2023-10-21T15:44:30.585 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-21T15:44:30.586 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368 --name yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --verbose -- /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/orig.config.yaml 2023-10-21T15:44:30.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T15:44:30.722 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7434368' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: f0804064d67a2a63dbabf1801ff452580a050cfc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f0804064d67a2a63dbabf1801ff452580a050cfc kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: f0804064d67a2a63dbabf1801ff452580a050cfc owner: scheduled_yuriw@teuthology priority: 76 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2514 sha1: f0804064d67a2a63dbabf1801ff452580a050cfc sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f0804064d67a2a63dbabf1801ff452580a050cfc/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f0804064d67a2a63dbabf1801ff452580a050cfc syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6fbGUyaNZpvY8oPhmMhokG1MwWoELqZEkefGzsAeko4jFlkENtZa/IW7SEG+imR/eOC9to3WOcrryOomavqFs= smithi124.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuo3XS6+QNnM2uGCiSRrpQv3oO0av8aS5sVXQ0s6Hpb+uYzKM7UdSZpafuhOAmzXO0hreAgrGR6pCLCsexlrdo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-20_18:46:15 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-21T15:44:30.723 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f0804064d67a2a63dbabf1801ff452580a050cfc/qa; will attempt to use it 2023-10-21T15:44:30.723 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f0804064d67a2a63dbabf1801ff452580a050cfc/qa/tasks 2023-10-21T15:44:30.724 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-21T15:44:30.725 INFO:teuthology.task.internal:Checking packages... 2023-10-21T15:44:30.750 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f0804064d67a2a63dbabf1801ff452580a050cfc' 2023-10-21T15:44:30.751 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-21T15:44:30.751 INFO:teuthology.packaging:ref: None 2023-10-21T15:44:30.752 INFO:teuthology.packaging:tag: None 2023-10-21T15:44:30.752 INFO:teuthology.packaging:branch: main 2023-10-21T15:44:30.752 INFO:teuthology.packaging:sha1: f0804064d67a2a63dbabf1801ff452580a050cfc 2023-10-21T15:44:30.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-10-21T15:44:31.006 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6820.gf0804064 2023-10-21T15:44:31.007 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-21T15:44:31.014 INFO:teuthology.task.internal:no buildpackages task found 2023-10-21T15:44:31.014 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-21T15:44:31.032 INFO:teuthology.task.internal:Saving configuration 2023-10-21T15:44:31.051 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-21T15:44:31.057 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T15:44:31.083 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 15:39:31.284207', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6fbGUyaNZpvY8oPhmMhokG1MwWoELqZEkefGzsAeko4jFlkENtZa/IW7SEG+imR/eOC9to3WOcrryOomavqFs='} 2023-10-21T15:44:31.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 15:39:31.282771', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuo3XS6+QNnM2uGCiSRrpQv3oO0av8aS5sVXQ0s6Hpb+uYzKM7UdSZpafuhOAmzXO0hreAgrGR6pCLCsexlrdo='} 2023-10-21T15:44:31.104 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-21T15:44:31.110 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-21T15:44:31.110 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-21T15:44:31.111 INFO:teuthology.run_tasks:Running task console_log... 2023-10-21T15:44:31.185 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8417d3db80>, signals=[15]) 2023-10-21T15:44:31.187 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-21T15:44:31.195 INFO:teuthology.task.internal:Opening connections... 2023-10-21T15:44:31.196 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T15:44:31.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:44:31.297 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T15:44:31.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:44:31.381 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-21T15:44:31.388 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-10-21T15:44:31.406 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-10-21T15:44:31.407 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-10-21T15:44:31.465 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2023-10-21T15:44:31.465 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2023-10-21T15:44:31.466 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2023-10-21T15:44:31.466 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2023-10-21T15:44:31.466 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2023-10-21T15:44:31.466 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2023-10-21T15:44:31.466 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T15:44:31.467 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T15:44:31.468 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T15:44:31.468 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-10-21T15:44:31.494 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2023-10-21T15:44:31.512 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2023-10-21T15:44:31.512 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-10-21T15:44:31.568 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2023-10-21T15:44:31.568 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2023-10-21T15:44:31.569 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2023-10-21T15:44:31.569 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2023-10-21T15:44:31.569 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2023-10-21T15:44:31.569 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2023-10-21T15:44:31.569 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T15:44:31.570 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-10-21T15:44:31.570 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2023-10-21T15:44:31.570 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T15:44:31.570 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2023-10-21T15:44:31.570 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T15:44:31.571 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T15:44:31.571 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T15:44:31.571 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-10-21T15:44:31.594 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-21T15:44:31.604 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-21T15:44:31.610 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-21T15:44:31.610 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2023-10-21T15:44:31.613 DEBUG:teuthology.orchestra.run.smithi124:> test '!' -e /home/ubuntu/cephtest 2023-10-21T15:44:31.628 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-21T15:44:31.634 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-21T15:44:31.635 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2023-10-21T15:44:31.671 DEBUG:teuthology.orchestra.run.smithi124:> test -z $(ls -A /var/lib/ceph) 2023-10-21T15:44:31.696 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-21T15:44:31.747 INFO:teuthology.run_tasks:Running task kernel... 2023-10-21T15:44:31.765 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-21T15:44:31.766 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-21T15:44:31.766 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-21T15:44:31.766 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-21T15:44:31.767 DEBUG:teuthology.orchestra.run.smithi124:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-21T15:44:31.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T15:44:31.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-21T15:44:31.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T15:44:31.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-21T15:44:31.956 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:44:31.956 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-21T15:44:31.957 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2023-10-21T15:44:31.976 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-361.el9.x86_64 2023-10-21T15:44:31.976 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:44:31.977 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-21T15:44:31.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum upgrade -y linux-firmware 2023-10-21T15:44:32.026 DEBUG:teuthology.task.kernel:sha1 for client.1 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:44:32.027 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-21T15:44:32.027 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2023-10-21T15:44:32.045 INFO:teuthology.orchestra.run.smithi124.stdout:5.14.0-361.el9.x86_64 2023-10-21T15:44:32.046 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:44:32.046 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-21T15:44:32.047 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum upgrade -y linux-firmware 2023-10-21T15:44:34.145 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-21T15:44:34.162 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-10-21T15:44:35.471 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2023-10-21T15:44:35.512 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2023-10-21T15:44:36.986 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 7.9 MB 00:01 2023-10-21T15:44:37.387 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 7.9 MB 00:01 2023-10-21T15:44:38.787 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 26 MB/s | 18 MB 00:00 2023-10-21T15:44:42.138 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 18 MB 00:03 2023-10-21T15:44:43.021 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 5.7 MB 00:00 2023-10-21T15:44:44.873 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2023-10-21T15:44:45.881 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.7 MB 00:00 2023-10-21T15:44:47.315 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2023-10-21T15:44:48.184 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 26 MB/s | 19 MB 00:00 2023-10-21T15:44:49.267 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 19 MB 00:04 2023-10-21T15:44:52.700 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2023-10-21T15:44:53.981 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2023-10-21T15:44:54.973 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2023-10-21T15:44:54.981 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2023-10-21T15:44:54.981 INFO:teuthology.orchestra.run.smithi124.stdout: Package Arch Version Repository Size 2023-10-21T15:44:54.982 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2023-10-21T15:44:54.982 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrading: 2023-10-21T15:44:54.982 INFO:teuthology.orchestra.run.smithi124.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-21T15:44:54.982 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction Summary 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrade 1 Package 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout:Total download size: 326 M 2023-10-21T15:44:54.983 INFO:teuthology.orchestra.run.smithi124.stdout:Downloading Packages: 2023-10-21T15:44:56.269 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-10-21T15:44:56.284 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-10-21T15:44:56.284 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-10-21T15:44:56.284 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-10-21T15:44:56.285 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2023-10-21T15:44:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-21T15:44:56.285 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:44:56.285 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-10-21T15:44:56.285 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-10-21T15:44:56.286 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 1 Package 2023-10-21T15:44:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:44:56.286 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 326 M 2023-10-21T15:44:56.287 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-10-21T15:45:03.632 INFO:teuthology.orchestra.run.smithi053.stdout:linux-firmware-20230926-140.el9.noarch.rpm 44 MB/s | 326 MB 00:07 2023-10-21T15:45:03.633 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-10-21T15:45:03.633 INFO:teuthology.orchestra.run.smithi053.stdout:Total 44 MB/s | 326 MB 00:07 2023-10-21T15:45:03.634 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-10-21T15:45:03.729 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-10-21T15:45:03.729 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-10-21T15:45:05.926 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-10-21T15:45:05.926 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-10-21T15:45:06.260 INFO:teuthology.orchestra.run.smithi124.stdout:linux-firmware-20230926-140.el9.noarch.rpm 29 MB/s | 326 MB 00:11 2023-10-21T15:45:06.261 INFO:teuthology.orchestra.run.smithi124.stdout:-------------------------------------------------------------------------------- 2023-10-21T15:45:06.261 INFO:teuthology.orchestra.run.smithi124.stdout:Total 29 MB/s | 326 MB 00:11 2023-10-21T15:45:06.261 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction check 2023-10-21T15:45:06.360 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction check succeeded. 2023-10-21T15:45:06.361 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction test 2023-10-21T15:45:08.435 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction test succeeded. 2023-10-21T15:45:08.435 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction 2023-10-21T15:45:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-10-21T15:45:12.225 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-21T15:45:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-21T15:45:13.717 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:13.717 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:45:13.718 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2023-10-21T15:45:13.718 INFO:teuthology.orchestra.run.smithi053.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-21T15:45:13.718 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:45:13.718 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-10-21T15:45:14.179 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-21T15:45:14.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:45:14.329 INFO:teuthology.orchestra.run.smithi124.stdout: Preparing : 1/1 2023-10-21T15:45:14.402 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-21T15:45:14.403 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-21T15:45:14.405 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi053.front.sepia.ceph.com to complete... 2023-10-21T15:45:14.744 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-21T15:45:14.834 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:15.848 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:15.849 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-21T15:45:16.247 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-21T15:45:16.248 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:45:16.248 INFO:teuthology.orchestra.run.smithi124.stdout:Upgraded: 2023-10-21T15:45:16.248 INFO:teuthology.orchestra.run.smithi124.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-21T15:45:16.248 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:45:16.248 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2023-10-21T15:45:16.540 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-21T15:45:16.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-21T15:45:16.684 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-21T15:45:16.684 DEBUG:teuthology.orchestra.run.smithi124:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-21T15:45:16.687 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi124.front.sepia.ceph.com to complete... 2023-10-21T15:45:48.010 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21 15:45:48 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-21T15:45:48.010 INFO:teuthology.orchestra.run.smithi124.stderr:FINISHED --2023-10-21 15:45:48-- 2023-10-21T15:45:48.010 INFO:teuthology.orchestra.run.smithi124.stderr:Total wall clock time: 31s 2023-10-21T15:45:48.011 INFO:teuthology.orchestra.run.smithi124.stderr:Downloaded: 1 files, 306M in 31s (9.95 MB/s) 2023-10-21T15:45:48.012 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-21T15:45:48.013 DEBUG:teuthology.orchestra.run.smithi124:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-21T15:45:49.864 INFO:teuthology.orchestra.run.smithi124.stdout:Verifying... ######################################## 2023-10-21T15:45:49.961 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing... ######################################## 2023-10-21T15:45:50.040 INFO:teuthology.orchestra.run.smithi124.stdout:Updating / installing... 2023-10-21T15:45:54.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21 15:45:54 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-21T15:45:54.344 INFO:teuthology.orchestra.run.smithi053.stderr:FINISHED --2023-10-21 15:45:54-- 2023-10-21T15:45:54.344 INFO:teuthology.orchestra.run.smithi053.stderr:Total wall clock time: 40s 2023-10-21T15:45:54.344 INFO:teuthology.orchestra.run.smithi053.stderr:Downloaded: 1 files, 306M in 39s (7.78 MB/s) 2023-10-21T15:45:54.345 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-21T15:45:54.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-21T15:45:56.278 INFO:teuthology.orchestra.run.smithi053.stdout:Verifying... ######################################## 2023-10-21T15:45:56.332 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing... ######################################## 2023-10-21T15:45:56.417 INFO:teuthology.orchestra.run.smithi053.stdout:Updating / installing... 2023-10-21T15:45:57.185 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-21T15:46:03.811 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-21T15:46:11.032 INFO:teuthology.orchestra.run.smithi124.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-21T15:46:17.961 INFO:teuthology.orchestra.run.smithi053.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-21T15:46:32.402 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi124.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-21T15:46:32.402 DEBUG:teuthology.orchestra.run.smithi124:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-21T15:46:32.476 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.476 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.476 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.477 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.477 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-21T15:46:32.477 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-21T15:46:32.477 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-21T15:46:32.477 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-21T15:46:32.478 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-21T15:46:32.478 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-21T15:46:32.478 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-21T15:46:32.478 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-21T15:46:32.478 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-21T15:46:32.479 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-21T15:46:32.479 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-21T15:46:32.479 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-21T15:46:32.479 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-21T15:46:32.479 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-21T15:46:32.480 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-21T15:46:32.480 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-21T15:46:32.480 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-21T15:46:32.480 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-21T15:46:32.480 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-21T15:46:32.481 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-21T15:46:32.481 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-21T15:46:32.481 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-21T15:46:32.481 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-21T15:46:32.481 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-21T15:46:32.482 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-21T15:46:32.482 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-21T15:46:32.482 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-21T15:46:32.482 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-21T15:46:32.482 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-21T15:46:32.483 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-21T15:46:32.483 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-21T15:46:32.483 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-21T15:46:32.483 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-21T15:46:32.483 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-21T15:46:32.484 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-21T15:46:32.485 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-21T15:46:32.485 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-21T15:46:32.485 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-21T15:46:32.485 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-21T15:46:32.485 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-21T15:46:32.486 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-21T15:46:32.486 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-21T15:46:32.486 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-21T15:46:32.486 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-21T15:46:32.486 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-21T15:46:32.487 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-21T15:46:32.487 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-21T15:46:32.487 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-21T15:46:32.487 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-21T15:46:32.487 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-21T15:46:32.488 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-21T15:46:32.488 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-21T15:46:32.488 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-21T15:46:32.488 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-21T15:46:32.488 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-21T15:46:32.489 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-21T15:46:32.490 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-21T15:46:32.490 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-21T15:46:32.490 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-21T15:46:32.490 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-21T15:46:32.490 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-21T15:46:32.491 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-21T15:46:32.491 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-21T15:46:32.491 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-21T15:46:32.491 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-21T15:46:32.491 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-21T15:46:32.492 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-21T15:46:32.492 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-21T15:46:32.492 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-21T15:46:32.492 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-21T15:46:32.492 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-21T15:46:32.493 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-21T15:46:32.493 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-21T15:46:32.493 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-21T15:46:32.493 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-21T15:46:32.493 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-21T15:46:32.494 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-21T15:46:32.494 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-21T15:46:32.494 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-21T15:46:32.494 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-21T15:46:32.494 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-21T15:46:32.495 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-21T15:46:32.495 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-21T15:46:32.495 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-21T15:46:32.495 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-21T15:46:32.495 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-21T15:46:32.496 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-21T15:46:32.496 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-21T15:46:32.496 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-21T15:46:32.496 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-21T15:46:32.496 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-21T15:46:32.497 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-21T15:46:32.498 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-21T15:46:32.498 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-21T15:46:32.498 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-21T15:46:32.498 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-21T15:46:32.498 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-21T15:46:32.499 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-21T15:46:32.499 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-21T15:46:32.499 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-21T15:46:32.499 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-21T15:46:32.499 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-21T15:46:32.500 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-21T15:46:32.500 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-21T15:46:32.500 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-21T15:46:32.500 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-21T15:46:32.500 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-21T15:46:32.501 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-21T15:46:32.502 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-21T15:46:32.502 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-21T15:46:32.502 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-21T15:46:32.502 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-21T15:46:32.502 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-21T15:46:32.503 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-21T15:46:32.503 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-21T15:46:32.503 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-21T15:46:32.503 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-21T15:46:32.503 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-21T15:46:32.504 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-21T15:46:32.505 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-21T15:46:32.505 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-21T15:46:32.505 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-21T15:46:32.505 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-21T15:46:32.505 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-21T15:46:32.506 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-21T15:46:32.506 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-21T15:46:32.506 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-21T15:46:32.506 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-21T15:46:32.506 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-21T15:46:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-21T15:46:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-21T15:46:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-21T15:46:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-21T15:46:32.507 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-21T15:46:32.508 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-21T15:46:32.509 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-21T15:46:32.509 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-21T15:46:32.509 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-21T15:46:32.509 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-21T15:46:32.509 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-21T15:46:32.510 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-21T15:46:32.510 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-21T15:46:32.510 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-21T15:46:32.510 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-21T15:46:32.510 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-21T15:46:32.511 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-21T15:46:32.512 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-21T15:46:32.512 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-21T15:46:32.512 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-21T15:46:32.512 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-21T15:46:32.512 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-21T15:46:32.513 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-21T15:46:32.513 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-21T15:46:32.513 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-21T15:46:32.513 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-21T15:46:32.513 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-21T15:46:32.514 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-21T15:46:32.514 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-21T15:46:32.514 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-21T15:46:32.514 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-21T15:46:32.514 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-21T15:46:32.515 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-21T15:46:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-21T15:46:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-21T15:46:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-21T15:46:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-21T15:46:32.516 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-21T15:46:32.517 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-21T15:46:32.517 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-21T15:46:32.517 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-21T15:46:32.517 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-21T15:46:32.517 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-21T15:46:32.518 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-21T15:46:32.518 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-21T15:46:32.518 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-21T15:46:32.518 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-21T15:46:32.518 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-21T15:46:32.519 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-21T15:46:32.519 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-21T15:46:32.519 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-21T15:46:32.519 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-21T15:46:32.519 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-21T15:46:32.520 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-21T15:46:32.520 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-21T15:46:32.520 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-21T15:46:32.520 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-21T15:46:32.520 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-21T15:46:32.521 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-21T15:46:32.522 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-21T15:46:32.522 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-21T15:46:32.522 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-21T15:46:32.522 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-21T15:46:32.522 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-21T15:46:32.523 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-21T15:46:32.523 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-21T15:46:32.523 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-21T15:46:32.523 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-21T15:46:32.523 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-21T15:46:32.524 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-21T15:46:32.524 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-21T15:46:32.524 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-21T15:46:32.524 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-21T15:46:32.524 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-21T15:46:32.525 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-21T15:46:32.525 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-21T15:46:32.525 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-21T15:46:32.525 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-21T15:46:32.525 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-21T15:46:32.526 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-21T15:46:32.527 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-21T15:46:32.527 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-21T15:46:32.527 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-21T15:46:32.527 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-21T15:46:32.527 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-21T15:46:32.528 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-21T15:46:32.528 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-21T15:46:32.528 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-21T15:46:32.528 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-21T15:46:32.528 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-21T15:46:32.529 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-21T15:46:32.530 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-21T15:46:32.530 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-21T15:46:32.530 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-21T15:46:32.530 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-21T15:46:32.530 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-21T15:46:32.531 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-21T15:46:32.531 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-21T15:46:32.531 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-21T15:46:32.531 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-21T15:46:32.531 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-21T15:46:32.532 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-21T15:46:32.532 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-21T15:46:32.532 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-21T15:46:32.532 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-21T15:46:32.532 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-21T15:46:32.533 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-21T15:46:32.534 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-21T15:46:32.534 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-21T15:46:32.534 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-21T15:46:32.534 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-21T15:46:32.534 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-21T15:46:32.535 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-21T15:46:32.535 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-21T15:46:32.535 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-21T15:46:32.535 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-21T15:46:32.535 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-21T15:46:32.536 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-21T15:46:32.537 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-21T15:46:32.537 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-21T15:46:32.537 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-21T15:46:32.537 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-21T15:46:32.537 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-21T15:46:32.538 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-21T15:46:32.539 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-21T15:46:32.539 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-21T15:46:32.539 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-21T15:46:32.539 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-21T15:46:32.539 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-21T15:46:32.540 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-21T15:46:32.540 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-21T15:46:32.540 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-21T15:46:32.540 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-21T15:46:32.540 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-21T15:46:32.541 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-21T15:46:32.542 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-21T15:46:32.542 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-21T15:46:32.542 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-21T15:46:32.542 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-21T15:46:32.542 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-21T15:46:32.543 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-21T15:46:32.543 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-21T15:46:32.543 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-21T15:46:32.543 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-21T15:46:32.543 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-21T15:46:32.544 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-21T15:46:32.545 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-21T15:46:32.545 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-21T15:46:32.545 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-21T15:46:32.545 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-21T15:46:32.545 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-21T15:46:32.546 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-21T15:46:32.546 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-21T15:46:32.546 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-21T15:46:32.546 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-21T15:46:32.546 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-21T15:46:32.547 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-21T15:46:32.548 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-21T15:46:32.548 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-21T15:46:32.548 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-21T15:46:32.548 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-21T15:46:32.548 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-21T15:46:32.549 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-21T15:46:32.549 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-21T15:46:32.549 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-21T15:46:32.549 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-21T15:46:32.549 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-21T15:46:32.550 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-21T15:46:32.551 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-21T15:46:32.551 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-21T15:46:32.551 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-21T15:46:32.551 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-21T15:46:32.551 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-21T15:46:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-21T15:46:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-21T15:46:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-21T15:46:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-21T15:46:32.552 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-21T15:46:32.553 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-21T15:46:32.554 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-21T15:46:32.554 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-21T15:46:32.554 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-21T15:46:32.554 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-21T15:46:32.554 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-21T15:46:32.555 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-21T15:46:32.555 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-21T15:46:32.555 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-21T15:46:32.555 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-21T15:46:32.555 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-21T15:46:32.556 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-21T15:46:32.557 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-21T15:46:32.557 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-21T15:46:32.557 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-21T15:46:32.557 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-21T15:46:32.557 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-21T15:46:32.558 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-21T15:46:32.558 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-21T15:46:32.558 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-21T15:46:32.558 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-21T15:46:32.558 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-21T15:46:32.559 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-21T15:46:32.560 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-21T15:46:32.560 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-21T15:46:32.560 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-21T15:46:32.560 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-21T15:46:32.560 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-21T15:46:32.561 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-21T15:46:32.561 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-21T15:46:32.561 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-21T15:46:32.561 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-21T15:46:32.562 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-21T15:46:32.562 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-21T15:46:32.562 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-21T15:46:32.562 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-21T15:46:32.562 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-21T15:46:32.563 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-21T15:46:32.564 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-21T15:46:32.564 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-21T15:46:32.564 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-21T15:46:32.564 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-21T15:46:32.564 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-21T15:46:32.565 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-21T15:46:32.565 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-21T15:46:32.565 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-21T15:46:32.565 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-21T15:46:32.565 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-21T15:46:32.566 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-21T15:46:32.566 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-21T15:46:32.566 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-21T15:46:32.566 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-21T15:46:32.566 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-21T15:46:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-21T15:46:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-21T15:46:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-21T15:46:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-21T15:46:32.567 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-21T15:46:32.568 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-21T15:46:32.569 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-21T15:46:32.569 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-21T15:46:32.569 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-21T15:46:32.569 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-21T15:46:32.569 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-21T15:46:32.570 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-21T15:46:32.570 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-21T15:46:32.570 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-21T15:46:32.570 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-21T15:46:32.570 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-21T15:46:32.571 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-21T15:46:32.572 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-21T15:46:32.572 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-21T15:46:32.572 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-21T15:46:32.572 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-21T15:46:32.572 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-21T15:46:32.573 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-21T15:46:32.573 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-21T15:46:32.573 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-21T15:46:32.573 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-21T15:46:32.573 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-21T15:46:32.574 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-21T15:46:32.575 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-21T15:46:32.575 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-21T15:46:32.575 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-21T15:46:32.575 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-21T15:46:32.575 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-21T15:46:32.576 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-21T15:46:32.576 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-21T15:46:32.576 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-21T15:46:32.576 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-21T15:46:32.576 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-21T15:46:32.577 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-21T15:46:32.578 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-21T15:46:32.578 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-21T15:46:32.578 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-21T15:46:32.578 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-21T15:46:32.578 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-21T15:46:32.579 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-21T15:46:32.579 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-21T15:46:32.579 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-21T15:46:32.579 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-21T15:46:32.579 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-21T15:46:32.580 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-21T15:46:32.580 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-21T15:46:32.580 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-21T15:46:32.580 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-21T15:46:32.580 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-21T15:46:32.581 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-21T15:46:32.582 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-21T15:46:32.582 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-21T15:46:32.582 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-21T15:46:32.582 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-21T15:46:32.582 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-21T15:46:32.583 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-21T15:46:32.583 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-21T15:46:32.583 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-21T15:46:32.583 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-21T15:46:32.583 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-21T15:46:32.584 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-21T15:46:32.585 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-21T15:46:32.585 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-21T15:46:32.585 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-21T15:46:32.585 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-21T15:46:32.585 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-21T15:46:32.586 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-21T15:46:32.586 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-21T15:46:32.586 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-21T15:46:32.586 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-21T15:46:32.586 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-21T15:46:32.587 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-21T15:46:32.587 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-21T15:46:32.587 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-21T15:46:32.587 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-21T15:46:32.587 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-21T15:46:32.588 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-21T15:46:32.589 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-21T15:46:32.589 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-21T15:46:32.589 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-21T15:46:32.589 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-21T15:46:32.589 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-21T15:46:32.590 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-21T15:46:32.590 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-21T15:46:32.590 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-21T15:46:32.590 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-21T15:46:32.590 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-21T15:46:32.591 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-21T15:46:32.591 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-21T15:46:32.591 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-21T15:46:32.591 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-21T15:46:32.591 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-21T15:46:32.592 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-21T15:46:32.593 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-21T15:46:32.593 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-21T15:46:32.593 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-21T15:46:32.593 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-21T15:46:32.593 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-21T15:46:32.594 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-21T15:46:32.594 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-21T15:46:32.594 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-21T15:46:32.594 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-21T15:46:32.594 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-21T15:46:32.595 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-21T15:46:32.595 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-21T15:46:32.595 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-21T15:46:32.595 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-21T15:46:32.595 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-21T15:46:32.596 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-21T15:46:32.596 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-21T15:46:32.596 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-21T15:46:32.596 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-21T15:46:32.596 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-21T15:46:32.597 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-21T15:46:32.597 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-21T15:46:32.597 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-21T15:46:32.597 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-21T15:46:32.597 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-21T15:46:32.598 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-21T15:46:32.599 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-21T15:46:32.599 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-21T15:46:32.599 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-21T15:46:32.599 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-21T15:46:32.599 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-21T15:46:32.600 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-21T15:46:32.600 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-21T15:46:32.600 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-21T15:46:32.600 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-21T15:46:32.600 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-21T15:46:32.601 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-21T15:46:32.602 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-21T15:46:32.602 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-21T15:46:32.602 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-21T15:46:32.602 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-21T15:46:32.602 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-21T15:46:32.603 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-21T15:46:32.603 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-21T15:46:32.603 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-21T15:46:32.603 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-21T15:46:32.603 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-21T15:46:32.604 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-21T15:46:32.604 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-21T15:46:32.604 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-21T15:46:32.604 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-21T15:46:32.604 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-21T15:46:32.605 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-21T15:46:32.605 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-21T15:46:32.605 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-21T15:46:32.605 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-21T15:46:32.605 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-21T15:46:32.606 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-21T15:46:32.606 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-21T15:46:32.606 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-21T15:46:32.606 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-21T15:46:32.606 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-21T15:46:32.607 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-21T15:46:32.607 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-21T15:46:32.607 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-21T15:46:32.607 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-21T15:46:32.607 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-21T15:46:32.608 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-21T15:46:32.608 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-21T15:46:32.608 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-21T15:46:32.608 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-21T15:46:32.608 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-21T15:46:32.609 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-21T15:46:32.609 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-21T15:46:32.609 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-21T15:46:32.609 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-21T15:46:32.609 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-21T15:46:32.610 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-21T15:46:32.610 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-21T15:46:32.610 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-21T15:46:32.610 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-21T15:46:32.610 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-21T15:46:32.611 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-21T15:46:32.611 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-21T15:46:32.611 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-21T15:46:32.611 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-21T15:46:32.611 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-21T15:46:32.612 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-21T15:46:32.613 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-21T15:46:32.613 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-21T15:46:32.613 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-21T15:46:32.613 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-21T15:46:32.613 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-21T15:46:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-21T15:46:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-21T15:46:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-21T15:46:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-21T15:46:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-21T15:46:32.615 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-21T15:46:32.615 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-21T15:46:32.615 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-21T15:46:32.615 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-21T15:46:32.615 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-21T15:46:32.616 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-21T15:46:32.617 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-21T15:46:32.617 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-21T15:46:32.617 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-21T15:46:32.617 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-21T15:46:32.617 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-21T15:46:32.618 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-21T15:46:32.618 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-21T15:46:32.618 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-21T15:46:32.618 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-21T15:46:32.618 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-21T15:46:32.619 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-21T15:46:32.619 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-21T15:46:32.619 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-21T15:46:32.619 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-21T15:46:32.619 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-21T15:46:32.620 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-21T15:46:32.620 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-21T15:46:32.620 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-21T15:46:32.620 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-21T15:46:32.620 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-21T15:46:32.621 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-21T15:46:32.621 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-21T15:46:32.621 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-21T15:46:32.621 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-21T15:46:32.621 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-21T15:46:32.622 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-21T15:46:32.623 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-21T15:46:32.623 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-21T15:46:32.623 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-21T15:46:32.623 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-21T15:46:32.623 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-21T15:46:32.624 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-21T15:46:32.624 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-21T15:46:32.624 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-21T15:46:32.624 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-21T15:46:32.624 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-21T15:46:32.625 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-21T15:46:32.626 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-21T15:46:32.626 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-21T15:46:32.626 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-21T15:46:32.626 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-21T15:46:32.626 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-21T15:46:32.627 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-21T15:46:32.627 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-21T15:46:32.627 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-21T15:46:32.627 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-21T15:46:32.627 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-21T15:46:32.628 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-21T15:46:32.628 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-21T15:46:32.628 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-21T15:46:32.628 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-21T15:46:32.628 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-21T15:46:32.629 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-21T15:46:32.630 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-21T15:46:32.630 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-21T15:46:32.630 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-21T15:46:32.630 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-21T15:46:32.630 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-21T15:46:32.631 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-21T15:46:32.631 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-21T15:46:32.631 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-21T15:46:32.631 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-21T15:46:32.631 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-21T15:46:32.632 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-21T15:46:32.632 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-21T15:46:32.632 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-21T15:46:32.632 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-21T15:46:32.632 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-21T15:46:32.633 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-21T15:46:32.634 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-21T15:46:32.634 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-21T15:46:32.634 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-21T15:46:32.634 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-21T15:46:32.634 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-21T15:46:32.635 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-21T15:46:32.636 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-21T15:46:32.636 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-21T15:46:32.636 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-21T15:46:32.636 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-21T15:46:32.636 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-21T15:46:32.637 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-21T15:46:32.637 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-21T15:46:32.637 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-21T15:46:32.637 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-21T15:46:32.637 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-21T15:46:32.638 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-21T15:46:32.639 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-21T15:46:32.639 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-21T15:46:32.639 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-21T15:46:32.639 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-21T15:46:32.639 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-21T15:46:32.640 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-21T15:46:32.641 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-21T15:46:32.641 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-21T15:46:32.641 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-21T15:46:32.641 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-21T15:46:32.641 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-21T15:46:32.642 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-21T15:46:32.642 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-21T15:46:32.642 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-21T15:46:32.642 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-21T15:46:32.642 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-21T15:46:32.643 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-21T15:46:32.644 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-21T15:46:32.644 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-21T15:46:32.644 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-21T15:46:32.644 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-21T15:46:32.644 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-21T15:46:32.645 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-21T15:46:32.645 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-21T15:46:32.645 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-21T15:46:32.645 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-21T15:46:32.645 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-21T15:46:32.646 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-21T15:46:32.647 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-21T15:46:32.647 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-21T15:46:32.647 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-21T15:46:32.647 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-21T15:46:32.647 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-21T15:46:32.648 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-21T15:46:32.649 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-21T15:46:32.649 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-21T15:46:32.649 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-21T15:46:32.649 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-21T15:46:32.649 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-21T15:46:32.650 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-21T15:46:32.650 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-21T15:46:32.650 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-21T15:46:32.650 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-21T15:46:32.650 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-21T15:46:32.651 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-21T15:46:32.651 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-21T15:46:32.651 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-21T15:46:32.651 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-21T15:46:32.651 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-21T15:46:32.652 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-21T15:46:32.653 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-21T15:46:32.653 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-21T15:46:32.653 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-21T15:46:32.653 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-21T15:46:32.653 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-21T15:46:32.654 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-21T15:46:32.654 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-21T15:46:32.654 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-21T15:46:32.654 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-21T15:46:32.654 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-21T15:46:32.655 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-21T15:46:32.656 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-21T15:46:32.656 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-21T15:46:32.656 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-21T15:46:32.656 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-21T15:46:32.656 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-21T15:46:32.657 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-21T15:46:32.658 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-21T15:46:32.658 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-21T15:46:32.658 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-21T15:46:32.658 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-21T15:46:32.658 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-21T15:46:32.659 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-21T15:46:32.659 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-21T15:46:32.659 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-21T15:46:32.659 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-21T15:46:32.659 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-21T15:46:32.660 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-21T15:46:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-21T15:46:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-21T15:46:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-21T15:46:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-21T15:46:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-21T15:46:32.662 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-21T15:46:32.662 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-21T15:46:32.662 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-21T15:46:32.662 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-21T15:46:32.662 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-21T15:46:32.663 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-21T15:46:32.663 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-21T15:46:32.663 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-21T15:46:32.663 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-21T15:46:32.663 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-21T15:46:32.664 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-21T15:46:32.665 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-21T15:46:32.665 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-21T15:46:32.665 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-21T15:46:32.665 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-21T15:46:32.665 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-21T15:46:32.666 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-21T15:46:32.667 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-21T15:46:32.667 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-21T15:46:32.667 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-21T15:46:32.667 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-21T15:46:32.667 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-21T15:46:32.668 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-21T15:46:32.668 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-21T15:46:32.668 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-21T15:46:32.668 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-21T15:46:32.668 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-21T15:46:32.669 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-21T15:46:32.670 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-21T15:46:32.670 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-21T15:46:32.670 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-21T15:46:32.670 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-21T15:46:32.670 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-21T15:46:32.671 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-21T15:46:32.672 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-21T15:46:32.672 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-21T15:46:32.672 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-21T15:46:32.672 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-21T15:46:32.672 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-21T15:46:32.673 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-21T15:46:32.674 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-21T15:46:32.674 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-21T15:46:32.674 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-21T15:46:32.674 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-21T15:46:32.674 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-21T15:46:32.675 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-21T15:46:32.675 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-21T15:46:32.675 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-21T15:46:32.675 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-21T15:46:32.675 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-21T15:46:32.676 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-21T15:46:32.677 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-21T15:46:32.677 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-21T15:46:32.677 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-21T15:46:32.677 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-21T15:46:32.677 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-21T15:46:32.678 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-21T15:46:32.679 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-21T15:46:32.679 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-21T15:46:32.679 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-21T15:46:32.679 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-21T15:46:32.679 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-21T15:46:32.680 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-21T15:46:32.681 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-21T15:46:32.681 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-21T15:46:32.681 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-21T15:46:32.681 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-21T15:46:32.681 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-21T15:46:32.682 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-21T15:46:32.682 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-21T15:46:32.682 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-21T15:46:32.682 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-21T15:46:32.682 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-21T15:46:32.683 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-21T15:46:32.684 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-21T15:46:32.684 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-21T15:46:32.684 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-21T15:46:32.684 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-21T15:46:32.684 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-21T15:46:32.685 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-21T15:46:32.686 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-21T15:46:32.686 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-21T15:46:32.686 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-21T15:46:32.686 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-21T15:46:32.686 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-21T15:46:32.687 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-21T15:46:32.688 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-21T15:46:32.688 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-21T15:46:32.688 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-21T15:46:32.688 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-21T15:46:32.688 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-21T15:46:32.689 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-21T15:46:32.689 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-21T15:46:32.689 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-21T15:46:32.689 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-21T15:46:32.689 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-21T15:46:32.690 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-21T15:46:32.690 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-21T15:46:32.690 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-21T15:46:32.690 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-21T15:46:32.690 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-21T15:46:32.691 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-21T15:46:32.692 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-21T15:46:32.692 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-21T15:46:32.692 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-21T15:46:32.692 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-21T15:46:32.692 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-21T15:46:32.693 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-21T15:46:32.694 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-21T15:46:32.694 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-21T15:46:32.694 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-21T15:46:32.694 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-21T15:46:32.694 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-21T15:46:32.695 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-21T15:46:32.695 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-21T15:46:32.695 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-21T15:46:32.695 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-21T15:46:32.695 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-21T15:46:32.696 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-21T15:46:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-21T15:46:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-21T15:46:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-21T15:46:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-21T15:46:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-21T15:46:32.698 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-21T15:46:32.698 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-21T15:46:32.698 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-21T15:46:32.698 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-21T15:46:32.698 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-21T15:46:32.699 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-21T15:46:32.700 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-21T15:46:32.700 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-21T15:46:32.700 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-21T15:46:32.700 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-21T15:46:32.700 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-21T15:46:32.701 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-21T15:46:32.701 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-21T15:46:32.701 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-21T15:46:32.701 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-21T15:46:32.701 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-21T15:46:32.702 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-21T15:46:32.703 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-21T15:46:32.703 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-21T15:46:32.703 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-21T15:46:32.703 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-21T15:46:32.703 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-21T15:46:32.704 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-21T15:46:32.704 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-21T15:46:32.704 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-21T15:46:32.704 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-21T15:46:32.704 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-21T15:46:32.705 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-21T15:46:32.706 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-21T15:46:32.706 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-21T15:46:32.706 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-21T15:46:32.706 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-21T15:46:32.706 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-21T15:46:32.707 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-21T15:46:32.707 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-21T15:46:32.707 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-21T15:46:32.707 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-21T15:46:32.707 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-21T15:46:32.708 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-21T15:46:32.709 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-21T15:46:32.709 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-21T15:46:32.709 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-21T15:46:32.709 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-21T15:46:32.709 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-21T15:46:32.710 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-21T15:46:32.710 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-21T15:46:32.710 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-21T15:46:32.710 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-21T15:46:32.710 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-21T15:46:32.711 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-21T15:46:32.712 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-21T15:46:32.712 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-21T15:46:32.712 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-21T15:46:32.712 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-21T15:46:32.712 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-21T15:46:32.713 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-21T15:46:32.713 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-21T15:46:32.713 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-21T15:46:32.713 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-21T15:46:32.713 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-21T15:46:32.714 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-21T15:46:32.714 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-21T15:46:32.714 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-21T15:46:32.714 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-21T15:46:32.714 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-21T15:46:32.715 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-21T15:46:32.716 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-21T15:46:32.716 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-21T15:46:32.716 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-21T15:46:32.716 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-21T15:46:32.716 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-21T15:46:32.717 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-21T15:46:32.718 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-21T15:46:32.718 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-21T15:46:32.718 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-21T15:46:32.718 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-21T15:46:32.718 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-21T15:46:32.719 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-21T15:46:32.720 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-21T15:46:32.720 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-21T15:46:32.720 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-21T15:46:32.720 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-21T15:46:32.720 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-21T15:46:32.721 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-21T15:46:32.721 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-21T15:46:32.721 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-21T15:46:32.721 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-21T15:46:32.721 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-21T15:46:32.722 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-21T15:46:32.723 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-21T15:46:32.723 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-21T15:46:32.723 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-21T15:46:32.723 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-21T15:46:32.723 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-21T15:46:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-21T15:46:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-21T15:46:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-21T15:46:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-21T15:46:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-21T15:46:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-21T15:46:32.726 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-21T15:46:32.727 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-21T15:46:32.727 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-21T15:46:32.727 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-21T15:46:32.727 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-21T15:46:32.727 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-21T15:46:32.728 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-21T15:46:32.728 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-21T15:46:32.728 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-21T15:46:32.728 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-21T15:46:32.728 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-21T15:46:32.729 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-21T15:46:32.730 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-21T15:46:32.730 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-21T15:46:32.730 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-21T15:46:32.730 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-21T15:46:32.730 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-21T15:46:32.731 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-21T15:46:32.732 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-21T15:46:32.732 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-21T15:46:32.732 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-21T15:46:32.732 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-21T15:46:32.732 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-21T15:46:32.733 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-21T15:46:32.733 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-21T15:46:32.733 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-21T15:46:32.733 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-21T15:46:32.733 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-21T15:46:32.734 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-21T15:46:32.734 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-21T15:46:32.734 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-21T15:46:32.737 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-21T15:46:32.738 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-21T15:46:32.738 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-21T15:46:32.738 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-21T15:46:32.738 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-21T15:46:32.739 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-21T15:46:32.739 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-21T15:46:32.739 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-21T15:46:32.739 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-21T15:46:32.739 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-21T15:46:32.740 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-21T15:46:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-21T15:46:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-21T15:46:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-21T15:46:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-21T15:46:32.741 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-21T15:46:32.742 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-21T15:46:32.743 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-21T15:46:32.743 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-21T15:46:32.743 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-21T15:46:32.743 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-21T15:46:32.743 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-21T15:46:32.744 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-21T15:46:32.745 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-21T15:46:32.745 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-21T15:46:32.745 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-21T15:46:32.745 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-21T15:46:32.745 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-21T15:46:32.746 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-21T15:46:32.746 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-21T15:46:32.746 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-21T15:46:32.746 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-21T15:46:32.746 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-21T15:46:32.747 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-21T15:46:32.748 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-21T15:46:32.748 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-21T15:46:32.748 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-21T15:46:32.748 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-21T15:46:32.748 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-21T15:46:32.749 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-21T15:46:32.750 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-21T15:46:32.750 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-21T15:46:32.750 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-21T15:46:32.750 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-21T15:46:32.750 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-21T15:46:32.751 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-21T15:46:32.752 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-21T15:46:32.752 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-21T15:46:32.752 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-21T15:46:32.752 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-21T15:46:32.752 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-21T15:46:32.753 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-21T15:46:32.754 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-21T15:46:32.754 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-21T15:46:32.754 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-21T15:46:32.754 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-21T15:46:32.754 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-21T15:46:32.755 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-21T15:46:32.756 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-21T15:46:32.756 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-21T15:46:32.756 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-21T15:46:32.756 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-21T15:46:32.756 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-21T15:46:32.757 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-21T15:46:32.758 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-21T15:46:32.758 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-21T15:46:32.758 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-21T15:46:32.758 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-21T15:46:32.758 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-21T15:46:32.759 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-21T15:46:32.760 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-21T15:46:32.760 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-21T15:46:32.760 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-21T15:46:32.760 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-21T15:46:32.760 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-21T15:46:32.761 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-21T15:46:32.761 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-21T15:46:32.761 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-21T15:46:32.761 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-21T15:46:32.761 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-21T15:46:32.762 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-21T15:46:32.763 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-21T15:46:32.763 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-21T15:46:32.763 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-21T15:46:32.763 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-21T15:46:32.763 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-21T15:46:32.764 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-21T15:46:32.765 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-21T15:46:32.765 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-21T15:46:32.765 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-21T15:46:32.765 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-21T15:46:32.765 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-21T15:46:32.766 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-21T15:46:32.767 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-21T15:46:32.768 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-21T15:46:32.769 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-21T15:46:32.770 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-21T15:46:32.770 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-21T15:46:32.770 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-21T15:46:32.770 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-21T15:46:32.770 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-21T15:46:32.771 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-21T15:46:32.771 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-21T15:46:32.771 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-21T15:46:32.771 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-21T15:46:32.771 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-21T15:46:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-21T15:46:32.773 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-21T15:46:32.774 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-21T15:46:32.774 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-21T15:46:32.774 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-21T15:46:32.774 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-21T15:46:32.774 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-21T15:46:32.775 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-21T15:46:32.775 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-21T15:46:32.775 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-21T15:46:32.775 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-21T15:46:32.775 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-21T15:46:32.776 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-21T15:46:32.777 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-21T15:46:32.777 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-21T15:46:32.777 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-21T15:46:32.777 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-21T15:46:32.777 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-21T15:46:32.778 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-21T15:46:32.778 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-21T15:46:32.778 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-21T15:46:32.778 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-21T15:46:32.778 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-21T15:46:32.779 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-21T15:46:32.779 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-21T15:46:32.779 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-21T15:46:32.779 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-21T15:46:32.779 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-21T15:46:32.780 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-21T15:46:32.781 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-21T15:46:32.781 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-21T15:46:32.781 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-21T15:46:32.781 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-21T15:46:32.781 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-21T15:46:32.782 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-21T15:46:32.783 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-21T15:46:32.783 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-21T15:46:32.783 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-21T15:46:32.783 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-21T15:46:32.783 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-21T15:46:32.784 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-21T15:46:32.785 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-21T15:46:32.785 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-21T15:46:32.785 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-21T15:46:32.785 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-21T15:46:32.785 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-21T15:46:32.786 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-21T15:46:32.786 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-21T15:46:32.786 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-21T15:46:32.786 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-21T15:46:32.786 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-21T15:46:32.787 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-21T15:46:32.788 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-21T15:46:32.789 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-21T15:46:32.789 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-21T15:46:32.789 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-21T15:46:32.789 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-21T15:46:32.789 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-21T15:46:32.790 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-21T15:46:32.791 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-21T15:46:32.791 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-21T15:46:32.791 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-21T15:46:32.791 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-21T15:46:32.791 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-21T15:46:32.792 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-21T15:46:32.793 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-21T15:46:32.793 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-21T15:46:32.793 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-21T15:46:32.793 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-21T15:46:32.793 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-21T15:46:32.794 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-21T15:46:32.795 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-21T15:46:32.795 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-21T15:46:32.795 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-21T15:46:32.795 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-21T15:46:32.795 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-21T15:46:32.796 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-21T15:46:32.797 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-21T15:46:32.797 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-21T15:46:32.797 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-21T15:46:32.797 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-21T15:46:32.797 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-21T15:46:32.798 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-21T15:46:32.799 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-21T15:46:32.799 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-21T15:46:32.799 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-21T15:46:32.799 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-21T15:46:32.799 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-21T15:46:32.800 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-21T15:46:32.801 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-21T15:46:32.801 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-21T15:46:32.801 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-21T15:46:32.801 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-21T15:46:32.801 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-21T15:46:32.802 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-21T15:46:32.803 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-21T15:46:32.803 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-21T15:46:32.803 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-21T15:46:32.803 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-21T15:46:32.803 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-21T15:46:32.804 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-21T15:46:32.805 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-21T15:46:32.805 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-21T15:46:32.805 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-21T15:46:32.805 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-21T15:46:32.805 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-21T15:46:32.806 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-21T15:46:32.807 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-21T15:46:32.807 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-21T15:46:32.807 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-21T15:46:32.807 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-21T15:46:32.807 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-21T15:46:32.808 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-21T15:46:32.809 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-21T15:46:32.809 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-21T15:46:32.809 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-21T15:46:32.809 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-21T15:46:32.809 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-21T15:46:32.810 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-21T15:46:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-21T15:46:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-21T15:46:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-21T15:46:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-21T15:46:32.811 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-21T15:46:32.812 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-21T15:46:32.813 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-21T15:46:32.813 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-21T15:46:32.813 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-21T15:46:32.813 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-21T15:46:32.813 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-21T15:46:32.814 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-21T15:46:32.814 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-21T15:46:32.814 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-21T15:46:32.814 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-21T15:46:32.814 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-21T15:46:32.815 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-21T15:46:32.816 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-21T15:46:32.817 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-21T15:46:32.817 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-21T15:46:32.817 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-21T15:46:32.817 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-21T15:46:32.817 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-21T15:46:32.818 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-21T15:46:32.818 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-21T15:46:32.818 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-21T15:46:32.818 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-21T15:46:32.818 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-21T15:46:32.819 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-21T15:46:32.820 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-21T15:46:32.821 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-21T15:46:32.821 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-21T15:46:32.821 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-21T15:46:32.821 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-21T15:46:32.821 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-21T15:46:32.822 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-21T15:46:32.823 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-21T15:46:32.823 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-21T15:46:32.823 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-21T15:46:32.823 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-21T15:46:32.823 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-21T15:46:32.824 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-21T15:46:32.825 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-21T15:46:32.825 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-21T15:46:32.825 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-21T15:46:32.825 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-21T15:46:32.825 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-21T15:46:32.826 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-21T15:46:32.827 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-21T15:46:32.827 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-21T15:46:32.827 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-21T15:46:32.827 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-21T15:46:32.827 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-21T15:46:32.828 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-21T15:46:32.829 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-21T15:46:32.829 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-21T15:46:32.829 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-21T15:46:32.829 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-21T15:46:32.829 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-21T15:46:32.830 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-21T15:46:32.831 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-21T15:46:32.831 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-21T15:46:32.831 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-21T15:46:32.831 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-21T15:46:32.831 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-21T15:46:32.832 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-21T15:46:32.833 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-21T15:46:32.833 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-21T15:46:32.833 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-21T15:46:32.833 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-21T15:46:32.833 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-21T15:46:32.834 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-21T15:46:32.835 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-21T15:46:32.835 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-21T15:46:32.835 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-21T15:46:32.835 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-21T15:46:32.835 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-21T15:46:32.836 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-21T15:46:32.837 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-21T15:46:32.837 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-21T15:46:32.837 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-21T15:46:32.837 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-21T15:46:32.837 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-21T15:46:32.838 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-21T15:46:32.839 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-21T15:46:32.839 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-21T15:46:32.839 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-21T15:46:32.839 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-21T15:46:32.839 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-21T15:46:32.840 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-21T15:46:32.841 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-21T15:46:32.841 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-21T15:46:32.841 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-21T15:46:32.841 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-21T15:46:32.841 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-21T15:46:32.842 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-21T15:46:32.843 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-21T15:46:32.843 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-21T15:46:32.843 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-21T15:46:32.843 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-21T15:46:32.843 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-21T15:46:32.844 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-21T15:46:32.845 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.845 DEBUG:teuthology.orchestra.run.smithi124:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-21T15:46:32.868 INFO:teuthology.orchestra.run.smithi124.stdout:postinstall scriptlet (using /bin/sh): 2023-10-21T15:46:32.868 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:32.869 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2023-10-21T15:46:32.870 INFO:teuthology.orchestra.run.smithi124.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-21T15:46:32.870 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-21T15:46:32.870 INFO:teuthology.orchestra.run.smithi124.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-21T15:46:32.870 INFO:teuthology.orchestra.run.smithi124.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-21T15:46:32.870 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.871 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2023-10-21T15:46:32.871 INFO:teuthology.orchestra.run.smithi124.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-21T15:46:32.871 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-21T15:46:32.871 INFO:teuthology.orchestra.run.smithi124.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.871 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2023-10-21T15:46:32.872 DEBUG:teuthology.orchestra.run.smithi124:> sudo rpm -qi grub2-tools 2023-10-21T15:46:32.946 INFO:teuthology.orchestra.run.smithi124.stdout:Name : grub2-tools 2023-10-21T15:46:32.947 INFO:teuthology.orchestra.run.smithi124.stdout:Epoch : 1 2023-10-21T15:46:32.947 INFO:teuthology.orchestra.run.smithi124.stdout:Version : 2.06 2023-10-21T15:46:32.947 INFO:teuthology.orchestra.run.smithi124.stdout:Release : 68.el9 2023-10-21T15:46:32.947 INFO:teuthology.orchestra.run.smithi124.stdout:Architecture: x86_64 2023-10-21T15:46:32.947 INFO:teuthology.orchestra.run.smithi124.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-21T15:46:32.948 INFO:teuthology.orchestra.run.smithi124.stdout:Group : Unspecified 2023-10-21T15:46:32.948 INFO:teuthology.orchestra.run.smithi124.stdout:Size : 8264254 2023-10-21T15:46:32.948 INFO:teuthology.orchestra.run.smithi124.stdout:License : GPLv3+ 2023-10-21T15:46:32.948 INFO:teuthology.orchestra.run.smithi124.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-21T15:46:32.948 INFO:teuthology.orchestra.run.smithi124.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:Packager : builder@centos.org 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:Vendor : CentOS 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-21T15:46:32.949 INFO:teuthology.orchestra.run.smithi124.stdout:Summary : Support tools for GRUB. 2023-10-21T15:46:32.950 INFO:teuthology.orchestra.run.smithi124.stdout:Description : 2023-10-21T15:46:32.950 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:46:32.950 INFO:teuthology.orchestra.run.smithi124.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-21T15:46:32.950 INFO:teuthology.orchestra.run.smithi124.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-21T15:46:32.950 INFO:teuthology.orchestra.run.smithi124.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-21T15:46:32.951 INFO:teuthology.orchestra.run.smithi124.stdout:hardware devices. 2023-10-21T15:46:32.951 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:46:32.951 INFO:teuthology.orchestra.run.smithi124.stdout:This subpackage provides tools for support of all platforms. 2023-10-21T15:46:32.952 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-21T15:46:32.952 INFO:teuthology.task.kernel:Updating grub on smithi124 to boot 6.6.0-rc5-g47672119149b 2023-10-21T15:46:32.953 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-21T15:46:33.580 INFO:teuthology.orchestra.run.smithi124.stderr:Generating grub configuration file ... 2023-10-21T15:46:34.990 INFO:teuthology.orchestra.run.smithi124.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-21T15:46:35.022 INFO:teuthology.orchestra.run.smithi124.stderr:done 2023-10-21T15:46:35.024 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-21T15:46:35.040 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.TL9sjWpMoa 2023-10-21T15:46:35.040 DEBUG:teuthology.orchestra.run.smithi124:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TL9sjWpMoa 2023-10-21T15:46:35.114 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.TL9sjWpMoa 2023-10-21T15:46:35.255 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.TL9sjWpMoa is 6KB 2023-10-21T15:46:35.303 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.TL9sjWpMoa 2023-10-21T15:46:35.318 DEBUG:teuthology.orchestra.run.smithi124:> sudo /bin/ls /boot/loader/entries || true 2023-10-21T15:46:35.382 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-21T15:46:35.383 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-21T15:46:35.383 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-21T15:46:35.383 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-21T15:46:35.384 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-21T15:46:35.527 DEBUG:teuthology.orchestra.run.smithi124:> sudo shutdown -r now 2023-10-21T15:46:39.706 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-21T15:46:39.706 DEBUG:teuthology.orchestra.run.smithi053:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-21T15:46:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-21T15:46:39.784 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-21T15:46:39.784 INFO:teuthology.orchestra.run.smithi053.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-21T15:46:39.784 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-21T15:46:39.785 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-21T15:46:39.785 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-21T15:46:39.785 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-21T15:46:39.785 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-21T15:46:39.785 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-21T15:46:39.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-21T15:46:39.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-21T15:46:39.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-21T15:46:39.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-21T15:46:39.786 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-21T15:46:39.787 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-21T15:46:39.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-21T15:46:39.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-21T15:46:39.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-21T15:46:39.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-21T15:46:39.788 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-21T15:46:39.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-21T15:46:39.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-21T15:46:39.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-21T15:46:39.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-21T15:46:39.789 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-21T15:46:39.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-21T15:46:39.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-21T15:46:39.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-21T15:46:39.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-21T15:46:39.790 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-21T15:46:39.791 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-21T15:46:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-21T15:46:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-21T15:46:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-21T15:46:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-21T15:46:39.792 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-21T15:46:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-21T15:46:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-21T15:46:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-21T15:46:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-21T15:46:39.793 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-21T15:46:39.794 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-21T15:46:39.795 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-21T15:46:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-21T15:46:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-21T15:46:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-21T15:46:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-21T15:46:39.796 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-21T15:46:39.797 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-21T15:46:39.798 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-21T15:46:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-21T15:46:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-21T15:46:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-21T15:46:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-21T15:46:39.799 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-21T15:46:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-21T15:46:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-21T15:46:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-21T15:46:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-21T15:46:39.800 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-21T15:46:39.801 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-21T15:46:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-21T15:46:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-21T15:46:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-21T15:46:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-21T15:46:39.802 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-21T15:46:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-21T15:46:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-21T15:46:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-21T15:46:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-21T15:46:39.803 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-21T15:46:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-21T15:46:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-21T15:46:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-21T15:46:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-21T15:46:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-21T15:46:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-21T15:46:39.806 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-21T15:46:39.807 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-21T15:46:39.808 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-21T15:46:39.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-21T15:46:39.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-21T15:46:39.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-21T15:46:39.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-21T15:46:39.809 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-21T15:46:39.810 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-21T15:46:39.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-21T15:46:39.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-21T15:46:39.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-21T15:46:39.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-21T15:46:39.811 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-21T15:46:39.812 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-21T15:46:39.813 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-21T15:46:39.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-21T15:46:39.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-21T15:46:39.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-21T15:46:39.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-21T15:46:39.814 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-21T15:46:39.815 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-21T15:46:39.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-21T15:46:39.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-21T15:46:39.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-21T15:46:39.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-21T15:46:39.816 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-21T15:46:39.817 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-21T15:46:39.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-21T15:46:39.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-21T15:46:39.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-21T15:46:39.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-21T15:46:39.818 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-21T15:46:39.819 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-21T15:46:39.820 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-21T15:46:39.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-21T15:46:39.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-21T15:46:39.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-21T15:46:39.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-21T15:46:39.821 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-21T15:46:39.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-21T15:46:39.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-21T15:46:39.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-21T15:46:39.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-21T15:46:39.822 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-21T15:46:39.823 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-21T15:46:39.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-21T15:46:39.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-21T15:46:39.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-21T15:46:39.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-21T15:46:39.824 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-21T15:46:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-21T15:46:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-21T15:46:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-21T15:46:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-21T15:46:39.825 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-21T15:46:39.826 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-21T15:46:39.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-21T15:46:39.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-21T15:46:39.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-21T15:46:39.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-21T15:46:39.827 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-21T15:46:39.828 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-21T15:46:39.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-21T15:46:39.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-21T15:46:39.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-21T15:46:39.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-21T15:46:39.829 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-21T15:46:39.830 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-21T15:46:39.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-21T15:46:39.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-21T15:46:39.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-21T15:46:39.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-21T15:46:39.831 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-21T15:46:39.832 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-21T15:46:39.833 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-21T15:46:39.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-21T15:46:39.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-21T15:46:39.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-21T15:46:39.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-21T15:46:39.834 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-21T15:46:39.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-21T15:46:39.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-21T15:46:39.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-21T15:46:39.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-21T15:46:39.835 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-21T15:46:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-21T15:46:39.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-21T15:46:39.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-21T15:46:39.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-21T15:46:39.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-21T15:46:39.837 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-21T15:46:39.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-21T15:46:39.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-21T15:46:39.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-21T15:46:39.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-21T15:46:39.838 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-21T15:46:39.839 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-21T15:46:39.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-21T15:46:39.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-21T15:46:39.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-21T15:46:39.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-21T15:46:39.840 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-21T15:46:39.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-21T15:46:39.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-21T15:46:39.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-21T15:46:39.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-21T15:46:39.841 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-21T15:46:39.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-21T15:46:39.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-21T15:46:39.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-21T15:46:39.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-21T15:46:39.842 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-21T15:46:39.843 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-21T15:46:39.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-21T15:46:39.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-21T15:46:39.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-21T15:46:39.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-21T15:46:39.844 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-21T15:46:39.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-21T15:46:39.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-21T15:46:39.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-21T15:46:39.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-21T15:46:39.845 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-21T15:46:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-21T15:46:39.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-21T15:46:39.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-21T15:46:39.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-21T15:46:39.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-21T15:46:39.847 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-21T15:46:39.848 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-21T15:46:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-21T15:46:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-21T15:46:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-21T15:46:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-21T15:46:39.849 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-21T15:46:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-21T15:46:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-21T15:46:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-21T15:46:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-21T15:46:39.850 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-21T15:46:39.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-21T15:46:39.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-21T15:46:39.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-21T15:46:39.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-21T15:46:39.851 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-21T15:46:39.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-21T15:46:39.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-21T15:46:39.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-21T15:46:39.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-21T15:46:39.852 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-21T15:46:39.853 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-21T15:46:39.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-21T15:46:39.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-21T15:46:39.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-21T15:46:39.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-21T15:46:39.854 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-21T15:46:39.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-21T15:46:39.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-21T15:46:39.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-21T15:46:39.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-21T15:46:39.855 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-21T15:46:39.856 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-21T15:46:39.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-21T15:46:39.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-21T15:46:39.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-21T15:46:39.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-21T15:46:39.857 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-21T15:46:39.858 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-21T15:46:39.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-21T15:46:39.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-21T15:46:39.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-21T15:46:39.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-21T15:46:39.859 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-21T15:46:39.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-21T15:46:39.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-21T15:46:39.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-21T15:46:39.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-21T15:46:39.860 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-21T15:46:39.861 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-21T15:46:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-21T15:46:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-21T15:46:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-21T15:46:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-21T15:46:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-21T15:46:39.863 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-21T15:46:39.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-21T15:46:39.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-21T15:46:39.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-21T15:46:39.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-21T15:46:39.864 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-21T15:46:39.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-21T15:46:39.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-21T15:46:39.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-21T15:46:39.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-21T15:46:39.865 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-21T15:46:39.866 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-21T15:46:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-21T15:46:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-21T15:46:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-21T15:46:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-21T15:46:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-21T15:46:39.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-21T15:46:39.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-21T15:46:39.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-21T15:46:39.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-21T15:46:39.868 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-21T15:46:39.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-21T15:46:39.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-21T15:46:39.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-21T15:46:39.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-21T15:46:39.869 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-21T15:46:39.870 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-21T15:46:39.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-21T15:46:39.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-21T15:46:39.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-21T15:46:39.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-21T15:46:39.871 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-21T15:46:39.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-21T15:46:39.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-21T15:46:39.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-21T15:46:39.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-21T15:46:39.872 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-21T15:46:39.873 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-21T15:46:39.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-21T15:46:39.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-21T15:46:39.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-21T15:46:39.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-21T15:46:39.874 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-21T15:46:39.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-21T15:46:39.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-21T15:46:39.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-21T15:46:39.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-21T15:46:39.875 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-21T15:46:39.876 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-21T15:46:39.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-21T15:46:39.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-21T15:46:39.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-21T15:46:39.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-21T15:46:39.877 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-21T15:46:39.878 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-21T15:46:39.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-21T15:46:39.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-21T15:46:39.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-21T15:46:39.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-21T15:46:39.879 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-21T15:46:39.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-21T15:46:39.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-21T15:46:39.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-21T15:46:39.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-21T15:46:39.880 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-21T15:46:39.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-21T15:46:39.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-21T15:46:39.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-21T15:46:39.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-21T15:46:39.881 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-21T15:46:39.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-21T15:46:39.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-21T15:46:39.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-21T15:46:39.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-21T15:46:39.882 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-21T15:46:39.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-21T15:46:39.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-21T15:46:39.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-21T15:46:39.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-21T15:46:39.883 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-21T15:46:39.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-21T15:46:39.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-21T15:46:39.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-21T15:46:39.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-21T15:46:39.884 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-21T15:46:39.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-21T15:46:39.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-21T15:46:39.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-21T15:46:39.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-21T15:46:39.885 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-21T15:46:39.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-21T15:46:39.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-21T15:46:39.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-21T15:46:39.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-21T15:46:39.886 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-21T15:46:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-21T15:46:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-21T15:46:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-21T15:46:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-21T15:46:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-21T15:46:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-21T15:46:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-21T15:46:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-21T15:46:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-21T15:46:39.888 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-21T15:46:39.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-21T15:46:39.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-21T15:46:39.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-21T15:46:39.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-21T15:46:39.889 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-21T15:46:39.890 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-21T15:46:39.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-21T15:46:39.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-21T15:46:39.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-21T15:46:39.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-21T15:46:39.891 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-21T15:46:39.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-21T15:46:39.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-21T15:46:39.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-21T15:46:39.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-21T15:46:39.892 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-21T15:46:39.893 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-21T15:46:39.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-21T15:46:39.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-21T15:46:39.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-21T15:46:39.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-21T15:46:39.894 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-21T15:46:39.895 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-21T15:46:39.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-21T15:46:39.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-21T15:46:39.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-21T15:46:39.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-21T15:46:39.896 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-21T15:46:39.897 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-21T15:46:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-21T15:46:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-21T15:46:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-21T15:46:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-21T15:46:39.898 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-21T15:46:39.899 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-21T15:46:39.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-21T15:46:39.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-21T15:46:39.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-21T15:46:39.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-21T15:46:39.900 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-21T15:46:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-21T15:46:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-21T15:46:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-21T15:46:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-21T15:46:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-21T15:46:39.902 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-21T15:46:39.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-21T15:46:39.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-21T15:46:39.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-21T15:46:39.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-21T15:46:39.903 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-21T15:46:39.904 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-21T15:46:39.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-21T15:46:39.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-21T15:46:39.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-21T15:46:39.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-21T15:46:39.905 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-21T15:46:39.906 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-21T15:46:39.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-21T15:46:39.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-21T15:46:39.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-21T15:46:39.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-21T15:46:39.907 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-21T15:46:39.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-21T15:46:39.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-21T15:46:39.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-21T15:46:39.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-21T15:46:39.908 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-21T15:46:39.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-21T15:46:39.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-21T15:46:39.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-21T15:46:39.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-21T15:46:39.909 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-21T15:46:39.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-21T15:46:39.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-21T15:46:39.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-21T15:46:39.910 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-21T15:46:39.911 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-21T15:46:39.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-21T15:46:39.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-21T15:46:39.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-21T15:46:39.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-21T15:46:39.912 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-21T15:46:39.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-21T15:46:39.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-21T15:46:39.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-21T15:46:39.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-21T15:46:39.913 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-21T15:46:39.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-21T15:46:39.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-21T15:46:39.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-21T15:46:39.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-21T15:46:39.914 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-21T15:46:39.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-21T15:46:39.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-21T15:46:39.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-21T15:46:39.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-21T15:46:39.915 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-21T15:46:39.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-21T15:46:39.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-21T15:46:39.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-21T15:46:39.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-21T15:46:39.916 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-21T15:46:39.917 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-21T15:46:39.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-21T15:46:39.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-21T15:46:39.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-21T15:46:39.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-21T15:46:39.918 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-21T15:46:39.919 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-21T15:46:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-21T15:46:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-21T15:46:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-21T15:46:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-21T15:46:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-21T15:46:39.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-21T15:46:39.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-21T15:46:39.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-21T15:46:39.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-21T15:46:39.922 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-21T15:46:39.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-21T15:46:39.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-21T15:46:39.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-21T15:46:39.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-21T15:46:39.923 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-21T15:46:39.924 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-21T15:46:39.925 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-21T15:46:39.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-21T15:46:39.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-21T15:46:39.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-21T15:46:39.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-21T15:46:39.926 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-21T15:46:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-21T15:46:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-21T15:46:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-21T15:46:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-21T15:46:39.927 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-21T15:46:39.928 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-21T15:46:39.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-21T15:46:39.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-21T15:46:39.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-21T15:46:39.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-21T15:46:39.929 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-21T15:46:39.930 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-21T15:46:39.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-21T15:46:39.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-21T15:46:39.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-21T15:46:39.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-21T15:46:39.931 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-21T15:46:39.932 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-21T15:46:39.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-21T15:46:39.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-21T15:46:39.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-21T15:46:39.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-21T15:46:39.933 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-21T15:46:39.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-21T15:46:39.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-21T15:46:39.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-21T15:46:39.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-21T15:46:39.934 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-21T15:46:39.935 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-21T15:46:39.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-21T15:46:39.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-21T15:46:39.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-21T15:46:39.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-21T15:46:39.936 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-21T15:46:39.937 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-21T15:46:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-21T15:46:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-21T15:46:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-21T15:46:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-21T15:46:39.938 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-21T15:46:39.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-21T15:46:39.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-21T15:46:39.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-21T15:46:39.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-21T15:46:39.939 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-21T15:46:39.940 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-21T15:46:39.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-21T15:46:39.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-21T15:46:39.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-21T15:46:39.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-21T15:46:39.941 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-21T15:46:39.942 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-21T15:46:39.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-21T15:46:39.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-21T15:46:39.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-21T15:46:39.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-21T15:46:39.943 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-21T15:46:39.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-21T15:46:39.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-21T15:46:39.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-21T15:46:39.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-21T15:46:39.944 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-21T15:46:39.945 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-21T15:46:39.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-21T15:46:39.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-21T15:46:39.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-21T15:46:39.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-21T15:46:39.946 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-21T15:46:39.947 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-21T15:46:39.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-21T15:46:39.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-21T15:46:39.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-21T15:46:39.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-21T15:46:39.948 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-21T15:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-21T15:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-21T15:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-21T15:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-21T15:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-21T15:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-21T15:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-21T15:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-21T15:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-21T15:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-21T15:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-21T15:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-21T15:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-21T15:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-21T15:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-21T15:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-21T15:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-21T15:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-21T15:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-21T15:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-21T15:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-21T15:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-21T15:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-21T15:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-21T15:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-21T15:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-21T15:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-21T15:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-21T15:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-21T15:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-21T15:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-21T15:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-21T15:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-21T15:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-21T15:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-21T15:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-21T15:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-21T15:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-21T15:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-21T15:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-21T15:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-21T15:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-21T15:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-21T15:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-21T15:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-21T15:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-21T15:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-21T15:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-21T15:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-21T15:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-21T15:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-21T15:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-21T15:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-21T15:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-21T15:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-21T15:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-21T15:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-21T15:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-21T15:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-21T15:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-21T15:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-21T15:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-21T15:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-21T15:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-21T15:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-21T15:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-21T15:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-21T15:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-21T15:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-21T15:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-21T15:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-21T15:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-21T15:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-21T15:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-21T15:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-21T15:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-21T15:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-21T15:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-21T15:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-21T15:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-21T15:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-21T15:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-21T15:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-21T15:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-21T15:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-21T15:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-21T15:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-21T15:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-21T15:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-21T15:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-21T15:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-21T15:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-21T15:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-21T15:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-21T15:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-21T15:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-21T15:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-21T15:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-21T15:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-21T15:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-21T15:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-21T15:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-21T15:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-21T15:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-21T15:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-21T15:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-21T15:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-21T15:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-21T15:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-21T15:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-21T15:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-21T15:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-21T15:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-21T15:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-21T15:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-21T15:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-21T15:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-21T15:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-21T15:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-21T15:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-21T15:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-21T15:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-21T15:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-21T15:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-21T15:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-21T15:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-21T15:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-21T15:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-21T15:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-21T15:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-21T15:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-21T15:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-21T15:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-21T15:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-21T15:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-21T15:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-21T15:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-21T15:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-21T15:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-21T15:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-21T15:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-21T15:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-21T15:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-21T15:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-21T15:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-21T15:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-21T15:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-21T15:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-21T15:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-21T15:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-21T15:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-21T15:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-21T15:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-21T15:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-21T15:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-21T15:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-21T15:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-21T15:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-21T15:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-21T15:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-21T15:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-21T15:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-21T15:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-21T15:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-21T15:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-21T15:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-21T15:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-21T15:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-21T15:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-21T15:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-21T15:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-21T15:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-21T15:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-21T15:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-21T15:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-21T15:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-21T15:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-21T15:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-21T15:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-21T15:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-21T15:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-21T15:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-21T15:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-21T15:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-21T15:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-21T15:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-21T15:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-21T15:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-21T15:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-21T15:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-21T15:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-21T15:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-21T15:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-21T15:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-21T15:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-21T15:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-21T15:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-21T15:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-21T15:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-21T15:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-21T15:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-21T15:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-21T15:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-21T15:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-21T15:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-21T15:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-21T15:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-21T15:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-21T15:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-21T15:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-21T15:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-21T15:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-21T15:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-21T15:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-21T15:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-21T15:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-21T15:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-21T15:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-21T15:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-21T15:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-21T15:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-21T15:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-21T15:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-21T15:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-21T15:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-21T15:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-21T15:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-21T15:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-21T15:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-21T15:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-21T15:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-21T15:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-21T15:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-21T15:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-21T15:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-21T15:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-21T15:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-21T15:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-21T15:46:40.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-21T15:46:40.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-21T15:46:40.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-21T15:46:40.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-21T15:46:40.019 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-21T15:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-21T15:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-21T15:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-21T15:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-21T15:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-21T15:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-21T15:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-21T15:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-21T15:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-21T15:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-21T15:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-21T15:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-21T15:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-21T15:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-21T15:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-21T15:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-21T15:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-21T15:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-21T15:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-21T15:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-21T15:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-21T15:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-21T15:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-21T15:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-21T15:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-21T15:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-21T15:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-21T15:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-21T15:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-21T15:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-21T15:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-21T15:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-21T15:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-21T15:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-21T15:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-21T15:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-21T15:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-21T15:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-21T15:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-21T15:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-21T15:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-21T15:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-21T15:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-21T15:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-21T15:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-21T15:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-21T15:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-21T15:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-21T15:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-21T15:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-21T15:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-21T15:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-21T15:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-21T15:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-21T15:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-21T15:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-21T15:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-21T15:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-21T15:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-21T15:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-21T15:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-21T15:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-21T15:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-21T15:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-21T15:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-21T15:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-21T15:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-21T15:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-21T15:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-21T15:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-21T15:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-21T15:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-21T15:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-21T15:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-21T15:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-21T15:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-21T15:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-21T15:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-21T15:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-21T15:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-21T15:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-21T15:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-21T15:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-21T15:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-21T15:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-21T15:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-21T15:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-21T15:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-21T15:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-21T15:46:40.046 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-21T15:46:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-21T15:46:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-21T15:46:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-21T15:46:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-21T15:46:40.047 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-21T15:46:40.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-21T15:46:40.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-21T15:46:40.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-21T15:46:40.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-21T15:46:40.048 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-21T15:46:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-21T15:46:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-21T15:46:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-21T15:46:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-21T15:46:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-21T15:46:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-21T15:46:40.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-21T15:46:40.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-21T15:46:40.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-21T15:46:40.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-21T15:46:40.051 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-21T15:46:40.052 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-21T15:46:40.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-21T15:46:40.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-21T15:46:40.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-21T15:46:40.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-21T15:46:40.053 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-21T15:46:40.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-21T15:46:40.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-21T15:46:40.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-21T15:46:40.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-21T15:46:40.054 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-21T15:46:40.055 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-21T15:46:40.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-21T15:46:40.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-21T15:46:40.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-21T15:46:40.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-21T15:46:40.056 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-21T15:46:40.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-21T15:46:40.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-21T15:46:40.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-21T15:46:40.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-21T15:46:40.057 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-21T15:46:40.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-21T15:46:40.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-21T15:46:40.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-21T15:46:40.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-21T15:46:40.058 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-21T15:46:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-21T15:46:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-21T15:46:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-21T15:46:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-21T15:46:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-21T15:46:40.060 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-21T15:46:40.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-21T15:46:40.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-21T15:46:40.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-21T15:46:40.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-21T15:46:40.061 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-21T15:46:40.062 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-21T15:46:40.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-21T15:46:40.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-21T15:46:40.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-21T15:46:40.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-21T15:46:40.063 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-21T15:46:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-21T15:46:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-21T15:46:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-21T15:46:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-21T15:46:40.064 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-21T15:46:40.065 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-21T15:46:40.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-21T15:46:40.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-21T15:46:40.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-21T15:46:40.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-21T15:46:40.066 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-21T15:46:40.067 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-21T15:46:40.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-21T15:46:40.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-21T15:46:40.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-21T15:46:40.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-21T15:46:40.068 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-21T15:46:40.069 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-21T15:46:40.070 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-21T15:46:40.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-21T15:46:40.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-21T15:46:40.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-21T15:46:40.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-21T15:46:40.071 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-21T15:46:40.072 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-21T15:46:40.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-21T15:46:40.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-21T15:46:40.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-21T15:46:40.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-21T15:46:40.073 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-21T15:46:40.074 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-21T15:46:40.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-21T15:46:40.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-21T15:46:40.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-21T15:46:40.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-21T15:46:40.075 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-21T15:46:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-21T15:46:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-21T15:46:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-21T15:46:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-21T15:46:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-21T15:46:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-21T15:46:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-21T15:46:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-21T15:46:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-21T15:46:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-21T15:46:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-21T15:46:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-21T15:46:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-21T15:46:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-21T15:46:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-21T15:46:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-21T15:46:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-21T15:46:40.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-21T15:46:40.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-21T15:46:40.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-21T15:46:40.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-21T15:46:40.081 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-21T15:46:40.082 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-21T15:46:40.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-21T15:46:40.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-21T15:46:40.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-21T15:46:40.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-21T15:46:40.083 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-21T15:46:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-21T15:46:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-21T15:46:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-21T15:46:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-21T15:46:40.084 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-21T15:46:40.085 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-21T15:46:40.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-21T15:46:40.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-21T15:46:40.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-21T15:46:40.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-21T15:46:40.086 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-21T15:46:40.087 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-21T15:46:40.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-21T15:46:40.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-21T15:46:40.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-21T15:46:40.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-21T15:46:40.088 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-21T15:46:40.089 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-21T15:46:40.090 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-21T15:46:40.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-21T15:46:40.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-21T15:46:40.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-21T15:46:40.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-21T15:46:40.091 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-21T15:46:40.092 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-21T15:46:40.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-21T15:46:40.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-21T15:46:40.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-21T15:46:40.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-21T15:46:40.093 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-21T15:46:40.094 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-21T15:46:40.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-21T15:46:40.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-21T15:46:40.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-21T15:46:40.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-21T15:46:40.095 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-21T15:46:40.096 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-21T15:46:40.097 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-21T15:46:40.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-21T15:46:40.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-21T15:46:40.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-21T15:46:40.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-21T15:46:40.098 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-21T15:46:40.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-21T15:46:40.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-21T15:46:40.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-21T15:46:40.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-21T15:46:40.099 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-21T15:46:40.100 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-21T15:46:40.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-21T15:46:40.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-21T15:46:40.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-21T15:46:40.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-21T15:46:40.101 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-21T15:46:40.102 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-21T15:46:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-21T15:46:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-21T15:46:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-21T15:46:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-21T15:46:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-21T15:46:40.104 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-21T15:46:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-21T15:46:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-21T15:46:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-21T15:46:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-21T15:46:40.105 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-21T15:46:40.106 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-21T15:46:40.106 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-21T15:46:40.106 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-21T15:46:40.106 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-21T15:46:40.106 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-21T15:46:40.107 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-21T15:46:40.108 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-21T15:46:40.108 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-21T15:46:40.108 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-21T15:46:40.108 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-21T15:46:40.108 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-21T15:46:40.109 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-21T15:46:40.109 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-21T15:46:40.109 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-21T15:46:40.109 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-21T15:46:40.109 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-21T15:46:40.110 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-21T15:46:40.111 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-21T15:46:40.111 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-21T15:46:40.111 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-21T15:46:40.111 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-21T15:46:40.111 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-21T15:46:40.112 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-21T15:46:40.113 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-21T15:46:40.113 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-21T15:46:40.113 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-21T15:46:40.113 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-21T15:46:40.113 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-21T15:46:40.114 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-21T15:46:40.115 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-21T15:46:40.116 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-21T15:46:40.116 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-21T15:46:40.116 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-21T15:46:40.116 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-21T15:46:40.116 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-21T15:46:40.117 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-21T15:46:40.118 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-21T15:46:40.118 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-21T15:46:40.118 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-21T15:46:40.118 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-21T15:46:40.118 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-21T15:46:40.119 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-21T15:46:40.119 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-21T15:46:40.119 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-21T15:46:40.119 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-21T15:46:40.119 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-21T15:46:40.120 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-21T15:46:40.121 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-21T15:46:40.121 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-21T15:46:40.121 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-21T15:46:40.121 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-21T15:46:40.121 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-21T15:46:40.122 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-21T15:46:40.122 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-21T15:46:40.122 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-21T15:46:40.122 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-21T15:46:40.122 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-21T15:46:40.123 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-21T15:46:40.124 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-21T15:46:40.125 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-21T15:46:40.125 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-21T15:46:40.125 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-21T15:46:40.125 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-21T15:46:40.125 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-21T15:46:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-21T15:46:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-21T15:46:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-21T15:46:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-21T15:46:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-21T15:46:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-21T15:46:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-21T15:46:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-21T15:46:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-21T15:46:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-21T15:46:40.128 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-21T15:46:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-21T15:46:40.130 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-21T15:46:40.131 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-21T15:46:40.131 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-21T15:46:40.131 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-21T15:46:40.131 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-21T15:46:40.131 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-21T15:46:40.132 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-21T15:46:40.133 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-21T15:46:40.134 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-21T15:46:40.134 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-21T15:46:40.134 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-21T15:46:40.134 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-21T15:46:40.134 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-21T15:46:40.135 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-21T15:46:40.135 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-21T15:46:40.135 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-21T15:46:40.135 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-21T15:46:40.135 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-21T15:46:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-21T15:46:40.137 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-21T15:46:40.138 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-21T15:46:40.138 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-21T15:46:40.138 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-21T15:46:40.138 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-21T15:46:40.138 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-21T15:46:40.139 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-21T15:46:40.140 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-21T15:46:40.140 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-21T15:46:40.140 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-21T15:46:40.140 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-21T15:46:40.140 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-21T15:46:40.141 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-21T15:46:40.142 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-21T15:46:40.142 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-21T15:46:40.142 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-21T15:46:40.142 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-21T15:46:40.142 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-21T15:46:40.143 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-21T15:46:40.144 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-21T15:46:40.144 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-21T15:46:40.144 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-21T15:46:40.144 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-21T15:46:40.144 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-21T15:46:40.145 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-21T15:46:40.145 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-21T15:46:40.145 INFO:teuthology.orchestra.run.smithi053.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-21T15:46:40.146 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-21T15:46:40.146 DEBUG:teuthology.orchestra.run.smithi053:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-21T15:46:40.172 INFO:teuthology.orchestra.run.smithi053.stdout:postinstall scriptlet (using /bin/sh): 2023-10-21T15:46:40.172 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-21T15:46:40.173 INFO:teuthology.orchestra.run.smithi053.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:40.173 INFO:teuthology.orchestra.run.smithi053.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:40.173 INFO:teuthology.orchestra.run.smithi053.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-21T15:46:40.173 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:40.173 INFO:teuthology.orchestra.run.smithi053.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-21T15:46:40.174 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-10-21T15:46:40.174 INFO:teuthology.orchestra.run.smithi053.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-21T15:46:40.174 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-21T15:46:40.174 INFO:teuthology.orchestra.run.smithi053.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-21T15:46:40.174 INFO:teuthology.orchestra.run.smithi053.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-21T15:46:40.175 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-21T15:46:40.175 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-10-21T15:46:40.175 INFO:teuthology.orchestra.run.smithi053.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-21T15:46:40.175 INFO:teuthology.orchestra.run.smithi053.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-21T15:46:40.175 INFO:teuthology.orchestra.run.smithi053.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-21T15:46:40.176 INFO:teuthology.orchestra.run.smithi053.stdout:fi 2023-10-21T15:46:40.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2023-10-21T15:46:40.261 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2023-10-21T15:46:40.261 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2023-10-21T15:46:40.262 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2023-10-21T15:46:40.262 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 68.el9 2023-10-21T15:46:40.262 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2023-10-21T15:46:40.262 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-21T15:46:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2023-10-21T15:46:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8264254 2023-10-21T15:46:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2023-10-21T15:46:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-21T15:46:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-21T15:46:40.264 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-21T15:46:40.264 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-21T15:46:40.264 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2023-10-21T15:46:40.264 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2023-10-21T15:46:40.264 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-21T15:46:40.265 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2023-10-21T15:46:40.265 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2023-10-21T15:46:40.265 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:46:40.265 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-21T15:46:40.265 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-21T15:46:40.266 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-21T15:46:40.266 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2023-10-21T15:46:40.266 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:46:40.266 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2023-10-21T15:46:40.267 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-21T15:46:40.267 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 6.6.0-rc5-g47672119149b 2023-10-21T15:46:40.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-21T15:46:40.854 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2023-10-21T15:46:42.318 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-21T15:46:42.339 INFO:teuthology.orchestra.run.smithi053.stderr:done 2023-10-21T15:46:42.350 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2023-10-21T15:46:42.368 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.o60lM1O9ag 2023-10-21T15:46:42.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.o60lM1O9ag 2023-10-21T15:46:42.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.o60lM1O9ag 2023-10-21T15:46:42.591 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.o60lM1O9ag is 6KB 2023-10-21T15:46:42.640 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.o60lM1O9ag 2023-10-21T15:46:42.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2023-10-21T15:46:42.723 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-21T15:46:42.724 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-21T15:46:42.724 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-21T15:46:42.724 INFO:teuthology.orchestra.run.smithi053.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-21T15:46:42.725 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-21T15:46:42.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2023-10-21T15:47:05.545 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-21T15:47:05.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:47:05.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:47:12.919 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-21T15:47:13.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:47:13.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:47:23.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-21T15:47:31.582 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-10-21T15:47:32.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:47:32.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:47:35.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-21T15:47:40.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:47:40.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:47:47.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:47:47.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:47:58.971 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-10-21T15:48:06.394 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-10-21T15:48:10.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:48:10.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:48:14.059 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-10-21T15:48:14.855 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:48:14.855 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-21T15:48:14.856 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2023-10-21T15:48:14.898 INFO:teuthology.orchestra.run.smithi053.stdout:6.6.0-rc5-g47672119149b 2023-10-21T15:48:14.899 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-21T15:48:14.899 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-21T15:48:14.899 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-21T15:48:15.901 DEBUG:teuthology.parallel:result is None 2023-10-21T15:48:21.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:48:21.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:48:22.274 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-10-21T15:48:22.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:48:22.865 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g47672119149b"... 2023-10-21T15:48:22.866 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2023-10-21T15:48:22.905 INFO:teuthology.orchestra.run.smithi124.stdout:6.6.0-rc5-g47672119149b 2023-10-21T15:48:22.905 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-21T15:48:22.906 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-21T15:48:22.906 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-21T15:48:23.907 DEBUG:teuthology.parallel:result is None 2023-10-21T15:48:23.908 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-21T15:48:23.918 INFO:teuthology.task.internal:Creating test directory... 2023-10-21T15:48:23.918 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-21T15:48:23.926 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-21T15:48:23.981 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-21T15:48:23.989 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-21T15:48:24.001 INFO:teuthology.task.internal:Creating archive directory... 2023-10-21T15:48:24.001 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-21T15:48:24.007 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-21T15:48:24.076 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-21T15:48:24.084 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-21T15:48:24.085 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-21T15:48:24.090 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-21T15:48:24.209 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-21T15:48:24.225 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-21T15:48:24.277 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-21T15:48:24.299 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-21T15:48:24.304 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-21T15:48:24.312 INFO:teuthology.task.internal:Configuring sudo... 2023-10-21T15:48:24.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-21T15:48:24.320 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-21T15:48:24.403 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-21T15:48:24.413 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-21T15:48:24.413 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-21T15:48:24.419 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-21T15:48:24.462 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-21T15:48:24.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-21T15:48:24.631 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-21T15:48:24.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-21T15:48:24.783 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T15:48:24.783 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-21T15:48:24.860 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-21T15:48:24.942 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-21T15:48:25.029 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-21T15:48:25.097 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-21T15:48:25.189 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T15:48:25.189 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-21T15:48:25.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2023-10-21T15:48:25.268 DEBUG:teuthology.orchestra.run.smithi124:> sudo service rsyslog restart 2023-10-21T15:48:25.396 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-21T15:48:25.407 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-21T15:48:25.689 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-21T15:48:25.702 INFO:teuthology.task.internal:Starting timer... 2023-10-21T15:48:25.703 INFO:teuthology.run_tasks:Running task pcp... 2023-10-21T15:48:25.734 INFO:teuthology.run_tasks:Running task selinux... 2023-10-21T15:48:25.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2023-10-21T15:48:25.874 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2023-10-21T15:48:25.879 DEBUG:teuthology.orchestra.run.smithi124:> sudo service auditd rotate 2023-10-21T15:48:26.013 INFO:teuthology.orchestra.run.smithi124.stdout:Rotating logs: 2023-10-21T15:48:26.017 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-21T15:48:26.017 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2023-10-21T15:48:26.079 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2023-10-21T15:48:26.079 DEBUG:teuthology.orchestra.run.smithi124:> /usr/sbin/getenforce 2023-10-21T15:48:26.139 INFO:teuthology.orchestra.run.smithi124.stdout:Permissive 2023-10-21T15:48:26.139 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi124.front.sepia.ceph.com': 'permissive'} 2023-10-21T15:48:26.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-21T15:48:26.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T15:48:26.224 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-21T15:48:26.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T15:48:26.304 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-21T15:48:26.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2023-10-21T15:48:26.409 DEBUG:teuthology.orchestra.run.smithi124:> sudo /usr/sbin/setenforce permissive 2023-10-21T15:48:26.493 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-21T15:48:26.504 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-21T15:48:26.509 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-21T15:48:26.648 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-21T15:48:26.664 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-21T15:48:26.665 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi124.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-21T15:58:42.964 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi124.front.sepia.ceph.com')] 2023-10-21T15:58:42.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:58:42.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:58:43.114 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-10-21T15:58:43.268 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-10-21T15:58:43.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:58:43.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T15:58:43.406 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-10-21T15:58:43.560 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-10-21T15:58:43.561 INFO:teuthology.run_tasks:Running task clock... 2023-10-21T15:58:43.572 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-21T15:58:43.572 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-21T15:58:43.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-21T15:58:43.582 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-21T15:58:43.582 DEBUG:teuthology.orchestra.run.smithi124:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-21T15:58:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-21T15:58:43.691 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-21T15:58:43.743 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-21T15:58:43.743 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-21T15:58:43.833 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: ntpd: command not found 2023-10-21T15:58:43.850 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2023-10-21T15:58:43.873 INFO:teuthology.orchestra.run.smithi124.stdout:506 Cannot talk to daemon 2023-10-21T15:58:43.896 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2023-10-21T15:58:43.924 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-21T15:58:43.952 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-21T15:58:43.978 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-21T15:58:44.010 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-21T15:58:44.105 INFO:teuthology.orchestra.run.smithi124.stderr:bash: line 1: ntpq: command not found 2023-10-21T15:58:44.114 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-21T15:58:44.114 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2023-10-21T15:58:44.115 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.115 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.115 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.115 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2023-10-21T15:58:44.163 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-21T15:58:44.163 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-10-21T15:58:44.163 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.163 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.163 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.164 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T15:58:44.165 INFO:teuthology.run_tasks:Running task install... 2023-10-21T15:58:44.176 DEBUG:teuthology.task.install:project ceph 2023-10-21T15:58:44.177 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'}} 2023-10-21T15:58:44.177 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'} 2023-10-21T15:58:44.177 INFO:teuthology.task.install:Using flavor: default 2023-10-21T15:58:44.184 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-21T15:58:44.184 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-21T15:58:44.185 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc', 'tag': None, 'wait_for_package': False} 2023-10-21T15:58:44.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f0804064d67a2a63dbabf1801ff452580a050cfc 2023-10-21T15:58:44.188 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc', 'tag': None, 'wait_for_package': False} 2023-10-21T15:58:44.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f0804064d67a2a63dbabf1801ff452580a050cfc 2023-10-21T15:58:44.415 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/ 2023-10-21T15:58:44.415 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6820.gf0804064 2023-10-21T15:58:44.432 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/ 2023-10-21T15:58:44.433 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6820.gf0804064 2023-10-21T15:58:44.546 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-21T15:58:44.547 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T15:58:44.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-21T15:58:44.562 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/f0804064d67a2a63dbabf1801ff452580a050cfc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-21T15:58:44.562 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T15:58:44.562 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-21T15:58:44.660 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-21T15:58:44.661 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f0804064d67a2a63dbabf1801ff452580a050cfc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-21T15:58:44.664 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-21T15:58:44.664 DEBUG:teuthology.orchestra.run.smithi124:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f0804064d67a2a63dbabf1801ff452580a050cfc/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-21T15:58:44.804 DEBUG:teuthology.orchestra.run.smithi124:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-21T15:58:44.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-21T15:58:44.940 DEBUG:teuthology.orchestra.run.smithi124:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-21T15:58:44.951 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-21T15:58:45.029 INFO:teuthology.orchestra.run.smithi124.stdout:check_obsoletes = 1 2023-10-21T15:58:45.033 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum clean all 2023-10-21T15:58:45.050 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2023-10-21T15:58:45.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2023-10-21T15:58:45.392 INFO:teuthology.orchestra.run.smithi124.stdout:72 files removed 2023-10-21T15:58:45.431 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-21T15:58:45.441 INFO:teuthology.orchestra.run.smithi053.stdout:72 files removed 2023-10-21T15:58:45.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-21T15:58:46.533 INFO:teuthology.orchestra.run.smithi124.stdout:ceph packages for x86_64 132 kB/s | 78 kB 00:00 2023-10-21T15:58:46.644 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 122 kB/s | 78 kB 00:00 2023-10-21T15:58:47.033 INFO:teuthology.orchestra.run.smithi124.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-10-21T15:58:47.162 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-10-21T15:58:47.526 INFO:teuthology.orchestra.run.smithi124.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2023-10-21T15:58:47.738 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-10-21T15:58:48.187 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-10-21T15:58:48.333 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-21T15:58:49.846 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2023-10-21T15:58:49.925 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2023-10-21T15:58:50.840 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 7.9 MB 00:00 2023-10-21T15:58:50.896 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 7.9 MB 00:00 2023-10-21T15:58:53.968 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 18 MB 00:01 2023-10-21T15:58:55.122 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 18 MB 00:03 2023-10-21T15:58:58.714 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 5.7 MB 00:01 2023-10-21T15:58:59.382 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.7 MB 00:00 2023-10-21T15:59:00.256 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 15 kB 00:00 2023-10-21T15:59:00.984 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2023-10-21T15:59:01.949 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 13 MB/s | 19 MB 00:01 2023-10-21T15:59:03.177 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 19 MB 00:02 2023-10-21T15:59:07.158 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-10-21T15:59:08.644 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2023-10-21T15:59:09.768 INFO:teuthology.orchestra.run.smithi124.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-21T15:59:09.770 INFO:teuthology.orchestra.run.smithi124.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-21T15:59:09.791 INFO:teuthology.orchestra.run.smithi124.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-21T15:59:09.792 INFO:teuthology.orchestra.run.smithi124.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-21T15:59:09.793 INFO:teuthology.orchestra.run.smithi124.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-21T15:59:09.795 INFO:teuthology.orchestra.run.smithi124.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-21T15:59:09.797 INFO:teuthology.orchestra.run.smithi124.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-21T15:59:09.801 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-21T15:59:09.802 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-21T15:59:09.879 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2023-10-21T15:59:09.898 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2023-10-21T15:59:09.899 INFO:teuthology.orchestra.run.smithi124.stdout: Package Arch Version Repository Size 2023-10-21T15:59:09.899 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2023-10-21T15:59:09.899 INFO:teuthology.orchestra.run.smithi124.stdout:Installing: 2023-10-21T15:59:09.899 INFO:teuthology.orchestra.run.smithi124.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-21T15:59:09.900 INFO:teuthology.orchestra.run.smithi124.stdout: ceph x86_64 2:18.0.0-6820.gf0804064.el9 ceph 6.5 k 2023-10-21T15:59:09.900 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base x86_64 2:18.0.0-6820.gf0804064.el9 ceph 5.2 M 2023-10-21T15:59:09.900 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 855 k 2023-10-21T15:59:09.900 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6820.gf0804064.el9 ceph 141 k 2023-10-21T15:59:09.900 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr x86_64 2:18.0.0-6820.gf0804064.el9 ceph 1.5 M 2023-10-21T15:59:09.901 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 132 k 2023-10-21T15:59:09.901 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 1.8 M 2023-10-21T15:59:09.901 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 7.4 M 2023-10-21T15:59:09.901 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 48 k 2023-10-21T15:59:09.901 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw x86_64 2:18.0.0-6820.gf0804064.el9 ceph 8.7 M 2023-10-21T15:59:09.902 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test x86_64 2:18.0.0-6820.gf0804064.el9 ceph 41 M 2023-10-21T15:59:09.902 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 262 k 2023-10-21T15:59:09.902 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 270 k 2023-10-21T15:59:09.902 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror x86_64 2:18.0.0-6820.gf0804064.el9 ceph 218 k 2023-10-21T15:59:09.902 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 18 k 2023-10-21T15:59:09.903 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-21T15:59:09.903 INFO:teuthology.orchestra.run.smithi124.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-21T15:59:09.903 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-21T15:59:09.903 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel x86_64 2:18.0.0-6820.gf0804064.el9 ceph 31 k 2023-10-21T15:59:09.904 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 716 k 2023-10-21T15:59:09.904 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel x86_64 2:18.0.0-6820.gf0804064.el9 ceph 126 k 2023-10-21T15:59:09.904 INFO:teuthology.orchestra.run.smithi124.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-21T15:59:09.904 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-21T15:59:09.904 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-21T15:59:09.905 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-21T15:59:09.905 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-21T15:59:09.905 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-21T15:59:09.905 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs x86_64 2:18.0.0-6820.gf0804064.el9 ceph 162 k 2023-10-21T15:59:09.906 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados x86_64 2:18.0.0-6820.gf0804064.el9 ceph 320 k 2023-10-21T15:59:09.906 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 298 k 2023-10-21T15:59:09.906 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw x86_64 2:18.0.0-6820.gf0804064.el9 ceph 99 k 2023-10-21T15:59:09.906 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 85 k 2023-10-21T15:59:09.906 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.0 M 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 168 k 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrading: 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager x86_64 1:1.45.5-1.el9 CentOS-BaseOS 2.2 M 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-libnm x86_64 1:1.45.5-1.el9 CentOS-BaseOS 1.8 M 2023-10-21T15:59:09.907 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-team x86_64 1:1.45.5-1.el9 CentOS-BaseOS 42 k 2023-10-21T15:59:09.908 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-tui x86_64 1:1.45.5-1.el9 CentOS-BaseOS 245 k 2023-10-21T15:59:09.908 INFO:teuthology.orchestra.run.smithi124.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-21T15:59:09.908 INFO:teuthology.orchestra.run.smithi124.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-21T15:59:09.908 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-21T15:59:09.908 INFO:teuthology.orchestra.run.smithi124.stdout: librados2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.3 M 2023-10-21T15:59:09.909 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.0 M 2023-10-21T15:59:09.909 INFO:teuthology.orchestra.run.smithi124.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-21T15:59:09.909 INFO:teuthology.orchestra.run.smithi124.stdout:Installing dependencies: 2023-10-21T15:59:09.909 INFO:teuthology.orchestra.run.smithi124.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-21T15:59:09.909 INFO:teuthology.orchestra.run.smithi124.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-21T15:59:09.910 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common x86_64 2:18.0.0-6820.gf0804064.el9 ceph 19 M 2023-10-21T15:59:09.910 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 23 k 2023-10-21T15:59:09.910 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds x86_64 2:18.0.0-6820.gf0804064.el9 ceph 2.1 M 2023-10-21T15:59:09.910 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 243 k 2023-10-21T15:59:09.910 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon x86_64 2:18.0.0-6820.gf0804064.el9 ceph 4.4 M 2023-10-21T15:59:09.911 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 18 M 2023-10-21T15:59:09.911 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 15 k 2023-10-21T15:59:09.911 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux x86_64 2:18.0.0-6820.gf0804064.el9 ceph 24 k 2023-10-21T15:59:09.911 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-21T15:59:09.912 INFO:teuthology.orchestra.run.smithi124.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-21T15:59:09.913 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-21T15:59:09.913 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-21T15:59:09.913 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite x86_64 2:18.0.0-6820.gf0804064.el9 ceph 164 k 2023-10-21T15:59:09.913 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-21T15:59:09.913 INFO:teuthology.orchestra.run.smithi124.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-21T15:59:09.914 INFO:teuthology.orchestra.run.smithi124.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-21T15:59:09.914 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-21T15:59:09.914 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-21T15:59:09.914 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 472 k 2023-10-21T15:59:09.914 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-21T15:59:09.915 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 4.8 M 2023-10-21T15:59:09.915 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-21T15:59:09.915 INFO:teuthology.orchestra.run.smithi124.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-21T15:59:09.915 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-21T15:59:09.916 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-21T15:59:09.917 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-21T15:59:09.917 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-21T15:59:09.917 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-21T15:59:09.917 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 45 k 2023-10-21T15:59:09.917 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common x86_64 2:18.0.0-6820.gf0804064.el9 ceph 127 k 2023-10-21T15:59:09.918 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-21T15:59:09.918 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-21T15:59:09.918 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-21T15:59:09.918 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-21T15:59:09.918 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-21T15:59:09.919 INFO:teuthology.orchestra.run.smithi124.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-21T15:59:09.919 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-21T15:59:09.919 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-21T15:59:09.919 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-21T15:59:09.919 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-21T15:59:09.920 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-21T15:59:09.920 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-21T15:59:09.920 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-21T15:59:09.920 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-21T15:59:09.920 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-21T15:59:09.921 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-21T15:59:09.921 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-21T15:59:09.921 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-21T15:59:09.921 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-21T15:59:09.921 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-21T15:59:09.922 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-21T15:59:09.922 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-21T15:59:09.922 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-21T15:59:09.922 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-21T15:59:09.922 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-21T15:59:09.923 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-21T15:59:09.923 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-21T15:59:09.923 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-21T15:59:09.923 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-21T15:59:09.923 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-21T15:59:09.924 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-21T15:59:09.924 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-21T15:59:09.924 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-21T15:59:09.924 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-21T15:59:09.924 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-21T15:59:09.925 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-21T15:59:09.925 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-21T15:59:09.925 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-21T15:59:09.925 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-21T15:59:09.925 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-21T15:59:09.926 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-21T15:59:09.926 INFO:teuthology.orchestra.run.smithi124.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-21T15:59:09.926 INFO:teuthology.orchestra.run.smithi124.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-21T15:59:09.926 INFO:teuthology.orchestra.run.smithi124.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-21T15:59:09.926 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-21T15:59:09.927 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-21T15:59:09.927 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-21T15:59:09.927 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-21T15:59:09.927 INFO:teuthology.orchestra.run.smithi124.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-21T15:59:09.927 INFO:teuthology.orchestra.run.smithi124.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout:Installing weak dependencies: 2023-10-21T15:59:09.928 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-21T15:59:09.929 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:59:09.929 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction Summary 2023-10-21T15:59:09.929 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2023-10-21T15:59:09.929 INFO:teuthology.orchestra.run.smithi124.stdout:Install 133 Packages 2023-10-21T15:59:09.929 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrade 10 Packages 2023-10-21T15:59:09.930 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:59:09.930 INFO:teuthology.orchestra.run.smithi124.stdout:Total download size: 200 M 2023-10-21T15:59:09.930 INFO:teuthology.orchestra.run.smithi124.stdout:Downloading Packages: 2023-10-21T15:59:10.326 INFO:teuthology.orchestra.run.smithi124.stdout:(1/143): ceph-18.0.0-6820.gf0804064.el9.x86_64. 30 kB/s | 6.5 kB 00:00 2023-10-21T15:59:10.545 INFO:teuthology.orchestra.run.smithi124.stdout:(2/143): ceph-fuse-18.0.0-6820.gf0804064.el9.x8 3.8 MB/s | 855 kB 00:00 2023-10-21T15:59:10.627 INFO:teuthology.orchestra.run.smithi124.stdout:(3/143): ceph-immutable-object-cache-18.0.0-682 1.7 MB/s | 141 kB 00:00 2023-10-21T15:59:10.969 INFO:teuthology.orchestra.run.smithi124.stdout:(4/143): ceph-mds-18.0.0-6820.gf0804064.el9.x86 6.1 MB/s | 2.1 MB 00:00 2023-10-21T15:59:11.244 INFO:teuthology.orchestra.run.smithi124.stdout:(5/143): ceph-base-18.0.0-6820.gf0804064.el9.x8 4.6 MB/s | 5.2 MB 00:01 2023-10-21T15:59:11.303 INFO:teuthology.orchestra.run.smithi124.stdout:(6/143): ceph-mgr-18.0.0-6820.gf0804064.el9.x86 4.4 MB/s | 1.5 MB 00:00 2023-10-21T15:59:11.370 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-21T15:59:11.372 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-21T15:59:11.394 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-21T15:59:11.396 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-21T15:59:11.397 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-21T15:59:11.399 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-21T15:59:11.402 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-21T15:59:11.406 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-21T15:59:11.407 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-21T15:59:11.488 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-10-21T15:59:11.507 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2023-10-21T15:59:11.508 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-10-21T15:59:11.508 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2023-10-21T15:59:11.508 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2023-10-21T15:59:11.508 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-21T15:59:11.509 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.0.0-6820.gf0804064.el9 ceph 6.5 k 2023-10-21T15:59:11.509 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.0.0-6820.gf0804064.el9 ceph 5.2 M 2023-10-21T15:59:11.509 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 855 k 2023-10-21T15:59:11.509 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6820.gf0804064.el9 ceph 141 k 2023-10-21T15:59:11.510 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.0.0-6820.gf0804064.el9 ceph 1.5 M 2023-10-21T15:59:11.510 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 132 k 2023-10-21T15:59:11.510 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 1.8 M 2023-10-21T15:59:11.510 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 7.4 M 2023-10-21T15:59:11.510 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 48 k 2023-10-21T15:59:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.0.0-6820.gf0804064.el9 ceph 8.7 M 2023-10-21T15:59:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.0.0-6820.gf0804064.el9 ceph 41 M 2023-10-21T15:59:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 262 k 2023-10-21T15:59:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 270 k 2023-10-21T15:59:11.511 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.0.0-6820.gf0804064.el9 ceph 218 k 2023-10-21T15:59:11.512 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 18 k 2023-10-21T15:59:11.512 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-21T15:59:11.512 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-21T15:59:11.512 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-21T15:59:11.512 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.0.0-6820.gf0804064.el9 ceph 31 k 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 716 k 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.0.0-6820.gf0804064.el9 ceph 126 k 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-21T15:59:11.513 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-21T15:59:11.514 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-21T15:59:11.514 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-21T15:59:11.514 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.0.0-6820.gf0804064.el9 ceph 162 k 2023-10-21T15:59:11.514 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.0.0-6820.gf0804064.el9 ceph 320 k 2023-10-21T15:59:11.514 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 298 k 2023-10-21T15:59:11.515 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.0.0-6820.gf0804064.el9 ceph 99 k 2023-10-21T15:59:11.515 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 85 k 2023-10-21T15:59:11.515 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.0 M 2023-10-21T15:59:11.515 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 168 k 2023-10-21T15:59:11.515 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-21T15:59:11.516 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2023-10-21T15:59:11.516 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.45.5-1.el9 CentOS-BaseOS 2.2 M 2023-10-21T15:59:11.516 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.45.5-1.el9 CentOS-BaseOS 1.8 M 2023-10-21T15:59:11.516 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.45.5-1.el9 CentOS-BaseOS 42 k 2023-10-21T15:59:11.516 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.45.5-1.el9 CentOS-BaseOS 245 k 2023-10-21T15:59:11.517 INFO:teuthology.orchestra.run.smithi053.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-21T15:59:11.517 INFO:teuthology.orchestra.run.smithi053.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-21T15:59:11.517 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-21T15:59:11.517 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.3 M 2023-10-21T15:59:11.517 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 3.0 M 2023-10-21T15:59:11.518 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-21T15:59:11.518 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-10-21T15:59:11.518 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-21T15:59:11.518 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-21T15:59:11.518 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.0.0-6820.gf0804064.el9 ceph 19 M 2023-10-21T15:59:11.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 23 k 2023-10-21T15:59:11.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.0.0-6820.gf0804064.el9 ceph 2.1 M 2023-10-21T15:59:11.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 243 k 2023-10-21T15:59:11.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.0.0-6820.gf0804064.el9 ceph 4.4 M 2023-10-21T15:59:11.519 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.0.0-6820.gf0804064.el9 ceph 18 M 2023-10-21T15:59:11.520 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6820.gf0804064.el9 ceph-noarch 15 k 2023-10-21T15:59:11.520 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.0.0-6820.gf0804064.el9 ceph 24 k 2023-10-21T15:59:11.520 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-21T15:59:11.520 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-21T15:59:11.520 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-21T15:59:11.521 INFO:teuthology.orchestra.run.smithi053.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-21T15:59:11.521 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-21T15:59:11.521 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-21T15:59:11.521 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-21T15:59:11.521 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.0.0-6820.gf0804064.el9 ceph 164 k 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-21T15:59:11.522 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-21T15:59:11.523 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-21T15:59:11.523 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 472 k 2023-10-21T15:59:11.523 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-21T15:59:11.523 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.0.0-6820.gf0804064.el9 ceph 4.8 M 2023-10-21T15:59:11.523 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-21T15:59:11.524 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-21T15:59:11.524 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-21T15:59:11.524 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-21T15:59:11.524 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-21T15:59:11.524 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-21T15:59:11.525 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-21T15:59:11.525 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-21T15:59:11.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-21T15:59:11.525 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-21T15:59:11.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-21T15:59:11.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-21T15:59:11.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.0.0-6820.gf0804064.el9 ceph 45 k 2023-10-21T15:59:11.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.0.0-6820.gf0804064.el9 ceph 127 k 2023-10-21T15:59:11.526 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-21T15:59:11.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-21T15:59:11.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-21T15:59:11.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-21T15:59:11.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-21T15:59:11.527 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-21T15:59:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-10-21T15:59:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-21T15:59:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-21T15:59:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-21T15:59:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-21T15:59:11.529 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-21T15:59:11.529 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-21T15:59:11.529 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-21T15:59:11.529 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-21T15:59:11.529 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-21T15:59:11.530 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-21T15:59:11.530 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-21T15:59:11.530 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-21T15:59:11.530 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-21T15:59:11.530 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-21T15:59:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-21T15:59:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-21T15:59:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-21T15:59:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-21T15:59:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-21T15:59:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-21T15:59:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-21T15:59:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-21T15:59:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-21T15:59:11.532 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-21T15:59:11.533 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-21T15:59:11.534 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-21T15:59:11.534 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-21T15:59:11.534 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-21T15:59:11.534 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-21T15:59:11.534 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-21T15:59:11.535 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-21T15:59:11.535 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-10-21T15:59:11.535 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-21T15:59:11.535 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-21T15:59:11.535 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-21T15:59:11.536 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-21T15:59:11.536 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-21T15:59:11.536 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-21T15:59:11.536 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-21T15:59:11.536 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-21T15:59:11.537 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-21T15:59:11.537 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-21T15:59:11.537 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2023-10-21T15:59:11.537 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-21T15:59:11.537 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:59:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-10-21T15:59:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================ 2023-10-21T15:59:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:Install 133 Packages 2023-10-21T15:59:11.538 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 10 Packages 2023-10-21T15:59:11.538 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T15:59:11.539 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 200 M 2023-10-21T15:59:11.539 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-10-21T15:59:11.762 INFO:teuthology.orchestra.run.smithi124.stdout:(7/143): ceph-common-18.0.0-6820.gf0804064.el9. 11 MB/s | 19 MB 00:01 2023-10-21T15:59:12.103 INFO:teuthology.orchestra.run.smithi124.stdout:(8/143): ceph-mon-18.0.0-6820.gf0804064.el9.x86 5.1 MB/s | 4.4 MB 00:00 2023-10-21T15:59:12.229 INFO:teuthology.orchestra.run.smithi124.stdout:(9/143): ceph-selinux-18.0.0-6820.gf0804064.el9 193 kB/s | 24 kB 00:00 2023-10-21T15:59:12.279 INFO:teuthology.orchestra.run.smithi053.stdout:(1/143): ceph-18.0.0-6820.gf0804064.el9.x86_64. 20 kB/s | 6.5 kB 00:00 2023-10-21T15:59:12.462 INFO:teuthology.orchestra.run.smithi124.stdout:(10/143): ceph-radosgw-18.0.0-6820.gf0804064.el 12 MB/s | 8.7 MB 00:00 2023-10-21T15:59:12.496 INFO:teuthology.orchestra.run.smithi053.stdout:(2/143): ceph-fuse-18.0.0-6820.gf0804064.el9.x8 3.9 MB/s | 855 kB 00:00 2023-10-21T15:59:12.521 INFO:teuthology.orchestra.run.smithi124.stdout:(11/143): cephfs-mirror-18.0.0-6820.gf0804064.e 3.7 MB/s | 218 kB 00:00 2023-10-21T15:59:12.546 INFO:teuthology.orchestra.run.smithi053.stdout:(3/143): ceph-immutable-object-cache-18.0.0-682 2.8 MB/s | 141 kB 00:00 2023-10-21T15:59:12.588 INFO:teuthology.orchestra.run.smithi124.stdout:(12/143): libcephfs-devel-18.0.0-6820.gf0804064 474 kB/s | 31 kB 00:00 2023-10-21T15:59:12.696 INFO:teuthology.orchestra.run.smithi124.stdout:(13/143): libcephfs2-18.0.0-6820.gf0804064.el9. 6.5 MB/s | 716 kB 00:00 2023-10-21T15:59:12.763 INFO:teuthology.orchestra.run.smithi124.stdout:(14/143): libcephsqlite-18.0.0-6820.gf0804064.e 2.4 MB/s | 164 kB 00:00 2023-10-21T15:59:12.804 INFO:teuthology.orchestra.run.smithi053.stdout:(4/143): ceph-mds-18.0.0-6820.gf0804064.el9.x86 8.1 MB/s | 2.1 MB 00:00 2023-10-21T15:59:12.841 INFO:teuthology.orchestra.run.smithi124.stdout:(15/143): librados-devel-18.0.0-6820.gf0804064. 1.6 MB/s | 126 kB 00:00 2023-10-21T15:59:12.922 INFO:teuthology.orchestra.run.smithi124.stdout:(16/143): libradosstriper1-18.0.0-6820.gf080406 5.8 MB/s | 472 kB 00:00 2023-10-21T15:59:12.946 INFO:teuthology.orchestra.run.smithi053.stdout:(5/143): ceph-mgr-18.0.0-6820.gf0804064.el9.x86 10 MB/s | 1.5 MB 00:00 2023-10-21T15:59:13.423 INFO:teuthology.orchestra.run.smithi124.stdout:(17/143): librgw2-18.0.0-6820.gf0804064.el9.x86 9.6 MB/s | 4.8 MB 00:00 2023-10-21T15:59:13.473 INFO:teuthology.orchestra.run.smithi124.stdout:(18/143): python3-ceph-argparse-18.0.0-6820.gf0 923 kB/s | 45 kB 00:00 2023-10-21T15:59:13.497 INFO:teuthology.orchestra.run.smithi053.stdout:(6/143): ceph-mon-18.0.0-6820.gf0804064.el9.x86 8.0 MB/s | 4.4 MB 00:00 2023-10-21T15:59:13.531 INFO:teuthology.orchestra.run.smithi124.stdout:(19/143): python3-ceph-common-18.0.0-6820.gf080 2.2 MB/s | 127 kB 00:00 2023-10-21T15:59:13.607 INFO:teuthology.orchestra.run.smithi124.stdout:(20/143): python3-cephfs-18.0.0-6820.gf0804064. 2.1 MB/s | 162 kB 00:00 2023-10-21T15:59:13.866 INFO:teuthology.orchestra.run.smithi124.stdout:(21/143): ceph-osd-18.0.0-6820.gf0804064.el9.x8 6.9 MB/s | 18 MB 00:02 2023-10-21T15:59:13.916 INFO:teuthology.orchestra.run.smithi124.stdout:(22/143): python3-rados-18.0.0-6820.gf0804064.e 1.0 MB/s | 320 kB 00:00 2023-10-21T15:59:13.975 INFO:teuthology.orchestra.run.smithi124.stdout:(23/143): python3-rbd-18.0.0-6820.gf0804064.el9 2.7 MB/s | 298 kB 00:00 2023-10-21T15:59:14.008 INFO:teuthology.orchestra.run.smithi124.stdout:(24/143): python3-rgw-18.0.0-6820.gf0804064.el9 1.1 MB/s | 99 kB 00:00 2023-10-21T15:59:14.033 INFO:teuthology.orchestra.run.smithi124.stdout:(25/143): rbd-fuse-18.0.0-6820.gf0804064.el9.x8 1.4 MB/s | 85 kB 00:00 2023-10-21T15:59:14.117 INFO:teuthology.orchestra.run.smithi124.stdout:(26/143): rbd-nbd-18.0.0-6820.gf0804064.el9.x86 2.0 MB/s | 168 kB 00:00 2023-10-21T15:59:14.201 INFO:teuthology.orchestra.run.smithi124.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6820.g 280 kB/s | 23 kB 00:00 2023-10-21T15:59:14.318 INFO:teuthology.orchestra.run.smithi124.stdout:(28/143): ceph-mgr-cephadm-18.0.0-6820.gf080406 1.1 MB/s | 132 kB 00:00 2023-10-21T15:59:14.402 INFO:teuthology.orchestra.run.smithi124.stdout:(29/143): rbd-mirror-18.0.0-6820.gf0804064.el9. 7.7 MB/s | 3.0 MB 00:00 2023-10-21T15:59:14.812 INFO:teuthology.orchestra.run.smithi124.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6820.gf0804 3.6 MB/s | 1.8 MB 00:00 2023-10-21T15:59:14.938 INFO:teuthology.orchestra.run.smithi053.stdout:(7/143): ceph-osd-18.0.0-6820.gf0804064.el9.x86 12 MB/s | 18 MB 00:01 2023-10-21T15:59:15.046 INFO:teuthology.orchestra.run.smithi124.stdout:(31/143): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-10-21T15:59:15.079 INFO:teuthology.orchestra.run.smithi124.stdout:(32/143): ceph-mgr-modules-core-18.0.0-6820.gf0 912 kB/s | 243 kB 00:00 2023-10-21T15:59:15.113 INFO:teuthology.orchestra.run.smithi124.stdout:(33/143): ceph-mgr-rook-18.0.0-6820.gf0804064.e 723 kB/s | 48 kB 00:00 2023-10-21T15:59:15.138 INFO:teuthology.orchestra.run.smithi124.stdout:(34/143): ceph-prometheus-alerts-18.0.0-6820.gf 261 kB/s | 15 kB 00:00 2023-10-21T15:59:15.155 INFO:teuthology.orchestra.run.smithi053.stdout:(8/143): ceph-base-18.0.0-6820.gf0804064.el9.x8 1.6 MB/s | 5.2 MB 00:03 2023-10-21T15:59:15.206 INFO:teuthology.orchestra.run.smithi053.stdout:(9/143): ceph-selinux-18.0.0-6820.gf0804064.el9 488 kB/s | 24 kB 00:00 2023-10-21T15:59:15.209 INFO:teuthology.orchestra.run.smithi124.stdout:(35/143): ceph-volume-18.0.0-6820.gf0804064.el9 2.7 MB/s | 262 kB 00:00 2023-10-21T15:59:15.255 INFO:teuthology.orchestra.run.smithi124.stdout:(36/143): cephadm-18.0.0-6820.gf0804064.el9.noa 2.3 MB/s | 270 kB 00:00 2023-10-21T15:59:15.281 INFO:teuthology.orchestra.run.smithi124.stdout:(37/143): cephfs-top-18.0.0-6820.gf0804064.el9. 260 kB/s | 18 kB 00:00 2023-10-21T15:59:15.589 INFO:teuthology.orchestra.run.smithi053.stdout:(10/143): ceph-radosgw-18.0.0-6820.gf0804064.el 13 MB/s | 8.7 MB 00:00 2023-10-21T15:59:15.668 INFO:teuthology.orchestra.run.smithi053.stdout:(11/143): cephfs-mirror-18.0.0-6820.gf0804064.e 2.7 MB/s | 218 kB 00:00 2023-10-21T15:59:15.698 INFO:teuthology.orchestra.run.smithi124.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 254 kB/s | 106 kB 00:00 2023-10-21T15:59:15.715 INFO:teuthology.orchestra.run.smithi053.stdout:(12/143): libcephfs-devel-18.0.0-6820.gf0804064 690 kB/s | 31 kB 00:00 2023-10-21T15:59:15.798 INFO:teuthology.orchestra.run.smithi053.stdout:(13/143): libcephfs2-18.0.0-6820.gf0804064.el9. 8.5 MB/s | 716 kB 00:00 2023-10-21T15:59:15.849 INFO:teuthology.orchestra.run.smithi053.stdout:(14/143): libcephsqlite-18.0.0-6820.gf0804064.e 3.3 MB/s | 164 kB 00:00 2023-10-21T15:59:15.890 INFO:teuthology.orchestra.run.smithi124.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 920 kB 00:00 2023-10-21T15:59:15.899 INFO:teuthology.orchestra.run.smithi053.stdout:(15/143): librados-devel-18.0.0-6820.gf0804064. 2.5 MB/s | 126 kB 00:00 2023-10-21T15:59:15.923 INFO:teuthology.orchestra.run.smithi124.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 307 kB 00:00 2023-10-21T15:59:15.949 INFO:teuthology.orchestra.run.smithi124.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 573 kB/s | 32 kB 00:00 2023-10-21T15:59:15.974 INFO:teuthology.orchestra.run.smithi053.stdout:(16/143): libradosstriper1-18.0.0-6820.gf080406 6.2 MB/s | 472 kB 00:00 2023-10-21T15:59:16.149 INFO:teuthology.orchestra.run.smithi124.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2023-10-21T15:59:16.174 INFO:teuthology.orchestra.run.smithi124.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 76 kB/s | 17 kB 00:00 2023-10-21T15:59:16.241 INFO:teuthology.orchestra.run.smithi124.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 154 kB/s | 10 kB 00:00 2023-10-21T15:59:16.266 INFO:teuthology.orchestra.run.smithi124.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2023-10-21T15:59:16.325 INFO:teuthology.orchestra.run.smithi124.stdout:(46/143): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2023-10-21T15:59:16.341 INFO:teuthology.orchestra.run.smithi053.stdout:(17/143): librgw2-18.0.0-6820.gf0804064.el9.x86 13 MB/s | 4.8 MB 00:00 2023-10-21T15:59:16.375 INFO:teuthology.orchestra.run.smithi124.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2023-10-21T15:59:16.391 INFO:teuthology.orchestra.run.smithi053.stdout:(18/143): python3-ceph-argparse-18.0.0-6820.gf0 922 kB/s | 45 kB 00:00 2023-10-21T15:59:16.400 INFO:teuthology.orchestra.run.smithi124.stdout:(48/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 666 kB/s | 46 kB 00:00 2023-10-21T15:59:16.442 INFO:teuthology.orchestra.run.smithi124.stdout:(49/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-10-21T15:59:16.450 INFO:teuthology.orchestra.run.smithi053.stdout:(19/143): python3-ceph-common-18.0.0-6820.gf080 2.2 MB/s | 127 kB 00:00 2023-10-21T15:59:16.476 INFO:teuthology.orchestra.run.smithi124.stdout:(50/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2023-10-21T15:59:16.492 INFO:teuthology.orchestra.run.smithi053.stdout:(20/143): python3-cephfs-18.0.0-6820.gf0804064. 3.9 MB/s | 162 kB 00:00 2023-10-21T15:59:16.501 INFO:teuthology.orchestra.run.smithi124.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 608 kB/s | 34 kB 00:00 2023-10-21T15:59:16.551 INFO:teuthology.orchestra.run.smithi053.stdout:(21/143): python3-rados-18.0.0-6820.gf0804064.e 5.4 MB/s | 320 kB 00:00 2023-10-21T15:59:16.609 INFO:teuthology.orchestra.run.smithi053.stdout:(22/143): python3-rbd-18.0.0-6820.gf0804064.el9 5.1 MB/s | 298 kB 00:00 2023-10-21T15:59:16.652 INFO:teuthology.orchestra.run.smithi053.stdout:(23/143): python3-rgw-18.0.0-6820.gf0804064.el9 2.3 MB/s | 99 kB 00:00 2023-10-21T15:59:16.702 INFO:teuthology.orchestra.run.smithi053.stdout:(24/143): rbd-fuse-18.0.0-6820.gf0804064.el9.x8 1.7 MB/s | 85 kB 00:00 2023-10-21T15:59:16.735 INFO:teuthology.orchestra.run.smithi124.stdout:(52/143): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2023-10-21T15:59:16.793 INFO:teuthology.orchestra.run.smithi124.stdout:(53/143): postgresql-13.11-1.el9.x86_64.rpm 5.2 MB/s | 1.5 MB 00:00 2023-10-21T15:59:16.844 INFO:teuthology.orchestra.run.smithi124.stdout:(54/143): postgresql-contrib-13.11-1.el9.x86_64 8.2 MB/s | 811 kB 00:00 2023-10-21T15:59:16.869 INFO:teuthology.orchestra.run.smithi124.stdout:(55/143): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2023-10-21T15:59:16.969 INFO:teuthology.orchestra.run.smithi053.stdout:(25/143): rbd-mirror-18.0.0-6820.gf0804064.el9. 11 MB/s | 3.0 MB 00:00 2023-10-21T15:59:17.011 INFO:teuthology.orchestra.run.smithi053.stdout:(26/143): rbd-nbd-18.0.0-6820.gf0804064.el9.x86 4.1 MB/s | 168 kB 00:00 2023-10-21T15:59:17.061 INFO:teuthology.orchestra.run.smithi053.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6820.g 470 kB/s | 23 kB 00:00 2023-10-21T15:59:17.111 INFO:teuthology.orchestra.run.smithi053.stdout:(28/143): ceph-mgr-cephadm-18.0.0-6820.gf080406 2.6 MB/s | 132 kB 00:00 2023-10-21T15:59:17.161 INFO:teuthology.orchestra.run.smithi124.stdout:(56/143): postgresql-server-13.11-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2023-10-21T15:59:17.273 INFO:teuthology.orchestra.run.smithi053.stdout:(29/143): ceph-mgr-dashboard-18.0.0-6820.gf0804 11 MB/s | 1.8 MB 00:00 2023-10-21T15:59:17.286 INFO:teuthology.orchestra.run.smithi124.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2023-10-21T15:59:17.320 INFO:teuthology.orchestra.run.smithi124.stdout:(58/143): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 205 kB 00:00 2023-10-21T15:59:17.353 INFO:teuthology.orchestra.run.smithi124.stdout:(59/143): python3-jinja2-2.11.3-4.el9.noarch.rp 3.5 MB/s | 228 kB 00:00 2023-10-21T15:59:17.379 INFO:teuthology.orchestra.run.smithi124.stdout:(60/143): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 24 kB 00:00 2023-10-21T15:59:17.404 INFO:teuthology.orchestra.run.smithi124.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 368 kB/s | 18 kB 00:00 2023-10-21T15:59:17.429 INFO:teuthology.orchestra.run.smithi124.stdout:(62/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-10-21T15:59:17.455 INFO:teuthology.orchestra.run.smithi124.stdout:(63/143): python3-markupsafe-1.1.1-12.el9.x86_6 653 kB/s | 32 kB 00:00 2023-10-21T15:59:17.605 INFO:teuthology.orchestra.run.smithi124.stdout:(64/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2023-10-21T15:59:17.638 INFO:teuthology.orchestra.run.smithi124.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2023-10-21T15:59:17.655 INFO:teuthology.orchestra.run.smithi124.stdout:(66/143): python3-oauthlib-3.1.1-5.el9.noarch.r 3.6 MB/s | 174 kB 00:00 2023-10-21T15:59:17.681 INFO:teuthology.orchestra.run.smithi124.stdout:(67/143): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-10-21T15:59:17.698 INFO:teuthology.orchestra.run.smithi124.stdout:(68/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2023-10-21T15:59:17.731 INFO:teuthology.orchestra.run.smithi124.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2023-10-21T15:59:17.756 INFO:teuthology.orchestra.run.smithi124.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 738 kB/s | 43 kB 00:00 2023-10-21T15:59:17.781 INFO:teuthology.orchestra.run.smithi053.stdout:(30/143): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-10-21T15:59:17.840 INFO:teuthology.orchestra.run.smithi053.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6820.gf0 4.1 MB/s | 243 kB 00:00 2023-10-21T15:59:17.882 INFO:teuthology.orchestra.run.smithi053.stdout:(32/143): ceph-mgr-rook-18.0.0-6820.gf0804064.e 1.1 MB/s | 48 kB 00:00 2023-10-21T15:59:17.924 INFO:teuthology.orchestra.run.smithi053.stdout:(33/143): ceph-prometheus-alerts-18.0.0-6820.gf 370 kB/s | 15 kB 00:00 2023-10-21T15:59:17.966 INFO:teuthology.orchestra.run.smithi124.stdout:(71/143): python3-toml-0.10.2-6.el9.noarch.rpm 186 kB/s | 39 kB 00:00 2023-10-21T15:59:17.983 INFO:teuthology.orchestra.run.smithi053.stdout:(34/143): ceph-volume-18.0.0-6820.gf0804064.el9 4.5 MB/s | 262 kB 00:00 2023-10-21T15:59:18.041 INFO:teuthology.orchestra.run.smithi053.stdout:(35/143): cephadm-18.0.0-6820.gf0804064.el9.noa 4.6 MB/s | 270 kB 00:00 2023-10-21T15:59:18.092 INFO:teuthology.orchestra.run.smithi053.stdout:(36/143): cephfs-top-18.0.0-6820.gf0804064.el9. 368 kB/s | 18 kB 00:00 2023-10-21T15:59:18.257 INFO:teuthology.orchestra.run.smithi124.stdout:(72/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2023-10-21T15:59:18.290 INFO:teuthology.orchestra.run.smithi124.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 967 kB/s | 300 kB 00:00 2023-10-21T15:59:18.316 INFO:teuthology.orchestra.run.smithi124.stdout:(74/143): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-10-21T15:59:18.341 INFO:teuthology.orchestra.run.smithi124.stdout:(75/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-10-21T15:59:18.366 INFO:teuthology.orchestra.run.smithi124.stdout:(76/143): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-10-21T15:59:18.375 INFO:teuthology.orchestra.run.smithi053.stdout:(37/143): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2023-10-21T15:59:18.425 INFO:teuthology.orchestra.run.smithi053.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2023-10-21T15:59:18.458 INFO:teuthology.orchestra.run.smithi124.stdout:(77/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2023-10-21T15:59:18.484 INFO:teuthology.orchestra.run.smithi053.stdout:(39/143): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2023-10-21T15:59:18.492 INFO:teuthology.orchestra.run.smithi124.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2023-10-21T15:59:18.525 INFO:teuthology.orchestra.run.smithi124.stdout:(79/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-10-21T15:59:18.534 INFO:teuthology.orchestra.run.smithi053.stdout:(40/143): flexiblas-3.0.4-8.el9.x86_64.rpm 640 kB/s | 32 kB 00:00 2023-10-21T15:59:18.542 INFO:teuthology.orchestra.run.smithi124.stdout:(80/143): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2023-10-21T15:59:18.576 INFO:teuthology.orchestra.run.smithi124.stdout:(81/143): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-10-21T15:59:18.601 INFO:teuthology.orchestra.run.smithi124.stdout:(82/143): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2023-10-21T15:59:18.643 INFO:teuthology.orchestra.run.smithi124.stdout:(83/143): python3-urllib3-1.26.5-4.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2023-10-21T15:59:18.710 INFO:teuthology.orchestra.run.smithi053.stdout:(41/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2023-10-21T15:59:18.752 INFO:teuthology.orchestra.run.smithi053.stdout:(42/143): flexiblas-openblas-openmp-3.0.4-8.el9 421 kB/s | 17 kB 00:00 2023-10-21T15:59:18.794 INFO:teuthology.orchestra.run.smithi053.stdout:(43/143): libaio-devel-0.3.111-13.el9.x86_64.rp 246 kB/s | 10 kB 00:00 2023-10-21T15:59:18.835 INFO:teuthology.orchestra.run.smithi124.stdout:(84/143): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2023-10-21T15:59:18.844 INFO:teuthology.orchestra.run.smithi053.stdout:(44/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-10-21T15:59:18.911 INFO:teuthology.orchestra.run.smithi053.stdout:(45/143): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2023-10-21T15:59:18.978 INFO:teuthology.orchestra.run.smithi053.stdout:(46/143): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2023-10-21T15:59:18.994 INFO:teuthology.orchestra.run.smithi124.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 142 kB/s | 22 kB 00:00 2023-10-21T15:59:19.028 INFO:teuthology.orchestra.run.smithi053.stdout:(47/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 46 kB 00:00 2023-10-21T15:59:19.036 INFO:teuthology.orchestra.run.smithi124.stdout:(86/143): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2023-10-21T15:59:19.087 INFO:teuthology.orchestra.run.smithi053.stdout:(48/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-10-21T15:59:19.102 INFO:teuthology.orchestra.run.smithi124.stdout:(87/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 687 kB/s | 45 kB 00:00 2023-10-21T15:59:19.137 INFO:teuthology.orchestra.run.smithi053.stdout:(49/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-10-21T15:59:19.169 INFO:teuthology.orchestra.run.smithi124.stdout:(88/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 387 kB/s | 26 kB 00:00 2023-10-21T15:59:19.188 INFO:teuthology.orchestra.run.smithi053.stdout:(50/143): openblas-0.3.21-2.el9.x86_64.rpm 697 kB/s | 34 kB 00:00 2023-10-21T15:59:19.195 INFO:teuthology.orchestra.run.smithi124.stdout:(89/143): jq-1.6-15.el9.x86_64.rpm 935 kB/s | 187 kB 00:00 2023-10-21T15:59:19.287 INFO:teuthology.orchestra.run.smithi124.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 562 kB/s | 51 kB 00:00 2023-10-21T15:59:19.388 INFO:teuthology.orchestra.run.smithi053.stdout:(51/143): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2023-10-21T15:59:19.389 INFO:teuthology.orchestra.run.smithi124.stdout:(91/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 853 kB/s | 86 kB 00:00 2023-10-21T15:59:19.509 INFO:teuthology.orchestra.run.smithi053.stdout:(52/143): postgresql-13.11-1.el9.x86_64.rpm 13 MB/s | 1.5 MB 00:00 2023-10-21T15:59:19.521 INFO:teuthology.orchestra.run.smithi124.stdout:(92/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2023-10-21T15:59:19.545 INFO:teuthology.orchestra.run.smithi124.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 290 kB/s | 45 kB 00:00 2023-10-21T15:59:19.589 INFO:teuthology.orchestra.run.smithi053.stdout:(53/143): postgresql-contrib-13.11-1.el9.x86_64 10 MB/s | 811 kB 00:00 2023-10-21T15:59:19.639 INFO:teuthology.orchestra.run.smithi053.stdout:(54/143): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2023-10-21T15:59:19.848 INFO:teuthology.orchestra.run.smithi053.stdout:(55/143): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2023-10-21T15:59:19.871 INFO:teuthology.orchestra.run.smithi124.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 317 kB/s | 111 kB 00:00 2023-10-21T15:59:19.996 INFO:teuthology.orchestra.run.smithi124.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 686 kB/s | 309 kB 00:00 2023-10-21T15:59:20.040 INFO:teuthology.orchestra.run.smithi053.stdout:(56/143): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2023-10-21T15:59:20.072 INFO:teuthology.orchestra.run.smithi124.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 328 kB/s | 24 kB 00:00 2023-10-21T15:59:20.090 INFO:teuthology.orchestra.run.smithi053.stdout:(57/143): python3-devel-3.9.17-2.el9.x86_64.rpm 4.1 MB/s | 205 kB 00:00 2023-10-21T15:59:20.155 INFO:teuthology.orchestra.run.smithi124.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 589 kB/s | 49 kB 00:00 2023-10-21T15:59:20.165 INFO:teuthology.orchestra.run.smithi053.stdout:(58/143): python3-jinja2-2.11.3-4.el9.noarch.rp 3.0 MB/s | 228 kB 00:00 2023-10-21T15:59:20.207 INFO:teuthology.orchestra.run.smithi053.stdout:(59/143): python3-jsonpatch-1.21-16.el9.noarch. 600 kB/s | 24 kB 00:00 2023-10-21T15:59:20.239 INFO:teuthology.orchestra.run.smithi124.stdout:(98/143): libunwind-1.6.2-1.el9.x86_64.rpm 816 kB/s | 67 kB 00:00 2023-10-21T15:59:20.249 INFO:teuthology.orchestra.run.smithi053.stdout:(60/143): python3-jsonpointer-2.0-4.el9.noarch. 442 kB/s | 18 kB 00:00 2023-10-21T15:59:20.316 INFO:teuthology.orchestra.run.smithi053.stdout:(61/143): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-10-21T15:59:20.367 INFO:teuthology.orchestra.run.smithi053.stdout:(62/143): python3-markupsafe-1.1.1-12.el9.x86_6 640 kB/s | 32 kB 00:00 2023-10-21T15:59:20.389 INFO:teuthology.orchestra.run.smithi124.stdout:(99/143): libarrow-9.0.0-9.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2023-10-21T15:59:20.556 INFO:teuthology.orchestra.run.smithi124.stdout:(100/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.6 MB/s | 837 kB 00:00 2023-10-21T15:59:20.598 INFO:teuthology.orchestra.run.smithi124.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 2.6 MB/s | 548 kB 00:00 2023-10-21T15:59:20.617 INFO:teuthology.orchestra.run.smithi053.stdout:(63/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2023-10-21T15:59:20.623 INFO:teuthology.orchestra.run.smithi124.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 656 kB/s | 43 kB 00:00 2023-10-21T15:59:20.665 INFO:teuthology.orchestra.run.smithi124.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 487 kB/s | 32 kB 00:00 2023-10-21T15:59:20.667 INFO:teuthology.orchestra.run.smithi053.stdout:(64/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.4 MB/s | 217 kB 00:00 2023-10-21T15:59:20.690 INFO:teuthology.orchestra.run.smithi124.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 226 kB/s | 15 kB 00:00 2023-10-21T15:59:20.718 INFO:teuthology.orchestra.run.smithi053.stdout:(65/143): python3-oauthlib-3.1.1-5.el9.noarch.r 3.5 MB/s | 174 kB 00:00 2023-10-21T15:59:20.733 INFO:teuthology.orchestra.run.smithi124.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2023-10-21T15:59:20.797 INFO:teuthology.orchestra.run.smithi053.stdout:(66/143): python3-prettytable-0.7.2-27.el9.noar 524 kB/s | 41 kB 00:00 2023-10-21T15:59:20.851 INFO:teuthology.orchestra.run.smithi053.stdout:(67/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 132 kB 00:00 2023-10-21T15:59:20.902 INFO:teuthology.orchestra.run.smithi053.stdout:(68/143): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 211 kB 00:00 2023-10-21T15:59:20.933 INFO:teuthology.orchestra.run.smithi124.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2023-10-21T15:59:20.944 INFO:teuthology.orchestra.run.smithi053.stdout:(69/143): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2023-10-21T15:59:20.967 INFO:teuthology.orchestra.run.smithi124.stdout:(107/143): python3-google-auth-2.23.3-1.el9.noa 845 kB/s | 197 kB 00:00 2023-10-21T15:59:21.000 INFO:teuthology.orchestra.run.smithi124.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 162 kB/s | 11 kB 00:00 2023-10-21T15:59:21.042 INFO:teuthology.orchestra.run.smithi124.stdout:(109/143): python3-jaraco-classes-3.2.1-5.el9.n 237 kB/s | 18 kB 00:00 2023-10-21T15:59:21.076 INFO:teuthology.orchestra.run.smithi124.stdout:(110/143): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2023-10-21T15:59:21.126 INFO:teuthology.orchestra.run.smithi124.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2023-10-21T15:59:21.151 INFO:teuthology.orchestra.run.smithi124.stdout:(112/143): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2023-10-21T15:59:21.193 INFO:teuthology.orchestra.run.smithi124.stdout:(113/143): python3-jwt+crypto-2.4.0-1.el9.noarc 137 kB/s | 9.0 kB 00:00 2023-10-21T15:59:21.235 INFO:teuthology.orchestra.run.smithi124.stdout:(114/143): python3-jwt-2.4.0-1.el9.noarch.rpm 493 kB/s | 41 kB 00:00 2023-10-21T15:59:21.293 INFO:teuthology.orchestra.run.smithi124.stdout:(115/143): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2023-10-21T15:59:21.319 INFO:teuthology.orchestra.run.smithi124.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 560 kB/s | 46 kB 00:00 2023-10-21T15:59:21.353 INFO:teuthology.orchestra.run.smithi124.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2023-10-21T15:59:21.378 INFO:teuthology.orchestra.run.smithi124.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 997 kB/s | 58 kB 00:00 2023-10-21T15:59:21.420 INFO:teuthology.orchestra.run.smithi124.stdout:(119/143): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2023-10-21T15:59:21.445 INFO:teuthology.orchestra.run.smithi124.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 248 kB/s | 16 kB 00:00 2023-10-21T15:59:21.470 INFO:teuthology.orchestra.run.smithi124.stdout:(121/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2023-10-21T15:59:21.477 INFO:teuthology.orchestra.run.smithi053.stdout:(70/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2023-10-21T15:59:21.495 INFO:teuthology.orchestra.run.smithi124.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2023-10-21T15:59:21.521 INFO:teuthology.orchestra.run.smithi053.stdout:(71/143): python3-toml-0.10.2-6.el9.noarch.rpm 907 kB/s | 39 kB 00:00 2023-10-21T15:59:21.529 INFO:teuthology.orchestra.run.smithi124.stdout:(123/143): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2023-10-21T15:59:21.546 INFO:teuthology.orchestra.run.smithi124.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2023-10-21T15:59:21.571 INFO:teuthology.orchestra.run.smithi053.stdout:(72/143): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2023-10-21T15:59:21.588 INFO:teuthology.orchestra.run.smithi124.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 617 kB/s | 36 kB 00:00 2023-10-21T15:59:21.605 INFO:teuthology.orchestra.run.smithi053.stdout:(73/143): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2023-10-21T15:59:21.613 INFO:teuthology.orchestra.run.smithi124.stdout:(126/143): python3-typing-extensions-4.4.0-2.el 767 kB/s | 51 kB 00:00 2023-10-21T15:59:21.647 INFO:teuthology.orchestra.run.smithi124.stdout:(127/143): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2023-10-21T15:59:21.656 INFO:teuthology.orchestra.run.smithi053.stdout:(74/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-10-21T15:59:21.689 INFO:teuthology.orchestra.run.smithi053.stdout:(75/143): fuse-2.9.9-15.el9.x86_64.rpm 2.4 MB/s | 79 kB 00:00 2023-10-21T15:59:21.705 INFO:teuthology.orchestra.run.smithi124.stdout:(128/143): python3-websocket-client-1.2.3-2.el9 983 kB/s | 90 kB 00:00 2023-10-21T15:59:21.747 INFO:teuthology.orchestra.run.smithi124.stdout:(129/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-10-21T15:59:21.748 INFO:teuthology.orchestra.run.smithi053.stdout:(76/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 14 MB/s | 803 kB 00:00 2023-10-21T15:59:21.773 INFO:teuthology.orchestra.run.smithi124.stdout:(130/143): python3-zc-lockfile-2.0-10.el9.noarc 302 kB/s | 20 kB 00:00 2023-10-21T15:59:21.790 INFO:teuthology.orchestra.run.smithi053.stdout:(77/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 4.7 MB/s | 195 kB 00:00 2023-10-21T15:59:21.806 INFO:teuthology.orchestra.run.smithi124.stdout:(131/143): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2023-10-21T15:59:21.832 INFO:teuthology.orchestra.run.smithi053.stdout:(78/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.8 MB/s | 241 kB 00:00 2023-10-21T15:59:21.874 INFO:teuthology.orchestra.run.smithi053.stdout:(79/143): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 103 kB 00:00 2023-10-21T15:59:21.916 INFO:teuthology.orchestra.run.smithi053.stdout:(80/143): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2023-10-21T15:59:21.932 INFO:teuthology.orchestra.run.smithi124.stdout:(132/143): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-10-21T15:59:21.958 INFO:teuthology.orchestra.run.smithi053.stdout:(81/143): python3-requests-2.25.1-7.el9.noarch. 2.7 MB/s | 113 kB 00:00 2023-10-21T15:59:22.000 INFO:teuthology.orchestra.run.smithi053.stdout:(82/143): python3-urllib3-1.26.5-4.el9.noarch.r 4.5 MB/s | 187 kB 00:00 2023-10-21T15:59:22.200 INFO:teuthology.orchestra.run.smithi053.stdout:(83/143): python3-cryptography-36.0.1-4.el9.x86 6.2 MB/s | 1.2 MB 00:00 2023-10-21T15:59:22.315 INFO:teuthology.orchestra.run.smithi124.stdout:(133/143): librbd1-18.0.0-6820.gf0804064.el9.x8 7.9 MB/s | 3.0 MB 00:00 2023-10-21T15:59:22.349 INFO:teuthology.orchestra.run.smithi124.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-10-21T15:59:22.466 INFO:teuthology.orchestra.run.smithi124.stdout:(135/143): librados2-18.0.0-6820.gf0804064.el9. 5.0 MB/s | 3.3 MB 00:00 2023-10-21T15:59:22.524 INFO:teuthology.orchestra.run.smithi124.stdout:(136/143): NetworkManager-1.45.5-1.el9.x86_64.r 13 MB/s | 2.2 MB 00:00 2023-10-21T15:59:22.611 INFO:teuthology.orchestra.run.smithi124.stdout:(137/143): NetworkManager-libnm-1.45.5-1.el9.x8 12 MB/s | 1.8 MB 00:00 2023-10-21T15:59:22.641 INFO:teuthology.orchestra.run.smithi124.stdout:(138/143): NetworkManager-team-1.45.5-1.el9.x86 405 kB/s | 42 kB 00:00 2023-10-21T15:59:22.675 INFO:teuthology.orchestra.run.smithi124.stdout:(139/143): NetworkManager-tui-1.45.5-1.el9.x86_ 3.8 MB/s | 245 kB 00:00 2023-10-21T15:59:22.700 INFO:teuthology.orchestra.run.smithi124.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-10-21T15:59:22.705 INFO:teuthology.orchestra.run.smithi053.stdout:(84/143): ceph-test-18.0.0-6820.gf0804064.el9.x 5.5 MB/s | 41 MB 00:07 2023-10-21T15:59:22.726 INFO:teuthology.orchestra.run.smithi124.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 450 kB/s | 22 kB 00:00 2023-10-21T15:59:22.759 INFO:teuthology.orchestra.run.smithi124.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-10-21T15:59:22.784 INFO:teuthology.orchestra.run.smithi053.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 290 kB/s | 22 kB 00:00 2023-10-21T15:59:22.917 INFO:teuthology.orchestra.run.smithi053.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-10-21T15:59:22.959 INFO:teuthology.orchestra.run.smithi053.stdout:(87/143): automake-1.16.2-8.el9.noarch.rpm 909 kB/s | 689 kB 00:00 2023-10-21T15:59:22.984 INFO:teuthology.orchestra.run.smithi053.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 685 kB/s | 45 kB 00:00 2023-10-21T15:59:23.010 INFO:teuthology.orchestra.run.smithi053.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 519 kB/s | 26 kB 00:00 2023-10-21T15:59:23.060 INFO:teuthology.orchestra.run.smithi053.stdout:(90/143): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2023-10-21T15:59:23.118 INFO:teuthology.orchestra.run.smithi053.stdout:(91/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.5 MB/s | 86 kB 00:00 2023-10-21T15:59:23.177 INFO:teuthology.orchestra.run.smithi053.stdout:(92/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 786 kB/s | 45 kB 00:00 2023-10-21T15:59:23.844 INFO:teuthology.orchestra.run.smithi053.stdout:(93/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 4.8 MB/s | 4.1 MB 00:00 2023-10-21T15:59:23.870 INFO:teuthology.orchestra.run.smithi053.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 160 kB/s | 111 kB 00:00 2023-10-21T15:59:24.843 INFO:teuthology.orchestra.run.smithi124.stdout:(143/143): ceph-test-18.0.0-6820.gf0804064.el9. 3.2 MB/s | 41 MB 00:12 2023-10-21T15:59:24.852 INFO:teuthology.orchestra.run.smithi124.stdout:-------------------------------------------------------------------------------- 2023-10-21T15:59:24.853 INFO:teuthology.orchestra.run.smithi124.stdout:Total 13 MB/s | 200 MB 00:14 2023-10-21T15:59:24.986 INFO:teuthology.orchestra.run.smithi053.stdout:(95/143): ceph-common-18.0.0-6820.gf0804064.el9 1.5 MB/s | 19 MB 00:13 2023-10-21T15:59:25.020 INFO:teuthology.orchestra.run.smithi053.stdout:(96/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 263 kB/s | 309 kB 00:01 2023-10-21T15:59:25.228 INFO:teuthology.orchestra.run.smithi053.stdout:(97/143): liboath-2.6.7-2.el9.x86_64.rpm 233 kB/s | 49 kB 00:00 2023-10-21T15:59:25.286 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction check 2023-10-21T15:59:25.287 INFO:teuthology.orchestra.run.smithi053.stdout:(98/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 81 kB/s | 24 kB 00:00 2023-10-21T15:59:25.489 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction check succeeded. 2023-10-21T15:59:25.489 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction test 2023-10-21T15:59:25.529 INFO:teuthology.orchestra.run.smithi053.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 225 kB/s | 67 kB 00:00 2023-10-21T15:59:26.996 INFO:teuthology.orchestra.run.smithi053.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 374 kB/s | 548 kB 00:01 2023-10-21T15:59:27.213 INFO:teuthology.orchestra.run.smithi053.stdout:(101/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 200 kB/s | 43 kB 00:00 2023-10-21T15:59:27.330 INFO:teuthology.orchestra.run.smithi053.stdout:(102/143): python3-cachetools-4.2.4-1.el9.noarc 278 kB/s | 32 kB 00:00 2023-10-21T15:59:27.447 INFO:teuthology.orchestra.run.smithi053.stdout:(103/143): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2023-10-21T15:59:27.624 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction test succeeded. 2023-10-21T15:59:27.624 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction 2023-10-21T15:59:27.930 INFO:teuthology.orchestra.run.smithi053.stdout:(104/143): python3-cheroot-8.6.0-4.el9.noarch.r 356 kB/s | 172 kB 00:00 2023-10-21T15:59:27.972 INFO:teuthology.orchestra.run.smithi053.stdout:(105/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 312 kB/s | 837 kB 00:02 2023-10-21T15:59:28.547 INFO:teuthology.orchestra.run.smithi053.stdout:(106/143): python3-google-auth-2.23.3-1.el9.noa 343 kB/s | 197 kB 00:00 2023-10-21T15:59:28.823 INFO:teuthology.orchestra.run.smithi053.stdout:(107/143): python3-jaraco-8.2.1-3.el9.noarch.rp 39 kB/s | 11 kB 00:00 2023-10-21T15:59:28.856 INFO:teuthology.orchestra.run.smithi053.stdout:(108/143): python3-cherrypy-18.6.1-2.el9.noarch 387 kB/s | 358 kB 00:00 2023-10-21T15:59:28.957 INFO:teuthology.orchestra.run.smithi053.stdout:(109/143): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2023-10-21T15:59:28.999 INFO:teuthology.orchestra.run.smithi053.stdout:(110/143): python3-jaraco-collections-3.0.0-8.e 163 kB/s | 23 kB 00:00 2023-10-21T15:59:29.091 INFO:teuthology.orchestra.run.smithi053.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2023-10-21T15:59:29.157 INFO:teuthology.orchestra.run.smithi053.stdout:(112/143): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2023-10-21T15:59:29.224 INFO:teuthology.orchestra.run.smithi053.stdout:(113/143): python3-jwt+crypto-2.4.0-1.el9.noarc 68 kB/s | 9.0 kB 00:00 2023-10-21T15:59:29.291 INFO:teuthology.orchestra.run.smithi053.stdout:(114/143): python3-jwt-2.4.0-1.el9.noarch.rpm 308 kB/s | 41 kB 00:00 2023-10-21T15:59:29.525 INFO:teuthology.orchestra.run.smithi053.stdout:(115/143): python3-logutils-0.3.5-21.el9.noarch 199 kB/s | 46 kB 00:00 2023-10-21T15:59:29.758 INFO:teuthology.orchestra.run.smithi053.stdout:(116/143): python3-more-itertools-8.12.0-2.el9. 340 kB/s | 79 kB 00:00 2023-10-21T15:59:29.984 INFO:teuthology.orchestra.run.smithi053.stdout:(117/143): python3-natsort-7.1.1-5.el9.noarch.r 256 kB/s | 58 kB 00:00 2023-10-21T15:59:30.642 INFO:teuthology.orchestra.run.smithi053.stdout:(118/143): python3-pecan-1.4.2-3.el9.noarch.rpm 414 kB/s | 272 kB 00:00 2023-10-21T15:59:30.793 INFO:teuthology.orchestra.run.smithi053.stdout:(119/143): python3-portend-3.1.0-2.el9.noarch.r 110 kB/s | 16 kB 00:00 2023-10-21T15:59:31.031 INFO:teuthology.orchestra.run.smithi053.stdout:(120/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 379 kB/s | 90 kB 00:00 2023-10-21T15:59:31.168 INFO:teuthology.orchestra.run.smithi053.stdout:(121/143): python3-repoze-lru-0.7-16.el9.noarch 226 kB/s | 31 kB 00:00 2023-10-21T15:59:31.560 INFO:teuthology.orchestra.run.smithi053.stdout:(122/143): python3-routes-2.5.1-5.el9.noarch.rp 482 kB/s | 188 kB 00:00 2023-10-21T15:59:31.668 INFO:teuthology.orchestra.run.smithi053.stdout:(123/143): python3-kubernetes-26.1.0-2.el9.noar 428 kB/s | 1.0 MB 00:02 2023-10-21T15:59:31.836 INFO:teuthology.orchestra.run.smithi053.stdout:(124/143): python3-rsa-4.9-2.el9.noarch.rpm 215 kB/s | 59 kB 00:00 2023-10-21T15:59:31.886 INFO:teuthology.orchestra.run.smithi053.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 165 kB/s | 36 kB 00:00 2023-10-21T15:59:32.045 INFO:teuthology.orchestra.run.smithi053.stdout:(126/143): python3-typing-extensions-4.4.0-2.el 245 kB/s | 51 kB 00:00 2023-10-21T15:59:32.253 INFO:teuthology.orchestra.run.smithi053.stdout:(127/143): python3-websocket-client-1.2.3-2.el9 432 kB/s | 90 kB 00:00 2023-10-21T15:59:32.379 INFO:teuthology.orchestra.run.smithi053.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 467 kB/s | 230 kB 00:00 2023-10-21T15:59:32.496 INFO:teuthology.orchestra.run.smithi053.stdout:(129/143): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2023-10-21T15:59:32.879 INFO:teuthology.orchestra.run.smithi053.stdout:(130/143): re2-20211101-3.el9.x86_64.rpm 501 kB/s | 192 kB 00:00 2023-10-21T15:59:33.096 INFO:teuthology.orchestra.run.smithi053.stdout:(131/143): python3-werkzeug-2.0.3-3.el9.1.noarc 507 kB/s | 427 kB 00:00 2023-10-21T15:59:33.205 INFO:teuthology.orchestra.run.smithi053.stdout:(132/143): libarrow-9.0.0-9.el9.x86_64.rpm 485 kB/s | 4.4 MB 00:09 2023-10-21T15:59:33.463 INFO:teuthology.orchestra.run.smithi053.stdout:(133/143): librados2-18.0.0-6820.gf0804064.el9. 9.0 MB/s | 3.3 MB 00:00 2023-10-21T15:59:33.505 INFO:teuthology.orchestra.run.smithi053.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-21T15:59:33.631 INFO:teuthology.orchestra.run.smithi053.stdout:(135/143): NetworkManager-1.45.5-1.el9.x86_64.r 18 MB/s | 2.2 MB 00:00 2023-10-21T15:59:33.703 INFO:teuthology.orchestra.run.smithi124.stdout: Preparing : 1/1 2023-10-21T15:59:33.731 INFO:teuthology.orchestra.run.smithi053.stdout:(136/143): librbd1-18.0.0-6820.gf0804064.el9.x8 5.8 MB/s | 3.0 MB 00:00 2023-10-21T15:59:33.783 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-21T15:59:33.813 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-21T15:59:33.832 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-21T15:59:33.849 INFO:teuthology.orchestra.run.smithi053.stdout:(137/143): NetworkManager-libnm-1.45.5-1.el9.x8 8.2 MB/s | 1.8 MB 00:00 2023-10-21T15:59:33.873 INFO:teuthology.orchestra.run.smithi053.stdout:(138/143): NetworkManager-team-1.45.5-1.el9.x86 302 kB/s | 42 kB 00:00 2023-10-21T15:59:33.907 INFO:teuthology.orchestra.run.smithi053.stdout:(139/143): NetworkManager-tui-1.45.5-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-10-21T15:59:33.932 INFO:teuthology.orchestra.run.smithi053.stdout:(140/143): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-10-21T15:59:33.957 INFO:teuthology.orchestra.run.smithi053.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2023-10-21T15:59:33.999 INFO:teuthology.orchestra.run.smithi053.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2023-10-21T15:59:34.132 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-21T15:59:34.573 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-21T15:59:34.638 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librados2-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T15:59:34.796 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T15:59:34.831 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T15:59:34.954 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T15:59:35.014 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 8/153 2023-10-21T15:59:35.068 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-21T15:59:35.326 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-21T15:59:35.385 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-21T15:59:35.410 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T15:59:35.574 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T15:59:35.611 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 13/153 2023-10-21T15:59:35.703 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 13/153 2023-10-21T15:59:35.757 INFO:teuthology.orchestra.run.smithi053.stdout:(143/143): thrift-0.15.0-2.el9.x86_64.rpm 563 kB/s | 1.6 MB 00:02 2023-10-21T15:59:35.767 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-10-21T15:59:35.768 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.2 MB/s | 200 MB 00:24 2023-10-21T15:59:36.147 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-21T15:59:36.212 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-10-21T15:59:36.266 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-21T15:59:36.355 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-21T15:59:36.431 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-10-21T15:59:36.431 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-10-21T15:59:36.572 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:36.635 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:37.117 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:37.166 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-21T15:59:37.228 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-21T15:59:37.351 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-21T15:59:37.426 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/153 2023-10-21T15:59:37.462 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-21T15:59:37.515 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-21T15:59:37.610 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-21T15:59:37.654 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-21T15:59:37.688 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-21T15:59:37.793 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-21T15:59:37.834 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-common-2:18.0.0-6820.gf0804064.el9. 27/153 2023-10-21T15:59:37.884 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-argparse-2:18.0.0-6820.gf0804064.el 28/153 2023-10-21T15:59:37.999 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_6 29/153 2023-10-21T15:59:38.181 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-21T15:59:38.281 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-21T15:59:38.633 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-10-21T15:59:38.634 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-10-21T15:59:38.983 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-21T15:59:39.076 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-21T15:59:39.138 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-21T15:59:39.206 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librados-devel-2:18.0.0-6820.gf0804064.el9.x86_6 35/153 2023-10-21T15:59:39.406 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-21T15:59:39.587 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-21T15:59:39.636 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-21T15:59:39.733 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-21T15:59:39.838 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-21T15:59:39.878 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-21T15:59:39.917 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-21T15:59:39.985 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-21T15:59:40.041 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-21T15:59:40.101 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-21T15:59:40.314 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-21T15:59:40.368 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-21T15:59:40.423 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-21T15:59:40.561 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-21T15:59:40.618 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-21T15:59:40.677 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-21T15:59:40.717 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-21T15:59:40.757 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-21T15:59:40.799 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-21T15:59:40.852 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-21T15:59:41.007 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-21T15:59:41.086 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-21T15:59:41.157 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-21T15:59:41.246 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-21T15:59:41.529 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-21T15:59:41.590 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-21T15:59:41.767 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-21T15:59:41.817 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-21T15:59:41.850 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-21T15:59:41.888 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-21T15:59:41.960 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-21T15:59:42.020 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-21T15:59:42.082 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-21T15:59:42.174 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-21T15:59:42.226 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-21T15:59:42.268 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-21T15:59:42.403 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-21T15:59:42.447 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-21T15:59:42.482 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-21T15:59:42.537 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-21T15:59:42.578 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-21T15:59:42.731 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-21T15:59:42.776 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-21T15:59:42.919 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-21T15:59:42.990 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-21T15:59:43.920 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-21T15:59:43.981 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-21T15:59:44.040 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-21T15:59:44.174 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-21T15:59:44.308 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-21T15:59:44.558 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-21T15:59:44.595 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-21T15:59:44.692 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-21T15:59:44.919 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-10-21T15:59:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-21T15:59:45.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-21T15:59:45.126 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-21T15:59:45.227 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-21T15:59:45.512 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-21T15:59:45.651 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-21T15:59:45.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-21T15:59:45.703 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T15:59:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T15:59:45.929 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T15:59:46.055 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T15:59:46.118 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 8/153 2023-10-21T15:59:46.162 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-21T15:59:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-21T15:59:46.476 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-21T15:59:46.500 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T15:59:46.641 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T15:59:46.676 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 13/153 2023-10-21T15:59:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 13/153 2023-10-21T15:59:47.220 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-21T15:59:47.244 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-21T15:59:47.330 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-21T15:59:47.348 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-21T15:59:47.377 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-21T15:59:47.415 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-21T15:59:47.471 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 15/153 2023-10-21T15:59:47.489 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-21T15:59:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:47.900 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-21T15:59:47.970 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 97/153 2023-10-21T15:59:48.096 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 97/153 2023-10-21T15:59:48.127 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 98/153 2023-10-21T15:59:48.185 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-21T15:59:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.45.5-1.el9.x86_64 16/153 2023-10-21T15:59:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-21T15:59:48.386 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-21T15:59:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-21T15:59:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/153 2023-10-21T15:59:48.644 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-21T15:59:48.680 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-21T15:59:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-21T15:59:48.723 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 101/153 2023-10-21T15:59:48.814 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-21T15:59:48.815 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 101/153 2023-10-21T15:59:48.863 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-21T15:59:48.867 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 102/153 2023-10-21T15:59:48.904 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-21T15:59:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-21T15:59:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.0.0-6820.gf0804064.el9. 27/153 2023-10-21T15:59:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.0.0-6820.gf0804064.el 28/153 2023-10-21T15:59:49.270 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_6 29/153 2023-10-21T15:59:49.459 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-21T15:59:49.522 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-21T15:59:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-21T15:59:50.389 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-21T15:59:50.504 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-21T15:59:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.0.0-6820.gf0804064.el9.x86_6 35/153 2023-10-21T15:59:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-21T15:59:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-21T15:59:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-21T15:59:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-21T15:59:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-21T15:59:51.019 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-21T15:59:51.059 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-21T15:59:51.130 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-21T15:59:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-21T15:59:51.261 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-21T15:59:51.322 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T15:59:51.362 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T15:59:51.462 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-21T15:59:51.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-21T15:59:51.595 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-21T15:59:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-21T15:59:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/153 2023-10-21T15:59:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-21T15:59:51.899 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-21T15:59:51.939 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-21T15:59:51.981 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-21T15:59:51.994 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T15:59:52.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-21T15:59:52.068 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 104/153 2023-10-21T15:59:52.203 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-21T15:59:52.249 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 104/153 2023-10-21T15:59:52.250 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-21T15:59:52.250 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-21T15:59:52.250 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T15:59:52.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-21T15:59:52.300 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 105/153 2023-10-21T15:59:52.353 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-21T15:59:52.437 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-21T15:59:52.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-21T15:59:52.774 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-21T15:59:52.975 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-21T15:59:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-21T15:59:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-21T15:59:53.109 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-21T15:59:53.154 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-21T15:59:53.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-21T15:59:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-21T15:59:53.341 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-21T15:59:53.403 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-21T15:59:53.470 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-21T15:59:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-21T15:59:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-21T15:59:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-21T15:59:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-21T15:59:53.822 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-21T15:59:53.978 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-21T15:59:54.018 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-21T15:59:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-21T15:59:54.269 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-21T15:59:55.306 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-21T15:59:55.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-21T15:59:55.416 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-21T15:59:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-21T15:59:55.599 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-21T15:59:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-21T15:59:55.879 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-21T15:59:55.976 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-21T15:59:56.852 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-21T15:59:56.989 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-21T15:59:58.662 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-21T15:59:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-21T15:59:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-21T15:59:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-21T15:59:58.857 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-21T15:59:59.251 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-21T15:59:59.289 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 97/153 2023-10-21T15:59:59.437 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 97/153 2023-10-21T15:59:59.479 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 98/153 2023-10-21T15:59:59.539 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-21T16:00:00.065 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-21T16:00:00.100 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 101/153 2023-10-21T16:00:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 101/153 2023-10-21T16:00:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 102/153 2023-10-21T16:00:02.816 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T16:00:02.848 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T16:00:03.529 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 103/153 2023-10-21T16:00:03.567 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 104/153 2023-10-21T16:00:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 104/153 2023-10-21T16:00:03.748 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-21T16:00:03.749 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-21T16:00:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:03.819 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 105/153 2023-10-21T16:00:05.205 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 105/153 2023-10-21T16:00:05.206 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /sys 2023-10-21T16:00:05.206 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /proc 2023-10-21T16:00:05.206 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /mnt 2023-10-21T16:00:05.206 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /var/tmp 2023-10-21T16:00:05.207 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /home 2023-10-21T16:00:05.207 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /root 2023-10-21T16:00:05.207 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /tmp 2023-10-21T16:00:05.207 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:05.433 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 106/153 2023-10-21T16:00:05.478 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 106/153 2023-10-21T16:00:05.478 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:05.478 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-21T16:00:05.479 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:07.129 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 107/153 2023-10-21T16:00:07.223 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 107/153 2023-10-21T16:00:07.224 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:07.224 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-21T16:00:07.224 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-21T16:00:07.224 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-21T16:00:07.225 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:07.454 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 108/153 2023-10-21T16:00:07.531 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 108/153 2023-10-21T16:00:07.531 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:07.532 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-21T16:00:07.532 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-21T16:00:07.532 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-21T16:00:07.532 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:08.021 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 109/153 2023-10-21T16:00:08.092 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 109/153 2023-10-21T16:00:08.093 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:08.093 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-21T16:00:08.093 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-21T16:00:08.093 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-21T16:00:08.093 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:08.803 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephadm-2:18.0.0-6820.gf0804064.el9.noarch 110/153 2023-10-21T16:00:08.849 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephadm-2:18.0.0-6820.gf0804064.el9.noarch 110/153 2023-10-21T16:00:08.895 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.e 111/153 2023-10-21T16:00:09.008 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6820.gf0804064. 112/153 2023-10-21T16:00:09.035 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 113/153 2023-10-21T16:00:09.359 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 113/153 2023-10-21T16:00:09.402 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 114/153 2023-10-21T16:00:10.529 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 114/153 2023-10-21T16:00:10.561 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 115/153 2023-10-21T16:00:10.702 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 115/153 2023-10-21T16:00:10.869 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el 116/153 2023-10-21T16:00:10.896 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 117/153 2023-10-21T16:00:10.970 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 117/153 2023-10-21T16:00:10.971 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:10.971 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-21T16:00:10.971 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-21T16:00:10.972 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-21T16:00:10.972 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:11.022 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 118/153 2023-10-21T16:00:11.062 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 118/153 2023-10-21T16:00:11.964 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-2:18.0.0-6820.gf0804064.el9.x86_64 119/153 2023-10-21T16:00:12.006 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 120/153 2023-10-21T16:00:12.079 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 120/153 2023-10-21T16:00:12.079 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:12.079 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-21T16:00:12.080 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-21T16:00:12.080 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-21T16:00:12.080 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:12.140 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6820.gf0804 121/153 2023-10-21T16:00:12.215 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6820.gf0804 121/153 2023-10-21T16:00:12.215 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:12.215 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-21T16:00:12.216 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:12.291 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 122/153 2023-10-21T16:00:12.366 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 122/153 2023-10-21T16:00:12.366 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:12.366 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-21T16:00:12.367 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-21T16:00:12.367 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-21T16:00:12.367 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:12.741 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 123/153 2023-10-21T16:00:12.814 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 123/153 2023-10-21T16:00:12.814 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:12.815 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-21T16:00:12.815 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-21T16:00:12.815 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-21T16:00:12.815 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:16.970 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 105/153 2023-10-21T16:00:16.970 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2023-10-21T16:00:16.970 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2023-10-21T16:00:16.970 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2023-10-21T16:00:16.971 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2023-10-21T16:00:16.971 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2023-10-21T16:00:16.971 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2023-10-21T16:00:16.971 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2023-10-21T16:00:16.971 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:16.981 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 124/153 2023-10-21T16:00:17.039 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 125/153 2023-10-21T16:00:17.082 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 126/153 2023-10-21T16:00:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 106/153 2023-10-21T16:00:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 106/153 2023-10-21T16:00:17.238 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:17.238 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-21T16:00:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:17.355 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-21T16:00:17.481 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-21T16:00:17.554 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 129/153 2023-10-21T16:00:17.598 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-21T16:00:17.636 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-21T16:00:17.676 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_ 132/153 2023-10-21T16:00:17.730 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-team-1:1.45.5-1.el9.x86_64 133/153 2023-10-21T16:00:17.838 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-tui-1:1.45.5-1.el9.x86_64 134/153 2023-10-21T16:00:17.931 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-21T16:00:17.978 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-21T16:00:18.950 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 107/153 2023-10-21T16:00:19.000 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:19.057 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:19.088 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 107/153 2023-10-21T16:00:19.088 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:19.088 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-21T16:00:19.089 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-21T16:00:19.089 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-21T16:00:19.089 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:19.120 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 138/153 2023-10-21T16:00:19.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 108/153 2023-10-21T16:00:19.407 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 108/153 2023-10-21T16:00:19.407 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:19.407 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-21T16:00:19.407 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-21T16:00:19.408 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-21T16:00:19.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:19.926 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 109/153 2023-10-21T16:00:19.998 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 109/153 2023-10-21T16:00:19.998 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:19.999 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-21T16:00:19.999 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-21T16:00:20.000 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-21T16:00:20.000 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:20.548 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-21T16:00:20.656 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-21T16:00:20.690 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.0.0-6820.gf0804064.el9.noarch 110/153 2023-10-21T16:00:20.709 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-21T16:00:20.719 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.0.0-6820.gf0804064.el9.noarch 110/153 2023-10-21T16:00:20.774 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.e 111/153 2023-10-21T16:00:20.832 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-21T16:00:20.866 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-21T16:00:20.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6820.gf0804064. 112/153 2023-10-21T16:00:20.903 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-21T16:00:20.903 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-21T16:00:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 113/153 2023-10-21T16:00:21.050 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-21T16:00:21.091 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-21T16:00:21.136 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-21T16:00:21.177 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-21T16:00:21.217 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:21.217 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 113/153 2023-10-21T16:00:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 114/153 2023-10-21T16:00:21.569 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:21.569 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-21T16:00:21.700 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-21T16:00:21.730 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-21T16:00:21.730 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-21T16:00:21.852 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-21T16:00:22.002 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:22.267 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:22.267 INFO:teuthology.orchestra.run.smithi124.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-21T16:00:22.267 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:22.528 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 114/153 2023-10-21T16:00:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 115/153 2023-10-21T16:00:22.716 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 115/153 2023-10-21T16:00:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el 116/153 2023-10-21T16:00:22.925 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 117/153 2023-10-21T16:00:23.002 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 117/153 2023-10-21T16:00:23.002 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:23.002 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-21T16:00:23.002 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-21T16:00:23.003 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-21T16:00:23.003 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:23.059 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 118/153 2023-10-21T16:00:23.100 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 118/153 2023-10-21T16:00:24.029 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.0.0-6820.gf0804064.el9.x86_64 119/153 2023-10-21T16:00:24.068 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 120/153 2023-10-21T16:00:24.152 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 120/153 2023-10-21T16:00:24.153 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:24.153 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-21T16:00:24.153 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-21T16:00:24.153 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-21T16:00:24.153 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:24.219 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6820.gf0804 121/153 2023-10-21T16:00:24.312 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6820.gf0804 121/153 2023-10-21T16:00:24.312 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:24.312 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-21T16:00:24.312 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 122/153 2023-10-21T16:00:24.519 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 122/153 2023-10-21T16:00:24.519 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:24.519 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-21T16:00:24.519 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-21T16:00:24.520 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-21T16:00:24.520 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 123/153 2023-10-21T16:00:25.050 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 123/153 2023-10-21T16:00:25.050 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-21T16:00:25.051 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-21T16:00:25.051 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-21T16:00:25.051 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-21T16:00:25.051 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:26.985 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-2:18.0.0-6820.gf0804064.el9.x86_64 1/153 2023-10-21T16:00:26.985 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 2/153 2023-10-21T16:00:26.985 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 3/153 2023-10-21T16:00:26.986 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 4/153 2023-10-21T16:00:26.986 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6820.gf0804 5/153 2023-10-21T16:00:26.986 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T16:00:26.986 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T16:00:26.987 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 8/153 2023-10-21T16:00:26.987 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 9/153 2023-10-21T16:00:26.987 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 10/153 2023-10-21T16:00:26.987 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 11/153 2023-10-21T16:00:26.987 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T16:00:26.988 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 13/153 2023-10-21T16:00:26.988 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_ 14/153 2023-10-21T16:00:26.988 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 15/153 2023-10-21T16:00:26.988 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 16/153 2023-10-21T16:00:26.988 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados-devel-2:18.0.0-6820.gf0804064.el9.x86_6 17/153 2023-10-21T16:00:26.989 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 18/153 2023-10-21T16:00:26.989 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 19/153 2023-10-21T16:00:26.989 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6820.gf0804064.el 20/153 2023-10-21T16:00:26.989 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-common-2:18.0.0-6820.gf0804064.el9. 21/153 2023-10-21T16:00:26.989 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_6 22/153 2023-10-21T16:00:26.990 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 23/153 2023-10-21T16:00:26.990 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 24/153 2023-10-21T16:00:26.990 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 25/153 2023-10-21T16:00:26.990 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 26/153 2023-10-21T16:00:26.990 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 27/153 2023-10-21T16:00:26.991 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 28/153 2023-10-21T16:00:26.991 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6820.gf0804064. 29/153 2023-10-21T16:00:26.991 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 30/153 2023-10-21T16:00:26.991 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 31/153 2023-10-21T16:00:26.991 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 32/153 2023-10-21T16:00:26.992 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el 33/153 2023-10-21T16:00:26.992 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 34/153 2023-10-21T16:00:26.992 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.e 35/153 2023-10-21T16:00:26.992 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 36/153 2023-10-21T16:00:26.992 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephadm-2:18.0.0-6820.gf0804064.el9.noarch 37/153 2023-10-21T16:00:26.993 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 38/153 2023-10-21T16:00:26.993 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-21T16:00:26.993 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-21T16:00:26.993 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-21T16:00:26.993 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-21T16:00:26.994 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-21T16:00:26.994 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-21T16:00:26.994 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-21T16:00:26.994 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-21T16:00:26.994 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-21T16:00:26.995 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-21T16:00:26.995 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-21T16:00:26.995 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-21T16:00:26.995 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-21T16:00:26.995 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-21T16:00:26.996 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-21T16:00:26.996 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-21T16:00:26.996 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-21T16:00:26.996 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-21T16:00:26.996 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-21T16:00:26.997 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-21T16:00:26.998 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-21T16:00:26.998 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-21T16:00:26.998 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-21T16:00:26.998 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-21T16:00:26.998 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-21T16:00:26.999 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-21T16:00:26.999 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-21T16:00:26.999 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-21T16:00:26.999 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-21T16:00:26.999 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-21T16:00:27.000 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-21T16:00:27.000 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-21T16:00:27.000 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-21T16:00:27.000 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-21T16:00:27.000 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-21T16:00:27.001 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-21T16:00:27.001 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-21T16:00:27.001 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-21T16:00:27.001 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-21T16:00:27.001 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-21T16:00:27.002 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 84/153 2023-10-21T16:00:27.002 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-21T16:00:27.002 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-21T16:00:27.002 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-21T16:00:27.002 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-21T16:00:27.003 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-21T16:00:27.004 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-21T16:00:27.004 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-21T16:00:27.004 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-21T16:00:27.004 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-21T16:00:27.004 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-21T16:00:27.005 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-21T16:00:27.005 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-21T16:00:27.005 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-21T16:00:27.005 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-21T16:00:27.005 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-21T16:00:27.006 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-21T16:00:27.006 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-21T16:00:27.006 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-21T16:00:27.006 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-21T16:00:27.006 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-21T16:00:27.007 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-21T16:00:27.007 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-21T16:00:27.007 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-21T16:00:27.007 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-21T16:00:27.007 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-21T16:00:27.008 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-21T16:00:27.008 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-21T16:00:27.008 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-21T16:00:27.008 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-21T16:00:27.008 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-21T16:00:27.009 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-21T16:00:27.009 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-21T16:00:27.009 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-21T16:00:27.009 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-21T16:00:27.009 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-21T16:00:27.010 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-21T16:00:27.010 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-21T16:00:27.010 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-21T16:00:27.010 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-21T16:00:27.010 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-2:18.0.0-6820.gf0804064.el9.x86_64 134/153 2023-10-21T16:00:27.011 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-21T16:00:27.012 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 136/153 2023-10-21T16:00:27.012 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-21T16:00:27.012 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-21T16:00:27.012 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-21T16:00:27.012 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-1:1.45.5-1.el9.x86_64 140/153 2023-10-21T16:00:27.013 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-21T16:00:27.013 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 142/153 2023-10-21T16:00:27.013 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-21T16:00:27.013 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-team-1:1.45.5-1.el9.x86_64 144/153 2023-10-21T16:00:27.013 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-21T16:00:27.014 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-tui-1:1.45.5-1.el9.x86_64 146/153 2023-10-21T16:00:27.014 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-21T16:00:27.014 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-21T16:00:27.014 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-21T16:00:27.014 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-21T16:00:27.015 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-21T16:00:27.015 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-21T16:00:29.120 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:29.121 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:29.121 INFO:teuthology.orchestra.run.smithi124.stdout:Upgraded: 2023-10-21T16:00:29.121 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:29.121 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:29.121 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-team-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-tui-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: acl-2.3.1-4.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: librados2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.122 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.123 INFO:teuthology.orchestra.run.smithi124.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-21T16:00:29.123 INFO:teuthology.orchestra.run.smithi124.stdout:Installed: 2023-10-21T16:00:29.123 INFO:teuthology.orchestra.run.smithi124.stdout: automake-1.16.2-8.el9.noarch 2023-10-21T16:00:29.123 INFO:teuthology.orchestra.run.smithi124.stdout: bison-3.7.4-5.el9.x86_64 2023-10-21T16:00:29.123 INFO:teuthology.orchestra.run.smithi124.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-21T16:00:29.124 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.124 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.124 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.124 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.124 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.125 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.126 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.126 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.126 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.126 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.126 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.127 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.128 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:29.128 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-21T16:00:29.128 INFO:teuthology.orchestra.run.smithi124.stdout: flex-2.6.4-9.el9.x86_64 2023-10-21T16:00:29.128 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-21T16:00:29.128 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-21T16:00:29.129 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-21T16:00:29.129 INFO:teuthology.orchestra.run.smithi124.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-21T16:00:29.129 INFO:teuthology.orchestra.run.smithi124.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-21T16:00:29.129 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-21T16:00:29.129 INFO:teuthology.orchestra.run.smithi124.stdout: jq-1.6-15.el9.x86_64 2023-10-21T16:00:29.130 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-21T16:00:29.130 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-21T16:00:29.130 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-21T16:00:29.130 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.130 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.132 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.133 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-21T16:00:29.133 INFO:teuthology.orchestra.run.smithi124.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-21T16:00:29.134 INFO:teuthology.orchestra.run.smithi124.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-21T16:00:29.134 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-21T16:00:29.135 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-21T16:00:29.135 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.135 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.135 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-21T16:00:29.135 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.136 INFO:teuthology.orchestra.run.smithi124.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-21T16:00:29.136 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-21T16:00:29.136 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-21T16:00:29.136 INFO:teuthology.orchestra.run.smithi124.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-21T16:00:29.136 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-21T16:00:29.137 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-13.11-1.el9.x86_64 2023-10-21T16:00:29.138 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-21T16:00:29.138 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-21T16:00:29.138 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-21T16:00:29.138 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-21T16:00:29.138 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-21T16:00:29.139 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-21T16:00:29.139 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-21T16:00:29.139 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.139 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.139 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.140 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-21T16:00:29.140 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-21T16:00:29.140 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-21T16:00:29.140 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-21T16:00:29.140 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-21T16:00:29.141 INFO:teuthology.orchestra.run.smithi124.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-21T16:00:29.141 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-21T16:00:29.141 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-21T16:00:29.141 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-21T16:00:29.141 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-21T16:00:29.142 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-21T16:00:29.143 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-21T16:00:29.143 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-21T16:00:29.143 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-21T16:00:29.143 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-21T16:00:29.143 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-21T16:00:29.144 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-21T16:00:29.144 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-21T16:00:29.144 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-21T16:00:29.144 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-21T16:00:29.144 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-21T16:00:29.145 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-21T16:00:29.145 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ply-3.11-14.el9.noarch 2023-10-21T16:00:29.145 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-21T16:00:29.145 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-21T16:00:29.145 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-21T16:00:29.146 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-21T16:00:29.146 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-21T16:00:29.146 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-21T16:00:29.146 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-21T16:00:29.146 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.147 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.147 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-21T16:00:29.147 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-21T16:00:29.147 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-21T16:00:29.148 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.148 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-21T16:00:29.148 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-21T16:00:29.148 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-21T16:00:29.148 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-21T16:00:29.149 INFO:teuthology.orchestra.run.smithi124.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-21T16:00:29.149 INFO:teuthology.orchestra.run.smithi124.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-21T16:00:29.149 INFO:teuthology.orchestra.run.smithi124.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-21T16:00:29.149 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-21T16:00:29.149 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-21T16:00:29.150 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-21T16:00:29.150 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-21T16:00:29.150 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.150 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.150 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: re2-1:20211101-3.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-21T16:00:29.151 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-21T16:00:29.152 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-21T16:00:29.152 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:29.152 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2023-10-21T16:00:29.486 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 124/153 2023-10-21T16:00:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 125/153 2023-10-21T16:00:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 126/153 2023-10-21T16:00:29.695 DEBUG:teuthology.parallel:result is None 2023-10-21T16:00:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-21T16:00:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-21T16:00:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 129/153 2023-10-21T16:00:30.267 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-21T16:00:30.305 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-21T16:00:30.335 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_ 132/153 2023-10-21T16:00:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.45.5-1.el9.x86_64 133/153 2023-10-21T16:00:30.508 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.45.5-1.el9.x86_64 134/153 2023-10-21T16:00:30.598 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-21T16:00:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-21T16:00:31.962 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:32.011 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:32.069 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-21T16:00:32.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 138/153 2023-10-21T16:00:33.660 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-21T16:00:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-21T16:00:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-21T16:00:33.938 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-21T16:00:33.979 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-21T16:00:34.017 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-21T16:00:34.017 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-21T16:00:34.144 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-21T16:00:34.185 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-21T16:00:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-21T16:00:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-21T16:00:34.312 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:34.313 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-21T16:00:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-21T16:00:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-21T16:00:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-21T16:00:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-21T16:00:34.933 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-21T16:00:35.068 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:35.364 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:35.365 INFO:teuthology.orchestra.run.smithi053.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-21T16:00:35.365 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.0.0-6820.gf0804064.el9.x86_64 1/153 2023-10-21T16:00:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 2/153 2023-10-21T16:00:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 3/153 2023-10-21T16:00:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 4/153 2023-10-21T16:00:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6820.gf0804 5/153 2023-10-21T16:00:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 6/153 2023-10-21T16:00:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 7/153 2023-10-21T16:00:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 8/153 2023-10-21T16:00:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 9/153 2023-10-21T16:00:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 10/153 2023-10-21T16:00:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 11/153 2023-10-21T16:00:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 12/153 2023-10-21T16:00:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 13/153 2023-10-21T16:00:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_ 14/153 2023-10-21T16:00:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 15/153 2023-10-21T16:00:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 16/153 2023-10-21T16:00:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.0.0-6820.gf0804064.el9.x86_6 17/153 2023-10-21T16:00:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86 18/153 2023-10-21T16:00:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 19/153 2023-10-21T16:00:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6820.gf0804064.el 20/153 2023-10-21T16:00:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.0.0-6820.gf0804064.el9. 21/153 2023-10-21T16:00:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_6 22/153 2023-10-21T16:00:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 23/153 2023-10-21T16:00:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 24/153 2023-10-21T16:00:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 25/153 2023-10-21T16:00:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 26/153 2023-10-21T16:00:40.019 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 27/153 2023-10-21T16:00:40.019 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 28/153 2023-10-21T16:00:40.019 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6820.gf0804064. 29/153 2023-10-21T16:00:40.019 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noa 30/153 2023-10-21T16:00:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.n 31/153 2023-10-21T16:00:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6820.gf08 32/153 2023-10-21T16:00:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el 33/153 2023-10-21T16:00:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 34/153 2023-10-21T16:00:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.e 35/153 2023-10-21T16:00:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 36/153 2023-10-21T16:00:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.0.0-6820.gf0804064.el9.noarch 37/153 2023-10-21T16:00:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 38/153 2023-10-21T16:00:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-21T16:00:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-21T16:00:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-21T16:00:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-21T16:00:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-21T16:00:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-21T16:00:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-21T16:00:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-21T16:00:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-21T16:00:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-21T16:00:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-21T16:00:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-21T16:00:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-21T16:00:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-21T16:00:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-21T16:00:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-21T16:00:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-21T16:00:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-21T16:00:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-21T16:00:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-21T16:00:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-21T16:00:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-21T16:00:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-21T16:00:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-21T16:00:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-21T16:00:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-21T16:00:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-21T16:00:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-21T16:00:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-21T16:00:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-21T16:00:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-21T16:00:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-21T16:00:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-21T16:00:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-21T16:00:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-21T16:00:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-21T16:00:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-21T16:00:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-21T16:00:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-21T16:00:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-21T16:00:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 84/153 2023-10-21T16:00:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-21T16:00:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-21T16:00:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-21T16:00:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-21T16:00:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-21T16:00:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-21T16:00:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-21T16:00:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-21T16:00:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-21T16:00:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-21T16:00:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-21T16:00:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-21T16:00:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-21T16:00:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 108/153 2023-10-21T16:00:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-21T16:00:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-21T16:00:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-21T16:00:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-21T16:00:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-21T16:00:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-21T16:00:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-21T16:00:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-21T16:00:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-21T16:00:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-21T16:00:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-21T16:00:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-21T16:00:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-21T16:00:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-21T16:00:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-21T16:00:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-21T16:00:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-21T16:00:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-21T16:00:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.0.0-6820.gf0804064.el9.x86_64 134/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 136/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-21T16:00:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-21T16:00:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-21T16:00:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.45.5-1.el9.x86_64 140/153 2023-10-21T16:00:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-21T16:00:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.45.5-1.el9.x86_64 142/153 2023-10-21T16:00:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-21T16:00:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.45.5-1.el9.x86_64 144/153 2023-10-21T16:00:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-21T16:00:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.45.5-1.el9.x86_64 146/153 2023-10-21T16:00:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-21T16:00:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-21T16:00:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-21T16:00:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-21T16:00:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-21T16:00:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-21T16:00:42.145 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-21T16:00:42.146 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:42.146 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2023-10-21T16:00:42.146 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:42.146 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:42.146 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:42.147 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.45.5-1.el9.x86_64 2023-10-21T16:00:42.147 INFO:teuthology.orchestra.run.smithi053.stdout: acl-2.3.1-4.el9.x86_64 2023-10-21T16:00:42.147 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-21T16:00:42.147 INFO:teuthology.orchestra.run.smithi053.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-21T16:00:42.147 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.148 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.148 INFO:teuthology.orchestra.run.smithi053.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-21T16:00:42.148 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-10-21T16:00:42.148 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2023-10-21T16:00:42.148 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2023-10-21T16:00:42.149 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-21T16:00:42.149 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.149 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.149 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.149 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.150 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.150 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.150 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.150 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.150 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.151 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.151 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.151 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.151 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.151 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.152 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.152 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.152 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.152 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.152 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.153 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.153 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.153 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.154 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.0.0-6820.gf0804064.el9.noarch 2023-10-21T16:00:42.154 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-21T16:00:42.154 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2023-10-21T16:00:42.154 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-21T16:00:42.155 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-21T16:00:42.155 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-21T16:00:42.155 INFO:teuthology.orchestra.run.smithi053.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-21T16:00:42.155 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-21T16:00:42.156 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-21T16:00:42.156 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-15.el9.x86_64 2023-10-21T16:00:42.156 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-21T16:00:42.156 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-21T16:00:42.156 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-21T16:00:42.157 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.157 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.157 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.157 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-21T16:00:42.158 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-21T16:00:42.158 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-21T16:00:42.158 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-21T16:00:42.158 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-21T16:00:42.158 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.159 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.159 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-21T16:00:42.159 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.159 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-21T16:00:42.159 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-21T16:00:42.160 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-21T16:00:42.160 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-21T16:00:42.160 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-21T16:00:42.160 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-21T16:00:42.160 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.11-1.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-21T16:00:42.161 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-21T16:00:42.162 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-21T16:00:42.162 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-21T16:00:42.162 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-21T16:00:42.162 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-21T16:00:42.162 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-21T16:00:42.163 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.163 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.163 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.163 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-21T16:00:42.163 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-21T16:00:42.164 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-21T16:00:42.164 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-21T16:00:42.164 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-21T16:00:42.164 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-21T16:00:42.164 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-10-21T16:00:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-21T16:00:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-21T16:00:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-21T16:00:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-21T16:00:42.165 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-21T16:00:42.166 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-21T16:00:42.166 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-21T16:00:42.166 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-21T16:00:42.166 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-21T16:00:42.166 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-21T16:00:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-21T16:00:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-21T16:00:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-21T16:00:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-21T16:00:42.167 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-21T16:00:42.168 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-21T16:00:42.168 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-21T16:00:42.168 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-21T16:00:42.168 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-21T16:00:42.168 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-21T16:00:42.169 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-21T16:00:42.170 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-21T16:00:42.170 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-21T16:00:42.170 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.170 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.170 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-21T16:00:42.171 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-21T16:00:42.171 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-21T16:00:42.171 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.171 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-21T16:00:42.171 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-21T16:00:42.172 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-21T16:00:42.172 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-21T16:00:42.173 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-21T16:00:42.173 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-21T16:00:42.173 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-10-21T16:00:42.173 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-21T16:00:42.173 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-21T16:00:42.174 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-21T16:00:42.174 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-21T16:00:42.174 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.174 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.174 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.0.0-6820.gf0804064.el9.x86_64 2023-10-21T16:00:42.175 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-3.el9.x86_64 2023-10-21T16:00:42.175 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-21T16:00:42.175 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-21T16:00:42.175 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-21T16:00:42.175 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-21T16:00:42.176 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-21T16:00:42.176 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-21T16:00:42.176 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:42.177 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-10-21T16:00:42.654 DEBUG:teuthology.parallel:result is None 2023-10-21T16:00:42.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f0804064d67a2a63dbabf1801ff452580a050cfc 2023-10-21T16:00:42.803 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-21T16:00:42.867 INFO:teuthology.orchestra.run.smithi053.stdout:18.0.0-6820.gf0804064.el9 2023-10-21T16:00:42.868 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6820.gf0804064.el9 2023-10-21T16:00:42.868 INFO:teuthology.task.install:The correct ceph version 18.0.0-6820.gf0804064 is installed. 2023-10-21T16:00:42.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f0804064d67a2a63dbabf1801ff452580a050cfc 2023-10-21T16:00:43.018 DEBUG:teuthology.orchestra.run.smithi124:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-21T16:00:43.094 INFO:teuthology.orchestra.run.smithi124.stdout:18.0.0-6820.gf0804064.el9 2023-10-21T16:00:43.094 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6820.gf0804064.el9 2023-10-21T16:00:43.095 INFO:teuthology.task.install:The correct ceph version 18.0.0-6820.gf0804064 is installed. 2023-10-21T16:00:43.095 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-21T16:00:43.096 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:43.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-21T16:00:43.184 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:43.185 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-21T16:00:43.291 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-21T16:00:43.292 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:43.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2023-10-21T16:00:43.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-21T16:00:43.492 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:43.493 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/daemon-helper 2023-10-21T16:00:43.593 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-21T16:00:43.736 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-21T16:00:43.737 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:43.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-21T16:00:43.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-21T16:00:43.936 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:43.937 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-21T16:00:44.034 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-21T16:00:44.175 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-21T16:00:44.176 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:44.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2023-10-21T16:00:44.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-21T16:00:44.373 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:44.373 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/stdin-killer 2023-10-21T16:00:44.478 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-21T16:00:44.575 INFO:teuthology.run_tasks:Running task ceph... 2023-10-21T16:00:44.644 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-21T16:00:44.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2023-10-21T16:00:44.654 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 777 /var/log/ceph 2023-10-21T16:00:44.755 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-21T16:00:44.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-21T16:00:44.761 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-21T16:00:44.859 INFO:tasks.ceph:Creating extra log directories... 2023-10-21T16:00:44.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-21T16:00:44.866 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-21T16:00:44.974 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-21T16:00:44.975 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-21T16:00:44.975 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-21T16:00:44.976 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:44.976 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-21T16:00:45.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-21T16:00:45.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-21T16:00:45.332 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:45.333 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-21T16:00:45.394 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-21T16:00:45.673 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-21T16:00:45.776 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-21T16:00:45.776 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-21T16:00:45.777 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7434368', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 76, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2514, 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f0804064d67a2a63dbabf1801ff452580a050cfc/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6fbGUyaNZpvY8oPhmMhokG1MwWoELqZEkefGzsAeko4jFlkENtZa/IW7SEG+imR/eOC9to3WOcrryOomavqFs=', 'smithi124.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuo3XS6+QNnM2uGCiSRrpQv3oO0av8aS5sVXQ0s6Hpb+uYzKM7UdSZpafuhOAmzXO0hreAgrGR6pCLCsexlrdo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f0804064d67a2a63dbabf1801ff452580a050cfc', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-20_18:46:15', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-21T16:00:45.778 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-21T16:00:45.786 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-21T16:00:45.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2023-10-21T16:00:45.870 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/run/ceph 2023-10-21T16:00:45.970 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:45.970 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2023-10-21T16:00:46.014 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-21T16:00:46.014 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2023-10-21T16:00:46.098 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-21T16:00:46.098 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:46.098 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-10-21T16:00:46.099 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:46.099 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:46.099 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-10-21 16:00:35.813764881 +0000 2023-10-21T16:00:46.099 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-10-21 15:57:58.748056510 +0000 2023-10-21T16:00:46.099 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-10-21 15:57:58.748056510 +0000 2023-10-21T16:00:46.100 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-10-21 15:57:58.748056510 +0000 2023-10-21T16:00:46.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-21T16:00:46.214 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-10-21T16:00:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-10-21T16:00:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000229266 s, 2.2 MB/s 2023-10-21T16:00:46.217 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-21T16:00:46.263 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2023-10-21T16:00:46.344 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-21T16:00:46.345 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:46.345 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-21T16:00:46.345 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:46.345 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:46.346 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-10-21 16:00:35.814764862 +0000 2023-10-21T16:00:46.346 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-10-21 15:57:59.789036465 +0000 2023-10-21T16:00:46.346 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-10-21 15:57:59.789036465 +0000 2023-10-21T16:00:46.346 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-10-21 15:57:59.789036465 +0000 2023-10-21T16:00:46.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-21T16:00:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-10-21T16:00:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-10-21T16:00:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000197515 s, 2.6 MB/s 2023-10-21T16:00:46.467 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-21T16:00:46.516 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2023-10-21T16:00:46.558 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-21T16:00:46.558 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:46.558 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-10-21T16:00:46.559 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:46.559 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:46.559 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-10-21 16:00:35.815764843 +0000 2023-10-21T16:00:46.559 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-10-21 15:58:00.879015476 +0000 2023-10-21T16:00:46.559 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-10-21 15:58:00.879015476 +0000 2023-10-21T16:00:46.560 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-10-21 15:58:00.879015476 +0000 2023-10-21T16:00:46.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-21T16:00:46.668 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-10-21T16:00:46.669 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-10-21T16:00:46.669 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000186667 s, 2.7 MB/s 2023-10-21T16:00:46.671 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-21T16:00:46.716 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2023-10-21T16:00:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-21T16:00:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:46.799 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-10-21T16:00:46.800 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:46.800 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:46.800 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-10-21 16:00:35.816764824 +0000 2023-10-21T16:00:46.800 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-10-21 15:58:01.942994987 +0000 2023-10-21T16:00:46.801 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-10-21 15:58:01.942994987 +0000 2023-10-21T16:00:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2023-10-21 15:58:01.942994987 +0000 2023-10-21T16:00:46.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-21T16:00:46.921 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-10-21T16:00:46.922 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-10-21T16:00:46.922 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000198128 s, 2.6 MB/s 2023-10-21T16:00:46.926 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-21T16:00:46.966 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:46.966 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:46.966 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/scratch_devs of=/dev/stdout 2023-10-21T16:00:47.027 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-21T16:00:47.028 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_1 2023-10-21T16:00:47.081 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-21T16:00:47.082 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:47.082 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 630 Links: 1 2023-10-21T16:00:47.082 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:47.082 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:47.082 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-21 16:00:22.689778545 +0000 2023-10-21T16:00:47.083 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-21 15:57:32.793110072 +0000 2023-10-21T16:00:47.083 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-21 15:57:32.793110072 +0000 2023-10-21T16:00:47.083 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2023-10-21 15:57:32.793110072 +0000 2023-10-21T16:00:47.083 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-21T16:00:47.172 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-21T16:00:47.173 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-21T16:00:47.173 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000321757 s, 1.6 MB/s 2023-10-21T16:00:47.176 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-21T16:00:47.235 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_2 2023-10-21T16:00:47.289 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-21T16:00:47.289 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:47.289 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 649 Links: 1 2023-10-21T16:00:47.289 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:47.290 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:47.290 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-21 16:00:22.690778529 +0000 2023-10-21T16:00:47.290 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-21 15:57:33.808092941 +0000 2023-10-21T16:00:47.290 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-21 15:57:33.808092941 +0000 2023-10-21T16:00:47.290 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2023-10-21 15:57:33.808092941 +0000 2023-10-21T16:00:47.291 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-21T16:00:47.381 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-21T16:00:47.381 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-21T16:00:47.381 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000371763 s, 1.4 MB/s 2023-10-21T16:00:47.384 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-21T16:00:47.442 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_3 2023-10-21T16:00:47.494 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-21T16:00:47.494 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:47.495 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 666 Links: 1 2023-10-21T16:00:47.495 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:47.495 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:47.495 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-21 16:00:22.690778529 +0000 2023-10-21T16:00:47.495 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-21 15:57:34.851075337 +0000 2023-10-21T16:00:47.496 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-21 15:57:34.851075337 +0000 2023-10-21T16:00:47.496 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2023-10-21 15:57:34.851075337 +0000 2023-10-21T16:00:47.496 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-21T16:00:47.585 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-21T16:00:47.585 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-21T16:00:47.585 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000372761 s, 1.4 MB/s 2023-10-21T16:00:47.588 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-21T16:00:47.642 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_4 2023-10-21T16:00:47.695 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-21T16:00:47.695 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-21T16:00:47.696 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 685 Links: 1 2023-10-21T16:00:47.696 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-21T16:00:47.696 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2023-10-21T16:00:47.696 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2023-10-21 16:00:22.691778512 +0000 2023-10-21T16:00:47.697 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2023-10-21 15:57:35.877058020 +0000 2023-10-21T16:00:47.697 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2023-10-21 15:57:35.877058020 +0000 2023-10-21T16:00:47.697 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2023-10-21 15:57:35.877058020 +0000 2023-10-21T16:00:47.697 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-21T16:00:47.784 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2023-10-21T16:00:47.784 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2023-10-21T16:00:47.784 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.00030735 s, 1.7 MB/s 2023-10-21T16:00:47.787 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-21T16:00:47.847 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:47.847 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-21T16:00:47.847 INFO:tasks.ceph:Generating config... 2023-10-21T16:00:47.849 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-21T16:00:47.849 INFO:tasks.ceph:[client] debug client = 20 2023-10-21T16:00:47.849 INFO:tasks.ceph:[client] debug ms = 1 2023-10-21T16:00:47.849 INFO:tasks.ceph:[client] pid file = 2023-10-21T16:00:47.850 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-21T16:00:47.850 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-21T16:00:47.850 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-21T16:00:47.851 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-21T16:00:47.851 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-21T16:00:47.851 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-21T16:00:47.851 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-21T16:00:47.852 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-21T16:00:47.852 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-21T16:00:47.852 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-21T16:00:47.852 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-21T16:00:47.853 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-21T16:00:47.853 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-21T16:00:47.853 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-21T16:00:47.853 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-21T16:00:47.854 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-21T16:00:47.854 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-21T16:00:47.854 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-21T16:00:47.854 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-21T16:00:47.855 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-21T16:00:47.855 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-21T16:00:47.855 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-21T16:00:47.855 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-21T16:00:47.856 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-21T16:00:47.856 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-21T16:00:47.856 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-21T16:00:47.856 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-21T16:00:47.857 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-21T16:00:47.857 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-21T16:00:47.857 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-21T16:00:47.857 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-21T16:00:47.858 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-21T16:00:47.858 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-21T16:00:47.858 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-21T16:00:47.859 INFO:tasks.ceph:Setting up mon.a... 2023-10-21T16:00:47.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-21T16:00:48.024 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.keyring 2023-10-21T16:00:48.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-21T16:00:48.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-21T16:00:48.283 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.53'), ('mon.b', '172.21.15.124'), ('mon.c', '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]')] 2023-10-21T16:00:48.284 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.53,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-21T16:00:48.285 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:48.285 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-21T16:00:48.347 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.53 --add b 172.21.15.124 --addv c '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:48.513 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:48.514 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: generated fsid d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56 2023-10-21T16:00:48.514 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = pacific 2023-10-21T16:00:48.514 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 0 2023-10-21T16:00:48.514 INFO:teuthology.orchestra.run.smithi053.stdout:fsid d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56 2023-10-21T16:00:48.515 INFO:teuthology.orchestra.run.smithi053.stdout:last_changed 2023-10-21T16:00:48.512997+0000 2023-10-21T16:00:48.515 INFO:teuthology.orchestra.run.smithi053.stdout:created 2023-10-21T16:00:48.512997+0000 2023-10-21T16:00:48.515 INFO:teuthology.orchestra.run.smithi053.stdout:min_mon_release 16 (pacific) 2023-10-21T16:00:48.515 INFO:teuthology.orchestra.run.smithi053.stdout:election_strategy: 1 2023-10-21T16:00:48.515 INFO:teuthology.orchestra.run.smithi053.stdout:0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2023-10-21T16:00:48.516 INFO:teuthology.orchestra.run.smithi053.stdout:1: [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] mon.b 2023-10-21T16:00:48.516 INFO:teuthology.orchestra.run.smithi053.stdout:2: [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] mon.c 2023-10-21T16:00:48.516 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-21T16:00:48.517 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-21T16:00:48.570 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56... 2023-10-21T16:00:48.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-21T16:00:48.578 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-21T16:00:48.751 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2023-10-21T16:00:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2023-10-21T16:00:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-21T16:00:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2023-10-21T16:00:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2023-10-21T16:00:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2023-10-21T16:00:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2023-10-21T16:00:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-21T16:00:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2023-10-21T16:00:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2023-10-21T16:00:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-21T16:00:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2023-10-21T16:00:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2023-10-21T16:00:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2023-10-21T16:00:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2023-10-21T16:00:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-21T16:00:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2023-10-21T16:00:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-21T16:00:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2023-10-21T16:00:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2023-10-21T16:00:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2023-10-21T16:00:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2023-10-21T16:00:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on skipped message = False 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2023-10-21T16:00:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2023-10-21T16:00:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2023-10-21T16:00:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2023-10-21T16:00:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2023-10-21T16:00:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2023-10-21T16:00:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2023-10-21T16:00:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2023-10-21T16:00:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2023-10-21T16:00:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2023-10-21T16:00:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2023-10-21T16:00:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2023-10-21T16:00:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2023-10-21T16:00:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 20 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 20 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2023-10-21T16:00:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 10 2023-10-21T16:00:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2023-10-21T16:00:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2023-10-21T16:00:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2023-10-21T16:00:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2023-10-21T16:00:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2023-10-21T16:00:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2023-10-21T16:00:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2023-10-21T16:00:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2023-10-21T16:00:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2023-10-21T16:00:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2023-10-21T16:00:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2023-10-21T16:00:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: debug finisher = 20 2023-10-21T16:00:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.770 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2023-10-21T16:00:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-10-21T16:00:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2023-10-21T16:00:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2023-10-21T16:00:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2023-10-21T16:00:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2023-10-21T16:00:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-21T16:00:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-21T16:00:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2023-10-21T16:00:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2023-10-21T16:00:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2023-10-21T16:00:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2023-10-21T16:00:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:00:48.775 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2023-10-21T16:00:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2023-10-21T16:00:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2023-10-21T16:00:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-10-21T16:00:48.776 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2023-10-21T16:00:48.777 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2023-10-21T16:00:48.777 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2023-10-21T16:00:48.777 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2023-10-21T16:00:48.777 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2023-10-21T16:00:48.777 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2023-10-21T16:00:48.778 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2023-10-21T16:00:48.778 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2023-10-21T16:00:48.778 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2023-10-21T16:00:48.778 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2023-10-21T16:00:48.778 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2023-10-21T16:00:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2023-10-21T16:00:48.780 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2023-10-21T16:00:48.780 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2023-10-21T16:00:48.780 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2023-10-21T16:00:48.780 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2023-10-21T16:00:48.781 INFO:teuthology.orchestra.run.smithi124.stdout:[global] 2023-10-21T16:00:48.781 INFO:teuthology.orchestra.run.smithi124.stdout: chdir = "" 2023-10-21T16:00:48.781 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-21T16:00:48.782 INFO:teuthology.orchestra.run.smithi124.stdout: auth supported = cephx 2023-10-21T16:00:48.782 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.782 INFO:teuthology.orchestra.run.smithi124.stdout: filestore xattr use omap = true 2023-10-21T16:00:48.782 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.782 INFO:teuthology.orchestra.run.smithi124.stdout: mon clock drift allowed = 1.000 2023-10-21T16:00:48.783 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.783 INFO:teuthology.orchestra.run.smithi124.stdout: osd crush chooseleaf type = 0 2023-10-21T16:00:48.783 INFO:teuthology.orchestra.run.smithi124.stdout: auth debug = true 2023-10-21T16:00:48.783 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on old message = true 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on bug = true 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: mon pg warn max object skew = 0 2023-10-21T16:00:48.784 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.785 INFO:teuthology.orchestra.run.smithi124.stdout: # disable pg_autoscaler by default for new pools 2023-10-21T16:00:48.785 INFO:teuthology.orchestra.run.smithi124.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-21T16:00:48.785 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.785 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default size = 2 2023-10-21T16:00:48.785 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.786 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow primary affinity = true 2023-10-21T16:00:48.786 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow pg remap = true 2023-10-21T16:00:48.786 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on legacy crush tunables = false 2023-10-21T16:00:48.786 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on crush straw calc version zero = false 2023-10-21T16:00:48.786 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on no sortbitwise = false 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on osd down out interval zero = false 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on too few osds = false 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: mon_allow_pool_size_one = true 2023-10-21T16:00:48.787 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.788 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-21T16:00:48.788 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.788 INFO:teuthology.orchestra.run.smithi124.stdout: osd default data pool replay window = 5 2023-10-21T16:00:48.788 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.788 INFO:teuthology.orchestra.run.smithi124.stdout: mon allow pool delete = true 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: mon cluster log file level = debug 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: debug asserts on shutdown = true 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: mon health detail to clog = false 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: mon host = "172.21.15.53,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2023-10-21T16:00:48.789 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on skipped message = False 2023-10-21T16:00:48.790 INFO:teuthology.orchestra.run.smithi124.stdout: fsid = d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56 2023-10-21T16:00:48.790 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.790 INFO:teuthology.orchestra.run.smithi124.stdout:[osd] 2023-10-21T16:00:48.790 INFO:teuthology.orchestra.run.smithi124.stdout: osd journal size = 100 2023-10-21T16:00:48.790 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.791 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub load threshold = 5.0 2023-10-21T16:00:48.791 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub max interval = 600 2023-10-21T16:00:48.791 INFO:teuthology.orchestra.run.smithi124.stdout: osd mclock profile = high_recovery_ops 2023-10-21T16:00:48.791 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.791 INFO:teuthology.orchestra.run.smithi124.stdout: osd recover clone overlap = true 2023-10-21T16:00:48.792 INFO:teuthology.orchestra.run.smithi124.stdout: osd recovery max chunk = 1048576 2023-10-21T16:00:48.792 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.792 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug shutdown = true 2023-10-21T16:00:48.792 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug op order = true 2023-10-21T16:00:48.792 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug verify stray on activate = true 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: osd open classes on start = true 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug pg log writeout = true 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: osd deep scrub update digest min age = 30 2023-10-21T16:00:48.793 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.794 INFO:teuthology.orchestra.run.smithi124.stdout: osd map max advance = 10 2023-10-21T16:00:48.794 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.794 INFO:teuthology.orchestra.run.smithi124.stdout: journal zero on create = true 2023-10-21T16:00:48.794 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.794 INFO:teuthology.orchestra.run.smithi124.stdout: filestore ondisk finisher threads = 3 2023-10-21T16:00:48.795 INFO:teuthology.orchestra.run.smithi124.stdout: filestore apply finisher threads = 3 2023-10-21T16:00:48.795 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.795 INFO:teuthology.orchestra.run.smithi124.stdout: bdev debug aio = true 2023-10-21T16:00:48.795 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug misdirected ops = true 2023-10-21T16:00:48.795 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore block size = 96636764160 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore fsck on mount = True 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluefs = 20 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluestore = 20 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: debug osd = 20 2023-10-21T16:00:48.796 INFO:teuthology.orchestra.run.smithi124.stdout: debug rocksdb = 10 2023-10-21T16:00:48.797 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd backfillfull_ratio = 0.85 2023-10-21T16:00:48.797 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd full ratio = 0.9 2023-10-21T16:00:48.797 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd nearfull ratio = 0.8 2023-10-21T16:00:48.797 INFO:teuthology.orchestra.run.smithi124.stdout: osd failsafe full ratio = 0.95 2023-10-21T16:00:48.797 INFO:teuthology.orchestra.run.smithi124.stdout: osd objectstore = bluestore 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr] 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout: debug mgr = 20 2023-10-21T16:00:48.798 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2023-10-21T16:00:48.799 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2023-10-21T16:00:48.799 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2023-10-21T16:00:48.799 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2023-10-21T16:00:48.799 INFO:teuthology.orchestra.run.smithi124.stdout: mgr/telemetry/nag = false 2023-10-21T16:00:48.799 INFO:teuthology.orchestra.run.smithi124.stdout: debug client = 20 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout: debug finisher = 20 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout:[mon] 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2023-10-21T16:00:48.800 INFO:teuthology.orchestra.run.smithi124.stdout: debug paxos = 20 2023-10-21T16:00:48.801 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2023-10-21T16:00:48.801 INFO:teuthology.orchestra.run.smithi124.stdout: mon data avail warn = 5 2023-10-21T16:00:48.801 INFO:teuthology.orchestra.run.smithi124.stdout: mon mgr mkfs grace = 240 2023-10-21T16:00:48.801 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2023-10-21T16:00:48.801 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd reporter subtree level = osd 2023-10-21T16:00:48.802 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd prime pg temp = true 2023-10-21T16:00:48.802 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2023-10-21T16:00:48.802 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.802 INFO:teuthology.orchestra.run.smithi124.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-21T16:00:48.802 INFO:teuthology.orchestra.run.smithi124.stdout: auth mon ticket ttl = 660 # 11m 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: auth service ticket ttl = 240 # 4m 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: # don't complain about insecure global_id in the test suite 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-21T16:00:48.803 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.804 INFO:teuthology.orchestra.run.smithi124.stdout: # 1m isn't quite enough 2023-10-21T16:00:48.804 INFO:teuthology.orchestra.run.smithi124.stdout: mon_down_mkfs_grace = 2m 2023-10-21T16:00:48.804 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.804 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_filestore_osds = false 2023-10-21T16:00:48.805 INFO:teuthology.orchestra.run.smithi124.stdout: mon op complaint time = 120 2023-10-21T16:00:48.805 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-10-21T16:00:48.805 INFO:teuthology.orchestra.run.smithi124.stdout:[client] 2023-10-21T16:00:48.805 INFO:teuthology.orchestra.run.smithi124.stdout: rgw cache enabled = true 2023-10-21T16:00:48.805 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable ops log = true 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable usage log = true 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: client mount timeout = 600 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: debug client = 20 2023-10-21T16:00:48.806 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-21T16:00:48.807 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = "" 2023-10-21T16:00:48.807 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2023-10-21T16:00:48.807 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2023-10-21T16:00:48.807 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.a] 2023-10-21T16:00:48.807 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.b] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.c] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2023-10-21T16:00:48.808 INFO:teuthology.orchestra.run.smithi124.stdout:[mds] 2023-10-21T16:00:48.809 INFO:teuthology.orchestra.run.smithi124.stdout: debug mds = 20 2023-10-21T16:00:48.809 INFO:teuthology.orchestra.run.smithi124.stdout: debug mds balancer = 20 2023-10-21T16:00:48.809 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2023-10-21T16:00:48.809 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug frag = True 2023-10-21T16:00:48.809 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug scatterstat = True 2023-10-21T16:00:48.810 INFO:teuthology.orchestra.run.smithi124.stdout: mds op complaint time = 180 2023-10-21T16:00:48.810 INFO:teuthology.orchestra.run.smithi124.stdout: mds verify scatter = True 2023-10-21T16:00:48.810 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2023-10-21T16:00:48.810 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2023-10-21T16:00:48.810 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2023-10-21T16:00:48.812 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-21T16:00:48.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-21T16:00:48.969 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-21T16:00:48.969 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:48.970 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-21T16:00:49.022 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:49.022 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-21T16:00:49.076 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:00:49.077 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:49.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-21T16:00:49.078 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-21T16:00:49.239 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:00:49.239 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:49.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:00:49.344 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:49.344 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-21T16:00:49.344 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-21T16:00:49.504 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:00:49.505 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:49.604 INFO:tasks.ceph:Setting up mon nodes... 2023-10-21T16:00:49.605 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-21T16:00:49.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-21T16:00:49.799 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-21T16:00:49.809 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-21T16:00:50.016 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-21T16:00:50.027 INFO:tasks.ceph:Setting up mds nodes... 2023-10-21T16:00:50.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-21T16:00:50.237 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-21T16:00:50.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-21T16:00:50.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-21T16:00:50.550 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-21T16:00:50.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-21T16:00:50.661 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-21T16:00:50.852 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-21T16:00:50.861 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-21T16:00:50.951 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-21T16:00:51.133 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-21T16:00:51.142 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-21T16:00:51.237 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-21T16:00:51.237 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-21T16:00:51.379 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-21T16:00:51.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-21T16:00:51.580 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-21T16:00:51.646 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-21T16:00:51.792 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-21T16:00:51.848 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-21T16:00:51.976 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-21T16:00:52.029 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-21T16:00:52.030 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-21T16:00:52.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-21T16:00:52.144 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:52.144 INFO:tasks.ceph:role: osd.0 2023-10-21T16:00:52.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:00:52.145 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-21T16:00:52.245 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:00:52.245 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:00:52.245 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:00:52.246 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:00:52.246 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:00:52.246 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:00:52.246 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:00:52.246 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:00:52.247 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:00:52.247 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:00:52.278 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-10-21T16:00:52.284 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-10-21T16:00:52.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-21T16:00:52.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-21T16:00:52.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-21T16:00:52.570 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:52.571 INFO:tasks.ceph:role: osd.1 2023-10-21T16:00:52.571 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:00:52.571 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-21T16:00:52.666 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:00:52.666 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:00:52.666 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:00:52.666 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:00:52.667 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:00:52.703 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-10-21T16:00:52.709 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-10-21T16:00:52.710 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-21T16:00:52.830 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-21T16:00:52.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-21T16:00:53.032 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:53.032 INFO:tasks.ceph:role: osd.2 2023-10-21T16:00:53.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:00:53.033 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-21T16:00:53.133 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:00:53.134 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:00:53.134 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:00:53.134 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:00:53.134 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:00:53.134 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:00:53.135 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:00:53.135 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:00:53.135 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:00:53.135 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:00:53.168 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-10-21T16:00:53.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-10-21T16:00:53.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-21T16:00:53.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-21T16:00:53.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-21T16:00:53.496 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-21T16:00:53.496 INFO:tasks.ceph:role: osd.3 2023-10-21T16:00:53.497 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:00:53.497 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-21T16:00:53.597 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:00:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:00:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:00:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:00:53.598 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:00:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:00:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:00:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:00:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:00:53.599 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:00:53.631 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2023-10-21T16:00:53.637 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2023-10-21T16:00:53.638 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-21T16:00:53.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-21T16:00:53.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:54.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.024+0000 7fd24ac0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-21T16:00:54.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.025+0000 7fd24ac0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-21T16:00:54.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.025+0000 7fd24ac0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-21T16:00:54.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.025+0000 7fd24ac0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-21T16:00:54.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.025+0000 7fd24ac0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-21T16:00:54.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:54.029+0000 7fd24ac0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-21T16:00:57.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-21T16:00:57.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:00:57.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.431+0000 7fe6ab62b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-21T16:00:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.431+0000 7fe6ab62b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-21T16:00:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.431+0000 7fe6ab62b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-21T16:00:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.431+0000 7fe6ab62b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-21T16:00:57.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.431+0000 7fe6ab62b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-21T16:00:57.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:00:57.433+0000 7fe6ab62b500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-21T16:01:00.638 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-21T16:01:00.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.895+0000 7fcd11823500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-21T16:01:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.895+0000 7fcd11823500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-21T16:01:00.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.895+0000 7fcd11823500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-21T16:01:00.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.895+0000 7fcd11823500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-21T16:01:00.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.895+0000 7fcd11823500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-21T16:01:00.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:00.898+0000 7fcd11823500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-21T16:01:04.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-21T16:01:04.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.328+0000 7f20f4e15500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-21T16:01:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.329+0000 7f20f4e15500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-21T16:01:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.329+0000 7f20f4e15500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-21T16:01:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.329+0000 7f20f4e15500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-21T16:01:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.329+0000 7f20f4e15500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-21T16:01:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:04.331+0000 7f20f4e15500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-21T16:01:07.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-21T16:01:07.612 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-21T16:01:07.717 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-21T16:01:07.718 INFO:tasks.ceph:role: osd.4 2023-10-21T16:01:07.718 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:01:07.718 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-21T16:01:07.819 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:01:07.820 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:01:07.820 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:01:07.820 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:01:07.820 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:01:07.821 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:01:07.821 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:01:07.821 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:01:07.821 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:01:07.821 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:01:08.340 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-21T16:01:08.346 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-21T16:01:08.347 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-21T16:01:08.502 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-21T16:01:08.595 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-21T16:01:08.675 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-21T16:01:08.676 INFO:tasks.ceph:role: osd.5 2023-10-21T16:01:08.676 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:01:08.676 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-21T16:01:08.751 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:01:08.752 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:01:08.752 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:01:08.752 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:01:08.752 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:01:08.752 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:01:08.753 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:01:08.753 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:01:08.753 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:01:08.753 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:01:09.238 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-21T16:01:09.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-21T16:01:09.245 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-21T16:01:09.369 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-21T16:01:09.454 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-21T16:01:09.537 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-21T16:01:09.538 INFO:tasks.ceph:role: osd.6 2023-10-21T16:01:09.538 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:01:09.538 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-21T16:01:09.630 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:01:09.631 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:01:09.631 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:01:09.631 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:01:09.631 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:01:09.632 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:01:09.632 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:01:09.632 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:01:09.632 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:01:09.632 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:01:10.151 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-21T16:01:10.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-21T16:01:10.157 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-21T16:01:10.339 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-21T16:01:10.440 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-21T16:01:10.532 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-21T16:01:10.532 INFO:tasks.ceph:role: osd.7 2023-10-21T16:01:10.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:01:10.533 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-21T16:01:10.631 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-21T16:01:10.631 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-21T16:01:10.631 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-21T16:01:10.632 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-21T16:01:10.632 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-21T16:01:10.632 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2023-10-21T16:01:10.632 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-21T16:01:10.632 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-21T16:01:10.633 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-21T16:01:10.633 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-21T16:01:11.122 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2023-10-21T16:01:11.127 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2023-10-21T16:01:11.128 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-21T16:01:11.263 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-21T16:01:11.359 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:11.522 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.520+0000 7fb617192500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-21T16:01:11.523 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.521+0000 7fb617192500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-21T16:01:11.523 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.521+0000 7fb617192500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-21T16:01:11.523 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.521+0000 7fb617192500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-21T16:01:11.523 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.521+0000 7fb617192500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-21T16:01:11.532 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:11.530+0000 7fb617192500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-21T16:01:14.708 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-21T16:01:14.799 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:14.925 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.923+0000 7f5722a1a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-21T16:01:14.926 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.923+0000 7f5722a1a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-21T16:01:14.926 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.924+0000 7f5722a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-21T16:01:14.926 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.924+0000 7f5722a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-21T16:01:14.927 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.924+0000 7f5722a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-21T16:01:14.928 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:14.926+0000 7f5722a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-21T16:01:15.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:01:15.789 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:01:18.092 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-21T16:01:18.190 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:18.332 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.329+0000 7f16bc618500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-21T16:01:18.333 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.330+0000 7f16bc618500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-21T16:01:18.333 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.330+0000 7f16bc618500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-21T16:01:18.334 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.330+0000 7f16bc618500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-21T16:01:18.334 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.330+0000 7f16bc618500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-21T16:01:18.335 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:18.332+0000 7f16bc618500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-21T16:01:21.503 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-21T16:01:21.604 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:21.742 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.740+0000 7f6b0590e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-21T16:01:21.743 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.740+0000 7f6b0590e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-21T16:01:21.744 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.741+0000 7f6b0590e500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-21T16:01:21.744 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.741+0000 7f6b0590e500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-21T16:01:21.744 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.741+0000 7f6b0590e500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-21T16:01:21.745 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:21.743+0000 7f6b0590e500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-21T16:01:24.923 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-21T16:01:25.022 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-21T16:01:25.023 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-21T16:01:25.132 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-21T16:01:25.240 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-21T16:01:25.350 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-21T16:01:25.458 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-21T16:01:25.563 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-21T16:01:25.667 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:25.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-21T16:01:25.769 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:25.769 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-21T16:01:25.874 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:25.874 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-21T16:01:25.970 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:25.971 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-21T16:01:26.063 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.064 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-21T16:01:26.160 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-21T16:01:26.254 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.254 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-21T16:01:26.349 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.349 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-21T16:01:26.447 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:26.448 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-21T16:01:26.511 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:26.512 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-21T16:01:26.568 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.568 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-21T16:01:26.628 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:26.629 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-21T16:01:26.684 INFO:tasks.ceph:Adding keys to all mons... 2023-10-21T16:01:26.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-21T16:01:26.693 DEBUG:teuthology.orchestra.run.smithi124:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-21T16:01:26.787 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.y] 2023-10-21T16:01:26.787 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ== 2023-10-21T16:01:26.787 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2023-10-21T16:01:26.787 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q== 2023-10-21T16:01:26.788 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2023-10-21T16:01:26.788 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA== 2023-10-21T16:01:26.788 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.0] 2023-10-21T16:01:26.788 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw== 2023-10-21T16:01:26.788 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.1] 2023-10-21T16:01:26.789 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ== 2023-10-21T16:01:26.789 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.2] 2023-10-21T16:01:26.789 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA== 2023-10-21T16:01:26.789 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.3] 2023-10-21T16:01:26.789 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q== 2023-10-21T16:01:26.790 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.x] 2023-10-21T16:01:26.790 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew== 2023-10-21T16:01:26.790 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2023-10-21T16:01:26.791 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ== 2023-10-21T16:01:26.791 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2023-10-21T16:01:26.791 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw== 2023-10-21T16:01:26.791 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.4] 2023-10-21T16:01:26.791 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ== 2023-10-21T16:01:26.792 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.5] 2023-10-21T16:01:26.792 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ== 2023-10-21T16:01:26.792 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.6] 2023-10-21T16:01:26.792 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ== 2023-10-21T16:01:26.792 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.7] 2023-10-21T16:01:26.793 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA== 2023-10-21T16:01:26.793 INFO:teuthology.orchestra.run.smithi124.stdout:[client.0] 2023-10-21T16:01:26.793 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:01:26.793 INFO:teuthology.orchestra.run.smithi124.stdout:[client.2] 2023-10-21T16:01:26.793 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg== 2023-10-21T16:01:26.794 INFO:teuthology.orchestra.run.smithi124.stdout:[client.1] 2023-10-21T16:01:26.794 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg== 2023-10-21T16:01:26.794 INFO:teuthology.orchestra.run.smithi124.stdout:[client.3] 2023-10-21T16:01:26.794 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA== 2023-10-21T16:01:26.796 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2023-10-21T16:01:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ== 2023-10-21T16:01:26.797 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2023-10-21T16:01:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q== 2023-10-21T16:01:26.797 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.c] 2023-10-21T16:01:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA== 2023-10-21T16:01:26.797 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2023-10-21T16:01:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw== 2023-10-21T16:01:26.798 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2023-10-21T16:01:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ== 2023-10-21T16:01:26.798 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2023-10-21T16:01:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA== 2023-10-21T16:01:26.799 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2023-10-21T16:01:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q== 2023-10-21T16:01:26.799 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2023-10-21T16:01:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew== 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ== 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.d] 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw== 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2023-10-21T16:01:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ== 2023-10-21T16:01:26.801 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2023-10-21T16:01:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ== 2023-10-21T16:01:26.801 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2023-10-21T16:01:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ== 2023-10-21T16:01:26.801 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2023-10-21T16:01:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA== 2023-10-21T16:01:26.802 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:01:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:01:26.802 INFO:teuthology.orchestra.run.smithi053.stdout:[client.2] 2023-10-21T16:01:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg== 2023-10-21T16:01:26.803 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2023-10-21T16:01:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg== 2023-10-21T16:01:26.803 INFO:teuthology.orchestra.run.smithi053.stdout:[client.3] 2023-10-21T16:01:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA== 2023-10-21T16:01:26.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-21T16:01:26.813 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-21T16:01:26.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:26.971 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:27.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:27.130 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:27.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.286 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.438 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.588 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.742 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:27.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-21T16:01:27.897 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-21T16:01:28.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:28.050 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:28.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:28.198 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-21T16:01:28.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.358 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.516 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.666 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.675 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.826 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-21T16:01:28.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:28.984 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.136 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.295 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.451 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-21T16:01:29.594 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-21T16:01:29.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-21T16:01:29.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-21T16:01:29.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-21T16:01:29.965 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-21T16:01:30.067 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-21T16:01:30.226 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-21T16:01:30.307 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-21T16:01:30.403 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-21T16:01:30.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-21T16:01:30.636 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:30.645 DEBUG:teuthology.orchestra.run.smithi124:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-21T16:01:30.700 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-21T16:01:30.701 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-21T16:01:30.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-21T16:01:30.708 INFO:tasks.ceph.mon.a:Started 2023-10-21T16:01:30.709 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-21T16:01:30.709 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-21T16:01:30.714 INFO:tasks.ceph.mon.b:Started 2023-10-21T16:01:30.714 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-21T16:01:30.715 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-21T16:01:30.720 INFO:tasks.ceph.mon.c:Started 2023-10-21T16:01:30.720 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-21T16:01:30.720 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-21T16:01:30.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-21T16:01:30.726 INFO:tasks.ceph.mgr.y:Started 2023-10-21T16:01:30.727 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-21T16:01:30.727 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-21T16:01:30.732 INFO:tasks.ceph.mgr.x:Started 2023-10-21T16:01:30.732 INFO:tasks.ceph:Setting crush tunables to default 2023-10-21T16:01:30.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd crush tunables default 2023-10-21T16:01:30.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.921+0000 7fab62fd5640 1 Processor -- start 2023-10-21T16:01:30.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.921+0000 7fab62fd5640 1 -- start start 2023-10-21T16:01:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.922+0000 7fab62fd5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.922+0000 7fab62fd5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.922+0000 7fab62fd5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c06a130 con 0x7fab5c06dab0 2023-10-21T16:01:30.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.922+0000 7fab62fd5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c06b710 con 0x7fab5c15cc80 2023-10-21T16:01:30.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.922+0000 7fab62fd5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06a410 con 0x7fab5c0cd240 2023-10-21T16:01:30.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab61572640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:30.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab5bfff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fab5c06dab0 0x7fab5c0ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:56422/0 (socket says 172.21.15.53:56422) 2023-10-21T16:01:30.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab60d71640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:30.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 learned_addr learned my addr 172.21.15.53:0/1149712532 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:30.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:30.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:30.923+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.124+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.124+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-21T16:01:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:01:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.125+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:01:31.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.526+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.526+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:31.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:31.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-21T16:01:31.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:31.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:31.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:01:31.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:31.527+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:01:32.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.328+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:32.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:32.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:32.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:32.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:32.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:01:32.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-21T16:01:32.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:32.329+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:01:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.922+0000 7fab5a7fc640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 msgr2=0x7fab5c0c95b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.922+0000 7fab5a7fc640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab5c0cd240 0x7fab5c0c95b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.922+0000 7fab5a7fc640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 msgr2=0x7fab5c06b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.922+0000 7fab5a7fc640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.124:3300/0 conn(0x7fab5c15cc80 0x7fab5c06b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.923+0000 7fab5a7fc640 1 -- 172.21.15.53:0/1149712532 >> v1:172.21.15.124:6789/0 conn(0x7fab5c06dab0 legacy=0x7fab5c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.923+0000 7fab5a7fc640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.923+0000 7fab5a7fc640 1 -- 172.21.15.53:0/1149712532 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c06a6f0 con 0x7fab400036a0 2023-10-21T16:01:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.923+0000 7fab5a7fc640 1 -- 172.21.15.53:0/1149712532 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c06b710 con 0x7fab40007330 2023-10-21T16:01:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.923+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.924+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.924+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.924+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:33.924+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.125+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.126+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.126+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.126+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.126+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:01:34.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.527+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.527+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.527+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.527+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:34.528+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:01:35.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.329+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.330+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:35.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.330+0000 7fab5bfff640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:35.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.330+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:35.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.330+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:01:35.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.951+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706218807 0 0) 0x7fab5c06a6f0 con 0x7fab400036a0 2023-10-21T16:01:35.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.951+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab4c003520 con 0x7fab400036a0 2023-10-21T16:01:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.953+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987636692 0 0) 0x7fab4c003520 con 0x7fab400036a0 2023-10-21T16:01:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 msgr2=0x7fab400097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab5b7fe640 1 --2- 172.21.15.53:0/1149712532 >> v2:172.21.15.53:3300/0 conn(0x7fab40007330 0x7fab400097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4000aef0 con 0x7fab400036a0 2023-10-21T16:01:35.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fab5000d580 con 0x7fab400036a0 2023-10-21T16:01:35.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 >> v1:172.21.15.53:6789/0 conn(0x7fab400036a0 legacy=0x7fab40005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.954+0000 7fab61572640 1 -- 172.21.15.53:0/1149712532 reap_dead start 2023-10-21T16:01:35.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 0x7fab4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c0047c0 0x7fab4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 0x7fab4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06b710 con 0x7fab4c0039c0 2023-10-21T16:01:35.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c00c240 con 0x7fab4c005380 2023-10-21T16:01:35.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.955+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c00c400 con 0x7fab4c0047c0 2023-10-21T16:01:35.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 0x7fab4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 0x7fab4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c0047c0 0x7fab4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c0047c0 msgr2=0x7fab4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:35.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c0047c0 0x7fab4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 msgr2=0x7fab4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:35.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 0x7fab4c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab60d71640 1 -- 172.21.15.53:0/1149712532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c003520 con 0x7fab4c0039c0 2023-10-21T16:01:35.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.956+0000 7fab61572640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 0x7fab4c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:01:35.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.958+0000 7fab60d71640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 0x7fab4c003dc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25492ce0b91e83f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:35.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.958+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5002d5b0 con 0x7fab4c0039c0 2023-10-21T16:01:35.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.958+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50031070 con 0x7fab4c0039c0 2023-10-21T16:01:35.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.958+0000 7fab5b7fe640 1 -- 172.21.15.53:0/1149712532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5002d950 con 0x7fab4c0039c0 2023-10-21T16:01:35.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 -- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 msgr2=0x7fab4c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:35.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 0x7fab4c003dc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab50006fd0 tx=0x7fab4c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 -- 172.21.15.53:0/1149712532 shutdown_connections 2023-10-21T16:01:35.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c005380 0x7fab4c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c0047c0 0x7fab4c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1149712532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0039c0 0x7fab4c003dc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.960+0000 7fab62fd5640 1 -- 172.21.15.53:0/1149712532 >> 172.21.15.53:0/1149712532 conn(0x7fab5c0b0ae0 msgr2=0x7fab5c0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:35.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.961+0000 7fab62fd5640 1 -- 172.21.15.53:0/1149712532 shutdown_connections 2023-10-21T16:01:35.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.961+0000 7fab62fd5640 1 -- 172.21.15.53:0/1149712532 wait complete. 2023-10-21T16:01:35.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.965+0000 7fab62fd5640 1 Processor -- start 2023-10-21T16:01:35.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.966+0000 7fab62fd5640 1 -- start start 2023-10-21T16:01:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.969+0000 7fab62fd5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c0039c0 0x7fab5c0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.969+0000 7fab62fd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.969+0000 7fab62fd5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:35.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.969+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41024/0 (socket says 172.21.15.53:41024) 2023-10-21T16:01:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 -- 172.21.15.53:0/1422314515 learned_addr learned my addr 172.21.15.53:0/1422314515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab60d71640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c0039c0 0x7fab5c0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b35f0 con 0x7fab4c0047c0 2023-10-21T16:01:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0c8fe0 con 0x7fab4c0039c0 2023-10-21T16:01:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b2f30 con 0x7fab4c005380 2023-10-21T16:01:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab61572640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab61572640 1 -- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 msgr2=0x7fab5c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-21T16:01:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab61572640 1 -- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 msgr2=0x7fab5c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab61572640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 -- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 msgr2=0x7fab5c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab61572640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 -- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c0039c0 msgr2=0x7fab5c0c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c0039c0 0x7fab5c0c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.970+0000 7fab5bfff640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0b3210 con 0x7fab4c0047c0 2023-10-21T16:01:35.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab5bfff640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab480077f0 tx=0x7fab48001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48017840 con 0x7fab4c0047c0 2023-10-21T16:01:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab480179a0 con 0x7fab4c0047c0 2023-10-21T16:01:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48017d30 con 0x7fab4c0047c0 2023-10-21T16:01:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0c92c0 con 0x7fab4c0047c0 2023-10-21T16:01:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.971+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0c69c0 con 0x7fab4c0047c0 2023-10-21T16:01:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.972+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fab480163c0 con 0x7fab4c0047c0 2023-10-21T16:01:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.972+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fab48015700 con 0x7fab4c0047c0 2023-10-21T16:01:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.973+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c04e160 con 0x7fab4c0047c0 2023-10-21T16:01:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:35.976+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fab48040b40 con 0x7fab4c0047c0 2023-10-21T16:01:36.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.050+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fab5c065c60 con 0x7fab4c0047c0 2023-10-21T16:01:36.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.054+0000 7fab597fa640 1 -- 172.21.15.53:0/1422314515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fab4802f070 con 0x7fab4c0047c0 2023-10-21T16:01:36.056 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2023-10-21T16:01:36.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 msgr2=0x7fab5c0c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:36.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab480077f0 tx=0x7fab48001630 comp rx=0 tx=0).stop 2023-10-21T16:01:36.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 shutdown_connections 2023-10-21T16:01:36.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab4c005380 0x7fab5c0c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:36.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab4c0047c0 0x7fab5c0c5030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:36.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 --2- 172.21.15.53:0/1422314515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab4c0039c0 0x7fab5c0c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:36.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 >> 172.21.15.53:0/1422314515 conn(0x7fab5c0b0ae0 msgr2=0x7fab5c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:36.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 shutdown_connections 2023-10-21T16:01:36.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:36.055+0000 7fab62fd5640 1 -- 172.21.15.53:0/1422314515 wait complete. 2023-10-21T16:01:36.076 INFO:tasks.ceph:check_enable_crimson: False 2023-10-21T16:01:36.076 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-21T16:01:36.077 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:36.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-21T16:01:36.163 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:36.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-21T16:01:36.301 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:36.301 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-21T16:01:36.389 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:01:36.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-21T16:01:36.481 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:36.482 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-21T16:01:36.499 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:36.496+0000 7f0ea5241040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:01:36.541 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:36.540+0000 7f48ed97b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:01:36.561 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:36.559+0000 7f0ea5241040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:01:36.574 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:36.575 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-21T16:01:36.603 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:36.602+0000 7f48ed97b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:01:36.702 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:36.702 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-21T16:01:36.778 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:01:36.778 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-21T16:01:36.899 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 4a16f7f4-3a30-462a-8bf0-41e8182c027c 0 2023-10-21T16:01:37.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.131+0000 7f5f7cde2640 1 Processor -- start 2023-10-21T16:01:37.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.131+0000 7f5f7cde2640 1 -- start start 2023-10-21T16:01:37.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f7cde2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f7cde2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f780cda40 0x7f5f780cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f7cde2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f780cc9b0 con 0x7f5f780b1600 2023-10-21T16:01:37.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f7cde2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f780cdf10 con 0x7f5f780cda40 2023-10-21T16:01:37.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f7cde2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780cd1c0 con 0x7f5f7815ce70 2023-10-21T16:01:37.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f77fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f780cda40 0x7f5f780cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:38176/0 (socket says 172.21.15.124:38176) 2023-10-21T16:01:37.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.132+0000 7f5f777fe640 1 -- 172.21.15.124:0/666768457 learned_addr learned my addr 172.21.15.124:0/666768457 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:37.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f777fe640 1 -- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f7806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-21T16:01:37.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f777fe640 1 -- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f7806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:37.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f777fe640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:37.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f777fe640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:37.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f77fff640 1 -- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f7806c520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:37.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f77fff640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f77fff640 1 -- 172.21.15.124:0/666768457 >> v1:172.21.15.53:6789/0 conn(0x7f5f780b1600 legacy=0x7f5f780cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.139 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.133+0000 7f5f77fff640 1 -- 172.21.15.124:0/666768457 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f780ca6c0 con 0x7f5f780cda40 2023-10-21T16:01:37.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.134+0000 7f5f77fff640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780cda40 0x7f5f780cab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009d90 tx=0x7f5f7002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78edf790baa3d0a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.134+0000 7f5f767fc640 1 -- 172.21.15.124:0/666768457 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7002cc50 con 0x7f5f780cda40 2023-10-21T16:01:37.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.134+0000 7f5f767fc640 1 -- 172.21.15.124:0/666768457 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7002cdb0 con 0x7f5f780cda40 2023-10-21T16:01:37.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.134+0000 7f5f767fc640 1 -- 172.21.15.124:0/666768457 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70034660 con 0x7f5f780cda40 2023-10-21T16:01:37.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 -- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780cda40 msgr2=0x7f5f780cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780cda40 0x7f5f780cab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009d90 tx=0x7f5f7002bea0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 -- 172.21.15.124:0/666768457 shutdown_connections 2023-10-21T16:01:37.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780cda40 0x7f5f780cab50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/666768457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f7806c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 -- 172.21.15.124:0/666768457 >> 172.21.15.124:0/666768457 conn(0x7f5f780b08d0 msgr2=0x7f5f780b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 -- 172.21.15.124:0/666768457 shutdown_connections 2023-10-21T16:01:37.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.135+0000 7f5f7cde2640 1 -- 172.21.15.124:0/666768457 wait complete. 2023-10-21T16:01:37.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.137+0000 7f5f7cde2640 1 Processor -- start 2023-10-21T16:01:37.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.137+0000 7f5f7cde2640 1 -- start start 2023-10-21T16:01:37.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f780cda40 0x7f5f781f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780b4330 con 0x7f5f780cda40 2023-10-21T16:01:37.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f78158bf0 con 0x7f5f780b1600 2023-10-21T16:01:37.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f7cde2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780b3460 con 0x7f5f7815ce70 2023-10-21T16:01:37.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f777fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:46942/0 (socket says 172.21.15.124:46942) 2023-10-21T16:01:37.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f777fe640 1 -- 172.21.15.124:0/214950415 learned_addr learned my addr 172.21.15.124:0/214950415 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f77fff640 1 -- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f781f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f77fff640 1 -- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f781f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f77fff640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.138+0000 7f5f76ffd640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f780cda40 0x7f5f781f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f77fff640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:37.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 -- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 msgr2=0x7f5f781f62a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:37.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 -- 172.21.15.124:0/214950415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f780cda40 msgr2=0x7f5f781f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f780cda40 0x7f5f781f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 -- 172.21.15.124:0/214950415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70009a70 con 0x7f5f780b1600 2023-10-21T16:01:37.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f777fe640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e870 tx=0x7f5f6800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004220 con 0x7f5f780b1600 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f68004380 con 0x7f5f780b1600 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800c910 con 0x7f5f780b1600 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.139+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f780b36b0 con 0x7f5f780b1600 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.140+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f780b3c70 con 0x7f5f780b1600 2023-10-21T16:01:37.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.140+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5f6800cd70 con 0x7f5f780b1600 2023-10-21T16:01:37.150 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.140+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5f68003e20 con 0x7f5f780b1600 2023-10-21T16:01:37.150 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.141+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f78074150 con 0x7f5f780b1600 2023-10-21T16:01:37.151 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.146+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5f68020060 con 0x7f5f780b1600 2023-10-21T16:01:37.217 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.215+0000 7f0ea5241040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:01:37.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.224+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4a16f7f4-3a30-462a-8bf0-41e8182c027c", "id": 0} v 0) v1 -- 0x7f5f780b4d60 con 0x7f5f780b1600 2023-10-21T16:01:37.233 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.231+0000 7f5f5ffff640 1 -- 172.21.15.124:0/214950415 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a16f7f4-3a30-462a-8bf0-41e8182c027c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f680290b0 con 0x7f5f780b1600 2023-10-21T16:01:37.233 INFO:teuthology.orchestra.run.smithi124.stdout:0 2023-10-21T16:01:37.235 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 msgr2=0x7f5f7809f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.235 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800e870 tx=0x7f5f6800ed30 comp rx=0 tx=0).stop 2023-10-21T16:01:37.235 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 shutdown_connections 2023-10-21T16:01:37.236 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7815ce70 0x7f5f781f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.236 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f780cda40 0x7f5f781f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.236 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 --2- 172.21.15.124:0/214950415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f780b1600 0x7f5f7809f120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.237 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 >> 172.21.15.124:0/214950415 conn(0x7f5f780b08d0 msgr2=0x7f5f780caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.237 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 shutdown_connections 2023-10-21T16:01:37.237 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.232+0000 7f5f7cde2640 1 -- 172.21.15.124:0/214950415 wait complete. 2023-10-21T16:01:37.252 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new c97255a9-80ac-47bc-8525-410c20b486ed 1 2023-10-21T16:01:37.255 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.253+0000 7f48ed97b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:01:37.290 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.288+0000 7f0ea5241040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:01:37.320 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.318+0000 7f48ed97b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:01:37.345 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.343+0000 7f0ea5241040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:01:37.373 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.371+0000 7f48ed97b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:01:37.423 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.421+0000 7f0ea5241040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:01:37.437 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.435+0000 7f1d83773640 1 Processor -- start 2023-10-21T16:01:37.437 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.435+0000 7f1d83773640 1 -- start start 2023-10-21T16:01:37.438 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.436+0000 7f1d83773640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.439 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.436+0000 7f1d83773640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d7c06db60 0x7f1d7c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.439 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.436+0000 7f1d83773640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c15b310 con 0x7f1d7c15aa00 2023-10-21T16:01:37.439 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.436+0000 7f1d83773640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0ca230 con 0x7f1d7c06db60 2023-10-21T16:01:37.439 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.436+0000 7f1d83773640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c06d290 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.440 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.437+0000 7f48ed97b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:01:37.442 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.442 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:46948/0 (socket says 172.21.15.124:46948) 2023-10-21T16:01:37.443 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 -- 172.21.15.124:0/2613009297 learned_addr learned my addr 172.21.15.124:0/2613009297 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:37.443 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d80d0e640 1 --2- 172.21.15.124:0/2613009297 >> v2:172.21.15.53:3300/0 conn(0x7f1d7c06db60 0x7f1d7c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.443 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 -- 172.21.15.124:0/2613009297 >> v2:172.21.15.53:3300/0 conn(0x7f1d7c06db60 msgr2=0x7f1d7c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:37.443 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 --2- 172.21.15.124:0/2613009297 >> v2:172.21.15.53:3300/0 conn(0x7f1d7c06db60 0x7f1d7c06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.444 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 -- 172.21.15.124:0/2613009297 >> v1:172.21.15.53:6789/0 conn(0x7f1d7c15aa00 legacy=0x7f1d7c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.444 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.437+0000 7f1d8150f640 1 -- 172.21.15.124:0/2613009297 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0c9860 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.444 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.438+0000 7f1d8150f640 1 --2- 172.21.15.124:0/2613009297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0716b0 tx=0x7f1d7802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4931a4718fb47a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.444 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d73fff640 1 -- 172.21.15.124:0/2613009297 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78030860 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.444 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d73fff640 1 -- 172.21.15.124:0/2613009297 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d780309c0 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.445 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d73fff640 1 -- 172.21.15.124:0/2613009297 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78030d50 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.445 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 -- 172.21.15.124:0/2613009297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 msgr2=0x7f1d7c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.445 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 --2- 172.21.15.124:0/2613009297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0716b0 tx=0x7f1d7802b9a0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.445 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 -- 172.21.15.124:0/2613009297 shutdown_connections 2023-10-21T16:01:37.445 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 --2- 172.21.15.124:0/2613009297 >> v2:172.21.15.53:3300/0 conn(0x7f1d7c06db60 0x7f1d7c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.446 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 --2- 172.21.15.124:0/2613009297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c0c9cf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.446 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 -- 172.21.15.124:0/2613009297 >> 172.21.15.124:0/2613009297 conn(0x7f1d7c06a5d0 msgr2=0x7f1d7c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.446 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.439+0000 7f1d83773640 1 -- 172.21.15.124:0/2613009297 shutdown_connections 2023-10-21T16:01:37.446 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.440+0000 7f1d83773640 1 -- 172.21.15.124:0/2613009297 wait complete. 2023-10-21T16:01:37.449 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.442+0000 7f1d83773640 1 Processor -- start 2023-10-21T16:01:37.449 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.442+0000 7f1d83773640 1 -- start start 2023-10-21T16:01:37.449 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.450 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 0x7f1d7c0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.450 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.450 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c4570 con 0x7f1d7c06db60 2023-10-21T16:01:37.450 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c4850 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.451 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d83773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0c4b30 con 0x7f1d7c15aa00 2023-10-21T16:01:37.451 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d8150f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.451 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d81d10640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.451 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d8150f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:46962/0 (socket says 172.21.15.124:46962) 2023-10-21T16:01:37.452 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d8150f640 1 -- 172.21.15.124:0/386177174 learned_addr learned my addr 172.21.15.124:0/386177174 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:37.452 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d81d10640 1 -- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 msgr2=0x7f1d7c0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-21T16:01:37.452 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d80d0e640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 0x7f1d7c0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.452 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d81d10640 1 -- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 msgr2=0x7f1d7c0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:37.452 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d81d10640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:37.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.444+0000 7f1d81d10640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:37.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 -- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 msgr2=0x7f1d7c0c7610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:37.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 -- 172.21.15.124:0/386177174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 msgr2=0x7f1d7c0c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 0x7f1d7c0c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 -- 172.21.15.124:0/386177174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780094d0 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d80d0e640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 0x7f1d7c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:01:37.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d8150f640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d7802b9e0 tx=0x7f1d78004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78039d60 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78043400 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780437b0 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c0c4e10 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.445+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1f2470 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.446+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d78043c10 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.456 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.446+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1d7803e070 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.456 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.447+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c04e160 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.456 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.449+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1d78034070 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.479 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.477+0000 7f0ea5241040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:01:37.490 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.489+0000 7f48ed97b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:01:37.528 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.526+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c97255a9-80ac-47bc-8525-410c20b486ed", "id": 1} v 0) v1 -- 0x7f1d7c15b4b0 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.534 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.532+0000 7f1d71ffb640 1 -- 172.21.15.124:0/386177174 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c97255a9-80ac-47bc-8525-410c20b486ed", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1d7804d460 con 0x7f1d7c06c3a0 2023-10-21T16:01:37.534 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-21T16:01:37.535 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 msgr2=0x7f1d7c1f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.535 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d7802b9e0 tx=0x7f1d78004730 comp rx=0 tx=0).stop 2023-10-21T16:01:37.536 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 shutdown_connections 2023-10-21T16:01:37.536 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d7c15aa00 0x7f1d7c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.536 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d7c06db60 0x7f1d7c0c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.536 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 --2- 172.21.15.124:0/386177174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d7c06c3a0 0x7f1d7c1f1f30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.536 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 >> 172.21.15.124:0/386177174 conn(0x7f1d7c06a5d0 msgr2=0x7f1d7c0cb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.537 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 shutdown_connections 2023-10-21T16:01:37.537 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.533+0000 7f1d83773640 1 -- 172.21.15.124:0/386177174 wait complete. 2023-10-21T16:01:37.550 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new f35949b0-800b-4e69-a1e7-aeeb07bc546a 2 2023-10-21T16:01:37.600 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.598+0000 7f0ea5241040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:01:37.609 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.608+0000 7f48ed97b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:01:37.684 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.682+0000 7f0ea5241040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:01:37.747 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.692+0000 7f48ed97b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:01:37.747 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.739+0000 7f0ea5241040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:01:37.749 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.747+0000 7f48ed97b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:01:37.781 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.779+0000 7f4478e0d640 1 Processor -- start 2023-10-21T16:01:37.781 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.779+0000 7f4478e0d640 1 -- start start 2023-10-21T16:01:37.782 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f4478e0d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f44740cf690 0x7f44740d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.782 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f4478e0d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740cd520 con 0x7f44740ce510 2023-10-21T16:01:37.783 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f4478e0d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740d1910 con 0x7f44740cf2b0 2023-10-21T16:01:37.783 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f4478e0d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740ce080 con 0x7f44740cf690 2023-10-21T16:01:37.783 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f44737fe640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f44740cf2b0 0x7f44740d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:57668/0 (socket says 172.21.15.124:57668) 2023-10-21T16:01:37.783 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.780+0000 7f44737fe640 1 -- 172.21.15.124:0/3331987899 learned_addr learned my addr 172.21.15.124:0/3331987899 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-21T16:01:37.785 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3331987899 >> v2:172.21.15.53:3300/0 conn(0x7f44740cf690 0x7f44740d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.785 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725679812 0 0) 0x7f44740d1910 con 0x7f44740cf2b0 2023-10-21T16:01:37.786 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f445c003520 con 0x7f44740cf2b0 2023-10-21T16:01:37.786 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784827617 0 0) 0x7f445c003520 con 0x7f44740cf2b0 2023-10-21T16:01:37.786 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 >> v2:172.21.15.53:3300/0 conn(0x7f44740cf690 msgr2=0x7f44740d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:37.786 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 --2- 172.21.15.124:0/3331987899 >> v2:172.21.15.53:3300/0 conn(0x7f44740cf690 0x7f44740d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.787 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 >> v1:172.21.15.53:6789/0 conn(0x7f44740ce510 legacy=0x7f447415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.787 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740cd750 con 0x7f44740cf2b0 2023-10-21T16:01:37.787 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4464001670 con 0x7f44740cf2b0 2023-10-21T16:01:37.787 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.781+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 >> v1:172.21.15.124:6789/0 conn(0x7f44740cf2b0 legacy=0x7f44740d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.787 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c003970 0x7f445c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.788 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 0x7f445c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.788 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c005180 0x7f445c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.788 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740ce080 con 0x7f445c005180 2023-10-21T16:01:37.788 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00c120 con 0x7f445c0047c0 2023-10-21T16:01:37.788 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00c2e0 con 0x7f445c003970 2023-10-21T16:01:37.789 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44737fe640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c003970 0x7f445c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.789 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 0x7f445c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.789 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4473fff640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c005180 0x7f445c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.789 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 -- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c003970 msgr2=0x7f445c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:37.790 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c003970 0x7f445c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.790 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 -- 172.21.15.124:0/3331987899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c005180 msgr2=0x7f445c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:37.790 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c005180 0x7f445c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.790 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f4472ffd640 1 -- 172.21.15.124:0/3331987899 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c003520 con 0x7f445c0047c0 2023-10-21T16:01:37.791 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.782+0000 7f44737fe640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c003970 0x7f445c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:01:37.791 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.783+0000 7f4473fff640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f445c005180 0x7f445c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:37.791 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.783+0000 7f4473fff640 1 -- 172.21.15.124:0/3331987899 reap_dead start 2023-10-21T16:01:37.791 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.783+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 0x7f445c004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4468009ac0 tx=0x7f445c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c534aa82f73b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.794 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.784+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446802c6f0 con 0x7f445c0047c0 2023-10-21T16:01:37.795 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.784+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446802c850 con 0x7f445c0047c0 2023-10-21T16:01:37.795 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.784+0000 7f44727fc640 1 -- 172.21.15.124:0/3331987899 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446802cbe0 con 0x7f445c0047c0 2023-10-21T16:01:37.795 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.784+0000 7f4478e0d640 1 -- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 msgr2=0x7f445c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.795 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.784+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 0x7f445c004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4468009ac0 tx=0x7f445c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.795 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.785+0000 7f4478e0d640 1 -- 172.21.15.124:0/3331987899 shutdown_connections 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.785+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3331987899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f445c0047c0 0x7f445c004c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.785+0000 7f4478e0d640 1 -- 172.21.15.124:0/3331987899 >> 172.21.15.124:0/3331987899 conn(0x7f447409bc40 msgr2=0x7f44740d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.785+0000 7f4478e0d640 1 -- 172.21.15.124:0/3331987899 shutdown_connections 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.785+0000 7f4478e0d640 1 -- 172.21.15.124:0/3331987899 wait complete. 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.787+0000 7f4478e0d640 1 Processor -- start 2023-10-21T16:01:37.796 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.787+0000 7f4478e0d640 1 -- start start 2023-10-21T16:01:37.797 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.797 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44740b1ab0 0x7f44740b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.797 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:37.797 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474150860 con 0x7f44740b1ab0 2023-10-21T16:01:37.797 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b63c0 con 0x7f44740b5f40 2023-10-21T16:01:37.798 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4478e0d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b6f80 con 0x7f445c0047c0 2023-10-21T16:01:37.798 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f44737fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.798 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4473fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.798 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4473fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:46980/0 (socket says 172.21.15.124:46980) 2023-10-21T16:01:37.798 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4473fff640 1 -- 172.21.15.124:0/3569522168 learned_addr learned my addr 172.21.15.124:0/3569522168 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:37.799 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f4472ffd640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44740b1ab0 0x7f44740b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:37.799 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f44737fe640 1 -- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 msgr2=0x7f44741f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-21T16:01:37.799 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.788+0000 7f44737fe640 1 -- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 msgr2=0x7f44741f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:37.799 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f44737fe640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:37.800 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f44737fe640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:37.800 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 -- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 msgr2=0x7f44741f67d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:37.800 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.800 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 -- 172.21.15.124:0/3569522168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44740b1ab0 msgr2=0x7f44740b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.800 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44740b1ab0 0x7f44740b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.801 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 -- 172.21.15.124:0/3569522168 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740b7260 con 0x7f44740b5f40 2023-10-21T16:01:37.801 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4473fff640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4474160830 tx=0x7f446c00d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:37.801 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c003d80 con 0x7f44740b5f40 2023-10-21T16:01:37.801 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c005070 con 0x7f44740b5f40 2023-10-21T16:01:37.801 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c005420 con 0x7f44740b5f40 2023-10-21T16:01:37.802 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.789+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44740b7540 con 0x7f44740b5f40 2023-10-21T16:01:37.802 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.790+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44740b7b00 con 0x7f44740b5f40 2023-10-21T16:01:37.802 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.790+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f446c005580 con 0x7f44740b5f40 2023-10-21T16:01:37.802 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.790+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f446c0123d0 con 0x7f44740b5f40 2023-10-21T16:01:37.802 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.791+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44740cdcc0 con 0x7f44740b5f40 2023-10-21T16:01:37.803 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.793+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f446c018070 con 0x7f44740b5f40 2023-10-21T16:01:37.807 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.805+0000 7f0ea5241040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:01:37.812 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.811+0000 7f48ed97b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:01:37.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:37.864+0000 7f0ea5241040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:01:37.869 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:37.868+0000 7f48ed97b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:01:37.877 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.875+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f35949b0-800b-4e69-a1e7-aeeb07bc546a", "id": 2} v 0) v1 -- 0x7f4474074150 con 0x7f44740b5f40 2023-10-21T16:01:37.882 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.880+0000 7f445bfff640 1 -- 172.21.15.124:0/3569522168 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f35949b0-800b-4e69-a1e7-aeeb07bc546a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4464007ce0 con 0x7f44740b5f40 2023-10-21T16:01:37.882 INFO:teuthology.orchestra.run.smithi124.stdout:2 2023-10-21T16:01:37.884 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 msgr2=0x7f4474153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:37.884 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4474160830 tx=0x7f446c00d9d0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.884 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 shutdown_connections 2023-10-21T16:01:37.885 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44740b5f40 0x7f4474153f00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.885 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44740b1ab0 0x7f44740b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.885 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 --2- 172.21.15.124:0/3569522168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f445c0047c0 0x7f44741f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:37.885 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 >> 172.21.15.124:0/3569522168 conn(0x7f447409bc40 msgr2=0x7f44740b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:37.885 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 shutdown_connections 2023-10-21T16:01:37.886 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:37.882+0000 7f4478e0d640 1 -- 172.21.15.124:0/3569522168 wait complete. 2023-10-21T16:01:37.899 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 2a33ca07-a7e3-4e34-9c42-35cafc6374f6 3 2023-10-21T16:01:38.124 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.122+0000 7f197c341640 1 Processor -- start 2023-10-21T16:01:38.125 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.122+0000 7f197c341640 1 -- start start 2023-10-21T16:01:38.129 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197c341640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f19741612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.130 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197c341640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19740b4e10 0x7f19740d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.130 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197c341640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19740b8c30 con 0x7f19740cd4e0 2023-10-21T16:01:38.130 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197c341640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19740cdd80 con 0x7f19740b4e10 2023-10-21T16:01:38.131 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197c341640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19740ce060 con 0x7f197415cd70 2023-10-21T16:01:38.131 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a0dd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f19741612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.131 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f19798dc640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f19740cd4e0 0x7f19740b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:57674/0 (socket says 172.21.15.124:57674) 2023-10-21T16:01:38.131 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f19798dc640 1 -- 172.21.15.124:0/1223238128 learned_addr learned my addr 172.21.15.124:0/1223238128 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-21T16:01:38.132 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a8de640 1 --2- 172.21.15.124:0/1223238128 >> v2:172.21.15.53:3300/0 conn(0x7f19740b4e10 0x7f19740d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.132 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a0dd640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 msgr2=0x7f19741612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-21T16:01:38.132 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a0dd640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 msgr2=0x7f19741612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:38.132 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a0dd640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f19741612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.132 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f197a0dd640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f19741612f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.124+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772417925 0 0) 0x7f19740b8c30 con 0x7f19740cd4e0 2023-10-21T16:01:38.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1960003520 con 0x7f19740cd4e0 2023-10-21T16:01:38.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263648056 0 0) 0x7f1960003520 con 0x7f19740cd4e0 2023-10-21T16:01:38.133 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 msgr2=0x7f19741612f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:38.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f19741612f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 >> v2:172.21.15.53:3300/0 conn(0x7f19740b4e10 msgr2=0x7f19740d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:38.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 --2- 172.21.15.124:0/1223238128 >> v2:172.21.15.53:3300/0 conn(0x7f19740b4e10 0x7f19740d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740ce340 con 0x7f19740cd4e0 2023-10-21T16:01:38.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1964001690 con 0x7f19740cd4e0 2023-10-21T16:01:38.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 >> v1:172.21.15.124:6789/0 conn(0x7f19740cd4e0 legacy=0x7f19740b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f1960003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1960003f40 0x7f19600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.135 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f1960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740cdd80 con 0x7f1960003f40 2023-10-21T16:01:38.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740ce060 con 0x7f1960004d00 2023-10-21T16:01:38.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196000ba30 con 0x7f197415cd70 2023-10-21T16:01:38.136 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f197a0dd640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f1960003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.125+0000 7f197a8de640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f1960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f19798dc640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1960003f40 0x7f19600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f197a8de640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 msgr2=0x7f1960003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:38.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f197a8de640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415cd70 0x7f1960003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.137 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f197a8de640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1960003f40 msgr2=0x7f19600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f197a8de640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1960003f40 0x7f19600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.126+0000 7f197a8de640 1 -- 172.21.15.124:0/1223238128 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960003520 con 0x7f1960004d00 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.127+0000 7f197a8de640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f1960005180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fc0 tx=0x7f19600057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d45c17a28656de server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.127+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c02cc40 con 0x7f1960004d00 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.127+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c00f040 con 0x7f1960004d00 2023-10-21T16:01:38.138 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.127+0000 7f19790db640 1 -- 172.21.15.124:0/1223238128 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c034440 con 0x7f1960004d00 2023-10-21T16:01:38.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 -- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 msgr2=0x7f1960005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 --2- 172.21.15.124:0/1223238128 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f1960005180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fc0 tx=0x7f19600057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.142 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197a8de640 1 -- 172.21.15.124:0/1223238128 reap_dead start 2023-10-21T16:01:38.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 -- 172.21.15.124:0/1223238128 shutdown_connections 2023-10-21T16:01:38.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 -- 172.21.15.124:0/1223238128 >> 172.21.15.124:0/1223238128 conn(0x7f19740cf700 msgr2=0x7f19740cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 -- 172.21.15.124:0/1223238128 shutdown_connections 2023-10-21T16:01:38.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.128+0000 7f197c341640 1 -- 172.21.15.124:0/1223238128 wait complete. 2023-10-21T16:01:38.143 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.131+0000 7f197c341640 1 Processor -- start 2023-10-21T16:01:38.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.131+0000 7f197c341640 1 -- start start 2023-10-21T16:01:38.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f197414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 0x7f19740c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.144 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197415b8a0 con 0x7f19740c8070 2023-10-21T16:01:38.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197414a6d0 con 0x7f1960004d00 2023-10-21T16:01:38.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197c341640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19740c8ac0 con 0x7f197415aee0 2023-10-21T16:01:38.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a0dd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f197414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:38260/0 (socket says 172.21.15.124:38260) 2023-10-21T16:01:38.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 -- 172.21.15.124:0/2829263865 learned_addr learned my addr 172.21.15.124:0/2829263865 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:38.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f19798dc640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 0x7f19740c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 -- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 msgr2=0x7f197415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2023-10-21T16:01:38.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 -- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 msgr2=0x7f197415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:38.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a8de640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a0dd640 1 -- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 msgr2=0x7f197415b360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:38.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.133+0000 7f197a0dd640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197a0dd640 1 -- 172.21.15.124:0/2829263865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 msgr2=0x7f19740c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197a0dd640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 0x7f19740c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197a0dd640 1 -- 172.21.15.124:0/2829263865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740c8da0 con 0x7f1960004d00 2023-10-21T16:01:38.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f19798dc640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 0x7f19740c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:01:38.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197a0dd640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f197414a190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f19700108c0 tx=0x7f1970010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000e9c0 con 0x7f1960004d00 2023-10-21T16:01:38.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000eb20 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740c9080 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197000eeb0 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.134+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1974157970 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.135+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1970016780 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.135+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1970014750 con 0x7f1960004d00 2023-10-21T16:01:38.149 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.135+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974157b60 con 0x7f1960004d00 2023-10-21T16:01:38.150 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.139+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f197003eb40 con 0x7f1960004d00 2023-10-21T16:01:38.219 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.216+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a33ca07-a7e3-4e34-9c42-35cafc6374f6", "id": 3} v 0) v1 -- 0x7f197404e160 con 0x7f1960004d00 2023-10-21T16:01:38.224 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.222+0000 7f196affd640 1 -- 172.21.15.124:0/2829263865 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a33ca07-a7e3-4e34-9c42-35cafc6374f6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f197002d070 con 0x7f1960004d00 2023-10-21T16:01:38.224 INFO:teuthology.orchestra.run.smithi124.stdout:3 2023-10-21T16:01:38.225 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 msgr2=0x7f197414a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f197414a190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f19700108c0 tx=0x7f1970010d80 comp rx=0 tx=0).stop 2023-10-21T16:01:38.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 shutdown_connections 2023-10-21T16:01:38.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f197415aee0 0x7f197415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19740c8070 0x7f19740c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.226 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 --2- 172.21.15.124:0/2829263865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1960004d00 0x7f197414a190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.227 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 >> 172.21.15.124:0/2829263865 conn(0x7f19740cf700 msgr2=0x7f19740b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.227 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 shutdown_connections 2023-10-21T16:01:38.227 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.223+0000 7f197c341640 1 -- 172.21.15.124:0/2829263865 wait complete. 2023-10-21T16:01:38.241 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 5c45510d-a729-470e-bca4-4a10297aeb06 4 2023-10-21T16:01:38.344 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.342+0000 7f48ed97b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:01:38.348 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.346+0000 7f0ea5241040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:01:38.419 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.417+0000 7f86a4333640 1 Processor -- start 2023-10-21T16:01:38.419 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.417+0000 7f86a4333640 1 -- start start 2023-10-21T16:01:38.421 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.418+0000 7f86a4333640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f869c15cd70 0x7f869c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.421 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.418+0000 7f86a4333640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.422 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.418+0000 7f86a4333640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c0b8c30 con 0x7f869c0cd4e0 2023-10-21T16:01:38.422 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.418+0000 7f86a4333640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c0cdd80 con 0x7f869c15cd70 2023-10-21T16:01:38.422 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.418+0000 7f86a4333640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0ce060 con 0x7f869c0b4e10 2023-10-21T16:01:38.422 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f869c15cd70 0x7f869c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f869c15cd70 0x7f869c1612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:47000/0 (socket says 172.21.15.124:47000) 2023-10-21T16:01:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a28d0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 -- 172.21.15.124:0/4115793699 learned_addr learned my addr 172.21.15.124:0/4115793699 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a28d0640 1 -- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-21T16:01:38.423 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a28d0640 1 -- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a28d0640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a28d0640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 -- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c0d1650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 -- 172.21.15.124:0/4115793699 >> v1:172.21.15.53:6789/0 conn(0x7f869c0cd4e0 legacy=0x7f869c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.424 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.419+0000 7f86a20cf640 1 -- 172.21.15.124:0/4115793699 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0ce340 con 0x7f869c15cd70 2023-10-21T16:01:38.426 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.420+0000 7f86a20cf640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c1612f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8698009870 tx=0x7f869802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770077f4a3868c7c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.426 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.420+0000 7f86a10cd640 1 -- 172.21.15.124:0/4115793699 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698030840 con 0x7f869c15cd70 2023-10-21T16:01:38.426 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a10cd640 1 -- 172.21.15.124:0/4115793699 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86980309a0 con 0x7f869c15cd70 2023-10-21T16:01:38.427 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a10cd640 1 -- 172.21.15.124:0/4115793699 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698030d30 con 0x7f869c15cd70 2023-10-21T16:01:38.427 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 -- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 msgr2=0x7f869c1612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.427 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c1612f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8698009870 tx=0x7f869802b980 comp rx=0 tx=0).stop 2023-10-21T16:01:38.427 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 -- 172.21.15.124:0/4115793699 shutdown_connections 2023-10-21T16:01:38.427 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.428 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 --2- 172.21.15.124:0/4115793699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c1612f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.428 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 -- 172.21.15.124:0/4115793699 >> 172.21.15.124:0/4115793699 conn(0x7f869c0cf700 msgr2=0x7f869c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.428 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 -- 172.21.15.124:0/4115793699 shutdown_connections 2023-10-21T16:01:38.428 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.421+0000 7f86a4333640 1 -- 172.21.15.124:0/4115793699 wait complete. 2023-10-21T16:01:38.428 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.424+0000 7f86a4333640 1 Processor -- start 2023-10-21T16:01:38.429 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.424+0000 7f86a4333640 1 -- start start 2023-10-21T16:01:38.431 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.431 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f869c0cd4e0 0x7f869c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.431 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.431 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0ca430 con 0x7f869c0cd4e0 2023-10-21T16:01:38.432 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0ccba0 con 0x7f869c15cd70 2023-10-21T16:01:38.432 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.425+0000 7f86a4333640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15fe80 con 0x7f869c0b4e10 2023-10-21T16:01:38.432 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.432 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:47014/0 (socket says 172.21.15.124:47014) 2023-10-21T16:01:38.433 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a20cf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.433 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 -- 172.21.15.124:0/3623025627 learned_addr learned my addr 172.21.15.124:0/3623025627 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:38.433 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a20cf640 1 -- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-21T16:01:38.433 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a20cf640 1 -- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:38.433 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a20cf640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.434 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a18ce640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f869c0cd4e0 0x7f869c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.434 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a20cf640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.434 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 -- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 msgr2=0x7f869c1559b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:38.434 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.434 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 -- 172.21.15.124:0/3623025627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f869c0cd4e0 msgr2=0x7f869c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.435 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f869c0cd4e0 0x7f869c15d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.435 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 -- 172.21.15.124:0/3623025627 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980094d0 con 0x7f869c15cd70 2023-10-21T16:01:38.435 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.426+0000 7f86a28d0640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8694007d10 tx=0x7f869400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.435 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694014420 con 0x7f869c15cd70 2023-10-21T16:01:38.435 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694014580 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694014910 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c160160 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c1566a0 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f869401d410 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.427+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f8694014dc0 con 0x7f869c15cd70 2023-10-21T16:01:38.436 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.428+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c0ca150 con 0x7f869c15cd70 2023-10-21T16:01:38.437 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.432+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f869c0ca150 con 0x7f869c15cd70 2023-10-21T16:01:38.512 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.509+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5c45510d-a729-470e-bca4-4a10297aeb06", "id": 4} v 0) v1 -- 0x7f869c0c9d50 con 0x7f869c15cd70 2023-10-21T16:01:38.512 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.510+0000 7f48ed97b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:01:38.516 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.514+0000 7f0ea5241040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:01:38.517 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.515+0000 7f8692ffd640 1 -- 172.21.15.124:0/3623025627 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5c45510d-a729-470e-bca4-4a10297aeb06", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f869401ddd0 con 0x7f869c15cd70 2023-10-21T16:01:38.518 INFO:teuthology.orchestra.run.smithi124.stdout:4 2023-10-21T16:01:38.519 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.516+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 msgr2=0x7f869c0ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.519 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.516+0000 7f86a4333640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8694007d10 tx=0x7f869400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.519 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 shutdown_connections 2023-10-21T16:01:38.519 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c15cd70 0x7f869c0ca6a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.520 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f869c0cd4e0 0x7f869c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.520 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 --2- 172.21.15.124:0/3623025627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0b4e10 0x7f869c1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.520 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 >> 172.21.15.124:0/3623025627 conn(0x7f869c0cf700 msgr2=0x7f869c0b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.520 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 shutdown_connections 2023-10-21T16:01:38.521 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.517+0000 7f86a4333640 1 -- 172.21.15.124:0/3623025627 wait complete. 2023-10-21T16:01:38.533 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 80f7d069-f0f5-4f93-b281-46d047166231 5 2023-10-21T16:01:38.642 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.641+0000 7f48ed97b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:01:38.662 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.660+0000 7f0ea5241040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:01:38.698 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.696+0000 7f48ed97b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:01:38.716 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.714+0000 7f9c85ae0640 1 Processor -- start 2023-10-21T16:01:38.716 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.714+0000 7f9c85ae0640 1 -- start start 2023-10-21T16:01:38.717 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f0ea5241040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:01:38.718 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f9c85ae0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.719 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f9c85ae0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c800cdf70 0x7f9c800d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.719 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f9c85ae0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8015d160 con 0x7f9c800b6730 2023-10-21T16:01:38.719 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f9c85ae0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800b7b90 con 0x7f9c800cdf70 2023-10-21T16:01:38.719 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.715+0000 7f9c85ae0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800b8020 con 0x7f9c8015cd80 2023-10-21T16:01:38.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c77fff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f9c800b6730 0x7f9c800cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:57680/0 (socket says 172.21.15.124:57680) 2023-10-21T16:01:38.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c77fff640 1 -- 172.21.15.124:0/3266473609 learned_addr learned my addr 172.21.15.124:0/3266473609 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-21T16:01:38.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/3266473609 >> v2:172.21.15.124:3300/0 conn(0x7f9c800cdf70 0x7f9c800d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.720 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7f7fe640 1 -- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 msgr2=0x7f9c800b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-21T16:01:38.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7f7fe640 1 -- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 msgr2=0x7f9c800b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:38.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7f7fe640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7f7fe640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.721 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7effd640 1 -- 172.21.15.124:0/3266473609 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979430281 0 0) 0x7f9c8015d160 con 0x7f9c800b6730 2023-10-21T16:01:38.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7effd640 1 -- 172.21.15.124:0/3266473609 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c6c003520 con 0x7f9c800b6730 2023-10-21T16:01:38.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7ffff640 1 -- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 msgr2=0x7f9c800b87d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:38.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7ffff640 1 -- 172.21.15.124:0/3266473609 >> v1:172.21.15.124:6789/0 conn(0x7f9c800b6730 legacy=0x7f9c800cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.722 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.716+0000 7f9c7ffff640 1 -- 172.21.15.124:0/3266473609 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800b6f80 con 0x7f9c800cdf70 2023-10-21T16:01:38.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.717+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c800cdf70 0x7f9c800d0ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c7802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd72762f95942d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c7effd640 1 -- 172.21.15.124:0/3266473609 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802cc60 con 0x7f9c800cdf70 2023-10-21T16:01:38.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c7effd640 1 -- 172.21.15.124:0/3266473609 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7802cdc0 con 0x7f9c800cdf70 2023-10-21T16:01:38.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c7effd640 1 -- 172.21.15.124:0/3266473609 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78034660 con 0x7f9c800cdf70 2023-10-21T16:01:38.726 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 -- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c800cdf70 msgr2=0x7f9c800d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c800cdf70 0x7f9c800d0ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c7802beb0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 -- 172.21.15.124:0/3266473609 shutdown_connections 2023-10-21T16:01:38.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c800cdf70 0x7f9c800d0ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/3266473609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8015cd80 0x7f9c800b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.727 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.718+0000 7f9c85ae0640 1 -- 172.21.15.124:0/3266473609 >> 172.21.15.124:0/3266473609 conn(0x7f9c8006b5d0 msgr2=0x7f9c8006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.719+0000 7f9c85ae0640 1 -- 172.21.15.124:0/3266473609 shutdown_connections 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.719+0000 7f9c85ae0640 1 -- 172.21.15.124:0/3266473609 wait complete. 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.720+0000 7f9c85ae0640 1 Processor -- start 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.720+0000 7f9c85ae0640 1 -- start start 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.728 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c800cdf70 0x7f9c80156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.729 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:38.729 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8015f240 con 0x7f9c800cdf70 2023-10-21T16:01:38.729 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c801f68c0 con 0x7f9c8015cd80 2023-10-21T16:01:38.729 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c85ae0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8015f440 con 0x7f9c800b6730 2023-10-21T16:01:38.729 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c7f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.730 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.730 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.721+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:47028/0 (socket says 172.21.15.124:47028) 2023-10-21T16:01:38.730 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 -- 172.21.15.124:0/2606969485 learned_addr learned my addr 172.21.15.124:0/2606969485 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:38.730 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c77fff640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c800cdf70 0x7f9c80156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:38.730 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7f7fe640 1 -- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 msgr2=0x7f9c801f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-21T16:01:38.731 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7f7fe640 1 -- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 msgr2=0x7f9c801f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:38.731 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7f7fe640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:38.731 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7f7fe640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:38.731 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 -- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 msgr2=0x7f9c801f5d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:38.731 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.732 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 -- 172.21.15.124:0/2606969485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c800cdf70 msgr2=0x7f9c80156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.732 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c800cdf70 0x7f9c80156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.732 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 -- 172.21.15.124:0/2606969485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009a70 con 0x7f9c8015cd80 2023-10-21T16:01:38.732 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.722+0000 7f9c7ffff640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c78004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:38.732 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78040650 con 0x7f9c8015cd80 2023-10-21T16:01:38.733 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c780407b0 con 0x7f9c8015cd80 2023-10-21T16:01:38.733 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78040b60 con 0x7f9c8015cd80 2023-10-21T16:01:38.733 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8015f720 con 0x7f9c8015cd80 2023-10-21T16:01:38.733 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8015fce0 con 0x7f9c8015cd80 2023-10-21T16:01:38.733 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c78049400 con 0x7f9c8015cd80 2023-10-21T16:01:38.734 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.723+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9c78049c30 con 0x7f9c8015cd80 2023-10-21T16:01:38.734 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.724+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8004e160 con 0x7f9c8015cd80 2023-10-21T16:01:38.734 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.726+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9c78067b40 con 0x7f9c8015cd80 2023-10-21T16:01:38.812 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.810+0000 7f48ed97b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:01:38.820 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.818+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80f7d069-f0f5-4f93-b281-46d047166231", "id": 5} v 0) v1 -- 0x7f9c8015a1a0 con 0x7f9c8015cd80 2023-10-21T16:01:38.827 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.825+0000 7f9c7cff9640 1 -- 172.21.15.124:0/2606969485 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80f7d069-f0f5-4f93-b281-46d047166231", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c78056030 con 0x7f9c8015cd80 2023-10-21T16:01:38.828 INFO:teuthology.orchestra.run.smithi124.stdout:5 2023-10-21T16:01:38.829 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 msgr2=0x7f9c8014dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:38.829 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009d90 tx=0x7f9c78004a40 comp rx=0 tx=0).stop 2023-10-21T16:01:38.830 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 shutdown_connections 2023-10-21T16:01:38.830 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8015cd80 0x7f9c8014dde0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.830 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c800cdf70 0x7f9c80156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.830 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 --2- 172.21.15.124:0/2606969485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c800b6730 0x7f9c801f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:38.830 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 >> 172.21.15.124:0/2606969485 conn(0x7f9c8006b5d0 msgr2=0x7f9c8015a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:38.831 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 shutdown_connections 2023-10-21T16:01:38.831 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:38.827+0000 7f9c85ae0640 1 -- 172.21.15.124:0/2606969485 wait complete. 2023-10-21T16:01:38.839 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.837+0000 7f0ea5241040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:01:38.851 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new ed6646ad-cdd5-41d8-b1da-6fccc6570435 6 2023-10-21T16:01:38.928 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:38.927+0000 7f48ed97b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:01:38.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:38.958+0000 7f0ea5241040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:01:39.046 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.044+0000 7ff98e70f640 1 Processor -- start 2023-10-21T16:01:39.046 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.044+0000 7ff98e70f640 1 -- start start 2023-10-21T16:01:39.049 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff98e70f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.049 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff98e70f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff988161920 con 0x7ff9880cd470 2023-10-21T16:01:39.049 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff98e70f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880cb8b0 con 0x7ff988161090 2023-10-21T16:01:39.049 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff98e70f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff988161640 con 0x7ff98815ce00 2023-10-21T16:01:39.049 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff987fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff987fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:38308/0 (socket says 172.21.15.124:38308) 2023-10-21T16:01:39.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 learned_addr learned my addr 172.21.15.124:0/2100121905 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:39.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.046+0000 7ff98ccac640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7ff988161090 0x7ff98806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:57692/0 (socket says 172.21.15.124:57692) 2023-10-21T16:01:39.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 msgr2=0x7ff9880cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-21T16:01:39.050 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 msgr2=0x7ff9880cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:39.051 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:39.051 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:39.051 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728778083 0 0) 0x7ff9880cb8b0 con 0x7ff988161090 2023-10-21T16:01:39.051 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff974003520 con 0x7ff988161090 2023-10-21T16:01:39.053 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4238894369 0 0) 0x7ff974003520 con 0x7ff988161090 2023-10-21T16:01:39.054 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 msgr2=0x7ff9880cb370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:39.054 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff9880cb370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.054 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 >> v1:172.21.15.53:6789/0 conn(0x7ff9880cd470 legacy=0x7ff988160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.054 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98806b690 con 0x7ff988161090 2023-10-21T16:01:39.055 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff9800019f0 con 0x7ff988161090 2023-10-21T16:01:39.055 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 >> v1:172.21.15.124:6789/0 conn(0x7ff988161090 legacy=0x7ff98806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.055 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff974003970 0x7ff974003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.055 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.055 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff974005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.056 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff988161640 con 0x7ff9740047c0 2023-10-21T16:01:39.056 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97400bb40 con 0x7ff974003970 2023-10-21T16:01:39.056 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97400bd00 con 0x7ff98815ce00 2023-10-21T16:01:39.056 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.047+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff974003970 0x7ff974003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.056 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff98ccac640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff974005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff9877fe640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 msgr2=0x7ff974005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98815ce00 0x7ff974005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9740047c0 msgr2=0x7ff974004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.057 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.048+0000 7ff987fff640 1 -- 172.21.15.124:0/2100121905 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974003520 con 0x7ff974003970 2023-10-21T16:01:39.058 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff987fff640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff974003970 0x7ff974003d90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff970009880 tx=0x7ff9740057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec6c19e3a8884f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.058 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970030660 con 0x7ff974003970 2023-10-21T16:01:39.058 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9700307c0 con 0x7ff974003970 2023-10-21T16:01:39.058 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff986ffd640 1 -- 172.21.15.124:0/2100121905 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970030b70 con 0x7ff974003970 2023-10-21T16:01:39.058 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff98e70f640 1 -- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff974003970 msgr2=0x7ff974003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.059 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.050+0000 7ff98e70f640 1 --2- 172.21.15.124:0/2100121905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff974003970 0x7ff974003d90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff970009880 tx=0x7ff9740057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.059 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.051+0000 7ff98ccac640 1 -- 172.21.15.124:0/2100121905 reap_dead start 2023-10-21T16:01:39.059 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.051+0000 7ff98e70f640 1 -- 172.21.15.124:0/2100121905 shutdown_connections 2023-10-21T16:01:39.059 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.051+0000 7ff98e70f640 1 -- 172.21.15.124:0/2100121905 >> 172.21.15.124:0/2100121905 conn(0x7ff98806e650 msgr2=0x7ff98806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.059 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.051+0000 7ff98e70f640 1 -- 172.21.15.124:0/2100121905 shutdown_connections 2023-10-21T16:01:39.060 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.051+0000 7ff98e70f640 1 -- 172.21.15.124:0/2100121905 wait complete. 2023-10-21T16:01:39.062 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.052+0000 7ff98e70f640 1 Processor -- start 2023-10-21T16:01:39.063 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.053+0000 7ff98e70f640 1 -- start start 2023-10-21T16:01:39.063 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.053+0000 7ff98e70f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.063 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.053+0000 7ff98e70f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.063 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98e70f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 0x7ff9880c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.063 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98e70f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c7520 con 0x7ff9880c6b50 2023-10-21T16:01:39.064 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98e70f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880b15d0 con 0x7ff9880b04c0 2023-10-21T16:01:39.064 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98e70f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff988151470 con 0x7ff988151090 2023-10-21T16:01:39.064 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.064 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.064 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:38314/0 (socket says 172.21.15.124:38314) 2023-10-21T16:01:39.065 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:47044/0 (socket says 172.21.15.124:47044) 2023-10-21T16:01:39.065 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 -- 172.21.15.124:0/1492914493 learned_addr learned my addr 172.21.15.124:0/1492914493 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:39.065 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 -- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 msgr2=0x7ff9880b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-21T16:01:39.065 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 -- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 msgr2=0x7ff9880b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:39.065 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:39.066 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff987fff640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:39.066 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98ccac640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 0x7ff9880c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.066 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 -- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 msgr2=0x7ff9880b1a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:39.066 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.066 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 -- 172.21.15.124:0/1492914493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 msgr2=0x7ff9880c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 0x7ff9880c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 -- 172.21.15.124:0/1492914493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880b09c0 con 0x7ff9880b04c0 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff98ccac640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 0x7ff9880c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.054+0000 7ff9877fe640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff97800e3c0 tx=0x7ff97800e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780052e0 con 0x7ff9880b04c0 2023-10-21T16:01:39.067 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff978005440 con 0x7ff9880b04c0 2023-10-21T16:01:39.068 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978003810 con 0x7ff9880b04c0 2023-10-21T16:01:39.068 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9880b0ca0 con 0x7ff9880b04c0 2023-10-21T16:01:39.068 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9880b1260 con 0x7ff9880b04c0 2023-10-21T16:01:39.068 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff978003c70 con 0x7ff9880b04c0 2023-10-21T16:01:39.068 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.055+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff978007020 con 0x7ff9880b04c0 2023-10-21T16:01:39.069 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.056+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9880b13f0 con 0x7ff9880b04c0 2023-10-21T16:01:39.069 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.058+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff978019070 con 0x7ff9880b04c0 2023-10-21T16:01:39.095 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:39.094+0000 7f48ed97b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:01:39.130 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:39.127+0000 7f0ea5241040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:01:39.134 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.132+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ed6646ad-cdd5-41d8-b1da-6fccc6570435", "id": 6} v 0) v1 -- 0x7ff98806cff0 con 0x7ff9880b04c0 2023-10-21T16:01:39.145 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.142+0000 7ff984ff9640 1 -- 172.21.15.124:0/1492914493 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed6646ad-cdd5-41d8-b1da-6fccc6570435", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff978007320 con 0x7ff9880b04c0 2023-10-21T16:01:39.145 INFO:teuthology.orchestra.run.smithi124.stdout:6 2023-10-21T16:01:39.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 msgr2=0x7ff9880b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff97800e3c0 tx=0x7ff97800e880 comp rx=0 tx=0).stop 2023-10-21T16:01:39.146 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 shutdown_connections 2023-10-21T16:01:39.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9880c6b50 0x7ff9880c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9880b04c0 0x7ff9880b5a90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 --2- 172.21.15.124:0/1492914493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff988151090 0x7ff9880b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 >> 172.21.15.124:0/1492914493 conn(0x7ff98806e650 msgr2=0x7ff9880c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.147 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 shutdown_connections 2023-10-21T16:01:39.148 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.144+0000 7ff98e70f640 1 -- 172.21.15.124:0/1492914493 wait complete. 2023-10-21T16:01:39.148 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:39.146+0000 7f48ed97b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:01:39.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new ffe40f79-ab92-472f-a4b2-fbdc385ebc6b 7 2023-10-21T16:01:39.184 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:39.182+0000 7f0ea5241040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:01:39.313 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:39.311+0000 7f48ed97b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:01:39.347 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:39.345+0000 7f0ea5241040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:01:39.348 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.346+0000 7f51b37a9640 1 Processor -- start 2023-10-21T16:01:39.349 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.347+0000 7f51b37a9640 1 -- start start 2023-10-21T16:01:39.352 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b37a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51ac15cd70 0x7f51ac1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.352 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b37a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f51ac0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.352 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b37a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0b5090 con 0x7f51ac0b4c40 2023-10-21T16:01:39.352 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b37a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0b8c30 con 0x7f51ac15cd70 2023-10-21T16:01:39.353 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b37a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0cdd80 con 0x7f51ac0cd4e0 2023-10-21T16:01:39.353 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b1d46640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f51ac0b4c40 0x7f51ac06b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:57702/0 (socket says 172.21.15.124:57702) 2023-10-21T16:01:39.353 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b0d44640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f51ac0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.353 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.348+0000 7f51b1d46640 1 -- 172.21.15.124:0/3885376520 learned_addr learned my addr 172.21.15.124:0/3885376520 (peer_addr_for_me v1:172.21.15.124:0/0) 2023-10-21T16:01:39.353 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51b1545640 1 --2- 172.21.15.124:0/3885376520 >> v2:172.21.15.53:3300/0 conn(0x7f51ac15cd70 0x7f51ac1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.354 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51b0d44640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 msgr2=0x7f51ac0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-21T16:01:39.354 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51b0d44640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 msgr2=0x7f51ac0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-21T16:01:39.354 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51b0d44640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f51ac0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:39.354 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51b0d44640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f51ac0b4700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:39.354 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164516414 0 0) 0x7f51a4001490 con 0x7f51ac0b4c40 2023-10-21T16:01:39.355 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5198003520 con 0x7f51ac0b4c40 2023-10-21T16:01:39.355 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2436847813 0 0) 0x7f5198003520 con 0x7f51ac0b4c40 2023-10-21T16:01:39.355 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 msgr2=0x7f51ac0b4700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:01:39.355 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f51ac0b4700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.355 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 >> v2:172.21.15.53:3300/0 conn(0x7f51ac15cd70 msgr2=0x7f51ac1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 --2- 172.21.15.124:0/3885376520 >> v2:172.21.15.53:3300/0 conn(0x7f51ac15cd70 0x7f51ac1612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0ce060 con 0x7f51ac0b4c40 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f51a4001b80 con 0x7f51ac0b4c40 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 >> v1:172.21.15.124:6789/0 conn(0x7f51ac0b4c40 legacy=0x7f51ac06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f5198003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.356 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980047c0 0x7f5198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.357 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f5198005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.357 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b8c30 con 0x7f51980047c0 2023-10-21T16:01:39.357 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0cdd80 con 0x7f5198003970 2023-10-21T16:01:39.357 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.349+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519800b9b0 con 0x7f51ac0cd4e0 2023-10-21T16:01:39.357 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f5198003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.358 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1d46640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f5198005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.358 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b0d44640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980047c0 0x7f5198004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.358 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 msgr2=0x7f5198005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.358 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac0cd4e0 0x7f5198005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.358 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980047c0 msgr2=0x7f5198004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.359 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980047c0 0x7f5198004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.359 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.350+0000 7f51b1545640 1 -- 172.21.15.124:0/3885376520 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198003520 con 0x7f5198003970 2023-10-21T16:01:39.362 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.351+0000 7f51b1545640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f5198003d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009ac0 tx=0x7f51980057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c54a4c260c98336 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.362 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.351+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a802c710 con 0x7f5198003970 2023-10-21T16:01:39.362 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.351+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a802c870 con 0x7f5198003970 2023-10-21T16:01:39.362 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.351+0000 7f51a3fff640 1 -- 172.21.15.124:0/3885376520 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a802cc00 con 0x7f5198003970 2023-10-21T16:01:39.363 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 -- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 msgr2=0x7f5198003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.363 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 --2- 172.21.15.124:0/3885376520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f5198003d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009ac0 tx=0x7f51980057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.363 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b1d46640 1 -- 172.21.15.124:0/3885376520 reap_dead start 2023-10-21T16:01:39.363 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 -- 172.21.15.124:0/3885376520 shutdown_connections 2023-10-21T16:01:39.363 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 -- 172.21.15.124:0/3885376520 >> 172.21.15.124:0/3885376520 conn(0x7f51ac0cf700 msgr2=0x7f51ac0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.364 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 -- 172.21.15.124:0/3885376520 shutdown_connections 2023-10-21T16:01:39.364 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.352+0000 7f51b37a9640 1 -- 172.21.15.124:0/3885376520 wait complete. 2023-10-21T16:01:39.364 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.354+0000 7f51b37a9640 1 Processor -- start 2023-10-21T16:01:39.364 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.354+0000 7f51b37a9640 1 -- start start 2023-10-21T16:01:39.364 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac0cbd50 0x7f51ac15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f22b0 con 0x7f51ac0cbd50 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0ca770 con 0x7f5198003970 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b37a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0caa50 con 0x7f51ac15db60 2023-10-21T16:01:39.365 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.366 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b0d44640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.366 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:47056/0 (socket says 172.21.15.124:47056) 2023-10-21T16:01:39.366 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 -- 172.21.15.124:0/2001365471 learned_addr learned my addr 172.21.15.124:0/2001365471 (peer_addr_for_me v2:172.21.15.124:0/0) 2023-10-21T16:01:39.366 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b0d44640 1 -- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 msgr2=0x7f51ac0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2023-10-21T16:01:39.366 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b0d44640 1 -- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 msgr2=0x7f51ac0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:39.367 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b0d44640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:39.367 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b0d44640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:39.367 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1d46640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac0cbd50 0x7f51ac15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.367 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 -- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 msgr2=0x7f51ac0cb810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:39.367 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.368 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 -- 172.21.15.124:0/2001365471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac0cbd50 msgr2=0x7f51ac15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.368 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac0cbd50 0x7f51ac15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.368 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.355+0000 7f51b1545640 1 -- 172.21.15.124:0/2001365471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0cad30 con 0x7f5198003970 2023-10-21T16:01:39.368 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51b1545640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51a802c2b0 tx=0x7f51a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.368 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8003810 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a802de90 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac0cb010 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8003b70 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac155ff0 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f51a8005990 con 0x7f5198003970 2023-10-21T16:01:39.369 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.356+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f51a8006020 con 0x7f5198003970 2023-10-21T16:01:39.370 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.357+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac1561b0 con 0x7f5198003970 2023-10-21T16:01:39.370 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.359+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f51a8044020 con 0x7f5198003970 2023-10-21T16:01:39.438 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.436+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ffe40f79-ab92-472f-a4b2-fbdc385ebc6b", "id": 7} v 0) v1 -- 0x7f51ac04e160 con 0x7f5198003970 2023-10-21T16:01:39.451 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.449+0000 7f51a1ffb640 1 -- 172.21.15.124:0/2001365471 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ffe40f79-ab92-472f-a4b2-fbdc385ebc6b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f51a8039070 con 0x7f5198003970 2023-10-21T16:01:39.452 INFO:teuthology.orchestra.run.smithi124.stdout:7 2023-10-21T16:01:39.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.450+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 msgr2=0x7f51ac1f1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.450+0000 7f51b37a9640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51a802c2b0 tx=0x7f51a8004730 comp rx=0 tx=0).stop 2023-10-21T16:01:39.453 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 shutdown_connections 2023-10-21T16:01:39.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51ac0cbd50 0x7f51ac15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51ac15db60 0x7f51ac0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 --2- 172.21.15.124:0/2001365471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198003970 0x7f51ac1f1d70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 >> 172.21.15.124:0/2001365471 conn(0x7f51ac0cf700 msgr2=0x7f51ac0d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.454 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 shutdown_connections 2023-10-21T16:01:39.455 INFO:teuthology.orchestra.run.smithi124.stderr:2023-10-21T16:01:39.451+0000 7f51b37a9640 1 -- 172.21.15.124:0/2001365471 wait complete. 2023-10-21T16:01:39.470 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-21T16:01:39.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-21T16:01:39.479 INFO:tasks.ceph.osd.0:Started 2023-10-21T16:01:39.480 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-21T16:01:39.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-21T16:01:39.488 INFO:tasks.ceph.osd.1:Started 2023-10-21T16:01:39.489 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-21T16:01:39.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-21T16:01:39.494 INFO:tasks.ceph.osd.2:Started 2023-10-21T16:01:39.495 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-21T16:01:39.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-21T16:01:39.501 INFO:tasks.ceph.osd.3:Started 2023-10-21T16:01:39.501 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-21T16:01:39.502 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-21T16:01:39.519 INFO:tasks.ceph.osd.4:Started 2023-10-21T16:01:39.520 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-21T16:01:39.520 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-21T16:01:39.526 INFO:tasks.ceph.osd.5:Started 2023-10-21T16:01:39.527 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-21T16:01:39.527 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-21T16:01:39.534 INFO:tasks.ceph.osd.6:Started 2023-10-21T16:01:39.534 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-21T16:01:39.534 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-21T16:01:39.540 INFO:tasks.ceph.osd.7:Started 2023-10-21T16:01:39.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:01:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.757+0000 7fad54dc9640 1 Processor -- start 2023-10-21T16:01:39.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.757+0000 7fad54dc9640 1 -- start start 2023-10-21T16:01:39.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad54dc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad5015aad0 0x7fad500cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad54dc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad500cc570 0x7fad500cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad54dc9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad500cb580 con 0x7fad5015aeb0 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad54dc9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500b2390 con 0x7fad5015aad0 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad54dc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500cedc0 con 0x7fad500cc570 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad5015aad0 0x7fad500cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad5015aad0 0x7fad500cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41066/0 (socket says 172.21.15.53:41066) 2023-10-21T16:01:39.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4e59c640 1 -- 172.21.15.53:0/906527509 learned_addr learned my addr 172.21.15.53:0/906527509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:39.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4ed9d640 1 --2- 172.21.15.53:0/906527509 >> v2:172.21.15.124:3300/0 conn(0x7fad500cc570 0x7fad500cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4e59c640 1 -- 172.21.15.53:0/906527509 >> v2:172.21.15.124:3300/0 conn(0x7fad500cc570 msgr2=0x7fad500cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:39.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.758+0000 7fad4e59c640 1 --2- 172.21.15.53:0/906527509 >> v2:172.21.15.124:3300/0 conn(0x7fad500cc570 0x7fad500cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.759+0000 7fad4e59c640 1 -- 172.21.15.53:0/906527509 >> v1:172.21.15.53:6789/0 conn(0x7fad5015aeb0 legacy=0x7fad500ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.759+0000 7fad4e59c640 1 -- 172.21.15.53:0/906527509 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500cc0e0 con 0x7fad5015aad0 2023-10-21T16:01:39.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad4e59c640 1 --2- 172.21.15.53:0/906527509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500cdf60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad500d1ac0 tx=0x7fad4002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9589aa3dabeb54db server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad4dd9b640 1 -- 172.21.15.53:0/906527509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4002f850 con 0x7fad5015aad0 2023-10-21T16:01:39.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad4dd9b640 1 -- 172.21.15.53:0/906527509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4002f9b0 con 0x7fad5015aad0 2023-10-21T16:01:39.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad4dd9b640 1 -- 172.21.15.53:0/906527509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4002fd40 con 0x7fad5015aad0 2023-10-21T16:01:39.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 -- 172.21.15.53:0/906527509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 msgr2=0x7fad500cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 --2- 172.21.15.53:0/906527509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500cdf60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad500d1ac0 tx=0x7fad4002ba30 comp rx=0 tx=0).stop 2023-10-21T16:01:39.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 -- 172.21.15.53:0/906527509 shutdown_connections 2023-10-21T16:01:39.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 --2- 172.21.15.53:0/906527509 >> v2:172.21.15.124:3300/0 conn(0x7fad500cc570 0x7fad500cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 --2- 172.21.15.53:0/906527509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500cdf60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.760+0000 7fad54dc9640 1 -- 172.21.15.53:0/906527509 >> 172.21.15.53:0/906527509 conn(0x7fad500d09e0 msgr2=0x7fad500d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.761+0000 7fad54dc9640 1 -- 172.21.15.53:0/906527509 shutdown_connections 2023-10-21T16:01:39.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.761+0000 7fad54dc9640 1 -- 172.21.15.53:0/906527509 wait complete. 2023-10-21T16:01:39.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.762+0000 7fad54dc9640 1 Processor -- start 2023-10-21T16:01:39.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.762+0000 7fad54dc9640 1 -- start start 2023-10-21T16:01:39.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5015aeb0 0x7fad501f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:39.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500c7ab0 con 0x7fad5015aad0 2023-10-21T16:01:39.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad501f5e60 con 0x7fad5015aeb0 2023-10-21T16:01:39.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad54dc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad501f6100 con 0x7fad500cc570 2023-10-21T16:01:39.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.763+0000 7fad47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41074/0 (socket says 172.21.15.53:41074) 2023-10-21T16:01:39.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 -- 172.21.15.53:0/1489788218 learned_addr learned my addr 172.21.15.53:0/1489788218 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:39.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4e59c640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4e59c640 1 -- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 msgr2=0x7fad500c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2023-10-21T16:01:39.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4e59c640 1 -- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 msgr2=0x7fad500c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-21T16:01:39.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4e59c640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-21T16:01:39.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4e59c640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:01:39.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad4ed9d640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5015aeb0 0x7fad501f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:39.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 -- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 msgr2=0x7fad500c9800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:01:39.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 -- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5015aeb0 msgr2=0x7fad501f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5015aeb0 0x7fad501f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 -- 172.21.15.53:0/1489788218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009470 con 0x7fad5015aad0 2023-10-21T16:01:39.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad47fff640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c400 tx=0x7fad3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:39.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad380052d0 con 0x7fad5015aad0 2023-10-21T16:01:39.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad38005430 con 0x7fad5015aad0 2023-10-21T16:01:39.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38003830 con 0x7fad5015aad0 2023-10-21T16:01:39.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.764+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad501f63e0 con 0x7fad5015aad0 2023-10-21T16:01:39.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.765+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad500c7cc0 con 0x7fad5015aad0 2023-10-21T16:01:39.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.765+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fad38003c90 con 0x7fad5015aad0 2023-10-21T16:01:39.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.765+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fad38012420 con 0x7fad5015aad0 2023-10-21T16:01:39.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.766+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad500c6b50 con 0x7fad5015aad0 2023-10-21T16:01:39.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.768+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fad38017070 con 0x7fad5015aad0 2023-10-21T16:01:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.833+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad500c7ee0 con 0x7fad5015aad0 2023-10-21T16:01:39.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.834+0000 7fad46ffd640 1 -- 172.21.15.53:0/1489788218 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fad38011590 con 0x7fad5015aad0 2023-10-21T16:01:39.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:39.836 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:01:39.439175+0000","last_up_change":"0.000000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:01:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 msgr2=0x7fad500c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:39.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c400 tx=0x7fad3800c8c0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 shutdown_connections 2023-10-21T16:01:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5015aeb0 0x7fad501f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad5015aad0 0x7fad500c6610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 --2- 172.21.15.53:0/1489788218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad500cc570 0x7fad500c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:39.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 >> 172.21.15.53:0/1489788218 conn(0x7fad500d09e0 msgr2=0x7fad5006ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:39.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 shutdown_connections 2023-10-21T16:01:39.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:39.835+0000 7fad54dc9640 1 -- 172.21.15.53:0/1489788218 wait complete. 2023-10-21T16:01:39.853 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-21T16:01:39.853 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-21T16:01:40.189 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-10-21T16:01:40.187+0000 7feae3413500 -1 Falling back to public interface 2023-10-21T16:01:40.201 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-10-21T16:01:40.199+0000 7f149bd48500 -1 Falling back to public interface 2023-10-21T16:01:40.204 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-21T16:01:40.202+0000 7f4f142af500 -1 Falling back to public interface 2023-10-21T16:01:40.213 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-10-21T16:01:40.212+0000 7f80ad524500 -1 Falling back to public interface 2023-10-21T16:01:40.218 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-21T16:01:40.216+0000 7f0c16613500 -1 Falling back to public interface 2023-10-21T16:01:40.219 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-21T16:01:40.216+0000 7f33d9e1d500 -1 Falling back to public interface 2023-10-21T16:01:40.219 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-21T16:01:40.217+0000 7f33d8825640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6809/0: (98) Address already in use 2023-10-21T16:01:40.223 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-21T16:01:40.221+0000 7f0c14ed9640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6817/0: (98) Address already in use 2023-10-21T16:01:40.247 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-10-21T16:01:40.246+0000 7f5f8d411500 -1 Falling back to public interface 2023-10-21T16:01:40.266 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-21T16:01:40.264+0000 7fb4cc00c500 -1 Falling back to public interface 2023-10-21T16:01:40.477 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:40.475+0000 7f0ea5241040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:01:40.507 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:40.505+0000 7f48ed97b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:01:40.659 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-21T16:01:40.807 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:40.805+0000 7f0ea5241040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:01:40.820 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.818+0000 7f4600977640 1 Processor -- start 2023-10-21T16:01:40.820 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.819+0000 7f4600977640 1 -- start start 2023-10-21T16:01:40.823 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f4600977640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45fc0cf550 0x7f45fc0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:40.823 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f4600977640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc0d28e0 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.823 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f4600977640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45fc15d2a0 con 0x7f45fc15cec0 2023-10-21T16:01:40.823 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f4600977640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45fc0cff20 con 0x7f45fc0cf550 2023-10-21T16:01:40.823 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f45fa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45fc0cf550 0x7f45fc0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:40.824 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f45fa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45fc0cf550 0x7f45fc0cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41090/0 (socket says 172.21.15.53:41090) 2023-10-21T16:01:40.824 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f45fa59c640 1 -- 172.21.15.53:0/1977690832 learned_addr learned my addr 172.21.15.53:0/1977690832 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:40.824 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f45fb59e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f45fc0b5bd0 0x7f45fc0b6020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33302/0 (socket says 172.21.15.53:33302) 2023-10-21T16:01:40.825 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.821+0000 7f45fa59c640 1 -- 172.21.15.53:0/1977690832 >> v1:172.21.15.124:6789/0 conn(0x7f45fc15cec0 legacy=0x7f45fc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.826 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.822+0000 7f45fa59c640 1 -- 172.21.15.53:0/1977690832 >> v1:172.21.15.53:6789/0 conn(0x7f45fc0b5bd0 legacy=0x7f45fc0b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.826 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.822+0000 7f45fa59c640 1 -- 172.21.15.53:0/1977690832 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0d1840 con 0x7f45fc0cf550 2023-10-21T16:01:40.826 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f45fa59c640 1 --2- 172.21.15.53:0/1977690832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0cf550 0x7f45fc0cf9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009870 tx=0x7f45f002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db0689ada87fb19 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:40.826 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f45f9d9b640 1 -- 172.21.15.53:0/1977690832 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f002f850 con 0x7f45fc0cf550 2023-10-21T16:01:40.827 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f45f9d9b640 1 -- 172.21.15.53:0/1977690832 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f002f9b0 con 0x7f45fc0cf550 2023-10-21T16:01:40.827 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f45f9d9b640 1 -- 172.21.15.53:0/1977690832 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f002fd40 con 0x7f45fc0cf550 2023-10-21T16:01:40.827 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f4600977640 1 -- 172.21.15.53:0/1977690832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0cf550 msgr2=0x7f45fc0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.827 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.823+0000 7f4600977640 1 --2- 172.21.15.53:0/1977690832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0cf550 0x7f45fc0cf9e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009870 tx=0x7f45f002ba30 comp rx=0 tx=0).stop 2023-10-21T16:01:40.827 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.824+0000 7f4600977640 1 -- 172.21.15.53:0/1977690832 shutdown_connections 2023-10-21T16:01:40.828 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.824+0000 7f4600977640 1 --2- 172.21.15.53:0/1977690832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0cf550 0x7f45fc0cf9e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.828 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.824+0000 7f4600977640 1 -- 172.21.15.53:0/1977690832 >> 172.21.15.53:0/1977690832 conn(0x7f45fc06b120 msgr2=0x7f45fc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:40.828 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.824+0000 7f4600977640 1 -- 172.21.15.53:0/1977690832 shutdown_connections 2023-10-21T16:01:40.828 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.824+0000 7f4600977640 1 -- 172.21.15.53:0/1977690832 wait complete. 2023-10-21T16:01:40.831 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.825+0000 7f4600977640 1 Processor -- start 2023-10-21T16:01:40.831 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.825+0000 7f4600977640 1 -- start start 2023-10-21T16:01:40.831 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:40.831 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 0x7f45fc0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:40.832 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45fc15cec0 0x7f45fc0b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:40.832 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1f7460 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.832 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1f7670 con 0x7f45fc0cf550 2023-10-21T16:01:40.832 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.826+0000 7f4600977640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1f7950 con 0x7f45fc15cec0 2023-10-21T16:01:40.832 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:40.833 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41092/0 (socket says 172.21.15.53:41092) 2023-10-21T16:01:40.833 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 -- 172.21.15.53:0/3874732001 learned_addr learned my addr 172.21.15.53:0/3874732001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:40.833 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fa59c640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 0x7f45fc0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:40.833 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fb59e640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45fc15cec0 0x7f45fc0b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:40.834 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 -- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45fc15cec0 msgr2=0x7f45fc0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.834 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45fc15cec0 0x7f45fc0b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.834 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 -- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 msgr2=0x7f45fc0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.834 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 0x7f45fc0b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.835 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 -- 172.21.15.53:0/3874732001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00094d0 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.835 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fa59c640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 0x7f45fc0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:01:40.835 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.827+0000 7f45fad9d640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c400 tx=0x7f45ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:40.835 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0052d0 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.835 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec005430 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.836 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec003830 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.836 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc1f7c30 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.836 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc1f7020 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.836 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f45ec003c90 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.836 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.828+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f45ec012420 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.837 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:40.829+0000 7f48ed97b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:01:40.837 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.829+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45fc04e160 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.837 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.832+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f45ec017070 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.871 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:40.869+0000 7f0ea5241040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:01:40.893 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:40.892+0000 7f48ed97b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:01:40.900 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.899+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45fc0ccf00 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.902 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.900+0000 7f45e37fe640 1 -- 172.21.15.53:0/3874732001 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f45ec011590 con 0x7f45fc0b5bd0 2023-10-21T16:01:40.902 INFO:teuthology.misc.health.smithi053.stdout: 2023-10-21T16:01:40.902 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":10,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:01:39.439175+0000","last_up_change":"0.000000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:01:40.903 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 msgr2=0x7f45fc1f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:40.903 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c400 tx=0x7f45ec00c8c0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.904 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 shutdown_connections 2023-10-21T16:01:40.904 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45fc15cec0 0x7f45fc0b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.904 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45fc0cf550 0x7f45fc0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.904 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 --2- 172.21.15.53:0/3874732001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45fc0b5bd0 0x7f45fc1f6ae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:40.904 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 >> 172.21.15.53:0/3874732001 conn(0x7f45fc06b120 msgr2=0x7f45fc073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:40.905 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 shutdown_connections 2023-10-21T16:01:40.905 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:40.901+0000 7f4600977640 1 -- 172.21.15.53:0/3874732001 wait complete. 2023-10-21T16:01:40.916 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-21T16:01:40.987 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:40.985+0000 7f0ea5241040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:01:41.015 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:01:41.013+0000 7f48ed97b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:01:42.039 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:42.037+0000 7f0e46651640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-21T16:01:42.039 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:01:42.037+0000 7f0e46651640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-21T16:01:43.774 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-21T16:01:43.772+0000 7f0c16613500 -1 osd.5 0 log_to_monitors true 2023-10-21T16:01:43.778 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-21T16:01:43.777+0000 7f33d9e1d500 -1 osd.6 0 log_to_monitors true 2023-10-21T16:01:43.779 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-10-21T16:01:43.778+0000 7f80ad524500 -1 osd.2 0 log_to_monitors true 2023-10-21T16:01:43.802 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-10-21T16:01:43.801+0000 7f5f8d411500 -1 osd.3 0 log_to_monitors true 2023-10-21T16:01:43.818 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-21T16:01:43.816+0000 7fb4cc00c500 -1 osd.7 0 log_to_monitors true 2023-10-21T16:01:43.969 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-10-21T16:01:43.967+0000 7feae3413500 -1 osd.1 0 log_to_monitors true 2023-10-21T16:01:43.989 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-21T16:01:43.987+0000 7f4f142af500 -1 osd.4 0 log_to_monitors true 2023-10-21T16:01:43.997 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-10-21T16:01:43.994+0000 7f149bd48500 -1 osd.0 0 log_to_monitors true 2023-10-21T16:01:45.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:01:45.954 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:01:46.778 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-10-21T16:01:46.776+0000 7f5f88ccc640 -1 osd.3 0 waiting for initial osdmap 2023-10-21T16:01:46.787 INFO:tasks.ceph.osd.3.smithi053.stderr:2023-10-21T16:01:46.785+0000 7f5f80aed640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:46.828 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-10-21T16:01:46.827+0000 7f1497cd4640 -1 osd.0 0 waiting for initial osdmap 2023-10-21T16:01:46.835 INFO:tasks.ceph.osd.0.smithi053.stderr:2023-10-21T16:01:46.834+0000 7f148faf5640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:46.880 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-10-21T16:01:46.878+0000 7f80a94b0640 -1 osd.2 0 waiting for initial osdmap 2023-10-21T16:01:46.888 INFO:tasks.ceph.osd.2.smithi053.stderr:2023-10-21T16:01:46.886+0000 7f80a12d1640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:46.922 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-10-21T16:01:46.921+0000 7feadecbe640 -1 osd.1 0 waiting for initial osdmap 2023-10-21T16:01:46.929 INFO:tasks.ceph.osd.1.smithi053.stderr:2023-10-21T16:01:46.927+0000 7fead6adf640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:47.288 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-21T16:01:47.287+0000 7f4f1023b640 -1 osd.4 0 waiting for initial osdmap 2023-10-21T16:01:47.308 INFO:tasks.ceph.osd.4.smithi124.stderr:2023-10-21T16:01:47.304+0000 7f4f0805c640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:47.378 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-21T16:01:47.377+0000 7f33d580a640 -1 osd.6 0 waiting for initial osdmap 2023-10-21T16:01:47.385 INFO:tasks.ceph.osd.6.smithi124.stderr:2023-10-21T16:01:47.385+0000 7f33cd62b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:47.420 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-21T16:01:47.419+0000 7f0c11ebe640 -1 osd.5 0 waiting for initial osdmap 2023-10-21T16:01:47.426 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-21T16:01:47.425+0000 7fb4c80e0640 -1 osd.7 0 waiting for initial osdmap 2023-10-21T16:01:47.426 INFO:tasks.ceph.osd.5.smithi124.stderr:2023-10-21T16:01:47.425+0000 7f0c09cdf640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:47.440 INFO:tasks.ceph.osd.7.smithi124.stderr:2023-10-21T16:01:47.439+0000 7fb4bf6eb640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-21T16:01:47.721 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-21T16:01:47.912 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.910+0000 7fd6db501640 1 Processor -- start 2023-10-21T16:01:47.912 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.910+0000 7fd6db501640 1 -- start start 2023-10-21T16:01:47.913 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6db501640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd6d415ceb0 0x7fd6d40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.914 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6db501640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cd780 0x7fd6d40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.914 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6db501640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40cdbe0 con 0x7fd6d40cfe10 2023-10-21T16:01:47.914 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6db501640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40d0500 con 0x7fd6d415ceb0 2023-10-21T16:01:47.914 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6db501640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40d02a0 con 0x7fd6d40cd780 2023-10-21T16:01:47.915 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6d929d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd6d415ceb0 0x7fd6d40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:47.915 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6d929d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd6d415ceb0 0x7fd6d40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:52804/0 (socket says 172.21.15.53:52804) 2023-10-21T16:01:47.916 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.912+0000 7fd6d929d640 1 -- 172.21.15.53:0/4128344783 learned_addr learned my addr 172.21.15.53:0/4128344783 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:47.916 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.913+0000 7fd6d8a9c640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cd780 0x7fd6d40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:47.916 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.913+0000 7fd6d929d640 1 -- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cd780 msgr2=0x7fd6d40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:47.916 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.913+0000 7fd6d929d640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cd780 0x7fd6d40cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:47.916 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.913+0000 7fd6d929d640 1 -- 172.21.15.53:0/4128344783 >> v1:172.21.15.124:6789/0 conn(0x7fd6d40cfe10 legacy=0x7fd6d40b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:47.917 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.913+0000 7fd6d929d640 1 -- 172.21.15.53:0/4128344783 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d415d290 con 0x7fd6d415ceb0 2023-10-21T16:01:47.918 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.914+0000 7fd6d929d640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d40cd240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009850 tx=0x7fd6d002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74ce4a3b53c7a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:47.918 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6cbfff640 1 -- 172.21.15.53:0/4128344783 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d002e070 con 0x7fd6d415ceb0 2023-10-21T16:01:47.918 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6cbfff640 1 -- 172.21.15.53:0/4128344783 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d0004030 con 0x7fd6d415ceb0 2023-10-21T16:01:47.918 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6cbfff640 1 -- 172.21.15.53:0/4128344783 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0004420 con 0x7fd6d415ceb0 2023-10-21T16:01:47.919 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 -- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 msgr2=0x7fd6d40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:47.919 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d40cd240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009850 tx=0x7fd6d002b840 comp rx=0 tx=0).stop 2023-10-21T16:01:47.919 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 -- 172.21.15.53:0/4128344783 shutdown_connections 2023-10-21T16:01:47.919 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cd780 0x7fd6d40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:47.919 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 --2- 172.21.15.53:0/4128344783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d40cd240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:47.920 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.915+0000 7fd6db501640 1 -- 172.21.15.53:0/4128344783 >> 172.21.15.53:0/4128344783 conn(0x7fd6d40d09c0 msgr2=0x7fd6d40d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:47.920 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.916+0000 7fd6db501640 1 -- 172.21.15.53:0/4128344783 shutdown_connections 2023-10-21T16:01:47.920 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.916+0000 7fd6db501640 1 -- 172.21.15.53:0/4128344783 wait complete. 2023-10-21T16:01:47.922 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.917+0000 7fd6db501640 1 Processor -- start 2023-10-21T16:01:47.922 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.917+0000 7fd6db501640 1 -- start start 2023-10-21T16:01:47.922 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.922 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cfe10 0x7fd6d41f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.923 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.923 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41f2a10 con 0x7fd6d40cd780 2023-10-21T16:01:47.923 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41f6110 con 0x7fd6d415ceb0 2023-10-21T16:01:47.923 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.918+0000 7fd6db501640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41f63b0 con 0x7fd6d40cfe10 2023-10-21T16:01:47.923 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:47.924 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58440/0 (socket says 172.21.15.53:58440) 2023-10-21T16:01:47.924 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 -- 172.21.15.53:0/1720594787 learned_addr learned my addr 172.21.15.53:0/1720594787 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:47.924 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d9a9e640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:47.924 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 -- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cfe10 msgr2=0x7fd6d41f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:47.924 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cfe10 0x7fd6d41f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:47.925 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 -- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 msgr2=0x7fd6d415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:47.925 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d415b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:47.925 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 -- 172.21.15.53:0/1720594787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009460 con 0x7fd6d40cd780 2023-10-21T16:01:47.925 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.919+0000 7fd6d929d640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0035040 tx=0x7fd6d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:47.925 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.920+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003920 con 0x7fd6d40cd780 2023-10-21T16:01:47.926 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.920+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d0003a80 con 0x7fd6d40cd780 2023-10-21T16:01:47.926 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.920+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003df0 con 0x7fd6d40cd780 2023-10-21T16:01:47.926 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.920+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d41f6690 con 0x7fd6d40cd780 2023-10-21T16:01:47.926 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.920+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d41f25e0 con 0x7fd6d40cd780 2023-10-21T16:01:47.927 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.921+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d4074090 con 0x7fd6d40cd780 2023-10-21T16:01:47.927 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.922+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fd6d0006040 con 0x7fd6d40cd780 2023-10-21T16:01:47.928 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.923+0000 7fd6c9ffb640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 0x7fd6b806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:47.928 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.923+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd6d00aaa10 con 0x7fd6d40cd780 2023-10-21T16:01:47.928 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.923+0000 7fd6d8a9c640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 0x7fd6b806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:47.928 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.925+0000 7fd6d8a9c640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 0x7fd6b806d2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006a90 tx=0x7fd6c4006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:47.929 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:47.928+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6d0075fd0 con 0x7fd6d40cd780 2023-10-21T16:01:48.118 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.116+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d40cadd0 con 0x7fd6d40cd780 2023-10-21T16:01:48.119 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.118+0000 7fd6c9ffb640 1 -- 172.21.15.53:0/1720594787 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7fd6d00795f0 con 0x7fd6d40cd780 2023-10-21T16:01:48.120 INFO:teuthology.misc.health.smithi053.stdout: 2023-10-21T16:01:48.120 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":13,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:01:47.773219+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:01:48.121 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 msgr2=0x7fd6b806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.122 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 0x7fd6b806d2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4006a90 tx=0x7fd6c4006a20 comp rx=0 tx=0).stop 2023-10-21T16:01:48.122 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 msgr2=0x7fd6d409e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.122 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0035040 tx=0x7fd6d00046c0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.123 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 shutdown_connections 2023-10-21T16:01:48.123 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d415ceb0 0x7fd6d415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.123 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d40cfe10 0x7fd6d41f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.123 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd6b806ade0 0x7fd6b806d2a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.123 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 --2- 172.21.15.53:0/1720594787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d40cd780 0x7fd6d409e0f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.124 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 >> 172.21.15.53:0/1720594787 conn(0x7fd6d40d09c0 msgr2=0x7fd6d406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:48.124 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.120+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 shutdown_connections 2023-10-21T16:01:48.124 INFO:teuthology.misc.health.smithi053.stderr:2023-10-21T16:01:48.121+0000 7fd6db501640 1 -- 172.21.15.53:0/1720594787 wait complete. 2023-10-21T16:01:48.136 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-21T16:01:48.136 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-21T16:01:48.137 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:01:48.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:01:48.146 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:01:48.146 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:01:48.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:01:48.155 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:01:48.156 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:01:48.157 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:01:48.167 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:01:48.168 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:01:48.168 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:01:48.178 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:01:48.179 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-21T16:01:48.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:01:48.301 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:01:48.302 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:01:48.322 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:01:48.338 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:01:48.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.366+0000 7f4bb256c640 1 Processor -- start 2023-10-21T16:01:48.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.367+0000 7f4bb256c640 1 -- start start 2023-10-21T16:01:48.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb256c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4bac15aad0 0x7f4bac0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb256c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb256c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac0cb580 con 0x7f4bac15aeb0 2023-10-21T16:01:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb256c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0b2390 con 0x7f4bac15aad0 2023-10-21T16:01:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb256c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0cedc0 con 0x7f4bac0cc570 2023-10-21T16:01:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4ba3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4bac15aeb0 0x7f4bac0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49596/0 (socket says 172.21.15.53:49596) 2023-10-21T16:01:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4ba3fff640 1 -- 172.21.15.53:0/4145258007 learned_addr learned my addr 172.21.15.53:0/4145258007 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:48.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4145258007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4babfff640 1 --2- 172.21.15.53:0/4145258007 >> v2:172.21.15.124:3300/0 conn(0x7f4bac15aad0 0x7f4bac0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bab7fe640 1 -- 172.21.15.53:0/4145258007 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676480988 0 0) 0x7f4bac0cb580 con 0x7f4bac15aeb0 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.368+0000 7f4bab7fe640 1 -- 172.21.15.53:0/4145258007 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b98003520 con 0x7f4bac15aeb0 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.369+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4145258007 >> v2:172.21.15.124:3300/0 conn(0x7f4bac15aad0 msgr2=0x7f4bac0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.369+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4145258007 >> v2:172.21.15.124:3300/0 conn(0x7f4bac15aad0 0x7f4bac0cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.369+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4145258007 >> v1:172.21.15.53:6789/0 conn(0x7f4bac15aeb0 legacy=0x7f4bac0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.369+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4145258007 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0cc0e0 con 0x7f4bac0cc570 2023-10-21T16:01:48.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.370+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4145258007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac0cefa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009d90 tx=0x7f4ba402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cd0d753928f4e07 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.370+0000 7f4bab7fe640 1 -- 172.21.15.53:0/4145258007 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba402cce0 con 0x7f4bac0cc570 2023-10-21T16:01:48.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.370+0000 7f4bab7fe640 1 -- 172.21.15.53:0/4145258007 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba402ce40 con 0x7f4bac0cc570 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.370+0000 7f4bab7fe640 1 -- 172.21.15.53:0/4145258007 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4034660 con 0x7f4bac0cc570 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 -- 172.21.15.53:0/4145258007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 msgr2=0x7f4bac0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4145258007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac0cefa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009d90 tx=0x7f4ba402bf30 comp rx=0 tx=0).stop 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 -- 172.21.15.53:0/4145258007 shutdown_connections 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4145258007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac0cefa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4145258007 >> v2:172.21.15.124:3300/0 conn(0x7f4bac15aad0 0x7f4bac0cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 -- 172.21.15.53:0/4145258007 >> 172.21.15.53:0/4145258007 conn(0x7f4bac0d09e0 msgr2=0x7f4bac0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 -- 172.21.15.53:0/4145258007 shutdown_connections 2023-10-21T16:01:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.371+0000 7f4bb256c640 1 -- 172.21.15.53:0/4145258007 wait complete. 2023-10-21T16:01:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.372+0000 7f4bb256c640 1 Processor -- start 2023-10-21T16:01:48.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 -- start start 2023-10-21T16:01:48.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bac15aad0 0x7f4bac06f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac14f270 con 0x7f4bac15aeb0 2023-10-21T16:01:48.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0c6eb0 con 0x7f4bac15aad0 2023-10-21T16:01:48.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.373+0000 7f4bb256c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0c5390 con 0x7f4bac0cc570 2023-10-21T16:01:48.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58480/0 (socket says 172.21.15.53:58480) 2023-10-21T16:01:48.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4018137641 learned_addr learned my addr 172.21.15.53:0/4018137641 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:48.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 msgr2=0x7f4bac14ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bac15aad0 msgr2=0x7f4bac06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bac15aad0 0x7f4bac06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 -- 172.21.15.53:0/4018137641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4009a70 con 0x7f4bac15aeb0 2023-10-21T16:01:48.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.374+0000 7f4bb0b09640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009eb0 tx=0x7f4ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.375+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba403d600 con 0x7f4bac15aeb0 2023-10-21T16:01:48.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.375+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba403d780 con 0x7f4bac15aeb0 2023-10-21T16:01:48.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.375+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba403db30 con 0x7f4bac15aeb0 2023-10-21T16:01:48.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.375+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac0c7340 con 0x7f4bac15aeb0 2023-10-21T16:01:48.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.375+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac0c6860 con 0x7f4bac15aeb0 2023-10-21T16:01:48.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.376+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac0cb730 con 0x7f4bac15aeb0 2023-10-21T16:01:48.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.377+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f4ba403b040 con 0x7f4bac15aeb0 2023-10-21T16:01:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.377+0000 7f4ba97fa640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 0x7f4b8c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.377+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f4ba40ab560 con 0x7f4bac15aeb0 2023-10-21T16:01:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.378+0000 7f4babfff640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 0x7f4b8c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.378+0000 7f4babfff640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 0x7f4b8c06d2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006a30 tx=0x7f4b9c0069c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.381+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ba4076ba0 con 0x7f4bac15aeb0 2023-10-21T16:01:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.579+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bac067490 con 0x7f4bac15aeb0 2023-10-21T16:01:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.580+0000 7f4ba97fa640 1 -- 172.21.15.53:0/4018137641 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7f4ba404e090 con 0x7f4bac15aeb0 2023-10-21T16:01:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:48.583 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":13,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:01:47.773219+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:01:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 msgr2=0x7f4b8c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 0x7f4b8c06d2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006a30 tx=0x7f4b9c0069c0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 msgr2=0x7f4bac06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009eb0 tx=0x7f4ba4004a40 comp rx=0 tx=0).stop 2023-10-21T16:01:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 shutdown_connections 2023-10-21T16:01:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4b8c06ae30 0x7f4b8c06d2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bac15aeb0 0x7f4bac06fc40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bac15aad0 0x7f4bac06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 --2- 172.21.15.53:0/4018137641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bac0cc570 0x7f4bac14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.582+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 >> 172.21.15.53:0/4018137641 conn(0x7f4bac0d09e0 msgr2=0x7f4bac0ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.583+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 shutdown_connections 2023-10-21T16:01:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.583+0000 7f4bb256c640 1 -- 172.21.15.53:0/4018137641 wait complete. 2023-10-21T16:01:48.601 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-21T16:01:48.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:01:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.798+0000 7f879f59e640 1 Processor -- start 2023-10-21T16:01:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.798+0000 7f879f59e640 1 -- start start 2023-10-21T16:01:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87a00adc70 0x7f87a00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f87a00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a015f7d0 con 0x7f87a015ab30 2023-10-21T16:01:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a015efb0 con 0x7f87a00adc70 2023-10-21T16:01:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015f210 con 0x7f87a015af10 2023-10-21T16:01:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.799+0000 7f879e59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f87a015ab30 0x7f87a015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49610/0 (socket says 172.21.15.53:49610) 2023-10-21T16:01:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879e59c640 1 -- 172.21.15.53:0/3063200055 learned_addr learned my addr 172.21.15.53:0/3063200055 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:48.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029193802 0 0) 0x7f87a015f7d0 con 0x7f87a015ab30 2023-10-21T16:01:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f8797fff640 1 --2- 172.21.15.53:0/3063200055 >> v2:172.21.15.124:3300/0 conn(0x7f87a00adc70 0x7f87a00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f878c003520 con 0x7f87a015ab30 2023-10-21T16:01:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879ed9d640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f87a00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1768324146 0 0) 0x7f878c003520 con 0x7f87a015ab30 2023-10-21T16:01:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 msgr2=0x7f87a00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f87a00b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 >> v2:172.21.15.124:3300/0 conn(0x7f87a00adc70 msgr2=0x7f87a00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 --2- 172.21.15.53:0/3063200055 >> v2:172.21.15.124:3300/0 conn(0x7f87a00adc70 0x7f87a00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a015f4f0 con 0x7f87a015ab30 2023-10-21T16:01:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8790001690 con 0x7f87a015ab30 2023-10-21T16:01:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 >> v1:172.21.15.53:6789/0 conn(0x7f87a015ab30 legacy=0x7f87a015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c003970 0x7f878c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.800+0000 7f879dd9b640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f878c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879dd9b640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c004d00 0x7f878c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015efb0 con 0x7f878c004d00 2023-10-21T16:01:48.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015f210 con 0x7f878c003970 2023-10-21T16:01:48.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c00bab0 con 0x7f87a015af10 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c004d00 0x7f878c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879e59c640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c003970 0x7f878c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f8797fff640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f878c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 -- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 msgr2=0x7f878c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a015af10 0x7f878c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 -- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c003970 msgr2=0x7f878c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:48.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c003970 0x7f878c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879ed9d640 1 -- 172.21.15.53:0/3063200055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c003520 con 0x7f878c004d00 2023-10-21T16:01:48.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.801+0000 7f879e59c640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c003970 0x7f878c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:01:48.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.803+0000 7f879ed9d640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c004d00 0x7f878c005180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8798007360 tx=0x7f878c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f056f689f99de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.803+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879802ccd0 con 0x7f878c004d00 2023-10-21T16:01:48.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.803+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879802ce30 con 0x7f878c004d00 2023-10-21T16:01:48.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.803+0000 7f879dd9b640 1 -- 172.21.15.53:0/3063200055 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798034660 con 0x7f878c004d00 2023-10-21T16:01:48.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 -- 172.21.15.53:0/3063200055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c004d00 msgr2=0x7f878c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 --2- 172.21.15.53:0/3063200055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c004d00 0x7f878c005180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8798007360 tx=0x7f878c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879ed9d640 1 -- 172.21.15.53:0/3063200055 reap_dead start 2023-10-21T16:01:48.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 -- 172.21.15.53:0/3063200055 shutdown_connections 2023-10-21T16:01:48.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 -- 172.21.15.53:0/3063200055 >> 172.21.15.53:0/3063200055 conn(0x7f87a00cb0c0 msgr2=0x7f87a00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:48.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 -- 172.21.15.53:0/3063200055 shutdown_connections 2023-10-21T16:01:48.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.804+0000 7f879f59e640 1 -- 172.21.15.53:0/3063200055 wait complete. 2023-10-21T16:01:48.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.805+0000 7f879f59e640 1 Processor -- start 2023-10-21T16:01:48.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.806+0000 7f879f59e640 1 -- start start 2023-10-21T16:01:48.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.806+0000 7f879f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a00acdf0 0x7f87a015dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.806+0000 7f879f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a00ad1d0 0x7f87a00c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.806+0000 7f879f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015cd00 con 0x7f87a0157710 2023-10-21T16:01:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015cfe0 con 0x7f87a00acdf0 2023-10-21T16:01:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015d2c0 con 0x7f87a00ad1d0 2023-10-21T16:01:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58496/0 (socket says 172.21.15.53:58496) 2023-10-21T16:01:48.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 -- 172.21.15.53:0/4139583956 learned_addr learned my addr 172.21.15.53:0/4139583956 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:48.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879e59c640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a00acdf0 0x7f87a015dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f8797fff640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a00ad1d0 0x7f87a00c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 -- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a00ad1d0 msgr2=0x7f87a00c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a00ad1d0 0x7f87a00c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 -- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a00acdf0 msgr2=0x7f87a015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:48.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a00acdf0 0x7f87a015dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:48.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.807+0000 7f879ed9d640 1 -- 172.21.15.53:0/4139583956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a015d5a0 con 0x7f87a0157710 2023-10-21T16:01:48.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f879ed9d640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8798037040 tx=0x7f8798004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879803e640 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879803e7a0 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879803eb30 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a015d880 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.808+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a00c6190 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.809+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a006c530 con 0x7f87a0157710 2023-10-21T16:01:48.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.810+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f879803ec90 con 0x7f87a0157710 2023-10-21T16:01:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.811+0000 7f8796ffd640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 0x7f878006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.811+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f87980ab680 con 0x7f87a0157710 2023-10-21T16:01:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.812+0000 7f879e59c640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 0x7f878006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:48.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.813+0000 7f879e59c640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 0x7f878006d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8790004840 tx=0x7f8790009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:48.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:48.815+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87a006c530 con 0x7f87a0157710 2023-10-21T16:01:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.046+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87a006c530 con 0x7f87a0157710 2023-10-21T16:01:49.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.094+0000 7f8796ffd640 1 -- 172.21.15.53:0/4139583956 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f87a006c530 con 0x7f87a0157710 2023-10-21T16:01:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:49.097 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-21T16:01:49.097 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 2 2023-10-21T16:01:49.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 msgr2=0x7f878006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:49.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 0x7f878006d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8790004840 tx=0x7f8790009040 comp rx=0 tx=0).stop 2023-10-21T16:01:49.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 msgr2=0x7f87a00c5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:49.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8798037040 tx=0x7f8798004ab0 comp rx=0 tx=0).stop 2023-10-21T16:01:49.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 shutdown_connections 2023-10-21T16:01:49.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f878006ade0 0x7f878006d2a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:49.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a0157710 0x7f87a00c5c50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:49.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a00ad1d0 0x7f87a00c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:49.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 --2- 172.21.15.53:0/4139583956 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a00acdf0 0x7f87a015dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:49.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 >> 172.21.15.53:0/4139583956 conn(0x7f87a00cb0c0 msgr2=0x7f87a00af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:49.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 shutdown_connections 2023-10-21T16:01:49.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:49.097+0000 7f879f59e640 1 -- 172.21.15.53:0/4139583956 wait complete. 2023-10-21T16:01:51.119 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:01:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.362+0000 7fca07187640 1 Processor -- start 2023-10-21T16:01:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.362+0000 7fca07187640 1 -- start start 2023-10-21T16:01:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca000d16f0 0x7fca000cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 0x7fca000ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0015d040 con 0x7fca000d16f0 2023-10-21T16:01:51.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca000d0650 con 0x7fca000cdfd0 2023-10-21T16:01:51.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca07187640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca000d0930 con 0x7fca0015cc60 2023-10-21T16:01:51.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca000d16f0 0x7fca000cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fca000d16f0 0x7fca000cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58510/0 (socket says 172.21.15.53:58510) 2023-10-21T16:01:51.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 -- 172.21.15.53:0/342789171 learned_addr learned my addr 172.21.15.53:0/342789171 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca06185640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca06986640 1 --2- 172.21.15.53:0/342789171 >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 0x7fca000ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 -- 172.21.15.53:0/342789171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 msgr2=0x7fca000cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.364+0000 7fca05984640 1 -- 172.21.15.53:0/342789171 >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 msgr2=0x7fca000ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.365+0000 7fca05984640 1 --2- 172.21.15.53:0/342789171 >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 0x7fca000ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.365+0000 7fca05984640 1 -- 172.21.15.53:0/342789171 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca000d0c10 con 0x7fca000d16f0 2023-10-21T16:01:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.365+0000 7fca06986640 1 --2- 172.21.15.53:0/342789171 >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 0x7fca000ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.366+0000 7fca05984640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000cf880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca000cf0e0 tx=0x7fc9f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e3c3969eedde5e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.366+0000 7fca05183640 1 -- 172.21.15.53:0/342789171 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0031870 con 0x7fca000d16f0 2023-10-21T16:01:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.366+0000 7fca05183640 1 -- 172.21.15.53:0/342789171 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f00319d0 con 0x7fca000d16f0 2023-10-21T16:01:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.366+0000 7fca05183640 1 -- 172.21.15.53:0/342789171 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0031d60 con 0x7fca000d16f0 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 -- 172.21.15.53:0/342789171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 msgr2=0x7fca000cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000cf880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca000cf0e0 tx=0x7fc9f002b910 comp rx=0 tx=0).stop 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 -- 172.21.15.53:0/342789171 shutdown_connections 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 --2- 172.21.15.53:0/342789171 >> v2:172.21.15.124:3300/0 conn(0x7fca000cdfd0 0x7fca000ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000cf880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 --2- 172.21.15.53:0/342789171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 -- 172.21.15.53:0/342789171 >> 172.21.15.53:0/342789171 conn(0x7fca0009bcf0 msgr2=0x7fca000d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 -- 172.21.15.53:0/342789171 shutdown_connections 2023-10-21T16:01:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.367+0000 7fca07187640 1 -- 172.21.15.53:0/342789171 wait complete. 2023-10-21T16:01:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.369+0000 7fca07187640 1 Processor -- start 2023-10-21T16:01:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.369+0000 7fca07187640 1 -- start start 2023-10-21T16:01:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca000cdfd0 0x7fca001f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca001f6330 con 0x7fca000d16f0 2023-10-21T16:01:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca001f5a00 con 0x7fca000cdfd0 2023-10-21T16:01:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca07187640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca001f5ce0 con 0x7fca0015cc60 2023-10-21T16:01:51.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca05984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca05984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58524/0 (socket says 172.21.15.53:58524) 2023-10-21T16:01:51.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.370+0000 7fca05984640 1 -- 172.21.15.53:0/3839782499 learned_addr learned my addr 172.21.15.53:0/3839782499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:51.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca06185640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca000cdfd0 0x7fca001f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca06986640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 -- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 msgr2=0x7fca000b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 -- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca000cdfd0 msgr2=0x7fca001f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca000cdfd0 0x7fca001f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 -- 172.21.15.53:0/3839782499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00094d0 con 0x7fca000d16f0 2023-10-21T16:01:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca06986640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca05984640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00090b0 tx=0x7fc9f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f003ad20 con 0x7fca000d16f0 2023-10-21T16:01:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f00443f0 con 0x7fca000d16f0 2023-10-21T16:01:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f003aea0 con 0x7fca000d16f0 2023-10-21T16:01:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.371+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca001f5fc0 con 0x7fca000d16f0 2023-10-21T16:01:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.372+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca001f67c0 con 0x7fca000d16f0 2023-10-21T16:01:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.373+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0004e160 con 0x7fca000d16f0 2023-10-21T16:01:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.374+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fc9f004d450 con 0x7fca000d16f0 2023-10-21T16:01:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.375+0000 7fc9f6ffd640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 0x7fc9e406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.375+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9f002d040 con 0x7fca000d16f0 2023-10-21T16:01:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.376+0000 7fca06185640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 0x7fc9e406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.377+0000 7fca06185640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 0x7fc9e406d2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009ab0 tx=0x7fc9fc003990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.381+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9f0076c10 con 0x7fca000d16f0 2023-10-21T16:01:51.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.622+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca000d3080 con 0x7fca000d16f0 2023-10-21T16:01:51.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.623+0000 7fc9f6ffd640 1 -- 172.21.15.53:0/3839782499 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3702 (secure 0 0 0) 0x7fc9f0035070 con 0x7fca000d16f0 2023-10-21T16:01:51.625 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:51.625 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34163,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-21T16:01:51.626 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 3 2023-10-21T16:01:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 msgr2=0x7fc9e406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 0x7fc9e406d2d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc009ab0 tx=0x7fc9fc003990 comp rx=0 tx=0).stop 2023-10-21T16:01:51.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 msgr2=0x7fca000b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00090b0 tx=0x7fc9f00046c0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 shutdown_connections 2023-10-21T16:01:51.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca0015cc60 0x7fca000b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc9e406ae10 0x7fc9e406d2d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca000d16f0 0x7fca000b7730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 --2- 172.21.15.53:0/3839782499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca000cdfd0 0x7fca001f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 >> 172.21.15.53:0/3839782499 conn(0x7fca0009bcf0 msgr2=0x7fca0006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:51.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 shutdown_connections 2023-10-21T16:01:51.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.626+0000 7fca07187640 1 -- 172.21.15.53:0/3839782499 wait complete. 2023-10-21T16:01:51.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:01:51.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.844+0000 7ff984f1d640 1 Processor -- start 2023-10-21T16:01:51.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.845+0000 7ff984f1d640 1 -- start start 2023-10-21T16:01:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff984f1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff980069fc0 0x7ff980071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff984f1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff98006a3a0 0x7ff980161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff984f1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff980072480 con 0x7ff98015fdb0 2023-10-21T16:01:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff984f1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff980161980 con 0x7ff980069fc0 2023-10-21T16:01:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff984f1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9800ce280 con 0x7ff98006a3a0 2023-10-21T16:01:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff97e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff980069fc0 0x7ff980071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff97e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff980069fc0 0x7ff980071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58530/0 (socket says 172.21.15.53:58530) 2023-10-21T16:01:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.846+0000 7ff97e59c640 1 -- 172.21.15.53:0/3294898574 learned_addr learned my addr 172.21.15.53:0/3294898574 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:51.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.847+0000 7ff977fff640 1 --2- 172.21.15.53:0/3294898574 >> v2:172.21.15.124:3300/0 conn(0x7ff98006a3a0 0x7ff980161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.847+0000 7ff97e59c640 1 -- 172.21.15.53:0/3294898574 >> v2:172.21.15.124:3300/0 conn(0x7ff98006a3a0 msgr2=0x7ff980161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:51.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.847+0000 7ff97e59c640 1 --2- 172.21.15.53:0/3294898574 >> v2:172.21.15.124:3300/0 conn(0x7ff98006a3a0 0x7ff980161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.847+0000 7ff97e59c640 1 -- 172.21.15.53:0/3294898574 >> v1:172.21.15.124:6789/0 conn(0x7ff98015fdb0 legacy=0x7ff980160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.847+0000 7ff97e59c640 1 -- 172.21.15.53:0/3294898574 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800ce560 con 0x7ff980069fc0 2023-10-21T16:01:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.848+0000 7ff97e59c640 1 --2- 172.21.15.53:0/3294898574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff980069fc0 0x7ff980071f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9800b2470 tx=0x7ff97002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c0bcd9825dad7c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff97dd9b640 1 -- 172.21.15.53:0/3294898574 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004230 con 0x7ff980069fc0 2023-10-21T16:01:51.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff97dd9b640 1 -- 172.21.15.53:0/3294898574 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970004390 con 0x7ff980069fc0 2023-10-21T16:01:51.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff97dd9b640 1 -- 172.21.15.53:0/3294898574 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970030840 con 0x7ff980069fc0 2023-10-21T16:01:51.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 -- 172.21.15.53:0/3294898574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff980069fc0 msgr2=0x7ff980071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 --2- 172.21.15.53:0/3294898574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff980069fc0 0x7ff980071f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9800b2470 tx=0x7ff97002b980 comp rx=0 tx=0).stop 2023-10-21T16:01:51.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 -- 172.21.15.53:0/3294898574 shutdown_connections 2023-10-21T16:01:51.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 --2- 172.21.15.53:0/3294898574 >> v2:172.21.15.124:3300/0 conn(0x7ff98006a3a0 0x7ff980161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 --2- 172.21.15.53:0/3294898574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff980069fc0 0x7ff980071f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 -- 172.21.15.53:0/3294898574 >> 172.21.15.53:0/3294898574 conn(0x7ff98006d450 msgr2=0x7ff98006d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:51.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 -- 172.21.15.53:0/3294898574 shutdown_connections 2023-10-21T16:01:51.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.849+0000 7ff984f1d640 1 -- 172.21.15.53:0/3294898574 wait complete. 2023-10-21T16:01:51.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.851+0000 7ff984f1d640 1 Processor -- start 2023-10-21T16:01:51.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.851+0000 7ff984f1d640 1 -- start start 2023-10-21T16:01:51.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff980069fc0 0x7ff9800c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98006a3a0 0x7ff9801f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800cb960 con 0x7ff98015fdb0 2023-10-21T16:01:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800c7c60 con 0x7ff980069fc0 2023-10-21T16:01:51.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff984f1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800c7f40 con 0x7ff98006a3a0 2023-10-21T16:01:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.852+0000 7ff97ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58536/0 (socket says 172.21.15.53:58536) 2023-10-21T16:01:51.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 -- 172.21.15.53:0/109891709 learned_addr learned my addr 172.21.15.53:0/109891709 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97e59c640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff980069fc0 0x7ff9800c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff977fff640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98006a3a0 0x7ff9801f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 -- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98006a3a0 msgr2=0x7ff9801f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98006a3a0 0x7ff9801f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 -- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff980069fc0 msgr2=0x7ff9800c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff980069fc0 0x7ff9800c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 -- 172.21.15.53:0/109891709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009460 con 0x7ff98015fdb0 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.853+0000 7ff97ed9d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b970 tx=0x7ff97800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.854+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800cb40 con 0x7ff98015fdb0 2023-10-21T16:01:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.854+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97800cca0 con 0x7ff98015fdb0 2023-10-21T16:01:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.854+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978014680 con 0x7ff98015fdb0 2023-10-21T16:01:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.854+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800c8220 con 0x7ff98015fdb0 2023-10-21T16:01:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.854+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800cb4f0 con 0x7ff98015fdb0 2023-10-21T16:01:51.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.855+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980066d80 con 0x7ff98015fdb0 2023-10-21T16:01:51.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.856+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7ff97801d3b0 con 0x7ff98015fdb0 2023-10-21T16:01:51.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.857+0000 7ff976ffd640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 0x7ff96006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:51.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.857+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff978089080 con 0x7ff98015fdb0 2023-10-21T16:01:51.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.858+0000 7ff97e59c640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 0x7ff96006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:51.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.859+0000 7ff97e59c640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 0x7ff96006d2d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff970035000 tx=0x7ff970008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:51.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:51.861+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9780bbca0 con 0x7ff98015fdb0 2023-10-21T16:01:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.062+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff980071450 con 0x7ff98015fdb0 2023-10-21T16:01:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.063+0000 7ff976ffd640 1 -- 172.21.15.53:0/109891709 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7ff9780544c0 con 0x7ff98015fdb0 2023-10-21T16:01:52.066 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:52.066 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:01:51.901142+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:01:52.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.066+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 msgr2=0x7ff96006d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.066+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 0x7ff96006d2d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff970035000 tx=0x7ff970008030 comp rx=0 tx=0).stop 2023-10-21T16:01:52.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.066+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 msgr2=0x7ff9800b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.066+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b970 tx=0x7ff97800be30 comp rx=0 tx=0).stop 2023-10-21T16:01:52.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 shutdown_connections 2023-10-21T16:01:52.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff96006ae10 0x7ff96006d2d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff98015fdb0 0x7ff9800b48e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff98006a3a0 0x7ff9801f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 --2- 172.21.15.53:0/109891709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff980069fc0 0x7ff9800c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 >> 172.21.15.53:0/109891709 conn(0x7ff98006d450 msgr2=0x7ff98015b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:52.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 shutdown_connections 2023-10-21T16:01:52.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.067+0000 7ff984f1d640 1 -- 172.21.15.53:0/109891709 wait complete. 2023-10-21T16:01:52.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:01:52.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:01:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.281+0000 7f1a21f2c640 1 Processor -- start 2023-10-21T16:01:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.281+0000 7f1a21f2c640 1 -- start start 2023-10-21T16:01:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.282+0000 7f1a21f2c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a21f2c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a1c0cd070 0x7f1a1c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a21f2c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c0cd480 con 0x7f1a1c15cc80 2023-10-21T16:01:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a21f2c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c06a130 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a21f2c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c06a410 con 0x7f1a1c0cd070 2023-10-21T16:01:52.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58546/0 (socket says 172.21.15.53:58546) 2023-10-21T16:01:52.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 -- 172.21.15.53:0/795757876 learned_addr learned my addr 172.21.15.53:0/795757876 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1bfff640 1 --2- 172.21.15.53:0/795757876 >> v2:172.21.15.124:3300/0 conn(0x7f1a1c0cd070 0x7f1a1c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 -- 172.21.15.53:0/795757876 >> v2:172.21.15.124:3300/0 conn(0x7f1a1c0cd070 msgr2=0x7f1a1c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 --2- 172.21.15.53:0/795757876 >> v2:172.21.15.124:3300/0 conn(0x7f1a1c0cd070 0x7f1a1c0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 -- 172.21.15.53:0/795757876 >> v1:172.21.15.53:6789/0 conn(0x7f1a1c15cc80 legacy=0x7f1a1c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.283+0000 7f1a1affd640 1 -- 172.21.15.53:0/795757876 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c06a6f0 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.284+0000 7f1a1affd640 1 --2- 172.21.15.53:0/795757876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c071ff0 tx=0x7f1a1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dcd1d3933695f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a1a7fc640 1 -- 172.21.15.53:0/795757876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10030860 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a1a7fc640 1 -- 172.21.15.53:0/795757876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a100309c0 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a1a7fc640 1 -- 172.21.15.53:0/795757876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10030d50 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a21f2c640 1 -- 172.21.15.53:0/795757876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c06b1d0 msgr2=0x7f1a1c0ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/795757876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c071ff0 tx=0x7f1a1002b9a0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a21f2c640 1 -- 172.21.15.53:0/795757876 shutdown_connections 2023-10-21T16:01:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/795757876 >> v2:172.21.15.124:3300/0 conn(0x7f1a1c0cd070 0x7f1a1c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.285+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/795757876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c06b1d0 0x7f1a1c0ccb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.286+0000 7f1a21f2c640 1 -- 172.21.15.53:0/795757876 >> 172.21.15.53:0/795757876 conn(0x7f1a1c0b0ae0 msgr2=0x7f1a1c0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.286+0000 7f1a21f2c640 1 -- 172.21.15.53:0/795757876 shutdown_connections 2023-10-21T16:01:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.286+0000 7f1a21f2c640 1 -- 172.21.15.53:0/795757876 wait complete. 2023-10-21T16:01:52.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.287+0000 7f1a21f2c640 1 Processor -- start 2023-10-21T16:01:52.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.288+0000 7f1a21f2c640 1 -- start start 2023-10-21T16:01:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 0x7f1a1c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 0x7f1a1c0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0cb650 con 0x7f1a1c0cd070 2023-10-21T16:01:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0c2ea0 con 0x7f1a1c15cc80 2023-10-21T16:01:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a21f2c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c15d5a0 con 0x7f1a1c06b1d0 2023-10-21T16:01:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58550/0 (socket says 172.21.15.53:58550) 2023-10-21T16:01:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 -- 172.21.15.53:0/3663050451 learned_addr learned my addr 172.21.15.53:0/3663050451 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1bfff640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 0x7f1a1c0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1b7fe640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 0x7f1a1c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 -- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 msgr2=0x7f1a1c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 0x7f1a1c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 -- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 msgr2=0x7f1a1c0c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 0x7f1a1c0c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1affd640 1 -- 172.21.15.53:0/3663050451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a100094d0 con 0x7f1a1c0cd070 2023-10-21T16:01:52.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1bfff640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 0x7f1a1c0c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:52.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.289+0000 7f1a1b7fe640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 0x7f1a1c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:52.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a1affd640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a1002b9e0 tx=0x7f1a10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10030850 con 0x7f1a1c0cd070 2023-10-21T16:01:52.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a10039d90 con 0x7f1a1c0cd070 2023-10-21T16:01:52.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10043650 con 0x7f1a1c0cd070 2023-10-21T16:01:52.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0cb090 con 0x7f1a1c0cd070 2023-10-21T16:01:52.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.290+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c0c2b30 con 0x7f1a1c0cd070 2023-10-21T16:01:52.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.292+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c0c3010 con 0x7f1a1c0cd070 2023-10-21T16:01:52.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.293+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f1a1002f3b0 con 0x7f1a1c0cd070 2023-10-21T16:01:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.293+0000 7f1a03fff640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 0x7f1a0806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.294+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1a100ab670 con 0x7f1a1c0cd070 2023-10-21T16:01:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.294+0000 7f1a1b7fe640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 0x7f1a0806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.295+0000 7f1a1b7fe640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 0x7f1a0806d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000f60 tx=0x7f1a0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.298+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a10076bd0 con 0x7f1a1c0cd070 2023-10-21T16:01:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.495+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a1c04e160 con 0x7f1a1c0cd070 2023-10-21T16:01:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.496+0000 7f1a03fff640 1 -- 172.21.15.53:0/3663050451 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a10048020 con 0x7f1a1c0cd070 2023-10-21T16:01:52.498 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:01:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 msgr2=0x7f1a0806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 0x7f1a0806d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000f60 tx=0x7f1a0c009040 comp rx=0 tx=0).stop 2023-10-21T16:01:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 msgr2=0x7f1a1c0c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a1002b9e0 tx=0x7f1a10004730 comp rx=0 tx=0).stop 2023-10-21T16:01:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 shutdown_connections 2023-10-21T16:01:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a1c15cc80 0x7f1a1c0c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1a0806ade0 0x7f1a0806d2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a1c0cd070 0x7f1a1c0c4cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 --2- 172.21.15.53:0/3663050451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a1c06b1d0 0x7f1a1c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 >> 172.21.15.53:0/3663050451 conn(0x7f1a1c0b0ae0 msgr2=0x7f1a1c15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 shutdown_connections 2023-10-21T16:01:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.499+0000 7f1a21f2c640 1 -- 172.21.15.53:0/3663050451 wait complete. 2023-10-21T16:01:52.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:01:52.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.741+0000 7f9ba759e640 1 Processor -- start 2023-10-21T16:01:52.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.741+0000 7f9ba759e640 1 -- start start 2023-10-21T16:01:52.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.742+0000 7f9ba759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba81614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba80c9020 con 0x7f9ba815cca0 2023-10-21T16:01:52.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba80ccc20 con 0x7f9ba80b07d0 2023-10-21T16:01:52.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c92e0 con 0x7f9ba80cab40 2023-10-21T16:01:52.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9ba815cca0 0x7f9ba80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49628/0 (socket says 172.21.15.53:49628) 2023-10-21T16:01:52.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba659c640 1 -- 172.21.15.53:0/3137508478 learned_addr learned my addr 172.21.15.53:0/3137508478 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243804793 0 0) 0x7f9ba80c9020 con 0x7f9ba815cca0 2023-10-21T16:01:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b90003520 con 0x7f9ba815cca0 2023-10-21T16:01:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.743+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba81614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533701404 0 0) 0x7f9ba80ccc20 con 0x7f9ba80b07d0 2023-10-21T16:01:52.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba80c9020 con 0x7f9ba80b07d0 2023-10-21T16:01:52.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1577349928 0 0) 0x7f9b90003520 con 0x7f9ba815cca0 2023-10-21T16:01:52.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 msgr2=0x7f9ba81614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:52.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba81614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 >> v1:172.21.15.124:6789/0 conn(0x7f9ba80b07d0 legacy=0x7f9ba80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80cc7b0 con 0x7f9ba815cca0 2023-10-21T16:01:52.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9b98001670 con 0x7f9ba815cca0 2023-10-21T16:01:52.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 >> v1:172.21.15.53:6789/0 conn(0x7f9ba815cca0 legacy=0x7f9ba80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9b90003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b900040e0 0x7f9b900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9b900051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c92e0 con 0x7f9b90004d70 2023-10-21T16:01:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000bc80 con 0x7f9b900040e0 2023-10-21T16:01:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000be40 con 0x7f9ba80cab40 2023-10-21T16:01:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.744+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9b900051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba659c640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9b90003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba5d9b640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b900040e0 0x7f9b900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba6d9d640 1 -- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 msgr2=0x7f9b90003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9b90003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba6d9d640 1 -- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b900040e0 msgr2=0x7f9b900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b900040e0 0x7f9b900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.745+0000 7f9ba6d9d640 1 -- 172.21.15.53:0/3137508478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90003520 con 0x7f9b90004d70 2023-10-21T16:01:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.746+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9b900051f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0007360 tx=0x7f9b900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca46ccc0b34880f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.746+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba002ccd0 con 0x7f9b90004d70 2023-10-21T16:01:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.746+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba002ce30 con 0x7f9b90004d70 2023-10-21T16:01:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.746+0000 7f9ba559a640 1 -- 172.21.15.53:0/3137508478 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0034660 con 0x7f9b90004d70 2023-10-21T16:01:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 -- 172.21.15.53:0/3137508478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 msgr2=0x7f9b900051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 --2- 172.21.15.53:0/3137508478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9b900051f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0007360 tx=0x7f9b900057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba6d9d640 1 -- 172.21.15.53:0/3137508478 reap_dead start 2023-10-21T16:01:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 -- 172.21.15.53:0/3137508478 shutdown_connections 2023-10-21T16:01:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 -- 172.21.15.53:0/3137508478 >> 172.21.15.53:0/3137508478 conn(0x7f9ba80cb0c0 msgr2=0x7f9ba80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 -- 172.21.15.53:0/3137508478 shutdown_connections 2023-10-21T16:01:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.747+0000 7f9ba759e640 1 -- 172.21.15.53:0/3137508478 wait complete. 2023-10-21T16:01:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.749+0000 7f9ba759e640 1 Processor -- start 2023-10-21T16:01:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.749+0000 7f9ba759e640 1 -- start start 2023-10-21T16:01:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba81588d0 0x7f9ba8158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 0x7f9ba81f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c5490 con 0x7f9b90004d70 2023-10-21T16:01:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8159290 con 0x7f9ba81f2600 2023-10-21T16:01:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba81f2fc0 con 0x7f9ba81588d0 2023-10-21T16:01:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58576/0 (socket says 172.21.15.53:58576) 2023-10-21T16:01:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 -- 172.21.15.53:0/160542630 learned_addr learned my addr 172.21.15.53:0/160542630 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 0x7f9ba81f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 -- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba81588d0 msgr2=0x7f9ba8158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba5d9b640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba81588d0 0x7f9ba8158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba81588d0 0x7f9ba8158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 -- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 msgr2=0x7f9ba81f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 0x7f9ba81f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba659c640 1 -- 172.21.15.53:0/160542630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba81f3530 con 0x7f9b90004d70 2023-10-21T16:01:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.751+0000 7f9ba6d9d640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 0x7f9ba81f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9ba659c640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800e730 tx=0x7f9b9800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800ee40 con 0x7f9b90004d70 2023-10-21T16:01:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9800c850 con 0x7f9b90004d70 2023-10-21T16:01:52.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800cc00 con 0x7f9b90004d70 2023-10-21T16:01:52.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba81f3250 con 0x7f9b90004d70 2023-10-21T16:01:52.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.752+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba81f3b00 con 0x7f9b90004d70 2023-10-21T16:01:52.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.754+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba804e160 con 0x7f9b90004d70 2023-10-21T16:01:52.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.754+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f9b9801d460 con 0x7f9b90004d70 2023-10-21T16:01:52.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.755+0000 7f9b96ffd640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 0x7f9b8806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:52.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.755+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b98089030 con 0x7f9b90004d70 2023-10-21T16:01:52.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.756+0000 7f9ba5d9b640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 0x7f9b8806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:52.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.757+0000 7f9ba5d9b640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 0x7f9b8806d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80cb5f0 tx=0x7f9b9c001060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:52.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.760+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b980545a0 con 0x7f9b90004d70 2023-10-21T16:01:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.956+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9ba80cb620 con 0x7f9b90004d70 2023-10-21T16:01:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.957+0000 7f9b96ffd640 1 -- 172.21.15.53:0/160542630 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9b98057bc0 con 0x7f9b90004d70 2023-10-21T16:01:52.959 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:01:52.960 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:01:52.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:01:52.960 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:01:52.960 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:01:52.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:01:52.961 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:01:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 msgr2=0x7f9b8806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 0x7f9b8806d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80cb5f0 tx=0x7f9b9c001060 comp rx=0 tx=0).stop 2023-10-21T16:01:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 msgr2=0x7f9ba8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:52.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800e730 tx=0x7f9b9800ebf0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 shutdown_connections 2023-10-21T16:01:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba81f2600 0x7f9ba81f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba81588d0 0x7f9ba8158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b8806ade0 0x7f9b8806d2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 --2- 172.21.15.53:0/160542630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b90004d70 0x7f9ba8070340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 >> 172.21.15.53:0/160542630 conn(0x7f9ba80cb0c0 msgr2=0x7f9ba80accf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 shutdown_connections 2023-10-21T16:01:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:52.960+0000 7f9ba759e640 1 -- 172.21.15.53:0/160542630 wait complete. 2023-10-21T16:01:52.981 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:01:52.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:01:53.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.195+0000 7f9d910a3640 1 Processor -- start 2023-10-21T16:01:53.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.195+0000 7f9d910a3640 1 -- start start 2023-10-21T16:01:53.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.196+0000 7f9d910a3640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d8c15aab0 0x7f9d8c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.196+0000 7f9d910a3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d8c0b0670 0x7f9d8c0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.196+0000 7f9d910a3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c15ae90 con 0x7f9d8c0caef0 2023-10-21T16:01:53.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.196+0000 7f9d910a3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0cbdb0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.196+0000 7f9d910a3640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0cb7f0 con 0x7f9d8c15aab0 2023-10-21T16:01:53.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d837fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9d8c0caef0 0x7f9d8c0aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49632/0 (socket says 172.21.15.53:49632) 2023-10-21T16:01:53.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d908a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d8c0b0670 0x7f9d8c0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:53.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d837fe640 1 -- 172.21.15.53:0/3819065 learned_addr learned my addr 172.21.15.53:0/3819065 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:53.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d8bfff640 1 --2- 172.21.15.53:0/3819065 >> v2:172.21.15.124:3300/0 conn(0x7f9d8c15aab0 0x7f9d8c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:53.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d908a2640 1 -- 172.21.15.53:0/3819065 >> v2:172.21.15.124:3300/0 conn(0x7f9d8c15aab0 msgr2=0x7f9d8c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:53.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d908a2640 1 --2- 172.21.15.53:0/3819065 >> v2:172.21.15.124:3300/0 conn(0x7f9d8c15aab0 0x7f9d8c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:53.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d908a2640 1 -- 172.21.15.53:0/3819065 >> v1:172.21.15.53:6789/0 conn(0x7f9d8c0caef0 legacy=0x7f9d8c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:53.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d8b7fe640 1 -- 172.21.15.53:0/3819065 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238314402 0 0) 0x7f9d74001110 con 0x7f9d8c0caef0 2023-10-21T16:01:53.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.197+0000 7f9d908a2640 1 -- 172.21.15.53:0/3819065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0cbad0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.198+0000 7f9d908a2640 1 --2- 172.21.15.53:0/3819065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c0b1c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009d90 tx=0x7f9d8402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321ab28ea5173f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:53.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d8b7fe640 1 -- 172.21.15.53:0/3819065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8402cbd0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d8b7fe640 1 -- 172.21.15.53:0/3819065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8402cd30 con 0x7f9d8c0b0670 2023-10-21T16:01:53.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d8b7fe640 1 -- 172.21.15.53:0/3819065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8402cea0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 -- 172.21.15.53:0/3819065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 msgr2=0x7f9d8c0b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:53.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 --2- 172.21.15.53:0/3819065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c0b1c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009d90 tx=0x7f9d8402be20 comp rx=0 tx=0).stop 2023-10-21T16:01:53.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 -- 172.21.15.53:0/3819065 shutdown_connections 2023-10-21T16:01:53.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 --2- 172.21.15.53:0/3819065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c0b1c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:53.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 --2- 172.21.15.53:0/3819065 >> v2:172.21.15.124:3300/0 conn(0x7f9d8c15aab0 0x7f9d8c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:53.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 -- 172.21.15.53:0/3819065 >> 172.21.15.53:0/3819065 conn(0x7f9d8c06c010 msgr2=0x7f9d8c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:53.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 -- 172.21.15.53:0/3819065 shutdown_connections 2023-10-21T16:01:53.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.199+0000 7f9d910a3640 1 -- 172.21.15.53:0/3819065 wait complete. 2023-10-21T16:01:53.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.201+0000 7f9d910a3640 1 Processor -- start 2023-10-21T16:01:53.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.201+0000 7f9d910a3640 1 -- start start 2023-10-21T16:01:53.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8c0caef0 0x7f9d8c0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d8c15aab0 0x7f9d8c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c15c910 con 0x7f9d8c0b0670 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c150f70 con 0x7f9d8c15aab0 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d910a3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c1f2ea0 con 0x7f9d8c0caef0 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:53.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58606/0 (socket says 172.21.15.53:58606) 2023-10-21T16:01:53.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d8bfff640 1 -- 172.21.15.53:0/2328309194 learned_addr learned my addr 172.21.15.53:0/2328309194 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:53.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.202+0000 7f9d837fe640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8c0caef0 0x7f9d8c0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:53.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 -- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8c0caef0 msgr2=0x7f9d8c0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:53.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8c0caef0 0x7f9d8c0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:53.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 -- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d8c15aab0 msgr2=0x7f9d8c0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:53.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d8c15aab0 0x7f9d8c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:53.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 -- 172.21.15.53:0/2328309194 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d84009a70 con 0x7f9d8c0b0670 2023-10-21T16:01:53.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d8bfff640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00e870 tx=0x7f9d7c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:53.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0094f0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c00c940 con 0x7f9d8c0b0670 2023-10-21T16:01:53.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00ccf0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c1f2290 con 0x7f9d8c0b0670 2023-10-21T16:01:53.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.203+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c1f2850 con 0x7f9d8c0b0670 2023-10-21T16:01:53.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.204+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c074150 con 0x7f9d8c0b0670 2023-10-21T16:01:53.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.206+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f9d7c013470 con 0x7f9d8c0b0670 2023-10-21T16:01:53.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.207+0000 7f9d897fa640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 0x7f9d6c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:53.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.207+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d7c0892d0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.207+0000 7f9d837fe640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 0x7f9d6c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:53.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.208+0000 7f9d837fe640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 0x7f9d6c06d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0b0040 tx=0x7f9d740079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:53.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.211+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d7c0547b0 con 0x7f9d8c0b0670 2023-10-21T16:01:53.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:53.406+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9d8c0c70e0 con 0x7f9d8c0b0670 2023-10-21T16:01:54.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.111+0000 7f9d897fa640 1 -- 172.21.15.53:0/2328309194 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9d7c057dd0 con 0x7f9d8c0b0670 2023-10-21T16:01:54.113 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:01:54.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.114+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 msgr2=0x7f9d6c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.114+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 0x7f9d6c06d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0b0040 tx=0x7f9d740079b0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.114+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 msgr2=0x7f9d8c150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.114+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00e870 tx=0x7f9d7c00ed30 comp rx=0 tx=0).stop 2023-10-21T16:01:54.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 shutdown_connections 2023-10-21T16:01:54.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d8c15aab0 0x7f9d8c0cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d8c0caef0 0x7f9d8c0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9d6c06ade0 0x7f9d6c06d2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 --2- 172.21.15.53:0/2328309194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d8c0b0670 0x7f9d8c150a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 >> 172.21.15.53:0/2328309194 conn(0x7f9d8c06c010 msgr2=0x7f9d8c0adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:54.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.115+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 shutdown_connections 2023-10-21T16:01:54.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.117+0000 7f9d910a3640 1 -- 172.21.15.53:0/2328309194 wait complete. 2023-10-21T16:01:54.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-21T16:01:54.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.356+0000 7fc4e3197640 1 Processor -- start 2023-10-21T16:01:54.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.356+0000 7fc4e3197640 1 -- start start 2023-10-21T16:01:54.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.357+0000 7fc4e3197640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.357+0000 7fc4e3197640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc4dc0b16e0 0x7fc4dc0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e3197640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc0b2890 con 0x7fc4dc0cba40 2023-10-21T16:01:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e3197640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc0cbeb0 con 0x7fc4dc0b16e0 2023-10-21T16:01:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e3197640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc15d8b0 con 0x7fc4dc159d50 2023-10-21T16:01:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e0f33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e0f33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:37488/0 (socket says 172.21.15.53:37488) 2023-10-21T16:01:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4dbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc4dc0b16e0 0x7fc4dc0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e0f33640 1 -- 172.21.15.53:0/924361729 learned_addr learned my addr 172.21.15.53:0/924361729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.358+0000 7fc4e1734640 1 --1- 172.21.15.53:0/924361729 >> v1:172.21.15.53:6789/0 conn(0x7fc4dc0cba40 0x7fc4dc0c9310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33750/0 (socket says 172.21.15.53:33750) 2023-10-21T16:01:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.359+0000 7fc4db7fe640 1 -- 172.21.15.53:0/924361729 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060816079 0 0) 0x7fc4dc0b2890 con 0x7fc4dc0cba40 2023-10-21T16:01:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.359+0000 7fc4e0f33640 1 -- 172.21.15.53:0/924361729 >> v2:172.21.15.124:3300/0 conn(0x7fc4dc0b16e0 msgr2=0x7fc4dc0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.359+0000 7fc4e0f33640 1 --2- 172.21.15.53:0/924361729 >> v2:172.21.15.124:3300/0 conn(0x7fc4dc0b16e0 0x7fc4dc0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.359+0000 7fc4e0f33640 1 -- 172.21.15.53:0/924361729 >> v1:172.21.15.53:6789/0 conn(0x7fc4dc0cba40 legacy=0x7fc4dc0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.359+0000 7fc4e0f33640 1 -- 172.21.15.53:0/924361729 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc0b1ff0 con 0x7fc4dc159d50 2023-10-21T16:01:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.361+0000 7fc4e0f33640 1 --2- 172.21.15.53:0/924361729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009e50 tx=0x7fc4cc02bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3b71c9146a65f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:54.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.370+0000 7fc4db7fe640 1 -- 172.21.15.53:0/924361729 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc02caf0 con 0x7fc4dc159d50 2023-10-21T16:01:54.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.370+0000 7fc4db7fe640 1 -- 172.21.15.53:0/924361729 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc02cc50 con 0x7fc4dc159d50 2023-10-21T16:01:54.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.370+0000 7fc4db7fe640 1 -- 172.21.15.53:0/924361729 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0343f0 con 0x7fc4dc159d50 2023-10-21T16:01:54.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 -- 172.21.15.53:0/924361729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 msgr2=0x7fc4dc15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 --2- 172.21.15.53:0/924361729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009e50 tx=0x7fc4cc02bf50 comp rx=0 tx=0).stop 2023-10-21T16:01:54.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 -- 172.21.15.53:0/924361729 shutdown_connections 2023-10-21T16:01:54.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 --2- 172.21.15.53:0/924361729 >> v2:172.21.15.124:3300/0 conn(0x7fc4dc0b16e0 0x7fc4dc0cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 --2- 172.21.15.53:0/924361729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc159d50 0x7fc4dc15e300 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 -- 172.21.15.53:0/924361729 >> 172.21.15.53:0/924361729 conn(0x7fc4dc06a140 msgr2=0x7fc4dc06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 -- 172.21.15.53:0/924361729 shutdown_connections 2023-10-21T16:01:54.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.371+0000 7fc4e3197640 1 -- 172.21.15.53:0/924361729 wait complete. 2023-10-21T16:01:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.373+0000 7fc4e3197640 1 Processor -- start 2023-10-21T16:01:54.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.373+0000 7fc4e3197640 1 -- start start 2023-10-21T16:01:54.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc0b16e0 0x7fc4dc0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4dc159d50 0x7fc4dc0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc152f80 con 0x7fc4dc0cba40 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc153260 con 0x7fc4dc159d50 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e3197640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc153540 con 0x7fc4dc0b16e0 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43818/0 (socket says 172.21.15.53:43818) 2023-10-21T16:01:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4dbfff640 1 -- 172.21.15.53:0/1190977993 learned_addr learned my addr 172.21.15.53:0/1190977993 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.374+0000 7fc4e0f33640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc0b16e0 0x7fc4dc0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4e1734640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4dc159d50 0x7fc4dc0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 -- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc0b16e0 msgr2=0x7fc4dc0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc0b16e0 0x7fc4dc0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 -- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4dc159d50 msgr2=0x7fc4dc0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4dc159d50 0x7fc4dc0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 -- 172.21.15.53:0/1190977993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009b30 con 0x7fc4dc0cba40 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.375+0000 7fc4dbfff640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e9b0 tx=0x7fc4c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.376+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4009670 con 0x7fc4dc0cba40 2023-10-21T16:01:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.376+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc14dcb0 con 0x7fc4dc0cba40 2023-10-21T16:01:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.376+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400c9a0 con 0x7fc4dc0cba40 2023-10-21T16:01:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.376+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400cd50 con 0x7fc4dc0cba40 2023-10-21T16:01:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.376+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc150420 con 0x7fc4dc0cba40 2023-10-21T16:01:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.377+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc071cb0 con 0x7fc4dc0cba40 2023-10-21T16:01:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.379+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4c4013470 con 0x7fc4dc0cba40 2023-10-21T16:01:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.380+0000 7fc4d97fa640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 0x7fc4bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.381+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc4c4089650 con 0x7fc4dc0cba40 2023-10-21T16:01:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.381+0000 7fc4e0f33640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 0x7fc4bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.382+0000 7fc4e0f33640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 0x7fc4bc06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc035000 tx=0x7fc4cc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:54.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.386+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4c4054910 con 0x7fc4dc0cba40 2023-10-21T16:01:54.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:54.587+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc4dc04e240 con 0x7fc4dc0cba40 2023-10-21T16:01:55.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.121+0000 7fc4d97fa640 1 -- 172.21.15.53:0/1190977993 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc4c4057f30 con 0x7fc4dc0cba40 2023-10-21T16:01:55.124 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:01:55.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.125+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 msgr2=0x7fc4bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.125+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 0x7fc4bc06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc035000 tx=0x7fc4cc030000 comp rx=0 tx=0).stop 2023-10-21T16:01:55.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.125+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 msgr2=0x7fc4dc14d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.125+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400e9b0 tx=0x7fc4c400ee70 comp rx=0 tx=0).stop 2023-10-21T16:01:55.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 shutdown_connections 2023-10-21T16:01:55.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4dc159d50 0x7fc4dc0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4bc06ae70 0x7fc4bc06d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4dc0cba40 0x7fc4dc14d410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 --2- 172.21.15.53:0/1190977993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4dc0b16e0 0x7fc4dc0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 >> 172.21.15.53:0/1190977993 conn(0x7fc4dc06a140 msgr2=0x7fc4dc0c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:55.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 shutdown_connections 2023-10-21T16:01:55.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.126+0000 7fc4e3197640 1 -- 172.21.15.53:0/1190977993 wait complete. 2023-10-21T16:01:55.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:01:55.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.351+0000 7fb78a8c9640 1 Processor -- start 2023-10-21T16:01:55.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.351+0000 7fb78a8c9640 1 -- start start 2023-10-21T16:01:55.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.352+0000 7fb78a8c9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb78406db60 0x7fb7840b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.352+0000 7fb78a8c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78415b310 con 0x7fb78406c3a0 2023-10-21T16:01:55.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.352+0000 7fb78a8c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78406d290 con 0x7fb7840c9cf0 2023-10-21T16:01:55.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.352+0000 7fb78a8c9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78415b110 con 0x7fb78406db60 2023-10-21T16:01:55.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb783fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb78406c3a0 0x7fb78406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33754/0 (socket says 172.21.15.53:33754) 2023-10-21T16:01:55.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb783fff640 1 -- 172.21.15.53:0/1940133769 learned_addr learned my addr 172.21.15.53:0/1940133769 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:55.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb788e66640 1 --2- 172.21.15.53:0/1940133769 >> v2:172.21.15.124:3300/0 conn(0x7fb78406db60 0x7fb7840b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573603506 0 0) 0x7fb774001110 con 0x7fb78406c3a0 2023-10-21T16:01:55.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb770003520 con 0x7fb78406c3a0 2023-10-21T16:01:55.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 779435307 0 0) 0x7fb770003520 con 0x7fb78406c3a0 2023-10-21T16:01:55.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 >> v2:172.21.15.124:3300/0 conn(0x7fb78406db60 msgr2=0x7fb7840b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:55.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 --2- 172.21.15.53:0/1940133769 >> v2:172.21.15.124:3300/0 conn(0x7fb78406db60 0x7fb7840b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 >> v1:172.21.15.124:6789/0 conn(0x7fb7840c9cf0 legacy=0x7fb78415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.353+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840c9860 con 0x7fb78406c3a0 2023-10-21T16:01:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb774001800 con 0x7fb78406c3a0 2023-10-21T16:01:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 >> v1:172.21.15.53:6789/0 conn(0x7fb78406c3a0 legacy=0x7fb78406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7700039c0 0x7fb770003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7700047c0 0x7fb770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 0x7fb77000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78415b110 con 0x7fb770005180 2023-10-21T16:01:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77000e290 con 0x7fb7700047c0 2023-10-21T16:01:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77000e450 con 0x7fb7700039c0 2023-10-21T16:01:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb788e66640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 0x7fb77000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb77bfff640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7700047c0 0x7fb770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb783fff640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7700039c0 0x7fb770003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb788e66640 1 -- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7700039c0 msgr2=0x7fb770003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb788e66640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7700039c0 0x7fb770003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb788e66640 1 -- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7700047c0 msgr2=0x7fb770004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:55.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.354+0000 7fb788e66640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7700047c0 0x7fb770004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.355+0000 7fb788e66640 1 -- 172.21.15.53:0/1940133769 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770003520 con 0x7fb770005180 2023-10-21T16:01:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.355+0000 7fb788e66640 1 -- 172.21.15.53:0/1940133769 reap_dead start 2023-10-21T16:01:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb788e66640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 0x7fb77000db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009fc0 tx=0x7fb7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcbf3052653e482 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c02cc60 con 0x7fb770005180 2023-10-21T16:01:55.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c00f040 con 0x7fb770005180 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb7837fe640 1 -- 172.21.15.53:0/1940133769 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003990 con 0x7fb770005180 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 -- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 msgr2=0x7fb77000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 0x7fb77000db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009fc0 tx=0x7fb7700057f0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 -- 172.21.15.53:0/1940133769 shutdown_connections 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/1940133769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb770005180 0x7fb77000db90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 -- 172.21.15.53:0/1940133769 >> 172.21.15.53:0/1940133769 conn(0x7fb78406a5d0 msgr2=0x7fb7840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:55.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.356+0000 7fb78a8c9640 1 -- 172.21.15.53:0/1940133769 shutdown_connections 2023-10-21T16:01:55.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.357+0000 7fb78a8c9640 1 -- 172.21.15.53:0/1940133769 wait complete. 2023-10-21T16:01:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.358+0000 7fb78a8c9640 1 Processor -- start 2023-10-21T16:01:55.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.358+0000 7fb78a8c9640 1 -- start start 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb770005180 0x7fb7840c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7840cdaa0 0x7fb7840cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840ce460 con 0x7fb78415c480 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78415ce40 con 0x7fb7840cdaa0 2023-10-21T16:01:55.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.359+0000 7fb78a8c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840cf270 con 0x7fb770005180 2023-10-21T16:01:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43838/0 (socket says 172.21.15.53:43838) 2023-10-21T16:01:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 -- 172.21.15.53:0/248031205 learned_addr learned my addr 172.21.15.53:0/248031205 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb783fff640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb770005180 0x7fb7840c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 -- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb770005180 msgr2=0x7fb7840c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb770005180 0x7fb7840c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 -- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7840cdaa0 msgr2=0x7fb7840cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7840cdaa0 0x7fb7840cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 -- 172.21.15.53:0/248031205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840ce660 con 0x7fb78415c480 2023-10-21T16:01:55.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.360+0000 7fb788e66640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e230 tx=0x7fb77c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:55.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.361+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0042f0 con 0x7fb78415c480 2023-10-21T16:01:55.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.361+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c004450 con 0x7fb78415c480 2023-10-21T16:01:55.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.361+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c004800 con 0x7fb78415c480 2023-10-21T16:01:55.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.361+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7840ce940 con 0x7fb78415c480 2023-10-21T16:01:55.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.361+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7840cef00 con 0x7fb78415c480 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.362+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78404e160 con 0x7fb78415c480 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.363+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb77c03f470 con 0x7fb78415c480 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.364+0000 7fb7817fa640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 0x7fb76406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.364+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb77c0abeb0 con 0x7fb78415c480 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.365+0000 7fb783fff640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 0x7fb76406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:55.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.366+0000 7fb783fff640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 0x7fb76406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb774001eb0 tx=0x7fb774009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:55.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.369+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb77c077040 con 0x7fb78415c480 2023-10-21T16:01:55.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:55.602+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7840705c0 con 0x7fb78415c480 2023-10-21T16:01:56.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.124+0000 7fb7817fa640 1 -- 172.21.15.53:0/248031205 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb77c07a660 con 0x7fb78415c480 2023-10-21T16:01:56.127 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 2 and data pool 3 2023-10-21T16:01:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 msgr2=0x7fb76406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 0x7fb76406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb774001eb0 tx=0x7fb774009040 comp rx=0 tx=0).stop 2023-10-21T16:01:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 msgr2=0x7fb78415c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e230 tx=0x7fb77c009750 comp rx=0 tx=0).stop 2023-10-21T16:01:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 shutdown_connections 2023-10-21T16:01:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb76406ae70 0x7fb76406d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb78415c480 0x7fb78415c900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7840cdaa0 0x7fb7840cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 --2- 172.21.15.53:0/248031205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb770005180 0x7fb7840c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 >> 172.21.15.53:0/248031205 conn(0x7fb78406a5d0 msgr2=0x7fb78415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:56.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 shutdown_connections 2023-10-21T16:01:56.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.128+0000 7fb78a8c9640 1 -- 172.21.15.53:0/248031205 wait complete. 2023-10-21T16:01:56.156 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-21T16:01:56.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-21T16:01:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.360+0000 7f87c5b08640 1 Processor -- start 2023-10-21T16:01:56.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.360+0000 7f87c5b08640 1 -- start start 2023-10-21T16:01:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87c01432f0 0x7f87c00cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c00b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 0x7f87c006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c01436d0 con 0x7f87c01432f0 2023-10-21T16:01:56.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c00cbb30 con 0x7f87c00b4070 2023-10-21T16:01:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87c5b08640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c015d440 con 0x7f87c00cb6b0 2023-10-21T16:01:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87c01432f0 0x7f87c00cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87c01432f0 0x7f87c00cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43848/0 (socket says 172.21.15.53:43848) 2023-10-21T16:01:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.362+0000 7f87bf7fe640 1 -- 172.21.15.53:0/323153872 learned_addr learned my addr 172.21.15.53:0/323153872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bffff640 1 --2- 172.21.15.53:0/323153872 >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 0x7f87c006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bf7fe640 1 -- 172.21.15.53:0/323153872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 msgr2=0x7f87c00b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87beffd640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c00b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c00b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bf7fe640 1 -- 172.21.15.53:0/323153872 >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 msgr2=0x7f87c006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/323153872 >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 0x7f87c006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bf7fe640 1 -- 172.21.15.53:0/323153872 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c015d690 con 0x7f87c01432f0 2023-10-21T16:01:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.363+0000 7f87bffff640 1 --2- 172.21.15.53:0/323153872 >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 0x7f87c006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:56.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.364+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c01432f0 0x7f87c00cb170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87c00ce750 tx=0x7f87b002e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6216dbf78d2ce8e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:56.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.364+0000 7f87be7fc640 1 -- 172.21.15.53:0/323153872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b002ec00 con 0x7f87c01432f0 2023-10-21T16:01:56.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.364+0000 7f87be7fc640 1 -- 172.21.15.53:0/323153872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b002ed60 con 0x7f87c01432f0 2023-10-21T16:01:56.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.364+0000 7f87be7fc640 1 -- 172.21.15.53:0/323153872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0036610 con 0x7f87c01432f0 2023-10-21T16:01:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 -- 172.21.15.53:0/323153872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c01432f0 msgr2=0x7f87c00cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c01432f0 0x7f87c00cb170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f87c00ce750 tx=0x7f87b002e060 comp rx=0 tx=0).stop 2023-10-21T16:01:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 -- 172.21.15.53:0/323153872 shutdown_connections 2023-10-21T16:01:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c00b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 --2- 172.21.15.53:0/323153872 >> v2:172.21.15.124:3300/0 conn(0x7f87c00b4070 0x7f87c006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 --2- 172.21.15.53:0/323153872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c01432f0 0x7f87c00cb170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 -- 172.21.15.53:0/323153872 >> 172.21.15.53:0/323153872 conn(0x7f87c0069d50 msgr2=0x7f87c006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:56.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 -- 172.21.15.53:0/323153872 shutdown_connections 2023-10-21T16:01:56.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.365+0000 7f87c5b08640 1 -- 172.21.15.53:0/323153872 wait complete. 2023-10-21T16:01:56.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.367+0000 7f87c5b08640 1 Processor -- start 2023-10-21T16:01:56.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.367+0000 7f87c5b08640 1 -- start start 2023-10-21T16:01:56.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c0138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 0x7f87c01f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0137210 con 0x7f87c00b4070 2023-10-21T16:01:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0157870 con 0x7f87c01432f0 2023-10-21T16:01:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87c5b08640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c0139b20 con 0x7f87c00cb6b0 2023-10-21T16:01:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43852/0 (socket says 172.21.15.53:43852) 2023-10-21T16:01:56.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.368+0000 7f87bf7fe640 1 -- 172.21.15.53:0/2179866209 learned_addr learned my addr 172.21.15.53:0/2179866209 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:56.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 -- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 msgr2=0x7f87c0138b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c0138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 -- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 msgr2=0x7f87c01f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bffff640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 0x7f87c01f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 0x7f87c01f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 -- 172.21.15.53:0/2179866209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009b30 con 0x7f87c00b4070 2023-10-21T16:01:56.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bffff640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 0x7f87c01f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:01:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87bf7fe640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87b0038040 tx=0x7f87b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.369+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b002ec00 con 0x7f87c00b4070 2023-10-21T16:01:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.370+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b003e3f0 con 0x7f87c00b4070 2023-10-21T16:01:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.370+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b003e7a0 con 0x7f87c00b4070 2023-10-21T16:01:56.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.370+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c0139560 con 0x7f87c00b4070 2023-10-21T16:01:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.370+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c0136ea0 con 0x7f87c00b4070 2023-10-21T16:01:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.371+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c0136d00 con 0x7f87c00b4070 2023-10-21T16:01:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.372+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f87b003e900 con 0x7f87c00b4070 2023-10-21T16:01:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.373+0000 7f87b7fff640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 0x7f87a006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:56.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.373+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87b00aba00 con 0x7f87c00b4070 2023-10-21T16:01:56.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.373+0000 7f87beffd640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 0x7f87a006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:56.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.374+0000 7f87beffd640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 0x7f87a006d380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006b10 tx=0x7f87a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:56.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.377+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87c0136d00 con 0x7f87c00b4070 2023-10-21T16:01:56.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.585+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f87c0136d00 con 0x7f87c00b4070 2023-10-21T16:01:56.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.907+0000 7f87b7fff640 1 -- 172.21.15.53:0/2179866209 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f87c0136d00 con 0x7f87c00b4070 2023-10-21T16:01:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.911+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 msgr2=0x7f87a006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.911+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 0x7f87a006d380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006b10 tx=0x7f87a8008040 comp rx=0 tx=0).stop 2023-10-21T16:01:56.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.911+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 msgr2=0x7f87c0157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:56.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.911+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87b0038040 tx=0x7f87b0004a40 comp rx=0 tx=0).stop 2023-10-21T16:01:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 shutdown_connections 2023-10-21T16:01:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c01432f0 0x7f87c01f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c00cb6b0 0x7f87c0138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f87a006aec0 0x7f87a006d380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 --2- 172.21.15.53:0/2179866209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c00b4070 0x7f87c0157330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.912+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 >> 172.21.15.53:0/2179866209 conn(0x7f87c0069d50 msgr2=0x7f87c006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.913+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 shutdown_connections 2023-10-21T16:01:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:56.913+0000 7f87c5b08640 1 -- 172.21.15.53:0/2179866209 wait complete. 2023-10-21T16:01:56.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:01:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.168+0000 7f9c7c1a5640 1 Processor -- start 2023-10-21T16:01:57.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.168+0000 7f9c7c1a5640 1 -- start start 2023-10-21T16:01:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.170+0000 7f9c7c1a5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c7406d710 0x7f9c740c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.170+0000 7f9c7c1a5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740c8980 con 0x7f9c7415acf0 2023-10-21T16:01:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.170+0000 7f9c7c1a5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740c9990 con 0x7f9c7406cb00 2023-10-21T16:01:57.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.170+0000 7f9c7c1a5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7406db20 con 0x7f9c7406d710 2023-10-21T16:01:57.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.171+0000 7f9c79f41640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9c7415acf0 0x7f9c7406c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33764/0 (socket says 172.21.15.53:33764) 2023-10-21T16:01:57.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.171+0000 7f9c79f41640 1 -- 172.21.15.53:0/2362571471 learned_addr learned my addr 172.21.15.53:0/2362571471 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:01:57.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.171+0000 7f9c7a742640 1 --2- 172.21.15.53:0/2362571471 >> v2:172.21.15.124:3300/0 conn(0x7f9c7406d710 0x7f9c740c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.171+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9121822 0 0) 0x7f9c740c8980 con 0x7f9c7415acf0 2023-10-21T16:01:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.171+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c60003520 con 0x7f9c7415acf0 2023-10-21T16:01:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.172+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388767078 0 0) 0x7f9c60003520 con 0x7f9c7415acf0 2023-10-21T16:01:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.172+0000 7f9c7a742640 1 -- 172.21.15.53:0/2362571471 >> v1:172.21.15.124:6789/0 conn(0x7f9c7406cb00 legacy=0x7f9c740c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:57.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.172+0000 7f9c7a742640 1 -- 172.21.15.53:0/2362571471 >> v1:172.21.15.53:6789/0 conn(0x7f9c7415acf0 legacy=0x7f9c7406c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:57.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.172+0000 7f9c7a742640 1 -- 172.21.15.53:0/2362571471 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740c9c70 con 0x7f9c7406d710 2023-10-21T16:01:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.173+0000 7f9c7a742640 1 --2- 172.21.15.53:0/2362571471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406d710 0x7f9c740c8f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c02bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2ed7ab7de74b7fa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70001670 con 0x7f9c7406d710 2023-10-21T16:01:57.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c02c830 con 0x7f9c7406d710 2023-10-21T16:01:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c78f3f640 1 -- 172.21.15.53:0/2362571471 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c02cbc0 con 0x7f9c7406d710 2023-10-21T16:01:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2362571471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406d710 msgr2=0x7f9c740c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2362571471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406d710 0x7f9c740c8f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c02bd00 comp rx=0 tx=0).stop 2023-10-21T16:01:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2362571471 shutdown_connections 2023-10-21T16:01:57.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2362571471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406d710 0x7f9c740c8f10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2362571471 >> 172.21.15.53:0/2362571471 conn(0x7f9c74069d50 msgr2=0x7f9c7406a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2362571471 shutdown_connections 2023-10-21T16:01:57.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.174+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2362571471 wait complete. 2023-10-21T16:01:57.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.176+0000 7f9c7c1a5640 1 Processor -- start 2023-10-21T16:01:57.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.177+0000 7f9c7c1a5640 1 -- start start 2023-10-21T16:01:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406cb00 0x7f9c741591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c7406d710 0x7f9c740c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:57.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7415ddf0 con 0x7f9c7415acf0 2023-10-21T16:01:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7415cd50 con 0x7f9c7406cb00 2023-10-21T16:01:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7c1a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7415d030 con 0x7f9c7406d710 2023-10-21T16:01:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7a742640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7a742640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43854/0 (socket says 172.21.15.53:43854) 2023-10-21T16:01:57.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c7a742640 1 -- 172.21.15.53:0/2142534520 learned_addr learned my addr 172.21.15.53:0/2142534520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c79740640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c7406d710 0x7f9c740c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.178+0000 7f9c79f41640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406cb00 0x7f9c741591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 -- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c7406d710 msgr2=0x7f9c740c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c7406d710 0x7f9c740c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 -- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406cb00 msgr2=0x7f9c741591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:57.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406cb00 0x7f9c741591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 -- 172.21.15.53:0/2142534520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009a70 con 0x7f9c7415acf0 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7a742640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c009450 con 0x7f9c7415acf0 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c034d00 con 0x7f9c7415acf0 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7415d310 con 0x7f9c7415acf0 2023-10-21T16:01:57.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.179+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c003bd0 con 0x7f9c7415acf0 2023-10-21T16:01:57.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.180+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7415d8d0 con 0x7f9c7415acf0 2023-10-21T16:01:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.181+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7415dac0 con 0x7f9c7415acf0 2023-10-21T16:01:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.182+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c6c03b050 con 0x7f9c7415acf0 2023-10-21T16:01:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.184+0000 7f9c6a7fc640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 0x7f9c5806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.184+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f9c6c0aba50 con 0x7f9c7415acf0 2023-10-21T16:01:57.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.184+0000 7f9c79f41640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 0x7f9c5806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:57.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.186+0000 7f9c79f41640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 0x7f9c5806d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c700017a0 tx=0x7f9c70001610 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:57.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.188+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c6c076a10 con 0x7f9c7415acf0 2023-10-21T16:01:57.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:57.390+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c74067490 con 0x7f9c7415acf0 2023-10-21T16:01:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.144+0000 7f9c6a7fc640 1 -- 172.21.15.53:0/2142534520 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f9c6c04d020 con 0x7f9c7415acf0 2023-10-21T16:01:59.147 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data_ec' created 2023-10-21T16:01:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 msgr2=0x7f9c5806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 0x7f9c5806d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c700017a0 tx=0x7f9c70001610 comp rx=0 tx=0).stop 2023-10-21T16:01:59.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 msgr2=0x7f9c740c2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:59.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009d90 tx=0x7f9c6c004a40 comp rx=0 tx=0).stop 2023-10-21T16:01:59.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 shutdown_connections 2023-10-21T16:01:59.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9c5806aea0 0x7f9c5806d360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c7415acf0 0x7f9c740c2930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c7406d710 0x7f9c740c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 --2- 172.21.15.53:0/2142534520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c7406cb00 0x7f9c741591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 >> 172.21.15.53:0/2142534520 conn(0x7f9c74069d50 msgr2=0x7f9c7406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:59.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.148+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 shutdown_connections 2023-10-21T16:01:59.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.149+0000 7f9c7c1a5640 1 -- 172.21.15.53:0/2142534520 wait complete. 2023-10-21T16:01:59.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-21T16:01:59.340 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-21T16:01:59.338+0000 7fe02d27c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:01:59.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.375+0000 7fc6a0947640 1 Processor -- start 2023-10-21T16:01:59.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.375+0000 7fc6a0947640 1 -- start start 2023-10-21T16:01:59.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc69c15aab0 0x7fc69c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc69c0caef0 0x7fc69c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0b0930 con 0x7fc69c0caef0 2023-10-21T16:01:59.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c15ae90 con 0x7fc69c15aab0 2023-10-21T16:01:59.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc6a0947640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0cbdb0 con 0x7fc69c0b04a0 2023-10-21T16:01:59.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc69affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc69c0caef0 0x7fc69c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:59.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc69affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc69c0caef0 0x7fc69c0aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43868/0 (socket says 172.21.15.53:43868) 2023-10-21T16:01:59.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.376+0000 7fc69affd640 1 -- 172.21.15.53:0/216455293 learned_addr learned my addr 172.21.15.53:0/216455293 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:59.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69b7fe640 1 --2- 172.21.15.53:0/216455293 >> v2:172.21.15.124:3300/0 conn(0x7fc69c15aab0 0x7fc69c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:59.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69affd640 1 -- 172.21.15.53:0/216455293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 msgr2=0x7fc69c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:59.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69affd640 1 --2- 172.21.15.53:0/216455293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69affd640 1 -- 172.21.15.53:0/216455293 >> v2:172.21.15.124:3300/0 conn(0x7fc69c15aab0 msgr2=0x7fc69c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:01:59.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69affd640 1 --2- 172.21.15.53:0/216455293 >> v2:172.21.15.124:3300/0 conn(0x7fc69c15aab0 0x7fc69c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.377+0000 7fc69affd640 1 -- 172.21.15.53:0/216455293 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0cb7f0 con 0x7fc69c0caef0 2023-10-21T16:01:59.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.378+0000 7fc69affd640 1 --2- 172.21.15.53:0/216455293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c0caef0 0x7fc69c0aff60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc684009870 tx=0x7fc68402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e52d64927c3fce server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:59.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc69a7fc640 1 -- 172.21.15.53:0/216455293 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840317c0 con 0x7fc69c0caef0 2023-10-21T16:01:59.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc69a7fc640 1 -- 172.21.15.53:0/216455293 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684031920 con 0x7fc69c0caef0 2023-10-21T16:01:59.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc69a7fc640 1 -- 172.21.15.53:0/216455293 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684031cb0 con 0x7fc69c0caef0 2023-10-21T16:01:59.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 -- 172.21.15.53:0/216455293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c0caef0 msgr2=0x7fc69c0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:59.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 --2- 172.21.15.53:0/216455293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c0caef0 0x7fc69c0aff60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc684009870 tx=0x7fc68402b860 comp rx=0 tx=0).stop 2023-10-21T16:01:59.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 -- 172.21.15.53:0/216455293 shutdown_connections 2023-10-21T16:01:59.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 --2- 172.21.15.53:0/216455293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c0c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 --2- 172.21.15.53:0/216455293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c0caef0 0x7fc69c0aff60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 --2- 172.21.15.53:0/216455293 >> v2:172.21.15.124:3300/0 conn(0x7fc69c15aab0 0x7fc69c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 -- 172.21.15.53:0/216455293 >> 172.21.15.53:0/216455293 conn(0x7fc69c06c010 msgr2=0x7fc69c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:01:59.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 -- 172.21.15.53:0/216455293 shutdown_connections 2023-10-21T16:01:59.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.379+0000 7fc6a0947640 1 -- 172.21.15.53:0/216455293 wait complete. 2023-10-21T16:01:59.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.381+0000 7fc6a0947640 1 Processor -- start 2023-10-21T16:01:59.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.382+0000 7fc6a0947640 1 -- start start 2023-10-21T16:01:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69c0caef0 0x7fc69c1f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c15c970 con 0x7fc69c15aab0 2023-10-21T16:01:59.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c14bf50 con 0x7fc69c0caef0 2023-10-21T16:01:59.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc6a0947640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1f5e20 con 0x7fc69c0b04a0 2023-10-21T16:01:59.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:59.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43882/0 (socket says 172.21.15.53:43882) 2023-10-21T16:01:59.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 -- 172.21.15.53:0/3326189244 learned_addr learned my addr 172.21.15.53:0/3326189244 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:01:59.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69affd640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69c0caef0 0x7fc69c1f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:59.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 -- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 msgr2=0x7fc69c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:59.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 -- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69c0caef0 msgr2=0x7fc69c1f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:01:59.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.383+0000 7fc69bfff640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69c0caef0 0x7fc69c1f5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:01:59.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc69bfff640 1 -- 172.21.15.53:0/3326189244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840094d0 con 0x7fc69c15aab0 2023-10-21T16:01:59.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc69bfff640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc694009fd0 tx=0x7fc69400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:59.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694011830 con 0x7fc69c15aab0 2023-10-21T16:01:59.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694011990 con 0x7fc69c15aab0 2023-10-21T16:01:59.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c1f6100 con 0x7fc69c15aab0 2023-10-21T16:01:59.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694011d20 con 0x7fc69c15aab0 2023-10-21T16:01:59.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.384+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c15c280 con 0x7fc69c15aab0 2023-10-21T16:01:59.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.385+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c15c470 con 0x7fc69c15aab0 2023-10-21T16:01:59.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.387+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc69401b460 con 0x7fc69c15aab0 2023-10-21T16:01:59.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.388+0000 7fc693fff640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 0x7fc67c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:01:59.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.388+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fc694089750 con 0x7fc69c15aab0 2023-10-21T16:01:59.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.388+0000 7fc69b7fe640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 0x7fc67c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:01:59.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.389+0000 7fc69b7fe640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 0x7fc67c06d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0070b0 tx=0x7fc68c007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:01:59.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.393+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc694057fc0 con 0x7fc69c15aab0 2023-10-21T16:01:59.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:01:59.585+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc69c074150 con 0x7fc69c15aab0 2023-10-21T16:02:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.148+0000 7fc693fff640 1 -- 172.21.15.53:0/3326189244 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc694057b50 con 0x7fc69c15aab0 2023-10-21T16:02:00.151 INFO:teuthology.orchestra.run.smithi053.stderr:set pool 4 allow_ec_overwrites to true 2023-10-21T16:02:00.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.153+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 msgr2=0x7fc67c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.153+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 0x7fc67c06d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0070b0 tx=0x7fc68c007040 comp rx=0 tx=0).stop 2023-10-21T16:02:00.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.153+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 msgr2=0x7fc69c0aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.153+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc694009fd0 tx=0x7fc69400b8e0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.154+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 shutdown_connections 2023-10-21T16:02:00.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.154+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc67c06ae70 0x7fc67c06d330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.154+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc69c15aab0 0x7fc69c0aca50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.154+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69c0caef0 0x7fc69c1f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.154+0000 7fc6a0947640 1 --2- 172.21.15.53:0/3326189244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69c0b04a0 0x7fc69c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.155+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 >> 172.21.15.53:0/3326189244 conn(0x7fc69c06c010 msgr2=0x7fc69c070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:00.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.158+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 shutdown_connections 2023-10-21T16:02:00.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.158+0000 7fc6a0947640 1 -- 172.21.15.53:0/3326189244 wait complete. 2023-10-21T16:02:00.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-21T16:02:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.386+0000 7f2c288bb640 1 Processor -- start 2023-10-21T16:02:00.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.386+0000 7f2c288bb640 1 -- start start 2023-10-21T16:02:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.387+0000 7f2c288bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.387+0000 7f2c288bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.387+0000 7f2c288bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2415b310 con 0x7f2c2406db60 2023-10-21T16:02:00.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.387+0000 7f2c288bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c240ca230 con 0x7f2c2406c3a0 2023-10-21T16:02:00.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.387+0000 7f2c288bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2406d290 con 0x7f2c2415aa00 2023-10-21T16:02:00.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.388+0000 7f2c2259c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.388+0000 7f2c22d9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.388+0000 7f2c22d9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:37576/0 (socket says 172.21.15.53:37576) 2023-10-21T16:02:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.388+0000 7f2c2259c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59916/0 (socket says 172.21.15.53:59916) 2023-10-21T16:02:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.388+0000 7f2c22d9d640 1 -- 172.21.15.53:0/1212763824 learned_addr learned my addr 172.21.15.53:0/1212763824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.389+0000 7f2c22d9d640 1 -- 172.21.15.53:0/1212763824 >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 msgr2=0x7f2c240c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.389+0000 7f2c22d9d640 1 --2- 172.21.15.53:0/1212763824 >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.389+0000 7f2c22d9d640 1 -- 172.21.15.53:0/1212763824 >> v1:172.21.15.124:6789/0 conn(0x7f2c2406db60 legacy=0x7f2c2406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.389+0000 7f2c22d9d640 1 -- 172.21.15.53:0/1212763824 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c240c9860 con 0x7f2c2415aa00 2023-10-21T16:02:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.389+0000 7f2c2259c640 1 --2- 172.21.15.53:0/1212763824 >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:00.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.390+0000 7f2c22d9d640 1 --2- 172.21.15.53:0/1212763824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00bb10 tx=0x7f2c1c02e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619f6a1dcb637168 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c2159a640 1 -- 172.21.15.53:0/1212763824 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c036440 con 0x7f2c2415aa00 2023-10-21T16:02:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c2159a640 1 -- 172.21.15.53:0/1212763824 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c0365a0 con 0x7f2c2415aa00 2023-10-21T16:02:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c2159a640 1 -- 172.21.15.53:0/1212763824 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c036930 con 0x7f2c2415aa00 2023-10-21T16:02:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 -- 172.21.15.53:0/1212763824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 msgr2=0x7f2c240b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 --2- 172.21.15.53:0/1212763824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00bb10 tx=0x7f2c1c02e150 comp rx=0 tx=0).stop 2023-10-21T16:02:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 -- 172.21.15.53:0/1212763824 shutdown_connections 2023-10-21T16:02:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 --2- 172.21.15.53:0/1212763824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c240b2a90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 --2- 172.21.15.53:0/1212763824 >> v2:172.21.15.124:3300/0 conn(0x7f2c2406c3a0 0x7f2c240c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.391+0000 7f2c288bb640 1 -- 172.21.15.53:0/1212763824 >> 172.21.15.53:0/1212763824 conn(0x7f2c2406a5d0 msgr2=0x7f2c240b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:00.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.392+0000 7f2c288bb640 1 -- 172.21.15.53:0/1212763824 shutdown_connections 2023-10-21T16:02:00.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.392+0000 7f2c288bb640 1 -- 172.21.15.53:0/1212763824 wait complete. 2023-10-21T16:02:00.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.393+0000 7f2c288bb640 1 Processor -- start 2023-10-21T16:02:00.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.394+0000 7f2c288bb640 1 -- start start 2023-10-21T16:02:00.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.394+0000 7f2c288bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2406c3a0 0x7f2c241f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c288bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c288bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c2415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c288bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2415d830 con 0x7f2c2406db60 2023-10-21T16:02:00.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c288bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240ad4d0 con 0x7f2c2406c3a0 2023-10-21T16:02:00.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c288bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2415cf90 con 0x7f2c2415aa00 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43884/0 (socket says 172.21.15.53:43884) 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 -- 172.21.15.53:0/910420008 learned_addr learned my addr 172.21.15.53:0/910420008 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c2259c640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2406c3a0 0x7f2c241f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c22d9d640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c2415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 -- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 msgr2=0x7f2c2415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c2415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 -- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2406c3a0 msgr2=0x7f2c241f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2406c3a0 0x7f2c241f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.395+0000 7f2c21d9b640 1 -- 172.21.15.53:0/910420008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009400 con 0x7f2c2406db60 2023-10-21T16:02:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c21d9b640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c790 tx=0x7f2c0c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c00cea0 con 0x7f2c2406db60 2023-10-21T16:02:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c006d50 con 0x7f2c2406db60 2023-10-21T16:02:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c003bb0 con 0x7f2c2406db60 2023-10-21T16:02:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c00b770 con 0x7f2c2406db60 2023-10-21T16:02:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.396+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2415d550 con 0x7f2c2406db60 2023-10-21T16:02:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.398+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2404e160 con 0x7f2c2406db60 2023-10-21T16:02:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.398+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c0c003d10 con 0x7f2c2406db60 2023-10-21T16:02:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.399+0000 7f2c1affd640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 0x7f2c0406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.399+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f2c0c089330 con 0x7f2c2406db60 2023-10-21T16:02:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.400+0000 7f2c2259c640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 0x7f2c0406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.401+0000 7f2c2259c640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 0x7f2c0406d360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c240cb010 tx=0x7f2c14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.404+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c0c057b90 con 0x7f2c2406db60 2023-10-21T16:02:00.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:00.630+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f2c2415b310 con 0x7f2c2406db60 2023-10-21T16:02:01.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.226+0000 7f2c1affd640 1 -- 172.21.15.53:0/910420008 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2c2415b310 con 0x7f2c2406db60 2023-10-21T16:02:01.228 INFO:teuthology.orchestra.run.smithi053.stderr:added data pool 4 to fsmap 2023-10-21T16:02:01.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 msgr2=0x7f2c0406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 0x7f2c0406d360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c240cb010 tx=0x7f2c14008040 comp rx=0 tx=0).stop 2023-10-21T16:02:01.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 msgr2=0x7f2c240ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c790 tx=0x7f2c0c00cc50 comp rx=0 tx=0).stop 2023-10-21T16:02:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 shutdown_connections 2023-10-21T16:02:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c2415aa00 0x7f2c2415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c0406aea0 0x7f2c0406d360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c2406db60 0x7f2c240ab940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 --2- 172.21.15.53:0/910420008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2406c3a0 0x7f2c241f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 >> 172.21.15.53:0/910420008 conn(0x7f2c2406a5d0 msgr2=0x7f2c240afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:01.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.230+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 shutdown_connections 2023-10-21T16:02:01.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.231+0000 7f2c288bb640 1 -- 172.21.15.53:0/910420008 wait complete. 2023-10-21T16:02:01.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:01.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.482+0000 7f01069a0640 1 Processor -- start 2023-10-21T16:02:01.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.482+0000 7f01069a0640 1 -- start start 2023-10-21T16:02:01.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f01069a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f0100165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f01069a0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015f7d0 con 0x7f01000adc70 2023-10-21T16:02:01.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f01069a0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015efb0 con 0x7f010015ab30 2023-10-21T16:02:01.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f01069a0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010015f210 con 0x7f01000b07d0 2023-10-21T16:02:01.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f010519d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f01000adc70 0x7f01000b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33772/0 (socket says 172.21.15.53:33772) 2023-10-21T16:02:01.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f010519d640 1 -- 172.21.15.53:0/508940500 learned_addr learned my addr 172.21.15.53:0/508940500 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:01.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938981173 0 0) 0x7f010015f7d0 con 0x7f01000adc70 2023-10-21T16:02:01.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f010619f640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f0100165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.484+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ec003520 con 0x7f01000adc70 2023-10-21T16:02:01.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1671053930 0 0) 0x7f00ec003520 con 0x7f01000adc70 2023-10-21T16:02:01.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 msgr2=0x7f0100165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:01.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f0100165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 >> v1:172.21.15.124:6789/0 conn(0x7f010015ab30 legacy=0x7f010015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010015f4f0 con 0x7f01000adc70 2023-10-21T16:02:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f00f0001670 con 0x7f01000adc70 2023-10-21T16:02:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 >> v1:172.21.15.53:6789/0 conn(0x7f01000adc70 legacy=0x7f01000b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f00ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f00ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00ec004d00 0x7f00ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010015f210 con 0x7f00ec0039a0 2023-10-21T16:02:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00bb10 con 0x7f00ec004d00 2023-10-21T16:02:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00bcd0 con 0x7f01000b07d0 2023-10-21T16:02:01.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.485+0000 7f010599e640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f00ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010519d640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f00ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010619f640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00ec004d00 0x7f00ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010599e640 1 -- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 msgr2=0x7f00ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:01.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010599e640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f00ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010599e640 1 -- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00ec004d00 msgr2=0x7f00ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:01.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010599e640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00ec004d00 0x7f00ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010599e640 1 -- 172.21.15.53:0/508940500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec003520 con 0x7f00ec0039a0 2023-10-21T16:02:01.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.486+0000 7f010519d640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01000b07d0 0x7f00ec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:01.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.487+0000 7f010599e640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f00ec003dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f00fc001720 tx=0x7f00ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a1344b285e36ca server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:01.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.487+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0052d0 con 0x7f00ec0039a0 2023-10-21T16:02:01.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.487+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc02e050 con 0x7f00ec0039a0 2023-10-21T16:02:01.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.487+0000 7f010499c640 1 -- 172.21.15.53:0/508940500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0035f0 con 0x7f00ec0039a0 2023-10-21T16:02:01.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 -- 172.21.15.53:0/508940500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 msgr2=0x7f00ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 --2- 172.21.15.53:0/508940500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f00ec003dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f00fc001720 tx=0x7f00ec0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f010619f640 1 -- 172.21.15.53:0/508940500 reap_dead start 2023-10-21T16:02:01.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 -- 172.21.15.53:0/508940500 shutdown_connections 2023-10-21T16:02:01.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 -- 172.21.15.53:0/508940500 >> 172.21.15.53:0/508940500 conn(0x7f01000cb0c0 msgr2=0x7f01000cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:01.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 -- 172.21.15.53:0/508940500 shutdown_connections 2023-10-21T16:02:01.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.488+0000 7f01069a0640 1 -- 172.21.15.53:0/508940500 wait complete. 2023-10-21T16:02:01.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.490+0000 7f01069a0640 1 Processor -- start 2023-10-21T16:02:01.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.490+0000 7f01069a0640 1 -- start start 2023-10-21T16:02:01.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 0x7f010014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01001550f0 0x7f0100155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0100155ac0 con 0x7f00ec0039a0 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b2940 con 0x7f01000ce030 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f01069a0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000ccf90 con 0x7f01001550f0 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43904/0 (socket says 172.21.15.53:43904) 2023-10-21T16:02:01.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 -- 172.21.15.53:0/1508767682 learned_addr learned my addr 172.21.15.53:0/1508767682 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:01.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010519d640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 0x7f010014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 -- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01001550f0 msgr2=0x7f0100155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010619f640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01001550f0 0x7f0100155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01001550f0 0x7f0100155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 -- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 msgr2=0x7f010014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 0x7f010014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010599e640 1 -- 172.21.15.53:0/1508767682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cd270 con 0x7f00ec0039a0 2023-10-21T16:02:01.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.491+0000 7f010519d640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 0x7f010014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:01.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f010599e640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00fc02bbe0 tx=0x7f00fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:01.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0076c0 con 0x7f00ec0039a0 2023-10-21T16:02:01.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc007a50 con 0x7f00ec0039a0 2023-10-21T16:02:01.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc006be0 con 0x7f00ec0039a0 2023-10-21T16:02:01.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000cd550 con 0x7f00ec0039a0 2023-10-21T16:02:01.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.492+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000cdb10 con 0x7f00ec0039a0 2023-10-21T16:02:01.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.494+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01000cdca0 con 0x7f00ec0039a0 2023-10-21T16:02:01.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.494+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f00fc006d40 con 0x7f00ec0039a0 2023-10-21T16:02:01.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.495+0000 7f00f67fc640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 0x7f00e406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.495+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00fc0ab580 con 0x7f00ec0039a0 2023-10-21T16:02:01.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.496+0000 7f010519d640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 0x7f00e406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.497+0000 7f010519d640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 0x7f00e406d380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00f0001eb0 tx=0x7f00f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:01.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.500+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00fc0764b0 con 0x7f00ec0039a0 2023-10-21T16:02:01.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.731+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f010004e160 con 0x7f00ec0039a0 2023-10-21T16:02:01.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.733+0000 7f00f67fc640 1 -- 172.21.15.53:0/1508767682 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f00fc079ad0 con 0x7f00ec0039a0 2023-10-21T16:02:01.735 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:01.735 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:00.632870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-21T16:02:01.735 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2023-10-21T16:02:01.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 msgr2=0x7f00e406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 0x7f00e406d380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00f0001eb0 tx=0x7f00f0009040 comp rx=0 tx=0).stop 2023-10-21T16:02:01.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 msgr2=0x7f01000b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00fc02bbe0 tx=0x7f00fc001120 comp rx=0 tx=0).stop 2023-10-21T16:02:01.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 shutdown_connections 2023-10-21T16:02:01.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01001550f0 0x7f0100155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01000ce030 0x7f010014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f00e406aec0 0x7f00e406d380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 --2- 172.21.15.53:0/1508767682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00ec0039a0 0x7f01000b2400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.735+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 >> 172.21.15.53:0/1508767682 conn(0x7f01000cb0c0 msgr2=0x7f01000af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:01.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.736+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 shutdown_connections 2023-10-21T16:02:01.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.736+0000 7f01069a0640 1 -- 172.21.15.53:0/1508767682 wait complete. 2023-10-21T16:02:01.757 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:01.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.979+0000 7f851735e640 1 Processor -- start 2023-10-21T16:02:01.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.979+0000 7f851735e640 1 -- start start 2023-10-21T16:02:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f851015ab40 0x7f85100ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f85100c90b0 0x7f85100adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100cae70 0x7f85100cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851015bf30 con 0x7f851015ab40 2023-10-21T16:02:01.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85100cb840 con 0x7f85100c90b0 2023-10-21T16:02:01.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f851735e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851015b2b0 con 0x7f85100cae70 2023-10-21T16:02:01.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.981+0000 7f85150fa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f851015ab40 0x7f85100ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f851015ab40 0x7f85100ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43908/0 (socket says 172.21.15.53:43908) 2023-10-21T16:02:01.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 -- 172.21.15.53:0/909530044 learned_addr learned my addr 172.21.15.53:0/909530044 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:01.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85148f9640 1 --2- 172.21.15.53:0/909530044 >> v2:172.21.15.124:3300/0 conn(0x7f85100c90b0 0x7f85100adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85158fb640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100cae70 0x7f85100cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 -- 172.21.15.53:0/909530044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100cae70 msgr2=0x7f85100cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:01.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100cae70 0x7f85100cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 -- 172.21.15.53:0/909530044 >> v2:172.21.15.124:3300/0 conn(0x7f85100c90b0 msgr2=0x7f85100adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:01.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 --2- 172.21.15.53:0/909530044 >> v2:172.21.15.124:3300/0 conn(0x7f85100c90b0 0x7f85100adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.982+0000 7f85150fa640 1 -- 172.21.15.53:0/909530044 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851015b590 con 0x7f851015ab40 2023-10-21T16:02:01.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.984+0000 7f85150fa640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f851015ab40 0x7f85100ad720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f851006e560 tx=0x7f850c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fccd21260d7e64 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:01.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.984+0000 7f8507fff640 1 -- 172.21.15.53:0/909530044 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c031870 con 0x7f851015ab40 2023-10-21T16:02:01.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.984+0000 7f8507fff640 1 -- 172.21.15.53:0/909530044 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c0319d0 con 0x7f851015ab40 2023-10-21T16:02:01.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.984+0000 7f8507fff640 1 -- 172.21.15.53:0/909530044 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c031d60 con 0x7f851015ab40 2023-10-21T16:02:01.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 -- 172.21.15.53:0/909530044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f851015ab40 msgr2=0x7f85100ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f851015ab40 0x7f85100ad720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f851006e560 tx=0x7f850c02b910 comp rx=0 tx=0).stop 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 -- 172.21.15.53:0/909530044 shutdown_connections 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100cae70 0x7f85100cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 --2- 172.21.15.53:0/909530044 >> v2:172.21.15.124:3300/0 conn(0x7f85100c90b0 0x7f85100adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 --2- 172.21.15.53:0/909530044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f851015ab40 0x7f85100ad720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 -- 172.21.15.53:0/909530044 >> 172.21.15.53:0/909530044 conn(0x7f851006a380 msgr2=0x7f85100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:01.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 -- 172.21.15.53:0/909530044 shutdown_connections 2023-10-21T16:02:01.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.985+0000 7f851735e640 1 -- 172.21.15.53:0/909530044 wait complete. 2023-10-21T16:02:01.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.987+0000 7f851735e640 1 Processor -- start 2023-10-21T16:02:01.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.987+0000 7f851735e640 1 -- start start 2023-10-21T16:02:01.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100c90b0 0x7f851014e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 0x7f8510158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851014e620 con 0x7f85100cae70 2023-10-21T16:02:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85100c7f10 con 0x7f851015ab40 2023-10-21T16:02:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f851735e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510156ab0 con 0x7f85100c90b0 2023-10-21T16:02:01.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43914/0 (socket says 172.21.15.53:43914) 2023-10-21T16:02:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 -- 172.21.15.53:0/2404781309 learned_addr learned my addr 172.21.15.53:0/2404781309 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85158fb640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 0x7f8510158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 -- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100c90b0 msgr2=0x7f851014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100c90b0 0x7f851014e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 -- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 msgr2=0x7f8510158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 0x7f8510158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85148f9640 1 -- 172.21.15.53:0/2404781309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0094d0 con 0x7f85100cae70 2023-10-21T16:02:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.988+0000 7f85158fb640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 0x7f8510158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:01.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f85148f9640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f850000d7d0 tx=0x7f850000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000096a0 con 0x7f85100cae70 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8500009800 con 0x7f85100cae70 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8500009bb0 con 0x7f85100cae70 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8510156d90 con 0x7f85100cae70 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.989+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510157350 con 0x7f85100cae70 2023-10-21T16:02:01.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.991+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85100ce340 con 0x7f85100cae70 2023-10-21T16:02:01.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.991+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8500009d10 con 0x7f85100cae70 2023-10-21T16:02:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.993+0000 7f8505ffb640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.993+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8500089e60 con 0x7f85100cae70 2023-10-21T16:02:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.993+0000 7f85150fa640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.994+0000 7f85150fa640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 0x7f84f406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f850c036000 tx=0x7f850c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:01.997+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8500054c40 con 0x7f85100cae70 2023-10-21T16:02:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.191+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8510067490 con 0x7f85100cae70 2023-10-21T16:02:02.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.194+0000 7f8505ffb640 1 -- 172.21.15.53:0/2404781309 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f8500058260 con 0x7f85100cae70 2023-10-21T16:02:02.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:02.197 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:02.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.196+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 msgr2=0x7f84f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 0x7f84f406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f850c036000 tx=0x7f850c030000 comp rx=0 tx=0).stop 2023-10-21T16:02:02.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 msgr2=0x7f85100c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f850000d7d0 tx=0x7f850000dc90 comp rx=0 tx=0).stop 2023-10-21T16:02:02.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 shutdown_connections 2023-10-21T16:02:02.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f851015ab40 0x7f8510158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85100cae70 0x7f85100c79d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 --2- 172.21.15.53:0/2404781309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85100c90b0 0x7f851014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 >> 172.21.15.53:0/2404781309 conn(0x7f851006a380 msgr2=0x7f85100ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:02.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 shutdown_connections 2023-10-21T16:02:02.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.197+0000 7f851735e640 1 -- 172.21.15.53:0/2404781309 wait complete. 2023-10-21T16:02:02.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.436+0000 7fa1d2dfb640 1 Processor -- start 2023-10-21T16:02:02.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.436+0000 7fa1d2dfb640 1 -- start start 2023-10-21T16:02:02.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d2dfb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1cc0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d2dfb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0cb580 con 0x7fa1cc15aad0 2023-10-21T16:02:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d2dfb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0cc0e0 con 0x7fa1cc0cc570 2023-10-21T16:02:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d2dfb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0b2390 con 0x7fa1cc15aeb0 2023-10-21T16:02:02.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d0b97640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa1cc15aad0 0x7fa1cc0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33774/0 (socket says 172.21.15.53:33774) 2023-10-21T16:02:02.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1d0b97640 1 -- 172.21.15.53:0/4153455309 learned_addr learned my addr 172.21.15.53:0/4153455309 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1cbfff640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1cc0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.438+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652495254 0 0) 0x7fa1cc0cb580 con 0x7fa1cc15aad0 2023-10-21T16:02:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b8003520 con 0x7fa1cc15aad0 2023-10-21T16:02:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158939483 0 0) 0x7fa1b8003520 con 0x7fa1cc15aad0 2023-10-21T16:02:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 msgr2=0x7fa1cc0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:02.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1cc0cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 >> v1:172.21.15.124:6789/0 conn(0x7fa1cc0cc570 legacy=0x7fa1cc0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0cb7b0 con 0x7fa1cc15aad0 2023-10-21T16:02:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa1bc001b70 con 0x7fa1cc15aad0 2023-10-21T16:02:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 >> v1:172.21.15.53:6789/0 conn(0x7fa1cc15aad0 legacy=0x7fa1cc0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8004d00 0x7fa1b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0b2390 con 0x7fa1b80039a0 2023-10-21T16:02:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b800baf0 con 0x7fa1b8004d00 2023-10-21T16:02:02.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b800bcb0 con 0x7fa1cc15aeb0 2023-10-21T16:02:02.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.439+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d1398640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8004d00 0x7fa1b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1cbfff640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d0b97640 1 -- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 msgr2=0x7fa1b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:02.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc15aeb0 0x7fa1b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d0b97640 1 -- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8004d00 msgr2=0x7fa1b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:02.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8004d00 0x7fa1b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.440+0000 7fa1d0b97640 1 -- 172.21.15.53:0/4153455309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8003520 con 0x7fa1b80039a0 2023-10-21T16:02:02.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.441+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1b8003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc002230 tx=0x7fa1b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7345a1eff435732 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.441+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc005d20 con 0x7fa1b80039a0 2023-10-21T16:02:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.441+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc02cb10 con 0x7fa1b80039a0 2023-10-21T16:02:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.441+0000 7fa1cb7fe640 1 -- 172.21.15.53:0/4153455309 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc005ea0 con 0x7fa1b80039a0 2023-10-21T16:02:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/4153455309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 msgr2=0x7fa1b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/4153455309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1b8003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc002230 tx=0x7fa1b80057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d1398640 1 -- 172.21.15.53:0/4153455309 reap_dead start 2023-10-21T16:02:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/4153455309 shutdown_connections 2023-10-21T16:02:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/4153455309 >> 172.21.15.53:0/4153455309 conn(0x7fa1cc0d09e0 msgr2=0x7fa1cc0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/4153455309 shutdown_connections 2023-10-21T16:02:02.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.442+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/4153455309 wait complete. 2023-10-21T16:02:02.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.444+0000 7fa1d2dfb640 1 Processor -- start 2023-10-21T16:02:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.444+0000 7fa1d2dfb640 1 -- start start 2023-10-21T16:02:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1cc1e7280 0x7fa1cc1e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc1e4b00 0x7fa1cc0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0cc9e0 con 0x7fa1b80039a0 2023-10-21T16:02:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0cedc0 con 0x7fa1cc1e7280 2023-10-21T16:02:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d2dfb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc1e5e20 con 0x7fa1cc1e4b00 2023-10-21T16:02:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d0b97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d0b97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43926/0 (socket says 172.21.15.53:43926) 2023-10-21T16:02:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.445+0000 7fa1d0b97640 1 -- 172.21.15.53:0/3678210702 learned_addr learned my addr 172.21.15.53:0/3678210702 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d1398640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc1e4b00 0x7fa1cc0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1cbfff640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1cc1e7280 0x7fa1cc1e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 -- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc1e4b00 msgr2=0x7fa1cc0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc1e4b00 0x7fa1cc0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 -- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1cc1e7280 msgr2=0x7fa1cc1e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1cc1e7280 0x7fa1cc1e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 -- 172.21.15.53:0/3678210702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0c5ed0 con 0x7fa1b80039a0 2023-10-21T16:02:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d0b97640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001270 tx=0x7fa1cc0cc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc039070 con 0x7fa1b80039a0 2023-10-21T16:02:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc007480 con 0x7fa1b80039a0 2023-10-21T16:02:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc03c040 con 0x7fa1b80039a0 2023-10-21T16:02:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.446+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc1e51a0 con 0x7fa1b80039a0 2023-10-21T16:02:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.447+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc1e5760 con 0x7fa1b80039a0 2023-10-21T16:02:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.448+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc1e4f10 con 0x7fa1b80039a0 2023-10-21T16:02:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.449+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa1bc032070 con 0x7fa1b80039a0 2023-10-21T16:02:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.449+0000 7fa1c97fa640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 0x7fa1ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.450+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1bc0aaca0 con 0x7fa1b80039a0 2023-10-21T16:02:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.450+0000 7fa1cbfff640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 0x7fa1ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.451+0000 7fa1cbfff640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 0x7fa1ac06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc1e7660 tx=0x7fa1b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.454+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1bc075b00 con 0x7fa1b80039a0 2023-10-21T16:02:02.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.647+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1cc067490 con 0x7fa1b80039a0 2023-10-21T16:02:02.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.649+0000 7fa1c97fa640 1 -- 172.21.15.53:0/3678210702 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fa1bc046090 con 0x7fa1b80039a0 2023-10-21T16:02:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:02.652 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 msgr2=0x7fa1ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 0x7fa1ac06d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc1e7660 tx=0x7fa1b4008040 comp rx=0 tx=0).stop 2023-10-21T16:02:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 msgr2=0x7fa1cc1e6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001270 tx=0x7fa1cc0cc050 comp rx=0 tx=0).stop 2023-10-21T16:02:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 shutdown_connections 2023-10-21T16:02:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1cc1e4b00 0x7fa1cc0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1cc1e7280 0x7fa1cc1e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1ac06ae40 0x7fa1ac06d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 --2- 172.21.15.53:0/3678210702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80039a0 0x7fa1cc1e6d40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 >> 172.21.15.53:0/3678210702 conn(0x7fa1cc0d09e0 msgr2=0x7fa1cc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:02.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 shutdown_connections 2023-10-21T16:02:02.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.652+0000 7fa1d2dfb640 1 -- 172.21.15.53:0/3678210702 wait complete. 2023-10-21T16:02:02.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:02.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.922+0000 7fcfb7a9b640 1 Processor -- start 2023-10-21T16:02:02.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.922+0000 7fcfb7a9b640 1 -- start start 2023-10-21T16:02:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.923+0000 7fcfb7a9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfb015ad70 0x7fcfb0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb7a9b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 0x7fcfb00b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb7a9b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00724a0 con 0x7fcfb015a990 2023-10-21T16:02:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb7a9b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb0071ad0 con 0x7fcfb015ad70 2023-10-21T16:02:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb7a9b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb00b5500 con 0x7fcfb006a460 2023-10-21T16:02:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfb015ad70 0x7fcfb0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5837640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcfb015a990 0x7fcfb0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:33790/0 (socket says 172.21.15.53:33790) 2023-10-21T16:02:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfb015ad70 0x7fcfb0071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43930/0 (socket says 172.21.15.53:43930) 2023-10-21T16:02:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5837640 1 -- 172.21.15.53:0/4274907421 learned_addr learned my addr 172.21.15.53:0/4274907421 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb6038640 1 --2- 172.21.15.53:0/4274907421 >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 0x7fcfb00b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 -- 172.21.15.53:0/4274907421 >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 msgr2=0x7fcfb00b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 --2- 172.21.15.53:0/4274907421 >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 0x7fcfb00b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 -- 172.21.15.53:0/4274907421 >> v1:172.21.15.53:6789/0 conn(0x7fcfb015a990 legacy=0x7fcfb0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb5036640 1 -- 172.21.15.53:0/4274907421 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0071480 con 0x7fcfb015ad70 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb6038640 1 --2- 172.21.15.53:0/4274907421 >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 0x7fcfb00b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.924+0000 7fcfb4835640 1 -- 172.21.15.53:0/4274907421 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873030396 0 0) 0x7fcfa4001490 con 0x7fcfb015a990 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb5036640 1 --2- 172.21.15.53:0/4274907421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb0071f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00c3390 tx=0x7fcf9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0178f11d509078e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb4835640 1 -- 172.21.15.53:0/4274907421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c031830 con 0x7fcfb015ad70 2023-10-21T16:02:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb4835640 1 -- 172.21.15.53:0/4274907421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c031990 con 0x7fcfb015ad70 2023-10-21T16:02:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb4835640 1 -- 172.21.15.53:0/4274907421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c031d20 con 0x7fcfb015ad70 2023-10-21T16:02:02.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4274907421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 msgr2=0x7fcfb0071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.926+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4274907421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb0071f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00c3390 tx=0x7fcf9c02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4274907421 shutdown_connections 2023-10-21T16:02:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4274907421 >> v2:172.21.15.124:3300/0 conn(0x7fcfb006a460 0x7fcfb00b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4274907421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb0071f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4274907421 >> 172.21.15.53:0/4274907421 conn(0x7fcfb006d450 msgr2=0x7fcfb006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4274907421 shutdown_connections 2023-10-21T16:02:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.927+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4274907421 wait complete. 2023-10-21T16:02:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.928+0000 7fcfb7a9b640 1 Processor -- start 2023-10-21T16:02:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.929+0000 7fcfb7a9b640 1 -- start start 2023-10-21T16:02:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.929+0000 7fcfb7a9b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfb006a460 0x7fcfb0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb7a9b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfb015a990 0x7fcfb00cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb7a9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb7a9b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb0154540 con 0x7fcfb015ad70 2023-10-21T16:02:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb7a9b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00ce460 con 0x7fcfb006a460 2023-10-21T16:02:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb7a9b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00cd460 con 0x7fcfb015a990 2023-10-21T16:02:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43932/0 (socket says 172.21.15.53:43932) 2023-10-21T16:02:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 -- 172.21.15.53:0/4052269450 learned_addr learned my addr 172.21.15.53:0/4052269450 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb5837640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfb006a460 0x7fcfb0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb5036640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfb015a990 0x7fcfb00cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 -- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfb015a990 msgr2=0x7fcfb00cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfb015a990 0x7fcfb00cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 -- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfb006a460 msgr2=0x7fcfb0154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfb006a460 0x7fcfb0154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.930+0000 7fcfb6038640 1 -- 172.21.15.53:0/4052269450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009400 con 0x7fcfb015ad70 2023-10-21T16:02:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfb6038640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c5f0 tx=0x7fcfa800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8011800 con 0x7fcfb015ad70 2023-10-21T16:02:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8011980 con 0x7fcfb015ad70 2023-10-21T16:02:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8011d30 con 0x7fcfb015ad70 2023-10-21T16:02:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb00addb0 con 0x7fcfb015ad70 2023-10-21T16:02:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.931+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb00cd720 con 0x7fcfb015ad70 2023-10-21T16:02:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.932+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb0067490 con 0x7fcfb015ad70 2023-10-21T16:02:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.933+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfa8022350 con 0x7fcfb015ad70 2023-10-21T16:02:02.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.934+0000 7fcfae7fc640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 0x7fcf9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:02.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.934+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcfa8089990 con 0x7fcfb015ad70 2023-10-21T16:02:02.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.935+0000 7fcfb5837640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 0x7fcf9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:02.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.936+0000 7fcfb5837640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 0x7fcf9406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40012e0 tx=0x7fcfa4007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:02.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:02.938+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfa80546c0 con 0x7fcfb015ad70 2023-10-21T16:02:03.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.173+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb00cbe60 con 0x7fcfb015ad70 2023-10-21T16:02:03.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.175+0000 7fcfae7fc640 1 -- 172.21.15.53:0/4052269450 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4919 (secure 0 0 0) 0x7fcfa8057ce0 con 0x7fcfb015ad70 2023-10-21T16:02:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:03.177 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:00.632870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-21T16:02:03.177 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 7 2023-10-21T16:02:03.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 msgr2=0x7fcf9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 0x7fcf9406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40012e0 tx=0x7fcfa4007d30 comp rx=0 tx=0).stop 2023-10-21T16:02:03.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 msgr2=0x7fcfb00ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c5f0 tx=0x7fcfa800cab0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 shutdown_connections 2023-10-21T16:02:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcf9406ae70 0x7fcf9406d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb015ad70 0x7fcfb00ad240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfb015a990 0x7fcfb00cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 --2- 172.21.15.53:0/4052269450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfb006a460 0x7fcfb0154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 >> 172.21.15.53:0/4052269450 conn(0x7fcfb006d450 msgr2=0x7fcfb00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.177+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 shutdown_connections 2023-10-21T16:02:03.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.178+0000 7fcfb7a9b640 1 -- 172.21.15.53:0/4052269450 wait complete. 2023-10-21T16:02:03.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:03.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.427+0000 7fd94756a640 1 Processor -- start 2023-10-21T16:02:03.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.427+0000 7fd94756a640 1 -- start start 2023-10-21T16:02:03.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.428+0000 7fd94756a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.428+0000 7fd94756a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 0x7fd94006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.428+0000 7fd94756a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94006e000 con 0x7fd94006ed20 2023-10-21T16:02:03.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.428+0000 7fd94756a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94015d150 con 0x7fd9401607c0 2023-10-21T16:02:03.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.428+0000 7fd94756a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940161010 con 0x7fd94015cd70 2023-10-21T16:02:03.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.429+0000 7fd945d67640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 0x7fd94006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.429+0000 7fd946568640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.429+0000 7fd946568640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:37660/0 (socket says 172.21.15.53:37660) 2023-10-21T16:02:03.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.429+0000 7fd946568640 1 -- 172.21.15.53:0/4154668214 learned_addr learned my addr 172.21.15.53:0/4154668214 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:03.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.430+0000 7fd946568640 1 -- 172.21.15.53:0/4154668214 >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 msgr2=0x7fd94006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:03.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.430+0000 7fd946568640 1 --2- 172.21.15.53:0/4154668214 >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 0x7fd94006e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.430+0000 7fd946568640 1 -- 172.21.15.53:0/4154668214 >> v1:172.21.15.124:6789/0 conn(0x7fd94006ed20 legacy=0x7fd9400c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.430+0000 7fd946568640 1 -- 172.21.15.53:0/4154668214 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9401612f0 con 0x7fd94015cd70 2023-10-21T16:02:03.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.430+0000 7fd945d67640 1 --2- 172.21.15.53:0/4154668214 >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 0x7fd94006e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:03.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.431+0000 7fd946568640 1 --2- 172.21.15.53:0/4154668214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009e50 tx=0x7fd93c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d3b50010d6041 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.432+0000 7fd945566640 1 -- 172.21.15.53:0/4154668214 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c02cb00 con 0x7fd94015cd70 2023-10-21T16:02:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.432+0000 7fd945566640 1 -- 172.21.15.53:0/4154668214 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c02cc60 con 0x7fd94015cd70 2023-10-21T16:02:03.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.432+0000 7fd945566640 1 -- 172.21.15.53:0/4154668214 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0343f0 con 0x7fd94015cd70 2023-10-21T16:02:03.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.432+0000 7fd94756a640 1 -- 172.21.15.53:0/4154668214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 msgr2=0x7fd940071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.432+0000 7fd94756a640 1 --2- 172.21.15.53:0/4154668214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009e50 tx=0x7fd93c02bf60 comp rx=0 tx=0).stop 2023-10-21T16:02:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 -- 172.21.15.53:0/4154668214 shutdown_connections 2023-10-21T16:02:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 --2- 172.21.15.53:0/4154668214 >> v2:172.21.15.124:3300/0 conn(0x7fd9401607c0 0x7fd94006e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 --2- 172.21.15.53:0/4154668214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd940071450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 -- 172.21.15.53:0/4154668214 >> 172.21.15.53:0/4154668214 conn(0x7fd9400cb8f0 msgr2=0x7fd9400cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:03.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 -- 172.21.15.53:0/4154668214 shutdown_connections 2023-10-21T16:02:03.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.433+0000 7fd94756a640 1 -- 172.21.15.53:0/4154668214 wait complete. 2023-10-21T16:02:03.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.434+0000 7fd94756a640 1 Processor -- start 2023-10-21T16:02:03.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.435+0000 7fd94756a640 1 -- start start 2023-10-21T16:02:03.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.435+0000 7fd94756a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd94006ed20 0x7fd94015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd94756a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd9400c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd94756a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd94756a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940143f60 con 0x7fd9401607c0 2023-10-21T16:02:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd94756a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940144800 con 0x7fd94006ed20 2023-10-21T16:02:03.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd94756a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400ad0a0 con 0x7fd94015cd70 2023-10-21T16:02:03.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43940/0 (socket says 172.21.15.53:43940) 2023-10-21T16:02:03.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 -- 172.21.15.53:0/399683845 learned_addr learned my addr 172.21.15.53:0/399683845 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:03.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946568640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd94006ed20 0x7fd94015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd945d67640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd9400c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 -- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 msgr2=0x7fd9400c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd9400c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 -- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd94006ed20 msgr2=0x7fd94015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd94006ed20 0x7fd94015fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.436+0000 7fd946d69640 1 -- 172.21.15.53:0/399683845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930009400 con 0x7fd9401607c0 2023-10-21T16:02:03.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd946d69640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c3f0 tx=0x7fd93800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:03.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380052d0 con 0x7fd9401607c0 2023-10-21T16:02:03.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd938005450 con 0x7fd9401607c0 2023-10-21T16:02:03.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93c009b30 con 0x7fd9401607c0 2023-10-21T16:02:03.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938007260 con 0x7fd9401607c0 2023-10-21T16:02:03.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.437+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940144490 con 0x7fd9401607c0 2023-10-21T16:02:03.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.439+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd940144990 con 0x7fd9401607c0 2023-10-21T16:02:03.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.440+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd9380073c0 con 0x7fd9401607c0 2023-10-21T16:02:03.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.441+0000 7fd936ffd640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 0x7fd92406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:03.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.441+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd938088c30 con 0x7fd9401607c0 2023-10-21T16:02:03.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.442+0000 7fd946568640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 0x7fd92406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:03.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.443+0000 7fd946568640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 0x7fd92406d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd93c035000 tx=0x7fd93c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:03.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.446+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9380539e0 con 0x7fd9401607c0 2023-10-21T16:02:03.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.640+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd94004e160 con 0x7fd9401607c0 2023-10-21T16:02:03.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.643+0000 7fd936ffd640 1 -- 172.21.15.53:0/399683845 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fd938057000 con 0x7fd9401607c0 2023-10-21T16:02:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:03.646 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:03.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 msgr2=0x7fd92406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 0x7fd92406d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd93c035000 tx=0x7fd93c030000 comp rx=0 tx=0).stop 2023-10-21T16:02:03.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 msgr2=0x7fd9400c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:03.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd93800c3f0 tx=0x7fd93800c8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 shutdown_connections 2023-10-21T16:02:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd92406aea0 0x7fd92406d360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9401607c0 0x7fd9400c6480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd94015cd70 0x7fd9400c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 --2- 172.21.15.53:0/399683845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd94006ed20 0x7fd94015fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:03.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 >> 172.21.15.53:0/399683845 conn(0x7fd9400cb8f0 msgr2=0x7fd9400ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:03.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.646+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 shutdown_connections 2023-10-21T16:02:03.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:03.647+0000 7fd94756a640 1 -- 172.21.15.53:0/399683845 wait complete. 2023-10-21T16:02:04.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:01:55.604470+0000', 'modified': '2023-10-21T16:02:00.632870+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34163': {'gid': 34163, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/2621260973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 2621260973}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 2621260973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-21T16:02:04.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:02:04.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-21T16:02:04.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.310+0000 7fc2cb3a7640 1 Processor -- start 2023-10-21T16:02:04.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.310+0000 7fc2cb3a7640 1 -- start start 2023-10-21T16:02:04.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2cb3a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2c415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2cb3a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40cab60 con 0x7fc2c40ca680 2023-10-21T16:02:04.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2cb3a7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40cb960 con 0x7fc2c415a820 2023-10-21T16:02:04.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2cb3a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40cbdf0 con 0x7fc2c40b0680 2023-10-21T16:02:04.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2caba6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2c40ca680 0x7fc2c415ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48628/0 (socket says 172.21.15.53:48628) 2023-10-21T16:02:04.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.312+0000 7fc2caba6640 1 -- 172.21.15.53:0/2073626860 learned_addr learned my addr 172.21.15.53:0/2073626860 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:04.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2c415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289092974 0 0) 0x7fc2c40cab60 con 0x7fc2c40ca680 2023-10-21T16:02:04.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b0003520 con 0x7fc2c40ca680 2023-10-21T16:02:04.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401614658 0 0) 0x7fc2b0003520 con 0x7fc2c40ca680 2023-10-21T16:02:04.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 msgr2=0x7fc2c415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2c415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 >> v1:172.21.15.124:6789/0 conn(0x7fc2c415a820 legacy=0x7fc2c40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40cb030 con 0x7fc2c40ca680 2023-10-21T16:02:04.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc2bc0019f0 con 0x7fc2c40ca680 2023-10-21T16:02:04.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 >> v1:172.21.15.53:6789/0 conn(0x7fc2c40ca680 legacy=0x7fc2c415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2b00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2b0003f20 0x7fc2b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40cbdf0 con 0x7fc2b0004d00 2023-10-21T16:02:04.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000bb10 con 0x7fc2b0003f20 2023-10-21T16:02:04.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.313+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000bcd0 con 0x7fc2c40b0680 2023-10-21T16:02:04.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2b00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2b0003f20 0x7fc2b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 -- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 msgr2=0x7fc2b00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c40b0680 0x7fc2b00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 -- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2b0003f20 msgr2=0x7fc2b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2b0003f20 0x7fc2b00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.314+0000 7fc2caba6640 1 -- 172.21.15.53:0/2073626860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0003520 con 0x7fc2b0004d00 2023-10-21T16:02:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.315+0000 7fc2caba6640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2b0005180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40caa90 tx=0x7fc2b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a61aa163a6b927 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.315+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0072d0 con 0x7fc2b0004d00 2023-10-21T16:02:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.315+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc02f070 con 0x7fc2b0004d00 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.315+0000 7fc2c93a3640 1 -- 172.21.15.53:0/2073626860 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0076b0 con 0x7fc2b0004d00 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2073626860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 msgr2=0x7fc2b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2073626860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2b0005180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40caa90 tx=0x7fc2b00057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2caba6640 1 -- 172.21.15.53:0/2073626860 reap_dead start 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2073626860 shutdown_connections 2023-10-21T16:02:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2073626860 >> 172.21.15.53:0/2073626860 conn(0x7fc2c406c050 msgr2=0x7fc2c406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2073626860 shutdown_connections 2023-10-21T16:02:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.316+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2073626860 wait complete. 2023-10-21T16:02:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.318+0000 7fc2cb3a7640 1 Processor -- start 2023-10-21T16:02:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.318+0000 7fc2cb3a7640 1 -- start start 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c415ac00 0x7fc2c41f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c41f5980 0x7fc2c41f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c415bdb0 con 0x7fc2b0004d00 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c415b1a0 con 0x7fc2c415ac00 2023-10-21T16:02:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2cb3a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c415b480 con 0x7fc2c41f5980 2023-10-21T16:02:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2ca3a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2ca3a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54458/0 (socket says 172.21.15.53:54458) 2023-10-21T16:02:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.319+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/2206688219 learned_addr learned my addr 172.21.15.53:0/2206688219 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c41f5980 msgr2=0x7fc2c41f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c41f5980 0x7fc2c41f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c415ac00 0x7fc2c41f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c415ac00 msgr2=0x7fc2c41f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c415ac00 0x7fc2c41f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/2206688219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c415b760 con 0x7fc2b0004d00 2023-10-21T16:02:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e410 tx=0x7fc2c000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2baffd640 1 -- 172.21.15.53:0/2206688219 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00052e0 con 0x7fc2b0004d00 2023-10-21T16:02:04.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2baffd640 1 -- 172.21.15.53:0/2206688219 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0005440 con 0x7fc2b0004d00 2023-10-21T16:02:04.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2baffd640 1 -- 172.21.15.53:0/2206688219 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003810 con 0x7fc2b0004d00 2023-10-21T16:02:04.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.320+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c415ba40 con 0x7fc2b0004d00 2023-10-21T16:02:04.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.321+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c41f16a0 con 0x7fc2b0004d00 2023-10-21T16:02:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.323+0000 7fc2baffd640 1 -- 172.21.15.53:0/2206688219 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2c0003970 con 0x7fc2b0004d00 2023-10-21T16:02:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.323+0000 7fc2baffd640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.324+0000 7fc2baffd640 1 -- 172.21.15.53:0/2206688219 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2c0089090 con 0x7fc2b0004d00 2023-10-21T16:02:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.324+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.324+0000 7fc2cf1a0740 1 Processor -- start 2023-10-21T16:02:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.324+0000 7fc2cf1a0740 1 -- start start 2023-10-21T16:02:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009910 tx=0x7fc2b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571170 0x55c204544070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c2045445b0 0x55c204574440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2044647a0 con 0x55c204571550 2023-10-21T16:02:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2044b6740 con 0x55c2045445b0 2023-10-21T16:02:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.325+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2044b3e30 con 0x55c204571170 2023-10-21T16:02:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54468/0 (socket says 172.21.15.53:54468) 2023-10-21T16:02:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3266466079 learned_addr learned my addr 172.21.15.53:0/3266466079 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571170 0x55c204544070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2caba6640 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c2045445b0 0x55c204574440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571170 msgr2=0x55c204544070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571170 0x55c204544070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c2045445b0 msgr2=0x55c204574440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c2045445b0 0x55c204574440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3266466079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c204473910 con 0x55c204571550 2023-10-21T16:02:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.326+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4011c60 tx=0x7fc2b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6e78c3ddbbf905 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2a7fff640 1 -- 172.21.15.53:0/3266466079 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40095c0 con 0x55c204571550 2023-10-21T16:02:04.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2a7fff640 1 -- 172.21.15.53:0/3266466079 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4011c90 con 0x55c204571550 2023-10-21T16:02:04.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2a7fff640 1 -- 172.21.15.53:0/3266466079 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4021440 con 0x55c204571550 2023-10-21T16:02:04.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3266466079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 msgr2=0x55c204573f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4011c60 tx=0x7fc2b4014000 comp rx=0 tx=0).stop 2023-10-21T16:02:04.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3266466079 shutdown_connections 2023-10-21T16:02:04.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c2045445b0 0x55c204574440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204573f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3266466079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571170 0x55c204544070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.327+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3266466079 >> 172.21.15.53:0/3266466079 conn(0x55c204546380 msgr2=0x55c20458ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.328+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3266466079 shutdown_connections 2023-10-21T16:02:04.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.328+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3266466079 wait complete. 2023-10-21T16:02:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.328+0000 7fc2cf1a0740 1 -- start start 2023-10-21T16:02:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.329+0000 7fc2cf1a0740 1 Processor -- start 2023-10-21T16:02:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c204560fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204577570 0x55c20457a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2044af720 con 0x55c204571170 2023-10-21T16:02:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20444d970 con 0x55c204577570 2023-10-21T16:02:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c20444d610 con 0x55c204571550 2023-10-21T16:02:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2cf1a0740 10 client.0 _ll_register_callbacks cb 0x55c20446f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:02:04.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54470/0 (socket says 172.21.15.53:54470) 2023-10-21T16:02:04.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/4283538377 learned_addr learned my addr 172.21.15.53:0/4283538377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2caba6640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204577570 0x55c20457a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c204560fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 msgr2=0x55c204560fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c204560fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204577570 msgr2=0x55c20457a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204577570 0x55c20457a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.330+0000 7fc2ca3a5640 1 -- 172.21.15.53:0/4283538377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b40118e0 con 0x55c204571170 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0054170 tx=0x7fc2c005bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75a77deab803aab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:02:04.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00560d0 con 0x55c204571170 2023-10-21T16:02:04.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2c0057160 con 0x55c204571170 2023-10-21T16:02:04.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2044d6160 con 0x55c204571170 2023-10-21T16:02:04.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c0057320 con 0x55c204571170 2023-10-21T16:02:04.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2cf1a0740 10 client.4398 fetch_fsmap learned FSMap version 7 2023-10-21T16:02:04.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c2044adda0 con 0x55c204571170 2023-10-21T16:02:04.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.331+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2044d6160 con 0x55c204571170 2023-10-21T16:02:04.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fc2c00762f0 con 0x55c204571170 2023-10-21T16:02:04.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2023-10-21T16:02:04.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 10 client.4398 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 20 client.4398 populate_metadata read hostname 'smithi053' 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20457a270 0x55c204574ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 4 client.4398 mds_command: new command op to 34163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 --> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c204481cc0 con 0x55c20457a270 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.332+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20457a270 0x55c204574ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.333+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20457a270 0x55c204574ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22899b28e62d39ee server_cookie=db5bbe4e5aea9fb0 in_seq=0 out_seq=0 2023-10-21T16:02:04.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.333+0000 7fc2a57fa640 10 client.4398 ms_handle_connect on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:04.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2a57fa640 1 -- 172.21.15.53:0/4283538377 <== mds.0 v2:172.21.15.124:6836/2621260973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c204481cc0 con 0x55c20457a270 2023-10-21T16:02:04.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2a57fa640 10 client.4398 handle_command_reply: tid=0 2023-10-21T16:02:04.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 client.4398 shutdown 2023-10-21T16:02:04.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 msgr2=0x55c20458fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0054170 tx=0x7fc2c005bdd0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 shutdown_connections 2023-10-21T16:02:04.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20457a270 0x55c204574ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204577570 0x55c20457a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c204560fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2a57fa640 0 client.4398 ms_handle_reset on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:04.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/4283538377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571170 0x55c20458fa90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.335+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 >> 172.21.15.53:0/4283538377 conn(0x55c2045445b0 msgr2=0x55c204547f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.336+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 shutdown_connections 2023-10-21T16:02:04.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.336+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/4283538377 wait complete. 2023-10-21T16:02:04.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.336+0000 7fc2cf1a0740 20 client.4398 trim_cache size 0 max 16384 2023-10-21T16:02:04.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.353+0000 7fc2cf1a0740 1 Processor -- start 2023-10-21T16:02:04.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.353+0000 7fc2cf1a0740 1 -- start start 2023-10-21T16:02:04.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c20457b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c2045f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c204213f20 con 0x55c204577570 2023-10-21T16:02:04.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2044555c0 con 0x55c204571170 2023-10-21T16:02:04.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c20458f460 con 0x55c204571550 2023-10-21T16:02:04.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2caba6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2caba6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54486/0 (socket says 172.21.15.53:54486) 2023-10-21T16:02:04.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2caba6640 1 -- 172.21.15.53:0/1576094861 learned_addr learned my addr 172.21.15.53:0/1576094861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c20457b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.354+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c2045f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 -- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 msgr2=0x55c2045f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c2045f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 -- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 msgr2=0x55c20457b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c20457b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 -- 172.21.15.53:0/1576094861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20444cb80 con 0x55c204577570 2023-10-21T16:02:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c20457b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2caba6640 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00a8f0 tx=0x7fc2bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06f8326d05c4491 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2b8ff9640 1 -- 172.21.15.53:0/1576094861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc044070 con 0x55c204577570 2023-10-21T16:02:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2b8ff9640 1 -- 172.21.15.53:0/1576094861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc001250 con 0x55c204577570 2023-10-21T16:02:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.355+0000 7fc2b8ff9640 1 -- 172.21.15.53:0/1576094861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0017f0 con 0x55c204577570 2023-10-21T16:02:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/1576094861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 msgr2=0x55c20453f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00a8f0 tx=0x7fc2bc02e000 comp rx=0 tx=0).stop 2023-10-21T16:02:04.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/1576094861 shutdown_connections 2023-10-21T16:02:04.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204577570 0x55c20453f9c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204571550 0x55c2045f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/1576094861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c20457b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/1576094861 >> 172.21.15.53:0/1576094861 conn(0x55c2045445b0 msgr2=0x55c20454b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/1576094861 shutdown_connections 2023-10-21T16:02:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.356+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/1576094861 wait complete. 2023-10-21T16:02:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.357+0000 7fc2cf1a0740 1 -- start start 2023-10-21T16:02:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.357+0000 7fc2cf1a0740 1 Processor -- start 2023-10-21T16:02:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c2045eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204577570 0x55c2045481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2045f79f0 con 0x55c204571550 2023-10-21T16:02:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20458ec80 con 0x55c204571170 2023-10-21T16:02:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2045604a0 con 0x55c204577570 2023-10-21T16:02:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2cf1a0740 10 client.0 _ll_register_callbacks cb 0x7fc2c0053fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:02:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54492/0 (socket says 172.21.15.53:54492) 2023-10-21T16:02:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3746565397 learned_addr learned my addr 172.21.15.53:0/3746565397 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c2045eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204577570 msgr2=0x55c2045481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204577570 0x55c2045481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 msgr2=0x55c2045eda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c2045eda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2c9ba4640 1 -- 172.21.15.53:0/3746565397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc038040 con 0x55c204571550 2023-10-21T16:02:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.358+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c2045eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2c9ba4640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4008010 tx=0x7fc2b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2b82d1fc88bb15 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:02:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4031040 con 0x55c204571550 2023-10-21T16:02:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2b40332c0 con 0x55c204571550 2023-10-21T16:02:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2b4011c90 con 0x55c204571550 2023-10-21T16:02:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b40095c0 con 0x55c204571550 2023-10-21T16:02:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2cf1a0740 10 client.4404 fetch_fsmap learned FSMap version 7 2023-10-21T16:02:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.359+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55c204472170 con 0x55c204571550 2023-10-21T16:02:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4011c90 con 0x55c204571550 2023-10-21T16:02:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fc2b4033810 con 0x55c204571550 2023-10-21T16:02:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 10 client.4404 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 20 client.4404 populate_metadata read hostname 'smithi053' 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20456e670 0x55c20456ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 4 client.4404 mds_command: new command op to 34163 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 --> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c2044a8700 con 0x55c20456e670 2023-10-21T16:02:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.360+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20456e670 0x55c20456ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.361+0000 7fc2ca3a5640 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20456e670 0x55c20456ea50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5994c3d9053c9ac server_cookie=ca7dc79865821bc7 in_seq=0 out_seq=0 2023-10-21T16:02:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.361+0000 7fc2a57fa640 10 client.4404 ms_handle_connect on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2a57fa640 1 -- 172.21.15.53:0/3746565397 <== mds.0 v2:172.21.15.124:6836/2621260973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55c2044a8700 con 0x55c20456e670 2023-10-21T16:02:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2a57fa640 10 client.4404 handle_command_reply: tid=0 2023-10-21T16:02:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 client.4404 shutdown 2023-10-21T16:02:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 msgr2=0x55c204547c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4008010 tx=0x7fc2b40046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 shutdown_connections 2023-10-21T16:02:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x55c20456e670 0x55c20456ea50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c204577570 0x55c2045481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c204571550 0x55c204547c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 --2- 172.21.15.53:0/3746565397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c204571170 0x55c2045eda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2a57fa640 0 client.4404 ms_handle_reset on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 >> 172.21.15.53:0/3746565397 conn(0x55c2045445b0 msgr2=0x55c204592410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 shutdown_connections 2023-10-21T16:02:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 1 -- 172.21.15.53:0/3746565397 wait complete. 2023-10-21T16:02:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.362+0000 7fc2cf1a0740 20 client.4404 trim_cache size 0 max 16384 2023-10-21T16:02:04.377 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:02:04.377 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:02:04.378 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:02:04.378 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 34163, 2023-10-21T16:02:04.378 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:02:04.378 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:02:04.378 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:02:04.379 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 8.234863893, 2023-10-21T16:02:04.379 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 7, 2023-10-21T16:02:04.379 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 26, 2023-10-21T16:02:04.379 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 25, 2023-10-21T16:02:04.379 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 16.021881977 2023-10-21T16:02:04.380 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:02:04.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 msgr2=0x7fc2a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009910 tx=0x7fc2b4008040 comp rx=0 tx=0).stop 2023-10-21T16:02:04.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 msgr2=0x7fc2c415d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000e410 tx=0x7fc2c000e8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 shutdown_connections 2023-10-21T16:02:04.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2c41f5980 0x7fc2c41f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c415ac00 0x7fc2c41f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 --2- 172.21.15.53:0/2206688219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2b0004d00 0x7fc2c415d290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 >> 172.21.15.53:0/2206688219 conn(0x7fc2c406c050 msgr2=0x7fc2c40ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 shutdown_connections 2023-10-21T16:02:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.363+0000 7fc2cb3a7640 1 -- 172.21.15.53:0/2206688219 wait complete. 2023-10-21T16:02:04.384 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-21T16:02:04.385 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:04.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:02:04.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.604+0000 7f9675380640 1 Processor -- start 2023-10-21T16:02:04.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.604+0000 7f9675380640 1 -- start start 2023-10-21T16:02:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.605+0000 7f9675380640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.605+0000 7f9675380640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 0x7f96700ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.605+0000 7f9675380640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f967006cfd0 con 0x7f967006dd00 2023-10-21T16:02:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.605+0000 7f9675380640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700cbe20 con 0x7f967015a970 2023-10-21T16:02:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.605+0000 7f9675380640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967006d870 con 0x7f96700cb990 2023-10-21T16:02:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.606+0000 7f966ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.606+0000 7f9674b7f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 0x7f96700ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.606+0000 7f966ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51064/0 (socket says 172.21.15.53:51064) 2023-10-21T16:02:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.606+0000 7f966ffff640 1 -- 172.21.15.53:0/3575182065 learned_addr learned my addr 172.21.15.53:0/3575182065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.606+0000 7f966effd640 1 -- 172.21.15.53:0/3575182065 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490358818 0 0) 0x7f967006cfd0 con 0x7f967006dd00 2023-10-21T16:02:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f966effd640 1 -- 172.21.15.53:0/3575182065 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9658003520 con 0x7f967006dd00 2023-10-21T16:02:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f9674b7f640 1 -- 172.21.15.53:0/3575182065 >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 msgr2=0x7f96700cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f9674b7f640 1 --2- 172.21.15.53:0/3575182065 >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f9674b7f640 1 -- 172.21.15.53:0/3575182065 >> v1:172.21.15.124:6789/0 conn(0x7f967006dd00 legacy=0x7f96700c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f9674b7f640 1 -- 172.21.15.53:0/3575182065 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967015ad50 con 0x7f96700cb990 2023-10-21T16:02:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.607+0000 7f966ffff640 1 --2- 172.21.15.53:0/3575182065 >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.608+0000 7f9674b7f640 1 --2- 172.21.15.53:0/3575182065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 0x7f96700ca570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9668009d90 tx=0x7f966802bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24cf2982a6207047 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.608+0000 7f966effd640 1 -- 172.21.15.53:0/3575182065 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966802cb40 con 0x7f96700cb990 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f966effd640 1 -- 172.21.15.53:0/3575182065 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966802cca0 con 0x7f96700cb990 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f966effd640 1 -- 172.21.15.53:0/3575182065 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668034660 con 0x7f96700cb990 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 -- 172.21.15.53:0/3575182065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 msgr2=0x7f96700ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 --2- 172.21.15.53:0/3575182065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 0x7f96700ca570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9668009d90 tx=0x7f966802bd90 comp rx=0 tx=0).stop 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 -- 172.21.15.53:0/3575182065 shutdown_connections 2023-10-21T16:02:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 --2- 172.21.15.53:0/3575182065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96700cb990 0x7f96700ca570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 --2- 172.21.15.53:0/3575182065 >> v2:172.21.15.124:3300/0 conn(0x7f967015a970 0x7f96700cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 -- 172.21.15.53:0/3575182065 >> 172.21.15.53:0/3575182065 conn(0x7f96700ce9d0 msgr2=0x7f96700cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 -- 172.21.15.53:0/3575182065 shutdown_connections 2023-10-21T16:02:04.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.609+0000 7f9675380640 1 -- 172.21.15.53:0/3575182065 wait complete. 2023-10-21T16:02:04.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.611+0000 7f9675380640 1 Processor -- start 2023-10-21T16:02:04.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.611+0000 7f9675380640 1 -- start start 2023-10-21T16:02:04.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.612+0000 7f9675380640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 0x7f96700cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.612+0000 7f9675380640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.612+0000 7f9675380640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f967015a970 0x7f96700b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.612+0000 7f9675380640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700c8640 con 0x7f96700cb990 2023-10-21T16:02:04.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f9675380640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700ccc50 con 0x7f967006dd00 2023-10-21T16:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f9675380640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700cc320 con 0x7f967015a970 2023-10-21T16:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54494/0 (socket says 172.21.15.53:54494) 2023-10-21T16:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 -- 172.21.15.53:0/2282700742 learned_addr learned my addr 172.21.15.53:0/2282700742 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966ffff640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 0x7f96700cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 -- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f967015a970 msgr2=0x7f96700b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f9674b7f640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f967015a970 0x7f96700b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f967015a970 0x7f96700b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 -- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 msgr2=0x7f96700cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 0x7f96700cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966f7fe640 1 -- 172.21.15.53:0/2282700742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96600094d0 con 0x7f96700cb990 2023-10-21T16:02:04.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.613+0000 7f966ffff640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 0x7f96700cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:04.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f966f7fe640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f966400c3c0 tx=0x7f966400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96640052d0 con 0x7f96700cb990 2023-10-21T16:02:04.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9664005450 con 0x7f96700cb990 2023-10-21T16:02:04.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664003830 con 0x7f96700cb990 2023-10-21T16:02:04.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668009a70 con 0x7f96700cb990 2023-10-21T16:02:04.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.614+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700cc8e0 con 0x7f96700cb990 2023-10-21T16:02:04.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.615+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967004e160 con 0x7f96700cb990 2023-10-21T16:02:04.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.616+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9664003990 con 0x7f96700cb990 2023-10-21T16:02:04.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.618+0000 7f966cff9640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 0x7f965c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:04.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.618+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9664088f50 con 0x7f96700cb990 2023-10-21T16:02:04.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.618+0000 7f966ffff640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 0x7f965c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:04.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.619+0000 7f966ffff640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 0x7f965c06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9660009c50 tx=0x7f9660008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:04.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.622+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9664053d30 con 0x7f96700cb990 2023-10-21T16:02:04.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.854+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f967006cb80 con 0x7f96700cb990 2023-10-21T16:02:04.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.855+0000 7f966cff9640 1 -- 172.21.15.53:0/2282700742 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9664017070 con 0x7f96700cb990 2023-10-21T16:02:04.857 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-21T16:02:04.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 msgr2=0x7f965c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 0x7f965c06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9660009c50 tx=0x7f9660008040 comp rx=0 tx=0).stop 2023-10-21T16:02:04.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 msgr2=0x7f96700c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:04.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f966400c3c0 tx=0x7f966400c880 comp rx=0 tx=0).stop 2023-10-21T16:02:04.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 shutdown_connections 2023-10-21T16:02:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f967015a970 0x7f96700b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f965c06ae70 0x7f965c06d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96700cb990 0x7f96700c7f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 --2- 172.21.15.53:0/2282700742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f967006dd00 0x7f96700cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:04.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 >> 172.21.15.53:0/2282700742 conn(0x7f96700ce9d0 msgr2=0x7f967015cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:04.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 shutdown_connections 2023-10-21T16:02:04.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:04.858+0000 7f9675380640 1 -- 172.21.15.53:0/2282700742 wait complete. 2023-10-21T16:02:04.879 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:02:04.879 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:02:04.880 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:02:04.880 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-21T16:02:04.880 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:04.881 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:02:04.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-21T16:02:04.881 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:02:04.881 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-21T16:02:04.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:04.882 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:02:04.937 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:04.937 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:04.937 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:04.938 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:04.938 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:04.938 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:04.938 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:04.938 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:04.939 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:02:04.939 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:04.939 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:04.939 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:02:04.939 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:02:04.940 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:04.940 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:02:04.940 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:02:04.940 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42368sec preferred_lft 42368sec 2023-10-21T16:02:04.940 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:02:04.941 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:04.941 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:04.941 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:04.941 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:02:04.942 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-21T16:02:04.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:04.943 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:04.944 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:04.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2023-10-21T16:02:04.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2023-10-21T16:02:04.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2023-10-21T16:02:04.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-21T16:02:04.945 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:05.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:05.432 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:02:05.519 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:02:05.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:05.523 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:02:05.605 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:02:05.605 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:02:05.605 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:02:05.606 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:02:05.606 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:02:05.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:05.607 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:05.607 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:05.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:02:05.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:02:05.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:02:05.608 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:06.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:06.385 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:02:06.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:06.431 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-10-21T16:02:06.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:06.527 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:06.528 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:06.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:06.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-21T16:02:06.529 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:06.676 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-21T16:02:06.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:06.677 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:06.677 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:06.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-21T16:02:06.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-21T16:02:06.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-21T16:02:06.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-21T16:02:06.679 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-21T16:02:06.679 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:06.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:06.952 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:06.952 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:06.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:02:06.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2023-10-21T16:02:06.953 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:07.108 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-21T16:02:07.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:07.109 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:07.158 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-21T16:02:07.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:07.159 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:07.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2023-10-21T16:02:07.299 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2023-10-21T16:02:07.350 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2023-10-21T16:02:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2023-10-21T16:02:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2023-10-21T16:02:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2023-10-21T16:02:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2023-10-21T16:02:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2023-10-21T16:02:07.352 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2023-10-21T16:02:07.353 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2023-10-21T16:02:07.353 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2023-10-21T16:02:07.353 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2023-10-21T16:02:07.353 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2023-10-21T16:02:07.353 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-21T16:02:07.354 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2023-10-21T16:02:07.354 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2023-10-21T16:02:07.354 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2023-10-21T16:02:07.354 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-21T16:02:07.354 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-0 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-1 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-2 2023-10-21T16:02:07.355 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-3 2023-10-21T16:02:07.356 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2023-10-21T16:02:07.356 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:07.356 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:07.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:07.358 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2023-10-21T16:02:07.402 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-21T16:02:07.403 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-21T16:02:07.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo modprobe fuse 2023-10-21T16:02:07.535 DEBUG:teuthology.orchestra.run.smithi053:> cat /proc/self/mounts | awk '{print $2}' 2023-10-21T16:02:07.579 INFO:teuthology.orchestra.run.smithi053.stdout:/proc 2023-10-21T16:02:07.579 INFO:teuthology.orchestra.run.smithi053.stdout:/sys 2023-10-21T16:02:07.580 INFO:teuthology.orchestra.run.smithi053.stdout:/dev 2023-10-21T16:02:07.580 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/security 2023-10-21T16:02:07.580 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/shm 2023-10-21T16:02:07.581 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/pts 2023-10-21T16:02:07.581 INFO:teuthology.orchestra.run.smithi053.stdout:/run 2023-10-21T16:02:07.581 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/cgroup 2023-10-21T16:02:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/pstore 2023-10-21T16:02:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/ 2023-10-21T16:02:07.582 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/selinux 2023-10-21T16:02:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/proc/sys/fs/binfmt_misc 2023-10-21T16:02:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/hugepages 2023-10-21T16:02:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/dev/mqueue 2023-10-21T16:02:07.583 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/debug 2023-10-21T16:02:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/tracing 2023-10-21T16:02:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-21T16:02:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/fs/fuse/connections 2023-10-21T16:02:07.584 INFO:teuthology.orchestra.run.smithi053.stdout:/sys/kernel/config 2023-10-21T16:02:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-sysctl.service 2023-10-21T16:02:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-21T16:02:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/nfs/rpc_pipefs 2023-10-21T16:02:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/run/user/1000 2023-10-21T16:02:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph 2023-10-21T16:02:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-0 2023-10-21T16:02:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-1 2023-10-21T16:02:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-2 2023-10-21T16:02:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:/var/lib/ceph/osd/ceph-3 2023-10-21T16:02:07.586 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns 2023-10-21T16:02:07.587 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:07.587 INFO:teuthology.orchestra.run.smithi053.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:07.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:07.588 DEBUG:teuthology.orchestra.run.smithi053:> ls /sys/fs/fuse/connections 2023-10-21T16:02:07.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:2023-10-21T16:02:07.650+0000 7f35b5a41480 -1 init, newargv = 0x55715b73cd90 newargc=15 2023-10-21T16:02:07.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[92795]: starting ceph client 2023-10-21T16:02:07.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[92795]: starting fuse 2023-10-21T16:02:07.676 INFO:teuthology.orchestra.run.smithi053.stdout:39 2023-10-21T16:02:07.676 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-10-21T16:02:07.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo stdin-killer -- python3 -c ' 2023-10-21T16:02:07.677 DEBUG:teuthology.orchestra.run.smithi053:> import glob 2023-10-21T16:02:07.678 DEBUG:teuthology.orchestra.run.smithi053:> import re 2023-10-21T16:02:07.678 DEBUG:teuthology.orchestra.run.smithi053:> import os 2023-10-21T16:02:07.678 DEBUG:teuthology.orchestra.run.smithi053:> import subprocess 2023-10-21T16:02:07.679 DEBUG:teuthology.orchestra.run.smithi053:> 2023-10-21T16:02:07.679 DEBUG:teuthology.orchestra.run.smithi053:> def _find_admin_socket(client_name): 2023-10-21T16:02:07.679 DEBUG:teuthology.orchestra.run.smithi053:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-21T16:02:07.680 DEBUG:teuthology.orchestra.run.smithi053:> files = glob.glob(asok_path) 2023-10-21T16:02:07.680 DEBUG:teuthology.orchestra.run.smithi053:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-21T16:02:07.680 DEBUG:teuthology.orchestra.run.smithi053:> 2023-10-21T16:02:07.681 DEBUG:teuthology.orchestra.run.smithi053:> # Given a non-glob path, it better be there 2023-10-21T16:02:07.681 DEBUG:teuthology.orchestra.run.smithi053:> if "*" not in asok_path: 2023-10-21T16:02:07.681 DEBUG:teuthology.orchestra.run.smithi053:> assert(len(files) == 1) 2023-10-21T16:02:07.682 DEBUG:teuthology.orchestra.run.smithi053:> return files[0] 2023-10-21T16:02:07.682 DEBUG:teuthology.orchestra.run.smithi053:> 2023-10-21T16:02:07.682 DEBUG:teuthology.orchestra.run.smithi053:> for f in files: 2023-10-21T16:02:07.683 DEBUG:teuthology.orchestra.run.smithi053:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-21T16:02:07.683 DEBUG:teuthology.orchestra.run.smithi053:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-21T16:02:07.684 DEBUG:teuthology.orchestra.run.smithi053:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-21T16:02:07.684 DEBUG:teuthology.orchestra.run.smithi053:> contents = proc_f.read() 2023-10-21T16:02:07.685 DEBUG:teuthology.orchestra.run.smithi053:> if mountpoint in contents: 2023-10-21T16:02:07.685 DEBUG:teuthology.orchestra.run.smithi053:> return f 2023-10-21T16:02:07.686 DEBUG:teuthology.orchestra.run.smithi053:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-21T16:02:07.686 DEBUG:teuthology.orchestra.run.smithi053:> 2023-10-21T16:02:07.686 DEBUG:teuthology.orchestra.run.smithi053:> print(_find_admin_socket("client.admin")) 2023-10-21T16:02:07.687 DEBUG:teuthology.orchestra.run.smithi053:> ' 2023-10-21T16:02:07.895 INFO:teuthology.orchestra.run.smithi053.stdout:/var/run/ceph/ceph-client.admin.92795.asok 2023-10-21T16:02:07.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-21T16:02:07.914 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.92795.asok 2023-10-21T16:02:07.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:07.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.92795.asok status 2023-10-21T16:02:08.166 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:02:08.167 INFO:teuthology.orchestra.run.smithi053.stdout: "metadata": { 2023-10-21T16:02:08.167 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_sha1": "f0804064d67a2a63dbabf1801ff452580a050cfc", 2023-10-21T16:02:08.167 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 18.0.0-6820-gf0804064 (f0804064d67a2a63dbabf1801ff452580a050cfc) reef (dev)", 2023-10-21T16:02:08.167 INFO:teuthology.orchestra.run.smithi053.stdout: "entity_id": "admin", 2023-10-21T16:02:08.167 INFO:teuthology.orchestra.run.smithi053.stdout: "hostname": "smithi053", 2023-10-21T16:02:08.168 INFO:teuthology.orchestra.run.smithi053.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-21T16:02:08.168 INFO:teuthology.orchestra.run.smithi053.stdout: "pid": "92795", 2023-10-21T16:02:08.168 INFO:teuthology.orchestra.run.smithi053.stdout: "root": "/" 2023-10-21T16:02:08.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:02:08.168 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_count": 0, 2023-10-21T16:02:08.169 INFO:teuthology.orchestra.run.smithi053.stdout: "dentry_pinned_count": 0, 2023-10-21T16:02:08.169 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 4413, 2023-10-21T16:02:08.169 INFO:teuthology.orchestra.run.smithi053.stdout: "inst": { 2023-10-21T16:02:08.169 INFO:teuthology.orchestra.run.smithi053.stdout: "name": { 2023-10-21T16:02:08.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "client", 2023-10-21T16:02:08.170 INFO:teuthology.orchestra.run.smithi053.stdout: "num": 4413 2023-10-21T16:02:08.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:02:08.170 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2023-10-21T16:02:08.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2023-10-21T16:02:08.170 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2023-10-21T16:02:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1628543261 2023-10-21T16:02:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:02:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:02:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": { 2023-10-21T16:02:08.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2023-10-21T16:02:08.172 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "192.168.0.1:0", 2023-10-21T16:02:08.172 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1628543261 2023-10-21T16:02:08.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:02:08.172 INFO:teuthology.orchestra.run.smithi053.stdout: "inst_str": "client.4413 192.168.0.1:0/1628543261", 2023-10-21T16:02:08.173 INFO:teuthology.orchestra.run.smithi053.stdout: "addr_str": "192.168.0.1:0/1628543261", 2023-10-21T16:02:08.173 INFO:teuthology.orchestra.run.smithi053.stdout: "inode_count": 1, 2023-10-21T16:02:08.173 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_epoch": 7, 2023-10-21T16:02:08.173 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch": 26, 2023-10-21T16:02:08.173 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_epoch_barrier": 0, 2023-10-21T16:02:08.174 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklisted": false, 2023-10-21T16:02:08.174 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs" 2023-10-21T16:02:08.174 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:02:08.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:08.182 DEBUG:teuthology.orchestra.run.smithi053:> stat --file-system '--printf=%T 2023-10-21T16:02:08.183 DEBUG:teuthology.orchestra.run.smithi053:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:08.240 INFO:teuthology.orchestra.run.smithi053.stdout:fuseblk 2023-10-21T16:02:08.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:08.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:08.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:08.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:08.346 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-21T16:02:08.442 INFO:teuthology.orchestra.run.smithi053.stdout:# file: . 2023-10-21T16:02:08.443 INFO:teuthology.orchestra.run.smithi053.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-21T16:02:08.443 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:08.444 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:02:08.444 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:08.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.733+0000 7feca1b41640 1 Processor -- start 2023-10-21T16:02:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.733+0000 7feca1b41640 1 -- start start 2023-10-21T16:02:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7feca1b41640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec9c0b09c0 0x7fec9c0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7feca1b41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15af40 0x7fec9c06c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7feca1b41640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c15ec40 con 0x7fec9c15ab60 2023-10-21T16:02:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7feca1b41640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0b1e00 con 0x7fec9c0b09c0 2023-10-21T16:02:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7feca1b41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15ef20 con 0x7fec9c15af40 2023-10-21T16:02:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7fec92ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec9c0b09c0 0x7fec9c0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7fec9b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fec9c15ab60 0x7fec9c0b02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48632/0 (socket says 172.21.15.53:48632) 2023-10-21T16:02:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7fec92ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec9c0b09c0 0x7fec9c0b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54504/0 (socket says 172.21.15.53:54504) 2023-10-21T16:02:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7fec9b7fe640 1 -- 172.21.15.53:0/3588788506 learned_addr learned my addr 172.21.15.53:0/3588788506 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.735+0000 7fec9bfff640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15af40 0x7fec9c06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.736+0000 7fec92ffd640 1 -- 172.21.15.53:0/3588788506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15af40 msgr2=0x7fec9c06c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.736+0000 7fec92ffd640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15af40 0x7fec9c06c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.736+0000 7fec92ffd640 1 -- 172.21.15.53:0/3588788506 >> v1:172.21.15.53:6789/0 conn(0x7fec9c15ab60 legacy=0x7fec9c0b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.736+0000 7fec92ffd640 1 -- 172.21.15.53:0/3588788506 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c15f200 con 0x7fec9c0b09c0 2023-10-21T16:02:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.736+0000 7fec9affd640 1 -- 172.21.15.53:0/3588788506 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073786975 0 0) 0x7fec9c15ec40 con 0x7fec9c15ab60 2023-10-21T16:02:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.737+0000 7fec92ffd640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c0b09c0 0x7fec9c0b18c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec840097a0 tx=0x7fec8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b5213b405f73d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:08.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.737+0000 7fec9affd640 1 -- 172.21.15.53:0/3588788506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8402d040 con 0x7fec9c0b09c0 2023-10-21T16:02:08.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.737+0000 7fec9affd640 1 -- 172.21.15.53:0/3588788506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84031810 con 0x7fec9c0b09c0 2023-10-21T16:02:08.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.737+0000 7fec9affd640 1 -- 172.21.15.53:0/3588788506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84031ba0 con 0x7fec9c0b09c0 2023-10-21T16:02:08.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 -- 172.21.15.53:0/3588788506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c0b09c0 msgr2=0x7fec9c0b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c0b09c0 0x7fec9c0b18c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec840097a0 tx=0x7fec8402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 -- 172.21.15.53:0/3588788506 shutdown_connections 2023-10-21T16:02:08.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c15af40 0x7fec9c06c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 --2- 172.21.15.53:0/3588788506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c0b09c0 0x7fec9c0b18c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 -- 172.21.15.53:0/3588788506 >> 172.21.15.53:0/3588788506 conn(0x7fec9c0cb2b0 msgr2=0x7fec9c0cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:08.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 -- 172.21.15.53:0/3588788506 shutdown_connections 2023-10-21T16:02:08.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.738+0000 7feca1b41640 1 -- 172.21.15.53:0/3588788506 wait complete. 2023-10-21T16:02:08.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.741+0000 7feca1b41640 1 Processor -- start 2023-10-21T16:02:08.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.741+0000 7feca1b41640 1 -- start start 2023-10-21T16:02:08.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b09c0 0x7fec9c1f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15ab60 0x7fec9c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1f24d0 con 0x7fec9c15af40 2023-10-21T16:02:08.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c38b0 con 0x7fec9c15ab60 2023-10-21T16:02:08.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7feca1b41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0c3b90 con 0x7fec9c0b09c0 2023-10-21T16:02:08.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7fec9bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7fec9bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54518/0 (socket says 172.21.15.53:54518) 2023-10-21T16:02:08.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.742+0000 7fec9bfff640 1 -- 172.21.15.53:0/4150294782 learned_addr learned my addr 172.21.15.53:0/4150294782 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:08.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec92ffd640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15ab60 0x7fec9c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9b7fe640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b09c0 0x7fec9c1f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 -- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b09c0 msgr2=0x7fec9c1f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b09c0 0x7fec9c1f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 -- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15ab60 msgr2=0x7fec9c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15ab60 0x7fec9c1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 -- 172.21.15.53:0/4150294782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84009400 con 0x7fec9c15af40 2023-10-21T16:02:08.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.743+0000 7fec9bfff640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec9400d4e0 tx=0x7fec9400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:08.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.744+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec940125f0 con 0x7fec9c15af40 2023-10-21T16:02:08.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.744+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94012770 con 0x7fec9c15af40 2023-10-21T16:02:08.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.744+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94012b20 con 0x7fec9c15af40 2023-10-21T16:02:08.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.744+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0c3e70 con 0x7fec9c15af40 2023-10-21T16:02:08.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.744+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c1f3cd0 con 0x7fec9c15af40 2023-10-21T16:02:08.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.745+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c071170 con 0x7fec9c15af40 2023-10-21T16:02:08.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.746+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fec94012c80 con 0x7fec9c15af40 2023-10-21T16:02:08.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.747+0000 7fec98ff9640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 0x7fec7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:08.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.747+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fec94089d50 con 0x7fec9c15af40 2023-10-21T16:02:08.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.748+0000 7fec9b7fe640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 0x7fec7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:08.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.749+0000 7fec9b7fe640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 0x7fec7c06d360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0012e0 tx=0x7fec8c007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:08.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.753+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec94054b00 con 0x7fec9c15af40 2023-10-21T16:02:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.949+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9c067490 con 0x7fec9c15af40 2023-10-21T16:02:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.950+0000 7fec98ff9640 1 -- 172.21.15.53:0/4150294782 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fec94058120 con 0x7fec9c15af40 2023-10-21T16:02:08.953 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:08.953 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 msgr2=0x7fec7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 0x7fec7c06d360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0012e0 tx=0x7fec8c007ef0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 msgr2=0x7fec9c0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec9400d4e0 tx=0x7fec9400d9a0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 shutdown_connections 2023-10-21T16:02:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fec7c06aea0 0x7fec7c06d360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec9c15af40 0x7fec9c0ccf10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec9c15ab60 0x7fec9c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 --2- 172.21.15.53:0/4150294782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec9c0b09c0 0x7fec9c1f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 >> 172.21.15.53:0/4150294782 conn(0x7fec9c0cb2b0 msgr2=0x7fec9c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 shutdown_connections 2023-10-21T16:02:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:08.953+0000 7feca1b41640 1 -- 172.21.15.53:0/4150294782 wait complete. 2023-10-21T16:02:08.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:08.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.202+0000 7fceffabb640 1 Processor -- start 2023-10-21T16:02:09.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.202+0000 7fceffabb640 1 -- start start 2023-10-21T16:02:09.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.203+0000 7fceffabb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcef815a820 0x7fcef8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.203+0000 7fceffabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fceffabb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef80c7ad0 con 0x7fcef8070b50 2023-10-21T16:02:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fceffabb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef80c9a40 con 0x7fcef815a820 2023-10-21T16:02:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fceffabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef815ac00 con 0x7fcef80c9080 2023-10-21T16:02:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcef815a820 0x7fcef8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcef815a820 0x7fcef8160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54520/0 (socket says 172.21.15.53:54520) 2023-10-21T16:02:09.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 -- 172.21.15.53:0/1051010466 learned_addr learned my addr 172.21.15.53:0/1051010466 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:09.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd056640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 -- 172.21.15.53:0/1051010466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 msgr2=0x7fcef80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:09.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef80c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 -- 172.21.15.53:0/1051010466 >> v1:172.21.15.124:6789/0 conn(0x7fcef8070b50 legacy=0x7fcef80c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:09.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.204+0000 7fcefd857640 1 -- 172.21.15.53:0/1051010466 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80cb210 con 0x7fcef815a820 2023-10-21T16:02:09.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.206+0000 7fcefd857640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef8160750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fceec009e50 tx=0x7fceec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e152aa7470e7ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.206+0000 7fcefc855640 1 -- 172.21.15.53:0/1051010466 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec02ca80 con 0x7fcef815a820 2023-10-21T16:02:09.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.206+0000 7fcefc855640 1 -- 172.21.15.53:0/1051010466 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec02cbe0 con 0x7fcef815a820 2023-10-21T16:02:09.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.206+0000 7fcefc855640 1 -- 172.21.15.53:0/1051010466 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0343f0 con 0x7fcef815a820 2023-10-21T16:02:09.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 -- 172.21.15.53:0/1051010466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 msgr2=0x7fcef8160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef8160750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fceec009e50 tx=0x7fceec02bee0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 -- 172.21.15.53:0/1051010466 shutdown_connections 2023-10-21T16:02:09.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 --2- 172.21.15.53:0/1051010466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef8160750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 -- 172.21.15.53:0/1051010466 >> 172.21.15.53:0/1051010466 conn(0x7fcef8069d50 msgr2=0x7fcef806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:09.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 -- 172.21.15.53:0/1051010466 shutdown_connections 2023-10-21T16:02:09.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.207+0000 7fceffabb640 1 -- 172.21.15.53:0/1051010466 wait complete. 2023-10-21T16:02:09.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.209+0000 7fceffabb640 1 Processor -- start 2023-10-21T16:02:09.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.209+0000 7fceffabb640 1 -- start start 2023-10-21T16:02:09.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcef8070b50 0x7fcef8148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef8160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8160ef0 con 0x7fcef815a820 2023-10-21T16:02:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8154840 con 0x7fcef8070b50 2023-10-21T16:02:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fceffabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8154560 con 0x7fcef80c9080 2023-10-21T16:02:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fcefe058640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fcefe058640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54536/0 (socket says 172.21.15.53:54536) 2023-10-21T16:02:09.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fcefe058640 1 -- 172.21.15.53:0/2702930475 learned_addr learned my addr 172.21.15.53:0/2702930475 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:09.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fcefd056640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.210+0000 7fcefd857640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcef8070b50 0x7fcef8148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 -- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 msgr2=0x7fcef8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 -- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcef8070b50 msgr2=0x7fcef8148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcef8070b50 0x7fcef8148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 -- 172.21.15.53:0/2702930475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec009b30 con 0x7fcef815a820 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcefe058640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c410 tx=0x7fcef000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0011790 con 0x7fcef815a820 2023-10-21T16:02:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00118f0 con 0x7fcef815a820 2023-10-21T16:02:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0011ce0 con 0x7fcef815a820 2023-10-21T16:02:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef815d270 con 0x7fcef815a820 2023-10-21T16:02:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.211+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef815d830 con 0x7fcef815a820 2023-10-21T16:02:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.213+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef804e160 con 0x7fcef815a820 2023-10-21T16:02:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.214+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcef001c460 con 0x7fcef815a820 2023-10-21T16:02:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.215+0000 7fcef67fc640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 0x7fcedc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.215+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcef0089680 con 0x7fcef815a820 2023-10-21T16:02:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.216+0000 7fcefd857640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 0x7fcedc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.217+0000 7fcefd857640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 0x7fcedc06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fceec035000 tx=0x7fceec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.220+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcef00543a0 con 0x7fcef815a820 2023-10-21T16:02:09.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.411+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcef80b4380 con 0x7fcef815a820 2023-10-21T16:02:09.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.412+0000 7fcef67fc640 1 -- 172.21.15.53:0/2702930475 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcef00579c0 con 0x7fcef815a820 2023-10-21T16:02:09.415 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:09.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 msgr2=0x7fcedc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 0x7fcedc06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fceec035000 tx=0x7fceec030000 comp rx=0 tx=0).stop 2023-10-21T16:02:09.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 msgr2=0x7fcef81609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c410 tx=0x7fcef000c8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 shutdown_connections 2023-10-21T16:02:09.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fcedc06ae70 0x7fcedc06d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef815a820 0x7fcef81609b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcef80c9080 0x7fcef8160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 --2- 172.21.15.53:0/2702930475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcef8070b50 0x7fcef8148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 >> 172.21.15.53:0/2702930475 conn(0x7fcef8069d50 msgr2=0x7fcef8154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:09.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 shutdown_connections 2023-10-21T16:02:09.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.415+0000 7fceffabb640 1 -- 172.21.15.53:0/2702930475 wait complete. 2023-10-21T16:02:09.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.654+0000 7fc9267e9640 1 Processor -- start 2023-10-21T16:02:09.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.655+0000 7fc9267e9640 1 -- start start 2023-10-21T16:02:09.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.656+0000 7fc9267e9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9200aff70 0x7fc9200b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.656+0000 7fc9267e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc9200ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.656+0000 7fc9267e9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9200c9250 con 0x7fc9200c18d0 2023-10-21T16:02:09.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.656+0000 7fc9267e9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc920161590 con 0x7fc9200aff70 2023-10-21T16:02:09.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.656+0000 7fc9267e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200cc570 con 0x7fc920164d10 2023-10-21T16:02:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.657+0000 7fc9257e7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9200aff70 0x7fc9200b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.657+0000 7fc9257e7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9200aff70 0x7fc9200b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54552/0 (socket says 172.21.15.53:54552) 2023-10-21T16:02:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.657+0000 7fc9257e7640 1 -- 172.21.15.53:0/1519638746 learned_addr learned my addr 172.21.15.53:0/1519638746 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.657+0000 7fc925fe8640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc9200ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.657+0000 7fc9257e7640 1 -- 172.21.15.53:0/1519638746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 msgr2=0x7fc9200ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:09.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.658+0000 7fc9257e7640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc9200ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.658+0000 7fc9257e7640 1 -- 172.21.15.53:0/1519638746 >> v1:172.21.15.124:6789/0 conn(0x7fc9200c18d0 legacy=0x7fc9200c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.658+0000 7fc9257e7640 1 -- 172.21.15.53:0/1519638746 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200c94b0 con 0x7fc9200aff70 2023-10-21T16:02:09.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.658+0000 7fc925fe8640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc9200ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:09.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.659+0000 7fc9257e7640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200b0350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009870 tx=0x7fc91c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3eec81f33a448c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc917fff640 1 -- 172.21.15.53:0/1519638746 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c031640 con 0x7fc9200aff70 2023-10-21T16:02:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc917fff640 1 -- 172.21.15.53:0/1519638746 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c0317a0 con 0x7fc9200aff70 2023-10-21T16:02:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc917fff640 1 -- 172.21.15.53:0/1519638746 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c031b30 con 0x7fc9200aff70 2023-10-21T16:02:09.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 -- 172.21.15.53:0/1519638746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 msgr2=0x7fc9200b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200b0350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009870 tx=0x7fc91c02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 -- 172.21.15.53:0/1519638746 shutdown_connections 2023-10-21T16:02:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc9200ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1519638746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200b0350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 -- 172.21.15.53:0/1519638746 >> 172.21.15.53:0/1519638746 conn(0x7fc92006c050 msgr2=0x7fc92006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 -- 172.21.15.53:0/1519638746 shutdown_connections 2023-10-21T16:02:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.660+0000 7fc9267e9640 1 -- 172.21.15.53:0/1519638746 wait complete. 2023-10-21T16:02:09.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.662+0000 7fc9267e9640 1 Processor -- start 2023-10-21T16:02:09.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.662+0000 7fc9267e9640 1 -- start start 2023-10-21T16:02:09.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9200c18d0 0x7fc9200c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc92015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9201625b0 con 0x7fc9200aff70 2023-10-21T16:02:09.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92015f040 con 0x7fc9200c18d0 2023-10-21T16:02:09.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.663+0000 7fc9267e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200c3ae0 con 0x7fc920164d10 2023-10-21T16:02:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54558/0 (socket says 172.21.15.53:54558) 2023-10-21T16:02:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 -- 172.21.15.53:0/1085292377 learned_addr learned my addr 172.21.15.53:0/1085292377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc925fe8640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc92015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc924fe6640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9200c18d0 0x7fc9200c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 -- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 msgr2=0x7fc92015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc92015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 -- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9200c18d0 msgr2=0x7fc9200c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9200c18d0 0x7fc9200c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 -- 172.21.15.53:0/1085292377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0094d0 con 0x7fc9200aff70 2023-10-21T16:02:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.664+0000 7fc9257e7640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.665+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c03ac60 con 0x7fc9200aff70 2023-10-21T16:02:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.665+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c03adc0 con 0x7fc9200aff70 2023-10-21T16:02:09.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.665+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c044630 con 0x7fc9200aff70 2023-10-21T16:02:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.665+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200ad8d0 con 0x7fc9200aff70 2023-10-21T16:02:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.665+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9200c4060 con 0x7fc9200aff70 2023-10-21T16:02:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.666+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920074150 con 0x7fc9200aff70 2023-10-21T16:02:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.667+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc91c02f050 con 0x7fc9200aff70 2023-10-21T16:02:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.669+0000 7fc915ffb640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 0x7fc90406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.669+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc91c0ab710 con 0x7fc9200aff70 2023-10-21T16:02:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.669+0000 7fc924fe6640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 0x7fc90406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.670+0000 7fc924fe6640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 0x7fc90406d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc9200c4fc0 tx=0x7fc910009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.673+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc91c076560 con 0x7fc9200aff70 2023-10-21T16:02:09.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.871+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9200b4470 con 0x7fc9200aff70 2023-10-21T16:02:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.872+0000 7fc915ffb640 1 -- 172.21.15.53:0/1085292377 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc91c031e60 con 0x7fc9200aff70 2023-10-21T16:02:09.875 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 msgr2=0x7fc90406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 0x7fc90406d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc9200c4fc0 tx=0x7fc910009040 comp rx=0 tx=0).stop 2023-10-21T16:02:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 msgr2=0x7fc9200714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 shutdown_connections 2023-10-21T16:02:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.874+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc920164d10 0x7fc92015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9200c18d0 0x7fc9200c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc90406ae70 0x7fc90406d330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 --2- 172.21.15.53:0/1085292377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9200aff70 0x7fc9200714a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 >> 172.21.15.53:0/1085292377 conn(0x7fc92006c050 msgr2=0x7fc920070960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 shutdown_connections 2023-10-21T16:02:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:09.875+0000 7fc9267e9640 1 -- 172.21.15.53:0/1085292377 wait complete. 2023-10-21T16:02:09.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.122+0000 7ffbe359e640 1 Processor -- start 2023-10-21T16:02:10.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.122+0000 7ffbe359e640 1 -- start start 2023-10-21T16:02:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffbe415a820 0x7ffbe40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe40ca680 0x7ffbe40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbe415ac00 0x7ffbe40c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40cab60 con 0x7ffbe415ac00 2023-10-21T16:02:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40cb960 con 0x7ffbe415a820 2023-10-21T16:02:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40cbdf0 con 0x7ffbe40ca680 2023-10-21T16:02:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe2d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbe415ac00 0x7ffbe40c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe2d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffbe415ac00 0x7ffbe40c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54572/0 (socket says 172.21.15.53:54572) 2023-10-21T16:02:10.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.124+0000 7ffbe2d9d640 1 -- 172.21.15.53:0/1558023759 learned_addr learned my addr 172.21.15.53:0/1558023759 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbdbfff640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe40ca680 0x7ffbe40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe259c640 1 --2- 172.21.15.53:0/1558023759 >> v2:172.21.15.124:3300/0 conn(0x7ffbe415a820 0x7ffbe40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe2d9d640 1 -- 172.21.15.53:0/1558023759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe40ca680 msgr2=0x7ffbe40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe2d9d640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe40ca680 0x7ffbe40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe2d9d640 1 -- 172.21.15.53:0/1558023759 >> v2:172.21.15.124:3300/0 conn(0x7ffbe415a820 msgr2=0x7ffbe40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe2d9d640 1 --2- 172.21.15.53:0/1558023759 >> v2:172.21.15.124:3300/0 conn(0x7ffbe415a820 0x7ffbe40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.125+0000 7ffbe2d9d640 1 -- 172.21.15.53:0/1558023759 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe40cb030 con 0x7ffbe415ac00 2023-10-21T16:02:10.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.126+0000 7ffbe2d9d640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415ac00 0x7ffbe40c9100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffbe406da70 tx=0x7ffbdc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c1c9f5765f5322 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe1d9b640 1 -- 172.21.15.53:0/1558023759 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc02c9f0 con 0x7ffbe415ac00 2023-10-21T16:02:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe1d9b640 1 -- 172.21.15.53:0/1558023759 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc02cb50 con 0x7ffbe415ac00 2023-10-21T16:02:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe1d9b640 1 -- 172.21.15.53:0/1558023759 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0343f0 con 0x7ffbe415ac00 2023-10-21T16:02:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 -- 172.21.15.53:0/1558023759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415ac00 msgr2=0x7ffbe40c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415ac00 0x7ffbe40c9100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffbe406da70 tx=0x7ffbdc02be50 comp rx=0 tx=0).stop 2023-10-21T16:02:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 -- 172.21.15.53:0/1558023759 shutdown_connections 2023-10-21T16:02:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe40ca680 0x7ffbe40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1558023759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415ac00 0x7ffbe40c9100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1558023759 >> v2:172.21.15.124:3300/0 conn(0x7ffbe415a820 0x7ffbe40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 -- 172.21.15.53:0/1558023759 >> 172.21.15.53:0/1558023759 conn(0x7ffbe406c050 msgr2=0x7ffbe406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 -- 172.21.15.53:0/1558023759 shutdown_connections 2023-10-21T16:02:10.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.127+0000 7ffbe359e640 1 -- 172.21.15.53:0/1558023759 wait complete. 2023-10-21T16:02:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.129+0000 7ffbe359e640 1 Processor -- start 2023-10-21T16:02:10.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.129+0000 7ffbe359e640 1 -- start start 2023-10-21T16:02:10.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffbe40ca680 0x7ffbe4154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 0x7ffbe415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40ada40 con 0x7ffbe415a820 2023-10-21T16:02:10.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe4155250 con 0x7ffbe40ca680 2023-10-21T16:02:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.130+0000 7ffbe359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40c25e0 con 0x7ffbe415ac00 2023-10-21T16:02:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54582/0 (socket says 172.21.15.53:54582) 2023-10-21T16:02:10.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 -- 172.21.15.53:0/1301941817 learned_addr learned my addr 172.21.15.53:0/1301941817 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbe2d9d640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 0x7ffbe415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbe259c640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffbe40ca680 0x7ffbe4154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 -- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 msgr2=0x7ffbe415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 0x7ffbe415d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 -- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffbe40ca680 msgr2=0x7ffbe4154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffbe40ca680 0x7ffbe4154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbdbfff640 1 -- 172.21.15.53:0/1301941817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc009b30 con 0x7ffbe415a820 2023-10-21T16:02:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.131+0000 7ffbe2d9d640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 0x7ffbe415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbdbfff640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00d620 tx=0x7ffbcc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0125f0 con 0x7ffbe415a820 2023-10-21T16:02:10.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe40c28c0 con 0x7ffbe415a820 2023-10-21T16:02:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc012750 con 0x7ffbe415a820 2023-10-21T16:02:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc012ac0 con 0x7ffbe415a820 2023-10-21T16:02:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.132+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe40c2e80 con 0x7ffbe415a820 2023-10-21T16:02:10.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.133+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe4074150 con 0x7ffbe415a820 2023-10-21T16:02:10.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.134+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffbcc012c20 con 0x7ffbe415a820 2023-10-21T16:02:10.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.135+0000 7ffbdaffd640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 0x7ffbc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.135+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffbcc089d50 con 0x7ffbe415a820 2023-10-21T16:02:10.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.136+0000 7ffbe259c640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 0x7ffbc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.137+0000 7ffbe259c640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 0x7ffbc406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4006fd0 tx=0x7ffbd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.139+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffbcc054b00 con 0x7ffbe415a820 2023-10-21T16:02:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.331+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbe40ccc50 con 0x7ffbe415a820 2023-10-21T16:02:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.332+0000 7ffbdaffd640 1 -- 172.21.15.53:0/1301941817 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbcc012ed0 con 0x7ffbe415a820 2023-10-21T16:02:10.335 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 msgr2=0x7ffbc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 0x7ffbc406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4006fd0 tx=0x7ffbd4008040 comp rx=0 tx=0).stop 2023-10-21T16:02:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 msgr2=0x7ffbe415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00d620 tx=0x7ffbcc00dae0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 shutdown_connections 2023-10-21T16:02:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffbe415ac00 0x7ffbe415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffbc406ae70 0x7ffbc406d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbe415a820 0x7ffbe415d340 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 --2- 172.21.15.53:0/1301941817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffbe40ca680 0x7ffbe4154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.335+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 >> 172.21.15.53:0/1301941817 conn(0x7ffbe406c050 msgr2=0x7ffbe40cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:10.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.336+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 shutdown_connections 2023-10-21T16:02:10.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.336+0000 7ffbe359e640 1 -- 172.21.15.53:0/1301941817 wait complete. 2023-10-21T16:02:10.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:10.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.583+0000 7f3488b3a640 1 Processor -- start 2023-10-21T16:02:10.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.583+0000 7f3488b3a640 1 -- start start 2023-10-21T16:02:10.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.584+0000 7f3488b3a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3484161690 0x7f348406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f3488b3a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 0x7f34840cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f3488b3a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348406cc50 con 0x7f34840cb150 2023-10-21T16:02:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f3488b3a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34840cbb40 con 0x7f3484161690 2023-10-21T16:02:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f3488b3a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348406cf30 con 0x7f34840b2f30 2023-10-21T16:02:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f348259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3484161690 0x7f348406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f348259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3484161690 0x7f348406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54590/0 (socket says 172.21.15.53:54590) 2023-10-21T16:02:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f348259c640 1 -- 172.21.15.53:0/2712028910 learned_addr learned my addr 172.21.15.53:0/2712028910 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.585+0000 7f3482d9d640 1 --2- 172.21.15.53:0/2712028910 >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 0x7f34840cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f348259c640 1 -- 172.21.15.53:0/2712028910 >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 msgr2=0x7f34840cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f348259c640 1 --2- 172.21.15.53:0/2712028910 >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 0x7f34840cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f348259c640 1 -- 172.21.15.53:0/2712028910 >> v1:172.21.15.53:6789/0 conn(0x7f34840cb150 legacy=0x7f34840702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f3481d9b640 1 -- 172.21.15.53:0/2712028910 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107709120 0 0) 0x7f348406cc50 con 0x7f34840cb150 2023-10-21T16:02:10.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f348259c640 1 -- 172.21.15.53:0/2712028910 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348406d210 con 0x7f3484161690 2023-10-21T16:02:10.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.586+0000 7f3482d9d640 1 --2- 172.21.15.53:0/2712028910 >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 0x7f34840cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:10.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.587+0000 7f348259c640 1 --2- 172.21.15.53:0/2712028910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f348406dad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f348406d890 tx=0x7f347402b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d653ebc6e5f49da server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.587+0000 7f3481d9b640 1 -- 172.21.15.53:0/2712028910 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474030880 con 0x7f3484161690 2023-10-21T16:02:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.587+0000 7f3481d9b640 1 -- 172.21.15.53:0/2712028910 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34740309e0 con 0x7f3484161690 2023-10-21T16:02:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.587+0000 7f3481d9b640 1 -- 172.21.15.53:0/2712028910 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474030d70 con 0x7f3484161690 2023-10-21T16:02:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 -- 172.21.15.53:0/2712028910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 msgr2=0x7f348406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2712028910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f348406dad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f348406d890 tx=0x7f347402b9c0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 -- 172.21.15.53:0/2712028910 shutdown_connections 2023-10-21T16:02:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2712028910 >> v2:172.21.15.124:3300/0 conn(0x7f34840b2f30 0x7f34840cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2712028910 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f348406dad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 -- 172.21.15.53:0/2712028910 >> 172.21.15.53:0/2712028910 conn(0x7f348406a5d0 msgr2=0x7f34840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 -- 172.21.15.53:0/2712028910 shutdown_connections 2023-10-21T16:02:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.588+0000 7f3488b3a640 1 -- 172.21.15.53:0/2712028910 wait complete. 2023-10-21T16:02:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.590+0000 7f3488b3a640 1 Processor -- start 2023-10-21T16:02:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.590+0000 7f3488b3a640 1 -- start start 2023-10-21T16:02:10.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34840b2f30 0x7f3484162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34840cb150 0x7f3484157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c7040 con 0x7f3484161690 2023-10-21T16:02:10.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c66a0 con 0x7f34840b2f30 2023-10-21T16:02:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3488b3a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c6980 con 0x7f34840cb150 2023-10-21T16:02:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3482d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54592/0 (socket says 172.21.15.53:54592) 2023-10-21T16:02:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.591+0000 7f3482d9d640 1 -- 172.21.15.53:0/2166198836 learned_addr learned my addr 172.21.15.53:0/2166198836 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f347bfff640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34840cb150 0x7f3484157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f348259c640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34840b2f30 0x7f3484162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 -- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34840cb150 msgr2=0x7f3484157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34840cb150 0x7f3484157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 -- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34840b2f30 msgr2=0x7f3484162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34840b2f30 0x7f3484162260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 -- 172.21.15.53:0/2166198836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34740094d0 con 0x7f3484161690 2023-10-21T16:02:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3482d9d640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f347c007ef0 tx=0x7f347c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c003b90 con 0x7f3484161690 2023-10-21T16:02:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c00f040 con 0x7f3484161690 2023-10-21T16:02:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00de80 con 0x7f3484161690 2023-10-21T16:02:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.592+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840c6c60 con 0x7f3484161690 2023-10-21T16:02:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.593+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34841627a0 con 0x7f3484161690 2023-10-21T16:02:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.594+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34840c7180 con 0x7f3484161690 2023-10-21T16:02:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.595+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f347c004030 con 0x7f3484161690 2023-10-21T16:02:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.596+0000 7f347affd640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 0x7f346406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.596+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f347c089b70 con 0x7f3484161690 2023-10-21T16:02:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.596+0000 7f348259c640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 0x7f346406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.598+0000 7f348259c640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 0x7f346406d380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3474036000 tx=0x7f347403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.600+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34840c7180 con 0x7f3484161690 2023-10-21T16:02:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.801+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f34840c7180 con 0x7f3484161690 2023-10-21T16:02:10.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.802+0000 7f347affd640 1 -- 172.21.15.53:0/2166198836 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34840c7180 con 0x7f3484161690 2023-10-21T16:02:10.805 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:10.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 msgr2=0x7f346406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 0x7f346406d380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3474036000 tx=0x7f347403b040 comp rx=0 tx=0).stop 2023-10-21T16:02:10.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 msgr2=0x7f34840c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:10.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f347c007ef0 tx=0x7f347c00d4a0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 shutdown_connections 2023-10-21T16:02:10.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f346406aec0 0x7f346406d380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3484161690 0x7f34840c7460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34840cb150 0x7f3484157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 --2- 172.21.15.53:0/2166198836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34840b2f30 0x7f3484162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.804+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 >> 172.21.15.53:0/2166198836 conn(0x7f348406a5d0 msgr2=0x7f34840b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.805+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 shutdown_connections 2023-10-21T16:02:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:10.805+0000 7f3488b3a640 1 -- 172.21.15.53:0/2166198836 wait complete. 2023-10-21T16:02:10.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:02:11.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.051+0000 7fe2dd173640 1 Processor -- start 2023-10-21T16:02:11.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.052+0000 7fe2dd173640 1 -- start start 2023-10-21T16:02:11.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.053+0000 7fe2dd173640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 0x7fe2d80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.053+0000 7fe2dd173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 0x7fe2d80b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.053+0000 7fe2dd173640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d80d2870 con 0x7fe2d80d0b90 2023-10-21T16:02:11.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.053+0000 7fe2dd173640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d80d1150 con 0x7fe2d815ccb0 2023-10-21T16:02:11.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.053+0000 7fe2dd173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80d1430 con 0x7fe2d80d00e0 2023-10-21T16:02:11.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.054+0000 7fe2d7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 0x7fe2d80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.054+0000 7fe2dc972640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 0x7fe2d80b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.054+0000 7fe2d7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 0x7fe2d80d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51152/0 (socket says 172.21.15.53:51152) 2023-10-21T16:02:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.054+0000 7fe2d7fff640 1 -- 172.21.15.53:0/2434181133 learned_addr learned my addr 172.21.15.53:0/2434181133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.055+0000 7fe2dc972640 1 -- 172.21.15.53:0/2434181133 >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 msgr2=0x7fe2d80d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.055+0000 7fe2dc972640 1 --2- 172.21.15.53:0/2434181133 >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 0x7fe2d80d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.055+0000 7fe2dc972640 1 -- 172.21.15.53:0/2434181133 >> v1:172.21.15.124:6789/0 conn(0x7fe2d80d0b90 legacy=0x7fe2d80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:11.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.055+0000 7fe2dc972640 1 -- 172.21.15.53:0/2434181133 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80d1710 con 0x7fe2d80d00e0 2023-10-21T16:02:11.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.056+0000 7fe2dc972640 1 --2- 172.21.15.53:0/2434181133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 0x7fe2d80b6060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d90 tx=0x7fe2d002bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f87f520afbd2089 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.056+0000 7fe2d6ffd640 1 -- 172.21.15.53:0/2434181133 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d002cb30 con 0x7fe2d80d00e0 2023-10-21T16:02:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2d6ffd640 1 -- 172.21.15.53:0/2434181133 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d002cc90 con 0x7fe2d80d00e0 2023-10-21T16:02:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2d6ffd640 1 -- 172.21.15.53:0/2434181133 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0034440 con 0x7fe2d80d00e0 2023-10-21T16:02:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 -- 172.21.15.53:0/2434181133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 msgr2=0x7fe2d80b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 --2- 172.21.15.53:0/2434181133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 0x7fe2d80b6060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d90 tx=0x7fe2d002bd80 comp rx=0 tx=0).stop 2023-10-21T16:02:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 -- 172.21.15.53:0/2434181133 shutdown_connections 2023-10-21T16:02:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 --2- 172.21.15.53:0/2434181133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d80d00e0 0x7fe2d80b6060 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 --2- 172.21.15.53:0/2434181133 >> v2:172.21.15.124:3300/0 conn(0x7fe2d815ccb0 0x7fe2d80d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 -- 172.21.15.53:0/2434181133 >> 172.21.15.53:0/2434181133 conn(0x7fe2d8071660 msgr2=0x7fe2d8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.057+0000 7fe2dd173640 1 -- 172.21.15.53:0/2434181133 shutdown_connections 2023-10-21T16:02:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.058+0000 7fe2dd173640 1 -- 172.21.15.53:0/2434181133 wait complete. 2023-10-21T16:02:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.060+0000 7fe2dd173640 1 Processor -- start 2023-10-21T16:02:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.060+0000 7fe2dd173640 1 -- start start 2023-10-21T16:02:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 0x7fe2d80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d815ccb0 0x7fe2d815f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80c8c30 con 0x7fe2d80d0b90 2023-10-21T16:02:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81f29c0 con 0x7fe2d80d00e0 2023-10-21T16:02:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dd173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81f2ca0 con 0x7fe2d815ccb0 2023-10-21T16:02:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2d77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2d77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54594/0 (socket says 172.21.15.53:54594) 2023-10-21T16:02:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2d77fe640 1 -- 172.21.15.53:0/1509813121 learned_addr learned my addr 172.21.15.53:0/1509813121 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2dc972640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d815ccb0 0x7fe2d815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.061+0000 7fe2d7fff640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 0x7fe2d80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 -- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d815ccb0 msgr2=0x7fe2d815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d815ccb0 0x7fe2d815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 -- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 msgr2=0x7fe2d80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 0x7fe2d80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 -- 172.21.15.53:0/1509813121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0009a70 con 0x7fe2d80d0b90 2023-10-21T16:02:11.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d7fff640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 0x7fe2d80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:11.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d77fe640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c430 tx=0x7fe2cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.062+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc011790 con 0x7fe2d80d0b90 2023-10-21T16:02:11.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.063+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc0118f0 con 0x7fe2d80d0b90 2023-10-21T16:02:11.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.063+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc011ce0 con 0x7fe2d80d0b90 2023-10-21T16:02:11.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.063+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d81f2f80 con 0x7fe2d80d0b90 2023-10-21T16:02:11.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.063+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d81f3260 con 0x7fe2d80d0b90 2023-10-21T16:02:11.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.064+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d80c79c0 con 0x7fe2d80d0b90 2023-10-21T16:02:11.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.065+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2cc01c460 con 0x7fe2d80d0b90 2023-10-21T16:02:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.066+0000 7fe2d4ff9640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 0x7fe2c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.066+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe2cc089780 con 0x7fe2d80d0b90 2023-10-21T16:02:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.066+0000 7fe2d7fff640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 0x7fe2c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.067+0000 7fe2d7fff640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 0x7fe2c406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009b80 tx=0x7fe2c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.070+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2cc0545d0 con 0x7fe2d80d0b90 2023-10-21T16:02:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.283+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe2d80b5b10 con 0x7fe2d80d0b90 2023-10-21T16:02:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.284+0000 7fe2d4ff9640 1 -- 172.21.15.53:0/1509813121 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe2cc057bf0 con 0x7fe2d80d0b90 2023-10-21T16:02:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:02:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 msgr2=0x7fe2c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 0x7fe2c406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009b80 tx=0x7fe2c8008040 comp rx=0 tx=0).stop 2023-10-21T16:02:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 msgr2=0x7fe2d815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c430 tx=0x7fe2cc00c8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 shutdown_connections 2023-10-21T16:02:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2d815ccb0 0x7fe2d815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe2c406ae70 0x7fe2c406d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2d80d0b90 0x7fe2d815d090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 --2- 172.21.15.53:0/1509813121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2d80d00e0 0x7fe2d80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 >> 172.21.15.53:0/1509813121 conn(0x7fe2d8071660 msgr2=0x7fe2d815c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 shutdown_connections 2023-10-21T16:02:11.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.287+0000 7fe2dd173640 1 -- 172.21.15.53:0/1509813121 wait complete. 2023-10-21T16:02:11.309 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:02:11.309 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:11.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:11.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.580+0000 7f26eed33640 1 Processor -- start 2023-10-21T16:02:11.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.580+0000 7f26eed33640 1 -- start start 2023-10-21T16:02:11.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.581+0000 7f26eed33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 0x7f26e80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26eed33640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26e80cb580 0x7f26e80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26eed33640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e815b2b0 con 0x7f26e80c90b0 2023-10-21T16:02:11.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26eed33640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e815bf30 con 0x7f26e80cb580 2023-10-21T16:02:11.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26eed33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e815f7a0 con 0x7f26e815ab40 2023-10-21T16:02:11.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26ecacf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 0x7f26e80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.582+0000 7f26ed2d0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26e80cb580 0x7f26e80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26e80cb580 0x7f26e80adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51158/0 (socket says 172.21.15.53:51158) 2023-10-21T16:02:11.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26e7fff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f26e80c90b0 0x7f26e80cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:51992/0 (socket says 172.21.15.53:51992) 2023-10-21T16:02:11.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 -- 172.21.15.53:0/1733160681 learned_addr learned my addr 172.21.15.53:0/1733160681 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:11.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ecacf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 0x7f26e80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:36686/0 (socket says 172.21.15.53:36686) 2023-10-21T16:02:11.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 -- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 msgr2=0x7f26e80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:11.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 --2- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 0x7f26e80ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 -- 172.21.15.53:0/1733160681 >> v1:172.21.15.124:6789/0 conn(0x7f26e80c90b0 legacy=0x7f26e80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.583+0000 7f26ed2d0640 1 -- 172.21.15.53:0/1733160681 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e815b590 con 0x7f26e80cb580 2023-10-21T16:02:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.585+0000 7f26ed2d0640 1 --2- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e80cb580 0x7f26e80adc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009e50 tx=0x7f26e002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=990f096cf02de90d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.585+0000 7f26e77fe640 1 -- 172.21.15.53:0/1733160681 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e002cb00 con 0x7f26e80cb580 2023-10-21T16:02:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.585+0000 7f26e77fe640 1 -- 172.21.15.53:0/1733160681 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e002cc60 con 0x7f26e80cb580 2023-10-21T16:02:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.585+0000 7f26e77fe640 1 -- 172.21.15.53:0/1733160681 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00343f0 con 0x7f26e80cb580 2023-10-21T16:02:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 -- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e80cb580 msgr2=0x7f26e80adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 --2- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e80cb580 0x7f26e80adc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009e50 tx=0x7f26e002bf60 comp rx=0 tx=0).stop 2023-10-21T16:02:11.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 -- 172.21.15.53:0/1733160681 shutdown_connections 2023-10-21T16:02:11.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 --2- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e80cb580 0x7f26e80adc60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 --2- 172.21.15.53:0/1733160681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e815ab40 0x7f26e80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 -- 172.21.15.53:0/1733160681 >> 172.21.15.53:0/1733160681 conn(0x7f26e806a380 msgr2=0x7f26e80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:11.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 -- 172.21.15.53:0/1733160681 shutdown_connections 2023-10-21T16:02:11.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.586+0000 7f26eed33640 1 -- 172.21.15.53:0/1733160681 wait complete. 2023-10-21T16:02:11.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.588+0000 7f26eed33640 1 Processor -- start 2023-10-21T16:02:11.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.588+0000 7f26eed33640 1 -- start start 2023-10-21T16:02:11.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e80c90b0 0x7f26e8153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e815ab40 0x7f26e80701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81f0b20 con 0x7f26e80cb580 2023-10-21T16:02:11.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81f0260 con 0x7f26e815ab40 2023-10-21T16:02:11.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26eed33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c62d0 con 0x7f26e80c90b0 2023-10-21T16:02:11.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54598/0 (socket says 172.21.15.53:54598) 2023-10-21T16:02:11.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26e7fff640 1 -- 172.21.15.53:0/2586201016 learned_addr learned my addr 172.21.15.53:0/2586201016 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:11.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26ed2d0640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e815ab40 0x7f26e80701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.589+0000 7f26ecacf640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e80c90b0 0x7f26e8153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 -- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e80c90b0 msgr2=0x7f26e8153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e80c90b0 0x7f26e8153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 -- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e815ab40 msgr2=0x7f26e80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e815ab40 0x7f26e80701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 -- 172.21.15.53:0/2586201016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0009b30 con 0x7f26e80cb580 2023-10-21T16:02:11.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e7fff640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3f0 tx=0x7f26d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80074e0 con 0x7f26e80cb580 2023-10-21T16:02:11.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d8007640 con 0x7f26e80cb580 2023-10-21T16:02:11.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004a00 con 0x7f26e80cb580 2023-10-21T16:02:11.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.590+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e80c65b0 con 0x7f26e80cb580 2023-10-21T16:02:11.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.591+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81f07b0 con 0x7f26e80cb580 2023-10-21T16:02:11.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.592+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e8067490 con 0x7f26e80cb580 2023-10-21T16:02:11.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.593+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f26d8004b60 con 0x7f26e80cb580 2023-10-21T16:02:11.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.594+0000 7f26e57fa640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 0x7f26d406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:11.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.594+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f26d8089200 con 0x7f26e80cb580 2023-10-21T16:02:11.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.594+0000 7f26ecacf640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 0x7f26d406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:11.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.595+0000 7f26ecacf640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 0x7f26d406d380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009ad0 tx=0x7f26dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:11.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.598+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26d8053fa0 con 0x7f26e80cb580 2023-10-21T16:02:11.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.791+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26e81f0970 con 0x7f26e80cb580 2023-10-21T16:02:11.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.794+0000 7f26e57fa640 1 -- 172.21.15.53:0/2586201016 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f26d80575c0 con 0x7f26e80cb580 2023-10-21T16:02:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:11.798 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:11.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 msgr2=0x7f26d406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 0x7f26d406d380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009ad0 tx=0x7f26dc008040 comp rx=0 tx=0).stop 2023-10-21T16:02:11.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 msgr2=0x7f26e80c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:11.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c3f0 tx=0x7f26d800c8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 shutdown_connections 2023-10-21T16:02:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e815ab40 0x7f26e80701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f26d406aec0 0x7f26d406d380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e80cb580 0x7f26e80c5850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 --2- 172.21.15.53:0/2586201016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e80c90b0 0x7f26e8153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 >> 172.21.15.53:0/2586201016 conn(0x7f26e806a380 msgr2=0x7f26e80cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:11.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 shutdown_connections 2023-10-21T16:02:11.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:11.797+0000 7f26eed33640 1 -- 172.21.15.53:0/2586201016 wait complete. 2023-10-21T16:02:11.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:11.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:12.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.053+0000 7fedb6289640 1 Processor -- start 2023-10-21T16:02:12.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.053+0000 7fedb6289640 1 -- start start 2023-10-21T16:02:12.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7fedb6289640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fedb0146d10 0x7fedb0162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7fedb6289640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7fedb6289640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb01650b0 con 0x7fedb0146930 2023-10-21T16:02:12.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7fedb6289640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb0165b80 con 0x7fedb0146d10 2023-10-21T16:02:12.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7fedb6289640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0162540 con 0x7fedb0144020 2023-10-21T16:02:12.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7feda7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fedb0146d10 0x7fedb0162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7feda7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fedb0146d10 0x7fedb0162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54614/0 (socket says 172.21.15.53:54614) 2023-10-21T16:02:12.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.055+0000 7feda7fff640 1 -- 172.21.15.53:0/2323547216 learned_addr learned my addr 172.21.15.53:0/2323547216 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.056+0000 7fedb4826640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.056+0000 7feda7fff640 1 -- 172.21.15.53:0/2323547216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 msgr2=0x7fedb0164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.056+0000 7feda7fff640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0164b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.056+0000 7feda7fff640 1 -- 172.21.15.53:0/2323547216 >> v1:172.21.15.124:6789/0 conn(0x7fedb0146930 legacy=0x7fedb0143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.056+0000 7feda7fff640 1 -- 172.21.15.53:0/2323547216 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb006c9d0 con 0x7fedb0146d10 2023-10-21T16:02:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.057+0000 7feda7fff640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146d10 0x7fedb0162000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed98009870 tx=0x7fed9802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3db4edc96c25681 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.057+0000 7fedaf7fe640 1 -- 172.21.15.53:0/2323547216 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98030840 con 0x7fedb0146d10 2023-10-21T16:02:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.057+0000 7fedaf7fe640 1 -- 172.21.15.53:0/2323547216 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed980309a0 con 0x7fedb0146d10 2023-10-21T16:02:12.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.057+0000 7fedaf7fe640 1 -- 172.21.15.53:0/2323547216 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98030d30 con 0x7fedb0146d10 2023-10-21T16:02:12.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 -- 172.21.15.53:0/2323547216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146d10 msgr2=0x7fedb0162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146d10 0x7fedb0162000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed98009870 tx=0x7fed9802b980 comp rx=0 tx=0).stop 2023-10-21T16:02:12.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 -- 172.21.15.53:0/2323547216 shutdown_connections 2023-10-21T16:02:12.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 --2- 172.21.15.53:0/2323547216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146d10 0x7fedb0162000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 -- 172.21.15.53:0/2323547216 >> 172.21.15.53:0/2323547216 conn(0x7fedb0069d50 msgr2=0x7fedb006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 -- 172.21.15.53:0/2323547216 shutdown_connections 2023-10-21T16:02:12.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.058+0000 7fedb6289640 1 -- 172.21.15.53:0/2323547216 wait complete. 2023-10-21T16:02:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.060+0000 7fedb6289640 1 Processor -- start 2023-10-21T16:02:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.060+0000 7fedb6289640 1 -- start start 2023-10-21T16:02:12.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0162720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fedb0146d10 0x7fedb01f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f7910 con 0x7fedb0146930 2023-10-21T16:02:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f6d90 con 0x7fedb0146d10 2023-10-21T16:02:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7fedb6289640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f7070 con 0x7fedb0144020 2023-10-21T16:02:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7feda7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7feda7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54622/0 (socket says 172.21.15.53:54622) 2023-10-21T16:02:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.061+0000 7feda7fff640 1 -- 172.21.15.53:0/3155499421 learned_addr learned my addr 172.21.15.53:0/3155499421 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7fedaffff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7fedb4826640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fedb0146d10 0x7fedb01f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 -- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 msgr2=0x7fedb0162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 -- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fedb0146d10 msgr2=0x7fedb01f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fedb0146d10 0x7fedb01f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 -- 172.21.15.53:0/3155499421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980094d0 con 0x7fedb0146930 2023-10-21T16:02:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.062+0000 7feda7fff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fed9802b9c0 tx=0x7fed980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.063+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9803e070 con 0x7fedb0146930 2023-10-21T16:02:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.063+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed98039a10 con 0x7fedb0146930 2023-10-21T16:02:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.063+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb01f7350 con 0x7fedb0146930 2023-10-21T16:02:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.063+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98039de0 con 0x7fedb0146930 2023-10-21T16:02:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.063+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb0162c60 con 0x7fedb0146930 2023-10-21T16:02:12.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.064+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb004e160 con 0x7fedb0146930 2023-10-21T16:02:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.065+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed9804b400 con 0x7fedb0146930 2023-10-21T16:02:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.066+0000 7fedad7fa640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 0x7fed9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.067+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed980abf20 con 0x7fedb0146930 2023-10-21T16:02:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.067+0000 7fedaffff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 0x7fed9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.068+0000 7fedaffff640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 0x7fed9006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedb0163b00 tx=0x7feda0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.071+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed98076ce0 con 0x7fedb0146930 2023-10-21T16:02:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.263+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedb0144640 con 0x7fedb0146930 2023-10-21T16:02:12.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.264+0000 7fedad7fa640 1 -- 172.21.15.53:0/3155499421 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed980340a0 con 0x7fedb0146930 2023-10-21T16:02:12.267 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:12.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 msgr2=0x7fed9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 0x7fed9006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fedb0163b00 tx=0x7feda0009040 comp rx=0 tx=0).stop 2023-10-21T16:02:12.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 msgr2=0x7fedb00ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fed9802b9c0 tx=0x7fed980046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 shutdown_connections 2023-10-21T16:02:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fedb0146d10 0x7fedb01f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fed9006ae70 0x7fed9006d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fedb0146930 0x7fedb00ccd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 --2- 172.21.15.53:0/3155499421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fedb0144020 0x7fedb0162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 >> 172.21.15.53:0/3155499421 conn(0x7fedb0069d50 msgr2=0x7fedb00701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 shutdown_connections 2023-10-21T16:02:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.267+0000 7fedb6289640 1 -- 172.21.15.53:0/3155499421 wait complete. 2023-10-21T16:02:12.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:12.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.516+0000 7f3edf2a2640 1 Processor -- start 2023-10-21T16:02:12.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.517+0000 7f3edf2a2640 1 -- start start 2023-10-21T16:02:12.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.518+0000 7f3edf2a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ed80cb150 0x7f3ed80cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edf2a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed806e1e0 con 0x7f3ed80b1f20 2023-10-21T16:02:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edf2a2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed806cc50 con 0x7f3ed8161690 2023-10-21T16:02:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edf2a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed80cbb40 con 0x7f3ed80cb150 2023-10-21T16:02:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edc83d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ed80cb150 0x7f3ed80cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edd83f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3ed80b1f20 0x7f3ed80b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48648/0 (socket says 172.21.15.53:48648) 2023-10-21T16:02:12.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edd83f640 1 -- 172.21.15.53:0/1731357042 learned_addr learned my addr 172.21.15.53:0/1731357042 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:12.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.519+0000 7f3edc83d640 1 -- 172.21.15.53:0/1731357042 >> v1:172.21.15.124:6789/0 conn(0x7f3ed8161690 legacy=0x7f3ed806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.520+0000 7f3edc83d640 1 -- 172.21.15.53:0/1731357042 >> v1:172.21.15.53:6789/0 conn(0x7f3ed80b1f20 legacy=0x7f3ed80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.520+0000 7f3edc83d640 1 -- 172.21.15.53:0/1731357042 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed806cf30 con 0x7f3ed80cb150 2023-10-21T16:02:12.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.520+0000 7f3ecffff640 1 -- 172.21.15.53:0/1731357042 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856827541 0 0) 0x7f3ed806e1e0 con 0x7f3ed80b1f20 2023-10-21T16:02:12.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.521+0000 7f3edc83d640 1 --2- 172.21.15.53:0/1731357042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cb600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80097a0 tx=0x7f3ec802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9aa8f2bf2aa437 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.521+0000 7f3ecffff640 1 -- 172.21.15.53:0/1731357042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8030610 con 0x7f3ed80cb150 2023-10-21T16:02:12.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.521+0000 7f3ecffff640 1 -- 172.21.15.53:0/1731357042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec8030770 con 0x7f3ed80cb150 2023-10-21T16:02:12.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 -- 172.21.15.53:0/1731357042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 msgr2=0x7f3ed80cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/1731357042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cb600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80097a0 tx=0x7f3ec802b960 comp rx=0 tx=0).stop 2023-10-21T16:02:12.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3ecffff640 1 -- 172.21.15.53:0/1731357042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8030b00 con 0x7f3ed80cb150 2023-10-21T16:02:12.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 -- 172.21.15.53:0/1731357042 shutdown_connections 2023-10-21T16:02:12.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/1731357042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cb600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 -- 172.21.15.53:0/1731357042 >> 172.21.15.53:0/1731357042 conn(0x7f3ed806a5d0 msgr2=0x7f3ed80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:12.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 -- 172.21.15.53:0/1731357042 shutdown_connections 2023-10-21T16:02:12.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.522+0000 7f3edf2a2640 1 -- 172.21.15.53:0/1731357042 wait complete. 2023-10-21T16:02:12.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.524+0000 7f3edf2a2640 1 Processor -- start 2023-10-21T16:02:12.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.524+0000 7f3edf2a2640 1 -- start start 2023-10-21T16:02:12.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ed80b1f20 0x7f3ed80c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ed8161690 0x7f3ed80c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80c5330 con 0x7f3ed80cb150 2023-10-21T16:02:12.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80c4d70 con 0x7f3ed80b1f20 2023-10-21T16:02:12.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edf2a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80c5050 con 0x7f3ed8161690 2023-10-21T16:02:12.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edc83d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edc83d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54636/0 (socket says 172.21.15.53:54636) 2023-10-21T16:02:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.525+0000 7f3edc83d640 1 -- 172.21.15.53:0/570827838 learned_addr learned my addr 172.21.15.53:0/570827838 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 -- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ed8161690 msgr2=0x7f3ed80c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ed8161690 0x7f3ed80c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 -- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ed80b1f20 msgr2=0x7f3ed80c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ed80b1f20 0x7f3ed80c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 -- 172.21.15.53:0/570827838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec8009400 con 0x7f3ed80cb150 2023-10-21T16:02:12.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edc83d640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8030610 con 0x7f3ed80cb150 2023-10-21T16:02:12.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec803f040 con 0x7f3ed80cb150 2023-10-21T16:02:12.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8030b00 con 0x7f3ed80cb150 2023-10-21T16:02:12.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed80cd590 con 0x7f3ed80cb150 2023-10-21T16:02:12.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.526+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed80c4a20 con 0x7f3ed80cb150 2023-10-21T16:02:12.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.528+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed80cdab0 con 0x7f3ed80cb150 2023-10-21T16:02:12.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.529+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ec8004030 con 0x7f3ed80cb150 2023-10-21T16:02:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.529+0000 7f3ecdffb640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 0x7f3ebc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.530+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ec80ab660 con 0x7f3ed80cb150 2023-10-21T16:02:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.530+0000 7f3edd03e640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 0x7f3ebc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.531+0000 7f3edd03e640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 0x7f3ebc06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4000f60 tx=0x7f3ed40079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.534+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ed80cdab0 con 0x7f3ed80cb150 2023-10-21T16:02:12.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.727+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ed80cdab0 con 0x7f3ed80cb150 2023-10-21T16:02:12.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.728+0000 7f3ecdffb640 1 -- 172.21.15.53:0/570827838 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ed80cdab0 con 0x7f3ed80cb150 2023-10-21T16:02:12.731 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:12.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 msgr2=0x7f3ebc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 0x7f3ebc06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4000f60 tx=0x7f3ed40079b0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 msgr2=0x7f3ed80cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8000c00 tx=0x7f3ec80046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 shutdown_connections 2023-10-21T16:02:12.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ed8161690 0x7f3ed80c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ebc06ae70 0x7f3ebc06d330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ed80cb150 0x7f3ed80cdd90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 --2- 172.21.15.53:0/570827838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ed80b1f20 0x7f3ed80c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 >> 172.21.15.53:0/570827838 conn(0x7f3ed806a5d0 msgr2=0x7f3ed80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 shutdown_connections 2023-10-21T16:02:12.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.731+0000 7f3edf2a2640 1 -- 172.21.15.53:0/570827838 wait complete. 2023-10-21T16:02:12.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:12.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.972+0000 7face6dff640 1 Processor -- start 2023-10-21T16:02:12.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.973+0000 7face6dff640 1 -- start start 2023-10-21T16:02:12.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.974+0000 7face6dff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7face0071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.974+0000 7face6dff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face0160d10 con 0x7face0069fc0 2023-10-21T16:02:12.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face6dff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face0072070 con 0x7face006ec00 2023-10-21T16:02:12.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face6dff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face0071580 con 0x7face0160790 2023-10-21T16:02:12.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face5dfd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7face0069fc0 0x7face00c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48656/0 (socket says 172.21.15.53:48656) 2023-10-21T16:02:12.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face5dfd640 1 -- 172.21.15.53:0/2101492106 learned_addr learned my addr 172.21.15.53:0/2101492106 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:12.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930035455 0 0) 0x7face0160d10 con 0x7face0069fc0 2023-10-21T16:02:12.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc003520 con 0x7face0069fc0 2023-10-21T16:02:12.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.975+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7face0071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3632888208 0 0) 0x7faccc003520 con 0x7face0069fc0 2023-10-21T16:02:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 msgr2=0x7face0071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7face0071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 >> v1:172.21.15.124:6789/0 conn(0x7face006ec00 legacy=0x7face0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0072330 con 0x7face0069fc0 2023-10-21T16:02:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7facdc001670 con 0x7face0069fc0 2023-10-21T16:02:12.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7face0071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:12.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 >> v1:172.21.15.53:6789/0 conn(0x7face0069fc0 legacy=0x7face00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faccc0039a0 0x7faccc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7faccc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faccc004d00 0x7faccc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face0071580 con 0x7faccc004d00 2023-10-21T16:02:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00bb10 con 0x7faccc0039a0 2023-10-21T16:02:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00bcd0 con 0x7face0160790 2023-10-21T16:02:12.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faccc004d00 0x7faccc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face5dfd640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faccc0039a0 0x7faccc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.976+0000 7face55fc640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7faccc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face65fe640 1 -- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 msgr2=0x7faccc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7face0160790 0x7faccc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face65fe640 1 -- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faccc0039a0 msgr2=0x7faccc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:12.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faccc0039a0 0x7faccc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face65fe640 1 -- 172.21.15.53:0/2101492106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc003520 con 0x7faccc004d00 2023-10-21T16:02:12.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.977+0000 7face5dfd640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faccc0039a0 0x7faccc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:12.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.978+0000 7face65fe640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faccc004d00 0x7faccc005180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7face0160be0 tx=0x7faccc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cddcbbcb760b606 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.978+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd802ccd0 con 0x7faccc004d00 2023-10-21T16:02:12.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.978+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd802ce30 con 0x7faccc004d00 2023-10-21T16:02:12.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.978+0000 7face4dfb640 1 -- 172.21.15.53:0/2101492106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8034640 con 0x7faccc004d00 2023-10-21T16:02:12.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 -- 172.21.15.53:0/2101492106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faccc004d00 msgr2=0x7faccc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 --2- 172.21.15.53:0/2101492106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faccc004d00 0x7faccc005180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7face0160be0 tx=0x7faccc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face65fe640 1 -- 172.21.15.53:0/2101492106 reap_dead start 2023-10-21T16:02:12.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 -- 172.21.15.53:0/2101492106 shutdown_connections 2023-10-21T16:02:12.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 -- 172.21.15.53:0/2101492106 >> 172.21.15.53:0/2101492106 conn(0x7face00cf5f0 msgr2=0x7face00cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:12.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 -- 172.21.15.53:0/2101492106 shutdown_connections 2023-10-21T16:02:12.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.979+0000 7face6dff640 1 -- 172.21.15.53:0/2101492106 wait complete. 2023-10-21T16:02:12.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.981+0000 7face6dff640 1 Processor -- start 2023-10-21T16:02:12.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.981+0000 7face6dff640 1 -- start start 2023-10-21T16:02:12.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faccc004d00 0x7face014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face01e68c0 0x7face01e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face00c9260 con 0x7face00b43d0 2023-10-21T16:02:12.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face01e7260 con 0x7face01e68c0 2023-10-21T16:02:12.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.982+0000 7face6dff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00b3f40 con 0x7faccc004d00 2023-10-21T16:02:12.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54664/0 (socket says 172.21.15.53:54664) 2023-10-21T16:02:12.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 -- 172.21.15.53:0/1240457326 learned_addr learned my addr 172.21.15.53:0/1240457326 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:12.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face55fc640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face01e68c0 0x7face01e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face5dfd640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faccc004d00 0x7face014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 -- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faccc004d00 msgr2=0x7face014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faccc004d00 0x7face014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 -- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face01e68c0 msgr2=0x7face01e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:12.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face01e68c0 0x7face01e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:12.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 -- 172.21.15.53:0/1240457326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00b3610 con 0x7face00b43d0 2023-10-21T16:02:12.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.983+0000 7face65fe640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facd802c740 tx=0x7facd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.984+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8045070 con 0x7face00b43d0 2023-10-21T16:02:12.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.984+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8003990 con 0x7face00b43d0 2023-10-21T16:02:12.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.984+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8003d00 con 0x7face00b43d0 2023-10-21T16:02:12.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.984+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face00b38f0 con 0x7face00b43d0 2023-10-21T16:02:12.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.984+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face014f140 con 0x7face00b43d0 2023-10-21T16:02:12.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.985+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0074150 con 0x7face00b43d0 2023-10-21T16:02:12.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.986+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7facd803b040 con 0x7face00b43d0 2023-10-21T16:02:12.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.987+0000 7facd67fc640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 0x7facc406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:12.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.987+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7facd80aaf20 con 0x7face00b43d0 2023-10-21T16:02:12.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.988+0000 7face5dfd640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 0x7facc406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:12.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.989+0000 7face5dfd640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 0x7facc406d380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facdc001c40 tx=0x7facdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:12.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:12.991+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facd8075dc0 con 0x7face00b43d0 2023-10-21T16:02:13.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.187+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7face00cec70 con 0x7face00b43d0 2023-10-21T16:02:13.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.188+0000 7facd67fc640 1 -- 172.21.15.53:0/1240457326 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7facd80793e0 con 0x7face00b43d0 2023-10-21T16:02:13.191 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:13.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 msgr2=0x7facc406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 0x7facc406d380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facdc001c40 tx=0x7facdc009040 comp rx=0 tx=0).stop 2023-10-21T16:02:13.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 msgr2=0x7face00b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7facd802c740 tx=0x7facd8004ab0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 shutdown_connections 2023-10-21T16:02:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7facc406aec0 0x7facc406d380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7face00b43d0 0x7face00b4850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7face01e68c0 0x7face01e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 --2- 172.21.15.53:0/1240457326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faccc004d00 0x7face014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 >> 172.21.15.53:0/1240457326 conn(0x7face00cf5f0 msgr2=0x7face00ac790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 shutdown_connections 2023-10-21T16:02:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.191+0000 7face6dff640 1 -- 172.21.15.53:0/1240457326 wait complete. 2023-10-21T16:02:13.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:13.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.436+0000 7fb6cea75640 1 Processor -- start 2023-10-21T16:02:13.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.436+0000 7fb6cea75640 1 -- start start 2023-10-21T16:02:13.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb6c80cf690 0x7fb6c80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 0x7fb6c815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c80cd520 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c80d1910 con 0x7fb6c80cf690 2023-10-21T16:02:13.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cea75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c80ce080 con 0x7fb6c80d1100 2023-10-21T16:02:13.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54676/0 (socket says 172.21.15.53:54676) 2023-10-21T16:02:13.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 -- 172.21.15.53:0/2798619738 learned_addr learned my addr 172.21.15.53:0/2798619738 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:13.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cd272640 1 --2- 172.21.15.53:0/2798619738 >> v2:172.21.15.124:3300/0 conn(0x7fb6c80cf690 0x7fb6c80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6ce274640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 0x7fb6c815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 -- 172.21.15.53:0/2798619738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 msgr2=0x7fb6c815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:13.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 0x7fb6c815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 -- 172.21.15.53:0/2798619738 >> v2:172.21.15.124:3300/0 conn(0x7fb6c80cf690 msgr2=0x7fb6c80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:13.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 --2- 172.21.15.53:0/2798619738 >> v2:172.21.15.124:3300/0 conn(0x7fb6c80cf690 0x7fb6c80ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6cda73640 1 -- 172.21.15.53:0/2798619738 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80cd750 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.438+0000 7fb6ce274640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 0x7fb6c815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:13.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.439+0000 7fb6cda73640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6c808baa0 tx=0x7fb6c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9efec9fe5740859 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cca71640 1 -- 172.21.15.53:0/2798619738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40317e0 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cca71640 1 -- 172.21.15.53:0/2798619738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c4031940 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cca71640 1 -- 172.21.15.53:0/2798619738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4031cd0 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 -- 172.21.15.53:0/2798619738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80cf2b0 msgr2=0x7fb6c80d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6c808baa0 tx=0x7fb6c402b880 comp rx=0 tx=0).stop 2023-10-21T16:02:13.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 -- 172.21.15.53:0/2798619738 shutdown_connections 2023-10-21T16:02:13.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80d1100 0x7fb6c815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 --2- 172.21.15.53:0/2798619738 >> v2:172.21.15.124:3300/0 conn(0x7fb6c80cf690 0x7fb6c80ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 --2- 172.21.15.53:0/2798619738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80cf2b0 0x7fb6c80d34d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 -- 172.21.15.53:0/2798619738 >> 172.21.15.53:0/2798619738 conn(0x7fb6c809bc40 msgr2=0x7fb6c80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:13.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 -- 172.21.15.53:0/2798619738 shutdown_connections 2023-10-21T16:02:13.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.440+0000 7fb6cea75640 1 -- 172.21.15.53:0/2798619738 wait complete. 2023-10-21T16:02:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.442+0000 7fb6cea75640 1 Processor -- start 2023-10-21T16:02:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.442+0000 7fb6cea75640 1 -- start start 2023-10-21T16:02:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80cf2b0 0x7fb6c814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c80cf690 0x7fb6c815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c809d530 con 0x7fb6c80d1100 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c809d810 con 0x7fb6c80cf690 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6cea75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c809daf0 con 0x7fb6c80cf2b0 2023-10-21T16:02:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6ce274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6ce274640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54692/0 (socket says 172.21.15.53:54692) 2023-10-21T16:02:13.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.443+0000 7fb6ce274640 1 -- 172.21.15.53:0/690757092 learned_addr learned my addr 172.21.15.53:0/690757092 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:13.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6cd272640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c80cf690 0x7fb6c815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6cda73640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80cf2b0 0x7fb6c814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 -- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80cf2b0 msgr2=0x7fb6c814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80cf2b0 0x7fb6c814cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 -- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c80cf690 msgr2=0x7fb6c815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c80cf690 0x7fb6c815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 -- 172.21.15.53:0/690757092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40094d0 con 0x7fb6c80d1100 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6ce274640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000cd50 tx=0x7fb6c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000f680 con 0x7fb6c80d1100 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c000f800 con 0x7fb6c80d1100 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000fbb0 con 0x7fb6c80d1100 2023-10-21T16:02:13.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.444+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c809ddd0 con 0x7fb6c80d1100 2023-10-21T16:02:13.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.445+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c814d160 con 0x7fb6c80d1100 2023-10-21T16:02:13.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.446+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c815c8e0 con 0x7fb6c80d1100 2023-10-21T16:02:13.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.446+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb6c000b040 con 0x7fb6c80d1100 2023-10-21T16:02:13.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.447+0000 7fb6be7fc640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 0x7fb6ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.447+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb6c008a2a0 con 0x7fb6c80d1100 2023-10-21T16:02:13.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.448+0000 7fb6cda73640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 0x7fb6ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.449+0000 7fb6cda73640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 0x7fb6ac06d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4036000 tx=0x7fb6c403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.453+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6c0055050 con 0x7fb6c80d1100 2023-10-21T16:02:13.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.643+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb6c80cda00 con 0x7fb6c80d1100 2023-10-21T16:02:13.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.644+0000 7fb6be7fc640 1 -- 172.21.15.53:0/690757092 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb6c0058670 con 0x7fb6c80d1100 2023-10-21T16:02:13.647 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 msgr2=0x7fb6ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 0x7fb6ac06d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4036000 tx=0x7fb6c403d040 comp rx=0 tx=0).stop 2023-10-21T16:02:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 msgr2=0x7fb6c809e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000cd50 tx=0x7fb6c000a650 comp rx=0 tx=0).stop 2023-10-21T16:02:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 shutdown_connections 2023-10-21T16:02:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb6ac06ae70 0x7fb6ac06d330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6c80d1100 0x7fb6c809e280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c80cf690 0x7fb6c815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 --2- 172.21.15.53:0/690757092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c80cf2b0 0x7fb6c814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 >> 172.21.15.53:0/690757092 conn(0x7fb6c809bc40 msgr2=0x7fb6c80d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.646+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 shutdown_connections 2023-10-21T16:02:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.647+0000 7fb6cea75640 1 -- 172.21.15.53:0/690757092 wait complete. 2023-10-21T16:02:13.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:02:13.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.887+0000 7f2cae621640 1 Processor -- start 2023-10-21T16:02:13.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.887+0000 7f2cae621640 1 -- start start 2023-10-21T16:02:13.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.888+0000 7f2cae621640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2ca80cf690 0x7f2ca80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.888+0000 7f2cae621640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 0x7f2ca815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.888+0000 7f2cae621640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca80cd520 con 0x7f2ca80cf2b0 2023-10-21T16:02:13.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.888+0000 7f2cae621640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca80d1910 con 0x7f2ca80cf690 2023-10-21T16:02:13.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.888+0000 7f2cae621640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80ce080 con 0x7f2ca80ce510 2023-10-21T16:02:13.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cace1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2ca80cf690 0x7f2ca80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cad61f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2ca80cf2b0 0x7f2ca80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48666/0 (socket says 172.21.15.53:48666) 2023-10-21T16:02:13.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cad61f640 1 -- 172.21.15.53:0/1164839168 learned_addr learned my addr 172.21.15.53:0/1164839168 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:13.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cade20640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 0x7f2ca815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cace1e640 1 -- 172.21.15.53:0/1164839168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 msgr2=0x7f2ca815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:13.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cace1e640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 0x7f2ca815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cace1e640 1 -- 172.21.15.53:0/1164839168 >> v1:172.21.15.53:6789/0 conn(0x7f2ca80cf2b0 legacy=0x7f2ca80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cace1e640 1 -- 172.21.15.53:0/1164839168 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80cd750 con 0x7f2ca80cf690 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2cade20640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 0x7f2ca815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.889+0000 7f2ca7fff640 1 -- 172.21.15.53:0/1164839168 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906761673 0 0) 0x7f2ca80cd520 con 0x7f2ca80cf2b0 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.890+0000 7f2cace1e640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80cf690 0x7f2ca80d34d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0097a0 tx=0x7f2c9c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00c477eac262e6e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.890+0000 7f2ca7fff640 1 -- 172.21.15.53:0/1164839168 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c031780 con 0x7f2ca80cf690 2023-10-21T16:02:13.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.890+0000 7f2ca7fff640 1 -- 172.21.15.53:0/1164839168 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0318e0 con 0x7f2ca80cf690 2023-10-21T16:02:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.890+0000 7f2ca7fff640 1 -- 172.21.15.53:0/1164839168 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c031c70 con 0x7f2ca80cf690 2023-10-21T16:02:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 -- 172.21.15.53:0/1164839168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80cf690 msgr2=0x7f2ca80d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80cf690 0x7f2ca80d34d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0097a0 tx=0x7f2c9c02b820 comp rx=0 tx=0).stop 2023-10-21T16:02:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 -- 172.21.15.53:0/1164839168 shutdown_connections 2023-10-21T16:02:13.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80ce510 0x7f2ca815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 --2- 172.21.15.53:0/1164839168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80cf690 0x7f2ca80d34d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 -- 172.21.15.53:0/1164839168 >> 172.21.15.53:0/1164839168 conn(0x7f2ca809bc40 msgr2=0x7f2ca80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 -- 172.21.15.53:0/1164839168 shutdown_connections 2023-10-21T16:02:13.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.891+0000 7f2cae621640 1 -- 172.21.15.53:0/1164839168 wait complete. 2023-10-21T16:02:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.893+0000 7f2cae621640 1 Processor -- start 2023-10-21T16:02:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.893+0000 7f2cae621640 1 -- start start 2023-10-21T16:02:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 0x7f2ca809cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80cf690 0x7f2ca80b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca815c460 con 0x7f2ca80ce510 2023-10-21T16:02:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca809d500 con 0x7f2ca80cf2b0 2023-10-21T16:02:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cae621640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81f6a60 con 0x7f2ca80cf690 2023-10-21T16:02:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cad61f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cad61f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54720/0 (socket says 172.21.15.53:54720) 2023-10-21T16:02:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.894+0000 7f2cad61f640 1 -- 172.21.15.53:0/2809772997 learned_addr learned my addr 172.21.15.53:0/2809772997 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cace1e640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 0x7f2ca809cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 -- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80cf690 msgr2=0x7f2ca80b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cade20640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80cf690 0x7f2ca80b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80cf690 0x7f2ca80b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 -- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 msgr2=0x7f2ca809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 0x7f2ca809cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 -- 172.21.15.53:0/2809772997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c009400 con 0x7f2ca80ce510 2023-10-21T16:02:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cace1e640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 0x7f2ca809cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:13.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2cad61f640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c4d0 tx=0x7f2c9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980074e0 con 0x7f2ca80ce510 2023-10-21T16:02:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98007660 con 0x7f2ca80ce510 2023-10-21T16:02:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.895+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004a20 con 0x7f2ca80ce510 2023-10-21T16:02:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.896+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca81f6d40 con 0x7f2ca80ce510 2023-10-21T16:02:13.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.896+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81f7300 con 0x7f2ca80ce510 2023-10-21T16:02:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.897+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8074150 con 0x7f2ca80ce510 2023-10-21T16:02:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.899+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c98004b80 con 0x7f2ca80ce510 2023-10-21T16:02:13.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.899+0000 7f2ca5ffb640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 0x7f2c9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:13.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.899+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c98088b90 con 0x7f2ca80ce510 2023-10-21T16:02:13.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.899+0000 7f2cace1e640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 0x7f2c9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:13.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.900+0000 7f2cace1e640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 0x7f2c9406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c036000 tx=0x7f2c9c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:13.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:13.904+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c980538b0 con 0x7f2ca80ce510 2023-10-21T16:02:14.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.104+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ca80d0070 con 0x7f2ca80ce510 2023-10-21T16:02:14.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.105+0000 7f2ca5ffb640 1 -- 172.21.15.53:0/2809772997 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2c98018020 con 0x7f2ca80ce510 2023-10-21T16:02:14.108 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:02:14.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.107+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 msgr2=0x7f2c9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:14.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.107+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 0x7f2c9406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c036000 tx=0x7f2c9c03d040 comp rx=0 tx=0).stop 2023-10-21T16:02:14.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 msgr2=0x7f2ca815bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:14.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c4d0 tx=0x7f2c9800c990 comp rx=0 tx=0).stop 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 shutdown_connections 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ca80cf690 0x7f2ca80b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ca80cf2b0 0x7f2ca809cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2c9406ae70 0x7f2c9406d330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 --2- 172.21.15.53:0/2809772997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ca80ce510 0x7f2ca815bf20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 >> 172.21.15.53:0/2809772997 conn(0x7f2ca809bc40 msgr2=0x7f2ca815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:14.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 shutdown_connections 2023-10-21T16:02:14.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:14.109+0000 7f2cae621640 1 -- 172.21.15.53:0/2809772997 wait complete. 2023-10-21T16:02:14.128 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi053.front.sepia.ceph.com... 2023-10-21T16:02:14.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:14.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-21T16:02:14.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi053.stderr:ceph-fuse[92795]: fuse finished with error 0 and tester_r 0 2023-10-21T16:02:14.234 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:02:16.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:02:16.103 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:02:20.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:02:20.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:20.337 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-21T16:02:20.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:20.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.644+0000 7f90b5034640 1 Processor -- start 2023-10-21T16:02:20.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.644+0000 7f90b5034640 1 -- start start 2023-10-21T16:02:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90b5034640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90b5034640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90b00c90b0 0x7f90b00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90b5034640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b015b2b0 con 0x7f90b00cae70 2023-10-21T16:02:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90b5034640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b015bf30 con 0x7f90b00c90b0 2023-10-21T16:02:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90b5034640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b015f7a0 con 0x7f90b015ab40 2023-10-21T16:02:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90a7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90b00c90b0 0x7f90b00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90a7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90b00c90b0 0x7f90b00adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:36834/0 (socket says 172.21.15.53:36834) 2023-10-21T16:02:20.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90a7fff640 1 -- 172.21.15.53:0/1749754237 learned_addr learned my addr 172.21.15.53:0/1749754237 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:20.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.646+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:52156/0 (socket says 172.21.15.53:52156) 2023-10-21T16:02:20.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.647+0000 7f90a7fff640 1 -- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 msgr2=0x7f90b00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:20.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.647+0000 7f90a7fff640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.647+0000 7f90a7fff640 1 -- 172.21.15.53:0/1749754237 >> v1:172.21.15.124:6789/0 conn(0x7f90b00cae70 legacy=0x7f90b00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.647+0000 7f90a7fff640 1 -- 172.21.15.53:0/1749754237 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b015b590 con 0x7f90b00c90b0 2023-10-21T16:02:20.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.647+0000 7f90aed9d640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.649+0000 7f90a7fff640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9098009870 tx=0x7f909802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f634dde43b4269 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.649+0000 7f90ae59c640 1 -- 172.21.15.53:0/1749754237 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098030840 con 0x7f90b00c90b0 2023-10-21T16:02:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.649+0000 7f90ae59c640 1 -- 172.21.15.53:0/1749754237 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90980309a0 con 0x7f90b00c90b0 2023-10-21T16:02:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.649+0000 7f90ae59c640 1 -- 172.21.15.53:0/1749754237 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098030d30 con 0x7f90b00c90b0 2023-10-21T16:02:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 -- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00c90b0 msgr2=0x7f90b00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9098009870 tx=0x7f909802b980 comp rx=0 tx=0).stop 2023-10-21T16:02:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 -- 172.21.15.53:0/1749754237 shutdown_connections 2023-10-21T16:02:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 --2- 172.21.15.53:0/1749754237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b00ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 -- 172.21.15.53:0/1749754237 >> 172.21.15.53:0/1749754237 conn(0x7f90b006a380 msgr2=0x7f90b00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 -- 172.21.15.53:0/1749754237 shutdown_connections 2023-10-21T16:02:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.650+0000 7f90b5034640 1 -- 172.21.15.53:0/1749754237 wait complete. 2023-10-21T16:02:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.653+0000 7f90b5034640 1 Processor -- start 2023-10-21T16:02:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.653+0000 7f90b5034640 1 -- start start 2023-10-21T16:02:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00cae70 0x7f90b0147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b01403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0149250 con 0x7f90b00c90b0 2023-10-21T16:02:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01485d0 con 0x7f90b00cae70 2023-10-21T16:02:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90b5034640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01488b0 con 0x7f90b015ab40 2023-10-21T16:02:20.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.654+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41534/0 (socket says 172.21.15.53:41534) 2023-10-21T16:02:20.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 -- 172.21.15.53:0/3696404064 learned_addr learned my addr 172.21.15.53:0/3696404064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:20.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90a7fff640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00cae70 0x7f90b0147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90af59e640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b01403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 -- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 msgr2=0x7f90b01403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b01403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 -- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00cae70 msgr2=0x7f90b0147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00cae70 0x7f90b0147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 -- 172.21.15.53:0/3696404064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009400 con 0x7f90b00c90b0 2023-10-21T16:02:20.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.655+0000 7f90aed9d640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ebe0 tx=0x7f90a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:20.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.656+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0009760 con 0x7f90b00c90b0 2023-10-21T16:02:20.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.656+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000cc30 con 0x7f90b00c90b0 2023-10-21T16:02:20.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.656+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000eee0 con 0x7f90b00c90b0 2023-10-21T16:02:20.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.656+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90980094d0 con 0x7f90b00c90b0 2023-10-21T16:02:20.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.656+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b0148e70 con 0x7f90b00c90b0 2023-10-21T16:02:20.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.657+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b01494c0 con 0x7f90b00c90b0 2023-10-21T16:02:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.658+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f90a0013470 con 0x7f90b00c90b0 2023-10-21T16:02:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.659+0000 7f90a77fe640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 0x7f909006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.659+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f90a0089bb0 con 0x7f90b00c90b0 2023-10-21T16:02:20.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.660+0000 7f90a7fff640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 0x7f909006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:20.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.661+0000 7f90a7fff640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 0x7f909006d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9098036000 tx=0x7f9098008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:20.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.664+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90a0054a10 con 0x7f90b00c90b0 2023-10-21T16:02:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.856+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90b00cc0a0 con 0x7f90b00c90b0 2023-10-21T16:02:20.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.859+0000 7f90a77fe640 1 -- 172.21.15.53:0/3696404064 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f90a0017090 con 0x7f90b00c90b0 2023-10-21T16:02:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:20.861 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 msgr2=0x7f909006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 0x7f909006d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9098036000 tx=0x7f9098008030 comp rx=0 tx=0).stop 2023-10-21T16:02:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 msgr2=0x7f90b00adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f90a000ebe0 tx=0x7f90a000c520 comp rx=0 tx=0).stop 2023-10-21T16:02:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 shutdown_connections 2023-10-21T16:02:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90b015ab40 0x7f90b01403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90b00cae70 0x7f90b0147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f909006aea0 0x7f909006d360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 --2- 172.21.15.53:0/3696404064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90b00c90b0 0x7f90b00adc80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.861+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 >> 172.21.15.53:0/3696404064 conn(0x7f90b006a380 msgr2=0x7f90b00ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.862+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 shutdown_connections 2023-10-21T16:02:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:20.862+0000 7f90b5034640 1 -- 172.21.15.53:0/3696404064 wait complete. 2023-10-21T16:02:20.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.106+0000 7f37c2fc9640 1 Processor -- start 2023-10-21T16:02:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.107+0000 7f37c2fc9640 1 -- start start 2023-10-21T16:02:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.108+0000 7f37c2fc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37bc15ab40 0x7f37bc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.108+0000 7f37c2fc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc0c90b0 0x7f37bc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.108+0000 7f37c2fc9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc15b2b0 con 0x7f37bc0cae70 2023-10-21T16:02:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.108+0000 7f37c2fc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc15bf30 con 0x7f37bc15ab40 2023-10-21T16:02:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.108+0000 7f37c2fc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc15f7a0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37bc15ab40 0x7f37bc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37bbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc0c90b0 0x7f37bc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37bc15ab40 0x7f37bc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:36844/0 (socket says 172.21.15.53:36844) 2023-10-21T16:02:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 -- 172.21.15.53:0/3793050038 learned_addr learned my addr 172.21.15.53:0/3793050038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 -- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc0c90b0 msgr2=0x7f37bc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 --2- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc0c90b0 0x7f37bc0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 -- 172.21.15.53:0/3793050038 >> v1:172.21.15.124:6789/0 conn(0x7f37bc0cae70 legacy=0x7f37bc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.109+0000 7f37c0d65640 1 -- 172.21.15.53:0/3793050038 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc15b590 con 0x7f37bc15ab40 2023-10-21T16:02:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.110+0000 7f37c0d65640 1 --2- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc15ab40 0x7f37bc0ad720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009870 tx=0x7f37ac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88fc05768fa78fcf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.111+0000 7f37bb7fe640 1 -- 172.21.15.53:0/3793050038 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac030840 con 0x7f37bc15ab40 2023-10-21T16:02:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.111+0000 7f37bb7fe640 1 -- 172.21.15.53:0/3793050038 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac0309a0 con 0x7f37bc15ab40 2023-10-21T16:02:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.111+0000 7f37bb7fe640 1 -- 172.21.15.53:0/3793050038 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac030d30 con 0x7f37bc15ab40 2023-10-21T16:02:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 -- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc15ab40 msgr2=0x7f37bc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc15ab40 0x7f37bc0ad720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009870 tx=0x7f37ac02b980 comp rx=0 tx=0).stop 2023-10-21T16:02:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 -- 172.21.15.53:0/3793050038 shutdown_connections 2023-10-21T16:02:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc0c90b0 0x7f37bc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/3793050038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc15ab40 0x7f37bc0ad720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 -- 172.21.15.53:0/3793050038 >> 172.21.15.53:0/3793050038 conn(0x7f37bc06a380 msgr2=0x7f37bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 -- 172.21.15.53:0/3793050038 shutdown_connections 2023-10-21T16:02:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.112+0000 7f37c2fc9640 1 -- 172.21.15.53:0/3793050038 wait complete. 2023-10-21T16:02:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.114+0000 7f37c2fc9640 1 Processor -- start 2023-10-21T16:02:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.114+0000 7f37c2fc9640 1 -- start start 2023-10-21T16:02:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 0x7f37bc155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 0x7f37bc1efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0c7080 con 0x7f37bc0c90b0 2023-10-21T16:02:21.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0c6d50 con 0x7f37bc0cae70 2023-10-21T16:02:21.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c2fc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc1f43f0 con 0x7f37bc15ab40 2023-10-21T16:02:21.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41540/0 (socket says 172.21.15.53:41540) 2023-10-21T16:02:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 -- 172.21.15.53:0/1713814498 learned_addr learned my addr 172.21.15.53:0/1713814498 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37bbfff640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 0x7f37bc155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 -- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 msgr2=0x7f37bc1efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c1566640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 0x7f37bc1efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 0x7f37bc1efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 -- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 msgr2=0x7f37bc155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 0x7f37bc155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37c0d65640 1 -- 172.21.15.53:0/1713814498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0094d0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.115+0000 7f37bbfff640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 0x7f37bc155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:21.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37c1566640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 0x7f37bc1efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37c0d65640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37ac000c00 tx=0x7f37ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac039c90 con 0x7f37bc0c90b0 2023-10-21T16:02:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac039df0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc1f46d0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac043640 con 0x7f37bc0c90b0 2023-10-21T16:02:21.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.116+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1f40e0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.117+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc067490 con 0x7f37bc0c90b0 2023-10-21T16:02:21.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.118+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37ac0437a0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.119+0000 7f37b97fa640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 0x7f37a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.119+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f37ac03e070 con 0x7f37bc0c90b0 2023-10-21T16:02:21.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.120+0000 7f37bbfff640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 0x7f37a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.121+0000 7f37bbfff640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 0x7f37a406d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37bc150e70 tx=0x7f37a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.124+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37ac0776f0 con 0x7f37bc0c90b0 2023-10-21T16:02:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.318+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37bc0b4780 con 0x7f37bc0c90b0 2023-10-21T16:02:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.322+0000 7f37b97fa640 1 -- 172.21.15.53:0/1713814498 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f37ac07ad10 con 0x7f37bc0c90b0 2023-10-21T16:02:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:21.325 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.324+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 msgr2=0x7f37a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.324+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 0x7f37a406d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f37bc150e70 tx=0x7f37a8008040 comp rx=0 tx=0).stop 2023-10-21T16:02:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.324+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 msgr2=0x7f37bc1f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.324+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37ac000c00 tx=0x7f37ac004730 comp rx=0 tx=0).stop 2023-10-21T16:02:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.324+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 shutdown_connections 2023-10-21T16:02:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37bc15ab40 0x7f37bc1efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37bc0cae70 0x7f37bc155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f37a406ae70 0x7f37a406d330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 --2- 172.21.15.53:0/1713814498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37bc0c90b0 0x7f37bc1f3ba0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 >> 172.21.15.53:0/1713814498 conn(0x7f37bc06a380 msgr2=0x7f37bc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 shutdown_connections 2023-10-21T16:02:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.325+0000 7f37c2fc9640 1 -- 172.21.15.53:0/1713814498 wait complete. 2023-10-21T16:02:21.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:02:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.563+0000 7f50ab0f2640 1 Processor -- start 2023-10-21T16:02:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.563+0000 7f50ab0f2640 1 -- start start 2023-10-21T16:02:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.564+0000 7f50ab0f2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50a406c3a0 0x7f50a40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.564+0000 7f50ab0f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f50a415aa00 0x7f50a40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.564+0000 7f50ab0f2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a415b310 con 0x7f50a406db60 2023-10-21T16:02:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.564+0000 7f50ab0f2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a40ca230 con 0x7f50a415aa00 2023-10-21T16:02:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.564+0000 7f50ab0f2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a406d290 con 0x7f50a406c3a0 2023-10-21T16:02:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a968f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f50a415aa00 0x7f50a40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f50a406db60 0x7f50a406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55022/0 (socket says 172.21.15.53:55022) 2023-10-21T16:02:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a3fff640 1 -- 172.21.15.53:0/2396974816 learned_addr learned my addr 172.21.15.53:0/2396974816 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a8e8e640 1 --2- 172.21.15.53:0/2396974816 >> v2:172.21.15.124:3300/0 conn(0x7f50a406c3a0 0x7f50a40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a37fe640 1 -- 172.21.15.53:0/2396974816 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797906847 0 0) 0x7f5090001110 con 0x7f50a406db60 2023-10-21T16:02:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a37fe640 1 -- 172.21.15.53:0/2396974816 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5094005730 con 0x7f50a406db60 2023-10-21T16:02:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a968f640 1 -- 172.21.15.53:0/2396974816 >> v2:172.21.15.124:3300/0 conn(0x7f50a406c3a0 msgr2=0x7f50a40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a968f640 1 --2- 172.21.15.53:0/2396974816 >> v2:172.21.15.124:3300/0 conn(0x7f50a406c3a0 0x7f50a40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a968f640 1 -- 172.21.15.53:0/2396974816 >> v1:172.21.15.53:6789/0 conn(0x7f50a406db60 legacy=0x7f50a406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.565+0000 7f50a968f640 1 -- 172.21.15.53:0/2396974816 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40c9860 con 0x7f50a415aa00 2023-10-21T16:02:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.567+0000 7f50a968f640 1 --2- 172.21.15.53:0/2396974816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a415aa00 0x7f50a40b2a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50a40716b0 tx=0x7f509c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d984430c0e8df513 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50a37fe640 1 -- 172.21.15.53:0/2396974816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c02cc80 con 0x7f50a415aa00 2023-10-21T16:02:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50a37fe640 1 -- 172.21.15.53:0/2396974816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c02cde0 con 0x7f50a415aa00 2023-10-21T16:02:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50a37fe640 1 -- 172.21.15.53:0/2396974816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c034640 con 0x7f50a415aa00 2023-10-21T16:02:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 -- 172.21.15.53:0/2396974816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a415aa00 msgr2=0x7f50a40b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/2396974816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a415aa00 0x7f50a40b2a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50a40716b0 tx=0x7f509c02bed0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 -- 172.21.15.53:0/2396974816 shutdown_connections 2023-10-21T16:02:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/2396974816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a415aa00 0x7f50a40b2a90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/2396974816 >> v2:172.21.15.124:3300/0 conn(0x7f50a406c3a0 0x7f50a40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 -- 172.21.15.53:0/2396974816 >> 172.21.15.53:0/2396974816 conn(0x7f50a406a5d0 msgr2=0x7f50a40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:21.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 -- 172.21.15.53:0/2396974816 shutdown_connections 2023-10-21T16:02:21.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.568+0000 7f50ab0f2640 1 -- 172.21.15.53:0/2396974816 wait complete. 2023-10-21T16:02:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.570+0000 7f50ab0f2640 1 Processor -- start 2023-10-21T16:02:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.570+0000 7f50ab0f2640 1 -- start start 2023-10-21T16:02:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50a406db60 0x7f50a4154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50a415aa00 0x7f50a4154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a41551f0 con 0x7f50a406c3a0 2023-10-21T16:02:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40c6e10 con 0x7f50a415aa00 2023-10-21T16:02:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.571+0000 7f50ab0f2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4154f70 con 0x7f50a406db60 2023-10-21T16:02:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41556/0 (socket says 172.21.15.53:41556) 2023-10-21T16:02:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 -- 172.21.15.53:0/3920020829 learned_addr learned my addr 172.21.15.53:0/3920020829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a968f640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50a415aa00 0x7f50a4154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 -- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50a406db60 msgr2=0x7f50a4154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50a406db60 0x7f50a4154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 -- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50a415aa00 msgr2=0x7f50a4154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50a415aa00 0x7f50a4154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 -- 172.21.15.53:0/3920020829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009a70 con 0x7f50a406c3a0 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.572+0000 7f50a8e8e640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9e0 tx=0x7f509800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.573+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800c9d0 con 0x7f50a406c3a0 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.573+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800cb30 con 0x7f50a406c3a0 2023-10-21T16:02:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.573+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800cee0 con 0x7f50a406c3a0 2023-10-21T16:02:21.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.573+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a40c64e0 con 0x7f50a406c3a0 2023-10-21T16:02:21.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.573+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a40c6aa0 con 0x7f50a406c3a0 2023-10-21T16:02:21.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.574+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a404e160 con 0x7f50a406c3a0 2023-10-21T16:02:21.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.575+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5098013470 con 0x7f50a406c3a0 2023-10-21T16:02:21.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.576+0000 7f50a17fa640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 0x7f508806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:21.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.576+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5098089ba0 con 0x7f50a406c3a0 2023-10-21T16:02:21.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.577+0000 7f50a3fff640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 0x7f508806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:21.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.578+0000 7f50a3fff640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 0x7f508806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5090001370 tx=0x7f5090001130 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:21.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.581+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5098054950 con 0x7f50a406c3a0 2023-10-21T16:02:21.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.812+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50a4070e40 con 0x7f50a406c3a0 2023-10-21T16:02:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.980+0000 7f50a17fa640 1 -- 172.21.15.53:0/3920020829 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50980170b0 con 0x7f50a406c3a0 2023-10-21T16:02:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 msgr2=0x7f508806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 0x7f508806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5090001370 tx=0x7f5090001130 comp rx=0 tx=0).stop 2023-10-21T16:02:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 msgr2=0x7f50a40708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9e0 tx=0x7f509800eea0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 shutdown_connections 2023-10-21T16:02:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50a415aa00 0x7f50a4154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50a406db60 0x7f50a4154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f508806ae70 0x7f508806d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 --2- 172.21.15.53:0/3920020829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50a406c3a0 0x7f50a40708b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 >> 172.21.15.53:0/3920020829 conn(0x7f50a406a5d0 msgr2=0x7f50a4158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 shutdown_connections 2023-10-21T16:02:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:21.984+0000 7f50ab0f2640 1 -- 172.21.15.53:0/3920020829 wait complete. 2023-10-21T16:02:22.007 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-21T16:02:22.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:02:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.236+0000 7fdbf33b7640 1 Processor -- start 2023-10-21T16:02:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.237+0000 7fdbf33b7640 1 -- start start 2023-10-21T16:02:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.238+0000 7fdbf33b7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdbec06a460 0x7fdbec0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.238+0000 7fdbf33b7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0ce210 con 0x7fdbec071c30 2023-10-21T16:02:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.238+0000 7fdbf33b7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec0cdb40 con 0x7fdbec1601b0 2023-10-21T16:02:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.238+0000 7fdbf33b7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec0cdee0 con 0x7fdbec06a460 2023-10-21T16:02:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf1954640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdbec06a460 0x7fdbec0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf0952640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdbec071c30 0x7fdbec069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55028/0 (socket says 172.21.15.53:55028) 2023-10-21T16:02:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf0952640 1 -- 172.21.15.53:0/1568223019 learned_addr learned my addr 172.21.15.53:0/1568223019 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf1954640 1 -- 172.21.15.53:0/1568223019 >> v1:172.21.15.124:6789/0 conn(0x7fdbec1601b0 legacy=0x7fdbec0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf1954640 1 -- 172.21.15.53:0/1568223019 >> v1:172.21.15.53:6789/0 conn(0x7fdbec071c30 legacy=0x7fdbec069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.239+0000 7fdbf1954640 1 -- 172.21.15.53:0/1568223019 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec161530 con 0x7fdbec06a460 2023-10-21T16:02:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.241+0000 7fdbf1954640 1 --2- 172.21.15.53:0/1568223019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec0ca090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a3ebccb945c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.241+0000 7fdbe3fff640 1 -- 172.21.15.53:0/1568223019 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe402cc60 con 0x7fdbec06a460 2023-10-21T16:02:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.241+0000 7fdbe3fff640 1 -- 172.21.15.53:0/1568223019 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe402cdc0 con 0x7fdbec06a460 2023-10-21T16:02:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.241+0000 7fdbe3fff640 1 -- 172.21.15.53:0/1568223019 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4034660 con 0x7fdbec06a460 2023-10-21T16:02:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.241+0000 7fdbf33b7640 1 -- 172.21.15.53:0/1568223019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 msgr2=0x7fdbec0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/1568223019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec0ca090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d90 tx=0x7fdbe402beb0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 -- 172.21.15.53:0/1568223019 shutdown_connections 2023-10-21T16:02:22.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/1568223019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec0ca090 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 -- 172.21.15.53:0/1568223019 >> 172.21.15.53:0/1568223019 conn(0x7fdbec06d430 msgr2=0x7fdbec06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:22.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 -- 172.21.15.53:0/1568223019 shutdown_connections 2023-10-21T16:02:22.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.242+0000 7fdbf33b7640 1 -- 172.21.15.53:0/1568223019 wait complete. 2023-10-21T16:02:22.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.244+0000 7fdbf33b7640 1 Processor -- start 2023-10-21T16:02:22.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.244+0000 7fdbf33b7640 1 -- start start 2023-10-21T16:02:22.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:22.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 0x7fdbec15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 0x7fdbec0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0abd20 con 0x7fdbec06a460 2023-10-21T16:02:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c5a10 con 0x7fdbec1601b0 2023-10-21T16:02:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf33b7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0c6180 con 0x7fdbec071c30 2023-10-21T16:02:22.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf1153640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:22.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf1153640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41566/0 (socket says 172.21.15.53:41566) 2023-10-21T16:02:22.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf1153640 1 -- 172.21.15.53:0/2697079393 learned_addr learned my addr 172.21.15.53:0/2697079393 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:22.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.245+0000 7fdbf1954640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 0x7fdbec0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:22.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf0952640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 0x7fdbec15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:22.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 -- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 msgr2=0x7fdbec15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 0x7fdbec15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 -- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 msgr2=0x7fdbec0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 0x7fdbec0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 -- 172.21.15.53:0/2697079393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4009a70 con 0x7fdbec06a460 2023-10-21T16:02:22.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1954640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 0x7fdbec0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf0952640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 0x7fdbec15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf1153640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b420 tx=0x7fdbe800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8011830 con 0x7fdbec06a460 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8011990 con 0x7fdbec06a460 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8011d40 con 0x7fdbec06a460 2023-10-21T16:02:22.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0ac3c0 con 0x7fdbec06a460 2023-10-21T16:02:22.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.246+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0c5c60 con 0x7fdbec06a460 2023-10-21T16:02:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.248+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec04e160 con 0x7fdbec06a460 2023-10-21T16:02:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.249+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdbe801b460 con 0x7fdbec06a460 2023-10-21T16:02:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.250+0000 7fdbe1ffb640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 0x7fdbd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.250+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbe8089400 con 0x7fdbec06a460 2023-10-21T16:02:22.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.250+0000 7fdbf0952640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 0x7fdbd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.251+0000 7fdbf0952640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 0x7fdbd006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007fd0 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:22.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.254+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbe8054250 con 0x7fdbec06a460 2023-10-21T16:02:22.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.487+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdbec0b13b0 con 0x7fdbec06a460 2023-10-21T16:02:22.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.983+0000 7fdbe1ffb640 1 -- 172.21.15.53:0/2697079393 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdbe8057870 con 0x7fdbec06a460 2023-10-21T16:02:22.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.986+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 msgr2=0x7fdbd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.986+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 0x7fdbd006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc007fd0 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2023-10-21T16:02:22.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 msgr2=0x7fdbec15b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:22.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b420 tx=0x7fdbe800b8e0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 shutdown_connections 2023-10-21T16:02:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbec1601b0 0x7fdbec0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbec071c30 0x7fdbec15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdbd006ae70 0x7fdbd006d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 --2- 172.21.15.53:0/2697079393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbec06a460 0x7fdbec15b7d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 >> 172.21.15.53:0/2697079393 conn(0x7fdbec06d430 msgr2=0x7fdbec069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 shutdown_connections 2023-10-21T16:02:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:22.987+0000 7fdbf33b7640 1 -- 172.21.15.53:0/2697079393 wait complete. 2023-10-21T16:02:23.010 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:02:23.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:23.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.234+0000 7f3adcedf640 1 Processor -- start 2023-10-21T16:02:23.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.234+0000 7f3adcedf640 1 -- start start 2023-10-21T16:02:23.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3adcedf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3ad815ab30 0x7f3ad815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3adcedf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 0x7f3ad80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3adcedf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad815f7d0 con 0x7f3ad815af10 2023-10-21T16:02:23.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3adcedf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad815f210 con 0x7f3ad815ab30 2023-10-21T16:02:23.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3adcedf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad815f4f0 con 0x7f3ad80adc70 2023-10-21T16:02:23.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3ad7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3ad815af10 0x7f3ad80701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55038/0 (socket says 172.21.15.53:55038) 2023-10-21T16:02:23.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.236+0000 7f3ad7fff640 1 -- 172.21.15.53:0/1516862594 learned_addr learned my addr 172.21.15.53:0/1516862594 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:23.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1516862594 >> v2:172.21.15.124:3300/0 conn(0x7f3ad815ab30 0x7f3ad815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad6ffd640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 0x7f3ad80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303302261 0 0) 0x7f3ad815f7d0 con 0x7f3ad815af10 2023-10-21T16:02:23.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac0003520 con 0x7f3ad815af10 2023-10-21T16:02:23.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3113178167 0 0) 0x7f3ac0003520 con 0x7f3ad815af10 2023-10-21T16:02:23.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 msgr2=0x7f3ad80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 0x7f3ad80b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 >> v2:172.21.15.124:3300/0 conn(0x7f3ad815ab30 msgr2=0x7f3ad815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 --2- 172.21.15.53:0/1516862594 >> v2:172.21.15.124:3300/0 conn(0x7f3ad815ab30 0x7f3ad815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad806c550 con 0x7f3ad815af10 2023-10-21T16:02:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3ad0001a10 con 0x7f3ad815af10 2023-10-21T16:02:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 >> v1:172.21.15.53:6789/0 conn(0x7f3ad815af10 legacy=0x7f3ad80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0003970 0x7f3ac0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 0x7f3ac00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0004d00 0x7f3ac0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.237+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad815f210 con 0x7f3ac0004d00 2023-10-21T16:02:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad815f4f0 con 0x7f3ac0003970 2023-10-21T16:02:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac000b980 con 0x7f3ad80adc70 2023-10-21T16:02:23.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0004d00 0x7f3ac0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0003970 0x7f3ac0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 -- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 msgr2=0x7f3ac00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80adc70 0x7f3ac00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 -- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0003970 msgr2=0x7f3ac0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0003970 0x7f3ac0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad7fff640 1 -- 172.21.15.53:0/1516862594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0003520 con 0x7f3ac0004d00 2023-10-21T16:02:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.238+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0003970 0x7f3ac0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.239+0000 7f3ad7fff640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0004d00 0x7f3ac0005180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000a880 tx=0x7f3ac00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c02d1882a45db8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad002cea0 con 0x7f3ac0004d00 2023-10-21T16:02:23.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad002f070 con 0x7f3ac0004d00 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3ad67fc640 1 -- 172.21.15.53:0/1516862594 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0034870 con 0x7f3ac0004d00 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 -- 172.21.15.53:0/1516862594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0004d00 msgr2=0x7f3ac0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1516862594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac0004d00 0x7f3ac0005180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000a880 tx=0x7f3ac00057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3ad7fff640 1 -- 172.21.15.53:0/1516862594 reap_dead start 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 -- 172.21.15.53:0/1516862594 shutdown_connections 2023-10-21T16:02:23.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 -- 172.21.15.53:0/1516862594 >> 172.21.15.53:0/1516862594 conn(0x7f3ad80cb0c0 msgr2=0x7f3ad80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 -- 172.21.15.53:0/1516862594 shutdown_connections 2023-10-21T16:02:23.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.240+0000 7f3adcedf640 1 -- 172.21.15.53:0/1516862594 wait complete. 2023-10-21T16:02:23.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.242+0000 7f3adcedf640 1 Processor -- start 2023-10-21T16:02:23.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.242+0000 7f3adcedf640 1 -- start start 2023-10-21T16:02:23.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0004d00 0x7f3ad815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80b3830 0x7f3ad80b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b41f0 con 0x7f3ad8156730 2023-10-21T16:02:23.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b2850 con 0x7f3ac0004d00 2023-10-21T16:02:23.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.243+0000 7f3adcedf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c8140 con 0x7f3ad80b3830 2023-10-21T16:02:23.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41584/0 (socket says 172.21.15.53:41584) 2023-10-21T16:02:23.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 -- 172.21.15.53:0/1569561541 learned_addr learned my addr 172.21.15.53:0/1569561541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:23.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0004d00 0x7f3ad815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad7fff640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80b3830 0x7f3ad80b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 -- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80b3830 msgr2=0x7f3ad80b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80b3830 0x7f3ad80b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 -- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0004d00 msgr2=0x7f3ad815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0004d00 0x7f3ad815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 -- 172.21.15.53:0/1569561541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80c8420 con 0x7f3ad8156730 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.244+0000 7f3ad6ffd640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00e7f0 tx=0x7f3acc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.245+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00eea0 con 0x7f3ad8156730 2023-10-21T16:02:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.245+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc00c940 con 0x7f3ad8156730 2023-10-21T16:02:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.245+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00ccd0 con 0x7f3ad8156730 2023-10-21T16:02:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.245+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad80b2af0 con 0x7f3ad8156730 2023-10-21T16:02:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.245+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad815e070 con 0x7f3ad8156730 2023-10-21T16:02:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.246+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad804e160 con 0x7f3ad8156730 2023-10-21T16:02:23.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.247+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3acc013470 con 0x7f3ad8156730 2023-10-21T16:02:23.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.248+0000 7f3abffff640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 0x7f3ac406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.249+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3acc089840 con 0x7f3ad8156730 2023-10-21T16:02:23.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.249+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 0x7f3ac406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.250+0000 7f3ad77fe640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 0x7f3ac406d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009b40 tx=0x7f3ac8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.253+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3acc054740 con 0x7f3ad8156730 2023-10-21T16:02:23.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.484+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ad80b0f70 con 0x7f3ad8156730 2023-10-21T16:02:23.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.485+0000 7f3abffff640 1 -- 172.21.15.53:0/1569561541 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f3acc057d60 con 0x7f3ad8156730 2023-10-21T16:02:23.487 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:23.488 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:02:23.488 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:02:23.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 msgr2=0x7f3ac406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 0x7f3ac406d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009b40 tx=0x7f3ac8008040 comp rx=0 tx=0).stop 2023-10-21T16:02:23.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 msgr2=0x7f3ad80b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00e7f0 tx=0x7f3acc00ecb0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 shutdown_connections 2023-10-21T16:02:23.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ad80b3830 0x7f3ad80b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ac406ae70 0x7f3ac406d330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ad8156730 0x7f3ad80b32f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 --2- 172.21.15.53:0/1569561541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac0004d00 0x7f3ad815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 >> 172.21.15.53:0/1569561541 conn(0x7f3ad80cb0c0 msgr2=0x7f3ad80cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:23.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 shutdown_connections 2023-10-21T16:02:23.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.488+0000 7f3adcedf640 1 -- 172.21.15.53:0/1569561541 wait complete. 2023-10-21T16:02:23.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.730+0000 7fa20746f640 1 Processor -- start 2023-10-21T16:02:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.731+0000 7fa20746f640 1 -- start start 2023-10-21T16:02:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.732+0000 7fa20746f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa200165cc0 0x7fa2001680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.732+0000 7fa20746f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000c48f0 con 0x7fa200160940 2023-10-21T16:02:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.732+0000 7fa20746f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000710f0 con 0x7fa20015cd90 2023-10-21T16:02:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.732+0000 7fa20746f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000c4d80 con 0x7fa200165cc0 2023-10-21T16:02:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.733+0000 7fa204a0a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa200160940 0x7fa2000709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55042/0 (socket says 172.21.15.53:55042) 2023-10-21T16:02:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.733+0000 7fa204a0a640 1 -- 172.21.15.53:0/2455941787 learned_addr learned my addr 172.21.15.53:0/2455941787 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.733+0000 7fa205a0c640 1 --2- 172.21.15.53:0/2455941787 >> v2:172.21.15.124:3300/0 conn(0x7fa200165cc0 0x7fa2001680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.733+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389602568 0 0) 0x7fa2000c48f0 con 0x7fa200160940 2023-10-21T16:02:23.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.733+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec003520 con 0x7fa200160940 2023-10-21T16:02:23.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 794422591 0 0) 0x7fa1ec003520 con 0x7fa200160940 2023-10-21T16:02:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 >> v2:172.21.15.124:3300/0 conn(0x7fa200165cc0 msgr2=0x7fa2001680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 --2- 172.21.15.53:0/2455941787 >> v2:172.21.15.124:3300/0 conn(0x7fa200165cc0 0x7fa2001680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 >> v1:172.21.15.124:6789/0 conn(0x7fa20015cd90 legacy=0x7fa2000702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000c3ce0 con 0x7fa200160940 2023-10-21T16:02:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa1f0001670 con 0x7fa200160940 2023-10-21T16:02:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa205a0c640 1 --2- 172.21.15.53:0/2455941787 >> v2:172.21.15.124:3300/0 conn(0x7fa200165cc0 0x7fa2001680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 >> v1:172.21.15.53:6789/0 conn(0x7fa200160940 legacy=0x7fa2000709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1ec0039a0 0x7fa1ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa1ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1ec005180 0x7fa1ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00c050 con 0x7fa1ec0047c0 2023-10-21T16:02:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00c190 con 0x7fa1ec005180 2023-10-21T16:02:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00c350 con 0x7fa1ec0039a0 2023-10-21T16:02:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa204a0a640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa1ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.734+0000 7fa205a0c640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1ec005180 0x7fa1ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa20520b640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1ec0039a0 0x7fa1ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa204a0a640 1 -- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1ec0039a0 msgr2=0x7fa1ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa204a0a640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1ec0039a0 0x7fa1ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa204a0a640 1 -- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1ec005180 msgr2=0x7fa1ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:23.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa204a0a640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1ec005180 0x7fa1ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa204a0a640 1 -- 172.21.15.53:0/2455941787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec003520 con 0x7fa1ec0047c0 2023-10-21T16:02:23.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa20520b640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1ec0039a0 0x7fa1ec003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:23.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa205a0c640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1ec005180 0x7fa1ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:23.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.735+0000 7fa205a0c640 1 -- 172.21.15.53:0/2455941787 reap_dead start 2023-10-21T16:02:23.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.736+0000 7fa204a0a640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa1ec004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c240 tx=0x7fa1ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b5b1c8ea8194d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.736+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0005ad0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.736+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0005c30 con 0x7fa1ec0047c0 2023-10-21T16:02:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.736+0000 7fa1f7fff640 1 -- 172.21.15.53:0/2455941787 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002cab0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 -- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 msgr2=0x7fa1ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa1ec004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c240 tx=0x7fa1ec0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 -- 172.21.15.53:0/2455941787 shutdown_connections 2023-10-21T16:02:23.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 --2- 172.21.15.53:0/2455941787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa1ec004c40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 -- 172.21.15.53:0/2455941787 >> 172.21.15.53:0/2455941787 conn(0x7fa20006a380 msgr2=0x7fa2000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:23.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 -- 172.21.15.53:0/2455941787 shutdown_connections 2023-10-21T16:02:23.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.737+0000 7fa20746f640 1 -- 172.21.15.53:0/2455941787 wait complete. 2023-10-21T16:02:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.739+0000 7fa20746f640 1 Processor -- start 2023-10-21T16:02:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.739+0000 7fa20746f640 1 -- start start 2023-10-21T16:02:23.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2000acb70 0x7fa2000acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000b2970 0x7fa2000b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000ad530 con 0x7fa1ec0047c0 2023-10-21T16:02:23.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b3330 con 0x7fa2000b2970 2023-10-21T16:02:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20746f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b4160 con 0x7fa2000acb70 2023-10-21T16:02:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20520b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20520b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41594/0 (socket says 172.21.15.53:41594) 2023-10-21T16:02:23.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.740+0000 7fa20520b640 1 -- 172.21.15.53:0/1556386068 learned_addr learned my addr 172.21.15.53:0/1556386068 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa205a0c640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000b2970 0x7fa2000b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 -- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2000acb70 msgr2=0x7fa2000acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2000acb70 0x7fa2000acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 -- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000b2970 msgr2=0x7fa2000b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000b2970 0x7fa2000b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 -- 172.21.15.53:0/1556386068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa200149800 con 0x7fa1ec0047c0 2023-10-21T16:02:23.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa20520b640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00d430 tx=0x7fa1fc00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00b070 con 0x7fa1ec0047c0 2023-10-21T16:02:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc0054e0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.741+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc003850 con 0x7fa1ec0047c0 2023-10-21T16:02:23.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.742+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa200152fd0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.742+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2000b3830 con 0x7fa1ec0047c0 2023-10-21T16:02:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.743+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa200067490 con 0x7fa1ec0047c0 2023-10-21T16:02:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.744+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa1fc0039b0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.745+0000 7fa1f5ffb640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 0x7fa1e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.745+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1fc089140 con 0x7fa1ec0047c0 2023-10-21T16:02:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.745+0000 7fa204a0a640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 0x7fa1e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.746+0000 7fa204a0a640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 0x7fa1e406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0034000 tx=0x7fa1f0039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:23.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.749+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1fc053fb0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.937+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2000b0ef0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.940+0000 7fa1f5ffb640 1 -- 172.21.15.53:0/1556386068 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fa1fc0575d0 con 0x7fa1ec0047c0 2023-10-21T16:02:23.942 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:23.943 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:23.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.942+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 msgr2=0x7fa1e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 0x7fa1e406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0034000 tx=0x7fa1f0039040 comp rx=0 tx=0).stop 2023-10-21T16:02:23.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 msgr2=0x7fa2001492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:23.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00d430 tx=0x7fa1fc00d8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 shutdown_connections 2023-10-21T16:02:23.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000b2970 0x7fa2000b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2000acb70 0x7fa2000acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa1e406ae70 0x7fa1e406d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 --2- 172.21.15.53:0/1556386068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1ec0047c0 0x7fa2001492c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:23.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 >> 172.21.15.53:0/1556386068 conn(0x7fa20006a380 msgr2=0x7fa2001568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:23.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 shutdown_connections 2023-10-21T16:02:23.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:23.943+0000 7fa20746f640 1 -- 172.21.15.53:0/1556386068 wait complete. 2023-10-21T16:02:23.964 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:02:23.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:02:24.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.186+0000 7ff05942e640 1 Processor -- start 2023-10-21T16:02:24.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.186+0000 7ff05942e640 1 -- start start 2023-10-21T16:02:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.187+0000 7ff05942e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff05415cc60 0x7ff0540cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.187+0000 7ff05942e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff0540d16f0 0x7ff0540cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff05942e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05415d040 con 0x7ff0540cdfd0 2023-10-21T16:02:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff05942e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540d0650 con 0x7ff05415cc60 2023-10-21T16:02:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff05942e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0540d0930 con 0x7ff0540d16f0 2023-10-21T16:02:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff053fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff05415cc60 0x7ff0540cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff058c2d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff0540cdfd0 0x7ff0540ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59622/0 (socket says 172.21.15.53:59622) 2023-10-21T16:02:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff058c2d640 1 -- 172.21.15.53:0/1240359119 learned_addr learned my addr 172.21.15.53:0/1240359119 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff053fff640 1 -- 172.21.15.53:0/1240359119 >> v2:172.21.15.124:3300/0 conn(0x7ff0540d16f0 msgr2=0x7ff0540cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:24.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.188+0000 7ff053fff640 1 --2- 172.21.15.53:0/1240359119 >> v2:172.21.15.124:3300/0 conn(0x7ff0540d16f0 0x7ff0540cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.189+0000 7ff053fff640 1 -- 172.21.15.53:0/1240359119 >> v1:172.21.15.53:6789/0 conn(0x7ff0540cdfd0 legacy=0x7ff0540ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.189+0000 7ff053fff640 1 -- 172.21.15.53:0/1240359119 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0540d0c10 con 0x7ff05415cc60 2023-10-21T16:02:24.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.190+0000 7ff053fff640 1 --2- 172.21.15.53:0/1240359119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff0540cf340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff0540b5560 tx=0x7ff04402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10043ba8cbf6a48 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.190+0000 7ff0537fe640 1 -- 172.21.15.53:0/1240359119 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031830 con 0x7ff05415cc60 2023-10-21T16:02:24.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.190+0000 7ff0537fe640 1 -- 172.21.15.53:0/1240359119 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044031990 con 0x7ff05415cc60 2023-10-21T16:02:24.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.190+0000 7ff0537fe640 1 -- 172.21.15.53:0/1240359119 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044031d20 con 0x7ff05415cc60 2023-10-21T16:02:24.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 -- 172.21.15.53:0/1240359119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 msgr2=0x7ff0540cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 --2- 172.21.15.53:0/1240359119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff0540cf340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff0540b5560 tx=0x7ff04402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 -- 172.21.15.53:0/1240359119 shutdown_connections 2023-10-21T16:02:24.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 --2- 172.21.15.53:0/1240359119 >> v2:172.21.15.124:3300/0 conn(0x7ff0540d16f0 0x7ff0540cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 --2- 172.21.15.53:0/1240359119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff0540cf340 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 -- 172.21.15.53:0/1240359119 >> 172.21.15.53:0/1240359119 conn(0x7ff05409bcf0 msgr2=0x7ff0540d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:24.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 -- 172.21.15.53:0/1240359119 shutdown_connections 2023-10-21T16:02:24.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.191+0000 7ff05942e640 1 -- 172.21.15.53:0/1240359119 wait complete. 2023-10-21T16:02:24.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.193+0000 7ff05942e640 1 Processor -- start 2023-10-21T16:02:24.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.193+0000 7ff05942e640 1 -- start start 2023-10-21T16:02:24.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0540cdfd0 0x7ff0541529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0540d16f0 0x7ff0541555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540c80c0 con 0x7ff05415cc60 2023-10-21T16:02:24.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054152f00 con 0x7ff0540d16f0 2023-10-21T16:02:24.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.194+0000 7ff05942e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff054154310 con 0x7ff0540cdfd0 2023-10-21T16:02:24.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59394/0 (socket says 172.21.15.53:59394) 2023-10-21T16:02:24.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 -- 172.21.15.53:0/2659297550 learned_addr learned my addr 172.21.15.53:0/2659297550 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:24.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff053fff640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0540cdfd0 0x7ff0541529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff04b7fe640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0540d16f0 0x7ff0541555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 -- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0540cdfd0 msgr2=0x7ff0541529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0540cdfd0 0x7ff0541529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 -- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0540d16f0 msgr2=0x7ff0541555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0540d16f0 0x7ff0541555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 -- 172.21.15.53:0/2659297550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0440093a0 con 0x7ff05415cc60 2023-10-21T16:02:24.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.195+0000 7ff058c2d640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00ca80 tx=0x7ff04c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.196+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00ac30 con 0x7ff05415cc60 2023-10-21T16:02:24.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.196+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c00ad90 con 0x7ff05415cc60 2023-10-21T16:02:24.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.196+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c012600 con 0x7ff05415cc60 2023-10-21T16:02:24.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.196+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0541545f0 con 0x7ff05415cc60 2023-10-21T16:02:24.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.196+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff054154bb0 con 0x7ff05415cc60 2023-10-21T16:02:24.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.197+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff054154da0 con 0x7ff05415cc60 2023-10-21T16:02:24.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.198+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff04c007070 con 0x7ff05415cc60 2023-10-21T16:02:24.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.199+0000 7ff0517fa640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 0x7ff03406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.199+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff04c089680 con 0x7ff05415cc60 2023-10-21T16:02:24.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.200+0000 7ff053fff640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 0x7ff03406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.201+0000 7ff053fff640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 0x7ff03406d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff044036000 tx=0x7ff04403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.204+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff04c054580 con 0x7ff05415cc60 2023-10-21T16:02:24.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.393+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff054154480 con 0x7ff03406ae70 2023-10-21T16:02:24.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.427+0000 7ff0517fa640 1 -- 172.21.15.53:0/2659297550 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345122 (secure 0 0 0) 0x7ff054154480 con 0x7ff03406ae70 2023-10-21T16:02:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:02:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.433+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 msgr2=0x7ff03406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.433+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 0x7ff03406d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff044036000 tx=0x7ff04403c040 comp rx=0 tx=0).stop 2023-10-21T16:02:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.433+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 msgr2=0x7ff054155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.433+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00ca80 tx=0x7ff04c00cf40 comp rx=0 tx=0).stop 2023-10-21T16:02:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 shutdown_connections 2023-10-21T16:02:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff03406ae70 0x7ff03406d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff05415cc60 0x7ff054155b00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0540d16f0 0x7ff0541555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 --2- 172.21.15.53:0/2659297550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0540cdfd0 0x7ff0541529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 >> 172.21.15.53:0/2659297550 conn(0x7ff05409bcf0 msgr2=0x7ff05415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 shutdown_connections 2023-10-21T16:02:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.434+0000 7ff05942e640 1 -- 172.21.15.53:0/2659297550 wait complete. 2023-10-21T16:02:24.457 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-21T16:02:23.025103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":211,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3481600,"data_stored":2082142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005038"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928748+0000","last_change":"2023-10-21T16:01:55.158448+0000","last_active":"2023-10-21T16:02:00.928748+0000","last_peered":"2023-10-21T16:02:00.928748+0000","last_clean":"2023-10-21T16:02:00.928748+0000","last_became_active":"2023-10-21T16:01:55.158229+0000","last_became_peered":"2023-10-21T16:01:55.158229+0000","last_unstale":"2023-10-21T16:02:00.928748+0000","last_undegraded":"2023-10-21T16:02:00.928748+0000","last_fullsized":"2023-10-21T16:02:00.928748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947239+0000","last_change":"2023-10-21T16:01:55.937779+0000","last_active":"2023-10-21T16:02:00.947239+0000","last_peered":"2023-10-21T16:02:00.947239+0000","last_clean":"2023-10-21T16:02:00.947239+0000","last_became_active":"2023-10-21T16:01:55.937491+0000","last_became_peered":"2023-10-21T16:01:55.937491+0000","last_unstale":"2023-10-21T16:02:00.947239+0000","last_undegraded":"2023-10-21T16:02:00.947239+0000","last_fullsized":"2023-10-21T16:02:00.947239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895601+0000","last_change":"2023-10-21T16:02:00.754927+0000","last_active":"2023-10-21T16:02:02.895601+0000","last_peered":"2023-10-21T16:02:02.895601+0000","last_clean":"2023-10-21T16:02:02.895601+0000","last_became_active":"2023-10-21T16:02:00.753883+0000","last_became_peered":"2023-10-21T16:02:00.753883+0000","last_unstale":"2023-10-21T16:02:02.895601+0000","last_undegraded":"2023-10-21T16:02:02.895601+0000","last_fullsized":"2023-10-21T16:02:02.895601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:45.898604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947499+0000","last_change":"2023-10-21T16:01:55.153722+0000","last_active":"2023-10-21T16:02:00.947499+0000","last_peered":"2023-10-21T16:02:00.947499+0000","last_clean":"2023-10-21T16:02:00.947499+0000","last_became_active":"2023-10-21T16:01:55.153434+0000","last_became_peered":"2023-10-21T16:01:55.153434+0000","last_unstale":"2023-10-21T16:02:00.947499+0000","last_undegraded":"2023-10-21T16:02:00.947499+0000","last_fullsized":"2023-10-21T16:02:00.947499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928468+0000","last_change":"2023-10-21T16:01:55.954746+0000","last_active":"2023-10-21T16:02:00.928468+0000","last_peered":"2023-10-21T16:02:00.928468+0000","last_clean":"2023-10-21T16:02:00.928468+0000","last_became_active":"2023-10-21T16:01:55.953670+0000","last_became_peered":"2023-10-21T16:01:55.953670+0000","last_unstale":"2023-10-21T16:02:00.928468+0000","last_undegraded":"2023-10-21T16:02:00.928468+0000","last_fullsized":"2023-10-21T16:02:00.928468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918749+0000","last_change":"2023-10-21T16:02:00.185538+0000","last_active":"2023-10-21T16:02:01.918749+0000","last_peered":"2023-10-21T16:02:01.918749+0000","last_clean":"2023-10-21T16:02:01.918749+0000","last_became_active":"2023-10-21T16:02:00.185127+0000","last_became_peered":"2023-10-21T16:02:00.185127+0000","last_unstale":"2023-10-21T16:02:01.918749+0000","last_undegraded":"2023-10-21T16:02:01.918749+0000","last_fullsized":"2023-10-21T16:02:01.918749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415743+0000","last_change":"2023-10-21T16:01:55.169060+0000","last_active":"2023-10-21T16:02:01.415743+0000","last_peered":"2023-10-21T16:02:01.415743+0000","last_clean":"2023-10-21T16:02:01.415743+0000","last_became_active":"2023-10-21T16:01:55.168772+0000","last_became_peered":"2023-10-21T16:01:55.168772+0000","last_unstale":"2023-10-21T16:02:01.415743+0000","last_undegraded":"2023-10-21T16:02:01.415743+0000","last_fullsized":"2023-10-21T16:02:01.415743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004837+0000","last_change":"2023-10-21T16:01:55.950057+0000","last_active":"2023-10-21T16:02:01.004837+0000","last_peered":"2023-10-21T16:02:01.004837+0000","last_clean":"2023-10-21T16:02:01.004837+0000","last_became_active":"2023-10-21T16:01:55.949780+0000","last_became_peered":"2023-10-21T16:01:55.949780+0000","last_unstale":"2023-10-21T16:02:01.004837+0000","last_undegraded":"2023-10-21T16:02:01.004837+0000","last_fullsized":"2023-10-21T16:02:01.004837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921013+0000","last_change":"2023-10-21T16:02:00.184728+0000","last_active":"2023-10-21T16:02:01.921013+0000","last_peered":"2023-10-21T16:02:01.921013+0000","last_clean":"2023-10-21T16:02:01.921013+0000","last_became_active":"2023-10-21T16:02:00.183993+0000","last_became_peered":"2023-10-21T16:02:00.183993+0000","last_unstale":"2023-10-21T16:02:01.921013+0000","last_undegraded":"2023-10-21T16:02:01.921013+0000","last_fullsized":"2023-10-21T16:02:01.921013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415376+0000","last_change":"2023-10-21T16:01:55.171185+0000","last_active":"2023-10-21T16:02:01.415376+0000","last_peered":"2023-10-21T16:02:01.415376+0000","last_clean":"2023-10-21T16:02:01.415376+0000","last_became_active":"2023-10-21T16:01:55.170714+0000","last_became_peered":"2023-10-21T16:01:55.170714+0000","last_unstale":"2023-10-21T16:02:01.415376+0000","last_undegraded":"2023-10-21T16:02:01.415376+0000","last_fullsized":"2023-10-21T16:02:01.415376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004885+0000","last_change":"2023-10-21T16:01:55.957250+0000","last_active":"2023-10-21T16:02:01.004885+0000","last_peered":"2023-10-21T16:02:01.004885+0000","last_clean":"2023-10-21T16:02:01.004885+0000","last_became_active":"2023-10-21T16:01:55.956926+0000","last_became_peered":"2023-10-21T16:01:55.956926+0000","last_unstale":"2023-10-21T16:02:01.004885+0000","last_undegraded":"2023-10-21T16:02:01.004885+0000","last_fullsized":"2023-10-21T16:02:01.004885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.920949+0000","last_change":"2023-10-21T16:02:00.190590+0000","last_active":"2023-10-21T16:02:01.920949+0000","last_peered":"2023-10-21T16:02:01.920949+0000","last_clean":"2023-10-21T16:02:01.920949+0000","last_became_active":"2023-10-21T16:02:00.190131+0000","last_became_peered":"2023-10-21T16:02:00.190131+0000","last_unstale":"2023-10-21T16:02:01.920949+0000","last_undegraded":"2023-10-21T16:02:01.920949+0000","last_fullsized":"2023-10-21T16:02:01.920949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413796+0000","last_change":"2023-10-21T16:01:55.948227+0000","last_active":"2023-10-21T16:02:01.413796+0000","last_peered":"2023-10-21T16:02:01.413796+0000","last_clean":"2023-10-21T16:02:01.413796+0000","last_became_active":"2023-10-21T16:01:55.947879+0000","last_became_peered":"2023-10-21T16:01:55.947879+0000","last_unstale":"2023-10-21T16:02:01.413796+0000","last_undegraded":"2023-10-21T16:02:01.413796+0000","last_fullsized":"2023-10-21T16:02:01.413796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415138+0000","last_change":"2023-10-21T16:01:55.161941+0000","last_active":"2023-10-21T16:02:01.415138+0000","last_peered":"2023-10-21T16:02:01.415138+0000","last_clean":"2023-10-21T16:02:01.415138+0000","last_became_active":"2023-10-21T16:01:55.160804+0000","last_became_peered":"2023-10-21T16:01:55.160804+0000","last_unstale":"2023-10-21T16:02:01.415138+0000","last_undegraded":"2023-10-21T16:02:01.415138+0000","last_fullsized":"2023-10-21T16:02:01.415138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919532+0000","last_change":"2023-10-21T16:02:00.179242+0000","last_active":"2023-10-21T16:02:01.919532+0000","last_peered":"2023-10-21T16:02:01.919532+0000","last_clean":"2023-10-21T16:02:01.919532+0000","last_became_active":"2023-10-21T16:02:00.178744+0000","last_became_peered":"2023-10-21T16:02:00.178744+0000","last_unstale":"2023-10-21T16:02:01.919532+0000","last_undegraded":"2023-10-21T16:02:01.919532+0000","last_fullsized":"2023-10-21T16:02:01.919532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:45.186023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946147+0000","last_change":"2023-10-21T16:01:55.943291+0000","last_active":"2023-10-21T16:02:00.946147+0000","last_peered":"2023-10-21T16:02:00.946147+0000","last_clean":"2023-10-21T16:02:00.946147+0000","last_became_active":"2023-10-21T16:01:55.942937+0000","last_became_peered":"2023-10-21T16:01:55.942937+0000","last_unstale":"2023-10-21T16:02:00.946147+0000","last_undegraded":"2023-10-21T16:02:00.946147+0000","last_fullsized":"2023-10-21T16:02:00.946147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911689+0000","last_change":"2023-10-21T16:01:55.147810+0000","last_active":"2023-10-21T16:02:00.911689+0000","last_peered":"2023-10-21T16:02:00.911689+0000","last_clean":"2023-10-21T16:02:00.911689+0000","last_became_active":"2023-10-21T16:01:55.147532+0000","last_became_peered":"2023-10-21T16:01:55.147532+0000","last_unstale":"2023-10-21T16:02:00.911689+0000","last_undegraded":"2023-10-21T16:02:00.911689+0000","last_fullsized":"2023-10-21T16:02:00.911689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921974+0000","last_change":"2023-10-21T16:02:00.192496+0000","last_active":"2023-10-21T16:02:01.921974+0000","last_peered":"2023-10-21T16:02:01.921974+0000","last_clean":"2023-10-21T16:02:01.921974+0000","last_became_active":"2023-10-21T16:02:00.192090+0000","last_became_peered":"2023-10-21T16:02:00.192090+0000","last_unstale":"2023-10-21T16:02:01.921974+0000","last_undegraded":"2023-10-21T16:02:01.921974+0000","last_fullsized":"2023-10-21T16:02:01.921974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946330+0000","last_change":"2023-10-21T16:01:55.936744+0000","last_active":"2023-10-21T16:02:00.946330+0000","last_peered":"2023-10-21T16:02:00.946330+0000","last_clean":"2023-10-21T16:02:00.946330+0000","last_became_active":"2023-10-21T16:01:55.936451+0000","last_became_peered":"2023-10-21T16:01:55.936451+0000","last_unstale":"2023-10-21T16:02:00.946330+0000","last_undegraded":"2023-10-21T16:02:00.946330+0000","last_fullsized":"2023-10-21T16:02:00.946330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910754+0000","last_change":"2023-10-21T16:01:55.159913+0000","last_active":"2023-10-21T16:02:00.910754+0000","last_peered":"2023-10-21T16:02:00.910754+0000","last_clean":"2023-10-21T16:02:00.910754+0000","last_became_active":"2023-10-21T16:01:55.159480+0000","last_became_peered":"2023-10-21T16:01:55.159480+0000","last_unstale":"2023-10-21T16:02:00.910754+0000","last_undegraded":"2023-10-21T16:02:00.910754+0000","last_fullsized":"2023-10-21T16:02:00.910754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:41.390056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895508+0000","last_change":"2023-10-21T16:02:00.748804+0000","last_active":"2023-10-21T16:02:02.895508+0000","last_peered":"2023-10-21T16:02:02.895508+0000","last_clean":"2023-10-21T16:02:02.895508+0000","last_became_active":"2023-10-21T16:02:00.746585+0000","last_became_peered":"2023-10-21T16:02:00.746585+0000","last_unstale":"2023-10-21T16:02:02.895508+0000","last_undegraded":"2023-10-21T16:02:02.895508+0000","last_fullsized":"2023-10-21T16:02:02.895508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913021+0000","last_change":"2023-10-21T16:01:55.936909+0000","last_active":"2023-10-21T16:02:00.913021+0000","last_peered":"2023-10-21T16:02:00.913021+0000","last_clean":"2023-10-21T16:02:00.913021+0000","last_became_active":"2023-10-21T16:01:55.930874+0000","last_became_peered":"2023-10-21T16:01:55.930874+0000","last_unstale":"2023-10-21T16:02:00.913021+0000","last_undegraded":"2023-10-21T16:02:00.913021+0000","last_fullsized":"2023-10-21T16:02:00.913021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:19.306040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414418+0000","last_change":"2023-10-21T16:01:55.161120+0000","last_active":"2023-10-21T16:02:01.414418+0000","last_peered":"2023-10-21T16:02:01.414418+0000","last_clean":"2023-10-21T16:02:01.414418+0000","last_became_active":"2023-10-21T16:01:55.160853+0000","last_became_peered":"2023-10-21T16:01:55.160853+0000","last_unstale":"2023-10-21T16:02:01.414418+0000","last_undegraded":"2023-10-21T16:02:01.414418+0000","last_fullsized":"2023-10-21T16:02:01.414418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919323+0000","last_change":"2023-10-21T16:02:00.192783+0000","last_active":"2023-10-21T16:02:01.919323+0000","last_peered":"2023-10-21T16:02:01.919323+0000","last_clean":"2023-10-21T16:02:01.919323+0000","last_became_active":"2023-10-21T16:02:00.192399+0000","last_became_peered":"2023-10-21T16:02:00.192399+0000","last_unstale":"2023-10-21T16:02:01.919323+0000","last_undegraded":"2023-10-21T16:02:01.919323+0000","last_fullsized":"2023-10-21T16:02:01.919323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928409+0000","last_change":"2023-10-21T16:01:55.942303+0000","last_active":"2023-10-21T16:02:00.928409+0000","last_peered":"2023-10-21T16:02:00.928409+0000","last_clean":"2023-10-21T16:02:00.928409+0000","last_became_active":"2023-10-21T16:01:55.941848+0000","last_became_peered":"2023-10-21T16:01:55.941848+0000","last_unstale":"2023-10-21T16:02:00.928409+0000","last_undegraded":"2023-10-21T16:02:00.928409+0000","last_fullsized":"2023-10-21T16:02:00.928409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417420+0000","last_change":"2023-10-21T16:01:55.164579+0000","last_active":"2023-10-21T16:02:01.417420+0000","last_peered":"2023-10-21T16:02:01.417420+0000","last_clean":"2023-10-21T16:02:01.417420+0000","last_became_active":"2023-10-21T16:01:55.162946+0000","last_became_peered":"2023-10-21T16:01:55.162946+0000","last_unstale":"2023-10-21T16:02:01.417420+0000","last_undegraded":"2023-10-21T16:02:01.417420+0000","last_fullsized":"2023-10-21T16:02:01.417420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919836+0000","last_change":"2023-10-21T16:02:00.748601+0000","last_active":"2023-10-21T16:02:01.919836+0000","last_peered":"2023-10-21T16:02:01.919836+0000","last_clean":"2023-10-21T16:02:01.919836+0000","last_became_active":"2023-10-21T16:02:00.747592+0000","last_became_peered":"2023-10-21T16:02:00.747592+0000","last_unstale":"2023-10-21T16:02:01.919836+0000","last_undegraded":"2023-10-21T16:02:01.919836+0000","last_fullsized":"2023-10-21T16:02:01.919836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947881+0000","last_change":"2023-10-21T16:01:55.941119+0000","last_active":"2023-10-21T16:02:00.947881+0000","last_peered":"2023-10-21T16:02:00.947881+0000","last_clean":"2023-10-21T16:02:00.947881+0000","last_became_active":"2023-10-21T16:01:55.940145+0000","last_became_peered":"2023-10-21T16:01:55.940145+0000","last_unstale":"2023-10-21T16:02:00.947881+0000","last_undegraded":"2023-10-21T16:02:00.947881+0000","last_fullsized":"2023-10-21T16:02:00.947881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895162+0000","last_change":"2023-10-21T16:01:55.165800+0000","last_active":"2023-10-21T16:02:02.895162+0000","last_peered":"2023-10-21T16:02:02.895162+0000","last_clean":"2023-10-21T16:02:02.895162+0000","last_became_active":"2023-10-21T16:01:55.165500+0000","last_became_peered":"2023-10-21T16:01:55.165500+0000","last_unstale":"2023-10-21T16:02:02.895162+0000","last_undegraded":"2023-10-21T16:02:02.895162+0000","last_fullsized":"2023-10-21T16:02:02.895162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894525+0000","last_change":"2023-10-21T16:02:00.742519+0000","last_active":"2023-10-21T16:02:02.894525+0000","last_peered":"2023-10-21T16:02:02.894525+0000","last_clean":"2023-10-21T16:02:02.894525+0000","last_became_active":"2023-10-21T16:02:00.740937+0000","last_became_peered":"2023-10-21T16:02:00.740937+0000","last_unstale":"2023-10-21T16:02:02.894525+0000","last_undegraded":"2023-10-21T16:02:02.894525+0000","last_fullsized":"2023-10-21T16:02:02.894525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:22.482979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913147+0000","last_change":"2023-10-21T16:01:55.939913+0000","last_active":"2023-10-21T16:02:00.913147+0000","last_peered":"2023-10-21T16:02:00.913147+0000","last_clean":"2023-10-21T16:02:00.913147+0000","last_became_active":"2023-10-21T16:01:55.939419+0000","last_became_peered":"2023-10-21T16:01:55.939419+0000","last_unstale":"2023-10-21T16:02:00.913147+0000","last_undegraded":"2023-10-21T16:02:00.913147+0000","last_fullsized":"2023-10-21T16:02:00.913147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415967+0000","last_change":"2023-10-21T16:01:55.163977+0000","last_active":"2023-10-21T16:02:01.415967+0000","last_peered":"2023-10-21T16:02:01.415967+0000","last_clean":"2023-10-21T16:02:01.415967+0000","last_became_active":"2023-10-21T16:01:55.163739+0000","last_became_peered":"2023-10-21T16:01:55.163739+0000","last_unstale":"2023-10-21T16:02:01.415967+0000","last_undegraded":"2023-10-21T16:02:01.415967+0000","last_fullsized":"2023-10-21T16:02:01.415967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:17.767707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007711+0000","last_change":"2023-10-21T16:02:00.751077+0000","last_active":"2023-10-21T16:02:02.007711+0000","last_peered":"2023-10-21T16:02:02.007711+0000","last_clean":"2023-10-21T16:02:02.007711+0000","last_became_active":"2023-10-21T16:02:00.749997+0000","last_became_peered":"2023-10-21T16:02:00.749997+0000","last_unstale":"2023-10-21T16:02:02.007711+0000","last_undegraded":"2023-10-21T16:02:02.007711+0000","last_fullsized":"2023-10-21T16:02:02.007711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005010+0000","last_change":"2023-10-21T16:01:55.951660+0000","last_active":"2023-10-21T16:02:01.005010+0000","last_peered":"2023-10-21T16:02:01.005010+0000","last_clean":"2023-10-21T16:02:01.005010+0000","last_became_active":"2023-10-21T16:01:55.951279+0000","last_became_peered":"2023-10-21T16:01:55.951279+0000","last_unstale":"2023-10-21T16:02:01.005010+0000","last_undegraded":"2023-10-21T16:02:01.005010+0000","last_fullsized":"2023-10-21T16:02:01.005010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913993+0000","last_change":"2023-10-21T16:01:55.149562+0000","last_active":"2023-10-21T16:02:00.913993+0000","last_peered":"2023-10-21T16:02:00.913993+0000","last_clean":"2023-10-21T16:02:00.913993+0000","last_became_active":"2023-10-21T16:01:55.149274+0000","last_became_peered":"2023-10-21T16:01:55.149274+0000","last_unstale":"2023-10-21T16:02:00.913993+0000","last_undegraded":"2023-10-21T16:02:00.913993+0000","last_fullsized":"2023-10-21T16:02:00.913993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.892788+0000","last_change":"2023-10-21T16:02:00.189379+0000","last_active":"2023-10-21T16:02:02.892788+0000","last_peered":"2023-10-21T16:02:02.892788+0000","last_clean":"2023-10-21T16:02:02.892788+0000","last_became_active":"2023-10-21T16:02:00.188693+0000","last_became_peered":"2023-10-21T16:02:00.188693+0000","last_unstale":"2023-10-21T16:02:02.892788+0000","last_undegraded":"2023-10-21T16:02:02.892788+0000","last_fullsized":"2023-10-21T16:02:02.892788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913564+0000","last_change":"2023-10-21T16:01:55.941518+0000","last_active":"2023-10-21T16:02:00.913564+0000","last_peered":"2023-10-21T16:02:00.913564+0000","last_clean":"2023-10-21T16:02:00.913564+0000","last_became_active":"2023-10-21T16:01:55.938726+0000","last_became_peered":"2023-10-21T16:01:55.938726+0000","last_unstale":"2023-10-21T16:02:00.913564+0000","last_undegraded":"2023-10-21T16:02:00.913564+0000","last_fullsized":"2023-10-21T16:02:00.913564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:39.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004473+0000","last_change":"2023-10-21T16:01:55.165664+0000","last_active":"2023-10-21T16:02:01.004473+0000","last_peered":"2023-10-21T16:02:01.004473+0000","last_clean":"2023-10-21T16:02:01.004473+0000","last_became_active":"2023-10-21T16:01:55.165070+0000","last_became_peered":"2023-10-21T16:01:55.165070+0000","last_unstale":"2023-10-21T16:02:01.004473+0000","last_undegraded":"2023-10-21T16:02:01.004473+0000","last_fullsized":"2023-10-21T16:02:01.004473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:02.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893925+0000","last_change":"2023-10-21T16:02:00.186762+0000","last_active":"2023-10-21T16:02:02.893925+0000","last_peered":"2023-10-21T16:02:02.893925+0000","last_clean":"2023-10-21T16:02:02.893925+0000","last_became_active":"2023-10-21T16:02:00.185437+0000","last_became_peered":"2023-10-21T16:02:00.185437+0000","last_unstale":"2023-10-21T16:02:02.893925+0000","last_undegraded":"2023-10-21T16:02:02.893925+0000","last_fullsized":"2023-10-21T16:02:02.893925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911065+0000","last_change":"2023-10-21T16:01:55.935243+0000","last_active":"2023-10-21T16:02:00.911065+0000","last_peered":"2023-10-21T16:02:00.911065+0000","last_clean":"2023-10-21T16:02:00.911065+0000","last_became_active":"2023-10-21T16:01:55.934353+0000","last_became_peered":"2023-10-21T16:01:55.934353+0000","last_unstale":"2023-10-21T16:02:00.911065+0000","last_undegraded":"2023-10-21T16:02:00.911065+0000","last_fullsized":"2023-10-21T16:02:00.911065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914987+0000","last_change":"2023-10-21T16:01:55.147043+0000","last_active":"2023-10-21T16:02:00.914987+0000","last_peered":"2023-10-21T16:02:00.914987+0000","last_clean":"2023-10-21T16:02:00.914987+0000","last_became_active":"2023-10-21T16:01:55.146674+0000","last_became_peered":"2023-10-21T16:01:55.146674+0000","last_unstale":"2023-10-21T16:02:00.914987+0000","last_undegraded":"2023-10-21T16:02:00.914987+0000","last_fullsized":"2023-10-21T16:02:00.914987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:18.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895819+0000","last_change":"2023-10-21T16:02:00.191162+0000","last_active":"2023-10-21T16:02:02.895819+0000","last_peered":"2023-10-21T16:02:02.895819+0000","last_clean":"2023-10-21T16:02:02.895819+0000","last_became_active":"2023-10-21T16:02:00.186599+0000","last_became_peered":"2023-10-21T16:02:00.186599+0000","last_unstale":"2023-10-21T16:02:02.895819+0000","last_undegraded":"2023-10-21T16:02:02.895819+0000","last_fullsized":"2023-10-21T16:02:02.895819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.926918+0000","last_change":"2023-10-21T16:01:55.948611+0000","last_active":"2023-10-21T16:02:00.926918+0000","last_peered":"2023-10-21T16:02:00.926918+0000","last_clean":"2023-10-21T16:02:00.926918+0000","last_became_active":"2023-10-21T16:01:55.948172+0000","last_became_peered":"2023-10-21T16:01:55.948172+0000","last_unstale":"2023-10-21T16:02:00.926918+0000","last_undegraded":"2023-10-21T16:02:00.926918+0000","last_fullsized":"2023-10-21T16:02:00.926918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414533+0000","last_change":"2023-10-21T16:01:55.161709+0000","last_active":"2023-10-21T16:02:01.414533+0000","last_peered":"2023-10-21T16:02:01.414533+0000","last_clean":"2023-10-21T16:02:01.414533+0000","last_became_active":"2023-10-21T16:01:55.160426+0000","last_became_peered":"2023-10-21T16:01:55.160426+0000","last_unstale":"2023-10-21T16:02:01.414533+0000","last_undegraded":"2023-10-21T16:02:01.414533+0000","last_fullsized":"2023-10-21T16:02:01.414533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007971+0000","last_change":"2023-10-21T16:02:00.742783+0000","last_active":"2023-10-21T16:02:02.007971+0000","last_peered":"2023-10-21T16:02:02.007971+0000","last_clean":"2023-10-21T16:02:02.007971+0000","last_became_active":"2023-10-21T16:02:00.740932+0000","last_became_peered":"2023-10-21T16:02:00.740932+0000","last_unstale":"2023-10-21T16:02:02.007971+0000","last_undegraded":"2023-10-21T16:02:02.007971+0000","last_fullsized":"2023-10-21T16:02:02.007971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946103+0000","last_change":"2023-10-21T16:01:55.955491+0000","last_active":"2023-10-21T16:02:00.946103+0000","last_peered":"2023-10-21T16:02:00.946103+0000","last_clean":"2023-10-21T16:02:00.946103+0000","last_became_active":"2023-10-21T16:01:55.955163+0000","last_became_peered":"2023-10-21T16:01:55.955163+0000","last_unstale":"2023-10-21T16:02:00.946103+0000","last_undegraded":"2023-10-21T16:02:00.946103+0000","last_fullsized":"2023-10-21T16:02:00.946103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004498+0000","last_change":"2023-10-21T16:01:55.160793+0000","last_active":"2023-10-21T16:02:01.004498+0000","last_peered":"2023-10-21T16:02:01.004498+0000","last_clean":"2023-10-21T16:02:01.004498+0000","last_became_active":"2023-10-21T16:01:55.157151+0000","last_became_peered":"2023-10-21T16:01:55.157151+0000","last_unstale":"2023-10-21T16:02:01.004498+0000","last_undegraded":"2023-10-21T16:02:01.004498+0000","last_fullsized":"2023-10-21T16:02:01.004498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:36.886516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414663+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:02:01.414663+0000","last_peered":"2023-10-21T16:02:01.414663+0000","last_clean":"2023-10-21T16:02:01.414663+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:02:01.414663+0000","last_undegraded":"2023-10-21T16:02:01.414663+0000","last_fullsized":"2023-10-21T16:02:01.414663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895666+0000","last_change":"2023-10-21T16:02:00.752861+0000","last_active":"2023-10-21T16:02:02.895666+0000","last_peered":"2023-10-21T16:02:02.895666+0000","last_clean":"2023-10-21T16:02:02.895666+0000","last_became_active":"2023-10-21T16:02:00.748880+0000","last_became_peered":"2023-10-21T16:02:00.748880+0000","last_unstale":"2023-10-21T16:02:02.895666+0000","last_undegraded":"2023-10-21T16:02:02.895666+0000","last_fullsized":"2023-10-21T16:02:02.895666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415527+0000","last_change":"2023-10-21T16:01:55.954245+0000","last_active":"2023-10-21T16:02:01.415527+0000","last_peered":"2023-10-21T16:02:01.415527+0000","last_clean":"2023-10-21T16:02:01.415527+0000","last_became_active":"2023-10-21T16:01:55.953197+0000","last_became_peered":"2023-10-21T16:01:55.953197+0000","last_unstale":"2023-10-21T16:02:01.415527+0000","last_undegraded":"2023-10-21T16:02:01.415527+0000","last_fullsized":"2023-10-21T16:02:01.415527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915622+0000","last_change":"2023-10-21T16:01:55.168127+0000","last_active":"2023-10-21T16:02:00.915622+0000","last_peered":"2023-10-21T16:02:00.915622+0000","last_clean":"2023-10-21T16:02:00.915622+0000","last_became_active":"2023-10-21T16:01:55.167766+0000","last_became_peered":"2023-10-21T16:01:55.167766+0000","last_unstale":"2023-10-21T16:02:00.915622+0000","last_undegraded":"2023-10-21T16:02:00.915622+0000","last_fullsized":"2023-10-21T16:02:00.915622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:28:10.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893224+0000","last_change":"2023-10-21T16:02:00.742293+0000","last_active":"2023-10-21T16:02:02.893224+0000","last_peered":"2023-10-21T16:02:02.893224+0000","last_clean":"2023-10-21T16:02:02.893224+0000","last_became_active":"2023-10-21T16:02:00.741230+0000","last_became_peered":"2023-10-21T16:02:00.741230+0000","last_unstale":"2023-10-21T16:02:02.893224+0000","last_undegraded":"2023-10-21T16:02:02.893224+0000","last_fullsized":"2023-10-21T16:02:02.893224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005008+0000","last_change":"2023-10-21T16:01:55.948576+0000","last_active":"2023-10-21T16:02:01.005008+0000","last_peered":"2023-10-21T16:02:01.005008+0000","last_clean":"2023-10-21T16:02:01.005008+0000","last_became_active":"2023-10-21T16:01:55.947390+0000","last_became_peered":"2023-10-21T16:01:55.947390+0000","last_unstale":"2023-10-21T16:02:01.005008+0000","last_undegraded":"2023-10-21T16:02:01.005008+0000","last_fullsized":"2023-10-21T16:02:01.005008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914480+0000","last_change":"2023-10-21T16:01:55.151627+0000","last_active":"2023-10-21T16:02:00.914480+0000","last_peered":"2023-10-21T16:02:00.914480+0000","last_clean":"2023-10-21T16:02:00.914480+0000","last_became_active":"2023-10-21T16:01:55.151283+0000","last_became_peered":"2023-10-21T16:01:55.151283+0000","last_unstale":"2023-10-21T16:02:00.914480+0000","last_undegraded":"2023-10-21T16:02:00.914480+0000","last_fullsized":"2023-10-21T16:02:00.914480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007984+0000","last_change":"2023-10-21T16:02:00.183073+0000","last_active":"2023-10-21T16:02:02.007984+0000","last_peered":"2023-10-21T16:02:02.007984+0000","last_clean":"2023-10-21T16:02:02.007984+0000","last_became_active":"2023-10-21T16:02:00.182604+0000","last_became_peered":"2023-10-21T16:02:00.182604+0000","last_unstale":"2023-10-21T16:02:02.007984+0000","last_undegraded":"2023-10-21T16:02:02.007984+0000","last_fullsized":"2023-10-21T16:02:02.007984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416904+0000","last_change":"2023-10-21T16:01:55.951113+0000","last_active":"2023-10-21T16:02:01.416904+0000","last_peered":"2023-10-21T16:02:01.416904+0000","last_clean":"2023-10-21T16:02:01.416904+0000","last_became_active":"2023-10-21T16:01:55.950792+0000","last_became_peered":"2023-10-21T16:01:55.950792+0000","last_unstale":"2023-10-21T16:02:01.416904+0000","last_undegraded":"2023-10-21T16:02:01.416904+0000","last_fullsized":"2023-10-21T16:02:01.416904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915704+0000","last_change":"2023-10-21T16:01:55.149597+0000","last_active":"2023-10-21T16:02:00.915704+0000","last_peered":"2023-10-21T16:02:00.915704+0000","last_clean":"2023-10-21T16:02:00.915704+0000","last_became_active":"2023-10-21T16:01:55.149258+0000","last_became_peered":"2023-10-21T16:01:55.149258+0000","last_unstale":"2023-10-21T16:02:00.915704+0000","last_undegraded":"2023-10-21T16:02:00.915704+0000","last_fullsized":"2023-10-21T16:02:00.915704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.914975+0000","last_change":"2023-10-21T16:02:00.750068+0000","last_active":"2023-10-21T16:02:01.914975+0000","last_peered":"2023-10-21T16:02:01.914975+0000","last_clean":"2023-10-21T16:02:01.914975+0000","last_became_active":"2023-10-21T16:02:00.749055+0000","last_became_peered":"2023-10-21T16:02:00.749055+0000","last_unstale":"2023-10-21T16:02:01.914975+0000","last_undegraded":"2023-10-21T16:02:01.914975+0000","last_fullsized":"2023-10-21T16:02:01.914975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416138+0000","last_change":"2023-10-21T16:01:55.953586+0000","last_active":"2023-10-21T16:02:01.416138+0000","last_peered":"2023-10-21T16:02:01.416138+0000","last_clean":"2023-10-21T16:02:01.416138+0000","last_became_active":"2023-10-21T16:01:55.952550+0000","last_became_peered":"2023-10-21T16:01:55.952550+0000","last_unstale":"2023-10-21T16:02:01.416138+0000","last_undegraded":"2023-10-21T16:02:01.416138+0000","last_fullsized":"2023-10-21T16:02:01.416138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913456+0000","last_change":"2023-10-21T16:01:55.154502+0000","last_active":"2023-10-21T16:02:00.913456+0000","last_peered":"2023-10-21T16:02:00.913456+0000","last_clean":"2023-10-21T16:02:00.913456+0000","last_became_active":"2023-10-21T16:01:55.150688+0000","last_became_peered":"2023-10-21T16:01:55.150688+0000","last_unstale":"2023-10-21T16:02:00.913456+0000","last_undegraded":"2023-10-21T16:02:00.913456+0000","last_fullsized":"2023-10-21T16:02:00.913456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:32.709310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918774+0000","last_change":"2023-10-21T16:02:00.184524+0000","last_active":"2023-10-21T16:02:01.918774+0000","last_peered":"2023-10-21T16:02:01.918774+0000","last_clean":"2023-10-21T16:02:01.918774+0000","last_became_active":"2023-10-21T16:02:00.184081+0000","last_became_peered":"2023-10-21T16:02:00.184081+0000","last_unstale":"2023-10-21T16:02:01.918774+0000","last_undegraded":"2023-10-21T16:02:01.918774+0000","last_fullsized":"2023-10-21T16:02:01.918774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414960+0000","last_change":"2023-10-21T16:01:55.951983+0000","last_active":"2023-10-21T16:02:01.414960+0000","last_peered":"2023-10-21T16:02:01.414960+0000","last_clean":"2023-10-21T16:02:01.414960+0000","last_became_active":"2023-10-21T16:01:55.951633+0000","last_became_peered":"2023-10-21T16:01:55.951633+0000","last_unstale":"2023-10-21T16:02:01.414960+0000","last_undegraded":"2023-10-21T16:02:01.414960+0000","last_fullsized":"2023-10-21T16:02:01.414960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914399+0000","last_change":"2023-10-21T16:01:55.147150+0000","last_active":"2023-10-21T16:02:00.914399+0000","last_peered":"2023-10-21T16:02:00.914399+0000","last_clean":"2023-10-21T16:02:00.914399+0000","last_became_active":"2023-10-21T16:01:55.141230+0000","last_became_peered":"2023-10-21T16:01:55.141230+0000","last_unstale":"2023-10-21T16:02:00.914399+0000","last_undegraded":"2023-10-21T16:02:00.914399+0000","last_fullsized":"2023-10-21T16:02:00.914399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007865+0000","last_change":"2023-10-21T16:02:00.187875+0000","last_active":"2023-10-21T16:02:02.007865+0000","last_peered":"2023-10-21T16:02:02.007865+0000","last_clean":"2023-10-21T16:02:02.007865+0000","last_became_active":"2023-10-21T16:02:00.187427+0000","last_became_peered":"2023-10-21T16:02:00.187427+0000","last_unstale":"2023-10-21T16:02:02.007865+0000","last_undegraded":"2023-10-21T16:02:02.007865+0000","last_fullsized":"2023-10-21T16:02:02.007865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414077+0000","last_change":"2023-10-21T16:01:55.944005+0000","last_active":"2023-10-21T16:02:01.414077+0000","last_peered":"2023-10-21T16:02:01.414077+0000","last_clean":"2023-10-21T16:02:01.414077+0000","last_became_active":"2023-10-21T16:01:55.942887+0000","last_became_peered":"2023-10-21T16:01:55.942887+0000","last_unstale":"2023-10-21T16:02:01.414077+0000","last_undegraded":"2023-10-21T16:02:01.414077+0000","last_fullsized":"2023-10-21T16:02:01.414077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413985+0000","last_change":"2023-10-21T16:01:55.160115+0000","last_active":"2023-10-21T16:02:01.413985+0000","last_peered":"2023-10-21T16:02:01.413985+0000","last_clean":"2023-10-21T16:02:01.413985+0000","last_became_active":"2023-10-21T16:01:55.159788+0000","last_became_peered":"2023-10-21T16:01:55.159788+0000","last_unstale":"2023-10-21T16:02:01.413985+0000","last_undegraded":"2023-10-21T16:02:01.413985+0000","last_fullsized":"2023-10-21T16:02:01.413985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897390+0000","last_change":"2023-10-21T16:02:00.745714+0000","last_active":"2023-10-21T16:02:02.897390+0000","last_peered":"2023-10-21T16:02:02.897390+0000","last_clean":"2023-10-21T16:02:02.897390+0000","last_became_active":"2023-10-21T16:02:00.744779+0000","last_became_peered":"2023-10-21T16:02:00.744779+0000","last_unstale":"2023-10-21T16:02:02.897390+0000","last_undegraded":"2023-10-21T16:02:02.897390+0000","last_fullsized":"2023-10-21T16:02:02.897390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:53.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414998+0000","last_change":"2023-10-21T16:01:55.955734+0000","last_active":"2023-10-21T16:02:01.414998+0000","last_peered":"2023-10-21T16:02:01.414998+0000","last_clean":"2023-10-21T16:02:01.414998+0000","last_became_active":"2023-10-21T16:01:55.955333+0000","last_became_peered":"2023-10-21T16:01:55.955333+0000","last_unstale":"2023-10-21T16:02:01.414998+0000","last_undegraded":"2023-10-21T16:02:01.414998+0000","last_fullsized":"2023-10-21T16:02:01.414998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004475+0000","last_change":"2023-10-21T16:01:55.167450+0000","last_active":"2023-10-21T16:02:01.004475+0000","last_peered":"2023-10-21T16:02:01.004475+0000","last_clean":"2023-10-21T16:02:01.004475+0000","last_became_active":"2023-10-21T16:01:55.166857+0000","last_became_peered":"2023-10-21T16:01:55.166857+0000","last_unstale":"2023-10-21T16:02:01.004475+0000","last_undegraded":"2023-10-21T16:02:01.004475+0000","last_fullsized":"2023-10-21T16:02:01.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.896683+0000","last_change":"2023-10-21T16:02:00.742809+0000","last_active":"2023-10-21T16:02:02.896683+0000","last_peered":"2023-10-21T16:02:02.896683+0000","last_clean":"2023-10-21T16:02:02.896683+0000","last_became_active":"2023-10-21T16:02:00.741398+0000","last_became_peered":"2023-10-21T16:02:00.741398+0000","last_unstale":"2023-10-21T16:02:02.896683+0000","last_undegraded":"2023-10-21T16:02:02.896683+0000","last_fullsized":"2023-10-21T16:02:02.896683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:35:32.899715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927531+0000","last_change":"2023-10-21T16:01:55.937515+0000","last_active":"2023-10-21T16:02:00.927531+0000","last_peered":"2023-10-21T16:02:00.927531+0000","last_clean":"2023-10-21T16:02:00.927531+0000","last_became_active":"2023-10-21T16:01:55.937165+0000","last_became_peered":"2023-10-21T16:01:55.937165+0000","last_unstale":"2023-10-21T16:02:00.927531+0000","last_undegraded":"2023-10-21T16:02:00.927531+0000","last_fullsized":"2023-10-21T16:02:00.927531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928104+0000","last_change":"2023-10-21T16:01:55.151763+0000","last_active":"2023-10-21T16:02:00.928104+0000","last_peered":"2023-10-21T16:02:00.928104+0000","last_clean":"2023-10-21T16:02:00.928104+0000","last_became_active":"2023-10-21T16:01:55.151316+0000","last_became_peered":"2023-10-21T16:01:55.151316+0000","last_unstale":"2023-10-21T16:02:00.928104+0000","last_undegraded":"2023-10-21T16:02:00.928104+0000","last_fullsized":"2023-10-21T16:02:00.928104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.917604+0000","last_change":"2023-10-21T16:02:00.742437+0000","last_active":"2023-10-21T16:02:01.917604+0000","last_peered":"2023-10-21T16:02:01.917604+0000","last_clean":"2023-10-21T16:02:01.917604+0000","last_became_active":"2023-10-21T16:02:00.741212+0000","last_became_peered":"2023-10-21T16:02:00.741212+0000","last_unstale":"2023-10-21T16:02:01.917604+0000","last_undegraded":"2023-10-21T16:02:01.917604+0000","last_fullsized":"2023-10-21T16:02:01.917604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415373+0000","last_change":"2023-10-21T16:01:55.942841+0000","last_active":"2023-10-21T16:02:01.415373+0000","last_peered":"2023-10-21T16:02:01.415373+0000","last_clean":"2023-10-21T16:02:01.415373+0000","last_became_active":"2023-10-21T16:01:55.942471+0000","last_became_peered":"2023-10-21T16:01:55.942471+0000","last_unstale":"2023-10-21T16:02:01.415373+0000","last_undegraded":"2023-10-21T16:02:01.415373+0000","last_fullsized":"2023-10-21T16:02:01.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:14.233631+0000","last_change":"2023-10-21T16:01:55.156634+0000","last_active":"2023-10-21T16:02:14.233631+0000","last_peered":"2023-10-21T16:02:14.233631+0000","last_clean":"2023-10-21T16:02:14.233631+0000","last_became_active":"2023-10-21T16:01:55.156340+0000","last_became_peered":"2023-10-21T16:01:55.156340+0000","last_unstale":"2023-10-21T16:02:14.233631+0000","last_undegraded":"2023-10-21T16:02:14.233631+0000","last_fullsized":"2023-10-21T16:02:14.233631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893349+0000","last_change":"2023-10-21T16:02:00.189158+0000","last_active":"2023-10-21T16:02:02.893349+0000","last_peered":"2023-10-21T16:02:02.893349+0000","last_clean":"2023-10-21T16:02:02.893349+0000","last_became_active":"2023-10-21T16:02:00.188097+0000","last_became_peered":"2023-10-21T16:02:00.188097+0000","last_unstale":"2023-10-21T16:02:02.893349+0000","last_undegraded":"2023-10-21T16:02:02.893349+0000","last_fullsized":"2023-10-21T16:02:02.893349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914268+0000","last_change":"2023-10-21T16:01:55.944799+0000","last_active":"2023-10-21T16:02:00.914268+0000","last_peered":"2023-10-21T16:02:00.914268+0000","last_clean":"2023-10-21T16:02:00.914268+0000","last_became_active":"2023-10-21T16:01:55.944472+0000","last_became_peered":"2023-10-21T16:01:55.944472+0000","last_unstale":"2023-10-21T16:02:00.914268+0000","last_undegraded":"2023-10-21T16:02:00.914268+0000","last_fullsized":"2023-10-21T16:02:00.914268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:58.256813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914405+0000","last_change":"2023-10-21T16:01:55.162979+0000","last_active":"2023-10-21T16:02:00.914405+0000","last_peered":"2023-10-21T16:02:00.914405+0000","last_clean":"2023-10-21T16:02:00.914405+0000","last_became_active":"2023-10-21T16:01:55.162557+0000","last_became_peered":"2023-10-21T16:01:55.162557+0000","last_unstale":"2023-10-21T16:02:00.914405+0000","last_undegraded":"2023-10-21T16:02:00.914405+0000","last_fullsized":"2023-10-21T16:02:00.914405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921560+0000","last_change":"2023-10-21T16:02:00.188187+0000","last_active":"2023-10-21T16:02:01.921560+0000","last_peered":"2023-10-21T16:02:01.921560+0000","last_clean":"2023-10-21T16:02:01.921560+0000","last_became_active":"2023-10-21T16:02:00.187688+0000","last_became_peered":"2023-10-21T16:02:00.187688+0000","last_unstale":"2023-10-21T16:02:01.921560+0000","last_undegraded":"2023-10-21T16:02:01.921560+0000","last_fullsized":"2023-10-21T16:02:01.921560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928702+0000","last_change":"2023-10-21T16:01:55.954216+0000","last_active":"2023-10-21T16:02:00.928702+0000","last_peered":"2023-10-21T16:02:00.928702+0000","last_clean":"2023-10-21T16:02:00.928702+0000","last_became_active":"2023-10-21T16:01:55.953211+0000","last_became_peered":"2023-10-21T16:01:55.953211+0000","last_unstale":"2023-10-21T16:02:00.928702+0000","last_undegraded":"2023-10-21T16:02:00.928702+0000","last_fullsized":"2023-10-21T16:02:00.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005397+0000","last_change":"2023-10-21T16:01:55.169893+0000","last_active":"2023-10-21T16:02:01.005397+0000","last_peered":"2023-10-21T16:02:01.005397+0000","last_clean":"2023-10-21T16:02:01.005397+0000","last_became_active":"2023-10-21T16:01:55.169587+0000","last_became_peered":"2023-10-21T16:01:55.169587+0000","last_unstale":"2023-10-21T16:02:01.005397+0000","last_undegraded":"2023-10-21T16:02:01.005397+0000","last_fullsized":"2023-10-21T16:02:01.005397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894928+0000","last_change":"2023-10-21T16:02:00.185249+0000","last_active":"2023-10-21T16:02:02.894928+0000","last_peered":"2023-10-21T16:02:02.894928+0000","last_clean":"2023-10-21T16:02:02.894928+0000","last_became_active":"2023-10-21T16:02:00.184448+0000","last_became_peered":"2023-10-21T16:02:00.184448+0000","last_unstale":"2023-10-21T16:02:02.894928+0000","last_undegraded":"2023-10-21T16:02:02.894928+0000","last_fullsized":"2023-10-21T16:02:02.894928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:18:23.475051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005291+0000","last_change":"2023-10-21T16:01:55.954077+0000","last_active":"2023-10-21T16:02:01.005291+0000","last_peered":"2023-10-21T16:02:01.005291+0000","last_clean":"2023-10-21T16:02:01.005291+0000","last_became_active":"2023-10-21T16:01:55.951152+0000","last_became_peered":"2023-10-21T16:01:55.951152+0000","last_unstale":"2023-10-21T16:02:01.005291+0000","last_undegraded":"2023-10-21T16:02:01.005291+0000","last_fullsized":"2023-10-21T16:02:01.005291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417259+0000","last_change":"2023-10-21T16:01:55.171136+0000","last_active":"2023-10-21T16:02:01.417259+0000","last_peered":"2023-10-21T16:02:01.417259+0000","last_clean":"2023-10-21T16:02:01.417259+0000","last_became_active":"2023-10-21T16:01:55.170850+0000","last_became_peered":"2023-10-21T16:01:55.170850+0000","last_unstale":"2023-10-21T16:02:01.417259+0000","last_undegraded":"2023-10-21T16:02:01.417259+0000","last_fullsized":"2023-10-21T16:02:01.417259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895695+0000","last_change":"2023-10-21T16:02:00.753599+0000","last_active":"2023-10-21T16:02:02.895695+0000","last_peered":"2023-10-21T16:02:02.895695+0000","last_clean":"2023-10-21T16:02:02.895695+0000","last_became_active":"2023-10-21T16:02:00.751869+0000","last_became_peered":"2023-10-21T16:02:00.751869+0000","last_unstale":"2023-10-21T16:02:02.895695+0000","last_undegraded":"2023-10-21T16:02:02.895695+0000","last_fullsized":"2023-10-21T16:02:02.895695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:09.960025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007811+0000","last_change":"2023-10-21T16:02:00.750967+0000","last_active":"2023-10-21T16:02:02.007811+0000","last_peered":"2023-10-21T16:02:02.007811+0000","last_clean":"2023-10-21T16:02:02.007811+0000","last_became_active":"2023-10-21T16:02:00.749696+0000","last_became_peered":"2023-10-21T16:02:00.749696+0000","last_unstale":"2023-10-21T16:02:02.007811+0000","last_undegraded":"2023-10-21T16:02:02.007811+0000","last_fullsized":"2023-10-21T16:02:02.007811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910430+0000","last_change":"2023-10-21T16:01:55.938421+0000","last_active":"2023-10-21T16:02:00.910430+0000","last_peered":"2023-10-21T16:02:00.910430+0000","last_clean":"2023-10-21T16:02:00.910430+0000","last_became_active":"2023-10-21T16:01:55.938105+0000","last_became_peered":"2023-10-21T16:01:55.938105+0000","last_unstale":"2023-10-21T16:02:00.910430+0000","last_undegraded":"2023-10-21T16:02:00.910430+0000","last_fullsized":"2023-10-21T16:02:00.910430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004053+0000","last_change":"2023-10-21T16:01:55.166389+0000","last_active":"2023-10-21T16:02:01.004053+0000","last_peered":"2023-10-21T16:02:01.004053+0000","last_clean":"2023-10-21T16:02:01.004053+0000","last_became_active":"2023-10-21T16:01:55.166065+0000","last_became_peered":"2023-10-21T16:01:55.166065+0000","last_unstale":"2023-10-21T16:02:01.004053+0000","last_undegraded":"2023-10-21T16:02:01.004053+0000","last_fullsized":"2023-10-21T16:02:01.004053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895651+0000","last_change":"2023-10-21T16:02:00.195541+0000","last_active":"2023-10-21T16:02:02.895651+0000","last_peered":"2023-10-21T16:02:02.895651+0000","last_clean":"2023-10-21T16:02:02.895651+0000","last_became_active":"2023-10-21T16:02:00.195020+0000","last_became_peered":"2023-10-21T16:02:00.195020+0000","last_unstale":"2023-10-21T16:02:02.895651+0000","last_undegraded":"2023-10-21T16:02:02.895651+0000","last_fullsized":"2023-10-21T16:02:02.895651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946124+0000","last_change":"2023-10-21T16:01:55.932516+0000","last_active":"2023-10-21T16:02:00.946124+0000","last_peered":"2023-10-21T16:02:00.946124+0000","last_clean":"2023-10-21T16:02:00.946124+0000","last_became_active":"2023-10-21T16:01:55.932175+0000","last_became_peered":"2023-10-21T16:01:55.932175+0000","last_unstale":"2023-10-21T16:02:00.946124+0000","last_undegraded":"2023-10-21T16:02:00.946124+0000","last_fullsized":"2023-10-21T16:02:00.946124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945962+0000","last_change":"2023-10-21T16:01:55.156263+0000","last_active":"2023-10-21T16:02:00.945962+0000","last_peered":"2023-10-21T16:02:00.945962+0000","last_clean":"2023-10-21T16:02:00.945962+0000","last_became_active":"2023-10-21T16:01:55.155789+0000","last_became_peered":"2023-10-21T16:01:55.155789+0000","last_unstale":"2023-10-21T16:02:00.945962+0000","last_undegraded":"2023-10-21T16:02:00.945962+0000","last_fullsized":"2023-10-21T16:02:00.945962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:52.609080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415216+0000","last_change":"2023-10-21T16:01:55.160677+0000","last_active":"2023-10-21T16:02:01.415216+0000","last_peered":"2023-10-21T16:02:01.415216+0000","last_clean":"2023-10-21T16:02:01.415216+0000","last_became_active":"2023-10-21T16:01:55.160265+0000","last_became_peered":"2023-10-21T16:01:55.160265+0000","last_unstale":"2023-10-21T16:02:01.415216+0000","last_undegraded":"2023-10-21T16:02:01.415216+0000","last_fullsized":"2023-10-21T16:02:01.415216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928119+0000","last_change":"2023-10-21T16:01:55.947536+0000","last_active":"2023-10-21T16:02:00.928119+0000","last_peered":"2023-10-21T16:02:00.928119+0000","last_clean":"2023-10-21T16:02:00.928119+0000","last_became_active":"2023-10-21T16:01:55.947243+0000","last_became_peered":"2023-10-21T16:01:55.947243+0000","last_unstale":"2023-10-21T16:02:00.928119+0000","last_undegraded":"2023-10-21T16:02:00.928119+0000","last_fullsized":"2023-10-21T16:02:00.928119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.916131+0000","last_change":"2023-10-21T16:02:00.181884+0000","last_active":"2023-10-21T16:02:01.916131+0000","last_peered":"2023-10-21T16:02:01.916131+0000","last_clean":"2023-10-21T16:02:01.916131+0000","last_became_active":"2023-10-21T16:02:00.179404+0000","last_became_peered":"2023-10-21T16:02:00.179404+0000","last_unstale":"2023-10-21T16:02:01.916131+0000","last_undegraded":"2023-10-21T16:02:01.916131+0000","last_fullsized":"2023-10-21T16:02:01.916131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:42.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946429+0000","last_change":"2023-10-21T16:01:55.160559+0000","last_active":"2023-10-21T16:02:00.946429+0000","last_peered":"2023-10-21T16:02:00.946429+0000","last_clean":"2023-10-21T16:02:00.946429+0000","last_became_active":"2023-10-21T16:01:55.160241+0000","last_became_peered":"2023-10-21T16:01:55.160241+0000","last_unstale":"2023-10-21T16:02:00.946429+0000","last_undegraded":"2023-10-21T16:02:00.946429+0000","last_fullsized":"2023-10-21T16:02:00.946429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913498+0000","last_change":"2023-10-21T16:01:55.939951+0000","last_active":"2023-10-21T16:02:00.913498+0000","last_peered":"2023-10-21T16:02:00.913498+0000","last_clean":"2023-10-21T16:02:00.913498+0000","last_became_active":"2023-10-21T16:01:55.939552+0000","last_became_peered":"2023-10-21T16:01:55.939552+0000","last_unstale":"2023-10-21T16:02:00.913498+0000","last_undegraded":"2023-10-21T16:02:00.913498+0000","last_fullsized":"2023-10-21T16:02:00.913498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:38.621663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897399+0000","last_change":"2023-10-21T16:02:00.182272+0000","last_active":"2023-10-21T16:02:02.897399+0000","last_peered":"2023-10-21T16:02:02.897399+0000","last_clean":"2023-10-21T16:02:02.897399+0000","last_became_active":"2023-10-21T16:02:00.181705+0000","last_became_peered":"2023-10-21T16:02:00.181705+0000","last_unstale":"2023-10-21T16:02:02.897399+0000","last_undegraded":"2023-10-21T16:02:02.897399+0000","last_fullsized":"2023-10-21T16:02:02.897399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:28.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413688+0000","last_change":"2023-10-21T16:01:55.943801+0000","last_active":"2023-10-21T16:02:01.413688+0000","last_peered":"2023-10-21T16:02:01.413688+0000","last_clean":"2023-10-21T16:02:01.413688+0000","last_became_active":"2023-10-21T16:01:55.942546+0000","last_became_peered":"2023-10-21T16:01:55.942546+0000","last_unstale":"2023-10-21T16:02:01.413688+0000","last_undegraded":"2023-10-21T16:02:01.413688+0000","last_fullsized":"2023-10-21T16:02:01.413688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915001+0000","last_change":"2023-10-21T16:01:55.953970+0000","last_active":"2023-10-21T16:02:00.915001+0000","last_peered":"2023-10-21T16:02:00.915001+0000","last_clean":"2023-10-21T16:02:00.915001+0000","last_became_active":"2023-10-21T16:01:55.953164+0000","last_became_peered":"2023-10-21T16:01:55.953164+0000","last_unstale":"2023-10-21T16:02:00.915001+0000","last_undegraded":"2023-10-21T16:02:00.915001+0000","last_fullsized":"2023-10-21T16:02:00.915001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:16.452088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004198+0000","last_change":"2023-10-21T16:01:55.948469+0000","last_active":"2023-10-21T16:02:01.004198+0000","last_peered":"2023-10-21T16:02:01.004198+0000","last_clean":"2023-10-21T16:02:01.004198+0000","last_became_active":"2023-10-21T16:01:55.948198+0000","last_became_peered":"2023-10-21T16:01:55.948198+0000","last_unstale":"2023-10-21T16:02:01.004198+0000","last_undegraded":"2023-10-21T16:02:01.004198+0000","last_fullsized":"2023-10-21T16:02:01.004198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927645+0000","last_change":"2023-10-21T16:01:55.933799+0000","last_active":"2023-10-21T16:02:00.927645+0000","last_peered":"2023-10-21T16:02:00.927645+0000","last_clean":"2023-10-21T16:02:00.927645+0000","last_became_active":"2023-10-21T16:01:55.933431+0000","last_became_peered":"2023-10-21T16:01:55.933431+0000","last_unstale":"2023-10-21T16:02:00.927645+0000","last_undegraded":"2023-10-21T16:02:00.927645+0000","last_fullsized":"2023-10-21T16:02:00.927645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947274+0000","last_change":"2023-10-21T16:01:55.925050+0000","last_active":"2023-10-21T16:02:00.947274+0000","last_peered":"2023-10-21T16:02:00.947274+0000","last_clean":"2023-10-21T16:02:00.947274+0000","last_became_active":"2023-10-21T16:01:55.924510+0000","last_became_peered":"2023-10-21T16:01:55.924510+0000","last_unstale":"2023-10-21T16:02:00.947274+0000","last_undegraded":"2023-10-21T16:02:00.947274+0000","last_fullsized":"2023-10-21T16:02:00.947274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913921+0000","last_change":"2023-10-21T16:01:55.939109+0000","last_active":"2023-10-21T16:02:00.913921+0000","last_peered":"2023-10-21T16:02:00.913921+0000","last_clean":"2023-10-21T16:02:00.913921+0000","last_became_active":"2023-10-21T16:01:55.938852+0000","last_became_peered":"2023-10-21T16:01:55.938852+0000","last_unstale":"2023-10-21T16:02:00.913921+0000","last_undegraded":"2023-10-21T16:02:00.913921+0000","last_fullsized":"2023-10-21T16:02:00.913921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416735+0000","last_change":"2023-10-21T16:01:55.939268+0000","last_active":"2023-10-21T16:02:01.416735+0000","last_peered":"2023-10-21T16:02:01.416735+0000","last_clean":"2023-10-21T16:02:01.416735+0000","last_became_active":"2023-10-21T16:01:55.938991+0000","last_became_peered":"2023-10-21T16:01:55.938991+0000","last_unstale":"2023-10-21T16:02:01.416735+0000","last_undegraded":"2023-10-21T16:02:01.416735+0000","last_fullsized":"2023-10-21T16:02:01.416735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947882+0000","last_change":"2023-10-21T16:01:55.930486+0000","last_active":"2023-10-21T16:02:00.947882+0000","last_peered":"2023-10-21T16:02:00.947882+0000","last_clean":"2023-10-21T16:02:00.947882+0000","last_became_active":"2023-10-21T16:01:55.930187+0000","last_became_peered":"2023-10-21T16:01:55.930187+0000","last_unstale":"2023-10-21T16:02:00.947882+0000","last_undegraded":"2023-10-21T16:02:00.947882+0000","last_fullsized":"2023-10-21T16:02:00.947882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003895+0000","last_change":"2023-10-21T16:01:55.943056+0000","last_active":"2023-10-21T16:02:01.003895+0000","last_peered":"2023-10-21T16:02:01.003895+0000","last_clean":"2023-10-21T16:02:01.003895+0000","last_became_active":"2023-10-21T16:01:55.942724+0000","last_became_peered":"2023-10-21T16:01:55.942724+0000","last_unstale":"2023-10-21T16:02:01.003895+0000","last_undegraded":"2023-10-21T16:02:01.003895+0000","last_fullsized":"2023-10-21T16:02:01.003895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946201+0000","last_change":"2023-10-21T16:01:55.922812+0000","last_active":"2023-10-21T16:02:00.946201+0000","last_peered":"2023-10-21T16:02:00.946201+0000","last_clean":"2023-10-21T16:02:00.946201+0000","last_became_active":"2023-10-21T16:01:55.922134+0000","last_became_peered":"2023-10-21T16:01:55.922134+0000","last_unstale":"2023-10-21T16:02:00.946201+0000","last_undegraded":"2023-10-21T16:02:00.946201+0000","last_fullsized":"2023-10-21T16:02:00.946201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004186+0000","last_change":"2023-10-21T16:01:55.954470+0000","last_active":"2023-10-21T16:02:01.004186+0000","last_peered":"2023-10-21T16:02:01.004186+0000","last_clean":"2023-10-21T16:02:01.004186+0000","last_became_active":"2023-10-21T16:01:55.954107+0000","last_became_peered":"2023-10-21T16:01:55.954107+0000","last_unstale":"2023-10-21T16:02:01.004186+0000","last_undegraded":"2023-10-21T16:02:01.004186+0000","last_fullsized":"2023-10-21T16:02:01.004186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927404+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.927404+0000","last_peered":"2023-10-21T16:02:00.927404+0000","last_clean":"2023-10-21T16:02:00.927404+0000","last_became_active":"2023-10-21T16:01:55.946552+0000","last_became_peered":"2023-10-21T16:01:55.946552+0000","last_unstale":"2023-10-21T16:02:00.927404+0000","last_undegraded":"2023-10-21T16:02:00.927404+0000","last_fullsized":"2023-10-21T16:02:00.927404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416045+0000","last_change":"2023-10-21T16:01:55.947979+0000","last_active":"2023-10-21T16:02:01.416045+0000","last_peered":"2023-10-21T16:02:01.416045+0000","last_clean":"2023-10-21T16:02:01.416045+0000","last_became_active":"2023-10-21T16:01:55.947572+0000","last_became_peered":"2023-10-21T16:01:55.947572+0000","last_unstale":"2023-10-21T16:02:01.416045+0000","last_undegraded":"2023-10-21T16:02:01.416045+0000","last_fullsized":"2023-10-21T16:02:01.416045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415776+0000","last_change":"2023-10-21T16:01:55.946625+0000","last_active":"2023-10-21T16:02:01.415776+0000","last_peered":"2023-10-21T16:02:01.415776+0000","last_clean":"2023-10-21T16:02:01.415776+0000","last_became_active":"2023-10-21T16:01:55.945299+0000","last_became_peered":"2023-10-21T16:01:55.945299+0000","last_unstale":"2023-10-21T16:02:01.415776+0000","last_undegraded":"2023-10-21T16:02:01.415776+0000","last_fullsized":"2023-10-21T16:02:01.415776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915640+0000","last_change":"2023-10-21T16:01:55.938698+0000","last_active":"2023-10-21T16:02:00.915640+0000","last_peered":"2023-10-21T16:02:00.915640+0000","last_clean":"2023-10-21T16:02:00.915640+0000","last_became_active":"2023-10-21T16:01:55.938414+0000","last_became_peered":"2023-10-21T16:01:55.938414+0000","last_unstale":"2023-10-21T16:02:00.915640+0000","last_undegraded":"2023-10-21T16:02:00.915640+0000","last_fullsized":"2023-10-21T16:02:00.915640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:51.423697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005523+0000","last_change":"2023-10-21T16:01:55.953465+0000","last_active":"2023-10-21T16:02:01.005523+0000","last_peered":"2023-10-21T16:02:01.005523+0000","last_clean":"2023-10-21T16:02:01.005523+0000","last_became_active":"2023-10-21T16:01:55.952044+0000","last_became_peered":"2023-10-21T16:01:55.952044+0000","last_unstale":"2023-10-21T16:02:01.005523+0000","last_undegraded":"2023-10-21T16:02:01.005523+0000","last_fullsized":"2023-10-21T16:02:01.005523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003802+0000","last_change":"2023-10-21T16:01:55.943713+0000","last_active":"2023-10-21T16:02:01.003802+0000","last_peered":"2023-10-21T16:02:01.003802+0000","last_clean":"2023-10-21T16:02:01.003802+0000","last_became_active":"2023-10-21T16:01:55.939815+0000","last_became_peered":"2023-10-21T16:01:55.939815+0000","last_unstale":"2023-10-21T16:02:01.003802+0000","last_undegraded":"2023-10-21T16:02:01.003802+0000","last_fullsized":"2023-10-21T16:02:01.003802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910545+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.910545+0000","last_peered":"2023-10-21T16:02:00.910545+0000","last_clean":"2023-10-21T16:02:00.910545+0000","last_became_active":"2023-10-21T16:01:55.946640+0000","last_became_peered":"2023-10-21T16:01:55.946640+0000","last_unstale":"2023-10-21T16:02:00.910545+0000","last_undegraded":"2023-10-21T16:02:00.910545+0000","last_fullsized":"2023-10-21T16:02:00.910545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927490+0000","last_change":"2023-10-21T16:01:55.944028+0000","last_active":"2023-10-21T16:02:00.927490+0000","last_peered":"2023-10-21T16:02:00.927490+0000","last_clean":"2023-10-21T16:02:00.927490+0000","last_became_active":"2023-10-21T16:01:55.940832+0000","last_became_peered":"2023-10-21T16:01:55.940832+0000","last_unstale":"2023-10-21T16:02:00.927490+0000","last_undegraded":"2023-10-21T16:02:00.927490+0000","last_fullsized":"2023-10-21T16:02:00.927490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:42.589308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946064+0000","last_change":"2023-10-21T16:01:55.939040+0000","last_active":"2023-10-21T16:02:00.946064+0000","last_peered":"2023-10-21T16:02:00.946064+0000","last_clean":"2023-10-21T16:02:00.946064+0000","last_became_active":"2023-10-21T16:01:55.936534+0000","last_became_peered":"2023-10-21T16:01:55.936534+0000","last_unstale":"2023-10-21T16:02:00.946064+0000","last_undegraded":"2023-10-21T16:02:00.946064+0000","last_fullsized":"2023-10-21T16:02:00.946064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914180+0000","last_change":"2023-10-21T16:01:55.938149+0000","last_active":"2023-10-21T16:02:00.914180+0000","last_peered":"2023-10-21T16:02:00.914180+0000","last_clean":"2023-10-21T16:02:00.914180+0000","last_became_active":"2023-10-21T16:01:55.937906+0000","last_became_peered":"2023-10-21T16:01:55.937906+0000","last_unstale":"2023-10-21T16:02:00.914180+0000","last_undegraded":"2023-10-21T16:02:00.914180+0000","last_fullsized":"2023-10-21T16:02:00.914180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:19.855104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416487+0000","last_change":"2023-10-21T16:01:55.953883+0000","last_active":"2023-10-21T16:02:01.416487+0000","last_peered":"2023-10-21T16:02:01.416487+0000","last_clean":"2023-10-21T16:02:01.416487+0000","last_became_active":"2023-10-21T16:01:55.953252+0000","last_became_peered":"2023-10-21T16:01:55.953252+0000","last_unstale":"2023-10-21T16:02:01.416487+0000","last_undegraded":"2023-10-21T16:02:01.416487+0000","last_fullsized":"2023-10-21T16:02:01.416487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416712+0000","last_change":"2023-10-21T16:01:55.955305+0000","last_active":"2023-10-21T16:02:01.416712+0000","last_peered":"2023-10-21T16:02:01.416712+0000","last_clean":"2023-10-21T16:02:01.416712+0000","last_became_active":"2023-10-21T16:01:55.955019+0000","last_became_peered":"2023-10-21T16:01:55.955019+0000","last_unstale":"2023-10-21T16:02:01.416712+0000","last_undegraded":"2023-10-21T16:02:01.416712+0000","last_fullsized":"2023-10-21T16:02:01.416712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416673+0000","last_change":"2023-10-21T16:01:55.958446+0000","last_active":"2023-10-21T16:02:01.416673+0000","last_peered":"2023-10-21T16:02:01.416673+0000","last_clean":"2023-10-21T16:02:01.416673+0000","last_became_active":"2023-10-21T16:01:55.958056+0000","last_became_peered":"2023-10-21T16:01:55.958056+0000","last_unstale":"2023-10-21T16:02:01.416673+0000","last_undegraded":"2023-10-21T16:02:01.416673+0000","last_fullsized":"2023-10-21T16:02:01.416673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945856+0000","last_change":"2023-10-21T16:01:55.942133+0000","last_active":"2023-10-21T16:02:00.945856+0000","last_peered":"2023-10-21T16:02:00.945856+0000","last_clean":"2023-10-21T16:02:00.945856+0000","last_became_active":"2023-10-21T16:01:55.941833+0000","last_became_peered":"2023-10-21T16:01:55.941833+0000","last_unstale":"2023-10-21T16:02:00.945856+0000","last_undegraded":"2023-10-21T16:02:00.945856+0000","last_fullsized":"2023-10-21T16:02:00.945856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414741+0000","last_change":"2023-10-21T16:01:55.940356+0000","last_active":"2023-10-21T16:02:01.414741+0000","last_peered":"2023-10-21T16:02:01.414741+0000","last_clean":"2023-10-21T16:02:01.414741+0000","last_became_active":"2023-10-21T16:01:55.939602+0000","last_became_peered":"2023-10-21T16:01:55.939602+0000","last_unstale":"2023-10-21T16:02:01.414741+0000","last_undegraded":"2023-10-21T16:02:01.414741+0000","last_fullsized":"2023-10-21T16:02:01.414741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004002+0000","last_change":"2023-10-21T16:01:55.952103+0000","last_active":"2023-10-21T16:02:01.004002+0000","last_peered":"2023-10-21T16:02:01.004002+0000","last_clean":"2023-10-21T16:02:01.004002+0000","last_became_active":"2023-10-21T16:01:55.951051+0000","last_became_peered":"2023-10-21T16:01:55.951051+0000","last_unstale":"2023-10-21T16:02:01.004002+0000","last_undegraded":"2023-10-21T16:02:01.004002+0000","last_fullsized":"2023-10-21T16:02:01.004002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414699+0000","last_change":"2023-10-21T16:01:55.940974+0000","last_active":"2023-10-21T16:02:01.414699+0000","last_peered":"2023-10-21T16:02:01.414699+0000","last_clean":"2023-10-21T16:02:01.414699+0000","last_became_active":"2023-10-21T16:01:55.940667+0000","last_became_peered":"2023-10-21T16:01:55.940667+0000","last_unstale":"2023-10-21T16:02:01.414699+0000","last_undegraded":"2023-10-21T16:02:01.414699+0000","last_fullsized":"2023-10-21T16:02:01.414699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004788+0000","last_change":"2023-10-21T16:01:55.954831+0000","last_active":"2023-10-21T16:02:01.004788+0000","last_peered":"2023-10-21T16:02:01.004788+0000","last_clean":"2023-10-21T16:02:01.004788+0000","last_became_active":"2023-10-21T16:01:55.954453+0000","last_became_peered":"2023-10-21T16:01:55.954453+0000","last_unstale":"2023-10-21T16:02:01.004788+0000","last_undegraded":"2023-10-21T16:02:01.004788+0000","last_fullsized":"2023-10-21T16:02:01.004788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928170+0000","last_change":"2023-10-21T16:01:55.936408+0000","last_active":"2023-10-21T16:02:00.928170+0000","last_peered":"2023-10-21T16:02:00.928170+0000","last_clean":"2023-10-21T16:02:00.928170+0000","last_became_active":"2023-10-21T16:01:55.935946+0000","last_became_peered":"2023-10-21T16:01:55.935946+0000","last_unstale":"2023-10-21T16:02:00.928170+0000","last_undegraded":"2023-10-21T16:02:00.928170+0000","last_fullsized":"2023-10-21T16:02:00.928170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913325+0000","last_change":"2023-10-21T16:01:55.938274+0000","last_active":"2023-10-21T16:02:00.913325+0000","last_peered":"2023-10-21T16:02:00.913325+0000","last_clean":"2023-10-21T16:02:00.913325+0000","last_became_active":"2023-10-21T16:01:55.937878+0000","last_became_peered":"2023-10-21T16:01:55.937878+0000","last_unstale":"2023-10-21T16:02:00.913325+0000","last_undegraded":"2023-10-21T16:02:00.913325+0000","last_fullsized":"2023-10-21T16:02:00.913325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416466+0000","last_change":"2023-10-21T16:01:55.945925+0000","last_active":"2023-10-21T16:02:01.416466+0000","last_peered":"2023-10-21T16:02:01.416466+0000","last_clean":"2023-10-21T16:02:01.416466+0000","last_became_active":"2023-10-21T16:01:55.945457+0000","last_became_peered":"2023-10-21T16:01:55.945457+0000","last_unstale":"2023-10-21T16:02:01.416466+0000","last_undegraded":"2023-10-21T16:02:01.416466+0000","last_fullsized":"2023-10-21T16:02:01.416466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":702466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0219999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.258}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8260000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":111642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":111516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":111676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":702522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:02:24.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:02:24.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.730+0000 7f15b894e640 1 Processor -- start 2023-10-21T16:02:24.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.730+0000 7f15b894e640 1 -- start start 2023-10-21T16:02:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b406a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f15b40cb880 0x7f15b40b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15b40cbce0 0x7f15b40b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40ce550 con 0x7f15b40cbce0 2023-10-21T16:02:24.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b41609d0 con 0x7f15b40cb880 2023-10-21T16:02:24.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.731+0000 7f15b894e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40cdcf0 con 0x7f15b4069d50 2023-10-21T16:02:24.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15b40cbce0 0x7f15b40b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15b40cbce0 0x7f15b40b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59400/0 (socket says 172.21.15.53:59400) 2023-10-21T16:02:24.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 -- 172.21.15.53:0/3853847323 learned_addr learned my addr 172.21.15.53:0/3853847323 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:24.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b259c640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b406a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b1d9b640 1 --2- 172.21.15.53:0/3853847323 >> v2:172.21.15.124:3300/0 conn(0x7f15b40cb880 0x7f15b40b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 -- 172.21.15.53:0/3853847323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 msgr2=0x7f15b406a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:24.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b406a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 -- 172.21.15.53:0/3853847323 >> v2:172.21.15.124:3300/0 conn(0x7f15b40cb880 msgr2=0x7f15b40b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:24.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 --2- 172.21.15.53:0/3853847323 >> v2:172.21.15.124:3300/0 conn(0x7f15b40cb880 0x7f15b40b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.732+0000 7f15b2d9d640 1 -- 172.21.15.53:0/3853847323 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b406a670 con 0x7f15b40cbce0 2023-10-21T16:02:24.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.733+0000 7f15b2d9d640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cbce0 0x7f15b40b1850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15b406d000 tx=0x7f15ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67da490eb8b837a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.734+0000 7f15b159a640 1 -- 172.21.15.53:0/3853847323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac031870 con 0x7f15b40cbce0 2023-10-21T16:02:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.734+0000 7f15b159a640 1 -- 172.21.15.53:0/3853847323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac0319d0 con 0x7f15b40cbce0 2023-10-21T16:02:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.734+0000 7f15b159a640 1 -- 172.21.15.53:0/3853847323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac031d60 con 0x7f15b40cbce0 2023-10-21T16:02:24.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.734+0000 7f15b894e640 1 -- 172.21.15.53:0/3853847323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cbce0 msgr2=0x7f15b40b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.734+0000 7f15b894e640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cbce0 0x7f15b40b1850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f15b406d000 tx=0x7f15ac02b910 comp rx=0 tx=0).stop 2023-10-21T16:02:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 -- 172.21.15.53:0/3853847323 shutdown_connections 2023-10-21T16:02:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cbce0 0x7f15b40b1850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 --2- 172.21.15.53:0/3853847323 >> v2:172.21.15.124:3300/0 conn(0x7f15b40cb880 0x7f15b40b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 --2- 172.21.15.53:0/3853847323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b406a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 -- 172.21.15.53:0/3853847323 >> 172.21.15.53:0/3853847323 conn(0x7f15b406d450 msgr2=0x7f15b406d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 -- 172.21.15.53:0/3853847323 shutdown_connections 2023-10-21T16:02:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.735+0000 7f15b894e640 1 -- 172.21.15.53:0/3853847323 wait complete. 2023-10-21T16:02:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.737+0000 7f15b894e640 1 Processor -- start 2023-10-21T16:02:24.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.737+0000 7f15b894e640 1 -- start start 2023-10-21T16:02:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b40cbce0 0x7f15b40abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41592d0 con 0x7f15b40cb880 2023-10-21T16:02:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40acd50 con 0x7f15b40cbce0 2023-10-21T16:02:24.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b894e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40ac4b0 con 0x7f15b4069d50 2023-10-21T16:02:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b2d9d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b40cbce0 0x7f15b40abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59408/0 (socket says 172.21.15.53:59408) 2023-10-21T16:02:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57792/0 (socket says 172.21.15.53:57792) 2023-10-21T16:02:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.738+0000 7f15b1d9b640 1 -- 172.21.15.53:0/2329358523 learned_addr learned my addr 172.21.15.53:0/2329358523 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:24.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 -- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 msgr2=0x7f15b40ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b40ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 -- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b40cbce0 msgr2=0x7f15b40abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b40cbce0 0x7f15b40abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 -- 172.21.15.53:0/2329358523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0094d0 con 0x7f15b40cb880 2023-10-21T16:02:24.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.739+0000 7f15b1d9b640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f159c00d5f0 tx=0x7f159c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.740+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0125f0 con 0x7f15b40cb880 2023-10-21T16:02:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.740+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c012750 con 0x7f15b40cb880 2023-10-21T16:02:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.740+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c012b00 con 0x7f15b40cb880 2023-10-21T16:02:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.740+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b40ac790 con 0x7f15b40cb880 2023-10-21T16:02:24.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.740+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b415d360 con 0x7f15b40cb880 2023-10-21T16:02:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.741+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b415d520 con 0x7f15b40cb880 2023-10-21T16:02:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.743+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f159c012c60 con 0x7f15b40cb880 2023-10-21T16:02:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.743+0000 7f15aaffd640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 0x7f159406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:24.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.744+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f159c089cc0 con 0x7f15b40cb880 2023-10-21T16:02:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.744+0000 7f15b259c640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 0x7f159406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:24.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.745+0000 7f15b259c640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 0x7f159406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15a40039e0 tx=0x7f15a4009790 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:24.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.748+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f159c090080 con 0x7f15b40cb880 2023-10-21T16:02:24.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.940+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15b40ce270 con 0x7f159406ae70 2023-10-21T16:02:24.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.966+0000 7f15aaffd640 1 -- 172.21.15.53:0/2329358523 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345122 (secure 0 0 0) 0x7f15a4004640 con 0x7f159406ae70 2023-10-21T16:02:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:02:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 msgr2=0x7f159406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 0x7f159406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15a40039e0 tx=0x7f15a4009790 comp rx=0 tx=0).stop 2023-10-21T16:02:24.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 msgr2=0x7f15b40ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:24.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f159c00d5f0 tx=0x7f159c00dab0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 shutdown_connections 2023-10-21T16:02:24.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b40cbce0 0x7f15b40abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.974+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f159406ae70 0x7f159406d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.975+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15b40cb880 0x7f15b40ab8d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.975+0000 7f15b894e640 1 --2- 172.21.15.53:0/2329358523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b4069d50 0x7f15b40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.975+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 >> 172.21.15.53:0/2329358523 conn(0x7f15b406d450 msgr2=0x7f15b40c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.975+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 shutdown_connections 2023-10-21T16:02:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:24.975+0000 7f15b894e640 1 -- 172.21.15.53:0/2329358523 wait complete. 2023-10-21T16:02:24.998 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-21T16:02:23.025103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":211,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236204,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738516,"statfs":{"total":773094113280,"available":772852240384,"internally_reserved":0,"allocated":3481600,"data_stored":2082142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005038"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928748+0000","last_change":"2023-10-21T16:01:55.158448+0000","last_active":"2023-10-21T16:02:00.928748+0000","last_peered":"2023-10-21T16:02:00.928748+0000","last_clean":"2023-10-21T16:02:00.928748+0000","last_became_active":"2023-10-21T16:01:55.158229+0000","last_became_peered":"2023-10-21T16:01:55.158229+0000","last_unstale":"2023-10-21T16:02:00.928748+0000","last_undegraded":"2023-10-21T16:02:00.928748+0000","last_fullsized":"2023-10-21T16:02:00.928748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947239+0000","last_change":"2023-10-21T16:01:55.937779+0000","last_active":"2023-10-21T16:02:00.947239+0000","last_peered":"2023-10-21T16:02:00.947239+0000","last_clean":"2023-10-21T16:02:00.947239+0000","last_became_active":"2023-10-21T16:01:55.937491+0000","last_became_peered":"2023-10-21T16:01:55.937491+0000","last_unstale":"2023-10-21T16:02:00.947239+0000","last_undegraded":"2023-10-21T16:02:00.947239+0000","last_fullsized":"2023-10-21T16:02:00.947239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895601+0000","last_change":"2023-10-21T16:02:00.754927+0000","last_active":"2023-10-21T16:02:02.895601+0000","last_peered":"2023-10-21T16:02:02.895601+0000","last_clean":"2023-10-21T16:02:02.895601+0000","last_became_active":"2023-10-21T16:02:00.753883+0000","last_became_peered":"2023-10-21T16:02:00.753883+0000","last_unstale":"2023-10-21T16:02:02.895601+0000","last_undegraded":"2023-10-21T16:02:02.895601+0000","last_fullsized":"2023-10-21T16:02:02.895601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:45.898604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947499+0000","last_change":"2023-10-21T16:01:55.153722+0000","last_active":"2023-10-21T16:02:00.947499+0000","last_peered":"2023-10-21T16:02:00.947499+0000","last_clean":"2023-10-21T16:02:00.947499+0000","last_became_active":"2023-10-21T16:01:55.153434+0000","last_became_peered":"2023-10-21T16:01:55.153434+0000","last_unstale":"2023-10-21T16:02:00.947499+0000","last_undegraded":"2023-10-21T16:02:00.947499+0000","last_fullsized":"2023-10-21T16:02:00.947499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928468+0000","last_change":"2023-10-21T16:01:55.954746+0000","last_active":"2023-10-21T16:02:00.928468+0000","last_peered":"2023-10-21T16:02:00.928468+0000","last_clean":"2023-10-21T16:02:00.928468+0000","last_became_active":"2023-10-21T16:01:55.953670+0000","last_became_peered":"2023-10-21T16:01:55.953670+0000","last_unstale":"2023-10-21T16:02:00.928468+0000","last_undegraded":"2023-10-21T16:02:00.928468+0000","last_fullsized":"2023-10-21T16:02:00.928468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918749+0000","last_change":"2023-10-21T16:02:00.185538+0000","last_active":"2023-10-21T16:02:01.918749+0000","last_peered":"2023-10-21T16:02:01.918749+0000","last_clean":"2023-10-21T16:02:01.918749+0000","last_became_active":"2023-10-21T16:02:00.185127+0000","last_became_peered":"2023-10-21T16:02:00.185127+0000","last_unstale":"2023-10-21T16:02:01.918749+0000","last_undegraded":"2023-10-21T16:02:01.918749+0000","last_fullsized":"2023-10-21T16:02:01.918749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415743+0000","last_change":"2023-10-21T16:01:55.169060+0000","last_active":"2023-10-21T16:02:01.415743+0000","last_peered":"2023-10-21T16:02:01.415743+0000","last_clean":"2023-10-21T16:02:01.415743+0000","last_became_active":"2023-10-21T16:01:55.168772+0000","last_became_peered":"2023-10-21T16:01:55.168772+0000","last_unstale":"2023-10-21T16:02:01.415743+0000","last_undegraded":"2023-10-21T16:02:01.415743+0000","last_fullsized":"2023-10-21T16:02:01.415743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004837+0000","last_change":"2023-10-21T16:01:55.950057+0000","last_active":"2023-10-21T16:02:01.004837+0000","last_peered":"2023-10-21T16:02:01.004837+0000","last_clean":"2023-10-21T16:02:01.004837+0000","last_became_active":"2023-10-21T16:01:55.949780+0000","last_became_peered":"2023-10-21T16:01:55.949780+0000","last_unstale":"2023-10-21T16:02:01.004837+0000","last_undegraded":"2023-10-21T16:02:01.004837+0000","last_fullsized":"2023-10-21T16:02:01.004837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921013+0000","last_change":"2023-10-21T16:02:00.184728+0000","last_active":"2023-10-21T16:02:01.921013+0000","last_peered":"2023-10-21T16:02:01.921013+0000","last_clean":"2023-10-21T16:02:01.921013+0000","last_became_active":"2023-10-21T16:02:00.183993+0000","last_became_peered":"2023-10-21T16:02:00.183993+0000","last_unstale":"2023-10-21T16:02:01.921013+0000","last_undegraded":"2023-10-21T16:02:01.921013+0000","last_fullsized":"2023-10-21T16:02:01.921013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415376+0000","last_change":"2023-10-21T16:01:55.171185+0000","last_active":"2023-10-21T16:02:01.415376+0000","last_peered":"2023-10-21T16:02:01.415376+0000","last_clean":"2023-10-21T16:02:01.415376+0000","last_became_active":"2023-10-21T16:01:55.170714+0000","last_became_peered":"2023-10-21T16:01:55.170714+0000","last_unstale":"2023-10-21T16:02:01.415376+0000","last_undegraded":"2023-10-21T16:02:01.415376+0000","last_fullsized":"2023-10-21T16:02:01.415376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004885+0000","last_change":"2023-10-21T16:01:55.957250+0000","last_active":"2023-10-21T16:02:01.004885+0000","last_peered":"2023-10-21T16:02:01.004885+0000","last_clean":"2023-10-21T16:02:01.004885+0000","last_became_active":"2023-10-21T16:01:55.956926+0000","last_became_peered":"2023-10-21T16:01:55.956926+0000","last_unstale":"2023-10-21T16:02:01.004885+0000","last_undegraded":"2023-10-21T16:02:01.004885+0000","last_fullsized":"2023-10-21T16:02:01.004885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.920949+0000","last_change":"2023-10-21T16:02:00.190590+0000","last_active":"2023-10-21T16:02:01.920949+0000","last_peered":"2023-10-21T16:02:01.920949+0000","last_clean":"2023-10-21T16:02:01.920949+0000","last_became_active":"2023-10-21T16:02:00.190131+0000","last_became_peered":"2023-10-21T16:02:00.190131+0000","last_unstale":"2023-10-21T16:02:01.920949+0000","last_undegraded":"2023-10-21T16:02:01.920949+0000","last_fullsized":"2023-10-21T16:02:01.920949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413796+0000","last_change":"2023-10-21T16:01:55.948227+0000","last_active":"2023-10-21T16:02:01.413796+0000","last_peered":"2023-10-21T16:02:01.413796+0000","last_clean":"2023-10-21T16:02:01.413796+0000","last_became_active":"2023-10-21T16:01:55.947879+0000","last_became_peered":"2023-10-21T16:01:55.947879+0000","last_unstale":"2023-10-21T16:02:01.413796+0000","last_undegraded":"2023-10-21T16:02:01.413796+0000","last_fullsized":"2023-10-21T16:02:01.413796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415138+0000","last_change":"2023-10-21T16:01:55.161941+0000","last_active":"2023-10-21T16:02:01.415138+0000","last_peered":"2023-10-21T16:02:01.415138+0000","last_clean":"2023-10-21T16:02:01.415138+0000","last_became_active":"2023-10-21T16:01:55.160804+0000","last_became_peered":"2023-10-21T16:01:55.160804+0000","last_unstale":"2023-10-21T16:02:01.415138+0000","last_undegraded":"2023-10-21T16:02:01.415138+0000","last_fullsized":"2023-10-21T16:02:01.415138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919532+0000","last_change":"2023-10-21T16:02:00.179242+0000","last_active":"2023-10-21T16:02:01.919532+0000","last_peered":"2023-10-21T16:02:01.919532+0000","last_clean":"2023-10-21T16:02:01.919532+0000","last_became_active":"2023-10-21T16:02:00.178744+0000","last_became_peered":"2023-10-21T16:02:00.178744+0000","last_unstale":"2023-10-21T16:02:01.919532+0000","last_undegraded":"2023-10-21T16:02:01.919532+0000","last_fullsized":"2023-10-21T16:02:01.919532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:45.186023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946147+0000","last_change":"2023-10-21T16:01:55.943291+0000","last_active":"2023-10-21T16:02:00.946147+0000","last_peered":"2023-10-21T16:02:00.946147+0000","last_clean":"2023-10-21T16:02:00.946147+0000","last_became_active":"2023-10-21T16:01:55.942937+0000","last_became_peered":"2023-10-21T16:01:55.942937+0000","last_unstale":"2023-10-21T16:02:00.946147+0000","last_undegraded":"2023-10-21T16:02:00.946147+0000","last_fullsized":"2023-10-21T16:02:00.946147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911689+0000","last_change":"2023-10-21T16:01:55.147810+0000","last_active":"2023-10-21T16:02:00.911689+0000","last_peered":"2023-10-21T16:02:00.911689+0000","last_clean":"2023-10-21T16:02:00.911689+0000","last_became_active":"2023-10-21T16:01:55.147532+0000","last_became_peered":"2023-10-21T16:01:55.147532+0000","last_unstale":"2023-10-21T16:02:00.911689+0000","last_undegraded":"2023-10-21T16:02:00.911689+0000","last_fullsized":"2023-10-21T16:02:00.911689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921974+0000","last_change":"2023-10-21T16:02:00.192496+0000","last_active":"2023-10-21T16:02:01.921974+0000","last_peered":"2023-10-21T16:02:01.921974+0000","last_clean":"2023-10-21T16:02:01.921974+0000","last_became_active":"2023-10-21T16:02:00.192090+0000","last_became_peered":"2023-10-21T16:02:00.192090+0000","last_unstale":"2023-10-21T16:02:01.921974+0000","last_undegraded":"2023-10-21T16:02:01.921974+0000","last_fullsized":"2023-10-21T16:02:01.921974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946330+0000","last_change":"2023-10-21T16:01:55.936744+0000","last_active":"2023-10-21T16:02:00.946330+0000","last_peered":"2023-10-21T16:02:00.946330+0000","last_clean":"2023-10-21T16:02:00.946330+0000","last_became_active":"2023-10-21T16:01:55.936451+0000","last_became_peered":"2023-10-21T16:01:55.936451+0000","last_unstale":"2023-10-21T16:02:00.946330+0000","last_undegraded":"2023-10-21T16:02:00.946330+0000","last_fullsized":"2023-10-21T16:02:00.946330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910754+0000","last_change":"2023-10-21T16:01:55.159913+0000","last_active":"2023-10-21T16:02:00.910754+0000","last_peered":"2023-10-21T16:02:00.910754+0000","last_clean":"2023-10-21T16:02:00.910754+0000","last_became_active":"2023-10-21T16:01:55.159480+0000","last_became_peered":"2023-10-21T16:01:55.159480+0000","last_unstale":"2023-10-21T16:02:00.910754+0000","last_undegraded":"2023-10-21T16:02:00.910754+0000","last_fullsized":"2023-10-21T16:02:00.910754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:41.390056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895508+0000","last_change":"2023-10-21T16:02:00.748804+0000","last_active":"2023-10-21T16:02:02.895508+0000","last_peered":"2023-10-21T16:02:02.895508+0000","last_clean":"2023-10-21T16:02:02.895508+0000","last_became_active":"2023-10-21T16:02:00.746585+0000","last_became_peered":"2023-10-21T16:02:00.746585+0000","last_unstale":"2023-10-21T16:02:02.895508+0000","last_undegraded":"2023-10-21T16:02:02.895508+0000","last_fullsized":"2023-10-21T16:02:02.895508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913021+0000","last_change":"2023-10-21T16:01:55.936909+0000","last_active":"2023-10-21T16:02:00.913021+0000","last_peered":"2023-10-21T16:02:00.913021+0000","last_clean":"2023-10-21T16:02:00.913021+0000","last_became_active":"2023-10-21T16:01:55.930874+0000","last_became_peered":"2023-10-21T16:01:55.930874+0000","last_unstale":"2023-10-21T16:02:00.913021+0000","last_undegraded":"2023-10-21T16:02:00.913021+0000","last_fullsized":"2023-10-21T16:02:00.913021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:19.306040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414418+0000","last_change":"2023-10-21T16:01:55.161120+0000","last_active":"2023-10-21T16:02:01.414418+0000","last_peered":"2023-10-21T16:02:01.414418+0000","last_clean":"2023-10-21T16:02:01.414418+0000","last_became_active":"2023-10-21T16:01:55.160853+0000","last_became_peered":"2023-10-21T16:01:55.160853+0000","last_unstale":"2023-10-21T16:02:01.414418+0000","last_undegraded":"2023-10-21T16:02:01.414418+0000","last_fullsized":"2023-10-21T16:02:01.414418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919323+0000","last_change":"2023-10-21T16:02:00.192783+0000","last_active":"2023-10-21T16:02:01.919323+0000","last_peered":"2023-10-21T16:02:01.919323+0000","last_clean":"2023-10-21T16:02:01.919323+0000","last_became_active":"2023-10-21T16:02:00.192399+0000","last_became_peered":"2023-10-21T16:02:00.192399+0000","last_unstale":"2023-10-21T16:02:01.919323+0000","last_undegraded":"2023-10-21T16:02:01.919323+0000","last_fullsized":"2023-10-21T16:02:01.919323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928409+0000","last_change":"2023-10-21T16:01:55.942303+0000","last_active":"2023-10-21T16:02:00.928409+0000","last_peered":"2023-10-21T16:02:00.928409+0000","last_clean":"2023-10-21T16:02:00.928409+0000","last_became_active":"2023-10-21T16:01:55.941848+0000","last_became_peered":"2023-10-21T16:01:55.941848+0000","last_unstale":"2023-10-21T16:02:00.928409+0000","last_undegraded":"2023-10-21T16:02:00.928409+0000","last_fullsized":"2023-10-21T16:02:00.928409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417420+0000","last_change":"2023-10-21T16:01:55.164579+0000","last_active":"2023-10-21T16:02:01.417420+0000","last_peered":"2023-10-21T16:02:01.417420+0000","last_clean":"2023-10-21T16:02:01.417420+0000","last_became_active":"2023-10-21T16:01:55.162946+0000","last_became_peered":"2023-10-21T16:01:55.162946+0000","last_unstale":"2023-10-21T16:02:01.417420+0000","last_undegraded":"2023-10-21T16:02:01.417420+0000","last_fullsized":"2023-10-21T16:02:01.417420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919836+0000","last_change":"2023-10-21T16:02:00.748601+0000","last_active":"2023-10-21T16:02:01.919836+0000","last_peered":"2023-10-21T16:02:01.919836+0000","last_clean":"2023-10-21T16:02:01.919836+0000","last_became_active":"2023-10-21T16:02:00.747592+0000","last_became_peered":"2023-10-21T16:02:00.747592+0000","last_unstale":"2023-10-21T16:02:01.919836+0000","last_undegraded":"2023-10-21T16:02:01.919836+0000","last_fullsized":"2023-10-21T16:02:01.919836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947881+0000","last_change":"2023-10-21T16:01:55.941119+0000","last_active":"2023-10-21T16:02:00.947881+0000","last_peered":"2023-10-21T16:02:00.947881+0000","last_clean":"2023-10-21T16:02:00.947881+0000","last_became_active":"2023-10-21T16:01:55.940145+0000","last_became_peered":"2023-10-21T16:01:55.940145+0000","last_unstale":"2023-10-21T16:02:00.947881+0000","last_undegraded":"2023-10-21T16:02:00.947881+0000","last_fullsized":"2023-10-21T16:02:00.947881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895162+0000","last_change":"2023-10-21T16:01:55.165800+0000","last_active":"2023-10-21T16:02:02.895162+0000","last_peered":"2023-10-21T16:02:02.895162+0000","last_clean":"2023-10-21T16:02:02.895162+0000","last_became_active":"2023-10-21T16:01:55.165500+0000","last_became_peered":"2023-10-21T16:01:55.165500+0000","last_unstale":"2023-10-21T16:02:02.895162+0000","last_undegraded":"2023-10-21T16:02:02.895162+0000","last_fullsized":"2023-10-21T16:02:02.895162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894525+0000","last_change":"2023-10-21T16:02:00.742519+0000","last_active":"2023-10-21T16:02:02.894525+0000","last_peered":"2023-10-21T16:02:02.894525+0000","last_clean":"2023-10-21T16:02:02.894525+0000","last_became_active":"2023-10-21T16:02:00.740937+0000","last_became_peered":"2023-10-21T16:02:00.740937+0000","last_unstale":"2023-10-21T16:02:02.894525+0000","last_undegraded":"2023-10-21T16:02:02.894525+0000","last_fullsized":"2023-10-21T16:02:02.894525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:22.482979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913147+0000","last_change":"2023-10-21T16:01:55.939913+0000","last_active":"2023-10-21T16:02:00.913147+0000","last_peered":"2023-10-21T16:02:00.913147+0000","last_clean":"2023-10-21T16:02:00.913147+0000","last_became_active":"2023-10-21T16:01:55.939419+0000","last_became_peered":"2023-10-21T16:01:55.939419+0000","last_unstale":"2023-10-21T16:02:00.913147+0000","last_undegraded":"2023-10-21T16:02:00.913147+0000","last_fullsized":"2023-10-21T16:02:00.913147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415967+0000","last_change":"2023-10-21T16:01:55.163977+0000","last_active":"2023-10-21T16:02:01.415967+0000","last_peered":"2023-10-21T16:02:01.415967+0000","last_clean":"2023-10-21T16:02:01.415967+0000","last_became_active":"2023-10-21T16:01:55.163739+0000","last_became_peered":"2023-10-21T16:01:55.163739+0000","last_unstale":"2023-10-21T16:02:01.415967+0000","last_undegraded":"2023-10-21T16:02:01.415967+0000","last_fullsized":"2023-10-21T16:02:01.415967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:17.767707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007711+0000","last_change":"2023-10-21T16:02:00.751077+0000","last_active":"2023-10-21T16:02:02.007711+0000","last_peered":"2023-10-21T16:02:02.007711+0000","last_clean":"2023-10-21T16:02:02.007711+0000","last_became_active":"2023-10-21T16:02:00.749997+0000","last_became_peered":"2023-10-21T16:02:00.749997+0000","last_unstale":"2023-10-21T16:02:02.007711+0000","last_undegraded":"2023-10-21T16:02:02.007711+0000","last_fullsized":"2023-10-21T16:02:02.007711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005010+0000","last_change":"2023-10-21T16:01:55.951660+0000","last_active":"2023-10-21T16:02:01.005010+0000","last_peered":"2023-10-21T16:02:01.005010+0000","last_clean":"2023-10-21T16:02:01.005010+0000","last_became_active":"2023-10-21T16:01:55.951279+0000","last_became_peered":"2023-10-21T16:01:55.951279+0000","last_unstale":"2023-10-21T16:02:01.005010+0000","last_undegraded":"2023-10-21T16:02:01.005010+0000","last_fullsized":"2023-10-21T16:02:01.005010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913993+0000","last_change":"2023-10-21T16:01:55.149562+0000","last_active":"2023-10-21T16:02:00.913993+0000","last_peered":"2023-10-21T16:02:00.913993+0000","last_clean":"2023-10-21T16:02:00.913993+0000","last_became_active":"2023-10-21T16:01:55.149274+0000","last_became_peered":"2023-10-21T16:01:55.149274+0000","last_unstale":"2023-10-21T16:02:00.913993+0000","last_undegraded":"2023-10-21T16:02:00.913993+0000","last_fullsized":"2023-10-21T16:02:00.913993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.892788+0000","last_change":"2023-10-21T16:02:00.189379+0000","last_active":"2023-10-21T16:02:02.892788+0000","last_peered":"2023-10-21T16:02:02.892788+0000","last_clean":"2023-10-21T16:02:02.892788+0000","last_became_active":"2023-10-21T16:02:00.188693+0000","last_became_peered":"2023-10-21T16:02:00.188693+0000","last_unstale":"2023-10-21T16:02:02.892788+0000","last_undegraded":"2023-10-21T16:02:02.892788+0000","last_fullsized":"2023-10-21T16:02:02.892788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913564+0000","last_change":"2023-10-21T16:01:55.941518+0000","last_active":"2023-10-21T16:02:00.913564+0000","last_peered":"2023-10-21T16:02:00.913564+0000","last_clean":"2023-10-21T16:02:00.913564+0000","last_became_active":"2023-10-21T16:01:55.938726+0000","last_became_peered":"2023-10-21T16:01:55.938726+0000","last_unstale":"2023-10-21T16:02:00.913564+0000","last_undegraded":"2023-10-21T16:02:00.913564+0000","last_fullsized":"2023-10-21T16:02:00.913564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:39.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004473+0000","last_change":"2023-10-21T16:01:55.165664+0000","last_active":"2023-10-21T16:02:01.004473+0000","last_peered":"2023-10-21T16:02:01.004473+0000","last_clean":"2023-10-21T16:02:01.004473+0000","last_became_active":"2023-10-21T16:01:55.165070+0000","last_became_peered":"2023-10-21T16:01:55.165070+0000","last_unstale":"2023-10-21T16:02:01.004473+0000","last_undegraded":"2023-10-21T16:02:01.004473+0000","last_fullsized":"2023-10-21T16:02:01.004473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:02.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893925+0000","last_change":"2023-10-21T16:02:00.186762+0000","last_active":"2023-10-21T16:02:02.893925+0000","last_peered":"2023-10-21T16:02:02.893925+0000","last_clean":"2023-10-21T16:02:02.893925+0000","last_became_active":"2023-10-21T16:02:00.185437+0000","last_became_peered":"2023-10-21T16:02:00.185437+0000","last_unstale":"2023-10-21T16:02:02.893925+0000","last_undegraded":"2023-10-21T16:02:02.893925+0000","last_fullsized":"2023-10-21T16:02:02.893925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911065+0000","last_change":"2023-10-21T16:01:55.935243+0000","last_active":"2023-10-21T16:02:00.911065+0000","last_peered":"2023-10-21T16:02:00.911065+0000","last_clean":"2023-10-21T16:02:00.911065+0000","last_became_active":"2023-10-21T16:01:55.934353+0000","last_became_peered":"2023-10-21T16:01:55.934353+0000","last_unstale":"2023-10-21T16:02:00.911065+0000","last_undegraded":"2023-10-21T16:02:00.911065+0000","last_fullsized":"2023-10-21T16:02:00.911065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914987+0000","last_change":"2023-10-21T16:01:55.147043+0000","last_active":"2023-10-21T16:02:00.914987+0000","last_peered":"2023-10-21T16:02:00.914987+0000","last_clean":"2023-10-21T16:02:00.914987+0000","last_became_active":"2023-10-21T16:01:55.146674+0000","last_became_peered":"2023-10-21T16:01:55.146674+0000","last_unstale":"2023-10-21T16:02:00.914987+0000","last_undegraded":"2023-10-21T16:02:00.914987+0000","last_fullsized":"2023-10-21T16:02:00.914987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:18.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895819+0000","last_change":"2023-10-21T16:02:00.191162+0000","last_active":"2023-10-21T16:02:02.895819+0000","last_peered":"2023-10-21T16:02:02.895819+0000","last_clean":"2023-10-21T16:02:02.895819+0000","last_became_active":"2023-10-21T16:02:00.186599+0000","last_became_peered":"2023-10-21T16:02:00.186599+0000","last_unstale":"2023-10-21T16:02:02.895819+0000","last_undegraded":"2023-10-21T16:02:02.895819+0000","last_fullsized":"2023-10-21T16:02:02.895819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.926918+0000","last_change":"2023-10-21T16:01:55.948611+0000","last_active":"2023-10-21T16:02:00.926918+0000","last_peered":"2023-10-21T16:02:00.926918+0000","last_clean":"2023-10-21T16:02:00.926918+0000","last_became_active":"2023-10-21T16:01:55.948172+0000","last_became_peered":"2023-10-21T16:01:55.948172+0000","last_unstale":"2023-10-21T16:02:00.926918+0000","last_undegraded":"2023-10-21T16:02:00.926918+0000","last_fullsized":"2023-10-21T16:02:00.926918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414533+0000","last_change":"2023-10-21T16:01:55.161709+0000","last_active":"2023-10-21T16:02:01.414533+0000","last_peered":"2023-10-21T16:02:01.414533+0000","last_clean":"2023-10-21T16:02:01.414533+0000","last_became_active":"2023-10-21T16:01:55.160426+0000","last_became_peered":"2023-10-21T16:01:55.160426+0000","last_unstale":"2023-10-21T16:02:01.414533+0000","last_undegraded":"2023-10-21T16:02:01.414533+0000","last_fullsized":"2023-10-21T16:02:01.414533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007971+0000","last_change":"2023-10-21T16:02:00.742783+0000","last_active":"2023-10-21T16:02:02.007971+0000","last_peered":"2023-10-21T16:02:02.007971+0000","last_clean":"2023-10-21T16:02:02.007971+0000","last_became_active":"2023-10-21T16:02:00.740932+0000","last_became_peered":"2023-10-21T16:02:00.740932+0000","last_unstale":"2023-10-21T16:02:02.007971+0000","last_undegraded":"2023-10-21T16:02:02.007971+0000","last_fullsized":"2023-10-21T16:02:02.007971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946103+0000","last_change":"2023-10-21T16:01:55.955491+0000","last_active":"2023-10-21T16:02:00.946103+0000","last_peered":"2023-10-21T16:02:00.946103+0000","last_clean":"2023-10-21T16:02:00.946103+0000","last_became_active":"2023-10-21T16:01:55.955163+0000","last_became_peered":"2023-10-21T16:01:55.955163+0000","last_unstale":"2023-10-21T16:02:00.946103+0000","last_undegraded":"2023-10-21T16:02:00.946103+0000","last_fullsized":"2023-10-21T16:02:00.946103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004498+0000","last_change":"2023-10-21T16:01:55.160793+0000","last_active":"2023-10-21T16:02:01.004498+0000","last_peered":"2023-10-21T16:02:01.004498+0000","last_clean":"2023-10-21T16:02:01.004498+0000","last_became_active":"2023-10-21T16:01:55.157151+0000","last_became_peered":"2023-10-21T16:01:55.157151+0000","last_unstale":"2023-10-21T16:02:01.004498+0000","last_undegraded":"2023-10-21T16:02:01.004498+0000","last_fullsized":"2023-10-21T16:02:01.004498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:36.886516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414663+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:02:01.414663+0000","last_peered":"2023-10-21T16:02:01.414663+0000","last_clean":"2023-10-21T16:02:01.414663+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:02:01.414663+0000","last_undegraded":"2023-10-21T16:02:01.414663+0000","last_fullsized":"2023-10-21T16:02:01.414663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895666+0000","last_change":"2023-10-21T16:02:00.752861+0000","last_active":"2023-10-21T16:02:02.895666+0000","last_peered":"2023-10-21T16:02:02.895666+0000","last_clean":"2023-10-21T16:02:02.895666+0000","last_became_active":"2023-10-21T16:02:00.748880+0000","last_became_peered":"2023-10-21T16:02:00.748880+0000","last_unstale":"2023-10-21T16:02:02.895666+0000","last_undegraded":"2023-10-21T16:02:02.895666+0000","last_fullsized":"2023-10-21T16:02:02.895666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415527+0000","last_change":"2023-10-21T16:01:55.954245+0000","last_active":"2023-10-21T16:02:01.415527+0000","last_peered":"2023-10-21T16:02:01.415527+0000","last_clean":"2023-10-21T16:02:01.415527+0000","last_became_active":"2023-10-21T16:01:55.953197+0000","last_became_peered":"2023-10-21T16:01:55.953197+0000","last_unstale":"2023-10-21T16:02:01.415527+0000","last_undegraded":"2023-10-21T16:02:01.415527+0000","last_fullsized":"2023-10-21T16:02:01.415527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915622+0000","last_change":"2023-10-21T16:01:55.168127+0000","last_active":"2023-10-21T16:02:00.915622+0000","last_peered":"2023-10-21T16:02:00.915622+0000","last_clean":"2023-10-21T16:02:00.915622+0000","last_became_active":"2023-10-21T16:01:55.167766+0000","last_became_peered":"2023-10-21T16:01:55.167766+0000","last_unstale":"2023-10-21T16:02:00.915622+0000","last_undegraded":"2023-10-21T16:02:00.915622+0000","last_fullsized":"2023-10-21T16:02:00.915622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:28:10.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893224+0000","last_change":"2023-10-21T16:02:00.742293+0000","last_active":"2023-10-21T16:02:02.893224+0000","last_peered":"2023-10-21T16:02:02.893224+0000","last_clean":"2023-10-21T16:02:02.893224+0000","last_became_active":"2023-10-21T16:02:00.741230+0000","last_became_peered":"2023-10-21T16:02:00.741230+0000","last_unstale":"2023-10-21T16:02:02.893224+0000","last_undegraded":"2023-10-21T16:02:02.893224+0000","last_fullsized":"2023-10-21T16:02:02.893224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005008+0000","last_change":"2023-10-21T16:01:55.948576+0000","last_active":"2023-10-21T16:02:01.005008+0000","last_peered":"2023-10-21T16:02:01.005008+0000","last_clean":"2023-10-21T16:02:01.005008+0000","last_became_active":"2023-10-21T16:01:55.947390+0000","last_became_peered":"2023-10-21T16:01:55.947390+0000","last_unstale":"2023-10-21T16:02:01.005008+0000","last_undegraded":"2023-10-21T16:02:01.005008+0000","last_fullsized":"2023-10-21T16:02:01.005008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914480+0000","last_change":"2023-10-21T16:01:55.151627+0000","last_active":"2023-10-21T16:02:00.914480+0000","last_peered":"2023-10-21T16:02:00.914480+0000","last_clean":"2023-10-21T16:02:00.914480+0000","last_became_active":"2023-10-21T16:01:55.151283+0000","last_became_peered":"2023-10-21T16:01:55.151283+0000","last_unstale":"2023-10-21T16:02:00.914480+0000","last_undegraded":"2023-10-21T16:02:00.914480+0000","last_fullsized":"2023-10-21T16:02:00.914480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007984+0000","last_change":"2023-10-21T16:02:00.183073+0000","last_active":"2023-10-21T16:02:02.007984+0000","last_peered":"2023-10-21T16:02:02.007984+0000","last_clean":"2023-10-21T16:02:02.007984+0000","last_became_active":"2023-10-21T16:02:00.182604+0000","last_became_peered":"2023-10-21T16:02:00.182604+0000","last_unstale":"2023-10-21T16:02:02.007984+0000","last_undegraded":"2023-10-21T16:02:02.007984+0000","last_fullsized":"2023-10-21T16:02:02.007984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416904+0000","last_change":"2023-10-21T16:01:55.951113+0000","last_active":"2023-10-21T16:02:01.416904+0000","last_peered":"2023-10-21T16:02:01.416904+0000","last_clean":"2023-10-21T16:02:01.416904+0000","last_became_active":"2023-10-21T16:01:55.950792+0000","last_became_peered":"2023-10-21T16:01:55.950792+0000","last_unstale":"2023-10-21T16:02:01.416904+0000","last_undegraded":"2023-10-21T16:02:01.416904+0000","last_fullsized":"2023-10-21T16:02:01.416904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915704+0000","last_change":"2023-10-21T16:01:55.149597+0000","last_active":"2023-10-21T16:02:00.915704+0000","last_peered":"2023-10-21T16:02:00.915704+0000","last_clean":"2023-10-21T16:02:00.915704+0000","last_became_active":"2023-10-21T16:01:55.149258+0000","last_became_peered":"2023-10-21T16:01:55.149258+0000","last_unstale":"2023-10-21T16:02:00.915704+0000","last_undegraded":"2023-10-21T16:02:00.915704+0000","last_fullsized":"2023-10-21T16:02:00.915704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.914975+0000","last_change":"2023-10-21T16:02:00.750068+0000","last_active":"2023-10-21T16:02:01.914975+0000","last_peered":"2023-10-21T16:02:01.914975+0000","last_clean":"2023-10-21T16:02:01.914975+0000","last_became_active":"2023-10-21T16:02:00.749055+0000","last_became_peered":"2023-10-21T16:02:00.749055+0000","last_unstale":"2023-10-21T16:02:01.914975+0000","last_undegraded":"2023-10-21T16:02:01.914975+0000","last_fullsized":"2023-10-21T16:02:01.914975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416138+0000","last_change":"2023-10-21T16:01:55.953586+0000","last_active":"2023-10-21T16:02:01.416138+0000","last_peered":"2023-10-21T16:02:01.416138+0000","last_clean":"2023-10-21T16:02:01.416138+0000","last_became_active":"2023-10-21T16:01:55.952550+0000","last_became_peered":"2023-10-21T16:01:55.952550+0000","last_unstale":"2023-10-21T16:02:01.416138+0000","last_undegraded":"2023-10-21T16:02:01.416138+0000","last_fullsized":"2023-10-21T16:02:01.416138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913456+0000","last_change":"2023-10-21T16:01:55.154502+0000","last_active":"2023-10-21T16:02:00.913456+0000","last_peered":"2023-10-21T16:02:00.913456+0000","last_clean":"2023-10-21T16:02:00.913456+0000","last_became_active":"2023-10-21T16:01:55.150688+0000","last_became_peered":"2023-10-21T16:01:55.150688+0000","last_unstale":"2023-10-21T16:02:00.913456+0000","last_undegraded":"2023-10-21T16:02:00.913456+0000","last_fullsized":"2023-10-21T16:02:00.913456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:32.709310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918774+0000","last_change":"2023-10-21T16:02:00.184524+0000","last_active":"2023-10-21T16:02:01.918774+0000","last_peered":"2023-10-21T16:02:01.918774+0000","last_clean":"2023-10-21T16:02:01.918774+0000","last_became_active":"2023-10-21T16:02:00.184081+0000","last_became_peered":"2023-10-21T16:02:00.184081+0000","last_unstale":"2023-10-21T16:02:01.918774+0000","last_undegraded":"2023-10-21T16:02:01.918774+0000","last_fullsized":"2023-10-21T16:02:01.918774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414960+0000","last_change":"2023-10-21T16:01:55.951983+0000","last_active":"2023-10-21T16:02:01.414960+0000","last_peered":"2023-10-21T16:02:01.414960+0000","last_clean":"2023-10-21T16:02:01.414960+0000","last_became_active":"2023-10-21T16:01:55.951633+0000","last_became_peered":"2023-10-21T16:01:55.951633+0000","last_unstale":"2023-10-21T16:02:01.414960+0000","last_undegraded":"2023-10-21T16:02:01.414960+0000","last_fullsized":"2023-10-21T16:02:01.414960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914399+0000","last_change":"2023-10-21T16:01:55.147150+0000","last_active":"2023-10-21T16:02:00.914399+0000","last_peered":"2023-10-21T16:02:00.914399+0000","last_clean":"2023-10-21T16:02:00.914399+0000","last_became_active":"2023-10-21T16:01:55.141230+0000","last_became_peered":"2023-10-21T16:01:55.141230+0000","last_unstale":"2023-10-21T16:02:00.914399+0000","last_undegraded":"2023-10-21T16:02:00.914399+0000","last_fullsized":"2023-10-21T16:02:00.914399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007865+0000","last_change":"2023-10-21T16:02:00.187875+0000","last_active":"2023-10-21T16:02:02.007865+0000","last_peered":"2023-10-21T16:02:02.007865+0000","last_clean":"2023-10-21T16:02:02.007865+0000","last_became_active":"2023-10-21T16:02:00.187427+0000","last_became_peered":"2023-10-21T16:02:00.187427+0000","last_unstale":"2023-10-21T16:02:02.007865+0000","last_undegraded":"2023-10-21T16:02:02.007865+0000","last_fullsized":"2023-10-21T16:02:02.007865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414077+0000","last_change":"2023-10-21T16:01:55.944005+0000","last_active":"2023-10-21T16:02:01.414077+0000","last_peered":"2023-10-21T16:02:01.414077+0000","last_clean":"2023-10-21T16:02:01.414077+0000","last_became_active":"2023-10-21T16:01:55.942887+0000","last_became_peered":"2023-10-21T16:01:55.942887+0000","last_unstale":"2023-10-21T16:02:01.414077+0000","last_undegraded":"2023-10-21T16:02:01.414077+0000","last_fullsized":"2023-10-21T16:02:01.414077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413985+0000","last_change":"2023-10-21T16:01:55.160115+0000","last_active":"2023-10-21T16:02:01.413985+0000","last_peered":"2023-10-21T16:02:01.413985+0000","last_clean":"2023-10-21T16:02:01.413985+0000","last_became_active":"2023-10-21T16:01:55.159788+0000","last_became_peered":"2023-10-21T16:01:55.159788+0000","last_unstale":"2023-10-21T16:02:01.413985+0000","last_undegraded":"2023-10-21T16:02:01.413985+0000","last_fullsized":"2023-10-21T16:02:01.413985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897390+0000","last_change":"2023-10-21T16:02:00.745714+0000","last_active":"2023-10-21T16:02:02.897390+0000","last_peered":"2023-10-21T16:02:02.897390+0000","last_clean":"2023-10-21T16:02:02.897390+0000","last_became_active":"2023-10-21T16:02:00.744779+0000","last_became_peered":"2023-10-21T16:02:00.744779+0000","last_unstale":"2023-10-21T16:02:02.897390+0000","last_undegraded":"2023-10-21T16:02:02.897390+0000","last_fullsized":"2023-10-21T16:02:02.897390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:53.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414998+0000","last_change":"2023-10-21T16:01:55.955734+0000","last_active":"2023-10-21T16:02:01.414998+0000","last_peered":"2023-10-21T16:02:01.414998+0000","last_clean":"2023-10-21T16:02:01.414998+0000","last_became_active":"2023-10-21T16:01:55.955333+0000","last_became_peered":"2023-10-21T16:01:55.955333+0000","last_unstale":"2023-10-21T16:02:01.414998+0000","last_undegraded":"2023-10-21T16:02:01.414998+0000","last_fullsized":"2023-10-21T16:02:01.414998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004475+0000","last_change":"2023-10-21T16:01:55.167450+0000","last_active":"2023-10-21T16:02:01.004475+0000","last_peered":"2023-10-21T16:02:01.004475+0000","last_clean":"2023-10-21T16:02:01.004475+0000","last_became_active":"2023-10-21T16:01:55.166857+0000","last_became_peered":"2023-10-21T16:01:55.166857+0000","last_unstale":"2023-10-21T16:02:01.004475+0000","last_undegraded":"2023-10-21T16:02:01.004475+0000","last_fullsized":"2023-10-21T16:02:01.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.896683+0000","last_change":"2023-10-21T16:02:00.742809+0000","last_active":"2023-10-21T16:02:02.896683+0000","last_peered":"2023-10-21T16:02:02.896683+0000","last_clean":"2023-10-21T16:02:02.896683+0000","last_became_active":"2023-10-21T16:02:00.741398+0000","last_became_peered":"2023-10-21T16:02:00.741398+0000","last_unstale":"2023-10-21T16:02:02.896683+0000","last_undegraded":"2023-10-21T16:02:02.896683+0000","last_fullsized":"2023-10-21T16:02:02.896683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:35:32.899715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927531+0000","last_change":"2023-10-21T16:01:55.937515+0000","last_active":"2023-10-21T16:02:00.927531+0000","last_peered":"2023-10-21T16:02:00.927531+0000","last_clean":"2023-10-21T16:02:00.927531+0000","last_became_active":"2023-10-21T16:01:55.937165+0000","last_became_peered":"2023-10-21T16:01:55.937165+0000","last_unstale":"2023-10-21T16:02:00.927531+0000","last_undegraded":"2023-10-21T16:02:00.927531+0000","last_fullsized":"2023-10-21T16:02:00.927531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928104+0000","last_change":"2023-10-21T16:01:55.151763+0000","last_active":"2023-10-21T16:02:00.928104+0000","last_peered":"2023-10-21T16:02:00.928104+0000","last_clean":"2023-10-21T16:02:00.928104+0000","last_became_active":"2023-10-21T16:01:55.151316+0000","last_became_peered":"2023-10-21T16:01:55.151316+0000","last_unstale":"2023-10-21T16:02:00.928104+0000","last_undegraded":"2023-10-21T16:02:00.928104+0000","last_fullsized":"2023-10-21T16:02:00.928104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.917604+0000","last_change":"2023-10-21T16:02:00.742437+0000","last_active":"2023-10-21T16:02:01.917604+0000","last_peered":"2023-10-21T16:02:01.917604+0000","last_clean":"2023-10-21T16:02:01.917604+0000","last_became_active":"2023-10-21T16:02:00.741212+0000","last_became_peered":"2023-10-21T16:02:00.741212+0000","last_unstale":"2023-10-21T16:02:01.917604+0000","last_undegraded":"2023-10-21T16:02:01.917604+0000","last_fullsized":"2023-10-21T16:02:01.917604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415373+0000","last_change":"2023-10-21T16:01:55.942841+0000","last_active":"2023-10-21T16:02:01.415373+0000","last_peered":"2023-10-21T16:02:01.415373+0000","last_clean":"2023-10-21T16:02:01.415373+0000","last_became_active":"2023-10-21T16:01:55.942471+0000","last_became_peered":"2023-10-21T16:01:55.942471+0000","last_unstale":"2023-10-21T16:02:01.415373+0000","last_undegraded":"2023-10-21T16:02:01.415373+0000","last_fullsized":"2023-10-21T16:02:01.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:14.233631+0000","last_change":"2023-10-21T16:01:55.156634+0000","last_active":"2023-10-21T16:02:14.233631+0000","last_peered":"2023-10-21T16:02:14.233631+0000","last_clean":"2023-10-21T16:02:14.233631+0000","last_became_active":"2023-10-21T16:01:55.156340+0000","last_became_peered":"2023-10-21T16:01:55.156340+0000","last_unstale":"2023-10-21T16:02:14.233631+0000","last_undegraded":"2023-10-21T16:02:14.233631+0000","last_fullsized":"2023-10-21T16:02:14.233631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893349+0000","last_change":"2023-10-21T16:02:00.189158+0000","last_active":"2023-10-21T16:02:02.893349+0000","last_peered":"2023-10-21T16:02:02.893349+0000","last_clean":"2023-10-21T16:02:02.893349+0000","last_became_active":"2023-10-21T16:02:00.188097+0000","last_became_peered":"2023-10-21T16:02:00.188097+0000","last_unstale":"2023-10-21T16:02:02.893349+0000","last_undegraded":"2023-10-21T16:02:02.893349+0000","last_fullsized":"2023-10-21T16:02:02.893349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914268+0000","last_change":"2023-10-21T16:01:55.944799+0000","last_active":"2023-10-21T16:02:00.914268+0000","last_peered":"2023-10-21T16:02:00.914268+0000","last_clean":"2023-10-21T16:02:00.914268+0000","last_became_active":"2023-10-21T16:01:55.944472+0000","last_became_peered":"2023-10-21T16:01:55.944472+0000","last_unstale":"2023-10-21T16:02:00.914268+0000","last_undegraded":"2023-10-21T16:02:00.914268+0000","last_fullsized":"2023-10-21T16:02:00.914268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:58.256813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914405+0000","last_change":"2023-10-21T16:01:55.162979+0000","last_active":"2023-10-21T16:02:00.914405+0000","last_peered":"2023-10-21T16:02:00.914405+0000","last_clean":"2023-10-21T16:02:00.914405+0000","last_became_active":"2023-10-21T16:01:55.162557+0000","last_became_peered":"2023-10-21T16:01:55.162557+0000","last_unstale":"2023-10-21T16:02:00.914405+0000","last_undegraded":"2023-10-21T16:02:00.914405+0000","last_fullsized":"2023-10-21T16:02:00.914405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921560+0000","last_change":"2023-10-21T16:02:00.188187+0000","last_active":"2023-10-21T16:02:01.921560+0000","last_peered":"2023-10-21T16:02:01.921560+0000","last_clean":"2023-10-21T16:02:01.921560+0000","last_became_active":"2023-10-21T16:02:00.187688+0000","last_became_peered":"2023-10-21T16:02:00.187688+0000","last_unstale":"2023-10-21T16:02:01.921560+0000","last_undegraded":"2023-10-21T16:02:01.921560+0000","last_fullsized":"2023-10-21T16:02:01.921560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928702+0000","last_change":"2023-10-21T16:01:55.954216+0000","last_active":"2023-10-21T16:02:00.928702+0000","last_peered":"2023-10-21T16:02:00.928702+0000","last_clean":"2023-10-21T16:02:00.928702+0000","last_became_active":"2023-10-21T16:01:55.953211+0000","last_became_peered":"2023-10-21T16:01:55.953211+0000","last_unstale":"2023-10-21T16:02:00.928702+0000","last_undegraded":"2023-10-21T16:02:00.928702+0000","last_fullsized":"2023-10-21T16:02:00.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005397+0000","last_change":"2023-10-21T16:01:55.169893+0000","last_active":"2023-10-21T16:02:01.005397+0000","last_peered":"2023-10-21T16:02:01.005397+0000","last_clean":"2023-10-21T16:02:01.005397+0000","last_became_active":"2023-10-21T16:01:55.169587+0000","last_became_peered":"2023-10-21T16:01:55.169587+0000","last_unstale":"2023-10-21T16:02:01.005397+0000","last_undegraded":"2023-10-21T16:02:01.005397+0000","last_fullsized":"2023-10-21T16:02:01.005397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894928+0000","last_change":"2023-10-21T16:02:00.185249+0000","last_active":"2023-10-21T16:02:02.894928+0000","last_peered":"2023-10-21T16:02:02.894928+0000","last_clean":"2023-10-21T16:02:02.894928+0000","last_became_active":"2023-10-21T16:02:00.184448+0000","last_became_peered":"2023-10-21T16:02:00.184448+0000","last_unstale":"2023-10-21T16:02:02.894928+0000","last_undegraded":"2023-10-21T16:02:02.894928+0000","last_fullsized":"2023-10-21T16:02:02.894928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:18:23.475051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005291+0000","last_change":"2023-10-21T16:01:55.954077+0000","last_active":"2023-10-21T16:02:01.005291+0000","last_peered":"2023-10-21T16:02:01.005291+0000","last_clean":"2023-10-21T16:02:01.005291+0000","last_became_active":"2023-10-21T16:01:55.951152+0000","last_became_peered":"2023-10-21T16:01:55.951152+0000","last_unstale":"2023-10-21T16:02:01.005291+0000","last_undegraded":"2023-10-21T16:02:01.005291+0000","last_fullsized":"2023-10-21T16:02:01.005291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417259+0000","last_change":"2023-10-21T16:01:55.171136+0000","last_active":"2023-10-21T16:02:01.417259+0000","last_peered":"2023-10-21T16:02:01.417259+0000","last_clean":"2023-10-21T16:02:01.417259+0000","last_became_active":"2023-10-21T16:01:55.170850+0000","last_became_peered":"2023-10-21T16:01:55.170850+0000","last_unstale":"2023-10-21T16:02:01.417259+0000","last_undegraded":"2023-10-21T16:02:01.417259+0000","last_fullsized":"2023-10-21T16:02:01.417259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895695+0000","last_change":"2023-10-21T16:02:00.753599+0000","last_active":"2023-10-21T16:02:02.895695+0000","last_peered":"2023-10-21T16:02:02.895695+0000","last_clean":"2023-10-21T16:02:02.895695+0000","last_became_active":"2023-10-21T16:02:00.751869+0000","last_became_peered":"2023-10-21T16:02:00.751869+0000","last_unstale":"2023-10-21T16:02:02.895695+0000","last_undegraded":"2023-10-21T16:02:02.895695+0000","last_fullsized":"2023-10-21T16:02:02.895695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:09.960025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007811+0000","last_change":"2023-10-21T16:02:00.750967+0000","last_active":"2023-10-21T16:02:02.007811+0000","last_peered":"2023-10-21T16:02:02.007811+0000","last_clean":"2023-10-21T16:02:02.007811+0000","last_became_active":"2023-10-21T16:02:00.749696+0000","last_became_peered":"2023-10-21T16:02:00.749696+0000","last_unstale":"2023-10-21T16:02:02.007811+0000","last_undegraded":"2023-10-21T16:02:02.007811+0000","last_fullsized":"2023-10-21T16:02:02.007811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910430+0000","last_change":"2023-10-21T16:01:55.938421+0000","last_active":"2023-10-21T16:02:00.910430+0000","last_peered":"2023-10-21T16:02:00.910430+0000","last_clean":"2023-10-21T16:02:00.910430+0000","last_became_active":"2023-10-21T16:01:55.938105+0000","last_became_peered":"2023-10-21T16:01:55.938105+0000","last_unstale":"2023-10-21T16:02:00.910430+0000","last_undegraded":"2023-10-21T16:02:00.910430+0000","last_fullsized":"2023-10-21T16:02:00.910430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004053+0000","last_change":"2023-10-21T16:01:55.166389+0000","last_active":"2023-10-21T16:02:01.004053+0000","last_peered":"2023-10-21T16:02:01.004053+0000","last_clean":"2023-10-21T16:02:01.004053+0000","last_became_active":"2023-10-21T16:01:55.166065+0000","last_became_peered":"2023-10-21T16:01:55.166065+0000","last_unstale":"2023-10-21T16:02:01.004053+0000","last_undegraded":"2023-10-21T16:02:01.004053+0000","last_fullsized":"2023-10-21T16:02:01.004053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895651+0000","last_change":"2023-10-21T16:02:00.195541+0000","last_active":"2023-10-21T16:02:02.895651+0000","last_peered":"2023-10-21T16:02:02.895651+0000","last_clean":"2023-10-21T16:02:02.895651+0000","last_became_active":"2023-10-21T16:02:00.195020+0000","last_became_peered":"2023-10-21T16:02:00.195020+0000","last_unstale":"2023-10-21T16:02:02.895651+0000","last_undegraded":"2023-10-21T16:02:02.895651+0000","last_fullsized":"2023-10-21T16:02:02.895651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946124+0000","last_change":"2023-10-21T16:01:55.932516+0000","last_active":"2023-10-21T16:02:00.946124+0000","last_peered":"2023-10-21T16:02:00.946124+0000","last_clean":"2023-10-21T16:02:00.946124+0000","last_became_active":"2023-10-21T16:01:55.932175+0000","last_became_peered":"2023-10-21T16:01:55.932175+0000","last_unstale":"2023-10-21T16:02:00.946124+0000","last_undegraded":"2023-10-21T16:02:00.946124+0000","last_fullsized":"2023-10-21T16:02:00.946124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945962+0000","last_change":"2023-10-21T16:01:55.156263+0000","last_active":"2023-10-21T16:02:00.945962+0000","last_peered":"2023-10-21T16:02:00.945962+0000","last_clean":"2023-10-21T16:02:00.945962+0000","last_became_active":"2023-10-21T16:01:55.155789+0000","last_became_peered":"2023-10-21T16:01:55.155789+0000","last_unstale":"2023-10-21T16:02:00.945962+0000","last_undegraded":"2023-10-21T16:02:00.945962+0000","last_fullsized":"2023-10-21T16:02:00.945962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:52.609080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415216+0000","last_change":"2023-10-21T16:01:55.160677+0000","last_active":"2023-10-21T16:02:01.415216+0000","last_peered":"2023-10-21T16:02:01.415216+0000","last_clean":"2023-10-21T16:02:01.415216+0000","last_became_active":"2023-10-21T16:01:55.160265+0000","last_became_peered":"2023-10-21T16:01:55.160265+0000","last_unstale":"2023-10-21T16:02:01.415216+0000","last_undegraded":"2023-10-21T16:02:01.415216+0000","last_fullsized":"2023-10-21T16:02:01.415216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928119+0000","last_change":"2023-10-21T16:01:55.947536+0000","last_active":"2023-10-21T16:02:00.928119+0000","last_peered":"2023-10-21T16:02:00.928119+0000","last_clean":"2023-10-21T16:02:00.928119+0000","last_became_active":"2023-10-21T16:01:55.947243+0000","last_became_peered":"2023-10-21T16:01:55.947243+0000","last_unstale":"2023-10-21T16:02:00.928119+0000","last_undegraded":"2023-10-21T16:02:00.928119+0000","last_fullsized":"2023-10-21T16:02:00.928119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.916131+0000","last_change":"2023-10-21T16:02:00.181884+0000","last_active":"2023-10-21T16:02:01.916131+0000","last_peered":"2023-10-21T16:02:01.916131+0000","last_clean":"2023-10-21T16:02:01.916131+0000","last_became_active":"2023-10-21T16:02:00.179404+0000","last_became_peered":"2023-10-21T16:02:00.179404+0000","last_unstale":"2023-10-21T16:02:01.916131+0000","last_undegraded":"2023-10-21T16:02:01.916131+0000","last_fullsized":"2023-10-21T16:02:01.916131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:42.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946429+0000","last_change":"2023-10-21T16:01:55.160559+0000","last_active":"2023-10-21T16:02:00.946429+0000","last_peered":"2023-10-21T16:02:00.946429+0000","last_clean":"2023-10-21T16:02:00.946429+0000","last_became_active":"2023-10-21T16:01:55.160241+0000","last_became_peered":"2023-10-21T16:01:55.160241+0000","last_unstale":"2023-10-21T16:02:00.946429+0000","last_undegraded":"2023-10-21T16:02:00.946429+0000","last_fullsized":"2023-10-21T16:02:00.946429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913498+0000","last_change":"2023-10-21T16:01:55.939951+0000","last_active":"2023-10-21T16:02:00.913498+0000","last_peered":"2023-10-21T16:02:00.913498+0000","last_clean":"2023-10-21T16:02:00.913498+0000","last_became_active":"2023-10-21T16:01:55.939552+0000","last_became_peered":"2023-10-21T16:01:55.939552+0000","last_unstale":"2023-10-21T16:02:00.913498+0000","last_undegraded":"2023-10-21T16:02:00.913498+0000","last_fullsized":"2023-10-21T16:02:00.913498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:38.621663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897399+0000","last_change":"2023-10-21T16:02:00.182272+0000","last_active":"2023-10-21T16:02:02.897399+0000","last_peered":"2023-10-21T16:02:02.897399+0000","last_clean":"2023-10-21T16:02:02.897399+0000","last_became_active":"2023-10-21T16:02:00.181705+0000","last_became_peered":"2023-10-21T16:02:00.181705+0000","last_unstale":"2023-10-21T16:02:02.897399+0000","last_undegraded":"2023-10-21T16:02:02.897399+0000","last_fullsized":"2023-10-21T16:02:02.897399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:28.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413688+0000","last_change":"2023-10-21T16:01:55.943801+0000","last_active":"2023-10-21T16:02:01.413688+0000","last_peered":"2023-10-21T16:02:01.413688+0000","last_clean":"2023-10-21T16:02:01.413688+0000","last_became_active":"2023-10-21T16:01:55.942546+0000","last_became_peered":"2023-10-21T16:01:55.942546+0000","last_unstale":"2023-10-21T16:02:01.413688+0000","last_undegraded":"2023-10-21T16:02:01.413688+0000","last_fullsized":"2023-10-21T16:02:01.413688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915001+0000","last_change":"2023-10-21T16:01:55.953970+0000","last_active":"2023-10-21T16:02:00.915001+0000","last_peered":"2023-10-21T16:02:00.915001+0000","last_clean":"2023-10-21T16:02:00.915001+0000","last_became_active":"2023-10-21T16:01:55.953164+0000","last_became_peered":"2023-10-21T16:01:55.953164+0000","last_unstale":"2023-10-21T16:02:00.915001+0000","last_undegraded":"2023-10-21T16:02:00.915001+0000","last_fullsized":"2023-10-21T16:02:00.915001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:16.452088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004198+0000","last_change":"2023-10-21T16:01:55.948469+0000","last_active":"2023-10-21T16:02:01.004198+0000","last_peered":"2023-10-21T16:02:01.004198+0000","last_clean":"2023-10-21T16:02:01.004198+0000","last_became_active":"2023-10-21T16:01:55.948198+0000","last_became_peered":"2023-10-21T16:01:55.948198+0000","last_unstale":"2023-10-21T16:02:01.004198+0000","last_undegraded":"2023-10-21T16:02:01.004198+0000","last_fullsized":"2023-10-21T16:02:01.004198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927645+0000","last_change":"2023-10-21T16:01:55.933799+0000","last_active":"2023-10-21T16:02:00.927645+0000","last_peered":"2023-10-21T16:02:00.927645+0000","last_clean":"2023-10-21T16:02:00.927645+0000","last_became_active":"2023-10-21T16:01:55.933431+0000","last_became_peered":"2023-10-21T16:01:55.933431+0000","last_unstale":"2023-10-21T16:02:00.927645+0000","last_undegraded":"2023-10-21T16:02:00.927645+0000","last_fullsized":"2023-10-21T16:02:00.927645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947274+0000","last_change":"2023-10-21T16:01:55.925050+0000","last_active":"2023-10-21T16:02:00.947274+0000","last_peered":"2023-10-21T16:02:00.947274+0000","last_clean":"2023-10-21T16:02:00.947274+0000","last_became_active":"2023-10-21T16:01:55.924510+0000","last_became_peered":"2023-10-21T16:01:55.924510+0000","last_unstale":"2023-10-21T16:02:00.947274+0000","last_undegraded":"2023-10-21T16:02:00.947274+0000","last_fullsized":"2023-10-21T16:02:00.947274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913921+0000","last_change":"2023-10-21T16:01:55.939109+0000","last_active":"2023-10-21T16:02:00.913921+0000","last_peered":"2023-10-21T16:02:00.913921+0000","last_clean":"2023-10-21T16:02:00.913921+0000","last_became_active":"2023-10-21T16:01:55.938852+0000","last_became_peered":"2023-10-21T16:01:55.938852+0000","last_unstale":"2023-10-21T16:02:00.913921+0000","last_undegraded":"2023-10-21T16:02:00.913921+0000","last_fullsized":"2023-10-21T16:02:00.913921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416735+0000","last_change":"2023-10-21T16:01:55.939268+0000","last_active":"2023-10-21T16:02:01.416735+0000","last_peered":"2023-10-21T16:02:01.416735+0000","last_clean":"2023-10-21T16:02:01.416735+0000","last_became_active":"2023-10-21T16:01:55.938991+0000","last_became_peered":"2023-10-21T16:01:55.938991+0000","last_unstale":"2023-10-21T16:02:01.416735+0000","last_undegraded":"2023-10-21T16:02:01.416735+0000","last_fullsized":"2023-10-21T16:02:01.416735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947882+0000","last_change":"2023-10-21T16:01:55.930486+0000","last_active":"2023-10-21T16:02:00.947882+0000","last_peered":"2023-10-21T16:02:00.947882+0000","last_clean":"2023-10-21T16:02:00.947882+0000","last_became_active":"2023-10-21T16:01:55.930187+0000","last_became_peered":"2023-10-21T16:01:55.930187+0000","last_unstale":"2023-10-21T16:02:00.947882+0000","last_undegraded":"2023-10-21T16:02:00.947882+0000","last_fullsized":"2023-10-21T16:02:00.947882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003895+0000","last_change":"2023-10-21T16:01:55.943056+0000","last_active":"2023-10-21T16:02:01.003895+0000","last_peered":"2023-10-21T16:02:01.003895+0000","last_clean":"2023-10-21T16:02:01.003895+0000","last_became_active":"2023-10-21T16:01:55.942724+0000","last_became_peered":"2023-10-21T16:01:55.942724+0000","last_unstale":"2023-10-21T16:02:01.003895+0000","last_undegraded":"2023-10-21T16:02:01.003895+0000","last_fullsized":"2023-10-21T16:02:01.003895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946201+0000","last_change":"2023-10-21T16:01:55.922812+0000","last_active":"2023-10-21T16:02:00.946201+0000","last_peered":"2023-10-21T16:02:00.946201+0000","last_clean":"2023-10-21T16:02:00.946201+0000","last_became_active":"2023-10-21T16:01:55.922134+0000","last_became_peered":"2023-10-21T16:01:55.922134+0000","last_unstale":"2023-10-21T16:02:00.946201+0000","last_undegraded":"2023-10-21T16:02:00.946201+0000","last_fullsized":"2023-10-21T16:02:00.946201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004186+0000","last_change":"2023-10-21T16:01:55.954470+0000","last_active":"2023-10-21T16:02:01.004186+0000","last_peered":"2023-10-21T16:02:01.004186+0000","last_clean":"2023-10-21T16:02:01.004186+0000","last_became_active":"2023-10-21T16:01:55.954107+0000","last_became_peered":"2023-10-21T16:01:55.954107+0000","last_unstale":"2023-10-21T16:02:01.004186+0000","last_undegraded":"2023-10-21T16:02:01.004186+0000","last_fullsized":"2023-10-21T16:02:01.004186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927404+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.927404+0000","last_peered":"2023-10-21T16:02:00.927404+0000","last_clean":"2023-10-21T16:02:00.927404+0000","last_became_active":"2023-10-21T16:01:55.946552+0000","last_became_peered":"2023-10-21T16:01:55.946552+0000","last_unstale":"2023-10-21T16:02:00.927404+0000","last_undegraded":"2023-10-21T16:02:00.927404+0000","last_fullsized":"2023-10-21T16:02:00.927404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416045+0000","last_change":"2023-10-21T16:01:55.947979+0000","last_active":"2023-10-21T16:02:01.416045+0000","last_peered":"2023-10-21T16:02:01.416045+0000","last_clean":"2023-10-21T16:02:01.416045+0000","last_became_active":"2023-10-21T16:01:55.947572+0000","last_became_peered":"2023-10-21T16:01:55.947572+0000","last_unstale":"2023-10-21T16:02:01.416045+0000","last_undegraded":"2023-10-21T16:02:01.416045+0000","last_fullsized":"2023-10-21T16:02:01.416045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415776+0000","last_change":"2023-10-21T16:01:55.946625+0000","last_active":"2023-10-21T16:02:01.415776+0000","last_peered":"2023-10-21T16:02:01.415776+0000","last_clean":"2023-10-21T16:02:01.415776+0000","last_became_active":"2023-10-21T16:01:55.945299+0000","last_became_peered":"2023-10-21T16:01:55.945299+0000","last_unstale":"2023-10-21T16:02:01.415776+0000","last_undegraded":"2023-10-21T16:02:01.415776+0000","last_fullsized":"2023-10-21T16:02:01.415776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915640+0000","last_change":"2023-10-21T16:01:55.938698+0000","last_active":"2023-10-21T16:02:00.915640+0000","last_peered":"2023-10-21T16:02:00.915640+0000","last_clean":"2023-10-21T16:02:00.915640+0000","last_became_active":"2023-10-21T16:01:55.938414+0000","last_became_peered":"2023-10-21T16:01:55.938414+0000","last_unstale":"2023-10-21T16:02:00.915640+0000","last_undegraded":"2023-10-21T16:02:00.915640+0000","last_fullsized":"2023-10-21T16:02:00.915640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:51.423697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005523+0000","last_change":"2023-10-21T16:01:55.953465+0000","last_active":"2023-10-21T16:02:01.005523+0000","last_peered":"2023-10-21T16:02:01.005523+0000","last_clean":"2023-10-21T16:02:01.005523+0000","last_became_active":"2023-10-21T16:01:55.952044+0000","last_became_peered":"2023-10-21T16:01:55.952044+0000","last_unstale":"2023-10-21T16:02:01.005523+0000","last_undegraded":"2023-10-21T16:02:01.005523+0000","last_fullsized":"2023-10-21T16:02:01.005523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003802+0000","last_change":"2023-10-21T16:01:55.943713+0000","last_active":"2023-10-21T16:02:01.003802+0000","last_peered":"2023-10-21T16:02:01.003802+0000","last_clean":"2023-10-21T16:02:01.003802+0000","last_became_active":"2023-10-21T16:01:55.939815+0000","last_became_peered":"2023-10-21T16:01:55.939815+0000","last_unstale":"2023-10-21T16:02:01.003802+0000","last_undegraded":"2023-10-21T16:02:01.003802+0000","last_fullsized":"2023-10-21T16:02:01.003802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910545+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.910545+0000","last_peered":"2023-10-21T16:02:00.910545+0000","last_clean":"2023-10-21T16:02:00.910545+0000","last_became_active":"2023-10-21T16:01:55.946640+0000","last_became_peered":"2023-10-21T16:01:55.946640+0000","last_unstale":"2023-10-21T16:02:00.910545+0000","last_undegraded":"2023-10-21T16:02:00.910545+0000","last_fullsized":"2023-10-21T16:02:00.910545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927490+0000","last_change":"2023-10-21T16:01:55.944028+0000","last_active":"2023-10-21T16:02:00.927490+0000","last_peered":"2023-10-21T16:02:00.927490+0000","last_clean":"2023-10-21T16:02:00.927490+0000","last_became_active":"2023-10-21T16:01:55.940832+0000","last_became_peered":"2023-10-21T16:01:55.940832+0000","last_unstale":"2023-10-21T16:02:00.927490+0000","last_undegraded":"2023-10-21T16:02:00.927490+0000","last_fullsized":"2023-10-21T16:02:00.927490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:42.589308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946064+0000","last_change":"2023-10-21T16:01:55.939040+0000","last_active":"2023-10-21T16:02:00.946064+0000","last_peered":"2023-10-21T16:02:00.946064+0000","last_clean":"2023-10-21T16:02:00.946064+0000","last_became_active":"2023-10-21T16:01:55.936534+0000","last_became_peered":"2023-10-21T16:01:55.936534+0000","last_unstale":"2023-10-21T16:02:00.946064+0000","last_undegraded":"2023-10-21T16:02:00.946064+0000","last_fullsized":"2023-10-21T16:02:00.946064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914180+0000","last_change":"2023-10-21T16:01:55.938149+0000","last_active":"2023-10-21T16:02:00.914180+0000","last_peered":"2023-10-21T16:02:00.914180+0000","last_clean":"2023-10-21T16:02:00.914180+0000","last_became_active":"2023-10-21T16:01:55.937906+0000","last_became_peered":"2023-10-21T16:01:55.937906+0000","last_unstale":"2023-10-21T16:02:00.914180+0000","last_undegraded":"2023-10-21T16:02:00.914180+0000","last_fullsized":"2023-10-21T16:02:00.914180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:19.855104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416487+0000","last_change":"2023-10-21T16:01:55.953883+0000","last_active":"2023-10-21T16:02:01.416487+0000","last_peered":"2023-10-21T16:02:01.416487+0000","last_clean":"2023-10-21T16:02:01.416487+0000","last_became_active":"2023-10-21T16:01:55.953252+0000","last_became_peered":"2023-10-21T16:01:55.953252+0000","last_unstale":"2023-10-21T16:02:01.416487+0000","last_undegraded":"2023-10-21T16:02:01.416487+0000","last_fullsized":"2023-10-21T16:02:01.416487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416712+0000","last_change":"2023-10-21T16:01:55.955305+0000","last_active":"2023-10-21T16:02:01.416712+0000","last_peered":"2023-10-21T16:02:01.416712+0000","last_clean":"2023-10-21T16:02:01.416712+0000","last_became_active":"2023-10-21T16:01:55.955019+0000","last_became_peered":"2023-10-21T16:01:55.955019+0000","last_unstale":"2023-10-21T16:02:01.416712+0000","last_undegraded":"2023-10-21T16:02:01.416712+0000","last_fullsized":"2023-10-21T16:02:01.416712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416673+0000","last_change":"2023-10-21T16:01:55.958446+0000","last_active":"2023-10-21T16:02:01.416673+0000","last_peered":"2023-10-21T16:02:01.416673+0000","last_clean":"2023-10-21T16:02:01.416673+0000","last_became_active":"2023-10-21T16:01:55.958056+0000","last_became_peered":"2023-10-21T16:01:55.958056+0000","last_unstale":"2023-10-21T16:02:01.416673+0000","last_undegraded":"2023-10-21T16:02:01.416673+0000","last_fullsized":"2023-10-21T16:02:01.416673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945856+0000","last_change":"2023-10-21T16:01:55.942133+0000","last_active":"2023-10-21T16:02:00.945856+0000","last_peered":"2023-10-21T16:02:00.945856+0000","last_clean":"2023-10-21T16:02:00.945856+0000","last_became_active":"2023-10-21T16:01:55.941833+0000","last_became_peered":"2023-10-21T16:01:55.941833+0000","last_unstale":"2023-10-21T16:02:00.945856+0000","last_undegraded":"2023-10-21T16:02:00.945856+0000","last_fullsized":"2023-10-21T16:02:00.945856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414741+0000","last_change":"2023-10-21T16:01:55.940356+0000","last_active":"2023-10-21T16:02:01.414741+0000","last_peered":"2023-10-21T16:02:01.414741+0000","last_clean":"2023-10-21T16:02:01.414741+0000","last_became_active":"2023-10-21T16:01:55.939602+0000","last_became_peered":"2023-10-21T16:01:55.939602+0000","last_unstale":"2023-10-21T16:02:01.414741+0000","last_undegraded":"2023-10-21T16:02:01.414741+0000","last_fullsized":"2023-10-21T16:02:01.414741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004002+0000","last_change":"2023-10-21T16:01:55.952103+0000","last_active":"2023-10-21T16:02:01.004002+0000","last_peered":"2023-10-21T16:02:01.004002+0000","last_clean":"2023-10-21T16:02:01.004002+0000","last_became_active":"2023-10-21T16:01:55.951051+0000","last_became_peered":"2023-10-21T16:01:55.951051+0000","last_unstale":"2023-10-21T16:02:01.004002+0000","last_undegraded":"2023-10-21T16:02:01.004002+0000","last_fullsized":"2023-10-21T16:02:01.004002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414699+0000","last_change":"2023-10-21T16:01:55.940974+0000","last_active":"2023-10-21T16:02:01.414699+0000","last_peered":"2023-10-21T16:02:01.414699+0000","last_clean":"2023-10-21T16:02:01.414699+0000","last_became_active":"2023-10-21T16:01:55.940667+0000","last_became_peered":"2023-10-21T16:01:55.940667+0000","last_unstale":"2023-10-21T16:02:01.414699+0000","last_undegraded":"2023-10-21T16:02:01.414699+0000","last_fullsized":"2023-10-21T16:02:01.414699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004788+0000","last_change":"2023-10-21T16:01:55.954831+0000","last_active":"2023-10-21T16:02:01.004788+0000","last_peered":"2023-10-21T16:02:01.004788+0000","last_clean":"2023-10-21T16:02:01.004788+0000","last_became_active":"2023-10-21T16:01:55.954453+0000","last_became_peered":"2023-10-21T16:01:55.954453+0000","last_unstale":"2023-10-21T16:02:01.004788+0000","last_undegraded":"2023-10-21T16:02:01.004788+0000","last_fullsized":"2023-10-21T16:02:01.004788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928170+0000","last_change":"2023-10-21T16:01:55.936408+0000","last_active":"2023-10-21T16:02:00.928170+0000","last_peered":"2023-10-21T16:02:00.928170+0000","last_clean":"2023-10-21T16:02:00.928170+0000","last_became_active":"2023-10-21T16:01:55.935946+0000","last_became_peered":"2023-10-21T16:01:55.935946+0000","last_unstale":"2023-10-21T16:02:00.928170+0000","last_undegraded":"2023-10-21T16:02:00.928170+0000","last_fullsized":"2023-10-21T16:02:00.928170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913325+0000","last_change":"2023-10-21T16:01:55.938274+0000","last_active":"2023-10-21T16:02:00.913325+0000","last_peered":"2023-10-21T16:02:00.913325+0000","last_clean":"2023-10-21T16:02:00.913325+0000","last_became_active":"2023-10-21T16:01:55.937878+0000","last_became_peered":"2023-10-21T16:01:55.937878+0000","last_unstale":"2023-10-21T16:02:00.913325+0000","last_undegraded":"2023-10-21T16:02:00.913325+0000","last_fullsized":"2023-10-21T16:02:00.913325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416466+0000","last_change":"2023-10-21T16:01:55.945925+0000","last_active":"2023-10-21T16:02:01.416466+0000","last_peered":"2023-10-21T16:02:01.416466+0000","last_clean":"2023-10-21T16:02:01.416466+0000","last_became_active":"2023-10-21T16:01:55.945457+0000","last_became_peered":"2023-10-21T16:01:55.945457+0000","last_unstale":"2023-10-21T16:02:01.416466+0000","last_undegraded":"2023-10-21T16:02:01.416466+0000","last_fullsized":"2023-10-21T16:02:01.416466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":702466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6539999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0219999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.258}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1509999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8260000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5940000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":111642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":111516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":111676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":702522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.288}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:02:25.005 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:02:25.006 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-21T16:02:25.006 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-21T16:02:25.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-21T16:02:25.007 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-21T16:02:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.271+0000 7fa77f435640 1 Processor -- start 2023-10-21T16:02:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.271+0000 7fa77f435640 1 -- start start 2023-10-21T16:02:25.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77f435640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 0x7fa7780cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77f435640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa778164f70 0x7fa7780c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77f435640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa778165470 con 0x7fa778161320 2023-10-21T16:02:25.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77f435640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa778165750 con 0x7fa778164f70 2023-10-21T16:02:25.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77f435640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa778165a30 con 0x7fa7780b07d0 2023-10-21T16:02:25.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77ec34640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa778164f70 0x7fa7780c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77ec34640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa778164f70 0x7fa7780c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59424/0 (socket says 172.21.15.53:59424) 2023-10-21T16:02:25.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77ec34640 1 -- 172.21.15.53:0/2768842094 learned_addr learned my addr 172.21.15.53:0/2768842094 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:25.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.273+0000 7fa77dc32640 1 --2- 172.21.15.53:0/2768842094 >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 0x7fa7780cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.274+0000 7fa77ec34640 1 -- 172.21.15.53:0/2768842094 >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 msgr2=0x7fa7780cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.274+0000 7fa77ec34640 1 --2- 172.21.15.53:0/2768842094 >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 0x7fa7780cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.274+0000 7fa77ec34640 1 -- 172.21.15.53:0/2768842094 >> v1:172.21.15.124:6789/0 conn(0x7fa778161320 legacy=0x7fa7780b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.274+0000 7fa77ec34640 1 -- 172.21.15.53:0/2768842094 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778165d10 con 0x7fa778164f70 2023-10-21T16:02:25.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.274+0000 7fa77dc32640 1 --2- 172.21.15.53:0/2768842094 >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 0x7fa7780cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:25.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.275+0000 7fa77ec34640 1 --2- 172.21.15.53:0/2768842094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778164f70 0x7fa7780c95b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cbe60 tx=0x7fa77002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a811762b8881fa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.275+0000 7fa77d431640 1 -- 172.21.15.53:0/2768842094 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77002cb20 con 0x7fa778164f70 2023-10-21T16:02:25.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.275+0000 7fa77d431640 1 -- 172.21.15.53:0/2768842094 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77002cc80 con 0x7fa778164f70 2023-10-21T16:02:25.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.275+0000 7fa77d431640 1 -- 172.21.15.53:0/2768842094 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7700343f0 con 0x7fa778164f70 2023-10-21T16:02:25.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 -- 172.21.15.53:0/2768842094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778164f70 msgr2=0x7fa7780c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 --2- 172.21.15.53:0/2768842094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778164f70 0x7fa7780c95b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cbe60 tx=0x7fa77002bf80 comp rx=0 tx=0).stop 2023-10-21T16:02:25.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 -- 172.21.15.53:0/2768842094 shutdown_connections 2023-10-21T16:02:25.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 --2- 172.21.15.53:0/2768842094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778164f70 0x7fa7780c95b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 --2- 172.21.15.53:0/2768842094 >> v2:172.21.15.124:3300/0 conn(0x7fa7780b07d0 0x7fa7780cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 -- 172.21.15.53:0/2768842094 >> 172.21.15.53:0/2768842094 conn(0x7fa7780cb0c0 msgr2=0x7fa7780cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:25.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 -- 172.21.15.53:0/2768842094 shutdown_connections 2023-10-21T16:02:25.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.276+0000 7fa77f435640 1 -- 172.21.15.53:0/2768842094 wait complete. 2023-10-21T16:02:25.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.278+0000 7fa77f435640 1 Processor -- start 2023-10-21T16:02:25.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.278+0000 7fa77f435640 1 -- start start 2023-10-21T16:02:25.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7780b07d0 0x7fa7781fa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 0x7fa7781f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781faa50 con 0x7fa778161320 2023-10-21T16:02:25.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780c88b0 con 0x7fa778164f70 2023-10-21T16:02:25.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77f435640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7781f6f20 con 0x7fa7780b07d0 2023-10-21T16:02:25.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77dc32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77dc32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59440/0 (socket says 172.21.15.53:59440) 2023-10-21T16:02:25.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.279+0000 7fa77dc32640 1 -- 172.21.15.53:0/2217985187 learned_addr learned my addr 172.21.15.53:0/2217985187 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:25.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77ec34640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 0x7fa7781f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77e433640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7780b07d0 0x7fa7781fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 -- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7780b07d0 msgr2=0x7fa7781fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7780b07d0 0x7fa7781fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 -- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 msgr2=0x7fa7781f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 0x7fa7781f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 -- 172.21.15.53:0/2217985187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770009b30 con 0x7fa778161320 2023-10-21T16:02:25.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77ec34640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 0x7fa7781f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:25.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77dc32640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e760 tx=0x7fa76c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c004230 con 0x7fa778161320 2023-10-21T16:02:25.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c004390 con 0x7fa778161320 2023-10-21T16:02:25.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00c8f0 con 0x7fa778161320 2023-10-21T16:02:25.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.280+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7781fac40 con 0x7fa778161320 2023-10-21T16:02:25.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.281+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7781fb200 con 0x7fa778161320 2023-10-21T16:02:25.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.282+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77804e160 con 0x7fa778161320 2023-10-21T16:02:25.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.283+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa76c00ca50 con 0x7fa778161320 2023-10-21T16:02:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.284+0000 7fa76affd640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 0x7fa75c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.284+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa76c0895c0 con 0x7fa778161320 2023-10-21T16:02:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.284+0000 7fa77e433640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 0x7fa75c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.285+0000 7fa77e433640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 0x7fa75c06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa774004ea0 tx=0x7fa774009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.288+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa76c0542e0 con 0x7fa778161320 2023-10-21T16:02:25.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.530+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa77815f270 con 0x7fa778161320 2023-10-21T16:02:25.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.534+0000 7fa76affd640 1 -- 172.21.15.53:0/2217985187 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171036 (secure 0 0 0) 0x7fa76c057900 con 0x7fa778161320 2023-10-21T16:02:25.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 msgr2=0x7fa75c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 0x7fa75c06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa774004ea0 tx=0x7fa774009040 comp rx=0 tx=0).stop 2023-10-21T16:02:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 msgr2=0x7fa7781509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e760 tx=0x7fa76c00ec20 comp rx=0 tx=0).stop 2023-10-21T16:02:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 shutdown_connections 2023-10-21T16:02:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa778164f70 0x7fa7781f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa75c06ae70 0x7fa75c06d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa778161320 0x7fa7781509d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 --2- 172.21.15.53:0/2217985187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7780b07d0 0x7fa7781fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 >> 172.21.15.53:0/2217985187 conn(0x7fa7780cb0c0 msgr2=0x7fa77815f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.539+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 shutdown_connections 2023-10-21T16:02:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.540+0000 7fa77f435640 1 -- 172.21.15.53:0/2217985187 wait complete. 2023-10-21T16:02:25.562 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3409974301},{"type":"v1","addr":"172.21.15.124:6833","nonce":3409974301}]},"active_addr":"172.21.15.124:6833/3409974301","active_change":"2023-10-21T16:01:40.992438+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":269363197}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":2484781998}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":2311155348}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":1953414266}]}]} 2023-10-21T16:02:25.566 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-21T16:02:25.566 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-21T16:02:25.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.787+0000 7f9b768e0640 1 Processor -- start 2023-10-21T16:02:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.787+0000 7f9b768e0640 1 -- start start 2023-10-21T16:02:25.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b768e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b700702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b768e0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9b7006c890 0x7f9b7006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b768e0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b700c48f0 con 0x7f9b70160940 2023-10-21T16:02:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b768e0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b700c4d80 con 0x7f9b7006c890 2023-10-21T16:02:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b768e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c3ce0 con 0x7f9b7015cd90 2023-10-21T16:02:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b67fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9b70160940 0x7f9b70070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59634/0 (socket says 172.21.15.53:59634) 2023-10-21T16:02:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.789+0000 7f9b67fff640 1 -- 172.21.15.53:0/403924207 learned_addr learned my addr 172.21.15.53:0/403924207 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b700702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b74e7d640 1 --2- 172.21.15.53:0/403924207 >> v2:172.21.15.124:3300/0 conn(0x7f9b7006c890 0x7f9b7006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596083323 0 0) 0x7f9b700c48f0 con 0x7f9b70160940 2023-10-21T16:02:25.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b5c003520 con 0x7f9b70160940 2023-10-21T16:02:25.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665301962 0 0) 0x7f9b5c003520 con 0x7f9b70160940 2023-10-21T16:02:25.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 msgr2=0x7f9b700702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b700702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 >> v2:172.21.15.124:3300/0 conn(0x7f9b7006c890 msgr2=0x7f9b7006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 --2- 172.21.15.53:0/403924207 >> v2:172.21.15.124:3300/0 conn(0x7f9b7006c890 0x7f9b7006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700c3fc0 con 0x7f9b70160940 2023-10-21T16:02:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9b58001670 con 0x7f9b70160940 2023-10-21T16:02:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 >> v1:172.21.15.53:6789/0 conn(0x7f9b70160940 legacy=0x7f9b70070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.790+0000 7f9b6f7fe640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6f7fe640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0047c0 0x7f9b5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6f7fe640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c3ce0 con 0x7f9b5c003970 2023-10-21T16:02:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c00b980 con 0x7f9b5c0047c0 2023-10-21T16:02:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c00bac0 con 0x7f9b7015cd90 2023-10-21T16:02:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b67fff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0047c0 0x7f9b5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b74e7d640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 -- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 msgr2=0x7f9b5c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b7015cd90 0x7f9b5c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 -- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0047c0 msgr2=0x7f9b5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5c0047c0 0x7f9b5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.791+0000 7f9b6ffff640 1 -- 172.21.15.53:0/403924207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c003520 con 0x7f9b5c003970 2023-10-21T16:02:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b5c003d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009ac0 tx=0x7f9b5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5418b434f3947a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6002c6b0 con 0x7f9b5c003970 2023-10-21T16:02:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6002c810 con 0x7f9b5c003970 2023-10-21T16:02:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b6f7fe640 1 -- 172.21.15.53:0/403924207 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6002cba0 con 0x7f9b5c003970 2023-10-21T16:02:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b768e0640 1 -- 172.21.15.53:0/403924207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 msgr2=0x7f9b5c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b768e0640 1 --2- 172.21.15.53:0/403924207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b5c003d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009ac0 tx=0x7f9b5c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b74e7d640 1 -- 172.21.15.53:0/403924207 reap_dead start 2023-10-21T16:02:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b768e0640 1 -- 172.21.15.53:0/403924207 shutdown_connections 2023-10-21T16:02:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.793+0000 7f9b768e0640 1 -- 172.21.15.53:0/403924207 >> 172.21.15.53:0/403924207 conn(0x7f9b7006a380 msgr2=0x7f9b700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:25.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.794+0000 7f9b768e0640 1 -- 172.21.15.53:0/403924207 shutdown_connections 2023-10-21T16:02:25.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.794+0000 7f9b768e0640 1 -- 172.21.15.53:0/403924207 wait complete. 2023-10-21T16:02:25.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.795+0000 7f9b768e0640 1 Processor -- start 2023-10-21T16:02:25.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.796+0000 7f9b768e0640 1 -- start start 2023-10-21T16:02:25.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b700c73e0 0x7f9b700ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b700ace10 0x7f9b70149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700ad2a0 con 0x7f9b5c003970 2023-10-21T16:02:25.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c84d0 con 0x7f9b700c73e0 2023-10-21T16:02:25.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b768e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c78c0 con 0x7f9b700ace10 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59466/0 (socket says 172.21.15.53:59466) 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 -- 172.21.15.53:0/2345881188 learned_addr learned my addr 172.21.15.53:0/2345881188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b67fff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b700c73e0 0x7f9b700ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b74e7d640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b700ace10 0x7f9b70149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 -- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b700ace10 msgr2=0x7f9b70149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b700ace10 0x7f9b70149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 -- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b700c73e0 msgr2=0x7f9b700ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:25.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b700c73e0 0x7f9b700ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:25.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.797+0000 7f9b6ffff640 1 -- 172.21.15.53:0/2345881188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700c7ba0 con 0x7f9b5c003970 2023-10-21T16:02:25.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b6ffff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009ac0 tx=0x7f9b60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60003810 con 0x7f9b5c003970 2023-10-21T16:02:25.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b60003970 con 0x7f9b5c003970 2023-10-21T16:02:25.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b60003d00 con 0x7f9b5c003970 2023-10-21T16:02:25.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b700c7e80 con 0x7f9b5c003970 2023-10-21T16:02:25.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.798+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7015f460 con 0x7f9b5c003970 2023-10-21T16:02:25.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.799+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b70067490 con 0x7f9b5c003970 2023-10-21T16:02:25.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.801+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b60040450 con 0x7f9b5c003970 2023-10-21T16:02:25.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.801+0000 7f9b6d7fa640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 0x7f9b5006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:25.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.802+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9b600abca0 con 0x7f9b5c003970 2023-10-21T16:02:25.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.802+0000 7f9b67fff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 0x7f9b5006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:25.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.803+0000 7f9b67fff640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 0x7f9b5006d360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b58001eb0 tx=0x7f9b58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:25.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:25.806+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b600769c0 con 0x7f9b5c003970 2023-10-21T16:02:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.007+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b700b0ef0 con 0x7f9b5c003970 2023-10-21T16:02:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.012+0000 7f9b6d7fa640 1 -- 172.21.15.53:0/2345881188 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f9b600044e0 con 0x7f9b5c003970 2023-10-21T16:02:26.014 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:26.014 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:26.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 msgr2=0x7f9b5006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 0x7f9b5006d360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b58001eb0 tx=0x7f9b58009040 comp rx=0 tx=0).stop 2023-10-21T16:02:26.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 msgr2=0x7f9b700c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009ac0 tx=0x7f9b60004730 comp rx=0 tx=0).stop 2023-10-21T16:02:26.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 shutdown_connections 2023-10-21T16:02:26.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b700ace10 0x7f9b70149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b700c73e0 0x7f9b700ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9b5006aea0 0x7f9b5006d360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 --2- 172.21.15.53:0/2345881188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b5c003970 0x7f9b700c5850 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 >> 172.21.15.53:0/2345881188 conn(0x7f9b7006a380 msgr2=0x7f9b700abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 shutdown_connections 2023-10-21T16:02:26.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.015+0000 7f9b768e0640 1 -- 172.21.15.53:0/2345881188 wait complete. 2023-10-21T16:02:26.036 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-21T16:02:26.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.259+0000 7f3cd3301640 1 Processor -- start 2023-10-21T16:02:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.260+0000 7f3cd3301640 1 -- start start 2023-10-21T16:02:26.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.261+0000 7f3cd3301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc06ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.261+0000 7f3cd3301640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.261+0000 7f3cd3301640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0b0340 con 0x7f3ccc06b520 2023-10-21T16:02:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.261+0000 7f3cd3301640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc06b3a0 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.261+0000 7f3cd3301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b0150 con 0x7f3ccc0af8b0 2023-10-21T16:02:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59474/0 (socket says 172.21.15.53:59474) 2023-10-21T16:02:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 -- 172.21.15.53:0/2317581995 learned_addr learned my addr 172.21.15.53:0/2317581995 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd089c640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc06ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 -- 172.21.15.53:0/2317581995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 msgr2=0x7f3ccc06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc06ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 -- 172.21.15.53:0/2317581995 >> v1:172.21.15.124:6789/0 conn(0x7f3ccc06b520 legacy=0x7f3ccc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.262+0000 7f3cd189e640 1 -- 172.21.15.53:0/2317581995 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0cf8f0 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.263+0000 7f3cd189e640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009e50 tx=0x7f3cc402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec3c83d1412cf45 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cc3fff640 1 -- 172.21.15.53:0/2317581995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc402ca60 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cc3fff640 1 -- 172.21.15.53:0/2317581995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc402cbc0 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cc3fff640 1 -- 172.21.15.53:0/2317581995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40343f0 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 -- 172.21.15.53:0/2317581995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc0b0ab0 msgr2=0x7f3ccc0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009e50 tx=0x7f3cc402bec0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 -- 172.21.15.53:0/2317581995 shutdown_connections 2023-10-21T16:02:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc06ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2317581995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc0b0f40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 -- 172.21.15.53:0/2317581995 >> 172.21.15.53:0/2317581995 conn(0x7f3ccc0d09e0 msgr2=0x7f3ccc0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.264+0000 7f3cd3301640 1 -- 172.21.15.53:0/2317581995 shutdown_connections 2023-10-21T16:02:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.265+0000 7f3cd3301640 1 -- 172.21.15.53:0/2317581995 wait complete. 2023-10-21T16:02:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.266+0000 7f3cd3301640 1 Processor -- start 2023-10-21T16:02:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.266+0000 7f3cd3301640 1 -- start start 2023-10-21T16:02:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc160170 con 0x7f3ccc06b520 2023-10-21T16:02:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c6c80 con 0x7f3ccc0b0ab0 2023-10-21T16:02:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.267+0000 7f3cd3301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0c6f60 con 0x7f3ccc0af8b0 2023-10-21T16:02:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59484/0 (socket says 172.21.15.53:59484) 2023-10-21T16:02:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 -- 172.21.15.53:0/2148643960 learned_addr learned my addr 172.21.15.53:0/2148643960 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd189e640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd089c640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 -- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 msgr2=0x7f3ccc0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 -- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 msgr2=0x7f3ccc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 -- 172.21.15.53:0/2148643960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009b30 con 0x7f3ccc06b520 2023-10-21T16:02:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd189e640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.268+0000 7f3cd109d640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c3a0 tx=0x7f3cc800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.269+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80052d0 con 0x7f3ccc06b520 2023-10-21T16:02:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.269+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8005450 con 0x7f3ccc06b520 2023-10-21T16:02:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.269+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8003880 con 0x7f3ccc06b520 2023-10-21T16:02:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.269+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc0c7240 con 0x7f3ccc06b520 2023-10-21T16:02:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.269+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0c4a50 con 0x7f3ccc06b520 2023-10-21T16:02:26.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.270+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc0c7760 con 0x7f3ccc06b520 2023-10-21T16:02:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.271+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3cc80039e0 con 0x7f3ccc06b520 2023-10-21T16:02:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.272+0000 7f3cc1ffb640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 0x7f3cb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.272+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cc8088ef0 con 0x7f3ccc06b520 2023-10-21T16:02:26.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.273+0000 7f3cd089c640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 0x7f3cb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.274+0000 7f3cd089c640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 0x7f3cb006d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006fd0 tx=0x7f3cbc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.277+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ccc0c7760 con 0x7f3ccc06b520 2023-10-21T16:02:26.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.479+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ccc0c7760 con 0x7f3ccc06b520 2023-10-21T16:02:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.482+0000 7f3cc1ffb640 1 -- 172.21.15.53:0/2148643960 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f3ccc0c7760 con 0x7f3ccc06b520 2023-10-21T16:02:26.484 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:26.485 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 msgr2=0x7f3cb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 0x7f3cb006d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc006fd0 tx=0x7f3cbc008040 comp rx=0 tx=0).stop 2023-10-21T16:02:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 msgr2=0x7f3ccc0c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c3a0 tx=0x7f3cc800c860 comp rx=0 tx=0).stop 2023-10-21T16:02:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 shutdown_connections 2023-10-21T16:02:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ccc0b0ab0 0x7f3ccc1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ccc0af8b0 0x7f3ccc0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3cb006ae70 0x7f3cb006d330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 --2- 172.21.15.53:0/2148643960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ccc06b520 0x7f3ccc0c4510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 >> 172.21.15.53:0/2148643960 conn(0x7f3ccc0d09e0 msgr2=0x7f3ccc0d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 shutdown_connections 2023-10-21T16:02:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.485+0000 7f3cd3301640 1 -- 172.21.15.53:0/2148643960 wait complete. 2023-10-21T16:02:26.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-21T16:02:26.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-21T16:02:26.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-21T16:02:26.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-21T16:02:26.511 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-21T16:02:26.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-21T16:02:26.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-21T16:02:26.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-21T16:02:26.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.835+0000 7faf17fff640 1 Processor -- start 2023-10-21T16:02:26.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.835+0000 7faf17fff640 1 -- start start 2023-10-21T16:02:26.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf10055400 0x7faf100557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf100543c0 0x7faf10039140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf10049ee0 0x7faf10051930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf100e8920 con 0x7faf100543c0 2023-10-21T16:02:26.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf10055d20 con 0x7faf10055400 2023-10-21T16:02:26.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.837+0000 7faf17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100532c0 con 0x7faf10049ee0 2023-10-21T16:02:26.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.839+0000 7faf16ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf10055400 0x7faf100557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.839+0000 7faf167fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf100543c0 0x7faf10039140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.839+0000 7faf167fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf100543c0 0x7faf10039140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59486/0 (socket says 172.21.15.53:59486) 2023-10-21T16:02:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.839+0000 7faf167fc640 1 -- 172.21.15.53:0/3374790332 learned_addr learned my addr 172.21.15.53:0/3374790332 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf177fe640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf10049ee0 0x7faf10051930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf167fc640 1 -- 172.21.15.53:0/3374790332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf10049ee0 msgr2=0x7faf10051930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf167fc640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf10049ee0 0x7faf10051930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf167fc640 1 -- 172.21.15.53:0/3374790332 >> v2:172.21.15.124:3300/0 conn(0x7faf10055400 msgr2=0x7faf100557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf167fc640 1 --2- 172.21.15.53:0/3374790332 >> v2:172.21.15.124:3300/0 conn(0x7faf10055400 0x7faf100557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.840+0000 7faf167fc640 1 -- 172.21.15.53:0/3374790332 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100e8e10 con 0x7faf100543c0 2023-10-21T16:02:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.842+0000 7faf167fc640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf100543c0 0x7faf10039140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf10057ef0 tx=0x7faf0c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc522ef2bdcb5dba server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.842+0000 7faf15ffb640 1 -- 172.21.15.53:0/3374790332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c030850 con 0x7faf100543c0 2023-10-21T16:02:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.842+0000 7faf15ffb640 1 -- 172.21.15.53:0/3374790332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c0309b0 con 0x7faf100543c0 2023-10-21T16:02:26.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.842+0000 7faf15ffb640 1 -- 172.21.15.53:0/3374790332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c030d40 con 0x7faf100543c0 2023-10-21T16:02:26.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.843+0000 7f347059e640 1 Processor -- start 2023-10-21T16:02:26.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.843+0000 7f347059e640 1 -- start start 2023-10-21T16:02:26.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 -- 172.21.15.53:0/3374790332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf100543c0 msgr2=0x7faf10039140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf100543c0 0x7faf10039140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf10057ef0 tx=0x7faf0c02b990 comp rx=0 tx=0).stop 2023-10-21T16:02:26.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 -- 172.21.15.53:0/3374790332 shutdown_connections 2023-10-21T16:02:26.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf10049ee0 0x7faf10051930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 --2- 172.21.15.53:0/3374790332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf100543c0 0x7faf10039140 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf10057ef0 tx=0x7faf0c02b990 comp rx=0 tx=0).stop 2023-10-21T16:02:26.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 --2- 172.21.15.53:0/3374790332 >> v2:172.21.15.124:3300/0 conn(0x7faf10055400 0x7faf100557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.844+0000 7faf17fff640 1 -- 172.21.15.53:0/3374790332 >> 172.21.15.53:0/3374790332 conn(0x7faf100538a0 msgr2=0x7faf10053cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f0fd9705640 1 Processor -- start 2023-10-21T16:02:26.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7faf17fff640 1 -- 172.21.15.53:0/3374790332 shutdown_connections 2023-10-21T16:02:26.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f347059e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 0x7f346806cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f347059e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34680cb3c0 con 0x7f34680cede0 2023-10-21T16:02:26.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f347059e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34680cb6a0 con 0x7f34681591d0 2023-10-21T16:02:26.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f347059e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34680cb980 con 0x7f34680ca220 2023-10-21T16:02:26.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.845+0000 7f0fd9705640 1 -- start start 2023-10-21T16:02:26.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.846+0000 7f346db39640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f34680cede0 0x7f346806c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59650/0 (socket says 172.21.15.53:59650) 2023-10-21T16:02:26.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.846+0000 7f346e33a640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f34681591d0 0x7f34680ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39476/0 (socket says 172.21.15.53:39476) 2023-10-21T16:02:26.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.846+0000 7f346db39640 1 -- 172.21.15.53:0/3046807226 learned_addr learned my addr 172.21.15.53:0/3046807226 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.846+0000 7f346eb3b640 1 --2- 172.21.15.53:0/3046807226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 0x7f346806cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd9705640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0fd40cdf70 0x7f0fd4167830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd9705640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd415d160 con 0x7f0fd40b87d0 2023-10-21T16:02:26.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd9705640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40b7b90 con 0x7f0fd415cd80 2023-10-21T16:02:26.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd9705640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40b8020 con 0x7f0fd40cdf70 2023-10-21T16:02:26.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f57151ff640 1 Processor -- start 2023-10-21T16:02:26.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0fd40cdf70 0x7f0fd4167830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0fd40cdf70 0x7f0fd4167830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59488/0 (socket says 172.21.15.53:59488) 2023-10-21T16:02:26.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f0fd37fe640 1 -- 172.21.15.53:0/3523542045 learned_addr learned my addr 172.21.15.53:0/3523542045 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f57151ff640 1 -- start start 2023-10-21T16:02:26.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7f346d338640 1 -- 172.21.15.53:0/3046807226 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834311436 0 0) 0x7f34680cb6a0 con 0x7f34681591d0 2023-10-21T16:02:26.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f346eb3b640 1 -- 172.21.15.53:0/3046807226 >> v1:172.21.15.124:6789/0 conn(0x7f34681591d0 legacy=0x7f34680ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f0fd37fe640 1 -- 172.21.15.53:0/3523542045 >> v1:172.21.15.124:6789/0 conn(0x7f0fd415cd80 legacy=0x7f0fd40b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f0fd37fe640 1 -- 172.21.15.53:0/3523542045 >> v1:172.21.15.53:6789/0 conn(0x7f0fd40b87d0 legacy=0x7f0fd40cd860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:02:26.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f346eb3b640 1 -- 172.21.15.53:0/3046807226 >> v1:172.21.15.53:6789/0 conn(0x7f34680cede0 legacy=0x7f346806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.848+0000 7f346eb3b640 1 -- 172.21.15.53:0/3046807226 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680cbc60 con 0x7f34680ca220 2023-10-21T16:02:26.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.847+0000 7faf17fff640 1 -- 172.21.15.53:0/3374790332 wait complete. 2023-10-21T16:02:26.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.849+0000 7f0fd37fe640 1 -- 172.21.15.53:0/3523542045 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40b6f80 con 0x7f0fd40cdf70 2023-10-21T16:02:26.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f346eb3b640 1 --2- 172.21.15.53:0/3046807226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 0x7f346806cd90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f34580097d0 tx=0x7f345802db50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a632a2cff40ed61 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f57151ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57100cf2b0 0x7f57100d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f57151ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100cd520 con 0x7f57100ce510 2023-10-21T16:02:26.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f57151ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100d1910 con 0x7f57100cf690 2023-10-21T16:02:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f57151ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100ce080 con 0x7f57100cf2b0 2023-10-21T16:02:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f346d338640 1 -- 172.21.15.53:0/3046807226 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345802f910 con 0x7f34680ca220 2023-10-21T16:02:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f346d338640 1 -- 172.21.15.53:0/3046807226 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345802fa70 con 0x7f34680ca220 2023-10-21T16:02:26.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f346d338640 1 -- 172.21.15.53:0/3046807226 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345802fe00 con 0x7f34680ca220 2023-10-21T16:02:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.851+0000 7f57077fe640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f57100cf690 0x7f57100d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39498/0 (socket says 172.21.15.53:39498) 2023-10-21T16:02:26.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.851+0000 7f57077fe640 1 -- 172.21.15.53:0/3017464014 learned_addr learned my addr 172.21.15.53:0/3017464014 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.850+0000 7f0fd37fe640 1 --2- 172.21.15.53:0/3523542045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4167830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009870 tx=0x7f0fc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9584a4b218d9c2b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.851+0000 7f570ffff640 1 --2- 172.21.15.53:0/3017464014 >> v2:172.21.15.53:3300/0 conn(0x7f57100cf2b0 0x7f57100d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.851+0000 7f0fd1ffb640 1 -- 172.21.15.53:0/3523542045 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4031870 con 0x7f0fd40cdf70 2023-10-21T16:02:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f570ffff640 1 -- 172.21.15.53:0/3017464014 >> v1:172.21.15.124:6789/0 conn(0x7f57100cf690 legacy=0x7f57100d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f0fd1ffb640 1 -- 172.21.15.53:0/3523542045 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40319d0 con 0x7f0fd40cdf70 2023-10-21T16:02:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f347059e640 1 -- 172.21.15.53:0/3046807226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 msgr2=0x7f346806cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f347059e640 1 --2- 172.21.15.53:0/3046807226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 0x7f346806cd90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f34580097d0 tx=0x7f345802db50 comp rx=0 tx=0).stop 2023-10-21T16:02:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f570ffff640 1 -- 172.21.15.53:0/3017464014 >> v1:172.21.15.53:6789/0 conn(0x7f57100ce510 legacy=0x7f571015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f0fd9705640 1 -- 172.21.15.53:0/3523542045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd40cdf70 msgr2=0x7f0fd4167830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f0fd1ffb640 1 -- 172.21.15.53:0/3523542045 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4031870 con 0x7f0fd40cdf70 2023-10-21T16:02:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f0fd9705640 1 --2- 172.21.15.53:0/3523542045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4167830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009870 tx=0x7f0fc402b910 comp rx=0 tx=0).stop 2023-10-21T16:02:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.852+0000 7f570ffff640 1 -- 172.21.15.53:0/3017464014 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100cd750 con 0x7f57100cf2b0 2023-10-21T16:02:26.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f347059e640 1 -- 172.21.15.53:0/3046807226 shutdown_connections 2023-10-21T16:02:26.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f347059e640 1 --2- 172.21.15.53:0/3046807226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680ca220 0x7f346806cd90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f347059e640 1 -- 172.21.15.53:0/3046807226 >> 172.21.15.53:0/3046807226 conn(0x7f34680660e0 msgr2=0x7f3468069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7faf17fff640 1 Processor -- start 2023-10-21T16:02:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f0fd9705640 1 -- 172.21.15.53:0/3523542045 shutdown_connections 2023-10-21T16:02:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f0fd9705640 1 --2- 172.21.15.53:0/3523542045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4167830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.853+0000 7f0fd9705640 1 -- 172.21.15.53:0/3523542045 >> 172.21.15.53:0/3523542045 conn(0x7f0fd406b5d0 msgr2=0x7f0fd406b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.854+0000 7f347059e640 1 -- 172.21.15.53:0/3046807226 shutdown_connections 2023-10-21T16:02:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.854+0000 7f347059e640 1 -- 172.21.15.53:0/3046807226 wait complete. 2023-10-21T16:02:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.854+0000 7faf17fff640 1 -- start start 2023-10-21T16:02:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7f0fd9705640 1 -- 172.21.15.53:0/3523542045 shutdown_connections 2023-10-21T16:02:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 0x7faf100e4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7f0fd9705640 1 -- 172.21.15.53:0/3523542045 wait complete. 2023-10-21T16:02:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf10055400 0x7faf100e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1004bbe0 0x7faf1004c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100d9150 con 0x7faf10049ee0 2023-10-21T16:02:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1004c5b0 con 0x7faf10055400 2023-10-21T16:02:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100d5b70 con 0x7faf1004bbe0 2023-10-21T16:02:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 0x7faf100e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 0x7faf100e4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59504/0 (socket says 172.21.15.53:59504) 2023-10-21T16:02:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7f570ffff640 1 --2- 172.21.15.53:0/3017464014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100d34d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5700009d90 tx=0x7f570002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3e6689ec851239 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7f39ec95a640 1 Processor -- start 2023-10-21T16:02:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.855+0000 7faf16ffd640 1 -- 172.21.15.53:0/71787858 learned_addr learned my addr 172.21.15.53:0/71787858 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 -- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1004bbe0 msgr2=0x7faf1004c070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:02:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1004bbe0 0x7faf1004c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 -- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf10055400 msgr2=0x7faf100e6850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-21T16:02:26.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7f39ec95a640 1 -- start start 2023-10-21T16:02:26.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf10055400 0x7faf100e6850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0094d0 con 0x7faf10049ee0 2023-10-21T16:02:26.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.856+0000 7faf16ffd640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 0x7faf100e4290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e870 tx=0x7faf0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7f570f7fe640 1 -- 172.21.15.53:0/3017464014 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002cc60 con 0x7f57100cf2b0 2023-10-21T16:02:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7f570f7fe640 1 -- 172.21.15.53:0/3017464014 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570002cdc0 con 0x7f57100cf2b0 2023-10-21T16:02:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7f570f7fe640 1 -- 172.21.15.53:0/3017464014 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700034660 con 0x7f57100cf2b0 2023-10-21T16:02:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800c8e0 con 0x7faf10049ee0 2023-10-21T16:02:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0800ca40 con 0x7faf10049ee0 2023-10-21T16:02:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800cdd0 con 0x7faf10049ee0 2023-10-21T16:02:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f57151ff640 1 -- 172.21.15.53:0/3017464014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 msgr2=0x7f57100d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f57151ff640 1 --2- 172.21.15.53:0/3017464014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100d34d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5700009d90 tx=0x7f570002beb0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f57151ff640 1 -- 172.21.15.53:0/3017464014 shutdown_connections 2023-10-21T16:02:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f57151ff640 1 --2- 172.21.15.53:0/3017464014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100d34d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f57151ff640 1 -- 172.21.15.53:0/3017464014 >> 172.21.15.53:0/3017464014 conn(0x7f571009bc40 msgr2=0x7f57100d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf100d8e70 con 0x7faf10049ee0 2023-10-21T16:02:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f57151ff640 1 -- 172.21.15.53:0/3017464014 shutdown_connections 2023-10-21T16:02:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f0fd9705640 1 Processor -- start 2023-10-21T16:02:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf100e6050 con 0x7faf10049ee0 2023-10-21T16:02:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f7f7df4e640 1 Processor -- start 2023-10-21T16:02:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f0fd9705640 1 -- start start 2023-10-21T16:02:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7f347059e640 1 Processor -- start 2023-10-21T16:02:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.857+0000 7f347059e640 1 -- start start 2023-10-21T16:02:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f347059e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34680ca220 0x7f3468152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f347059e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 0x7f34681f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f347059e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34681591d0 0x7f34680cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.858+0000 7f347059e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34680c6eb0 con 0x7f34680ca220 2023-10-21T16:02:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f347059e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34681f77c0 con 0x7f34681591d0 2023-10-21T16:02:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f347059e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34680c4cb0 con 0x7f34680cede0 2023-10-21T16:02:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f346db39640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 0x7f34681f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f346db39640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 0x7f34681f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57866/0 (socket says 172.21.15.53:57866) 2023-10-21T16:02:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f346db39640 1 -- 172.21.15.53:0/1386123676 learned_addr learned my addr 172.21.15.53:0/1386123676 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f346eb3b640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34681591d0 0x7f34680cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.860+0000 7f7f7df4e640 1 -- start start 2023-10-21T16:02:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.860+0000 7f346e33a640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34680ca220 0x7f3468152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39ec95a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e80d1ef0 0x7f39e81679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.859+0000 7f57151ff640 1 -- 172.21.15.53:0/3017464014 wait complete. 2023-10-21T16:02:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f57151ff640 1 Processor -- start 2023-10-21T16:02:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f346db39640 1 -- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34681591d0 msgr2=0x7f34680cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f346db39640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34681591d0 0x7f34680cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f346db39640 1 -- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34680ca220 msgr2=0x7f3468152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f346db39640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34680ca220 0x7f3468152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f346db39640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009400 con 0x7f34680cede0 2023-10-21T16:02:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f57151ff640 1 -- start start 2023-10-21T16:02:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39ec95a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e80b54a0 con 0x7f39e815cf80 2023-10-21T16:02:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39ec95a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e80b7130 con 0x7f39e815cba0 2023-10-21T16:02:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39ec95a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b8bb0 con 0x7f39e80d1ef0 2023-10-21T16:02:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39e5d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39e815cf80 0x7f39e80b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59684/0 (socket says 172.21.15.53:59684) 2023-10-21T16:02:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.861+0000 7f39e5d9b640 1 -- 172.21.15.53:0/3458424426 learned_addr learned my addr 172.21.15.53:0/3458424426 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585357960 0 0) 0x7f39d8001110 con 0x7f39e815cf80 2023-10-21T16:02:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d0003520 con 0x7f39e815cf80 2023-10-21T16:02:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f7f7df4e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 0x7f7f7003ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f7f7df4e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7003c070 con 0x7f7f70006b70 2023-10-21T16:02:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f7f7df4e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700070f0 con 0x7f7f70054360 2023-10-21T16:02:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f7f7df4e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7003ef80 con 0x7f7f700e3070 2023-10-21T16:02:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf10038940 con 0x7faf10049ee0 2023-10-21T16:02:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f0fd9705640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fd40b87d0 0x7f0fd414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584361410 0 0) 0x7f39d0003520 con 0x7f39e815cf80 2023-10-21T16:02:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e80d1ef0 msgr2=0x7f39e81679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e80d1ef0 0x7f39e81679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 >> v1:172.21.15.124:6789/0 conn(0x7f39e815cba0 legacy=0x7f39e80b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80b7410 con 0x7f39e815cf80 2023-10-21T16:02:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f39d8001800 con 0x7f39e815cf80 2023-10-21T16:02:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.862+0000 7f7f77fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7f70006b70 0x7f7f70053c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59686/0 (socket says 172.21.15.53:59686) 2023-10-21T16:02:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f7f77fff640 1 -- 172.21.15.53:0/3421030964 learned_addr learned my addr 172.21.15.53:0/3421030964 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f7f7cf4c640 1 --2- 172.21.15.53:0/3421030964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 0x7f7f7003ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 >> v1:172.21.15.53:6789/0 conn(0x7f39e815cf80 legacy=0x7f39e80b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f346db39640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 0x7f34681f7280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f346400c4b0 tx=0x7f346400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f0fd9705640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f7f777fe640 1 -- 172.21.15.53:0/3421030964 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635468020 0 0) 0x7f7f7003c070 con 0x7f7f70006b70 2023-10-21T16:02:26.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f7f777fe640 1 -- 172.21.15.53:0/3421030964 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f68003520 con 0x7f7f70006b70 2023-10-21T16:02:26.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d00039a0 0x7f39d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39d00047c0 0x7f39d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00051f0 0x7f39d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b8bb0 con 0x7f39d00051f0 2023-10-21T16:02:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d000c100 con 0x7f39d00047c0 2023-10-21T16:02:26.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d000c2c0 con 0x7f39d00039a0 2023-10-21T16:02:26.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f0fd9705640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 0x7f0fd40c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 0x7f0fd40c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 0x7f0fd40c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59522/0 (socket says 172.21.15.53:59522) 2023-10-21T16:02:26.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd37fe640 1 -- 172.21.15.53:0/1388003158 learned_addr learned my addr 172.21.15.53:0/1388003158 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c7c50 con 0x7f0fd415cd80 2023-10-21T16:02:26.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c7f30 con 0x7f0fd40cdf70 2023-10-21T16:02:26.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c8210 con 0x7f0fd40b87d0 2023-10-21T16:02:26.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f0fd2ffd640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fd40b87d0 0x7f0fd414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464006480 con 0x7f34680cede0 2023-10-21T16:02:26.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34640065e0 con 0x7f34680cede0 2023-10-21T16:02:26.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640049e0 con 0x7f34680cede0 2023-10-21T16:02:26.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34680c4f90 con 0x7f34680cede0 2023-10-21T16:02:26.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f7f7cf4c640 1 -- 172.21.15.53:0/3421030964 >> v1:172.21.15.124:6789/0 conn(0x7f7f70054360 legacy=0x7f7f70057720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f7f7cf4c640 1 -- 172.21.15.53:0/3421030964 >> v1:172.21.15.53:6789/0 conn(0x7f7f70006b70 legacy=0x7f7f70053c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f7f7cf4c640 1 -- 172.21.15.53:0/3421030964 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700073b0 con 0x7f7f700e3070 2023-10-21T16:02:26.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57100ce510 0x7f5710155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57100cf690 0x7f57100b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101527e0 con 0x7f57100cf2b0 2023-10-21T16:02:26.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100c97e0 con 0x7f57100cf690 2023-10-21T16:02:26.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57151ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5710153010 con 0x7f57100ce510 2023-10-21T16:02:26.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f0fd37fe640 1 -- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fd40b87d0 msgr2=0x7f0fd414f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f0fd27fc640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f0fd37fe640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fd40b87d0 0x7f0fd414f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f0fd37fe640 1 -- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fd40cdf70 msgr2=0x7f0fd4145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7fef03fff640 1 Processor -- start 2023-10-21T16:02:26.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.865+0000 7f57077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59526/0 (socket says 172.21.15.53:59526) 2023-10-21T16:02:26.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f0fd37fe640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f57077fe640 1 -- 172.21.15.53:0/722001745 learned_addr learned my addr 172.21.15.53:0/722001745 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f570ffff640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57100ce510 0x7f5710155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.863+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39d00047c0 0x7f39d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f39e5d9b640 1 -- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d00039a0 msgr2=0x7f39d0003dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:02:26.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d00039a0 0x7f39d0003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f39e5d9b640 1 -- 172.21.15.53:0/3458424426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00051f0 msgr2=0x7f39d000b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-21T16:02:26.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d00051f0 0x7f39d000b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.864+0000 7f39e5d9b640 1 -- 172.21.15.53:0/3458424426 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80b7410 con 0x7f39d00047c0 2023-10-21T16:02:26.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7fef03fff640 1 -- start start 2023-10-21T16:02:26.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7fe7df14c640 1 Processor -- start 2023-10-21T16:02:26.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f0fd37fe640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40094d0 con 0x7f0fd415cd80 2023-10-21T16:02:26.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39d00047c0 0x7f39d0004c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39d8007e40 tx=0x7f39d00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720da84ff515545c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf08013470 con 0x7faf10049ee0 2023-10-21T16:02:26.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f7f7cf4c640 1 --2- 172.21.15.53:0/3421030964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 0x7f7f7003ea40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0097d0 tx=0x7f7f6c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d16c1cb8dc1d5a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f57149fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57100cf690 0x7f57100b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f0fd2ffd640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fd40b87d0 0x7f0fd414f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:26.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34680c6b40 con 0x7f34680cede0 2023-10-21T16:02:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 -- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57100ce510 msgr2=0x7f5710155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57100ce510 0x7f5710155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 -- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57100cf690 msgr2=0x7f57100b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57100cf690 0x7f57100b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8005ad0 con 0x7f39d00047c0 2023-10-21T16:02:26.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700009a70 con 0x7f57100cf2b0 2023-10-21T16:02:26.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8037070 con 0x7f39d00047c0 2023-10-21T16:02:26.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f39e559a640 1 -- 172.21.15.53:0/3458424426 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8005e90 con 0x7f39d00047c0 2023-10-21T16:02:26.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f0fd37fe640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 0x7f0fd40c6ed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40090b0 tx=0x7f0fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f7f777fe640 1 -- 172.21.15.53:0/3421030964 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c02d040 con 0x7f7f700e3070 2023-10-21T16:02:26.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f7f777fe640 1 -- 172.21.15.53:0/3421030964 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c031810 con 0x7f7f700e3070 2023-10-21T16:02:26.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f0fd27fc640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fd40cdf70 0x7f0fd4145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f57077fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100b3790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c4b0 tx=0x7f56f800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7f7f777fe640 1 -- 172.21.15.53:0/3421030964 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c031b80 con 0x7f7f700e3070 2023-10-21T16:02:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34680b1e50 con 0x7f34680cede0 2023-10-21T16:02:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fef03fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef040cf690 0x7fef040d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fef03fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef040cd520 con 0x7fef040ce510 2023-10-21T16:02:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8005ce0 con 0x7f57100cf2b0 2023-10-21T16:02:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fef03fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef040d1910 con 0x7fef040cf2b0 2023-10-21T16:02:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fef03fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef040ce080 con 0x7fef040cf690 2023-10-21T16:02:26.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8005e40 con 0x7f57100cf2b0 2023-10-21T16:02:26.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8007270 con 0x7f57100cf2b0 2023-10-21T16:02:26.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f57151ff640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100b4210 con 0x7f57100cf2b0 2023-10-21T16:02:26.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.866+0000 7fe7df14c640 1 -- start start 2023-10-21T16:02:26.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7faeff7fe640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf0406af40 0x7faf0406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7fef02ffd640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fef040cf2b0 0x7fef040d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39534/0 (socket says 172.21.15.53:39534) 2023-10-21T16:02:26.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4003a20 con 0x7f0fd415cd80 2023-10-21T16:02:26.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.867+0000 7fe7df14c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7d80cede0 0x7fe7d806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fe7df14c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d80cb3c0 con 0x7fe7d81591d0 2023-10-21T16:02:26.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fe7df14c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d80cb6a0 con 0x7fe7d80ca220 2023-10-21T16:02:26.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fe7df14c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d80cb980 con 0x7fe7d80cede0 2023-10-21T16:02:26.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fe7dd6e9640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fe7d80ca220 0x7fe7d806cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39520/0 (socket says 172.21.15.53:39520) 2023-10-21T16:02:26.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1599238022 learned_addr learned my addr 172.21.15.53:0/1599238022 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f57151ff640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57100b7ce0 con 0x7f57100cf2b0 2023-10-21T16:02:26.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf08089980 con 0x7faf10049ee0 2023-10-21T16:02:26.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7fef02ffd640 1 -- 172.21.15.53:0/3412371702 learned_addr learned my addr 172.21.15.53:0/3412371702 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:26.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f39ec95a640 1 -- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39d00047c0 msgr2=0x7f39d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.868+0000 7f39ec95a640 1 --2- 172.21.15.53:0/3458424426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39d00047c0 0x7f39d0004c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f39d8007e40 tx=0x7f39d00057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f39e6d9d640 1 -- 172.21.15.53:0/3458424426 reap_dead start 2023-10-21T16:02:26.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f39ec95a640 1 -- 172.21.15.53:0/3458424426 shutdown_connections 2023-10-21T16:02:26.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f39ec95a640 1 -- 172.21.15.53:0/3458424426 >> 172.21.15.53:0/3458424426 conn(0x7f39e80718a0 msgr2=0x7f39e8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713635881 0 0) 0x7fe7d80cb6a0 con 0x7fe7d80ca220 2023-10-21T16:02:26.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7faeff7fe640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7faf04074500 0x7faf04076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c0003520 con 0x7fe7d80ca220 2023-10-21T16:02:26.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f346401b3b0 con 0x7f34680cede0 2023-10-21T16:02:26.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f39ec95a640 1 -- 172.21.15.53:0/3458424426 shutdown_connections 2023-10-21T16:02:26.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7f39ec95a640 1 -- 172.21.15.53:0/3458424426 wait complete. 2023-10-21T16:02:26.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc403f070 con 0x7f0fd415cd80 2023-10-21T16:02:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1599238022 >> v2:172.21.15.53:3300/0 conn(0x7fe7d80cede0 0x7fe7d806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274870318 0 0) 0x7fe7c0003520 con 0x7fe7d80ca220 2023-10-21T16:02:26.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.869+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf04077010 con 0x7faf04074500 2023-10-21T16:02:26.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf080545e0 con 0x7faf10049ee0 2023-10-21T16:02:26.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 >> v2:172.21.15.53:3300/0 conn(0x7fe7d80cede0 msgr2=0x7fe7d806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57100cdd10 con 0x7f57100cf2b0 2023-10-21T16:02:26.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 --2- 172.21.15.53:0/1599238022 >> v2:172.21.15.53:3300/0 conn(0x7fe7d80cede0 0x7fe7d806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 >> v1:172.21.15.53:6789/0 conn(0x7fe7d81591d0 legacy=0x7fe7d80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80cbc60 con 0x7fe7d80ca220 2023-10-21T16:02:26.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe7d0001670 con 0x7fe7d80ca220 2023-10-21T16:02:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 >> v1:172.21.15.124:6789/0 conn(0x7fe7d80ca220 legacy=0x7fe7d806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7c0003760 0x7fe7c0003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c00040a0 0x7fe7c000b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 0x7fe7c000d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80cb980 con 0x7fe7c000b550 2023-10-21T16:02:26.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c000e260 con 0x7fe7c0003760 2023-10-21T16:02:26.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c000e420 con 0x7fe7c00040a0 2023-10-21T16:02:26.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7faf177fe640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7faf04074500 0x7faf04076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1599238022 >> v2:172.21.15.53:3300/0 conn(0x7fe7d80cede0 0x7fe7d806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:26.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd40c84f0 con 0x7f0fd415cd80 2023-10-21T16:02:26.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd40c7410 con 0x7f0fd415cd80 2023-10-21T16:02:26.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4003da0 con 0x7f0fd415cd80 2023-10-21T16:02:26.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465985857 0 0) 0x7fef040d1910 con 0x7fef040cf2b0 2023-10-21T16:02:26.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003520 con 0x7fef040cf2b0 2023-10-21T16:02:26.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7faf167fc640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf0406af40 0x7faf0406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f7f7df4e640 1 -- 172.21.15.53:0/3421030964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 msgr2=0x7f7f7003ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.870+0000 7f7f7df4e640 1 --2- 172.21.15.53:0/3421030964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 0x7f7f7003ea40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0097d0 tx=0x7f7f6c02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7f345effd640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f344c06aec0 0x7f344c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 0x7fe7c000d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7fe7dcee8640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7c0003760 0x7fe7c0003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.871+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34640897f0 con 0x7f34680cede0 2023-10-21T16:02:26.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c00040a0 0x7fe7c000b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7faf177fe640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7faf04074500 0x7faf04076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250367529 0 0) 0x7feeec003520 con 0x7fef040cf2b0 2023-10-21T16:02:26.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 >> v2:172.21.15.53:3300/0 conn(0x7fef040cf690 msgr2=0x7fef040d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 --2- 172.21.15.53:0/3412371702 >> v2:172.21.15.53:3300/0 conn(0x7fef040cf690 0x7fef040d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 >> v1:172.21.15.53:6789/0 conn(0x7fef040ce510 legacy=0x7fef0415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040cd750 con 0x7fef040cf2b0 2023-10-21T16:02:26.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7feef8001670 con 0x7fef040cf2b0 2023-10-21T16:02:26.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 >> v1:172.21.15.124:6789/0 conn(0x7fef040cf2b0 legacy=0x7fef040d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec003970 0x7feeec003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeec005380 0x7feeec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef040ce080 con 0x7feeec003970 2023-10-21T16:02:26.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c220 con 0x7feeec0047c0 2023-10-21T16:02:26.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c3e0 con 0x7feeec005380 2023-10-21T16:02:26.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f345effd640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7f344c074500 0x7f344c076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f344c077010 con 0x7f344c074500 2023-10-21T16:02:26.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3464089c90 con 0x7f34680cede0 2023-10-21T16:02:26.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f7f7df4e640 1 -- 172.21.15.53:0/3421030964 shutdown_connections 2023-10-21T16:02:26.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f7f7df4e640 1 --2- 172.21.15.53:0/3421030964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f700e3070 0x7f7f7003ea40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7f7f7df4e640 1 -- 172.21.15.53:0/3421030964 >> 172.21.15.53:0/3421030964 conn(0x7f7f70055a90 msgr2=0x7f7f70055e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7f7f7df4e640 1 -- 172.21.15.53:0/3421030964 shutdown_connections 2023-10-21T16:02:26.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7f346eb3b640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7f344c074500 0x7f344c076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7faf167fc640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf0406af40 0x7faf0406d400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf0c036000 tx=0x7faf0c03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7f39ec95a640 1 Processor -- start 2023-10-21T16:02:26.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c00040a0 msgr2=0x7fe7c000b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c00040a0 0x7fe7c000b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7c0003760 msgr2=0x7fe7c0003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7c0003760 0x7fe7c0003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1599238022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0003520 con 0x7fe7c000b550 2023-10-21T16:02:26.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.872+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c00040a0 0x7fe7c000b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:26.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1599238022 reap_dead start 2023-10-21T16:02:26.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fb4000fc0 con 0x7f0fd415cd80 2023-10-21T16:02:26.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== osd.0 v2:172.21.15.53:6808/716916716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7faf04077010 con 0x7faf04074500 2023-10-21T16:02:26.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.873+0000 7f39ec95a640 1 -- start start 2023-10-21T16:02:26.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.874+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 0x7fe7c000d980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c160 tx=0x7fe7c00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91f881945c58104 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.874+0000 7fef027fc640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0005ad0 con 0x7fe7c000b550 2023-10-21T16:02:26.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0005c30 con 0x7fe7c000b550 2023-10-21T16:02:26.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f39ec95a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e81552f0 0x7f39e806d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f39ec95a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 0x7f39e806ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f39ec95a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e806f110 0x7f39e81f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7fe7cf7fe640 1 -- 172.21.15.53:0/1599238022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00070d0 con 0x7fe7c000b550 2023-10-21T16:02:26.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f39ec95a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80c7580 con 0x7f39e81552f0 2023-10-21T16:02:26.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f39ec95a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80c6900 con 0x7f39e806f110 2023-10-21T16:02:26.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39ec95a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80c6be0 con 0x7f39e81f2280 2023-10-21T16:02:26.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 0x7f39e806ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.874+0000 7f346eb3b640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7f344c074500 0x7f344c076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.874+0000 7f346e33a640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f344c06aec0 0x7f344c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f346e33a640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f344c06aec0 0x7f344c06d380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3460004bf0 tx=0x7f3460009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 0x7f39e806ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57956/0 (socket says 172.21.15.53:57956) 2023-10-21T16:02:26.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 -- 172.21.15.53:0/871160241 learned_addr learned my addr 172.21.15.53:0/871160241 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7fef037fe640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeec005380 0x7feeec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e6d9d640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e806f110 0x7f39e81f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 -- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e806f110 msgr2=0x7f39e81f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e806f110 0x7f39e81f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 -- 172.21.15.53:0/871160241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e81552f0 msgr2=0x7f39e806d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e81552f0 0x7f39e806d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f39e5d9b640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80c6ec0 con 0x7f39e81f2280 2023-10-21T16:02:26.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0fc403a430 con 0x7f0fd415cd80 2023-10-21T16:02:26.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7f39e6d9d640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e806f110 0x7f39e81f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:26.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef02ffd640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec003970 0x7feeec003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef027fc640 1 -- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeec005380 msgr2=0x7feeec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef027fc640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feeec005380 0x7feeec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef027fc640 1 -- 172.21.15.53:0/3412371702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec003970 msgr2=0x7feeec003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef027fc640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feeec003970 0x7feeec003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef027fc640 1 -- 172.21.15.53:0/3412371702 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040cd750 con 0x7feeec0047c0 2023-10-21T16:02:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fe7df14c640 1 -- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 msgr2=0x7fe7c000d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fe7df14c640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 0x7fe7c000d980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c160 tx=0x7fe7c00047e0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7fef037fe640 1 -- 172.21.15.53:0/3412371702 reap_dead start 2023-10-21T16:02:26.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.876+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56f80073d0 con 0x7f57100cf2b0 2023-10-21T16:02:26.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7f39e5d9b640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 0x7f39e806ebd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034d50 tx=0x7f39d8001ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.875+0000 7f7f7df4e640 1 -- 172.21.15.53:0/3421030964 wait complete. 2023-10-21T16:02:26.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.877+0000 7f7f7df4e640 1 Processor -- start 2023-10-21T16:02:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== osd.2 v2:172.21.15.53:6816/3435187098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f344c077010 con 0x7f344c074500 2023-10-21T16:02:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7fe7df14c640 1 -- 172.21.15.53:0/1599238022 shutdown_connections 2023-10-21T16:02:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7fe7df14c640 1 --2- 172.21.15.53:0/1599238022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7c000b550 0x7fe7c000d980 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7fe7df14c640 1 -- 172.21.15.53:0/1599238022 >> 172.21.15.53:0/1599238022 conn(0x7fe7d80660e0 msgr2=0x7fe7d8069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f0fbb7fe640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0fc006ae70 0x7f0fc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0fc4041070 con 0x7f0fd415cd80 2023-10-21T16:02:26.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f7f7df4e640 1 -- start start 2023-10-21T16:02:26.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004cb0 con 0x7f39e81f2280 2023-10-21T16:02:26.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8004e10 con 0x7f39e81f2280 2023-10-21T16:02:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8007270 con 0x7f39e81f2280 2023-10-21T16:02:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f570d7fa640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56f006af70 0x7f56f006d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56f8089470 con 0x7f57100cf2b0 2023-10-21T16:02:26.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f570d7fa640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f56f00745b0 0x7f56f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56f00770c0 con 0x7f56f00745b0 2023-10-21T16:02:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f8089820 con 0x7f57100cf2b0 2023-10-21T16:02:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f0fbb7fe640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7f0fc0074430 0x7f0fc0076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.878+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc0076f40 con 0x7f0fc0074430 2023-10-21T16:02:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc40ace60 con 0x7f0fd415cd80 2023-10-21T16:02:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7fef027fc640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feef4009860 tx=0x7feeec0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f201bc7cd80b793 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7fe7df14c640 1 -- 172.21.15.53:0/1599238022 shutdown_connections 2023-10-21T16:02:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f57149fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f56f00745b0 0x7f56f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e80c71a0 con 0x7f39e81f2280 2023-10-21T16:02:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e81558b0 con 0x7f39e81f2280 2023-10-21T16:02:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4032600 con 0x7feeec0047c0 2023-10-21T16:02:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef402e050 con 0x7feeec0047c0 2023-10-21T16:02:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7fef01ffb640 1 -- 172.21.15.53:0/3412371702 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40329c0 con 0x7feeec0047c0 2023-10-21T16:02:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f0fd2ffd640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0fc006ae70 0x7f0fc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f0fd27fc640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7f0fc0074430 0x7f0fc0076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f570ffff640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56f006af70 0x7f56f006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f70006b70 0x7f7f70038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f70054360 0x7f7f70186340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 0x7f7f70188710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70186a80 con 0x7f7f700e3070 2023-10-21T16:02:26.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70186880 con 0x7f7f70054360 2023-10-21T16:02:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.879+0000 7f7f7df4e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70188c50 con 0x7f7f70006b70 2023-10-21T16:02:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 0x7f7f70188710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 0x7f7f70188710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59578/0 (socket says 172.21.15.53:59578) 2023-10-21T16:02:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 -- 172.21.15.53:0/418153965 learned_addr learned my addr 172.21.15.53:0/418153965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 -- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f70006b70 msgr2=0x7f7f70038d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f70006b70 0x7f7f70038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 -- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f70054360 msgr2=0x7f7f70186340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f70054360 0x7f7f70186340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c009400 con 0x7f7f700e3070 2023-10-21T16:02:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 -- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 msgr2=0x7feeec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feef4009860 tx=0x7feeec0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39e80b79d0 con 0x7f39e81f2280 2023-10-21T16:02:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fe7df14c640 1 -- 172.21.15.53:0/1599238022 wait complete. 2023-10-21T16:02:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.880+0000 7f7f7d74d640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 0x7f7f70188710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f78069460 tx=0x7f7f780734e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 -- 172.21.15.53:0/3412371702 shutdown_connections 2023-10-21T16:02:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 --2- 172.21.15.53:0/3412371702 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 -- 172.21.15.53:0/3412371702 >> 172.21.15.53:0/3412371702 conn(0x7fef0409bc40 msgr2=0x7fef040d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 -- 172.21.15.53:0/3412371702 shutdown_connections 2023-10-21T16:02:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7fef03fff640 1 -- 172.21.15.53:0/3412371702 wait complete. 2023-10-21T16:02:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f0fd2ffd640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0fc006ae70 0x7f0fc006d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007e90 tx=0x7f0fcc0073a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780723d0 con 0x7f7f700e3070 2023-10-21T16:02:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78091400 con 0x7f7f700e3070 2023-10-21T16:02:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780723d0 con 0x7f7f700e3070 2023-10-21T16:02:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f570ffff640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56f006af70 0x7f56f006d430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5700036000 tx=0x7f5700030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.882+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f70039360 con 0x7f7f700e3070 2023-10-21T16:02:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.883+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70039920 con 0x7f7f700e3070 2023-10-21T16:02:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.883+0000 7fe7df14c640 1 Processor -- start 2023-10-21T16:02:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.883+0000 7fe7df14c640 1 -- start start 2023-10-21T16:02:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39d8039030 con 0x7f39e81f2280 2023-10-21T16:02:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c000b550 0x7fe7d8157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7d80c78b0 0x7fe7d80c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 0x7fe7d80c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80c8270 con 0x7fe7d80c5ad0 2023-10-21T16:02:26.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80c6490 con 0x7fe7d80c78b0 2023-10-21T16:02:26.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.884+0000 7fe7df14c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80c2f70 con 0x7fe7c000b550 2023-10-21T16:02:26.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.883+0000 7f57149fe640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f56f00745b0 0x7f56f00769f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.881+0000 7f0fd27fc640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7f0fc0074430 0x7f0fc0076870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fe7dd6e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 0x7fe7d80c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fe7dd6e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 0x7fe7d80c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59590/0 (socket says 172.21.15.53:59590) 2023-10-21T16:02:26.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1600693163 learned_addr learned my addr 172.21.15.53:0/1600693163 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:26.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== osd.1 v2:172.21.15.53:6800/752205661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0fc0076f40 con 0x7f0fc0074430 2023-10-21T16:02:26.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7d80c78b0 0x7fe7d80c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fe7dcee8640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c000b550 0x7fe7d8157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.885+0000 7fef03fff640 1 Processor -- start 2023-10-21T16:02:26.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c000b550 msgr2=0x7fe7d8157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7c000b550 0x7fe7d8157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7d80c78b0 msgr2=0x7fe7d80c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:26.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7d80c78b0 0x7fe7d80c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:26.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80c2960 con 0x7fe7d80c5ad0 2023-10-21T16:02:26.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f7809a3a0 con 0x7f7f700e3070 2023-10-21T16:02:26.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f70007970 con 0x7f7f700e3070 2023-10-21T16:02:26.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.886+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== osd.3 v2:172.21.15.53:6824/1838632921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f56f00770c0 con 0x7f56f00745b0 2023-10-21T16:02:26.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.887+0000 7f7f757fa640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f5c06aed0 0x7f7f5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7f78102bc0 con 0x7f7f700e3070 2023-10-21T16:02:26.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.887+0000 7f39d6ffd640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f39c806af40 0x7f39c806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39d80b3a80 con 0x7f39e81f2280 2023-10-21T16:02:26.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7f39d6ffd640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7f39c8074580 0x7f39c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39c8077090 con 0x7f39c8074580 2023-10-21T16:02:26.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d80b3e30 con 0x7f39e81f2280 2023-10-21T16:02:26.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.888+0000 7fef03fff640 1 -- start start 2023-10-21T16:02:26.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f7f757fa640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f7f5c074510 0x7f7f5c076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:26.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f7f7cf4c640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f5c06aed0 0x7f7f5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f5c077020 con 0x7f7f5c074510 2023-10-21T16:02:26.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7fe7dd6e9640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 0x7fe7d80c5f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7d002c400 tx=0x7fe7d0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:26.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f78107280 con 0x7f7f700e3070 2023-10-21T16:02:26.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f39e659c640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f39c806af40 0x7f39c806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f39e6d9d640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7f39c8074580 0x7f39c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.889+0000 7f7f77fff640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f7f5c074510 0x7f7f5c076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:26.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00074e0 con 0x7fe7d80c5ad0 2023-10-21T16:02:26.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0007640 con 0x7fe7d80c5ad0 2023-10-21T16:02:26.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d002ca00 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d80c8470 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7f7f7cf4c640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f5c06aed0 0x7f7f5c06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c036000 tx=0x7f7f6c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7fef041f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef040af780 0x7fef040afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 0x7fef0415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef040b0140 con 0x7fef040af780 2023-10-21T16:02:27.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef041f6aa0 con 0x7feeec0047c0 2023-10-21T16:02:27.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fef03fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef040b6f20 con 0x7fef0415cf20 2023-10-21T16:02:27.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.890+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d80c2c00 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7f7f77fff640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f7f5c074510 0x7f7f5c076950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7fef02ffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7fef041f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7fef027fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef040af780 0x7fef040afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7fef037fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 0x7fef0415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7fef037fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 0x7fef0415d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57986/0 (socket says 172.21.15.53:57986) 2023-10-21T16:02:27.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7fef037fe640 1 -- 172.21.15.53:0/2779018505 learned_addr learned my addr 172.21.15.53:0/2779018505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7f39e659c640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f39c806af40 0x7f39c806d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39e80c6420 tx=0x7f39e0007d10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.891+0000 7f39e6d9d640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7f39c8074580 0x7f39c80769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7d80b1e50 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef027fc640 1 -- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 msgr2=0x7fef0415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef027fc640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 0x7fef0415d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef027fc640 1 -- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 msgr2=0x7fef041f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef027fc640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feeec0047c0 0x7fef041f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef027fc640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040b7200 con 0x7fef040af780 2023-10-21T16:02:27.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.892+0000 7fef037fe640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0415cf20 0x7fef0415d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:27.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.893+0000 7fef027fc640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef040af780 0x7fef040afc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feef4032000 tx=0x7feef40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.893+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== osd.4 v2:172.21.15.124:6800/1000673860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7f5c077020 con 0x7f7f5c074510 2023-10-21T16:02:27.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.894+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== osd.5 v2:172.21.15.124:6808/3990975910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f39c8077090 con 0x7f39c8074580 2023-10-21T16:02:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.894+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef403c880 con 0x7fef040af780 2023-10-21T16:02:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.894+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef403c9e0 con 0x7fef040af780 2023-10-21T16:02:27.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.894+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef403cd90 con 0x7fef040af780 2023-10-21T16:02:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.895+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040b74e0 con 0x7fef040af780 2023-10-21T16:02:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.895+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef040b7aa0 con 0x7fef040af780 2023-10-21T16:02:27.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.895+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe7d0032070 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.897+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef040cdd10 con 0x7fef040af780 2023-10-21T16:02:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.897+0000 7fe7cd7fa640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe7c406af40 0x7fe7c406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.898+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe7d00aaa00 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.898+0000 7fe7cd7fa640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fe7c4074580 0x7fe7c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.898+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7c4077090 con 0x7fe7c4074580 2023-10-21T16:02:27.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.899+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d00aadb0 con 0x7fe7d80c5ad0 2023-10-21T16:02:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.899+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fe7c4074580 0x7fe7c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.899+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 --> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf100e8b30 con 0x7faf04074500 2023-10-21T16:02:27.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.899+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feef4039040 con 0x7fef040af780 2023-10-21T16:02:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.900+0000 7faeff7fe640 1 -- 172.21.15.53:0/71787858 <== osd.0 v2:172.21.15.53:6808/716916716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faf1804f050 con 0x7faf04074500 2023-10-21T16:02:27.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.900+0000 7fe7cffff640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fe7c4074580 0x7fe7c40769c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.901+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7faf04074500 msgr2=0x7faf04076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.901+0000 7faf17fff640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7faf04074500 0x7faf04076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.901+0000 7feeeb7fe640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef006aec0 0x7feef006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.901+0000 7fe7dcee8640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe7c406af40 0x7fe7c406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.901+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feef40ab630 con 0x7fef040af780 2023-10-21T16:02:27.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7feeeb7fe640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7feef0074500 0x7feef0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf0406af40 msgr2=0x7faf0406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf0406af40 0x7faf0406d400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf0c036000 tx=0x7faf0c03b040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feef0077010 con 0x7feef0074500 2023-10-21T16:02:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef40ab9a0 con 0x7fef040af780 2023-10-21T16:02:27.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 msgr2=0x7faf100e4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 --2- 172.21.15.53:0/71787858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf10049ee0 0x7faf100e4290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e870 tx=0x7faf0800ed30 comp rx=0 tx=0).stop 2023-10-21T16:02:27.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf177fe640 1 -- 172.21.15.53:0/71787858 reap_dead start 2023-10-21T16:02:27.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7fef02ffd640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef006aec0 0x7feef006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 shutdown_connections 2023-10-21T16:02:27.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 >> 172.21.15.53:0/71787858 conn(0x7faf100538a0 msgr2=0x7faf1003ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.902+0000 7fe7dcee8640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe7c406af40 0x7fe7c406d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005160 tx=0x7fe7d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.903+0000 7fef037fe640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7feef0074500 0x7feef0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.903+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 shutdown_connections 2023-10-21T16:02:27.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.903+0000 7faf17fff640 1 -- 172.21.15.53:0/71787858 wait complete. 2023-10-21T16:02:27.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.903+0000 7fef02ffd640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef006aec0 0x7feef006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feef8001ca0 tx=0x7feef8007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.903+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== osd.7 v2:172.21.15.124:6824/1993556734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe7c4077090 con 0x7fe7c4074580 2023-10-21T16:02:27.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.905+0000 7fef037fe640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7feef0074500 0x7feef0076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.907+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 --> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34680cbf40 con 0x7f344c074500 2023-10-21T16:02:27.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.908+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== osd.6 v2:172.21.15.124:6810/3032138525 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7feef0077010 con 0x7feef0074500 2023-10-21T16:02:27.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.908+0000 7f345effd640 1 -- 172.21.15.53:0/1386123676 <== osd.2 v2:172.21.15.53:6816/3435187098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f345802f7f0 con 0x7f344c074500 2023-10-21T16:02:27.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7f344c074500 msgr2=0x7f344c076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7f344c074500 0x7f344c076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f344c06aec0 msgr2=0x7f344c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f344c06aec0 0x7f344c06d380 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3460004bf0 tx=0x7f3460009040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 msgr2=0x7f34681f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 --2- 172.21.15.53:0/1386123676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34680cede0 0x7f34681f7280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f346400c4b0 tx=0x7f346400c970 comp rx=0 tx=0).stop 2023-10-21T16:02:27.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f346eb3b640 1 -- 172.21.15.53:0/1386123676 reap_dead start 2023-10-21T16:02:27.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 shutdown_connections 2023-10-21T16:02:27.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 >> 172.21.15.53:0/1386123676 conn(0x7f34680660e0 msgr2=0x7f346806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 shutdown_connections 2023-10-21T16:02:27.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.909+0000 7f347059e640 1 -- 172.21.15.53:0/1386123676 wait complete. 2023-10-21T16:02:27.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.918+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 --> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57100cda30 con 0x7f56f00745b0 2023-10-21T16:02:27.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.920+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 --> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fb4002cb0 con 0x7f0fc0074430 2023-10-21T16:02:27.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.920+0000 7f570d7fa640 1 -- 172.21.15.53:0/722001745 <== osd.3 v2:172.21.15.53:6824/1838632921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57080013a0 con 0x7f56f00745b0 2023-10-21T16:02:27.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f56f00745b0 msgr2=0x7f56f00769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f0fbb7fe640 1 -- 172.21.15.53:0/1388003158 <== osd.1 v2:172.21.15.53:6800/752205661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0fb4002cb0 con 0x7f0fc0074430 2023-10-21T16:02:27.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f56f00745b0 0x7f56f00769f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56f006af70 msgr2=0x7f56f006d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56f006af70 0x7f56f006d430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5700036000 tx=0x7f5700030000 comp rx=0 tx=0).stop 2023-10-21T16:02:27.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 msgr2=0x7f57100b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.921+0000 7f57067fc640 1 --2- 172.21.15.53:0/722001745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57100cf2b0 0x7f57100b3790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c4b0 tx=0x7f56f800c970 comp rx=0 tx=0).stop 2023-10-21T16:02:27.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f57149fe640 1 -- 172.21.15.53:0/722001745 reap_dead start 2023-10-21T16:02:27.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7f0fc0074430 msgr2=0x7f0fc0076870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7f0fc0074430 0x7f0fc0076870 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0fc006ae70 msgr2=0x7f0fc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0fc006ae70 0x7f0fc006d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007e90 tx=0x7f0fcc0073a0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 msgr2=0x7f0fd40c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.922+0000 7f0fd9705640 1 --2- 172.21.15.53:0/1388003158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd415cd80 0x7f0fd40c6ed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40090b0 tx=0x7f0fc40046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f0fd37fe640 1 -- 172.21.15.53:0/1388003158 reap_dead start 2023-10-21T16:02:27.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 shutdown_connections 2023-10-21T16:02:27.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 >> 172.21.15.53:0/1388003158 conn(0x7f0fd406b5d0 msgr2=0x7f0fd40d1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 shutdown_connections 2023-10-21T16:02:27.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 >> 172.21.15.53:0/722001745 conn(0x7f571009bc40 msgr2=0x7f57100b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 shutdown_connections 2023-10-21T16:02:27.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.923+0000 7f57067fc640 1 -- 172.21.15.53:0/722001745 wait complete. 2023-10-21T16:02:27.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.925+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 shutdown_connections 2023-10-21T16:02:27.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.927+0000 7f0fd9705640 1 -- 172.21.15.53:0/1388003158 wait complete. 2023-10-21T16:02:27.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.928+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 --> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f70007690 con 0x7f7f5c074510 2023-10-21T16:02:27.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.929+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 --> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39e80b76f0 con 0x7f39c8074580 2023-10-21T16:02:27.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.929+0000 7f7f757fa640 1 -- 172.21.15.53:0/418153965 <== osd.4 v2:172.21.15.124:6800/1000673860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f640012d0 con 0x7f7f5c074510 2023-10-21T16:02:27.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 --> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7d80cbf40 con 0x7fe7c4074580 2023-10-21T16:02:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f7f5c074510 msgr2=0x7f7f5c076950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f7f5c074510 0x7f7f5c076950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f5c06aed0 msgr2=0x7f7f5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f5c06aed0 0x7f7f5c06d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c036000 tx=0x7f7f6c030000 comp rx=0 tx=0).stop 2023-10-21T16:02:27.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 msgr2=0x7f7f70188710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 --2- 172.21.15.53:0/418153965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f700e3070 0x7f7f70188710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f78069460 tx=0x7f7f780734e0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7d74d640 1 -- 172.21.15.53:0/418153965 reap_dead start 2023-10-21T16:02:27.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 shutdown_connections 2023-10-21T16:02:27.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 >> 172.21.15.53:0/418153965 conn(0x7f7f70055a90 msgr2=0x7f7f70051020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.931+0000 7fe7cd7fa640 1 -- 172.21.15.53:0/1600693163 <== osd.7 v2:172.21.15.124:6824/1993556734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7c4077090 con 0x7fe7c4074580 2023-10-21T16:02:27.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.930+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 shutdown_connections 2023-10-21T16:02:27.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.931+0000 7f7f7df4e640 1 -- 172.21.15.53:0/418153965 wait complete. 2023-10-21T16:02:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fe7c4074580 msgr2=0x7fe7c40769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fe7c4074580 0x7fe7c40769c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe7c406af40 msgr2=0x7fe7c406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe7c406af40 0x7fe7c406d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4005160 tx=0x7fe7d4009040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 msgr2=0x7fe7d80c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.932+0000 7fe7df14c640 1 --2- 172.21.15.53:0/1600693163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7d80c5ad0 0x7fe7d80c5f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7d002c400 tx=0x7fe7d0001930 comp rx=0 tx=0).stop 2023-10-21T16:02:27.042 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.042 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-21T16:02:27.043 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:27.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7f39d6ffd640 1 -- 172.21.15.53:0/871160241 <== osd.5 v2:172.21.15.124:6808/3990975910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39dc00b2f0 con 0x7f39c8074580 2023-10-21T16:02:27.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-21T16:02:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7fe7dd6e9640 1 -- 172.21.15.53:0/1600693163 reap_dead start 2023-10-21T16:02:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 shutdown_connections 2023-10-21T16:02:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 >> 172.21.15.53:0/1600693163 conn(0x7fe7d80660e0 msgr2=0x7fe7d80cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7f39c8074580 msgr2=0x7f39c80769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7f39ec95a640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7f39c8074580 0x7f39c80769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 shutdown_connections 2023-10-21T16:02:27.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7fe7df14c640 1 -- 172.21.15.53:0/1600693163 wait complete. 2023-10-21T16:02:27.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f39c806af40 msgr2=0x7f39c806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.933+0000 7f39ec95a640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f39c806af40 0x7f39c806d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39e80c6420 tx=0x7f39e0007d10 comp rx=0 tx=0).stop 2023-10-21T16:02:27.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 msgr2=0x7f39e806ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 --2- 172.21.15.53:0/871160241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e81f2280 0x7f39e806ebd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034d50 tx=0x7f39d8001ac0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39e6d9d640 1 -- 172.21.15.53:0/871160241 reap_dead start 2023-10-21T16:02:27.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 shutdown_connections 2023-10-21T16:02:27.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 >> 172.21.15.53:0/871160241 conn(0x7f39e80718a0 msgr2=0x7f39e8159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 shutdown_connections 2023-10-21T16:02:27.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.934+0000 7f39ec95a640 1 -- 172.21.15.53:0/871160241 wait complete. 2023-10-21T16:02:27.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.940+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 --> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef040cda30 con 0x7feef0074500 2023-10-21T16:02:27.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.941+0000 7feeeb7fe640 1 -- 172.21.15.53:0/2779018505 <== osd.6 v2:172.21.15.124:6810/3032138525 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7feefc005560 con 0x7feef0074500 2023-10-21T16:02:27.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.941+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7feef0074500 msgr2=0x7feef0076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.941+0000 7fef03fff640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7feef0074500 0x7feef0076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef006aec0 msgr2=0x7feef006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef006aec0 0x7feef006d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feef8001ca0 tx=0x7feef8007040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef040af780 msgr2=0x7fef040afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 --2- 172.21.15.53:0/2779018505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef040af780 0x7fef040afc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feef4032000 tx=0x7feef40047d0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef037fe640 1 -- 172.21.15.53:0/2779018505 reap_dead start 2023-10-21T16:02:27.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 shutdown_connections 2023-10-21T16:02:27.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 >> 172.21.15.53:0/2779018505 conn(0x7fef0409bc40 msgr2=0x7fef040b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 shutdown_connections 2023-10-21T16:02:27.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:26.942+0000 7fef03fff640 1 -- 172.21.15.53:0/2779018505 wait complete. 2023-10-21T16:02:27.053 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-21T16:02:27.055 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-21T16:02:27.056 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-21T16:02:27.057 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-21T16:02:27.059 INFO:teuthology.orchestra.run.smithi053.stdout:55834574859 2023-10-21T16:02:27.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-21T16:02:27.060 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:27.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-21T16:02:27.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.385+0000 7fd432b1d640 1 Processor -- start 2023-10-21T16:02:27.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.385+0000 7fd432b1d640 1 -- start start 2023-10-21T16:02:27.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.387+0000 7fd432b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.388+0000 7fd432b1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd42c070300 0x7fd42c165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.388+0000 7fd432b1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c070760 con 0x7fd42c1655c0 2023-10-21T16:02:27.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.388+0000 7fd432b1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c165c20 con 0x7fd42c070300 2023-10-21T16:02:27.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.388+0000 7fd432b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0724b0 con 0x7fd42c164ca0 2023-10-21T16:02:27.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.389+0000 7fd423fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd42c070300 0x7fd42c165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.389+0000 7fd4308b9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.389+0000 7fd4308b9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57990/0 (socket says 172.21.15.53:57990) 2023-10-21T16:02:27.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.389+0000 7fd4308b9640 1 -- 172.21.15.53:0/815108519 learned_addr learned my addr 172.21.15.53:0/815108519 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.389+0000 7fd4310ba640 1 --1- 172.21.15.53:0/815108519 >> v1:172.21.15.124:6789/0 conn(0x7fd42c1655c0 0x7fd42c0c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39544/0 (socket says 172.21.15.53:39544) 2023-10-21T16:02:27.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.390+0000 7fd4308b9640 1 -- 172.21.15.53:0/815108519 >> v2:172.21.15.124:3300/0 conn(0x7fd42c070300 msgr2=0x7fd42c165080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.390+0000 7fd4308b9640 1 --2- 172.21.15.53:0/815108519 >> v2:172.21.15.124:3300/0 conn(0x7fd42c070300 0x7fd42c165080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.390+0000 7fd4308b9640 1 -- 172.21.15.53:0/815108519 >> v1:172.21.15.124:6789/0 conn(0x7fd42c1655c0 legacy=0x7fd42c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.390+0000 7fd4308b9640 1 -- 172.21.15.53:0/815108519 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0c9710 con 0x7fd42c164ca0 2023-10-21T16:02:27.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.392+0000 7fd4308b9640 1 --2- 172.21.15.53:0/815108519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab30b7a76a1b002 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.392+0000 7fd42bfff640 1 -- 172.21.15.53:0/815108519 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c02cb00 con 0x7fd42c164ca0 2023-10-21T16:02:27.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.393+0000 7fd42bfff640 1 -- 172.21.15.53:0/815108519 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c02cc60 con 0x7fd42c164ca0 2023-10-21T16:02:27.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.393+0000 7fd42bfff640 1 -- 172.21.15.53:0/815108519 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0343f0 con 0x7fd42c164ca0 2023-10-21T16:02:27.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.394+0000 7fd432b1d640 1 -- 172.21.15.53:0/815108519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 msgr2=0x7fd42c0cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.394+0000 7fd432b1d640 1 --2- 172.21.15.53:0/815108519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c02bf60 comp rx=0 tx=0).stop 2023-10-21T16:02:27.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.394+0000 7fd432b1d640 1 -- 172.21.15.53:0/815108519 shutdown_connections 2023-10-21T16:02:27.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.395+0000 7fd432b1d640 1 --2- 172.21.15.53:0/815108519 >> v2:172.21.15.124:3300/0 conn(0x7fd42c070300 0x7fd42c165080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.395+0000 7fd432b1d640 1 --2- 172.21.15.53:0/815108519 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c164ca0 0x7fd42c0cbab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.395+0000 7fd432b1d640 1 -- 172.21.15.53:0/815108519 >> 172.21.15.53:0/815108519 conn(0x7fd42c0caf90 msgr2=0x7fd42c0cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.395+0000 7fd432b1d640 1 -- 172.21.15.53:0/815108519 shutdown_connections 2023-10-21T16:02:27.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.396+0000 7fd432b1d640 1 -- 172.21.15.53:0/815108519 wait complete. 2023-10-21T16:02:27.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.400+0000 7fd432b1d640 1 Processor -- start 2023-10-21T16:02:27.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.400+0000 7fd432b1d640 1 -- start start 2023-10-21T16:02:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.402+0000 7fd432b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.402+0000 7fd432b1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 0x7fd42c0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.403+0000 7fd432b1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd42c1655c0 0x7fd42c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.403+0000 7fd432b1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0b0a10 con 0x7fd42c164ca0 2023-10-21T16:02:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.403+0000 7fd432b1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0afd90 con 0x7fd42c1655c0 2023-10-21T16:02:27.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.403+0000 7fd432b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c0b0070 con 0x7fd42c070300 2023-10-21T16:02:27.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.404+0000 7fd4308b9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.404+0000 7fd4308b9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57994/0 (socket says 172.21.15.53:57994) 2023-10-21T16:02:27.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.404+0000 7fd4308b9640 1 -- 172.21.15.53:0/692485016 learned_addr learned my addr 172.21.15.53:0/692485016 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.404+0000 7fd423fff640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 0x7fd42c0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.405+0000 7fd4310ba640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd42c1655c0 0x7fd42c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.406+0000 7fd4308b9640 1 -- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd42c1655c0 msgr2=0x7fd42c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.406+0000 7fd4308b9640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd42c1655c0 0x7fd42c0c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.406+0000 7fd4308b9640 1 -- 172.21.15.53:0/692485016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 msgr2=0x7fd42c0c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.406+0000 7fd4308b9640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 0x7fd42c0c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.406+0000 7fd4308b9640 1 -- 172.21.15.53:0/692485016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c009b30 con 0x7fd42c070300 2023-10-21T16:02:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.407+0000 7fd423fff640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 0x7fd42c0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.408+0000 7fd4308b9640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.409+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c03e450 con 0x7fd42c070300 2023-10-21T16:02:27.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.409+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c03e5b0 con 0x7fd42c070300 2023-10-21T16:02:27.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.409+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c03e960 con 0x7fd42c070300 2023-10-21T16:02:27.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.409+0000 7fd432b1d640 1 -- 172.21.15.53:0/692485016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c0b0350 con 0x7fd42c070300 2023-10-21T16:02:27.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.410+0000 7fd432b1d640 1 -- 172.21.15.53:0/692485016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c0af9b0 con 0x7fd42c070300 2023-10-21T16:02:27.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.414+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd404005250 con 0x7fd42c070300 2023-10-21T16:02:27.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.415+0000 7f584fd33640 1 Processor -- start 2023-10-21T16:02:27.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.415+0000 7f584fd33640 1 -- start start 2023-10-21T16:02:27.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.415+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd41c03eac0 con 0x7fd42c070300 2023-10-21T16:02:27.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.416+0000 7fd429ffb640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 0x7fd40c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.416+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd41c0abf00 con 0x7fd42c070300 2023-10-21T16:02:27.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.417+0000 7f584fd33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 0x7f58480d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.417+0000 7f584fd33640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f58480d0a40 0x7f58480d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.417+0000 7fd423fff640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 0x7fd40c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584fd33640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480d3550 con 0x7f584815ce00 2023-10-21T16:02:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584fd33640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480cd5f0 con 0x7f58480d0a40 2023-10-21T16:02:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584fd33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cd890 con 0x7f584815ca20 2023-10-21T16:02:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584e2d0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f584815ce00 0x7f58481607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59716/0 (socket says 172.21.15.53:59716) 2023-10-21T16:02:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584e2d0640 1 -- 172.21.15.53:0/1222409327 learned_addr learned my addr 172.21.15.53:0/1222409327 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:27.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.418+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1222409327 >> v2:172.21.15.124:3300/0 conn(0x7f58480d0a40 0x7f58480d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.419+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915647002 0 0) 0x7f58480d3550 con 0x7f584815ce00 2023-10-21T16:02:27.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.419+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5834003520 con 0x7f584815ce00 2023-10-21T16:02:27.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.419+0000 7fd423fff640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 0x7fd40c06d400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0b1070 tx=0x7fd414008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.419+0000 7f584d2ce640 1 -- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 msgr2=0x7f58480d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373298434 0 0) 0x7f5834003520 con 0x7f584815ce00 2023-10-21T16:02:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584dacf640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 0x7f58480d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 0x7f58480d0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584d2ce640 1 -- 172.21.15.53:0/1222409327 >> v1:172.21.15.53:6789/0 conn(0x7f584815ce00 legacy=0x7f58481607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584d2ce640 1 -- 172.21.15.53:0/1222409327 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480cdb70 con 0x7f58480d0a40 2023-10-21T16:02:27.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.420+0000 7f584dacf640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 0x7f58480d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.421+0000 7fc4c25dd640 1 Processor -- start 2023-10-21T16:02:27.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.421+0000 7fc4c25dd640 1 -- start start 2023-10-21T16:02:27.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.422+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480d0ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b6150b2b66f996 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.422+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400019f0 con 0x7f58480d0a40 2023-10-21T16:02:27.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.422+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583802e050 con 0x7f58480d0a40 2023-10-21T16:02:27.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.422+0000 7f584cacd640 1 -- 172.21.15.53:0/1222409327 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838030730 con 0x7f58480d0a40 2023-10-21T16:02:27.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7fc4c25dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4b40e3320 0x7fc4b40539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7fc4c25dd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc4b40500f0 0x7fc4b40067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7fc4c25dd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b4006310 con 0x7fc4b403a830 2023-10-21T16:02:27.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7fc4c25dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b40e3700 con 0x7fc4b40e3320 2023-10-21T16:02:27.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7fc4c25dd640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b4005fe0 con 0x7fc4b40500f0 2023-10-21T16:02:27.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7f584fd33640 1 -- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 msgr2=0x7f58480d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.423+0000 7f584fd33640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480d0ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802b790 comp rx=0 tx=0).stop 2023-10-21T16:02:27.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7fc4c15db640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4b40e3320 0x7fc4b40539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7fc4c0dda640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc4b403a830 0x7fc4b404f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59732/0 (socket says 172.21.15.53:59732) 2023-10-21T16:02:27.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7fc4c0dda640 1 -- 172.21.15.53:0/2661527141 learned_addr learned my addr 172.21.15.53:0/2661527141 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:27.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7f584fd33640 1 -- 172.21.15.53:0/1222409327 shutdown_connections 2023-10-21T16:02:27.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7f584fd33640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480d0ef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7f584fd33640 1 --2- 172.21.15.53:0/1222409327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ca20 0x7f58480d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7f584fd33640 1 -- 172.21.15.53:0/1222409327 >> 172.21.15.53:0/1222409327 conn(0x7f584809bc60 msgr2=0x7f58480d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.424+0000 7fc4c1ddc640 1 --2- 172.21.15.53:0/2661527141 >> v2:172.21.15.124:3300/0 conn(0x7fc4b40500f0 0x7fc4b40067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2661527141 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782573180 0 0) 0x7fc4b4006310 con 0x7fc4b403a830 2023-10-21T16:02:27.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7f584fd33640 1 -- 172.21.15.53:0/1222409327 shutdown_connections 2023-10-21T16:02:27.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2661527141 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a8003520 con 0x7fc4b403a830 2023-10-21T16:02:27.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4c15db640 1 -- 172.21.15.53:0/2661527141 >> v2:172.21.15.124:3300/0 conn(0x7fc4b40500f0 msgr2=0x7fc4b40067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4c15db640 1 --2- 172.21.15.53:0/2661527141 >> v2:172.21.15.124:3300/0 conn(0x7fc4b40500f0 0x7fc4b40067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4c15db640 1 -- 172.21.15.53:0/2661527141 >> v1:172.21.15.53:6789/0 conn(0x7fc4b403a830 legacy=0x7fc4b404f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7fc4c15db640 1 -- 172.21.15.53:0/2661527141 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4054240 con 0x7fc4b40e3320 2023-10-21T16:02:27.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.425+0000 7f584fd33640 1 -- 172.21.15.53:0/1222409327 wait complete. 2023-10-21T16:02:27.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.427+0000 7fc4c15db640 1 --2- 172.21.15.53:0/2661527141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40539f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4055c30 tx=0x7fc4b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61649ce6a360fa46 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.427+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2661527141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031830 con 0x7fc4b40e3320 2023-10-21T16:02:27.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.427+0000 7f584fd33640 1 Processor -- start 2023-10-21T16:02:27.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.427+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2661527141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8031990 con 0x7fc4b40e3320 2023-10-21T16:02:27.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.427+0000 7fc4b3fff640 1 -- 172.21.15.53:0/2661527141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031d20 con 0x7fc4b40e3320 2023-10-21T16:02:27.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.428+0000 7f583402d640 1 Processor -- start 2023-10-21T16:02:27.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.428+0000 7f584fd33640 1 -- start start 2023-10-21T16:02:27.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.428+0000 7fc4c25dd640 1 -- 172.21.15.53:0/2661527141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 msgr2=0x7fc4b40539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.428+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/2661527141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40539f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4055c30 tx=0x7fc4b802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.428+0000 7f583402d640 1 -- start start 2023-10-21T16:02:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 -- 172.21.15.53:0/2661527141 shutdown_connections 2023-10-21T16:02:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/2661527141 >> v2:172.21.15.124:3300/0 conn(0x7fc4b40500f0 0x7fc4b40067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/2661527141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40539f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 -- 172.21.15.53:0/2661527141 >> 172.21.15.53:0/2661527141 conn(0x7fc4b4054c10 msgr2=0x7fc4b4054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 -- 172.21.15.53:0/2661527141 shutdown_connections 2023-10-21T16:02:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7fc4c25dd640 1 -- 172.21.15.53:0/2661527141 wait complete. 2023-10-21T16:02:27.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 0x7f58480affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 0x7f58480c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c97f0 con 0x7f584815ca20 2023-10-21T16:02:27.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480c9510 con 0x7f58480d0a40 2023-10-21T16:02:27.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.429+0000 7f584fd33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b6af0 con 0x7f584815ce00 2023-10-21T16:02:27.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f583402d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f583402d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 0x7f582c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f583402d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c0cb3c0 con 0x7f582c06a460 2023-10-21T16:02:27.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f584dacf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f583402d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0cb6a0 con 0x7f582c0ca220 2023-10-21T16:02:27.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f583402d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0cb980 con 0x7f582c0cede0 2023-10-21T16:02:27.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f58315c8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f58315c8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58016/0 (socket says 172.21.15.53:58016) 2023-10-21T16:02:27.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f58315c8640 1 -- 172.21.15.53:0/3722489811 learned_addr learned my addr 172.21.15.53:0/3722489811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f58325ca640 1 --2- 172.21.15.53:0/3722489811 >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 0x7f582c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f58315c8640 1 -- 172.21.15.53:0/3722489811 >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 msgr2=0x7f582c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f584e2d0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 0x7f58480c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f584e2d0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 0x7f58480c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58002/0 (socket says 172.21.15.53:58002) 2023-10-21T16:02:27.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f584d2ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 0x7f58480affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f584e2d0640 1 -- 172.21.15.53:0/1773811860 learned_addr learned my addr 172.21.15.53:0/1773811860 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.430+0000 7f584dacf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49242/0 (socket says 172.21.15.53:49242) 2023-10-21T16:02:27.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.431+0000 7f584d2ce640 1 -- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 msgr2=0x7f58480c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7fc4c25dd640 1 Processor -- start 2023-10-21T16:02:27.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f58315c8640 1 --2- 172.21.15.53:0/3722489811 >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 0x7f582c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f58315c8640 1 -- 172.21.15.53:0/3722489811 >> v1:172.21.15.53:6789/0 conn(0x7f582c06a460 legacy=0x7f582c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 0x7f58480c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f584d2ce640 1 -- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 msgr2=0x7f58480b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480b75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f584d2ce640 1 -- 172.21.15.53:0/1773811860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009400 con 0x7f584815ca20 2023-10-21T16:02:27.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f584d2ce640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 0x7f58480affd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5838032000 tx=0x7f58380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.433+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583803b8d0 con 0x7f584815ca20 2023-10-21T16:02:27.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.433+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583803ba50 con 0x7f584815ca20 2023-10-21T16:02:27.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.433+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583803be00 con 0x7f584815ca20 2023-10-21T16:02:27.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.433+0000 7f584fd33640 1 -- 172.21.15.53:0/1773811860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58480b6dd0 con 0x7f584815ca20 2023-10-21T16:02:27.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7fc4c25dd640 1 -- start start 2023-10-21T16:02:27.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f58315c8640 1 -- 172.21.15.53:0/3722489811 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0cbc60 con 0x7f582c0cede0 2023-10-21T16:02:27.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.432+0000 7f58325ca640 1 --2- 172.21.15.53:0/3722489811 >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 0x7f582c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:27.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.434+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd41c076de0 con 0x7fd42c070300 2023-10-21T16:02:27.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.433+0000 7f584fd33640 1 -- 172.21.15.53:0/1773811860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480b7b10 con 0x7f584815ca20 2023-10-21T16:02:27.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.434+0000 7f58315c8640 1 --2- 172.21.15.53:0/3722489811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f581c009870 tx=0x7f581c02b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6845ae77cc985d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.434+0000 7f5830dc7640 1 -- 172.21.15.53:0/3722489811 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c030860 con 0x7f582c0cede0 2023-10-21T16:02:27.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.434+0000 7f5830dc7640 1 -- 172.21.15.53:0/3722489811 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c0309c0 con 0x7f582c0cede0 2023-10-21T16:02:27.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.434+0000 7f5830dc7640 1 -- 172.21.15.53:0/3722489811 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c030d50 con 0x7f582c0cede0 2023-10-21T16:02:27.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.435+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5824005250 con 0x7f584815ca20 2023-10-21T16:02:27.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.435+0000 7fc4c25dd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.435+0000 7fc4c25dd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4b40500f0 0x7fc4b40e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c25dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40e1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 -- 172.21.15.53:0/3722489811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 msgr2=0x7f582c161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 --2- 172.21.15.53:0/3722489811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f581c009870 tx=0x7f581c02b9a0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c25dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40334d0 con 0x7fc4b40e3320 2023-10-21T16:02:27.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c25dd640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b417bc00 con 0x7fc4b40500f0 2023-10-21T16:02:27.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c25dd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b40ccc70 con 0x7fc4b403a830 2023-10-21T16:02:27.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 -- 172.21.15.53:0/3722489811 shutdown_connections 2023-10-21T16:02:27.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 --2- 172.21.15.53:0/3722489811 >> v2:172.21.15.53:3300/0 conn(0x7f582c0ca220 0x7f582c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 --2- 172.21.15.53:0/3722489811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0cede0 0x7f582c161d00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 -- 172.21.15.53:0/3722489811 >> 172.21.15.53:0/3722489811 conn(0x7f582c0660e0 msgr2=0x7f582c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c15db640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c15db640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58026/0 (socket says 172.21.15.53:58026) 2023-10-21T16:02:27.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c15db640 1 -- 172.21.15.53:0/1609190375 learned_addr learned my addr 172.21.15.53:0/1609190375 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7fc4c0dda640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4b40500f0 0x7fc4b40e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.436+0000 7f583402d640 1 -- 172.21.15.53:0/3722489811 shutdown_connections 2023-10-21T16:02:27.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7f583402d640 1 -- 172.21.15.53:0/3722489811 wait complete. 2023-10-21T16:02:27.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7f2f55fb0640 1 Processor -- start 2023-10-21T16:02:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c1ddc640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c15db640 1 -- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4b40500f0 msgr2=0x7fc4b40e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c15db640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4b40500f0 0x7fc4b40e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c15db640 1 -- 172.21.15.53:0/1609190375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 msgr2=0x7fc4b40e1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c15db640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40e1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7fc4c15db640 1 -- 172.21.15.53:0/1609190375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009400 con 0x7fc4b403a830 2023-10-21T16:02:27.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5838039040 con 0x7f584815ca20 2023-10-21T16:02:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.438+0000 7f583e7fc640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 0x7f582c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.438+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58380aace0 con 0x7f584815ca20 2023-10-21T16:02:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.438+0000 7fc4c15db640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8000c00 tx=0x7fc4b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.438+0000 7f584dacf640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 0x7f582c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.437+0000 7f2f55fb0640 1 -- start start 2023-10-21T16:02:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b803a830 con 0x7fc4b403a830 2023-10-21T16:02:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b803a9b0 con 0x7fc4b403a830 2023-10-21T16:02:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b803ad60 con 0x7fc4b403a830 2023-10-21T16:02:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7f583402d640 1 Processor -- start 2023-10-21T16:02:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7f2f55fb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b40328c0 con 0x7fc4b403a830 2023-10-21T16:02:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.439+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b4032e80 con 0x7fc4b403a830 2023-10-21T16:02:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f584dacf640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 0x7f582c06d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f584400a950 tx=0x7f5844008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.440+0000 7f583402d640 1 -- start start 2023-10-21T16:02:27.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.440+0000 7f2f55fb0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 0x7f2f500cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.440+0000 7f2f55fb0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2f500702b0 0x7f2f50070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.440+0000 7f2f55fb0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500cbb10 con 0x7f2f500cb150 2023-10-21T16:02:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f2f55fb0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5006cc50 con 0x7f2f500702b0 2023-10-21T16:02:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f2f55fb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5006cf30 con 0x7f2f50161690 2023-10-21T16:02:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.440+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58038/0 (socket says 172.21.15.53:58038) 2023-10-21T16:02:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f2f4f7fe640 1 -- 172.21.15.53:0/1189807145 learned_addr learned my addr 172.21.15.53:0/1189807145 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f2f4effd640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 0x7f2f500cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f583402d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 0x7f582c143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.441+0000 7f583402d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 0x7f582c0c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f583402d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 0x7f582c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f583402d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c1440e0 con 0x7f582c0cede0 2023-10-21T16:02:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f583402d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0c3f60 con 0x7f582c06a460 2023-10-21T16:02:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f583402d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c15c320 con 0x7f582c0ca220 2023-10-21T16:02:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4f7fe640 1 -- 172.21.15.53:0/1189807145 >> v2:172.21.15.124:3300/0 conn(0x7f2f500702b0 msgr2=0x7f2f50070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.124:3300/0 conn(0x7f2f500702b0 0x7f2f50070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b4005490 con 0x7fc4b403a830 2023-10-21T16:02:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f58315c8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 0x7f582c0c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4f7fe640 1 -- 172.21.15.53:0/1189807145 >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 msgr2=0x7f2f500cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 0x7f2f500cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f58325ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 0x7f582c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f58315c8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 0x7f582c0c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58046/0 (socket says 172.21.15.53:58046) 2023-10-21T16:02:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f58315c8640 1 -- 172.21.15.53:0/4120372998 learned_addr learned my addr 172.21.15.53:0/4120372998 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4f7fe640 1 -- 172.21.15.53:0/1189807145 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5006d210 con 0x7f2f50161690 2023-10-21T16:02:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.442+0000 7f2f4effd640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 0x7f2f500cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f82ea51d640 1 Processor -- start 2023-10-21T16:02:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f5831dc9640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 0x7f582c143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f58325ca640 1 -- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 msgr2=0x7f582c0c3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f58325ca640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 0x7f582c0c3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f82ea51d640 1 -- start start 2023-10-21T16:02:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f58325ca640 1 -- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 msgr2=0x7f582c143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f58325ca640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 0x7f582c143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f58325ca640 1 -- 172.21.15.53:0/4120372998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0094d0 con 0x7f582c0cede0 2023-10-21T16:02:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.443+0000 7f5831dc9640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 0x7f582c143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f58325ca640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 0x7f582c06c680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f582400cdf0 tx=0x7f582400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7fe9f5c7c640 1 Processor -- start 2023-10-21T16:02:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7fe9f5c7c640 1 -- start start 2023-10-21T16:02:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f82ea51d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 0x7f82e406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f82ea51d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 0x7f82e40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f82ea51d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e406cc50 con 0x7f82e40702b0 2023-10-21T16:02:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f82ea51d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40cbb10 con 0x7f82e4161690 2023-10-21T16:02:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.444+0000 7f82ea51d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e406cf30 con 0x7f82e40cb150 2023-10-21T16:02:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f82e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 0x7f82e406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f82e8aba640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f82e40702b0 0x7f82e4070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:39548/0 (socket says 172.21.15.53:39548) 2023-10-21T16:02:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f82e8aba640 1 -- 172.21.15.53:0/2605568483 learned_addr learned my addr 172.21.15.53:0/2605568483 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f82e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 0x7f82e406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59678/0 (socket says 172.21.15.53:59678) 2023-10-21T16:02:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f82dbfff640 1 --2- 172.21.15.53:0/2605568483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 0x7f82e40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.445+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1189807145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f500b1760 tx=0x7f2f4802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebeddc12ea631aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82e37fe640 1 -- 172.21.15.53:0/2605568483 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244790949 0 0) 0x7f82e406cc50 con 0x7f82e40702b0 2023-10-21T16:02:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82e37fe640 1 -- 172.21.15.53:0/2605568483 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d0003520 con 0x7f82e40702b0 2023-10-21T16:02:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82dbfff640 1 -- 172.21.15.53:0/2605568483 >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 msgr2=0x7f82e406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7fe9f5c7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9f0159eb0 0x7fe9f00b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7fe9f5c7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 0x7fe9f00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7fe9f5c7c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00c8d90 con 0x7fe9f00cec10 2023-10-21T16:02:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7fe9f5c7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf080 con 0x7fe9f0159eb0 2023-10-21T16:02:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7fe9f5c7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00b3f20 con 0x7fe9f00b3a00 2023-10-21T16:02:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82dbfff640 1 --2- 172.21.15.53:0/2605568483 >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 0x7f82e406dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82dbfff640 1 -- 172.21.15.53:0/2605568483 >> v1:172.21.15.124:6789/0 conn(0x7f82e40702b0 legacy=0x7f82e4070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.446+0000 7f82dbfff640 1 -- 172.21.15.53:0/2605568483 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e406d210 con 0x7f82e40cb150 2023-10-21T16:02:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4b804d450 con 0x7fc4b403a830 2023-10-21T16:02:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7fe9ef7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9f0159eb0 0x7fe9f00b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7ff3eeec9640 1 Processor -- start 2023-10-21T16:02:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7fe9ef7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9f0159eb0 0x7fe9f00b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59680/0 (socket says 172.21.15.53:59680) 2023-10-21T16:02:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7fe9ef7fe640 1 -- 172.21.15.53:0/3966290168 learned_addr learned my addr 172.21.15.53:0/3966290168 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.447+0000 7fe9eeffd640 1 --2- 172.21.15.53:0/3966290168 >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 0x7fe9f00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fe9ef7fe640 1 -- 172.21.15.53:0/3966290168 >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 msgr2=0x7fe9f00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7f82dbfff640 1 --2- 172.21.15.53:0/2605568483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 0x7f82e40cb5d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82d40097a0 tx=0x7f82d402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbe63da9225709c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7ff3eeec9640 1 -- start start 2023-10-21T16:02:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fc4b1ffb640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 0x7fc4a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc4b80abdc0 con 0x7fc4b403a830 2023-10-21T16:02:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/3966290168 >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 0x7fe9f00ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fe9ef7fe640 1 -- 172.21.15.53:0/3966290168 >> v1:172.21.15.124:6789/0 conn(0x7fe9f00cec10 legacy=0x7fe9f00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fe9ef7fe640 1 -- 172.21.15.53:0/3966290168 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00c9050 con 0x7fe9f0159eb0 2023-10-21T16:02:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.448+0000 7fe9eeffd640 1 --2- 172.21.15.53:0/3966290168 >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 0x7fe9f00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.449+0000 7fc4c0dda640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 0x7fc4a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.449+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400d840 con 0x7f582c0cede0 2023-10-21T16:02:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.449+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582400d9a0 con 0x7f582c0cede0 2023-10-21T16:02:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.449+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400dd90 con 0x7f582c0cede0 2023-10-21T16:02:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.449+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5838075a00 con 0x7f584815ca20 2023-10-21T16:02:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c15ac90 con 0x7f582c0cede0 2023-10-21T16:02:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c15b250 con 0x7f582c0cede0 2023-10-21T16:02:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3eeec9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff3e80b4e10 0x7ff3e81612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3eeec9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3eeec9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80b8c30 con 0x7ff3e815cd70 2023-10-21T16:02:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3eeec9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80cdd80 con 0x7ff3e80b4e10 2023-10-21T16:02:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3eeec9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80ce060 con 0x7ff3e80cd4e0 2023-10-21T16:02:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7fc4c0dda640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 0x7fc4a006d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40078b0 tx=0x7fc4ac007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/3966290168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f0159eb0 0x7fe9f00b2a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9f006a630 tx=0x7fe9e802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7421dc9ee5b437d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3ecc65640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3e815cd70 0x7ff3e80b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59764/0 (socket says 172.21.15.53:59764) 2023-10-21T16:02:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3ecc65640 1 -- 172.21.15.53:0/1335763588 learned_addr learned my addr 172.21.15.53:0/1335763588 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.450+0000 7ff3e7fff640 1 --2- 172.21.15.53:0/1335763588 >> v2:172.21.15.124:3300/0 conn(0x7ff3e80b4e10 0x7ff3e81612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7ff3ed466640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7fe9ee7fc640 1 -- 172.21.15.53:0/3966290168 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8030850 con 0x7fe9f0159eb0 2023-10-21T16:02:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7fe9ee7fc640 1 -- 172.21.15.53:0/3966290168 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80309b0 con 0x7fe9f0159eb0 2023-10-21T16:02:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7fe9ee7fc640 1 -- 172.21.15.53:0/3966290168 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8030d40 con 0x7fe9f0159eb0 2023-10-21T16:02:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7ff3e7fff640 1 -- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 msgr2=0x7ff3e80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7ff3e7fff640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e80d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7ff3e7fff640 1 -- 172.21.15.53:0/1335763588 >> v1:172.21.15.53:6789/0 conn(0x7ff3e815cd70 legacy=0x7ff3e80b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.451+0000 7ff3e7fff640 1 -- 172.21.15.53:0/1335763588 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80ce340 con 0x7ff3e80b4e10 2023-10-21T16:02:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c15b440 con 0x7f582c0cede0 2023-10-21T16:02:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7ff3ed466640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/3966290168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f0159eb0 msgr2=0x7fe9f00b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/3966290168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f0159eb0 0x7fe9f00b2a20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe9f006a630 tx=0x7fe9e802b990 comp rx=0 tx=0).stop 2023-10-21T16:02:27.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/3966290168 shutdown_connections 2023-10-21T16:02:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/3966290168 >> v2:172.21.15.124:3300/0 conn(0x7fe9f00b3a00 0x7fe9f00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/3966290168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f0159eb0 0x7fe9f00b2a20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/3966290168 >> 172.21.15.53:0/3966290168 conn(0x7fe9f00660e0 msgr2=0x7fe9f0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.452+0000 7ff3e7fff640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e81612f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40097a0 tx=0x7ff3d402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc821ac5b177e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7ff3e77fe640 1 -- 172.21.15.53:0/1335763588 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d402d040 con 0x7ff3e80b4e10 2023-10-21T16:02:27.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f582401e3b0 con 0x7f582c0cede0 2023-10-21T16:02:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7ff3e77fe640 1 -- 172.21.15.53:0/1335763588 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4031810 con 0x7ff3e80b4e10 2023-10-21T16:02:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7ff3e77fe640 1 -- 172.21.15.53:0/1335763588 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4031ba0 con 0x7ff3e80b4e10 2023-10-21T16:02:27.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/3966290168 shutdown_connections 2023-10-21T16:02:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 -- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 msgr2=0x7ff3e81612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e81612f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40097a0 tx=0x7ff3d402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.453+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/3966290168 wait complete. 2023-10-21T16:02:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 -- 172.21.15.53:0/1335763588 shutdown_connections 2023-10-21T16:02:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/1335763588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e81612f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 -- 172.21.15.53:0/1335763588 >> 172.21.15.53:0/1335763588 conn(0x7ff3e80cf700 msgr2=0x7ff3e80cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 -- 172.21.15.53:0/1335763588 shutdown_connections 2023-10-21T16:02:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7f58227fc640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 0x7f581006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.454+0000 7ff3eeec9640 1 -- 172.21.15.53:0/1335763588 wait complete. 2023-10-21T16:02:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.455+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f582408a2a0 con 0x7f582c0cede0 2023-10-21T16:02:27.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.455+0000 7f5831dc9640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 0x7f581006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.456+0000 7fe9f5c7c640 1 Processor -- start 2023-10-21T16:02:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.456+0000 7fe9f5c7c640 1 -- start start 2023-10-21T16:02:27.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.456+0000 7ff3eeec9640 1 Processor -- start 2023-10-21T16:02:27.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.456+0000 7f5831dc9640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 0x7f581006d380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5828007fd0 tx=0x7f5828009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 0x7fe9f0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9f0159eb0 0x7fe9f014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0153a30 con 0x7fe9f00b3a00 2023-10-21T16:02:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0153d10 con 0x7fe9f00cec10 2023-10-21T16:02:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7fe9f5c7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0153ff0 con 0x7fe9f0159eb0 2023-10-21T16:02:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.457+0000 7ff3eeec9640 1 -- start start 2023-10-21T16:02:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e80cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e814adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 0x7ff3e814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c7710 con 0x7ff3e815cd70 2023-10-21T16:02:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c79f0 con 0x7ff3e80b4e10 2023-10-21T16:02:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9effff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9f0159eb0 0x7fe9f014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49288/0 (socket says 172.21.15.53:49288) 2023-10-21T16:02:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 -- 172.21.15.53:0/4256542022 learned_addr learned my addr 172.21.15.53:0/4256542022 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 0x7fe9f0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3eeec9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c7cd0 con 0x7ff3e80cd4e0 2023-10-21T16:02:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 -- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9f0159eb0 msgr2=0x7fe9f014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9f0159eb0 0x7fe9f014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7fe9eeffd640 1 -- 172.21.15.53:0/4256542022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 msgr2=0x7fe9f0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e814adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3ed466640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 0x7ff3e814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.458+0000 7ff3ecc65640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e80cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ecc65640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e80cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49304/0 (socket says 172.21.15.53:49304) 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ecc65640 1 -- 172.21.15.53:0/4048342400 learned_addr learned my addr 172.21.15.53:0/4048342400 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e814adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58060/0 (socket says 172.21.15.53:58060) 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 -- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 msgr2=0x7ff3e814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e814adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 -- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 msgr2=0x7ff3e80cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e80cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 -- 172.21.15.53:0/4048342400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009400 con 0x7ff3e815cd70 2023-10-21T16:02:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7fe9eeffd640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 0x7fe9f0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7fe9eeffd640 1 -- 172.21.15.53:0/4256542022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e80094d0 con 0x7fe9f00cec10 2023-10-21T16:02:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 0x7fe9f0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:27.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7ff3ed466640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 0x7ff3e814b300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007d40 tx=0x7ff3e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.459+0000 7fe9eeffd640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000cf60 tx=0x7fe9e000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0007e90 con 0x7ff3e815cd70 2023-10-21T16:02:27.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80c7fb0 con 0x7ff3e815cd70 2023-10-21T16:02:27.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000d840 con 0x7fe9f00cec10 2023-10-21T16:02:27.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e000d9a0 con 0x7fe9f00cec10 2023-10-21T16:02:27.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000dd30 con 0x7fe9f00cec10 2023-10-21T16:02:27.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f01542d0 con 0x7fe9f00cec10 2023-10-21T16:02:27.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e80cc2c0 con 0x7ff3e815cd70 2023-10-21T16:02:27.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58240550f0 con 0x7f582c0cede0 2023-10-21T16:02:27.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000f040 con 0x7ff3e815cd70 2023-10-21T16:02:27.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f2f4e7fc640 1 -- 172.21.15.53:0/1189807145 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4802ca80 con 0x7f2f50161690 2023-10-21T16:02:27.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.460+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0003bd0 con 0x7ff3e815cd70 2023-10-21T16:02:27.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f2f4e7fc640 1 -- 172.21.15.53:0/1189807145 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4802cbe0 con 0x7f2f50161690 2023-10-21T16:02:27.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f2f4e7fc640 1 -- 172.21.15.53:0/1189807145 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f480343f0 con 0x7f2f50161690 2023-10-21T16:02:27.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f82e37fe640 1 -- 172.21.15.53:0/2605568483 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d402d040 con 0x7f82e40cb150 2023-10-21T16:02:27.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f82e37fe640 1 -- 172.21.15.53:0/2605568483 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d4031810 con 0x7f82e40cb150 2023-10-21T16:02:27.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.461+0000 7f82e37fe640 1 -- 172.21.15.53:0/2605568483 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4031ba0 con 0x7f82e40cb150 2023-10-21T16:02:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1189807145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 msgr2=0x7f2f5006dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1189807145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f500b1760 tx=0x7f2f4802bf00 comp rx=0 tx=0).stop 2023-10-21T16:02:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 -- 172.21.15.53:0/2605568483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 msgr2=0x7f82e40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 --2- 172.21.15.53:0/2605568483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 0x7f82e40cb5d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82d40097a0 tx=0x7f82d402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 -- 172.21.15.53:0/2605568483 shutdown_connections 2023-10-21T16:02:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 --2- 172.21.15.53:0/2605568483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40cb150 0x7f82e40cb5d0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82d40097a0 tx=0x7f82d402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 --2- 172.21.15.53:0/2605568483 >> v2:172.21.15.53:3300/0 conn(0x7f82e4161690 0x7f82e406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f82ea51d640 1 -- 172.21.15.53:0/2605568483 >> 172.21.15.53:0/2605568483 conn(0x7f82e406a5d0 msgr2=0x7f82e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1189807145 shutdown_connections 2023-10-21T16:02:27.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.124:3300/0 conn(0x7f2f500702b0 0x7f2f50070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1189807145 >> v2:172.21.15.53:3300/0 conn(0x7f2f500cb150 0x7f2f500cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1189807145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f50161690 0x7f2f5006dad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.462+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1189807145 >> 172.21.15.53:0/1189807145 conn(0x7f2f5006a5d0 msgr2=0x7f2f500b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.463+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f01596a0 con 0x7fe9f00cec10 2023-10-21T16:02:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.464+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4b8076bf0 con 0x7fc4b403a830 2023-10-21T16:02:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.464+0000 7f82ea51d640 1 -- 172.21.15.53:0/2605568483 shutdown_connections 2023-10-21T16:02:27.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.464+0000 7f82ea51d640 1 -- 172.21.15.53:0/2605568483 wait complete. 2023-10-21T16:02:27.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.464+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f004e160 con 0x7fe9f00cec10 2023-10-21T16:02:27.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.463+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3e0003d30 con 0x7ff3e815cd70 2023-10-21T16:02:27.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.464+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c8005250 con 0x7ff3e815cd70 2023-10-21T16:02:27.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.465+0000 7ff3e57fa640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 0x7ff3cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.465+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3e0089a70 con 0x7ff3e815cd70 2023-10-21T16:02:27.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.467+0000 7f82ea51d640 1 Processor -- start 2023-10-21T16:02:27.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.463+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1189807145 shutdown_connections 2023-10-21T16:02:27.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.463+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1189807145 wait complete. 2023-10-21T16:02:27.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.465+0000 7f2f55fb0640 1 Processor -- start 2023-10-21T16:02:27.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.467+0000 7f82ea51d640 1 -- start start 2023-10-21T16:02:27.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.468+0000 7f2f55fb0640 1 -- start start 2023-10-21T16:02:27.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 0x7f82e4157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e4156e00 0x7f82e41597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e41572b0 con 0x7f82e40702b0 2023-10-21T16:02:27.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e415a510 con 0x7f82e4156e00 2023-10-21T16:02:27.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7f82ea51d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e4161ba0 con 0x7f82e4161690 2023-10-21T16:02:27.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.465+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe9e003a060 con 0x7fe9f00cec10 2023-10-21T16:02:27.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.467+0000 7fe9e7fff640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 0x7fe9d006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.467+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9e0038070 con 0x7fe9f00cec10 2023-10-21T16:02:27.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.469+0000 7ff3ecc65640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 0x7ff3cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f82dbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58068/0 (socket says 172.21.15.53:58068) 2023-10-21T16:02:27.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f82dbfff640 1 -- 172.21.15.53:0/115097351 learned_addr learned my addr 172.21.15.53:0/115097351 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f2f55fb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f500702b0 0x7f2f501f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f2f55fb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 0x7f2f50159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 0x7fe9d006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.470+0000 7f2f55fb0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 0x7f2f50159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f2f55fb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5014a8a0 con 0x7f2f500cb150 2023-10-21T16:02:27.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7ff3ecc65640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 0x7ff3cc06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc000f60 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82e3fff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 0x7f82e4157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 -- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e4156e00 msgr2=0x7f82e41597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e4156e00 0x7f82e41597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 -- 172.21.15.53:0/115097351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 msgr2=0x7f82e4157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 0x7f82e4157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 -- 172.21.15.53:0/115097351 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4009400 con 0x7f82e4161690 2023-10-21T16:02:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f2f55fb0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5014ab40 con 0x7f2f50161690 2023-10-21T16:02:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f2f55fb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5014ae20 con 0x7f2f500702b0 2023-10-21T16:02:27.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82e3fff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 0x7f82e4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:27.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f2f4effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 0x7f2f50159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f2f4effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 0x7f2f50159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59712/0 (socket says 172.21.15.53:59712) 2023-10-21T16:02:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.471+0000 7f82dbfff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f82d402be40 tx=0x7f82d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.472+0000 7f2f4effd640 1 -- 172.21.15.53:0/1270503717 learned_addr learned my addr 172.21.15.53:0/1270503717 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.472+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4006e10 con 0x7f82e4161690 2023-10-21T16:02:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.472+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d403ad60 con 0x7f82e4161690 2023-10-21T16:02:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.472+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4044400 con 0x7f82e4161690 2023-10-21T16:02:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.472+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e4161e80 con 0x7f82e4161690 2023-10-21T16:02:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.473+0000 7f2f4ffff640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 0x7f2f50159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.473+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e4162440 con 0x7f82e4161690 2023-10-21T16:02:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.474+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f500702b0 0x7f2f501f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.474+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3e0054840 con 0x7ff3e815cd70 2023-10-21T16:02:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e4162620 con 0x7f82e4161690 2023-10-21T16:02:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f2f4ffff640 1 -- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f500702b0 msgr2=0x7f2f501f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f2f4ffff640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f500702b0 0x7f2f501f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f2f4ffff640 1 -- 172.21.15.53:0/1270503717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 msgr2=0x7f2f50159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f2f4ffff640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 0x7f2f50159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7f2f4ffff640 1 -- 172.21.15.53:0/1270503717 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009400 con 0x7f2f50161690 2023-10-21T16:02:27.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.475+0000 7fe9ef7fe640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 0x7fe9d006d380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8036000 tx=0x7fe9e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.477+0000 7f2f4ffff640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 0x7f2f50159e50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800b390 tx=0x7f2f3800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.477+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f82d4044560 con 0x7f82e4161690 2023-10-21T16:02:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.478+0000 7f82e17fa640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 0x7f82c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.478+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f82d40ab7a0 con 0x7f82e4161690 2023-10-21T16:02:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.478+0000 7f82e3fff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 0x7f82c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.479+0000 7f82e3fff640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 0x7f82c406d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009a20 tx=0x7f82cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.485+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011830 con 0x7f2f50161690 2023-10-21T16:02:27.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.485+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48009b30 con 0x7f2f50161690 2023-10-21T16:02:27.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.485+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f501649b0 con 0x7f2f50161690 2023-10-21T16:02:27.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.486+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9e00555f0 con 0x7fe9f00cec10 2023-10-21T16:02:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.486+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38011990 con 0x7f2f50161690 2023-10-21T16:02:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.486+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38011d20 con 0x7f2f50161690 2023-10-21T16:02:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.487+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f34005250 con 0x7f2f50161690 2023-10-21T16:02:27.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.489+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f3801a400 con 0x7f2f50161690 2023-10-21T16:02:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.489+0000 7f2f3ffff640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 0x7f2f3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.490+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f38089e70 con 0x7f2f50161690 2023-10-21T16:02:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.490+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 0x7f2f3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:27.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.491+0000 7f2f4f7fe640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 0x7f2f3006d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035000 tx=0x7f2f48030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:27.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.492+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82d40765f0 con 0x7f82e4161690 2023-10-21T16:02:27.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.502+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f38054cc0 con 0x7f2f50161690 2023-10-21T16:02:27.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.873+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f582c0cae10 con 0x7f582c0cede0 2023-10-21T16:02:27.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.874+0000 7f58227fc640 1 -- 172.21.15.53:0/4120372998 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5824058710 con 0x7f582c0cede0 2023-10-21T16:02:27.879 INFO:teuthology.orchestra.run.smithi053.stdout:55834574856 2023-10-21T16:02:27.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.876+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd404005500 con 0x7fd42c070300 2023-10-21T16:02:27.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.877+0000 7fd429ffb640 1 -- 172.21.15.53:0/692485016 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd41c043020 con 0x7fd42c070300 2023-10-21T16:02:27.882 INFO:teuthology.orchestra.run.smithi053.stdout:55834574856 2023-10-21T16:02:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.880+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 msgr2=0x7fd40c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 0x7fd40c06d400 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0b1070 tx=0x7fd414008040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 msgr2=0x7fd42c1608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 msgr2=0x7f581006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 0x7f581006d380 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5828007fd0 tx=0x7f5828009040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 msgr2=0x7f582c06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.881+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 0x7f582c06c680 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f582400cdf0 tx=0x7f582400a710 comp rx=0 tx=0).stop 2023-10-21T16:02:27.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.882+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c004a40 comp rx=0 tx=0).stop 2023-10-21T16:02:27.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.882+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 shutdown_connections 2023-10-21T16:02:27.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.882+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd42c1655c0 0x7fd42c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd40c06af40 0x7fd40c06d400 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd42c164ca0 0x7fd42c0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7fd422ffd640 1 --2- 172.21.15.53:0/692485016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd42c070300 0x7fd42c1608e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 >> 172.21.15.53:0/692485016 conn(0x7fd42c0caf90 msgr2=0x7fd42c0c0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 shutdown_connections 2023-10-21T16:02:27.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f581006aec0 0x7f581006d380 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5828007fd0 tx=0x7f5828009040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c0cede0 0x7f582c06c680 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f582c0ca220 0x7f582c0c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 --2- 172.21.15.53:0/4120372998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f582c06a460 0x7f582c143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.883+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 >> 172.21.15.53:0/4120372998 conn(0x7f582c0660e0 msgr2=0x7f582c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.885+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 shutdown_connections 2023-10-21T16:02:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.886+0000 7fd422ffd640 1 -- 172.21.15.53:0/692485016 wait complete. 2023-10-21T16:02:27.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.886+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 shutdown_connections 2023-10-21T16:02:27.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.886+0000 7f583402d640 1 -- 172.21.15.53:0/4120372998 wait complete. 2023-10-21T16:02:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.887+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5824005500 con 0x7f584815ca20 2023-10-21T16:02:27.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.888+0000 7f583e7fc640 1 -- 172.21.15.53:0/1773811860 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5838037090 con 0x7f584815ca20 2023-10-21T16:02:27.902 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 msgr2=0x7f582c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 0x7f582c06d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f584400a950 tx=0x7f5844008040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 msgr2=0x7f58480affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 0x7f58480affd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5838032000 tx=0x7f58380046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 shutdown_connections 2023-10-21T16:02:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f584815ce00 0x7f58480c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f582c06ae70 0x7f582c06d330 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f584400a950 tx=0x7f5844008040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f584815ca20 0x7f58480affd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 --2- 172.21.15.53:0/1773811860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58480d0a40 0x7f58480b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.894+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 >> 172.21.15.53:0/1773811860 conn(0x7f584809bc60 msgr2=0x7f58480b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.895+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 shutdown_connections 2023-10-21T16:02:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.895+0000 7f582bfff640 1 -- 172.21.15.53:0/1773811860 wait complete. 2023-10-21T16:02:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.898+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc4b40528b0 con 0x7fc4b403a830 2023-10-21T16:02:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.900+0000 7fc4b1ffb640 1 -- 172.21.15.53:0/1609190375 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4b8035020 con 0x7fc4b403a830 2023-10-21T16:02:27.908 INFO:teuthology.orchestra.run.smithi053.stdout:55834574856 2023-10-21T16:02:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.901+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe9f006d2d0 con 0x7fe9f00cec10 2023-10-21T16:02:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.902+0000 7fe9e7fff640 1 -- 172.21.15.53:0/4256542022 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe9e0058c10 con 0x7fe9f00cec10 2023-10-21T16:02:27.909 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.904+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff3c8005500 con 0x7ff3e815cd70 2023-10-21T16:02:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.904+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 msgr2=0x7fc4a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.904+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 0x7fc4a006d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40078b0 tx=0x7fc4ac007d30 comp rx=0 tx=0).stop 2023-10-21T16:02:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.904+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 msgr2=0x7fc4b417b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.904+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8000c00 tx=0x7fc4b80046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 shutdown_connections 2023-10-21T16:02:27.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc4a006ae70 0x7fc4a006d330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b40e3320 0x7fc4b40e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4b40500f0 0x7fc4b40e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 --2- 172.21.15.53:0/1609190375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4b403a830 0x7fc4b417b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 >> 172.21.15.53:0/1609190375 conn(0x7fc4b4054c10 msgr2=0x7fc4b4049b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7ff3e57fa640 1 -- 172.21.15.53:0/4048342400 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3e001a090 con 0x7ff3e815cd70 2023-10-21T16:02:27.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 msgr2=0x7fe9d006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.913 INFO:teuthology.orchestra.run.smithi053.stdout:55834574857 2023-10-21T16:02:27.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 shutdown_connections 2023-10-21T16:02:27.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fc4c25dd640 1 -- 172.21.15.53:0/1609190375 wait complete. 2023-10-21T16:02:27.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 0x7fe9d006d380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8036000 tx=0x7fe9e8008030 comp rx=0 tx=0).stop 2023-10-21T16:02:27.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 msgr2=0x7fe9f0153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.905+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000cf60 tx=0x7fe9e000cf90 comp rx=0 tx=0).stop 2023-10-21T16:02:27.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 shutdown_connections 2023-10-21T16:02:27.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9f0159eb0 0x7fe9f014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9f00cec10 0x7fe9f0153190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe9d006aec0 0x7fe9d006d380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 --2- 172.21.15.53:0/4256542022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9f00b3a00 0x7fe9f0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.906+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 >> 172.21.15.53:0/4256542022 conn(0x7fe9f00660e0 msgr2=0x7fe9f00ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.907+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 shutdown_connections 2023-10-21T16:02:27.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.907+0000 7fe9f5c7c640 1 -- 172.21.15.53:0/4256542022 wait complete. 2023-10-21T16:02:27.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.909+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 msgr2=0x7ff3cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.909+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 0x7ff3cc06d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc000f60 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.909+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 msgr2=0x7ff3e814b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.909+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 0x7ff3e814b300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0007d40 tx=0x7ff3e000d4a0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 shutdown_connections 2023-10-21T16:02:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff3cc06ae70 0x7ff3cc06d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e815cd70 0x7ff3e814b300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e80cd4e0 0x7ff3e814adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 --2- 172.21.15.53:0/4048342400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e80b4e10 0x7ff3e80cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.910+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 >> 172.21.15.53:0/4048342400 conn(0x7ff3e80cf700 msgr2=0x7ff3e80b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.913+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 shutdown_connections 2023-10-21T16:02:27.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.913+0000 7ff3eeec9640 1 -- 172.21.15.53:0/4048342400 wait complete. 2023-10-21T16:02:27.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-21T16:02:27.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-10-21T16:02:27.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2023-10-21T16:02:27.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2023-10-21T16:02:27.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.3 2023-10-21T16:02:27.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.933+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2f340057d0 con 0x7f2f50161690 2023-10-21T16:02:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.934+0000 7f2f3ffff640 1 -- 172.21.15.53:0/1270503717 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f38015020 con 0x7f2f50161690 2023-10-21T16:02:27.936 INFO:teuthology.orchestra.run.smithi053.stdout:55834574856 2023-10-21T16:02:27.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-10-21T16:02:27.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 msgr2=0x7f2f3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 0x7f2f3006d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2f48035000 tx=0x7f2f48030000 comp rx=0 tx=0).stop 2023-10-21T16:02:27.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 msgr2=0x7f2f50159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 0x7f2f50159e50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800b390 tx=0x7f2f3800b850 comp rx=0 tx=0).stop 2023-10-21T16:02:27.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 shutdown_connections 2023-10-21T16:02:27.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f50161690 0x7f2f50159e50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2f3006ae70 0x7f2f3006d330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f500cb150 0x7f2f50159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 --2- 172.21.15.53:0/1270503717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f500702b0 0x7f2f501f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 >> 172.21.15.53:0/1270503717 conn(0x7f2f5006a5d0 msgr2=0x7f2f500c5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 shutdown_connections 2023-10-21T16:02:27.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.937+0000 7f2f55fb0640 1 -- 172.21.15.53:0/1270503717 wait complete. 2023-10-21T16:02:27.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.951+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f82e4161ce0 con 0x7f82e4161690 2023-10-21T16:02:27.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.952+0000 7f82e17fa640 1 -- 172.21.15.53:0/115097351 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82e4161ce0 con 0x7f82e4161690 2023-10-21T16:02:27.953 INFO:teuthology.orchestra.run.smithi053.stdout:55834574856 2023-10-21T16:02:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 msgr2=0x7f82c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 0x7f82c406d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009a20 tx=0x7f82cc008040 comp rx=0 tx=0).stop 2023-10-21T16:02:27.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 msgr2=0x7f82e41568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:27.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f82d402be40 tx=0x7f82d40046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 shutdown_connections 2023-10-21T16:02:27.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e4156e00 0x7f82e41597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e4161690 0x7f82e41568c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f82c406ae70 0x7f82c406d330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 --2- 172.21.15.53:0/115097351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e40702b0 0x7f82e4157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:27.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 >> 172.21.15.53:0/115097351 conn(0x7f82e406a5d0 msgr2=0x7f82e40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:27.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 shutdown_connections 2023-10-21T16:02:27.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:27.954+0000 7f82ea51d640 1 -- 172.21.15.53:0/115097351 wait complete. 2023-10-21T16:02:27.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-10-21T16:02:27.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-10-21T16:02:28.922 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-21T16:02:28.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-21T16:02:28.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-21T16:02:28.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-21T16:02:28.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-21T16:02:28.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-21T16:02:28.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-21T16:02:28.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-21T16:02:29.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.229+0000 7fa3b1fe7640 1 Processor -- start 2023-10-21T16:02:29.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.230+0000 7fa3b1fe7640 1 -- start start 2023-10-21T16:02:29.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.232+0000 7fa3b1fe7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3ac15d070 0x7fa3ac1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.232+0000 7fa3b1fe7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac06d3b0 0x7fa3ac0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.232+0000 7fa3b1fe7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac0cbb40 con 0x7fa3ac0cb150 2023-10-21T16:02:29.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.232+0000 7fa3b1fe7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac161af0 con 0x7fa3ac15d070 2023-10-21T16:02:29.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.232+0000 7fa3b1fe7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0c8d90 con 0x7fa3ac06d3b0 2023-10-21T16:02:29.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.233+0000 7fa3ab7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3ac15d070 0x7fa3ac1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.233+0000 7fa3ab7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3ac15d070 0x7fa3ac1615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59714/0 (socket says 172.21.15.53:59714) 2023-10-21T16:02:29.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.233+0000 7fa3ab7fe640 1 -- 172.21.15.53:0/547707462 learned_addr learned my addr 172.21.15.53:0/547707462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.233+0000 7fa3abfff640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac06d3b0 0x7fa3ac0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.234+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625576571 0 0) 0x7fa3ac0cbb40 con 0x7fa3ac0cb150 2023-10-21T16:02:29.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.234+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa394003520 con 0x7fa3ac0cb150 2023-10-21T16:02:29.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443554967 0 0) 0x7fa394003520 con 0x7fa3ac0cb150 2023-10-21T16:02:29.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac06d3b0 msgr2=0x7fa3ac0cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac06d3b0 0x7fa3ac0cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 >> v2:172.21.15.53:3300/0 conn(0x7fa3ac15d070 msgr2=0x7fa3ac1615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 --2- 172.21.15.53:0/547707462 >> v2:172.21.15.53:3300/0 conn(0x7fa3ac15d070 0x7fa3ac1615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0c9070 con 0x7fa3ac0cb150 2023-10-21T16:02:29.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa39c001670 con 0x7fa3ac0cb150 2023-10-21T16:02:29.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 >> v1:172.21.15.53:6789/0 conn(0x7fa3ac0cb150 legacy=0x7fa3ac0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa394003970 0x7fa394003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3940047c0 0x7fa394004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa394005180 0x7fa39400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0c8d90 con 0x7fa394005180 2023-10-21T16:02:29.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39400bec0 con 0x7fa394003970 2023-10-21T16:02:29.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39400c000 con 0x7fa3940047c0 2023-10-21T16:02:29.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.235+0000 7fa3abfff640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac06d3b0 0x7fa3ac0cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:29.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.236+0000 7fa108a2e640 1 Processor -- start 2023-10-21T16:02:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.236+0000 7fa3abfff640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa394005180 0x7fa39400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.236+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3940047c0 0x7fa394004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.236+0000 7fa3ab7fe640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa394003970 0x7fa394003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.237+0000 7fa3aaffd640 1 -- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa394003970 msgr2=0x7fa394003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.237+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa394003970 0x7fa394003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.237+0000 7fa3aaffd640 1 -- 172.21.15.53:0/547707462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa394005180 msgr2=0x7fa39400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.237+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa394005180 0x7fa39400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.238+0000 7fa3aaffd640 1 -- 172.21.15.53:0/547707462 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0c9070 con 0x7fa3940047c0 2023-10-21T16:02:29.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.236+0000 7fa108a2e640 1 -- start start 2023-10-21T16:02:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.238+0000 7fa108a2e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1040b1b90 0x7fa10406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.238+0000 7fa108a2e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040d1a00 con 0x7fa1040c5910 2023-10-21T16:02:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.238+0000 7fa108a2e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040d33d0 con 0x7fa1040b1f70 2023-10-21T16:02:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.238+0000 7fa108a2e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040d3670 con 0x7fa1040b1b90 2023-10-21T16:02:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.239+0000 7fa3abfff640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa394005180 0x7fa39400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.239+0000 7fa1037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1040b1b90 0x7fa10406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.239+0000 7fa1037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1040b1b90 0x7fa10406afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59736/0 (socket says 172.21.15.53:59736) 2023-10-21T16:02:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.239+0000 7fa1037fe640 1 -- 172.21.15.53:0/2386419782 learned_addr learned my addr 172.21.15.53:0/2386419782 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.240+0000 7fa1037fe640 1 -- 172.21.15.53:0/2386419782 >> v1:172.21.15.124:6789/0 conn(0x7fa1040b1f70 legacy=0x7fa1040cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.240+0000 7fa1037fe640 1 -- 172.21.15.53:0/2386419782 >> v1:172.21.15.53:6789/0 conn(0x7fa1040c5910 legacy=0x7fa1040d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.240+0000 7fa102ffd640 1 -- 172.21.15.53:0/2386419782 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112632012 0 0) 0x7fa1040d33d0 con 0x7fa1040b1f70 2023-10-21T16:02:29.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.241+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3940047c0 0x7fa394004c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c190 tx=0x7fa3940057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a159a41682e3a4b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.241+0000 7fb2e446f640 1 Processor -- start 2023-10-21T16:02:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.241+0000 7fa1037fe640 1 -- 172.21.15.53:0/2386419782 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040cd870 con 0x7fa1040b1b90 2023-10-21T16:02:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7f397caec640 1 Processor -- start 2023-10-21T16:02:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7f397caec640 1 -- start start 2023-10-21T16:02:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7fb2e446f640 1 -- start start 2023-10-21T16:02:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c02ca30 con 0x7fa3940047c0 2023-10-21T16:02:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c02cbb0 con 0x7fa3940047c0 2023-10-21T16:02:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7fa3aa7fc640 1 -- 172.21.15.53:0/547707462 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c034440 con 0x7fa3940047c0 2023-10-21T16:02:29.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.242+0000 7fa1037fe640 1 --2- 172.21.15.53:0/2386419782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1b90 0x7fa10406afb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009e50 tx=0x7fa0f402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e383c8763615b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3940047c0 msgr2=0x7fa394004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/547707462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3940047c0 0x7fa394004c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c190 tx=0x7fa3940057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa3abfff640 1 -- 172.21.15.53:0/547707462 reap_dead start 2023-10-21T16:02:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7f397caec640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f397806c3a0 0x7f39780c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7f397caec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 0x7f39780b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7f397caec640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397815b310 con 0x7f397806db60 2023-10-21T16:02:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7f397caec640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39780ca230 con 0x7f397806c3a0 2023-10-21T16:02:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7f397caec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397806d290 con 0x7f397815aa00 2023-10-21T16:02:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/547707462 shutdown_connections 2023-10-21T16:02:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/547707462 >> 172.21.15.53:0/547707462 conn(0x7fa3ac0660e0 msgr2=0x7fa3ac069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.243+0000 7fa102ffd640 1 -- 172.21.15.53:0/2386419782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f402cb20 con 0x7fa1040b1b90 2023-10-21T16:02:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.243+0000 7fa102ffd640 1 -- 172.21.15.53:0/2386419782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f402cc80 con 0x7fa1040b1b90 2023-10-21T16:02:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.243+0000 7fa102ffd640 1 -- 172.21.15.53:0/2386419782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40343f0 con 0x7fa1040b1b90 2023-10-21T16:02:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa108a2e640 1 -- 172.21.15.53:0/2386419782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1b90 msgr2=0x7fa10406afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2386419782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1b90 0x7fa10406afb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009e50 tx=0x7fa0f402bf80 comp rx=0 tx=0).stop 2023-10-21T16:02:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fb2e446f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.244+0000 7fb2e446f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc0b4d10 0x7fb2dc0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e446f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc0b54a0 con 0x7fb2dc15cf80 2023-10-21T16:02:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e446f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc0b7130 con 0x7fb2dc15cba0 2023-10-21T16:02:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e446f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0b7410 con 0x7fb2dc0b4d10 2023-10-21T16:02:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fa108a2e640 1 -- 172.21.15.53:0/2386419782 shutdown_connections 2023-10-21T16:02:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2386419782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1b90 0x7fa10406afb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fa108a2e640 1 -- 172.21.15.53:0/2386419782 >> 172.21.15.53:0/2386419782 conn(0x7fa10409bc40 msgr2=0x7fa1040d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7f396dd9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f397806db60 0x7f397806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59784/0 (socket says 172.21.15.53:59784) 2023-10-21T16:02:29.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7f396dd9b640 1 -- 172.21.15.53:0/3997446203 learned_addr learned my addr 172.21.15.53:0/3997446203 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:29.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e220b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/547707462 shutdown_connections 2023-10-21T16:02:29.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fa108a2e640 1 -- 172.21.15.53:0/2386419782 shutdown_connections 2023-10-21T16:02:29.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e220b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49352/0 (socket says 172.21.15.53:49352) 2023-10-21T16:02:29.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7fb2e220b640 1 -- 172.21.15.53:0/2016853324 learned_addr learned my addr 172.21.15.53:0/2016853324 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/547707462 wait complete. 2023-10-21T16:02:29.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.245+0000 7f3976d9d640 1 --2- 172.21.15.53:0/3997446203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 0x7f39780b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7f397659c640 1 --2- 172.21.15.53:0/3997446203 >> v2:172.21.15.124:3300/0 conn(0x7f397806c3a0 0x7f39780c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7f3976d9d640 1 -- 172.21.15.53:0/3997446203 >> v2:172.21.15.124:3300/0 conn(0x7f397806c3a0 msgr2=0x7f39780c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7f3976d9d640 1 --2- 172.21.15.53:0/3997446203 >> v2:172.21.15.124:3300/0 conn(0x7f397806c3a0 0x7f39780c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7f3976d9d640 1 -- 172.21.15.53:0/3997446203 >> v1:172.21.15.53:6789/0 conn(0x7f397806db60 legacy=0x7f397806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7f3976d9d640 1 -- 172.21.15.53:0/3997446203 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780c9860 con 0x7f397815aa00 2023-10-21T16:02:29.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fa108a2e640 1 -- 172.21.15.53:0/2386419782 wait complete. 2023-10-21T16:02:29.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fb2e220b640 1 -- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc0b4d10 msgr2=0x7fb2dc0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fb2e220b640 1 --2- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc0b4d10 0x7fb2dc0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fb2e220b640 1 -- 172.21.15.53:0/2016853324 >> v1:172.21.15.124:6789/0 conn(0x7fb2dc15cf80 legacy=0x7fb2dc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.246+0000 7fb2e220b640 1 -- 172.21.15.53:0/2016853324 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0b76f0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fbd1b1cb640 1 Processor -- start 2023-10-21T16:02:29.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fbd1b1cb640 1 -- start start 2023-10-21T16:02:29.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fa108a2e640 1 Processor -- start 2023-10-21T16:02:29.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7f3976d9d640 1 --2- 172.21.15.53:0/3997446203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 0x7f39780b2a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f397000bfa0 tx=0x7f397002e0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d9a5fb32dbbd51 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fb2e220b640 1 --2- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009e80 tx=0x7fb2d002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ebb1da3225f9dd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fa108a2e640 1 -- start start 2023-10-21T16:02:29.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.248+0000 7fa3b1fe7640 1 Processor -- start 2023-10-21T16:02:29.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7fa3b1fe7640 1 -- start start 2023-10-21T16:02:29.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7f3975d9b640 1 -- 172.21.15.53:0/3997446203 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397002ea40 con 0x7f397815aa00 2023-10-21T16:02:29.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7f3975d9b640 1 -- 172.21.15.53:0/3997446203 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397002eba0 con 0x7f397815aa00 2023-10-21T16:02:29.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7f3975d9b640 1 -- 172.21.15.53:0/3997446203 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970009540 con 0x7f397815aa00 2023-10-21T16:02:29.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7fb2e1209640 1 -- 172.21.15.53:0/2016853324 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d002ca70 con 0x7fb2dc15cba0 2023-10-21T16:02:29.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7fb2e1209640 1 -- 172.21.15.53:0/2016853324 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d002cbd0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.249+0000 7fb2e1209640 1 -- 172.21.15.53:0/2016853324 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00343f0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 -- 172.21.15.53:0/3997446203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 msgr2=0x7f39780b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 --2- 172.21.15.53:0/3997446203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 0x7f39780b2a90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f397000bfa0 tx=0x7f397002e0a0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 -- 172.21.15.53:0/3997446203 shutdown_connections 2023-10-21T16:02:29.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 --2- 172.21.15.53:0/3997446203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397815aa00 0x7f39780b2a90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 --2- 172.21.15.53:0/3997446203 >> v2:172.21.15.124:3300/0 conn(0x7f397806c3a0 0x7f39780c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 -- 172.21.15.53:0/3997446203 >> 172.21.15.53:0/3997446203 conn(0x7f397806a5d0 msgr2=0x7f39780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fbd1b1cb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fbd1b1cb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd140ce510 0x7fbd1415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fbd1b1cb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd140cd520 con 0x7fbd140cf690 2023-10-21T16:02:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fbd1b1cb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd140d1910 con 0x7fbd140ce510 2023-10-21T16:02:29.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fbd1b1cb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140ce080 con 0x7fbd140cf2b0 2023-10-21T16:02:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fb2e446f640 1 -- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cba0 msgr2=0x7fb2dc0b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fb2e446f640 1 --2- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009e80 tx=0x7fb2d002bed0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fa108a2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1040b1b90 0x7fa10415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fa108a2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7fa108a2e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1040c5910 0x7fa1040c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.250+0000 7f397caec640 1 -- 172.21.15.53:0/3997446203 shutdown_connections 2023-10-21T16:02:29.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fa108a2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c68c0 con 0x7fa1040b1f70 2023-10-21T16:02:29.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fa108a2e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040afe10 con 0x7fa1040c5910 2023-10-21T16:02:29.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 -- 172.21.15.53:0/2016853324 shutdown_connections 2023-10-21T16:02:29.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 --2- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc0b4d10 0x7fb2dc0d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 --2- 172.21.15.53:0/2016853324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cba0 0x7fb2dc0b84a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 -- 172.21.15.53:0/2016853324 >> 172.21.15.53:0/2016853324 conn(0x7fb2dc0718a0 msgr2=0x7fb2dc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 -- 172.21.15.53:0/2016853324 shutdown_connections 2023-10-21T16:02:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fb2e446f640 1 -- 172.21.15.53:0/2016853324 wait complete. 2023-10-21T16:02:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fbd1a9ca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd140ce510 0x7fbd1415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7f397caec640 1 -- 172.21.15.53:0/3997446203 wait complete. 2023-10-21T16:02:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fbd1a9ca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd140ce510 0x7fbd1415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59750/0 (socket says 172.21.15.53:59750) 2023-10-21T16:02:29.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1561818308 learned_addr learned my addr 172.21.15.53:0/1561818308 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fa3b1fe7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3940047c0 0x7fa3ac0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fa3b1fe7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fa3b1fe7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 0x7fa3ac15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fa9088c6640 1 Processor -- start 2023-10-21T16:02:29.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fbd1a1c9640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3aaffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49380/0 (socket says 172.21.15.53:49380) 2023-10-21T16:02:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 0x7fa3ac15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 0x7fa3ac15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58168/0 (socket says 172.21.15.53:58168) 2023-10-21T16:02:29.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3aaffd640 1 -- 172.21.15.53:0/4031157708 learned_addr learned my addr 172.21.15.53:0/4031157708 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1561818308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 msgr2=0x7fbd140d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fbd1a9ca640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1561818308 >> v1:172.21.15.124:6789/0 conn(0x7fbd140cf690 legacy=0x7fbd140d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15fde0 con 0x7fa3940047c0 2023-10-21T16:02:29.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac1600c0 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa9088c6640 1 -- start start 2023-10-21T16:02:29.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1561818308 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140cd750 con 0x7fbd140ce510 2023-10-21T16:02:29.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.254+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac1603a0 con 0x7fa3ac15ee60 2023-10-21T16:02:29.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.251+0000 7fa108a2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b00f0 con 0x7fa1040b1b90 2023-10-21T16:02:29.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.252+0000 7fa0faffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59740/0 (socket says 172.21.15.53:59740) 2023-10-21T16:02:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 -- 172.21.15.53:0/2059549380 learned_addr learned my addr 172.21.15.53:0/2059549380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 -- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1040b1b90 msgr2=0x7fa10415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa1037fe640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1040b1b90 0x7fa10415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1040b1b90 0x7fa10415f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 -- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1040c5910 msgr2=0x7fa1040c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1040c5910 0x7fa1040c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7fa0faffd640 1 -- 172.21.15.53:0/2059549380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4009b30 con 0x7fa1040b1f70 2023-10-21T16:02:29.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.254+0000 7fa0faffd640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c430 tx=0x7fa0ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.254+0000 7fb2e446f640 1 Processor -- start 2023-10-21T16:02:29.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011790 con 0x7fa1040b1f70 2023-10-21T16:02:29.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fb2e446f640 1 -- start start 2023-10-21T16:02:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fbd1a1c9640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa3ab7fe640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3940047c0 0x7fa3ac0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fbd1a9ca640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140ce510 0x7fbd1415c820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009870 tx=0x7fbd0c02b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78343c3b79ed9f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 -- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 msgr2=0x7fa3ac15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 0x7fa3ac15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040c64e0 con 0x7fa1040b1f70 2023-10-21T16:02:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 -- 172.21.15.53:0/4031157708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3940047c0 msgr2=0x7fa3ac0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3940047c0 0x7fa3ac0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac1e5840 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0118f0 con 0x7fa1040b1f70 2023-10-21T16:02:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011790 con 0x7fa1040b1f70 2023-10-21T16:02:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa9088c6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9040d1ef0 0x7fa9041679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa9088c6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9040b54a0 con 0x7fa90415cf80 2023-10-21T16:02:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa9088c6640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9040b7130 con 0x7fa90415cba0 2023-10-21T16:02:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7fa9088c6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040b8bb0 con 0x7fa9040d1ef0 2023-10-21T16:02:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.253+0000 7f397caec640 1 Processor -- start 2023-10-21T16:02:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.254+0000 7f397caec640 1 -- start start 2023-10-21T16:02:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397806c3a0 0x7f39781f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f397806db60 0x7f39780cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978201100 con 0x7f397815aa00 2023-10-21T16:02:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39782004f0 con 0x7f397806db60 2023-10-21T16:02:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.255+0000 7f397caec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39782007d0 con 0x7f397806c3a0 2023-10-21T16:02:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59774/0 (socket says 172.21.15.53:59774) 2023-10-21T16:02:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7f3976d9d640 1 -- 172.21.15.53:0/279784105 learned_addr learned my addr 172.21.15.53:0/279784105 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fb2e446f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2dc0b4d10 0x7fb2dc158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e446f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e446f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cf80 0x7fb2dc0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa8f9d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa90415cf80 0x7fa9040b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59786/0 (socket says 172.21.15.53:59786) 2023-10-21T16:02:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa8f9d9b640 1 -- 172.21.15.53:0/3633080878 learned_addr learned my addr 172.21.15.53:0/3633080878 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa902d9d640 1 --2- 172.21.15.53:0/3633080878 >> v2:172.21.15.53:3300/0 conn(0x7fa9040d1ef0 0x7fa9041679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa3aaffd640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c2b0 tx=0x7fa39c000fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0037f0 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac1e5b20 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac15f820 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.256+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10415f640 con 0x7fa1040b1f70 2023-10-21T16:02:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e446f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1e7dc0 con 0x7fb2dc0b4d10 2023-10-21T16:02:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e446f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1e8010 con 0x7fb2dc15cf80 2023-10-21T16:02:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e446f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1e82f0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e220b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2dc0b4d10 0x7fb2dc158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e1a0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e1a0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:58200/0 (socket says 172.21.15.53:58200) 2023-10-21T16:02:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fb2e1a0a640 1 -- 172.21.15.53:0/3032046094 learned_addr learned my addr 172.21.15.53:0/3032046094 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e2a0c640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cf80 0x7fb2dc0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f397659c640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397806c3a0 0x7f39781f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f3976d9d640 1 -- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397806c3a0 msgr2=0x7f39781f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f3976d9d640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397806c3a0 0x7f39781f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f3976d9d640 1 -- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f397806db60 msgr2=0x7f39780cc640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-21T16:02:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f3976d9d640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f397806db60 0x7f39780cc640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7f3976d9d640 1 -- 172.21.15.53:0/279784105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397000bc80 con 0x7f397815aa00 2023-10-21T16:02:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fbd191c7640 1 -- 172.21.15.53:0/1561818308 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c030710 con 0x7fbd140ce510 2023-10-21T16:02:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fbd191c7640 1 -- 172.21.15.53:0/1561818308 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c02e050 con 0x7fbd140ce510 2023-10-21T16:02:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.257+0000 7fbd191c7640 1 -- 172.21.15.53:0/1561818308 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c030ad0 con 0x7fbd140ce510 2023-10-21T16:02:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c003950 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e1a0a640 1 -- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cf80 msgr2=0x7fb2dc0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e1a0a640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cf80 0x7fb2dc0d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e1a0a640 1 -- 172.21.15.53:0/3032046094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2dc0b4d10 msgr2=0x7fb2dc158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e1a0a640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2dc0b4d10 0x7fb2dc158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fb2e1a0a640 1 -- 172.21.15.53:0/3032046094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0009b30 con 0x7fb2dc15cba0 2023-10-21T16:02:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003ce0 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7f3976d9d640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3970038040 tx=0x7f3970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.258+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292287198 0 0) 0x7fa8f4001490 con 0x7fa90415cf80 2023-10-21T16:02:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f0003520 con 0x7fa90415cf80 2023-10-21T16:02:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fb2e1a0a640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b3f0 tx=0x7fb2c800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775883466 0 0) 0x7fa9040b7130 con 0x7fa90415cba0 2023-10-21T16:02:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397003f640 con 0x7f397815aa00 2023-10-21T16:02:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8011830 con 0x7fb2dc15cba0 2023-10-21T16:02:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3978200ab0 con 0x7f397815aa00 2023-10-21T16:02:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397003f7a0 con 0x7f397815aa00 2023-10-21T16:02:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397003f640 con 0x7f397815aa00 2023-10-21T16:02:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39781f6010 con 0x7f397815aa00 2023-10-21T16:02:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc1e85d0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8011990 con 0x7fb2dc15cba0 2023-10-21T16:02:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8011830 con 0x7fb2dc15cba0 2023-10-21T16:02:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f4001490 con 0x7fa90415cba0 2023-10-21T16:02:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc159390 con 0x7fb2dc15cba0 2023-10-21T16:02:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.259+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38c005250 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.261+0000 7fa902d9d640 1 -- 172.21.15.53:0/3633080878 >> v1:172.21.15.124:6789/0 conn(0x7fa90415cba0 legacy=0x7fa9040b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.261+0000 7fa902d9d640 1 -- 172.21.15.53:0/3633080878 >> v1:172.21.15.53:6789/0 conn(0x7fa90415cf80 legacy=0x7fa9040b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.261+0000 7fa902d9d640 1 -- 172.21.15.53:0/3633080878 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040b7410 con 0x7fa9040d1ef0 2023-10-21T16:02:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.260+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0ec011aa0 con 0x7fa1040b1f70 2023-10-21T16:02:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.261+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1040d1870 con 0x7fa1040b1f70 2023-10-21T16:02:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.262+0000 7fa100ff9640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 0x7fa0e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.262+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa0ec089410 con 0x7fa1040b1f70 2023-10-21T16:02:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.262+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc04e160 con 0x7fb2dc15cba0 2023-10-21T16:02:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.263+0000 7fa902d9d640 1 --2- 172.21.15.53:0/3633080878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa9041679a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0097d0 tx=0x7fa8fc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd5e58a144ba063 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.263+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa39c048450 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.263+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978201270 con 0x7f397815aa00 2023-10-21T16:02:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1561818308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140ce510 msgr2=0x7fbd1415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140ce510 0x7fbd1415c820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009870 tx=0x7fbd0c02b760 comp rx=0 tx=0).stop 2023-10-21T16:02:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2c8011b00 con 0x7fb2dc15cba0 2023-10-21T16:02:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fa1037fe640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 0x7fa0e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1561818308 shutdown_connections 2023-10-21T16:02:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140ce510 0x7fbd1415c820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1561818308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.264+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1561818308 >> 172.21.15.53:0/1561818308 conn(0x7fbd1409bc40 msgr2=0x7fbd140d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1561818308 shutdown_connections 2023-10-21T16:02:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1561818308 wait complete. 2023-10-21T16:02:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fb2daffd640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 0x7fb2c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb2c8089040 con 0x7fb2dc15cba0 2023-10-21T16:02:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fa393fff640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 0x7fa39806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.265+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa39c0abc90 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.266+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f397003f910 con 0x7f397815aa00 2023-10-21T16:02:29.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.267+0000 7fb2e220b640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 0x7fb2c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.267+0000 7fa3ab7fe640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 0x7fa39806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.267+0000 7fa1037fe640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 0x7fa0e406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4035000 tx=0x7fa0f4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7fbd1b1cb640 1 Processor -- start 2023-10-21T16:02:29.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7fb2e220b640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 0x7fb2c006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0035000 tx=0x7fb2d0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7fa3ab7fe640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 0x7fa39806d360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009a90 tx=0x7fa3a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7f396f7fe640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 0x7f395806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f39700aca00 con 0x7f397815aa00 2023-10-21T16:02:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.268+0000 7fbd1b1cb640 1 -- start start 2023-10-21T16:02:29.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7f397659c640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 0x7f395806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd140ce510 0x7fbd140b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140af4a0 con 0x7fbd140cf690 2023-10-21T16:02:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140b1870 con 0x7fbd140ce510 2023-10-21T16:02:29.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7f397659c640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 0x7f395806d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3968009a10 tx=0x7f3968008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.269+0000 7fbd1b1cb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140b04b0 con 0x7fbd140cf2b0 2023-10-21T16:02:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59798/0 (socket says 172.21.15.53:59798) 2023-10-21T16:02:29.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1491656572 learned_addr learned my addr 172.21.15.53:0/1491656572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a1c9640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd140ce510 0x7fbd140b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd199c8640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc02d040 con 0x7fa9040d1ef0 2023-10-21T16:02:29.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 msgr2=0x7fbd140b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd140ce510 msgr2=0x7fbd140b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd140ce510 0x7fbd140b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.270+0000 7fbd1a9ca640 1 -- 172.21.15.53:0/1491656572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0094d0 con 0x7fbd140cf690 2023-10-21T16:02:29.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.271+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0ec054260 con 0x7fa1040b1f70 2023-10-21T16:02:29.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.271+0000 7fbd1a9ca640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0090b0 tx=0x7fbd0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.278+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa39c0769d0 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc031810 con 0x7fa9040d1ef0 2023-10-21T16:02:29.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0329d0 con 0x7fbd140cf690 2023-10-21T16:02:29.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c032b30 con 0x7fbd140cf690 2023-10-21T16:02:29.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c03b400 con 0x7fbd140cf690 2023-10-21T16:02:29.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd140af6f0 con 0x7fbd140cf690 2023-10-21T16:02:29.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39700777c0 con 0x7f397815aa00 2023-10-21T16:02:29.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.279+0000 7fa901d9b640 1 -- 172.21.15.53:0/3633080878 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc031b80 con 0x7fa9040d1ef0 2023-10-21T16:02:29.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.280+0000 7fa9088c6640 1 -- 172.21.15.53:0/3633080878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 msgr2=0x7fa9041679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.280+0000 7fa9088c6640 1 --2- 172.21.15.53:0/3633080878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa9041679a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0097d0 tx=0x7fa8fc02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.280+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2c80bbca0 con 0x7fb2dc15cba0 2023-10-21T16:02:29.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.280+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd140afcb0 con 0x7fbd140cf690 2023-10-21T16:02:29.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.281+0000 7fa9088c6640 1 -- 172.21.15.53:0/3633080878 shutdown_connections 2023-10-21T16:02:29.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.281+0000 7fa9088c6640 1 --2- 172.21.15.53:0/3633080878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa9041679a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.281+0000 7fa9088c6640 1 -- 172.21.15.53:0/3633080878 >> 172.21.15.53:0/3633080878 conn(0x7fa9040718a0 msgr2=0x7fa904071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.281+0000 7fa9088c6640 1 -- 172.21.15.53:0/3633080878 shutdown_connections 2023-10-21T16:02:29.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.281+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd14074150 con 0x7fbd140cf690 2023-10-21T16:02:29.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.283+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbd0c039040 con 0x7fbd140cf690 2023-10-21T16:02:29.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.285+0000 7fbd12ffd640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 0x7fbcf806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.285+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbd0c0ab1f0 con 0x7fbd140cf690 2023-10-21T16:02:29.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.282+0000 7fa9088c6640 1 -- 172.21.15.53:0/3633080878 wait complete. 2023-10-21T16:02:29.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.284+0000 7fa9088c6640 1 Processor -- start 2023-10-21T16:02:29.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.287+0000 7fbd1a1c9640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 0x7fbcf806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.287+0000 7fa9088c6640 1 -- start start 2023-10-21T16:02:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90415cf80 0x7fa90406f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040b8f30 con 0x7fa9040d1ef0 2023-10-21T16:02:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406de20 con 0x7fa90415cba0 2023-10-21T16:02:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.289+0000 7fa9088c6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904157050 con 0x7fa90415cf80 2023-10-21T16:02:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.290+0000 7fa90259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.290+0000 7fa8f9d9b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.291+0000 7fa8f9d9b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49416/0 (socket says 172.21.15.53:49416) 2023-10-21T16:02:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.291+0000 7fa90259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59800/0 (socket says 172.21.15.53:59800) 2023-10-21T16:02:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.291+0000 7fa90259c640 1 -- 172.21.15.53:0/2871103242 learned_addr learned my addr 172.21.15.53:0/2871103242 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.291+0000 7fa902d9d640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90415cf80 0x7fa90406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.292+0000 7fa90259c640 1 -- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90415cf80 msgr2=0x7fa90406f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.292+0000 7fa90259c640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90415cf80 0x7fa90406f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.292+0000 7fa90259c640 1 -- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 msgr2=0x7fa904156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.292+0000 7fa90259c640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.293+0000 7fbd1a1c9640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 0x7fbcf806d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d620 tx=0x7fbd08007810 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.292+0000 7fa90259c640 1 -- 172.21.15.53:0/2871103242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009400 con 0x7fa9040d1ef0 2023-10-21T16:02:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.293+0000 7fa90259c640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c780 tx=0x7fa8ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.293+0000 7fa8f9d9b640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.294+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0052d0 con 0x7fa9040d1ef0 2023-10-21T16:02:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.294+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd0c076160 con 0x7fbd140cf690 2023-10-21T16:02:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.294+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec005430 con 0x7fa9040d1ef0 2023-10-21T16:02:29.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.294+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00a860 con 0x7fa9040d1ef0 2023-10-21T16:02:29.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.294+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90406d210 con 0x7fa9040d1ef0 2023-10-21T16:02:29.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.295+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90406d7d0 con 0x7fa9040d1ef0 2023-10-21T16:02:29.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.299+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90404e160 con 0x7fa9040d1ef0 2023-10-21T16:02:29.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.299+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8ec00a9c0 con 0x7fa9040d1ef0 2023-10-21T16:02:29.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.300+0000 7fa8fb7fe640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 0x7fa8e406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.301+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8ec089420 con 0x7fa9040d1ef0 2023-10-21T16:02:29.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.302+0000 7fa8f9d9b640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 0x7fa8e406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.307+0000 7fa8f9d9b640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 0x7fa8e406d400 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa904159680 tx=0x7fa8f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.311+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8ec042e60 con 0x7fa9040d1ef0 2023-10-21T16:02:29.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.316+0000 7f7f5d13e640 1 Processor -- start 2023-10-21T16:02:29.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.317+0000 7f7f5d13e640 1 -- start start 2023-10-21T16:02:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.318+0000 7f7f5d13e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7f580afda0 0x7f7f580c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5d13e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5816d910 con 0x7f7f58070f30 2023-10-21T16:02:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5d13e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f580cbef0 con 0x7f7f5806d440 2023-10-21T16:02:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5d13e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5806d1c0 con 0x7f7f580afda0 2023-10-21T16:02:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7f580afda0 0x7f7f580c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7f580afda0 0x7f7f580c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49428/0 (socket says 172.21.15.53:49428) 2023-10-21T16:02:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.319+0000 7f7f5759e640 1 -- 172.21.15.53:0/3682082828 learned_addr learned my addr 172.21.15.53:0/3682082828 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.322+0000 7f1803cc9640 1 Processor -- start 2023-10-21T16:02:29.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.322+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744460547 0 0) 0x7f7f580cbef0 con 0x7f7f5806d440 2023-10-21T16:02:29.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.322+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f40003520 con 0x7f7f5806d440 2023-10-21T16:02:29.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767748379 0 0) 0x7f7f5816d910 con 0x7f7f58070f30 2023-10-21T16:02:29.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f580cbef0 con 0x7f7f58070f30 2023-10-21T16:02:29.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430052367 0 0) 0x7f7f40003520 con 0x7f7f5806d440 2023-10-21T16:02:29.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 >> v2:172.21.15.124:3300/0 conn(0x7f7f580afda0 msgr2=0x7f7f580c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 --2- 172.21.15.53:0/3682082828 >> v2:172.21.15.124:3300/0 conn(0x7f7f580afda0 0x7f7f580c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 >> v1:172.21.15.53:6789/0 conn(0x7f7f58070f30 legacy=0x7f7f5806cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58089a90 con 0x7f7f5806d440 2023-10-21T16:02:29.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7f48001670 con 0x7f7f5806d440 2023-10-21T16:02:29.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 >> v1:172.21.15.124:6789/0 conn(0x7f7f5806d440 legacy=0x7f7f5806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 0x7f7f40003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f400047c0 0x7f7f40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f40005180 0x7f7f4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5806d1c0 con 0x7f7f40003b00 2023-10-21T16:02:29.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4000c000 con 0x7f7f400047c0 2023-10-21T16:02:29.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.323+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4000c1c0 con 0x7f7f40005180 2023-10-21T16:02:29.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 0x7f7f40003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.322+0000 7f1803cc9640 1 -- start start 2023-10-21T16:02:29.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1803cc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17f4050270 0x7f17f4007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1803cc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 0x7f17f4054150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1803cc9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f40063d0 con 0x7f17f40df3f0 2023-10-21T16:02:29.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1803cc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f40df7d0 con 0x7f17f4050270 2023-10-21T16:02:29.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1803cc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40066b0 con 0x7f17f4005ed0 2023-10-21T16:02:29.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f7f5759e640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f40005180 0x7f7f4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1801a65640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f17f40df3f0 0x7f17f404fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59810/0 (socket says 172.21.15.53:59810) 2023-10-21T16:02:29.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f1801a65640 1 -- 172.21.15.53:0/690496180 learned_addr learned my addr 172.21.15.53:0/690496180 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:29.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.324+0000 7f7f5659c640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f400047c0 0x7f7f40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f56d9d640 1 -- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f40005180 msgr2=0x7f7f4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f40005180 0x7f7f4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f56d9d640 1 -- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f400047c0 msgr2=0x7f7f40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f400047c0 0x7f7f40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f56d9d640 1 -- 172.21.15.53:0/3682082828 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58089a90 con 0x7f7f40003b00 2023-10-21T16:02:29.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f7f5759e640 1 -- 172.21.15.53:0/3682082828 reap_dead start 2023-10-21T16:02:29.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.327+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 0x7f7f40003f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c0e0 tx=0x7f7f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec928252db0de9cb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.327+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48030db0 con 0x7f7f40003b00 2023-10-21T16:02:29.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.327+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48038440 con 0x7f7f40003b00 2023-10-21T16:02:29.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.327+0000 7f7f55d9b640 1 -- 172.21.15.53:0/3682082828 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480387f0 con 0x7f7f40003b00 2023-10-21T16:02:29.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1802266640 1 --2- 172.21.15.53:0/690496180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 0x7f17f4054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1801264640 1 --2- 172.21.15.53:0/690496180 >> v2:172.21.15.124:3300/0 conn(0x7f17f4050270 0x7f17f4007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1802266640 1 -- 172.21.15.53:0/690496180 >> v2:172.21.15.124:3300/0 conn(0x7f17f4050270 msgr2=0x7f17f4007470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:29.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1802266640 1 --2- 172.21.15.53:0/690496180 >> v2:172.21.15.124:3300/0 conn(0x7f17f4050270 0x7f17f4007470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1802266640 1 -- 172.21.15.53:0/690496180 >> v1:172.21.15.53:6789/0 conn(0x7f17f40df3f0 legacy=0x7f17f404fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.325+0000 7f1802266640 1 -- 172.21.15.53:0/690496180 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4006990 con 0x7f17f4005ed0 2023-10-21T16:02:29.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.327+0000 7f1802266640 1 --2- 172.21.15.53:0/690496180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 0x7f17f4054150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17fc02cb40 tx=0x7f17fc0a56f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88dbd7c10ef678b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.328+0000 7f1800a63640 1 -- 172.21.15.53:0/690496180 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0aa7d0 con 0x7f17f4005ed0 2023-10-21T16:02:29.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.328+0000 7f1800a63640 1 -- 172.21.15.53:0/690496180 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0a8050 con 0x7f17f4005ed0 2023-10-21T16:02:29.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.328+0000 7f1800a63640 1 -- 172.21.15.53:0/690496180 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0aabb0 con 0x7f17f4005ed0 2023-10-21T16:02:29.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.328+0000 7f7f5d13e640 1 -- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 msgr2=0x7f7f40003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.328+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 0x7f7f40003f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c0e0 tx=0x7f7f400057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f7f5d13e640 1 -- 172.21.15.53:0/3682082828 shutdown_connections 2023-10-21T16:02:29.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/3682082828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f40003b00 0x7f7f40003f00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f7f5d13e640 1 -- 172.21.15.53:0/3682082828 >> 172.21.15.53:0/3682082828 conn(0x7f7f580660e0 msgr2=0x7f7f58069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f7f5d13e640 1 -- 172.21.15.53:0/3682082828 shutdown_connections 2023-10-21T16:02:29.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f7f5d13e640 1 -- 172.21.15.53:0/3682082828 wait complete. 2023-10-21T16:02:29.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f1803cc9640 1 -- 172.21.15.53:0/690496180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 msgr2=0x7f17f4054150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.329+0000 7f1803cc9640 1 --2- 172.21.15.53:0/690496180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 0x7f17f4054150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17fc02cb40 tx=0x7f17fc0a56f0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.330+0000 7f1803cc9640 1 -- 172.21.15.53:0/690496180 shutdown_connections 2023-10-21T16:02:29.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.330+0000 7f1803cc9640 1 --2- 172.21.15.53:0/690496180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4005ed0 0x7f17f4054150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.330+0000 7f1803cc9640 1 --2- 172.21.15.53:0/690496180 >> v2:172.21.15.124:3300/0 conn(0x7f17f4050270 0x7f17f4007470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.330+0000 7f1803cc9640 1 -- 172.21.15.53:0/690496180 >> 172.21.15.53:0/690496180 conn(0x7f17f4054b00 msgr2=0x7f17f4054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.331+0000 7f1803cc9640 1 -- 172.21.15.53:0/690496180 shutdown_connections 2023-10-21T16:02:29.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.331+0000 7f1803cc9640 1 -- 172.21.15.53:0/690496180 wait complete. 2023-10-21T16:02:29.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.332+0000 7f7f5d13e640 1 Processor -- start 2023-10-21T16:02:29.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.332+0000 7f7f5d13e640 1 -- start start 2023-10-21T16:02:29.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.334+0000 7f1803cc9640 1 Processor -- start 2023-10-21T16:02:29.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.334+0000 7f1803cc9640 1 -- start start 2023-10-21T16:02:29.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.334+0000 7f7f5d13e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 0x7f7f5816d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.334+0000 7f7f5d13e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f56d9d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 0x7f7f5816d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f56d9d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 0x7f7f5816d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:49444/0 (socket says 172.21.15.53:49444) 2023-10-21T16:02:29.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f5659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f5659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59816/0 (socket says 172.21.15.53:59816) 2023-10-21T16:02:29.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f56d9d640 1 -- 172.21.15.53:0/1191631241 learned_addr learned my addr 172.21.15.53:0/1191631241 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 0x7f7f580a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f1803cc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17f4005ed0 0x7f17f40d7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.335+0000 7f1803cc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4050270 0x7f17f404ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1803cc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1803cc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40d6740 con 0x7f17f40df3f0 2023-10-21T16:02:29.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1803cc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40d6a20 con 0x7f17f4005ed0 2023-10-21T16:02:29.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1803cc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40d6d00 con 0x7f17f4050270 2023-10-21T16:02:29.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f580a7eb0 con 0x7f7f580c8370 2023-10-21T16:02:29.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5816d620 con 0x7f7f40003b00 2023-10-21T16:02:29.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f581fef10 con 0x7f7f580a74c0 2023-10-21T16:02:29.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1802266640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1802266640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59832/0 (socket says 172.21.15.53:59832) 2023-10-21T16:02:29.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1801a65640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17f4005ed0 0x7f17f40d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1802266640 1 -- 172.21.15.53:0/2639815380 learned_addr learned my addr 172.21.15.53:0/2639815380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:29.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.336+0000 7f1801264640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4050270 0x7f17f404ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5759e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 0x7f7f580a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5659c640 1 -- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 msgr2=0x7f7f580a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5659c640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 0x7f7f580a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5659c640 1 -- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 msgr2=0x7f7f5816d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5659c640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 0x7f7f5816d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5659c640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5816a5b0 con 0x7f7f580c8370 2023-10-21T16:02:29.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f7f5759e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 0x7f7f580a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:29.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f1802266640 1 -- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4050270 msgr2=0x7f17f404ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f1802266640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4050270 0x7f17f404ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f1802266640 1 -- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17f4005ed0 msgr2=0x7f17f40d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.338+0000 7f7f5659c640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f580639b0 tx=0x7f7f4c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f1802266640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17f4005ed0 0x7f17f40d7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.337+0000 7f1802266640 1 -- 172.21.15.53:0/2639815380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f40d6fe0 con 0x7f17f40df3f0 2023-10-21T16:02:29.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.338+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c005110 con 0x7f7f580c8370 2023-10-21T16:02:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.338+0000 7f1802266640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17fc069580 tx=0x7f17fc061b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.339+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0b68e0 con 0x7f17f40df3f0 2023-10-21T16:02:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.339+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0b6a40 con 0x7f17f40df3f0 2023-10-21T16:02:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.339+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0b6db0 con 0x7f17f40df3f0 2023-10-21T16:02:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.339+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f40d63d0 con 0x7f17f40df3f0 2023-10-21T16:02:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.340+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c005270 con 0x7f7f580c8370 2023-10-21T16:02:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.340+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5816a890 con 0x7f7f580c8370 2023-10-21T16:02:29.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.339+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f40d7f40 con 0x7f17f40df3f0 2023-10-21T16:02:29.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.340+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c007050 con 0x7f7f580c8370 2023-10-21T16:02:29.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.340+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5816ae50 con 0x7f7f580c8370 2023-10-21T16:02:29.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.343+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5804e160 con 0x7f7f580c8370 2023-10-21T16:02:29.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.344+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f4050c60 con 0x7f17f40df3f0 2023-10-21T16:02:29.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.344+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17fc0b3050 con 0x7f17f40df3f0 2023-10-21T16:02:29.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.346+0000 7f17f27fc640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 0x7f17e006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.346+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f17fc124360 con 0x7f17f40df3f0 2023-10-21T16:02:29.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.347+0000 7f1801a65640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 0x7f17e006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.344+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f4c0071b0 con 0x7f7f580c8370 2023-10-21T16:02:29.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.346+0000 7f7f3f7fe640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 0x7f7f4406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:29.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.346+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7f4c088e70 con 0x7f7f580c8370 2023-10-21T16:02:29.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.348+0000 7f1801a65640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 0x7f17e006d380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000f60 tx=0x7f17f8007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.348+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 0x7f7f4406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.349+0000 7f7f56d9d640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 0x7f7f4406d360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f480011e0 tx=0x7f7f48009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:29.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.354+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17fc0ef2b0 con 0x7f17f40df3f0 2023-10-21T16:02:29.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.366+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f4c053c40 con 0x7f7f580c8370 2023-10-21T16:02:29.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.684+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f397804e160 con 0x7f397815aa00 2023-10-21T16:02:29.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.685+0000 7f396f7fe640 1 -- 172.21.15.53:0/279784105 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f397007ade0 con 0x7f397815aa00 2023-10-21T16:02:29.687 INFO:teuthology.orchestra.run.smithi053.stdout:55834574860 2023-10-21T16:02:29.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.688+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 msgr2=0x7f395806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 0x7f395806d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3968009a10 tx=0x7f3968008040 comp rx=0 tx=0).stop 2023-10-21T16:02:29.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 msgr2=0x7f39780ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3970038040 tx=0x7f3970004a40 comp rx=0 tx=0).stop 2023-10-21T16:02:29.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 shutdown_connections 2023-10-21T16:02:29.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f395806aed0 0x7f395806d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397815aa00 0x7f39780ccb80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f397806db60 0x7f39780cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 --2- 172.21.15.53:0/279784105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f397806c3a0 0x7f39781f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.689+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 >> 172.21.15.53:0/279784105 conn(0x7f397806a5d0 msgr2=0x7f3978073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.690+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 shutdown_connections 2023-10-21T16:02:29.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.690+0000 7f397caec640 1 -- 172.21.15.53:0/279784105 wait complete. 2023-10-21T16:02:29.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.700+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa1040c6a00 con 0x7fa1040b1f70 2023-10-21T16:02:29.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.701+0000 7fa100ff9640 1 -- 172.21.15.53:0/2059549380 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1040c6a00 con 0x7fa1040b1f70 2023-10-21T16:02:29.707 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:29.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 msgr2=0x7fa0e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 0x7fa0e406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4035000 tx=0x7fa0f4030000 comp rx=0 tx=0).stop 2023-10-21T16:02:29.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 msgr2=0x7fa1040af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c430 tx=0x7fa0ec00c8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 shutdown_connections 2023-10-21T16:02:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1040c5910 0x7fa1040c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa0e406aed0 0x7fa0e406d390 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4035000 tx=0x7fa0f4030000 comp rx=0 tx=0).stop 2023-10-21T16:02:29.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1040b1f70 0x7fa1040af4a0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c430 tx=0x7fa0ec00c8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 --2- 172.21.15.53:0/2059549380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1040b1b90 0x7fa10415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.708+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 >> 172.21.15.53:0/2059549380 conn(0x7fa10409bc40 msgr2=0x7fa1040cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.709+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 shutdown_connections 2023-10-21T16:02:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.710+0000 7fa108a2e640 1 -- 172.21.15.53:0/2059549380 wait complete. 2023-10-21T16:02:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.711+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb2dc0d3530 con 0x7fb2dc15cba0 2023-10-21T16:02:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.712+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa38c0057d0 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.712+0000 7fb2daffd640 1 -- 172.21.15.53:0/3032046094 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb2c8015090 con 0x7fb2dc15cba0 2023-10-21T16:02:29.720 INFO:teuthology.orchestra.run.smithi053.stdout:55834574859 2023-10-21T16:02:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.713+0000 7fa393fff640 1 -- 172.21.15.53:0/4031157708 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa39c039090 con 0x7fa3ac15a8a0 2023-10-21T16:02:29.721 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:29.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 msgr2=0x7fb2c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 0x7fb2c006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0035000 tx=0x7fb2d0030000 comp rx=0 tx=0).stop 2023-10-21T16:02:29.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 msgr2=0x7fa39806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 0x7fa39806d360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009a90 tx=0x7fa3a0008040 comp rx=0 tx=0).stop 2023-10-21T16:02:29.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 msgr2=0x7fa3ac15ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c2b0 tx=0x7fa39c000fe0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 msgr2=0x7fb2dc0d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.716+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800b3f0 tx=0x7fb2c800b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 shutdown_connections 2023-10-21T16:02:29.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2dc15cf80 0x7fb2dc0d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2dc15cba0 0x7fb2dc0d16a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb2c006aed0 0x7fb2c006d390 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0035000 tx=0x7fb2d0030000 comp rx=0 tx=0).stop 2023-10-21T16:02:29.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 --2- 172.21.15.53:0/3032046094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2dc0b4d10 0x7fb2dc158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 >> 172.21.15.53:0/3032046094 conn(0x7fb2dc0718a0 msgr2=0x7fb2dc0d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.5 2023-10-21T16:02:29.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 shutdown_connections 2023-10-21T16:02:29.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3ac15ee60 0x7fa3ac15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3ac15a8a0 0x7fa3ac15ace0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa39806aea0 0x7fa39806d360 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 --2- 172.21.15.53:0/4031157708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3940047c0 0x7fa3ac0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 >> 172.21.15.53:0/4031157708 conn(0x7fa3ac0660e0 msgr2=0x7fa3ac0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 shutdown_connections 2023-10-21T16:02:29.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fb2e446f640 1 -- 172.21.15.53:0/3032046094 wait complete. 2023-10-21T16:02:29.737 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 shutdown_connections 2023-10-21T16:02:29.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.717+0000 7fa3b1fe7640 1 -- 172.21.15.53:0/4031157708 wait complete. 2023-10-21T16:02:29.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.1 2023-10-21T16:02:29.740 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.739+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa9040b7780 con 0x7fa9040d1ef0 2023-10-21T16:02:29.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.740+0000 7fa8fb7fe640 1 -- 172.21.15.53:0/2871103242 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8ec0541c0 con 0x7fa9040d1ef0 2023-10-21T16:02:29.742 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:29.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.740+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbd140d2c40 con 0x7fbd140cf690 2023-10-21T16:02:29.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.741+0000 7fbd12ffd640 1 -- 172.21.15.53:0/1491656572 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbd0c079780 con 0x7fbd140cf690 2023-10-21T16:02:29.743 INFO:teuthology.orchestra.run.smithi053.stdout:55834574859 2023-10-21T16:02:29.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 msgr2=0x7fbcf806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 0x7fbcf806d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d620 tx=0x7fbd08007810 comp rx=0 tx=0).stop 2023-10-21T16:02:29.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 msgr2=0x7fa8e406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 0x7fa8e406d400 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa904159680 tx=0x7fa8f4009040 comp rx=0 tx=0).stop 2023-10-21T16:02:29.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 msgr2=0x7fa904156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c780 tx=0x7fa8ec00cc40 comp rx=0 tx=0).stop 2023-10-21T16:02:29.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 msgr2=0x7fbd140b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0090b0 tx=0x7fbd0c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 shutdown_connections 2023-10-21T16:02:29.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90415cf80 0x7fa90406f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90415cba0 0x7fa904156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa8e406af40 0x7fa8e406d400 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 --2- 172.21.15.53:0/2871103242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9040d1ef0 0x7fa904156b10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 >> 172.21.15.53:0/2871103242 conn(0x7fa9040718a0 msgr2=0x7fa9040cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.2 2023-10-21T16:02:29.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 shutdown_connections 2023-10-21T16:02:29.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.744+0000 7fa9088c6640 1 -- 172.21.15.53:0/2871103242 wait complete. 2023-10-21T16:02:29.753 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.0 2023-10-21T16:02:29.756 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 shutdown_connections 2023-10-21T16:02:29.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbcf806af40 0x7fbcf806d400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd140cf690 0x7fbd140b1330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd140cf2b0 0x7fbd140b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 --2- 172.21.15.53:0/1491656572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd140ce510 0x7fbd140b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 >> 172.21.15.53:0/1491656572 conn(0x7fbd1409bc40 msgr2=0x7fbd140b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 shutdown_connections 2023-10-21T16:02:29.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.745+0000 7fbd1b1cb640 1 -- 172.21.15.53:0/1491656572 wait complete. 2023-10-21T16:02:29.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2023-10-21T16:02:29.764 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.767+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7f58070ae0 con 0x7f7f580c8370 2023-10-21T16:02:29.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.768+0000 7f7f3f7fe640 1 -- 172.21.15.53:0/1191631241 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f4c01a090 con 0x7f7f580c8370 2023-10-21T16:02:29.770 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:29.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.3 2023-10-21T16:02:29.771 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 msgr2=0x7f7f4406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 0x7f7f4406d360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7f480011e0 tx=0x7f7f48009030 comp rx=0 tx=0).stop 2023-10-21T16:02:29.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 msgr2=0x7f7f58167af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f580639b0 tx=0x7f7f4c00fc00 comp rx=0 tx=0).stop 2023-10-21T16:02:29.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 shutdown_connections 2023-10-21T16:02:29.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f580a74c0 0x7f7f580a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7f4406aea0 0x7f7f4406d360 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f580c8370 0x7f7f58167af0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 --2- 172.21.15.53:0/1191631241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f40003b00 0x7f7f5816d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 >> 172.21.15.53:0/1191631241 conn(0x7f7f580660e0 msgr2=0x7f7f58168bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 shutdown_connections 2023-10-21T16:02:29.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.770+0000 7f7f5d13e640 1 -- 172.21.15.53:0/1191631241 wait complete. 2023-10-21T16:02:29.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.772+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f17f40d7f40 con 0x7f17f40df3f0 2023-10-21T16:02:29.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.772+0000 7f17f27fc640 1 -- 172.21.15.53:0/2639815380 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17f40d7f40 con 0x7f17f40df3f0 2023-10-21T16:02:29.777 INFO:teuthology.orchestra.run.smithi053.stdout:55834574858 2023-10-21T16:02:29.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.774+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 msgr2=0x7f17e006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.774+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 0x7f17e006d380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000f60 tx=0x7f17f8007e00 comp rx=0 tx=0).stop 2023-10-21T16:02:29.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.774+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 msgr2=0x7f17f40d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:29.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.774+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17fc069580 tx=0x7f17fc061b90 comp rx=0 tx=0).stop 2023-10-21T16:02:29.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 shutdown_connections 2023-10-21T16:02:29.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f17e006aec0 0x7f17e006d380 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17f40df3f0 0x7f17f40d4a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17f4050270 0x7f17f404ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 --2- 172.21.15.53:0/2639815380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17f4005ed0 0x7f17f40d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 >> 172.21.15.53:0/2639815380 conn(0x7f17f4054b00 msgr2=0x7f17f40380c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 shutdown_connections 2023-10-21T16:02:29.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:29.775+0000 7f1803cc9640 1 -- 172.21.15.53:0/2639815380 wait complete. 2023-10-21T16:02:29.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.4 2023-10-21T16:02:29.791 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574858 for osd.7 2023-10-21T16:02:29.795 DEBUG:teuthology.parallel:result is None 2023-10-21T16:02:29.795 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-21T16:02:29.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:02:30.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.051+0000 7feabb59e640 1 Processor -- start 2023-10-21T16:02:30.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.052+0000 7feabb59e640 1 -- start start 2023-10-21T16:02:30.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc15cc60 0x7feabc0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feabc0ce6e0 0x7feabc0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabb59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc15d040 con 0x7feabc0d16f0 2023-10-21T16:02:30.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0d0650 con 0x7feabc0ce6e0 2023-10-21T16:02:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0d0930 con 0x7feabc15cc60 2023-10-21T16:02:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feabc0ce6e0 0x7feabc0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feabc0ce6e0 0x7feabc0cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59838/0 (socket says 172.21.15.53:59838) 2023-10-21T16:02:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.053+0000 7feabad9d640 1 -- 172.21.15.53:0/1448665850 learned_addr learned my addr 172.21.15.53:0/1448665850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.054+0000 7feaba59c640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc15cc60 0x7feabc0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.054+0000 7feabad9d640 1 -- 172.21.15.53:0/1448665850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc15cc60 msgr2=0x7feabc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.054+0000 7feabad9d640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc15cc60 0x7feabc0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.054+0000 7feabad9d640 1 -- 172.21.15.53:0/1448665850 >> v1:172.21.15.124:6789/0 conn(0x7feabc0d16f0 legacy=0x7feabc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:30.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.054+0000 7feabad9d640 1 -- 172.21.15.53:0/1448665850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0d0c10 con 0x7feabc0ce6e0 2023-10-21T16:02:30.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.055+0000 7feabad9d640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc0ce6e0 0x7feabc0cf880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc84d81678fdcf29 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.055+0000 7feab959a640 1 -- 172.21.15.53:0/1448665850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402ca70 con 0x7feabc0ce6e0 2023-10-21T16:02:30.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.055+0000 7feab959a640 1 -- 172.21.15.53:0/1448665850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab402cbd0 con 0x7feabc0ce6e0 2023-10-21T16:02:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.055+0000 7feab959a640 1 -- 172.21.15.53:0/1448665850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40343f0 con 0x7feabc0ce6e0 2023-10-21T16:02:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 -- 172.21.15.53:0/1448665850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc0ce6e0 msgr2=0x7feabc0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc0ce6e0 0x7feabc0cf880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab402bed0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 -- 172.21.15.53:0/1448665850 shutdown_connections 2023-10-21T16:02:30.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc0ce6e0 0x7feabc0cf880 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 --2- 172.21.15.53:0/1448665850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc15cc60 0x7feabc0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 -- 172.21.15.53:0/1448665850 >> 172.21.15.53:0/1448665850 conn(0x7feabc09bcf0 msgr2=0x7feabc0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 -- 172.21.15.53:0/1448665850 shutdown_connections 2023-10-21T16:02:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.056+0000 7feabb59e640 1 -- 172.21.15.53:0/1448665850 wait complete. 2023-10-21T16:02:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.058+0000 7feabb59e640 1 Processor -- start 2023-10-21T16:02:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.058+0000 7feabb59e640 1 -- start start 2023-10-21T16:02:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc0ce6e0 0x7feabc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feabc0d16f0 0x7feabc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0cac10 con 0x7feabc15cc60 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0c93d0 con 0x7feabc0d16f0 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0ca000 con 0x7feabc0ce6e0 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.059+0000 7feabad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59854/0 (socket says 172.21.15.53:59854) 2023-10-21T16:02:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 -- 172.21.15.53:0/1688219479 learned_addr learned my addr 172.21.15.53:0/1688219479 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feaba59c640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc0ce6e0 0x7feabc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feab9d9b640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feabc0d16f0 0x7feabc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 -- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc0ce6e0 msgr2=0x7feabc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc0ce6e0 0x7feabc153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 -- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feabc0d16f0 msgr2=0x7feabc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feabc0d16f0 0x7feabc0c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 -- 172.21.15.53:0/1688219479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009b30 con 0x7feabc15cc60 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.060+0000 7feabad9d640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.061+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4009510 con 0x7feabc15cc60 2023-10-21T16:02:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.061+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab403e450 con 0x7feabc15cc60 2023-10-21T16:02:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.061+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab403e7e0 con 0x7feabc15cc60 2023-10-21T16:02:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.061+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc0ca2e0 con 0x7feabc15cc60 2023-10-21T16:02:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.061+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc0ca8a0 con 0x7feabc15cc60 2023-10-21T16:02:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.062+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc0cadc0 con 0x7feabc15cc60 2023-10-21T16:02:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.063+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feab403e940 con 0x7feabc15cc60 2023-10-21T16:02:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.064+0000 7feab2ffd640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 0x7fea9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.064+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feab40ac690 con 0x7feabc15cc60 2023-10-21T16:02:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.065+0000 7feaba59c640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 0x7fea9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.065+0000 7feaba59c640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 0x7fea9c06d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feabc1f5800 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.068+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feabc0cadc0 con 0x7feabc15cc60 2023-10-21T16:02:30.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.249+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feabc0d11d0 con 0x7fea9c06ae70 2023-10-21T16:02:30.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.276+0000 7feab2ffd640 1 -- 172.21.15.53:0/1688219479 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345114 (secure 0 0 0) 0x7feaac00b1a0 con 0x7fea9c06ae70 2023-10-21T16:02:30.280 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:30.286 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:02:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.284+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 msgr2=0x7fea9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.284+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 0x7fea9c06d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feabc1f5800 tx=0x7feaac008040 comp rx=0 tx=0).stop 2023-10-21T16:02:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 msgr2=0x7feabc0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feab4009e50 tx=0x7feab4004a40 comp rx=0 tx=0).stop 2023-10-21T16:02:30.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 shutdown_connections 2023-10-21T16:02:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fea9c06ae70 0x7fea9c06d330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feabc15cc60 0x7feabc0b3bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feabc0d16f0 0x7feabc0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 --2- 172.21.15.53:0/1688219479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feabc0ce6e0 0x7feabc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 >> 172.21.15.53:0/1688219479 conn(0x7feabc09bcf0 msgr2=0x7feabc0b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 shutdown_connections 2023-10-21T16:02:30.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.285+0000 7feabb59e640 1 -- 172.21.15.53:0/1688219479 wait complete. 2023-10-21T16:02:30.307 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-21T16:02:29.028013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":213,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3481600,"data_stored":2082142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005411"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928748+0000","last_change":"2023-10-21T16:01:55.158448+0000","last_active":"2023-10-21T16:02:00.928748+0000","last_peered":"2023-10-21T16:02:00.928748+0000","last_clean":"2023-10-21T16:02:00.928748+0000","last_became_active":"2023-10-21T16:01:55.158229+0000","last_became_peered":"2023-10-21T16:01:55.158229+0000","last_unstale":"2023-10-21T16:02:00.928748+0000","last_undegraded":"2023-10-21T16:02:00.928748+0000","last_fullsized":"2023-10-21T16:02:00.928748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947239+0000","last_change":"2023-10-21T16:01:55.937779+0000","last_active":"2023-10-21T16:02:00.947239+0000","last_peered":"2023-10-21T16:02:00.947239+0000","last_clean":"2023-10-21T16:02:00.947239+0000","last_became_active":"2023-10-21T16:01:55.937491+0000","last_became_peered":"2023-10-21T16:01:55.937491+0000","last_unstale":"2023-10-21T16:02:00.947239+0000","last_undegraded":"2023-10-21T16:02:00.947239+0000","last_fullsized":"2023-10-21T16:02:00.947239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895601+0000","last_change":"2023-10-21T16:02:00.754927+0000","last_active":"2023-10-21T16:02:02.895601+0000","last_peered":"2023-10-21T16:02:02.895601+0000","last_clean":"2023-10-21T16:02:02.895601+0000","last_became_active":"2023-10-21T16:02:00.753883+0000","last_became_peered":"2023-10-21T16:02:00.753883+0000","last_unstale":"2023-10-21T16:02:02.895601+0000","last_undegraded":"2023-10-21T16:02:02.895601+0000","last_fullsized":"2023-10-21T16:02:02.895601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:45.898604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947499+0000","last_change":"2023-10-21T16:01:55.153722+0000","last_active":"2023-10-21T16:02:00.947499+0000","last_peered":"2023-10-21T16:02:00.947499+0000","last_clean":"2023-10-21T16:02:00.947499+0000","last_became_active":"2023-10-21T16:01:55.153434+0000","last_became_peered":"2023-10-21T16:01:55.153434+0000","last_unstale":"2023-10-21T16:02:00.947499+0000","last_undegraded":"2023-10-21T16:02:00.947499+0000","last_fullsized":"2023-10-21T16:02:00.947499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928468+0000","last_change":"2023-10-21T16:01:55.954746+0000","last_active":"2023-10-21T16:02:00.928468+0000","last_peered":"2023-10-21T16:02:00.928468+0000","last_clean":"2023-10-21T16:02:00.928468+0000","last_became_active":"2023-10-21T16:01:55.953670+0000","last_became_peered":"2023-10-21T16:01:55.953670+0000","last_unstale":"2023-10-21T16:02:00.928468+0000","last_undegraded":"2023-10-21T16:02:00.928468+0000","last_fullsized":"2023-10-21T16:02:00.928468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918749+0000","last_change":"2023-10-21T16:02:00.185538+0000","last_active":"2023-10-21T16:02:01.918749+0000","last_peered":"2023-10-21T16:02:01.918749+0000","last_clean":"2023-10-21T16:02:01.918749+0000","last_became_active":"2023-10-21T16:02:00.185127+0000","last_became_peered":"2023-10-21T16:02:00.185127+0000","last_unstale":"2023-10-21T16:02:01.918749+0000","last_undegraded":"2023-10-21T16:02:01.918749+0000","last_fullsized":"2023-10-21T16:02:01.918749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415743+0000","last_change":"2023-10-21T16:01:55.169060+0000","last_active":"2023-10-21T16:02:01.415743+0000","last_peered":"2023-10-21T16:02:01.415743+0000","last_clean":"2023-10-21T16:02:01.415743+0000","last_became_active":"2023-10-21T16:01:55.168772+0000","last_became_peered":"2023-10-21T16:01:55.168772+0000","last_unstale":"2023-10-21T16:02:01.415743+0000","last_undegraded":"2023-10-21T16:02:01.415743+0000","last_fullsized":"2023-10-21T16:02:01.415743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004837+0000","last_change":"2023-10-21T16:01:55.950057+0000","last_active":"2023-10-21T16:02:01.004837+0000","last_peered":"2023-10-21T16:02:01.004837+0000","last_clean":"2023-10-21T16:02:01.004837+0000","last_became_active":"2023-10-21T16:01:55.949780+0000","last_became_peered":"2023-10-21T16:01:55.949780+0000","last_unstale":"2023-10-21T16:02:01.004837+0000","last_undegraded":"2023-10-21T16:02:01.004837+0000","last_fullsized":"2023-10-21T16:02:01.004837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921013+0000","last_change":"2023-10-21T16:02:00.184728+0000","last_active":"2023-10-21T16:02:01.921013+0000","last_peered":"2023-10-21T16:02:01.921013+0000","last_clean":"2023-10-21T16:02:01.921013+0000","last_became_active":"2023-10-21T16:02:00.183993+0000","last_became_peered":"2023-10-21T16:02:00.183993+0000","last_unstale":"2023-10-21T16:02:01.921013+0000","last_undegraded":"2023-10-21T16:02:01.921013+0000","last_fullsized":"2023-10-21T16:02:01.921013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415376+0000","last_change":"2023-10-21T16:01:55.171185+0000","last_active":"2023-10-21T16:02:01.415376+0000","last_peered":"2023-10-21T16:02:01.415376+0000","last_clean":"2023-10-21T16:02:01.415376+0000","last_became_active":"2023-10-21T16:01:55.170714+0000","last_became_peered":"2023-10-21T16:01:55.170714+0000","last_unstale":"2023-10-21T16:02:01.415376+0000","last_undegraded":"2023-10-21T16:02:01.415376+0000","last_fullsized":"2023-10-21T16:02:01.415376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004885+0000","last_change":"2023-10-21T16:01:55.957250+0000","last_active":"2023-10-21T16:02:01.004885+0000","last_peered":"2023-10-21T16:02:01.004885+0000","last_clean":"2023-10-21T16:02:01.004885+0000","last_became_active":"2023-10-21T16:01:55.956926+0000","last_became_peered":"2023-10-21T16:01:55.956926+0000","last_unstale":"2023-10-21T16:02:01.004885+0000","last_undegraded":"2023-10-21T16:02:01.004885+0000","last_fullsized":"2023-10-21T16:02:01.004885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.920949+0000","last_change":"2023-10-21T16:02:00.190590+0000","last_active":"2023-10-21T16:02:01.920949+0000","last_peered":"2023-10-21T16:02:01.920949+0000","last_clean":"2023-10-21T16:02:01.920949+0000","last_became_active":"2023-10-21T16:02:00.190131+0000","last_became_peered":"2023-10-21T16:02:00.190131+0000","last_unstale":"2023-10-21T16:02:01.920949+0000","last_undegraded":"2023-10-21T16:02:01.920949+0000","last_fullsized":"2023-10-21T16:02:01.920949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413796+0000","last_change":"2023-10-21T16:01:55.948227+0000","last_active":"2023-10-21T16:02:01.413796+0000","last_peered":"2023-10-21T16:02:01.413796+0000","last_clean":"2023-10-21T16:02:01.413796+0000","last_became_active":"2023-10-21T16:01:55.947879+0000","last_became_peered":"2023-10-21T16:01:55.947879+0000","last_unstale":"2023-10-21T16:02:01.413796+0000","last_undegraded":"2023-10-21T16:02:01.413796+0000","last_fullsized":"2023-10-21T16:02:01.413796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415138+0000","last_change":"2023-10-21T16:01:55.161941+0000","last_active":"2023-10-21T16:02:01.415138+0000","last_peered":"2023-10-21T16:02:01.415138+0000","last_clean":"2023-10-21T16:02:01.415138+0000","last_became_active":"2023-10-21T16:01:55.160804+0000","last_became_peered":"2023-10-21T16:01:55.160804+0000","last_unstale":"2023-10-21T16:02:01.415138+0000","last_undegraded":"2023-10-21T16:02:01.415138+0000","last_fullsized":"2023-10-21T16:02:01.415138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919532+0000","last_change":"2023-10-21T16:02:00.179242+0000","last_active":"2023-10-21T16:02:01.919532+0000","last_peered":"2023-10-21T16:02:01.919532+0000","last_clean":"2023-10-21T16:02:01.919532+0000","last_became_active":"2023-10-21T16:02:00.178744+0000","last_became_peered":"2023-10-21T16:02:00.178744+0000","last_unstale":"2023-10-21T16:02:01.919532+0000","last_undegraded":"2023-10-21T16:02:01.919532+0000","last_fullsized":"2023-10-21T16:02:01.919532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:45.186023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946147+0000","last_change":"2023-10-21T16:01:55.943291+0000","last_active":"2023-10-21T16:02:00.946147+0000","last_peered":"2023-10-21T16:02:00.946147+0000","last_clean":"2023-10-21T16:02:00.946147+0000","last_became_active":"2023-10-21T16:01:55.942937+0000","last_became_peered":"2023-10-21T16:01:55.942937+0000","last_unstale":"2023-10-21T16:02:00.946147+0000","last_undegraded":"2023-10-21T16:02:00.946147+0000","last_fullsized":"2023-10-21T16:02:00.946147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911689+0000","last_change":"2023-10-21T16:01:55.147810+0000","last_active":"2023-10-21T16:02:00.911689+0000","last_peered":"2023-10-21T16:02:00.911689+0000","last_clean":"2023-10-21T16:02:00.911689+0000","last_became_active":"2023-10-21T16:01:55.147532+0000","last_became_peered":"2023-10-21T16:01:55.147532+0000","last_unstale":"2023-10-21T16:02:00.911689+0000","last_undegraded":"2023-10-21T16:02:00.911689+0000","last_fullsized":"2023-10-21T16:02:00.911689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921974+0000","last_change":"2023-10-21T16:02:00.192496+0000","last_active":"2023-10-21T16:02:01.921974+0000","last_peered":"2023-10-21T16:02:01.921974+0000","last_clean":"2023-10-21T16:02:01.921974+0000","last_became_active":"2023-10-21T16:02:00.192090+0000","last_became_peered":"2023-10-21T16:02:00.192090+0000","last_unstale":"2023-10-21T16:02:01.921974+0000","last_undegraded":"2023-10-21T16:02:01.921974+0000","last_fullsized":"2023-10-21T16:02:01.921974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946330+0000","last_change":"2023-10-21T16:01:55.936744+0000","last_active":"2023-10-21T16:02:00.946330+0000","last_peered":"2023-10-21T16:02:00.946330+0000","last_clean":"2023-10-21T16:02:00.946330+0000","last_became_active":"2023-10-21T16:01:55.936451+0000","last_became_peered":"2023-10-21T16:01:55.936451+0000","last_unstale":"2023-10-21T16:02:00.946330+0000","last_undegraded":"2023-10-21T16:02:00.946330+0000","last_fullsized":"2023-10-21T16:02:00.946330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910754+0000","last_change":"2023-10-21T16:01:55.159913+0000","last_active":"2023-10-21T16:02:00.910754+0000","last_peered":"2023-10-21T16:02:00.910754+0000","last_clean":"2023-10-21T16:02:00.910754+0000","last_became_active":"2023-10-21T16:01:55.159480+0000","last_became_peered":"2023-10-21T16:01:55.159480+0000","last_unstale":"2023-10-21T16:02:00.910754+0000","last_undegraded":"2023-10-21T16:02:00.910754+0000","last_fullsized":"2023-10-21T16:02:00.910754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:41.390056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895508+0000","last_change":"2023-10-21T16:02:00.748804+0000","last_active":"2023-10-21T16:02:02.895508+0000","last_peered":"2023-10-21T16:02:02.895508+0000","last_clean":"2023-10-21T16:02:02.895508+0000","last_became_active":"2023-10-21T16:02:00.746585+0000","last_became_peered":"2023-10-21T16:02:00.746585+0000","last_unstale":"2023-10-21T16:02:02.895508+0000","last_undegraded":"2023-10-21T16:02:02.895508+0000","last_fullsized":"2023-10-21T16:02:02.895508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913021+0000","last_change":"2023-10-21T16:01:55.936909+0000","last_active":"2023-10-21T16:02:00.913021+0000","last_peered":"2023-10-21T16:02:00.913021+0000","last_clean":"2023-10-21T16:02:00.913021+0000","last_became_active":"2023-10-21T16:01:55.930874+0000","last_became_peered":"2023-10-21T16:01:55.930874+0000","last_unstale":"2023-10-21T16:02:00.913021+0000","last_undegraded":"2023-10-21T16:02:00.913021+0000","last_fullsized":"2023-10-21T16:02:00.913021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:19.306040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414418+0000","last_change":"2023-10-21T16:01:55.161120+0000","last_active":"2023-10-21T16:02:01.414418+0000","last_peered":"2023-10-21T16:02:01.414418+0000","last_clean":"2023-10-21T16:02:01.414418+0000","last_became_active":"2023-10-21T16:01:55.160853+0000","last_became_peered":"2023-10-21T16:01:55.160853+0000","last_unstale":"2023-10-21T16:02:01.414418+0000","last_undegraded":"2023-10-21T16:02:01.414418+0000","last_fullsized":"2023-10-21T16:02:01.414418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919323+0000","last_change":"2023-10-21T16:02:00.192783+0000","last_active":"2023-10-21T16:02:01.919323+0000","last_peered":"2023-10-21T16:02:01.919323+0000","last_clean":"2023-10-21T16:02:01.919323+0000","last_became_active":"2023-10-21T16:02:00.192399+0000","last_became_peered":"2023-10-21T16:02:00.192399+0000","last_unstale":"2023-10-21T16:02:01.919323+0000","last_undegraded":"2023-10-21T16:02:01.919323+0000","last_fullsized":"2023-10-21T16:02:01.919323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928409+0000","last_change":"2023-10-21T16:01:55.942303+0000","last_active":"2023-10-21T16:02:00.928409+0000","last_peered":"2023-10-21T16:02:00.928409+0000","last_clean":"2023-10-21T16:02:00.928409+0000","last_became_active":"2023-10-21T16:01:55.941848+0000","last_became_peered":"2023-10-21T16:01:55.941848+0000","last_unstale":"2023-10-21T16:02:00.928409+0000","last_undegraded":"2023-10-21T16:02:00.928409+0000","last_fullsized":"2023-10-21T16:02:00.928409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417420+0000","last_change":"2023-10-21T16:01:55.164579+0000","last_active":"2023-10-21T16:02:01.417420+0000","last_peered":"2023-10-21T16:02:01.417420+0000","last_clean":"2023-10-21T16:02:01.417420+0000","last_became_active":"2023-10-21T16:01:55.162946+0000","last_became_peered":"2023-10-21T16:01:55.162946+0000","last_unstale":"2023-10-21T16:02:01.417420+0000","last_undegraded":"2023-10-21T16:02:01.417420+0000","last_fullsized":"2023-10-21T16:02:01.417420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919836+0000","last_change":"2023-10-21T16:02:00.748601+0000","last_active":"2023-10-21T16:02:01.919836+0000","last_peered":"2023-10-21T16:02:01.919836+0000","last_clean":"2023-10-21T16:02:01.919836+0000","last_became_active":"2023-10-21T16:02:00.747592+0000","last_became_peered":"2023-10-21T16:02:00.747592+0000","last_unstale":"2023-10-21T16:02:01.919836+0000","last_undegraded":"2023-10-21T16:02:01.919836+0000","last_fullsized":"2023-10-21T16:02:01.919836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947881+0000","last_change":"2023-10-21T16:01:55.941119+0000","last_active":"2023-10-21T16:02:00.947881+0000","last_peered":"2023-10-21T16:02:00.947881+0000","last_clean":"2023-10-21T16:02:00.947881+0000","last_became_active":"2023-10-21T16:01:55.940145+0000","last_became_peered":"2023-10-21T16:01:55.940145+0000","last_unstale":"2023-10-21T16:02:00.947881+0000","last_undegraded":"2023-10-21T16:02:00.947881+0000","last_fullsized":"2023-10-21T16:02:00.947881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:13.347675+0000","last_change":"2023-10-21T16:01:55.165800+0000","last_active":"2023-10-21T16:02:13.347675+0000","last_peered":"2023-10-21T16:02:13.347675+0000","last_clean":"2023-10-21T16:02:13.347675+0000","last_became_active":"2023-10-21T16:01:55.165500+0000","last_became_peered":"2023-10-21T16:01:55.165500+0000","last_unstale":"2023-10-21T16:02:13.347675+0000","last_undegraded":"2023-10-21T16:02:13.347675+0000","last_fullsized":"2023-10-21T16:02:13.347675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894525+0000","last_change":"2023-10-21T16:02:00.742519+0000","last_active":"2023-10-21T16:02:02.894525+0000","last_peered":"2023-10-21T16:02:02.894525+0000","last_clean":"2023-10-21T16:02:02.894525+0000","last_became_active":"2023-10-21T16:02:00.740937+0000","last_became_peered":"2023-10-21T16:02:00.740937+0000","last_unstale":"2023-10-21T16:02:02.894525+0000","last_undegraded":"2023-10-21T16:02:02.894525+0000","last_fullsized":"2023-10-21T16:02:02.894525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:22.482979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913147+0000","last_change":"2023-10-21T16:01:55.939913+0000","last_active":"2023-10-21T16:02:00.913147+0000","last_peered":"2023-10-21T16:02:00.913147+0000","last_clean":"2023-10-21T16:02:00.913147+0000","last_became_active":"2023-10-21T16:01:55.939419+0000","last_became_peered":"2023-10-21T16:01:55.939419+0000","last_unstale":"2023-10-21T16:02:00.913147+0000","last_undegraded":"2023-10-21T16:02:00.913147+0000","last_fullsized":"2023-10-21T16:02:00.913147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415967+0000","last_change":"2023-10-21T16:01:55.163977+0000","last_active":"2023-10-21T16:02:01.415967+0000","last_peered":"2023-10-21T16:02:01.415967+0000","last_clean":"2023-10-21T16:02:01.415967+0000","last_became_active":"2023-10-21T16:01:55.163739+0000","last_became_peered":"2023-10-21T16:01:55.163739+0000","last_unstale":"2023-10-21T16:02:01.415967+0000","last_undegraded":"2023-10-21T16:02:01.415967+0000","last_fullsized":"2023-10-21T16:02:01.415967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:17.767707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007711+0000","last_change":"2023-10-21T16:02:00.751077+0000","last_active":"2023-10-21T16:02:02.007711+0000","last_peered":"2023-10-21T16:02:02.007711+0000","last_clean":"2023-10-21T16:02:02.007711+0000","last_became_active":"2023-10-21T16:02:00.749997+0000","last_became_peered":"2023-10-21T16:02:00.749997+0000","last_unstale":"2023-10-21T16:02:02.007711+0000","last_undegraded":"2023-10-21T16:02:02.007711+0000","last_fullsized":"2023-10-21T16:02:02.007711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005010+0000","last_change":"2023-10-21T16:01:55.951660+0000","last_active":"2023-10-21T16:02:01.005010+0000","last_peered":"2023-10-21T16:02:01.005010+0000","last_clean":"2023-10-21T16:02:01.005010+0000","last_became_active":"2023-10-21T16:01:55.951279+0000","last_became_peered":"2023-10-21T16:01:55.951279+0000","last_unstale":"2023-10-21T16:02:01.005010+0000","last_undegraded":"2023-10-21T16:02:01.005010+0000","last_fullsized":"2023-10-21T16:02:01.005010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913993+0000","last_change":"2023-10-21T16:01:55.149562+0000","last_active":"2023-10-21T16:02:00.913993+0000","last_peered":"2023-10-21T16:02:00.913993+0000","last_clean":"2023-10-21T16:02:00.913993+0000","last_became_active":"2023-10-21T16:01:55.149274+0000","last_became_peered":"2023-10-21T16:01:55.149274+0000","last_unstale":"2023-10-21T16:02:00.913993+0000","last_undegraded":"2023-10-21T16:02:00.913993+0000","last_fullsized":"2023-10-21T16:02:00.913993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.892788+0000","last_change":"2023-10-21T16:02:00.189379+0000","last_active":"2023-10-21T16:02:02.892788+0000","last_peered":"2023-10-21T16:02:02.892788+0000","last_clean":"2023-10-21T16:02:02.892788+0000","last_became_active":"2023-10-21T16:02:00.188693+0000","last_became_peered":"2023-10-21T16:02:00.188693+0000","last_unstale":"2023-10-21T16:02:02.892788+0000","last_undegraded":"2023-10-21T16:02:02.892788+0000","last_fullsized":"2023-10-21T16:02:02.892788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913564+0000","last_change":"2023-10-21T16:01:55.941518+0000","last_active":"2023-10-21T16:02:00.913564+0000","last_peered":"2023-10-21T16:02:00.913564+0000","last_clean":"2023-10-21T16:02:00.913564+0000","last_became_active":"2023-10-21T16:01:55.938726+0000","last_became_peered":"2023-10-21T16:01:55.938726+0000","last_unstale":"2023-10-21T16:02:00.913564+0000","last_undegraded":"2023-10-21T16:02:00.913564+0000","last_fullsized":"2023-10-21T16:02:00.913564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:39.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004473+0000","last_change":"2023-10-21T16:01:55.165664+0000","last_active":"2023-10-21T16:02:01.004473+0000","last_peered":"2023-10-21T16:02:01.004473+0000","last_clean":"2023-10-21T16:02:01.004473+0000","last_became_active":"2023-10-21T16:01:55.165070+0000","last_became_peered":"2023-10-21T16:01:55.165070+0000","last_unstale":"2023-10-21T16:02:01.004473+0000","last_undegraded":"2023-10-21T16:02:01.004473+0000","last_fullsized":"2023-10-21T16:02:01.004473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:02.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893925+0000","last_change":"2023-10-21T16:02:00.186762+0000","last_active":"2023-10-21T16:02:02.893925+0000","last_peered":"2023-10-21T16:02:02.893925+0000","last_clean":"2023-10-21T16:02:02.893925+0000","last_became_active":"2023-10-21T16:02:00.185437+0000","last_became_peered":"2023-10-21T16:02:00.185437+0000","last_unstale":"2023-10-21T16:02:02.893925+0000","last_undegraded":"2023-10-21T16:02:02.893925+0000","last_fullsized":"2023-10-21T16:02:02.893925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911065+0000","last_change":"2023-10-21T16:01:55.935243+0000","last_active":"2023-10-21T16:02:00.911065+0000","last_peered":"2023-10-21T16:02:00.911065+0000","last_clean":"2023-10-21T16:02:00.911065+0000","last_became_active":"2023-10-21T16:01:55.934353+0000","last_became_peered":"2023-10-21T16:01:55.934353+0000","last_unstale":"2023-10-21T16:02:00.911065+0000","last_undegraded":"2023-10-21T16:02:00.911065+0000","last_fullsized":"2023-10-21T16:02:00.911065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914987+0000","last_change":"2023-10-21T16:01:55.147043+0000","last_active":"2023-10-21T16:02:00.914987+0000","last_peered":"2023-10-21T16:02:00.914987+0000","last_clean":"2023-10-21T16:02:00.914987+0000","last_became_active":"2023-10-21T16:01:55.146674+0000","last_became_peered":"2023-10-21T16:01:55.146674+0000","last_unstale":"2023-10-21T16:02:00.914987+0000","last_undegraded":"2023-10-21T16:02:00.914987+0000","last_fullsized":"2023-10-21T16:02:00.914987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:18.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895819+0000","last_change":"2023-10-21T16:02:00.191162+0000","last_active":"2023-10-21T16:02:02.895819+0000","last_peered":"2023-10-21T16:02:02.895819+0000","last_clean":"2023-10-21T16:02:02.895819+0000","last_became_active":"2023-10-21T16:02:00.186599+0000","last_became_peered":"2023-10-21T16:02:00.186599+0000","last_unstale":"2023-10-21T16:02:02.895819+0000","last_undegraded":"2023-10-21T16:02:02.895819+0000","last_fullsized":"2023-10-21T16:02:02.895819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.926918+0000","last_change":"2023-10-21T16:01:55.948611+0000","last_active":"2023-10-21T16:02:00.926918+0000","last_peered":"2023-10-21T16:02:00.926918+0000","last_clean":"2023-10-21T16:02:00.926918+0000","last_became_active":"2023-10-21T16:01:55.948172+0000","last_became_peered":"2023-10-21T16:01:55.948172+0000","last_unstale":"2023-10-21T16:02:00.926918+0000","last_undegraded":"2023-10-21T16:02:00.926918+0000","last_fullsized":"2023-10-21T16:02:00.926918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414533+0000","last_change":"2023-10-21T16:01:55.161709+0000","last_active":"2023-10-21T16:02:01.414533+0000","last_peered":"2023-10-21T16:02:01.414533+0000","last_clean":"2023-10-21T16:02:01.414533+0000","last_became_active":"2023-10-21T16:01:55.160426+0000","last_became_peered":"2023-10-21T16:01:55.160426+0000","last_unstale":"2023-10-21T16:02:01.414533+0000","last_undegraded":"2023-10-21T16:02:01.414533+0000","last_fullsized":"2023-10-21T16:02:01.414533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007971+0000","last_change":"2023-10-21T16:02:00.742783+0000","last_active":"2023-10-21T16:02:02.007971+0000","last_peered":"2023-10-21T16:02:02.007971+0000","last_clean":"2023-10-21T16:02:02.007971+0000","last_became_active":"2023-10-21T16:02:00.740932+0000","last_became_peered":"2023-10-21T16:02:00.740932+0000","last_unstale":"2023-10-21T16:02:02.007971+0000","last_undegraded":"2023-10-21T16:02:02.007971+0000","last_fullsized":"2023-10-21T16:02:02.007971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946103+0000","last_change":"2023-10-21T16:01:55.955491+0000","last_active":"2023-10-21T16:02:00.946103+0000","last_peered":"2023-10-21T16:02:00.946103+0000","last_clean":"2023-10-21T16:02:00.946103+0000","last_became_active":"2023-10-21T16:01:55.955163+0000","last_became_peered":"2023-10-21T16:01:55.955163+0000","last_unstale":"2023-10-21T16:02:00.946103+0000","last_undegraded":"2023-10-21T16:02:00.946103+0000","last_fullsized":"2023-10-21T16:02:00.946103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004498+0000","last_change":"2023-10-21T16:01:55.160793+0000","last_active":"2023-10-21T16:02:01.004498+0000","last_peered":"2023-10-21T16:02:01.004498+0000","last_clean":"2023-10-21T16:02:01.004498+0000","last_became_active":"2023-10-21T16:01:55.157151+0000","last_became_peered":"2023-10-21T16:01:55.157151+0000","last_unstale":"2023-10-21T16:02:01.004498+0000","last_undegraded":"2023-10-21T16:02:01.004498+0000","last_fullsized":"2023-10-21T16:02:01.004498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:36.886516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414663+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:02:01.414663+0000","last_peered":"2023-10-21T16:02:01.414663+0000","last_clean":"2023-10-21T16:02:01.414663+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:02:01.414663+0000","last_undegraded":"2023-10-21T16:02:01.414663+0000","last_fullsized":"2023-10-21T16:02:01.414663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895666+0000","last_change":"2023-10-21T16:02:00.752861+0000","last_active":"2023-10-21T16:02:02.895666+0000","last_peered":"2023-10-21T16:02:02.895666+0000","last_clean":"2023-10-21T16:02:02.895666+0000","last_became_active":"2023-10-21T16:02:00.748880+0000","last_became_peered":"2023-10-21T16:02:00.748880+0000","last_unstale":"2023-10-21T16:02:02.895666+0000","last_undegraded":"2023-10-21T16:02:02.895666+0000","last_fullsized":"2023-10-21T16:02:02.895666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415527+0000","last_change":"2023-10-21T16:01:55.954245+0000","last_active":"2023-10-21T16:02:01.415527+0000","last_peered":"2023-10-21T16:02:01.415527+0000","last_clean":"2023-10-21T16:02:01.415527+0000","last_became_active":"2023-10-21T16:01:55.953197+0000","last_became_peered":"2023-10-21T16:01:55.953197+0000","last_unstale":"2023-10-21T16:02:01.415527+0000","last_undegraded":"2023-10-21T16:02:01.415527+0000","last_fullsized":"2023-10-21T16:02:01.415527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915622+0000","last_change":"2023-10-21T16:01:55.168127+0000","last_active":"2023-10-21T16:02:00.915622+0000","last_peered":"2023-10-21T16:02:00.915622+0000","last_clean":"2023-10-21T16:02:00.915622+0000","last_became_active":"2023-10-21T16:01:55.167766+0000","last_became_peered":"2023-10-21T16:01:55.167766+0000","last_unstale":"2023-10-21T16:02:00.915622+0000","last_undegraded":"2023-10-21T16:02:00.915622+0000","last_fullsized":"2023-10-21T16:02:00.915622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:28:10.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893224+0000","last_change":"2023-10-21T16:02:00.742293+0000","last_active":"2023-10-21T16:02:02.893224+0000","last_peered":"2023-10-21T16:02:02.893224+0000","last_clean":"2023-10-21T16:02:02.893224+0000","last_became_active":"2023-10-21T16:02:00.741230+0000","last_became_peered":"2023-10-21T16:02:00.741230+0000","last_unstale":"2023-10-21T16:02:02.893224+0000","last_undegraded":"2023-10-21T16:02:02.893224+0000","last_fullsized":"2023-10-21T16:02:02.893224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005008+0000","last_change":"2023-10-21T16:01:55.948576+0000","last_active":"2023-10-21T16:02:01.005008+0000","last_peered":"2023-10-21T16:02:01.005008+0000","last_clean":"2023-10-21T16:02:01.005008+0000","last_became_active":"2023-10-21T16:01:55.947390+0000","last_became_peered":"2023-10-21T16:01:55.947390+0000","last_unstale":"2023-10-21T16:02:01.005008+0000","last_undegraded":"2023-10-21T16:02:01.005008+0000","last_fullsized":"2023-10-21T16:02:01.005008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914480+0000","last_change":"2023-10-21T16:01:55.151627+0000","last_active":"2023-10-21T16:02:00.914480+0000","last_peered":"2023-10-21T16:02:00.914480+0000","last_clean":"2023-10-21T16:02:00.914480+0000","last_became_active":"2023-10-21T16:01:55.151283+0000","last_became_peered":"2023-10-21T16:01:55.151283+0000","last_unstale":"2023-10-21T16:02:00.914480+0000","last_undegraded":"2023-10-21T16:02:00.914480+0000","last_fullsized":"2023-10-21T16:02:00.914480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007984+0000","last_change":"2023-10-21T16:02:00.183073+0000","last_active":"2023-10-21T16:02:02.007984+0000","last_peered":"2023-10-21T16:02:02.007984+0000","last_clean":"2023-10-21T16:02:02.007984+0000","last_became_active":"2023-10-21T16:02:00.182604+0000","last_became_peered":"2023-10-21T16:02:00.182604+0000","last_unstale":"2023-10-21T16:02:02.007984+0000","last_undegraded":"2023-10-21T16:02:02.007984+0000","last_fullsized":"2023-10-21T16:02:02.007984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416904+0000","last_change":"2023-10-21T16:01:55.951113+0000","last_active":"2023-10-21T16:02:01.416904+0000","last_peered":"2023-10-21T16:02:01.416904+0000","last_clean":"2023-10-21T16:02:01.416904+0000","last_became_active":"2023-10-21T16:01:55.950792+0000","last_became_peered":"2023-10-21T16:01:55.950792+0000","last_unstale":"2023-10-21T16:02:01.416904+0000","last_undegraded":"2023-10-21T16:02:01.416904+0000","last_fullsized":"2023-10-21T16:02:01.416904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915704+0000","last_change":"2023-10-21T16:01:55.149597+0000","last_active":"2023-10-21T16:02:00.915704+0000","last_peered":"2023-10-21T16:02:00.915704+0000","last_clean":"2023-10-21T16:02:00.915704+0000","last_became_active":"2023-10-21T16:01:55.149258+0000","last_became_peered":"2023-10-21T16:01:55.149258+0000","last_unstale":"2023-10-21T16:02:00.915704+0000","last_undegraded":"2023-10-21T16:02:00.915704+0000","last_fullsized":"2023-10-21T16:02:00.915704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.914975+0000","last_change":"2023-10-21T16:02:00.750068+0000","last_active":"2023-10-21T16:02:01.914975+0000","last_peered":"2023-10-21T16:02:01.914975+0000","last_clean":"2023-10-21T16:02:01.914975+0000","last_became_active":"2023-10-21T16:02:00.749055+0000","last_became_peered":"2023-10-21T16:02:00.749055+0000","last_unstale":"2023-10-21T16:02:01.914975+0000","last_undegraded":"2023-10-21T16:02:01.914975+0000","last_fullsized":"2023-10-21T16:02:01.914975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416138+0000","last_change":"2023-10-21T16:01:55.953586+0000","last_active":"2023-10-21T16:02:01.416138+0000","last_peered":"2023-10-21T16:02:01.416138+0000","last_clean":"2023-10-21T16:02:01.416138+0000","last_became_active":"2023-10-21T16:01:55.952550+0000","last_became_peered":"2023-10-21T16:01:55.952550+0000","last_unstale":"2023-10-21T16:02:01.416138+0000","last_undegraded":"2023-10-21T16:02:01.416138+0000","last_fullsized":"2023-10-21T16:02:01.416138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913456+0000","last_change":"2023-10-21T16:01:55.154502+0000","last_active":"2023-10-21T16:02:00.913456+0000","last_peered":"2023-10-21T16:02:00.913456+0000","last_clean":"2023-10-21T16:02:00.913456+0000","last_became_active":"2023-10-21T16:01:55.150688+0000","last_became_peered":"2023-10-21T16:01:55.150688+0000","last_unstale":"2023-10-21T16:02:00.913456+0000","last_undegraded":"2023-10-21T16:02:00.913456+0000","last_fullsized":"2023-10-21T16:02:00.913456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:32.709310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918774+0000","last_change":"2023-10-21T16:02:00.184524+0000","last_active":"2023-10-21T16:02:01.918774+0000","last_peered":"2023-10-21T16:02:01.918774+0000","last_clean":"2023-10-21T16:02:01.918774+0000","last_became_active":"2023-10-21T16:02:00.184081+0000","last_became_peered":"2023-10-21T16:02:00.184081+0000","last_unstale":"2023-10-21T16:02:01.918774+0000","last_undegraded":"2023-10-21T16:02:01.918774+0000","last_fullsized":"2023-10-21T16:02:01.918774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414960+0000","last_change":"2023-10-21T16:01:55.951983+0000","last_active":"2023-10-21T16:02:01.414960+0000","last_peered":"2023-10-21T16:02:01.414960+0000","last_clean":"2023-10-21T16:02:01.414960+0000","last_became_active":"2023-10-21T16:01:55.951633+0000","last_became_peered":"2023-10-21T16:01:55.951633+0000","last_unstale":"2023-10-21T16:02:01.414960+0000","last_undegraded":"2023-10-21T16:02:01.414960+0000","last_fullsized":"2023-10-21T16:02:01.414960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914399+0000","last_change":"2023-10-21T16:01:55.147150+0000","last_active":"2023-10-21T16:02:00.914399+0000","last_peered":"2023-10-21T16:02:00.914399+0000","last_clean":"2023-10-21T16:02:00.914399+0000","last_became_active":"2023-10-21T16:01:55.141230+0000","last_became_peered":"2023-10-21T16:01:55.141230+0000","last_unstale":"2023-10-21T16:02:00.914399+0000","last_undegraded":"2023-10-21T16:02:00.914399+0000","last_fullsized":"2023-10-21T16:02:00.914399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007865+0000","last_change":"2023-10-21T16:02:00.187875+0000","last_active":"2023-10-21T16:02:02.007865+0000","last_peered":"2023-10-21T16:02:02.007865+0000","last_clean":"2023-10-21T16:02:02.007865+0000","last_became_active":"2023-10-21T16:02:00.187427+0000","last_became_peered":"2023-10-21T16:02:00.187427+0000","last_unstale":"2023-10-21T16:02:02.007865+0000","last_undegraded":"2023-10-21T16:02:02.007865+0000","last_fullsized":"2023-10-21T16:02:02.007865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414077+0000","last_change":"2023-10-21T16:01:55.944005+0000","last_active":"2023-10-21T16:02:01.414077+0000","last_peered":"2023-10-21T16:02:01.414077+0000","last_clean":"2023-10-21T16:02:01.414077+0000","last_became_active":"2023-10-21T16:01:55.942887+0000","last_became_peered":"2023-10-21T16:01:55.942887+0000","last_unstale":"2023-10-21T16:02:01.414077+0000","last_undegraded":"2023-10-21T16:02:01.414077+0000","last_fullsized":"2023-10-21T16:02:01.414077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413985+0000","last_change":"2023-10-21T16:01:55.160115+0000","last_active":"2023-10-21T16:02:01.413985+0000","last_peered":"2023-10-21T16:02:01.413985+0000","last_clean":"2023-10-21T16:02:01.413985+0000","last_became_active":"2023-10-21T16:01:55.159788+0000","last_became_peered":"2023-10-21T16:01:55.159788+0000","last_unstale":"2023-10-21T16:02:01.413985+0000","last_undegraded":"2023-10-21T16:02:01.413985+0000","last_fullsized":"2023-10-21T16:02:01.413985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897390+0000","last_change":"2023-10-21T16:02:00.745714+0000","last_active":"2023-10-21T16:02:02.897390+0000","last_peered":"2023-10-21T16:02:02.897390+0000","last_clean":"2023-10-21T16:02:02.897390+0000","last_became_active":"2023-10-21T16:02:00.744779+0000","last_became_peered":"2023-10-21T16:02:00.744779+0000","last_unstale":"2023-10-21T16:02:02.897390+0000","last_undegraded":"2023-10-21T16:02:02.897390+0000","last_fullsized":"2023-10-21T16:02:02.897390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:53.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414998+0000","last_change":"2023-10-21T16:01:55.955734+0000","last_active":"2023-10-21T16:02:01.414998+0000","last_peered":"2023-10-21T16:02:01.414998+0000","last_clean":"2023-10-21T16:02:01.414998+0000","last_became_active":"2023-10-21T16:01:55.955333+0000","last_became_peered":"2023-10-21T16:01:55.955333+0000","last_unstale":"2023-10-21T16:02:01.414998+0000","last_undegraded":"2023-10-21T16:02:01.414998+0000","last_fullsized":"2023-10-21T16:02:01.414998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004475+0000","last_change":"2023-10-21T16:01:55.167450+0000","last_active":"2023-10-21T16:02:01.004475+0000","last_peered":"2023-10-21T16:02:01.004475+0000","last_clean":"2023-10-21T16:02:01.004475+0000","last_became_active":"2023-10-21T16:01:55.166857+0000","last_became_peered":"2023-10-21T16:01:55.166857+0000","last_unstale":"2023-10-21T16:02:01.004475+0000","last_undegraded":"2023-10-21T16:02:01.004475+0000","last_fullsized":"2023-10-21T16:02:01.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.896683+0000","last_change":"2023-10-21T16:02:00.742809+0000","last_active":"2023-10-21T16:02:02.896683+0000","last_peered":"2023-10-21T16:02:02.896683+0000","last_clean":"2023-10-21T16:02:02.896683+0000","last_became_active":"2023-10-21T16:02:00.741398+0000","last_became_peered":"2023-10-21T16:02:00.741398+0000","last_unstale":"2023-10-21T16:02:02.896683+0000","last_undegraded":"2023-10-21T16:02:02.896683+0000","last_fullsized":"2023-10-21T16:02:02.896683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:35:32.899715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927531+0000","last_change":"2023-10-21T16:01:55.937515+0000","last_active":"2023-10-21T16:02:00.927531+0000","last_peered":"2023-10-21T16:02:00.927531+0000","last_clean":"2023-10-21T16:02:00.927531+0000","last_became_active":"2023-10-21T16:01:55.937165+0000","last_became_peered":"2023-10-21T16:01:55.937165+0000","last_unstale":"2023-10-21T16:02:00.927531+0000","last_undegraded":"2023-10-21T16:02:00.927531+0000","last_fullsized":"2023-10-21T16:02:00.927531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928104+0000","last_change":"2023-10-21T16:01:55.151763+0000","last_active":"2023-10-21T16:02:00.928104+0000","last_peered":"2023-10-21T16:02:00.928104+0000","last_clean":"2023-10-21T16:02:00.928104+0000","last_became_active":"2023-10-21T16:01:55.151316+0000","last_became_peered":"2023-10-21T16:01:55.151316+0000","last_unstale":"2023-10-21T16:02:00.928104+0000","last_undegraded":"2023-10-21T16:02:00.928104+0000","last_fullsized":"2023-10-21T16:02:00.928104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.917604+0000","last_change":"2023-10-21T16:02:00.742437+0000","last_active":"2023-10-21T16:02:01.917604+0000","last_peered":"2023-10-21T16:02:01.917604+0000","last_clean":"2023-10-21T16:02:01.917604+0000","last_became_active":"2023-10-21T16:02:00.741212+0000","last_became_peered":"2023-10-21T16:02:00.741212+0000","last_unstale":"2023-10-21T16:02:01.917604+0000","last_undegraded":"2023-10-21T16:02:01.917604+0000","last_fullsized":"2023-10-21T16:02:01.917604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415373+0000","last_change":"2023-10-21T16:01:55.942841+0000","last_active":"2023-10-21T16:02:01.415373+0000","last_peered":"2023-10-21T16:02:01.415373+0000","last_clean":"2023-10-21T16:02:01.415373+0000","last_became_active":"2023-10-21T16:01:55.942471+0000","last_became_peered":"2023-10-21T16:01:55.942471+0000","last_unstale":"2023-10-21T16:02:01.415373+0000","last_undegraded":"2023-10-21T16:02:01.415373+0000","last_fullsized":"2023-10-21T16:02:01.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:14.233631+0000","last_change":"2023-10-21T16:01:55.156634+0000","last_active":"2023-10-21T16:02:14.233631+0000","last_peered":"2023-10-21T16:02:14.233631+0000","last_clean":"2023-10-21T16:02:14.233631+0000","last_became_active":"2023-10-21T16:01:55.156340+0000","last_became_peered":"2023-10-21T16:01:55.156340+0000","last_unstale":"2023-10-21T16:02:14.233631+0000","last_undegraded":"2023-10-21T16:02:14.233631+0000","last_fullsized":"2023-10-21T16:02:14.233631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893349+0000","last_change":"2023-10-21T16:02:00.189158+0000","last_active":"2023-10-21T16:02:02.893349+0000","last_peered":"2023-10-21T16:02:02.893349+0000","last_clean":"2023-10-21T16:02:02.893349+0000","last_became_active":"2023-10-21T16:02:00.188097+0000","last_became_peered":"2023-10-21T16:02:00.188097+0000","last_unstale":"2023-10-21T16:02:02.893349+0000","last_undegraded":"2023-10-21T16:02:02.893349+0000","last_fullsized":"2023-10-21T16:02:02.893349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914268+0000","last_change":"2023-10-21T16:01:55.944799+0000","last_active":"2023-10-21T16:02:00.914268+0000","last_peered":"2023-10-21T16:02:00.914268+0000","last_clean":"2023-10-21T16:02:00.914268+0000","last_became_active":"2023-10-21T16:01:55.944472+0000","last_became_peered":"2023-10-21T16:01:55.944472+0000","last_unstale":"2023-10-21T16:02:00.914268+0000","last_undegraded":"2023-10-21T16:02:00.914268+0000","last_fullsized":"2023-10-21T16:02:00.914268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:58.256813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914405+0000","last_change":"2023-10-21T16:01:55.162979+0000","last_active":"2023-10-21T16:02:00.914405+0000","last_peered":"2023-10-21T16:02:00.914405+0000","last_clean":"2023-10-21T16:02:00.914405+0000","last_became_active":"2023-10-21T16:01:55.162557+0000","last_became_peered":"2023-10-21T16:01:55.162557+0000","last_unstale":"2023-10-21T16:02:00.914405+0000","last_undegraded":"2023-10-21T16:02:00.914405+0000","last_fullsized":"2023-10-21T16:02:00.914405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921560+0000","last_change":"2023-10-21T16:02:00.188187+0000","last_active":"2023-10-21T16:02:01.921560+0000","last_peered":"2023-10-21T16:02:01.921560+0000","last_clean":"2023-10-21T16:02:01.921560+0000","last_became_active":"2023-10-21T16:02:00.187688+0000","last_became_peered":"2023-10-21T16:02:00.187688+0000","last_unstale":"2023-10-21T16:02:01.921560+0000","last_undegraded":"2023-10-21T16:02:01.921560+0000","last_fullsized":"2023-10-21T16:02:01.921560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928702+0000","last_change":"2023-10-21T16:01:55.954216+0000","last_active":"2023-10-21T16:02:00.928702+0000","last_peered":"2023-10-21T16:02:00.928702+0000","last_clean":"2023-10-21T16:02:00.928702+0000","last_became_active":"2023-10-21T16:01:55.953211+0000","last_became_peered":"2023-10-21T16:01:55.953211+0000","last_unstale":"2023-10-21T16:02:00.928702+0000","last_undegraded":"2023-10-21T16:02:00.928702+0000","last_fullsized":"2023-10-21T16:02:00.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005397+0000","last_change":"2023-10-21T16:01:55.169893+0000","last_active":"2023-10-21T16:02:01.005397+0000","last_peered":"2023-10-21T16:02:01.005397+0000","last_clean":"2023-10-21T16:02:01.005397+0000","last_became_active":"2023-10-21T16:01:55.169587+0000","last_became_peered":"2023-10-21T16:01:55.169587+0000","last_unstale":"2023-10-21T16:02:01.005397+0000","last_undegraded":"2023-10-21T16:02:01.005397+0000","last_fullsized":"2023-10-21T16:02:01.005397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894928+0000","last_change":"2023-10-21T16:02:00.185249+0000","last_active":"2023-10-21T16:02:02.894928+0000","last_peered":"2023-10-21T16:02:02.894928+0000","last_clean":"2023-10-21T16:02:02.894928+0000","last_became_active":"2023-10-21T16:02:00.184448+0000","last_became_peered":"2023-10-21T16:02:00.184448+0000","last_unstale":"2023-10-21T16:02:02.894928+0000","last_undegraded":"2023-10-21T16:02:02.894928+0000","last_fullsized":"2023-10-21T16:02:02.894928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:18:23.475051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005291+0000","last_change":"2023-10-21T16:01:55.954077+0000","last_active":"2023-10-21T16:02:01.005291+0000","last_peered":"2023-10-21T16:02:01.005291+0000","last_clean":"2023-10-21T16:02:01.005291+0000","last_became_active":"2023-10-21T16:01:55.951152+0000","last_became_peered":"2023-10-21T16:01:55.951152+0000","last_unstale":"2023-10-21T16:02:01.005291+0000","last_undegraded":"2023-10-21T16:02:01.005291+0000","last_fullsized":"2023-10-21T16:02:01.005291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417259+0000","last_change":"2023-10-21T16:01:55.171136+0000","last_active":"2023-10-21T16:02:01.417259+0000","last_peered":"2023-10-21T16:02:01.417259+0000","last_clean":"2023-10-21T16:02:01.417259+0000","last_became_active":"2023-10-21T16:01:55.170850+0000","last_became_peered":"2023-10-21T16:01:55.170850+0000","last_unstale":"2023-10-21T16:02:01.417259+0000","last_undegraded":"2023-10-21T16:02:01.417259+0000","last_fullsized":"2023-10-21T16:02:01.417259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895695+0000","last_change":"2023-10-21T16:02:00.753599+0000","last_active":"2023-10-21T16:02:02.895695+0000","last_peered":"2023-10-21T16:02:02.895695+0000","last_clean":"2023-10-21T16:02:02.895695+0000","last_became_active":"2023-10-21T16:02:00.751869+0000","last_became_peered":"2023-10-21T16:02:00.751869+0000","last_unstale":"2023-10-21T16:02:02.895695+0000","last_undegraded":"2023-10-21T16:02:02.895695+0000","last_fullsized":"2023-10-21T16:02:02.895695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:09.960025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007811+0000","last_change":"2023-10-21T16:02:00.750967+0000","last_active":"2023-10-21T16:02:02.007811+0000","last_peered":"2023-10-21T16:02:02.007811+0000","last_clean":"2023-10-21T16:02:02.007811+0000","last_became_active":"2023-10-21T16:02:00.749696+0000","last_became_peered":"2023-10-21T16:02:00.749696+0000","last_unstale":"2023-10-21T16:02:02.007811+0000","last_undegraded":"2023-10-21T16:02:02.007811+0000","last_fullsized":"2023-10-21T16:02:02.007811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910430+0000","last_change":"2023-10-21T16:01:55.938421+0000","last_active":"2023-10-21T16:02:00.910430+0000","last_peered":"2023-10-21T16:02:00.910430+0000","last_clean":"2023-10-21T16:02:00.910430+0000","last_became_active":"2023-10-21T16:01:55.938105+0000","last_became_peered":"2023-10-21T16:01:55.938105+0000","last_unstale":"2023-10-21T16:02:00.910430+0000","last_undegraded":"2023-10-21T16:02:00.910430+0000","last_fullsized":"2023-10-21T16:02:00.910430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004053+0000","last_change":"2023-10-21T16:01:55.166389+0000","last_active":"2023-10-21T16:02:01.004053+0000","last_peered":"2023-10-21T16:02:01.004053+0000","last_clean":"2023-10-21T16:02:01.004053+0000","last_became_active":"2023-10-21T16:01:55.166065+0000","last_became_peered":"2023-10-21T16:01:55.166065+0000","last_unstale":"2023-10-21T16:02:01.004053+0000","last_undegraded":"2023-10-21T16:02:01.004053+0000","last_fullsized":"2023-10-21T16:02:01.004053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895651+0000","last_change":"2023-10-21T16:02:00.195541+0000","last_active":"2023-10-21T16:02:02.895651+0000","last_peered":"2023-10-21T16:02:02.895651+0000","last_clean":"2023-10-21T16:02:02.895651+0000","last_became_active":"2023-10-21T16:02:00.195020+0000","last_became_peered":"2023-10-21T16:02:00.195020+0000","last_unstale":"2023-10-21T16:02:02.895651+0000","last_undegraded":"2023-10-21T16:02:02.895651+0000","last_fullsized":"2023-10-21T16:02:02.895651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946124+0000","last_change":"2023-10-21T16:01:55.932516+0000","last_active":"2023-10-21T16:02:00.946124+0000","last_peered":"2023-10-21T16:02:00.946124+0000","last_clean":"2023-10-21T16:02:00.946124+0000","last_became_active":"2023-10-21T16:01:55.932175+0000","last_became_peered":"2023-10-21T16:01:55.932175+0000","last_unstale":"2023-10-21T16:02:00.946124+0000","last_undegraded":"2023-10-21T16:02:00.946124+0000","last_fullsized":"2023-10-21T16:02:00.946124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945962+0000","last_change":"2023-10-21T16:01:55.156263+0000","last_active":"2023-10-21T16:02:00.945962+0000","last_peered":"2023-10-21T16:02:00.945962+0000","last_clean":"2023-10-21T16:02:00.945962+0000","last_became_active":"2023-10-21T16:01:55.155789+0000","last_became_peered":"2023-10-21T16:01:55.155789+0000","last_unstale":"2023-10-21T16:02:00.945962+0000","last_undegraded":"2023-10-21T16:02:00.945962+0000","last_fullsized":"2023-10-21T16:02:00.945962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:52.609080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415216+0000","last_change":"2023-10-21T16:01:55.160677+0000","last_active":"2023-10-21T16:02:01.415216+0000","last_peered":"2023-10-21T16:02:01.415216+0000","last_clean":"2023-10-21T16:02:01.415216+0000","last_became_active":"2023-10-21T16:01:55.160265+0000","last_became_peered":"2023-10-21T16:01:55.160265+0000","last_unstale":"2023-10-21T16:02:01.415216+0000","last_undegraded":"2023-10-21T16:02:01.415216+0000","last_fullsized":"2023-10-21T16:02:01.415216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928119+0000","last_change":"2023-10-21T16:01:55.947536+0000","last_active":"2023-10-21T16:02:00.928119+0000","last_peered":"2023-10-21T16:02:00.928119+0000","last_clean":"2023-10-21T16:02:00.928119+0000","last_became_active":"2023-10-21T16:01:55.947243+0000","last_became_peered":"2023-10-21T16:01:55.947243+0000","last_unstale":"2023-10-21T16:02:00.928119+0000","last_undegraded":"2023-10-21T16:02:00.928119+0000","last_fullsized":"2023-10-21T16:02:00.928119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.916131+0000","last_change":"2023-10-21T16:02:00.181884+0000","last_active":"2023-10-21T16:02:01.916131+0000","last_peered":"2023-10-21T16:02:01.916131+0000","last_clean":"2023-10-21T16:02:01.916131+0000","last_became_active":"2023-10-21T16:02:00.179404+0000","last_became_peered":"2023-10-21T16:02:00.179404+0000","last_unstale":"2023-10-21T16:02:01.916131+0000","last_undegraded":"2023-10-21T16:02:01.916131+0000","last_fullsized":"2023-10-21T16:02:01.916131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:42.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946429+0000","last_change":"2023-10-21T16:01:55.160559+0000","last_active":"2023-10-21T16:02:00.946429+0000","last_peered":"2023-10-21T16:02:00.946429+0000","last_clean":"2023-10-21T16:02:00.946429+0000","last_became_active":"2023-10-21T16:01:55.160241+0000","last_became_peered":"2023-10-21T16:01:55.160241+0000","last_unstale":"2023-10-21T16:02:00.946429+0000","last_undegraded":"2023-10-21T16:02:00.946429+0000","last_fullsized":"2023-10-21T16:02:00.946429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913498+0000","last_change":"2023-10-21T16:01:55.939951+0000","last_active":"2023-10-21T16:02:00.913498+0000","last_peered":"2023-10-21T16:02:00.913498+0000","last_clean":"2023-10-21T16:02:00.913498+0000","last_became_active":"2023-10-21T16:01:55.939552+0000","last_became_peered":"2023-10-21T16:01:55.939552+0000","last_unstale":"2023-10-21T16:02:00.913498+0000","last_undegraded":"2023-10-21T16:02:00.913498+0000","last_fullsized":"2023-10-21T16:02:00.913498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:38.621663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897399+0000","last_change":"2023-10-21T16:02:00.182272+0000","last_active":"2023-10-21T16:02:02.897399+0000","last_peered":"2023-10-21T16:02:02.897399+0000","last_clean":"2023-10-21T16:02:02.897399+0000","last_became_active":"2023-10-21T16:02:00.181705+0000","last_became_peered":"2023-10-21T16:02:00.181705+0000","last_unstale":"2023-10-21T16:02:02.897399+0000","last_undegraded":"2023-10-21T16:02:02.897399+0000","last_fullsized":"2023-10-21T16:02:02.897399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:28.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413688+0000","last_change":"2023-10-21T16:01:55.943801+0000","last_active":"2023-10-21T16:02:01.413688+0000","last_peered":"2023-10-21T16:02:01.413688+0000","last_clean":"2023-10-21T16:02:01.413688+0000","last_became_active":"2023-10-21T16:01:55.942546+0000","last_became_peered":"2023-10-21T16:01:55.942546+0000","last_unstale":"2023-10-21T16:02:01.413688+0000","last_undegraded":"2023-10-21T16:02:01.413688+0000","last_fullsized":"2023-10-21T16:02:01.413688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915001+0000","last_change":"2023-10-21T16:01:55.953970+0000","last_active":"2023-10-21T16:02:00.915001+0000","last_peered":"2023-10-21T16:02:00.915001+0000","last_clean":"2023-10-21T16:02:00.915001+0000","last_became_active":"2023-10-21T16:01:55.953164+0000","last_became_peered":"2023-10-21T16:01:55.953164+0000","last_unstale":"2023-10-21T16:02:00.915001+0000","last_undegraded":"2023-10-21T16:02:00.915001+0000","last_fullsized":"2023-10-21T16:02:00.915001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:16.452088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004198+0000","last_change":"2023-10-21T16:01:55.948469+0000","last_active":"2023-10-21T16:02:01.004198+0000","last_peered":"2023-10-21T16:02:01.004198+0000","last_clean":"2023-10-21T16:02:01.004198+0000","last_became_active":"2023-10-21T16:01:55.948198+0000","last_became_peered":"2023-10-21T16:01:55.948198+0000","last_unstale":"2023-10-21T16:02:01.004198+0000","last_undegraded":"2023-10-21T16:02:01.004198+0000","last_fullsized":"2023-10-21T16:02:01.004198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927645+0000","last_change":"2023-10-21T16:01:55.933799+0000","last_active":"2023-10-21T16:02:00.927645+0000","last_peered":"2023-10-21T16:02:00.927645+0000","last_clean":"2023-10-21T16:02:00.927645+0000","last_became_active":"2023-10-21T16:01:55.933431+0000","last_became_peered":"2023-10-21T16:01:55.933431+0000","last_unstale":"2023-10-21T16:02:00.927645+0000","last_undegraded":"2023-10-21T16:02:00.927645+0000","last_fullsized":"2023-10-21T16:02:00.927645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947274+0000","last_change":"2023-10-21T16:01:55.925050+0000","last_active":"2023-10-21T16:02:00.947274+0000","last_peered":"2023-10-21T16:02:00.947274+0000","last_clean":"2023-10-21T16:02:00.947274+0000","last_became_active":"2023-10-21T16:01:55.924510+0000","last_became_peered":"2023-10-21T16:01:55.924510+0000","last_unstale":"2023-10-21T16:02:00.947274+0000","last_undegraded":"2023-10-21T16:02:00.947274+0000","last_fullsized":"2023-10-21T16:02:00.947274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913921+0000","last_change":"2023-10-21T16:01:55.939109+0000","last_active":"2023-10-21T16:02:00.913921+0000","last_peered":"2023-10-21T16:02:00.913921+0000","last_clean":"2023-10-21T16:02:00.913921+0000","last_became_active":"2023-10-21T16:01:55.938852+0000","last_became_peered":"2023-10-21T16:01:55.938852+0000","last_unstale":"2023-10-21T16:02:00.913921+0000","last_undegraded":"2023-10-21T16:02:00.913921+0000","last_fullsized":"2023-10-21T16:02:00.913921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416735+0000","last_change":"2023-10-21T16:01:55.939268+0000","last_active":"2023-10-21T16:02:01.416735+0000","last_peered":"2023-10-21T16:02:01.416735+0000","last_clean":"2023-10-21T16:02:01.416735+0000","last_became_active":"2023-10-21T16:01:55.938991+0000","last_became_peered":"2023-10-21T16:01:55.938991+0000","last_unstale":"2023-10-21T16:02:01.416735+0000","last_undegraded":"2023-10-21T16:02:01.416735+0000","last_fullsized":"2023-10-21T16:02:01.416735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947882+0000","last_change":"2023-10-21T16:01:55.930486+0000","last_active":"2023-10-21T16:02:00.947882+0000","last_peered":"2023-10-21T16:02:00.947882+0000","last_clean":"2023-10-21T16:02:00.947882+0000","last_became_active":"2023-10-21T16:01:55.930187+0000","last_became_peered":"2023-10-21T16:01:55.930187+0000","last_unstale":"2023-10-21T16:02:00.947882+0000","last_undegraded":"2023-10-21T16:02:00.947882+0000","last_fullsized":"2023-10-21T16:02:00.947882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003895+0000","last_change":"2023-10-21T16:01:55.943056+0000","last_active":"2023-10-21T16:02:01.003895+0000","last_peered":"2023-10-21T16:02:01.003895+0000","last_clean":"2023-10-21T16:02:01.003895+0000","last_became_active":"2023-10-21T16:01:55.942724+0000","last_became_peered":"2023-10-21T16:01:55.942724+0000","last_unstale":"2023-10-21T16:02:01.003895+0000","last_undegraded":"2023-10-21T16:02:01.003895+0000","last_fullsized":"2023-10-21T16:02:01.003895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946201+0000","last_change":"2023-10-21T16:01:55.922812+0000","last_active":"2023-10-21T16:02:00.946201+0000","last_peered":"2023-10-21T16:02:00.946201+0000","last_clean":"2023-10-21T16:02:00.946201+0000","last_became_active":"2023-10-21T16:01:55.922134+0000","last_became_peered":"2023-10-21T16:01:55.922134+0000","last_unstale":"2023-10-21T16:02:00.946201+0000","last_undegraded":"2023-10-21T16:02:00.946201+0000","last_fullsized":"2023-10-21T16:02:00.946201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004186+0000","last_change":"2023-10-21T16:01:55.954470+0000","last_active":"2023-10-21T16:02:01.004186+0000","last_peered":"2023-10-21T16:02:01.004186+0000","last_clean":"2023-10-21T16:02:01.004186+0000","last_became_active":"2023-10-21T16:01:55.954107+0000","last_became_peered":"2023-10-21T16:01:55.954107+0000","last_unstale":"2023-10-21T16:02:01.004186+0000","last_undegraded":"2023-10-21T16:02:01.004186+0000","last_fullsized":"2023-10-21T16:02:01.004186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927404+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.927404+0000","last_peered":"2023-10-21T16:02:00.927404+0000","last_clean":"2023-10-21T16:02:00.927404+0000","last_became_active":"2023-10-21T16:01:55.946552+0000","last_became_peered":"2023-10-21T16:01:55.946552+0000","last_unstale":"2023-10-21T16:02:00.927404+0000","last_undegraded":"2023-10-21T16:02:00.927404+0000","last_fullsized":"2023-10-21T16:02:00.927404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416045+0000","last_change":"2023-10-21T16:01:55.947979+0000","last_active":"2023-10-21T16:02:01.416045+0000","last_peered":"2023-10-21T16:02:01.416045+0000","last_clean":"2023-10-21T16:02:01.416045+0000","last_became_active":"2023-10-21T16:01:55.947572+0000","last_became_peered":"2023-10-21T16:01:55.947572+0000","last_unstale":"2023-10-21T16:02:01.416045+0000","last_undegraded":"2023-10-21T16:02:01.416045+0000","last_fullsized":"2023-10-21T16:02:01.416045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415776+0000","last_change":"2023-10-21T16:01:55.946625+0000","last_active":"2023-10-21T16:02:01.415776+0000","last_peered":"2023-10-21T16:02:01.415776+0000","last_clean":"2023-10-21T16:02:01.415776+0000","last_became_active":"2023-10-21T16:01:55.945299+0000","last_became_peered":"2023-10-21T16:01:55.945299+0000","last_unstale":"2023-10-21T16:02:01.415776+0000","last_undegraded":"2023-10-21T16:02:01.415776+0000","last_fullsized":"2023-10-21T16:02:01.415776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915640+0000","last_change":"2023-10-21T16:01:55.938698+0000","last_active":"2023-10-21T16:02:00.915640+0000","last_peered":"2023-10-21T16:02:00.915640+0000","last_clean":"2023-10-21T16:02:00.915640+0000","last_became_active":"2023-10-21T16:01:55.938414+0000","last_became_peered":"2023-10-21T16:01:55.938414+0000","last_unstale":"2023-10-21T16:02:00.915640+0000","last_undegraded":"2023-10-21T16:02:00.915640+0000","last_fullsized":"2023-10-21T16:02:00.915640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:51.423697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005523+0000","last_change":"2023-10-21T16:01:55.953465+0000","last_active":"2023-10-21T16:02:01.005523+0000","last_peered":"2023-10-21T16:02:01.005523+0000","last_clean":"2023-10-21T16:02:01.005523+0000","last_became_active":"2023-10-21T16:01:55.952044+0000","last_became_peered":"2023-10-21T16:01:55.952044+0000","last_unstale":"2023-10-21T16:02:01.005523+0000","last_undegraded":"2023-10-21T16:02:01.005523+0000","last_fullsized":"2023-10-21T16:02:01.005523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003802+0000","last_change":"2023-10-21T16:01:55.943713+0000","last_active":"2023-10-21T16:02:01.003802+0000","last_peered":"2023-10-21T16:02:01.003802+0000","last_clean":"2023-10-21T16:02:01.003802+0000","last_became_active":"2023-10-21T16:01:55.939815+0000","last_became_peered":"2023-10-21T16:01:55.939815+0000","last_unstale":"2023-10-21T16:02:01.003802+0000","last_undegraded":"2023-10-21T16:02:01.003802+0000","last_fullsized":"2023-10-21T16:02:01.003802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910545+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.910545+0000","last_peered":"2023-10-21T16:02:00.910545+0000","last_clean":"2023-10-21T16:02:00.910545+0000","last_became_active":"2023-10-21T16:01:55.946640+0000","last_became_peered":"2023-10-21T16:01:55.946640+0000","last_unstale":"2023-10-21T16:02:00.910545+0000","last_undegraded":"2023-10-21T16:02:00.910545+0000","last_fullsized":"2023-10-21T16:02:00.910545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927490+0000","last_change":"2023-10-21T16:01:55.944028+0000","last_active":"2023-10-21T16:02:00.927490+0000","last_peered":"2023-10-21T16:02:00.927490+0000","last_clean":"2023-10-21T16:02:00.927490+0000","last_became_active":"2023-10-21T16:01:55.940832+0000","last_became_peered":"2023-10-21T16:01:55.940832+0000","last_unstale":"2023-10-21T16:02:00.927490+0000","last_undegraded":"2023-10-21T16:02:00.927490+0000","last_fullsized":"2023-10-21T16:02:00.927490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:42.589308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946064+0000","last_change":"2023-10-21T16:01:55.939040+0000","last_active":"2023-10-21T16:02:00.946064+0000","last_peered":"2023-10-21T16:02:00.946064+0000","last_clean":"2023-10-21T16:02:00.946064+0000","last_became_active":"2023-10-21T16:01:55.936534+0000","last_became_peered":"2023-10-21T16:01:55.936534+0000","last_unstale":"2023-10-21T16:02:00.946064+0000","last_undegraded":"2023-10-21T16:02:00.946064+0000","last_fullsized":"2023-10-21T16:02:00.946064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914180+0000","last_change":"2023-10-21T16:01:55.938149+0000","last_active":"2023-10-21T16:02:00.914180+0000","last_peered":"2023-10-21T16:02:00.914180+0000","last_clean":"2023-10-21T16:02:00.914180+0000","last_became_active":"2023-10-21T16:01:55.937906+0000","last_became_peered":"2023-10-21T16:01:55.937906+0000","last_unstale":"2023-10-21T16:02:00.914180+0000","last_undegraded":"2023-10-21T16:02:00.914180+0000","last_fullsized":"2023-10-21T16:02:00.914180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:19.855104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416487+0000","last_change":"2023-10-21T16:01:55.953883+0000","last_active":"2023-10-21T16:02:01.416487+0000","last_peered":"2023-10-21T16:02:01.416487+0000","last_clean":"2023-10-21T16:02:01.416487+0000","last_became_active":"2023-10-21T16:01:55.953252+0000","last_became_peered":"2023-10-21T16:01:55.953252+0000","last_unstale":"2023-10-21T16:02:01.416487+0000","last_undegraded":"2023-10-21T16:02:01.416487+0000","last_fullsized":"2023-10-21T16:02:01.416487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416712+0000","last_change":"2023-10-21T16:01:55.955305+0000","last_active":"2023-10-21T16:02:01.416712+0000","last_peered":"2023-10-21T16:02:01.416712+0000","last_clean":"2023-10-21T16:02:01.416712+0000","last_became_active":"2023-10-21T16:01:55.955019+0000","last_became_peered":"2023-10-21T16:01:55.955019+0000","last_unstale":"2023-10-21T16:02:01.416712+0000","last_undegraded":"2023-10-21T16:02:01.416712+0000","last_fullsized":"2023-10-21T16:02:01.416712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416673+0000","last_change":"2023-10-21T16:01:55.958446+0000","last_active":"2023-10-21T16:02:01.416673+0000","last_peered":"2023-10-21T16:02:01.416673+0000","last_clean":"2023-10-21T16:02:01.416673+0000","last_became_active":"2023-10-21T16:01:55.958056+0000","last_became_peered":"2023-10-21T16:01:55.958056+0000","last_unstale":"2023-10-21T16:02:01.416673+0000","last_undegraded":"2023-10-21T16:02:01.416673+0000","last_fullsized":"2023-10-21T16:02:01.416673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945856+0000","last_change":"2023-10-21T16:01:55.942133+0000","last_active":"2023-10-21T16:02:00.945856+0000","last_peered":"2023-10-21T16:02:00.945856+0000","last_clean":"2023-10-21T16:02:00.945856+0000","last_became_active":"2023-10-21T16:01:55.941833+0000","last_became_peered":"2023-10-21T16:01:55.941833+0000","last_unstale":"2023-10-21T16:02:00.945856+0000","last_undegraded":"2023-10-21T16:02:00.945856+0000","last_fullsized":"2023-10-21T16:02:00.945856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414741+0000","last_change":"2023-10-21T16:01:55.940356+0000","last_active":"2023-10-21T16:02:01.414741+0000","last_peered":"2023-10-21T16:02:01.414741+0000","last_clean":"2023-10-21T16:02:01.414741+0000","last_became_active":"2023-10-21T16:01:55.939602+0000","last_became_peered":"2023-10-21T16:01:55.939602+0000","last_unstale":"2023-10-21T16:02:01.414741+0000","last_undegraded":"2023-10-21T16:02:01.414741+0000","last_fullsized":"2023-10-21T16:02:01.414741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004002+0000","last_change":"2023-10-21T16:01:55.952103+0000","last_active":"2023-10-21T16:02:01.004002+0000","last_peered":"2023-10-21T16:02:01.004002+0000","last_clean":"2023-10-21T16:02:01.004002+0000","last_became_active":"2023-10-21T16:01:55.951051+0000","last_became_peered":"2023-10-21T16:01:55.951051+0000","last_unstale":"2023-10-21T16:02:01.004002+0000","last_undegraded":"2023-10-21T16:02:01.004002+0000","last_fullsized":"2023-10-21T16:02:01.004002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414699+0000","last_change":"2023-10-21T16:01:55.940974+0000","last_active":"2023-10-21T16:02:01.414699+0000","last_peered":"2023-10-21T16:02:01.414699+0000","last_clean":"2023-10-21T16:02:01.414699+0000","last_became_active":"2023-10-21T16:01:55.940667+0000","last_became_peered":"2023-10-21T16:01:55.940667+0000","last_unstale":"2023-10-21T16:02:01.414699+0000","last_undegraded":"2023-10-21T16:02:01.414699+0000","last_fullsized":"2023-10-21T16:02:01.414699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004788+0000","last_change":"2023-10-21T16:01:55.954831+0000","last_active":"2023-10-21T16:02:01.004788+0000","last_peered":"2023-10-21T16:02:01.004788+0000","last_clean":"2023-10-21T16:02:01.004788+0000","last_became_active":"2023-10-21T16:01:55.954453+0000","last_became_peered":"2023-10-21T16:01:55.954453+0000","last_unstale":"2023-10-21T16:02:01.004788+0000","last_undegraded":"2023-10-21T16:02:01.004788+0000","last_fullsized":"2023-10-21T16:02:01.004788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928170+0000","last_change":"2023-10-21T16:01:55.936408+0000","last_active":"2023-10-21T16:02:00.928170+0000","last_peered":"2023-10-21T16:02:00.928170+0000","last_clean":"2023-10-21T16:02:00.928170+0000","last_became_active":"2023-10-21T16:01:55.935946+0000","last_became_peered":"2023-10-21T16:01:55.935946+0000","last_unstale":"2023-10-21T16:02:00.928170+0000","last_undegraded":"2023-10-21T16:02:00.928170+0000","last_fullsized":"2023-10-21T16:02:00.928170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913325+0000","last_change":"2023-10-21T16:01:55.938274+0000","last_active":"2023-10-21T16:02:00.913325+0000","last_peered":"2023-10-21T16:02:00.913325+0000","last_clean":"2023-10-21T16:02:00.913325+0000","last_became_active":"2023-10-21T16:01:55.937878+0000","last_became_peered":"2023-10-21T16:01:55.937878+0000","last_unstale":"2023-10-21T16:02:00.913325+0000","last_undegraded":"2023-10-21T16:02:00.913325+0000","last_fullsized":"2023-10-21T16:02:00.913325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416466+0000","last_change":"2023-10-21T16:01:55.945925+0000","last_active":"2023-10-21T16:02:01.416466+0000","last_peered":"2023-10-21T16:02:01.416466+0000","last_clean":"2023-10-21T16:02:01.416466+0000","last_became_active":"2023-10-21T16:01:55.945457+0000","last_became_peered":"2023-10-21T16:01:55.945457+0000","last_unstale":"2023-10-21T16:02:01.416466+0000","last_undegraded":"2023-10-21T16:02:01.416466+0000","last_fullsized":"2023-10-21T16:02:01.416466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":702466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":111642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":111516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":286720,"data_stored":111676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":702522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.742}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:02:30.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:02:30.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.575+0000 7f7b2e0d1640 1 Processor -- start 2023-10-21T16:02:30.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.575+0000 7f7b2e0d1640 1 -- start start 2023-10-21T16:02:30.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2e0d1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 0x7f7b280cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2e0d1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7b28164f70 0x7f7b280b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2e0d1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b28165470 con 0x7f7b2806c710 2023-10-21T16:02:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2e0d1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b28165750 con 0x7f7b28164f70 2023-10-21T16:02:30.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2e0d1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b28165a30 con 0x7f7b28161320 2023-10-21T16:02:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.576+0000 7f7b2c8ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7b28164f70 0x7f7b280b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7b28164f70 0x7f7b280b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59866/0 (socket says 172.21.15.53:59866) 2023-10-21T16:02:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 -- 172.21.15.53:0/2777527950 learned_addr learned my addr 172.21.15.53:0/2777527950 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2d0cf640 1 --2- 172.21.15.53:0/2777527950 >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 0x7f7b280cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 -- 172.21.15.53:0/2777527950 >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 msgr2=0x7f7b280cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 --2- 172.21.15.53:0/2777527950 >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 0x7f7b280cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 -- 172.21.15.53:0/2777527950 >> v1:172.21.15.53:6789/0 conn(0x7f7b2806c710 legacy=0x7f7b280c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2c8ce640 1 -- 172.21.15.53:0/2777527950 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28165d10 con 0x7f7b28164f70 2023-10-21T16:02:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.577+0000 7f7b2d0cf640 1 --2- 172.21.15.53:0/2777527950 >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 0x7f7b280cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:30.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.578+0000 7f7b2c8ce640 1 --2- 172.21.15.53:0/2777527950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b28164f70 0x7f7b280b00c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cf6e0 tx=0x7f7b1802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f791967f10d2a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.578+0000 7f7b1ffff640 1 -- 172.21.15.53:0/2777527950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18030870 con 0x7f7b28164f70 2023-10-21T16:02:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.578+0000 7f7b1ffff640 1 -- 172.21.15.53:0/2777527950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b180309d0 con 0x7f7b28164f70 2023-10-21T16:02:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.578+0000 7f7b1ffff640 1 -- 172.21.15.53:0/2777527950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18030d60 con 0x7f7b28164f70 2023-10-21T16:02:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/2777527950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b28164f70 msgr2=0x7f7b280b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/2777527950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b28164f70 0x7f7b280b00c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cf6e0 tx=0x7f7b1802b9b0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/2777527950 shutdown_connections 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/2777527950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b28164f70 0x7f7b280b00c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/2777527950 >> v2:172.21.15.124:3300/0 conn(0x7f7b28161320 0x7f7b280cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/2777527950 >> 172.21.15.53:0/2777527950 conn(0x7f7b280cb0c0 msgr2=0x7f7b280cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/2777527950 shutdown_connections 2023-10-21T16:02:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.579+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/2777527950 wait complete. 2023-10-21T16:02:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.581+0000 7f7b2e0d1640 1 Processor -- start 2023-10-21T16:02:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.581+0000 7f7b2e0d1640 1 -- start start 2023-10-21T16:02:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b28161320 0x7f7b280b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 0x7f7b280c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b2e80 con 0x7f7b2806c710 2023-10-21T16:02:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280b32b0 con 0x7f7b28164f70 2023-10-21T16:02:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2e0d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b280c88f0 con 0x7f7b28161320 2023-10-21T16:02:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2d0cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2d0cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59882/0 (socket says 172.21.15.53:59882) 2023-10-21T16:02:30.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2d0cf640 1 -- 172.21.15.53:0/20201215 learned_addr learned my addr 172.21.15.53:0/20201215 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2d8d0640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 0x7f7b280c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.582+0000 7f7b2c8ce640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b28161320 0x7f7b280b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 -- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b28161320 msgr2=0x7f7b280b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b28161320 0x7f7b280b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 -- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 msgr2=0x7f7b280c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 0x7f7b280c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 -- 172.21.15.53:0/20201215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180094d0 con 0x7f7b2806c710 2023-10-21T16:02:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d8d0640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 0x7f7b280c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2d0cf640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c7f0 tx=0x7f7b2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24007a30 con 0x7f7b2806c710 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24007b90 con 0x7f7b2806c710 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400ce40 con 0x7f7b2806c710 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b280b3550 con 0x7f7b2806c710 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.583+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b280b3b10 con 0x7f7b2806c710 2023-10-21T16:02:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.584+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b280b2ff0 con 0x7f7b2806c710 2023-10-21T16:02:30.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.585+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b240113f0 con 0x7f7b2806c710 2023-10-21T16:02:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.586+0000 7f7b1dffb640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 0x7f7b0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.586+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7b24089b40 con 0x7f7b2806c710 2023-10-21T16:02:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.587+0000 7f7b2c8ce640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 0x7f7b0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.588+0000 7f7b2c8ce640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 0x7f7b0c06d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b18036000 tx=0x7f7b1803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.590+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b24054860 con 0x7f7b2806c710 2023-10-21T16:02:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.772+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b280c9020 con 0x7f7b0c06ae70 2023-10-21T16:02:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.798+0000 7f7b1dffb640 1 -- 172.21.15.53:0/20201215 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345114 (secure 0 0 0) 0x7f7b18030bc0 con 0x7f7b0c06ae70 2023-10-21T16:02:30.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:30.806 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:02:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 msgr2=0x7f7b0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 0x7f7b0c06d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7b18036000 tx=0x7f7b1803b040 comp rx=0 tx=0).stop 2023-10-21T16:02:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 msgr2=0x7f7b280b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400c7f0 tx=0x7f7b2400cbf0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 shutdown_connections 2023-10-21T16:02:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b28164f70 0x7f7b280c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b28161320 0x7f7b280b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7b0c06ae70 0x7f7b0c06d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 --2- 172.21.15.53:0/20201215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b2806c710 0x7f7b280b4030 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.806+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 >> 172.21.15.53:0/20201215 conn(0x7f7b280cb0c0 msgr2=0x7f7b2806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.807+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 shutdown_connections 2023-10-21T16:02:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:30.807+0000 7f7b2e0d1640 1 -- 172.21.15.53:0/20201215 wait complete. 2023-10-21T16:02:30.830 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-10-21T16:02:29.028013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":213,"num_write_kb":2436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3481600,"data_stored":2082142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005411"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928748+0000","last_change":"2023-10-21T16:01:55.158448+0000","last_active":"2023-10-21T16:02:00.928748+0000","last_peered":"2023-10-21T16:02:00.928748+0000","last_clean":"2023-10-21T16:02:00.928748+0000","last_became_active":"2023-10-21T16:01:55.158229+0000","last_became_peered":"2023-10-21T16:01:55.158229+0000","last_unstale":"2023-10-21T16:02:00.928748+0000","last_undegraded":"2023-10-21T16:02:00.928748+0000","last_fullsized":"2023-10-21T16:02:00.928748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947239+0000","last_change":"2023-10-21T16:01:55.937779+0000","last_active":"2023-10-21T16:02:00.947239+0000","last_peered":"2023-10-21T16:02:00.947239+0000","last_clean":"2023-10-21T16:02:00.947239+0000","last_became_active":"2023-10-21T16:01:55.937491+0000","last_became_peered":"2023-10-21T16:01:55.937491+0000","last_unstale":"2023-10-21T16:02:00.947239+0000","last_undegraded":"2023-10-21T16:02:00.947239+0000","last_fullsized":"2023-10-21T16:02:00.947239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895601+0000","last_change":"2023-10-21T16:02:00.754927+0000","last_active":"2023-10-21T16:02:02.895601+0000","last_peered":"2023-10-21T16:02:02.895601+0000","last_clean":"2023-10-21T16:02:02.895601+0000","last_became_active":"2023-10-21T16:02:00.753883+0000","last_became_peered":"2023-10-21T16:02:00.753883+0000","last_unstale":"2023-10-21T16:02:02.895601+0000","last_undegraded":"2023-10-21T16:02:02.895601+0000","last_fullsized":"2023-10-21T16:02:02.895601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:45.898604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947499+0000","last_change":"2023-10-21T16:01:55.153722+0000","last_active":"2023-10-21T16:02:00.947499+0000","last_peered":"2023-10-21T16:02:00.947499+0000","last_clean":"2023-10-21T16:02:00.947499+0000","last_became_active":"2023-10-21T16:01:55.153434+0000","last_became_peered":"2023-10-21T16:01:55.153434+0000","last_unstale":"2023-10-21T16:02:00.947499+0000","last_undegraded":"2023-10-21T16:02:00.947499+0000","last_fullsized":"2023-10-21T16:02:00.947499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928468+0000","last_change":"2023-10-21T16:01:55.954746+0000","last_active":"2023-10-21T16:02:00.928468+0000","last_peered":"2023-10-21T16:02:00.928468+0000","last_clean":"2023-10-21T16:02:00.928468+0000","last_became_active":"2023-10-21T16:01:55.953670+0000","last_became_peered":"2023-10-21T16:01:55.953670+0000","last_unstale":"2023-10-21T16:02:00.928468+0000","last_undegraded":"2023-10-21T16:02:00.928468+0000","last_fullsized":"2023-10-21T16:02:00.928468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918749+0000","last_change":"2023-10-21T16:02:00.185538+0000","last_active":"2023-10-21T16:02:01.918749+0000","last_peered":"2023-10-21T16:02:01.918749+0000","last_clean":"2023-10-21T16:02:01.918749+0000","last_became_active":"2023-10-21T16:02:00.185127+0000","last_became_peered":"2023-10-21T16:02:00.185127+0000","last_unstale":"2023-10-21T16:02:01.918749+0000","last_undegraded":"2023-10-21T16:02:01.918749+0000","last_fullsized":"2023-10-21T16:02:01.918749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415743+0000","last_change":"2023-10-21T16:01:55.169060+0000","last_active":"2023-10-21T16:02:01.415743+0000","last_peered":"2023-10-21T16:02:01.415743+0000","last_clean":"2023-10-21T16:02:01.415743+0000","last_became_active":"2023-10-21T16:01:55.168772+0000","last_became_peered":"2023-10-21T16:01:55.168772+0000","last_unstale":"2023-10-21T16:02:01.415743+0000","last_undegraded":"2023-10-21T16:02:01.415743+0000","last_fullsized":"2023-10-21T16:02:01.415743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004837+0000","last_change":"2023-10-21T16:01:55.950057+0000","last_active":"2023-10-21T16:02:01.004837+0000","last_peered":"2023-10-21T16:02:01.004837+0000","last_clean":"2023-10-21T16:02:01.004837+0000","last_became_active":"2023-10-21T16:01:55.949780+0000","last_became_peered":"2023-10-21T16:01:55.949780+0000","last_unstale":"2023-10-21T16:02:01.004837+0000","last_undegraded":"2023-10-21T16:02:01.004837+0000","last_fullsized":"2023-10-21T16:02:01.004837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921013+0000","last_change":"2023-10-21T16:02:00.184728+0000","last_active":"2023-10-21T16:02:01.921013+0000","last_peered":"2023-10-21T16:02:01.921013+0000","last_clean":"2023-10-21T16:02:01.921013+0000","last_became_active":"2023-10-21T16:02:00.183993+0000","last_became_peered":"2023-10-21T16:02:00.183993+0000","last_unstale":"2023-10-21T16:02:01.921013+0000","last_undegraded":"2023-10-21T16:02:01.921013+0000","last_fullsized":"2023-10-21T16:02:01.921013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415376+0000","last_change":"2023-10-21T16:01:55.171185+0000","last_active":"2023-10-21T16:02:01.415376+0000","last_peered":"2023-10-21T16:02:01.415376+0000","last_clean":"2023-10-21T16:02:01.415376+0000","last_became_active":"2023-10-21T16:01:55.170714+0000","last_became_peered":"2023-10-21T16:01:55.170714+0000","last_unstale":"2023-10-21T16:02:01.415376+0000","last_undegraded":"2023-10-21T16:02:01.415376+0000","last_fullsized":"2023-10-21T16:02:01.415376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004885+0000","last_change":"2023-10-21T16:01:55.957250+0000","last_active":"2023-10-21T16:02:01.004885+0000","last_peered":"2023-10-21T16:02:01.004885+0000","last_clean":"2023-10-21T16:02:01.004885+0000","last_became_active":"2023-10-21T16:01:55.956926+0000","last_became_peered":"2023-10-21T16:01:55.956926+0000","last_unstale":"2023-10-21T16:02:01.004885+0000","last_undegraded":"2023-10-21T16:02:01.004885+0000","last_fullsized":"2023-10-21T16:02:01.004885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.920949+0000","last_change":"2023-10-21T16:02:00.190590+0000","last_active":"2023-10-21T16:02:01.920949+0000","last_peered":"2023-10-21T16:02:01.920949+0000","last_clean":"2023-10-21T16:02:01.920949+0000","last_became_active":"2023-10-21T16:02:00.190131+0000","last_became_peered":"2023-10-21T16:02:00.190131+0000","last_unstale":"2023-10-21T16:02:01.920949+0000","last_undegraded":"2023-10-21T16:02:01.920949+0000","last_fullsized":"2023-10-21T16:02:01.920949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413796+0000","last_change":"2023-10-21T16:01:55.948227+0000","last_active":"2023-10-21T16:02:01.413796+0000","last_peered":"2023-10-21T16:02:01.413796+0000","last_clean":"2023-10-21T16:02:01.413796+0000","last_became_active":"2023-10-21T16:01:55.947879+0000","last_became_peered":"2023-10-21T16:01:55.947879+0000","last_unstale":"2023-10-21T16:02:01.413796+0000","last_undegraded":"2023-10-21T16:02:01.413796+0000","last_fullsized":"2023-10-21T16:02:01.413796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415138+0000","last_change":"2023-10-21T16:01:55.161941+0000","last_active":"2023-10-21T16:02:01.415138+0000","last_peered":"2023-10-21T16:02:01.415138+0000","last_clean":"2023-10-21T16:02:01.415138+0000","last_became_active":"2023-10-21T16:01:55.160804+0000","last_became_peered":"2023-10-21T16:01:55.160804+0000","last_unstale":"2023-10-21T16:02:01.415138+0000","last_undegraded":"2023-10-21T16:02:01.415138+0000","last_fullsized":"2023-10-21T16:02:01.415138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919532+0000","last_change":"2023-10-21T16:02:00.179242+0000","last_active":"2023-10-21T16:02:01.919532+0000","last_peered":"2023-10-21T16:02:01.919532+0000","last_clean":"2023-10-21T16:02:01.919532+0000","last_became_active":"2023-10-21T16:02:00.178744+0000","last_became_peered":"2023-10-21T16:02:00.178744+0000","last_unstale":"2023-10-21T16:02:01.919532+0000","last_undegraded":"2023-10-21T16:02:01.919532+0000","last_fullsized":"2023-10-21T16:02:01.919532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:45.186023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946147+0000","last_change":"2023-10-21T16:01:55.943291+0000","last_active":"2023-10-21T16:02:00.946147+0000","last_peered":"2023-10-21T16:02:00.946147+0000","last_clean":"2023-10-21T16:02:00.946147+0000","last_became_active":"2023-10-21T16:01:55.942937+0000","last_became_peered":"2023-10-21T16:01:55.942937+0000","last_unstale":"2023-10-21T16:02:00.946147+0000","last_undegraded":"2023-10-21T16:02:00.946147+0000","last_fullsized":"2023-10-21T16:02:00.946147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911689+0000","last_change":"2023-10-21T16:01:55.147810+0000","last_active":"2023-10-21T16:02:00.911689+0000","last_peered":"2023-10-21T16:02:00.911689+0000","last_clean":"2023-10-21T16:02:00.911689+0000","last_became_active":"2023-10-21T16:01:55.147532+0000","last_became_peered":"2023-10-21T16:01:55.147532+0000","last_unstale":"2023-10-21T16:02:00.911689+0000","last_undegraded":"2023-10-21T16:02:00.911689+0000","last_fullsized":"2023-10-21T16:02:00.911689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921974+0000","last_change":"2023-10-21T16:02:00.192496+0000","last_active":"2023-10-21T16:02:01.921974+0000","last_peered":"2023-10-21T16:02:01.921974+0000","last_clean":"2023-10-21T16:02:01.921974+0000","last_became_active":"2023-10-21T16:02:00.192090+0000","last_became_peered":"2023-10-21T16:02:00.192090+0000","last_unstale":"2023-10-21T16:02:01.921974+0000","last_undegraded":"2023-10-21T16:02:01.921974+0000","last_fullsized":"2023-10-21T16:02:01.921974+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946330+0000","last_change":"2023-10-21T16:01:55.936744+0000","last_active":"2023-10-21T16:02:00.946330+0000","last_peered":"2023-10-21T16:02:00.946330+0000","last_clean":"2023-10-21T16:02:00.946330+0000","last_became_active":"2023-10-21T16:01:55.936451+0000","last_became_peered":"2023-10-21T16:01:55.936451+0000","last_unstale":"2023-10-21T16:02:00.946330+0000","last_undegraded":"2023-10-21T16:02:00.946330+0000","last_fullsized":"2023-10-21T16:02:00.946330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910754+0000","last_change":"2023-10-21T16:01:55.159913+0000","last_active":"2023-10-21T16:02:00.910754+0000","last_peered":"2023-10-21T16:02:00.910754+0000","last_clean":"2023-10-21T16:02:00.910754+0000","last_became_active":"2023-10-21T16:01:55.159480+0000","last_became_peered":"2023-10-21T16:01:55.159480+0000","last_unstale":"2023-10-21T16:02:00.910754+0000","last_undegraded":"2023-10-21T16:02:00.910754+0000","last_fullsized":"2023-10-21T16:02:00.910754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:41.390056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895508+0000","last_change":"2023-10-21T16:02:00.748804+0000","last_active":"2023-10-21T16:02:02.895508+0000","last_peered":"2023-10-21T16:02:02.895508+0000","last_clean":"2023-10-21T16:02:02.895508+0000","last_became_active":"2023-10-21T16:02:00.746585+0000","last_became_peered":"2023-10-21T16:02:00.746585+0000","last_unstale":"2023-10-21T16:02:02.895508+0000","last_undegraded":"2023-10-21T16:02:02.895508+0000","last_fullsized":"2023-10-21T16:02:02.895508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913021+0000","last_change":"2023-10-21T16:01:55.936909+0000","last_active":"2023-10-21T16:02:00.913021+0000","last_peered":"2023-10-21T16:02:00.913021+0000","last_clean":"2023-10-21T16:02:00.913021+0000","last_became_active":"2023-10-21T16:01:55.930874+0000","last_became_peered":"2023-10-21T16:01:55.930874+0000","last_unstale":"2023-10-21T16:02:00.913021+0000","last_undegraded":"2023-10-21T16:02:00.913021+0000","last_fullsized":"2023-10-21T16:02:00.913021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:19.306040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414418+0000","last_change":"2023-10-21T16:01:55.161120+0000","last_active":"2023-10-21T16:02:01.414418+0000","last_peered":"2023-10-21T16:02:01.414418+0000","last_clean":"2023-10-21T16:02:01.414418+0000","last_became_active":"2023-10-21T16:01:55.160853+0000","last_became_peered":"2023-10-21T16:01:55.160853+0000","last_unstale":"2023-10-21T16:02:01.414418+0000","last_undegraded":"2023-10-21T16:02:01.414418+0000","last_fullsized":"2023-10-21T16:02:01.414418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:37.422410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919323+0000","last_change":"2023-10-21T16:02:00.192783+0000","last_active":"2023-10-21T16:02:01.919323+0000","last_peered":"2023-10-21T16:02:01.919323+0000","last_clean":"2023-10-21T16:02:01.919323+0000","last_became_active":"2023-10-21T16:02:00.192399+0000","last_became_peered":"2023-10-21T16:02:00.192399+0000","last_unstale":"2023-10-21T16:02:01.919323+0000","last_undegraded":"2023-10-21T16:02:01.919323+0000","last_fullsized":"2023-10-21T16:02:01.919323+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928409+0000","last_change":"2023-10-21T16:01:55.942303+0000","last_active":"2023-10-21T16:02:00.928409+0000","last_peered":"2023-10-21T16:02:00.928409+0000","last_clean":"2023-10-21T16:02:00.928409+0000","last_became_active":"2023-10-21T16:01:55.941848+0000","last_became_peered":"2023-10-21T16:01:55.941848+0000","last_unstale":"2023-10-21T16:02:00.928409+0000","last_undegraded":"2023-10-21T16:02:00.928409+0000","last_fullsized":"2023-10-21T16:02:00.928409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417420+0000","last_change":"2023-10-21T16:01:55.164579+0000","last_active":"2023-10-21T16:02:01.417420+0000","last_peered":"2023-10-21T16:02:01.417420+0000","last_clean":"2023-10-21T16:02:01.417420+0000","last_became_active":"2023-10-21T16:01:55.162946+0000","last_became_peered":"2023-10-21T16:01:55.162946+0000","last_unstale":"2023-10-21T16:02:01.417420+0000","last_undegraded":"2023-10-21T16:02:01.417420+0000","last_fullsized":"2023-10-21T16:02:01.417420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.919836+0000","last_change":"2023-10-21T16:02:00.748601+0000","last_active":"2023-10-21T16:02:01.919836+0000","last_peered":"2023-10-21T16:02:01.919836+0000","last_clean":"2023-10-21T16:02:01.919836+0000","last_became_active":"2023-10-21T16:02:00.747592+0000","last_became_peered":"2023-10-21T16:02:00.747592+0000","last_unstale":"2023-10-21T16:02:01.919836+0000","last_undegraded":"2023-10-21T16:02:01.919836+0000","last_fullsized":"2023-10-21T16:02:01.919836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947881+0000","last_change":"2023-10-21T16:01:55.941119+0000","last_active":"2023-10-21T16:02:00.947881+0000","last_peered":"2023-10-21T16:02:00.947881+0000","last_clean":"2023-10-21T16:02:00.947881+0000","last_became_active":"2023-10-21T16:01:55.940145+0000","last_became_peered":"2023-10-21T16:01:55.940145+0000","last_unstale":"2023-10-21T16:02:00.947881+0000","last_undegraded":"2023-10-21T16:02:00.947881+0000","last_fullsized":"2023-10-21T16:02:00.947881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:13.347675+0000","last_change":"2023-10-21T16:01:55.165800+0000","last_active":"2023-10-21T16:02:13.347675+0000","last_peered":"2023-10-21T16:02:13.347675+0000","last_clean":"2023-10-21T16:02:13.347675+0000","last_became_active":"2023-10-21T16:01:55.165500+0000","last_became_peered":"2023-10-21T16:01:55.165500+0000","last_unstale":"2023-10-21T16:02:13.347675+0000","last_undegraded":"2023-10-21T16:02:13.347675+0000","last_fullsized":"2023-10-21T16:02:13.347675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894525+0000","last_change":"2023-10-21T16:02:00.742519+0000","last_active":"2023-10-21T16:02:02.894525+0000","last_peered":"2023-10-21T16:02:02.894525+0000","last_clean":"2023-10-21T16:02:02.894525+0000","last_became_active":"2023-10-21T16:02:00.740937+0000","last_became_peered":"2023-10-21T16:02:00.740937+0000","last_unstale":"2023-10-21T16:02:02.894525+0000","last_undegraded":"2023-10-21T16:02:02.894525+0000","last_fullsized":"2023-10-21T16:02:02.894525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:22.482979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913147+0000","last_change":"2023-10-21T16:01:55.939913+0000","last_active":"2023-10-21T16:02:00.913147+0000","last_peered":"2023-10-21T16:02:00.913147+0000","last_clean":"2023-10-21T16:02:00.913147+0000","last_became_active":"2023-10-21T16:01:55.939419+0000","last_became_peered":"2023-10-21T16:01:55.939419+0000","last_unstale":"2023-10-21T16:02:00.913147+0000","last_undegraded":"2023-10-21T16:02:00.913147+0000","last_fullsized":"2023-10-21T16:02:00.913147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415967+0000","last_change":"2023-10-21T16:01:55.163977+0000","last_active":"2023-10-21T16:02:01.415967+0000","last_peered":"2023-10-21T16:02:01.415967+0000","last_clean":"2023-10-21T16:02:01.415967+0000","last_became_active":"2023-10-21T16:01:55.163739+0000","last_became_peered":"2023-10-21T16:01:55.163739+0000","last_unstale":"2023-10-21T16:02:01.415967+0000","last_undegraded":"2023-10-21T16:02:01.415967+0000","last_fullsized":"2023-10-21T16:02:01.415967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:17.767707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007711+0000","last_change":"2023-10-21T16:02:00.751077+0000","last_active":"2023-10-21T16:02:02.007711+0000","last_peered":"2023-10-21T16:02:02.007711+0000","last_clean":"2023-10-21T16:02:02.007711+0000","last_became_active":"2023-10-21T16:02:00.749997+0000","last_became_peered":"2023-10-21T16:02:00.749997+0000","last_unstale":"2023-10-21T16:02:02.007711+0000","last_undegraded":"2023-10-21T16:02:02.007711+0000","last_fullsized":"2023-10-21T16:02:02.007711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005010+0000","last_change":"2023-10-21T16:01:55.951660+0000","last_active":"2023-10-21T16:02:01.005010+0000","last_peered":"2023-10-21T16:02:01.005010+0000","last_clean":"2023-10-21T16:02:01.005010+0000","last_became_active":"2023-10-21T16:01:55.951279+0000","last_became_peered":"2023-10-21T16:01:55.951279+0000","last_unstale":"2023-10-21T16:02:01.005010+0000","last_undegraded":"2023-10-21T16:02:01.005010+0000","last_fullsized":"2023-10-21T16:02:01.005010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913993+0000","last_change":"2023-10-21T16:01:55.149562+0000","last_active":"2023-10-21T16:02:00.913993+0000","last_peered":"2023-10-21T16:02:00.913993+0000","last_clean":"2023-10-21T16:02:00.913993+0000","last_became_active":"2023-10-21T16:01:55.149274+0000","last_became_peered":"2023-10-21T16:01:55.149274+0000","last_unstale":"2023-10-21T16:02:00.913993+0000","last_undegraded":"2023-10-21T16:02:00.913993+0000","last_fullsized":"2023-10-21T16:02:00.913993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.892788+0000","last_change":"2023-10-21T16:02:00.189379+0000","last_active":"2023-10-21T16:02:02.892788+0000","last_peered":"2023-10-21T16:02:02.892788+0000","last_clean":"2023-10-21T16:02:02.892788+0000","last_became_active":"2023-10-21T16:02:00.188693+0000","last_became_peered":"2023-10-21T16:02:00.188693+0000","last_unstale":"2023-10-21T16:02:02.892788+0000","last_undegraded":"2023-10-21T16:02:02.892788+0000","last_fullsized":"2023-10-21T16:02:02.892788+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:15.481295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913564+0000","last_change":"2023-10-21T16:01:55.941518+0000","last_active":"2023-10-21T16:02:00.913564+0000","last_peered":"2023-10-21T16:02:00.913564+0000","last_clean":"2023-10-21T16:02:00.913564+0000","last_became_active":"2023-10-21T16:01:55.938726+0000","last_became_peered":"2023-10-21T16:01:55.938726+0000","last_unstale":"2023-10-21T16:02:00.913564+0000","last_undegraded":"2023-10-21T16:02:00.913564+0000","last_fullsized":"2023-10-21T16:02:00.913564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:39.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004473+0000","last_change":"2023-10-21T16:01:55.165664+0000","last_active":"2023-10-21T16:02:01.004473+0000","last_peered":"2023-10-21T16:02:01.004473+0000","last_clean":"2023-10-21T16:02:01.004473+0000","last_became_active":"2023-10-21T16:01:55.165070+0000","last_became_peered":"2023-10-21T16:01:55.165070+0000","last_unstale":"2023-10-21T16:02:01.004473+0000","last_undegraded":"2023-10-21T16:02:01.004473+0000","last_fullsized":"2023-10-21T16:02:01.004473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:02.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893925+0000","last_change":"2023-10-21T16:02:00.186762+0000","last_active":"2023-10-21T16:02:02.893925+0000","last_peered":"2023-10-21T16:02:02.893925+0000","last_clean":"2023-10-21T16:02:02.893925+0000","last_became_active":"2023-10-21T16:02:00.185437+0000","last_became_peered":"2023-10-21T16:02:00.185437+0000","last_unstale":"2023-10-21T16:02:02.893925+0000","last_undegraded":"2023-10-21T16:02:02.893925+0000","last_fullsized":"2023-10-21T16:02:02.893925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:01.977808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.911065+0000","last_change":"2023-10-21T16:01:55.935243+0000","last_active":"2023-10-21T16:02:00.911065+0000","last_peered":"2023-10-21T16:02:00.911065+0000","last_clean":"2023-10-21T16:02:00.911065+0000","last_became_active":"2023-10-21T16:01:55.934353+0000","last_became_peered":"2023-10-21T16:01:55.934353+0000","last_unstale":"2023-10-21T16:02:00.911065+0000","last_undegraded":"2023-10-21T16:02:00.911065+0000","last_fullsized":"2023-10-21T16:02:00.911065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914987+0000","last_change":"2023-10-21T16:01:55.147043+0000","last_active":"2023-10-21T16:02:00.914987+0000","last_peered":"2023-10-21T16:02:00.914987+0000","last_clean":"2023-10-21T16:02:00.914987+0000","last_became_active":"2023-10-21T16:01:55.146674+0000","last_became_peered":"2023-10-21T16:01:55.146674+0000","last_unstale":"2023-10-21T16:02:00.914987+0000","last_undegraded":"2023-10-21T16:02:00.914987+0000","last_fullsized":"2023-10-21T16:02:00.914987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:18.106788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895819+0000","last_change":"2023-10-21T16:02:00.191162+0000","last_active":"2023-10-21T16:02:02.895819+0000","last_peered":"2023-10-21T16:02:02.895819+0000","last_clean":"2023-10-21T16:02:02.895819+0000","last_became_active":"2023-10-21T16:02:00.186599+0000","last_became_peered":"2023-10-21T16:02:00.186599+0000","last_unstale":"2023-10-21T16:02:02.895819+0000","last_undegraded":"2023-10-21T16:02:02.895819+0000","last_fullsized":"2023-10-21T16:02:02.895819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.926918+0000","last_change":"2023-10-21T16:01:55.948611+0000","last_active":"2023-10-21T16:02:00.926918+0000","last_peered":"2023-10-21T16:02:00.926918+0000","last_clean":"2023-10-21T16:02:00.926918+0000","last_became_active":"2023-10-21T16:01:55.948172+0000","last_became_peered":"2023-10-21T16:01:55.948172+0000","last_unstale":"2023-10-21T16:02:00.926918+0000","last_undegraded":"2023-10-21T16:02:00.926918+0000","last_fullsized":"2023-10-21T16:02:00.926918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414533+0000","last_change":"2023-10-21T16:01:55.161709+0000","last_active":"2023-10-21T16:02:01.414533+0000","last_peered":"2023-10-21T16:02:01.414533+0000","last_clean":"2023-10-21T16:02:01.414533+0000","last_became_active":"2023-10-21T16:01:55.160426+0000","last_became_peered":"2023-10-21T16:01:55.160426+0000","last_unstale":"2023-10-21T16:02:01.414533+0000","last_undegraded":"2023-10-21T16:02:01.414533+0000","last_fullsized":"2023-10-21T16:02:01.414533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:57:38.217905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007971+0000","last_change":"2023-10-21T16:02:00.742783+0000","last_active":"2023-10-21T16:02:02.007971+0000","last_peered":"2023-10-21T16:02:02.007971+0000","last_clean":"2023-10-21T16:02:02.007971+0000","last_became_active":"2023-10-21T16:02:00.740932+0000","last_became_peered":"2023-10-21T16:02:00.740932+0000","last_unstale":"2023-10-21T16:02:02.007971+0000","last_undegraded":"2023-10-21T16:02:02.007971+0000","last_fullsized":"2023-10-21T16:02:02.007971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946103+0000","last_change":"2023-10-21T16:01:55.955491+0000","last_active":"2023-10-21T16:02:00.946103+0000","last_peered":"2023-10-21T16:02:00.946103+0000","last_clean":"2023-10-21T16:02:00.946103+0000","last_became_active":"2023-10-21T16:01:55.955163+0000","last_became_peered":"2023-10-21T16:01:55.955163+0000","last_unstale":"2023-10-21T16:02:00.946103+0000","last_undegraded":"2023-10-21T16:02:00.946103+0000","last_fullsized":"2023-10-21T16:02:00.946103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004498+0000","last_change":"2023-10-21T16:01:55.160793+0000","last_active":"2023-10-21T16:02:01.004498+0000","last_peered":"2023-10-21T16:02:01.004498+0000","last_clean":"2023-10-21T16:02:01.004498+0000","last_became_active":"2023-10-21T16:01:55.157151+0000","last_became_peered":"2023-10-21T16:01:55.157151+0000","last_unstale":"2023-10-21T16:02:01.004498+0000","last_undegraded":"2023-10-21T16:02:01.004498+0000","last_fullsized":"2023-10-21T16:02:01.004498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:36.886516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414663+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:02:01.414663+0000","last_peered":"2023-10-21T16:02:01.414663+0000","last_clean":"2023-10-21T16:02:01.414663+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:02:01.414663+0000","last_undegraded":"2023-10-21T16:02:01.414663+0000","last_fullsized":"2023-10-21T16:02:01.414663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895666+0000","last_change":"2023-10-21T16:02:00.752861+0000","last_active":"2023-10-21T16:02:02.895666+0000","last_peered":"2023-10-21T16:02:02.895666+0000","last_clean":"2023-10-21T16:02:02.895666+0000","last_became_active":"2023-10-21T16:02:00.748880+0000","last_became_peered":"2023-10-21T16:02:00.748880+0000","last_unstale":"2023-10-21T16:02:02.895666+0000","last_undegraded":"2023-10-21T16:02:02.895666+0000","last_fullsized":"2023-10-21T16:02:02.895666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:28.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415527+0000","last_change":"2023-10-21T16:01:55.954245+0000","last_active":"2023-10-21T16:02:01.415527+0000","last_peered":"2023-10-21T16:02:01.415527+0000","last_clean":"2023-10-21T16:02:01.415527+0000","last_became_active":"2023-10-21T16:01:55.953197+0000","last_became_peered":"2023-10-21T16:01:55.953197+0000","last_unstale":"2023-10-21T16:02:01.415527+0000","last_undegraded":"2023-10-21T16:02:01.415527+0000","last_fullsized":"2023-10-21T16:02:01.415527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915622+0000","last_change":"2023-10-21T16:01:55.168127+0000","last_active":"2023-10-21T16:02:00.915622+0000","last_peered":"2023-10-21T16:02:00.915622+0000","last_clean":"2023-10-21T16:02:00.915622+0000","last_became_active":"2023-10-21T16:01:55.167766+0000","last_became_peered":"2023-10-21T16:01:55.167766+0000","last_unstale":"2023-10-21T16:02:00.915622+0000","last_undegraded":"2023-10-21T16:02:00.915622+0000","last_fullsized":"2023-10-21T16:02:00.915622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:28:10.704795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893224+0000","last_change":"2023-10-21T16:02:00.742293+0000","last_active":"2023-10-21T16:02:02.893224+0000","last_peered":"2023-10-21T16:02:02.893224+0000","last_clean":"2023-10-21T16:02:02.893224+0000","last_became_active":"2023-10-21T16:02:00.741230+0000","last_became_peered":"2023-10-21T16:02:00.741230+0000","last_unstale":"2023-10-21T16:02:02.893224+0000","last_undegraded":"2023-10-21T16:02:02.893224+0000","last_fullsized":"2023-10-21T16:02:02.893224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005008+0000","last_change":"2023-10-21T16:01:55.948576+0000","last_active":"2023-10-21T16:02:01.005008+0000","last_peered":"2023-10-21T16:02:01.005008+0000","last_clean":"2023-10-21T16:02:01.005008+0000","last_became_active":"2023-10-21T16:01:55.947390+0000","last_became_peered":"2023-10-21T16:01:55.947390+0000","last_unstale":"2023-10-21T16:02:01.005008+0000","last_undegraded":"2023-10-21T16:02:01.005008+0000","last_fullsized":"2023-10-21T16:02:01.005008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914480+0000","last_change":"2023-10-21T16:01:55.151627+0000","last_active":"2023-10-21T16:02:00.914480+0000","last_peered":"2023-10-21T16:02:00.914480+0000","last_clean":"2023-10-21T16:02:00.914480+0000","last_became_active":"2023-10-21T16:01:55.151283+0000","last_became_peered":"2023-10-21T16:01:55.151283+0000","last_unstale":"2023-10-21T16:02:00.914480+0000","last_undegraded":"2023-10-21T16:02:00.914480+0000","last_fullsized":"2023-10-21T16:02:00.914480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007984+0000","last_change":"2023-10-21T16:02:00.183073+0000","last_active":"2023-10-21T16:02:02.007984+0000","last_peered":"2023-10-21T16:02:02.007984+0000","last_clean":"2023-10-21T16:02:02.007984+0000","last_became_active":"2023-10-21T16:02:00.182604+0000","last_became_peered":"2023-10-21T16:02:00.182604+0000","last_unstale":"2023-10-21T16:02:02.007984+0000","last_undegraded":"2023-10-21T16:02:02.007984+0000","last_fullsized":"2023-10-21T16:02:02.007984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416904+0000","last_change":"2023-10-21T16:01:55.951113+0000","last_active":"2023-10-21T16:02:01.416904+0000","last_peered":"2023-10-21T16:02:01.416904+0000","last_clean":"2023-10-21T16:02:01.416904+0000","last_became_active":"2023-10-21T16:01:55.950792+0000","last_became_peered":"2023-10-21T16:01:55.950792+0000","last_unstale":"2023-10-21T16:02:01.416904+0000","last_undegraded":"2023-10-21T16:02:01.416904+0000","last_fullsized":"2023-10-21T16:02:01.416904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915704+0000","last_change":"2023-10-21T16:01:55.149597+0000","last_active":"2023-10-21T16:02:00.915704+0000","last_peered":"2023-10-21T16:02:00.915704+0000","last_clean":"2023-10-21T16:02:00.915704+0000","last_became_active":"2023-10-21T16:01:55.149258+0000","last_became_peered":"2023-10-21T16:01:55.149258+0000","last_unstale":"2023-10-21T16:02:00.915704+0000","last_undegraded":"2023-10-21T16:02:00.915704+0000","last_fullsized":"2023-10-21T16:02:00.915704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.914975+0000","last_change":"2023-10-21T16:02:00.750068+0000","last_active":"2023-10-21T16:02:01.914975+0000","last_peered":"2023-10-21T16:02:01.914975+0000","last_clean":"2023-10-21T16:02:01.914975+0000","last_became_active":"2023-10-21T16:02:00.749055+0000","last_became_peered":"2023-10-21T16:02:00.749055+0000","last_unstale":"2023-10-21T16:02:01.914975+0000","last_undegraded":"2023-10-21T16:02:01.914975+0000","last_fullsized":"2023-10-21T16:02:01.914975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:26.171565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416138+0000","last_change":"2023-10-21T16:01:55.953586+0000","last_active":"2023-10-21T16:02:01.416138+0000","last_peered":"2023-10-21T16:02:01.416138+0000","last_clean":"2023-10-21T16:02:01.416138+0000","last_became_active":"2023-10-21T16:01:55.952550+0000","last_became_peered":"2023-10-21T16:01:55.952550+0000","last_unstale":"2023-10-21T16:02:01.416138+0000","last_undegraded":"2023-10-21T16:02:01.416138+0000","last_fullsized":"2023-10-21T16:02:01.416138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:16.649604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913456+0000","last_change":"2023-10-21T16:01:55.154502+0000","last_active":"2023-10-21T16:02:00.913456+0000","last_peered":"2023-10-21T16:02:00.913456+0000","last_clean":"2023-10-21T16:02:00.913456+0000","last_became_active":"2023-10-21T16:01:55.150688+0000","last_became_peered":"2023-10-21T16:01:55.150688+0000","last_unstale":"2023-10-21T16:02:00.913456+0000","last_undegraded":"2023-10-21T16:02:00.913456+0000","last_fullsized":"2023-10-21T16:02:00.913456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:32.709310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.918774+0000","last_change":"2023-10-21T16:02:00.184524+0000","last_active":"2023-10-21T16:02:01.918774+0000","last_peered":"2023-10-21T16:02:01.918774+0000","last_clean":"2023-10-21T16:02:01.918774+0000","last_became_active":"2023-10-21T16:02:00.184081+0000","last_became_peered":"2023-10-21T16:02:00.184081+0000","last_unstale":"2023-10-21T16:02:01.918774+0000","last_undegraded":"2023-10-21T16:02:01.918774+0000","last_fullsized":"2023-10-21T16:02:01.918774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:05:20.676938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414960+0000","last_change":"2023-10-21T16:01:55.951983+0000","last_active":"2023-10-21T16:02:01.414960+0000","last_peered":"2023-10-21T16:02:01.414960+0000","last_clean":"2023-10-21T16:02:01.414960+0000","last_became_active":"2023-10-21T16:01:55.951633+0000","last_became_peered":"2023-10-21T16:01:55.951633+0000","last_unstale":"2023-10-21T16:02:01.414960+0000","last_undegraded":"2023-10-21T16:02:01.414960+0000","last_fullsized":"2023-10-21T16:02:01.414960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914399+0000","last_change":"2023-10-21T16:01:55.147150+0000","last_active":"2023-10-21T16:02:00.914399+0000","last_peered":"2023-10-21T16:02:00.914399+0000","last_clean":"2023-10-21T16:02:00.914399+0000","last_became_active":"2023-10-21T16:01:55.141230+0000","last_became_peered":"2023-10-21T16:01:55.141230+0000","last_unstale":"2023-10-21T16:02:00.914399+0000","last_undegraded":"2023-10-21T16:02:00.914399+0000","last_fullsized":"2023-10-21T16:02:00.914399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007865+0000","last_change":"2023-10-21T16:02:00.187875+0000","last_active":"2023-10-21T16:02:02.007865+0000","last_peered":"2023-10-21T16:02:02.007865+0000","last_clean":"2023-10-21T16:02:02.007865+0000","last_became_active":"2023-10-21T16:02:00.187427+0000","last_became_peered":"2023-10-21T16:02:00.187427+0000","last_unstale":"2023-10-21T16:02:02.007865+0000","last_undegraded":"2023-10-21T16:02:02.007865+0000","last_fullsized":"2023-10-21T16:02:02.007865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:25:51.576442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414077+0000","last_change":"2023-10-21T16:01:55.944005+0000","last_active":"2023-10-21T16:02:01.414077+0000","last_peered":"2023-10-21T16:02:01.414077+0000","last_clean":"2023-10-21T16:02:01.414077+0000","last_became_active":"2023-10-21T16:01:55.942887+0000","last_became_peered":"2023-10-21T16:01:55.942887+0000","last_unstale":"2023-10-21T16:02:01.414077+0000","last_undegraded":"2023-10-21T16:02:01.414077+0000","last_fullsized":"2023-10-21T16:02:01.414077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:04.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413985+0000","last_change":"2023-10-21T16:01:55.160115+0000","last_active":"2023-10-21T16:02:01.413985+0000","last_peered":"2023-10-21T16:02:01.413985+0000","last_clean":"2023-10-21T16:02:01.413985+0000","last_became_active":"2023-10-21T16:01:55.159788+0000","last_became_peered":"2023-10-21T16:01:55.159788+0000","last_unstale":"2023-10-21T16:02:01.413985+0000","last_undegraded":"2023-10-21T16:02:01.413985+0000","last_fullsized":"2023-10-21T16:02:01.413985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:57.057561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897390+0000","last_change":"2023-10-21T16:02:00.745714+0000","last_active":"2023-10-21T16:02:02.897390+0000","last_peered":"2023-10-21T16:02:02.897390+0000","last_clean":"2023-10-21T16:02:02.897390+0000","last_became_active":"2023-10-21T16:02:00.744779+0000","last_became_peered":"2023-10-21T16:02:00.744779+0000","last_unstale":"2023-10-21T16:02:02.897390+0000","last_undegraded":"2023-10-21T16:02:02.897390+0000","last_fullsized":"2023-10-21T16:02:02.897390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:53.297692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414998+0000","last_change":"2023-10-21T16:01:55.955734+0000","last_active":"2023-10-21T16:02:01.414998+0000","last_peered":"2023-10-21T16:02:01.414998+0000","last_clean":"2023-10-21T16:02:01.414998+0000","last_became_active":"2023-10-21T16:01:55.955333+0000","last_became_peered":"2023-10-21T16:01:55.955333+0000","last_unstale":"2023-10-21T16:02:01.414998+0000","last_undegraded":"2023-10-21T16:02:01.414998+0000","last_fullsized":"2023-10-21T16:02:01.414998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004475+0000","last_change":"2023-10-21T16:01:55.167450+0000","last_active":"2023-10-21T16:02:01.004475+0000","last_peered":"2023-10-21T16:02:01.004475+0000","last_clean":"2023-10-21T16:02:01.004475+0000","last_became_active":"2023-10-21T16:01:55.166857+0000","last_became_peered":"2023-10-21T16:01:55.166857+0000","last_unstale":"2023-10-21T16:02:01.004475+0000","last_undegraded":"2023-10-21T16:02:01.004475+0000","last_fullsized":"2023-10-21T16:02:01.004475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:03.337694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.896683+0000","last_change":"2023-10-21T16:02:00.742809+0000","last_active":"2023-10-21T16:02:02.896683+0000","last_peered":"2023-10-21T16:02:02.896683+0000","last_clean":"2023-10-21T16:02:02.896683+0000","last_became_active":"2023-10-21T16:02:00.741398+0000","last_became_peered":"2023-10-21T16:02:00.741398+0000","last_unstale":"2023-10-21T16:02:02.896683+0000","last_undegraded":"2023-10-21T16:02:02.896683+0000","last_fullsized":"2023-10-21T16:02:02.896683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:35:32.899715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927531+0000","last_change":"2023-10-21T16:01:55.937515+0000","last_active":"2023-10-21T16:02:00.927531+0000","last_peered":"2023-10-21T16:02:00.927531+0000","last_clean":"2023-10-21T16:02:00.927531+0000","last_became_active":"2023-10-21T16:01:55.937165+0000","last_became_peered":"2023-10-21T16:01:55.937165+0000","last_unstale":"2023-10-21T16:02:00.927531+0000","last_undegraded":"2023-10-21T16:02:00.927531+0000","last_fullsized":"2023-10-21T16:02:00.927531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928104+0000","last_change":"2023-10-21T16:01:55.151763+0000","last_active":"2023-10-21T16:02:00.928104+0000","last_peered":"2023-10-21T16:02:00.928104+0000","last_clean":"2023-10-21T16:02:00.928104+0000","last_became_active":"2023-10-21T16:01:55.151316+0000","last_became_peered":"2023-10-21T16:01:55.151316+0000","last_unstale":"2023-10-21T16:02:00.928104+0000","last_undegraded":"2023-10-21T16:02:00.928104+0000","last_fullsized":"2023-10-21T16:02:00.928104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.917604+0000","last_change":"2023-10-21T16:02:00.742437+0000","last_active":"2023-10-21T16:02:01.917604+0000","last_peered":"2023-10-21T16:02:01.917604+0000","last_clean":"2023-10-21T16:02:01.917604+0000","last_became_active":"2023-10-21T16:02:00.741212+0000","last_became_peered":"2023-10-21T16:02:00.741212+0000","last_unstale":"2023-10-21T16:02:01.917604+0000","last_undegraded":"2023-10-21T16:02:01.917604+0000","last_fullsized":"2023-10-21T16:02:01.917604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:34:02.135246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415373+0000","last_change":"2023-10-21T16:01:55.942841+0000","last_active":"2023-10-21T16:02:01.415373+0000","last_peered":"2023-10-21T16:02:01.415373+0000","last_clean":"2023-10-21T16:02:01.415373+0000","last_became_active":"2023-10-21T16:01:55.942471+0000","last_became_peered":"2023-10-21T16:01:55.942471+0000","last_unstale":"2023-10-21T16:02:01.415373+0000","last_undegraded":"2023-10-21T16:02:01.415373+0000","last_fullsized":"2023-10-21T16:02:01.415373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:14.233631+0000","last_change":"2023-10-21T16:01:55.156634+0000","last_active":"2023-10-21T16:02:14.233631+0000","last_peered":"2023-10-21T16:02:14.233631+0000","last_clean":"2023-10-21T16:02:14.233631+0000","last_became_active":"2023-10-21T16:01:55.156340+0000","last_became_peered":"2023-10-21T16:01:55.156340+0000","last_unstale":"2023-10-21T16:02:14.233631+0000","last_undegraded":"2023-10-21T16:02:14.233631+0000","last_fullsized":"2023-10-21T16:02:14.233631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:23.492370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.893349+0000","last_change":"2023-10-21T16:02:00.189158+0000","last_active":"2023-10-21T16:02:02.893349+0000","last_peered":"2023-10-21T16:02:02.893349+0000","last_clean":"2023-10-21T16:02:02.893349+0000","last_became_active":"2023-10-21T16:02:00.188097+0000","last_became_peered":"2023-10-21T16:02:00.188097+0000","last_unstale":"2023-10-21T16:02:02.893349+0000","last_undegraded":"2023-10-21T16:02:02.893349+0000","last_fullsized":"2023-10-21T16:02:02.893349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:07.271546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914268+0000","last_change":"2023-10-21T16:01:55.944799+0000","last_active":"2023-10-21T16:02:00.914268+0000","last_peered":"2023-10-21T16:02:00.914268+0000","last_clean":"2023-10-21T16:02:00.914268+0000","last_became_active":"2023-10-21T16:01:55.944472+0000","last_became_peered":"2023-10-21T16:01:55.944472+0000","last_unstale":"2023-10-21T16:02:00.914268+0000","last_undegraded":"2023-10-21T16:02:00.914268+0000","last_fullsized":"2023-10-21T16:02:00.914268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:58.256813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914405+0000","last_change":"2023-10-21T16:01:55.162979+0000","last_active":"2023-10-21T16:02:00.914405+0000","last_peered":"2023-10-21T16:02:00.914405+0000","last_clean":"2023-10-21T16:02:00.914405+0000","last_became_active":"2023-10-21T16:01:55.162557+0000","last_became_peered":"2023-10-21T16:01:55.162557+0000","last_unstale":"2023-10-21T16:02:00.914405+0000","last_undegraded":"2023-10-21T16:02:00.914405+0000","last_fullsized":"2023-10-21T16:02:00.914405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.921560+0000","last_change":"2023-10-21T16:02:00.188187+0000","last_active":"2023-10-21T16:02:01.921560+0000","last_peered":"2023-10-21T16:02:01.921560+0000","last_clean":"2023-10-21T16:02:01.921560+0000","last_became_active":"2023-10-21T16:02:00.187688+0000","last_became_peered":"2023-10-21T16:02:00.187688+0000","last_unstale":"2023-10-21T16:02:01.921560+0000","last_undegraded":"2023-10-21T16:02:01.921560+0000","last_fullsized":"2023-10-21T16:02:01.921560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928702+0000","last_change":"2023-10-21T16:01:55.954216+0000","last_active":"2023-10-21T16:02:00.928702+0000","last_peered":"2023-10-21T16:02:00.928702+0000","last_clean":"2023-10-21T16:02:00.928702+0000","last_became_active":"2023-10-21T16:01:55.953211+0000","last_became_peered":"2023-10-21T16:01:55.953211+0000","last_unstale":"2023-10-21T16:02:00.928702+0000","last_undegraded":"2023-10-21T16:02:00.928702+0000","last_fullsized":"2023-10-21T16:02:00.928702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005397+0000","last_change":"2023-10-21T16:01:55.169893+0000","last_active":"2023-10-21T16:02:01.005397+0000","last_peered":"2023-10-21T16:02:01.005397+0000","last_clean":"2023-10-21T16:02:01.005397+0000","last_became_active":"2023-10-21T16:01:55.169587+0000","last_became_peered":"2023-10-21T16:01:55.169587+0000","last_unstale":"2023-10-21T16:02:01.005397+0000","last_undegraded":"2023-10-21T16:02:01.005397+0000","last_fullsized":"2023-10-21T16:02:01.005397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:18.653419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.894928+0000","last_change":"2023-10-21T16:02:00.185249+0000","last_active":"2023-10-21T16:02:02.894928+0000","last_peered":"2023-10-21T16:02:02.894928+0000","last_clean":"2023-10-21T16:02:02.894928+0000","last_became_active":"2023-10-21T16:02:00.184448+0000","last_became_peered":"2023-10-21T16:02:00.184448+0000","last_unstale":"2023-10-21T16:02:02.894928+0000","last_undegraded":"2023-10-21T16:02:02.894928+0000","last_fullsized":"2023-10-21T16:02:02.894928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:18:23.475051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005291+0000","last_change":"2023-10-21T16:01:55.954077+0000","last_active":"2023-10-21T16:02:01.005291+0000","last_peered":"2023-10-21T16:02:01.005291+0000","last_clean":"2023-10-21T16:02:01.005291+0000","last_became_active":"2023-10-21T16:01:55.951152+0000","last_became_peered":"2023-10-21T16:01:55.951152+0000","last_unstale":"2023-10-21T16:02:01.005291+0000","last_undegraded":"2023-10-21T16:02:01.005291+0000","last_fullsized":"2023-10-21T16:02:01.005291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:41.158688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.417259+0000","last_change":"2023-10-21T16:01:55.171136+0000","last_active":"2023-10-21T16:02:01.417259+0000","last_peered":"2023-10-21T16:02:01.417259+0000","last_clean":"2023-10-21T16:02:01.417259+0000","last_became_active":"2023-10-21T16:01:55.170850+0000","last_became_peered":"2023-10-21T16:01:55.170850+0000","last_unstale":"2023-10-21T16:02:01.417259+0000","last_undegraded":"2023-10-21T16:02:01.417259+0000","last_fullsized":"2023-10-21T16:02:01.417259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895695+0000","last_change":"2023-10-21T16:02:00.753599+0000","last_active":"2023-10-21T16:02:02.895695+0000","last_peered":"2023-10-21T16:02:02.895695+0000","last_clean":"2023-10-21T16:02:02.895695+0000","last_became_active":"2023-10-21T16:02:00.751869+0000","last_became_peered":"2023-10-21T16:02:00.751869+0000","last_unstale":"2023-10-21T16:02:02.895695+0000","last_undegraded":"2023-10-21T16:02:02.895695+0000","last_fullsized":"2023-10-21T16:02:02.895695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:09.960025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.007811+0000","last_change":"2023-10-21T16:02:00.750967+0000","last_active":"2023-10-21T16:02:02.007811+0000","last_peered":"2023-10-21T16:02:02.007811+0000","last_clean":"2023-10-21T16:02:02.007811+0000","last_became_active":"2023-10-21T16:02:00.749696+0000","last_became_peered":"2023-10-21T16:02:00.749696+0000","last_unstale":"2023-10-21T16:02:02.007811+0000","last_undegraded":"2023-10-21T16:02:02.007811+0000","last_fullsized":"2023-10-21T16:02:02.007811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:58.990695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910430+0000","last_change":"2023-10-21T16:01:55.938421+0000","last_active":"2023-10-21T16:02:00.910430+0000","last_peered":"2023-10-21T16:02:00.910430+0000","last_clean":"2023-10-21T16:02:00.910430+0000","last_became_active":"2023-10-21T16:01:55.938105+0000","last_became_peered":"2023-10-21T16:01:55.938105+0000","last_unstale":"2023-10-21T16:02:00.910430+0000","last_undegraded":"2023-10-21T16:02:00.910430+0000","last_fullsized":"2023-10-21T16:02:00.910430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004053+0000","last_change":"2023-10-21T16:01:55.166389+0000","last_active":"2023-10-21T16:02:01.004053+0000","last_peered":"2023-10-21T16:02:01.004053+0000","last_clean":"2023-10-21T16:02:01.004053+0000","last_became_active":"2023-10-21T16:01:55.166065+0000","last_became_peered":"2023-10-21T16:01:55.166065+0000","last_unstale":"2023-10-21T16:02:01.004053+0000","last_undegraded":"2023-10-21T16:02:01.004053+0000","last_fullsized":"2023-10-21T16:02:01.004053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:18.655852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.895651+0000","last_change":"2023-10-21T16:02:00.195541+0000","last_active":"2023-10-21T16:02:02.895651+0000","last_peered":"2023-10-21T16:02:02.895651+0000","last_clean":"2023-10-21T16:02:02.895651+0000","last_became_active":"2023-10-21T16:02:00.195020+0000","last_became_peered":"2023-10-21T16:02:00.195020+0000","last_unstale":"2023-10-21T16:02:02.895651+0000","last_undegraded":"2023-10-21T16:02:02.895651+0000","last_fullsized":"2023-10-21T16:02:02.895651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:35.591226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946124+0000","last_change":"2023-10-21T16:01:55.932516+0000","last_active":"2023-10-21T16:02:00.946124+0000","last_peered":"2023-10-21T16:02:00.946124+0000","last_clean":"2023-10-21T16:02:00.946124+0000","last_became_active":"2023-10-21T16:01:55.932175+0000","last_became_peered":"2023-10-21T16:01:55.932175+0000","last_unstale":"2023-10-21T16:02:00.946124+0000","last_undegraded":"2023-10-21T16:02:00.946124+0000","last_fullsized":"2023-10-21T16:02:00.946124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945962+0000","last_change":"2023-10-21T16:01:55.156263+0000","last_active":"2023-10-21T16:02:00.945962+0000","last_peered":"2023-10-21T16:02:00.945962+0000","last_clean":"2023-10-21T16:02:00.945962+0000","last_became_active":"2023-10-21T16:01:55.155789+0000","last_became_peered":"2023-10-21T16:01:55.155789+0000","last_unstale":"2023-10-21T16:02:00.945962+0000","last_undegraded":"2023-10-21T16:02:00.945962+0000","last_fullsized":"2023-10-21T16:02:00.945962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:52.609080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415216+0000","last_change":"2023-10-21T16:01:55.160677+0000","last_active":"2023-10-21T16:02:01.415216+0000","last_peered":"2023-10-21T16:02:01.415216+0000","last_clean":"2023-10-21T16:02:01.415216+0000","last_became_active":"2023-10-21T16:01:55.160265+0000","last_became_peered":"2023-10-21T16:01:55.160265+0000","last_unstale":"2023-10-21T16:02:01.415216+0000","last_undegraded":"2023-10-21T16:02:01.415216+0000","last_fullsized":"2023-10-21T16:02:01.415216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:15.252836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928119+0000","last_change":"2023-10-21T16:01:55.947536+0000","last_active":"2023-10-21T16:02:00.928119+0000","last_peered":"2023-10-21T16:02:00.928119+0000","last_clean":"2023-10-21T16:02:00.928119+0000","last_became_active":"2023-10-21T16:01:55.947243+0000","last_became_peered":"2023-10-21T16:01:55.947243+0000","last_unstale":"2023-10-21T16:02:00.928119+0000","last_undegraded":"2023-10-21T16:02:00.928119+0000","last_fullsized":"2023-10-21T16:02:00.928119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.916131+0000","last_change":"2023-10-21T16:02:00.181884+0000","last_active":"2023-10-21T16:02:01.916131+0000","last_peered":"2023-10-21T16:02:01.916131+0000","last_clean":"2023-10-21T16:02:01.916131+0000","last_became_active":"2023-10-21T16:02:00.179404+0000","last_became_peered":"2023-10-21T16:02:00.179404+0000","last_unstale":"2023-10-21T16:02:01.916131+0000","last_undegraded":"2023-10-21T16:02:01.916131+0000","last_fullsized":"2023-10-21T16:02:01.916131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:42.113103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946429+0000","last_change":"2023-10-21T16:01:55.160559+0000","last_active":"2023-10-21T16:02:00.946429+0000","last_peered":"2023-10-21T16:02:00.946429+0000","last_clean":"2023-10-21T16:02:00.946429+0000","last_became_active":"2023-10-21T16:01:55.160241+0000","last_became_peered":"2023-10-21T16:01:55.160241+0000","last_unstale":"2023-10-21T16:02:00.946429+0000","last_undegraded":"2023-10-21T16:02:00.946429+0000","last_fullsized":"2023-10-21T16:02:00.946429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:53.913010+0000","last_clean_scrub_stamp":"2023-10-21T16:01:53.913010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:50.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913498+0000","last_change":"2023-10-21T16:01:55.939951+0000","last_active":"2023-10-21T16:02:00.913498+0000","last_peered":"2023-10-21T16:02:00.913498+0000","last_clean":"2023-10-21T16:02:00.913498+0000","last_became_active":"2023-10-21T16:01:55.939552+0000","last_became_peered":"2023-10-21T16:01:55.939552+0000","last_unstale":"2023-10-21T16:02:00.913498+0000","last_undegraded":"2023-10-21T16:02:00.913498+0000","last_fullsized":"2023-10-21T16:02:00.913498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:38.621663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-10-21T16:02:02.897399+0000","last_change":"2023-10-21T16:02:00.182272+0000","last_active":"2023-10-21T16:02:02.897399+0000","last_peered":"2023-10-21T16:02:02.897399+0000","last_clean":"2023-10-21T16:02:02.897399+0000","last_became_active":"2023-10-21T16:02:00.181705+0000","last_became_peered":"2023-10-21T16:02:00.181705+0000","last_unstale":"2023-10-21T16:02:02.897399+0000","last_undegraded":"2023-10-21T16:02:02.897399+0000","last_fullsized":"2023-10-21T16:02:02.897399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:59.139597+0000","last_clean_scrub_stamp":"2023-10-21T16:01:59.139597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:28.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.413688+0000","last_change":"2023-10-21T16:01:55.943801+0000","last_active":"2023-10-21T16:02:01.413688+0000","last_peered":"2023-10-21T16:02:01.413688+0000","last_clean":"2023-10-21T16:02:01.413688+0000","last_became_active":"2023-10-21T16:01:55.942546+0000","last_became_peered":"2023-10-21T16:01:55.942546+0000","last_unstale":"2023-10-21T16:02:01.413688+0000","last_undegraded":"2023-10-21T16:02:01.413688+0000","last_fullsized":"2023-10-21T16:02:01.413688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915001+0000","last_change":"2023-10-21T16:01:55.953970+0000","last_active":"2023-10-21T16:02:00.915001+0000","last_peered":"2023-10-21T16:02:00.915001+0000","last_clean":"2023-10-21T16:02:00.915001+0000","last_became_active":"2023-10-21T16:01:55.953164+0000","last_became_peered":"2023-10-21T16:01:55.953164+0000","last_unstale":"2023-10-21T16:02:00.915001+0000","last_undegraded":"2023-10-21T16:02:00.915001+0000","last_fullsized":"2023-10-21T16:02:00.915001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:46:16.452088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004198+0000","last_change":"2023-10-21T16:01:55.948469+0000","last_active":"2023-10-21T16:02:01.004198+0000","last_peered":"2023-10-21T16:02:01.004198+0000","last_clean":"2023-10-21T16:02:01.004198+0000","last_became_active":"2023-10-21T16:01:55.948198+0000","last_became_peered":"2023-10-21T16:01:55.948198+0000","last_unstale":"2023-10-21T16:02:01.004198+0000","last_undegraded":"2023-10-21T16:02:01.004198+0000","last_fullsized":"2023-10-21T16:02:01.004198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:45.956214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927645+0000","last_change":"2023-10-21T16:01:55.933799+0000","last_active":"2023-10-21T16:02:00.927645+0000","last_peered":"2023-10-21T16:02:00.927645+0000","last_clean":"2023-10-21T16:02:00.927645+0000","last_became_active":"2023-10-21T16:01:55.933431+0000","last_became_peered":"2023-10-21T16:01:55.933431+0000","last_unstale":"2023-10-21T16:02:00.927645+0000","last_undegraded":"2023-10-21T16:02:00.927645+0000","last_fullsized":"2023-10-21T16:02:00.927645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947274+0000","last_change":"2023-10-21T16:01:55.925050+0000","last_active":"2023-10-21T16:02:00.947274+0000","last_peered":"2023-10-21T16:02:00.947274+0000","last_clean":"2023-10-21T16:02:00.947274+0000","last_became_active":"2023-10-21T16:01:55.924510+0000","last_became_peered":"2023-10-21T16:01:55.924510+0000","last_unstale":"2023-10-21T16:02:00.947274+0000","last_undegraded":"2023-10-21T16:02:00.947274+0000","last_fullsized":"2023-10-21T16:02:00.947274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913921+0000","last_change":"2023-10-21T16:01:55.939109+0000","last_active":"2023-10-21T16:02:00.913921+0000","last_peered":"2023-10-21T16:02:00.913921+0000","last_clean":"2023-10-21T16:02:00.913921+0000","last_became_active":"2023-10-21T16:01:55.938852+0000","last_became_peered":"2023-10-21T16:01:55.938852+0000","last_unstale":"2023-10-21T16:02:00.913921+0000","last_undegraded":"2023-10-21T16:02:00.913921+0000","last_fullsized":"2023-10-21T16:02:00.913921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416735+0000","last_change":"2023-10-21T16:01:55.939268+0000","last_active":"2023-10-21T16:02:01.416735+0000","last_peered":"2023-10-21T16:02:01.416735+0000","last_clean":"2023-10-21T16:02:01.416735+0000","last_became_active":"2023-10-21T16:01:55.938991+0000","last_became_peered":"2023-10-21T16:01:55.938991+0000","last_unstale":"2023-10-21T16:02:01.416735+0000","last_undegraded":"2023-10-21T16:02:01.416735+0000","last_fullsized":"2023-10-21T16:02:01.416735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.947882+0000","last_change":"2023-10-21T16:01:55.930486+0000","last_active":"2023-10-21T16:02:00.947882+0000","last_peered":"2023-10-21T16:02:00.947882+0000","last_clean":"2023-10-21T16:02:00.947882+0000","last_became_active":"2023-10-21T16:01:55.930187+0000","last_became_peered":"2023-10-21T16:01:55.930187+0000","last_unstale":"2023-10-21T16:02:00.947882+0000","last_undegraded":"2023-10-21T16:02:00.947882+0000","last_fullsized":"2023-10-21T16:02:00.947882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003895+0000","last_change":"2023-10-21T16:01:55.943056+0000","last_active":"2023-10-21T16:02:01.003895+0000","last_peered":"2023-10-21T16:02:01.003895+0000","last_clean":"2023-10-21T16:02:01.003895+0000","last_became_active":"2023-10-21T16:01:55.942724+0000","last_became_peered":"2023-10-21T16:01:55.942724+0000","last_unstale":"2023-10-21T16:02:01.003895+0000","last_undegraded":"2023-10-21T16:02:01.003895+0000","last_fullsized":"2023-10-21T16:02:01.003895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:08:31.303247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946201+0000","last_change":"2023-10-21T16:01:55.922812+0000","last_active":"2023-10-21T16:02:00.946201+0000","last_peered":"2023-10-21T16:02:00.946201+0000","last_clean":"2023-10-21T16:02:00.946201+0000","last_became_active":"2023-10-21T16:01:55.922134+0000","last_became_peered":"2023-10-21T16:01:55.922134+0000","last_unstale":"2023-10-21T16:02:00.946201+0000","last_undegraded":"2023-10-21T16:02:00.946201+0000","last_fullsized":"2023-10-21T16:02:00.946201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004186+0000","last_change":"2023-10-21T16:01:55.954470+0000","last_active":"2023-10-21T16:02:01.004186+0000","last_peered":"2023-10-21T16:02:01.004186+0000","last_clean":"2023-10-21T16:02:01.004186+0000","last_became_active":"2023-10-21T16:01:55.954107+0000","last_became_peered":"2023-10-21T16:01:55.954107+0000","last_unstale":"2023-10-21T16:02:01.004186+0000","last_undegraded":"2023-10-21T16:02:01.004186+0000","last_fullsized":"2023-10-21T16:02:01.004186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927404+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.927404+0000","last_peered":"2023-10-21T16:02:00.927404+0000","last_clean":"2023-10-21T16:02:00.927404+0000","last_became_active":"2023-10-21T16:01:55.946552+0000","last_became_peered":"2023-10-21T16:01:55.946552+0000","last_unstale":"2023-10-21T16:02:00.927404+0000","last_undegraded":"2023-10-21T16:02:00.927404+0000","last_fullsized":"2023-10-21T16:02:00.927404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416045+0000","last_change":"2023-10-21T16:01:55.947979+0000","last_active":"2023-10-21T16:02:01.416045+0000","last_peered":"2023-10-21T16:02:01.416045+0000","last_clean":"2023-10-21T16:02:01.416045+0000","last_became_active":"2023-10-21T16:01:55.947572+0000","last_became_peered":"2023-10-21T16:01:55.947572+0000","last_unstale":"2023-10-21T16:02:01.416045+0000","last_undegraded":"2023-10-21T16:02:01.416045+0000","last_fullsized":"2023-10-21T16:02:01.416045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:19.852672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.415776+0000","last_change":"2023-10-21T16:01:55.946625+0000","last_active":"2023-10-21T16:02:01.415776+0000","last_peered":"2023-10-21T16:02:01.415776+0000","last_clean":"2023-10-21T16:02:01.415776+0000","last_became_active":"2023-10-21T16:01:55.945299+0000","last_became_peered":"2023-10-21T16:01:55.945299+0000","last_unstale":"2023-10-21T16:02:01.415776+0000","last_undegraded":"2023-10-21T16:02:01.415776+0000","last_fullsized":"2023-10-21T16:02:01.415776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:25.751842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.915640+0000","last_change":"2023-10-21T16:01:55.938698+0000","last_active":"2023-10-21T16:02:00.915640+0000","last_peered":"2023-10-21T16:02:00.915640+0000","last_clean":"2023-10-21T16:02:00.915640+0000","last_became_active":"2023-10-21T16:01:55.938414+0000","last_became_peered":"2023-10-21T16:01:55.938414+0000","last_unstale":"2023-10-21T16:02:00.915640+0000","last_undegraded":"2023-10-21T16:02:00.915640+0000","last_fullsized":"2023-10-21T16:02:00.915640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:51.423697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.005523+0000","last_change":"2023-10-21T16:01:55.953465+0000","last_active":"2023-10-21T16:02:01.005523+0000","last_peered":"2023-10-21T16:02:01.005523+0000","last_clean":"2023-10-21T16:02:01.005523+0000","last_became_active":"2023-10-21T16:01:55.952044+0000","last_became_peered":"2023-10-21T16:01:55.952044+0000","last_unstale":"2023-10-21T16:02:01.005523+0000","last_undegraded":"2023-10-21T16:02:01.005523+0000","last_fullsized":"2023-10-21T16:02:01.005523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:46.990623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.003802+0000","last_change":"2023-10-21T16:01:55.943713+0000","last_active":"2023-10-21T16:02:01.003802+0000","last_peered":"2023-10-21T16:02:01.003802+0000","last_clean":"2023-10-21T16:02:01.003802+0000","last_became_active":"2023-10-21T16:01:55.939815+0000","last_became_peered":"2023-10-21T16:01:55.939815+0000","last_unstale":"2023-10-21T16:02:01.003802+0000","last_undegraded":"2023-10-21T16:02:01.003802+0000","last_fullsized":"2023-10-21T16:02:01.003802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.910545+0000","last_change":"2023-10-21T16:01:55.946915+0000","last_active":"2023-10-21T16:02:00.910545+0000","last_peered":"2023-10-21T16:02:00.910545+0000","last_clean":"2023-10-21T16:02:00.910545+0000","last_became_active":"2023-10-21T16:01:55.946640+0000","last_became_peered":"2023-10-21T16:01:55.946640+0000","last_unstale":"2023-10-21T16:02:00.910545+0000","last_undegraded":"2023-10-21T16:02:00.910545+0000","last_fullsized":"2023-10-21T16:02:00.910545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.927490+0000","last_change":"2023-10-21T16:01:55.944028+0000","last_active":"2023-10-21T16:02:00.927490+0000","last_peered":"2023-10-21T16:02:00.927490+0000","last_clean":"2023-10-21T16:02:00.927490+0000","last_became_active":"2023-10-21T16:01:55.940832+0000","last_became_peered":"2023-10-21T16:01:55.940832+0000","last_unstale":"2023-10-21T16:02:00.927490+0000","last_undegraded":"2023-10-21T16:02:00.927490+0000","last_fullsized":"2023-10-21T16:02:00.927490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:42.589308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.946064+0000","last_change":"2023-10-21T16:01:55.939040+0000","last_active":"2023-10-21T16:02:00.946064+0000","last_peered":"2023-10-21T16:02:00.946064+0000","last_clean":"2023-10-21T16:02:00.946064+0000","last_became_active":"2023-10-21T16:01:55.936534+0000","last_became_peered":"2023-10-21T16:01:55.936534+0000","last_unstale":"2023-10-21T16:02:00.946064+0000","last_undegraded":"2023-10-21T16:02:00.946064+0000","last_fullsized":"2023-10-21T16:02:00.946064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:00.016277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.914180+0000","last_change":"2023-10-21T16:01:55.938149+0000","last_active":"2023-10-21T16:02:00.914180+0000","last_peered":"2023-10-21T16:02:00.914180+0000","last_clean":"2023-10-21T16:02:00.914180+0000","last_became_active":"2023-10-21T16:01:55.937906+0000","last_became_peered":"2023-10-21T16:01:55.937906+0000","last_unstale":"2023-10-21T16:02:00.914180+0000","last_undegraded":"2023-10-21T16:02:00.914180+0000","last_fullsized":"2023-10-21T16:02:00.914180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:19.855104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416487+0000","last_change":"2023-10-21T16:01:55.953883+0000","last_active":"2023-10-21T16:02:01.416487+0000","last_peered":"2023-10-21T16:02:01.416487+0000","last_clean":"2023-10-21T16:02:01.416487+0000","last_became_active":"2023-10-21T16:01:55.953252+0000","last_became_peered":"2023-10-21T16:01:55.953252+0000","last_unstale":"2023-10-21T16:02:01.416487+0000","last_undegraded":"2023-10-21T16:02:01.416487+0000","last_fullsized":"2023-10-21T16:02:01.416487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:13.627139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416712+0000","last_change":"2023-10-21T16:01:55.955305+0000","last_active":"2023-10-21T16:02:01.416712+0000","last_peered":"2023-10-21T16:02:01.416712+0000","last_clean":"2023-10-21T16:02:01.416712+0000","last_became_active":"2023-10-21T16:01:55.955019+0000","last_became_peered":"2023-10-21T16:01:55.955019+0000","last_unstale":"2023-10-21T16:02:01.416712+0000","last_undegraded":"2023-10-21T16:02:01.416712+0000","last_fullsized":"2023-10-21T16:02:01.416712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:03.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416673+0000","last_change":"2023-10-21T16:01:55.958446+0000","last_active":"2023-10-21T16:02:01.416673+0000","last_peered":"2023-10-21T16:02:01.416673+0000","last_clean":"2023-10-21T16:02:01.416673+0000","last_became_active":"2023-10-21T16:01:55.958056+0000","last_became_peered":"2023-10-21T16:01:55.958056+0000","last_unstale":"2023-10-21T16:02:01.416673+0000","last_undegraded":"2023-10-21T16:02:01.416673+0000","last_fullsized":"2023-10-21T16:02:01.416673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:24.447892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.945856+0000","last_change":"2023-10-21T16:01:55.942133+0000","last_active":"2023-10-21T16:02:00.945856+0000","last_peered":"2023-10-21T16:02:00.945856+0000","last_clean":"2023-10-21T16:02:00.945856+0000","last_became_active":"2023-10-21T16:01:55.941833+0000","last_became_peered":"2023-10-21T16:01:55.941833+0000","last_unstale":"2023-10-21T16:02:00.945856+0000","last_undegraded":"2023-10-21T16:02:00.945856+0000","last_fullsized":"2023-10-21T16:02:00.945856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:38.085769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414741+0000","last_change":"2023-10-21T16:01:55.940356+0000","last_active":"2023-10-21T16:02:01.414741+0000","last_peered":"2023-10-21T16:02:01.414741+0000","last_clean":"2023-10-21T16:02:01.414741+0000","last_became_active":"2023-10-21T16:01:55.939602+0000","last_became_peered":"2023-10-21T16:01:55.939602+0000","last_unstale":"2023-10-21T16:02:01.414741+0000","last_undegraded":"2023-10-21T16:02:01.414741+0000","last_fullsized":"2023-10-21T16:02:01.414741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004002+0000","last_change":"2023-10-21T16:01:55.952103+0000","last_active":"2023-10-21T16:02:01.004002+0000","last_peered":"2023-10-21T16:02:01.004002+0000","last_clean":"2023-10-21T16:02:01.004002+0000","last_became_active":"2023-10-21T16:01:55.951051+0000","last_became_peered":"2023-10-21T16:01:55.951051+0000","last_unstale":"2023-10-21T16:02:01.004002+0000","last_undegraded":"2023-10-21T16:02:01.004002+0000","last_fullsized":"2023-10-21T16:02:01.004002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:32:49.807955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.414699+0000","last_change":"2023-10-21T16:01:55.940974+0000","last_active":"2023-10-21T16:02:01.414699+0000","last_peered":"2023-10-21T16:02:01.414699+0000","last_clean":"2023-10-21T16:02:01.414699+0000","last_became_active":"2023-10-21T16:01:55.940667+0000","last_became_peered":"2023-10-21T16:01:55.940667+0000","last_unstale":"2023-10-21T16:02:01.414699+0000","last_undegraded":"2023-10-21T16:02:01.414699+0000","last_fullsized":"2023-10-21T16:02:01.414699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:11.799646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.004788+0000","last_change":"2023-10-21T16:01:55.954831+0000","last_active":"2023-10-21T16:02:01.004788+0000","last_peered":"2023-10-21T16:02:01.004788+0000","last_clean":"2023-10-21T16:02:01.004788+0000","last_became_active":"2023-10-21T16:01:55.954453+0000","last_became_peered":"2023-10-21T16:01:55.954453+0000","last_unstale":"2023-10-21T16:02:01.004788+0000","last_undegraded":"2023-10-21T16:02:01.004788+0000","last_fullsized":"2023-10-21T16:02:01.004788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:24.052080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.928170+0000","last_change":"2023-10-21T16:01:55.936408+0000","last_active":"2023-10-21T16:02:00.928170+0000","last_peered":"2023-10-21T16:02:00.928170+0000","last_clean":"2023-10-21T16:02:00.928170+0000","last_became_active":"2023-10-21T16:01:55.935946+0000","last_became_peered":"2023-10-21T16:01:55.935946+0000","last_unstale":"2023-10-21T16:02:00.928170+0000","last_undegraded":"2023-10-21T16:02:00.928170+0000","last_fullsized":"2023-10-21T16:02:00.928170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:38.449910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:00.913325+0000","last_change":"2023-10-21T16:01:55.938274+0000","last_active":"2023-10-21T16:02:00.913325+0000","last_peered":"2023-10-21T16:02:00.913325+0000","last_clean":"2023-10-21T16:02:00.913325+0000","last_became_active":"2023-10-21T16:01:55.937878+0000","last_became_peered":"2023-10-21T16:01:55.937878+0000","last_unstale":"2023-10-21T16:02:00.913325+0000","last_undegraded":"2023-10-21T16:02:00.913325+0000","last_fullsized":"2023-10-21T16:02:00.913325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:18.966960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-21T16:02:01.416466+0000","last_change":"2023-10-21T16:01:55.945925+0000","last_active":"2023-10-21T16:02:01.416466+0000","last_peered":"2023-10-21T16:02:01.416466+0000","last_clean":"2023-10-21T16:02:01.416466+0000","last_became_active":"2023-10-21T16:01:55.945457+0000","last_became_peered":"2023-10-21T16:01:55.945457+0000","last_unstale":"2023-10-21T16:02:01.416466+0000","last_undegraded":"2023-10-21T16:02:01.416466+0000","last_fullsized":"2023-10-21T16:02:01.416466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:55.112262+0000","last_clean_scrub_stamp":"2023-10-21T16:01:55.112262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:31.563892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":702466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.244}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.208}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5089999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5369999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":111606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.321}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.849}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":111642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":290816,"data_stored":115357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":111516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":286720,"data_stored":111676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":702522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.742}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:02:30.837 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-21T16:02:30.838 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-21T16:02:30.838 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-21T16:02:30.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-21T16:02:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.094+0000 7f0c88275640 1 Processor -- start 2023-10-21T16:02:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.094+0000 7f0c88275640 1 -- start start 2023-10-21T16:02:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c88275640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c80151aa0 0x7f0c800c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c88275640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 0x7f0c8015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c88275640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800b1320 con 0x7f0c8015ec30 2023-10-21T16:02:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c88275640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c800b0a80 con 0x7f0c80151aa0 2023-10-21T16:02:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c88275640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c800b0d60 con 0x7f0c800c93c0 2023-10-21T16:02:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86011640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c80151aa0 0x7f0c800c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86812640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0c8015ec30 0x7f0c8015b730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59830/0 (socket says 172.21.15.53:59830) 2023-10-21T16:02:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86812640 1 -- 172.21.15.53:0/702148247 learned_addr learned my addr 172.21.15.53:0/702148247 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:31.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c85810640 1 --2- 172.21.15.53:0/702148247 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 0x7f0c8015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86011640 1 -- 172.21.15.53:0/702148247 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 msgr2=0x7f0c8015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:31.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86011640 1 --2- 172.21.15.53:0/702148247 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 0x7f0c8015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c8500f640 1 -- 172.21.15.53:0/702148247 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944225472 0 0) 0x7f0c800b1320 con 0x7f0c8015ec30 2023-10-21T16:02:31.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86011640 1 -- 172.21.15.53:0/702148247 >> v1:172.21.15.53:6789/0 conn(0x7f0c8015ec30 legacy=0x7f0c8015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c86011640 1 -- 172.21.15.53:0/702148247 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800b1040 con 0x7f0c80151aa0 2023-10-21T16:02:31.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.096+0000 7f0c85810640 1 --2- 172.21.15.53:0/702148247 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 0x7f0c8015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:31.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c86011640 1 --2- 172.21.15.53:0/702148247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c80151aa0 0x7f0c800c8e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b4a60 tx=0x7f0c7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d10031ed83b8c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c8500f640 1 -- 172.21.15.53:0/702148247 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c031830 con 0x7f0c80151aa0 2023-10-21T16:02:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c8500f640 1 -- 172.21.15.53:0/702148247 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c031990 con 0x7f0c80151aa0 2023-10-21T16:02:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c8500f640 1 -- 172.21.15.53:0/702148247 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c031d20 con 0x7f0c80151aa0 2023-10-21T16:02:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 -- 172.21.15.53:0/702148247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c80151aa0 msgr2=0x7f0c800c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 --2- 172.21.15.53:0/702148247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c80151aa0 0x7f0c800c8e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c800b4a60 tx=0x7f0c7c02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 -- 172.21.15.53:0/702148247 shutdown_connections 2023-10-21T16:02:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 --2- 172.21.15.53:0/702148247 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c93c0 0x7f0c8015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 --2- 172.21.15.53:0/702148247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c80151aa0 0x7f0c800c8e80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.098+0000 7f0c88275640 1 -- 172.21.15.53:0/702148247 >> 172.21.15.53:0/702148247 conn(0x7f0c800b20b0 msgr2=0x7f0c80069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.099+0000 7f0c88275640 1 -- 172.21.15.53:0/702148247 shutdown_connections 2023-10-21T16:02:31.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.099+0000 7f0c88275640 1 -- 172.21.15.53:0/702148247 wait complete. 2023-10-21T16:02:31.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.100+0000 7f0c88275640 1 Processor -- start 2023-10-21T16:02:31.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.100+0000 7f0c88275640 1 -- start start 2023-10-21T16:02:31.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c80151aa0 0x7f0c800c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c8015ec30 0x7f0c800c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c76e0 con 0x7f0c800c93c0 2023-10-21T16:02:31.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801534d0 con 0x7f0c80151aa0 2023-10-21T16:02:31.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.101+0000 7f0c88275640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c42a0 con 0x7f0c8015ec30 2023-10-21T16:02:31.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59900/0 (socket says 172.21.15.53:59900) 2023-10-21T16:02:31.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 -- 172.21.15.53:0/3966530183 learned_addr learned my addr 172.21.15.53:0/3966530183 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:31.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c85810640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c80151aa0 0x7f0c800c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86812640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c8015ec30 0x7f0c800c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 -- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c8015ec30 msgr2=0x7f0c800c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c8015ec30 0x7f0c800c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 -- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c80151aa0 msgr2=0x7f0c800c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c80151aa0 0x7f0c800c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.102+0000 7f0c86011640 1 -- 172.21.15.53:0/3966530183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c009400 con 0x7f0c800c93c0 2023-10-21T16:02:31.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c86011640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009890 tx=0x7f0c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c03ba70 con 0x7f0c800c93c0 2023-10-21T16:02:31.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c03bbf0 con 0x7f0c800c93c0 2023-10-21T16:02:31.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c045400 con 0x7f0c800c93c0 2023-10-21T16:02:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c800c6e40 con 0x7f0c800c93c0 2023-10-21T16:02:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.103+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c800c7400 con 0x7f0c800c93c0 2023-10-21T16:02:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.104+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c801532a0 con 0x7f0c800c93c0 2023-10-21T16:02:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.105+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c7c04e440 con 0x7f0c800c93c0 2023-10-21T16:02:31.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.106+0000 7f0c76ffd640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 0x7f0c6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.106+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0c7c0abc40 con 0x7f0c800c93c0 2023-10-21T16:02:31.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.107+0000 7f0c85810640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 0x7f0c6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.108+0000 7f0c85810640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 0x7f0c6406d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c80151a40 tx=0x7f0c70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.111+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c801532a0 con 0x7f0c800c93c0 2023-10-21T16:02:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.340+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0c801532a0 con 0x7f0c800c93c0 2023-10-21T16:02:31.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.341+0000 7f0c76ffd640 1 -- 172.21.15.53:0/3966530183 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0c7c035040 con 0x7f0c800c93c0 2023-10-21T16:02:31.343 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:31.343 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-21T16:02:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.343+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 msgr2=0x7f0c6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.343+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 0x7f0c6406d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c80151a40 tx=0x7f0c70008040 comp rx=0 tx=0).stop 2023-10-21T16:02:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.343+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 msgr2=0x7f0c8014d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.343+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009890 tx=0x7f0c7c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 shutdown_connections 2023-10-21T16:02:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c8015ec30 0x7f0c800c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c80151aa0 0x7f0c800c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f0c6406ae70 0x7f0c6406d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 --2- 172.21.15.53:0/3966530183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c93c0 0x7f0c8014d870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 >> 172.21.15.53:0/3966530183 conn(0x7f0c800b20b0 msgr2=0x7f0c800cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 shutdown_connections 2023-10-21T16:02:31.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.344+0000 7f0c88275640 1 -- 172.21.15.53:0/3966530183 wait complete. 2023-10-21T16:02:31.364 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-21T16:02:31.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.586+0000 7fad4c9d2640 1 Processor -- start 2023-10-21T16:02:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.587+0000 7fad4c9d2640 1 -- start start 2023-10-21T16:02:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad4c9d2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad480adc70 0x7fad480b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad4c9d2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815af10 0x7fad480b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad4c9d2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4815f7d0 con 0x7fad4815ab30 2023-10-21T16:02:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad4c9d2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4815efb0 con 0x7fad480adc70 2023-10-21T16:02:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad4c9d2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815f210 con 0x7fad4815af10 2023-10-21T16:02:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad477fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fad4815ab30 0x7fad4815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59846/0 (socket says 172.21.15.53:59846) 2023-10-21T16:02:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad477fe640 1 -- 172.21.15.53:0/2461378575 learned_addr learned my addr 172.21.15.53:0/2461378575 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:31.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.588+0000 7fad46ffd640 1 --2- 172.21.15.53:0/2461378575 >> v2:172.21.15.53:3300/0 conn(0x7fad480adc70 0x7fad480b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad47fff640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815af10 0x7fad480b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad467fc640 1 -- 172.21.15.53:0/2461378575 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753778702 0 0) 0x7fad4815f7d0 con 0x7fad4815ab30 2023-10-21T16:02:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad467fc640 1 -- 172.21.15.53:0/2461378575 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad30003520 con 0x7fad4815ab30 2023-10-21T16:02:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad46ffd640 1 -- 172.21.15.53:0/2461378575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815af10 msgr2=0x7fad480b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad46ffd640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815af10 0x7fad480b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad46ffd640 1 -- 172.21.15.53:0/2461378575 >> v1:172.21.15.53:6789/0 conn(0x7fad4815ab30 legacy=0x7fad4815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.589+0000 7fad46ffd640 1 -- 172.21.15.53:0/2461378575 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4815f4f0 con 0x7fad480adc70 2023-10-21T16:02:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.590+0000 7fad46ffd640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad480b00c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0097a0 tx=0x7fad3c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945c69f2acf01b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.590+0000 7fad467fc640 1 -- 172.21.15.53:0/2461378575 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c02d040 con 0x7fad480adc70 2023-10-21T16:02:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.590+0000 7fad467fc640 1 -- 172.21.15.53:0/2461378575 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c031810 con 0x7fad480adc70 2023-10-21T16:02:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.590+0000 7fad467fc640 1 -- 172.21.15.53:0/2461378575 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c031b80 con 0x7fad480adc70 2023-10-21T16:02:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 -- 172.21.15.53:0/2461378575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 msgr2=0x7fad480b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad480b00c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0097a0 tx=0x7fad3c02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 -- 172.21.15.53:0/2461378575 shutdown_connections 2023-10-21T16:02:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815af10 0x7fad480b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/2461378575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad480b00c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 -- 172.21.15.53:0/2461378575 >> 172.21.15.53:0/2461378575 conn(0x7fad480cb0c0 msgr2=0x7fad480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 -- 172.21.15.53:0/2461378575 shutdown_connections 2023-10-21T16:02:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.591+0000 7fad4c9d2640 1 -- 172.21.15.53:0/2461378575 wait complete. 2023-10-21T16:02:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.593+0000 7fad4c9d2640 1 Processor -- start 2023-10-21T16:02:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.593+0000 7fad4c9d2640 1 -- start start 2023-10-21T16:02:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815ab30 0x7fad480b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad4815af10 0x7fad480b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad481f2cd0 con 0x7fad480adc70 2023-10-21T16:02:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad480cc530 con 0x7fad4815af10 2023-10-21T16:02:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad4c9d2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4814be10 con 0x7fad4815ab30 2023-10-21T16:02:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59906/0 (socket says 172.21.15.53:59906) 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.594+0000 7fad477fe640 1 -- 172.21.15.53:0/1349376188 learned_addr learned my addr 172.21.15.53:0/1349376188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad46ffd640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815ab30 0x7fad480b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad47fff640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad4815af10 0x7fad480b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 -- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815ab30 msgr2=0x7fad480b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815ab30 0x7fad480b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 -- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad4815af10 msgr2=0x7fad480b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad4815af10 0x7fad480b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 -- 172.21.15.53:0/1349376188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c009400 con 0x7fad480adc70 2023-10-21T16:02:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad477fe640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad38001460 tx=0x7fad3800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800ace0 con 0x7fad480adc70 2023-10-21T16:02:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3800ae60 con 0x7fad480adc70 2023-10-21T16:02:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38012600 con 0x7fad480adc70 2023-10-21T16:02:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.595+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4814c0f0 con 0x7fad480adc70 2023-10-21T16:02:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.596+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad481f2960 con 0x7fad480adc70 2023-10-21T16:02:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.597+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4804e160 con 0x7fad480adc70 2023-10-21T16:02:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.598+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fad38007070 con 0x7fad480adc70 2023-10-21T16:02:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.599+0000 7fad2ffff640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 0x7fad3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.599+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad380899c0 con 0x7fad480adc70 2023-10-21T16:02:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.600+0000 7fad46ffd640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 0x7fad3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.601+0000 7fad46ffd640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 0x7fad3406d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036000 tx=0x7fad3c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.604+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad380546e0 con 0x7fad480adc70 2023-10-21T16:02:31.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.799+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad480ada50 con 0x7fad480adc70 2023-10-21T16:02:31.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.800+0000 7fad2ffff640 1 -- 172.21.15.53:0/1349376188 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fad38017090 con 0x7fad480adc70 2023-10-21T16:02:31.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:31.803 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:31.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 msgr2=0x7fad3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 0x7fad3406d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036000 tx=0x7fad3c03c040 comp rx=0 tx=0).stop 2023-10-21T16:02:31.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 msgr2=0x7fad481f3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:31.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad38001460 tx=0x7fad3800a430 comp rx=0 tx=0).stop 2023-10-21T16:02:31.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 shutdown_connections 2023-10-21T16:02:31.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad4815af10 0x7fad480b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad4815ab30 0x7fad480b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fad3406ae70 0x7fad3406d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 --2- 172.21.15.53:0/1349376188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad480adc70 0x7fad481f3160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:31.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 >> 172.21.15.53:0/1349376188 conn(0x7fad480cb0c0 msgr2=0x7fad480ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:31.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 shutdown_connections 2023-10-21T16:02:31.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:31.803+0000 7fad4c9d2640 1 -- 172.21.15.53:0/1349376188 wait complete. 2023-10-21T16:02:31.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:31.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:32.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.057+0000 7f58caaa9640 1 Processor -- start 2023-10-21T16:02:32.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.057+0000 7f58caaa9640 1 -- start start 2023-10-21T16:02:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58caaa9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f58c415aad0 0x7f58c40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58caaa9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 0x7f58c40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58caaa9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c415bf30 con 0x7f58c406cd00 2023-10-21T16:02:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58caaa9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c415b320 con 0x7f58c415aad0 2023-10-21T16:02:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58caaa9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415b600 con 0x7f58c40c82b0 2023-10-21T16:02:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58c9046640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f58c406cd00 0x7f58c4070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59854/0 (socket says 172.21.15.53:59854) 2023-10-21T16:02:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.059+0000 7f58c9046640 1 -- 172.21.15.53:0/3396773226 learned_addr learned my addr 172.21.15.53:0/3396773226 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c8845640 1 --2- 172.21.15.53:0/3396773226 >> v2:172.21.15.124:3300/0 conn(0x7f58c415aad0 0x7f58c40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c3fff640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 0x7f58c40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102200764 0 0) 0x7f58c415bf30 con 0x7f58c406cd00 2023-10-21T16:02:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ac003520 con 0x7f58c406cd00 2023-10-21T16:02:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980561558 0 0) 0x7f58ac003520 con 0x7f58c406cd00 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 msgr2=0x7f58c40c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 0x7f58c40c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 >> v2:172.21.15.124:3300/0 conn(0x7f58c415aad0 msgr2=0x7f58c40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 --2- 172.21.15.53:0/3396773226 >> v2:172.21.15.124:3300/0 conn(0x7f58c415aad0 0x7f58c40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c415b8e0 con 0x7f58c406cd00 2023-10-21T16:02:32.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f58bc001670 con 0x7f58c406cd00 2023-10-21T16:02:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 >> v1:172.21.15.53:6789/0 conn(0x7f58c406cd00 legacy=0x7f58c4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.060+0000 7f58c37fe640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003970 0x7f58ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c37fe640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 0x7f58ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c37fe640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac004d00 0x7f58ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415b320 con 0x7f58ac003970 2023-10-21T16:02:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415b600 con 0x7f58ac004d00 2023-10-21T16:02:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac00fd90 con 0x7f58c40c82b0 2023-10-21T16:02:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003970 0x7f58ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c9046640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac004d00 0x7f58ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 -- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 msgr2=0x7f58ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c40c82b0 0x7f58ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 -- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac004d00 msgr2=0x7f58ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac004d00 0x7f58ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.061+0000 7f58c8845640 1 -- 172.21.15.53:0/3396773226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac003520 con 0x7f58ac003970 2023-10-21T16:02:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.063+0000 7f58c8845640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003970 0x7f58ac003d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009ac0 tx=0x7f58ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b62565cdc9f3daf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.063+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b402c710 con 0x7f58ac003970 2023-10-21T16:02:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.063+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b402c870 con 0x7f58ac003970 2023-10-21T16:02:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.063+0000 7f58c37fe640 1 -- 172.21.15.53:0/3396773226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b402cc00 con 0x7f58ac003970 2023-10-21T16:02:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 -- 172.21.15.53:0/3396773226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003970 msgr2=0x7f58ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 --2- 172.21.15.53:0/3396773226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58ac003970 0x7f58ac003d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009ac0 tx=0x7f58ac0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58c9046640 1 -- 172.21.15.53:0/3396773226 reap_dead start 2023-10-21T16:02:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 -- 172.21.15.53:0/3396773226 shutdown_connections 2023-10-21T16:02:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 -- 172.21.15.53:0/3396773226 >> 172.21.15.53:0/3396773226 conn(0x7f58c4069d50 msgr2=0x7f58c406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 -- 172.21.15.53:0/3396773226 shutdown_connections 2023-10-21T16:02:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.064+0000 7f58caaa9640 1 -- 172.21.15.53:0/3396773226 wait complete. 2023-10-21T16:02:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.066+0000 7f58caaa9640 1 Processor -- start 2023-10-21T16:02:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.066+0000 7f58caaa9640 1 -- start start 2023-10-21T16:02:32.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 0x7f58c414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 0x7f58c414f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41edfe0 con 0x7f58c415cd40 2023-10-21T16:02:32.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415d700 con 0x7f58ac003970 2023-10-21T16:02:32.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58caaa9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c414fea0 con 0x7f58c4159f80 2023-10-21T16:02:32.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58c9046640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58c9046640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59920/0 (socket says 172.21.15.53:59920) 2023-10-21T16:02:32.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.067+0000 7f58c9046640 1 -- 172.21.15.53:0/1119080760 learned_addr learned my addr 172.21.15.53:0/1119080760 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:32.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c8845640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 0x7f58c414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c3fff640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 0x7f58c414f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 -- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 msgr2=0x7f58c414f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 0x7f58c414f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 -- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 msgr2=0x7f58c414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 0x7f58c414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 -- 172.21.15.53:0/1119080760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c41ee1e0 con 0x7f58c415cd40 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c8845640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 0x7f58c414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c3fff640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 0x7f58c414f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c9046640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58bc000c00 tx=0x7f58bc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.068+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00eed0 con 0x7f58c415cd40 2023-10-21T16:02:32.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.069+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc00c690 con 0x7f58c415cd40 2023-10-21T16:02:32.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.069+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00ca20 con 0x7f58c415cd40 2023-10-21T16:02:32.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.069+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c414fc00 con 0x7f58c415cd40 2023-10-21T16:02:32.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.069+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c41ee480 con 0x7f58c415cd40 2023-10-21T16:02:32.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.070+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c40cf680 con 0x7f58c415cd40 2023-10-21T16:02:32.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.071+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f58bc00cb80 con 0x7f58c415cd40 2023-10-21T16:02:32.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.073+0000 7f58c17fa640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 0x7f58b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.073+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58bc089650 con 0x7f58c415cd40 2023-10-21T16:02:32.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.074+0000 7f58c8845640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 0x7f58b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.075+0000 7f58c8845640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 0x7f58b006d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f58b4036000 tx=0x7f58b4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.079+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58bc057e90 con 0x7f58c415cd40 2023-10-21T16:02:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.279+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58c404e160 con 0x7f58c415cd40 2023-10-21T16:02:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.280+0000 7f58c17fa640 1 -- 172.21.15.53:0/1119080760 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58bc057a20 con 0x7f58c415cd40 2023-10-21T16:02:32.282 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 msgr2=0x7f58b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 0x7f58b006d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f58b4036000 tx=0x7f58b4030000 comp rx=0 tx=0).stop 2023-10-21T16:02:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 msgr2=0x7f58c415d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f58bc000c00 tx=0x7f58bc00eb90 comp rx=0 tx=0).stop 2023-10-21T16:02:32.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 shutdown_connections 2023-10-21T16:02:32.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f58b006ae70 0x7f58b006d330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58c415cd40 0x7f58c415d1c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58c4159f80 0x7f58c414f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 --2- 172.21.15.53:0/1119080760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58ac003970 0x7f58c414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 >> 172.21.15.53:0/1119080760 conn(0x7f58c4069d50 msgr2=0x7f58c40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 shutdown_connections 2023-10-21T16:02:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.283+0000 7f58caaa9640 1 -- 172.21.15.53:0/1119080760 wait complete. 2023-10-21T16:02:32.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:32.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.530+0000 7fe56f0b4640 1 Processor -- start 2023-10-21T16:02:32.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.531+0000 7fe56f0b4640 1 -- start start 2023-10-21T16:02:32.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56f0b4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5681599d0 0x7fe5680cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56f0b4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5680b3c50 0x7fe5680caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56f0b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe568159db0 con 0x7fe5680cb410 2023-10-21T16:02:32.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56f0b4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe568071fe0 con 0x7fe5681599d0 2023-10-21T16:02:32.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56f0b4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680b44c0 con 0x7fe5680b3c50 2023-10-21T16:02:32.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56e8b3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe5680cb410 0x7fe5680c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59860/0 (socket says 172.21.15.53:59860) 2023-10-21T16:02:32.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56e8b3640 1 -- 172.21.15.53:0/702717890 learned_addr learned my addr 172.21.15.53:0/702717890 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:32.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.532+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/702717890 >> v2:172.21.15.53:3300/0 conn(0x7fe5681599d0 0x7fe5680cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.533+0000 7fe56d8b1640 1 --2- 172.21.15.53:0/702717890 >> v2:172.21.15.124:3300/0 conn(0x7fe5680b3c50 0x7fe5680caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.533+0000 7fe56e0b2640 1 -- 172.21.15.53:0/702717890 >> v2:172.21.15.124:3300/0 conn(0x7fe5680b3c50 msgr2=0x7fe5680caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.533+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/702717890 >> v2:172.21.15.124:3300/0 conn(0x7fe5680b3c50 0x7fe5680caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.533+0000 7fe56e0b2640 1 -- 172.21.15.53:0/702717890 >> v1:172.21.15.53:6789/0 conn(0x7fe5680cb410 legacy=0x7fe5680c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.533+0000 7fe56e0b2640 1 -- 172.21.15.53:0/702717890 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680cb860 con 0x7fe5681599d0 2023-10-21T16:02:32.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.534+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/702717890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5681599d0 0x7fe5680cdcd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5680709e0 tx=0x7fe56402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aea1e03f50a0c99 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.534+0000 7fe56d0b0640 1 -- 172.21.15.53:0/702717890 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56402ee80 con 0x7fe5681599d0 2023-10-21T16:02:32.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.534+0000 7fe56d0b0640 1 -- 172.21.15.53:0/702717890 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe564036440 con 0x7fe5681599d0 2023-10-21T16:02:32.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.534+0000 7fe56d0b0640 1 -- 172.21.15.53:0/702717890 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640367b0 con 0x7fe5681599d0 2023-10-21T16:02:32.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 -- 172.21.15.53:0/702717890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5681599d0 msgr2=0x7fe5680cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/702717890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5681599d0 0x7fe5680cdcd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe5680709e0 tx=0x7fe56402e0d0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 -- 172.21.15.53:0/702717890 shutdown_connections 2023-10-21T16:02:32.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/702717890 >> v2:172.21.15.124:3300/0 conn(0x7fe5680b3c50 0x7fe5680caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/702717890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5681599d0 0x7fe5680cdcd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 -- 172.21.15.53:0/702717890 >> 172.21.15.53:0/702717890 conn(0x7fe56806c050 msgr2=0x7fe56806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:32.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 -- 172.21.15.53:0/702717890 shutdown_connections 2023-10-21T16:02:32.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.535+0000 7fe56f0b4640 1 -- 172.21.15.53:0/702717890 wait complete. 2023-10-21T16:02:32.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.537+0000 7fe56f0b4640 1 Processor -- start 2023-10-21T16:02:32.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.537+0000 7fe56f0b4640 1 -- start start 2023-10-21T16:02:32.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5680b3c50 0x7fe5680c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 0x7fe5680af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c83c0 con 0x7fe5680cb410 2023-10-21T16:02:32.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c6780 con 0x7fe5680b3c50 2023-10-21T16:02:32.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56f0b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c6a60 con 0x7fe5681599d0 2023-10-21T16:02:32.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56d8b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56d8b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59940/0 (socket says 172.21.15.53:59940) 2023-10-21T16:02:32.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56d8b1640 1 -- 172.21.15.53:0/374635557 learned_addr learned my addr 172.21.15.53:0/374635557 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:32.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5680b3c50 0x7fe5680c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.538+0000 7fe56e8b3640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 0x7fe5680af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 -- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 msgr2=0x7fe5680af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 0x7fe5680af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 -- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5680b3c50 msgr2=0x7fe5680c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5680b3c50 0x7fe5680c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 -- 172.21.15.53:0/374635557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe564009a70 con 0x7fe5680cb410 2023-10-21T16:02:32.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56e8b3640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 0x7fe5680af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:32.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56d8b1640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d800 tx=0x7fe55800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800b6c0 con 0x7fe5680cb410 2023-10-21T16:02:32.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800b820 con 0x7fe5680cb410 2023-10-21T16:02:32.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800bbb0 con 0x7fe5680cb410 2023-10-21T16:02:32.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5680c6d40 con 0x7fe5680cb410 2023-10-21T16:02:32.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.539+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5680c7300 con 0x7fe5680cb410 2023-10-21T16:02:32.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.541+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5680c74c0 con 0x7fe5680cb410 2023-10-21T16:02:32.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.541+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe55800bd10 con 0x7fe5680cb410 2023-10-21T16:02:32.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.542+0000 7fe55effd640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 0x7fe54c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.543+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe558089f40 con 0x7fe5680cb410 2023-10-21T16:02:32.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.543+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 0x7fe54c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.544+0000 7fe56e0b2640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 0x7fe54c06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe564038000 tx=0x7fe564032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.546+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe558054c60 con 0x7fe5680cb410 2023-10-21T16:02:32.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.739+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe568074150 con 0x7fe5680cb410 2023-10-21T16:02:32.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.740+0000 7fe55effd640 1 -- 172.21.15.53:0/374635557 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe558058280 con 0x7fe5680cb410 2023-10-21T16:02:32.742 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:32.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 msgr2=0x7fe54c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 0x7fe54c06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe564038000 tx=0x7fe564032000 comp rx=0 tx=0).stop 2023-10-21T16:02:32.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 msgr2=0x7fe5680af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d800 tx=0x7fe55800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 shutdown_connections 2023-10-21T16:02:32.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5681599d0 0x7fe5680af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe54c06ae70 0x7fe54c06d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5680cb410 0x7fe5680af210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 --2- 172.21.15.53:0/374635557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5680b3c50 0x7fe5680c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 >> 172.21.15.53:0/374635557 conn(0x7fe56806c050 msgr2=0x7fe56815ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:32.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 shutdown_connections 2023-10-21T16:02:32.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.743+0000 7fe56f0b4640 1 -- 172.21.15.53:0/374635557 wait complete. 2023-10-21T16:02:32.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:32.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.981+0000 7f175e193640 1 Processor -- start 2023-10-21T16:02:32.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.982+0000 7f175e193640 1 -- start start 2023-10-21T16:02:32.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.983+0000 7f175e193640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17580cf2b0 0x7f17580d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.983+0000 7f175e193640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.983+0000 7f175e193640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cd520 con 0x7f17580d1100 2023-10-21T16:02:32.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175e193640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580d1910 con 0x7f17580cf2b0 2023-10-21T16:02:32.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175e193640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580ce080 con 0x7f17580cf690 2023-10-21T16:02:32.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d191640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17580cf2b0 0x7f17580d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d992640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f17580d1100 0x7f175815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59872/0 (socket says 172.21.15.53:59872) 2023-10-21T16:02:32.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d992640 1 -- 172.21.15.53:0/471902074 learned_addr learned my addr 172.21.15.53:0/471902074 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:32.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175c990640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d191640 1 -- 172.21.15.53:0/471902074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 msgr2=0x7f17580ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:32.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d191640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f175d191640 1 -- 172.21.15.53:0/471902074 >> v1:172.21.15.53:6789/0 conn(0x7f17580d1100 legacy=0x7f175815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.985+0000 7f175d191640 1 -- 172.21.15.53:0/471902074 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580cd750 con 0x7f17580cf2b0 2023-10-21T16:02:32.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.984+0000 7f174ffff640 1 -- 172.21.15.53:0/471902074 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571595700 0 0) 0x7f17580cd520 con 0x7f17580d1100 2023-10-21T16:02:32.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.986+0000 7f175d191640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580cf2b0 0x7f17580d34d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17540097a0 tx=0x7f175402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183996a878be20b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.986+0000 7f174ffff640 1 -- 172.21.15.53:0/471902074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754031780 con 0x7f17580cf2b0 2023-10-21T16:02:32.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.986+0000 7f174ffff640 1 -- 172.21.15.53:0/471902074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17540318e0 con 0x7f17580cf2b0 2023-10-21T16:02:32.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.986+0000 7f174ffff640 1 -- 172.21.15.53:0/471902074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754031c70 con 0x7f17580cf2b0 2023-10-21T16:02:32.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 -- 172.21.15.53:0/471902074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580cf2b0 msgr2=0x7f17580d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580cf2b0 0x7f17580d34d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17540097a0 tx=0x7f175402b820 comp rx=0 tx=0).stop 2023-10-21T16:02:32.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 -- 172.21.15.53:0/471902074 shutdown_connections 2023-10-21T16:02:32.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 --2- 172.21.15.53:0/471902074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580cf2b0 0x7f17580d34d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 -- 172.21.15.53:0/471902074 >> 172.21.15.53:0/471902074 conn(0x7f175809bc40 msgr2=0x7f17580d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:32.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 -- 172.21.15.53:0/471902074 shutdown_connections 2023-10-21T16:02:32.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.987+0000 7f175e193640 1 -- 172.21.15.53:0/471902074 wait complete. 2023-10-21T16:02:32.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.989+0000 7f175e193640 1 Processor -- start 2023-10-21T16:02:32.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.989+0000 7f175e193640 1 -- start start 2023-10-21T16:02:32.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17580cf2b0 0x7f1758155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:32.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175815c5f0 con 0x7f17580d1100 2023-10-21T16:02:32.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580ccb90 con 0x7f17580cf2b0 2023-10-21T16:02:32.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175e193640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cce70 con 0x7f17580cf690 2023-10-21T16:02:32.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175d992640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175d992640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59954/0 (socket says 172.21.15.53:59954) 2023-10-21T16:02:32.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.990+0000 7f175d992640 1 -- 172.21.15.53:0/2342474958 learned_addr learned my addr 172.21.15.53:0/2342474958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:32.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175c990640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d191640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17580cf2b0 0x7f1758155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:32.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 -- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 msgr2=0x7f17580cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 -- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17580cf2b0 msgr2=0x7f1758155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:32.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17580cf2b0 0x7f1758155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 -- 172.21.15.53:0/2342474958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1754009400 con 0x7f17580d1100 2023-10-21T16:02:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175d992640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f175000c7e0 tx=0x7f175000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000a640 con 0x7f17580d1100 2023-10-21T16:02:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175000a7c0 con 0x7f17580d1100 2023-10-21T16:02:32.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175815c8b0 con 0x7f17580d1100 2023-10-21T16:02:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.991+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000ab70 con 0x7f17580d1100 2023-10-21T16:02:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.992+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17581556a0 con 0x7f17580d1100 2023-10-21T16:02:32.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.993+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175815cdd0 con 0x7f17580d1100 2023-10-21T16:02:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.994+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f175000acd0 con 0x7f17580d1100 2023-10-21T16:02:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.994+0000 7f174dffb640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 0x7f173c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.995+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1750088fc0 con 0x7f17580d1100 2023-10-21T16:02:33.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.995+0000 7f175d191640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 0x7f173c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.996+0000 7f175d191640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 0x7f173c06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1754036000 tx=0x7f175403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:32.999+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f175815cdd0 con 0x7f17580d1100 2023-10-21T16:02:33.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.197+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f175815cdd0 con 0x7f17580d1100 2023-10-21T16:02:33.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.197+0000 7f174dffb640 1 -- 172.21.15.53:0/2342474958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f175815cdd0 con 0x7f17580d1100 2023-10-21T16:02:33.199 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:33.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 msgr2=0x7f173c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 0x7f173c06d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1754036000 tx=0x7f175403d040 comp rx=0 tx=0).stop 2023-10-21T16:02:33.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 msgr2=0x7f175815d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f175000c7e0 tx=0x7f175000cca0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 shutdown_connections 2023-10-21T16:02:33.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f173c06ae70 0x7f173c06d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17580d1100 0x7f175815d0b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17580cf690 0x7f17580cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 --2- 172.21.15.53:0/2342474958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17580cf2b0 0x7f1758155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 >> 172.21.15.53:0/2342474958 conn(0x7f175809bc40 msgr2=0x7f17580d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:33.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 shutdown_connections 2023-10-21T16:02:33.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.200+0000 7f175e193640 1 -- 172.21.15.53:0/2342474958 wait complete. 2023-10-21T16:02:33.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.449+0000 7f72cc8af640 1 Processor -- start 2023-10-21T16:02:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.449+0000 7f72cc8af640 1 -- start start 2023-10-21T16:02:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72cc8af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72c80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72cc8af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c815b310 con 0x7f72c815aa00 2023-10-21T16:02:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72cc8af640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c806d290 con 0x7f72c806db60 2023-10-21T16:02:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72cc8af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80ca230 con 0x7f72c806c3a0 2023-10-21T16:02:33.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72c6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f72c815aa00 0x7f72c80b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59878/0 (socket says 172.21.15.53:59878) 2023-10-21T16:02:33.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72c6d9d640 1 -- 172.21.15.53:0/2114791808 learned_addr learned my addr 172.21.15.53:0/2114791808 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:33.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.451+0000 7f72c659c640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72c80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679292040 0 0) 0x7f72c0001110 con 0x7f72c815aa00 2023-10-21T16:02:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b4003520 con 0x7f72c815aa00 2023-10-21T16:02:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2117949066 0 0) 0x7f72b4003520 con 0x7f72c815aa00 2023-10-21T16:02:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 msgr2=0x7f72c80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72c80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 >> v1:172.21.15.124:6789/0 conn(0x7f72c806db60 legacy=0x7f72c806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80c9860 con 0x7f72c815aa00 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f72c0001800 con 0x7f72c815aa00 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 >> v1:172.21.15.53:6789/0 conn(0x7f72c815aa00 legacy=0x7f72c80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40039a0 0x7f72b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b40047c0 0x7f72b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b400ba30 con 0x7f72b40047c0 2023-10-21T16:02:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b400bba0 con 0x7f72b40039a0 2023-10-21T16:02:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.452+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b400bd60 con 0x7f72c806c3a0 2023-10-21T16:02:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b40047c0 0x7f72b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c659c640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40039a0 0x7f72b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c6d9d640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 msgr2=0x7f72b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c806c3a0 0x7f72b4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40039a0 msgr2=0x7f72b4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40039a0 0x7f72b4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2114791808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4003520 con 0x7f72b40047c0 2023-10-21T16:02:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.453+0000 7f72c659c640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40039a0 0x7f72b4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.454+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b40047c0 0x7f72b4004c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001720 tx=0x7f72b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27c386468c722f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.454+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00052d0 con 0x7f72b40047c0 2023-10-21T16:02:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.454+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b002e050 con 0x7f72b40047c0 2023-10-21T16:02:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.454+0000 7f72c559a640 1 -- 172.21.15.53:0/2114791808 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00035f0 con 0x7f72b40047c0 2023-10-21T16:02:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 -- 172.21.15.53:0/2114791808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b40047c0 msgr2=0x7f72b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2114791808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72b40047c0 0x7f72b4004c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f72b0001720 tx=0x7f72b40057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72c6d9d640 1 -- 172.21.15.53:0/2114791808 reap_dead start 2023-10-21T16:02:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 -- 172.21.15.53:0/2114791808 shutdown_connections 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 -- 172.21.15.53:0/2114791808 >> 172.21.15.53:0/2114791808 conn(0x7f72c806a5d0 msgr2=0x7f72c80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 -- 172.21.15.53:0/2114791808 shutdown_connections 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.455+0000 7f72cc8af640 1 -- 172.21.15.53:0/2114791808 wait complete. 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.457+0000 7f72cc8af640 1 Processor -- start 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.457+0000 7f72cc8af640 1 -- start start 2023-10-21T16:02:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40047c0 0x7f72c80c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c80b3450 0x7f72c80c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80ae6c0 con 0x7f72c80b2a90 2023-10-21T16:02:33.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80b3900 con 0x7f72b40047c0 2023-10-21T16:02:33.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.458+0000 7f72cc8af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80c3060 con 0x7f72c80b3450 2023-10-21T16:02:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59974/0 (socket says 172.21.15.53:59974) 2023-10-21T16:02:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2035913732 learned_addr learned my addr 172.21.15.53:0/2035913732 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c80b3450 msgr2=0x7f72c80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c80b3450 0x7f72c80c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40047c0 msgr2=0x7f72c80c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40047c0 0x7f72c80c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 -- 172.21.15.53:0/2035913732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80c3340 con 0x7f72c80b2a90 2023-10-21T16:02:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.459+0000 7f72c5d9b640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72b0008fd0 tx=0x7f72b0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.460+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00076c0 con 0x7f72c80b2a90 2023-10-21T16:02:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.460+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b00069e0 con 0x7f72c80b2a90 2023-10-21T16:02:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.460+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0006d50 con 0x7f72c80b2a90 2023-10-21T16:02:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.460+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c80c3620 con 0x7f72c80b2a90 2023-10-21T16:02:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.460+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c80c5af0 con 0x7f72c80b2a90 2023-10-21T16:02:33.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.461+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c80c5c80 con 0x7f72c80b2a90 2023-10-21T16:02:33.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.462+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f72b00035f0 con 0x7f72c80b2a90 2023-10-21T16:02:33.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.463+0000 7f72beffd640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 0x7f72a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.463+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f72b00ab6f0 con 0x7f72c80b2a90 2023-10-21T16:02:33.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.464+0000 7f72c659c640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 0x7f72a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.465+0000 7f72c659c640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 0x7f72a806d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f72c80b19c0 tx=0x7f72b8009750 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.467+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72b0076530 con 0x7f72c80b2a90 2023-10-21T16:02:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.659+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f72c804e160 con 0x7f72c80b2a90 2023-10-21T16:02:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.660+0000 7f72beffd640 1 -- 172.21.15.53:0/2035913732 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72b0079b50 con 0x7f72c80b2a90 2023-10-21T16:02:33.663 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 msgr2=0x7f72a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 0x7f72a806d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f72c80b19c0 tx=0x7f72b8009750 comp rx=0 tx=0).stop 2023-10-21T16:02:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 msgr2=0x7f72c80b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72b0008fd0 tx=0x7f72b0001120 comp rx=0 tx=0).stop 2023-10-21T16:02:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 shutdown_connections 2023-10-21T16:02:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72c80b3450 0x7f72c80c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f72a806aea0 0x7f72a806d360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c80b2a90 0x7f72c80b2f10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 --2- 172.21.15.53:0/2035913732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72b40047c0 0x7f72c80c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 >> 172.21.15.53:0/2035913732 conn(0x7f72c806a5d0 msgr2=0x7f72c80c7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:33.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 shutdown_connections 2023-10-21T16:02:33.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.663+0000 7f72cc8af640 1 -- 172.21.15.53:0/2035913732 wait complete. 2023-10-21T16:02:33.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:33.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.911+0000 7efdc58bb640 1 Processor -- start 2023-10-21T16:02:33.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.912+0000 7efdc58bb640 1 -- start start 2023-10-21T16:02:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdc58bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc0162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdc58bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efdc00cb150 0x7efdc00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdc58bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc00ae140 con 0x7efdc00add60 2023-10-21T16:02:33.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdc58bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc0161f10 con 0x7efdc00cb150 2023-10-21T16:02:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdc58bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00cbb20 con 0x7efdc006d4c0 2023-10-21T16:02:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdbf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efdc00cb150 0x7efdc00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdbf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efdc00cb150 0x7efdc00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59988/0 (socket says 172.21.15.53:59988) 2023-10-21T16:02:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.914+0000 7efdbf7fe640 1 -- 172.21.15.53:0/528217569 learned_addr learned my addr 172.21.15.53:0/528217569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:33.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.915+0000 7efdbe7fc640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc0162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.915+0000 7efdbf7fe640 1 -- 172.21.15.53:0/528217569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 msgr2=0x7efdc0162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:33.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.915+0000 7efdbf7fe640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc0162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.915+0000 7efdbf7fe640 1 -- 172.21.15.53:0/528217569 >> v1:172.21.15.124:6789/0 conn(0x7efdc00add60 legacy=0x7efdc00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.915+0000 7efdbf7fe640 1 -- 172.21.15.53:0/528217569 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00ccf20 con 0x7efdc00cb150 2023-10-21T16:02:33.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.916+0000 7efdbf7fe640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00cb150 0x7efdc00cb5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5c48de1ff96a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.916+0000 7efdbdffb640 1 -- 172.21.15.53:0/528217569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8031640 con 0x7efdc00cb150 2023-10-21T16:02:33.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.916+0000 7efdbdffb640 1 -- 172.21.15.53:0/528217569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb80317a0 con 0x7efdc00cb150 2023-10-21T16:02:33.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.916+0000 7efdbdffb640 1 -- 172.21.15.53:0/528217569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8031b30 con 0x7efdc00cb150 2023-10-21T16:02:33.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 -- 172.21.15.53:0/528217569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00cb150 msgr2=0x7efdc00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00cb150 0x7efdc00cb5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009870 tx=0x7efdb802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 -- 172.21.15.53:0/528217569 shutdown_connections 2023-10-21T16:02:33.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc0162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 --2- 172.21.15.53:0/528217569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00cb150 0x7efdc00cb5e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.917+0000 7efdc58bb640 1 -- 172.21.15.53:0/528217569 >> 172.21.15.53:0/528217569 conn(0x7efdc006a380 msgr2=0x7efdc00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:33.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.918+0000 7efdc58bb640 1 -- 172.21.15.53:0/528217569 shutdown_connections 2023-10-21T16:02:33.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.918+0000 7efdc58bb640 1 -- 172.21.15.53:0/528217569 wait complete. 2023-10-21T16:02:33.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.919+0000 7efdc58bb640 1 Processor -- start 2023-10-21T16:02:33.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 -- start start 2023-10-21T16:02:33.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc00b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 0x7efdc0151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00b21c0 con 0x7efdc00add60 2023-10-21T16:02:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00b15b0 con 0x7efdc00cb150 2023-10-21T16:02:33.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.920+0000 7efdc58bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc00b1890 con 0x7efdc006d4c0 2023-10-21T16:02:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59992/0 (socket says 172.21.15.53:59992) 2023-10-21T16:02:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 -- 172.21.15.53:0/4084753453 learned_addr learned my addr 172.21.15.53:0/4084753453 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbf7fe640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 0x7efdc0151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbeffd640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc00b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 -- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 msgr2=0x7efdc00b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc00b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 -- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 msgr2=0x7efdc0151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 0x7efdc0151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbe7fc640 1 -- 172.21.15.53:0/4084753453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80094d0 con 0x7efdc00add60 2023-10-21T16:02:33.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbf7fe640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 0x7efdc0151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.921+0000 7efdbeffd640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc00b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdbe7fc640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efda800d800 tx=0x7efda800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800b6c0 con 0x7efdc00add60 2023-10-21T16:02:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda800b820 con 0x7efdc00add60 2023-10-21T16:02:33.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda800bbb0 con 0x7efdc00add60 2023-10-21T16:02:33.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc00b1b70 con 0x7efdc00add60 2023-10-21T16:02:33.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.922+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc00b2b90 con 0x7efdc00add60 2023-10-21T16:02:33.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.923+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc00b2330 con 0x7efdc00add60 2023-10-21T16:02:33.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.924+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efda800bd10 con 0x7efdc00add60 2023-10-21T16:02:33.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.925+0000 7efdb7fff640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 0x7efda006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:33.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.925+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efda8089e10 con 0x7efdc00add60 2023-10-21T16:02:33.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.926+0000 7efdbeffd640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 0x7efda006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:33.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.927+0000 7efdbeffd640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 0x7efda006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efdb00012e0 tx=0x7efdb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:33.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:33.930+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efda8054c60 con 0x7efdc00add60 2023-10-21T16:02:34.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.163+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdc0067490 con 0x7efdc00add60 2023-10-21T16:02:34.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.164+0000 7efdb7fff640 1 -- 172.21.15.53:0/4084753453 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7efda8058280 con 0x7efdc00add60 2023-10-21T16:02:34.166 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:34.166 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:02:34.167 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:02:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 msgr2=0x7efda006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 0x7efda006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efdb00012e0 tx=0x7efdb0009040 comp rx=0 tx=0).stop 2023-10-21T16:02:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 msgr2=0x7efdc00b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efda800d800 tx=0x7efda800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 shutdown_connections 2023-10-21T16:02:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdc00cb150 0x7efdc0151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efda006ae70 0x7efda006d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdc00add60 0x7efdc00b0370 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 --2- 172.21.15.53:0/4084753453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdc006d4c0 0x7efdc00b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 >> 172.21.15.53:0/4084753453 conn(0x7efdc006a380 msgr2=0x7efdc006c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 shutdown_connections 2023-10-21T16:02:34.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.167+0000 7efdc58bb640 1 -- 172.21.15.53:0/4084753453 wait complete. 2023-10-21T16:02:34.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:34.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.401+0000 7f4920864640 1 Processor -- start 2023-10-21T16:02:34.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.401+0000 7f4920864640 1 -- start start 2023-10-21T16:02:34.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.402+0000 7f4920864640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 0x7f491c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.402+0000 7f4920864640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f4920864640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c15f7d0 con 0x7f491c0adc70 2023-10-21T16:02:34.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f4920864640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491c15f210 con 0x7f491c0b07d0 2023-10-21T16:02:34.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f4920864640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c15f4f0 con 0x7f491c15ab30 2023-10-21T16:02:34.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f491bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f491b7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 0x7f491c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f491bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:52846/0 (socket says 172.21.15.53:52846) 2023-10-21T16:02:34.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.403+0000 7f491bfff640 1 -- 172.21.15.53:0/2042948250 learned_addr learned my addr 172.21.15.53:0/2042948250 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:34.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.404+0000 7f491b7fe640 1 -- 172.21.15.53:0/2042948250 >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 msgr2=0x7f491c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:34.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.404+0000 7f491b7fe640 1 --2- 172.21.15.53:0/2042948250 >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.404+0000 7f491b7fe640 1 -- 172.21.15.53:0/2042948250 >> v1:172.21.15.124:6789/0 conn(0x7f491c0adc70 legacy=0x7f491c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.404+0000 7f491b7fe640 1 -- 172.21.15.53:0/2042948250 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c06c550 con 0x7f491c15ab30 2023-10-21T16:02:34.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.404+0000 7f491bfff640 1 --2- 172.21.15.53:0/2042948250 >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.405+0000 7f491b7fe640 1 --2- 172.21.15.53:0/2042948250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 0x7f491c15e8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097a0 tx=0x7f490c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85eb75ee339fc7b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.406+0000 7f491a7fc640 1 -- 172.21.15.53:0/2042948250 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c02d040 con 0x7f491c15ab30 2023-10-21T16:02:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.406+0000 7f491a7fc640 1 -- 172.21.15.53:0/2042948250 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c031810 con 0x7f491c15ab30 2023-10-21T16:02:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.406+0000 7f491a7fc640 1 -- 172.21.15.53:0/2042948250 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c031ba0 con 0x7f491c15ab30 2023-10-21T16:02:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.406+0000 7f4920864640 1 -- 172.21.15.53:0/2042948250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 msgr2=0x7f491c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.406+0000 7f4920864640 1 --2- 172.21.15.53:0/2042948250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 0x7f491c15e8a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097a0 tx=0x7f490c02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 -- 172.21.15.53:0/2042948250 shutdown_connections 2023-10-21T16:02:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 --2- 172.21.15.53:0/2042948250 >> v2:172.21.15.124:3300/0 conn(0x7f491c0b07d0 0x7f491c06cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 --2- 172.21.15.53:0/2042948250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c15ab30 0x7f491c15e8a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 -- 172.21.15.53:0/2042948250 >> 172.21.15.53:0/2042948250 conn(0x7f491c0cb0c0 msgr2=0x7f491c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 -- 172.21.15.53:0/2042948250 shutdown_connections 2023-10-21T16:02:34.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.407+0000 7f4920864640 1 -- 172.21.15.53:0/2042948250 wait complete. 2023-10-21T16:02:34.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.408+0000 7f4920864640 1 Processor -- start 2023-10-21T16:02:34.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.409+0000 7f4920864640 1 -- start start 2023-10-21T16:02:34.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.409+0000 7f4920864640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.409+0000 7f4920864640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c0b07d0 0x7f491c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f4920864640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f491c15ab30 0x7f491c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f4920864640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0c2780 con 0x7f491c0adc70 2023-10-21T16:02:34.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f4920864640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c15d920 con 0x7f491c15ab30 2023-10-21T16:02:34.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f4920864640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c15dc00 con 0x7f491c0b07d0 2023-10-21T16:02:34.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55194/0 (socket says 172.21.15.53:55194) 2023-10-21T16:02:34.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 -- 172.21.15.53:0/3011230196 learned_addr learned my addr 172.21.15.53:0/3011230196 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:34.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491bfff640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f491c15ab30 0x7f491c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491affd640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c0b07d0 0x7f491c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 -- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c0b07d0 msgr2=0x7f491c15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c0b07d0 0x7f491c15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 -- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f491c15ab30 msgr2=0x7f491c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f491c15ab30 0x7f491c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.410+0000 7f491b7fe640 1 -- 172.21.15.53:0/3011230196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4914009b30 con 0x7f491c0adc70 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f491b7fe640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f490c000c00 tx=0x7f490c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c006e10 con 0x7f491c0adc70 2023-10-21T16:02:34.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c03ad60 con 0x7f491c0adc70 2023-10-21T16:02:34.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c044630 con 0x7f491c0adc70 2023-10-21T16:02:34.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c009400 con 0x7f491c0adc70 2023-10-21T16:02:34.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.411+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c0c2410 con 0x7f491c0adc70 2023-10-21T16:02:34.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.413+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c04e160 con 0x7f491c0adc70 2023-10-21T16:02:34.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.413+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f490c04d450 con 0x7f491c0adc70 2023-10-21T16:02:34.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.414+0000 7f4903fff640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 0x7f490806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.414+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f490c0abb10 con 0x7f491c0adc70 2023-10-21T16:02:34.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.415+0000 7f491affd640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 0x7f490806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.416+0000 7f491affd640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 0x7f490806d3b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f491c1f6080 tx=0x7f4910009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.419+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f490c0768b0 con 0x7f491c0adc70 2023-10-21T16:02:34.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.613+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f491c0b0f70 con 0x7f491c0adc70 2023-10-21T16:02:34.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.615+0000 7f4903fff640 1 -- 172.21.15.53:0/3011230196 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f490c079ed0 con 0x7f491c0adc70 2023-10-21T16:02:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:34.617 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:34.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 msgr2=0x7f490806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 0x7f490806d3b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f491c1f6080 tx=0x7f4910009040 comp rx=0 tx=0).stop 2023-10-21T16:02:34.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 msgr2=0x7f491c154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f490c000c00 tx=0x7f490c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 shutdown_connections 2023-10-21T16:02:34.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f491c15ab30 0x7f491c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f491c0b07d0 0x7f491c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f490806aef0 0x7f490806d3b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 --2- 172.21.15.53:0/3011230196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f491c0adc70 0x7f491c154a10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 >> 172.21.15.53:0/3011230196 conn(0x7f491c0cb0c0 msgr2=0x7f491c0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.618+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 shutdown_connections 2023-10-21T16:02:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.619+0000 7f4920864640 1 -- 172.21.15.53:0/3011230196 wait complete. 2023-10-21T16:02:34.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:34.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.871+0000 7f49a1d2e640 1 Processor -- start 2023-10-21T16:02:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.872+0000 7f49a1d2e640 1 -- start start 2023-10-21T16:02:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.873+0000 7f49a1d2e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f499c15aab0 0x7f499c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.873+0000 7f49a1d2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.873+0000 7f49a1d2e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c15ae90 con 0x7f499c0caef0 2023-10-21T16:02:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.873+0000 7f49a1d2e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0cbdb0 con 0x7f499c15aab0 2023-10-21T16:02:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.873+0000 7f49a1d2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0cb7f0 con 0x7f499c0b0670 2023-10-21T16:02:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f4993fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f499c0caef0 0x7f499c0aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36002/0 (socket says 172.21.15.53:36002) 2023-10-21T16:02:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f4993fff640 1 -- 172.21.15.53:0/4281038710 learned_addr learned my addr 172.21.15.53:0/4281038710 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/4281038710 >> v2:172.21.15.124:3300/0 conn(0x7f499c15aab0 0x7f499c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f49a152d640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f49937fe640 1 -- 172.21.15.53:0/4281038710 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285175926 0 0) 0x7f498c001110 con 0x7f499c0caef0 2023-10-21T16:02:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.874+0000 7f49937fe640 1 -- 172.21.15.53:0/4281038710 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4984003520 con 0x7f499c0caef0 2023-10-21T16:02:34.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.875+0000 7f49a0d2c640 1 -- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 msgr2=0x7f499c0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:34.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.875+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.875+0000 7f49a0d2c640 1 -- 172.21.15.53:0/4281038710 >> v1:172.21.15.53:6789/0 conn(0x7f499c0caef0 legacy=0x7f499c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.875+0000 7f49a0d2c640 1 -- 172.21.15.53:0/4281038710 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0cbad0 con 0x7f499c15aab0 2023-10-21T16:02:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.876+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c15aab0 0x7f499c070b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4998009dc0 tx=0x7f499802bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47fa2e12d3e61946 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.876+0000 7f49937fe640 1 -- 172.21.15.53:0/4281038710 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499802cba0 con 0x7f499c15aab0 2023-10-21T16:02:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.876+0000 7f49937fe640 1 -- 172.21.15.53:0/4281038710 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499802cd00 con 0x7f499c15aab0 2023-10-21T16:02:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.876+0000 7f49937fe640 1 -- 172.21.15.53:0/4281038710 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998034640 con 0x7f499c15aab0 2023-10-21T16:02:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 -- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c15aab0 msgr2=0x7f499c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c15aab0 0x7f499c070b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4998009dc0 tx=0x7f499802bdf0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 -- 172.21.15.53:0/4281038710 shutdown_connections 2023-10-21T16:02:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/4281038710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c15aab0 0x7f499c070b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 -- 172.21.15.53:0/4281038710 >> 172.21.15.53:0/4281038710 conn(0x7f499c06c010 msgr2=0x7f499c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.877+0000 7f49a1d2e640 1 -- 172.21.15.53:0/4281038710 shutdown_connections 2023-10-21T16:02:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.878+0000 7f49a1d2e640 1 -- 172.21.15.53:0/4281038710 wait complete. 2023-10-21T16:02:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.880+0000 7f49a1d2e640 1 Processor -- start 2023-10-21T16:02:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.880+0000 7f49a1d2e640 1 -- start start 2023-10-21T16:02:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 0x7f499c1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b27b0 con 0x7f499c15aab0 2023-10-21T16:02:34.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1590f0 con 0x7f499c0caef0 2023-10-21T16:02:34.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a1d2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c6990 con 0x7f499c0b0670 2023-10-21T16:02:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a152d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a152d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55204/0 (socket says 172.21.15.53:55204) 2023-10-21T16:02:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.881+0000 7f49a152d640 1 -- 172.21.15.53:0/998321314 learned_addr learned my addr 172.21.15.53:0/998321314 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f4993fff640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 0x7f499c1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 -- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 msgr2=0x7f499c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 -- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 msgr2=0x7f499c1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:34.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 0x7f499c1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:34.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 -- 172.21.15.53:0/998321314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998009a70 con 0x7f499c15aab0 2023-10-21T16:02:34.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f4993fff640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 0x7f499c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:34.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:34.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.882+0000 7f49a152d640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f499400e7f0 tx=0x7f499400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.883+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994009510 con 0x7f499c15aab0 2023-10-21T16:02:34.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.883+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499400ee40 con 0x7f499c15aab0 2023-10-21T16:02:34.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.883+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400cb80 con 0x7f499c15aab0 2023-10-21T16:02:34.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.883+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c0c6c30 con 0x7f499c15aab0 2023-10-21T16:02:34.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.883+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c0c71f0 con 0x7f499c15aab0 2023-10-21T16:02:34.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.884+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c0c7410 con 0x7f499c15aab0 2023-10-21T16:02:34.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.885+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f499400cce0 con 0x7f499c15aab0 2023-10-21T16:02:34.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.886+0000 7f49917fa640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 0x7f498806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:34.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.886+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4994089a50 con 0x7f499c15aab0 2023-10-21T16:02:34.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.887+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 0x7f498806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:34.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.888+0000 7f49a0d2c640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 0x7f498806d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4998036000 tx=0x7f4998030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:34.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:34.890+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4994054770 con 0x7f499c15aab0 2023-10-21T16:02:35.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.085+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499c074150 con 0x7f499c15aab0 2023-10-21T16:02:35.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.086+0000 7f49917fa640 1 -- 172.21.15.53:0/998321314 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49940170a0 con 0x7f499c15aab0 2023-10-21T16:02:35.088 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.088+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 msgr2=0x7f498806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.088+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 0x7f498806d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4998036000 tx=0x7f4998030000 comp rx=0 tx=0).stop 2023-10-21T16:02:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.088+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 msgr2=0x7f499c15bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.088+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f499400e7f0 tx=0x7f499400ebf0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 shutdown_connections 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f498806ae70 0x7f498806d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f499c15aab0 0x7f499c15bb80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f499c0caef0 0x7f499c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 --2- 172.21.15.53:0/998321314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f499c0b0670 0x7f499c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 >> 172.21.15.53:0/998321314 conn(0x7f499c06c010 msgr2=0x7f499c0acc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:35.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 shutdown_connections 2023-10-21T16:02:35.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.089+0000 7f49a1d2e640 1 -- 172.21.15.53:0/998321314 wait complete. 2023-10-21T16:02:35.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:35.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.330+0000 7f6f2dfb3640 1 Processor -- start 2023-10-21T16:02:35.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.330+0000 7f6f2dfb3640 1 -- start start 2023-10-21T16:02:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2dfb3640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f2815cf30 0x7f6f280d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2dfb3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280cd520 con 0x7f6f280cf2b0 2023-10-21T16:02:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2dfb3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280d1910 con 0x7f6f280cf690 2023-10-21T16:02:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2dfb3640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280ce080 con 0x7f6f2815cf30 2023-10-21T16:02:35.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2cfb1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6f280cf2b0 0x7f6f280d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36018/0 (socket says 172.21.15.53:36018) 2023-10-21T16:02:35.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2cfb1640 1 -- 172.21.15.53:0/3506065740 learned_addr learned my addr 172.21.15.53:0/3506065740 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:35.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f2d7b2640 1 --2- 172.21.15.53:0/3506065740 >> v2:172.21.15.124:3300/0 conn(0x7f6f2815cf30 0x7f6f280d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.332+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264265457 0 0) 0x7f6f280cd520 con 0x7f6f280cf2b0 2023-10-21T16:02:35.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f10003520 con 0x7f6f280cf2b0 2023-10-21T16:02:35.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428635228 0 0) 0x7f6f10003520 con 0x7f6f280cf2b0 2023-10-21T16:02:35.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 >> v2:172.21.15.124:3300/0 conn(0x7f6f2815cf30 msgr2=0x7f6f280d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 --2- 172.21.15.53:0/3506065740 >> v2:172.21.15.124:3300/0 conn(0x7f6f2815cf30 0x7f6f280d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 >> v1:172.21.15.124:6789/0 conn(0x7f6f280cf690 legacy=0x7f6f2815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280cd750 con 0x7f6f280cf2b0 2023-10-21T16:02:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6f180019f0 con 0x7f6f280cf2b0 2023-10-21T16:02:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 >> v1:172.21.15.53:6789/0 conn(0x7f6f280cf2b0 legacy=0x7f6f280d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f10003970 0x7f6f10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 0x7f6f10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f10005180 0x7f6f1000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280ce080 con 0x7f6f100047c0 2023-10-21T16:02:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1000c120 con 0x7f6f10005180 2023-10-21T16:02:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1000c2e0 con 0x7f6f10003970 2023-10-21T16:02:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.333+0000 7f6f27fff640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 0x7f6f10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f10003970 0x7f6f10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f27fff640 1 -- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f10003970 msgr2=0x7f6f10003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f2d7b2640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f10005180 0x7f6f1000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f27fff640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f10003970 0x7f6f10003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f27fff640 1 -- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f10005180 msgr2=0x7f6f1000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f27fff640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f10005180 0x7f6f1000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f2d7b2640 1 -- 172.21.15.53:0/3506065740 reap_dead start 2023-10-21T16:02:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f27fff640 1 -- 172.21.15.53:0/3506065740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10003520 con 0x7f6f100047c0 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.334+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f10003970 0x7f6f10003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.335+0000 7f6f27fff640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 0x7f6f10004c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c001750 tx=0x7f6f100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f885d337d1c8873f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.335+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0052d0 con 0x7f6f100047c0 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.335+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c02e050 con 0x7f6f100047c0 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.335+0000 7f6f277fe640 1 -- 172.21.15.53:0/3506065740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c003830 con 0x7f6f100047c0 2023-10-21T16:02:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 msgr2=0x7f6f10004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 0x7f6f10004c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c001750 tx=0x7f6f100057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/3506065740 shutdown_connections 2023-10-21T16:02:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/3506065740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f100047c0 0x7f6f10004c40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/3506065740 >> 172.21.15.53:0/3506065740 conn(0x7f6f2809bc40 msgr2=0x7f6f280d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/3506065740 shutdown_connections 2023-10-21T16:02:35.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.336+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/3506065740 wait complete. 2023-10-21T16:02:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.338+0000 7f6f2dfb3640 1 Processor -- start 2023-10-21T16:02:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.338+0000 7f6f2dfb3640 1 -- start start 2023-10-21T16:02:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 0x7f6f2809fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f2815a9c0 0x7f6f2809eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2809f860 con 0x7f6f2809f3e0 2023-10-21T16:02:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280cbe60 con 0x7f6f2815a9c0 2023-10-21T16:02:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2dfb3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2809e2c0 con 0x7f6f100047c0 2023-10-21T16:02:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55226/0 (socket says 172.21.15.53:55226) 2023-10-21T16:02:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 -- 172.21.15.53:0/1665562051 learned_addr learned my addr 172.21.15.53:0/1665562051 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f27fff640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f2815a9c0 0x7f6f2809eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 0x7f6f2809fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 -- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 msgr2=0x7f6f2809fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 0x7f6f2809fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 -- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f2815a9c0 msgr2=0x7f6f2809eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f2815a9c0 0x7f6f2809eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2d7b2640 1 -- 172.21.15.53:0/1665562051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280cb570 con 0x7f6f2809f3e0 2023-10-21T16:02:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.339+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 0x7f6f2809fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f2d7b2640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f2815d3a0 tx=0x7f6f20010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20004230 con 0x7f6f2809f3e0 2023-10-21T16:02:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f200043b0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f280cb810 con 0x7f6f2809f3e0 2023-10-21T16:02:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20004520 con 0x7f6f2809f3e0 2023-10-21T16:02:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.340+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f280cc2f0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.342+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f28074150 con 0x7f6f2809f3e0 2023-10-21T16:02:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.342+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f2000e8b0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.343+0000 7f6f257fa640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 0x7f6f1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.344+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f200898f0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.344+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 0x7f6f1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.345+0000 7f6f2cfb1640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 0x7f6f1406d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6f18007c40 tx=0x7f6f18009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.348+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f200546b0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.540+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f280d0070 con 0x7f6f2809f3e0 2023-10-21T16:02:35.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.541+0000 7f6f257fa640 1 -- 172.21.15.53:0/1665562051 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f20057cd0 con 0x7f6f2809f3e0 2023-10-21T16:02:35.543 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:35.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 msgr2=0x7f6f1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 0x7f6f1406d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6f18007c40 tx=0x7f6f18009040 comp rx=0 tx=0).stop 2023-10-21T16:02:35.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 msgr2=0x7f6f281f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f2815d3a0 tx=0x7f6f20010c20 comp rx=0 tx=0).stop 2023-10-21T16:02:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 shutdown_connections 2023-10-21T16:02:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6f1406ae70 0x7f6f1406d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f2809f3e0 0x7f6f281f20d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f2815a9c0 0x7f6f2809eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 --2- 172.21.15.53:0/1665562051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f100047c0 0x7f6f2809fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 >> 172.21.15.53:0/1665562051 conn(0x7f6f2809bc40 msgr2=0x7f6f280b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:35.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 shutdown_connections 2023-10-21T16:02:35.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.544+0000 7f6f2dfb3640 1 -- 172.21.15.53:0/1665562051 wait complete. 2023-10-21T16:02:35.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.788+0000 7f6e4e656640 1 Processor -- start 2023-10-21T16:02:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.788+0000 7f6e4e656640 1 -- start start 2023-10-21T16:02:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.790+0000 7f6e4e656640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806c3a0 0x7f6e480c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.790+0000 7f6e4e656640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e4815aa00 0x7f6e480b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.790+0000 7f6e4e656640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4815b310 con 0x7f6e4806db60 2023-10-21T16:02:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.790+0000 7f6e4e656640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480ca230 con 0x7f6e4815aa00 2023-10-21T16:02:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.790+0000 7f6e4e656640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4806d290 con 0x7f6e4806c3a0 2023-10-21T16:02:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e4815aa00 0x7f6e480b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e4815aa00 0x7f6e480b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55232/0 (socket says 172.21.15.53:55232) 2023-10-21T16:02:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/1973213311 learned_addr learned my addr 172.21.15.53:0/1973213311 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e47fff640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806c3a0 0x7f6e480c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/1973213311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806c3a0 msgr2=0x7f6e480c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806c3a0 0x7f6e480c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/1973213311 >> v1:172.21.15.124:6789/0 conn(0x7f6e4806db60 legacy=0x7f6e4806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.791+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/1973213311 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480c9860 con 0x7f6e4815aa00 2023-10-21T16:02:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.793+0000 7f6e4cbf3640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e480b2a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009e50 tx=0x7f6e4002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18c4edb5eec8562 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.793+0000 7f6e46ffd640 1 -- 172.21.15.53:0/1973213311 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4002cb00 con 0x7f6e4815aa00 2023-10-21T16:02:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.793+0000 7f6e46ffd640 1 -- 172.21.15.53:0/1973213311 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4002cc60 con 0x7f6e4815aa00 2023-10-21T16:02:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.793+0000 7f6e46ffd640 1 -- 172.21.15.53:0/1973213311 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e400343f0 con 0x7f6e4815aa00 2023-10-21T16:02:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 -- 172.21.15.53:0/1973213311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 msgr2=0x7f6e480b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e480b2a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009e50 tx=0x7f6e4002bf60 comp rx=0 tx=0).stop 2023-10-21T16:02:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 -- 172.21.15.53:0/1973213311 shutdown_connections 2023-10-21T16:02:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e480b2a90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 --2- 172.21.15.53:0/1973213311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806c3a0 0x7f6e480c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 -- 172.21.15.53:0/1973213311 >> 172.21.15.53:0/1973213311 conn(0x7f6e4806a5d0 msgr2=0x7f6e480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 -- 172.21.15.53:0/1973213311 shutdown_connections 2023-10-21T16:02:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.794+0000 7f6e4e656640 1 -- 172.21.15.53:0/1973213311 wait complete. 2023-10-21T16:02:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.796+0000 7f6e4e656640 1 Processor -- start 2023-10-21T16:02:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.796+0000 7f6e4e656640 1 -- start start 2023-10-21T16:02:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4806c3a0 0x7f6e4815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806db60 0x7f6e481f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480c51c0 con 0x7f6e4815aa00 2023-10-21T16:02:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48157510 con 0x7f6e4806c3a0 2023-10-21T16:02:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.797+0000 7f6e4e656640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480c3c40 con 0x7f6e4806db60 2023-10-21T16:02:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55234/0 (socket says 172.21.15.53:55234) 2023-10-21T16:02:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/4140537001 learned_addr learned my addr 172.21.15.53:0/4140537001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e47fff640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4806c3a0 0x7f6e4815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e477fe640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806db60 0x7f6e481f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806db60 msgr2=0x7f6e481f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806db60 0x7f6e481f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4806c3a0 msgr2=0x7f6e4815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4806c3a0 0x7f6e4815ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.798+0000 7f6e4cbf3640 1 -- 172.21.15.53:0/4140537001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009b30 con 0x7f6e4815aa00 2023-10-21T16:02:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e4cbf3640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036040 tx=0x7f6e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40040670 con 0x7f6e4815aa00 2023-10-21T16:02:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40040810 con 0x7f6e4815aa00 2023-10-21T16:02:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e481f2730 con 0x7f6e4815aa00 2023-10-21T16:02:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40040bc0 con 0x7f6e4815aa00 2023-10-21T16:02:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.799+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e480c4c00 con 0x7f6e4815aa00 2023-10-21T16:02:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.801+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e481f2500 con 0x7f6e4815aa00 2023-10-21T16:02:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.801+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e40040d20 con 0x7f6e4815aa00 2023-10-21T16:02:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.802+0000 7f6e44ff9640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 0x7f6e3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.802+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6e400ac7e0 con 0x7f6e4815aa00 2023-10-21T16:02:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.803+0000 7f6e47fff640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 0x7f6e3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.804+0000 7f6e47fff640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 0x7f6e3406d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e480c5320 tx=0x7f6e38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:35.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:35.807+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e481f2500 con 0x7f6e4815aa00 2023-10-21T16:02:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.008+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e481f2500 con 0x7f6e4815aa00 2023-10-21T16:02:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.009+0000 7f6e44ff9640 1 -- 172.21.15.53:0/4140537001 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e481f2500 con 0x7f6e4815aa00 2023-10-21T16:02:36.011 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.011+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 msgr2=0x7f6e3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.011+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 0x7f6e3406d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6e480c5320 tx=0x7f6e38008040 comp rx=0 tx=0).stop 2023-10-21T16:02:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.011+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 msgr2=0x7f6e481f35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.011+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036040 tx=0x7f6e40004a40 comp rx=0 tx=0).stop 2023-10-21T16:02:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 shutdown_connections 2023-10-21T16:02:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6e3406ae70 0x7f6e3406d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e4815aa00 0x7f6e481f35a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4806db60 0x7f6e481f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 --2- 172.21.15.53:0/4140537001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4806c3a0 0x7f6e4815ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 >> 172.21.15.53:0/4140537001 conn(0x7f6e4806a5d0 msgr2=0x7f6e48159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:36.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 shutdown_connections 2023-10-21T16:02:36.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.012+0000 7f6e4e656640 1 -- 172.21.15.53:0/4140537001 wait complete. 2023-10-21T16:02:36.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.258+0000 7fef0bfff640 1 Processor -- start 2023-10-21T16:02:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.258+0000 7fef0bfff640 1 -- start start 2023-10-21T16:02:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef0c15a820 0x7fef0c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15ac00 0x7fef0c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c0cab60 con 0x7fef0c0ca680 2023-10-21T16:02:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0bfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0cb960 con 0x7fef0c15a820 2023-10-21T16:02:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0cbdf0 con 0x7fef0c15ac00 2023-10-21T16:02:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef0c15a820 0x7fef0c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef0c15a820 0x7fef0c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55246/0 (socket says 172.21.15.53:55246) 2023-10-21T16:02:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0affd640 1 -- 172.21.15.53:0/2326584658 learned_addr learned my addr 172.21.15.53:0/2326584658 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.260+0000 7fef0b7fe640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15ac00 0x7fef0c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.261+0000 7fef0affd640 1 -- 172.21.15.53:0/2326584658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15ac00 msgr2=0x7fef0c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.261+0000 7fef0affd640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15ac00 0x7fef0c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.261+0000 7fef0affd640 1 -- 172.21.15.53:0/2326584658 >> v1:172.21.15.124:6789/0 conn(0x7fef0c0ca680 legacy=0x7fef0c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.261+0000 7fef0affd640 1 -- 172.21.15.53:0/2326584658 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0cb030 con 0x7fef0c15a820 2023-10-21T16:02:36.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.262+0000 7fef0affd640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15a820 0x7fef0c0aff70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62815bcf2b61241f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.262+0000 7fef09ffb640 1 -- 172.21.15.53:0/2326584658 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0317c0 con 0x7fef0c15a820 2023-10-21T16:02:36.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.262+0000 7fef09ffb640 1 -- 172.21.15.53:0/2326584658 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc031920 con 0x7fef0c15a820 2023-10-21T16:02:36.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.262+0000 7fef09ffb640 1 -- 172.21.15.53:0/2326584658 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc031cb0 con 0x7fef0c15a820 2023-10-21T16:02:36.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 -- 172.21.15.53:0/2326584658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15a820 msgr2=0x7fef0c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15a820 0x7fef0c0aff70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02b860 comp rx=0 tx=0).stop 2023-10-21T16:02:36.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 -- 172.21.15.53:0/2326584658 shutdown_connections 2023-10-21T16:02:36.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15ac00 0x7fef0c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 --2- 172.21.15.53:0/2326584658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15a820 0x7fef0c0aff70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 -- 172.21.15.53:0/2326584658 >> 172.21.15.53:0/2326584658 conn(0x7fef0c06c050 msgr2=0x7fef0c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 -- 172.21.15.53:0/2326584658 shutdown_connections 2023-10-21T16:02:36.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.263+0000 7fef0bfff640 1 -- 172.21.15.53:0/2326584658 wait complete. 2023-10-21T16:02:36.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.265+0000 7fef0bfff640 1 Processor -- start 2023-10-21T16:02:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.265+0000 7fef0bfff640 1 -- start start 2023-10-21T16:02:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef0c0ca680 0x7fef0c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15a820 0x7fef0c0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0c74d0 con 0x7fef0c15ac00 2023-10-21T16:02:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0c8310 con 0x7fef0c0ca680 2023-10-21T16:02:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.266+0000 7fef0bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0c7700 con 0x7fef0c15a820 2023-10-21T16:02:36.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55260/0 (socket says 172.21.15.53:55260) 2023-10-21T16:02:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 -- 172.21.15.53:0/3459382595 learned_addr learned my addr 172.21.15.53:0/3459382595 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0a7fc640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15a820 0x7fef0c0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0affd640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef0c0ca680 0x7fef0c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 -- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15a820 msgr2=0x7fef0c0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15a820 0x7fef0c0b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 -- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef0c0ca680 msgr2=0x7fef0c14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef0c0ca680 0x7fef0c14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 -- 172.21.15.53:0/3459382595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0094d0 con 0x7fef0c15ac00 2023-10-21T16:02:36.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.267+0000 7fef0b7fe640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef04007e30 tx=0x7fef0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.268+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04014620 con 0x7fef0c15ac00 2023-10-21T16:02:36.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.268+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400ee90 con 0x7fef0c15ac00 2023-10-21T16:02:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.268+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c0c79e0 con 0x7fef0c15ac00 2023-10-21T16:02:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.268+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040149c0 con 0x7fef0c15ac00 2023-10-21T16:02:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.268+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c0c7fa0 con 0x7fef0c15ac00 2023-10-21T16:02:36.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.269+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c0c84a0 con 0x7fef0c15ac00 2023-10-21T16:02:36.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.270+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fef04014b20 con 0x7fef0c15ac00 2023-10-21T16:02:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.271+0000 7feef37fe640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 0x7feef806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.271+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef0408a300 con 0x7fef0c15ac00 2023-10-21T16:02:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.272+0000 7fef0affd640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 0x7feef806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.273+0000 7fef0affd640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 0x7feef806d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feefc036000 tx=0x7feefc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.276+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef04055020 con 0x7fef0c15ac00 2023-10-21T16:02:36.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.466+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fef0c074150 con 0x7fef0c15ac00 2023-10-21T16:02:36.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.467+0000 7feef37fe640 1 -- 172.21.15.53:0/3459382595 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fef04058640 con 0x7fef0c15ac00 2023-10-21T16:02:36.469 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:36.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 msgr2=0x7feef806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 0x7feef806d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feefc036000 tx=0x7feefc03d040 comp rx=0 tx=0).stop 2023-10-21T16:02:36.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 msgr2=0x7fef0c0b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef04007e30 tx=0x7fef0400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 shutdown_connections 2023-10-21T16:02:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feef806ae70 0x7feef806d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef0c15ac00 0x7fef0c0b23d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef0c15a820 0x7fef0c0b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 --2- 172.21.15.53:0/3459382595 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef0c0ca680 0x7fef0c14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 >> 172.21.15.53:0/3459382595 conn(0x7fef0c06c050 msgr2=0x7fef0c15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:36.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 shutdown_connections 2023-10-21T16:02:36.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.469+0000 7fef0bfff640 1 -- 172.21.15.53:0/3459382595 wait complete. 2023-10-21T16:02:36.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:36.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.705+0000 7f4d4a183640 1 Processor -- start 2023-10-21T16:02:36.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.705+0000 7f4d4a183640 1 -- start start 2023-10-21T16:02:36.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d4a183640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4d4415ab40 0x7f4d440ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d4a183640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4415b2b0 con 0x7f4d440cb580 2023-10-21T16:02:36.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d4a183640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4415b590 con 0x7f4d440c90b0 2023-10-21T16:02:36.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d4a183640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4415bf30 con 0x7f4d4415ab40 2023-10-21T16:02:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d43fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4d440cb580 0x7f4d4415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36020/0 (socket says 172.21.15.53:36020) 2023-10-21T16:02:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d43fff640 1 -- 172.21.15.53:0/3453122677 learned_addr learned my addr 172.21.15.53:0/3453122677 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.707+0000 7f4d437fe640 1 --2- 172.21.15.53:0/3453122677 >> v2:172.21.15.124:3300/0 conn(0x7f4d4415ab40 0x7f4d440ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645467801 0 0) 0x7f4d4415b2b0 con 0x7f4d440cb580 2023-10-21T16:02:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d30003520 con 0x7f4d440cb580 2023-10-21T16:02:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961857881 0 0) 0x7f4d30003520 con 0x7f4d440cb580 2023-10-21T16:02:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 >> v2:172.21.15.124:3300/0 conn(0x7f4d4415ab40 msgr2=0x7f4d440ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 --2- 172.21.15.53:0/3453122677 >> v2:172.21.15.124:3300/0 conn(0x7f4d4415ab40 0x7f4d440ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 >> v1:172.21.15.124:6789/0 conn(0x7f4d440c90b0 legacy=0x7f4d440cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4415b870 con 0x7f4d440cb580 2023-10-21T16:02:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4d3c001670 con 0x7f4d440cb580 2023-10-21T16:02:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 >> v1:172.21.15.53:6789/0 conn(0x7f4d440cb580 legacy=0x7f4d4415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 0x7f4d30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d30005180 0x7f4d3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4415bf30 con 0x7f4d300047c0 2023-10-21T16:02:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3000c080 con 0x7f4d300039c0 2023-10-21T16:02:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.708+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3000c240 con 0x7f4d30005180 2023-10-21T16:02:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 0x7f4d30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d437fe640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d43fff640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d30005180 0x7f4d3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 -- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d30005180 msgr2=0x7f4d3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d30005180 0x7f4d3000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 -- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d300039c0 msgr2=0x7f4d30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d42ffd640 1 -- 172.21.15.53:0/3453122677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30003520 con 0x7f4d300047c0 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.709+0000 7f4d43fff640 1 -- 172.21.15.53:0/3453122677 reap_dead start 2023-10-21T16:02:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.710+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 0x7f4d30004c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001720 tx=0x7f4d300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684db90dec2b5b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.710+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0052d0 con 0x7f4d300047c0 2023-10-21T16:02:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.710+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c02e050 con 0x7f4d300047c0 2023-10-21T16:02:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.710+0000 7f4d427fc640 1 -- 172.21.15.53:0/3453122677 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c003830 con 0x7f4d300047c0 2023-10-21T16:02:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 -- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 msgr2=0x7f4d30004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 0x7f4d30004c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001720 tx=0x7f4d300057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 -- 172.21.15.53:0/3453122677 shutdown_connections 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3453122677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d300047c0 0x7f4d30004c40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 -- 172.21.15.53:0/3453122677 >> 172.21.15.53:0/3453122677 conn(0x7f4d4406a380 msgr2=0x7f4d440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 -- 172.21.15.53:0/3453122677 shutdown_connections 2023-10-21T16:02:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.711+0000 7f4d4a183640 1 -- 172.21.15.53:0/3453122677 wait complete. 2023-10-21T16:02:36.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.713+0000 7f4d4a183640 1 Processor -- start 2023-10-21T16:02:36.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.713+0000 7f4d4a183640 1 -- start start 2023-10-21T16:02:36.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d300047c0 0x7f4d440b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 0x7f4d440c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440ac6e0 con 0x7f4d440c5390 2023-10-21T16:02:36.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440aba60 con 0x7f4d4414c150 2023-10-21T16:02:36.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d4a183640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440abd40 con 0x7f4d300047c0 2023-10-21T16:02:36.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d43fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d43fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55276/0 (socket says 172.21.15.53:55276) 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.714+0000 7f4d43fff640 1 -- 172.21.15.53:0/3161745800 learned_addr learned my addr 172.21.15.53:0/3161745800 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 0x7f4d440c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 -- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d300047c0 msgr2=0x7f4d440b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d300047c0 0x7f4d440b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 -- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 msgr2=0x7f4d440c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 0x7f4d440c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 -- 172.21.15.53:0/3161745800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440ac020 con 0x7f4d440c5390 2023-10-21T16:02:36.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d42ffd640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 0x7f4d440c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:36.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d43fff640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d4406e3f0 tx=0x7f4d3c00d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0074e0 con 0x7f4d440c5390 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c007660 con 0x7f4d440c5390 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d440ac300 con 0x7f4d440c5390 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c004a20 con 0x7f4d440c5390 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.715+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4414c5d0 con 0x7f4d440c5390 2023-10-21T16:02:36.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.716+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4414c7c0 con 0x7f4d440c5390 2023-10-21T16:02:36.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.717+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d3c004b80 con 0x7f4d440c5390 2023-10-21T16:02:36.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.718+0000 7f4d3bfff640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 0x7f4d2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:36.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.718+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4d3c089060 con 0x7f4d440c5390 2023-10-21T16:02:36.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.719+0000 7f4d437fe640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 0x7f4d2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:36.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.720+0000 7f4d437fe640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 0x7f4d2406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d4415c290 tx=0x7f4d34009750 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:36.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.722+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d3c053eb0 con 0x7f4d440c5390 2023-10-21T16:02:36.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.946+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d440ac820 con 0x7f4d440c5390 2023-10-21T16:02:36.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.947+0000 7f4d3bfff640 1 -- 172.21.15.53:0/3161745800 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f4d440ac820 con 0x7f4d440c5390 2023-10-21T16:02:36.949 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:36.950 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:02:36.950 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:02:36.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 msgr2=0x7f4d2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 0x7f4d2406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d4415c290 tx=0x7f4d34009750 comp rx=0 tx=0).stop 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 msgr2=0x7f4d440c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d4406e3f0 tx=0x7f4d3c00d880 comp rx=0 tx=0).stop 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 shutdown_connections 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4d2406ae70 0x7f4d2406d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d440c5390 0x7f4d440c9b80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d4414c150 0x7f4d440c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 --2- 172.21.15.53:0/3161745800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d300047c0 0x7f4d440b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.950+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 >> 172.21.15.53:0/3161745800 conn(0x7f4d4406a380 msgr2=0x7f4d440af400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.951+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 shutdown_connections 2023-10-21T16:02:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:36.951+0000 7f4d4a183640 1 -- 172.21.15.53:0/3161745800 wait complete. 2023-10-21T16:02:36.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:37.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.194+0000 7f7d1d685640 1 Processor -- start 2023-10-21T16:02:37.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.194+0000 7f7d1d685640 1 -- start start 2023-10-21T16:02:37.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d1d685640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d1815ab30 0x7f7d1815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d1d685640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180b07d0 0x7f7d1806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d1d685640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1815f7d0 con 0x7f7d180adc70 2023-10-21T16:02:37.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d1d685640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1815f210 con 0x7f7d1815ab30 2023-10-21T16:02:37.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d1d685640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1815f4f0 con 0x7f7d180b07d0 2023-10-21T16:02:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d1815ab30 0x7f7d1815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d1815ab30 0x7f7d1815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55282/0 (socket says 172.21.15.53:55282) 2023-10-21T16:02:37.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.196+0000 7f7d17fff640 1 -- 172.21.15.53:0/1736906276 learned_addr learned my addr 172.21.15.53:0/1736906276 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:37.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.197+0000 7f7d17fff640 1 -- 172.21.15.53:0/1736906276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180b07d0 msgr2=0x7f7d1806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.197+0000 7f7d17fff640 1 --2- 172.21.15.53:0/1736906276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180b07d0 0x7f7d1806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.197+0000 7f7d17fff640 1 -- 172.21.15.53:0/1736906276 >> v1:172.21.15.124:6789/0 conn(0x7f7d180adc70 legacy=0x7f7d180b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.197+0000 7f7d17fff640 1 -- 172.21.15.53:0/1736906276 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1806c550 con 0x7f7d1815ab30 2023-10-21T16:02:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.198+0000 7f7d17fff640 1 --2- 172.21.15.53:0/1736906276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1815ab30 0x7f7d1815e8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009870 tx=0x7f7d0802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5597e1a9f6629789 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.198+0000 7f7d177fe640 1 -- 172.21.15.53:0/1736906276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08031640 con 0x7f7d1815ab30 2023-10-21T16:02:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.198+0000 7f7d177fe640 1 -- 172.21.15.53:0/1736906276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d080317a0 con 0x7f7d1815ab30 2023-10-21T16:02:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.198+0000 7f7d177fe640 1 -- 172.21.15.53:0/1736906276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08031b30 con 0x7f7d1815ab30 2023-10-21T16:02:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 -- 172.21.15.53:0/1736906276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1815ab30 msgr2=0x7f7d1815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 --2- 172.21.15.53:0/1736906276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1815ab30 0x7f7d1815e8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009870 tx=0x7f7d0802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 -- 172.21.15.53:0/1736906276 shutdown_connections 2023-10-21T16:02:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 --2- 172.21.15.53:0/1736906276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180b07d0 0x7f7d1806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 --2- 172.21.15.53:0/1736906276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d1815ab30 0x7f7d1815e8a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 -- 172.21.15.53:0/1736906276 >> 172.21.15.53:0/1736906276 conn(0x7f7d180cb0c0 msgr2=0x7f7d180cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 -- 172.21.15.53:0/1736906276 shutdown_connections 2023-10-21T16:02:37.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.199+0000 7f7d1d685640 1 -- 172.21.15.53:0/1736906276 wait complete. 2023-10-21T16:02:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.201+0000 7f7d1d685640 1 Processor -- start 2023-10-21T16:02:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.201+0000 7f7d1d685640 1 -- start start 2023-10-21T16:02:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180adc70 0x7f7d18156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 0x7f7d180ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c8160 con 0x7f7d180b07d0 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180ab8d0 con 0x7f7d1815ab30 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1d685640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c70f0 con 0x7f7d180adc70 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55286/0 (socket says 172.21.15.53:55286) 2023-10-21T16:02:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 -- 172.21.15.53:0/2121492149 learned_addr learned my addr 172.21.15.53:0/2121492149 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d17fff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180adc70 0x7f7d18156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1ce84640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 0x7f7d180ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 -- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180adc70 msgr2=0x7f7d18156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180adc70 0x7f7d18156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 -- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 msgr2=0x7f7d180ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 0x7f7d180ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d0ffff640 1 -- 172.21.15.53:0/2121492149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080094d0 con 0x7f7d180b07d0 2023-10-21T16:02:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.202+0000 7f7d1ce84640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 0x7f7d180ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d0ffff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c440 tx=0x7f7d0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00005480 con 0x7f7d180b07d0 2023-10-21T16:02:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000035f0 con 0x7f7d180b07d0 2023-10-21T16:02:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000039c0 con 0x7f7d180b07d0 2023-10-21T16:02:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180c73d0 con 0x7f7d180b07d0 2023-10-21T16:02:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.203+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d180abe00 con 0x7f7d180b07d0 2023-10-21T16:02:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.204+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1804e160 con 0x7f7d180b07d0 2023-10-21T16:02:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.205+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d00003b20 con 0x7f7d180b07d0 2023-10-21T16:02:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.206+0000 7f7d157fa640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 0x7f7cf806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.206+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d00088b00 con 0x7f7d180b07d0 2023-10-21T16:02:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.207+0000 7f7d17fff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 0x7f7cf806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.208+0000 7f7d17fff640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 0x7f7cf806d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d08036000 tx=0x7f7d0803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.210+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d000538c0 con 0x7f7d180b07d0 2023-10-21T16:02:37.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.404+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d180ca220 con 0x7f7d180b07d0 2023-10-21T16:02:37.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.406+0000 7f7d157fa640 1 -- 172.21.15.53:0/2121492149 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f7d00017020 con 0x7f7d180b07d0 2023-10-21T16:02:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:37.409 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":26,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:01.905448+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:37.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 msgr2=0x7f7cf806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 0x7f7cf806d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d08036000 tx=0x7f7d0803c040 comp rx=0 tx=0).stop 2023-10-21T16:02:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 msgr2=0x7f7d181f3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000c440 tx=0x7f7d0000c900 comp rx=0 tx=0).stop 2023-10-21T16:02:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 shutdown_connections 2023-10-21T16:02:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d1815ab30 0x7f7d180ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7cf806ae70 0x7f7cf806d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180b07d0 0x7f7d181f3a30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 --2- 172.21.15.53:0/2121492149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180adc70 0x7f7d18156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 >> 172.21.15.53:0/2121492149 conn(0x7f7d180cb0c0 msgr2=0x7f7d180701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:37.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.409+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 shutdown_connections 2023-10-21T16:02:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.410+0000 7f7d1d685640 1 -- 172.21.15.53:0/2121492149 wait complete. 2023-10-21T16:02:37.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:02:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.657+0000 7f1adf316640 1 Processor -- start 2023-10-21T16:02:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.657+0000 7f1adf316640 1 -- start start 2023-10-21T16:02:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1ad80d07c0 0x7f1ad80d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ad80d1470 0x7f1ad8160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80d1900 con 0x7f1ad80d1470 2023-10-21T16:02:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad80ce610 con 0x7f1ad80d07c0 2023-10-21T16:02:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.659+0000 7f1adf316640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad815d1d0 con 0x7f1ad815cdf0 2023-10-21T16:02:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ad80d1470 0x7f1ad8160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ad80d1470 0x7f1ad8160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55288/0 (socket says 172.21.15.53:55288) 2023-10-21T16:02:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 -- 172.21.15.53:0/1555193220 learned_addr learned my addr 172.21.15.53:0/1555193220 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1addb13640 1 --2- 172.21.15.53:0/1555193220 >> v2:172.21.15.124:3300/0 conn(0x7f1ad80d07c0 0x7f1ad80d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1ade314640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 -- 172.21.15.53:0/1555193220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 msgr2=0x7f1ad80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad80ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 -- 172.21.15.53:0/1555193220 >> v2:172.21.15.124:3300/0 conn(0x7f1ad80d07c0 msgr2=0x7f1ad80d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 --2- 172.21.15.53:0/1555193220 >> v2:172.21.15.124:3300/0 conn(0x7f1ad80d07c0 0x7f1ad80d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.660+0000 7f1adeb15640 1 -- 172.21.15.53:0/1555193220 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80ceb00 con 0x7f1ad80d1470 2023-10-21T16:02:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.661+0000 7f1adeb15640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d1470 0x7f1ad8160850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8067060 tx=0x7f1ad002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87209f63c1005369 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.662+0000 7f1add312640 1 -- 172.21.15.53:0/1555193220 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad002c960 con 0x7f1ad80d1470 2023-10-21T16:02:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.662+0000 7f1add312640 1 -- 172.21.15.53:0/1555193220 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad002cac0 con 0x7f1ad80d1470 2023-10-21T16:02:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.662+0000 7f1add312640 1 -- 172.21.15.53:0/1555193220 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad002ce50 con 0x7f1ad80d1470 2023-10-21T16:02:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.662+0000 7f1adf316640 1 -- 172.21.15.53:0/1555193220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d1470 msgr2=0x7f1ad8160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d1470 0x7f1ad8160850 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8067060 tx=0x7f1ad002bdc0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 -- 172.21.15.53:0/1555193220 shutdown_connections 2023-10-21T16:02:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d1470 0x7f1ad8160850 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 --2- 172.21.15.53:0/1555193220 >> v2:172.21.15.124:3300/0 conn(0x7f1ad80d07c0 0x7f1ad80d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 --2- 172.21.15.53:0/1555193220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad80ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 -- 172.21.15.53:0/1555193220 >> 172.21.15.53:0/1555193220 conn(0x7f1ad809bcb0 msgr2=0x7f1ad80d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 -- 172.21.15.53:0/1555193220 shutdown_connections 2023-10-21T16:02:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.663+0000 7f1adf316640 1 -- 172.21.15.53:0/1555193220 wait complete. 2023-10-21T16:02:37.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.665+0000 7f1adf316640 1 Processor -- start 2023-10-21T16:02:37.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.665+0000 7f1adf316640 1 -- start start 2023-10-21T16:02:37.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ad80d1470 0x7f1ad81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad81f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8157420 con 0x7f1ad80d07c0 2023-10-21T16:02:37.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8157700 con 0x7f1ad80d1470 2023-10-21T16:02:37.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1adf316640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad81579e0 con 0x7f1ad815cdf0 2023-10-21T16:02:37.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1ade314640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1ade314640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55294/0 (socket says 172.21.15.53:55294) 2023-10-21T16:02:37.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1ade314640 1 -- 172.21.15.53:0/1732622518 learned_addr learned my addr 172.21.15.53:0/1732622518 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:37.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.666+0000 7f1addb13640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ad80d1470 0x7f1ad81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1adeb15640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad81f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 -- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 msgr2=0x7f1ad81f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad81f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 -- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ad80d1470 msgr2=0x7f1ad81597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ad80d1470 0x7f1ad81597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 -- 172.21.15.53:0/1732622518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0009b30 con 0x7f1ad80d07c0 2023-10-21T16:02:37.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1ade314640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400d800 tx=0x7f1ad400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400b6c0 con 0x7f1ad80d07c0 2023-10-21T16:02:37.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad400b820 con 0x7f1ad80d07c0 2023-10-21T16:02:37.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad8157cc0 con 0x7f1ad80d07c0 2023-10-21T16:02:37.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.667+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400bbb0 con 0x7f1ad80d07c0 2023-10-21T16:02:37.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.668+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad8159d10 con 0x7f1ad80d07c0 2023-10-21T16:02:37.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.669+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad80b51b0 con 0x7f1ad80d07c0 2023-10-21T16:02:37.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.670+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1ad400bd10 con 0x7f1ad80d07c0 2023-10-21T16:02:37.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.670+0000 7f1aceffd640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 0x7f1abc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:37.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.671+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ad4089e10 con 0x7f1ad80d07c0 2023-10-21T16:02:37.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.671+0000 7f1addb13640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 0x7f1abc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:37.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.672+0000 7f1addb13640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 0x7f1abc06d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8006fd0 tx=0x7f1ac8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:37.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.675+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ad4054c60 con 0x7f1ad80d07c0 2023-10-21T16:02:37.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.907+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ad80b4cc0 con 0x7f1ad80d07c0 2023-10-21T16:02:37.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.908+0000 7f1aceffd640 1 -- 172.21.15.53:0/1732622518 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f1ad4058280 con 0x7f1ad80d07c0 2023-10-21T16:02:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:37.911 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:02:37.912 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:02:37.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 msgr2=0x7f1abc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 0x7f1abc06d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8006fd0 tx=0x7f1ac8008040 comp rx=0 tx=0).stop 2023-10-21T16:02:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 msgr2=0x7f1ad8156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400d800 tx=0x7f1ad400dcc0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 shutdown_connections 2023-10-21T16:02:37.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ad815cdf0 0x7f1ad81f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ad80d1470 0x7f1ad81597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1abc06ae70 0x7f1abc06d330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 --2- 172.21.15.53:0/1732622518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ad80d07c0 0x7f1ad8156db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:37.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 >> 172.21.15.53:0/1732622518 conn(0x7f1ad809bcb0 msgr2=0x7f1ad806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:37.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 shutdown_connections 2023-10-21T16:02:37.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:37.911+0000 7f1adf316640 1 -- 172.21.15.53:0/1732622518 wait complete. 2023-10-21T16:02:38.336 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:01:55.604470+0000', 'modified': '2023-10-21T16:02:22.488758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34163': {'gid': 34163, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/2621260973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 2621260973}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 2621260973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:02:38.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:02:38.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-21T16:02:38.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.566+0000 7f02f9378640 1 Processor -- start 2023-10-21T16:02:38.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.567+0000 7f02f9378640 1 -- start start 2023-10-21T16:02:38.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.568+0000 7f02f9378640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f02f4070cc0 0x7f02f40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.568+0000 7f02f9378640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f40cb450 0x7f02f40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.568+0000 7f02f9378640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f4071170 con 0x7f02f4160660 2023-10-21T16:02:38.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.568+0000 7f02f9378640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f406e150 con 0x7f02f4070cc0 2023-10-21T16:02:38.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.569+0000 7f02f9378640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40cb8c0 con 0x7f02f40cb450 2023-10-21T16:02:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.569+0000 7f02f27fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f02f4070cc0 0x7f02f40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.569+0000 7f02f27fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f02f4070cc0 0x7f02f40caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:52926/0 (socket says 172.21.15.53:52926) 2023-10-21T16:02:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.569+0000 7f02f27fc640 1 -- 172.21.15.53:0/1077277625 learned_addr learned my addr 172.21.15.53:0/1077277625 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.569+0000 7f02f37fe640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f40cb450 0x7f02f40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.570+0000 7f02f27fc640 1 -- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f40cb450 msgr2=0x7f02f40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.570+0000 7f02f27fc640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f40cb450 0x7f02f40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.570+0000 7f02f27fc640 1 -- 172.21.15.53:0/1077277625 >> v1:172.21.15.124:6789/0 conn(0x7f02f4160660 legacy=0x7f02f40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.570+0000 7f02f27fc640 1 -- 172.21.15.53:0/1077277625 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f406de70 con 0x7f02f4070cc0 2023-10-21T16:02:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.571+0000 7f02f27fc640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f40caf10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009870 tx=0x7f02dc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4992c9d7e1b93281 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f1ffb640 1 -- 172.21.15.53:0/1077277625 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc030840 con 0x7f02f4070cc0 2023-10-21T16:02:38.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f1ffb640 1 -- 172.21.15.53:0/1077277625 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0309a0 con 0x7f02f4070cc0 2023-10-21T16:02:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f1ffb640 1 -- 172.21.15.53:0/1077277625 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc030d30 con 0x7f02f4070cc0 2023-10-21T16:02:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 -- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 msgr2=0x7f02f40caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f40caf10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009870 tx=0x7f02dc02b980 comp rx=0 tx=0).stop 2023-10-21T16:02:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 -- 172.21.15.53:0/1077277625 shutdown_connections 2023-10-21T16:02:38.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f40cb450 0x7f02f40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 --2- 172.21.15.53:0/1077277625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f40caf10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 -- 172.21.15.53:0/1077277625 >> 172.21.15.53:0/1077277625 conn(0x7f02f406a5d0 msgr2=0x7f02f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 -- 172.21.15.53:0/1077277625 shutdown_connections 2023-10-21T16:02:38.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.572+0000 7f02f9378640 1 -- 172.21.15.53:0/1077277625 wait complete. 2023-10-21T16:02:38.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.574+0000 7f02f9378640 1 Processor -- start 2023-10-21T16:02:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.574+0000 7f02f9378640 1 -- start start 2023-10-21T16:02:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f41f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f4160660 0x7f02f40c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f41f3690 con 0x7f02f40cb450 2023-10-21T16:02:38.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f4153260 con 0x7f02f4070cc0 2023-10-21T16:02:38.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.575+0000 7f02f9378640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f40c2250 con 0x7f02f4160660 2023-10-21T16:02:38.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55310/0 (socket says 172.21.15.53:55310) 2023-10-21T16:02:38.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 -- 172.21.15.53:0/2905868941 learned_addr learned my addr 172.21.15.53:0/2905868941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f41f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f37fe640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f4160660 0x7f02f40c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 -- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f4160660 msgr2=0x7f02f40c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f4160660 0x7f02f40c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 -- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 msgr2=0x7f02f41f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f41f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.576+0000 7f02f27fc640 1 -- 172.21.15.53:0/2905868941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc0094d0 con 0x7f02f40cb450 2023-10-21T16:02:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02dc000c00 tx=0x7f02dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02eb7fe640 1 -- 172.21.15.53:0/2905868941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc039c90 con 0x7f02f40cb450 2023-10-21T16:02:38.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02eb7fe640 1 -- 172.21.15.53:0/2905868941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc039df0 con 0x7f02f40cb450 2023-10-21T16:02:38.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02eb7fe640 1 -- 172.21.15.53:0/2905868941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc043640 con 0x7f02f40cb450 2023-10-21T16:02:38.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f40c2530 con 0x7f02f40cb450 2023-10-21T16:02:38.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.577+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f415b6e0 con 0x7f02f40cb450 2023-10-21T16:02:38.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.579+0000 7f02eb7fe640 1 -- 172.21.15.53:0/2905868941 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02dc0437a0 con 0x7f02f40cb450 2023-10-21T16:02:38.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.580+0000 7f02eb7fe640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 0x7f02d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.580+0000 7f02eb7fe640 1 -- 172.21.15.53:0/2905868941 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02dc03e070 con 0x7f02f40cb450 2023-10-21T16:02:38.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.581+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 0x7f02d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.581+0000 7f02fbfe8740 1 Processor -- start 2023-10-21T16:02:38.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.581+0000 7f02fbfe8740 1 -- start start 2023-10-21T16:02:38.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 0x7f02d406d360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02e40012e0 tx=0x7f02e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254703e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 0x564254734250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642546247a0 con 0x564254731360 2023-10-21T16:02:38.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254676740 con 0x5642547043c0 2023-10-21T16:02:38.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.582+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564254673e30 con 0x564254730f80 2023-10-21T16:02:38.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55316/0 (socket says 172.21.15.53:55316) 2023-10-21T16:02:38.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 -- 172.21.15.53:0/1433801844 learned_addr learned my addr 172.21.15.53:0/1433801844 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254703e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f37fe640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 0x564254734250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 -- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 msgr2=0x564254703e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254703e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 -- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 msgr2=0x564254734250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 0x564254734250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 -- 172.21.15.53:0/1433801844 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564254633910 con 0x564254731360 2023-10-21T16:02:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f37fe640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 0x564254734250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.583+0000 7f02f27fc640 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02dc07b230 tx=0x7f02dc07b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f477a3d1e48e1d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02e8ff9640 1 -- 172.21.15.53:0/1433801844 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc081040 con 0x564254731360 2023-10-21T16:02:38.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02e8ff9640 1 -- 172.21.15.53:0/1433801844 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0835a0 con 0x564254731360 2023-10-21T16:02:38.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02e8ff9640 1 -- 172.21.15.53:0/1433801844 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc079640 con 0x564254731360 2023-10-21T16:02:38.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 -- 172.21.15.53:0/1433801844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 msgr2=0x564254733d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f02dc07b230 tx=0x7f02dc07b9a0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 -- 172.21.15.53:0/1433801844 shutdown_connections 2023-10-21T16:02:38.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5642547043c0 0x564254734250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254733d10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/1433801844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254703e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 -- 172.21.15.53:0/1433801844 >> 172.21.15.53:0/1433801844 conn(0x564254706190 msgr2=0x56425474ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.584+0000 7f02fbfe8740 1 -- 172.21.15.53:0/1433801844 shutdown_connections 2023-10-21T16:02:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.585+0000 7f02fbfe8740 1 -- 172.21.15.53:0/1433801844 wait complete. 2023-10-21T16:02:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.585+0000 7f02fbfe8740 1 -- start start 2023-10-21T16:02:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.585+0000 7f02fbfe8740 1 Processor -- start 2023-10-21T16:02:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254730f80 0x56425472e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x564254747990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425466f720 con 0x564254731360 2023-10-21T16:02:38.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425460d970 con 0x564254730f80 2023-10-21T16:02:38.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.586+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x56425460d610 con 0x56425474f940 2023-10-21T16:02:38.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02fbfe8740 10 client.0 _ll_register_callbacks cb 0x56425462f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:02:38.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55318/0 (socket says 172.21.15.53:55318) 2023-10-21T16:02:38.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 -- 172.21.15.53:0/2121271958 learned_addr learned my addr 172.21.15.53:0/2121271958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254730f80 0x56425472e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f37fe640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x564254747990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 -- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 msgr2=0x564254747990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x564254747990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 -- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254730f80 msgr2=0x56425472e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254730f80 0x56425472e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 -- 172.21.15.53:0/2121271958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f415b6e0 con 0x564254731360 2023-10-21T16:02:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f37fe640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x564254747990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02f27fc640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02dc07b230 tx=0x7f02dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12a17f0606f9bf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.587+0000 7f02d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:02:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc081040 con 0x564254731360 2023-10-21T16:02:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02dc083310 con 0x564254731360 2023-10-21T16:02:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564254696160 con 0x564254731360 2023-10-21T16:02:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02dc0811a0 con 0x564254731360 2023-10-21T16:02:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 10 client.4821 fetch_fsmap learned FSMap version 9 2023-10-21T16:02:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x56425466dda0 con 0x564254731360 2023-10-21T16:02:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564254696160 con 0x564254731360 2023-10-21T16:02:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f02dc081300 con 0x564254731360 2023-10-21T16:02:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2023-10-21T16:02:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 10 client.4821 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T16:02:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.588+0000 7f02fbfe8740 20 client.4821 populate_metadata read hostname 'smithi053' 2023-10-21T16:02:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.589+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254746520 0x564254735880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.589+0000 7f02fbfe8740 4 client.4821 mds_command: new command op to 34163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:02:38.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.589+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 --> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564254641cc0 con 0x564254746520 2023-10-21T16:02:38.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.589+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254746520 0x564254735880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.590+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254746520 0x564254735880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb862d61be0649c server_cookie=339de7e939523e3e in_seq=0 out_seq=0 2023-10-21T16:02:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.590+0000 7f02d1ffb640 10 client.4821 ms_handle_connect on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02d1ffb640 1 -- 172.21.15.53:0/2121271958 <== mds.0 v2:172.21.15.124:6836/2621260973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564254641cc0 con 0x564254746520 2023-10-21T16:02:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02d1ffb640 10 client.4821 handle_command_reply: tid=0 2023-10-21T16:02:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 client.4821 shutdown 2023-10-21T16:02:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 msgr2=0x564254736ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02dc07b230 tx=0x7f02dc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 shutdown_connections 2023-10-21T16:02:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254746520 0x564254735880 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x564254747990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254736ef0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/2121271958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254730f80 0x56425472e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02d1ffb640 0 client.4821 ms_handle_reset on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.592+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 >> 172.21.15.53:0/2121271958 conn(0x5642547043c0 msgr2=0x564254707f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.593+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 shutdown_connections 2023-10-21T16:02:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.593+0000 7f02fbfe8740 1 -- 172.21.15.53:0/2121271958 wait complete. 2023-10-21T16:02:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.593+0000 7f02fbfe8740 20 client.4821 trim_cache size 0 max 16384 2023-10-21T16:02:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.609+0000 7f02fbfe8740 1 Processor -- start 2023-10-21T16:02:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 -- start start 2023-10-21T16:02:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254750e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254737d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56425474f940 0x564254752410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642543d3f20 con 0x564254731360 2023-10-21T16:02:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642547368b0 con 0x56425474f940 2023-10-21T16:02:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.610+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564254748d80 con 0x564254730f80 2023-10-21T16:02:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.611+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254750e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.611+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254750e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:34444/0 (socket says 172.21.15.53:34444) 2023-10-21T16:02:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.611+0000 7f02f2ffd640 1 -- 172.21.15.53:0/869348060 learned_addr learned my addr 172.21.15.53:0/869348060 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.611+0000 7f02f27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254737d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.611+0000 7f02f37fe640 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56425474f940 0x564254752410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 -- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 msgr2=0x564254750e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254750e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 -- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56425474f940 msgr2=0x564254752410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56425474f940 0x564254752410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 -- 172.21.15.53:0/869348060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56425460cb80 con 0x564254731360 2023-10-21T16:02:38.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02f27fc640 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254737d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02dc02bf90 tx=0x7f02dc04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b67771addde6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02e97fa640 1 -- 172.21.15.53:0/869348060 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc081040 con 0x564254731360 2023-10-21T16:02:38.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02e97fa640 1 -- 172.21.15.53:0/869348060 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc083310 con 0x564254731360 2023-10-21T16:02:38.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.612+0000 7f02e97fa640 1 -- 172.21.15.53:0/869348060 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0813c0 con 0x564254731360 2023-10-21T16:02:38.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 -- 172.21.15.53:0/869348060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 msgr2=0x564254737d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254737d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02dc02bf90 tx=0x7f02dc04f000 comp rx=0 tx=0).stop 2023-10-21T16:02:38.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 -- 172.21.15.53:0/869348060 shutdown_connections 2023-10-21T16:02:38.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x56425474f940 0x564254752410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254731360 0x564254737d30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/869348060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564254730f80 0x564254750e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 -- 172.21.15.53:0/869348060 >> 172.21.15.53:0/869348060 conn(0x5642547043c0 msgr2=0x564254771340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 -- 172.21.15.53:0/869348060 shutdown_connections 2023-10-21T16:02:38.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.613+0000 7f02fbfe8740 1 -- 172.21.15.53:0/869348060 wait complete. 2023-10-21T16:02:38.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.614+0000 7f02fbfe8740 1 -- start start 2023-10-21T16:02:38.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.614+0000 7f02fbfe8740 1 Processor -- start 2023-10-21T16:02:38.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254731360 0x5642547b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x56425475d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425475dd90 con 0x564254730f80 2023-10-21T16:02:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425472ddf0 con 0x564254731360 2023-10-21T16:02:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564254750790 con 0x56425474f940 2023-10-21T16:02:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02fbfe8740 10 client.0 _ll_register_callbacks cb 0x7f02dc07ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:02:38.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55338/0 (socket says 172.21.15.53:55338) 2023-10-21T16:02:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f2ffd640 1 -- 172.21.15.53:0/3573989467 learned_addr learned my addr 172.21.15.53:0/3573989467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f37fe640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x56425475d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f27fc640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254731360 0x5642547b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f2ffd640 1 -- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 msgr2=0x56425475d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.615+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x56425475d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02f2ffd640 1 -- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254731360 msgr2=0x5642547b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254731360 0x5642547b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02f2ffd640 1 -- 172.21.15.53:0/3573989467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56425466dda0 con 0x564254730f80 2023-10-21T16:02:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02f2ffd640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02e4001c20 tx=0x7f02e4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe3a6af2e3fdca server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:02:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4014450 con 0x564254730f80 2023-10-21T16:02:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02e402fdb0 con 0x564254730f80 2023-10-21T16:02:38.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02dc0835a0 con 0x564254730f80 2023-10-21T16:02:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02e40145b0 con 0x564254730f80 2023-10-21T16:02:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02fbfe8740 10 client.4827 fetch_fsmap learned FSMap version 9 2023-10-21T16:02:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.616+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x564254632170 con 0x564254730f80 2023-10-21T16:02:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0835a0 con 0x564254730f80 2023-10-21T16:02:38.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f02e4012070 con 0x564254730f80 2023-10-21T16:02:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2023-10-21T16:02:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 10 client.4827 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T16:02:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 20 client.4827 populate_metadata read hostname 'smithi053' 2023-10-21T16:02:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254736c30 0x564254746900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:38.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 4 client.4827 mds_command: new command op to 34163 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:02:38.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 --> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564254668700 con 0x564254736c30 2023-10-21T16:02:38.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.617+0000 7f02f27fc640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254736c30 0x564254746900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:38.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.618+0000 7f02d1ffb640 10 client.4827 ms_handle_connect on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:38.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.618+0000 7f02f27fc640 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254736c30 0x564254746900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c809db18ddfa3c42 server_cookie=f84d23243f4ba6e2 in_seq=0 out_seq=0 2023-10-21T16:02:38.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02d1ffb640 1 -- 172.21.15.53:0/3573989467 <== mds.0 v2:172.21.15.124:6836/2621260973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564254668700 con 0x564254736c30 2023-10-21T16:02:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02d1ffb640 10 client.4827 handle_command_reply: tid=0 2023-10-21T16:02:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 client.4827 shutdown 2023-10-21T16:02:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 msgr2=0x5642547b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02e4001c20 tx=0x7f02e4001250 comp rx=0 tx=0).stop 2023-10-21T16:02:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 shutdown_connections 2023-10-21T16:02:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:6836/2621260973,v1:172.21.15.124:6837/2621260973] conn(0x564254736c30 0x564254746900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x56425474f940 0x56425475d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564254731360 0x5642547b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02d1ffb640 0 client.4827 ms_handle_reset on v2:172.21.15.124:6836/2621260973 2023-10-21T16:02:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 --2- 172.21.15.53:0/3573989467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564254730f80 0x5642547b03c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 >> 172.21.15.53:0/3573989467 conn(0x5642547043c0 msgr2=0x564254771340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 shutdown_connections 2023-10-21T16:02:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 1 -- 172.21.15.53:0/3573989467 wait complete. 2023-10-21T16:02:38.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.619+0000 7f02fbfe8740 20 client.4827 trim_cache size 0 max 16384 2023-10-21T16:02:38.637 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:02:38.637 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:02:38.637 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:02:38.638 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 34163, 2023-10-21T16:02:38.638 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:02:38.638 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:02:38.638 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:02:38.638 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 42.491879025999999, 2023-10-21T16:02:38.639 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 9, 2023-10-21T16:02:38.639 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 26, 2023-10-21T16:02:38.639 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 26, 2023-10-21T16:02:38.639 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 50.278899197000001 2023-10-21T16:02:38.639 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:02:38.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 msgr2=0x7f02d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 0x7f02d406d360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f02e40012e0 tx=0x7f02e4009040 comp rx=0 tx=0).stop 2023-10-21T16:02:38.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 msgr2=0x7f02f4152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:38.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02dc000c00 tx=0x7f02dc004730 comp rx=0 tx=0).stop 2023-10-21T16:02:38.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 shutdown_connections 2023-10-21T16:02:38.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02f4160660 0x7f02f40c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f02d406aea0 0x7f02d406d360 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02f40cb450 0x7f02f4152d20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 --2- 172.21.15.53:0/2905868941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02f4070cc0 0x7f02f41f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 >> 172.21.15.53:0/2905868941 conn(0x7f02f406a5d0 msgr2=0x7f02f40cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 shutdown_connections 2023-10-21T16:02:38.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:38.620+0000 7f02f9378640 1 -- 172.21.15.53:0/2905868941 wait complete. 2023-10-21T16:02:38.644 INFO:teuthology.run_tasks:Running task kclient... 2023-10-21T16:02:38.657 INFO:tasks.kclient:Mounting kernel clients... 2023-10-21T16:02:38.658 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-21T16:02:38.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:38.659 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:02:38.717 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-21T16:02:38.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:38.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-21T16:02:38.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:38.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-10-21T16:02:38.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:38.959 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:02:39.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:39.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-10-21T16:02:39.158 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-10-21T16:02:39.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T16:02:39.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:39.166 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T16:02:39.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:39.228 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-21T16:02:39.331 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-21T16:02:39.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T16:02:39.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:39.336 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T16:02:39.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:39.392 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-21T16:02:39.493 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-21T16:02:39.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T16:02:39.498 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-21T16:02:39.498 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-21T16:02:39.499 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:02:39.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:02:39.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.734+0000 7f2b263ce640 1 Processor -- start 2023-10-21T16:02:39.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.734+0000 7f2b263ce640 1 -- start start 2023-10-21T16:02:39.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.736+0000 7f2b263ce640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2b2015ab40 0x7f2b200ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.736+0000 7f2b263ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b200adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.736+0000 7f2b263ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2015b2b0 con 0x7f2b200cae70 2023-10-21T16:02:39.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.736+0000 7f2b263ce640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2015bf30 con 0x7f2b2015ab40 2023-10-21T16:02:39.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.736+0000 7f2b263ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2015f7a0 con 0x7f2b200c90b0 2023-10-21T16:02:39.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b2496b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2b200cae70 0x7f2b200702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36022/0 (socket says 172.21.15.53:36022) 2023-10-21T16:02:39.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b2496b640 1 -- 172.21.15.53:0/3467709350 learned_addr learned my addr 172.21.15.53:0/3467709350 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:39.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3467709350 >> v2:172.21.15.124:3300/0 conn(0x7f2b2015ab40 0x7f2b200ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b200adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544057723 0 0) 0x7f2b2015b2b0 con 0x7f2b200cae70 2023-10-21T16:02:39.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b0c003520 con 0x7f2b200cae70 2023-10-21T16:02:39.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.737+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3491673849 0 0) 0x7f2b0c003520 con 0x7f2b200cae70 2023-10-21T16:02:39.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 msgr2=0x7f2b200adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:39.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b200adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 >> v2:172.21.15.124:3300/0 conn(0x7f2b2015ab40 msgr2=0x7f2b200ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 --2- 172.21.15.53:0/3467709350 >> v2:172.21.15.124:3300/0 conn(0x7f2b2015ab40 0x7f2b200ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2015b590 con 0x7f2b200cae70 2023-10-21T16:02:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2b180019f0 con 0x7f2b200cae70 2023-10-21T16:02:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3467709350 >> v2:172.21.15.124:3300/0 conn(0x7f2b2015ab40 0x7f2b200ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 >> v1:172.21.15.53:6789/0 conn(0x7f2b200cae70 legacy=0x7f2b200702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b0c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b0c003f40 0x7f2b0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c004d00 0x7f2b0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2015bf30 con 0x7f2b0c003f40 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2015f7a0 con 0x7f2b0c004d00 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c00b980 con 0x7f2b200c90b0 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b0c003f40 0x7f2b0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.738+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b0c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b177fe640 1 -- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 msgr2=0x7f2b0c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c90b0 0x7f2b0c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b2496b640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c004d00 0x7f2b0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b177fe640 1 -- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c004d00 msgr2=0x7f2b0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c004d00 0x7f2b0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b177fe640 1 -- 172.21.15.53:0/3467709350 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c003520 con 0x7f2b0c003f40 2023-10-21T16:02:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.739+0000 7f2b2496b640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c004d00 0x7f2b0c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.740+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b0c003f40 0x7f2b0c0047c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009af0 tx=0x7f2b0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c4a420ead7c04d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.740+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0802c6b0 con 0x7f2b0c003f40 2023-10-21T16:02:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.740+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0802c810 con 0x7f2b0c003f40 2023-10-21T16:02:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.740+0000 7f2b1f7fe640 1 -- 172.21.15.53:0/3467709350 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0802cba0 con 0x7f2b0c003f40 2023-10-21T16:02:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 -- 172.21.15.53:0/3467709350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b0c003f40 msgr2=0x7f2b0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3467709350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b0c003f40 0x7f2b0c0047c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009af0 tx=0x7f2b0c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b2496b640 1 -- 172.21.15.53:0/3467709350 reap_dead start 2023-10-21T16:02:39.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 -- 172.21.15.53:0/3467709350 shutdown_connections 2023-10-21T16:02:39.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 -- 172.21.15.53:0/3467709350 >> 172.21.15.53:0/3467709350 conn(0x7f2b2006a380 msgr2=0x7f2b200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:39.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 -- 172.21.15.53:0/3467709350 shutdown_connections 2023-10-21T16:02:39.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.741+0000 7f2b263ce640 1 -- 172.21.15.53:0/3467709350 wait complete. 2023-10-21T16:02:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.743+0000 7f2b263ce640 1 Processor -- start 2023-10-21T16:02:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.743+0000 7f2b263ce640 1 -- start start 2023-10-21T16:02:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 0x7f2b201f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c3840 0x7f2b201f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c56b0 con 0x7f2b200c4a60 2023-10-21T16:02:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200b2bd0 con 0x7f2b0c003f40 2023-10-21T16:02:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b263ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200b25c0 con 0x7f2b200c3840 2023-10-21T16:02:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55350/0 (socket says 172.21.15.53:55350) 2023-10-21T16:02:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 -- 172.21.15.53:0/3623684101 learned_addr learned my addr 172.21.15.53:0/3623684101 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 0x7f2b201f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b177fe640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c3840 0x7f2b201f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 -- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c3840 msgr2=0x7f2b201f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c3840 0x7f2b201f55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 -- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 msgr2=0x7f2b201f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 0x7f2b201f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b2496b640 1 -- 172.21.15.53:0/3623684101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200c53d0 con 0x7f2b200c4a60 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.744+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 0x7f2b201f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b2496b640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b200b0c20 tx=0x7f2b1800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800de90 con 0x7f2b200c4a60 2023-10-21T16:02:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800bb20 con 0x7f2b200c4a60 2023-10-21T16:02:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b200b2860 con 0x7f2b200c4a60 2023-10-21T16:02:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800bed0 con 0x7f2b200c4a60 2023-10-21T16:02:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.745+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b200c50f0 con 0x7f2b200c4a60 2023-10-21T16:02:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.746+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b200b2420 con 0x7f2b200c4a60 2023-10-21T16:02:39.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.748+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b18007070 con 0x7f2b200c4a60 2023-10-21T16:02:39.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.749+0000 7f2b1d7fa640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 0x7f2b0006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.749+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2b18089d20 con 0x7f2b200c4a60 2023-10-21T16:02:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.750+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 0x7f2b0006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.751+0000 7f2b1ffff640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 0x7f2b0006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009c10 tx=0x7f2b10008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.754+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b200b2420 con 0x7f2b200c4a60 2023-10-21T16:02:39.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.990+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b200b2420 con 0x7f2b200c4a60 2023-10-21T16:02:39.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.991+0000 7f2b1d7fa640 1 -- 172.21.15.53:0/3623684101 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2b200b2420 con 0x7f2b200c4a60 2023-10-21T16:02:39.992 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-21T16:02:39.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.993+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 msgr2=0x7f2b0006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.993+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 0x7f2b0006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009c10 tx=0x7f2b10008040 comp rx=0 tx=0).stop 2023-10-21T16:02:39.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.993+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 msgr2=0x7f2b200abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:39.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.993+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b200b0c20 tx=0x7f2b1800b4a0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 shutdown_connections 2023-10-21T16:02:39.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0006aea0 0x7f2b0006d360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200c4a60 0x7f2b200abf20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c3840 0x7f2b201f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 --2- 172.21.15.53:0/3623684101 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b0c003f40 0x7f2b201f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:39.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 >> 172.21.15.53:0/3623684101 conn(0x7f2b2006a380 msgr2=0x7f2b200cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:39.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 shutdown_connections 2023-10-21T16:02:39.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:39.994+0000 7f2b263ce640 1 -- 172.21.15.53:0/3623684101 wait complete. 2023-10-21T16:02:40.016 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-21T16:02:40.017 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:02:40.017 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:02:40.017 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:02:40.018 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:02:40.018 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-21T16:02:40.018 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-21T16:02:40.018 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:02:40.019 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:02:40.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.019 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:02:40.076 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:40.076 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:40.077 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:40.077 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:02:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:02:40.078 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:02:40.078 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:02:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:02:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42333sec preferred_lft 42333sec 2023-10-21T16:02:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:02:40.079 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:02:40.081 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-21T16:02:40.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.082 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:40.082 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:40.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add name ceph-brx type bridge 2023-10-21T16:02:40.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr flush dev ceph-brx 2023-10-21T16:02:40.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set ceph-brx up 2023-10-21T16:02:40.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-21T16:02:40.083 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:40.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.324 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:02:40.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.417 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:02:40.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.475 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:40.475 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:40.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:02:40.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:02:40.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:02:40.476 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:40.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.674 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:02:40.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.733 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-10-21T16:02:40.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.791 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:40.791 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:40.791 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:02:40.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-21T16:02:40.792 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:40.943 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-21T16:02:40.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:40.944 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:40.944 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:40.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-21T16:02:40.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-21T16:02:40.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-21T16:02:40.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-21T16:02:40.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-21T16:02:40.946 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:41.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:41.210 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:41.211 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:41.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:02:41.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.0 master ceph-brx 2023-10-21T16:02:41.212 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:41.387 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:41.388 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:02:41.438 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:02:41.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:41.439 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:02:41.488 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:02:41.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:41.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:41.634 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:41.635 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:42.122 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:42.123 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:42.123 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:02:42.123 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:02:42.123 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:02:42.125 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:02:42.126 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:02:42.126 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:02:42.126 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:02:42.126 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:02:42.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:02:42.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:42.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:02:42.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:42.217 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:42.278 INFO:teuthology.orchestra.run.smithi053.stdout:client34505 2023-10-21T16:02:42.278 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:02:42.279 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:42.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:42.338 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:42.400 INFO:teuthology.orchestra.run.smithi053.stdout:client34505 2023-10-21T16:02:42.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:42.401 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:42.462 INFO:teuthology.orchestra.run.smithi053.stdout:client34505 2023-10-21T16:02:42.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:42.463 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.client34505/status') 2023-10-21T16:02:42.560 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.34505 (3)172.21.15.53:0/3264325898 2023-10-21T16:02:42.561 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2023-10-21T16:02:42.561 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2023-10-21T16:02:42.561 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2023-10-21T16:02:42.561 INFO:teuthology.orchestra.run.smithi053.stderr:71 bytes copied, 7.1872e-05 s, 988 kB/s 2023-10-21T16:02:42.564 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-21T16:02:42.565 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-21T16:02:42.565 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-21T16:02:42.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:02:42.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.782+0000 7f585c30f640 1 Processor -- start 2023-10-21T16:02:42.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.782+0000 7f585c30f640 1 -- start start 2023-10-21T16:02:42.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.784+0000 7f585c30f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5854159ac0 0x7f585406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.784+0000 7f585c30f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f58540709e0 0x7f58540c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.784+0000 7f585c30f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585406dee0 con 0x7f58540ad790 2023-10-21T16:02:42.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.784+0000 7f585c30f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585415d850 con 0x7f5854159ac0 2023-10-21T16:02:42.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.784+0000 7f585c30f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585415db30 con 0x7f58540709e0 2023-10-21T16:02:42.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5854159ac0 0x7f585406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5854159ac0 0x7f585406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55366/0 (socket says 172.21.15.53:55366) 2023-10-21T16:02:42.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 -- 172.21.15.53:0/2972165534 learned_addr learned my addr 172.21.15.53:0/2972165534 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:42.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f58598aa640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f58540ad790 0x7f58540702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36036/0 (socket says 172.21.15.53:36036) 2023-10-21T16:02:42.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a8ac640 1 --2- 172.21.15.53:0/2972165534 >> v2:172.21.15.124:3300/0 conn(0x7f58540709e0 0x7f58540c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 -- 172.21.15.53:0/2972165534 >> v2:172.21.15.124:3300/0 conn(0x7f58540709e0 msgr2=0x7f58540c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:42.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 --2- 172.21.15.53:0/2972165534 >> v2:172.21.15.124:3300/0 conn(0x7f58540709e0 0x7f58540c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 -- 172.21.15.53:0/2972165534 >> v1:172.21.15.53:6789/0 conn(0x7f58540ad790 legacy=0x7f58540702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:42.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f585a0ab640 1 -- 172.21.15.53:0/2972165534 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585415de10 con 0x7f5854159ac0 2023-10-21T16:02:42.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.785+0000 7f58590a9640 1 -- 172.21.15.53:0/2972165534 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757785128 0 0) 0x7f585406dee0 con 0x7f58540ad790 2023-10-21T16:02:42.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.786+0000 7f585a0ab640 1 --2- 172.21.15.53:0/2972165534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5854159ac0 0x7f585406d5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58540caab0 tx=0x7f585002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86ade5c7eb06698 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:42.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f58590a9640 1 -- 172.21.15.53:0/2972165534 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850030860 con 0x7f5854159ac0 2023-10-21T16:02:42.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f58590a9640 1 -- 172.21.15.53:0/2972165534 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58500309c0 con 0x7f5854159ac0 2023-10-21T16:02:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f58590a9640 1 -- 172.21.15.53:0/2972165534 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850030d50 con 0x7f5854159ac0 2023-10-21T16:02:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 -- 172.21.15.53:0/2972165534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5854159ac0 msgr2=0x7f585406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 --2- 172.21.15.53:0/2972165534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5854159ac0 0x7f585406d5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58540caab0 tx=0x7f585002b9a0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 -- 172.21.15.53:0/2972165534 shutdown_connections 2023-10-21T16:02:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 --2- 172.21.15.53:0/2972165534 >> v2:172.21.15.124:3300/0 conn(0x7f58540709e0 0x7f58540c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 --2- 172.21.15.53:0/2972165534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5854159ac0 0x7f585406d5e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 -- 172.21.15.53:0/2972165534 >> 172.21.15.53:0/2972165534 conn(0x7f585406a380 msgr2=0x7f58540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.787+0000 7f585c30f640 1 -- 172.21.15.53:0/2972165534 shutdown_connections 2023-10-21T16:02:42.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.788+0000 7f585c30f640 1 -- 172.21.15.53:0/2972165534 wait complete. 2023-10-21T16:02:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.789+0000 7f585c30f640 1 Processor -- start 2023-10-21T16:02:42.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.789+0000 7f585c30f640 1 -- start start 2023-10-21T16:02:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540ad790 0x7f58541582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854159ac0 0x7f58540cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854151630 con 0x7f58540709e0 2023-10-21T16:02:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854151910 con 0x7f58540ad790 2023-10-21T16:02:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.790+0000 7f585c30f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5854151bf0 con 0x7f5854159ac0 2023-10-21T16:02:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55382/0 (socket says 172.21.15.53:55382) 2023-10-21T16:02:42.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 -- 172.21.15.53:0/4107323821 learned_addr learned my addr 172.21.15.53:0/4107323821 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:42.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a8ac640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854159ac0 0x7f58540cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f58598aa640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540ad790 0x7f58541582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 -- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854159ac0 msgr2=0x7f58540cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:42.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854159ac0 0x7f58540cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 -- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540ad790 msgr2=0x7f58541582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:42.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540ad790 0x7f58541582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:42.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.791+0000 7f585a0ab640 1 -- 172.21.15.53:0/4107323821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58500094d0 con 0x7f58540709e0 2023-10-21T16:02:42.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f585a0ab640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f585002b9e0 tx=0x7f58500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:42.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585003e070 con 0x7f58540709e0 2023-10-21T16:02:42.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5850004030 con 0x7f58540709e0 2023-10-21T16:02:42.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58500043a0 con 0x7f58540709e0 2023-10-21T16:02:42.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854151ed0 con 0x7f58540709e0 2023-10-21T16:02:42.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.792+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585414f7d0 con 0x7f58540709e0 2023-10-21T16:02:42.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.793+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58541523f0 con 0x7f58540709e0 2023-10-21T16:02:42.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.794+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5850042070 con 0x7f58540709e0 2023-10-21T16:02:42.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.795+0000 7f584affd640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 0x7f583806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:42.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.795+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f58500ab860 con 0x7f58540709e0 2023-10-21T16:02:42.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.796+0000 7f58598aa640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 0x7f583806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:42.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.797+0000 7f58598aa640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 0x7f583806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5844007fd0 tx=0x7f5844009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:42.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:42.799+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58541523f0 con 0x7f58540709e0 2023-10-21T16:02:43.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.039+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58541523f0 con 0x7f58540709e0 2023-10-21T16:02:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.040+0000 7f584affd640 1 -- 172.21.15.53:0/4107323821 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5850034070 con 0x7f58540709e0 2023-10-21T16:02:43.042 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-21T16:02:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 msgr2=0x7f583806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 0x7f583806d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5844007fd0 tx=0x7f5844009040 comp rx=0 tx=0).stop 2023-10-21T16:02:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 msgr2=0x7f585414f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f585002b9e0 tx=0x7f58500046c0 comp rx=0 tx=0).stop 2023-10-21T16:02:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 shutdown_connections 2023-10-21T16:02:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854159ac0 0x7f58540cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540ad790 0x7f58541582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f583806ae70 0x7f583806d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 --2- 172.21.15.53:0/4107323821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58540709e0 0x7f585414f290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 >> 172.21.15.53:0/4107323821 conn(0x7f585406a380 msgr2=0x7f5854156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 shutdown_connections 2023-10-21T16:02:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:43.043+0000 7f585c30f640 1 -- 172.21.15.53:0/4107323821 wait complete. 2023-10-21T16:02:43.065 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-21T16:02:43.066 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:02:43.066 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:02:43.066 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-21T16:02:43.067 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-21T16:02:43.067 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-21T16:02:43.067 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-21T16:02:43.067 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:02:43.068 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-21T16:02:43.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.068 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:02:43.123 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:43.123 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:43.123 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:43.124 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.124 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:43.124 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.124 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:43.125 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:43.125 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:02:43.125 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:43.125 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:43.125 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:02:43.126 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:02:43.126 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:43.126 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:02:43.126 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:02:43.126 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-10-21T16:02:43.127 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:02:43.127 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.127 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:43.127 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:43.127 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:02:43.128 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.129 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:02:43.129 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:02:43.129 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link tentative 2023-10-21T16:02:43.129 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.130 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:02:43.211 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:02:43.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.215 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:02:43.272 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:02:43.272 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:02:43.272 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:02:43.273 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:02:43.273 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:02:43.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.274 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:43.274 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:43.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:02:43.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:02:43.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:02:43.275 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:43.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.469 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:02:43.522 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:02:43.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.524 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list-id 2023-10-21T16:02:43.577 INFO:teuthology.orchestra.run.smithi053.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-21T16:02:43.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.579 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:43.580 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:43.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:02:43.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-21T16:02:43.581 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:43.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-21T16:02:43.830 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:43.830 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:43.830 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:43.831 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.831 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:43.831 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.831 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:43.831 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether e2:11:d8:45:59:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-21T16:02:43.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-21T16:02:43.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e011:d8ff:fe45:5950/64 scope link 2023-10-21T16:02:43.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-21T16:02:43.942 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:43.942 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:43.943 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:43.943 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.943 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:43.943 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.944 INFO:teuthology.orchestra.run.smithi053.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:43.944 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether e2:11:d8:45:59:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-21T16:02:43.944 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-21T16:02:43.944 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.944 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e011:d8ff:fe45:5950/64 scope link 2023-10-21T16:02:43.945 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:43.949 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-21T16:02:43.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:43.950 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:43.950 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:43.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-21T16:02:43.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-21T16:02:43.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-21T16:02:43.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-21T16:02:43.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-21T16:02:43.952 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:44.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:44.231 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:02:44.231 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:02:44.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.1 up 2023-10-21T16:02:44.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set dev brx.1 master ceph-brx 2023-10-21T16:02:44.232 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:02:44.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:44.418 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-21T16:02:44.470 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-21T16:02:44.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:44.471 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-21T16:02:44.524 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-21T16:02:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:44.525 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:44.671 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:44.672 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:44.713 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:44.714 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:44.714 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:02:44.714 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 2@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:02:44.714 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:02:44.717 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-10-21T16:02:44.717 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:02:44.717 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:02:44.718 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:02:44.718 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:02:44.722 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:02:44.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:44.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-21T16:02:44.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:44.824 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:44.889 INFO:teuthology.orchestra.run.smithi053.stdout:client34508 2023-10-21T16:02:44.889 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:02:44.890 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:44.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:44.952 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:45.014 INFO:teuthology.orchestra.run.smithi053.stdout:client34508 2023-10-21T16:02:45.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:45.015 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:45.082 INFO:teuthology.orchestra.run.smithi053.stdout:client34508 2023-10-21T16:02:45.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:45.083 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.client34508/status') 2023-10-21T16:02:45.186 INFO:teuthology.orchestra.run.smithi053.stdout:instance: client.34508 (3)172.21.15.53:0/3472169862 2023-10-21T16:02:45.187 INFO:teuthology.orchestra.run.smithi053.stdout:blocklisted: false 2023-10-21T16:02:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records in 2023-10-21T16:02:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:0+1 records out 2023-10-21T16:02:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:71 bytes copied, 9.5199e-05 s, 746 kB/s 2023-10-21T16:02:45.191 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-21T16:02:45.191 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-21T16:02:45.191 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-21T16:02:45.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:02:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.419+0000 7f454504d640 1 Processor -- start 2023-10-21T16:02:45.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.419+0000 7f454504d640 1 -- start start 2023-10-21T16:02:45.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454504d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f454015cc40 0x7f45400cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454504d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454504d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400b4d20 con 0x7f45400d33f0 2023-10-21T16:02:45.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454504d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400d19e0 con 0x7f454015cc40 2023-10-21T16:02:45.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454504d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400d0510 con 0x7f45400b43b0 2023-10-21T16:02:45.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454484c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f453ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f454015cc40 0x7f45400cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454484c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:51384/0 (socket says 172.21.15.53:51384) 2023-10-21T16:02:45.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f454484c640 1 -- 172.21.15.53:0/1785849305 learned_addr learned my addr 172.21.15.53:0/1785849305 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:45.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.421+0000 7f453ffff640 1 --2- 172.21.15.53:0/1785849305 >> v2:172.21.15.124:3300/0 conn(0x7f454015cc40 0x7f45400cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:52178/0 (socket says 172.21.15.53:52178) 2023-10-21T16:02:45.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.422+0000 7f453ffff640 1 -- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 msgr2=0x7f45400b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:45.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.422+0000 7f453ffff640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.422+0000 7f453ffff640 1 -- 172.21.15.53:0/1785849305 >> v1:172.21.15.124:6789/0 conn(0x7f45400d33f0 legacy=0x7f45400d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.422+0000 7f453ffff640 1 -- 172.21.15.53:0/1785849305 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400d07f0 con 0x7f454015cc40 2023-10-21T16:02:45.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.422+0000 7f454484c640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:45.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.424+0000 7f453ffff640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f45400cdf80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45300097a0 tx=0x7f453002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b586758047a76b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:45.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.424+0000 7f453f7fe640 1 -- 172.21.15.53:0/1785849305 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530031830 con 0x7f454015cc40 2023-10-21T16:02:45.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.424+0000 7f453f7fe640 1 -- 172.21.15.53:0/1785849305 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530031990 con 0x7f454015cc40 2023-10-21T16:02:45.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.424+0000 7f453f7fe640 1 -- 172.21.15.53:0/1785849305 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530031d20 con 0x7f454015cc40 2023-10-21T16:02:45.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 -- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 msgr2=0x7f45400cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f45400cdf80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45300097a0 tx=0x7f453002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 -- 172.21.15.53:0/1785849305 shutdown_connections 2023-10-21T16:02:45.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45400b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 --2- 172.21.15.53:0/1785849305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f45400cdf80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 -- 172.21.15.53:0/1785849305 >> 172.21.15.53:0/1785849305 conn(0x7f454009bc60 msgr2=0x7f45400d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:45.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 -- 172.21.15.53:0/1785849305 shutdown_connections 2023-10-21T16:02:45.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.425+0000 7f454504d640 1 -- 172.21.15.53:0/1785849305 wait complete. 2023-10-21T16:02:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.427+0000 7f454504d640 1 Processor -- start 2023-10-21T16:02:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.427+0000 7f454504d640 1 -- start start 2023-10-21T16:02:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45401f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f454006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401f3e40 con 0x7f45400d33f0 2023-10-21T16:02:45.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401f7340 con 0x7f454015cc40 2023-10-21T16:02:45.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454504d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454015a410 con 0x7f45400b43b0 2023-10-21T16:02:45.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f45377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f45377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53254/0 (socket says 172.21.15.53:53254) 2023-10-21T16:02:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f45377fe640 1 -- 172.21.15.53:0/2640299361 learned_addr learned my addr 172.21.15.53:0/2640299361 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f453ffff640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45401f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.428+0000 7f454484c640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f454006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 -- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 msgr2=0x7f45401f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45401f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 -- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 msgr2=0x7f454006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f454006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 -- 172.21.15.53:0/2640299361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538009b30 con 0x7f45400d33f0 2023-10-21T16:02:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f453ffff640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45401f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f45377fe640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f452800c400 tx=0x7f452800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280052d0 con 0x7f45400d33f0 2023-10-21T16:02:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4528005430 con 0x7f45400d33f0 2023-10-21T16:02:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4530009400 con 0x7f45400d33f0 2023-10-21T16:02:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528003830 con 0x7f45400d33f0 2023-10-21T16:02:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.429+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45401f6aa0 con 0x7f45400d33f0 2023-10-21T16:02:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.431+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400c6050 con 0x7f45400d33f0 2023-10-21T16:02:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.432+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4528003990 con 0x7f45400d33f0 2023-10-21T16:02:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.434+0000 7f453d7fa640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.435+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4528088f30 con 0x7f45400d33f0 2023-10-21T16:02:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.435+0000 7f453ffff640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.436+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4528053d00 con 0x7f45400d33f0 2023-10-21T16:02:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.437+0000 7f453ffff640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 0x7f452006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4530036000 tx=0x7f453003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.674+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f45400c6050 con 0x7f45400d33f0 2023-10-21T16:02:45.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.675+0000 7f453d7fa640 1 -- 172.21.15.53:0/2640299361 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f45400c6050 con 0x7f45400d33f0 2023-10-21T16:02:45.677 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-21T16:02:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 msgr2=0x7f452006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 0x7f452006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4530036000 tx=0x7f453003c040 comp rx=0 tx=0).stop 2023-10-21T16:02:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 msgr2=0x7f4540159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f452800c400 tx=0x7f452800c8c0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 shutdown_connections 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f454015cc40 0x7f454006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45400d33f0 0x7f4540159ed0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 --2- 172.21.15.53:0/2640299361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45400b43b0 0x7f45401f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 >> 172.21.15.53:0/2640299361 conn(0x7f454009bc60 msgr2=0x7f454015f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.678+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 shutdown_connections 2023-10-21T16:02:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:45.679+0000 7f454504d640 1 -- 172.21.15.53:0/2640299361 wait complete. 2023-10-21T16:02:45.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-21T16:02:45.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:02:45.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2023-10-21T16:02:45.701 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-21T16:02:45.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-21T16:02:45.702 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-21T16:02:45.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-21T16:02:45.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:02:45.702 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-21T16:02:45.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:45.703 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2023-10-21T16:02:45.763 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:45.764 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:45.764 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:45.764 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:45.764 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:45.764 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:45.765 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:45.765 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:45.765 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2023-10-21T16:02:45.765 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:45.765 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:45.766 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2023-10-21T16:02:45.766 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:45.766 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:45.766 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:02:45.766 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:45.767 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-21T16:02:45.767 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42336sec preferred_lft 42336sec 2023-10-21T16:02:45.767 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2023-10-21T16:02:45.767 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:45.768 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-21T16:02:45.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:45.769 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:45.769 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:45.770 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add name ceph-brx type bridge 2023-10-21T16:02:45.770 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr flush dev ceph-brx 2023-10-21T16:02:45.770 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set ceph-brx up 2023-10-21T16:02:45.770 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-21T16:02:45.771 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:46.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:46.229 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:02:46.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:02:46.256 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:02:46.311 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-21T16:02:46.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:46.315 DEBUG:teuthology.orchestra.run.smithi124:> route 2023-10-21T16:02:46.426 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2023-10-21T16:02:46.426 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:02:46.426 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-21T16:02:46.426 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-21T16:02:46.427 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:02:46.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:46.427 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:46.428 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:46.428 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-21T16:02:46.428 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-21T16:02:46.429 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-21T16:02:46.429 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:47.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:47.196 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T16:02:47.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:47.256 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2023-10-21T16:02:47.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:47.312 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:47.312 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:47.313 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-21T16:02:47.313 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-21T16:02:47.314 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:47.458 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-21T16:02:47.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:47.459 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:47.459 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:47.459 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-21T16:02:47.460 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-21T16:02:47.460 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-21T16:02:47.460 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-21T16:02:47.461 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-21T16:02:47.461 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:47.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:47.755 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:47.755 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:47.755 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2023-10-21T16:02:47.756 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.0 master ceph-brx 2023-10-21T16:02:47.756 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:47.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:47.925 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-21T16:02:47.968 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-21T16:02:47.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:47.969 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-21T16:02:48.065 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-21T16:02:48.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:48.065 DEBUG:teuthology.orchestra.run.smithi124:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:48.215 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:48.215 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:48.706 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:48.707 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:48.707 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:02:48.707 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 1@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:02:48.707 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:02:48.711 INFO:teuthology.orchestra.run.smithi124.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-21T16:02:48.712 INFO:teuthology.orchestra.run.smithi124.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:02:48.712 INFO:teuthology.orchestra.run.smithi124.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:02:48.712 INFO:teuthology.orchestra.run.smithi124.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:02:48.712 INFO:teuthology.orchestra.run.smithi124.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:02:48.716 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:02:48.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:48.716 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-21T16:02:48.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:48.807 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:48.869 INFO:teuthology.orchestra.run.smithi124.stdout:client34514 2023-10-21T16:02:48.869 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:02:48.869 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:48.923 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:49.026 INFO:teuthology.orchestra.run.smithi124.stdout:client34514 2023-10-21T16:02:49.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.027 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:49.080 INFO:teuthology.orchestra.run.smithi124.stdout:client34514 2023-10-21T16:02:49.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.081 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.client34514/status') 2023-10-21T16:02:49.168 INFO:teuthology.orchestra.run.smithi124.stdout:instance: client.34514 (3)192.168.0.1:0/916431169 2023-10-21T16:02:49.169 INFO:teuthology.orchestra.run.smithi124.stdout:blocklisted: false 2023-10-21T16:02:49.169 INFO:teuthology.orchestra.run.smithi124.stderr:0+1 records in 2023-10-21T16:02:49.169 INFO:teuthology.orchestra.run.smithi124.stderr:0+1 records out 2023-10-21T16:02:49.170 INFO:teuthology.orchestra.run.smithi124.stderr:69 bytes copied, 6.6365e-05 s, 1.0 MB/s 2023-10-21T16:02:49.173 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-21T16:02:49.173 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-21T16:02:49.173 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-21T16:02:49.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:02:49.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.406+0000 7f1bab0bb640 1 Processor -- start 2023-10-21T16:02:49.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.406+0000 7f1bab0bb640 1 -- start start 2023-10-21T16:02:49.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1bab0bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1ba40ca620 0x7f1ba40b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1bab0bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba40cade0 con 0x7f1ba40cbb60 2023-10-21T16:02:49.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1bab0bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba40addf0 con 0x7f1ba40ada10 2023-10-21T16:02:49.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1bab0bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba406c270 con 0x7f1ba40ca620 2023-10-21T16:02:49.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1ba98b8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1ba40cbb60 0x7f1ba415d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38990/0 (socket says 172.21.15.53:38990) 2023-10-21T16:02:49.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1ba98b8640 1 -- 172.21.15.53:0/4025669045 learned_addr learned my addr 172.21.15.53:0/4025669045 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:49.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1baa8ba640 1 --2- 172.21.15.53:0/4025669045 >> v2:172.21.15.124:3300/0 conn(0x7f1ba40ca620 0x7f1ba40b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.408+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056226299 0 0) 0x7f1ba40cade0 con 0x7f1ba40cbb60 2023-10-21T16:02:49.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b90005730 con 0x7f1ba40cbb60 2023-10-21T16:02:49.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407908658 0 0) 0x7f1b90005730 con 0x7f1ba40cbb60 2023-10-21T16:02:49.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 >> v2:172.21.15.124:3300/0 conn(0x7f1ba40ca620 msgr2=0x7f1ba40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:49.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 --2- 172.21.15.53:0/4025669045 >> v2:172.21.15.124:3300/0 conn(0x7f1ba40ca620 0x7f1ba40b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 >> v1:172.21.15.124:6789/0 conn(0x7f1ba40ada10 legacy=0x7f1ba40cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba406c550 con 0x7f1ba40cbb60 2023-10-21T16:02:49.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1b94001670 con 0x7f1ba40cbb60 2023-10-21T16:02:49.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 >> v1:172.21.15.53:6789/0 conn(0x7f1ba40cbb60 legacy=0x7f1ba415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 0x7f1b90005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b900069d0 0x7f1b90006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b900073e0 0x7f1b9000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba406c270 con 0x7f1b90005bd0 2023-10-21T16:02:49.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9000e330 con 0x7f1b900073e0 2023-10-21T16:02:49.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.409+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9000e4f0 con 0x7f1b900069d0 2023-10-21T16:02:49.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 0x7f1b90005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1ba98b8640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b900069d0 0x7f1b90006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa8ba640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b900073e0 0x7f1b9000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 -- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b900069d0 msgr2=0x7f1b90006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:49.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b900069d0 0x7f1b90006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 -- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b900073e0 msgr2=0x7f1b9000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:49.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b900073e0 0x7f1b9000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa0b9640 1 -- 172.21.15.53:0/4025669045 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90005730 con 0x7f1b90005bd0 2023-10-21T16:02:49.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1baa8ba640 1 -- 172.21.15.53:0/4025669045 reap_dead start 2023-10-21T16:02:49.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.410+0000 7f1ba98b8640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b900069d0 0x7f1b90006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:49.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.411+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 0x7f1b90005ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0001720 tx=0x7f1b90007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee78787a94ca39 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:49.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.411+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00052d0 con 0x7f1b90005bd0 2023-10-21T16:02:49.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.411+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba002e050 con 0x7f1b90005bd0 2023-10-21T16:02:49.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.411+0000 7f1ba90b7640 1 -- 172.21.15.53:0/4025669045 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0003830 con 0x7f1b90005bd0 2023-10-21T16:02:49.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 -- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 msgr2=0x7f1b90005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 0x7f1b90005ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0001720 tx=0x7f1b90007a00 comp rx=0 tx=0).stop 2023-10-21T16:02:49.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 -- 172.21.15.53:0/4025669045 shutdown_connections 2023-10-21T16:02:49.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/4025669045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b90005bd0 0x7f1b90005ff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 -- 172.21.15.53:0/4025669045 >> 172.21.15.53:0/4025669045 conn(0x7f1ba40ce9d0 msgr2=0x7f1ba40cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:49.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 -- 172.21.15.53:0/4025669045 shutdown_connections 2023-10-21T16:02:49.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.412+0000 7f1bab0bb640 1 -- 172.21.15.53:0/4025669045 wait complete. 2023-10-21T16:02:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.414+0000 7f1bab0bb640 1 Processor -- start 2023-10-21T16:02:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.414+0000 7f1bab0bb640 1 -- start start 2023-10-21T16:02:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 0x7f1ba409b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 0x7f1ba40afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba409c1b0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba41ee520 con 0x7f1b90005bd0 2023-10-21T16:02:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1bab0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4156ef0 con 0x7f1ba41ef210 2023-10-21T16:02:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1ba98b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1ba98b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53268/0 (socket says 172.21.15.53:53268) 2023-10-21T16:02:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.415+0000 7f1ba98b8640 1 -- 172.21.15.53:0/2385778906 learned_addr learned my addr 172.21.15.53:0/2385778906 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:49.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1baa8ba640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 0x7f1ba40afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 0x7f1ba409b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 -- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 msgr2=0x7f1ba40afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 0x7f1ba40afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 -- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 msgr2=0x7f1ba409b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 0x7f1ba409b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 -- 172.21.15.53:0/2385778906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba41eed80 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1baa8ba640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 0x7f1ba40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:49.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 0x7f1ba409b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:49.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1ba98b8640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400d770 tx=0x7f1b9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:49.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940072d0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94007430 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940047e0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.416+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba40c4640 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.417+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba409bed0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.418+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba404e160 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.419+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b94004940 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.420+0000 7f1b9affd640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 0x7f1b8806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.420+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1b94089290 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.420+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 0x7f1b8806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:49.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.421+0000 7f1baa0b9640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 0x7f1b8806d320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0033000 tx=0x7f1ba0009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:49.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.424+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b94053ee0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.659+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ba415aff0 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.660+0000 7f1b9affd640 1 -- 172.21.15.53:0/2385778906 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1b94057500 con 0x7f1ba40ac7d0 2023-10-21T16:02:49.661 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-21T16:02:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 msgr2=0x7f1b8806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 0x7f1b8806d320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0033000 tx=0x7f1ba0009030 comp rx=0 tx=0).stop 2023-10-21T16:02:49.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 msgr2=0x7f1ba40af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400d770 tx=0x7f1b9400dc30 comp rx=0 tx=0).stop 2023-10-21T16:02:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 shutdown_connections 2023-10-21T16:02:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba41ef210 0x7f1ba40afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1b8806ae60 0x7f1b8806d320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.662+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba40ac7d0 0x7f1ba40af700 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.663+0000 7f1bab0bb640 1 --2- 172.21.15.53:0/2385778906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b90005bd0 0x7f1ba409b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:49.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.663+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 >> 172.21.15.53:0/2385778906 conn(0x7f1ba40ce9d0 msgr2=0x7f1ba40716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:49.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.663+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 shutdown_connections 2023-10-21T16:02:49.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:49.663+0000 7f1bab0bb640 1 -- 172.21.15.53:0/2385778906 wait complete. 2023-10-21T16:02:49.685 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-21T16:02:49.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:02:49.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2023-10-21T16:02:49.686 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-21T16:02:49.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-21T16:02:49.686 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-21T16:02:49.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-21T16:02:49.687 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:02:49.687 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-21T16:02:49.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.688 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2023-10-21T16:02:49.749 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:49.750 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:49.750 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:49.750 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.750 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:49.750 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.751 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:49.751 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:49.751 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2023-10-21T16:02:49.751 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:49.751 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:49.752 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2023-10-21T16:02:49.752 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:02:49.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:49.752 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:02:49.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:49.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-21T16:02:49.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42332sec preferred_lft 42332sec 2023-10-21T16:02:49.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2023-10-21T16:02:49.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.753 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:49.754 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 8e:70:44:83:79:21 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:02:49.754 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:02:49.754 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.754 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::c05f:10ff:fe7c:9b45/64 scope link 2023-10-21T16:02:49.755 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.755 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:02:49.755 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 8e:70:44:83:79:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-21T16:02:49.756 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::8c70:44ff:fe83:7921/64 scope link 2023-10-21T16:02:49.756 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:49.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.758 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:02:49.856 INFO:teuthology.orchestra.run.smithi124.stdout:1 2023-10-21T16:02:49.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.860 DEBUG:teuthology.orchestra.run.smithi124:> route 2023-10-21T16:02:49.917 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2023-10-21T16:02:49.918 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:02:49.918 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-21T16:02:49.918 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-21T16:02:49.918 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:02:49.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:49.919 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:49.919 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:49.920 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-21T16:02:49.920 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-21T16:02:49.920 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-21T16:02:49.920 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:50.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.106 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T16:02:50.158 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-21T16:02:50.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.159 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2023-10-21T16:02:50.211 INFO:teuthology.orchestra.run.smithi124.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-21T16:02:50.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.213 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:50.213 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:50.213 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-21T16:02:50.213 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-21T16:02:50.214 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:50.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.359 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-21T16:02:50.452 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.453 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:50.454 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 4e:e2:e9:72:ee:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-21T16:02:50.454 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-21T16:02:50.454 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.454 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4ce2:e9ff:fe72:eee7/64 scope link 2023-10-21T16:02:50.454 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-21T16:02:50.554 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:02:50.555 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:02:50.555 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:02:50.555 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.555 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2023-10-21T16:02:50.555 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.556 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:02:50.556 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 4e:e2:e9:72:ee:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-21T16:02:50.556 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-21T16:02:50.556 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.556 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4ce2:e9ff:fe72:eee7/64 scope link 2023-10-21T16:02:50.557 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:02:50.558 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-21T16:02:50.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.559 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:50.559 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:50.559 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-21T16:02:50.560 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-21T16:02:50.560 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-21T16:02:50.561 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-21T16:02:50.561 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-21T16:02:50.562 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:50.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:50.834 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2023-10-21T16:02:50.834 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2023-10-21T16:02:50.835 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.1 up 2023-10-21T16:02:50.835 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.1 master ceph-brx 2023-10-21T16:02:50.835 DEBUG:teuthology.orchestra.run.smithi124:> ') 2023-10-21T16:02:51.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:51.004 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-21T16:02:51.052 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-21T16:02:51.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:02:51.053 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-21T16:02:51.150 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-21T16:02:51.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.151 DEBUG:teuthology.orchestra.run.smithi124:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:51.279 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:51.280 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:51.312 INFO:teuthology.orchestra.run.smithi124.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:02:51.312 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:02:51.312 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:02:51.313 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: trying mount with new device syntax: 3@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:02:51.313 INFO:teuthology.orchestra.run.smithi124.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:02:51.314 INFO:teuthology.orchestra.run.smithi124.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-10-21T16:02:51.314 INFO:teuthology.orchestra.run.smithi124.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:02:51.314 INFO:teuthology.orchestra.run.smithi124.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:02:51.314 INFO:teuthology.orchestra.run.smithi124.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:02:51.315 INFO:teuthology.orchestra.run.smithi124.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:02:51.318 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:02:51.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.318 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-21T16:02:51.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.408 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:51.462 INFO:teuthology.orchestra.run.smithi124.stdout:client4714 2023-10-21T16:02:51.462 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:02:51.463 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:02:51.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.516 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:51.621 INFO:teuthology.orchestra.run.smithi124.stdout:client4714 2023-10-21T16:02:51.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.622 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:02:51.681 INFO:teuthology.orchestra.run.smithi124.stdout:client4714 2023-10-21T16:02:51.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:02:51.682 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.client4714/status') 2023-10-21T16:02:51.774 INFO:teuthology.orchestra.run.smithi124.stdout:instance: client.4714 (3)192.168.0.2:0/3835673624 2023-10-21T16:02:51.775 INFO:teuthology.orchestra.run.smithi124.stdout:blocklisted: false 2023-10-21T16:02:51.775 INFO:teuthology.orchestra.run.smithi124.stderr:0+1 records in 2023-10-21T16:02:51.775 INFO:teuthology.orchestra.run.smithi124.stderr:0+1 records out 2023-10-21T16:02:51.776 INFO:teuthology.orchestra.run.smithi124.stderr:69 bytes copied, 6.6759e-05 s, 1.0 MB/s 2023-10-21T16:02:51.778 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-21T16:02:51.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.026+0000 7f407dbc9640 1 Processor -- start 2023-10-21T16:02:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.027+0000 7f407dbc9640 1 -- start start 2023-10-21T16:02:52.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f407dbc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f407815ab70 0x7f40780cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f407dbc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f40780d04e0 0x7f40780cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f407dbc9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407815af50 con 0x7f40780cdca0 2023-10-21T16:02:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f407dbc9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780ce110 con 0x7f407815ab70 2023-10-21T16:02:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f407dbc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780ced20 con 0x7f40780d04e0 2023-10-21T16:02:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f40777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f407815ab70 0x7f40780cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f40777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f407815ab70 0x7f40780cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53274/0 (socket says 172.21.15.53:53274) 2023-10-21T16:02:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f40777fe640 1 -- 172.21.15.53:0/3878997515 learned_addr learned my addr 172.21.15.53:0/3878997515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f40777fe640 1 -- 172.21.15.53:0/3878997515 >> v2:172.21.15.124:3300/0 conn(0x7f40780d04e0 msgr2=0x7f40780cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.028+0000 7f40777fe640 1 --2- 172.21.15.53:0/3878997515 >> v2:172.21.15.124:3300/0 conn(0x7f40780d04e0 0x7f40780cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.029+0000 7f40777fe640 1 -- 172.21.15.53:0/3878997515 >> v1:172.21.15.124:6789/0 conn(0x7f40780cdca0 legacy=0x7f40780d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.029+0000 7f40777fe640 1 -- 172.21.15.53:0/3878997515 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780cf000 con 0x7f407815ab70 2023-10-21T16:02:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f40777fe640 1 --2- 172.21.15.53:0/3878997515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f40780cffa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40780ce440 tx=0x7f406802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905d9d03a26a8d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f40767fc640 1 -- 172.21.15.53:0/3878997515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406802ca80 con 0x7f407815ab70 2023-10-21T16:02:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f40767fc640 1 -- 172.21.15.53:0/3878997515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406802cbe0 con 0x7f407815ab70 2023-10-21T16:02:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f40767fc640 1 -- 172.21.15.53:0/3878997515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680343f0 con 0x7f407815ab70 2023-10-21T16:02:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f407dbc9640 1 -- 172.21.15.53:0/3878997515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 msgr2=0x7f40780cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.030+0000 7f407dbc9640 1 --2- 172.21.15.53:0/3878997515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f40780cffa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40780ce440 tx=0x7f406802bee0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 -- 172.21.15.53:0/3878997515 shutdown_connections 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 --2- 172.21.15.53:0/3878997515 >> v2:172.21.15.124:3300/0 conn(0x7f40780d04e0 0x7f40780cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 --2- 172.21.15.53:0/3878997515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f40780cffa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 -- 172.21.15.53:0/3878997515 >> 172.21.15.53:0/3878997515 conn(0x7f4078069f30 msgr2=0x7f407806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:52.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 -- 172.21.15.53:0/3878997515 shutdown_connections 2023-10-21T16:02:52.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.031+0000 7f407dbc9640 1 -- 172.21.15.53:0/3878997515 wait complete. 2023-10-21T16:02:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.032+0000 7f407dbc9640 1 Processor -- start 2023-10-21T16:02:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 -- start start 2023-10-21T16:02:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40780cdca0 0x7f40780c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40780d04e0 0x7f40780c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780c5650 con 0x7f407815ab70 2023-10-21T16:02:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407806be00 con 0x7f40780cdca0 2023-10-21T16:02:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.033+0000 7f407dbc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407806b840 con 0x7f40780d04e0 2023-10-21T16:02:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53282/0 (socket says 172.21.15.53:53282) 2023-10-21T16:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 -- 172.21.15.53:0/2436921126 learned_addr learned my addr 172.21.15.53:0/2436921126 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f40777fe640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40780cdca0 0x7f40780c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 -- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40780d04e0 msgr2=0x7f40780c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40780d04e0 0x7f40780c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 -- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40780cdca0 msgr2=0x7f40780c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40780cdca0 0x7f40780c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.034+0000 7f4077fff640 1 -- 172.21.15.53:0/2436921126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068009b30 con 0x7f407815ab70 2023-10-21T16:02:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f4077fff640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f407000c390 tx=0x7f407000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700035f0 con 0x7f407815ab70 2023-10-21T16:02:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4070003750 con 0x7f407815ab70 2023-10-21T16:02:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070003b00 con 0x7f407815ab70 2023-10-21T16:02:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407806bb20 con 0x7f407815ab70 2023-10-21T16:02:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.035+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407815c710 con 0x7f407815ab70 2023-10-21T16:02:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.036+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078066d80 con 0x7f407815ab70 2023-10-21T16:02:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.037+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4070003c60 con 0x7f407815ab70 2023-10-21T16:02:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.038+0000 7f405ffff640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 0x7f406406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.038+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4070088c70 con 0x7f407815ab70 2023-10-21T16:02:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.039+0000 7f40777fe640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 0x7f406406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.040+0000 7f40777fe640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 0x7f406406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4068035000 tx=0x7f4068030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.043+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4070053950 con 0x7f407815ab70 2023-10-21T16:02:52.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.237+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40780c4020 con 0x7f407815ab70 2023-10-21T16:02:52.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.239+0000 7f405ffff640 1 -- 172.21.15.53:0/2436921126 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f4070017090 con 0x7f407815ab70 2023-10-21T16:02:52.241 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:52.241 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.241+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 msgr2=0x7f406406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.241+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 0x7f406406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4068035000 tx=0x7f4068030000 comp rx=0 tx=0).stop 2023-10-21T16:02:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.241+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 msgr2=0x7f407806ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.241+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f407000c390 tx=0x7f407000c850 comp rx=0 tx=0).stop 2023-10-21T16:02:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 shutdown_connections 2023-10-21T16:02:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f406406ae70 0x7f406406d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f407815ab70 0x7f407806ae60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40780d04e0 0x7f40780c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 --2- 172.21.15.53:0/2436921126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40780cdca0 0x7f40780c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 >> 172.21.15.53:0/2436921126 conn(0x7f4078069f30 msgr2=0x7f40780b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:52.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 shutdown_connections 2023-10-21T16:02:52.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.242+0000 7f407dbc9640 1 -- 172.21.15.53:0/2436921126 wait complete. 2023-10-21T16:02:52.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:52.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.482+0000 7fd4ee48a640 1 Processor -- start 2023-10-21T16:02:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.482+0000 7fd4ee48a640 1 -- start start 2023-10-21T16:02:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ee48a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4e815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ee48a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80d3800 con 0x7fd4e80d06c0 2023-10-21T16:02:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ee48a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80ce370 con 0x7fd4e815cd80 2023-10-21T16:02:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ee48a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e815d160 con 0x7fd4e80d15d0 2023-10-21T16:02:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ecc87640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd4e80d06c0 0x7fd4e80d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38994/0 (socket says 172.21.15.53:38994) 2023-10-21T16:02:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4ecc87640 1 -- 172.21.15.53:0/4289951780 learned_addr learned my addr 172.21.15.53:0/4289951780 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4edc89640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4e815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099007841 0 0) 0x7fd4e80d3800 con 0x7fd4e80d06c0 2023-10-21T16:02:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.484+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d4003520 con 0x7fd4e80d06c0 2023-10-21T16:02:52.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624881019 0 0) 0x7fd4d4003520 con 0x7fd4e80d06c0 2023-10-21T16:02:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 msgr2=0x7fd4e815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4e815dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 >> v1:172.21.15.124:6789/0 conn(0x7fd4e815cd80 legacy=0x7fd4e815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e815ecf0 con 0x7fd4e80d06c0 2023-10-21T16:02:52.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd4d8001690 con 0x7fd4e80d06c0 2023-10-21T16:02:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 >> v1:172.21.15.53:6789/0 conn(0x7fd4e80d06c0 legacy=0x7fd4e80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d4004d00 0x7fd4d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e815d160 con 0x7fd4d40039a0 2023-10-21T16:02:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d400bb70 con 0x7fd4d4004d00 2023-10-21T16:02:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d400bd30 con 0x7fd4e80d15d0 2023-10-21T16:02:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.485+0000 7fd4ed488640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ecc87640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4edc89640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d4004d00 0x7fd4d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ed488640 1 -- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 msgr2=0x7fd4d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ed488640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80d15d0 0x7fd4d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ed488640 1 -- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d4004d00 msgr2=0x7fd4d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ed488640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d4004d00 0x7fd4d4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.486+0000 7fd4ed488640 1 -- 172.21.15.53:0/4289951780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4003520 con 0x7fd4d40039a0 2023-10-21T16:02:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.487+0000 7fd4ed488640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4d4003dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4001770 tx=0x7fd4d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cbdcfb81065e62 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.487+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40052d0 con 0x7fd4d40039a0 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.487+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e402e050 con 0x7fd4d40039a0 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.487+0000 7fd4dffff640 1 -- 172.21.15.53:0/4289951780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4003640 con 0x7fd4d40039a0 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 -- 172.21.15.53:0/4289951780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 msgr2=0x7fd4d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/4289951780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4d4003dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4001770 tx=0x7fd4d40057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4edc89640 1 -- 172.21.15.53:0/4289951780 reap_dead start 2023-10-21T16:02:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 -- 172.21.15.53:0/4289951780 shutdown_connections 2023-10-21T16:02:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 -- 172.21.15.53:0/4289951780 >> 172.21.15.53:0/4289951780 conn(0x7fd4e809bc40 msgr2=0x7fd4e80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 -- 172.21.15.53:0/4289951780 shutdown_connections 2023-10-21T16:02:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.488+0000 7fd4ee48a640 1 -- 172.21.15.53:0/4289951780 wait complete. 2023-10-21T16:02:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.490+0000 7fd4ee48a640 1 Processor -- start 2023-10-21T16:02:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.490+0000 7fd4ee48a640 1 -- start start 2023-10-21T16:02:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4e80c81c0 0x7fd4e80c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80cbb80 0x7fd4e80cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80cc540 con 0x7fd4d40039a0 2023-10-21T16:02:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80b6360 con 0x7fd4e80c81c0 2023-10-21T16:02:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ee48a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80b6640 con 0x7fd4e80cbb80 2023-10-21T16:02:52.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53296/0 (socket says 172.21.15.53:53296) 2023-10-21T16:02:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 -- 172.21.15.53:0/3439363277 learned_addr learned my addr 172.21.15.53:0/3439363277 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ecc87640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4e80c81c0 0x7fd4e80c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4edc89640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80cbb80 0x7fd4e80cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 -- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80cbb80 msgr2=0x7fd4e80cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80cbb80 0x7fd4e80cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 -- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4e80c81c0 msgr2=0x7fd4e80c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4e80c81c0 0x7fd4e80c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.491+0000 7fd4ed488640 1 -- 172.21.15.53:0/3439363277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80b6920 con 0x7fd4d40039a0 2023-10-21T16:02:52.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ed488640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4e402bc00 tx=0x7fd4e4001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4006e60 con 0x7fd4d40039a0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4007040 con 0x7fd4d40039a0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e40073f0 con 0x7fd4d40039a0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80b6c00 con 0x7fd4d40039a0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.492+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e80b5fa0 con 0x7fd4d40039a0 2023-10-21T16:02:52.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.493+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8074150 con 0x7fd4d40039a0 2023-10-21T16:02:52.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.494+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4e4007550 con 0x7fd4d40039a0 2023-10-21T16:02:52.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.496+0000 7fd4ddffb640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 0x7fd4cc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.496+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd4e40ab7f0 con 0x7fd4d40039a0 2023-10-21T16:02:52.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.497+0000 7fd4ecc87640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 0x7fd4cc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.498+0000 7fd4ecc87640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 0x7fd4cc06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8001bd0 tx=0x7fd4d8000f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.501+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4e4076440 con 0x7fd4d40039a0 2023-10-21T16:02:52.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.696+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4e80b88f0 con 0x7fd4d40039a0 2023-10-21T16:02:52.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.697+0000 7fd4ddffb640 1 -- 172.21.15.53:0/3439363277 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4e4079a60 con 0x7fd4d40039a0 2023-10-21T16:02:52.698 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.699+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 msgr2=0x7fd4cc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.699+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 0x7fd4cc06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8001bd0 tx=0x7fd4d8000f50 comp rx=0 tx=0).stop 2023-10-21T16:02:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.699+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 msgr2=0x7fd4e8148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.699+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4e402bc00 tx=0x7fd4e4001170 comp rx=0 tx=0).stop 2023-10-21T16:02:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 shutdown_connections 2023-10-21T16:02:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4e80cbb80 0x7fd4e80cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4e80c81c0 0x7fd4e80c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd4cc06aea0 0x7fd4cc06d360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 --2- 172.21.15.53:0/3439363277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40039a0 0x7fd4e8148f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 >> 172.21.15.53:0/3439363277 conn(0x7fd4e809bc40 msgr2=0x7fd4e815ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 shutdown_connections 2023-10-21T16:02:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.700+0000 7fd4ee48a640 1 -- 172.21.15.53:0/3439363277 wait complete. 2023-10-21T16:02:52.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:52.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.955+0000 7f9f84b83640 1 Processor -- start 2023-10-21T16:02:52.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.955+0000 7f9f84b83640 1 -- start start 2023-10-21T16:02:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.956+0000 7f9f84b83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f80160660 0x7f9f80070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.956+0000 7f9f84b83640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f800c9910 0x7f9f800caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f84b83640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8006e150 con 0x7f9f800cb450 2023-10-21T16:02:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f84b83640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f800cb8c0 con 0x7f9f800c9910 2023-10-21T16:02:52.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f84b83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8006de70 con 0x7f9f80160660 2023-10-21T16:02:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f7ed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9f800cb450 0x7f9f800b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39004/0 (socket says 172.21.15.53:39004) 2023-10-21T16:02:52.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/1333619972 learned_addr learned my addr 172.21.15.53:0/1333619972 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f80160660 0x7f9f80070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.957+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> v2:172.21.15.124:3300/0 conn(0x7f9f800c9910 0x7f9f800caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929881819 0 0) 0x7f9f8006e150 con 0x7f9f800cb450 2023-10-21T16:02:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f6c003520 con 0x7f9f800cb450 2023-10-21T16:02:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423850340 0 0) 0x7f9f6c003520 con 0x7f9f800cb450 2023-10-21T16:02:52.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f80160660 msgr2=0x7f9f80070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f80160660 0x7f9f80070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 >> v2:172.21.15.124:3300/0 conn(0x7f9f800c9910 msgr2=0x7f9f800caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 --2- 172.21.15.53:0/1333619972 >> v2:172.21.15.124:3300/0 conn(0x7f9f800c9910 0x7f9f800caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800c8e30 con 0x7f9f800cb450 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9f78001670 con 0x7f9f800cb450 2023-10-21T16:02:52.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 >> v1:172.21.15.53:6789/0 conn(0x7f9f800cb450 legacy=0x7f9f800b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c003970 0x7f9f6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6c0047c0 0x7f9f6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6c005180 0x7f9f6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800cb8c0 con 0x7f9f6c0047c0 2023-10-21T16:02:52.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8006de70 con 0x7f9f6c005180 2023-10-21T16:02:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c00bec0 con 0x7f9f6c003970 2023-10-21T16:02:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f80160660 0x7f9f80070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:52.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.958+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> v2:172.21.15.124:3300/0 conn(0x7f9f800c9910 0x7f9f800caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:02:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6c0047c0 0x7f9f6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7ed9d640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6c005180 0x7f9f6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c003970 0x7f9f6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 -- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c003970 msgr2=0x7f9f6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c003970 0x7f9f6c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 -- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6c005180 msgr2=0x7f9f6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6c005180 0x7f9f6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7dd9b640 1 -- 172.21.15.53:0/1333619972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c003520 con 0x7f9f6c0047c0 2023-10-21T16:02:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.959+0000 7f9f7ed9d640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6c005180 0x7f9f6c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:52.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.960+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6c0047c0 0x7f9f6c004c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800e380 tx=0x7f9f6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbba765a2e7a5a02 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6802cde0 con 0x7f9f6c0047c0 2023-10-21T16:02:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6800f040 con 0x7f9f6c0047c0 2023-10-21T16:02:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f7d59a640 1 -- 172.21.15.53:0/1333619972 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68003b90 con 0x7f9f6c0047c0 2023-10-21T16:02:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f84b83640 1 -- 172.21.15.53:0/1333619972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6c0047c0 msgr2=0x7f9f6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f84b83640 1 --2- 172.21.15.53:0/1333619972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6c0047c0 0x7f9f6c004c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800e380 tx=0x7f9f6c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/1333619972 reap_dead start 2023-10-21T16:02:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f84b83640 1 -- 172.21.15.53:0/1333619972 shutdown_connections 2023-10-21T16:02:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f84b83640 1 -- 172.21.15.53:0/1333619972 >> 172.21.15.53:0/1333619972 conn(0x7f9f8006a5d0 msgr2=0x7f9f800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:52.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.961+0000 7f9f84b83640 1 -- 172.21.15.53:0/1333619972 shutdown_connections 2023-10-21T16:02:52.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.962+0000 7f9f84b83640 1 -- 172.21.15.53:0/1333619972 wait complete. 2023-10-21T16:02:52.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.963+0000 7f9f84b83640 1 Processor -- start 2023-10-21T16:02:52.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.963+0000 7f9f84b83640 1 -- start start 2023-10-21T16:02:52.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c0047c0 0x7f9f80150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8015c860 0x7f9f8015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80158180 con 0x7f9f801f5b30 2023-10-21T16:02:52.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8015d220 con 0x7f9f8015c860 2023-10-21T16:02:52.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.964+0000 7f9f84b83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f801f6a90 con 0x7f9f6c0047c0 2023-10-21T16:02:52.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53306/0 (socket says 172.21.15.53:53306) 2023-10-21T16:02:52.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/3878187972 learned_addr learned my addr 172.21.15.53:0/3878187972 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:52.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7dd9b640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8015c860 0x7f9f8015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c0047c0 msgr2=0x7f9f80150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c0047c0 0x7f9f80150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c0047c0 0x7f9f80150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8015c860 msgr2=0x7f9f8015cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:52.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8015c860 0x7f9f8015cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:52.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.965+0000 7f9f7ed9d640 1 -- 172.21.15.53:0/3878187972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f801f64f0 con 0x7f9f801f5b30 2023-10-21T16:02:52.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f7ed9d640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b1760 tx=0x7f9f7800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007440 con 0x7f9f801f5b30 2023-10-21T16:02:52.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f780075a0 con 0x7f9f801f5b30 2023-10-21T16:02:52.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004a00 con 0x7f9f801f5b30 2023-10-21T16:02:52.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f801f67b0 con 0x7f9f801f5b30 2023-10-21T16:02:52.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.966+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f80151cf0 con 0x7f9f801f5b30 2023-10-21T16:02:52.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.967+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f800b0f60 con 0x7f9f801f5b30 2023-10-21T16:02:52.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.968+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f78004b60 con 0x7f9f801f5b30 2023-10-21T16:02:52.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.969+0000 7f9f76ffd640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 0x7f9f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:52.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.969+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9f78089660 con 0x7f9f801f5b30 2023-10-21T16:02:52.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.970+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 0x7f9f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:52.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.971+0000 7f9f7e59c640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 0x7f9f6006d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b0d80 tx=0x7f9f70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:52.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:52.973+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f780542b0 con 0x7f9f801f5b30 2023-10-21T16:02:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.170+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f8004e160 con 0x7f9f801f5b30 2023-10-21T16:02:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.171+0000 7f9f76ffd640 1 -- 172.21.15.53:0/3878187972 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f780578d0 con 0x7f9f801f5b30 2023-10-21T16:02:53.173 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:53.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.173+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 msgr2=0x7f9f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.173+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 0x7f9f6006d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b0d80 tx=0x7f9f70008040 comp rx=0 tx=0).stop 2023-10-21T16:02:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.173+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 msgr2=0x7f9f801f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.173+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b1760 tx=0x7f9f7800d9b0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 shutdown_connections 2023-10-21T16:02:53.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9f6006ae70 0x7f9f6006d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f801f5b30 0x7f9f801f5fb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8015c860 0x7f9f8015cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 --2- 172.21.15.53:0/3878187972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6c0047c0 0x7f9f80150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 >> 172.21.15.53:0/3878187972 conn(0x7f9f8006a5d0 msgr2=0x7f9f8015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 shutdown_connections 2023-10-21T16:02:53.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.174+0000 7f9f84b83640 1 -- 172.21.15.53:0/3878187972 wait complete. 2023-10-21T16:02:53.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:53.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.413+0000 7f253461f640 1 Processor -- start 2023-10-21T16:02:53.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.413+0000 7f253461f640 1 -- start start 2023-10-21T16:02:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f253461f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f252c0cb150 0x7f252c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f253461f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f253461f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c06e1e0 con 0x7f252c161690 2023-10-21T16:02:53.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f253461f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cbb40 con 0x7f252c0cb150 2023-10-21T16:02:53.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f253461f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c06cc50 con 0x7f252c0b1f20 2023-10-21T16:02:53.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f25323bb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f252c161690 0x7f252c06dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39014/0 (socket says 172.21.15.53:39014) 2023-10-21T16:02:53.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f25323bb640 1 -- 172.21.15.53:0/2098275469 learned_addr learned my addr 172.21.15.53:0/2098275469 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:53.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f2531bba640 1 --2- 172.21.15.53:0/2098275469 >> v2:172.21.15.124:3300/0 conn(0x7f252c0cb150 0x7f252c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.415+0000 7f2532bbc640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f25313b9640 1 -- 172.21.15.53:0/2098275469 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605310205 0 0) 0x7f252c06e1e0 con 0x7f252c161690 2023-10-21T16:02:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f25313b9640 1 -- 172.21.15.53:0/2098275469 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2518003520 con 0x7f252c161690 2023-10-21T16:02:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f2531bba640 1 -- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 msgr2=0x7f252c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:53.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f2531bba640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f2531bba640 1 -- 172.21.15.53:0/2098275469 >> v1:172.21.15.53:6789/0 conn(0x7f252c161690 legacy=0x7f252c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.416+0000 7f2531bba640 1 -- 172.21.15.53:0/2098275469 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c06cf30 con 0x7f252c0cb150 2023-10-21T16:02:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.417+0000 7f2531bba640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0cb600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25200097a0 tx=0x7f252002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a747c60a15b607 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.418+0000 7f25313b9640 1 -- 172.21.15.53:0/2098275469 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252002d040 con 0x7f252c0cb150 2023-10-21T16:02:53.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.418+0000 7f25313b9640 1 -- 172.21.15.53:0/2098275469 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520031810 con 0x7f252c0cb150 2023-10-21T16:02:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.418+0000 7f25313b9640 1 -- 172.21.15.53:0/2098275469 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520031b80 con 0x7f252c0cb150 2023-10-21T16:02:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 -- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 msgr2=0x7f252c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0cb600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25200097a0 tx=0x7f252002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 -- 172.21.15.53:0/2098275469 shutdown_connections 2023-10-21T16:02:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 --2- 172.21.15.53:0/2098275469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0cb600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 -- 172.21.15.53:0/2098275469 >> 172.21.15.53:0/2098275469 conn(0x7f252c06a5d0 msgr2=0x7f252c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 -- 172.21.15.53:0/2098275469 shutdown_connections 2023-10-21T16:02:53.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.419+0000 7f253461f640 1 -- 172.21.15.53:0/2098275469 wait complete. 2023-10-21T16:02:53.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.421+0000 7f253461f640 1 Processor -- start 2023-10-21T16:02:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.421+0000 7f253461f640 1 -- start start 2023-10-21T16:02:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c162780 con 0x7f252c161690 2023-10-21T16:02:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c162a60 con 0x7f252c0cb150 2023-10-21T16:02:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f253461f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c162d40 con 0x7f252c0b1f20 2023-10-21T16:02:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f2532bbc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f2532bbc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53322/0 (socket says 172.21.15.53:53322) 2023-10-21T16:02:53.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.422+0000 7f2532bbc640 1 -- 172.21.15.53:0/2687846868 learned_addr learned my addr 172.21.15.53:0/2687846868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2531bba640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f25323bb640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 -- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 msgr2=0x7f252c152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 -- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 msgr2=0x7f252c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 -- 172.21.15.53:0/2687846868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520009400 con 0x7f252c161690 2023-10-21T16:02:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f2532bbc640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f252400efc0 tx=0x7f252400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524009ba0 con 0x7f252c161690 2023-10-21T16:02:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400f040 con 0x7f252c161690 2023-10-21T16:02:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.423+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240146c0 con 0x7f252c161690 2023-10-21T16:02:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.424+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c163020 con 0x7f252c161690 2023-10-21T16:02:53.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.424+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c153030 con 0x7f252c161690 2023-10-21T16:02:53.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.425+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c153440 con 0x7f252c161690 2023-10-21T16:02:53.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.426+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2524014820 con 0x7f252c161690 2023-10-21T16:02:53.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.427+0000 7f251effd640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.427+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f252408a5d0 con 0x7f252c161690 2023-10-21T16:02:53.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.428+0000 7f25323bb640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.429+0000 7f25323bb640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 0x7f251006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2528001200 tx=0x7f2528009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.431+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f252c153440 con 0x7f252c161690 2023-10-21T16:02:53.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.622+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f252c153440 con 0x7f252c161690 2023-10-21T16:02:53.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.623+0000 7f251effd640 1 -- 172.21.15.53:0/2687846868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f252c153440 con 0x7f252c161690 2023-10-21T16:02:53.624 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 msgr2=0x7f251006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 0x7f251006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2528001200 tx=0x7f2528009040 comp rx=0 tx=0).stop 2023-10-21T16:02:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 msgr2=0x7f252c0c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f252400efc0 tx=0x7f252400c490 comp rx=0 tx=0).stop 2023-10-21T16:02:53.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 shutdown_connections 2023-10-21T16:02:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f252c161690 0x7f252c0c7d60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f252c0cb150 0x7f252c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 --2- 172.21.15.53:0/2687846868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f252c0b1f20 0x7f252c152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 >> 172.21.15.53:0/2687846868 conn(0x7f252c06a5d0 msgr2=0x7f252c160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:53.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 shutdown_connections 2023-10-21T16:02:53.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.625+0000 7f253461f640 1 -- 172.21.15.53:0/2687846868 wait complete. 2023-10-21T16:02:53.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.868+0000 7fc45135e640 1 Processor -- start 2023-10-21T16:02:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.868+0000 7fc45135e640 1 -- start start 2023-10-21T16:02:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.870+0000 7fc45135e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.870+0000 7fc45135e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0d2870 con 0x7fc44c0d00e0 2023-10-21T16:02:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.870+0000 7fc45135e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0d1150 con 0x7fc44c15ccb0 2023-10-21T16:02:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.870+0000 7fc45135e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0d1430 con 0x7fc44c0d0b90 2023-10-21T16:02:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53330/0 (socket says 172.21.15.53:53330) 2023-10-21T16:02:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc443fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc44c0d00e0 0x7fc44c0b6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39020/0 (socket says 172.21.15.53:39020) 2023-10-21T16:02:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 -- 172.21.15.53:0/1513720088 learned_addr learned my addr 172.21.15.53:0/1513720088 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 -- 172.21.15.53:0/1513720088 >> v1:172.21.15.124:6789/0 conn(0x7fc44c15ccb0 legacy=0x7fc44c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 -- 172.21.15.53:0/1513720088 >> v1:172.21.15.53:6789/0 conn(0x7fc44c0d00e0 legacy=0x7fc44c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc450b5d640 1 -- 172.21.15.53:0/1513720088 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0d1710 con 0x7fc44c0d0b90 2023-10-21T16:02:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.871+0000 7fc44b7fe640 1 -- 172.21.15.53:0/1513720088 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429933404 0 0) 0x7fc44c0d2870 con 0x7fc44c0d00e0 2023-10-21T16:02:53.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc450b5d640 1 --2- 172.21.15.53:0/1513720088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e80 tx=0x7fc44402bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d30f4dcdd9cf43 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc44b7fe640 1 -- 172.21.15.53:0/1513720088 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44402cb80 con 0x7fc44c0d0b90 2023-10-21T16:02:53.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc44b7fe640 1 -- 172.21.15.53:0/1513720088 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44402cce0 con 0x7fc44c0d0b90 2023-10-21T16:02:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc44b7fe640 1 -- 172.21.15.53:0/1513720088 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444034610 con 0x7fc44c0d0b90 2023-10-21T16:02:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc45135e640 1 -- 172.21.15.53:0/1513720088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 msgr2=0x7fc44c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc45135e640 1 --2- 172.21.15.53:0/1513720088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e80 tx=0x7fc44402bfe0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc45135e640 1 -- 172.21.15.53:0/1513720088 shutdown_connections 2023-10-21T16:02:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.873+0000 7fc45135e640 1 --2- 172.21.15.53:0/1513720088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0d1fb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.874+0000 7fc45135e640 1 -- 172.21.15.53:0/1513720088 >> 172.21.15.53:0/1513720088 conn(0x7fc44c071660 msgr2=0x7fc44c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.874+0000 7fc45135e640 1 -- 172.21.15.53:0/1513720088 shutdown_connections 2023-10-21T16:02:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.874+0000 7fc45135e640 1 -- 172.21.15.53:0/1513720088 wait complete. 2023-10-21T16:02:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.875+0000 7fc45135e640 1 Processor -- start 2023-10-21T16:02:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.876+0000 7fc45135e640 1 -- start start 2023-10-21T16:02:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc44c0d00e0 0x7fc44c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc44c15ccb0 0x7fc44c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c8aa0 con 0x7fc44c0d0b90 2023-10-21T16:02:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c7a00 con 0x7fc44c15ccb0 2023-10-21T16:02:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc45135e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c7ce0 con 0x7fc44c0d00e0 2023-10-21T16:02:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53336/0 (socket says 172.21.15.53:53336) 2023-10-21T16:02:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 -- 172.21.15.53:0/2256422774 learned_addr learned my addr 172.21.15.53:0/2256422774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc450b5d640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc44c15ccb0 0x7fc44c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 -- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc44c0d00e0 msgr2=0x7fc44c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc44c0d00e0 0x7fc44c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 -- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc44c15ccb0 msgr2=0x7fc44c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc44c15ccb0 0x7fc44c0b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.877+0000 7fc443fff640 1 -- 172.21.15.53:0/2256422774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009b30 con 0x7fc44c0d0b90 2023-10-21T16:02:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc443fff640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c4a0 tx=0x7fc43400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340074e0 con 0x7fc44c0d0b90 2023-10-21T16:02:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434007640 con 0x7fc44c0d0b90 2023-10-21T16:02:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434004a00 con 0x7fc44c0d0b90 2023-10-21T16:02:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c0c7fc0 con 0x7fc44c0d0b90 2023-10-21T16:02:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.878+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c0c8580 con 0x7fc44c0d0b90 2023-10-21T16:02:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.879+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c074150 con 0x7fc44c0d0b90 2023-10-21T16:02:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.880+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc434004b60 con 0x7fc44c0d0b90 2023-10-21T16:02:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.881+0000 7fc4497fa640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 0x7fc42c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.881+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc434088bc0 con 0x7fc44c0d0b90 2023-10-21T16:02:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.882+0000 7fc44bfff640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 0x7fc42c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.883+0000 7fc44bfff640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 0x7fc42c06d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f60 tx=0x7fc43c0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:53.885+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc434053810 con 0x7fc44c0d0b90 2023-10-21T16:02:54.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.081+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc44c0c8740 con 0x7fc44c0d0b90 2023-10-21T16:02:54.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.081+0000 7fc4497fa640 1 -- 172.21.15.53:0/2256422774 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc434018020 con 0x7fc44c0d0b90 2023-10-21T16:02:54.084 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:54.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 msgr2=0x7fc42c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 0x7fc42c06d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000f60 tx=0x7fc43c0079b0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 msgr2=0x7fc44c0c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c4a0 tx=0x7fc43400c960 comp rx=0 tx=0).stop 2023-10-21T16:02:54.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 shutdown_connections 2023-10-21T16:02:54.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc44c15ccb0 0x7fc44c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc42c06ae70 0x7fc42c06d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc44c0d0b90 0x7fc44c0c7100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 --2- 172.21.15.53:0/2256422774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc44c0d00e0 0x7fc44c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 >> 172.21.15.53:0/2256422774 conn(0x7fc44c071660 msgr2=0x7fc44c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:54.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 shutdown_connections 2023-10-21T16:02:54.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.084+0000 7fc45135e640 1 -- 172.21.15.53:0/2256422774 wait complete. 2023-10-21T16:02:54.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:54.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.314+0000 7efd2d9ff640 1 Processor -- start 2023-10-21T16:02:54.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.314+0000 7efd2d9ff640 1 -- start start 2023-10-21T16:02:54.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.315+0000 7efd2d9ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd28069d50 0x7efd280cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.315+0000 7efd2d9ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cc5b0 con 0x7efd2806a130 2023-10-21T16:02:54.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.315+0000 7efd2d9ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cc890 con 0x7efd280b4fb0 2023-10-21T16:02:54.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.315+0000 7efd2d9ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd280b18a0 con 0x7efd28069d50 2023-10-21T16:02:54.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd27fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efd2806a130 0x7efd280b1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43668/0 (socket says 172.21.15.53:43668) 2023-10-21T16:02:54.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd27fff640 1 -- 172.21.15.53:0/3574952185 learned_addr learned my addr 172.21.15.53:0/3574952185 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:54.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd2c9fd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd28069d50 0x7efd280cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd277fe640 1 -- 172.21.15.53:0/3574952185 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025503154 0 0) 0x7efd280cc5b0 con 0x7efd2806a130 2023-10-21T16:02:54.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd277fe640 1 -- 172.21.15.53:0/3574952185 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd18003520 con 0x7efd2806a130 2023-10-21T16:02:54.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd2c9fd640 1 -- 172.21.15.53:0/3574952185 >> v1:172.21.15.124:6789/0 conn(0x7efd280b4fb0 legacy=0x7efd2815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd2c9fd640 1 -- 172.21.15.53:0/3574952185 >> v1:172.21.15.53:6789/0 conn(0x7efd2806a130 legacy=0x7efd280b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.316+0000 7efd2c9fd640 1 -- 172.21.15.53:0/3574952185 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280ccb70 con 0x7efd28069d50 2023-10-21T16:02:54.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.318+0000 7efd2c9fd640 1 --2- 172.21.15.53:0/3574952185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28069d50 0x7efd280cbfb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009d90 tx=0x7efd1c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52495bd65334aa82 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.318+0000 7efd277fe640 1 -- 172.21.15.53:0/3574952185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c02cd00 con 0x7efd28069d50 2023-10-21T16:02:54.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.318+0000 7efd277fe640 1 -- 172.21.15.53:0/3574952185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c02ce60 con 0x7efd28069d50 2023-10-21T16:02:54.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.318+0000 7efd277fe640 1 -- 172.21.15.53:0/3574952185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c034660 con 0x7efd28069d50 2023-10-21T16:02:54.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3574952185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28069d50 msgr2=0x7efd280cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3574952185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28069d50 0x7efd280cbfb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009d90 tx=0x7efd1c02bf50 comp rx=0 tx=0).stop 2023-10-21T16:02:54.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3574952185 shutdown_connections 2023-10-21T16:02:54.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3574952185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd28069d50 0x7efd280cbfb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3574952185 >> 172.21.15.53:0/3574952185 conn(0x7efd2806ae60 msgr2=0x7efd2806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:54.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3574952185 shutdown_connections 2023-10-21T16:02:54.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.319+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3574952185 wait complete. 2023-10-21T16:02:54.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.321+0000 7efd2d9ff640 1 Processor -- start 2023-10-21T16:02:54.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.321+0000 7efd2d9ff640 1 -- start start 2023-10-21T16:02:54.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd28069d50 0x7efd281e1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd280b4fb0 0x7efd280ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281e1970 con 0x7efd2806a130 2023-10-21T16:02:54.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281e1c50 con 0x7efd280b4fb0 2023-10-21T16:02:54.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d9ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd281e1f30 con 0x7efd28069d50 2023-10-21T16:02:54.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60124/0 (socket says 172.21.15.53:60124) 2023-10-21T16:02:54.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 -- 172.21.15.53:0/3294069098 learned_addr learned my addr 172.21.15.53:0/3294069098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2c9fd640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd28069d50 0x7efd281e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 -- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd28069d50 msgr2=0x7efd281e1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd28069d50 0x7efd281e1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd27fff640 1 -- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd280b4fb0 msgr2=0x7efd280ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.322+0000 7efd2d1fe640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd280b4fb0 0x7efd280ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd27fff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd280b4fb0 0x7efd280ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd27fff640 1 -- 172.21.15.53:0/3294069098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c009a70 con 0x7efd2806a130 2023-10-21T16:02:54.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd27fff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c500 tx=0x7efd1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14005ce0 con 0x7efd2806a130 2023-10-21T16:02:54.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14005e40 con 0x7efd2806a130 2023-10-21T16:02:54.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14007330 con 0x7efd2806a130 2023-10-21T16:02:54.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd281e2210 con 0x7efd2806a130 2023-10-21T16:02:54.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.323+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd281e15b0 con 0x7efd2806a130 2023-10-21T16:02:54.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.324+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2815a520 con 0x7efd2806a130 2023-10-21T16:02:54.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.325+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd14007490 con 0x7efd2806a130 2023-10-21T16:02:54.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.326+0000 7efd257fa640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 0x7efd0c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.326+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efd14089070 con 0x7efd2806a130 2023-10-21T16:02:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.327+0000 7efd2c9fd640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 0x7efd0c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.328+0000 7efd2c9fd640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 0x7efd0c06d360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd1c036000 tx=0x7efd1c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.330+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd140bbca0 con 0x7efd2806a130 2023-10-21T16:02:54.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.514+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd2815a240 con 0x7efd2806a130 2023-10-21T16:02:54.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.516+0000 7efd257fa640 1 -- 172.21.15.53:0/3294069098 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7efd14017020 con 0x7efd2806a130 2023-10-21T16:02:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:54.519 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:54.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.518+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 msgr2=0x7efd0c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.518+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 0x7efd0c06d360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd1c036000 tx=0x7efd1c030000 comp rx=0 tx=0).stop 2023-10-21T16:02:54.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.518+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 msgr2=0x7efd2814e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.518+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd1400c500 tx=0x7efd1400c9c0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 shutdown_connections 2023-10-21T16:02:54.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd280b4fb0 0x7efd280ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efd0c06aea0 0x7efd0c06d360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd2806a130 0x7efd2814e900 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 --2- 172.21.15.53:0/3294069098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd28069d50 0x7efd281e1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 >> 172.21.15.53:0/3294069098 conn(0x7efd2806ae60 msgr2=0x7efd280cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:54.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 shutdown_connections 2023-10-21T16:02:54.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.519+0000 7efd2d9ff640 1 -- 172.21.15.53:0/3294069098 wait complete. 2023-10-21T16:02:54.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:54.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.756+0000 7f6b31399640 1 Processor -- start 2023-10-21T16:02:54.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.756+0000 7f6b31399640 1 -- start start 2023-10-21T16:02:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b31399640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6b2c15dcd0 0x7f6b2c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b31399640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0ce290 con 0x7f6b2c0d1120 2023-10-21T16:02:54.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b31399640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0d0a80 con 0x7f6b2c0d1500 2023-10-21T16:02:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b31399640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b1fe0 con 0x7f6b2c15dcd0 2023-10-21T16:02:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b2bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6b2c0d1120 0x7f6b2c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43678/0 (socket says 172.21.15.53:43678) 2023-10-21T16:02:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b2bfff640 1 -- 172.21.15.53:0/704770321 learned_addr learned my addr 172.21.15.53:0/704770321 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888986222 0 0) 0x7f6b2c0ce290 con 0x7f6b2c0d1120 2023-10-21T16:02:54.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b18003520 con 0x7f6b2c0d1120 2023-10-21T16:02:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.758+0000 7f6b30b98640 1 --2- 172.21.15.53:0/704770321 >> v2:172.21.15.124:3300/0 conn(0x7f6b2c15dcd0 0x7f6b2c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 269458985 0 0) 0x7f6b18003520 con 0x7f6b2c0d1120 2023-10-21T16:02:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 >> v2:172.21.15.124:3300/0 conn(0x7f6b2c15dcd0 msgr2=0x7f6b2c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 --2- 172.21.15.53:0/704770321 >> v2:172.21.15.124:3300/0 conn(0x7f6b2c15dcd0 0x7f6b2c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 >> v1:172.21.15.124:6789/0 conn(0x7f6b2c0d1500 legacy=0x7f6b2c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0b4580 con 0x7f6b2c0d1120 2023-10-21T16:02:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6b1c001b70 con 0x7f6b2c0d1120 2023-10-21T16:02:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 >> v1:172.21.15.53:6789/0 conn(0x7f6b2c0d1120 legacy=0x7f6b2c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b180039c0 0x7f6b18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180047c0 0x7f6b18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 0x7f6b1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0b1fe0 con 0x7f6b180051d0 2023-10-21T16:02:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1800c120 con 0x7f6b180047c0 2023-10-21T16:02:54.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.759+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1800c2e0 con 0x7f6b180039c0 2023-10-21T16:02:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 0x7f6b1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b2bfff640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b180039c0 0x7f6b18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b2b7fe640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180047c0 0x7f6b18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 -- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b180039c0 msgr2=0x7f6b18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:54.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b180039c0 0x7f6b18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 -- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180047c0 msgr2=0x7f6b18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180047c0 0x7f6b18004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.760+0000 7f6b30b98640 1 -- 172.21.15.53:0/704770321 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18003520 con 0x7f6b180051d0 2023-10-21T16:02:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.761+0000 7f6b30b98640 1 -- 172.21.15.53:0/704770321 reap_dead start 2023-10-21T16:02:54.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.761+0000 7f6b30b98640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 0x7f6b1800b980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009a90 tx=0x7f6b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cdcf24207e9787 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.761+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24004020 con 0x7f6b180051d0 2023-10-21T16:02:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.761+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b24004180 con 0x7f6b180051d0 2023-10-21T16:02:54.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.761+0000 7f6b2affd640 1 -- 172.21.15.53:0/704770321 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240044f0 con 0x7f6b180051d0 2023-10-21T16:02:54.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 -- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 msgr2=0x7f6b1800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 0x7f6b1800b980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009a90 tx=0x7f6b180057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 -- 172.21.15.53:0/704770321 shutdown_connections 2023-10-21T16:02:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 --2- 172.21.15.53:0/704770321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b180051d0 0x7f6b1800b980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 -- 172.21.15.53:0/704770321 >> 172.21.15.53:0/704770321 conn(0x7f6b2c0d2960 msgr2=0x7f6b2c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 -- 172.21.15.53:0/704770321 shutdown_connections 2023-10-21T16:02:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.762+0000 7f6b31399640 1 -- 172.21.15.53:0/704770321 wait complete. 2023-10-21T16:02:54.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.764+0000 7f6b31399640 1 Processor -- start 2023-10-21T16:02:54.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.764+0000 7f6b31399640 1 -- start start 2023-10-21T16:02:54.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180051d0 0x7f6b2c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 0x7f6b2c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c15fd00 con 0x7f6b2c15bac0 2023-10-21T16:02:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c7800 con 0x7f6b180051d0 2023-10-21T16:02:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b31399640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c8620 con 0x7f6b2c15f310 2023-10-21T16:02:54.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60142/0 (socket says 172.21.15.53:60142) 2023-10-21T16:02:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.765+0000 7f6b2b7fe640 1 -- 172.21.15.53:0/2064786949 learned_addr learned my addr 172.21.15.53:0/2064786949 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2bfff640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180051d0 0x7f6b2c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b30b98640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 0x7f6b2c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 -- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 msgr2=0x7f6b2c15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 0x7f6b2c15f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 -- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180051d0 msgr2=0x7f6b2c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180051d0 0x7f6b2c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 -- 172.21.15.53:0/2064786949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c156400 con 0x7f6b2c15bac0 2023-10-21T16:02:54.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b30b98640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 0x7f6b2c15f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b2b7fe640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e410 tx=0x7f6b1400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140052e0 con 0x7f6b2c15bac0 2023-10-21T16:02:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b14005440 con 0x7f6b2c15bac0 2023-10-21T16:02:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.766+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c15b550 con 0x7f6b2c15bac0 2023-10-21T16:02:54.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.767+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14003810 con 0x7f6b2c15bac0 2023-10-21T16:02:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.767+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0c7cf0 con 0x7f6b2c15bac0 2023-10-21T16:02:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.768+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0b4b20 con 0x7f6b2c15bac0 2023-10-21T16:02:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.769+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b14003970 con 0x7f6b2c15bac0 2023-10-21T16:02:54.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.770+0000 7f6b28ff9640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 0x7f6b0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.770+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6b14089190 con 0x7f6b2c15bac0 2023-10-21T16:02:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.770+0000 7f6b2bfff640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 0x7f6b0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.771+0000 7f6b2bfff640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 0x7f6b0c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002230 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:54.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.774+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b2c0b4b20 con 0x7f6b2c15bac0 2023-10-21T16:02:54.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.958+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b2c0b4b20 con 0x7f6b2c15bac0 2023-10-21T16:02:54.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.959+0000 7f6b28ff9640 1 -- 172.21.15.53:0/2064786949 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b14019020 con 0x7f6b2c15bac0 2023-10-21T16:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 msgr2=0x7f6b0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 0x7f6b0c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c002230 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2023-10-21T16:02:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 msgr2=0x7f6b2c15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:54.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e410 tx=0x7f6b1400e8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 shutdown_connections 2023-10-21T16:02:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b2c15f310 0x7f6b2c15f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6b0c06ae70 0x7f6b0c06d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b2c15bac0 0x7f6b2c15b010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 --2- 172.21.15.53:0/2064786949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b180051d0 0x7f6b2c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:54.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 >> 172.21.15.53:0/2064786949 conn(0x7f6b2c0d2960 msgr2=0x7f6b2c160820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:54.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 shutdown_connections 2023-10-21T16:02:54.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:54.962+0000 7f6b31399640 1 -- 172.21.15.53:0/2064786949 wait complete. 2023-10-21T16:02:54.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.201+0000 7f5f3490c640 1 Processor -- start 2023-10-21T16:02:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.202+0000 7f5f3490c640 1 -- start start 2023-10-21T16:02:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.203+0000 7f5f3490c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f3006dfc0 0x7f5f30070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.203+0000 7f5f3490c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300b1cc0 con 0x7f5f300afbe0 2023-10-21T16:02:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.203+0000 7f5f3490c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f300b2620 con 0x7f5f30071070 2023-10-21T16:02:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f3490c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f300b2060 con 0x7f5f3006dfc0 2023-10-21T16:02:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f25d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f3006dfc0 0x7f5f30070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f2e59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5f300afbe0 0x7f5f300b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43694/0 (socket says 172.21.15.53:43694) 2023-10-21T16:02:55.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f2e59c640 1 -- 172.21.15.53:0/1246275597 learned_addr learned my addr 172.21.15.53:0/1246275597 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f25d9b640 1 -- 172.21.15.53:0/1246275597 >> v1:172.21.15.124:6789/0 conn(0x7f5f30071070 legacy=0x7f5f300cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:55.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f25d9b640 1 -- 172.21.15.53:0/1246275597 >> v1:172.21.15.53:6789/0 conn(0x7f5f300afbe0 legacy=0x7f5f300b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f25d9b640 1 -- 172.21.15.53:0/1246275597 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300b2340 con 0x7f5f3006dfc0 2023-10-21T16:02:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.204+0000 7f5f2dd9b640 1 -- 172.21.15.53:0/1246275597 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762935509 0 0) 0x7f5f300b1cc0 con 0x7f5f300afbe0 2023-10-21T16:02:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.205+0000 7f5f25d9b640 1 --2- 172.21.15.53:0/1246275597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f30070b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f300b2810 tx=0x7f5f1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66487b94a864277 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.206+0000 7f5f2dd9b640 1 -- 172.21.15.53:0/1246275597 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18031830 con 0x7f5f3006dfc0 2023-10-21T16:02:55.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.206+0000 7f5f2dd9b640 1 -- 172.21.15.53:0/1246275597 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18031990 con 0x7f5f3006dfc0 2023-10-21T16:02:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.206+0000 7f5f2dd9b640 1 -- 172.21.15.53:0/1246275597 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18031d20 con 0x7f5f3006dfc0 2023-10-21T16:02:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.206+0000 7f5f3490c640 1 -- 172.21.15.53:0/1246275597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 msgr2=0x7f5f30070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.206+0000 7f5f3490c640 1 --2- 172.21.15.53:0/1246275597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f30070b30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f300b2810 tx=0x7f5f1802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.207+0000 7f5f3490c640 1 -- 172.21.15.53:0/1246275597 shutdown_connections 2023-10-21T16:02:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.207+0000 7f5f3490c640 1 --2- 172.21.15.53:0/1246275597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f30070b30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.207+0000 7f5f3490c640 1 -- 172.21.15.53:0/1246275597 >> 172.21.15.53:0/1246275597 conn(0x7f5f3006a5d0 msgr2=0x7f5f300b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.207+0000 7f5f3490c640 1 -- 172.21.15.53:0/1246275597 shutdown_connections 2023-10-21T16:02:55.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.207+0000 7f5f3490c640 1 -- 172.21.15.53:0/1246275597 wait complete. 2023-10-21T16:02:55.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.208+0000 7f5f3490c640 1 Processor -- start 2023-10-21T16:02:55.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.209+0000 7f5f3490c640 1 -- start start 2023-10-21T16:02:55.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.209+0000 7f5f3490c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.209+0000 7f5f3490c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f30071070 0x7f5f300cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f3490c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f300afbe0 0x7f5f300ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f3490c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3014f7a0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f3490c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300c3d30 con 0x7f5f30071070 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f3490c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300c4010 con 0x7f5f300afbe0 2023-10-21T16:02:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60164/0 (socket says 172.21.15.53:60164) 2023-10-21T16:02:55.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 -- 172.21.15.53:0/937415013 learned_addr learned my addr 172.21.15.53:0/937415013 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:55.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 -- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f300afbe0 msgr2=0x7f5f300ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f300afbe0 0x7f5f300ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 -- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f30071070 msgr2=0x7f5f300cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f30071070 0x7f5f300cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.210+0000 7f5f2e59c640 1 -- 172.21.15.53:0/937415013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009400 con 0x7f5f3006dfc0 2023-10-21T16:02:55.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f2e59c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c4d0 tx=0x7f5f2000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200074e0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f20007660 con 0x7f5f3006dfc0 2023-10-21T16:02:55.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004a20 con 0x7f5f3006dfc0 2023-10-21T16:02:55.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300c42f0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.211+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f300c48b0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.212+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3004e160 con 0x7f5f3006dfc0 2023-10-21T16:02:55.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.213+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f20004b80 con 0x7f5f3006dfc0 2023-10-21T16:02:55.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.214+0000 7f5f277fe640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 0x7f5f1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.214+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f20088b70 con 0x7f5f3006dfc0 2023-10-21T16:02:55.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.215+0000 7f5f25d9b640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 0x7f5f1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.216+0000 7f5f25d9b640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 0x7f5f1006d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f18036000 tx=0x7f5f18030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.219+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f20053850 con 0x7f5f3006dfc0 2023-10-21T16:02:55.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.411+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f300b0ef0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.412+0000 7f5f277fe640 1 -- 172.21.15.53:0/937415013 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f200180a0 con 0x7f5f3006dfc0 2023-10-21T16:02:55.413 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 msgr2=0x7f5f1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 0x7f5f1006d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f18036000 tx=0x7f5f18030000 comp rx=0 tx=0).stop 2023-10-21T16:02:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 msgr2=0x7f5f300c3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000c4d0 tx=0x7f5f2000c990 comp rx=0 tx=0).stop 2023-10-21T16:02:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 shutdown_connections 2023-10-21T16:02:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f300afbe0 0x7f5f300ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f30071070 0x7f5f300cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f1006ae70 0x7f5f1006d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 --2- 172.21.15.53:0/937415013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3006dfc0 0x7f5f300c3350 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 >> 172.21.15.53:0/937415013 conn(0x7f5f3006a5d0 msgr2=0x7f5f3006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 shutdown_connections 2023-10-21T16:02:55.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.414+0000 7f5f3490c640 1 -- 172.21.15.53:0/937415013 wait complete. 2023-10-21T16:02:55.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.664+0000 7fafb1d6a640 1 Processor -- start 2023-10-21T16:02:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.664+0000 7fafb1d6a640 1 -- start start 2023-10-21T16:02:55.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafb1d6a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7fafac0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafb1d6a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac0c94f0 con 0x7fafac0c8b40 2023-10-21T16:02:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafb1d6a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac0ccbc0 con 0x7fafac15cae0 2023-10-21T16:02:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafb1d6a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0ce1f0 con 0x7fafac0b0680 2023-10-21T16:02:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafabfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fafac0c8b40 0x7fafac0aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43706/0 (socket says 172.21.15.53:43706) 2023-10-21T16:02:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafabfff640 1 -- 172.21.15.53:0/4049298516 learned_addr learned my addr 172.21.15.53:0/4049298516 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafb1569640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7fafac0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.666+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204977264 0 0) 0x7fafac0c94f0 con 0x7fafac0c8b40 2023-10-21T16:02:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf94003520 con 0x7fafac0c8b40 2023-10-21T16:02:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2074792595 0 0) 0x7faf94003520 con 0x7fafac0c8b40 2023-10-21T16:02:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 msgr2=0x7fafac0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7fafac0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 >> v1:172.21.15.124:6789/0 conn(0x7fafac15cae0 legacy=0x7fafac0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0c9720 con 0x7fafac0c8b40 2023-10-21T16:02:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fafa0001670 con 0x7fafac0c8b40 2023-10-21T16:02:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 >> v1:172.21.15.53:6789/0 conn(0x7fafac0c8b40 legacy=0x7fafac0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7faf940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf94004d00 0x7faf94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0ce1f0 con 0x7faf940039a0 2023-10-21T16:02:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9400ba60 con 0x7faf94004d00 2023-10-21T16:02:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9400bc20 con 0x7fafac0b0680 2023-10-21T16:02:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.667+0000 7fafb0d68640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb1569640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf94004d00 0x7faf94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb0d68640 1 -- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 msgr2=0x7faf940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafabfff640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7faf940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb0d68640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0b0680 0x7faf940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb0d68640 1 -- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf94004d00 msgr2=0x7faf94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb0d68640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf94004d00 0x7faf94005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb0d68640 1 -- 172.21.15.53:0/4049298516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94003520 con 0x7faf940039a0 2023-10-21T16:02:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.668+0000 7fafb1569640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf94004d00 0x7faf94005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.669+0000 7fafb0d68640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf9c001720 tx=0x7faf940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b556602a1e301156 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.669+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0052d0 con 0x7faf940039a0 2023-10-21T16:02:55.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.669+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c02e050 con 0x7faf940039a0 2023-10-21T16:02:55.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.669+0000 7fafab7fe640 1 -- 172.21.15.53:0/4049298516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0035f0 con 0x7faf940039a0 2023-10-21T16:02:55.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 -- 172.21.15.53:0/4049298516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940039a0 msgr2=0x7faf94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/4049298516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940039a0 0x7faf94003dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf9c001720 tx=0x7faf940057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1569640 1 -- 172.21.15.53:0/4049298516 reap_dead start 2023-10-21T16:02:55.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 -- 172.21.15.53:0/4049298516 shutdown_connections 2023-10-21T16:02:55.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 -- 172.21.15.53:0/4049298516 >> 172.21.15.53:0/4049298516 conn(0x7fafac06c050 msgr2=0x7fafac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:55.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 -- 172.21.15.53:0/4049298516 shutdown_connections 2023-10-21T16:02:55.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.670+0000 7fafb1d6a640 1 -- 172.21.15.53:0/4049298516 wait complete. 2023-10-21T16:02:55.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.672+0000 7fafb1d6a640 1 Processor -- start 2023-10-21T16:02:55.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.672+0000 7fafb1d6a640 1 -- start start 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940039a0 0x7fafac15a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0c3ee0 0x7fafac0c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0c48d0 con 0x7fafac0c7ed0 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac15aa90 con 0x7faf940039a0 2023-10-21T16:02:55.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafb1d6a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0c5760 con 0x7fafac0c3ee0 2023-10-21T16:02:55.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafabfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafabfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60184/0 (socket says 172.21.15.53:60184) 2023-10-21T16:02:55.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.673+0000 7fafabfff640 1 -- 172.21.15.53:0/2796540632 learned_addr learned my addr 172.21.15.53:0/2796540632 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:55.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 -- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0c3ee0 msgr2=0x7fafac0c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0c3ee0 0x7fafac0c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 -- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940039a0 msgr2=0x7fafac15a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940039a0 0x7fafac15a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 -- 172.21.15.53:0/2796540632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0c5a40 con 0x7fafac0c7ed0 2023-10-21T16:02:55.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafabfff640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafa000d7a0 tx=0x7fafa000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa00074e0 con 0x7fafac0c7ed0 2023-10-21T16:02:55.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0007640 con 0x7fafac0c7ed0 2023-10-21T16:02:55.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0004a20 con 0x7fafac0c7ed0 2023-10-21T16:02:55.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.674+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac0c5d20 con 0x7fafac0c7ed0 2023-10-21T16:02:55.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.675+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac0c8350 con 0x7fafac0c7ed0 2023-10-21T16:02:55.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.676+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac0c55a0 con 0x7fafac0c7ed0 2023-10-21T16:02:55.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.677+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fafa0004b80 con 0x7fafac0c7ed0 2023-10-21T16:02:55.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.678+0000 7fafa97fa640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 0x7faf9806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:55.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.678+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fafa00893c0 con 0x7fafac0c7ed0 2023-10-21T16:02:55.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.678+0000 7fafb0d68640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 0x7faf9806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:55.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.679+0000 7fafb0d68640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 0x7faf9806d3b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf9c033000 tx=0x7faf9c02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:55.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.682+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafa0054240 con 0x7fafac0c7ed0 2023-10-21T16:02:55.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.877+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafac074150 con 0x7fafac0c7ed0 2023-10-21T16:02:55.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.879+0000 7fafa97fa640 1 -- 172.21.15.53:0/2796540632 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafa0057860 con 0x7fafac0c7ed0 2023-10-21T16:02:55.880 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:55.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.881+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 msgr2=0x7faf9806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.881+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 0x7faf9806d3b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf9c033000 tx=0x7faf9c02f000 comp rx=0 tx=0).stop 2023-10-21T16:02:55.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.881+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 msgr2=0x7fafac1f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:55.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.881+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fafa000d7a0 tx=0x7fafa000dc60 comp rx=0 tx=0).stop 2023-10-21T16:02:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 shutdown_connections 2023-10-21T16:02:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafac0c3ee0 0x7fafac0c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faf9806aef0 0x7faf9806d3b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafac0c7ed0 0x7fafac1f2a80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 --2- 172.21.15.53:0/2796540632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940039a0 0x7fafac15a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 >> 172.21.15.53:0/2796540632 conn(0x7fafac06c050 msgr2=0x7fafac0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 shutdown_connections 2023-10-21T16:02:55.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:55.882+0000 7fafb1d6a640 1 -- 172.21.15.53:0/2796540632 wait complete. 2023-10-21T16:02:55.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:56.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.133+0000 7f36b4885640 1 Processor -- start 2023-10-21T16:02:56.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.133+0000 7f36b4885640 1 -- start start 2023-10-21T16:02:56.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36b4885640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 0x7f36b00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36b4885640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36b00c8b40 0x7f36b00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36b4885640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b00ccbc0 con 0x7f36b00aff70 2023-10-21T16:02:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36b4885640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00c9510 con 0x7f36b00c8b40 2023-10-21T16:02:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36b4885640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00ce1f0 con 0x7f36b015cae0 2023-10-21T16:02:56.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36aeffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36b00c8b40 0x7f36b00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36aeffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36b00c8b40 0x7f36b00c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60188/0 (socket says 172.21.15.53:60188) 2023-10-21T16:02:56.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36aeffd640 1 -- 172.21.15.53:0/2883253765 learned_addr learned my addr 172.21.15.53:0/2883253765 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:56.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.135+0000 7f36af7fe640 1 --2- 172.21.15.53:0/2883253765 >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 0x7f36b00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.136+0000 7f36aeffd640 1 -- 172.21.15.53:0/2883253765 >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 msgr2=0x7f36b00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:56.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.136+0000 7f36aeffd640 1 --2- 172.21.15.53:0/2883253765 >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 0x7f36b00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.136+0000 7f36aeffd640 1 -- 172.21.15.53:0/2883253765 >> v1:172.21.15.124:6789/0 conn(0x7f36b00aff70 legacy=0x7f36b00b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:56.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.136+0000 7f36aeffd640 1 -- 172.21.15.53:0/2883253765 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00cd590 con 0x7f36b00c8b40 2023-10-21T16:02:56.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.136+0000 7f36af7fe640 1 --2- 172.21.15.53:0/2883253765 >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 0x7f36b00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:56.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.137+0000 7f36aeffd640 1 --2- 172.21.15.53:0/2883253765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b00c8b40 0x7f36b00c8fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f36b00caed0 tx=0x7f36a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33116d5cae00a9eb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.137+0000 7f36ae7fc640 1 -- 172.21.15.53:0/2883253765 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4031870 con 0x7f36b00c8b40 2023-10-21T16:02:56.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.137+0000 7f36ae7fc640 1 -- 172.21.15.53:0/2883253765 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a40319d0 con 0x7f36b00c8b40 2023-10-21T16:02:56.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.137+0000 7f36ae7fc640 1 -- 172.21.15.53:0/2883253765 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4031d60 con 0x7f36b00c8b40 2023-10-21T16:02:56.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 -- 172.21.15.53:0/2883253765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b00c8b40 msgr2=0x7f36b00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 --2- 172.21.15.53:0/2883253765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b00c8b40 0x7f36b00c8fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f36b00caed0 tx=0x7f36a402b910 comp rx=0 tx=0).stop 2023-10-21T16:02:56.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 -- 172.21.15.53:0/2883253765 shutdown_connections 2023-10-21T16:02:56.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 --2- 172.21.15.53:0/2883253765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b00c8b40 0x7f36b00c8fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 --2- 172.21.15.53:0/2883253765 >> v2:172.21.15.124:3300/0 conn(0x7f36b015cae0 0x7f36b00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 -- 172.21.15.53:0/2883253765 >> 172.21.15.53:0/2883253765 conn(0x7f36b006c050 msgr2=0x7f36b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:56.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 -- 172.21.15.53:0/2883253765 shutdown_connections 2023-10-21T16:02:56.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.138+0000 7f36b4885640 1 -- 172.21.15.53:0/2883253765 wait complete. 2023-10-21T16:02:56.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.141+0000 7f36b4885640 1 Processor -- start 2023-10-21T16:02:56.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.141+0000 7f36b4885640 1 -- start start 2023-10-21T16:02:56.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b00aff70 0x7f36b0157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b00c8b40 0x7f36b00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00b1f90 con 0x7f36b015cae0 2023-10-21T16:02:56.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00c55c0 con 0x7f36b00aff70 2023-10-21T16:02:56.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36b4885640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00c58a0 con 0x7f36b00c8b40 2023-10-21T16:02:56.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.142+0000 7f36affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60198/0 (socket says 172.21.15.53:60198) 2023-10-21T16:02:56.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 -- 172.21.15.53:0/2804412623 learned_addr learned my addr 172.21.15.53:0/2804412623 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:56.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36af7fe640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b00aff70 0x7f36b0157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36aeffd640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b00c8b40 0x7f36b00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 -- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b00c8b40 msgr2=0x7f36b00c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b00c8b40 0x7f36b00c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 -- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b00aff70 msgr2=0x7f36b0157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b00aff70 0x7f36b0157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 -- 172.21.15.53:0/2804412623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40094d0 con 0x7f36b015cae0 2023-10-21T16:02:56.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.143+0000 7f36affff640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c780 tx=0x7f36a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.144+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004b60 con 0x7f36b015cae0 2023-10-21T16:02:56.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.144+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a800ce90 con 0x7f36b015cae0 2023-10-21T16:02:56.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.144+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b00c5b80 con 0x7f36b015cae0 2023-10-21T16:02:56.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.144+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800a850 con 0x7f36b015cae0 2023-10-21T16:02:56.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.144+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b00c6140 con 0x7f36b015cae0 2023-10-21T16:02:56.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.145+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0074150 con 0x7f36b015cae0 2023-10-21T16:02:56.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.146+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36a800a9b0 con 0x7f36b015cae0 2023-10-21T16:02:56.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.147+0000 7f3697fff640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 0x7f369c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.147+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f36a8089b30 con 0x7f36b015cae0 2023-10-21T16:02:56.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.148+0000 7f36af7fe640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 0x7f369c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.149+0000 7f36af7fe640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 0x7f369c06d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a950 tx=0x7f36a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.152+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36a8054820 con 0x7f36b015cae0 2023-10-21T16:02:56.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.349+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36b00afbc0 con 0x7f36b015cae0 2023-10-21T16:02:56.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.350+0000 7f3697fff640 1 -- 172.21.15.53:0/2804412623 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36a8057e40 con 0x7f36b015cae0 2023-10-21T16:02:56.352 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:56.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 msgr2=0x7f369c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 0x7f369c06d360 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a950 tx=0x7f36a0008040 comp rx=0 tx=0).stop 2023-10-21T16:02:56.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 msgr2=0x7f36b00c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c780 tx=0x7f36a800cc40 comp rx=0 tx=0).stop 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 shutdown_connections 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369c06aea0 0x7f369c06d360 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b015cae0 0x7f36b00c7a40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b00c8b40 0x7f36b00c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 --2- 172.21.15.53:0/2804412623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b00aff70 0x7f36b0157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 >> 172.21.15.53:0/2804412623 conn(0x7f36b006c050 msgr2=0x7f36b0070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.353+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 shutdown_connections 2023-10-21T16:02:56.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.354+0000 7f36b4885640 1 -- 172.21.15.53:0/2804412623 wait complete. 2023-10-21T16:02:56.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:56.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.604+0000 7f773bfff640 1 Processor -- start 2023-10-21T16:02:56.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.604+0000 7f773bfff640 1 -- start start 2023-10-21T16:02:56.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c0adc70 0x7f773c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f773c15af10 0x7f773c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c15f7d0 con 0x7f773c15ab30 2023-10-21T16:02:56.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773bfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c15efb0 con 0x7f773c15af10 2023-10-21T16:02:56.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c15f210 con 0x7f773c0adc70 2023-10-21T16:02:56.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f773c15af10 0x7f773c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f773c15af10 0x7f773c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60208/0 (socket says 172.21.15.53:60208) 2023-10-21T16:02:56.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 -- 172.21.15.53:0/100684044 learned_addr learned my addr 172.21.15.53:0/100684044 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:56.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773a7fc640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c0adc70 0x7f773c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 -- 172.21.15.53:0/100684044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c0adc70 msgr2=0x7f773c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:56.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c0adc70 0x7f773c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.606+0000 7f773b7fe640 1 -- 172.21.15.53:0/100684044 >> v1:172.21.15.124:6789/0 conn(0x7f773c15ab30 legacy=0x7f773c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:56.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.607+0000 7f773b7fe640 1 -- 172.21.15.53:0/100684044 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c15f4f0 con 0x7f773c15af10 2023-10-21T16:02:56.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.608+0000 7f773b7fe640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c15af10 0x7f773c0b0600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cb000 tx=0x7f773402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9415dcbffe4a72f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.608+0000 7f7739ffb640 1 -- 172.21.15.53:0/100684044 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773402ca30 con 0x7f773c15af10 2023-10-21T16:02:56.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.608+0000 7f7739ffb640 1 -- 172.21.15.53:0/100684044 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773402cb90 con 0x7f773c15af10 2023-10-21T16:02:56.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.608+0000 7f7739ffb640 1 -- 172.21.15.53:0/100684044 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340343f0 con 0x7f773c15af10 2023-10-21T16:02:56.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 -- 172.21.15.53:0/100684044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c15af10 msgr2=0x7f773c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c15af10 0x7f773c0b0600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cb000 tx=0x7f773402be90 comp rx=0 tx=0).stop 2023-10-21T16:02:56.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 -- 172.21.15.53:0/100684044 shutdown_connections 2023-10-21T16:02:56.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c0adc70 0x7f773c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 --2- 172.21.15.53:0/100684044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c15af10 0x7f773c0b0600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 -- 172.21.15.53:0/100684044 >> 172.21.15.53:0/100684044 conn(0x7f773c0cb0c0 msgr2=0x7f773c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:56.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 -- 172.21.15.53:0/100684044 shutdown_connections 2023-10-21T16:02:56.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.609+0000 7f773bfff640 1 -- 172.21.15.53:0/100684044 wait complete. 2023-10-21T16:02:56.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.611+0000 7f773bfff640 1 Processor -- start 2023-10-21T16:02:56.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.611+0000 7f773bfff640 1 -- start start 2023-10-21T16:02:56.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 0x7f773c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c15af10 0x7f773c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c8360 con 0x7f773c0adc70 2023-10-21T16:02:56.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c156340 con 0x7f773c15ab30 2023-10-21T16:02:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c7750 con 0x7f773c15af10 2023-10-21T16:02:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60216/0 (socket says 172.21.15.53:60216) 2023-10-21T16:02:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.612+0000 7f773affd640 1 -- 172.21.15.53:0/3711908943 learned_addr learned my addr 172.21.15.53:0/3711908943 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:56.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773a7fc640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 0x7f773c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773b7fe640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c15af10 0x7f773c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 -- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c15af10 msgr2=0x7f773c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c15af10 0x7f773c0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 -- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 msgr2=0x7f773c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 0x7f773c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 -- 172.21.15.53:0/3711908943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009b30 con 0x7f773c0adc70 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773a7fc640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 0x7f773c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773affd640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c430 tx=0x7f772c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011790 con 0x7f773c0adc70 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c0118f0 con 0x7f773c0adc70 2023-10-21T16:02:56.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011ce0 con 0x7f773c0adc70 2023-10-21T16:02:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.613+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0c7a30 con 0x7f773c0adc70 2023-10-21T16:02:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.614+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c0c7ff0 con 0x7f773c0adc70 2023-10-21T16:02:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.615+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c0cb4a0 con 0x7f773c0adc70 2023-10-21T16:02:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.616+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f772c01c460 con 0x7f773c0adc70 2023-10-21T16:02:56.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.616+0000 7f77237fe640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 0x7f772806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:56.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.617+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f772c0899b0 con 0x7f773c0adc70 2023-10-21T16:02:56.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.617+0000 7f773a7fc640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 0x7f772806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:56.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.618+0000 7f773a7fc640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 0x7f772806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f773c0ac060 tx=0x7f7730008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:56.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.621+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f772c054600 con 0x7f773c0adc70 2023-10-21T16:02:56.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.816+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f773c04e160 con 0x7f773c0adc70 2023-10-21T16:02:56.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.819+0000 7f77237fe640 1 -- 172.21.15.53:0/3711908943 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f772c057c20 con 0x7f773c0adc70 2023-10-21T16:02:56.822 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:56.822 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:56.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 msgr2=0x7f772806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 0x7f772806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f773c0ac060 tx=0x7f7730008040 comp rx=0 tx=0).stop 2023-10-21T16:02:56.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 msgr2=0x7f773c0c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:56.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c430 tx=0x7f772c00c8f0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 shutdown_connections 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f773c15af10 0x7f773c0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f773c15ab30 0x7f773c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f772806ae70 0x7f772806d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 --2- 172.21.15.53:0/3711908943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f773c0adc70 0x7f773c0c7170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:56.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 >> 172.21.15.53:0/3711908943 conn(0x7f773c0cb0c0 msgr2=0x7f773c0ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:56.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 shutdown_connections 2023-10-21T16:02:56.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:56.822+0000 7f773bfff640 1 -- 172.21.15.53:0/3711908943 wait complete. 2023-10-21T16:02:56.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:56.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:57.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.075+0000 7f56e889a640 1 Processor -- start 2023-10-21T16:02:57.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.075+0000 7f56e889a640 1 -- start start 2023-10-21T16:02:57.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.076+0000 7f56e889a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56e40d06c0 0x7f56e415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e889a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 0x7f56e40d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e889a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e40d3800 con 0x7f56e415cd80 2023-10-21T16:02:57.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e889a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e415d160 con 0x7f56e40d06c0 2023-10-21T16:02:57.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e889a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40ce370 con 0x7f56e40d0ec0 2023-10-21T16:02:57.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56e40d06c0 0x7f56e415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56e40d06c0 0x7f56e415dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60218/0 (socket says 172.21.15.53:60218) 2023-10-21T16:02:57.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e259c640 1 -- 172.21.15.53:0/2964262654 learned_addr learned my addr 172.21.15.53:0/2964262654 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:57.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e259c640 1 -- 172.21.15.53:0/2964262654 >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 msgr2=0x7f56e40d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:57.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.077+0000 7f56e359e640 1 --2- 172.21.15.53:0/2964262654 >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 0x7f56e40d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.078+0000 7f56e259c640 1 --2- 172.21.15.53:0/2964262654 >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 0x7f56e40d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.078+0000 7f56e259c640 1 -- 172.21.15.53:0/2964262654 >> v1:172.21.15.124:6789/0 conn(0x7f56e415cd80 legacy=0x7f56e415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.078+0000 7f56e259c640 1 -- 172.21.15.53:0/2964262654 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e415ecf0 con 0x7f56e40d06c0 2023-10-21T16:02:57.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.078+0000 7f56e359e640 1 --2- 172.21.15.53:0/2964262654 >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 0x7f56e40d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:57.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.079+0000 7f56e259c640 1 --2- 172.21.15.53:0/2964262654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e415dcf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56e40d0de0 tx=0x7f56d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce655631846a680 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.079+0000 7f56e1d9b640 1 -- 172.21.15.53:0/2964262654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8031870 con 0x7f56e40d06c0 2023-10-21T16:02:57.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.079+0000 7f56e1d9b640 1 -- 172.21.15.53:0/2964262654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80319d0 con 0x7f56e40d06c0 2023-10-21T16:02:57.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.079+0000 7f56e1d9b640 1 -- 172.21.15.53:0/2964262654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8031d60 con 0x7f56e40d06c0 2023-10-21T16:02:57.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 -- 172.21.15.53:0/2964262654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 msgr2=0x7f56e415dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 --2- 172.21.15.53:0/2964262654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e415dcf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56e40d0de0 tx=0x7f56d802b910 comp rx=0 tx=0).stop 2023-10-21T16:02:57.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 -- 172.21.15.53:0/2964262654 shutdown_connections 2023-10-21T16:02:57.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 --2- 172.21.15.53:0/2964262654 >> v2:172.21.15.124:3300/0 conn(0x7f56e40d0ec0 0x7f56e40d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 --2- 172.21.15.53:0/2964262654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e415dcf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 -- 172.21.15.53:0/2964262654 >> 172.21.15.53:0/2964262654 conn(0x7f56e409bc40 msgr2=0x7f56e40d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:57.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 -- 172.21.15.53:0/2964262654 shutdown_connections 2023-10-21T16:02:57.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.080+0000 7f56e889a640 1 -- 172.21.15.53:0/2964262654 wait complete. 2023-10-21T16:02:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.082+0000 7f56e889a640 1 Processor -- start 2023-10-21T16:02:57.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.082+0000 7f56e889a640 1 -- start start 2023-10-21T16:02:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e40d0ec0 0x7f56e4140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e415cd80 0x7f56e40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b8730 con 0x7f56e40d06c0 2023-10-21T16:02:57.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40c9b50 con 0x7f56e40d0ec0 2023-10-21T16:02:57.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e889a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40c9e30 con 0x7f56e415cd80 2023-10-21T16:02:57.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60226/0 (socket says 172.21.15.53:60226) 2023-10-21T16:02:57.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 -- 172.21.15.53:0/3081028627 learned_addr learned my addr 172.21.15.53:0/3081028627 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 -- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e415cd80 msgr2=0x7f56e40ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e415cd80 0x7f56e40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 -- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e40d0ec0 msgr2=0x7f56e4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e40d0ec0 0x7f56e4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.083+0000 7f56e2d9d640 1 -- 172.21.15.53:0/3081028627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80094d0 con 0x7f56e40d06c0 2023-10-21T16:02:57.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56e2d9d640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c730 tx=0x7f56d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4003bc0 con 0x7f56e40d06c0 2023-10-21T16:02:57.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d4003d40 con 0x7f56e40d06c0 2023-10-21T16:02:57.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400aa90 con 0x7f56e40d06c0 2023-10-21T16:02:57.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e40ca110 con 0x7f56e40d06c0 2023-10-21T16:02:57.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.084+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41541d0 con 0x7f56e40d06c0 2023-10-21T16:02:57.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.085+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e40ca630 con 0x7f56e40d06c0 2023-10-21T16:02:57.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.086+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56d400abf0 con 0x7f56e40d06c0 2023-10-21T16:02:57.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.087+0000 7f56cb7fe640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 0x7f56d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.087+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f56d4089860 con 0x7f56e40d06c0 2023-10-21T16:02:57.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.088+0000 7f56e259c640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 0x7f56d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.089+0000 7f56e259c640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 0x7f56d006d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56d8036000 tx=0x7f56d8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.091+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56d4054540 con 0x7f56e40d06c0 2023-10-21T16:02:57.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.288+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56e4074150 con 0x7f56e40d06c0 2023-10-21T16:02:57.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.289+0000 7f56cb7fe640 1 -- 172.21.15.53:0/3081028627 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56d4057b60 con 0x7f56e40d06c0 2023-10-21T16:02:57.291 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:57.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 msgr2=0x7f56d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 0x7f56d006d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56d8036000 tx=0x7f56d8030000 comp rx=0 tx=0).stop 2023-10-21T16:02:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 msgr2=0x7f56e40b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c730 tx=0x7f56d400cbf0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 shutdown_connections 2023-10-21T16:02:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56e415cd80 0x7f56e40ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56e40d0ec0 0x7f56e4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f56d006ae70 0x7f56d006d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 --2- 172.21.15.53:0/3081028627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56e40d06c0 0x7f56e40b5640 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 >> 172.21.15.53:0/3081028627 conn(0x7f56e409bc40 msgr2=0x7f56e4157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 shutdown_connections 2023-10-21T16:02:57.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.292+0000 7f56e889a640 1 -- 172.21.15.53:0/3081028627 wait complete. 2023-10-21T16:02:57.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:57.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.533+0000 7fd63e67e640 1 Processor -- start 2023-10-21T16:02:57.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.533+0000 7fd63e67e640 1 -- start start 2023-10-21T16:02:57.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63e67e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd6380d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63e67e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380cd520 con 0x7fd6380cf2b0 2023-10-21T16:02:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63e67e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380d1910 con 0x7fd6380cf690 2023-10-21T16:02:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63e67e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380ce080 con 0x7fd63815cf30 2023-10-21T16:02:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63d67c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd6380cf2b0 0x7fd6380d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43718/0 (socket says 172.21.15.53:43718) 2023-10-21T16:02:57.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63d67c640 1 -- 172.21.15.53:0/1931338011 learned_addr learned my addr 172.21.15.53:0/1931338011 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:57.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.535+0000 7fd63de7d640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd6380d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811224064 0 0) 0x7fd6380cd520 con 0x7fd6380cf2b0 2023-10-21T16:02:57.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd624003520 con 0x7fd6380cf2b0 2023-10-21T16:02:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052650786 0 0) 0x7fd6380d1910 con 0x7fd6380cf690 2023-10-21T16:02:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6380cd520 con 0x7fd6380cf690 2023-10-21T16:02:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327610293 0 0) 0x7fd624003520 con 0x7fd6380cf2b0 2023-10-21T16:02:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 msgr2=0x7fd6380d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd6380d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 >> v1:172.21.15.124:6789/0 conn(0x7fd6380cf690 legacy=0x7fd63815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380cd750 con 0x7fd6380cf2b0 2023-10-21T16:02:57.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd634001670 con 0x7fd6380cf2b0 2023-10-21T16:02:57.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 >> v1:172.21.15.53:6789/0 conn(0x7fd6380cf2b0 legacy=0x7fd6380d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd624003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd6240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd624004d00 0x7fd624005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380ce080 con 0x7fd624004090 2023-10-21T16:02:57.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62400bb10 con 0x7fd624004d00 2023-10-21T16:02:57.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.536+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62400bca0 con 0x7fd63815cf30 2023-10-21T16:02:57.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd6240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63de7d640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd624004d00 0x7fd624005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63d67c640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd624003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 -- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 msgr2=0x7fd624003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:57.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd624003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 -- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd624004d00 msgr2=0x7fd624005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:57.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd624004d00 0x7fd624005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63ce7b640 1 -- 172.21.15.53:0/1931338011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624003520 con 0x7fd624004090 2023-10-21T16:02:57.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.537+0000 7fd63d67c640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63815cf30 0x7fd624003b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:57.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.538+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd6240047c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd628001fd0 tx=0x7fd6240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe09923c54a64d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.538+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628003b70 con 0x7fd624004090 2023-10-21T16:02:57.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628003cd0 con 0x7fd624004090 2023-10-21T16:02:57.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd62ffff640 1 -- 172.21.15.53:0/1931338011 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280052f0 con 0x7fd624004090 2023-10-21T16:02:57.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 -- 172.21.15.53:0/1931338011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 msgr2=0x7fd6240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1931338011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd6240047c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd628001fd0 tx=0x7fd6240057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63de7d640 1 -- 172.21.15.53:0/1931338011 reap_dead start 2023-10-21T16:02:57.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 -- 172.21.15.53:0/1931338011 shutdown_connections 2023-10-21T16:02:57.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 -- 172.21.15.53:0/1931338011 >> 172.21.15.53:0/1931338011 conn(0x7fd63809bc40 msgr2=0x7fd6380d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:57.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 -- 172.21.15.53:0/1931338011 shutdown_connections 2023-10-21T16:02:57.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.539+0000 7fd63e67e640 1 -- 172.21.15.53:0/1931338011 wait complete. 2023-10-21T16:02:57.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.541+0000 7fd63e67e640 1 Processor -- start 2023-10-21T16:02:57.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.541+0000 7fd63e67e640 1 -- start start 2023-10-21T16:02:57.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63809f1d0 0x7fd6380b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6380b70a0 0x7fd6381f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380b7520 con 0x7fd624004090 2023-10-21T16:02:57.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63809ed40 con 0x7fd6380b70a0 2023-10-21T16:02:57.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.542+0000 7fd63e67e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63815b2a0 con 0x7fd63809f1d0 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60250/0 (socket says 172.21.15.53:60250) 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 -- 172.21.15.53:0/1674892021 learned_addr learned my addr 172.21.15.53:0/1674892021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 -- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63809f1d0 msgr2=0x7fd6380b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63809f1d0 0x7fd6380b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63de7d640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6380b70a0 0x7fd6381f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 -- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6380b70a0 msgr2=0x7fd6381f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6380b70a0 0x7fd6381f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.543+0000 7fd63d67c640 1 -- 172.21.15.53:0/1674892021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63809e130 con 0x7fd624004090 2023-10-21T16:02:57.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd63d67c640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd63400d7e0 tx=0x7fd63400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340074e0 con 0x7fd624004090 2023-10-21T16:02:57.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634007640 con 0x7fd624004090 2023-10-21T16:02:57.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340049e0 con 0x7fd624004090 2023-10-21T16:02:57.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63809e410 con 0x7fd624004090 2023-10-21T16:02:57.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.544+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63809e9d0 con 0x7fd624004090 2023-10-21T16:02:57.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.545+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638074150 con 0x7fd624004090 2023-10-21T16:02:57.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.546+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd634004b40 con 0x7fd624004090 2023-10-21T16:02:57.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.547+0000 7fd62dffb640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 0x7fd61c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:57.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.547+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd634089410 con 0x7fd624004090 2023-10-21T16:02:57.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.548+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 0x7fd61c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:57.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.549+0000 7fd63ce7b640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 0x7fd61c06d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd628034000 tx=0x7fd628030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:57.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.551+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd634053fd0 con 0x7fd624004090 2023-10-21T16:02:57.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.742+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6380d0e70 con 0x7fd624004090 2023-10-21T16:02:57.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.743+0000 7fd62dffb640 1 -- 172.21.15.53:0/1674892021 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd634019020 con 0x7fd624004090 2023-10-21T16:02:57.745 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:57.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 msgr2=0x7fd61c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 0x7fd61c06d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd628034000 tx=0x7fd628030000 comp rx=0 tx=0).stop 2023-10-21T16:02:57.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 msgr2=0x7fd638151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:57.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd63400d7e0 tx=0x7fd63400dca0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 shutdown_connections 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6380b70a0 0x7fd6381f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd63809f1d0 0x7fd6380b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd61c06ae70 0x7fd61c06d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 --2- 172.21.15.53:0/1674892021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd624004090 0x7fd638151f30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 >> 172.21.15.53:0/1674892021 conn(0x7fd63809bc40 msgr2=0x7fd63815fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:57.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 shutdown_connections 2023-10-21T16:02:57.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.746+0000 7fd63e67e640 1 -- 172.21.15.53:0/1674892021 wait complete. 2023-10-21T16:02:57.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:02:57.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.993+0000 7f6981589640 1 Processor -- start 2023-10-21T16:02:57.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.994+0000 7f6981589640 1 -- start start 2023-10-21T16:02:58.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.995+0000 7f6981589640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f697c071c30 0x7f697c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.995+0000 7f6981589640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f697c0720c0 0x7f697c06a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.995+0000 7f6981589640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0ce210 con 0x7f697c1601b0 2023-10-21T16:02:58.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.995+0000 7f6981589640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c0cdb40 con 0x7f697c0720c0 2023-10-21T16:02:58.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.995+0000 7f6981589640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c0cdee0 con 0x7f697c071c30 2023-10-21T16:02:58.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f697c0720c0 0x7f697c06a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f697c0720c0 0x7f697c06a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60266/0 (socket says 172.21.15.53:60266) 2023-10-21T16:02:58.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 -- 172.21.15.53:0/1974323587 learned_addr learned my addr 172.21.15.53:0/1974323587 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:58.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f6973fff640 1 --2- 172.21.15.53:0/1974323587 >> v2:172.21.15.124:3300/0 conn(0x7f697c071c30 0x7f697c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 -- 172.21.15.53:0/1974323587 >> v2:172.21.15.124:3300/0 conn(0x7f697c071c30 msgr2=0x7f697c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 --2- 172.21.15.53:0/1974323587 >> v2:172.21.15.124:3300/0 conn(0x7f697c071c30 0x7f697c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 -- 172.21.15.53:0/1974323587 >> v1:172.21.15.124:6789/0 conn(0x7f697c1601b0 legacy=0x7f697c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.996+0000 7f697b7fe640 1 -- 172.21.15.53:0/1974323587 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c161530 con 0x7f697c0720c0 2023-10-21T16:02:58.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.998+0000 7f697b7fe640 1 --2- 172.21.15.53:0/1974323587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0720c0 0x7f697c06a290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f697c067230 tx=0x7f697402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18a253082fa5b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.998+0000 7f697a7fc640 1 -- 172.21.15.53:0/1974323587 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697402ca80 con 0x7f697c0720c0 2023-10-21T16:02:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.998+0000 7f697a7fc640 1 -- 172.21.15.53:0/1974323587 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697402cbe0 con 0x7f697c0720c0 2023-10-21T16:02:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.998+0000 7f697a7fc640 1 -- 172.21.15.53:0/1974323587 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740343f0 con 0x7f697c0720c0 2023-10-21T16:02:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 -- 172.21.15.53:0/1974323587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0720c0 msgr2=0x7f697c06a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 --2- 172.21.15.53:0/1974323587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0720c0 0x7f697c06a290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f697c067230 tx=0x7f697402bee0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 -- 172.21.15.53:0/1974323587 shutdown_connections 2023-10-21T16:02:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 --2- 172.21.15.53:0/1974323587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c0720c0 0x7f697c06a290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 --2- 172.21.15.53:0/1974323587 >> v2:172.21.15.124:3300/0 conn(0x7f697c071c30 0x7f697c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 -- 172.21.15.53:0/1974323587 >> 172.21.15.53:0/1974323587 conn(0x7f697c06d430 msgr2=0x7f697c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:58.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 -- 172.21.15.53:0/1974323587 shutdown_connections 2023-10-21T16:02:58.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:57.999+0000 7f6981589640 1 -- 172.21.15.53:0/1974323587 wait complete. 2023-10-21T16:02:58.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.001+0000 7f6981589640 1 Processor -- start 2023-10-21T16:02:58.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.001+0000 7f6981589640 1 -- start start 2023-10-21T16:02:58.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 0x7f697c0ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697c0720c0 0x7f697c1f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 0x7f697c1f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0ae5f0 con 0x7f697c071c30 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c14a9f0 con 0x7f697c1601b0 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f6981589640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0adfe0 con 0x7f697c0720c0 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 0x7f697c1f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.002+0000 7f697affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 0x7f697c0ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 0x7f697c1f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:57176/0 (socket says 172.21.15.53:57176) 2023-10-21T16:02:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697b7fe640 1 -- 172.21.15.53:0/2683322764 learned_addr learned my addr 172.21.15.53:0/2683322764 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f6973fff640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697c0720c0 0x7f697c1f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 -- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697c0720c0 msgr2=0x7f697c1f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697c0720c0 0x7f697c1f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 -- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 msgr2=0x7f697c1f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 0x7f697c1f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 -- 172.21.15.53:0/2683322764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009b30 con 0x7f697c071c30 2023-10-21T16:02:58.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.003+0000 7f697affd640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 0x7f697c0ad630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f696c00c390 tx=0x7f696c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.004+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0052d0 con 0x7f697c071c30 2023-10-21T16:02:58.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.004+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c005430 con 0x7f697c071c30 2023-10-21T16:02:58.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.004+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c003830 con 0x7f697c071c30 2023-10-21T16:02:58.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.004+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c0ab8d0 con 0x7f697c071c30 2023-10-21T16:02:58.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.004+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c0ae280 con 0x7f697c071c30 2023-10-21T16:02:58.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.005+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0ae760 con 0x7f697c071c30 2023-10-21T16:02:58.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.006+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f696c003990 con 0x7f697c071c30 2023-10-21T16:02:58.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.007+0000 7f69737fe640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 0x7f695c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.007+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f696c088e00 con 0x7f697c071c30 2023-10-21T16:02:58.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.008+0000 7f6973fff640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 0x7f695c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.009+0000 7f6973fff640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 0x7f695c06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6964006fd0 tx=0x7f6964008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.012+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f696c053b90 con 0x7f697c071c30 2023-10-21T16:02:58.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.212+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f697c04e160 con 0x7f697c071c30 2023-10-21T16:02:58.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.213+0000 7f69737fe640 1 -- 172.21.15.53:0/2683322764 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f696c0170a0 con 0x7f697c071c30 2023-10-21T16:02:58.215 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:02:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 msgr2=0x7f695c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 0x7f695c06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6964006fd0 tx=0x7f6964008040 comp rx=0 tx=0).stop 2023-10-21T16:02:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 msgr2=0x7f697c0ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 0x7f697c0ad630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f696c00c390 tx=0x7f696c00c850 comp rx=0 tx=0).stop 2023-10-21T16:02:58.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 shutdown_connections 2023-10-21T16:02:58.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697c1601b0 0x7f697c1f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697c0720c0 0x7f697c1f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f695c06ae70 0x7f695c06d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 --2- 172.21.15.53:0/2683322764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f697c071c30 0x7f697c0ad630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.216+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 >> 172.21.15.53:0/2683322764 conn(0x7f697c06d430 msgr2=0x7f697c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:58.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.217+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 shutdown_connections 2023-10-21T16:02:58.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.217+0000 7f6981589640 1 -- 172.21.15.53:0/2683322764 wait complete. 2023-10-21T16:02:58.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:02:58.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.464+0000 7f5f822ad640 1 Processor -- start 2023-10-21T16:02:58.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.464+0000 7f5f822ad640 1 -- start start 2023-10-21T16:02:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 0x7f5f7c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cab60 con 0x7f5f7c0ca680 2023-10-21T16:02:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cb960 con 0x7f5f7c15ac00 2023-10-21T16:02:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f822ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cbdf0 con 0x7f5f7c15a820 2023-10-21T16:02:58.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f80aaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f80aaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60286/0 (socket says 172.21.15.53:60286) 2023-10-21T16:02:58.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f80aaa640 1 -- 172.21.15.53:0/884622448 learned_addr learned my addr 172.21.15.53:0/884622448 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:58.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.466+0000 7f5f81aac640 1 --2- 172.21.15.53:0/884622448 >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 0x7f5f7c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f812ab640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f80aaa640 1 -- 172.21.15.53:0/884622448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 msgr2=0x7f5f7c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:58.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f80aaa640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f80aaa640 1 -- 172.21.15.53:0/884622448 >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 msgr2=0x7f5f7c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:58.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f80aaa640 1 --2- 172.21.15.53:0/884622448 >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 0x7f5f7c0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f80aaa640 1 -- 172.21.15.53:0/884622448 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0cb030 con 0x7f5f7c0ca680 2023-10-21T16:02:58.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.467+0000 7f5f81aac640 1 --2- 172.21.15.53:0/884622448 >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 0x7f5f7c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:58.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.468+0000 7f5f80aaa640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ce340 tx=0x7f5f6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b13d882bbd27c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.468+0000 7f5f73fff640 1 -- 172.21.15.53:0/884622448 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0317e0 con 0x7f5f7c0ca680 2023-10-21T16:02:58.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.468+0000 7f5f73fff640 1 -- 172.21.15.53:0/884622448 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c031940 con 0x7f5f7c0ca680 2023-10-21T16:02:58.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.468+0000 7f5f73fff640 1 -- 172.21.15.53:0/884622448 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c031cd0 con 0x7f5f7c0ca680 2023-10-21T16:02:58.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 -- 172.21.15.53:0/884622448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c0ca680 msgr2=0x7f5f7c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ce340 tx=0x7f5f6c02b880 comp rx=0 tx=0).stop 2023-10-21T16:02:58.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 -- 172.21.15.53:0/884622448 shutdown_connections 2023-10-21T16:02:58.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 --2- 172.21.15.53:0/884622448 >> v2:172.21.15.124:3300/0 conn(0x7f5f7c15ac00 0x7f5f7c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c0b04b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 --2- 172.21.15.53:0/884622448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 -- 172.21.15.53:0/884622448 >> 172.21.15.53:0/884622448 conn(0x7f5f7c06c050 msgr2=0x7f5f7c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:58.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 -- 172.21.15.53:0/884622448 shutdown_connections 2023-10-21T16:02:58.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.469+0000 7f5f822ad640 1 -- 172.21.15.53:0/884622448 wait complete. 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.471+0000 7f5f822ad640 1 Processor -- start 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.471+0000 7f5f822ad640 1 -- start start 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c1f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c1f5050 con 0x7f5f7c15ac00 2023-10-21T16:02:58.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c15cfa0 con 0x7f5f7c0ca680 2023-10-21T16:02:58.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.472+0000 7f5f822ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c15d280 con 0x7f5f7c15a820 2023-10-21T16:02:58.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60302/0 (socket says 172.21.15.53:60302) 2023-10-21T16:02:58.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 -- 172.21.15.53:0/2644769772 learned_addr learned my addr 172.21.15.53:0/2644769772 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:58.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f812ab640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c1f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f80aaa640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 -- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 msgr2=0x7f5f7c15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 -- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f7c0ca680 msgr2=0x7f5f7c1f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c1f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 -- 172.21.15.53:0/2644769772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0094d0 con 0x7f5f7c15ac00 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f80aaa640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.473+0000 7f5f81aac640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f74007e10 tx=0x7f5f7400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.474+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74003b90 con 0x7f5f7c15ac00 2023-10-21T16:02:58.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.474+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7400f040 con 0x7f5f7c15ac00 2023-10-21T16:02:58.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.474+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400de80 con 0x7f5f7c15ac00 2023-10-21T16:02:58.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.474+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c15d560 con 0x7f5f7c15ac00 2023-10-21T16:02:58.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.474+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c15db20 con 0x7f5f7c15ac00 2023-10-21T16:02:58.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.475+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c15dd10 con 0x7f5f7c15ac00 2023-10-21T16:02:58.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.476+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f74004030 con 0x7f5f7c15ac00 2023-10-21T16:02:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.478+0000 7f5f71ffb640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 0x7f5f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.478+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f74089cc0 con 0x7f5f7c15ac00 2023-10-21T16:02:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.478+0000 7f5f812ab640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 0x7f5f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.479+0000 7f5f812ab640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 0x7f5f6006d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f78006fd0 tx=0x7f5f78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.482+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f74090080 con 0x7f5f7c15ac00 2023-10-21T16:02:58.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.678+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f7c0b1310 con 0x7f5f7c15ac00 2023-10-21T16:02:58.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.679+0000 7f5f71ffb640 1 -- 172.21.15.53:0/2644769772 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f7401a0a0 con 0x7f5f7c15ac00 2023-10-21T16:02:58.681 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:02:58.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 msgr2=0x7f5f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 0x7f5f6006d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f78006fd0 tx=0x7f5f78008040 comp rx=0 tx=0).stop 2023-10-21T16:02:58.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 msgr2=0x7f5f7c0ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f74007e10 tx=0x7f5f7400d4a0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 shutdown_connections 2023-10-21T16:02:58.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f5f6006ae70 0x7f5f6006d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f7c15ac00 0x7f5f7c0ac920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.681+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f7c15a820 0x7f5f7c15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.682+0000 7f5f822ad640 1 --2- 172.21.15.53:0/2644769772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f7c0ca680 0x7f5f7c1f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.682+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 >> 172.21.15.53:0/2644769772 conn(0x7f5f7c06c050 msgr2=0x7f5f7c0c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.682+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 shutdown_connections 2023-10-21T16:02:58.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.682+0000 7f5f822ad640 1 -- 172.21.15.53:0/2644769772 wait complete. 2023-10-21T16:02:58.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:02:58.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.919+0000 7fbc3349f640 1 Processor -- start 2023-10-21T16:02:58.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.919+0000 7fbc3349f640 1 -- start start 2023-10-21T16:02:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc3349f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc3349f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c15ced0 con 0x7fbc2c0d0a00 2023-10-21T16:02:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc3349f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c0d2470 con 0x7fbc2c15caf0 2023-10-21T16:02:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc3349f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d2750 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc31c9c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc32c9e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbc2c0d0a00 0x7fbc2c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43732/0 (socket says 172.21.15.53:43732) 2023-10-21T16:02:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc32c9e640 1 -- 172.21.15.53:0/3945635860 learned_addr learned my addr 172.21.15.53:0/3945635860 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.920+0000 7fbc31c9c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60308/0 (socket says 172.21.15.53:60308) 2023-10-21T16:02:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.921+0000 7fbc31c9c640 1 -- 172.21.15.53:0/3945635860 >> v1:172.21.15.124:6789/0 conn(0x7fbc2c15caf0 legacy=0x7fbc2c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.921+0000 7fbc31c9c640 1 -- 172.21.15.53:0/3945635860 >> v1:172.21.15.53:6789/0 conn(0x7fbc2c0d0a00 legacy=0x7fbc2c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.921+0000 7fbc31c9c640 1 -- 172.21.15.53:0/3945635860 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0d2a30 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.921+0000 7fbc3149b640 1 -- 172.21.15.53:0/3945635860 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675178911 0 0) 0x7fbc2c15ced0 con 0x7fbc2c0d0a00 2023-10-21T16:02:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.922+0000 7fbc31c9c640 1 --2- 172.21.15.53:0/3945635860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0097a0 tx=0x7fbc1c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3074c9c045008826 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.922+0000 7fbc3149b640 1 -- 172.21.15.53:0/3945635860 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c030610 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.922+0000 7fbc3149b640 1 -- 172.21.15.53:0/3945635860 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c030770 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3149b640 1 -- 172.21.15.53:0/3945635860 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c030b00 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 -- 172.21.15.53:0/3945635860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c0cd8e0 msgr2=0x7fbc2c0b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 --2- 172.21.15.53:0/3945635860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0097a0 tx=0x7fbc1c02b960 comp rx=0 tx=0).stop 2023-10-21T16:02:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 -- 172.21.15.53:0/3945635860 shutdown_connections 2023-10-21T16:02:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 --2- 172.21.15.53:0/3945635860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b5a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 -- 172.21.15.53:0/3945635860 >> 172.21.15.53:0/3945635860 conn(0x7fbc2c071870 msgr2=0x7fbc2c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:58.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 -- 172.21.15.53:0/3945635860 shutdown_connections 2023-10-21T16:02:58.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.923+0000 7fbc3349f640 1 -- 172.21.15.53:0/3945635860 wait complete. 2023-10-21T16:02:58.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.925+0000 7fbc3349f640 1 Processor -- start 2023-10-21T16:02:58.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.925+0000 7fbc3349f640 1 -- start start 2023-10-21T16:02:58.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc2c0d0a00 0x7fbc2c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0cbb80 con 0x7fbc2c15caf0 2023-10-21T16:02:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d20e0 con 0x7fbc2c0cd8e0 2023-10-21T16:02:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc3349f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c15b320 con 0x7fbc2c0d0a00 2023-10-21T16:02:58.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.926+0000 7fbc32c9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60324/0 (socket says 172.21.15.53:60324) 2023-10-21T16:02:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 -- 172.21.15.53:0/2512323644 learned_addr learned my addr 172.21.15.53:0/2512323644 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc3249d640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc31c9c640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc2c0d0a00 0x7fbc2c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 -- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc2c0d0a00 msgr2=0x7fbc2c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc2c0d0a00 0x7fbc2c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 -- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc2c0cd8e0 msgr2=0x7fbc2c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:58.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:58.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 -- 172.21.15.53:0/2512323644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009400 con 0x7fbc2c15caf0 2023-10-21T16:02:58.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.927+0000 7fbc32c9e640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c450 tx=0x7fbc2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.928+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240074e0 con 0x7fbc2c15caf0 2023-10-21T16:02:58.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.928+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24007660 con 0x7fbc2c15caf0 2023-10-21T16:02:58.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.928+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24004a20 con 0x7fbc2c15caf0 2023-10-21T16:02:58.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.928+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c158740 con 0x7fbc2c15caf0 2023-10-21T16:02:58.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.928+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c158d00 con 0x7fbc2c15caf0 2023-10-21T16:02:58.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.930+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c0d1460 con 0x7fbc2c15caf0 2023-10-21T16:02:58.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.930+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc24004b80 con 0x7fbc2c15caf0 2023-10-21T16:02:58.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.931+0000 7fbc22ffd640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 0x7fbc1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:58.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.931+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbc24088ad0 con 0x7fbc2c15caf0 2023-10-21T16:02:58.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.932+0000 7fbc3249d640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 0x7fbc1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:58.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.933+0000 7fbc3249d640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 0x7fbc1006d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc280012e0 tx=0x7fbc28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:58.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:58.937+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc24053750 con 0x7fbc2c15caf0 2023-10-21T16:02:59.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.137+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc2c0cbd30 con 0x7fbc2c15caf0 2023-10-21T16:02:59.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.140+0000 7fbc22ffd640 1 -- 172.21.15.53:0/2512323644 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fbc2c0cbd30 con 0x7fbc2c15caf0 2023-10-21T16:02:59.142 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:02:59.142 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:02:59.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 msgr2=0x7fbc1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 0x7fbc1006d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc280012e0 tx=0x7fbc28009040 comp rx=0 tx=0).stop 2023-10-21T16:02:59.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 msgr2=0x7fbc2c15ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c450 tx=0x7fbc2400c910 comp rx=0 tx=0).stop 2023-10-21T16:02:59.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 shutdown_connections 2023-10-21T16:02:59.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbc1006aea0 0x7fbc1006d360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc2c15caf0 0x7fbc2c15ac40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc2c0d0a00 0x7fbc2c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 --2- 172.21.15.53:0/2512323644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc2c0cd8e0 0x7fbc2c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.142+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 >> 172.21.15.53:0/2512323644 conn(0x7fbc2c071870 msgr2=0x7fbc2c0b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.143+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 shutdown_connections 2023-10-21T16:02:59.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.143+0000 7fbc3349f640 1 -- 172.21.15.53:0/2512323644 wait complete. 2023-10-21T16:02:59.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:02:59.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:02:59.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.400+0000 7f9e4759e640 1 Processor -- start 2023-10-21T16:02:59.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.400+0000 7f9e4759e640 1 -- start start 2023-10-21T16:02:59.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.401+0000 7f9e4759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e4815aab0 0x7f9e48070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.401+0000 7f9e4759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 0x7f9e480b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.401+0000 7f9e4759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4815ae90 con 0x7f9e480caef0 2023-10-21T16:02:59.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.401+0000 7f9e4759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e480cbdb0 con 0x7f9e4815aab0 2023-10-21T16:02:59.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.401+0000 7f9e4759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cb7f0 con 0x7f9e480b0670 2023-10-21T16:02:59.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e3dd9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9e480caef0 0x7f9e480aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43748/0 (socket says 172.21.15.53:43748) 2023-10-21T16:02:59.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e3dd9b640 1 -- 172.21.15.53:0/2731958908 learned_addr learned my addr 172.21.15.53:0/2731958908 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:59.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e46d9d640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 0x7f9e480b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e4659c640 1 --2- 172.21.15.53:0/2731958908 >> v2:172.21.15.124:3300/0 conn(0x7f9e4815aab0 0x7f9e48070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679657034 0 0) 0x7f9e30001110 con 0x7f9e480caef0 2023-10-21T16:02:59.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e34003520 con 0x7f9e480caef0 2023-10-21T16:02:59.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2176193116 0 0) 0x7f9e34003520 con 0x7f9e480caef0 2023-10-21T16:02:59.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 msgr2=0x7f9e480b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 0x7f9e480b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 >> v2:172.21.15.124:3300/0 conn(0x7f9e4815aab0 msgr2=0x7f9e48070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.402+0000 7f9e45d9b640 1 --2- 172.21.15.53:0/2731958908 >> v2:172.21.15.124:3300/0 conn(0x7f9e4815aab0 0x7f9e48070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480cbad0 con 0x7f9e480caef0 2023-10-21T16:02:59.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9e30001800 con 0x7f9e480caef0 2023-10-21T16:02:59.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 >> v1:172.21.15.53:6789/0 conn(0x7f9e480caef0 legacy=0x7f9e480aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e340047c0 0x7f9e34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 0x7f9e34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cbdb0 con 0x7f9e34003970 2023-10-21T16:02:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cb7f0 con 0x7f9e340047c0 2023-10-21T16:02:59.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3400ba60 con 0x7f9e480b0670 2023-10-21T16:02:59.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e3dd9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e340047c0 0x7f9e34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 -- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 msgr2=0x7f9e34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b0670 0x7f9e34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 -- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e340047c0 msgr2=0x7f9e34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e340047c0 0x7f9e34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.403+0000 7f9e4659c640 1 -- 172.21.15.53:0/2731958908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34003520 con 0x7f9e34003970 2023-10-21T16:02:59.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.404+0000 7f9e3dd9b640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e340047c0 0x7f9e34004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:59.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e4659c640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e34003d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800ba00 tx=0x7f9e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d876430319ceab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3802d040 con 0x7f9e34003970 2023-10-21T16:02:59.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38033610 con 0x7f9e34003970 2023-10-21T16:02:59.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e45d9b640 1 -- 172.21.15.53:0/2731958908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380339e0 con 0x7f9e34003970 2023-10-21T16:02:59.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e4759e640 1 -- 172.21.15.53:0/2731958908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 msgr2=0x7f9e34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e4759e640 1 --2- 172.21.15.53:0/2731958908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e34003d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800ba00 tx=0x7f9e340057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.405+0000 7f9e46d9d640 1 -- 172.21.15.53:0/2731958908 reap_dead start 2023-10-21T16:02:59.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.406+0000 7f9e4759e640 1 -- 172.21.15.53:0/2731958908 shutdown_connections 2023-10-21T16:02:59.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.406+0000 7f9e4759e640 1 -- 172.21.15.53:0/2731958908 >> 172.21.15.53:0/2731958908 conn(0x7f9e4806c010 msgr2=0x7f9e4806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:59.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.406+0000 7f9e4759e640 1 -- 172.21.15.53:0/2731958908 shutdown_connections 2023-10-21T16:02:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.406+0000 7f9e4759e640 1 -- 172.21.15.53:0/2731958908 wait complete. 2023-10-21T16:02:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.408+0000 7f9e4759e640 1 Processor -- start 2023-10-21T16:02:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.408+0000 7f9e4759e640 1 -- start start 2023-10-21T16:02:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e4815b030 0x7f9e480b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b30f0 0x7f9e481f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480b3570 con 0x7f9e34003970 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4815c1c0 con 0x7f9e4815b030 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480c4610 con 0x7f9e480b30f0 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60342/0 (socket says 172.21.15.53:60342) 2023-10-21T16:02:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 -- 172.21.15.53:0/1741725629 learned_addr learned my addr 172.21.15.53:0/1741725629 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 -- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b30f0 msgr2=0x7f9e481f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b30f0 0x7f9e481f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 -- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e4815b030 msgr2=0x7f9e480b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e4815b030 0x7f9e480b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.409+0000 7f9e4659c640 1 -- 172.21.15.53:0/1741725629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4815b5b0 con 0x7f9e34003970 2023-10-21T16:02:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e4659c640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800ba60 tx=0x7f9e38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004220 con 0x7f9e34003970 2023-10-21T16:02:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38040070 con 0x7f9e34003970 2023-10-21T16:02:59.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38003610 con 0x7f9e34003970 2023-10-21T16:02:59.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4815b890 con 0x7f9e34003970 2023-10-21T16:02:59.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.410+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4815be50 con 0x7f9e34003970 2023-10-21T16:02:59.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.411+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e48074150 con 0x7f9e34003970 2023-10-21T16:02:59.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.412+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9e38003770 con 0x7f9e34003970 2023-10-21T16:02:59.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.413+0000 7f9e3f7fe640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 0x7f9e2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.413+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e380ab950 con 0x7f9e34003970 2023-10-21T16:02:59.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.414+0000 7f9e3dd9b640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 0x7f9e2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.415+0000 7f9e3dd9b640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 0x7f9e2806d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e30000f60 tx=0x7f9e30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.418+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e38076510 con 0x7f9e34003970 2023-10-21T16:02:59.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.615+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e4815bfe0 con 0x7f9e34003970 2023-10-21T16:02:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.616+0000 7f9e3f7fe640 1 -- 172.21.15.53:0/1741725629 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e380420a0 con 0x7f9e34003970 2023-10-21T16:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:02:59.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 msgr2=0x7f9e2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 0x7f9e2806d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e30000f60 tx=0x7f9e30009040 comp rx=0 tx=0).stop 2023-10-21T16:02:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 msgr2=0x7f9e480c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800ba60 tx=0x7f9e38004730 comp rx=0 tx=0).stop 2023-10-21T16:02:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 shutdown_connections 2023-10-21T16:02:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e480b30f0 0x7f9e481f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e4815b030 0x7f9e480b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e2806ae70 0x7f9e2806d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 --2- 172.21.15.53:0/1741725629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e34003970 0x7f9e480c40d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.618+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 >> 172.21.15.53:0/1741725629 conn(0x7f9e4806c010 msgr2=0x7f9e480ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.619+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 shutdown_connections 2023-10-21T16:02:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.619+0000 7f9e4759e640 1 -- 172.21.15.53:0/1741725629 wait complete. 2023-10-21T16:02:59.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:02:59.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.867+0000 7f42dbfff640 1 Processor -- start 2023-10-21T16:02:59.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.867+0000 7f42dbfff640 1 -- start start 2023-10-21T16:02:59.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.868+0000 7f42dbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42dc0d00e0 0x7f42dc0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.868+0000 7f42dbfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0d2870 con 0x7f42dc15ccb0 2023-10-21T16:02:59.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.868+0000 7f42dbfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0d1150 con 0x7f42dc0d0b90 2023-10-21T16:02:59.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.868+0000 7f42dbfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0d1430 con 0x7f42dc0d00e0 2023-10-21T16:02:59.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.869+0000 7f42daffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f42dc15ccb0 0x7f42dc0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43760/0 (socket says 172.21.15.53:43760) 2023-10-21T16:02:59.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.869+0000 7f42daffd640 1 -- 172.21.15.53:0/708808264 learned_addr learned my addr 172.21.15.53:0/708808264 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:02:59.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.869+0000 7f42d27fc640 1 --2- 172.21.15.53:0/708808264 >> v2:172.21.15.124:3300/0 conn(0x7f42dc0d00e0 0x7f42dc0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.869+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271779585 0 0) 0x7f42dc0d2870 con 0x7f42dc15ccb0 2023-10-21T16:02:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.869+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42bc003520 con 0x7f42dc15ccb0 2023-10-21T16:02:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219606953 0 0) 0x7f42bc003520 con 0x7f42dc15ccb0 2023-10-21T16:02:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 >> v2:172.21.15.124:3300/0 conn(0x7f42dc0d00e0 msgr2=0x7f42dc0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 --2- 172.21.15.53:0/708808264 >> v2:172.21.15.124:3300/0 conn(0x7f42dc0d00e0 0x7f42dc0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 >> v1:172.21.15.124:6789/0 conn(0x7f42dc0d0b90 legacy=0x7f42dc0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0d1710 con 0x7f42dc15ccb0 2023-10-21T16:02:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f42cc001670 con 0x7f42dc15ccb0 2023-10-21T16:02:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 >> v1:172.21.15.53:6789/0 conn(0x7f42dc15ccb0 legacy=0x7f42dc0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42bc0039c0 0x7f42bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0047c0 0x7f42bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 0x7f42bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0d1430 con 0x7f42bc0051d0 2023-10-21T16:02:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc00c120 con 0x7f42bc0039c0 2023-10-21T16:02:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc00c2e0 con 0x7f42bc0047c0 2023-10-21T16:02:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42d27fc640 1 --2- 172.21.15.53:0/708808264 >> v2:172.21.15.124:3300/0 conn(0x7f42dc0d00e0 0x7f42dc0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:02:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.870+0000 7f42db7fe640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 0x7f42bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42d27fc640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0047c0 0x7f42bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42db7fe640 1 -- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0047c0 msgr2=0x7f42bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42daffd640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42bc0039c0 0x7f42bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42db7fe640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0047c0 0x7f42bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42db7fe640 1 -- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42bc0039c0 msgr2=0x7f42bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:02:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42db7fe640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42bc0039c0 0x7f42bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42db7fe640 1 -- 172.21.15.53:0/708808264 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc003520 con 0x7f42bc0051d0 2023-10-21T16:02:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42d27fc640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0047c0 0x7f42bc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.871+0000 7f42daffd640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42bc0039c0 0x7f42bc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:02:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.872+0000 7f42db7fe640 1 -- 172.21.15.53:0/708808264 reap_dead start 2023-10-21T16:02:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.872+0000 7f42db7fe640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 0x7f42bc00b980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001e50 tx=0x7f42bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28264477304a987 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.872+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4003b70 con 0x7f42bc0051d0 2023-10-21T16:02:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.872+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4003d10 con 0x7f42bc0051d0 2023-10-21T16:02:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.872+0000 7f42da7fc640 1 -- 172.21.15.53:0/708808264 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4005310 con 0x7f42bc0051d0 2023-10-21T16:02:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 -- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 msgr2=0x7f42bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 0x7f42bc00b980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001e50 tx=0x7f42bc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 -- 172.21.15.53:0/708808264 shutdown_connections 2023-10-21T16:02:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 --2- 172.21.15.53:0/708808264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42bc0051d0 0x7f42bc00b980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 -- 172.21.15.53:0/708808264 >> 172.21.15.53:0/708808264 conn(0x7f42dc071660 msgr2=0x7f42dc071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:02:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 -- 172.21.15.53:0/708808264 shutdown_connections 2023-10-21T16:02:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.873+0000 7f42dbfff640 1 -- 172.21.15.53:0/708808264 wait complete. 2023-10-21T16:02:59.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.875+0000 7f42dbfff640 1 Processor -- start 2023-10-21T16:02:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.875+0000 7f42dbfff640 1 -- start start 2023-10-21T16:02:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0051d0 0x7f42dc1fa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc15a140 0x7f42dc0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c8d30 con 0x7f42dc0cc670 2023-10-21T16:02:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1faad0 con 0x7f42dc15a140 2023-10-21T16:02:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42dbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1e5450 con 0x7f42bc0051d0 2023-10-21T16:02:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60360/0 (socket says 172.21.15.53:60360) 2023-10-21T16:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 -- 172.21.15.53:0/2969032784 learned_addr learned my addr 172.21.15.53:0/2969032784 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42daffd640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0051d0 0x7f42dc1fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 -- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0051d0 msgr2=0x7f42dc1fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42db7fe640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc15a140 0x7f42dc0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0051d0 0x7f42dc1fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 -- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc15a140 msgr2=0x7f42dc0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc15a140 0x7f42dc0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.876+0000 7f42d27fc640 1 -- 172.21.15.53:0/2969032784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc1e5730 con 0x7f42dc0cc670 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42d27fc640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42c400f550 tx=0x7f42c400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4014890 con 0x7f42dc0cc670 2023-10-21T16:02:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c400c030 con 0x7f42dc0cc670 2023-10-21T16:02:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4014bf0 con 0x7f42dc0cc670 2023-10-21T16:02:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc1e5a10 con 0x7f42dc0cc670 2023-10-21T16:02:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.877+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1e5fd0 con 0x7f42dc0cc670 2023-10-21T16:02:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.879+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc1e6160 con 0x7f42dc0cc670 2023-10-21T16:02:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.880+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42c4014d50 con 0x7f42dc0cc670 2023-10-21T16:02:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.881+0000 7f42d3fff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:02:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.881+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f42c4089c50 con 0x7f42dc0cc670 2023-10-21T16:02:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.881+0000 7f42daffd640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:02:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.882+0000 7f42daffd640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 0x7f42c006d360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001200 tx=0x7f42cc000f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:02:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:02:59.885+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42c408f020 con 0x7f42dc0cc670 2023-10-21T16:03:00.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.086+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42dc074150 con 0x7f42dc0cc670 2023-10-21T16:03:00.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.087+0000 7f42d3fff640 1 -- 172.21.15.53:0/2969032784 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42c4058460 con 0x7f42dc0cc670 2023-10-21T16:03:00.089 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:00.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 msgr2=0x7f42c006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 0x7f42c006d360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001200 tx=0x7f42cc000f50 comp rx=0 tx=0).stop 2023-10-21T16:03:00.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 msgr2=0x7f42dc0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42c400f550 tx=0x7f42c400fa10 comp rx=0 tx=0).stop 2023-10-21T16:03:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 shutdown_connections 2023-10-21T16:03:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42dc15a140 0x7f42dc0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f42c006aea0 0x7f42c006d360 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42dc0cc670 0x7f42dc0c7e60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 --2- 172.21.15.53:0/2969032784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42bc0051d0 0x7f42dc1fa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 >> 172.21.15.53:0/2969032784 conn(0x7f42dc071660 msgr2=0x7f42dc15b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:00.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 shutdown_connections 2023-10-21T16:03:00.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.090+0000 7f42dbfff640 1 -- 172.21.15.53:0/2969032784 wait complete. 2023-10-21T16:03:00.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.339+0000 7fb1853a9640 1 Processor -- start 2023-10-21T16:03:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.339+0000 7fb1853a9640 1 -- start start 2023-10-21T16:03:00.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.340+0000 7fb1853a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb18015cc70 0x7fb1800cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.340+0000 7fb1853a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb1800b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb1853a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800b8b00 con 0x7fb1800d31a0 2023-10-21T16:03:00.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb1853a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800b4fd0 con 0x7fb18015cc70 2023-10-21T16:03:00.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb1853a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b8540 con 0x7fb1800b4600 2023-10-21T16:03:00.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb1800d31a0 0x7fb1800b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43772/0 (socket says 172.21.15.53:43772) 2023-10-21T16:03:00.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17e7fc640 1 -- 172.21.15.53:0/1847118791 learned_addr learned my addr 172.21.15.53:0/1847118791 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb1800b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17effd640 1 --2- 172.21.15.53:0/1847118791 >> v2:172.21.15.124:3300/0 conn(0x7fb18015cc70 0x7fb1800cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805636381 0 0) 0x7fb1800b8b00 con 0x7fb1800d31a0 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.341+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb168003520 con 0x7fb1800d31a0 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967057081 0 0) 0x7fb168003520 con 0x7fb1800d31a0 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 msgr2=0x7fb1800b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb1800b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 >> v2:172.21.15.124:3300/0 conn(0x7fb18015cc70 msgr2=0x7fb1800cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 --2- 172.21.15.53:0/1847118791 >> v2:172.21.15.124:3300/0 conn(0x7fb18015cc70 0x7fb1800cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800b8820 con 0x7fb1800d31a0 2023-10-21T16:03:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb174001670 con 0x7fb1800d31a0 2023-10-21T16:03:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17effd640 1 --2- 172.21.15.53:0/1847118791 >> v2:172.21.15.124:3300/0 conn(0x7fb18015cc70 0x7fb1800cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 >> v1:172.21.15.53:6789/0 conn(0x7fb1800d31a0 legacy=0x7fb1800b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb168003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168003ef0 0x7fb1680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168004d00 0x7fb168005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b4fd0 con 0x7fb168004d00 2023-10-21T16:03:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800b8540 con 0x7fb168003ef0 2023-10-21T16:03:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16800ba80 con 0x7fb1800b4600 2023-10-21T16:03:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.342+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168004d00 0x7fb168005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17effd640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb168003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17e7fc640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168003ef0 0x7fb1680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17f7fe640 1 -- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 msgr2=0x7fb168003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:00.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800b4600 0x7fb168003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17f7fe640 1 -- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168003ef0 msgr2=0x7fb1680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:00.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168003ef0 0x7fb1680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.343+0000 7fb17f7fe640 1 -- 172.21.15.53:0/1847118791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb168003520 con 0x7fb168004d00 2023-10-21T16:03:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.344+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168004d00 0x7fb168005180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb178009af0 tx=0x7fb1680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c0e7126a0501ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.344+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17802c6b0 con 0x7fb168004d00 2023-10-21T16:03:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.344+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17802c810 con 0x7fb168004d00 2023-10-21T16:03:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.344+0000 7fb17dffb640 1 -- 172.21.15.53:0/1847118791 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17802cba0 con 0x7fb168004d00 2023-10-21T16:03:00.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 -- 172.21.15.53:0/1847118791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168004d00 msgr2=0x7fb168005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 --2- 172.21.15.53:0/1847118791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168004d00 0x7fb168005180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb178009af0 tx=0x7fb1680057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb17f7fe640 1 -- 172.21.15.53:0/1847118791 reap_dead start 2023-10-21T16:03:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 -- 172.21.15.53:0/1847118791 shutdown_connections 2023-10-21T16:03:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 -- 172.21.15.53:0/1847118791 >> 172.21.15.53:0/1847118791 conn(0x7fb1800718a0 msgr2=0x7fb180071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 -- 172.21.15.53:0/1847118791 shutdown_connections 2023-10-21T16:03:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.345+0000 7fb1853a9640 1 -- 172.21.15.53:0/1847118791 wait complete. 2023-10-21T16:03:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.347+0000 7fb1853a9640 1 Processor -- start 2023-10-21T16:03:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.347+0000 7fb1853a9640 1 -- start start 2023-10-21T16:03:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 0x7fb18015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800d1ec0 0x7fb1800d2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1800d28b0 con 0x7fb1801f2bf0 2023-10-21T16:03:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015dfd0 con 0x7fb168004d00 2023-10-21T16:03:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.348+0000 7fb1853a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015e2b0 con 0x7fb1800d1ec0 2023-10-21T16:03:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60388/0 (socket says 172.21.15.53:60388) 2023-10-21T16:03:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 -- 172.21.15.53:0/3639651065 learned_addr learned my addr 172.21.15.53:0/3639651065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17effd640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 0x7fb18015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800d1ec0 0x7fb1800d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 -- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800d1ec0 msgr2=0x7fb1800d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800d1ec0 0x7fb1800d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 -- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 msgr2=0x7fb18015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 0x7fb18015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 -- 172.21.15.53:0/3639651065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18015e590 con 0x7fb1801f2bf0 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17effd640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 0x7fb18015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.349+0000 7fb17e7fc640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb17400e8c0 tx=0x7fb17400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.350+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400c8d0 con 0x7fb1801f2bf0 2023-10-21T16:03:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.350+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17400ca50 con 0x7fb1801f2bf0 2023-10-21T16:03:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.350+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1800c76e0 con 0x7fb1801f2bf0 2023-10-21T16:03:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.350+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17400ce00 con 0x7fb1801f2bf0 2023-10-21T16:03:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.350+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18015af60 con 0x7fb1801f2bf0 2023-10-21T16:03:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.351+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1800c7c00 con 0x7fb1801f2bf0 2023-10-21T16:03:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.352+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb17401d460 con 0x7fb1801f2bf0 2023-10-21T16:03:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.353+0000 7fb1677fe640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 0x7fb16c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.353+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb174089a80 con 0x7fb1801f2bf0 2023-10-21T16:03:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.354+0000 7fb17effd640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 0x7fb16c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.355+0000 7fb17effd640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 0x7fb16c06d360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb170009960 tx=0x7fb17000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.357+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1800c7c00 con 0x7fb1801f2bf0 2023-10-21T16:03:00.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.553+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1800c7c00 con 0x7fb1801f2bf0 2023-10-21T16:03:00.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.553+0000 7fb1677fe640 1 -- 172.21.15.53:0/3639651065 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1800c7c00 con 0x7fb1801f2bf0 2023-10-21T16:03:00.555 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:00.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 msgr2=0x7fb16c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 0x7fb16c06d360 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb170009960 tx=0x7fb17000b040 comp rx=0 tx=0).stop 2023-10-21T16:03:00.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 msgr2=0x7fb18015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb17400e8c0 tx=0x7fb17400ed80 comp rx=0 tx=0).stop 2023-10-21T16:03:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 shutdown_connections 2023-10-21T16:03:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1800d1ec0 0x7fb1800d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16c06aea0 0x7fb16c06d360 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1801f2bf0 0x7fb18015d660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 --2- 172.21.15.53:0/3639651065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb168004d00 0x7fb18015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 >> 172.21.15.53:0/3639651065 conn(0x7fb1800718a0 msgr2=0x7fb1800b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.556+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 shutdown_connections 2023-10-21T16:03:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.557+0000 7fb1853a9640 1 -- 172.21.15.53:0/3639651065 wait complete. 2023-10-21T16:03:00.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.803+0000 7f9e86ebb640 1 Processor -- start 2023-10-21T16:03:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.804+0000 7f9e86ebb640 1 -- start start 2023-10-21T16:03:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e86ebb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e800b0680 0x7f9e8015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e86ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800ca680 0x7f9e8015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e86ebb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e800cab60 con 0x7f9e8015a820 2023-10-21T16:03:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e86ebb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e800cb960 con 0x7f9e800b0680 2023-10-21T16:03:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e86ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800cbdf0 con 0x7f9e800ca680 2023-10-21T16:03:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e856b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e800b0680 0x7f9e8015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e856b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e800b0680 0x7f9e8015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60392/0 (socket says 172.21.15.53:60392) 2023-10-21T16:03:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.805+0000 7f9e856b8640 1 -- 172.21.15.53:0/3931009708 learned_addr learned my addr 172.21.15.53:0/3931009708 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.806+0000 7f9e866ba640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800ca680 0x7f9e8015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.806+0000 7f9e856b8640 1 -- 172.21.15.53:0/3931009708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800ca680 msgr2=0x7f9e8015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.806+0000 7f9e856b8640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800ca680 0x7f9e8015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.806+0000 7f9e856b8640 1 -- 172.21.15.53:0/3931009708 >> v1:172.21.15.124:6789/0 conn(0x7f9e8015a820 legacy=0x7f9e800aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.806+0000 7f9e856b8640 1 -- 172.21.15.53:0/3931009708 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800cb030 con 0x7f9e800b0680 2023-10-21T16:03:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.807+0000 7f9e856b8640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800b0680 0x7f9e8015e540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009870 tx=0x7f9e7002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd08805f9e295ce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e84eb7640 1 -- 172.21.15.53:0/3931009708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70031640 con 0x7f9e800b0680 2023-10-21T16:03:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e84eb7640 1 -- 172.21.15.53:0/3931009708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700317a0 con 0x7f9e800b0680 2023-10-21T16:03:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e84eb7640 1 -- 172.21.15.53:0/3931009708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70031b30 con 0x7f9e800b0680 2023-10-21T16:03:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3931009708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800b0680 msgr2=0x7f9e8015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800b0680 0x7f9e8015e540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009870 tx=0x7f9e7002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3931009708 shutdown_connections 2023-10-21T16:03:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800ca680 0x7f9e8015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3931009708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800b0680 0x7f9e8015e540 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3931009708 >> 172.21.15.53:0/3931009708 conn(0x7f9e8006c050 msgr2=0x7f9e8006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.808+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3931009708 shutdown_connections 2023-10-21T16:03:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.809+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3931009708 wait complete. 2023-10-21T16:03:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.810+0000 7f9e86ebb640 1 Processor -- start 2023-10-21T16:03:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.810+0000 7f9e86ebb640 1 -- start start 2023-10-21T16:03:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800b0680 0x7f9e80153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 0x7f9e801506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800c69b0 con 0x7f9e800ca680 2023-10-21T16:03:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e801f0490 con 0x7f9e8015a820 2023-10-21T16:03:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.811+0000 7f9e86ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80146e50 con 0x7f9e800b0680 2023-10-21T16:03:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60400/0 (socket says 172.21.15.53:60400) 2023-10-21T16:03:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 -- 172.21.15.53:0/3647130578 learned_addr learned my addr 172.21.15.53:0/3647130578 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:00.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e866ba640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 0x7f9e801506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 -- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800b0680 msgr2=0x7f9e80153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e85eb9640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800b0680 0x7f9e80153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800b0680 0x7f9e80153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 -- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 msgr2=0x7f9e801506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:00.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 0x7f9e801506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 -- 172.21.15.53:0/3647130578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700094d0 con 0x7f9e800ca680 2023-10-21T16:03:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e866ba640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 0x7f9e801506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.812+0000 7f9e856b8640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9e7002b930 tx=0x7f9e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.813+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7003ac60 con 0x7f9e800ca680 2023-10-21T16:03:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.813+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7003adc0 con 0x7f9e800ca680 2023-10-21T16:03:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.813+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70042630 con 0x7f9e800ca680 2023-10-21T16:03:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.813+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e80150be0 con 0x7f9e800ca680 2023-10-21T16:03:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.813+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e800c62f0 con 0x7f9e800ca680 2023-10-21T16:03:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.814+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e800c6af0 con 0x7f9e800ca680 2023-10-21T16:03:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.815+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9e7002f050 con 0x7f9e800ca680 2023-10-21T16:03:00.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.816+0000 7f9e767fc640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 0x7f9e6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:00.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.816+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e700ab2f0 con 0x7f9e800ca680 2023-10-21T16:03:00.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.816+0000 7f9e85eb9640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 0x7f9e6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:00.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.817+0000 7f9e85eb9640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 0x7f9e6406d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e801503c0 tx=0x7f9e7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:00.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:00.820+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e800c6af0 con 0x7f9e800ca680 2023-10-21T16:03:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.018+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9e800c6af0 con 0x7f9e800ca680 2023-10-21T16:03:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.019+0000 7f9e767fc640 1 -- 172.21.15.53:0/3647130578 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9e800c6af0 con 0x7f9e800ca680 2023-10-21T16:03:01.022 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:01.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.021+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 msgr2=0x7f9e6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.021+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 0x7f9e6406d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e801503c0 tx=0x7f9e7c009040 comp rx=0 tx=0).stop 2023-10-21T16:03:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 msgr2=0x7f9e80146910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9e7002b930 tx=0x7f9e700046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 shutdown_connections 2023-10-21T16:03:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e8015a820 0x7f9e801506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9e6406ae70 0x7f9e6406d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e800ca680 0x7f9e80146910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 --2- 172.21.15.53:0/3647130578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800b0680 0x7f9e80153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 >> 172.21.15.53:0/3647130578 conn(0x7f9e8006c050 msgr2=0x7f9e80157010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 shutdown_connections 2023-10-21T16:03:01.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.022+0000 7f9e86ebb640 1 -- 172.21.15.53:0/3647130578 wait complete. 2023-10-21T16:03:01.090 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-21T16:03:01.091 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:03:01.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:01.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.317+0000 7f055b3a9640 1 Processor -- start 2023-10-21T16:03:01.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.318+0000 7f055b3a9640 1 -- start start 2023-10-21T16:03:01.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.319+0000 7f055b3a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f05541600d0 0x7f05540d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.319+0000 7f055b3a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f05540cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.319+0000 7f055b3a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05540d1750 con 0x7f05540d0120 2023-10-21T16:03:01.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.319+0000 7f055b3a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540d0590 con 0x7f05541600d0 2023-10-21T16:03:01.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.319+0000 7f055b3a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540d1a30 con 0x7f05540d11d0 2023-10-21T16:03:01.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f0559946640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05540d0120 0x7f05540d2ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43778/0 (socket says 172.21.15.53:43778) 2023-10-21T16:03:01.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f0559946640 1 -- 172.21.15.53:0/3560641823 learned_addr learned my addr 172.21.15.53:0/3560641823 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f0559145640 1 --2- 172.21.15.53:0/3560641823 >> v2:172.21.15.124:3300/0 conn(0x7f05541600d0 0x7f05540d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f0558944640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f05540cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475310349 0 0) 0x7f05540d1750 con 0x7f05540d0120 2023-10-21T16:03:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.320+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0540003520 con 0x7f05540d0120 2023-10-21T16:03:01.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1786347499 0 0) 0x7f0540003520 con 0x7f05540d0120 2023-10-21T16:03:01.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 msgr2=0x7f05540cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:01.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f05540cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 >> v2:172.21.15.124:3300/0 conn(0x7f05541600d0 msgr2=0x7f05540d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:01.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 --2- 172.21.15.53:0/3560641823 >> v2:172.21.15.124:3300/0 conn(0x7f05541600d0 0x7f05540d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05540d1d10 con 0x7f05540d0120 2023-10-21T16:03:01.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f054c0019f0 con 0x7f05540d0120 2023-10-21T16:03:01.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 >> v1:172.21.15.53:6789/0 conn(0x7f05540d0120 legacy=0x7f05540d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0540003970 0x7f0540003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0540004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f0540005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540d0590 con 0x7f05400047c0 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540d1a30 con 0x7f0540003970 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054000b980 con 0x7f05540d11d0 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f0558944640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0540004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f0559145640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0540003970 0x7f0540003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.321+0000 7f0559946640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f0540005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0558944640 1 -- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 msgr2=0x7f0540005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0558944640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f0540005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0558944640 1 -- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0540003970 msgr2=0x7f0540003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0558944640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0540003970 0x7f0540003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0558944640 1 -- 172.21.15.53:0/3560641823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540003520 con 0x7f05400047c0 2023-10-21T16:03:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.322+0000 7f0559946640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05540d11d0 0x7f0540005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f0558944640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0540004c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0544009ac0 tx=0x7f05400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6eab5e2daef39f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054402c6b0 con 0x7f05400047c0 2023-10-21T16:03:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054402c810 con 0x7f05400047c0 2023-10-21T16:03:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f054bfff640 1 -- 172.21.15.53:0/3560641823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054402cba0 con 0x7f05400047c0 2023-10-21T16:03:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f055b3a9640 1 -- 172.21.15.53:0/3560641823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 msgr2=0x7f0540004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f055b3a9640 1 --2- 172.21.15.53:0/3560641823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0540004c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0544009ac0 tx=0x7f05400057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.323+0000 7f0559946640 1 -- 172.21.15.53:0/3560641823 reap_dead start 2023-10-21T16:03:01.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.324+0000 7f055b3a9640 1 -- 172.21.15.53:0/3560641823 shutdown_connections 2023-10-21T16:03:01.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.324+0000 7f055b3a9640 1 -- 172.21.15.53:0/3560641823 >> 172.21.15.53:0/3560641823 conn(0x7f05540718a0 msgr2=0x7f0554071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:01.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.324+0000 7f055b3a9640 1 -- 172.21.15.53:0/3560641823 shutdown_connections 2023-10-21T16:03:01.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.324+0000 7f055b3a9640 1 -- 172.21.15.53:0/3560641823 wait complete. 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.325+0000 7f055b3a9640 1 Processor -- start 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.326+0000 7f055b3a9640 1 -- start start 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.326+0000 7f055b3a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.326+0000 7f055b3a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05540c6410 0x7f05540c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.326+0000 7f055b3a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0554155070 0x7f05541554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f055b3a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0554155a30 con 0x7f05400047c0 2023-10-21T16:03:01.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f055b3a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05541f3520 con 0x7f05540c6410 2023-10-21T16:03:01.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f055b3a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540c77a0 con 0x7f0554155070 2023-10-21T16:03:01.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60412/0 (socket says 172.21.15.53:60412) 2023-10-21T16:03:01.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 -- 172.21.15.53:0/1274532414 learned_addr learned my addr 172.21.15.53:0/1274532414 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0558944640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05540c6410 0x7f05540c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 -- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0554155070 msgr2=0x7f05541554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0554155070 0x7f05541554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 -- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05540c6410 msgr2=0x7f05540c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05540c6410 0x7f05540c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.327+0000 7f0559145640 1 -- 172.21.15.53:0/1274532414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554153640 con 0x7f05400047c0 2023-10-21T16:03:01.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f0559145640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05500116d0 tx=0x7f0550011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000f9c0 con 0x7f05400047c0 2023-10-21T16:03:01.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055000fb20 con 0x7f05400047c0 2023-10-21T16:03:01.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000fe90 con 0x7f05400047c0 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05540c6ed0 con 0x7f05400047c0 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.328+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05540c79f0 con 0x7f05400047c0 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.329+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05540c73f0 con 0x7f05400047c0 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.330+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0550015460 con 0x7f05400047c0 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.332+0000 7f0549ffb640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 0x7f053806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.332+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0550089fa0 con 0x7f05400047c0 2023-10-21T16:03:01.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.332+0000 7f0558944640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 0x7f053806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.333+0000 7f0558944640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 0x7f053806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0544036000 tx=0x7f0544030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.336+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05500bcca0 con 0x7f05400047c0 2023-10-21T16:03:01.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.532+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0554067490 con 0x7f05400047c0 2023-10-21T16:03:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.535+0000 7f0549ffb640 1 -- 172.21.15.53:0/1274532414 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f0550022090 con 0x7f05400047c0 2023-10-21T16:03:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:01.538 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:01.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.537+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 msgr2=0x7f053806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.537+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 0x7f053806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0544036000 tx=0x7f0544030000 comp rx=0 tx=0).stop 2023-10-21T16:03:01.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.537+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 msgr2=0x7f0554153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.537+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f05500116d0 tx=0x7f0550011b90 comp rx=0 tx=0).stop 2023-10-21T16:03:01.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 shutdown_connections 2023-10-21T16:03:01.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0554155070 0x7f05541554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05540c6410 0x7f05540c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f053806ae70 0x7f053806d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 --2- 172.21.15.53:0/1274532414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05400047c0 0x7f0554153100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 >> 172.21.15.53:0/1274532414 conn(0x7f05540718a0 msgr2=0x7f05540b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:01.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 shutdown_connections 2023-10-21T16:03:01.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.538+0000 7f055b3a9640 1 -- 172.21.15.53:0/1274532414 wait complete. 2023-10-21T16:03:01.561 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:01.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.791+0000 7f3e31516640 1 Processor -- start 2023-10-21T16:03:01.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.791+0000 7f3e31516640 1 -- start start 2023-10-21T16:03:01.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.792+0000 7f3e31516640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e2c15cd80 0x7f3e2c15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e31516640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d06c0 0x7f3e2c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e31516640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2c0d3800 con 0x7f3e2c0d0ec0 2023-10-21T16:03:01.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e31516640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2c15d160 con 0x7f3e2c15cd80 2023-10-21T16:03:01.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e31516640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0ce370 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e30d15640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f3e2c0d0ec0 0x7f3e2c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:57806/0 (socket says 172.21.15.53:57806) 2023-10-21T16:03:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e2b7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d06c0 0x7f3e2c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e2bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e2c15cd80 0x7f3e2c15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.793+0000 7f3e30d15640 1 -- 172.21.15.53:0/543348583 learned_addr learned my addr 172.21.15.53:0/543348583 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.794+0000 7f3e2bfff640 1 -- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d06c0 msgr2=0x7f3e2c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.794+0000 7f3e2bfff640 1 --2- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d06c0 0x7f3e2c15e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.794+0000 7f3e2bfff640 1 -- 172.21.15.53:0/543348583 >> v1:172.21.15.124:6789/0 conn(0x7f3e2c0d0ec0 legacy=0x7f3e2c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.794+0000 7f3e2bfff640 1 -- 172.21.15.53:0/543348583 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0cda40 con 0x7f3e2c15cd80 2023-10-21T16:03:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.796+0000 7f3e2bfff640 1 --2- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15dcf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009de0 tx=0x7f3e1c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da08be3dffe4280f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e2affd640 1 -- 172.21.15.53:0/543348583 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c02cc00 con 0x7f3e2c15cd80 2023-10-21T16:03:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e2affd640 1 -- 172.21.15.53:0/543348583 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c02cd60 con 0x7f3e2c15cd80 2023-10-21T16:03:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e2affd640 1 -- 172.21.15.53:0/543348583 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c034660 con 0x7f3e2c15cd80 2023-10-21T16:03:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 -- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 msgr2=0x7f3e2c15dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 --2- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15dcf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009de0 tx=0x7f3e1c02be50 comp rx=0 tx=0).stop 2023-10-21T16:03:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 -- 172.21.15.53:0/543348583 shutdown_connections 2023-10-21T16:03:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 --2- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d06c0 0x7f3e2c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 --2- 172.21.15.53:0/543348583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15dcf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.797+0000 7f3e31516640 1 -- 172.21.15.53:0/543348583 >> 172.21.15.53:0/543348583 conn(0x7f3e2c09bc40 msgr2=0x7f3e2c0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:01.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.798+0000 7f3e31516640 1 -- 172.21.15.53:0/543348583 shutdown_connections 2023-10-21T16:03:01.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.798+0000 7f3e31516640 1 -- 172.21.15.53:0/543348583 wait complete. 2023-10-21T16:03:01.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.799+0000 7f3e31516640 1 Processor -- start 2023-10-21T16:03:01.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 -- start start 2023-10-21T16:03:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 0x7f3e2c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c09ca90 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c09cd70 con 0x7f3e2c15cd80 2023-10-21T16:03:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.800+0000 7f3e31516640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c09d050 con 0x7f3e2c0d0ec0 2023-10-21T16:03:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60416/0 (socket says 172.21.15.53:60416) 2023-10-21T16:03:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 -- 172.21.15.53:0/786675727 learned_addr learned my addr 172.21.15.53:0/786675727 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e30d15640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2b7fe640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 0x7f3e2c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 -- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 msgr2=0x7f3e2c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 0x7f3e2c0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 -- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 msgr2=0x7f3e2c15a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2bfff640 1 -- 172.21.15.53:0/786675727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c009a70 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.801+0000 7f3e2b7fe640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 0x7f3e2c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e2bfff640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009de0 tx=0x7f3e1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c040650 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c0407b0 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c040b60 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c09d330 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.802+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c09db30 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.803+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2c09d850 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.804+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e1c040cc0 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.805+0000 7f3e28ff9640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 0x7f3e1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:01.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.805+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3e1c0ac7d0 con 0x7f3e2c0d06c0 2023-10-21T16:03:01.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.806+0000 7f3e2b7fe640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 0x7f3e1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:01.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.807+0000 7f3e2b7fe640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 0x7f3e1806d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0cc5c0 tx=0x7f3e20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:01.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:01.810+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e2c09d850 con 0x7f3e2c0d06c0 2023-10-21T16:03:02.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.003+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e2c09d850 con 0x7f3e2c0d06c0 2023-10-21T16:03:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.004+0000 7f3e28ff9640 1 -- 172.21.15.53:0/786675727 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e1c044020 con 0x7f3e2c0d06c0 2023-10-21T16:03:02.007 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:02.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 msgr2=0x7f3e1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 0x7f3e1806d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0cc5c0 tx=0x7f3e20008040 comp rx=0 tx=0).stop 2023-10-21T16:03:02.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 msgr2=0x7f3e2c0b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009de0 tx=0x7f3e1c004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:02.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 shutdown_connections 2023-10-21T16:03:02.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e2c15cd80 0x7f3e2c15a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e2c0d0ec0 0x7f3e2c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3e1806ae70 0x7f3e1806d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 --2- 172.21.15.53:0/786675727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e2c0d06c0 0x7f3e2c0b7c90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 >> 172.21.15.53:0/786675727 conn(0x7f3e2c09bc40 msgr2=0x7f3e2c0c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:02.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 shutdown_connections 2023-10-21T16:03:02.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.007+0000 7f3e31516640 1 -- 172.21.15.53:0/786675727 wait complete. 2023-10-21T16:03:02.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:02.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.260+0000 7fae44aec640 1 Processor -- start 2023-10-21T16:03:02.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.260+0000 7fae44aec640 1 -- start start 2023-10-21T16:03:02.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.261+0000 7fae44aec640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae400cf2b0 0x7fae400d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.261+0000 7fae44aec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 0x7fae4015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.261+0000 7fae44aec640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae400cd520 con 0x7fae400cf690 2023-10-21T16:03:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.261+0000 7fae44aec640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400d1910 con 0x7fae400cf2b0 2023-10-21T16:03:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.261+0000 7fae44aec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400ce080 con 0x7fae400ce510 2023-10-21T16:03:02.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae400cf2b0 0x7fae400d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fae400cf690 0x7fae400d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43788/0 (socket says 172.21.15.53:43788) 2023-10-21T16:03:02.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae400cf2b0 0x7fae400d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60422/0 (socket says 172.21.15.53:60422) 2023-10-21T16:03:02.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3effd640 1 -- 172.21.15.53:0/488168394 learned_addr learned my addr 172.21.15.53:0/488168394 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:02.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3ffff640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 0x7fae4015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 -- 172.21.15.53:0/488168394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 msgr2=0x7fae4015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 0x7fae4015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 -- 172.21.15.53:0/488168394 >> v1:172.21.15.53:6789/0 conn(0x7fae400cf690 legacy=0x7fae400d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3f7fe640 1 -- 172.21.15.53:0/488168394 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400cd750 con 0x7fae400cf2b0 2023-10-21T16:03:02.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3e7fc640 1 -- 172.21.15.53:0/488168394 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529337509 0 0) 0x7fae400cd520 con 0x7fae400cf690 2023-10-21T16:03:02.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.262+0000 7fae3ffff640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 0x7fae4015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:02.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.263+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf2b0 0x7fae400d34d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae30009d90 tx=0x7fae3002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c450f40b2c4b860a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae3e7fc640 1 -- 172.21.15.53:0/488168394 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3002cbd0 con 0x7fae400cf2b0 2023-10-21T16:03:02.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae3e7fc640 1 -- 172.21.15.53:0/488168394 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3002cd30 con 0x7fae400cf2b0 2023-10-21T16:03:02.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae3e7fc640 1 -- 172.21.15.53:0/488168394 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3002cea0 con 0x7fae400cf2b0 2023-10-21T16:03:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 -- 172.21.15.53:0/488168394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf2b0 msgr2=0x7fae400d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf2b0 0x7fae400d34d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae30009d90 tx=0x7fae3002be20 comp rx=0 tx=0).stop 2023-10-21T16:03:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 -- 172.21.15.53:0/488168394 shutdown_connections 2023-10-21T16:03:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400ce510 0x7fae4015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 --2- 172.21.15.53:0/488168394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf2b0 0x7fae400d34d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.264+0000 7fae44aec640 1 -- 172.21.15.53:0/488168394 >> 172.21.15.53:0/488168394 conn(0x7fae4009bc40 msgr2=0x7fae400d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:02.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.265+0000 7fae44aec640 1 -- 172.21.15.53:0/488168394 shutdown_connections 2023-10-21T16:03:02.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.265+0000 7fae44aec640 1 -- 172.21.15.53:0/488168394 wait complete. 2023-10-21T16:03:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.266+0000 7fae44aec640 1 Processor -- start 2023-10-21T16:03:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.267+0000 7fae44aec640 1 -- start start 2023-10-21T16:03:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.267+0000 7fae44aec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 0x7fae4009e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.267+0000 7fae44aec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400cf2b0 0x7fae4009eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae44aec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae44aec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015c180 con 0x7fae400cf690 2023-10-21T16:03:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae44aec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c99e0 con 0x7fae400ce510 2023-10-21T16:03:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae44aec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4009dba0 con 0x7fae400cf2b0 2023-10-21T16:03:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60436/0 (socket says 172.21.15.53:60436) 2023-10-21T16:03:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 -- 172.21.15.53:0/118814719 learned_addr learned my addr 172.21.15.53:0/118814719 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3effd640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400cf2b0 0x7fae4009eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 0x7fae4009e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 -- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400cf2b0 msgr2=0x7fae4009eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400cf2b0 0x7fae4009eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 -- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 msgr2=0x7fae4009e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 0x7fae4009e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3ffff640 1 -- 172.21.15.53:0/118814719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009a70 con 0x7fae400cf690 2023-10-21T16:03:02.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.268+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 0x7fae4009e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:02.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae3ffff640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c7f0 tx=0x7fae3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38007820 con 0x7fae400cf690 2023-10-21T16:03:02.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38007980 con 0x7fae400cf690 2023-10-21T16:03:02.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38007d70 con 0x7fae400cf690 2023-10-21T16:03:02.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4009de80 con 0x7fae400cf690 2023-10-21T16:03:02.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.269+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400c9ca0 con 0x7fae400cf690 2023-10-21T16:03:02.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.270+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae400c9ec0 con 0x7fae400cf690 2023-10-21T16:03:02.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.272+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae380113f0 con 0x7fae400cf690 2023-10-21T16:03:02.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.273+0000 7fae37fff640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 0x7fae2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.273+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fae38089e50 con 0x7fae400cf690 2023-10-21T16:03:02.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.274+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 0x7fae2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.275+0000 7fae3f7fe640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 0x7fae2006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae30036000 tx=0x7fae30030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.277+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae38054aa0 con 0x7fae400cf690 2023-10-21T16:03:02.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.469+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae40074150 con 0x7fae400cf690 2023-10-21T16:03:02.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.470+0000 7fae37fff640 1 -- 172.21.15.53:0/118814719 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae380580c0 con 0x7fae400cf690 2023-10-21T16:03:02.473 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:02.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.472+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 msgr2=0x7fae2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.472+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 0x7fae2006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae30036000 tx=0x7fae30030000 comp rx=0 tx=0).stop 2023-10-21T16:03:02.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.472+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 msgr2=0x7fae4009cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.472+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c7f0 tx=0x7fae3800ccb0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 shutdown_connections 2023-10-21T16:03:02.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae2006ae70 0x7fae2006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae400cf690 0x7fae4009cfc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae400cf2b0 0x7fae4009eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 --2- 172.21.15.53:0/118814719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae400ce510 0x7fae4009e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 >> 172.21.15.53:0/118814719 conn(0x7fae4009bc40 msgr2=0x7fae4015fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:02.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 shutdown_connections 2023-10-21T16:03:02.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.473+0000 7fae44aec640 1 -- 172.21.15.53:0/118814719 wait complete. 2023-10-21T16:03:02.494 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:02.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.720+0000 7fa08cfd2640 1 Processor -- start 2023-10-21T16:03:02.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.720+0000 7fa08cfd2640 1 -- start start 2023-10-21T16:03:02.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa08cfd2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa0880b0680 0x7fa0880c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa08cfd2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0880c94f0 con 0x7fa0880c8b40 2023-10-21T16:03:02.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa08cfd2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0880ccbc0 con 0x7fa08815cae0 2023-10-21T16:03:02.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa08cfd2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0880ce1f0 con 0x7fa0880b0680 2023-10-21T16:03:02.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa07ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa0880c8b40 0x7fa0880aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43790/0 (socket says 172.21.15.53:43790) 2023-10-21T16:03:02.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa07ffff640 1 -- 172.21.15.53:0/2193722436 learned_addr learned my addr 172.21.15.53:0/2193722436 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:02.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.722+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754068744 0 0) 0x7fa0880c94f0 con 0x7fa0880c8b40 2023-10-21T16:03:02.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa074003520 con 0x7fa0880c8b40 2023-10-21T16:03:02.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa087fff640 1 --2- 172.21.15.53:0/2193722436 >> v2:172.21.15.124:3300/0 conn(0x7fa0880b0680 0x7fa0880c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695843371 0 0) 0x7fa074003520 con 0x7fa0880c8b40 2023-10-21T16:03:02.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 >> v2:172.21.15.124:3300/0 conn(0x7fa0880b0680 msgr2=0x7fa0880c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:02.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 --2- 172.21.15.53:0/2193722436 >> v2:172.21.15.124:3300/0 conn(0x7fa0880b0680 0x7fa0880c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 >> v1:172.21.15.124:6789/0 conn(0x7fa08815cae0 legacy=0x7fa0880cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880c9720 con 0x7fa0880c8b40 2023-10-21T16:03:02.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa070001670 con 0x7fa0880c8b40 2023-10-21T16:03:02.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 >> v1:172.21.15.53:6789/0 conn(0x7fa0880c8b40 legacy=0x7fa0880aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074003970 0x7fa074003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0740047c0 0x7fa074004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 0x7fa07400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0880ce1f0 con 0x7fa074005180 2023-10-21T16:03:02.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07400c0d0 con 0x7fa074003970 2023-10-21T16:03:02.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07400c290 con 0x7fa0740047c0 2023-10-21T16:03:02.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.723+0000 7fa087fff640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 0x7fa07400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa0877fe640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074003970 0x7fa074003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa087fff640 1 -- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0740047c0 msgr2=0x7fa074004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:02.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa087fff640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0740047c0 0x7fa074004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa087fff640 1 -- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074003970 msgr2=0x7fa074003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:02.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa087fff640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074003970 0x7fa074003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.724+0000 7fa087fff640 1 -- 172.21.15.53:0/2193722436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074003520 con 0x7fa074005180 2023-10-21T16:03:02.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.725+0000 7fa087fff640 1 -- 172.21.15.53:0/2193722436 reap_dead start 2023-10-21T16:03:02.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.725+0000 7fa087fff640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 0x7fa07400b980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa080007360 tx=0x7fa0740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31771f4a419c2a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.725+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08002ccf0 con 0x7fa074005180 2023-10-21T16:03:02.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.725+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08002ce90 con 0x7fa074005180 2023-10-21T16:03:02.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.725+0000 7fa086ffd640 1 -- 172.21.15.53:0/2193722436 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080003bd0 con 0x7fa074005180 2023-10-21T16:03:02.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 -- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 msgr2=0x7fa07400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 0x7fa07400b980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa080007360 tx=0x7fa0740057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 -- 172.21.15.53:0/2193722436 shutdown_connections 2023-10-21T16:03:02.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/2193722436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa074005180 0x7fa07400b980 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 -- 172.21.15.53:0/2193722436 >> 172.21.15.53:0/2193722436 conn(0x7fa08806c050 msgr2=0x7fa08806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:02.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 -- 172.21.15.53:0/2193722436 shutdown_connections 2023-10-21T16:03:02.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.726+0000 7fa08cfd2640 1 -- 172.21.15.53:0/2193722436 wait complete. 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.728+0000 7fa08cfd2640 1 Processor -- start 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.728+0000 7fa08cfd2640 1 -- start start 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074005180 0x7fa0880b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0880b1650 0x7fa0880c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0880c1fe0 con 0x7fa0880c7840 2023-10-21T16:03:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0880c8590 con 0x7fa074005180 2023-10-21T16:03:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa08cfd2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0880c7cf0 con 0x7fa0880b1650 2023-10-21T16:03:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa087fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa087fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60456/0 (socket says 172.21.15.53:60456) 2023-10-21T16:03:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.729+0000 7fa087fff640 1 -- 172.21.15.53:0/3755078343 learned_addr learned my addr 172.21.15.53:0/3755078343 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa0877fe640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074005180 0x7fa0880b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa07ffff640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0880b1650 0x7fa0880c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 -- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0880b1650 msgr2=0x7fa0880c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0880b1650 0x7fa0880c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 -- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074005180 msgr2=0x7fa0880b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074005180 0x7fa0880b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 -- 172.21.15.53:0/3755078343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880c21c0 con 0x7fa0880c7840 2023-10-21T16:03:02.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.730+0000 7fa087fff640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa08002c760 tx=0x7fa080004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.731+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080004700 con 0x7fa0880c7840 2023-10-21T16:03:02.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.731+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080004860 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.731+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08003f6b0 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.731+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0880c24a0 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.731+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0880c2a60 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.732+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0880cc550 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.733+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa080034440 con 0x7fa0880c7840 2023-10-21T16:03:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.734+0000 7fa084ff9640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 0x7fa06806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.734+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa08003b070 con 0x7fa0880c7840 2023-10-21T16:03:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.735+0000 7fa0877fe640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 0x7fa06806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.736+0000 7fa0877fe640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 0x7fa06806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa078000c00 tx=0x7fa078009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.738+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0880cc550 con 0x7fa0880c7840 2023-10-21T16:03:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.932+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0880cc550 con 0x7fa0880c7840 2023-10-21T16:03:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.933+0000 7fa084ff9640 1 -- 172.21.15.53:0/3755078343 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0880cc550 con 0x7fa0880c7840 2023-10-21T16:03:02.936 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 msgr2=0x7fa06806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 0x7fa06806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa078000c00 tx=0x7fa078009040 comp rx=0 tx=0).stop 2023-10-21T16:03:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 msgr2=0x7fa0880c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa08002c760 tx=0x7fa080004ab0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 shutdown_connections 2023-10-21T16:03:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa06806ae70 0x7fa06806d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0880c7840 0x7fa0880c2f80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0880b1650 0x7fa0880c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 --2- 172.21.15.53:0/3755078343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa074005180 0x7fa0880b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:02.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.936+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 >> 172.21.15.53:0/3755078343 conn(0x7fa08806c050 msgr2=0x7fa0880aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:02.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.937+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 shutdown_connections 2023-10-21T16:03:02.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:02.937+0000 7fa08cfd2640 1 -- 172.21.15.53:0/3755078343 wait complete. 2023-10-21T16:03:02.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:03.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.186+0000 7fd908602640 1 Processor -- start 2023-10-21T16:03:03.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.186+0000 7fd908602640 1 -- start start 2023-10-21T16:03:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.188+0000 7fd908602640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd900160660 0x7fd900070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.188+0000 7fd908602640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9000c9910 0x7fd9000caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.188+0000 7fd908602640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90006e150 con 0x7fd9000cb450 2023-10-21T16:03:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.188+0000 7fd908602640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9000cb8c0 con 0x7fd900160660 2023-10-21T16:03:03.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.188+0000 7fd908602640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90006de70 con 0x7fd9000c9910 2023-10-21T16:03:03.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd900160660 0x7fd900070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd900160660 0x7fd900070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60470/0 (socket says 172.21.15.53:60470) 2023-10-21T16:03:03.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 -- 172.21.15.53:0/216851646 learned_addr learned my addr 172.21.15.53:0/216851646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:03.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd905b9d640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9000c9910 0x7fd9000caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 -- 172.21.15.53:0/216851646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9000c9910 msgr2=0x7fd9000caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:03.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9000c9910 0x7fd9000caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 -- 172.21.15.53:0/216851646 >> v1:172.21.15.124:6789/0 conn(0x7fd9000cb450 legacy=0x7fd9000b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.189+0000 7fd90639e640 1 -- 172.21.15.53:0/216851646 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000c8e30 con 0x7fd900160660 2023-10-21T16:03:03.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.191+0000 7fd90639e640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd900160660 0x7fd900070cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009870 tx=0x7fd8fc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5201c1701a7f9517 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.191+0000 7fd90539c640 1 -- 172.21.15.53:0/216851646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc030840 con 0x7fd900160660 2023-10-21T16:03:03.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.191+0000 7fd90539c640 1 -- 172.21.15.53:0/216851646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc0309a0 con 0x7fd900160660 2023-10-21T16:03:03.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.191+0000 7fd90539c640 1 -- 172.21.15.53:0/216851646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc030d30 con 0x7fd900160660 2023-10-21T16:03:03.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 -- 172.21.15.53:0/216851646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd900160660 msgr2=0x7fd900070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd900160660 0x7fd900070cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009870 tx=0x7fd8fc02b980 comp rx=0 tx=0).stop 2023-10-21T16:03:03.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 -- 172.21.15.53:0/216851646 shutdown_connections 2023-10-21T16:03:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9000c9910 0x7fd9000caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 --2- 172.21.15.53:0/216851646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd900160660 0x7fd900070cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 -- 172.21.15.53:0/216851646 >> 172.21.15.53:0/216851646 conn(0x7fd90006a5d0 msgr2=0x7fd9000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 -- 172.21.15.53:0/216851646 shutdown_connections 2023-10-21T16:03:03.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.192+0000 7fd908602640 1 -- 172.21.15.53:0/216851646 wait complete. 2023-10-21T16:03:03.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.194+0000 7fd908602640 1 Processor -- start 2023-10-21T16:03:03.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.194+0000 7fd908602640 1 -- start start 2023-10-21T16:03:03.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9000c9910 0x7fd900150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 0x7fd9001598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001583c0 con 0x7fd9000cb450 2023-10-21T16:03:03.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001f6c10 con 0x7fd9000c9910 2023-10-21T16:03:03.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd908602640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001579e0 con 0x7fd900160660 2023-10-21T16:03:03.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd905b9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd905b9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60472/0 (socket says 172.21.15.53:60472) 2023-10-21T16:03:03.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.195+0000 7fd905b9d640 1 -- 172.21.15.53:0/2241769754 learned_addr learned my addr 172.21.15.53:0/2241769754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:03.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd90639e640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9000c9910 0x7fd900150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd906b9f640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 0x7fd9001598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 -- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 msgr2=0x7fd9001598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 0x7fd9001598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 -- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9000c9910 msgr2=0x7fd900150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9000c9910 0x7fd900150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 -- 172.21.15.53:0/2241769754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0094d0 con 0x7fd9000cb450 2023-10-21T16:03:03.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd906b9f640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 0x7fd9001598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:03.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd905b9d640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000d830 tx=0x7fd8f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000b6c0 con 0x7fd9000cb450 2023-10-21T16:03:03.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f000b820 con 0x7fd9000cb450 2023-10-21T16:03:03.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9001580e0 con 0x7fd9000cb450 2023-10-21T16:03:03.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.196+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000bbb0 con 0x7fd9000cb450 2023-10-21T16:03:03.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.197+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd900157700 con 0x7fd9000cb450 2023-10-21T16:03:03.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.198+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90004e160 con 0x7fd9000cb450 2023-10-21T16:03:03.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.199+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8f001e460 con 0x7fd9000cb450 2023-10-21T16:03:03.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.200+0000 7fd8f6ffd640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 0x7fd8e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.200+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd8f008a190 con 0x7fd9000cb450 2023-10-21T16:03:03.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.200+0000 7fd90639e640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 0x7fd8e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.202+0000 7fd90639e640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 0x7fd8e406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc036000 tx=0x7fd8fc03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.204+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8f0054e70 con 0x7fd9000cb450 2023-10-21T16:03:03.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.401+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd9000cd7f0 con 0x7fd9000cb450 2023-10-21T16:03:03.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.402+0000 7fd8f6ffd640 1 -- 172.21.15.53:0/2241769754 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8f0058490 con 0x7fd9000cb450 2023-10-21T16:03:03.405 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 msgr2=0x7fd8e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 0x7fd8e406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc036000 tx=0x7fd8fc03b040 comp rx=0 tx=0).stop 2023-10-21T16:03:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 msgr2=0x7fd9001f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000d830 tx=0x7fd8f000dcf0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 shutdown_connections 2023-10-21T16:03:03.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd900160660 0x7fd9001598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd8e406ae70 0x7fd8e406d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9000cb450 0x7fd9001f66d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 --2- 172.21.15.53:0/2241769754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9000c9910 0x7fd900150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 >> 172.21.15.53:0/2241769754 conn(0x7fd90006a5d0 msgr2=0x7fd9000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 shutdown_connections 2023-10-21T16:03:03.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.405+0000 7fd908602640 1 -- 172.21.15.53:0/2241769754 wait complete. 2023-10-21T16:03:03.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-21T16:03:03.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.656+0000 7fb140901640 1 Processor -- start 2023-10-21T16:03:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.656+0000 7fb140901640 1 -- start start 2023-10-21T16:03:03.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.657+0000 7fb140901640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb13c06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.657+0000 7fb140901640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c1614f0 con 0x7fb13c071700 2023-10-21T16:03:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.657+0000 7fb140901640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c161a20 con 0x7fb13c0b1cd0 2023-10-21T16:03:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.658+0000 7fb140901640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0af1f0 con 0x7fb13c069d50 2023-10-21T16:03:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.658+0000 7fb139d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb13c071700 0x7fb13c0ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43794/0 (socket says 172.21.15.53:43794) 2023-10-21T16:03:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.658+0000 7fb139d9b640 1 -- 172.21.15.53:0/2947494665 learned_addr learned my addr 172.21.15.53:0/2947494665 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:03.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.658+0000 7fb13a59c640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb13c06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.658+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865459210 0 0) 0x7fb13c1614f0 con 0x7fb13c071700 2023-10-21T16:03:03.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb128003520 con 0x7fb13c071700 2023-10-21T16:03:03.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719711914 0 0) 0x7fb13c161a20 con 0x7fb13c0b1cd0 2023-10-21T16:03:03.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13c1614f0 con 0x7fb13c0b1cd0 2023-10-21T16:03:03.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520892695 0 0) 0x7fb128003520 con 0x7fb13c071700 2023-10-21T16:03:03.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 msgr2=0x7fb13c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:03.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb13c06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 >> v1:172.21.15.124:6789/0 conn(0x7fb13c0b1cd0 legacy=0x7fb13c0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c06a670 con 0x7fb13c071700 2023-10-21T16:03:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb1240019f0 con 0x7fb13c071700 2023-10-21T16:03:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 >> v1:172.21.15.53:6789/0 conn(0x7fb13c071700 legacy=0x7fb13c0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb128003b30 0x7fb128003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb128004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb1280051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0af1f0 con 0x7fb1280047c0 2023-10-21T16:03:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12800bb10 con 0x7fb128003b30 2023-10-21T16:03:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.659+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12800bca0 con 0x7fb13c069d50 2023-10-21T16:03:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb128004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb13a59c640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb128003b30 0x7fb128003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb13ad9d640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb1280051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 -- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 msgr2=0x7fb1280051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb1280051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 -- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb128003b30 msgr2=0x7fb128003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb128003b30 0x7fb128003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb139d9b640 1 -- 172.21.15.53:0/2947494665 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb128003520 con 0x7fb1280047c0 2023-10-21T16:03:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb13ad9d640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c069d50 0x7fb1280051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.660+0000 7fb13a59c640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb128003b30 0x7fb128003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:03.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb139d9b640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb128004c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb12400a730 tx=0x7fb1280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77176f8b6b0ea66 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124034630 con 0x7fb1280047c0 2023-10-21T16:03:03.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12402f070 con 0x7fb1280047c0 2023-10-21T16:03:03.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb13959a640 1 -- 172.21.15.53:0/2947494665 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240349f0 con 0x7fb1280047c0 2023-10-21T16:03:03.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb140901640 1 -- 172.21.15.53:0/2947494665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 msgr2=0x7fb128004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb140901640 1 --2- 172.21.15.53:0/2947494665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb128004c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb12400a730 tx=0x7fb1280057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb13ad9d640 1 -- 172.21.15.53:0/2947494665 reap_dead start 2023-10-21T16:03:03.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb140901640 1 -- 172.21.15.53:0/2947494665 shutdown_connections 2023-10-21T16:03:03.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.662+0000 7fb140901640 1 -- 172.21.15.53:0/2947494665 >> 172.21.15.53:0/2947494665 conn(0x7fb13c06d450 msgr2=0x7fb13c06d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:03.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.663+0000 7fb140901640 1 -- 172.21.15.53:0/2947494665 shutdown_connections 2023-10-21T16:03:03.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.663+0000 7fb140901640 1 -- 172.21.15.53:0/2947494665 wait complete. 2023-10-21T16:03:03.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.664+0000 7fb140901640 1 Processor -- start 2023-10-21T16:03:03.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.664+0000 7fb140901640 1 -- start start 2023-10-21T16:03:03.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c0ab8d0 0x7fb13c0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb13c0abd50 0x7fb13c0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c1f23b0 con 0x7fb1280047c0 2023-10-21T16:03:03.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0ade40 con 0x7fb13c0abd50 2023-10-21T16:03:03.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb140901640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0c4830 con 0x7fb13c0ab8d0 2023-10-21T16:03:03.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60488/0 (socket says 172.21.15.53:60488) 2023-10-21T16:03:03.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 -- 172.21.15.53:0/1511416140 learned_addr learned my addr 172.21.15.53:0/1511416140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:03.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13ad9d640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb13c0abd50 0x7fb13c0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb139d9b640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c0ab8d0 0x7fb13c0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 -- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c0ab8d0 msgr2=0x7fb13c0ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c0ab8d0 0x7fb13c0ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 -- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb13c0abd50 msgr2=0x7fb13c0ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb13c0abd50 0x7fb13c0ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.666+0000 7fb13a59c640 1 -- 172.21.15.53:0/1511416140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c0c4b10 con 0x7fb1280047c0 2023-10-21T16:03:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb13a59c640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0116d0 tx=0x7fb12c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00f9c0 con 0x7fb1280047c0 2023-10-21T16:03:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00fb20 con 0x7fb1280047c0 2023-10-21T16:03:03.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00fed0 con 0x7fb1280047c0 2023-10-21T16:03:03.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c0c4df0 con 0x7fb1280047c0 2023-10-21T16:03:03.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.667+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c0c6110 con 0x7fb1280047c0 2023-10-21T16:03:03.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.668+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0c6300 con 0x7fb1280047c0 2023-10-21T16:03:03.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.669+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb12c015460 con 0x7fb1280047c0 2023-10-21T16:03:03.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.670+0000 7fb132ffd640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 0x7fb11c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:03.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.670+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb12c08a290 con 0x7fb1280047c0 2023-10-21T16:03:03.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.671+0000 7fb139d9b640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 0x7fb11c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:03.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.672+0000 7fb139d9b640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 0x7fb11c06d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb124034000 tx=0x7fb12402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:03.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.674+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb13c0c6300 con 0x7fb1280047c0 2023-10-21T16:03:03.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.935+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fb13c0c6300 con 0x7fb1280047c0 2023-10-21T16:03:03.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.991+0000 7fb132ffd640 1 -- 172.21.15.53:0/1511416140 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb13c0c6300 con 0x7fb1280047c0 2023-10-21T16:03:03.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 msgr2=0x7fb11c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 0x7fb11c06d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb124034000 tx=0x7fb12402f000 comp rx=0 tx=0).stop 2023-10-21T16:03:03.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 msgr2=0x7fb13c1547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:03.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0116d0 tx=0x7fb12c011b90 comp rx=0 tx=0).stop 2023-10-21T16:03:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 shutdown_connections 2023-10-21T16:03:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb13c0abd50 0x7fb13c0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb13c0ab8d0 0x7fb13c0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb11c06ae70 0x7fb11c06d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 --2- 172.21.15.53:0/1511416140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb1280047c0 0x7fb13c1547c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:03.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 >> 172.21.15.53:0/1511416140 conn(0x7fb13c06d450 msgr2=0x7fb13c0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:03.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 shutdown_connections 2023-10-21T16:03:03.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:03.994+0000 7fb140901640 1 -- 172.21.15.53:0/1511416140 wait complete. 2023-10-21T16:03:04.019 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:03:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.250+0000 7f73ae47c640 1 Processor -- start 2023-10-21T16:03:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.250+0000 7f73ae47c640 1 -- start start 2023-10-21T16:03:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73ae47c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a806c3a0 0x7f73a80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73ae47c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73a815aa00 0x7f73a80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73ae47c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a815b310 con 0x7f73a806db60 2023-10-21T16:03:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73ae47c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a80ca230 con 0x7f73a806c3a0 2023-10-21T16:03:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73ae47c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a806d290 con 0x7f73a815aa00 2023-10-21T16:03:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73a7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a806c3a0 0x7f73a80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73a7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73a806c3a0 0x7f73a80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56120/0 (socket says 172.21.15.53:56120) 2023-10-21T16:03:04.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.252+0000 7f73a7fff640 1 -- 172.21.15.53:0/1921169830 learned_addr learned my addr 172.21.15.53:0/1921169830 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:04.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.253+0000 7f73a7fff640 1 -- 172.21.15.53:0/1921169830 >> v2:172.21.15.124:3300/0 conn(0x7f73a815aa00 msgr2=0x7f73a80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.253+0000 7f73aca19640 1 --2- 172.21.15.53:0/1921169830 >> v2:172.21.15.124:3300/0 conn(0x7f73a815aa00 0x7f73a80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.253+0000 7f73a7fff640 1 --2- 172.21.15.53:0/1921169830 >> v2:172.21.15.124:3300/0 conn(0x7f73a815aa00 0x7f73a80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.253+0000 7f73a7fff640 1 -- 172.21.15.53:0/1921169830 >> v1:172.21.15.53:6789/0 conn(0x7f73a806db60 legacy=0x7f73a806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.253+0000 7f73a7fff640 1 -- 172.21.15.53:0/1921169830 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80c9860 con 0x7f73a806c3a0 2023-10-21T16:03:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.254+0000 7f73a7fff640 1 --2- 172.21.15.53:0/1921169830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a806c3a0 0x7f73a80c9cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73a80c57a0 tx=0x7f739802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f626425abd8a73 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.254+0000 7f73a77fe640 1 -- 172.21.15.53:0/1921169830 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398036440 con 0x7f73a806c3a0 2023-10-21T16:03:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.254+0000 7f73a77fe640 1 -- 172.21.15.53:0/1921169830 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980365a0 con 0x7f73a806c3a0 2023-10-21T16:03:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.254+0000 7f73a77fe640 1 -- 172.21.15.53:0/1921169830 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398036930 con 0x7f73a806c3a0 2023-10-21T16:03:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 -- 172.21.15.53:0/1921169830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a806c3a0 msgr2=0x7f73a80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1921169830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a806c3a0 0x7f73a80c9cf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73a80c57a0 tx=0x7f739802e170 comp rx=0 tx=0).stop 2023-10-21T16:03:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 -- 172.21.15.53:0/1921169830 shutdown_connections 2023-10-21T16:03:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1921169830 >> v2:172.21.15.124:3300/0 conn(0x7f73a815aa00 0x7f73a80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1921169830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a806c3a0 0x7f73a80c9cf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 -- 172.21.15.53:0/1921169830 >> 172.21.15.53:0/1921169830 conn(0x7f73a806a5d0 msgr2=0x7f73a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:04.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 -- 172.21.15.53:0/1921169830 shutdown_connections 2023-10-21T16:03:04.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.255+0000 7f73ae47c640 1 -- 172.21.15.53:0/1921169830 wait complete. 2023-10-21T16:03:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.257+0000 7f73ae47c640 1 Processor -- start 2023-10-21T16:03:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.257+0000 7f73ae47c640 1 -- start start 2023-10-21T16:03:04.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73a806c3a0 0x7f73a80c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73a806db60 0x7f73a81f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80c7650 con 0x7f73a815aa00 2023-10-21T16:03:04.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80c84b0 con 0x7f73a806db60 2023-10-21T16:03:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73ae47c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80c6760 con 0x7f73a806c3a0 2023-10-21T16:03:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73aca19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73aca19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56132/0 (socket says 172.21.15.53:56132) 2023-10-21T16:03:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.258+0000 7f73aca19640 1 -- 172.21.15.53:0/1739574931 learned_addr learned my addr 172.21.15.53:0/1739574931 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:04.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73a7fff640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73a806c3a0 0x7f73a80c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f739ffff640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73a806db60 0x7f73a81f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 -- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73a806c3a0 msgr2=0x7f73a80c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73a806c3a0 0x7f73a80c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 -- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73a806db60 msgr2=0x7f73a81f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73a806db60 0x7f73a81f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 -- 172.21.15.53:0/1739574931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739800b770 con 0x7f73a815aa00 2023-10-21T16:03:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73aca19640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73a000cc60 tx=0x7f73a000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a000ce70 con 0x7f73a815aa00 2023-10-21T16:03:04.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0003990 con 0x7f73a815aa00 2023-10-21T16:03:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a80c78a0 con 0x7f73a815aa00 2023-10-21T16:03:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.259+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0003d20 con 0x7f73a815aa00 2023-10-21T16:03:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.260+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a80c7e60 con 0x7f73a815aa00 2023-10-21T16:03:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.261+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a80c8620 con 0x7f73a815aa00 2023-10-21T16:03:04.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.262+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f73a0003e80 con 0x7f73a815aa00 2023-10-21T16:03:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.263+0000 7f73a57fa640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 0x7f738806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:04.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.263+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f73a0089d00 con 0x7f73a815aa00 2023-10-21T16:03:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.263+0000 7f73a7fff640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 0x7f738806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.265+0000 7f73a7fff640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 0x7f738806d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7398038000 tx=0x7f7398008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:04.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.267+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73a00549f0 con 0x7f73a815aa00 2023-10-21T16:03:04.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.465+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f73a804e160 con 0x7f73a815aa00 2023-10-21T16:03:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.469+0000 7f73a57fa640 1 -- 172.21.15.53:0/1739574931 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f73a001d070 con 0x7f73a815aa00 2023-10-21T16:03:04.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.472+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 msgr2=0x7f738806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.472+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 0x7f738806d360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7398038000 tx=0x7f7398008040 comp rx=0 tx=0).stop 2023-10-21T16:03:04.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.472+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 msgr2=0x7f73a81566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:04.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.472+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73a000cc60 tx=0x7f73a000a5c0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 shutdown_connections 2023-10-21T16:03:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f738806aea0 0x7f738806d360 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73a815aa00 0x7f73a81566e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73a806db60 0x7f73a81f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 --2- 172.21.15.53:0/1739574931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73a806c3a0 0x7f73a80c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 >> 172.21.15.53:0/1739574931 conn(0x7f73a806a5d0 msgr2=0x7f73a8159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 shutdown_connections 2023-10-21T16:03:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:04.473+0000 7f73ae47c640 1 -- 172.21.15.53:0/1739574931 wait complete. 2023-10-21T16:03:04.494 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:03:04.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:03:04.609 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:03:04.748 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:03:04.749 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:04.749 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:04.812 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:04.813 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:04.869 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:04.870 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:04.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:05.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.159+0000 7fa95359e640 1 Processor -- start 2023-10-21T16:03:05.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.160+0000 7fa95359e640 1 -- start start 2023-10-21T16:03:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.161+0000 7fa95359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9540d16f0 0x7fa9540cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.161+0000 7fa95359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540cfa50 0x7fa9540ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.161+0000 7fa95359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9540cea50 con 0x7fa95415cc60 2023-10-21T16:03:05.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.161+0000 7fa95359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95415d040 con 0x7fa9540d16f0 2023-10-21T16:03:05.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.161+0000 7fa95359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540d0650 con 0x7fa9540cfa50 2023-10-21T16:03:05.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9540d16f0 0x7fa9540cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9540d16f0 0x7fa9540cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56144/0 (socket says 172.21.15.53:56144) 2023-10-21T16:03:05.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 -- 172.21.15.53:0/1505899593 learned_addr learned my addr 172.21.15.53:0/1505899593 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:05.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 -- 172.21.15.53:0/1505899593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540cfa50 msgr2=0x7fa9540ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 --2- 172.21.15.53:0/1505899593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540cfa50 0x7fa9540ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 -- 172.21.15.53:0/1505899593 >> v1:172.21.15.124:6789/0 conn(0x7fa95415cc60 legacy=0x7fa9540cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.162+0000 7fa951d9b640 1 -- 172.21.15.53:0/1505899593 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540d0930 con 0x7fa9540d16f0 2023-10-21T16:03:05.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.163+0000 7fa951d9b640 1 --2- 172.21.15.53:0/1505899593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9540d16f0 0x7fa9540cdfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1079016c2eb4ffc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95159a640 1 -- 172.21.15.53:0/1505899593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c031870 con 0x7fa9540d16f0 2023-10-21T16:03:05.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95159a640 1 -- 172.21.15.53:0/1505899593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c0319d0 con 0x7fa9540d16f0 2023-10-21T16:03:05.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95159a640 1 -- 172.21.15.53:0/1505899593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c031d60 con 0x7fa9540d16f0 2023-10-21T16:03:05.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 -- 172.21.15.53:0/1505899593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9540d16f0 msgr2=0x7fa9540cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 --2- 172.21.15.53:0/1505899593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9540d16f0 0x7fa9540cdfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c02b910 comp rx=0 tx=0).stop 2023-10-21T16:03:05.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 -- 172.21.15.53:0/1505899593 shutdown_connections 2023-10-21T16:03:05.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 --2- 172.21.15.53:0/1505899593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540cfa50 0x7fa9540ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 --2- 172.21.15.53:0/1505899593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9540d16f0 0x7fa9540cdfd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 -- 172.21.15.53:0/1505899593 >> 172.21.15.53:0/1505899593 conn(0x7fa95409bcf0 msgr2=0x7fa9540d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:05.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 -- 172.21.15.53:0/1505899593 shutdown_connections 2023-10-21T16:03:05.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.164+0000 7fa95359e640 1 -- 172.21.15.53:0/1505899593 wait complete. 2023-10-21T16:03:05.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.166+0000 7fa95359e640 1 Processor -- start 2023-10-21T16:03:05.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.166+0000 7fa95359e640 1 -- start start 2023-10-21T16:03:05.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 0x7fa9540c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540d16f0 0x7fa954158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95415d570 con 0x7fa95415cc60 2023-10-21T16:03:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa954160110 con 0x7fa9540cfa50 2023-10-21T16:03:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.167+0000 7fa95359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540c8c00 con 0x7fa9540d16f0 2023-10-21T16:03:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56160/0 (socket says 172.21.15.53:56160) 2023-10-21T16:03:05.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 -- 172.21.15.53:0/3545331790 learned_addr learned my addr 172.21.15.53:0/3545331790 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:05.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa95259c640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 0x7fa9540c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa951d9b640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540d16f0 0x7fa954158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 -- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540d16f0 msgr2=0x7fa954158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540d16f0 0x7fa954158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 -- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 msgr2=0x7fa9540c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 0x7fa9540c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa952d9d640 1 -- 172.21.15.53:0/3545331790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0094d0 con 0x7fa95415cc60 2023-10-21T16:03:05.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.168+0000 7fa95259c640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 0x7fa9540c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:05.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa952d9d640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b3e0 tx=0x7fa94c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011830 con 0x7fa95415cc60 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c011990 con 0x7fa95415cc60 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011d20 con 0x7fa95415cc60 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9540c8ee0 con 0x7fa95415cc60 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.169+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954159420 con 0x7fa95415cc60 2023-10-21T16:03:05.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.170+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95404e160 con 0x7fa95415cc60 2023-10-21T16:03:05.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.171+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa94c01a400 con 0x7fa95415cc60 2023-10-21T16:03:05.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.172+0000 7fa94affd640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 0x7fa93406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.172+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa94c089ad0 con 0x7fa95415cc60 2023-10-21T16:03:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.173+0000 7fa95259c640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 0x7fa93406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.174+0000 7fa95259c640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 0x7fa93406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa9540b5500 tx=0x7fa944009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.177+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa94c0547b0 con 0x7fa95415cc60 2023-10-21T16:03:05.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.371+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9540d3080 con 0x7fa95415cc60 2023-10-21T16:03:05.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.373+0000 7fa94affd640 1 -- 172.21.15.53:0/3545331790 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fa94c015090 con 0x7fa95415cc60 2023-10-21T16:03:05.377 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:05.377 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 msgr2=0x7fa93406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 0x7fa93406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa9540b5500 tx=0x7fa944009040 comp rx=0 tx=0).stop 2023-10-21T16:03:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 msgr2=0x7fa9540c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00b3e0 tx=0x7fa94c00b8a0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 shutdown_connections 2023-10-21T16:03:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa93406ae70 0x7fa93406d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa95415cc60 0x7fa9540c8030 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9540d16f0 0x7fa954158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 --2- 172.21.15.53:0/3545331790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9540cfa50 0x7fa9540c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 >> 172.21.15.53:0/3545331790 conn(0x7fa95409bcf0 msgr2=0x7fa9540b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.376+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 shutdown_connections 2023-10-21T16:03:05.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.377+0000 7fa95359e640 1 -- 172.21.15.53:0/3545331790 wait complete. 2023-10-21T16:03:05.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:05.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:05.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.636+0000 7feaf9301640 1 Processor -- start 2023-10-21T16:03:05.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.637+0000 7feaf9301640 1 -- start start 2023-10-21T16:03:05.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf9301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feaf40cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf9301640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feaf40cb6b0 0x7feaf40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf9301640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf40cbb30 con 0x7feaf40b4070 2023-10-21T16:03:05.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf9301640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf41436d0 con 0x7feaf40cb6b0 2023-10-21T16:03:05.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf9301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415ae60 con 0x7feaf41432f0 2023-10-21T16:03:05.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feaf40b4070 0x7feaf406c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49306/0 (socket says 172.21.15.53:49306) 2023-10-21T16:03:05.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf37fe640 1 -- 172.21.15.53:0/3719733188 learned_addr learned my addr 172.21.15.53:0/3719733188 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:05.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> v2:172.21.15.124:3300/0 conn(0x7feaf40cb6b0 0x7feaf40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feaf40cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628812735 0 0) 0x7feaf40cbb30 con 0x7feaf40b4070 2023-10-21T16:03:05.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.639+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae0005730 con 0x7feaf40b4070 2023-10-21T16:03:05.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3341937020 0 0) 0x7feae0005730 con 0x7feaf40b4070 2023-10-21T16:03:05.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 msgr2=0x7feaf40cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feaf40cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 >> v2:172.21.15.124:3300/0 conn(0x7feaf40cb6b0 msgr2=0x7feaf40b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 --2- 172.21.15.53:0/3719733188 >> v2:172.21.15.124:3300/0 conn(0x7feaf40cb6b0 0x7feaf40b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf415d440 con 0x7feaf40b4070 2023-10-21T16:03:05.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7feaec001670 con 0x7feaf40b4070 2023-10-21T16:03:05.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 >> v1:172.21.15.53:6789/0 conn(0x7feaf40b4070 legacy=0x7feaf406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feae0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae0006150 0x7feae00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae0006f10 0x7feae0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf41436d0 con 0x7feae0006150 2023-10-21T16:03:05.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415ae60 con 0x7feae0006f10 2023-10-21T16:03:05.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae000dc40 con 0x7feaf41432f0 2023-10-21T16:03:05.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> v2:172.21.15.124:3300/0 conn(0x7feaf40cb6b0 0x7feaf40b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:03:05.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.640+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae0006150 0x7feae00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feae0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf37fe640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae0006f10 0x7feae0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf27fc640 1 -- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 msgr2=0x7feae0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feae0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf27fc640 1 -- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae0006f10 msgr2=0x7feae0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:05.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae0006f10 0x7feae0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf27fc640 1 -- 172.21.15.53:0/3719733188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0005730 con 0x7feae0006150 2023-10-21T16:03:05.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf37fe640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae0006f10 0x7feae0007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:05.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.641+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feaf41432f0 0x7feae0005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:05.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.642+0000 7feaf27fc640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae0006150 0x7feae00069d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feadc009c70 tx=0x7feae0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0462b1cdc86a7a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.642+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc004020 con 0x7feae0006150 2023-10-21T16:03:05.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.642+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc004180 con 0x7feae0006150 2023-10-21T16:03:05.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.642+0000 7feaf1ffb640 1 -- 172.21.15.53:0/3719733188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02c980 con 0x7feae0006150 2023-10-21T16:03:05.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 -- 172.21.15.53:0/3719733188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae0006150 msgr2=0x7feae00069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 --2- 172.21.15.53:0/3719733188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae0006150 0x7feae00069d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feadc009c70 tx=0x7feae0007a00 comp rx=0 tx=0).stop 2023-10-21T16:03:05.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf37fe640 1 -- 172.21.15.53:0/3719733188 reap_dead start 2023-10-21T16:03:05.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 -- 172.21.15.53:0/3719733188 shutdown_connections 2023-10-21T16:03:05.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 -- 172.21.15.53:0/3719733188 >> 172.21.15.53:0/3719733188 conn(0x7feaf4069d50 msgr2=0x7feaf406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:05.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 -- 172.21.15.53:0/3719733188 shutdown_connections 2023-10-21T16:03:05.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.643+0000 7feaf9301640 1 -- 172.21.15.53:0/3719733188 wait complete. 2023-10-21T16:03:05.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.645+0000 7feaf9301640 1 Processor -- start 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.645+0000 7feaf9301640 1 -- start start 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae0006150 0x7feaf40aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feaf415c1a0 0x7feaf415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40a9070 con 0x7feaf41f16b0 2023-10-21T16:03:05.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40a8480 con 0x7feaf415c1a0 2023-10-21T16:03:05.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf9301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40a8740 con 0x7feae0006150 2023-10-21T16:03:05.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56176/0 (socket says 172.21.15.53:56176) 2023-10-21T16:03:05.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.646+0000 7feaf27fc640 1 -- 172.21.15.53:0/1951231451 learned_addr learned my addr 172.21.15.53:0/1951231451 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:05.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf37fe640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feaf415c1a0 0x7feaf415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae0006150 0x7feaf40aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 -- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae0006150 msgr2=0x7feaf40aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae0006150 0x7feaf40aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 -- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feaf415c1a0 msgr2=0x7feaf415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feaf415c1a0 0x7feaf415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 -- 172.21.15.53:0/1951231451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40a8a20 con 0x7feaf41f16b0 2023-10-21T16:03:05.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf27fc640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feadc009d90 tx=0x7feadc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc005ba0 con 0x7feaf41f16b0 2023-10-21T16:03:05.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc005d00 con 0x7feaf41f16b0 2023-10-21T16:03:05.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc006260 con 0x7feaf41f16b0 2023-10-21T16:03:05.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.647+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf40a8d00 con 0x7feaf41f16b0 2023-10-21T16:03:05.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.648+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf415cb40 con 0x7feaf41f16b0 2023-10-21T16:03:05.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.649+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf40a91e0 con 0x7feaf41f16b0 2023-10-21T16:03:05.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.650+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feadc0063c0 con 0x7feaf41f16b0 2023-10-21T16:03:05.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.651+0000 7feaeb7fe640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 0x7fead406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:05.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.651+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feadc0abef0 con 0x7feaf41f16b0 2023-10-21T16:03:05.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.651+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 0x7fead406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:05.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.652+0000 7feaf2ffd640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 0x7fead406d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feae40073e0 tx=0x7feae400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:05.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.655+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feadc076be0 con 0x7feaf41f16b0 2023-10-21T16:03:05.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.850+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feaf404e160 con 0x7feaf41f16b0 2023-10-21T16:03:05.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.851+0000 7feaeb7fe640 1 -- 172.21.15.53:0/1951231451 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7feadc04d020 con 0x7feaf41f16b0 2023-10-21T16:03:05.854 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:05.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 msgr2=0x7fead406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 0x7fead406d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feae40073e0 tx=0x7feae400d040 comp rx=0 tx=0).stop 2023-10-21T16:03:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 msgr2=0x7feaf41f1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feadc009d90 tx=0x7feadc004730 comp rx=0 tx=0).stop 2023-10-21T16:03:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 shutdown_connections 2023-10-21T16:03:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feaf415c1a0 0x7feaf415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fead406aea0 0x7fead406d360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feaf41f16b0 0x7feaf41f1b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 --2- 172.21.15.53:0/1951231451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae0006150 0x7feaf40aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 >> 172.21.15.53:0/1951231451 conn(0x7feaf4069d50 msgr2=0x7feaf40b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 shutdown_connections 2023-10-21T16:03:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:05.854+0000 7feaf9301640 1 -- 172.21.15.53:0/1951231451 wait complete. 2023-10-21T16:03:05.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:06.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.098+0000 7fb1871ce640 1 Processor -- start 2023-10-21T16:03:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.098+0000 7fb1871ce640 1 -- start start 2023-10-21T16:03:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.099+0000 7fb1871ce640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb1800ca370 0x7fb18006d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.099+0000 7fb1871ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18006c3a0 con 0x7fb1800702b0 2023-10-21T16:03:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.099+0000 7fb1871ce640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18006c640 con 0x7fb1800b21f0 2023-10-21T16:03:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.099+0000 7fb1871ce640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18006c920 con 0x7fb1800ca370 2023-10-21T16:03:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.100+0000 7fb184f6a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb1800702b0 0x7fb180070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49316/0 (socket says 172.21.15.53:49316) 2023-10-21T16:03:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.100+0000 7fb184f6a640 1 -- 172.21.15.53:0/2118430480 learned_addr learned my addr 172.21.15.53:0/2118430480 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.100+0000 7fb18576b640 1 --2- 172.21.15.53:0/2118430480 >> v2:172.21.15.124:3300/0 conn(0x7fb1800ca370 0x7fb18006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.100+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208129950 0 0) 0x7fb18006c3a0 con 0x7fb1800702b0 2023-10-21T16:03:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.100+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb170003520 con 0x7fb1800702b0 2023-10-21T16:03:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2544807271 0 0) 0x7fb170003520 con 0x7fb1800702b0 2023-10-21T16:03:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 >> v2:172.21.15.124:3300/0 conn(0x7fb1800ca370 msgr2=0x7fb18006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/2118430480 >> v2:172.21.15.124:3300/0 conn(0x7fb1800ca370 0x7fb18006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 >> v1:172.21.15.124:6789/0 conn(0x7fb1800b21f0 legacy=0x7fb1800c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18006cc00 con 0x7fb1800702b0 2023-10-21T16:03:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb174001670 con 0x7fb1800702b0 2023-10-21T16:03:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 >> v1:172.21.15.53:6789/0 conn(0x7fb1800702b0 legacy=0x7fb180070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 0x7fb170003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1700047c0 0x7fb170004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1700051f0 0x7fb17000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006c920 con 0x7fb170003970 2023-10-21T16:03:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17000e360 con 0x7fb1700051f0 2023-10-21T16:03:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17000e520 con 0x7fb1700047c0 2023-10-21T16:03:06.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.101+0000 7fb184f6a640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 0x7fb170003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb18576b640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1700051f0 0x7fb17000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb17ffff640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1700047c0 0x7fb170004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb184f6a640 1 -- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1700047c0 msgr2=0x7fb170004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:06.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb184f6a640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1700047c0 0x7fb170004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb184f6a640 1 -- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1700051f0 msgr2=0x7fb17000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb184f6a640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1700051f0 0x7fb17000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb184f6a640 1 -- 172.21.15.53:0/2118430480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170003520 con 0x7fb170003970 2023-10-21T16:03:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.102+0000 7fb18576b640 1 -- 172.21.15.53:0/2118430480 reap_dead start 2023-10-21T16:03:06.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.104+0000 7fb184f6a640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 0x7fb170003d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c190 tx=0x7fb1700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208db4cbec7b0a03 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.104+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174005ad0 con 0x7fb170003970 2023-10-21T16:03:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.104+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174005c30 con 0x7fb170003970 2023-10-21T16:03:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.104+0000 7fb17f7fe640 1 -- 172.21.15.53:0/2118430480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174007070 con 0x7fb170003970 2023-10-21T16:03:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 -- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 msgr2=0x7fb170003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 0x7fb170003d50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c190 tx=0x7fb1700057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 -- 172.21.15.53:0/2118430480 shutdown_connections 2023-10-21T16:03:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 --2- 172.21.15.53:0/2118430480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb170003970 0x7fb170003d50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 -- 172.21.15.53:0/2118430480 >> 172.21.15.53:0/2118430480 conn(0x7fb18006a5d0 msgr2=0x7fb1800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 -- 172.21.15.53:0/2118430480 shutdown_connections 2023-10-21T16:03:06.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.105+0000 7fb1871ce640 1 -- 172.21.15.53:0/2118430480 wait complete. 2023-10-21T16:03:06.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.108+0000 7fb1871ce640 1 Processor -- start 2023-10-21T16:03:06.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.108+0000 7fb1871ce640 1 -- start start 2023-10-21T16:03:06.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb170003970 0x7fb1800ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1800ac9a0 0x7fb1800b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015a260 con 0x7fb18015bc80 2023-10-21T16:03:06.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015c100 con 0x7fb1800ac9a0 2023-10-21T16:03:06.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015c380 con 0x7fb170003970 2023-10-21T16:03:06.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56188/0 (socket says 172.21.15.53:56188) 2023-10-21T16:03:06.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 -- 172.21.15.53:0/1892005872 learned_addr learned my addr 172.21.15.53:0/1892005872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:06.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb18576b640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1800ac9a0 0x7fb1800b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb184f6a640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb170003970 0x7fb1800ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 -- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb170003970 msgr2=0x7fb1800ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb170003970 0x7fb1800ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 -- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1800ac9a0 msgr2=0x7fb1800b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1800ac9a0 0x7fb1800b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 -- 172.21.15.53:0/1892005872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18015c660 con 0x7fb18015bc80 2023-10-21T16:03:06.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17ffff640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00e480 tx=0x7fb16c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0053a0 con 0x7fb18015bc80 2023-10-21T16:03:06.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c0035f0 con 0x7fb18015bc80 2023-10-21T16:03:06.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0039a0 con 0x7fb18015bc80 2023-10-21T16:03:06.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.109+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb180159ba0 con 0x7fb18015bc80 2023-10-21T16:03:06.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.110+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb180153b90 con 0x7fb18015bc80 2023-10-21T16:03:06.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.111+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18015a3a0 con 0x7fb18015bc80 2023-10-21T16:03:06.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.113+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb16c003b00 con 0x7fb18015bc80 2023-10-21T16:03:06.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.113+0000 7fb17d7fa640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 0x7fb16406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.114+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb16c0897d0 con 0x7fb18015bc80 2023-10-21T16:03:06.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.114+0000 7fb184f6a640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 0x7fb16406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.115+0000 7fb184f6a640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 0x7fb16406d360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb174034000 tx=0x7fb174009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.119+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb18015a3a0 con 0x7fb18015bc80 2023-10-21T16:03:06.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.309+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb18015a3a0 con 0x7fb18015bc80 2023-10-21T16:03:06.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.309+0000 7fb17d7fa640 1 -- 172.21.15.53:0/1892005872 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb18015a3a0 con 0x7fb18015bc80 2023-10-21T16:03:06.312 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:06.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 msgr2=0x7fb16406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 0x7fb16406d360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb174034000 tx=0x7fb174009030 comp rx=0 tx=0).stop 2023-10-21T16:03:06.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 msgr2=0x7fb180153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00e480 tx=0x7fb16c00e940 comp rx=0 tx=0).stop 2023-10-21T16:03:06.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 shutdown_connections 2023-10-21T16:03:06.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1800ac9a0 0x7fb1800b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb16406aea0 0x7fb16406d360 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb18015bc80 0x7fb180153650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 --2- 172.21.15.53:0/1892005872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb170003970 0x7fb1800ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 >> 172.21.15.53:0/1892005872 conn(0x7fb18006a5d0 msgr2=0x7fb1800b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:06.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.312+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 shutdown_connections 2023-10-21T16:03:06.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.313+0000 7fb1871ce640 1 -- 172.21.15.53:0/1892005872 wait complete. 2023-10-21T16:03:06.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.566+0000 7fa381b87640 1 Processor -- start 2023-10-21T16:03:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.566+0000 7fa381b87640 1 -- start start 2023-10-21T16:03:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381b87640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa37c15a820 0x7fa37c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381b87640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15ac00 0x7fa37c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381b87640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c0cab60 con 0x7fa37c0ca680 2023-10-21T16:03:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381b87640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0cb960 con 0x7fa37c15a820 2023-10-21T16:03:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381b87640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0cbdf0 con 0x7fa37c15ac00 2023-10-21T16:03:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa37c15a820 0x7fa37c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa37c15a820 0x7fa37c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56190/0 (socket says 172.21.15.53:56190) 2023-10-21T16:03:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 -- 172.21.15.53:0/1813996621 learned_addr learned my addr 172.21.15.53:0/1813996621 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa381386640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15ac00 0x7fa37c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 -- 172.21.15.53:0/1813996621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15ac00 msgr2=0x7fa37c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15ac00 0x7fa37c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 -- 172.21.15.53:0/1813996621 >> v1:172.21.15.124:6789/0 conn(0x7fa37c0ca680 legacy=0x7fa37c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.568+0000 7fa380b85640 1 -- 172.21.15.53:0/1813996621 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0cb030 con 0x7fa37c15a820 2023-10-21T16:03:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.570+0000 7fa380b85640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15a820 0x7fa37c0aff70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa370009870 tx=0x7fa37002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9da720321abf8f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa37b7fe640 1 -- 172.21.15.53:0/1813996621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370031640 con 0x7fa37c15a820 2023-10-21T16:03:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa37b7fe640 1 -- 172.21.15.53:0/1813996621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3700317a0 con 0x7fa37c15a820 2023-10-21T16:03:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa37b7fe640 1 -- 172.21.15.53:0/1813996621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370031b30 con 0x7fa37c15a820 2023-10-21T16:03:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 -- 172.21.15.53:0/1813996621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15a820 msgr2=0x7fa37c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15a820 0x7fa37c0aff70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa370009870 tx=0x7fa37002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 -- 172.21.15.53:0/1813996621 shutdown_connections 2023-10-21T16:03:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15ac00 0x7fa37c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 --2- 172.21.15.53:0/1813996621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15a820 0x7fa37c0aff70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.571+0000 7fa381b87640 1 -- 172.21.15.53:0/1813996621 >> 172.21.15.53:0/1813996621 conn(0x7fa37c06c050 msgr2=0x7fa37c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.572+0000 7fa381b87640 1 -- 172.21.15.53:0/1813996621 shutdown_connections 2023-10-21T16:03:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.572+0000 7fa381b87640 1 -- 172.21.15.53:0/1813996621 wait complete. 2023-10-21T16:03:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.572+0000 7fa381b87640 1 Processor -- start 2023-10-21T16:03:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.572+0000 7fa381b87640 1 -- start start 2023-10-21T16:03:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 0x7fa37c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15a820 0x7fa37c1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0c84c0 con 0x7fa37c15ac00 2023-10-21T16:03:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0c2020 con 0x7fa37c0ca680 2023-10-21T16:03:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.573+0000 7fa381b87640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c1f0ca0 con 0x7fa37c15a820 2023-10-21T16:03:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56204/0 (socket says 172.21.15.53:56204) 2023-10-21T16:03:06.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 -- 172.21.15.53:0/2012893865 learned_addr learned my addr 172.21.15.53:0/2012893865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:06.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa37bfff640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15a820 0x7fa37c1f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa380b85640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 0x7fa37c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 -- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15a820 msgr2=0x7fa37c1f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15a820 0x7fa37c1f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 -- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 msgr2=0x7fa37c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 0x7fa37c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa381386640 1 -- 172.21.15.53:0/2012893865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700094d0 con 0x7fa37c15ac00 2023-10-21T16:03:06.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.575+0000 7fa380b85640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 0x7fa37c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa381386640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa374007e30 tx=0x7fa37400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374014620 con 0x7fa37c15ac00 2023-10-21T16:03:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37400ee90 con 0x7fa37c15ac00 2023-10-21T16:03:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740149c0 con 0x7fa37c15ac00 2023-10-21T16:03:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c1f0f80 con 0x7fa37c15ac00 2023-10-21T16:03:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.576+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c0c25c0 con 0x7fa37c15ac00 2023-10-21T16:03:06.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.577+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c074150 con 0x7fa37c15ac00 2023-10-21T16:03:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.578+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa374014b20 con 0x7fa37c15ac00 2023-10-21T16:03:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.579+0000 7fa3797fa640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 0x7fa36006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.579+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa37408a340 con 0x7fa37c15ac00 2023-10-21T16:03:06.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.580+0000 7fa380b85640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 0x7fa36006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.580+0000 7fa380b85640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 0x7fa36006d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa370036000 tx=0x7fa37003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:06.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.583+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa374055020 con 0x7fa37c15ac00 2023-10-21T16:03:06.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.776+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa37c0c2780 con 0x7fa37c15ac00 2023-10-21T16:03:06.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.776+0000 7fa3797fa640 1 -- 172.21.15.53:0/2012893865 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa374058640 con 0x7fa37c15ac00 2023-10-21T16:03:06.778 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.779+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 msgr2=0x7fa36006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.779+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 0x7fa36006d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa370036000 tx=0x7fa37003c040 comp rx=0 tx=0).stop 2023-10-21T16:03:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.779+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 msgr2=0x7fa37c1f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.779+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa374007e30 tx=0x7fa37400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 shutdown_connections 2023-10-21T16:03:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa36006aea0 0x7fa36006d360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa37c15ac00 0x7fa37c1f00e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa37c15a820 0x7fa37c1f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 --2- 172.21.15.53:0/2012893865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa37c0ca680 0x7fa37c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 >> 172.21.15.53:0/2012893865 conn(0x7fa37c06c050 msgr2=0x7fa37c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:06.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 shutdown_connections 2023-10-21T16:03:06.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:06.780+0000 7fa381b87640 1 -- 172.21.15.53:0/2012893865 wait complete. 2023-10-21T16:03:06.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.037+0000 7f2b27b9e640 1 Processor -- start 2023-10-21T16:03:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.038+0000 7f2b27b9e640 1 -- start start 2023-10-21T16:03:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.039+0000 7f2b27b9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b20160660 0x7f2b20070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.039+0000 7f2b27b9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b200cb620 0x7f2b200b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.039+0000 7f2b27b9e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2006e150 con 0x7f2b200c9910 2023-10-21T16:03:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.039+0000 7f2b27b9e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2006de70 con 0x7f2b200cb620 2023-10-21T16:03:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.039+0000 7f2b27b9e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c8e30 con 0x7f2b20160660 2023-10-21T16:03:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b200cb620 0x7f2b200b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b200cb620 0x7f2b200b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56206/0 (socket says 172.21.15.53:56206) 2023-10-21T16:03:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b25139640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2b200c9910 0x7f2b200caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49318/0 (socket says 172.21.15.53:49318) 2023-10-21T16:03:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 -- 172.21.15.53:0/415265416 learned_addr learned my addr 172.21.15.53:0/415265416 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:07.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2593a640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b20160660 0x7f2b20070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 -- 172.21.15.53:0/415265416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b20160660 msgr2=0x7f2b20070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b20160660 0x7f2b20070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 -- 172.21.15.53:0/415265416 >> v1:172.21.15.53:6789/0 conn(0x7f2b200c9910 legacy=0x7f2b200caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b2613b640 1 -- 172.21.15.53:0/415265416 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200c9110 con 0x7f2b200cb620 2023-10-21T16:03:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.040+0000 7f2b24938640 1 -- 172.21.15.53:0/415265416 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649592114 0 0) 0x7f2b2006e150 con 0x7f2b200c9910 2023-10-21T16:03:07.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.041+0000 7f2b2613b640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200cb620 0x7f2b200b1990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009e50 tx=0x7f2b1802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb7b6112d46fe57 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b24938640 1 -- 172.21.15.53:0/415265416 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1802caf0 con 0x7f2b200cb620 2023-10-21T16:03:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b24938640 1 -- 172.21.15.53:0/415265416 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1802cc50 con 0x7f2b200cb620 2023-10-21T16:03:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b24938640 1 -- 172.21.15.53:0/415265416 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180343f0 con 0x7f2b200cb620 2023-10-21T16:03:07.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 -- 172.21.15.53:0/415265416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200cb620 msgr2=0x7f2b200b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200cb620 0x7f2b200b1990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009e50 tx=0x7f2b1802bf50 comp rx=0 tx=0).stop 2023-10-21T16:03:07.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 -- 172.21.15.53:0/415265416 shutdown_connections 2023-10-21T16:03:07.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b200cb620 0x7f2b200b1990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/415265416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b20160660 0x7f2b20070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 -- 172.21.15.53:0/415265416 >> 172.21.15.53:0/415265416 conn(0x7f2b2006a5d0 msgr2=0x7f2b200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:07.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.042+0000 7f2b27b9e640 1 -- 172.21.15.53:0/415265416 shutdown_connections 2023-10-21T16:03:07.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.043+0000 7f2b27b9e640 1 -- 172.21.15.53:0/415265416 wait complete. 2023-10-21T16:03:07.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.044+0000 7f2b27b9e640 1 Processor -- start 2023-10-21T16:03:07.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 -- start start 2023-10-21T16:03:07.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c9910 0x7f2b200cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 0x7f2b201592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200ccb80 con 0x7f2b20160660 2023-10-21T16:03:07.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200cce20 con 0x7f2b200cb620 2023-10-21T16:03:07.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.045+0000 7f2b27b9e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200cd100 con 0x7f2b200c9910 2023-10-21T16:03:07.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56220/0 (socket says 172.21.15.53:56220) 2023-10-21T16:03:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 -- 172.21.15.53:0/1401633823 learned_addr learned my addr 172.21.15.53:0/1401633823 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2593a640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c9910 0x7f2b200cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 -- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c9910 msgr2=0x7f2b200cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c9910 0x7f2b200cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 -- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 msgr2=0x7f2b201592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b25139640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 0x7f2b201592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 0x7f2b201592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b2613b640 1 -- 172.21.15.53:0/1401633823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009b30 con 0x7f2b20160660 2023-10-21T16:03:07.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.046+0000 7f2b25139640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 0x7f2b201592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b2613b640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009e50 tx=0x7f2b18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18009510 con 0x7f2b20160660 2023-10-21T16:03:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1803e450 con 0x7f2b20160660 2023-10-21T16:03:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b200cd3e0 con 0x7f2b20160660 2023-10-21T16:03:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1803e800 con 0x7f2b20160660 2023-10-21T16:03:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.047+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b200cda20 con 0x7f2b20160660 2023-10-21T16:03:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.048+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b200cdc10 con 0x7f2b20160660 2023-10-21T16:03:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.049+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b1803e960 con 0x7f2b20160660 2023-10-21T16:03:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.050+0000 7f2b167fc640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 0x7f2b0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.051+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b180ac760 con 0x7f2b20160660 2023-10-21T16:03:07.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.051+0000 7f2b2593a640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 0x7f2b0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.052+0000 7f2b2593a640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 0x7f2b0406d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c006fd0 tx=0x7f2b1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.055+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b18077440 con 0x7f2b20160660 2023-10-21T16:03:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.253+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2b2004e160 con 0x7f2b20160660 2023-10-21T16:03:07.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.254+0000 7f2b167fc640 1 -- 172.21.15.53:0/1401633823 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2b1807aa60 con 0x7f2b20160660 2023-10-21T16:03:07.255 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 msgr2=0x7f2b0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 0x7f2b0406d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c006fd0 tx=0x7f2b1c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 msgr2=0x7f2b200c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009e50 tx=0x7f2b18004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 shutdown_connections 2023-10-21T16:03:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2b0406ae70 0x7f2b0406d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b20160660 0x7f2b200c4420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b200cb620 0x7f2b201592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 --2- 172.21.15.53:0/1401633823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b200c9910 0x7f2b200cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.256+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 >> 172.21.15.53:0/1401633823 conn(0x7f2b2006a5d0 msgr2=0x7f2b2015ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.257+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 shutdown_connections 2023-10-21T16:03:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.257+0000 7f2b27b9e640 1 -- 172.21.15.53:0/1401633823 wait complete. 2023-10-21T16:03:07.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:03:07.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.504+0000 7f2a7bfff640 1 Processor -- start 2023-10-21T16:03:07.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.504+0000 7f2a7bfff640 1 -- start start 2023-10-21T16:03:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a7c15a820 0x7f2a7c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2a7c15ac00 0x7f2a7c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c0cab60 con 0x7f2a7c0ca680 2023-10-21T16:03:07.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7bfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c0cb960 con 0x7f2a7c15a820 2023-10-21T16:03:07.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c0cbdf0 con 0x7f2a7c15ac00 2023-10-21T16:03:07.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a7c15a820 0x7f2a7c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a7c15a820 0x7f2a7c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56222/0 (socket says 172.21.15.53:56222) 2023-10-21T16:03:07.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.506+0000 7f2a7affd640 1 -- 172.21.15.53:0/2070368569 learned_addr learned my addr 172.21.15.53:0/2070368569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.507+0000 7f2a7b7fe640 1 --2- 172.21.15.53:0/2070368569 >> v2:172.21.15.124:3300/0 conn(0x7f2a7c15ac00 0x7f2a7c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.507+0000 7f2a7affd640 1 -- 172.21.15.53:0/2070368569 >> v2:172.21.15.124:3300/0 conn(0x7f2a7c15ac00 msgr2=0x7f2a7c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.507+0000 7f2a7affd640 1 --2- 172.21.15.53:0/2070368569 >> v2:172.21.15.124:3300/0 conn(0x7f2a7c15ac00 0x7f2a7c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.507+0000 7f2a7affd640 1 -- 172.21.15.53:0/2070368569 >> v1:172.21.15.124:6789/0 conn(0x7f2a7c0ca680 legacy=0x7f2a7c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:07.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.507+0000 7f2a7affd640 1 -- 172.21.15.53:0/2070368569 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0cb030 con 0x7f2a7c15a820 2023-10-21T16:03:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.509+0000 7f2a7affd640 1 --2- 172.21.15.53:0/2070368569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c15a820 0x7f2a7c0aff70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0b0e00 tx=0x7f2a6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b405f1051425078 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.509+0000 7f2a79ffb640 1 -- 172.21.15.53:0/2070368569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031870 con 0x7f2a7c15a820 2023-10-21T16:03:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.509+0000 7f2a79ffb640 1 -- 172.21.15.53:0/2070368569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0319d0 con 0x7f2a7c15a820 2023-10-21T16:03:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.509+0000 7f2a79ffb640 1 -- 172.21.15.53:0/2070368569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c031d60 con 0x7f2a7c15a820 2023-10-21T16:03:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 -- 172.21.15.53:0/2070368569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c15a820 msgr2=0x7f2a7c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/2070368569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c15a820 0x7f2a7c0aff70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0b0e00 tx=0x7f2a6c02b910 comp rx=0 tx=0).stop 2023-10-21T16:03:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 -- 172.21.15.53:0/2070368569 shutdown_connections 2023-10-21T16:03:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/2070368569 >> v2:172.21.15.124:3300/0 conn(0x7f2a7c15ac00 0x7f2a7c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/2070368569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c15a820 0x7f2a7c0aff70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 -- 172.21.15.53:0/2070368569 >> 172.21.15.53:0/2070368569 conn(0x7f2a7c06c050 msgr2=0x7f2a7c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 -- 172.21.15.53:0/2070368569 shutdown_connections 2023-10-21T16:03:07.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.510+0000 7f2a7bfff640 1 -- 172.21.15.53:0/2070368569 wait complete. 2023-10-21T16:03:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.512+0000 7f2a7bfff640 1 Processor -- start 2023-10-21T16:03:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.512+0000 7f2a7bfff640 1 -- start start 2023-10-21T16:03:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a7c15ac00 0x7f2a7c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a7c154050 0x7f2a7c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0af590 con 0x7f2a7c0ca680 2023-10-21T16:03:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0c7c40 con 0x7f2a7c154050 2023-10-21T16:03:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.513+0000 7f2a7bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0c67b0 con 0x7f2a7c15ac00 2023-10-21T16:03:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56232/0 (socket says 172.21.15.53:56232) 2023-10-21T16:03:07.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 -- 172.21.15.53:0/756939948 learned_addr learned my addr 172.21.15.53:0/756939948 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:07.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 -- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a7c15ac00 msgr2=0x7f2a7c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a7c15ac00 0x7f2a7c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 -- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a7c154050 msgr2=0x7f2a7c0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a7c154050 0x7f2a7c0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 -- 172.21.15.53:0/756939948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0094d0 con 0x7f2a7c0ca680 2023-10-21T16:03:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.514+0000 7f2a7affd640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c02bea0 tx=0x7f2a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.515+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c03ad30 con 0x7f2a7c0ca680 2023-10-21T16:03:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.515+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c0443f0 con 0x7f2a7c0ca680 2023-10-21T16:03:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.515+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0447a0 con 0x7f2a7c0ca680 2023-10-21T16:03:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.515+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c0c7ee0 con 0x7f2a7c0ca680 2023-10-21T16:03:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.515+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c0c84a0 con 0x7f2a7c0ca680 2023-10-21T16:03:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.516+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c0c8660 con 0x7f2a7c0ca680 2023-10-21T16:03:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.517+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a6c04d450 con 0x7f2a7c0ca680 2023-10-21T16:03:07.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.518+0000 7f2a63fff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 0x7f2a6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:07.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.518+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2a6c02d040 con 0x7f2a7c0ca680 2023-10-21T16:03:07.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.519+0000 7f2a7a7fc640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 0x7f2a6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:07.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.520+0000 7f2a7a7fc640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 0x7f2a6806d360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000f60 tx=0x7f2a700079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:07.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.522+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a6c076c70 con 0x7f2a7c0ca680 2023-10-21T16:03:07.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.729+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a7c074150 con 0x7f2a7c0ca680 2023-10-21T16:03:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.731+0000 7f2a63fff640 1 -- 172.21.15.53:0/756939948 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2a6c035020 con 0x7f2a7c0ca680 2023-10-21T16:03:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:03:07.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 msgr2=0x7f2a6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 0x7f2a6806d360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a70000f60 tx=0x7f2a700079b0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 msgr2=0x7f2a7c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:07.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c02bea0 tx=0x7f2a6c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 shutdown_connections 2023-10-21T16:03:07.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a7c154050 0x7f2a7c0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a7c15ac00 0x7f2a7c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f2a6806aea0 0x7f2a6806d360 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 --2- 172.21.15.53:0/756939948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a7c0ca680 0x7f2a7c0c5980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:07.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 >> 172.21.15.53:0/756939948 conn(0x7f2a7c06c050 msgr2=0x7f2a7c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:07.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.733+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 shutdown_connections 2023-10-21T16:03:07.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:07.734+0000 7f2a7bfff640 1 -- 172.21.15.53:0/756939948 wait complete. 2023-10-21T16:03:07.755 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:03:07.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:07.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-21T16:03:07.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:03:07.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:07.859 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:03:07.909 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:07.910 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:07.974 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:07.974 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:08.027 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:08.028 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:08.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:08.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.312+0000 7f3bc63dc640 1 Processor -- start 2023-10-21T16:03:08.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.313+0000 7f3bc63dc640 1 -- start start 2023-10-21T16:03:08.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.314+0000 7f3bc63dc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3bc00ad790 0x7f3bc00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.314+0000 7f3bc63dc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc006dee0 con 0x7f3bc015e330 2023-10-21T16:03:08.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.314+0000 7f3bc63dc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc015d850 con 0x7f3bc0159ac0 2023-10-21T16:03:08.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bc63dc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc006dcf0 con 0x7f3bc00ad790 2023-10-21T16:03:08.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bc4979640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3bc015e330 0x7f3bc00c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49320/0 (socket says 172.21.15.53:49320) 2023-10-21T16:03:08.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bc4979640 1 -- 172.21.15.53:0/2626315116 learned_addr learned my addr 172.21.15.53:0/2626315116 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:08.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/2626315116 >> v2:172.21.15.124:3300/0 conn(0x7f3bc00ad790 0x7f3bc00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459430463 0 0) 0x7f3bc006dee0 con 0x7f3bc015e330 2023-10-21T16:03:08.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.315+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba8003520 con 0x7f3bc015e330 2023-10-21T16:03:08.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572667572 0 0) 0x7f3bc015d850 con 0x7f3bc0159ac0 2023-10-21T16:03:08.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc006dee0 con 0x7f3bc0159ac0 2023-10-21T16:03:08.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052968909 0 0) 0x7f3ba8003520 con 0x7f3bc015e330 2023-10-21T16:03:08.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 >> v2:172.21.15.124:3300/0 conn(0x7f3bc00ad790 msgr2=0x7f3bc00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:08.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 --2- 172.21.15.53:0/2626315116 >> v2:172.21.15.124:3300/0 conn(0x7f3bc00ad790 0x7f3bc00702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 >> v1:172.21.15.124:6789/0 conn(0x7f3bc0159ac0 legacy=0x7f3bc006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc015db30 con 0x7f3bc015e330 2023-10-21T16:03:08.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3bb8001670 con 0x7f3bc015e330 2023-10-21T16:03:08.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 >> v1:172.21.15.53:6789/0 conn(0x7f3bc015e330 legacy=0x7f3bc00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ba8003b50 0x7f3ba8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 0x7f3ba8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba8005180 0x7f3ba800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba800c050 con 0x7f3ba80047c0 2023-10-21T16:03:08.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba800c190 con 0x7f3ba8005180 2023-10-21T16:03:08.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba800c350 con 0x7f3ba8003b50 2023-10-21T16:03:08.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.316+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 0x7f3ba8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbffff640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ba8003b50 0x7f3ba8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ba8003b50 msgr2=0x7f3ba8003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:08.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ba8003b50 0x7f3ba8003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba8005180 msgr2=0x7f3ba800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:08.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bc4979640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba8005180 0x7f3ba800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba8005180 0x7f3ba800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/2626315116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8003520 con 0x7f3ba80047c0 2023-10-21T16:03:08.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bc4979640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba8005180 0x7f3ba800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:08.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.317+0000 7f3bc4979640 1 -- 172.21.15.53:0/2626315116 reap_dead start 2023-10-21T16:03:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.318+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 0x7f3ba8004c40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009ac0 tx=0x7f3ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d24334bd2e00ee server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.318+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb402c6b0 con 0x7f3ba80047c0 2023-10-21T16:03:08.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.318+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb402c810 con 0x7f3ba80047c0 2023-10-21T16:03:08.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.318+0000 7f3bbeffd640 1 -- 172.21.15.53:0/2626315116 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb402cba0 con 0x7f3ba80047c0 2023-10-21T16:03:08.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 -- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 msgr2=0x7f3ba8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 0x7f3ba8004c40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009ac0 tx=0x7f3ba80057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 -- 172.21.15.53:0/2626315116 shutdown_connections 2023-10-21T16:03:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/2626315116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80047c0 0x7f3ba8004c40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 -- 172.21.15.53:0/2626315116 >> 172.21.15.53:0/2626315116 conn(0x7f3bc006a380 msgr2=0x7f3bc00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 -- 172.21.15.53:0/2626315116 shutdown_connections 2023-10-21T16:03:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.319+0000 7f3bc63dc640 1 -- 172.21.15.53:0/2626315116 wait complete. 2023-10-21T16:03:08.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.321+0000 7f3bc63dc640 1 Processor -- start 2023-10-21T16:03:08.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.321+0000 7f3bc63dc640 1 -- start start 2023-10-21T16:03:08.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba80047c0 0x7f3bc015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bc015a560 0x7f3bc0158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015b6f0 con 0x7f3bc01512b0 2023-10-21T16:03:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015aae0 con 0x7f3ba80047c0 2023-10-21T16:03:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bc63dc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc015adc0 con 0x7f3bc015a560 2023-10-21T16:03:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bbf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56242/0 (socket says 172.21.15.53:56242) 2023-10-21T16:03:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.322+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/3712005648 learned_addr learned my addr 172.21.15.53:0/3712005648 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbffff640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba80047c0 0x7f3bc015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bc015a560 msgr2=0x7f3bc0158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bc015a560 0x7f3bc0158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba80047c0 msgr2=0x7f3bc015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba80047c0 0x7f3bc015bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 -- 172.21.15.53:0/3712005648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc015b0a0 con 0x7f3bc01512b0 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.323+0000 7f3bbf7fe640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3bb402c250 tx=0x7f3bb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.324+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003810 con 0x7f3bc01512b0 2023-10-21T16:03:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.324+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb4003970 con 0x7f3bc01512b0 2023-10-21T16:03:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.324+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003d00 con 0x7f3bc01512b0 2023-10-21T16:03:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.324+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc015b380 con 0x7f3bc01512b0 2023-10-21T16:03:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.324+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc01530f0 con 0x7f3bc01512b0 2023-10-21T16:03:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.325+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc015b8a0 con 0x7f3bc01512b0 2023-10-21T16:03:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.326+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3bb4047450 con 0x7f3bc01512b0 2023-10-21T16:03:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.327+0000 7f3bbcff9640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 0x7f3bac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.327+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3bb40abcd0 con 0x7f3bc01512b0 2023-10-21T16:03:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.327+0000 7f3bbffff640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 0x7f3bac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.329+0000 7f3bbffff640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 0x7f3bac06d360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0003fd0 tx=0x7f3bb0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.331+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3bc015b8a0 con 0x7f3bc01512b0 2023-10-21T16:03:08.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.524+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc0067490 con 0x7f3bc01512b0 2023-10-21T16:03:08.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.528+0000 7f3bbcff9640 1 -- 172.21.15.53:0/3712005648 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f3bb404c090 con 0x7f3bc01512b0 2023-10-21T16:03:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:08.530 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:08.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 msgr2=0x7f3bac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 0x7f3bac06d360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0003fd0 tx=0x7f3bb0009040 comp rx=0 tx=0).stop 2023-10-21T16:03:08.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 msgr2=0x7f3bc015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3bb402c250 tx=0x7f3bb4004730 comp rx=0 tx=0).stop 2023-10-21T16:03:08.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 shutdown_connections 2023-10-21T16:03:08.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bc015a560 0x7f3bc0158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3bac06aea0 0x7f3bac06d360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc01512b0 0x7f3bc015a020 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 --2- 172.21.15.53:0/3712005648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ba80047c0 0x7f3bc015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 >> 172.21.15.53:0/3712005648 conn(0x7f3bc006a380 msgr2=0x7f3bc01549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:08.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.530+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 shutdown_connections 2023-10-21T16:03:08.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.531+0000 7f3bc63dc640 1 -- 172.21.15.53:0/3712005648 wait complete. 2023-10-21T16:03:08.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:08.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:08.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.783+0000 7f17b31cb640 1 Processor -- start 2023-10-21T16:03:08.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.783+0000 7f17b31cb640 1 -- start start 2023-10-21T16:03:08.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.784+0000 7f17b31cb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 0x7f17ac0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.784+0000 7f17b31cb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17ac15dcd0 0x7f17ac1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.784+0000 7f17b31cb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac0d0a80 con 0x7f17ac0d1120 2023-10-21T16:03:08.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b31cb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac161620 con 0x7f17ac15dcd0 2023-10-21T16:03:08.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b31cb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0b1fe0 con 0x7f17ac0d1500 2023-10-21T16:03:08.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b21c9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f17ac0d1120 0x7f17ac0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49330/0 (socket says 172.21.15.53:49330) 2023-10-21T16:03:08.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b21c9640 1 -- 172.21.15.53:0/3204330135 learned_addr learned my addr 172.21.15.53:0/3204330135 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:08.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b19c8640 1 --2- 172.21.15.53:0/3204330135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 0x7f17ac0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b29ca640 1 --2- 172.21.15.53:0/3204330135 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15dcd0 0x7f17ac1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858194234 0 0) 0x7f17ac0d0a80 con 0x7f17ac0d1120 2023-10-21T16:03:08.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.785+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1798003520 con 0x7f17ac0d1120 2023-10-21T16:03:08.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.786+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 931568382 0 0) 0x7f1798003520 con 0x7f17ac0d1120 2023-10-21T16:03:08.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.786+0000 7f17b19c8640 1 -- 172.21.15.53:0/3204330135 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15dcd0 msgr2=0x7f17ac1610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:08.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.786+0000 7f17b19c8640 1 --2- 172.21.15.53:0/3204330135 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15dcd0 0x7f17ac1610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.786+0000 7f17b19c8640 1 -- 172.21.15.53:0/3204330135 >> v1:172.21.15.53:6789/0 conn(0x7f17ac0d1120 legacy=0x7f17ac0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.786+0000 7f17b19c8640 1 -- 172.21.15.53:0/3204330135 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0b4580 con 0x7f17ac0d1500 2023-10-21T16:03:08.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.787+0000 7f17b19c8640 1 --2- 172.21.15.53:0/3204330135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 0x7f17ac0d04a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17a00097a0 tx=0x7f17a002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19090ee1df3c4ea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.788+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a80017f0 con 0x7f17ac0d1500 2023-10-21T16:03:08.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.788+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a002e050 con 0x7f17ac0d1500 2023-10-21T16:03:08.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.788+0000 7f17b11c7640 1 -- 172.21.15.53:0/3204330135 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0030730 con 0x7f17ac0d1500 2023-10-21T16:03:08.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 -- 172.21.15.53:0/3204330135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 msgr2=0x7f17ac0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 --2- 172.21.15.53:0/3204330135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 0x7f17ac0d04a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17a00097a0 tx=0x7f17a002b710 comp rx=0 tx=0).stop 2023-10-21T16:03:08.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 -- 172.21.15.53:0/3204330135 shutdown_connections 2023-10-21T16:03:08.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 --2- 172.21.15.53:0/3204330135 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15dcd0 0x7f17ac1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 --2- 172.21.15.53:0/3204330135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1500 0x7f17ac0d04a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 -- 172.21.15.53:0/3204330135 >> 172.21.15.53:0/3204330135 conn(0x7f17ac0d2960 msgr2=0x7f17ac0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:08.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 -- 172.21.15.53:0/3204330135 shutdown_connections 2023-10-21T16:03:08.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.789+0000 7f17b31cb640 1 -- 172.21.15.53:0/3204330135 wait complete. 2023-10-21T16:03:08.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.791+0000 7f17b31cb640 1 Processor -- start 2023-10-21T16:03:08.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.791+0000 7f17b31cb640 1 -- start start 2023-10-21T16:03:08.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 0x7f17ac0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 0x7f17ac15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0c9e60 con 0x7f17ac15dcd0 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0c7460 con 0x7f17ac0d1500 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b31cb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0c68e0 con 0x7f17ac0d1120 2023-10-21T16:03:08.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b29ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b29ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56258/0 (socket says 172.21.15.53:56258) 2023-10-21T16:03:08.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b29ca640 1 -- 172.21.15.53:0/2838483609 learned_addr learned my addr 172.21.15.53:0/2838483609 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:08.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b21c9640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 0x7f17ac0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.792+0000 7f17b19c8640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 0x7f17ac15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 -- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 msgr2=0x7f17ac0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 0x7f17ac0c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 -- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 msgr2=0x7f17ac15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:08.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 0x7f17ac15ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:08.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 -- 172.21.15.53:0/2838483609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009400 con 0x7f17ac15dcd0 2023-10-21T16:03:08.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b19c8640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 0x7f17ac15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:08.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b21c9640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 0x7f17ac0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:08.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b29ca640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17a400efc0 tx=0x7f17a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4009ba0 con 0x7f17ac15dcd0 2023-10-21T16:03:08.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a400f040 con 0x7f17ac15dcd0 2023-10-21T16:03:08.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a40146c0 con 0x7f17ac15dcd0 2023-10-21T16:03:08.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.793+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac0c6bc0 con 0x7f17ac15dcd0 2023-10-21T16:03:08.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.794+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac0c7180 con 0x7f17ac15dcd0 2023-10-21T16:03:08.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.795+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac1f37e0 con 0x7f17ac15dcd0 2023-10-21T16:03:08.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.796+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17a4014820 con 0x7f17ac15dcd0 2023-10-21T16:03:08.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.797+0000 7f179effd640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 0x7f179006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:08.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.797+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f17a408a760 con 0x7f17ac15dcd0 2023-10-21T16:03:08.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.797+0000 7f17b21c9640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 0x7f179006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:08.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.798+0000 7f17b21c9640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 0x7f179006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17a80010e0 tx=0x7f17a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:08.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:08.801+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17ac1f37e0 con 0x7f17ac15dcd0 2023-10-21T16:03:09.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.004+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17ac1f37e0 con 0x7f17ac15dcd0 2023-10-21T16:03:09.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.005+0000 7f179effd640 1 -- 172.21.15.53:0/2838483609 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17a4018020 con 0x7f17ac15dcd0 2023-10-21T16:03:09.007 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 msgr2=0x7f179006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 0x7f179006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17a80010e0 tx=0x7f17a8009040 comp rx=0 tx=0).stop 2023-10-21T16:03:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 msgr2=0x7f17ac15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17a400efc0 tx=0x7f17a400c490 comp rx=0 tx=0).stop 2023-10-21T16:03:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 shutdown_connections 2023-10-21T16:03:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f179006ae70 0x7f179006d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac15dcd0 0x7f17ac15b280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac0d1500 0x7f17ac15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 --2- 172.21.15.53:0/2838483609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0d1120 0x7f17ac0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 >> 172.21.15.53:0/2838483609 conn(0x7f17ac0d2960 msgr2=0x7f17ac15ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:09.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 shutdown_connections 2023-10-21T16:03:09.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.008+0000 7f17b31cb640 1 -- 172.21.15.53:0/2838483609 wait complete. 2023-10-21T16:03:09.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.260+0000 7f96c10ad640 1 Processor -- start 2023-10-21T16:03:09.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.260+0000 7f96c10ad640 1 -- start start 2023-10-21T16:03:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96c10ad640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96c10ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96c10ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc0ce9b0 con 0x7f96bc1612e0 2023-10-21T16:03:09.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96c10ad640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0cec90 con 0x7f96bc15ccd0 2023-10-21T16:03:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96c10ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0cef70 con 0x7f96bc06bcb0 2023-10-21T16:03:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96bbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96bbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56268/0 (socket says 172.21.15.53:56268) 2023-10-21T16:03:09.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.262+0000 7f96bbfff640 1 -- 172.21.15.53:0/2773833784 learned_addr learned my addr 172.21.15.53:0/2773833784 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.263+0000 7f96bbfff640 1 -- 172.21.15.53:0/2773833784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 msgr2=0x7f96bc160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.263+0000 7f96bbfff640 1 --2- 172.21.15.53:0/2773833784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.263+0000 7f96bbfff640 1 -- 172.21.15.53:0/2773833784 >> v1:172.21.15.124:6789/0 conn(0x7f96bc1612e0 legacy=0x7f96bc0b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:09.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.263+0000 7f96bbfff640 1 -- 172.21.15.53:0/2773833784 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc160880 con 0x7f96bc15ccd0 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.264+0000 7f96bbfff640 1 --2- 172.21.15.53:0/2773833784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009870 tx=0x7f96ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31178542b9fe4260 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.264+0000 7f96baffd640 1 -- 172.21.15.53:0/2773833784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac031640 con 0x7f96bc15ccd0 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.264+0000 7f96baffd640 1 -- 172.21.15.53:0/2773833784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac0317a0 con 0x7f96bc15ccd0 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.264+0000 7f96baffd640 1 -- 172.21.15.53:0/2773833784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac031b30 con 0x7f96bc15ccd0 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 -- 172.21.15.53:0/2773833784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc15ccd0 msgr2=0x7f96bc0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 --2- 172.21.15.53:0/2773833784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009870 tx=0x7f96ac02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 -- 172.21.15.53:0/2773833784 shutdown_connections 2023-10-21T16:03:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 --2- 172.21.15.53:0/2773833784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 --2- 172.21.15.53:0/2773833784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc15ccd0 0x7f96bc0ce2d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 -- 172.21.15.53:0/2773833784 >> 172.21.15.53:0/2773833784 conn(0x7f96bc0cd180 msgr2=0x7f96bc0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:09.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 -- 172.21.15.53:0/2773833784 shutdown_connections 2023-10-21T16:03:09.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.265+0000 7f96c10ad640 1 -- 172.21.15.53:0/2773833784 wait complete. 2023-10-21T16:03:09.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.267+0000 7f96c10ad640 1 Processor -- start 2023-10-21T16:03:09.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.268+0000 7f96c10ad640 1 -- start start 2023-10-21T16:03:09.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc15ccd0 0x7f96bc1f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1f5eb0 con 0x7f96bc1612e0 2023-10-21T16:03:09.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1521f0 con 0x7f96bc15ccd0 2023-10-21T16:03:09.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c10ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc15de50 con 0x7f96bc06bcb0 2023-10-21T16:03:09.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c08ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c08ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56274/0 (socket says 172.21.15.53:56274) 2023-10-21T16:03:09.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.269+0000 7f96c08ac640 1 -- 172.21.15.53:0/223971379 learned_addr learned my addr 172.21.15.53:0/223971379 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96bbfff640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96bb7fe640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc15ccd0 0x7f96bc1f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 -- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 msgr2=0x7f96bc151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 -- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc15ccd0 msgr2=0x7f96bc1f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc15ccd0 0x7f96bc1f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 -- 172.21.15.53:0/223971379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0094d0 con 0x7f96bc1612e0 2023-10-21T16:03:09.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96c08ac640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b400b490 tx=0x7f96b400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011830 con 0x7f96bc1612e0 2023-10-21T16:03:09.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.270+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b40119b0 con 0x7f96bc1612e0 2023-10-21T16:03:09.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.271+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4011d60 con 0x7f96bc1612e0 2023-10-21T16:03:09.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.271+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc15d5b0 con 0x7f96bc1612e0 2023-10-21T16:03:09.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.271+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc15db70 con 0x7f96bc1612e0 2023-10-21T16:03:09.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.272+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc1f6020 con 0x7f96bc1612e0 2023-10-21T16:03:09.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.273+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96b401a400 con 0x7f96bc1612e0 2023-10-21T16:03:09.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.274+0000 7f96b8ff9640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 0x7f969c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.274+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f96b4089b80 con 0x7f96bc1612e0 2023-10-21T16:03:09.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.275+0000 7f96bbfff640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 0x7f969c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.276+0000 7f96bbfff640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 0x7f969c06d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96ac036000 tx=0x7f96ac03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.280+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96b40547d0 con 0x7f96bc1612e0 2023-10-21T16:03:09.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.474+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96bc1f5ca0 con 0x7f96bc1612e0 2023-10-21T16:03:09.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.475+0000 7f96b8ff9640 1 -- 172.21.15.53:0/223971379 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96b4089e60 con 0x7f96bc1612e0 2023-10-21T16:03:09.476 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 msgr2=0x7f969c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 0x7f969c06d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96ac036000 tx=0x7f96ac03c040 comp rx=0 tx=0).stop 2023-10-21T16:03:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 msgr2=0x7f96bc0c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b400b490 tx=0x7f96b400b950 comp rx=0 tx=0).stop 2023-10-21T16:03:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 shutdown_connections 2023-10-21T16:03:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f969c06ae70 0x7f969c06d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96bc1612e0 0x7f96bc0c7fc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc15ccd0 0x7f96bc1f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 --2- 172.21.15.53:0/223971379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc06bcb0 0x7f96bc151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 >> 172.21.15.53:0/223971379 conn(0x7f96bc0cd180 msgr2=0x7f96bc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.477+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 shutdown_connections 2023-10-21T16:03:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.478+0000 7f96c10ad640 1 -- 172.21.15.53:0/223971379 wait complete. 2023-10-21T16:03:09.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:09.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.721+0000 7f4f7672e640 1 Processor -- start 2023-10-21T16:03:09.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.721+0000 7f4f7672e640 1 -- start start 2023-10-21T16:03:09.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.722+0000 7f4f7672e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f701599d0 0x7f4f700cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.722+0000 7f4f7672e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f700cb5e0 0x7f4f700cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.722+0000 7f4f7672e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f70159db0 con 0x7f4f700b3c50 2023-10-21T16:03:09.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.722+0000 7f4f7672e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f70071fe0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.722+0000 7f4f7672e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f700b44c0 con 0x7f4f701599d0 2023-10-21T16:03:09.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f75f2d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f700cb5e0 0x7f4f700cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f74f2b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4f700b3c50 0x7f4f700caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49336/0 (socket says 172.21.15.53:49336) 2023-10-21T16:03:09.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f74f2b640 1 -- 172.21.15.53:0/3528373738 learned_addr learned my addr 172.21.15.53:0/3528373738 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:09.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f7572c640 1 --2- 172.21.15.53:0/3528373738 >> v2:172.21.15.124:3300/0 conn(0x7f4f701599d0 0x7f4f700cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f75f2d640 1 -- 172.21.15.53:0/3528373738 >> v2:172.21.15.124:3300/0 conn(0x7f4f701599d0 msgr2=0x7f4f700cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:09.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f75f2d640 1 --2- 172.21.15.53:0/3528373738 >> v2:172.21.15.124:3300/0 conn(0x7f4f701599d0 0x7f4f700cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.723+0000 7f4f75f2d640 1 -- 172.21.15.53:0/3528373738 >> v1:172.21.15.53:6789/0 conn(0x7f4f700b3c50 legacy=0x7f4f700caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.724+0000 7f4f75f2d640 1 -- 172.21.15.53:0/3528373738 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700713d0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.724+0000 7f4f67fff640 1 -- 172.21.15.53:0/3528373738 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728222018 0 0) 0x7f4f60001110 con 0x7f4f700b3c50 2023-10-21T16:03:09.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f75f2d640 1 --2- 172.21.15.53:0/3528373738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700cc0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f7006e150 tx=0x7f4f6802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b319fdfa4075a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f67fff640 1 -- 172.21.15.53:0/3528373738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6802cc70 con 0x7f4f700cb5e0 2023-10-21T16:03:09.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f67fff640 1 -- 172.21.15.53:0/3528373738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6802cdd0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f67fff640 1 -- 172.21.15.53:0/3528373738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68034660 con 0x7f4f700cb5e0 2023-10-21T16:03:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f7672e640 1 -- 172.21.15.53:0/3528373738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 msgr2=0x7f4f700cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.725+0000 7f4f7672e640 1 --2- 172.21.15.53:0/3528373738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700cc0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f7006e150 tx=0x7f4f6802bec0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 -- 172.21.15.53:0/3528373738 shutdown_connections 2023-10-21T16:03:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 --2- 172.21.15.53:0/3528373738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700cc0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 --2- 172.21.15.53:0/3528373738 >> v2:172.21.15.124:3300/0 conn(0x7f4f701599d0 0x7f4f700cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 -- 172.21.15.53:0/3528373738 >> 172.21.15.53:0/3528373738 conn(0x7f4f7006c050 msgr2=0x7f4f7006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:09.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 -- 172.21.15.53:0/3528373738 shutdown_connections 2023-10-21T16:03:09.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.726+0000 7f4f7672e640 1 -- 172.21.15.53:0/3528373738 wait complete. 2023-10-21T16:03:09.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.728+0000 7f4f7672e640 1 Processor -- start 2023-10-21T16:03:09.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.728+0000 7f4f7672e640 1 -- start start 2023-10-21T16:03:09.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 0x7f4f700c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 0x7f4f700b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700c8f00 con 0x7f4f700cb5e0 2023-10-21T16:03:09.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700c6d10 con 0x7f4f701599d0 2023-10-21T16:03:09.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7672e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f700c6100 con 0x7f4f700b3c50 2023-10-21T16:03:09.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56300/0 (socket says 172.21.15.53:56300) 2023-10-21T16:03:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 -- 172.21.15.53:0/1748750080 learned_addr learned my addr 172.21.15.53:0/1748750080 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f75f2d640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 0x7f4f700b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7572c640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 0x7f4f700c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 -- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 msgr2=0x7f4f700c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 0x7f4f700c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 -- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 msgr2=0x7f4f700b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 0x7f4f700b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f74f2b640 1 -- 172.21.15.53:0/1748750080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68009a70 con 0x7f4f700cb5e0 2023-10-21T16:03:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f7572c640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 0x7f4f700c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:09.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.729+0000 7f4f75f2d640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 0x7f4f700b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:09.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f74f2b640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c430 tx=0x7f4f6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f600074e0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f60007660 con 0x7f4f700cb5e0 2023-10-21T16:03:09.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60004a20 con 0x7f4f700cb5e0 2023-10-21T16:03:09.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f700c63e0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.730+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f700c69a0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.731+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f70074150 con 0x7f4f700cb5e0 2023-10-21T16:03:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.732+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f60004b80 con 0x7f4f700cb5e0 2023-10-21T16:03:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.733+0000 7f4f65ffb640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 0x7f4f5406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.733+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4f60089750 con 0x7f4f700cb5e0 2023-10-21T16:03:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.734+0000 7f4f7572c640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 0x7f4f5406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:09.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.735+0000 7f4f7572c640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 0x7f4f5406d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006fd0 tx=0x7f4f6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:09.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.737+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f600543a0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.929+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f70159db0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.930+0000 7f4f65ffb640 1 -- 172.21.15.53:0/1748750080 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f70159db0 con 0x7f4f700cb5e0 2023-10-21T16:03:09.931 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 msgr2=0x7f4f5406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 0x7f4f5406d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006fd0 tx=0x7f4f6c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 msgr2=0x7f4f700b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c430 tx=0x7f4f6000c8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 shutdown_connections 2023-10-21T16:03:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f701599d0 0x7f4f700b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f4f5406aea0 0x7f4f5406d360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f700cb5e0 0x7f4f700b2650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 --2- 172.21.15.53:0/1748750080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f700b3c50 0x7f4f700c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.932+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 >> 172.21.15.53:0/1748750080 conn(0x7f4f7006c050 msgr2=0x7f4f70159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.933+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 shutdown_connections 2023-10-21T16:03:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:09.933+0000 7f4f7672e640 1 -- 172.21.15.53:0/1748750080 wait complete. 2023-10-21T16:03:09.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:10.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.163+0000 7f0577547640 1 Processor -- start 2023-10-21T16:03:10.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.164+0000 7f0577547640 1 -- start start 2023-10-21T16:03:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.165+0000 7f0577547640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f057015a820 0x7f05700aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.165+0000 7f0577547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.165+0000 7f0577547640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05700cab60 con 0x7f05700ca680 2023-10-21T16:03:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.165+0000 7f0577547640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05700cb960 con 0x7f057015a820 2023-10-21T16:03:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.165+0000 7f0577547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700cbdf0 con 0x7f057015ac00 2023-10-21T16:03:10.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0575d44640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05700ca680 0x7f05700c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49348/0 (socket says 172.21.15.53:49348) 2023-10-21T16:03:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0575d44640 1 -- 172.21.15.53:0/4239722139 learned_addr learned my addr 172.21.15.53:0/4239722139 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576545640 1 --2- 172.21.15.53:0/4239722139 >> v2:172.21.15.53:3300/0 conn(0x7f057015a820 0x7f05700aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576d46640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576545640 1 -- 172.21.15.53:0/4239722139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 msgr2=0x7f05700b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576545640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576545640 1 -- 172.21.15.53:0/4239722139 >> v1:172.21.15.53:6789/0 conn(0x7f05700ca680 legacy=0x7f05700c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576545640 1 -- 172.21.15.53:0/4239722139 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700cb030 con 0x7f057015a820 2023-10-21T16:03:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0575543640 1 -- 172.21.15.53:0/4239722139 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755317810 0 0) 0x7f05700cab60 con 0x7f05700ca680 2023-10-21T16:03:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.166+0000 7f0576d46640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:10.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.167+0000 7f0576545640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057015a820 0x7f05700aff70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f056c0097a0 tx=0x7f056c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6a03f74aaf9c8d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0575543640 1 -- 172.21.15.53:0/4239722139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c031780 con 0x7f057015a820 2023-10-21T16:03:10.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0575543640 1 -- 172.21.15.53:0/4239722139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c0318e0 con 0x7f057015a820 2023-10-21T16:03:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0575543640 1 -- 172.21.15.53:0/4239722139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c031c70 con 0x7f057015a820 2023-10-21T16:03:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 -- 172.21.15.53:0/4239722139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057015a820 msgr2=0x7f05700aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057015a820 0x7f05700aff70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f056c0097a0 tx=0x7f056c02b820 comp rx=0 tx=0).stop 2023-10-21T16:03:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 -- 172.21.15.53:0/4239722139 shutdown_connections 2023-10-21T16:03:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 --2- 172.21.15.53:0/4239722139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f057015a820 0x7f05700aff70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 -- 172.21.15.53:0/4239722139 >> 172.21.15.53:0/4239722139 conn(0x7f057006c050 msgr2=0x7f057006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.168+0000 7f0577547640 1 -- 172.21.15.53:0/4239722139 shutdown_connections 2023-10-21T16:03:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.169+0000 7f0577547640 1 -- 172.21.15.53:0/4239722139 wait complete. 2023-10-21T16:03:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.170+0000 7f0577547640 1 Processor -- start 2023-10-21T16:03:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.170+0000 7f0577547640 1 -- start start 2023-10-21T16:03:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f057015a820 0x7f05701ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05700c7500 con 0x7f05700ca680 2023-10-21T16:03:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05701efc00 con 0x7f057015a820 2023-10-21T16:03:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.171+0000 7f0577547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05700c6610 con 0x7f057015ac00 2023-10-21T16:03:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56320/0 (socket says 172.21.15.53:56320) 2023-10-21T16:03:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 -- 172.21.15.53:0/3413842916 learned_addr learned my addr 172.21.15.53:0/3413842916 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0575d44640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f057015a820 0x7f05701ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576d46640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 -- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 msgr2=0x7f05700c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 -- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f057015a820 msgr2=0x7f05701ef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f057015a820 0x7f05701ef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 -- 172.21.15.53:0/3413842916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c009400 con 0x7f05700ca680 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.172+0000 7f0576545640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.173+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c004230 con 0x7f05700ca680 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.173+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c004390 con 0x7f05700ca680 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.173+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05700c68f0 con 0x7f05700ca680 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.173+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c032860 con 0x7f05700ca680 2023-10-21T16:03:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.173+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05700c6eb0 con 0x7f05700ca680 2023-10-21T16:03:10.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.174+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0570074150 con 0x7f05700ca680 2023-10-21T16:03:10.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.175+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f056c0329c0 con 0x7f05700ca680 2023-10-21T16:03:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.176+0000 7f0566ffd640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 0x7f055406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.176+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f056c0abd10 con 0x7f05700ca680 2023-10-21T16:03:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.177+0000 7f0575d44640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 0x7f055406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.178+0000 7f0575d44640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 0x7f055406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0570153c60 tx=0x7f0560009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.180+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f056c0b1020 con 0x7f05700ca680 2023-10-21T16:03:10.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.373+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f05700b1310 con 0x7f05700ca680 2023-10-21T16:03:10.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.374+0000 7f0566ffd640 1 -- 172.21.15.53:0/3413842916 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f056c076b10 con 0x7f05700ca680 2023-10-21T16:03:10.376 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:10.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 msgr2=0x7f055406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 0x7f055406d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0570153c60 tx=0x7f0560009040 comp rx=0 tx=0).stop 2023-10-21T16:03:10.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 msgr2=0x7f0570157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 shutdown_connections 2023-10-21T16:03:10.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f057015ac00 0x7f05700c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f057015a820 0x7f05701ef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f055406ae70 0x7f055406d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 --2- 172.21.15.53:0/3413842916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05700ca680 0x7f0570157230 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 >> 172.21.15.53:0/3413842916 conn(0x7f057006c050 msgr2=0x7f05700b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:10.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 shutdown_connections 2023-10-21T16:03:10.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.377+0000 7f0577547640 1 -- 172.21.15.53:0/3413842916 wait complete. 2023-10-21T16:03:10.399 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:03:10.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.626+0000 7fcd0f3ee640 1 Processor -- start 2023-10-21T16:03:10.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.626+0000 7fcd0f3ee640 1 -- start start 2023-10-21T16:03:10.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.627+0000 7fcd0f3ee640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.627+0000 7fcd0f3ee640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcd0806a460 0x7fcd080ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.627+0000 7fcd0f3ee640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd080ce210 con 0x7fcd08071c30 2023-10-21T16:03:10.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.627+0000 7fcd0f3ee640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd080cdee0 con 0x7fcd0806a460 2023-10-21T16:03:10.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.627+0000 7fcd0f3ee640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08161530 con 0x7fcd081601b0 2023-10-21T16:03:10.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.628+0000 7fcd0d18a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.628+0000 7fcd0d18a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:49098/0 (socket says 172.21.15.53:49098) 2023-10-21T16:03:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.628+0000 7fcd0d18a640 1 -- 172.21.15.53:0/1070629732 learned_addr learned my addr 172.21.15.53:0/1070629732 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.628+0000 7fcd0d98b640 1 --2- 172.21.15.53:0/1070629732 >> v2:172.21.15.124:3300/0 conn(0x7fcd0806a460 0x7fcd080ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.629+0000 7fcd0d18a640 1 -- 172.21.15.53:0/1070629732 >> v2:172.21.15.124:3300/0 conn(0x7fcd0806a460 msgr2=0x7fcd080ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.629+0000 7fcd0d18a640 1 --2- 172.21.15.53:0/1070629732 >> v2:172.21.15.124:3300/0 conn(0x7fcd0806a460 0x7fcd080ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.629+0000 7fcd0d18a640 1 -- 172.21.15.53:0/1070629732 >> v1:172.21.15.124:6789/0 conn(0x7fcd08071c30 legacy=0x7fcd08069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.629+0000 7fcd0d18a640 1 -- 172.21.15.53:0/1070629732 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08161810 con 0x7fcd081601b0 2023-10-21T16:03:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.629+0000 7fcd07fff640 1 -- 172.21.15.53:0/1070629732 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364994481 0 0) 0x7fcd080ce210 con 0x7fcd08071c30 2023-10-21T16:03:10.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.630+0000 7fcd0d18a640 1 --2- 172.21.15.53:0/1070629732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009e50 tx=0x7fccf802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b9399e680c79d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd07fff640 1 -- 172.21.15.53:0/1070629732 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf802cb00 con 0x7fcd081601b0 2023-10-21T16:03:10.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd07fff640 1 -- 172.21.15.53:0/1070629732 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf802cc60 con 0x7fcd081601b0 2023-10-21T16:03:10.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd07fff640 1 -- 172.21.15.53:0/1070629732 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf80343f0 con 0x7fcd081601b0 2023-10-21T16:03:10.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/1070629732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 msgr2=0x7fcd080cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/1070629732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009e50 tx=0x7fccf802bf60 comp rx=0 tx=0).stop 2023-10-21T16:03:10.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/1070629732 shutdown_connections 2023-10-21T16:03:10.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/1070629732 >> v2:172.21.15.124:3300/0 conn(0x7fcd0806a460 0x7fcd080ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/1070629732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd081601b0 0x7fcd080cd430 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/1070629732 >> 172.21.15.53:0/1070629732 conn(0x7fcd0806d430 msgr2=0x7fcd0806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:10.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/1070629732 shutdown_connections 2023-10-21T16:03:10.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.631+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/1070629732 wait complete. 2023-10-21T16:03:10.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.633+0000 7fcd0f3ee640 1 Processor -- start 2023-10-21T16:03:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.633+0000 7fcd0f3ee640 1 -- start start 2023-10-21T16:03:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0806a460 0x7fcd0814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd08071c30 0x7fcd0815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0815edd0 con 0x7fcd081601b0 2023-10-21T16:03:10.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08150190 con 0x7fcd08071c30 2023-10-21T16:03:10.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.634+0000 7fcd0f3ee640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0815df20 con 0x7fcd0806a460 2023-10-21T16:03:10.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56332/0 (socket says 172.21.15.53:56332) 2023-10-21T16:03:10.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 -- 172.21.15.53:0/2813643433 learned_addr learned my addr 172.21.15.53:0/2813643433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 -- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0806a460 msgr2=0x7fcd0814fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0806a460 0x7fcd0814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 -- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd08071c30 msgr2=0x7fcd0815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd08071c30 0x7fcd0815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 -- 172.21.15.53:0/2813643433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8009b30 con 0x7fcd081601b0 2023-10-21T16:03:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.635+0000 7fcd0d98b640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e820 tx=0x7fcd0000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.636+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000c730 con 0x7fcd081601b0 2023-10-21T16:03:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.636+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000c890 con 0x7fcd081601b0 2023-10-21T16:03:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.636+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000cc20 con 0x7fcd081601b0 2023-10-21T16:03:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.636+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0815e1c0 con 0x7fcd081601b0 2023-10-21T16:03:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.636+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0815e780 con 0x7fcd081601b0 2023-10-21T16:03:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.637+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0804e160 con 0x7fcd081601b0 2023-10-21T16:03:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.638+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd0000cd80 con 0x7fcd081601b0 2023-10-21T16:03:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.639+0000 7fcd05ffb640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 0x7fccf406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.639+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcd00089aa0 con 0x7fcd081601b0 2023-10-21T16:03:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.640+0000 7fcd0d18a640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 0x7fccf406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:10.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.641+0000 7fcd0d18a640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 0x7fccf406d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccf8035000 tx=0x7fccf802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:10.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.644+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd00054790 con 0x7fcd081601b0 2023-10-21T16:03:10.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.849+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd0815ef80 con 0x7fcd081601b0 2023-10-21T16:03:10.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.850+0000 7fcd05ffb640 1 -- 172.21.15.53:0/2813643433 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd0815ef80 con 0x7fcd081601b0 2023-10-21T16:03:10.852 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:03:10.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 msgr2=0x7fccf406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 0x7fccf406d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccf8035000 tx=0x7fccf802e000 comp rx=0 tx=0).stop 2023-10-21T16:03:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 msgr2=0x7fcd0815f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e820 tx=0x7fcd0000ece0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 shutdown_connections 2023-10-21T16:03:10.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fccf406aea0 0x7fccf406d360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd081601b0 0x7fcd0815f260 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd08071c30 0x7fcd0815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 --2- 172.21.15.53:0/2813643433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0806a460 0x7fcd0814fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.853+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 >> 172.21.15.53:0/2813643433 conn(0x7fcd0806d430 msgr2=0x7fcd080b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:10.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.854+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 shutdown_connections 2023-10-21T16:03:10.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:10.854+0000 7fcd0f3ee640 1 -- 172.21.15.53:0/2813643433 wait complete. 2023-10-21T16:03:10.873 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-21T16:03:10.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:10.874 DEBUG:teuthology.orchestra.run.smithi124:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-21T16:03:10.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:03:10.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:10.989 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-21T16:03:11.056 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:11.056 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:11.118 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:11.119 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:11.176 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:11.176 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:11.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:11.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.464+0000 7f15de3dd640 1 Processor -- start 2023-10-21T16:03:11.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.464+0000 7f15de3dd640 1 -- start start 2023-10-21T16:03:11.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d815ab30 0x7f15d815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 0x7f15d80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d815f7d0 con 0x7f15d815ab30 2023-10-21T16:03:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d815f210 con 0x7f15d815af10 2023-10-21T16:03:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15de3dd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d815f4f0 con 0x7f15d80adc70 2023-10-21T16:03:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15dd3db640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d815ab30 0x7f15d815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15dd3db640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d815ab30 0x7f15d815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56336/0 (socket says 172.21.15.53:56336) 2023-10-21T16:03:11.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.466+0000 7f15dd3db640 1 -- 172.21.15.53:0/2691569467 learned_addr learned my addr 172.21.15.53:0/2691569467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dcbda640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15ddbdc640 1 --2- 172.21.15.53:0/2691569467 >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 0x7f15d80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dd3db640 1 -- 172.21.15.53:0/2691569467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 msgr2=0x7f15d80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dd3db640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d80b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dd3db640 1 -- 172.21.15.53:0/2691569467 >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 msgr2=0x7f15d80b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dd3db640 1 --2- 172.21.15.53:0/2691569467 >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 0x7f15d80b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dd3db640 1 -- 172.21.15.53:0/2691569467 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d806c550 con 0x7f15d815ab30 2023-10-21T16:03:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15dcbda640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.467+0000 7f15ddbdc640 1 --2- 172.21.15.53:0/2691569467 >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 0x7f15d80b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.468+0000 7f15dd3db640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815ab30 0x7f15d815e8a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f15d80cb000 tx=0x7f15d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2607f06f13270 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.468+0000 7f15cffff640 1 -- 172.21.15.53:0/2691569467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4031870 con 0x7f15d815ab30 2023-10-21T16:03:11.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.468+0000 7f15cffff640 1 -- 172.21.15.53:0/2691569467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d40319d0 con 0x7f15d815ab30 2023-10-21T16:03:11.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.468+0000 7f15cffff640 1 -- 172.21.15.53:0/2691569467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4031d60 con 0x7f15d815ab30 2023-10-21T16:03:11.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 -- 172.21.15.53:0/2691569467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815ab30 msgr2=0x7f15d815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815ab30 0x7f15d815e8a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f15d80cb000 tx=0x7f15d402b910 comp rx=0 tx=0).stop 2023-10-21T16:03:11.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 -- 172.21.15.53:0/2691569467 shutdown_connections 2023-10-21T16:03:11.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 --2- 172.21.15.53:0/2691569467 >> v2:172.21.15.124:3300/0 conn(0x7f15d815af10 0x7f15d80b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 --2- 172.21.15.53:0/2691569467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815ab30 0x7f15d815e8a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 -- 172.21.15.53:0/2691569467 >> 172.21.15.53:0/2691569467 conn(0x7f15d80cb0c0 msgr2=0x7f15d80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 -- 172.21.15.53:0/2691569467 shutdown_connections 2023-10-21T16:03:11.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.469+0000 7f15de3dd640 1 -- 172.21.15.53:0/2691569467 wait complete. 2023-10-21T16:03:11.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.472+0000 7f15de3dd640 1 Processor -- start 2023-10-21T16:03:11.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.472+0000 7f15de3dd640 1 -- start start 2023-10-21T16:03:11.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d81f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d815ab30 0x7f15d8146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81f64b0 con 0x7f15d815af10 2023-10-21T16:03:11.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81f6e10 con 0x7f15d815ab30 2023-10-21T16:03:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15de3dd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81f33e0 con 0x7f15d80adc70 2023-10-21T16:03:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15ddbdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15ddbdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56346/0 (socket says 172.21.15.53:56346) 2023-10-21T16:03:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.473+0000 7f15ddbdc640 1 -- 172.21.15.53:0/1742979788 learned_addr learned my addr 172.21.15.53:0/1742979788 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:11.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15dd3db640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d81f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 -- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 msgr2=0x7f15d81f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d81f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 -- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d815ab30 msgr2=0x7f15d8146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d815ab30 0x7f15d8146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 -- 172.21.15.53:0/1742979788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40094d0 con 0x7f15d815af10 2023-10-21T16:03:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15ddbdc640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f15d000cef0 tx=0x7f15d000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000d840 con 0x7f15d815af10 2023-10-21T16:03:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000d9c0 con 0x7f15d815af10 2023-10-21T16:03:11.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000dd70 con 0x7f15d815af10 2023-10-21T16:03:11.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.474+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d81f36c0 con 0x7f15d815af10 2023-10-21T16:03:11.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.475+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8146e70 con 0x7f15d815af10 2023-10-21T16:03:11.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.476+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d81f65f0 con 0x7f15d815af10 2023-10-21T16:03:11.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.477+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15d001e3b0 con 0x7f15d815af10 2023-10-21T16:03:11.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.478+0000 7f15cdffb640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 0x7f15bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.478+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f15d008a4c0 con 0x7f15d815af10 2023-10-21T16:03:11.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.478+0000 7f15dd3db640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 0x7f15bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.480+0000 7f15dd3db640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 0x7f15bc06d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15d4036000 tx=0x7f15d403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.483+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15d81f65f0 con 0x7f15d815af10 2023-10-21T16:03:11.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.676+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d81f65f0 con 0x7f15d815af10 2023-10-21T16:03:11.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.678+0000 7f15cdffb640 1 -- 172.21.15.53:0/1742979788 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f15d81f65f0 con 0x7f15d815af10 2023-10-21T16:03:11.680 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:11.680 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 msgr2=0x7f15bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 0x7f15bc06d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15d4036000 tx=0x7f15d403c040 comp rx=0 tx=0).stop 2023-10-21T16:03:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 msgr2=0x7f15d81f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f15d000cef0 tx=0x7f15d000a710 comp rx=0 tx=0).stop 2023-10-21T16:03:11.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 shutdown_connections 2023-10-21T16:03:11.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f15bc06ae70 0x7f15bc06d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d815af10 0x7f15d81f2940 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d815ab30 0x7f15d8146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 --2- 172.21.15.53:0/1742979788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d80adc70 0x7f15d81f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 >> 172.21.15.53:0/1742979788 conn(0x7f15d80cb0c0 msgr2=0x7f15d8158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:11.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 shutdown_connections 2023-10-21T16:03:11.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.681+0000 7f15de3dd640 1 -- 172.21.15.53:0/1742979788 wait complete. 2023-10-21T16:03:11.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:11.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:11.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.943+0000 7f408f59e640 1 Processor -- start 2023-10-21T16:03:11.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.943+0000 7f408f59e640 1 -- start start 2023-10-21T16:03:11.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408f59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f40900d1120 0x7f40900d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f409015dcd0 0x7f40901610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900d0a80 con 0x7f40900d1500 2023-10-21T16:03:11.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408f59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4090161620 con 0x7f40900d1120 2023-10-21T16:03:11.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40900b1fe0 con 0x7f409015dcd0 2023-10-21T16:03:11.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f40900d1120 0x7f40900d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f4085d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f40900d1500 0x7f40900cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49356/0 (socket says 172.21.15.53:49356) 2023-10-21T16:03:11.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f4085d9b640 1 -- 172.21.15.53:0/2833568736 learned_addr learned my addr 172.21.15.53:0/2833568736 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:11.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.945+0000 7f408dd9b640 1 -- 172.21.15.53:0/2833568736 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429701468 0 0) 0x7f40900d0a80 con 0x7f40900d1500 2023-10-21T16:03:11.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408dd9b640 1 -- 172.21.15.53:0/2833568736 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f407c003520 con 0x7f40900d1500 2023-10-21T16:03:11.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408ed9d640 1 --2- 172.21.15.53:0/2833568736 >> v2:172.21.15.124:3300/0 conn(0x7f409015dcd0 0x7f40901610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408e59c640 1 -- 172.21.15.53:0/2833568736 >> v2:172.21.15.124:3300/0 conn(0x7f409015dcd0 msgr2=0x7f40901610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:11.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408e59c640 1 --2- 172.21.15.53:0/2833568736 >> v2:172.21.15.124:3300/0 conn(0x7f409015dcd0 0x7f40901610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408e59c640 1 -- 172.21.15.53:0/2833568736 >> v1:172.21.15.53:6789/0 conn(0x7f40900d1500 legacy=0x7f40900cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.946+0000 7f408e59c640 1 -- 172.21.15.53:0/2833568736 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900b4580 con 0x7f40900d1120 2023-10-21T16:03:11.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.947+0000 7f408e59c640 1 --2- 172.21.15.53:0/2833568736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f40900d04a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f40900cceb0 tx=0x7f408002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec8d5326b2948fc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408dd9b640 1 -- 172.21.15.53:0/2833568736 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408002e070 con 0x7f40900d1120 2023-10-21T16:03:11.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408dd9b640 1 -- 172.21.15.53:0/2833568736 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800315a0 con 0x7f40900d1120 2023-10-21T16:03:11.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408dd9b640 1 -- 172.21.15.53:0/2833568736 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080031910 con 0x7f40900d1120 2023-10-21T16:03:11.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 -- 172.21.15.53:0/2833568736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 msgr2=0x7f40900d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 --2- 172.21.15.53:0/2833568736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f40900d04a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f40900cceb0 tx=0x7f408002b840 comp rx=0 tx=0).stop 2023-10-21T16:03:11.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 -- 172.21.15.53:0/2833568736 shutdown_connections 2023-10-21T16:03:11.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 --2- 172.21.15.53:0/2833568736 >> v2:172.21.15.124:3300/0 conn(0x7f409015dcd0 0x7f40901610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 --2- 172.21.15.53:0/2833568736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f40900d04a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.948+0000 7f408f59e640 1 -- 172.21.15.53:0/2833568736 >> 172.21.15.53:0/2833568736 conn(0x7f40900d2960 msgr2=0x7f40900d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:11.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.949+0000 7f408f59e640 1 -- 172.21.15.53:0/2833568736 shutdown_connections 2023-10-21T16:03:11.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.949+0000 7f408f59e640 1 -- 172.21.15.53:0/2833568736 wait complete. 2023-10-21T16:03:11.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.950+0000 7f408f59e640 1 Processor -- start 2023-10-21T16:03:11.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.951+0000 7f408f59e640 1 -- start start 2023-10-21T16:03:11.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40900d1500 0x7f40900c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 0x7f40900cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900c8f20 con 0x7f40900d1120 2023-10-21T16:03:11.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409015a4b0 con 0x7f409015dcd0 2023-10-21T16:03:11.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40900c8c40 con 0x7f40900d1500 2023-10-21T16:03:11.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56366/0 (socket says 172.21.15.53:56366) 2023-10-21T16:03:11.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 -- 172.21.15.53:0/3135835265 learned_addr learned my addr 172.21.15.53:0/3135835265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:11.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408ed9d640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 0x7f40900cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f4085d9b640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40900d1500 0x7f40900c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 -- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40900d1500 msgr2=0x7f40900c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40900d1500 0x7f40900c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 -- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 msgr2=0x7f40900cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:11.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.952+0000 7f408e59c640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 0x7f40900cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:11.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f408e59c640 1 -- 172.21.15.53:0/3135835265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800093a0 con 0x7f40900d1120 2023-10-21T16:03:11.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f408ed9d640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 0x7f40900cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:11.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f408e59c640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4080034040 tx=0x7f40800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080031f10 con 0x7f40900d1120 2023-10-21T16:03:11.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4080032840 con 0x7f40900d1120 2023-10-21T16:03:11.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080032bd0 con 0x7f40900d1120 2023-10-21T16:03:11.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40900c6220 con 0x7f40900d1120 2023-10-21T16:03:11.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.953+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40900c67e0 con 0x7f40900d1120 2023-10-21T16:03:11.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.955+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40900c6080 con 0x7f40900d1120 2023-10-21T16:03:11.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.955+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4080032d30 con 0x7f40900d1120 2023-10-21T16:03:11.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.956+0000 7f40877fe640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 0x7f407006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:11.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.956+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f40800ab5a0 con 0x7f40900d1120 2023-10-21T16:03:11.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.957+0000 7f4085d9b640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 0x7f407006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:11.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.958+0000 7f4085d9b640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 0x7f407006d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40900caf70 tx=0x7f4078009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:11.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:11.961+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40900c6080 con 0x7f40900d1120 2023-10-21T16:03:12.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.156+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40900c6080 con 0x7f40900d1120 2023-10-21T16:03:12.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.157+0000 7f40877fe640 1 -- 172.21.15.53:0/3135835265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40900c6080 con 0x7f40900d1120 2023-10-21T16:03:12.159 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:12.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 msgr2=0x7f407006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 0x7f407006d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f40900caf70 tx=0x7f4078009040 comp rx=0 tx=0).stop 2023-10-21T16:03:12.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 msgr2=0x7f4090159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4080034040 tx=0x7f40800046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 shutdown_connections 2023-10-21T16:03:12.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f409015dcd0 0x7f40900cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40900d1500 0x7f40900c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f407006ae70 0x7f407006d330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 --2- 172.21.15.53:0/3135835265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40900d1120 0x7f4090159f70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 >> 172.21.15.53:0/3135835265 conn(0x7f40900d2960 msgr2=0x7f40900b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:12.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 shutdown_connections 2023-10-21T16:03:12.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.160+0000 7f408f59e640 1 -- 172.21.15.53:0/3135835265 wait complete. 2023-10-21T16:03:12.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:12.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.409+0000 7fb093fff640 1 Processor -- start 2023-10-21T16:03:12.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.409+0000 7fb093fff640 1 -- start start 2023-10-21T16:03:12.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.411+0000 7fb093fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb09415cf30 0x7fb0940d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.411+0000 7fb093fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940cd520 con 0x7fb0940cf690 2023-10-21T16:03:12.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.411+0000 7fb093fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0940d1910 con 0x7fb0940cf2b0 2023-10-21T16:03:12.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.411+0000 7fb093fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940ce080 con 0x7fb09415cf30 2023-10-21T16:03:12.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.411+0000 7fb0937fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb09415cf30 0x7fb0940d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0937fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb09415cf30 0x7fb0940d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56382/0 (socket says 172.21.15.53:56382) 2023-10-21T16:03:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0927fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb0940cf690 0x7fb09415c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49370/0 (socket says 172.21.15.53:49370) 2023-10-21T16:03:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0937fe640 1 -- 172.21.15.53:0/352386882 learned_addr learned my addr 172.21.15.53:0/352386882 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb092ffd640 1 --1- 172.21.15.53:0/352386882 >> v1:172.21.15.124:6789/0 conn(0x7fb0940cf2b0 0x7fb0940d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:40850/0 (socket says 172.21.15.53:40850) 2023-10-21T16:03:12.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb091ffb640 1 -- 172.21.15.53:0/352386882 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251097237 0 0) 0x7fb0940cd520 con 0x7fb0940cf690 2023-10-21T16:03:12.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb091ffb640 1 -- 172.21.15.53:0/352386882 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c003520 con 0x7fb0940cf690 2023-10-21T16:03:12.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0937fe640 1 -- 172.21.15.53:0/352386882 >> v1:172.21.15.124:6789/0 conn(0x7fb0940cf2b0 legacy=0x7fb0940d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0937fe640 1 -- 172.21.15.53:0/352386882 >> v1:172.21.15.53:6789/0 conn(0x7fb0940cf690 legacy=0x7fb09415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.412+0000 7fb0937fe640 1 -- 172.21.15.53:0/352386882 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940cd750 con 0x7fb09415cf30 2023-10-21T16:03:12.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.414+0000 7fb0937fe640 1 --2- 172.21.15.53:0/352386882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb0940d34d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009e50 tx=0x7fb08c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ffbdfa86266cda server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.414+0000 7fb091ffb640 1 -- 172.21.15.53:0/352386882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c02cb00 con 0x7fb09415cf30 2023-10-21T16:03:12.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.414+0000 7fb091ffb640 1 -- 172.21.15.53:0/352386882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c02cc60 con 0x7fb09415cf30 2023-10-21T16:03:12.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.414+0000 7fb091ffb640 1 -- 172.21.15.53:0/352386882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0343f0 con 0x7fb09415cf30 2023-10-21T16:03:12.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 -- 172.21.15.53:0/352386882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 msgr2=0x7fb0940d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 --2- 172.21.15.53:0/352386882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb0940d34d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009e50 tx=0x7fb08c02bf60 comp rx=0 tx=0).stop 2023-10-21T16:03:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 -- 172.21.15.53:0/352386882 shutdown_connections 2023-10-21T16:03:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 --2- 172.21.15.53:0/352386882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb0940d34d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 -- 172.21.15.53:0/352386882 >> 172.21.15.53:0/352386882 conn(0x7fb09409bc40 msgr2=0x7fb0940d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 -- 172.21.15.53:0/352386882 shutdown_connections 2023-10-21T16:03:12.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.415+0000 7fb093fff640 1 -- 172.21.15.53:0/352386882 wait complete. 2023-10-21T16:03:12.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.417+0000 7fb093fff640 1 Processor -- start 2023-10-21T16:03:12.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.417+0000 7fb093fff640 1 -- start start 2023-10-21T16:03:12.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0940cf2b0 0x7fb094158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0940cf690 0x7fb0941f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941f6210 con 0x7fb09415cf30 2023-10-21T16:03:12.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941f64f0 con 0x7fb0940cf2b0 2023-10-21T16:03:12.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb093fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409cbd0 con 0x7fb0940cf690 2023-10-21T16:03:12.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56388/0 (socket says 172.21.15.53:56388) 2023-10-21T16:03:12.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 -- 172.21.15.53:0/2776487445 learned_addr learned my addr 172.21.15.53:0/2776487445 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:12.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 -- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0940cf690 msgr2=0x7fb0941f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0940cf690 0x7fb0941f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 -- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0940cf2b0 msgr2=0x7fb094158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.418+0000 7fb0937fe640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0940cf2b0 0x7fb094158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb0937fe640 1 -- 172.21.15.53:0/2776487445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009b30 con 0x7fb09415cf30 2023-10-21T16:03:12.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb0937fe640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb08c036040 tx=0x7fb08c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c009510 con 0x7fb09415cf30 2023-10-21T16:03:12.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c03e450 con 0x7fb09415cf30 2023-10-21T16:03:12.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c03e5c0 con 0x7fb09415cf30 2023-10-21T16:03:12.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09409ceb0 con 0x7fb09415cf30 2023-10-21T16:03:12.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.419+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb094158d20 con 0x7fb09415cf30 2023-10-21T16:03:12.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.421+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09409ca30 con 0x7fb09415cf30 2023-10-21T16:03:12.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.422+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb08c03e720 con 0x7fb09415cf30 2023-10-21T16:03:12.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.422+0000 7fb07b7fe640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 0x7fb08006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.423+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb08c0ac630 con 0x7fb09415cf30 2023-10-21T16:03:12.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.423+0000 7fb092ffd640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 0x7fb08006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.424+0000 7fb092ffd640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 0x7fb08006d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0840015e0 tx=0x7fb0840079c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.427+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb09409ca30 con 0x7fb09415cf30 2023-10-21T16:03:12.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.622+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb09409ca30 con 0x7fb09415cf30 2023-10-21T16:03:12.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.622+0000 7fb07b7fe640 1 -- 172.21.15.53:0/2776487445 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb09409ca30 con 0x7fb09415cf30 2023-10-21T16:03:12.624 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:12.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 msgr2=0x7fb08006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 0x7fb08006d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb0840015e0 tx=0x7fb0840079c0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 msgr2=0x7fb09415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb08c036040 tx=0x7fb08c004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 shutdown_connections 2023-10-21T16:03:12.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb08006ae70 0x7fb08006d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb09415cf30 0x7fb09415db00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0940cf690 0x7fb0941f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 --2- 172.21.15.53:0/2776487445 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0940cf2b0 0x7fb094158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 >> 172.21.15.53:0/2776487445 conn(0x7fb09409bc40 msgr2=0x7fb0940c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 shutdown_connections 2023-10-21T16:03:12.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.625+0000 7fb093fff640 1 -- 172.21.15.53:0/2776487445 wait complete. 2023-10-21T16:03:12.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:12.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.878+0000 7fc9a147c640 1 Processor -- start 2023-10-21T16:03:12.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.878+0000 7fc9a147c640 1 -- start start 2023-10-21T16:03:12.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc9a147c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc99c0b2c40 0x7fc99c167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc9a147c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c06ce20 con 0x7fc99c15d0c0 2023-10-21T16:03:12.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc9a147c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0cf850 con 0x7fc99c15cce0 2023-10-21T16:03:12.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc9a147c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c06c060 con 0x7fc99c0b2c40 2023-10-21T16:03:12.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc99a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc99c15d0c0 0x7fc99c0b2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49376/0 (socket says 172.21.15.53:49376) 2023-10-21T16:03:12.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.880+0000 7fc99a7fc640 1 -- 172.21.15.53:0/3242250846 learned_addr learned my addr 172.21.15.53:0/3242250846 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:12.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc99b7fe640 1 --2- 172.21.15.53:0/3242250846 >> v2:172.21.15.124:3300/0 conn(0x7fc99c0b2c40 0x7fc99c167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231662495 0 0) 0x7fc99c06ce20 con 0x7fc99c15d0c0 2023-10-21T16:03:12.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc988003520 con 0x7fc99c15d0c0 2023-10-21T16:03:12.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043700106 0 0) 0x7fc988003520 con 0x7fc99c15d0c0 2023-10-21T16:03:12.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 >> v2:172.21.15.124:3300/0 conn(0x7fc99c0b2c40 msgr2=0x7fc99c167ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:12.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 --2- 172.21.15.53:0/3242250846 >> v2:172.21.15.124:3300/0 conn(0x7fc99c0b2c40 0x7fc99c167ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 >> v1:172.21.15.124:6789/0 conn(0x7fc99c15cce0 legacy=0x7fc99c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c06c340 con 0x7fc99c15d0c0 2023-10-21T16:03:12.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc984001670 con 0x7fc99c15d0c0 2023-10-21T16:03:12.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.881+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 >> v1:172.21.15.53:6789/0 conn(0x7fc99c15d0c0 legacy=0x7fc99c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 0x7fc988003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc988004370 0x7fc988004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988005040 0x7fc98800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c06c060 con 0x7fc988003970 2023-10-21T16:03:12.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800bfa0 con 0x7fc988005040 2023-10-21T16:03:12.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800c160 con 0x7fc988004370 2023-10-21T16:03:12.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99affd640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 0x7fc988003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99a7fc640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc988004370 0x7fc988004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99affd640 1 -- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc988004370 msgr2=0x7fc988004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:12.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99affd640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc988004370 0x7fc988004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99b7fe640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988005040 0x7fc98800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.882+0000 7fc99affd640 1 -- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988005040 msgr2=0x7fc98800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:12.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.883+0000 7fc99affd640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988005040 0x7fc98800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.883+0000 7fc99affd640 1 -- 172.21.15.53:0/3242250846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988003520 con 0x7fc988003970 2023-10-21T16:03:12.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.883+0000 7fc99b7fe640 1 -- 172.21.15.53:0/3242250846 reap_dead start 2023-10-21T16:03:12.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc99affd640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 0x7fc988003d50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001f60 tx=0x7fc9880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d7d582faa91831 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c02d6f0 con 0x7fc988003970 2023-10-21T16:03:12.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c02d870 con 0x7fc988003970 2023-10-21T16:03:12.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc999ffb640 1 -- 172.21.15.53:0/3242250846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c02dc20 con 0x7fc988003970 2023-10-21T16:03:12.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc9a147c640 1 -- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 msgr2=0x7fc988003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.884+0000 7fc9a147c640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 0x7fc988003d50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001f60 tx=0x7fc9880057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.885+0000 7fc9a147c640 1 -- 172.21.15.53:0/3242250846 shutdown_connections 2023-10-21T16:03:12.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.885+0000 7fc9a147c640 1 --2- 172.21.15.53:0/3242250846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc988003970 0x7fc988003d50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.885+0000 7fc9a147c640 1 -- 172.21.15.53:0/3242250846 >> 172.21.15.53:0/3242250846 conn(0x7fc99c06e650 msgr2=0x7fc99c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:12.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.885+0000 7fc9a147c640 1 -- 172.21.15.53:0/3242250846 shutdown_connections 2023-10-21T16:03:12.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.885+0000 7fc9a147c640 1 -- 172.21.15.53:0/3242250846 wait complete. 2023-10-21T16:03:12.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.886+0000 7fc9a147c640 1 Processor -- start 2023-10-21T16:03:12.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.887+0000 7fc9a147c640 1 -- start start 2023-10-21T16:03:12.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.887+0000 7fc9a147c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988003970 0x7fc99c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.887+0000 7fc9a147c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc9a147c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc99c15e500 0x7fc99c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc9a147c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0cd5d0 con 0x7fc99c160000 2023-10-21T16:03:12.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc9a147c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0c4b10 con 0x7fc988003970 2023-10-21T16:03:12.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc9a147c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0c4df0 con 0x7fc99c15e500 2023-10-21T16:03:12.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56412/0 (socket says 172.21.15.53:56412) 2023-10-21T16:03:12.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 -- 172.21.15.53:0/132467941 learned_addr learned my addr 172.21.15.53:0/132467941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99affd640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988003970 0x7fc99c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99b7fe640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc99c15e500 0x7fc99c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 -- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc99c15e500 msgr2=0x7fc99c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc99c15e500 0x7fc99c15e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 -- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988003970 msgr2=0x7fc99c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:12.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988003970 0x7fc99c0ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:12.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.888+0000 7fc99a7fc640 1 -- 172.21.15.53:0/132467941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0c50d0 con 0x7fc99c160000 2023-10-21T16:03:12.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc99a7fc640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc98400e700 tx=0x7fc98400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98400ee10 con 0x7fc99c160000 2023-10-21T16:03:12.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98400c850 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98400cc00 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c0cd2c0 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.889+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c0c8740 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.890+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c0c55f0 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.891+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc98401d460 con 0x7fc99c160000 2023-10-21T16:03:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.892+0000 7fc9937fe640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 0x7fc97c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:12.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.892+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc9840897b0 con 0x7fc99c160000 2023-10-21T16:03:12.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.893+0000 7fc99affd640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 0x7fc97c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.894+0000 7fc99affd640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 0x7fc97c06d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:12.897+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc99c0c55f0 con 0x7fc99c160000 2023-10-21T16:03:13.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.092+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc99c0c8740 con 0x7fc99c160000 2023-10-21T16:03:13.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.092+0000 7fc9937fe640 1 -- 172.21.15.53:0/132467941 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc99c0c8740 con 0x7fc99c160000 2023-10-21T16:03:13.095 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 msgr2=0x7fc97c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 0x7fc97c06d360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c031000 comp rx=0 tx=0).stop 2023-10-21T16:03:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 msgr2=0x7fc99c0c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc98400e700 tx=0x7fc98400ebc0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 shutdown_connections 2023-10-21T16:03:13.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc99c15e500 0x7fc99c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc97c06aea0 0x7fc97c06d360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc99c160000 0x7fc99c0c8020 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 --2- 172.21.15.53:0/132467941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc988003970 0x7fc99c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 >> 172.21.15.53:0/132467941 conn(0x7fc99c06e650 msgr2=0x7fc99c0c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:13.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 shutdown_connections 2023-10-21T16:03:13.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.095+0000 7fc9a147c640 1 -- 172.21.15.53:0/132467941 wait complete. 2023-10-21T16:03:13.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.344+0000 7f67ccc78640 1 Processor -- start 2023-10-21T16:03:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.345+0000 7f67ccc78640 1 -- start start 2023-10-21T16:03:13.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67ccc78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67c81599d0 0x7f67c80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67ccc78640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 0x7f67c80cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67ccc78640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c8159db0 con 0x7f67c80b3c50 2023-10-21T16:03:13.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67ccc78640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c8071fe0 con 0x7f67c81599d0 2023-10-21T16:03:13.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67ccc78640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c80b44c0 con 0x7f67c80cb5e0 2023-10-21T16:03:13.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67c77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67c81599d0 0x7f67c80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67c77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67c81599d0 0x7f67c80cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56422/0 (socket says 172.21.15.53:56422) 2023-10-21T16:03:13.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67c77fe640 1 -- 172.21.15.53:0/1279008605 learned_addr learned my addr 172.21.15.53:0/1279008605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:13.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.346+0000 7f67c6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f67c80b3c50 0x7f67c80caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49378/0 (socket says 172.21.15.53:49378) 2023-10-21T16:03:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c7fff640 1 --2- 172.21.15.53:0/1279008605 >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 0x7f67c80cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c77fe640 1 -- 172.21.15.53:0/1279008605 >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 msgr2=0x7f67c80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1279008605 >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 0x7f67c80cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c77fe640 1 -- 172.21.15.53:0/1279008605 >> v1:172.21.15.53:6789/0 conn(0x7f67c80b3c50 legacy=0x7f67c80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c77fe640 1 -- 172.21.15.53:0/1279008605 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80713d0 con 0x7f67c81599d0 2023-10-21T16:03:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.347+0000 7f67c7fff640 1 --2- 172.21.15.53:0/1279008605 >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 0x7f67c80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.348+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1279008605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81599d0 0x7f67c80cdcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67c806ffd0 tx=0x7f67b802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c488ccbcd2c71 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.348+0000 7f67c67fc640 1 -- 172.21.15.53:0/1279008605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b802cb10 con 0x7f67c81599d0 2023-10-21T16:03:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67c67fc640 1 -- 172.21.15.53:0/1279008605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b802cc70 con 0x7f67c81599d0 2023-10-21T16:03:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67c67fc640 1 -- 172.21.15.53:0/1279008605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80343f0 con 0x7f67c81599d0 2023-10-21T16:03:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 -- 172.21.15.53:0/1279008605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81599d0 msgr2=0x7f67c80cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1279008605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81599d0 0x7f67c80cdcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67c806ffd0 tx=0x7f67b802bf70 comp rx=0 tx=0).stop 2023-10-21T16:03:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 -- 172.21.15.53:0/1279008605 shutdown_connections 2023-10-21T16:03:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1279008605 >> v2:172.21.15.124:3300/0 conn(0x7f67c80cb5e0 0x7f67c80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1279008605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c81599d0 0x7f67c80cdcd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 -- 172.21.15.53:0/1279008605 >> 172.21.15.53:0/1279008605 conn(0x7f67c806c050 msgr2=0x7f67c806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 -- 172.21.15.53:0/1279008605 shutdown_connections 2023-10-21T16:03:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.349+0000 7f67ccc78640 1 -- 172.21.15.53:0/1279008605 wait complete. 2023-10-21T16:03:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.351+0000 7f67ccc78640 1 Processor -- start 2023-10-21T16:03:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.351+0000 7f67ccc78640 1 -- start start 2023-10-21T16:03:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 0x7f67c80af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 0x7f67c814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80ad490 con 0x7f67c80cb5e0 2023-10-21T16:03:13.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80ad770 con 0x7f67c81599d0 2023-10-21T16:03:13.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67ccc78640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80ada50 con 0x7f67c80b3c50 2023-10-21T16:03:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.352+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56426/0 (socket says 172.21.15.53:56426) 2023-10-21T16:03:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 -- 172.21.15.53:0/1154515240 learned_addr learned my addr 172.21.15.53:0/1154515240 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 0x7f67c80af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c7fff640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 0x7f67c814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 -- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 msgr2=0x7f67c80af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 0x7f67c80af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 -- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 msgr2=0x7f67c814eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 0x7f67c814eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 -- 172.21.15.53:0/1154515240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8009b30 con 0x7f67c80cb5e0 2023-10-21T16:03:13.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 0x7f67c80af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:13.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c7fff640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 0x7f67c814eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:13.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.353+0000 7f67c6ffd640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3c0 tx=0x7f67b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.354+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00074e0 con 0x7f67c80cb5e0 2023-10-21T16:03:13.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.354+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0007640 con 0x7f67c80cb5e0 2023-10-21T16:03:13.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.354+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004a00 con 0x7f67c80cb5e0 2023-10-21T16:03:13.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.354+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c80add30 con 0x7f67c80cb5e0 2023-10-21T16:03:13.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.354+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c80ae530 con 0x7f67c80cb5e0 2023-10-21T16:03:13.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.355+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c80ae250 con 0x7f67c80cb5e0 2023-10-21T16:03:13.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.356+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67b0004b60 con 0x7f67c80cb5e0 2023-10-21T16:03:13.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.357+0000 7f67bffff640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 0x7f67a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.357+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67b0089280 con 0x7f67c80cb5e0 2023-10-21T16:03:13.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.358+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 0x7f67a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.359+0000 7f67c77fe640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 0x7f67a806d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67b8035000 tx=0x7f67b8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.361+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67c80ae250 con 0x7f67c80cb5e0 2023-10-21T16:03:13.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.554+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f67c8159db0 con 0x7f67c80cb5e0 2023-10-21T16:03:13.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.555+0000 7f67bffff640 1 -- 172.21.15.53:0/1154515240 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f67c8159db0 con 0x7f67c80cb5e0 2023-10-21T16:03:13.557 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:13.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 msgr2=0x7f67a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 0x7f67a806d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67b8035000 tx=0x7f67b8030000 comp rx=0 tx=0).stop 2023-10-21T16:03:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 msgr2=0x7f67c814e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3c0 tx=0x7f67b000c880 comp rx=0 tx=0).stop 2023-10-21T16:03:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 shutdown_connections 2023-10-21T16:03:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67c81599d0 0x7f67c814eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f67a806ae70 0x7f67a806d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67c80cb5e0 0x7f67c814e560 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 --2- 172.21.15.53:0/1154515240 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67c80b3c50 0x7f67c80af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.557+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 >> 172.21.15.53:0/1154515240 conn(0x7f67c806c050 msgr2=0x7f67c80affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.558+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 shutdown_connections 2023-10-21T16:03:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.558+0000 7f67ccc78640 1 -- 172.21.15.53:0/1154515240 wait complete. 2023-10-21T16:03:13.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:03:13.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.798+0000 7ff1104da640 1 Processor -- start 2023-10-21T16:03:13.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.798+0000 7ff1104da640 1 -- start start 2023-10-21T16:03:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff10815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff1080ad730 0x7ff1080cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080ccca0 0x7ff1080cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10815af50 con 0x7ff1080ccca0 2023-10-21T16:03:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1080ce1d0 con 0x7ff1080ad730 2023-10-21T16:03:13.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff1104da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080adc60 con 0x7ff1080cd8b0 2023-10-21T16:03:13.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff10ea77640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080ccca0 0x7ff1080cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff10ea77640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1080ccca0 0x7ff1080cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56436/0 (socket says 172.21.15.53:56436) 2023-10-21T16:03:13.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff10ea77640 1 -- 172.21.15.53:0/2706909903 learned_addr learned my addr 172.21.15.53:0/2706909903 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:13.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff10e276640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff10815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.800+0000 7ff10da75640 1 --2- 172.21.15.53:0/2706909903 >> v2:172.21.15.124:3300/0 conn(0x7ff1080ad730 0x7ff1080cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.801+0000 7ff10ea77640 1 -- 172.21.15.53:0/2706909903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 msgr2=0x7ff10815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:13.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.801+0000 7ff10ea77640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff10815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.801+0000 7ff10ea77640 1 -- 172.21.15.53:0/2706909903 >> v2:172.21.15.124:3300/0 conn(0x7ff1080ad730 msgr2=0x7ff1080cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:13.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.801+0000 7ff10ea77640 1 --2- 172.21.15.53:0/2706909903 >> v2:172.21.15.124:3300/0 conn(0x7ff1080ad730 0x7ff1080cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.801+0000 7ff10ea77640 1 -- 172.21.15.53:0/2706909903 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080adf40 con 0x7ff1080ccca0 2023-10-21T16:03:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.802+0000 7ff10ea77640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ccca0 0x7ff1080cd150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff10806e3c0 tx=0x7ff10002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea334a9df2062e0a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.803+0000 7ff10d274640 1 -- 172.21.15.53:0/2706909903 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10002ee90 con 0x7ff1080ccca0 2023-10-21T16:03:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.803+0000 7ff10d274640 1 -- 172.21.15.53:0/2706909903 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100036440 con 0x7ff1080ccca0 2023-10-21T16:03:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.803+0000 7ff10d274640 1 -- 172.21.15.53:0/2706909903 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000367d0 con 0x7ff1080ccca0 2023-10-21T16:03:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.803+0000 7ff1104da640 1 -- 172.21.15.53:0/2706909903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ccca0 msgr2=0x7ff1080cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.803+0000 7ff1104da640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ccca0 0x7ff1080cd150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff10806e3c0 tx=0x7ff10002e0e0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 -- 172.21.15.53:0/2706909903 shutdown_connections 2023-10-21T16:03:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ccca0 0x7ff1080cd150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 --2- 172.21.15.53:0/2706909903 >> v2:172.21.15.124:3300/0 conn(0x7ff1080ad730 0x7ff1080cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 --2- 172.21.15.53:0/2706909903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff10815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 -- 172.21.15.53:0/2706909903 >> 172.21.15.53:0/2706909903 conn(0x7ff108069d50 msgr2=0x7ff10806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:13.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 -- 172.21.15.53:0/2706909903 shutdown_connections 2023-10-21T16:03:13.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.804+0000 7ff1104da640 1 -- 172.21.15.53:0/2706909903 wait complete. 2023-10-21T16:03:13.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.806+0000 7ff1104da640 1 Processor -- start 2023-10-21T16:03:13.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.806+0000 7ff1104da640 1 -- start start 2023-10-21T16:03:13.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080ccca0 0x7ff1081554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff1081559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080c5820 con 0x7ff1080ad730 2023-10-21T16:03:13.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080ae4f0 con 0x7ff1080ccca0 2023-10-21T16:03:13.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.807+0000 7ff1104da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080ae7d0 con 0x7ff1080cd8b0 2023-10-21T16:03:13.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56446/0 (socket says 172.21.15.53:56446) 2023-10-21T16:03:13.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 -- 172.21.15.53:0/2357040688 learned_addr learned my addr 172.21.15.53:0/2357040688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:13.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10da75640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080ccca0 0x7ff1081554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10ea77640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff1081559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 -- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 msgr2=0x7ff1081559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff1081559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 -- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080ccca0 msgr2=0x7ff1081554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:13.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080ccca0 0x7ff1081554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:13.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 -- 172.21.15.53:0/2357040688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10000b770 con 0x7ff1080ad730 2023-10-21T16:03:13.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.808+0000 7ff10e276640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff104007ef0 tx=0x7ff10400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.809+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104014620 con 0x7ff1080ad730 2023-10-21T16:03:13.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.809+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff104014780 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.809+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104014b30 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.809+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1080aeab0 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.809+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1080ae180 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.810+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10804e160 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.811+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff104014c90 con 0x7ff1080ad730 2023-10-21T16:03:13.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.812+0000 7ff0feffd640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 0x7ff0ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:13.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.812+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff10408a3b0 con 0x7ff1080ad730 2023-10-21T16:03:13.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.813+0000 7ff10da75640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 0x7ff0ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:13.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.814+0000 7ff10da75640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 0x7ff0ec06d360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006fd0 tx=0x7ff0f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:13.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:13.816+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff104055090 con 0x7ff1080ad730 2023-10-21T16:03:14.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.025+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1080ca990 con 0x7ff1080ad730 2023-10-21T16:03:14.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.025+0000 7ff0feffd640 1 -- 172.21.15.53:0/2357040688 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff1040586b0 con 0x7ff1080ad730 2023-10-21T16:03:14.027 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:03:14.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 msgr2=0x7ff0ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 0x7ff0ec06d360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8006fd0 tx=0x7ff0f8008040 comp rx=0 tx=0).stop 2023-10-21T16:03:14.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 msgr2=0x7ff10814e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff104007ef0 tx=0x7ff10400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 shutdown_connections 2023-10-21T16:03:14.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1080cd8b0 0x7ff1081559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080ccca0 0x7ff1081554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff0ec06aea0 0x7ff0ec06d360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 --2- 172.21.15.53:0/2357040688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1080ad730 0x7ff10814e390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 >> 172.21.15.53:0/2357040688 conn(0x7ff108069d50 msgr2=0x7ff10806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:14.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 shutdown_connections 2023-10-21T16:03:14.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.028+0000 7ff1104da640 1 -- 172.21.15.53:0/2357040688 wait complete. 2023-10-21T16:03:14.049 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-21T16:03:14.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:14.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-21T16:03:14.160 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:03:14.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:14.161 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-21T16:03:14.214 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:14.214 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:14.275 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:14.275 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:14.330 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:03:14.331 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:14.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:14.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.616+0000 7f46df59e640 1 Processor -- start 2023-10-21T16:03:14.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.616+0000 7f46df59e640 1 -- start start 2023-10-21T16:03:14.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.618+0000 7f46df59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f46e006b680 0x7f46e015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.618+0000 7f46df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.618+0000 7f46df59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e00cde30 con 0x7f46e0157480 2023-10-21T16:03:14.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.618+0000 7f46df59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e0157860 con 0x7f46e006b680 2023-10-21T16:03:14.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.618+0000 7f46df59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e015e2c0 con 0x7f46e00ce2c0 2023-10-21T16:03:14.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.619+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.619+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:41740/0 (socket says 172.21.15.53:41740) 2023-10-21T16:03:14.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.619+0000 7f46ded9d640 1 -- 172.21.15.53:0/2744701901 learned_addr learned my addr 172.21.15.53:0/2744701901 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:14.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.619+0000 7f46d7fff640 1 --2- 172.21.15.53:0/2744701901 >> v2:172.21.15.124:3300/0 conn(0x7f46e006b680 0x7f46e015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.620+0000 7f46ded9d640 1 -- 172.21.15.53:0/2744701901 >> v2:172.21.15.124:3300/0 conn(0x7f46e006b680 msgr2=0x7f46e015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:14.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.620+0000 7f46ded9d640 1 --2- 172.21.15.53:0/2744701901 >> v2:172.21.15.124:3300/0 conn(0x7f46e006b680 0x7f46e015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.620+0000 7f46ded9d640 1 -- 172.21.15.53:0/2744701901 >> v1:172.21.15.124:6789/0 conn(0x7f46e0157480 legacy=0x7f46e006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.620+0000 7f46ded9d640 1 -- 172.21.15.53:0/2744701901 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00cd500 con 0x7f46e00ce2c0 2023-10-21T16:03:14.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.621+0000 7f46ded9d640 1 --2- 172.21.15.53:0/2744701901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3beff4c51d9f576 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:14.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.621+0000 7f46ddd9b640 1 -- 172.21.15.53:0/2744701901 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d802ca80 con 0x7f46e00ce2c0 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.621+0000 7f46ddd9b640 1 -- 172.21.15.53:0/2744701901 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d802cbe0 con 0x7f46e00ce2c0 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.621+0000 7f46ddd9b640 1 -- 172.21.15.53:0/2744701901 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80343f0 con 0x7f46e00ce2c0 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 -- 172.21.15.53:0/2744701901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 msgr2=0x7f46e015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 --2- 172.21.15.53:0/2744701901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e50 tx=0x7f46d802bee0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 -- 172.21.15.53:0/2744701901 shutdown_connections 2023-10-21T16:03:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 --2- 172.21.15.53:0/2744701901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e015da90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 --2- 172.21.15.53:0/2744701901 >> v2:172.21.15.124:3300/0 conn(0x7f46e006b680 0x7f46e015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 -- 172.21.15.53:0/2744701901 >> 172.21.15.53:0/2744701901 conn(0x7f46e00d2ae0 msgr2=0x7f46e00d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:14.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.622+0000 7f46df59e640 1 -- 172.21.15.53:0/2744701901 shutdown_connections 2023-10-21T16:03:14.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.623+0000 7f46df59e640 1 -- 172.21.15.53:0/2744701901 wait complete. 2023-10-21T16:03:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.625+0000 7f46df59e640 1 Processor -- start 2023-10-21T16:03:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.625+0000 7f46df59e640 1 -- start start 2023-10-21T16:03:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e01f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46e0157480 0x7f46e01f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00ca0d0 con 0x7f46e006b680 2023-10-21T16:03:14.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c9d50 con 0x7f46e0157480 2023-10-21T16:03:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.626+0000 7f46df59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b5450 con 0x7f46e00ce2c0 2023-10-21T16:03:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32872/0 (socket says 172.21.15.53:32872) 2023-10-21T16:03:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 -- 172.21.15.53:0/4129109948 learned_addr learned my addr 172.21.15.53:0/4129109948 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46ded9d640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46e0157480 0x7f46e01f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46d7fff640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e01f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 -- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 msgr2=0x7f46e01f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e01f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 -- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46e0157480 msgr2=0x7f46e01f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46e0157480 0x7f46e01f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 -- 172.21.15.53:0/4129109948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8009b30 con 0x7f46e006b680 2023-10-21T16:03:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.627+0000 7f46de59c640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c450 tx=0x7f46d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.628+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00052d0 con 0x7f46e006b680 2023-10-21T16:03:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.628+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0005430 con 0x7f46e006b680 2023-10-21T16:03:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.628+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0012870 con 0x7f46e006b680 2023-10-21T16:03:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.629+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e0155ef0 con 0x7f46e006b680 2023-10-21T16:03:14.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.629+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e01564b0 con 0x7f46e006b680 2023-10-21T16:03:14.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.630+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e004e160 con 0x7f46e006b680 2023-10-21T16:03:14.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.631+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f46d00129d0 con 0x7f46e006b680 2023-10-21T16:03:14.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.632+0000 7f46d6ffd640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 0x7f46c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:14.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.632+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46d0089540 con 0x7f46e006b680 2023-10-21T16:03:14.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.632+0000 7f46d7fff640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 0x7f46c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.633+0000 7f46d7fff640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 0x7f46c006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46e00b41d0 tx=0x7f46c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.636+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46d0054230 con 0x7f46e006b680 2023-10-21T16:03:14.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.830+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e00d21e0 con 0x7f46e006b680 2023-10-21T16:03:14.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.832+0000 7f46d6ffd640 1 -- 172.21.15.53:0/4129109948 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f46d0057850 con 0x7f46e006b680 2023-10-21T16:03:14.834 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:14.835 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 msgr2=0x7f46c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 0x7f46c006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46e00b41d0 tx=0x7f46c8008040 comp rx=0 tx=0).stop 2023-10-21T16:03:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 msgr2=0x7f46e00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f46d000c450 tx=0x7f46d000c910 comp rx=0 tx=0).stop 2023-10-21T16:03:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 shutdown_connections 2023-10-21T16:03:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46e0157480 0x7f46e01f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46e00ce2c0 0x7f46e01f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f46c006ae70 0x7f46c006d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 --2- 172.21.15.53:0/4129109948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46e006b680 0x7f46e00b4f10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 >> 172.21.15.53:0/4129109948 conn(0x7f46e00d2ae0 msgr2=0x7f46e00d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 shutdown_connections 2023-10-21T16:03:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:14.835+0000 7f46df59e640 1 -- 172.21.15.53:0/4129109948 wait complete. 2023-10-21T16:03:14.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:14.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.090+0000 7faebcd01640 1 Processor -- start 2023-10-21T16:03:15.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.091+0000 7faebcd01640 1 -- start start 2023-10-21T16:03:15.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.092+0000 7faebcd01640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faeb80cb880 0x7faeb80b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.092+0000 7faebcd01640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faeb8159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faebcd01640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb815d440 con 0x7faeb81432f0 2023-10-21T16:03:15.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faebcd01640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb81436d0 con 0x7faeb80cb880 2023-10-21T16:03:15.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faebcd01640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815d690 con 0x7faeb80b4070 2023-10-21T16:03:15.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faeb659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faeb81432f0 0x7faeb80cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55358/0 (socket says 172.21.15.53:55358) 2023-10-21T16:03:15.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faeb659c640 1 -- 172.21.15.53:0/3344007318 learned_addr learned my addr 172.21.15.53:0/3344007318 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:15.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faeb8159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/3344007318 >> v2:172.21.15.124:3300/0 conn(0x7faeb80cb880 0x7faeb80b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.093+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545853030 0 0) 0x7faeb815d440 con 0x7faeb81432f0 2023-10-21T16:03:15.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea4003520 con 0x7faeb81432f0 2023-10-21T16:03:15.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844289370 0 0) 0x7faea4003520 con 0x7faeb81432f0 2023-10-21T16:03:15.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 msgr2=0x7faeb8159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:15.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faeb8159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 >> v2:172.21.15.124:3300/0 conn(0x7faeb80cb880 msgr2=0x7faeb80b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:15.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 --2- 172.21.15.53:0/3344007318 >> v2:172.21.15.124:3300/0 conn(0x7faeb80cb880 0x7faeb80b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb815d970 con 0x7faeb81432f0 2023-10-21T16:03:15.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7faea8001670 con 0x7faeb81432f0 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 >> v1:172.21.15.53:6789/0 conn(0x7faeb81432f0 legacy=0x7faeb80cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea40039a0 0x7faea4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faea40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faea4004d00 0x7faea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81436d0 con 0x7faea4004d00 2023-10-21T16:03:15.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb815d690 con 0x7faea40039a0 2023-10-21T16:03:15.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea400b9d0 con 0x7faeb80b4070 2023-10-21T16:03:15.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.094+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faea4004d00 0x7faea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faea40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb6d9d640 1 -- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 msgr2=0x7faea40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:15.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb659c640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea40039a0 0x7faea4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faea40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb6d9d640 1 -- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea40039a0 msgr2=0x7faea4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:15.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea40039a0 0x7faea4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb6d9d640 1 -- 172.21.15.53:0/3344007318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4003520 con 0x7faea4004d00 2023-10-21T16:03:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeb80b4070 0x7faea40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.095+0000 7faeb659c640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea40039a0 0x7faea4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.096+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faea4004d00 0x7faea4005180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e170 tx=0x7faea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4abaa3ad1bed0b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.096+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb002cbf0 con 0x7faea4004d00 2023-10-21T16:03:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.096+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb000f040 con 0x7faea4004d00 2023-10-21T16:03:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.096+0000 7faeb559a640 1 -- 172.21.15.53:0/3344007318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0034440 con 0x7faea4004d00 2023-10-21T16:03:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 -- 172.21.15.53:0/3344007318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faea4004d00 msgr2=0x7faea4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 --2- 172.21.15.53:0/3344007318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faea4004d00 0x7faea4005180 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e170 tx=0x7faea40057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faeb6d9d640 1 -- 172.21.15.53:0/3344007318 reap_dead start 2023-10-21T16:03:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 -- 172.21.15.53:0/3344007318 shutdown_connections 2023-10-21T16:03:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 -- 172.21.15.53:0/3344007318 >> 172.21.15.53:0/3344007318 conn(0x7faeb8069d50 msgr2=0x7faeb806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 -- 172.21.15.53:0/3344007318 shutdown_connections 2023-10-21T16:03:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.097+0000 7faebcd01640 1 -- 172.21.15.53:0/3344007318 wait complete. 2023-10-21T16:03:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.099+0000 7faebcd01640 1 Processor -- start 2023-10-21T16:03:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.099+0000 7faebcd01640 1 -- start start 2023-10-21T16:03:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 0x7faeb8158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeb80701d0 0x7faeb8070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8159080 con 0x7faeb8133180 2023-10-21T16:03:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8070b90 con 0x7faeb80701d0 2023-10-21T16:03:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faebcd01640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8134f70 con 0x7faea4004d00 2023-10-21T16:03:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faeb5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faeb5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32894/0 (socket says 172.21.15.53:32894) 2023-10-21T16:03:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faeb5d9b640 1 -- 172.21.15.53:0/1397371225 learned_addr learned my addr 172.21.15.53:0/1397371225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faeb659c640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 0x7faeb8158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.100+0000 7faeb6d9d640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeb80701d0 0x7faeb8070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 -- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 msgr2=0x7faeb8158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 0x7faeb8158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 -- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeb80701d0 msgr2=0x7faeb8070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeb80701d0 0x7faeb8070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 -- 172.21.15.53:0/1397371225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8134c40 con 0x7faeb8133180 2023-10-21T16:03:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb659c640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 0x7faeb8158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeb5d9b640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faea00109d0 tx=0x7faea0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea000ec20 con 0x7faeb8133180 2023-10-21T16:03:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea000ed80 con 0x7faeb8133180 2023-10-21T16:03:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb8133600 con 0x7faeb8133180 2023-10-21T16:03:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea001c620 con 0x7faeb8133180 2023-10-21T16:03:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.101+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb8159550 con 0x7faeb8133180 2023-10-21T16:03:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.103+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8134aa0 con 0x7faeb8133180 2023-10-21T16:03:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.104+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faea001c780 con 0x7faeb8133180 2023-10-21T16:03:15.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.105+0000 7faeaeffd640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 0x7fae9806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.105+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faea008a0b0 con 0x7faeb8133180 2023-10-21T16:03:15.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.105+0000 7faeb659c640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 0x7fae9806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.107+0000 7faeb659c640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 0x7fae9806d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faea8001eb0 tx=0x7faea8001540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.109+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faeb8134aa0 con 0x7faeb8133180 2023-10-21T16:03:15.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.305+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faeb8134aa0 con 0x7faeb8133180 2023-10-21T16:03:15.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.306+0000 7faeaeffd640 1 -- 172.21.15.53:0/1397371225 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeb8134aa0 con 0x7faeb8133180 2023-10-21T16:03:15.308 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:15.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.308+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 msgr2=0x7fae9806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.308+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 0x7fae9806d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faea8001eb0 tx=0x7faea8001540 comp rx=0 tx=0).stop 2023-10-21T16:03:15.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 msgr2=0x7faeb8157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faea00109d0 tx=0x7faea0010e90 comp rx=0 tx=0).stop 2023-10-21T16:03:15.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 shutdown_connections 2023-10-21T16:03:15.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeb80701d0 0x7faeb8070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fae9806aea0 0x7fae9806d360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeb8133180 0x7faeb8157880 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 --2- 172.21.15.53:0/1397371225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea4004d00 0x7faeb8158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 >> 172.21.15.53:0/1397371225 conn(0x7faeb8069d50 msgr2=0x7faeb80c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:15.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 shutdown_connections 2023-10-21T16:03:15.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.309+0000 7faebcd01640 1 -- 172.21.15.53:0/1397371225 wait complete. 2023-10-21T16:03:15.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:15.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.559+0000 7f22053f5640 1 Processor -- start 2023-10-21T16:03:15.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.559+0000 7f22053f5640 1 -- start start 2023-10-21T16:03:15.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f22053f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22000afbe0 0x7f22000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f22053f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 0x7f22000cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f22053f5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22000cbad0 con 0x7f220006dfc0 2023-10-21T16:03:15.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f22053f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22000b2620 con 0x7f22000afbe0 2023-10-21T16:03:15.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f22053f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b2060 con 0x7f22000cb130 2023-10-21T16:03:15.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f21feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22000afbe0 0x7f22000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f21fe7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f220006dfc0 0x7f2200070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55360/0 (socket says 172.21.15.53:55360) 2023-10-21T16:03:15.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f21fe7fc640 1 -- 172.21.15.53:0/2778962146 learned_addr learned my addr 172.21.15.53:0/2778962146 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:15.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.561+0000 7f21ff7fe640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 0x7f22000cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.562+0000 7f21feffd640 1 -- 172.21.15.53:0/2778962146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 msgr2=0x7f22000cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:15.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.562+0000 7f21feffd640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 0x7f22000cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.562+0000 7f21feffd640 1 -- 172.21.15.53:0/2778962146 >> v1:172.21.15.53:6789/0 conn(0x7f220006dfc0 legacy=0x7f2200070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.562+0000 7f21feffd640 1 -- 172.21.15.53:0/2778962146 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000b2340 con 0x7f22000afbe0 2023-10-21T16:03:15.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.562+0000 7f21ff7fe640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 0x7f22000cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:15.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.563+0000 7f21feffd640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000afbe0 0x7f22000b15b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009d90 tx=0x7f21f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5ebda59a6f20b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.563+0000 7f21fdffb640 1 -- 172.21.15.53:0/2778962146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f002cc60 con 0x7f22000afbe0 2023-10-21T16:03:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.563+0000 7f21fdffb640 1 -- 172.21.15.53:0/2778962146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f002cdc0 con 0x7f22000afbe0 2023-10-21T16:03:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.563+0000 7f21fdffb640 1 -- 172.21.15.53:0/2778962146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0034660 con 0x7f22000afbe0 2023-10-21T16:03:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 -- 172.21.15.53:0/2778962146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000afbe0 msgr2=0x7f22000b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000afbe0 0x7f22000b15b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009d90 tx=0x7f21f002beb0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 -- 172.21.15.53:0/2778962146 shutdown_connections 2023-10-21T16:03:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22000cb130 0x7f22000cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 --2- 172.21.15.53:0/2778962146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000afbe0 0x7f22000b15b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 -- 172.21.15.53:0/2778962146 >> 172.21.15.53:0/2778962146 conn(0x7f220006a5d0 msgr2=0x7f22000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 -- 172.21.15.53:0/2778962146 shutdown_connections 2023-10-21T16:03:15.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.564+0000 7f22053f5640 1 -- 172.21.15.53:0/2778962146 wait complete. 2023-10-21T16:03:15.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.566+0000 7f22053f5640 1 Processor -- start 2023-10-21T16:03:15.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.566+0000 7f22053f5640 1 -- start start 2023-10-21T16:03:15.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 0x7f2200158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22000afbe0 0x7f22000c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22001ee380 con 0x7f22000cb130 2023-10-21T16:03:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22001f08c0 con 0x7f22000afbe0 2023-10-21T16:03:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f22053f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22001f0b30 con 0x7f220006dfc0 2023-10-21T16:03:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f21ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f21ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32902/0 (socket says 172.21.15.53:32902) 2023-10-21T16:03:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f21ff7fe640 1 -- 172.21.15.53:0/1153142143 learned_addr learned my addr 172.21.15.53:0/1153142143 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f21feffd640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 0x7f2200158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.567+0000 7f21fe7fc640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22000afbe0 0x7f22000c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 -- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 msgr2=0x7f2200158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 0x7f2200158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 -- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22000afbe0 msgr2=0x7f22000c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22000afbe0 0x7f22000c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 -- 172.21.15.53:0/1153142143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0009a70 con 0x7f22000cb130 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21feffd640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 0x7f2200158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21ff7fe640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c820 tx=0x7f21f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8007a30 con 0x7f22000cb130 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8007b90 con 0x7f22000cb130 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800ce10 con 0x7f22000cb130 2023-10-21T16:03:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.568+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22001f0e10 con 0x7f22000cb130 2023-10-21T16:03:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.569+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22001f13d0 con 0x7f22000cb130 2023-10-21T16:03:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.570+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220004e160 con 0x7f22000cb130 2023-10-21T16:03:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.571+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21f80113f0 con 0x7f22000cb130 2023-10-21T16:03:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.572+0000 7f21e77fe640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 0x7f21ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.572+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f21f8089be0 con 0x7f22000cb130 2023-10-21T16:03:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.572+0000 7f21feffd640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 0x7f21ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.573+0000 7f21feffd640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 0x7f21ec06d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.576+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f8054830 con 0x7f22000cb130 2023-10-21T16:03:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.771+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22000c5280 con 0x7f22000cb130 2023-10-21T16:03:15.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.772+0000 7f21e77fe640 1 -- 172.21.15.53:0/1153142143 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21f8057e50 con 0x7f22000cb130 2023-10-21T16:03:15.773 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:15.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 msgr2=0x7f21ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 0x7f21ec06d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f0030000 comp rx=0 tx=0).stop 2023-10-21T16:03:15.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 msgr2=0x7f22000c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:15.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c820 tx=0x7f21f800cce0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 shutdown_connections 2023-10-21T16:03:15.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f21ec06ae70 0x7f21ec06d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22000cb130 0x7f22000c3d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22000afbe0 0x7f22000c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 --2- 172.21.15.53:0/1153142143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220006dfc0 0x7f2200158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:15.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 >> 172.21.15.53:0/1153142143 conn(0x7f220006a5d0 msgr2=0x7f22000b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:15.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 shutdown_connections 2023-10-21T16:03:15.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:15.774+0000 7f22053f5640 1 -- 172.21.15.53:0/1153142143 wait complete. 2023-10-21T16:03:15.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:16.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.025+0000 7f0453a75640 1 Processor -- start 2023-10-21T16:03:16.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.025+0000 7f0453a75640 1 -- start start 2023-10-21T16:03:16.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0453a75640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f044c06dfc0 0x7f044c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0453a75640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f044c071070 0x7f044c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0453a75640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c0b1cc0 con 0x7f044c0afbe0 2023-10-21T16:03:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0453a75640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0b2620 con 0x7f044c071070 2023-10-21T16:03:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0453a75640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0b2060 con 0x7f044c06dfc0 2023-10-21T16:03:16.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0451811640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f044c0afbe0 0x7f044c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55362/0 (socket says 172.21.15.53:55362) 2023-10-21T16:03:16.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0451811640 1 -- 172.21.15.53:0/3335173671 learned_addr learned my addr 172.21.15.53:0/3335173671 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:16.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0452012640 1 --2- 172.21.15.53:0/3335173671 >> v2:172.21.15.53:3300/0 conn(0x7f044c071070 0x7f044c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f0451010640 1 --2- 172.21.15.53:0/3335173671 >> v2:172.21.15.124:3300/0 conn(0x7f044c06dfc0 0x7f044c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f045080f640 1 -- 172.21.15.53:0/3335173671 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568652796 0 0) 0x7f044c0b1cc0 con 0x7f044c0afbe0 2023-10-21T16:03:16.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.027+0000 7f045080f640 1 -- 172.21.15.53:0/3335173671 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0438003520 con 0x7f044c0afbe0 2023-10-21T16:03:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.028+0000 7f0452012640 1 -- 172.21.15.53:0/3335173671 >> v2:172.21.15.124:3300/0 conn(0x7f044c06dfc0 msgr2=0x7f044c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.028+0000 7f0452012640 1 --2- 172.21.15.53:0/3335173671 >> v2:172.21.15.124:3300/0 conn(0x7f044c06dfc0 0x7f044c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.028+0000 7f0452012640 1 -- 172.21.15.53:0/3335173671 >> v1:172.21.15.53:6789/0 conn(0x7f044c0afbe0 legacy=0x7f044c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.028+0000 7f0452012640 1 -- 172.21.15.53:0/3335173671 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0b2340 con 0x7f044c071070 2023-10-21T16:03:16.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.029+0000 7f0452012640 1 --2- 172.21.15.53:0/3335173671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c0cb130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f044c0b3980 tx=0x7f044402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ece9b06013a131 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.029+0000 7f045080f640 1 -- 172.21.15.53:0/3335173671 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044402cc70 con 0x7f044c071070 2023-10-21T16:03:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f045080f640 1 -- 172.21.15.53:0/3335173671 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044402cdd0 con 0x7f044c071070 2023-10-21T16:03:16.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f045080f640 1 -- 172.21.15.53:0/3335173671 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444034640 con 0x7f044c071070 2023-10-21T16:03:16.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 -- 172.21.15.53:0/3335173671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 msgr2=0x7f044c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 --2- 172.21.15.53:0/3335173671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c0cb130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f044c0b3980 tx=0x7f044402bec0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 -- 172.21.15.53:0/3335173671 shutdown_connections 2023-10-21T16:03:16.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 --2- 172.21.15.53:0/3335173671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c0cb130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 --2- 172.21.15.53:0/3335173671 >> v2:172.21.15.124:3300/0 conn(0x7f044c06dfc0 0x7f044c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 -- 172.21.15.53:0/3335173671 >> 172.21.15.53:0/3335173671 conn(0x7f044c06a5d0 msgr2=0x7f044c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:16.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 -- 172.21.15.53:0/3335173671 shutdown_connections 2023-10-21T16:03:16.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.030+0000 7f0453a75640 1 -- 172.21.15.53:0/3335173671 wait complete. 2023-10-21T16:03:16.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.032+0000 7f0453a75640 1 Processor -- start 2023-10-21T16:03:16.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.032+0000 7f0453a75640 1 -- start start 2023-10-21T16:03:16.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c06dfc0 0x7f044c0ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0afbe0 0x7f044c15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15c970 con 0x7f044c071070 2023-10-21T16:03:16.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15bcf0 con 0x7f044c0afbe0 2023-10-21T16:03:16.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0453a75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15bfd0 con 0x7f044c06dfc0 2023-10-21T16:03:16.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.033+0000 7f0451010640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32924/0 (socket says 172.21.15.53:32924) 2023-10-21T16:03:16.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 -- 172.21.15.53:0/493761166 learned_addr learned my addr 172.21.15.53:0/493761166 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:16.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451811640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c06dfc0 0x7f044c0ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0452012640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0afbe0 0x7f044c15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 -- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c06dfc0 msgr2=0x7f044c0ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c06dfc0 0x7f044c0ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 -- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0afbe0 msgr2=0x7f044c15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0afbe0 0x7f044c15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 -- 172.21.15.53:0/493761166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009a70 con 0x7f044c071070 2023-10-21T16:03:16.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.034+0000 7f0451010640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f044000d7d0 tx=0x7f044000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.035+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440018070 con 0x7f044c071070 2023-10-21T16:03:16.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.035+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440004020 con 0x7f044c071070 2023-10-21T16:03:16.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.035+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400041b0 con 0x7f044c071070 2023-10-21T16:03:16.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.035+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c15c2b0 con 0x7f044c071070 2023-10-21T16:03:16.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.035+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c15b930 con 0x7f044c071070 2023-10-21T16:03:16.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.036+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c04e160 con 0x7f044c071070 2023-10-21T16:03:16.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.037+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0440004310 con 0x7f044c071070 2023-10-21T16:03:16.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.038+0000 7f043e7fc640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 0x7f043006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.038+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f04400898f0 con 0x7f044c071070 2023-10-21T16:03:16.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.039+0000 7f0451811640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 0x7f043006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.040+0000 7f0451811640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 0x7f043006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0448005880 tx=0x7f0448009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.043+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04400545d0 con 0x7f044c071070 2023-10-21T16:03:16.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.234+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f044c0b0ef0 con 0x7f044c071070 2023-10-21T16:03:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.235+0000 7f043e7fc640 1 -- 172.21.15.53:0/493761166 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0440057bf0 con 0x7f044c071070 2023-10-21T16:03:16.238 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 msgr2=0x7f043006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 0x7f043006d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0448005880 tx=0x7f0448009040 comp rx=0 tx=0).stop 2023-10-21T16:03:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 msgr2=0x7f044c159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f044000d7d0 tx=0x7f044000dc90 comp rx=0 tx=0).stop 2023-10-21T16:03:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 shutdown_connections 2023-10-21T16:03:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0afbe0 0x7f044c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f043006ae70 0x7f043006d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f044c071070 0x7f044c159a60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 --2- 172.21.15.53:0/493761166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c06dfc0 0x7f044c0ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 >> 172.21.15.53:0/493761166 conn(0x7f044c06a5d0 msgr2=0x7f044c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 shutdown_connections 2023-10-21T16:03:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.238+0000 7f0453a75640 1 -- 172.21.15.53:0/493761166 wait complete. 2023-10-21T16:03:16.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:16.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:03:16.406 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:03:16.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.470+0000 7faee2773640 1 Processor -- start 2023-10-21T16:03:16.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.470+0000 7faee2773640 1 -- start start 2023-10-21T16:03:16.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.472+0000 7faee2773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.472+0000 7faee2773640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faedc0ca1a0 0x7faedc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.472+0000 7faee2773640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faedc06c3a0 con 0x7faedc0702b0 2023-10-21T16:03:16.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.472+0000 7faee2773640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faedc06c640 con 0x7faedc0ca1a0 2023-10-21T16:03:16.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.472+0000 7faee2773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc06c920 con 0x7faedc0b21f0 2023-10-21T16:03:16.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:41824/0 (socket says 172.21.15.53:41824) 2023-10-21T16:03:16.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 -- 172.21.15.53:0/1617850714 learned_addr learned my addr 172.21.15.53:0/1617850714 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:16.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faee0d10640 1 --2- 172.21.15.53:0/1617850714 >> v2:172.21.15.124:3300/0 conn(0x7faedc0ca1a0 0x7faedc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 -- 172.21.15.53:0/1617850714 >> v2:172.21.15.124:3300/0 conn(0x7faedc0ca1a0 msgr2=0x7faedc06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:16.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 --2- 172.21.15.53:0/1617850714 >> v2:172.21.15.124:3300/0 conn(0x7faedc0ca1a0 0x7faedc06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 -- 172.21.15.53:0/1617850714 >> v1:172.21.15.124:6789/0 conn(0x7faedc0702b0 legacy=0x7faedc070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.473+0000 7faedb7fe640 1 -- 172.21.15.53:0/1617850714 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc06cc00 con 0x7faedc0b21f0 2023-10-21T16:03:16.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.474+0000 7faedb7fe640 1 --2- 172.21.15.53:0/1617850714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faed000bb30 tx=0x7faed002df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9570eca4fbbf3b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.475+0000 7faedaffd640 1 -- 172.21.15.53:0/1617850714 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed002ecc0 con 0x7faedc0b21f0 2023-10-21T16:03:16.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.475+0000 7faedaffd640 1 -- 172.21.15.53:0/1617850714 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed0032070 con 0x7faedc0b21f0 2023-10-21T16:03:16.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.475+0000 7faedaffd640 1 -- 172.21.15.53:0/1617850714 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0038620 con 0x7faedc0b21f0 2023-10-21T16:03:16.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 -- 172.21.15.53:0/1617850714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 msgr2=0x7faedc0c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 --2- 172.21.15.53:0/1617850714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faed000bb30 tx=0x7faed002df10 comp rx=0 tx=0).stop 2023-10-21T16:03:16.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 -- 172.21.15.53:0/1617850714 shutdown_connections 2023-10-21T16:03:16.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 --2- 172.21.15.53:0/1617850714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0b21f0 0x7faedc0c9c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 --2- 172.21.15.53:0/1617850714 >> v2:172.21.15.124:3300/0 conn(0x7faedc0ca1a0 0x7faedc06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 -- 172.21.15.53:0/1617850714 >> 172.21.15.53:0/1617850714 conn(0x7faedc06a5d0 msgr2=0x7faedc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:16.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 -- 172.21.15.53:0/1617850714 shutdown_connections 2023-10-21T16:03:16.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.476+0000 7faee2773640 1 -- 172.21.15.53:0/1617850714 wait complete. 2023-10-21T16:03:16.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.478+0000 7faee2773640 1 Processor -- start 2023-10-21T16:03:16.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.478+0000 7faee2773640 1 -- start start 2023-10-21T16:03:16.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 0x7faedc148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faedc0ca1a0 0x7faedc0a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc1491c0 con 0x7faedc0b21f0 2023-10-21T16:03:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc14c4c0 con 0x7faedc0ca1a0 2023-10-21T16:03:16.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee2773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc14bfc0 con 0x7faedc0702b0 2023-10-21T16:03:16.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32932/0 (socket says 172.21.15.53:32932) 2023-10-21T16:03:16.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faedb7fe640 1 -- 172.21.15.53:0/2395331431 learned_addr learned my addr 172.21.15.53:0/2395331431 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:16.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faedbfff640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 0x7faedc148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.479+0000 7faee0d10640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faedc0ca1a0 0x7faedc0a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 -- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 msgr2=0x7faedc148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 0x7faedc148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 -- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faedc0ca1a0 msgr2=0x7faedc0a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faedc0ca1a0 0x7faedc0a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 -- 172.21.15.53:0/2395331431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed000b760 con 0x7faedc0b21f0 2023-10-21T16:03:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedbfff640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 0x7faedc148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:03:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faedb7fe640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faed003a040 tx=0x7faed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0006dd0 con 0x7faedc0b21f0 2023-10-21T16:03:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.480+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed0043470 con 0x7faedc0b21f0 2023-10-21T16:03:16.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.481+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0043800 con 0x7faedc0b21f0 2023-10-21T16:03:16.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.481+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc14a9e0 con 0x7faedc0b21f0 2023-10-21T16:03:16.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.481+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc14afa0 con 0x7faedc0b21f0 2023-10-21T16:03:16.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.482+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc04e160 con 0x7faedc0b21f0 2023-10-21T16:03:16.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.483+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faed0043960 con 0x7faedc0b21f0 2023-10-21T16:03:16.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.484+0000 7faed8ff9640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 0x7faec8071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.484+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faed00ac300 con 0x7faedc0b21f0 2023-10-21T16:03:16.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.485+0000 7faedbfff640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 0x7faec8071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.486+0000 7faedbfff640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 0x7faec8071740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faecc000f60 tx=0x7faecc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.488+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faed0077090 con 0x7faedc0b21f0 2023-10-21T16:03:16.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.683+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faedc0af770 con 0x7faedc0b21f0 2023-10-21T16:03:16.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.684+0000 7faed8ff9640 1 -- 172.21.15.53:0/2395331431 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7faed007a6b0 con 0x7faedc0b21f0 2023-10-21T16:03:16.687 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 msgr2=0x7faec8071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 0x7faec8071740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faecc000f60 tx=0x7faecc009040 comp rx=0 tx=0).stop 2023-10-21T16:03:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 msgr2=0x7faedc14ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faed003a040 tx=0x7faed00046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 shutdown_connections 2023-10-21T16:03:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faedc0ca1a0 0x7faedc0a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7faec806f280 0x7faec8071740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faedc0b21f0 0x7faedc14ba80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 --2- 172.21.15.53:0/2395331431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faedc0702b0 0x7faedc148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 >> 172.21.15.53:0/2395331431 conn(0x7faedc06a5d0 msgr2=0x7faedc15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:16.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 shutdown_connections 2023-10-21T16:03:16.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.687+0000 7faee2773640 1 -- 172.21.15.53:0/2395331431 wait complete. 2023-10-21T16:03:16.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:03:16.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.922+0000 7f7a197bd640 1 Processor -- start 2023-10-21T16:03:16.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.922+0000 7f7a197bd640 1 -- start start 2023-10-21T16:03:16.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.924+0000 7f7a197bd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a140cf2b0 0x7f7a140d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.924+0000 7f7a197bd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 0x7f7a1415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.924+0000 7f7a197bd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140cd520 con 0x7f7a140cf690 2023-10-21T16:03:16.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a197bd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a140d1910 con 0x7f7a140cf2b0 2023-10-21T16:03:16.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a197bd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140ce080 con 0x7f7a140ce510 2023-10-21T16:03:16.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a13fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a140cf2b0 0x7f7a140d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a0bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7a140cf690 0x7f7a140d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55378/0 (socket says 172.21.15.53:55378) 2023-10-21T16:03:16.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a0bfff640 1 -- 172.21.15.53:0/2808044886 learned_addr learned my addr 172.21.15.53:0/2808044886 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:16.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a18fbc640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 0x7f7a1415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a13fff640 1 -- 172.21.15.53:0/2808044886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 msgr2=0x7f7a1415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:16.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a13fff640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 0x7f7a1415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a13fff640 1 -- 172.21.15.53:0/2808044886 >> v1:172.21.15.53:6789/0 conn(0x7f7a140cf690 legacy=0x7f7a140d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a13fff640 1 -- 172.21.15.53:0/2808044886 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a140cd750 con 0x7f7a140cf2b0 2023-10-21T16:03:16.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.925+0000 7f7a18fbc640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 0x7f7a1415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:16.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.927+0000 7f7a13fff640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140cf2b0 0x7f7a140d34d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d393292d76b8fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.927+0000 7f7a137fe640 1 -- 172.21.15.53:0/2808044886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04031780 con 0x7f7a140cf2b0 2023-10-21T16:03:16.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.928+0000 7f7a137fe640 1 -- 172.21.15.53:0/2808044886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a040318e0 con 0x7f7a140cf2b0 2023-10-21T16:03:16.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.928+0000 7f7a137fe640 1 -- 172.21.15.53:0/2808044886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04031c70 con 0x7f7a140cf2b0 2023-10-21T16:03:16.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.928+0000 7f7a197bd640 1 -- 172.21.15.53:0/2808044886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140cf2b0 msgr2=0x7f7a140d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.928+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140cf2b0 0x7f7a140d34d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402b820 comp rx=0 tx=0).stop 2023-10-21T16:03:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 -- 172.21.15.53:0/2808044886 shutdown_connections 2023-10-21T16:03:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140ce510 0x7f7a1415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2808044886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140cf2b0 0x7f7a140d34d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 -- 172.21.15.53:0/2808044886 >> 172.21.15.53:0/2808044886 conn(0x7f7a1409bc40 msgr2=0x7f7a140d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:16.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 -- 172.21.15.53:0/2808044886 shutdown_connections 2023-10-21T16:03:16.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.929+0000 7f7a197bd640 1 -- 172.21.15.53:0/2808044886 wait complete. 2023-10-21T16:03:16.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.931+0000 7f7a197bd640 1 Processor -- start 2023-10-21T16:03:16.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.931+0000 7f7a197bd640 1 -- start start 2023-10-21T16:03:16.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 0x7f7a1409e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a140cf690 0x7f7a1409ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1409e490 con 0x7f7a140ce510 2023-10-21T16:03:16.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1409ded0 con 0x7f7a140cf690 2023-10-21T16:03:16.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a197bd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1409e1b0 con 0x7f7a140cf2b0 2023-10-21T16:03:16.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32956/0 (socket says 172.21.15.53:32956) 2023-10-21T16:03:16.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 -- 172.21.15.53:0/2675057759 learned_addr learned my addr 172.21.15.53:0/2675057759 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:16.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a0bfff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 0x7f7a1409e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a18fbc640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a140cf690 0x7f7a1409ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 -- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 msgr2=0x7f7a1409e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 0x7f7a1409e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 -- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a140cf690 msgr2=0x7f7a1409ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a140cf690 0x7f7a1409ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a13fff640 1 -- 172.21.15.53:0/2675057759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04009400 con 0x7f7a140ce510 2023-10-21T16:03:16.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.932+0000 7f7a0bfff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 0x7f7a1409e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:16.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a13fff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a04034040 tx=0x7f7a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04032960 con 0x7f7a140ce510 2023-10-21T16:03:16.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04032ac0 con 0x7f7a140ce510 2023-10-21T16:03:16.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04032e70 con 0x7f7a140ce510 2023-10-21T16:03:16.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a140c9b10 con 0x7f7a140ce510 2023-10-21T16:03:16.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.933+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1415b8a0 con 0x7f7a140ce510 2023-10-21T16:03:16.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.934+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a14074150 con 0x7f7a140ce510 2023-10-21T16:03:16.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.935+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7a04045400 con 0x7f7a140ce510 2023-10-21T16:03:16.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.937+0000 7f7a117fa640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 0x7f79f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:16.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.937+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7a040abe60 con 0x7f7a140ce510 2023-10-21T16:03:16.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.937+0000 7f7a0bfff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 0x7f79f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:16.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.938+0000 7f7a0bfff640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 0x7f79f406d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a140c97f0 tx=0x7f79fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:16.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:16.941+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a04076b40 con 0x7f7a140ce510 2023-10-21T16:03:17.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.163+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a140d0070 con 0x7f7a140ce510 2023-10-21T16:03:17.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.164+0000 7f7a117fa640 1 -- 172.21.15.53:0/2675057759 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7a0407a160 con 0x7f7a140ce510 2023-10-21T16:03:17.166 INFO:teuthology.orchestra.run.smithi053.stderr:listed 0 entries 2023-10-21T16:03:17.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 msgr2=0x7f79f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 0x7f79f406d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a140c97f0 tx=0x7f79fc009040 comp rx=0 tx=0).stop 2023-10-21T16:03:17.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 msgr2=0x7f7a1415b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a04034040 tx=0x7f7a040046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 shutdown_connections 2023-10-21T16:03:17.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a140cf690 0x7f7a1409ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a140cf2b0 0x7f7a1409e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f79f406aea0 0x7f79f406d360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 --2- 172.21.15.53:0/2675057759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a140ce510 0x7f7a1415b360 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 >> 172.21.15.53:0/2675057759 conn(0x7f7a1409bc40 msgr2=0x7f7a1415bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:17.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 shutdown_connections 2023-10-21T16:03:17.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.167+0000 7f7a197bd640 1 -- 172.21.15.53:0/2675057759 wait complete. 2023-10-21T16:03:17.188 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-21T16:03:17.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:17.189 DEBUG:teuthology.orchestra.run.smithi124:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-21T16:03:17.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T16:03:17.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:17.283 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-21T16:03:17.324 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:03:17.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:17.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.536+0000 7f1f6e6ff640 1 Processor -- start 2023-10-21T16:03:17.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.536+0000 7f1f6e6ff640 1 -- start start 2023-10-21T16:03:17.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f680d07c0 0x7f1f680d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f680d1470 0x7f1f68160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680d1900 con 0x7f1f680d07c0 2023-10-21T16:03:17.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680ce610 con 0x7f1f680d1470 2023-10-21T16:03:17.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6e6ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6815d1d0 con 0x7f1f6815cdf0 2023-10-21T16:03:17.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.537+0000 7f1f6cefc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f680d07c0 0x7f1f680d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f680d07c0 0x7f1f680d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32968/0 (socket says 172.21.15.53:32968) 2023-10-21T16:03:17.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2013301852 learned_addr learned my addr 172.21.15.53:0/2013301852 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:17.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6defe640 1 --2- 172.21.15.53:0/2013301852 >> v2:172.21.15.124:3300/0 conn(0x7f1f680d1470 0x7f1f68160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2013301852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 msgr2=0x7f1f680ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:17.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6d6fd640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2013301852 >> v2:172.21.15.124:3300/0 conn(0x7f1f680d1470 msgr2=0x7f1f68160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:17.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2013301852 >> v2:172.21.15.124:3300/0 conn(0x7f1f680d1470 0x7f1f68160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2013301852 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680ceb00 con 0x7f1f680d07c0 2023-10-21T16:03:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.538+0000 7f1f6d6fd640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.539+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d07c0 0x7f1f680d0f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f680d1c10 tx=0x7f1f5c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16fdedb896487bd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.539+0000 7f1f67fff640 1 -- 172.21.15.53:0/2013301852 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c031740 con 0x7f1f680d07c0 2023-10-21T16:03:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.539+0000 7f1f67fff640 1 -- 172.21.15.53:0/2013301852 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0318a0 con 0x7f1f680d07c0 2023-10-21T16:03:17.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.539+0000 7f1f67fff640 1 -- 172.21.15.53:0/2013301852 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c031c30 con 0x7f1f680d07c0 2023-10-21T16:03:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2013301852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d07c0 msgr2=0x7f1f680d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d07c0 0x7f1f680d0f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1f680d1c10 tx=0x7f1f5c02b7e0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2013301852 shutdown_connections 2023-10-21T16:03:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2013301852 >> v2:172.21.15.124:3300/0 conn(0x7f1f680d1470 0x7f1f68160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d07c0 0x7f1f680d0f30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2013301852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2013301852 >> 172.21.15.53:0/2013301852 conn(0x7f1f6809bcb0 msgr2=0x7f1f680d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2013301852 shutdown_connections 2023-10-21T16:03:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.540+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2013301852 wait complete. 2023-10-21T16:03:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.542+0000 7f1f6e6ff640 1 Processor -- start 2023-10-21T16:03:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.542+0000 7f1f6e6ff640 1 -- start start 2023-10-21T16:03:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f680d07c0 0x7f1f68155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68157220 con 0x7f1f680d1470 2023-10-21T16:03:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68156290 con 0x7f1f680d07c0 2023-10-21T16:03:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6e6ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68156570 con 0x7f1f6815cdf0 2023-10-21T16:03:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6cefc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6cefc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32978/0 (socket says 172.21.15.53:32978) 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2196199678 learned_addr learned my addr 172.21.15.53:0/2196199678 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6d6fd640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f680d07c0 0x7f1f68155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.543+0000 7f1f6defe640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 msgr2=0x7f1f680c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f680d07c0 msgr2=0x7f1f68155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f680d07c0 0x7f1f68155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 -- 172.21.15.53:0/2196199678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0094d0 con 0x7f1f680d1470 2023-10-21T16:03:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6defe640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6cefc640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097f0 tx=0x7f1f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c033c70 con 0x7f1f680d1470 2023-10-21T16:03:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c033dd0 con 0x7f1f680d1470 2023-10-21T16:03:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c03d6b0 con 0x7f1f680d1470 2023-10-21T16:03:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f68156850 con 0x7f1f680d1470 2023-10-21T16:03:17.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.544+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f680c91a0 con 0x7f1f680d1470 2023-10-21T16:03:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.545+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68157390 con 0x7f1f680d1470 2023-10-21T16:03:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.546+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f5c03a040 con 0x7f1f680d1470 2023-10-21T16:03:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.547+0000 7f1f65ffb640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 0x7f1f5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:17.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.547+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f5c0ab7d0 con 0x7f1f680d1470 2023-10-21T16:03:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.548+0000 7f1f6d6fd640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 0x7f1f5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.549+0000 7f1f6d6fd640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 0x7f1f5406d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f580075b0 tx=0x7f1f58007540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:17.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.552+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f68157390 con 0x7f1f680d1470 2023-10-21T16:03:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.779+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f68157360 con 0x7f1f680d1470 2023-10-21T16:03:17.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.780+0000 7f1f65ffb640 1 -- 172.21.15.53:0/2196199678 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f1f5c0764b0 con 0x7f1f680d1470 2023-10-21T16:03:17.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:17.783 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:03:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:03:17.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 msgr2=0x7f1f5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 0x7f1f5406d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f580075b0 tx=0x7f1f58007540 comp rx=0 tx=0).stop 2023-10-21T16:03:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 msgr2=0x7f1f68159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097f0 tx=0x7f1f5c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 shutdown_connections 2023-10-21T16:03:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f6815cdf0 0x7f1f680c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f1f5406ae70 0x7f1f5406d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f680d1470 0x7f1f68159ca0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 --2- 172.21.15.53:0/2196199678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f680d07c0 0x7f1f68155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 >> 172.21.15.53:0/2196199678 conn(0x7f1f6809bcb0 msgr2=0x7f1f6815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 shutdown_connections 2023-10-21T16:03:17.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:17.783+0000 7f1f6e6ff640 1 -- 172.21.15.53:0/2196199678 wait complete. 2023-10-21T16:03:17.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:18.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.004+0000 7fdcda5ce640 1 Processor -- start 2023-10-21T16:03:18.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.004+0000 7fdcda5ce640 1 -- start start 2023-10-21T16:03:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcda5ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcd40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcda5ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd415ec40 con 0x7fdcd40b09c0 2023-10-21T16:03:18.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcda5ce640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd415ef20 con 0x7fdcd415af40 2023-10-21T16:03:18.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcda5ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b1e00 con 0x7fdcd415ab60 2023-10-21T16:03:18.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcd8b6b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdcd40b09c0 0x7fdcd406c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55394/0 (socket says 172.21.15.53:55394) 2023-10-21T16:03:18.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.006+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/155212021 learned_addr learned my addr 172.21.15.53:0/155212021 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:18.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784084825 0 0) 0x7fdcd415ec40 con 0x7fdcd40b09c0 2023-10-21T16:03:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcbc003520 con 0x7fdcd40b09c0 2023-10-21T16:03:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcd40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323929877 0 0) 0x7fdcbc003520 con 0x7fdcd40b09c0 2023-10-21T16:03:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 msgr2=0x7fdcd40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcd40b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 >> v1:172.21.15.124:6789/0 conn(0x7fdcd415af40 legacy=0x7fdcd40b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd415f200 con 0x7fdcd40b09c0 2023-10-21T16:03:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fdccc0019f0 con 0x7fdcd40b09c0 2023-10-21T16:03:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 >> v1:172.21.15.53:6789/0 conn(0x7fdcd40b09c0 legacy=0x7fdcd406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcbc0039a0 0x7fdcbc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcbc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcbc004d00 0x7fdcbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40b1e00 con 0x7fdcbc0039a0 2023-10-21T16:03:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc00ba30 con 0x7fdcbc004d00 2023-10-21T16:03:18.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.007+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc00bbf0 con 0x7fdcd415ab60 2023-10-21T16:03:18.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.008+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcbc0039a0 0x7fdcbc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.008+0000 7fdcd8b6b640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcbc004d00 0x7fdcbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.008+0000 7fdcd37fe640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcbc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.008+0000 7fdcd3fff640 1 -- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 msgr2=0x7fdcbc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.008+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcd415ab60 0x7fdcbc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.009+0000 7fdcd3fff640 1 -- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcbc004d00 msgr2=0x7fdcbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.009+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcbc004d00 0x7fdcbc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.009+0000 7fdcd3fff640 1 -- 172.21.15.53:0/155212021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc003520 con 0x7fdcbc0039a0 2023-10-21T16:03:18.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.010+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcbc0039a0 0x7fdcbc003d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400ba00 tx=0x7fdcbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e06409380dad7a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.010+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4032620 con 0x7fdcbc0039a0 2023-10-21T16:03:18.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.010+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4032780 con 0x7fdcbc0039a0 2023-10-21T16:03:18.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.010+0000 7fdcd2ffd640 1 -- 172.21.15.53:0/155212021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4032b10 con 0x7fdcbc0039a0 2023-10-21T16:03:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 -- 172.21.15.53:0/155212021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcbc0039a0 msgr2=0x7fdcbc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/155212021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcbc0039a0 0x7fdcbc003d80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400ba00 tx=0x7fdcbc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/155212021 reap_dead start 2023-10-21T16:03:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 -- 172.21.15.53:0/155212021 shutdown_connections 2023-10-21T16:03:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 -- 172.21.15.53:0/155212021 >> 172.21.15.53:0/155212021 conn(0x7fdcd40cb2b0 msgr2=0x7fdcd40cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:18.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 -- 172.21.15.53:0/155212021 shutdown_connections 2023-10-21T16:03:18.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.011+0000 7fdcda5ce640 1 -- 172.21.15.53:0/155212021 wait complete. 2023-10-21T16:03:18.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.013+0000 7fdcda5ce640 1 Processor -- start 2023-10-21T16:03:18.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.013+0000 7fdcda5ce640 1 -- start start 2023-10-21T16:03:18.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcbc0039a0 0x7fdcd40ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcd41f2c00 0x7fdcd41f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40ad360 con 0x7fdcd40ac970 2023-10-21T16:03:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40ab8d0 con 0x7fdcd41f2c00 2023-10-21T16:03:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.014+0000 7fdcda5ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40abbb0 con 0x7fdcbc0039a0 2023-10-21T16:03:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32996/0 (socket says 172.21.15.53:32996) 2023-10-21T16:03:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/3441590222 learned_addr learned my addr 172.21.15.53:0/3441590222 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcbc0039a0 0x7fdcd40ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd37fe640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcd41f2c00 0x7fdcd41f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcbc0039a0 msgr2=0x7fdcd40ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcbc0039a0 0x7fdcd40ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcd41f2c00 msgr2=0x7fdcd41f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcd41f2c00 0x7fdcd41f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 -- 172.21.15.53:0/3441590222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40abe90 con 0x7fdcd40ac970 2023-10-21T16:03:18.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.015+0000 7fdcd8b6b640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdccc002230 tx=0x7fdccc007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.016+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00ab60 con 0x7fdcd40ac970 2023-10-21T16:03:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.016+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc00ad00 con 0x7fdcd40ac970 2023-10-21T16:03:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.016+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc013630 con 0x7fdcd40ac970 2023-10-21T16:03:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.016+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40ac170 con 0x7fdcd40ac970 2023-10-21T16:03:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.016+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd40cd330 con 0x7fdcd40ac970 2023-10-21T16:03:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.017+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd40ac690 con 0x7fdcd40ac970 2023-10-21T16:03:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.018+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdccc013790 con 0x7fdcd40ac970 2023-10-21T16:03:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.019+0000 7fdcd0ff9640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 0x7fdcc006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.019+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdccc08a030 con 0x7fdcd40ac970 2023-10-21T16:03:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.020+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 0x7fdcc006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.020+0000 7fdcd3fff640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 0x7fdcc006d360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc004740 tx=0x7fdcc403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.023+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcd40ac690 con 0x7fdcd40ac970 2023-10-21T16:03:18.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.221+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcd40ac690 con 0x7fdcd40ac970 2023-10-21T16:03:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.223+0000 7fdcd0ff9640 1 -- 172.21.15.53:0/3441590222 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fdcd40ac690 con 0x7fdcd40ac970 2023-10-21T16:03:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:18.225 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 msgr2=0x7fdcc006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 0x7fdcc006d360 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc004740 tx=0x7fdcc403d040 comp rx=0 tx=0).stop 2023-10-21T16:03:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 msgr2=0x7fdcd40ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdccc002230 tx=0x7fdccc007b00 comp rx=0 tx=0).stop 2023-10-21T16:03:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 shutdown_connections 2023-10-21T16:03:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fdcc006aea0 0x7fdcc006d360 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcd40ac970 0x7fdcd40ace20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcd41f2c00 0x7fdcd41f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 --2- 172.21.15.53:0/3441590222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcbc0039a0 0x7fdcd40ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 >> 172.21.15.53:0/3441590222 conn(0x7fdcd40cb2b0 msgr2=0x7fdcd40cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:18.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 shutdown_connections 2023-10-21T16:03:18.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.226+0000 7fdcda5ce640 1 -- 172.21.15.53:0/3441590222 wait complete. 2023-10-21T16:03:18.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:01:53.459159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:01:54.644921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-21T16:01:58.201944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-21T16:03:18.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.473+0000 7fc032652640 1 Processor -- start 2023-10-21T16:03:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.473+0000 7fc032652640 1 -- start start 2023-10-21T16:03:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c8fb0 0x7fc02c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc02c0c9390 0x7fc02c160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc02c160c90 0x7fc02c15c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c1610d0 con 0x7fc02c160c90 2023-10-21T16:03:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0b0260 con 0x7fc02c0c9390 2023-10-21T16:03:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc032652640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1612d0 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc031e51640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc02c160c90 0x7fc02c15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc031e51640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc02c160c90 0x7fc02c15c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:32998/0 (socket says 172.21.15.53:32998) 2023-10-21T16:03:18.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.475+0000 7fc031e51640 1 -- 172.21.15.53:0/521546504 learned_addr learned my addr 172.21.15.53:0/521546504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:18.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc030e4f640 1 --2- 172.21.15.53:0/521546504 >> v2:172.21.15.124:3300/0 conn(0x7fc02c0c9390 0x7fc02c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031650640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c8fb0 0x7fc02c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031e51640 1 -- 172.21.15.53:0/521546504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c8fb0 msgr2=0x7fc02c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031e51640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c8fb0 0x7fc02c0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031e51640 1 -- 172.21.15.53:0/521546504 >> v2:172.21.15.124:3300/0 conn(0x7fc02c0c9390 msgr2=0x7fc02c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031e51640 1 --2- 172.21.15.53:0/521546504 >> v2:172.21.15.124:3300/0 conn(0x7fc02c0c9390 0x7fc02c160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.476+0000 7fc031e51640 1 -- 172.21.15.53:0/521546504 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c161590 con 0x7fc02c160c90 2023-10-21T16:03:18.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.478+0000 7fc031e51640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c160c90 0x7fc02c15c630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cbfd0 tx=0x7fc02402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6812fd034cf75edd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.478+0000 7fc023fff640 1 -- 172.21.15.53:0/521546504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02402cb00 con 0x7fc02c160c90 2023-10-21T16:03:18.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.478+0000 7fc023fff640 1 -- 172.21.15.53:0/521546504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02402cc60 con 0x7fc02c160c90 2023-10-21T16:03:18.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.478+0000 7fc023fff640 1 -- 172.21.15.53:0/521546504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240343f0 con 0x7fc02c160c90 2023-10-21T16:03:18.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 -- 172.21.15.53:0/521546504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c160c90 msgr2=0x7fc02c15c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c160c90 0x7fc02c15c630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0cbfd0 tx=0x7fc02402bf60 comp rx=0 tx=0).stop 2023-10-21T16:03:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 -- 172.21.15.53:0/521546504 shutdown_connections 2023-10-21T16:03:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c160c90 0x7fc02c15c630 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 --2- 172.21.15.53:0/521546504 >> v2:172.21.15.124:3300/0 conn(0x7fc02c0c9390 0x7fc02c160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 --2- 172.21.15.53:0/521546504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c8fb0 0x7fc02c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 -- 172.21.15.53:0/521546504 >> 172.21.15.53:0/521546504 conn(0x7fc02c06c010 msgr2=0x7fc02c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 -- 172.21.15.53:0/521546504 shutdown_connections 2023-10-21T16:03:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.479+0000 7fc032652640 1 -- 172.21.15.53:0/521546504 wait complete. 2023-10-21T16:03:18.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.481+0000 7fc032652640 1 Processor -- start 2023-10-21T16:03:18.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.481+0000 7fc032652640 1 -- start start 2023-10-21T16:03:18.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.482+0000 7fc032652640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc032652640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c9390 0x7fc02c0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc032652640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc02c160c90 0x7fc02c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc032652640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0b2800 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc032652640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c153890 con 0x7fc02c160c90 2023-10-21T16:03:18.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc032652640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0b1b80 con 0x7fc02c0c9390 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33014/0 (socket says 172.21.15.53:33014) 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 -- 172.21.15.53:0/3518780742 learned_addr learned my addr 172.21.15.53:0/3518780742 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 -- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c9390 msgr2=0x7fc02c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c9390 0x7fc02c0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031e51640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc02c160c90 0x7fc02c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 -- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc02c160c90 msgr2=0x7fc02c0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc02c160c90 0x7fc02c0c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.483+0000 7fc031650640 1 -- 172.21.15.53:0/3518780742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc024009b30 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc031650640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc02800d620 tx=0x7fc02800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280125f0 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028012750 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028012b00 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0b1e60 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.484+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0b2420 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.485+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0b25b0 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.486+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc028012c60 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.487+0000 7fc021ffb640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 0x7fc01006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.487+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc028089e70 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.488+0000 7fc030e4f640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 0x7fc01006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.489+0000 7fc030e4f640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 0x7fc01006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc02c160650 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.492+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc028054b50 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.687+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc02c074150 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.688+0000 7fc021ffb640 1 -- 172.21.15.53:0/3518780742 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc028058170 con 0x7fc02c0c8fb0 2023-10-21T16:03:18.689 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:18.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 msgr2=0x7fc01006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 0x7fc01006d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc02c160650 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:18.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 msgr2=0x7fc02c153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc02800d620 tx=0x7fc02800dae0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 shutdown_connections 2023-10-21T16:03:18.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc02c160c90 0x7fc02c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c9390 0x7fc02c0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc01006ae70 0x7fc01006d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 --2- 172.21.15.53:0/3518780742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c0c8fb0 0x7fc02c153350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 >> 172.21.15.53:0/3518780742 conn(0x7fc02c06c010 msgr2=0x7fc02c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:18.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.690+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 shutdown_connections 2023-10-21T16:03:18.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.691+0000 7fc032652640 1 -- 172.21.15.53:0/3518780742 wait complete. 2023-10-21T16:03:18.710 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:03:18.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.940+0000 7f69d9467640 1 Processor -- start 2023-10-21T16:03:18.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.941+0000 7f69d9467640 1 -- start start 2023-10-21T16:03:18.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.942+0000 7f69d9467640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69d415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69d9467640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d40cab60 con 0x7f69d415a820 2023-10-21T16:03:18.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69d9467640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d40cb960 con 0x7f69d40ca680 2023-10-21T16:03:18.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69d9467640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cbdf0 con 0x7f69d40b0680 2023-10-21T16:03:18.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69d3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f69d415a820 0x7f69d40aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55396/0 (socket says 172.21.15.53:55396) 2023-10-21T16:03:18.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69d3fff640 1 -- 172.21.15.53:0/2479688334 learned_addr learned my addr 172.21.15.53:0/2479688334 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:18.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.943+0000 7f69cbfff640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69d415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876367187 0 0) 0x7f69d40cab60 con 0x7f69d415a820 2023-10-21T16:03:18.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c0003520 con 0x7f69d415a820 2023-10-21T16:03:18.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 201181287 0 0) 0x7f69c0003520 con 0x7f69d415a820 2023-10-21T16:03:18.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 msgr2=0x7f69d415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69d415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 >> v1:172.21.15.124:6789/0 conn(0x7f69d40ca680 legacy=0x7f69d415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d40cb030 con 0x7f69d415a820 2023-10-21T16:03:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f69c4001670 con 0x7f69d415a820 2023-10-21T16:03:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 >> v1:172.21.15.53:6789/0 conn(0x7f69d415a820 legacy=0x7f69d40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69c00039a0 0x7f69c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cbdf0 con 0x7f69c0004d00 2023-10-21T16:03:18.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c000bab0 con 0x7f69c00039a0 2023-10-21T16:03:18.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.944+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c000bc70 con 0x7f69d40b0680 2023-10-21T16:03:18.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d3fff640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69c00039a0 0x7f69c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69cbfff640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 -- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 msgr2=0x7f69c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d40b0680 0x7f69c00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 -- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69c00039a0 msgr2=0x7f69c0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:18.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69c00039a0 0x7f69c0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.945+0000 7f69d8c66640 1 -- 172.21.15.53:0/2479688334 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0003520 con 0x7f69c0004d00 2023-10-21T16:03:18.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.946+0000 7f69d8c66640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69c0005180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69d40caa90 tx=0x7f69c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcd403d9134aa18 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc003b70 con 0x7f69c0004d00 2023-10-21T16:03:18.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc003cd0 con 0x7f69c0004d00 2023-10-21T16:03:18.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d37fe640 1 -- 172.21.15.53:0/2479688334 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc005310 con 0x7f69c0004d00 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d9467640 1 -- 172.21.15.53:0/2479688334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 msgr2=0x7f69c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d9467640 1 --2- 172.21.15.53:0/2479688334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69c0005180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69d40caa90 tx=0x7f69c00057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d8c66640 1 -- 172.21.15.53:0/2479688334 reap_dead start 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d9467640 1 -- 172.21.15.53:0/2479688334 shutdown_connections 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.947+0000 7f69d9467640 1 -- 172.21.15.53:0/2479688334 >> 172.21.15.53:0/2479688334 conn(0x7f69d406c050 msgr2=0x7f69d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:18.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.948+0000 7f69d9467640 1 -- 172.21.15.53:0/2479688334 shutdown_connections 2023-10-21T16:03:18.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.948+0000 7f69d9467640 1 -- 172.21.15.53:0/2479688334 wait complete. 2023-10-21T16:03:18.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.949+0000 7f69d9467640 1 Processor -- start 2023-10-21T16:03:18.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.949+0000 7f69d9467640 1 -- start start 2023-10-21T16:03:18.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d4158b50 0x7f69d40cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d40ce0a0 0x7f69d41f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d41f30b0 con 0x7f69c0004d00 2023-10-21T16:03:18.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40ce520 con 0x7f69d40ce0a0 2023-10-21T16:03:18.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.950+0000 7f69d9467640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cd120 con 0x7f69d4158b50 2023-10-21T16:03:18.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33034/0 (socket says 172.21.15.53:33034) 2023-10-21T16:03:18.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 -- 172.21.15.53:0/2145941640 learned_addr learned my addr 172.21.15.53:0/2145941640 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:18.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 -- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d4158b50 msgr2=0x7f69d40cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d4158b50 0x7f69d40cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d8c66640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d40ce0a0 0x7f69d41f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 -- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d40ce0a0 msgr2=0x7f69d41f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d40ce0a0 0x7f69d41f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.951+0000 7f69d3fff640 1 -- 172.21.15.53:0/2145941640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d40cc510 con 0x7f69c0004d00 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d3fff640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f69c40072f0 tx=0x7f69c40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400f990 con 0x7f69c0004d00 2023-10-21T16:03:18.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c400faf0 con 0x7f69c0004d00 2023-10-21T16:03:18.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400fea0 con 0x7f69c0004d00 2023-10-21T16:03:18.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d40cc7f0 con 0x7f69c0004d00 2023-10-21T16:03:18.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.952+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d40ccdb0 con 0x7f69c0004d00 2023-10-21T16:03:18.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.953+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d40ccf40 con 0x7f69c0004d00 2023-10-21T16:03:18.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.954+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69c401d460 con 0x7f69c0004d00 2023-10-21T16:03:18.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.955+0000 7f69d17fa640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 0x7f69b406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:18.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.955+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f69c4089be0 con 0x7f69c0004d00 2023-10-21T16:03:18.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.956+0000 7f69cbfff640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 0x7f69b406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:18.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.957+0000 7f69cbfff640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 0x7f69b406d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69d40c1c20 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:18.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:18.960+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69c40548c0 con 0x7f69c0004d00 2023-10-21T16:03:19.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.154+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69d4074150 con 0x7f69c0004d00 2023-10-21T16:03:19.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.155+0000 7f69d17fa640 1 -- 172.21.15.53:0/2145941640 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69c4057ee0 con 0x7f69c0004d00 2023-10-21T16:03:19.157 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:19.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 msgr2=0x7f69b406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 0x7f69b406d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f69d40c1c20 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2023-10-21T16:03:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 msgr2=0x7f69d41f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f69c40072f0 tx=0x7f69c40077b0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 shutdown_connections 2023-10-21T16:03:19.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d40ce0a0 0x7f69d41f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69d4158b50 0x7f69d40cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f69b406aea0 0x7f69b406d360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 --2- 172.21.15.53:0/2145941640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c0004d00 0x7f69d41f2b70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 >> 172.21.15.53:0/2145941640 conn(0x7f69d406c050 msgr2=0x7f69d415cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:19.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 shutdown_connections 2023-10-21T16:03:19.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.158+0000 7f69d9467640 1 -- 172.21.15.53:0/2145941640 wait complete. 2023-10-21T16:03:19.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:03:19.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.405+0000 7f19f727e640 1 Processor -- start 2023-10-21T16:03:19.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.405+0000 7f19f727e640 1 -- start start 2023-10-21T16:03:19.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f727e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f19f00cf2b0 0x7f19f00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f727e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f00cd520 con 0x7f19f00ce510 2023-10-21T16:03:19.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f727e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f00d1910 con 0x7f19f00cf690 2023-10-21T16:03:19.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f727e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f00ce080 con 0x7f19f00cf2b0 2023-10-21T16:03:19.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f6a7d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19f00ce510 0x7f19f015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55404/0 (socket says 172.21.15.53:55404) 2023-10-21T16:03:19.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.407+0000 7f19f6a7d640 1 -- 172.21.15.53:0/4013870302 learned_addr learned my addr 172.21.15.53:0/4013870302 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:19.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f627c640 1 --2- 172.21.15.53:0/4013870302 >> v2:172.21.15.124:3300/0 conn(0x7f19f00cf2b0 0x7f19f00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171851897 0 0) 0x7f19f00cd520 con 0x7f19f00ce510 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19dc003520 con 0x7f19f00ce510 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898433782 0 0) 0x7f19dc003520 con 0x7f19f00ce510 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 >> v2:172.21.15.124:3300/0 conn(0x7f19f00cf2b0 msgr2=0x7f19f00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 --2- 172.21.15.53:0/4013870302 >> v2:172.21.15.124:3300/0 conn(0x7f19f00cf2b0 0x7f19f00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 >> v1:172.21.15.124:6789/0 conn(0x7f19f00cf690 legacy=0x7f19f00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00cd750 con 0x7f19f00ce510 2023-10-21T16:03:19.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f19e80019f0 con 0x7f19f00ce510 2023-10-21T16:03:19.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 >> v1:172.21.15.53:6789/0 conn(0x7f19f00ce510 legacy=0x7f19f015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.408+0000 7f19f527a640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 0x7f19dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f527a640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc0047c0 0x7f19dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f527a640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19dc005180 0x7f19dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00ce080 con 0x7f19dc003970 2023-10-21T16:03:19.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc00c120 con 0x7f19dc0047c0 2023-10-21T16:03:19.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc00c2e0 con 0x7f19dc005180 2023-10-21T16:03:19.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 0x7f19dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f5a7b640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc0047c0 0x7f19dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 -- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19dc005180 msgr2=0x7f19dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:19.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19dc005180 0x7f19dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 -- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc0047c0 msgr2=0x7f19dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:19.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc0047c0 0x7f19dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f627c640 1 -- 172.21.15.53:0/4013870302 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc003520 con 0x7f19dc003970 2023-10-21T16:03:19.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.409+0000 7f19f5a7b640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc0047c0 0x7f19dc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:19.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.410+0000 7f19f6a7d640 1 -- 172.21.15.53:0/4013870302 reap_dead start 2023-10-21T16:03:19.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.411+0000 7f19f627c640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 0x7f19dc003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ac0 tx=0x7f19dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b961f55cb6f33d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.411+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02c6b0 con 0x7f19dc003970 2023-10-21T16:03:19.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.411+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec02c810 con 0x7f19dc003970 2023-10-21T16:03:19.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.411+0000 7f19f527a640 1 -- 172.21.15.53:0/4013870302 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec02cba0 con 0x7f19dc003970 2023-10-21T16:03:19.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 -- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 msgr2=0x7f19dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 0x7f19dc003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009ac0 tx=0x7f19dc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 -- 172.21.15.53:0/4013870302 shutdown_connections 2023-10-21T16:03:19.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 --2- 172.21.15.53:0/4013870302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19dc003970 0x7f19dc003d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 -- 172.21.15.53:0/4013870302 >> 172.21.15.53:0/4013870302 conn(0x7f19f009bc40 msgr2=0x7f19f00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:19.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 -- 172.21.15.53:0/4013870302 shutdown_connections 2023-10-21T16:03:19.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.412+0000 7f19f727e640 1 -- 172.21.15.53:0/4013870302 wait complete. 2023-10-21T16:03:19.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.414+0000 7f19f727e640 1 Processor -- start 2023-10-21T16:03:19.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.414+0000 7f19f727e640 1 -- start start 2023-10-21T16:03:19.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc003970 0x7f19f0151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 0x7f19f015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00d2540 con 0x7f19f0153f70 2023-10-21T16:03:19.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0152550 con 0x7f19dc003970 2023-10-21T16:03:19.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f727e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f015e8f0 con 0x7f19f00d2090 2023-10-21T16:03:19.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f5a7b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f5a7b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33046/0 (socket says 172.21.15.53:33046) 2023-10-21T16:03:19.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.415+0000 7f19f5a7b640 1 -- 172.21.15.53:0/3178548059 learned_addr learned my addr 172.21.15.53:0/3178548059 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:19.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f627c640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc003970 0x7f19f0151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f6a7d640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 0x7f19f015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 -- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 msgr2=0x7f19f015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 0x7f19f015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 -- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc003970 msgr2=0x7f19f0151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc003970 0x7f19f0151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 -- 172.21.15.53:0/3178548059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f015ebd0 con 0x7f19f0153f70 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f6a7d640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 0x7f19f015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f5a7b640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d7a0 tx=0x7f19e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0017070 con 0x7f19f0153f70 2023-10-21T16:03:19.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e0005070 con 0x7f19f0153f70 2023-10-21T16:03:19.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0005420 con 0x7f19f0153f70 2023-10-21T16:03:19.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.416+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f015eeb0 con 0x7f19f0153f70 2023-10-21T16:03:19.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.417+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f015e100 con 0x7f19f0153f70 2023-10-21T16:03:19.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.418+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0074150 con 0x7f19f0153f70 2023-10-21T16:03:19.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.419+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f19e0007030 con 0x7f19f0153f70 2023-10-21T16:03:19.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.419+0000 7f19e6ffd640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 0x7f19d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.420+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f19e0089280 con 0x7f19f0153f70 2023-10-21T16:03:19.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.420+0000 7f19f627c640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 0x7f19d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.421+0000 7f19f627c640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 0x7f19d406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19ec036000 tx=0x7f19ec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.424+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19e0053f60 con 0x7f19f0153f70 2023-10-21T16:03:19.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.623+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19f00d0070 con 0x7f19f0153f70 2023-10-21T16:03:19.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.624+0000 7f19e6ffd640 1 -- 172.21.15.53:0/3178548059 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19e0057580 con 0x7f19f0153f70 2023-10-21T16:03:19.625 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:03:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 msgr2=0x7f19d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 0x7f19d406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19ec036000 tx=0x7f19ec030000 comp rx=0 tx=0).stop 2023-10-21T16:03:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 msgr2=0x7f19f00d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d7a0 tx=0x7f19e000dc60 comp rx=0 tx=0).stop 2023-10-21T16:03:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 shutdown_connections 2023-10-21T16:03:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19f00d2090 0x7f19f015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f19d406ae70 0x7f19d406d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19f0153f70 0x7f19f00d1b50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 --2- 172.21.15.53:0/3178548059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19dc003970 0x7f19f0151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.626+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 >> 172.21.15.53:0/3178548059 conn(0x7f19f009bc40 msgr2=0x7f19f00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.627+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 shutdown_connections 2023-10-21T16:03:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.627+0000 7f19f727e640 1 -- 172.21.15.53:0/3178548059 wait complete. 2023-10-21T16:03:19.647 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-21T16:03:19.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.875+0000 7f6be884b640 1 Processor -- start 2023-10-21T16:03:19.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.875+0000 7f6be884b640 1 -- start start 2023-10-21T16:03:19.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be884b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6be406c3a0 0x7f6be40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be884b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6be406db60 0x7f6be406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be884b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be415b310 con 0x7f6be415aa00 2023-10-21T16:03:19.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be884b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be40ca230 con 0x7f6be406c3a0 2023-10-21T16:03:19.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be884b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be406d290 con 0x7f6be406db60 2023-10-21T16:03:19.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6be406c3a0 0x7f6be40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6be406c3a0 0x7f6be40c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33058/0 (socket says 172.21.15.53:33058) 2023-10-21T16:03:19.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6be415aa00 0x7f6be40b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55414/0 (socket says 172.21.15.53:55414) 2023-10-21T16:03:19.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.877+0000 7f6be259c640 1 -- 172.21.15.53:0/3657080810 learned_addr learned my addr 172.21.15.53:0/3657080810 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:19.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.878+0000 7f6be259c640 1 -- 172.21.15.53:0/3657080810 >> v2:172.21.15.124:3300/0 conn(0x7f6be406db60 msgr2=0x7f6be406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:19.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.878+0000 7f6be259c640 1 --2- 172.21.15.53:0/3657080810 >> v2:172.21.15.124:3300/0 conn(0x7f6be406db60 0x7f6be406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.878+0000 7f6be259c640 1 -- 172.21.15.53:0/3657080810 >> v1:172.21.15.53:6789/0 conn(0x7f6be415aa00 legacy=0x7f6be40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.878+0000 7f6be159a640 1 -- 172.21.15.53:0/3657080810 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764307064 0 0) 0x7f6bdc001490 con 0x7f6be415aa00 2023-10-21T16:03:19.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.878+0000 7f6be259c640 1 -- 172.21.15.53:0/3657080810 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40c9860 con 0x7f6be406c3a0 2023-10-21T16:03:19.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.879+0000 7f6be259c640 1 --2- 172.21.15.53:0/3657080810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be406c3a0 0x7f6be40c9cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6be40715c0 tx=0x7f6bd402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a4dc437e608aa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.879+0000 7f6be159a640 1 -- 172.21.15.53:0/3657080810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4030860 con 0x7f6be406c3a0 2023-10-21T16:03:19.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.879+0000 7f6be159a640 1 -- 172.21.15.53:0/3657080810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd40309c0 con 0x7f6be406c3a0 2023-10-21T16:03:19.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.879+0000 7f6be159a640 1 -- 172.21.15.53:0/3657080810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4030d50 con 0x7f6be406c3a0 2023-10-21T16:03:19.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 -- 172.21.15.53:0/3657080810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be406c3a0 msgr2=0x7f6be40c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 --2- 172.21.15.53:0/3657080810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be406c3a0 0x7f6be40c9cf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6be40715c0 tx=0x7f6bd402b9a0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 -- 172.21.15.53:0/3657080810 shutdown_connections 2023-10-21T16:03:19.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 --2- 172.21.15.53:0/3657080810 >> v2:172.21.15.124:3300/0 conn(0x7f6be406db60 0x7f6be406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 --2- 172.21.15.53:0/3657080810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be406c3a0 0x7f6be40c9cf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 -- 172.21.15.53:0/3657080810 >> 172.21.15.53:0/3657080810 conn(0x7f6be406a5d0 msgr2=0x7f6be40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:19.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 -- 172.21.15.53:0/3657080810 shutdown_connections 2023-10-21T16:03:19.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.880+0000 7f6be884b640 1 -- 172.21.15.53:0/3657080810 wait complete. 2023-10-21T16:03:19.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.882+0000 7f6be884b640 1 Processor -- start 2023-10-21T16:03:19.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.882+0000 7f6be884b640 1 -- start start 2023-10-21T16:03:19.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6be406c3a0 0x7f6be41f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 0x7f6be415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40c3e20 con 0x7f6be415aa00 2023-10-21T16:03:19.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41f2290 con 0x7f6be406db60 2023-10-21T16:03:19.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be884b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40c31a0 con 0x7f6be406c3a0 2023-10-21T16:03:19.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.883+0000 7f6be2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33072/0 (socket says 172.21.15.53:33072) 2023-10-21T16:03:19.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 -- 172.21.15.53:0/3022341549 learned_addr learned my addr 172.21.15.53:0/3022341549 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:19.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be259c640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6be406c3a0 0x7f6be41f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be1d9b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 0x7f6be415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 -- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6be406c3a0 msgr2=0x7f6be41f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6be406c3a0 0x7f6be41f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 -- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 msgr2=0x7f6be415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:19.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 0x7f6be415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:19.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 -- 172.21.15.53:0/3022341549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40094d0 con 0x7f6be415aa00 2023-10-21T16:03:19.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be1d9b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 0x7f6be415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6be2d9d640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c780 tx=0x7f6bdc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.884+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00ce90 con 0x7f6be415aa00 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.885+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00a640 con 0x7f6be415aa00 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.885+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be40c3480 con 0x7f6be415aa00 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.885+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00a9d0 con 0x7f6be415aa00 2023-10-21T16:03:19.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.885+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be40c3a40 con 0x7f6be415aa00 2023-10-21T16:03:19.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.886+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be40c3f60 con 0x7f6be415aa00 2023-10-21T16:03:19.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.887+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6bdc00ab30 con 0x7f6be415aa00 2023-10-21T16:03:19.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.888+0000 7f6bd2ffd640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:19.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.888+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6bdc088fa0 con 0x7f6be415aa00 2023-10-21T16:03:19.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.889+0000 7f6be259c640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:19.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.890+0000 7f6be259c640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 0x7f6bc406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4036000 tx=0x7f6bd4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:19.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:19.893+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6be40c3f60 con 0x7f6be415aa00 2023-10-21T16:03:20.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.085+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6be415b310 con 0x7f6be415aa00 2023-10-21T16:03:20.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.086+0000 7f6bd2ffd640 1 -- 172.21.15.53:0/3022341549 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6be415b310 con 0x7f6be415aa00 2023-10-21T16:03:20.088 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:03:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 msgr2=0x7f6bc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 0x7f6bc406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4036000 tx=0x7f6bd4008030 comp rx=0 tx=0).stop 2023-10-21T16:03:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 msgr2=0x7f6be415c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c780 tx=0x7f6bdc00cc40 comp rx=0 tx=0).stop 2023-10-21T16:03:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 shutdown_connections 2023-10-21T16:03:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f6bc406ae70 0x7f6bc406d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6be415aa00 0x7f6be415c450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6be406db60 0x7f6be415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 --2- 172.21.15.53:0/3022341549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6be406c3a0 0x7f6be41f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 >> 172.21.15.53:0/3022341549 conn(0x7f6be406a5d0 msgr2=0x7f6be40ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 shutdown_connections 2023-10-21T16:03:20.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.089+0000 7f6be884b640 1 -- 172.21.15.53:0/3022341549 wait complete. 2023-10-21T16:03:20.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:20.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.341+0000 7f38cce90640 1 Processor -- start 2023-10-21T16:03:20.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.341+0000 7f38cce90640 1 -- start start 2023-10-21T16:03:20.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38cce90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38c815cd90 0x7f38c80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38cce90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f38c8160940 0x7f38c8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38cce90640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c80c48f0 con 0x7f38c8070d50 2023-10-21T16:03:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38cce90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80c4d80 con 0x7f38c815cd90 2023-10-21T16:03:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38cce90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80c3ce0 con 0x7f38c8160940 2023-10-21T16:03:20.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38c815cd90 0x7f38c80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38c659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38c815cd90 0x7f38c80702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33084/0 (socket says 172.21.15.53:33084) 2023-10-21T16:03:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38c659c640 1 -- 172.21.15.53:0/4018734953 learned_addr learned my addr 172.21.15.53:0/4018734953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.343+0000 7f38c5d9b640 1 --2- 172.21.15.53:0/4018734953 >> v2:172.21.15.124:3300/0 conn(0x7f38c8160940 0x7f38c8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.344+0000 7f38c659c640 1 -- 172.21.15.53:0/4018734953 >> v2:172.21.15.124:3300/0 conn(0x7f38c8160940 msgr2=0x7f38c8070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:20.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.344+0000 7f38c659c640 1 --2- 172.21.15.53:0/4018734953 >> v2:172.21.15.124:3300/0 conn(0x7f38c8160940 0x7f38c8070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.344+0000 7f38c659c640 1 -- 172.21.15.53:0/4018734953 >> v1:172.21.15.124:6789/0 conn(0x7f38c8070d50 legacy=0x7f38c80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:20.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.344+0000 7f38c659c640 1 -- 172.21.15.53:0/4018734953 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80c3fc0 con 0x7f38c815cd90 2023-10-21T16:03:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.345+0000 7f38c659c640 1 --2- 172.21.15.53:0/4018734953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80702d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f38c80cc090 tx=0x7f38b802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3c54753cac0f53 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.345+0000 7f38c559a640 1 -- 172.21.15.53:0/4018734953 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b802ca80 con 0x7f38c815cd90 2023-10-21T16:03:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.345+0000 7f38c559a640 1 -- 172.21.15.53:0/4018734953 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b802cbe0 con 0x7f38c815cd90 2023-10-21T16:03:20.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.345+0000 7f38c559a640 1 -- 172.21.15.53:0/4018734953 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b80343f0 con 0x7f38c815cd90 2023-10-21T16:03:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 -- 172.21.15.53:0/4018734953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 msgr2=0x7f38c80702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 --2- 172.21.15.53:0/4018734953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80702d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f38c80cc090 tx=0x7f38b802bee0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 -- 172.21.15.53:0/4018734953 shutdown_connections 2023-10-21T16:03:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 --2- 172.21.15.53:0/4018734953 >> v2:172.21.15.124:3300/0 conn(0x7f38c8160940 0x7f38c8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 --2- 172.21.15.53:0/4018734953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80702d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 -- 172.21.15.53:0/4018734953 >> 172.21.15.53:0/4018734953 conn(0x7f38c806a380 msgr2=0x7f38c80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 -- 172.21.15.53:0/4018734953 shutdown_connections 2023-10-21T16:03:20.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.346+0000 7f38cce90640 1 -- 172.21.15.53:0/4018734953 wait complete. 2023-10-21T16:03:20.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.349+0000 7f38cce90640 1 Processor -- start 2023-10-21T16:03:20.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.349+0000 7f38cce90640 1 -- start start 2023-10-21T16:03:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38c8070d50 0x7f38c81514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38c8160940 0x7f38c80b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80b2300 con 0x7f38c815cd90 2023-10-21T16:03:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80cb1d0 con 0x7f38c8070d50 2023-10-21T16:03:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.350+0000 7f38cce90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815de30 con 0x7f38c8160940 2023-10-21T16:03:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33088/0 (socket says 172.21.15.53:33088) 2023-10-21T16:03:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 -- 172.21.15.53:0/3626604809 learned_addr learned my addr 172.21.15.53:0/3626604809 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c659c640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38c8070d50 0x7f38c81514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 -- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38c8160940 msgr2=0x7f38c80b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38c8160940 0x7f38c80b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 -- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38c8070d50 msgr2=0x7f38c81514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38c8070d50 0x7f38c81514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 -- 172.21.15.53:0/3626604809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8009b30 con 0x7f38c815cd90 2023-10-21T16:03:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.351+0000 7f38c5d9b640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38b000d800 tx=0x7f38b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.352+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b000b6c0 con 0x7f38c815cd90 2023-10-21T16:03:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.352+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b000b820 con 0x7f38c815cd90 2023-10-21T16:03:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.352+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b000bbb0 con 0x7f38c815cd90 2023-10-21T16:03:20.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.352+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c80b1a60 con 0x7f38c815cd90 2023-10-21T16:03:20.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.352+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c80b2020 con 0x7f38c815cd90 2023-10-21T16:03:20.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.353+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c80aec50 con 0x7f38c815cd90 2023-10-21T16:03:20.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.354+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38b000bd10 con 0x7f38c815cd90 2023-10-21T16:03:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.355+0000 7f38beffd640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 0x7f38a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.355+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f38b0089f80 con 0x7f38c815cd90 2023-10-21T16:03:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.355+0000 7f38c659c640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 0x7f38a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.357+0000 7f38c659c640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 0x7f38a806d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38b8035000 tx=0x7f38b8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.360+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38b0054c60 con 0x7f38c815cd90 2023-10-21T16:03:20.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.592+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38c8067490 con 0x7f38c815cd90 2023-10-21T16:03:20.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.593+0000 7f38beffd640 1 -- 172.21.15.53:0/3626604809 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f38b0058280 con 0x7f38c815cd90 2023-10-21T16:03:20.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:20.596 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:03:20.596 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:03:20.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 msgr2=0x7f38a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 0x7f38a806d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38b8035000 tx=0x7f38b8030000 comp rx=0 tx=0).stop 2023-10-21T16:03:20.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 msgr2=0x7f38c80b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38b000d800 tx=0x7f38b000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 shutdown_connections 2023-10-21T16:03:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38c8160940 0x7f38c80b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f38a806ae70 0x7f38a806d330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38c815cd90 0x7f38c80b13d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 --2- 172.21.15.53:0/3626604809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38c8070d50 0x7f38c81514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 >> 172.21.15.53:0/3626604809 conn(0x7f38c806a380 msgr2=0x7f38c80ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:20.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 shutdown_connections 2023-10-21T16:03:20.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.596+0000 7f38cce90640 1 -- 172.21.15.53:0/3626604809 wait complete. 2023-10-21T16:03:20.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.844+0000 7f9287fff640 1 Processor -- start 2023-10-21T16:03:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.844+0000 7f9287fff640 1 -- start start 2023-10-21T16:03:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9288161320 0x7f92880cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9288164f70 0x7f92880b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 0x7f92880b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288165470 con 0x7f9288161320 2023-10-21T16:03:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9288165750 con 0x7f9288164f70 2023-10-21T16:03:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9287fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9288165a30 con 0x7f928806c710 2023-10-21T16:03:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.846+0000 7f9286ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9288161320 0x7f92880cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9288161320 0x7f92880cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33090/0 (socket says 172.21.15.53:33090) 2023-10-21T16:03:20.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 -- 172.21.15.53:0/1876020098 learned_addr learned my addr 172.21.15.53:0/1876020098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:20.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f927ffff640 1 --2- 172.21.15.53:0/1876020098 >> v2:172.21.15.124:3300/0 conn(0x7f9288164f70 0x7f92880b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f92877fe640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 0x7f92880b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 -- 172.21.15.53:0/1876020098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 msgr2=0x7f92880b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 0x7f92880b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 -- 172.21.15.53:0/1876020098 >> v2:172.21.15.124:3300/0 conn(0x7f9288164f70 msgr2=0x7f92880b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 --2- 172.21.15.53:0/1876020098 >> v2:172.21.15.124:3300/0 conn(0x7f9288164f70 0x7f92880b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f9286ffd640 1 -- 172.21.15.53:0/1876020098 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288165d10 con 0x7f9288161320 2023-10-21T16:03:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.847+0000 7f92877fe640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 0x7f92880b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.848+0000 7f9286ffd640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9288161320 0x7f92880cc210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92880cbe60 tx=0x7f927802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fbe4f4f6fbf22f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.848+0000 7f92867fc640 1 -- 172.21.15.53:0/1876020098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278030860 con 0x7f9288161320 2023-10-21T16:03:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.848+0000 7f92867fc640 1 -- 172.21.15.53:0/1876020098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780309c0 con 0x7f9288161320 2023-10-21T16:03:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.848+0000 7f92867fc640 1 -- 172.21.15.53:0/1876020098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278030d50 con 0x7f9288161320 2023-10-21T16:03:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 -- 172.21.15.53:0/1876020098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9288161320 msgr2=0x7f92880cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9288161320 0x7f92880cc210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f92880cbe60 tx=0x7f927802b9a0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 -- 172.21.15.53:0/1876020098 shutdown_connections 2023-10-21T16:03:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f928806c710 0x7f92880b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 --2- 172.21.15.53:0/1876020098 >> v2:172.21.15.124:3300/0 conn(0x7f9288164f70 0x7f92880b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 --2- 172.21.15.53:0/1876020098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9288161320 0x7f92880cc210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 -- 172.21.15.53:0/1876020098 >> 172.21.15.53:0/1876020098 conn(0x7f92880cb0c0 msgr2=0x7f92880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 -- 172.21.15.53:0/1876020098 shutdown_connections 2023-10-21T16:03:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.849+0000 7f9287fff640 1 -- 172.21.15.53:0/1876020098 wait complete. 2023-10-21T16:03:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.851+0000 7f9287fff640 1 Processor -- start 2023-10-21T16:03:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.851+0000 7f9287fff640 1 -- start start 2023-10-21T16:03:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9288161320 0x7f92880c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9288164f70 0x7f92880c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880c7a50 con 0x7f928806c710 2023-10-21T16:03:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92880c7d30 con 0x7f9288161320 2023-10-21T16:03:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9287fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880c8010 con 0x7f9288164f70 2023-10-21T16:03:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9286ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9286ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33100/0 (socket says 172.21.15.53:33100) 2023-10-21T16:03:20.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.852+0000 7f9286ffd640 1 -- 172.21.15.53:0/3183484831 learned_addr learned my addr 172.21.15.53:0/3183484831 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f927ffff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9288161320 0x7f92880c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f92877fe640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9288164f70 0x7f92880c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 -- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9288164f70 msgr2=0x7f92880c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9288164f70 0x7f92880c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 -- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9288161320 msgr2=0x7f92880c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9288161320 0x7f92880c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 -- 172.21.15.53:0/3183484831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780094d0 con 0x7f928806c710 2023-10-21T16:03:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.853+0000 7f9286ffd640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f927802b9e0 tx=0x7f9278004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.854+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278039d60 con 0x7f928806c710 2023-10-21T16:03:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.854+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9278043400 con 0x7f928806c710 2023-10-21T16:03:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.854+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780437b0 con 0x7f928806c710 2023-10-21T16:03:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.854+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880c82f0 con 0x7f928806c710 2023-10-21T16:03:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.854+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92880c75b0 con 0x7f928806c710 2023-10-21T16:03:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.855+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928804e160 con 0x7f928806c710 2023-10-21T16:03:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.856+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9278043910 con 0x7f928806c710 2023-10-21T16:03:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.857+0000 7f927f7fe640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.857+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f927803e070 con 0x7f928806c710 2023-10-21T16:03:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.858+0000 7f927ffff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.859+0000 7f927ffff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 0x7f926806d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9270006fd0 tx=0x7f9270008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:20.862+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9278077a50 con 0x7f928806c710 2023-10-21T16:03:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.064+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f928806e3c0 con 0x7f928806c710 2023-10-21T16:03:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.066+0000 7f927f7fe640 1 -- 172.21.15.53:0/3183484831 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7f927807b070 con 0x7f928806c710 2023-10-21T16:03:21.068 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:21.068 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 msgr2=0x7f926806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 0x7f926806d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9270006fd0 tx=0x7f9270008040 comp rx=0 tx=0).stop 2023-10-21T16:03:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 msgr2=0x7f92880c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f927802b9e0 tx=0x7f9278004730 comp rx=0 tx=0).stop 2023-10-21T16:03:21.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 shutdown_connections 2023-10-21T16:03:21.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9288164f70 0x7f92880c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9288161320 0x7f92880c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 --2- 172.21.15.53:0/3183484831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f928806c710 0x7f92880c7070 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 >> 172.21.15.53:0/3183484831 conn(0x7f92880cb0c0 msgr2=0x7f92880701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:21.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 shutdown_connections 2023-10-21T16:03:21.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.069+0000 7f9287fff640 1 -- 172.21.15.53:0/3183484831 wait complete. 2023-10-21T16:03:21.089 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:03:21.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:21.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.313+0000 7f4a0759e640 1 Processor -- start 2023-10-21T16:03:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.313+0000 7f4a0759e640 1 -- start start 2023-10-21T16:03:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.314+0000 7f4a0759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8a0 0x7f4a0806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.314+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cf720 0x7f4a0806b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.314+0000 7f4a0759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a080cb590 con 0x7f4a0806bd90 2023-10-21T16:03:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a0759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0806c220 con 0x7f4a080cd8a0 2023-10-21T16:03:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a0759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080cfc90 con 0x7f4a080cf720 2023-10-21T16:03:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a0659c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8a0 0x7f4a0806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a05d9b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cf720 0x7f4a0806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a0659c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8a0 0x7f4a0806b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:48300/0 (socket says 172.21.15.53:48300) 2023-10-21T16:03:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.315+0000 7f4a0659c640 1 -- 172.21.15.53:0/2407527989 learned_addr learned my addr 172.21.15.53:0/2407527989 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.316+0000 7f4a0659c640 1 -- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cf720 msgr2=0x7f4a0806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.316+0000 7f4a0659c640 1 --2- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cf720 0x7f4a0806b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.316+0000 7f4a0659c640 1 -- 172.21.15.53:0/2407527989 >> v1:172.21.15.124:6789/0 conn(0x7f4a0806bd90 legacy=0x7f4a081608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.316+0000 7f4a0659c640 1 -- 172.21.15.53:0/2407527989 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080cff70 con 0x7f4a080cd8a0 2023-10-21T16:03:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.317+0000 7f4a0659c640 1 --2- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080cd8a0 0x7f4a0806b070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009870 tx=0x7f49f002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96651fcc38833c6d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.318+0000 7f4a0559a640 1 -- 172.21.15.53:0/2407527989 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0030840 con 0x7f4a080cd8a0 2023-10-21T16:03:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.318+0000 7f4a0559a640 1 -- 172.21.15.53:0/2407527989 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f00309a0 con 0x7f4a080cd8a0 2023-10-21T16:03:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.318+0000 7f4a0559a640 1 -- 172.21.15.53:0/2407527989 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0030d30 con 0x7f4a080cd8a0 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.318+0000 7f4a0759e640 1 -- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080cd8a0 msgr2=0x7f4a0806b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.318+0000 7f4a0759e640 1 --2- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080cd8a0 0x7f4a0806b070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009870 tx=0x7f49f002b980 comp rx=0 tx=0).stop 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 -- 172.21.15.53:0/2407527989 shutdown_connections 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 --2- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cf720 0x7f4a0806b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 --2- 172.21.15.53:0/2407527989 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080cd8a0 0x7f4a0806b070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 -- 172.21.15.53:0/2407527989 >> 172.21.15.53:0/2407527989 conn(0x7f4a08099b60 msgr2=0x7f4a080d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 -- 172.21.15.53:0/2407527989 shutdown_connections 2023-10-21T16:03:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.319+0000 7f4a0759e640 1 -- 172.21.15.53:0/2407527989 wait complete. 2023-10-21T16:03:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.321+0000 7f4a0759e640 1 Processor -- start 2023-10-21T16:03:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.321+0000 7f4a0759e640 1 -- start start 2023-10-21T16:03:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a0806bd90 0x7f4a081f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cd8a0 0x7f4a0806f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081512a0 con 0x7f4a080cf720 2023-10-21T16:03:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081f3310 con 0x7f4a0806bd90 2023-10-21T16:03:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08150970 con 0x7f4a080cd8a0 2023-10-21T16:03:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a06d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a06d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33104/0 (socket says 172.21.15.53:33104) 2023-10-21T16:03:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a06d9d640 1 -- 172.21.15.53:0/1572431032 learned_addr learned my addr 172.21.15.53:0/1572431032 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:21.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a0659c640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a0806bd90 0x7f4a081f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.322+0000 7f4a05d9b640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cd8a0 0x7f4a0806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 -- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cd8a0 msgr2=0x7f4a0806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cd8a0 0x7f4a0806f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 -- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a0806bd90 msgr2=0x7f4a081f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a0806bd90 0x7f4a081f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 -- 172.21.15.53:0/1572431032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00094d0 con 0x7f4a080cf720 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a06d9d640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c390 tx=0x7f4a0000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000052d0 con 0x7f4a080cf720 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00005430 con 0x7f4a080cf720 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003810 con 0x7f4a080cf720 2023-10-21T16:03:21.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.323+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a08150c50 con 0x7f4a080cf720 2023-10-21T16:03:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.324+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a08151730 con 0x7f4a080cf720 2023-10-21T16:03:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.325+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0804e160 con 0x7f4a080cf720 2023-10-21T16:03:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.326+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a00003970 con 0x7f4a080cf720 2023-10-21T16:03:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.327+0000 7f49feffd640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.327+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a00088eb0 con 0x7f4a080cf720 2023-10-21T16:03:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.327+0000 7f4a0659c640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.329+0000 7f4a0659c640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49f0036000 tx=0x7f49f0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.332+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a00053b90 con 0x7f4a080cf720 2023-10-21T16:03:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.564+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a08160350 con 0x7f4a080cf720 2023-10-21T16:03:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.565+0000 7f49feffd640 1 -- 172.21.15.53:0/1572431032 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7f4a00017090 con 0x7f4a080cf720 2023-10-21T16:03:21.567 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:21.567 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:03:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:03:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 msgr2=0x7f49e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49f0036000 tx=0x7f49f0008030 comp rx=0 tx=0).stop 2023-10-21T16:03:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 msgr2=0x7f4a0806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c390 tx=0x7f4a0000c850 comp rx=0 tx=0).stop 2023-10-21T16:03:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 shutdown_connections 2023-10-21T16:03:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080cf720 0x7f4a0806f9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a080cd8a0 0x7f4a0806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 --2- 172.21.15.53:0/1572431032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a0806bd90 0x7f4a081f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 >> 172.21.15.53:0/1572431032 conn(0x7f4a08099b60 msgr2=0x7f4a080b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 shutdown_connections 2023-10-21T16:03:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.568+0000 7f4a0759e640 1 -- 172.21.15.53:0/1572431032 wait complete. 2023-10-21T16:03:21.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.817+0000 7fb7cd24d640 1 Processor -- start 2023-10-21T16:03:21.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.817+0000 7fb7cd24d640 1 -- start start 2023-10-21T16:03:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cd24d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb7c8150440 0x7fb7c8071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cd24d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c80c97d0 0x7fb7c80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cd24d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c80c9f90 con 0x7fb7c815ce40 2023-10-21T16:03:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cd24d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c8071550 con 0x7fb7c80c97d0 2023-10-21T16:03:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cd24d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c80ca270 con 0x7fb7c8150440 2023-10-21T16:03:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c80c97d0 0x7fb7c80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c80c97d0 0x7fb7c80b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33110/0 (socket says 172.21.15.53:33110) 2023-10-21T16:03:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7cca4c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb7c815ce40 0x7fb7c815d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55424/0 (socket says 172.21.15.53:55424) 2023-10-21T16:03:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.819+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/3995146670 learned_addr learned my addr 172.21.15.53:0/3995146670 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7c7fff640 1 --2- 172.21.15.53:0/3995146670 >> v2:172.21.15.124:3300/0 conn(0x7fb7c8150440 0x7fb7c8071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/3995146670 >> v2:172.21.15.124:3300/0 conn(0x7fb7c8150440 msgr2=0x7fb7c8071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/3995146670 >> v2:172.21.15.124:3300/0 conn(0x7fb7c8150440 0x7fb7c8071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/3995146670 >> v1:172.21.15.53:6789/0 conn(0x7fb7c815ce40 legacy=0x7fb7c815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7c77fe640 1 -- 172.21.15.53:0/3995146670 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706140767 0 0) 0x7fb7c80c9f90 con 0x7fb7c815ce40 2023-10-21T16:03:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.820+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/3995146670 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80ca550 con 0x7fb7c80c97d0 2023-10-21T16:03:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.821+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/3995146670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c80c97d0 0x7fb7c80b0130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80cbe60 tx=0x7fb7b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aac7db25a3106e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7c77fe640 1 -- 172.21.15.53:0/3995146670 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8030860 con 0x7fb7c80c97d0 2023-10-21T16:03:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7c77fe640 1 -- 172.21.15.53:0/3995146670 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b80309c0 con 0x7fb7c80c97d0 2023-10-21T16:03:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7c77fe640 1 -- 172.21.15.53:0/3995146670 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8030d50 con 0x7fb7c80c97d0 2023-10-21T16:03:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3995146670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c80c97d0 msgr2=0x7fb7c80b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3995146670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c80c97d0 0x7fb7c80b0130 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80cbe60 tx=0x7fb7b802b9a0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3995146670 shutdown_connections 2023-10-21T16:03:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3995146670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c80c97d0 0x7fb7c80b0130 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3995146670 >> v2:172.21.15.124:3300/0 conn(0x7fb7c8150440 0x7fb7c8071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.822+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3995146670 >> 172.21.15.53:0/3995146670 conn(0x7fb7c80cb0c0 msgr2=0x7fb7c80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.823+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3995146670 shutdown_connections 2023-10-21T16:03:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.823+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3995146670 wait complete. 2023-10-21T16:03:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.824+0000 7fb7cd24d640 1 Processor -- start 2023-10-21T16:03:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.825+0000 7fb7cd24d640 1 -- start start 2023-10-21T16:03:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 0x7fb7c814fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c8150440 0x7fb7c814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80c2960 con 0x7fb7c815ce40 2023-10-21T16:03:21.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c814e0b0 con 0x7fb7c8150440 2023-10-21T16:03:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cd24d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c814e360 con 0x7fb7c80c97d0 2023-10-21T16:03:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cca4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cca4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33118/0 (socket says 172.21.15.53:33118) 2023-10-21T16:03:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7cca4c640 1 -- 172.21.15.53:0/3676661526 learned_addr learned my addr 172.21.15.53:0/3676661526 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c8150440 0x7fb7c814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.826+0000 7fb7c7fff640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 0x7fb7c814fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 -- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 msgr2=0x7fb7c814fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 0x7fb7c814fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 -- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c8150440 msgr2=0x7fb7c814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c8150440 0x7fb7c814d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 -- 172.21.15.53:0/3676661526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b80094d0 con 0x7fb7c815ce40 2023-10-21T16:03:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7c7fff640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 0x7fb7c814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cca4c640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c7a0 tx=0x7fb7c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000a640 con 0x7fb7c815ce40 2023-10-21T16:03:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c000a7a0 con 0x7fb7c815ce40 2023-10-21T16:03:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000ab90 con 0x7fb7c815ce40 2023-10-21T16:03:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.827+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c80c22a0 con 0x7fb7c815ce40 2023-10-21T16:03:21.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.828+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c81501f0 con 0x7fb7c815ce40 2023-10-21T16:03:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.829+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c80c2aa0 con 0x7fb7c815ce40 2023-10-21T16:03:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.830+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb7c000acf0 con 0x7fb7c815ce40 2023-10-21T16:03:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.831+0000 7fb7c57fa640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 0x7fb7a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.831+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb7c0088ea0 con 0x7fb7c815ce40 2023-10-21T16:03:21.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.832+0000 7fb7c7fff640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 0x7fb7a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:21.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.833+0000 7fb7c7fff640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 0x7fb7a806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:21.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:21.835+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7c80c2aa0 con 0x7fb7c815ce40 2023-10-21T16:03:22.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.067+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c80c2aa0 con 0x7fb7c815ce40 2023-10-21T16:03:22.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.068+0000 7fb7c57fa640 1 -- 172.21.15.53:0/3676661526 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4919 (secure 0 0 0) 0x7fb7c80c2aa0 con 0x7fb7c815ce40 2023-10-21T16:03:22.070 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:22.070 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/3066075322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3066075322},{"type":"v1","addr":"172.21.15.53:6834","nonce":3066075322}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1634499341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1634499341},{"type":"v1","addr":"172.21.15.53:6835","nonce":1634499341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34157,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4258945131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4258945131},{"type":"v1","addr":"172.21.15.124:6835","nonce":4258945131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:01:55.604470+0000","modified":"2023-10-21T16:02:22.488758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34163":{"gid":34163,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2621260973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2621260973},{"type":"v1","addr":"172.21.15.124:6837","nonce":2621260973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-21T16:03:22.071 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 9 2023-10-21T16:03:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 msgr2=0x7fb7a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 0x7fb7a806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0008040 comp rx=0 tx=0).stop 2023-10-21T16:03:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 msgr2=0x7fb7c814db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000c7a0 tx=0x7fb7c000cc60 comp rx=0 tx=0).stop 2023-10-21T16:03:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 shutdown_connections 2023-10-21T16:03:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fb7a806ae70 0x7fb7a806d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c815ce40 0x7fb7c814db70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c8150440 0x7fb7c814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 --2- 172.21.15.53:0/3676661526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c80c97d0 0x7fb7c814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 >> 172.21.15.53:0/3676661526 conn(0x7fb7c80cb0c0 msgr2=0x7fb7c80ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 shutdown_connections 2023-10-21T16:03:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.071+0000 7fb7cd24d640 1 -- 172.21.15.53:0/3676661526 wait complete. 2023-10-21T16:03:22.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:22.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.323+0000 7fa99280d640 1 Processor -- start 2023-10-21T16:03:22.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.323+0000 7fa99280d640 1 -- start start 2023-10-21T16:03:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.324+0000 7fa99280d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 0x7fa98c1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.324+0000 7fa99280d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa98c0cab40 0x7fa98c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99280d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 0x7fa98c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99280d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0c9020 con 0x7fa98c0cab40 2023-10-21T16:03:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99280d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0ccc20 con 0x7fa98c15d080 2023-10-21T16:03:22.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99280d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c92e0 con 0x7fa98c15cca0 2023-10-21T16:03:22.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99100a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa98c0cab40 0x7fa98c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99100a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa98c0cab40 0x7fa98c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33130/0 (socket says 172.21.15.53:33130) 2023-10-21T16:03:22.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99100a640 1 -- 172.21.15.53:0/3514669069 learned_addr learned my addr 172.21.15.53:0/3514669069 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:22.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99180b640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 0x7fa98c1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.325+0000 7fa99200c640 1 --2- 172.21.15.53:0/3514669069 >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 0x7fa98c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99100a640 1 -- 172.21.15.53:0/3514669069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 msgr2=0x7fa98c1614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:22.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99100a640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 0x7fa98c1614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99100a640 1 -- 172.21.15.53:0/3514669069 >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 msgr2=0x7fa98c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:22.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99100a640 1 --2- 172.21.15.53:0/3514669069 >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 0x7fa98c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99100a640 1 -- 172.21.15.53:0/3514669069 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0cc7b0 con 0x7fa98c0cab40 2023-10-21T16:03:22.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99180b640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 0x7fa98c1614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:22.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.326+0000 7fa99200c640 1 --2- 172.21.15.53:0/3514669069 >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 0x7fa98c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:22.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.327+0000 7fa99100a640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c0b00c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0adac0 tx=0x7fa97c02b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadb4651b8621d28 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.327+0000 7fa990809640 1 -- 172.21.15.53:0/3514669069 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c031760 con 0x7fa98c0cab40 2023-10-21T16:03:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.327+0000 7fa990809640 1 -- 172.21.15.53:0/3514669069 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c0318c0 con 0x7fa98c0cab40 2023-10-21T16:03:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.327+0000 7fa990809640 1 -- 172.21.15.53:0/3514669069 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c031c50 con 0x7fa98c0cab40 2023-10-21T16:03:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 -- 172.21.15.53:0/3514669069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 msgr2=0x7fa98c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c0b00c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0adac0 tx=0x7fa97c02b800 comp rx=0 tx=0).stop 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 -- 172.21.15.53:0/3514669069 shutdown_connections 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 --2- 172.21.15.53:0/3514669069 >> v2:172.21.15.124:3300/0 conn(0x7fa98c15d080 0x7fa98c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c0b00c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 --2- 172.21.15.53:0/3514669069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15cca0 0x7fa98c1614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 -- 172.21.15.53:0/3514669069 >> 172.21.15.53:0/3514669069 conn(0x7fa98c0cb0c0 msgr2=0x7fa98c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:22.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 -- 172.21.15.53:0/3514669069 shutdown_connections 2023-10-21T16:03:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.328+0000 7fa99280d640 1 -- 172.21.15.53:0/3514669069 wait complete. 2023-10-21T16:03:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.330+0000 7fa99280d640 1 Processor -- start 2023-10-21T16:03:22.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.330+0000 7fa99280d640 1 -- start start 2023-10-21T16:03:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 0x7fa98c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 0x7fa98c1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c89a0 con 0x7fa98c0cab40 2023-10-21T16:03:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1f5b80 con 0x7fa98c15cca0 2023-10-21T16:03:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99280d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1f5e60 con 0x7fa98c15d080 2023-10-21T16:03:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99180b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99180b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33132/0 (socket says 172.21.15.53:33132) 2023-10-21T16:03:22.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.331+0000 7fa99180b640 1 -- 172.21.15.53:0/2675377847 learned_addr learned my addr 172.21.15.53:0/2675377847 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:22.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99100a640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 0x7fa98c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99200c640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 0x7fa98c1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 -- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 msgr2=0x7fa98c1573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 0x7fa98c1573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 -- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 msgr2=0x7fa98c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 0x7fa98c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 -- 172.21.15.53:0/2675377847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c0094d0 con 0x7fa98c0cab40 2023-10-21T16:03:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99100a640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 0x7fa98c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99200c640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 0x7fa98c1573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99180b640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c7f0 tx=0x7fa98800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007820 con 0x7fa98c0cab40 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988007980 con 0x7fa98c0cab40 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007cf0 con 0x7fa98c0cab40 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.332+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c1f6140 con 0x7fa98c0cab40 2023-10-21T16:03:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.333+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c15d5e0 con 0x7fa98c0cab40 2023-10-21T16:03:22.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.334+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c04e160 con 0x7fa98c0cab40 2023-10-21T16:03:22.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.335+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa9880113f0 con 0x7fa98c0cab40 2023-10-21T16:03:22.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.335+0000 7fa9827fc640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 0x7fa97006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.336+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa988089d10 con 0x7fa98c0cab40 2023-10-21T16:03:22.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.336+0000 7fa99100a640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 0x7fa97006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.337+0000 7fa99100a640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 0x7fa97006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa97c036000 tx=0x7fa97c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.340+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa988054a00 con 0x7fa98c0cab40 2023-10-21T16:03:22.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.533+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa98c0b45c0 con 0x7fa98c0cab40 2023-10-21T16:03:22.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.534+0000 7fa9827fc640 1 -- 172.21.15.53:0/2675377847 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18661 (secure 0 0 0) 0x7fa98801f090 con 0x7fa98c0cab40 2023-10-21T16:03:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:22.537 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":28,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:02:42.990139+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:01:53.459159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-21T16:01:54.644921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-21T16:01:58.201944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 msgr2=0x7fa97006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 0x7fa97006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa97c036000 tx=0x7fa97c03d040 comp rx=0 tx=0).stop 2023-10-21T16:03:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 msgr2=0x7fa98c1f55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c7f0 tx=0x7fa98800ccb0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 shutdown_connections 2023-10-21T16:03:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa98c15d080 0x7fa98c1573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa98c15cca0 0x7fa98c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa97006ae70 0x7fa97006d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 --2- 172.21.15.53:0/2675377847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa98c0cab40 0x7fa98c1f55a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 >> 172.21.15.53:0/2675377847 conn(0x7fa98c0cb0c0 msgr2=0x7fa98c0ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 shutdown_connections 2023-10-21T16:03:22.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.537+0000 7fa99280d640 1 -- 172.21.15.53:0/2675377847 wait complete. 2023-10-21T16:03:22.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:03:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.789+0000 7feb16967640 1 Processor -- start 2023-10-21T16:03:22.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.790+0000 7feb16967640 1 -- start start 2023-10-21T16:03:22.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015a820 0x7feb100aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16967640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb1015ac00 0x7feb100b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16967640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100cab60 con 0x7feb100ca680 2023-10-21T16:03:22.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16967640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100cb960 con 0x7feb1015ac00 2023-10-21T16:03:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100cbdf0 con 0x7feb1015a820 2023-10-21T16:03:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.791+0000 7feb16166640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb1015ac00 0x7feb100b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb15164640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb100ca680 0x7feb100c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55432/0 (socket says 172.21.15.53:55432) 2023-10-21T16:03:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb15164640 1 -- 172.21.15.53:0/3437674025 learned_addr learned my addr 172.21.15.53:0/3437674025 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:22.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb16166640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb1015ac00 0x7feb100b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33142/0 (socket says 172.21.15.53:33142) 2023-10-21T16:03:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb15965640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015a820 0x7feb100aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb16166640 1 -- 172.21.15.53:0/3437674025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015a820 msgr2=0x7feb100aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb16166640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015a820 0x7feb100aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb16166640 1 -- 172.21.15.53:0/3437674025 >> v1:172.21.15.53:6789/0 conn(0x7feb100ca680 legacy=0x7feb100c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.792+0000 7feb16166640 1 -- 172.21.15.53:0/3437674025 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100cb030 con 0x7feb1015ac00 2023-10-21T16:03:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.793+0000 7feb16166640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1015ac00 0x7feb100b04b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb08009d90 tx=0x7feb0802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c5b69a2f98e094 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.793+0000 7feb14963640 1 -- 172.21.15.53:0/3437674025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0802cbd0 con 0x7feb1015ac00 2023-10-21T16:03:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb14963640 1 -- 172.21.15.53:0/3437674025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0802cd30 con 0x7feb1015ac00 2023-10-21T16:03:22.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb14963640 1 -- 172.21.15.53:0/3437674025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0802cea0 con 0x7feb1015ac00 2023-10-21T16:03:22.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 -- 172.21.15.53:0/3437674025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1015ac00 msgr2=0x7feb100b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1015ac00 0x7feb100b04b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feb08009d90 tx=0x7feb0802be20 comp rx=0 tx=0).stop 2023-10-21T16:03:22.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 -- 172.21.15.53:0/3437674025 shutdown_connections 2023-10-21T16:03:22.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1015ac00 0x7feb100b04b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 --2- 172.21.15.53:0/3437674025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015a820 0x7feb100aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 -- 172.21.15.53:0/3437674025 >> 172.21.15.53:0/3437674025 conn(0x7feb1006c050 msgr2=0x7feb1006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:22.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 -- 172.21.15.53:0/3437674025 shutdown_connections 2023-10-21T16:03:22.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.794+0000 7feb16967640 1 -- 172.21.15.53:0/3437674025 wait complete. 2023-10-21T16:03:22.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.797+0000 7feb16967640 1 Processor -- start 2023-10-21T16:03:22.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.797+0000 7feb16967640 1 -- start start 2023-10-21T16:03:22.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 0x7feb100ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015ac00 0x7feb100c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100acfb0 con 0x7feb100ca680 2023-10-21T16:03:22.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100ac3a0 con 0x7feb1015a820 2023-10-21T16:03:22.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb16967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100ac680 con 0x7feb1015ac00 2023-10-21T16:03:22.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.798+0000 7feb15965640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33148/0 (socket says 172.21.15.53:33148) 2023-10-21T16:03:22.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 -- 172.21.15.53:0/2040030377 learned_addr learned my addr 172.21.15.53:0/2040030377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:22.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15164640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 0x7feb100ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 -- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015ac00 msgr2=0x7feb100c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015ac00 0x7feb100c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 -- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 msgr2=0x7feb100ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:22.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 0x7feb100ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:22.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 -- 172.21.15.53:0/2040030377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08009a70 con 0x7feb100ca680 2023-10-21T16:03:22.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15164640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 0x7feb100ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:22.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.799+0000 7feb15965640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00d800 tx=0x7feb0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.800+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00b6c0 con 0x7feb100ca680 2023-10-21T16:03:22.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.800+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c00b820 con 0x7feb100ca680 2023-10-21T16:03:22.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.800+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00bbb0 con 0x7feb100ca680 2023-10-21T16:03:22.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.800+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb100ac960 con 0x7feb100ca680 2023-10-21T16:03:22.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.800+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb100cd2f0 con 0x7feb100ca680 2023-10-21T16:03:22.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.801+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb10074150 con 0x7feb100ca680 2023-10-21T16:03:22.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.802+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb0c00bd10 con 0x7feb100ca680 2023-10-21T16:03:22.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.803+0000 7feb067fc640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 0x7feaf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:22.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.803+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feb0c089f70 con 0x7feb100ca680 2023-10-21T16:03:22.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.804+0000 7feb15164640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 0x7feaf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:22.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.805+0000 7feb15164640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 0x7feaf406d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb100c1b40 tx=0x7feb00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:22.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:22.807+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb0c0bcca0 con 0x7feb100ca680 2023-10-21T16:03:23.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.041+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feb100ca280 con 0x7feb100ca680 2023-10-21T16:03:23.044 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:03:23.043+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:03:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.047+0000 7feb067fc640 1 -- 172.21.15.53:0/2040030377 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb0c054c60 con 0x7feb100ca680 2023-10-21T16:03:23.048 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:03:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.050+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 msgr2=0x7feaf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.050+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 0x7feaf406d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb100c1b40 tx=0x7feb00009040 comp rx=0 tx=0).stop 2023-10-21T16:03:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.050+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 msgr2=0x7feb101f19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:23.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.050+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00d800 tx=0x7feb0c00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 shutdown_connections 2023-10-21T16:03:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1015ac00 0x7feb100c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1015a820 0x7feb100ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7feaf406ae70 0x7feaf406d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 --2- 172.21.15.53:0/2040030377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb100ca680 0x7feb101f19f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.051+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 >> 172.21.15.53:0/2040030377 conn(0x7feb1006c050 msgr2=0x7feb10159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:23.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.052+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 shutdown_connections 2023-10-21T16:03:23.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.052+0000 7feb16967640 1 -- 172.21.15.53:0/2040030377 wait complete. 2023-10-21T16:03:23.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:03:23.085 INFO:tasks.ceph.mds.d.smithi124.stderr: -5444> 2023-10-21T16:01:59.338+0000 7fe02d27c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:03:23.169 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:03:23.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.311+0000 7fd361bf4640 1 Processor -- start 2023-10-21T16:03:23.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.311+0000 7fd361bf4640 1 -- start start 2023-10-21T16:03:23.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.312+0000 7fd361bf4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.312+0000 7fd361bf4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd35c0ce510 0x7fd35c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.312+0000 7fd361bf4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c0cd520 con 0x7fd35c0cf690 2023-10-21T16:03:23.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.312+0000 7fd361bf4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c0d1910 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.312+0000 7fd361bf4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c0ce080 con 0x7fd35c0ce510 2023-10-21T16:03:23.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33158/0 (socket says 172.21.15.53:33158) 2023-10-21T16:03:23.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 -- 172.21.15.53:0/1423220134 learned_addr learned my addr 172.21.15.53:0/1423220134 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd3613f3640 1 --2- 172.21.15.53:0/1423220134 >> v2:172.21.15.124:3300/0 conn(0x7fd35c0ce510 0x7fd35c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 -- 172.21.15.53:0/1423220134 >> v2:172.21.15.124:3300/0 conn(0x7fd35c0ce510 msgr2=0x7fd35c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 --2- 172.21.15.53:0/1423220134 >> v2:172.21.15.124:3300/0 conn(0x7fd35c0ce510 0x7fd35c15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 -- 172.21.15.53:0/1423220134 >> v1:172.21.15.124:6789/0 conn(0x7fd35c0cf690 legacy=0x7fd35c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:23.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.313+0000 7fd360bf2640 1 -- 172.21.15.53:0/1423220134 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0cd750 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.314+0000 7fd360bf2640 1 --2- 172.21.15.53:0/1423220134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0cfb20 tx=0x7fd35002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b27b6c46c0328df server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.315+0000 7fd35b7fe640 1 -- 172.21.15.53:0/1423220134 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3500317e0 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.315+0000 7fd35b7fe640 1 -- 172.21.15.53:0/1423220134 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350031940 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.315+0000 7fd35b7fe640 1 -- 172.21.15.53:0/1423220134 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350031cd0 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.315+0000 7fd361bf4640 1 -- 172.21.15.53:0/1423220134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf2b0 msgr2=0x7fd35c0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:23.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.315+0000 7fd361bf4640 1 --2- 172.21.15.53:0/1423220134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0cfb20 tx=0x7fd35002b880 comp rx=0 tx=0).stop 2023-10-21T16:03:23.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 -- 172.21.15.53:0/1423220134 shutdown_connections 2023-10-21T16:03:23.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 --2- 172.21.15.53:0/1423220134 >> v2:172.21.15.124:3300/0 conn(0x7fd35c0ce510 0x7fd35c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 --2- 172.21.15.53:0/1423220134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf2b0 0x7fd35c0d34d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 -- 172.21.15.53:0/1423220134 >> 172.21.15.53:0/1423220134 conn(0x7fd35c09bc40 msgr2=0x7fd35c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:23.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 -- 172.21.15.53:0/1423220134 shutdown_connections 2023-10-21T16:03:23.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.316+0000 7fd361bf4640 1 -- 172.21.15.53:0/1423220134 wait complete. 2023-10-21T16:03:23.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.318+0000 7fd361bf4640 1 Processor -- start 2023-10-21T16:03:23.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.318+0000 7fd361bf4640 1 -- start start 2023-10-21T16:03:23.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0ce510 0x7fd35c1f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0cf2b0 0x7fd35c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c1f6d50 con 0x7fd35c0cf690 2023-10-21T16:03:23.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c15aa00 con 0x7fd35c0ce510 2023-10-21T16:03:23.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd361bf4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c1f2730 con 0x7fd35c0cf2b0 2023-10-21T16:03:23.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd3613f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd3613f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33168/0 (socket says 172.21.15.53:33168) 2023-10-21T16:03:23.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd3613f3640 1 -- 172.21.15.53:0/2994274516 learned_addr learned my addr 172.21.15.53:0/2994274516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:23.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.319+0000 7fd360bf2640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0ce510 0x7fd35c1f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd35bfff640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0cf2b0 0x7fd35c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 -- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0cf2b0 msgr2=0x7fd35c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:23.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0cf2b0 0x7fd35c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 -- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0ce510 msgr2=0x7fd35c1f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:23.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0ce510 0x7fd35c1f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:23.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 -- 172.21.15.53:0/2994274516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3500094d0 con 0x7fd35c0cf690 2023-10-21T16:03:23.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3613f3640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd35400cdd0 tx=0x7fd35400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:23.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354003b90 con 0x7fd35c0cf690 2023-10-21T16:03:23.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354003cf0 con 0x7fd35c0cf690 2023-10-21T16:03:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354004240 con 0x7fd35c0cf690 2023-10-21T16:03:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c1f6a70 con 0x7fd35c0cf690 2023-10-21T16:03:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.320+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c15a690 con 0x7fd35c0cf690 2023-10-21T16:03:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.321+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c074150 con 0x7fd35c0cf690 2023-10-21T16:03:23.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.322+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd3540043a0 con 0x7fd35c0cf690 2023-10-21T16:03:23.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.324+0000 7fd3597fa640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 0x7fd34006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:23.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.324+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fd354012070 con 0x7fd35c0cf690 2023-10-21T16:03:23.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.325+0000 7fd360bf2640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 0x7fd34006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:23.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.325+0000 7fd360bf2640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 0x7fd34006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd350036000 tx=0x7fd35003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:23.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.327+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3540550c0 con 0x7fd35c0cf690 2023-10-21T16:03:23.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:23.559+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd35c15ab40 con 0x7fd35c0cf690 2023-10-21T16:03:24.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.113+0000 7fd3597fa640 1 -- 172.21.15.53:0/2994274516 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd3540586e0 con 0x7fd35c0cf690 2023-10-21T16:03:24.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 msgr2=0x7fd34006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 0x7fd34006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd350036000 tx=0x7fd35003d040 comp rx=0 tx=0).stop 2023-10-21T16:03:24.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 msgr2=0x7fd35c0cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd35400cdd0 tx=0x7fd35400a660 comp rx=0 tx=0).stop 2023-10-21T16:03:24.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 shutdown_connections 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fd34006ae70 0x7fd34006d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35c0cf690 0x7fd35c0cbf80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0cf2b0 0x7fd35c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 --2- 172.21.15.53:0/2994274516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0ce510 0x7fd35c1f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 >> 172.21.15.53:0/2994274516 conn(0x7fd35c09bc40 msgr2=0x7fd35c15bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:24.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 shutdown_connections 2023-10-21T16:03:24.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.117+0000 7fd361bf4640 1 -- 172.21.15.53:0/2994274516 wait complete. 2023-10-21T16:03:24.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:03:24.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.371+0000 7f53762ea640 1 Processor -- start 2023-10-21T16:03:24.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.371+0000 7f53762ea640 1 -- start start 2023-10-21T16:03:24.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f53762ea640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f537015cae0 0x7f53700cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f53762ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53700b0680 0x7f53700c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f53762ea640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53700ccbc0 con 0x7f53700c8b40 2023-10-21T16:03:24.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f53762ea640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700c9510 con 0x7f53700b0680 2023-10-21T16:03:24.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f53762ea640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700ce1f0 con 0x7f537015cae0 2023-10-21T16:03:24.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f5375ae9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53700b0680 0x7f53700c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f5374ae7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f53700c8b40 0x7f53700aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50560/0 (socket says 172.21.15.53:50560) 2023-10-21T16:03:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.373+0000 7f5374ae7640 1 -- 172.21.15.53:0/2655865911 learned_addr learned my addr 172.21.15.53:0/2655865911 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.374+0000 7f53752e8640 1 --2- 172.21.15.53:0/2655865911 >> v2:172.21.15.124:3300/0 conn(0x7f537015cae0 0x7f53700cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.374+0000 7f5375ae9640 1 -- 172.21.15.53:0/2655865911 >> v2:172.21.15.124:3300/0 conn(0x7f537015cae0 msgr2=0x7f53700cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:24.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.374+0000 7f5375ae9640 1 --2- 172.21.15.53:0/2655865911 >> v2:172.21.15.124:3300/0 conn(0x7f537015cae0 0x7f53700cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.374+0000 7f5375ae9640 1 -- 172.21.15.53:0/2655865911 >> v1:172.21.15.53:6789/0 conn(0x7f53700c8b40 legacy=0x7f53700aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.374+0000 7f5375ae9640 1 -- 172.21.15.53:0/2655865911 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700cd590 con 0x7f53700b0680 2023-10-21T16:03:24.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.375+0000 7f5375ae9640 1 --2- 172.21.15.53:0/2655865911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c8fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f53700caed0 tx=0x7f536802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce605854f53b80b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:24.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f5367fff640 1 -- 172.21.15.53:0/2655865911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368031830 con 0x7f53700b0680 2023-10-21T16:03:24.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f5367fff640 1 -- 172.21.15.53:0/2655865911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5368031990 con 0x7f53700b0680 2023-10-21T16:03:24.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f5367fff640 1 -- 172.21.15.53:0/2655865911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368031d20 con 0x7f53700b0680 2023-10-21T16:03:24.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 -- 172.21.15.53:0/2655865911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 msgr2=0x7f53700c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 --2- 172.21.15.53:0/2655865911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c8fd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f53700caed0 tx=0x7f536802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 -- 172.21.15.53:0/2655865911 shutdown_connections 2023-10-21T16:03:24.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 --2- 172.21.15.53:0/2655865911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c8fd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 --2- 172.21.15.53:0/2655865911 >> v2:172.21.15.124:3300/0 conn(0x7f537015cae0 0x7f53700cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.376+0000 7f53762ea640 1 -- 172.21.15.53:0/2655865911 >> 172.21.15.53:0/2655865911 conn(0x7f537006c050 msgr2=0x7f537006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:24.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.377+0000 7f53762ea640 1 -- 172.21.15.53:0/2655865911 shutdown_connections 2023-10-21T16:03:24.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.377+0000 7f53762ea640 1 -- 172.21.15.53:0/2655865911 wait complete. 2023-10-21T16:03:24.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.378+0000 7f53762ea640 1 Processor -- start 2023-10-21T16:03:24.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.378+0000 7f53762ea640 1 -- start start 2023-10-21T16:03:24.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53700c8b40 0x7f53700c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f537015cae0 0x7f53700c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c8470 con 0x7f53700b0680 2023-10-21T16:03:24.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c6860 con 0x7f53700c8b40 2023-10-21T16:03:24.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.379+0000 7f53762ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700c5c50 con 0x7f537015cae0 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50946/0 (socket says 172.21.15.53:50946) 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 -- 172.21.15.53:0/2359490086 learned_addr learned my addr 172.21.15.53:0/2359490086 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 -- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f537015cae0 msgr2=0x7f53700c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f537015cae0 0x7f53700c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 -- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53700c8b40 msgr2=0x7f53700c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:24.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53700c8b40 0x7f53700c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:24.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 -- 172.21.15.53:0/2359490086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009400 con 0x7f53700b0680 2023-10-21T16:03:24.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.380+0000 7f53752e8640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536000d5f0 tx=0x7f536000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:24.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.381+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53600125f0 con 0x7f53700b0680 2023-10-21T16:03:24.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.381+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5360012750 con 0x7f53700b0680 2023-10-21T16:03:24.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.381+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5360012b00 con 0x7f53700b0680 2023-10-21T16:03:24.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.381+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53700c5f30 con 0x7f53700b0680 2023-10-21T16:03:24.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.381+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53700c64f0 con 0x7f53700b0680 2023-10-21T16:03:24.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.382+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5370074150 con 0x7f53700b0680 2023-10-21T16:03:24.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.383+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5360012c60 con 0x7f53700b0680 2023-10-21T16:03:24.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.384+0000 7f5365ffb640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 0x7f535c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:24.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.384+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f5360089ed0 con 0x7f53700b0680 2023-10-21T16:03:24.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.385+0000 7f5374ae7640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 0x7f535c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:24.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.386+0000 7f5374ae7640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 0x7f535c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f537015d5c0 tx=0x7f536c001660 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:24.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.389+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5360054b60 con 0x7f53700b0680 2023-10-21T16:03:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:24.582+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53700cdab0 con 0x7f53700b0680 2023-10-21T16:03:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.123+0000 7f5365ffb640 1 -- 172.21.15.53:0/2359490086 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5360058180 con 0x7f53700b0680 2023-10-21T16:03:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:03:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 msgr2=0x7f535c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 0x7f535c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f537015d5c0 tx=0x7f536c001660 comp rx=0 tx=0).stop 2023-10-21T16:03:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 msgr2=0x7f53700c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536000d5f0 tx=0x7f536000dab0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 shutdown_connections 2023-10-21T16:03:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f537015cae0 0x7f53700c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53700c8b40 0x7f53700c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f535c06ae70 0x7f535c06d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 --2- 172.21.15.53:0/2359490086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53700b0680 0x7f53700c55c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 >> 172.21.15.53:0/2359490086 conn(0x7f537006c050 msgr2=0x7f53700c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.127+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 shutdown_connections 2023-10-21T16:03:25.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.128+0000 7f53762ea640 1 -- 172.21.15.53:0/2359490086 wait complete. 2023-10-21T16:03:25.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:03:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.381+0000 7fe8b8cf2640 1 Processor -- start 2023-10-21T16:03:25.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.381+0000 7fe8b8cf2640 1 -- start start 2023-10-21T16:03:25.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b8cf2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8b40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b8cf2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe8b40707f0 0x7fe8b40c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b8cf2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b40cb890 con 0x7fe8b40b21f0 2023-10-21T16:03:25.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b8cf2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b4070c80 con 0x7fe8b40707f0 2023-10-21T16:03:25.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b8cf2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b406c3a0 con 0x7fe8b40b25d0 2023-10-21T16:03:25.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe8b40b21f0 0x7fe8b40cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50570/0 (socket says 172.21.15.53:50570) 2023-10-21T16:03:25.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b259c640 1 -- 172.21.15.53:0/2735633376 learned_addr learned my addr 172.21.15.53:0/2735633376 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8a9d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8b40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.383+0000 7fe8b2d9d640 1 --2- 172.21.15.53:0/2735633376 >> v2:172.21.15.124:3300/0 conn(0x7fe8b40707f0 0x7fe8b40c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79781749 0 0) 0x7fe8b40cb890 con 0x7fe8b40b21f0 2023-10-21T16:03:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe894003520 con 0x7fe8b40b21f0 2023-10-21T16:03:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113355995 0 0) 0x7fe894003520 con 0x7fe8b40b21f0 2023-10-21T16:03:25.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 msgr2=0x7fe8b40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8b40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 >> v2:172.21.15.124:3300/0 conn(0x7fe8b40707f0 msgr2=0x7fe8b40c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 --2- 172.21.15.53:0/2735633376 >> v2:172.21.15.124:3300/0 conn(0x7fe8b40707f0 0x7fe8b40c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b406c640 con 0x7fe8b40b21f0 2023-10-21T16:03:25.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe8a4001670 con 0x7fe8b40b21f0 2023-10-21T16:03:25.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 >> v1:172.21.15.53:6789/0 conn(0x7fe8b40b21f0 legacy=0x7fe8b40cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe894003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe894004d00 0x7fe894005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4070c80 con 0x7fe894003970 2023-10-21T16:03:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b406c3a0 con 0x7fe894004d00 2023-10-21T16:03:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89400db90 con 0x7fe8b40b25d0 2023-10-21T16:03:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.384+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe894003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b2d9d640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe894004d00 0x7fe894005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8a9d9b640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b259c640 1 -- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 msgr2=0x7fe8940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:25.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40b25d0 0x7fe8940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b259c640 1 -- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe894004d00 msgr2=0x7fe894005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:25.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe894004d00 0x7fe894005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.385+0000 7fe8b259c640 1 -- 172.21.15.53:0/2735633376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894003520 con 0x7fe894003970 2023-10-21T16:03:25.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.386+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe894003d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c110 tx=0x7fe8940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a357c9a8a39438f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:25.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.386+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4005ad0 con 0x7fe894003970 2023-10-21T16:03:25.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.386+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4005c50 con 0x7fe894003970 2023-10-21T16:03:25.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.386+0000 7fe8b1d9b640 1 -- 172.21.15.53:0/2735633376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a402c830 con 0x7fe894003970 2023-10-21T16:03:25.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2735633376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 msgr2=0x7fe894003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2735633376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe894003d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400c110 tx=0x7fe8940057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b2d9d640 1 -- 172.21.15.53:0/2735633376 reap_dead start 2023-10-21T16:03:25.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2735633376 shutdown_connections 2023-10-21T16:03:25.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2735633376 >> 172.21.15.53:0/2735633376 conn(0x7fe8b406a5d0 msgr2=0x7fe8b40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:25.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2735633376 shutdown_connections 2023-10-21T16:03:25.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.387+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2735633376 wait complete. 2023-10-21T16:03:25.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.389+0000 7fe8b8cf2640 1 Processor -- start 2023-10-21T16:03:25.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.389+0000 7fe8b8cf2640 1 -- start start 2023-10-21T16:03:25.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40acf90 0x7fe8b40ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 0x7fe8b40cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40ad410 con 0x7fe894003970 2023-10-21T16:03:25.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40ad600 con 0x7fe8b4148240 2023-10-21T16:03:25.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b8cf2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40ad8e0 con 0x7fe8b40acf90 2023-10-21T16:03:25.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.390+0000 7fe8b259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50958/0 (socket says 172.21.15.53:50958) 2023-10-21T16:03:25.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 -- 172.21.15.53:0/2460772842 learned_addr learned my addr 172.21.15.53:0/2460772842 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8a9d9b640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40acf90 0x7fe8b40ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b2d9d640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 0x7fe8b40cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 -- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40acf90 msgr2=0x7fe8b40ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40acf90 0x7fe8b40ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 -- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 msgr2=0x7fe8b40cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 0x7fe8b40cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:25.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 -- 172.21.15.53:0/2460772842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40adbc0 con 0x7fe894003970 2023-10-21T16:03:25.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b2d9d640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 0x7fe8b40cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:25.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.391+0000 7fe8b259c640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402c3d0 tx=0x7fe8a4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:25.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.392+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4039070 con 0x7fe894003970 2023-10-21T16:03:25.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.392+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a402cc40 con 0x7fe894003970 2023-10-21T16:03:25.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.392+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a403c040 con 0x7fe894003970 2023-10-21T16:03:25.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.392+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b40adea0 con 0x7fe894003970 2023-10-21T16:03:25.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.392+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41486c0 con 0x7fe894003970 2023-10-21T16:03:25.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.393+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b404e160 con 0x7fe894003970 2023-10-21T16:03:25.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.394+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8a4032070 con 0x7fe894003970 2023-10-21T16:03:25.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.395+0000 7fe8ab7fe640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 0x7fe89806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:25.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.395+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fe8a40aabe0 con 0x7fe894003970 2023-10-21T16:03:25.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.396+0000 7fe8a9d9b640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 0x7fe89806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:25.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.397+0000 7fe8a9d9b640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 0x7fe89806d320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4071830 tx=0x7fe89c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:25.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.400+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8a4075b50 con 0x7fe894003970 2023-10-21T16:03:25.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:25.596+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe8b406d900 con 0x7fe894003970 2023-10-21T16:03:26.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.136+0000 7fe8ab7fe640 1 -- 172.21.15.53:0/2460772842 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe8a4079170 con 0x7fe894003970 2023-10-21T16:03:26.138 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:03:26.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.141+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 msgr2=0x7fe89806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.141+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 0x7fe89806d320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4071830 tx=0x7fe89c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:26.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.141+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 msgr2=0x7fe8b40c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.141+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe8a402c3d0 tx=0x7fe8a4001930 comp rx=0 tx=0).stop 2023-10-21T16:03:26.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 shutdown_connections 2023-10-21T16:03:26.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8b4148240 0x7fe8b40cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8b40acf90 0x7fe8b40ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe89806ae60 0x7fe89806d320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 --2- 172.21.15.53:0/2460772842 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe894003970 0x7fe8b40c4670 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 >> 172.21.15.53:0/2460772842 conn(0x7fe8b406a5d0 msgr2=0x7fe8b40b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:26.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.142+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 shutdown_connections 2023-10-21T16:03:26.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.143+0000 7fe8b8cf2640 1 -- 172.21.15.53:0/2460772842 wait complete. 2023-10-21T16:03:26.172 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-21T16:03:26.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.383+0000 7f920481a640 1 Processor -- start 2023-10-21T16:03:26.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.383+0000 7f920481a640 1 -- start start 2023-10-21T16:03:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.385+0000 7f920481a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f920015a820 0x7f92000aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.385+0000 7f920481a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92000cab60 con 0x7f92000ca680 2023-10-21T16:03:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.385+0000 7f920481a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92000cb960 con 0x7f920015ac00 2023-10-21T16:03:26.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.385+0000 7f920481a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000cbdf0 con 0x7f920015a820 2023-10-21T16:03:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.385+0000 7f91ff7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f920015a820 0x7f92000aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91f7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f92000ca680 0x7f92000c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50582/0 (socket says 172.21.15.53:50582) 2023-10-21T16:03:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91f7fff640 1 -- 172.21.15.53:0/3797530158 learned_addr learned my addr 172.21.15.53:0/3797530158 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:26.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91ff7fe640 1 -- 172.21.15.53:0/3797530158 >> v1:172.21.15.124:6789/0 conn(0x7f920015ac00 legacy=0x7f92000c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:26.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91ff7fe640 1 -- 172.21.15.53:0/3797530158 >> v1:172.21.15.53:6789/0 conn(0x7f92000ca680 legacy=0x7f92000c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91ff7fe640 1 -- 172.21.15.53:0/3797530158 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000c9f20 con 0x7f920015a820 2023-10-21T16:03:26.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.386+0000 7f91feffd640 1 -- 172.21.15.53:0/3797530158 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153953102 0 0) 0x7f92000cab60 con 0x7f92000ca680 2023-10-21T16:03:26.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.387+0000 7f91ff7fe640 1 --2- 172.21.15.53:0/3797530158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92000aff70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f91f00097a0 tx=0x7f91f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2680e9d618ecc4a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:26.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f91feffd640 1 -- 172.21.15.53:0/3797530158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f002e070 con 0x7f920015a820 2023-10-21T16:03:26.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f91feffd640 1 -- 172.21.15.53:0/3797530158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00315a0 con 0x7f920015a820 2023-10-21T16:03:26.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f91feffd640 1 -- 172.21.15.53:0/3797530158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0031930 con 0x7f920015a820 2023-10-21T16:03:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f920481a640 1 -- 172.21.15.53:0/3797530158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 msgr2=0x7f92000aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f920481a640 1 --2- 172.21.15.53:0/3797530158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92000aff70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f91f00097a0 tx=0x7f91f002b840 comp rx=0 tx=0).stop 2023-10-21T16:03:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f920481a640 1 -- 172.21.15.53:0/3797530158 shutdown_connections 2023-10-21T16:03:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f920481a640 1 --2- 172.21.15.53:0/3797530158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92000aff70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.388+0000 7f920481a640 1 -- 172.21.15.53:0/3797530158 >> 172.21.15.53:0/3797530158 conn(0x7f920006c050 msgr2=0x7f920006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.389+0000 7f920481a640 1 -- 172.21.15.53:0/3797530158 shutdown_connections 2023-10-21T16:03:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.389+0000 7f920481a640 1 -- 172.21.15.53:0/3797530158 wait complete. 2023-10-21T16:03:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.390+0000 7f920481a640 1 Processor -- start 2023-10-21T16:03:26.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.390+0000 7f920481a640 1 -- start start 2023-10-21T16:03:26.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92000ca680 0x7f92000c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:26.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:26.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 0x7f92000c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920015cca0 con 0x7f920015a820 2023-10-21T16:03:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000c8470 con 0x7f920015ac00 2023-10-21T16:03:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.391+0000 7f920481a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920015c690 con 0x7f92000ca680 2023-10-21T16:03:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50976/0 (socket says 172.21.15.53:50976) 2023-10-21T16:03:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 -- 172.21.15.53:0/3583627905 learned_addr learned my addr 172.21.15.53:0/3583627905 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91fffff640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 0x7f92000c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 -- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92000ca680 msgr2=0x7f92000c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92000ca680 0x7f92000c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 -- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 msgr2=0x7f92000c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 0x7f92000c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91f7fff640 1 -- 172.21.15.53:0/3583627905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0009400 con 0x7f920015a820 2023-10-21T16:03:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.392+0000 7f91fffff640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 0x7f92000c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f91f7fff640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c4b0 tx=0x7f91e800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e80074e0 con 0x7f920015a820 2023-10-21T16:03:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8007660 con 0x7f920015a820 2023-10-21T16:03:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8004a20 con 0x7f920015a820 2023-10-21T16:03:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92000c8190 con 0x7f920015a820 2023-10-21T16:03:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.393+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001e48a0 con 0x7f920015a820 2023-10-21T16:03:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.394+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200074150 con 0x7f920015a820 2023-10-21T16:03:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.395+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f91e8004b80 con 0x7f920015a820 2023-10-21T16:03:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.396+0000 7f91fcff9640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 0x7f91e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.396+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f91e8088730 con 0x7f920015a820 2023-10-21T16:03:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.397+0000 7f91ff7fe640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 0x7f91e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.398+0000 7f91ff7fe640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 0x7f91e006d360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f91f0036000 tx=0x7f91f0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.401+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91e80538e0 con 0x7f920015a820 2023-10-21T16:03:26.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:26.595+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92000ac4f0 con 0x7f920015a820 2023-10-21T16:03:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.148+0000 7f91fcff9640 1 -- 172.21.15.53:0/3583627905 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7f91e80180d0 con 0x7f920015a820 2023-10-21T16:03:27.151 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data_ec' removed 2023-10-21T16:03:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 msgr2=0x7f91e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 0x7f91e006d360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f91f0036000 tx=0x7f91f0030000 comp rx=0 tx=0).stop 2023-10-21T16:03:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 msgr2=0x7f92001e4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f91e800c4b0 tx=0x7f91e800c970 comp rx=0 tx=0).stop 2023-10-21T16:03:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 shutdown_connections 2023-10-21T16:03:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f920015ac00 0x7f92000c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f91e006aea0 0x7f91e006d360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f920015a820 0x7f92001e4360 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 --2- 172.21.15.53:0/3583627905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92000ca680 0x7f92000c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.152+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 >> 172.21.15.53:0/3583627905 conn(0x7f920006c050 msgr2=0x7f92001e7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.153+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 shutdown_connections 2023-10-21T16:03:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.153+0000 7f920481a640 1 -- 172.21.15.53:0/3583627905 wait complete. 2023-10-21T16:03:27.178 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:03:27.179 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:03:27.179 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:03:27.179 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:03:27.180 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:03:27.180 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:03:27.180 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:03:27.180 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:03:27.181 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:03:27.181 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:03:27.182 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:03:27.182 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:03:27.183 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:03:27.183 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:03:27.183 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:03:27.184 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:03:27.200 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:03:27.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:03:27.203 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:03:27.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:03:27.205 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:03:27.205 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:03:27.206 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:03:27.207 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:03:27.251 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:03:27.251 DEBUG:teuthology.parallel:result is None 2023-10-21T16:03:27.254 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:03:27.255 DEBUG:teuthology.parallel:result is None 2023-10-21T16:03:27.259 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:03:27.259 DEBUG:teuthology.parallel:result is None 2023-10-21T16:03:27.262 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:03:27.263 DEBUG:teuthology.parallel:result is None 2023-10-21T16:03:27.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:03:27.416 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:03:27.416 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:03:27.417 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:03:27.425 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:03:27.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.456+0000 7f03a6fb0640 1 Processor -- start 2023-10-21T16:03:27.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.457+0000 7f03a6fb0640 1 -- start start 2023-10-21T16:03:27.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f03a6fb0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f03a015cdb0 0x7f03a00cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f03a6fb0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a015d620 con 0x7f03a015d190 2023-10-21T16:03:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f03a6fb0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a006b6a0 con 0x7f03a00ccb40 2023-10-21T16:03:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f03a6fb0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00b19e0 con 0x7f03a015cdb0 2023-10-21T16:03:27.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f039ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f03a015d190 0x7f03a006af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50588/0 (socket says 172.21.15.53:50588) 2023-10-21T16:03:27.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.458+0000 7f039ffff640 1 -- 172.21.15.53:0/3958608157 learned_addr learned my addr 172.21.15.53:0/3958608157 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f03a4d4c640 1 --2- 172.21.15.53:0/3958608157 >> v2:172.21.15.124:3300/0 conn(0x7f03a015cdb0 0x7f03a00cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879810414 0 0) 0x7f03a015d620 con 0x7f03a015d190 2023-10-21T16:03:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0390003520 con 0x7f03a015d190 2023-10-21T16:03:27.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648878234 0 0) 0x7f03a006b6a0 con 0x7f03a00ccb40 2023-10-21T16:03:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a015d620 con 0x7f03a00ccb40 2023-10-21T16:03:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f03a4d4c640 1 -- 172.21.15.53:0/3958608157 >> v1:172.21.15.124:6789/0 conn(0x7f03a00ccb40 legacy=0x7f03a00ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f03a4d4c640 1 -- 172.21.15.53:0/3958608157 >> v1:172.21.15.53:6789/0 conn(0x7f03a015d190 legacy=0x7f03a006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.459+0000 7f03a4d4c640 1 -- 172.21.15.53:0/3958608157 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0069ef0 con 0x7f03a015cdb0 2023-10-21T16:03:27.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.460+0000 7f03a4d4c640 1 --2- 172.21.15.53:0/3958608157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a015cdb0 0x7f03a00cea30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03940097a0 tx=0x7f039402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d511b2aba88080 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:27.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.461+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394030610 con 0x7f03a015cdb0 2023-10-21T16:03:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.461+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394030770 con 0x7f03a015cdb0 2023-10-21T16:03:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.461+0000 7f039f7fe640 1 -- 172.21.15.53:0/3958608157 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394030b00 con 0x7f03a015cdb0 2023-10-21T16:03:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.461+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3958608157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a015cdb0 msgr2=0x7f03a00cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.461+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3958608157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a015cdb0 0x7f03a00cea30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f03940097a0 tx=0x7f039402b960 comp rx=0 tx=0).stop 2023-10-21T16:03:27.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.462+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3958608157 shutdown_connections 2023-10-21T16:03:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.462+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3958608157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a015cdb0 0x7f03a00cea30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.462+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3958608157 >> 172.21.15.53:0/3958608157 conn(0x7f03a00b0af0 msgr2=0x7f03a00b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.462+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3958608157 shutdown_connections 2023-10-21T16:03:27.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.462+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3958608157 wait complete. 2023-10-21T16:03:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.463+0000 7f03a6fb0640 1 Processor -- start 2023-10-21T16:03:27.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.464+0000 7f03a6fb0640 1 -- start start 2023-10-21T16:03:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 0x7f03a01f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a015d190 0x7f03a006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a015c020 con 0x7f03a015cdb0 2023-10-21T16:03:27.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a015b410 con 0x7f03a00ccb40 2023-10-21T16:03:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a6fb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a015b6f0 con 0x7f03a015d190 2023-10-21T16:03:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f039ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a4d4c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 0x7f03a01f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f039ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51022/0 (socket says 172.21.15.53:51022) 2023-10-21T16:03:27.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f039ffff640 1 -- 172.21.15.53:0/3965560082 learned_addr learned my addr 172.21.15.53:0/3965560082 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.465+0000 7f03a554d640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a015d190 0x7f03a006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 -- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a015d190 msgr2=0x7f03a006dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a015d190 0x7f03a006dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 -- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 msgr2=0x7f03a01f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:27.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 0x7f03a01f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 -- 172.21.15.53:0/3965560082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394009400 con 0x7f03a015cdb0 2023-10-21T16:03:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f03a4d4c640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 0x7f03a01f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:03:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039ffff640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f038c00c410 tx=0x7f038c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c005ce0 con 0x7f03a015cdb0 2023-10-21T16:03:27.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c005e40 con 0x7f03a015cdb0 2023-10-21T16:03:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0072f0 con 0x7f03a015cdb0 2023-10-21T16:03:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a015b9d0 con 0x7f03a015cdb0 2023-10-21T16:03:27.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.466+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a01f24c0 con 0x7f03a015cdb0 2023-10-21T16:03:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.467+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a015c1d0 con 0x7f03a015cdb0 2023-10-21T16:03:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.469+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f038c007450 con 0x7f03a015cdb0 2023-10-21T16:03:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.470+0000 7f039d7fa640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 0x7f038406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.470+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f038c088c90 con 0x7f03a015cdb0 2023-10-21T16:03:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.470+0000 7f03a4d4c640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 0x7f038406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.471+0000 7f03a4d4c640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 0x7f038406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:27.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.473+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f038c0540e0 con 0x7f03a015cdb0 2023-10-21T16:03:27.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:27.678+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f03a004e160 con 0x7f03a015cdb0 2023-10-21T16:03:28.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.171+0000 7f039d7fa640 1 -- 172.21.15.53:0/3965560082 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f038c017080 con 0x7f03a015cdb0 2023-10-21T16:03:28.173 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:03:28.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.174+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 msgr2=0x7f038406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 0x7f038406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0394036000 tx=0x7f0394008030 comp rx=0 tx=0).stop 2023-10-21T16:03:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 msgr2=0x7f03a015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f038c00c410 tx=0x7f038c00c8d0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 shutdown_connections 2023-10-21T16:03:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a015d190 0x7f03a006dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f038406ae70 0x7f038406d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a015cdb0 0x7f03a015a9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 --2- 172.21.15.53:0/3965560082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a00ccb40 0x7f03a01f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 >> 172.21.15.53:0/3965560082 conn(0x7f03a00b0af0 msgr2=0x7f03a0160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 shutdown_connections 2023-10-21T16:03:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.175+0000 7f03a6fb0640 1 -- 172.21.15.53:0/3965560082 wait complete. 2023-10-21T16:03:28.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:03:28.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.415+0000 7fbdcca2e640 1 Processor -- start 2023-10-21T16:03:28.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.415+0000 7fbdcca2e640 1 -- start start 2023-10-21T16:03:28.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdcca2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc815ab30 0x7fbdc815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdcca2e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b07d0 0x7fbdc806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdcca2e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc815f7d0 con 0x7fbdc80adc70 2023-10-21T16:03:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdcca2e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc815f210 con 0x7fbdc80b07d0 2023-10-21T16:03:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdcca2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc815f4f0 con 0x7fbdc815ab30 2023-10-21T16:03:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.417+0000 7fbdc7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b07d0 0x7fbdc806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdc80b07d0 0x7fbdc806cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:33938/0 (socket says 172.21.15.53:33938) 2023-10-21T16:03:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 -- 172.21.15.53:0/2035501024 learned_addr learned my addr 172.21.15.53:0/2035501024 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc77fe640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc815ab30 0x7fbdc815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 -- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc815ab30 msgr2=0x7fbdc815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc815ab30 0x7fbdc815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 -- 172.21.15.53:0/2035501024 >> v1:172.21.15.124:6789/0 conn(0x7fbdc80adc70 legacy=0x7fbdc80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.418+0000 7fbdc7fff640 1 -- 172.21.15.53:0/2035501024 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc806c550 con 0x7fbdc80b07d0 2023-10-21T16:03:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.419+0000 7fbdc7fff640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b07d0 0x7fbdc806cd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009e50 tx=0x7fbdc002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bc2b17d074d9af server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdc67fc640 1 -- 172.21.15.53:0/2035501024 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc002cb00 con 0x7fbdc80b07d0 2023-10-21T16:03:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdc67fc640 1 -- 172.21.15.53:0/2035501024 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc002cc60 con 0x7fbdc80b07d0 2023-10-21T16:03:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdc67fc640 1 -- 172.21.15.53:0/2035501024 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00343f0 con 0x7fbdc80b07d0 2023-10-21T16:03:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 -- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b07d0 msgr2=0x7fbdc806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b07d0 0x7fbdc806cd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009e50 tx=0x7fbdc002bf60 comp rx=0 tx=0).stop 2023-10-21T16:03:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 -- 172.21.15.53:0/2035501024 shutdown_connections 2023-10-21T16:03:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc80b07d0 0x7fbdc806cd50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/2035501024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc815ab30 0x7fbdc815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 -- 172.21.15.53:0/2035501024 >> 172.21.15.53:0/2035501024 conn(0x7fbdc80cb0c0 msgr2=0x7fbdc80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 -- 172.21.15.53:0/2035501024 shutdown_connections 2023-10-21T16:03:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.420+0000 7fbdcca2e640 1 -- 172.21.15.53:0/2035501024 wait complete. 2023-10-21T16:03:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.422+0000 7fbdcca2e640 1 Processor -- start 2023-10-21T16:03:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.422+0000 7fbdcca2e640 1 -- start start 2023-10-21T16:03:28.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 0x7fbdc80af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc815ab30 0x7fbdc814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80cd9a0 con 0x7fbdc80b07d0 2023-10-21T16:03:28.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80ccd20 con 0x7fbdc815ab30 2023-10-21T16:03:28.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdcca2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80cd000 con 0x7fbdc80adc70 2023-10-21T16:03:28.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdc6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.423+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 0x7fbdc80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51032/0 (socket says 172.21.15.53:51032) 2023-10-21T16:03:28.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc6ffd640 1 -- 172.21.15.53:0/3198388169 learned_addr learned my addr 172.21.15.53:0/3198388169 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:28.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc7fff640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc815ab30 0x7fbdc814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 -- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc815ab30 msgr2=0x7fbdc814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc815ab30 0x7fbdc814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 -- 172.21.15.53:0/3198388169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 msgr2=0x7fbdc8155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 -- 172.21.15.53:0/3198388169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc0009b30 con 0x7fbdc80adc70 2023-10-21T16:03:28.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc6ffd640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:03:28.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.424+0000 7fbdc77fe640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 0x7fbdc80af160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e870 tx=0x7fbdb800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:28.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.425+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8009410 con 0x7fbdc80adc70 2023-10-21T16:03:28.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.425+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8009e40 con 0x7fbdc80adc70 2023-10-21T16:03:28.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.425+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb800cb80 con 0x7fbdc80adc70 2023-10-21T16:03:28.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.425+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc80cd2e0 con 0x7fbdc80adc70 2023-10-21T16:03:28.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.425+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc80af6a0 con 0x7fbdc80adc70 2023-10-21T16:03:28.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.426+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc804e160 con 0x7fbdc80adc70 2023-10-21T16:03:28.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.427+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbdb8004020 con 0x7fbdc80adc70 2023-10-21T16:03:28.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.428+0000 7fbdaffff640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 0x7fbdb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:28.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.428+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbdb8089160 con 0x7fbdc80adc70 2023-10-21T16:03:28.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.428+0000 7fbdc6ffd640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 0x7fbdb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:28.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.429+0000 7fbdc6ffd640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 0x7fbdb406d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80ac900 tx=0x7fbdbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:28.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.442+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdb8054540 con 0x7fbdc80adc70 2023-10-21T16:03:28.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.710+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbdc80cd140 con 0x7fbdc80adc70 2023-10-21T16:03:28.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.712+0000 7fbdaffff640 1 -- 172.21.15.53:0/3198388169 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fbdc80cd140 con 0x7fbdc80adc70 2023-10-21T16:03:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:28.714 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:03:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:03:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:03:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:03:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:03:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:03:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:03:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:03:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:03:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:03:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:03:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:03:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:03:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:03:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:03:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:03:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:03:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:03:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:03:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:03:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:03:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:03:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:03:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:03:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:03:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:03:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:03:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:03:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:03:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:03:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:03:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:03:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:03:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:03:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:03:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:03:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:03:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:03:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:03:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:03:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:03:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:03:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:03:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:03:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:03:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:03:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:03:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:03:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:03:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:03:28.746 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:03:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:03:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:03:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:03:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:03:28.748 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:03:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:03:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mds", 2023-10-21T16:03:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNl4MP8NhAAuXacP1tg4I6tr1yMZFPD1g==", 2023-10-21T16:03:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mds" 2023-10-21T16:03:28.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-mgr", 2023-10-21T16:03:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNl7t78NhAACID6pKc/F49eJm6dkrJYYw==", 2023-10-21T16:03:28.751 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-osd", 2023-10-21T16:03:28.752 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNlN/f8NhAAPdux/RjU0zdyzgS3CPQnFQ==", 2023-10-21T16:03:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-osd" 2023-10-21T16:03:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd", 2023-10-21T16:03:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNl8RT9NhAA9BNKSWTj/G3kl5sp3Ebd1g==", 2023-10-21T16:03:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-21T16:03:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-21T16:03:28.755 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNloS39NhAA9GXb0JvByCgK+h4/GqZUYw==", 2023-10-21T16:03:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-21T16:03:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.bootstrap-rgw", 2023-10-21T16:03:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDf9TNleUv9NhAAQS5EIoRzHPIhWCWc4ayOww==", 2023-10-21T16:03:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.757 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-21T16:03:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.758 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:03:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:03:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:03:28.760 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:03:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:03:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:03:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:03:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:28.762 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:03:28.762 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:03:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 msgr2=0x7fbdb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 0x7fbdb406d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80ac900 tx=0x7fbdbc009040 comp rx=0 tx=0).stop 2023-10-21T16:03:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 msgr2=0x7fbdc80af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 0x7fbdc80af160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e870 tx=0x7fbdb800ed30 comp rx=0 tx=0).stop 2023-10-21T16:03:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 shutdown_connections 2023-10-21T16:03:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdc815ab30 0x7fbdc814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fbdb406ae70 0x7fbdb406d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdc80b07d0 0x7fbdc8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 --2- 172.21.15.53:0/3198388169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdc80adc70 0x7fbdc80af160 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 >> 172.21.15.53:0/3198388169 conn(0x7fbdc80cb0c0 msgr2=0x7fbdc80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.714+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 shutdown_connections 2023-10-21T16:03:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.715+0000 7fbdcca2e640 1 -- 172.21.15.53:0/3198388169 wait complete. 2023-10-21T16:03:28.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-21T16:03:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.998+0000 7ff737fff640 1 Processor -- start 2023-10-21T16:03:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:28.998+0000 7ff737fff640 1 -- start start 2023-10-21T16:03:29.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff737fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7381599d0 0x7ff7380cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff737fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7380b3c50 0x7ff7380caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff737fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff738159db0 con 0x7ff7380cb410 2023-10-21T16:03:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff737fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff738071fe0 con 0x7ff7380b3c50 2023-10-21T16:03:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff737fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380b44c0 con 0x7ff7381599d0 2023-10-21T16:03:29.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff72e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7380b3c50 0x7ff7380caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff72e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7380b3c50 0x7ff7380caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51042/0 (socket says 172.21.15.53:51042) 2023-10-21T16:03:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff72e7fc640 1 -- 172.21.15.53:0/1234945811 learned_addr learned my addr 172.21.15.53:0/1234945811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:29.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff736ffd640 1 --2- 172.21.15.53:0/1234945811 >> v2:172.21.15.124:3300/0 conn(0x7ff7381599d0 0x7ff7380cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.000+0000 7ff72e7fc640 1 -- 172.21.15.53:0/1234945811 >> v2:172.21.15.124:3300/0 conn(0x7ff7381599d0 msgr2=0x7ff7380cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.001+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/1234945811 >> v2:172.21.15.124:3300/0 conn(0x7ff7381599d0 0x7ff7380cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.001+0000 7ff72e7fc640 1 -- 172.21.15.53:0/1234945811 >> v1:172.21.15.124:6789/0 conn(0x7ff7380cb410 legacy=0x7ff7380c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.001+0000 7ff72e7fc640 1 -- 172.21.15.53:0/1234945811 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380cb860 con 0x7ff7380b3c50 2023-10-21T16:03:29.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.002+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/1234945811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380caed0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff73806e150 tx=0x7ff72002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93f5a55978d3c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.002+0000 7ff7367fc640 1 -- 172.21.15.53:0/1234945811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72002ca80 con 0x7ff7380b3c50 2023-10-21T16:03:29.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.002+0000 7ff7367fc640 1 -- 172.21.15.53:0/1234945811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72002cbe0 con 0x7ff7380b3c50 2023-10-21T16:03:29.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.002+0000 7ff7367fc640 1 -- 172.21.15.53:0/1234945811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7200343f0 con 0x7ff7380b3c50 2023-10-21T16:03:29.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 -- 172.21.15.53:0/1234945811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 msgr2=0x7ff7380caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 --2- 172.21.15.53:0/1234945811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380caed0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff73806e150 tx=0x7ff72002bee0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 -- 172.21.15.53:0/1234945811 shutdown_connections 2023-10-21T16:03:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 --2- 172.21.15.53:0/1234945811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380caed0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 --2- 172.21.15.53:0/1234945811 >> v2:172.21.15.124:3300/0 conn(0x7ff7381599d0 0x7ff7380cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 -- 172.21.15.53:0/1234945811 >> 172.21.15.53:0/1234945811 conn(0x7ff73806c050 msgr2=0x7ff73806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 -- 172.21.15.53:0/1234945811 shutdown_connections 2023-10-21T16:03:29.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.003+0000 7ff737fff640 1 -- 172.21.15.53:0/1234945811 wait complete. 2023-10-21T16:03:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.005+0000 7ff737fff640 1 Processor -- start 2023-10-21T16:03:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.005+0000 7ff737fff640 1 -- start start 2023-10-21T16:03:29.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 0x7ff7380c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 0x7ff7380c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c9b20 con 0x7ff7380b3c50 2023-10-21T16:03:29.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380b39e0 con 0x7ff7380cb410 2023-10-21T16:03:29.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff737fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380b3140 con 0x7ff7381599d0 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51056/0 (socket says 172.21.15.53:51056) 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 -- 172.21.15.53:0/241811629 learned_addr learned my addr 172.21.15.53:0/241811629 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 0x7ff7380c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff7377fe640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 0x7ff7380c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 -- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 msgr2=0x7ff7380c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 0x7ff7380c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 -- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 msgr2=0x7ff7380c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.006+0000 7ff736ffd640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 0x7ff7380c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff736ffd640 1 -- 172.21.15.53:0/241811629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009b30 con 0x7ff7380b3c50 2023-10-21T16:03:29.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 0x7ff7380c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff7377fe640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 0x7ff7380c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff736ffd640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d620 tx=0x7ff72800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800b620 con 0x7ff7380b3c50 2023-10-21T16:03:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72800b780 con 0x7ff7380b3c50 2023-10-21T16:03:29.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800bb30 con 0x7ff7380b3c50 2023-10-21T16:03:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7380b3420 con 0x7ff7380b3c50 2023-10-21T16:03:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.007+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380b00f0 con 0x7ff7380b3c50 2023-10-21T16:03:29.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.009+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7380b0280 con 0x7ff7380b3c50 2023-10-21T16:03:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.010+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff72800bc90 con 0x7ff7380b3c50 2023-10-21T16:03:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.011+0000 7ff734ff9640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.011+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff728089860 con 0x7ff7380b3c50 2023-10-21T16:03:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.012+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.013+0000 7ff72e7fc640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 0x7ff71806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff720035000 tx=0x7ff720030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.015+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff728054b90 con 0x7ff7380b3c50 2023-10-21T16:03:29.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.284+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff738074150 con 0x7ff7380b3c50 2023-10-21T16:03:29.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.289+0000 7ff734ff9640 1 -- 172.21.15.53:0/241811629 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff7280581b0 con 0x7ff7380b3c50 2023-10-21T16:03:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 msgr2=0x7ff71806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 0x7ff71806d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff720035000 tx=0x7ff720030000 comp rx=0 tx=0).stop 2023-10-21T16:03:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 msgr2=0x7ff7380c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d620 tx=0x7ff72800dae0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 shutdown_connections 2023-10-21T16:03:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7381599d0 0x7ff7380c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7380cb410 0x7ff7380c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 --2- 172.21.15.53:0/241811629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7380b3c50 0x7ff7380c78d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 >> 172.21.15.53:0/241811629 conn(0x7ff73806c050 msgr2=0x7ff7380aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.292+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 shutdown_connections 2023-10-21T16:03:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.293+0000 7ff737fff640 1 -- 172.21.15.53:0/241811629 wait complete. 2023-10-21T16:03:29.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-21T16:03:29.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.523+0000 7fe8cc859640 1 Processor -- start 2023-10-21T16:03:29.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.524+0000 7fe8cc859640 1 -- start start 2023-10-21T16:03:29.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8c815a820 0x7fe8c80aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe8c80ca680 0x7fe8c80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c815ac00 0x7fe8c80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80cab60 con 0x7fe8c815a820 2023-10-21T16:03:29.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c80cb960 con 0x7fe8c80ca680 2023-10-21T16:03:29.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.525+0000 7fe8cc859640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80cbdf0 con 0x7fe8c815ac00 2023-10-21T16:03:29.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8c815a820 0x7fe8c80aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8c815a820 0x7fe8c80aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51072/0 (socket says 172.21.15.53:51072) 2023-10-21T16:03:29.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 -- 172.21.15.53:0/3600577156 learned_addr learned my addr 172.21.15.53:0/3600577156 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:29.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c7fff640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c815ac00 0x7fe8c80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c6ffd640 1 --2- 172.21.15.53:0/3600577156 >> v2:172.21.15.124:3300/0 conn(0x7fe8c80ca680 0x7fe8c80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 -- 172.21.15.53:0/3600577156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c815ac00 msgr2=0x7fe8c80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:29.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c815ac00 0x7fe8c80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 -- 172.21.15.53:0/3600577156 >> v2:172.21.15.124:3300/0 conn(0x7fe8c80ca680 msgr2=0x7fe8c80b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/3600577156 >> v2:172.21.15.124:3300/0 conn(0x7fe8c80ca680 0x7fe8c80b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.526+0000 7fe8c77fe640 1 -- 172.21.15.53:0/3600577156 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80cb030 con 0x7fe8c815a820 2023-10-21T16:03:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.528+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815a820 0x7fe8c80aff70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80718e0 tx=0x7fe8b802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a3115ff30af3d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.528+0000 7fe8c67fc640 1 -- 172.21.15.53:0/3600577156 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80317d0 con 0x7fe8c815a820 2023-10-21T16:03:29.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.528+0000 7fe8c67fc640 1 -- 172.21.15.53:0/3600577156 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8031930 con 0x7fe8c815a820 2023-10-21T16:03:29.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.528+0000 7fe8c67fc640 1 -- 172.21.15.53:0/3600577156 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8031cc0 con 0x7fe8c815a820 2023-10-21T16:03:29.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 -- 172.21.15.53:0/3600577156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815a820 msgr2=0x7fe8c80aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815a820 0x7fe8c80aff70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80718e0 tx=0x7fe8b802b870 comp rx=0 tx=0).stop 2023-10-21T16:03:29.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 -- 172.21.15.53:0/3600577156 shutdown_connections 2023-10-21T16:03:29.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c815ac00 0x7fe8c80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 --2- 172.21.15.53:0/3600577156 >> v2:172.21.15.124:3300/0 conn(0x7fe8c80ca680 0x7fe8c80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 --2- 172.21.15.53:0/3600577156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815a820 0x7fe8c80aff70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 -- 172.21.15.53:0/3600577156 >> 172.21.15.53:0/3600577156 conn(0x7fe8c806c050 msgr2=0x7fe8c806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 -- 172.21.15.53:0/3600577156 shutdown_connections 2023-10-21T16:03:29.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.529+0000 7fe8cc859640 1 -- 172.21.15.53:0/3600577156 wait complete. 2023-10-21T16:03:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.531+0000 7fe8cc859640 1 Processor -- start 2023-10-21T16:03:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.531+0000 7fe8cc859640 1 -- start start 2023-10-21T16:03:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c80ca680 0x7fe8c814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8c815a820 0x7fe8c80cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80af290 con 0x7fe8c815ac00 2023-10-21T16:03:29.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c815dd50 con 0x7fe8c815a820 2023-10-21T16:03:29.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8cc859640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c814f280 con 0x7fe8c80ca680 2023-10-21T16:03:29.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51080/0 (socket says 172.21.15.53:51080) 2023-10-21T16:03:29.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.532+0000 7fe8c7fff640 1 -- 172.21.15.53:0/2012568339 learned_addr learned my addr 172.21.15.53:0/2012568339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:29.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c80ca680 0x7fe8c814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c6ffd640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8c815a820 0x7fe8c80cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 -- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c80ca680 msgr2=0x7fe8c814ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c80ca680 0x7fe8c814ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 -- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8c815a820 msgr2=0x7fe8c80cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8c815a820 0x7fe8c80cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 -- 172.21.15.53:0/2012568339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80094d0 con 0x7fe8c815ac00 2023-10-21T16:03:29.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8c7fff640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007e90 tx=0x7fe8c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0014620 con 0x7fe8c815ac00 2023-10-21T16:03:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c0014780 con 0x7fe8c815ac00 2023-10-21T16:03:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0014b30 con 0x7fe8c815ac00 2023-10-21T16:03:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.533+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c80ae5a0 con 0x7fe8c815ac00 2023-10-21T16:03:29.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.534+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c80aeb60 con 0x7fe8c815ac00 2023-10-21T16:03:29.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.535+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c80af3d0 con 0x7fe8c815ac00 2023-10-21T16:03:29.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.536+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8c0014c90 con 0x7fe8c815ac00 2023-10-21T16:03:29.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.536+0000 7fe8affff640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 0x7fe8b406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:29.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.537+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe8c0089b10 con 0x7fe8c815ac00 2023-10-21T16:03:29.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.537+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 0x7fe8b406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:29.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.538+0000 7fe8c77fe640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 0x7fe8b406d3b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8036000 tx=0x7fe8b803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:29.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.541+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8c80af3d0 con 0x7fe8c815ac00 2023-10-21T16:03:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.807+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe8c80af3d0 con 0x7fe8c815ac00 2023-10-21T16:03:29.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.811+0000 7fe8affff640 1 -- 172.21.15.53:0/2012568339 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe8c80af3d0 con 0x7fe8c815ac00 2023-10-21T16:03:29.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 msgr2=0x7fe8b406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 0x7fe8b406d3b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8036000 tx=0x7fe8b803d040 comp rx=0 tx=0).stop 2023-10-21T16:03:29.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 msgr2=0x7fe8c80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:29.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0007e90 tx=0x7fe8c000e4b0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 shutdown_connections 2023-10-21T16:03:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fe8b406aef0 0x7fe8b406d3b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8c815ac00 0x7fe8c80cd7f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8c815a820 0x7fe8c80cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 --2- 172.21.15.53:0/2012568339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8c80ca680 0x7fe8c814ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 >> 172.21.15.53:0/2012568339 conn(0x7fe8c806c050 msgr2=0x7fe8c80cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 shutdown_connections 2023-10-21T16:03:29.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:29.815+0000 7fe8cc859640 1 -- 172.21.15.53:0/2012568339 wait complete. 2023-10-21T16:03:29.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-21T16:03:30.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.054+0000 7f948274e640 1 Processor -- start 2023-10-21T16:03:30.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.054+0000 7f948274e640 1 -- start start 2023-10-21T16:03:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948274e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f947c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948274e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c0d2870 con 0x7f947c15ccb0 2023-10-21T16:03:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948274e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c0d1150 con 0x7f947c0d00e0 2023-10-21T16:03:30.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948274e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0d1430 con 0x7f947c0d0b90 2023-10-21T16:03:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948174c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f947c15ccb0 0x7f947c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50598/0 (socket says 172.21.15.53:50598) 2023-10-21T16:03:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.055+0000 7f948174c640 1 -- 172.21.15.53:0/1049271353 learned_addr learned my addr 172.21.15.53:0/1049271353 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190714362 0 0) 0x7f947c0d2870 con 0x7f947c15ccb0 2023-10-21T16:03:30.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f9481f4d640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f947c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c003520 con 0x7f947c15ccb0 2023-10-21T16:03:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4158268059 0 0) 0x7f946c003520 con 0x7f947c15ccb0 2023-10-21T16:03:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 msgr2=0x7f947c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f947c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 >> v1:172.21.15.124:6789/0 conn(0x7f947c0d00e0 legacy=0x7f947c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0d1710 con 0x7f947c15ccb0 2023-10-21T16:03:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f94700019f0 con 0x7f947c15ccb0 2023-10-21T16:03:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 >> v1:172.21.15.53:6789/0 conn(0x7f947c15ccb0 legacy=0x7f947c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f946c0039a0 0x7f946c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f946c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f946c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0d1430 con 0x7f946c0047c0 2023-10-21T16:03:30.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c00bb40 con 0x7f946c0039a0 2023-10-21T16:03:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.056+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c00bd00 con 0x7f947c0d0b90 2023-10-21T16:03:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f946c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f948174c640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f946c0039a0 0x7f946c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9481f4d640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f946c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 -- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 msgr2=0x7f946c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0d0b90 0x7f946c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 -- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f946c0039a0 msgr2=0x7f946c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f946c0039a0 0x7f946c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.057+0000 7f9480f4b640 1 -- 172.21.15.53:0/1049271353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c003520 con 0x7f946c0047c0 2023-10-21T16:03:30.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.058+0000 7f9480f4b640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f946c004c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9468001750 tx=0x7f946c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70414cc9dd2f176 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.058+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680052d0 con 0x7f946c0047c0 2023-10-21T16:03:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.058+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946802e050 con 0x7f946c0047c0 2023-10-21T16:03:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.058+0000 7f947bfff640 1 -- 172.21.15.53:0/1049271353 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680035f0 con 0x7f946c0047c0 2023-10-21T16:03:30.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 -- 172.21.15.53:0/1049271353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 msgr2=0x7f946c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 --2- 172.21.15.53:0/1049271353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f946c004c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9468001750 tx=0x7f946c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f9481f4d640 1 -- 172.21.15.53:0/1049271353 reap_dead start 2023-10-21T16:03:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 -- 172.21.15.53:0/1049271353 shutdown_connections 2023-10-21T16:03:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 -- 172.21.15.53:0/1049271353 >> 172.21.15.53:0/1049271353 conn(0x7f947c071660 msgr2=0x7f947c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:30.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 -- 172.21.15.53:0/1049271353 shutdown_connections 2023-10-21T16:03:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.059+0000 7f948274e640 1 -- 172.21.15.53:0/1049271353 wait complete. 2023-10-21T16:03:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.061+0000 7f948274e640 1 Processor -- start 2023-10-21T16:03:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.061+0000 7f948274e640 1 -- start start 2023-10-21T16:03:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f947c1509c0 0x7f947c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0cc320 0x7f947c1f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cc7a0 con 0x7f946c0047c0 2023-10-21T16:03:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1f2f40 con 0x7f947c1509c0 2023-10-21T16:03:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948274e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1f2c60 con 0x7f947c0cc320 2023-10-21T16:03:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948174c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948174c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51096/0 (socket says 172.21.15.53:51096) 2023-10-21T16:03:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.062+0000 7f948174c640 1 -- 172.21.15.53:0/2619825930 learned_addr learned my addr 172.21.15.53:0/2619825930 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 -- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0cc320 msgr2=0x7f947c1f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0cc320 0x7f947c1f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 -- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f947c1509c0 msgr2=0x7f947c0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f947c1509c0 0x7f947c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 -- 172.21.15.53:0/2619825930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c15d1c0 con 0x7f946c0047c0 2023-10-21T16:03:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948174c640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f94700013d0 tx=0x7f947000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947000ee50 con 0x7f946c0047c0 2023-10-21T16:03:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470013420 con 0x7f946c0047c0 2023-10-21T16:03:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94700137d0 con 0x7f946c0047c0 2023-10-21T16:03:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.063+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c15d4a0 con 0x7f946c0047c0 2023-10-21T16:03:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.064+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c150e40 con 0x7f946c0047c0 2023-10-21T16:03:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.065+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c15d9c0 con 0x7f946c0047c0 2023-10-21T16:03:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.065+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9470013930 con 0x7f946c0047c0 2023-10-21T16:03:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.066+0000 7f9479ffb640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 0x7f946006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.066+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9470089b40 con 0x7f946c0047c0 2023-10-21T16:03:30.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.067+0000 7f9480f4b640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 0x7f946006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.068+0000 7f9480f4b640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 0x7f946006d360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9468033000 tx=0x7f946802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.071+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f947c15d9c0 con 0x7f946c0047c0 2023-10-21T16:03:30.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.336+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f947c15d9c0 con 0x7f946c0047c0 2023-10-21T16:03:30.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.342+0000 7f9479ffb640 1 -- 172.21.15.53:0/2619825930 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f947c15d9c0 con 0x7f946c0047c0 2023-10-21T16:03:30.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 msgr2=0x7f946006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 0x7f946006d360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9468033000 tx=0x7f946802f000 comp rx=0 tx=0).stop 2023-10-21T16:03:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 msgr2=0x7f947c15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f94700013d0 tx=0x7f947000e530 comp rx=0 tx=0).stop 2023-10-21T16:03:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 shutdown_connections 2023-10-21T16:03:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f947c0cc320 0x7f947c1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f947c1509c0 0x7f947c0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f946006aea0 0x7f946006d360 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 --2- 172.21.15.53:0/2619825930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f946c0047c0 0x7f947c15dca0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.344+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 >> 172.21.15.53:0/2619825930 conn(0x7f947c071660 msgr2=0x7f947c15c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.345+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 shutdown_connections 2023-10-21T16:03:30.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.345+0000 7f948274e640 1 -- 172.21.15.53:0/2619825930 wait complete. 2023-10-21T16:03:30.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-21T16:03:30.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.582+0000 7fcaaa2df640 1 Processor -- start 2023-10-21T16:03:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.582+0000 7fcaaa2df640 1 -- start start 2023-10-21T16:03:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.583+0000 7fcaaa2df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fcaa40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaaa2df640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcaa40b3c50 0x7fcaa40cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaaa2df640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa4159db0 con 0x7fcaa41599d0 2023-10-21T16:03:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaaa2df640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa4071fe0 con 0x7fcaa40b3c50 2023-10-21T16:03:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaaa2df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b44c0 con 0x7fcaa40cb5e0 2023-10-21T16:03:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaa92dd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcaa41599d0 0x7fcaa40caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50606/0 (socket says 172.21.15.53:50606) 2023-10-21T16:03:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaa92dd640 1 -- 172.21.15.53:0/3569322601 learned_addr learned my addr 172.21.15.53:0/3569322601 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaa9ade640 1 --2- 172.21.15.53:0/3569322601 >> v2:172.21.15.124:3300/0 conn(0x7fcaa40b3c50 0x7fcaa40cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fcaa8adc640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fcaa40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030557680 0 0) 0x7fcaa0001110 con 0x7fcaa41599d0 2023-10-21T16:03:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.584+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca90003520 con 0x7fcaa41599d0 2023-10-21T16:03:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990842192 0 0) 0x7fca90003520 con 0x7fcaa41599d0 2023-10-21T16:03:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 msgr2=0x7fcaa40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fcaa40cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 >> v2:172.21.15.124:3300/0 conn(0x7fcaa40b3c50 msgr2=0x7fcaa40cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 --2- 172.21.15.53:0/3569322601 >> v2:172.21.15.124:3300/0 conn(0x7fcaa40b3c50 0x7fcaa40cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40713d0 con 0x7fcaa41599d0 2023-10-21T16:03:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcaa0001800 con 0x7fcaa41599d0 2023-10-21T16:03:30.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 >> v1:172.21.15.53:6789/0 conn(0x7fcaa41599d0 legacy=0x7fcaa40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90003970 0x7fca90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca900047c0 0x7fca90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fca90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4071fe0 con 0x7fca90003970 2023-10-21T16:03:30.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b44c0 con 0x7fca900047c0 2023-10-21T16:03:30.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9000ba80 con 0x7fcaa40cb5e0 2023-10-21T16:03:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.585+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90003970 0x7fca90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa8adc640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca900047c0 0x7fca90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa92dd640 1 -- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 msgr2=0x7fca90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa9ade640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fca90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa40cb5e0 0x7fca90005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa92dd640 1 -- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca900047c0 msgr2=0x7fca90004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca900047c0 0x7fca90004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa92dd640 1 -- 172.21.15.53:0/3569322601 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca90003520 con 0x7fca90003970 2023-10-21T16:03:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.586+0000 7fcaa8adc640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca900047c0 0x7fca90004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.587+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90003970 0x7fca90003d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c0e0 tx=0x7fca900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccf5e927156ad0d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.587+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0005ad0 con 0x7fca90003970 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.587+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa0005c50 con 0x7fca90003970 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.587+0000 7fca9bfff640 1 -- 172.21.15.53:0/3569322601 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa002c7c0 con 0x7fca90003970 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3569322601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90003970 msgr2=0x7fca90003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3569322601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90003970 0x7fca90003d90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000c0e0 tx=0x7fca900057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaa9ade640 1 -- 172.21.15.53:0/3569322601 reap_dead start 2023-10-21T16:03:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3569322601 shutdown_connections 2023-10-21T16:03:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3569322601 >> 172.21.15.53:0/3569322601 conn(0x7fcaa406c050 msgr2=0x7fcaa406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3569322601 shutdown_connections 2023-10-21T16:03:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.588+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3569322601 wait complete. 2023-10-21T16:03:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.590+0000 7fcaaa2df640 1 Processor -- start 2023-10-21T16:03:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.590+0000 7fcaaa2df640 1 -- start start 2023-10-21T16:03:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 0x7fcaa414d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa414d860 0x7fcaa4154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b3800 con 0x7fcaa41548b0 2023-10-21T16:03:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa414dce0 con 0x7fca90003970 2023-10-21T16:03:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaaa2df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4155270 con 0x7fcaa414d860 2023-10-21T16:03:30.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51116/0 (socket says 172.21.15.53:51116) 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 -- 172.21.15.53:0/3607349620 learned_addr learned my addr 172.21.15.53:0/3607349620 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 0x7fcaa414d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa8adc640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa414d860 0x7fcaa4154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 -- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa414d860 msgr2=0x7fcaa4154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa414d860 0x7fcaa4154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 -- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 msgr2=0x7fcaa414d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 0x7fcaa414d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.591+0000 7fcaa9ade640 1 -- 172.21.15.53:0/3607349620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa414a660 con 0x7fcaa41548b0 2023-10-21T16:03:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 0x7fcaa414d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fcaa9ade640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4070700 tx=0x7fca9c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c007cf0 con 0x7fcaa41548b0 2023-10-21T16:03:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c007e70 con 0x7fcaa41548b0 2023-10-21T16:03:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c014620 con 0x7fcaa41548b0 2023-10-21T16:03:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa4155480 con 0x7fcaa41548b0 2023-10-21T16:03:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.592+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa4155a40 con 0x7fcaa41548b0 2023-10-21T16:03:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.593+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa414a810 con 0x7fcaa41548b0 2023-10-21T16:03:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.594+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca9c01d3b0 con 0x7fcaa41548b0 2023-10-21T16:03:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.595+0000 7fca99ffb640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 0x7fca8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.595+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fca9c0890f0 con 0x7fcaa41548b0 2023-10-21T16:03:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.596+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 0x7fca8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.597+0000 7fcaa92dd640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 0x7fca8806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034000 tx=0x7fcaa0007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.600+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcaa414a810 con 0x7fcaa41548b0 2023-10-21T16:03:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.860+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcaa414a810 con 0x7fcaa41548b0 2023-10-21T16:03:30.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.864+0000 7fca99ffb640 1 -- 172.21.15.53:0/3607349620 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcaa414a810 con 0x7fcaa41548b0 2023-10-21T16:03:30.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.866+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 msgr2=0x7fca8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 0x7fca8806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034000 tx=0x7fcaa0007000 comp rx=0 tx=0).stop 2023-10-21T16:03:30.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 msgr2=0x7fcaa4154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:30.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4070700 tx=0x7fca9c009cb0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 shutdown_connections 2023-10-21T16:03:30.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fca8806ae70 0x7fca8806d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa41548b0 0x7fcaa4154d30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa414d860 0x7fcaa4154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 --2- 172.21.15.53:0/3607349620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003970 0x7fcaa414d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:30.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 >> 172.21.15.53:0/3607349620 conn(0x7fcaa406c050 msgr2=0x7fcaa40af470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:30.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 shutdown_connections 2023-10-21T16:03:30.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:30.867+0000 7fcaaa2df640 1 -- 172.21.15.53:0/3607349620 wait complete. 2023-10-21T16:03:30.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-21T16:03:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.110+0000 7f13c9413640 1 Processor -- start 2023-10-21T16:03:31.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.111+0000 7f13c9413640 1 -- start start 2023-10-21T16:03:31.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.112+0000 7f13c9413640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 0x7f13c40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.112+0000 7f13c9413640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 0x7f13c40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.112+0000 7f13c9413640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c40cab60 con 0x7f13c40ca680 2023-10-21T16:03:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.112+0000 7f13c9413640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40cb960 con 0x7f13c415a820 2023-10-21T16:03:31.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.112+0000 7f13c9413640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cbdf0 con 0x7f13c415ac00 2023-10-21T16:03:31.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.113+0000 7f13c3fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 0x7f13c40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.113+0000 7f13c8c12640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 0x7f13c40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.113+0000 7f13bbfff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f13c40ca680 0x7f13c40c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:40104/0 (socket says 172.21.15.53:40104) 2023-10-21T16:03:31.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.113+0000 7f13bbfff640 1 -- 172.21.15.53:0/3009225832 learned_addr learned my addr 172.21.15.53:0/3009225832 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:31.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.113+0000 7f13c3fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 0x7f13c40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:34038/0 (socket says 172.21.15.53:34038) 2023-10-21T16:03:31.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.114+0000 7f13c8c12640 1 -- 172.21.15.53:0/3009225832 >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 msgr2=0x7f13c40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:31.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.114+0000 7f13c8c12640 1 --2- 172.21.15.53:0/3009225832 >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 0x7f13c40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.114+0000 7f13c8c12640 1 -- 172.21.15.53:0/3009225832 >> v1:172.21.15.124:6789/0 conn(0x7f13c40ca680 legacy=0x7f13c40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.114+0000 7f13c8c12640 1 -- 172.21.15.53:0/3009225832 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40cb030 con 0x7f13c415ac00 2023-10-21T16:03:31.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.115+0000 7f13c8c12640 1 --2- 172.21.15.53:0/3009225832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 0x7f13c40b04b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009d90 tx=0x7f13bc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1831c53c76515375 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.115+0000 7f13c37fe640 1 -- 172.21.15.53:0/3009225832 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc02cbc0 con 0x7f13c415ac00 2023-10-21T16:03:31.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.116+0000 7f13c37fe640 1 -- 172.21.15.53:0/3009225832 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc02cd20 con 0x7f13c415ac00 2023-10-21T16:03:31.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.116+0000 7f13c37fe640 1 -- 172.21.15.53:0/3009225832 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc034660 con 0x7f13c415ac00 2023-10-21T16:03:31.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.116+0000 7f13c9413640 1 -- 172.21.15.53:0/3009225832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 msgr2=0x7f13c40b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.116+0000 7f13c9413640 1 --2- 172.21.15.53:0/3009225832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 0x7f13c40b04b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009d90 tx=0x7f13bc02be10 comp rx=0 tx=0).stop 2023-10-21T16:03:31.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 -- 172.21.15.53:0/3009225832 shutdown_connections 2023-10-21T16:03:31.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 --2- 172.21.15.53:0/3009225832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415ac00 0x7f13c40b04b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 --2- 172.21.15.53:0/3009225832 >> v2:172.21.15.124:3300/0 conn(0x7f13c415a820 0x7f13c40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 -- 172.21.15.53:0/3009225832 >> 172.21.15.53:0/3009225832 conn(0x7f13c406c050 msgr2=0x7f13c406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:31.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 -- 172.21.15.53:0/3009225832 shutdown_connections 2023-10-21T16:03:31.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.117+0000 7f13c9413640 1 -- 172.21.15.53:0/3009225832 wait complete. 2023-10-21T16:03:31.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.119+0000 7f13c9413640 1 Processor -- start 2023-10-21T16:03:31.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.119+0000 7f13c9413640 1 -- start start 2023-10-21T16:03:31.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.120+0000 7f13c9413640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.120+0000 7f13c9413640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415a820 0x7f13c414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.120+0000 7f13c9413640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c415ac00 0x7f13c40c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c9413640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40ce0c0 con 0x7f13c40ca680 2023-10-21T16:03:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c9413640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cdb00 con 0x7f13c415ac00 2023-10-21T16:03:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c9413640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cdde0 con 0x7f13c415a820 2023-10-21T16:03:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51118/0 (socket says 172.21.15.53:51118) 2023-10-21T16:03:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 -- 172.21.15.53:0/3876712923 learned_addr learned my addr 172.21.15.53:0/3876712923 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13bbfff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415a820 0x7f13c414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 -- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415a820 msgr2=0x7f13c414c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415a820 0x7f13c414c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 -- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c415ac00 msgr2=0x7f13c40c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c8c12640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c415ac00 0x7f13c40c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c415ac00 0x7f13c40c3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.121+0000 7f13c3fff640 1 -- 172.21.15.53:0/3876712923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc009a70 con 0x7f13c40ca680 2023-10-21T16:03:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c3fff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e9b0 tx=0x7f13b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4009670 con 0x7f13c40ca680 2023-10-21T16:03:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b400c9a0 con 0x7f13c40ca680 2023-10-21T16:03:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c414d340 con 0x7f13c40ca680 2023-10-21T16:03:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b400cd50 con 0x7f13c40ca680 2023-10-21T16:03:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.122+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c40b3450 con 0x7f13c40ca680 2023-10-21T16:03:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.123+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4074150 con 0x7f13c40ca680 2023-10-21T16:03:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.124+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f13b4013470 con 0x7f13c40ca680 2023-10-21T16:03:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.125+0000 7f13c17fa640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 0x7f13a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.125+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f13b40897d0 con 0x7f13c40ca680 2023-10-21T16:03:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.126+0000 7f13bbfff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 0x7f13a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.127+0000 7f13bbfff640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 0x7f13a406d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13c415c380 tx=0x7f13ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.130+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13b4054970 con 0x7f13c40ca680 2023-10-21T16:03:31.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.395+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f13c415b2f0 con 0x7f13c40ca680 2023-10-21T16:03:31.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.401+0000 7f13c17fa640 1 -- 172.21.15.53:0/3876712923 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f13b4057f90 con 0x7f13c40ca680 2023-10-21T16:03:31.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 msgr2=0x7f13a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 0x7f13a406d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13c415c380 tx=0x7f13ac009040 comp rx=0 tx=0).stop 2023-10-21T16:03:31.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 msgr2=0x7f13c4153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e9b0 tx=0x7f13b400ee70 comp rx=0 tx=0).stop 2023-10-21T16:03:31.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 shutdown_connections 2023-10-21T16:03:31.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13c415ac00 0x7f13c40c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13c415a820 0x7f13c414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f13a406ae70 0x7f13a406d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 --2- 172.21.15.53:0/3876712923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13c40ca680 0x7f13c4153d30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 >> 172.21.15.53:0/3876712923 conn(0x7f13c406c050 msgr2=0x7f13c40cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:31.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 shutdown_connections 2023-10-21T16:03:31.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.403+0000 7f13c9413640 1 -- 172.21.15.53:0/3876712923 wait complete. 2023-10-21T16:03:31.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-21T16:03:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.631+0000 7fc2c0ad1640 1 Processor -- start 2023-10-21T16:03:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.631+0000 7fc2c0ad1640 1 -- start start 2023-10-21T16:03:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2bc06bcb0 0x7fc2bc160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 0x7fc2bc0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0ce9b0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0cec90 con 0x7fc2bc1612e0 2023-10-21T16:03:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.632+0000 7fc2c0ad1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0cef70 con 0x7fc2bc15ccd0 2023-10-21T16:03:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2bc06bcb0 0x7fc2bc160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc2bc06bcb0 0x7fc2bc160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51128/0 (socket says 172.21.15.53:51128) 2023-10-21T16:03:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 -- 172.21.15.53:0/2869752424 learned_addr learned my addr 172.21.15.53:0/2869752424 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2bbfff640 1 --2- 172.21.15.53:0/2869752424 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 0x7fc2bc0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 -- 172.21.15.53:0/2869752424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 msgr2=0x7fc2bc0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2bb7fe640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 -- 172.21.15.53:0/2869752424 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 msgr2=0x7fc2bc0b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 --2- 172.21.15.53:0/2869752424 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 0x7fc2bc0b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2baffd640 1 -- 172.21.15.53:0/2869752424 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc160880 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2bb7fe640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.633+0000 7fc2bbfff640 1 --2- 172.21.15.53:0/2869752424 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 0x7fc2bc0b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2baffd640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc160da0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0721e0 tx=0x7fc2b002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21efb52cf9010c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2ba7fc640 1 -- 172.21.15.53:0/2869752424 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0031860 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2ba7fc640 1 -- 172.21.15.53:0/2869752424 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00319c0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2ba7fc640 1 -- 172.21.15.53:0/2869752424 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0031d50 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/2869752424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 msgr2=0x7fc2bc160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.635+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc160da0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0721e0 tx=0x7fc2b002b900 comp rx=0 tx=0).stop 2023-10-21T16:03:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/2869752424 shutdown_connections 2023-10-21T16:03:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/2869752424 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc1612e0 0x7fc2bc0b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc160da0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/2869752424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/2869752424 >> 172.21.15.53:0/2869752424 conn(0x7fc2bc0cd180 msgr2=0x7fc2bc0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/2869752424 shutdown_connections 2023-10-21T16:03:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.636+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/2869752424 wait complete. 2023-10-21T16:03:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.637+0000 7fc2c0ad1640 1 Processor -- start 2023-10-21T16:03:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 -- start start 2023-10-21T16:03:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc1612e0 0x7fc2bc14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc1600c0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0b46a0 con 0x7fc2bc1612e0 2023-10-21T16:03:31.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.638+0000 7fc2c0ad1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0b4980 con 0x7fc2bc15ccd0 2023-10-21T16:03:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51136/0 (socket says 172.21.15.53:51136) 2023-10-21T16:03:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 -- 172.21.15.53:0/3103730120 learned_addr learned my addr 172.21.15.53:0/3103730120 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 -- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 msgr2=0x7fc2bc15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 -- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc1612e0 msgr2=0x7fc2bc14eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc1612e0 0x7fc2bc14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 -- 172.21.15.53:0/3103730120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00094d0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.639+0000 7fc2bb7fe640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c7f0 tx=0x7fc2ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.640+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac007a30 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.640+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac007b90 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.640+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00ce40 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.640+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc0b4c60 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.640+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc0b5220 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.641+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc074150 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.642+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2ac0113f0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.643+0000 7fc2a3fff640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.643+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc2ac0895e0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.643+0000 7fc2baffd640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:31.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.645+0000 7fc2baffd640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0036000 tx=0x7fc2b0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:31.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.647+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2ac054890 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.899+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc2bc0b5410 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.903+0000 7fc2a3fff640 1 -- 172.21.15.53:0/3103730120 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc2ac057eb0 con 0x7fc2bc06bcb0 2023-10-21T16:03:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 msgr2=0x7fc2a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0036000 tx=0x7fc2b0030000 comp rx=0 tx=0).stop 2023-10-21T16:03:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 msgr2=0x7fc2bc0b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c7f0 tx=0x7fc2ac00cbf0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 shutdown_connections 2023-10-21T16:03:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc1612e0 0x7fc2bc14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc15ccd0 0x7fc2bc15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fc2a806ae70 0x7fc2a806d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 --2- 172.21.15.53:0/3103730120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2bc06bcb0 0x7fc2bc0b3fc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:31.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 >> 172.21.15.53:0/3103730120 conn(0x7fc2bc0cd180 msgr2=0x7fc2bc0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:31.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 shutdown_connections 2023-10-21T16:03:31.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:31.906+0000 7fc2c0ad1640 1 -- 172.21.15.53:0/3103730120 wait complete. 2023-10-21T16:03:31.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.142+0000 7fa89aa21640 1 Processor -- start 2023-10-21T16:03:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.142+0000 7fa89aa21640 1 -- start start 2023-10-21T16:03:32.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa89aa21640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa89415ab40 0x7fa8940ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa89aa21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa8940adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa89aa21640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89415b2b0 con 0x7fa8940cae70 2023-10-21T16:03:32.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa89aa21640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89415bf30 con 0x7fa89415ab40 2023-10-21T16:03:32.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa89aa21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89415f7a0 con 0x7fa8940c90b0 2023-10-21T16:03:32.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa898fbe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa8940cae70 0x7fa8940702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50612/0 (socket says 172.21.15.53:50612) 2023-10-21T16:03:32.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.144+0000 7fa898fbe640 1 -- 172.21.15.53:0/760718714 learned_addr learned my addr 172.21.15.53:0/760718714 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:32.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa893fff640 1 --2- 172.21.15.53:0/760718714 >> v2:172.21.15.124:3300/0 conn(0x7fa89415ab40 0x7fa8940ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa8937fe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa8940adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737384285 0 0) 0x7fa89415b2b0 con 0x7fa8940cae70 2023-10-21T16:03:32.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c003520 con 0x7fa8940cae70 2023-10-21T16:03:32.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771456169 0 0) 0x7fa87c003520 con 0x7fa8940cae70 2023-10-21T16:03:32.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 msgr2=0x7fa8940adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:32.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa8940adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 >> v2:172.21.15.124:3300/0 conn(0x7fa89415ab40 msgr2=0x7fa8940ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:32.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 --2- 172.21.15.53:0/760718714 >> v2:172.21.15.124:3300/0 conn(0x7fa89415ab40 0x7fa8940ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89415b590 con 0x7fa8940cae70 2023-10-21T16:03:32.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa88c0019f0 con 0x7fa8940cae70 2023-10-21T16:03:32.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 >> v1:172.21.15.53:6789/0 conn(0x7fa8940cae70 legacy=0x7fa8940702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa87c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.145+0000 7fa892ffd640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa87c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89415bf30 con 0x7fa87c004d00 2023-10-21T16:03:32.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89415f7a0 con 0x7fa87c003970 2023-10-21T16:03:32.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c00b980 con 0x7fa8940c90b0 2023-10-21T16:03:32.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa87c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa8937fe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa87c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa893fff640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 -- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 msgr2=0x7fa87c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:32.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c90b0 0x7fa87c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 -- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa87c003970 msgr2=0x7fa87c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:32.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa87c003970 0x7fa87c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.146+0000 7fa898fbe640 1 -- 172.21.15.53:0/760718714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c003520 con 0x7fa87c004d00 2023-10-21T16:03:32.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.147+0000 7fa898fbe640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa87c005180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00a810 tx=0x7fa87c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb713464fca701a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c034630 con 0x7fa87c004d00 2023-10-21T16:03:32.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c02f070 con 0x7fa87c004d00 2023-10-21T16:03:32.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa892ffd640 1 -- 172.21.15.53:0/760718714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0349f0 con 0x7fa87c004d00 2023-10-21T16:03:32.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 -- 172.21.15.53:0/760718714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 msgr2=0x7fa87c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 --2- 172.21.15.53:0/760718714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa87c005180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00a810 tx=0x7fa87c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa898fbe640 1 -- 172.21.15.53:0/760718714 reap_dead start 2023-10-21T16:03:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 -- 172.21.15.53:0/760718714 shutdown_connections 2023-10-21T16:03:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 -- 172.21.15.53:0/760718714 >> 172.21.15.53:0/760718714 conn(0x7fa89406a380 msgr2=0x7fa8940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 -- 172.21.15.53:0/760718714 shutdown_connections 2023-10-21T16:03:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.148+0000 7fa89aa21640 1 -- 172.21.15.53:0/760718714 wait complete. 2023-10-21T16:03:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.150+0000 7fa89aa21640 1 Processor -- start 2023-10-21T16:03:32.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.150+0000 7fa89aa21640 1 -- start start 2023-10-21T16:03:32.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c66a0 0x7fa8940b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8940b3f20 0x7fa894151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940c6b20 con 0x7fa87c004d00 2023-10-21T16:03:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940b26b0 con 0x7fa8940b3f20 2023-10-21T16:03:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.151+0000 7fa89aa21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8940c6d30 con 0x7fa8940c66a0 2023-10-21T16:03:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51156/0 (socket says 172.21.15.53:51156) 2023-10-21T16:03:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 -- 172.21.15.53:0/2654624826 learned_addr learned my addr 172.21.15.53:0/2654624826 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 -- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c66a0 msgr2=0x7fa8940b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c66a0 0x7fa8940b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 -- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8940b3f20 msgr2=0x7fa894151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8940b3f20 0x7fa894151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 -- 172.21.15.53:0/2654624826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940c7010 con 0x7fa87c004d00 2023-10-21T16:03:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.152+0000 7fa893fff640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa8840108e0 tx=0x7fa884010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.153+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400eae0 con 0x7fa87c004d00 2023-10-21T16:03:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.153+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88400ec40 con 0x7fa87c004d00 2023-10-21T16:03:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.153+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88401c400 con 0x7fa87c004d00 2023-10-21T16:03:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.153+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8940c72f0 con 0x7fa87c004d00 2023-10-21T16:03:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.153+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8940c78b0 con 0x7fa87c004d00 2023-10-21T16:03:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.154+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa894067490 con 0x7fa87c004d00 2023-10-21T16:03:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.155+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa88401c560 con 0x7fa87c004d00 2023-10-21T16:03:32.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.156+0000 7fa890ff9640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 0x7fa88006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.156+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa884089a80 con 0x7fa87c004d00 2023-10-21T16:03:32.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.157+0000 7fa8937fe640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 0x7fa88006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.158+0000 7fa8937fe640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 0x7fa88006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa89406e210 tx=0x7fa888008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.160+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa884054d50 con 0x7fa87c004d00 2023-10-21T16:03:32.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.357+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8940b0ef0 con 0x7fa87c004d00 2023-10-21T16:03:32.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.359+0000 7fa890ff9640 1 -- 172.21.15.53:0/2654624826 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13050 (secure 0 0 0) 0x7fa884058370 con 0x7fa87c004d00 2023-10-21T16:03:32.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:32.362 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:03:28.186453+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 msgr2=0x7fa88006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 0x7fa88006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa89406e210 tx=0x7fa888008040 comp rx=0 tx=0).stop 2023-10-21T16:03:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 msgr2=0x7fa8940c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa8840108e0 tx=0x7fa884010da0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 shutdown_connections 2023-10-21T16:03:32.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8940b3f20 0x7fa894151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8940c66a0 0x7fa8940b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fa88006aea0 0x7fa88006d360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 --2- 172.21.15.53:0/2654624826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa87c004d00 0x7fa8940c7dd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 >> 172.21.15.53:0/2654624826 conn(0x7fa89406a380 msgr2=0x7fa8940cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:32.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 shutdown_connections 2023-10-21T16:03:32.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.362+0000 7fa89aa21640 1 -- 172.21.15.53:0/2654624826 wait complete. 2023-10-21T16:03:32.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:03:32.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:03:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.609+0000 7f85006f2640 1 Processor -- start 2023-10-21T16:03:32.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.609+0000 7f85006f2640 1 -- start start 2023-10-21T16:03:32.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.611+0000 7f85006f2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f84f815aad0 0x7f84f80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.611+0000 7f85006f2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f806d410 0x7f84f80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.611+0000 7f85006f2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f815bf30 con 0x7f84f80c82b0 2023-10-21T16:03:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.611+0000 7f85006f2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f815b320 con 0x7f84f815aad0 2023-10-21T16:03:32.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.611+0000 7f85006f2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f815b600 con 0x7f84f806d410 2023-10-21T16:03:32.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f84f815aad0 0x7f84f80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f84f815aad0 0x7f84f80b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:34102/0 (socket says 172.21.15.53:34102) 2023-10-21T16:03:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 -- 172.21.15.53:0/1436829454 learned_addr learned my addr 172.21.15.53:0/1436829454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fec8f640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f806d410 0x7f84f80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 -- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f806d410 msgr2=0x7f84f80c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f806d410 0x7f84f80c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 -- 172.21.15.53:0/1436829454 >> v1:172.21.15.124:6789/0 conn(0x7f84f80c82b0 legacy=0x7f84f806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.612+0000 7f84fe48e640 1 -- 172.21.15.53:0/1436829454 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f815b8e0 con 0x7f84f815aad0 2023-10-21T16:03:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.614+0000 7f84fe48e640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f815aad0 0x7f84f80b1750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009870 tx=0x7f84f402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a098efd8aef3d001 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.614+0000 7f84fd48c640 1 -- 172.21.15.53:0/1436829454 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4030840 con 0x7f84f815aad0 2023-10-21T16:03:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.614+0000 7f84fd48c640 1 -- 172.21.15.53:0/1436829454 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f40309a0 con 0x7f84f815aad0 2023-10-21T16:03:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.614+0000 7f84fd48c640 1 -- 172.21.15.53:0/1436829454 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4030d30 con 0x7f84f815aad0 2023-10-21T16:03:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 -- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f815aad0 msgr2=0x7f84f80b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f815aad0 0x7f84f80b1750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009870 tx=0x7f84f402b980 comp rx=0 tx=0).stop 2023-10-21T16:03:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 -- 172.21.15.53:0/1436829454 shutdown_connections 2023-10-21T16:03:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f806d410 0x7f84f80c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 --2- 172.21.15.53:0/1436829454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f815aad0 0x7f84f80b1750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 -- 172.21.15.53:0/1436829454 >> 172.21.15.53:0/1436829454 conn(0x7f84f8069d50 msgr2=0x7f84f806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 -- 172.21.15.53:0/1436829454 shutdown_connections 2023-10-21T16:03:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.615+0000 7f85006f2640 1 -- 172.21.15.53:0/1436829454 wait complete. 2023-10-21T16:03:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.617+0000 7f85006f2640 1 Processor -- start 2023-10-21T16:03:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.617+0000 7f85006f2640 1 -- start start 2023-10-21T16:03:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f806d410 0x7f84f8154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 0x7f84f81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f81f4330 con 0x7f84f815aad0 2023-10-21T16:03:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f81f4610 con 0x7f84f806d410 2023-10-21T16:03:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f85006f2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f8151ea0 con 0x7f84f80c82b0 2023-10-21T16:03:32.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f84fec8f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f84fec8f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51172/0 (socket says 172.21.15.53:51172) 2023-10-21T16:03:32.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.618+0000 7f84fec8f640 1 -- 172.21.15.53:0/466702167 learned_addr learned my addr 172.21.15.53:0/466702167 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:32.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fe48e640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f806d410 0x7f84f8154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fdc8d640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 0x7f84f81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 -- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 msgr2=0x7f84f81f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 0x7f84f81f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 -- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f806d410 msgr2=0x7f84f8154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f806d410 0x7f84f8154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 -- 172.21.15.53:0/466702167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f40094d0 con 0x7f84f815aad0 2023-10-21T16:03:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fdc8d640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 0x7f84f81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84fec8f640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84f000ef90 tx=0x7f84f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0009b20 con 0x7f84f815aad0 2023-10-21T16:03:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f000f040 con 0x7f84f815aad0 2023-10-21T16:03:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f00146c0 con 0x7f84f815aad0 2023-10-21T16:03:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.619+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f8152180 con 0x7f84f815aad0 2023-10-21T16:03:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.620+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f8155240 con 0x7f84f815aad0 2023-10-21T16:03:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.621+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f8151d00 con 0x7f84f815aad0 2023-10-21T16:03:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.622+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f84f0014820 con 0x7f84f815aad0 2023-10-21T16:03:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.622+0000 7f84eeffd640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 0x7f84dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:32.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.623+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f84f008a0a0 con 0x7f84f815aad0 2023-10-21T16:03:32.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.623+0000 7f84fe48e640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 0x7f84dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:32.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.624+0000 7f84fe48e640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 0x7f84dc06d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84f4036000 tx=0x7f84f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:32.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.627+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84f8151d00 con 0x7f84f815aad0 2023-10-21T16:03:32.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.821+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84f8155240 con 0x7f84f815aad0 2023-10-21T16:03:32.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.821+0000 7f84eeffd640 1 -- 172.21.15.53:0/466702167 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84f8155240 con 0x7f84f815aad0 2023-10-21T16:03:32.823 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:03:32.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 msgr2=0x7f84dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 0x7f84dc06d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f84f4036000 tx=0x7f84f4008030 comp rx=0 tx=0).stop 2023-10-21T16:03:32.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 msgr2=0x7f84f814c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:32.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84f000ef90 tx=0x7f84f000c490 comp rx=0 tx=0).stop 2023-10-21T16:03:32.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 shutdown_connections 2023-10-21T16:03:32.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f84dc06ae70 0x7f84dc06d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f815aad0 0x7f84f814c180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84f80c82b0 0x7f84f81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 --2- 172.21.15.53:0/466702167 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84f806d410 0x7f84f8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:32.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.824+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 >> 172.21.15.53:0/466702167 conn(0x7f84f8069d50 msgr2=0x7f84f80ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:32.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.825+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 shutdown_connections 2023-10-21T16:03:32.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:32.825+0000 7f85006f2640 1 -- 172.21.15.53:0/466702167 wait complete. 2023-10-21T16:03:32.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:03:33.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.067+0000 7f81648ba640 1 Processor -- start 2023-10-21T16:03:33.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.067+0000 7f81648ba640 1 -- start start 2023-10-21T16:03:33.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f81648ba640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f816015ab30 0x7f816015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f81648ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f81600b07d0 0x7f816006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f81648ba640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816015f7d0 con 0x7f81600adc70 2023-10-21T16:03:33.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f81648ba640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816015f210 con 0x7f81600b07d0 2023-10-21T16:03:33.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f81648ba640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816015f4f0 con 0x7f816015ab30 2023-10-21T16:03:33.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f815f59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f81600b07d0 0x7f816006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f815f59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f81600b07d0 0x7f816006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51178/0 (socket says 172.21.15.53:51178) 2023-10-21T16:03:33.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f815f59e640 1 -- 172.21.15.53:0/1840836721 learned_addr learned my addr 172.21.15.53:0/1840836721 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:33.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.069+0000 7f815ed9d640 1 --2- 172.21.15.53:0/1840836721 >> v2:172.21.15.124:3300/0 conn(0x7f816015ab30 0x7f816015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.070+0000 7f815f59e640 1 -- 172.21.15.53:0/1840836721 >> v2:172.21.15.124:3300/0 conn(0x7f816015ab30 msgr2=0x7f816015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.070+0000 7f815f59e640 1 --2- 172.21.15.53:0/1840836721 >> v2:172.21.15.124:3300/0 conn(0x7f816015ab30 0x7f816015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.070+0000 7f815f59e640 1 -- 172.21.15.53:0/1840836721 >> v1:172.21.15.124:6789/0 conn(0x7f81600adc70 legacy=0x7f81600b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:33.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.070+0000 7f815f59e640 1 -- 172.21.15.53:0/1840836721 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816006c550 con 0x7f81600b07d0 2023-10-21T16:03:33.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.071+0000 7f815f59e640 1 --2- 172.21.15.53:0/1840836721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81600b07d0 0x7f816006cd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f816006ada0 tx=0x7f815802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652f00baccf75571 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.071+0000 7f815e59c640 1 -- 172.21.15.53:0/1840836721 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815802ca80 con 0x7f81600b07d0 2023-10-21T16:03:33.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f815e59c640 1 -- 172.21.15.53:0/1840836721 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815802cbe0 con 0x7f81600b07d0 2023-10-21T16:03:33.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f815e59c640 1 -- 172.21.15.53:0/1840836721 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81580343f0 con 0x7f81600b07d0 2023-10-21T16:03:33.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 -- 172.21.15.53:0/1840836721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81600b07d0 msgr2=0x7f816006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 --2- 172.21.15.53:0/1840836721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81600b07d0 0x7f816006cd50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f816006ada0 tx=0x7f815802bee0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 -- 172.21.15.53:0/1840836721 shutdown_connections 2023-10-21T16:03:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 --2- 172.21.15.53:0/1840836721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81600b07d0 0x7f816006cd50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 --2- 172.21.15.53:0/1840836721 >> v2:172.21.15.124:3300/0 conn(0x7f816015ab30 0x7f816015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 -- 172.21.15.53:0/1840836721 >> 172.21.15.53:0/1840836721 conn(0x7f81600cb0c0 msgr2=0x7f81600cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 -- 172.21.15.53:0/1840836721 shutdown_connections 2023-10-21T16:03:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.072+0000 7f81648ba640 1 -- 172.21.15.53:0/1840836721 wait complete. 2023-10-21T16:03:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.074+0000 7f81648ba640 1 Processor -- start 2023-10-21T16:03:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.074+0000 7f81648ba640 1 -- start start 2023-10-21T16:03:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81600adc70 0x7f8160157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 0x7f81600c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600ab8d0 con 0x7f816015ab30 2023-10-21T16:03:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81600cc750 con 0x7f81600b07d0 2023-10-21T16:03:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f81648ba640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816015c1e0 con 0x7f81600adc70 2023-10-21T16:03:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f815f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f815f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51186/0 (socket says 172.21.15.53:51186) 2023-10-21T16:03:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.075+0000 7f815f59e640 1 -- 172.21.15.53:0/2889597435 learned_addr learned my addr 172.21.15.53:0/2889597435 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:33.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815ed9d640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81600adc70 0x7f8160157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 -- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81600adc70 msgr2=0x7f8160157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f8157fff640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 0x7f81600c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81600adc70 0x7f8160157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 -- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 msgr2=0x7f81600c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 0x7f81600c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 -- 172.21.15.53:0/2889597435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8158009b30 con 0x7f816015ab30 2023-10-21T16:03:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f8157fff640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 0x7f81600c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f815f59e640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8158009f40 tx=0x7f8158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.076+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158009510 con 0x7f816015ab30 2023-10-21T16:03:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.077+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815803e450 con 0x7f816015ab30 2023-10-21T16:03:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.077+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815803e7e0 con 0x7f816015ab30 2023-10-21T16:03:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.077+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81600cc980 con 0x7f816015ab30 2023-10-21T16:03:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.077+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81600ccf40 con 0x7f816015ab30 2023-10-21T16:03:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.078+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816015c3b0 con 0x7f816015ab30 2023-10-21T16:03:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.079+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f815802cd50 con 0x7f816015ab30 2023-10-21T16:03:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.080+0000 7f81577fe640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 0x7f814006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.080+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f81580ac250 con 0x7f816015ab30 2023-10-21T16:03:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.080+0000 7f815ed9d640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 0x7f814006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.082+0000 7f815ed9d640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 0x7f814006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8150006fd0 tx=0x7f8150008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.084+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f816015c3b0 con 0x7f816015ab30 2023-10-21T16:03:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.279+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f816015c3b0 con 0x7f816015ab30 2023-10-21T16:03:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.279+0000 7f81577fe640 1 -- 172.21.15.53:0/2889597435 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f816015c3b0 con 0x7f816015ab30 2023-10-21T16:03:33.281 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:33.282 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:03:33.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:33.282 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:03:33.282 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:03:33.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:33.283 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:03:33.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 msgr2=0x7f814006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 0x7f814006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8150006fd0 tx=0x7f8150008040 comp rx=0 tx=0).stop 2023-10-21T16:03:33.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 msgr2=0x7f81600cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8158009f40 tx=0x7f8158004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 shutdown_connections 2023-10-21T16:03:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f814006ae70 0x7f814006d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816015ab30 0x7f81600cc210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f81600b07d0 0x7f81600c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 --2- 172.21.15.53:0/2889597435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f81600adc70 0x7f8160157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 >> 172.21.15.53:0/2889597435 conn(0x7f81600cb0c0 msgr2=0x7f81600b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:33.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.282+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 shutdown_connections 2023-10-21T16:03:33.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.283+0000 7f81648ba640 1 -- 172.21.15.53:0/2889597435 wait complete. 2023-10-21T16:03:33.301 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:03:33.302 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:03:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.531+0000 7ff2968e5640 1 Processor -- start 2023-10-21T16:03:33.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.531+0000 7ff2968e5640 1 -- start start 2023-10-21T16:03:33.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.533+0000 7ff2968e5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff290146930 0x7ff290162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.533+0000 7ff2968e5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.533+0000 7ff2968e5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2901650b0 con 0x7ff290143910 2023-10-21T16:03:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.533+0000 7ff2968e5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff290165b80 con 0x7ff290146930 2023-10-21T16:03:33.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.533+0000 7ff2968e5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290162540 con 0x7ff290146d10 2023-10-21T16:03:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff290146930 0x7ff290162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:44770/0 (socket says 172.21.15.53:44770) 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28f7fe640 1 -- 172.21.15.53:0/1409437164 learned_addr learned my addr 172.21.15.53:0/1409437164 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28f7fe640 1 -- 172.21.15.53:0/1409437164 >> v2:172.21.15.124:3300/0 conn(0x7ff290146930 msgr2=0x7ff290162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.534+0000 7ff28f7fe640 1 --2- 172.21.15.53:0/1409437164 >> v2:172.21.15.124:3300/0 conn(0x7ff290146930 0x7ff290162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.535+0000 7ff28f7fe640 1 -- 172.21.15.53:0/1409437164 >> v1:172.21.15.124:6789/0 conn(0x7ff290143910 legacy=0x7ff290143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:33.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.535+0000 7ff28f7fe640 1 -- 172.21.15.53:0/1409437164 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29006c9d0 con 0x7ff290146d10 2023-10-21T16:03:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.536+0000 7ff28f7fe640 1 --2- 172.21.15.53:0/1409437164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff28402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bddbf6e270ee14e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.536+0000 7ff28effd640 1 -- 172.21.15.53:0/1409437164 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284031640 con 0x7ff290146d10 2023-10-21T16:03:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.537+0000 7ff28effd640 1 -- 172.21.15.53:0/1409437164 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2840317a0 con 0x7ff290146d10 2023-10-21T16:03:33.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.537+0000 7ff28effd640 1 -- 172.21.15.53:0/1409437164 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284031b30 con 0x7ff290146d10 2023-10-21T16:03:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.537+0000 7ff2968e5640 1 -- 172.21.15.53:0/1409437164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 msgr2=0x7ff290164b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.537+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1409437164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff284009870 tx=0x7ff28402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 -- 172.21.15.53:0/1409437164 shutdown_connections 2023-10-21T16:03:33.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1409437164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146d10 0x7ff290164b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1409437164 >> v2:172.21.15.124:3300/0 conn(0x7ff290146930 0x7ff290162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 -- 172.21.15.53:0/1409437164 >> 172.21.15.53:0/1409437164 conn(0x7ff290069d50 msgr2=0x7ff29006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 -- 172.21.15.53:0/1409437164 shutdown_connections 2023-10-21T16:03:33.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.538+0000 7ff2968e5640 1 -- 172.21.15.53:0/1409437164 wait complete. 2023-10-21T16:03:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.540+0000 7ff2968e5640 1 Processor -- start 2023-10-21T16:03:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.540+0000 7ff2968e5640 1 -- start start 2023-10-21T16:03:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff290143910 0x7ff2900b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146930 0x7ff290140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900b2f00 con 0x7ff290146d10 2023-10-21T16:03:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290162850 con 0x7ff290143910 2023-10-21T16:03:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.541+0000 7ff2968e5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900b2940 con 0x7ff290146930 2023-10-21T16:03:33.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51200/0 (socket says 172.21.15.53:51200) 2023-10-21T16:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 -- 172.21.15.53:0/1422210914 learned_addr learned my addr 172.21.15.53:0/1422210914 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff28f7fe640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146930 0x7ff290140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff28ffff640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff290143910 0x7ff2900b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 -- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146930 msgr2=0x7ff290140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146930 0x7ff290140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 -- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff290143910 msgr2=0x7ff2900b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff290143910 0x7ff2900b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.542+0000 7ff294e82640 1 -- 172.21.15.53:0/1422210914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840094d0 con 0x7ff290146d10 2023-10-21T16:03:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff294e82640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff28800b970 tx=0x7ff28800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28800cb40 con 0x7ff290146d10 2023-10-21T16:03:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28800cca0 con 0x7ff290146d10 2023-10-21T16:03:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288014680 con 0x7ff290146d10 2023-10-21T16:03:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2900b2c20 con 0x7ff290146d10 2023-10-21T16:03:33.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.543+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2900b2100 con 0x7ff290146d10 2023-10-21T16:03:33.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.544+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29015b900 con 0x7ff290146d10 2023-10-21T16:03:33.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.545+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff28801d3b0 con 0x7ff290146d10 2023-10-21T16:03:33.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.546+0000 7ff28cff9640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 0x7ff27c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:33.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.546+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ff2880891a0 con 0x7ff290146d10 2023-10-21T16:03:33.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.547+0000 7ff28ffff640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 0x7ff27c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:33.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.548+0000 7ff28ffff640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 0x7ff27c06d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff280009ad0 tx=0x7ff280008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:33.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.551+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2880544f0 con 0x7ff290146d10 2023-10-21T16:03:33.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:33.744+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff290162a10 con 0x7ff290146d10 2023-10-21T16:03:34.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.205+0000 7ff28cff9640 1 -- 172.21.15.53:0/1422210914 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff288057b10 con 0x7ff290146d10 2023-10-21T16:03:34.207 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:03:34.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.210+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 msgr2=0x7ff27c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.210+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 0x7ff27c06d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff280009ad0 tx=0x7ff280008040 comp rx=0 tx=0).stop 2023-10-21T16:03:34.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.210+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 msgr2=0x7ff2900b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.210+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff28800b970 tx=0x7ff28800be30 comp rx=0 tx=0).stop 2023-10-21T16:03:34.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 shutdown_connections 2023-10-21T16:03:34.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ff27c06ae70 0x7ff27c06d330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff290146d10 0x7ff2900b1bc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff290146930 0x7ff290140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 --2- 172.21.15.53:0/1422210914 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff290143910 0x7ff2900b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.211+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 >> 172.21.15.53:0/1422210914 conn(0x7ff290069d50 msgr2=0x7ff2900ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:34.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.212+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 shutdown_connections 2023-10-21T16:03:34.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.212+0000 7ff2968e5640 1 -- 172.21.15.53:0/1422210914 wait complete. 2023-10-21T16:03:34.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:03:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.451+0000 7f044967c640 1 Processor -- start 2023-10-21T16:03:34.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.451+0000 7f044967c640 1 -- start start 2023-10-21T16:03:34.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.453+0000 7f044967c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f04440ca990 0x7f04440c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f044967c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0444070cf0 con 0x7f04440cb500 2023-10-21T16:03:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f044967c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04440cae20 con 0x7f04440ca530 2023-10-21T16:03:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f044967c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0444161910 con 0x7f04440ca990 2023-10-21T16:03:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f0442ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f04440cb500 0x7f04440cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55632/0 (socket says 172.21.15.53:55632) 2023-10-21T16:03:34.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f0442ffd640 1 -- 172.21.15.53:0/1308515921 learned_addr learned my addr 172.21.15.53:0/1308515921 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:34.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.454+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063729789 0 0) 0x7f0444070cf0 con 0x7f04440cb500 2023-10-21T16:03:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04437fe640 1 --2- 172.21.15.53:0/1308515921 >> v2:172.21.15.124:3300/0 conn(0x7f04440ca990 0x7f04440c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0430003520 con 0x7f04440cb500 2023-10-21T16:03:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343777201 0 0) 0x7f0430003520 con 0x7f04440cb500 2023-10-21T16:03:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 >> v2:172.21.15.124:3300/0 conn(0x7f04440ca990 msgr2=0x7f04440c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:34.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 --2- 172.21.15.53:0/1308515921 >> v2:172.21.15.124:3300/0 conn(0x7f04440ca990 0x7f04440c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 >> v1:172.21.15.124:6789/0 conn(0x7f04440ca530 legacy=0x7f04440c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04440cb080 con 0x7f04440cb500 2023-10-21T16:03:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0434001670 con 0x7f04440cb500 2023-10-21T16:03:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 >> v1:172.21.15.53:6789/0 conn(0x7f04440cb500 legacy=0x7f04440cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0430003970 0x7f0430003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04300047c0 0x7f0430004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 0x7f043000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0444161910 con 0x7f0430005180 2023-10-21T16:03:34.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043000c120 con 0x7f0430003970 2023-10-21T16:03:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.455+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043000c2e0 con 0x7f04300047c0 2023-10-21T16:03:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 0x7f043000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f0442ffd640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0430003970 0x7f0430003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f043a7fc640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04300047c0 0x7f0430004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 -- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04300047c0 msgr2=0x7f0430004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04300047c0 0x7f0430004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 -- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0430003970 msgr2=0x7f0430003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0430003970 0x7f0430003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.456+0000 7f04437fe640 1 -- 172.21.15.53:0/1308515921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430003520 con 0x7f0430005180 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.457+0000 7f04437fe640 1 -- 172.21.15.53:0/1308515921 reap_dead start 2023-10-21T16:03:34.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.457+0000 7f04437fe640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 0x7f043000b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f043c009fc0 tx=0x7f04300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ff56fdb955974a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.457+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c02cc60 con 0x7f0430005180 2023-10-21T16:03:34.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c00f040 con 0x7f0430005180 2023-10-21T16:03:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f04427fc640 1 -- 172.21.15.53:0/1308515921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c003990 con 0x7f0430005180 2023-10-21T16:03:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 -- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 msgr2=0x7f043000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 0x7f043000b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f043c009fc0 tx=0x7f04300057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 -- 172.21.15.53:0/1308515921 shutdown_connections 2023-10-21T16:03:34.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 --2- 172.21.15.53:0/1308515921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0430005180 0x7f043000b980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 -- 172.21.15.53:0/1308515921 >> 172.21.15.53:0/1308515921 conn(0x7f044406a140 msgr2=0x7f044406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 -- 172.21.15.53:0/1308515921 shutdown_connections 2023-10-21T16:03:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.458+0000 7f044967c640 1 -- 172.21.15.53:0/1308515921 wait complete. 2023-10-21T16:03:34.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.460+0000 7f044967c640 1 Processor -- start 2023-10-21T16:03:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.460+0000 7f044967c640 1 -- start start 2023-10-21T16:03:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0430005180 0x7f044415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0444151050 0x7f044415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04440bd430 con 0x7f04440bf190 2023-10-21T16:03:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04440bfb50 con 0x7f0444151050 2023-10-21T16:03:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.461+0000 7f044967c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044415c530 con 0x7f0430005180 2023-10-21T16:03:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52424/0 (socket says 172.21.15.53:52424) 2023-10-21T16:03:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 -- 172.21.15.53:0/2739063411 learned_addr learned my addr 172.21.15.53:0/2739063411 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f0442ffd640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0430005180 0x7f044415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f043a7fc640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0444151050 0x7f044415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 -- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0430005180 msgr2=0x7f044415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0430005180 0x7f044415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 -- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0444151050 msgr2=0x7f044415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0444151050 0x7f044415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.462+0000 7f04437fe640 1 -- 172.21.15.53:0/2739063411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044415c800 con 0x7f04440bf190 2023-10-21T16:03:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f04437fe640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f043c02c6d0 tx=0x7f043c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:34.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c035e40 con 0x7f04440bf190 2023-10-21T16:03:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c003990 con 0x7f04440bf190 2023-10-21T16:03:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044415cae0 con 0x7f04440bf190 2023-10-21T16:03:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c003d80 con 0x7f04440bf190 2023-10-21T16:03:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.463+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04440bd640 con 0x7f04440bf190 2023-10-21T16:03:34.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.464+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0444071cb0 con 0x7f04440bf190 2023-10-21T16:03:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.467+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f043c004030 con 0x7f04440bf190 2023-10-21T16:03:34.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.467+0000 7f043bfff640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 0x7f042406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.468+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f043c0acd50 con 0x7f04440bf190 2023-10-21T16:03:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.468+0000 7f0442ffd640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 0x7f042406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.469+0000 7f0442ffd640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 0x7f042406d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0434001eb0 tx=0x7f0434009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:34.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.471+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f043c078030 con 0x7f04440bf190 2023-10-21T16:03:34.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:34.669+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f044404e240 con 0x7f04440bf190 2023-10-21T16:03:35.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.212+0000 7f043bfff640 1 -- 172.21.15.53:0/2739063411 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f043c07b650 con 0x7f04440bf190 2023-10-21T16:03:35.214 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:03:35.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 msgr2=0x7f042406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:35.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 0x7f042406d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0434001eb0 tx=0x7f0434009040 comp rx=0 tx=0).stop 2023-10-21T16:03:35.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 msgr2=0x7f04440bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:35.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f043c02c6d0 tx=0x7f043c009750 comp rx=0 tx=0).stop 2023-10-21T16:03:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 shutdown_connections 2023-10-21T16:03:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f042406ae70 0x7f042406d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04440bf190 0x7f04440bf610 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0444151050 0x7f044415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 --2- 172.21.15.53:0/2739063411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0430005180 0x7f044415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.216+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 >> 172.21.15.53:0/2739063411 conn(0x7f044406a140 msgr2=0x7f04440c4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.217+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 shutdown_connections 2023-10-21T16:03:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.217+0000 7f044967c640 1 -- 172.21.15.53:0/2739063411 wait complete. 2023-10-21T16:03:35.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:03:35.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.446+0000 7f607a84b640 1 Processor -- start 2023-10-21T16:03:35.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.446+0000 7f607a84b640 1 -- start start 2023-10-21T16:03:35.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.448+0000 7f607a84b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6074069fc0 0x7f60740ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.448+0000 7f607a84b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6074071810 0x7f60740b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.448+0000 7f607a84b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074071650 con 0x7f607406a3a0 2023-10-21T16:03:35.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.448+0000 7f607a84b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607406f260 con 0x7f6074071810 2023-10-21T16:03:35.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.448+0000 7f607a84b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740ca3b0 con 0x7f6074069fc0 2023-10-21T16:03:35.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6074071810 0x7f60740b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6074071810 0x7f60740b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52436/0 (socket says 172.21.15.53:52436) 2023-10-21T16:03:35.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 -- 172.21.15.53:0/1672423646 learned_addr learned my addr 172.21.15.53:0/1672423646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:35.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f6073fff640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6074069fc0 0x7f60740ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 -- 172.21.15.53:0/1672423646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6074069fc0 msgr2=0x7f60740ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:35.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6074069fc0 0x7f60740ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 -- 172.21.15.53:0/1672423646 >> v1:172.21.15.124:6789/0 conn(0x7f607406a3a0 legacy=0x7f60740b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:35.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.449+0000 7f60737fe640 1 -- 172.21.15.53:0/1672423646 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607406ef80 con 0x7f6074071810 2023-10-21T16:03:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.450+0000 7f60737fe640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740b12d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f605c009870 tx=0x7f605c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9983593816a1db32 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f6072ffd640 1 -- 172.21.15.53:0/1672423646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c031640 con 0x7f6074071810 2023-10-21T16:03:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f6072ffd640 1 -- 172.21.15.53:0/1672423646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c0317a0 con 0x7f6074071810 2023-10-21T16:03:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f6072ffd640 1 -- 172.21.15.53:0/1672423646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c031b30 con 0x7f6074071810 2023-10-21T16:03:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 -- 172.21.15.53:0/1672423646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 msgr2=0x7f60740b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740b12d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f605c009870 tx=0x7f605c02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 -- 172.21.15.53:0/1672423646 shutdown_connections 2023-10-21T16:03:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740b12d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 --2- 172.21.15.53:0/1672423646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6074069fc0 0x7f60740ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.451+0000 7f607a84b640 1 -- 172.21.15.53:0/1672423646 >> 172.21.15.53:0/1672423646 conn(0x7f60740cc2d0 msgr2=0x7f60740cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.452+0000 7f607a84b640 1 -- 172.21.15.53:0/1672423646 shutdown_connections 2023-10-21T16:03:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.452+0000 7f607a84b640 1 -- 172.21.15.53:0/1672423646 wait complete. 2023-10-21T16:03:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.453+0000 7f607a84b640 1 Processor -- start 2023-10-21T16:03:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.454+0000 7f607a84b640 1 -- start start 2023-10-21T16:03:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.454+0000 7f607a84b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6074069fc0 0x7f60740c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f607a84b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607406a3a0 0x7f60740c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f607a84b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f607a84b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c7420 con 0x7f6074071810 2023-10-21T16:03:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f607a84b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c8cc0 con 0x7f6074069fc0 2023-10-21T16:03:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f607a84b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c6ac0 con 0x7f607406a3a0 2023-10-21T16:03:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52450/0 (socket says 172.21.15.53:52450) 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 -- 172.21.15.53:0/3325039909 learned_addr learned my addr 172.21.15.53:0/3325039909 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6073fff640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6074069fc0 0x7f60740c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f60737fe640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607406a3a0 0x7f60740c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 -- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607406a3a0 msgr2=0x7f60740c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607406a3a0 0x7f60740c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 -- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6074069fc0 msgr2=0x7f60740c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6074069fc0 0x7f60740c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.455+0000 7f6078de8640 1 -- 172.21.15.53:0/3325039909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0094d0 con 0x7f6074071810 2023-10-21T16:03:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f6078de8640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c440 tx=0x7f606c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c011790 con 0x7f6074071810 2023-10-21T16:03:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0118f0 con 0x7f6074071810 2023-10-21T16:03:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60740c6d60 con 0x7f6074071810 2023-10-21T16:03:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c011ca0 con 0x7f6074071810 2023-10-21T16:03:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.456+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60740c48d0 con 0x7f6074071810 2023-10-21T16:03:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.457+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607404e160 con 0x7f6074071810 2023-10-21T16:03:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.458+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f606c01c460 con 0x7f6074071810 2023-10-21T16:03:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.459+0000 7f6070ff9640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 0x7f605406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.459+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f606c089610 con 0x7f6074071810 2023-10-21T16:03:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.460+0000 7f6073fff640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 0x7f605406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.461+0000 7f6073fff640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 0x7f605406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60640070b0 tx=0x7f6064007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.464+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f606c0545d0 con 0x7f6074071810 2023-10-21T16:03:35.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:35.693+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f60741654b0 con 0x7f6074071810 2023-10-21T16:03:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.243+0000 7f6070ff9640 1 -- 172.21.15.53:0/3325039909 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f606c057bf0 con 0x7f6074071810 2023-10-21T16:03:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 5 and data pool 6 2023-10-21T16:03:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 msgr2=0x7f605406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 0x7f605406d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60640070b0 tx=0x7f6064007040 comp rx=0 tx=0).stop 2023-10-21T16:03:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 msgr2=0x7f60740c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c440 tx=0x7f606c00c900 comp rx=0 tx=0).stop 2023-10-21T16:03:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 shutdown_connections 2023-10-21T16:03:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f605406ae70 0x7f605406d330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6074071810 0x7f60740c4390 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607406a3a0 0x7f60740c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.247+0000 7f607a84b640 1 --2- 172.21.15.53:0/3325039909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6074069fc0 0x7f60740c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.248+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 >> 172.21.15.53:0/3325039909 conn(0x7f60740cc2d0 msgr2=0x7f6074164d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.248+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 shutdown_connections 2023-10-21T16:03:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.248+0000 7f607a84b640 1 -- 172.21.15.53:0/3325039909 wait complete. 2023-10-21T16:03:36.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:36.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.468+0000 7f9bd3661640 1 Processor -- start 2023-10-21T16:03:36.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.469+0000 7f9bd3661640 1 -- start start 2023-10-21T16:03:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.470+0000 7f9bd3661640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bcc159ac0 0x7f9bcc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.470+0000 7f9bd3661640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.470+0000 7f9bd3661640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc06dee0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.470+0000 7f9bd3661640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc15d850 con 0x7f9bcc159ac0 2023-10-21T16:03:36.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.470+0000 7f9bd3661640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc15db30 con 0x7f9bcc0709e0 2023-10-21T16:03:36.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.471+0000 7f9bd1bfe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.471+0000 7f9bd13fd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bcc159ac0 0x7f9bcc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.471+0000 7f9bd1bfe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:40598/0 (socket says 172.21.15.53:40598) 2023-10-21T16:03:36.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.471+0000 7f9bd1bfe640 1 -- 172.21.15.53:0/1873704362 learned_addr learned my addr 172.21.15.53:0/1873704362 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:36.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.472+0000 7f9bd1bfe640 1 -- 172.21.15.53:0/1873704362 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc159ac0 msgr2=0x7f9bcc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:36.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.472+0000 7f9bd1bfe640 1 --2- 172.21.15.53:0/1873704362 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc159ac0 0x7f9bcc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.472+0000 7f9bd1bfe640 1 -- 172.21.15.53:0/1873704362 >> v1:172.21.15.124:6789/0 conn(0x7f9bcc0ad790 legacy=0x7f9bcc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:36.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.472+0000 7f9bd1bfe640 1 -- 172.21.15.53:0/1873704362 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc15de10 con 0x7f9bcc0709e0 2023-10-21T16:03:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.473+0000 7f9bd1bfe640 1 --2- 172.21.15.53:0/1873704362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc06e790 tx=0x7f9bc402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12697faaced3bcf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bc3fff640 1 -- 172.21.15.53:0/1873704362 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc402ca30 con 0x7f9bcc0709e0 2023-10-21T16:03:36.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bc3fff640 1 -- 172.21.15.53:0/1873704362 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc402cb90 con 0x7f9bcc0709e0 2023-10-21T16:03:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bc3fff640 1 -- 172.21.15.53:0/1873704362 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40343f0 con 0x7f9bcc0709e0 2023-10-21T16:03:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 -- 172.21.15.53:0/1873704362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 msgr2=0x7f9bcc0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1873704362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc06e790 tx=0x7f9bc402be90 comp rx=0 tx=0).stop 2023-10-21T16:03:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 -- 172.21.15.53:0/1873704362 shutdown_connections 2023-10-21T16:03:36.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1873704362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc0c8dd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1873704362 >> v2:172.21.15.124:3300/0 conn(0x7f9bcc159ac0 0x7f9bcc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 -- 172.21.15.53:0/1873704362 >> 172.21.15.53:0/1873704362 conn(0x7f9bcc06a380 msgr2=0x7f9bcc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.474+0000 7f9bd3661640 1 -- 172.21.15.53:0/1873704362 shutdown_connections 2023-10-21T16:03:36.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.475+0000 7f9bd3661640 1 -- 172.21.15.53:0/1873704362 wait complete. 2023-10-21T16:03:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.476+0000 7f9bd3661640 1 Processor -- start 2023-10-21T16:03:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.476+0000 7f9bd3661640 1 -- start start 2023-10-21T16:03:36.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc13bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc159ac0 0x7f9bcc158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc13c040 con 0x7f9bcc0ad790 2023-10-21T16:03:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b2170 con 0x7f9bcc159ac0 2023-10-21T16:03:36.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.477+0000 7f9bd3661640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc13f110 con 0x7f9bcc0709e0 2023-10-21T16:03:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52452/0 (socket says 172.21.15.53:52452) 2023-10-21T16:03:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 -- 172.21.15.53:0/1365882348 learned_addr learned my addr 172.21.15.53:0/1365882348 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd13fd640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc13bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd1bfe640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc159ac0 0x7f9bcc158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 -- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 msgr2=0x7f9bcc13bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc13bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 -- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc159ac0 msgr2=0x7f9bcc158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc159ac0 0x7f9bcc158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd0bfc640 1 -- 172.21.15.53:0/1365882348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4009b30 con 0x7f9bcc0ad790 2023-10-21T16:03:36.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.478+0000 7f9bd13fd640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:36.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bd0bfc640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00c430 tx=0x7f9bbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011790 con 0x7f9bcc0ad790 2023-10-21T16:03:36.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc0118f0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011ca0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0b3900 con 0x7f9bcc0ad790 2023-10-21T16:03:36.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.479+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0b3ec0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.480+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc067490 con 0x7f9bcc0ad790 2023-10-21T16:03:36.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.481+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9bbc01c460 con 0x7f9bcc0ad790 2023-10-21T16:03:36.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.482+0000 7f9bc1ffb640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.482+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9bbc0897a0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.483+0000 7f9bd13fd640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.484+0000 7f9bd13fd640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 0x7f9bb006d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009af0 tx=0x7f9bc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.487+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bbc054680 con 0x7f9bcc0ad790 2023-10-21T16:03:36.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.676+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bcc0b0ff0 con 0x7f9bcc0ad790 2023-10-21T16:03:36.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.678+0000 7f9bc1ffb640 1 -- 172.21.15.53:0/1365882348 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16775 (secure 0 0 0) 0x7f9bbc06fe60 con 0x7f9bcc0ad790 2023-10-21T16:03:36.680 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:36.680 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":39,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:03:35.695050+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 msgr2=0x7f9bb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 0x7f9bb006d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009af0 tx=0x7f9bc8008040 comp rx=0 tx=0).stop 2023-10-21T16:03:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 msgr2=0x7f9bcc1587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00c430 tx=0x7f9bbc00c8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 shutdown_connections 2023-10-21T16:03:36.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bcc159ac0 0x7f9bcc158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f9bb006ae70 0x7f9bb006d330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bcc0ad790 0x7f9bcc1587c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 --2- 172.21.15.53:0/1365882348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bcc0709e0 0x7f9bcc13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 >> 172.21.15.53:0/1365882348 conn(0x7f9bcc06a380 msgr2=0x7f9bcc156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:36.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 shutdown_connections 2023-10-21T16:03:36.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.681+0000 7f9bd3661640 1 -- 172.21.15.53:0/1365882348 wait complete. 2023-10-21T16:03:36.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:36.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.908+0000 7f519eda9640 1 Processor -- start 2023-10-21T16:03:36.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.908+0000 7f519eda9640 1 -- start start 2023-10-21T16:03:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519eda9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5198159ac0 0x7f519806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519eda9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51980709e0 0x7f51980c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519eda9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519806dee0 con 0x7f51980ad790 2023-10-21T16:03:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519eda9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519815d850 con 0x7f51980709e0 2023-10-21T16:03:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519eda9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519815db30 con 0x7f5198159ac0 2023-10-21T16:03:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f519d346640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51980709e0 0x7f51980c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f5197fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f51980ad790 0x7f51980702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55636/0 (socket says 172.21.15.53:55636) 2023-10-21T16:03:36.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.910+0000 7f5197fff640 1 -- 172.21.15.53:0/3123264672 learned_addr learned my addr 172.21.15.53:0/3123264672 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:36.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.911+0000 7f519cb45640 1 --2- 172.21.15.53:0/3123264672 >> v2:172.21.15.124:3300/0 conn(0x7f5198159ac0 0x7f519806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.911+0000 7f519d346640 1 -- 172.21.15.53:0/3123264672 >> v2:172.21.15.124:3300/0 conn(0x7f5198159ac0 msgr2=0x7f519806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:36.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.911+0000 7f519d346640 1 --2- 172.21.15.53:0/3123264672 >> v2:172.21.15.124:3300/0 conn(0x7f5198159ac0 0x7f519806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.911+0000 7f519d346640 1 -- 172.21.15.53:0/3123264672 >> v1:172.21.15.53:6789/0 conn(0x7f51980ad790 legacy=0x7f51980702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.911+0000 7f519d346640 1 -- 172.21.15.53:0/3123264672 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519815de10 con 0x7f51980709e0 2023-10-21T16:03:36.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.912+0000 7f519d346640 1 --2- 172.21.15.53:0/3123264672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980709e0 0x7f51980c8dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f519806e790 tx=0x7f519002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52912438237e7b7a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f51977fe640 1 -- 172.21.15.53:0/3123264672 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519002cc70 con 0x7f51980709e0 2023-10-21T16:03:36.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f51977fe640 1 -- 172.21.15.53:0/3123264672 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519002cdd0 con 0x7f51980709e0 2023-10-21T16:03:36.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f51977fe640 1 -- 172.21.15.53:0/3123264672 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190034660 con 0x7f51980709e0 2023-10-21T16:03:36.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 -- 172.21.15.53:0/3123264672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980709e0 msgr2=0x7f51980c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 --2- 172.21.15.53:0/3123264672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980709e0 0x7f51980c8dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f519806e790 tx=0x7f519002bec0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 -- 172.21.15.53:0/3123264672 shutdown_connections 2023-10-21T16:03:36.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 --2- 172.21.15.53:0/3123264672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980709e0 0x7f51980c8dd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 --2- 172.21.15.53:0/3123264672 >> v2:172.21.15.124:3300/0 conn(0x7f5198159ac0 0x7f519806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 -- 172.21.15.53:0/3123264672 >> 172.21.15.53:0/3123264672 conn(0x7f519806a380 msgr2=0x7f51980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:36.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 -- 172.21.15.53:0/3123264672 shutdown_connections 2023-10-21T16:03:36.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.913+0000 7f519eda9640 1 -- 172.21.15.53:0/3123264672 wait complete. 2023-10-21T16:03:36.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.915+0000 7f519eda9640 1 Processor -- start 2023-10-21T16:03:36.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.915+0000 7f519eda9640 1 -- start start 2023-10-21T16:03:36.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51980709e0 0x7f519814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198159ac0 0x7f519815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981f6390 con 0x7f51980ad790 2023-10-21T16:03:36.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5198151100 con 0x7f5198159ac0 2023-10-21T16:03:36.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.916+0000 7f519eda9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51981f3530 con 0x7f51980709e0 2023-10-21T16:03:36.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52470/0 (socket says 172.21.15.53:52470) 2023-10-21T16:03:36.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 -- 172.21.15.53:0/495792886 learned_addr learned my addr 172.21.15.53:0/495792886 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:36.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f519cb45640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51980709e0 0x7f519814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f519d346640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198159ac0 0x7f519815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 -- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51980709e0 msgr2=0x7f519814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51980709e0 0x7f519814bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 -- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198159ac0 msgr2=0x7f519815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:36.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198159ac0 0x7f519815c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:36.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 -- 172.21.15.53:0/495792886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009a70 con 0x7f51980ad790 2023-10-21T16:03:36.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.917+0000 7f5197fff640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f518400c4a0 tx=0x7f518400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.918+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184005ce0 con 0x7f51980ad790 2023-10-21T16:03:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.918+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184005e40 con 0x7f51980ad790 2023-10-21T16:03:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.918+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184007270 con 0x7f51980ad790 2023-10-21T16:03:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.918+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519815a1a0 con 0x7f51980ad790 2023-10-21T16:03:36.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.918+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51981513a0 con 0x7f51980ad790 2023-10-21T16:03:36.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.919+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5198151590 con 0x7f51980ad790 2023-10-21T16:03:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.920+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5184007490 con 0x7f51980ad790 2023-10-21T16:03:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.921+0000 7f51957fa640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 0x7f517c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.921+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5184088c30 con 0x7f51980ad790 2023-10-21T16:03:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.922+0000 7f519cb45640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 0x7f517c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:36.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.923+0000 7f519cb45640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 0x7f517c06d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f518c006fd0 tx=0x7f518c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:36.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:36.925+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5184053bb0 con 0x7f51980ad790 2023-10-21T16:03:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.125+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5198067490 con 0x7f51980ad790 2023-10-21T16:03:37.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.128+0000 7f51957fa640 1 -- 172.21.15.53:0/495792886 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16762 (secure 0 0 0) 0x7f5184017020 con 0x7f51980ad790 2023-10-21T16:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":40,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:03:36.939133+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 msgr2=0x7f517c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 0x7f517c06d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f518c006fd0 tx=0x7f518c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 msgr2=0x7f51981f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f518400c4a0 tx=0x7f518400c960 comp rx=0 tx=0).stop 2023-10-21T16:03:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 shutdown_connections 2023-10-21T16:03:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5198159ac0 0x7f519815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f517c06ae70 0x7f517c06d330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51980ad790 0x7f51981f2ff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 --2- 172.21.15.53:0/495792886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51980709e0 0x7f519814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 >> 172.21.15.53:0/495792886 conn(0x7f519806a380 msgr2=0x7f51980c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 shutdown_connections 2023-10-21T16:03:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.131+0000 7f519eda9640 1 -- 172.21.15.53:0/495792886 wait complete. 2023-10-21T16:03:37.152 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:03:37.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.372+0000 7efc99827640 1 Processor -- start 2023-10-21T16:03:37.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.372+0000 7efc99827640 1 -- start start 2023-10-21T16:03:37.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.373+0000 7efc99827640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc940cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc99827640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940ccbc0 con 0x7efc940c8b40 2023-10-21T16:03:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc99827640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940ce1f0 con 0x7efc940b0680 2023-10-21T16:03:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc99827640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940cd590 con 0x7efc9415cae0 2023-10-21T16:03:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc93fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efc940c8b40 0x7efc940aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55640/0 (socket says 172.21.15.53:55640) 2023-10-21T16:03:37.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc93fff640 1 -- 172.21.15.53:0/4267432186 learned_addr learned my addr 172.21.15.53:0/4267432186 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc98825640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc940cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.374+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915935957 0 0) 0x7efc940ccbc0 con 0x7efc940c8b40 2023-10-21T16:03:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc84003520 con 0x7efc940c8b40 2023-10-21T16:03:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041676430 0 0) 0x7efc84003520 con 0x7efc940c8b40 2023-10-21T16:03:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 msgr2=0x7efc940cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc940cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 >> v1:172.21.15.124:6789/0 conn(0x7efc940b0680 legacy=0x7efc94070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940c9720 con 0x7efc940c8b40 2023-10-21T16:03:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7efc80001670 con 0x7efc940c8b40 2023-10-21T16:03:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 >> v1:172.21.15.53:6789/0 conn(0x7efc940c8b40 legacy=0x7efc940aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc840039a0 0x7efc84003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc84004d00 0x7efc84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940cd590 con 0x7efc840039a0 2023-10-21T16:03:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8400ba60 con 0x7efc84004d00 2023-10-21T16:03:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8400bc20 con 0x7efc9415cae0 2023-10-21T16:03:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.375+0000 7efc98825640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc840039a0 0x7efc84003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc93fff640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc99026640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc84004d00 0x7efc84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc98825640 1 -- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 msgr2=0x7efc840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc98825640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc98825640 1 -- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc84004d00 msgr2=0x7efc84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc98825640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc84004d00 0x7efc84005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc98825640 1 -- 172.21.15.53:0/4267432186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84003520 con 0x7efc840039a0 2023-10-21T16:03:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc99026640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc84004d00 0x7efc84005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.376+0000 7efc93fff640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc9415cae0 0x7efc840047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.377+0000 7efc98825640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc840039a0 0x7efc84003dc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc88009a90 tx=0x7efc840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db5a52fea0bacc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:37.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.377+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8802c680 con 0x7efc840039a0 2023-10-21T16:03:37.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.377+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8802c7e0 con 0x7efc840039a0 2023-10-21T16:03:37.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.377+0000 7efc937fe640 1 -- 172.21.15.53:0/4267432186 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8802cb70 con 0x7efc840039a0 2023-10-21T16:03:37.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 -- 172.21.15.53:0/4267432186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc840039a0 msgr2=0x7efc84003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 --2- 172.21.15.53:0/4267432186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc840039a0 0x7efc84003dc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efc88009a90 tx=0x7efc840057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99026640 1 -- 172.21.15.53:0/4267432186 reap_dead start 2023-10-21T16:03:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 -- 172.21.15.53:0/4267432186 shutdown_connections 2023-10-21T16:03:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 -- 172.21.15.53:0/4267432186 >> 172.21.15.53:0/4267432186 conn(0x7efc9406c050 msgr2=0x7efc9406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:37.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 -- 172.21.15.53:0/4267432186 shutdown_connections 2023-10-21T16:03:37.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.378+0000 7efc99827640 1 -- 172.21.15.53:0/4267432186 wait complete. 2023-10-21T16:03:37.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.380+0000 7efc99827640 1 Processor -- start 2023-10-21T16:03:37.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.380+0000 7efc99827640 1 -- start start 2023-10-21T16:03:37.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc840039a0 0x7efc940c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 0x7efc941f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940c3d00 con 0x7efc9414e4d0 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940c3180 con 0x7efc940c2a10 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99827640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940c3460 con 0x7efc840039a0 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99026640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99026640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52496/0 (socket says 172.21.15.53:52496) 2023-10-21T16:03:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.381+0000 7efc99026640 1 -- 172.21.15.53:0/1103602876 learned_addr learned my addr 172.21.15.53:0/1103602876 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 -- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc840039a0 msgr2=0x7efc940c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc840039a0 0x7efc940c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc93fff640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 0x7efc941f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 -- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 msgr2=0x7efc941f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 0x7efc941f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 -- 172.21.15.53:0/1103602876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940c3740 con 0x7efc9414e4d0 2023-10-21T16:03:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc93fff640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 0x7efc941f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99026640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000f80 tx=0x7efc8c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:37.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003970 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c003af0 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003ea0 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc940c3a20 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.382+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9414eec0 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.384+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc940c5650 con 0x7efc9414e4d0 2023-10-21T16:03:37.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.384+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efc8c0050f0 con 0x7efc9414e4d0 2023-10-21T16:03:37.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.385+0000 7efc917fa640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 0x7efc7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.386+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc8c0894e0 con 0x7efc9414e4d0 2023-10-21T16:03:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.386+0000 7efc98825640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 0x7efc7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.387+0000 7efc98825640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 0x7efc7806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc88036000 tx=0x7efc8802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:37.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.389+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc940c5650 con 0x7efc9414e4d0 2023-10-21T16:03:37.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:37.624+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efc940c5650 con 0x7efc9414e4d0 2023-10-21T16:03:38.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.277+0000 7efc917fa640 1 -- 172.21.15.53:0/1103602876 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7efc940c5650 con 0x7efc9414e4d0 2023-10-21T16:03:38.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 msgr2=0x7efc7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 0x7efc7806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc88036000 tx=0x7efc8802e000 comp rx=0 tx=0).stop 2023-10-21T16:03:38.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 msgr2=0x7efc9414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efc8c000f80 tx=0x7efc8c00d9f0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 shutdown_connections 2023-10-21T16:03:38.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7efc7806ae70 0x7efc7806d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc9414e4d0 0x7efc9414e980 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc940c2a10 0x7efc941f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 --2- 172.21.15.53:0/1103602876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc840039a0 0x7efc940c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 >> 172.21.15.53:0/1103602876 conn(0x7efc9406c050 msgr2=0x7efc9415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:38.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 shutdown_connections 2023-10-21T16:03:38.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.282+0000 7efc99827640 1 -- 172.21.15.53:0/1103602876 wait complete. 2023-10-21T16:03:38.307 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:03:38.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:03:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.537+0000 7f3136400640 1 Processor -- start 2023-10-21T16:03:38.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.537+0000 7f3136400640 1 -- start start 2023-10-21T16:03:38.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f3136400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31300afbe0 0x7f31300b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f3136400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 0x7f31300cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f3136400640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31300cbad0 con 0x7f313006dfc0 2023-10-21T16:03:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f3136400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300b2620 con 0x7f31300afbe0 2023-10-21T16:03:38.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f3136400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300b2060 con 0x7f31300cb130 2023-10-21T16:03:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f312ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31300afbe0 0x7f31300b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f312f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f313006dfc0 0x7f3130070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55652/0 (socket says 172.21.15.53:55652) 2023-10-21T16:03:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f312ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31300afbe0 0x7f31300b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52504/0 (socket says 172.21.15.53:52504) 2023-10-21T16:03:38.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.539+0000 7f312f7fe640 1 -- 172.21.15.53:0/2074435220 learned_addr learned my addr 172.21.15.53:0/2074435220 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:38.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f313499d640 1 --2- 172.21.15.53:0/2074435220 >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 0x7f31300cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f312ffff640 1 -- 172.21.15.53:0/2074435220 >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 msgr2=0x7f31300cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:38.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f312ffff640 1 --2- 172.21.15.53:0/2074435220 >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 0x7f31300cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f312ffff640 1 -- 172.21.15.53:0/2074435220 >> v1:172.21.15.53:6789/0 conn(0x7f313006dfc0 legacy=0x7f3130070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f312ffff640 1 -- 172.21.15.53:0/2074435220 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31300b2340 con 0x7f31300afbe0 2023-10-21T16:03:38.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.540+0000 7f313499d640 1 --2- 172.21.15.53:0/2074435220 >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 0x7f31300cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:38.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.541+0000 7f312ffff640 1 --2- 172.21.15.53:0/2074435220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f31300b15b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f31300c8d10 tx=0x7f312002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e37cb29c91798ca server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:38.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.541+0000 7f312effd640 1 -- 172.21.15.53:0/2074435220 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312002cc70 con 0x7f31300afbe0 2023-10-21T16:03:38.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.541+0000 7f312effd640 1 -- 172.21.15.53:0/2074435220 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312002cdd0 con 0x7f31300afbe0 2023-10-21T16:03:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.541+0000 7f312effd640 1 -- 172.21.15.53:0/2074435220 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120034660 con 0x7f31300afbe0 2023-10-21T16:03:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 -- 172.21.15.53:0/2074435220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 msgr2=0x7f31300b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 --2- 172.21.15.53:0/2074435220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f31300b15b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f31300c8d10 tx=0x7f312002bec0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 -- 172.21.15.53:0/2074435220 shutdown_connections 2023-10-21T16:03:38.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 --2- 172.21.15.53:0/2074435220 >> v2:172.21.15.124:3300/0 conn(0x7f31300cb130 0x7f31300cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 --2- 172.21.15.53:0/2074435220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f31300b15b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 -- 172.21.15.53:0/2074435220 >> 172.21.15.53:0/2074435220 conn(0x7f313006a5d0 msgr2=0x7f31300b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 -- 172.21.15.53:0/2074435220 shutdown_connections 2023-10-21T16:03:38.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.542+0000 7f3136400640 1 -- 172.21.15.53:0/2074435220 wait complete. 2023-10-21T16:03:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.544+0000 7f3136400640 1 Processor -- start 2023-10-21T16:03:38.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.544+0000 7f3136400640 1 -- start start 2023-10-21T16:03:38.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313006dfc0 0x7f3130158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31300cb130 0x7f313015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31301592a0 con 0x7f31300afbe0 2023-10-21T16:03:38.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c2380 con 0x7f31300cb130 2023-10-21T16:03:38.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.545+0000 7f3136400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c2660 con 0x7f313006dfc0 2023-10-21T16:03:38.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313006dfc0 0x7f3130158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52520/0 (socket says 172.21.15.53:52520) 2023-10-21T16:03:38.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 -- 172.21.15.53:0/4061244123 learned_addr learned my addr 172.21.15.53:0/4061244123 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f313499d640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31300cb130 0x7f313015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 -- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313006dfc0 msgr2=0x7f3130158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313006dfc0 0x7f3130158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 -- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31300cb130 msgr2=0x7f313015cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31300cb130 0x7f313015cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:38.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.546+0000 7f312f7fe640 1 -- 172.21.15.53:0/4061244123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009a70 con 0x7f31300afbe0 2023-10-21T16:03:38.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f312f7fe640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f311800c500 tx=0x7f311800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:38.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118005ce0 con 0x7f31300afbe0 2023-10-21T16:03:38.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118005e40 con 0x7f31300afbe0 2023-10-21T16:03:38.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180072f0 con 0x7f31300afbe0 2023-10-21T16:03:38.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31300c2940 con 0x7f31300afbe0 2023-10-21T16:03:38.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.547+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31300c2f00 con 0x7f31300afbe0 2023-10-21T16:03:38.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.548+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31300c30f0 con 0x7f31300afbe0 2023-10-21T16:03:38.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.549+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3118007510 con 0x7f31300afbe0 2023-10-21T16:03:38.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.551+0000 7f312cff9640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 0x7f311006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:38.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.551+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3118088c50 con 0x7f31300afbe0 2023-10-21T16:03:38.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.551+0000 7f312ffff640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 0x7f311006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:38.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.553+0000 7f312ffff640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 0x7f311006d380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3120036000 tx=0x7f3120030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:38.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.555+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3118053ce0 con 0x7f31300afbe0 2023-10-21T16:03:38.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:38.788+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f313004e160 con 0x7f31300afbe0 2023-10-21T16:03:39.190 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-10-21T16:03:39.189+0000 7fcc0f14c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:03:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.281+0000 7f312cff9640 1 -- 172.21.15.53:0/4061244123 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3118017020 con 0x7f31300afbe0 2023-10-21T16:03:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 msgr2=0x7f311006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 0x7f311006d380 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3120036000 tx=0x7f3120030000 comp rx=0 tx=0).stop 2023-10-21T16:03:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 msgr2=0x7f313015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f311800c500 tx=0x7f311800c9c0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 shutdown_connections 2023-10-21T16:03:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31300cb130 0x7f313015cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f311006aec0 0x7f311006d380 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31300afbe0 0x7f313015c840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 --2- 172.21.15.53:0/4061244123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313006dfc0 0x7f3130158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 >> 172.21.15.53:0/4061244123 conn(0x7f313006a5d0 msgr2=0x7f313006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:39.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.284+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 shutdown_connections 2023-10-21T16:03:39.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.285+0000 7f3136400640 1 -- 172.21.15.53:0/4061244123 wait complete. 2023-10-21T16:03:39.309 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:03:39.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:39.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.534+0000 7f857effb640 1 Processor -- start 2023-10-21T16:03:39.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.534+0000 7f857effb640 1 -- start start 2023-10-21T16:03:39.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.535+0000 7f857effb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f857815cc60 0x7f85780cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.535+0000 7f857effb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f85780ce6e0 0x7f85780cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.535+0000 7f857effb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857815d040 con 0x7f85780d16f0 2023-10-21T16:03:39.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.535+0000 7f857effb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780d0650 con 0x7f857815cc60 2023-10-21T16:03:39.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.535+0000 7f857effb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780d0930 con 0x7f85780ce6e0 2023-10-21T16:03:39.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f857815cc60 0x7f85780cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f857815cc60 0x7f85780cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52528/0 (socket says 172.21.15.53:52528) 2023-10-21T16:03:39.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 -- 172.21.15.53:0/3356143541 learned_addr learned my addr 172.21.15.53:0/3356143541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:39.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857e7fa640 1 --2- 172.21.15.53:0/3356143541 >> v2:172.21.15.124:3300/0 conn(0x7f85780ce6e0 0x7f85780cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 -- 172.21.15.53:0/3356143541 >> v2:172.21.15.124:3300/0 conn(0x7f85780ce6e0 msgr2=0x7f85780cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:39.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 --2- 172.21.15.53:0/3356143541 >> v2:172.21.15.124:3300/0 conn(0x7f85780ce6e0 0x7f85780cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 -- 172.21.15.53:0/3356143541 >> v1:172.21.15.124:6789/0 conn(0x7f85780d16f0 legacy=0x7f85780cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:39.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.536+0000 7f857dff9640 1 -- 172.21.15.53:0/3356143541 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780d0c10 con 0x7f857815cc60 2023-10-21T16:03:39.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.537+0000 7f857dff9640 1 --2- 172.21.15.53:0/3356143541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857815cc60 0x7f85780cf340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f85780b8980 tx=0x7f857402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246a13ecceddb8d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:39.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857cff7640 1 -- 172.21.15.53:0/3356143541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857402cad0 con 0x7f857815cc60 2023-10-21T16:03:39.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857cff7640 1 -- 172.21.15.53:0/3356143541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857402cc30 con 0x7f857815cc60 2023-10-21T16:03:39.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857cff7640 1 -- 172.21.15.53:0/3356143541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85740343f0 con 0x7f857815cc60 2023-10-21T16:03:39.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 -- 172.21.15.53:0/3356143541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857815cc60 msgr2=0x7f85780cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 --2- 172.21.15.53:0/3356143541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857815cc60 0x7f85780cf340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f85780b8980 tx=0x7f857402bf30 comp rx=0 tx=0).stop 2023-10-21T16:03:39.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 -- 172.21.15.53:0/3356143541 shutdown_connections 2023-10-21T16:03:39.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 --2- 172.21.15.53:0/3356143541 >> v2:172.21.15.124:3300/0 conn(0x7f85780ce6e0 0x7f85780cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 --2- 172.21.15.53:0/3356143541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f857815cc60 0x7f85780cf340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 -- 172.21.15.53:0/3356143541 >> 172.21.15.53:0/3356143541 conn(0x7f857809bcf0 msgr2=0x7f85780d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:39.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 -- 172.21.15.53:0/3356143541 shutdown_connections 2023-10-21T16:03:39.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.538+0000 7f857effb640 1 -- 172.21.15.53:0/3356143541 wait complete. 2023-10-21T16:03:39.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.540+0000 7f857effb640 1 Processor -- start 2023-10-21T16:03:39.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.540+0000 7f857effb640 1 -- start start 2023-10-21T16:03:39.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85780ce6e0 0x7f8578153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f857815cc60 0x7f857815ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85781f6620 con 0x7f85780d16f0 2023-10-21T16:03:39.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c7160 con 0x7f85780ce6e0 2023-10-21T16:03:39.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.541+0000 7f857effb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c7440 con 0x7f857815cc60 2023-10-21T16:03:39.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52534/0 (socket says 172.21.15.53:52534) 2023-10-21T16:03:39.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 -- 172.21.15.53:0/934428953 learned_addr learned my addr 172.21.15.53:0/934428953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:39.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857dff9640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85780ce6e0 0x7f8578153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857e7fa640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f857815cc60 0x7f857815ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 -- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f857815cc60 msgr2=0x7f857815ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f857815cc60 0x7f857815ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 -- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85780ce6e0 msgr2=0x7f8578153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85780ce6e0 0x7f8578153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 -- 172.21.15.53:0/934428953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009b30 con 0x7f85780d16f0 2023-10-21T16:03:39.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.542+0000 7f857d7f8640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f856800b3f0 tx=0x7f856800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:39.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.543+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568011830 con 0x7f85780d16f0 2023-10-21T16:03:39.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.543+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568011990 con 0x7f85780d16f0 2023-10-21T16:03:39.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.543+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85780c7720 con 0x7f85780d16f0 2023-10-21T16:03:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.543+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568011d40 con 0x7f85780d16f0 2023-10-21T16:03:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.543+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85780c7ce0 con 0x7f85780d16f0 2023-10-21T16:03:39.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.544+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857804e160 con 0x7f85780d16f0 2023-10-21T16:03:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.545+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f856801a400 con 0x7f85780d16f0 2023-10-21T16:03:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.546+0000 7f856e7fc640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 0x7f855c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:39.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.546+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f85680895a0 con 0x7f85780d16f0 2023-10-21T16:03:39.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.547+0000 7f857dff9640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 0x7f855c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:39.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.548+0000 7f857dff9640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 0x7f855c06d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8574035000 tx=0x7f8574030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:39.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.550+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8568054500 con 0x7f85780d16f0 2023-10-21T16:03:39.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.779+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85780d3080 con 0x7f85780d16f0 2023-10-21T16:03:39.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.781+0000 7f856e7fc640 1 -- 172.21.15.53:0/934428953 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f8568057b20 con 0x7f85780d16f0 2023-10-21T16:03:39.782 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:39.783 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:03:39.783 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:03:39.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 msgr2=0x7f855c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 0x7f855c06d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8574035000 tx=0x7f8574030000 comp rx=0 tx=0).stop 2023-10-21T16:03:39.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 msgr2=0x7f85781f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:39.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f856800b3f0 tx=0x7f856800b8b0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 shutdown_connections 2023-10-21T16:03:39.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f857815cc60 0x7f857815ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f855c06ae70 0x7f855c06d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85780d16f0 0x7f85781f60e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 --2- 172.21.15.53:0/934428953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85780ce6e0 0x7f8578153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:39.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 >> 172.21.15.53:0/934428953 conn(0x7f857809bcf0 msgr2=0x7f85780b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:39.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 shutdown_connections 2023-10-21T16:03:39.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:39.783+0000 7f857effb640 1 -- 172.21.15.53:0/934428953 wait complete. 2023-10-21T16:03:39.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:03:40.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.023+0000 7f8ac139e640 1 Processor -- start 2023-10-21T16:03:40.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.023+0000 7f8ac139e640 1 -- start start 2023-10-21T16:03:40.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8ac139e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8abc165cc0 0x7f8abc1680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8ac139e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0c48f0 con 0x7f8abc160940 2023-10-21T16:03:40.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8ac139e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0710f0 con 0x7f8abc15cd90 2023-10-21T16:03:40.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8ac139e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0c4d80 con 0x7f8abc165cc0 2023-10-21T16:03:40.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8aba7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8abc160940 0x7f8abc0709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55656/0 (socket says 172.21.15.53:55656) 2023-10-21T16:03:40.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8aba7fc640 1 -- 172.21.15.53:0/1482792902 learned_addr learned my addr 172.21.15.53:0/1482792902 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:40.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.025+0000 7f8abb7fe640 1 --2- 172.21.15.53:0/1482792902 >> v2:172.21.15.53:3300/0 conn(0x7f8abc165cc0 0x7f8abc1680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.026+0000 7f8ab9ffb640 1 -- 172.21.15.53:0/1482792902 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652278784 0 0) 0x7f8abc0c48f0 con 0x7f8abc160940 2023-10-21T16:03:40.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.026+0000 7f8ab9ffb640 1 -- 172.21.15.53:0/1482792902 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa8003520 con 0x7f8abc160940 2023-10-21T16:03:40.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.026+0000 7f8abb7fe640 1 -- 172.21.15.53:0/1482792902 >> v1:172.21.15.124:6789/0 conn(0x7f8abc15cd90 legacy=0x7f8abc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.026+0000 7f8abb7fe640 1 -- 172.21.15.53:0/1482792902 >> v1:172.21.15.53:6789/0 conn(0x7f8abc160940 legacy=0x7f8abc0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.026+0000 7f8abb7fe640 1 -- 172.21.15.53:0/1482792902 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0c3ce0 con 0x7f8abc165cc0 2023-10-21T16:03:40.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.027+0000 7f8abb7fe640 1 --2- 172.21.15.53:0/1482792902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc165cc0 0x7f8abc1680b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097d0 tx=0x7f8ab402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e726f7e3c220d180 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.027+0000 7f8ab9ffb640 1 -- 172.21.15.53:0/1482792902 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4030800 con 0x7f8abc165cc0 2023-10-21T16:03:40.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.027+0000 7f8ab9ffb640 1 -- 172.21.15.53:0/1482792902 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4030960 con 0x7f8abc165cc0 2023-10-21T16:03:40.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.027+0000 7f8ab9ffb640 1 -- 172.21.15.53:0/1482792902 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4030cf0 con 0x7f8abc165cc0 2023-10-21T16:03:40.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 -- 172.21.15.53:0/1482792902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc165cc0 msgr2=0x7f8abc1680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 --2- 172.21.15.53:0/1482792902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc165cc0 0x7f8abc1680b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097d0 tx=0x7f8ab402b940 comp rx=0 tx=0).stop 2023-10-21T16:03:40.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 -- 172.21.15.53:0/1482792902 shutdown_connections 2023-10-21T16:03:40.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 --2- 172.21.15.53:0/1482792902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc165cc0 0x7f8abc1680b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 -- 172.21.15.53:0/1482792902 >> 172.21.15.53:0/1482792902 conn(0x7f8abc06a380 msgr2=0x7f8abc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:40.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 -- 172.21.15.53:0/1482792902 shutdown_connections 2023-10-21T16:03:40.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.028+0000 7f8ac139e640 1 -- 172.21.15.53:0/1482792902 wait complete. 2023-10-21T16:03:40.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.030+0000 7f8ac139e640 1 Processor -- start 2023-10-21T16:03:40.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.030+0000 7f8ac139e640 1 -- start start 2023-10-21T16:03:40.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8abc15cd90 0x7f8abc0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 0x7f8abc0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0addd0 con 0x7f8abc160940 2023-10-21T16:03:40.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0c9370 con 0x7f8abc165cc0 2023-10-21T16:03:40.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8ac139e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0ad1c0 con 0x7f8abc15cd90 2023-10-21T16:03:40.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.031+0000 7f8aba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52558/0 (socket says 172.21.15.53:52558) 2023-10-21T16:03:40.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 -- 172.21.15.53:0/2523252652 learned_addr learned my addr 172.21.15.53:0/2523252652 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:40.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8abaffd640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8abc15cd90 0x7f8abc0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8abb7fe640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 0x7f8abc0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 -- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8abc15cd90 msgr2=0x7f8abc0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8abc15cd90 0x7f8abc0acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 -- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 msgr2=0x7f8abc0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 0x7f8abc0ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 -- 172.21.15.53:0/2523252652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009400 con 0x7f8abc160940 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8abb7fe640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 0x7f8abc0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.032+0000 7f8aba7fc640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cbf0 tx=0x7f8aa400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.033+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400ad50 con 0x7f8abc160940 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.033+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4012400 con 0x7f8abc160940 2023-10-21T16:03:40.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.033+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4012790 con 0x7f8abc160940 2023-10-21T16:03:40.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.033+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0ad4a0 con 0x7f8abc160940 2023-10-21T16:03:40.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.033+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0ada60 con 0x7f8abc160940 2023-10-21T16:03:40.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.034+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc067490 con 0x7f8abc160940 2023-10-21T16:03:40.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.035+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8aa4007070 con 0x7f8abc160940 2023-10-21T16:03:40.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.036+0000 7f8ab37fe640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 0x7f8a9c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.036+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8aa4089840 con 0x7f8abc160940 2023-10-21T16:03:40.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.037+0000 7f8abaffd640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 0x7f8a9c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.038+0000 7f8abaffd640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 0x7f8a9c06d360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0cf620 tx=0x7f8aac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.040+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8aa4054710 con 0x7f8abc160940 2023-10-21T16:03:40.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.235+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc0b0ef0 con 0x7f8abc160940 2023-10-21T16:03:40.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.238+0000 7f8ab37fe640 1 -- 172.21.15.53:0/2523252652 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16762 (secure 0 0 0) 0x7f8aa4057d30 con 0x7f8abc160940 2023-10-21T16:03:40.240 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:40.240 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":40,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:03:36.939133+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:03:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.240+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 msgr2=0x7f8a9c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.240+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 0x7f8a9c06d360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0cf620 tx=0x7f8aac009040 comp rx=0 tx=0).stop 2023-10-21T16:03:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.240+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 msgr2=0x7f8abc0ac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.240+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400cbf0 tx=0x7f8aa400a430 comp rx=0 tx=0).stop 2023-10-21T16:03:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 shutdown_connections 2023-10-21T16:03:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8abc165cc0 0x7f8abc0ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f8a9c06aea0 0x7f8a9c06d360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8abc160940 0x7f8abc0ac480 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 --2- 172.21.15.53:0/2523252652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8abc15cd90 0x7f8abc0acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 >> 172.21.15.53:0/2523252652 conn(0x7f8abc06a380 msgr2=0x7f8abc153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 shutdown_connections 2023-10-21T16:03:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.241+0000 7f8ac139e640 1 -- 172.21.15.53:0/2523252652 wait complete. 2023-10-21T16:03:40.263 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:03:40.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:03:40.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.482+0000 7f4a0d6da640 1 Processor -- start 2023-10-21T16:03:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.483+0000 7f4a0d6da640 1 -- start start 2023-10-21T16:03:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a0d6da640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8d0 0x7f4a080cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a0d6da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a080b8bf0 con 0x7f4a080b1840 2023-10-21T16:03:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a0d6da640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a080d39b0 con 0x7f4a080b1d90 2023-10-21T16:03:40.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a0d6da640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a080d2910 con 0x7f4a080cd8d0 2023-10-21T16:03:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a067fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4a080b1840 0x7f4a080d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55662/0 (socket says 172.21.15.53:55662) 2023-10-21T16:03:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.484+0000 7f4a067fc640 1 -- 172.21.15.53:0/2988761734 learned_addr learned my addr 172.21.15.53:0/2988761734 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:40.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a06ffd640 1 --2- 172.21.15.53:0/2988761734 >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8d0 0x7f4a080cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268780725 0 0) 0x7f4a080b8bf0 con 0x7f4a080b1840 2023-10-21T16:03:40.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49f4003520 con 0x7f4a080b1840 2023-10-21T16:03:40.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170593124 0 0) 0x7f49f4003520 con 0x7f4a080b1840 2023-10-21T16:03:40.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8d0 msgr2=0x7f4a080cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:40.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/2988761734 >> v2:172.21.15.124:3300/0 conn(0x7f4a080cd8d0 0x7f4a080cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 >> v1:172.21.15.124:6789/0 conn(0x7f4a080b1d90 legacy=0x7f4a080d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080ce1f0 con 0x7f4a080b1840 2023-10-21T16:03:40.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f49f0001670 con 0x7f4a080b1840 2023-10-21T16:03:40.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 >> v1:172.21.15.53:6789/0 conn(0x7f4a080b1840 legacy=0x7f4a080d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49f4003970 0x7f49f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f40047c0 0x7f49f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 0x7f49f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080d2910 con 0x7f49f4005180 2023-10-21T16:03:40.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f400c020 con 0x7f49f4003970 2023-10-21T16:03:40.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.485+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f400c1e0 con 0x7f49f40047c0 2023-10-21T16:03:40.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 0x7f49f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a06ffd640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49f4003970 0x7f49f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a067fc640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f40047c0 0x7f49f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 -- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f40047c0 msgr2=0x7f49f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:40.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f40047c0 0x7f49f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 -- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49f4003970 msgr2=0x7f49f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:40.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49f4003970 0x7f49f4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.486+0000 7f4a077fe640 1 -- 172.21.15.53:0/2988761734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4003520 con 0x7f49f4005180 2023-10-21T16:03:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.487+0000 7f4a077fe640 1 -- 172.21.15.53:0/2988761734 reap_dead start 2023-10-21T16:03:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.487+0000 7f4a077fe640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 0x7f49f400b980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001750 tx=0x7f49f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551160dcfb9874bd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000052d0 con 0x7f49f4005180 2023-10-21T16:03:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0002e050 con 0x7f49f4005180 2023-10-21T16:03:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a05ffb640 1 -- 172.21.15.53:0/2988761734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00003830 con 0x7f49f4005180 2023-10-21T16:03:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 -- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 msgr2=0x7f49f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 0x7f49f400b980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001750 tx=0x7f49f40057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 -- 172.21.15.53:0/2988761734 shutdown_connections 2023-10-21T16:03:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/2988761734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49f4005180 0x7f49f400b980 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 -- 172.21.15.53:0/2988761734 >> 172.21.15.53:0/2988761734 conn(0x7f4a0806b6a0 msgr2=0x7f4a0806bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 -- 172.21.15.53:0/2988761734 shutdown_connections 2023-10-21T16:03:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.488+0000 7f4a0d6da640 1 -- 172.21.15.53:0/2988761734 wait complete. 2023-10-21T16:03:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.490+0000 7f4a0d6da640 1 Processor -- start 2023-10-21T16:03:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.490+0000 7f4a0d6da640 1 -- start start 2023-10-21T16:03:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f4005180 0x7f4a080c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080ca250 0x7f4a080d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080d3550 con 0x7f4a080d30d0 2023-10-21T16:03:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080cb3a0 con 0x7f4a080ca250 2023-10-21T16:03:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a0d6da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080ca790 con 0x7f49f4005180 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.491+0000 7f4a077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52578/0 (socket says 172.21.15.53:52578) 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 -- 172.21.15.53:0/917411412 learned_addr learned my addr 172.21.15.53:0/917411412 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a067fc640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080ca250 0x7f4a080d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a06ffd640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f4005180 0x7f4a080c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 -- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f4005180 msgr2=0x7f4a080c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f4005180 0x7f4a080c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 -- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080ca250 msgr2=0x7f4a080d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080ca250 0x7f4a080d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 -- 172.21.15.53:0/917411412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080caa70 con 0x7f4a080d30d0 2023-10-21T16:03:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.492+0000 7f4a077fe640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001870 tx=0x7f4a00001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.493+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00039070 con 0x7f4a080d30d0 2023-10-21T16:03:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.493+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00005450 con 0x7f4a080d30d0 2023-10-21T16:03:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.493+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007230 con 0x7f4a080d30d0 2023-10-21T16:03:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.493+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a080cad50 con 0x7f4a080d30d0 2023-10-21T16:03:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.493+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a080cb830 con 0x7f4a080d30d0 2023-10-21T16:03:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.494+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a08066d80 con 0x7f4a080d30d0 2023-10-21T16:03:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.495+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a0003c020 con 0x7f4a080d30d0 2023-10-21T16:03:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.496+0000 7f49ff7fe640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.496+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4a000a9a80 con 0x7f4a080d30d0 2023-10-21T16:03:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.497+0000 7f4a06ffd640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:40.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.498+0000 7f4a06ffd640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009fc0 tx=0x7f49f800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.500+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a00074a90 con 0x7f4a080d30d0 2023-10-21T16:03:40.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.691+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a080b8910 con 0x7f49e806aea0 2023-10-21T16:03:40.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.717+0000 7f49ff7fe640 1 -- 172.21.15.53:0/917411412 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286512 (secure 0 0 0) 0x7f49f8009c30 con 0x7f49e806aea0 2023-10-21T16:03:40.723 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:40.725 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:03:40.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 msgr2=0x7f49e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009fc0 tx=0x7f49f800b040 comp rx=0 tx=0).stop 2023-10-21T16:03:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 msgr2=0x7f4a08156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4a00001870 tx=0x7f4a00001120 comp rx=0 tx=0).stop 2023-10-21T16:03:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 shutdown_connections 2023-10-21T16:03:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f49e806aea0 0x7f49e806d360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a080d30d0 0x7f4a08156e10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a080ca250 0x7f4a080d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 --2- 172.21.15.53:0/917411412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49f4005180 0x7f4a080c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 >> 172.21.15.53:0/917411412 conn(0x7f4a0806b6a0 msgr2=0x7f4a0815c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 shutdown_connections 2023-10-21T16:03:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:40.724+0000 7f4a0d6da640 1 -- 172.21.15.53:0/917411412 wait complete. 2023-10-21T16:03:40.749 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-10-21T16:03:39.055122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237724,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736996,"statfs":{"total":773094113280,"available":772850683904,"internally_reserved":0,"allocated":4956160,"data_stored":2792724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845124"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110282+0000","last_change":"2023-10-21T16:03:35.968487+0000","last_active":"2023-10-21T16:03:37.110282+0000","last_peered":"2023-10-21T16:03:37.110282+0000","last_clean":"2023-10-21T16:03:37.110282+0000","last_became_active":"2023-10-21T16:03:35.967993+0000","last_became_peered":"2023-10-21T16:03:35.967993+0000","last_unstale":"2023-10-21T16:03:37.110282+0000","last_undegraded":"2023-10-21T16:03:37.110282+0000","last_fullsized":"2023-10-21T16:03:37.110282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:05.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109538+0000","last_change":"2023-10-21T16:03:35.976671+0000","last_active":"2023-10-21T16:03:37.109538+0000","last_peered":"2023-10-21T16:03:37.109538+0000","last_clean":"2023-10-21T16:03:37.109538+0000","last_became_active":"2023-10-21T16:03:35.976051+0000","last_became_peered":"2023-10-21T16:03:35.976051+0000","last_unstale":"2023-10-21T16:03:37.109538+0000","last_undegraded":"2023-10-21T16:03:37.109538+0000","last_fullsized":"2023-10-21T16:03:37.109538+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:43:31.874405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110186+0000","last_change":"2023-10-21T16:03:35.973243+0000","last_active":"2023-10-21T16:03:37.110186+0000","last_peered":"2023-10-21T16:03:37.110186+0000","last_clean":"2023-10-21T16:03:37.110186+0000","last_became_active":"2023-10-21T16:03:35.972967+0000","last_became_peered":"2023-10-21T16:03:35.972967+0000","last_unstale":"2023-10-21T16:03:37.110186+0000","last_undegraded":"2023-10-21T16:03:37.110186+0000","last_fullsized":"2023-10-21T16:03:37.110186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948195+0000","last_change":"2023-10-21T16:03:35.968486+0000","last_active":"2023-10-21T16:03:36.948195+0000","last_peered":"2023-10-21T16:03:36.948195+0000","last_clean":"2023-10-21T16:03:36.948195+0000","last_became_active":"2023-10-21T16:03:35.968120+0000","last_became_peered":"2023-10-21T16:03:35.968120+0000","last_unstale":"2023-10-21T16:03:36.948195+0000","last_undegraded":"2023-10-21T16:03:36.948195+0000","last_fullsized":"2023-10-21T16:03:36.948195+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:29.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.947993+0000","last_change":"2023-10-21T16:03:35.973338+0000","last_active":"2023-10-21T16:03:36.947993+0000","last_peered":"2023-10-21T16:03:36.947993+0000","last_clean":"2023-10-21T16:03:36.947993+0000","last_became_active":"2023-10-21T16:03:35.972648+0000","last_became_peered":"2023-10-21T16:03:35.972648+0000","last_unstale":"2023-10-21T16:03:36.947993+0000","last_undegraded":"2023-10-21T16:03:36.947993+0000","last_fullsized":"2023-10-21T16:03:36.947993+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:10.275412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944626+0000","last_change":"2023-10-21T16:03:35.961394+0000","last_active":"2023-10-21T16:03:36.944626+0000","last_peered":"2023-10-21T16:03:36.944626+0000","last_clean":"2023-10-21T16:03:36.944626+0000","last_became_active":"2023-10-21T16:03:35.961091+0000","last_became_peered":"2023-10-21T16:03:35.961091+0000","last_unstale":"2023-10-21T16:03:36.944626+0000","last_undegraded":"2023-10-21T16:03:36.944626+0000","last_fullsized":"2023-10-21T16:03:36.944626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111171+0000","last_change":"2023-10-21T16:03:35.975808+0000","last_active":"2023-10-21T16:03:37.111171+0000","last_peered":"2023-10-21T16:03:37.111171+0000","last_clean":"2023-10-21T16:03:37.111171+0000","last_became_active":"2023-10-21T16:03:35.975182+0000","last_became_peered":"2023-10-21T16:03:35.975182+0000","last_unstale":"2023-10-21T16:03:37.111171+0000","last_undegraded":"2023-10-21T16:03:37.111171+0000","last_fullsized":"2023-10-21T16:03:37.111171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:06.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108410+0000","last_change":"2023-10-21T16:03:35.976523+0000","last_active":"2023-10-21T16:03:37.108410+0000","last_peered":"2023-10-21T16:03:37.108410+0000","last_clean":"2023-10-21T16:03:37.108410+0000","last_became_active":"2023-10-21T16:03:35.976126+0000","last_became_peered":"2023-10-21T16:03:35.976126+0000","last_unstale":"2023-10-21T16:03:37.108410+0000","last_undegraded":"2023-10-21T16:03:37.108410+0000","last_fullsized":"2023-10-21T16:03:37.108410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:24.487305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111033+0000","last_change":"2023-10-21T16:03:35.978945+0000","last_active":"2023-10-21T16:03:37.111033+0000","last_peered":"2023-10-21T16:03:37.111033+0000","last_clean":"2023-10-21T16:03:37.111033+0000","last_became_active":"2023-10-21T16:03:35.978654+0000","last_became_peered":"2023-10-21T16:03:35.978654+0000","last_unstale":"2023-10-21T16:03:37.111033+0000","last_undegraded":"2023-10-21T16:03:37.111033+0000","last_fullsized":"2023-10-21T16:03:37.111033+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:26:17.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113519+0000","last_change":"2023-10-21T16:03:35.979372+0000","last_active":"2023-10-21T16:03:37.113519+0000","last_peered":"2023-10-21T16:03:37.113519+0000","last_clean":"2023-10-21T16:03:37.113519+0000","last_became_active":"2023-10-21T16:03:35.975419+0000","last_became_peered":"2023-10-21T16:03:35.975419+0000","last_unstale":"2023-10-21T16:03:37.113519+0000","last_undegraded":"2023-10-21T16:03:37.113519+0000","last_fullsized":"2023-10-21T16:03:37.113519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:19.547371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113701+0000","last_change":"2023-10-21T16:03:35.971219+0000","last_active":"2023-10-21T16:03:37.113701+0000","last_peered":"2023-10-21T16:03:37.113701+0000","last_clean":"2023-10-21T16:03:37.113701+0000","last_became_active":"2023-10-21T16:03:35.970563+0000","last_became_peered":"2023-10-21T16:03:35.970563+0000","last_unstale":"2023-10-21T16:03:37.113701+0000","last_undegraded":"2023-10-21T16:03:37.113701+0000","last_fullsized":"2023-10-21T16:03:37.113701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:17.247375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110342+0000","last_change":"2023-10-21T16:03:35.977449+0000","last_active":"2023-10-21T16:03:37.110342+0000","last_peered":"2023-10-21T16:03:37.110342+0000","last_clean":"2023-10-21T16:03:37.110342+0000","last_became_active":"2023-10-21T16:03:35.977153+0000","last_became_peered":"2023-10-21T16:03:35.977153+0000","last_unstale":"2023-10-21T16:03:37.110342+0000","last_undegraded":"2023-10-21T16:03:37.110342+0000","last_fullsized":"2023-10-21T16:03:37.110342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:48.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110734+0000","last_change":"2023-10-21T16:03:35.972649+0000","last_active":"2023-10-21T16:03:37.110734+0000","last_peered":"2023-10-21T16:03:37.110734+0000","last_clean":"2023-10-21T16:03:37.110734+0000","last_became_active":"2023-10-21T16:03:35.971694+0000","last_became_peered":"2023-10-21T16:03:35.971694+0000","last_unstale":"2023-10-21T16:03:37.110734+0000","last_undegraded":"2023-10-21T16:03:37.110734+0000","last_fullsized":"2023-10-21T16:03:37.110734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:50.206789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949605+0000","last_change":"2023-10-21T16:03:35.975971+0000","last_active":"2023-10-21T16:03:36.949605+0000","last_peered":"2023-10-21T16:03:36.949605+0000","last_clean":"2023-10-21T16:03:36.949605+0000","last_became_active":"2023-10-21T16:03:35.975686+0000","last_became_peered":"2023-10-21T16:03:35.975686+0000","last_unstale":"2023-10-21T16:03:36.949605+0000","last_undegraded":"2023-10-21T16:03:36.949605+0000","last_fullsized":"2023-10-21T16:03:36.949605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111243+0000","last_change":"2023-10-21T16:03:35.976446+0000","last_active":"2023-10-21T16:03:37.111243+0000","last_peered":"2023-10-21T16:03:37.111243+0000","last_clean":"2023-10-21T16:03:37.111243+0000","last_became_active":"2023-10-21T16:03:35.976091+0000","last_became_peered":"2023-10-21T16:03:35.976091+0000","last_unstale":"2023-10-21T16:03:37.111243+0000","last_undegraded":"2023-10-21T16:03:37.111243+0000","last_fullsized":"2023-10-21T16:03:37.111243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:30:28.375486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109302+0000","last_change":"2023-10-21T16:03:35.972767+0000","last_active":"2023-10-21T16:03:37.109302+0000","last_peered":"2023-10-21T16:03:37.109302+0000","last_clean":"2023-10-21T16:03:37.109302+0000","last_became_active":"2023-10-21T16:03:35.972472+0000","last_became_peered":"2023-10-21T16:03:35.972472+0000","last_unstale":"2023-10-21T16:03:37.109302+0000","last_undegraded":"2023-10-21T16:03:37.109302+0000","last_fullsized":"2023-10-21T16:03:37.109302+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:30.658334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113049+0000","last_change":"2023-10-21T16:03:35.979501+0000","last_active":"2023-10-21T16:03:37.113049+0000","last_peered":"2023-10-21T16:03:37.113049+0000","last_clean":"2023-10-21T16:03:37.113049+0000","last_became_active":"2023-10-21T16:03:35.977443+0000","last_became_peered":"2023-10-21T16:03:35.977443+0000","last_unstale":"2023-10-21T16:03:37.113049+0000","last_undegraded":"2023-10-21T16:03:37.113049+0000","last_fullsized":"2023-10-21T16:03:37.113049+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111408+0000","last_change":"2023-10-21T16:03:35.974760+0000","last_active":"2023-10-21T16:03:37.111408+0000","last_peered":"2023-10-21T16:03:37.111408+0000","last_clean":"2023-10-21T16:03:37.111408+0000","last_became_active":"2023-10-21T16:03:35.973128+0000","last_became_peered":"2023-10-21T16:03:35.973128+0000","last_unstale":"2023-10-21T16:03:37.111408+0000","last_undegraded":"2023-10-21T16:03:37.111408+0000","last_fullsized":"2023-10-21T16:03:37.111408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:17:56.898215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948470+0000","last_change":"2023-10-21T16:03:35.963948+0000","last_active":"2023-10-21T16:03:36.948470+0000","last_peered":"2023-10-21T16:03:36.948470+0000","last_clean":"2023-10-21T16:03:36.948470+0000","last_became_active":"2023-10-21T16:03:35.963610+0000","last_became_peered":"2023-10-21T16:03:35.963610+0000","last_unstale":"2023-10-21T16:03:36.948470+0000","last_undegraded":"2023-10-21T16:03:36.948470+0000","last_fullsized":"2023-10-21T16:03:36.948470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:13.113623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111784+0000","last_change":"2023-10-21T16:03:35.972955+0000","last_active":"2023-10-21T16:03:37.111784+0000","last_peered":"2023-10-21T16:03:37.111784+0000","last_clean":"2023-10-21T16:03:37.111784+0000","last_became_active":"2023-10-21T16:03:35.972517+0000","last_became_peered":"2023-10-21T16:03:35.972517+0000","last_unstale":"2023-10-21T16:03:37.111784+0000","last_undegraded":"2023-10-21T16:03:37.111784+0000","last_fullsized":"2023-10-21T16:03:37.111784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:27.519767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111089+0000","last_change":"2023-10-21T16:03:35.971099+0000","last_active":"2023-10-21T16:03:37.111089+0000","last_peered":"2023-10-21T16:03:37.111089+0000","last_clean":"2023-10-21T16:03:37.111089+0000","last_became_active":"2023-10-21T16:03:35.970803+0000","last_became_peered":"2023-10-21T16:03:35.970803+0000","last_unstale":"2023-10-21T16:03:37.111089+0000","last_undegraded":"2023-10-21T16:03:37.111089+0000","last_fullsized":"2023-10-21T16:03:37.111089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:11:14.049352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950148+0000","last_change":"2023-10-21T16:03:35.983252+0000","last_active":"2023-10-21T16:03:36.950148+0000","last_peered":"2023-10-21T16:03:36.950148+0000","last_clean":"2023-10-21T16:03:36.950148+0000","last_became_active":"2023-10-21T16:03:35.982898+0000","last_became_peered":"2023-10-21T16:03:35.982898+0000","last_unstale":"2023-10-21T16:03:36.950148+0000","last_undegraded":"2023-10-21T16:03:36.950148+0000","last_fullsized":"2023-10-21T16:03:36.950148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112990+0000","last_change":"2023-10-21T16:03:35.974047+0000","last_active":"2023-10-21T16:03:37.112990+0000","last_peered":"2023-10-21T16:03:37.112990+0000","last_clean":"2023-10-21T16:03:37.112990+0000","last_became_active":"2023-10-21T16:03:35.973692+0000","last_became_peered":"2023-10-21T16:03:35.973692+0000","last_unstale":"2023-10-21T16:03:37.112990+0000","last_undegraded":"2023-10-21T16:03:37.112990+0000","last_fullsized":"2023-10-21T16:03:37.112990+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:45.499630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.947991+0000","last_change":"2023-10-21T16:03:35.968156+0000","last_active":"2023-10-21T16:03:36.947991+0000","last_peered":"2023-10-21T16:03:36.947991+0000","last_clean":"2023-10-21T16:03:36.947991+0000","last_became_active":"2023-10-21T16:03:35.967121+0000","last_became_peered":"2023-10-21T16:03:35.967121+0000","last_unstale":"2023-10-21T16:03:36.947991+0000","last_undegraded":"2023-10-21T16:03:36.947991+0000","last_fullsized":"2023-10-21T16:03:36.947991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:36:23.012299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110713+0000","last_change":"2023-10-21T16:03:35.976947+0000","last_active":"2023-10-21T16:03:37.110713+0000","last_peered":"2023-10-21T16:03:37.110713+0000","last_clean":"2023-10-21T16:03:37.110713+0000","last_became_active":"2023-10-21T16:03:35.976672+0000","last_became_peered":"2023-10-21T16:03:35.976672+0000","last_unstale":"2023-10-21T16:03:37.110713+0000","last_undegraded":"2023-10-21T16:03:37.110713+0000","last_fullsized":"2023-10-21T16:03:37.110713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:32.562760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945098+0000","last_change":"2023-10-21T16:03:35.978739+0000","last_active":"2023-10-21T16:03:36.945098+0000","last_peered":"2023-10-21T16:03:36.945098+0000","last_clean":"2023-10-21T16:03:36.945098+0000","last_became_active":"2023-10-21T16:03:35.978395+0000","last_became_peered":"2023-10-21T16:03:35.978395+0000","last_unstale":"2023-10-21T16:03:36.945098+0000","last_undegraded":"2023-10-21T16:03:36.945098+0000","last_fullsized":"2023-10-21T16:03:36.945098+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110847+0000","last_change":"2023-10-21T16:03:35.976745+0000","last_active":"2023-10-21T16:03:37.110847+0000","last_peered":"2023-10-21T16:03:37.110847+0000","last_clean":"2023-10-21T16:03:37.110847+0000","last_became_active":"2023-10-21T16:03:35.976436+0000","last_became_peered":"2023-10-21T16:03:35.976436+0000","last_unstale":"2023-10-21T16:03:37.110847+0000","last_undegraded":"2023-10-21T16:03:37.110847+0000","last_fullsized":"2023-10-21T16:03:37.110847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:40.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112524+0000","last_change":"2023-10-21T16:03:35.970032+0000","last_active":"2023-10-21T16:03:37.112524+0000","last_peered":"2023-10-21T16:03:37.112524+0000","last_clean":"2023-10-21T16:03:37.112524+0000","last_became_active":"2023-10-21T16:03:35.969280+0000","last_became_peered":"2023-10-21T16:03:35.969280+0000","last_unstale":"2023-10-21T16:03:37.112524+0000","last_undegraded":"2023-10-21T16:03:37.112524+0000","last_fullsized":"2023-10-21T16:03:37.112524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:44:57.794604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110887+0000","last_change":"2023-10-21T16:03:35.975780+0000","last_active":"2023-10-21T16:03:37.110887+0000","last_peered":"2023-10-21T16:03:37.110887+0000","last_clean":"2023-10-21T16:03:37.110887+0000","last_became_active":"2023-10-21T16:03:35.972648+0000","last_became_peered":"2023-10-21T16:03:35.972648+0000","last_unstale":"2023-10-21T16:03:37.110887+0000","last_undegraded":"2023-10-21T16:03:37.110887+0000","last_fullsized":"2023-10-21T16:03:37.110887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:03.488974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110430+0000","last_change":"2023-10-21T16:03:35.971689+0000","last_active":"2023-10-21T16:03:37.110430+0000","last_peered":"2023-10-21T16:03:37.110430+0000","last_clean":"2023-10-21T16:03:37.110430+0000","last_became_active":"2023-10-21T16:03:35.971420+0000","last_became_peered":"2023-10-21T16:03:35.971420+0000","last_unstale":"2023-10-21T16:03:37.110430+0000","last_undegraded":"2023-10-21T16:03:37.110430+0000","last_fullsized":"2023-10-21T16:03:37.110430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:40.145592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113274+0000","last_change":"2023-10-21T16:03:35.977272+0000","last_active":"2023-10-21T16:03:37.113274+0000","last_peered":"2023-10-21T16:03:37.113274+0000","last_clean":"2023-10-21T16:03:37.113274+0000","last_became_active":"2023-10-21T16:03:35.974263+0000","last_became_peered":"2023-10-21T16:03:35.974263+0000","last_unstale":"2023-10-21T16:03:37.113274+0000","last_undegraded":"2023-10-21T16:03:37.113274+0000","last_fullsized":"2023-10-21T16:03:37.113274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944456+0000","last_change":"2023-10-21T16:03:35.973976+0000","last_active":"2023-10-21T16:03:36.944456+0000","last_peered":"2023-10-21T16:03:36.944456+0000","last_clean":"2023-10-21T16:03:36.944456+0000","last_became_active":"2023-10-21T16:03:35.973706+0000","last_became_peered":"2023-10-21T16:03:35.973706+0000","last_unstale":"2023-10-21T16:03:36.944456+0000","last_undegraded":"2023-10-21T16:03:36.944456+0000","last_fullsized":"2023-10-21T16:03:36.944456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:45.499630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113758+0000","last_change":"2023-10-21T16:03:35.973726+0000","last_active":"2023-10-21T16:03:37.113758+0000","last_peered":"2023-10-21T16:03:37.113758+0000","last_clean":"2023-10-21T16:03:37.113758+0000","last_became_active":"2023-10-21T16:03:35.971267+0000","last_became_peered":"2023-10-21T16:03:35.971267+0000","last_unstale":"2023-10-21T16:03:37.113758+0000","last_undegraded":"2023-10-21T16:03:37.113758+0000","last_fullsized":"2023-10-21T16:03:37.113758+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110763+0000","last_change":"2023-10-21T16:03:35.251024+0000","last_active":"2023-10-21T16:03:37.110763+0000","last_peered":"2023-10-21T16:03:37.110763+0000","last_clean":"2023-10-21T16:03:37.110763+0000","last_became_active":"2023-10-21T16:03:35.250733+0000","last_became_peered":"2023-10-21T16:03:35.250733+0000","last_unstale":"2023-10-21T16:03:37.110763+0000","last_undegraded":"2023-10-21T16:03:37.110763+0000","last_fullsized":"2023-10-21T16:03:37.110763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:22.567411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945235+0000","last_change":"2023-10-21T16:03:35.971408+0000","last_active":"2023-10-21T16:03:36.945235+0000","last_peered":"2023-10-21T16:03:36.945235+0000","last_clean":"2023-10-21T16:03:36.945235+0000","last_became_active":"2023-10-21T16:03:35.970832+0000","last_became_peered":"2023-10-21T16:03:35.970832+0000","last_unstale":"2023-10-21T16:03:36.945235+0000","last_undegraded":"2023-10-21T16:03:36.945235+0000","last_fullsized":"2023-10-21T16:03:36.945235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110576+0000","last_change":"2023-10-21T16:03:35.256409+0000","last_active":"2023-10-21T16:03:37.110576+0000","last_peered":"2023-10-21T16:03:37.110576+0000","last_clean":"2023-10-21T16:03:37.110576+0000","last_became_active":"2023-10-21T16:03:35.256124+0000","last_became_peered":"2023-10-21T16:03:35.256124+0000","last_unstale":"2023-10-21T16:03:37.110576+0000","last_undegraded":"2023-10-21T16:03:37.110576+0000","last_fullsized":"2023-10-21T16:03:37.110576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:28.145159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950678+0000","last_change":"2023-10-21T16:03:35.980000+0000","last_active":"2023-10-21T16:03:36.950678+0000","last_peered":"2023-10-21T16:03:36.950678+0000","last_clean":"2023-10-21T16:03:36.950678+0000","last_became_active":"2023-10-21T16:03:35.979382+0000","last_became_peered":"2023-10-21T16:03:35.979382+0000","last_unstale":"2023-10-21T16:03:36.950678+0000","last_undegraded":"2023-10-21T16:03:36.950678+0000","last_fullsized":"2023-10-21T16:03:36.950678+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:48.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945305+0000","last_change":"2023-10-21T16:03:35.255798+0000","last_active":"2023-10-21T16:03:36.945305+0000","last_peered":"2023-10-21T16:03:36.945305+0000","last_clean":"2023-10-21T16:03:36.945305+0000","last_became_active":"2023-10-21T16:03:35.255452+0000","last_became_peered":"2023-10-21T16:03:35.255452+0000","last_unstale":"2023-10-21T16:03:36.945305+0000","last_undegraded":"2023-10-21T16:03:36.945305+0000","last_fullsized":"2023-10-21T16:03:36.945305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:26.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110026+0000","last_change":"2023-10-21T16:03:35.977872+0000","last_active":"2023-10-21T16:03:37.110026+0000","last_peered":"2023-10-21T16:03:37.110026+0000","last_clean":"2023-10-21T16:03:37.110026+0000","last_became_active":"2023-10-21T16:03:35.977603+0000","last_became_peered":"2023-10-21T16:03:35.977603+0000","last_unstale":"2023-10-21T16:03:37.110026+0000","last_undegraded":"2023-10-21T16:03:37.110026+0000","last_fullsized":"2023-10-21T16:03:37.110026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:06.085872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950255+0000","last_change":"2023-10-21T16:03:35.248275+0000","last_active":"2023-10-21T16:03:36.950255+0000","last_peered":"2023-10-21T16:03:36.950255+0000","last_clean":"2023-10-21T16:03:36.950255+0000","last_became_active":"2023-10-21T16:03:35.248029+0000","last_became_peered":"2023-10-21T16:03:35.248029+0000","last_unstale":"2023-10-21T16:03:36.950255+0000","last_undegraded":"2023-10-21T16:03:36.950255+0000","last_fullsized":"2023-10-21T16:03:36.950255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:02.475292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109565+0000","last_change":"2023-10-21T16:03:35.968152+0000","last_active":"2023-10-21T16:03:37.109565+0000","last_peered":"2023-10-21T16:03:37.109565+0000","last_clean":"2023-10-21T16:03:37.109565+0000","last_became_active":"2023-10-21T16:03:35.966209+0000","last_became_peered":"2023-10-21T16:03:35.966209+0000","last_unstale":"2023-10-21T16:03:37.109565+0000","last_undegraded":"2023-10-21T16:03:37.109565+0000","last_fullsized":"2023-10-21T16:03:37.109565+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111325+0000","last_change":"2023-10-21T16:03:35.251641+0000","last_active":"2023-10-21T16:03:37.111325+0000","last_peered":"2023-10-21T16:03:37.111325+0000","last_clean":"2023-10-21T16:03:37.111325+0000","last_became_active":"2023-10-21T16:03:35.251344+0000","last_became_peered":"2023-10-21T16:03:35.251344+0000","last_unstale":"2023-10-21T16:03:37.111325+0000","last_undegraded":"2023-10-21T16:03:37.111325+0000","last_fullsized":"2023-10-21T16:03:37.111325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:06.733145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111538+0000","last_change":"2023-10-21T16:03:35.978958+0000","last_active":"2023-10-21T16:03:37.111538+0000","last_peered":"2023-10-21T16:03:37.111538+0000","last_clean":"2023-10-21T16:03:37.111538+0000","last_became_active":"2023-10-21T16:03:35.978671+0000","last_became_peered":"2023-10-21T16:03:35.978671+0000","last_unstale":"2023-10-21T16:03:37.111538+0000","last_undegraded":"2023-10-21T16:03:37.111538+0000","last_fullsized":"2023-10-21T16:03:37.111538+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:01.855993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112148+0000","last_change":"2023-10-21T16:03:35.256677+0000","last_active":"2023-10-21T16:03:37.112148+0000","last_peered":"2023-10-21T16:03:37.112148+0000","last_clean":"2023-10-21T16:03:37.112148+0000","last_became_active":"2023-10-21T16:03:35.256416+0000","last_became_peered":"2023-10-21T16:03:35.256416+0000","last_unstale":"2023-10-21T16:03:37.112148+0000","last_undegraded":"2023-10-21T16:03:37.112148+0000","last_fullsized":"2023-10-21T16:03:37.112148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:09.261730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950196+0000","last_change":"2023-10-21T16:03:35.973558+0000","last_active":"2023-10-21T16:03:36.950196+0000","last_peered":"2023-10-21T16:03:36.950196+0000","last_clean":"2023-10-21T16:03:36.950196+0000","last_became_active":"2023-10-21T16:03:35.972353+0000","last_became_peered":"2023-10-21T16:03:35.972353+0000","last_unstale":"2023-10-21T16:03:36.950196+0000","last_undegraded":"2023-10-21T16:03:36.950196+0000","last_fullsized":"2023-10-21T16:03:36.950196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944730+0000","last_change":"2023-10-21T16:03:35.257801+0000","last_active":"2023-10-21T16:03:36.944730+0000","last_peered":"2023-10-21T16:03:36.944730+0000","last_clean":"2023-10-21T16:03:36.944730+0000","last_became_active":"2023-10-21T16:03:35.257497+0000","last_became_peered":"2023-10-21T16:03:35.257497+0000","last_unstale":"2023-10-21T16:03:36.944730+0000","last_undegraded":"2023-10-21T16:03:36.944730+0000","last_fullsized":"2023-10-21T16:03:36.944730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:56.238578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108839+0000","last_change":"2023-10-21T16:03:35.975278+0000","last_active":"2023-10-21T16:03:37.108839+0000","last_peered":"2023-10-21T16:03:37.108839+0000","last_clean":"2023-10-21T16:03:37.108839+0000","last_became_active":"2023-10-21T16:03:35.974029+0000","last_became_peered":"2023-10-21T16:03:35.974029+0000","last_unstale":"2023-10-21T16:03:37.108839+0000","last_undegraded":"2023-10-21T16:03:37.108839+0000","last_fullsized":"2023-10-21T16:03:37.108839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:52.770641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949491+0000","last_change":"2023-10-21T16:03:35.242677+0000","last_active":"2023-10-21T16:03:36.949491+0000","last_peered":"2023-10-21T16:03:36.949491+0000","last_clean":"2023-10-21T16:03:36.949491+0000","last_became_active":"2023-10-21T16:03:35.242319+0000","last_became_peered":"2023-10-21T16:03:35.242319+0000","last_unstale":"2023-10-21T16:03:36.949491+0000","last_undegraded":"2023-10-21T16:03:36.949491+0000","last_fullsized":"2023-10-21T16:03:36.949491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:13.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948553+0000","last_change":"2023-10-21T16:03:35.973905+0000","last_active":"2023-10-21T16:03:36.948553+0000","last_peered":"2023-10-21T16:03:36.948553+0000","last_clean":"2023-10-21T16:03:36.948553+0000","last_became_active":"2023-10-21T16:03:35.973634+0000","last_became_peered":"2023-10-21T16:03:35.973634+0000","last_unstale":"2023-10-21T16:03:36.948553+0000","last_undegraded":"2023-10-21T16:03:36.948553+0000","last_fullsized":"2023-10-21T16:03:36.948553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:29.567131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110240+0000","last_change":"2023-10-21T16:03:35.249161+0000","last_active":"2023-10-21T16:03:37.110240+0000","last_peered":"2023-10-21T16:03:37.110240+0000","last_clean":"2023-10-21T16:03:37.110240+0000","last_became_active":"2023-10-21T16:03:35.248865+0000","last_became_peered":"2023-10-21T16:03:35.248865+0000","last_unstale":"2023-10-21T16:03:37.110240+0000","last_undegraded":"2023-10-21T16:03:37.110240+0000","last_fullsized":"2023-10-21T16:03:37.110240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:45.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113870+0000","last_change":"2023-10-21T16:03:35.976123+0000","last_active":"2023-10-21T16:03:37.113870+0000","last_peered":"2023-10-21T16:03:37.113870+0000","last_clean":"2023-10-21T16:03:37.113870+0000","last_became_active":"2023-10-21T16:03:35.975847+0000","last_became_peered":"2023-10-21T16:03:35.975847+0000","last_unstale":"2023-10-21T16:03:37.113870+0000","last_undegraded":"2023-10-21T16:03:37.113870+0000","last_fullsized":"2023-10-21T16:03:37.113870+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113955+0000","last_change":"2023-10-21T16:03:35.244588+0000","last_active":"2023-10-21T16:03:37.113955+0000","last_peered":"2023-10-21T16:03:37.113955+0000","last_clean":"2023-10-21T16:03:37.113955+0000","last_became_active":"2023-10-21T16:03:35.244301+0000","last_became_peered":"2023-10-21T16:03:35.244301+0000","last_unstale":"2023-10-21T16:03:37.113955+0000","last_undegraded":"2023-10-21T16:03:37.113955+0000","last_fullsized":"2023-10-21T16:03:37.113955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:47.476019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110669+0000","last_change":"2023-10-21T16:03:35.974229+0000","last_active":"2023-10-21T16:03:37.110669+0000","last_peered":"2023-10-21T16:03:37.110669+0000","last_clean":"2023-10-21T16:03:37.110669+0000","last_became_active":"2023-10-21T16:03:35.973946+0000","last_became_peered":"2023-10-21T16:03:35.973946+0000","last_unstale":"2023-10-21T16:03:37.110669+0000","last_undegraded":"2023-10-21T16:03:37.110669+0000","last_fullsized":"2023-10-21T16:03:37.110669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:55.578300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113930+0000","last_change":"2023-10-21T16:03:35.254230+0000","last_active":"2023-10-21T16:03:37.113930+0000","last_peered":"2023-10-21T16:03:37.113930+0000","last_clean":"2023-10-21T16:03:37.113930+0000","last_became_active":"2023-10-21T16:03:35.253801+0000","last_became_peered":"2023-10-21T16:03:35.253801+0000","last_unstale":"2023-10-21T16:03:37.113930+0000","last_undegraded":"2023-10-21T16:03:37.113930+0000","last_fullsized":"2023-10-21T16:03:37.113930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:49.046573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950991+0000","last_change":"2023-10-21T16:03:35.975452+0000","last_active":"2023-10-21T16:03:36.950991+0000","last_peered":"2023-10-21T16:03:36.950991+0000","last_clean":"2023-10-21T16:03:36.950991+0000","last_became_active":"2023-10-21T16:03:35.975173+0000","last_became_peered":"2023-10-21T16:03:35.975173+0000","last_unstale":"2023-10-21T16:03:36.950991+0000","last_undegraded":"2023-10-21T16:03:36.950991+0000","last_fullsized":"2023-10-21T16:03:36.950991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:55.578300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108865+0000","last_change":"2023-10-21T16:03:35.251735+0000","last_active":"2023-10-21T16:03:37.108865+0000","last_peered":"2023-10-21T16:03:37.108865+0000","last_clean":"2023-10-21T16:03:37.108865+0000","last_became_active":"2023-10-21T16:03:35.251449+0000","last_became_peered":"2023-10-21T16:03:35.251449+0000","last_unstale":"2023-10-21T16:03:37.108865+0000","last_undegraded":"2023-10-21T16:03:37.108865+0000","last_fullsized":"2023-10-21T16:03:37.108865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:40.625703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108715+0000","last_change":"2023-10-21T16:03:35.973338+0000","last_active":"2023-10-21T16:03:37.108715+0000","last_peered":"2023-10-21T16:03:37.108715+0000","last_clean":"2023-10-21T16:03:37.108715+0000","last_became_active":"2023-10-21T16:03:35.973047+0000","last_became_peered":"2023-10-21T16:03:35.973047+0000","last_unstale":"2023-10-21T16:03:37.108715+0000","last_undegraded":"2023-10-21T16:03:37.108715+0000","last_fullsized":"2023-10-21T16:03:37.108715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109489+0000","last_change":"2023-10-21T16:03:35.245144+0000","last_active":"2023-10-21T16:03:37.109489+0000","last_peered":"2023-10-21T16:03:37.109489+0000","last_clean":"2023-10-21T16:03:37.109489+0000","last_became_active":"2023-10-21T16:03:35.244663+0000","last_became_peered":"2023-10-21T16:03:35.244663+0000","last_unstale":"2023-10-21T16:03:37.109489+0000","last_undegraded":"2023-10-21T16:03:37.109489+0000","last_fullsized":"2023-10-21T16:03:37.109489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:00.842311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108529+0000","last_change":"2023-10-21T16:03:35.965705+0000","last_active":"2023-10-21T16:03:37.108529+0000","last_peered":"2023-10-21T16:03:37.108529+0000","last_clean":"2023-10-21T16:03:37.108529+0000","last_became_active":"2023-10-21T16:03:35.965378+0000","last_became_peered":"2023-10-21T16:03:35.965378+0000","last_unstale":"2023-10-21T16:03:37.108529+0000","last_undegraded":"2023-10-21T16:03:37.108529+0000","last_fullsized":"2023-10-21T16:03:37.108529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:14.604478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944730+0000","last_change":"2023-10-21T16:03:35.257206+0000","last_active":"2023-10-21T16:03:36.944730+0000","last_peered":"2023-10-21T16:03:36.944730+0000","last_clean":"2023-10-21T16:03:36.944730+0000","last_became_active":"2023-10-21T16:03:35.256937+0000","last_became_peered":"2023-10-21T16:03:35.256937+0000","last_unstale":"2023-10-21T16:03:36.944730+0000","last_undegraded":"2023-10-21T16:03:36.944730+0000","last_fullsized":"2023-10-21T16:03:36.944730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:02.475292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111673+0000","last_change":"2023-10-21T16:03:35.976267+0000","last_active":"2023-10-21T16:03:37.111673+0000","last_peered":"2023-10-21T16:03:37.111673+0000","last_clean":"2023-10-21T16:03:37.111673+0000","last_became_active":"2023-10-21T16:03:35.973415+0000","last_became_peered":"2023-10-21T16:03:35.973415+0000","last_unstale":"2023-10-21T16:03:37.111673+0000","last_undegraded":"2023-10-21T16:03:37.111673+0000","last_fullsized":"2023-10-21T16:03:37.111673+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:57.252260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110063+0000","last_change":"2023-10-21T16:03:35.251081+0000","last_active":"2023-10-21T16:03:37.110063+0000","last_peered":"2023-10-21T16:03:37.110063+0000","last_clean":"2023-10-21T16:03:37.110063+0000","last_became_active":"2023-10-21T16:03:35.250827+0000","last_became_peered":"2023-10-21T16:03:35.250827+0000","last_unstale":"2023-10-21T16:03:37.110063+0000","last_undegraded":"2023-10-21T16:03:37.110063+0000","last_fullsized":"2023-10-21T16:03:37.110063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:04:30.130546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109216+0000","last_change":"2023-10-21T16:03:35.965925+0000","last_active":"2023-10-21T16:03:37.109216+0000","last_peered":"2023-10-21T16:03:37.109216+0000","last_clean":"2023-10-21T16:03:37.109216+0000","last_became_active":"2023-10-21T16:03:35.965120+0000","last_became_peered":"2023-10-21T16:03:35.965120+0000","last_unstale":"2023-10-21T16:03:37.109216+0000","last_undegraded":"2023-10-21T16:03:37.109216+0000","last_fullsized":"2023-10-21T16:03:37.109216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:36.297055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109323+0000","last_change":"2023-10-21T16:03:35.258028+0000","last_active":"2023-10-21T16:03:37.109323+0000","last_peered":"2023-10-21T16:03:37.109323+0000","last_clean":"2023-10-21T16:03:37.109323+0000","last_became_active":"2023-10-21T16:03:35.257316+0000","last_became_peered":"2023-10-21T16:03:35.257316+0000","last_unstale":"2023-10-21T16:03:37.109323+0000","last_undegraded":"2023-10-21T16:03:37.109323+0000","last_fullsized":"2023-10-21T16:03:37.109323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:17:55.884533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108771+0000","last_change":"2023-10-21T16:03:35.975586+0000","last_active":"2023-10-21T16:03:37.108771+0000","last_peered":"2023-10-21T16:03:37.108771+0000","last_clean":"2023-10-21T16:03:37.108771+0000","last_became_active":"2023-10-21T16:03:35.975271+0000","last_became_peered":"2023-10-21T16:03:35.975271+0000","last_unstale":"2023-10-21T16:03:37.108771+0000","last_undegraded":"2023-10-21T16:03:37.108771+0000","last_fullsized":"2023-10-21T16:03:37.108771+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:51.725141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112600+0000","last_change":"2023-10-21T16:03:35.258356+0000","last_active":"2023-10-21T16:03:37.112600+0000","last_peered":"2023-10-21T16:03:37.112600+0000","last_clean":"2023-10-21T16:03:37.112600+0000","last_became_active":"2023-10-21T16:03:35.257132+0000","last_became_peered":"2023-10-21T16:03:35.257132+0000","last_unstale":"2023-10-21T16:03:37.112600+0000","last_undegraded":"2023-10-21T16:03:37.112600+0000","last_fullsized":"2023-10-21T16:03:37.112600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:54.564618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113776+0000","last_change":"2023-10-21T16:03:35.981591+0000","last_active":"2023-10-21T16:03:37.113776+0000","last_peered":"2023-10-21T16:03:37.113776+0000","last_clean":"2023-10-21T16:03:37.113776+0000","last_became_active":"2023-10-21T16:03:35.980501+0000","last_became_peered":"2023-10-21T16:03:35.980501+0000","last_unstale":"2023-10-21T16:03:37.113776+0000","last_undegraded":"2023-10-21T16:03:37.113776+0000","last_fullsized":"2023-10-21T16:03:37.113776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:46.170039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950834+0000","last_change":"2023-10-21T16:03:35.253716+0000","last_active":"2023-10-21T16:03:36.950834+0000","last_peered":"2023-10-21T16:03:36.950834+0000","last_clean":"2023-10-21T16:03:36.950834+0000","last_became_active":"2023-10-21T16:03:35.249915+0000","last_became_peered":"2023-10-21T16:03:35.249915+0000","last_unstale":"2023-10-21T16:03:36.950834+0000","last_undegraded":"2023-10-21T16:03:36.950834+0000","last_fullsized":"2023-10-21T16:03:36.950834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:56.238578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950627+0000","last_change":"2023-10-21T16:03:35.960275+0000","last_active":"2023-10-21T16:03:36.950627+0000","last_peered":"2023-10-21T16:03:36.950627+0000","last_clean":"2023-10-21T16:03:36.950627+0000","last_became_active":"2023-10-21T16:03:35.959878+0000","last_became_peered":"2023-10-21T16:03:35.959878+0000","last_unstale":"2023-10-21T16:03:36.950627+0000","last_undegraded":"2023-10-21T16:03:36.950627+0000","last_fullsized":"2023-10-21T16:03:36.950627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948634+0000","last_change":"2023-10-21T16:03:35.254518+0000","last_active":"2023-10-21T16:03:36.948634+0000","last_peered":"2023-10-21T16:03:36.948634+0000","last_clean":"2023-10-21T16:03:36.948634+0000","last_became_active":"2023-10-21T16:03:35.254140+0000","last_became_peered":"2023-10-21T16:03:35.254140+0000","last_unstale":"2023-10-21T16:03:36.948634+0000","last_undegraded":"2023-10-21T16:03:36.948634+0000","last_fullsized":"2023-10-21T16:03:36.948634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:12.360673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948464+0000","last_change":"2023-10-21T16:03:35.970456+0000","last_active":"2023-10-21T16:03:36.948464+0000","last_peered":"2023-10-21T16:03:36.948464+0000","last_clean":"2023-10-21T16:03:36.948464+0000","last_became_active":"2023-10-21T16:03:35.970098+0000","last_became_peered":"2023-10-21T16:03:35.970098+0000","last_unstale":"2023-10-21T16:03:36.948464+0000","last_undegraded":"2023-10-21T16:03:36.948464+0000","last_fullsized":"2023-10-21T16:03:36.948464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:04:31.144228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944985+0000","last_change":"2023-10-21T16:03:35.256557+0000","last_active":"2023-10-21T16:03:36.944985+0000","last_peered":"2023-10-21T16:03:36.944985+0000","last_clean":"2023-10-21T16:03:36.944985+0000","last_became_active":"2023-10-21T16:03:35.256265+0000","last_became_peered":"2023-10-21T16:03:35.256265+0000","last_unstale":"2023-10-21T16:03:36.944985+0000","last_undegraded":"2023-10-21T16:03:36.944985+0000","last_fullsized":"2023-10-21T16:03:36.944985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:39.747058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111959+0000","last_change":"2023-10-21T16:03:35.978613+0000","last_active":"2023-10-21T16:03:37.111959+0000","last_peered":"2023-10-21T16:03:37.111959+0000","last_clean":"2023-10-21T16:03:37.111959+0000","last_became_active":"2023-10-21T16:03:35.978309+0000","last_became_peered":"2023-10-21T16:03:35.978309+0000","last_unstale":"2023-10-21T16:03:37.111959+0000","last_undegraded":"2023-10-21T16:03:37.111959+0000","last_fullsized":"2023-10-21T16:03:37.111959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:08.467477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948070+0000","last_change":"2023-10-21T16:03:35.240577+0000","last_active":"2023-10-21T16:03:36.948070+0000","last_peered":"2023-10-21T16:03:36.948070+0000","last_clean":"2023-10-21T16:03:36.948070+0000","last_became_active":"2023-10-21T16:03:35.240292+0000","last_became_peered":"2023-10-21T16:03:35.240292+0000","last_unstale":"2023-10-21T16:03:36.948070+0000","last_undegraded":"2023-10-21T16:03:36.948070+0000","last_fullsized":"2023-10-21T16:03:36.948070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:35.909043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109149+0000","last_change":"2023-10-21T16:03:35.967733+0000","last_active":"2023-10-21T16:03:37.109149+0000","last_peered":"2023-10-21T16:03:37.109149+0000","last_clean":"2023-10-21T16:03:37.109149+0000","last_became_active":"2023-10-21T16:03:35.967439+0000","last_became_peered":"2023-10-21T16:03:35.967439+0000","last_unstale":"2023-10-21T16:03:37.109149+0000","last_undegraded":"2023-10-21T16:03:37.109149+0000","last_fullsized":"2023-10-21T16:03:37.109149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948135+0000","last_change":"2023-10-21T16:03:35.249102+0000","last_active":"2023-10-21T16:03:36.948135+0000","last_peered":"2023-10-21T16:03:36.948135+0000","last_clean":"2023-10-21T16:03:36.948135+0000","last_became_active":"2023-10-21T16:03:35.248793+0000","last_became_peered":"2023-10-21T16:03:35.248793+0000","last_unstale":"2023-10-21T16:03:36.948135+0000","last_undegraded":"2023-10-21T16:03:36.948135+0000","last_fullsized":"2023-10-21T16:03:36.948135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:52.663008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112318+0000","last_change":"2023-10-21T16:03:35.983491+0000","last_active":"2023-10-21T16:03:37.112318+0000","last_peered":"2023-10-21T16:03:37.112318+0000","last_clean":"2023-10-21T16:03:37.112318+0000","last_became_active":"2023-10-21T16:03:35.981131+0000","last_became_peered":"2023-10-21T16:03:35.981131+0000","last_unstale":"2023-10-21T16:03:37.112318+0000","last_undegraded":"2023-10-21T16:03:37.112318+0000","last_fullsized":"2023-10-21T16:03:37.112318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:36:22.885882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113448+0000","last_change":"2023-10-21T16:03:35.252926+0000","last_active":"2023-10-21T16:03:37.113448+0000","last_peered":"2023-10-21T16:03:37.113448+0000","last_clean":"2023-10-21T16:03:37.113448+0000","last_became_active":"2023-10-21T16:03:35.252623+0000","last_became_peered":"2023-10-21T16:03:35.252623+0000","last_unstale":"2023-10-21T16:03:37.113448+0000","last_undegraded":"2023-10-21T16:03:37.113448+0000","last_fullsized":"2023-10-21T16:03:37.113448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:39.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109437+0000","last_change":"2023-10-21T16:03:35.975257+0000","last_active":"2023-10-21T16:03:37.109437+0000","last_peered":"2023-10-21T16:03:37.109437+0000","last_clean":"2023-10-21T16:03:37.109437+0000","last_became_active":"2023-10-21T16:03:35.972983+0000","last_became_peered":"2023-10-21T16:03:35.972983+0000","last_unstale":"2023-10-21T16:03:37.109437+0000","last_undegraded":"2023-10-21T16:03:37.109437+0000","last_fullsized":"2023-10-21T16:03:37.109437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:02:44.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948009+0000","last_change":"2023-10-21T16:03:35.246281+0000","last_active":"2023-10-21T16:03:36.948009+0000","last_peered":"2023-10-21T16:03:36.948009+0000","last_clean":"2023-10-21T16:03:36.948009+0000","last_became_active":"2023-10-21T16:03:35.246012+0000","last_became_peered":"2023-10-21T16:03:35.246012+0000","last_unstale":"2023-10-21T16:03:36.948009+0000","last_undegraded":"2023-10-21T16:03:36.948009+0000","last_fullsized":"2023-10-21T16:03:36.948009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:01.677983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944866+0000","last_change":"2023-10-21T16:03:35.971443+0000","last_active":"2023-10-21T16:03:36.944866+0000","last_peered":"2023-10-21T16:03:36.944866+0000","last_clean":"2023-10-21T16:03:36.944866+0000","last_became_active":"2023-10-21T16:03:35.971183+0000","last_became_peered":"2023-10-21T16:03:35.971183+0000","last_unstale":"2023-10-21T16:03:36.944866+0000","last_undegraded":"2023-10-21T16:03:36.944866+0000","last_fullsized":"2023-10-21T16:03:36.944866+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113037+0000","last_change":"2023-10-21T16:03:35.249268+0000","last_active":"2023-10-21T16:03:37.113037+0000","last_peered":"2023-10-21T16:03:37.113037+0000","last_clean":"2023-10-21T16:03:37.113037+0000","last_became_active":"2023-10-21T16:03:35.248956+0000","last_became_peered":"2023-10-21T16:03:35.248956+0000","last_unstale":"2023-10-21T16:03:37.113037+0000","last_undegraded":"2023-10-21T16:03:37.113037+0000","last_fullsized":"2023-10-21T16:03:37.113037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:26:16.532600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113005+0000","last_change":"2023-10-21T16:03:35.982746+0000","last_active":"2023-10-21T16:03:37.113005+0000","last_peered":"2023-10-21T16:03:37.113005+0000","last_clean":"2023-10-21T16:03:37.113005+0000","last_became_active":"2023-10-21T16:03:35.981895+0000","last_became_peered":"2023-10-21T16:03:35.981895+0000","last_unstale":"2023-10-21T16:03:37.113005+0000","last_undegraded":"2023-10-21T16:03:37.113005+0000","last_fullsized":"2023-10-21T16:03:37.113005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.427846+0000","last_change":"2023-10-21T16:03:35.238616+0000","last_active":"2023-10-21T16:03:37.427846+0000","last_peered":"2023-10-21T16:03:37.427846+0000","last_clean":"2023-10-21T16:03:37.427846+0000","last_became_active":"2023-10-21T16:03:35.238161+0000","last_became_peered":"2023-10-21T16:03:35.238161+0000","last_unstale":"2023-10-21T16:03:37.427846+0000","last_undegraded":"2023-10-21T16:03:37.427846+0000","last_fullsized":"2023-10-21T16:03:37.427846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:23.215304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109881+0000","last_change":"2023-10-21T16:03:35.975555+0000","last_active":"2023-10-21T16:03:37.109881+0000","last_peered":"2023-10-21T16:03:37.109881+0000","last_clean":"2023-10-21T16:03:37.109881+0000","last_became_active":"2023-10-21T16:03:35.974303+0000","last_became_peered":"2023-10-21T16:03:35.974303+0000","last_unstale":"2023-10-21T16:03:37.109881+0000","last_undegraded":"2023-10-21T16:03:37.109881+0000","last_fullsized":"2023-10-21T16:03:37.109881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:31.749731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110277+0000","last_change":"2023-10-21T16:03:35.252320+0000","last_active":"2023-10-21T16:03:37.110277+0000","last_peered":"2023-10-21T16:03:37.110277+0000","last_clean":"2023-10-21T16:03:37.110277+0000","last_became_active":"2023-10-21T16:03:35.251995+0000","last_became_peered":"2023-10-21T16:03:35.251995+0000","last_unstale":"2023-10-21T16:03:37.110277+0000","last_undegraded":"2023-10-21T16:03:37.110277+0000","last_fullsized":"2023-10-21T16:03:37.110277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:07.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109085+0000","last_change":"2023-10-21T16:03:35.965099+0000","last_active":"2023-10-21T16:03:37.109085+0000","last_peered":"2023-10-21T16:03:37.109085+0000","last_clean":"2023-10-21T16:03:37.109085+0000","last_became_active":"2023-10-21T16:03:35.964769+0000","last_became_peered":"2023-10-21T16:03:35.964769+0000","last_unstale":"2023-10-21T16:03:37.109085+0000","last_undegraded":"2023-10-21T16:03:37.109085+0000","last_fullsized":"2023-10-21T16:03:37.109085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110272+0000","last_change":"2023-10-21T16:03:35.252315+0000","last_active":"2023-10-21T16:03:37.110272+0000","last_peered":"2023-10-21T16:03:37.110272+0000","last_clean":"2023-10-21T16:03:37.110272+0000","last_became_active":"2023-10-21T16:03:35.251813+0000","last_became_peered":"2023-10-21T16:03:35.251813+0000","last_unstale":"2023-10-21T16:03:37.110272+0000","last_undegraded":"2023-10-21T16:03:37.110272+0000","last_fullsized":"2023-10-21T16:03:37.110272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:13.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944669+0000","last_change":"2023-10-21T16:03:35.972407+0000","last_active":"2023-10-21T16:03:36.944669+0000","last_peered":"2023-10-21T16:03:36.944669+0000","last_clean":"2023-10-21T16:03:36.944669+0000","last_became_active":"2023-10-21T16:03:35.972151+0000","last_became_peered":"2023-10-21T16:03:35.972151+0000","last_unstale":"2023-10-21T16:03:36.944669+0000","last_undegraded":"2023-10-21T16:03:36.944669+0000","last_fullsized":"2023-10-21T16:03:36.944669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110587+0000","last_change":"2023-10-21T16:03:35.246336+0000","last_active":"2023-10-21T16:03:37.110587+0000","last_peered":"2023-10-21T16:03:37.110587+0000","last_clean":"2023-10-21T16:03:37.110587+0000","last_became_active":"2023-10-21T16:03:35.246024+0000","last_became_peered":"2023-10-21T16:03:35.246024+0000","last_unstale":"2023-10-21T16:03:37.110587+0000","last_undegraded":"2023-10-21T16:03:37.110587+0000","last_fullsized":"2023-10-21T16:03:37.110587+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:28.553449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949856+0000","last_change":"2023-10-21T16:03:35.961422+0000","last_active":"2023-10-21T16:03:36.949856+0000","last_peered":"2023-10-21T16:03:36.949856+0000","last_clean":"2023-10-21T16:03:36.949856+0000","last_became_active":"2023-10-21T16:03:35.960740+0000","last_became_peered":"2023-10-21T16:03:35.960740+0000","last_unstale":"2023-10-21T16:03:36.949856+0000","last_undegraded":"2023-10-21T16:03:36.949856+0000","last_fullsized":"2023-10-21T16:03:36.949856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949513+0000","last_change":"2023-10-21T16:03:35.248648+0000","last_active":"2023-10-21T16:03:36.949513+0000","last_peered":"2023-10-21T16:03:36.949513+0000","last_clean":"2023-10-21T16:03:36.949513+0000","last_became_active":"2023-10-21T16:03:35.248313+0000","last_became_peered":"2023-10-21T16:03:35.248313+0000","last_unstale":"2023-10-21T16:03:36.949513+0000","last_undegraded":"2023-10-21T16:03:36.949513+0000","last_fullsized":"2023-10-21T16:03:36.949513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:39.747058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112374+0000","last_change":"2023-10-21T16:03:35.974517+0000","last_active":"2023-10-21T16:03:37.112374+0000","last_peered":"2023-10-21T16:03:37.112374+0000","last_clean":"2023-10-21T16:03:37.112374+0000","last_became_active":"2023-10-21T16:03:35.972628+0000","last_became_peered":"2023-10-21T16:03:35.972628+0000","last_unstale":"2023-10-21T16:03:37.112374+0000","last_undegraded":"2023-10-21T16:03:37.112374+0000","last_fullsized":"2023-10-21T16:03:37.112374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:29.158841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108300+0000","last_change":"2023-10-21T16:03:35.241484+0000","last_active":"2023-10-21T16:03:37.108300+0000","last_peered":"2023-10-21T16:03:37.108300+0000","last_clean":"2023-10-21T16:03:37.108300+0000","last_became_active":"2023-10-21T16:03:35.241199+0000","last_became_peered":"2023-10-21T16:03:35.241199+0000","last_unstale":"2023-10-21T16:03:37.108300+0000","last_undegraded":"2023-10-21T16:03:37.108300+0000","last_fullsized":"2023-10-21T16:03:37.108300+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:49.193107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112144+0000","last_change":"2023-10-21T16:03:35.976782+0000","last_active":"2023-10-21T16:03:37.112144+0000","last_peered":"2023-10-21T16:03:37.112144+0000","last_clean":"2023-10-21T16:03:37.112144+0000","last_became_active":"2023-10-21T16:03:35.976508+0000","last_became_peered":"2023-10-21T16:03:35.976508+0000","last_unstale":"2023-10-21T16:03:37.112144+0000","last_undegraded":"2023-10-21T16:03:37.112144+0000","last_fullsized":"2023-10-21T16:03:37.112144+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:41.639386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113499+0000","last_change":"2023-10-21T16:03:35.250845+0000","last_active":"2023-10-21T16:03:37.113499+0000","last_peered":"2023-10-21T16:03:37.113499+0000","last_clean":"2023-10-21T16:03:37.113499+0000","last_became_active":"2023-10-21T16:03:35.250511+0000","last_became_peered":"2023-10-21T16:03:35.250511+0000","last_unstale":"2023-10-21T16:03:37.113499+0000","last_undegraded":"2023-10-21T16:03:37.113499+0000","last_fullsized":"2023-10-21T16:03:37.113499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:28.553449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-21T16:03:35.962277+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:03:35.962277+0000","last_peered":"2023-10-21T16:03:35.962277+0000","last_clean":"2023-10-21T16:03:35.962277+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:03:35.962277+0000","last_undegraded":"2023-10-21T16:03:35.962277+0000","last_fullsized":"2023-10-21T16:03:35.962277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1073152,"data_stored":792946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.5900000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.478},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.6379999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.823},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.387}]},{"osd":3,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.9830000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.679},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.778}]},{"osd":5,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":2.1059999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.7270000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.657,"15min":1.657},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.883},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.6970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":201478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.93400000000000005}]},{"osd":1,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.034}]},{"osd":2,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.097},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.3120000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.21}]},{"osd":4,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":1.256},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.3700000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.444},{"interface":"front","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":1.105,"5min":1.105,"15min":1.105},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":1.534}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":201008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3899999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.278},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.997}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.032},{"interface":"front","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.5600000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.878},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.681},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.331},{"interface":"front","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.4770000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":201500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":3.6080000000000001,"5min":3.6080000000000001,"15min":3.6080000000000001},"last":1.401}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.80200000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2,"5min":2,"15min":2},"last":0.84499999999999997}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.97599999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.631},{"interface":"front","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":3.9300000000000002,"5min":3.9300000000000002,"15min":3.9300000000000002},"last":1.2130000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.452},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":1.788}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":1.784,"15min":1.784},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.548}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":479232,"data_stored":202542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.393},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.02}]},{"osd":1,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.1419999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.5469999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.597},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.2869999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.7250000000000001,"5min":3.7250000000000001,"15min":3.7250000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.1930000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":1.456},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.573,"5min":3.573,"15min":3.573},"last":1.6459999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.704}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":466944,"data_stored":200966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.667},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.5329999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.7869999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.169}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.476}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.413},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.05,"5min":1.05,"15min":1.05},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":1.722}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":200986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.292}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.756}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.377},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.907}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.998},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4890000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":4.1260000000000003,"5min":4.1260000000000003,"15min":4.1260000000000003},"last":2.198},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.8140000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":2.3130000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":791298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.1060000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.306,"5min":3.306,"15min":3.306},"last":1.5009999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.8849999999999998,"5min":3.8849999999999998,"15min":3.8849999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.6080000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":4.3019999999999996,"5min":4.3019999999999996,"15min":4.3019999999999996},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":1.6659999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":3.8140000000000001,"5min":3.8140000000000001,"15min":3.8140000000000001},"last":0.996},{"interface":"front","average":{"1min":1.72,"5min":1.72,"15min":1.72},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":1.444}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:03:40.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:03:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.025+0000 7fda3acd4640 1 Processor -- start 2023-10-21T16:03:41.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.025+0000 7fda3acd4640 1 -- start start 2023-10-21T16:03:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda3acd4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda341600d0 0x7fda340d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda3acd4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda340d02f0 0x7fda340b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda3acd4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda340d1750 con 0x7fda340d11d0 2023-10-21T16:03:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda3acd4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340d1a30 con 0x7fda341600d0 2023-10-21T16:03:41.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda3acd4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340d1d10 con 0x7fda340d02f0 2023-10-21T16:03:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda38a70640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda341600d0 0x7fda340d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda38a70640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda341600d0 0x7fda340d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52584/0 (socket says 172.21.15.53:52584) 2023-10-21T16:03:41.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.027+0000 7fda38a70640 1 -- 172.21.15.53:0/4224623259 learned_addr learned my addr 172.21.15.53:0/4224623259 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.028+0000 7fda38a70640 1 -- 172.21.15.53:0/4224623259 >> v2:172.21.15.124:3300/0 conn(0x7fda340d02f0 msgr2=0x7fda340b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.028+0000 7fda38a70640 1 --2- 172.21.15.53:0/4224623259 >> v2:172.21.15.124:3300/0 conn(0x7fda340d02f0 0x7fda340b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.028+0000 7fda38a70640 1 -- 172.21.15.53:0/4224623259 >> v1:172.21.15.53:6789/0 conn(0x7fda340d11d0 legacy=0x7fda340cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.028+0000 7fda38a70640 1 -- 172.21.15.53:0/4224623259 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340d1ff0 con 0x7fda341600d0 2023-10-21T16:03:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.029+0000 7fda38a70640 1 --2- 172.21.15.53:0/4224623259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda340d0c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda340cedf0 tx=0x7fda2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c6b9f25c9aeab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.029+0000 7fda337fe640 1 -- 172.21.15.53:0/4224623259 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28031870 con 0x7fda341600d0 2023-10-21T16:03:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.029+0000 7fda337fe640 1 -- 172.21.15.53:0/4224623259 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda280319d0 con 0x7fda341600d0 2023-10-21T16:03:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.029+0000 7fda337fe640 1 -- 172.21.15.53:0/4224623259 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28031d60 con 0x7fda341600d0 2023-10-21T16:03:41.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 -- 172.21.15.53:0/4224623259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 msgr2=0x7fda340d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 --2- 172.21.15.53:0/4224623259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda340d0c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fda340cedf0 tx=0x7fda2802b910 comp rx=0 tx=0).stop 2023-10-21T16:03:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 -- 172.21.15.53:0/4224623259 shutdown_connections 2023-10-21T16:03:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 --2- 172.21.15.53:0/4224623259 >> v2:172.21.15.124:3300/0 conn(0x7fda340d02f0 0x7fda340b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 --2- 172.21.15.53:0/4224623259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda340d0c90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 -- 172.21.15.53:0/4224623259 >> 172.21.15.53:0/4224623259 conn(0x7fda340718a0 msgr2=0x7fda34071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 -- 172.21.15.53:0/4224623259 shutdown_connections 2023-10-21T16:03:41.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.030+0000 7fda3acd4640 1 -- 172.21.15.53:0/4224623259 wait complete. 2023-10-21T16:03:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.032+0000 7fda3acd4640 1 Processor -- start 2023-10-21T16:03:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.032+0000 7fda3acd4640 1 -- start start 2023-10-21T16:03:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 0x7fda34154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340d11d0 0x7fda34152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340cab00 con 0x7fda341600d0 2023-10-21T16:03:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340c9e80 con 0x7fda340d02f0 2023-10-21T16:03:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda3acd4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340ca160 con 0x7fda340d11d0 2023-10-21T16:03:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.033+0000 7fda39271640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52586/0 (socket says 172.21.15.53:52586) 2023-10-21T16:03:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 -- 172.21.15.53:0/3226616387 learned_addr learned my addr 172.21.15.53:0/3226616387 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda33fff640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340d11d0 0x7fda34152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda38a70640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 0x7fda34154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 -- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340d11d0 msgr2=0x7fda34152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340d11d0 0x7fda34152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 -- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 msgr2=0x7fda34154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 0x7fda34154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 -- 172.21.15.53:0/3226616387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280094d0 con 0x7fda341600d0 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda38a70640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 0x7fda34154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.034+0000 7fda39271640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b9e0 tx=0x7fda2c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.035+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00cb20 con 0x7fda341600d0 2023-10-21T16:03:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.035+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c00cca0 con 0x7fda341600d0 2023-10-21T16:03:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.035+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c014680 con 0x7fda341600d0 2023-10-21T16:03:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.035+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda340ca440 con 0x7fda341600d0 2023-10-21T16:03:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.035+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda34159ad0 con 0x7fda341600d0 2023-10-21T16:03:41.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.036+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda340cac40 con 0x7fda341600d0 2023-10-21T16:03:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.037+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda2c01d3b0 con 0x7fda341600d0 2023-10-21T16:03:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.038+0000 7fda317fa640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 0x7fda1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.038+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fda2c089680 con 0x7fda341600d0 2023-10-21T16:03:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.039+0000 7fda38a70640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 0x7fda1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.040+0000 7fda38a70640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 0x7fda1806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda28036000 tx=0x7fda2803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.042+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda340cac40 con 0x7fda341600d0 2023-10-21T16:03:41.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.233+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda340ca2d0 con 0x7fda1806ae70 2023-10-21T16:03:41.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.257+0000 7fda317fa640 1 -- 172.21.15.53:0/3226616387 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286512 (secure 0 0 0) 0x7fda340ca2d0 con 0x7fda1806ae70 2023-10-21T16:03:41.262 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:03:41.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.264+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 msgr2=0x7fda1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.264+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 0x7fda1806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda28036000 tx=0x7fda2803c040 comp rx=0 tx=0).stop 2023-10-21T16:03:41.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.264+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 msgr2=0x7fda34158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.264+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b9e0 tx=0x7fda2c00bea0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 shutdown_connections 2023-10-21T16:03:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7fda1806ae70 0x7fda1806d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda341600d0 0x7fda34158590 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340d11d0 0x7fda34152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 --2- 172.21.15.53:0/3226616387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda340d02f0 0x7fda34154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 >> 172.21.15.53:0/3226616387 conn(0x7fda340718a0 msgr2=0x7fda3406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:41.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 shutdown_connections 2023-10-21T16:03:41.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.265+0000 7fda3acd4640 1 -- 172.21.15.53:0/3226616387 wait complete. 2023-10-21T16:03:41.289 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-10-21T16:03:41.056036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237724,"kb_used_data":4840,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736996,"statfs":{"total":773094113280,"available":772850683904,"internally_reserved":0,"allocated":4956160,"data_stored":2792724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238276153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002079"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110282+0000","last_change":"2023-10-21T16:03:35.968487+0000","last_active":"2023-10-21T16:03:37.110282+0000","last_peered":"2023-10-21T16:03:37.110282+0000","last_clean":"2023-10-21T16:03:37.110282+0000","last_became_active":"2023-10-21T16:03:35.967993+0000","last_became_peered":"2023-10-21T16:03:35.967993+0000","last_unstale":"2023-10-21T16:03:37.110282+0000","last_undegraded":"2023-10-21T16:03:37.110282+0000","last_fullsized":"2023-10-21T16:03:37.110282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:05.136225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109538+0000","last_change":"2023-10-21T16:03:35.976671+0000","last_active":"2023-10-21T16:03:37.109538+0000","last_peered":"2023-10-21T16:03:37.109538+0000","last_clean":"2023-10-21T16:03:37.109538+0000","last_became_active":"2023-10-21T16:03:35.976051+0000","last_became_peered":"2023-10-21T16:03:35.976051+0000","last_unstale":"2023-10-21T16:03:37.109538+0000","last_undegraded":"2023-10-21T16:03:37.109538+0000","last_fullsized":"2023-10-21T16:03:37.109538+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:43:31.874405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110186+0000","last_change":"2023-10-21T16:03:35.973243+0000","last_active":"2023-10-21T16:03:37.110186+0000","last_peered":"2023-10-21T16:03:37.110186+0000","last_clean":"2023-10-21T16:03:37.110186+0000","last_became_active":"2023-10-21T16:03:35.972967+0000","last_became_peered":"2023-10-21T16:03:35.972967+0000","last_unstale":"2023-10-21T16:03:37.110186+0000","last_undegraded":"2023-10-21T16:03:37.110186+0000","last_fullsized":"2023-10-21T16:03:37.110186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948195+0000","last_change":"2023-10-21T16:03:35.968486+0000","last_active":"2023-10-21T16:03:36.948195+0000","last_peered":"2023-10-21T16:03:36.948195+0000","last_clean":"2023-10-21T16:03:36.948195+0000","last_became_active":"2023-10-21T16:03:35.968120+0000","last_became_peered":"2023-10-21T16:03:35.968120+0000","last_unstale":"2023-10-21T16:03:36.948195+0000","last_undegraded":"2023-10-21T16:03:36.948195+0000","last_fullsized":"2023-10-21T16:03:36.948195+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:29.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.947993+0000","last_change":"2023-10-21T16:03:35.973338+0000","last_active":"2023-10-21T16:03:36.947993+0000","last_peered":"2023-10-21T16:03:36.947993+0000","last_clean":"2023-10-21T16:03:36.947993+0000","last_became_active":"2023-10-21T16:03:35.972648+0000","last_became_peered":"2023-10-21T16:03:35.972648+0000","last_unstale":"2023-10-21T16:03:36.947993+0000","last_undegraded":"2023-10-21T16:03:36.947993+0000","last_fullsized":"2023-10-21T16:03:36.947993+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:10.275412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944626+0000","last_change":"2023-10-21T16:03:35.961394+0000","last_active":"2023-10-21T16:03:36.944626+0000","last_peered":"2023-10-21T16:03:36.944626+0000","last_clean":"2023-10-21T16:03:36.944626+0000","last_became_active":"2023-10-21T16:03:35.961091+0000","last_became_peered":"2023-10-21T16:03:35.961091+0000","last_unstale":"2023-10-21T16:03:36.944626+0000","last_undegraded":"2023-10-21T16:03:36.944626+0000","last_fullsized":"2023-10-21T16:03:36.944626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111171+0000","last_change":"2023-10-21T16:03:35.975808+0000","last_active":"2023-10-21T16:03:37.111171+0000","last_peered":"2023-10-21T16:03:37.111171+0000","last_clean":"2023-10-21T16:03:37.111171+0000","last_became_active":"2023-10-21T16:03:35.975182+0000","last_became_peered":"2023-10-21T16:03:35.975182+0000","last_unstale":"2023-10-21T16:03:37.111171+0000","last_undegraded":"2023-10-21T16:03:37.111171+0000","last_fullsized":"2023-10-21T16:03:37.111171+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:06.569611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108410+0000","last_change":"2023-10-21T16:03:35.976523+0000","last_active":"2023-10-21T16:03:37.108410+0000","last_peered":"2023-10-21T16:03:37.108410+0000","last_clean":"2023-10-21T16:03:37.108410+0000","last_became_active":"2023-10-21T16:03:35.976126+0000","last_became_peered":"2023-10-21T16:03:35.976126+0000","last_unstale":"2023-10-21T16:03:37.108410+0000","last_undegraded":"2023-10-21T16:03:37.108410+0000","last_fullsized":"2023-10-21T16:03:37.108410+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:24.487305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111033+0000","last_change":"2023-10-21T16:03:35.978945+0000","last_active":"2023-10-21T16:03:37.111033+0000","last_peered":"2023-10-21T16:03:37.111033+0000","last_clean":"2023-10-21T16:03:37.111033+0000","last_became_active":"2023-10-21T16:03:35.978654+0000","last_became_peered":"2023-10-21T16:03:35.978654+0000","last_unstale":"2023-10-21T16:03:37.111033+0000","last_undegraded":"2023-10-21T16:03:37.111033+0000","last_fullsized":"2023-10-21T16:03:37.111033+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:26:17.546283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113519+0000","last_change":"2023-10-21T16:03:35.979372+0000","last_active":"2023-10-21T16:03:37.113519+0000","last_peered":"2023-10-21T16:03:37.113519+0000","last_clean":"2023-10-21T16:03:37.113519+0000","last_became_active":"2023-10-21T16:03:35.975419+0000","last_became_peered":"2023-10-21T16:03:35.975419+0000","last_unstale":"2023-10-21T16:03:37.113519+0000","last_undegraded":"2023-10-21T16:03:37.113519+0000","last_fullsized":"2023-10-21T16:03:37.113519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:19.547371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113701+0000","last_change":"2023-10-21T16:03:35.971219+0000","last_active":"2023-10-21T16:03:37.113701+0000","last_peered":"2023-10-21T16:03:37.113701+0000","last_clean":"2023-10-21T16:03:37.113701+0000","last_became_active":"2023-10-21T16:03:35.970563+0000","last_became_peered":"2023-10-21T16:03:35.970563+0000","last_unstale":"2023-10-21T16:03:37.113701+0000","last_undegraded":"2023-10-21T16:03:37.113701+0000","last_fullsized":"2023-10-21T16:03:37.113701+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:17.247375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110342+0000","last_change":"2023-10-21T16:03:35.977449+0000","last_active":"2023-10-21T16:03:37.110342+0000","last_peered":"2023-10-21T16:03:37.110342+0000","last_clean":"2023-10-21T16:03:37.110342+0000","last_became_active":"2023-10-21T16:03:35.977153+0000","last_became_peered":"2023-10-21T16:03:35.977153+0000","last_unstale":"2023-10-21T16:03:37.110342+0000","last_undegraded":"2023-10-21T16:03:37.110342+0000","last_fullsized":"2023-10-21T16:03:37.110342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:48.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110734+0000","last_change":"2023-10-21T16:03:35.972649+0000","last_active":"2023-10-21T16:03:37.110734+0000","last_peered":"2023-10-21T16:03:37.110734+0000","last_clean":"2023-10-21T16:03:37.110734+0000","last_became_active":"2023-10-21T16:03:35.971694+0000","last_became_peered":"2023-10-21T16:03:35.971694+0000","last_unstale":"2023-10-21T16:03:37.110734+0000","last_undegraded":"2023-10-21T16:03:37.110734+0000","last_fullsized":"2023-10-21T16:03:37.110734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:50.206789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949605+0000","last_change":"2023-10-21T16:03:35.975971+0000","last_active":"2023-10-21T16:03:36.949605+0000","last_peered":"2023-10-21T16:03:36.949605+0000","last_clean":"2023-10-21T16:03:36.949605+0000","last_became_active":"2023-10-21T16:03:35.975686+0000","last_became_peered":"2023-10-21T16:03:35.975686+0000","last_unstale":"2023-10-21T16:03:36.949605+0000","last_undegraded":"2023-10-21T16:03:36.949605+0000","last_fullsized":"2023-10-21T16:03:36.949605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111243+0000","last_change":"2023-10-21T16:03:35.976446+0000","last_active":"2023-10-21T16:03:37.111243+0000","last_peered":"2023-10-21T16:03:37.111243+0000","last_clean":"2023-10-21T16:03:37.111243+0000","last_became_active":"2023-10-21T16:03:35.976091+0000","last_became_peered":"2023-10-21T16:03:35.976091+0000","last_unstale":"2023-10-21T16:03:37.111243+0000","last_undegraded":"2023-10-21T16:03:37.111243+0000","last_fullsized":"2023-10-21T16:03:37.111243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:30:28.375486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109302+0000","last_change":"2023-10-21T16:03:35.972767+0000","last_active":"2023-10-21T16:03:37.109302+0000","last_peered":"2023-10-21T16:03:37.109302+0000","last_clean":"2023-10-21T16:03:37.109302+0000","last_became_active":"2023-10-21T16:03:35.972472+0000","last_became_peered":"2023-10-21T16:03:35.972472+0000","last_unstale":"2023-10-21T16:03:37.109302+0000","last_undegraded":"2023-10-21T16:03:37.109302+0000","last_fullsized":"2023-10-21T16:03:37.109302+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:30.658334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113049+0000","last_change":"2023-10-21T16:03:35.979501+0000","last_active":"2023-10-21T16:03:37.113049+0000","last_peered":"2023-10-21T16:03:37.113049+0000","last_clean":"2023-10-21T16:03:37.113049+0000","last_became_active":"2023-10-21T16:03:35.977443+0000","last_became_peered":"2023-10-21T16:03:35.977443+0000","last_unstale":"2023-10-21T16:03:37.113049+0000","last_undegraded":"2023-10-21T16:03:37.113049+0000","last_fullsized":"2023-10-21T16:03:37.113049+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111408+0000","last_change":"2023-10-21T16:03:35.974760+0000","last_active":"2023-10-21T16:03:37.111408+0000","last_peered":"2023-10-21T16:03:37.111408+0000","last_clean":"2023-10-21T16:03:37.111408+0000","last_became_active":"2023-10-21T16:03:35.973128+0000","last_became_peered":"2023-10-21T16:03:35.973128+0000","last_unstale":"2023-10-21T16:03:37.111408+0000","last_undegraded":"2023-10-21T16:03:37.111408+0000","last_fullsized":"2023-10-21T16:03:37.111408+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:17:56.898215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948470+0000","last_change":"2023-10-21T16:03:35.963948+0000","last_active":"2023-10-21T16:03:36.948470+0000","last_peered":"2023-10-21T16:03:36.948470+0000","last_clean":"2023-10-21T16:03:36.948470+0000","last_became_active":"2023-10-21T16:03:35.963610+0000","last_became_peered":"2023-10-21T16:03:35.963610+0000","last_unstale":"2023-10-21T16:03:36.948470+0000","last_undegraded":"2023-10-21T16:03:36.948470+0000","last_fullsized":"2023-10-21T16:03:36.948470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:13.113623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111784+0000","last_change":"2023-10-21T16:03:35.972955+0000","last_active":"2023-10-21T16:03:37.111784+0000","last_peered":"2023-10-21T16:03:37.111784+0000","last_clean":"2023-10-21T16:03:37.111784+0000","last_became_active":"2023-10-21T16:03:35.972517+0000","last_became_peered":"2023-10-21T16:03:35.972517+0000","last_unstale":"2023-10-21T16:03:37.111784+0000","last_undegraded":"2023-10-21T16:03:37.111784+0000","last_fullsized":"2023-10-21T16:03:37.111784+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:27.519767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111089+0000","last_change":"2023-10-21T16:03:35.971099+0000","last_active":"2023-10-21T16:03:37.111089+0000","last_peered":"2023-10-21T16:03:37.111089+0000","last_clean":"2023-10-21T16:03:37.111089+0000","last_became_active":"2023-10-21T16:03:35.970803+0000","last_became_peered":"2023-10-21T16:03:35.970803+0000","last_unstale":"2023-10-21T16:03:37.111089+0000","last_undegraded":"2023-10-21T16:03:37.111089+0000","last_fullsized":"2023-10-21T16:03:37.111089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:11:14.049352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950148+0000","last_change":"2023-10-21T16:03:35.983252+0000","last_active":"2023-10-21T16:03:36.950148+0000","last_peered":"2023-10-21T16:03:36.950148+0000","last_clean":"2023-10-21T16:03:36.950148+0000","last_became_active":"2023-10-21T16:03:35.982898+0000","last_became_peered":"2023-10-21T16:03:35.982898+0000","last_unstale":"2023-10-21T16:03:36.950148+0000","last_undegraded":"2023-10-21T16:03:36.950148+0000","last_fullsized":"2023-10-21T16:03:36.950148+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112990+0000","last_change":"2023-10-21T16:03:35.974047+0000","last_active":"2023-10-21T16:03:37.112990+0000","last_peered":"2023-10-21T16:03:37.112990+0000","last_clean":"2023-10-21T16:03:37.112990+0000","last_became_active":"2023-10-21T16:03:35.973692+0000","last_became_peered":"2023-10-21T16:03:35.973692+0000","last_unstale":"2023-10-21T16:03:37.112990+0000","last_undegraded":"2023-10-21T16:03:37.112990+0000","last_fullsized":"2023-10-21T16:03:37.112990+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:45.499630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.947991+0000","last_change":"2023-10-21T16:03:35.968156+0000","last_active":"2023-10-21T16:03:36.947991+0000","last_peered":"2023-10-21T16:03:36.947991+0000","last_clean":"2023-10-21T16:03:36.947991+0000","last_became_active":"2023-10-21T16:03:35.967121+0000","last_became_peered":"2023-10-21T16:03:35.967121+0000","last_unstale":"2023-10-21T16:03:36.947991+0000","last_undegraded":"2023-10-21T16:03:36.947991+0000","last_fullsized":"2023-10-21T16:03:36.947991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:36:23.012299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110713+0000","last_change":"2023-10-21T16:03:35.976947+0000","last_active":"2023-10-21T16:03:37.110713+0000","last_peered":"2023-10-21T16:03:37.110713+0000","last_clean":"2023-10-21T16:03:37.110713+0000","last_became_active":"2023-10-21T16:03:35.976672+0000","last_became_peered":"2023-10-21T16:03:35.976672+0000","last_unstale":"2023-10-21T16:03:37.110713+0000","last_undegraded":"2023-10-21T16:03:37.110713+0000","last_fullsized":"2023-10-21T16:03:37.110713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:32.562760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945098+0000","last_change":"2023-10-21T16:03:35.978739+0000","last_active":"2023-10-21T16:03:36.945098+0000","last_peered":"2023-10-21T16:03:36.945098+0000","last_clean":"2023-10-21T16:03:36.945098+0000","last_became_active":"2023-10-21T16:03:35.978395+0000","last_became_peered":"2023-10-21T16:03:35.978395+0000","last_unstale":"2023-10-21T16:03:36.945098+0000","last_undegraded":"2023-10-21T16:03:36.945098+0000","last_fullsized":"2023-10-21T16:03:36.945098+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110847+0000","last_change":"2023-10-21T16:03:35.976745+0000","last_active":"2023-10-21T16:03:37.110847+0000","last_peered":"2023-10-21T16:03:37.110847+0000","last_clean":"2023-10-21T16:03:37.110847+0000","last_became_active":"2023-10-21T16:03:35.976436+0000","last_became_peered":"2023-10-21T16:03:35.976436+0000","last_unstale":"2023-10-21T16:03:37.110847+0000","last_undegraded":"2023-10-21T16:03:37.110847+0000","last_fullsized":"2023-10-21T16:03:37.110847+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:40.760740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112524+0000","last_change":"2023-10-21T16:03:35.970032+0000","last_active":"2023-10-21T16:03:37.112524+0000","last_peered":"2023-10-21T16:03:37.112524+0000","last_clean":"2023-10-21T16:03:37.112524+0000","last_became_active":"2023-10-21T16:03:35.969280+0000","last_became_peered":"2023-10-21T16:03:35.969280+0000","last_unstale":"2023-10-21T16:03:37.112524+0000","last_undegraded":"2023-10-21T16:03:37.112524+0000","last_fullsized":"2023-10-21T16:03:37.112524+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:44:57.794604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110887+0000","last_change":"2023-10-21T16:03:35.975780+0000","last_active":"2023-10-21T16:03:37.110887+0000","last_peered":"2023-10-21T16:03:37.110887+0000","last_clean":"2023-10-21T16:03:37.110887+0000","last_became_active":"2023-10-21T16:03:35.972648+0000","last_became_peered":"2023-10-21T16:03:35.972648+0000","last_unstale":"2023-10-21T16:03:37.110887+0000","last_undegraded":"2023-10-21T16:03:37.110887+0000","last_fullsized":"2023-10-21T16:03:37.110887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:03.488974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110430+0000","last_change":"2023-10-21T16:03:35.971689+0000","last_active":"2023-10-21T16:03:37.110430+0000","last_peered":"2023-10-21T16:03:37.110430+0000","last_clean":"2023-10-21T16:03:37.110430+0000","last_became_active":"2023-10-21T16:03:35.971420+0000","last_became_peered":"2023-10-21T16:03:35.971420+0000","last_unstale":"2023-10-21T16:03:37.110430+0000","last_undegraded":"2023-10-21T16:03:37.110430+0000","last_fullsized":"2023-10-21T16:03:37.110430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:40.145592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113274+0000","last_change":"2023-10-21T16:03:35.977272+0000","last_active":"2023-10-21T16:03:37.113274+0000","last_peered":"2023-10-21T16:03:37.113274+0000","last_clean":"2023-10-21T16:03:37.113274+0000","last_became_active":"2023-10-21T16:03:35.974263+0000","last_became_peered":"2023-10-21T16:03:35.974263+0000","last_unstale":"2023-10-21T16:03:37.113274+0000","last_undegraded":"2023-10-21T16:03:37.113274+0000","last_fullsized":"2023-10-21T16:03:37.113274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944456+0000","last_change":"2023-10-21T16:03:35.973976+0000","last_active":"2023-10-21T16:03:36.944456+0000","last_peered":"2023-10-21T16:03:36.944456+0000","last_clean":"2023-10-21T16:03:36.944456+0000","last_became_active":"2023-10-21T16:03:35.973706+0000","last_became_peered":"2023-10-21T16:03:35.973706+0000","last_unstale":"2023-10-21T16:03:36.944456+0000","last_undegraded":"2023-10-21T16:03:36.944456+0000","last_fullsized":"2023-10-21T16:03:36.944456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:45.499630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113758+0000","last_change":"2023-10-21T16:03:35.973726+0000","last_active":"2023-10-21T16:03:37.113758+0000","last_peered":"2023-10-21T16:03:37.113758+0000","last_clean":"2023-10-21T16:03:37.113758+0000","last_became_active":"2023-10-21T16:03:35.971267+0000","last_became_peered":"2023-10-21T16:03:35.971267+0000","last_unstale":"2023-10-21T16:03:37.113758+0000","last_undegraded":"2023-10-21T16:03:37.113758+0000","last_fullsized":"2023-10-21T16:03:37.113758+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110763+0000","last_change":"2023-10-21T16:03:35.251024+0000","last_active":"2023-10-21T16:03:37.110763+0000","last_peered":"2023-10-21T16:03:37.110763+0000","last_clean":"2023-10-21T16:03:37.110763+0000","last_became_active":"2023-10-21T16:03:35.250733+0000","last_became_peered":"2023-10-21T16:03:35.250733+0000","last_unstale":"2023-10-21T16:03:37.110763+0000","last_undegraded":"2023-10-21T16:03:37.110763+0000","last_fullsized":"2023-10-21T16:03:37.110763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:22.567411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945235+0000","last_change":"2023-10-21T16:03:35.971408+0000","last_active":"2023-10-21T16:03:36.945235+0000","last_peered":"2023-10-21T16:03:36.945235+0000","last_clean":"2023-10-21T16:03:36.945235+0000","last_became_active":"2023-10-21T16:03:35.970832+0000","last_became_peered":"2023-10-21T16:03:35.970832+0000","last_unstale":"2023-10-21T16:03:36.945235+0000","last_undegraded":"2023-10-21T16:03:36.945235+0000","last_fullsized":"2023-10-21T16:03:36.945235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110576+0000","last_change":"2023-10-21T16:03:35.256409+0000","last_active":"2023-10-21T16:03:37.110576+0000","last_peered":"2023-10-21T16:03:37.110576+0000","last_clean":"2023-10-21T16:03:37.110576+0000","last_became_active":"2023-10-21T16:03:35.256124+0000","last_became_peered":"2023-10-21T16:03:35.256124+0000","last_unstale":"2023-10-21T16:03:37.110576+0000","last_undegraded":"2023-10-21T16:03:37.110576+0000","last_fullsized":"2023-10-21T16:03:37.110576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:28.145159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950678+0000","last_change":"2023-10-21T16:03:35.980000+0000","last_active":"2023-10-21T16:03:36.950678+0000","last_peered":"2023-10-21T16:03:36.950678+0000","last_clean":"2023-10-21T16:03:36.950678+0000","last_became_active":"2023-10-21T16:03:35.979382+0000","last_became_peered":"2023-10-21T16:03:35.979382+0000","last_unstale":"2023-10-21T16:03:36.950678+0000","last_undegraded":"2023-10-21T16:03:36.950678+0000","last_fullsized":"2023-10-21T16:03:36.950678+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:48.489701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.945305+0000","last_change":"2023-10-21T16:03:35.255798+0000","last_active":"2023-10-21T16:03:36.945305+0000","last_peered":"2023-10-21T16:03:36.945305+0000","last_clean":"2023-10-21T16:03:36.945305+0000","last_became_active":"2023-10-21T16:03:35.255452+0000","last_became_peered":"2023-10-21T16:03:35.255452+0000","last_unstale":"2023-10-21T16:03:36.945305+0000","last_undegraded":"2023-10-21T16:03:36.945305+0000","last_fullsized":"2023-10-21T16:03:36.945305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:26.506085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110026+0000","last_change":"2023-10-21T16:03:35.977872+0000","last_active":"2023-10-21T16:03:37.110026+0000","last_peered":"2023-10-21T16:03:37.110026+0000","last_clean":"2023-10-21T16:03:37.110026+0000","last_became_active":"2023-10-21T16:03:35.977603+0000","last_became_peered":"2023-10-21T16:03:35.977603+0000","last_unstale":"2023-10-21T16:03:37.110026+0000","last_undegraded":"2023-10-21T16:03:37.110026+0000","last_fullsized":"2023-10-21T16:03:37.110026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:06.085872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950255+0000","last_change":"2023-10-21T16:03:35.248275+0000","last_active":"2023-10-21T16:03:36.950255+0000","last_peered":"2023-10-21T16:03:36.950255+0000","last_clean":"2023-10-21T16:03:36.950255+0000","last_became_active":"2023-10-21T16:03:35.248029+0000","last_became_peered":"2023-10-21T16:03:35.248029+0000","last_unstale":"2023-10-21T16:03:36.950255+0000","last_undegraded":"2023-10-21T16:03:36.950255+0000","last_fullsized":"2023-10-21T16:03:36.950255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:02.475292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109565+0000","last_change":"2023-10-21T16:03:35.968152+0000","last_active":"2023-10-21T16:03:37.109565+0000","last_peered":"2023-10-21T16:03:37.109565+0000","last_clean":"2023-10-21T16:03:37.109565+0000","last_became_active":"2023-10-21T16:03:35.966209+0000","last_became_peered":"2023-10-21T16:03:35.966209+0000","last_unstale":"2023-10-21T16:03:37.109565+0000","last_undegraded":"2023-10-21T16:03:37.109565+0000","last_fullsized":"2023-10-21T16:03:37.109565+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111325+0000","last_change":"2023-10-21T16:03:35.251641+0000","last_active":"2023-10-21T16:03:37.111325+0000","last_peered":"2023-10-21T16:03:37.111325+0000","last_clean":"2023-10-21T16:03:37.111325+0000","last_became_active":"2023-10-21T16:03:35.251344+0000","last_became_peered":"2023-10-21T16:03:35.251344+0000","last_unstale":"2023-10-21T16:03:37.111325+0000","last_undegraded":"2023-10-21T16:03:37.111325+0000","last_fullsized":"2023-10-21T16:03:37.111325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:06.733145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111538+0000","last_change":"2023-10-21T16:03:35.978958+0000","last_active":"2023-10-21T16:03:37.111538+0000","last_peered":"2023-10-21T16:03:37.111538+0000","last_clean":"2023-10-21T16:03:37.111538+0000","last_became_active":"2023-10-21T16:03:35.978671+0000","last_became_peered":"2023-10-21T16:03:35.978671+0000","last_unstale":"2023-10-21T16:03:37.111538+0000","last_undegraded":"2023-10-21T16:03:37.111538+0000","last_fullsized":"2023-10-21T16:03:37.111538+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:01.855993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112148+0000","last_change":"2023-10-21T16:03:35.256677+0000","last_active":"2023-10-21T16:03:37.112148+0000","last_peered":"2023-10-21T16:03:37.112148+0000","last_clean":"2023-10-21T16:03:37.112148+0000","last_became_active":"2023-10-21T16:03:35.256416+0000","last_became_peered":"2023-10-21T16:03:35.256416+0000","last_unstale":"2023-10-21T16:03:37.112148+0000","last_undegraded":"2023-10-21T16:03:37.112148+0000","last_fullsized":"2023-10-21T16:03:37.112148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:09.261730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950196+0000","last_change":"2023-10-21T16:03:35.973558+0000","last_active":"2023-10-21T16:03:36.950196+0000","last_peered":"2023-10-21T16:03:36.950196+0000","last_clean":"2023-10-21T16:03:36.950196+0000","last_became_active":"2023-10-21T16:03:35.972353+0000","last_became_peered":"2023-10-21T16:03:35.972353+0000","last_unstale":"2023-10-21T16:03:36.950196+0000","last_undegraded":"2023-10-21T16:03:36.950196+0000","last_fullsized":"2023-10-21T16:03:36.950196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:07.746827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944730+0000","last_change":"2023-10-21T16:03:35.257801+0000","last_active":"2023-10-21T16:03:36.944730+0000","last_peered":"2023-10-21T16:03:36.944730+0000","last_clean":"2023-10-21T16:03:36.944730+0000","last_became_active":"2023-10-21T16:03:35.257497+0000","last_became_peered":"2023-10-21T16:03:35.257497+0000","last_unstale":"2023-10-21T16:03:36.944730+0000","last_undegraded":"2023-10-21T16:03:36.944730+0000","last_fullsized":"2023-10-21T16:03:36.944730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:56.238578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108839+0000","last_change":"2023-10-21T16:03:35.975278+0000","last_active":"2023-10-21T16:03:37.108839+0000","last_peered":"2023-10-21T16:03:37.108839+0000","last_clean":"2023-10-21T16:03:37.108839+0000","last_became_active":"2023-10-21T16:03:35.974029+0000","last_became_peered":"2023-10-21T16:03:35.974029+0000","last_unstale":"2023-10-21T16:03:37.108839+0000","last_undegraded":"2023-10-21T16:03:37.108839+0000","last_fullsized":"2023-10-21T16:03:37.108839+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:52.770641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949491+0000","last_change":"2023-10-21T16:03:35.242677+0000","last_active":"2023-10-21T16:03:36.949491+0000","last_peered":"2023-10-21T16:03:36.949491+0000","last_clean":"2023-10-21T16:03:36.949491+0000","last_became_active":"2023-10-21T16:03:35.242319+0000","last_became_peered":"2023-10-21T16:03:35.242319+0000","last_unstale":"2023-10-21T16:03:36.949491+0000","last_undegraded":"2023-10-21T16:03:36.949491+0000","last_fullsized":"2023-10-21T16:03:36.949491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:13.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948553+0000","last_change":"2023-10-21T16:03:35.973905+0000","last_active":"2023-10-21T16:03:36.948553+0000","last_peered":"2023-10-21T16:03:36.948553+0000","last_clean":"2023-10-21T16:03:36.948553+0000","last_became_active":"2023-10-21T16:03:35.973634+0000","last_became_peered":"2023-10-21T16:03:35.973634+0000","last_unstale":"2023-10-21T16:03:36.948553+0000","last_undegraded":"2023-10-21T16:03:36.948553+0000","last_fullsized":"2023-10-21T16:03:36.948553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:29.567131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110240+0000","last_change":"2023-10-21T16:03:35.249161+0000","last_active":"2023-10-21T16:03:37.110240+0000","last_peered":"2023-10-21T16:03:37.110240+0000","last_clean":"2023-10-21T16:03:37.110240+0000","last_became_active":"2023-10-21T16:03:35.248865+0000","last_became_peered":"2023-10-21T16:03:35.248865+0000","last_unstale":"2023-10-21T16:03:37.110240+0000","last_undegraded":"2023-10-21T16:03:37.110240+0000","last_fullsized":"2023-10-21T16:03:37.110240+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:45.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113870+0000","last_change":"2023-10-21T16:03:35.976123+0000","last_active":"2023-10-21T16:03:37.113870+0000","last_peered":"2023-10-21T16:03:37.113870+0000","last_clean":"2023-10-21T16:03:37.113870+0000","last_became_active":"2023-10-21T16:03:35.975847+0000","last_became_peered":"2023-10-21T16:03:35.975847+0000","last_unstale":"2023-10-21T16:03:37.113870+0000","last_undegraded":"2023-10-21T16:03:37.113870+0000","last_fullsized":"2023-10-21T16:03:37.113870+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113955+0000","last_change":"2023-10-21T16:03:35.244588+0000","last_active":"2023-10-21T16:03:37.113955+0000","last_peered":"2023-10-21T16:03:37.113955+0000","last_clean":"2023-10-21T16:03:37.113955+0000","last_became_active":"2023-10-21T16:03:35.244301+0000","last_became_peered":"2023-10-21T16:03:35.244301+0000","last_unstale":"2023-10-21T16:03:37.113955+0000","last_undegraded":"2023-10-21T16:03:37.113955+0000","last_fullsized":"2023-10-21T16:03:37.113955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:34:47.476019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110669+0000","last_change":"2023-10-21T16:03:35.974229+0000","last_active":"2023-10-21T16:03:37.110669+0000","last_peered":"2023-10-21T16:03:37.110669+0000","last_clean":"2023-10-21T16:03:37.110669+0000","last_became_active":"2023-10-21T16:03:35.973946+0000","last_became_peered":"2023-10-21T16:03:35.973946+0000","last_unstale":"2023-10-21T16:03:37.110669+0000","last_undegraded":"2023-10-21T16:03:37.110669+0000","last_fullsized":"2023-10-21T16:03:37.110669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:55.578300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113930+0000","last_change":"2023-10-21T16:03:35.254230+0000","last_active":"2023-10-21T16:03:37.113930+0000","last_peered":"2023-10-21T16:03:37.113930+0000","last_clean":"2023-10-21T16:03:37.113930+0000","last_became_active":"2023-10-21T16:03:35.253801+0000","last_became_peered":"2023-10-21T16:03:35.253801+0000","last_unstale":"2023-10-21T16:03:37.113930+0000","last_undegraded":"2023-10-21T16:03:37.113930+0000","last_fullsized":"2023-10-21T16:03:37.113930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:49.046573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950991+0000","last_change":"2023-10-21T16:03:35.975452+0000","last_active":"2023-10-21T16:03:36.950991+0000","last_peered":"2023-10-21T16:03:36.950991+0000","last_clean":"2023-10-21T16:03:36.950991+0000","last_became_active":"2023-10-21T16:03:35.975173+0000","last_became_peered":"2023-10-21T16:03:35.975173+0000","last_unstale":"2023-10-21T16:03:36.950991+0000","last_undegraded":"2023-10-21T16:03:36.950991+0000","last_fullsized":"2023-10-21T16:03:36.950991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:55.578300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108865+0000","last_change":"2023-10-21T16:03:35.251735+0000","last_active":"2023-10-21T16:03:37.108865+0000","last_peered":"2023-10-21T16:03:37.108865+0000","last_clean":"2023-10-21T16:03:37.108865+0000","last_became_active":"2023-10-21T16:03:35.251449+0000","last_became_peered":"2023-10-21T16:03:35.251449+0000","last_unstale":"2023-10-21T16:03:37.108865+0000","last_undegraded":"2023-10-21T16:03:37.108865+0000","last_fullsized":"2023-10-21T16:03:37.108865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:40.625703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108715+0000","last_change":"2023-10-21T16:03:35.973338+0000","last_active":"2023-10-21T16:03:37.108715+0000","last_peered":"2023-10-21T16:03:37.108715+0000","last_clean":"2023-10-21T16:03:37.108715+0000","last_became_active":"2023-10-21T16:03:35.973047+0000","last_became_peered":"2023-10-21T16:03:35.973047+0000","last_unstale":"2023-10-21T16:03:37.108715+0000","last_undegraded":"2023-10-21T16:03:37.108715+0000","last_fullsized":"2023-10-21T16:03:37.108715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109489+0000","last_change":"2023-10-21T16:03:35.245144+0000","last_active":"2023-10-21T16:03:37.109489+0000","last_peered":"2023-10-21T16:03:37.109489+0000","last_clean":"2023-10-21T16:03:37.109489+0000","last_became_active":"2023-10-21T16:03:35.244663+0000","last_became_peered":"2023-10-21T16:03:35.244663+0000","last_unstale":"2023-10-21T16:03:37.109489+0000","last_undegraded":"2023-10-21T16:03:37.109489+0000","last_fullsized":"2023-10-21T16:03:37.109489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:00.842311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108529+0000","last_change":"2023-10-21T16:03:35.965705+0000","last_active":"2023-10-21T16:03:37.108529+0000","last_peered":"2023-10-21T16:03:37.108529+0000","last_clean":"2023-10-21T16:03:37.108529+0000","last_became_active":"2023-10-21T16:03:35.965378+0000","last_became_peered":"2023-10-21T16:03:35.965378+0000","last_unstale":"2023-10-21T16:03:37.108529+0000","last_undegraded":"2023-10-21T16:03:37.108529+0000","last_fullsized":"2023-10-21T16:03:37.108529+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:14.604478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944730+0000","last_change":"2023-10-21T16:03:35.257206+0000","last_active":"2023-10-21T16:03:36.944730+0000","last_peered":"2023-10-21T16:03:36.944730+0000","last_clean":"2023-10-21T16:03:36.944730+0000","last_became_active":"2023-10-21T16:03:35.256937+0000","last_became_peered":"2023-10-21T16:03:35.256937+0000","last_unstale":"2023-10-21T16:03:36.944730+0000","last_undegraded":"2023-10-21T16:03:36.944730+0000","last_fullsized":"2023-10-21T16:03:36.944730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:28:02.475292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111673+0000","last_change":"2023-10-21T16:03:35.976267+0000","last_active":"2023-10-21T16:03:37.111673+0000","last_peered":"2023-10-21T16:03:37.111673+0000","last_clean":"2023-10-21T16:03:37.111673+0000","last_became_active":"2023-10-21T16:03:35.973415+0000","last_became_peered":"2023-10-21T16:03:35.973415+0000","last_unstale":"2023-10-21T16:03:37.111673+0000","last_undegraded":"2023-10-21T16:03:37.111673+0000","last_fullsized":"2023-10-21T16:03:37.111673+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:57.252260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110063+0000","last_change":"2023-10-21T16:03:35.251081+0000","last_active":"2023-10-21T16:03:37.110063+0000","last_peered":"2023-10-21T16:03:37.110063+0000","last_clean":"2023-10-21T16:03:37.110063+0000","last_became_active":"2023-10-21T16:03:35.250827+0000","last_became_peered":"2023-10-21T16:03:35.250827+0000","last_unstale":"2023-10-21T16:03:37.110063+0000","last_undegraded":"2023-10-21T16:03:37.110063+0000","last_fullsized":"2023-10-21T16:03:37.110063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:04:30.130546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109216+0000","last_change":"2023-10-21T16:03:35.965925+0000","last_active":"2023-10-21T16:03:37.109216+0000","last_peered":"2023-10-21T16:03:37.109216+0000","last_clean":"2023-10-21T16:03:37.109216+0000","last_became_active":"2023-10-21T16:03:35.965120+0000","last_became_peered":"2023-10-21T16:03:35.965120+0000","last_unstale":"2023-10-21T16:03:37.109216+0000","last_undegraded":"2023-10-21T16:03:37.109216+0000","last_fullsized":"2023-10-21T16:03:37.109216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:14:36.297055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109323+0000","last_change":"2023-10-21T16:03:35.258028+0000","last_active":"2023-10-21T16:03:37.109323+0000","last_peered":"2023-10-21T16:03:37.109323+0000","last_clean":"2023-10-21T16:03:37.109323+0000","last_became_active":"2023-10-21T16:03:35.257316+0000","last_became_peered":"2023-10-21T16:03:35.257316+0000","last_unstale":"2023-10-21T16:03:37.109323+0000","last_undegraded":"2023-10-21T16:03:37.109323+0000","last_fullsized":"2023-10-21T16:03:37.109323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:17:55.884533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108771+0000","last_change":"2023-10-21T16:03:35.975586+0000","last_active":"2023-10-21T16:03:37.108771+0000","last_peered":"2023-10-21T16:03:37.108771+0000","last_clean":"2023-10-21T16:03:37.108771+0000","last_became_active":"2023-10-21T16:03:35.975271+0000","last_became_peered":"2023-10-21T16:03:35.975271+0000","last_unstale":"2023-10-21T16:03:37.108771+0000","last_undegraded":"2023-10-21T16:03:37.108771+0000","last_fullsized":"2023-10-21T16:03:37.108771+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:51.725141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112600+0000","last_change":"2023-10-21T16:03:35.258356+0000","last_active":"2023-10-21T16:03:37.112600+0000","last_peered":"2023-10-21T16:03:37.112600+0000","last_clean":"2023-10-21T16:03:37.112600+0000","last_became_active":"2023-10-21T16:03:35.257132+0000","last_became_peered":"2023-10-21T16:03:35.257132+0000","last_unstale":"2023-10-21T16:03:37.112600+0000","last_undegraded":"2023-10-21T16:03:37.112600+0000","last_fullsized":"2023-10-21T16:03:37.112600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:34:54.564618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113776+0000","last_change":"2023-10-21T16:03:35.981591+0000","last_active":"2023-10-21T16:03:37.113776+0000","last_peered":"2023-10-21T16:03:37.113776+0000","last_clean":"2023-10-21T16:03:37.113776+0000","last_became_active":"2023-10-21T16:03:35.980501+0000","last_became_peered":"2023-10-21T16:03:35.980501+0000","last_unstale":"2023-10-21T16:03:37.113776+0000","last_undegraded":"2023-10-21T16:03:37.113776+0000","last_fullsized":"2023-10-21T16:03:37.113776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:46.170039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950834+0000","last_change":"2023-10-21T16:03:35.253716+0000","last_active":"2023-10-21T16:03:36.950834+0000","last_peered":"2023-10-21T16:03:36.950834+0000","last_clean":"2023-10-21T16:03:36.950834+0000","last_became_active":"2023-10-21T16:03:35.249915+0000","last_became_peered":"2023-10-21T16:03:35.249915+0000","last_unstale":"2023-10-21T16:03:36.950834+0000","last_undegraded":"2023-10-21T16:03:36.950834+0000","last_fullsized":"2023-10-21T16:03:36.950834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:52:56.238578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.950627+0000","last_change":"2023-10-21T16:03:35.960275+0000","last_active":"2023-10-21T16:03:36.950627+0000","last_peered":"2023-10-21T16:03:36.950627+0000","last_clean":"2023-10-21T16:03:36.950627+0000","last_became_active":"2023-10-21T16:03:35.959878+0000","last_became_peered":"2023-10-21T16:03:35.959878+0000","last_unstale":"2023-10-21T16:03:36.950627+0000","last_undegraded":"2023-10-21T16:03:36.950627+0000","last_fullsized":"2023-10-21T16:03:36.950627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:50.060255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948634+0000","last_change":"2023-10-21T16:03:35.254518+0000","last_active":"2023-10-21T16:03:36.948634+0000","last_peered":"2023-10-21T16:03:36.948634+0000","last_clean":"2023-10-21T16:03:36.948634+0000","last_became_active":"2023-10-21T16:03:35.254140+0000","last_became_peered":"2023-10-21T16:03:35.254140+0000","last_unstale":"2023-10-21T16:03:36.948634+0000","last_undegraded":"2023-10-21T16:03:36.948634+0000","last_fullsized":"2023-10-21T16:03:36.948634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:12.360673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948464+0000","last_change":"2023-10-21T16:03:35.970456+0000","last_active":"2023-10-21T16:03:36.948464+0000","last_peered":"2023-10-21T16:03:36.948464+0000","last_clean":"2023-10-21T16:03:36.948464+0000","last_became_active":"2023-10-21T16:03:35.970098+0000","last_became_peered":"2023-10-21T16:03:35.970098+0000","last_unstale":"2023-10-21T16:03:36.948464+0000","last_undegraded":"2023-10-21T16:03:36.948464+0000","last_fullsized":"2023-10-21T16:03:36.948464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:04:31.144228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944985+0000","last_change":"2023-10-21T16:03:35.256557+0000","last_active":"2023-10-21T16:03:36.944985+0000","last_peered":"2023-10-21T16:03:36.944985+0000","last_clean":"2023-10-21T16:03:36.944985+0000","last_became_active":"2023-10-21T16:03:35.256265+0000","last_became_peered":"2023-10-21T16:03:35.256265+0000","last_unstale":"2023-10-21T16:03:36.944985+0000","last_undegraded":"2023-10-21T16:03:36.944985+0000","last_fullsized":"2023-10-21T16:03:36.944985+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:39.747058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.111959+0000","last_change":"2023-10-21T16:03:35.978613+0000","last_active":"2023-10-21T16:03:37.111959+0000","last_peered":"2023-10-21T16:03:37.111959+0000","last_clean":"2023-10-21T16:03:37.111959+0000","last_became_active":"2023-10-21T16:03:35.978309+0000","last_became_peered":"2023-10-21T16:03:35.978309+0000","last_unstale":"2023-10-21T16:03:37.111959+0000","last_undegraded":"2023-10-21T16:03:37.111959+0000","last_fullsized":"2023-10-21T16:03:37.111959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:08.467477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948070+0000","last_change":"2023-10-21T16:03:35.240577+0000","last_active":"2023-10-21T16:03:36.948070+0000","last_peered":"2023-10-21T16:03:36.948070+0000","last_clean":"2023-10-21T16:03:36.948070+0000","last_became_active":"2023-10-21T16:03:35.240292+0000","last_became_peered":"2023-10-21T16:03:35.240292+0000","last_unstale":"2023-10-21T16:03:36.948070+0000","last_undegraded":"2023-10-21T16:03:36.948070+0000","last_fullsized":"2023-10-21T16:03:36.948070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:35.909043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109149+0000","last_change":"2023-10-21T16:03:35.967733+0000","last_active":"2023-10-21T16:03:37.109149+0000","last_peered":"2023-10-21T16:03:37.109149+0000","last_clean":"2023-10-21T16:03:37.109149+0000","last_became_active":"2023-10-21T16:03:35.967439+0000","last_became_peered":"2023-10-21T16:03:35.967439+0000","last_unstale":"2023-10-21T16:03:37.109149+0000","last_undegraded":"2023-10-21T16:03:37.109149+0000","last_fullsized":"2023-10-21T16:03:37.109149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948135+0000","last_change":"2023-10-21T16:03:35.249102+0000","last_active":"2023-10-21T16:03:36.948135+0000","last_peered":"2023-10-21T16:03:36.948135+0000","last_clean":"2023-10-21T16:03:36.948135+0000","last_became_active":"2023-10-21T16:03:35.248793+0000","last_became_peered":"2023-10-21T16:03:35.248793+0000","last_unstale":"2023-10-21T16:03:36.948135+0000","last_undegraded":"2023-10-21T16:03:36.948135+0000","last_fullsized":"2023-10-21T16:03:36.948135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:52.663008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112318+0000","last_change":"2023-10-21T16:03:35.983491+0000","last_active":"2023-10-21T16:03:37.112318+0000","last_peered":"2023-10-21T16:03:37.112318+0000","last_clean":"2023-10-21T16:03:37.112318+0000","last_became_active":"2023-10-21T16:03:35.981131+0000","last_became_peered":"2023-10-21T16:03:35.981131+0000","last_unstale":"2023-10-21T16:03:37.112318+0000","last_undegraded":"2023-10-21T16:03:37.112318+0000","last_fullsized":"2023-10-21T16:03:37.112318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:36:22.885882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113448+0000","last_change":"2023-10-21T16:03:35.252926+0000","last_active":"2023-10-21T16:03:37.113448+0000","last_peered":"2023-10-21T16:03:37.113448+0000","last_clean":"2023-10-21T16:03:37.113448+0000","last_became_active":"2023-10-21T16:03:35.252623+0000","last_became_peered":"2023-10-21T16:03:35.252623+0000","last_unstale":"2023-10-21T16:03:37.113448+0000","last_undegraded":"2023-10-21T16:03:37.113448+0000","last_fullsized":"2023-10-21T16:03:37.113448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:39.131909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109437+0000","last_change":"2023-10-21T16:03:35.975257+0000","last_active":"2023-10-21T16:03:37.109437+0000","last_peered":"2023-10-21T16:03:37.109437+0000","last_clean":"2023-10-21T16:03:37.109437+0000","last_became_active":"2023-10-21T16:03:35.972983+0000","last_became_peered":"2023-10-21T16:03:35.972983+0000","last_unstale":"2023-10-21T16:03:37.109437+0000","last_undegraded":"2023-10-21T16:03:37.109437+0000","last_fullsized":"2023-10-21T16:03:37.109437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:02:44.039250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.948009+0000","last_change":"2023-10-21T16:03:35.246281+0000","last_active":"2023-10-21T16:03:36.948009+0000","last_peered":"2023-10-21T16:03:36.948009+0000","last_clean":"2023-10-21T16:03:36.948009+0000","last_became_active":"2023-10-21T16:03:35.246012+0000","last_became_peered":"2023-10-21T16:03:35.246012+0000","last_unstale":"2023-10-21T16:03:36.948009+0000","last_undegraded":"2023-10-21T16:03:36.948009+0000","last_fullsized":"2023-10-21T16:03:36.948009+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:01.677983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944866+0000","last_change":"2023-10-21T16:03:35.971443+0000","last_active":"2023-10-21T16:03:36.944866+0000","last_peered":"2023-10-21T16:03:36.944866+0000","last_clean":"2023-10-21T16:03:36.944866+0000","last_became_active":"2023-10-21T16:03:35.971183+0000","last_became_peered":"2023-10-21T16:03:35.971183+0000","last_unstale":"2023-10-21T16:03:36.944866+0000","last_undegraded":"2023-10-21T16:03:36.944866+0000","last_fullsized":"2023-10-21T16:03:36.944866+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:22:38.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113037+0000","last_change":"2023-10-21T16:03:35.249268+0000","last_active":"2023-10-21T16:03:37.113037+0000","last_peered":"2023-10-21T16:03:37.113037+0000","last_clean":"2023-10-21T16:03:37.113037+0000","last_became_active":"2023-10-21T16:03:35.248956+0000","last_became_peered":"2023-10-21T16:03:35.248956+0000","last_unstale":"2023-10-21T16:03:37.113037+0000","last_undegraded":"2023-10-21T16:03:37.113037+0000","last_fullsized":"2023-10-21T16:03:37.113037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:26:16.532600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113005+0000","last_change":"2023-10-21T16:03:35.982746+0000","last_active":"2023-10-21T16:03:37.113005+0000","last_peered":"2023-10-21T16:03:37.113005+0000","last_clean":"2023-10-21T16:03:37.113005+0000","last_became_active":"2023-10-21T16:03:35.981895+0000","last_became_peered":"2023-10-21T16:03:35.981895+0000","last_unstale":"2023-10-21T16:03:37.113005+0000","last_undegraded":"2023-10-21T16:03:37.113005+0000","last_fullsized":"2023-10-21T16:03:37.113005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:10.759306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.427846+0000","last_change":"2023-10-21T16:03:35.238616+0000","last_active":"2023-10-21T16:03:37.427846+0000","last_peered":"2023-10-21T16:03:37.427846+0000","last_clean":"2023-10-21T16:03:37.427846+0000","last_became_active":"2023-10-21T16:03:35.238161+0000","last_became_peered":"2023-10-21T16:03:35.238161+0000","last_unstale":"2023-10-21T16:03:37.427846+0000","last_undegraded":"2023-10-21T16:03:37.427846+0000","last_fullsized":"2023-10-21T16:03:37.427846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:23.215304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109881+0000","last_change":"2023-10-21T16:03:35.975555+0000","last_active":"2023-10-21T16:03:37.109881+0000","last_peered":"2023-10-21T16:03:37.109881+0000","last_clean":"2023-10-21T16:03:37.109881+0000","last_became_active":"2023-10-21T16:03:35.974303+0000","last_became_peered":"2023-10-21T16:03:35.974303+0000","last_unstale":"2023-10-21T16:03:37.109881+0000","last_undegraded":"2023-10-21T16:03:37.109881+0000","last_fullsized":"2023-10-21T16:03:37.109881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:31.749731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110277+0000","last_change":"2023-10-21T16:03:35.252320+0000","last_active":"2023-10-21T16:03:37.110277+0000","last_peered":"2023-10-21T16:03:37.110277+0000","last_clean":"2023-10-21T16:03:37.110277+0000","last_became_active":"2023-10-21T16:03:35.251995+0000","last_became_peered":"2023-10-21T16:03:35.251995+0000","last_unstale":"2023-10-21T16:03:37.110277+0000","last_undegraded":"2023-10-21T16:03:37.110277+0000","last_fullsized":"2023-10-21T16:03:37.110277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:07.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.109085+0000","last_change":"2023-10-21T16:03:35.965099+0000","last_active":"2023-10-21T16:03:37.109085+0000","last_peered":"2023-10-21T16:03:37.109085+0000","last_clean":"2023-10-21T16:03:37.109085+0000","last_became_active":"2023-10-21T16:03:35.964769+0000","last_became_peered":"2023-10-21T16:03:35.964769+0000","last_unstale":"2023-10-21T16:03:37.109085+0000","last_undegraded":"2023-10-21T16:03:37.109085+0000","last_fullsized":"2023-10-21T16:03:37.109085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110272+0000","last_change":"2023-10-21T16:03:35.252315+0000","last_active":"2023-10-21T16:03:37.110272+0000","last_peered":"2023-10-21T16:03:37.110272+0000","last_clean":"2023-10-21T16:03:37.110272+0000","last_became_active":"2023-10-21T16:03:35.251813+0000","last_became_peered":"2023-10-21T16:03:35.251813+0000","last_unstale":"2023-10-21T16:03:37.110272+0000","last_undegraded":"2023-10-21T16:03:37.110272+0000","last_fullsized":"2023-10-21T16:03:37.110272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:13.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.944669+0000","last_change":"2023-10-21T16:03:35.972407+0000","last_active":"2023-10-21T16:03:36.944669+0000","last_peered":"2023-10-21T16:03:36.944669+0000","last_clean":"2023-10-21T16:03:36.944669+0000","last_became_active":"2023-10-21T16:03:35.972151+0000","last_became_peered":"2023-10-21T16:03:35.972151+0000","last_unstale":"2023-10-21T16:03:36.944669+0000","last_undegraded":"2023-10-21T16:03:36.944669+0000","last_fullsized":"2023-10-21T16:03:36.944669+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.110587+0000","last_change":"2023-10-21T16:03:35.246336+0000","last_active":"2023-10-21T16:03:37.110587+0000","last_peered":"2023-10-21T16:03:37.110587+0000","last_clean":"2023-10-21T16:03:37.110587+0000","last_became_active":"2023-10-21T16:03:35.246024+0000","last_became_peered":"2023-10-21T16:03:35.246024+0000","last_unstale":"2023-10-21T16:03:37.110587+0000","last_undegraded":"2023-10-21T16:03:37.110587+0000","last_fullsized":"2023-10-21T16:03:37.110587+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:28.553449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949856+0000","last_change":"2023-10-21T16:03:35.961422+0000","last_active":"2023-10-21T16:03:36.949856+0000","last_peered":"2023-10-21T16:03:36.949856+0000","last_clean":"2023-10-21T16:03:36.949856+0000","last_became_active":"2023-10-21T16:03:35.960740+0000","last_became_peered":"2023-10-21T16:03:35.960740+0000","last_unstale":"2023-10-21T16:03:36.949856+0000","last_undegraded":"2023-10-21T16:03:36.949856+0000","last_fullsized":"2023-10-21T16:03:36.949856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:51.514046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:36.949513+0000","last_change":"2023-10-21T16:03:35.248648+0000","last_active":"2023-10-21T16:03:36.949513+0000","last_peered":"2023-10-21T16:03:36.949513+0000","last_clean":"2023-10-21T16:03:36.949513+0000","last_became_active":"2023-10-21T16:03:35.248313+0000","last_became_peered":"2023-10-21T16:03:35.248313+0000","last_unstale":"2023-10-21T16:03:36.949513+0000","last_undegraded":"2023-10-21T16:03:36.949513+0000","last_fullsized":"2023-10-21T16:03:36.949513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:39.747058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112374+0000","last_change":"2023-10-21T16:03:35.974517+0000","last_active":"2023-10-21T16:03:37.112374+0000","last_peered":"2023-10-21T16:03:37.112374+0000","last_clean":"2023-10-21T16:03:37.112374+0000","last_became_active":"2023-10-21T16:03:35.972628+0000","last_became_peered":"2023-10-21T16:03:35.972628+0000","last_unstale":"2023-10-21T16:03:37.112374+0000","last_undegraded":"2023-10-21T16:03:37.112374+0000","last_fullsized":"2023-10-21T16:03:37.112374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:29.158841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.108300+0000","last_change":"2023-10-21T16:03:35.241484+0000","last_active":"2023-10-21T16:03:37.108300+0000","last_peered":"2023-10-21T16:03:37.108300+0000","last_clean":"2023-10-21T16:03:37.108300+0000","last_became_active":"2023-10-21T16:03:35.241199+0000","last_became_peered":"2023-10-21T16:03:35.241199+0000","last_unstale":"2023-10-21T16:03:37.108300+0000","last_undegraded":"2023-10-21T16:03:37.108300+0000","last_fullsized":"2023-10-21T16:03:37.108300+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:49.193107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.112144+0000","last_change":"2023-10-21T16:03:35.976782+0000","last_active":"2023-10-21T16:03:37.112144+0000","last_peered":"2023-10-21T16:03:37.112144+0000","last_clean":"2023-10-21T16:03:37.112144+0000","last_became_active":"2023-10-21T16:03:35.976508+0000","last_became_peered":"2023-10-21T16:03:35.976508+0000","last_unstale":"2023-10-21T16:03:37.112144+0000","last_undegraded":"2023-10-21T16:03:37.112144+0000","last_fullsized":"2023-10-21T16:03:37.112144+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:35.205638+0000","last_clean_scrub_stamp":"2023-10-21T16:03:35.205638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:54:41.639386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-10-21T16:03:37.113499+0000","last_change":"2023-10-21T16:03:35.250845+0000","last_active":"2023-10-21T16:03:37.113499+0000","last_peered":"2023-10-21T16:03:37.113499+0000","last_clean":"2023-10-21T16:03:37.113499+0000","last_became_active":"2023-10-21T16:03:35.250511+0000","last_became_peered":"2023-10-21T16:03:35.250511+0000","last_unstale":"2023-10-21T16:03:37.113499+0000","last_undegraded":"2023-10-21T16:03:37.113499+0000","last_fullsized":"2023-10-21T16:03:37.113499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:03:34.191956+0000","last_clean_scrub_stamp":"2023-10-21T16:03:34.191956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:28.553449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-21T16:03:35.962277+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:03:35.962277+0000","last_peered":"2023-10-21T16:03:35.962277+0000","last_clean":"2023-10-21T16:03:35.962277+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:03:35.962277+0000","last_undegraded":"2023-10-21T16:03:35.962277+0000","last_fullsized":"2023-10-21T16:03:35.962277+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1073152,"data_stored":792946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.5900000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.478},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.6379999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.823},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.387}]},{"osd":3,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.9830000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.679},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.778}]},{"osd":5,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":2.1059999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.7270000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.657,"5min":1.657,"15min":1.657},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":1.883},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.6970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":201478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.93400000000000005}]},{"osd":1,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.034}]},{"osd":2,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.097},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.3120000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.21}]},{"osd":4,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":1.256},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.5470000000000002,"5min":2.5470000000000002,"15min":2.5470000000000002},"last":1.3700000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:02:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4930000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"max":{"1min":3.71,"5min":3.71,"15min":3.71},"last":1.444},{"interface":"front","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":1.105,"5min":1.105,"15min":1.105},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":1.534}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":201008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.80200000000000005}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3899999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.278},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.997}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.032},{"interface":"front","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.5600000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.878},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.681},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.1100000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.077,"5min":2.077,"15min":2.077},"last":1.331},{"interface":"front","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.4770000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":201500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":3.6080000000000001,"5min":3.6080000000000001,"15min":3.6080000000000001},"last":1.401}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.80200000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2,"5min":2,"15min":2},"last":0.84499999999999997}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5,"15min":1.5},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":1.02,"5min":1.02,"15min":1.02},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.97599999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":1.631},{"interface":"front","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":3.9300000000000002,"5min":3.9300000000000002,"15min":3.9300000000000002},"last":1.2130000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.21},"last":1.452},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.9609999999999999,"5min":3.9609999999999999,"15min":3.9609999999999999},"last":1.788}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":1.784,"15min":1.784},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.548}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":479232,"data_stored":202542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.393},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.02}]},{"osd":1,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.1419999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.5469999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.597},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.2869999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.7250000000000001,"5min":3.7250000000000001,"15min":3.7250000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.1930000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":1.456},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":3.573,"5min":3.573,"15min":3.573},"last":1.6459999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:02:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.704}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31648,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340192,"statfs":{"total":96636764160,"available":96604356608,"internally_reserved":0,"allocated":466944,"data_stored":200966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.667},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.5329999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":1.7869999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.169}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.488,"15min":1.488},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.476}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.413},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"min":{"1min":1.05,"5min":1.05,"15min":1.05},"max":{"1min":3.6150000000000002,"5min":3.6150000000000002,"15min":3.6150000000000002},"last":1.722}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":200986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.292}]},{"osd":3,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.756}]},{"osd":4,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.377},{"interface":"front","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.907}]},{"osd":5,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6030000000000002,"5min":2.6030000000000002,"15min":2.6030000000000002},"last":1.998},{"interface":"front","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.2570000000000001,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.4890000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":4.1260000000000003,"5min":4.1260000000000003,"15min":4.1260000000000003},"last":2.198},{"interface":"front","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.101,"5min":3.101,"15min":3.101},"last":1.8140000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:02:50 2023","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":2.8180000000000001,"5min":2.8180000000000001,"15min":2.8180000000000001},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":3.952,"5min":3.952,"15min":3.952},"last":2.3130000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":791298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.1060000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.49,"15min":1.49},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":3.306,"5min":3.306,"15min":3.306},"last":1.5009999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":3.8849999999999998,"5min":3.8849999999999998,"15min":3.8849999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":3.1789999999999998,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.6080000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":4.3019999999999996,"5min":4.3019999999999996,"15min":4.3019999999999996},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":1.6659999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:02:53 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":3.8140000000000001,"5min":3.8140000000000001,"15min":3.8140000000000001},"last":0.996},{"interface":"front","average":{"1min":1.72,"5min":1.72,"15min":1.72},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":1.444}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:03:41.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:03:41.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:03:41.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.573+0000 7ffb4eeca640 1 Processor -- start 2023-10-21T16:03:41.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.573+0000 7ffb4eeca640 1 -- start start 2023-10-21T16:03:41.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4eeca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb480ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4eeca640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb480c9c90 con 0x7ffb4815de70 2023-10-21T16:03:41.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4eeca640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4815e250 con 0x7ffb4815e490 2023-10-21T16:03:41.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4eeca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4815f480 con 0x7ffb480cb970 2023-10-21T16:03:41.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4dec8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ffb4815de70 0x7ffb480cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55678/0 (socket says 172.21.15.53:55678) 2023-10-21T16:03:41.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.574+0000 7ffb4dec8640 1 -- 172.21.15.53:0/3009450719 learned_addr learned my addr 172.21.15.53:0/3009450719 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873136964 0 0) 0x7ffb480c9c90 con 0x7ffb4815de70 2023-10-21T16:03:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb480ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003520 con 0x7ffb4815de70 2023-10-21T16:03:41.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4901429 0 0) 0x7ffb34003520 con 0x7ffb4815de70 2023-10-21T16:03:41.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 msgr2=0x7ffb480ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:41.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb480ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 >> v1:172.21.15.124:6789/0 conn(0x7ffb4815e490 legacy=0x7ffb4815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4815f1a0 con 0x7ffb4815de70 2023-10-21T16:03:41.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.575+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ffb44001670 con 0x7ffb4815de70 2023-10-21T16:03:41.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 >> v1:172.21.15.53:6789/0 conn(0x7ffb4815de70 legacy=0x7ffb480cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb34003f20 0x7ffb340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb34004d00 0x7ffb34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4815f480 con 0x7ffb34003f20 2023-10-21T16:03:41.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3400ba60 con 0x7ffb34004d00 2023-10-21T16:03:41.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3400bc20 con 0x7ffb480cb970 2023-10-21T16:03:41.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb34003f20 0x7ffb340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4dec8640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 msgr2=0x7ffb340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:41.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4e6c9640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb34004d00 0x7ffb34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb480cb970 0x7ffb340039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb34004d00 msgr2=0x7ffb34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:41.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb34004d00 0x7ffb34005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/3009450719 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34003520 con 0x7ffb34003f20 2023-10-21T16:03:41.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.576+0000 7ffb4e6c9640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb34004d00 0x7ffb34005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:41.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb34003f20 0x7ffb340047c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ac0 tx=0x7ffb340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140adfcedbabf50c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802c6b0 con 0x7ffb34003f20 2023-10-21T16:03:41.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3802c810 con 0x7ffb34003f20 2023-10-21T16:03:41.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4cec6640 1 -- 172.21.15.53:0/3009450719 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802cba0 con 0x7ffb34003f20 2023-10-21T16:03:41.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4eeca640 1 -- 172.21.15.53:0/3009450719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb34003f20 msgr2=0x7ffb340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/3009450719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb34003f20 0x7ffb340047c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ac0 tx=0x7ffb340057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4e6c9640 1 -- 172.21.15.53:0/3009450719 reap_dead start 2023-10-21T16:03:41.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4eeca640 1 -- 172.21.15.53:0/3009450719 shutdown_connections 2023-10-21T16:03:41.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.578+0000 7ffb4eeca640 1 -- 172.21.15.53:0/3009450719 >> 172.21.15.53:0/3009450719 conn(0x7ffb480ce9f0 msgr2=0x7ffb480cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:41.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.579+0000 7ffb4eeca640 1 -- 172.21.15.53:0/3009450719 shutdown_connections 2023-10-21T16:03:41.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.579+0000 7ffb4eeca640 1 -- 172.21.15.53:0/3009450719 wait complete. 2023-10-21T16:03:41.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.580+0000 7ffb4eeca640 1 Processor -- start 2023-10-21T16:03:41.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.581+0000 7ffb4eeca640 1 -- start start 2023-10-21T16:03:41.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.581+0000 7ffb4eeca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb34003f20 0x7ffb48097aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.581+0000 7ffb4eeca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4eeca640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 0x7ffb4815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4eeca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4815d260 con 0x7ffb480c6f90 2023-10-21T16:03:41.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4eeca640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c86d0 con 0x7ffb4815c8a0 2023-10-21T16:03:41.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4eeca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480c7ac0 con 0x7ffb34003f20 2023-10-21T16:03:41.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52606/0 (socket says 172.21.15.53:52606) 2023-10-21T16:03:41.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/199298376 learned_addr learned my addr 172.21.15.53:0/199298376 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:41.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4dec8640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb34003f20 0x7ffb48097aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4e6c9640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 0x7ffb4815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb34003f20 msgr2=0x7ffb48097aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb34003f20 0x7ffb48097aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 msgr2=0x7ffb4815cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 0x7ffb4815cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4d6c7640 1 -- 172.21.15.53:0/199298376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480c7da0 con 0x7ffb480c6f90 2023-10-21T16:03:41.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.582+0000 7ffb4e6c9640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 0x7ffb4815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:41.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb4d6c7640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ac0 tx=0x7ffb38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38003810 con 0x7ffb480c6f90 2023-10-21T16:03:41.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38003970 con 0x7ffb480c6f90 2023-10-21T16:03:41.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38003d00 con 0x7ffb480c6f90 2023-10-21T16:03:41.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480c8080 con 0x7ffb480c6f90 2023-10-21T16:03:41.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.583+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb480c5880 con 0x7ffb480c6f90 2023-10-21T16:03:41.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.584+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4804e160 con 0x7ffb480c6f90 2023-10-21T16:03:41.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.585+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb38040450 con 0x7ffb480c6f90 2023-10-21T16:03:41.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.586+0000 7ffb3e7fc640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 0x7ffb2c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:41.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.586+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffb380ab710 con 0x7ffb480c6f90 2023-10-21T16:03:41.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.587+0000 7ffb4dec8640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 0x7ffb2c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:41.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.588+0000 7ffb4dec8640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 0x7ffb2c06d3b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb44000fd0 tx=0x7ffb44001600 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:41.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.590+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb38076660 con 0x7ffb480c6f90 2023-10-21T16:03:41.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.853+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffb4806cab0 con 0x7ffb480c6f90 2023-10-21T16:03:41.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.857+0000 7ffb3e7fc640 1 -- 172.21.15.53:0/199298376 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffb38003e60 con 0x7ffb480c6f90 2023-10-21T16:03:41.859 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:03:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:03:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:03:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:03:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:03:41.860 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:03:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 msgr2=0x7ffb2c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 0x7ffb2c06d3b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb44000fd0 tx=0x7ffb44001600 comp rx=0 tx=0).stop 2023-10-21T16:03:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 msgr2=0x7ffb480c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:41.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009ac0 tx=0x7ffb38004730 comp rx=0 tx=0).stop 2023-10-21T16:03:41.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 shutdown_connections 2023-10-21T16:03:41.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4815c8a0 0x7ffb4815cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7ffb2c06aef0 0x7ffb2c06d3b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb480c6f90 0x7ffb480c7410 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 --2- 172.21.15.53:0/199298376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb34003f20 0x7ffb48097aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 >> 172.21.15.53:0/199298376 conn(0x7ffb480ce9f0 msgr2=0x7ffb480ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 shutdown_connections 2023-10-21T16:03:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:41.860+0000 7ffb4eeca640 1 -- 172.21.15.53:0/199298376 wait complete. 2023-10-21T16:03:41.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:03:42.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.111+0000 7f896bfff640 1 Processor -- start 2023-10-21T16:03:42.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.111+0000 7f896bfff640 1 -- start start 2023-10-21T16:03:42.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f896c0cc210 0x7f896c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c0cacf0 con 0x7f896c1599d0 2023-10-21T16:03:42.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896c0ca3c0 con 0x7f896c159db0 2023-10-21T16:03:42.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896c0b10f0 con 0x7f896c0cc210 2023-10-21T16:03:42.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f896c1599d0 0x7f896c0ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55684/0 (socket says 172.21.15.53:55684) 2023-10-21T16:03:42.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896affd640 1 -- 172.21.15.53:0/388407967 learned_addr learned my addr 172.21.15.53:0/388407967 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:42.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.113+0000 7f896a7fc640 1 --2- 172.21.15.53:0/388407967 >> v2:172.21.15.124:3300/0 conn(0x7f896c0cc210 0x7f896c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837275358 0 0) 0x7f896c0cacf0 con 0x7f896c1599d0 2023-10-21T16:03:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8958003520 con 0x7f896c1599d0 2023-10-21T16:03:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803310370 0 0) 0x7f8958003520 con 0x7f896c1599d0 2023-10-21T16:03:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 >> v2:172.21.15.124:3300/0 conn(0x7f896c0cc210 msgr2=0x7f896c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:42.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 --2- 172.21.15.53:0/388407967 >> v2:172.21.15.124:3300/0 conn(0x7f896c0cc210 0x7f896c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 >> v1:172.21.15.124:6789/0 conn(0x7f896c159db0 legacy=0x7f896c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c0ca6a0 con 0x7f896c1599d0 2023-10-21T16:03:42.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f895c0019f0 con 0x7f896c1599d0 2023-10-21T16:03:42.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 >> v1:172.21.15.53:6789/0 conn(0x7f896c1599d0 legacy=0x7f896c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f8958003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89580047c0 0x7f8958004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8958005180 0x7f895800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c0b10f0 con 0x7f89580039c0 2023-10-21T16:03:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895800c080 con 0x7f89580047c0 2023-10-21T16:03:42.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895800c240 con 0x7f8958005180 2023-10-21T16:03:42.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.114+0000 7f896affd640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f8958003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896a7fc640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89580047c0 0x7f8958004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896b7fe640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8958005180 0x7f895800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896affd640 1 -- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8958005180 msgr2=0x7f895800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:42.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896affd640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8958005180 0x7f895800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896affd640 1 -- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89580047c0 msgr2=0x7f8958004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896affd640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89580047c0 0x7f8958004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896affd640 1 -- 172.21.15.53:0/388407967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958003520 con 0x7f89580039c0 2023-10-21T16:03:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896b7fe640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8958005180 0x7f895800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:42.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896a7fc640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89580047c0 0x7f8958004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:42.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.115+0000 7f896b7fe640 1 -- 172.21.15.53:0/388407967 reap_dead start 2023-10-21T16:03:42.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.116+0000 7f896affd640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f8958003de0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a760 tx=0x7f89580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77ea81abaa5e6c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:42.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.116+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c034630 con 0x7f89580039c0 2023-10-21T16:03:42.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.116+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c02f070 con 0x7f89580039c0 2023-10-21T16:03:42.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.116+0000 7f8969ffb640 1 -- 172.21.15.53:0/388407967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c034a50 con 0x7f89580039c0 2023-10-21T16:03:42.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 -- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 msgr2=0x7f8958003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f8958003de0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a760 tx=0x7f89580057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 -- 172.21.15.53:0/388407967 shutdown_connections 2023-10-21T16:03:42.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 --2- 172.21.15.53:0/388407967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f8958003de0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 -- 172.21.15.53:0/388407967 >> 172.21.15.53:0/388407967 conn(0x7f896c0cb0c0 msgr2=0x7f896c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 -- 172.21.15.53:0/388407967 shutdown_connections 2023-10-21T16:03:42.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.117+0000 7f896bfff640 1 -- 172.21.15.53:0/388407967 wait complete. 2023-10-21T16:03:42.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.119+0000 7f896bfff640 1 Processor -- start 2023-10-21T16:03:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.119+0000 7f896bfff640 1 -- start start 2023-10-21T16:03:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f896c0c5f40 0x7f896c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f896c0c63c0 0x7f896c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1e6bf0 con 0x7f89580039c0 2023-10-21T16:03:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1e4630 con 0x7f896c0c5f40 2023-10-21T16:03:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1510d0 con 0x7f896c0c63c0 2023-10-21T16:03:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52634/0 (socket says 172.21.15.53:52634) 2023-10-21T16:03:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.120+0000 7f896affd640 1 -- 172.21.15.53:0/483792950 learned_addr learned my addr 172.21.15.53:0/483792950 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 -- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f896c0c63c0 msgr2=0x7f896c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896b7fe640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f896c0c63c0 0x7f896c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896a7fc640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f896c0c5f40 0x7f896c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f896c0c63c0 0x7f896c0cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 -- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f896c0c5f40 msgr2=0x7f896c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f896c0c5f40 0x7f896c157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 -- 172.21.15.53:0/483792950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c0b35b0 con 0x7f89580039c0 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896affd640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f895c02cab0 tx=0x7f895c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c004d60 con 0x7f89580039c0 2023-10-21T16:03:42.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c001970 con 0x7f89580039c0 2023-10-21T16:03:42.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.121+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c0b3890 con 0x7f89580039c0 2023-10-21T16:03:42.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.122+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c034650 con 0x7f89580039c0 2023-10-21T16:03:42.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.122+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c0b3e50 con 0x7f89580039c0 2023-10-21T16:03:42.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.123+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c04e160 con 0x7f89580039c0 2023-10-21T16:03:42.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.124+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f895c031030 con 0x7f89580039c0 2023-10-21T16:03:42.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.125+0000 7f89637fe640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 0x7f894c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:42.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.125+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f895c0ab540 con 0x7f89580039c0 2023-10-21T16:03:42.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.125+0000 7f896a7fc640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 0x7f894c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:42.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.127+0000 7f896a7fc640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 0x7f894c06d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f896c1e5ab0 tx=0x7f8954008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:42.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.129+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f895c0764a0 con 0x7f89580039c0 2023-10-21T16:03:42.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.362+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f896c04e160 con 0x7f89580039c0 2023-10-21T16:03:42.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.362+0000 7f89637fe640 1 -- 172.21.15.53:0/483792950 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f895c079ac0 con 0x7f89580039c0 2023-10-21T16:03:42.364 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:03:42.365 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:03:42.365 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:03:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 msgr2=0x7f894c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 0x7f894c06d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f896c1e5ab0 tx=0x7f8954008040 comp rx=0 tx=0).stop 2023-10-21T16:03:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 msgr2=0x7f896c150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f895c02cab0 tx=0x7f895c001cb0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 shutdown_connections 2023-10-21T16:03:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f896c0c63c0 0x7f896c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f896c0c5f40 0x7f896c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f894c06ae70 0x7f894c06d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 --2- 172.21.15.53:0/483792950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89580039c0 0x7f896c150b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 >> 172.21.15.53:0/483792950 conn(0x7f896c0cb0c0 msgr2=0x7f896c0aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 shutdown_connections 2023-10-21T16:03:42.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:42.365+0000 7f896bfff640 1 -- 172.21.15.53:0/483792950 wait complete. 2023-10-21T16:03:42.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:03:35.695582+0000', 'modified': '2023-10-21T16:03:38.789790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34679': {'gid': 34679, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6833/1236966553', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1236966553}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1236966553}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:03:42.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:03:42.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-21T16:03:43.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.027+0000 7f75834e7640 1 Processor -- start 2023-10-21T16:03:43.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.027+0000 7f75834e7640 1 -- start start 2023-10-21T16:03:43.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.028+0000 7f75834e7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f757c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f75834e7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0b1cc0 con 0x7f757c0afbe0 2023-10-21T16:03:43.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f75834e7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c0b2620 con 0x7f757c071070 2023-10-21T16:03:43.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f75834e7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b2060 con 0x7f757c06dfc0 2023-10-21T16:03:43.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f7581283640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f757c0afbe0 0x7f757c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55696/0 (socket says 172.21.15.53:55696) 2023-10-21T16:03:43.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f7581283640 1 -- 172.21.15.53:0/4000469285 learned_addr learned my addr 172.21.15.53:0/4000469285 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:43.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f7580a82640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f757c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169526624 0 0) 0x7f757c0b1cc0 con 0x7f757c0afbe0 2023-10-21T16:03:43.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.029+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7568003520 con 0x7f757c0afbe0 2023-10-21T16:03:43.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909824130 0 0) 0x7f7568003520 con 0x7f757c0afbe0 2023-10-21T16:03:43.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 msgr2=0x7f757c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f757c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 >> v1:172.21.15.124:6789/0 conn(0x7f757c071070 legacy=0x7f757c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0b2340 con 0x7f757c0afbe0 2023-10-21T16:03:43.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f75780019f0 con 0x7f757c0afbe0 2023-10-21T16:03:43.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 >> v1:172.21.15.53:6789/0 conn(0x7f757c0afbe0 legacy=0x7f757c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f7568003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f75680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7568004d00 0x7f7568005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b2060 con 0x7f75680039a0 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756800bac0 con 0x7f7568004d00 2023-10-21T16:03:43.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756800bc80 con 0x7f757c06dfc0 2023-10-21T16:03:43.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.030+0000 7f7581283640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f7568003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7580a82640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f75680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581a84640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7568004d00 0x7f7568005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581283640 1 -- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 msgr2=0x7f75680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581283640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c06dfc0 0x7f75680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581283640 1 -- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7568004d00 msgr2=0x7f7568005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581283640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7568004d00 0x7f7568005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581283640 1 -- 172.21.15.53:0/4000469285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568003520 con 0x7f75680039a0 2023-10-21T16:03:43.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.031+0000 7f7581a84640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7568004d00 0x7f7568005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:43.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.032+0000 7f7581283640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f7568003dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f757800a760 tx=0x7f75680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8618496a3098cbbb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.032+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578034630 con 0x7f75680039a0 2023-10-21T16:03:43.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.032+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757802f070 con 0x7f75680039a0 2023-10-21T16:03:43.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.032+0000 7f7573fff640 1 -- 172.21.15.53:0/4000469285 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578034a10 con 0x7f75680039a0 2023-10-21T16:03:43.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 -- 172.21.15.53:0/4000469285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 msgr2=0x7f7568003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 --2- 172.21.15.53:0/4000469285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f7568003dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f757800a760 tx=0x7f75680057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f7581a84640 1 -- 172.21.15.53:0/4000469285 reap_dead start 2023-10-21T16:03:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 -- 172.21.15.53:0/4000469285 shutdown_connections 2023-10-21T16:03:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 -- 172.21.15.53:0/4000469285 >> 172.21.15.53:0/4000469285 conn(0x7f757c06a5d0 msgr2=0x7f757c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 -- 172.21.15.53:0/4000469285 shutdown_connections 2023-10-21T16:03:43.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.033+0000 7f75834e7640 1 -- 172.21.15.53:0/4000469285 wait complete. 2023-10-21T16:03:43.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.035+0000 7f75834e7640 1 Processor -- start 2023-10-21T16:03:43.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.035+0000 7f75834e7640 1 -- start start 2023-10-21T16:03:43.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f757c0cc950 0x7f757c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c0ce980 0x7f757c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c15ca60 con 0x7f75680039a0 2023-10-21T16:03:43.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0cd310 con 0x7f757c0cc950 2023-10-21T16:03:43.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f75834e7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0cf640 con 0x7f757c0ce980 2023-10-21T16:03:43.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.036+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52646/0 (socket says 172.21.15.53:52646) 2023-10-21T16:03:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 -- 172.21.15.53:0/409346694 learned_addr learned my addr 172.21.15.53:0/409346694 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7580a82640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f757c0cc950 0x7f757c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581a84640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c0ce980 0x7f757c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 -- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c0ce980 msgr2=0x7f757c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c0ce980 0x7f757c0cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 -- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f757c0cc950 msgr2=0x7f757c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f757c0cc950 0x7f757c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 -- 172.21.15.53:0/409346694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c1490a0 con 0x7f75680039a0 2023-10-21T16:03:43.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.037+0000 7f7581283640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f757802c9f0 tx=0x7f7578001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.038+0000 7f7571ffb640 1 -- 172.21.15.53:0/409346694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578004e30 con 0x7f75680039a0 2023-10-21T16:03:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.038+0000 7f7571ffb640 1 -- 172.21.15.53:0/409346694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578035540 con 0x7f75680039a0 2023-10-21T16:03:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.038+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c15c100 con 0x7f75680039a0 2023-10-21T16:03:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.038+0000 7f7571ffb640 1 -- 172.21.15.53:0/409346694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780358b0 con 0x7f75680039a0 2023-10-21T16:03:43.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.038+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c15c3a0 con 0x7f75680039a0 2023-10-21T16:03:43.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.041+0000 7f7571ffb640 1 -- 172.21.15.53:0/409346694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7578031030 con 0x7f75680039a0 2023-10-21T16:03:43.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.042+0000 7f7571ffb640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 0x7f756006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.042+0000 7f7571ffb640 1 -- 172.21.15.53:0/409346694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f75780abe30 con 0x7f75680039a0 2023-10-21T16:03:43.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.042+0000 7f7586084740 1 Processor -- start 2023-10-21T16:03:43.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7580a82640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 0x7f756006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 -- start start 2023-10-21T16:03:43.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 0x55bae3ee8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3f18d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3e097a0 con 0x55bae3ee93c0 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3e5b740 con 0x55bae3f16360 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.043+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae3e58e30 con 0x55bae3f15f80 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52652/0 (socket says 172.21.15.53:52652) 2023-10-21T16:03:43.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 -- 172.21.15.53:0/4127268271 learned_addr learned my addr 172.21.15.53:0/4127268271 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7580a82640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 0x7f756006d330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f757c15bef0 tx=0x7f756c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581283640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 0x55bae3ee8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7580a82640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3f18d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 -- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 msgr2=0x55bae3ee8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 0x55bae3ee8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 -- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 msgr2=0x55bae3f18d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3f18d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581a84640 1 -- 172.21.15.53:0/4127268271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae3e18910 con 0x55bae3ee93c0 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7581283640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 0x55bae3ee8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.044+0000 7f7580a82640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3f18d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:43.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.045+0000 7f7581a84640 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f757c0cb080 tx=0x7f7574001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd08d7934ae1ec4b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.045+0000 7f7566ffd640 1 -- 172.21.15.53:0/4127268271 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574005480 con 0x55bae3ee93c0 2023-10-21T16:03:43.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.045+0000 7f7566ffd640 1 -- 172.21.15.53:0/4127268271 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400d040 con 0x55bae3ee93c0 2023-10-21T16:03:43.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.045+0000 7f7566ffd640 1 -- 172.21.15.53:0/4127268271 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574003830 con 0x55bae3ee93c0 2023-10-21T16:03:43.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 -- 172.21.15.53:0/4127268271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 msgr2=0x55bae3f19250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f757c0cb080 tx=0x7f7574001720 comp rx=0 tx=0).stop 2023-10-21T16:03:43.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 -- 172.21.15.53:0/4127268271 shutdown_connections 2023-10-21T16:03:43.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3ee93c0 0x55bae3f19250 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3f18d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 --2- 172.21.15.53:0/4127268271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f15f80 0x55bae3ee8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 -- 172.21.15.53:0/4127268271 >> 172.21.15.53:0/4127268271 conn(0x55bae3eeb190 msgr2=0x55bae3f33d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 -- 172.21.15.53:0/4127268271 shutdown_connections 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.046+0000 7f7586084740 1 -- 172.21.15.53:0/4127268271 wait complete. 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.047+0000 7f7586084740 1 -- start start 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.047+0000 7f7586084740 1 Processor -- start 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f16360 0x55bae3f479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f2e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3e54720 con 0x55bae3f15f80 2023-10-21T16:03:43.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3df2970 con 0x55bae3f31410 2023-10-21T16:03:43.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae3df2610 con 0x55bae3f16360 2023-10-21T16:03:43.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7586084740 10 client.0 _ll_register_callbacks cb 0x55bae3e149d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:03:43.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52660/0 (socket says 172.21.15.53:52660) 2023-10-21T16:03:43.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.048+0000 7f7581283640 1 -- 172.21.15.53:0/821700628 learned_addr learned my addr 172.21.15.53:0/821700628 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 -- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f16360 msgr2=0x55bae3f479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f16360 0x55bae3f479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581a84640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f2e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 -- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 msgr2=0x55bae3f2e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f2e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 -- 172.21.15.53:0/821700628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c0b2620 con 0x55bae3f15f80 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581a84640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f2e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7581283640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f75780aa130 tx=0x7f757807b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825a952488831b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f755bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:03:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578077140 con 0x55bae3f15f80 2023-10-21T16:03:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f75780772a0 con 0x55bae3f15f80 2023-10-21T16:03:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bae3e7b160 con 0x55bae3f15f80 2023-10-21T16:03:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.049+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7578092180 con 0x55bae3f15f80 2023-10-21T16:03:43.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 10 client.5478 fetch_fsmap learned FSMap version 18 2023-10-21T16:03:43.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bae3e52da0 con 0x55bae3f15f80 2023-10-21T16:03:43.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578096070 con 0x55bae3f15f80 2023-10-21T16:03:43.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f757804f020 con 0x55bae3f15f80 2023-10-21T16:03:43.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 10 client.5478 fetch_fsmap finished waiting for FSMap version 18 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 10 client.5478 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 20 client.5478 populate_metadata read hostname 'smithi053' 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f2ee80 0x55bae3f0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 4 client.5478 mds_command: new command op to 34679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 --> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bae3e26cc0 con 0x55bae3f2ee80 2023-10-21T16:03:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.050+0000 7f7580a82640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f2ee80 0x55bae3f0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.051+0000 7f7580a82640 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f2ee80 0x55bae3f0a1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62dd98ed79516acf server_cookie=8a7c172c07bc9050 in_seq=0 out_seq=0 2023-10-21T16:03:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.051+0000 7f755bfff640 10 client.5478 ms_handle_connect on v2:172.21.15.53:6832/1236966553 2023-10-21T16:03:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f755bfff640 1 -- 172.21.15.53:0/821700628 <== mds.0 v2:172.21.15.53:6832/1236966553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bae3e26cc0 con 0x55bae3f2ee80 2023-10-21T16:03:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f755bfff640 10 client.5478 handle_command_reply: tid=0 2023-10-21T16:03:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 client.5478 shutdown 2023-10-21T16:03:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 msgr2=0x55bae3f1c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f75780aa130 tx=0x7f757807b270 comp rx=0 tx=0).stop 2023-10-21T16:03:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 shutdown_connections 2023-10-21T16:03:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f2ee80 0x55bae3f0a1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f2e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f16360 0x55bae3f479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 --2- 172.21.15.53:0/821700628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f15f80 0x55bae3f1c190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 >> 172.21.15.53:0/821700628 conn(0x55bae3ee93c0 msgr2=0x55bae3eece30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f755bfff640 0 client.5478 ms_handle_reset on v2:172.21.15.53:6832/1236966553 2023-10-21T16:03:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 shutdown_connections 2023-10-21T16:03:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 1 -- 172.21.15.53:0/821700628 wait complete. 2023-10-21T16:03:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.052+0000 7f7586084740 20 client.5478 trim_cache size 0 max 16384 2023-10-21T16:03:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.070+0000 7f7586084740 1 Processor -- start 2023-10-21T16:03:43.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.070+0000 7f7586084740 1 -- start start 2023-10-21T16:03:43.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3eeca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f46940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3bb8f20 con 0x55bae3f31410 2023-10-21T16:03:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3eee900 con 0x55bae3f16360 2023-10-21T16:03:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae3eed530 con 0x55bae3f36340 2023-10-21T16:03:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52676/0 (socket says 172.21.15.53:52676) 2023-10-21T16:03:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 -- 172.21.15.53:0/1641749984 learned_addr learned my addr 172.21.15.53:0/1641749984 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 -- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 msgr2=0x55bae3f46940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f46940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 -- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 msgr2=0x55bae3eeca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3eeca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.071+0000 7f7580a82640 1 -- 172.21.15.53:0/1641749984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae3df1b80 con 0x55bae3f31410 2023-10-21T16:03:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.072+0000 7f7580a82640 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f757c06e410 tx=0x7f756c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca1a050fbc33007 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.072+0000 7f75677fe640 1 -- 172.21.15.53:0/1641749984 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0042f0 con 0x55bae3f31410 2023-10-21T16:03:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.072+0000 7f75677fe640 1 -- 172.21.15.53:0/1641749984 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c02c070 con 0x55bae3f31410 2023-10-21T16:03:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.072+0000 7f75677fe640 1 -- 172.21.15.53:0/1641749984 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c028050 con 0x55bae3f31410 2023-10-21T16:03:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- 172.21.15.53:0/1641749984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 msgr2=0x55bae3f46400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f757c06e410 tx=0x7f756c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- 172.21.15.53:0/1641749984 shutdown_connections 2023-10-21T16:03:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f46940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f31410 0x55bae3f46400 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 --2- 172.21.15.53:0/1641749984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f16360 0x55bae3eeca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- 172.21.15.53:0/1641749984 >> 172.21.15.53:0/1641749984 conn(0x55bae3ee93c0 msgr2=0x55bae3f0a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- 172.21.15.53:0/1641749984 shutdown_connections 2023-10-21T16:03:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- 172.21.15.53:0/1641749984 wait complete. 2023-10-21T16:03:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.073+0000 7f7586084740 1 -- start start 2023-10-21T16:03:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.074+0000 7f7586084740 1 Processor -- start 2023-10-21T16:03:43.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f9cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3f9da10 con 0x55bae3f16360 2023-10-21T16:03:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae3f1cf10 con 0x55bae3f31410 2023-10-21T16:03:43.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae3eece50 con 0x55bae3f36340 2023-10-21T16:03:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7586084740 10 client.0 _ll_register_callbacks cb 0x7f7578076d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:03:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7581283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52688/0 (socket says 172.21.15.53:52688) 2023-10-21T16:03:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.075+0000 7f7581283640 1 -- 172.21.15.53:0/145937462 learned_addr learned my addr 172.21.15.53:0/145937462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 -- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 msgr2=0x55bae3f9cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f9cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 -- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 msgr2=0x55bae3f093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 -- 172.21.15.53:0/145937462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c01d040 con 0x55bae3f16360 2023-10-21T16:03:43.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7581283640 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f757807a9c0 tx=0x7f7578004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba64821dcfa5c30e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f755bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:03:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757807a800 con 0x55bae3f16360 2023-10-21T16:03:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f757809aea0 con 0x55bae3f16360 2023-10-21T16:03:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f757400d040 con 0x55bae3f16360 2023-10-21T16:03:43.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f757800a3b0 con 0x55bae3f16360 2023-10-21T16:03:43.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7586084740 10 client.5484 fetch_fsmap learned FSMap version 18 2023-10-21T16:03:43.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.076+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bae3e17170 con 0x55bae3f16360 2023-10-21T16:03:43.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400d040 con 0x55bae3f16360 2023-10-21T16:03:43.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75780a0020 con 0x55bae3f16360 2023-10-21T16:03:43.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 10 client.5484 fetch_fsmap finished waiting for FSMap version 18 2023-10-21T16:03:43.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 10 client.5484 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:03:43.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 20 client.5484 populate_metadata read hostname 'smithi053' 2023-10-21T16:03:43.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f9b9d0 0x55bae3f9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 4 client.5484 mds_command: new command op to 34679 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:03:43.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 --> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bae3e4d700 con 0x55bae3f9b9d0 2023-10-21T16:03:43.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.077+0000 7f7580a82640 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f9b9d0 0x55bae3f9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7580a82640 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f9b9d0 0x55bae3f9bdb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0bcec03a37ac36 server_cookie=ce5bdb0c27a7d9a1 in_seq=0 out_seq=0 2023-10-21T16:03:43.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f755bfff640 10 client.5484 ms_handle_connect on v2:172.21.15.53:6832/1236966553 2023-10-21T16:03:43.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f755bfff640 1 -- 172.21.15.53:0/145937462 <== mds.0 v2:172.21.15.53:6832/1236966553 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55bae3e4d700 con 0x55bae3f9b9d0 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f755bfff640 10 client.5484 handle_command_reply: tid=0 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 client.5484 shutdown 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 msgr2=0x55bae3f356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f757807a9c0 tx=0x7f7578004f90 comp rx=0 tx=0).stop 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 shutdown_connections 2023-10-21T16:03:43.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:6832/1236966553,v1:172.21.15.53:6833/1236966553] conn(0x55bae3f9b9d0 0x55bae3f9bdb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55bae3f36340 0x55bae3f9cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55bae3f31410 0x55bae3f093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 --2- 172.21.15.53:0/145937462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bae3f16360 0x55bae3f356c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f755bfff640 0 client.5484 ms_handle_reset on v2:172.21.15.53:6832/1236966553 2023-10-21T16:03:43.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.078+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 >> 172.21.15.53:0/145937462 conn(0x55bae3ee93c0 msgr2=0x55bae3f46f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 shutdown_connections 2023-10-21T16:03:43.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f7586084740 1 -- 172.21.15.53:0/145937462 wait complete. 2023-10-21T16:03:43.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f7586084740 20 client.5484 trim_cache size 0 max 16384 2023-10-21T16:03:43.094 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:03:43.094 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:03:43.094 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:03:43.095 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 34679, 2023-10-21T16:03:43.095 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:03:43.095 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:03:43.095 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:03:43.095 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.833223577, 2023-10-21T16:03:43.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 18, 2023-10-21T16:03:43.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 41, 2023-10-21T16:03:43.096 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 40, 2023-10-21T16:03:43.096 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 15.658182793 2023-10-21T16:03:43.096 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:03:43.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 msgr2=0x7f756006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 0x7f756006d330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f757c15bef0 tx=0x7f756c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:43.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 msgr2=0x7f757c148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f757802c9f0 tx=0x7f7578001cb0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 shutdown_connections 2023-10-21T16:03:43.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f757c0ce980 0x7f757c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f757c0cc950 0x7f757c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f756006ae70 0x7f756006d330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.079+0000 7f75834e7640 1 --2- 172.21.15.53:0/409346694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680039a0 0x7f757c148500 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.080+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 >> 172.21.15.53:0/409346694 conn(0x7f757c06a5d0 msgr2=0x7f757c0c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.080+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 shutdown_connections 2023-10-21T16:03:43.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.080+0000 7f75834e7640 1 -- 172.21.15.53:0/409346694 wait complete. 2023-10-21T16:03:43.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:03:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.315+0000 7f8493f89640 1 Processor -- start 2023-10-21T16:03:43.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.315+0000 7f8493f89640 1 -- start start 2023-10-21T16:03:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.316+0000 7f8493f89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f848c157000 0x7f848c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.316+0000 7f8493f89640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0adfc0 con 0x7f848c1597d0 2023-10-21T16:03:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.316+0000 7f8493f89640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0ccce0 con 0x7f848c0cb190 2023-10-21T16:03:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.316+0000 7f8493f89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c159cb0 con 0x7f848c157000 2023-10-21T16:03:43.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491d25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f848c157000 0x7f848c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491524640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f848c1597d0 0x7f848c06ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55712/0 (socket says 172.21.15.53:55712) 2023-10-21T16:03:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491524640 1 -- 172.21.15.53:0/2258355485 learned_addr learned my addr 172.21.15.53:0/2258355485 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:43.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491d25640 1 -- 172.21.15.53:0/2258355485 >> v1:172.21.15.124:6789/0 conn(0x7f848c0cb190 legacy=0x7f848c15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491d25640 1 -- 172.21.15.53:0/2258355485 >> v1:172.21.15.53:6789/0 conn(0x7f848c1597d0 legacy=0x7f848c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8491d25640 1 -- 172.21.15.53:0/2258355485 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c159f90 con 0x7f848c157000 2023-10-21T16:03:43.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.317+0000 7f8490d23640 1 -- 172.21.15.53:0/2258355485 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134790458 0 0) 0x7f848c0adfc0 con 0x7f848c1597d0 2023-10-21T16:03:43.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.318+0000 7f8491d25640 1 --2- 172.21.15.53:0/2258355485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c157000 0x7f848c0cc7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f848800ba40 tx=0x7f848802e090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca09f2ef7e47632 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8490d23640 1 -- 172.21.15.53:0/2258355485 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848802ee40 con 0x7f848c157000 2023-10-21T16:03:43.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8490d23640 1 -- 172.21.15.53:0/2258355485 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488036440 con 0x7f848c157000 2023-10-21T16:03:43.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8490d23640 1 -- 172.21.15.53:0/2258355485 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84880367d0 con 0x7f848c157000 2023-10-21T16:03:43.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 -- 172.21.15.53:0/2258355485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c157000 msgr2=0x7f848c0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 --2- 172.21.15.53:0/2258355485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c157000 0x7f848c0cc7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f848800ba40 tx=0x7f848802e090 comp rx=0 tx=0).stop 2023-10-21T16:03:43.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 -- 172.21.15.53:0/2258355485 shutdown_connections 2023-10-21T16:03:43.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 --2- 172.21.15.53:0/2258355485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c157000 0x7f848c0cc7a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 -- 172.21.15.53:0/2258355485 >> 172.21.15.53:0/2258355485 conn(0x7f848c06a380 msgr2=0x7f848c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.319+0000 7f8493f89640 1 -- 172.21.15.53:0/2258355485 shutdown_connections 2023-10-21T16:03:43.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.320+0000 7f8493f89640 1 -- 172.21.15.53:0/2258355485 wait complete. 2023-10-21T16:03:43.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.321+0000 7f8493f89640 1 Processor -- start 2023-10-21T16:03:43.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.321+0000 7f8493f89640 1 -- start start 2023-10-21T16:03:43.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.322+0000 7f8493f89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.322+0000 7f8493f89640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f848c1597d0 0x7f848c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.322+0000 7f8493f89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f848c0b3330 0x7f848c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8493f89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c155cb0 con 0x7f848c0cb190 2023-10-21T16:03:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8493f89640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b37e0 con 0x7f848c1597d0 2023-10-21T16:03:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8493f89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b42a0 con 0x7f848c0b3330 2023-10-21T16:03:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52704/0 (socket says 172.21.15.53:52704) 2023-10-21T16:03:43.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 -- 172.21.15.53:0/731676887 learned_addr learned my addr 172.21.15.53:0/731676887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 -- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f848c0b3330 msgr2=0x7f848c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f848c0b3330 0x7f848c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 -- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f848c1597d0 msgr2=0x7f848c0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f848c1597d0 0x7f848c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.323+0000 7f8491d25640 1 -- 172.21.15.53:0/731676887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848800b6a0 con 0x7f848c0cb190 2023-10-21T16:03:43.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f8491d25640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f848800bb60 tx=0x7f84880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488034a10 con 0x7f848c0cb190 2023-10-21T16:03:43.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488034b90 con 0x7f848c0cb190 2023-10-21T16:03:43.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488040410 con 0x7f848c0cb190 2023-10-21T16:03:43.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c0b3a00 con 0x7f848c0cb190 2023-10-21T16:03:43.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.324+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0b3fc0 con 0x7f848c0cb190 2023-10-21T16:03:43.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.325+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c067490 con 0x7f848c0cb190 2023-10-21T16:03:43.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.326+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f848803d050 con 0x7f848c0cb190 2023-10-21T16:03:43.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.327+0000 7f84827fc640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 0x7f847006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:43.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.327+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84880ae620 con 0x7f848c0cb190 2023-10-21T16:03:43.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.328+0000 7f8491524640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 0x7f847006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:43.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.329+0000 7f8491524640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 0x7f847006d360 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:43.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.332+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8488079560 con 0x7f848c0cb190 2023-10-21T16:03:43.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.570+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f848c06c6d0 con 0x7f848c0cb190 2023-10-21T16:03:43.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.571+0000 7f84827fc640 1 -- 172.21.15.53:0/731676887 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f848807cb80 con 0x7f848c0cb190 2023-10-21T16:03:43.573 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:03:43.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.573+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 msgr2=0x7f847006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.573+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 0x7f847006d360 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f847c007fd0 tx=0x7f847c007960 comp rx=0 tx=0).stop 2023-10-21T16:03:43.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 msgr2=0x7f848c0ce9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:43.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f848800bb60 tx=0x7f84880046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 shutdown_connections 2023-10-21T16:03:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f848c0b3330 0x7f848c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f848c1597d0 0x7f848c0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f847006aea0 0x7f847006d360 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 --2- 172.21.15.53:0/731676887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0cb190 0x7f848c0ce9f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 >> 172.21.15.53:0/731676887 conn(0x7f848c06a380 msgr2=0x7f848c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 shutdown_connections 2023-10-21T16:03:43.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:43.574+0000 7f8493f89640 1 -- 172.21.15.53:0/731676887 wait complete. 2023-10-21T16:03:43.595 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:03:43.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:03:43.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:03:43.596 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:03:43.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:03:43.597 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:03:43.597 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:03:43.597 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:03:43.598 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:03:43.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:43.599 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:03:43.655 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:03:43.656 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:03:43.656 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:03:43.656 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.656 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:03:43.657 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.657 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:03:43.657 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:03:43.657 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:03:43.657 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:03:43.658 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:03:43.659 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-10-21T16:03:43.659 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:03:43.659 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.659 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:03:43.659 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:03:43.660 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:03:43.660 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:03:43.660 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:03:43.660 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:03:43.660 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:03:43.661 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.662 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:03:43.662 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:03:43.662 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:03:43.662 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:03:43.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:43.664 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:03:43.761 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:03:43.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:43.766 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:03:43.826 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:03:43.827 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:03:43.827 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:03:43.827 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:03:43.828 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:03:43.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:43.829 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:03:43.830 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:03:43.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:03:43.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:03:43.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:03:43.832 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:03:44.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:44.045 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:03:44.099 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:03:44.099 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:03:44.099 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:03:44.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:44.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:03:44.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:03:44.199 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:03:44.256 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:03:44.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:03:44.257 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:03:44.310 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:03:44.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:44.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:03:44.472 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:03:44.473 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:03:44.511 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:03:44.511 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:03:44.511 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:03:44.511 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:03:44.512 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:03:44.514 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:03:44.514 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:03:44.514 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:03:44.514 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:03:44.515 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:03:44.519 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:03:44.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:44.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:03:44.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:44.609 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:03:44.678 INFO:teuthology.orchestra.run.smithi053.stdout:client34754 2023-10-21T16:03:44.679 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:03:44.679 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:03:44.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:03:44.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.940+0000 7f7af61b7640 1 Processor -- start 2023-10-21T16:03:44.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.940+0000 7f7af61b7640 1 -- start start 2023-10-21T16:03:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.941+0000 7f7af61b7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7af00c9cf0 0x7f7af006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.941+0000 7f7af61b7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7af015aa00 0x7f7af00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7af61b7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af015b310 con 0x7f7af006c3a0 2023-10-21T16:03:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7af61b7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af006e0a0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7af61b7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af006d290 con 0x7f7af015aa00 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7af00c9cf0 0x7f7af006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7af00c9cf0 0x7f7af006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45836/0 (socket says 172.21.15.53:45836) 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 -- 172.21.15.53:0/2261805258 learned_addr learned my addr 172.21.15.53:0/2261805258 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7aeffff640 1 --2- 172.21.15.53:0/2261805258 >> v2:172.21.15.124:3300/0 conn(0x7f7af015aa00 0x7f7af00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 -- 172.21.15.53:0/2261805258 >> v2:172.21.15.124:3300/0 conn(0x7f7af015aa00 msgr2=0x7f7af00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:44.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 --2- 172.21.15.53:0/2261805258 >> v2:172.21.15.124:3300/0 conn(0x7f7af015aa00 0x7f7af00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:44.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 -- 172.21.15.53:0/2261805258 >> v1:172.21.15.124:6789/0 conn(0x7f7af006c3a0 legacy=0x7f7af006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:44.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.942+0000 7f7ae7fff640 1 -- 172.21.15.53:0/2261805258 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af00c9860 con 0x7f7af00c9cf0 2023-10-21T16:03:44.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.944+0000 7f7ae7fff640 1 --2- 172.21.15.53:0/2261805258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af006db60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7af00c57a0 tx=0x7f7ad802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784debcea632da7c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:44.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.944+0000 7f7aeeffd640 1 -- 172.21.15.53:0/2261805258 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8031870 con 0x7f7af00c9cf0 2023-10-21T16:03:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.944+0000 7f7aeeffd640 1 -- 172.21.15.53:0/2261805258 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80319d0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.944+0000 7f7aeeffd640 1 -- 172.21.15.53:0/2261805258 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8031d60 con 0x7f7af00c9cf0 2023-10-21T16:03:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 -- 172.21.15.53:0/2261805258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 msgr2=0x7f7af006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 --2- 172.21.15.53:0/2261805258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af006db60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7af00c57a0 tx=0x7f7ad802b910 comp rx=0 tx=0).stop 2023-10-21T16:03:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 -- 172.21.15.53:0/2261805258 shutdown_connections 2023-10-21T16:03:44.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 --2- 172.21.15.53:0/2261805258 >> v2:172.21.15.124:3300/0 conn(0x7f7af015aa00 0x7f7af00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:44.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 --2- 172.21.15.53:0/2261805258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af006db60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:44.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 -- 172.21.15.53:0/2261805258 >> 172.21.15.53:0/2261805258 conn(0x7f7af006a5d0 msgr2=0x7f7af00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:44.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 -- 172.21.15.53:0/2261805258 shutdown_connections 2023-10-21T16:03:44.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.945+0000 7f7af61b7640 1 -- 172.21.15.53:0/2261805258 wait complete. 2023-10-21T16:03:44.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.947+0000 7f7af61b7640 1 Processor -- start 2023-10-21T16:03:44.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.947+0000 7f7af61b7640 1 -- start start 2023-10-21T16:03:44.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7af006c3a0 0x7f7af00c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7af015aa00 0x7f7af014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c6ac0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af014b0a0 con 0x7f7af006c3a0 2023-10-21T16:03:44.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.948+0000 7f7af61b7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c8550 con 0x7f7af015aa00 2023-10-21T16:03:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45848/0 (socket says 172.21.15.53:45848) 2023-10-21T16:03:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 -- 172.21.15.53:0/862969982 learned_addr learned my addr 172.21.15.53:0/862969982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7aef7fe640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7af006c3a0 0x7f7af00c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7aeffff640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7af015aa00 0x7f7af014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 -- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7af015aa00 msgr2=0x7f7af014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:44.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7af015aa00 0x7f7af014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:44.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 -- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7af006c3a0 msgr2=0x7f7af00c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:44.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7af006c3a0 0x7f7af00c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:44.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.949+0000 7f7ae7fff640 1 -- 172.21.15.53:0/862969982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad80094d0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7ae7fff640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000c00 tx=0x7f7ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:44.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad803abe0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad803ad40 con 0x7f7af00c9cf0 2023-10-21T16:03:44.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8044610 con 0x7f7af00c9cf0 2023-10-21T16:03:44.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af00c6220 con 0x7f7af00c9cf0 2023-10-21T16:03:44.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.950+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af00c67e0 con 0x7f7af00c9cf0 2023-10-21T16:03:44.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.951+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af004e160 con 0x7f7af00c9cf0 2023-10-21T16:03:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.952+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ad804d450 con 0x7f7af00c9cf0 2023-10-21T16:03:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.954+0000 7f7aecff9640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 0x7f7ad006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.954+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ad802d040 con 0x7f7af00c9cf0 2023-10-21T16:03:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.955+0000 7f7aef7fe640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 0x7f7ad006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:44.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.956+0000 7f7aef7fe640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 0x7f7ad006d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0004ea0 tx=0x7f7ae0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:44.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:44.958+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ad8076b20 con 0x7f7af00c9cf0 2023-10-21T16:03:45.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.177+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7af00c8f30 con 0x7f7ad006aea0 2023-10-21T16:03:45.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.181+0000 7f7aecff9640 1 -- 172.21.15.53:0/862969982 <== mgr.4108 v2:172.21.15.124:6832/3409974301 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7ae0005420 con 0x7f7ad006aea0 2023-10-21T16:03:45.183 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:03:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:03:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:03:45.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:03:45.184 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:03:45.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.184+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 msgr2=0x7f7ad006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:45.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.184+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 0x7f7ad006d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0004ea0 tx=0x7f7ae0009040 comp rx=0 tx=0).stop 2023-10-21T16:03:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.184+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 msgr2=0x7f7af0159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.184+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8000c00 tx=0x7f7ad80046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 shutdown_connections 2023-10-21T16:03:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7af015aa00 0x7f7af014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f7ad006aea0 0x7f7ad006d360 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7af00c9cf0 0x7f7af0159d20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 --2- 172.21.15.53:0/862969982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7af006c3a0 0x7f7af00c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 >> 172.21.15.53:0/862969982 conn(0x7f7af006a5d0 msgr2=0x7f7af00ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 shutdown_connections 2023-10-21T16:03:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.185+0000 7f7af61b7640 1 -- 172.21.15.53:0/862969982 wait complete. 2023-10-21T16:03:45.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:03:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.430+0000 7f3ac9890640 1 Processor -- start 2023-10-21T16:03:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.430+0000 7f3ac9890640 1 -- start start 2023-10-21T16:03:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac415a820 0x7f3ac40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3ac40ca680 0x7f3ac40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ac415ac00 0x7f3ac40c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40cab60 con 0x7f3ac415ac00 2023-10-21T16:03:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40cb960 con 0x7f3ac40ca680 2023-10-21T16:03:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac9890640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40cbdf0 con 0x7f3ac415a820 2023-10-21T16:03:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.432+0000 7f3ac908f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ac415ac00 0x7f3ac40c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ac415ac00 0x7f3ac40c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45862/0 (socket says 172.21.15.53:45862) 2023-10-21T16:03:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 -- 172.21.15.53:0/3933722878 learned_addr learned my addr 172.21.15.53:0/3933722878 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3abbfff640 1 --2- 172.21.15.53:0/3933722878 >> v2:172.21.15.124:3300/0 conn(0x7f3ac40ca680 0x7f3ac40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac888e640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac415a820 0x7f3ac40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 -- 172.21.15.53:0/3933722878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac415a820 msgr2=0x7f3ac40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac415a820 0x7f3ac40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 -- 172.21.15.53:0/3933722878 >> v2:172.21.15.124:3300/0 conn(0x7f3ac40ca680 msgr2=0x7f3ac40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 --2- 172.21.15.53:0/3933722878 >> v2:172.21.15.124:3300/0 conn(0x7f3ac40ca680 0x7f3ac40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.433+0000 7f3ac908f640 1 -- 172.21.15.53:0/3933722878 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac40cb030 con 0x7f3ac415ac00 2023-10-21T16:03:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.435+0000 7f3ac908f640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40c9100 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3ac406dc80 tx=0x7f3abc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abbc864d1d4ffdd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.435+0000 7f3abb7fe640 1 -- 172.21.15.53:0/3933722878 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc02c9e0 con 0x7f3ac415ac00 2023-10-21T16:03:45.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.435+0000 7f3abb7fe640 1 -- 172.21.15.53:0/3933722878 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc02cb40 con 0x7f3ac415ac00 2023-10-21T16:03:45.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.435+0000 7f3abb7fe640 1 -- 172.21.15.53:0/3933722878 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0343f0 con 0x7f3ac415ac00 2023-10-21T16:03:45.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 -- 172.21.15.53:0/3933722878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 msgr2=0x7f3ac40c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:45.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40c9100 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3ac406dc80 tx=0x7f3abc02be40 comp rx=0 tx=0).stop 2023-10-21T16:03:45.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 -- 172.21.15.53:0/3933722878 shutdown_connections 2023-10-21T16:03:45.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40c9100 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 --2- 172.21.15.53:0/3933722878 >> v2:172.21.15.124:3300/0 conn(0x7f3ac40ca680 0x7f3ac40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 --2- 172.21.15.53:0/3933722878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac415a820 0x7f3ac40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 -- 172.21.15.53:0/3933722878 >> 172.21.15.53:0/3933722878 conn(0x7f3ac406c050 msgr2=0x7f3ac406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:45.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 -- 172.21.15.53:0/3933722878 shutdown_connections 2023-10-21T16:03:45.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.436+0000 7f3ac9890640 1 -- 172.21.15.53:0/3933722878 wait complete. 2023-10-21T16:03:45.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.438+0000 7f3ac9890640 1 Processor -- start 2023-10-21T16:03:45.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.438+0000 7f3ac9890640 1 -- start start 2023-10-21T16:03:45.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac40ca680 0x7f3ac40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac415a820 0x7f3ac40ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40aef70 con 0x7f3ac415ac00 2023-10-21T16:03:45.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40af250 con 0x7f3ac415a820 2023-10-21T16:03:45.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac9890640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40af530 con 0x7f3ac40ca680 2023-10-21T16:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac908f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac908f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45868/0 (socket says 172.21.15.53:45868) 2023-10-21T16:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac908f640 1 -- 172.21.15.53:0/4150724646 learned_addr learned my addr 172.21.15.53:0/4150724646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3abbfff640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac415a820 0x7f3ac40ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.439+0000 7f3ac888e640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac40ca680 0x7f3ac40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 -- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac40ca680 msgr2=0x7f3ac40ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac40ca680 0x7f3ac40ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 -- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac415a820 msgr2=0x7f3ac40ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:45.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac415a820 0x7f3ac40ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 -- 172.21.15.53:0/4150724646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc009b30 con 0x7f3ac415ac00 2023-10-21T16:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac908f640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3abc036040 tx=0x7f3abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc040670 con 0x7f3ac415ac00 2023-10-21T16:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc0407d0 con 0x7f3ac415ac00 2023-10-21T16:03:45.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc040b80 con 0x7f3ac415ac00 2023-10-21T16:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.440+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac40af810 con 0x7f3ac415ac00 2023-10-21T16:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.441+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac40ad350 con 0x7f3ac415ac00 2023-10-21T16:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.442+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac40afd30 con 0x7f3ac415ac00 2023-10-21T16:03:45.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.443+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3abc040ce0 con 0x7f3ac415ac00 2023-10-21T16:03:45.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.444+0000 7f3ab97fa640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 0x7f3ab006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:45.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.444+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3abc0ac880 con 0x7f3ac415ac00 2023-10-21T16:03:45.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.444+0000 7f3ac888e640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 0x7f3ab006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:45.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.446+0000 7f3ac888e640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 0x7f3ab006d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006fd0 tx=0x7f3ac0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:45.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.450+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ac40afd30 con 0x7f3ac415ac00 2023-10-21T16:03:45.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.691+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f3ac40afd30 con 0x7f3ac415ac00 2023-10-21T16:03:46.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:45.999+0000 7f3ab97fa640 1 -- 172.21.15.53:0/4150724646 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3ac40afd30 con 0x7f3ac415ac00 2023-10-21T16:03:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 msgr2=0x7f3ab006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 0x7f3ab006d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006fd0 tx=0x7f3ac0008040 comp rx=0 tx=0).stop 2023-10-21T16:03:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 msgr2=0x7f3ac40cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3abc036040 tx=0x7f3abc004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:46.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 shutdown_connections 2023-10-21T16:03:46.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3ab006ae70 0x7f3ab006d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ac415ac00 0x7f3ac40cc7b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ac415a820 0x7f3ac40ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 --2- 172.21.15.53:0/4150724646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ac40ca680 0x7f3ac40ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.003+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 >> 172.21.15.53:0/4150724646 conn(0x7f3ac406c050 msgr2=0x7f3ac40c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:46.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.004+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 shutdown_connections 2023-10-21T16:03:46.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.004+0000 7f3ac9890640 1 -- 172.21.15.53:0/4150724646 wait complete. 2023-10-21T16:03:46.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:03:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.236+0000 7f36b8a64640 1 Processor -- start 2023-10-21T16:03:46.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.236+0000 7f36b8a64640 1 -- start start 2023-10-21T16:03:46.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.237+0000 7f36b8a64640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b41601b0 0x7f36b40cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.237+0000 7f36b8a64640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36b406a460 0x7f36b40ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.237+0000 7f36b8a64640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b40ce210 con 0x7f36b4071c30 2023-10-21T16:03:46.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.237+0000 7f36b8a64640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40cdee0 con 0x7f36b406a460 2023-10-21T16:03:46.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.237+0000 7f36b8a64640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4161530 con 0x7f36b41601b0 2023-10-21T16:03:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.238+0000 7f36b259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b41601b0 0x7f36b40cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.238+0000 7f36b2d9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36b406a460 0x7f36b40ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.238+0000 7f36b2d9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36b406a460 0x7f36b40ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:53562/0 (socket says 172.21.15.53:53562) 2023-10-21T16:03:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.238+0000 7f36b2d9d640 1 -- 172.21.15.53:0/1012528179 learned_addr learned my addr 172.21.15.53:0/1012528179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.239+0000 7f36b2d9d640 1 -- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b41601b0 msgr2=0x7f36b40cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.239+0000 7f36b2d9d640 1 --2- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b41601b0 0x7f36b40cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.239+0000 7f36b2d9d640 1 -- 172.21.15.53:0/1012528179 >> v1:172.21.15.124:6789/0 conn(0x7f36b4071c30 legacy=0x7f36b4069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.239+0000 7f36b2d9d640 1 -- 172.21.15.53:0/1012528179 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4161810 con 0x7f36b406a460 2023-10-21T16:03:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.240+0000 7f36b2d9d640 1 --2- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b406a460 0x7f36b40ca090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009870 tx=0x7f36ac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4520cd4fc4b6a50 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:46.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.241+0000 7f36b159a640 1 -- 172.21.15.53:0/1012528179 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac030840 con 0x7f36b406a460 2023-10-21T16:03:46.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.241+0000 7f36b159a640 1 -- 172.21.15.53:0/1012528179 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac0309a0 con 0x7f36b406a460 2023-10-21T16:03:46.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.241+0000 7f36b159a640 1 -- 172.21.15.53:0/1012528179 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac030d30 con 0x7f36b406a460 2023-10-21T16:03:46.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.241+0000 7f36b8a64640 1 -- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b406a460 msgr2=0x7f36b40ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.241+0000 7f36b8a64640 1 --2- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b406a460 0x7f36b40ca090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009870 tx=0x7f36ac02b980 comp rx=0 tx=0).stop 2023-10-21T16:03:46.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 -- 172.21.15.53:0/1012528179 shutdown_connections 2023-10-21T16:03:46.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 --2- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b406a460 0x7f36b40ca090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 --2- 172.21.15.53:0/1012528179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b41601b0 0x7f36b40cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 -- 172.21.15.53:0/1012528179 >> 172.21.15.53:0/1012528179 conn(0x7f36b406d430 msgr2=0x7f36b406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 -- 172.21.15.53:0/1012528179 shutdown_connections 2023-10-21T16:03:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.242+0000 7f36b8a64640 1 -- 172.21.15.53:0/1012528179 wait complete. 2023-10-21T16:03:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.243+0000 7f36b8a64640 1 Processor -- start 2023-10-21T16:03:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.244+0000 7f36b8a64640 1 -- start start 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.244+0000 7f36b8a64640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b406a460 0x7f36b40c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.244+0000 7f36b8a64640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 0x7f36b40c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.244+0000 7f36b8a64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.244+0000 7f36b8a64640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40ae220 con 0x7f36b41601b0 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b8a64640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40c5f60 con 0x7f36b4071c30 2023-10-21T16:03:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b8a64640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40ad610 con 0x7f36b406a460 2023-10-21T16:03:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45894/0 (socket says 172.21.15.53:45894) 2023-10-21T16:03:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b2d9d640 1 -- 172.21.15.53:0/2499683789 learned_addr learned my addr 172.21.15.53:0/2499683789 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b259c640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b406a460 0x7f36b40c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.245+0000 7f36b1d9b640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 0x7f36b40c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b406a460 msgr2=0x7f36b40c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b406a460 0x7f36b40c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 msgr2=0x7f36b40c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 0x7f36b40c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 -- 172.21.15.53:0/2499683789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0094d0 con 0x7f36b41601b0 2023-10-21T16:03:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b1d9b640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 0x7f36b40c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36b2d9d640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f36ac02b9c0 tx=0x7f36ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac039a90 con 0x7f36b41601b0 2023-10-21T16:03:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac039c10 con 0x7f36b41601b0 2023-10-21T16:03:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.246+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac043410 con 0x7f36b41601b0 2023-10-21T16:03:46.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.247+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b40ad8f0 con 0x7f36b41601b0 2023-10-21T16:03:46.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.247+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b40adeb0 con 0x7f36b41601b0 2023-10-21T16:03:46.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.248+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b404e160 con 0x7f36b41601b0 2023-10-21T16:03:46.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.250+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f36ac04c3b0 con 0x7f36b41601b0 2023-10-21T16:03:46.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.251+0000 7f36aaffd640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 0x7f369406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.251+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36ac0abc30 con 0x7f36b41601b0 2023-10-21T16:03:46.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.251+0000 7f36b259c640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 msgr2=0x7f369406d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:46.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.251+0000 7f36b259c640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 0x7f369406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:03:46.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.256+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36ac076c20 con 0x7f36b41601b0 2023-10-21T16:03:46.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.452+0000 7f36b259c640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 msgr2=0x7f369406d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:46.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.452+0000 7f36b259c640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 0x7f369406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:03:46.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.452+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f36b406a0d0 con 0x7f36b41601b0 2023-10-21T16:03:46.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.458+0000 7f36aaffd640 1 -- 172.21.15.53:0/2499683789 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f36ac0480d0 con 0x7f36b41601b0 2023-10-21T16:03:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 msgr2=0x7f369406d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:03:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 0x7f369406d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 msgr2=0x7f36b40c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f36ac02b9c0 tx=0x7f36ac004730 comp rx=0 tx=0).stop 2023-10-21T16:03:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 shutdown_connections 2023-10-21T16:03:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f369406aef0 0x7f369406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36b41601b0 0x7f36b40c5a20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36b4071c30 0x7f36b40c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 --2- 172.21.15.53:0/2499683789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36b406a460 0x7f36b40c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 >> 172.21.15.53:0/2499683789 conn(0x7f36b406d430 msgr2=0x7f36b40ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 shutdown_connections 2023-10-21T16:03:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.461+0000 7f36b8a64640 1 -- 172.21.15.53:0/2499683789 wait complete. 2023-10-21T16:03:46.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:03:46.536 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:46.535+0000 7f40f644c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:03:46.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:03:46.543 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:03:46.583 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:46.582+0000 7ff0c044f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:03:46.591 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:46.590+0000 7f40f644c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:03:46.645 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:46.643+0000 7ff0c044f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:03:46.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.691+0000 7f08c1c40640 1 Processor -- start 2023-10-21T16:03:46.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.691+0000 7f08c1c40640 1 -- start start 2023-10-21T16:03:46.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c1c40640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f08bc1599d0 0x7f08bc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c1c40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc0b3c50 0x7f08bc0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c1c40640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc0cb8a0 con 0x7f08bc0cb410 2023-10-21T16:03:46.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c1c40640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc159db0 con 0x7f08bc1599d0 2023-10-21T16:03:46.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c1c40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc071fe0 con 0x7f08bc0b3c50 2023-10-21T16:03:46.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c143f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f08bc0cb410 0x7f08bc0c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50994/0 (socket says 172.21.15.53:50994) 2023-10-21T16:03:46.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.693+0000 7f08c143f640 1 -- 172.21.15.53:0/217439166 learned_addr learned my addr 172.21.15.53:0/217439166 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:46.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/217439166 >> v2:172.21.15.124:3300/0 conn(0x7f08bc1599d0 0x7f08bc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08bbfff640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc0b3c50 0x7f08bc0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08bb7fe640 1 -- 172.21.15.53:0/217439166 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736259771 0 0) 0x7f08bc0cb8a0 con 0x7f08bc0cb410 2023-10-21T16:03:46.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08bb7fe640 1 -- 172.21.15.53:0/217439166 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ac003520 con 0x7f08bc0cb410 2023-10-21T16:03:46.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08c0c3e640 1 -- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc0b3c50 msgr2=0x7f08bc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:46.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc0b3c50 0x7f08bc0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08c0c3e640 1 -- 172.21.15.53:0/217439166 >> v1:172.21.15.53:6789/0 conn(0x7f08bc0cb410 legacy=0x7f08bc0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.694+0000 7f08c0c3e640 1 -- 172.21.15.53:0/217439166 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc0b44c0 con 0x7f08bc1599d0 2023-10-21T16:03:46.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.695+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc1599d0 0x7f08bc0cdcd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08b00097a0 tx=0x7f08b002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7a6eb94e0d4aef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:46.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.696+0000 7f08bb7fe640 1 -- 172.21.15.53:0/217439166 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b002d040 con 0x7f08bc1599d0 2023-10-21T16:03:46.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.696+0000 7f08bb7fe640 1 -- 172.21.15.53:0/217439166 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0031810 con 0x7f08bc1599d0 2023-10-21T16:03:46.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.696+0000 7f08bb7fe640 1 -- 172.21.15.53:0/217439166 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0031ba0 con 0x7f08bc1599d0 2023-10-21T16:03:46.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 -- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc1599d0 msgr2=0x7f08bc0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc1599d0 0x7f08bc0cdcd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08b00097a0 tx=0x7f08b002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 -- 172.21.15.53:0/217439166 shutdown_connections 2023-10-21T16:03:46.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc0b3c50 0x7f08bc0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 --2- 172.21.15.53:0/217439166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc1599d0 0x7f08bc0cdcd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 -- 172.21.15.53:0/217439166 >> 172.21.15.53:0/217439166 conn(0x7f08bc06c050 msgr2=0x7f08bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:46.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 -- 172.21.15.53:0/217439166 shutdown_connections 2023-10-21T16:03:46.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.697+0000 7f08c1c40640 1 -- 172.21.15.53:0/217439166 wait complete. 2023-10-21T16:03:46.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.699+0000 7f08c1c40640 1 Processor -- start 2023-10-21T16:03:46.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.699+0000 7f08c1c40640 1 -- start start 2023-10-21T16:03:46.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc0cb410 0x7f08bc1f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc1599d0 0x7f08bc14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1f1cb0 con 0x7f08bc0b3c50 2023-10-21T16:03:46.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc14b5e0 con 0x7f08bc0cb410 2023-10-21T16:03:46.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.700+0000 7f08c1c40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc15b580 con 0x7f08bc1599d0 2023-10-21T16:03:46.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45902/0 (socket says 172.21.15.53:45902) 2023-10-21T16:03:46.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 -- 172.21.15.53:0/238778476 learned_addr learned my addr 172.21.15.53:0/238778476 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:46.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08bbfff640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc0cb410 0x7f08bc1f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c143f640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc1599d0 0x7f08bc14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc1599d0 msgr2=0x7f08bc14b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc1599d0 0x7f08bc14b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc0cb410 msgr2=0x7f08bc1f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc0cb410 0x7f08bc1f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.701+0000 7f08c0c3e640 1 -- 172.21.15.53:0/238778476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0009400 con 0x7f08bc0b3c50 2023-10-21T16:03:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08c0c3e640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b003ad60 con 0x7f08bc0b3c50 2023-10-21T16:03:46.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b00443f0 con 0x7f08bc0b3c50 2023-10-21T16:03:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00445a0 con 0x7f08bc0b3c50 2023-10-21T16:03:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc15b860 con 0x7f08bc0b3c50 2023-10-21T16:03:46.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.702+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc15be20 con 0x7f08bc0b3c50 2023-10-21T16:03:46.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.703+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc074150 con 0x7f08bc0b3c50 2023-10-21T16:03:46.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.704+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f08b004d450 con 0x7f08bc0b3c50 2023-10-21T16:03:46.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.705+0000 7f08b97fa640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 0x7f08a006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.705+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08b00ab840 con 0x7f08bc0b3c50 2023-10-21T16:03:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.705+0000 7f08bbfff640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 msgr2=0x7f08a006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.706+0000 7f08bbfff640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 0x7f08a006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:03:46.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.710+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08b0076780 con 0x7f08bc0b3c50 2023-10-21T16:03:46.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.901+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f08bc158e80 con 0x7f08bc0b3c50 2023-10-21T16:03:46.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.906+0000 7f08b97fa640 1 -- 172.21.15.53:0/238778476 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f08b0079da0 con 0x7f08bc0b3c50 2023-10-21T16:03:46.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.907+0000 7f08bbfff640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 msgr2=0x7f08a006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:46.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.907+0000 7f08bbfff640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 0x7f08a006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:03:46.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 msgr2=0x7f08a006d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:03:46.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 0x7f08a006d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 msgr2=0x7f08bc15afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:46.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f08b0000c00 tx=0x7f08b00046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 shutdown_connections 2023-10-21T16:03:46.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08bc1599d0 0x7f08bc14b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08bc0cb410 0x7f08bc1f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f08a006aec0 0x7f08a006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 --2- 172.21.15.53:0/238778476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08bc0b3c50 0x7f08bc15afa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:46.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 >> 172.21.15.53:0/238778476 conn(0x7f08bc06c050 msgr2=0x7f08bc0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:46.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.909+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 shutdown_connections 2023-10-21T16:03:46.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:46.910+0000 7f08c1c40640 1 -- 172.21.15.53:0/238778476 wait complete. 2023-10-21T16:03:46.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:46.929 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T16:03:46.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:46.986 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-21T16:03:47.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2023-10-21T16:03:47.048 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.047+0000 7f40f644c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:03:47.100 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.099+0000 7f40f644c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:03:47.132 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.130+0000 7ff0c044f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:03:47.150 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.149+0000 7f40f644c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:03:47.187 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.186+0000 7ff0c044f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:03:47.210 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.209+0000 7f40f644c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:03:47.241 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.239+0000 7ff0c044f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:03:47.258 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.257+0000 7f40f644c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:03:47.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.274+0000 7f3b20d0a640 1 Processor -- start 2023-10-21T16:03:47.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.275+0000 7f3b20d0a640 1 -- start start 2023-10-21T16:03:47.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.276+0000 7f3b20d0a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3b1c069fc0 0x7f3b1c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.276+0000 7f3b20d0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b1c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.276+0000 7f3b20d0a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c160d10 con 0x7f3b1c0ca210 2023-10-21T16:03:47.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.276+0000 7f3b20d0a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c072070 con 0x7f3b1c069fc0 2023-10-21T16:03:47.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.276+0000 7f3b20d0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c071580 con 0x7f3b1c06ec00 2023-10-21T16:03:47.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3b1c0ca210 0x7f3b1c160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51006/0 (socket says 172.21.15.53:51006) 2023-10-21T16:03:47.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1bfff640 1 -- 172.21.15.53:0/3493695538 learned_addr learned my addr 172.21.15.53:0/3493695538 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:47.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3493695538 >> v2:172.21.15.124:3300/0 conn(0x7f3b1c069fc0 0x7f3b1c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b1c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038570318 0 0) 0x7f3b1c160d10 con 0x7f3b1c0ca210 2023-10-21T16:03:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b08003520 con 0x7f3b1c0ca210 2023-10-21T16:03:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.277+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3014142103 0 0) 0x7f3b08003520 con 0x7f3b1c0ca210 2023-10-21T16:03:47.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 msgr2=0x7f3b1c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:47.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b1c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 >> v2:172.21.15.124:3300/0 conn(0x7f3b1c069fc0 msgr2=0x7f3b1c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:47.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 --2- 172.21.15.53:0/3493695538 >> v2:172.21.15.124:3300/0 conn(0x7f3b1c069fc0 0x7f3b1c071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c072330 con 0x7f3b1c0ca210 2023-10-21T16:03:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3b140019f0 con 0x7f3b1c0ca210 2023-10-21T16:03:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 >> v1:172.21.15.53:6789/0 conn(0x7f3b1c0ca210 legacy=0x7f3b1c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b08003970 0x7f3b08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b080047c0 0x7f3b08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c072070 con 0x7f3b080047c0 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c071580 con 0x7f3b08003970 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0800b980 con 0x7f3b1c06ec00 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b080047c0 0x7f3b08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b08003970 0x7f3b08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b1bfff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b13fff640 1 -- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 msgr2=0x7f3b08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:47.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b1c06ec00 0x7f3b08005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b13fff640 1 -- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b08003970 msgr2=0x7f3b08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:47.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.278+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b08003970 0x7f3b08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.279+0000 7f3b13fff640 1 -- 172.21.15.53:0/3493695538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08003520 con 0x7f3b080047c0 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.279+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b08003970 0x7f3b08003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.280+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b080047c0 0x7f3b08004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009a90 tx=0x7f3b080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660fddc59f1f251 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.280+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0402c6b0 con 0x7f3b080047c0 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.280+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0402c810 con 0x7f3b080047c0 2023-10-21T16:03:47.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.280+0000 7f3b1affd640 1 -- 172.21.15.53:0/3493695538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0402cba0 con 0x7f3b080047c0 2023-10-21T16:03:47.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3493695538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b080047c0 msgr2=0x7f3b08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:47.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 --2- 172.21.15.53:0/3493695538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b080047c0 0x7f3b08004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009a90 tx=0x7f3b080057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b1bfff640 1 -- 172.21.15.53:0/3493695538 reap_dead start 2023-10-21T16:03:47.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3493695538 shutdown_connections 2023-10-21T16:03:47.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3493695538 >> 172.21.15.53:0/3493695538 conn(0x7f3b1c0cf5f0 msgr2=0x7f3b1c0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:47.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3493695538 shutdown_connections 2023-10-21T16:03:47.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.281+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3493695538 wait complete. 2023-10-21T16:03:47.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.282+0000 7f3b20d0a640 1 Processor -- start 2023-10-21T16:03:47.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.283+0000 7f3b20d0a640 1 -- start start 2023-10-21T16:03:47.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b080047c0 0x7f3b1c0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 0x7f3b1c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b1c14aad0 0x7f3b1c14d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c14c890 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0c59e0 con 0x7f3b1c14aad0 2023-10-21T16:03:47.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b20d0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0c5140 con 0x7f3b080047c0 2023-10-21T16:03:47.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 0x7f3b1c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 0x7f3b1c14a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45918/0 (socket says 172.21.15.53:45918) 2023-10-21T16:03:47.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 -- 172.21.15.53:0/3381376409 learned_addr learned my addr 172.21.15.53:0/3381376409 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:47.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b1bfff640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b1c14aad0 0x7f3b1c14d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b080047c0 0x7f3b1c0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:47.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b080047c0 msgr2=0x7f3b1c0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:47.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b080047c0 0x7f3b1c0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b1c14aad0 msgr2=0x7f3b1c14d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:47.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b1c14aad0 0x7f3b1c14d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:47.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b13fff640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0c5420 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b080047c0 0x7f3b1c0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.284+0000 7f3b1bfff640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b1c14aad0 0x7f3b1c14d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b13fff640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 0x7f3b1c14a590 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009a90 tx=0x7f3b04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04003810 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04003970 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04003d00 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c0c5700 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.285+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0af750 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.286+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c074150 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.287+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7f3b04040450 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.288+0000 7f3b18ff9640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:47.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.288+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b040ab660 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.288+0000 7f3b1b7fe640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 msgr2=0x7f3afc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:47.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.288+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:03:47.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.292+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b040765b0 con 0x7f3b1c0af2d0 2023-10-21T16:03:47.308 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.307+0000 7ff0c044f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:03:47.360 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.359+0000 7ff0c044f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:03:47.368 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.367+0000 7f40f644c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:03:47.431 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.430+0000 7f40f644c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:03:47.478 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.476+0000 7ff0c044f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:03:47.480 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.479+0000 7f40f644c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:03:47.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.489+0000 7f3b1b7fe640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 msgr2=0x7f3afc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:47.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.489+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:03:47.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.503+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b1c160790 con 0x7f3afc06aef0 2023-10-21T16:03:47.538 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.537+0000 7f40f644c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:03:47.544 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.543+0000 7ff0c044f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:03:47.587 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.586+0000 7f40f644c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:03:47.596 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.594+0000 7ff0c044f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:03:47.655 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.654+0000 7ff0c044f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:03:47.708 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:47.706+0000 7ff0c044f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:03:47.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.890+0000 7f3b1b7fe640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 msgr2=0x7f3afc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:47.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:47.890+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:03:47.971 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:47.970+0000 7f40f644c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:03:48.113 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.112+0000 7ff0c044f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:03:48.138 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.137+0000 7f40f644c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:03:48.253 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.252+0000 7f40f644c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:03:48.281 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.279+0000 7ff0c044f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:03:48.305 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.304+0000 7f40f644c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:03:48.394 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.393+0000 7ff0c044f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:03:48.419 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.418+0000 7f40f644c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:03:48.447 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.445+0000 7ff0c044f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:03:48.532 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.531+0000 7f40f644c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:03:48.559 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.558+0000 7ff0c044f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:03:48.672 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.670+0000 7ff0c044f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:03:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:48.691+0000 7f3b1b7fe640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 msgr2=0x7f3afc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3409974301 2023-10-21T16:03:48.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:48.691+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:03:48.696 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.694+0000 7f40f644c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:03:48.747 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.745+0000 7f40f644c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:03:48.836 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.833+0000 7ff0c044f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:03:48.871 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:48.870+0000 7f40f644c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:03:48.886 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:48.885+0000 7ff0c044f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:03:49.012 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:49.010+0000 7ff0c044f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:03:49.698 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:49.697+0000 7f40f644c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:03:49.840 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:49.838+0000 7ff0c044f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:03:49.992 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:49.990+0000 7f40f644c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:03:50.049 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.047+0000 7f40f644c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:03:50.153 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:50.151+0000 7ff0c044f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:03:50.162 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.161+0000 7f40f644c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:03:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:50.185+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 7) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f3b04076980 con 0x7f3b1c0af2d0 2023-10-21T16:03:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:50.185+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 msgr2=0x7f3afc06d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:03:50.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:50.185+0000 7f3b18ff9640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/3409974301,v1:172.21.15.124:6833/3409974301] conn(0x7f3afc06aef0 0x7f3afc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:50.221 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:50.220+0000 7ff0c044f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:03:50.351 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:03:50.349+0000 7ff0c044f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:03:50.386 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f407571c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.386 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f407571c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.386 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f407571c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.387 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f4072ed7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.387 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f407571c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.387 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f407571c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.387 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f4072ed7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.387 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f4072ed7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.388 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f4072ed7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:50.388 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:03:50.385+0000 7f4072ed7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:03:51.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.195+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3b0408dfc0 con 0x7f3b1c0af2d0 2023-10-21T16:03:51.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.196+0000 7f3b18ff9640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3afc035ad0 0x7f3afc075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.196+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3afc008050 con 0x7f3afc035ad0 2023-10-21T16:03:51.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.197+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3afc035ad0 0x7f3afc075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.204+0000 7f3b1b7fe640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3afc035ad0 0x7f3afc075320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0093d0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.414+0000 7f3b18ff9640 1 -- 172.21.15.53:0/3381376409 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f3afc008050 con 0x7f3afc035ad0 2023-10-21T16:03:51.416 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-21T16:03:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3afc035ad0 msgr2=0x7f3afc075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3afc035ad0 0x7f3afc075320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0093d0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 msgr2=0x7f3b1c14a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 --2- 172.21.15.53:0/3381376409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b1c0af2d0 0x7f3b1c14a590 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009a90 tx=0x7f3b04004730 comp rx=0 tx=0).stop 2023-10-21T16:03:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b1bfff640 1 -- 172.21.15.53:0/3381376409 reap_dead start 2023-10-21T16:03:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 shutdown_connections 2023-10-21T16:03:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.417+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 >> 172.21.15.53:0/3381376409 conn(0x7f3b1c0cf5f0 msgr2=0x7f3b1c15bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.418+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 shutdown_connections 2023-10-21T16:03:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.418+0000 7f3b20d0a640 1 -- 172.21.15.53:0/3381376409 wait complete. 2023-10-21T16:03:51.443 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-21T16:03:51.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:51.444 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-21T16:03:51.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2023-10-21T16:03:51.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.718+0000 7f6463a90640 1 Processor -- start 2023-10-21T16:03:51.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.718+0000 7f6463a90640 1 -- start start 2023-10-21T16:03:51.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.719+0000 7f6463a90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f645c160660 0x7f645c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.719+0000 7f6463a90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 0x7f645c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.719+0000 7f6463a90640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c06e150 con 0x7f645c0cb450 2023-10-21T16:03:51.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.719+0000 7f6463a90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0cb8c0 con 0x7f645c160660 2023-10-21T16:03:51.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.719+0000 7f6463a90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c06de70 con 0x7f645c0c9910 2023-10-21T16:03:51.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f645c160660 0x7f645c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f645c160660 0x7f645c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45928/0 (socket says 172.21.15.53:45928) 2023-10-21T16:03:51.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 -- 172.21.15.53:0/3855077723 learned_addr learned my addr 172.21.15.53:0/3855077723 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:51.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646102b640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 0x7f645c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 -- 172.21.15.53:0/3855077723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 msgr2=0x7f645c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:51.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 0x7f645c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 -- 172.21.15.53:0/3855077723 >> v1:172.21.15.124:6789/0 conn(0x7f645c0cb450 legacy=0x7f645c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:51.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646182c640 1 -- 172.21.15.53:0/3855077723 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0c8e30 con 0x7f645c160660 2023-10-21T16:03:51.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.720+0000 7f646102b640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 0x7f645c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:51.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.722+0000 7f646182c640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c160660 0x7f645c070cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6458009e50 tx=0x7f645802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa26c5c50691266 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:51.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.722+0000 7f646082a640 1 -- 172.21.15.53:0/3855077723 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645802ca70 con 0x7f645c160660 2023-10-21T16:03:51.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.722+0000 7f646082a640 1 -- 172.21.15.53:0/3855077723 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645802cbd0 con 0x7f645c160660 2023-10-21T16:03:51.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.722+0000 7f646082a640 1 -- 172.21.15.53:0/3855077723 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580343f0 con 0x7f645c160660 2023-10-21T16:03:51.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 -- 172.21.15.53:0/3855077723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c160660 msgr2=0x7f645c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:51.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c160660 0x7f645c070cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6458009e50 tx=0x7f645802bed0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 -- 172.21.15.53:0/3855077723 shutdown_connections 2023-10-21T16:03:51.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c0c9910 0x7f645c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 --2- 172.21.15.53:0/3855077723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c160660 0x7f645c070cc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 -- 172.21.15.53:0/3855077723 >> 172.21.15.53:0/3855077723 conn(0x7f645c06a5d0 msgr2=0x7f645c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:51.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 -- 172.21.15.53:0/3855077723 shutdown_connections 2023-10-21T16:03:51.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.723+0000 7f6463a90640 1 -- 172.21.15.53:0/3855077723 wait complete. 2023-10-21T16:03:51.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.725+0000 7f6463a90640 1 Processor -- start 2023-10-21T16:03:51.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.725+0000 7f6463a90640 1 -- start start 2023-10-21T16:03:51.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f645c0cb450 0x7f645c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c160660 0x7f645c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c155b60 con 0x7f645c0c9910 2023-10-21T16:03:51.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c155230 con 0x7f645c0cb450 2023-10-21T16:03:51.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f6463a90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1554a0 con 0x7f645c160660 2023-10-21T16:03:51.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f646182c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f646182c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45936/0 (socket says 172.21.15.53:45936) 2023-10-21T16:03:51.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f646182c640 1 -- 172.21.15.53:0/3141982065 learned_addr learned my addr 172.21.15.53:0/3141982065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:51.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f646202d640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c160660 0x7f645c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.726+0000 7f646102b640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f645c0cb450 0x7f645c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 -- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c160660 msgr2=0x7f645c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:51.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c160660 0x7f645c15e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 -- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f645c0cb450 msgr2=0x7f645c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:51.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f645c0cb450 0x7f645c159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:51.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 -- 172.21.15.53:0/3141982065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458009b30 con 0x7f645c0c9910 2023-10-21T16:03:51.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f646182c640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6458009e50 tx=0x7f6458004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:51.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645803e450 con 0x7f645c0c9910 2023-10-21T16:03:51.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645803e5f0 con 0x7f645c0c9910 2023-10-21T16:03:51.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645803e9a0 con 0x7f645c0c9910 2023-10-21T16:03:51.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.727+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c155780 con 0x7f645c0c9910 2023-10-21T16:03:51.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.728+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c15a1b0 con 0x7f645c0c9910 2023-10-21T16:03:51.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.729+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c15a460 con 0x7f645c0c9910 2023-10-21T16:03:51.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.729+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f645803eb00 con 0x7f645c0c9910 2023-10-21T16:03:51.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.730+0000 7f64527fc640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 0x7f644006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:51.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.730+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f64580ac570 con 0x7f645c0c9910 2023-10-21T16:03:51.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.731+0000 7f646102b640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 0x7f644006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:51.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.732+0000 7f646102b640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 0x7f644006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f645c15ed90 tx=0x7f644c008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:51.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.735+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f645c15a460 con 0x7f645c0c9910 2023-10-21T16:03:51.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:51.949+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f645c0c93f0 con 0x7f644006ad10 2023-10-21T16:03:52.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.109+0000 7f64527fc640 1 -- 172.21.15.53:0/3141982065 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f644c00c040 con 0x7f644006ad10 2023-10-21T16:03:52.111 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-21T16:03:52.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 msgr2=0x7f644006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 0x7f644006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f645c15ed90 tx=0x7f644c008040 comp rx=0 tx=0).stop 2023-10-21T16:03:52.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 msgr2=0x7f645c1507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6458009e50 tx=0x7f6458004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:52.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 shutdown_connections 2023-10-21T16:03:52.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f644006ad10 0x7f644006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f645c160660 0x7f645c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f645c0cb450 0x7f645c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 --2- 172.21.15.53:0/3141982065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f645c0c9910 0x7f645c1507b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 >> 172.21.15.53:0/3141982065 conn(0x7f645c06a5d0 msgr2=0x7f645c0cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:52.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.112+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 shutdown_connections 2023-10-21T16:03:52.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.113+0000 7f6463a90640 1 -- 172.21.15.53:0/3141982065 wait complete. 2023-10-21T16:03:52.137 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-21T16:03:52.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:52.138 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-21T16:03:52.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2023-10-21T16:03:52.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.435+0000 7f5edf46f640 1 Processor -- start 2023-10-21T16:03:52.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.435+0000 7f5edf46f640 1 -- start start 2023-10-21T16:03:52.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.436+0000 7f5edf46f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ed80afbe0 0x7f5ed80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.436+0000 7f5edf46f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 0x7f5ed8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edf46f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80b2620 con 0x7f5ed8071070 2023-10-21T16:03:52.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edf46f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed80b2060 con 0x7f5ed80afbe0 2023-10-21T16:03:52.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edf46f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80b2340 con 0x7f5ed806dfc0 2023-10-21T16:03:52.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ed80afbe0 0x7f5ed80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:52.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ed80afbe0 0x7f5ed80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45948/0 (socket says 172.21.15.53:45948) 2023-10-21T16:03:52.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 -- 172.21.15.53:0/3486832085 learned_addr learned my addr 172.21.15.53:0/3486832085 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:52.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edca0a640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 0x7f5ed8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:52.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 -- 172.21.15.53:0/3486832085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 msgr2=0x7f5ed8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:52.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 0x7f5ed8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 -- 172.21.15.53:0/3486832085 >> v1:172.21.15.124:6789/0 conn(0x7f5ed8071070 legacy=0x7f5ed80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:52.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edd20b640 1 -- 172.21.15.53:0/3486832085 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8070350 con 0x7f5ed80afbe0 2023-10-21T16:03:52.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.437+0000 7f5edca0a640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 0x7f5ed8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:52.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.438+0000 7f5edd20b640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80b15b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2101790b4b8851b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:52.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.439+0000 7f5ecffff640 1 -- 172.21.15.53:0/3486832085 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4031640 con 0x7f5ed80afbe0 2023-10-21T16:03:52.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.439+0000 7f5ecffff640 1 -- 172.21.15.53:0/3486832085 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed40317a0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.439+0000 7f5ecffff640 1 -- 172.21.15.53:0/3486832085 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4031b30 con 0x7f5ed80afbe0 2023-10-21T16:03:52.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.439+0000 7f5edf46f640 1 -- 172.21.15.53:0/3486832085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 msgr2=0x7f5ed80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.439+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80b15b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009870 tx=0x7f5ed402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 -- 172.21.15.53:0/3486832085 shutdown_connections 2023-10-21T16:03:52.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed806dfc0 0x7f5ed8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3486832085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80b15b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 -- 172.21.15.53:0/3486832085 >> 172.21.15.53:0/3486832085 conn(0x7f5ed806a5d0 msgr2=0x7f5ed80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:52.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 -- 172.21.15.53:0/3486832085 shutdown_connections 2023-10-21T16:03:52.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.440+0000 7f5edf46f640 1 -- 172.21.15.53:0/3486832085 wait complete. 2023-10-21T16:03:52.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.441+0000 7f5edf46f640 1 Processor -- start 2023-10-21T16:03:52.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.442+0000 7f5edf46f640 1 -- start start 2023-10-21T16:03:52.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.442+0000 7f5edf46f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ed806dfc0 0x7f5ed80c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edf46f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed8071070 0x7f5ed80cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edf46f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edf46f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed814fe90 con 0x7f5ed80afbe0 2023-10-21T16:03:52.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edf46f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c3c30 con 0x7f5ed806dfc0 2023-10-21T16:03:52.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edf46f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c3f10 con 0x7f5ed8071070 2023-10-21T16:03:52.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:52.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45960/0 (socket says 172.21.15.53:45960) 2023-10-21T16:03:52.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 -- 172.21.15.53:0/3977906718 learned_addr learned my addr 172.21.15.53:0/3977906718 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:52.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edca0a640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed8071070 0x7f5ed80cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:52.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 -- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed8071070 msgr2=0x7f5ed80cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed8071070 0x7f5ed80cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 -- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ed806dfc0 msgr2=0x7f5ed80c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ed806dfc0 0x7f5ed80c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.443+0000 7f5edda0c640 1 -- 172.21.15.53:0/3977906718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40094d0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5edda0c640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000b970 tx=0x7f5ed000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:52.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed000cb40 con 0x7f5ed80afbe0 2023-10-21T16:03:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed000cca0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0014680 con 0x7f5ed80afbe0 2023-10-21T16:03:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed80c41f0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.444+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed80c47b0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.445+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed804e160 con 0x7f5ed80afbe0 2023-10-21T16:03:52.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.446+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f5ed001d3b0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.447+0000 7f5ecdffb640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 0x7f5ebc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:52.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.447+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5ed0089920 con 0x7f5ed80afbe0 2023-10-21T16:03:52.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.448+0000 7f5edd20b640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 0x7f5ebc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:52.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.449+0000 7f5edd20b640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 0x7f5ebc06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4036000 tx=0x7f5ed4030000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:52.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.451+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ed00545c0 con 0x7f5ed80afbe0 2023-10-21T16:03:52.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.669+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed80affc0 con 0x7f5ebc06afe0 2023-10-21T16:03:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.833+0000 7f5ecdffb640 1 -- 172.21.15.53:0/3977906718 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5ed4031a10 con 0x7f5ebc06afe0 2023-10-21T16:03:52.835 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-21T16:03:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 msgr2=0x7f5ebc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 0x7f5ebc06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4036000 tx=0x7f5ed4030000 comp rx=0 tx=0).stop 2023-10-21T16:03:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 msgr2=0x7f5ed80c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000b970 tx=0x7f5ed000be30 comp rx=0 tx=0).stop 2023-10-21T16:03:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 shutdown_connections 2023-10-21T16:03:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5ebc06afe0 0x7f5ebc06d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ed80afbe0 0x7f5ed80c4cd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ed8071070 0x7f5ed80cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 --2- 172.21.15.53:0/3977906718 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ed806dfc0 0x7f5ed80c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 >> 172.21.15.53:0/3977906718 conn(0x7f5ed806a5d0 msgr2=0x7f5ed806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 shutdown_connections 2023-10-21T16:03:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:52.837+0000 7f5edf46f640 1 -- 172.21.15.53:0/3977906718 wait complete. 2023-10-21T16:03:52.861 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-21T16:03:52.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:52.862 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-21T16:03:52.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2023-10-21T16:03:53.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.149+0000 7fbb474af640 1 Processor -- start 2023-10-21T16:03:53.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.149+0000 7fbb474af640 1 -- start start 2023-10-21T16:03:53.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb474af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb400cb150 0x7fbb400cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb474af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb400b1f20 0x7fbb400b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb474af640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4006e1e0 con 0x7fbb40161690 2023-10-21T16:03:53.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb474af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400cbb40 con 0x7fbb400cb150 2023-10-21T16:03:53.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb474af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4006cc50 con 0x7fbb400b1f20 2023-10-21T16:03:53.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb44a4a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb400cb150 0x7fbb400cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb4524b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbb40161690 0x7fbb4006dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51012/0 (socket says 172.21.15.53:51012) 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.151+0000 7fbb4524b640 1 -- 172.21.15.53:0/1549591054 learned_addr learned my addr 172.21.15.53:0/1549591054 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.152+0000 7fbb45a4c640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb400b1f20 0x7fbb400b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.152+0000 7fbb44a4a640 1 -- 172.21.15.53:0/1549591054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb400b1f20 msgr2=0x7fbb400b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.152+0000 7fbb44a4a640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb400b1f20 0x7fbb400b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.152+0000 7fbb44a4a640 1 -- 172.21.15.53:0/1549591054 >> v1:172.21.15.53:6789/0 conn(0x7fbb40161690 legacy=0x7fbb4006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.152+0000 7fbb44a4a640 1 -- 172.21.15.53:0/1549591054 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4006cf30 con 0x7fbb400cb150 2023-10-21T16:03:53.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.153+0000 7fbb44a4a640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb400cb600 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097a0 tx=0x7fbb3002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735afce8e7ba14ec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.153+0000 7fbb37fff640 1 -- 172.21.15.53:0/1549591054 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3002d040 con 0x7fbb400cb150 2023-10-21T16:03:53.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.153+0000 7fbb37fff640 1 -- 172.21.15.53:0/1549591054 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30031810 con 0x7fbb400cb150 2023-10-21T16:03:53.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.153+0000 7fbb37fff640 1 -- 172.21.15.53:0/1549591054 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb30031ba0 con 0x7fbb400cb150 2023-10-21T16:03:53.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 -- 172.21.15.53:0/1549591054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 msgr2=0x7fbb400cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb400cb600 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097a0 tx=0x7fbb3002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 -- 172.21.15.53:0/1549591054 shutdown_connections 2023-10-21T16:03:53.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb400b1f20 0x7fbb400b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 --2- 172.21.15.53:0/1549591054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb400cb600 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 -- 172.21.15.53:0/1549591054 >> 172.21.15.53:0/1549591054 conn(0x7fbb4006a5d0 msgr2=0x7fbb400b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:53.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 -- 172.21.15.53:0/1549591054 shutdown_connections 2023-10-21T16:03:53.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.154+0000 7fbb474af640 1 -- 172.21.15.53:0/1549591054 wait complete. 2023-10-21T16:03:53.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.156+0000 7fbb474af640 1 Processor -- start 2023-10-21T16:03:53.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.156+0000 7fbb474af640 1 -- start start 2023-10-21T16:03:53.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb400b1f20 0x7fbb40162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb40161690 0x7fbb400c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400c1600 con 0x7fbb400cb150 2023-10-21T16:03:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015fde0 con 0x7fbb400b1f20 2023-10-21T16:03:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb474af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40162b40 con 0x7fbb40161690 2023-10-21T16:03:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb44a4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb44a4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45988/0 (socket says 172.21.15.53:45988) 2023-10-21T16:03:53.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.157+0000 7fbb44a4a640 1 -- 172.21.15.53:0/106808587 learned_addr learned my addr 172.21.15.53:0/106808587 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:53.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 -- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb40161690 msgr2=0x7fbb400c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb40161690 0x7fbb400c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 -- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb400b1f20 msgr2=0x7fbb40162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb400b1f20 0x7fbb40162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 -- 172.21.15.53:0/106808587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30009400 con 0x7fbb400cb150 2023-10-21T16:03:53.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb44a4a640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbb300098c0 tx=0x7fbb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3003ad60 con 0x7fbb400cb150 2023-10-21T16:03:53.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb30044400 con 0x7fbb400cb150 2023-10-21T16:03:53.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300447b0 con 0x7fbb400cb150 2023-10-21T16:03:53.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.158+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb40162e20 con 0x7fbb400cb150 2023-10-21T16:03:53.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.159+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb401638a0 con 0x7fbb400cb150 2023-10-21T16:03:53.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.160+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4004e160 con 0x7fbb400cb150 2023-10-21T16:03:53.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.161+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fbb3004d440 con 0x7fbb400cb150 2023-10-21T16:03:53.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.162+0000 7fbb35ffb640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 0x7fbb2406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.162+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fbb300abf80 con 0x7fbb400cb150 2023-10-21T16:03:53.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.162+0000 7fbb4524b640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 0x7fbb2406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.164+0000 7fbb4524b640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 0x7fbb2406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005880 tx=0x7fbb3c007a00 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.167+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb300deca0 con 0x7fbb400cb150 2023-10-21T16:03:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.381+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4006d4f0 con 0x7fbb2406afe0 2023-10-21T16:03:53.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.528+0000 7fbb35ffb640 1 -- 172.21.15.53:0/106808587 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbb3c0058b0 con 0x7fbb2406afe0 2023-10-21T16:03:53.531 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-21T16:03:53.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 msgr2=0x7fbb2406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 0x7fbb2406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005880 tx=0x7fbb3c007a00 comp rx=0 tx=0).stop 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 msgr2=0x7fbb4015f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbb300098c0 tx=0x7fbb300046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 shutdown_connections 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fbb2406afe0 0x7fbb2406d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb40161690 0x7fbb400c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb400cb150 0x7fbb4015f8a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 --2- 172.21.15.53:0/106808587 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb400b1f20 0x7fbb40162480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.531+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 >> 172.21.15.53:0/106808587 conn(0x7fbb4006a5d0 msgr2=0x7fbb400b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:53.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.532+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 shutdown_connections 2023-10-21T16:03:53.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.532+0000 7fbb474af640 1 -- 172.21.15.53:0/106808587 wait complete. 2023-10-21T16:03:53.556 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-21T16:03:53.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:53.557 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-21T16:03:53.614 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2023-10-21T16:03:53.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.839+0000 7f7a253d5640 1 Processor -- start 2023-10-21T16:03:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.840+0000 7f7a253d5640 1 -- start start 2023-10-21T16:03:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.841+0000 7f7a253d5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a200b1960 0x7f7a2015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.841+0000 7f7a253d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 0x7f7a20160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.841+0000 7f7a253d5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a200b1df0 con 0x7f7a200b10c0 2023-10-21T16:03:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.841+0000 7f7a253d5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200d14d0 con 0x7f7a200b1960 2023-10-21T16:03:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.841+0000 7f7a253d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200b1680 con 0x7f7a2006bf60 2023-10-21T16:03:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.842+0000 7f7a167fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a200b1960 0x7f7a2015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.842+0000 7f7a1f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 0x7f7a20160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.842+0000 7f7a167fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a200b1960 0x7f7a2015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:53642/0 (socket says 172.21.15.53:53642) 2023-10-21T16:03:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.842+0000 7f7a167fc640 1 -- 172.21.15.53:0/2303801247 learned_addr learned my addr 172.21.15.53:0/2303801247 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.843+0000 7f7a167fc640 1 -- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 msgr2=0x7f7a20160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.843+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 0x7f7a20160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.843+0000 7f7a167fc640 1 -- 172.21.15.53:0/2303801247 >> v1:172.21.15.124:6789/0 conn(0x7f7a200b10c0 legacy=0x7f7a2006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.843+0000 7f7a167fc640 1 -- 172.21.15.53:0/2303801247 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2006b1a0 con 0x7f7a200b1960 2023-10-21T16:03:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.843+0000 7f7a1f7fe640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 0x7f7a20160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:03:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.844+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b1960 0x7f7a2015e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45df2bd365f0f242 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.845+0000 7f7a1e7fc640 1 -- 172.21.15.53:0/2303801247 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08031640 con 0x7f7a200b1960 2023-10-21T16:03:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.845+0000 7f7a1e7fc640 1 -- 172.21.15.53:0/2303801247 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a080317a0 con 0x7f7a200b1960 2023-10-21T16:03:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.845+0000 7f7a1e7fc640 1 -- 172.21.15.53:0/2303801247 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08031b30 con 0x7f7a200b1960 2023-10-21T16:03:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 -- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b1960 msgr2=0x7f7a2015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b1960 0x7f7a2015e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 -- 172.21.15.53:0/2303801247 shutdown_connections 2023-10-21T16:03:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a2006bf60 0x7f7a20160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2303801247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b1960 0x7f7a2015e9f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 -- 172.21.15.53:0/2303801247 >> 172.21.15.53:0/2303801247 conn(0x7f7a20099d40 msgr2=0x7f7a200d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 -- 172.21.15.53:0/2303801247 shutdown_connections 2023-10-21T16:03:53.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.846+0000 7f7a253d5640 1 -- 172.21.15.53:0/2303801247 wait complete. 2023-10-21T16:03:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.848+0000 7f7a253d5640 1 Processor -- start 2023-10-21T16:03:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.848+0000 7f7a253d5640 1 -- start start 2023-10-21T16:03:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 0x7f7a200b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b5fd0 0x7f7a200cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d08a0 con 0x7f7a2006bf60 2023-10-21T16:03:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200cfd20 con 0x7f7a200b5fd0 2023-10-21T16:03:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a253d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200d0000 con 0x7f7a200b10c0 2023-10-21T16:03:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45994/0 (socket says 172.21.15.53:45994) 2023-10-21T16:03:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 -- 172.21.15.53:0/2006350628 learned_addr learned my addr 172.21.15.53:0/2006350628 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 0x7f7a200b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1f7fe640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b5fd0 0x7f7a200cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 -- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 msgr2=0x7f7a200b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 0x7f7a200b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 -- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b5fd0 msgr2=0x7f7a200cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b5fd0 0x7f7a200cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.850+0000 7f7a1effd640 1 -- 172.21.15.53:0/2006350628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18009a70 con 0x7f7a2006bf60 2023-10-21T16:03:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 0x7f7a200b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a1effd640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000c430 tx=0x7f7a1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100052d0 con 0x7f7a2006bf60 2023-10-21T16:03:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a10005430 con 0x7f7a2006bf60 2023-10-21T16:03:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a080094d0 con 0x7f7a2006bf60 2023-10-21T16:03:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10003850 con 0x7f7a2006bf60 2023-10-21T16:03:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.851+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a200d05c0 con 0x7f7a2006bf60 2023-10-21T16:03:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.853+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2004e160 con 0x7f7a2006bf60 2023-10-21T16:03:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.854+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f7a100039b0 con 0x7f7a2006bf60 2023-10-21T16:03:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.854+0000 7f7a17fff640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 0x7f7a0006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.855+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7a10088f70 con 0x7f7a2006bf60 2023-10-21T16:03:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.855+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 0x7f7a0006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.856+0000 7f7a167fc640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 0x7f7a0006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a08036000 tx=0x7f7a08008030 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:53.859+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a100bbca0 con 0x7f7a2006bf60 2023-10-21T16:03:54.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.073+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a2006b480 con 0x7f7a0006afe0 2023-10-21T16:03:54.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.222+0000 7f7a17fff640 1 -- 172.21.15.53:0/2006350628 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7a08031990 con 0x7f7a0006afe0 2023-10-21T16:03:54.226 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-21T16:03:54.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 msgr2=0x7f7a0006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 0x7f7a0006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a08036000 tx=0x7f7a08008030 comp rx=0 tx=0).stop 2023-10-21T16:03:54.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 msgr2=0x7f7a20152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000c430 tx=0x7f7a1000c8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 shutdown_connections 2023-10-21T16:03:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7a0006afe0 0x7f7a0006d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a200b5fd0 0x7f7a200cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a200b10c0 0x7f7a200b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 --2- 172.21.15.53:0/2006350628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a2006bf60 0x7f7a20152e00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.226+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 >> 172.21.15.53:0/2006350628 conn(0x7f7a20099d40 msgr2=0x7f7a200d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:54.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.227+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 shutdown_connections 2023-10-21T16:03:54.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.227+0000 7f7a253d5640 1 -- 172.21.15.53:0/2006350628 wait complete. 2023-10-21T16:03:54.251 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-21T16:03:54.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:54.252 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-21T16:03:54.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2023-10-21T16:03:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.534+0000 7ff084e8a640 1 Processor -- start 2023-10-21T16:03:54.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.534+0000 7ff084e8a640 1 -- start start 2023-10-21T16:03:54.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.535+0000 7ff084e8a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff0800cf690 0x7ff0800d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff084e8a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800cd520 con 0x7ff0800cf2b0 2023-10-21T16:03:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff084e8a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0800d1910 con 0x7ff0800ce510 2023-10-21T16:03:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff084e8a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0800ce080 con 0x7ff0800cf690 2023-10-21T16:03:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff07f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff0800cf2b0 0x7ff0800d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48162/0 (socket says 172.21.15.53:48162) 2023-10-21T16:03:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff07f7fe640 1 -- 172.21.15.53:0/1167483959 learned_addr learned my addr 172.21.15.53:0/1167483959 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:54.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653522395 0 0) 0x7ff0800cd520 con 0x7ff0800cf2b0 2023-10-21T16:03:54.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff06c003520 con 0x7ff0800cf2b0 2023-10-21T16:03:54.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.536+0000 7ff077fff640 1 --2- 172.21.15.53:0/1167483959 >> v2:172.21.15.124:3300/0 conn(0x7ff0800cf690 0x7ff0800d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688636367 0 0) 0x7ff06c003520 con 0x7ff0800cf2b0 2023-10-21T16:03:54.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 >> v2:172.21.15.124:3300/0 conn(0x7ff0800cf690 msgr2=0x7ff0800d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 --2- 172.21.15.53:0/1167483959 >> v2:172.21.15.124:3300/0 conn(0x7ff0800cf690 0x7ff0800d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 >> v1:172.21.15.124:6789/0 conn(0x7ff0800ce510 legacy=0x7ff08015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800cd750 con 0x7ff0800cf2b0 2023-10-21T16:03:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff070001670 con 0x7ff0800cf2b0 2023-10-21T16:03:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 >> v1:172.21.15.53:6789/0 conn(0x7ff0800cf2b0 legacy=0x7ff0800d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c0039c0 0x7ff06c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff06c0047c0 0x7ff06c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 0x7ff06c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800ce080 con 0x7ff06c005250 2023-10-21T16:03:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c00c1a0 con 0x7ff06c0047c0 2023-10-21T16:03:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c00c360 con 0x7ff06c0039c0 2023-10-21T16:03:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.537+0000 7ff07ffff640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 0x7ff06c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff077fff640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff06c0047c0 0x7ff06c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07f7fe640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c0039c0 0x7ff06c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07ffff640 1 -- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c0039c0 msgr2=0x7ff06c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07ffff640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c0039c0 0x7ff06c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07ffff640 1 -- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff06c0047c0 msgr2=0x7ff06c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07ffff640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff06c0047c0 0x7ff06c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.538+0000 7ff07ffff640 1 -- 172.21.15.53:0/1167483959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c003520 con 0x7ff06c005250 2023-10-21T16:03:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.539+0000 7ff07ffff640 1 -- 172.21.15.53:0/1167483959 reap_dead start 2023-10-21T16:03:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.539+0000 7ff07ffff640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 0x7ff06c00b980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff0780044b0 tx=0x7ff06c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a349945808b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.539+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078006d50 con 0x7ff06c005250 2023-10-21T16:03:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.539+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078007020 con 0x7ff06c005250 2023-10-21T16:03:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.539+0000 7ff07effd640 1 -- 172.21.15.53:0/1167483959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780073d0 con 0x7ff06c005250 2023-10-21T16:03:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 -- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 msgr2=0x7ff06c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 0x7ff06c00b980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff0780044b0 tx=0x7ff06c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 -- 172.21.15.53:0/1167483959 shutdown_connections 2023-10-21T16:03:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 --2- 172.21.15.53:0/1167483959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff06c005250 0x7ff06c00b980 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 -- 172.21.15.53:0/1167483959 >> 172.21.15.53:0/1167483959 conn(0x7ff08009bc40 msgr2=0x7ff0800d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:54.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 -- 172.21.15.53:0/1167483959 shutdown_connections 2023-10-21T16:03:54.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.540+0000 7ff084e8a640 1 -- 172.21.15.53:0/1167483959 wait complete. 2023-10-21T16:03:54.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.542+0000 7ff084e8a640 1 Processor -- start 2023-10-21T16:03:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.542+0000 7ff084e8a640 1 -- start start 2023-10-21T16:03:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c005250 0x7ff08015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 0x7ff08014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800b6720 con 0x7ff0801f5820 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff080149e00 con 0x7ff0801f5ca0 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff084e8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08015f700 con 0x7ff06c005250 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51140/0 (socket says 172.21.15.53:51140) 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.543+0000 7ff077fff640 1 -- 172.21.15.53:0/900339942 learned_addr learned my addr 172.21.15.53:0/900339942 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:54.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07f7fe640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c005250 0x7ff08015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07ffff640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 0x7ff08014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 -- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c005250 msgr2=0x7ff08015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c005250 0x7ff08015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 -- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 msgr2=0x7ff08014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 0x7ff08014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 -- 172.21.15.53:0/900339942 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08015ee60 con 0x7ff0801f5820 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07ffff640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 0x7ff08014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff077fff640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff06800f5b0 tx=0x7ff06800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068013810 con 0x7ff0801f5820 2023-10-21T16:03:54.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068013970 con 0x7ff0801f5820 2023-10-21T16:03:54.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08015f140 con 0x7ff0801f5820 2023-10-21T16:03:54.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.544+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068013ce0 con 0x7ff0801f5820 2023-10-21T16:03:54.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.545+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08015eac0 con 0x7ff0801f5820 2023-10-21T16:03:54.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.546+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0800b6860 con 0x7ff0801f5820 2023-10-21T16:03:54.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.547+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff06801f460 con 0x7ff0801f5820 2023-10-21T16:03:54.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.548+0000 7ff07cff9640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 0x7ff06006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:54.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.548+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7ff06808a060 con 0x7ff0801f5820 2023-10-21T16:03:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.548+0000 7ff07f7fe640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 0x7ff06006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.550+0000 7ff07f7fe640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 0x7ff06006d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff070000c00 tx=0x7ff070009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:54.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.552+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0680bcca0 con 0x7ff0801f5820 2023-10-21T16:03:54.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.772+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0800cdd10 con 0x7ff06006b070 2023-10-21T16:03:54.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.931+0000 7ff07cff9640 1 -- 172.21.15.53:0/900339942 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff07000d070 con 0x7ff06006b070 2023-10-21T16:03:54.934 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-21T16:03:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 msgr2=0x7ff06006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 0x7ff06006d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff070000c00 tx=0x7ff070009040 comp rx=0 tx=0).stop 2023-10-21T16:03:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 msgr2=0x7ff08014f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff06800f5b0 tx=0x7ff06800fa70 comp rx=0 tx=0).stop 2023-10-21T16:03:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 shutdown_connections 2023-10-21T16:03:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff06006b070 0x7ff06006d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0801f5ca0 0x7ff08014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0801f5820 0x7ff08014f590 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 --2- 172.21.15.53:0/900339942 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff06c005250 0x7ff08015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.934+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 >> 172.21.15.53:0/900339942 conn(0x7ff08009bc40 msgr2=0x7ff0800b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.935+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 shutdown_connections 2023-10-21T16:03:54.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:54.935+0000 7ff084e8a640 1 -- 172.21.15.53:0/900339942 wait complete. 2023-10-21T16:03:54.959 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-21T16:03:54.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:54.960 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-21T16:03:55.019 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2023-10-21T16:03:55.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.242+0000 7f2b549a4640 1 Processor -- start 2023-10-21T16:03:55.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.243+0000 7f2b549a4640 1 -- start start 2023-10-21T16:03:55.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b549a4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2b501599d0 0x7f2b500cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b549a4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b500b3c50 0x7f2b500caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b549a4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b50159db0 con 0x7f2b500cb410 2023-10-21T16:03:55.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b549a4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b50071fe0 con 0x7f2b500b3c50 2023-10-21T16:03:55.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b549a4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500b44c0 con 0x7f2b501599d0 2023-10-21T16:03:55.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b4e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b500b3c50 0x7f2b500caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b4e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b500b3c50 0x7f2b500caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51142/0 (socket says 172.21.15.53:51142) 2023-10-21T16:03:55.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.244+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1163714566 learned_addr learned my addr 172.21.15.53:0/1163714566 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:55.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.245+0000 7f2b4ed9d640 1 --2- 172.21.15.53:0/1163714566 >> v2:172.21.15.124:3300/0 conn(0x7f2b501599d0 0x7f2b500cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.245+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1163714566 >> v2:172.21.15.124:3300/0 conn(0x7f2b501599d0 msgr2=0x7f2b500cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:55.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.245+0000 7f2b4e59c640 1 --2- 172.21.15.53:0/1163714566 >> v2:172.21.15.124:3300/0 conn(0x7f2b501599d0 0x7f2b500cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.245+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1163714566 >> v1:172.21.15.124:6789/0 conn(0x7f2b500cb410 legacy=0x7f2b500c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:55.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.245+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1163714566 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500cb860 con 0x7f2b500b3c50 2023-10-21T16:03:55.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.246+0000 7f2b4e59c640 1 --2- 172.21.15.53:0/1163714566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500b3c50 0x7f2b500caed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006e150 tx=0x7f2b3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423a2519e5cc1066 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.246+0000 7f2b4dd9b640 1 -- 172.21.15.53:0/1163714566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38031870 con 0x7f2b500b3c50 2023-10-21T16:03:55.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.246+0000 7f2b4dd9b640 1 -- 172.21.15.53:0/1163714566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b380319d0 con 0x7f2b500b3c50 2023-10-21T16:03:55.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.246+0000 7f2b4dd9b640 1 -- 172.21.15.53:0/1163714566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38031d60 con 0x7f2b500b3c50 2023-10-21T16:03:55.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 -- 172.21.15.53:0/1163714566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500b3c50 msgr2=0x7f2b500caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1163714566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500b3c50 0x7f2b500caed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006e150 tx=0x7f2b3802b910 comp rx=0 tx=0).stop 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 -- 172.21.15.53:0/1163714566 shutdown_connections 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1163714566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500b3c50 0x7f2b500caed0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1163714566 >> v2:172.21.15.124:3300/0 conn(0x7f2b501599d0 0x7f2b500cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 -- 172.21.15.53:0/1163714566 >> 172.21.15.53:0/1163714566 conn(0x7f2b5006c050 msgr2=0x7f2b5006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:55.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 -- 172.21.15.53:0/1163714566 shutdown_connections 2023-10-21T16:03:55.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.247+0000 7f2b549a4640 1 -- 172.21.15.53:0/1163714566 wait complete. 2023-10-21T16:03:55.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.249+0000 7f2b549a4640 1 Processor -- start 2023-10-21T16:03:55.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.249+0000 7f2b549a4640 1 -- start start 2023-10-21T16:03:55.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b500b3c50 0x7f2b501524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 0x7f2b501e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b1560 con 0x7f2b500cb410 2023-10-21T16:03:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b0cc0 con 0x7f2b501599d0 2023-10-21T16:03:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b549a4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b0fa0 con 0x7f2b500b3c50 2023-10-21T16:03:55.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51144/0 (socket says 172.21.15.53:51144) 2023-10-21T16:03:55.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1637701253 learned_addr learned my addr 172.21.15.53:0/1637701253 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:55.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b4ed9d640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b500b3c50 0x7f2b501524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.250+0000 7f2b4f59e640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 0x7f2b501e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b500b3c50 msgr2=0x7f2b501524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b500b3c50 0x7f2b501524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 msgr2=0x7f2b501e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 0x7f2b501e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 -- 172.21.15.53:0/1637701253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380094d0 con 0x7f2b500cb410 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4f59e640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 0x7f2b501e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b4e59c640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2b38000c00 tx=0x7f2b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3803abe0 con 0x7f2b500cb410 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3803ad40 con 0x7f2b500cb410 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38044610 con 0x7f2b500cb410 2023-10-21T16:03:55.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.251+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b500b1280 con 0x7f2b500cb410 2023-10-21T16:03:55.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.252+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b50152c80 con 0x7f2b500cb410 2023-10-21T16:03:55.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.253+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b501548a0 con 0x7f2b500cb410 2023-10-21T16:03:55.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.254+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f2b3804d450 con 0x7f2b500cb410 2023-10-21T16:03:55.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.254+0000 7f2b477fe640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 0x7f2b3006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.255+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2b3802d040 con 0x7f2b500cb410 2023-10-21T16:03:55.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.255+0000 7f2b4ed9d640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 0x7f2b3006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.256+0000 7f2b4ed9d640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 0x7f2b3006d560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b400099c0 tx=0x7f2b40003990 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.259+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b501548a0 con 0x7f2b500cb410 2023-10-21T16:03:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.482+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b500716b0 con 0x7f2b3006b0a0 2023-10-21T16:03:55.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.657+0000 7f2b477fe640 1 -- 172.21.15.53:0/1637701253 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2b40009650 con 0x7f2b3006b0a0 2023-10-21T16:03:55.660 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-21T16:03:55.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 msgr2=0x7f2b3006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 0x7f2b3006d560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2b400099c0 tx=0x7f2b40003990 comp rx=0 tx=0).stop 2023-10-21T16:03:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 msgr2=0x7f2b50151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2b38000c00 tx=0x7f2b380046c0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 shutdown_connections 2023-10-21T16:03:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2b3006b0a0 0x7f2b3006d560 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b501599d0 0x7f2b501e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b500cb410 0x7f2b50151e80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 --2- 172.21.15.53:0/1637701253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b500b3c50 0x7f2b501524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 >> 172.21.15.53:0/1637701253 conn(0x7f2b5006c050 msgr2=0x7f2b5015c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 shutdown_connections 2023-10-21T16:03:55.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.660+0000 7f2b549a4640 1 -- 172.21.15.53:0/1637701253 wait complete. 2023-10-21T16:03:55.683 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-21T16:03:55.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:55.685 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-21T16:03:55.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2023-10-21T16:03:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.962+0000 7fae85c7c640 1 Processor -- start 2023-10-21T16:03:55.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.962+0000 7fae85c7c640 1 -- start start 2023-10-21T16:03:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae85c7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 0x7fae8006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae85c7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae8015aa00 0x7fae800b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae85c7c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8015b310 con 0x7fae8006c3a0 2023-10-21T16:03:55.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae85c7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8006e0a0 con 0x7fae8015aa00 2023-10-21T16:03:55.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae85c7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8006d290 con 0x7fae800c9cf0 2023-10-21T16:03:55.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae7ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fae8015aa00 0x7fae800b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae7f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fae8006c3a0 0x7fae8006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48176/0 (socket says 172.21.15.53:48176) 2023-10-21T16:03:55.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.964+0000 7fae7f7fe640 1 -- 172.21.15.53:0/725064598 learned_addr learned my addr 172.21.15.53:0/725064598 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7effd640 1 --2- 172.21.15.53:0/725064598 >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 0x7fae8006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7ffff640 1 -- 172.21.15.53:0/725064598 >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 msgr2=0x7fae8006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:55.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7ffff640 1 --2- 172.21.15.53:0/725064598 >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 0x7fae8006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7ffff640 1 -- 172.21.15.53:0/725064598 >> v1:172.21.15.53:6789/0 conn(0x7fae8006c3a0 legacy=0x7fae8006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7ffff640 1 -- 172.21.15.53:0/725064598 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae800c9860 con 0x7fae8015aa00 2023-10-21T16:03:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7e7fc640 1 -- 172.21.15.53:0/725064598 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307574902 0 0) 0x7fae70001110 con 0x7fae8006c3a0 2023-10-21T16:03:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.965+0000 7fae7effd640 1 --2- 172.21.15.53:0/725064598 >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 0x7fae8006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:55.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.966+0000 7fae7ffff640 1 --2- 172.21.15.53:0/725064598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae8015aa00 0x7fae800b2a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae800715c0 tx=0x7fae7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb814813fb4f384 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.966+0000 7fae7e7fc640 1 -- 172.21.15.53:0/725064598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7802cc80 con 0x7fae8015aa00 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae7e7fc640 1 -- 172.21.15.53:0/725064598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7802cde0 con 0x7fae8015aa00 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae7e7fc640 1 -- 172.21.15.53:0/725064598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78034660 con 0x7fae8015aa00 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 -- 172.21.15.53:0/725064598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae8015aa00 msgr2=0x7fae800b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 --2- 172.21.15.53:0/725064598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae8015aa00 0x7fae800b2a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fae800715c0 tx=0x7fae7802bed0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 -- 172.21.15.53:0/725064598 shutdown_connections 2023-10-21T16:03:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 --2- 172.21.15.53:0/725064598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae8015aa00 0x7fae800b2a90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 --2- 172.21.15.53:0/725064598 >> v2:172.21.15.124:3300/0 conn(0x7fae800c9cf0 0x7fae8006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 -- 172.21.15.53:0/725064598 >> 172.21.15.53:0/725064598 conn(0x7fae8006a5d0 msgr2=0x7fae800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 -- 172.21.15.53:0/725064598 shutdown_connections 2023-10-21T16:03:55.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.967+0000 7fae85c7c640 1 -- 172.21.15.53:0/725064598 wait complete. 2023-10-21T16:03:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.970+0000 7fae85c7c640 1 Processor -- start 2023-10-21T16:03:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.970+0000 7fae85c7c640 1 -- start start 2023-10-21T16:03:55.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae8006c3a0 0x7fae801e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae8015aa00 0x7fae800702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae800c6120 con 0x7fae800c9cf0 2023-10-21T16:03:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae800c5810 con 0x7fae8015aa00 2023-10-21T16:03:55.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.971+0000 7fae85c7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae800c1120 con 0x7fae8006c3a0 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51166/0 (socket says 172.21.15.53:51166) 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 -- 172.21.15.53:0/823676690 learned_addr learned my addr 172.21.15.53:0/823676690 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7ffff640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae8015aa00 0x7fae800702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7f7fe640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae8006c3a0 0x7fae801e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 -- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae8006c3a0 msgr2=0x7fae801e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae8006c3a0 0x7fae801e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 -- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae8015aa00 msgr2=0x7fae800702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae8015aa00 0x7fae800702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.972+0000 7fae7effd640 1 -- 172.21.15.53:0/823676690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae78009a70 con 0x7fae800c9cf0 2023-10-21T16:03:55.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae7effd640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c7f0 tx=0x7fae7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74007a30 con 0x7fae800c9cf0 2023-10-21T16:03:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74007b90 con 0x7fae800c9cf0 2023-10-21T16:03:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400ce40 con 0x7fae800c9cf0 2023-10-21T16:03:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae800c5ad0 con 0x7fae800c9cf0 2023-10-21T16:03:55.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.973+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae800c8550 con 0x7fae800c9cf0 2023-10-21T16:03:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.974+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae800c6290 con 0x7fae800c9cf0 2023-10-21T16:03:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.975+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fae740113f0 con 0x7fae800c9cf0 2023-10-21T16:03:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.976+0000 7fae67fff640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 0x7fae6c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.976+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fae74089e60 con 0x7fae800c9cf0 2023-10-21T16:03:55.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.977+0000 7fae7f7fe640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 0x7fae6c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:55.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.978+0000 7fae7f7fe640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 0x7fae6c06d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fae70000f60 tx=0x7fae700079b0 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:55.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:55.981+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae74054910 con 0x7fae800c9cf0 2023-10-21T16:03:56.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.196+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fae800c7950 con 0x7fae6c06b070 2023-10-21T16:03:56.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.360+0000 7fae67fff640 1 -- 172.21.15.53:0/823676690 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fae800c7950 con 0x7fae6c06b070 2023-10-21T16:03:56.363 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-21T16:03:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 msgr2=0x7fae6c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 0x7fae6c06d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fae70000f60 tx=0x7fae700079b0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 msgr2=0x7fae800c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c7f0 tx=0x7fae7400cbf0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 shutdown_connections 2023-10-21T16:03:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fae6c06b070 0x7fae6c06d530 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae8015aa00 0x7fae800702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae800c9cf0 0x7fae800c8010 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 --2- 172.21.15.53:0/823676690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae8006c3a0 0x7fae801e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.363+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 >> 172.21.15.53:0/823676690 conn(0x7fae8006a5d0 msgr2=0x7fae800caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.364+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 shutdown_connections 2023-10-21T16:03:56.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.364+0000 7fae85c7c640 1 -- 172.21.15.53:0/823676690 wait complete. 2023-10-21T16:03:56.386 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-21T16:03:56.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:56.387 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-21T16:03:56.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2023-10-21T16:03:56.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.674+0000 7f789a0ad640 1 Processor -- start 2023-10-21T16:03:56.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.674+0000 7f789a0ad640 1 -- start start 2023-10-21T16:03:56.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f789a0ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78940cf2b0 0x7f78940d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f789a0ad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940cd520 con 0x7f78940ce510 2023-10-21T16:03:56.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f789a0ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940d1910 con 0x7f78940cf690 2023-10-21T16:03:56.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f789a0ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78940ce080 con 0x7f78940cf2b0 2023-10-21T16:03:56.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f78998ac640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f78940ce510 0x7f789415c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48186/0 (socket says 172.21.15.53:48186) 2023-10-21T16:03:56.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.675+0000 7f78998ac640 1 -- 172.21.15.53:0/3875076373 learned_addr learned my addr 172.21.15.53:0/3875076373 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:56.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f78990ab640 1 --2- 172.21.15.53:0/3875076373 >> v2:172.21.15.124:3300/0 conn(0x7f78940cf2b0 0x7f78940d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224329193 0 0) 0x7f78940cd520 con 0x7f78940ce510 2023-10-21T16:03:56.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787c003520 con 0x7f78940ce510 2023-10-21T16:03:56.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 500424734 0 0) 0x7f787c003520 con 0x7f78940ce510 2023-10-21T16:03:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 >> v2:172.21.15.124:3300/0 conn(0x7f78940cf2b0 msgr2=0x7f78940d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 --2- 172.21.15.53:0/3875076373 >> v2:172.21.15.124:3300/0 conn(0x7f78940cf2b0 0x7f78940d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 >> v1:172.21.15.124:6789/0 conn(0x7f78940cf690 legacy=0x7f78940d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940cd750 con 0x7f78940ce510 2023-10-21T16:03:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f788c001670 con 0x7f78940ce510 2023-10-21T16:03:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 >> v1:172.21.15.53:6789/0 conn(0x7f78940ce510 legacy=0x7f789415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787c003970 0x7f787c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c0047c0 0x7f787c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 0x7f787c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940ce080 con 0x7f787c005180 2023-10-21T16:03:56.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c00c120 con 0x7f787c003970 2023-10-21T16:03:56.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.676+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c00c2e0 con 0x7f787c0047c0 2023-10-21T16:03:56.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 0x7f787c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78990ab640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787c003970 0x7f787c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78988aa640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c0047c0 0x7f787c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 -- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c0047c0 msgr2=0x7f787c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c0047c0 0x7f787c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 -- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787c003970 msgr2=0x7f787c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787c003970 0x7f787c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78998ac640 1 -- 172.21.15.53:0/3875076373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c003520 con 0x7f787c005180 2023-10-21T16:03:56.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.677+0000 7f78990ab640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787c003970 0x7f787c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:56.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.678+0000 7f78998ac640 1 -- 172.21.15.53:0/3875076373 reap_dead start 2023-10-21T16:03:56.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f78998ac640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 0x7f787c00b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c240 tx=0x7f787c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197a40916c617a58 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:56.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c005ad0 con 0x7f787c005180 2023-10-21T16:03:56.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c005c70 con 0x7f787c005180 2023-10-21T16:03:56.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f7893fff640 1 -- 172.21.15.53:0/3875076373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c02c8b0 con 0x7f787c005180 2023-10-21T16:03:56.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f789a0ad640 1 -- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 msgr2=0x7f787c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.679+0000 7f789a0ad640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 0x7f787c00b980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c240 tx=0x7f787c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.680+0000 7f789a0ad640 1 -- 172.21.15.53:0/3875076373 shutdown_connections 2023-10-21T16:03:56.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.680+0000 7f789a0ad640 1 --2- 172.21.15.53:0/3875076373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787c005180 0x7f787c00b980 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.680+0000 7f789a0ad640 1 -- 172.21.15.53:0/3875076373 >> 172.21.15.53:0/3875076373 conn(0x7f789409bc40 msgr2=0x7f78940d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:56.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.680+0000 7f789a0ad640 1 -- 172.21.15.53:0/3875076373 shutdown_connections 2023-10-21T16:03:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.680+0000 7f789a0ad640 1 -- 172.21.15.53:0/3875076373 wait complete. 2023-10-21T16:03:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.682+0000 7f789a0ad640 1 Processor -- start 2023-10-21T16:03:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.682+0000 7f789a0ad640 1 -- start start 2023-10-21T16:03:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c005180 0x7f78940b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7894156d10 0x7f7894155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b7de0 con 0x7f78941e6ce0 2023-10-21T16:03:56.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941e5cf0 con 0x7f7894156d10 2023-10-21T16:03:56.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f789a0ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7894156440 con 0x7f787c005180 2023-10-21T16:03:56.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51188/0 (socket says 172.21.15.53:51188) 2023-10-21T16:03:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 -- 172.21.15.53:0/2183874748 learned_addr learned my addr 172.21.15.53:0/2183874748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 -- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c005180 msgr2=0x7f78940b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c005180 0x7f78940b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 -- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7894156d10 msgr2=0x7f7894155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:56.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7894156d10 0x7f7894155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:56.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.683+0000 7f78988aa640 1 -- 172.21.15.53:0/2183874748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894155410 con 0x7f78941e6ce0 2023-10-21T16:03:56.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f78988aa640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f788800e4e0 tx=0x7f788800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:56.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880052e0 con 0x7f78941e6ce0 2023-10-21T16:03:56.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7888005440 con 0x7f78941e6ce0 2023-10-21T16:03:56.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940b7b00 con 0x7f78941e6ce0 2023-10-21T16:03:56.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888003810 con 0x7f78941e6ce0 2023-10-21T16:03:56.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.684+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78941e5980 con 0x7f78941e6ce0 2023-10-21T16:03:56.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.685+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78941e5b40 con 0x7f78941e6ce0 2023-10-21T16:03:56.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.686+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f7888003970 con 0x7f78941e6ce0 2023-10-21T16:03:56.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.687+0000 7f7891ffb640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 0x7f788006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:56.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.687+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f78880897c0 con 0x7f78941e6ce0 2023-10-21T16:03:56.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.688+0000 7f78990ab640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 0x7f788006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:56.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.689+0000 7f78990ab640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 0x7f788006d5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7884009f80 tx=0x7f7884009750 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:56.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.692+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7888054400 con 0x7f78941e6ce0 2023-10-21T16:03:56.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:56.906+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f78940cdd10 con 0x7f788006b0f0 2023-10-21T16:03:57.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.058+0000 7f7891ffb640 1 -- 172.21.15.53:0/2183874748 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f788400f040 con 0x7f788006b0f0 2023-10-21T16:03:57.060 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-21T16:03:57.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 msgr2=0x7f788006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 0x7f788006d5b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7884009f80 tx=0x7f7884009750 comp rx=0 tx=0).stop 2023-10-21T16:03:57.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 msgr2=0x7f7894154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f788800e4e0 tx=0x7f788800e9a0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 shutdown_connections 2023-10-21T16:03:57.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f788006b0f0 0x7f788006d5b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7894156d10 0x7f7894155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78941e6ce0 0x7f7894154ed0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 --2- 172.21.15.53:0/2183874748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f787c005180 0x7f78940b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 >> 172.21.15.53:0/2183874748 conn(0x7f789409bc40 msgr2=0x7f78940b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:57.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 shutdown_connections 2023-10-21T16:03:57.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.060+0000 7f789a0ad640 1 -- 172.21.15.53:0/2183874748 wait complete. 2023-10-21T16:03:57.085 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-21T16:03:57.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:03:57.086 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-21T16:03:57.144 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2023-10-21T16:03:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.365+0000 7f1e166bb640 1 Processor -- start 2023-10-21T16:03:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.365+0000 7f1e166bb640 1 -- start start 2023-10-21T16:03:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1e1015a820 0x7f1e100aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 0x7f1e100b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e1015ac00 0x7f1e100c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100cab60 con 0x7f1e1015ac00 2023-10-21T16:03:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100cb960 con 0x7f1e1015a820 2023-10-21T16:03:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e166bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100cbdf0 con 0x7f1e100ca680 2023-10-21T16:03:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e15eba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e1015ac00 0x7f1e100c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e15eba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e1015ac00 0x7f1e100c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51196/0 (socket says 172.21.15.53:51196) 2023-10-21T16:03:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e15eba640 1 -- 172.21.15.53:0/3029752999 learned_addr learned my addr 172.21.15.53:0/3029752999 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e156b9640 1 --2- 172.21.15.53:0/3029752999 >> v2:172.21.15.124:3300/0 conn(0x7f1e1015a820 0x7f1e100aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.367+0000 7f1e14eb8640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 0x7f1e100b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e15eba640 1 -- 172.21.15.53:0/3029752999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 msgr2=0x7f1e100b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e15eba640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 0x7f1e100b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e15eba640 1 -- 172.21.15.53:0/3029752999 >> v2:172.21.15.124:3300/0 conn(0x7f1e1015a820 msgr2=0x7f1e100aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e15eba640 1 --2- 172.21.15.53:0/3029752999 >> v2:172.21.15.124:3300/0 conn(0x7f1e1015a820 0x7f1e100aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e15eba640 1 -- 172.21.15.53:0/3029752999 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100cb030 con 0x7f1e1015ac00 2023-10-21T16:03:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.368+0000 7f1e14eb8640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 0x7f1e100b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:03:57.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.369+0000 7f1e15eba640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e1015ac00 0x7f1e100c9100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1e1006da70 tx=0x7f1e0802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fda4829c831c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.370+0000 7f1e07fff640 1 -- 172.21.15.53:0/3029752999 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080317d0 con 0x7f1e1015ac00 2023-10-21T16:03:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.370+0000 7f1e07fff640 1 -- 172.21.15.53:0/3029752999 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08031930 con 0x7f1e1015ac00 2023-10-21T16:03:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.370+0000 7f1e07fff640 1 -- 172.21.15.53:0/3029752999 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08031cc0 con 0x7f1e1015ac00 2023-10-21T16:03:57.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 -- 172.21.15.53:0/3029752999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e1015ac00 msgr2=0x7f1e100c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e1015ac00 0x7f1e100c9100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1e1006da70 tx=0x7f1e0802b870 comp rx=0 tx=0).stop 2023-10-21T16:03:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 -- 172.21.15.53:0/3029752999 shutdown_connections 2023-10-21T16:03:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e100ca680 0x7f1e100b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3029752999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e1015ac00 0x7f1e100c9100 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3029752999 >> v2:172.21.15.124:3300/0 conn(0x7f1e1015a820 0x7f1e100aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 -- 172.21.15.53:0/3029752999 >> 172.21.15.53:0/3029752999 conn(0x7f1e1006c050 msgr2=0x7f1e1006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:57.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 -- 172.21.15.53:0/3029752999 shutdown_connections 2023-10-21T16:03:57.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.371+0000 7f1e166bb640 1 -- 172.21.15.53:0/3029752999 wait complete. 2023-10-21T16:03:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.373+0000 7f1e166bb640 1 Processor -- start 2023-10-21T16:03:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.373+0000 7f1e166bb640 1 -- start start 2023-10-21T16:03:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e1015a820 0x7f1e100cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 0x7f1e101e4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100ad940 con 0x7f1e100ca680 2023-10-21T16:03:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100aba70 con 0x7f1e1015ac00 2023-10-21T16:03:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e166bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e100abd50 con 0x7f1e1015a820 2023-10-21T16:03:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51202/0 (socket says 172.21.15.53:51202) 2023-10-21T16:03:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 -- 172.21.15.53:0/3584374921 learned_addr learned my addr 172.21.15.53:0/3584374921 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e15eba640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 0x7f1e101e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 -- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e1015a820 msgr2=0x7f1e100cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e1015a820 0x7f1e100cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.374+0000 7f1e156b9640 1 -- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 msgr2=0x7f1e101e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e156b9640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 0x7f1e101e4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e156b9640 1 -- 172.21.15.53:0/3584374921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080094d0 con 0x7f1e100ca680 2023-10-21T16:03:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e15eba640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 0x7f1e101e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e156b9640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00efc0 tx=0x7f1e0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c014480 con 0x7f1e100ca680 2023-10-21T16:03:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00f040 con 0x7f1e100ca680 2023-10-21T16:03:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c014800 con 0x7f1e100ca680 2023-10-21T16:03:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e100ac030 con 0x7f1e100ca680 2023-10-21T16:03:57.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.375+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e100ac5f0 con 0x7f1e100ca680 2023-10-21T16:03:57.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.377+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e10074150 con 0x7f1e100ca680 2023-10-21T16:03:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.377+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f1e0c014960 con 0x7f1e100ca680 2023-10-21T16:03:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.379+0000 7f1e05ffb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 0x7f1df406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.379+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f1e0c08a850 con 0x7f1e100ca680 2023-10-21T16:03:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.380+0000 7f1e14eb8640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 0x7f1df406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.381+0000 7f1e14eb8640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 0x7f1df406d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:57.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.383+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e0c0553b0 con 0x7f1e100ca680 2023-10-21T16:03:57.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.599+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e100c6f50 con 0x7f1df406b0a0 2023-10-21T16:03:57.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.754+0000 7f1e05ffb640 1 -- 172.21.15.53:0/3584374921 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1e100c6f50 con 0x7f1df406b0a0 2023-10-21T16:03:57.757 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-21T16:03:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 msgr2=0x7f1df406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 0x7f1df406d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00008040 comp rx=0 tx=0).stop 2023-10-21T16:03:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 msgr2=0x7f1e100b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:57.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00efc0 tx=0x7f1e0c00c490 comp rx=0 tx=0).stop 2023-10-21T16:03:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 shutdown_connections 2023-10-21T16:03:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1df406b0a0 0x7f1df406d560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e1015ac00 0x7f1e101e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e1015a820 0x7f1e100cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 --2- 172.21.15.53:0/3584374921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e100ca680 0x7f1e100b0190 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 >> 172.21.15.53:0/3584374921 conn(0x7f1e1006c050 msgr2=0x7f1e100b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 shutdown_connections 2023-10-21T16:03:57.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:57.757+0000 7f1e166bb640 1 -- 172.21.15.53:0/3584374921 wait complete. 2023-10-21T16:03:57.781 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-21T16:03:57.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2023-10-21T16:03:58.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.012+0000 7f30f359e640 1 Processor -- start 2023-10-21T16:03:58.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.013+0000 7f30f359e640 1 -- start start 2023-10-21T16:03:58.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.014+0000 7f30f359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f415cae0 0x7f30f40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.014+0000 7f30f359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30f40c8b40 0x7f30f40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.014+0000 7f30f359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f40c94e0 con 0x7f30f40aff70 2023-10-21T16:03:58.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.014+0000 7f30f359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f40ccbc0 con 0x7f30f40c8b40 2023-10-21T16:03:58.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.014+0000 7f30f359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40ce1f0 con 0x7f30f415cae0 2023-10-21T16:03:58.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30ebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30f40c8b40 0x7f30f40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30f2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f30f40aff70 0x7f30f4070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48200/0 (socket says 172.21.15.53:48200) 2023-10-21T16:03:58.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30f2d9d640 1 -- 172.21.15.53:0/4196670364 learned_addr learned my addr 172.21.15.53:0/4196670364 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:58.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30f259c640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f415cae0 0x7f30f40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30f1d9b640 1 -- 172.21.15.53:0/4196670364 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795524109 0 0) 0x7f30ec001490 con 0x7f30f40aff70 2023-10-21T16:03:58.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30f1d9b640 1 -- 172.21.15.53:0/4196670364 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e0003520 con 0x7f30f40aff70 2023-10-21T16:03:58.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30ebfff640 1 -- 172.21.15.53:0/4196670364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f415cae0 msgr2=0x7f30f40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:58.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30ebfff640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f415cae0 0x7f30f40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30ebfff640 1 -- 172.21.15.53:0/4196670364 >> v1:172.21.15.53:6789/0 conn(0x7f30f40aff70 legacy=0x7f30f4070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.015+0000 7f30ebfff640 1 -- 172.21.15.53:0/4196670364 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40cd590 con 0x7f30f40c8b40 2023-10-21T16:03:58.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.017+0000 7f30ebfff640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40c8b40 0x7f30f40c8fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0097a0 tx=0x7f30dc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e63ccfedaebf97 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.017+0000 7f30f1d9b640 1 -- 172.21.15.53:0/4196670364 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc02d040 con 0x7f30f40c8b40 2023-10-21T16:03:58.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.017+0000 7f30f1d9b640 1 -- 172.21.15.53:0/4196670364 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc031810 con 0x7f30f40c8b40 2023-10-21T16:03:58.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.017+0000 7f30f1d9b640 1 -- 172.21.15.53:0/4196670364 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc031b80 con 0x7f30f40c8b40 2023-10-21T16:03:58.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 -- 172.21.15.53:0/4196670364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40c8b40 msgr2=0x7f30f40c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40c8b40 0x7f30f40c8fa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0097a0 tx=0x7f30dc02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 -- 172.21.15.53:0/4196670364 shutdown_connections 2023-10-21T16:03:58.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40c8b40 0x7f30f40c8fa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 --2- 172.21.15.53:0/4196670364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f415cae0 0x7f30f40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 -- 172.21.15.53:0/4196670364 >> 172.21.15.53:0/4196670364 conn(0x7f30f406c050 msgr2=0x7f30f406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:58.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 -- 172.21.15.53:0/4196670364 shutdown_connections 2023-10-21T16:03:58.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.018+0000 7f30f359e640 1 -- 172.21.15.53:0/4196670364 wait complete. 2023-10-21T16:03:58.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.020+0000 7f30f359e640 1 Processor -- start 2023-10-21T16:03:58.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.020+0000 7f30f359e640 1 -- start start 2023-10-21T16:03:58.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f40c8b40 0x7f30f41f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30f415cae0 0x7f30f41f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f415c120 con 0x7f30f40aff70 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40c7e50 con 0x7f30f415cae0 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40c7890 con 0x7f30f40c8b40 2023-10-21T16:03:58.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51208/0 (socket says 172.21.15.53:51208) 2023-10-21T16:03:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.021+0000 7f30f259c640 1 -- 172.21.15.53:0/2678621032 learned_addr learned my addr 172.21.15.53:0/2678621032 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 -- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f40c8b40 msgr2=0x7f30f41f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f2d9d640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30f415cae0 0x7f30f41f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30ebfff640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f40c8b40 0x7f30f41f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f40c8b40 0x7f30f41f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 -- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30f415cae0 msgr2=0x7f30f41f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30f415cae0 0x7f30f41f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 -- 172.21.15.53:0/2678621032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc009400 con 0x7f30f40aff70 2023-10-21T16:03:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f259c640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30e400d5f0 tx=0x7f30e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e40125f0 con 0x7f30f40aff70 2023-10-21T16:03:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e4012750 con 0x7f30f40aff70 2023-10-21T16:03:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4012b00 con 0x7f30f40aff70 2023-10-21T16:03:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.022+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f40c7b70 con 0x7f30f40aff70 2023-10-21T16:03:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.023+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40b1490 con 0x7f30f40aff70 2023-10-21T16:03:58.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.024+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f40b1620 con 0x7f30f40aff70 2023-10-21T16:03:58.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.025+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f30e4012c60 con 0x7f30f40aff70 2023-10-21T16:03:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.026+0000 7f30eaffd640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 0x7f30d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.026+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f30e408a0d0 con 0x7f30f40aff70 2023-10-21T16:03:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.027+0000 7f30ebfff640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 0x7f30d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.028+0000 7f30ebfff640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 0x7f30d406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30dc036000 tx=0x7f30dc03c040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.030+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30e4054c90 con 0x7f30f40aff70 2023-10-21T16:03:58.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.246+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f30f40cc570 con 0x7f30d406b070 2023-10-21T16:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.377+0000 7f30eaffd640 1 -- 172.21.15.53:0/2678621032 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f30dc031b00 con 0x7f30d406b070 2023-10-21T16:03:58.380 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-21T16:03:58.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 msgr2=0x7f30d406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 0x7f30d406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30dc036000 tx=0x7f30dc03c040 comp rx=0 tx=0).stop 2023-10-21T16:03:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 msgr2=0x7f30f40b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30e400d5f0 tx=0x7f30e400dab0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 shutdown_connections 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f30d406b070 0x7f30d406d530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30f415cae0 0x7f30f41f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30f40c8b40 0x7f30f41f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 --2- 172.21.15.53:0/2678621032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30f40aff70 0x7f30f40b3a80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 >> 172.21.15.53:0/2678621032 conn(0x7f30f406c050 msgr2=0x7f30f40caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:58.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 shutdown_connections 2023-10-21T16:03:58.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.382+0000 7f30f359e640 1 -- 172.21.15.53:0/2678621032 wait complete. 2023-10-21T16:03:58.405 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-21T16:03:58.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2023-10-21T16:03:58.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 Processor -- start 2023-10-21T16:03:58.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 -- start start 2023-10-21T16:03:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7c01599d0 0x7fa7c00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa7c00cb5e0 0x7fa7c00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c0159db0 con 0x7fa7c00b3c50 2023-10-21T16:03:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c0071fe0 con 0x7fa7c01599d0 2023-10-21T16:03:58.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.640+0000 7fa7c7143640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c00b44c0 con 0x7fa7c00cb5e0 2023-10-21T16:03:58.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7c01599d0 0x7fa7c00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7c01599d0 0x7fa7c00cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51222/0 (socket says 172.21.15.53:51222) 2023-10-21T16:03:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 -- 172.21.15.53:0/412265879 learned_addr learned my addr 172.21.15.53:0/412265879 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6942640 1 --2- 172.21.15.53:0/412265879 >> v2:172.21.15.124:3300/0 conn(0x7fa7c00cb5e0 0x7fa7c00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 -- 172.21.15.53:0/412265879 >> v2:172.21.15.124:3300/0 conn(0x7fa7c00cb5e0 msgr2=0x7fa7c00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:58.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 --2- 172.21.15.53:0/412265879 >> v2:172.21.15.124:3300/0 conn(0x7fa7c00cb5e0 0x7fa7c00cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 -- 172.21.15.53:0/412265879 >> v1:172.21.15.124:6789/0 conn(0x7fa7c00b3c50 legacy=0x7fa7c00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.641+0000 7fa7c6141640 1 -- 172.21.15.53:0/412265879 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00713d0 con 0x7fa7c01599d0 2023-10-21T16:03:58.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c6141640 1 --2- 172.21.15.53:0/412265879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c01599d0 0x7fa7c00cdcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7c006ffd0 tx=0x7fa7bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763a77dc983e3cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c513f640 1 -- 172.21.15.53:0/412265879 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc02ca80 con 0x7fa7c01599d0 2023-10-21T16:03:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c513f640 1 -- 172.21.15.53:0/412265879 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc02cbe0 con 0x7fa7c01599d0 2023-10-21T16:03:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c513f640 1 -- 172.21.15.53:0/412265879 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0343f0 con 0x7fa7c01599d0 2023-10-21T16:03:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 -- 172.21.15.53:0/412265879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c01599d0 msgr2=0x7fa7c00cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 --2- 172.21.15.53:0/412265879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c01599d0 0x7fa7c00cdcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7c006ffd0 tx=0x7fa7bc02bee0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 -- 172.21.15.53:0/412265879 shutdown_connections 2023-10-21T16:03:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 --2- 172.21.15.53:0/412265879 >> v2:172.21.15.124:3300/0 conn(0x7fa7c00cb5e0 0x7fa7c00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 --2- 172.21.15.53:0/412265879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c01599d0 0x7fa7c00cdcd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 -- 172.21.15.53:0/412265879 >> 172.21.15.53:0/412265879 conn(0x7fa7c006c050 msgr2=0x7fa7c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:58.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 -- 172.21.15.53:0/412265879 shutdown_connections 2023-10-21T16:03:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.643+0000 7fa7c7143640 1 -- 172.21.15.53:0/412265879 wait complete. 2023-10-21T16:03:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.646+0000 7fa7c7143640 1 Processor -- start 2023-10-21T16:03:58.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.646+0000 7fa7c7143640 1 -- start start 2023-10-21T16:03:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7c00cb5e0 0x7fa7c01e6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7c01599d0 0x7fa7c01e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01e3390 con 0x7fa7c00b3c50 2023-10-21T16:03:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01e3670 con 0x7fa7c00cb5e0 2023-10-21T16:03:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c7143640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c01e3950 con 0x7fa7c01599d0 2023-10-21T16:03:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51228/0 (socket says 172.21.15.53:51228) 2023-10-21T16:03:58.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 -- 172.21.15.53:0/3801676233 learned_addr learned my addr 172.21.15.53:0/3801676233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c5940640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7c00cb5e0 0x7fa7c01e6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6942640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7c01599d0 0x7fa7c01e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 -- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7c01599d0 msgr2=0x7fa7c01e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7c01599d0 0x7fa7c01e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 -- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7c00cb5e0 msgr2=0x7fa7c01e6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7c00cb5e0 0x7fa7c01e6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.647+0000 7fa7c6141640 1 -- 172.21.15.53:0/3801676233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009b30 con 0x7fa7c00b3c50 2023-10-21T16:03:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7c6141640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc036040 tx=0x7fa7bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc009510 con 0x7fa7c00b3c50 2023-10-21T16:03:58.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc03e450 con 0x7fa7c00b3c50 2023-10-21T16:03:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc03e7e0 con 0x7fa7c00b3c50 2023-10-21T16:03:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c01e3c30 con 0x7fa7c00b3c50 2023-10-21T16:03:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.648+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c01571a0 con 0x7fa7c00b3c50 2023-10-21T16:03:58.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.649+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c0157330 con 0x7fa7c00b3c50 2023-10-21T16:03:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.650+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fa7bc02cd50 con 0x7fa7c00b3c50 2023-10-21T16:03:58.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.651+0000 7fa7b6ffd640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 0x7fa7a406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:58.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.651+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa7bc0ac680 con 0x7fa7c00b3c50 2023-10-21T16:03:58.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.652+0000 7fa7c5940640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 0x7fa7a406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:58.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.653+0000 7fa7c5940640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 0x7fa7a406d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7c01e4350 tx=0x7fa7b0008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:58.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.656+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7bc077260 con 0x7fa7c00b3c50 2023-10-21T16:03:58.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:58.873+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7c0071990 con 0x7fa7a406b070 2023-10-21T16:03:59.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.004+0000 7fa7b6ffd640 1 -- 172.21.15.53:0/3801676233 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa7b000a380 con 0x7fa7a406b070 2023-10-21T16:03:59.006 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-21T16:03:59.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 msgr2=0x7fa7a406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 0x7fa7a406d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7c01e4350 tx=0x7fa7b0008040 comp rx=0 tx=0).stop 2023-10-21T16:03:59.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 msgr2=0x7fa7c01e2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc036040 tx=0x7fa7bc004a40 comp rx=0 tx=0).stop 2023-10-21T16:03:59.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 shutdown_connections 2023-10-21T16:03:59.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa7a406b070 0x7fa7a406d530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7c01599d0 0x7fa7c01e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7c00cb5e0 0x7fa7c01e6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 --2- 172.21.15.53:0/3801676233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7c00b3c50 0x7fa7c01e2940 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.008+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 >> 172.21.15.53:0/3801676233 conn(0x7fa7c006c050 msgr2=0x7fa7c015c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:59.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.009+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 shutdown_connections 2023-10-21T16:03:59.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.009+0000 7fa7c7143640 1 -- 172.21.15.53:0/3801676233 wait complete. 2023-10-21T16:03:59.035 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-21T16:03:59.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2023-10-21T16:03:59.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.269+0000 7f10fffff640 1 Processor -- start 2023-10-21T16:03:59.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.269+0000 7f10fffff640 1 -- start start 2023-10-21T16:03:59.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10fffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11000b0680 0x7f110015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10fffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000ca680 0x7f110015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10fffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000cab60 con 0x7f110015a820 2023-10-21T16:03:59.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10fffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000cb960 con 0x7f11000b0680 2023-10-21T16:03:59.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10fffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000cbdf0 con 0x7f11000ca680 2023-10-21T16:03:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11000b0680 0x7f110015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10feffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f110015a820 0x7f11000aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48216/0 (socket says 172.21.15.53:48216) 2023-10-21T16:03:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f11000b0680 0x7f110015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51242/0 (socket says 172.21.15.53:51242) 2023-10-21T16:03:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.271+0000 7f10feffd640 1 -- 172.21.15.53:0/1605799096 learned_addr learned my addr 172.21.15.53:0/1605799096 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:03:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.272+0000 7f10ff7fe640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000ca680 0x7f110015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.272+0000 7f10f7fff640 1 -- 172.21.15.53:0/1605799096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000ca680 msgr2=0x7f110015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.272+0000 7f10f7fff640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000ca680 0x7f110015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.272+0000 7f10f7fff640 1 -- 172.21.15.53:0/1605799096 >> v1:172.21.15.53:6789/0 conn(0x7f110015a820 legacy=0x7f11000aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.272+0000 7f10f7fff640 1 -- 172.21.15.53:0/1605799096 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000cb030 con 0x7f11000b0680 2023-10-21T16:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.273+0000 7f10f7fff640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11000b0680 0x7f110015e540 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10e80097a0 tx=0x7f10e802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3628e20896da50 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.273+0000 7f10fe7fc640 1 -- 172.21.15.53:0/1605799096 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8031780 con 0x7f11000b0680 2023-10-21T16:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.273+0000 7f10fe7fc640 1 -- 172.21.15.53:0/1605799096 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e80318e0 con 0x7f11000b0680 2023-10-21T16:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.273+0000 7f10fe7fc640 1 -- 172.21.15.53:0/1605799096 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8031c70 con 0x7f11000b0680 2023-10-21T16:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 -- 172.21.15.53:0/1605799096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11000b0680 msgr2=0x7f110015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11000b0680 0x7f110015e540 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f10e80097a0 tx=0x7f10e802b820 comp rx=0 tx=0).stop 2023-10-21T16:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 -- 172.21.15.53:0/1605799096 shutdown_connections 2023-10-21T16:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000ca680 0x7f110015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 --2- 172.21.15.53:0/1605799096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11000b0680 0x7f110015e540 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 -- 172.21.15.53:0/1605799096 >> 172.21.15.53:0/1605799096 conn(0x7f110006c050 msgr2=0x7f110006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 -- 172.21.15.53:0/1605799096 shutdown_connections 2023-10-21T16:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.274+0000 7f10fffff640 1 -- 172.21.15.53:0/1605799096 wait complete. 2023-10-21T16:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.276+0000 7f10fffff640 1 Processor -- start 2023-10-21T16:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.276+0000 7f10fffff640 1 -- start start 2023-10-21T16:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000b0680 0x7f11001f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 0x7f11001f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000acaa0 con 0x7f110015a820 2023-10-21T16:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000acd30 con 0x7f11000ca680 2023-10-21T16:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10fffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000ad010 con 0x7f11000b0680 2023-10-21T16:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51250/0 (socket says 172.21.15.53:51250) 2023-10-21T16:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.277+0000 7f10ff7fe640 1 -- 172.21.15.53:0/4250929407 learned_addr learned my addr 172.21.15.53:0/4250929407 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10feffd640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000b0680 0x7f11001f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10f7fff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 0x7f11001f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 -- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000b0680 msgr2=0x7f11001f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000b0680 0x7f11001f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 -- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 msgr2=0x7f11001f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 0x7f11001f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 -- 172.21.15.53:0/4250929407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009400 con 0x7f110015a820 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10f7fff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 0x7f11001f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10ff7fe640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f10f8007e30 tx=0x7f10f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8014620 con 0x7f110015a820 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f800ee90 con 0x7f110015a820 2023-10-21T16:03:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11001f2690 con 0x7f110015a820 2023-10-21T16:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.278+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f80149c0 con 0x7f110015a820 2023-10-21T16:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.279+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11001f2c50 con 0x7f110015a820 2023-10-21T16:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.280+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11001f2e40 con 0x7f110015a820 2023-10-21T16:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.281+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f10f8014b20 con 0x7f110015a820 2023-10-21T16:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.281+0000 7f10f77fe640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 0x7f10e006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.282+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f10f808a6b0 con 0x7f110015a820 2023-10-21T16:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.282+0000 7f10feffd640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 0x7f10e006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.283+0000 7f10feffd640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 0x7f10e006d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005880 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.286+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10f80551e0 con 0x7f110015a820 2023-10-21T16:03:59.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.504+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f11000cb5f0 con 0x7f10e006b070 2023-10-21T16:03:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.627+0000 7f10f77fe640 1 -- 172.21.15.53:0/4250929407 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f10f00012c0 con 0x7f10e006b070 2023-10-21T16:03:59.629 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-21T16:03:59.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 msgr2=0x7f10e006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 0x7f10e006d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005880 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2023-10-21T16:03:59.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 msgr2=0x7f11000ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f10f8007e30 tx=0x7f10f800e4b0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 shutdown_connections 2023-10-21T16:03:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f10e006b070 0x7f10e006d530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015a820 0x7f11000ac560 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11000ca680 0x7f11001f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 --2- 172.21.15.53:0/4250929407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11000b0680 0x7f11001f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.630+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 >> 172.21.15.53:0/4250929407 conn(0x7f110006c050 msgr2=0x7f1100157010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.631+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 shutdown_connections 2023-10-21T16:03:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.631+0000 7f10fffff640 1 -- 172.21.15.53:0/4250929407 wait complete. 2023-10-21T16:03:59.654 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-21T16:03:59.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2023-10-21T16:03:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.887+0000 7ff44b59e640 1 Processor -- start 2023-10-21T16:03:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.888+0000 7ff44b59e640 1 -- start start 2023-10-21T16:03:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.889+0000 7ff44b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff44c15a820 0x7ff44c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.889+0000 7ff44b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.889+0000 7ff44b59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0cab60 con 0x7ff44c15ac00 2023-10-21T16:03:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.889+0000 7ff44b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0cb960 con 0x7ff44c15a820 2023-10-21T16:03:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.889+0000 7ff44b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0cbdf0 con 0x7ff44c0ca680 2023-10-21T16:03:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.890+0000 7ff44a59c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff44c15a820 0x7ff44c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.890+0000 7ff443fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.890+0000 7ff443fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:59762/0 (socket says 172.21.15.53:59762) 2023-10-21T16:03:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.890+0000 7ff443fff640 1 -- 172.21.15.53:0/393991388 learned_addr learned my addr 172.21.15.53:0/393991388 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.891+0000 7ff443fff640 1 -- 172.21.15.53:0/393991388 >> v2:172.21.15.124:3300/0 conn(0x7ff44c15a820 msgr2=0x7ff44c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:03:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.891+0000 7ff443fff640 1 --2- 172.21.15.53:0/393991388 >> v2:172.21.15.124:3300/0 conn(0x7ff44c15a820 0x7ff44c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.891+0000 7ff443fff640 1 -- 172.21.15.53:0/393991388 >> v1:172.21.15.124:6789/0 conn(0x7ff44c15ac00 legacy=0x7ff44c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.891+0000 7ff443fff640 1 -- 172.21.15.53:0/393991388 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0cb030 con 0x7ff44c0ca680 2023-10-21T16:03:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.892+0000 7ff443fff640 1 --2- 172.21.15.53:0/393991388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff434009870 tx=0x7ff43402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9451f866d3338f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.893+0000 7ff449d9b640 1 -- 172.21.15.53:0/393991388 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434031640 con 0x7ff44c0ca680 2023-10-21T16:03:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.893+0000 7ff449d9b640 1 -- 172.21.15.53:0/393991388 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4340317a0 con 0x7ff44c0ca680 2023-10-21T16:03:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.893+0000 7ff449d9b640 1 -- 172.21.15.53:0/393991388 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434031b30 con 0x7ff44c0ca680 2023-10-21T16:03:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 -- 172.21.15.53:0/393991388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 msgr2=0x7ff44c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 --2- 172.21.15.53:0/393991388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff434009870 tx=0x7ff43402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 -- 172.21.15.53:0/393991388 shutdown_connections 2023-10-21T16:03:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 --2- 172.21.15.53:0/393991388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c0ca680 0x7ff44c0b04b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 --2- 172.21.15.53:0/393991388 >> v2:172.21.15.124:3300/0 conn(0x7ff44c15a820 0x7ff44c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 -- 172.21.15.53:0/393991388 >> 172.21.15.53:0/393991388 conn(0x7ff44c06c050 msgr2=0x7ff44c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:03:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 -- 172.21.15.53:0/393991388 shutdown_connections 2023-10-21T16:03:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.894+0000 7ff44b59e640 1 -- 172.21.15.53:0/393991388 wait complete. 2023-10-21T16:03:59.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.896+0000 7ff44b59e640 1 Processor -- start 2023-10-21T16:03:59.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.896+0000 7ff44b59e640 1 -- start start 2023-10-21T16:03:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c15a820 0x7ff44c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff44c15ac00 0x7ff44c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0c55c0 con 0x7ff44c0ca680 2023-10-21T16:03:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c15c1f0 con 0x7ff44c15ac00 2023-10-21T16:03:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c15c4d0 con 0x7ff44c15a820 2023-10-21T16:03:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51254/0 (socket says 172.21.15.53:51254) 2023-10-21T16:03:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff44a59c640 1 -- 172.21.15.53:0/200432899 learned_addr learned my addr 172.21.15.53:0/200432899 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:03:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.897+0000 7ff443fff640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c15a820 0x7ff44c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44ad9d640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff44c15ac00 0x7ff44c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 -- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c15a820 msgr2=0x7ff44c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c15a820 0x7ff44c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 -- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff44c15ac00 msgr2=0x7ff44c0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:03:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff44c15ac00 0x7ff44c0ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:03:59.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 -- 172.21.15.53:0/200432899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340094d0 con 0x7ff44c0ca680 2023-10-21T16:03:59.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff44a59c640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00e9b0 tx=0x7ff43c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c009670 con 0x7ff44c0ca680 2023-10-21T16:03:59.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.898+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c00c9a0 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.899+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c15c7b0 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.899+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00cd50 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.899+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0cd4d0 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.900+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c074150 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.901+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7ff43c013470 con 0x7ff44c0ca680 2023-10-21T16:03:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.902+0000 7ff442ffd640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 0x7ff42c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:03:59.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.902+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7ff43c089db0 con 0x7ff44c0ca680 2023-10-21T16:03:59.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.903+0000 7ff443fff640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 0x7ff42c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:03:59.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.904+0000 7ff443fff640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 0x7ff42c06d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff434036000 tx=0x7ff43403c040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:03:59.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:03:59.907+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff43c054a10 con 0x7ff44c0ca680 2023-10-21T16:04:00.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:04:00.081+0000 7f405d62c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:04:00.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:04:00.081+0000 7f405d62c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:04:00.083 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:04:00.081+0000 7f405d62c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:04:00.083 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:04:00.081+0000 7f405d62c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:04:00.083 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:04:00.081+0000 7f405d62c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:04:00.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.146+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ff44c0cb5f0 con 0x7ff42c06b070 2023-10-21T16:04:00.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.445+0000 7ff442ffd640 1 -- 172.21.15.53:0/200432899 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff434031990 con 0x7ff42c06b070 2023-10-21T16:04:00.448 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-21T16:04:00.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 msgr2=0x7ff42c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 0x7ff42c06d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff434036000 tx=0x7ff43403c040 comp rx=0 tx=0).stop 2023-10-21T16:04:00.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 msgr2=0x7ff44c1526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00e9b0 tx=0x7ff43c00ee70 comp rx=0 tx=0).stop 2023-10-21T16:04:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 shutdown_connections 2023-10-21T16:04:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7ff42c06b070 0x7ff42c06d530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff44c15ac00 0x7ff44c0ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff44c15a820 0x7ff44c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 --2- 172.21.15.53:0/200432899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff44c0ca680 0x7ff44c1526f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 >> 172.21.15.53:0/200432899 conn(0x7ff44c06c050 msgr2=0x7ff44c0ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:00.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.449+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 shutdown_connections 2023-10-21T16:04:00.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.450+0000 7ff44b59e640 1 -- 172.21.15.53:0/200432899 wait complete. 2023-10-21T16:04:00.475 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-21T16:04:00.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2023-10-21T16:04:00.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.706+0000 7fa8cde7c640 1 Processor -- start 2023-10-21T16:04:00.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.706+0000 7fa8cde7c640 1 -- start start 2023-10-21T16:04:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8cde7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa8c80b21f0 0x7fa8c80c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8cde7c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c806c3a0 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8cde7c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c806c640 con 0x7fa8c80702b0 2023-10-21T16:04:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8cde7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c806c920 con 0x7fa8c80b21f0 2023-10-21T16:04:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8c7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa8c80ca1a0 0x7fa8c806d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48224/0 (socket says 172.21.15.53:48224) 2023-10-21T16:04:00.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8c7fff640 1 -- 172.21.15.53:0/2105349509 learned_addr learned my addr 172.21.15.53:0/2105349509 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:04:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.708+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/2105349509 >> v2:172.21.15.124:3300/0 conn(0x7fa8c80b21f0 0x7fa8c80c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362822099 0 0) 0x7fa8c806c3a0 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b0003440 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945964172 0 0) 0x7fa8b0003440 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 >> v2:172.21.15.124:3300/0 conn(0x7fa8c80b21f0 msgr2=0x7fa8c80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:00.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 --2- 172.21.15.53:0/2105349509 >> v2:172.21.15.124:3300/0 conn(0x7fa8c80b21f0 0x7fa8c80c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 >> v1:172.21.15.124:6789/0 conn(0x7fa8c80702b0 legacy=0x7fa8c8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c806cc00 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa8c00019f0 con 0x7fa8c80ca1a0 2023-10-21T16:04:00.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 >> v1:172.21.15.53:6789/0 conn(0x7fa8c80ca1a0 legacy=0x7fa8c806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b0003a60 0x7fa8b0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8b000ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b000b480 0x7fa8b000d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c806c920 con 0x7fa8b0008850 2023-10-21T16:04:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b000e220 con 0x7fa8b000b480 2023-10-21T16:04:00.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.709+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b000e3e0 con 0x7fa8b0003a60 2023-10-21T16:04:00.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8b000ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c77fe640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b0003a60 0x7fa8b0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c7fff640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b000b480 0x7fa8b000d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 -- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b0003a60 msgr2=0x7fa8b0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b0003a60 0x7fa8b0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 -- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b000b480 msgr2=0x7fa8b000d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b000b480 0x7fa8b000d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c6ffd640 1 -- 172.21.15.53:0/2105349509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b0003440 con 0x7fa8b0008850 2023-10-21T16:04:00.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c7fff640 1 -- 172.21.15.53:0/2105349509 reap_dead start 2023-10-21T16:04:00.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.710+0000 7fa8c77fe640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b0003a60 0x7fa8b0003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:00.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.711+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8b000ac80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009ac0 tx=0x7fa8b00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f36afc101c0ffa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02c6b0 con 0x7fa8b0008850 2023-10-21T16:04:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc02c810 con 0x7fa8b0008850 2023-10-21T16:04:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8c67fc640 1 -- 172.21.15.53:0/2105349509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02cba0 con 0x7fa8b0008850 2023-10-21T16:04:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8cde7c640 1 -- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 msgr2=0x7fa8b000ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8b000ac80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009ac0 tx=0x7fa8b00036f0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8cde7c640 1 -- 172.21.15.53:0/2105349509 shutdown_connections 2023-10-21T16:04:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/2105349509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8b000ac80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.712+0000 7fa8cde7c640 1 -- 172.21.15.53:0/2105349509 >> 172.21.15.53:0/2105349509 conn(0x7fa8c806a5d0 msgr2=0x7fa8c80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.713+0000 7fa8cde7c640 1 -- 172.21.15.53:0/2105349509 shutdown_connections 2023-10-21T16:04:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.713+0000 7fa8cde7c640 1 -- 172.21.15.53:0/2105349509 wait complete. 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.715+0000 7fa8cde7c640 1 Processor -- start 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.715+0000 7fa8cde7c640 1 -- start start 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 0x7fa8c8146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 0x7fa8c8147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8145cb0 con 0x7fa8b0008850 2023-10-21T16:04:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8145f90 con 0x7fa8c815aae0 2023-10-21T16:04:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8cde7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8146270 con 0x7fa8c815a660 2023-10-21T16:04:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51280/0 (socket says 172.21.15.53:51280) 2023-10-21T16:04:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.716+0000 7fa8c77fe640 1 -- 172.21.15.53:0/3847995964 learned_addr learned my addr 172.21.15.53:0/3847995964 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 0x7fa8c8146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c7fff640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 0x7fa8c8147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 -- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 msgr2=0x7fa8c8146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 0x7fa8c8146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 -- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 msgr2=0x7fa8c8147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:00.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 0x7fa8c8147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:00.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 -- 172.21.15.53:0/3847995964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8146550 con 0x7fa8b0008850 2023-10-21T16:04:00.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 0x7fa8c8146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:00.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c7fff640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 0x7fa8c8147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:00.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.717+0000 7fa8c77fe640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d400 tx=0x7fa8b800d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.718+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800b070 con 0x7fa8b0008850 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.718+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80054e0 con 0x7fa8b0008850 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.718+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003830 con 0x7fa8b0008850 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.718+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c8146830 con 0x7fa8b0008850 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.718+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81458d0 con 0x7fa8b0008850 2023-10-21T16:04:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.719+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c804e160 con 0x7fa8b0008850 2023-10-21T16:04:00.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.720+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fa8b8003990 con 0x7fa8b0008850 2023-10-21T16:04:00.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.721+0000 7fa8affff640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 0x7fa8b406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:00.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.721+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa8b80894c0 con 0x7fa8b0008850 2023-10-21T16:04:00.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.722+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 0x7fa8b406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:00.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.723+0000 7fa8c6ffd640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 0x7fa8b406d560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc030000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:00.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.726+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8b8054030 con 0x7fa8b0008850 2023-10-21T16:04:00.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:00.961+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c8146100 con 0x7fa8b406b0a0 2023-10-21T16:04:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.000+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa8b8054400 con 0x7fa8b0008850 2023-10-21T16:04:01.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.714+0000 7fa8affff640 1 -- 172.21.15.53:0/3847995964 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa8c8146100 con 0x7fa8b406b0a0 2023-10-21T16:04:01.716 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-21T16:04:01.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 msgr2=0x7fa8b406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 0x7fa8b406d560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc030000 comp rx=0 tx=0).stop 2023-10-21T16:04:01.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 msgr2=0x7fa8c80c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800d400 tx=0x7fa8b800d8c0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 shutdown_connections 2023-10-21T16:04:01.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.717+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fa8b406b0a0 0x7fa8b406d560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8c815aae0 0x7fa8c8147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8c815a660 0x7fa8c8146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 --2- 172.21.15.53:0/3847995964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8b0008850 0x7fa8c80c4880 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 >> 172.21.15.53:0/3847995964 conn(0x7fa8c806a5d0 msgr2=0x7fa8c80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:01.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 shutdown_connections 2023-10-21T16:04:01.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.718+0000 7fa8cde7c640 1 -- 172.21.15.53:0/3847995964 wait complete. 2023-10-21T16:04:01.742 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-21T16:04:01.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2023-10-21T16:04:01.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.960+0000 7fa01d0bb640 1 Processor -- start 2023-10-21T16:04:01.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.960+0000 7fa01d0bb640 1 -- start start 2023-10-21T16:04:01.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.962+0000 7fa01d0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 0x7fa0180caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.962+0000 7fa01d0bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 0x7fa0180b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.962+0000 7fa01d0bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa018071140 con 0x7fa018160660 2023-10-21T16:04:01.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.962+0000 7fa01d0bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01806e150 con 0x7fa0180cb450 2023-10-21T16:04:01.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.962+0000 7fa01d0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cb8c0 con 0x7fa018070cc0 2023-10-21T16:04:01.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.963+0000 7fa01759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 0x7fa0180b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.963+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 0x7fa0180caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.963+0000 7fa01759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 0x7fa0180b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:54652/0 (socket says 172.21.15.53:54652) 2023-10-21T16:04:01.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.963+0000 7fa01759e640 1 -- 172.21.15.53:0/822104896 learned_addr learned my addr 172.21.15.53:0/822104896 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:01.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.964+0000 7fa00ffff640 1 -- 172.21.15.53:0/822104896 >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 msgr2=0x7fa0180b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:01.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.964+0000 7fa00ffff640 1 --2- 172.21.15.53:0/822104896 >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 0x7fa0180b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.964+0000 7fa00ffff640 1 -- 172.21.15.53:0/822104896 >> v1:172.21.15.124:6789/0 conn(0x7fa018160660 legacy=0x7fa0180c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.964+0000 7fa00ffff640 1 -- 172.21.15.53:0/822104896 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01806de70 con 0x7fa018070cc0 2023-10-21T16:04:01.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.965+0000 7fa00ffff640 1 --2- 172.21.15.53:0/822104896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 0x7fa0180caf10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0000097a0 tx=0x7fa00002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc537b6b288944a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:01.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.966+0000 7fa01659c640 1 -- 172.21.15.53:0/822104896 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00002d040 con 0x7fa018070cc0 2023-10-21T16:04:01.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.966+0000 7fa01659c640 1 -- 172.21.15.53:0/822104896 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000031810 con 0x7fa018070cc0 2023-10-21T16:04:01.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.966+0000 7fa01659c640 1 -- 172.21.15.53:0/822104896 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000031ba0 con 0x7fa018070cc0 2023-10-21T16:04:01.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.966+0000 7fa01d0bb640 1 -- 172.21.15.53:0/822104896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 msgr2=0x7fa0180caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.966+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/822104896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 0x7fa0180caf10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0000097a0 tx=0x7fa00002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 -- 172.21.15.53:0/822104896 shutdown_connections 2023-10-21T16:04:01.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/822104896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa018070cc0 0x7fa0180caf10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/822104896 >> v2:172.21.15.124:3300/0 conn(0x7fa0180cb450 0x7fa0180b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 -- 172.21.15.53:0/822104896 >> 172.21.15.53:0/822104896 conn(0x7fa01806a5d0 msgr2=0x7fa0180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:01.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 -- 172.21.15.53:0/822104896 shutdown_connections 2023-10-21T16:04:01.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.967+0000 7fa01d0bb640 1 -- 172.21.15.53:0/822104896 wait complete. 2023-10-21T16:04:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.968+0000 7fa01d0bb640 1 Processor -- start 2023-10-21T16:04:01.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.969+0000 7fa01d0bb640 1 -- start start 2023-10-21T16:04:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.969+0000 7fa01d0bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa018070cc0 0x7fa0180c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01d0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0180cb450 0x7fa0181f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01d0bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01d0bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c7b70 con 0x7fa018160660 2023-10-21T16:04:01.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01d0bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c7e50 con 0x7fa018070cc0 2023-10-21T16:04:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01d0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c8130 con 0x7fa0180cb450 2023-10-21T16:04:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51286/0 (socket says 172.21.15.53:51286) 2023-10-21T16:04:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 -- 172.21.15.53:0/2762871611 learned_addr learned my addr 172.21.15.53:0/2762871611 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:01.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa016d9d640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa018070cc0 0x7fa0180c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa00ffff640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0180cb450 0x7fa0181f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 -- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0180cb450 msgr2=0x7fa0181f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0180cb450 0x7fa0181f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 -- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa018070cc0 msgr2=0x7fa0180c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa018070cc0 0x7fa0180c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:01.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.970+0000 7fa01759e640 1 -- 172.21.15.53:0/2762871611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000009400 con 0x7fa018160660 2023-10-21T16:04:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa01759e640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa01000efc0 tx=0x7fa01000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cd20 con 0x7fa018160660 2023-10-21T16:04:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01000f040 con 0x7fa018160660 2023-10-21T16:04:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010003bd0 con 0x7fa018160660 2023-10-21T16:04:01.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0181486f0 con 0x7fa018160660 2023-10-21T16:04:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.971+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180c77b0 con 0x7fa018160660 2023-10-21T16:04:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.972+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa018148c10 con 0x7fa018160660 2023-10-21T16:04:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.973+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa010003d30 con 0x7fa018160660 2023-10-21T16:04:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.974+0000 7fa00f7fe640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 0x7f9ff806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.974+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa01008a490 con 0x7fa018160660 2023-10-21T16:04:01.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.975+0000 7fa016d9d640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 0x7f9ff806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.976+0000 7fa016d9d640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 0x7f9ff806d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa008000f60 tx=0x7fa008009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:01.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:01.979+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa018148c10 con 0x7fa018160660 2023-10-21T16:04:02.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.203+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0180c9110 con 0x7f9ff806b100 2023-10-21T16:04:02.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.331+0000 7fa00f7fe640 1 -- 172.21.15.53:0/2762871611 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa0080012c0 con 0x7f9ff806b100 2023-10-21T16:04:02.333 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-21T16:04:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 msgr2=0x7f9ff806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 0x7f9ff806d5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa008000f60 tx=0x7fa008009040 comp rx=0 tx=0).stop 2023-10-21T16:04:02.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 msgr2=0x7fa0181f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa01000efc0 tx=0x7fa01000c490 comp rx=0 tx=0).stop 2023-10-21T16:04:02.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 shutdown_connections 2023-10-21T16:04:02.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9ff806b100 0x7f9ff806d5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa018160660 0x7fa0181f62a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0180cb450 0x7fa0181f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 --2- 172.21.15.53:0/2762871611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa018070cc0 0x7fa0180c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 >> 172.21.15.53:0/2762871611 conn(0x7fa01806a5d0 msgr2=0x7fa0180cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.334+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 shutdown_connections 2023-10-21T16:04:02.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.335+0000 7fa01d0bb640 1 -- 172.21.15.53:0/2762871611 wait complete. 2023-10-21T16:04:02.358 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-21T16:04:02.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2023-10-21T16:04:02.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.586+0000 7f0598a15640 1 Processor -- start 2023-10-21T16:04:02.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.587+0000 7f0598a15640 1 -- start start 2023-10-21T16:04:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.588+0000 7f0598a15640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 0x7f05940b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.588+0000 7f0598a15640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059415b310 con 0x7f059406c3a0 2023-10-21T16:04:02.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.588+0000 7f0598a15640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059406d290 con 0x7f05940c9cf0 2023-10-21T16:04:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.588+0000 7f0598a15640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059415b110 con 0x7f059406db60 2023-10-21T16:04:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f059259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f059406c3a0 0x7f059406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48236/0 (socket says 172.21.15.53:48236) 2023-10-21T16:04:02.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f059259c640 1 -- 172.21.15.53:0/159272475 learned_addr learned my addr 172.21.15.53:0/159272475 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:04:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0592d9d640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 0x7f05940b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176628247 0 0) 0x7f0584001110 con 0x7f059406c3a0 2023-10-21T16:04:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0580003520 con 0x7f059406c3a0 2023-10-21T16:04:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2760757545 0 0) 0x7f0580003520 con 0x7f059406c3a0 2023-10-21T16:04:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 msgr2=0x7f05940b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 0x7f05940b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 >> v1:172.21.15.124:6789/0 conn(0x7f05940c9cf0 legacy=0x7f059415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05940c9860 con 0x7f059406c3a0 2023-10-21T16:04:02.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0584001800 con 0x7f059406c3a0 2023-10-21T16:04:02.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.589+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 >> v1:172.21.15.53:6789/0 conn(0x7f059406c3a0 legacy=0x7f059406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 0x7f05800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580003f70 0x7f05800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580004d00 0x7f0580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059415b110 con 0x7f0580004d00 2023-10-21T16:04:02.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058000bac0 con 0x7f0580003f70 2023-10-21T16:04:02.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058000bc80 con 0x7f059406db60 2023-10-21T16:04:02.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580004d00 0x7f0580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f058bfff640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580003f70 0x7f05800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 -- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 msgr2=0x7f05800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:02.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f059406db60 0x7f05800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 -- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580003f70 msgr2=0x7f05800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:02.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580003f70 0x7f05800047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.590+0000 7f0592d9d640 1 -- 172.21.15.53:0/159272475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580003520 con 0x7f0580004d00 2023-10-21T16:04:02.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.591+0000 7f0592d9d640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580004d00 0x7f0580005180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f058c009fc0 tx=0x7f05800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e53e71573b9faa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:02.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c02cc60 con 0x7f0580004d00 2023-10-21T16:04:02.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c00f040 con 0x7f0580004d00 2023-10-21T16:04:02.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0591d9b640 1 -- 172.21.15.53:0/159272475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c034440 con 0x7f0580004d00 2023-10-21T16:04:02.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0598a15640 1 -- 172.21.15.53:0/159272475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580004d00 msgr2=0x7f0580005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0598a15640 1 --2- 172.21.15.53:0/159272475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0580004d00 0x7f0580005180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f058c009fc0 tx=0x7f05800057f0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0592d9d640 1 -- 172.21.15.53:0/159272475 reap_dead start 2023-10-21T16:04:02.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0598a15640 1 -- 172.21.15.53:0/159272475 shutdown_connections 2023-10-21T16:04:02.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.592+0000 7f0598a15640 1 -- 172.21.15.53:0/159272475 >> 172.21.15.53:0/159272475 conn(0x7f059406a5d0 msgr2=0x7f05940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:02.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.593+0000 7f0598a15640 1 -- 172.21.15.53:0/159272475 shutdown_connections 2023-10-21T16:04:02.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.593+0000 7f0598a15640 1 -- 172.21.15.53:0/159272475 wait complete. 2023-10-21T16:04:02.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.594+0000 7f0598a15640 1 Processor -- start 2023-10-21T16:04:02.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 -- start start 2023-10-21T16:04:02.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 0x7f0594154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0594154a10 0x7f0594163a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594164440 con 0x7f0594163fa0 2023-10-21T16:04:02.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594164720 con 0x7f0580004d00 2023-10-21T16:04:02.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.595+0000 7f0598a15640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0594164a00 con 0x7f0594154a10 2023-10-21T16:04:02.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51300/0 (socket says 172.21.15.53:51300) 2023-10-21T16:04:02.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 -- 172.21.15.53:0/500626012 learned_addr learned my addr 172.21.15.53:0/500626012 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:02.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f059259c640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 0x7f0594154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 -- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0594154a10 msgr2=0x7f0594163a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f058bfff640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0594154a10 0x7f0594163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0594154a10 0x7f0594163a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 -- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 msgr2=0x7f0594154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 0x7f0594154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f0592d9d640 1 -- 172.21.15.53:0/500626012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594164ce0 con 0x7f0594163fa0 2023-10-21T16:04:02.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.596+0000 7f059259c640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 0x7f0594154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:02.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f0592d9d640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f058c002230 tx=0x7f058c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c034cd0 con 0x7f0594163fa0 2023-10-21T16:04:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c034e30 con 0x7f0594163fa0 2023-10-21T16:04:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c003b90 con 0x7f0594163fa0 2023-10-21T16:04:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594164fc0 con 0x7f0594163fa0 2023-10-21T16:04:02.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.597+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05941656c0 con 0x7f0594163fa0 2023-10-21T16:04:02.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.598+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059406d700 con 0x7f0594163fa0 2023-10-21T16:04:02.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.599+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f058c003cf0 con 0x7f0594163fa0 2023-10-21T16:04:02.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.600+0000 7f058affd640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 0x7f057406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:02.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.600+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f058c0ab4f0 con 0x7f0594163fa0 2023-10-21T16:04:02.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.601+0000 7f059259c640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 0x7f057406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:02.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.602+0000 7f059259c640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 0x7f057406d5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0584001270 tx=0x7f0584009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:02.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.604+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f058c075ff0 con 0x7f0594163fa0 2023-10-21T16:04:02.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.820+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0594164890 con 0x7f057406b100 2023-10-21T16:04:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.958+0000 7f058affd640 1 -- 172.21.15.53:0/500626012 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0594164890 con 0x7f057406b100 2023-10-21T16:04:02.961 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-21T16:04:02.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.961+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 msgr2=0x7f057406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 0x7f057406d5c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0584001270 tx=0x7f0584009040 comp rx=0 tx=0).stop 2023-10-21T16:04:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 msgr2=0x7f05940c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f058c002230 tx=0x7f058c009750 comp rx=0 tx=0).stop 2023-10-21T16:04:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 shutdown_connections 2023-10-21T16:04:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f057406b100 0x7f057406d5c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0594163fa0 0x7f05940c5f00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0594154a10 0x7f0594163a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 --2- 172.21.15.53:0/500626012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0580004d00 0x7f0594154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 >> 172.21.15.53:0/500626012 conn(0x7f059406a5d0 msgr2=0x7f0594163150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 shutdown_connections 2023-10-21T16:04:02.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:02.962+0000 7f0598a15640 1 -- 172.21.15.53:0/500626012 wait complete. 2023-10-21T16:04:02.983 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-21T16:04:02.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2023-10-21T16:04:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.223+0000 7f26418a2640 1 Processor -- start 2023-10-21T16:04:03.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.223+0000 7f26418a2640 1 -- start start 2023-10-21T16:04:03.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f26418a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f263c0c9cf0 0x7f263c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f26418a2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 0x7f263c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f26418a2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c15b310 con 0x7f263c06c3a0 2023-10-21T16:04:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f26418a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c06e0a0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f26418a2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c06d290 con 0x7f263c15aa00 2023-10-21T16:04:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f263a7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f263c0c9cf0 0x7f263c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.225+0000 7f263a7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f263c0c9cf0 0x7f263c06db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51310/0 (socket says 172.21.15.53:51310) 2023-10-21T16:04:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263a7fc640 1 -- 172.21.15.53:0/3467903626 learned_addr learned my addr 172.21.15.53:0/3467903626 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263b7fe640 1 --2- 172.21.15.53:0/3467903626 >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 0x7f263c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263a7fc640 1 -- 172.21.15.53:0/3467903626 >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 msgr2=0x7f263c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263a7fc640 1 --2- 172.21.15.53:0/3467903626 >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 0x7f263c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263a7fc640 1 -- 172.21.15.53:0/3467903626 >> v1:172.21.15.124:6789/0 conn(0x7f263c06c3a0 legacy=0x7f263c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263a7fc640 1 -- 172.21.15.53:0/3467903626 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0c9860 con 0x7f263c0c9cf0 2023-10-21T16:04:03.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.226+0000 7f263b7fe640 1 --2- 172.21.15.53:0/3467903626 >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 0x7f263c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.227+0000 7f263a7fc640 1 --2- 172.21.15.53:0/3467903626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c06db60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f263c0c57a0 tx=0x7f263002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15552d7540e29c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.227+0000 7f2639ffb640 1 -- 172.21.15.53:0/3467903626 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263002eea0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f2639ffb640 1 -- 172.21.15.53:0/3467903626 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630036440 con 0x7f263c0c9cf0 2023-10-21T16:04:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f2639ffb640 1 -- 172.21.15.53:0/3467903626 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26300367d0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 -- 172.21.15.53:0/3467903626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 msgr2=0x7f263c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 --2- 172.21.15.53:0/3467903626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c06db60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f263c0c57a0 tx=0x7f263002e0f0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 -- 172.21.15.53:0/3467903626 shutdown_connections 2023-10-21T16:04:03.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 --2- 172.21.15.53:0/3467903626 >> v2:172.21.15.124:3300/0 conn(0x7f263c15aa00 0x7f263c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 --2- 172.21.15.53:0/3467903626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c06db60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 -- 172.21.15.53:0/3467903626 >> 172.21.15.53:0/3467903626 conn(0x7f263c06a5d0 msgr2=0x7f263c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 -- 172.21.15.53:0/3467903626 shutdown_connections 2023-10-21T16:04:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.228+0000 7f26418a2640 1 -- 172.21.15.53:0/3467903626 wait complete. 2023-10-21T16:04:03.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.230+0000 7f26418a2640 1 Processor -- start 2023-10-21T16:04:03.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.230+0000 7f26418a2640 1 -- start start 2023-10-21T16:04:03.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 0x7f263c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f263c15aa00 0x7f263c0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c1553f0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0c6220 con 0x7f263c06c3a0 2023-10-21T16:04:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.231+0000 7f26418a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0c4ff0 con 0x7f263c15aa00 2023-10-21T16:04:03.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51318/0 (socket says 172.21.15.53:51318) 2023-10-21T16:04:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 -- 172.21.15.53:0/807060345 learned_addr learned my addr 172.21.15.53:0/807060345 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 -- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f263c15aa00 msgr2=0x7f263c0c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f263c15aa00 0x7f263c0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 -- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 msgr2=0x7f263c0c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263affd640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 0x7f263c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 0x7f263c0c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 -- 172.21.15.53:0/807060345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263000b770 con 0x7f263c0c9cf0 2023-10-21T16:04:03.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263affd640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 0x7f263c0c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:04:03.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.232+0000 7f263a7fc640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f263002e130 tx=0x7f2630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.233+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630034b80 con 0x7f263c0c9cf0 2023-10-21T16:04:03.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.233+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630034d00 con 0x7f263c0c9cf0 2023-10-21T16:04:03.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.233+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26300416c0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.233+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0c6470 con 0x7f263c0c9cf0 2023-10-21T16:04:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.233+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c0c6a30 con 0x7f263c0c9cf0 2023-10-21T16:04:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.234+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c0667d0 con 0x7f263c0c9cf0 2023-10-21T16:04:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.235+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2630041820 con 0x7f263c0c9cf0 2023-10-21T16:04:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.236+0000 7f2623fff640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 0x7f262806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.236+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f26300ace40 con 0x7f263c0c9cf0 2023-10-21T16:04:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.237+0000 7f263affd640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 0x7f262806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.238+0000 7f263affd640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 0x7f262806d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.241+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2630077920 con 0x7f263c0c9cf0 2023-10-21T16:04:03.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.461+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f263c0c8f30 con 0x7f262806b100 2023-10-21T16:04:03.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.587+0000 7f2623fff640 1 -- 172.21.15.53:0/807060345 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f262c0012c0 con 0x7f262806b100 2023-10-21T16:04:03.589 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-21T16:04:03.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 msgr2=0x7f262806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 0x7f262806d5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).stop 2023-10-21T16:04:03.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 msgr2=0x7f263c154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f263002e130 tx=0x7f2630004730 comp rx=0 tx=0).stop 2023-10-21T16:04:03.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 shutdown_connections 2023-10-21T16:04:03.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f262806b100 0x7f262806d5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f263c15aa00 0x7f263c0c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f263c0c9cf0 0x7f263c154eb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 --2- 172.21.15.53:0/807060345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f263c06c3a0 0x7f263c0c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 >> 172.21.15.53:0/807060345 conn(0x7f263c06a5d0 msgr2=0x7f263c0cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:03.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 shutdown_connections 2023-10-21T16:04:03.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.590+0000 7f26418a2640 1 -- 172.21.15.53:0/807060345 wait complete. 2023-10-21T16:04:03.614 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-21T16:04:03.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2023-10-21T16:04:03.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.843+0000 7f5d4ea90640 1 Processor -- start 2023-10-21T16:04:03.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.843+0000 7f5d4ea90640 1 -- start start 2023-10-21T16:04:03.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.844+0000 7f5d4ea90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d480afbe0 0x7f5d480b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.844+0000 7f5d4ea90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d4806dfc0 0x7f5d48070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4ea90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d48071070 0x7f5d480cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4ea90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480b2620 con 0x7f5d48071070 2023-10-21T16:04:03.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4ea90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480b2060 con 0x7f5d4806dfc0 2023-10-21T16:04:03.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4ea90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480b2340 con 0x7f5d480afbe0 2023-10-21T16:04:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4d02d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d48071070 0x7f5d480cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4d02d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d48071070 0x7f5d480cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51322/0 (socket says 172.21.15.53:51322) 2023-10-21T16:04:03.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4215438000 learned_addr learned my addr 172.21.15.53:0/4215438000 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:03.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d47fff640 1 --2- 172.21.15.53:0/4215438000 >> v2:172.21.15.124:3300/0 conn(0x7f5d4806dfc0 0x7f5d48070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.845+0000 7f5d4c82c640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d480afbe0 0x7f5d480b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.846+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4215438000 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d480afbe0 msgr2=0x7f5d480b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:03.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.846+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d480afbe0 0x7f5d480b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.846+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4215438000 >> v2:172.21.15.124:3300/0 conn(0x7f5d4806dfc0 msgr2=0x7f5d48070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:03.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.846+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4215438000 >> v2:172.21.15.124:3300/0 conn(0x7f5d4806dfc0 0x7f5d48070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.846+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4215438000 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d48070350 con 0x7f5d48071070 2023-10-21T16:04:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.847+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d48071070 0x7f5d480cb130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5d480cb0b0 tx=0x7f5d4002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f84946af44609e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.847+0000 7f5d477fe640 1 -- 172.21.15.53:0/4215438000 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40031860 con 0x7f5d48071070 2023-10-21T16:04:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.847+0000 7f5d477fe640 1 -- 172.21.15.53:0/4215438000 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d400319c0 con 0x7f5d48071070 2023-10-21T16:04:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.847+0000 7f5d477fe640 1 -- 172.21.15.53:0/4215438000 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40031d50 con 0x7f5d48071070 2023-10-21T16:04:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4215438000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d48071070 msgr2=0x7f5d480cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d48071070 0x7f5d480cb130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5d480cb0b0 tx=0x7f5d4002b900 comp rx=0 tx=0).stop 2023-10-21T16:04:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4215438000 shutdown_connections 2023-10-21T16:04:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d48071070 0x7f5d480cb130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4215438000 >> v2:172.21.15.124:3300/0 conn(0x7f5d4806dfc0 0x7f5d48070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4215438000 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d480afbe0 0x7f5d480b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4215438000 >> 172.21.15.53:0/4215438000 conn(0x7f5d4806a5d0 msgr2=0x7f5d480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4215438000 shutdown_connections 2023-10-21T16:04:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.848+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4215438000 wait complete. 2023-10-21T16:04:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.850+0000 7f5d4ea90640 1 Processor -- start 2023-10-21T16:04:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.850+0000 7f5d4ea90640 1 -- start start 2023-10-21T16:04:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4806dfc0 0x7f5d481efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d48071070 0x7f5d480c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c4080 con 0x7f5d480afbe0 2023-10-21T16:04:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c42d0 con 0x7f5d4806dfc0 2023-10-21T16:04:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.851+0000 7f5d4ea90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480c45b0 con 0x7f5d48071070 2023-10-21T16:04:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51330/0 (socket says 172.21.15.53:51330) 2023-10-21T16:04:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4019075646 learned_addr learned my addr 172.21.15.53:0/4019075646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4c82c640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4806dfc0 0x7f5d481efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d47fff640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d48071070 0x7f5d480c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d48071070 msgr2=0x7f5d480c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d48071070 0x7f5d480c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4806dfc0 msgr2=0x7f5d481efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4806dfc0 0x7f5d481efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.852+0000 7f5d4d02d640 1 -- 172.21.15.53:0/4019075646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400094d0 con 0x7f5d480afbe0 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d4d02d640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d400090b0 tx=0x7f5d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4003ac60 con 0x7f5d480afbe0 2023-10-21T16:04:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4003adc0 con 0x7f5d480afbe0 2023-10-21T16:04:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40044630 con 0x7f5d480afbe0 2023-10-21T16:04:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480c4890 con 0x7f5d480afbe0 2023-10-21T16:04:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.853+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480c4e50 con 0x7f5d480afbe0 2023-10-21T16:04:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.854+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4804e160 con 0x7f5d480afbe0 2023-10-21T16:04:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.855+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5d4004d450 con 0x7f5d480afbe0 2023-10-21T16:04:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.856+0000 7f5d457fa640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 0x7f5d3406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.856+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5d400ac3e0 con 0x7f5d480afbe0 2023-10-21T16:04:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.857+0000 7f5d4c82c640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 0x7f5d3406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.858+0000 7f5d4c82c640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 0x7f5d3406d5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009a60 tx=0x7f5d38003990 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:03.860+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d40076d20 con 0x7f5d480afbe0 2023-10-21T16:04:04.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.080+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d480affc0 con 0x7f5d3406b100 2023-10-21T16:04:04.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.209+0000 7f5d457fa640 1 -- 172.21.15.53:0/4019075646 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5d380096f0 con 0x7f5d3406b100 2023-10-21T16:04:04.212 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-21T16:04:04.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.212+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 msgr2=0x7f5d3406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.212+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 0x7f5d3406d5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009a60 tx=0x7f5d38003990 comp rx=0 tx=0).stop 2023-10-21T16:04:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 msgr2=0x7f5d480ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d400090b0 tx=0x7f5d400046c0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 shutdown_connections 2023-10-21T16:04:04.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f5d3406b100 0x7f5d3406d5c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d480afbe0 0x7f5d480ce9c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d48071070 0x7f5d480c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 --2- 172.21.15.53:0/4019075646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4806dfc0 0x7f5d481efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 >> 172.21.15.53:0/4019075646 conn(0x7f5d4806a5d0 msgr2=0x7f5d4806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 shutdown_connections 2023-10-21T16:04:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.213+0000 7f5d4ea90640 1 -- 172.21.15.53:0/4019075646 wait complete. 2023-10-21T16:04:04.235 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-21T16:04:04.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2023-10-21T16:04:04.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.464+0000 7f22bbfff640 1 Processor -- start 2023-10-21T16:04:04.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.464+0000 7f22bbfff640 1 -- start start 2023-10-21T16:04:04.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.465+0000 7f22bbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22bc0b0680 0x7f22bc15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.465+0000 7f22bbfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0cab60 con 0x7f22bc0ca680 2023-10-21T16:04:04.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.466+0000 7f22bbfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0cb960 con 0x7f22bc15a820 2023-10-21T16:04:04.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.466+0000 7f22bbfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc0cbdf0 con 0x7f22bc0b0680 2023-10-21T16:04:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.466+0000 7f22bb7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f22bc0ca680 0x7f22bc15ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:44286/0 (socket says 172.21.15.53:44286) 2023-10-21T16:04:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.466+0000 7f22bb7fe640 1 -- 172.21.15.53:0/4049655159 learned_addr learned my addr 172.21.15.53:0/4049655159 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:04:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.466+0000 7f22b3fff640 1 --2- 172.21.15.53:0/4049655159 >> v2:172.21.15.124:3300/0 conn(0x7f22bc0b0680 0x7f22bc15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:04.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483988246 0 0) 0x7f22bc0cab60 con 0x7f22bc0ca680 2023-10-21T16:04:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a8003520 con 0x7f22bc0ca680 2023-10-21T16:04:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395178474 0 0) 0x7f22a8003520 con 0x7f22bc0ca680 2023-10-21T16:04:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 >> v2:172.21.15.124:3300/0 conn(0x7f22bc0b0680 msgr2=0x7f22bc15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 --2- 172.21.15.53:0/4049655159 >> v2:172.21.15.124:3300/0 conn(0x7f22bc0b0680 0x7f22bc15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 >> v1:172.21.15.124:6789/0 conn(0x7f22bc15a820 legacy=0x7f22bc0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0cb030 con 0x7f22bc0ca680 2023-10-21T16:04:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f22b40019f0 con 0x7f22bc0ca680 2023-10-21T16:04:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 >> v1:172.21.15.53:6789/0 conn(0x7f22bc0ca680 legacy=0x7f22bc15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22a80047c0 0x7f22a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22a8005180 0x7f22a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0cbdf0 con 0x7f22a8003970 2023-10-21T16:04:04.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a800c120 con 0x7f22a8005180 2023-10-21T16:04:04.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a800c2e0 con 0x7f22a80047c0 2023-10-21T16:04:04.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.467+0000 7f22baffd640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:04.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22b3fff640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22a80047c0 0x7f22a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:04.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22baffd640 1 -- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22a80047c0 msgr2=0x7f22a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:04.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22baffd640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22a80047c0 0x7f22a8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22baffd640 1 -- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22a8005180 msgr2=0x7f22a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:04:04.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22baffd640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22a8005180 0x7f22a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22baffd640 1 -- 172.21.15.53:0/4049655159 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8003520 con 0x7f22a8003970 2023-10-21T16:04:04.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.468+0000 7f22bb7fe640 1 -- 172.21.15.53:0/4049655159 reap_dead start 2023-10-21T16:04:04.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.469+0000 7f22baffd640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22a8003d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f22ac001750 tx=0x7f22a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d067ebd96bdf77a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.469+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0052d0 con 0x7f22a8003970 2023-10-21T16:04:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.469+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac02e050 con 0x7f22a8003970 2023-10-21T16:04:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.469+0000 7f22ba7fc640 1 -- 172.21.15.53:0/4049655159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003830 con 0x7f22a8003970 2023-10-21T16:04:04.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 -- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 msgr2=0x7f22a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22a8003d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f22ac001750 tx=0x7f22a80057f0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 -- 172.21.15.53:0/4049655159 shutdown_connections 2023-10-21T16:04:04.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 --2- 172.21.15.53:0/4049655159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22a8003d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 -- 172.21.15.53:0/4049655159 >> 172.21.15.53:0/4049655159 conn(0x7f22bc06c050 msgr2=0x7f22bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:04.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 -- 172.21.15.53:0/4049655159 shutdown_connections 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.470+0000 7f22bbfff640 1 -- 172.21.15.53:0/4049655159 wait complete. 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.472+0000 7f22bbfff640 1 Processor -- start 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.472+0000 7f22bbfff640 1 -- start start 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22bc14fcf0 0x7f22bc1455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22bc1f3b00 0x7f22bc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c7de0 con 0x7f22a8003970 2023-10-21T16:04:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c36e0 con 0x7f22bc14fcf0 2023-10-21T16:04:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22bbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c85b0 con 0x7f22bc1f3b00 2023-10-21T16:04:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22baffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22baffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36600/0 (socket says 172.21.15.53:36600) 2023-10-21T16:04:04.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22baffd640 1 -- 172.21.15.53:0/3996238753 learned_addr learned my addr 172.21.15.53:0/3996238753 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:04:04.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22baffd640 1 -- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22bc1f3b00 msgr2=0x7f22bc0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.473+0000 7f22baffd640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22bc1f3b00 0x7f22bc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22baffd640 1 -- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22bc14fcf0 msgr2=0x7f22bc1455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22baffd640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22bc14fcf0 0x7f22bc1455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22baffd640 1 -- 172.21.15.53:0/3996238753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0c3910 con 0x7f22a8003970 2023-10-21T16:04:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22baffd640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f22ac02bc30 tx=0x7f22ac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0076c0 con 0x7f22a8003970 2023-10-21T16:04:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac007a30 con 0x7f22a8003970 2023-10-21T16:04:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc0c7ff0 con 0x7f22a8003970 2023-10-21T16:04:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac006be0 con 0x7f22a8003970 2023-10-21T16:04:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.474+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc0c3bb0 con 0x7f22a8003970 2023-10-21T16:04:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.476+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc074150 con 0x7f22a8003970 2023-10-21T16:04:04.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.476+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 11) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f22ac006d40 con 0x7f22a8003970 2023-10-21T16:04:04.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.477+0000 7f22b37fe640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 0x7f229c06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:04:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.478+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f22ac0ab890 con 0x7f22a8003970 2023-10-21T16:04:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.478+0000 7f22b3fff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 0x7f229c06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:04:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.479+0000 7f22b3fff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 0x7f229c06d5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1462a0 tx=0x7f22a4008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:04:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.482+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22ac076340 con 0x7f22a8003970 2023-10-21T16:04:04.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.699+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f22bc0cb5f0 con 0x7f229c06b130 2023-10-21T16:04:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.823+0000 7f22b37fe640 1 -- 172.21.15.53:0/3996238753 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f22a4006bd0 con 0x7f229c06b130 2023-10-21T16:04:04.825 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-21T16:04:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 msgr2=0x7f229c06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 0x7f229c06d5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1462a0 tx=0x7f22a4008040 comp rx=0 tx=0).stop 2023-10-21T16:04:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 msgr2=0x7f22bc0c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:04:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f22ac02bc30 tx=0x7f22ac001120 comp rx=0 tx=0).stop 2023-10-21T16:04:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 shutdown_connections 2023-10-21T16:04:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f229c06b130 0x7f229c06d5f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22bc1f3b00 0x7f22bc0b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22bc14fcf0 0x7f22bc1455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 --2- 172.21.15.53:0/3996238753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22a8003970 0x7f22bc0c55c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:04:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 >> 172.21.15.53:0/3996238753 conn(0x7f22bc06c050 msgr2=0x7f22bc070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:04:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 shutdown_connections 2023-10-21T16:04:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:04:04.826+0000 7f22bbfff640 1 -- 172.21.15.53:0/3996238753 wait complete. 2023-10-21T16:04:04.854 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-21T16:04:16.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:04:16.688 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:04:46.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:04:46.838 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:05:16.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:05:16.986 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:05:47.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:05:47.132 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:06:17.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:06:17.296 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:06:47.559 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:06:47.569 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:07:01.010 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:07:01.009+0000 7f405f630640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:07:01.011 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:07:01.009+0000 7f405f630640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:07:01.011 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:07:01.009+0000 7f405f630640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:07:01.011 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:07:01.009+0000 7f405f630640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:07:01.011 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:07:01.009+0000 7f405f630640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:07:17.716 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:07:17.727 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:07:47.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:07:47.875 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:08:18.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:08:18.036 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:08:48.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:08:48.210 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:09:18.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:09:18.368 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:09:48.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:09:48.516 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:10:18.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:10:18.659 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:10:48.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:10:48.820 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:11:18.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:11:18.967 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:11:49.126 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:11:49.139 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:12:19.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:12:19.308 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:12:49.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:12:49.455 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:13:19.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:13:19.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:13:49.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:13:49.743 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:14:19.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:14:19.896 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:14:22.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2023-10-21T16:14:23.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.097+0000 7fe191b0a640 1 Processor -- start 2023-10-21T16:14:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.097+0000 7fe191b0a640 1 -- start start 2023-10-21T16:14:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe191b0a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe18c070cc0 0x7fe18c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe191b0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c0cb450 0x7fe18c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe191b0a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c071170 con 0x7fe18c160660 2023-10-21T16:14:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe191b0a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c06e150 con 0x7fe18c070cc0 2023-10-21T16:14:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe191b0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0cb8c0 con 0x7fe18c0cb450 2023-10-21T16:14:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe183fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe18c070cc0 0x7fe18c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe183fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe18c070cc0 0x7fe18c0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44502/0 (socket says 172.21.15.53:44502) 2023-10-21T16:14:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.099+0000 7fe183fff640 1 -- 172.21.15.53:0/1266611043 learned_addr learned my addr 172.21.15.53:0/1266611043 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.100+0000 7fe18bfff640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c0cb450 0x7fe18c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.100+0000 7fe183fff640 1 -- 172.21.15.53:0/1266611043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c0cb450 msgr2=0x7fe18c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:23.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.100+0000 7fe183fff640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c0cb450 0x7fe18c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.100+0000 7fe183fff640 1 -- 172.21.15.53:0/1266611043 >> v1:172.21.15.124:6789/0 conn(0x7fe18c160660 legacy=0x7fe18c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:23.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.100+0000 7fe183fff640 1 -- 172.21.15.53:0/1266611043 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c06de70 con 0x7fe18c070cc0 2023-10-21T16:14:23.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.101+0000 7fe183fff640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c070cc0 0x7fe18c0caf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe174009870 tx=0x7fe17402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf497ca746432d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe18affd640 1 -- 172.21.15.53:0/1266611043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174030840 con 0x7fe18c070cc0 2023-10-21T16:14:23.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe18affd640 1 -- 172.21.15.53:0/1266611043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1740309a0 con 0x7fe18c070cc0 2023-10-21T16:14:23.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe18affd640 1 -- 172.21.15.53:0/1266611043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174030d30 con 0x7fe18c070cc0 2023-10-21T16:14:23.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 -- 172.21.15.53:0/1266611043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c070cc0 msgr2=0x7fe18c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c070cc0 0x7fe18c0caf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe174009870 tx=0x7fe17402b980 comp rx=0 tx=0).stop 2023-10-21T16:14:23.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 -- 172.21.15.53:0/1266611043 shutdown_connections 2023-10-21T16:14:23.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c0cb450 0x7fe18c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 --2- 172.21.15.53:0/1266611043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c070cc0 0x7fe18c0caf10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.102+0000 7fe191b0a640 1 -- 172.21.15.53:0/1266611043 >> 172.21.15.53:0/1266611043 conn(0x7fe18c06a5d0 msgr2=0x7fe18c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:23.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.103+0000 7fe191b0a640 1 -- 172.21.15.53:0/1266611043 shutdown_connections 2023-10-21T16:14:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.103+0000 7fe191b0a640 1 -- 172.21.15.53:0/1266611043 wait complete. 2023-10-21T16:14:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.104+0000 7fe191b0a640 1 Processor -- start 2023-10-21T16:14:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 -- start start 2023-10-21T16:14:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c070cc0 0x7fe18c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 0x7fe18c1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0c5140 con 0x7fe18c160660 2023-10-21T16:14:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c157f00 con 0x7fe18c0cb450 2023-10-21T16:14:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.105+0000 7fe191b0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c1581c0 con 0x7fe18c070cc0 2023-10-21T16:14:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44512/0 (socket says 172.21.15.53:44512) 2023-10-21T16:14:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 -- 172.21.15.53:0/3191583252 learned_addr learned my addr 172.21.15.53:0/3191583252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe183fff640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 0x7fe18c1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 -- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c070cc0 msgr2=0x7fe18c0c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18b7fe640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c070cc0 0x7fe18c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c070cc0 0x7fe18c0c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 -- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 msgr2=0x7fe18c1f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 0x7fe18c1f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe18bfff640 1 -- 172.21.15.53:0/3191583252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1740094d0 con 0x7fe18c160660 2023-10-21T16:14:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.106+0000 7fe183fff640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 0x7fe18c1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe18bfff640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe184007e40 tx=0x7fe18400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184014620 con 0x7fe18c160660 2023-10-21T16:14:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18400ee90 con 0x7fe18c160660 2023-10-21T16:14:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840149e0 con 0x7fe18c160660 2023-10-21T16:14:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c1584a0 con 0x7fe18c160660 2023-10-21T16:14:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.107+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c158a60 con 0x7fe18c160660 2023-10-21T16:14:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.108+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c04e160 con 0x7fe18c160660 2023-10-21T16:14:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.109+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe184014b40 con 0x7fe18c160660 2023-10-21T16:14:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.110+0000 7fe188ff9640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 0x7fe16c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.110+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe18408a6c0 con 0x7fe18c160660 2023-10-21T16:14:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.111+0000 7fe18b7fe640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 0x7fe16c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.112+0000 7fe18b7fe640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 0x7fe16c06d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000f60 tx=0x7fe17c009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.115+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1840551f0 con 0x7fe18c160660 2023-10-21T16:14:23.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.335+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe18c0c9110 con 0x7fe16c06b070 2023-10-21T16:14:23.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.509+0000 7fe188ff9640 1 -- 172.21.15.53:0/3191583252 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe17c0012e0 con 0x7fe16c06b070 2023-10-21T16:14:23.511 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-21T16:14:23.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 msgr2=0x7fe16c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 0x7fe16c06d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000f60 tx=0x7fe17c009040 comp rx=0 tx=0).stop 2023-10-21T16:14:23.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 msgr2=0x7fe18c0c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe184007e40 tx=0x7fe18400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 shutdown_connections 2023-10-21T16:14:23.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fe16c06b070 0x7fe16c06d530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe18c160660 0x7fe18c0c6d10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe18c0cb450 0x7fe18c1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 --2- 172.21.15.53:0/3191583252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe18c070cc0 0x7fe18c0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.512+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 >> 172.21.15.53:0/3191583252 conn(0x7fe18c06a5d0 msgr2=0x7fe18c0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.513+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 shutdown_connections 2023-10-21T16:14:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.513+0000 7fe191b0a640 1 -- 172.21.15.53:0/3191583252 wait complete. 2023-10-21T16:14:23.536 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-21T16:14:23.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2023-10-21T16:14:23.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.761+0000 7f9bc0967640 1 Processor -- start 2023-10-21T16:14:23.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.762+0000 7f9bc0967640 1 -- start start 2023-10-21T16:14:23.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bc0967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9bbc0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bc0967640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0ccbc0 con 0x7f9bbc0aff70 2023-10-21T16:14:23.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bc0967640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0ce1f0 con 0x7f9bbc15cae0 2023-10-21T16:14:23.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bc0967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0c94c0 con 0x7f9bbc0c8b40 2023-10-21T16:14:23.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bbbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9bbc0aff70 0x7f9bbc070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57792/0 (socket says 172.21.15.53:57792) 2023-10-21T16:14:23.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.763+0000 7f9bbbfff640 1 -- 172.21.15.53:0/478483348 learned_addr learned my addr 172.21.15.53:0/478483348 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9bbc0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061865120 0 0) 0x7f9bbc0ccbc0 con 0x7f9bbc0aff70 2023-10-21T16:14:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba8003520 con 0x7f9bbc0aff70 2023-10-21T16:14:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3996840830 0 0) 0x7f9ba8003520 con 0x7f9bbc0aff70 2023-10-21T16:14:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 msgr2=0x7f9bbc0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9bbc0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 >> v1:172.21.15.124:6789/0 conn(0x7f9bbc15cae0 legacy=0x7f9bbc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0c9720 con 0x7f9bbc0aff70 2023-10-21T16:14:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9bb4001670 con 0x7f9bbc0aff70 2023-10-21T16:14:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 >> v1:172.21.15.53:6789/0 conn(0x7f9bbc0aff70 legacy=0x7f9bbc070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9ba80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.764+0000 7f9bba7fc640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9ba80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bba7fc640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba8004d00 0x7f9ba8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0c94c0 con 0x7f9ba8003f70 2023-10-21T16:14:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba800bac0 con 0x7f9ba8004d00 2023-10-21T16:14:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba800bc80 con 0x7f9bbc0c8b40 2023-10-21T16:14:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9ba80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbbfff640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba8004d00 0x7f9ba8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 -- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 msgr2=0x7f9ba80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbb7fe640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9ba80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0c8b40 0x7f9ba80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 -- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba8004d00 msgr2=0x7f9ba8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba8004d00 0x7f9ba8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbaffd640 1 -- 172.21.15.53:0/478483348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba8003520 con 0x7f9ba8003f70 2023-10-21T16:14:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.765+0000 7f9bbbfff640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba8004d00 0x7f9ba8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.766+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9ba80047c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fc0 tx=0x7f9ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effbda993fad4386 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba402cb40 con 0x7f9ba8003f70 2023-10-21T16:14:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400f040 con 0x7f9ba8003f70 2023-10-21T16:14:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bba7fc640 1 -- 172.21.15.53:0/478483348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4034440 con 0x7f9ba8003f70 2023-10-21T16:14:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bc0967640 1 -- 172.21.15.53:0/478483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 msgr2=0x7f9ba80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bc0967640 1 --2- 172.21.15.53:0/478483348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9ba80047c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fc0 tx=0x7f9ba80057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bbbfff640 1 -- 172.21.15.53:0/478483348 reap_dead start 2023-10-21T16:14:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bc0967640 1 -- 172.21.15.53:0/478483348 shutdown_connections 2023-10-21T16:14:23.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.767+0000 7f9bc0967640 1 -- 172.21.15.53:0/478483348 >> 172.21.15.53:0/478483348 conn(0x7f9bbc06c050 msgr2=0x7f9bbc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:23.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.768+0000 7f9bc0967640 1 -- 172.21.15.53:0/478483348 shutdown_connections 2023-10-21T16:14:23.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.768+0000 7f9bc0967640 1 -- 172.21.15.53:0/478483348 wait complete. 2023-10-21T16:14:23.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.769+0000 7f9bc0967640 1 Processor -- start 2023-10-21T16:14:23.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 -- start start 2023-10-21T16:14:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 0x7f9bbc154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0b19e0 0x7f9bbc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15a7e0 con 0x7f9ba8003f70 2023-10-21T16:14:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc154770 con 0x7f9bbc0b10d0 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.770+0000 7f9bc0967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc1f3230 con 0x7f9bbc0b19e0 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44532/0 (socket says 172.21.15.53:44532) 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 -- 172.21.15.53:0/1746028298 learned_addr learned my addr 172.21.15.53:0/1746028298 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbbfff640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0b19e0 0x7f9bbc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 0x7f9bbc154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 -- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0b19e0 msgr2=0x7f9bbc0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0b19e0 0x7f9bbc0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 -- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 msgr2=0x7f9bbc154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 0x7f9bbc154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbb7fe640 1 -- 172.21.15.53:0/1746028298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc154120 con 0x7f9ba8003f70 2023-10-21T16:14:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.771+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 0x7f9bbc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bbb7fe640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d570 tx=0x7f9bac00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac005480 con 0x7f9ba8003f70 2023-10-21T16:14:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac003610 con 0x7f9ba8003f70 2023-10-21T16:14:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc154400 con 0x7f9ba8003f70 2023-10-21T16:14:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003980 con 0x7f9ba8003f70 2023-10-21T16:14:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.772+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc1f2ee0 con 0x7f9ba8003f70 2023-10-21T16:14:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.773+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc074150 con 0x7f9ba8003f70 2023-10-21T16:14:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.774+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9bac003ae0 con 0x7f9ba8003f70 2023-10-21T16:14:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.775+0000 7f9bb3fff640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 0x7f9b9c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.775+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f9bac089bd0 con 0x7f9ba8003f70 2023-10-21T16:14:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.776+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 0x7f9b9c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.777+0000 7f9bbaffd640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 0x7f9b9c06d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4036000 tx=0x7f9ba402e000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:23.780+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bac054680 con 0x7f9ba8003f70 2023-10-21T16:14:24.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.012+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bbc0cc850 con 0x7f9b9c06b070 2023-10-21T16:14:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.183+0000 7f9bb3fff640 1 -- 172.21.15.53:0/1746028298 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9ba40440c0 con 0x7f9b9c06b070 2023-10-21T16:14:24.185 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-21T16:14:24.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.186+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 msgr2=0x7f9b9c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.186+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 0x7f9b9c06d530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4036000 tx=0x7f9ba402e000 comp rx=0 tx=0).stop 2023-10-21T16:14:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.186+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 msgr2=0x7f9bbc15a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.186+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d570 tx=0x7f9bac00da30 comp rx=0 tx=0).stop 2023-10-21T16:14:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 shutdown_connections 2023-10-21T16:14:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f9b9c06b070 0x7f9b9c06d530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bbc0b19e0 0x7f9bbc0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bbc0b10d0 0x7f9bbc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 --2- 172.21.15.53:0/1746028298 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba8003f70 0x7f9bbc15a2a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 >> 172.21.15.53:0/1746028298 conn(0x7f9bbc06c050 msgr2=0x7f9bbc070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 shutdown_connections 2023-10-21T16:14:24.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.187+0000 7f9bc0967640 1 -- 172.21.15.53:0/1746028298 wait complete. 2023-10-21T16:14:24.209 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-21T16:14:24.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2023-10-21T16:14:24.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.446+0000 7f7eaffff640 1 Processor -- start 2023-10-21T16:14:24.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.446+0000 7f7eaffff640 1 -- start start 2023-10-21T16:14:24.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eb015a820 0x7f7eb00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb00cab60 con 0x7f7eb00ca680 2023-10-21T16:14:24.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb00cb960 con 0x7f7eb015a820 2023-10-21T16:14:24.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00cbdf0 con 0x7f7eb015ac00 2023-10-21T16:14:24.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.448+0000 7f7eaeffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eb015a820 0x7f7eb00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:24.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eb015a820 0x7f7eb00aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47558/0 (socket says 172.21.15.53:47558) 2023-10-21T16:14:24.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 -- 172.21.15.53:0/3398784955 learned_addr learned my addr 172.21.15.53:0/3398784955 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:24.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaf7fe640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:24.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 -- 172.21.15.53:0/3398784955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 msgr2=0x7f7eb00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:24.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb00b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 -- 172.21.15.53:0/3398784955 >> v1:172.21.15.124:6789/0 conn(0x7f7eb00ca680 legacy=0x7f7eb00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:24.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.449+0000 7f7eaeffd640 1 -- 172.21.15.53:0/3398784955 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00cb030 con 0x7f7eb015a820 2023-10-21T16:14:24.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.451+0000 7f7eaeffd640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb015a820 0x7f7eb00aff70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a905de0adc6b9f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:24.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.451+0000 7f7eae7fc640 1 -- 172.21.15.53:0/3398784955 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00317c0 con 0x7f7eb015a820 2023-10-21T16:14:24.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.451+0000 7f7eae7fc640 1 -- 172.21.15.53:0/3398784955 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0031920 con 0x7f7eb015a820 2023-10-21T16:14:24.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.451+0000 7f7eae7fc640 1 -- 172.21.15.53:0/3398784955 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0031cb0 con 0x7f7eb015a820 2023-10-21T16:14:24.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 -- 172.21.15.53:0/3398784955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb015a820 msgr2=0x7f7eb00aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb015a820 0x7f7eb00aff70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea002b860 comp rx=0 tx=0).stop 2023-10-21T16:14:24.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 -- 172.21.15.53:0/3398784955 shutdown_connections 2023-10-21T16:14:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 --2- 172.21.15.53:0/3398784955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb015a820 0x7f7eb00aff70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 -- 172.21.15.53:0/3398784955 >> 172.21.15.53:0/3398784955 conn(0x7f7eb006c050 msgr2=0x7f7eb006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 -- 172.21.15.53:0/3398784955 shutdown_connections 2023-10-21T16:14:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.452+0000 7f7eaffff640 1 -- 172.21.15.53:0/3398784955 wait complete. 2023-10-21T16:14:24.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.454+0000 7f7eaffff640 1 Processor -- start 2023-10-21T16:14:24.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.455+0000 7f7eaffff640 1 -- start start 2023-10-21T16:14:24.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb015a820 0x7f7eb00c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00ae810 con 0x7f7eb00ca680 2023-10-21T16:14:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00aeaf0 con 0x7f7eb015a820 2023-10-21T16:14:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00aedd0 con 0x7f7eb015ac00 2023-10-21T16:14:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47570/0 (socket says 172.21.15.53:47570) 2023-10-21T16:14:24.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.456+0000 7f7eaeffd640 1 -- 172.21.15.53:0/4138507312 learned_addr learned my addr 172.21.15.53:0/4138507312 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:24.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 -- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 msgr2=0x7f7eb015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7ea7fff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb015a820 0x7f7eb00c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 -- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb015a820 msgr2=0x7f7eb00c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb015a820 0x7f7eb00c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 -- 172.21.15.53:0/4138507312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00094d0 con 0x7f7eb00ca680 2023-10-21T16:14:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7eaeffd640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.457+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0032860 con 0x7f7eb00ca680 2023-10-21T16:14:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.458+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea00329c0 con 0x7f7eb00ca680 2023-10-21T16:14:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.458+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0032d70 con 0x7f7eb00ca680 2023-10-21T16:14:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.458+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb00af0b0 con 0x7f7eb00ca680 2023-10-21T16:14:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.458+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb00af8b0 con 0x7f7eb00ca680 2023-10-21T16:14:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.459+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb00afa40 con 0x7f7eb00ca680 2023-10-21T16:14:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.460+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f7ea004d3f0 con 0x7f7eb00ca680 2023-10-21T16:14:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.461+0000 7f7ea77fe640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 0x7f7e9006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.461+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7ea00abb30 con 0x7f7eb00ca680 2023-10-21T16:14:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.462+0000 7f7ea7fff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 0x7f7e9006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.463+0000 7f7ea7fff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 0x7f7e9006d530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00ccbc0 tx=0x7f7e98009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.465+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7eb00afa40 con 0x7f7eb00ca680 2023-10-21T16:14:24.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.683+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb00cb5f0 con 0x7f7e9006b070 2023-10-21T16:14:24.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.851+0000 7f7ea77fe640 1 -- 172.21.15.53:0/4138507312 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7e98003980 con 0x7f7e9006b070 2023-10-21T16:14:24.853 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-21T16:14:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 msgr2=0x7f7e9006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 0x7f7e9006d530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00ccbc0 tx=0x7f7e98009040 comp rx=0 tx=0).stop 2023-10-21T16:14:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 msgr2=0x7f7eb0151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:24.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009870 tx=0x7f7ea00046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 shutdown_connections 2023-10-21T16:14:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7e9006b070 0x7f7e9006d530 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb015ac00 0x7f7eb015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb015a820 0x7f7eb00c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 --2- 172.21.15.53:0/4138507312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb00ca680 0x7f7eb0151b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.855+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 >> 172.21.15.53:0/4138507312 conn(0x7f7eb006c050 msgr2=0x7f7eb00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.856+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 shutdown_connections 2023-10-21T16:14:24.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:24.856+0000 7f7eaffff640 1 -- 172.21.15.53:0/4138507312 wait complete. 2023-10-21T16:14:24.880 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-21T16:14:24.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2023-10-21T16:14:25.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.097+0000 7f825bfff640 1 Processor -- start 2023-10-21T16:14:25.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.097+0000 7f825bfff640 1 -- start start 2023-10-21T16:14:25.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f825c15a820 0x7f825c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f825c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c0cab60 con 0x7f825c15ac00 2023-10-21T16:14:25.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0cb960 con 0x7f825c15a820 2023-10-21T16:14:25.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cbdf0 con 0x7f825c0ca680 2023-10-21T16:14:25.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f825c15ac00 0x7f825c0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53858/0 (socket says 172.21.15.53:53858) 2023-10-21T16:14:25.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.099+0000 7f825b7fe640 1 -- 172.21.15.53:0/328053980 learned_addr learned my addr 172.21.15.53:0/328053980 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:25.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f825affd640 1 --2- 172.21.15.53:0/328053980 >> v2:172.21.15.124:3300/0 conn(0x7f825c15a820 0x7f825c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f825a7fc640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f825c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318660571 0 0) 0x7f825c0cab60 con 0x7f825c15ac00 2023-10-21T16:14:25.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8244003520 con 0x7f825c15ac00 2023-10-21T16:14:25.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066834264 0 0) 0x7f8244003520 con 0x7f825c15ac00 2023-10-21T16:14:25.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 msgr2=0x7f825c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f825c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 >> v2:172.21.15.124:3300/0 conn(0x7f825c15a820 msgr2=0x7f825c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 --2- 172.21.15.53:0/328053980 >> v2:172.21.15.124:3300/0 conn(0x7f825c15a820 0x7f825c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0cb030 con 0x7f825c15ac00 2023-10-21T16:14:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f82540019f0 con 0x7f825c15ac00 2023-10-21T16:14:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 >> v1:172.21.15.53:6789/0 conn(0x7f825c15ac00 legacy=0x7f825c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f8244003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8244003f40 0x7f82440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f8244005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cb960 con 0x7f8244004d00 2023-10-21T16:14:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cbdf0 con 0x7f8244003f40 2023-10-21T16:14:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.100+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824400ba30 con 0x7f825c0ca680 2023-10-21T16:14:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f8244005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825affd640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f8244003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825a7fc640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8244003f40 0x7f82440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 -- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 msgr2=0x7f8244003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c0ca680 0x7f8244003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 -- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8244003f40 msgr2=0x7f82440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8244003f40 0x7f82440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.101+0000 7f825b7fe640 1 -- 172.21.15.53:0/328053980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244003520 con 0x7f8244004d00 2023-10-21T16:14:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.102+0000 7f825b7fe640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f8244005180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f825c15b010 tx=0x7f82440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88328c5d3c728121 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254034630 con 0x7f8244004d00 2023-10-21T16:14:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825402f070 con 0x7f8244004d00 2023-10-21T16:14:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f8259ffb640 1 -- 172.21.15.53:0/328053980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254034a30 con 0x7f8244004d00 2023-10-21T16:14:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 -- 172.21.15.53:0/328053980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 msgr2=0x7f8244005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 --2- 172.21.15.53:0/328053980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f8244005180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f825c15b010 tx=0x7f82440057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825b7fe640 1 -- 172.21.15.53:0/328053980 reap_dead start 2023-10-21T16:14:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 -- 172.21.15.53:0/328053980 shutdown_connections 2023-10-21T16:14:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 -- 172.21.15.53:0/328053980 >> 172.21.15.53:0/328053980 conn(0x7f825c06c050 msgr2=0x7f825c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 -- 172.21.15.53:0/328053980 shutdown_connections 2023-10-21T16:14:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.103+0000 7f825bfff640 1 -- 172.21.15.53:0/328053980 wait complete. 2023-10-21T16:14:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.105+0000 7f825bfff640 1 Processor -- start 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.105+0000 7f825bfff640 1 -- start start 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 0x7f825c1f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c1f5910 0x7f825c1f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0c7f20 con 0x7f8244004d00 2023-10-21T16:14:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0c7310 con 0x7f825c0cd680 2023-10-21T16:14:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.106+0000 7f825bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0c75f0 con 0x7f825c1f5910 2023-10-21T16:14:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47582/0 (socket says 172.21.15.53:47582) 2023-10-21T16:14:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 -- 172.21.15.53:0/3076042265 learned_addr learned my addr 172.21.15.53:0/3076042265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825a7fc640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 0x7f825c1f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825b7fe640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c1f5910 0x7f825c1f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 -- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c1f5910 msgr2=0x7f825c1f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c1f5910 0x7f825c1f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 -- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 msgr2=0x7f825c1f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 0x7f825c1f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 -- 172.21.15.53:0/3076042265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0c78d0 con 0x7f8244004d00 2023-10-21T16:14:25.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825a7fc640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 0x7f825c1f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:25.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.107+0000 7f825affd640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f824c0116d0 tx=0x7f824c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.108+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00f9c0 con 0x7f8244004d00 2023-10-21T16:14:25.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.108+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00fb20 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.108+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00fe90 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.108+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0c7bb0 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.108+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c0c83b0 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.109+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c074150 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.110+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f824c015460 con 0x7f8244004d00 2023-10-21T16:14:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.111+0000 7f82437fe640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 0x7f824806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.111+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f824c08a170 con 0x7f8244004d00 2023-10-21T16:14:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.112+0000 7f825a7fc640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 0x7f824806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.113+0000 7f825a7fc640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 0x7f824806d560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8250009980 tx=0x7f8250008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.117+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f824c054df0 con 0x7f8244004d00 2023-10-21T16:14:25.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.335+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f825c0cb5f0 con 0x7f824806b0a0 2023-10-21T16:14:25.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.517+0000 7f82437fe640 1 -- 172.21.15.53:0/3076042265 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8250012950 con 0x7f824806b0a0 2023-10-21T16:14:25.519 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-21T16:14:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.520+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 msgr2=0x7f824806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.520+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 0x7f824806d560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8250009980 tx=0x7f8250008040 comp rx=0 tx=0).stop 2023-10-21T16:14:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.520+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 msgr2=0x7f825c0c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.520+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f824c0116d0 tx=0x7f824c011b90 comp rx=0 tx=0).stop 2023-10-21T16:14:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 shutdown_connections 2023-10-21T16:14:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f824806b0a0 0x7f824806d560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f825c1f5910 0x7f825c1f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825c0cd680 0x7f825c1f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 --2- 172.21.15.53:0/3076042265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8244004d00 0x7f825c0c6cc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 >> 172.21.15.53:0/3076042265 conn(0x7f825c06c050 msgr2=0x7f825c0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 shutdown_connections 2023-10-21T16:14:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.521+0000 7f825bfff640 1 -- 172.21.15.53:0/3076042265 wait complete. 2023-10-21T16:14:25.543 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-21T16:14:25.544 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2023-10-21T16:14:25.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.767+0000 7f36eb802640 1 Processor -- start 2023-10-21T16:14:25.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.767+0000 7f36eb802640 1 -- start start 2023-10-21T16:14:25.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.768+0000 7f36eb802640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36e406dfc0 0x7f36e4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.768+0000 7f36eb802640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36e4071070 0x7f36e40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.768+0000 7f36eb802640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e40b1cc0 con 0x7f36e40afbe0 2023-10-21T16:14:25.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.768+0000 7f36eb802640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40b2620 con 0x7f36e4071070 2023-10-21T16:14:25.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.768+0000 7f36eb802640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40b2060 con 0x7f36e406dfc0 2023-10-21T16:14:25.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36e4071070 0x7f36e40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f36e4071070 0x7f36e40cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47596/0 (socket says 172.21.15.53:47596) 2023-10-21T16:14:25.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1413844596 learned_addr learned my addr 172.21.15.53:0/1413844596 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:25.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e8d9d640 1 --2- 172.21.15.53:0/1413844596 >> v2:172.21.15.124:3300/0 conn(0x7f36e406dfc0 0x7f36e4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1413844596 >> v2:172.21.15.124:3300/0 conn(0x7f36e406dfc0 msgr2=0x7f36e4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 --2- 172.21.15.53:0/1413844596 >> v2:172.21.15.124:3300/0 conn(0x7f36e406dfc0 0x7f36e4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1413844596 >> v1:172.21.15.124:6789/0 conn(0x7f36e40afbe0 legacy=0x7f36e40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:25.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.769+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1413844596 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40b2340 con 0x7f36e4071070 2023-10-21T16:14:25.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.770+0000 7f36e9d9f640 1 --2- 172.21.15.53:0/1413844596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e4071070 0x7f36e40cb130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36e40b3980 tx=0x7f36dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2277cf15c59a835e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.771+0000 7f36dbfff640 1 -- 172.21.15.53:0/1413844596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc02ca80 con 0x7f36e4071070 2023-10-21T16:14:25.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.771+0000 7f36dbfff640 1 -- 172.21.15.53:0/1413844596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc02cbe0 con 0x7f36e4071070 2023-10-21T16:14:25.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.771+0000 7f36dbfff640 1 -- 172.21.15.53:0/1413844596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0343f0 con 0x7f36e4071070 2023-10-21T16:14:25.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.771+0000 7f36eb802640 1 -- 172.21.15.53:0/1413844596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e4071070 msgr2=0x7f36e40cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.771+0000 7f36eb802640 1 --2- 172.21.15.53:0/1413844596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e4071070 0x7f36e40cb130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36e40b3980 tx=0x7f36dc02bee0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 -- 172.21.15.53:0/1413844596 shutdown_connections 2023-10-21T16:14:25.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 --2- 172.21.15.53:0/1413844596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e4071070 0x7f36e40cb130 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 --2- 172.21.15.53:0/1413844596 >> v2:172.21.15.124:3300/0 conn(0x7f36e406dfc0 0x7f36e4070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 -- 172.21.15.53:0/1413844596 >> 172.21.15.53:0/1413844596 conn(0x7f36e406a5d0 msgr2=0x7f36e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:25.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 -- 172.21.15.53:0/1413844596 shutdown_connections 2023-10-21T16:14:25.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.772+0000 7f36eb802640 1 -- 172.21.15.53:0/1413844596 wait complete. 2023-10-21T16:14:25.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.773+0000 7f36eb802640 1 Processor -- start 2023-10-21T16:14:25.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 -- start start 2023-10-21T16:14:25.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 0x7f36e40c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e4071070 0x7f36e41f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4147380 con 0x7f36e40afbe0 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40c3540 con 0x7f36e406dfc0 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.774+0000 7f36eb802640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40c3820 con 0x7f36e4071070 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47598/0 (socket says 172.21.15.53:47598) 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1966691463 learned_addr learned my addr 172.21.15.53:0/1966691463 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e4071070 msgr2=0x7f36e41f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e4071070 0x7f36e41f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 msgr2=0x7f36e40c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e959e640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 0x7f36e40c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 0x7f36e40c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e9d9f640 1 -- 172.21.15.53:0/1966691463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc009b30 con 0x7f36e40afbe0 2023-10-21T16:14:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.775+0000 7f36e959e640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 0x7f36e40c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36e9d9f640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f36dc036040 tx=0x7f36dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc009510 con 0x7f36e40afbe0 2023-10-21T16:14:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc03e450 con 0x7f36e40afbe0 2023-10-21T16:14:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc03e7e0 con 0x7f36e40afbe0 2023-10-21T16:14:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e40c3b00 con 0x7f36e40afbe0 2023-10-21T16:14:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.776+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e40c3180 con 0x7f36e40afbe0 2023-10-21T16:14:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.777+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4147530 con 0x7f36e40afbe0 2023-10-21T16:14:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.778+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f36dc02cd50 con 0x7f36e40afbe0 2023-10-21T16:14:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.779+0000 7f36d9ffb640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 0x7f36c806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.779+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f36dc0ac7a0 con 0x7f36e40afbe0 2023-10-21T16:14:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.780+0000 7f36e959e640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 0x7f36c806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.781+0000 7f36e959e640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 0x7f36c806d530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36e000a810 tx=0x7f36e0008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.784+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36e4147530 con 0x7f36e40afbe0 2023-10-21T16:14:25.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:25.997+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f36e4070630 con 0x7f36c806b070 2023-10-21T16:14:26.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.170+0000 7f36d9ffb640 1 -- 172.21.15.53:0/1966691463 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f36e000ad90 con 0x7f36c806b070 2023-10-21T16:14:26.172 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-21T16:14:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 msgr2=0x7f36c806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 0x7f36c806d530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36e000a810 tx=0x7f36e0008040 comp rx=0 tx=0).stop 2023-10-21T16:14:26.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 msgr2=0x7f36e40cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f36dc036040 tx=0x7f36dc004a40 comp rx=0 tx=0).stop 2023-10-21T16:14:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 shutdown_connections 2023-10-21T16:14:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f36c806b070 0x7f36c806d530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e40afbe0 0x7f36e40cd8c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36e4071070 0x7f36e41f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 --2- 172.21.15.53:0/1966691463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36e406dfc0 0x7f36e40c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.173+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 >> 172.21.15.53:0/1966691463 conn(0x7f36e406a5d0 msgr2=0x7f36e40c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.174+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 shutdown_connections 2023-10-21T16:14:26.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.174+0000 7f36eb802640 1 -- 172.21.15.53:0/1966691463 wait complete. 2023-10-21T16:14:26.196 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-21T16:14:26.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2023-10-21T16:14:26.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.428+0000 7f229cadc640 1 Processor -- start 2023-10-21T16:14:26.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.428+0000 7f229cadc640 1 -- start start 2023-10-21T16:14:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f229cadc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22980cf2b0 0x7f22980d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f229cadc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22980ce510 0x7f229815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f229cadc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22980cd520 con 0x7f22980cf690 2023-10-21T16:14:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f229cadc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980d1910 con 0x7f22980cf2b0 2023-10-21T16:14:26.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f229cadc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980ce080 con 0x7f22980ce510 2023-10-21T16:14:26.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.430+0000 7f22977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22980cf2b0 0x7f22980d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22980cf2b0 0x7f22980d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47606/0 (socket says 172.21.15.53:47606) 2023-10-21T16:14:26.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 -- 172.21.15.53:0/3261448705 learned_addr learned my addr 172.21.15.53:0/3261448705 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f2297fff640 1 --2- 172.21.15.53:0/3261448705 >> v2:172.21.15.124:3300/0 conn(0x7f22980ce510 0x7f229815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 -- 172.21.15.53:0/3261448705 >> v2:172.21.15.124:3300/0 conn(0x7f22980ce510 msgr2=0x7f229815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 --2- 172.21.15.53:0/3261448705 >> v2:172.21.15.124:3300/0 conn(0x7f22980ce510 0x7f229815c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 -- 172.21.15.53:0/3261448705 >> v1:172.21.15.53:6789/0 conn(0x7f22980cf690 legacy=0x7f22980d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22977fe640 1 -- 172.21.15.53:0/3261448705 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980cd750 con 0x7f22980cf2b0 2023-10-21T16:14:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.431+0000 7f22967fc640 1 -- 172.21.15.53:0/3261448705 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479412904 0 0) 0x7f22980cd520 con 0x7f22980cf690 2023-10-21T16:14:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.432+0000 7f22977fe640 1 --2- 172.21.15.53:0/3261448705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22980d34d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f22980cfb20 tx=0x7f228802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ae5bc5075e17ed server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.432+0000 7f22967fc640 1 -- 172.21.15.53:0/3261448705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288031880 con 0x7f22980cf2b0 2023-10-21T16:14:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f22967fc640 1 -- 172.21.15.53:0/3261448705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22880319e0 con 0x7f22980cf2b0 2023-10-21T16:14:26.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f22967fc640 1 -- 172.21.15.53:0/3261448705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288031d70 con 0x7f22980cf2b0 2023-10-21T16:14:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 -- 172.21.15.53:0/3261448705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 msgr2=0x7f22980d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 --2- 172.21.15.53:0/3261448705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22980d34d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f22980cfb20 tx=0x7f228802b920 comp rx=0 tx=0).stop 2023-10-21T16:14:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 -- 172.21.15.53:0/3261448705 shutdown_connections 2023-10-21T16:14:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 --2- 172.21.15.53:0/3261448705 >> v2:172.21.15.124:3300/0 conn(0x7f22980ce510 0x7f229815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 --2- 172.21.15.53:0/3261448705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22980d34d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 -- 172.21.15.53:0/3261448705 >> 172.21.15.53:0/3261448705 conn(0x7f229809bc40 msgr2=0x7f22980d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 -- 172.21.15.53:0/3261448705 shutdown_connections 2023-10-21T16:14:26.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.433+0000 7f229cadc640 1 -- 172.21.15.53:0/3261448705 wait complete. 2023-10-21T16:14:26.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.435+0000 7f229cadc640 1 Processor -- start 2023-10-21T16:14:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.435+0000 7f229cadc640 1 -- start start 2023-10-21T16:14:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22980ce510 0x7f22980cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22980cf690 0x7f22980c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2298159810 con 0x7f22980cf2b0 2023-10-21T16:14:26.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2298159af0 con 0x7f22980ce510 2023-10-21T16:14:26.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f229cadc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2298159dd0 con 0x7f22980cf690 2023-10-21T16:14:26.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f2296ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f2296ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47618/0 (socket says 172.21.15.53:47618) 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.436+0000 7f2296ffd640 1 -- 172.21.15.53:0/3262578064 learned_addr learned my addr 172.21.15.53:0/3262578064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f22977fe640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22980ce510 0x7f22980cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2297fff640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22980cf690 0x7f22980c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 -- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22980cf690 msgr2=0x7f22980c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22980cf690 0x7f22980c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 -- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22980ce510 msgr2=0x7f22980cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22980ce510 0x7f22980cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 -- 172.21.15.53:0/3262578064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880094d0 con 0x7f22980cf2b0 2023-10-21T16:14:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f2296ffd640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f228c00c410 tx=0x7f228c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.437+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c0035f0 con 0x7f22980cf2b0 2023-10-21T16:14:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.438+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c003770 con 0x7f22980cf2b0 2023-10-21T16:14:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.438+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c003b20 con 0x7f22980cf2b0 2023-10-21T16:14:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.438+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229815a0b0 con 0x7f22980cf2b0 2023-10-21T16:14:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.438+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980cb350 con 0x7f22980cf2b0 2023-10-21T16:14:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.439+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229815a5d0 con 0x7f22980cf2b0 2023-10-21T16:14:26.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.440+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f228c003c80 con 0x7f22980cf2b0 2023-10-21T16:14:26.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.441+0000 7f227ffff640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 0x7f228406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:26.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.441+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f228c089220 con 0x7f22980cf2b0 2023-10-21T16:14:26.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.442+0000 7f22977fe640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 0x7f228406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:26.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.443+0000 7f22977fe640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 0x7f228406d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2288036000 tx=0x7f2288008030 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:26.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.446+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f229815a5d0 con 0x7f22980cf2b0 2023-10-21T16:14:26.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.663+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f22980cdd10 con 0x7f228406b070 2023-10-21T16:14:26.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.836+0000 7f227ffff640 1 -- 172.21.15.53:0/3262578064 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2288031cf0 con 0x7f228406b070 2023-10-21T16:14:26.838 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-21T16:14:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 msgr2=0x7f228406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 0x7f228406d530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2288036000 tx=0x7f2288008030 comp rx=0 tx=0).stop 2023-10-21T16:14:26.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 msgr2=0x7f22981e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f228c00c410 tx=0x7f228c00c8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 shutdown_connections 2023-10-21T16:14:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f228406b070 0x7f228406d530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22980cf690 0x7f22980c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22980cf2b0 0x7f22981e6830 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 --2- 172.21.15.53:0/3262578064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22980ce510 0x7f22980cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.840+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 >> 172.21.15.53:0/3262578064 conn(0x7f229809bc40 msgr2=0x7f229806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.841+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 shutdown_connections 2023-10-21T16:14:26.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:26.841+0000 7f229cadc640 1 -- 172.21.15.53:0/3262578064 wait complete. 2023-10-21T16:14:26.861 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-21T16:14:26.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2023-10-21T16:14:27.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.090+0000 7f7542173640 1 Processor -- start 2023-10-21T16:14:27.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.091+0000 7f7542173640 1 -- start start 2023-10-21T16:14:27.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.092+0000 7f7542173640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f753c0b0680 0x7f753c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.092+0000 7f7542173640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0c94f0 con 0x7f753c0c8b40 2023-10-21T16:14:27.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.092+0000 7f7542173640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0ccbc0 con 0x7f753c15cae0 2023-10-21T16:14:27.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.092+0000 7f7542173640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c0ce1f0 con 0x7f753c0b0680 2023-10-21T16:14:27.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.093+0000 7f7540970640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f753c0c8b40 0x7f753c0aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53862/0 (socket says 172.21.15.53:53862) 2023-10-21T16:14:27.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.093+0000 7f7540970640 1 -- 172.21.15.53:0/1512890528 learned_addr learned my addr 172.21.15.53:0/1512890528 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:27.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.093+0000 7f7541972640 1 --2- 172.21.15.53:0/1512890528 >> v2:172.21.15.124:3300/0 conn(0x7f753c0b0680 0x7f753c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.093+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336857905 0 0) 0x7f753c0c94f0 con 0x7f753c0c8b40 2023-10-21T16:14:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.093+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7524003520 con 0x7f753c0c8b40 2023-10-21T16:14:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1679782400 0 0) 0x7f7524003520 con 0x7f753c0c8b40 2023-10-21T16:14:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 >> v2:172.21.15.124:3300/0 conn(0x7f753c0b0680 msgr2=0x7f753c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 --2- 172.21.15.53:0/1512890528 >> v2:172.21.15.124:3300/0 conn(0x7f753c0b0680 0x7f753c0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 >> v1:172.21.15.124:6789/0 conn(0x7f753c15cae0 legacy=0x7f753c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0c9720 con 0x7f753c0c8b40 2023-10-21T16:14:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7530001670 con 0x7f753c0c8b40 2023-10-21T16:14:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 >> v1:172.21.15.53:6789/0 conn(0x7f753c0c8b40 legacy=0x7f753c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7524003970 0x7f7524003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7524005180 0x7f752400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0ce1f0 con 0x7f75240047c0 2023-10-21T16:14:27.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752400c020 con 0x7f7524003970 2023-10-21T16:14:27.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752400c1e0 con 0x7f7524005180 2023-10-21T16:14:27.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f7540970640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.094+0000 7f7541171640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7524003970 0x7f7524003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7541972640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7524005180 0x7f752400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7540970640 1 -- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7524005180 msgr2=0x7f752400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:27.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7540970640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7524005180 0x7f752400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7540970640 1 -- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7524003970 msgr2=0x7f7524003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:27.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7540970640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7524003970 0x7f7524003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7541972640 1 -- 172.21.15.53:0/1512890528 reap_dead start 2023-10-21T16:14:27.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.095+0000 7f7540970640 1 -- 172.21.15.53:0/1512890528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524003520 con 0x7f75240047c0 2023-10-21T16:14:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.096+0000 7f7540970640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 0x7f7524004c40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f753000c240 tx=0x7f75240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a392219c8f7381b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.096+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530005ad0 con 0x7f75240047c0 2023-10-21T16:14:27.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.096+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7530005c30 con 0x7f75240047c0 2023-10-21T16:14:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.096+0000 7f753bfff640 1 -- 172.21.15.53:0/1512890528 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753002ca00 con 0x7f75240047c0 2023-10-21T16:14:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 -- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 msgr2=0x7f7524004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 0x7f7524004c40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f753000c240 tx=0x7f75240057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 -- 172.21.15.53:0/1512890528 shutdown_connections 2023-10-21T16:14:27.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 --2- 172.21.15.53:0/1512890528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75240047c0 0x7f7524004c40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 -- 172.21.15.53:0/1512890528 >> 172.21.15.53:0/1512890528 conn(0x7f753c06c050 msgr2=0x7f753c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:27.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 -- 172.21.15.53:0/1512890528 shutdown_connections 2023-10-21T16:14:27.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.097+0000 7f7542173640 1 -- 172.21.15.53:0/1512890528 wait complete. 2023-10-21T16:14:27.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.099+0000 7f7542173640 1 Processor -- start 2023-10-21T16:14:27.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.099+0000 7f7542173640 1 -- start start 2023-10-21T16:14:27.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75240047c0 0x7f753c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f753c159ba0 0x7f753c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c151a90 con 0x7f753c151610 2023-10-21T16:14:27.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c15aa50 con 0x7f753c159ba0 2023-10-21T16:14:27.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7542173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c15a490 con 0x7f75240047c0 2023-10-21T16:14:27.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7540970640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7540970640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47626/0 (socket says 172.21.15.53:47626) 2023-10-21T16:14:27.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.100+0000 7f7540970640 1 -- 172.21.15.53:0/1397704177 learned_addr learned my addr 172.21.15.53:0/1397704177 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:27.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7541171640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75240047c0 0x7f753c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7541972640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f753c159ba0 0x7f753c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 -- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75240047c0 msgr2=0x7f753c1510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75240047c0 0x7f753c1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 -- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f753c159ba0 msgr2=0x7f753c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f753c159ba0 0x7f753c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 -- 172.21.15.53:0/1397704177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c15a770 con 0x7f753c151610 2023-10-21T16:14:27.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7540970640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7530001930 tx=0x7f753002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753003c040 con 0x7f753c151610 2023-10-21T16:14:27.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753002cbd0 con 0x7f753c151610 2023-10-21T16:14:27.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.101+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c0c56f0 con 0x7f753c151610 2023-10-21T16:14:27.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.102+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530007070 con 0x7f753c151610 2023-10-21T16:14:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.102+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c14f240 con 0x7f753c151610 2023-10-21T16:14:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.103+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c074150 con 0x7f753c151610 2023-10-21T16:14:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.104+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f7530032070 con 0x7f753c151610 2023-10-21T16:14:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.105+0000 7f7539ffb640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 0x7f752806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.105+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f75300aae70 con 0x7f753c151610 2023-10-21T16:14:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.105+0000 7f7541171640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 0x7f752806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.107+0000 7f7541171640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 0x7f752806d560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.109+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7530075ad0 con 0x7f753c151610 2023-10-21T16:14:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.328+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f753c0cc850 con 0x7f752806b0a0 2023-10-21T16:14:27.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.504+0000 7f7539ffb640 1 -- 172.21.15.53:0/1397704177 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f752c00d050 con 0x7f752806b0a0 2023-10-21T16:14:27.506 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-21T16:14:27.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 msgr2=0x7f752806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 0x7f752806d560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f752c007fd0 tx=0x7f752c009040 comp rx=0 tx=0).stop 2023-10-21T16:14:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 msgr2=0x7f753c159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7530001930 tx=0x7f753002e070 comp rx=0 tx=0).stop 2023-10-21T16:14:27.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 shutdown_connections 2023-10-21T16:14:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f752806b0a0 0x7f752806d560 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f753c159ba0 0x7f753c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f753c151610 0x7f753c159660 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 --2- 172.21.15.53:0/1397704177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75240047c0 0x7f753c1510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 >> 172.21.15.53:0/1397704177 conn(0x7f753c06c050 msgr2=0x7f753c0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 shutdown_connections 2023-10-21T16:14:27.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.508+0000 7f7542173640 1 -- 172.21.15.53:0/1397704177 wait complete. 2023-10-21T16:14:27.529 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-21T16:14:27.529 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2023-10-21T16:14:27.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.761+0000 7fd6d759e640 1 Processor -- start 2023-10-21T16:14:27.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.761+0000 7fd6d759e640 1 -- start start 2023-10-21T16:14:27.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 0x7fd6d80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 0x7fd6d80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6d80aff70 0x7fd6d80b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80c94e0 con 0x7fd6d80aff70 2023-10-21T16:14:27.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d80ccbc0 con 0x7fd6d80c8b40 2023-10-21T16:14:27.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.763+0000 7fd6d759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80b0940 con 0x7fd6d815cae0 2023-10-21T16:14:27.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6d80aff70 0x7fd6d80b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6d80aff70 0x7fd6d80b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47634/0 (socket says 172.21.15.53:47634) 2023-10-21T16:14:27.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/4159099002 learned_addr learned my addr 172.21.15.53:0/4159099002 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:27.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d5d9b640 1 --2- 172.21.15.53:0/4159099002 >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 0x7fd6d80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d659c640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 0x7fd6d80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/4159099002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 msgr2=0x7fd6d80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 0x7fd6d80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/4159099002 >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 msgr2=0x7fd6d80c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.764+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/4159099002 >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 0x7fd6d80c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.765+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/4159099002 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80ce1f0 con 0x7fd6d80aff70 2023-10-21T16:14:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.765+0000 7fd6d5d9b640 1 --2- 172.21.15.53:0/4159099002 >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 0x7fd6d80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:14:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.765+0000 7fd6d659c640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 0x7fd6d80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:14:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.766+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d80aff70 0x7fd6d80b0400 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6d806e550 tx=0x7fd6d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d14bf4e65e5cddb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.767+0000 7fd6d559a640 1 -- 172.21.15.53:0/4159099002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d002ca70 con 0x7fd6d80aff70 2023-10-21T16:14:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.767+0000 7fd6d559a640 1 -- 172.21.15.53:0/4159099002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d002cbd0 con 0x7fd6d80aff70 2023-10-21T16:14:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.767+0000 7fd6d559a640 1 -- 172.21.15.53:0/4159099002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00343f0 con 0x7fd6d80aff70 2023-10-21T16:14:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.767+0000 7fd6d759e640 1 -- 172.21.15.53:0/4159099002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d80aff70 msgr2=0x7fd6d80b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.767+0000 7fd6d759e640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d80aff70 0x7fd6d80b0400 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6d806e550 tx=0x7fd6d002bed0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 -- 172.21.15.53:0/4159099002 shutdown_connections 2023-10-21T16:14:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d80aff70 0x7fd6d80b0400 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 --2- 172.21.15.53:0/4159099002 >> v2:172.21.15.124:3300/0 conn(0x7fd6d80c8b40 0x7fd6d80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 --2- 172.21.15.53:0/4159099002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d815cae0 0x7fd6d80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 -- 172.21.15.53:0/4159099002 >> 172.21.15.53:0/4159099002 conn(0x7fd6d806c050 msgr2=0x7fd6d806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 -- 172.21.15.53:0/4159099002 shutdown_connections 2023-10-21T16:14:27.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.768+0000 7fd6d759e640 1 -- 172.21.15.53:0/4159099002 wait complete. 2023-10-21T16:14:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.770+0000 7fd6d759e640 1 Processor -- start 2023-10-21T16:14:27.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.771+0000 7fd6d759e640 1 -- start start 2023-10-21T16:14:27.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d80aff70 0x7fd6d815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d80c8b40 0x7fd6d80c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c2ee0 con 0x7fd6d815cae0 2023-10-21T16:14:27.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c36c0 con 0x7fd6d80c8b40 2023-10-21T16:14:27.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d80c39a0 con 0x7fd6d80aff70 2023-10-21T16:14:27.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47638/0 (socket says 172.21.15.53:47638) 2023-10-21T16:14:27.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.772+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/243033546 learned_addr learned my addr 172.21.15.53:0/243033546 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:27.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d659c640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d80aff70 0x7fd6d815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d5d9b640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d80c8b40 0x7fd6d80c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d80aff70 msgr2=0x7fd6d815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d80aff70 0x7fd6d815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d80c8b40 msgr2=0x7fd6d80c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d80c8b40 0x7fd6d80c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:27.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 -- 172.21.15.53:0/243033546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0009b30 con 0x7fd6d815cae0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.773+0000 7fd6d6d9d640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0036040 tx=0x7fd6d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.774+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0009510 con 0x7fd6d815cae0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.774+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d003e450 con 0x7fd6d815cae0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.774+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d80c3c80 con 0x7fd6d815cae0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.774+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d003e7e0 con 0x7fd6d815cae0 2023-10-21T16:14:27.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.774+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d80c4240 con 0x7fd6d815cae0 2023-10-21T16:14:27.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.775+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d80c4430 con 0x7fd6d815cae0 2023-10-21T16:14:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.776+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd6d003e940 con 0x7fd6d815cae0 2023-10-21T16:14:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.777+0000 7fd6c6ffd640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 0x7fd6b806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.777+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd6d00ac9e0 con 0x7fd6d815cae0 2023-10-21T16:14:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.778+0000 7fd6d659c640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 0x7fd6b806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.779+0000 7fd6d659c640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 0x7fd6b806d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:27.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.782+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6d0077640 con 0x7fd6d815cae0 2023-10-21T16:14:28.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:27.999+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d80c9720 con 0x7fd6b806b070 2023-10-21T16:14:28.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.162+0000 7fd6c6ffd640 1 -- 172.21.15.53:0/243033546 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd6c800de20 con 0x7fd6b806b070 2023-10-21T16:14:28.164 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-21T16:14:28.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 msgr2=0x7fd6b806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 0x7fd6b806d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8008040 comp rx=0 tx=0).stop 2023-10-21T16:14:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 msgr2=0x7fd6d81f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0036040 tx=0x7fd6d0004a40 comp rx=0 tx=0).stop 2023-10-21T16:14:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 shutdown_connections 2023-10-21T16:14:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fd6b806b070 0x7fd6b806d530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6d815cae0 0x7fd6d81f5690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6d80c8b40 0x7fd6d80c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 --2- 172.21.15.53:0/243033546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6d80aff70 0x7fd6d815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 >> 172.21.15.53:0/243033546 conn(0x7fd6d806c050 msgr2=0x7fd6d80af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:28.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.166+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 shutdown_connections 2023-10-21T16:14:28.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.167+0000 7fd6d759e640 1 -- 172.21.15.53:0/243033546 wait complete. 2023-10-21T16:14:28.191 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-21T16:14:28.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2023-10-21T16:14:28.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.404+0000 7f1ef554e640 1 Processor -- start 2023-10-21T16:14:28.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.405+0000 7f1ef554e640 1 -- start start 2023-10-21T16:14:28.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.406+0000 7f1ef554e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1ef0070cc0 0x7f1ef00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1ef554e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef0071170 con 0x7f1ef00cb450 2023-10-21T16:14:28.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1ef554e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006e150 con 0x7f1ef0160660 2023-10-21T16:14:28.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1ef554e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef006de70 con 0x7f1ef0070cc0 2023-10-21T16:14:28.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1eef7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1ef00cb450 0x7f1ef00b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53876/0 (socket says 172.21.15.53:53876) 2023-10-21T16:14:28.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1eef7fe640 1 -- 172.21.15.53:0/3461460432 learned_addr learned my addr 172.21.15.53:0/3461460432 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:28.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/3461460432 >> v2:172.21.15.124:3300/0 conn(0x7f1ef0070cc0 0x7f1ef00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764321493 0 0) 0x7f1ee8001490 con 0x7f1ef00cb450 2023-10-21T16:14:28.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.407+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1edc003520 con 0x7f1ef00cb450 2023-10-21T16:14:28.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637322170 0 0) 0x7f1edc003520 con 0x7f1ef00cb450 2023-10-21T16:14:28.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 >> v2:172.21.15.124:3300/0 conn(0x7f1ef0070cc0 msgr2=0x7f1ef00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:28.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 --2- 172.21.15.53:0/3461460432 >> v2:172.21.15.124:3300/0 conn(0x7f1ef0070cc0 0x7f1ef00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 >> v1:172.21.15.124:6789/0 conn(0x7f1ef0160660 legacy=0x7f1ef00c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00c8e30 con 0x7f1ef00cb450 2023-10-21T16:14:28.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1ee8001b80 con 0x7f1ef00cb450 2023-10-21T16:14:28.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 >> v1:172.21.15.53:6789/0 conn(0x7f1ef00cb450 legacy=0x7f1ef00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1edc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1edc005180 0x7f1edc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef006de70 con 0x7f1edc003970 2023-10-21T16:14:28.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00c050 con 0x7f1edc005180 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc00c210 con 0x7f1edc0047c0 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.408+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1edc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eef7fe640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1edc005180 0x7f1edc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eeeffd640 1 -- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1edc0047c0 msgr2=0x7f1edc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eeeffd640 1 -- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1edc005180 msgr2=0x7f1edc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:28.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1edc005180 0x7f1edc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eeeffd640 1 -- 172.21.15.53:0/3461460432 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc003520 con 0x7f1edc003970 2023-10-21T16:14:28.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1eef7fe640 1 -- 172.21.15.53:0/3461460432 reap_dead start 2023-10-21T16:14:28.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.409+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1edc0047c0 0x7f1edc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:28.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.410+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1edc003db0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0001750 tx=0x7f1edc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e8761a29a4cd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.410+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00052d0 con 0x7f1edc003970 2023-10-21T16:14:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.410+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee002e050 con 0x7f1edc003970 2023-10-21T16:14:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.410+0000 7f1eee7fc640 1 -- 172.21.15.53:0/3461460432 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00035f0 con 0x7f1edc003970 2023-10-21T16:14:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 -- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 msgr2=0x7f1edc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1edc003db0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0001750 tx=0x7f1edc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 -- 172.21.15.53:0/3461460432 shutdown_connections 2023-10-21T16:14:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 --2- 172.21.15.53:0/3461460432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1edc003db0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 -- 172.21.15.53:0/3461460432 >> 172.21.15.53:0/3461460432 conn(0x7f1ef006a5d0 msgr2=0x7f1ef00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 -- 172.21.15.53:0/3461460432 shutdown_connections 2023-10-21T16:14:28.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.411+0000 7f1ef554e640 1 -- 172.21.15.53:0/3461460432 wait complete. 2023-10-21T16:14:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.413+0000 7f1ef554e640 1 Processor -- start 2023-10-21T16:14:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.413+0000 7f1ef554e640 1 -- start start 2023-10-21T16:14:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef0154420 0x7f1ef00ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef0147760 0x7f1ef0147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0148120 con 0x7f1edc003970 2023-10-21T16:14:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01597c0 con 0x7f1ef0147760 2023-10-21T16:14:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.414+0000 7f1ef554e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0158480 con 0x7f1ef0154420 2023-10-21T16:14:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47650/0 (socket says 172.21.15.53:47650) 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 -- 172.21.15.53:0/4121998314 learned_addr learned my addr 172.21.15.53:0/4121998314 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef0154420 0x7f1ef00ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eef7fe640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef0147760 0x7f1ef0147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 -- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef0154420 msgr2=0x7f1ef00ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef0154420 0x7f1ef00ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 -- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef0147760 msgr2=0x7f1ef0147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef0147760 0x7f1ef0147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 -- 172.21.15.53:0/4121998314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0158720 con 0x7f1edc003970 2023-10-21T16:14:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.415+0000 7f1eeeffd640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.416+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00076c0 con 0x7f1edc003970 2023-10-21T16:14:28.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.416+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0007a30 con 0x7f1edc003970 2023-10-21T16:14:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.416+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0006be0 con 0x7f1edc003970 2023-10-21T16:14:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.416+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef0158a00 con 0x7f1edc003970 2023-10-21T16:14:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.416+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef0158fc0 con 0x7f1edc003970 2023-10-21T16:14:28.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.417+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef004e160 con 0x7f1edc003970 2023-10-21T16:14:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.418+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f1ee0006d40 con 0x7f1edc003970 2023-10-21T16:14:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.419+0000 7f1ee77fe640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 0x7f1ed006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:28.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.419+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f1ee00ab970 con 0x7f1edc003970 2023-10-21T16:14:28.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.420+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 0x7f1ed006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:28.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.421+0000 7f1ee7fff640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 0x7f1ed006d560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0154970 tx=0x7f1ed8008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:28.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.423+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ee0076390 con 0x7f1edc003970 2023-10-21T16:14:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.645+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ef00c93f0 con 0x7f1ed006b0a0 2023-10-21T16:14:28.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.816+0000 7f1ee77fe640 1 -- 172.21.15.53:0/4121998314 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1ed8009620 con 0x7f1ed006b0a0 2023-10-21T16:14:28.819 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-21T16:14:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 msgr2=0x7f1ed006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 0x7f1ed006d560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0154970 tx=0x7f1ed8008040 comp rx=0 tx=0).stop 2023-10-21T16:14:28.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 msgr2=0x7f1ef0157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:28.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee0001120 comp rx=0 tx=0).stop 2023-10-21T16:14:28.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 shutdown_connections 2023-10-21T16:14:28.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f1ed006b0a0 0x7f1ed006d560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef0147760 0x7f1ef0147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef0154420 0x7f1ef00ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 --2- 172.21.15.53:0/4121998314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1edc003970 0x7f1ef0157f40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:28.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 >> 172.21.15.53:0/4121998314 conn(0x7f1ef006a5d0 msgr2=0x7f1ef015e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:28.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 shutdown_connections 2023-10-21T16:14:28.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:28.820+0000 7f1ef554e640 1 -- 172.21.15.53:0/4121998314 wait complete. 2023-10-21T16:14:28.841 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-21T16:14:28.841 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2023-10-21T16:14:29.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.072+0000 7f42bc8d1640 1 Processor -- start 2023-10-21T16:14:29.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.072+0000 7f42bc8d1640 1 -- start start 2023-10-21T16:14:29.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42bc8d1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42b815e400 0x7f42b80cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42bc8d1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42b815b4b0 0x7f42b815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42bc8d1640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80b63e0 con 0x7f42b81585a0 2023-10-21T16:14:29.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42bc8d1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b815be00 con 0x7f42b815b4b0 2023-10-21T16:14:29.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42bc8d1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b815e8b0 con 0x7f42b815e400 2023-10-21T16:14:29.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42b7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42b815b4b0 0x7f42b815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42b7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42b815b4b0 0x7f42b815b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47658/0 (socket says 172.21.15.53:47658) 2023-10-21T16:14:29.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.074+0000 7f42b7fff640 1 -- 172.21.15.53:0/823781072 learned_addr learned my addr 172.21.15.53:0/823781072 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:29.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.075+0000 7f42affff640 1 --2- 172.21.15.53:0/823781072 >> v2:172.21.15.124:3300/0 conn(0x7f42b815e400 0x7f42b80cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.075+0000 7f42b7fff640 1 -- 172.21.15.53:0/823781072 >> v2:172.21.15.124:3300/0 conn(0x7f42b815e400 msgr2=0x7f42b80cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:29.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.075+0000 7f42b7fff640 1 --2- 172.21.15.53:0/823781072 >> v2:172.21.15.124:3300/0 conn(0x7f42b815e400 0x7f42b80cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.075+0000 7f42b7fff640 1 -- 172.21.15.53:0/823781072 >> v1:172.21.15.124:6789/0 conn(0x7f42b81585a0 legacy=0x7f42b80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.075+0000 7f42b7fff640 1 -- 172.21.15.53:0/823781072 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b815b020 con 0x7f42b815b4b0 2023-10-21T16:14:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.076+0000 7f42b7fff640 1 --2- 172.21.15.53:0/823781072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b815b8c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f42b80afde0 tx=0x7f42b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad368ecefceff45 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.076+0000 7f42b6ffd640 1 -- 172.21.15.53:0/823781072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b002ca80 con 0x7f42b815b4b0 2023-10-21T16:14:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42b6ffd640 1 -- 172.21.15.53:0/823781072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b002cbe0 con 0x7f42b815b4b0 2023-10-21T16:14:29.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42b6ffd640 1 -- 172.21.15.53:0/823781072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00343f0 con 0x7f42b815b4b0 2023-10-21T16:14:29.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 -- 172.21.15.53:0/823781072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 msgr2=0x7f42b815b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/823781072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b815b8c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f42b80afde0 tx=0x7f42b002bee0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 -- 172.21.15.53:0/823781072 shutdown_connections 2023-10-21T16:14:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/823781072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b815b8c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/823781072 >> v2:172.21.15.124:3300/0 conn(0x7f42b815e400 0x7f42b80cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.077+0000 7f42bc8d1640 1 -- 172.21.15.53:0/823781072 >> 172.21.15.53:0/823781072 conn(0x7f42b80cd180 msgr2=0x7f42b80cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:29.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.078+0000 7f42bc8d1640 1 -- 172.21.15.53:0/823781072 shutdown_connections 2023-10-21T16:14:29.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.078+0000 7f42bc8d1640 1 -- 172.21.15.53:0/823781072 wait complete. 2023-10-21T16:14:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.079+0000 7f42bc8d1640 1 Processor -- start 2023-10-21T16:14:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.080+0000 7f42bc8d1640 1 -- start start 2023-10-21T16:14:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42b81585a0 0x7f42b8156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 0x7f42b80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c55f0 con 0x7f42b815b4b0 2023-10-21T16:14:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c5840 con 0x7f42b81585a0 2023-10-21T16:14:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42bc8d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c5b20 con 0x7f42b815e400 2023-10-21T16:14:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47666/0 (socket says 172.21.15.53:47666) 2023-10-21T16:14:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 -- 172.21.15.53:0/1606866152 learned_addr learned my addr 172.21.15.53:0/1606866152 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42b77fe640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42b81585a0 0x7f42b8156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42b7fff640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 0x7f42b80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 -- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 msgr2=0x7f42b80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 0x7f42b80c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 -- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42b81585a0 msgr2=0x7f42b8156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.081+0000 7f42affff640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42b81585a0 0x7f42b8156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42affff640 1 -- 172.21.15.53:0/1606866152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0009b30 con 0x7f42b815b4b0 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42b7fff640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 0x7f42b80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42affff640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f42a000d5f0 tx=0x7f42a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a00125f0 con 0x7f42b815b4b0 2023-10-21T16:14:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0012750 con 0x7f42b815b4b0 2023-10-21T16:14:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0012b00 con 0x7f42b815b4b0 2023-10-21T16:14:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80c5e00 con 0x7f42b815b4b0 2023-10-21T16:14:29.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.082+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b80c63c0 con 0x7f42b815b4b0 2023-10-21T16:14:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.083+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b8074150 con 0x7f42b815b4b0 2023-10-21T16:14:29.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.084+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f42a0012c60 con 0x7f42b815b4b0 2023-10-21T16:14:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.085+0000 7f42b4ff9640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 0x7f429806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.085+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f42a008a170 con 0x7f42b815b4b0 2023-10-21T16:14:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.086+0000 7f42b77fe640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 0x7f429806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.087+0000 7f42b77fe640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 0x7f429806d530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42a8007fd0 tx=0x7f42a8009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.089+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42a0054ca0 con 0x7f42b815b4b0 2023-10-21T16:14:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.310+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f42b80c5c90 con 0x7f429806b070 2023-10-21T16:14:29.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.486+0000 7f42b4ff9640 1 -- 172.21.15.53:0/1606866152 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f42b80c5c90 con 0x7f429806b070 2023-10-21T16:14:29.488 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-21T16:14:29.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.490+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 msgr2=0x7f429806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.490+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 0x7f429806d530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42a8007fd0 tx=0x7f42a8009040 comp rx=0 tx=0).stop 2023-10-21T16:14:29.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.490+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 msgr2=0x7f42b80c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.490+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f42a000d5f0 tx=0x7f42a000dab0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.490+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 shutdown_connections 2023-10-21T16:14:29.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f429806b070 0x7f429806d530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42b815e400 0x7f42b80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42b815b4b0 0x7f42b80c4c20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 --2- 172.21.15.53:0/1606866152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42b81585a0 0x7f42b8156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 >> 172.21.15.53:0/1606866152 conn(0x7f42b80cd180 msgr2=0x7f42b80cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:29.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 shutdown_connections 2023-10-21T16:14:29.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.491+0000 7f42bc8d1640 1 -- 172.21.15.53:0/1606866152 wait complete. 2023-10-21T16:14:29.515 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-21T16:14:29.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:29.516 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-21T16:14:29.580 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:29.580 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:29.581 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:29.581 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:29.581 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:29.581 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:29.581 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:29.582 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:29.582 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:29.582 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:29.582 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:29.583 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_01_UTC', 'scheduled-2023-10-21-16_06_01_UTC', 'scheduled-2023-10-21-16_07_01_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_01_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:29.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-10-21T16:14:29.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.820+0000 7f0876c2e640 1 Processor -- start 2023-10-21T16:14:29.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.820+0000 7f0876c2e640 1 -- start start 2023-10-21T16:14:29.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015a820 0x7f08700aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08700ca680 0x7f08700b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f087015ac00 0x7f08700c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08700cab60 con 0x7f08700ca680 2023-10-21T16:14:29.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08700cb960 con 0x7f087015ac00 2023-10-21T16:14:29.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.822+0000 7f0876c2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08700cbdf0 con 0x7f087015a820 2023-10-21T16:14:29.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08700ca680 0x7f08700b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f08700ca680 0x7f08700b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47672/0 (socket says 172.21.15.53:47672) 2023-10-21T16:14:29.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 -- 172.21.15.53:0/922752235 learned_addr learned my addr 172.21.15.53:0/922752235 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:29.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f0875c2c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015a820 0x7f08700aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087642d640 1 --2- 172.21.15.53:0/922752235 >> v2:172.21.15.124:3300/0 conn(0x7f087015ac00 0x7f08700c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 -- 172.21.15.53:0/922752235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015a820 msgr2=0x7f08700aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:29.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 --2- 172.21.15.53:0/922752235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015a820 0x7f08700aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 -- 172.21.15.53:0/922752235 >> v2:172.21.15.124:3300/0 conn(0x7f087015ac00 msgr2=0x7f08700c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:29.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 --2- 172.21.15.53:0/922752235 >> v2:172.21.15.124:3300/0 conn(0x7f087015ac00 0x7f08700c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.823+0000 7f087542b640 1 -- 172.21.15.53:0/922752235 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700cb030 con 0x7f08700ca680 2023-10-21T16:14:29.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.825+0000 7f087542b640 1 --2- 172.21.15.53:0/922752235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08700ca680 0x7f08700b04b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08700ce340 tx=0x7f086002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a30653255efffb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.825+0000 7f0874c2a640 1 -- 172.21.15.53:0/922752235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600317d0 con 0x7f08700ca680 2023-10-21T16:14:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.825+0000 7f0874c2a640 1 -- 172.21.15.53:0/922752235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860031930 con 0x7f08700ca680 2023-10-21T16:14:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.825+0000 7f0874c2a640 1 -- 172.21.15.53:0/922752235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860031cc0 con 0x7f08700ca680 2023-10-21T16:14:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 -- 172.21.15.53:0/922752235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08700ca680 msgr2=0x7f08700b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 --2- 172.21.15.53:0/922752235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08700ca680 0x7f08700b04b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f08700ce340 tx=0x7f086002b870 comp rx=0 tx=0).stop 2023-10-21T16:14:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 -- 172.21.15.53:0/922752235 shutdown_connections 2023-10-21T16:14:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 --2- 172.21.15.53:0/922752235 >> v2:172.21.15.124:3300/0 conn(0x7f087015ac00 0x7f08700c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 --2- 172.21.15.53:0/922752235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08700ca680 0x7f08700b04b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 --2- 172.21.15.53:0/922752235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015a820 0x7f08700aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 -- 172.21.15.53:0/922752235 >> 172.21.15.53:0/922752235 conn(0x7f087006c050 msgr2=0x7f087006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 -- 172.21.15.53:0/922752235 shutdown_connections 2023-10-21T16:14:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.826+0000 7f0876c2e640 1 -- 172.21.15.53:0/922752235 wait complete. 2023-10-21T16:14:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.828+0000 7f0876c2e640 1 Processor -- start 2023-10-21T16:14:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.828+0000 7f0876c2e640 1 -- start start 2023-10-21T16:14:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08700ca680 0x7f08701f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 0x7f08700b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08701502e0 con 0x7f087015a820 2023-10-21T16:14:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08700ac730 con 0x7f08700ca680 2023-10-21T16:14:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f0876c2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08700af490 con 0x7f087015ac00 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f087542b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.829+0000 7f087542b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47676/0 (socket says 172.21.15.53:47676) 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 -- 172.21.15.53:0/2314464879 learned_addr learned my addr 172.21.15.53:0/2314464879 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f0875c2c640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08700ca680 0x7f08701f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 -- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 msgr2=0x7f08700b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087642d640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 0x7f08700b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 0x7f08700b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 -- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08700ca680 msgr2=0x7f08701f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08700ca680 0x7f08701f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 -- 172.21.15.53:0/2314464879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600094d0 con 0x7f087015a820 2023-10-21T16:14:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087642d640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 0x7f08700b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.830+0000 7f087542b640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f086002b8b0 tx=0x7f08600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.831+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860032870 con 0x7f087015a820 2023-10-21T16:14:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.831+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600329f0 con 0x7f087015a820 2023-10-21T16:14:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.831+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860032da0 con 0x7f087015a820 2023-10-21T16:14:29.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.831+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08700af6e0 con 0x7f087015a820 2023-10-21T16:14:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.831+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08700afca0 con 0x7f087015a820 2023-10-21T16:14:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.832+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08700b1320 con 0x7f087015a820 2023-10-21T16:14:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.833+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f0860030070 con 0x7f087015a820 2023-10-21T16:14:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.834+0000 7f08667fc640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 0x7f085406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.834+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f08600abac0 con 0x7f087015a820 2023-10-21T16:14:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.835+0000 7f0875c2c640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 0x7f085406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.836+0000 7f0875c2c640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 0x7f085406d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08700c5350 tx=0x7f086c008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:29.838+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08700b1320 con 0x7f087015a820 2023-10-21T16:14:30.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.056+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08700cb5f0 con 0x7f085406b070 2023-10-21T16:14:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.075+0000 7f08667fc640 1 -- 172.21.15.53:0/2314464879 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f086c009820 con 0x7f085406b070 2023-10-21T16:14:30.077 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:30.077 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:51", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.078+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 msgr2=0x7f085406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.078+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 0x7f085406d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08700c5350 tx=0x7f086c008040 comp rx=0 tx=0).stop 2023-10-21T16:14:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.078+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 msgr2=0x7f08700b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.078+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f086002b8b0 tx=0x7f08600046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 shutdown_connections 2023-10-21T16:14:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f085406b070 0x7f085406d530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f087015ac00 0x7f08700b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f087015a820 0x7f08700b35e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 --2- 172.21.15.53:0/2314464879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08700ca680 0x7f08701f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 >> 172.21.15.53:0/2314464879 conn(0x7f087006c050 msgr2=0x7f087015d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 shutdown_connections 2023-10-21T16:14:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.079+0000 7f0876c2e640 1 -- 172.21.15.53:0/2314464879 wait complete. 2023-10-21T16:14:30.099 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:51", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:30.100 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:51', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:30.100 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:30.101 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:30.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:30.101 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-21T16:14:30.157 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:30.158 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:30.158 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:30.158 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:30.160 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:30.160 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:30.161 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_01_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_00_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_01_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:30.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-10-21T16:14:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.389+0000 7f8f323b0640 1 Processor -- start 2023-10-21T16:14:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.390+0000 7f8f323b0640 1 -- start start 2023-10-21T16:14:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f2c15abe0 0x7f8f2c0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cde00 con 0x7f8f2c0cd430 2023-10-21T16:14:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2c0b69d0 con 0x7f8f2c15abe0 2023-10-21T16:14:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.391+0000 7f8f323b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0cf770 con 0x7f8f2c0b62a0 2023-10-21T16:14:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47680/0 (socket says 172.21.15.53:47680) 2023-10-21T16:14:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 -- 172.21.15.53:0/1197237215 learned_addr learned my addr 172.21.15.53:0/1197237215 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f2bfff640 1 --2- 172.21.15.53:0/1197237215 >> v2:172.21.15.124:3300/0 conn(0x7f8f2c15abe0 0x7f8f2c0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f23fff640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 -- 172.21.15.53:0/1197237215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 msgr2=0x7f8f2c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 -- 172.21.15.53:0/1197237215 >> v2:172.21.15.124:3300/0 conn(0x7f8f2c15abe0 msgr2=0x7f8f2c0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 --2- 172.21.15.53:0/1197237215 >> v2:172.21.15.124:3300/0 conn(0x7f8f2c15abe0 0x7f8f2c0d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.392+0000 7f8f3094d640 1 -- 172.21.15.53:0/1197237215 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c0d0140 con 0x7f8f2c0cd430 2023-10-21T16:14:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f3094d640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c070ff0 tx=0x7f8f2402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36540520b31427f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f2b7fe640 1 -- 172.21.15.53:0/1197237215 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2402ca70 con 0x7f8f2c0cd430 2023-10-21T16:14:30.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f2b7fe640 1 -- 172.21.15.53:0/1197237215 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2402cbd0 con 0x7f8f2c0cd430 2023-10-21T16:14:30.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f2b7fe640 1 -- 172.21.15.53:0/1197237215 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240343f0 con 0x7f8f2c0cd430 2023-10-21T16:14:30.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f323b0640 1 -- 172.21.15.53:0/1197237215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 msgr2=0x7f8f2c0cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.394+0000 7f8f323b0640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c070ff0 tx=0x7f8f2402bed0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 -- 172.21.15.53:0/1197237215 shutdown_connections 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 --2- 172.21.15.53:0/1197237215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c0cd8c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 --2- 172.21.15.53:0/1197237215 >> v2:172.21.15.124:3300/0 conn(0x7f8f2c15abe0 0x7f8f2c0d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 -- 172.21.15.53:0/1197237215 >> 172.21.15.53:0/1197237215 conn(0x7f8f2c06e650 msgr2=0x7f8f2c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:30.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 -- 172.21.15.53:0/1197237215 shutdown_connections 2023-10-21T16:14:30.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.395+0000 7f8f323b0640 1 -- 172.21.15.53:0/1197237215 wait complete. 2023-10-21T16:14:30.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.397+0000 7f8f323b0640 1 Processor -- start 2023-10-21T16:14:30.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.397+0000 7f8f323b0640 1 -- start start 2023-10-21T16:14:30.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f2c15abe0 0x7f8f2c0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1525c0 con 0x7f8f2c0cd430 2023-10-21T16:14:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c15d220 con 0x7f8f2c15abe0 2023-10-21T16:14:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f323b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c0b4750 con 0x7f8f2c0b62a0 2023-10-21T16:14:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47684/0 (socket says 172.21.15.53:47684) 2023-10-21T16:14:30.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 -- 172.21.15.53:0/2303710132 learned_addr learned my addr 172.21.15.53:0/2303710132 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:30.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 -- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 msgr2=0x7f8f2c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 -- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f2c15abe0 msgr2=0x7f8f2c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f2c15abe0 0x7f8f2c0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.398+0000 7f8f23fff640 1 -- 172.21.15.53:0/2303710132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24009b30 con 0x7f8f2c0cd430 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f23fff640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400d5f0 tx=0x7f8f1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140125f0 con 0x7f8f2c0cd430 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14012750 con 0x7f8f2c0cd430 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14012b00 con 0x7f8f2c0cd430 2023-10-21T16:14:30.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c0b4470 con 0x7f8f2c0cd430 2023-10-21T16:14:30.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.399+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c15d730 con 0x7f8f2c0cd430 2023-10-21T16:14:30.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.400+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c04e160 con 0x7f8f2c0cd430 2023-10-21T16:14:30.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.401+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f8f14012c60 con 0x7f8f2c0cd430 2023-10-21T16:14:30.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.402+0000 7f8f297fa640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 0x7f8f0c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.402+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8f1408a140 con 0x7f8f2c0cd430 2023-10-21T16:14:30.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.403+0000 7f8f2bfff640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 0x7f8f0c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.404+0000 7f8f2bfff640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 0x7f8f0c06d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006ea0 tx=0x7f8f1c008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.407+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f14054c70 con 0x7f8f2c0cd430 2023-10-21T16:14:30.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.621+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f2c0cee40 con 0x7f8f0c06b070 2023-10-21T16:14:30.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.640+0000 7f8f297fa640 1 -- 172.21.15.53:0/2303710132 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8f1c00be20 con 0x7f8f0c06b070 2023-10-21T16:14:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:30.642 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:51", "first": "2023-10-21T16:04:01", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.642+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 msgr2=0x7f8f0c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.642+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 0x7f8f0c06d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c006ea0 tx=0x7f8f1c008040 comp rx=0 tx=0).stop 2023-10-21T16:14:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.642+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 msgr2=0x7f8f2c15b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.642+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400d5f0 tx=0x7f8f1400dab0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 shutdown_connections 2023-10-21T16:14:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8f0c06b070 0x7f8f0c06d530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f2c15abe0 0x7f8f2c0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f2c0cd430 0x7f8f2c15b7d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 --2- 172.21.15.53:0/2303710132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f2c0b62a0 0x7f8f2c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 >> 172.21.15.53:0/2303710132 conn(0x7f8f2c06e650 msgr2=0x7f8f2c0c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 shutdown_connections 2023-10-21T16:14:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.643+0000 7f8f323b0640 1 -- 172.21.15.53:0/2303710132 wait complete. 2023-10-21T16:14:30.664 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:51", "first": "2023-10-21T16:04:01", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:30.665 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:51', 'first': '2023-10-21T16:04:01', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:30.665 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:30.665 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:30.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:30.666 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-21T16:14:30.727 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:30.727 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:30.728 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:30.728 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:30.728 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:30.728 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:30.728 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:30.729 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:30.729 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:30.729 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:30.729 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:30.730 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_01_UTC', 'scheduled-2023-10-21-16_11_00_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:30.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-10-21T16:14:30.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.959+0000 7f2eb759e640 1 Processor -- start 2023-10-21T16:14:30.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.959+0000 7f2eb759e640 1 -- start start 2023-10-21T16:14:30.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2eb80b0680 0x7f2eb815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80cab60 con 0x7f2eb815a820 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb80cb960 con 0x7f2eb80b0680 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80cbdf0 con 0x7f2eb80ca680 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2eb815a820 0x7f2eb80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53892/0 (socket says 172.21.15.53:53892) 2023-10-21T16:14:30.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.961+0000 7f2eb659c640 1 -- 172.21.15.53:0/2221495865 learned_addr learned my addr 172.21.15.53:0/2221495865 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:30.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb6d9d640 1 --2- 172.21.15.53:0/2221495865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eaffff640 1 --2- 172.21.15.53:0/2221495865 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80b0680 0x7f2eb815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb5d9b640 1 -- 172.21.15.53:0/2221495865 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498493176 0 0) 0x7f2eb80cab60 con 0x7f2eb815a820 2023-10-21T16:14:30.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb5d9b640 1 -- 172.21.15.53:0/2221495865 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea4003520 con 0x7f2eb815a820 2023-10-21T16:14:30.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb6d9d640 1 -- 172.21.15.53:0/2221495865 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80b0680 msgr2=0x7f2eb815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:30.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb6d9d640 1 --2- 172.21.15.53:0/2221495865 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80b0680 0x7f2eb815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb6d9d640 1 -- 172.21.15.53:0/2221495865 >> v1:172.21.15.53:6789/0 conn(0x7f2eb815a820 legacy=0x7f2eb80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.962+0000 7f2eb6d9d640 1 -- 172.21.15.53:0/2221495865 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80cb030 con 0x7f2eb80ca680 2023-10-21T16:14:30.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.964+0000 7f2eb6d9d640 1 --2- 172.21.15.53:0/2221495865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb815ea80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009d90 tx=0x7f2eb002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c7730bb38884e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.964+0000 7f2eb5d9b640 1 -- 172.21.15.53:0/2221495865 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb002cbc0 con 0x7f2eb80ca680 2023-10-21T16:14:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.964+0000 7f2eb5d9b640 1 -- 172.21.15.53:0/2221495865 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb002cd20 con 0x7f2eb80ca680 2023-10-21T16:14:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb5d9b640 1 -- 172.21.15.53:0/2221495865 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0034660 con 0x7f2eb80ca680 2023-10-21T16:14:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 -- 172.21.15.53:0/2221495865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 msgr2=0x7f2eb815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 --2- 172.21.15.53:0/2221495865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb815ea80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009d90 tx=0x7f2eb002be10 comp rx=0 tx=0).stop 2023-10-21T16:14:30.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 -- 172.21.15.53:0/2221495865 shutdown_connections 2023-10-21T16:14:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 --2- 172.21.15.53:0/2221495865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb815ea80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 --2- 172.21.15.53:0/2221495865 >> v2:172.21.15.124:3300/0 conn(0x7f2eb80b0680 0x7f2eb815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 -- 172.21.15.53:0/2221495865 >> 172.21.15.53:0/2221495865 conn(0x7f2eb806c050 msgr2=0x7f2eb806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 -- 172.21.15.53:0/2221495865 shutdown_connections 2023-10-21T16:14:30.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.965+0000 7f2eb759e640 1 -- 172.21.15.53:0/2221495865 wait complete. 2023-10-21T16:14:30.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.967+0000 7f2eb759e640 1 Processor -- start 2023-10-21T16:14:30.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.967+0000 7f2eb759e640 1 -- start start 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb80ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb815a820 0x7f2eb81f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81f6020 con 0x7f2eb80b0680 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb814a420 con 0x7f2eb815a820 2023-10-21T16:14:30.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.968+0000 7f2eb759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81f53a0 con 0x7f2eb80ca680 2023-10-21T16:14:30.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47692/0 (socket says 172.21.15.53:47692) 2023-10-21T16:14:30.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 -- 172.21.15.53:0/1269022934 learned_addr learned my addr 172.21.15.53:0/1269022934 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:30.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eaffff640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb80ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb6d9d640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb815a820 0x7f2eb81f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 -- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 msgr2=0x7f2eb80ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb80ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 -- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb815a820 msgr2=0x7f2eb81f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:30.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb815a820 0x7f2eb81f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:30.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 -- 172.21.15.53:0/1269022934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb0009a70 con 0x7f2eb80b0680 2023-10-21T16:14:30.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.969+0000 7f2eb659c640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c530 tx=0x7f2ea800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.970+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8005ce0 con 0x7f2eb80b0680 2023-10-21T16:14:30.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.970+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8005e40 con 0x7f2eb80b0680 2023-10-21T16:14:30.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.970+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80072b0 con 0x7f2eb80b0680 2023-10-21T16:14:30.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.970+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb81f5680 con 0x7f2eb80b0680 2023-10-21T16:14:30.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.970+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81f5c40 con 0x7f2eb80b0680 2023-10-21T16:14:30.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.971+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb8074150 con 0x7f2eb80b0680 2023-10-21T16:14:30.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.972+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f2ea8007410 con 0x7f2eb80b0680 2023-10-21T16:14:30.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.973+0000 7f2eaeffd640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 0x7f2e9806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:30.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.973+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2ea80891d0 con 0x7f2eb80b0680 2023-10-21T16:14:30.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.974+0000 7f2eaffff640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 0x7f2e9806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:30.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.975+0000 7f2eaffff640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 0x7f2e9806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8150de0 tx=0x7f2ea0008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:30.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:30.977+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ea8053e50 con 0x7f2eb80b0680 2023-10-21T16:14:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.198+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2eb80cb5f0 con 0x7f2e9806b070 2023-10-21T16:14:31.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.218+0000 7f2eaeffd640 1 -- 172.21.15.53:0/1269022934 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f2ea0009300 con 0x7f2e9806b070 2023-10-21T16:14:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:31.220 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:52", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:31.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 msgr2=0x7f2e9806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 0x7f2e9806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8150de0 tx=0x7f2ea0008040 comp rx=0 tx=0).stop 2023-10-21T16:14:31.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 msgr2=0x7f2eb80acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c530 tx=0x7f2ea800c9f0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 shutdown_connections 2023-10-21T16:14:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2e9806b070 0x7f2e9806d530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2eb815a820 0x7f2eb81f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2eb80ca680 0x7f2eb80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 --2- 172.21.15.53:0/1269022934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2eb80b0680 0x7f2eb80acb70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.221+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 >> 172.21.15.53:0/1269022934 conn(0x7f2eb806c050 msgr2=0x7f2eb80b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.222+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 shutdown_connections 2023-10-21T16:14:31.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.222+0000 7f2eb759e640 1 -- 172.21.15.53:0/1269022934 wait complete. 2023-10-21T16:14:31.244 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:52", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:31.244 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:52', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:31.244 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:31.245 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:31.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:31.245 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-21T16:14:31.308 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:31.308 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:31.308 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:31.308 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:31.309 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:31.309 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:31.309 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:31.309 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:31.309 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:31.310 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:31.311 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_01_UTC', 'scheduled-2023-10-21-16_07_01_UTC', 'scheduled-2023-10-21-16_08_01_UTC', 'scheduled-2023-10-21-16_09_01_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_00_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:31.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-10-21T16:14:31.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.539+0000 7f6d8f59e640 1 Processor -- start 2023-10-21T16:14:31.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.539+0000 7f6d8f59e640 1 -- start start 2023-10-21T16:14:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d8f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6d9015cf30 0x7f6d900d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d8f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900cd520 con 0x7f6d900cf690 2023-10-21T16:14:31.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d8f59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900d1910 con 0x7f6d900cf2b0 2023-10-21T16:14:31.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d8f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900ce080 con 0x7f6d9015cf30 2023-10-21T16:14:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d87fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6d900cf690 0x7f6d9015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53902/0 (socket says 172.21.15.53:53902) 2023-10-21T16:14:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d87fff640 1 -- 172.21.15.53:0/2803190029 learned_addr learned my addr 172.21.15.53:0/2803190029 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.541+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/2803190029 >> v2:172.21.15.124:3300/0 conn(0x7f6d9015cf30 0x7f6d900d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256957291 0 0) 0x7f6d900cd520 con 0x7f6d900cf690 2023-10-21T16:14:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7c003520 con 0x7f6d900cf690 2023-10-21T16:14:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465951389 0 0) 0x7f6d7c003520 con 0x7f6d900cf690 2023-10-21T16:14:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 >> v2:172.21.15.124:3300/0 conn(0x7f6d9015cf30 msgr2=0x7f6d900d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 --2- 172.21.15.53:0/2803190029 >> v2:172.21.15.124:3300/0 conn(0x7f6d9015cf30 0x7f6d900d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 >> v1:172.21.15.124:6789/0 conn(0x7f6d900cf2b0 legacy=0x7f6d900d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900cd750 con 0x7f6d900cf690 2023-10-21T16:14:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6d78001670 con 0x7f6d900cf690 2023-10-21T16:14:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 >> v1:172.21.15.53:6789/0 conn(0x7f6d900cf690 legacy=0x7f6d9015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d7c0039c0 0x7f6d7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d7c0047c0 0x7f6d7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ce080 con 0x7f6d7c0051d0 2023-10-21T16:14:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c00c120 con 0x7f6d7c0047c0 2023-10-21T16:14:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.542+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c00c2e0 con 0x7f6d7c0039c0 2023-10-21T16:14:31.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8e59c640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d7c0039c0 0x7f6d7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d87fff640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d7c0047c0 0x7f6d7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 -- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d7c0039c0 msgr2=0x7f6d7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d7c0039c0 0x7f6d7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 -- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d7c0047c0 msgr2=0x7f6d7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:31.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d7c0047c0 0x7f6d7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d8ed9d640 1 -- 172.21.15.53:0/2803190029 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c003520 con 0x7f6d7c0051d0 2023-10-21T16:14:31.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.543+0000 7f6d87fff640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d7c0047c0 0x7f6d7c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.544+0000 7f6d8ed9d640 1 -- 172.21.15.53:0/2803190029 reap_dead start 2023-10-21T16:14:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.544+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d7c00b980 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d9015d3a0 tx=0x7f6d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ceae20dcdc503b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8802cbd0 con 0x7f6d7c0051d0 2023-10-21T16:14:31.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800f040 con 0x7f6d7c0051d0 2023-10-21T16:14:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8dd9b640 1 -- 172.21.15.53:0/2803190029 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003990 con 0x7f6d7c0051d0 2023-10-21T16:14:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 -- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 msgr2=0x7f6d7c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d7c00b980 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d9015d3a0 tx=0x7f6d7c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 -- 172.21.15.53:0/2803190029 shutdown_connections 2023-10-21T16:14:31.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/2803190029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d7c00b980 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 -- 172.21.15.53:0/2803190029 >> 172.21.15.53:0/2803190029 conn(0x7f6d9009bc40 msgr2=0x7f6d900d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:31.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 -- 172.21.15.53:0/2803190029 shutdown_connections 2023-10-21T16:14:31.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.545+0000 7f6d8f59e640 1 -- 172.21.15.53:0/2803190029 wait complete. 2023-10-21T16:14:31.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.547+0000 7f6d8f59e640 1 Processor -- start 2023-10-21T16:14:31.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.547+0000 7f6d8f59e640 1 -- start start 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 0x7f6d9009ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d9009f180 0x7f6d901f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d901f2c90 con 0x7f6d7c0051d0 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9009f600 con 0x7f6d9009c5b0 2023-10-21T16:14:31.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.548+0000 7f6d8f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9009e4e0 con 0x7f6d9009f180 2023-10-21T16:14:31.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47704/0 (socket says 172.21.15.53:47704) 2023-10-21T16:14:31.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 -- 172.21.15.53:0/1508327453 learned_addr learned my addr 172.21.15.53:0/1508327453 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:31.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d87fff640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 0x7f6d9009ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8ed9d640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d9009f180 0x7f6d901f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 -- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d9009f180 msgr2=0x7f6d901f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d9009f180 0x7f6d901f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 -- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 msgr2=0x7f6d9009ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 0x7f6d9009ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 -- 172.21.15.53:0/1508327453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900cac60 con 0x7f6d7c0051d0 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d87fff640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 0x7f6d9009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.549+0000 7f6d8e59c640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e4d0 tx=0x7f6d8000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.550+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800052e0 con 0x7f6d7c0051d0 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.550+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d80005440 con 0x7f6d7c0051d0 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.550+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003810 con 0x7f6d7c0051d0 2023-10-21T16:14:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.550+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9015c150 con 0x7f6d7c0051d0 2023-10-21T16:14:31.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.550+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9009df20 con 0x7f6d7c0051d0 2023-10-21T16:14:31.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.551+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900caa30 con 0x7f6d7c0051d0 2023-10-21T16:14:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.552+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f6d80003970 con 0x7f6d7c0051d0 2023-10-21T16:14:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.553+0000 7f6d86ffd640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 0x7f6d7006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.553+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f6d80089780 con 0x7f6d7c0051d0 2023-10-21T16:14:31.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.554+0000 7f6d87fff640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 0x7f6d7006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:31.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.555+0000 7f6d87fff640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 0x7f6d7006d5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d900ce630 tx=0x7f6d78009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:31.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.557+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d900caa30 con 0x7f6d7c0051d0 2023-10-21T16:14:31.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.773+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d900cdd10 con 0x7f6d7006b0f0 2023-10-21T16:14:31.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.793+0000 7f6d86ffd640 1 -- 172.21.15.53:0/1508327453 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f6d7800d040 con 0x7f6d7006b0f0 2023-10-21T16:14:31.795 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:31.795 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:53", "first": "2023-10-21T16:05:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 10, "pruned_count": 0, "active": false}] 2023-10-21T16:14:31.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 msgr2=0x7f6d7006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 0x7f6d7006d5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d900ce630 tx=0x7f6d78009040 comp rx=0 tx=0).stop 2023-10-21T16:14:31.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 msgr2=0x7f6d90150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:31.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e4d0 tx=0x7f6d8000e990 comp rx=0 tx=0).stop 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 shutdown_connections 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f6d7006b0f0 0x7f6d7006d5b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6d9009f180 0x7f6d901f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6d9009c5b0 0x7f6d9009ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 --2- 172.21.15.53:0/1508327453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6d7c0051d0 0x7f6d90150e00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:31.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 >> 172.21.15.53:0/1508327453 conn(0x7f6d9009bc40 msgr2=0x7f6d900d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:31.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 shutdown_connections 2023-10-21T16:14:31.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:31.796+0000 7f6d8f59e640 1 -- 172.21.15.53:0/1508327453 wait complete. 2023-10-21T16:14:31.815 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:53", "first": "2023-10-21T16:05:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 10, "pruned_count": 0, "active": false}] 2023-10-21T16:14:31.816 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:53', 'first': '2023-10-21T16:05:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 10, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:31.816 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 10 2023-10-21T16:14:31.816 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 10 2023-10-21T16:14:31.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:31.817 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-21T16:14:31.871 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:31.871 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:31.872 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:31.873 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:31.873 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:31.873 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:31.874 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_01_UTC', 'scheduled-2023-10-21-16_08_01_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_00_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_01_UTC'] 2023-10-21T16:14:31.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-10-21T16:14:32.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.108+0000 7fdc246a2640 1 Processor -- start 2023-10-21T16:14:32.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.109+0000 7fdc246a2640 1 -- start start 2023-10-21T16:14:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc246a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc246a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc1c072090 0x7fdc1c06a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc246a2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0ce210 con 0x7fdc1c1601b0 2023-10-21T16:14:32.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc246a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cdb40 con 0x7fdc1c072090 2023-10-21T16:14:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc246a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cdee0 con 0x7fdc1c071c30 2023-10-21T16:14:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc22c3f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc1c072090 0x7fdc1c06a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc22c3f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc1c072090 0x7fdc1c06a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47710/0 (socket says 172.21.15.53:47710) 2023-10-21T16:14:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc22c3f640 1 -- 172.21.15.53:0/3724833810 learned_addr learned my addr 172.21.15.53:0/3724833810 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:32.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.110+0000 7fdc21c3d640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.111+0000 7fdc22c3f640 1 -- 172.21.15.53:0/3724833810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 msgr2=0x7fdc1c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.111+0000 7fdc22c3f640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.111+0000 7fdc22c3f640 1 -- 172.21.15.53:0/3724833810 >> v1:172.21.15.124:6789/0 conn(0x7fdc1c1601b0 legacy=0x7fdc1c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.111+0000 7fdc22c3f640 1 -- 172.21.15.53:0/3724833810 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c161530 con 0x7fdc1c072090 2023-10-21T16:14:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.112+0000 7fdc22c3f640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c072090 0x7fdc1c06a290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e50 tx=0x7fdc1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208ab3b0620ba58d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.112+0000 7fdc2143c640 1 -- 172.21.15.53:0/3724833810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1402ca70 con 0x7fdc1c072090 2023-10-21T16:14:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.112+0000 7fdc2143c640 1 -- 172.21.15.53:0/3724833810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1402cbd0 con 0x7fdc1c072090 2023-10-21T16:14:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.112+0000 7fdc2143c640 1 -- 172.21.15.53:0/3724833810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140343f0 con 0x7fdc1c072090 2023-10-21T16:14:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 -- 172.21.15.53:0/3724833810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c072090 msgr2=0x7fdc1c06a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c072090 0x7fdc1c06a290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e50 tx=0x7fdc1402bed0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 -- 172.21.15.53:0/3724833810 shutdown_connections 2023-10-21T16:14:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 --2- 172.21.15.53:0/3724833810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c072090 0x7fdc1c06a290 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 -- 172.21.15.53:0/3724833810 >> 172.21.15.53:0/3724833810 conn(0x7fdc1c06d430 msgr2=0x7fdc1c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 -- 172.21.15.53:0/3724833810 shutdown_connections 2023-10-21T16:14:32.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.113+0000 7fdc246a2640 1 -- 172.21.15.53:0/3724833810 wait complete. 2023-10-21T16:14:32.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.115+0000 7fdc246a2640 1 Processor -- start 2023-10-21T16:14:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.115+0000 7fdc246a2640 1 -- start start 2023-10-21T16:14:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc1c072090 0x7fdc1c1f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b5430 con 0x7fdc1c1601b0 2023-10-21T16:14:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b2300 con 0x7fdc1c072090 2023-10-21T16:14:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc246a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c151150 con 0x7fdc1c071c30 2023-10-21T16:14:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc22c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc22c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47726/0 (socket says 172.21.15.53:47726) 2023-10-21T16:14:32.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.116+0000 7fdc22c3f640 1 -- 172.21.15.53:0/226735388 learned_addr learned my addr 172.21.15.53:0/226735388 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:32.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc21c3d640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc1c072090 0x7fdc1c1f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc2243e640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 -- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 msgr2=0x7fdc1c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c0b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 -- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc1c072090 msgr2=0x7fdc1c1f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc1c072090 0x7fdc1c1f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 -- 172.21.15.53:0/226735388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009b30 con 0x7fdc1c1601b0 2023-10-21T16:14:32.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc22c3f640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdc14036040 tx=0x7fdc14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14009510 con 0x7fdc1c1601b0 2023-10-21T16:14:32.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.117+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1403e450 con 0x7fdc1c1601b0 2023-10-21T16:14:32.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.118+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1403e7e0 con 0x7fdc1c1601b0 2023-10-21T16:14:32.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.118+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0c3530 con 0x7fdc1c1601b0 2023-10-21T16:14:32.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.118+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0b2550 con 0x7fdc1c1601b0 2023-10-21T16:14:32.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.119+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c0c39f0 con 0x7fdc1c1601b0 2023-10-21T16:14:32.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.120+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fdc1403e940 con 0x7fdc1c1601b0 2023-10-21T16:14:32.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.121+0000 7fdc12ffd640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 0x7fdc0006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.121+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fdc140aca50 con 0x7fdc1c1601b0 2023-10-21T16:14:32.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.121+0000 7fdc2243e640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 0x7fdc0006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.122+0000 7fdc2243e640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 0x7fdc0006d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800a750 tx=0x7fdc18008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.125+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc14077580 con 0x7fdc1c1601b0 2023-10-21T16:14:32.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.339+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc1c161810 con 0x7fdc0006b070 2023-10-21T16:14:32.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.358+0000 7fdc12ffd640 1 -- 172.21.15.53:0/226735388 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fdc1800a380 con 0x7fdc0006b070 2023-10-21T16:14:32.360 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:32.360 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:54", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:32.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 msgr2=0x7fdc0006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 0x7fdc0006d530 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800a750 tx=0x7fdc18008040 comp rx=0 tx=0).stop 2023-10-21T16:14:32.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 msgr2=0x7fdc1c0ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdc14036040 tx=0x7fdc14004a40 comp rx=0 tx=0).stop 2023-10-21T16:14:32.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 shutdown_connections 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdc0006b070 0x7fdc0006d530 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c1601b0 0x7fdc1c0ca6a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc1c072090 0x7fdc1c1f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 --2- 172.21.15.53:0/226735388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc1c071c30 0x7fdc1c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 >> 172.21.15.53:0/226735388 conn(0x7fdc1c06d430 msgr2=0x7fdc1c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:32.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 shutdown_connections 2023-10-21T16:14:32.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.361+0000 7fdc246a2640 1 -- 172.21.15.53:0/226735388 wait complete. 2023-10-21T16:14:32.382 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:54", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:32.382 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:54', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:32.382 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:32.383 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:32.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:32.383 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-21T16:14:32.439 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:32.440 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:32.440 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:32.440 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:32.440 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:32.440 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:32.441 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:32.441 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:32.441 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:32.441 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:32.441 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:32.442 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_01_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_01_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_01_UTC', 'scheduled-2023-10-21-16_10_01_UTC', 'scheduled-2023-10-21-16_11_01_UTC', 'scheduled-2023-10-21-16_12_01_UTC', 'scheduled-2023-10-21-16_13_01_UTC', 'scheduled-2023-10-21-16_14_01_UTC'] 2023-10-21T16:14:32.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-10-21T16:14:32.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.671+0000 7fc0e359e640 1 Processor -- start 2023-10-21T16:14:32.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.672+0000 7fc0e359e640 1 -- start start 2023-10-21T16:14:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.673+0000 7fc0e359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0e415a820 0x7fc0e40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.673+0000 7fc0e359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 0x7fc0e40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.673+0000 7fc0e359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e40cab60 con 0x7fc0e415ac00 2023-10-21T16:14:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.673+0000 7fc0e359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e40cb960 con 0x7fc0e415a820 2023-10-21T16:14:32.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.673+0000 7fc0e359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cbdf0 con 0x7fc0e40ca680 2023-10-21T16:14:32.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0e2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc0e415ac00 0x7fc0e40c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53914/0 (socket says 172.21.15.53:53914) 2023-10-21T16:14:32.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0e2d9d640 1 -- 172.21.15.53:0/2358177560 learned_addr learned my addr 172.21.15.53:0/2358177560 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:32.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0e259c640 1 --2- 172.21.15.53:0/2358177560 >> v2:172.21.15.124:3300/0 conn(0x7fc0e415a820 0x7fc0e40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0dbfff640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 0x7fc0e40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190795855 0 0) 0x7fc0e40cab60 con 0x7fc0e415ac00 2023-10-21T16:14:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.674+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d0003520 con 0x7fc0e415ac00 2023-10-21T16:14:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4143937131 0 0) 0x7fc0d0003520 con 0x7fc0e415ac00 2023-10-21T16:14:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 msgr2=0x7fc0e40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 0x7fc0e40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 >> v2:172.21.15.124:3300/0 conn(0x7fc0e415a820 msgr2=0x7fc0e40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 --2- 172.21.15.53:0/2358177560 >> v2:172.21.15.124:3300/0 conn(0x7fc0e415a820 0x7fc0e40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40cb030 con 0x7fc0e415ac00 2023-10-21T16:14:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc0dc0019f0 con 0x7fc0e415ac00 2023-10-21T16:14:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 >> v1:172.21.15.53:6789/0 conn(0x7fc0e415ac00 legacy=0x7fc0e40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d0003970 0x7fc0d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 0x7fc0d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0004d00 0x7fc0d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cb960 con 0x7fc0d0003970 2023-10-21T16:14:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cbdf0 con 0x7fc0d0004d00 2023-10-21T16:14:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d000b980 con 0x7fc0e40ca680 2023-10-21T16:14:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.675+0000 7fc0e259c640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d0003970 0x7fc0d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e2d9d640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0004d00 0x7fc0d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e259c640 1 -- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 msgr2=0x7fc0d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e259c640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40ca680 0x7fc0d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e259c640 1 -- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0004d00 msgr2=0x7fc0d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e259c640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0004d00 0x7fc0d0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.676+0000 7fc0e259c640 1 -- 172.21.15.53:0/2358177560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0003520 con 0x7fc0d0003970 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.677+0000 7fc0e259c640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d0003970 0x7fc0d0003d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009ac0 tx=0x7fc0d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ff7253e95a7d12 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.677+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d402c710 con 0x7fc0d0003970 2023-10-21T16:14:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.677+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d402c870 con 0x7fc0d0003970 2023-10-21T16:14:32.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.677+0000 7fc0e1d9b640 1 -- 172.21.15.53:0/2358177560 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d402cc00 con 0x7fc0d0003970 2023-10-21T16:14:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 -- 172.21.15.53:0/2358177560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d0003970 msgr2=0x7fc0d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 --2- 172.21.15.53:0/2358177560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d0003970 0x7fc0d0003d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009ac0 tx=0x7fc0d00057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e2d9d640 1 -- 172.21.15.53:0/2358177560 reap_dead start 2023-10-21T16:14:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 -- 172.21.15.53:0/2358177560 shutdown_connections 2023-10-21T16:14:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 -- 172.21.15.53:0/2358177560 >> 172.21.15.53:0/2358177560 conn(0x7fc0e406c050 msgr2=0x7fc0e406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 -- 172.21.15.53:0/2358177560 shutdown_connections 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.678+0000 7fc0e359e640 1 -- 172.21.15.53:0/2358177560 wait complete. 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.680+0000 7fc0e359e640 1 Processor -- start 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.680+0000 7fc0e359e640 1 -- start start 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 0x7fc0e40afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40c82c0 0x7fc0e41f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e414a280 con 0x7fc0e40c7e40 2023-10-21T16:14:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40af570 con 0x7fc0d0003970 2023-10-21T16:14:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e415d890 con 0x7fc0e40c82c0 2023-10-21T16:14:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47732/0 (socket says 172.21.15.53:47732) 2023-10-21T16:14:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0dbfff640 1 -- 172.21.15.53:0/3613822997 learned_addr learned my addr 172.21.15.53:0/3613822997 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.681+0000 7fc0e259c640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 0x7fc0e40afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 -- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40c82c0 msgr2=0x7fc0e41f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0e2d9d640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40c82c0 0x7fc0e41f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40c82c0 0x7fc0e41f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 -- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 msgr2=0x7fc0e40afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 0x7fc0e40afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 -- 172.21.15.53:0/3613822997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4153490 con 0x7fc0e40c7e40 2023-10-21T16:14:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0e259c640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 0x7fc0e40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0dbfff640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e7c0 tx=0x7fc0cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00eed0 con 0x7fc0e40c7e40 2023-10-21T16:14:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00c940 con 0x7fc0e40c7e40 2023-10-21T16:14:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00ccd0 con 0x7fc0e40c7e40 2023-10-21T16:14:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e4153770 con 0x7fc0e40c7e40 2023-10-21T16:14:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.682+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e40b0290 con 0x7fc0e40c7e40 2023-10-21T16:14:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.684+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e4153960 con 0x7fc0e40c7e40 2023-10-21T16:14:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.685+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc0cc013470 con 0x7fc0e40c7e40 2023-10-21T16:14:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.686+0000 7fc0daffd640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 0x7fc0c406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.686+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fc0cc089d20 con 0x7fc0e40c7e40 2023-10-21T16:14:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.686+0000 7fc0e259c640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 0x7fc0c406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.688+0000 7fc0e259c640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 0x7fc0c406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4036000 tx=0x7fc0d4030000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.690+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0cc0548e0 con 0x7fc0e40c7e40 2023-10-21T16:14:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.908+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0e40af3f0 con 0x7fc0c406b070 2023-10-21T16:14:32.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.928+0000 7fc0daffd640 1 -- 172.21.15.53:0/3613822997 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fc0d402cae0 con 0x7fc0c406b070 2023-10-21T16:14:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:32.930 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:54", "first": "2023-10-21T16:04:01", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:32.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 msgr2=0x7fc0c406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 0x7fc0c406d530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4036000 tx=0x7fc0d4030000 comp rx=0 tx=0).stop 2023-10-21T16:14:32.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 msgr2=0x7fc0e41f33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:32.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00e7c0 tx=0x7fc0cc00ec80 comp rx=0 tx=0).stop 2023-10-21T16:14:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 shutdown_connections 2023-10-21T16:14:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fc0c406b070 0x7fc0c406d530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0e40c82c0 0x7fc0e41f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e40c7e40 0x7fc0e41f33c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 --2- 172.21.15.53:0/3613822997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d0003970 0x7fc0e40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 >> 172.21.15.53:0/3613822997 conn(0x7fc0e406c050 msgr2=0x7fc0e40ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 shutdown_connections 2023-10-21T16:14:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:32.931+0000 7fc0e359e640 1 -- 172.21.15.53:0/3613822997 wait complete. 2023-10-21T16:14:32.953 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:54", "first": "2023-10-21T16:04:01", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:32.954 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:54', 'first': '2023-10-21T16:04:01', 'last': '2023-10-21T16:14:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:32.954 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:32.954 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:32.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:32.955 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-21T16:14:33.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:33.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:33.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:33.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:33.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:33.012 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:33.012 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:33.012 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:33.012 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:33.012 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:33.013 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:33.014 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_01_UTC', 'scheduled-2023-10-21-16_06_01_UTC', 'scheduled-2023-10-21-16_07_01_UTC', 'scheduled-2023-10-21-16_08_01_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_01_UTC', 'scheduled-2023-10-21-16_12_01_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:33.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-10-21T16:14:33.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.241+0000 7f3dae587640 1 Processor -- start 2023-10-21T16:14:33.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.241+0000 7f3dae587640 1 -- start start 2023-10-21T16:14:33.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dae587640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3da80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dae587640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3da80ce510 0x7f3da815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dae587640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80cd520 con 0x7f3da80cf2b0 2023-10-21T16:14:33.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dae587640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da80d1910 con 0x7f3da80ce510 2023-10-21T16:14:33.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dae587640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80ce080 con 0x7f3da80cf690 2023-10-21T16:14:33.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dad585640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3da80cf2b0 0x7f3da80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53922/0 (socket says 172.21.15.53:53922) 2023-10-21T16:14:33.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.242+0000 7f3dad585640 1 -- 172.21.15.53:0/191380118 learned_addr learned my addr 172.21.15.53:0/191380118 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:33.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3dadd86640 1 --2- 172.21.15.53:0/191380118 >> v2:172.21.15.124:3300/0 conn(0x7f3da80ce510 0x7f3da815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667200048 0 0) 0x7f3da80cd520 con 0x7f3da80cf2b0 2023-10-21T16:14:33.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d94003520 con 0x7f3da80cf2b0 2023-10-21T16:14:33.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3dacd84640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3da80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 307407492 0 0) 0x7f3d94003520 con 0x7f3da80cf2b0 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 msgr2=0x7f3da80d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3da80d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 >> v2:172.21.15.124:3300/0 conn(0x7f3da80ce510 msgr2=0x7f3da815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 --2- 172.21.15.53:0/191380118 >> v2:172.21.15.124:3300/0 conn(0x7f3da80ce510 0x7f3da815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80cd750 con 0x7f3da80cf2b0 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3da40019f0 con 0x7f3da80cf2b0 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 >> v1:172.21.15.53:6789/0 conn(0x7f3da80cf2b0 legacy=0x7f3da80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3d94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d940047c0 0x7f3d94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3d94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80d1910 con 0x7f3d94003970 2023-10-21T16:14:33.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80ce080 con 0x7f3d940047c0 2023-10-21T16:14:33.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.243+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9400bae0 con 0x7f3da80cf690 2023-10-21T16:14:33.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3d94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dacd84640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d940047c0 0x7f3d94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dadd86640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3d94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 -- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 msgr2=0x7f3d94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cf690 0x7f3d94005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 -- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d940047c0 msgr2=0x7f3d94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d940047c0 0x7f3d94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dad585640 1 -- 172.21.15.53:0/191380118 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d94003520 con 0x7f3d94003970 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.244+0000 7f3dacd84640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d940047c0 0x7f3d94004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.245+0000 7f3dad585640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3d94003d90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3da400a830 tx=0x7f3d940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983cba711501191 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.245+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4034630 con 0x7f3d94003970 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.245+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da402f070 con 0x7f3d94003970 2023-10-21T16:14:33.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.245+0000 7f3d9ffff640 1 -- 172.21.15.53:0/191380118 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40349f0 con 0x7f3d94003970 2023-10-21T16:14:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 -- 172.21.15.53:0/191380118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 msgr2=0x7f3d94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 --2- 172.21.15.53:0/191380118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3d94003d90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3da400a830 tx=0x7f3d940057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dadd86640 1 -- 172.21.15.53:0/191380118 reap_dead start 2023-10-21T16:14:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 -- 172.21.15.53:0/191380118 shutdown_connections 2023-10-21T16:14:33.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 -- 172.21.15.53:0/191380118 >> 172.21.15.53:0/191380118 conn(0x7f3da809bc40 msgr2=0x7f3da80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:33.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 -- 172.21.15.53:0/191380118 shutdown_connections 2023-10-21T16:14:33.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.246+0000 7f3dae587640 1 -- 172.21.15.53:0/191380118 wait complete. 2023-10-21T16:14:33.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.248+0000 7f3dae587640 1 Processor -- start 2023-10-21T16:14:33.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.248+0000 7f3dae587640 1 -- start start 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3da815a0b0 0x7f3da80cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 0x7f3da81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da815ce70 con 0x7f3d94003970 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80cbc20 con 0x7f3da815a0b0 2023-10-21T16:14:33.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.249+0000 7f3dae587640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80b8330 con 0x7f3da80cb7a0 2023-10-21T16:14:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dadd86640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 0x7f3da81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dacd84640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3da815a0b0 0x7f3da80cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47752/0 (socket says 172.21.15.53:47752) 2023-10-21T16:14:33.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 -- 172.21.15.53:0/4225324024 learned_addr learned my addr 172.21.15.53:0/4225324024 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dadd86640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 0x7f3da81f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57106/0 (socket says 172.21.15.53:57106) 2023-10-21T16:14:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 -- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 msgr2=0x7f3da81f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 0x7f3da81f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 -- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3da815a0b0 msgr2=0x7f3da80cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3da815a0b0 0x7f3da80cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.250+0000 7f3dad585640 1 -- 172.21.15.53:0/4225324024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80ca7b0 con 0x7f3d94003970 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3dad585640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3da400a950 tx=0x7f3da4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4005030 con 0x7f3d94003970 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da402ce90 con 0x7f3d94003970 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4005390 con 0x7f3d94003970 2023-10-21T16:14:33.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da80caa90 con 0x7f3d94003970 2023-10-21T16:14:33.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.251+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da80b59b0 con 0x7f3d94003970 2023-10-21T16:14:33.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.252+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da80b5530 con 0x7f3d94003970 2023-10-21T16:14:33.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.253+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3da4031030 con 0x7f3d94003970 2023-10-21T16:14:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.254+0000 7f3d9dffb640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 0x7f3d8c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.255+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3da40abfd0 con 0x7f3d94003970 2023-10-21T16:14:33.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.255+0000 7f3dacd84640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 0x7f3d8c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.257+0000 7f3dacd84640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 0x7f3d8c06d530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3da806c940 tx=0x7f3d98008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.259+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3da80b5530 con 0x7f3d94003970 2023-10-21T16:14:33.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.472+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3da80cdd10 con 0x7f3d8c06b070 2023-10-21T16:14:33.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.494+0000 7f3d9dffb640 1 -- 172.21.15.53:0/4225324024 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3d98009e50 con 0x7f3d8c06b070 2023-10-21T16:14:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:33.496 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:55", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 msgr2=0x7f3d8c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 0x7f3d8c06d530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3da806c940 tx=0x7f3d98008040 comp rx=0 tx=0).stop 2023-10-21T16:14:33.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 msgr2=0x7f3da80b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3da400a950 tx=0x7f3da4001d80 comp rx=0 tx=0).stop 2023-10-21T16:14:33.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 shutdown_connections 2023-10-21T16:14:33.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3d8c06b070 0x7f3d8c06d530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3da80cb7a0 0x7f3da81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3da815a0b0 0x7f3da80cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 --2- 172.21.15.53:0/4225324024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d94003970 0x7f3da80b7df0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.497+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 >> 172.21.15.53:0/4225324024 conn(0x7f3da809bc40 msgr2=0x7f3da80b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:33.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.498+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 shutdown_connections 2023-10-21T16:14:33.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.498+0000 7f3dae587640 1 -- 172.21.15.53:0/4225324024 wait complete. 2023-10-21T16:14:33.519 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:55", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:33.519 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:55', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:33.520 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:33.520 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:33.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:33.521 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-21T16:14:33.577 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:33.577 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:33.577 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:33.578 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:33.579 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:33.579 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:33.580 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_01_UTC', 'scheduled-2023-10-21-16_11_01_UTC', 'scheduled-2023-10-21-16_12_01_UTC', 'scheduled-2023-10-21-16_13_01_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:33.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-10-21T16:14:33.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.807+0000 7fb8650a2640 1 Processor -- start 2023-10-21T16:14:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.807+0000 7fb8650a2640 1 -- start start 2023-10-21T16:14:33.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb8650a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 0x7fb86015e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb8650a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86015aef0 con 0x7fb860070b20 2023-10-21T16:14:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb8650a2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86006f4a0 con 0x7fb8600cb390 2023-10-21T16:14:33.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb8650a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015ee00 con 0x7fb86015ab10 2023-10-21T16:14:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb85f59e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb860070b20 0x7fb860069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53926/0 (socket says 172.21.15.53:53926) 2023-10-21T16:14:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.808+0000 7fb85f59e640 1 -- 172.21.15.53:0/1369665219 learned_addr learned my addr 172.21.15.53:0/1369665219 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.809+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 0x7fb86015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.809+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619824709 0 0) 0x7fb858001110 con 0x7fb860070b20 2023-10-21T16:14:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.809+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb84c003520 con 0x7fb860070b20 2023-10-21T16:14:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887560483 0 0) 0x7fb84c003520 con 0x7fb860070b20 2023-10-21T16:14:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 msgr2=0x7fb86015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 0x7fb86015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 >> v1:172.21.15.124:6789/0 conn(0x7fb8600cb390 legacy=0x7fb86006ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86015f0e0 con 0x7fb860070b20 2023-10-21T16:14:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb858001800 con 0x7fb860070b20 2023-10-21T16:14:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 >> v1:172.21.15.53:6789/0 conn(0x7fb860070b20 legacy=0x7fb860069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb84c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb84c0047c0 0x7fb84c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 0x7fb84c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015ee00 con 0x7fb84c0039a0 2023-10-21T16:14:33.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c00ba30 con 0x7fb84c0047c0 2023-10-21T16:14:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c00bbf0 con 0x7fb86015ab10 2023-10-21T16:14:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb84c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 -- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 msgr2=0x7fb84c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86015ab10 0x7fb84c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 -- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb84c0047c0 msgr2=0x7fb84c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb84c0047c0 0x7fb84c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.810+0000 7fb85ed9d640 1 -- 172.21.15.53:0/1369665219 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c003520 con 0x7fb84c0039a0 2023-10-21T16:14:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb84c003dc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb850009a90 tx=0x7fb84c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f7dd5cfded4969 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85002c6b0 con 0x7fb84c0039a0 2023-10-21T16:14:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85002c810 con 0x7fb84c0039a0 2023-10-21T16:14:33.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb85e59c640 1 -- 172.21.15.53:0/1369665219 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85002cba0 con 0x7fb84c0039a0 2023-10-21T16:14:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb8650a2640 1 -- 172.21.15.53:0/1369665219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 msgr2=0x7fb84c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.812+0000 7fb8650a2640 1 --2- 172.21.15.53:0/1369665219 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb84c003dc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb850009a90 tx=0x7fb84c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.813+0000 7fb85f59e640 1 -- 172.21.15.53:0/1369665219 reap_dead start 2023-10-21T16:14:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.813+0000 7fb8650a2640 1 -- 172.21.15.53:0/1369665219 shutdown_connections 2023-10-21T16:14:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.813+0000 7fb8650a2640 1 -- 172.21.15.53:0/1369665219 >> 172.21.15.53:0/1369665219 conn(0x7fb8600ceef0 msgr2=0x7fb8600cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.813+0000 7fb8650a2640 1 -- 172.21.15.53:0/1369665219 shutdown_connections 2023-10-21T16:14:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.813+0000 7fb8650a2640 1 -- 172.21.15.53:0/1369665219 wait complete. 2023-10-21T16:14:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.815+0000 7fb8650a2640 1 Processor -- start 2023-10-21T16:14:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.815+0000 7fb8650a2640 1 -- start start 2023-10-21T16:14:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8600c4420 0x7fb8600b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86009f330 0x7fb86009f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86009fcf0 con 0x7fb84c0039a0 2023-10-21T16:14:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600ccb60 con 0x7fb8600c4420 2023-10-21T16:14:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb8650a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8600c4c60 con 0x7fb86009f330 2023-10-21T16:14:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb85ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb85ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47772/0 (socket says 172.21.15.53:47772) 2023-10-21T16:14:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.816+0000 7fb85ed9d640 1 -- 172.21.15.53:0/3319521227 learned_addr learned my addr 172.21.15.53:0/3319521227 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85659c640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8600c4420 0x7fb8600b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 -- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86009f330 msgr2=0x7fb86009f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86009f330 0x7fb86009f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 -- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8600c4420 msgr2=0x7fb8600b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8600c4420 0x7fb8600b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 -- 172.21.15.53:0/3319521227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8600cce00 con 0x7fb84c0039a0 2023-10-21T16:14:33.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb85ed9d640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb85002c250 tx=0x7fb850004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850003810 con 0x7fb84c0039a0 2023-10-21T16:14:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850003970 con 0x7fb84c0039a0 2023-10-21T16:14:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850003ce0 con 0x7fb84c0039a0 2023-10-21T16:14:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8600cd0e0 con 0x7fb84c0039a0 2023-10-21T16:14:33.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.817+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8601509b0 con 0x7fb84c0039a0 2023-10-21T16:14:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.819+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004e160 con 0x7fb84c0039a0 2023-10-21T16:14:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.820+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb850040450 con 0x7fb84c0039a0 2023-10-21T16:14:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.820+0000 7fb857fff640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 0x7fb84006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.821+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fb8500ab9e0 con 0x7fb84c0039a0 2023-10-21T16:14:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.821+0000 7fb85659c640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 0x7fb84006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:33.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.822+0000 7fb85659c640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 0x7fb84006d560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8601f6100 tx=0x7fb848009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:33.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:33.825+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb850076510 con 0x7fb84c0039a0 2023-10-21T16:14:34.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.039+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb86006e870 con 0x7fb84006b0a0 2023-10-21T16:14:34.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.058+0000 7fb857fff640 1 -- 172.21.15.53:0/3319521227 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fb8480019c0 con 0x7fb84006b0a0 2023-10-21T16:14:34.060 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:34.060 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:56", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 msgr2=0x7fb84006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 0x7fb84006d560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8601f6100 tx=0x7fb848009040 comp rx=0 tx=0).stop 2023-10-21T16:14:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 msgr2=0x7fb8600cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb85002c250 tx=0x7fb850004730 comp rx=0 tx=0).stop 2023-10-21T16:14:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 shutdown_connections 2023-10-21T16:14:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb84006b0a0 0x7fb84006d560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86009f330 0x7fb86009f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8600c4420 0x7fb8600b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 --2- 172.21.15.53:0/3319521227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0039a0 0x7fb8600cadd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 >> 172.21.15.53:0/3319521227 conn(0x7fb8600ceef0 msgr2=0x7fb86015b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.061+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 shutdown_connections 2023-10-21T16:14:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.062+0000 7fb8650a2640 1 -- 172.21.15.53:0/3319521227 wait complete. 2023-10-21T16:14:34.083 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:56", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:34.083 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:56', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:34.083 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:34.084 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:34.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:34.084 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-21T16:14:34.137 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:34.137 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:34.137 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:34.137 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:34.138 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:34.138 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:34.138 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:34.138 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:34.139 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:34.139 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:34.139 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:34.140 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_00_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_00_UTC', 'scheduled-2023-10-21-16_09_00_UTC', 'scheduled-2023-10-21-16_10_00_UTC', 'scheduled-2023-10-21-16_11_00_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_00_UTC', 'scheduled-2023-10-21-16_14_00_UTC'] 2023-10-21T16:14:34.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-10-21T16:14:34.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.367+0000 7fceeeab4640 1 Processor -- start 2023-10-21T16:14:34.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.367+0000 7fceeeab4640 1 -- start start 2023-10-21T16:14:34.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceeeab4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcee806dfc0 0x7fcee8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceeeab4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcee8071070 0x7fcee80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceeeab4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee80b1cc0 con 0x7fcee80afbe0 2023-10-21T16:14:34.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceeeab4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80b2620 con 0x7fcee806dfc0 2023-10-21T16:14:34.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceeeab4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80b2060 con 0x7fcee8071070 2023-10-21T16:14:34.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceec850640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcee80afbe0 0x7fcee80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48972/0 (socket says 172.21.15.53:48972) 2023-10-21T16:14:34.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.369+0000 7fceec850640 1 -- 172.21.15.53:0/1041275221 learned_addr learned my addr 172.21.15.53:0/1041275221 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:34.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee7fff640 1 --2- 172.21.15.53:0/1041275221 >> v2:172.21.15.53:3300/0 conn(0x7fcee806dfc0 0x7fcee8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fceed051640 1 --2- 172.21.15.53:0/1041275221 >> v2:172.21.15.124:3300/0 conn(0x7fcee8071070 0x7fcee80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee77fe640 1 -- 172.21.15.53:0/1041275221 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951219046 0 0) 0x7fcee80b1cc0 con 0x7fcee80afbe0 2023-10-21T16:14:34.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee77fe640 1 -- 172.21.15.53:0/1041275221 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced8003520 con 0x7fcee80afbe0 2023-10-21T16:14:34.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee7fff640 1 -- 172.21.15.53:0/1041275221 >> v2:172.21.15.124:3300/0 conn(0x7fcee8071070 msgr2=0x7fcee80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee7fff640 1 --2- 172.21.15.53:0/1041275221 >> v2:172.21.15.124:3300/0 conn(0x7fcee8071070 0x7fcee80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee7fff640 1 -- 172.21.15.53:0/1041275221 >> v1:172.21.15.53:6789/0 conn(0x7fcee80afbe0 legacy=0x7fcee80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.370+0000 7fcee7fff640 1 -- 172.21.15.53:0/1041275221 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80b2340 con 0x7fcee806dfc0 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.371+0000 7fcee7fff640 1 --2- 172.21.15.53:0/1041275221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee8070b30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcee80c8d10 tx=0x7fced402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ed413e5d1f0c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.372+0000 7fcee77fe640 1 -- 172.21.15.53:0/1041275221 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4031830 con 0x7fcee806dfc0 2023-10-21T16:14:34.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.372+0000 7fcee77fe640 1 -- 172.21.15.53:0/1041275221 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4031990 con 0x7fcee806dfc0 2023-10-21T16:14:34.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.372+0000 7fcee77fe640 1 -- 172.21.15.53:0/1041275221 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4031d20 con 0x7fcee806dfc0 2023-10-21T16:14:34.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.372+0000 7fceeeab4640 1 -- 172.21.15.53:0/1041275221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 msgr2=0x7fcee8070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.372+0000 7fceeeab4640 1 --2- 172.21.15.53:0/1041275221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee8070b30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcee80c8d10 tx=0x7fced402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 -- 172.21.15.53:0/1041275221 shutdown_connections 2023-10-21T16:14:34.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 --2- 172.21.15.53:0/1041275221 >> v2:172.21.15.124:3300/0 conn(0x7fcee8071070 0x7fcee80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 --2- 172.21.15.53:0/1041275221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee8070b30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 -- 172.21.15.53:0/1041275221 >> 172.21.15.53:0/1041275221 conn(0x7fcee806a5d0 msgr2=0x7fcee80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:34.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 -- 172.21.15.53:0/1041275221 shutdown_connections 2023-10-21T16:14:34.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.373+0000 7fceeeab4640 1 -- 172.21.15.53:0/1041275221 wait complete. 2023-10-21T16:14:34.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.374+0000 7fceeeab4640 1 Processor -- start 2023-10-21T16:14:34.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.375+0000 7fceeeab4640 1 -- start start 2023-10-21T16:14:34.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.375+0000 7fceeeab4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.375+0000 7fceeeab4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcee8071070 0x7fcee80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceeeab4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcee80afbe0 0x7fcee80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceeeab4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c5090 con 0x7fcee806dfc0 2023-10-21T16:14:34.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceeeab4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c4480 con 0x7fcee8071070 2023-10-21T16:14:34.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceeeab4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee80c4760 con 0x7fcee80afbe0 2023-10-21T16:14:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53794/0 (socket says 172.21.15.53:53794) 2023-10-21T16:14:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 -- 172.21.15.53:0/2317956961 learned_addr learned my addr 172.21.15.53:0/2317956961 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 -- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcee80afbe0 msgr2=0x7fcee80c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcee80afbe0 0x7fcee80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 -- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcee8071070 msgr2=0x7fcee80c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcee8071070 0x7fcee80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.376+0000 7fceec850640 1 -- 172.21.15.53:0/2317956961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4009400 con 0x7fcee806dfc0 2023-10-21T16:14:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fceec850640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c410 tx=0x7fcedc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc005ce0 con 0x7fcee806dfc0 2023-10-21T16:14:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc005e40 con 0x7fcee806dfc0 2023-10-21T16:14:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007310 con 0x7fcee806dfc0 2023-10-21T16:14:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee80c4a40 con 0x7fcee806dfc0 2023-10-21T16:14:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.377+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee80c3fe0 con 0x7fcee806dfc0 2023-10-21T16:14:34.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.378+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80c4230 con 0x7fcee806dfc0 2023-10-21T16:14:34.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.379+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fcedc007470 con 0x7fcee806dfc0 2023-10-21T16:14:34.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.380+0000 7fcee57fa640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 0x7fcecc06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.380+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fcedc089210 con 0x7fcee806dfc0 2023-10-21T16:14:34.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.381+0000 7fcee7fff640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 0x7fcecc06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.382+0000 7fcee7fff640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 0x7fcecc06d500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fced4036000 tx=0x7fced4030000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.384+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcedc053de0 con 0x7fcee806dfc0 2023-10-21T16:14:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.599+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcee8070630 con 0x7fcecc06b040 2023-10-21T16:14:34.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.617+0000 7fcee57fa640 1 -- 172.21.15.53:0/2317956961 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fced4031b00 con 0x7fcecc06b040 2023-10-21T16:14:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:34.619 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:56", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 msgr2=0x7fcecc06d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 0x7fcecc06d500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fced4036000 tx=0x7fced4030000 comp rx=0 tx=0).stop 2023-10-21T16:14:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 msgr2=0x7fcee80c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c410 tx=0x7fcedc00c8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 shutdown_connections 2023-10-21T16:14:34.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fcecc06b040 0x7fcecc06d500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcee80afbe0 0x7fcee80c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcee8071070 0x7fcee80c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 --2- 172.21.15.53:0/2317956961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee806dfc0 0x7fcee80c3aa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 >> 172.21.15.53:0/2317956961 conn(0x7fcee806a5d0 msgr2=0x7fcee806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 shutdown_connections 2023-10-21T16:14:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.620+0000 7fceeeab4640 1 -- 172.21.15.53:0/2317956961 wait complete. 2023-10-21T16:14:34.642 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:56", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:34.642 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:56', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:34.642 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:34.643 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:34.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:34.643 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-21T16:14:34.703 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:34.704 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:34.704 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:34.704 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:34.705 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:34.705 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:34.705 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:34.705 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:34.705 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:34.706 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:34.706 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:34.707 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_04_00_UTC', 'scheduled-2023-10-21-16_05_01_UTC', 'scheduled-2023-10-21-16_06_00_UTC', 'scheduled-2023-10-21-16_07_00_UTC', 'scheduled-2023-10-21-16_08_01_UTC', 'scheduled-2023-10-21-16_09_01_UTC', 'scheduled-2023-10-21-16_10_01_UTC', 'scheduled-2023-10-21-16_11_01_UTC', 'scheduled-2023-10-21-16_12_00_UTC', 'scheduled-2023-10-21-16_13_01_UTC', 'scheduled-2023-10-21-16_14_01_UTC'] 2023-10-21T16:14:34.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-10-21T16:14:34.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.943+0000 7fed13cb4640 1 Processor -- start 2023-10-21T16:14:34.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.943+0000 7fed13cb4640 1 -- start start 2023-10-21T16:14:34.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed13cb4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed13cb4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed0c0b2f30 0x7fed0c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed13cb4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c06cc50 con 0x7fed0c0cb150 2023-10-21T16:14:34.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed13cb4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0cbb10 con 0x7fed0c0b2f30 2023-10-21T16:14:34.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed13cb4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c06cf30 con 0x7fed0c161690 2023-10-21T16:14:34.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed12251640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed0c0b2f30 0x7fed0c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed1124f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fed0c0cb150 0x7fed0c0702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48974/0 (socket says 172.21.15.53:48974) 2023-10-21T16:14:34.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed1124f640 1 -- 172.21.15.53:0/3390491032 learned_addr learned my addr 172.21.15.53:0/3390491032 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:34.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.945+0000 7fed11a50640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.946+0000 7fed12251640 1 -- 172.21.15.53:0/3390491032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 msgr2=0x7fed0c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:34.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.946+0000 7fed12251640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.946+0000 7fed12251640 1 -- 172.21.15.53:0/3390491032 >> v1:172.21.15.53:6789/0 conn(0x7fed0c0cb150 legacy=0x7fed0c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.946+0000 7fed12251640 1 -- 172.21.15.53:0/3390491032 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c06d210 con 0x7fed0c0b2f30 2023-10-21T16:14:34.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.947+0000 7fed12251640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0b2f30 0x7fed0c0cb5d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed040097a0 tx=0x7fed0402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4618d62378c35f43 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.947+0000 7fed10a4e640 1 -- 172.21.15.53:0/3390491032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0402d040 con 0x7fed0c0b2f30 2023-10-21T16:14:34.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.947+0000 7fed10a4e640 1 -- 172.21.15.53:0/3390491032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04031810 con 0x7fed0c0b2f30 2023-10-21T16:14:34.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.947+0000 7fed10a4e640 1 -- 172.21.15.53:0/3390491032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04031ba0 con 0x7fed0c0b2f30 2023-10-21T16:14:34.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 -- 172.21.15.53:0/3390491032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0b2f30 msgr2=0x7fed0c0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0b2f30 0x7fed0c0cb5d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed040097a0 tx=0x7fed0402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 -- 172.21.15.53:0/3390491032 shutdown_connections 2023-10-21T16:14:34.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0b2f30 0x7fed0c0cb5d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3390491032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 -- 172.21.15.53:0/3390491032 >> 172.21.15.53:0/3390491032 conn(0x7fed0c06a5d0 msgr2=0x7fed0c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:34.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 -- 172.21.15.53:0/3390491032 shutdown_connections 2023-10-21T16:14:34.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.948+0000 7fed13cb4640 1 -- 172.21.15.53:0/3390491032 wait complete. 2023-10-21T16:14:34.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.950+0000 7fed13cb4640 1 Processor -- start 2023-10-21T16:14:34.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.950+0000 7fed13cb4640 1 -- start start 2023-10-21T16:14:34.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c0b2f30 0x7fed0c1f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c1f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c164a50 con 0x7fed0c0cb150 2023-10-21T16:14:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1faaf0 con 0x7fed0c0b2f30 2023-10-21T16:14:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed13cb4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1588e0 con 0x7fed0c161690 2023-10-21T16:14:34.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53814/0 (socket says 172.21.15.53:53814) 2023-10-21T16:14:34.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 -- 172.21.15.53:0/3412111297 learned_addr learned my addr 172.21.15.53:0/3412111297 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:34.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 -- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 msgr2=0x7fed0c1f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c1f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed11a50640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c0b2f30 0x7fed0c1f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 -- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c0b2f30 msgr2=0x7fed0c1f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:34.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c0b2f30 0x7fed0c1f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:34.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.951+0000 7fed1124f640 1 -- 172.21.15.53:0/3412111297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04009400 con 0x7fed0c0cb150 2023-10-21T16:14:34.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed1124f640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c4a0 tx=0x7fecfc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc005ce0 con 0x7fed0c0cb150 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc005e40 con 0x7fed0c0cb150 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007270 con 0x7fed0c0cb150 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c158bc0 con 0x7fed0c0cb150 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.952+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c159180 con 0x7fed0c0cb150 2023-10-21T16:14:34.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.953+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c1593a0 con 0x7fed0c0cb150 2023-10-21T16:14:34.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.954+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fecfc007490 con 0x7fed0c0cb150 2023-10-21T16:14:34.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.956+0000 7fed027fc640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 0x7fecf006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:34.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.956+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fecfc088ff0 con 0x7fed0c0cb150 2023-10-21T16:14:34.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.956+0000 7fed11a50640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 0x7fecf006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:34.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.958+0000 7fed11a50640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 0x7fecf006d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed08009e90 tx=0x7fed0800b040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:34.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:34.960+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecfc053cc0 con 0x7fed0c0cb150 2023-10-21T16:14:35.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.177+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed0c0b1990 con 0x7fecf006b070 2023-10-21T16:14:35.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.195+0000 7fed027fc640 1 -- 172.21.15.53:0/3412111297 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fed08007e40 con 0x7fecf006b070 2023-10-21T16:14:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:35.197 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:57", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:35.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 msgr2=0x7fecf006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 0x7fecf006d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed08009e90 tx=0x7fed0800b040 comp rx=0 tx=0).stop 2023-10-21T16:14:35.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 msgr2=0x7fed0c14a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c4a0 tx=0x7fecfc00c960 comp rx=0 tx=0).stop 2023-10-21T16:14:35.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 shutdown_connections 2023-10-21T16:14:35.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fecf006b070 0x7fecf006d530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed0c161690 0x7fed0c1f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed0c0cb150 0x7fed0c14a700 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 --2- 172.21.15.53:0/3412111297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed0c0b2f30 0x7fed0c1f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 >> 172.21.15.53:0/3412111297 conn(0x7fed0c06a5d0 msgr2=0x7fed0c0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:35.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 shutdown_connections 2023-10-21T16:14:35.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.199+0000 7fed13cb4640 1 -- 172.21.15.53:0/3412111297 wait complete. 2023-10-21T16:14:35.223 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:03:57", "first": "2023-10-21T16:04:00", "last": "2023-10-21T16:14:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-21T16:14:35.224 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:03:57', 'first': '2023-10-21T16:04:00', 'last': '2023-10-21T16:14:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-21T16:14:35.224 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T16:14:35.224 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T16:14:35.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2023-10-21T16:14:35.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.455+0000 7f3871d5e640 1 Processor -- start 2023-10-21T16:14:35.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.456+0000 7f3871d5e640 1 -- start start 2023-10-21T16:14:35.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c15cae0 0x7f386c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f386c0c8b40 0x7f386c0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f386c0aff70 0x7f386c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0c94e0 con 0x7f386c0aff70 2023-10-21T16:14:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0ccbc0 con 0x7f386c0c8b40 2023-10-21T16:14:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.457+0000 7f3871d5e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b0940 con 0x7f386c15cae0 2023-10-21T16:14:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f386c0aff70 0x7f386c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f386c0aff70 0x7f386c0b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53828/0 (socket says 172.21.15.53:53828) 2023-10-21T16:14:35.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 -- 172.21.15.53:0/423920320 learned_addr learned my addr 172.21.15.53:0/423920320 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f386bfff640 1 --2- 172.21.15.53:0/423920320 >> v2:172.21.15.124:3300/0 conn(0x7f386c0c8b40 0x7f386c0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f3870d5c640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c15cae0 0x7f386c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 -- 172.21.15.53:0/423920320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c15cae0 msgr2=0x7f386c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c15cae0 0x7f386c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 -- 172.21.15.53:0/423920320 >> v2:172.21.15.124:3300/0 conn(0x7f386c0c8b40 msgr2=0x7f386c0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 --2- 172.21.15.53:0/423920320 >> v2:172.21.15.124:3300/0 conn(0x7f386c0c8b40 0x7f386c0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.458+0000 7f387155d640 1 -- 172.21.15.53:0/423920320 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0ce1f0 con 0x7f386c0aff70 2023-10-21T16:14:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.459+0000 7f387155d640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c0b0400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f386c06e550 tx=0x7f386402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29202eea5b4ffb1f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.460+0000 7f386b7fe640 1 -- 172.21.15.53:0/423920320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386402ca70 con 0x7f386c0aff70 2023-10-21T16:14:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.460+0000 7f386b7fe640 1 -- 172.21.15.53:0/423920320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386402cbd0 con 0x7f386c0aff70 2023-10-21T16:14:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.460+0000 7f386b7fe640 1 -- 172.21.15.53:0/423920320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640343f0 con 0x7f386c0aff70 2023-10-21T16:14:35.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.460+0000 7f3871d5e640 1 -- 172.21.15.53:0/423920320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 msgr2=0x7f386c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.460+0000 7f3871d5e640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c0b0400 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f386c06e550 tx=0x7f386402bed0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 -- 172.21.15.53:0/423920320 shutdown_connections 2023-10-21T16:14:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c0b0400 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 --2- 172.21.15.53:0/423920320 >> v2:172.21.15.124:3300/0 conn(0x7f386c0c8b40 0x7f386c0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 --2- 172.21.15.53:0/423920320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c15cae0 0x7f386c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 -- 172.21.15.53:0/423920320 >> 172.21.15.53:0/423920320 conn(0x7f386c06c050 msgr2=0x7f386c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 -- 172.21.15.53:0/423920320 shutdown_connections 2023-10-21T16:14:35.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.461+0000 7f3871d5e640 1 -- 172.21.15.53:0/423920320 wait complete. 2023-10-21T16:14:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.463+0000 7f3871d5e640 1 Processor -- start 2023-10-21T16:14:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.463+0000 7f3871d5e640 1 -- start start 2023-10-21T16:14:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c0c8b40 0x7f386c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 0x7f386c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0c4230 con 0x7f386c0aff70 2023-10-21T16:14:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c158850 con 0x7f386c15cae0 2023-10-21T16:14:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3871d5e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c158b30 con 0x7f386c0c8b40 2023-10-21T16:14:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3870d5c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3870d5c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53834/0 (socket says 172.21.15.53:53834) 2023-10-21T16:14:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3870d5c640 1 -- 172.21.15.53:0/430774443 learned_addr learned my addr 172.21.15.53:0/430774443 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f387155d640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 0x7f386c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f386bfff640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c0c8b40 0x7f386c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3870d5c640 1 -- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c0c8b40 msgr2=0x7f386c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.465+0000 7f3870d5c640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c0c8b40 0x7f386c15abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3870d5c640 1 -- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 msgr2=0x7f386c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3870d5c640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 0x7f386c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3870d5c640 1 -- 172.21.15.53:0/430774443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864009b30 con 0x7f386c0aff70 2023-10-21T16:14:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f387155d640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 0x7f386c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3870d5c640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f386000d5f0 tx=0x7f386000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600125f0 con 0x7f386c0aff70 2023-10-21T16:14:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860012750 con 0x7f386c0aff70 2023-10-21T16:14:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860012b00 con 0x7f386c0aff70 2023-10-21T16:14:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c158e10 con 0x7f386c0aff70 2023-10-21T16:14:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.466+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c0c4510 con 0x7f386c0aff70 2023-10-21T16:14:35.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.467+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c0cc550 con 0x7f386c0aff70 2023-10-21T16:14:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.469+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3860012c60 con 0x7f386c0aff70 2023-10-21T16:14:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.469+0000 7f38697fa640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 0x7f385006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.470+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3860089fd0 con 0x7f386c0aff70 2023-10-21T16:14:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.470+0000 7f386bfff640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 0x7f385006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.471+0000 7f386bfff640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 0x7f385006d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f386c15ba50 tx=0x7f3858008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:35.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.474+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f386c0cc550 con 0x7f386c0aff70 2023-10-21T16:14:35.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.695+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f386c0c9720 con 0x7f385006b070 2023-10-21T16:14:35.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.885+0000 7f38697fa640 1 -- 172.21.15.53:0/430774443 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3858009520 con 0x7f385006b070 2023-10-21T16:14:35.887 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-21T16:14:35.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.888+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 msgr2=0x7f385006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.888+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 0x7f385006d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f386c15ba50 tx=0x7f3858008040 comp rx=0 tx=0).stop 2023-10-21T16:14:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 msgr2=0x7f386c1576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f386000d5f0 tx=0x7f386000dab0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 shutdown_connections 2023-10-21T16:14:35.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f385006b070 0x7f385006d530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f386c15cae0 0x7f386c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f386c0c8b40 0x7f386c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 --2- 172.21.15.53:0/430774443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f386c0aff70 0x7f386c1576d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:35.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 >> 172.21.15.53:0/430774443 conn(0x7f386c06c050 msgr2=0x7f386c0acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:35.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 shutdown_connections 2023-10-21T16:14:35.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:35.889+0000 7f3871d5e640 1 -- 172.21.15.53:0/430774443 wait complete. 2023-10-21T16:14:35.912 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-21T16:14:35.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:35.913 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-21T16:14:35.971 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:35.971 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:35.971 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:35.972 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:35.972 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:35.972 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:35.972 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:35.972 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:35.973 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:35.973 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:35.973 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:35.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:35.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:35.975 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:36.042 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:36.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.043 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_05_01_UTC) 2023-10-21T16:14:36.115 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:36.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.116 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_06_01_UTC) 2023-10-21T16:14:36.192 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:36.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.193 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_07_01_UTC) 2023-10-21T16:14:36.266 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:36.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.267 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:36.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:36.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.336 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:36.403 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:36.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.404 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:36.475 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:36.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.476 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_11_01_UTC) 2023-10-21T16:14:36.549 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:36.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.550 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:36.613 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:36.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.614 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:36.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.684 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:36.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:36.755 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-21T16:14:36.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2023-10-21T16:14:37.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.047+0000 7f3336d89640 1 Processor -- start 2023-10-21T16:14:37.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.047+0000 7f3336d89640 1 -- start start 2023-10-21T16:14:37.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3336d89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f333015cd80 0x7f33300b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3336d89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300b6730 0x7f33300cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3336d89640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333015d160 con 0x7f33300cdda0 2023-10-21T16:14:37.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3336d89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33300ce230 con 0x7f333015cd80 2023-10-21T16:14:37.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3336d89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300b7b90 con 0x7f33300b6730 2023-10-21T16:14:37.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3334b25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f333015cd80 0x7f33300b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3334b25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f333015cd80 0x7f33300b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53836/0 (socket says 172.21.15.53:53836) 2023-10-21T16:14:37.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3334b25640 1 -- 172.21.15.53:0/2523702168 learned_addr learned my addr 172.21.15.53:0/2523702168 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:37.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.049+0000 7f3335326640 1 --1- 172.21.15.53:0/2523702168 >> v1:172.21.15.53:6789/0 conn(0x7f33300cdda0 0x7f33300b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48976/0 (socket says 172.21.15.53:48976) 2023-10-21T16:14:37.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.050+0000 7f3327fff640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300b6730 0x7f33300cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.050+0000 7f3334b25640 1 -- 172.21.15.53:0/2523702168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300b6730 msgr2=0x7f33300cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:37.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.050+0000 7f3334b25640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300b6730 0x7f33300cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.050+0000 7f3334b25640 1 -- 172.21.15.53:0/2523702168 >> v1:172.21.15.53:6789/0 conn(0x7f33300cdda0 legacy=0x7f33300b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.050+0000 7f3334b25640 1 -- 172.21.15.53:0/2523702168 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300b8020 con 0x7f333015cd80 2023-10-21T16:14:37.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.051+0000 7f3334b25640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300b87d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f332c009870 tx=0x7f332c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf1b9e2f00140bc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.051+0000 7f33277fe640 1 -- 172.21.15.53:0/2523702168 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c030840 con 0x7f333015cd80 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.051+0000 7f33277fe640 1 -- 172.21.15.53:0/2523702168 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c0309a0 con 0x7f333015cd80 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.051+0000 7f33277fe640 1 -- 172.21.15.53:0/2523702168 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c030d30 con 0x7f333015cd80 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 -- 172.21.15.53:0/2523702168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 msgr2=0x7f33300b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300b87d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f332c009870 tx=0x7f332c02b980 comp rx=0 tx=0).stop 2023-10-21T16:14:37.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 -- 172.21.15.53:0/2523702168 shutdown_connections 2023-10-21T16:14:37.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300b6730 0x7f33300cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 --2- 172.21.15.53:0/2523702168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300b87d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 -- 172.21.15.53:0/2523702168 >> 172.21.15.53:0/2523702168 conn(0x7f333006b5d0 msgr2=0x7f333006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:37.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 -- 172.21.15.53:0/2523702168 shutdown_connections 2023-10-21T16:14:37.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.052+0000 7f3336d89640 1 -- 172.21.15.53:0/2523702168 wait complete. 2023-10-21T16:14:37.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.054+0000 7f3336d89640 1 Processor -- start 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.054+0000 7f3336d89640 1 -- start start 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33300b6730 0x7f3330159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300cdda0 0x7f33300aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333015e030 con 0x7f333015cd80 2023-10-21T16:14:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333015d420 con 0x7f33300b6730 2023-10-21T16:14:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3336d89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333015d700 con 0x7f33300cdda0 2023-10-21T16:14:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3335326640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3335326640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53852/0 (socket says 172.21.15.53:53852) 2023-10-21T16:14:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.055+0000 7f3335326640 1 -- 172.21.15.53:0/506854602 learned_addr learned my addr 172.21.15.53:0/506854602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3327fff640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300cdda0 0x7f33300aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3334b25640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33300b6730 0x7f3330159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 -- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300cdda0 msgr2=0x7f33300aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300cdda0 0x7f33300aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 -- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33300b6730 msgr2=0x7f3330159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33300b6730 0x7f3330159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 -- 172.21.15.53:0/506854602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0094d0 con 0x7f333015cd80 2023-10-21T16:14:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f3335326640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f332800c780 tx=0x7f332800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.056+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800ce90 con 0x7f333015cd80 2023-10-21T16:14:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.057+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800a640 con 0x7f333015cd80 2023-10-21T16:14:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.057+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333015d9e0 con 0x7f333015cd80 2023-10-21T16:14:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.057+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800a9f0 con 0x7f333015cd80 2023-10-21T16:14:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.057+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33301596b0 con 0x7f333015cd80 2023-10-21T16:14:37.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.058+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333015e1a0 con 0x7f333015cd80 2023-10-21T16:14:37.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.059+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f332800ab50 con 0x7f333015cd80 2023-10-21T16:14:37.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.060+0000 7f33257fa640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 0x7f331c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:37.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.060+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3328089330 con 0x7f333015cd80 2023-10-21T16:14:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.060+0000 7f3334b25640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 0x7f331c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.062+0000 7f3334b25640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 0x7f331c06d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f332c036000 tx=0x7f332c03b040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.065+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3328053fa0 con 0x7f333015cd80 2023-10-21T16:14:37.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.284+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f33300b7260 con 0x7f331c06b070 2023-10-21T16:14:37.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.480+0000 7f33257fa640 1 -- 172.21.15.53:0/506854602 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f332c030b90 con 0x7f331c06b070 2023-10-21T16:14:37.482 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-21T16:14:37.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 msgr2=0x7f331c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 0x7f331c06d530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f332c036000 tx=0x7f332c03b040 comp rx=0 tx=0).stop 2023-10-21T16:14:37.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 msgr2=0x7f33300cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:37.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f332800c780 tx=0x7f332800cc40 comp rx=0 tx=0).stop 2023-10-21T16:14:37.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 shutdown_connections 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f331c06b070 0x7f331c06d530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f333015cd80 0x7f33300cb440 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33300cdda0 0x7f33300aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 --2- 172.21.15.53:0/506854602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33300b6730 0x7f3330159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 >> 172.21.15.53:0/506854602 conn(0x7f333006b5d0 msgr2=0x7f33300c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:37.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 shutdown_connections 2023-10-21T16:14:37.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:37.484+0000 7f3336d89640 1 -- 172.21.15.53:0/506854602 wait complete. 2023-10-21T16:14:37.506 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-21T16:14:37.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.507 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-21T16:14:37.562 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:37.563 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:37.563 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:37.563 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:37.563 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:37.564 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:37.564 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:37.564 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:37.564 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:37.565 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:37.565 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:37.566 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:37.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.567 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_04_01_UTC) 2023-10-21T16:14:37.633 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:37.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.634 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:37.702 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:37.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.703 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:37.778 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:37.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.779 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:37.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:37.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.853 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:37.933 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:37.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:37.933 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:38.013 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:38.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.014 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:38.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:38.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.088 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_11_00_UTC) 2023-10-21T16:14:38.158 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:38.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.159 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:38.233 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:38.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.233 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_13_01_UTC) 2023-10-21T16:14:38.327 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:38.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.328 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:38.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:38.380 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-21T16:14:38.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2023-10-21T16:14:38.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.688+0000 7fb978761640 1 Processor -- start 2023-10-21T16:14:38.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.688+0000 7fb978761640 1 -- start start 2023-10-21T16:14:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.691+0000 7fb978761640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb970159b50 0x7fb97015e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.691+0000 7fb978761640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9700ae740 0x7fb9700aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.691+0000 7fb978761640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9700cd430 con 0x7fb9700ceaa0 2023-10-21T16:14:38.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.691+0000 7fb978761640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9700af0f0 con 0x7fb9700ae740 2023-10-21T16:14:38.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.691+0000 7fb978761640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9700cf260 con 0x7fb970159b50 2023-10-21T16:14:38.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9700ae740 0x7fb9700aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9700ae740 0x7fb9700aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53862/0 (socket says 172.21.15.53:53862) 2023-10-21T16:14:38.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 -- 172.21.15.53:0/1874132747 learned_addr learned my addr 172.21.15.53:0/1874132747 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:38.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb9764fd640 1 --2- 172.21.15.53:0/1874132747 >> v2:172.21.15.124:3300/0 conn(0x7fb970159b50 0x7fb97015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 -- 172.21.15.53:0/1874132747 >> v2:172.21.15.124:3300/0 conn(0x7fb970159b50 msgr2=0x7fb97015e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:38.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 --2- 172.21.15.53:0/1874132747 >> v2:172.21.15.124:3300/0 conn(0x7fb970159b50 0x7fb97015e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:38.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 -- 172.21.15.53:0/1874132747 >> v1:172.21.15.124:6789/0 conn(0x7fb9700ceaa0 legacy=0x7fb9700cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:38.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.692+0000 7fb976cfe640 1 -- 172.21.15.53:0/1874132747 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700cf540 con 0x7fb9700ae740 2023-10-21T16:14:38.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.694+0000 7fb976cfe640 1 --2- 172.21.15.53:0/1874132747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9700aebb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb970067110 tx=0x7fb96802e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98449a471ec95541 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:38.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.695+0000 7fb9754fb640 1 -- 172.21.15.53:0/1874132747 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968035600 con 0x7fb9700ae740 2023-10-21T16:14:38.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.695+0000 7fb9754fb640 1 -- 172.21.15.53:0/1874132747 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968031050 con 0x7fb9700ae740 2023-10-21T16:14:38.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.695+0000 7fb9754fb640 1 -- 172.21.15.53:0/1874132747 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968035980 con 0x7fb9700ae740 2023-10-21T16:14:38.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.696+0000 7fb978761640 1 -- 172.21.15.53:0/1874132747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 msgr2=0x7fb9700aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:38.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.696+0000 7fb978761640 1 --2- 172.21.15.53:0/1874132747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9700aebb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb970067110 tx=0x7fb96802e410 comp rx=0 tx=0).stop 2023-10-21T16:14:38.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 -- 172.21.15.53:0/1874132747 shutdown_connections 2023-10-21T16:14:38.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 --2- 172.21.15.53:0/1874132747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9700aebb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:38.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 --2- 172.21.15.53:0/1874132747 >> v2:172.21.15.124:3300/0 conn(0x7fb970159b50 0x7fb97015e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:38.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 -- 172.21.15.53:0/1874132747 >> 172.21.15.53:0/1874132747 conn(0x7fb97006d430 msgr2=0x7fb97006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:38.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 -- 172.21.15.53:0/1874132747 shutdown_connections 2023-10-21T16:14:38.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.697+0000 7fb978761640 1 -- 172.21.15.53:0/1874132747 wait complete. 2023-10-21T16:14:38.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.700+0000 7fb978761640 1 Processor -- start 2023-10-21T16:14:38.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.700+0000 7fb978761640 1 -- start start 2023-10-21T16:14:38.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9700ceaa0 0x7fb9700c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb970159b50 0x7fb9700c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701f78f0 con 0x7fb9700ae740 2023-10-21T16:14:38.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970159360 con 0x7fb970159b50 2023-10-21T16:14:38.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.702+0000 7fb978761640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb970144150 con 0x7fb9700ceaa0 2023-10-21T16:14:38.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.703+0000 7fb9764fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.703+0000 7fb9764fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53874/0 (socket says 172.21.15.53:53874) 2023-10-21T16:14:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.703+0000 7fb9764fd640 1 -- 172.21.15.53:0/1191258967 learned_addr learned my addr 172.21.15.53:0/1191258967 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.703+0000 7fb975cfc640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9700ceaa0 0x7fb9700c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.703+0000 7fb976cfe640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb970159b50 0x7fb9700c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.704+0000 7fb9764fd640 1 -- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9700ceaa0 msgr2=0x7fb9700c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.704+0000 7fb9764fd640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9700ceaa0 0x7fb9700c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.704+0000 7fb9764fd640 1 -- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb970159b50 msgr2=0x7fb9700c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:38.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.704+0000 7fb9764fd640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb970159b50 0x7fb9700c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:38.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.704+0000 7fb9764fd640 1 -- 172.21.15.53:0/1191258967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968009b30 con 0x7fb9700ae740 2023-10-21T16:14:38.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.705+0000 7fb9764fd640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00e9b0 tx=0x7fb96c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:38.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.705+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c009670 con 0x7fb9700ae740 2023-10-21T16:14:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.706+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00c9a0 con 0x7fb9700ae740 2023-10-21T16:14:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.706+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00cd50 con 0x7fb9700ae740 2023-10-21T16:14:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.706+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9701f6f50 con 0x7fb9700ae740 2023-10-21T16:14:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.706+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701f72e0 con 0x7fb9700ae740 2023-10-21T16:14:38.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.710+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9701f7e30 con 0x7fb9700ae740 2023-10-21T16:14:38.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.712+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb96c013470 con 0x7fb9700ae740 2023-10-21T16:14:38.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.713+0000 7fb966ffd640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 0x7fb95406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.714+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb96c089f20 con 0x7fb9700ae740 2023-10-21T16:14:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.715+0000 7fb975cfc640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 0x7fb95406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.716+0000 7fb975cfc640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 0x7fb95406d580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb970155400 tx=0x7fb960009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:38.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.721+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb96c054a80 con 0x7fb9700ae740 2023-10-21T16:14:38.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:38.953+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9700cd9b0 con 0x7fb95406b0c0 2023-10-21T16:14:39.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.154+0000 7fb966ffd640 1 -- 172.21.15.53:0/1191258967 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb9600014a0 con 0x7fb95406b0c0 2023-10-21T16:14:39.156 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-21T16:14:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 msgr2=0x7fb95406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 0x7fb95406d580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb970155400 tx=0x7fb960009040 comp rx=0 tx=0).stop 2023-10-21T16:14:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 msgr2=0x7fb9701f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00e9b0 tx=0x7fb96c00ee70 comp rx=0 tx=0).stop 2023-10-21T16:14:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 shutdown_connections 2023-10-21T16:14:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb95406b0c0 0x7fb95406d580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb970159b50 0x7fb9700c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9700ceaa0 0x7fb9700c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 --2- 172.21.15.53:0/1191258967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9700ae740 0x7fb9701f6920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:39.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 >> 172.21.15.53:0/1191258967 conn(0x7fb97006d430 msgr2=0x7fb97006a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 shutdown_connections 2023-10-21T16:14:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:39.158+0000 7fb978761640 1 -- 172.21.15.53:0/1191258967 wait complete. 2023-10-21T16:14:39.180 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-21T16:14:39.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.181 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-21T16:14:39.243 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:39.244 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:39.244 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:39.244 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:39.244 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:39.244 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:39.245 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:39.246 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:39.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.247 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:39.343 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:39.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.344 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:39.397 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:39.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.398 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:39.455 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:39.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.456 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:39.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:39.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.514 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:39.570 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:39.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.571 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:39.633 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:39.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.634 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_10_01_UTC) 2023-10-21T16:14:39.696 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:39.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.696 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_11_00_UTC) 2023-10-21T16:14:39.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:39.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.769 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:39.831 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:39.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.832 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:39.904 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:39.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.905 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:39.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:39.985 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-21T16:14:40.032 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2023-10-21T16:14:40.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.302+0000 7faf23be9640 1 Processor -- start 2023-10-21T16:14:40.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.302+0000 7faf23be9640 1 -- start start 2023-10-21T16:14:40.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.303+0000 7faf23be9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf1c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.303+0000 7faf23be9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1c0b2620 con 0x7faf1c0cb130 2023-10-21T16:14:40.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.303+0000 7faf23be9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1c0b2060 con 0x7faf1c06dfc0 2023-10-21T16:14:40.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.303+0000 7faf23be9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0b2340 con 0x7faf1c0afbe0 2023-10-21T16:14:40.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.304+0000 7faf22186640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faf1c0cb130 0x7faf1c06c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48984/0 (socket says 172.21.15.53:48984) 2023-10-21T16:14:40.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.304+0000 7faf22186640 1 -- 172.21.15.53:0/2351253811 learned_addr learned my addr 172.21.15.53:0/2351253811 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:40.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.304+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf1c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.304+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366888075 0 0) 0x7faf1c0b2620 con 0x7faf1c0cb130 2023-10-21T16:14:40.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.304+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf08003520 con 0x7faf1c0cb130 2023-10-21T16:14:40.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175199911 0 0) 0x7faf1c0b2060 con 0x7faf1c06dfc0 2023-10-21T16:14:40.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf1c0b2620 con 0x7faf1c06dfc0 2023-10-21T16:14:40.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879940646 0 0) 0x7faf08003520 con 0x7faf1c0cb130 2023-10-21T16:14:40.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 msgr2=0x7faf1c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:40.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf1c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 >> v1:172.21.15.124:6789/0 conn(0x7faf1c06dfc0 legacy=0x7faf1c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c070350 con 0x7faf1c0cb130 2023-10-21T16:14:40.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7faf14001670 con 0x7faf1c0cb130 2023-10-21T16:14:40.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf1c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:14:40.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 >> v1:172.21.15.53:6789/0 conn(0x7faf1c0cb130 legacy=0x7faf1c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf08003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf08004d00 0x7faf08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0b2340 con 0x7faf08003b30 2023-10-21T16:14:40.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0800ba30 con 0x7faf08004d00 2023-10-21T16:14:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0800bbf0 con 0x7faf1c0afbe0 2023-10-21T16:14:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf08003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.305+0000 7faf21184640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf22186640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf08004d00 0x7faf08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf21985640 1 -- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 msgr2=0x7faf080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0afbe0 0x7faf080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf21985640 1 -- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf08004d00 msgr2=0x7faf08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf08004d00 0x7faf08005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf21985640 1 -- 172.21.15.53:0/2351253811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009a70 con 0x7faf08003b30 2023-10-21T16:14:40.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.306+0000 7faf22186640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf08004d00 0x7faf08005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.307+0000 7faf21985640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf08003f30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e350 tx=0x7faf080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f983ea48a3e923 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.307+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1802cd70 con 0x7faf08003b30 2023-10-21T16:14:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.307+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1800f040 con 0x7faf08003b30 2023-10-21T16:14:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.307+0000 7faf20983640 1 -- 172.21.15.53:0/2351253811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18034640 con 0x7faf08003b30 2023-10-21T16:14:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 -- 172.21.15.53:0/2351253811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 msgr2=0x7faf08003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 --2- 172.21.15.53:0/2351253811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf08003f30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faf1800e350 tx=0x7faf080057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf22186640 1 -- 172.21.15.53:0/2351253811 reap_dead start 2023-10-21T16:14:40.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 -- 172.21.15.53:0/2351253811 shutdown_connections 2023-10-21T16:14:40.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 -- 172.21.15.53:0/2351253811 >> 172.21.15.53:0/2351253811 conn(0x7faf1c06a5d0 msgr2=0x7faf1c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:40.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 -- 172.21.15.53:0/2351253811 shutdown_connections 2023-10-21T16:14:40.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.308+0000 7faf23be9640 1 -- 172.21.15.53:0/2351253811 wait complete. 2023-10-21T16:14:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.310+0000 7faf23be9640 1 Processor -- start 2023-10-21T16:14:40.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.310+0000 7faf23be9640 1 -- start start 2023-10-21T16:14:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1c0a7ed0 0x7faf1c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 0x7faf1c14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0ab4e0 con 0x7faf08003b30 2023-10-21T16:14:40.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0c5810 con 0x7faf1c0a7ed0 2023-10-21T16:14:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf23be9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0a8350 con 0x7faf1c0a8ea0 2023-10-21T16:14:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53880/0 (socket says 172.21.15.53:53880) 2023-10-21T16:14:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 -- 172.21.15.53:0/1813507541 learned_addr learned my addr 172.21.15.53:0/1813507541 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:40.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf22186640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 0x7faf1c14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21184640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1c0a7ed0 0x7faf1c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 -- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 msgr2=0x7faf1c14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 0x7faf1c14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 -- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1c0a7ed0 msgr2=0x7faf1c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1c0a7ed0 0x7faf1c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.311+0000 7faf21985640 1 -- 172.21.15.53:0/1813507541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0c5a20 con 0x7faf08003b30 2023-10-21T16:14:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf22186640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 0x7faf1c14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf21985640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faf1802c710 tx=0x7faf18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1803e600 con 0x7faf08003b30 2023-10-21T16:14:40.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1803e7a0 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1803eb50 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c0c5d00 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.312+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c0c8410 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.314+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1c04e160 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.315+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7faf1803ecb0 con 0x7faf08003b30 2023-10-21T16:14:40.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.316+0000 7faf127fc640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 0x7faf0006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:40.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.316+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6708+0+0 (secure 0 0 0) 0x7faf180ac470 con 0x7faf08003b30 2023-10-21T16:14:40.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.316+0000 7faf21184640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 0x7faf0006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:40.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.317+0000 7faf21184640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 0x7faf0006d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0c7c00 tx=0x7faf0c009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:40.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.320+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf18076f50 con 0x7faf08003b30 2023-10-21T16:14:40.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.545+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7faf1c071240 con 0x7faf0006b070 2023-10-21T16:14:40.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.758+0000 7faf127fc640 1 -- 172.21.15.53:0/1813507541 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7faf0c00d050 con 0x7faf0006b070 2023-10-21T16:14:40.760 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-21T16:14:40.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.761+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 msgr2=0x7faf0006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.761+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 0x7faf0006d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0c7c00 tx=0x7faf0c009040 comp rx=0 tx=0).stop 2023-10-21T16:14:40.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.761+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 msgr2=0x7faf1c0c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:40.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.761+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faf1802c710 tx=0x7faf18009750 comp rx=0 tx=0).stop 2023-10-21T16:14:40.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 shutdown_connections 2023-10-21T16:14:40.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7faf0006b070 0x7faf0006d530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf1c0a8ea0 0x7faf1c14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1c0a7ed0 0x7faf1c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 --2- 172.21.15.53:0/1813507541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf08003b30 0x7faf1c0c8780 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:40.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 >> 172.21.15.53:0/1813507541 conn(0x7faf1c06a5d0 msgr2=0x7faf1c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:40.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 shutdown_connections 2023-10-21T16:14:40.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:40.762+0000 7faf23be9640 1 -- 172.21.15.53:0/1813507541 wait complete. 2023-10-21T16:14:40.784 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-21T16:14:40.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:40.785 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-21T16:14:40.841 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:40.842 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:40.842 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:40.842 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:40.842 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:40.843 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:40.843 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:40.843 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:40.843 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:40.843 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:40.844 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:40.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:40.845 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:40.926 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:40.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:40.927 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_06_01_UTC) 2023-10-21T16:14:40.986 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:40.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:40.987 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_07_01_UTC) 2023-10-21T16:14:41.047 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:41.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.048 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_08_01_UTC) 2023-10-21T16:14:41.122 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:41.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.123 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_09_01_UTC) 2023-10-21T16:14:41.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:41.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.198 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:41.269 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:41.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.270 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_11_00_UTC) 2023-10-21T16:14:41.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:41.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.376 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:41.460 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:41.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.461 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:41.517 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:41.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.518 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:41.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:41.569 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-21T16:14:41.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2023-10-21T16:14:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.883+0000 7fb25ed1a640 1 Processor -- start 2023-10-21T16:14:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.883+0000 7fb25ed1a640 1 -- start start 2023-10-21T16:14:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.884+0000 7fb25ed1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2580d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.884+0000 7fb25ed1a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2580cf690 0x7fb2580ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.884+0000 7fb25ed1a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2580cd520 con 0x7fb2580d1100 2023-10-21T16:14:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.884+0000 7fb25ed1a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2580d1910 con 0x7fb2580cf690 2023-10-21T16:14:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.884+0000 7fb25ed1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580ce080 con 0x7fb2580cf2b0 2023-10-21T16:14:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25e519640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb2580d1100 0x7fb25815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48990/0 (socket says 172.21.15.53:48990) 2023-10-21T16:14:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25e519640 1 -- 172.21.15.53:0/67717373 learned_addr learned my addr 172.21.15.53:0/67717373 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:41.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25d517640 1 --2- 172.21.15.53:0/67717373 >> v2:172.21.15.124:3300/0 conn(0x7fb2580cf690 0x7fb2580ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25dd18640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2580d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792837503 0 0) 0x7fb2580cd520 con 0x7fb2580d1100 2023-10-21T16:14:41.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb244003520 con 0x7fb2580d1100 2023-10-21T16:14:41.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3079148744 0 0) 0x7fb244003520 con 0x7fb2580d1100 2023-10-21T16:14:41.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 msgr2=0x7fb2580d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:41.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2580d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 >> v2:172.21.15.124:3300/0 conn(0x7fb2580cf690 msgr2=0x7fb2580ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:41.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 --2- 172.21.15.53:0/67717373 >> v2:172.21.15.124:3300/0 conn(0x7fb2580cf690 0x7fb2580ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580cd750 con 0x7fb2580d1100 2023-10-21T16:14:41.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.885+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb2500019f0 con 0x7fb2580d1100 2023-10-21T16:14:41.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 >> v1:172.21.15.53:6789/0 conn(0x7fb2580d1100 legacy=0x7fb25815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:41.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244003970 0x7fb244003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb244004d00 0x7fb244005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580d1910 con 0x7fb244004d00 2023-10-21T16:14:41.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2580ce080 con 0x7fb244003970 2023-10-21T16:14:41.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb24400b9b0 con 0x7fb2580cf2b0 2023-10-21T16:14:41.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb244004d00 0x7fb244005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25dd18640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244003970 0x7fb244003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25d517640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 -- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 msgr2=0x7fb2440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:41.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2580cf2b0 0x7fb2440047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 -- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244003970 msgr2=0x7fb244003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:41.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244003970 0x7fb244003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.886+0000 7fb25e519640 1 -- 172.21.15.53:0/67717373 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb244003520 con 0x7fb244004d00 2023-10-21T16:14:41.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.887+0000 7fb25e519640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb244004d00 0x7fb244005180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb25000a780 tx=0x7fb2440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee48478bdd84542 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250034630 con 0x7fb244004d00 2023-10-21T16:14:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25002f070 con 0x7fb244004d00 2023-10-21T16:14:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25cd16640 1 -- 172.21.15.53:0/67717373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500349d0 con 0x7fb244004d00 2023-10-21T16:14:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25ed1a640 1 -- 172.21.15.53:0/67717373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb244004d00 msgr2=0x7fb244005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/67717373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb244004d00 0x7fb244005180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb25000a780 tx=0x7fb2440057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25e519640 1 -- 172.21.15.53:0/67717373 reap_dead start 2023-10-21T16:14:41.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25ed1a640 1 -- 172.21.15.53:0/67717373 shutdown_connections 2023-10-21T16:14:41.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.888+0000 7fb25ed1a640 1 -- 172.21.15.53:0/67717373 >> 172.21.15.53:0/67717373 conn(0x7fb25809bc40 msgr2=0x7fb2580d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:41.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.889+0000 7fb25ed1a640 1 -- 172.21.15.53:0/67717373 shutdown_connections 2023-10-21T16:14:41.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.889+0000 7fb25ed1a640 1 -- 172.21.15.53:0/67717373 wait complete. 2023-10-21T16:14:41.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.890+0000 7fb25ed1a640 1 Processor -- start 2023-10-21T16:14:41.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.890+0000 7fb25ed1a640 1 -- start start 2023-10-21T16:14:41.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244004d00 0x7fb2581e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2581f2350 0x7fb2581f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25815c5f0 con 0x7fb258158870 2023-10-21T16:14:41.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb258159230 con 0x7fb244004d00 2023-10-21T16:14:41.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.891+0000 7fb25ed1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581f3080 con 0x7fb2581f2350 2023-10-21T16:14:41.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53902/0 (socket says 172.21.15.53:53902) 2023-10-21T16:14:41.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 -- 172.21.15.53:0/3310165642 learned_addr learned my addr 172.21.15.53:0/3310165642 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:41.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 -- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2581f2350 msgr2=0x7fb2581f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:41.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2581f2350 0x7fb2581f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 -- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244004d00 msgr2=0x7fb2581e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:41.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244004d00 0x7fb2581e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:41.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 -- 172.21.15.53:0/3310165642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25815c800 con 0x7fb258158870 2023-10-21T16:14:41.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.892+0000 7fb25d517640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb2480108e0 tx=0x7fb248010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:41.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.893+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24800eae0 con 0x7fb258158870 2023-10-21T16:14:41.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.893+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24800ec40 con 0x7fb258158870 2023-10-21T16:14:41.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.893+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24801c400 con 0x7fb258158870 2023-10-21T16:14:41.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.893+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25815cae0 con 0x7fb258158870 2023-10-21T16:14:41.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.893+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2581f2cf0 con 0x7fb258158870 2023-10-21T16:14:41.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.894+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb258074150 con 0x7fb258158870 2023-10-21T16:14:41.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.895+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb24801c560 con 0x7fb258158870 2023-10-21T16:14:41.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.896+0000 7fb24e7fc640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 0x7fb23c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:41.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.896+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6948+0+0 (secure 0 0 0) 0x7fb24808a560 con 0x7fb258158870 2023-10-21T16:14:41.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.897+0000 7fb25dd18640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 0x7fb23c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:41.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.898+0000 7fb25dd18640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 0x7fb23c06d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb254009890 tx=0x7fb254008040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:41.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:41.901+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb248054ec0 con 0x7fb258158870 2023-10-21T16:14:42.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.117+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2580cda30 con 0x7fb23c06b070 2023-10-21T16:14:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.350+0000 7fb24e7fc640 1 -- 172.21.15.53:0/3310165642 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb254009e70 con 0x7fb23c06b070 2023-10-21T16:14:42.352 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-21T16:14:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 msgr2=0x7fb23c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 0x7fb23c06d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb254009890 tx=0x7fb254008040 comp rx=0 tx=0).stop 2023-10-21T16:14:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 msgr2=0x7fb258158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb2480108e0 tx=0x7fb248010da0 comp rx=0 tx=0).stop 2023-10-21T16:14:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 shutdown_connections 2023-10-21T16:14:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fb23c06b070 0x7fb23c06d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2581f2350 0x7fb2581f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb258158870 0x7fb258158cf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 --2- 172.21.15.53:0/3310165642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb244004d00 0x7fb2581e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 >> 172.21.15.53:0/3310165642 conn(0x7fb25809bc40 msgr2=0x7fb25815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 shutdown_connections 2023-10-21T16:14:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:42.354+0000 7fb25ed1a640 1 -- 172.21.15.53:0/3310165642 wait complete. 2023-10-21T16:14:42.375 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-21T16:14:42.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.376 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-21T16:14:42.419 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:42.419 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:42.419 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:42.420 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:42.420 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:42.420 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:42.420 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:42.420 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:42.421 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:42.421 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:42.421 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:42.422 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:42.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.422 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:42.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:42.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.531 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:42.586 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:42.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.587 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:42.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:42.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.653 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_07_01_UTC) 2023-10-21T16:14:42.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:42.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.726 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_08_01_UTC) 2023-10-21T16:14:42.800 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:42.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.801 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:42.856 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:42.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.857 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:42.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:42.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.917 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_11_00_UTC) 2023-10-21T16:14:42.989 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:42.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:42.989 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:43.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:43.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:43.061 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:43.129 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:43.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:43.130 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-21-16_14_01_UTC) 2023-10-21T16:14:43.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:43.197 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-21T16:14:43.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2023-10-21T16:14:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.483+0000 7f3e935c9640 1 Processor -- start 2023-10-21T16:14:43.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.483+0000 7f3e935c9640 1 -- start start 2023-10-21T16:14:43.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e935c9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e8c0d05b0 0x7f3e8c0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e935c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 0x7f3e8c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e935c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c0cdde0 con 0x7f3e8c15adf0 2023-10-21T16:14:43.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e935c9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8c15b1d0 con 0x7f3e8c0d05b0 2023-10-21T16:14:43.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e935c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cf510 con 0x7f3e8c0cd410 2023-10-21T16:14:43.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e91365640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3e8c15adf0 0x7f3e8c0ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49000/0 (socket says 172.21.15.53:49000) 2023-10-21T16:14:43.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e91365640 1 -- 172.21.15.53:0/3673505778 learned_addr learned my addr 172.21.15.53:0/3673505778 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.485+0000 7f3e91b66640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 0x7f3e8c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e90b64640 1 --2- 172.21.15.53:0/3673505778 >> v2:172.21.15.124:3300/0 conn(0x7f3e8c0d05b0 0x7f3e8c0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346540912 0 0) 0x7f3e8c0cdde0 con 0x7f3e8c15adf0 2023-10-21T16:14:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e78003520 con 0x7f3e8c15adf0 2023-10-21T16:14:43.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3817100992 0 0) 0x7f3e78003520 con 0x7f3e8c15adf0 2023-10-21T16:14:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 msgr2=0x7f3e8c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 0x7f3e8c0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 >> v2:172.21.15.124:3300/0 conn(0x7f3e8c0d05b0 msgr2=0x7f3e8c0b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 --2- 172.21.15.53:0/3673505778 >> v2:172.21.15.124:3300/0 conn(0x7f3e8c0d05b0 0x7f3e8c0b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0cf7f0 con 0x7f3e8c15adf0 2023-10-21T16:14:43.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3e88001670 con 0x7f3e8c15adf0 2023-10-21T16:14:43.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 >> v1:172.21.15.53:6789/0 conn(0x7f3e8c15adf0 legacy=0x7f3e8c0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e78003970 0x7f3e78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e780047c0 0x7f3e78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 0x7f3e78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c15b1d0 con 0x7f3e78003970 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cf510 con 0x7f3e780047c0 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.486+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7800b980 con 0x7f3e8c0cd410 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e78003970 0x7f3e78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 -- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 msgr2=0x7f3e78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:43.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e8c0cd410 0x7f3e78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 -- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e780047c0 msgr2=0x7f3e78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:43.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e780047c0 0x7f3e78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.487+0000 7f3e91365640 1 -- 172.21.15.53:0/3673505778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78003520 con 0x7f3e78003970 2023-10-21T16:14:43.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.488+0000 7f3e91365640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e78003970 0x7f3e78003d90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800c160 tx=0x7f3e780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c8f85ceb3116a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:43.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.488+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88005ad0 con 0x7f3e78003970 2023-10-21T16:14:43.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.488+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88005c30 con 0x7f3e78003970 2023-10-21T16:14:43.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.488+0000 7f3e7ffff640 1 -- 172.21.15.53:0/3673505778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8802c810 con 0x7f3e78003970 2023-10-21T16:14:43.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 -- 172.21.15.53:0/3673505778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e78003970 msgr2=0x7f3e78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 --2- 172.21.15.53:0/3673505778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e78003970 0x7f3e78003d90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800c160 tx=0x7f3e780057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e91b66640 1 -- 172.21.15.53:0/3673505778 reap_dead start 2023-10-21T16:14:43.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 -- 172.21.15.53:0/3673505778 shutdown_connections 2023-10-21T16:14:43.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 -- 172.21.15.53:0/3673505778 >> 172.21.15.53:0/3673505778 conn(0x7f3e8c06e630 msgr2=0x7f3e8c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:43.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 -- 172.21.15.53:0/3673505778 shutdown_connections 2023-10-21T16:14:43.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.489+0000 7f3e935c9640 1 -- 172.21.15.53:0/3673505778 wait complete. 2023-10-21T16:14:43.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.491+0000 7f3e935c9640 1 Processor -- start 2023-10-21T16:14:43.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.491+0000 7f3e935c9640 1 -- start start 2023-10-21T16:14:43.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e78003970 0x7f3e8c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 0x7f3e8c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b31f0 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b0d30 con 0x7f3e8c0c6d60 2023-10-21T16:14:43.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.492+0000 7f3e935c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0c6880 con 0x7f3e78003970 2023-10-21T16:14:43.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53916/0 (socket says 172.21.15.53:53916) 2023-10-21T16:14:43.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 -- 172.21.15.53:0/1512237027 learned_addr learned my addr 172.21.15.53:0/1512237027 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:43.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91365640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e78003970 0x7f3e8c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e90b64640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 0x7f3e8c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 -- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e78003970 msgr2=0x7f3e8c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e78003970 0x7f3e8c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 -- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 msgr2=0x7f3e8c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 0x7f3e8c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 -- 172.21.15.53:0/1512237027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0b0f80 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e90b64640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 0x7f3e8c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:43.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.493+0000 7f3e91b66640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c06fa50 tx=0x7f3e8400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:43.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.494+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84010040 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.494+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8400bae0 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.494+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400be70 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.494+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0b1260 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.494+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c0b1820 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.495+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c0c6a80 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.496+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3e8401d400 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.497+0000 7f3e7dffb640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 0x7f3e7006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:43.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.497+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6900+0+0 (secure 0 0 0) 0x7f3e8408a620 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.498+0000 7f3e91365640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 0x7f3e7006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:43.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.499+0000 7f3e91365640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 0x7f3e7006d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e88001050 tx=0x7f3e88007000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:43.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.502+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e8c0c6a80 con 0x7f3e8c0c71e0 2023-10-21T16:14:43.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.723+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e8c0cfdb0 con 0x7f3e7006b070 2023-10-21T16:14:43.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.930+0000 7f3e7dffb640 1 -- 172.21.15.53:0/1512237027 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3e88001530 con 0x7f3e7006b070 2023-10-21T16:14:43.932 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-21T16:14:43.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.933+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 msgr2=0x7f3e7006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.933+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 0x7f3e7006d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e88001050 tx=0x7f3e88007000 comp rx=0 tx=0).stop 2023-10-21T16:14:43.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.933+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 msgr2=0x7f3e8c15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:43.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.933+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c06fa50 tx=0x7f3e8400b480 comp rx=0 tx=0).stop 2023-10-21T16:14:43.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 shutdown_connections 2023-10-21T16:14:43.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f3e7006b070 0x7f3e7006d530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e8c0c71e0 0x7f3e8c15c1e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e8c0c6d60 0x7f3e8c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 --2- 172.21.15.53:0/1512237027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e78003970 0x7f3e8c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 >> 172.21.15.53:0/1512237027 conn(0x7f3e8c06e630 msgr2=0x7f3e8c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 shutdown_connections 2023-10-21T16:14:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:43.934+0000 7f3e935c9640 1 -- 172.21.15.53:0/1512237027 wait complete. 2023-10-21T16:14:43.957 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-21T16:14:43.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:43.958 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-21T16:14:44.016 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:44.016 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:44.017 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:44.018 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:44.018 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:44.018 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:44.018 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:44.019 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_04_01_UTC 2023-10-21T16:14:44.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.020 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_04_01_UTC) 2023-10-21T16:14:44.086 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:44.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.087 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:44.158 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:44.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.159 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_06_01_UTC) 2023-10-21T16:14:44.221 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:44.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.222 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:44.290 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:44.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.290 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:44.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:44.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.374 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_09_01_UTC) 2023-10-21T16:14:44.432 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:44.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.433 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_10_01_UTC) 2023-10-21T16:14:44.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:44.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.482 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_11_01_UTC) 2023-10-21T16:14:44.536 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:44.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.537 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_12_01_UTC) 2023-10-21T16:14:44.594 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:44.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.595 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_13_01_UTC) 2023-10-21T16:14:44.662 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:44.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.663 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-21-16_14_01_UTC) 2023-10-21T16:14:44.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:44.732 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-21T16:14:44.783 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2023-10-21T16:14:45.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.028+0000 7fabbe86f640 1 Processor -- start 2023-10-21T16:14:45.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.028+0000 7fabbe86f640 1 -- start start 2023-10-21T16:14:45.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabbe86f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fabb806db60 0x7fabb80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:45.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabbe86f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb815b310 con 0x7fabb80c9cf0 2023-10-21T16:14:45.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabbe86f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb806d290 con 0x7fabb806c3a0 2023-10-21T16:14:45.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabbe86f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb815b110 con 0x7fabb806db60 2023-10-21T16:14:45.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabaffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fabb80c9cf0 0x7fabb815aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58122/0 (socket says 172.21.15.53:58122) 2023-10-21T16:14:45.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabaffff640 1 -- 172.21.15.53:0/3687489724 learned_addr learned my addr 172.21.15.53:0/3687489724 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:45.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.029+0000 7fabbce0c640 1 --2- 172.21.15.53:0/3687489724 >> v2:172.21.15.124:3300/0 conn(0x7fabb806db60 0x7fabb80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.030+0000 7fabbce0c640 1 -- 172.21.15.53:0/3687489724 >> v1:172.21.15.124:6789/0 conn(0x7fabb806c3a0 legacy=0x7fabb806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.030+0000 7fabbce0c640 1 -- 172.21.15.53:0/3687489724 >> v1:172.21.15.53:6789/0 conn(0x7fabb80c9cf0 legacy=0x7fabb815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.030+0000 7fabb77fe640 1 -- 172.21.15.53:0/3687489724 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474598442 0 0) 0x7faba0001110 con 0x7fabb80c9cf0 2023-10-21T16:14:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.030+0000 7fabbce0c640 1 -- 172.21.15.53:0/3687489724 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80c9860 con 0x7fabb806db60 2023-10-21T16:14:45.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.031+0000 7fabbce0c640 1 --2- 172.21.15.53:0/3687489724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806db60 0x7fabb80b2a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb002e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bb9c9d795ff4520 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.031+0000 7fabb77fe640 1 -- 172.21.15.53:0/3687489724 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0036440 con 0x7fabb806db60 2023-10-21T16:14:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabb77fe640 1 -- 172.21.15.53:0/3687489724 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb00365a0 con 0x7fabb806db60 2023-10-21T16:14:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabb77fe640 1 -- 172.21.15.53:0/3687489724 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0036930 con 0x7fabb806db60 2023-10-21T16:14:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 -- 172.21.15.53:0/3687489724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806db60 msgr2=0x7fabb80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3687489724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806db60 0x7fabb80b2a90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb002e160 comp rx=0 tx=0).stop 2023-10-21T16:14:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 -- 172.21.15.53:0/3687489724 shutdown_connections 2023-10-21T16:14:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3687489724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806db60 0x7fabb80b2a90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 -- 172.21.15.53:0/3687489724 >> 172.21.15.53:0/3687489724 conn(0x7fabb806a5d0 msgr2=0x7fabb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 -- 172.21.15.53:0/3687489724 shutdown_connections 2023-10-21T16:14:45.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.032+0000 7fabbe86f640 1 -- 172.21.15.53:0/3687489724 wait complete. 2023-10-21T16:14:45.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.034+0000 7fabbe86f640 1 Processor -- start 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.034+0000 7fabbe86f640 1 -- start start 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 0x7fabb815ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabb806db60 0x7fabb80c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c77b0 con 0x7fabb80c9cf0 2023-10-21T16:14:45.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c79e0 con 0x7fabb806c3a0 2023-10-21T16:14:45.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbe86f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80c2310 con 0x7fabb806db60 2023-10-21T16:14:45.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbce0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:45.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbce0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41546/0 (socket says 172.21.15.53:41546) 2023-10-21T16:14:45.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabbce0c640 1 -- 172.21.15.53:0/3692184792 learned_addr learned my addr 172.21.15.53:0/3692184792 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:45.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabb7fff640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 0x7fabb815ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.035+0000 7fabaffff640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabb806db60 0x7fabb80c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 -- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabb806db60 msgr2=0x7fabb80c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabb806db60 0x7fabb80c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 -- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 msgr2=0x7fabb815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 0x7fabb815ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 -- 172.21.15.53:0/3692184792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009a70 con 0x7fabb80c9cf0 2023-10-21T16:14:45.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabb7fff640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 0x7fabb815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:14:45.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabbce0c640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:45.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.036+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb003f830 con 0x7fabb80c9cf0 2023-10-21T16:14:45.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.037+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb003f9b0 con 0x7fabb80c9cf0 2023-10-21T16:14:45.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.037+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb003fd60 con 0x7fabb80c9cf0 2023-10-21T16:14:45.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.037+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb80c25f0 con 0x7fabb80c9cf0 2023-10-21T16:14:45.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.037+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb80c2bb0 con 0x7fabb80c9cf0 2023-10-21T16:14:45.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.038+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb804e160 con 0x7fabb80c9cf0 2023-10-21T16:14:45.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.039+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fabb003d070 con 0x7fabb80c9cf0 2023-10-21T16:14:45.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.040+0000 7fabb57fa640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 0x7fab9806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:45.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.040+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7028+0+0 (secure 0 0 0) 0x7fabb00acd80 con 0x7fabb80c9cf0 2023-10-21T16:14:45.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.041+0000 7fabb7fff640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 0x7fab9806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:45.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.041+0000 7fabb7fff640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 0x7fab9806d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabb80b01b0 tx=0x7faba8009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:45.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.045+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabb0077720 con 0x7fabb80c9cf0 2023-10-21T16:14:45.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.259+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fabb806dff0 con 0x7fab9806b070 2023-10-21T16:14:45.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.430+0000 7fabb57fa640 1 -- 172.21.15.53:0/3692184792 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7faba80018b0 con 0x7fab9806b070 2023-10-21T16:14:45.432 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-21T16:14:45.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 msgr2=0x7fab9806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 0x7fab9806d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabb80b01b0 tx=0x7faba8009040 comp rx=0 tx=0).stop 2023-10-21T16:14:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 msgr2=0x7fabb814daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009d90 tx=0x7fabb0004a40 comp rx=0 tx=0).stop 2023-10-21T16:14:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 shutdown_connections 2023-10-21T16:14:45.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fab9806b070 0x7fab9806d530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabb80c9cf0 0x7fabb814daa0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabb806db60 0x7fabb80c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 --2- 172.21.15.53:0/3692184792 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabb806c3a0 0x7fabb815ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 >> 172.21.15.53:0/3692184792 conn(0x7fabb806a5d0 msgr2=0x7fabb80b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:45.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 shutdown_connections 2023-10-21T16:14:45.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:45.433+0000 7fabbe86f640 1 -- 172.21.15.53:0/3692184792 wait complete. 2023-10-21T16:14:45.453 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-21T16:14:45.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.453 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-21T16:14:45.500 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:45.500 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:45.501 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:45.501 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:45.501 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:45.501 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:45.502 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:45.502 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:45.502 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:45.502 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:45.503 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:45.503 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:45.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.504 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:45.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:45.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.618 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_05_01_UTC) 2023-10-21T16:14:45.694 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_06_01_UTC 2023-10-21T16:14:45.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.694 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_06_01_UTC) 2023-10-21T16:14:45.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_07_01_UTC 2023-10-21T16:14:45.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.768 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_07_01_UTC) 2023-10-21T16:14:45.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:45.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.841 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_08_01_UTC) 2023-10-21T16:14:45.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:45.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.919 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:45.993 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:45.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:45.994 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:46.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:46.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:46.067 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_11_01_UTC) 2023-10-21T16:14:46.139 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:46.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:46.140 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_12_01_UTC) 2023-10-21T16:14:46.211 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:46.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:46.212 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:46.291 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:46.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:46.292 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:46.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:46.346 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-21T16:14:46.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2023-10-21T16:14:46.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.648+0000 7f803967c640 1 Processor -- start 2023-10-21T16:14:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.648+0000 7f803967c640 1 -- start start 2023-10-21T16:14:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.649+0000 7f803967c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8034159b40 0x7f80340cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.649+0000 7f803967c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80340aec80 0x7f80340cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f803967c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cb230 con 0x7f80340cc7e0 2023-10-21T16:14:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f803967c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340ccc90 con 0x7f80340aec80 2023-10-21T16:14:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f803967c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80340cf610 con 0x7f8034159b40 2023-10-21T16:14:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80340aec80 0x7f80340cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80340aec80 0x7f80340cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41548/0 (socket says 172.21.15.53:41548) 2023-10-21T16:14:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 -- 172.21.15.53:0/458292434 learned_addr learned my addr 172.21.15.53:0/458292434 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80337fe640 1 --1- 172.21.15.53:0/458292434 >> v1:172.21.15.53:6789/0 conn(0x7f80340cc7e0 0x7f80340b1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58134/0 (socket says 172.21.15.53:58134) 2023-10-21T16:14:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f8032ffd640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8034159b40 0x7f80340cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 -- 172.21.15.53:0/458292434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8034159b40 msgr2=0x7f80340cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8034159b40 0x7f80340cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.650+0000 7f80327fc640 1 -- 172.21.15.53:0/458292434 >> v1:172.21.15.53:6789/0 conn(0x7f80340cc7e0 legacy=0x7f80340b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.651+0000 7f80327fc640 1 -- 172.21.15.53:0/458292434 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340ce410 con 0x7f80340aec80 2023-10-21T16:14:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.652+0000 7f80327fc640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80340cc2a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f801c009870 tx=0x7f801c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f20276d7379d653 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.652+0000 7f8031ffb640 1 -- 172.21.15.53:0/458292434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c030840 con 0x7f80340aec80 2023-10-21T16:14:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.652+0000 7f8031ffb640 1 -- 172.21.15.53:0/458292434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c0309a0 con 0x7f80340aec80 2023-10-21T16:14:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.652+0000 7f8031ffb640 1 -- 172.21.15.53:0/458292434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c030d30 con 0x7f80340aec80 2023-10-21T16:14:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 -- 172.21.15.53:0/458292434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 msgr2=0x7f80340cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:46.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80340cc2a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f801c009870 tx=0x7f801c02b980 comp rx=0 tx=0).stop 2023-10-21T16:14:46.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 -- 172.21.15.53:0/458292434 shutdown_connections 2023-10-21T16:14:46.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80340cc2a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:46.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 --2- 172.21.15.53:0/458292434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8034159b40 0x7f80340cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:46.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 -- 172.21.15.53:0/458292434 >> 172.21.15.53:0/458292434 conn(0x7f803406b250 msgr2=0x7f803406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:46.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 -- 172.21.15.53:0/458292434 shutdown_connections 2023-10-21T16:14:46.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.653+0000 7f803967c640 1 -- 172.21.15.53:0/458292434 wait complete. 2023-10-21T16:14:46.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.655+0000 7f803967c640 1 Processor -- start 2023-10-21T16:14:46.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.655+0000 7f803967c640 1 -- start start 2023-10-21T16:14:46.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80340cc7e0 0x7f80341541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8034159b40 0x7f80340c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341584e0 con 0x7f80340aec80 2023-10-21T16:14:46.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803415c8d0 con 0x7f8034159b40 2023-10-21T16:14:46.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f803967c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803415cbb0 con 0x7f80340cc7e0 2023-10-21T16:14:46.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f8032ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:46.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f8032ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41558/0 (socket says 172.21.15.53:41558) 2023-10-21T16:14:46.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.656+0000 7f8032ffd640 1 -- 172.21.15.53:0/3069284171 learned_addr learned my addr 172.21.15.53:0/3069284171 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:46.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 -- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80340cc7e0 msgr2=0x7f80341541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:46.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80340cc7e0 0x7f80341541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:46.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 -- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8034159b40 msgr2=0x7f80340c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:46.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8034159b40 0x7f80340c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:46.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 -- 172.21.15.53:0/3069284171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0094d0 con 0x7f80340aec80 2023-10-21T16:14:46.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f8032ffd640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8024007e40 tx=0x7f802400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:46.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024014620 con 0x7f80340aec80 2023-10-21T16:14:46.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400ee90 con 0x7f80340aec80 2023-10-21T16:14:46.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240149c0 con 0x7f80340aec80 2023-10-21T16:14:46.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.657+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80341546e0 con 0x7f80340aec80 2023-10-21T16:14:46.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.658+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8034155bf0 con 0x7f80340aec80 2023-10-21T16:14:46.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.659+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80340ce8e0 con 0x7f80340aec80 2023-10-21T16:14:46.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.660+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f8024014b20 con 0x7f80340aec80 2023-10-21T16:14:46.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.660+0000 7f802b7fe640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 0x7f801406d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:46.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.661+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7280+0+0 (secure 0 0 0) 0x7f802408a940 con 0x7f80340aec80 2023-10-21T16:14:46.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.661+0000 7f80327fc640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 0x7f801406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:46.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.662+0000 7f80327fc640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 0x7f801406d580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f801c036000 tx=0x7f801c03b040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:46.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.666+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8024055150 con 0x7f80340aec80 2023-10-21T16:14:46.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:46.907+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f803415ca40 con 0x7f801406b0c0 2023-10-21T16:14:47.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.114+0000 7f802b7fe640 1 -- 172.21.15.53:0/3069284171 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f803415ca40 con 0x7f801406b0c0 2023-10-21T16:14:47.116 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-21T16:14:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 msgr2=0x7f801406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 0x7f801406d580 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f801c036000 tx=0x7f801c03b040 comp rx=0 tx=0).stop 2023-10-21T16:14:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 msgr2=0x7f80341549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8024007e40 tx=0x7f802400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:14:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 shutdown_connections 2023-10-21T16:14:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f801406b0c0 0x7f801406d580 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8034159b40 0x7f80340c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80340cc7e0 0x7f80341541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 --2- 172.21.15.53:0/3069284171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80340aec80 0x7f80341549d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 >> 172.21.15.53:0/3069284171 conn(0x7f803406b250 msgr2=0x7f803406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:47.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 shutdown_connections 2023-10-21T16:14:47.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:47.118+0000 7f803967c640 1 -- 172.21.15.53:0/3069284171 wait complete. 2023-10-21T16:14:47.139 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-21T16:14:47.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.140 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-21T16:14:47.197 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:47.197 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:47.197 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:47.197 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:47.198 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:47.198 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:47.198 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:47.198 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:47.198 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:47.199 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:47.199 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:47.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:47.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.201 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:47.293 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:47.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.294 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:47.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:47.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.349 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:47.397 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:47.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.398 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:47.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:47.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.460 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:47.516 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:47.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.517 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:47.567 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:47.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.568 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_10_01_UTC) 2023-10-21T16:14:47.616 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:47.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.617 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_11_01_UTC) 2023-10-21T16:14:47.678 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_12_01_UTC 2023-10-21T16:14:47.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.679 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_12_01_UTC) 2023-10-21T16:14:47.732 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:47.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.734 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_13_01_UTC) 2023-10-21T16:14:47.790 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.791 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:47.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:47.854 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-21T16:14:47.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2023-10-21T16:14:48.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.143+0000 7fefdd0da640 1 Processor -- start 2023-10-21T16:14:48.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.143+0000 7fefdd0da640 1 -- start start 2023-10-21T16:14:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefdd0da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fefd80b4e10 0x7fefd81612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefdd0da640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 0x7fefd80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefdd0da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd80b8c30 con 0x7fefd815cd70 2023-10-21T16:14:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefdd0da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80cdd80 con 0x7fefd80b4e10 2023-10-21T16:14:48.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefdd0da640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd80ce060 con 0x7fefd80cd4e0 2023-10-21T16:14:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefd6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fefd815cd70 0x7fefd80b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58144/0 (socket says 172.21.15.53:58144) 2023-10-21T16:14:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefd6d9d640 1 -- 172.21.15.53:0/1719323828 learned_addr learned my addr 172.21.15.53:0/1719323828 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:48.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.144+0000 7fefce59c640 1 --2- 172.21.15.53:0/1719323828 >> v2:172.21.15.53:3300/0 conn(0x7fefd80b4e10 0x7fefd81612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefd759e640 1 --2- 172.21.15.53:0/1719323828 >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 0x7fefd80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefce59c640 1 -- 172.21.15.53:0/1719323828 >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 msgr2=0x7fefd80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefd659c640 1 -- 172.21.15.53:0/1719323828 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385532482 0 0) 0x7fefd80b8c30 con 0x7fefd815cd70 2023-10-21T16:14:48.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefce59c640 1 --2- 172.21.15.53:0/1719323828 >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 0x7fefd80d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefce59c640 1 -- 172.21.15.53:0/1719323828 >> v1:172.21.15.53:6789/0 conn(0x7fefd815cd70 legacy=0x7fefd80b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefce59c640 1 -- 172.21.15.53:0/1719323828 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80ce340 con 0x7fefd80b4e10 2023-10-21T16:14:48.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.145+0000 7fefd759e640 1 --2- 172.21.15.53:0/1719323828 >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 0x7fefd80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:48.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.146+0000 7fefce59c640 1 --2- 172.21.15.53:0/1719323828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd81612f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fefd80d06c0 tx=0x7fefc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ff6e2079323c47 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:48.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.146+0000 7fefd659c640 1 -- 172.21.15.53:0/1719323828 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031830 con 0x7fefd80b4e10 2023-10-21T16:14:48.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.146+0000 7fefd659c640 1 -- 172.21.15.53:0/1719323828 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc0031990 con 0x7fefd80b4e10 2023-10-21T16:14:48.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.146+0000 7fefd659c640 1 -- 172.21.15.53:0/1719323828 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0031d20 con 0x7fefd80b4e10 2023-10-21T16:14:48.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 -- 172.21.15.53:0/1719323828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 msgr2=0x7fefd81612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1719323828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd81612f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fefd80d06c0 tx=0x7fefc002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 -- 172.21.15.53:0/1719323828 shutdown_connections 2023-10-21T16:14:48.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1719323828 >> v2:172.21.15.124:3300/0 conn(0x7fefd80cd4e0 0x7fefd80d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1719323828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd81612f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 -- 172.21.15.53:0/1719323828 >> 172.21.15.53:0/1719323828 conn(0x7fefd80cf700 msgr2=0x7fefd80cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:48.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 -- 172.21.15.53:0/1719323828 shutdown_connections 2023-10-21T16:14:48.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.147+0000 7fefdd0da640 1 -- 172.21.15.53:0/1719323828 wait complete. 2023-10-21T16:14:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.149+0000 7fefdd0da640 1 Processor -- start 2023-10-21T16:14:48.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.149+0000 7fefdd0da640 1 -- start start 2023-10-21T16:14:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd80cd4e0 0x7fefd81e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd815cd70 0x7fefd81e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80cb300 con 0x7fefd80b4e10 2023-10-21T16:14:48.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80c79d0 con 0x7fefd815cd70 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefdd0da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80c7cb0 con 0x7fefd80cd4e0 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefd6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefd6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41574/0 (socket says 172.21.15.53:41574) 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.150+0000 7fefd6d9d640 1 -- 172.21.15.53:0/1555423773 learned_addr learned my addr 172.21.15.53:0/1555423773 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 -- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd80cd4e0 msgr2=0x7fefd81e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd80cd4e0 0x7fefd81e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 -- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd815cd70 msgr2=0x7fefd81e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd815cd70 0x7fefd81e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 -- 172.21.15.53:0/1555423773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc0009400 con 0x7fefd80b4e10 2023-10-21T16:14:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefd6d9d640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c4d0 tx=0x7fefc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:48.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80074e0 con 0x7fefd80b4e10 2023-10-21T16:14:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc8007660 con 0x7fefd80b4e10 2023-10-21T16:14:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd80c7f90 con 0x7fefd80b4e10 2023-10-21T16:14:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004a20 con 0x7fefd80b4e10 2023-10-21T16:14:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.151+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd80cae90 con 0x7fefd80b4e10 2023-10-21T16:14:48.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.153+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd804e160 con 0x7fefd80b4e10 2023-10-21T16:14:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.154+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fefc8004b80 con 0x7fefd80b4e10 2023-10-21T16:14:48.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.154+0000 7fefcffff640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 0x7fefb806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:48.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.155+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6996+0+0 (secure 0 0 0) 0x7fefc8089140 con 0x7fefd80b4e10 2023-10-21T16:14:48.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.155+0000 7fefce59c640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 0x7fefb806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:48.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.156+0000 7fefce59c640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 0x7fefb806d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc0036000 tx=0x7fefc0030000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:48.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.159+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefc80539c0 con 0x7fefd80b4e10 2023-10-21T16:14:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.428+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fefd80cb170 con 0x7fefb806b070 2023-10-21T16:14:48.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.578+0000 7fefcffff640 1 -- 172.21.15.53:0/1555423773 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fefd80cb170 con 0x7fefb806b070 2023-10-21T16:14:48.580 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-21T16:14:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.581+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 msgr2=0x7fefb806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.581+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 0x7fefb806d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc0036000 tx=0x7fefc0030000 comp rx=0 tx=0).stop 2023-10-21T16:14:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.581+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 msgr2=0x7fefd80c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.581+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fefc800c4d0 tx=0x7fefc800c990 comp rx=0 tx=0).stop 2023-10-21T16:14:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.581+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 shutdown_connections 2023-10-21T16:14:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fefb806b070 0x7fefb806d530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefd815cd70 0x7fefd81e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefd80cd4e0 0x7fefd81e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 --2- 172.21.15.53:0/1555423773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefd80b4e10 0x7fefd80c7160 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 >> 172.21.15.53:0/1555423773 conn(0x7fefd80cf700 msgr2=0x7fefd80d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 shutdown_connections 2023-10-21T16:14:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:48.582+0000 7fefdd0da640 1 -- 172.21.15.53:0/1555423773 wait complete. 2023-10-21T16:14:48.603 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-21T16:14:48.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.604 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-21T16:14:48.659 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:48.659 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:48.660 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:48.660 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:48.660 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:48.660 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:48.660 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:48.661 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:48.661 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:48.661 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:48.661 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:48.662 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:48.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.663 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:48.732 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_05_00_UTC 2023-10-21T16:14:48.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.733 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_05_00_UTC) 2023-10-21T16:14:48.808 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:48.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.809 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:48.890 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:48.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.891 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:48.970 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_08_00_UTC 2023-10-21T16:14:48.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:48.971 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_08_00_UTC) 2023-10-21T16:14:49.051 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_09_00_UTC 2023-10-21T16:14:49.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.052 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_09_00_UTC) 2023-10-21T16:14:49.133 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_10_00_UTC 2023-10-21T16:14:49.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.134 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_10_00_UTC) 2023-10-21T16:14:49.209 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_11_00_UTC 2023-10-21T16:14:49.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.210 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_11_00_UTC) 2023-10-21T16:14:49.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:49.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.318 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:49.381 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_13_00_UTC 2023-10-21T16:14:49.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.382 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_13_00_UTC) 2023-10-21T16:14:49.439 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_14_00_UTC 2023-10-21T16:14:49.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.440 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-21-16_14_00_UTC) 2023-10-21T16:14:49.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:49.495 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-21T16:14:49.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2023-10-21T16:14:49.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.772+0000 7f61cb11d640 1 Processor -- start 2023-10-21T16:14:49.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.773+0000 7f61cb11d640 1 -- start start 2023-10-21T16:14:49.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61cb11d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61c40afbe0 0x7f61c40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61cb11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61c40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61cb11d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40cbad0 con 0x7f61c406dfc0 2023-10-21T16:14:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61cb11d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40b2620 con 0x7f61c40afbe0 2023-10-21T16:14:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61cb11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b2060 con 0x7f61c40cb130 2023-10-21T16:14:49.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61c3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f61c406dfc0 0x7f61c4070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58148/0 (socket says 172.21.15.53:58148) 2023-10-21T16:14:49.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.774+0000 7f61c3fff640 1 -- 172.21.15.53:0/2034175444 learned_addr learned my addr 172.21.15.53:0/2034175444 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:49.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/2034175444 >> v2:172.21.15.124:3300/0 conn(0x7f61c40afbe0 0x7f61c40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c96ba640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61c40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536229824 0 0) 0x7f61c40cbad0 con 0x7f61c406dfc0 2023-10-21T16:14:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b0003520 con 0x7f61c406dfc0 2023-10-21T16:14:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923460264 0 0) 0x7f61b0003520 con 0x7f61c406dfc0 2023-10-21T16:14:49.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 msgr2=0x7f61c40cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61c40cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 >> v2:172.21.15.124:3300/0 conn(0x7f61c40afbe0 msgr2=0x7f61c40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 --2- 172.21.15.53:0/2034175444 >> v2:172.21.15.124:3300/0 conn(0x7f61c40afbe0 0x7f61c40b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40b2340 con 0x7f61c406dfc0 2023-10-21T16:14:49.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f61ac001670 con 0x7f61c406dfc0 2023-10-21T16:14:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 >> v1:172.21.15.53:6789/0 conn(0x7f61c406dfc0 legacy=0x7f61c4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0003970 0x7f61b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b0004d00 0x7f61b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b2620 con 0x7f61b0003970 2023-10-21T16:14:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b2060 con 0x7f61b0004d00 2023-10-21T16:14:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.775+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b000bae0 con 0x7f61c40cb130 2023-10-21T16:14:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0003970 0x7f61b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c3fff640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 -- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 msgr2=0x7f61b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61c40cb130 0x7f61b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 -- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b0004d00 msgr2=0x7f61b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c96ba640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b0004d00 0x7f61b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61b0004d00 0x7f61b0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.776+0000 7f61c8eb9640 1 -- 172.21.15.53:0/2034175444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0003520 con 0x7f61b0003970 2023-10-21T16:14:49.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.777+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0003970 0x7f61b0003d90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007360 tx=0x7f61b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6538e44b5f7db48 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:49.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b402ccf0 con 0x7f61b0003970 2023-10-21T16:14:49.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b402ce50 con 0x7f61b0003970 2023-10-21T16:14:49.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61c37fe640 1 -- 172.21.15.53:0/2034175444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4034660 con 0x7f61b0003970 2023-10-21T16:14:49.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61cb11d640 1 -- 172.21.15.53:0/2034175444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0003970 msgr2=0x7f61b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:49.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61cb11d640 1 --2- 172.21.15.53:0/2034175444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61b0003970 0x7f61b0003d90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007360 tx=0x7f61b00057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61c96ba640 1 -- 172.21.15.53:0/2034175444 reap_dead start 2023-10-21T16:14:49.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61cb11d640 1 -- 172.21.15.53:0/2034175444 shutdown_connections 2023-10-21T16:14:49.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.778+0000 7f61cb11d640 1 -- 172.21.15.53:0/2034175444 >> 172.21.15.53:0/2034175444 conn(0x7f61c406a5d0 msgr2=0x7f61c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:49.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.779+0000 7f61cb11d640 1 -- 172.21.15.53:0/2034175444 shutdown_connections 2023-10-21T16:14:49.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.779+0000 7f61cb11d640 1 -- 172.21.15.53:0/2034175444 wait complete. 2023-10-21T16:14:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.780+0000 7f61cb11d640 1 Processor -- start 2023-10-21T16:14:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.781+0000 7f61cb11d640 1 -- start start 2023-10-21T16:14:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.781+0000 7f61cb11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b0003970 0x7f61c40ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.781+0000 7f61cb11d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61cb11d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61c4147380 0x7f61c4147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61cb11d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c3f30 con 0x7f61c4149a50 2023-10-21T16:14:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61cb11d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415be80 con 0x7f61c4147380 2023-10-21T16:14:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61cb11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415c160 con 0x7f61b0003970 2023-10-21T16:14:49.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41596/0 (socket says 172.21.15.53:41596) 2023-10-21T16:14:49.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 -- 172.21.15.53:0/1069508724 learned_addr learned my addr 172.21.15.53:0/1069508724 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:49.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 -- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b0003970 msgr2=0x7f61c40ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:49.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b0003970 0x7f61c40ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 -- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61c4147380 msgr2=0x7f61c4147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:49.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61c4147380 0x7f61c4147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:49.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.782+0000 7f61c3fff640 1 -- 172.21.15.53:0/1069508724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c415ce20 con 0x7f61c4149a50 2023-10-21T16:14:49.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61c3fff640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e780 tx=0x7f61ac00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:49.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00ee30 con 0x7f61c4149a50 2023-10-21T16:14:49.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac00c8d0 con 0x7f61c4149a50 2023-10-21T16:14:49.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00cc80 con 0x7f61c4149a50 2023-10-21T16:14:49.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c415d100 con 0x7f61c4149a50 2023-10-21T16:14:49.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.783+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40cac80 con 0x7f61c4149a50 2023-10-21T16:14:49.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.784+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c415cc80 con 0x7f61c4149a50 2023-10-21T16:14:49.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.785+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f61ac01d460 con 0x7f61c4149a50 2023-10-21T16:14:49.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.786+0000 7f61c17fa640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 0x7f61a406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:49.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.786+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6964+0+0 (secure 0 0 0) 0x7f61ac089c40 con 0x7f61c4149a50 2023-10-21T16:14:49.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.787+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 0x7f61a406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:49.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.788+0000 7f61c8eb9640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 0x7f61a406d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4036000 tx=0x7f61b402e000 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:49.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:49.790+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61c415cc80 con 0x7f61c4149a50 2023-10-21T16:14:50.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.010+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 --> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c4070630 con 0x7f61a406b070 2023-10-21T16:14:50.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:14:50.052 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:14:50.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.147+0000 7f61c17fa640 1 -- 172.21.15.53:0/1069508724 <== mgr.5143 v2:172.21.15.124:6832/637025846 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f61b4003990 con 0x7f61a406b070 2023-10-21T16:14:50.149 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-21T16:14:50.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.149+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 msgr2=0x7f61a406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.149+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 0x7f61a406d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61b4036000 tx=0x7f61b402e000 comp rx=0 tx=0).stop 2023-10-21T16:14:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 msgr2=0x7f61c415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00e780 tx=0x7f61ac00ec40 comp rx=0 tx=0).stop 2023-10-21T16:14:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 shutdown_connections 2023-10-21T16:14:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f61a406b070 0x7f61a406d530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61c4147380 0x7f61c4147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61c4149a50 0x7f61c415b510 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 --2- 172.21.15.53:0/1069508724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61b0003970 0x7f61c40ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 >> 172.21.15.53:0/1069508724 conn(0x7f61c406a5d0 msgr2=0x7f61c40cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 shutdown_connections 2023-10-21T16:14:50.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:50.150+0000 7f61cb11d640 1 -- 172.21.15.53:0/1069508724 wait complete. 2023-10-21T16:14:50.171 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-21T16:14:50.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.172 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-21T16:14:50.287 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:50.287 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:50.287 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:50.287 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:50.288 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:50.288 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:50.288 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:50.288 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:50.289 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:50.289 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:50.289 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:50.290 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_04_00_UTC 2023-10-21T16:14:50.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.291 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_04_00_UTC) 2023-10-21T16:14:50.369 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_05_01_UTC 2023-10-21T16:14:50.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.369 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_05_01_UTC) 2023-10-21T16:14:50.426 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_06_00_UTC 2023-10-21T16:14:50.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.427 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_06_00_UTC) 2023-10-21T16:14:50.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_07_00_UTC 2023-10-21T16:14:50.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.483 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_07_00_UTC) 2023-10-21T16:14:50.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_08_01_UTC 2023-10-21T16:14:50.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.542 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_08_01_UTC) 2023-10-21T16:14:50.619 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_09_01_UTC 2023-10-21T16:14:50.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.620 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_09_01_UTC) 2023-10-21T16:14:50.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_10_01_UTC 2023-10-21T16:14:50.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.683 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_10_01_UTC) 2023-10-21T16:14:50.737 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_11_01_UTC 2023-10-21T16:14:50.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.738 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_11_01_UTC) 2023-10-21T16:14:50.797 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_12_00_UTC 2023-10-21T16:14:50.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.799 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_12_00_UTC) 2023-10-21T16:14:50.862 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_13_01_UTC 2023-10-21T16:14:50.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.862 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_13_01_UTC) 2023-10-21T16:14:50.926 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_14_01_UTC 2023-10-21T16:14:50.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.927 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-21-16_14_01_UTC) 2023-10-21T16:14:50.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:50.992 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-21T16:14:51.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:14:51.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.249+0000 7f473fee0640 1 Processor -- start 2023-10-21T16:14:51.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.249+0000 7f473fee0640 1 -- start start 2023-10-21T16:14:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473fee0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f47380b21f0 0x7f47380c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473fee0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473806c3a0 con 0x7f47380ca1a0 2023-10-21T16:14:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473fee0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473806c640 con 0x7f47380702b0 2023-10-21T16:14:51.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473fee0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473806c920 con 0x7f47380b21f0 2023-10-21T16:14:51.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473e47d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f47380ca1a0 0x7f473806d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58164/0 (socket says 172.21.15.53:58164) 2023-10-21T16:14:51.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.251+0000 7f473e47d640 1 -- 172.21.15.53:0/3914299199 learned_addr learned my addr 172.21.15.53:0/3914299199 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:51.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473d47b640 1 --2- 172.21.15.53:0/3914299199 >> v2:172.21.15.124:3300/0 conn(0x7f47380b21f0 0x7f47380c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127769768 0 0) 0x7f473806c3a0 con 0x7f47380ca1a0 2023-10-21T16:14:51.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4724003520 con 0x7f47380ca1a0 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093585158 0 0) 0x7f4724003520 con 0x7f47380ca1a0 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 >> v2:172.21.15.124:3300/0 conn(0x7f47380b21f0 msgr2=0x7f47380c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 --2- 172.21.15.53:0/3914299199 >> v2:172.21.15.124:3300/0 conn(0x7f47380b21f0 0x7f47380c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 >> v1:172.21.15.124:6789/0 conn(0x7f47380702b0 legacy=0x7f4738070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473806cc00 con 0x7f47380ca1a0 2023-10-21T16:14:51.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.252+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f47300019f0 con 0x7f47380ca1a0 2023-10-21T16:14:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 >> v1:172.21.15.53:6789/0 conn(0x7f47380ca1a0 legacy=0x7f473806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4724003970 0x7f4724003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 0x7f4724004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724005040 0x7f472400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473806c920 con 0x7f4724004370 2023-10-21T16:14:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472400bfa0 con 0x7f4724003970 2023-10-21T16:14:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472400c160 con 0x7f4724005040 2023-10-21T16:14:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 0x7f4724004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473dc7c640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4724003970 0x7f4724003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 -- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724005040 msgr2=0x7f472400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:51.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724005040 0x7f472400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 -- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4724003970 msgr2=0x7f4724003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:51.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4724003970 0x7f4724003d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.253+0000 7f473d47b640 1 -- 172.21.15.53:0/3914299199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724003520 con 0x7f4724004370 2023-10-21T16:14:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.254+0000 7f473e47d640 1 -- 172.21.15.53:0/3914299199 reap_dead start 2023-10-21T16:14:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.255+0000 7f473d47b640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 0x7f4724004840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f472800aac0 tx=0x7f47240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81563966024e19b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:51.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.255+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728035440 con 0x7f4724004370 2023-10-21T16:14:51.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.255+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47280355a0 con 0x7f4724004370 2023-10-21T16:14:51.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.255+0000 7f473cc7a640 1 -- 172.21.15.53:0/3914299199 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728035930 con 0x7f4724004370 2023-10-21T16:14:51.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 -- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 msgr2=0x7f4724004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:51.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 0x7f4724004840 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f472800aac0 tx=0x7f47240057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 -- 172.21.15.53:0/3914299199 shutdown_connections 2023-10-21T16:14:51.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 --2- 172.21.15.53:0/3914299199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4724004370 0x7f4724004840 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 -- 172.21.15.53:0/3914299199 >> 172.21.15.53:0/3914299199 conn(0x7f473806a5d0 msgr2=0x7f47380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 -- 172.21.15.53:0/3914299199 shutdown_connections 2023-10-21T16:14:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.256+0000 7f473fee0640 1 -- 172.21.15.53:0/3914299199 wait complete. 2023-10-21T16:14:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.258+0000 7f473fee0640 1 Processor -- start 2023-10-21T16:14:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.258+0000 7f473fee0640 1 -- start start 2023-10-21T16:14:51.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724004370 0x7f4738155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47380ac630 0x7f47380cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47381f2360 con 0x7f4738156800 2023-10-21T16:14:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380acb80 con 0x7f47380ac630 2023-10-21T16:14:51.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473fee0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380ace60 con 0x7f4724004370 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473e47d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473e47d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41622/0 (socket says 172.21.15.53:41622) 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473e47d640 1 -- 172.21.15.53:0/346473982 learned_addr learned my addr 172.21.15.53:0/346473982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.259+0000 7f473dc7c640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724004370 0x7f4738155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473d47b640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47380ac630 0x7f47380cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 -- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724004370 msgr2=0x7f4738155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:51.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724004370 0x7f4738155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 -- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47380ac630 msgr2=0x7f47380cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:51.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47380ac630 0x7f47380cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:51.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 -- 172.21.15.53:0/346473982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380ad140 con 0x7f4738156800 2023-10-21T16:14:51.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f473e47d640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4738071370 tx=0x7f473000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:51.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.260+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000aa80 con 0x7f4738156800 2023-10-21T16:14:51.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.261+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473000ac00 con 0x7f4738156800 2023-10-21T16:14:51.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.261+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730010300 con 0x7f4738156800 2023-10-21T16:14:51.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.261+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380c61d0 con 0x7f4738156800 2023-10-21T16:14:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.261+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380af540 con 0x7f4738156800 2023-10-21T16:14:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.262+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473804e160 con 0x7f4738156800 2023-10-21T16:14:51.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.264+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f4730010460 con 0x7f4738156800 2023-10-21T16:14:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.265+0000 7f472e7fc640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 0x7f471c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:51.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.265+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7312+0+0 (secure 0 0 0) 0x7f47300891d0 con 0x7f4738156800 2023-10-21T16:14:51.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.266+0000 7f473dc7c640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 0x7f471c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:51.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.267+0000 7f473dc7c640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 0x7f471c06d560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4734001510 tx=0x7f4734009040 comp rx=0 tx=0).ready entity=mgr.5143 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:51.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.270+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4730053a50 con 0x7f4738156800 2023-10-21T16:14:51.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:51.516+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f47380cca80 con 0x7f4738156800 2023-10-21T16:14:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.283+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4730057070 con 0x7f4738156800 2023-10-21T16:14:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.286+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 msgr2=0x7f471c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.286+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 0x7f471c06d560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4734001510 tx=0x7f4734009040 comp rx=0 tx=0).stop 2023-10-21T16:14:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.286+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 msgr2=0x7f47380cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.286+0000 7f472e7fc640 1 -- 172.21.15.53:0/346473982 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f4730053e20 con 0x7f4738156800 2023-10-21T16:14:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.286+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4738071370 tx=0x7f473000cf00 comp rx=0 tx=0).stop 2023-10-21T16:14:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 shutdown_connections 2023-10-21T16:14:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c06b0a0 0x7f471c06d560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4738156800 0x7f47380cd9d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47380ac630 0x7f47380cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4724004370 0x7f4738155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 >> 172.21.15.53:0/346473982 conn(0x7f473806a5d0 msgr2=0x7f4738070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f472e7fc640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c0362f0 0x7f471c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 shutdown_connections 2023-10-21T16:14:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 --2- 172.21.15.53:0/346473982 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f471c0362f0 0x7f471c06d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.287+0000 7f473fee0640 1 -- 172.21.15.53:0/346473982 wait complete. 2023-10-21T16:14:52.305 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:14:52.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:14:52.410 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:14:52.536 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:14:52.536 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:14:52.537 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:14:52.594 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:14:52.595 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:14:52.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:14:52.857 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:52.855+0000 7f14ef812040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:14:52.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.870+0000 7f2ee32bb640 1 Processor -- start 2023-10-21T16:14:52.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.870+0000 7f2ee32bb640 1 -- start start 2023-10-21T16:14:52.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee32bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2edc15b670 0x7f2edc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee32bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2edc0cd840 0x7f2edc0cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee32bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc1626e0 con 0x7f2edc0b5410 2023-10-21T16:14:52.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee32bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc1658d0 con 0x7f2edc15b670 2023-10-21T16:14:52.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee32bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc164cc0 con 0x7f2edc0cd840 2023-10-21T16:14:52.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.872+0000 7f2ee1057640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2edc15b670 0x7f2edc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:52.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2edc15b670 0x7f2edc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41658/0 (socket says 172.21.15.53:41658) 2023-10-21T16:14:52.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 -- 172.21.15.53:0/1746964907 learned_addr learned my addr 172.21.15.53:0/1746964907 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:52.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee0856640 1 --2- 172.21.15.53:0/1746964907 >> v2:172.21.15.124:3300/0 conn(0x7f2edc0cd840 0x7f2edc0cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:52.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 -- 172.21.15.53:0/1746964907 >> v2:172.21.15.124:3300/0 conn(0x7f2edc0cd840 msgr2=0x7f2edc0cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:52.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 --2- 172.21.15.53:0/1746964907 >> v2:172.21.15.124:3300/0 conn(0x7f2edc0cd840 0x7f2edc0cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 -- 172.21.15.53:0/1746964907 >> v1:172.21.15.53:6789/0 conn(0x7f2edc0b5410 legacy=0x7f2edc0d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ed3fff640 1 -- 172.21.15.53:0/1746964907 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382049885 0 0) 0x7f2edc1626e0 con 0x7f2edc0b5410 2023-10-21T16:14:52.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.873+0000 7f2ee1057640 1 -- 172.21.15.53:0/1746964907 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc164fa0 con 0x7f2edc15b670 2023-10-21T16:14:52.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.875+0000 7f2ee1057640 1 --2- 172.21.15.53:0/1746964907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc0cd300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0c66d0 tx=0x7f2ed802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3040f52530770c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:52.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.875+0000 7f2ed3fff640 1 -- 172.21.15.53:0/1746964907 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8031870 con 0x7f2edc15b670 2023-10-21T16:14:52.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.875+0000 7f2ed3fff640 1 -- 172.21.15.53:0/1746964907 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed80319d0 con 0x7f2edc15b670 2023-10-21T16:14:52.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.875+0000 7f2ed3fff640 1 -- 172.21.15.53:0/1746964907 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8031d60 con 0x7f2edc15b670 2023-10-21T16:14:52.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 -- 172.21.15.53:0/1746964907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 msgr2=0x7f2edc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/1746964907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc0cd300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0c66d0 tx=0x7f2ed802b910 comp rx=0 tx=0).stop 2023-10-21T16:14:52.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 -- 172.21.15.53:0/1746964907 shutdown_connections 2023-10-21T16:14:52.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/1746964907 >> v2:172.21.15.124:3300/0 conn(0x7f2edc0cd840 0x7f2edc0cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/1746964907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc0cd300 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 -- 172.21.15.53:0/1746964907 >> 172.21.15.53:0/1746964907 conn(0x7f2edc06b6a0 msgr2=0x7f2edc06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:52.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 -- 172.21.15.53:0/1746964907 shutdown_connections 2023-10-21T16:14:52.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.876+0000 7f2ee32bb640 1 -- 172.21.15.53:0/1746964907 wait complete. 2023-10-21T16:14:52.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.878+0000 7f2ee32bb640 1 Processor -- start 2023-10-21T16:14:52.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.878+0000 7f2ee32bb640 1 -- start start 2023-10-21T16:14:52.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2edc0b5410 0x7f2edc15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2edc0cd840 0x7f2edc06d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06fb00 con 0x7f2edc15b670 2023-10-21T16:14:52.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0cad20 con 0x7f2edc0b5410 2023-10-21T16:14:52.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee32bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc06f260 con 0x7f2edc0cd840 2023-10-21T16:14:52.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee1858640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:52.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee1858640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41662/0 (socket says 172.21.15.53:41662) 2023-10-21T16:14:52.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.879+0000 7f2ee1858640 1 -- 172.21.15.53:0/226543882 learned_addr learned my addr 172.21.15.53:0/226543882 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:52.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2edc0cd840 msgr2=0x7f2edc06d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2edc0cd840 0x7f2edc06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2edc0b5410 msgr2=0x7f2edc15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:52.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2edc0b5410 0x7f2edc15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:52.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 -- 172.21.15.53:0/226543882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed80094d0 con 0x7f2edc15b670 2023-10-21T16:14:52.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee1858640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400c760 tx=0x7f2ed400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:52.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003da0 con 0x7f2edc15b670 2023-10-21T16:14:52.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400ce70 con 0x7f2edc15b670 2023-10-21T16:14:52.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4005290 con 0x7f2edc15b670 2023-10-21T16:14:52.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc06f540 con 0x7f2edc15b670 2023-10-21T16:14:52.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.880+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc0caf30 con 0x7f2edc15b670 2023-10-21T16:14:52.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.881+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc066d80 con 0x7f2edc15b670 2023-10-21T16:14:52.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.882+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f2ed40053f0 con 0x7f2edc15b670 2023-10-21T16:14:52.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.883+0000 7f2ed1ffb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 0x7f2ec006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:52.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.883+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f2ed4089c30 con 0x7f2edc15b670 2023-10-21T16:14:52.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.884+0000 7f2ee1057640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 msgr2=0x7f2ec006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:52.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.884+0000 7f2ee1057640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 0x7f2ec006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:52.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:52.888+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ed40545e0 con 0x7f2edc15b670 2023-10-21T16:14:52.901 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:52.899+0000 7fcc7182c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:14:52.915 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:52.914+0000 7f14ef812040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:14:52.957 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:52.955+0000 7fcc7182c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:14:53.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.082+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2edc0b5060 con 0x7f2edc15b670 2023-10-21T16:14:53.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.084+0000 7f2ed1ffb640 1 -- 172.21.15.53:0/226543882 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18367 (secure 0 0 0) 0x7f2ed4057c00 con 0x7f2edc15b670 2023-10-21T16:14:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:53.086 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":58,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:14:52.216759+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":218,"snap_epoch":58,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2484781998":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2311155348":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/1953414266":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/269363197":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6832/3409974301":"2023-10-22T16:03:50.166436+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":7,"length":3},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":18,"length":1},{"begin":20,"length":2},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":2},{"begin":36,"length":1},{"begin":38,"length":1},{"begin":42,"length":2},{"begin":46,"length":2},{"begin":50,"length":1},{"begin":54,"length":3},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":68,"length":3},{"begin":72,"length":2},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":87,"length":4},{"begin":95,"length":4},{"begin":100,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":110,"length":1},{"begin":168,"length":51}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":8,"length":2},{"begin":18,"length":1},{"begin":26,"length":1},{"begin":34,"length":1},{"begin":50,"length":1},{"begin":60,"length":1},{"begin":70,"length":1},{"begin":73,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":97,"length":1},{"begin":105,"length":1},{"begin":208,"length":11}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:14:53.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.085+0000 7f2ee1057640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 msgr2=0x7f2ec006d530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:53.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.085+0000 7f2ee1057640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 0x7f2ec006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:53.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 msgr2=0x7f2ec006d530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:53.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 0x7f2ec006d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 msgr2=0x7f2edc06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400c760 tx=0x7f2ed400cc20 comp rx=0 tx=0).stop 2023-10-21T16:14:53.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 shutdown_connections 2023-10-21T16:14:53.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f2ec006b070 0x7f2ec006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2edc15b670 0x7f2edc06d5b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2edc0cd840 0x7f2edc06d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 --2- 172.21.15.53:0/226543882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2edc0b5410 0x7f2edc15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 >> 172.21.15.53:0/226543882 conn(0x7f2edc06b6a0 msgr2=0x7f2edc071450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:53.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.086+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 shutdown_connections 2023-10-21T16:14:53.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.087+0000 7f2ee32bb640 1 -- 172.21.15.53:0/226543882 wait complete. 2023-10-21T16:14:53.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:03:33.797436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:03:34.721730+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 218, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:14:53.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:14:53.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.317+0000 7f7f0dc6f640 1 Processor -- start 2023-10-21T16:14:53.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.317+0000 7f7f0dc6f640 1 -- start start 2023-10-21T16:14:53.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f0dc6f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f0815cf50 0x7f7f081612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f0dc6f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7f080cd460 0x7f7f080cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f0dc6f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f08161810 con 0x7f7f080cef60 2023-10-21T16:14:53.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f0dc6f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0806c440 con 0x7f7f0815cf50 2023-10-21T16:14:53.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f0dc6f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080cde10 con 0x7f7f080cd460 2023-10-21T16:14:53.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f077fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f0815cf50 0x7f7f081612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f077fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f0815cf50 0x7f7f081612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41674/0 (socket says 172.21.15.53:41674) 2023-10-21T16:14:53.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f077fe640 1 -- 172.21.15.53:0/3017095498 learned_addr learned my addr 172.21.15.53:0/3017095498 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:53.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.319+0000 7f7f07fff640 1 --2- 172.21.15.53:0/3017095498 >> v2:172.21.15.124:3300/0 conn(0x7f7f080cd460 0x7f7f080cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.320+0000 7f7f077fe640 1 -- 172.21.15.53:0/3017095498 >> v2:172.21.15.124:3300/0 conn(0x7f7f080cd460 msgr2=0x7f7f080cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:53.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.320+0000 7f7f077fe640 1 --2- 172.21.15.53:0/3017095498 >> v2:172.21.15.124:3300/0 conn(0x7f7f080cd460 0x7f7f080cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.320+0000 7f7f077fe640 1 -- 172.21.15.53:0/3017095498 >> v1:172.21.15.124:6789/0 conn(0x7f7f080cef60 legacy=0x7f7f0806c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:53.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.320+0000 7f7f077fe640 1 -- 172.21.15.53:0/3017095498 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0806b4e0 con 0x7f7f0815cf50 2023-10-21T16:14:53.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.321+0000 7f7f077fe640 1 --2- 172.21.15.53:0/3017095498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0815cf50 0x7f7f081612d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f080d0940 tx=0x7f7ef802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9416353d2babfe13 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:53.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.321+0000 7f7f067fc640 1 -- 172.21.15.53:0/3017095498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80317d0 con 0x7f7f0815cf50 2023-10-21T16:14:53.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.321+0000 7f7f067fc640 1 -- 172.21.15.53:0/3017095498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef8031930 con 0x7f7f0815cf50 2023-10-21T16:14:53.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.321+0000 7f7f067fc640 1 -- 172.21.15.53:0/3017095498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8031cc0 con 0x7f7f0815cf50 2023-10-21T16:14:53.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/3017095498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0815cf50 msgr2=0x7f7f081612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/3017095498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0815cf50 0x7f7f081612d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f080d0940 tx=0x7f7ef802b870 comp rx=0 tx=0).stop 2023-10-21T16:14:53.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/3017095498 shutdown_connections 2023-10-21T16:14:53.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/3017095498 >> v2:172.21.15.124:3300/0 conn(0x7f7f080cd460 0x7f7f080cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/3017095498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f0815cf50 0x7f7f081612d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/3017095498 >> 172.21.15.53:0/3017095498 conn(0x7f7f0806e650 msgr2=0x7f7f0806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:53.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/3017095498 shutdown_connections 2023-10-21T16:14:53.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.322+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/3017095498 wait complete. 2023-10-21T16:14:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.324+0000 7f7f0dc6f640 1 Processor -- start 2023-10-21T16:14:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.324+0000 7f7f0dc6f640 1 -- start start 2023-10-21T16:14:53.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 0x7f7f080c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f0815cf50 0x7f7f080b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c9380 con 0x7f7f080cd460 2023-10-21T16:14:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080c53d0 con 0x7f7f080cef60 2023-10-21T16:14:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f0dc6f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080b50e0 con 0x7f7f0815cf50 2023-10-21T16:14:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41686/0 (socket says 172.21.15.53:41686) 2023-10-21T16:14:53.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f077fe640 1 -- 172.21.15.53:0/2895374012 learned_addr learned my addr 172.21.15.53:0/2895374012 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f06ffd640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 0x7f7f080c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f07fff640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f0815cf50 0x7f7f080b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.326+0000 7f7f077fe640 1 -- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f0815cf50 msgr2=0x7f7f080b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f077fe640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f0815cf50 0x7f7f080b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f077fe640 1 -- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 msgr2=0x7f7f080c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f077fe640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 0x7f7f080c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f077fe640 1 -- 172.21.15.53:0/2895374012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80094d0 con 0x7f7f080cd460 2023-10-21T16:14:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f06ffd640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 0x7f7f080c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f077fe640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c00 tx=0x7f7ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:53.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8032870 con 0x7f7f080cd460 2023-10-21T16:14:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80329f0 con 0x7f7f080cd460 2023-10-21T16:14:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8032da0 con 0x7f7f080cd460 2023-10-21T16:14:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.327+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080b5380 con 0x7f7f080cd460 2023-10-21T16:14:53.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.328+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f080c5950 con 0x7f7f080cd460 2023-10-21T16:14:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.329+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080c5230 con 0x7f7f080cd460 2023-10-21T16:14:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.330+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f7ef8030070 con 0x7f7f080cd460 2023-10-21T16:14:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.331+0000 7f7eeffff640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 0x7f7ef406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.331+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f7ef80ab9b0 con 0x7f7f080cd460 2023-10-21T16:14:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.331+0000 7f7f06ffd640 1 -- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 msgr2=0x7f7ef406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:53.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.331+0000 7f7f06ffd640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 0x7f7ef406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:53.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.335+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f080c5230 con 0x7f7f080cd460 2023-10-21T16:14:53.392 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.390+0000 7f14ef812040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:14:53.400 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.398+0000 7fcc7182c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:14:53.446 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.445+0000 7f14ef812040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:14:53.451 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.449+0000 7fcc7182c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:14:53.498 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.497+0000 7f14ef812040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:14:53.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.498+0000 7fcc7182c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:14:53.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.526+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f08161810 con 0x7f7f080cd460 2023-10-21T16:14:53.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.527+0000 7f7eeffff640 1 -- 172.21.15.53:0/2895374012 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f08161810 con 0x7f7f080cd460 2023-10-21T16:14:53.529 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:14:53.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 msgr2=0x7f7ef406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 0x7f7ef406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 msgr2=0x7f7f080c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c00 tx=0x7f7ef80046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 shutdown_connections 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.529+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f7ef406b020 0x7f7ef406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f0815cf50 0x7f7f080b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f080cef60 0x7f7f080c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 --2- 172.21.15.53:0/2895374012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f080cd460 0x7f7f080c7120 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 >> 172.21.15.53:0/2895374012 conn(0x7f7f0806e650 msgr2=0x7f7f080b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:53.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 shutdown_connections 2023-10-21T16:14:53.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.530+0000 7f7f0dc6f640 1 -- 172.21.15.53:0/2895374012 wait complete. 2023-10-21T16:14:53.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:14:53.560 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.558+0000 7fcc7182c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:14:53.564 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.563+0000 7f14ef812040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:14:53.608 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.607+0000 7fcc7182c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:14:53.616 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.615+0000 7f14ef812040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:14:53.718 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.715+0000 7fcc7182c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:14:53.734 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.733+0000 7f14ef812040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:14:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.755+0000 7fdfe266d640 1 Processor -- start 2023-10-21T16:14:53.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.755+0000 7fdfe266d640 1 -- start start 2023-10-21T16:14:53.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0cf690 0x7fdfdc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdfdc0d1100 0x7fdfdc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc0cd520 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfdc0d1910 con 0x7fdfdc0d1100 2023-10-21T16:14:53.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe266d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc0ce080 con 0x7fdfdc0cf690 2023-10-21T16:14:53.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe166b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe166b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41688/0 (socket says 172.21.15.53:41688) 2023-10-21T16:14:53.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.757+0000 7fdfe166b640 1 -- 172.21.15.53:0/3776898253 learned_addr learned my addr 172.21.15.53:0/3776898253 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:53.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.758+0000 7fdfe166b640 1 -- 172.21.15.53:0/3776898253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0cf690 msgr2=0x7fdfdc0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:53.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.758+0000 7fdfe166b640 1 --2- 172.21.15.53:0/3776898253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0cf690 0x7fdfdc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.758+0000 7fdfe166b640 1 -- 172.21.15.53:0/3776898253 >> v2:172.21.15.124:3300/0 conn(0x7fdfdc0d1100 msgr2=0x7fdfdc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.758+0000 7fdfe166b640 1 --2- 172.21.15.53:0/3776898253 >> v2:172.21.15.124:3300/0 conn(0x7fdfdc0d1100 0x7fdfdc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.758+0000 7fdfe166b640 1 -- 172.21.15.53:0/3776898253 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc0cd750 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.759+0000 7fdfe166b640 1 --2- 172.21.15.53:0/3776898253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0d0020 tx=0x7fdfd802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf114a4494039820 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.759+0000 7fdfd3fff640 1 -- 172.21.15.53:0/3776898253 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80317e0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.759+0000 7fdfd3fff640 1 -- 172.21.15.53:0/3776898253 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8031940 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.759+0000 7fdfd3fff640 1 -- 172.21.15.53:0/3776898253 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8031cd0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 -- 172.21.15.53:0/3776898253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 msgr2=0x7fdfdc0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 --2- 172.21.15.53:0/3776898253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0d0020 tx=0x7fdfd802b880 comp rx=0 tx=0).stop 2023-10-21T16:14:53.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 -- 172.21.15.53:0/3776898253 shutdown_connections 2023-10-21T16:14:53.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 --2- 172.21.15.53:0/3776898253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0cf690 0x7fdfdc0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 --2- 172.21.15.53:0/3776898253 >> v2:172.21.15.124:3300/0 conn(0x7fdfdc0d1100 0x7fdfdc15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 --2- 172.21.15.53:0/3776898253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc0d34d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 -- 172.21.15.53:0/3776898253 >> 172.21.15.53:0/3776898253 conn(0x7fdfdc09bc40 msgr2=0x7fdfdc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 -- 172.21.15.53:0/3776898253 shutdown_connections 2023-10-21T16:14:53.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.760+0000 7fdfe266d640 1 -- 172.21.15.53:0/3776898253 wait complete. 2023-10-21T16:14:53.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.763+0000 7fdfe266d640 1 Processor -- start 2023-10-21T16:14:53.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.763+0000 7fdfe266d640 1 -- start start 2023-10-21T16:14:53.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfdc0cf690 0x7fdfdc15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0d1100 0x7fdfdc0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc09eeb0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc09c5b0 con 0x7fdfdc0cf690 2023-10-21T16:14:53.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe266d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfdc09d410 con 0x7fdfdc0d1100 2023-10-21T16:14:53.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe166b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:53.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe166b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41692/0 (socket says 172.21.15.53:41692) 2023-10-21T16:14:53.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe166b640 1 -- 172.21.15.53:0/191356166 learned_addr learned my addr 172.21.15.53:0/191356166 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:53.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.765+0000 7fdfe166b640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0d1100 msgr2=0x7fdfdc0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe166b640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0d1100 0x7fdfdc0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe166b640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfdc0cf690 msgr2=0x7fdfdc15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe166b640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfdc0cf690 0x7fdfdc15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe166b640 1 -- 172.21.15.53:0/191356166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd80094d0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe166b640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdfd802b8c0 tx=0x7fdfd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:53.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8032940 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8032aa0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8032e50 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc1f2520 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.766+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc09cae0 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.768+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc074150 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.768+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fdfd804e450 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.769+0000 7fdfd1ffb640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 0x7fdfc006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.769+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7fdfd80ac240 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.770+0000 7fdfe0e6a640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 msgr2=0x7fdfc006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.770+0000 7fdfe0e6a640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 0x7fdfc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:53.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.778+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfd8076960 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.781 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.778+0000 7fcc7182c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:14:53.807 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.805+0000 7f14ef812040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:14:53.829 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.827+0000 7fcc7182c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:14:53.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.857+0000 7f14ef812040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:14:53.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.917+0000 7f14ef812040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:14:53.946 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.882+0000 7fcc7182c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:14:53.946 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:53.931+0000 7fcc7182c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:14:53.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.969+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfdc0d0070 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.971 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:53.969+0000 7f14ef812040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:14:53.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.970+0000 7fdfd1ffb640 1 -- 172.21.15.53:0/191356166 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfd8079f80 con 0x7fdfdc0cf2b0 2023-10-21T16:14:53.972 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:14:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.971+0000 7fdfe0e6a640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 msgr2=0x7fdfc006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:53.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.971+0000 7fdfe0e6a640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 0x7fdfc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 msgr2=0x7fdfc006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 0x7fdfc006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 msgr2=0x7fdfdc1f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:53.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdfd802b8c0 tx=0x7fdfd80046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 shutdown_connections 2023-10-21T16:14:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdfc006b020 0x7fdfc006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfdc0d1100 0x7fdfdc0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfdc0cf690 0x7fdfdc15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.972+0000 7fdfe266d640 1 --2- 172.21.15.53:0/191356166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfdc0cf2b0 0x7fdfdc1f1fe0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:53.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.973+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 >> 172.21.15.53:0/191356166 conn(0x7fdfdc09bc40 msgr2=0x7fdfdc1e6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:53.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.973+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 shutdown_connections 2023-10-21T16:14:53.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:53.973+0000 7fdfe266d640 1 -- 172.21.15.53:0/191356166 wait complete. 2023-10-21T16:14:53.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:14:54.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.249+0000 7f685ecbb640 1 Processor -- start 2023-10-21T16:14:54.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.249+0000 7f685ecbb640 1 -- start start 2023-10-21T16:14:54.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685ecbb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68580cf2b0 0x7f68580d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685ecbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f68580ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685ecbb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68580cd520 con 0x7f68580d1100 2023-10-21T16:14:54.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685ecbb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68580d1910 con 0x7f68580cf2b0 2023-10-21T16:14:54.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685ecbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68580ce080 con 0x7f68580cf690 2023-10-21T16:14:54.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685e4ba640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f68580d1100 0x7f685815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37444/0 (socket says 172.21.15.53:37444) 2023-10-21T16:14:54.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685e4ba640 1 -- 172.21.15.53:0/2496137187 learned_addr learned my addr 172.21.15.53:0/2496137187 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.251+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2496137187 >> v2:172.21.15.124:3300/0 conn(0x7f68580cf2b0 0x7f68580d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f68580ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111599177 0 0) 0x7f68580cd520 con 0x7f68580d1100 2023-10-21T16:14:54.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6844003520 con 0x7f68580d1100 2023-10-21T16:14:54.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187641254 0 0) 0x7f6844003520 con 0x7f68580d1100 2023-10-21T16:14:54.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 msgr2=0x7f68580ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:54.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f68580ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 >> v2:172.21.15.124:3300/0 conn(0x7f68580cf2b0 msgr2=0x7f68580d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:54.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 --2- 172.21.15.53:0/2496137187 >> v2:172.21.15.124:3300/0 conn(0x7f68580cf2b0 0x7f68580d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68580cd750 con 0x7f68580d1100 2023-10-21T16:14:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f68500019f0 con 0x7f68580d1100 2023-10-21T16:14:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 >> v1:172.21.15.53:6789/0 conn(0x7f68580d1100 legacy=0x7f685815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f6844003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6844003f40 0x7f68440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6844004d00 0x7f6844005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580d1910 con 0x7f6844003f40 2023-10-21T16:14:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580ce080 con 0x7f6844004d00 2023-10-21T16:14:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.252+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684400ba30 con 0x7f68580cf690 2023-10-21T16:14:54.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6844003f40 0x7f68440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f6844003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685e4ba640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6844004d00 0x7f6844005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 -- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 msgr2=0x7f6844003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f6844003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 -- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6844004d00 msgr2=0x7f6844005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:54.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6844004d00 0x7f6844005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685d4b8640 1 -- 172.21.15.53:0/2496137187 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844003520 con 0x7f6844003f40 2023-10-21T16:14:54.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.253+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68580cf690 0x7f6844003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:14:54.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.254+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6844003f40 0x7f68440047c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ac0 tx=0x7f68440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2bd77f91f33d0a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:54.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.254+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684802c6b0 con 0x7f6844003f40 2023-10-21T16:14:54.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.254+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684802c810 con 0x7f6844003f40 2023-10-21T16:14:54.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.254+0000 7f685ccb7640 1 -- 172.21.15.53:0/2496137187 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684802cba0 con 0x7f6844003f40 2023-10-21T16:14:54.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 -- 172.21.15.53:0/2496137187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6844003f40 msgr2=0x7f68440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2496137187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6844003f40 0x7f68440047c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ac0 tx=0x7f68440057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685e4ba640 1 -- 172.21.15.53:0/2496137187 reap_dead start 2023-10-21T16:14:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 -- 172.21.15.53:0/2496137187 shutdown_connections 2023-10-21T16:14:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 -- 172.21.15.53:0/2496137187 >> 172.21.15.53:0/2496137187 conn(0x7f685809bc40 msgr2=0x7f68580d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 -- 172.21.15.53:0/2496137187 shutdown_connections 2023-10-21T16:14:54.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.255+0000 7f685ecbb640 1 -- 172.21.15.53:0/2496137187 wait complete. 2023-10-21T16:14:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.257+0000 7f685ecbb640 1 Processor -- start 2023-10-21T16:14:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.257+0000 7f685ecbb640 1 -- start start 2023-10-21T16:14:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6844003f40 0x7f685815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68580b7090 0x7f68580b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580cc270 con 0x7f68580cb880 2023-10-21T16:14:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685815a680 con 0x7f68580b7090 2023-10-21T16:14:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685ecbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685815ce50 con 0x7f6844003f40 2023-10-21T16:14:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685e4ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685e4ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35884/0 (socket says 172.21.15.53:35884) 2023-10-21T16:14:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.258+0000 7f685e4ba640 1 -- 172.21.15.53:0/2393328357 learned_addr learned my addr 172.21.15.53:0/2393328357 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685d4b8640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68580b7090 0x7f68580b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6844003f40 0x7f685815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6844003f40 msgr2=0x7f685815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6844003f40 0x7f685815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68580b7090 msgr2=0x7f68580b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68580b7090 0x7f68580b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 -- 172.21.15.53:0/2393328357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68580b7550 con 0x7f68580cb880 2023-10-21T16:14:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f685e4ba640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f685806a9b0 tx=0x7f685000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:54.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000eb30 con 0x7f68580cb880 2023-10-21T16:14:54.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685000ec90 con 0x7f68580cb880 2023-10-21T16:14:54.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.259+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850014430 con 0x7f68580cb880 2023-10-21T16:14:54.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.260+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68580b7830 con 0x7f68580cb880 2023-10-21T16:14:54.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.260+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68580b7df0 con 0x7f68580cb880 2023-10-21T16:14:54.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.261+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68580b7f80 con 0x7f68580cb880 2023-10-21T16:14:54.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.263+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f6850014590 con 0x7f68580cb880 2023-10-21T16:14:54.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.263+0000 7f684e7fc640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 0x7f683c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.264+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f6850089ab0 con 0x7f68580cb880 2023-10-21T16:14:54.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.264+0000 7f685dcb9640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 msgr2=0x7f683c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:54.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.264+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 0x7f683c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:54.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.272+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6850054380 con 0x7f68580cb880 2023-10-21T16:14:54.311 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.309+0000 7fcc7182c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:14:54.382 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.380+0000 7f14ef812040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:14:54.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.465+0000 7f685dcb9640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 msgr2=0x7f683c06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:54.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.465+0000 7f685dcb9640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 0x7f683c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:54.468 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.467+0000 7fcc7182c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:14:54.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.468+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f685815d010 con 0x7f68580cb880 2023-10-21T16:14:54.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.469+0000 7f684e7fc640 1 -- 172.21.15.53:0/2393328357 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68500579a0 con 0x7f68580cb880 2023-10-21T16:14:54.470 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:14:54.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.471+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 msgr2=0x7f683c06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.471+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 0x7f683c06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.471+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 msgr2=0x7f68580cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.471+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f685806a9b0 tx=0x7f685000e530 comp rx=0 tx=0).stop 2023-10-21T16:14:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 shutdown_connections 2023-10-21T16:14:54.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f683c06b050 0x7f683c06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580cb880 0x7f68580cbd30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68580b7090 0x7f68580b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 --2- 172.21.15.53:0/2393328357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6844003f40 0x7f685815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 >> 172.21.15.53:0/2393328357 conn(0x7f685809bc40 msgr2=0x7f68580ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:54.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 shutdown_connections 2023-10-21T16:14:54.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.472+0000 7f685ecbb640 1 -- 172.21.15.53:0/2393328357 wait complete. 2023-10-21T16:14:54.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:14:54.552 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.551+0000 7f14ef812040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:14:54.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.573+0000 7fcc7182c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:14:54.625 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.622+0000 7fcc7182c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:14:54.667 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.665+0000 7f14ef812040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:14:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.696+0000 7fdf58690640 1 Processor -- start 2023-10-21T16:14:54.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.697+0000 7fdf58690640 1 -- start start 2023-10-21T16:14:54.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf58690640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf500afbe0 0x7fdf500b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf58690640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf500cb130 0x7fdf500cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf58690640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500cbad0 con 0x7fdf5006dfc0 2023-10-21T16:14:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf58690640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500b2620 con 0x7fdf500afbe0 2023-10-21T16:14:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf58690640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500b2060 con 0x7fdf500cb130 2023-10-21T16:14:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf5642c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf500afbe0 0x7fdf500b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf5642c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf500afbe0 0x7fdf500b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35900/0 (socket says 172.21.15.53:35900) 2023-10-21T16:14:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf5642c640 1 -- 172.21.15.53:0/1865154563 learned_addr learned my addr 172.21.15.53:0/1865154563 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.698+0000 7fdf55c2b640 1 --1- 172.21.15.53:0/1865154563 >> v1:172.21.15.53:6789/0 conn(0x7fdf5006dfc0 0x7fdf50070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37446/0 (socket says 172.21.15.53:37446) 2023-10-21T16:14:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.699+0000 7fdf56c2d640 1 --2- 172.21.15.53:0/1865154563 >> v2:172.21.15.124:3300/0 conn(0x7fdf500cb130 0x7fdf500cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.699+0000 7fdf5642c640 1 -- 172.21.15.53:0/1865154563 >> v2:172.21.15.124:3300/0 conn(0x7fdf500cb130 msgr2=0x7fdf500cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:54.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.699+0000 7fdf5642c640 1 --2- 172.21.15.53:0/1865154563 >> v2:172.21.15.124:3300/0 conn(0x7fdf500cb130 0x7fdf500cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.699+0000 7fdf5642c640 1 -- 172.21.15.53:0/1865154563 >> v1:172.21.15.53:6789/0 conn(0x7fdf5006dfc0 legacy=0x7fdf50070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.699+0000 7fdf5642c640 1 -- 172.21.15.53:0/1865154563 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500b2340 con 0x7fdf500afbe0 2023-10-21T16:14:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.700+0000 7fdf5642c640 1 --2- 172.21.15.53:0/1865154563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf500b15b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdf500c8d10 tx=0x7fdf4c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a086651d2c99c33b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf5542a640 1 -- 172.21.15.53:0/1865154563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c030860 con 0x7fdf500afbe0 2023-10-21T16:14:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf5542a640 1 -- 172.21.15.53:0/1865154563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0309c0 con 0x7fdf500afbe0 2023-10-21T16:14:54.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf5542a640 1 -- 172.21.15.53:0/1865154563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c030d50 con 0x7fdf500afbe0 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 -- 172.21.15.53:0/1865154563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 msgr2=0x7fdf500b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 --2- 172.21.15.53:0/1865154563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf500b15b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdf500c8d10 tx=0x7fdf4c02b9a0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 -- 172.21.15.53:0/1865154563 shutdown_connections 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 --2- 172.21.15.53:0/1865154563 >> v2:172.21.15.124:3300/0 conn(0x7fdf500cb130 0x7fdf500cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 --2- 172.21.15.53:0/1865154563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf500b15b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.701+0000 7fdf58690640 1 -- 172.21.15.53:0/1865154563 >> 172.21.15.53:0/1865154563 conn(0x7fdf5006a5d0 msgr2=0x7fdf500b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.702+0000 7fdf58690640 1 -- 172.21.15.53:0/1865154563 shutdown_connections 2023-10-21T16:14:54.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.702+0000 7fdf58690640 1 -- 172.21.15.53:0/1865154563 wait complete. 2023-10-21T16:14:54.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.703+0000 7fdf58690640 1 Processor -- start 2023-10-21T16:14:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.703+0000 7fdf58690640 1 -- start start 2023-10-21T16:14:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 0x7fdf500ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf500cb130 0x7fdf501f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500cc7e0 con 0x7fdf500afbe0 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50158e60 con 0x7fdf5006dfc0 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf58690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50159140 con 0x7fdf500cb130 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.704+0000 7fdf55c2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35914/0 (socket says 172.21.15.53:35914) 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 -- 172.21.15.53:0/397438176 learned_addr learned my addr 172.21.15.53:0/397438176 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:54.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf5642c640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 0x7fdf500ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf56c2d640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf500cb130 0x7fdf501f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf500cb130 msgr2=0x7fdf501f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf500cb130 0x7fdf501f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 msgr2=0x7fdf500ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 0x7fdf500ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 -- 172.21.15.53:0/397438176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0094d0 con 0x7fdf500afbe0 2023-10-21T16:14:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf5642c640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 0x7fdf500ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.705+0000 7fdf55c2b640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c450 tx=0x7fdf4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.706+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40007480 con 0x7fdf500afbe0 2023-10-21T16:14:54.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.706+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf400075e0 con 0x7fdf500afbe0 2023-10-21T16:14:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.706+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40004a00 con 0x7fdf500afbe0 2023-10-21T16:14:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.706+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50159420 con 0x7fdf500afbe0 2023-10-21T16:14:54.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.706+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf500cf530 con 0x7fdf500afbe0 2023-10-21T16:14:54.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.707+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf500cc920 con 0x7fdf500afbe0 2023-10-21T16:14:54.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.708+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fdf40004b60 con 0x7fdf500afbe0 2023-10-21T16:14:54.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.709+0000 7fdf46ffd640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 0x7fdf3406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:54.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.709+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7fdf40089060 con 0x7fdf500afbe0 2023-10-21T16:14:54.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.709+0000 7fdf5642c640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 msgr2=0x7fdf3406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:54.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.709+0000 7fdf5642c640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 0x7fdf3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:54.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.717+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf400bbca0 con 0x7fdf500afbe0 2023-10-21T16:14:54.724 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.723+0000 7f14ef812040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:14:54.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.730+0000 7fcc7182c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:14:54.837 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.835+0000 7fcc7182c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:14:54.838 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.836+0000 7f14ef812040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:14:54.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.910+0000 7fdf5642c640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 msgr2=0x7fdf3406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:54.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.910+0000 7fdf5642c640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 0x7fdf3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:54.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.923+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf5004e160 con 0x7fdf500afbe0 2023-10-21T16:14:54.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.924+0000 7fdf46ffd640 1 -- 172.21.15.53:0/397438176 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v58) v1 ==== 80+0+359 (secure 0 0 0) 0x7fdf40018020 con 0x7fdf500afbe0 2023-10-21T16:14:54.925 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2484781998 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2311155348 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1953414266 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/269363197 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3409974301 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3409974301 2023-10-22T16:03:50.166436+0000 2023-10-21T16:14:54.927 INFO:teuthology.orchestra.run.smithi053.stderr:listed 6 entries 2023-10-21T16:14:54.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 msgr2=0x7fdf3406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:54.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 0x7fdf3406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 msgr2=0x7fdf501f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:54.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c450 tx=0x7fdf4000c910 comp rx=0 tx=0).stop 2023-10-21T16:14:54.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 shutdown_connections 2023-10-21T16:14:54.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7fdf3406b020 0x7fdf3406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf500cb130 0x7fdf501f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf500afbe0 0x7fdf501f5e80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 --2- 172.21.15.53:0/397438176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf5006dfc0 0x7fdf500ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:54.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 >> 172.21.15.53:0/397438176 conn(0x7fdf5006a5d0 msgr2=0x7fdf500c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:54.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 shutdown_connections 2023-10-21T16:14:54.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:54.926+0000 7fdf58690640 1 -- 172.21.15.53:0/397438176 wait complete. 2023-10-21T16:14:54.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:14:54.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:54.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:14:54.954 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:54.952+0000 7f14ef812040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:14:54.991 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:54.989+0000 7fcc7182c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:14:55.039 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:55.038+0000 7fcc7182c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:14:55.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:14:55.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:14:55.055 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:14:55.107 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:14:55.108 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:14:55.130 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:55.129+0000 7f14ef812040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:14:55.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:14:55.160 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:14:55.160 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:14:55.163 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:55.161+0000 7fcc7182c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:14:55.184 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:55.182+0000 7f14ef812040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:14:55.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:14:55.216 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:14:55.217 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:14:55.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:14:55.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:14:55.315 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:55.313+0000 7f14ef812040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:14:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.489+0000 7f8dc6573640 1 Processor -- start 2023-10-21T16:14:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.490+0000 7f8dc6573640 1 -- start start 2023-10-21T16:14:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.491+0000 7f8dc6573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dc00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.491+0000 7f8dc6573640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc006b490 con 0x7f8dc00cde60 2023-10-21T16:14:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.491+0000 7f8dc6573640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc006b770 con 0x7f8dc00cb220 2023-10-21T16:14:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.491+0000 7f8dc6573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc006ba50 con 0x7f8dc015cd20 2023-10-21T16:14:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dc4b10640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8dc00cde60 0x7f8dc006c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37460/0 (socket says 172.21.15.53:37460) 2023-10-21T16:14:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dc4b10640 1 -- 172.21.15.53:0/189506133 learned_addr learned my addr 172.21.15.53:0/189506133 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dc00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083489501 0 0) 0x7f8dc006b490 con 0x7f8dc00cde60 2023-10-21T16:14:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dac003520 con 0x7f8dc00cde60 2023-10-21T16:14:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3943548768 0 0) 0x7f8dac003520 con 0x7f8dc00cde60 2023-10-21T16:14:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 msgr2=0x7f8dc00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:55.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dc00cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 >> v1:172.21.15.124:6789/0 conn(0x7f8dc00cb220 legacy=0x7f8dc00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc006bd30 con 0x7f8dc00cde60 2023-10-21T16:14:55.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8db8001a40 con 0x7f8dc00cde60 2023-10-21T16:14:55.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.492+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dc00cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:14:55.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 >> v1:172.21.15.53:6789/0 conn(0x7f8dc00cde60 legacy=0x7f8dc006c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dac003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dac004350 0x7f8dac004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dac005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc006ba50 con 0x7f8dac0039a0 2023-10-21T16:14:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac00b980 con 0x7f8dac004350 2023-10-21T16:14:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac00baf0 con 0x7f8dc015cd20 2023-10-21T16:14:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dac003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8db7fff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dac004350 0x7f8dac004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 -- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 msgr2=0x7f8dac005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc015cd20 0x7f8dac005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 -- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dac004350 msgr2=0x7f8dac004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dac004350 0x7f8dac004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.493+0000 7f8dbffff640 1 -- 172.21.15.53:0/189506133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0009400 con 0x7f8dac0039a0 2023-10-21T16:14:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.494+0000 7f8dbffff640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dac003d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009c90 tx=0x7f8dac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb37790f6118d1f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db002c950 con 0x7f8dac0039a0 2023-10-21T16:14:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db002cab0 con 0x7f8dac0039a0 2023-10-21T16:14:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dbf7fe640 1 -- 172.21.15.53:0/189506133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db002ce60 con 0x7f8dac0039a0 2023-10-21T16:14:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc6573640 1 -- 172.21.15.53:0/189506133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 msgr2=0x7f8dac003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc6573640 1 --2- 172.21.15.53:0/189506133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dac003d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009c90 tx=0x7f8dac0057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc4b10640 1 -- 172.21.15.53:0/189506133 reap_dead start 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc6573640 1 -- 172.21.15.53:0/189506133 shutdown_connections 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc6573640 1 -- 172.21.15.53:0/189506133 >> 172.21.15.53:0/189506133 conn(0x7f8dc0069f30 msgr2=0x7f8dc006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.495+0000 7f8dc6573640 1 -- 172.21.15.53:0/189506133 shutdown_connections 2023-10-21T16:14:55.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.496+0000 7f8dc6573640 1 -- 172.21.15.53:0/189506133 wait complete. 2023-10-21T16:14:55.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.497+0000 7f8dc6573640 1 Processor -- start 2023-10-21T16:14:55.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.497+0000 7f8dc6573640 1 -- start start 2023-10-21T16:14:55.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc00b4770 0x7f8dc00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dc00c4370 0x7f8dc00c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00c4d60 con 0x7f8dac0039a0 2023-10-21T16:14:55.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01f25c0 con 0x7f8dc00c4370 2023-10-21T16:14:55.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.498+0000 7f8dc6573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00c61e0 con 0x7f8dc00b4770 2023-10-21T16:14:55.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35926/0 (socket says 172.21.15.53:35926) 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 -- 172.21.15.53:0/1944274172 learned_addr learned my addr 172.21.15.53:0/1944274172 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dc4b10640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dc00c4370 0x7f8dc00c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8db7fff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc00b4770 0x7f8dc00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc00b4770 msgr2=0x7f8dc00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc00b4770 0x7f8dc00b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dc00c4370 msgr2=0x7f8dc00c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dc00c4370 0x7f8dc00c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 -- 172.21.15.53:0/1944274172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00ca5c0 con 0x7f8dac0039a0 2023-10-21T16:14:55.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.499+0000 7f8dbffff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8db002c430 tx=0x7f8db0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:55.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.500+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0006e30 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.500+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0004020 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.500+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00043b0 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.500+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00b41b0 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.500+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00c6440 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.501+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0066d80 con 0x7f8dac0039a0 2023-10-21T16:14:55.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.502+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f8db0003610 con 0x7f8dac0039a0 2023-10-21T16:14:55.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.503+0000 7f8dbd7fa640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 0x7f8da006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:55.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.503+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f8db00ac4f0 con 0x7f8dac0039a0 2023-10-21T16:14:55.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.503+0000 7f8db7fff640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 msgr2=0x7f8da006d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:55.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.503+0000 7f8db7fff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 0x7f8da006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:55.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.511+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8db0076db0 con 0x7f8dac0039a0 2023-10-21T16:14:55.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.704+0000 7f8db7fff640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 msgr2=0x7f8da006d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:55.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.704+0000 7f8db7fff640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 0x7f8da006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:55.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.745+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc00b6970 con 0x7f8dac0039a0 2023-10-21T16:14:55.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.747+0000 7f8dbd7fa640 1 -- 172.21.15.53:0/1944274172 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f8db007a3d0 con 0x7f8dac0039a0 2023-10-21T16:14:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:55.749 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:14:55.749 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:14:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 msgr2=0x7f8da006d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 0x7f8da006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 msgr2=0x7f8dc01f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8db002c430 tx=0x7f8db0004730 comp rx=0 tx=0).stop 2023-10-21T16:14:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 shutdown_connections 2023-10-21T16:14:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f8da006b050 0x7f8da006d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dc00c4370 0x7f8dc00c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dc00b4770 0x7f8dc00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.749+0000 7f8dc6573640 1 --2- 172.21.15.53:0/1944274172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dac0039a0 0x7f8dc01f2080 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.750+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 >> 172.21.15.53:0/1944274172 conn(0x7f8dc0069f30 msgr2=0x7f8dc006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.750+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 shutdown_connections 2023-10-21T16:14:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:55.750+0000 7f8dc6573640 1 -- 172.21.15.53:0/1944274172 wait complete. 2023-10-21T16:14:55.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:14:55.985 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:55.983+0000 7fcc7182c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:14:56.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.004+0000 7f6135399640 1 Processor -- start 2023-10-21T16:14:56.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.004+0000 7f6135399640 1 -- start start 2023-10-21T16:14:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f6135399640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61300ce2c0 0x7f613015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f6135399640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61300cde30 con 0x7f6130157480 2023-10-21T16:14:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f6135399640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6130157860 con 0x7f613006b680 2023-10-21T16:14:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f6135399640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613015e2c0 con 0x7f61300ce2c0 2023-10-21T16:14:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f612ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6130157480 0x7f613006af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37474/0 (socket says 172.21.15.53:37474) 2023-10-21T16:14:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f612ffff640 1 -- 172.21.15.53:0/1305086482 learned_addr learned my addr 172.21.15.53:0/1305086482 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.006+0000 7f6134b98640 1 --2- 172.21.15.53:0/1305086482 >> v2:172.21.15.124:3300/0 conn(0x7f61300ce2c0 0x7f613015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140129521 0 0) 0x7f61300cde30 con 0x7f6130157480 2023-10-21T16:14:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6118003520 con 0x7f6130157480 2023-10-21T16:14:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 32241083 0 0) 0x7f6118003520 con 0x7f6130157480 2023-10-21T16:14:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 >> v2:172.21.15.124:3300/0 conn(0x7f61300ce2c0 msgr2=0x7f613015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 --2- 172.21.15.53:0/1305086482 >> v2:172.21.15.124:3300/0 conn(0x7f61300ce2c0 0x7f613015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 >> v1:172.21.15.124:6789/0 conn(0x7f613006b680 legacy=0x7f613015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300cd500 con 0x7f6130157480 2023-10-21T16:14:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6120001670 con 0x7f6130157480 2023-10-21T16:14:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 >> v1:172.21.15.53:6789/0 conn(0x7f6130157480 legacy=0x7f613006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6118003970 0x7f6118003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61180047c0 0x7f6118004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 0x7f611800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613015e2c0 con 0x7f61180052a0 2023-10-21T16:14:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611800c150 con 0x7f61180047c0 2023-10-21T16:14:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.007+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611800c310 con 0x7f6118003970 2023-10-21T16:14:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 0x7f611800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f612ffff640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6118003970 0x7f6118003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f612f7fe640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61180047c0 0x7f6118004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 -- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6118003970 msgr2=0x7f6118003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6118003970 0x7f6118003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 -- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61180047c0 msgr2=0x7f6118004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61180047c0 0x7f6118004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.008+0000 7f6134b98640 1 -- 172.21.15.53:0/1305086482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6118003520 con 0x7f61180052a0 2023-10-21T16:14:56.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.009+0000 7f6134b98640 1 -- 172.21.15.53:0/1305086482 reap_dead start 2023-10-21T16:14:56.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.009+0000 7f6134b98640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 0x7f611800b980 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6128007330 tx=0x7f61180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9985c138d28c6e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.009+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612802ccd0 con 0x7f61180052a0 2023-10-21T16:14:56.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.009+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612802ce30 con 0x7f61180052a0 2023-10-21T16:14:56.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.009+0000 7f612effd640 1 -- 172.21.15.53:0/1305086482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128003bd0 con 0x7f61180052a0 2023-10-21T16:14:56.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 -- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 msgr2=0x7f611800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 0x7f611800b980 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6128007330 tx=0x7f61180057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 -- 172.21.15.53:0/1305086482 shutdown_connections 2023-10-21T16:14:56.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 --2- 172.21.15.53:0/1305086482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61180052a0 0x7f611800b980 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 -- 172.21.15.53:0/1305086482 >> 172.21.15.53:0/1305086482 conn(0x7f61300d2ae0 msgr2=0x7f61300d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 -- 172.21.15.53:0/1305086482 shutdown_connections 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.010+0000 7f6135399640 1 -- 172.21.15.53:0/1305086482 wait complete. 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.012+0000 7f6135399640 1 Processor -- start 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.012+0000 7f6135399640 1 -- start start 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61180052a0 0x7f61300cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f613014c1b0 0x7f6130149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613014cfe0 con 0x7f61300c5ab0 2023-10-21T16:14:56.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300ccb70 con 0x7f613014c1b0 2023-10-21T16:14:56.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f6135399640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300cc5b0 con 0x7f61180052a0 2023-10-21T16:14:56.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.013+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35948/0 (socket says 172.21.15.53:35948) 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 -- 172.21.15.53:0/933629643 learned_addr learned my addr 172.21.15.53:0/933629643 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612ffff640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61180052a0 0x7f61300cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f6134b98640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f613014c1b0 0x7f6130149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61180052a0 msgr2=0x7f61300cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61180052a0 0x7f61300cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f613014c1b0 msgr2=0x7f6130149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f613014c1b0 0x7f6130149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 -- 172.21.15.53:0/933629643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300cc890 con 0x7f61300c5ab0 2023-10-21T16:14:56.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.014+0000 7f612f7fe640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f612400d4e0 tx=0x7f612400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.015+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124012800 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.015+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124012960 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.015+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124012d50 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.015+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613014cc70 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.015+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613014f4a0 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.016+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613014d150 con 0x7f61300c5ab0 2023-10-21T16:14:56.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.017+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f6124019360 con 0x7f61300c5ab0 2023-10-21T16:14:56.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.018+0000 7f612cff9640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 0x7f611c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.018+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f6124089ea0 con 0x7f61300c5ab0 2023-10-21T16:14:56.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.018+0000 7f612ffff640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 msgr2=0x7f611c06d560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:56.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.018+0000 7f612ffff640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 0x7f611c06d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:56.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.026+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61240547d0 con 0x7f61300c5ab0 2023-10-21T16:14:56.178 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:56.177+0000 7f14ef812040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:14:56.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.219+0000 7f612ffff640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 msgr2=0x7f611c06d560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:56.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.219+0000 7f612ffff640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 0x7f611c06d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:14:56.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.223+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f613004e160 con 0x7f61300c5ab0 2023-10-21T16:14:56.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.225+0000 7f612cff9640 1 -- 172.21.15.53:0/933629643 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+18367 (secure 0 0 0) 0x7f6124020020 con 0x7f61300c5ab0 2023-10-21T16:14:56.228 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:56.228 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":58,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:14:52.216759+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":218,"snap_epoch":58,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2484781998":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2311155348":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/1953414266":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/269363197":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6832/3409974301":"2023-10-22T16:03:50.166436+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":7,"length":3},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":18,"length":1},{"begin":20,"length":2},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":2},{"begin":36,"length":1},{"begin":38,"length":1},{"begin":42,"length":2},{"begin":46,"length":2},{"begin":50,"length":1},{"begin":54,"length":3},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":68,"length":3},{"begin":72,"length":2},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":87,"length":4},{"begin":95,"length":4},{"begin":100,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":110,"length":1},{"begin":168,"length":51}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":8,"length":2},{"begin":18,"length":1},{"begin":26,"length":1},{"begin":34,"length":1},{"begin":50,"length":1},{"begin":60,"length":1},{"begin":70,"length":1},{"begin":73,"length":1},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":97,"length":1},{"begin":105,"length":1},{"begin":208,"length":11}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:14:56.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 msgr2=0x7f611c06d560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:56.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 0x7f611c06d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 msgr2=0x7f613014ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f612400d4e0 tx=0x7f612400d9a0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 shutdown_connections 2023-10-21T16:14:56.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f611c06b0a0 0x7f611c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f613014c1b0 0x7f6130149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61300c5ab0 0x7f613014ef60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 --2- 172.21.15.53:0/933629643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61180052a0 0x7f61300cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 >> 172.21.15.53:0/933629643 conn(0x7f61300d2ae0 msgr2=0x7f613014b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:56.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 shutdown_connections 2023-10-21T16:14:56.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.228+0000 7f6135399640 1 -- 172.21.15.53:0/933629643 wait complete. 2023-10-21T16:14:56.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:03:33.797436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:03:34.721730+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 218, 'snap_epoch': 58, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:14:56.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:14:56.283 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.281+0000 7fcc7182c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:14:56.342 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.340+0000 7fcc7182c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:14:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.452+0000 7f0d69b4e640 1 Processor -- start 2023-10-21T16:14:56.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.452+0000 7f0d69b4e640 1 -- start start 2023-10-21T16:14:56.455 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.453+0000 7fcc7182c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:14:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d69b4e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d6415cca0 0x7f0d640b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d69b4e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 0x7f0d640cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d69b4e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d640b7e70 con 0x7f0d640cde70 2023-10-21T16:14:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d69b4e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d640ce2e0 con 0x7f0d6415cca0 2023-10-21T16:14:56.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d69b4e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6415d080 con 0x7f0d640b6cd0 2023-10-21T16:14:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d637fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d6415cca0 0x7f0d640b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d637fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0d6415cca0 0x7f0d640b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35950/0 (socket says 172.21.15.53:35950) 2023-10-21T16:14:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d637fe640 1 -- 172.21.15.53:0/3860764533 learned_addr learned my addr 172.21.15.53:0/3860764533 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:56.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.454+0000 7f0d63fff640 1 --1- 172.21.15.53:0/3860764533 >> v1:172.21.15.53:6789/0 conn(0x7f0d640cde70 0x7f0d640b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37476/0 (socket says 172.21.15.53:37476) 2023-10-21T16:14:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d5bfff640 1 --2- 172.21.15.53:0/3860764533 >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 0x7f0d640cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d637fe640 1 -- 172.21.15.53:0/3860764533 >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 msgr2=0x7f0d640cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d637fe640 1 --2- 172.21.15.53:0/3860764533 >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 0x7f0d640cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d637fe640 1 -- 172.21.15.53:0/3860764533 >> v1:172.21.15.53:6789/0 conn(0x7f0d640cde70 legacy=0x7f0d640b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d637fe640 1 -- 172.21.15.53:0/3860764533 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640b7260 con 0x7f0d6415cca0 2023-10-21T16:14:56.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.455+0000 7f0d5bfff640 1 --2- 172.21.15.53:0/3860764533 >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 0x7f0d640cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:14:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.456+0000 7f0d637fe640 1 --2- 172.21.15.53:0/3860764533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640b6790 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d640c9ce0 tx=0x7f0d5402ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeae2ad17e71ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.456+0000 7f0d62ffd640 1 -- 172.21.15.53:0/3860764533 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5402f860 con 0x7f0d6415cca0 2023-10-21T16:14:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.456+0000 7f0d62ffd640 1 -- 172.21.15.53:0/3860764533 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5402f9c0 con 0x7f0d6415cca0 2023-10-21T16:14:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.456+0000 7f0d62ffd640 1 -- 172.21.15.53:0/3860764533 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5402fd50 con 0x7f0d6415cca0 2023-10-21T16:14:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 -- 172.21.15.53:0/3860764533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 msgr2=0x7f0d640b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/3860764533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640b6790 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0d640c9ce0 tx=0x7f0d5402ba40 comp rx=0 tx=0).stop 2023-10-21T16:14:56.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 -- 172.21.15.53:0/3860764533 shutdown_connections 2023-10-21T16:14:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/3860764533 >> v2:172.21.15.124:3300/0 conn(0x7f0d640b6cd0 0x7f0d640cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/3860764533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640b6790 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 -- 172.21.15.53:0/3860764533 >> 172.21.15.53:0/3860764533 conn(0x7f0d640718a0 msgr2=0x7f0d64071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 -- 172.21.15.53:0/3860764533 shutdown_connections 2023-10-21T16:14:56.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.457+0000 7f0d69b4e640 1 -- 172.21.15.53:0/3860764533 wait complete. 2023-10-21T16:14:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.460+0000 7f0d69b4e640 1 Processor -- start 2023-10-21T16:14:56.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.460+0000 7f0d69b4e640 1 -- start start 2023-10-21T16:14:56.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d640b6cd0 0x7f0d64153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 0x7f0d640c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640cb860 con 0x7f0d6415cca0 2023-10-21T16:14:56.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640cbb00 con 0x7f0d640cde70 2023-10-21T16:14:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d69b4e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640cbde0 con 0x7f0d640b6cd0 2023-10-21T16:14:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35958/0 (socket says 172.21.15.53:35958) 2023-10-21T16:14:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d63fff640 1 -- 172.21.15.53:0/889540728 learned_addr learned my addr 172.21.15.53:0/889540728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:56.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d637fe640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d640b6cd0 0x7f0d64153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.461+0000 7f0d5bfff640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 0x7f0d640c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 -- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d640b6cd0 msgr2=0x7f0d64153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d640b6cd0 0x7f0d64153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 -- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 msgr2=0x7f0d640c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 0x7f0d640c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 -- 172.21.15.53:0/889540728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540094d0 con 0x7f0d6415cca0 2023-10-21T16:14:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d5bfff640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 0x7f0d640c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:14:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d63fff640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c780 tx=0x7f0d5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00ce90 con 0x7f0d6415cca0 2023-10-21T16:14:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00a640 con 0x7f0d6415cca0 2023-10-21T16:14:56.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00a9f0 con 0x7f0d6415cca0 2023-10-21T16:14:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d640cc0c0 con 0x7f0d6415cca0 2023-10-21T16:14:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.462+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d641536b0 con 0x7f0d6415cca0 2023-10-21T16:14:56.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.464+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6404e160 con 0x7f0d6415cca0 2023-10-21T16:14:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.465+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f0d5c00ab50 con 0x7f0d6415cca0 2023-10-21T16:14:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.466+0000 7f0d60ff9640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 0x7f0d4406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.466+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7076+0+0 (secure 0 0 0) 0x7f0d5c089500 con 0x7f0d6415cca0 2023-10-21T16:14:56.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.467+0000 7f0d637fe640 1 -- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 msgr2=0x7f0d4406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/637025846 2023-10-21T16:14:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.467+0000 7f0d637fe640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 0x7f0d4406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:14:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.469+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0d5c0542e0 con 0x7f0d6415cca0 2023-10-21T16:14:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.469+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 msgr2=0x7f0d4406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:14:56.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.469+0000 7f0d60ff9640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 0x7f0d4406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.476+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d5c072aa0 con 0x7f0d6415cca0 2023-10-21T16:14:56.504 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:56.502+0000 7f14ef812040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:14:56.549 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf33b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.550 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf33b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.550 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf33b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.550 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf33b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.550 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf33b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.551 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf4bb7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.551 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf4bb7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.551 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf4bb7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.551 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf4bb7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.551 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:14:56.547+0000 7fcbf4bb7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:14:56.569 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:56.568+0000 7f14ef812040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:14:56.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.676+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d6415c4c0 con 0x7f0d6415cca0 2023-10-21T16:14:56.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.677+0000 7f0d60ff9640 1 -- 172.21.15.53:0/889540728 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d5c057710 con 0x7f0d6415cca0 2023-10-21T16:14:56.679 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:14:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 msgr2=0x7f0d640d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c780 tx=0x7f0d5c00cc40 comp rx=0 tx=0).stop 2023-10-21T16:14:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 shutdown_connections 2023-10-21T16:14:56.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:6832/637025846,v1:172.21.15.124:6833/637025846] conn(0x7f0d4406b020 0x7f0d4406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6415cca0 0x7f0d640d2350 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d640cde70 0x7f0d640c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 --2- 172.21.15.53:0/889540728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d640b6cd0 0x7f0d64153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 >> 172.21.15.53:0/889540728 conn(0x7f0d640718a0 msgr2=0x7f0d640d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 shutdown_connections 2023-10-21T16:14:56.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.680+0000 7f0d69b4e640 1 -- 172.21.15.53:0/889540728 wait complete. 2023-10-21T16:14:56.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:14:56.701 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:14:56.699+0000 7f14ef812040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:14:56.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.903+0000 7fd85ded1640 1 Processor -- start 2023-10-21T16:14:56.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.903+0000 7fd85ded1640 1 -- start start 2023-10-21T16:14:56.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd85ded1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 0x7fd8580aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd85ded1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8580ca680 0x7fd8580b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd85ded1640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8580cab60 con 0x7fd85815ac00 2023-10-21T16:14:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd85ded1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8580cb960 con 0x7fd8580ca680 2023-10-21T16:14:56.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd85ded1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580cbdf0 con 0x7fd85815a820 2023-10-21T16:14:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd857fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8580ca680 0x7fd8580b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd857fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8580ca680 0x7fd8580b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35962/0 (socket says 172.21.15.53:35962) 2023-10-21T16:14:56.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.905+0000 7fd857fff640 1 -- 172.21.15.53:0/2760416602 learned_addr learned my addr 172.21.15.53:0/2760416602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd85cecf640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 0x7fd8580aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd857fff640 1 -- 172.21.15.53:0/2760416602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 msgr2=0x7fd8580aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd857fff640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 0x7fd8580aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd857fff640 1 -- 172.21.15.53:0/2760416602 >> v1:172.21.15.124:6789/0 conn(0x7fd85815ac00 legacy=0x7fd8580c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd857fff640 1 -- 172.21.15.53:0/2760416602 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8580cb030 con 0x7fd8580ca680 2023-10-21T16:14:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.906+0000 7fd85cecf640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 0x7fd8580aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.907+0000 7fd857fff640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8580ca680 0x7fd8580b04b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f24cb2ab4c397b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.907+0000 7fd8577fe640 1 -- 172.21.15.53:0/2760416602 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440317c0 con 0x7fd8580ca680 2023-10-21T16:14:56.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.907+0000 7fd8577fe640 1 -- 172.21.15.53:0/2760416602 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844031920 con 0x7fd8580ca680 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.907+0000 7fd8577fe640 1 -- 172.21.15.53:0/2760416602 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844031cb0 con 0x7fd8580ca680 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 -- 172.21.15.53:0/2760416602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8580ca680 msgr2=0x7fd8580b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8580ca680 0x7fd8580b04b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84402b860 comp rx=0 tx=0).stop 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 -- 172.21.15.53:0/2760416602 shutdown_connections 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8580ca680 0x7fd8580b04b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 --2- 172.21.15.53:0/2760416602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd85815a820 0x7fd8580aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 -- 172.21.15.53:0/2760416602 >> 172.21.15.53:0/2760416602 conn(0x7fd85806c050 msgr2=0x7fd85806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 -- 172.21.15.53:0/2760416602 shutdown_connections 2023-10-21T16:14:56.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.908+0000 7fd85ded1640 1 -- 172.21.15.53:0/2760416602 wait complete. 2023-10-21T16:14:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.911+0000 7fd85ded1640 1 Processor -- start 2023-10-21T16:14:56.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.911+0000 7fd85ded1640 1 -- start start 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8580ca680 0x7fd8580c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd85815a820 0x7fd8581f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c6340 con 0x7fd85815ac00 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8580c5080 con 0x7fd85815a820 2023-10-21T16:14:56.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85ded1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581f0e60 con 0x7fd8580ca680 2023-10-21T16:14:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85d6d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85d6d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35976/0 (socket says 172.21.15.53:35976) 2023-10-21T16:14:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.912+0000 7fd85d6d0640 1 -- 172.21.15.53:0/3607585357 learned_addr learned my addr 172.21.15.53:0/3607585357 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85cecf640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8580ca680 0x7fd8580c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:56.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 -- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8580ca680 msgr2=0x7fd8580c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8580ca680 0x7fd8580c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 -- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd85815a820 msgr2=0x7fd8581f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd85815a820 0x7fd8581f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 -- 172.21.15.53:0/3607585357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440094d0 con 0x7fd85815ac00 2023-10-21T16:14:56.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd85d6d0640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c410 tx=0x7fd85000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.913+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011790 con 0x7fd85815ac00 2023-10-21T16:14:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.914+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500118f0 con 0x7fd85815ac00 2023-10-21T16:14:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.914+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011ce0 con 0x7fd85815ac00 2023-10-21T16:14:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.914+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8580c7e20 con 0x7fd85815ac00 2023-10-21T16:14:56.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.914+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8580c8660 con 0x7fd85815ac00 2023-10-21T16:14:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.915+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd858074150 con 0x7fd85815ac00 2023-10-21T16:14:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.916+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd85001c460 con 0x7fd85815ac00 2023-10-21T16:14:56.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.917+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fd850089600 con 0x7fd85815ac00 2023-10-21T16:14:56.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:56.921+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8500544d0 con 0x7fd85815ac00 2023-10-21T16:14:57.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.112+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd85815dd00 con 0x7fd85815ac00 2023-10-21T16:14:57.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.113+0000 7fd8557fa640 1 -- 172.21.15.53:0/3607585357 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd850057af0 con 0x7fd85815ac00 2023-10-21T16:14:57.115 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:14:57.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 msgr2=0x7fd8581ef740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c410 tx=0x7fd85000c810 comp rx=0 tx=0).stop 2023-10-21T16:14:57.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 shutdown_connections 2023-10-21T16:14:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd85815ac00 0x7fd8581ef740 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd85815a820 0x7fd8581f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 --2- 172.21.15.53:0/3607585357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8580ca680 0x7fd8580c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.115+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 >> 172.21.15.53:0/3607585357 conn(0x7fd85806c050 msgr2=0x7fd8580b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:57.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.116+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 shutdown_connections 2023-10-21T16:14:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.116+0000 7fd85ded1640 1 -- 172.21.15.53:0/3607585357 wait complete. 2023-10-21T16:14:57.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:14:57.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.353+0000 7f53f8982640 1 Processor -- start 2023-10-21T16:14:57.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.353+0000 7f53f8982640 1 -- start start 2023-10-21T16:14:57.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f8982640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53f40b0680 0x7f53f415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f8982640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f40cab60 con 0x7f53f40ca680 2023-10-21T16:14:57.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f8982640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f40cb960 con 0x7f53f415a820 2023-10-21T16:14:57.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f8982640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f40cbdf0 con 0x7f53f40b0680 2023-10-21T16:14:57.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53f40b0680 0x7f53f415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f53f40b0680 0x7f53f415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35980/0 (socket says 172.21.15.53:35980) 2023-10-21T16:14:57.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f259c640 1 -- 172.21.15.53:0/2152547098 learned_addr learned my addr 172.21.15.53:0/2152547098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:57.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.355+0000 7f53f359e640 1 --1- 172.21.15.53:0/2152547098 >> v1:172.21.15.53:6789/0 conn(0x7f53f40ca680 0x7f53f415ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37488/0 (socket says 172.21.15.53:37488) 2023-10-21T16:14:57.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.356+0000 7f53f259c640 1 -- 172.21.15.53:0/2152547098 >> v1:172.21.15.124:6789/0 conn(0x7f53f415a820 legacy=0x7f53f40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.356+0000 7f53f259c640 1 -- 172.21.15.53:0/2152547098 >> v1:172.21.15.53:6789/0 conn(0x7f53f40ca680 legacy=0x7f53f415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.356+0000 7f53f259c640 1 -- 172.21.15.53:0/2152547098 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40cb030 con 0x7f53f40b0680 2023-10-21T16:14:57.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.356+0000 7f53f1d9b640 1 -- 172.21.15.53:0/2152547098 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428368735 0 0) 0x7f53f40cab60 con 0x7f53f40ca680 2023-10-21T16:14:57.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.357+0000 7f53f259c640 1 --2- 172.21.15.53:0/2152547098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f40b0680 0x7f53f415e540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf42e56724d6082 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.357+0000 7f53f1d9b640 1 -- 172.21.15.53:0/2152547098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8030860 con 0x7f53f40b0680 2023-10-21T16:14:57.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.357+0000 7f53f1d9b640 1 -- 172.21.15.53:0/2152547098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80309c0 con 0x7f53f40b0680 2023-10-21T16:14:57.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.357+0000 7f53f1d9b640 1 -- 172.21.15.53:0/2152547098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8030d50 con 0x7f53f40b0680 2023-10-21T16:14:57.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 -- 172.21.15.53:0/2152547098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f40b0680 msgr2=0x7f53f415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 --2- 172.21.15.53:0/2152547098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f40b0680 0x7f53f415e540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009870 tx=0x7f53e802b9a0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 -- 172.21.15.53:0/2152547098 shutdown_connections 2023-10-21T16:14:57.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 --2- 172.21.15.53:0/2152547098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f40b0680 0x7f53f415e540 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 -- 172.21.15.53:0/2152547098 >> 172.21.15.53:0/2152547098 conn(0x7f53f406c050 msgr2=0x7f53f406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:57.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 -- 172.21.15.53:0/2152547098 shutdown_connections 2023-10-21T16:14:57.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.358+0000 7f53f8982640 1 -- 172.21.15.53:0/2152547098 wait complete. 2023-10-21T16:14:57.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.360+0000 7f53f8982640 1 Processor -- start 2023-10-21T16:14:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.360+0000 7f53f8982640 1 -- start start 2023-10-21T16:14:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40b0680 0x7f53f40b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40ca680 0x7f53f40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4157c90 con 0x7f53f415a820 2023-10-21T16:14:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4157800 con 0x7f53f40b0680 2023-10-21T16:14:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f8982640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40c7c80 con 0x7f53f40ca680 2023-10-21T16:14:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35990/0 (socket says 172.21.15.53:35990) 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.361+0000 7f53f359e640 1 -- 172.21.15.53:0/1270290442 learned_addr learned my addr 172.21.15.53:0/1270290442 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f259c640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40ca680 0x7f53f40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f2d9d640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40b0680 0x7f53f40b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 -- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40ca680 msgr2=0x7f53f40b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40ca680 0x7f53f40b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 -- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40b0680 msgr2=0x7f53f40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40b0680 0x7f53f40b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 -- 172.21.15.53:0/1270290442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80094d0 con 0x7f53f415a820 2023-10-21T16:14:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.362+0000 7f53f359e640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c730 tx=0x7f53ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.363+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00ce40 con 0x7f53f415a820 2023-10-21T16:14:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.363+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec00a640 con 0x7f53f415a820 2023-10-21T16:14:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.363+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00a9f0 con 0x7f53f415a820 2023-10-21T16:14:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.363+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f4156ed0 con 0x7f53f415a820 2023-10-21T16:14:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.363+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f4157490 con 0x7f53f415a820 2023-10-21T16:14:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.364+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f4157970 con 0x7f53f415a820 2023-10-21T16:14:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.365+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f53ec00ab50 con 0x7f53f415a820 2023-10-21T16:14:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.366+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f53ec088ba0 con 0x7f53f415a820 2023-10-21T16:14:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.371+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53ec0539b0 con 0x7f53f415a820 2023-10-21T16:14:57.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.485+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f53ec053460 con 0x7f53f415a820 2023-10-21T16:14:57.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.486+0000 7f53db7fe640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 0x7f53e00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.487+0000 7f53f2d9d640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 0x7f53e00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.493+0000 7f53f2d9d640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 0x7f53e00734f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53f40c85f0 tx=0x7f53e4009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.577+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53f4074150 con 0x7f53f415a820 2023-10-21T16:14:57.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.578+0000 7f53db7fe640 1 -- 172.21.15.53:0/1270290442 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53ec073d10 con 0x7f53f415a820 2023-10-21T16:14:57.580 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:14:57.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.580+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 msgr2=0x7f53e00734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.580+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 0x7f53e00734f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f53f40c85f0 tx=0x7f53e4009040 comp rx=0 tx=0).stop 2023-10-21T16:14:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 msgr2=0x7f53f40c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c730 tx=0x7f53ec00cbf0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 shutdown_connections 2023-10-21T16:14:57.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f53e0071100 0x7f53e00734f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53f415a820 0x7f53f40c6ec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53f40ca680 0x7f53f40b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 --2- 172.21.15.53:0/1270290442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53f40b0680 0x7f53f40b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 >> 172.21.15.53:0/1270290442 conn(0x7f53f406c050 msgr2=0x7f53f40ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:57.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 shutdown_connections 2023-10-21T16:14:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.581+0000 7f53f8982640 1 -- 172.21.15.53:0/1270290442 wait complete. 2023-10-21T16:14:57.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:14:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.829+0000 7fb9e4e9f640 1 Processor -- start 2023-10-21T16:14:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.829+0000 7fb9e4e9f640 1 -- start start 2023-10-21T16:14:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9e4e9f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9e01599d0 0x7fb9e00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9e4e9f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 0x7fb9e00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9e4e9f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cacf0 con 0x7fb9e0159db0 2023-10-21T16:14:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9e4e9f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00b10f0 con 0x7fb9e01599d0 2023-10-21T16:14:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9e4e9f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00ca3c0 con 0x7fb9e00cc210 2023-10-21T16:14:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9df7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9e01599d0 0x7fb9e00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9dffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb9e0159db0 0x7fb9e015d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37496/0 (socket says 172.21.15.53:37496) 2023-10-21T16:14:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9df7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9e01599d0 0x7fb9e00ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35992/0 (socket says 172.21.15.53:35992) 2023-10-21T16:14:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.831+0000 7fb9dffff640 1 -- 172.21.15.53:0/4204119961 learned_addr learned my addr 172.21.15.53:0/4204119961 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9deffd640 1 --2- 172.21.15.53:0/4204119961 >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 0x7fb9e00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9df7fe640 1 -- 172.21.15.53:0/4204119961 >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 msgr2=0x7fb9e00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9df7fe640 1 --2- 172.21.15.53:0/4204119961 >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 0x7fb9e00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9df7fe640 1 -- 172.21.15.53:0/4204119961 >> v1:172.21.15.53:6789/0 conn(0x7fb9e0159db0 legacy=0x7fb9e015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9df7fe640 1 -- 172.21.15.53:0/4204119961 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00ca6a0 con 0x7fb9e01599d0 2023-10-21T16:14:57.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.832+0000 7fb9deffd640 1 --2- 172.21.15.53:0/4204119961 >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 0x7fb9e00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:57.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.833+0000 7fb9df7fe640 1 --2- 172.21.15.53:0/4204119961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e01599d0 0x7fb9e00ad730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00097a0 tx=0x7fb9d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13f67c72e729e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.833+0000 7fb9de7fc640 1 -- 172.21.15.53:0/4204119961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0031830 con 0x7fb9e01599d0 2023-10-21T16:14:57.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.833+0000 7fb9de7fc640 1 -- 172.21.15.53:0/4204119961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0031990 con 0x7fb9e01599d0 2023-10-21T16:14:57.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.833+0000 7fb9de7fc640 1 -- 172.21.15.53:0/4204119961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0031d20 con 0x7fb9e01599d0 2023-10-21T16:14:57.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/4204119961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e01599d0 msgr2=0x7fb9e00ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/4204119961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e01599d0 0x7fb9e00ad730 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00097a0 tx=0x7fb9d002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/4204119961 shutdown_connections 2023-10-21T16:14:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/4204119961 >> v2:172.21.15.124:3300/0 conn(0x7fb9e00cc210 0x7fb9e00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/4204119961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e01599d0 0x7fb9e00ad730 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/4204119961 >> 172.21.15.53:0/4204119961 conn(0x7fb9e00cb0c0 msgr2=0x7fb9e00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:57.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/4204119961 shutdown_connections 2023-10-21T16:14:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.834+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/4204119961 wait complete. 2023-10-21T16:14:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.836+0000 7fb9e4e9f640 1 Processor -- start 2023-10-21T16:14:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.836+0000 7fb9e4e9f640 1 -- start start 2023-10-21T16:14:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e01599d0 0x7fb9e0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0159db0 0x7fb9e0157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0150d40 con 0x7fb9e00cc210 2023-10-21T16:14:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e014d2b0 con 0x7fb9e0159db0 2023-10-21T16:14:57.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9e4e9f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0151b60 con 0x7fb9e01599d0 2023-10-21T16:14:57.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36004/0 (socket says 172.21.15.53:36004) 2023-10-21T16:14:57.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.837+0000 7fb9df7fe640 1 -- 172.21.15.53:0/3555275653 learned_addr learned my addr 172.21.15.53:0/3555275653 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:57.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 -- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e01599d0 msgr2=0x7fb9e0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9deffd640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e01599d0 0x7fb9e0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e01599d0 0x7fb9e0151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 -- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0159db0 msgr2=0x7fb9e0157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:57.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0159db0 0x7fb9e0157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:57.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 -- 172.21.15.53:0/3555275653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009400 con 0x7fb9e00cc210 2023-10-21T16:14:57.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9df7fe640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002bee0 tx=0x7fb9d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d003ba70 con 0x7fb9e00cc210 2023-10-21T16:14:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d003bbf0 con 0x7fb9e00cc210 2023-10-21T16:14:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e0150f50 con 0x7fb9e00cc210 2023-10-21T16:14:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.838+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00453f0 con 0x7fb9e00cc210 2023-10-21T16:14:57.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.839+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e0151510 con 0x7fb9e00cc210 2023-10-21T16:14:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.840+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e004e160 con 0x7fb9e00cc210 2023-10-21T16:14:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.841+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb9d004e450 con 0x7fb9e00cc210 2023-10-21T16:14:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.842+0000 7fb9c7fff640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 0x7fb9cc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.842+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fb9d00abba0 con 0x7fb9e00cc210 2023-10-21T16:14:57.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.843+0000 7fb9deffd640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 0x7fb9cc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.844+0000 7fb9deffd640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 0x7fb9cc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0154c30 tx=0x7fb9d4008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:57.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:57.847+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9d00769b0 con 0x7fb9e00cc210 2023-10-21T16:14:58.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.095+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e00ccd50 con 0x7fb9e00cc210 2023-10-21T16:14:58.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.096+0000 7fb9c7fff640 1 -- 172.21.15.53:0/3555275653 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7fb9d0035090 con 0x7fb9e00cc210 2023-10-21T16:14:58.097 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:58.098 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:14:58.098 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:14:58.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.098+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 msgr2=0x7fb9cc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.098+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 0x7fb9cc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0154c30 tx=0x7fb9d4008040 comp rx=0 tx=0).stop 2023-10-21T16:14:58.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.098+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 msgr2=0x7fb9e014cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.098+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002bee0 tx=0x7fb9d00046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 shutdown_connections 2023-10-21T16:14:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb9cc06acc0 0x7fb9cc06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0159db0 0x7fb9e0157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e01599d0 0x7fb9e0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 --2- 172.21.15.53:0/3555275653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9e00cc210 0x7fb9e014cd70 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 >> 172.21.15.53:0/3555275653 conn(0x7fb9e00cb0c0 msgr2=0x7fb9e015d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:58.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 shutdown_connections 2023-10-21T16:14:58.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.099+0000 7fb9e4e9f640 1 -- 172.21.15.53:0/3555275653 wait complete. 2023-10-21T16:14:58.119 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:14:58.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.332+0000 7fa884820640 1 Processor -- start 2023-10-21T16:14:58.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.333+0000 7fa884820640 1 -- start start 2023-10-21T16:14:58.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa884820640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa880071010 0x7fa88015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa884820640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88015d7d0 con 0x7fa88015d380 2023-10-21T16:14:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa884820640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8800714a0 con 0x7fa880150440 2023-10-21T16:14:58.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa884820640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800c9f90 con 0x7fa880071010 2023-10-21T16:14:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa877fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa880071010 0x7fa88015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa877fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa880071010 0x7fa88015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36008/0 (socket says 172.21.15.53:36008) 2023-10-21T16:14:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa877fff640 1 -- 172.21.15.53:0/621593637 learned_addr learned my addr 172.21.15.53:0/621593637 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:58.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.335+0000 7fa87f59e640 1 --1- 172.21.15.53:0/621593637 >> v1:172.21.15.53:6789/0 conn(0x7fa88015d380 0x7fa8800cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37506/0 (socket says 172.21.15.53:37506) 2023-10-21T16:14:58.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.336+0000 7fa877fff640 1 -- 172.21.15.53:0/621593637 >> v1:172.21.15.124:6789/0 conn(0x7fa880150440 legacy=0x7fa8800c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:58.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.336+0000 7fa877fff640 1 -- 172.21.15.53:0/621593637 >> v1:172.21.15.53:6789/0 conn(0x7fa88015d380 legacy=0x7fa8800cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.336+0000 7fa877fff640 1 -- 172.21.15.53:0/621593637 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800ca270 con 0x7fa880071010 2023-10-21T16:14:58.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.336+0000 7fa87e59c640 1 -- 172.21.15.53:0/621593637 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645156444 0 0) 0x7fa88015d7d0 con 0x7fa88015d380 2023-10-21T16:14:58.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.337+0000 7fa877fff640 1 --2- 172.21.15.53:0/621593637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880071010 0x7fa88015ce40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bb00 tx=0x7fa86802df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44796769eb3de0c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.337+0000 7fa87e59c640 1 -- 172.21.15.53:0/621593637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86802ece0 con 0x7fa880071010 2023-10-21T16:14:58.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa87e59c640 1 -- 172.21.15.53:0/621593637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868032070 con 0x7fa880071010 2023-10-21T16:14:58.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa87e59c640 1 -- 172.21.15.53:0/621593637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868038620 con 0x7fa880071010 2023-10-21T16:14:58.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 -- 172.21.15.53:0/621593637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880071010 msgr2=0x7fa88015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 --2- 172.21.15.53:0/621593637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880071010 0x7fa88015ce40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bb00 tx=0x7fa86802df30 comp rx=0 tx=0).stop 2023-10-21T16:14:58.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 -- 172.21.15.53:0/621593637 shutdown_connections 2023-10-21T16:14:58.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 --2- 172.21.15.53:0/621593637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880071010 0x7fa88015ce40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 -- 172.21.15.53:0/621593637 >> 172.21.15.53:0/621593637 conn(0x7fa8800cb0c0 msgr2=0x7fa8800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:58.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.338+0000 7fa884820640 1 -- 172.21.15.53:0/621593637 shutdown_connections 2023-10-21T16:14:58.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.339+0000 7fa884820640 1 -- 172.21.15.53:0/621593637 wait complete. 2023-10-21T16:14:58.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.340+0000 7fa884820640 1 Processor -- start 2023-10-21T16:14:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.340+0000 7fa884820640 1 -- start start 2023-10-21T16:14:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa880071010 0x7fa8800c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa88015d380 0x7fa880148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800c72d0 con 0x7fa880150440 2023-10-21T16:14:58.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88014ac90 con 0x7fa88015d380 2023-10-21T16:14:58.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.341+0000 7fa884820640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8801491e0 con 0x7fa880071010 2023-10-21T16:14:58.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36024/0 (socket says 172.21.15.53:36024) 2023-10-21T16:14:58.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 -- 172.21.15.53:0/3113422571 learned_addr learned my addr 172.21.15.53:0/3113422571 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:58.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa87ed9d640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa880071010 0x7fa8800c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa87f59e640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa88015d380 0x7fa880148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 -- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa880071010 msgr2=0x7fa8800c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa880071010 0x7fa8800c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 -- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa88015d380 msgr2=0x7fa880148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa88015d380 0x7fa880148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.342+0000 7fa877fff640 1 -- 172.21.15.53:0/3113422571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86800b760 con 0x7fa880150440 2023-10-21T16:14:58.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa877fff640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bc20 tx=0x7fa8680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868042430 con 0x7fa880150440 2023-10-21T16:14:58.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8680425b0 con 0x7fa880150440 2023-10-21T16:14:58.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88014aea0 con 0x7fa880150440 2023-10-21T16:14:58.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868042960 con 0x7fa880150440 2023-10-21T16:14:58.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.343+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88014b460 con 0x7fa880150440 2023-10-21T16:14:58.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.344+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88004e160 con 0x7fa880150440 2023-10-21T16:14:58.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.345+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa868042ac0 con 0x7fa880150440 2023-10-21T16:14:58.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.346+0000 7fa8777fe640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 0x7fa86006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.346+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fa8680ac7d0 con 0x7fa880150440 2023-10-21T16:14:58.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.347+0000 7fa87ed9d640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 0x7fa86006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.348+0000 7fa87ed9d640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 0x7fa86006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa870004ea0 tx=0x7fa870009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.351+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8680774c0 con 0x7fa880150440 2023-10-21T16:14:58.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.520+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa868077890 con 0x7fa880150440 2023-10-21T16:14:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.551+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8800cae30 con 0x7fa880150440 2023-10-21T16:14:58.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.554+0000 7fa8777fe640 1 -- 172.21.15.53:0/3113422571 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+18671 (secure 0 0 0) 0x7fa86804e8f0 con 0x7fa880150440 2023-10-21T16:14:58.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:58.557 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:14:57.467887+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":219,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/810479840":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/20933058":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/3285509161":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2020694858":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/269363197":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/1953414266":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2311155348":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2484781998":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6832/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/3629698444":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2591851543":"2023-10-22T16:14:56.458396+0000","172.21.15.124:6832/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/3840977991":"2023-10-22T16:14:56.458396+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":7,"length":3},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":18,"length":1},{"begin":20,"length":2},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":2},{"begin":36,"length":1},{"begin":38,"length":1},{"begin":42,"length":2},{"begin":46,"length":2},{"begin":50,"length":1},{"begin":54,"length":3},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":68,"length":3},{"begin":72,"length":2},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":87,"length":4},{"begin":95,"length":4},{"begin":100,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":107,"length":2},{"begin":110,"length":1},{"begin":168,"length":52}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:14:58.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 msgr2=0x7fa86006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 0x7fa86006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa870004ea0 tx=0x7fa870009040 comp rx=0 tx=0).stop 2023-10-21T16:14:58.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 msgr2=0x7fa88014a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bc20 tx=0x7fa8680046c0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 shutdown_connections 2023-10-21T16:14:58.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa86006acc0 0x7fa86006d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa88015d380 0x7fa880148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa880150440 0x7fa88014a750 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 --2- 172.21.15.53:0/3113422571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa880071010 0x7fa8800c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.557+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 >> 172.21.15.53:0/3113422571 conn(0x7fa8800cb0c0 msgr2=0x7fa88015a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.558+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 shutdown_connections 2023-10-21T16:14:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.558+0000 7fa884820640 1 -- 172.21.15.53:0/3113422571 wait complete. 2023-10-21T16:14:58.577 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:14:58.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:14:58.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.789+0000 7f5f95967640 1 Processor -- start 2023-10-21T16:14:58.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.789+0000 7f5f95967640 1 -- start start 2023-10-21T16:14:58.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.790+0000 7f5f95967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f90167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.790+0000 7f5f95967640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9015d040 con 0x7f5f900d16f0 2023-10-21T16:14:58.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.791+0000 7f5f95967640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900d0650 con 0x7f5f9015cc60 2023-10-21T16:14:58.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.791+0000 7f5f95967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900d0930 con 0x7f5f900cfa50 2023-10-21T16:14:58.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.791+0000 7f5f8ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5f900d16f0 0x7f5f900cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37512/0 (socket says 172.21.15.53:37512) 2023-10-21T16:14:58.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.791+0000 7f5f8ffff640 1 -- 172.21.15.53:0/929686563 learned_addr learned my addr 172.21.15.53:0/929686563 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.791+0000 7f5f95166640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f90167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447574207 0 0) 0x7f5f9015d040 con 0x7f5f900d16f0 2023-10-21T16:14:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f80003520 con 0x7f5f900d16f0 2023-10-21T16:14:58.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869036428 0 0) 0x7f5f80003520 con 0x7f5f900d16f0 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 msgr2=0x7f5f90167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f90167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 >> v1:172.21.15.124:6789/0 conn(0x7f5f9015cc60 legacy=0x7f5f900cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900d0c10 con 0x7f5f900d16f0 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5f7c001670 con 0x7f5f900d16f0 2023-10-21T16:14:58.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 >> v1:172.21.15.53:6789/0 conn(0x7f5f900d16f0 legacy=0x7f5f900cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800039a0 0x7f5f80003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f800047c0 0x7f5f80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900d0930 con 0x7f5f800047c0 2023-10-21T16:14:58.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8000ba30 con 0x7f5f800039a0 2023-10-21T16:14:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.792+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8000bba0 con 0x7f5f900cfa50 2023-10-21T16:14:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f800047c0 0x7f5f80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f95166640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f94965640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800039a0 0x7f5f80003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 -- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 msgr2=0x7f5f80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:58.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f900cfa50 0x7f5f80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 -- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800039a0 msgr2=0x7f5f80003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:58.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800039a0 0x7f5f80003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f8ffff640 1 -- 172.21.15.53:0/929686563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80003520 con 0x7f5f800047c0 2023-10-21T16:14:58.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.793+0000 7f5f94965640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800039a0 0x7f5f80003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:58.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.794+0000 7f5f8ffff640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f800047c0 0x7f5f80004c40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c160 tx=0x7f5f800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c08543e3d856882 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.794+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c005ad0 con 0x7f5f800047c0 2023-10-21T16:14:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c005c30 con 0x7f5f800047c0 2023-10-21T16:14:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f8f7fe640 1 -- 172.21.15.53:0/929686563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c02c790 con 0x7f5f800047c0 2023-10-21T16:14:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 -- 172.21.15.53:0/929686563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f800047c0 msgr2=0x7f5f80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 --2- 172.21.15.53:0/929686563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f800047c0 0x7f5f80004c40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c160 tx=0x7f5f800057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95166640 1 -- 172.21.15.53:0/929686563 reap_dead start 2023-10-21T16:14:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 -- 172.21.15.53:0/929686563 shutdown_connections 2023-10-21T16:14:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 -- 172.21.15.53:0/929686563 >> 172.21.15.53:0/929686563 conn(0x7f5f9009bcf0 msgr2=0x7f5f900d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 -- 172.21.15.53:0/929686563 shutdown_connections 2023-10-21T16:14:58.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.795+0000 7f5f95967640 1 -- 172.21.15.53:0/929686563 wait complete. 2023-10-21T16:14:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.797+0000 7f5f95967640 1 Processor -- start 2023-10-21T16:14:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.797+0000 7f5f95967640 1 -- start start 2023-10-21T16:14:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800047c0 0x7f5f900c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f90151bd0 0x7f5f900ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900ccbe0 con 0x7f5f900cc1f0 2023-10-21T16:14:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900ca990 con 0x7f5f800047c0 2023-10-21T16:14:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.798+0000 7f5f95967640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cac30 con 0x7f5f90151bd0 2023-10-21T16:14:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36040/0 (socket says 172.21.15.53:36040) 2023-10-21T16:14:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 -- 172.21.15.53:0/3574163368 learned_addr learned my addr 172.21.15.53:0/3574163368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f94965640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800047c0 0x7f5f900c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f8ffff640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f90151bd0 0x7f5f900ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 -- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f90151bd0 msgr2=0x7f5f900ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f90151bd0 0x7f5f900ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 -- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800047c0 msgr2=0x7f5f900c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800047c0 0x7f5f900c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 -- 172.21.15.53:0/3574163368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900caf10 con 0x7f5f900cc1f0 2023-10-21T16:14:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.799+0000 7f5f95166640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5f900cfe60 tx=0x7f5f880115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.800+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800e070 con 0x7f5f900cc1f0 2023-10-21T16:14:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.800+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88011e10 con 0x7f5f900cc1f0 2023-10-21T16:14:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.800+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88003990 con 0x7f5f900cc1f0 2023-10-21T16:14:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.800+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f900cb1f0 con 0x7f5f900cc1f0 2023-10-21T16:14:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.800+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90152050 con 0x7f5f900cc1f0 2023-10-21T16:14:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.801+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9004e160 con 0x7f5f900cc1f0 2023-10-21T16:14:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.802+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5f88003af0 con 0x7f5f900cc1f0 2023-10-21T16:14:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.803+0000 7f5f8d7fa640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 0x7f5f7406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.803+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f5f8808a4a0 con 0x7f5f900cc1f0 2023-10-21T16:14:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.804+0000 7f5f94965640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 0x7f5f7406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.805+0000 7f5f94965640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 0x7f5f7406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001900 tx=0x7f5f84009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:58.808+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f88054db0 con 0x7f5f900cc1f0 2023-10-21T16:14:59.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.038+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f900d3080 con 0x7f5f900cc1f0 2023-10-21T16:14:59.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.039+0000 7f5f8d7fa640 1 -- 172.21.15.53:0/3574163368 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f5f880583d0 con 0x7f5f900cc1f0 2023-10-21T16:14:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:59.042 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:14:59.042 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:14:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 msgr2=0x7f5f7406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 0x7f5f7406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001900 tx=0x7f5f84009040 comp rx=0 tx=0).stop 2023-10-21T16:14:59.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 msgr2=0x7f5f900cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5f900cfe60 tx=0x7f5f880115a0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 shutdown_connections 2023-10-21T16:14:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5f7406af00 0x7f5f7406d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.042+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f900cc1f0 0x7f5f900cc6a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.043+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f90151bd0 0x7f5f900ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.043+0000 7f5f95967640 1 --2- 172.21.15.53:0/3574163368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f800047c0 0x7f5f900c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.043+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 >> 172.21.15.53:0/3574163368 conn(0x7f5f9009bcf0 msgr2=0x7f5f900d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.043+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 shutdown_connections 2023-10-21T16:14:59.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.043+0000 7f5f95967640 1 -- 172.21.15.53:0/3574163368 wait complete. 2023-10-21T16:14:59.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:14:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.278+0000 7f5ef714e640 1 Processor -- start 2023-10-21T16:14:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.279+0000 7f5ef714e640 1 -- start start 2023-10-21T16:14:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.280+0000 7f5ef714e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ef01599d0 0x7f5ef00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef714e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00cacf0 con 0x7f5ef00b07d0 2023-10-21T16:14:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef714e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00ca3c0 con 0x7f5ef00cc210 2023-10-21T16:14:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef714e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef00b10f0 con 0x7f5ef01599d0 2023-10-21T16:14:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef614c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ef01599d0 0x7f5ef00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef694d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5ef00b07d0 0x7f5ef006c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37524/0 (socket says 172.21.15.53:37524) 2023-10-21T16:14:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef694d640 1 -- 172.21.15.53:0/1745139228 learned_addr learned my addr 172.21.15.53:0/1745139228 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef614c640 1 -- 172.21.15.53:0/1745139228 >> v1:172.21.15.124:6789/0 conn(0x7f5ef00cc210 legacy=0x7f5ef00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.281+0000 7f5ef614c640 1 -- 172.21.15.53:0/1745139228 >> v1:172.21.15.53:6789/0 conn(0x7f5ef00b07d0 legacy=0x7f5ef006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.282+0000 7f5ef614c640 1 -- 172.21.15.53:0/1745139228 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00ca6a0 con 0x7f5ef01599d0 2023-10-21T16:14:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.283+0000 7f5ef614c640 1 --2- 172.21.15.53:0/1745139228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef01599d0 0x7f5ef00ad730 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0097a0 tx=0x7f5eec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd63f08784a5b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.283+0000 7f5ef514a640 1 -- 172.21.15.53:0/1745139228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec031830 con 0x7f5ef01599d0 2023-10-21T16:14:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.283+0000 7f5ef514a640 1 -- 172.21.15.53:0/1745139228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec031990 con 0x7f5ef01599d0 2023-10-21T16:14:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.283+0000 7f5ef514a640 1 -- 172.21.15.53:0/1745139228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec031d20 con 0x7f5ef01599d0 2023-10-21T16:14:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 -- 172.21.15.53:0/1745139228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef01599d0 msgr2=0x7f5ef00ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 --2- 172.21.15.53:0/1745139228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef01599d0 0x7f5ef00ad730 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0097a0 tx=0x7f5eec02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 -- 172.21.15.53:0/1745139228 shutdown_connections 2023-10-21T16:14:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 --2- 172.21.15.53:0/1745139228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef01599d0 0x7f5ef00ad730 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 -- 172.21.15.53:0/1745139228 >> 172.21.15.53:0/1745139228 conn(0x7f5ef00cb0c0 msgr2=0x7f5ef00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 -- 172.21.15.53:0/1745139228 shutdown_connections 2023-10-21T16:14:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.284+0000 7f5ef714e640 1 -- 172.21.15.53:0/1745139228 wait complete. 2023-10-21T16:14:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.286+0000 7f5ef714e640 1 Processor -- start 2023-10-21T16:14:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.286+0000 7f5ef714e640 1 -- start start 2023-10-21T16:14:59.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.287+0000 7f5ef714e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 0x7f5ef014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.287+0000 7f5ef714e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.287+0000 7f5ef714e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ef01599d0 0x7f5ef00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef714e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef015b600 con 0x7f5ef00cc210 2023-10-21T16:14:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef714e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef01f0e60 con 0x7f5ef00b07d0 2023-10-21T16:14:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef714e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef015ab50 con 0x7f5ef01599d0 2023-10-21T16:14:59.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36066/0 (socket says 172.21.15.53:36066) 2023-10-21T16:14:59.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 -- 172.21.15.53:0/770719638 learned_addr learned my addr 172.21.15.53:0/770719638 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:59.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef614c640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 0x7f5ef014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 -- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ef01599d0 msgr2=0x7f5ef00cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef694d640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ef01599d0 0x7f5ef00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ef01599d0 0x7f5ef00cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 -- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 msgr2=0x7f5ef014da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 0x7f5ef014da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef594b640 1 -- 172.21.15.53:0/770719638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009400 con 0x7f5ef00cc210 2023-10-21T16:14:59.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.288+0000 7f5ef614c640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 0x7f5ef014da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:59.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ef594b640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c430 tx=0x7f5ee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011790 con 0x7f5ef00cc210 2023-10-21T16:14:59.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee00118f0 con 0x7f5ef00cc210 2023-10-21T16:14:59.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011ce0 con 0x7f5ef00cc210 2023-10-21T16:14:59.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef015b850 con 0x7f5ef00cc210 2023-10-21T16:14:59.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.289+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef015be10 con 0x7f5ef00cc210 2023-10-21T16:14:59.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.290+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef004e160 con 0x7f5ef00cc210 2023-10-21T16:14:59.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.291+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5ee001c460 con 0x7f5ef00cc210 2023-10-21T16:14:59.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.292+0000 7f5ee6ffd640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 0x7f5ed406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.292+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f5ee0089ae0 con 0x7f5ef00cc210 2023-10-21T16:14:59.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.293+0000 7f5ef614c640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 0x7f5ed406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.294+0000 7f5ef614c640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 0x7f5ed406d410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5eec036000 tx=0x7f5eec008030 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.297+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ee0054680 con 0x7f5ef00cc210 2023-10-21T16:14:59.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.528+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef00b3dc0 con 0x7f5ef00cc210 2023-10-21T16:14:59.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.529+0000 7f5ee6ffd640 1 -- 172.21.15.53:0/770719638 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f5ee0057ca0 con 0x7f5ef00cc210 2023-10-21T16:14:59.531 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:59.531 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:03:35.695582+0000","modified":"2023-10-21T16:03:38.789790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34679":{"gid":34679,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6833/1236966553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1236966553},{"type":"v1","addr":"172.21.15.53:6833","nonce":1236966553}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-21T16:14:59.531 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 18 2023-10-21T16:14:59.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.531+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 msgr2=0x7f5ed406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.531+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 0x7f5ed406d410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5eec036000 tx=0x7f5eec008030 comp rx=0 tx=0).stop 2023-10-21T16:14:59.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.531+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 msgr2=0x7f5ef00ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.531+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c430 tx=0x7f5ee000c8f0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.531+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 shutdown_connections 2023-10-21T16:14:59.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5ed406af50 0x7f5ed406d410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ef01599d0 0x7f5ef00cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ef00cc210 0x7f5ef00ce710 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 --2- 172.21.15.53:0/770719638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ef00b07d0 0x7f5ef014da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 >> 172.21.15.53:0/770719638 conn(0x7f5ef00cb0c0 msgr2=0x7f5ef01579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:59.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 shutdown_connections 2023-10-21T16:14:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.532+0000 7f5ef714e640 1 -- 172.21.15.53:0/770719638 wait complete. 2023-10-21T16:14:59.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:14:59.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.772+0000 7f14910bb640 1 Processor -- start 2023-10-21T16:14:59.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.772+0000 7f14910bb640 1 -- start start 2023-10-21T16:14:59.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f14910bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f148c0b07d0 0x7f148c165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f14910bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c15f7d0 con 0x7f148c0adc70 2023-10-21T16:14:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f14910bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148c15efb0 con 0x7f148c15ab30 2023-10-21T16:14:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f14910bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148c15f210 con 0x7f148c0b07d0 2023-10-21T16:14:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f1483fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f148c0adc70 0x7f148c0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37538/0 (socket says 172.21.15.53:37538) 2023-10-21T16:14:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f1483fff640 1 -- 172.21.15.53:0/2323311155 learned_addr learned my addr 172.21.15.53:0/2323311155 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:14:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f14908ba640 1 --2- 172.21.15.53:0/2323311155 >> v2:172.21.15.124:3300/0 conn(0x7f148c0b07d0 0x7f148c165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.774+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900441614 0 0) 0x7f148c15f7d0 con 0x7f148c0adc70 2023-10-21T16:14:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1478003520 con 0x7f148c0adc70 2023-10-21T16:14:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443324174 0 0) 0x7f1478003520 con 0x7f148c0adc70 2023-10-21T16:14:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 >> v2:172.21.15.124:3300/0 conn(0x7f148c0b07d0 msgr2=0x7f148c165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 --2- 172.21.15.53:0/2323311155 >> v2:172.21.15.124:3300/0 conn(0x7f148c0b07d0 0x7f148c165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 >> v1:172.21.15.124:6789/0 conn(0x7f148c15ab30 legacy=0x7f148c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c15f4f0 con 0x7f148c0adc70 2023-10-21T16:14:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1474001670 con 0x7f148c0adc70 2023-10-21T16:14:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 >> v1:172.21.15.53:6789/0 conn(0x7f148c0adc70 legacy=0x7f148c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 0x7f1478003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780047c0 0x7f1478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1478005180 0x7f147800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c15f210 con 0x7f14780039c0 2023-10-21T16:14:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147800c080 con 0x7f14780047c0 2023-10-21T16:14:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147800c240 con 0x7f1478005180 2023-10-21T16:14:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.775+0000 7f148bfff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 0x7f1478003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f1483fff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780047c0 0x7f1478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f148bfff640 1 -- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1478005180 msgr2=0x7f147800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f148bfff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1478005180 0x7f147800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f148bfff640 1 -- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780047c0 msgr2=0x7f1478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:14:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f148bfff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780047c0 0x7f1478004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f148bfff640 1 -- 172.21.15.53:0/2323311155 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478003520 con 0x7f14780039c0 2023-10-21T16:14:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f1483fff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780047c0 0x7f1478004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:59.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.776+0000 7f14908ba640 1 -- 172.21.15.53:0/2323311155 reap_dead start 2023-10-21T16:14:59.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.777+0000 7f148bfff640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 0x7f1478003de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f147c001720 tx=0x7f14780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e2921f2c482e10 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.777+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0052d0 con 0x7f14780039c0 2023-10-21T16:14:59.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.777+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c02e050 con 0x7f14780039c0 2023-10-21T16:14:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.777+0000 7f148b7fe640 1 -- 172.21.15.53:0/2323311155 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c003830 con 0x7f14780039c0 2023-10-21T16:14:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 -- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 msgr2=0x7f1478003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 0x7f1478003de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f147c001720 tx=0x7f14780057f0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 -- 172.21.15.53:0/2323311155 shutdown_connections 2023-10-21T16:14:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 --2- 172.21.15.53:0/2323311155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14780039c0 0x7f1478003de0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 -- 172.21.15.53:0/2323311155 >> 172.21.15.53:0/2323311155 conn(0x7f148c0cb0c0 msgr2=0x7f148c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:59.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 -- 172.21.15.53:0/2323311155 shutdown_connections 2023-10-21T16:14:59.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.778+0000 7f14910bb640 1 -- 172.21.15.53:0/2323311155 wait complete. 2023-10-21T16:14:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.780+0000 7f14910bb640 1 Processor -- start 2023-10-21T16:14:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.780+0000 7f14910bb640 1 -- start start 2023-10-21T16:14:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780039c0 0x7f148c14e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 0x7f148c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0cd6c0 con 0x7f148c0aca90 2023-10-21T16:14:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0cd9a0 con 0x7f14780039c0 2023-10-21T16:14:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14910bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148c0c57b0 con 0x7f148c14eed0 2023-10-21T16:14:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14908ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14908ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36088/0 (socket says 172.21.15.53:36088) 2023-10-21T16:14:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f14908ba640 1 -- 172.21.15.53:0/546782145 learned_addr learned my addr 172.21.15.53:0/546782145 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:14:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.781+0000 7f148bfff640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780039c0 0x7f148c14e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f1483fff640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 0x7f148c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 -- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 msgr2=0x7f148c0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 0x7f148c0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 -- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780039c0 msgr2=0x7f148c14e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780039c0 0x7f148c14e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 -- 172.21.15.53:0/546782145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0c5a90 con 0x7f148c0aca90 2023-10-21T16:14:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f1483fff640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 0x7f148c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:14:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14908ba640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f148c0b4730 tx=0x7f148400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484010040 con 0x7f148c0aca90 2023-10-21T16:14:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148400ba80 con 0x7f148c0aca90 2023-10-21T16:14:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148400be10 con 0x7f148c0aca90 2023-10-21T16:14:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.782+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148c0c5d70 con 0x7f148c0aca90 2023-10-21T16:14:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.783+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c0acf40 con 0x7f148c0aca90 2023-10-21T16:14:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.784+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148c04e160 con 0x7f148c0aca90 2023-10-21T16:14:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.785+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f148401f460 con 0x7f148c0aca90 2023-10-21T16:14:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.785+0000 7f14897fa640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 0x7f146c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:14:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.786+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f148408a180 con 0x7f148c0aca90 2023-10-21T16:14:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.786+0000 7f148bfff640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 0x7f146c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:14:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.787+0000 7f148bfff640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 0x7f146c06d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f147c033000 tx=0x7f147c009030 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:14:59.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.790+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1484054a00 con 0x7f148c0aca90 2023-10-21T16:14:59.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.984+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f148c0ada50 con 0x7f148c0aca90 2023-10-21T16:14:59.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.988+0000 7f14897fa640 1 -- 172.21.15.53:0/546782145 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+18671 (secure 0 0 0) 0x7f1484058020 con 0x7f148c0aca90 2023-10-21T16:14:59.989 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:14:59.990 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":60,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:14:57.467887+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:03:33.797436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-21T16:03:34.721730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":219,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/810479840":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/20933058":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/3285509161":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2020694858":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/269363197":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/1953414266":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2311155348":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2484781998":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6832/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/3629698444":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2591851543":"2023-10-22T16:14:56.458396+0000","172.21.15.124:6832/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/3840977991":"2023-10-22T16:14:56.458396+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1},{"begin":7,"length":3},{"begin":12,"length":1},{"begin":15,"length":1},{"begin":18,"length":1},{"begin":20,"length":2},{"begin":24,"length":1},{"begin":26,"length":1},{"begin":29,"length":1},{"begin":31,"length":1},{"begin":33,"length":2},{"begin":36,"length":1},{"begin":38,"length":1},{"begin":42,"length":2},{"begin":46,"length":2},{"begin":50,"length":1},{"begin":54,"length":3},{"begin":59,"length":3},{"begin":63,"length":1},{"begin":68,"length":3},{"begin":72,"length":2},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":83,"length":1},{"begin":87,"length":4},{"begin":95,"length":4},{"begin":100,"length":1},{"begin":102,"length":1},{"begin":105,"length":1},{"begin":107,"length":2},{"begin":110,"length":1},{"begin":168,"length":52}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:14:59.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.990+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 msgr2=0x7f146c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.990+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 0x7f146c06d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f147c033000 tx=0x7f147c009030 comp rx=0 tx=0).stop 2023-10-21T16:14:59.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.990+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 msgr2=0x7f148c0c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:14:59.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.990+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f148c0b4730 tx=0x7f148400b480 comp rx=0 tx=0).stop 2023-10-21T16:14:59.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 shutdown_connections 2023-10-21T16:14:59.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f146c06af00 0x7f146c06d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f148c0aca90 0x7f148c0c2130 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f148c14eed0 0x7f148c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 --2- 172.21.15.53:0/546782145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14780039c0 0x7f148c14e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:14:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 >> 172.21.15.53:0/546782145 conn(0x7f148c0cb0c0 msgr2=0x7f148c0ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:14:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 shutdown_connections 2023-10-21T16:14:59.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:14:59.991+0000 7f14910bb640 1 -- 172.21.15.53:0/546782145 wait complete. 2023-10-21T16:15:00.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:15:00.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.235+0000 7f872d6ad640 1 Processor -- start 2023-10-21T16:15:00.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.235+0000 7f872d6ad640 1 -- start start 2023-10-21T16:15:00.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.236+0000 7f872d6ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.236+0000 7f872d6ad640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87280cb5e0 0x7f87280cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f872d6ad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8728159db0 con 0x7f87280b3c50 2023-10-21T16:15:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f872d6ad640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8728071fe0 con 0x7f87280cb5e0 2023-10-21T16:15:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f872d6ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b44c0 con 0x7f87281599d0 2023-10-21T16:15:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f872ceac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87280cb5e0 0x7f87280cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f871ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f87280b3c50 0x7f87280caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37542/0 (socket says 172.21.15.53:37542) 2023-10-21T16:15:00.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f872ceac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87280cb5e0 0x7f87280cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36102/0 (socket says 172.21.15.53:36102) 2023-10-21T16:15:00.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f871ffff640 1 -- 172.21.15.53:0/1214201595 learned_addr learned my addr 172.21.15.53:0/1214201595 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:00.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.237+0000 7f8727fff640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f872ceac640 1 -- 172.21.15.53:0/1214201595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 msgr2=0x7f87280cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:00.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f872ceac640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f872ceac640 1 -- 172.21.15.53:0/1214201595 >> v1:172.21.15.53:6789/0 conn(0x7f87280b3c50 legacy=0x7f87280caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f872ceac640 1 -- 172.21.15.53:0/1214201595 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280713d0 con 0x7f87280cb5e0 2023-10-21T16:15:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f8727fff640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:15:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.238+0000 7f87277fe640 1 -- 172.21.15.53:0/1214201595 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608603069 0 0) 0x7f8710001110 con 0x7f87280b3c50 2023-10-21T16:15:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.239+0000 7f872ceac640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280cc0c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8720009d90 tx=0x7f872002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9a7e589353e5f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.239+0000 7f87277fe640 1 -- 172.21.15.53:0/1214201595 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872002cc50 con 0x7f87280cb5e0 2023-10-21T16:15:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.239+0000 7f87277fe640 1 -- 172.21.15.53:0/1214201595 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872002cdb0 con 0x7f87280cb5e0 2023-10-21T16:15:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.239+0000 7f87277fe640 1 -- 172.21.15.53:0/1214201595 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720034660 con 0x7f87280cb5e0 2023-10-21T16:15:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 -- 172.21.15.53:0/1214201595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 msgr2=0x7f87280cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280cc0c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8720009d90 tx=0x7f872002bea0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 -- 172.21.15.53:0/1214201595 shutdown_connections 2023-10-21T16:15:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280cc0c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 --2- 172.21.15.53:0/1214201595 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 -- 172.21.15.53:0/1214201595 >> 172.21.15.53:0/1214201595 conn(0x7f872806c050 msgr2=0x7f872806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 -- 172.21.15.53:0/1214201595 shutdown_connections 2023-10-21T16:15:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.240+0000 7f872d6ad640 1 -- 172.21.15.53:0/1214201595 wait complete. 2023-10-21T16:15:00.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.242+0000 7f872d6ad640 1 Processor -- start 2023-10-21T16:15:00.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.242+0000 7f872d6ad640 1 -- start start 2023-10-21T16:15:00.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87280b3c50 0x7f87280b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280c55c0 con 0x7f87280cb5e0 2023-10-21T16:15:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8728155460 con 0x7f87280b3c50 2023-10-21T16:15:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.243+0000 7f872d6ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b2df0 con 0x7f87281599d0 2023-10-21T16:15:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36106/0 (socket says 172.21.15.53:36106) 2023-10-21T16:15:00.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 -- 172.21.15.53:0/2336547481 learned_addr learned my addr 172.21.15.53:0/2336547481 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:00.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f872ceac640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f8727fff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87280b3c50 0x7f87280b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 -- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 msgr2=0x7f87280b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 -- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87280b3c50 msgr2=0x7f87280b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87280b3c50 0x7f87280b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 -- 172.21.15.53:0/2336547481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720009a70 con 0x7f87280cb5e0 2023-10-21T16:15:00.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.244+0000 7f871ffff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f871000c780 tx=0x7f871000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.245+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87100072d0 con 0x7f87280cb5e0 2023-10-21T16:15:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.245+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710007450 con 0x7f87280cb5e0 2023-10-21T16:15:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.245+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87100047e0 con 0x7f87280cb5e0 2023-10-21T16:15:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.245+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8728158530 con 0x7f87280cb5e0 2023-10-21T16:15:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.245+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87281556b0 con 0x7f87280cb5e0 2023-10-21T16:15:00.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.246+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87281558a0 con 0x7f87280cb5e0 2023-10-21T16:15:00.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.247+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8710004960 con 0x7f87280cb5e0 2023-10-21T16:15:00.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.248+0000 7f87257fa640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 0x7f870806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.248+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f8710088ce0 con 0x7f87280cb5e0 2023-10-21T16:15:00.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.249+0000 7f8727fff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 0x7f870806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.250+0000 7f8727fff640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 0x7f870806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87280b0cc0 tx=0x7f8718008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.252+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8710053700 con 0x7f87280cb5e0 2023-10-21T16:15:00.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.486+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8728159db0 con 0x7f87280cb5e0 2023-10-21T16:15:00.488 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:15:00.487+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:15:00.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.490+0000 7f87257fa640 1 -- 172.21.15.53:0/2336547481 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8728159db0 con 0x7f87280cb5e0 2023-10-21T16:15:00.492 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:15:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 msgr2=0x7f870806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 0x7f870806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87280b0cc0 tx=0x7f8718008040 comp rx=0 tx=0).stop 2023-10-21T16:15:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 msgr2=0x7f87280b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f871000c780 tx=0x7f871000cc40 comp rx=0 tx=0).stop 2023-10-21T16:15:00.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 shutdown_connections 2023-10-21T16:15:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f870806af30 0x7f870806d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87281599d0 0x7f87280b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87280cb5e0 0x7f87280b28b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 --2- 172.21.15.53:0/2336547481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87280b3c50 0x7f87280b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 >> 172.21.15.53:0/2336547481 conn(0x7f872806c050 msgr2=0x7f87280ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:00.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 shutdown_connections 2023-10-21T16:15:00.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.494+0000 7f872d6ad640 1 -- 172.21.15.53:0/2336547481 wait complete. 2023-10-21T16:15:00.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:15:00.585 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:15:00.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.730+0000 7f5171b90640 1 Processor -- start 2023-10-21T16:15:00.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.730+0000 7f5171b90640 1 -- start start 2023-10-21T16:15:00.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5171b90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5171b90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f516c06dfc0 0x7f516c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5171b90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c0b2620 con 0x7f516c071070 2023-10-21T16:15:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5171b90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c0b2060 con 0x7f516c06dfc0 2023-10-21T16:15:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5171b90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0b2340 con 0x7f516c0afbe0 2023-10-21T16:15:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f516bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f516c071070 0x7f516c0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37556/0 (socket says 172.21.15.53:37556) 2023-10-21T16:15:00.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f516bfff640 1 -- 172.21.15.53:0/3771885850 learned_addr learned my addr 172.21.15.53:0/3771885850 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:00.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f5162ffd640 1 --2- 172.21.15.53:0/3771885850 >> v2:172.21.15.53:3300/0 conn(0x7f516c06dfc0 0x7f516c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.732+0000 7f516b7fe640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f516affd640 1 -- 172.21.15.53:0/3771885850 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639668594 0 0) 0x7f516c0b2620 con 0x7f516c071070 2023-10-21T16:15:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f516affd640 1 -- 172.21.15.53:0/3771885850 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5158003520 con 0x7f516c071070 2023-10-21T16:15:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f5162ffd640 1 -- 172.21.15.53:0/3771885850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 msgr2=0x7f516c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f5162ffd640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f5162ffd640 1 -- 172.21.15.53:0/3771885850 >> v1:172.21.15.53:6789/0 conn(0x7f516c071070 legacy=0x7f516c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.733+0000 7f5162ffd640 1 -- 172.21.15.53:0/3771885850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c070350 con 0x7f516c06dfc0 2023-10-21T16:15:00.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.734+0000 7f5162ffd640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c06dfc0 0x7f516c070b30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f51540097a0 tx=0x7f515402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953451c6e6fadd1d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.734+0000 7f516affd640 1 -- 172.21.15.53:0/3771885850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515402d040 con 0x7f516c06dfc0 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.734+0000 7f516affd640 1 -- 172.21.15.53:0/3771885850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5154031810 con 0x7f516c06dfc0 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.734+0000 7f516affd640 1 -- 172.21.15.53:0/3771885850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154031ba0 con 0x7f516c06dfc0 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 -- 172.21.15.53:0/3771885850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c06dfc0 msgr2=0x7f516c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c06dfc0 0x7f516c070b30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f51540097a0 tx=0x7f515402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 -- 172.21.15.53:0/3771885850 shutdown_connections 2023-10-21T16:15:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c06dfc0 0x7f516c070b30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 --2- 172.21.15.53:0/3771885850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 -- 172.21.15.53:0/3771885850 >> 172.21.15.53:0/3771885850 conn(0x7f516c06a5d0 msgr2=0x7f516c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 -- 172.21.15.53:0/3771885850 shutdown_connections 2023-10-21T16:15:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.735+0000 7f5171b90640 1 -- 172.21.15.53:0/3771885850 wait complete. 2023-10-21T16:15:00.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.737+0000 7f5171b90640 1 Processor -- start 2023-10-21T16:15:00.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.737+0000 7f5171b90640 1 -- start start 2023-10-21T16:15:00.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f516c06dfc0 0x7f516c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0c8eb0 con 0x7f516c071070 2023-10-21T16:15:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0cc770 con 0x7f516c06dfc0 2023-10-21T16:15:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5171b90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c0cca50 con 0x7f516c0afbe0 2023-10-21T16:15:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5162ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5162ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36140/0 (socket says 172.21.15.53:36140) 2023-10-21T16:15:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.738+0000 7f5162ffd640 1 -- 172.21.15.53:0/4056142359 learned_addr learned my addr 172.21.15.53:0/4056142359 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f516bfff640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f516b7fe640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f516c06dfc0 0x7f516c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 -- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 msgr2=0x7f516c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 -- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f516c06dfc0 msgr2=0x7f516c14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:00.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f516c06dfc0 0x7f516c14ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:00.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 -- 172.21.15.53:0/4056142359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009400 con 0x7f516c071070 2023-10-21T16:15:00.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f516bfff640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:00.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5162ffd640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154006e10 con 0x7f516c071070 2023-10-21T16:15:00.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515403ad60 con 0x7f516c071070 2023-10-21T16:15:00.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154044630 con 0x7f516c071070 2023-10-21T16:15:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.739+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c0ccd30 con 0x7f516c071070 2023-10-21T16:15:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.740+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c0cee10 con 0x7f516c071070 2023-10-21T16:15:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.741+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c0cd250 con 0x7f516c071070 2023-10-21T16:15:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.742+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f515404d450 con 0x7f516c071070 2023-10-21T16:15:00.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.742+0000 7f5168ff9640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 0x7f514c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:00.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.743+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7329+0+0 (secure 0 0 0) 0x7f51540ac0f0 con 0x7f516c071070 2023-10-21T16:15:00.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.743+0000 7f516b7fe640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 0x7f514c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:00.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.744+0000 7f516b7fe640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 0x7f514c06d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:00.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.747+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f516c0cd250 con 0x7f516c071070 2023-10-21T16:15:00.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:00.983+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f516c0cd250 con 0x7f516c071070 2023-10-21T16:15:01.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.531+0000 7f5168ff9640 1 -- 172.21.15.53:0/4056142359 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f516c0cd250 con 0x7f516c071070 2023-10-21T16:15:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.534+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 msgr2=0x7f514c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.534+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 0x7f514c06d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f515c006fd0 tx=0x7f515c008040 comp rx=0 tx=0).stop 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.534+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 msgr2=0x7f516c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.534+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5154000c00 tx=0x7f51540046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 shutdown_connections 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f514c06af90 0x7f514c06d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f516c0afbe0 0x7f516c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f516c071070 0x7f516c157940 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 --2- 172.21.15.53:0/4056142359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f516c06dfc0 0x7f516c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 >> 172.21.15.53:0/4056142359 conn(0x7f516c06a5d0 msgr2=0x7f516c0c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:01.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 shutdown_connections 2023-10-21T16:15:01.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.535+0000 7f5171b90640 1 -- 172.21.15.53:0/4056142359 wait complete. 2023-10-21T16:15:01.560 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:15:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.769+0000 7f081a14e640 1 Processor -- start 2023-10-21T16:15:01.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.769+0000 7f081a14e640 1 -- start start 2023-10-21T16:15:01.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081a14e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f081415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081a14e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08140cab60 con 0x7f08140b0680 2023-10-21T16:15:01.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081a14e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08140cb960 con 0x7f081415a820 2023-10-21T16:15:01.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081a14e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08140cbdf0 con 0x7f08140ca680 2023-10-21T16:15:01.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081894b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f08140b0680 0x7f081415e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37572/0 (socket says 172.21.15.53:37572) 2023-10-21T16:15:01.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.771+0000 7f081894b640 1 -- 172.21.15.53:0/3038515815 learned_addr learned my addr 172.21.15.53:0/3038515815 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:01.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f081994d640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f081415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215319989 0 0) 0x7f08140cab60 con 0x7f08140b0680 2023-10-21T16:15:01.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0800003520 con 0x7f08140b0680 2023-10-21T16:15:01.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 611930424 0 0) 0x7f0800003520 con 0x7f08140b0680 2023-10-21T16:15:01.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 msgr2=0x7f081415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:01.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f081415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 >> v1:172.21.15.124:6789/0 conn(0x7f081415a820 legacy=0x7f08140aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08140cb030 con 0x7f08140b0680 2023-10-21T16:15:01.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0804001670 con 0x7f08140b0680 2023-10-21T16:15:01.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 >> v1:172.21.15.53:6789/0 conn(0x7f08140b0680 legacy=0x7f081415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f0800003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f08000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0800004d00 0x7f0800005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140cbdf0 con 0x7f08000039a0 2023-10-21T16:15:01.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080000bab0 con 0x7f0800004d00 2023-10-21T16:15:01.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.772+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080000bc70 con 0x7f08140ca680 2023-10-21T16:15:01.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f0800003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081894b640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f08000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081994d640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0800004d00 0x7f0800005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 -- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 msgr2=0x7f08000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140ca680 0x7f08000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 -- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0800004d00 msgr2=0x7f0800005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:01.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0800004d00 0x7f0800005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.773+0000 7f081914c640 1 -- 172.21.15.53:0/3038515815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800003520 con 0x7f08000039a0 2023-10-21T16:15:01.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.774+0000 7f081914c640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f0800003dc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0810001750 tx=0x7f08000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac36cb1dd1d309e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:01.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100052d0 con 0x7f08000039a0 2023-10-21T16:15:01.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081002e050 con 0x7f08000039a0 2023-10-21T16:15:01.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f080bfff640 1 -- 172.21.15.53:0/3038515815 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100035f0 con 0x7f08000039a0 2023-10-21T16:15:01.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 -- 172.21.15.53:0/3038515815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 msgr2=0x7f0800003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 --2- 172.21.15.53:0/3038515815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f0800003dc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0810001750 tx=0x7f08000057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081994d640 1 -- 172.21.15.53:0/3038515815 reap_dead start 2023-10-21T16:15:01.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 -- 172.21.15.53:0/3038515815 shutdown_connections 2023-10-21T16:15:01.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 -- 172.21.15.53:0/3038515815 >> 172.21.15.53:0/3038515815 conn(0x7f081406c050 msgr2=0x7f081406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:01.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 -- 172.21.15.53:0/3038515815 shutdown_connections 2023-10-21T16:15:01.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.775+0000 7f081a14e640 1 -- 172.21.15.53:0/3038515815 wait complete. 2023-10-21T16:15:01.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.777+0000 7f081a14e640 1 Processor -- start 2023-10-21T16:15:01.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.777+0000 7f081a14e640 1 -- start start 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140c7d80 0x7f08140c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081415b000 0x7f081415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140c7410 con 0x7f08000039a0 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140c6800 con 0x7f081415b000 2023-10-21T16:15:01.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.778+0000 7f081a14e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08140c6ae0 con 0x7f08140c7d80 2023-10-21T16:15:01.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36148/0 (socket says 172.21.15.53:36148) 2023-10-21T16:15:01.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 -- 172.21.15.53:0/3885451754 learned_addr learned my addr 172.21.15.53:0/3885451754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:01.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 -- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140c7d80 msgr2=0x7f08140c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140c7d80 0x7f08140c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 -- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081415b000 msgr2=0x7f081415b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:01.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081415b000 0x7f081415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:01.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 -- 172.21.15.53:0/3885451754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08140c6dc0 con 0x7f08000039a0 2023-10-21T16:15:01.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.779+0000 7f081914c640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0810008fd0 tx=0x7f0810001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:01.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.780+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810007a30 con 0x7f08000039a0 2023-10-21T16:15:01.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.780+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08100069e0 con 0x7f08000039a0 2023-10-21T16:15:01.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.780+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810006d50 con 0x7f08000039a0 2023-10-21T16:15:01.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.780+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08140c70a0 con 0x7f08000039a0 2023-10-21T16:15:01.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.780+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081415b9c0 con 0x7f08000039a0 2023-10-21T16:15:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.781+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814074150 con 0x7f08000039a0 2023-10-21T16:15:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.782+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08100035f0 con 0x7f08000039a0 2023-10-21T16:15:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.783+0000 7f0809ffb640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 0x7f07f806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.783+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f08100ac100 con 0x7f08000039a0 2023-10-21T16:15:01.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.783+0000 7f081894b640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 0x7f07f806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:01.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.785+0000 7f081894b640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 0x7f07f806d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f08140cf180 tx=0x7f08040011c0 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.788+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0810076870 con 0x7f08000039a0 2023-10-21T16:15:01.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:01.977+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f081406b0b0 con 0x7f08000039a0 2023-10-21T16:15:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.554+0000 7f0809ffb640 1 -- 172.21.15.53:0/3885451754 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0810079e90 con 0x7f08000039a0 2023-10-21T16:15:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:15:02.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 msgr2=0x7f07f806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 0x7f07f806d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f08140cf180 tx=0x7f08040011c0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 msgr2=0x7f08141589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0810008fd0 tx=0x7f0810001120 comp rx=0 tx=0).stop 2023-10-21T16:15:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 shutdown_connections 2023-10-21T16:15:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f07f806afc0 0x7f07f806d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081415b000 0x7f081415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08140c7d80 0x7f08140c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 --2- 172.21.15.53:0/3885451754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08000039a0 0x7f08141589c0 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0810008fd0 tx=0x7f0810001120 comp rx=0 tx=0).stop 2023-10-21T16:15:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.559+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 >> 172.21.15.53:0/3885451754 conn(0x7f081406c050 msgr2=0x7f08140c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.560+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 shutdown_connections 2023-10-21T16:15:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.560+0000 7f081a14e640 1 -- 172.21.15.53:0/3885451754 wait complete. 2023-10-21T16:15:02.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:15:02.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.821+0000 7f3455ead640 1 Processor -- start 2023-10-21T16:15:02.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.821+0000 7f3455ead640 1 -- start start 2023-10-21T16:15:02.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f3455ead640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34500d00e0 0x7f34500d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f3455ead640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f3455ead640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34500d2870 con 0x7f345015ccb0 2023-10-21T16:15:02.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f3455ead640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500d1150 con 0x7f34500d00e0 2023-10-21T16:15:02.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f3455ead640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500d1430 con 0x7f34500d0b90 2023-10-21T16:15:02.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f344ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34500d00e0 0x7f34500d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f34556ac640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.823+0000 7f344ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34500d00e0 0x7f34500d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:58108/0 (socket says 172.21.15.53:58108) 2023-10-21T16:15:02.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f344ffff640 1 -- 172.21.15.53:0/357630181 learned_addr learned my addr 172.21.15.53:0/357630181 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:02.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f344ffff640 1 -- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 msgr2=0x7f34500b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:02.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f344ffff640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f344ffff640 1 -- 172.21.15.53:0/357630181 >> v1:172.21.15.124:6789/0 conn(0x7f345015ccb0 legacy=0x7f34500cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f344ffff640 1 -- 172.21.15.53:0/357630181 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500d1710 con 0x7f34500d00e0 2023-10-21T16:15:02.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.824+0000 7f34556ac640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:15:02.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.826+0000 7f344ffff640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34500d00e0 0x7f34500d1fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0c50e6a8f658ab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:02.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.826+0000 7f344f7fe640 1 -- 172.21.15.53:0/357630181 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c030840 con 0x7f34500d00e0 2023-10-21T16:15:02.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f344f7fe640 1 -- 172.21.15.53:0/357630181 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0309a0 con 0x7f34500d00e0 2023-10-21T16:15:02.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f344f7fe640 1 -- 172.21.15.53:0/357630181 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c030d30 con 0x7f34500d00e0 2023-10-21T16:15:02.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 -- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34500d00e0 msgr2=0x7f34500d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34500d00e0 0x7f34500d1fb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f343c009870 tx=0x7f343c02b980 comp rx=0 tx=0).stop 2023-10-21T16:15:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 -- 172.21.15.53:0/357630181 shutdown_connections 2023-10-21T16:15:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 --2- 172.21.15.53:0/357630181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34500d00e0 0x7f34500d1fb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 -- 172.21.15.53:0/357630181 >> 172.21.15.53:0/357630181 conn(0x7f3450071660 msgr2=0x7f3450071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:02.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.827+0000 7f3455ead640 1 -- 172.21.15.53:0/357630181 shutdown_connections 2023-10-21T16:15:02.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.828+0000 7f3455ead640 1 -- 172.21.15.53:0/357630181 wait complete. 2023-10-21T16:15:02.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.829+0000 7f3455ead640 1 Processor -- start 2023-10-21T16:15:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.829+0000 7f3455ead640 1 -- start start 2023-10-21T16:15:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ccb0 0x7f34500b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500cb680 con 0x7f34500d00e0 2023-10-21T16:15:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34501f54d0 con 0x7f345015ccb0 2023-10-21T16:15:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.830+0000 7f3455ead640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34500c81f0 con 0x7f34500d0b90 2023-10-21T16:15:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36160/0 (socket says 172.21.15.53:36160) 2023-10-21T16:15:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 -- 172.21.15.53:0/1256535274 learned_addr learned my addr 172.21.15.53:0/1256535274 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f344ffff640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f34556ac640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ccb0 0x7f34500b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 -- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 msgr2=0x7f34500b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 -- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ccb0 msgr2=0x7f34500b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ccb0 0x7f34500b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.831+0000 7f3454eab640 1 -- 172.21.15.53:0/1256535274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009a70 con 0x7f34500d00e0 2023-10-21T16:15:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f3454eab640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f344400c3c0 tx=0x7f344400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440052d0 con 0x7f34500d00e0 2023-10-21T16:15:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444005450 con 0x7f34500d00e0 2023-10-21T16:15:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0094d0 con 0x7f34500d00e0 2023-10-21T16:15:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444007260 con 0x7f34500d00e0 2023-10-21T16:15:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.832+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34500cafc0 con 0x7f34500d00e0 2023-10-21T16:15:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.833+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34500b6200 con 0x7f34500d00e0 2023-10-21T16:15:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.834+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f34440073c0 con 0x7f34500d00e0 2023-10-21T16:15:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.835+0000 7f344d7fa640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 0x7f343406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.836+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6395+0+0 (secure 0 0 0) 0x7f3444088e00 con 0x7f34500d00e0 2023-10-21T16:15:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.836+0000 7f344ffff640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 0x7f343406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.837+0000 7f344ffff640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 0x7f343406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f343c036000 tx=0x7f343c008030 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:02.839+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3444053ae0 con 0x7f34500d00e0 2023-10-21T16:15:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.026+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f34500d19c0 con 0x7f34500d00e0 2023-10-21T16:15:03.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.567+0000 7f344d7fa640 1 -- 172.21.15.53:0/1256535274 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3444057100 con 0x7f34500d00e0 2023-10-21T16:15:03.570 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:15:03.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 msgr2=0x7f343406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 0x7f343406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f343c036000 tx=0x7f343c008030 comp rx=0 tx=0).stop 2023-10-21T16:15:03.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 msgr2=0x7f34501f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f344400c3c0 tx=0x7f344400c880 comp rx=0 tx=0).stop 2023-10-21T16:15:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 shutdown_connections 2023-10-21T16:15:03.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f343406afc0 0x7f343406d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ccb0 0x7f34500b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34500d0b90 0x7f34500b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 --2- 172.21.15.53:0/1256535274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34500d00e0 0x7f34501f4f90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.571+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 >> 172.21.15.53:0/1256535274 conn(0x7f3450071660 msgr2=0x7f345015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:03.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.573+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 shutdown_connections 2023-10-21T16:15:03.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.577+0000 7f3455ead640 1 -- 172.21.15.53:0/1256535274 wait complete. 2023-10-21T16:15:03.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:15:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.814+0000 7fdc9eed7640 1 Processor -- start 2023-10-21T16:15:03.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.814+0000 7fdc9eed7640 1 -- start start 2023-10-21T16:15:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.815+0000 7fdc9eed7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc9806c3a0 0x7fdc980c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9eed7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 0x7fdc980b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9eed7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9815b310 con 0x7fdc9806db60 2023-10-21T16:15:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9eed7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980ca230 con 0x7fdc9806c3a0 2023-10-21T16:15:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9eed7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9806d290 con 0x7fdc9815aa00 2023-10-21T16:15:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9cc73640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc9806c3a0 0x7fdc980c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc97fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdc9806db60 0x7fdc9806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37582/0 (socket says 172.21.15.53:37582) 2023-10-21T16:15:03.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc97fff640 1 -- 172.21.15.53:0/752453670 learned_addr learned my addr 172.21.15.53:0/752453670 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9cc73640 1 -- 172.21.15.53:0/752453670 >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 msgr2=0x7fdc980b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc9d474640 1 --2- 172.21.15.53:0/752453670 >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 0x7fdc980b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.816+0000 7fdc977fe640 1 -- 172.21.15.53:0/752453670 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732762460 0 0) 0x7fdc84001110 con 0x7fdc9806db60 2023-10-21T16:15:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.817+0000 7fdc9cc73640 1 --2- 172.21.15.53:0/752453670 >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 0x7fdc980b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.817+0000 7fdc9cc73640 1 -- 172.21.15.53:0/752453670 >> v1:172.21.15.53:6789/0 conn(0x7fdc9806db60 legacy=0x7fdc9806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.817+0000 7fdc9cc73640 1 -- 172.21.15.53:0/752453670 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980c9860 con 0x7fdc9806c3a0 2023-10-21T16:15:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.817+0000 7fdc9d474640 1 --2- 172.21.15.53:0/752453670 >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 0x7fdc980b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:03.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc9cc73640 1 --2- 172.21.15.53:0/752453670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc980c9cf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c57a0 tx=0x7fdc8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3310c488d76ba248 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:03.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc977fe640 1 -- 172.21.15.53:0/752453670 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c031830 con 0x7fdc9806c3a0 2023-10-21T16:15:03.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc977fe640 1 -- 172.21.15.53:0/752453670 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c031990 con 0x7fdc9806c3a0 2023-10-21T16:15:03.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc977fe640 1 -- 172.21.15.53:0/752453670 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c031d20 con 0x7fdc9806c3a0 2023-10-21T16:15:03.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc9eed7640 1 -- 172.21.15.53:0/752453670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 msgr2=0x7fdc980c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.818+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/752453670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc980c9cf0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c57a0 tx=0x7fdc8c02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 -- 172.21.15.53:0/752453670 shutdown_connections 2023-10-21T16:15:03.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/752453670 >> v2:172.21.15.124:3300/0 conn(0x7fdc9815aa00 0x7fdc980b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/752453670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc980c9cf0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 -- 172.21.15.53:0/752453670 >> 172.21.15.53:0/752453670 conn(0x7fdc9806a5d0 msgr2=0x7fdc980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:03.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 -- 172.21.15.53:0/752453670 shutdown_connections 2023-10-21T16:15:03.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.819+0000 7fdc9eed7640 1 -- 172.21.15.53:0/752453670 wait complete. 2023-10-21T16:15:03.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.820+0000 7fdc9eed7640 1 Processor -- start 2023-10-21T16:15:03.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 -- start start 2023-10-21T16:15:03.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc9806db60 0x7fdc98070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc9815aa00 0x7fdc98145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980c23f0 con 0x7fdc9806c3a0 2023-10-21T16:15:03.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980c26d0 con 0x7fdc9815aa00 2023-10-21T16:15:03.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.821+0000 7fdc9eed7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980c29b0 con 0x7fdc9806db60 2023-10-21T16:15:03.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:03.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:36176/0 (socket says 172.21.15.53:36176) 2023-10-21T16:15:03.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 -- 172.21.15.53:0/528226366 learned_addr learned my addr 172.21.15.53:0/528226366 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:03.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 -- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc9806db60 msgr2=0x7fdc98070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc9806db60 0x7fdc98070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 -- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc9815aa00 msgr2=0x7fdc98145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:03.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9d474640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc9815aa00 0x7fdc98145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:03.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc9815aa00 0x7fdc98145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:03.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.822+0000 7fdc9cc73640 1 -- 172.21.15.53:0/528226366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009400 con 0x7fdc9806c3a0 2023-10-21T16:15:03.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc9cc73640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009890 tx=0x7fdc8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:03.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c03ba70 con 0x7fdc9806c3a0 2023-10-21T16:15:03.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c03bbd0 con 0x7fdc9806c3a0 2023-10-21T16:15:03.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0453f0 con 0x7fdc9806c3a0 2023-10-21T16:15:03.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc980c2c90 con 0x7fdc9806c3a0 2023-10-21T16:15:03.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.823+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980c2f70 con 0x7fdc9806c3a0 2023-10-21T16:15:03.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.825+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9804e160 con 0x7fdc9806c3a0 2023-10-21T16:15:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.825+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc8c045550 con 0x7fdc9806c3a0 2023-10-21T16:15:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.826+0000 7fdc957fa640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 0x7fdc7c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.826+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fdc8c0abc40 con 0x7fdc9806c3a0 2023-10-21T16:15:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.827+0000 7fdc97fff640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 0x7fdc7c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.828+0000 7fdc97fff640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 0x7fdc7c06d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84001310 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:03.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:03.830+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc8c076c90 con 0x7fdc9806c3a0 2023-10-21T16:15:04.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.023+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdc9815d050 con 0x7fdc9806c3a0 2023-10-21T16:15:04.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.030+0000 7fdc957fa640 1 -- 172.21.15.53:0/528226366 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdc8c0ab0a0 con 0x7fdc9806c3a0 2023-10-21T16:15:04.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.032+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 msgr2=0x7fdc7c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.032+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 0x7fdc7c06d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84001310 comp rx=0 tx=0).stop 2023-10-21T16:15:04.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 msgr2=0x7fdc981597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009890 tx=0x7fdc8c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 shutdown_connections 2023-10-21T16:15:04.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdc7c06afe0 0x7fdc7c06d4a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc9815aa00 0x7fdc98145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc9806db60 0x7fdc98070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 --2- 172.21.15.53:0/528226366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc9806c3a0 0x7fdc981597a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 >> 172.21.15.53:0/528226366 conn(0x7fdc9806a5d0 msgr2=0x7fdc980cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:04.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 shutdown_connections 2023-10-21T16:15:04.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.033+0000 7fdc9eed7640 1 -- 172.21.15.53:0/528226366 wait complete. 2023-10-21T16:15:04.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:15:04.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.284+0000 7f583da1a640 1 Processor -- start 2023-10-21T16:15:04.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.285+0000 7f583da1a640 1 -- start start 2023-10-21T16:15:04.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.286+0000 7f583da1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f583815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.286+0000 7f583da1a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583815ac90 con 0x7f58380caef0 2023-10-21T16:15:04.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.286+0000 7f583da1a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583815a6d0 con 0x7f58380cdc30 2023-10-21T16:15:04.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.286+0000 7f583da1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583815a9b0 con 0x7f5838159bc0 2023-10-21T16:15:04.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.286+0000 7f583ca18640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f58380caef0 0x7f58380cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55466/0 (socket says 172.21.15.53:55466) 2023-10-21T16:15:04.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f583ca18640 1 -- 172.21.15.53:0/297835427 learned_addr learned my addr 172.21.15.53:0/297835427 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:04.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f583d219640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f583815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833350913 0 0) 0x7f583815ac90 con 0x7f58380caef0 2023-10-21T16:15:04.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5828005730 con 0x7f58380caef0 2023-10-21T16:15:04.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889246888 0 0) 0x7f5828005730 con 0x7f58380caef0 2023-10-21T16:15:04.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 msgr2=0x7f583815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:04.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f583815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 >> v1:172.21.15.124:6789/0 conn(0x7f58380cdc30 legacy=0x7f58380cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380cc150 con 0x7f58380caef0 2023-10-21T16:15:04.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.287+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f582c001670 con 0x7f58380caef0 2023-10-21T16:15:04.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 >> v1:172.21.15.53:6789/0 conn(0x7f58380caef0 legacy=0x7f58380cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5828005bb0 0x7f5828005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5828006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f5828007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583815a9b0 con 0x7f58280069d0 2023-10-21T16:15:04.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582800dc70 con 0x7f5828005bb0 2023-10-21T16:15:04.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582800de30 con 0x7f5838159bc0 2023-10-21T16:15:04.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5828006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f583d219640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f5828007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f583ca18640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5828005bb0 0x7f5828005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 -- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 msgr2=0x7f5828007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:04.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5838159bc0 0x7f5828007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 -- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5828005bb0 msgr2=0x7f5828005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:04.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5828005bb0 0x7f5828005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f5837fff640 1 -- 172.21.15.53:0/297835427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828005730 con 0x7f58280069d0 2023-10-21T16:15:04.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.288+0000 7f583ca18640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5828005bb0 0x7f5828005fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:04.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f5837fff640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5828006e50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5824001720 tx=0x7f5828007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf97065449e8775a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240052d0 con 0x7f58280069d0 2023-10-21T16:15:04.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582402e050 con 0x7f58280069d0 2023-10-21T16:15:04.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f58377fe640 1 -- 172.21.15.53:0/297835427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240035f0 con 0x7f58280069d0 2023-10-21T16:15:04.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f583da1a640 1 -- 172.21.15.53:0/297835427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 msgr2=0x7f5828006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f583da1a640 1 --2- 172.21.15.53:0/297835427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5828006e50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5824001720 tx=0x7f5828007a00 comp rx=0 tx=0).stop 2023-10-21T16:15:04.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.290+0000 7f583d219640 1 -- 172.21.15.53:0/297835427 reap_dead start 2023-10-21T16:15:04.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.291+0000 7f583da1a640 1 -- 172.21.15.53:0/297835427 shutdown_connections 2023-10-21T16:15:04.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.291+0000 7f583da1a640 1 -- 172.21.15.53:0/297835427 >> 172.21.15.53:0/297835427 conn(0x7f583806c010 msgr2=0x7f583806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:04.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.291+0000 7f583da1a640 1 -- 172.21.15.53:0/297835427 shutdown_connections 2023-10-21T16:15:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.291+0000 7f583da1a640 1 -- 172.21.15.53:0/297835427 wait complete. 2023-10-21T16:15:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.292+0000 7f583da1a640 1 Processor -- start 2023-10-21T16:15:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.293+0000 7f583da1a640 1 -- start start 2023-10-21T16:15:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.293+0000 7f583da1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.293+0000 7f583da1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58380c25c0 0x7f58380c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583da1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58380c42b0 0x7f58380c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583da1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c3e20 con 0x7f58280069d0 2023-10-21T16:15:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583da1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c2f80 con 0x7f58380c42b0 2023-10-21T16:15:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583da1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c3210 con 0x7f58380c25c0 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57348/0 (socket says 172.21.15.53:57348) 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 -- 172.21.15.53:0/2627182273 learned_addr learned my addr 172.21.15.53:0/2627182273 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 -- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58380c25c0 msgr2=0x7f58380c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58380c25c0 0x7f58380c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 -- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58380c42b0 msgr2=0x7f58380c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58380c42b0 0x7f58380c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.294+0000 7f583ca18640 1 -- 172.21.15.53:0/2627182273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380c34f0 con 0x7f58280069d0 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f583ca18640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d750 tx=0x7f582c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0074e0 con 0x7f58280069d0 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c007640 con 0x7f58280069d0 2023-10-21T16:15:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0049e0 con 0x7f58280069d0 2023-10-21T16:15:04.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58380c37d0 con 0x7f58280069d0 2023-10-21T16:15:04.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.295+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583814a2e0 con 0x7f58280069d0 2023-10-21T16:15:04.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.296+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583806ff20 con 0x7f58280069d0 2023-10-21T16:15:04.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.297+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f582c004b40 con 0x7f58280069d0 2023-10-21T16:15:04.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.298+0000 7f58357fa640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 0x7f581c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.298+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f582c0891c0 con 0x7f58280069d0 2023-10-21T16:15:04.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.299+0000 7f5837fff640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 0x7f581c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.300+0000 7f5837fff640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 0x7f581c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5824033000 tx=0x7f582402f000 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.303+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f582c053ff0 con 0x7f58280069d0 2023-10-21T16:15:04.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.496+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f5838074150 con 0x7f58280069d0 2023-10-21T16:15:04.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.502+0000 7f58357fa640 1 -- 172.21.15.53:0/2627182273 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f582c00de60 con 0x7f58280069d0 2023-10-21T16:15:04.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 msgr2=0x7f581c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 0x7f581c06d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5824033000 tx=0x7f582402f000 comp rx=0 tx=0).stop 2023-10-21T16:15:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 msgr2=0x7f5838147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d750 tx=0x7f582c00dc10 comp rx=0 tx=0).stop 2023-10-21T16:15:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 shutdown_connections 2023-10-21T16:15:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f581c06afc0 0x7f581c06d480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58380c42b0 0x7f58380c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58380c25c0 0x7f58380c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 --2- 172.21.15.53:0/2627182273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58280069d0 0x7f5838147350 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 >> 172.21.15.53:0/2627182273 conn(0x7f583806c010 msgr2=0x7f583815bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 shutdown_connections 2023-10-21T16:15:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.505+0000 7f583da1a640 1 -- 172.21.15.53:0/2627182273 wait complete. 2023-10-21T16:15:04.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:15:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.758+0000 7f3ab1a13640 1 Processor -- start 2023-10-21T16:15:04.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.759+0000 7f3ab1a13640 1 -- start start 2023-10-21T16:15:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1a13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac15a820 0x7f3aac0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1a13640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3aac15ac00 0x7f3aac0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1a13640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0cab60 con 0x7f3aac0ca680 2023-10-21T16:15:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1a13640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac0cb960 con 0x7f3aac15ac00 2023-10-21T16:15:04.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1a13640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cbdf0 con 0x7f3aac15a820 2023-10-21T16:15:04.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.760+0000 7f3ab1212640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3aac15ac00 0x7f3aac0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab0a11640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac15a820 0x7f3aac0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3aac15ac00 0x7f3aac0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:45124/0 (socket says 172.21.15.53:45124) 2023-10-21T16:15:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 -- 172.21.15.53:0/3636916234 learned_addr learned my addr 172.21.15.53:0/3636916234 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 -- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac15a820 msgr2=0x7f3aac0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 --2- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac15a820 0x7f3aac0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 -- 172.21.15.53:0/3636916234 >> v1:172.21.15.124:6789/0 conn(0x7f3aac0ca680 legacy=0x7f3aac0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.761+0000 7f3ab1212640 1 -- 172.21.15.53:0/3636916234 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0cb030 con 0x7f3aac15ac00 2023-10-21T16:15:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.763+0000 7f3ab1212640 1 --2- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15ac00 0x7f3aac0b04b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc4588ab41336f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.763+0000 7f3aab7fe640 1 -- 172.21.15.53:0/3636916234 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa402ca70 con 0x7f3aac15ac00 2023-10-21T16:15:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.763+0000 7f3aab7fe640 1 -- 172.21.15.53:0/3636916234 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa402cbd0 con 0x7f3aac15ac00 2023-10-21T16:15:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.763+0000 7f3aab7fe640 1 -- 172.21.15.53:0/3636916234 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40343f0 con 0x7f3aac15ac00 2023-10-21T16:15:04.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15ac00 msgr2=0x7f3aac0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15ac00 0x7f3aac0b04b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa402bed0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3636916234 shutdown_connections 2023-10-21T16:15:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15ac00 0x7f3aac0b04b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3636916234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac15a820 0x7f3aac0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3636916234 >> 172.21.15.53:0/3636916234 conn(0x7f3aac06c050 msgr2=0x7f3aac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:04.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3636916234 shutdown_connections 2023-10-21T16:15:04.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.764+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3636916234 wait complete. 2023-10-21T16:15:04.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.766+0000 7f3ab1a13640 1 Processor -- start 2023-10-21T16:15:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.766+0000 7f3ab1a13640 1 -- start start 2023-10-21T16:15:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 0x7f3aac1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15a820 0x7f3aac157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0c8b60 con 0x7f3aac15ac00 2023-10-21T16:15:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac156fa0 con 0x7f3aac15a820 2023-10-21T16:15:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.767+0000 7f3ab1a13640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1f5ee0 con 0x7f3aac0ca680 2023-10-21T16:15:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57358/0 (socket says 172.21.15.53:57358) 2023-10-21T16:15:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 -- 172.21.15.53:0/3122091249 learned_addr learned my addr 172.21.15.53:0/3122091249 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab0a11640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 0x7f3aac1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3aabfff640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15a820 0x7f3aac157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 -- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 msgr2=0x7f3aac1f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 0x7f3aac1f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 -- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15a820 msgr2=0x7f3aac157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15a820 0x7f3aac157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab1212640 1 -- 172.21.15.53:0/3122091249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4009b30 con 0x7f3aac15ac00 2023-10-21T16:15:04.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.768+0000 7f3ab0a11640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 0x7f3aac1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:15:04.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3ab1212640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4040670 con 0x7f3aac15ac00 2023-10-21T16:15:04.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4040810 con 0x7f3aac15ac00 2023-10-21T16:15:04.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4040bc0 con 0x7f3aac15ac00 2023-10-21T16:15:04.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1f6180 con 0x7f3aac15ac00 2023-10-21T16:15:04.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.769+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1f6740 con 0x7f3aac15ac00 2023-10-21T16:15:04.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.771+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac074150 con 0x7f3aac15ac00 2023-10-21T16:15:04.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.772+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3aa4040d20 con 0x7f3aac15ac00 2023-10-21T16:15:04.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.772+0000 7f3aa97fa640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 0x7f3a9006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:04.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.773+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f3aa40ac870 con 0x7f3aac15ac00 2023-10-21T16:15:04.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.773+0000 7f3ab0a11640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 0x7f3a9006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:04.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.774+0000 7f3ab0a11640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 0x7f3a9006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009b90 tx=0x7f3aa0008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:04.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.777+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3aa4077840 con 0x7f3aac15ac00 2023-10-21T16:15:04.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.970+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f3aac0b0e90 con 0x7f3aac15ac00 2023-10-21T16:15:04.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.978+0000 7f3aa97fa640 1 -- 172.21.15.53:0/3122091249 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3aa4009510 con 0x7f3aac15ac00 2023-10-21T16:15:04.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 msgr2=0x7f3a9006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 0x7f3a9006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009b90 tx=0x7f3aa0008040 comp rx=0 tx=0).stop 2023-10-21T16:15:04.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 msgr2=0x7f3aac0af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:04.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa4004a40 comp rx=0 tx=0).stop 2023-10-21T16:15:04.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 shutdown_connections 2023-10-21T16:15:04.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f3a9006af90 0x7f3a9006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3aac15ac00 0x7f3aac0af100 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aac15a820 0x7f3aac157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 --2- 172.21.15.53:0/3122091249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aac0ca680 0x7f3aac1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:04.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.981+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 >> 172.21.15.53:0/3122091249 conn(0x7f3aac06c050 msgr2=0x7f3aac15bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:04.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.982+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 shutdown_connections 2023-10-21T16:15:04.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:04.982+0000 7f3ab1a13640 1 -- 172.21.15.53:0/3122091249 wait complete. 2023-10-21T16:15:05.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-21T16:15:05.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.220+0000 7f064c607640 1 Processor -- start 2023-10-21T16:15:05.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.220+0000 7f064c607640 1 -- start start 2023-10-21T16:15:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.221+0000 7f064c607640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f06440702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064c607640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0644160940 0x7f0644070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064c607640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440c48f0 con 0x7f0644070d50 2023-10-21T16:15:05.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064c607640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440c4d80 con 0x7f0644160940 2023-10-21T16:15:05.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064c607640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c3ce0 con 0x7f064415cd90 2023-10-21T16:15:05.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064aba4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0644070d50 0x7f06440cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55470/0 (socket says 172.21.15.53:55470) 2023-10-21T16:15:05.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064aba4640 1 -- 172.21.15.53:0/1886738753 learned_addr learned my addr 172.21.15.53:0/1886738753 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f0649ba2640 1 --2- 172.21.15.53:0/1886738753 >> v2:172.21.15.124:3300/0 conn(0x7f0644160940 0x7f0644070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.222+0000 7f064a3a3640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f06440702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683240243 0 0) 0x7f06440c48f0 con 0x7f0644070d50 2023-10-21T16:15:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0634003520 con 0x7f0644070d50 2023-10-21T16:15:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753073363 0 0) 0x7f0634003520 con 0x7f0644070d50 2023-10-21T16:15:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 msgr2=0x7f06440702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f06440702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 >> v2:172.21.15.124:3300/0 conn(0x7f0644160940 msgr2=0x7f0644070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 --2- 172.21.15.53:0/1886738753 >> v2:172.21.15.124:3300/0 conn(0x7f0644160940 0x7f0644070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440c3fc0 con 0x7f0644070d50 2023-10-21T16:15:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f063c001670 con 0x7f0644070d50 2023-10-21T16:15:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 >> v1:172.21.15.53:6789/0 conn(0x7f0644070d50 legacy=0x7f06440cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f0634003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0634003f40 0x7f06340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f0634005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c3ce0 con 0x7f0634004d00 2023-10-21T16:15:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063400b980 con 0x7f0634003f40 2023-10-21T16:15:05.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063400bac0 con 0x7f064415cd90 2023-10-21T16:15:05.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.223+0000 7f0649ba2640 1 --2- 172.21.15.53:0/1886738753 >> v2:172.21.15.124:3300/0 conn(0x7f0644160940 0x7f0644070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:15:05.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f0634005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064a3a3640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f0634003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 -- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 msgr2=0x7f0634003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f0649ba2640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0634003f40 0x7f06340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f064415cd90 0x7f0634003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 -- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0634003f40 msgr2=0x7f06340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0634003f40 0x7f06340047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.224+0000 7f064aba4640 1 -- 172.21.15.53:0/1886738753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634003520 con 0x7f0634004d00 2023-10-21T16:15:05.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.225+0000 7f064aba4640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f0634005180 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c1c0 tx=0x7f06340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a198a225d12545e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c005ad0 con 0x7f0634004d00 2023-10-21T16:15:05.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c005c50 con 0x7f0634004d00 2023-10-21T16:15:05.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f06493a1640 1 -- 172.21.15.53:0/1886738753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c02c810 con 0x7f0634004d00 2023-10-21T16:15:05.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f064c607640 1 -- 172.21.15.53:0/1886738753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 msgr2=0x7f0634005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f064c607640 1 --2- 172.21.15.53:0/1886738753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f0634005180 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c1c0 tx=0x7f06340057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f064aba4640 1 -- 172.21.15.53:0/1886738753 reap_dead start 2023-10-21T16:15:05.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f064c607640 1 -- 172.21.15.53:0/1886738753 shutdown_connections 2023-10-21T16:15:05.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.226+0000 7f064c607640 1 -- 172.21.15.53:0/1886738753 >> 172.21.15.53:0/1886738753 conn(0x7f064406a380 msgr2=0x7f06440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:05.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.227+0000 7f064c607640 1 -- 172.21.15.53:0/1886738753 shutdown_connections 2023-10-21T16:15:05.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.227+0000 7f064c607640 1 -- 172.21.15.53:0/1886738753 wait complete. 2023-10-21T16:15:05.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.228+0000 7f064c607640 1 Processor -- start 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.229+0000 7f064c607640 1 -- start start 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.229+0000 7f064c607640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.229+0000 7f064c607640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06440cb630 0x7f06440c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064c607640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06440b17a0 0x7f06440b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064c607640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440b1560 con 0x7f0634004d00 2023-10-21T16:15:05.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064c607640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440cac70 con 0x7f06440b17a0 2023-10-21T16:15:05.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064c607640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064415ffd0 con 0x7f06440cb630 2023-10-21T16:15:05.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57372/0 (socket says 172.21.15.53:57372) 2023-10-21T16:15:05.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 -- 172.21.15.53:0/148569286 learned_addr learned my addr 172.21.15.53:0/148569286 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:05.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064aba4640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06440b17a0 0x7f06440b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f0649ba2640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06440cb630 0x7f06440c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 -- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06440cb630 msgr2=0x7f06440c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06440cb630 0x7f06440c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 -- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06440b17a0 msgr2=0x7f06440b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06440b17a0 0x7f06440b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.230+0000 7f064a3a3640 1 -- 172.21.15.53:0/148569286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0644154820 con 0x7f0634004d00 2023-10-21T16:15:05.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f064a3a3640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f063800f350 tx=0x7f063800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638017600 con 0x7f0634004d00 2023-10-21T16:15:05.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0638011040 con 0x7f0634004d00 2023-10-21T16:15:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380179c0 con 0x7f0634004d00 2023-10-21T16:15:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06440c9ff0 con 0x7f0634004d00 2023-10-21T16:15:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.231+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06440ca5b0 con 0x7f0634004d00 2023-10-21T16:15:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.232+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06440b0160 con 0x7f0634004d00 2023-10-21T16:15:05.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.233+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0638017b20 con 0x7f0634004d00 2023-10-21T16:15:05.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.234+0000 7f0642ffd640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 0x7f062806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.234+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f063808a0d0 con 0x7f0634004d00 2023-10-21T16:15:05.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.235+0000 7f0649ba2640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 0x7f062806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.236+0000 7f0649ba2640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 0x7f062806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06441616b0 tx=0x7f0630008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.238+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0638054f40 con 0x7f0634004d00 2023-10-21T16:15:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.506+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f0644067490 con 0x7f0634004d00 2023-10-21T16:15:05.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.572+0000 7f0642ffd640 1 -- 172.21.15.53:0/148569286 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v480) v1 ==== 158+0+0 (secure 0 0 0) 0x7f063801c0e0 con 0x7f0634004d00 2023-10-21T16:15:05.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 msgr2=0x7f062806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 0x7f062806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f06441616b0 tx=0x7f0630008040 comp rx=0 tx=0).stop 2023-10-21T16:15:05.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 msgr2=0x7f06441542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f063800f350 tx=0x7f063800f810 comp rx=0 tx=0).stop 2023-10-21T16:15:05.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 shutdown_connections 2023-10-21T16:15:05.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f062806af90 0x7f062806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06440b17a0 0x7f06440b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06440cb630 0x7f06440c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 --2- 172.21.15.53:0/148569286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0634004d00 0x7f06441542e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.575+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 >> 172.21.15.53:0/148569286 conn(0x7f064406a380 msgr2=0x7f06441571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.576+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 shutdown_connections 2023-10-21T16:15:05.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.576+0000 7f064c607640 1 -- 172.21.15.53:0/148569286 wait complete. 2023-10-21T16:15:05.599 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-21T16:15:05.599 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-21T16:15:05.600 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:15:05.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:15:05.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.816+0000 7fc6ca29f640 1 Processor -- start 2023-10-21T16:15:05.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.817+0000 7fc6ca29f640 1 -- start start 2023-10-21T16:15:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.818+0000 7fc6ca29f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c41599d0 0x7fc6c40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.818+0000 7fc6ca29f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc6c40cc210 0x7fc6c40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.818+0000 7fc6ca29f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c40cacf0 con 0x7fc6c4159db0 2023-10-21T16:15:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.818+0000 7fc6ca29f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40b10f0 con 0x7fc6c40cc210 2023-10-21T16:15:05.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.818+0000 7fc6ca29f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40ca3c0 con 0x7fc6c41599d0 2023-10-21T16:15:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc6c40cc210 0x7fc6c40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc6c40cc210 0x7fc6c40b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57382/0 (socket says 172.21.15.53:57382) 2023-10-21T16:15:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 -- 172.21.15.53:0/1011001071 learned_addr learned my addr 172.21.15.53:0/1011001071 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c929d640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c41599d0 0x7fc6c40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 -- 172.21.15.53:0/1011001071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c41599d0 msgr2=0x7fc6c40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c41599d0 0x7fc6c40ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 -- 172.21.15.53:0/1011001071 >> v1:172.21.15.124:6789/0 conn(0x7fc6c4159db0 legacy=0x7fc6c415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.819+0000 7fc6c8a9c640 1 -- 172.21.15.53:0/1011001071 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c40ca6a0 con 0x7fc6c40cc210 2023-10-21T16:15:05.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.820+0000 7fc6c8a9c640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c40b00c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009870 tx=0x7fc6b402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b97c19781f0d706 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.821+0000 7fc6bbfff640 1 -- 172.21.15.53:0/1011001071 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4031640 con 0x7fc6c40cc210 2023-10-21T16:15:05.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.821+0000 7fc6bbfff640 1 -- 172.21.15.53:0/1011001071 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b40317a0 con 0x7fc6c40cc210 2023-10-21T16:15:05.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.821+0000 7fc6bbfff640 1 -- 172.21.15.53:0/1011001071 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4031b30 con 0x7fc6c40cc210 2023-10-21T16:15:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.821+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1011001071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 msgr2=0x7fc6c40b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.821+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c40b00c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009870 tx=0x7fc6b402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1011001071 shutdown_connections 2023-10-21T16:15:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c40b00c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1011001071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c41599d0 0x7fc6c40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1011001071 >> 172.21.15.53:0/1011001071 conn(0x7fc6c40cb0c0 msgr2=0x7fc6c40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1011001071 shutdown_connections 2023-10-21T16:15:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.822+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1011001071 wait complete. 2023-10-21T16:15:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.824+0000 7fc6ca29f640 1 Processor -- start 2023-10-21T16:15:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.824+0000 7fc6ca29f640 1 -- start start 2023-10-21T16:15:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c41599d0 0x7fc6c415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c4159db0 0x7fc6c41e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b16f0 con 0x7fc6c40cc210 2023-10-21T16:15:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b1880 con 0x7fc6c41599d0 2023-10-21T16:15:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6ca29f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c415ac20 con 0x7fc6c4159db0 2023-10-21T16:15:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57388/0 (socket says 172.21.15.53:57388) 2023-10-21T16:15:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 -- 172.21.15.53:0/1558348777 learned_addr learned my addr 172.21.15.53:0/1558348777 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 -- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c4159db0 msgr2=0x7fc6c41e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c4159db0 0x7fc6c41e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 -- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c41599d0 msgr2=0x7fc6c415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c41599d0 0x7fc6c415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.825+0000 7fc6c929d640 1 -- 172.21.15.53:0/1558348777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40094d0 con 0x7fc6c40cc210 2023-10-21T16:15:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6c929d640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000d800 tx=0x7fc6c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000b6c0 con 0x7fc6c40cc210 2023-10-21T16:15:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c000b820 con 0x7fc6c40cc210 2023-10-21T16:15:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c415af00 con 0x7fc6c40cc210 2023-10-21T16:15:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000bbb0 con 0x7fc6c40cc210 2023-10-21T16:15:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.826+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c415b4c0 con 0x7fc6c40cc210 2023-10-21T16:15:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.828+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c415b650 con 0x7fc6c40cc210 2023-10-21T16:15:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.828+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc6c000bd10 con 0x7fc6c40cc210 2023-10-21T16:15:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.830+0000 7fc6b9ffb640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 0x7fc6a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.830+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fc6c0089dd0 con 0x7fc6c40cc210 2023-10-21T16:15:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.831+0000 7fc6c8a9c640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 0x7fc6a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.832+0000 7fc6c8a9c640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 0x7fc6a806d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4036000 tx=0x7fc6b4030000 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:05.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:05.834+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6c0054c90 con 0x7fc6c40cc210 2023-10-21T16:15:06.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.028+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6c404e160 con 0x7fc6c40cc210 2023-10-21T16:15:06.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.030+0000 7fc6b9ffb640 1 -- 172.21.15.53:0/1558348777 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+14200 (secure 0 0 0) 0x7fc6c00582b0 con 0x7fc6c40cc210 2023-10-21T16:15:06.033 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:06.033 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":64,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:15:03.556116+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/810479840":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/20933058":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/3285509161":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2020694858":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/269363197":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/1953414266":"2023-10-22T16:03:50.166436+0000","172.21.15.53:6833/1236966553":"2023-10-22T16:15:00.487643+0000","172.21.15.124:0/2311155348":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6833/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/2484781998":"2023-10-22T16:03:50.166436+0000","172.21.15.124:6832/3409974301":"2023-10-22T16:03:50.166436+0000","172.21.15.124:0/3629698444":"2023-10-22T16:14:56.458396+0000","172.21.15.124:0/2591851543":"2023-10-22T16:14:56.458396+0000","172.21.15.124:6832/637025846":"2023-10-22T16:14:56.458396+0000","172.21.15.53:6832/1236966553":"2023-10-22T16:15:00.487643+0000","172.21.15.124:0/3840977991":"2023-10-22T16:14:56.458396+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:15:06.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 msgr2=0x7fc6a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 0x7fc6a806d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4036000 tx=0x7fc6b4030000 comp rx=0 tx=0).stop 2023-10-21T16:15:06.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 msgr2=0x7fc6c414bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000d800 tx=0x7fc6c000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 shutdown_connections 2023-10-21T16:15:06.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc6a806afc0 0x7fc6a806d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6c4159db0 0x7fc6c41e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6c41599d0 0x7fc6c415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 --2- 172.21.15.53:0/1558348777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6c40cc210 0x7fc6c414bbb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 >> 172.21.15.53:0/1558348777 conn(0x7fc6c40cb0c0 msgr2=0x7fc6c406ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:06.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 shutdown_connections 2023-10-21T16:15:06.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.033+0000 7fc6ca29f640 1 -- 172.21.15.53:0/1558348777 wait complete. 2023-10-21T16:15:06.055 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:15:06.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:15:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.282+0000 7f363086e640 1 Processor -- start 2023-10-21T16:15:06.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.282+0000 7f363086e640 1 -- start start 2023-10-21T16:15:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.283+0000 7f363086e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f362c15dcd0 0x7f362c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.283+0000 7f363086e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0ce290 con 0x7f362c0d1500 2023-10-21T16:15:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.283+0000 7f363086e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0d0a80 con 0x7f362c0d1120 2023-10-21T16:15:06.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.283+0000 7f363086e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0b1fe0 con 0x7f362c15dcd0 2023-10-21T16:15:06.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.284+0000 7f362affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f362c0d1500 0x7f362c0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55476/0 (socket says 172.21.15.53:55476) 2023-10-21T16:15:06.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.284+0000 7f362affd640 1 -- 172.21.15.53:0/2349156742 learned_addr learned my addr 172.21.15.53:0/2349156742 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:06.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.284+0000 7f362bfff640 1 --2- 172.21.15.53:0/2349156742 >> v2:172.21.15.124:3300/0 conn(0x7f362c15dcd0 0x7f362c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.284+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846953978 0 0) 0x7f362c0ce290 con 0x7f362c0d1500 2023-10-21T16:15:06.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.284+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3618003520 con 0x7f362c0d1500 2023-10-21T16:15:06.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 772923841 0 0) 0x7f3618003520 con 0x7f362c0d1500 2023-10-21T16:15:06.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 >> v2:172.21.15.124:3300/0 conn(0x7f362c15dcd0 msgr2=0x7f362c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:06.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 --2- 172.21.15.53:0/2349156742 >> v2:172.21.15.124:3300/0 conn(0x7f362c15dcd0 0x7f362c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 >> v1:172.21.15.124:6789/0 conn(0x7f362c0d1120 legacy=0x7f362c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0b4580 con 0x7f362c0d1500 2023-10-21T16:15:06.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3614001670 con 0x7f362c0d1500 2023-10-21T16:15:06.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 >> v1:172.21.15.53:6789/0 conn(0x7f362c0d1500 legacy=0x7f362c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3618003970 0x7f3618003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180047c0 0x7f3618004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 0x7f361800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0b1fe0 con 0x7f3618005180 2023-10-21T16:15:06.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361800c120 con 0x7f3618003970 2023-10-21T16:15:06.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361800c2e0 con 0x7f36180047c0 2023-10-21T16:15:06.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.285+0000 7f362bfff640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 0x7f361800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362affd640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180047c0 0x7f3618004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362bfff640 1 -- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180047c0 msgr2=0x7f3618004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:06.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362b7fe640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3618003970 0x7f3618003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362bfff640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180047c0 0x7f3618004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362bfff640 1 -- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3618003970 msgr2=0x7f3618003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:06.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362bfff640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3618003970 0x7f3618003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362bfff640 1 -- 172.21.15.53:0/2349156742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618003520 con 0x7f3618005180 2023-10-21T16:15:06.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362affd640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36180047c0 0x7f3618004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:06.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.286+0000 7f362b7fe640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3618003970 0x7f3618003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:06.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.287+0000 7f362bfff640 1 -- 172.21.15.53:0/2349156742 reap_dead start 2023-10-21T16:15:06.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.287+0000 7f362bfff640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 0x7f361800b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3624009ac0 tx=0x7f36180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f58554663479c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.287+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624004020 con 0x7f3618005180 2023-10-21T16:15:06.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.287+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624004180 con 0x7f3618005180 2023-10-21T16:15:06.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.287+0000 7f362a7fc640 1 -- 172.21.15.53:0/2349156742 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36240044f0 con 0x7f3618005180 2023-10-21T16:15:06.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 -- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 msgr2=0x7f361800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 0x7f361800b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3624009ac0 tx=0x7f36180057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 -- 172.21.15.53:0/2349156742 shutdown_connections 2023-10-21T16:15:06.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 --2- 172.21.15.53:0/2349156742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3618005180 0x7f361800b980 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 -- 172.21.15.53:0/2349156742 >> 172.21.15.53:0/2349156742 conn(0x7f362c0d2960 msgr2=0x7f362c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:06.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 -- 172.21.15.53:0/2349156742 shutdown_connections 2023-10-21T16:15:06.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.288+0000 7f363086e640 1 -- 172.21.15.53:0/2349156742 wait complete. 2023-10-21T16:15:06.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.290+0000 7f363086e640 1 Processor -- start 2023-10-21T16:15:06.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.290+0000 7f363086e640 1 -- start start 2023-10-21T16:15:06.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3618005180 0x7f362c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f362c0ca0f0 0x7f362c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c159800 con 0x7f362c1593a0 2023-10-21T16:15:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c159b00 con 0x7f362c0ca0f0 2023-10-21T16:15:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f363086e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c159de0 con 0x7f3618005180 2023-10-21T16:15:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57414/0 (socket says 172.21.15.53:57414) 2023-10-21T16:15:06.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 -- 172.21.15.53:0/1318310150 learned_addr learned my addr 172.21.15.53:0/1318310150 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:06.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362b7fe640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3618005180 0x7f362c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 -- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3618005180 msgr2=0x7f362c0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362affd640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f362c0ca0f0 0x7f362c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3618005180 0x7f362c0b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 -- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f362c0ca0f0 msgr2=0x7f362c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f362c0ca0f0 0x7f362c158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.292+0000 7f362bfff640 1 -- 172.21.15.53:0/1318310150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c15a0c0 con 0x7f362c1593a0 2023-10-21T16:15:06.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f362bfff640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f362402c380 tx=0x7f3624004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624003810 con 0x7f362c1593a0 2023-10-21T16:15:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624003970 con 0x7f362c1593a0 2023-10-21T16:15:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c15a3a0 con 0x7f362c1593a0 2023-10-21T16:15:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624003d20 con 0x7f362c1593a0 2023-10-21T16:15:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.293+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0b0cb0 con 0x7f362c1593a0 2023-10-21T16:15:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.294+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c15a8c0 con 0x7f362c1593a0 2023-10-21T16:15:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.295+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3624003e80 con 0x7f362c1593a0 2023-10-21T16:15:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.296+0000 7f3623fff640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 0x7f360c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.296+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f362403b070 con 0x7f362c1593a0 2023-10-21T16:15:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.297+0000 7f362b7fe640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 0x7f360c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.298+0000 7f362b7fe640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 0x7f360c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f361c001330 tx=0x7f361c009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.301+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f362c15a8c0 con 0x7f362c1593a0 2023-10-21T16:15:06.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.494+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f362c15a8c0 con 0x7f362c1593a0 2023-10-21T16:15:06.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.495+0000 7f3623fff640 1 -- 172.21.15.53:0/1318310150 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f362c15a8c0 con 0x7f362c1593a0 2023-10-21T16:15:06.498 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:15:06.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 msgr2=0x7f360c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 0x7f360c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f361c001330 tx=0x7f361c009040 comp rx=0 tx=0).stop 2023-10-21T16:15:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 msgr2=0x7f362c0c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f362402c380 tx=0x7f3624004730 comp rx=0 tx=0).stop 2023-10-21T16:15:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 shutdown_connections 2023-10-21T16:15:06.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f360c06af90 0x7f360c06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f362c1593a0 0x7f362c0c7ac0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f362c0ca0f0 0x7f362c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 --2- 172.21.15.53:0/1318310150 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3618005180 0x7f362c0b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 >> 172.21.15.53:0/1318310150 conn(0x7f362c0d2960 msgr2=0x7f362c15ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:06.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.498+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 shutdown_connections 2023-10-21T16:15:06.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.499+0000 7f363086e640 1 -- 172.21.15.53:0/1318310150 wait complete. 2023-10-21T16:15:06.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-21T16:15:06.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.743+0000 7f55bca47640 1 Processor -- start 2023-10-21T16:15:06.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.743+0000 7f55bca47640 1 -- start start 2023-10-21T16:15:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55bca47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55bca47640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55b80b1940 0x7f55b806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55bca47640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b80cc920 con 0x7f55b80c1580 2023-10-21T16:15:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55bca47640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80ad8e0 con 0x7f55b80b1940 2023-10-21T16:15:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55bca47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80ccbc0 con 0x7f55b80cdc60 2023-10-21T16:15:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55b7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55b80b1940 0x7f55b806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55b7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55b80b1940 0x7f55b806c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57426/0 (socket says 172.21.15.53:57426) 2023-10-21T16:15:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.745+0000 7f55b7fff640 1 -- 172.21.15.53:0/2981481598 learned_addr learned my addr 172.21.15.53:0/2981481598 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:06.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b6ffd640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b7fff640 1 -- 172.21.15.53:0/2981481598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 msgr2=0x7f55b80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:06.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b7fff640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b80b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b7fff640 1 -- 172.21.15.53:0/2981481598 >> v1:172.21.15.124:6789/0 conn(0x7f55b80c1580 legacy=0x7f55b80c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:06.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b7fff640 1 -- 172.21.15.53:0/2981481598 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80ccea0 con 0x7f55b80b1940 2023-10-21T16:15:06.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.746+0000 7f55b6ffd640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:06.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.747+0000 7f55b7fff640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80b1940 0x7f55b806c140 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009e50 tx=0x7f55b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9285d630dce52 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.747+0000 7f55b67fc640 1 -- 172.21.15.53:0/2981481598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b002ca80 con 0x7f55b80b1940 2023-10-21T16:15:06.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.747+0000 7f55b67fc640 1 -- 172.21.15.53:0/2981481598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b002cbe0 con 0x7f55b80b1940 2023-10-21T16:15:06.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.747+0000 7f55b67fc640 1 -- 172.21.15.53:0/2981481598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00343f0 con 0x7f55b80b1940 2023-10-21T16:15:06.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 -- 172.21.15.53:0/2981481598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80b1940 msgr2=0x7f55b806c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80b1940 0x7f55b806c140 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009e50 tx=0x7f55b002bee0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 -- 172.21.15.53:0/2981481598 shutdown_connections 2023-10-21T16:15:06.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 --2- 172.21.15.53:0/2981481598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80b1940 0x7f55b806c140 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 -- 172.21.15.53:0/2981481598 >> 172.21.15.53:0/2981481598 conn(0x7f55b80ce9d0 msgr2=0x7f55b80cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:06.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 -- 172.21.15.53:0/2981481598 shutdown_connections 2023-10-21T16:15:06.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.748+0000 7f55bca47640 1 -- 172.21.15.53:0/2981481598 wait complete. 2023-10-21T16:15:06.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.750+0000 7f55bca47640 1 Processor -- start 2023-10-21T16:15:06.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.750+0000 7f55bca47640 1 -- start start 2023-10-21T16:15:06.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 0x7f55b80c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80c6e30 con 0x7f55b80c1580 2023-10-21T16:15:06.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b81facb0 con 0x7f55b80b1940 2023-10-21T16:15:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.751+0000 7f55bca47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80c4f40 con 0x7f55b80cdc60 2023-10-21T16:15:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57436/0 (socket says 172.21.15.53:57436) 2023-10-21T16:15:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 -- 172.21.15.53:0/3023847393 learned_addr learned my addr 172.21.15.53:0/3023847393 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:06.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b77fe640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 0x7f55b80c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 -- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 msgr2=0x7f55b815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 -- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 msgr2=0x7f55b80c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:06.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 0x7f55b80c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 -- 172.21.15.53:0/3023847393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0009b30 con 0x7f55b80c1580 2023-10-21T16:15:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b77fe640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 0x7f55b80c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.752+0000 7f55b6ffd640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c850 tx=0x7f55ac00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.753+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac004230 con 0x7f55b80c1580 2023-10-21T16:15:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.753+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac004390 con 0x7f55b80c1580 2023-10-21T16:15:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.753+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac007a70 con 0x7f55b80c1580 2023-10-21T16:15:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.753+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b81fa9d0 con 0x7f55b80c1580 2023-10-21T16:15:06.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.753+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b80c60f0 con 0x7f55b80c1580 2023-10-21T16:15:06.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.754+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b804e160 con 0x7f55b80c1580 2023-10-21T16:15:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.755+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55ac007bd0 con 0x7f55b80c1580 2023-10-21T16:15:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.756+0000 7f559ffff640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 0x7f55a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.756+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f55ac0898a0 con 0x7f55b80c1580 2023-10-21T16:15:06.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.757+0000 7f55b77fe640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 0x7f55a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.758+0000 7f55b77fe640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 0x7f55a406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55a8007fd0 tx=0x7f55a8009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:06.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:06.760+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55ac054650 con 0x7f55b80c1580 2023-10-21T16:15:07.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.022+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f55b8073820 con 0x7f55b80c1580 2023-10-21T16:15:07.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.577+0000 7f559ffff640 1 -- 172.21.15.53:0/3023847393 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v482) v1 ==== 157+0+0 (secure 0 0 0) 0x7f55ac01e0e0 con 0x7f55b80c1580 2023-10-21T16:15:07.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 msgr2=0x7f55a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 0x7f55a406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55a8007fd0 tx=0x7f55a8009040 comp rx=0 tx=0).stop 2023-10-21T16:15:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 msgr2=0x7f55b80c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c850 tx=0x7f55ac00cd10 comp rx=0 tx=0).stop 2023-10-21T16:15:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 shutdown_connections 2023-10-21T16:15:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f55a406af90 0x7f55a406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b80cdc60 0x7f55b815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b80c1580 0x7f55b80c4a00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 --2- 172.21.15.53:0/3023847393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b80b1940 0x7f55b80c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 >> 172.21.15.53:0/3023847393 conn(0x7f55b80ce9d0 msgr2=0x7f55b806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 shutdown_connections 2023-10-21T16:15:07.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.581+0000 7f55bca47640 1 -- 172.21.15.53:0/3023847393 wait complete. 2023-10-21T16:15:07.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:15:07.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.829+0000 7f9f2759e640 1 Processor -- start 2023-10-21T16:15:07.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.829+0000 7f9f2759e640 1 -- start start 2023-10-21T16:15:07.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 0x7f9f280d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f280cf690 0x7f9f280ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 0x7f9f2815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280cd520 con 0x7f9f280cf690 2023-10-21T16:15:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280d1910 con 0x7f9f280cf2b0 2023-10-21T16:15:07.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280ce080 con 0x7f9f280d1100 2023-10-21T16:15:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f280cf690 0x7f9f280ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f280cf690 0x7f9f280ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57438/0 (socket says 172.21.15.53:57438) 2023-10-21T16:15:07.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 -- 172.21.15.53:0/2640698147 learned_addr learned my addr 172.21.15.53:0/2640698147 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 -- 172.21.15.53:0/2640698147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 msgr2=0x7f9f2815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f26d9d640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 0x7f9f2815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 0x7f9f2815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2640698147 >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 0x7f9f280d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.831+0000 7f9f25d9b640 1 -- 172.21.15.53:0/2640698147 >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 msgr2=0x7f9f280d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:07.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.832+0000 7f9f25d9b640 1 --2- 172.21.15.53:0/2640698147 >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 0x7f9f280d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.832+0000 7f9f25d9b640 1 -- 172.21.15.53:0/2640698147 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280cd750 con 0x7f9f280cf690 2023-10-21T16:15:07.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.832+0000 7f9f26d9d640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 0x7f9f2815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:07.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.832+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2640698147 >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 0x7f9f280d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.833+0000 7f9f25d9b640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280cf690 0x7f9f280ce510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9f280b54b0 tx=0x7f9f1002b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5803bb70da00628b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.833+0000 7f9f2559a640 1 -- 172.21.15.53:0/2640698147 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100317f0 con 0x7f9f280cf690 2023-10-21T16:15:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.833+0000 7f9f2559a640 1 -- 172.21.15.53:0/2640698147 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10031950 con 0x7f9f280cf690 2023-10-21T16:15:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.833+0000 7f9f2559a640 1 -- 172.21.15.53:0/2640698147 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10031ce0 con 0x7f9f280cf690 2023-10-21T16:15:07.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 -- 172.21.15.53:0/2640698147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280cf690 msgr2=0x7f9f280ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280cf690 0x7f9f280ce510 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9f280b54b0 tx=0x7f9f1002b890 comp rx=0 tx=0).stop 2023-10-21T16:15:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 -- 172.21.15.53:0/2640698147 shutdown_connections 2023-10-21T16:15:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280d1100 0x7f9f2815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2640698147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280cf690 0x7f9f280ce510 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2640698147 >> v2:172.21.15.124:3300/0 conn(0x7f9f280cf2b0 0x7f9f280d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 -- 172.21.15.53:0/2640698147 >> 172.21.15.53:0/2640698147 conn(0x7f9f2809bc40 msgr2=0x7f9f280d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 -- 172.21.15.53:0/2640698147 shutdown_connections 2023-10-21T16:15:07.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.834+0000 7f9f2759e640 1 -- 172.21.15.53:0/2640698147 wait complete. 2023-10-21T16:15:07.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.836+0000 7f9f2759e640 1 Processor -- start 2023-10-21T16:15:07.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.836+0000 7f9f2759e640 1 -- start start 2023-10-21T16:15:07.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 0x7f9f281f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280cf690 0x7f9f280cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281f6a60 con 0x7f9f280d1100 2023-10-21T16:15:07.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cbba0 con 0x7f9f280cf2b0 2023-10-21T16:15:07.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f281523e0 con 0x7f9f280cf690 2023-10-21T16:15:07.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57452/0 (socket says 172.21.15.53:57452) 2023-10-21T16:15:07.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f26d9d640 1 -- 172.21.15.53:0/2486534841 learned_addr learned my addr 172.21.15.53:0/2486534841 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:07.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.837+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 0x7f9f281f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f25d9b640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280cf690 0x7f9f280cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 -- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280cf690 msgr2=0x7f9f280cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:07.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280cf690 0x7f9f280cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 -- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 msgr2=0x7f9f281f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:07.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 0x7f9f281f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:07.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 -- 172.21.15.53:0/2486534841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f100094d0 con 0x7f9f280d1100 2023-10-21T16:15:07.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 0x7f9f281f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:07.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f26d9d640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c7f0 tx=0x7f9f2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:07.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20004230 con 0x7f9f280d1100 2023-10-21T16:15:07.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20004390 con 0x7f9f280d1100 2023-10-21T16:15:07.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20007860 con 0x7f9f280d1100 2023-10-21T16:15:07.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280c9ba0 con 0x7f9f280d1100 2023-10-21T16:15:07.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.838+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f280cc070 con 0x7f9f280d1100 2023-10-21T16:15:07.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.839+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f280c9d30 con 0x7f9f280d1100 2023-10-21T16:15:07.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.841+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9f20007bd0 con 0x7f9f280d1100 2023-10-21T16:15:07.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.841+0000 7f9f1effd640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 0x7f9f0806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:07.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.842+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f9f20089d20 con 0x7f9f280d1100 2023-10-21T16:15:07.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.842+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 0x7f9f0806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:07.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.843+0000 7f9f2659c640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 0x7f9f0806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9f18007f40 tx=0x7f9f18007490 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:07.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:07.846+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f20054c10 con 0x7f9f280d1100 2023-10-21T16:15:08.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.037+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9f28074150 con 0x7f9f280d1100 2023-10-21T16:15:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.042+0000 7f9f1effd640 1 -- 172.21.15.53:0/2486534841 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f200150c0 con 0x7f9f280d1100 2023-10-21T16:15:08.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 msgr2=0x7f9f0806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:08.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 0x7f9f0806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9f18007f40 tx=0x7f9f18007490 comp rx=0 tx=0).stop 2023-10-21T16:15:08.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 msgr2=0x7f9f28151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:08.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000c7f0 tx=0x7f9f2000ccb0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 shutdown_connections 2023-10-21T16:15:08.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9f0806af90 0x7f9f0806d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f280d1100 0x7f9f28151ea0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f280cf690 0x7f9f280cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 --2- 172.21.15.53:0/2486534841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f280cf2b0 0x7f9f281f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.045+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 >> 172.21.15.53:0/2486534841 conn(0x7f9f2809bc40 msgr2=0x7f9f280b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:08.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.046+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 shutdown_connections 2023-10-21T16:15:08.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.046+0000 7f9f2759e640 1 -- 172.21.15.53:0/2486534841 wait complete. 2023-10-21T16:15:08.068 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:15:08.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:15:08.192 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:15:08.321 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:15:08.322 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:15:08.322 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:15:08.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:15:08.389 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:15:08.389 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:15:08.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:15:08.449 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:15:08.450 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:15:08.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:15:08.514 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:15:08.514 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:15:08.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:15:08.575 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:15:08.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:15:08.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.807+0000 7f9ae350f640 1 Processor -- start 2023-10-21T16:15:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.807+0000 7f9ae350f640 1 -- start start 2023-10-21T16:15:08.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc159ac0 0x7f9adc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9adc0ad790 0x7f9adc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 0x7f9adc070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc06dee0 con 0x7f9adc0ad790 2023-10-21T16:15:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc15d850 con 0x7f9adc15e330 2023-10-21T16:15:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae350f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc15db30 con 0x7f9adc159ac0 2023-10-21T16:15:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae0aaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9adc0ad790 0x7f9adc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae0aaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9adc0ad790 0x7f9adc0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57460/0 (socket says 172.21.15.53:57460) 2023-10-21T16:15:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.809+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/1574072300 learned_addr learned my addr 172.21.15.53:0/1574072300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae12ab640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc159ac0 0x7f9adc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae1aac640 1 --2- 172.21.15.53:0/1574072300 >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 0x7f9adc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/1574072300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc159ac0 msgr2=0x7f9adc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc159ac0 0x7f9adc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/1574072300 >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 msgr2=0x7f9adc070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/1574072300 >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 0x7f9adc070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/1574072300 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc15de10 con 0x7f9adc0ad790 2023-10-21T16:15:08.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.810+0000 7f9ae1aac640 1 --2- 172.21.15.53:0/1574072300 >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 0x7f9adc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.811+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc0ad790 0x7f9adc0702d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0ce730 tx=0x7f9acc02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e9c128116da26 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.811+0000 7f9ad3fff640 1 -- 172.21.15.53:0/1574072300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc02ee80 con 0x7f9adc0ad790 2023-10-21T16:15:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.811+0000 7f9ad3fff640 1 -- 172.21.15.53:0/1574072300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc036440 con 0x7f9adc0ad790 2023-10-21T16:15:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.811+0000 7f9ad3fff640 1 -- 172.21.15.53:0/1574072300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0367b0 con 0x7f9adc0ad790 2023-10-21T16:15:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 -- 172.21.15.53:0/1574072300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc0ad790 msgr2=0x7f9adc0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc0ad790 0x7f9adc0702d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0ce730 tx=0x7f9acc02e0d0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 -- 172.21.15.53:0/1574072300 shutdown_connections 2023-10-21T16:15:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 --2- 172.21.15.53:0/1574072300 >> v2:172.21.15.124:3300/0 conn(0x7f9adc15e330 0x7f9adc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc0ad790 0x7f9adc0702d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 --2- 172.21.15.53:0/1574072300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc159ac0 0x7f9adc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 -- 172.21.15.53:0/1574072300 >> 172.21.15.53:0/1574072300 conn(0x7f9adc06a380 msgr2=0x7f9adc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 -- 172.21.15.53:0/1574072300 shutdown_connections 2023-10-21T16:15:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.812+0000 7f9ae350f640 1 -- 172.21.15.53:0/1574072300 wait complete. 2023-10-21T16:15:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.814+0000 7f9ae350f640 1 Processor -- start 2023-10-21T16:15:08.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.814+0000 7f9ae350f640 1 -- start start 2023-10-21T16:15:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 0x7f9adc0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 0x7f9adc1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1f5960 con 0x7f9adc159ac0 2023-10-21T16:15:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc151900 con 0x7f9adc15e330 2023-10-21T16:15:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae350f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc151be0 con 0x7f9adc0ad790 2023-10-21T16:15:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae0aaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae0aaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57474/0 (socket says 172.21.15.53:57474) 2023-10-21T16:15:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.815+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/2384865949 learned_addr learned my addr 172.21.15.53:0/2384865949 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae12ab640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 0x7f9adc0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae1aac640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 0x7f9adc1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 msgr2=0x7f9adc0acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 0x7f9adc0acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 msgr2=0x7f9adc1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 0x7f9adc1f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 -- 172.21.15.53:0/2384865949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc00b770 con 0x7f9adc159ac0 2023-10-21T16:15:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae12ab640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 0x7f9adc0acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:15:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae1aac640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 0x7f9adc1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ae0aaa640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02e110 tx=0x7f9acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.816+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc034850 con 0x7f9adc159ac0 2023-10-21T16:15:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.817+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc0349b0 con 0x7f9adc159ac0 2023-10-21T16:15:08.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.817+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc034d60 con 0x7f9adc159ac0 2023-10-21T16:15:08.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.817+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc151ec0 con 0x7f9adc159ac0 2023-10-21T16:15:08.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.817+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1515d0 con 0x7f9adc159ac0 2023-10-21T16:15:08.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.818+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc1f5e60 con 0x7f9adc159ac0 2023-10-21T16:15:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.819+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9acc0403e0 con 0x7f9adc159ac0 2023-10-21T16:15:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.820+0000 7f9ad1ffb640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 0x7f9ac006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:08.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.820+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f9acc0abba0 con 0x7f9adc159ac0 2023-10-21T16:15:08.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.821+0000 7f9ae12ab640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 0x7f9ac006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:08.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.822+0000 7f9ae12ab640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 0x7f9ac006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:08.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:08.824+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9adc1f5e60 con 0x7f9adc159ac0 2023-10-21T16:15:09.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.054+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9adc1f5e60 con 0x7f9adc159ac0 2023-10-21T16:15:09.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.055+0000 7f9ad1ffb640 1 -- 172.21.15.53:0/2384865949 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3697 (secure 0 0 0) 0x7f9adc1f5e60 con 0x7f9adc159ac0 2023-10-21T16:15:09.058 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:09.058 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2452641650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2452641650},{"type":"v1","addr":"172.21.15.124:6836","nonce":2452641650}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/596282762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":596282762},{"type":"v1","addr":"172.21.15.53:6835","nonce":596282762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":6039,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2576795757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2576795757},{"type":"v1","addr":"172.21.15.53:6833","nonce":2576795757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34676,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/89230818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":89230818},{"type":"v1","addr":"172.21.15.124:6837","nonce":89230818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-21T16:15:09.058 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 20 2023-10-21T16:15:09.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 msgr2=0x7f9ac006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 0x7f9ac006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8008040 comp rx=0 tx=0).stop 2023-10-21T16:15:09.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 msgr2=0x7f9adc151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9acc02e110 tx=0x7f9acc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 shutdown_connections 2023-10-21T16:15:09.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9ac006af90 0x7f9ac006d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9adc15e330 0x7f9adc1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9adc159ac0 0x7f9adc151090 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 --2- 172.21.15.53:0/2384865949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9adc0ad790 0x7f9adc0acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 >> 172.21.15.53:0/2384865949 conn(0x7f9adc06a380 msgr2=0x7f9adc0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:09.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 shutdown_connections 2023-10-21T16:15:09.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.058+0000 7f9ae350f640 1 -- 172.21.15.53:0/2384865949 wait complete. 2023-10-21T16:15:09.082 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:15:09.082 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:15:09.082 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:15:09.083 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:15:09.083 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:15:09.083 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:15:09.084 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:15:09.084 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:15:09.084 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:15:09.084 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:15:09.085 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:15:09.085 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:15:09.085 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:15:09.085 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:15:09.086 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:15:09.086 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:15:09.100 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:15:09.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:15:09.106 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:15:09.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:15:09.107 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:15:09.108 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:15:09.111 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:15:09.111 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:15:09.151 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:15:09.152 DEBUG:teuthology.parallel:result is None 2023-10-21T16:15:09.156 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:15:09.157 DEBUG:teuthology.parallel:result is None 2023-10-21T16:15:09.160 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:15:09.160 DEBUG:teuthology.parallel:result is None 2023-10-21T16:15:09.170 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:15:09.170 DEBUG:teuthology.parallel:result is None 2023-10-21T16:15:09.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:15:09.312 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:15:09.313 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:15:09.324 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:15:09.342 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:15:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.362+0000 7fe000d89640 1 Processor -- start 2023-10-21T16:15:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.362+0000 7fe000d89640 1 -- start start 2023-10-21T16:15:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.363+0000 7fe000d89640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 0x7fdffc069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.363+0000 7fe000d89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 0x7fdffc06a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.363+0000 7fe000d89640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc0ce210 con 0x7fdffc1601b0 2023-10-21T16:15:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.363+0000 7fe000d89640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc0cdb40 con 0x7fdffc071c30 2023-10-21T16:15:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.363+0000 7fe000d89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0cdee0 con 0x7fdffc0720c0 2023-10-21T16:15:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.364+0000 7fdffad9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 0x7fdffc06a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.364+0000 7fdffa59c640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fdffc1601b0 0x7fdffc0cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:58902/0 (socket says 172.21.15.53:58902) 2023-10-21T16:15:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.364+0000 7fdff9d9b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 0x7fdffc069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.364+0000 7fdffa59c640 1 -- 172.21.15.53:0/573357757 learned_addr learned my addr 172.21.15.53:0/573357757 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdffad9d640 1 -- 172.21.15.53:0/573357757 >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 msgr2=0x7fdffc069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdffad9d640 1 --2- 172.21.15.53:0/573357757 >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 0x7fdffc069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdffad9d640 1 -- 172.21.15.53:0/573357757 >> v1:172.21.15.124:6789/0 conn(0x7fdffc1601b0 legacy=0x7fdffc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdffad9d640 1 -- 172.21.15.53:0/573357757 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc161530 con 0x7fdffc0720c0 2023-10-21T16:15:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdff9d9b640 1 --2- 172.21.15.53:0/573357757 >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 0x7fdffc069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:15:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.365+0000 7fdff959a640 1 -- 172.21.15.53:0/573357757 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911900113 0 0) 0x7fdffc0ce210 con 0x7fdffc1601b0 2023-10-21T16:15:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.366+0000 7fdffad9d640 1 --2- 172.21.15.53:0/573357757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 0x7fdffc06a290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009d90 tx=0x7fdff402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e420c660b727702 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:09.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.366+0000 7fdff959a640 1 -- 172.21.15.53:0/573357757 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff402cc50 con 0x7fdffc0720c0 2023-10-21T16:15:09.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.366+0000 7fdff959a640 1 -- 172.21.15.53:0/573357757 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff402cdb0 con 0x7fdffc0720c0 2023-10-21T16:15:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.366+0000 7fdff959a640 1 -- 172.21.15.53:0/573357757 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4034660 con 0x7fdffc0720c0 2023-10-21T16:15:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 -- 172.21.15.53:0/573357757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 msgr2=0x7fdffc06a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 --2- 172.21.15.53:0/573357757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 0x7fdffc06a290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009d90 tx=0x7fdff402bea0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 -- 172.21.15.53:0/573357757 shutdown_connections 2023-10-21T16:15:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 --2- 172.21.15.53:0/573357757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc0720c0 0x7fdffc06a290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 --2- 172.21.15.53:0/573357757 >> v2:172.21.15.124:3300/0 conn(0x7fdffc071c30 0x7fdffc069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 -- 172.21.15.53:0/573357757 >> 172.21.15.53:0/573357757 conn(0x7fdffc06d430 msgr2=0x7fdffc06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:09.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 -- 172.21.15.53:0/573357757 shutdown_connections 2023-10-21T16:15:09.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.367+0000 7fe000d89640 1 -- 172.21.15.53:0/573357757 wait complete. 2023-10-21T16:15:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.369+0000 7fe000d89640 1 Processor -- start 2023-10-21T16:15:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.369+0000 7fe000d89640 1 -- start start 2023-10-21T16:15:09.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 0x7fdffc1f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdffc1601b0 0x7fdffc14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc15ac30 con 0x7fdffc0720c0 2023-10-21T16:15:09.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc1f25e0 con 0x7fdffc1601b0 2023-10-21T16:15:09.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fe000d89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc14b2a0 con 0x7fdffc071c30 2023-10-21T16:15:09.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdffa59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdff9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 0x7fdffc1f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdffa59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:52532/0 (socket says 172.21.15.53:52532) 2023-10-21T16:15:09.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdffa59c640 1 -- 172.21.15.53:0/105493861 learned_addr learned my addr 172.21.15.53:0/105493861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdffad9d640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdffc1601b0 0x7fdffc14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdff9d9b640 1 -- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 msgr2=0x7fdffc0ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.370+0000 7fdff9d9b640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdff9d9b640 1 -- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdffc1601b0 msgr2=0x7fdffc14a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdff9d9b640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdffc1601b0 0x7fdffc14a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdff9d9b640 1 -- 172.21.15.53:0/105493861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0009400 con 0x7fdffc0720c0 2023-10-21T16:15:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdffa59c640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:15:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdff9d9b640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 0x7fdffc1f20a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e870 tx=0x7fdff000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000c940 con 0x7fdffc0720c0 2023-10-21T16:15:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000caa0 con 0x7fdffc0720c0 2023-10-21T16:15:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000ce30 con 0x7fdffc0720c0 2023-10-21T16:15:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4009a70 con 0x7fdffc0720c0 2023-10-21T16:15:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.371+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc14b860 con 0x7fdffc0720c0 2023-10-21T16:15:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.372+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc15ad70 con 0x7fdffc0720c0 2023-10-21T16:15:09.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.374+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdff0013470 con 0x7fdffc0720c0 2023-10-21T16:15:09.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.375+0000 7fdfeaffd640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 0x7fdfdc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:09.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.375+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fdff0089e60 con 0x7fdffc0720c0 2023-10-21T16:15:09.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.375+0000 7fdffa59c640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 0x7fdfdc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:09.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.376+0000 7fdffa59c640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 0x7fdfdc06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000f60 tx=0x7fdfec007ea0 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:09.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.378+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdff0054f50 con 0x7fdffc0720c0 2023-10-21T16:15:09.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:09.584+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdffc15ad70 con 0x7fdffc0720c0 2023-10-21T16:15:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.069+0000 7fdfeaffd640 1 -- 172.21.15.53:0/105493861 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdffc15ad70 con 0x7fdffc0720c0 2023-10-21T16:15:10.072 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:15:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 msgr2=0x7fdfdc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 0x7fdfdc06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfec000f60 tx=0x7fdfec007ea0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 msgr2=0x7fdffc1f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 0x7fdffc1f20a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e870 tx=0x7fdff000ed30 comp rx=0 tx=0).stop 2023-10-21T16:15:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 shutdown_connections 2023-10-21T16:15:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fdfdc06af90 0x7fdfdc06d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdffc1601b0 0x7fdffc14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdffc0720c0 0x7fdffc1f20a0 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdff000e870 tx=0x7fdff000ed30 comp rx=0 tx=0).stop 2023-10-21T16:15:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 --2- 172.21.15.53:0/105493861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdffc071c30 0x7fdffc0ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 >> 172.21.15.53:0/105493861 conn(0x7fdffc06d430 msgr2=0x7fdffc0caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 shutdown_connections 2023-10-21T16:15:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.074+0000 7fe000d89640 1 -- 172.21.15.53:0/105493861 wait complete. 2023-10-21T16:15:10.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:15:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.315+0000 7fc32d8df640 1 Processor -- start 2023-10-21T16:15:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.315+0000 7fc32d8df640 1 -- start start 2023-10-21T16:15:10.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d8df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc3280cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d8df640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc32806a460 0x7fc3280cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d8df640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32815b040 con 0x7fc3280d0120 2023-10-21T16:15:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d8df640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32815f4a0 con 0x7fc32806a460 2023-10-21T16:15:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d8df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815c060 con 0x7fc328155010 2023-10-21T16:15:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc327fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc3280d0120 0x7fc328069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55494/0 (socket says 172.21.15.53:55494) 2023-10-21T16:15:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc327fff640 1 -- 172.21.15.53:0/2677849543 learned_addr learned my addr 172.21.15.53:0/2677849543 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521422747 0 0) 0x7fc32815b040 con 0x7fc3280d0120 2023-10-21T16:15:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc318003520 con 0x7fc3280d0120 2023-10-21T16:15:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32c8dd640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc3280cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc32d0de640 1 --2- 172.21.15.53:0/2677849543 >> v2:172.21.15.124:3300/0 conn(0x7fc32806a460 0x7fc3280cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421395724 0 0) 0x7fc318003520 con 0x7fc3280d0120 2023-10-21T16:15:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 msgr2=0x7fc3280cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc3280cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 >> v2:172.21.15.124:3300/0 conn(0x7fc32806a460 msgr2=0x7fc3280cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 --2- 172.21.15.53:0/2677849543 >> v2:172.21.15.124:3300/0 conn(0x7fc32806a460 0x7fc3280cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32815bdc0 con 0x7fc3280d0120 2023-10-21T16:15:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc314001670 con 0x7fc3280d0120 2023-10-21T16:15:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 >> v1:172.21.15.53:6789/0 conn(0x7fc3280d0120 legacy=0x7fc328069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc318003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc318003ef0 0x7fc3180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc318005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815f4a0 con 0x7fc318004d00 2023-10-21T16:15:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31800b980 con 0x7fc318003ef0 2023-10-21T16:15:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.317+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31800bac0 con 0x7fc328155010 2023-10-21T16:15:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc318005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32c8dd640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc318003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc327fff640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc318003ef0 0x7fc3180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 -- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 msgr2=0x7fc318003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc328155010 0x7fc318003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 -- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc318003ef0 msgr2=0x7fc3180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc318003ef0 0x7fc3180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc32d0de640 1 -- 172.21.15.53:0/2677849543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318003520 con 0x7fc318004d00 2023-10-21T16:15:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.318+0000 7fc327fff640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc318003ef0 0x7fc3180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.319+0000 7fc32d0de640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc318005180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc320009f90 tx=0x7fc3180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02afe82ccad0ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.319+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32002cc00 con 0x7fc318004d00 2023-10-21T16:15:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.319+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32000f040 con 0x7fc318004d00 2023-10-21T16:15:10.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.319+0000 7fc3277fe640 1 -- 172.21.15.53:0/2677849543 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320034440 con 0x7fc318004d00 2023-10-21T16:15:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 -- 172.21.15.53:0/2677849543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 msgr2=0x7fc318005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2677849543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc318005180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc320009f90 tx=0x7fc3180057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d0de640 1 -- 172.21.15.53:0/2677849543 reap_dead start 2023-10-21T16:15:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 -- 172.21.15.53:0/2677849543 shutdown_connections 2023-10-21T16:15:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 -- 172.21.15.53:0/2677849543 >> 172.21.15.53:0/2677849543 conn(0x7fc328098a50 msgr2=0x7fc3280cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:10.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 -- 172.21.15.53:0/2677849543 shutdown_connections 2023-10-21T16:15:10.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.320+0000 7fc32d8df640 1 -- 172.21.15.53:0/2677849543 wait complete. 2023-10-21T16:15:10.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.322+0000 7fc32d8df640 1 Processor -- start 2023-10-21T16:15:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.322+0000 7fc32d8df640 1 -- start start 2023-10-21T16:15:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3280c7af0 0x7fc3281502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3280c67d0 0x7fc3280c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280c9100 con 0x7fc318004d00 2023-10-21T16:15:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280c76c0 con 0x7fc3280c67d0 2023-10-21T16:15:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32d8df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280c7190 con 0x7fc3280c7af0 2023-10-21T16:15:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32c8dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32c8dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57556/0 (socket says 172.21.15.53:57556) 2023-10-21T16:15:10.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.323+0000 7fc32c8dd640 1 -- 172.21.15.53:0/2517461289 learned_addr learned my addr 172.21.15.53:0/2517461289 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 -- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3280c7af0 msgr2=0x7fc3281502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc327fff640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3280c7af0 0x7fc3281502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3280c7af0 0x7fc3281502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 -- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3280c67d0 msgr2=0x7fc3280c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3280c67d0 0x7fc3280c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 -- 172.21.15.53:0/2517461289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328157400 con 0x7fc318004d00 2023-10-21T16:15:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32c8dd640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc31c011600 tx=0x7fc31c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c015870 con 0x7fc318004d00 2023-10-21T16:15:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c0159d0 con 0x7fc318004d00 2023-10-21T16:15:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c015d40 con 0x7fc318004d00 2023-10-21T16:15:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3280c8c70 con 0x7fc318004d00 2023-10-21T16:15:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.324+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3280c8340 con 0x7fc318004d00 2023-10-21T16:15:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.325+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32804e160 con 0x7fc318004d00 2023-10-21T16:15:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.326+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc31c013430 con 0x7fc318004d00 2023-10-21T16:15:10.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.327+0000 7fc3257fa640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 0x7fc30c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.327+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fc31c0898c0 con 0x7fc318004d00 2023-10-21T16:15:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.328+0000 7fc327fff640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 0x7fc30c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.329+0000 7fc327fff640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 0x7fc30c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3280c2250 tx=0x7fc314009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.331+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc31c0549a0 con 0x7fc318004d00 2023-10-21T16:15:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.598+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc328157400 con 0x7fc318004d00 2023-10-21T16:15:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.599+0000 7fc3257fa640 1 -- 172.21.15.53:0/2517461289 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc328157400 con 0x7fc318004d00 2023-10-21T16:15:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:10.602 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:15:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:15:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:15:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:15:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:15:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:15:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:15:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:15:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:15:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:15:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:15:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:15:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:15:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:15:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:15:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:15:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:15:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:15:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:15:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:15:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:15:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:15:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:15:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:15:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:15:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:15:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:15:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:15:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:15:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:15:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:15:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:15:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:15:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:15:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:15:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:15:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:15:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:15:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:15:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:15:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:15:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:15:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:15:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:15:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:15:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:15:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:15:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:15:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:15:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:15:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:15:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:15:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:15:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:15:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:15:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:15:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:15:10.638 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:15:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 msgr2=0x7fc30c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 0x7fc30c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc3280c2250 tx=0x7fc314009040 comp rx=0 tx=0).stop 2023-10-21T16:15:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 msgr2=0x7fc3281f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc31c011600 tx=0x7fc31c011ac0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 shutdown_connections 2023-10-21T16:15:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc30c06af90 0x7fc30c06d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3280c67d0 0x7fc3280c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3280c7af0 0x7fc3281502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 --2- 172.21.15.53:0/2517461289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc318004d00 0x7fc3281f2fa0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 >> 172.21.15.53:0/2517461289 conn(0x7fc328098a50 msgr2=0x7fc328159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 shutdown_connections 2023-10-21T16:15:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.602+0000 7fc32d8df640 1 -- 172.21.15.53:0/2517461289 wait complete. 2023-10-21T16:15:10.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:15:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.868+0000 7fb69b31d640 1 Processor -- start 2023-10-21T16:15:10.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.869+0000 7fb69b31d640 1 -- start start 2023-10-21T16:15:10.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb69415ab40 0x7fb6940ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940c90b0 0x7fb6940adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6940cae70 0x7fb6940cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69415bf30 con 0x7fb6940cae70 2023-10-21T16:15:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6940cb840 con 0x7fb69415ab40 2023-10-21T16:15:10.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb69b31d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415b2b0 con 0x7fb6940c90b0 2023-10-21T16:15:10.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.870+0000 7fb6998ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6940cae70 0x7fb6940cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6940cae70 0x7fb6940cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57566/0 (socket says 172.21.15.53:57566) 2023-10-21T16:15:10.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 -- 172.21.15.53:0/2648440134 learned_addr learned my addr 172.21.15.53:0/2648440134 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:10.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6990b9640 1 --2- 172.21.15.53:0/2648440134 >> v2:172.21.15.124:3300/0 conn(0x7fb69415ab40 0x7fb6940ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6988b8640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940c90b0 0x7fb6940adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 -- 172.21.15.53:0/2648440134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940c90b0 msgr2=0x7fb6940adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940c90b0 0x7fb6940adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 -- 172.21.15.53:0/2648440134 >> v2:172.21.15.124:3300/0 conn(0x7fb69415ab40 msgr2=0x7fb6940ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:10.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 --2- 172.21.15.53:0/2648440134 >> v2:172.21.15.124:3300/0 conn(0x7fb69415ab40 0x7fb6940ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.871+0000 7fb6998ba640 1 -- 172.21.15.53:0/2648440134 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69415b590 con 0x7fb6940cae70 2023-10-21T16:15:10.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.872+0000 7fb6998ba640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940cae70 0x7fb6940cb300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb69406e440 tx=0x7fb68c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad37b7dd9568cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb693fff640 1 -- 172.21.15.53:0/2648440134 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c031860 con 0x7fb6940cae70 2023-10-21T16:15:10.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb693fff640 1 -- 172.21.15.53:0/2648440134 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c0319c0 con 0x7fb6940cae70 2023-10-21T16:15:10.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb693fff640 1 -- 172.21.15.53:0/2648440134 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c031d50 con 0x7fb6940cae70 2023-10-21T16:15:10.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 -- 172.21.15.53:0/2648440134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940cae70 msgr2=0x7fb6940cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940cae70 0x7fb6940cb300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb69406e440 tx=0x7fb68c02b900 comp rx=0 tx=0).stop 2023-10-21T16:15:10.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 -- 172.21.15.53:0/2648440134 shutdown_connections 2023-10-21T16:15:10.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940c90b0 0x7fb6940adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2648440134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940cae70 0x7fb6940cb300 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2648440134 >> v2:172.21.15.124:3300/0 conn(0x7fb69415ab40 0x7fb6940ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.873+0000 7fb69b31d640 1 -- 172.21.15.53:0/2648440134 >> 172.21.15.53:0/2648440134 conn(0x7fb69406a380 msgr2=0x7fb6940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:10.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.874+0000 7fb69b31d640 1 -- 172.21.15.53:0/2648440134 shutdown_connections 2023-10-21T16:15:10.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.874+0000 7fb69b31d640 1 -- 172.21.15.53:0/2648440134 wait complete. 2023-10-21T16:15:10.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.875+0000 7fb69b31d640 1 Processor -- start 2023-10-21T16:15:10.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.876+0000 7fb69b31d640 1 -- start start 2023-10-21T16:15:10.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.876+0000 7fb69b31d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb69b31d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 0x7fb6940c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb69b31d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 0x7fb6940c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb69b31d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6941f12f0 con 0x7fb6940c90b0 2023-10-21T16:15:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb69b31d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6941f18d0 con 0x7fb69415ab40 2023-10-21T16:15:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb69b31d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6941f1bb0 con 0x7fb6940cae70 2023-10-21T16:15:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57570/0 (socket says 172.21.15.53:57570) 2023-10-21T16:15:10.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 -- 172.21.15.53:0/2548746267 learned_addr learned my addr 172.21.15.53:0/2548746267 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:10.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6988b8640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 0x7fb6940c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6998ba640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 0x7fb6940c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 -- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 msgr2=0x7fb6940c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 0x7fb6940c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 -- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 msgr2=0x7fb6940c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:10.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 0x7fb6940c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:10.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6990b9640 1 -- 172.21.15.53:0/2548746267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0094d0 con 0x7fb6940c90b0 2023-10-21T16:15:10.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6988b8640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 0x7fb6940c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:10.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.877+0000 7fb6998ba640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 0x7fb6940c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb6990b9640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb684007ef0 tx=0x7fb68400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684014620 con 0x7fb6940c90b0 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684014780 con 0x7fb6940c90b0 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684014b30 con 0x7fb6940c90b0 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6941f1e90 con 0x7fb6940c90b0 2023-10-21T16:15:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.878+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6941f15d0 con 0x7fb6940c90b0 2023-10-21T16:15:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.880+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb694067490 con 0x7fb6940c90b0 2023-10-21T16:15:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.880+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb684014c90 con 0x7fb6940c90b0 2023-10-21T16:15:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.881+0000 7fb691ffb640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 0x7fb68006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.881+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fb684089ff0 con 0x7fb6940c90b0 2023-10-21T16:15:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.882+0000 7fb6988b8640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 0x7fb68006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.883+0000 7fb6988b8640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 0x7fb68006d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb688006fd0 tx=0x7fb688008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:10.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:10.886+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb684055130 con 0x7fb6940c90b0 2023-10-21T16:15:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.082+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6940b0ef0 con 0x7fb6940c90b0 2023-10-21T16:15:11.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.083+0000 7fb691ffb640 1 -- 172.21.15.53:0/2548746267 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+13120 (secure 0 0 0) 0x7fb684058750 con 0x7fb6940c90b0 2023-10-21T16:15:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:11.085 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":68,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:15:10.082213+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:15:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 msgr2=0x7fb68006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 0x7fb68006d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb688006fd0 tx=0x7fb688008040 comp rx=0 tx=0).stop 2023-10-21T16:15:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 msgr2=0x7fb694148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb684007ef0 tx=0x7fb68400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 shutdown_connections 2023-10-21T16:15:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fb68006af90 0x7fb68006d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69415ab40 0x7fb6940c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6940cae70 0x7fb6940c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 --2- 172.21.15.53:0/2548746267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6940c90b0 0x7fb694148b20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 >> 172.21.15.53:0/2548746267 conn(0x7fb69406a380 msgr2=0x7fb6940af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 shutdown_connections 2023-10-21T16:15:11.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.086+0000 7fb69b31d640 1 -- 172.21.15.53:0/2548746267 wait complete. 2023-10-21T16:15:11.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:15:11.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:15:11.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.334+0000 7f4bdd26f640 1 Processor -- start 2023-10-21T16:15:11.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.334+0000 7f4bdd26f640 1 -- start start 2023-10-21T16:15:11.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.336+0000 7f4bdd26f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bd80b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.336+0000 7f4bdd26f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80b30a0 con 0x7f4bd80b1750 2023-10-21T16:15:11.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.336+0000 7f4bdd26f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80b3320 con 0x7f4bd80702d0 2023-10-21T16:15:11.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.336+0000 7f4bdd26f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd815dcf0 con 0x7f4bd815d710 2023-10-21T16:15:11.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd67fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4bd80b1750 0x7f4bd80c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55496/0 (socket says 172.21.15.53:55496) 2023-10-21T16:15:11.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd67fc640 1 -- 172.21.15.53:0/688618348 learned_addr learned my addr 172.21.15.53:0/688618348 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:11.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bd80b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053690896 0 0) 0x7f4bd80b30a0 con 0x7f4bd80b1750 2023-10-21T16:15:11.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc4003520 con 0x7f4bd80b1750 2023-10-21T16:15:11.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2772856312 0 0) 0x7f4bc4003520 con 0x7f4bd80b1750 2023-10-21T16:15:11.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 msgr2=0x7f4bd80b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:11.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bd80b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 >> v1:172.21.15.124:6789/0 conn(0x7f4bd80702d0 legacy=0x7f4bd80706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.337+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80b3600 con 0x7f4bd80b1750 2023-10-21T16:15:11.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4bc0001670 con 0x7f4bd80b1750 2023-10-21T16:15:11.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 >> v1:172.21.15.53:6789/0 conn(0x7f4bd80b1750 legacy=0x7f4bd80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc40039a0 0x7f4bc4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc4004d00 0x7f4bc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd815dcf0 con 0x7f4bc4004d00 2023-10-21T16:15:11.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc400dd50 con 0x7f4bc40039a0 2023-10-21T16:15:11.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc400df10 con 0x7f4bd815d710 2023-10-21T16:15:11.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc4004d00 0x7f4bc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd67fc640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd6ffd640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc40039a0 0x7f4bc4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.338+0000 7f4bd77fe640 1 -- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 msgr2=0x7f4bc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.339+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd815d710 0x7f4bc40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.339+0000 7f4bd77fe640 1 -- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc40039a0 msgr2=0x7f4bc4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.339+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc40039a0 0x7f4bc4003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.339+0000 7f4bd77fe640 1 -- 172.21.15.53:0/688618348 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4003520 con 0x7f4bc4004d00 2023-10-21T16:15:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc4004d00 0x7f4bc4005180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000a250 tx=0x7f4bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2931f47378887171 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd002cd80 con 0x7f4bc4004d00 2023-10-21T16:15:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd0030070 con 0x7f4bc4004d00 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bd5ffb640 1 -- 172.21.15.53:0/688618348 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0036620 con 0x7f4bc4004d00 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bdd26f640 1 -- 172.21.15.53:0/688618348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc4004d00 msgr2=0x7f4bc4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/688618348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bc4004d00 0x7f4bc4005180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000a250 tx=0x7f4bc40057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.340+0000 7f4bd77fe640 1 -- 172.21.15.53:0/688618348 reap_dead start 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.341+0000 7f4bdd26f640 1 -- 172.21.15.53:0/688618348 shutdown_connections 2023-10-21T16:15:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.341+0000 7f4bdd26f640 1 -- 172.21.15.53:0/688618348 >> 172.21.15.53:0/688618348 conn(0x7f4bd8069d50 msgr2=0x7f4bd806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.341+0000 7f4bdd26f640 1 -- 172.21.15.53:0/688618348 shutdown_connections 2023-10-21T16:15:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.341+0000 7f4bdd26f640 1 -- 172.21.15.53:0/688618348 wait complete. 2023-10-21T16:15:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.342+0000 7f4bdd26f640 1 Processor -- start 2023-10-21T16:15:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.343+0000 7f4bdd26f640 1 -- start start 2023-10-21T16:15:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc4004d00 0x7f4bd8158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 0x7f4bd81f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd81585a0 con 0x7f4bd815b7f0 2023-10-21T16:15:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8150a70 con 0x7f4bc4004d00 2023-10-21T16:15:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bdd26f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80c8d80 con 0x7f4bd81502f0 2023-10-21T16:15:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57590/0 (socket says 172.21.15.53:57590) 2023-10-21T16:15:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 -- 172.21.15.53:0/821246434 learned_addr learned my addr 172.21.15.53:0/821246434 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd67fc640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 0x7f4bd81f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 -- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 msgr2=0x7f4bd81f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 0x7f4bd81f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd6ffd640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc4004d00 0x7f4bd8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 -- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc4004d00 msgr2=0x7f4bd8158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.344+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc4004d00 0x7f4bd8158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bd77fe640 1 -- 172.21.15.53:0/821246434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80c9060 con 0x7f4bd815b7f0 2023-10-21T16:15:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bd67fc640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 0x7f4bd81f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bd77fe640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0038040 tx=0x7f4bd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0036e50 con 0x7f4bd815b7f0 2023-10-21T16:15:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd0041470 con 0x7f4bd815b7f0 2023-10-21T16:15:11.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0041800 con 0x7f4bd815b7f0 2023-10-21T16:15:11.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd80c9340 con 0x7f4bd815b7f0 2023-10-21T16:15:11.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.345+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd81f2610 con 0x7f4bd815b7f0 2023-10-21T16:15:11.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.346+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd80c8ec0 con 0x7f4bd815b7f0 2023-10-21T16:15:11.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.348+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4bd0041960 con 0x7f4bd815b7f0 2023-10-21T16:15:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.349+0000 7f4bcf7fe640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 0x7f4bb806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.349+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f4bd00ac070 con 0x7f4bd815b7f0 2023-10-21T16:15:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.349+0000 7f4bd6ffd640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 0x7f4bb806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.351+0000 7f4bd6ffd640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 0x7f4bb806d4d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001590 tx=0x7f4bc8009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.353+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bd80c8ec0 con 0x7f4bd815b7f0 2023-10-21T16:15:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.546+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bd80c9860 con 0x7f4bd815b7f0 2023-10-21T16:15:11.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.547+0000 7f4bcf7fe640 1 -- 172.21.15.53:0/821246434 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bd80c9860 con 0x7f4bd815b7f0 2023-10-21T16:15:11.549 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:15:11.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 msgr2=0x7f4bb806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 0x7f4bb806d4d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001590 tx=0x7f4bc8009040 comp rx=0 tx=0).stop 2023-10-21T16:15:11.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 msgr2=0x7f4bd8148db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0038040 tx=0x7f4bd0004a40 comp rx=0 tx=0).stop 2023-10-21T16:15:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 shutdown_connections 2023-10-21T16:15:11.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f4bb806b010 0x7f4bb806d4d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bd815b7f0 0x7f4bd8148db0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bd81502f0 0x7f4bd81f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 --2- 172.21.15.53:0/821246434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bc4004d00 0x7f4bd8158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 >> 172.21.15.53:0/821246434 conn(0x7f4bd8069d50 msgr2=0x7f4bd8159530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:11.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 shutdown_connections 2023-10-21T16:15:11.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.550+0000 7f4bdd26f640 1 -- 172.21.15.53:0/821246434 wait complete. 2023-10-21T16:15:11.572 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:15:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.799+0000 7f266b02e640 1 Processor -- start 2023-10-21T16:15:11.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.799+0000 7f266b02e640 1 -- start start 2023-10-21T16:15:11.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266b02e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26640adc70 0x7f26640b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266b02e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415af10 0x7f26640b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266b02e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266415f7d0 con 0x7f266415ab30 2023-10-21T16:15:11.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266b02e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415efb0 con 0x7f26640adc70 2023-10-21T16:15:11.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266b02e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266415f210 con 0x7f266415af10 2023-10-21T16:15:11.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266982b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26640adc70 0x7f26640b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266a02c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f266415ab30 0x7f266415e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55502/0 (socket says 172.21.15.53:55502) 2023-10-21T16:15:11.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.801+0000 7f266a02c640 1 -- 172.21.15.53:0/2858391206 learned_addr learned my addr 172.21.15.53:0/2858391206 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:11.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.802+0000 7f266a82d640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415af10 0x7f26640b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.802+0000 7f266982b640 1 -- 172.21.15.53:0/2858391206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415af10 msgr2=0x7f26640b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:11.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.802+0000 7f266982b640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415af10 0x7f26640b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.802+0000 7f266982b640 1 -- 172.21.15.53:0/2858391206 >> v1:172.21.15.53:6789/0 conn(0x7f266415ab30 legacy=0x7f266415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.802+0000 7f266982b640 1 -- 172.21.15.53:0/2858391206 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266415f4f0 con 0x7f26640adc70 2023-10-21T16:15:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.803+0000 7f266982b640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640b00c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26540097a0 tx=0x7f265402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cc5ba337f73a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266902a640 1 -- 172.21.15.53:0/2858391206 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265402d040 con 0x7f26640adc70 2023-10-21T16:15:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266902a640 1 -- 172.21.15.53:0/2858391206 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654031810 con 0x7f26640adc70 2023-10-21T16:15:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266902a640 1 -- 172.21.15.53:0/2858391206 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654031ba0 con 0x7f26640adc70 2023-10-21T16:15:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266b02e640 1 -- 172.21.15.53:0/2858391206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 msgr2=0x7f26640b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266b02e640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640b00c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26540097a0 tx=0x7f265402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266b02e640 1 -- 172.21.15.53:0/2858391206 shutdown_connections 2023-10-21T16:15:11.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266b02e640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415af10 0x7f26640b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.804+0000 7f266b02e640 1 --2- 172.21.15.53:0/2858391206 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640b00c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.805+0000 7f266b02e640 1 -- 172.21.15.53:0/2858391206 >> 172.21.15.53:0/2858391206 conn(0x7f26640cb0c0 msgr2=0x7f26640cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.805+0000 7f266b02e640 1 -- 172.21.15.53:0/2858391206 shutdown_connections 2023-10-21T16:15:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.805+0000 7f266b02e640 1 -- 172.21.15.53:0/2858391206 wait complete. 2023-10-21T16:15:11.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.807+0000 7f266b02e640 1 Processor -- start 2023-10-21T16:15:11.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.807+0000 7f266b02e640 1 -- start start 2023-10-21T16:15:11.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 0x7f26640cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f266415af10 0x7f26640b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640ab8d0 con 0x7f26640adc70 2023-10-21T16:15:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26641f35a0 con 0x7f266415af10 2023-10-21T16:15:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266b02e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640abae0 con 0x7f266415ab30 2023-10-21T16:15:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266a02c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266a02c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57610/0 (socket says 172.21.15.53:57610) 2023-10-21T16:15:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.809+0000 7f266a02c640 1 -- 172.21.15.53:0/2432647465 learned_addr learned my addr 172.21.15.53:0/2432647465 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266982b640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 0x7f26640cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 -- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 msgr2=0x7f26640cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 0x7f26640cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 -- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f266415af10 msgr2=0x7f26640b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f266415af10 0x7f26640b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:11.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 -- 172.21.15.53:0/2432647465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2654009400 con 0x7f26640adc70 2023-10-21T16:15:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266982b640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 0x7f26640cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266a02c640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f266000c450 tx=0x7f266000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660005ce0 con 0x7f26640adc70 2023-10-21T16:15:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660005e40 con 0x7f26640adc70 2023-10-21T16:15:11.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600072b0 con 0x7f26640adc70 2023-10-21T16:15:11.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.810+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640abdc0 con 0x7f26640adc70 2023-10-21T16:15:11.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.811+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640b2e40 con 0x7f26640adc70 2023-10-21T16:15:11.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.812+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26640ac2e0 con 0x7f26640adc70 2023-10-21T16:15:11.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.813+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2660007410 con 0x7f26640adc70 2023-10-21T16:15:11.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.814+0000 7f265affd640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 0x7f264806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:11.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.814+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f2660088a40 con 0x7f26640adc70 2023-10-21T16:15:11.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.814+0000 7f266982b640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 0x7f264806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:11.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.816+0000 7f266982b640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 0x7f264806d4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2654036000 tx=0x7f265403c040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:11.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:11.820+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26640ac2e0 con 0x7f26640adc70 2023-10-21T16:15:12.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.015+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f26640ac2e0 con 0x7f26640adc70 2023-10-21T16:15:12.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.016+0000 7f265affd640 1 -- 172.21.15.53:0/2432647465 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2660017020 con 0x7f26640adc70 2023-10-21T16:15:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:12.018 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:15:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:15:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:15:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:12.020 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:15:12.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 msgr2=0x7f264806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:12.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 0x7f264806d4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2654036000 tx=0x7f265403c040 comp rx=0 tx=0).stop 2023-10-21T16:15:12.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 msgr2=0x7f26640ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:12.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f266000c450 tx=0x7f266000c910 comp rx=0 tx=0).stop 2023-10-21T16:15:12.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 shutdown_connections 2023-10-21T16:15:12.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f264806afe0 0x7f264806d4a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f266415af10 0x7f26640b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f266415ab30 0x7f26640cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 --2- 172.21.15.53:0/2432647465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26640adc70 0x7f26640ac5c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 >> 172.21.15.53:0/2432647465 conn(0x7f26640cb0c0 msgr2=0x7f26640b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:12.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 shutdown_connections 2023-10-21T16:15:12.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.019+0000 7f266b02e640 1 -- 172.21.15.53:0/2432647465 wait complete. 2023-10-21T16:15:12.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:15:12.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:15:12.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.273+0000 7f64bd3c7640 1 Processor -- start 2023-10-21T16:15:12.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.274+0000 7f64bd3c7640 1 -- start start 2023-10-21T16:15:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.275+0000 7f64bd3c7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 0x7f64b8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.275+0000 7f64bd3c7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64b8071260 0x7f64b80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64bd3c7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80c7ad0 con 0x7f64b80c9080 2023-10-21T16:15:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64bd3c7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b80c9a40 con 0x7f64b8071260 2023-10-21T16:15:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64bd3c7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b815ac00 con 0x7f64b815a820 2023-10-21T16:15:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64b8071260 0x7f64b80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:12.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64b8071260 0x7f64b80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57622/0 (socket says 172.21.15.53:57622) 2023-10-21T16:15:12.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64b77fe640 1 -- 172.21.15.53:0/2183625385 learned_addr learned my addr 172.21.15.53:0/2183625385 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:12.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.276+0000 7f64b6ffd640 1 --2- 172.21.15.53:0/2183625385 >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 0x7f64b8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.277+0000 7f64b77fe640 1 -- 172.21.15.53:0/2183625385 >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 msgr2=0x7f64b8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.277+0000 7f64b77fe640 1 --2- 172.21.15.53:0/2183625385 >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 0x7f64b8160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.277+0000 7f64b77fe640 1 -- 172.21.15.53:0/2183625385 >> v1:172.21.15.124:6789/0 conn(0x7f64b80c9080 legacy=0x7f64b8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.277+0000 7f64b77fe640 1 -- 172.21.15.53:0/2183625385 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80cb210 con 0x7f64b8071260 2023-10-21T16:15:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.277+0000 7f64b6ffd640 1 --2- 172.21.15.53:0/2183625385 >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 0x7f64b8160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.278+0000 7f64b77fe640 1 --2- 172.21.15.53:0/2183625385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b80c9500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f64b806acb0 tx=0x7f64b002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b88ed2306f6497 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.278+0000 7f64b5ffb640 1 -- 172.21.15.53:0/2183625385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0031890 con 0x7f64b8071260 2023-10-21T16:15:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.278+0000 7f64b5ffb640 1 -- 172.21.15.53:0/2183625385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b00319f0 con 0x7f64b8071260 2023-10-21T16:15:12.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.278+0000 7f64b5ffb640 1 -- 172.21.15.53:0/2183625385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0031d80 con 0x7f64b8071260 2023-10-21T16:15:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2183625385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 msgr2=0x7f64b80c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2183625385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b80c9500 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f64b806acb0 tx=0x7f64b002b930 comp rx=0 tx=0).stop 2023-10-21T16:15:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2183625385 shutdown_connections 2023-10-21T16:15:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2183625385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b80c9500 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2183625385 >> v2:172.21.15.124:3300/0 conn(0x7f64b815a820 0x7f64b8160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2183625385 >> 172.21.15.53:0/2183625385 conn(0x7f64b8069d50 msgr2=0x7f64b806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2183625385 shutdown_connections 2023-10-21T16:15:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.279+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2183625385 wait complete. 2023-10-21T16:15:12.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.281+0000 7f64bd3c7640 1 Processor -- start 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.281+0000 7f64bd3c7640 1 -- start start 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 0x7f64b815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64b815a820 0x7f64b8158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b1540 con 0x7f64b8071260 2023-10-21T16:15:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b1820 con 0x7f64b815a820 2023-10-21T16:15:12.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64bd3c7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80b1b00 con 0x7f64b80c9080 2023-10-21T16:15:12.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:12.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57630/0 (socket says 172.21.15.53:57630) 2023-10-21T16:15:12.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.282+0000 7f64b6ffd640 1 -- 172.21.15.53:0/2785160077 learned_addr learned my addr 172.21.15.53:0/2785160077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 -- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 msgr2=0x7f64b815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b67fc640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 0x7f64b815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 0x7f64b815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 -- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64b815a820 msgr2=0x7f64b8158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64b815a820 0x7f64b8158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:12.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 -- 172.21.15.53:0/2785160077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b00094d0 con 0x7f64b8071260 2023-10-21T16:15:12.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b67fc640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 0x7f64b815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:12.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64b6ffd640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c7f0 tx=0x7f64a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:12.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0007820 con 0x7f64b8071260 2023-10-21T16:15:12.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a0007980 con 0x7f64b8071260 2023-10-21T16:15:12.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0007cf0 con 0x7f64b8071260 2023-10-21T16:15:12.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.283+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80b1de0 con 0x7f64b8071260 2023-10-21T16:15:12.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.284+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b815e5e0 con 0x7f64b8071260 2023-10-21T16:15:12.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.285+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b80cba90 con 0x7f64b8071260 2023-10-21T16:15:12.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.286+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64a00113f0 con 0x7f64b8071260 2023-10-21T16:15:12.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.287+0000 7f64af7fe640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 0x7f649806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:12.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.287+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f64a0089aa0 con 0x7f64b8071260 2023-10-21T16:15:12.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.287+0000 7f64b67fc640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 0x7f649806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:12.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.289+0000 7f64b67fc640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 0x7f649806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64b815d6a0 tx=0x7f64a8009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:12.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.291+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64b80cba90 con 0x7f64b8071260 2023-10-21T16:15:12.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:12.487+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f64b80cba90 con 0x7f64b8071260 2023-10-21T16:15:13.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.089+0000 7f64af7fe640 1 -- 172.21.15.53:0/2785160077 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7f64b80cba90 con 0x7f64b8071260 2023-10-21T16:15:13.091 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:15:13.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 msgr2=0x7f649806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 0x7f649806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f64b815d6a0 tx=0x7f64a8009040 comp rx=0 tx=0).stop 2023-10-21T16:15:13.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 msgr2=0x7f64b815e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f64a000c7f0 tx=0x7f64a000ccb0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 shutdown_connections 2023-10-21T16:15:13.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f649806af90 0x7f649806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64b815a820 0x7f64b8158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64b80c9080 0x7f64b815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 --2- 172.21.15.53:0/2785160077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64b8071260 0x7f64b815e0a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 >> 172.21.15.53:0/2785160077 conn(0x7f64b8069d50 msgr2=0x7f64b815c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.093+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 shutdown_connections 2023-10-21T16:15:13.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.094+0000 7f64bd3c7640 1 -- 172.21.15.53:0/2785160077 wait complete. 2023-10-21T16:15:13.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:15:13.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.336+0000 7f687ce40640 1 Processor -- start 2023-10-21T16:15:13.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.336+0000 7f687ce40640 1 -- start start 2023-10-21T16:15:13.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f687ce40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 0x7f68780ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f687ce40640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f687815e490 0x7f687815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f687ce40640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68780c9c90 con 0x7f687815de70 2023-10-21T16:15:13.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f687ce40640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687815f480 con 0x7f687815e490 2023-10-21T16:15:13.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f687ce40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687815e250 con 0x7f68780cb970 2023-10-21T16:15:13.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.337+0000 7f68777fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f687815de70 0x7f68780cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55510/0 (socket says 172.21.15.53:55510) 2023-10-21T16:15:13.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68777fe640 1 -- 172.21.15.53:0/1660139612 learned_addr learned my addr 172.21.15.53:0/1660139612 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:13.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f6877fff640 1 --2- 172.21.15.53:0/1660139612 >> v2:172.21.15.124:3300/0 conn(0x7f687815e490 0x7f687815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 0x7f68780ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455190911 0 0) 0x7f68780c9c90 con 0x7f687815de70 2023-10-21T16:15:13.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6864003520 con 0x7f687815de70 2023-10-21T16:15:13.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3604339746 0 0) 0x7f6864003520 con 0x7f687815de70 2023-10-21T16:15:13.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 msgr2=0x7f68780ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:13.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 0x7f68780ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 >> v2:172.21.15.124:3300/0 conn(0x7f687815e490 msgr2=0x7f687815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:13.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 --2- 172.21.15.53:0/1660139612 >> v2:172.21.15.124:3300/0 conn(0x7f687815e490 0x7f687815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687815f1a0 con 0x7f687815de70 2023-10-21T16:15:13.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.338+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f68680019f0 con 0x7f687815de70 2023-10-21T16:15:13.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 >> v1:172.21.15.53:6789/0 conn(0x7f687815de70 legacy=0x7f68780cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6864003970 0x7f6864003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f6864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 0x7f6864005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687815f480 con 0x7f68640047c0 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687815e250 con 0x7f6864003970 2023-10-21T16:15:13.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686400b980 con 0x7f68780cb970 2023-10-21T16:15:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f6864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 -- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 msgr2=0x7f6864005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68780cb970 0x7f6864005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 -- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6864003970 msgr2=0x7f6864003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:13.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6864003970 0x7f6864003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.339+0000 7f6876ffd640 1 -- 172.21.15.53:0/1660139612 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864003520 con 0x7f68640047c0 2023-10-21T16:15:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.340+0000 7f6876ffd640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f6864004c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ac0 tx=0x7f68640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da7d094d256b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686002c710 con 0x7f68640047c0 2023-10-21T16:15:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686002c870 con 0x7f68640047c0 2023-10-21T16:15:13.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f68767fc640 1 -- 172.21.15.53:0/1660139612 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686002cc00 con 0x7f68640047c0 2023-10-21T16:15:13.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f687ce40640 1 -- 172.21.15.53:0/1660139612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 msgr2=0x7f6864004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f687ce40640 1 --2- 172.21.15.53:0/1660139612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f6864004c40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6860009ac0 tx=0x7f68640057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f6877fff640 1 -- 172.21.15.53:0/1660139612 reap_dead start 2023-10-21T16:15:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f687ce40640 1 -- 172.21.15.53:0/1660139612 shutdown_connections 2023-10-21T16:15:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.341+0000 7f687ce40640 1 -- 172.21.15.53:0/1660139612 >> 172.21.15.53:0/1660139612 conn(0x7f68780ce9f0 msgr2=0x7f68780cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.342+0000 7f687ce40640 1 -- 172.21.15.53:0/1660139612 shutdown_connections 2023-10-21T16:15:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.342+0000 7f687ce40640 1 -- 172.21.15.53:0/1660139612 wait complete. 2023-10-21T16:15:13.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.343+0000 7f687ce40640 1 Processor -- start 2023-10-21T16:15:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.343+0000 7f687ce40640 1 -- start start 2023-10-21T16:15:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f687815bf80 0x7f687815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68781f28a0 0x7f68781f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687815c940 con 0x7f68640047c0 2023-10-21T16:15:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780cd3f0 con 0x7f687815bf80 2023-10-21T16:15:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.344+0000 7f687ce40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68781f5980 con 0x7f68781f28a0 2023-10-21T16:15:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57646/0 (socket says 172.21.15.53:57646) 2023-10-21T16:15:13.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 -- 172.21.15.53:0/316069830 learned_addr learned my addr 172.21.15.53:0/316069830 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 -- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68781f28a0 msgr2=0x7f68781f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68781f28a0 0x7f68781f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f6876ffd640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f687815bf80 0x7f687815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 -- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f687815bf80 msgr2=0x7f687815c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:13.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f687815bf80 0x7f687815c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 -- 172.21.15.53:0/316069830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68781f5c60 con 0x7f68640047c0 2023-10-21T16:15:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.345+0000 7f68777fe640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f686800ca20 tx=0x7f686800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.346+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800a7c0 con 0x7f68640047c0 2023-10-21T16:15:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.346+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800a940 con 0x7f68640047c0 2023-10-21T16:15:13.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.346+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800acf0 con 0x7f68640047c0 2023-10-21T16:15:13.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.346+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68781f5f40 con 0x7f68640047c0 2023-10-21T16:15:13.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.346+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68781f3260 con 0x7f68640047c0 2023-10-21T16:15:13.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.347+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68781f6460 con 0x7f68640047c0 2023-10-21T16:15:13.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.348+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68680070d0 con 0x7f68640047c0 2023-10-21T16:15:13.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.349+0000 7f686ffff640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 0x7f685806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:13.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.349+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f6868088f50 con 0x7f68640047c0 2023-10-21T16:15:13.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.350+0000 7f6876ffd640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 0x7f685806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.351+0000 7f6876ffd640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 0x7f685806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6860036000 tx=0x7f6860030000 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:13.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.354+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68781f6460 con 0x7f68640047c0 2023-10-21T16:15:13.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:13.549+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68781f6460 con 0x7f68640047c0 2023-10-21T16:15:14.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.096+0000 7f686ffff640 1 -- 172.21.15.53:0/316069830 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f68781f6460 con 0x7f68640047c0 2023-10-21T16:15:14.098 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:15:14.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 msgr2=0x7f685806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:14.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 0x7f685806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6860036000 tx=0x7f6860030000 comp rx=0 tx=0).stop 2023-10-21T16:15:14.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 msgr2=0x7f68780cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:14.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f686800ca20 tx=0x7f686800cee0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 shutdown_connections 2023-10-21T16:15:14.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f685806afc0 0x7f685806d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68781f28a0 0x7f68781f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f687815bf80 0x7f687815c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 --2- 172.21.15.53:0/316069830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68640047c0 0x7f68780cca50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.101+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 >> 172.21.15.53:0/316069830 conn(0x7f68780ce9f0 msgr2=0x7f68780abf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:14.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.102+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 shutdown_connections 2023-10-21T16:15:14.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.102+0000 7f687ce40640 1 -- 172.21.15.53:0/316069830 wait complete. 2023-10-21T16:15:14.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:15:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.329+0000 7f151431d640 1 Processor -- start 2023-10-21T16:15:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.330+0000 7f151431d640 1 -- start start 2023-10-21T16:15:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f151431d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f150c0afbe0 0x7f150c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f151431d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f150c0cb130 0x7f150c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f151431d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0cbad0 con 0x7f150c06dfc0 2023-10-21T16:15:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f151431d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0b2620 con 0x7f150c0cb130 2023-10-21T16:15:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f151431d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0b2060 con 0x7f150c0afbe0 2023-10-21T16:15:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f15128ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f150c0cb130 0x7f150c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:14.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f15128ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f150c0cb130 0x7f150c0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56268/0 (socket says 172.21.15.53:56268) 2023-10-21T16:15:14.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.331+0000 7f15128ba640 1 -- 172.21.15.53:0/1106658516 learned_addr learned my addr 172.21.15.53:0/1106658516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.332+0000 7f15128ba640 1 -- 172.21.15.53:0/1106658516 >> v2:172.21.15.124:3300/0 conn(0x7f150c0afbe0 msgr2=0x7f150c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.332+0000 7f15128ba640 1 --2- 172.21.15.53:0/1106658516 >> v2:172.21.15.124:3300/0 conn(0x7f150c0afbe0 0x7f150c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.332+0000 7f15128ba640 1 -- 172.21.15.53:0/1106658516 >> v1:172.21.15.124:6789/0 conn(0x7f150c06dfc0 legacy=0x7f150c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.332+0000 7f15128ba640 1 -- 172.21.15.53:0/1106658516 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0b2340 con 0x7f150c0cb130 2023-10-21T16:15:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.333+0000 7f15128ba640 1 --2- 172.21.15.53:0/1106658516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0cb130 0x7f150c0cb590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f150c06e640 tx=0x7f150402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e25b3c9e211bb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.333+0000 7f15110b7640 1 -- 172.21.15.53:0/1106658516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150402ca80 con 0x7f150c0cb130 2023-10-21T16:15:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f15110b7640 1 -- 172.21.15.53:0/1106658516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150402cbe0 con 0x7f150c0cb130 2023-10-21T16:15:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f15110b7640 1 -- 172.21.15.53:0/1106658516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040343f0 con 0x7f150c0cb130 2023-10-21T16:15:14.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 -- 172.21.15.53:0/1106658516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0cb130 msgr2=0x7f150c0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 --2- 172.21.15.53:0/1106658516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0cb130 0x7f150c0cb590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f150c06e640 tx=0x7f150402bee0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 -- 172.21.15.53:0/1106658516 shutdown_connections 2023-10-21T16:15:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 --2- 172.21.15.53:0/1106658516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0cb130 0x7f150c0cb590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 --2- 172.21.15.53:0/1106658516 >> v2:172.21.15.124:3300/0 conn(0x7f150c0afbe0 0x7f150c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 -- 172.21.15.53:0/1106658516 >> 172.21.15.53:0/1106658516 conn(0x7f150c06a5d0 msgr2=0x7f150c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:14.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 -- 172.21.15.53:0/1106658516 shutdown_connections 2023-10-21T16:15:14.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.334+0000 7f151431d640 1 -- 172.21.15.53:0/1106658516 wait complete. 2023-10-21T16:15:14.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.336+0000 7f151431d640 1 Processor -- start 2023-10-21T16:15:14.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.336+0000 7f151431d640 1 -- start start 2023-10-21T16:15:14.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150c06dfc0 0x7f150c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f150c0cb130 0x7f150c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0cd0c0 con 0x7f150c0afbe0 2023-10-21T16:15:14.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0cd3a0 con 0x7f150c0cb130 2023-10-21T16:15:14.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.337+0000 7f151431d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1542d0 con 0x7f150c06dfc0 2023-10-21T16:15:14.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:14.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56270/0 (socket says 172.21.15.53:56270) 2023-10-21T16:15:14.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 -- 172.21.15.53:0/4216762014 learned_addr learned my addr 172.21.15.53:0/4216762014 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:14.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15120b9640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150c06dfc0 0x7f150c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:14.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15128ba640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f150c0cb130 0x7f150c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:14.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 -- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150c06dfc0 msgr2=0x7f150c14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:14.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150c06dfc0 0x7f150c14eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 -- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f150c0cb130 msgr2=0x7f150c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f150c0cb130 0x7f150c15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.338+0000 7f15118b8640 1 -- 172.21.15.53:0/4216762014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009b30 con 0x7f150c0afbe0 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f15118b8640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3c0 tx=0x7f14fc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011830 con 0x7f150c0afbe0 2023-10-21T16:15:14.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc011990 con 0x7f150c0afbe0 2023-10-21T16:15:14.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011d60 con 0x7f150c0afbe0 2023-10-21T16:15:14.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c14f0b0 con 0x7f150c0afbe0 2023-10-21T16:15:14.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.339+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c1564c0 con 0x7f150c0afbe0 2023-10-21T16:15:14.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.340+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c04e160 con 0x7f150c0afbe0 2023-10-21T16:15:14.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.342+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f14fc01b460 con 0x7f150c0afbe0 2023-10-21T16:15:14.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.343+0000 7f1502ffd640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 0x7f14f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:14.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.343+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f14fc08a200 con 0x7f150c0afbe0 2023-10-21T16:15:14.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.343+0000 7f15120b9640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 0x7f14f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:14.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.345+0000 7f15120b9640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 0x7f14f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1508009890 tx=0x7f1508008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:14.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.347+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14fc055070 con 0x7f150c0afbe0 2023-10-21T16:15:14.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:14.578+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f150c0cd200 con 0x7f150c0afbe0 2023-10-21T16:15:15.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.152+0000 7f1502ffd640 1 -- 172.21.15.53:0/4216762014 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7f150c0cd200 con 0x7f150c0afbe0 2023-10-21T16:15:15.158 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 7 and data pool 8 2023-10-21T16:15:15.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.163+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 msgr2=0x7f14f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.163+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 0x7f14f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1508009890 tx=0x7f1508008040 comp rx=0 tx=0).stop 2023-10-21T16:15:15.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.163+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 msgr2=0x7f150c15c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.163+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3c0 tx=0x7f14fc00b880 comp rx=0 tx=0).stop 2023-10-21T16:15:15.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 shutdown_connections 2023-10-21T16:15:15.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f14f006af90 0x7f14f006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f150c0cb130 0x7f150c15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150c0afbe0 0x7f150c15c780 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 --2- 172.21.15.53:0/4216762014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150c06dfc0 0x7f150c14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.164+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 >> 172.21.15.53:0/4216762014 conn(0x7f150c06a5d0 msgr2=0x7f150c0c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:15.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.165+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 shutdown_connections 2023-10-21T16:15:15.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.166+0000 7f151431d640 1 -- 172.21.15.53:0/4216762014 wait complete. 2023-10-21T16:15:15.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:15:15.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.382+0000 7f64fd040640 1 Processor -- start 2023-10-21T16:15:15.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.383+0000 7f64fd040640 1 -- start start 2023-10-21T16:15:15.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64fd040640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64f815ab30 0x7f64f815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64fd040640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 0x7f64f806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64fd040640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f815f7d0 con 0x7f64f80adc70 2023-10-21T16:15:15.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64fd040640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f815f210 con 0x7f64f815ab30 2023-10-21T16:15:15.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64fd040640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f815f4f0 con 0x7f64f80b07d0 2023-10-21T16:15:15.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64effff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f64f80adc70 0x7f64f80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51706/0 (socket says 172.21.15.53:51706) 2023-10-21T16:15:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64f815ab30 0x7f64f815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.384+0000 7f64effff640 1 -- 172.21.15.53:0/1359640665 learned_addr learned my addr 172.21.15.53:0/1359640665 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64fc83f640 1 --2- 172.21.15.53:0/1359640665 >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 0x7f64f806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f77fe640 1 -- 172.21.15.53:0/1359640665 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614009127 0 0) 0x7f64f815f7d0 con 0x7f64f80adc70 2023-10-21T16:15:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f77fe640 1 -- 172.21.15.53:0/1359640665 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e4003520 con 0x7f64f80adc70 2023-10-21T16:15:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f7fff640 1 -- 172.21.15.53:0/1359640665 >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 msgr2=0x7f64f806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f7fff640 1 --2- 172.21.15.53:0/1359640665 >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 0x7f64f806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f7fff640 1 -- 172.21.15.53:0/1359640665 >> v1:172.21.15.53:6789/0 conn(0x7f64f80adc70 legacy=0x7f64f80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64f7fff640 1 -- 172.21.15.53:0/1359640665 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f806c550 con 0x7f64f815ab30 2023-10-21T16:15:15.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.385+0000 7f64fc83f640 1 --2- 172.21.15.53:0/1359640665 >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 0x7f64f806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:15:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64f7fff640 1 --2- 172.21.15.53:0/1359640665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f815e8a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f64f80ca4a0 tx=0x7f64e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2644d6de4a06ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64f77fe640 1 -- 172.21.15.53:0/1359640665 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8031830 con 0x7f64f815ab30 2023-10-21T16:15:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64f77fe640 1 -- 172.21.15.53:0/1359640665 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e8031990 con 0x7f64f815ab30 2023-10-21T16:15:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64f77fe640 1 -- 172.21.15.53:0/1359640665 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8031d20 con 0x7f64f815ab30 2023-10-21T16:15:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64fd040640 1 -- 172.21.15.53:0/1359640665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 msgr2=0x7f64f815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.387+0000 7f64fd040640 1 --2- 172.21.15.53:0/1359640665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f815e8a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f64f80ca4a0 tx=0x7f64e802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 -- 172.21.15.53:0/1359640665 shutdown_connections 2023-10-21T16:15:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 --2- 172.21.15.53:0/1359640665 >> v2:172.21.15.124:3300/0 conn(0x7f64f80b07d0 0x7f64f806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 --2- 172.21.15.53:0/1359640665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f815e8a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 -- 172.21.15.53:0/1359640665 >> 172.21.15.53:0/1359640665 conn(0x7f64f80cb0c0 msgr2=0x7f64f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 -- 172.21.15.53:0/1359640665 shutdown_connections 2023-10-21T16:15:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.388+0000 7f64fd040640 1 -- 172.21.15.53:0/1359640665 wait complete. 2023-10-21T16:15:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.389+0000 7f64fd040640 1 Processor -- start 2023-10-21T16:15:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.390+0000 7f64fd040640 1 -- start start 2023-10-21T16:15:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.390+0000 7f64fd040640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64f80adc70 0x7f64f80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fd040640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 0x7f64f815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fd040640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fd040640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f814b610 con 0x7f64f815ab30 2023-10-21T16:15:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fd040640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80c7d00 con 0x7f64f80b07d0 2023-10-21T16:15:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fd040640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80c70f0 con 0x7f64f80adc70 2023-10-21T16:15:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56282/0 (socket says 172.21.15.53:56282) 2023-10-21T16:15:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 -- 172.21.15.53:0/4131597594 learned_addr learned my addr 172.21.15.53:0/4131597594 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64f7fff640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64f80adc70 0x7f64f80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64effff640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 0x7f64f815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 -- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64f80adc70 msgr2=0x7f64f80c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64f80adc70 0x7f64f80c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 -- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 msgr2=0x7f64f815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 0x7f64f815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64fc83f640 1 -- 172.21.15.53:0/4131597594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e8009400 con 0x7f64f815ab30 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.391+0000 7f64effff640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 0x7f64f815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64fc83f640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f64f0007ef0 tx=0x7f64f000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f0003b90 con 0x7f64f815ab30 2023-10-21T16:15:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f000f040 con 0x7f64f815ab30 2023-10-21T16:15:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f000de80 con 0x7f64f815ab30 2023-10-21T16:15:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f80c73d0 con 0x7f64f815ab30 2023-10-21T16:15:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.392+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f80c7990 con 0x7f64f815ab30 2023-10-21T16:15:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.393+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f80c7e90 con 0x7f64f815ab30 2023-10-21T16:15:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.394+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64f0004030 con 0x7f64f815ab30 2023-10-21T16:15:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.395+0000 7f64f57fa640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 0x7f64d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.395+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f64f0089ce0 con 0x7f64f815ab30 2023-10-21T16:15:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.396+0000 7f64f7fff640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 0x7f64d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.397+0000 7f64f7fff640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 0x7f64d806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64e8036000 tx=0x7f64e803c040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.400+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64f0054a10 con 0x7f64f815ab30 2023-10-21T16:15:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.595+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64f804e160 con 0x7f64f815ab30 2023-10-21T16:15:15.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.598+0000 7f64f57fa640 1 -- 172.21.15.53:0/4131597594 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16845 (secure 0 0 0) 0x7f64f001a090 con 0x7f64f815ab30 2023-10-21T16:15:15.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:15.600 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":71,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:15:14.580426+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:15:15.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 msgr2=0x7f64d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 0x7f64d806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64e8036000 tx=0x7f64e803c040 comp rx=0 tx=0).stop 2023-10-21T16:15:15.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 msgr2=0x7f64f80af1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f64f0007ef0 tx=0x7f64f000d4a0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 shutdown_connections 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f64d806af90 0x7f64d806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64f815ab30 0x7f64f80af1d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64f80b07d0 0x7f64f815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 --2- 172.21.15.53:0/4131597594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64f80adc70 0x7f64f80c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.600+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 >> 172.21.15.53:0/4131597594 conn(0x7f64f80cb0c0 msgr2=0x7f64f80b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:15.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.601+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 shutdown_connections 2023-10-21T16:15:15.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.601+0000 7f64fd040640 1 -- 172.21.15.53:0/4131597594 wait complete. 2023-10-21T16:15:15.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:15:15.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.849+0000 7feb59547640 1 Processor -- start 2023-10-21T16:15:15.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.849+0000 7feb59547640 1 -- start start 2023-10-21T16:15:15.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb5415ab30 0x7feb5415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 0x7feb540b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 0x7feb540b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5415f7d0 con 0x7feb5415ab30 2023-10-21T16:15:15.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb5415f210 con 0x7feb5415af10 2023-10-21T16:15:15.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.850+0000 7feb59547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5415f4f0 con 0x7feb540adc70 2023-10-21T16:15:15.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb5415ab30 0x7feb5415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb5415ab30 0x7feb5415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56294/0 (socket says 172.21.15.53:56294) 2023-10-21T16:15:15.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 -- 172.21.15.53:0/1804712331 learned_addr learned my addr 172.21.15.53:0/1804712331 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:15.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb58d46640 1 --2- 172.21.15.53:0/1804712331 >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 0x7feb540b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb537fe640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 0x7feb540b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 -- 172.21.15.53:0/1804712331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 msgr2=0x7feb540b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:15.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 0x7feb540b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 -- 172.21.15.53:0/1804712331 >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 msgr2=0x7feb540b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 --2- 172.21.15.53:0/1804712331 >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 0x7feb540b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb53fff640 1 -- 172.21.15.53:0/1804712331 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5406c550 con 0x7feb5415ab30 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb58d46640 1 --2- 172.21.15.53:0/1804712331 >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 0x7feb540b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.851+0000 7feb537fe640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 0x7feb540b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.852+0000 7feb53fff640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5415ab30 0x7feb5415e8a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feb540cb000 tx=0x7feb4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918d157d44c9cf9f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.853+0000 7feb52ffd640 1 -- 172.21.15.53:0/1804712331 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44031870 con 0x7feb5415ab30 2023-10-21T16:15:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.853+0000 7feb52ffd640 1 -- 172.21.15.53:0/1804712331 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb440319d0 con 0x7feb5415ab30 2023-10-21T16:15:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.853+0000 7feb52ffd640 1 -- 172.21.15.53:0/1804712331 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44031d60 con 0x7feb5415ab30 2023-10-21T16:15:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.853+0000 7feb59547640 1 -- 172.21.15.53:0/1804712331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5415ab30 msgr2=0x7feb5415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.853+0000 7feb59547640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5415ab30 0x7feb5415e8a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feb540cb000 tx=0x7feb4402b910 comp rx=0 tx=0).stop 2023-10-21T16:15:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 -- 172.21.15.53:0/1804712331 shutdown_connections 2023-10-21T16:15:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb540adc70 0x7feb540b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 --2- 172.21.15.53:0/1804712331 >> v2:172.21.15.124:3300/0 conn(0x7feb5415af10 0x7feb540b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 --2- 172.21.15.53:0/1804712331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5415ab30 0x7feb5415e8a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 -- 172.21.15.53:0/1804712331 >> 172.21.15.53:0/1804712331 conn(0x7feb540cb0c0 msgr2=0x7feb540cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 -- 172.21.15.53:0/1804712331 shutdown_connections 2023-10-21T16:15:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.854+0000 7feb59547640 1 -- 172.21.15.53:0/1804712331 wait complete. 2023-10-21T16:15:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.856+0000 7feb59547640 1 Processor -- start 2023-10-21T16:15:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.856+0000 7feb59547640 1 -- start start 2023-10-21T16:15:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb5415ab30 0x7feb540b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 0x7feb5415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5415b3a0 con 0x7feb540adc70 2023-10-21T16:15:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5415c170 con 0x7feb5415ab30 2023-10-21T16:15:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb59547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5414be30 con 0x7feb5415af10 2023-10-21T16:15:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56304/0 (socket says 172.21.15.53:56304) 2023-10-21T16:15:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb53fff640 1 -- 172.21.15.53:0/3867559037 learned_addr learned my addr 172.21.15.53:0/3867559037 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.857+0000 7feb537fe640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb5415ab30 0x7feb540b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb58d46640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 0x7feb5415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 -- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 msgr2=0x7feb5415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 0x7feb5415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 -- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb5415ab30 msgr2=0x7feb540b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb5415ab30 0x7feb540b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 -- 172.21.15.53:0/3867559037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440094d0 con 0x7feb540adc70 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb58d46640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 0x7feb5415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb53fff640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7feb4402b950 tx=0x7feb440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44031860 con 0x7feb540adc70 2023-10-21T16:15:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb440443f0 con 0x7feb540adc70 2023-10-21T16:15:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb440447a0 con 0x7feb540adc70 2023-10-21T16:15:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.858+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5415b5f0 con 0x7feb540adc70 2023-10-21T16:15:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.859+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5415bbb0 con 0x7feb540adc70 2023-10-21T16:15:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.860+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5404e160 con 0x7feb540adc70 2023-10-21T16:15:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.861+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb4404d450 con 0x7feb540adc70 2023-10-21T16:15:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.861+0000 7feb50ff9640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 0x7feb4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.862+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb4402d040 con 0x7feb540adc70 2023-10-21T16:15:15.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.862+0000 7feb537fe640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 0x7feb4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:15.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.863+0000 7feb537fe640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 0x7feb4006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb540c8090 tx=0x7feb48008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:15.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:15.866+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb44076f00 con 0x7feb540adc70 2023-10-21T16:15:16.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.060+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb540ab8d0 con 0x7feb540adc70 2023-10-21T16:15:16.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.063+0000 7feb50ff9640 1 -- 172.21.15.53:0/3867559037 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16845 (secure 0 0 0) 0x7feb44035090 con 0x7feb540adc70 2023-10-21T16:15:16.065 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:16.065 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":71,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:15:14.580426+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:15:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 msgr2=0x7feb4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 0x7feb4006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb540c8090 tx=0x7feb48008040 comp rx=0 tx=0).stop 2023-10-21T16:15:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 msgr2=0x7feb5414b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7feb4402b950 tx=0x7feb440046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 shutdown_connections 2023-10-21T16:15:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7feb4006af90 0x7feb4006d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb5415af10 0x7feb5415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb5415ab30 0x7feb540b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 --2- 172.21.15.53:0/3867559037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb540adc70 0x7feb5414b8f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 >> 172.21.15.53:0/3867559037 conn(0x7feb540cb0c0 msgr2=0x7feb540b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 shutdown_connections 2023-10-21T16:15:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.066+0000 7feb59547640 1 -- 172.21.15.53:0/3867559037 wait complete. 2023-10-21T16:15:16.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:15:16.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.297+0000 7fc1f6a90640 1 Processor -- start 2023-10-21T16:15:16.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.298+0000 7fc1f6a90640 1 -- start start 2023-10-21T16:15:16.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.299+0000 7fc1f6a90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f006dfc0 0x7fc1f0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.299+0000 7fc1f6a90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc1f0071070 0x7fc1f00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.299+0000 7fc1f6a90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00b1cc0 con 0x7fc1f00afbe0 2023-10-21T16:15:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.299+0000 7fc1f6a90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00b2620 con 0x7fc1f0071070 2023-10-21T16:15:16.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.299+0000 7fc1f6a90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b2060 con 0x7fc1f006dfc0 2023-10-21T16:15:16.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc1f0071070 0x7fc1f00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc1f0071070 0x7fc1f00cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56314/0 (socket says 172.21.15.53:56314) 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 -- 172.21.15.53:0/3545675233 learned_addr learned my addr 172.21.15.53:0/3545675233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1effff640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f006dfc0 0x7fc1f0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 -- 172.21.15.53:0/3545675233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f006dfc0 msgr2=0x7fc1f0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f006dfc0 0x7fc1f0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 -- 172.21.15.53:0/3545675233 >> v1:172.21.15.53:6789/0 conn(0x7fc1f00afbe0 legacy=0x7fc1f00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:16.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.300+0000 7fc1f502d640 1 -- 172.21.15.53:0/3545675233 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00b2340 con 0x7fc1f0071070 2023-10-21T16:15:16.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.301+0000 7fc1f502d640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f0071070 0x7fc1f00cb130 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009e50 tx=0x7fc1e802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485bf1f27764586b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1ef7fe640 1 -- 172.21.15.53:0/3545675233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e802caf0 con 0x7fc1f0071070 2023-10-21T16:15:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1ef7fe640 1 -- 172.21.15.53:0/3545675233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e802cc50 con 0x7fc1f0071070 2023-10-21T16:15:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1ef7fe640 1 -- 172.21.15.53:0/3545675233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80343f0 con 0x7fc1f0071070 2023-10-21T16:15:16.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 -- 172.21.15.53:0/3545675233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f0071070 msgr2=0x7fc1f00cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f0071070 0x7fc1f00cb130 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009e50 tx=0x7fc1e802bf50 comp rx=0 tx=0).stop 2023-10-21T16:15:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 -- 172.21.15.53:0/3545675233 shutdown_connections 2023-10-21T16:15:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f0071070 0x7fc1f00cb130 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/3545675233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f006dfc0 0x7fc1f0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.302+0000 7fc1f6a90640 1 -- 172.21.15.53:0/3545675233 >> 172.21.15.53:0/3545675233 conn(0x7fc1f006a5d0 msgr2=0x7fc1f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:16.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.303+0000 7fc1f6a90640 1 -- 172.21.15.53:0/3545675233 shutdown_connections 2023-10-21T16:15:16.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.303+0000 7fc1f6a90640 1 -- 172.21.15.53:0/3545675233 wait complete. 2023-10-21T16:15:16.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.304+0000 7fc1f6a90640 1 Processor -- start 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.305+0000 7fc1f6a90640 1 -- start start 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.305+0000 7fc1f6a90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.305+0000 7fc1f6a90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1f0071070 0x7fc1f00c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f6a90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f00afbe0 0x7fc1f01580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f6a90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00cad30 con 0x7fc1f006dfc0 2023-10-21T16:15:16.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f6a90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00ca0b0 con 0x7fc1f0071070 2023-10-21T16:15:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f6a90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00ca390 con 0x7fc1f00afbe0 2023-10-21T16:15:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56328/0 (socket says 172.21.15.53:56328) 2023-10-21T16:15:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 -- 172.21.15.53:0/2831894993 learned_addr learned my addr 172.21.15.53:0/2831894993 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:16.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 -- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f00afbe0 msgr2=0x7fc1f01580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f00afbe0 0x7fc1f01580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 -- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1f0071070 msgr2=0x7fc1f00c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1f0071070 0x7fc1f00c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.306+0000 7fc1f482c640 1 -- 172.21.15.53:0/2831894993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8009b30 con 0x7fc1f006dfc0 2023-10-21T16:15:16.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1f482c640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c450 tx=0x7fc1e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:16.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40052d0 con 0x7fc1f006dfc0 2023-10-21T16:15:16.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4005430 con 0x7fc1f006dfc0 2023-10-21T16:15:16.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f00ca670 con 0x7fc1f006dfc0 2023-10-21T16:15:16.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4012870 con 0x7fc1f006dfc0 2023-10-21T16:15:16.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.307+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f01f60c0 con 0x7fc1f006dfc0 2023-10-21T16:15:16.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.308+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f004e160 con 0x7fc1f006dfc0 2023-10-21T16:15:16.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.309+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc1e40129d0 con 0x7fc1f006dfc0 2023-10-21T16:15:16.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.310+0000 7fc1ed7fa640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 0x7fc1d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:16.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.310+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc1e4089400 con 0x7fc1f006dfc0 2023-10-21T16:15:16.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.311+0000 7fc1effff640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 0x7fc1d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:16.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.312+0000 7fc1effff640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 0x7fc1d406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006fd0 tx=0x7fc1dc008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:16.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.315+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1e4054220 con 0x7fc1f006dfc0 2023-10-21T16:15:16.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:16.551+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc1f00b0ef0 con 0x7fc1f006dfc0 2023-10-21T16:15:17.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.231+0000 7fc1ed7fa640 1 -- 172.21.15.53:0/2831894993 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1e4057840 con 0x7fc1f006dfc0 2023-10-21T16:15:17.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 msgr2=0x7fc1d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 0x7fc1d406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006fd0 tx=0x7fc1dc008040 comp rx=0 tx=0).stop 2023-10-21T16:15:17.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 msgr2=0x7fc1f01f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c450 tx=0x7fc1e400c910 comp rx=0 tx=0).stop 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 shutdown_connections 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc1d406af90 0x7fc1d406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1f00afbe0 0x7fc1f01580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1f0071070 0x7fc1f00c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 --2- 172.21.15.53:0/2831894993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1f006dfc0 0x7fc1f01f2060 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.235+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 >> 172.21.15.53:0/2831894993 conn(0x7fc1f006a5d0 msgr2=0x7fc1f0073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:17.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.236+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 shutdown_connections 2023-10-21T16:15:17.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.236+0000 7fc1f6a90640 1 -- 172.21.15.53:0/2831894993 wait complete. 2023-10-21T16:15:17.258 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:15:17.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:15:17.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.482+0000 7ff19aa7e640 1 Processor -- start 2023-10-21T16:15:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.482+0000 7ff19aa7e640 1 -- start start 2023-10-21T16:15:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.483+0000 7ff19aa7e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff19415a820 0x7ff1940aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.483+0000 7ff19aa7e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1940ca680 0x7ff1940b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.483+0000 7ff19aa7e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940cab60 con 0x7ff19415ac00 2023-10-21T16:15:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff19aa7e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940cb960 con 0x7ff19415a820 2023-10-21T16:15:17.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff19aa7e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940cbdf0 con 0x7ff1940ca680 2023-10-21T16:15:17.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff199a7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff19415a820 0x7ff1940aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:17.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff19a27d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff19415ac00 0x7ff1940c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51724/0 (socket says 172.21.15.53:51724) 2023-10-21T16:15:17.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff19a27d640 1 -- 172.21.15.53:0/4253858511 learned_addr learned my addr 172.21.15.53:0/4253858511 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:17.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff19927b640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1940ca680 0x7ff1940b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:17.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff199a7c640 1 -- 172.21.15.53:0/4253858511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1940ca680 msgr2=0x7ff1940b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:17.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff199a7c640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1940ca680 0x7ff1940b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff199a7c640 1 -- 172.21.15.53:0/4253858511 >> v1:172.21.15.53:6789/0 conn(0x7ff19415ac00 legacy=0x7ff1940c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff199a7c640 1 -- 172.21.15.53:0/4253858511 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940cb030 con 0x7ff19415a820 2023-10-21T16:15:17.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.484+0000 7ff198a7a640 1 -- 172.21.15.53:0/4253858511 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756383318 0 0) 0x7ff1940cab60 con 0x7ff19415ac00 2023-10-21T16:15:17.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.486+0000 7ff199a7c640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff19415a820 0x7ff1940aff70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff1900097a0 tx=0x7ff19002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f8876ebf25fbb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:17.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.486+0000 7ff198a7a640 1 -- 172.21.15.53:0/4253858511 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190031780 con 0x7ff19415a820 2023-10-21T16:15:17.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.486+0000 7ff198a7a640 1 -- 172.21.15.53:0/4253858511 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1900318e0 con 0x7ff19415a820 2023-10-21T16:15:17.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.486+0000 7ff198a7a640 1 -- 172.21.15.53:0/4253858511 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190031c70 con 0x7ff19415a820 2023-10-21T16:15:17.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.486+0000 7ff19aa7e640 1 -- 172.21.15.53:0/4253858511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff19415a820 msgr2=0x7ff1940aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff19415a820 0x7ff1940aff70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff1900097a0 tx=0x7ff19002b820 comp rx=0 tx=0).stop 2023-10-21T16:15:17.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 -- 172.21.15.53:0/4253858511 shutdown_connections 2023-10-21T16:15:17.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1940ca680 0x7ff1940b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/4253858511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff19415a820 0x7ff1940aff70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 -- 172.21.15.53:0/4253858511 >> 172.21.15.53:0/4253858511 conn(0x7ff19406c050 msgr2=0x7ff19406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:17.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 -- 172.21.15.53:0/4253858511 shutdown_connections 2023-10-21T16:15:17.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.487+0000 7ff19aa7e640 1 -- 172.21.15.53:0/4253858511 wait complete. 2023-10-21T16:15:17.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.489+0000 7ff19aa7e640 1 Processor -- start 2023-10-21T16:15:17.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.489+0000 7ff19aa7e640 1 -- start start 2023-10-21T16:15:17.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff19415a820 0x7ff1941f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 0x7ff1940c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940aefd0 con 0x7ff1940ca680 2023-10-21T16:15:17.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940b07c0 con 0x7ff19415ac00 2023-10-21T16:15:17.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19aa7e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940afdf0 con 0x7ff19415a820 2023-10-21T16:15:17.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:17.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56352/0 (socket says 172.21.15.53:56352) 2023-10-21T16:15:17.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 -- 172.21.15.53:0/1093978275 learned_addr learned my addr 172.21.15.53:0/1093978275 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:17.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 -- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff19415a820 msgr2=0x7ff1941f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff19415a820 0x7ff1941f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff199a7c640 1 -- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 msgr2=0x7ff1940c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:17.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.490+0000 7ff19a27d640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 0x7ff1940c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:17.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff199a7c640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 0x7ff1940c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:17.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff199a7c640 1 -- 172.21.15.53:0/1093978275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009400 con 0x7ff1940ca680 2023-10-21T16:15:17.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff19a27d640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 0x7ff1940c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:17.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff199a7c640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:17.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190031f10 con 0x7ff1940ca680 2023-10-21T16:15:17.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190004250 con 0x7ff1940ca680 2023-10-21T16:15:17.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.491+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1940af1e0 con 0x7ff1940ca680 2023-10-21T16:15:17.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.492+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190032620 con 0x7ff1940ca680 2023-10-21T16:15:17.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.492+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1940af7a0 con 0x7ff1940ca680 2023-10-21T16:15:17.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.493+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff194074150 con 0x7ff1940ca680 2023-10-21T16:15:17.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.494+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff190032780 con 0x7ff1940ca680 2023-10-21T16:15:17.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.495+0000 7ff18a7fc640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 0x7ff17806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:17.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.495+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff1900abdb0 con 0x7ff1940ca680 2023-10-21T16:15:17.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.495+0000 7ff19927b640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 0x7ff17806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:17.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.497+0000 7ff19927b640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 0x7ff17806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff184006fd0 tx=0x7ff184008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:17.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.499+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1900b0070 con 0x7ff1940ca680 2023-10-21T16:15:17.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:17.731+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff1940b1310 con 0x7ff1940ca680 2023-10-21T16:15:18.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.234+0000 7ff18a7fc640 1 -- 172.21.15.53:0/1093978275 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff1900769b0 con 0x7ff1940ca680 2023-10-21T16:15:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 msgr2=0x7ff17806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 0x7ff17806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff184006fd0 tx=0x7ff184008040 comp rx=0 tx=0).stop 2023-10-21T16:15:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 msgr2=0x7ff1940b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff1900046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 shutdown_connections 2023-10-21T16:15:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7ff17806af90 0x7ff17806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff19415ac00 0x7ff1940c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff19415a820 0x7ff1941f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 --2- 172.21.15.53:0/1093978275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1940ca680 0x7ff1940b0280 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 >> 172.21.15.53:0/1093978275 conn(0x7ff19406c050 msgr2=0x7ff1940c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:18.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.238+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 shutdown_connections 2023-10-21T16:15:18.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.239+0000 7ff19aa7e640 1 -- 172.21.15.53:0/1093978275 wait complete. 2023-10-21T16:15:18.260 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:15:18.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:15:18.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.487+0000 7f5fc895d640 1 Processor -- start 2023-10-21T16:15:18.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.487+0000 7f5fc895d640 1 -- start start 2023-10-21T16:15:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.488+0000 7f5fc895d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fc415a9a0 0x7f5fc40cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.488+0000 7f5fc895d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.488+0000 7f5fc895d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc415b070 con 0x7f5fc40b54b0 2023-10-21T16:15:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.488+0000 7f5fc895d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cc590 con 0x7f5fc415a9a0 2023-10-21T16:15:18.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.488+0000 7f5fc895d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40cc870 con 0x7f5fc40b5050 2023-10-21T16:15:18.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.489+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.489+0000 7f5fc2d9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fc415a9a0 0x7f5fc40cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.489+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:57528/0 (socket says 172.21.15.53:57528) 2023-10-21T16:15:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.489+0000 7f5fc259c640 1 -- 172.21.15.53:0/908436375 learned_addr learned my addr 172.21.15.53:0/908436375 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:18.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.489+0000 7f5fc359e640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5fc40b54b0 0x7f5fc4069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:41906/0 (socket says 172.21.15.53:41906) 2023-10-21T16:15:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.490+0000 7f5fc259c640 1 -- 172.21.15.53:0/908436375 >> v2:172.21.15.124:3300/0 conn(0x7f5fc415a9a0 msgr2=0x7f5fc40cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.490+0000 7f5fc259c640 1 --2- 172.21.15.53:0/908436375 >> v2:172.21.15.124:3300/0 conn(0x7f5fc415a9a0 0x7f5fc40cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.490+0000 7f5fc259c640 1 -- 172.21.15.53:0/908436375 >> v1:172.21.15.124:6789/0 conn(0x7f5fc40b54b0 legacy=0x7f5fc4069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.490+0000 7f5fc1d9b640 1 -- 172.21.15.53:0/908436375 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18215156 0 0) 0x7f5fc415b070 con 0x7f5fc40b54b0 2023-10-21T16:15:18.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.490+0000 7f5fc259c640 1 -- 172.21.15.53:0/908436375 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40ccb50 con 0x7f5fc40b5050 2023-10-21T16:15:18.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.491+0000 7f5fc259c640 1 --2- 172.21.15.53:0/908436375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=345ced2c303b0d60 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.492+0000 7f5fc1d9b640 1 -- 172.21.15.53:0/908436375 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb802cb00 con 0x7f5fc40b5050 2023-10-21T16:15:18.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.492+0000 7f5fc1d9b640 1 -- 172.21.15.53:0/908436375 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb802cc60 con 0x7f5fc40b5050 2023-10-21T16:15:18.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.492+0000 7f5fc1d9b640 1 -- 172.21.15.53:0/908436375 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80343f0 con 0x7f5fc40b5050 2023-10-21T16:15:18.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.492+0000 7f5fc895d640 1 -- 172.21.15.53:0/908436375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 msgr2=0x7f5fc40cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.492+0000 7f5fc895d640 1 --2- 172.21.15.53:0/908436375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb802bf60 comp rx=0 tx=0).stop 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 -- 172.21.15.53:0/908436375 shutdown_connections 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 --2- 172.21.15.53:0/908436375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc40cf5f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 --2- 172.21.15.53:0/908436375 >> v2:172.21.15.124:3300/0 conn(0x7f5fc415a9a0 0x7f5fc40cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 -- 172.21.15.53:0/908436375 >> 172.21.15.53:0/908436375 conn(0x7f5fc406ae60 msgr2=0x7f5fc406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 -- 172.21.15.53:0/908436375 shutdown_connections 2023-10-21T16:15:18.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.493+0000 7f5fc895d640 1 -- 172.21.15.53:0/908436375 wait complete. 2023-10-21T16:15:18.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.495+0000 7f5fc895d640 1 Processor -- start 2023-10-21T16:15:18.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.495+0000 7f5fc895d640 1 -- start start 2023-10-21T16:15:18.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc41f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fc415a9a0 0x7f5fc40ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc41f2b30 con 0x7f5fc40b54b0 2023-10-21T16:15:18.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40c7810 con 0x7f5fc415a9a0 2023-10-21T16:15:18.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc895d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40c7af0 con 0x7f5fc40b5050 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56368/0 (socket says 172.21.15.53:56368) 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc259c640 1 -- 172.21.15.53:0/150233043 learned_addr learned my addr 172.21.15.53:0/150233043 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.496+0000 7f5fc2d9d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc41f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc359e640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fc415a9a0 0x7f5fc40ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 -- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 msgr2=0x7f5fc41f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc41f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 -- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fc415a9a0 msgr2=0x7f5fc40ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fc415a9a0 0x7f5fc40ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 -- 172.21.15.53:0/150233043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8009b30 con 0x7f5fc40b54b0 2023-10-21T16:15:18.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fc259c640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.497+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb803e450 con 0x7f5fc40b54b0 2023-10-21T16:15:18.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.498+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb803e5f0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.498+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb803e9a0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.498+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc40c7dd0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.498+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc40c8390 con 0x7f5fc40b54b0 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.499+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc404e160 con 0x7f5fc40b54b0 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.500+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5fb803eb00 con 0x7f5fc40b54b0 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.501+0000 7f5fab7fe640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 0x7f5fb006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.501+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5fb80ac5f0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.501+0000 7f5fc2d9d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 0x7f5fb006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.502+0000 7f5fc2d9d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 0x7f5fb006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009a30 tx=0x7f5fb4008040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.505+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fb80774b0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.739+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fc40c9a00 con 0x7f5fc40b54b0 2023-10-21T16:15:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.740+0000 7f5fab7fe640 1 -- 172.21.15.53:0/150233043 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f5fb803edb0 con 0x7f5fc40b54b0 2023-10-21T16:15:18.742 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:18.742 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:15:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:15:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 msgr2=0x7f5fb006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 0x7f5fb006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009a30 tx=0x7f5fb4008040 comp rx=0 tx=0).stop 2023-10-21T16:15:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 msgr2=0x7f5fc4151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009e50 tx=0x7f5fb8004a40 comp rx=0 tx=0).stop 2023-10-21T16:15:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 shutdown_connections 2023-10-21T16:15:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f5fb006af90 0x7f5fb006d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fc415a9a0 0x7f5fc40ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc40b54b0 0x7f5fc4151b20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 --2- 172.21.15.53:0/150233043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fc40b5050 0x7f5fc41f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 >> 172.21.15.53:0/150233043 conn(0x7f5fc406ae60 msgr2=0x7f5fc40ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:18.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 shutdown_connections 2023-10-21T16:15:18.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.743+0000 7f5fc895d640 1 -- 172.21.15.53:0/150233043 wait complete. 2023-10-21T16:15:18.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:15:18.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.980+0000 7fd6ce85d640 1 Processor -- start 2023-10-21T16:15:18.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.980+0000 7fd6ce85d640 1 -- start start 2023-10-21T16:15:18.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6ce85d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6c8165cc0 0x7fd6c81680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6ce85d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80c48f0 con 0x7fd6c815cd90 2023-10-21T16:15:18.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6ce85d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80710f0 con 0x7fd6c8160940 2023-10-21T16:15:18.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6ce85d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c80c4d80 con 0x7fd6c8165cc0 2023-10-21T16:15:18.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6c7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd6c815cd90 0x7fd6c80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51726/0 (socket says 172.21.15.53:51726) 2023-10-21T16:15:18.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.982+0000 7fd6c7fff640 1 -- 172.21.15.53:0/2182247652 learned_addr learned my addr 172.21.15.53:0/2182247652 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:18.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6ccdfa640 1 --2- 172.21.15.53:0/2182247652 >> v2:172.21.15.53:3300/0 conn(0x7fd6c8165cc0 0x7fd6c81680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6c77fe640 1 -- 172.21.15.53:0/2182247652 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028416491 0 0) 0x7fd6c80c48f0 con 0x7fd6c815cd90 2023-10-21T16:15:18.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6c77fe640 1 -- 172.21.15.53:0/2182247652 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003520 con 0x7fd6c815cd90 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6ccdfa640 1 -- 172.21.15.53:0/2182247652 >> v1:172.21.15.124:6789/0 conn(0x7fd6c8160940 legacy=0x7fd6c80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6ccdfa640 1 -- 172.21.15.53:0/2182247652 >> v1:172.21.15.53:6789/0 conn(0x7fd6c815cd90 legacy=0x7fd6c80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.983+0000 7fd6ccdfa640 1 -- 172.21.15.53:0/2182247652 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80c3ce0 con 0x7fd6c8165cc0 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.984+0000 7fd6ccdfa640 1 --2- 172.21.15.53:0/2182247652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8165cc0 0x7fd6c81680b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097d0 tx=0x7fd6c002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a763ff490e8cdefe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.984+0000 7fd6c77fe640 1 -- 172.21.15.53:0/2182247652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0030800 con 0x7fd6c8165cc0 2023-10-21T16:15:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6c77fe640 1 -- 172.21.15.53:0/2182247652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0030960 con 0x7fd6c8165cc0 2023-10-21T16:15:18.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6c77fe640 1 -- 172.21.15.53:0/2182247652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0030cf0 con 0x7fd6c8165cc0 2023-10-21T16:15:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 -- 172.21.15.53:0/2182247652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8165cc0 msgr2=0x7fd6c81680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/2182247652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8165cc0 0x7fd6c81680b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097d0 tx=0x7fd6c002b940 comp rx=0 tx=0).stop 2023-10-21T16:15:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 -- 172.21.15.53:0/2182247652 shutdown_connections 2023-10-21T16:15:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/2182247652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8165cc0 0x7fd6c81680b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 -- 172.21.15.53:0/2182247652 >> 172.21.15.53:0/2182247652 conn(0x7fd6c806a380 msgr2=0x7fd6c80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:18.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 -- 172.21.15.53:0/2182247652 shutdown_connections 2023-10-21T16:15:18.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.985+0000 7fd6ce85d640 1 -- 172.21.15.53:0/2182247652 wait complete. 2023-10-21T16:15:18.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.987+0000 7fd6ce85d640 1 Processor -- start 2023-10-21T16:15:18.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.987+0000 7fd6ce85d640 1 -- start start 2023-10-21T16:15:18.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6c815cd90 0x7fd6c80abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6c8165cc0 0x7fd6c80cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80ad340 con 0x7fd6c8160940 2023-10-21T16:15:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80ac730 con 0x7fd6c8165cc0 2023-10-21T16:15:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.988+0000 7fd6ce85d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80aca10 con 0x7fd6c815cd90 2023-10-21T16:15:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56388/0 (socket says 172.21.15.53:56388) 2023-10-21T16:15:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 -- 172.21.15.53:0/1476618578 learned_addr learned my addr 172.21.15.53:0/1476618578 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6c7fff640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6c815cd90 0x7fd6c80abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6ccdfa640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6c8165cc0 0x7fd6c80cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 -- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6c815cd90 msgr2=0x7fd6c80abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6c815cd90 0x7fd6c80abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 -- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6c8165cc0 msgr2=0x7fd6c80cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6c8165cc0 0x7fd6c80cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 -- 172.21.15.53:0/1476618578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0009400 con 0x7fd6c8160940 2023-10-21T16:15:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.989+0000 7fd6bf7fe640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c450 tx=0x7fd6b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.990+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00052d0 con 0x7fd6c8160940 2023-10-21T16:15:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.990+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0005430 con 0x7fd6c8160940 2023-10-21T16:15:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.990+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0012870 con 0x7fd6c8160940 2023-10-21T16:15:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.990+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80accf0 con 0x7fd6c8160940 2023-10-21T16:15:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.990+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80c8430 con 0x7fd6c8160940 2023-10-21T16:15:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.991+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8067490 con 0x7fd6c8160940 2023-10-21T16:15:18.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.992+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd6b00129d0 con 0x7fd6c8160940 2023-10-21T16:15:18.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.993+0000 7fd6c57fa640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 0x7fd6a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:18.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.993+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd6b00894a0 con 0x7fd6c8160940 2023-10-21T16:15:18.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.994+0000 7fd6c7fff640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 0x7fd6a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:18.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.995+0000 7fd6c7fff640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 0x7fd6a806d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001560 tx=0x7fd6b8009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:18.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:18.998+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6b0054230 con 0x7fd6c8160940 2023-10-21T16:15:19.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.192+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6c80b0ef0 con 0x7fd6c8160940 2023-10-21T16:15:19.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.194+0000 7fd6c57fa640 1 -- 172.21.15.53:0/1476618578 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16832 (secure 0 0 0) 0x7fd6b0057850 con 0x7fd6c8160940 2023-10-21T16:15:19.196 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:19.197 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":72,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:15:16.126132+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:15:19.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 msgr2=0x7fd6a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 0x7fd6a806d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8001560 tx=0x7fd6b8009040 comp rx=0 tx=0).stop 2023-10-21T16:15:19.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 msgr2=0x7fd6c80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c450 tx=0x7fd6b000c910 comp rx=0 tx=0).stop 2023-10-21T16:15:19.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 shutdown_connections 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fd6a806afc0 0x7fd6a806d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd6c8165cc0 0x7fd6c80cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6c8160940 0x7fd6c80caaf0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 --2- 172.21.15.53:0/1476618578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6c815cd90 0x7fd6c80abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 >> 172.21.15.53:0/1476618578 conn(0x7fd6c806a380 msgr2=0x7fd6c8160e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:19.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 shutdown_connections 2023-10-21T16:15:19.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.197+0000 7fd6ce85d640 1 -- 172.21.15.53:0/1476618578 wait complete. 2023-10-21T16:15:19.216 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:15:19.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:15:19.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.430+0000 7fa5f1a51640 1 Processor -- start 2023-10-21T16:15:19.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.431+0000 7fa5f1a51640 1 -- start start 2023-10-21T16:15:19.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.432+0000 7fa5f1a51640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5ec15cd90 0x7fa5ec0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.432+0000 7fa5f1a51640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0c48f0 con 0x7fa5ec160940 2023-10-21T16:15:19.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.432+0000 7fa5f1a51640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec0c3ce0 con 0x7fa5ec06c890 2023-10-21T16:15:19.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.432+0000 7fa5f1a51640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0c4d80 con 0x7fa5ec15cd90 2023-10-21T16:15:19.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5ea7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa5ec160940 0x7fa5ec070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51728/0 (socket says 172.21.15.53:51728) 2023-10-21T16:15:19.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5ea7fc640 1 -- 172.21.15.53:0/1189942694 learned_addr learned my addr 172.21.15.53:0/1189942694 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:19.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5eaffd640 1 --2- 172.21.15.53:0/1189942694 >> v2:172.21.15.53:3300/0 conn(0x7fa5ec15cd90 0x7fa5ec0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5e9ffb640 1 -- 172.21.15.53:0/1189942694 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816637853 0 0) 0x7fa5ec0c48f0 con 0x7fa5ec160940 2023-10-21T16:15:19.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5e9ffb640 1 -- 172.21.15.53:0/1189942694 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d8003520 con 0x7fa5ec160940 2023-10-21T16:15:19.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5eaffd640 1 -- 172.21.15.53:0/1189942694 >> v1:172.21.15.124:6789/0 conn(0x7fa5ec06c890 legacy=0x7fa5ec06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5eaffd640 1 -- 172.21.15.53:0/1189942694 >> v1:172.21.15.53:6789/0 conn(0x7fa5ec160940 legacy=0x7fa5ec070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.433+0000 7fa5eaffd640 1 -- 172.21.15.53:0/1189942694 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec0c3fc0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5eaffd640 1 --2- 172.21.15.53:0/1189942694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec0702d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009d90 tx=0x7fa5dc02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ca987aa0986742 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5e9ffb640 1 -- 172.21.15.53:0/1189942694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc02ccf0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5e9ffb640 1 -- 172.21.15.53:0/1189942694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc02ce50 con 0x7fa5ec15cd90 2023-10-21T16:15:19.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5e9ffb640 1 -- 172.21.15.53:0/1189942694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc034660 con 0x7fa5ec15cd90 2023-10-21T16:15:19.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5f1a51640 1 -- 172.21.15.53:0/1189942694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 msgr2=0x7fa5ec0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.435+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/1189942694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec0702d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009d90 tx=0x7fa5dc02bf40 comp rx=0 tx=0).stop 2023-10-21T16:15:19.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.436+0000 7fa5f1a51640 1 -- 172.21.15.53:0/1189942694 shutdown_connections 2023-10-21T16:15:19.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.436+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/1189942694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec0702d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.436+0000 7fa5f1a51640 1 -- 172.21.15.53:0/1189942694 >> 172.21.15.53:0/1189942694 conn(0x7fa5ec06a380 msgr2=0x7fa5ec0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:19.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.436+0000 7fa5f1a51640 1 -- 172.21.15.53:0/1189942694 shutdown_connections 2023-10-21T16:15:19.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.436+0000 7fa5f1a51640 1 -- 172.21.15.53:0/1189942694 wait complete. 2023-10-21T16:15:19.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.437+0000 7fa5f1a51640 1 Processor -- start 2023-10-21T16:15:19.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.438+0000 7fa5f1a51640 1 -- start start 2023-10-21T16:15:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.438+0000 7fa5f1a51640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5ec06c890 0x7fa5ec15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5f1a51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5f1a51640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5ec160940 0x7fa5ec1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5f1a51640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec15d690 con 0x7fa5ec15cd90 2023-10-21T16:15:19.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5f1a51640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec15d380 con 0x7fa5ec06c890 2023-10-21T16:15:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5f1a51640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec1f2dd0 con 0x7fa5ec160940 2023-10-21T16:15:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56402/0 (socket says 172.21.15.53:56402) 2023-10-21T16:15:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 -- 172.21.15.53:0/2967472984 learned_addr learned my addr 172.21.15.53:0/2967472984 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5eaffd640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5ec06c890 0x7fa5ec15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 -- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5ec160940 msgr2=0x7fa5ec1f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5ec160940 0x7fa5ec1f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 -- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5ec06c890 msgr2=0x7fa5ec15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5ec06c890 0x7fa5ec15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.439+0000 7fa5ea7fc640 1 -- 172.21.15.53:0/2967472984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009a70 con 0x7fa5ec15cd90 2023-10-21T16:15:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5ea7fc640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c410 tx=0x7fa5d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4005ce0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4005e40 con 0x7fa5ec15cd90 2023-10-21T16:15:19.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40072f0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec1f3030 con 0x7fa5ec15cd90 2023-10-21T16:15:19.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.440+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec1f35f0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.441+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec15d7d0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.442+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa5d4007510 con 0x7fa5ec15cd90 2023-10-21T16:15:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.443+0000 7fa5e3fff640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 0x7fa5cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.443+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa5d4088f20 con 0x7fa5ec15cd90 2023-10-21T16:15:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.444+0000 7fa5eaffd640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 0x7fa5cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.445+0000 7fa5eaffd640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 0x7fa5cc06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc036000 tx=0x7fa5dc030000 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.448+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5ec15d7d0 con 0x7fa5ec15cd90 2023-10-21T16:15:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.639+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 --> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5ec0c4580 con 0x7fa5cc06af90 2023-10-21T16:15:19.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.668+0000 7fa5e3fff640 1 -- 172.21.15.53:0/2967472984 <== mgr.34973 v2:172.21.15.124:6832/3288323666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286632 (secure 0 0 0) 0x7fa5dc044090 con 0x7fa5cc06af90 2023-10-21T16:15:19.673 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:19.675 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:15:19.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 msgr2=0x7fa5cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 0x7fa5cc06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc036000 tx=0x7fa5dc030000 comp rx=0 tx=0).stop 2023-10-21T16:15:19.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 msgr2=0x7fa5ec1f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c410 tx=0x7fa5d400c8d0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 shutdown_connections 2023-10-21T16:15:19.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa5cc06af90 0x7fa5cc06d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5ec160940 0x7fa5ec1f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ec15cd90 0x7fa5ec1f68b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 --2- 172.21.15.53:0/2967472984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5ec06c890 0x7fa5ec15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.674+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 >> 172.21.15.53:0/2967472984 conn(0x7fa5ec06a380 msgr2=0x7fa5ec0aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:19.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.675+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 shutdown_connections 2023-10-21T16:15:19.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.675+0000 7fa5f1a51640 1 -- 172.21.15.53:0/2967472984 wait complete. 2023-10-21T16:15:19.696 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-21T16:15:18.500791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318504,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754656216,"statfs":{"total":773094113280,"available":772767965184,"internally_reserved":0,"allocated":8167424,"data_stored":4639820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.408980"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346442+0000","last_change":"2023-10-21T16:15:15.158373+0000","last_active":"2023-10-21T16:15:16.346442+0000","last_peered":"2023-10-21T16:15:16.346442+0000","last_clean":"2023-10-21T16:15:16.346442+0000","last_became_active":"2023-10-21T16:15:15.158112+0000","last_became_peered":"2023-10-21T16:15:15.158112+0000","last_unstale":"2023-10-21T16:15:16.346442+0000","last_undegraded":"2023-10-21T16:15:16.346442+0000","last_fullsized":"2023-10-21T16:15:16.346442+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:06.393324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345267+0000","last_change":"2023-10-21T16:15:15.170422+0000","last_active":"2023-10-21T16:15:16.345267+0000","last_peered":"2023-10-21T16:15:16.345267+0000","last_clean":"2023-10-21T16:15:16.345267+0000","last_became_active":"2023-10-21T16:15:15.169565+0000","last_became_peered":"2023-10-21T16:15:15.169565+0000","last_unstale":"2023-10-21T16:15:16.345267+0000","last_undegraded":"2023-10-21T16:15:16.345267+0000","last_fullsized":"2023-10-21T16:15:16.345267+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:42.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344444+0000","last_change":"2023-10-21T16:15:15.181541+0000","last_active":"2023-10-21T16:15:16.344444+0000","last_peered":"2023-10-21T16:15:16.344444+0000","last_clean":"2023-10-21T16:15:16.344444+0000","last_became_active":"2023-10-21T16:15:15.169934+0000","last_became_peered":"2023-10-21T16:15:15.169934+0000","last_unstale":"2023-10-21T16:15:16.344444+0000","last_undegraded":"2023-10-21T16:15:16.344444+0000","last_fullsized":"2023-10-21T16:15:16.344444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:33:44.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344597+0000","last_change":"2023-10-21T16:15:15.181500+0000","last_active":"2023-10-21T16:15:16.344597+0000","last_peered":"2023-10-21T16:15:16.344597+0000","last_clean":"2023-10-21T16:15:16.344597+0000","last_became_active":"2023-10-21T16:15:15.181221+0000","last_became_peered":"2023-10-21T16:15:15.181221+0000","last_unstale":"2023-10-21T16:15:16.344597+0000","last_undegraded":"2023-10-21T16:15:16.344597+0000","last_fullsized":"2023-10-21T16:15:16.344597+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:16.588613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136130+0000","last_change":"2023-10-21T16:15:15.160958+0000","last_active":"2023-10-21T16:15:16.136130+0000","last_peered":"2023-10-21T16:15:16.136130+0000","last_clean":"2023-10-21T16:15:16.136130+0000","last_became_active":"2023-10-21T16:15:15.160577+0000","last_became_peered":"2023-10-21T16:15:15.160577+0000","last_unstale":"2023-10-21T16:15:16.136130+0000","last_undegraded":"2023-10-21T16:15:16.136130+0000","last_fullsized":"2023-10-21T16:15:16.136130+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:41:29.882157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345748+0000","last_change":"2023-10-21T16:15:15.170354+0000","last_active":"2023-10-21T16:15:16.345748+0000","last_peered":"2023-10-21T16:15:16.345748+0000","last_clean":"2023-10-21T16:15:16.345748+0000","last_became_active":"2023-10-21T16:15:15.165795+0000","last_became_peered":"2023-10-21T16:15:15.165795+0000","last_unstale":"2023-10-21T16:15:16.345748+0000","last_undegraded":"2023-10-21T16:15:16.345748+0000","last_fullsized":"2023-10-21T16:15:16.345748+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:06.124253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343036+0000","last_change":"2023-10-21T16:15:15.177108+0000","last_active":"2023-10-21T16:15:16.343036+0000","last_peered":"2023-10-21T16:15:16.343036+0000","last_clean":"2023-10-21T16:15:16.343036+0000","last_became_active":"2023-10-21T16:15:15.176806+0000","last_became_peered":"2023-10-21T16:15:15.176806+0000","last_unstale":"2023-10-21T16:15:16.343036+0000","last_undegraded":"2023-10-21T16:15:16.343036+0000","last_fullsized":"2023-10-21T16:15:16.343036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:12:12.503956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131460+0000","last_change":"2023-10-21T16:15:15.173603+0000","last_active":"2023-10-21T16:15:16.131460+0000","last_peered":"2023-10-21T16:15:16.131460+0000","last_clean":"2023-10-21T16:15:16.131460+0000","last_became_active":"2023-10-21T16:15:15.168881+0000","last_became_peered":"2023-10-21T16:15:15.168881+0000","last_unstale":"2023-10-21T16:15:16.131460+0000","last_undegraded":"2023-10-21T16:15:16.131460+0000","last_fullsized":"2023-10-21T16:15:16.131460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:55:54.669363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132370+0000","last_change":"2023-10-21T16:15:15.174759+0000","last_active":"2023-10-21T16:15:16.132370+0000","last_peered":"2023-10-21T16:15:16.132370+0000","last_clean":"2023-10-21T16:15:16.132370+0000","last_became_active":"2023-10-21T16:15:15.174483+0000","last_became_peered":"2023-10-21T16:15:15.174483+0000","last_unstale":"2023-10-21T16:15:16.132370+0000","last_undegraded":"2023-10-21T16:15:16.132370+0000","last_fullsized":"2023-10-21T16:15:16.132370+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:04.064641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346449+0000","last_change":"2023-10-21T16:15:15.151130+0000","last_active":"2023-10-21T16:15:16.346449+0000","last_peered":"2023-10-21T16:15:16.346449+0000","last_clean":"2023-10-21T16:15:16.346449+0000","last_became_active":"2023-10-21T16:15:15.149600+0000","last_became_peered":"2023-10-21T16:15:15.149600+0000","last_unstale":"2023-10-21T16:15:16.346449+0000","last_undegraded":"2023-10-21T16:15:16.346449+0000","last_fullsized":"2023-10-21T16:15:16.346449+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:51:30.423650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.133118+0000","last_change":"2023-10-21T16:15:15.160767+0000","last_active":"2023-10-21T16:15:16.133118+0000","last_peered":"2023-10-21T16:15:16.133118+0000","last_clean":"2023-10-21T16:15:16.133118+0000","last_became_active":"2023-10-21T16:15:15.160402+0000","last_became_peered":"2023-10-21T16:15:15.160402+0000","last_unstale":"2023-10-21T16:15:16.133118+0000","last_undegraded":"2023-10-21T16:15:16.133118+0000","last_fullsized":"2023-10-21T16:15:16.133118+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:42:17.688309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131814+0000","last_change":"2023-10-21T16:15:15.177615+0000","last_active":"2023-10-21T16:15:16.131814+0000","last_peered":"2023-10-21T16:15:16.131814+0000","last_clean":"2023-10-21T16:15:16.131814+0000","last_became_active":"2023-10-21T16:15:15.177192+0000","last_became_peered":"2023-10-21T16:15:15.177192+0000","last_unstale":"2023-10-21T16:15:16.131814+0000","last_undegraded":"2023-10-21T16:15:16.131814+0000","last_fullsized":"2023-10-21T16:15:16.131814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:09.858244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345520+0000","last_change":"2023-10-21T16:15:15.150507+0000","last_active":"2023-10-21T16:15:16.345520+0000","last_peered":"2023-10-21T16:15:16.345520+0000","last_clean":"2023-10-21T16:15:16.345520+0000","last_became_active":"2023-10-21T16:15:15.148274+0000","last_became_peered":"2023-10-21T16:15:15.148274+0000","last_unstale":"2023-10-21T16:15:16.345520+0000","last_undegraded":"2023-10-21T16:15:16.345520+0000","last_fullsized":"2023-10-21T16:15:16.345520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:59.390383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343616+0000","last_change":"2023-10-21T16:15:15.175870+0000","last_active":"2023-10-21T16:15:16.343616+0000","last_peered":"2023-10-21T16:15:16.343616+0000","last_clean":"2023-10-21T16:15:16.343616+0000","last_became_active":"2023-10-21T16:15:15.175126+0000","last_became_peered":"2023-10-21T16:15:15.175126+0000","last_unstale":"2023-10-21T16:15:16.343616+0000","last_undegraded":"2023-10-21T16:15:16.343616+0000","last_fullsized":"2023-10-21T16:15:16.343616+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:36:17.800515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345866+0000","last_change":"2023-10-21T16:15:15.171486+0000","last_active":"2023-10-21T16:15:16.345866+0000","last_peered":"2023-10-21T16:15:16.345866+0000","last_clean":"2023-10-21T16:15:16.345866+0000","last_became_active":"2023-10-21T16:15:15.171131+0000","last_became_peered":"2023-10-21T16:15:15.171131+0000","last_unstale":"2023-10-21T16:15:16.345866+0000","last_undegraded":"2023-10-21T16:15:16.345866+0000","last_fullsized":"2023-10-21T16:15:16.345866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:07:56.048213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344183+0000","last_change":"2023-10-21T16:15:15.143914+0000","last_active":"2023-10-21T16:15:16.344183+0000","last_peered":"2023-10-21T16:15:16.344183+0000","last_clean":"2023-10-21T16:15:16.344183+0000","last_became_active":"2023-10-21T16:15:15.143182+0000","last_became_peered":"2023-10-21T16:15:15.143182+0000","last_unstale":"2023-10-21T16:15:16.344183+0000","last_undegraded":"2023-10-21T16:15:16.344183+0000","last_fullsized":"2023-10-21T16:15:16.344183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:57.175736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344202+0000","last_change":"2023-10-21T16:15:15.164720+0000","last_active":"2023-10-21T16:15:16.344202+0000","last_peered":"2023-10-21T16:15:16.344202+0000","last_clean":"2023-10-21T16:15:16.344202+0000","last_became_active":"2023-10-21T16:15:15.163636+0000","last_became_peered":"2023-10-21T16:15:15.163636+0000","last_unstale":"2023-10-21T16:15:16.344202+0000","last_undegraded":"2023-10-21T16:15:16.344202+0000","last_fullsized":"2023-10-21T16:15:16.344202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:57.327377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345484+0000","last_change":"2023-10-21T16:15:15.170631+0000","last_active":"2023-10-21T16:15:16.345484+0000","last_peered":"2023-10-21T16:15:16.345484+0000","last_clean":"2023-10-21T16:15:16.345484+0000","last_became_active":"2023-10-21T16:15:15.170093+0000","last_became_peered":"2023-10-21T16:15:15.170093+0000","last_unstale":"2023-10-21T16:15:16.345484+0000","last_undegraded":"2023-10-21T16:15:16.345484+0000","last_fullsized":"2023-10-21T16:15:16.345484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:55:49.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344632+0000","last_change":"2023-10-21T16:15:15.182099+0000","last_active":"2023-10-21T16:15:16.344632+0000","last_peered":"2023-10-21T16:15:16.344632+0000","last_clean":"2023-10-21T16:15:16.344632+0000","last_became_active":"2023-10-21T16:15:15.181813+0000","last_became_peered":"2023-10-21T16:15:15.181813+0000","last_unstale":"2023-10-21T16:15:16.344632+0000","last_undegraded":"2023-10-21T16:15:16.344632+0000","last_fullsized":"2023-10-21T16:15:16.344632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:42.983243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347407+0000","last_change":"2023-10-21T16:15:15.168380+0000","last_active":"2023-10-21T16:15:16.347407+0000","last_peered":"2023-10-21T16:15:16.347407+0000","last_clean":"2023-10-21T16:15:16.347407+0000","last_became_active":"2023-10-21T16:15:15.168031+0000","last_became_peered":"2023-10-21T16:15:15.168031+0000","last_unstale":"2023-10-21T16:15:16.347407+0000","last_undegraded":"2023-10-21T16:15:16.347407+0000","last_fullsized":"2023-10-21T16:15:16.347407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:59:04.723830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347460+0000","last_change":"2023-10-21T16:15:15.167861+0000","last_active":"2023-10-21T16:15:16.347460+0000","last_peered":"2023-10-21T16:15:16.347460+0000","last_clean":"2023-10-21T16:15:16.347460+0000","last_became_active":"2023-10-21T16:15:15.166851+0000","last_became_peered":"2023-10-21T16:15:15.166851+0000","last_unstale":"2023-10-21T16:15:16.347460+0000","last_undegraded":"2023-10-21T16:15:16.347460+0000","last_fullsized":"2023-10-21T16:15:16.347460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:23:21.903410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131634+0000","last_change":"2023-10-21T16:15:15.187260+0000","last_active":"2023-10-21T16:15:16.131634+0000","last_peered":"2023-10-21T16:15:16.131634+0000","last_clean":"2023-10-21T16:15:16.131634+0000","last_became_active":"2023-10-21T16:15:15.186105+0000","last_became_peered":"2023-10-21T16:15:15.186105+0000","last_unstale":"2023-10-21T16:15:16.131634+0000","last_undegraded":"2023-10-21T16:15:16.131634+0000","last_fullsized":"2023-10-21T16:15:16.131634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:15:40.626855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132803+0000","last_change":"2023-10-21T16:15:15.157999+0000","last_active":"2023-10-21T16:15:16.132803+0000","last_peered":"2023-10-21T16:15:16.132803+0000","last_clean":"2023-10-21T16:15:16.132803+0000","last_became_active":"2023-10-21T16:15:15.157648+0000","last_became_peered":"2023-10-21T16:15:15.157648+0000","last_unstale":"2023-10-21T16:15:16.132803+0000","last_undegraded":"2023-10-21T16:15:16.132803+0000","last_fullsized":"2023-10-21T16:15:16.132803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:25.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343251+0000","last_change":"2023-10-21T16:15:15.167680+0000","last_active":"2023-10-21T16:15:16.343251+0000","last_peered":"2023-10-21T16:15:16.343251+0000","last_clean":"2023-10-21T16:15:16.343251+0000","last_became_active":"2023-10-21T16:15:15.167409+0000","last_became_peered":"2023-10-21T16:15:15.167409+0000","last_unstale":"2023-10-21T16:15:16.343251+0000","last_undegraded":"2023-10-21T16:15:16.343251+0000","last_fullsized":"2023-10-21T16:15:16.343251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:37:45.487017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344658+0000","last_change":"2023-10-21T16:15:15.179111+0000","last_active":"2023-10-21T16:15:16.344658+0000","last_peered":"2023-10-21T16:15:16.344658+0000","last_clean":"2023-10-21T16:15:16.344658+0000","last_became_active":"2023-10-21T16:15:15.178134+0000","last_became_peered":"2023-10-21T16:15:15.178134+0000","last_unstale":"2023-10-21T16:15:16.344658+0000","last_undegraded":"2023-10-21T16:15:16.344658+0000","last_fullsized":"2023-10-21T16:15:16.344658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:23.300910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132361+0000","last_change":"2023-10-21T16:15:15.169578+0000","last_active":"2023-10-21T16:15:16.132361+0000","last_peered":"2023-10-21T16:15:16.132361+0000","last_clean":"2023-10-21T16:15:16.132361+0000","last_became_active":"2023-10-21T16:15:15.168927+0000","last_became_peered":"2023-10-21T16:15:15.168927+0000","last_unstale":"2023-10-21T16:15:16.132361+0000","last_undegraded":"2023-10-21T16:15:16.132361+0000","last_fullsized":"2023-10-21T16:15:16.132361+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:12:18.158375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346274+0000","last_change":"2023-10-21T16:15:15.159089+0000","last_active":"2023-10-21T16:15:16.346274+0000","last_peered":"2023-10-21T16:15:16.346274+0000","last_clean":"2023-10-21T16:15:16.346274+0000","last_became_active":"2023-10-21T16:15:15.158826+0000","last_became_peered":"2023-10-21T16:15:15.158826+0000","last_unstale":"2023-10-21T16:15:16.346274+0000","last_undegraded":"2023-10-21T16:15:16.346274+0000","last_fullsized":"2023-10-21T16:15:16.346274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:31.769825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347768+0000","last_change":"2023-10-21T16:15:15.164652+0000","last_active":"2023-10-21T16:15:16.347768+0000","last_peered":"2023-10-21T16:15:16.347768+0000","last_clean":"2023-10-21T16:15:16.347768+0000","last_became_active":"2023-10-21T16:15:15.164313+0000","last_became_peered":"2023-10-21T16:15:15.164313+0000","last_unstale":"2023-10-21T16:15:16.347768+0000","last_undegraded":"2023-10-21T16:15:16.347768+0000","last_fullsized":"2023-10-21T16:15:16.347768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:58.100651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343936+0000","last_change":"2023-10-21T16:15:15.169012+0000","last_active":"2023-10-21T16:15:16.343936+0000","last_peered":"2023-10-21T16:15:16.343936+0000","last_clean":"2023-10-21T16:15:16.343936+0000","last_became_active":"2023-10-21T16:15:15.168675+0000","last_became_peered":"2023-10-21T16:15:15.168675+0000","last_unstale":"2023-10-21T16:15:16.343936+0000","last_undegraded":"2023-10-21T16:15:16.343936+0000","last_fullsized":"2023-10-21T16:15:16.343936+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:00:09.665089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344937+0000","last_change":"2023-10-21T16:15:15.169421+0000","last_active":"2023-10-21T16:15:16.344937+0000","last_peered":"2023-10-21T16:15:16.344937+0000","last_clean":"2023-10-21T16:15:16.344937+0000","last_became_active":"2023-10-21T16:15:15.169114+0000","last_became_peered":"2023-10-21T16:15:15.169114+0000","last_unstale":"2023-10-21T16:15:16.344937+0000","last_undegraded":"2023-10-21T16:15:16.344937+0000","last_fullsized":"2023-10-21T16:15:16.344937+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:44.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343227+0000","last_change":"2023-10-21T16:15:15.166358+0000","last_active":"2023-10-21T16:15:16.343227+0000","last_peered":"2023-10-21T16:15:16.343227+0000","last_clean":"2023-10-21T16:15:16.343227+0000","last_became_active":"2023-10-21T16:15:15.166100+0000","last_became_peered":"2023-10-21T16:15:15.166100+0000","last_unstale":"2023-10-21T16:15:16.343227+0000","last_undegraded":"2023-10-21T16:15:16.343227+0000","last_fullsized":"2023-10-21T16:15:16.343227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:50.332580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343847+0000","last_change":"2023-10-21T16:15:15.164650+0000","last_active":"2023-10-21T16:15:16.343847+0000","last_peered":"2023-10-21T16:15:16.343847+0000","last_clean":"2023-10-21T16:15:16.343847+0000","last_became_active":"2023-10-21T16:15:15.160824+0000","last_became_peered":"2023-10-21T16:15:15.160824+0000","last_unstale":"2023-10-21T16:15:16.343847+0000","last_undegraded":"2023-10-21T16:15:16.343847+0000","last_fullsized":"2023-10-21T16:15:16.343847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:41.282186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345125+0000","last_change":"2023-10-21T16:15:15.169834+0000","last_active":"2023-10-21T16:15:16.345125+0000","last_peered":"2023-10-21T16:15:16.345125+0000","last_clean":"2023-10-21T16:15:16.345125+0000","last_became_active":"2023-10-21T16:15:15.169506+0000","last_became_peered":"2023-10-21T16:15:15.169506+0000","last_unstale":"2023-10-21T16:15:16.345125+0000","last_undegraded":"2023-10-21T16:15:16.345125+0000","last_fullsized":"2023-10-21T16:15:16.345125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:40.166519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"71'4","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347552+0000","last_change":"2023-10-21T16:15:14.129206+0000","last_active":"2023-10-21T16:15:16.347552+0000","last_peered":"2023-10-21T16:15:16.347552+0000","last_clean":"2023-10-21T16:15:16.347552+0000","last_became_active":"2023-10-21T16:15:14.128887+0000","last_became_peered":"2023-10-21T16:15:14.128887+0000","last_unstale":"2023-10-21T16:15:16.347552+0000","last_undegraded":"2023-10-21T16:15:16.347552+0000","last_fullsized":"2023-10-21T16:15:16.347552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:23.192369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343889+0000","last_change":"2023-10-21T16:15:15.162035+0000","last_active":"2023-10-21T16:15:16.343889+0000","last_peered":"2023-10-21T16:15:16.343889+0000","last_clean":"2023-10-21T16:15:16.343889+0000","last_became_active":"2023-10-21T16:15:15.161621+0000","last_became_peered":"2023-10-21T16:15:15.161621+0000","last_unstale":"2023-10-21T16:15:16.343889+0000","last_undegraded":"2023-10-21T16:15:16.343889+0000","last_fullsized":"2023-10-21T16:15:16.343889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:09.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346709+0000","last_change":"2023-10-21T16:15:14.143580+0000","last_active":"2023-10-21T16:15:16.346709+0000","last_peered":"2023-10-21T16:15:16.346709+0000","last_clean":"2023-10-21T16:15:16.346709+0000","last_became_active":"2023-10-21T16:15:14.142794+0000","last_became_peered":"2023-10-21T16:15:14.142794+0000","last_unstale":"2023-10-21T16:15:16.346709+0000","last_undegraded":"2023-10-21T16:15:16.346709+0000","last_fullsized":"2023-10-21T16:15:16.346709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:39.403925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347130+0000","last_change":"2023-10-21T16:15:15.165643+0000","last_active":"2023-10-21T16:15:16.347130+0000","last_peered":"2023-10-21T16:15:16.347130+0000","last_clean":"2023-10-21T16:15:16.347130+0000","last_became_active":"2023-10-21T16:15:15.164402+0000","last_became_peered":"2023-10-21T16:15:15.164402+0000","last_unstale":"2023-10-21T16:15:16.347130+0000","last_undegraded":"2023-10-21T16:15:16.347130+0000","last_fullsized":"2023-10-21T16:15:16.347130+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:45:48.319841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346196+0000","last_change":"2023-10-21T16:15:14.138562+0000","last_active":"2023-10-21T16:15:16.346196+0000","last_peered":"2023-10-21T16:15:16.346196+0000","last_clean":"2023-10-21T16:15:16.346196+0000","last_became_active":"2023-10-21T16:15:14.137280+0000","last_became_peered":"2023-10-21T16:15:14.137280+0000","last_unstale":"2023-10-21T16:15:16.346196+0000","last_undegraded":"2023-10-21T16:15:16.346196+0000","last_fullsized":"2023-10-21T16:15:16.346196+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:47:32.188866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131608+0000","last_change":"2023-10-21T16:15:15.159423+0000","last_active":"2023-10-21T16:15:16.131608+0000","last_peered":"2023-10-21T16:15:16.131608+0000","last_clean":"2023-10-21T16:15:16.131608+0000","last_became_active":"2023-10-21T16:15:15.159131+0000","last_became_peered":"2023-10-21T16:15:15.159131+0000","last_unstale":"2023-10-21T16:15:16.131608+0000","last_undegraded":"2023-10-21T16:15:16.131608+0000","last_fullsized":"2023-10-21T16:15:16.131608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:43.781262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344071+0000","last_change":"2023-10-21T16:15:14.132213+0000","last_active":"2023-10-21T16:15:16.344071+0000","last_peered":"2023-10-21T16:15:16.344071+0000","last_clean":"2023-10-21T16:15:16.344071+0000","last_became_active":"2023-10-21T16:15:14.131899+0000","last_became_peered":"2023-10-21T16:15:14.131899+0000","last_unstale":"2023-10-21T16:15:16.344071+0000","last_undegraded":"2023-10-21T16:15:16.344071+0000","last_fullsized":"2023-10-21T16:15:16.344071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:14:48.327976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136103+0000","last_change":"2023-10-21T16:15:15.175489+0000","last_active":"2023-10-21T16:15:16.136103+0000","last_peered":"2023-10-21T16:15:16.136103+0000","last_clean":"2023-10-21T16:15:16.136103+0000","last_became_active":"2023-10-21T16:15:15.175120+0000","last_became_peered":"2023-10-21T16:15:15.175120+0000","last_unstale":"2023-10-21T16:15:16.136103+0000","last_undegraded":"2023-10-21T16:15:16.136103+0000","last_fullsized":"2023-10-21T16:15:16.136103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:34:15.676088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344075+0000","last_change":"2023-10-21T16:15:14.132492+0000","last_active":"2023-10-21T16:15:16.344075+0000","last_peered":"2023-10-21T16:15:16.344075+0000","last_clean":"2023-10-21T16:15:16.344075+0000","last_became_active":"2023-10-21T16:15:14.132024+0000","last_became_peered":"2023-10-21T16:15:14.132024+0000","last_unstale":"2023-10-21T16:15:16.344075+0000","last_undegraded":"2023-10-21T16:15:16.344075+0000","last_fullsized":"2023-10-21T16:15:16.344075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:22.360100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344965+0000","last_change":"2023-10-21T16:15:15.183915+0000","last_active":"2023-10-21T16:15:16.344965+0000","last_peered":"2023-10-21T16:15:16.344965+0000","last_clean":"2023-10-21T16:15:16.344965+0000","last_became_active":"2023-10-21T16:15:15.183662+0000","last_became_peered":"2023-10-21T16:15:15.183662+0000","last_unstale":"2023-10-21T16:15:16.344965+0000","last_undegraded":"2023-10-21T16:15:16.344965+0000","last_fullsized":"2023-10-21T16:15:16.344965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:01.951186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135404+0000","last_change":"2023-10-21T16:15:14.127769+0000","last_active":"2023-10-21T16:15:16.135404+0000","last_peered":"2023-10-21T16:15:16.135404+0000","last_clean":"2023-10-21T16:15:16.135404+0000","last_became_active":"2023-10-21T16:15:14.127287+0000","last_became_peered":"2023-10-21T16:15:14.127287+0000","last_unstale":"2023-10-21T16:15:16.135404+0000","last_undegraded":"2023-10-21T16:15:16.135404+0000","last_fullsized":"2023-10-21T16:15:16.135404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:42.412349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132620+0000","last_change":"2023-10-21T16:15:15.179343+0000","last_active":"2023-10-21T16:15:16.132620+0000","last_peered":"2023-10-21T16:15:16.132620+0000","last_clean":"2023-10-21T16:15:16.132620+0000","last_became_active":"2023-10-21T16:15:15.169289+0000","last_became_peered":"2023-10-21T16:15:15.169289+0000","last_unstale":"2023-10-21T16:15:16.132620+0000","last_undegraded":"2023-10-21T16:15:16.132620+0000","last_fullsized":"2023-10-21T16:15:16.132620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:01.484334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346962+0000","last_change":"2023-10-21T16:15:14.130970+0000","last_active":"2023-10-21T16:15:16.346962+0000","last_peered":"2023-10-21T16:15:16.346962+0000","last_clean":"2023-10-21T16:15:16.346962+0000","last_became_active":"2023-10-21T16:15:14.130311+0000","last_became_peered":"2023-10-21T16:15:14.130311+0000","last_unstale":"2023-10-21T16:15:16.346962+0000","last_undegraded":"2023-10-21T16:15:16.346962+0000","last_fullsized":"2023-10-21T16:15:16.346962+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:27.551565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344387+0000","last_change":"2023-10-21T16:15:15.178413+0000","last_active":"2023-10-21T16:15:16.344387+0000","last_peered":"2023-10-21T16:15:16.344387+0000","last_clean":"2023-10-21T16:15:16.344387+0000","last_became_active":"2023-10-21T16:15:15.178134+0000","last_became_peered":"2023-10-21T16:15:15.178134+0000","last_unstale":"2023-10-21T16:15:16.344387+0000","last_undegraded":"2023-10-21T16:15:16.344387+0000","last_fullsized":"2023-10-21T16:15:16.344387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:30.519586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343060+0000","last_change":"2023-10-21T16:15:14.126182+0000","last_active":"2023-10-21T16:15:16.343060+0000","last_peered":"2023-10-21T16:15:16.343060+0000","last_clean":"2023-10-21T16:15:16.343060+0000","last_became_active":"2023-10-21T16:15:14.125913+0000","last_became_peered":"2023-10-21T16:15:14.125913+0000","last_unstale":"2023-10-21T16:15:16.343060+0000","last_undegraded":"2023-10-21T16:15:16.343060+0000","last_fullsized":"2023-10-21T16:15:16.343060+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:20.773548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135329+0000","last_change":"2023-10-21T16:15:15.162339+0000","last_active":"2023-10-21T16:15:16.135329+0000","last_peered":"2023-10-21T16:15:16.135329+0000","last_clean":"2023-10-21T16:15:16.135329+0000","last_became_active":"2023-10-21T16:15:15.161520+0000","last_became_peered":"2023-10-21T16:15:15.161520+0000","last_unstale":"2023-10-21T16:15:16.135329+0000","last_undegraded":"2023-10-21T16:15:16.135329+0000","last_fullsized":"2023-10-21T16:15:16.135329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:14.919605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344661+0000","last_change":"2023-10-21T16:15:14.130505+0000","last_active":"2023-10-21T16:15:16.344661+0000","last_peered":"2023-10-21T16:15:16.344661+0000","last_clean":"2023-10-21T16:15:16.344661+0000","last_became_active":"2023-10-21T16:15:14.130218+0000","last_became_peered":"2023-10-21T16:15:14.130218+0000","last_unstale":"2023-10-21T16:15:16.344661+0000","last_undegraded":"2023-10-21T16:15:16.344661+0000","last_fullsized":"2023-10-21T16:15:16.344661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:26:04.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343504+0000","last_change":"2023-10-21T16:15:15.161969+0000","last_active":"2023-10-21T16:15:16.343504+0000","last_peered":"2023-10-21T16:15:16.343504+0000","last_clean":"2023-10-21T16:15:16.343504+0000","last_became_active":"2023-10-21T16:15:15.157990+0000","last_became_peered":"2023-10-21T16:15:15.157990+0000","last_unstale":"2023-10-21T16:15:16.343504+0000","last_undegraded":"2023-10-21T16:15:16.343504+0000","last_fullsized":"2023-10-21T16:15:16.343504+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:02:18.621279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136313+0000","last_change":"2023-10-21T16:15:14.137687+0000","last_active":"2023-10-21T16:15:16.136313+0000","last_peered":"2023-10-21T16:15:16.136313+0000","last_clean":"2023-10-21T16:15:16.136313+0000","last_became_active":"2023-10-21T16:15:14.137336+0000","last_became_peered":"2023-10-21T16:15:14.137336+0000","last_unstale":"2023-10-21T16:15:16.136313+0000","last_undegraded":"2023-10-21T16:15:16.136313+0000","last_fullsized":"2023-10-21T16:15:16.136313+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:33.007836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343861+0000","last_change":"2023-10-21T16:15:15.176131+0000","last_active":"2023-10-21T16:15:16.343861+0000","last_peered":"2023-10-21T16:15:16.343861+0000","last_clean":"2023-10-21T16:15:16.343861+0000","last_became_active":"2023-10-21T16:15:15.175480+0000","last_became_peered":"2023-10-21T16:15:15.175480+0000","last_unstale":"2023-10-21T16:15:16.343861+0000","last_undegraded":"2023-10-21T16:15:16.343861+0000","last_fullsized":"2023-10-21T16:15:16.343861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:54.706460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345416+0000","last_change":"2023-10-21T16:15:14.140547+0000","last_active":"2023-10-21T16:15:16.345416+0000","last_peered":"2023-10-21T16:15:16.345416+0000","last_clean":"2023-10-21T16:15:16.345416+0000","last_became_active":"2023-10-21T16:15:14.140055+0000","last_became_peered":"2023-10-21T16:15:14.140055+0000","last_unstale":"2023-10-21T16:15:16.345416+0000","last_undegraded":"2023-10-21T16:15:16.345416+0000","last_fullsized":"2023-10-21T16:15:16.345416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:22.286324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136421+0000","last_change":"2023-10-21T16:15:15.152639+0000","last_active":"2023-10-21T16:15:16.136421+0000","last_peered":"2023-10-21T16:15:16.136421+0000","last_clean":"2023-10-21T16:15:16.136421+0000","last_became_active":"2023-10-21T16:15:15.151821+0000","last_became_peered":"2023-10-21T16:15:15.151821+0000","last_unstale":"2023-10-21T16:15:16.136421+0000","last_undegraded":"2023-10-21T16:15:16.136421+0000","last_fullsized":"2023-10-21T16:15:16.136421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:40.759854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"71'6","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131973+0000","last_change":"2023-10-21T16:15:14.130084+0000","last_active":"2023-10-21T16:15:16.131973+0000","last_peered":"2023-10-21T16:15:16.131973+0000","last_clean":"2023-10-21T16:15:16.131973+0000","last_became_active":"2023-10-21T16:15:14.129788+0000","last_became_peered":"2023-10-21T16:15:14.129788+0000","last_unstale":"2023-10-21T16:15:16.131973+0000","last_undegraded":"2023-10-21T16:15:16.131973+0000","last_fullsized":"2023-10-21T16:15:16.131973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:37.224728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343452+0000","last_change":"2023-10-21T16:15:15.161670+0000","last_active":"2023-10-21T16:15:16.343452+0000","last_peered":"2023-10-21T16:15:16.343452+0000","last_clean":"2023-10-21T16:15:16.343452+0000","last_became_active":"2023-10-21T16:15:15.161344+0000","last_became_peered":"2023-10-21T16:15:15.161344+0000","last_unstale":"2023-10-21T16:15:16.343452+0000","last_undegraded":"2023-10-21T16:15:16.343452+0000","last_fullsized":"2023-10-21T16:15:16.343452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:01.206148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344217+0000","last_change":"2023-10-21T16:15:14.135807+0000","last_active":"2023-10-21T16:15:16.344217+0000","last_peered":"2023-10-21T16:15:16.344217+0000","last_clean":"2023-10-21T16:15:16.344217+0000","last_became_active":"2023-10-21T16:15:14.135403+0000","last_became_peered":"2023-10-21T16:15:14.135403+0000","last_unstale":"2023-10-21T16:15:16.344217+0000","last_undegraded":"2023-10-21T16:15:16.344217+0000","last_fullsized":"2023-10-21T16:15:16.344217+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:45.579353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344700+0000","last_change":"2023-10-21T16:15:15.185854+0000","last_active":"2023-10-21T16:15:16.344700+0000","last_peered":"2023-10-21T16:15:16.344700+0000","last_clean":"2023-10-21T16:15:16.344700+0000","last_became_active":"2023-10-21T16:15:15.184136+0000","last_became_peered":"2023-10-21T16:15:15.184136+0000","last_unstale":"2023-10-21T16:15:16.344700+0000","last_undegraded":"2023-10-21T16:15:16.344700+0000","last_fullsized":"2023-10-21T16:15:16.344700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:51.122703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347293+0000","last_change":"2023-10-21T16:15:14.136973+0000","last_active":"2023-10-21T16:15:16.347293+0000","last_peered":"2023-10-21T16:15:16.347293+0000","last_clean":"2023-10-21T16:15:16.347293+0000","last_became_active":"2023-10-21T16:15:14.136699+0000","last_became_peered":"2023-10-21T16:15:14.136699+0000","last_unstale":"2023-10-21T16:15:16.347293+0000","last_undegraded":"2023-10-21T16:15:16.347293+0000","last_fullsized":"2023-10-21T16:15:16.347293+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:07:04.467622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132365+0000","last_change":"2023-10-21T16:15:15.163276+0000","last_active":"2023-10-21T16:15:16.132365+0000","last_peered":"2023-10-21T16:15:16.132365+0000","last_clean":"2023-10-21T16:15:16.132365+0000","last_became_active":"2023-10-21T16:15:15.162999+0000","last_became_peered":"2023-10-21T16:15:15.162999+0000","last_unstale":"2023-10-21T16:15:16.132365+0000","last_undegraded":"2023-10-21T16:15:16.132365+0000","last_fullsized":"2023-10-21T16:15:16.132365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:30:51.816899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346009+0000","last_change":"2023-10-21T16:15:14.130710+0000","last_active":"2023-10-21T16:15:16.346009+0000","last_peered":"2023-10-21T16:15:16.346009+0000","last_clean":"2023-10-21T16:15:16.346009+0000","last_became_active":"2023-10-21T16:15:14.129585+0000","last_became_peered":"2023-10-21T16:15:14.129585+0000","last_unstale":"2023-10-21T16:15:16.346009+0000","last_undegraded":"2023-10-21T16:15:16.346009+0000","last_fullsized":"2023-10-21T16:15:16.346009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:56.312791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344351+0000","last_change":"2023-10-21T16:15:15.163981+0000","last_active":"2023-10-21T16:15:16.344351+0000","last_peered":"2023-10-21T16:15:16.344351+0000","last_clean":"2023-10-21T16:15:16.344351+0000","last_became_active":"2023-10-21T16:15:15.161362+0000","last_became_peered":"2023-10-21T16:15:15.161362+0000","last_unstale":"2023-10-21T16:15:16.344351+0000","last_undegraded":"2023-10-21T16:15:16.344351+0000","last_fullsized":"2023-10-21T16:15:16.344351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:24.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131848+0000","last_change":"2023-10-21T16:15:14.141613+0000","last_active":"2023-10-21T16:15:16.131848+0000","last_peered":"2023-10-21T16:15:16.131848+0000","last_clean":"2023-10-21T16:15:16.131848+0000","last_became_active":"2023-10-21T16:15:14.140539+0000","last_became_peered":"2023-10-21T16:15:14.140539+0000","last_unstale":"2023-10-21T16:15:16.131848+0000","last_undegraded":"2023-10-21T16:15:16.131848+0000","last_fullsized":"2023-10-21T16:15:16.131848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:02:17.606693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131716+0000","last_change":"2023-10-21T16:15:15.159705+0000","last_active":"2023-10-21T16:15:16.131716+0000","last_peered":"2023-10-21T16:15:16.131716+0000","last_clean":"2023-10-21T16:15:16.131716+0000","last_became_active":"2023-10-21T16:15:15.159410+0000","last_became_peered":"2023-10-21T16:15:15.159410+0000","last_unstale":"2023-10-21T16:15:16.131716+0000","last_undegraded":"2023-10-21T16:15:16.131716+0000","last_fullsized":"2023-10-21T16:15:16.131716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:31:42.531013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344450+0000","last_change":"2023-10-21T16:15:14.138208+0000","last_active":"2023-10-21T16:15:16.344450+0000","last_peered":"2023-10-21T16:15:16.344450+0000","last_clean":"2023-10-21T16:15:16.344450+0000","last_became_active":"2023-10-21T16:15:14.137474+0000","last_became_peered":"2023-10-21T16:15:14.137474+0000","last_unstale":"2023-10-21T16:15:16.344450+0000","last_undegraded":"2023-10-21T16:15:16.344450+0000","last_fullsized":"2023-10-21T16:15:16.344450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:39.856113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345219+0000","last_change":"2023-10-21T16:15:15.167813+0000","last_active":"2023-10-21T16:15:16.345219+0000","last_peered":"2023-10-21T16:15:16.345219+0000","last_clean":"2023-10-21T16:15:16.345219+0000","last_became_active":"2023-10-21T16:15:15.166307+0000","last_became_peered":"2023-10-21T16:15:15.166307+0000","last_unstale":"2023-10-21T16:15:16.345219+0000","last_undegraded":"2023-10-21T16:15:16.345219+0000","last_fullsized":"2023-10-21T16:15:16.345219+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:53:47.031790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.133070+0000","last_change":"2023-10-21T16:15:14.110821+0000","last_active":"2023-10-21T16:15:16.133070+0000","last_peered":"2023-10-21T16:15:16.133070+0000","last_clean":"2023-10-21T16:15:16.133070+0000","last_became_active":"2023-10-21T16:15:14.109764+0000","last_became_peered":"2023-10-21T16:15:14.109764+0000","last_unstale":"2023-10-21T16:15:16.133070+0000","last_undegraded":"2023-10-21T16:15:16.133070+0000","last_fullsized":"2023-10-21T16:15:16.133070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:33:21.253930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135432+0000","last_change":"2023-10-21T16:15:15.175816+0000","last_active":"2023-10-21T16:15:16.135432+0000","last_peered":"2023-10-21T16:15:16.135432+0000","last_clean":"2023-10-21T16:15:16.135432+0000","last_became_active":"2023-10-21T16:15:15.175562+0000","last_became_peered":"2023-10-21T16:15:15.175562+0000","last_unstale":"2023-10-21T16:15:16.135432+0000","last_undegraded":"2023-10-21T16:15:16.135432+0000","last_fullsized":"2023-10-21T16:15:16.135432+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:34.819605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347822+0000","last_change":"2023-10-21T16:15:14.140748+0000","last_active":"2023-10-21T16:15:16.347822+0000","last_peered":"2023-10-21T16:15:16.347822+0000","last_clean":"2023-10-21T16:15:16.347822+0000","last_became_active":"2023-10-21T16:15:14.140297+0000","last_became_peered":"2023-10-21T16:15:14.140297+0000","last_unstale":"2023-10-21T16:15:16.347822+0000","last_undegraded":"2023-10-21T16:15:16.347822+0000","last_fullsized":"2023-10-21T16:15:16.347822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:55.387335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345774+0000","last_change":"2023-10-21T16:15:15.148751+0000","last_active":"2023-10-21T16:15:16.345774+0000","last_peered":"2023-10-21T16:15:16.345774+0000","last_clean":"2023-10-21T16:15:16.345774+0000","last_became_active":"2023-10-21T16:15:15.147602+0000","last_became_peered":"2023-10-21T16:15:15.147602+0000","last_unstale":"2023-10-21T16:15:16.345774+0000","last_undegraded":"2023-10-21T16:15:16.345774+0000","last_fullsized":"2023-10-21T16:15:16.345774+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:54:58.214345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347862+0000","last_change":"2023-10-21T16:15:14.138362+0000","last_active":"2023-10-21T16:15:16.347862+0000","last_peered":"2023-10-21T16:15:16.347862+0000","last_clean":"2023-10-21T16:15:16.347862+0000","last_became_active":"2023-10-21T16:15:14.138054+0000","last_became_peered":"2023-10-21T16:15:14.138054+0000","last_unstale":"2023-10-21T16:15:16.347862+0000","last_undegraded":"2023-10-21T16:15:16.347862+0000","last_fullsized":"2023-10-21T16:15:16.347862+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:08:03.060972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343822+0000","last_change":"2023-10-21T16:15:15.164297+0000","last_active":"2023-10-21T16:15:16.343822+0000","last_peered":"2023-10-21T16:15:16.343822+0000","last_clean":"2023-10-21T16:15:16.343822+0000","last_became_active":"2023-10-21T16:15:15.163948+0000","last_became_peered":"2023-10-21T16:15:15.163948+0000","last_unstale":"2023-10-21T16:15:16.343822+0000","last_undegraded":"2023-10-21T16:15:16.343822+0000","last_fullsized":"2023-10-21T16:15:16.343822+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:38.239314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345738+0000","last_change":"2023-10-21T16:15:14.135993+0000","last_active":"2023-10-21T16:15:16.345738+0000","last_peered":"2023-10-21T16:15:16.345738+0000","last_clean":"2023-10-21T16:15:16.345738+0000","last_became_active":"2023-10-21T16:15:14.135657+0000","last_became_peered":"2023-10-21T16:15:14.135657+0000","last_unstale":"2023-10-21T16:15:16.345738+0000","last_undegraded":"2023-10-21T16:15:16.345738+0000","last_fullsized":"2023-10-21T16:15:16.345738+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:59:03.709244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347666+0000","last_change":"2023-10-21T16:15:15.169186+0000","last_active":"2023-10-21T16:15:16.347666+0000","last_peered":"2023-10-21T16:15:16.347666+0000","last_clean":"2023-10-21T16:15:16.347666+0000","last_became_active":"2023-10-21T16:15:15.168886+0000","last_became_peered":"2023-10-21T16:15:15.168886+0000","last_unstale":"2023-10-21T16:15:16.347666+0000","last_undegraded":"2023-10-21T16:15:16.347666+0000","last_fullsized":"2023-10-21T16:15:16.347666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:40.418511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131613+0000","last_change":"2023-10-21T16:15:14.129573+0000","last_active":"2023-10-21T16:15:16.131613+0000","last_peered":"2023-10-21T16:15:16.131613+0000","last_clean":"2023-10-21T16:15:16.131613+0000","last_became_active":"2023-10-21T16:15:14.129205+0000","last_became_peered":"2023-10-21T16:15:14.129205+0000","last_unstale":"2023-10-21T16:15:16.131613+0000","last_undegraded":"2023-10-21T16:15:16.131613+0000","last_fullsized":"2023-10-21T16:15:16.131613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:08.395341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344983+0000","last_change":"2023-10-21T16:15:15.171139+0000","last_active":"2023-10-21T16:15:16.344983+0000","last_peered":"2023-10-21T16:15:16.344983+0000","last_clean":"2023-10-21T16:15:16.344983+0000","last_became_active":"2023-10-21T16:15:15.170847+0000","last_became_peered":"2023-10-21T16:15:15.170847+0000","last_unstale":"2023-10-21T16:15:16.344983+0000","last_undegraded":"2023-10-21T16:15:16.344983+0000","last_fullsized":"2023-10-21T16:15:16.344983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:26:05.210587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343259+0000","last_change":"2023-10-21T16:15:14.140383+0000","last_active":"2023-10-21T16:15:16.343259+0000","last_peered":"2023-10-21T16:15:16.343259+0000","last_clean":"2023-10-21T16:15:16.343259+0000","last_became_active":"2023-10-21T16:15:14.140051+0000","last_became_peered":"2023-10-21T16:15:14.140051+0000","last_unstale":"2023-10-21T16:15:16.343259+0000","last_undegraded":"2023-10-21T16:15:16.343259+0000","last_fullsized":"2023-10-21T16:15:16.343259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:00.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346582+0000","last_change":"2023-10-21T16:15:15.177152+0000","last_active":"2023-10-21T16:15:16.346582+0000","last_peered":"2023-10-21T16:15:16.346582+0000","last_clean":"2023-10-21T16:15:16.346582+0000","last_became_active":"2023-10-21T16:15:15.175508+0000","last_became_peered":"2023-10-21T16:15:15.175508+0000","last_unstale":"2023-10-21T16:15:16.346582+0000","last_undegraded":"2023-10-21T16:15:16.346582+0000","last_fullsized":"2023-10-21T16:15:16.346582+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:43.426934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132595+0000","last_change":"2023-10-21T16:15:14.122127+0000","last_active":"2023-10-21T16:15:16.132595+0000","last_peered":"2023-10-21T16:15:16.132595+0000","last_clean":"2023-10-21T16:15:16.132595+0000","last_became_active":"2023-10-21T16:15:14.121120+0000","last_became_peered":"2023-10-21T16:15:14.121120+0000","last_unstale":"2023-10-21T16:15:16.132595+0000","last_undegraded":"2023-10-21T16:15:16.132595+0000","last_fullsized":"2023-10-21T16:15:16.132595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:36:37.374348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343464+0000","last_change":"2023-10-21T16:15:15.164762+0000","last_active":"2023-10-21T16:15:16.343464+0000","last_peered":"2023-10-21T16:15:16.343464+0000","last_clean":"2023-10-21T16:15:16.343464+0000","last_became_active":"2023-10-21T16:15:15.161224+0000","last_became_peered":"2023-10-21T16:15:15.161224+0000","last_unstale":"2023-10-21T16:15:16.343464+0000","last_undegraded":"2023-10-21T16:15:16.343464+0000","last_fullsized":"2023-10-21T16:15:16.343464+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:18.236248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347678+0000","last_change":"2023-10-21T16:15:14.137016+0000","last_active":"2023-10-21T16:15:16.347678+0000","last_peered":"2023-10-21T16:15:16.347678+0000","last_clean":"2023-10-21T16:15:16.347678+0000","last_became_active":"2023-10-21T16:15:14.136682+0000","last_became_peered":"2023-10-21T16:15:14.136682+0000","last_unstale":"2023-10-21T16:15:16.347678+0000","last_undegraded":"2023-10-21T16:15:16.347678+0000","last_fullsized":"2023-10-21T16:15:16.347678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:08.466171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345588+0000","last_change":"2023-10-21T16:15:15.161754+0000","last_active":"2023-10-21T16:15:16.345588+0000","last_peered":"2023-10-21T16:15:16.345588+0000","last_clean":"2023-10-21T16:15:16.345588+0000","last_became_active":"2023-10-21T16:15:15.161459+0000","last_became_peered":"2023-10-21T16:15:15.161459+0000","last_unstale":"2023-10-21T16:15:16.345588+0000","last_undegraded":"2023-10-21T16:15:16.345588+0000","last_fullsized":"2023-10-21T16:15:16.345588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:11.943378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345633+0000","last_change":"2023-10-21T16:15:14.123848+0000","last_active":"2023-10-21T16:15:16.345633+0000","last_peered":"2023-10-21T16:15:16.345633+0000","last_clean":"2023-10-21T16:15:16.345633+0000","last_became_active":"2023-10-21T16:15:14.123520+0000","last_became_peered":"2023-10-21T16:15:14.123520+0000","last_unstale":"2023-10-21T16:15:16.345633+0000","last_undegraded":"2023-10-21T16:15:16.345633+0000","last_fullsized":"2023-10-21T16:15:16.345633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:50.108117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131917+0000","last_change":"2023-10-21T16:15:15.164413+0000","last_active":"2023-10-21T16:15:16.131917+0000","last_peered":"2023-10-21T16:15:16.131917+0000","last_clean":"2023-10-21T16:15:16.131917+0000","last_became_active":"2023-10-21T16:15:15.164151+0000","last_became_peered":"2023-10-21T16:15:15.164151+0000","last_unstale":"2023-10-21T16:15:16.131917+0000","last_undegraded":"2023-10-21T16:15:16.131917+0000","last_fullsized":"2023-10-21T16:15:16.131917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:41:08.955167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344509+0000","last_change":"2023-10-21T16:15:14.124632+0000","last_active":"2023-10-21T16:15:16.344509+0000","last_peered":"2023-10-21T16:15:16.344509+0000","last_clean":"2023-10-21T16:15:16.344509+0000","last_became_active":"2023-10-21T16:15:14.123070+0000","last_became_peered":"2023-10-21T16:15:14.123070+0000","last_unstale":"2023-10-21T16:15:16.344509+0000","last_undegraded":"2023-10-21T16:15:16.344509+0000","last_fullsized":"2023-10-21T16:15:16.344509+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:48:27.256478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344284+0000","last_change":"2023-10-21T16:15:15.172753+0000","last_active":"2023-10-21T16:15:16.344284+0000","last_peered":"2023-10-21T16:15:16.344284+0000","last_clean":"2023-10-21T16:15:16.344284+0000","last_became_active":"2023-10-21T16:15:15.171529+0000","last_became_peered":"2023-10-21T16:15:15.171529+0000","last_unstale":"2023-10-21T16:15:16.344284+0000","last_undegraded":"2023-10-21T16:15:16.344284+0000","last_fullsized":"2023-10-21T16:15:16.344284+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:54:35.680365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132038+0000","last_change":"2023-10-21T16:15:14.128533+0000","last_active":"2023-10-21T16:15:16.132038+0000","last_peered":"2023-10-21T16:15:16.132038+0000","last_clean":"2023-10-21T16:15:16.132038+0000","last_became_active":"2023-10-21T16:15:14.128263+0000","last_became_peered":"2023-10-21T16:15:14.128263+0000","last_unstale":"2023-10-21T16:15:16.132038+0000","last_undegraded":"2023-10-21T16:15:16.132038+0000","last_fullsized":"2023-10-21T16:15:16.132038+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:08.727743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344189+0000","last_change":"2023-10-21T16:15:15.163219+0000","last_active":"2023-10-21T16:15:16.344189+0000","last_peered":"2023-10-21T16:15:16.344189+0000","last_clean":"2023-10-21T16:15:16.344189+0000","last_became_active":"2023-10-21T16:15:15.162037+0000","last_became_peered":"2023-10-21T16:15:15.162037+0000","last_unstale":"2023-10-21T16:15:16.344189+0000","last_undegraded":"2023-10-21T16:15:16.344189+0000","last_fullsized":"2023-10-21T16:15:16.344189+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:55:54.669363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131721+0000","last_change":"2023-10-21T16:15:14.143628+0000","last_active":"2023-10-21T16:15:16.131721+0000","last_peered":"2023-10-21T16:15:16.131721+0000","last_clean":"2023-10-21T16:15:16.131721+0000","last_became_active":"2023-10-21T16:15:14.143077+0000","last_became_peered":"2023-10-21T16:15:14.143077+0000","last_unstale":"2023-10-21T16:15:16.131721+0000","last_undegraded":"2023-10-21T16:15:16.131721+0000","last_fullsized":"2023-10-21T16:15:16.131721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:17.221662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344842+0000","last_change":"2023-10-21T16:15:15.165299+0000","last_active":"2023-10-21T16:15:16.344842+0000","last_peered":"2023-10-21T16:15:16.344842+0000","last_clean":"2023-10-21T16:15:16.344842+0000","last_became_active":"2023-10-21T16:15:15.165020+0000","last_became_peered":"2023-10-21T16:15:15.165020+0000","last_unstale":"2023-10-21T16:15:16.344842+0000","last_undegraded":"2023-10-21T16:15:16.344842+0000","last_fullsized":"2023-10-21T16:15:16.344842+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.135495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135706+0000","last_change":"2023-10-21T16:15:14.132685+0000","last_active":"2023-10-21T16:15:16.135706+0000","last_peered":"2023-10-21T16:15:16.135706+0000","last_clean":"2023-10-21T16:15:16.135706+0000","last_became_active":"2023-10-21T16:15:14.132306+0000","last_became_peered":"2023-10-21T16:15:14.132306+0000","last_unstale":"2023-10-21T16:15:16.135706+0000","last_undegraded":"2023-10-21T16:15:16.135706+0000","last_fullsized":"2023-10-21T16:15:16.135706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:05.378738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132663+0000","last_change":"2023-10-21T16:15:15.174061+0000","last_active":"2023-10-21T16:15:16.132663+0000","last_peered":"2023-10-21T16:15:16.132663+0000","last_clean":"2023-10-21T16:15:16.132663+0000","last_became_active":"2023-10-21T16:15:15.163769+0000","last_became_peered":"2023-10-21T16:15:15.163769+0000","last_unstale":"2023-10-21T16:15:16.132663+0000","last_undegraded":"2023-10-21T16:15:16.132663+0000","last_fullsized":"2023-10-21T16:15:16.132663+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:50.487376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343711+0000","last_change":"2023-10-21T16:15:14.138642+0000","last_active":"2023-10-21T16:15:16.343711+0000","last_peered":"2023-10-21T16:15:16.343711+0000","last_clean":"2023-10-21T16:15:16.343711+0000","last_became_active":"2023-10-21T16:15:14.138178+0000","last_became_peered":"2023-10-21T16:15:14.138178+0000","last_unstale":"2023-10-21T16:15:16.343711+0000","last_undegraded":"2023-10-21T16:15:16.343711+0000","last_fullsized":"2023-10-21T16:15:16.343711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:29.613027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347329+0000","last_change":"2023-10-21T16:15:15.167240+0000","last_active":"2023-10-21T16:15:16.347329+0000","last_peered":"2023-10-21T16:15:16.347329+0000","last_clean":"2023-10-21T16:15:16.347329+0000","last_became_active":"2023-10-21T16:15:15.166937+0000","last_became_peered":"2023-10-21T16:15:15.166937+0000","last_unstale":"2023-10-21T16:15:16.347329+0000","last_undegraded":"2023-10-21T16:15:16.347329+0000","last_fullsized":"2023-10-21T16:15:16.347329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:50.689542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345061+0000","last_change":"2023-10-21T16:15:14.132922+0000","last_active":"2023-10-21T16:15:16.345061+0000","last_peered":"2023-10-21T16:15:16.345061+0000","last_clean":"2023-10-21T16:15:16.345061+0000","last_became_active":"2023-10-21T16:15:14.132608+0000","last_became_peered":"2023-10-21T16:15:14.132608+0000","last_unstale":"2023-10-21T16:15:16.345061+0000","last_undegraded":"2023-10-21T16:15:16.345061+0000","last_fullsized":"2023-10-21T16:15:16.345061+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:13:38.423265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"60'139","reported_seq":293,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-21T16:15:15.155282+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:15:15.155282+0000","last_peered":"2023-10-21T16:15:15.155282+0000","last_clean":"2023-10-21T16:15:15.155282+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:15:15.155282+0000","last_undegraded":"2023-10-21T16:15:15.155282+0000","last_fullsized":"2023-10-21T16:15:15.155282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1466368,"data_stored":1022275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1890000000000001,"15min":1.1279999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5600000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.641},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.2370000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.181,"15min":1.145},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.5840000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.3260000000000001,"15min":1.294},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.736,"5min":2.085,"15min":2.085},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3200000000000001,"15min":1.2689999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.619}]},{"osd":3,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2909999999999999,"15min":1.2549999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.278},{"interface":"front","average":{"1min":1.369,"5min":1.4379999999999999,"15min":1.4219999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.9570000000000001,"5min":1.9750000000000001,"15min":2.0099999999999998},"last":1.417}]},{"osd":4,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3560000000000001,"15min":1.373},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.679},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4390000000000001,"15min":1.454},"min":{"1min":0.93100000000000005,"5min":0.82599999999999996,"15min":0.70399999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.7110000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3799999999999999,"15min":1.425},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.865,"5min":1.8720000000000001,"15min":2.258},"last":1.181},{"interface":"front","average":{"1min":1.498,"5min":1.446,"15min":1.4590000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.60899999999999999},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.4770000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5509999999999999,"15min":1.5669999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.762},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.4690000000000001,"15min":1.466},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.552}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":432435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.129,"15min":1.1220000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.4590000000000001,"5min":1.806,"15min":1.806},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.2350000000000001,"15min":1.212},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.60599999999999998},"max":{"1min":1.895,"5min":1.895,"15min":2.4350000000000001},"last":1.212}]},{"osd":1,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.212,"15min":1.21},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.492},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.2909999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.3}]},{"osd":2,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.244,"15min":1.2689999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2549999999999999,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.876,"15min":0.82899999999999996},"max":{"1min":1.5449999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.6459999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.343,"15min":1.3740000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":3.46},"last":1.054},{"interface":"front","average":{"1min":1.512,"5min":1.4430000000000001,"15min":1.4339999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":4.6210000000000004,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.575}]},{"osd":4,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.319,"15min":1.375},"min":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.76300000000000001},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.351,"15min":1.413},"min":{"1min":0.96099999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":0.96199999999999997}]},{"osd":5,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.516},"min":{"1min":0.85299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.6909999999999998},"last":1.401},{"interface":"front","average":{"1min":1.425,"5min":1.3320000000000001,"15min":1.3580000000000001},"min":{"1min":0.81499999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.732}]},{"osd":7,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.85999999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.339,"5min":2.339,"15min":3.71},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.5960000000000001,"15min":1.6240000000000001},"min":{"1min":1.1779999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.7650000000000001},"last":1.776}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":432365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1379999999999999,"15min":1.111},"min":{"1min":0.76900000000000002,"5min":0.73299999999999998,"15min":0.55400000000000005},"max":{"1min":1.5629999999999999,"5min":1.919,"15min":1.919},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.244},"min":{"1min":0.91400000000000003,"5min":0.78800000000000003,"15min":0.73599999999999999},"max":{"1min":1.526,"5min":1.7849999999999999,"15min":3.6080000000000001},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2709999999999999,"15min":1.282},"min":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.69399999999999995},"max":{"1min":1.6140000000000001,"5min":1.784,"15min":2.0699999999999998},"last":1.25},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2490000000000001,"15min":1.2270000000000001},"min":{"1min":0.86699999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"max":{"1min":1.5600000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.306}]},{"osd":2,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":0.95199999999999996,"5min":0.85199999999999998,"15min":0.73399999999999999},"max":{"1min":1.649,"5min":1.7450000000000001,"15min":1.954},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.873,"5min":0.873,"15min":0.77100000000000002},"max":{"1min":1.8260000000000001,"5min":1.9059999999999999,"15min":2},"last":1.3779999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3140000000000001,"15min":1.361},"min":{"1min":1.0389999999999999,"5min":0.82699999999999996,"15min":0.80600000000000005},"max":{"1min":1.6850000000000001,"5min":2.0630000000000002,"15min":3.8109999999999999},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3779999999999999,"15min":1.389},"min":{"1min":1.0249999999999999,"5min":0.97599999999999998,"15min":0.75900000000000001},"max":{"1min":1.6419999999999999,"5min":1.952,"15min":2.0830000000000002},"last":1.722}]},{"osd":5,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.286,"15min":1.361},"min":{"1min":0.91700000000000004,"5min":0.75700000000000001,"15min":0.67700000000000005},"max":{"1min":1.7110000000000001,"5min":1.766,"15min":2.6219999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3200000000000001,"15min":1.365},"min":{"1min":0.95799999999999996,"5min":0.878,"15min":0.65700000000000003},"max":{"1min":1.774,"5min":2.1459999999999999,"15min":3.9300000000000002},"last":1.474}]},{"osd":6,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.379,"15min":1.4350000000000001},"min":{"1min":0.997,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.871,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.911},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5620000000000001,"15min":1.633},"min":{"1min":1.369,"5min":0.90100000000000002,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.0920000000000001,"15min":3.9609999999999999},"last":1.514}]},{"osd":7,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.4530000000000001,"15min":1.5429999999999999},"min":{"1min":1.1220000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8300000000000001,"5min":2.4809999999999999,"15min":4.0190000000000001},"last":1.756},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5189999999999999,"15min":1.587},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.84699999999999998},"max":{"1min":1.8540000000000001,"5min":2.0609999999999999,"15min":2.879},"last":1.8080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":431783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.117,"15min":1.105},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3140000000000001,"5min":1.6919999999999999,"15min":2.0070000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.218,"15min":1.181},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6259999999999999,"5min":1.8080000000000001,"15min":1.9530000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.2390000000000001,"15min":1.2430000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.7150000000000001,"5min":1.8919999999999999,"15min":1.944},"last":1.298},{"interface":"front","average":{"1min":1.135,"5min":1.228,"15min":1.23},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.641,"5min":1.7050000000000001,"15min":1.907},"last":0.90600000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.244,"15min":1.2929999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.659,"5min":1.9199999999999999,"15min":1.95},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.1930000000000001,"15min":1.2490000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4410000000000001,"5min":1.7,"15min":2.0249999999999999},"last":1.4410000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.873,"5min":1.9259999999999999,"15min":2.0630000000000002},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3779999999999999,"15min":1.3819999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5329999999999999,"5min":1.833,"15min":1.8959999999999999},"last":1.3640000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2150000000000001,"15min":1.2370000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7529999999999999,"5min":1.843,"15min":2.0960000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3480000000000001,"15min":1.339},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.121},"last":1.4079999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.423,"15min":1.448},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.1360000000000001},"last":1.546},{"interface":"front","average":{"1min":1.365,"5min":1.4690000000000001,"15min":1.5},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":1.5229999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.506,"15min":1.506},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.9330000000000001,"5min":2.218,"15min":2.218},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36208,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335632,"statfs":{"total":96636764160,"available":96599687168,"internally_reserved":0,"allocated":880640,"data_stored":433417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.95199999999999996,"15min":0.92800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.5580000000000001,"5min":1.746,"15min":1.746},"last":1.4419999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.222,"15min":1.1830000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":2.04,"15min":2.5289999999999999},"last":0.878}]},{"osd":1,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0309999999999999,"15min":1.0109999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.0070000000000001,"5min":2.173,"15min":2.2519999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.2649999999999999,"15min":1.264},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":3.827},"last":1.2849999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.101,"15min":1.05},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.446,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.2190000000000001,"15min":1.212},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":2.847,"5min":2.847,"15min":2.847},"last":0.94699999999999995}]},{"osd":4,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.4119999999999999,"15min":1.3939999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6950000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.4019999999999999,"15min":1.4099999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":1.083}]},{"osd":5,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.401,"15min":1.4279999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.59399999999999997},"max":{"1min":1.774,"5min":1.8759999999999999,"15min":3.7250000000000001},"last":1.347},{"interface":"front","average":{"1min":1.333,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8100000000000001,"5min":2.423,"15min":2.423},"last":1.1259999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.502,"15min":1.528},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.5229999999999999,"15min":1.5429999999999999},"min":{"1min":0.871,"5min":0.82499999999999996,"15min":0.70299999999999996},"max":{"1min":1.9259999999999999,"5min":2.2290000000000001,"15min":3.573},"last":1.4890000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.635,"15min":1.593},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.966,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.599,"15min":1.6080000000000001},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3939999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58408,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313432,"statfs":{"total":96636764160,"available":96576954368,"internally_reserved":0,"allocated":876544,"data_stored":433395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.141,"5min":1.6359999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.755}]},{"osd":1,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.095,"15min":1.131},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.907,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.042},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.25,"15min":1.254},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6919999999999999,"5min":1.7749999999999999,"15min":1.8129999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0700000000000001,"15min":1.1140000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.379},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.827,"5min":1.879,"15min":2.0390000000000001},"last":1.375}]},{"osd":4,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.3600000000000001,"15min":1.359},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.6180000000000001,"5min":1.9239999999999999,"15min":2.206},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3009999999999999,"15min":1.337},"min":{"1min":0.752,"5min":0.752,"15min":0.71099999999999997},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.492,"15min":1.5049999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.86199999999999999},"max":{"1min":1.819,"5min":1.875,"15min":2.4870000000000001},"last":1.54}]},{"osd":6,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.462,"15min":1.4530000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.698},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.389,"15min":1.446},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.552,"5min":2.552,"15min":2.6179999999999999},"last":1.1479999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.5229999999999999,"15min":1.536},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.319},{"interface":"front","average":{"1min":1.321,"5min":1.492,"15min":1.532},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.2609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335648,"statfs":{"total":96636764160,"available":96599703552,"internally_reserved":0,"allocated":872448,"data_stored":431943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.92000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3400000000000001,"5min":1.905,"15min":1.905},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.028,"15min":1.014},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6559999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.161,"15min":1.194},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.546,"5min":1.974,"15min":2.29},"last":1.0229999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.3049999999999999,"15min":1.3180000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6930000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2350000000000001,"15min":1.238},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.7989999999999999,"5min":1.877,"15min":2.3620000000000001},"last":1.4219999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.306,"15min":1.29},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.778,"5min":1.944,"15min":1.944},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9370000000000001,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.319}]},{"osd":5,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.482,"15min":1.528},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.75900000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.478,"15min":1.5029999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.357}]},{"osd":6,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.488,"15min":1.548},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.2469999999999999,"5min":2.2959999999999998,"15min":4.1260000000000003},"last":1.282},{"interface":"front","average":{"1min":1.302,"5min":1.4630000000000001,"15min":1.51},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.206,"5min":2.2090000000000001,"15min":3.101},"last":1.087}]},{"osd":7,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.6040000000000001,"15min":1.637},"min":{"1min":0.90700000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1560000000000001,"5min":2.3730000000000002,"15min":2.8180000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.6080000000000001,"15min":1.6599999999999999},"min":{"1min":0.89600000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.1619999999999999,"5min":2.2349999999999999,"15min":3.952},"last":1.121}]}]},{"osd":0,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1466368,"data_stored":1022207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":0.95099999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.36399999999999999},"max":{"1min":1.3999999999999999,"5min":1.855,"15min":2.9260000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.101,"15min":1.0620000000000001},"min":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.42799999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.171,"15min":1.1519999999999999},"min":{"1min":0.82999999999999996,"5min":0.82099999999999995,"15min":0.41799999999999998},"max":{"1min":1.548,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":0.996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.177,"15min":1.149},"min":{"1min":0.89800000000000002,"5min":0.75900000000000001,"15min":0.42999999999999999},"max":{"1min":1.46,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.091}]},{"osd":3,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.151,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.68700000000000006,"15min":0.46500000000000002},"max":{"1min":1.718,"5min":1.8740000000000001,"15min":2.1429999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.3700000000000001,"15min":1.321},"min":{"1min":0.93999999999999995,"5min":0.879,"15min":0.44900000000000001},"max":{"1min":1.758,"5min":1.972,"15min":3.052},"last":1.224}]},{"osd":4,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.415,"15min":1.4410000000000001},"min":{"1min":0.98799999999999999,"5min":0.96299999999999997,"15min":0.88900000000000001},"max":{"1min":1.6619999999999999,"5min":1.954,"15min":3.3580000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.5860000000000001,"15min":1.548},"min":{"1min":1.1679999999999999,"5min":1.042,"15min":0.92400000000000004},"max":{"1min":2.246,"5min":2.2890000000000001,"15min":3.306},"last":1.4119999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4319999999999999,"15min":1.411},"min":{"1min":1.075,"5min":1.0429999999999999,"15min":0.78000000000000003},"max":{"1min":1.845,"5min":2.137,"15min":3.8849999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5760000000000001,"15min":1.5509999999999999},"min":{"1min":1.1639999999999999,"5min":1.135,"15min":0.81000000000000005},"max":{"1min":2.137,"5min":2.4670000000000001,"15min":3.1789999999999998},"last":1.3149999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4990000000000001,"15min":1.5209999999999999},"min":{"1min":1.119,"5min":1.044,"15min":0.76800000000000002},"max":{"1min":1.9079999999999999,"5min":2.2519999999999998,"15min":4.3019999999999996},"last":1.282},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.655,"15min":1.635},"min":{"1min":1.3799999999999999,"5min":1.2050000000000001,"15min":0.97099999999999997},"max":{"1min":2.355,"5min":2.403,"15min":3.5139999999999998},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":1.0549999999999999,"5min":1.048,"15min":0.83899999999999997},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.627,"15min":1.6339999999999999},"min":{"1min":1.3149999999999999,"5min":1.131,"15min":0.82899999999999996},"max":{"1min":2.0049999999999999,"5min":2.3540000000000001,"15min":4.6219999999999999},"last":1.5289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:15:19.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:15:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.970+0000 7f0931040640 1 Processor -- start 2023-10-21T16:15:19.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.971+0000 7f0931040640 1 -- start start 2023-10-21T16:15:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.972+0000 7f0931040640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f092c15ab30 0x7f092c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.972+0000 7f0931040640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f092c0b07d0 0x7f092c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.972+0000 7f0931040640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c15f7d0 con 0x7f092c0adc70 2023-10-21T16:15:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.972+0000 7f0931040640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c15f210 con 0x7f092c15ab30 2023-10-21T16:15:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.972+0000 7f0931040640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c15f4f0 con 0x7f092c0b07d0 2023-10-21T16:15:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f092c15ab30 0x7f092c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f092c15ab30 0x7f092c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56414/0 (socket says 172.21.15.53:56414) 2023-10-21T16:15:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 -- 172.21.15.53:0/707337977 learned_addr learned my addr 172.21.15.53:0/707337977 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 -- 172.21.15.53:0/707337977 >> v2:172.21.15.124:3300/0 conn(0x7f092c0b07d0 msgr2=0x7f092c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 --2- 172.21.15.53:0/707337977 >> v2:172.21.15.124:3300/0 conn(0x7f092c0b07d0 0x7f092c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 -- 172.21.15.53:0/707337977 >> v1:172.21.15.124:6789/0 conn(0x7f092c0adc70 legacy=0x7f092c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.973+0000 7f092bfff640 1 -- 172.21.15.53:0/707337977 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c06c550 con 0x7f092c15ab30 2023-10-21T16:15:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.974+0000 7f092bfff640 1 --2- 172.21.15.53:0/707337977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c15e8a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ca4a0 tx=0x7f091c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82cfac8e158b6ec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.975+0000 7f092b7fe640 1 -- 172.21.15.53:0/707337977 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c031870 con 0x7f092c15ab30 2023-10-21T16:15:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.975+0000 7f092b7fe640 1 -- 172.21.15.53:0/707337977 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c0319d0 con 0x7f092c15ab30 2023-10-21T16:15:19.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.975+0000 7f092b7fe640 1 -- 172.21.15.53:0/707337977 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c031d60 con 0x7f092c15ab30 2023-10-21T16:15:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.975+0000 7f0931040640 1 -- 172.21.15.53:0/707337977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 msgr2=0x7f092c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.975+0000 7f0931040640 1 --2- 172.21.15.53:0/707337977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c15e8a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ca4a0 tx=0x7f091c02b910 comp rx=0 tx=0).stop 2023-10-21T16:15:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 -- 172.21.15.53:0/707337977 shutdown_connections 2023-10-21T16:15:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 --2- 172.21.15.53:0/707337977 >> v2:172.21.15.124:3300/0 conn(0x7f092c0b07d0 0x7f092c06cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 --2- 172.21.15.53:0/707337977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c15e8a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 -- 172.21.15.53:0/707337977 >> 172.21.15.53:0/707337977 conn(0x7f092c0cb0c0 msgr2=0x7f092c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 -- 172.21.15.53:0/707337977 shutdown_connections 2023-10-21T16:15:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.976+0000 7f0931040640 1 -- 172.21.15.53:0/707337977 wait complete. 2023-10-21T16:15:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.978+0000 7f0931040640 1 Processor -- start 2023-10-21T16:15:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.978+0000 7f0931040640 1 -- start start 2023-10-21T16:15:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f092c0adc70 0x7f092c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f092c0b07d0 0x7f092c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c6060 con 0x7f092c15ab30 2023-10-21T16:15:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c8570 con 0x7f092c0adc70 2023-10-21T16:15:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f0931040640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c7510 con 0x7f092c0b07d0 2023-10-21T16:15:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f093083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f093083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56424/0 (socket says 172.21.15.53:56424) 2023-10-21T16:15:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f093083f640 1 -- 172.21.15.53:0/454021226 learned_addr learned my addr 172.21.15.53:0/454021226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.979+0000 7f092bfff640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f092c0adc70 0x7f092c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f09237fe640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f092c0b07d0 0x7f092c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 -- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f092c0b07d0 msgr2=0x7f092c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f092c0b07d0 0x7f092c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 -- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f092c0adc70 msgr2=0x7f092c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f092c0adc70 0x7f092c0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 -- 172.21.15.53:0/454021226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0094d0 con 0x7f092c15ab30 2023-10-21T16:15:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f093083f640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f092400b9a0 tx=0x7f092400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400cba0 con 0x7f092c15ab30 2023-10-21T16:15:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400cd20 con 0x7f092c15ab30 2023-10-21T16:15:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924014680 con 0x7f092c15ab30 2023-10-21T16:15:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c0c77b0 con 0x7f092c15ab30 2023-10-21T16:15:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.980+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0c8050 con 0x7f092c15ab30 2023-10-21T16:15:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.982+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c0c7bb0 con 0x7f092c15ab30 2023-10-21T16:15:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.983+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f092401d3b0 con 0x7f092c15ab30 2023-10-21T16:15:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.984+0000 7f09297fa640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 0x7f090c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.984+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0924089880 con 0x7f092c15ab30 2023-10-21T16:15:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.985+0000 7f092bfff640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 0x7f090c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.986+0000 7f092bfff640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 0x7f090c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091c036000 tx=0x7f091c03c040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:19.988+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09240546b0 con 0x7f092c15ab30 2023-10-21T16:15:20.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:15:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.178+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 --> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f092c06c830 con 0x7f090c06af90 2023-10-21T16:15:20.186 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:15:20.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.201+0000 7f09297fa640 1 -- 172.21.15.53:0/454021226 <== mgr.34973 v2:172.21.15.124:6832/3288323666 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286632 (secure 0 0 0) 0x7f091c031cc0 con 0x7f090c06af90 2023-10-21T16:15:20.206 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:20.207 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:15:20.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 msgr2=0x7f090c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 0x7f090c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f091c036000 tx=0x7f091c03c040 comp rx=0 tx=0).stop 2023-10-21T16:15:20.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 msgr2=0x7f092c158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f092400b9a0 tx=0x7f092400be60 comp rx=0 tx=0).stop 2023-10-21T16:15:20.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 shutdown_connections 2023-10-21T16:15:20.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f090c06af90 0x7f090c06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f092c15ab30 0x7f092c158e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f092c0b07d0 0x7f092c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 --2- 172.21.15.53:0/454021226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f092c0adc70 0x7f092c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 >> 172.21.15.53:0/454021226 conn(0x7f092c0cb0c0 msgr2=0x7f092c0b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:20.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 shutdown_connections 2023-10-21T16:15:20.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.206+0000 7f0931040640 1 -- 172.21.15.53:0/454021226 wait complete. 2023-10-21T16:15:20.227 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-21T16:15:18.500791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318504,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754656216,"statfs":{"total":773094113280,"available":772767965184,"internally_reserved":0,"allocated":8167424,"data_stored":4639820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.408980"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346442+0000","last_change":"2023-10-21T16:15:15.158373+0000","last_active":"2023-10-21T16:15:16.346442+0000","last_peered":"2023-10-21T16:15:16.346442+0000","last_clean":"2023-10-21T16:15:16.346442+0000","last_became_active":"2023-10-21T16:15:15.158112+0000","last_became_peered":"2023-10-21T16:15:15.158112+0000","last_unstale":"2023-10-21T16:15:16.346442+0000","last_undegraded":"2023-10-21T16:15:16.346442+0000","last_fullsized":"2023-10-21T16:15:16.346442+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:06.393324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345267+0000","last_change":"2023-10-21T16:15:15.170422+0000","last_active":"2023-10-21T16:15:16.345267+0000","last_peered":"2023-10-21T16:15:16.345267+0000","last_clean":"2023-10-21T16:15:16.345267+0000","last_became_active":"2023-10-21T16:15:15.169565+0000","last_became_peered":"2023-10-21T16:15:15.169565+0000","last_unstale":"2023-10-21T16:15:16.345267+0000","last_undegraded":"2023-10-21T16:15:16.345267+0000","last_fullsized":"2023-10-21T16:15:16.345267+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:42.417849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344444+0000","last_change":"2023-10-21T16:15:15.181541+0000","last_active":"2023-10-21T16:15:16.344444+0000","last_peered":"2023-10-21T16:15:16.344444+0000","last_clean":"2023-10-21T16:15:16.344444+0000","last_became_active":"2023-10-21T16:15:15.169934+0000","last_became_peered":"2023-10-21T16:15:15.169934+0000","last_unstale":"2023-10-21T16:15:16.344444+0000","last_undegraded":"2023-10-21T16:15:16.344444+0000","last_fullsized":"2023-10-21T16:15:16.344444+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:33:44.668072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344597+0000","last_change":"2023-10-21T16:15:15.181500+0000","last_active":"2023-10-21T16:15:16.344597+0000","last_peered":"2023-10-21T16:15:16.344597+0000","last_clean":"2023-10-21T16:15:16.344597+0000","last_became_active":"2023-10-21T16:15:15.181221+0000","last_became_peered":"2023-10-21T16:15:15.181221+0000","last_unstale":"2023-10-21T16:15:16.344597+0000","last_undegraded":"2023-10-21T16:15:16.344597+0000","last_fullsized":"2023-10-21T16:15:16.344597+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:16.588613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136130+0000","last_change":"2023-10-21T16:15:15.160958+0000","last_active":"2023-10-21T16:15:16.136130+0000","last_peered":"2023-10-21T16:15:16.136130+0000","last_clean":"2023-10-21T16:15:16.136130+0000","last_became_active":"2023-10-21T16:15:15.160577+0000","last_became_peered":"2023-10-21T16:15:15.160577+0000","last_unstale":"2023-10-21T16:15:16.136130+0000","last_undegraded":"2023-10-21T16:15:16.136130+0000","last_fullsized":"2023-10-21T16:15:16.136130+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:41:29.882157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345748+0000","last_change":"2023-10-21T16:15:15.170354+0000","last_active":"2023-10-21T16:15:16.345748+0000","last_peered":"2023-10-21T16:15:16.345748+0000","last_clean":"2023-10-21T16:15:16.345748+0000","last_became_active":"2023-10-21T16:15:15.165795+0000","last_became_peered":"2023-10-21T16:15:15.165795+0000","last_unstale":"2023-10-21T16:15:16.345748+0000","last_undegraded":"2023-10-21T16:15:16.345748+0000","last_fullsized":"2023-10-21T16:15:16.345748+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:06.124253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343036+0000","last_change":"2023-10-21T16:15:15.177108+0000","last_active":"2023-10-21T16:15:16.343036+0000","last_peered":"2023-10-21T16:15:16.343036+0000","last_clean":"2023-10-21T16:15:16.343036+0000","last_became_active":"2023-10-21T16:15:15.176806+0000","last_became_peered":"2023-10-21T16:15:15.176806+0000","last_unstale":"2023-10-21T16:15:16.343036+0000","last_undegraded":"2023-10-21T16:15:16.343036+0000","last_fullsized":"2023-10-21T16:15:16.343036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:12:12.503956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131460+0000","last_change":"2023-10-21T16:15:15.173603+0000","last_active":"2023-10-21T16:15:16.131460+0000","last_peered":"2023-10-21T16:15:16.131460+0000","last_clean":"2023-10-21T16:15:16.131460+0000","last_became_active":"2023-10-21T16:15:15.168881+0000","last_became_peered":"2023-10-21T16:15:15.168881+0000","last_unstale":"2023-10-21T16:15:16.131460+0000","last_undegraded":"2023-10-21T16:15:16.131460+0000","last_fullsized":"2023-10-21T16:15:16.131460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:55:54.669363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132370+0000","last_change":"2023-10-21T16:15:15.174759+0000","last_active":"2023-10-21T16:15:16.132370+0000","last_peered":"2023-10-21T16:15:16.132370+0000","last_clean":"2023-10-21T16:15:16.132370+0000","last_became_active":"2023-10-21T16:15:15.174483+0000","last_became_peered":"2023-10-21T16:15:15.174483+0000","last_unstale":"2023-10-21T16:15:16.132370+0000","last_undegraded":"2023-10-21T16:15:16.132370+0000","last_fullsized":"2023-10-21T16:15:16.132370+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:04.064641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346449+0000","last_change":"2023-10-21T16:15:15.151130+0000","last_active":"2023-10-21T16:15:16.346449+0000","last_peered":"2023-10-21T16:15:16.346449+0000","last_clean":"2023-10-21T16:15:16.346449+0000","last_became_active":"2023-10-21T16:15:15.149600+0000","last_became_peered":"2023-10-21T16:15:15.149600+0000","last_unstale":"2023-10-21T16:15:16.346449+0000","last_undegraded":"2023-10-21T16:15:16.346449+0000","last_fullsized":"2023-10-21T16:15:16.346449+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:51:30.423650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.133118+0000","last_change":"2023-10-21T16:15:15.160767+0000","last_active":"2023-10-21T16:15:16.133118+0000","last_peered":"2023-10-21T16:15:16.133118+0000","last_clean":"2023-10-21T16:15:16.133118+0000","last_became_active":"2023-10-21T16:15:15.160402+0000","last_became_peered":"2023-10-21T16:15:15.160402+0000","last_unstale":"2023-10-21T16:15:16.133118+0000","last_undegraded":"2023-10-21T16:15:16.133118+0000","last_fullsized":"2023-10-21T16:15:16.133118+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:42:17.688309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131814+0000","last_change":"2023-10-21T16:15:15.177615+0000","last_active":"2023-10-21T16:15:16.131814+0000","last_peered":"2023-10-21T16:15:16.131814+0000","last_clean":"2023-10-21T16:15:16.131814+0000","last_became_active":"2023-10-21T16:15:15.177192+0000","last_became_peered":"2023-10-21T16:15:15.177192+0000","last_unstale":"2023-10-21T16:15:16.131814+0000","last_undegraded":"2023-10-21T16:15:16.131814+0000","last_fullsized":"2023-10-21T16:15:16.131814+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:09.858244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345520+0000","last_change":"2023-10-21T16:15:15.150507+0000","last_active":"2023-10-21T16:15:16.345520+0000","last_peered":"2023-10-21T16:15:16.345520+0000","last_clean":"2023-10-21T16:15:16.345520+0000","last_became_active":"2023-10-21T16:15:15.148274+0000","last_became_peered":"2023-10-21T16:15:15.148274+0000","last_unstale":"2023-10-21T16:15:16.345520+0000","last_undegraded":"2023-10-21T16:15:16.345520+0000","last_fullsized":"2023-10-21T16:15:16.345520+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:59.390383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343616+0000","last_change":"2023-10-21T16:15:15.175870+0000","last_active":"2023-10-21T16:15:16.343616+0000","last_peered":"2023-10-21T16:15:16.343616+0000","last_clean":"2023-10-21T16:15:16.343616+0000","last_became_active":"2023-10-21T16:15:15.175126+0000","last_became_peered":"2023-10-21T16:15:15.175126+0000","last_unstale":"2023-10-21T16:15:16.343616+0000","last_undegraded":"2023-10-21T16:15:16.343616+0000","last_fullsized":"2023-10-21T16:15:16.343616+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:36:17.800515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345866+0000","last_change":"2023-10-21T16:15:15.171486+0000","last_active":"2023-10-21T16:15:16.345866+0000","last_peered":"2023-10-21T16:15:16.345866+0000","last_clean":"2023-10-21T16:15:16.345866+0000","last_became_active":"2023-10-21T16:15:15.171131+0000","last_became_peered":"2023-10-21T16:15:15.171131+0000","last_unstale":"2023-10-21T16:15:16.345866+0000","last_undegraded":"2023-10-21T16:15:16.345866+0000","last_fullsized":"2023-10-21T16:15:16.345866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:07:56.048213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344183+0000","last_change":"2023-10-21T16:15:15.143914+0000","last_active":"2023-10-21T16:15:16.344183+0000","last_peered":"2023-10-21T16:15:16.344183+0000","last_clean":"2023-10-21T16:15:16.344183+0000","last_became_active":"2023-10-21T16:15:15.143182+0000","last_became_peered":"2023-10-21T16:15:15.143182+0000","last_unstale":"2023-10-21T16:15:16.344183+0000","last_undegraded":"2023-10-21T16:15:16.344183+0000","last_fullsized":"2023-10-21T16:15:16.344183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:57.175736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344202+0000","last_change":"2023-10-21T16:15:15.164720+0000","last_active":"2023-10-21T16:15:16.344202+0000","last_peered":"2023-10-21T16:15:16.344202+0000","last_clean":"2023-10-21T16:15:16.344202+0000","last_became_active":"2023-10-21T16:15:15.163636+0000","last_became_peered":"2023-10-21T16:15:15.163636+0000","last_unstale":"2023-10-21T16:15:16.344202+0000","last_undegraded":"2023-10-21T16:15:16.344202+0000","last_fullsized":"2023-10-21T16:15:16.344202+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:57.327377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345484+0000","last_change":"2023-10-21T16:15:15.170631+0000","last_active":"2023-10-21T16:15:16.345484+0000","last_peered":"2023-10-21T16:15:16.345484+0000","last_clean":"2023-10-21T16:15:16.345484+0000","last_became_active":"2023-10-21T16:15:15.170093+0000","last_became_peered":"2023-10-21T16:15:15.170093+0000","last_unstale":"2023-10-21T16:15:16.345484+0000","last_undegraded":"2023-10-21T16:15:16.345484+0000","last_fullsized":"2023-10-21T16:15:16.345484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:55:49.716899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344632+0000","last_change":"2023-10-21T16:15:15.182099+0000","last_active":"2023-10-21T16:15:16.344632+0000","last_peered":"2023-10-21T16:15:16.344632+0000","last_clean":"2023-10-21T16:15:16.344632+0000","last_became_active":"2023-10-21T16:15:15.181813+0000","last_became_peered":"2023-10-21T16:15:15.181813+0000","last_unstale":"2023-10-21T16:15:16.344632+0000","last_undegraded":"2023-10-21T16:15:16.344632+0000","last_fullsized":"2023-10-21T16:15:16.344632+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:42.983243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347407+0000","last_change":"2023-10-21T16:15:15.168380+0000","last_active":"2023-10-21T16:15:16.347407+0000","last_peered":"2023-10-21T16:15:16.347407+0000","last_clean":"2023-10-21T16:15:16.347407+0000","last_became_active":"2023-10-21T16:15:15.168031+0000","last_became_peered":"2023-10-21T16:15:15.168031+0000","last_unstale":"2023-10-21T16:15:16.347407+0000","last_undegraded":"2023-10-21T16:15:16.347407+0000","last_fullsized":"2023-10-21T16:15:16.347407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:59:04.723830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347460+0000","last_change":"2023-10-21T16:15:15.167861+0000","last_active":"2023-10-21T16:15:16.347460+0000","last_peered":"2023-10-21T16:15:16.347460+0000","last_clean":"2023-10-21T16:15:16.347460+0000","last_became_active":"2023-10-21T16:15:15.166851+0000","last_became_peered":"2023-10-21T16:15:15.166851+0000","last_unstale":"2023-10-21T16:15:16.347460+0000","last_undegraded":"2023-10-21T16:15:16.347460+0000","last_fullsized":"2023-10-21T16:15:16.347460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:23:21.903410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131634+0000","last_change":"2023-10-21T16:15:15.187260+0000","last_active":"2023-10-21T16:15:16.131634+0000","last_peered":"2023-10-21T16:15:16.131634+0000","last_clean":"2023-10-21T16:15:16.131634+0000","last_became_active":"2023-10-21T16:15:15.186105+0000","last_became_peered":"2023-10-21T16:15:15.186105+0000","last_unstale":"2023-10-21T16:15:16.131634+0000","last_undegraded":"2023-10-21T16:15:16.131634+0000","last_fullsized":"2023-10-21T16:15:16.131634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:15:40.626855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132803+0000","last_change":"2023-10-21T16:15:15.157999+0000","last_active":"2023-10-21T16:15:16.132803+0000","last_peered":"2023-10-21T16:15:16.132803+0000","last_clean":"2023-10-21T16:15:16.132803+0000","last_became_active":"2023-10-21T16:15:15.157648+0000","last_became_peered":"2023-10-21T16:15:15.157648+0000","last_unstale":"2023-10-21T16:15:16.132803+0000","last_undegraded":"2023-10-21T16:15:16.132803+0000","last_fullsized":"2023-10-21T16:15:16.132803+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:25.746647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343251+0000","last_change":"2023-10-21T16:15:15.167680+0000","last_active":"2023-10-21T16:15:16.343251+0000","last_peered":"2023-10-21T16:15:16.343251+0000","last_clean":"2023-10-21T16:15:16.343251+0000","last_became_active":"2023-10-21T16:15:15.167409+0000","last_became_peered":"2023-10-21T16:15:15.167409+0000","last_unstale":"2023-10-21T16:15:16.343251+0000","last_undegraded":"2023-10-21T16:15:16.343251+0000","last_fullsized":"2023-10-21T16:15:16.343251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:37:45.487017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344658+0000","last_change":"2023-10-21T16:15:15.179111+0000","last_active":"2023-10-21T16:15:16.344658+0000","last_peered":"2023-10-21T16:15:16.344658+0000","last_clean":"2023-10-21T16:15:16.344658+0000","last_became_active":"2023-10-21T16:15:15.178134+0000","last_became_peered":"2023-10-21T16:15:15.178134+0000","last_unstale":"2023-10-21T16:15:16.344658+0000","last_undegraded":"2023-10-21T16:15:16.344658+0000","last_fullsized":"2023-10-21T16:15:16.344658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:23.300910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132361+0000","last_change":"2023-10-21T16:15:15.169578+0000","last_active":"2023-10-21T16:15:16.132361+0000","last_peered":"2023-10-21T16:15:16.132361+0000","last_clean":"2023-10-21T16:15:16.132361+0000","last_became_active":"2023-10-21T16:15:15.168927+0000","last_became_peered":"2023-10-21T16:15:15.168927+0000","last_unstale":"2023-10-21T16:15:16.132361+0000","last_undegraded":"2023-10-21T16:15:16.132361+0000","last_fullsized":"2023-10-21T16:15:16.132361+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:12:18.158375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346274+0000","last_change":"2023-10-21T16:15:15.159089+0000","last_active":"2023-10-21T16:15:16.346274+0000","last_peered":"2023-10-21T16:15:16.346274+0000","last_clean":"2023-10-21T16:15:16.346274+0000","last_became_active":"2023-10-21T16:15:15.158826+0000","last_became_peered":"2023-10-21T16:15:15.158826+0000","last_unstale":"2023-10-21T16:15:16.346274+0000","last_undegraded":"2023-10-21T16:15:16.346274+0000","last_fullsized":"2023-10-21T16:15:16.346274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:31.769825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347768+0000","last_change":"2023-10-21T16:15:15.164652+0000","last_active":"2023-10-21T16:15:16.347768+0000","last_peered":"2023-10-21T16:15:16.347768+0000","last_clean":"2023-10-21T16:15:16.347768+0000","last_became_active":"2023-10-21T16:15:15.164313+0000","last_became_peered":"2023-10-21T16:15:15.164313+0000","last_unstale":"2023-10-21T16:15:16.347768+0000","last_undegraded":"2023-10-21T16:15:16.347768+0000","last_fullsized":"2023-10-21T16:15:16.347768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:58.100651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343936+0000","last_change":"2023-10-21T16:15:15.169012+0000","last_active":"2023-10-21T16:15:16.343936+0000","last_peered":"2023-10-21T16:15:16.343936+0000","last_clean":"2023-10-21T16:15:16.343936+0000","last_became_active":"2023-10-21T16:15:15.168675+0000","last_became_peered":"2023-10-21T16:15:15.168675+0000","last_unstale":"2023-10-21T16:15:16.343936+0000","last_undegraded":"2023-10-21T16:15:16.343936+0000","last_fullsized":"2023-10-21T16:15:16.343936+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:00:09.665089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344937+0000","last_change":"2023-10-21T16:15:15.169421+0000","last_active":"2023-10-21T16:15:16.344937+0000","last_peered":"2023-10-21T16:15:16.344937+0000","last_clean":"2023-10-21T16:15:16.344937+0000","last_became_active":"2023-10-21T16:15:15.169114+0000","last_became_peered":"2023-10-21T16:15:15.169114+0000","last_unstale":"2023-10-21T16:15:16.344937+0000","last_undegraded":"2023-10-21T16:15:16.344937+0000","last_fullsized":"2023-10-21T16:15:16.344937+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:44.974084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343227+0000","last_change":"2023-10-21T16:15:15.166358+0000","last_active":"2023-10-21T16:15:16.343227+0000","last_peered":"2023-10-21T16:15:16.343227+0000","last_clean":"2023-10-21T16:15:16.343227+0000","last_became_active":"2023-10-21T16:15:15.166100+0000","last_became_peered":"2023-10-21T16:15:15.166100+0000","last_unstale":"2023-10-21T16:15:16.343227+0000","last_undegraded":"2023-10-21T16:15:16.343227+0000","last_fullsized":"2023-10-21T16:15:16.343227+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:50.332580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343847+0000","last_change":"2023-10-21T16:15:15.164650+0000","last_active":"2023-10-21T16:15:16.343847+0000","last_peered":"2023-10-21T16:15:16.343847+0000","last_clean":"2023-10-21T16:15:16.343847+0000","last_became_active":"2023-10-21T16:15:15.160824+0000","last_became_peered":"2023-10-21T16:15:15.160824+0000","last_unstale":"2023-10-21T16:15:16.343847+0000","last_undegraded":"2023-10-21T16:15:16.343847+0000","last_fullsized":"2023-10-21T16:15:16.343847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:41.282186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345125+0000","last_change":"2023-10-21T16:15:15.169834+0000","last_active":"2023-10-21T16:15:16.345125+0000","last_peered":"2023-10-21T16:15:16.345125+0000","last_clean":"2023-10-21T16:15:16.345125+0000","last_became_active":"2023-10-21T16:15:15.169506+0000","last_became_peered":"2023-10-21T16:15:15.169506+0000","last_unstale":"2023-10-21T16:15:16.345125+0000","last_undegraded":"2023-10-21T16:15:16.345125+0000","last_fullsized":"2023-10-21T16:15:16.345125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:40.166519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"71'4","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347552+0000","last_change":"2023-10-21T16:15:14.129206+0000","last_active":"2023-10-21T16:15:16.347552+0000","last_peered":"2023-10-21T16:15:16.347552+0000","last_clean":"2023-10-21T16:15:16.347552+0000","last_became_active":"2023-10-21T16:15:14.128887+0000","last_became_peered":"2023-10-21T16:15:14.128887+0000","last_unstale":"2023-10-21T16:15:16.347552+0000","last_undegraded":"2023-10-21T16:15:16.347552+0000","last_fullsized":"2023-10-21T16:15:16.347552+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:23.192369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343889+0000","last_change":"2023-10-21T16:15:15.162035+0000","last_active":"2023-10-21T16:15:16.343889+0000","last_peered":"2023-10-21T16:15:16.343889+0000","last_clean":"2023-10-21T16:15:16.343889+0000","last_became_active":"2023-10-21T16:15:15.161621+0000","last_became_peered":"2023-10-21T16:15:15.161621+0000","last_unstale":"2023-10-21T16:15:16.343889+0000","last_undegraded":"2023-10-21T16:15:16.343889+0000","last_fullsized":"2023-10-21T16:15:16.343889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:09.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346709+0000","last_change":"2023-10-21T16:15:14.143580+0000","last_active":"2023-10-21T16:15:16.346709+0000","last_peered":"2023-10-21T16:15:16.346709+0000","last_clean":"2023-10-21T16:15:16.346709+0000","last_became_active":"2023-10-21T16:15:14.142794+0000","last_became_peered":"2023-10-21T16:15:14.142794+0000","last_unstale":"2023-10-21T16:15:16.346709+0000","last_undegraded":"2023-10-21T16:15:16.346709+0000","last_fullsized":"2023-10-21T16:15:16.346709+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:39.403925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347130+0000","last_change":"2023-10-21T16:15:15.165643+0000","last_active":"2023-10-21T16:15:16.347130+0000","last_peered":"2023-10-21T16:15:16.347130+0000","last_clean":"2023-10-21T16:15:16.347130+0000","last_became_active":"2023-10-21T16:15:15.164402+0000","last_became_peered":"2023-10-21T16:15:15.164402+0000","last_unstale":"2023-10-21T16:15:16.347130+0000","last_undegraded":"2023-10-21T16:15:16.347130+0000","last_fullsized":"2023-10-21T16:15:16.347130+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:45:48.319841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346196+0000","last_change":"2023-10-21T16:15:14.138562+0000","last_active":"2023-10-21T16:15:16.346196+0000","last_peered":"2023-10-21T16:15:16.346196+0000","last_clean":"2023-10-21T16:15:16.346196+0000","last_became_active":"2023-10-21T16:15:14.137280+0000","last_became_peered":"2023-10-21T16:15:14.137280+0000","last_unstale":"2023-10-21T16:15:16.346196+0000","last_undegraded":"2023-10-21T16:15:16.346196+0000","last_fullsized":"2023-10-21T16:15:16.346196+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:47:32.188866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131608+0000","last_change":"2023-10-21T16:15:15.159423+0000","last_active":"2023-10-21T16:15:16.131608+0000","last_peered":"2023-10-21T16:15:16.131608+0000","last_clean":"2023-10-21T16:15:16.131608+0000","last_became_active":"2023-10-21T16:15:15.159131+0000","last_became_peered":"2023-10-21T16:15:15.159131+0000","last_unstale":"2023-10-21T16:15:16.131608+0000","last_undegraded":"2023-10-21T16:15:16.131608+0000","last_fullsized":"2023-10-21T16:15:16.131608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:43.781262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344071+0000","last_change":"2023-10-21T16:15:14.132213+0000","last_active":"2023-10-21T16:15:16.344071+0000","last_peered":"2023-10-21T16:15:16.344071+0000","last_clean":"2023-10-21T16:15:16.344071+0000","last_became_active":"2023-10-21T16:15:14.131899+0000","last_became_peered":"2023-10-21T16:15:14.131899+0000","last_unstale":"2023-10-21T16:15:16.344071+0000","last_undegraded":"2023-10-21T16:15:16.344071+0000","last_fullsized":"2023-10-21T16:15:16.344071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:14:48.327976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136103+0000","last_change":"2023-10-21T16:15:15.175489+0000","last_active":"2023-10-21T16:15:16.136103+0000","last_peered":"2023-10-21T16:15:16.136103+0000","last_clean":"2023-10-21T16:15:16.136103+0000","last_became_active":"2023-10-21T16:15:15.175120+0000","last_became_peered":"2023-10-21T16:15:15.175120+0000","last_unstale":"2023-10-21T16:15:16.136103+0000","last_undegraded":"2023-10-21T16:15:16.136103+0000","last_fullsized":"2023-10-21T16:15:16.136103+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:34:15.676088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344075+0000","last_change":"2023-10-21T16:15:14.132492+0000","last_active":"2023-10-21T16:15:16.344075+0000","last_peered":"2023-10-21T16:15:16.344075+0000","last_clean":"2023-10-21T16:15:16.344075+0000","last_became_active":"2023-10-21T16:15:14.132024+0000","last_became_peered":"2023-10-21T16:15:14.132024+0000","last_unstale":"2023-10-21T16:15:16.344075+0000","last_undegraded":"2023-10-21T16:15:16.344075+0000","last_fullsized":"2023-10-21T16:15:16.344075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:42:22.360100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344965+0000","last_change":"2023-10-21T16:15:15.183915+0000","last_active":"2023-10-21T16:15:16.344965+0000","last_peered":"2023-10-21T16:15:16.344965+0000","last_clean":"2023-10-21T16:15:16.344965+0000","last_became_active":"2023-10-21T16:15:15.183662+0000","last_became_peered":"2023-10-21T16:15:15.183662+0000","last_unstale":"2023-10-21T16:15:16.344965+0000","last_undegraded":"2023-10-21T16:15:16.344965+0000","last_fullsized":"2023-10-21T16:15:16.344965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:01.951186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135404+0000","last_change":"2023-10-21T16:15:14.127769+0000","last_active":"2023-10-21T16:15:16.135404+0000","last_peered":"2023-10-21T16:15:16.135404+0000","last_clean":"2023-10-21T16:15:16.135404+0000","last_became_active":"2023-10-21T16:15:14.127287+0000","last_became_peered":"2023-10-21T16:15:14.127287+0000","last_unstale":"2023-10-21T16:15:16.135404+0000","last_undegraded":"2023-10-21T16:15:16.135404+0000","last_fullsized":"2023-10-21T16:15:16.135404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:42.412349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132620+0000","last_change":"2023-10-21T16:15:15.179343+0000","last_active":"2023-10-21T16:15:16.132620+0000","last_peered":"2023-10-21T16:15:16.132620+0000","last_clean":"2023-10-21T16:15:16.132620+0000","last_became_active":"2023-10-21T16:15:15.169289+0000","last_became_peered":"2023-10-21T16:15:15.169289+0000","last_unstale":"2023-10-21T16:15:16.132620+0000","last_undegraded":"2023-10-21T16:15:16.132620+0000","last_fullsized":"2023-10-21T16:15:16.132620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:01.484334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346962+0000","last_change":"2023-10-21T16:15:14.130970+0000","last_active":"2023-10-21T16:15:16.346962+0000","last_peered":"2023-10-21T16:15:16.346962+0000","last_clean":"2023-10-21T16:15:16.346962+0000","last_became_active":"2023-10-21T16:15:14.130311+0000","last_became_peered":"2023-10-21T16:15:14.130311+0000","last_unstale":"2023-10-21T16:15:16.346962+0000","last_undegraded":"2023-10-21T16:15:16.346962+0000","last_fullsized":"2023-10-21T16:15:16.346962+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:27.551565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344387+0000","last_change":"2023-10-21T16:15:15.178413+0000","last_active":"2023-10-21T16:15:16.344387+0000","last_peered":"2023-10-21T16:15:16.344387+0000","last_clean":"2023-10-21T16:15:16.344387+0000","last_became_active":"2023-10-21T16:15:15.178134+0000","last_became_peered":"2023-10-21T16:15:15.178134+0000","last_unstale":"2023-10-21T16:15:16.344387+0000","last_undegraded":"2023-10-21T16:15:16.344387+0000","last_fullsized":"2023-10-21T16:15:16.344387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:30.519586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343060+0000","last_change":"2023-10-21T16:15:14.126182+0000","last_active":"2023-10-21T16:15:16.343060+0000","last_peered":"2023-10-21T16:15:16.343060+0000","last_clean":"2023-10-21T16:15:16.343060+0000","last_became_active":"2023-10-21T16:15:14.125913+0000","last_became_peered":"2023-10-21T16:15:14.125913+0000","last_unstale":"2023-10-21T16:15:16.343060+0000","last_undegraded":"2023-10-21T16:15:16.343060+0000","last_fullsized":"2023-10-21T16:15:16.343060+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:20.773548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135329+0000","last_change":"2023-10-21T16:15:15.162339+0000","last_active":"2023-10-21T16:15:16.135329+0000","last_peered":"2023-10-21T16:15:16.135329+0000","last_clean":"2023-10-21T16:15:16.135329+0000","last_became_active":"2023-10-21T16:15:15.161520+0000","last_became_peered":"2023-10-21T16:15:15.161520+0000","last_unstale":"2023-10-21T16:15:16.135329+0000","last_undegraded":"2023-10-21T16:15:16.135329+0000","last_fullsized":"2023-10-21T16:15:16.135329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:14.919605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344661+0000","last_change":"2023-10-21T16:15:14.130505+0000","last_active":"2023-10-21T16:15:16.344661+0000","last_peered":"2023-10-21T16:15:16.344661+0000","last_clean":"2023-10-21T16:15:16.344661+0000","last_became_active":"2023-10-21T16:15:14.130218+0000","last_became_peered":"2023-10-21T16:15:14.130218+0000","last_unstale":"2023-10-21T16:15:16.344661+0000","last_undegraded":"2023-10-21T16:15:16.344661+0000","last_fullsized":"2023-10-21T16:15:16.344661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:26:04.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343504+0000","last_change":"2023-10-21T16:15:15.161969+0000","last_active":"2023-10-21T16:15:16.343504+0000","last_peered":"2023-10-21T16:15:16.343504+0000","last_clean":"2023-10-21T16:15:16.343504+0000","last_became_active":"2023-10-21T16:15:15.157990+0000","last_became_peered":"2023-10-21T16:15:15.157990+0000","last_unstale":"2023-10-21T16:15:16.343504+0000","last_undegraded":"2023-10-21T16:15:16.343504+0000","last_fullsized":"2023-10-21T16:15:16.343504+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:02:18.621279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136313+0000","last_change":"2023-10-21T16:15:14.137687+0000","last_active":"2023-10-21T16:15:16.136313+0000","last_peered":"2023-10-21T16:15:16.136313+0000","last_clean":"2023-10-21T16:15:16.136313+0000","last_became_active":"2023-10-21T16:15:14.137336+0000","last_became_peered":"2023-10-21T16:15:14.137336+0000","last_unstale":"2023-10-21T16:15:16.136313+0000","last_undegraded":"2023-10-21T16:15:16.136313+0000","last_fullsized":"2023-10-21T16:15:16.136313+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:33.007836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343861+0000","last_change":"2023-10-21T16:15:15.176131+0000","last_active":"2023-10-21T16:15:16.343861+0000","last_peered":"2023-10-21T16:15:16.343861+0000","last_clean":"2023-10-21T16:15:16.343861+0000","last_became_active":"2023-10-21T16:15:15.175480+0000","last_became_peered":"2023-10-21T16:15:15.175480+0000","last_unstale":"2023-10-21T16:15:16.343861+0000","last_undegraded":"2023-10-21T16:15:16.343861+0000","last_fullsized":"2023-10-21T16:15:16.343861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:54.706460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345416+0000","last_change":"2023-10-21T16:15:14.140547+0000","last_active":"2023-10-21T16:15:16.345416+0000","last_peered":"2023-10-21T16:15:16.345416+0000","last_clean":"2023-10-21T16:15:16.345416+0000","last_became_active":"2023-10-21T16:15:14.140055+0000","last_became_peered":"2023-10-21T16:15:14.140055+0000","last_unstale":"2023-10-21T16:15:16.345416+0000","last_undegraded":"2023-10-21T16:15:16.345416+0000","last_fullsized":"2023-10-21T16:15:16.345416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:22.286324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.136421+0000","last_change":"2023-10-21T16:15:15.152639+0000","last_active":"2023-10-21T16:15:16.136421+0000","last_peered":"2023-10-21T16:15:16.136421+0000","last_clean":"2023-10-21T16:15:16.136421+0000","last_became_active":"2023-10-21T16:15:15.151821+0000","last_became_peered":"2023-10-21T16:15:15.151821+0000","last_unstale":"2023-10-21T16:15:16.136421+0000","last_undegraded":"2023-10-21T16:15:16.136421+0000","last_fullsized":"2023-10-21T16:15:16.136421+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:40.759854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"71'6","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131973+0000","last_change":"2023-10-21T16:15:14.130084+0000","last_active":"2023-10-21T16:15:16.131973+0000","last_peered":"2023-10-21T16:15:16.131973+0000","last_clean":"2023-10-21T16:15:16.131973+0000","last_became_active":"2023-10-21T16:15:14.129788+0000","last_became_peered":"2023-10-21T16:15:14.129788+0000","last_unstale":"2023-10-21T16:15:16.131973+0000","last_undegraded":"2023-10-21T16:15:16.131973+0000","last_fullsized":"2023-10-21T16:15:16.131973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:37.224728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343452+0000","last_change":"2023-10-21T16:15:15.161670+0000","last_active":"2023-10-21T16:15:16.343452+0000","last_peered":"2023-10-21T16:15:16.343452+0000","last_clean":"2023-10-21T16:15:16.343452+0000","last_became_active":"2023-10-21T16:15:15.161344+0000","last_became_peered":"2023-10-21T16:15:15.161344+0000","last_unstale":"2023-10-21T16:15:16.343452+0000","last_undegraded":"2023-10-21T16:15:16.343452+0000","last_fullsized":"2023-10-21T16:15:16.343452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:01.206148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344217+0000","last_change":"2023-10-21T16:15:14.135807+0000","last_active":"2023-10-21T16:15:16.344217+0000","last_peered":"2023-10-21T16:15:16.344217+0000","last_clean":"2023-10-21T16:15:16.344217+0000","last_became_active":"2023-10-21T16:15:14.135403+0000","last_became_peered":"2023-10-21T16:15:14.135403+0000","last_unstale":"2023-10-21T16:15:16.344217+0000","last_undegraded":"2023-10-21T16:15:16.344217+0000","last_fullsized":"2023-10-21T16:15:16.344217+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:45.579353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344700+0000","last_change":"2023-10-21T16:15:15.185854+0000","last_active":"2023-10-21T16:15:16.344700+0000","last_peered":"2023-10-21T16:15:16.344700+0000","last_clean":"2023-10-21T16:15:16.344700+0000","last_became_active":"2023-10-21T16:15:15.184136+0000","last_became_peered":"2023-10-21T16:15:15.184136+0000","last_unstale":"2023-10-21T16:15:16.344700+0000","last_undegraded":"2023-10-21T16:15:16.344700+0000","last_fullsized":"2023-10-21T16:15:16.344700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:51.122703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347293+0000","last_change":"2023-10-21T16:15:14.136973+0000","last_active":"2023-10-21T16:15:16.347293+0000","last_peered":"2023-10-21T16:15:16.347293+0000","last_clean":"2023-10-21T16:15:16.347293+0000","last_became_active":"2023-10-21T16:15:14.136699+0000","last_became_peered":"2023-10-21T16:15:14.136699+0000","last_unstale":"2023-10-21T16:15:16.347293+0000","last_undegraded":"2023-10-21T16:15:16.347293+0000","last_fullsized":"2023-10-21T16:15:16.347293+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:07:04.467622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132365+0000","last_change":"2023-10-21T16:15:15.163276+0000","last_active":"2023-10-21T16:15:16.132365+0000","last_peered":"2023-10-21T16:15:16.132365+0000","last_clean":"2023-10-21T16:15:16.132365+0000","last_became_active":"2023-10-21T16:15:15.162999+0000","last_became_peered":"2023-10-21T16:15:15.162999+0000","last_unstale":"2023-10-21T16:15:16.132365+0000","last_undegraded":"2023-10-21T16:15:16.132365+0000","last_fullsized":"2023-10-21T16:15:16.132365+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:30:51.816899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346009+0000","last_change":"2023-10-21T16:15:14.130710+0000","last_active":"2023-10-21T16:15:16.346009+0000","last_peered":"2023-10-21T16:15:16.346009+0000","last_clean":"2023-10-21T16:15:16.346009+0000","last_became_active":"2023-10-21T16:15:14.129585+0000","last_became_peered":"2023-10-21T16:15:14.129585+0000","last_unstale":"2023-10-21T16:15:16.346009+0000","last_undegraded":"2023-10-21T16:15:16.346009+0000","last_fullsized":"2023-10-21T16:15:16.346009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:56.312791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344351+0000","last_change":"2023-10-21T16:15:15.163981+0000","last_active":"2023-10-21T16:15:16.344351+0000","last_peered":"2023-10-21T16:15:16.344351+0000","last_clean":"2023-10-21T16:15:16.344351+0000","last_became_active":"2023-10-21T16:15:15.161362+0000","last_became_peered":"2023-10-21T16:15:15.161362+0000","last_unstale":"2023-10-21T16:15:16.344351+0000","last_undegraded":"2023-10-21T16:15:16.344351+0000","last_fullsized":"2023-10-21T16:15:16.344351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:24.477453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131848+0000","last_change":"2023-10-21T16:15:14.141613+0000","last_active":"2023-10-21T16:15:16.131848+0000","last_peered":"2023-10-21T16:15:16.131848+0000","last_clean":"2023-10-21T16:15:16.131848+0000","last_became_active":"2023-10-21T16:15:14.140539+0000","last_became_peered":"2023-10-21T16:15:14.140539+0000","last_unstale":"2023-10-21T16:15:16.131848+0000","last_undegraded":"2023-10-21T16:15:16.131848+0000","last_fullsized":"2023-10-21T16:15:16.131848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:02:17.606693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131716+0000","last_change":"2023-10-21T16:15:15.159705+0000","last_active":"2023-10-21T16:15:16.131716+0000","last_peered":"2023-10-21T16:15:16.131716+0000","last_clean":"2023-10-21T16:15:16.131716+0000","last_became_active":"2023-10-21T16:15:15.159410+0000","last_became_peered":"2023-10-21T16:15:15.159410+0000","last_unstale":"2023-10-21T16:15:16.131716+0000","last_undegraded":"2023-10-21T16:15:16.131716+0000","last_fullsized":"2023-10-21T16:15:16.131716+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:31:42.531013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344450+0000","last_change":"2023-10-21T16:15:14.138208+0000","last_active":"2023-10-21T16:15:16.344450+0000","last_peered":"2023-10-21T16:15:16.344450+0000","last_clean":"2023-10-21T16:15:16.344450+0000","last_became_active":"2023-10-21T16:15:14.137474+0000","last_became_peered":"2023-10-21T16:15:14.137474+0000","last_unstale":"2023-10-21T16:15:16.344450+0000","last_undegraded":"2023-10-21T16:15:16.344450+0000","last_fullsized":"2023-10-21T16:15:16.344450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:39.856113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345219+0000","last_change":"2023-10-21T16:15:15.167813+0000","last_active":"2023-10-21T16:15:16.345219+0000","last_peered":"2023-10-21T16:15:16.345219+0000","last_clean":"2023-10-21T16:15:16.345219+0000","last_became_active":"2023-10-21T16:15:15.166307+0000","last_became_peered":"2023-10-21T16:15:15.166307+0000","last_unstale":"2023-10-21T16:15:16.345219+0000","last_undegraded":"2023-10-21T16:15:16.345219+0000","last_fullsized":"2023-10-21T16:15:16.345219+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:53:47.031790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.133070+0000","last_change":"2023-10-21T16:15:14.110821+0000","last_active":"2023-10-21T16:15:16.133070+0000","last_peered":"2023-10-21T16:15:16.133070+0000","last_clean":"2023-10-21T16:15:16.133070+0000","last_became_active":"2023-10-21T16:15:14.109764+0000","last_became_peered":"2023-10-21T16:15:14.109764+0000","last_unstale":"2023-10-21T16:15:16.133070+0000","last_undegraded":"2023-10-21T16:15:16.133070+0000","last_fullsized":"2023-10-21T16:15:16.133070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:33:21.253930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135432+0000","last_change":"2023-10-21T16:15:15.175816+0000","last_active":"2023-10-21T16:15:16.135432+0000","last_peered":"2023-10-21T16:15:16.135432+0000","last_clean":"2023-10-21T16:15:16.135432+0000","last_became_active":"2023-10-21T16:15:15.175562+0000","last_became_peered":"2023-10-21T16:15:15.175562+0000","last_unstale":"2023-10-21T16:15:16.135432+0000","last_undegraded":"2023-10-21T16:15:16.135432+0000","last_fullsized":"2023-10-21T16:15:16.135432+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:34.819605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347822+0000","last_change":"2023-10-21T16:15:14.140748+0000","last_active":"2023-10-21T16:15:16.347822+0000","last_peered":"2023-10-21T16:15:16.347822+0000","last_clean":"2023-10-21T16:15:16.347822+0000","last_became_active":"2023-10-21T16:15:14.140297+0000","last_became_peered":"2023-10-21T16:15:14.140297+0000","last_unstale":"2023-10-21T16:15:16.347822+0000","last_undegraded":"2023-10-21T16:15:16.347822+0000","last_fullsized":"2023-10-21T16:15:16.347822+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:55.387335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345774+0000","last_change":"2023-10-21T16:15:15.148751+0000","last_active":"2023-10-21T16:15:16.345774+0000","last_peered":"2023-10-21T16:15:16.345774+0000","last_clean":"2023-10-21T16:15:16.345774+0000","last_became_active":"2023-10-21T16:15:15.147602+0000","last_became_peered":"2023-10-21T16:15:15.147602+0000","last_unstale":"2023-10-21T16:15:16.345774+0000","last_undegraded":"2023-10-21T16:15:16.345774+0000","last_fullsized":"2023-10-21T16:15:16.345774+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:54:58.214345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347862+0000","last_change":"2023-10-21T16:15:14.138362+0000","last_active":"2023-10-21T16:15:16.347862+0000","last_peered":"2023-10-21T16:15:16.347862+0000","last_clean":"2023-10-21T16:15:16.347862+0000","last_became_active":"2023-10-21T16:15:14.138054+0000","last_became_peered":"2023-10-21T16:15:14.138054+0000","last_unstale":"2023-10-21T16:15:16.347862+0000","last_undegraded":"2023-10-21T16:15:16.347862+0000","last_fullsized":"2023-10-21T16:15:16.347862+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:08:03.060972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343822+0000","last_change":"2023-10-21T16:15:15.164297+0000","last_active":"2023-10-21T16:15:16.343822+0000","last_peered":"2023-10-21T16:15:16.343822+0000","last_clean":"2023-10-21T16:15:16.343822+0000","last_became_active":"2023-10-21T16:15:15.163948+0000","last_became_peered":"2023-10-21T16:15:15.163948+0000","last_unstale":"2023-10-21T16:15:16.343822+0000","last_undegraded":"2023-10-21T16:15:16.343822+0000","last_fullsized":"2023-10-21T16:15:16.343822+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:38.239314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345738+0000","last_change":"2023-10-21T16:15:14.135993+0000","last_active":"2023-10-21T16:15:16.345738+0000","last_peered":"2023-10-21T16:15:16.345738+0000","last_clean":"2023-10-21T16:15:16.345738+0000","last_became_active":"2023-10-21T16:15:14.135657+0000","last_became_peered":"2023-10-21T16:15:14.135657+0000","last_unstale":"2023-10-21T16:15:16.345738+0000","last_undegraded":"2023-10-21T16:15:16.345738+0000","last_fullsized":"2023-10-21T16:15:16.345738+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:59:03.709244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347666+0000","last_change":"2023-10-21T16:15:15.169186+0000","last_active":"2023-10-21T16:15:16.347666+0000","last_peered":"2023-10-21T16:15:16.347666+0000","last_clean":"2023-10-21T16:15:16.347666+0000","last_became_active":"2023-10-21T16:15:15.168886+0000","last_became_peered":"2023-10-21T16:15:15.168886+0000","last_unstale":"2023-10-21T16:15:16.347666+0000","last_undegraded":"2023-10-21T16:15:16.347666+0000","last_fullsized":"2023-10-21T16:15:16.347666+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:40.418511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131613+0000","last_change":"2023-10-21T16:15:14.129573+0000","last_active":"2023-10-21T16:15:16.131613+0000","last_peered":"2023-10-21T16:15:16.131613+0000","last_clean":"2023-10-21T16:15:16.131613+0000","last_became_active":"2023-10-21T16:15:14.129205+0000","last_became_peered":"2023-10-21T16:15:14.129205+0000","last_unstale":"2023-10-21T16:15:16.131613+0000","last_undegraded":"2023-10-21T16:15:16.131613+0000","last_fullsized":"2023-10-21T16:15:16.131613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:08.395341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344983+0000","last_change":"2023-10-21T16:15:15.171139+0000","last_active":"2023-10-21T16:15:16.344983+0000","last_peered":"2023-10-21T16:15:16.344983+0000","last_clean":"2023-10-21T16:15:16.344983+0000","last_became_active":"2023-10-21T16:15:15.170847+0000","last_became_peered":"2023-10-21T16:15:15.170847+0000","last_unstale":"2023-10-21T16:15:16.344983+0000","last_undegraded":"2023-10-21T16:15:16.344983+0000","last_fullsized":"2023-10-21T16:15:16.344983+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:26:05.210587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343259+0000","last_change":"2023-10-21T16:15:14.140383+0000","last_active":"2023-10-21T16:15:16.343259+0000","last_peered":"2023-10-21T16:15:16.343259+0000","last_clean":"2023-10-21T16:15:16.343259+0000","last_became_active":"2023-10-21T16:15:14.140051+0000","last_became_peered":"2023-10-21T16:15:14.140051+0000","last_unstale":"2023-10-21T16:15:16.343259+0000","last_undegraded":"2023-10-21T16:15:16.343259+0000","last_fullsized":"2023-10-21T16:15:16.343259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:00.586161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.346582+0000","last_change":"2023-10-21T16:15:15.177152+0000","last_active":"2023-10-21T16:15:16.346582+0000","last_peered":"2023-10-21T16:15:16.346582+0000","last_clean":"2023-10-21T16:15:16.346582+0000","last_became_active":"2023-10-21T16:15:15.175508+0000","last_became_peered":"2023-10-21T16:15:15.175508+0000","last_unstale":"2023-10-21T16:15:16.346582+0000","last_undegraded":"2023-10-21T16:15:16.346582+0000","last_fullsized":"2023-10-21T16:15:16.346582+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:43.426934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132595+0000","last_change":"2023-10-21T16:15:14.122127+0000","last_active":"2023-10-21T16:15:16.132595+0000","last_peered":"2023-10-21T16:15:16.132595+0000","last_clean":"2023-10-21T16:15:16.132595+0000","last_became_active":"2023-10-21T16:15:14.121120+0000","last_became_peered":"2023-10-21T16:15:14.121120+0000","last_unstale":"2023-10-21T16:15:16.132595+0000","last_undegraded":"2023-10-21T16:15:16.132595+0000","last_fullsized":"2023-10-21T16:15:16.132595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:36:37.374348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343464+0000","last_change":"2023-10-21T16:15:15.164762+0000","last_active":"2023-10-21T16:15:16.343464+0000","last_peered":"2023-10-21T16:15:16.343464+0000","last_clean":"2023-10-21T16:15:16.343464+0000","last_became_active":"2023-10-21T16:15:15.161224+0000","last_became_peered":"2023-10-21T16:15:15.161224+0000","last_unstale":"2023-10-21T16:15:16.343464+0000","last_undegraded":"2023-10-21T16:15:16.343464+0000","last_fullsized":"2023-10-21T16:15:16.343464+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:18.236248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347678+0000","last_change":"2023-10-21T16:15:14.137016+0000","last_active":"2023-10-21T16:15:16.347678+0000","last_peered":"2023-10-21T16:15:16.347678+0000","last_clean":"2023-10-21T16:15:16.347678+0000","last_became_active":"2023-10-21T16:15:14.136682+0000","last_became_peered":"2023-10-21T16:15:14.136682+0000","last_unstale":"2023-10-21T16:15:16.347678+0000","last_undegraded":"2023-10-21T16:15:16.347678+0000","last_fullsized":"2023-10-21T16:15:16.347678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:08.466171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345588+0000","last_change":"2023-10-21T16:15:15.161754+0000","last_active":"2023-10-21T16:15:16.345588+0000","last_peered":"2023-10-21T16:15:16.345588+0000","last_clean":"2023-10-21T16:15:16.345588+0000","last_became_active":"2023-10-21T16:15:15.161459+0000","last_became_peered":"2023-10-21T16:15:15.161459+0000","last_unstale":"2023-10-21T16:15:16.345588+0000","last_undegraded":"2023-10-21T16:15:16.345588+0000","last_fullsized":"2023-10-21T16:15:16.345588+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:11.943378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345633+0000","last_change":"2023-10-21T16:15:14.123848+0000","last_active":"2023-10-21T16:15:16.345633+0000","last_peered":"2023-10-21T16:15:16.345633+0000","last_clean":"2023-10-21T16:15:16.345633+0000","last_became_active":"2023-10-21T16:15:14.123520+0000","last_became_peered":"2023-10-21T16:15:14.123520+0000","last_unstale":"2023-10-21T16:15:16.345633+0000","last_undegraded":"2023-10-21T16:15:16.345633+0000","last_fullsized":"2023-10-21T16:15:16.345633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:50.108117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131917+0000","last_change":"2023-10-21T16:15:15.164413+0000","last_active":"2023-10-21T16:15:16.131917+0000","last_peered":"2023-10-21T16:15:16.131917+0000","last_clean":"2023-10-21T16:15:16.131917+0000","last_became_active":"2023-10-21T16:15:15.164151+0000","last_became_peered":"2023-10-21T16:15:15.164151+0000","last_unstale":"2023-10-21T16:15:16.131917+0000","last_undegraded":"2023-10-21T16:15:16.131917+0000","last_fullsized":"2023-10-21T16:15:16.131917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:41:08.955167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344509+0000","last_change":"2023-10-21T16:15:14.124632+0000","last_active":"2023-10-21T16:15:16.344509+0000","last_peered":"2023-10-21T16:15:16.344509+0000","last_clean":"2023-10-21T16:15:16.344509+0000","last_became_active":"2023-10-21T16:15:14.123070+0000","last_became_peered":"2023-10-21T16:15:14.123070+0000","last_unstale":"2023-10-21T16:15:16.344509+0000","last_undegraded":"2023-10-21T16:15:16.344509+0000","last_fullsized":"2023-10-21T16:15:16.344509+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:48:27.256478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344284+0000","last_change":"2023-10-21T16:15:15.172753+0000","last_active":"2023-10-21T16:15:16.344284+0000","last_peered":"2023-10-21T16:15:16.344284+0000","last_clean":"2023-10-21T16:15:16.344284+0000","last_became_active":"2023-10-21T16:15:15.171529+0000","last_became_peered":"2023-10-21T16:15:15.171529+0000","last_unstale":"2023-10-21T16:15:16.344284+0000","last_undegraded":"2023-10-21T16:15:16.344284+0000","last_fullsized":"2023-10-21T16:15:16.344284+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:54:35.680365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132038+0000","last_change":"2023-10-21T16:15:14.128533+0000","last_active":"2023-10-21T16:15:16.132038+0000","last_peered":"2023-10-21T16:15:16.132038+0000","last_clean":"2023-10-21T16:15:16.132038+0000","last_became_active":"2023-10-21T16:15:14.128263+0000","last_became_peered":"2023-10-21T16:15:14.128263+0000","last_unstale":"2023-10-21T16:15:16.132038+0000","last_undegraded":"2023-10-21T16:15:16.132038+0000","last_fullsized":"2023-10-21T16:15:16.132038+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:08.727743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344189+0000","last_change":"2023-10-21T16:15:15.163219+0000","last_active":"2023-10-21T16:15:16.344189+0000","last_peered":"2023-10-21T16:15:16.344189+0000","last_clean":"2023-10-21T16:15:16.344189+0000","last_became_active":"2023-10-21T16:15:15.162037+0000","last_became_peered":"2023-10-21T16:15:15.162037+0000","last_unstale":"2023-10-21T16:15:16.344189+0000","last_undegraded":"2023-10-21T16:15:16.344189+0000","last_fullsized":"2023-10-21T16:15:16.344189+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:55:54.669363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.131721+0000","last_change":"2023-10-21T16:15:14.143628+0000","last_active":"2023-10-21T16:15:16.131721+0000","last_peered":"2023-10-21T16:15:16.131721+0000","last_clean":"2023-10-21T16:15:16.131721+0000","last_became_active":"2023-10-21T16:15:14.143077+0000","last_became_peered":"2023-10-21T16:15:14.143077+0000","last_unstale":"2023-10-21T16:15:16.131721+0000","last_undegraded":"2023-10-21T16:15:16.131721+0000","last_fullsized":"2023-10-21T16:15:16.131721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:17.221662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.344842+0000","last_change":"2023-10-21T16:15:15.165299+0000","last_active":"2023-10-21T16:15:16.344842+0000","last_peered":"2023-10-21T16:15:16.344842+0000","last_clean":"2023-10-21T16:15:16.344842+0000","last_became_active":"2023-10-21T16:15:15.165020+0000","last_became_peered":"2023-10-21T16:15:15.165020+0000","last_unstale":"2023-10-21T16:15:16.344842+0000","last_undegraded":"2023-10-21T16:15:16.344842+0000","last_fullsized":"2023-10-21T16:15:16.344842+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.135495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.135706+0000","last_change":"2023-10-21T16:15:14.132685+0000","last_active":"2023-10-21T16:15:16.135706+0000","last_peered":"2023-10-21T16:15:16.135706+0000","last_clean":"2023-10-21T16:15:16.135706+0000","last_became_active":"2023-10-21T16:15:14.132306+0000","last_became_peered":"2023-10-21T16:15:14.132306+0000","last_unstale":"2023-10-21T16:15:16.135706+0000","last_undegraded":"2023-10-21T16:15:16.135706+0000","last_fullsized":"2023-10-21T16:15:16.135706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:05.378738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.132663+0000","last_change":"2023-10-21T16:15:15.174061+0000","last_active":"2023-10-21T16:15:16.132663+0000","last_peered":"2023-10-21T16:15:16.132663+0000","last_clean":"2023-10-21T16:15:16.132663+0000","last_became_active":"2023-10-21T16:15:15.163769+0000","last_became_peered":"2023-10-21T16:15:15.163769+0000","last_unstale":"2023-10-21T16:15:16.132663+0000","last_undegraded":"2023-10-21T16:15:16.132663+0000","last_fullsized":"2023-10-21T16:15:16.132663+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:50.487376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.343711+0000","last_change":"2023-10-21T16:15:14.138642+0000","last_active":"2023-10-21T16:15:16.343711+0000","last_peered":"2023-10-21T16:15:16.343711+0000","last_clean":"2023-10-21T16:15:16.343711+0000","last_became_active":"2023-10-21T16:15:14.138178+0000","last_became_peered":"2023-10-21T16:15:14.138178+0000","last_unstale":"2023-10-21T16:15:16.343711+0000","last_undegraded":"2023-10-21T16:15:16.343711+0000","last_fullsized":"2023-10-21T16:15:16.343711+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:29.613027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.347329+0000","last_change":"2023-10-21T16:15:15.167240+0000","last_active":"2023-10-21T16:15:16.347329+0000","last_peered":"2023-10-21T16:15:16.347329+0000","last_clean":"2023-10-21T16:15:16.347329+0000","last_became_active":"2023-10-21T16:15:15.166937+0000","last_became_peered":"2023-10-21T16:15:15.166937+0000","last_unstale":"2023-10-21T16:15:16.347329+0000","last_undegraded":"2023-10-21T16:15:16.347329+0000","last_fullsized":"2023-10-21T16:15:16.347329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:14.089537+0000","last_clean_scrub_stamp":"2023-10-21T16:15:14.089537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:28:50.689542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-21T16:15:16.345061+0000","last_change":"2023-10-21T16:15:14.132922+0000","last_active":"2023-10-21T16:15:16.345061+0000","last_peered":"2023-10-21T16:15:16.345061+0000","last_clean":"2023-10-21T16:15:16.345061+0000","last_became_active":"2023-10-21T16:15:14.132608+0000","last_became_peered":"2023-10-21T16:15:14.132608+0000","last_unstale":"2023-10-21T16:15:16.345061+0000","last_undegraded":"2023-10-21T16:15:16.345061+0000","last_fullsized":"2023-10-21T16:15:16.345061+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:15:13.074951+0000","last_clean_scrub_stamp":"2023-10-21T16:15:13.074951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:13:38.423265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"60'139","reported_seq":293,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-21T16:15:15.155282+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:15:15.155282+0000","last_peered":"2023-10-21T16:15:15.155282+0000","last_clean":"2023-10-21T16:15:15.155282+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:15:15.155282+0000","last_undegraded":"2023-10-21T16:15:15.155282+0000","last_fullsized":"2023-10-21T16:15:15.155282+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1466368,"data_stored":1022275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1890000000000001,"15min":1.1279999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5600000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.641},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.2370000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.181,"15min":1.145},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.5840000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.3260000000000001,"15min":1.294},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.736,"5min":2.085,"15min":2.085},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3200000000000001,"15min":1.2689999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.619}]},{"osd":3,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2909999999999999,"15min":1.2549999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.278},{"interface":"front","average":{"1min":1.369,"5min":1.4379999999999999,"15min":1.4219999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.9570000000000001,"5min":1.9750000000000001,"15min":2.0099999999999998},"last":1.417}]},{"osd":4,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3560000000000001,"15min":1.373},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.679},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4390000000000001,"15min":1.454},"min":{"1min":0.93100000000000005,"5min":0.82599999999999996,"15min":0.70399999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.7110000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3799999999999999,"15min":1.425},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.865,"5min":1.8720000000000001,"15min":2.258},"last":1.181},{"interface":"front","average":{"1min":1.498,"5min":1.446,"15min":1.4590000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.60899999999999999},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.4770000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:15:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.5509999999999999,"15min":1.5669999999999999},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.762},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.4690000000000001,"15min":1.466},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.552}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":432435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.129,"15min":1.1220000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.4590000000000001,"5min":1.806,"15min":1.806},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.2350000000000001,"15min":1.212},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.60599999999999998},"max":{"1min":1.895,"5min":1.895,"15min":2.4350000000000001},"last":1.212}]},{"osd":1,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.212,"15min":1.21},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.492},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.2909999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.3}]},{"osd":2,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.244,"15min":1.2689999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2549999999999999,"15min":1.238},"min":{"1min":0.89400000000000002,"5min":0.876,"15min":0.82899999999999996},"max":{"1min":1.5449999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.6459999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.343,"15min":1.3740000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":3.46},"last":1.054},{"interface":"front","average":{"1min":1.512,"5min":1.4430000000000001,"15min":1.4339999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":4.6210000000000004,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.575}]},{"osd":4,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.319,"15min":1.375},"min":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.76300000000000001},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.351,"15min":1.413},"min":{"1min":0.96099999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":0.96199999999999997}]},{"osd":5,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.516},"min":{"1min":0.85299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.6909999999999998},"last":1.401},{"interface":"front","average":{"1min":1.425,"5min":1.3320000000000001,"15min":1.3580000000000001},"min":{"1min":0.81499999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.732}]},{"osd":7,"last update":"Sat Oct 21 16:15:11 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.85999999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.339,"5min":2.339,"15min":3.71},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.5960000000000001,"15min":1.6240000000000001},"min":{"1min":1.1779999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.7650000000000001},"last":1.776}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":432365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1379999999999999,"15min":1.111},"min":{"1min":0.76900000000000002,"5min":0.73299999999999998,"15min":0.55400000000000005},"max":{"1min":1.5629999999999999,"5min":1.919,"15min":1.919},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.244},"min":{"1min":0.91400000000000003,"5min":0.78800000000000003,"15min":0.73599999999999999},"max":{"1min":1.526,"5min":1.7849999999999999,"15min":3.6080000000000001},"last":0.86299999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2709999999999999,"15min":1.282},"min":{"1min":0.90300000000000002,"5min":0.83999999999999997,"15min":0.69399999999999995},"max":{"1min":1.6140000000000001,"5min":1.784,"15min":2.0699999999999998},"last":1.25},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2490000000000001,"15min":1.2270000000000001},"min":{"1min":0.86699999999999999,"5min":0.83099999999999996,"15min":0.80200000000000005},"max":{"1min":1.5600000000000001,"5min":1.9690000000000001,"15min":1.9690000000000001},"last":1.306}]},{"osd":2,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":0.95199999999999996,"5min":0.85199999999999998,"15min":0.73399999999999999},"max":{"1min":1.649,"5min":1.7450000000000001,"15min":1.954},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.873,"5min":0.873,"15min":0.77100000000000002},"max":{"1min":1.8260000000000001,"5min":1.9059999999999999,"15min":2},"last":1.3779999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3140000000000001,"15min":1.361},"min":{"1min":1.0389999999999999,"5min":0.82699999999999996,"15min":0.80600000000000005},"max":{"1min":1.6850000000000001,"5min":2.0630000000000002,"15min":3.8109999999999999},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3779999999999999,"15min":1.389},"min":{"1min":1.0249999999999999,"5min":0.97599999999999998,"15min":0.75900000000000001},"max":{"1min":1.6419999999999999,"5min":1.952,"15min":2.0830000000000002},"last":1.722}]},{"osd":5,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.286,"15min":1.361},"min":{"1min":0.91700000000000004,"5min":0.75700000000000001,"15min":0.67700000000000005},"max":{"1min":1.7110000000000001,"5min":1.766,"15min":2.6219999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3200000000000001,"15min":1.365},"min":{"1min":0.95799999999999996,"5min":0.878,"15min":0.65700000000000003},"max":{"1min":1.774,"5min":2.1459999999999999,"15min":3.9300000000000002},"last":1.474}]},{"osd":6,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.379,"15min":1.4350000000000001},"min":{"1min":0.997,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.871,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.911},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5620000000000001,"15min":1.633},"min":{"1min":1.369,"5min":0.90100000000000002,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.0920000000000001,"15min":3.9609999999999999},"last":1.514}]},{"osd":7,"last update":"Sat Oct 21 16:14:15 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.4530000000000001,"15min":1.5429999999999999},"min":{"1min":1.1220000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8300000000000001,"5min":2.4809999999999999,"15min":4.0190000000000001},"last":1.756},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5189999999999999,"15min":1.587},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.84699999999999998},"max":{"1min":1.8540000000000001,"5min":2.0609999999999999,"15min":2.879},"last":1.8080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":431783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.117,"15min":1.105},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3140000000000001,"5min":1.6919999999999999,"15min":2.0070000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.024,"5min":1.218,"15min":1.181},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.6259999999999999,"5min":1.8080000000000001,"15min":1.9530000000000001},"last":0.97099999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.2390000000000001,"15min":1.2430000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.7150000000000001,"5min":1.8919999999999999,"15min":1.944},"last":1.298},{"interface":"front","average":{"1min":1.135,"5min":1.228,"15min":1.23},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.641,"5min":1.7050000000000001,"15min":1.907},"last":0.90600000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.244,"15min":1.2929999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.659,"5min":1.9199999999999999,"15min":1.95},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.1930000000000001,"15min":1.2490000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4410000000000001,"5min":1.7,"15min":2.0249999999999999},"last":1.4410000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.873,"5min":1.9259999999999999,"15min":2.0630000000000002},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.3779999999999999,"15min":1.3819999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5329999999999999,"5min":1.833,"15min":1.8959999999999999},"last":1.3640000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2150000000000001,"15min":1.2370000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7529999999999999,"5min":1.843,"15min":2.0960000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3480000000000001,"15min":1.339},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.004,"5min":2.004,"15min":2.121},"last":1.4079999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.423,"15min":1.448},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.1360000000000001},"last":1.546},{"interface":"front","average":{"1min":1.365,"5min":1.4690000000000001,"15min":1.5},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":1.5229999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:15:17 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.506,"15min":1.506},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.9330000000000001,"5min":2.218,"15min":2.218},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.677}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36208,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335632,"statfs":{"total":96636764160,"available":96599687168,"internally_reserved":0,"allocated":880640,"data_stored":433417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.95199999999999996,"15min":0.92800000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.5580000000000001,"5min":1.746,"15min":1.746},"last":1.4419999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.222,"15min":1.1830000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.4490000000000001,"5min":2.04,"15min":2.5289999999999999},"last":0.878}]},{"osd":1,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0309999999999999,"15min":1.0109999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.0070000000000001,"5min":2.173,"15min":2.2519999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.2649999999999999,"15min":1.264},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":3.827},"last":1.2849999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.101,"15min":1.05},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.446,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.2190000000000001,"15min":1.212},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":2.847,"5min":2.847,"15min":2.847},"last":0.94699999999999995}]},{"osd":4,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.4119999999999999,"15min":1.3939999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6950000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.4019999999999999,"15min":1.4099999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":1.083}]},{"osd":5,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.401,"15min":1.4279999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.59399999999999997},"max":{"1min":1.774,"5min":1.8759999999999999,"15min":3.7250000000000001},"last":1.347},{"interface":"front","average":{"1min":1.333,"5min":1.524,"15min":1.4970000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8100000000000001,"5min":2.423,"15min":2.423},"last":1.1259999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.502,"15min":1.528},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.5229999999999999,"15min":1.5429999999999999},"min":{"1min":0.871,"5min":0.82499999999999996,"15min":0.70299999999999996},"max":{"1min":1.9259999999999999,"5min":2.2290000000000001,"15min":3.573},"last":1.4890000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.635,"15min":1.593},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.966,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.599,"15min":1.6080000000000001},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3939999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58408,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313432,"statfs":{"total":96636764160,"available":96576954368,"internally_reserved":0,"allocated":876544,"data_stored":433395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.83199999999999996,"15min":0.86199999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.141,"5min":1.6359999999999999,"15min":1.6819999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.755}]},{"osd":1,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":1.095,"15min":1.131},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.907,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.042},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.25,"15min":1.254},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6919999999999999,"5min":1.7749999999999999,"15min":1.8129999999999999},"last":1.0760000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.0700000000000001,"15min":1.1140000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.379},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.069,"5min":1.319,"15min":1.3480000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.827,"5min":1.879,"15min":2.0390000000000001},"last":1.375}]},{"osd":4,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.3600000000000001,"15min":1.359},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.6180000000000001,"5min":1.9239999999999999,"15min":2.206},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.357,"15min":1.353},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3009999999999999,"15min":1.337},"min":{"1min":0.752,"5min":0.752,"15min":0.71099999999999997},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.492,"15min":1.5049999999999999},"min":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.86199999999999999},"max":{"1min":1.819,"5min":1.875,"15min":2.4870000000000001},"last":1.54}]},{"osd":6,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.462,"15min":1.4530000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.698},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.389,"15min":1.446},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.552,"5min":2.552,"15min":2.6179999999999999},"last":1.1479999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:15:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.5229999999999999,"15min":1.536},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.319},{"interface":"front","average":{"1min":1.321,"5min":1.492,"15min":1.532},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.2609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335648,"statfs":{"total":96636764160,"available":96599703552,"internally_reserved":0,"allocated":872448,"data_stored":431943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.92000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3400000000000001,"5min":1.905,"15min":1.905},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.028,"15min":1.014},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6559999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.161,"15min":1.194},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.546,"5min":1.974,"15min":2.29},"last":1.0229999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.3049999999999999,"15min":1.3180000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6930000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.2350000000000001,"15min":1.238},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.7989999999999999,"5min":1.877,"15min":2.3620000000000001},"last":1.4219999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.306,"15min":1.29},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.778,"5min":1.944,"15min":1.944},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9370000000000001,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.319}]},{"osd":5,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.482,"15min":1.528},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.75900000000000001},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.478,"15min":1.5029999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.357}]},{"osd":6,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.488,"15min":1.548},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.2469999999999999,"5min":2.2959999999999998,"15min":4.1260000000000003},"last":1.282},{"interface":"front","average":{"1min":1.302,"5min":1.4630000000000001,"15min":1.51},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.206,"5min":2.2090000000000001,"15min":3.101},"last":1.087}]},{"osd":7,"last update":"Sat Oct 21 16:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.6040000000000001,"15min":1.637},"min":{"1min":0.90700000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.1560000000000001,"5min":2.3730000000000002,"15min":2.8180000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.6080000000000001,"15min":1.6599999999999999},"min":{"1min":0.89600000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.1619999999999999,"5min":2.2349999999999999,"15min":3.952},"last":1.121}]}]},{"osd":0,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1466368,"data_stored":1022207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":0.95099999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.36399999999999999},"max":{"1min":1.3999999999999999,"5min":1.855,"15min":2.9260000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.101,"15min":1.0620000000000001},"min":{"1min":0.80800000000000005,"5min":0.74299999999999999,"15min":0.42799999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.171,"15min":1.1519999999999999},"min":{"1min":0.82999999999999996,"5min":0.82099999999999995,"15min":0.41799999999999998},"max":{"1min":1.548,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":0.996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.177,"15min":1.149},"min":{"1min":0.89800000000000002,"5min":0.75900000000000001,"15min":0.42999999999999999},"max":{"1min":1.46,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.091}]},{"osd":3,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.151,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.68700000000000006,"15min":0.46500000000000002},"max":{"1min":1.718,"5min":1.8740000000000001,"15min":2.1429999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.3700000000000001,"15min":1.321},"min":{"1min":0.93999999999999995,"5min":0.879,"15min":0.44900000000000001},"max":{"1min":1.758,"5min":1.972,"15min":3.052},"last":1.224}]},{"osd":4,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.415,"15min":1.4410000000000001},"min":{"1min":0.98799999999999999,"5min":0.96299999999999997,"15min":0.88900000000000001},"max":{"1min":1.6619999999999999,"5min":1.954,"15min":3.3580000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.54,"5min":1.5860000000000001,"15min":1.548},"min":{"1min":1.1679999999999999,"5min":1.042,"15min":0.92400000000000004},"max":{"1min":2.246,"5min":2.2890000000000001,"15min":3.306},"last":1.4119999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4319999999999999,"15min":1.411},"min":{"1min":1.075,"5min":1.0429999999999999,"15min":0.78000000000000003},"max":{"1min":1.845,"5min":2.137,"15min":3.8849999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5760000000000001,"15min":1.5509999999999999},"min":{"1min":1.1639999999999999,"5min":1.135,"15min":0.81000000000000005},"max":{"1min":2.137,"5min":2.4670000000000001,"15min":3.1789999999999998},"last":1.3149999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4990000000000001,"15min":1.5209999999999999},"min":{"1min":1.119,"5min":1.044,"15min":0.76800000000000002},"max":{"1min":1.9079999999999999,"5min":2.2519999999999998,"15min":4.3019999999999996},"last":1.282},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.655,"15min":1.635},"min":{"1min":1.3799999999999999,"5min":1.2050000000000001,"15min":0.97099999999999997},"max":{"1min":2.355,"5min":2.403,"15min":3.5139999999999998},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:14:18 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4710000000000001,"15min":1.494},"min":{"1min":1.0549999999999999,"5min":1.048,"15min":0.83899999999999997},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.647,"5min":1.627,"15min":1.6339999999999999},"min":{"1min":1.3149999999999999,"5min":1.131,"15min":0.82899999999999996},"max":{"1min":2.0049999999999999,"5min":2.3540000000000001,"15min":4.6219999999999999},"last":1.5289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:15:20.233 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:15:20.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:15:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.507+0000 7fda9d8e0640 1 Processor -- start 2023-10-21T16:15:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.507+0000 7fda9d8e0640 1 -- start start 2023-10-21T16:15:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda9d8e0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda980b89f0 0x7fda980b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda9d8e0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda980cd9c0 0x7fda980cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda9d8e0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda980ce310 con 0x7fda9815cf00 2023-10-21T16:15:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda9d8e0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda980d0da0 con 0x7fda980b89f0 2023-10-21T16:15:20.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda9d8e0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda980d1040 con 0x7fda980cd9c0 2023-10-21T16:15:20.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda967fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda980b89f0 0x7fda980b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:20.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda967fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fda980b89f0 0x7fda980b1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56426/0 (socket says 172.21.15.53:56426) 2023-10-21T16:15:20.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.509+0000 7fda967fc640 1 -- 172.21.15.53:0/3768419660 learned_addr learned my addr 172.21.15.53:0/3768419660 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:20.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.510+0000 7fda967fc640 1 -- 172.21.15.53:0/3768419660 >> v2:172.21.15.124:3300/0 conn(0x7fda980cd9c0 msgr2=0x7fda980cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:20.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.510+0000 7fda967fc640 1 --2- 172.21.15.53:0/3768419660 >> v2:172.21.15.124:3300/0 conn(0x7fda980cd9c0 0x7fda980cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.510+0000 7fda967fc640 1 -- 172.21.15.53:0/3768419660 >> v1:172.21.15.124:6789/0 conn(0x7fda9815cf00 legacy=0x7fda980b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:20.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.510+0000 7fda967fc640 1 -- 172.21.15.53:0/3768419660 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda980d1320 con 0x7fda980b89f0 2023-10-21T16:15:20.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.511+0000 7fda967fc640 1 --2- 172.21.15.53:0/3768419660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda980b1df0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda9808c020 tx=0x7fda8002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f99fb2c6e0ac26e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:20.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.511+0000 7fda95ffb640 1 -- 172.21.15.53:0/3768419660 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80030860 con 0x7fda980b89f0 2023-10-21T16:15:20.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.511+0000 7fda95ffb640 1 -- 172.21.15.53:0/3768419660 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda800309c0 con 0x7fda980b89f0 2023-10-21T16:15:20.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.511+0000 7fda95ffb640 1 -- 172.21.15.53:0/3768419660 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80030d50 con 0x7fda980b89f0 2023-10-21T16:15:20.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3768419660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 msgr2=0x7fda980b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3768419660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda980b1df0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fda9808c020 tx=0x7fda8002b9a0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3768419660 shutdown_connections 2023-10-21T16:15:20.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3768419660 >> v2:172.21.15.124:3300/0 conn(0x7fda980cd9c0 0x7fda980cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3768419660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda980b1df0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3768419660 >> 172.21.15.53:0/3768419660 conn(0x7fda980cf700 msgr2=0x7fda980cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:20.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3768419660 shutdown_connections 2023-10-21T16:15:20.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.512+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3768419660 wait complete. 2023-10-21T16:15:20.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.514+0000 7fda9d8e0640 1 Processor -- start 2023-10-21T16:15:20.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.514+0000 7fda9d8e0640 1 -- start start 2023-10-21T16:15:20.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 0x7fda981592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda9815cf00 0x7fda98155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda98157370 con 0x7fda980b89f0 2023-10-21T16:15:20.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda981f2f00 con 0x7fda980cd9c0 2023-10-21T16:15:20.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda9d8e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda98159810 con 0x7fda9815cf00 2023-10-21T16:15:20.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.515+0000 7fda96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:20.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56434/0 (socket says 172.21.15.53:56434) 2023-10-21T16:15:20.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 -- 172.21.15.53:0/3874799991 learned_addr learned my addr 172.21.15.53:0/3874799991 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:20.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda977fe640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda9815cf00 0x7fda98155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:20.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda967fc640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 0x7fda981592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:20.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 -- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda9815cf00 msgr2=0x7fda98155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda9815cf00 0x7fda98155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 -- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 msgr2=0x7fda981592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 0x7fda981592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 -- 172.21.15.53:0/3874799991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800094d0 con 0x7fda980b89f0 2023-10-21T16:15:20.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda967fc640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 0x7fda981592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:20.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.516+0000 7fda96ffd640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda8800c7d0 tx=0x7fda8800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:20.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.517+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88003d80 con 0x7fda980b89f0 2023-10-21T16:15:20.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.517+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda88005070 con 0x7fda980b89f0 2023-10-21T16:15:20.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.517+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88005420 con 0x7fda980b89f0 2023-10-21T16:15:20.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.517+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9815a000 con 0x7fda980b89f0 2023-10-21T16:15:20.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.517+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda98159d20 con 0x7fda980b89f0 2023-10-21T16:15:20.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.518+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda981574b0 con 0x7fda980b89f0 2023-10-21T16:15:20.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.519+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda88005580 con 0x7fda980b89f0 2023-10-21T16:15:20.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.520+0000 7fda8ffff640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 0x7fda7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:20.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.520+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda88088da0 con 0x7fda980b89f0 2023-10-21T16:15:20.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.521+0000 7fda967fc640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 0x7fda7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:20.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.522+0000 7fda967fc640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 0x7fda7806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda80036000 tx=0x7fda8003b040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:20.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.525+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda981574b0 con 0x7fda980b89f0 2023-10-21T16:15:20.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.793+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fda981574b0 con 0x7fda980b89f0 2023-10-21T16:15:20.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.797+0000 7fda8ffff640 1 -- 172.21.15.53:0/3874799991 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fda981574b0 con 0x7fda980b89f0 2023-10-21T16:15:20.799 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:15:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:15:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:15:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:15:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:15:20.800 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:15:20.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.800+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 msgr2=0x7fda7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.800+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 0x7fda7806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda80036000 tx=0x7fda8003b040 comp rx=0 tx=0).stop 2023-10-21T16:15:20.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.800+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 msgr2=0x7fda981f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:20.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.800+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fda8800c7d0 tx=0x7fda8800cc90 comp rx=0 tx=0).stop 2023-10-21T16:15:20.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 shutdown_connections 2023-10-21T16:15:20.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fda7806af90 0x7fda7806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda9815cf00 0x7fda98155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda980cd9c0 0x7fda981592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 --2- 172.21.15.53:0/3874799991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda980b89f0 0x7fda981f29c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 >> 172.21.15.53:0/3874799991 conn(0x7fda980cf700 msgr2=0x7fda980b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 shutdown_connections 2023-10-21T16:15:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:20.801+0000 7fda9d8e0640 1 -- 172.21.15.53:0/3874799991 wait complete. 2023-10-21T16:15:20.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:15:21.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.044+0000 7f1b30646640 1 Processor -- start 2023-10-21T16:15:21.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.044+0000 7f1b30646640 1 -- start start 2023-10-21T16:15:21.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b30646640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b280c90b0 0x7f1b280cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b30646640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2815b2b0 con 0x7f1b2815ab40 2023-10-21T16:15:21.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b30646640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2815b590 con 0x7f1b280cb3b0 2023-10-21T16:15:21.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b30646640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2815bf30 con 0x7f1b280c90b0 2023-10-21T16:15:21.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b2e3e2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1b2815ab40 0x7f1b280ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51738/0 (socket says 172.21.15.53:51738) 2023-10-21T16:15:21.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.046+0000 7f1b2e3e2640 1 -- 172.21.15.53:0/3277269417 learned_addr learned my addr 172.21.15.53:0/3277269417 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:21.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2dbe1640 1 --2- 172.21.15.53:0/3277269417 >> v2:172.21.15.53:3300/0 conn(0x7f1b280c90b0 0x7f1b280cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2d3e0640 1 -- 172.21.15.53:0/3277269417 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918712924 0 0) 0x7f1b2815b2b0 con 0x7f1b2815ab40 2023-10-21T16:15:21.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2d3e0640 1 -- 172.21.15.53:0/3277269417 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b14003520 con 0x7f1b2815ab40 2023-10-21T16:15:21.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/3277269417 >> v1:172.21.15.124:6789/0 conn(0x7f1b280cb3b0 legacy=0x7f1b2815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/3277269417 >> v1:172.21.15.53:6789/0 conn(0x7f1b2815ab40 legacy=0x7f1b280ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.047+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/3277269417 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2815b870 con 0x7f1b280c90b0 2023-10-21T16:15:21.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.048+0000 7f1b2dbe1640 1 --2- 172.21.15.53:0/3277269417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280c90b0 0x7f1b280cae70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b180097a0 tx=0x7f1b1802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6212193f9c62ef9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b2d3e0640 1 -- 172.21.15.53:0/3277269417 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18030610 con 0x7f1b280c90b0 2023-10-21T16:15:21.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b2d3e0640 1 -- 172.21.15.53:0/3277269417 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18030770 con 0x7f1b280c90b0 2023-10-21T16:15:21.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b2d3e0640 1 -- 172.21.15.53:0/3277269417 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18030b00 con 0x7f1b280c90b0 2023-10-21T16:15:21.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 -- 172.21.15.53:0/3277269417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280c90b0 msgr2=0x7f1b280cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 --2- 172.21.15.53:0/3277269417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280c90b0 0x7f1b280cae70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b180097a0 tx=0x7f1b1802b960 comp rx=0 tx=0).stop 2023-10-21T16:15:21.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 -- 172.21.15.53:0/3277269417 shutdown_connections 2023-10-21T16:15:21.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 --2- 172.21.15.53:0/3277269417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280c90b0 0x7f1b280cae70 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 -- 172.21.15.53:0/3277269417 >> 172.21.15.53:0/3277269417 conn(0x7f1b2806a380 msgr2=0x7f1b280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:21.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 -- 172.21.15.53:0/3277269417 shutdown_connections 2023-10-21T16:15:21.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.049+0000 7f1b30646640 1 -- 172.21.15.53:0/3277269417 wait complete. 2023-10-21T16:15:21.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.051+0000 7f1b30646640 1 Processor -- start 2023-10-21T16:15:21.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.051+0000 7f1b30646640 1 -- start start 2023-10-21T16:15:21.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b280c90b0 0x7f1b28158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 0x7f1b280c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28159390 con 0x7f1b280cb3b0 2023-10-21T16:15:21.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280c5d90 con 0x7f1b280c90b0 2023-10-21T16:15:21.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.052+0000 7f1b30646640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28158710 con 0x7f1b2815ab40 2023-10-21T16:15:21.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56440/0 (socket says 172.21.15.53:56440) 2023-10-21T16:15:21.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/517550132 learned_addr learned my addr 172.21.15.53:0/517550132 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:21.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2e3e2640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b280c90b0 0x7f1b28158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2ebe3640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 0x7f1b280c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 msgr2=0x7f1b280c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 0x7f1b280c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b280c90b0 msgr2=0x7f1b28158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b280c90b0 0x7f1b28158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2dbe1640 1 -- 172.21.15.53:0/517550132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18009400 con 0x7f1b280cb3b0 2023-10-21T16:15:21.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.053+0000 7f1b2ebe3640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 0x7f1b280c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b2dbe1640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1b1802bf70 tx=0x7f1b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1803e040 con 0x7f1b280cb3b0 2023-10-21T16:15:21.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18039cb0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281589f0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18044630 con 0x7f1b280cb3b0 2023-10-21T16:15:21.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.054+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28158fb0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.056+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b281594d0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.057+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b18043390 con 0x7f1b280cb3b0 2023-10-21T16:15:21.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.058+0000 7f1b1effd640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 0x7f1b0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.058+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b180abb80 con 0x7f1b280cb3b0 2023-10-21T16:15:21.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.059+0000 7f1b2e3e2640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 0x7f1b0c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.060+0000 7f1b2e3e2640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 0x7f1b0c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001560 tx=0x7f1b24009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.062+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b281594d0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.086 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-10-21T16:15:21.085+0000 7ff053512640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:15:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.296+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b281594d0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.297+0000 7f1b1effd640 1 -- 172.21.15.53:0/517550132 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f1b281594d0 con 0x7f1b280cb3b0 2023-10-21T16:15:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:21.299 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:15:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:15:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.299+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 msgr2=0x7f1b0c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.299+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 0x7f1b0c06d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001560 tx=0x7f1b24009040 comp rx=0 tx=0).stop 2023-10-21T16:15:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.299+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 msgr2=0x7f1b28146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.299+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1b1802bf70 tx=0x7f1b18004730 comp rx=0 tx=0).stop 2023-10-21T16:15:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 shutdown_connections 2023-10-21T16:15:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f1b0c06af90 0x7f1b0c06d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b2815ab40 0x7f1b280c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b280cb3b0 0x7f1b28146320 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 --2- 172.21.15.53:0/517550132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b280c90b0 0x7f1b28158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 >> 172.21.15.53:0/517550132 conn(0x7f1b2806a380 msgr2=0x7f1b2806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 shutdown_connections 2023-10-21T16:15:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.300+0000 7f1b30646640 1 -- 172.21.15.53:0/517550132 wait complete. 2023-10-21T16:15:21.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:15:14.581274+0000', 'modified': '2023-10-21T16:15:17.733529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6126}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6126': {'gid': 6126, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/3351749776', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 3351749776}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 3351749776}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:15:21.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:15:21.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-21T16:15:21.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.957+0000 7f24bdb8a640 1 Processor -- start 2023-10-21T16:15:21.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.957+0000 7f24bdb8a640 1 -- start start 2023-10-21T16:15:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24bdb8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24bdb8a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24bdb8a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b8071140 con 0x7f24b8160660 2023-10-21T16:15:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24bdb8a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b806e150 con 0x7f24b80cb450 2023-10-21T16:15:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24bdb8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80cb8c0 con 0x7f24b8070cc0 2023-10-21T16:15:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.959+0000 7f24b7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:46554/0 (socket says 172.21.15.53:46554) 2023-10-21T16:15:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b7fff640 1 -- 172.21.15.53:0/4279674757 learned_addr learned my addr 172.21.15.53:0/4279674757 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4279674757 >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 msgr2=0x7f24b80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/4279674757 >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4279674757 >> v1:172.21.15.124:6789/0 conn(0x7f24b8160660 legacy=0x7f24b80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.960+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4279674757 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b806de70 con 0x7f24b8070cc0 2023-10-21T16:15:21.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.961+0000 7f24b7fff640 1 --2- 172.21.15.53:0/4279674757 >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:15:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.962+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/4279674757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b80caf10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd882a71f7187126 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.963+0000 7f24b67fc640 1 -- 172.21.15.53:0/4279674757 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031830 con 0x7f24b8070cc0 2023-10-21T16:15:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.963+0000 7f24b67fc640 1 -- 172.21.15.53:0/4279674757 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0031990 con 0x7f24b8070cc0 2023-10-21T16:15:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.963+0000 7f24b67fc640 1 -- 172.21.15.53:0/4279674757 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031d20 con 0x7f24b8070cc0 2023-10-21T16:15:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 -- 172.21.15.53:0/4279674757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 msgr2=0x7f24b80caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/4279674757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b80caf10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 -- 172.21.15.53:0/4279674757 shutdown_connections 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/4279674757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b80caf10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/4279674757 >> v2:172.21.15.124:3300/0 conn(0x7f24b80cb450 0x7f24b80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 -- 172.21.15.53:0/4279674757 >> 172.21.15.53:0/4279674757 conn(0x7f24b806a5d0 msgr2=0x7f24b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 -- 172.21.15.53:0/4279674757 shutdown_connections 2023-10-21T16:15:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.964+0000 7f24bdb8a640 1 -- 172.21.15.53:0/4279674757 wait complete. 2023-10-21T16:15:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.966+0000 7f24bdb8a640 1 Processor -- start 2023-10-21T16:15:21.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.966+0000 7f24bdb8a640 1 -- start start 2023-10-21T16:15:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b81f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 0x7f24b8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81f24d0 con 0x7f24b80cb450 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80cdb80 con 0x7f24b8160660 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24bdb8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b80c2c60 con 0x7f24b8070cc0 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56454/0 (socket says 172.21.15.53:56454) 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.967+0000 7f24b6ffd640 1 -- 172.21.15.53:0/2928519765 learned_addr learned my addr 172.21.15.53:0/2928519765 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b77fe640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b81f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b7fff640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 0x7f24b8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 -- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 msgr2=0x7f24b81f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b81f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 -- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 msgr2=0x7f24b8151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 0x7f24b8151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 -- 172.21.15.53:0/2928519765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00094d0 con 0x7f24b80cb450 2023-10-21T16:15:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b7fff640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 0x7f24b8151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.968+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f24a002bee0 tx=0x7f24a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.969+0000 7f24affff640 1 -- 172.21.15.53:0/2928519765 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a003a6f0 con 0x7f24b80cb450 2023-10-21T16:15:21.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.969+0000 7f24affff640 1 -- 172.21.15.53:0/2928519765 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a003a870 con 0x7f24b80cb450 2023-10-21T16:15:21.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.969+0000 7f24affff640 1 -- 172.21.15.53:0/2928519765 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a003ac20 con 0x7f24b80cb450 2023-10-21T16:15:21.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.969+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a0009400 con 0x7f24b80cb450 2023-10-21T16:15:21.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.969+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b80c3220 con 0x7f24b80cb450 2023-10-21T16:15:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.972+0000 7f24affff640 1 -- 172.21.15.53:0/2928519765 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f24a0030380 con 0x7f24b80cb450 2023-10-21T16:15:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.973+0000 7f24affff640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 0x7f249806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.973+0000 7f24affff640 1 -- 172.21.15.53:0/2928519765 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f24a00ab350 con 0x7f24b80cb450 2023-10-21T16:15:21.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.973+0000 7f24c07f5740 1 Processor -- start 2023-10-21T16:15:21.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.973+0000 7f24c07f5740 1 -- start start 2023-10-21T16:15:21.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.973+0000 7f24b77fe640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 0x7f249806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db8956e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 0x555db8987250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db88777a0 con 0x555db8984360 2023-10-21T16:15:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db88c9740 con 0x555db89573c0 2023-10-21T16:15:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db88c6e30 con 0x555db8983f80 2023-10-21T16:15:21.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56462/0 (socket says 172.21.15.53:56462) 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4122203336 learned_addr learned my addr 172.21.15.53:0/4122203336 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.974+0000 7f24b77fe640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db8956e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b7fff640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 0x555db8987250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b77fe640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 0x7f249806d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24a80012e0 tx=0x7f24a8009040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 msgr2=0x555db8956e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:21.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db8956e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 msgr2=0x555db8987250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:21.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 0x555db8987250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 -- 172.21.15.53:0/4122203336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db8886910 con 0x555db8984360 2023-10-21T16:15:21.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b7fff640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 0x555db8987250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f24a0079a20 tx=0x7f24a0076190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49100cda5538968e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24ad7fa640 1 -- 172.21.15.53:0/4122203336 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0082020 con 0x555db8984360 2023-10-21T16:15:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24ad7fa640 1 -- 172.21.15.53:0/4122203336 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00780c0 con 0x555db8984360 2023-10-21T16:15:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.975+0000 7f24ad7fa640 1 -- 172.21.15.53:0/4122203336 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00a9660 con 0x555db8984360 2023-10-21T16:15:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 -- 172.21.15.53:0/4122203336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 msgr2=0x555db8986d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f24a0079a20 tx=0x7f24a0076190 comp rx=0 tx=0).stop 2023-10-21T16:15:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 -- 172.21.15.53:0/4122203336 shutdown_connections 2023-10-21T16:15:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89573c0 0x555db8987250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8986d10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 --2- 172.21.15.53:0/4122203336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db8956e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 -- 172.21.15.53:0/4122203336 >> 172.21.15.53:0/4122203336 conn(0x555db8959190 msgr2=0x555db89a1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 -- 172.21.15.53:0/4122203336 shutdown_connections 2023-10-21T16:15:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.976+0000 7f24c07f5740 1 -- 172.21.15.53:0/4122203336 wait complete. 2023-10-21T16:15:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.977+0000 7f24c07f5740 1 -- start start 2023-10-21T16:15:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.977+0000 7f24c07f5740 1 Processor -- start 2023-10-21T16:15:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8984360 0x555db89b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89b5af0 0x555db89b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db88c2720 con 0x555db8983f80 2023-10-21T16:15:21.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8860970 con 0x555db89b5af0 2023-10-21T16:15:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db8860610 con 0x555db8984360 2023-10-21T16:15:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24c07f5740 10 client.0 _ll_register_callbacks cb 0x555db88829d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:15:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56472/0 (socket says 172.21.15.53:56472) 2023-10-21T16:15:21.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.978+0000 7f24b77fe640 1 -- 172.21.15.53:0/3355407889 learned_addr learned my addr 172.21.15.53:0/3355407889 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8984360 0x555db89b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 -- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8984360 msgr2=0x555db89b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8984360 0x555db89b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 -- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89b5af0 msgr2=0x555db89b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:21.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b7fff640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89b5af0 0x555db89b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89b5af0 0x555db89b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 -- 172.21.15.53:0/3355407889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b80c3220 con 0x555db8983f80 2023-10-21T16:15:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24b77fe640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24a80146e0 tx=0x7f24a8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6be80d66f025ddb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:21.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24967fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:15:21.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8003b80 con 0x555db8983f80 2023-10-21T16:15:21.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f24a8003ce0 con 0x555db8983f80 2023-10-21T16:15:21.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.979+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555db88e9160 con 0x555db8983f80 2023-10-21T16:15:21.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a8003ea0 con 0x555db8983f80 2023-10-21T16:15:21.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 10 client.6270 fetch_fsmap learned FSMap version 27 2023-10-21T16:15:21.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x555db88c0da0 con 0x555db8983f80 2023-10-21T16:15:21.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555db88e9160 con 0x555db8983f80 2023-10-21T16:15:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24a8005110 con 0x555db8983f80 2023-10-21T16:15:21.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 10 client.6270 fetch_fsmap finished waiting for FSMap version 27 2023-10-21T16:15:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 10 client.6270 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:15:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 20 client.6270 populate_metadata read hostname 'smithi053' 2023-10-21T16:15:21.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8989600 0x555db89bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 4 client.6270 mds_command: new command op to 6126 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:15:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.980+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 --> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db8894cc0 con 0x555db8989600 2023-10-21T16:15:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.981+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8989600 0x555db89bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.981+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8989600 0x555db89bf880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6060b36e57c6d2e6 server_cookie=ba40e8360d998dd9 in_seq=0 out_seq=0 2023-10-21T16:15:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.981+0000 7f24967fc640 10 client.6270 ms_handle_connect on v2:172.21.15.53:6833/3351749776 2023-10-21T16:15:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24967fc640 1 -- 172.21.15.53:0/3355407889 <== mds.0 v2:172.21.15.53:6833/3351749776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555db8894cc0 con 0x555db8989600 2023-10-21T16:15:22.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24967fc640 10 client.6270 handle_command_reply: tid=0 2023-10-21T16:15:22.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 client.6270 shutdown 2023-10-21T16:15:22.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 msgr2=0x555db897f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24a80146e0 tx=0x7f24a8001250 comp rx=0 tx=0).stop 2023-10-21T16:15:22.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 shutdown_connections 2023-10-21T16:15:22.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8989600 0x555db89bf880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db89b5af0 0x555db89b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8984360 0x555db89b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24c07f5740 1 --2- 172.21.15.53:0/3355407889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8983f80 0x555db897f090 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.982+0000 7f24967fc640 0 client.6270 ms_handle_reset on v2:172.21.15.53:6833/3351749776 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.983+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 >> 172.21.15.53:0/3355407889 conn(0x555db89573c0 msgr2=0x555db895af40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.983+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 shutdown_connections 2023-10-21T16:15:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.983+0000 7f24c07f5740 1 -- 172.21.15.53:0/3355407889 wait complete. 2023-10-21T16:15:22.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:21.983+0000 7f24c07f5740 20 client.6270 trim_cache size 0 max 16384 2023-10-21T16:15:22.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.000+0000 7f24c07f5740 1 Processor -- start 2023-10-21T16:15:22.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.000+0000 7f24c07f5740 1 -- start start 2023-10-21T16:15:22.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8983f80 0x555db895d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db89b5af0 0x555db89b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8626f20 con 0x555db8984360 2023-10-21T16:15:22.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8868590 con 0x555db8983f80 2023-10-21T16:15:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db89c4350 con 0x555db89b5af0 2023-10-21T16:15:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56476/0 (socket says 172.21.15.53:56476) 2023-10-21T16:15:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 -- 172.21.15.53:0/1711083452 learned_addr learned my addr 172.21.15.53:0/1711083452 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 -- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db89b5af0 msgr2=0x555db89b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db89b5af0 0x555db89b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 -- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8983f80 msgr2=0x555db895d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b77fe640 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8983f80 0x555db895d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8983f80 0x555db895d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.001+0000 7f24b6ffd640 1 -- 172.21.15.53:0/1711083452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db89885b0 con 0x555db8984360 2023-10-21T16:15:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.002+0000 7f24b6ffd640 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f24a002be60 tx=0x7f24a0035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9650eb315bc0b68a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.002+0000 7f24adffb640 1 -- 172.21.15.53:0/1711083452 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0082020 con 0x555db8984360 2023-10-21T16:15:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.002+0000 7f24adffb640 1 -- 172.21.15.53:0/1711083452 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00ab0b0 con 0x555db8984360 2023-10-21T16:15:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.002+0000 7f24adffb640 1 -- 172.21.15.53:0/1711083452 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0076430 con 0x555db8984360 2023-10-21T16:15:22.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- 172.21.15.53:0/1711083452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 msgr2=0x555db8984ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f24a002be60 tx=0x7f24a0035000 comp rx=0 tx=0).stop 2023-10-21T16:15:22.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- 172.21.15.53:0/1711083452 shutdown_connections 2023-10-21T16:15:22.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db89b5af0 0x555db89b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db8984360 0x555db8984ef0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1711083452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8983f80 0x555db895d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- 172.21.15.53:0/1711083452 >> 172.21.15.53:0/1711083452 conn(0x555db89573c0 msgr2=0x555db89873d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- 172.21.15.53:0/1711083452 shutdown_connections 2023-10-21T16:15:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- 172.21.15.53:0/1711083452 wait complete. 2023-10-21T16:15:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.003+0000 7f24c07f5740 1 -- start start 2023-10-21T16:15:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.004+0000 7f24c07f5740 1 Processor -- start 2023-10-21T16:15:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db89ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8984360 0x555db89a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db89b0780 con 0x555db89b5af0 2023-10-21T16:15:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db89a32f0 con 0x555db8984360 2023-10-21T16:15:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db89b0430 con 0x555db8983f80 2023-10-21T16:15:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24c07f5740 10 client.0 _ll_register_callbacks cb 0x7f24a800e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:15:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56488/0 (socket says 172.21.15.53:56488) 2023-10-21T16:15:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 -- 172.21.15.53:0/1089020389 learned_addr learned my addr 172.21.15.53:0/1089020389 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b77fe640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db89ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 -- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 msgr2=0x555db89ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db89ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 -- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8984360 msgr2=0x555db89a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8984360 0x555db89a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b7fff640 1 -- 172.21.15.53:0/1089020389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db8886910 con 0x555db89b5af0 2023-10-21T16:15:22.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.005+0000 7f24b77fe640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db89ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:22.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24b7fff640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24b0004700 tx=0x7f24b0012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d21fc217002848 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:22.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24967fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:15:22.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0014880 con 0x555db89b5af0 2023-10-21T16:15:22.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a00780c0 con 0x555db89b5af0 2023-10-21T16:15:22.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f24b00149e0 con 0x555db89b5af0 2023-10-21T16:15:22.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b0014ba0 con 0x555db89b5af0 2023-10-21T16:15:22.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24c07f5740 10 client.6276 fetch_fsmap learned FSMap version 27 2023-10-21T16:15:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.006+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x555db885fb80 con 0x555db89b5af0 2023-10-21T16:15:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a00780c0 con 0x555db89b5af0 2023-10-21T16:15:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24b001c020 con 0x555db89b5af0 2023-10-21T16:15:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 10 client.6276 fetch_fsmap finished waiting for FSMap version 27 2023-10-21T16:15:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 10 client.6276 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:15:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 20 client.6276 populate_metadata read hostname 'smithi053' 2023-10-21T16:15:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8982200 0x555db89825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 4 client.6276 mds_command: new command op to 6126 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:15:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 --> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555db88bb700 con 0x555db8982200 2023-10-21T16:15:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.007+0000 7f24b77fe640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8982200 0x555db89825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24b77fe640 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8982200 0x555db89825e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3923457110bba392 server_cookie=8b3dc3a0daf849e9 in_seq=0 out_seq=0 2023-10-21T16:15:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24967fc640 10 client.6276 ms_handle_connect on v2:172.21.15.53:6833/3351749776 2023-10-21T16:15:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24967fc640 1 -- 172.21.15.53:0/1089020389 <== mds.0 v2:172.21.15.53:6833/3351749776 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555db88bb700 con 0x555db8982200 2023-10-21T16:15:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24967fc640 10 client.6276 handle_command_reply: tid=0 2023-10-21T16:15:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 client.6276 shutdown 2023-10-21T16:15:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 msgr2=0x555db89a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f24b0004700 tx=0x7f24b0012a50 comp rx=0 tx=0).stop 2023-10-21T16:15:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 shutdown_connections 2023-10-21T16:15:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:6833/3351749776,v1:172.21.15.53:6835/3351749776] conn(0x555db8982200 0x555db89825e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555db89b5af0 0x555db89a2770 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555db8984360 0x555db89a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 --2- 172.21.15.53:0/1089020389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555db8983f80 0x555db89ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 >> 172.21.15.53:0/1089020389 conn(0x555db89573c0 msgr2=0x555db8987440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24967fc640 0 client.6276 ms_handle_reset on v2:172.21.15.53:6833/3351749776 2023-10-21T16:15:22.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 shutdown_connections 2023-10-21T16:15:22.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.008+0000 7f24c07f5740 1 -- 172.21.15.53:0/1089020389 wait complete. 2023-10-21T16:15:22.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24c07f5740 20 client.6276 trim_cache size 0 max 16384 2023-10-21T16:15:22.034 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:15:22.035 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:15:22.035 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:15:22.035 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 6126, 2023-10-21T16:15:22.036 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:15:22.036 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:15:22.036 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:15:22.037 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8521927399999996, 2023-10-21T16:15:22.037 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 27, 2023-10-21T16:15:22.038 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 72, 2023-10-21T16:15:22.038 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 72, 2023-10-21T16:15:22.038 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.690896116999999 2023-10-21T16:15:22.039 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:15:22.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 msgr2=0x7f249806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 0x7f249806d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f24a80012e0 tx=0x7f24a8009040 comp rx=0 tx=0).stop 2023-10-21T16:15:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 msgr2=0x7f24b80c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f24a002bee0 tx=0x7f24a00046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 shutdown_connections 2023-10-21T16:15:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f249806afc0 0x7f249806d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b8160660 0x7f24b8151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24b80cb450 0x7f24b80c3a20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 --2- 172.21.15.53:0/2928519765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b8070cc0 0x7f24b81f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 >> 172.21.15.53:0/2928519765 conn(0x7f24b806a5d0 msgr2=0x7f24b80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 shutdown_connections 2023-10-21T16:15:22.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.009+0000 7f24bdb8a640 1 -- 172.21.15.53:0/2928519765 wait complete. 2023-10-21T16:15:22.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:15:22.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.281+0000 7fc5cb59e640 1 Processor -- start 2023-10-21T16:15:22.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.281+0000 7fc5cb59e640 1 -- start start 2023-10-21T16:15:22.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.282+0000 7fc5cb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc15a970 0x7fc5cc0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.282+0000 7fc5cb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5cc06dd00 0x7fc5cc0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.282+0000 7fc5cb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc06cfd0 con 0x7fc5cc0c9e60 2023-10-21T16:15:22.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.282+0000 7fc5cb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0caa00 con 0x7fc5cc06dd00 2023-10-21T16:15:22.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.282+0000 7fc5cb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc06d870 con 0x7fc5cc15a970 2023-10-21T16:15:22.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c9d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5cc06dd00 0x7fc5cc0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5cad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc5cc0c9e60 0x7fc5cc0ca2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51744/0 (socket says 172.21.15.53:51744) 2023-10-21T16:15:22.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5cad9d640 1 -- 172.21.15.53:0/991880511 learned_addr learned my addr 172.21.15.53:0/991880511 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:22.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5ca59c640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc15a970 0x7fc5cc0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c9d9b640 1 -- 172.21.15.53:0/991880511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc15a970 msgr2=0x7fc5cc0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c9d9b640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc15a970 0x7fc5cc0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c9d9b640 1 -- 172.21.15.53:0/991880511 >> v1:172.21.15.53:6789/0 conn(0x7fc5cc0c9e60 legacy=0x7fc5cc0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c959a640 1 -- 172.21.15.53:0/991880511 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447372362 0 0) 0x7fc5cc06cfd0 con 0x7fc5cc0c9e60 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.283+0000 7fc5c9d9b640 1 -- 172.21.15.53:0/991880511 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc15ad50 con 0x7fc5cc06dd00 2023-10-21T16:15:22.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5c9d9b640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc0cb990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c5b44ecc6e06fd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:22.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5c959a640 1 -- 172.21.15.53:0/991880511 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b402cb30 con 0x7fc5cc06dd00 2023-10-21T16:15:22.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5c959a640 1 -- 172.21.15.53:0/991880511 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b402cc90 con 0x7fc5cc06dd00 2023-10-21T16:15:22.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5c959a640 1 -- 172.21.15.53:0/991880511 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4034440 con 0x7fc5cc06dd00 2023-10-21T16:15:22.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5cb59e640 1 -- 172.21.15.53:0/991880511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 msgr2=0x7fc5cc0cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc0cb990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b402bd80 comp rx=0 tx=0).stop 2023-10-21T16:15:22.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5cb59e640 1 -- 172.21.15.53:0/991880511 shutdown_connections 2023-10-21T16:15:22.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc0cb990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.285+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/991880511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc15a970 0x7fc5cc0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.286+0000 7fc5cb59e640 1 -- 172.21.15.53:0/991880511 >> 172.21.15.53:0/991880511 conn(0x7fc5cc0ce9d0 msgr2=0x7fc5cc0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.286+0000 7fc5cb59e640 1 -- 172.21.15.53:0/991880511 shutdown_connections 2023-10-21T16:15:22.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.286+0000 7fc5cb59e640 1 -- 172.21.15.53:0/991880511 wait complete. 2023-10-21T16:15:22.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.288+0000 7fc5cb59e640 1 Processor -- start 2023-10-21T16:15:22.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.288+0000 7fc5cb59e640 1 -- start start 2023-10-21T16:15:22.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc0c9e60 0x7fc5cc1f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5cc15a970 0x7fc5cc070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc1f69c0 con 0x7fc5cc06dd00 2023-10-21T16:15:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cd240 con 0x7fc5cc15a970 2023-10-21T16:15:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c57b0 con 0x7fc5cc0c9e60 2023-10-21T16:15:22.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56514/0 (socket says 172.21.15.53:56514) 2023-10-21T16:15:22.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 -- 172.21.15.53:0/3348994887 learned_addr learned my addr 172.21.15.53:0/3348994887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:22.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5c9d9b640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc0c9e60 0x7fc5cc1f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 -- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc0c9e60 msgr2=0x7fc5cc1f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5cad9d640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5cc15a970 0x7fc5cc070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc0c9e60 0x7fc5cc1f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.289+0000 7fc5ca59c640 1 -- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5cc15a970 msgr2=0x7fc5cc070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5ca59c640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5cc15a970 0x7fc5cc070110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5ca59c640 1 -- 172.21.15.53:0/3348994887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009a70 con 0x7fc5cc06dd00 2023-10-21T16:15:22.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5ca59c640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d620 tx=0x7fc5bc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00b620 con 0x7fc5cc06dd00 2023-10-21T16:15:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc00b780 con 0x7fc5cc06dd00 2023-10-21T16:15:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc1f2ad0 con 0x7fc5cc06dd00 2023-10-21T16:15:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00bb30 con 0x7fc5cc06dd00 2023-10-21T16:15:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.290+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc1f3090 con 0x7fc5cc06dd00 2023-10-21T16:15:22.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.292+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc04e160 con 0x7fc5cc06dd00 2023-10-21T16:15:22.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.293+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc5bc00bc90 con 0x7fc5cc06dd00 2023-10-21T16:15:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.293+0000 7fc5c2ffd640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 0x7fc5ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.294+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc5bc089ed0 con 0x7fc5cc06dd00 2023-10-21T16:15:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.294+0000 7fc5c9d9b640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 0x7fc5ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.295+0000 7fc5c9d9b640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 0x7fc5ac06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4036000 tx=0x7fc5b4030000 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:22.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.298+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5bc054c60 con 0x7fc5cc06dd00 2023-10-21T16:15:22.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.532+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc5cc0cbcb0 con 0x7fc5cc06dd00 2023-10-21T16:15:22.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.533+0000 7fc5c2ffd640 1 -- 172.21.15.53:0/3348994887 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc5bc058280 con 0x7fc5cc06dd00 2023-10-21T16:15:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:15:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 msgr2=0x7fc5ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 0x7fc5ac06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4036000 tx=0x7fc5b4030000 comp rx=0 tx=0).stop 2023-10-21T16:15:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 msgr2=0x7fc5cc1f2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d620 tx=0x7fc5bc00dae0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 shutdown_connections 2023-10-21T16:15:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fc5ac06af90 0x7fc5ac06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5cc15a970 0x7fc5cc070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5cc0c9e60 0x7fc5cc1f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 --2- 172.21.15.53:0/3348994887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5cc06dd00 0x7fc5cc1f2480 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:22.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 >> 172.21.15.53:0/3348994887 conn(0x7fc5cc0ce9d0 msgr2=0x7fc5cc071830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 shutdown_connections 2023-10-21T16:15:22.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:22.536+0000 7fc5cb59e640 1 -- 172.21.15.53:0/3348994887 wait complete. 2023-10-21T16:15:22.559 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:15:22.559 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:15:22.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:15:22.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:15:22.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:15:22.561 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:15:22.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:15:22.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:15:22.562 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:15:22.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:22.563 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:15:22.617 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:15:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:15:22.617 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:15:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:15:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.618 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:15:22.618 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:15:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:15:22.619 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:15:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:15:22.619 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:15:22.619 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41571sec preferred_lft 41571sec 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:15:22.620 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.621 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:15:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:15:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:15:22.621 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:15:22.621 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:15:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:15:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:15:22.622 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.622 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:15:22.623 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:15:22.624 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:15:22.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:22.625 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:15:22.720 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:15:22.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:22.725 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:15:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:15:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:15:22.786 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:15:22.787 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:15:22.787 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:15:22.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:22.788 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:15:22.788 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:15:22.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:15:22.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:15:22.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:15:22.789 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:15:22.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:22.999 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:15:23.054 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:15:23.054 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:15:23.055 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:15:23.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:23.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:15:23.164 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:15:23.165 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:15:23.222 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:15:23.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:15:23.223 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:15:23.277 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:15:23.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:23.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:15:23.421 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:15:23.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:15:23.454 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:15:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:15:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:15:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:15:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:15:23.458 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:15:23.458 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:15:23.458 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:15:23.458 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:15:23.458 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:15:23.463 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:15:23.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:23.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:15:23.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:23.552 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:15:23.625 INFO:teuthology.orchestra.run.smithi053.stdout:client35144 2023-10-21T16:15:23.625 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:15:23.626 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:15:23.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:15:23.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.905+0000 7f350759e640 1 Processor -- start 2023-10-21T16:15:23.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.905+0000 7f350759e640 1 -- start start 2023-10-21T16:15:23.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.906+0000 7f350759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f350815cae0 0x7f35080cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35080c8b40 0x7f35080c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35080ccbc0 con 0x7f35080aff70 2023-10-21T16:15:23.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35080c9510 con 0x7f350815cae0 2023-10-21T16:15:23.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35080ce1f0 con 0x7f35080c8b40 2023-10-21T16:15:23.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f350815cae0 0x7f35080cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:23.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f350815cae0 0x7f35080cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56526/0 (socket says 172.21.15.53:56526) 2023-10-21T16:15:23.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 -- 172.21.15.53:0/1179255223 learned_addr learned my addr 172.21.15.53:0/1179255223 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:23.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 -- 172.21.15.53:0/1179255223 >> v2:172.21.15.124:3300/0 conn(0x7f35080c8b40 msgr2=0x7f35080c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:23.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 --2- 172.21.15.53:0/1179255223 >> v2:172.21.15.124:3300/0 conn(0x7f35080c8b40 0x7f35080c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:23.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 -- 172.21.15.53:0/1179255223 >> v1:172.21.15.124:6789/0 conn(0x7f35080aff70 legacy=0x7f35080b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:23.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.907+0000 7f350659c640 1 -- 172.21.15.53:0/1179255223 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080cd590 con 0x7f350815cae0 2023-10-21T16:15:23.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f350659c640 1 --2- 172.21.15.53:0/1179255223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f350815cae0 0x7f35080cd050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f35080cc010 tx=0x7f34f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c6188ba1d320c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:23.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f3505d9b640 1 -- 172.21.15.53:0/1179255223 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8031870 con 0x7f350815cae0 2023-10-21T16:15:23.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f3505d9b640 1 -- 172.21.15.53:0/1179255223 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f80319d0 con 0x7f350815cae0 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f3505d9b640 1 -- 172.21.15.53:0/1179255223 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8031d60 con 0x7f350815cae0 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f350759e640 1 -- 172.21.15.53:0/1179255223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f350815cae0 msgr2=0x7f35080cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.909+0000 7f350759e640 1 --2- 172.21.15.53:0/1179255223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f350815cae0 0x7f35080cd050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f35080cc010 tx=0x7f34f802b910 comp rx=0 tx=0).stop 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 -- 172.21.15.53:0/1179255223 shutdown_connections 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 --2- 172.21.15.53:0/1179255223 >> v2:172.21.15.124:3300/0 conn(0x7f35080c8b40 0x7f35080c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 --2- 172.21.15.53:0/1179255223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f350815cae0 0x7f35080cd050 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:23.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 -- 172.21.15.53:0/1179255223 >> 172.21.15.53:0/1179255223 conn(0x7f350806c050 msgr2=0x7f350806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:23.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 -- 172.21.15.53:0/1179255223 shutdown_connections 2023-10-21T16:15:23.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.910+0000 7f350759e640 1 -- 172.21.15.53:0/1179255223 wait complete. 2023-10-21T16:15:23.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.912+0000 7f350759e640 1 Processor -- start 2023-10-21T16:15:23.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.912+0000 7f350759e640 1 -- start start 2023-10-21T16:15:23.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35080aff70 0x7f350815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f350815cae0 0x7f35080c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35080c4520 con 0x7f35080c8b40 2023-10-21T16:15:23.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350815da70 con 0x7f350815cae0 2023-10-21T16:15:23.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350815dd50 con 0x7f35080aff70 2023-10-21T16:15:23.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f34fdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f34fdd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56534/0 (socket says 172.21.15.53:56534) 2023-10-21T16:15:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f34fdd9b640 1 -- 172.21.15.53:0/2772130851 learned_addr learned my addr 172.21.15.53:0/2772130851 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f3506d9d640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f350815cae0 0x7f35080c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.913+0000 7f350659c640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35080aff70 0x7f350815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 -- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35080aff70 msgr2=0x7f350815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35080aff70 0x7f350815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 -- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f350815cae0 msgr2=0x7f35080c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f350815cae0 0x7f35080c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 -- 172.21.15.53:0/2772130851 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f80094d0 con 0x7f35080c8b40 2023-10-21T16:15:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34fdd9b640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b4a0 tx=0x7f34f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0010610 con 0x7f35080c8b40 2023-10-21T16:15:23.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f0010770 con 0x7f35080c8b40 2023-10-21T16:15:23.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0010b20 con 0x7f35080c8b40 2023-10-21T16:15:23.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350815e030 con 0x7f35080c8b40 2023-10-21T16:15:23.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.914+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350815e7e0 con 0x7f35080c8b40 2023-10-21T16:15:23.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.916+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35080c2240 con 0x7f35080c8b40 2023-10-21T16:15:23.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.917+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f34f0010c80 con 0x7f35080c8b40 2023-10-21T16:15:23.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.918+0000 7f34ff7fe640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 0x7f34ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:23.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.918+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f34f00894e0 con 0x7f35080c8b40 2023-10-21T16:15:23.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.918+0000 7f350659c640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 0x7f34ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:23.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.920+0000 7f350659c640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 0x7f34ec06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f34f8036000 tx=0x7f34f803c040 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:23.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:23.922+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35080c2240 con 0x7f35080c8b40 2023-10-21T16:15:24.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.139+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 --> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f35080cc570 con 0x7f34ec06af90 2023-10-21T16:15:24.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.143+0000 7f34ff7fe640 1 -- 172.21.15.53:0/2772130851 <== mgr.34973 v2:172.21.15.124:6832/3288323666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f34f8031d10 con 0x7f34ec06af90 2023-10-21T16:15:24.145 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:15:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:15:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:15:24.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:15:24.145 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:15:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 msgr2=0x7f34ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 0x7f34ec06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f34f8036000 tx=0x7f34f803c040 comp rx=0 tx=0).stop 2023-10-21T16:15:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 msgr2=0x7f35081f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f34f000b4a0 tx=0x7f34f000b960 comp rx=0 tx=0).stop 2023-10-21T16:15:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 shutdown_connections 2023-10-21T16:15:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f34ec06af90 0x7f34ec06d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f350815cae0 0x7f35080c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35080c8b40 0x7f35081f1ad0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 --2- 172.21.15.53:0/2772130851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35080aff70 0x7f350815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 >> 172.21.15.53:0/2772130851 conn(0x7f350806c050 msgr2=0x7f35080b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 shutdown_connections 2023-10-21T16:15:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.146+0000 7f350759e640 1 -- 172.21.15.53:0/2772130851 wait complete. 2023-10-21T16:15:24.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:15:24.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.401+0000 7f002ea59640 1 Processor -- start 2023-10-21T16:15:24.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.401+0000 7f002ea59640 1 -- start start 2023-10-21T16:15:24.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002ea59640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f00280cb5e0 0x7f00280cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002ea59640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f00280b3c50 0x7f00280cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002ea59640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0028159db0 con 0x7f00281599d0 2023-10-21T16:15:24.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002ea59640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0028071fe0 con 0x7f00280cb5e0 2023-10-21T16:15:24.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002ea59640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00280b44c0 con 0x7f00280b3c50 2023-10-21T16:15:24.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002da57640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f00281599d0 0x7f00280caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41046/0 (socket says 172.21.15.53:41046) 2023-10-21T16:15:24.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002da57640 1 -- 172.21.15.53:0/860155691 learned_addr learned my addr 172.21.15.53:0/860155691 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:24.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.403+0000 7f002d256640 1 --2- 172.21.15.53:0/860155691 >> v2:172.21.15.53:3300/0 conn(0x7f00280cb5e0 0x7f00280cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002e258640 1 --2- 172.21.15.53:0/860155691 >> v2:172.21.15.124:3300/0 conn(0x7f00280b3c50 0x7f00280cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002d256640 1 -- 172.21.15.53:0/860155691 >> v2:172.21.15.124:3300/0 conn(0x7f00280b3c50 msgr2=0x7f00280cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:24.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002d256640 1 --2- 172.21.15.53:0/860155691 >> v2:172.21.15.124:3300/0 conn(0x7f00280b3c50 0x7f00280cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002d256640 1 -- 172.21.15.53:0/860155691 >> v1:172.21.15.53:6789/0 conn(0x7f00281599d0 legacy=0x7f00280caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002d256640 1 -- 172.21.15.53:0/860155691 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00280713d0 con 0x7f00280cb5e0 2023-10-21T16:15:24.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.404+0000 7f002ca55640 1 -- 172.21.15.53:0/860155691 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687110807 0 0) 0x7f0024001110 con 0x7f00281599d0 2023-10-21T16:15:24.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.405+0000 7f002d256640 1 --2- 172.21.15.53:0/860155691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280cdcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f00280cd830 tx=0x7f001802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8a813f6a2e4167 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:24.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ca55640 1 -- 172.21.15.53:0/860155691 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018031830 con 0x7f00280cb5e0 2023-10-21T16:15:24.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ca55640 1 -- 172.21.15.53:0/860155691 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018031990 con 0x7f00280cb5e0 2023-10-21T16:15:24.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ca55640 1 -- 172.21.15.53:0/860155691 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018031d20 con 0x7f00280cb5e0 2023-10-21T16:15:24.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 -- 172.21.15.53:0/860155691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 msgr2=0x7f00280cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 --2- 172.21.15.53:0/860155691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280cdcd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f00280cd830 tx=0x7f001802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 -- 172.21.15.53:0/860155691 shutdown_connections 2023-10-21T16:15:24.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 --2- 172.21.15.53:0/860155691 >> v2:172.21.15.124:3300/0 conn(0x7f00280b3c50 0x7f00280cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 --2- 172.21.15.53:0/860155691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280cdcd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 -- 172.21.15.53:0/860155691 >> 172.21.15.53:0/860155691 conn(0x7f002806c050 msgr2=0x7f002806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:24.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 -- 172.21.15.53:0/860155691 shutdown_connections 2023-10-21T16:15:24.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.406+0000 7f002ea59640 1 -- 172.21.15.53:0/860155691 wait complete. 2023-10-21T16:15:24.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.408+0000 7f002ea59640 1 Processor -- start 2023-10-21T16:15:24.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.408+0000 7f002ea59640 1 -- start start 2023-10-21T16:15:24.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00280b3c50 0x7f002814bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00281599d0 0x7f002814d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002814d040 con 0x7f00280cb5e0 2023-10-21T16:15:24.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00280a8e90 con 0x7f00280b3c50 2023-10-21T16:15:24.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.409+0000 7f002ea59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002814c3c0 con 0x7f00281599d0 2023-10-21T16:15:24.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43536/0 (socket says 172.21.15.53:43536) 2023-10-21T16:15:24.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 -- 172.21.15.53:0/1555396420 learned_addr learned my addr 172.21.15.53:0/1555396420 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:24.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002da57640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00280b3c50 0x7f002814bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002e258640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00281599d0 0x7f002814d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 -- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00281599d0 msgr2=0x7f002814d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00281599d0 0x7f002814d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 -- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00280b3c50 msgr2=0x7f002814bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00280b3c50 0x7f002814bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.410+0000 7f002d256640 1 -- 172.21.15.53:0/1555396420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018009400 con 0x7f00280cb5e0 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f002d256640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0018009890 tx=0x7f00180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001803ba70 con 0x7f00280cb5e0 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001803bbf0 con 0x7f00280cb5e0 2023-10-21T16:15:24.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002814c6a0 con 0x7f00280cb5e0 2023-10-21T16:15:24.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018045400 con 0x7f00280cb5e0 2023-10-21T16:15:24.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.411+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002814cc60 con 0x7f00280cb5e0 2023-10-21T16:15:24.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.412+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002814d180 con 0x7f00280cb5e0 2023-10-21T16:15:24.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.414+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f001804e440 con 0x7f00280cb5e0 2023-10-21T16:15:24.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.415+0000 7f001e7fc640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 0x7f000c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:24.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.415+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00180abc10 con 0x7f00280cb5e0 2023-10-21T16:15:24.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.415+0000 7f002da57640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 0x7f000c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:24.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.416+0000 7f002da57640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 0x7f000c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0024000f60 tx=0x7f00240079b0 comp rx=0 tx=0).ready entity=mgr.34973 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:24.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.419+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f002814d180 con 0x7f00280cb5e0 2023-10-21T16:15:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:24.665+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f002814d180 con 0x7f00280cb5e0 2023-10-21T16:15:25.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.288+0000 7f001e7fc640 1 -- 172.21.15.53:0/1555396420 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f002814d180 con 0x7f00280cb5e0 2023-10-21T16:15:25.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.292+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 msgr2=0x7f000c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.292+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 0x7f000c06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0024000f60 tx=0x7f00240079b0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.292+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 msgr2=0x7f00280a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.292+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0018009890 tx=0x7f00180046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 shutdown_connections 2023-10-21T16:15:25.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f000c06af90 0x7f000c06d450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00281599d0 0x7f002814d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00280cb5e0 0x7f00280a8950 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 --2- 172.21.15.53:0/1555396420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00280b3c50 0x7f002814bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 >> 172.21.15.53:0/1555396420 conn(0x7f002806c050 msgr2=0x7f002815a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:25.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 shutdown_connections 2023-10-21T16:15:25.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.293+0000 7f002ea59640 1 -- 172.21.15.53:0/1555396420 wait complete. 2023-10-21T16:15:25.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:15:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.539+0000 7f9d403e9640 1 Processor -- start 2023-10-21T16:15:25.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.539+0000 7f9d403e9640 1 -- start start 2023-10-21T16:15:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d403e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d38161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d403e9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380cdd90 con 0x7f9d380cd3f0 2023-10-21T16:15:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d403e9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380ce070 con 0x7f9d3815cc80 2023-10-21T16:15:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d403e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380d05c0 con 0x7f9d380b4d20 2023-10-21T16:15:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d3e986640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9d380cd3f0 0x7f9d380d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41066/0 (socket says 172.21.15.53:41066) 2023-10-21T16:15:25.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.541+0000 7f9d3e986640 1 -- 172.21.15.53:0/608830504 learned_addr learned my addr 172.21.15.53:0/608830504 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:25.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d984640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d38161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885414581 0 0) 0x7f9d380cdd90 con 0x7f9d380cd3f0 2023-10-21T16:15:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d24003520 con 0x7f9d380cd3f0 2023-10-21T16:15:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769197080 0 0) 0x7f9d24003520 con 0x7f9d380cd3f0 2023-10-21T16:15:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 msgr2=0x7f9d38161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d38161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 >> v1:172.21.15.124:6789/0 conn(0x7f9d3815cc80 legacy=0x7f9d380b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380d1030 con 0x7f9d380cd3f0 2023-10-21T16:15:25.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9d300019f0 con 0x7f9d380cd3f0 2023-10-21T16:15:25.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 >> v1:172.21.15.53:6789/0 conn(0x7f9d380cd3f0 legacy=0x7f9d380d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.542+0000 7f9d3d183640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d240039a0 0x7f9d24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d183640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d183640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2400bac0 con 0x7f9d240039a0 2023-10-21T16:15:25.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2400bc30 con 0x7f9d24004d00 2023-10-21T16:15:25.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2400bdf0 con 0x7f9d380b4d20 2023-10-21T16:15:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d240039a0 0x7f9d24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3d984640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 -- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 msgr2=0x7f9d240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380b4d20 0x7f9d240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e986640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 -- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d24004d00 msgr2=0x7f9d24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.543+0000 7f9d3e185640 1 -- 172.21.15.53:0/608830504 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d24003520 con 0x7f9d240039a0 2023-10-21T16:15:25.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.544+0000 7f9d3e986640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d24004d00 0x7f9d24005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.545+0000 7f9d3e185640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d240039a0 0x7f9d24003dc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9d34001720 tx=0x7f9d240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b666ccfb549ebb7d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.545+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d340052d0 con 0x7f9d240039a0 2023-10-21T16:15:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.545+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3402e050 con 0x7f9d240039a0 2023-10-21T16:15:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.545+0000 7f9d3d183640 1 -- 172.21.15.53:0/608830504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d340035f0 con 0x7f9d240039a0 2023-10-21T16:15:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 -- 172.21.15.53:0/608830504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d240039a0 msgr2=0x7f9d24003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 --2- 172.21.15.53:0/608830504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d240039a0 0x7f9d24003dc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9d34001720 tx=0x7f9d240057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d3e986640 1 -- 172.21.15.53:0/608830504 reap_dead start 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 -- 172.21.15.53:0/608830504 shutdown_connections 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 -- 172.21.15.53:0/608830504 >> 172.21.15.53:0/608830504 conn(0x7f9d380cf610 msgr2=0x7f9d380cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 -- 172.21.15.53:0/608830504 shutdown_connections 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.546+0000 7f9d403e9640 1 -- 172.21.15.53:0/608830504 wait complete. 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.548+0000 7f9d403e9640 1 Processor -- start 2023-10-21T16:15:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.548+0000 7f9d403e9640 1 -- start start 2023-10-21T16:15:25.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 0x7f9d380cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d381f2360 0x7f9d38150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380ccb20 con 0x7f9d380c66c0 2023-10-21T16:15:25.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380b4050 con 0x7f9d381f2360 2023-10-21T16:15:25.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d403e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d380c8b80 con 0x7f9d240039a0 2023-10-21T16:15:25.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d3d984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d3d984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43552/0 (socket says 172.21.15.53:43552) 2023-10-21T16:15:25.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.549+0000 7f9d3d984640 1 -- 172.21.15.53:0/2918554534 learned_addr learned my addr 172.21.15.53:0/2918554534 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:25.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3e185640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 0x7f9d380cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3e986640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d381f2360 0x7f9d38150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 msgr2=0x7f9d380cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 0x7f9d380cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d381f2360 msgr2=0x7f9d38150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d381f2360 0x7f9d38150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 -- 172.21.15.53:0/2918554534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380c9550 con 0x7f9d380c66c0 2023-10-21T16:15:25.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3e185640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 0x7f9d380cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:15:25.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.550+0000 7f9d3d984640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9d28011630 tx=0x7f9d28011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:25.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.551+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800f920 con 0x7f9d380c66c0 2023-10-21T16:15:25.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.551+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2800fa80 con 0x7f9d380c66c0 2023-10-21T16:15:25.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.551+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800fdf0 con 0x7f9d380c66c0 2023-10-21T16:15:25.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.551+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d380c6b40 con 0x7f9d380c66c0 2023-10-21T16:15:25.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.551+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d380c8250 con 0x7f9d380c66c0 2023-10-21T16:15:25.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.552+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3804e160 con 0x7f9d380c66c0 2023-10-21T16:15:25.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.553+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9d28015460 con 0x7f9d380c66c0 2023-10-21T16:15:25.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.555+0000 7f9d2effd640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 0x7f9d1c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.555+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d28089d80 con 0x7f9d380c66c0 2023-10-21T16:15:25.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.555+0000 7f9d3e185640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 msgr2=0x7f9d1c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:25.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.555+0000 7f9d3e185640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 0x7f9d1c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:15:25.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.560+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d28054bb0 con 0x7f9d380c66c0 2023-10-21T16:15:25.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.754+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9d381e2ae0 con 0x7f9d380c66c0 2023-10-21T16:15:25.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.756+0000 7f9d3e185640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 msgr2=0x7f9d1c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:25.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.756+0000 7f9d3e185640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 0x7f9d1c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:15:25.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.759+0000 7f9d2effd640 1 -- 172.21.15.53:0/2918554534 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9d280220d0 con 0x7f9d380c66c0 2023-10-21T16:15:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 msgr2=0x7f9d1c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:15:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 0x7f9d1c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 msgr2=0x7f9d380c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9d28011630 tx=0x7f9d28011af0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 shutdown_connections 2023-10-21T16:15:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f9d1c06afe0 0x7f9d1c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d381f2360 0x7f9d38150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d380c66c0 0x7f9d380c9010 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 --2- 172.21.15.53:0/2918554534 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d240039a0 0x7f9d380cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 >> 172.21.15.53:0/2918554534 conn(0x7f9d380cf610 msgr2=0x7f9d38155300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:25.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 shutdown_connections 2023-10-21T16:15:25.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.762+0000 7f9d403e9640 1 -- 172.21.15.53:0/2918554534 wait complete. 2023-10-21T16:15:25.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:15:25.820 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:25.818+0000 7f4b91445040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:15:25.843 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:25.841+0000 7f51c9e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:15:25.872 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:25.870+0000 7f4b91445040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:15:25.900 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:25.898+0000 7f51c9e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:15:25.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.984+0000 7f30221ef640 1 Processor -- start 2023-10-21T16:15:25.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.985+0000 7f30221ef640 1 -- start start 2023-10-21T16:15:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.986+0000 7f30221ef640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f301c0b3c50 0x7f301c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.986+0000 7f30221ef640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c159db0 con 0x7f301c0cb5e0 2023-10-21T16:15:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.986+0000 7f30221ef640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c071fe0 con 0x7f301c1599d0 2023-10-21T16:15:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.986+0000 7f30221ef640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c0b44c0 con 0x7f301c0b3c50 2023-10-21T16:15:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f30209ec640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f301c0cb5e0 0x7f301c0cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41070/0 (socket says 172.21.15.53:41070) 2023-10-21T16:15:25.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f30209ec640 1 -- 172.21.15.53:0/2123051882 learned_addr learned my addr 172.21.15.53:0/2123051882 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:25.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f30219ee640 1 --2- 172.21.15.53:0/2123051882 >> v2:172.21.15.124:3300/0 conn(0x7f301c0b3c50 0x7f301c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549620350 0 0) 0x7f3010001110 con 0x7f301c0cb5e0 2023-10-21T16:15:25.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3004003520 con 0x7f301c0cb5e0 2023-10-21T16:15:25.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798519131 0 0) 0x7f3004003520 con 0x7f301c0cb5e0 2023-10-21T16:15:25.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 >> v2:172.21.15.124:3300/0 conn(0x7f301c0b3c50 msgr2=0x7f301c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 --2- 172.21.15.53:0/2123051882 >> v2:172.21.15.124:3300/0 conn(0x7f301c0b3c50 0x7f301c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 >> v1:172.21.15.124:6789/0 conn(0x7f301c1599d0 legacy=0x7f301c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0713d0 con 0x7f301c0cb5e0 2023-10-21T16:15:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3010001800 con 0x7f301c0cb5e0 2023-10-21T16:15:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.987+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 >> v1:172.21.15.53:6789/0 conn(0x7f301c0cb5e0 legacy=0x7f301c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30040039c0 0x7f3004003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30040047c0 0x7f3004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 0x7f300400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0b44c0 con 0x7f3004005180 2023-10-21T16:15:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300400c080 con 0x7f30040047c0 2023-10-21T16:15:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300400c240 con 0x7f30040039c0 2023-10-21T16:15:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30219ee640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 0x7f300400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30209ec640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30040047c0 0x7f3004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30211ed640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30040039c0 0x7f3004003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30219ee640 1 -- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30040039c0 msgr2=0x7f3004003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30219ee640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30040039c0 0x7f3004003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30219ee640 1 -- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30040047c0 msgr2=0x7f3004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.988+0000 7f30219ee640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30040047c0 0x7f3004004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.989+0000 7f30219ee640 1 -- 172.21.15.53:0/2123051882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004003520 con 0x7f3004005180 2023-10-21T16:15:25.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.990+0000 7f30219ee640 1 -- 172.21.15.53:0/2123051882 reap_dead start 2023-10-21T16:15:25.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.990+0000 7f30219ee640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 0x7f300400b980 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fc0 tx=0x7f30040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faef9a2b253fadd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:25.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301402cc60 con 0x7f3004005180 2023-10-21T16:15:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301400f040 con 0x7f3004005180 2023-10-21T16:15:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f301bfff640 1 -- 172.21.15.53:0/2123051882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014003990 con 0x7f3004005180 2023-10-21T16:15:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 -- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 msgr2=0x7f300400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 0x7f300400b980 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fc0 tx=0x7f30040057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:25.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 -- 172.21.15.53:0/2123051882 shutdown_connections 2023-10-21T16:15:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 --2- 172.21.15.53:0/2123051882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3004005180 0x7f300400b980 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 -- 172.21.15.53:0/2123051882 >> 172.21.15.53:0/2123051882 conn(0x7f301c06c050 msgr2=0x7f301c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 -- 172.21.15.53:0/2123051882 shutdown_connections 2023-10-21T16:15:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.991+0000 7f30221ef640 1 -- 172.21.15.53:0/2123051882 wait complete. 2023-10-21T16:15:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.993+0000 7f30221ef640 1 Processor -- start 2023-10-21T16:15:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.993+0000 7f30221ef640 1 -- start start 2023-10-21T16:15:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3004005180 0x7f301c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f301c1f2b10 0x7f301c0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0aafc0 con 0x7f301c0aab40 2023-10-21T16:15:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0a8690 con 0x7f3004005180 2023-10-21T16:15:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30221ef640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0a8970 con 0x7f301c1f2b10 2023-10-21T16:15:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.994+0000 7f30209ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43580/0 (socket says 172.21.15.53:43580) 2023-10-21T16:15:26.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 -- 172.21.15.53:0/147243728 learned_addr learned my addr 172.21.15.53:0/147243728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:26.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30211ed640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3004005180 0x7f301c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30219ee640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f301c1f2b10 0x7f301c0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 -- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f301c1f2b10 msgr2=0x7f301c0ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f301c1f2b10 0x7f301c0ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 -- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3004005180 msgr2=0x7f301c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3004005180 0x7f301c150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 -- 172.21.15.53:0/147243728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0a8c50 con 0x7f301c0aab40 2023-10-21T16:15:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.995+0000 7f30209ec640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3010001050 tx=0x7f301000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.996+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000c690 con 0x7f301c0aab40 2023-10-21T16:15:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.996+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301000c7f0 con 0x7f301c0aab40 2023-10-21T16:15:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.996+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301000cba0 con 0x7f301c0aab40 2023-10-21T16:15:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.996+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301c0a8f30 con 0x7f301c0aab40 2023-10-21T16:15:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.996+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c0ad930 con 0x7f301c0aab40 2023-10-21T16:15:26.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.997+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c0adb50 con 0x7f301c0aab40 2023-10-21T16:15:26.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.998+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f301000cd00 con 0x7f301c0aab40 2023-10-21T16:15:26.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.999+0000 7f3019ffb640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 0x7f300806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.999+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f30100896e0 con 0x7f301c0aab40 2023-10-21T16:15:26.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.999+0000 7f30211ed640 1 -- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 msgr2=0x7f300806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:26.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:25.999+0000 7f30211ed640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 0x7f300806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:15:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.003+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3010054470 con 0x7f301c0aab40 2023-10-21T16:15:26.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.192+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f301c159db0 con 0x7f301c0aab40 2023-10-21T16:15:26.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.198+0000 7f3019ffb640 1 -- 172.21.15.53:0/147243728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f301c159db0 con 0x7f301c0aab40 2023-10-21T16:15:26.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 msgr2=0x7f300806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-21T16:15:26.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 0x7f300806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 msgr2=0x7f301c1f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3010001050 tx=0x7f301000ec10 comp rx=0 tx=0).stop 2023-10-21T16:15:26.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 shutdown_connections 2023-10-21T16:15:26.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7f300806afe0 0x7f300806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f301c1f2b10 0x7f301c0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f301c0aab40 0x7f301c1f25d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 --2- 172.21.15.53:0/147243728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3004005180 0x7f301c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 >> 172.21.15.53:0/147243728 conn(0x7f301c06c050 msgr2=0x7f301c0cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:26.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 shutdown_connections 2023-10-21T16:15:26.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.200+0000 7f30221ef640 1 -- 172.21.15.53:0/147243728 wait complete. 2023-10-21T16:15:26.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:26.220 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T16:15:26.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-21T16:15:26.321 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.319+0000 7f4b91445040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:15:26.375 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.373+0000 7f4b91445040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:15:26.388 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.387+0000 7f51c9e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:15:26.425 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.423+0000 7f4b91445040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:15:26.444 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.442+0000 7f51c9e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:15:26.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.478+0000 7fa9e7fdc640 1 Processor -- start 2023-10-21T16:15:26.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.479+0000 7fa9e7fdc640 1 -- start start 2023-10-21T16:15:26.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.480+0000 7fa9e7fdc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9e00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.480+0000 7fa9e7fdc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e00b1cc0 con 0x7fa9e006dfc0 2023-10-21T16:15:26.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.480+0000 7fa9e7fdc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e00cbad0 con 0x7fa9e00afbe0 2023-10-21T16:15:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.480+0000 7fa9e7fdc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00b2620 con 0x7fa9e00cb130 2023-10-21T16:15:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e5577640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa9e006dfc0 0x7fa9e0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41078/0 (socket says 172.21.15.53:41078) 2023-10-21T16:15:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e5577640 1 -- 172.21.15.53:0/2341271126 learned_addr learned my addr 172.21.15.53:0/2341271126 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:15:26.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e6579640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9e00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577785374 0 0) 0x7fa9e00b1cc0 con 0x7fa9e006dfc0 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9cc003520 con 0x7fa9e006dfc0 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.481+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2339982429 0 0) 0x7fa9cc003520 con 0x7fa9e006dfc0 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 msgr2=0x7fa9e00cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9e00cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 >> v1:172.21.15.124:6789/0 conn(0x7fa9e00afbe0 legacy=0x7fa9e00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00b2060 con 0x7fa9e006dfc0 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa9d0001670 con 0x7fa9e006dfc0 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 >> v1:172.21.15.53:6789/0 conn(0x7fa9e006dfc0 legacy=0x7fa9e0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9cc0039a0 0x7fa9cc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9cc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9cc005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00b2620 con 0x7fa9cc004350 2023-10-21T16:15:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc00b9b0 con 0x7fa9cc0039a0 2023-10-21T16:15:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc00bb70 con 0x7fa9e00cb130 2023-10-21T16:15:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e5577640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9cc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.482+0000 7fa9e5d78640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9cc0039a0 0x7fa9cc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e6579640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9cc005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e5577640 1 -- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 msgr2=0x7fa9cc005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e5577640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9cc005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e5577640 1 -- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9cc0039a0 msgr2=0x7fa9cc003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e5577640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9cc0039a0 0x7fa9cc003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e5577640 1 -- 172.21.15.53:0/2341271126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc003520 con 0x7fa9cc004350 2023-10-21T16:15:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.483+0000 7fa9e6579640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00cb130 0x7fa9cc005080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.484+0000 7fa9e5577640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9cc004820 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006e410 tx=0x7fa9cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e53bdaebe4b771 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.484+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0005ad0 con 0x7fa9cc004350 2023-10-21T16:15:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.484+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0005c50 con 0x7fa9cc004350 2023-10-21T16:15:26.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.484+0000 7fa9e4d76640 1 -- 172.21.15.53:0/2341271126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d002c890 con 0x7fa9cc004350 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/2341271126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 msgr2=0x7fa9cc004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 --2- 172.21.15.53:0/2341271126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9cc004820 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006e410 tx=0x7fa9cc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e6579640 1 -- 172.21.15.53:0/2341271126 reap_dead start 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/2341271126 shutdown_connections 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/2341271126 >> 172.21.15.53:0/2341271126 conn(0x7fa9e006a5d0 msgr2=0x7fa9e00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:26.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/2341271126 shutdown_connections 2023-10-21T16:15:26.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.485+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/2341271126 wait complete. 2023-10-21T16:15:26.493 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.484+0000 7f4b91445040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:15:26.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.487+0000 7fa9e7fdc640 1 Processor -- start 2023-10-21T16:15:26.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.487+0000 7fa9e7fdc640 1 -- start start 2023-10-21T16:15:26.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9e00c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00c68b0 0x7fa9e015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9e015cf90 0x7fa9e00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00c6420 con 0x7fa9cc004350 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00c7dd0 con 0x7fa9e015cf90 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e7fdc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015d4a0 con 0x7fa9e00c68b0 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e5d78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9e00c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e5d78640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9e00c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43598/0 (socket says 172.21.15.53:43598) 2023-10-21T16:15:26.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e5d78640 1 -- 172.21.15.53:0/3793238426 learned_addr learned my addr 172.21.15.53:0/3793238426 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:26.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00c68b0 0x7fa9e015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.488+0000 7fa9e5d78640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00c68b0 msgr2=0x7fa9e015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5d78640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00c68b0 0x7fa9e015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5d78640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9e015cf90 msgr2=0x7fa9e00c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e6579640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9e015cf90 0x7fa9e00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5d78640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9e015cf90 0x7fa9e00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5d78640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00c5af0 con 0x7fa9cc004350 2023-10-21T16:15:26.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9e00c68b0 0x7fa9e015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e5d78640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9e00c7890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00e5f0 tx=0x7fa9dc00eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0052e0 con 0x7fa9cc004350 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc005440 con 0x7fa9cc004350 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc003830 con 0x7fa9cc004350 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e00c5dd0 con 0x7fa9cc004350 2023-10-21T16:15:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.489+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e00c6d30 con 0x7fa9cc004350 2023-10-21T16:15:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.491+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e004e160 con 0x7fa9cc004350 2023-10-21T16:15:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.492+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa9dc003990 con 0x7fa9cc004350 2023-10-21T16:15:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.493+0000 7fa9d67fc640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:26.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.493+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa9dc089840 con 0x7fa9cc004350 2023-10-21T16:15:26.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.493+0000 7fa9e5577640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 msgr2=0x7fa9c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:26.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.493+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:15:26.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.498+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9dc054730 con 0x7fa9cc004350 2023-10-21T16:15:26.503 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.499+0000 7f51c9e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:15:26.536 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.534+0000 7f4b91445040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:15:26.566 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.564+0000 7f51c9e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:15:26.618 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.617+0000 7f51c9e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:15:26.647 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.645+0000 7f4b91445040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:15:26.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.694+0000 7fa9e5577640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 msgr2=0x7fa9c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:26.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.694+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:15:26.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:26.706+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e015d610 con 0x7fa9c406afe0 2023-10-21T16:15:26.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.706+0000 7f4b91445040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:15:26.735 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.734+0000 7f51c9e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:15:26.758 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.756+0000 7f4b91445040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:15:26.803 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.802+0000 7f51c9e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:15:26.814 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.812+0000 7f4b91445040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:15:26.855 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.854+0000 7f51c9e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:15:26.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:26.862+0000 7f4b91445040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:15:26.915 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.914+0000 7f51c9e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:15:26.968 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:26.966+0000 7f51c9e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:15:27.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:27.095+0000 7fa9e5577640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 msgr2=0x7fa9c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:27.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:27.095+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:15:27.241 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.239+0000 7f4b91445040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:15:27.371 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.370+0000 7f51c9e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:15:27.399 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.397+0000 7f4b91445040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:15:27.506 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.504+0000 7f4b91445040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:15:27.539 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.538+0000 7f51c9e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:15:27.557 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.555+0000 7f4b91445040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:15:27.653 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.651+0000 7f51c9e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:15:27.664 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.662+0000 7f4b91445040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:15:27.705 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.704+0000 7f51c9e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:15:27.770 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.768+0000 7f4b91445040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:15:27.818 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.817+0000 7f51c9e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:15:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:27.896+0000 7fa9e5577640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 msgr2=0x7fa9c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3288323666 2023-10-21T16:15:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:27.896+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:15:27.925 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.922+0000 7f4b91445040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:15:27.931 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:27.930+0000 7f51c9e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:15:27.974 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:27.972+0000 7f4b91445040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:15:28.091 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:28.089+0000 7f4b91445040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:15:28.097 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:28.096+0000 7f51c9e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:15:28.149 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:28.148+0000 7f51c9e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:15:28.275 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:28.274+0000 7f51c9e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:15:28.913 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:28.911+0000 7f4b91445040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:15:29.147 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:29.145+0000 7f51c9e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:15:29.204 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.201+0000 7f4b91445040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:15:29.260 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.258+0000 7f4b91445040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:15:29.374 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.372+0000 7f4b91445040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:15:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:29.394+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 20) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fa9dc087bf0 con 0x7fa9cc004350 2023-10-21T16:15:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:29.395+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 msgr2=0x7fa9c406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:15:29.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:29.395+0000 7fa9d67fc640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3288323666,v1:172.21.15.124:6833/3288323666] conn(0x7fa9c406afe0 0x7fa9c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:29.475 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:29.474+0000 7f51c9e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:15:29.541 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:29.539+0000 7f51c9e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:15:29.572 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b1071c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.572 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b1071c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.572 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b1071c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b1071c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b1071c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b0d6d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b0d6d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b0d6d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b0d6d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:15:29.570+0000 7f4b0d6d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:15:29.664 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:15:29.662+0000 7f51c9e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:15:30.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.407+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa9dc054b00 con 0x7fa9cc004350 2023-10-21T16:15:30.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.408+0000 7fa9d67fc640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fa9c4035880 0x7fa9c4075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.408+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 --> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9c4035c60 con 0x7fa9c4035880 2023-10-21T16:15:30.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.417+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fa9c4035880 0x7fa9c4075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:30.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.422+0000 7fa9e5577640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fa9c4035880 0x7fa9c4075440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0034000 tx=0x7fa9d0009030 comp rx=0 tx=0).ready entity=mgr.5644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:30.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.580+0000 7fa9d67fc640 1 -- 172.21.15.53:0/3793238426 <== mgr.5644 v2:172.21.15.124:6832/3738768716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa9c4035c60 con 0x7fa9c4035880 2023-10-21T16:15:30.582 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T16:15:30.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fa9c4035880 msgr2=0x7fa9c4075440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fa9c4035880 0x7fa9c4075440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0034000 tx=0x7fa9d0009030 comp rx=0 tx=0).stop 2023-10-21T16:15:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 msgr2=0x7fa9e00c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 --2- 172.21.15.53:0/3793238426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9cc004350 0x7fa9e00c7890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00e5f0 tx=0x7fa9dc00eab0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e6579640 1 -- 172.21.15.53:0/3793238426 reap_dead start 2023-10-21T16:15:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 shutdown_connections 2023-10-21T16:15:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 >> 172.21.15.53:0/3793238426 conn(0x7fa9e006a5d0 msgr2=0x7fa9e006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:30.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 shutdown_connections 2023-10-21T16:15:30.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.583+0000 7fa9e7fdc640 1 -- 172.21.15.53:0/3793238426 wait complete. 2023-10-21T16:15:30.605 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T16:15:30.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2023-10-21T16:15:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.823+0000 7f71e17e9640 1 Processor -- start 2023-10-21T16:15:30.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.823+0000 7f71e17e9640 1 -- start start 2023-10-21T16:15:30.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.824+0000 7f71e17e9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71dc15a820 0x7f71dc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.824+0000 7f71e17e9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f71dc15ac00 0x7f71dc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.824+0000 7f71e17e9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc0cab60 con 0x7f71dc0ca680 2023-10-21T16:15:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.824+0000 7f71e17e9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc0cb960 con 0x7f71dc15a820 2023-10-21T16:15:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.824+0000 7f71e17e9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc0cbdf0 con 0x7f71dc15ac00 2023-10-21T16:15:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71dc15a820 0x7f71dc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71dc15a820 0x7f71dc0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43602/0 (socket says 172.21.15.53:43602) 2023-10-21T16:15:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 -- 172.21.15.53:0/54705218 learned_addr learned my addr 172.21.15.53:0/54705218 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71db7fe640 1 --1- 172.21.15.53:0/54705218 >> v1:172.21.15.53:6789/0 conn(0x7f71dc0ca680 0x7f71dc0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41088/0 (socket says 172.21.15.53:41088) 2023-10-21T16:15:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 -- 172.21.15.53:0/54705218 >> v2:172.21.15.124:3300/0 conn(0x7f71dc15ac00 msgr2=0x7f71dc0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 --2- 172.21.15.53:0/54705218 >> v2:172.21.15.124:3300/0 conn(0x7f71dc15ac00 0x7f71dc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 -- 172.21.15.53:0/54705218 >> v1:172.21.15.53:6789/0 conn(0x7f71dc0ca680 legacy=0x7f71dc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.825+0000 7f71dbfff640 1 -- 172.21.15.53:0/54705218 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0cb030 con 0x7f71dc15a820 2023-10-21T16:15:30.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.826+0000 7f71dbfff640 1 --2- 172.21.15.53:0/54705218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc15a820 0x7f71dc0aff70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0e00 tx=0x7f71cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc78c24b866d87d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:30.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71daffd640 1 -- 172.21.15.53:0/54705218 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc031870 con 0x7f71dc15a820 2023-10-21T16:15:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71daffd640 1 -- 172.21.15.53:0/54705218 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc0319d0 con 0x7f71dc15a820 2023-10-21T16:15:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71daffd640 1 -- 172.21.15.53:0/54705218 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc031d60 con 0x7f71dc15a820 2023-10-21T16:15:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 -- 172.21.15.53:0/54705218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc15a820 msgr2=0x7f71dc0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 --2- 172.21.15.53:0/54705218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc15a820 0x7f71dc0aff70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0e00 tx=0x7f71cc02b910 comp rx=0 tx=0).stop 2023-10-21T16:15:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 -- 172.21.15.53:0/54705218 shutdown_connections 2023-10-21T16:15:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 --2- 172.21.15.53:0/54705218 >> v2:172.21.15.124:3300/0 conn(0x7f71dc15ac00 0x7f71dc0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 --2- 172.21.15.53:0/54705218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc15a820 0x7f71dc0aff70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 -- 172.21.15.53:0/54705218 >> 172.21.15.53:0/54705218 conn(0x7f71dc06c050 msgr2=0x7f71dc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 -- 172.21.15.53:0/54705218 shutdown_connections 2023-10-21T16:15:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.827+0000 7f71e17e9640 1 -- 172.21.15.53:0/54705218 wait complete. 2023-10-21T16:15:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.829+0000 7f71e17e9640 1 Processor -- start 2023-10-21T16:15:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.829+0000 7f71e17e9640 1 -- start start 2023-10-21T16:15:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71dc15a820 0x7f71dc0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71dc15ac00 0x7f71dc0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0b0300 con 0x7f71dc0ca680 2023-10-21T16:15:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0afa60 con 0x7f71dc15a820 2023-10-21T16:15:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71e17e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0afd40 con 0x7f71dc15ac00 2023-10-21T16:15:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43616/0 (socket says 172.21.15.53:43616) 2023-10-21T16:15:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.830+0000 7f71dbfff640 1 -- 172.21.15.53:0/251329056 learned_addr learned my addr 172.21.15.53:0/251329056 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 -- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71dc15ac00 msgr2=0x7f71dc0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71dc15ac00 0x7f71dc0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 -- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71dc15a820 msgr2=0x7f71dc0af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71dc15a820 0x7f71dc0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 -- 172.21.15.53:0/251329056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0094d0 con 0x7f71dc0ca680 2023-10-21T16:15:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71dbfff640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f71cc000c00 tx=0x7f71cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc03ad10 con 0x7f71dc0ca680 2023-10-21T16:15:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc03ae90 con 0x7f71dc0ca680 2023-10-21T16:15:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc044630 con 0x7f71dc0ca680 2023-10-21T16:15:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.831+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc0b0020 con 0x7f71dc0ca680 2023-10-21T16:15:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.832+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc0af710 con 0x7f71dc0ca680 2023-10-21T16:15:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.833+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc0c58a0 con 0x7f71dc0ca680 2023-10-21T16:15:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.833+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f71cc04d450 con 0x7f71dc0ca680 2023-10-21T16:15:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.834+0000 7f71d8ff9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 0x7f71c806d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.834+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f71cc02d040 con 0x7f71dc0ca680 2023-10-21T16:15:30.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.835+0000 7f71db7fe640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 0x7f71c806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:30.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.836+0000 7f71db7fe640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 0x7f71c806d220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71dc1592c0 tx=0x7f71d00079c0 comp rx=0 tx=0).ready entity=mgr.5644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:30.839+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71cc076e50 con 0x7f71dc0ca680 2023-10-21T16:15:31.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.056+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 --> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) v1 -- 0x7f71dc0c29c0 con 0x7f71c806ad60 2023-10-21T16:15:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.195+0000 7f71d8ff9640 1 -- 172.21.15.53:0/251329056 <== mgr.5644 v2:172.21.15.124:6832/3738768716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71dc0c29c0 con 0x7f71c806ad60 2023-10-21T16:15:31.197 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T16:15:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 msgr2=0x7f71c806d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 0x7f71c806d220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71dc1592c0 tx=0x7f71d00079c0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 msgr2=0x7f71dc0aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f71cc000c00 tx=0x7f71cc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 shutdown_connections 2023-10-21T16:15:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f71c806ad60 0x7f71c806d220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71dc15ac00 0x7f71dc0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71dc15a820 0x7f71dc0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 --2- 172.21.15.53:0/251329056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71dc0ca680 0x7f71dc0aec90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 >> 172.21.15.53:0/251329056 conn(0x7f71dc06c050 msgr2=0x7f71dc159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 shutdown_connections 2023-10-21T16:15:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.199+0000 7f71e17e9640 1 -- 172.21.15.53:0/251329056 wait complete. 2023-10-21T16:15:31.224 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T16:15:31.224 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-21-16_16_00 in ~29s... 2023-10-21T16:15:31.225 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-21-16_17_00 in ~89s... 2023-10-21T16:15:31.225 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2023-10-21T16:15:31.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-21T16:15:31.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.452+0000 7fcd4afb3640 1 Processor -- start 2023-10-21T16:15:31.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.452+0000 7fcd4afb3640 1 -- start start 2023-10-21T16:15:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.454+0000 7fcd4afb3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd440ca680 0x7fcd4415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.454+0000 7fcd4afb3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440cab60 con 0x7fcd440b0680 2023-10-21T16:15:31.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.454+0000 7fcd4afb3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440cb960 con 0x7fcd4415a820 2023-10-21T16:15:31.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.454+0000 7fcd4afb3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440cbdf0 con 0x7fcd440ca680 2023-10-21T16:15:31.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.454+0000 7fcd4a7b2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd440ca680 0x7fcd4415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.455+0000 7fcd4a7b2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd440ca680 0x7fcd4415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43618/0 (socket says 172.21.15.53:43618) 2023-10-21T16:15:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.455+0000 7fcd4a7b2640 1 -- 172.21.15.53:0/3978124097 learned_addr learned my addr 172.21.15.53:0/3978124097 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:31.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.455+0000 7fcd4a7b2640 1 -- 172.21.15.53:0/3978124097 >> v1:172.21.15.124:6789/0 conn(0x7fcd4415a820 legacy=0x7fcd440aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:15:31.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.455+0000 7fcd4a7b2640 1 -- 172.21.15.53:0/3978124097 >> v1:172.21.15.53:6789/0 conn(0x7fcd440b0680 legacy=0x7fcd4415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.455+0000 7fcd4a7b2640 1 -- 172.21.15.53:0/3978124097 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440cb030 con 0x7fcd440ca680 2023-10-21T16:15:31.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.456+0000 7fcd4a7b2640 1 --2- 172.21.15.53:0/3978124097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4415ec50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a1c6abb3708c4f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:31.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd48faf640 1 -- 172.21.15.53:0/3978124097 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c02cb10 con 0x7fcd440ca680 2023-10-21T16:15:31.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd48faf640 1 -- 172.21.15.53:0/3978124097 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c02cc70 con 0x7fcd440ca680 2023-10-21T16:15:31.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd48faf640 1 -- 172.21.15.53:0/3978124097 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0343f0 con 0x7fcd440ca680 2023-10-21T16:15:31.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 -- 172.21.15.53:0/3978124097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 msgr2=0x7fcd4415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/3978124097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4415ec50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009e50 tx=0x7fcd3c02bf70 comp rx=0 tx=0).stop 2023-10-21T16:15:31.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 -- 172.21.15.53:0/3978124097 shutdown_connections 2023-10-21T16:15:31.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/3978124097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4415ec50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 -- 172.21.15.53:0/3978124097 >> 172.21.15.53:0/3978124097 conn(0x7fcd4406c050 msgr2=0x7fcd4406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:31.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 -- 172.21.15.53:0/3978124097 shutdown_connections 2023-10-21T16:15:31.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.457+0000 7fcd4afb3640 1 -- 172.21.15.53:0/3978124097 wait complete. 2023-10-21T16:15:31.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.459+0000 7fcd4afb3640 1 Processor -- start 2023-10-21T16:15:31.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.459+0000 7fcd4afb3640 1 -- start start 2023-10-21T16:15:31.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd440b0680 0x7fcd440cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:31.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:31.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd4415a820 0x7fcd440ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:31.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440ace20 con 0x7fcd440ca680 2023-10-21T16:15:31.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440ad240 con 0x7fcd4415a820 2023-10-21T16:15:31.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.460+0000 7fcd4afb3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440cca20 con 0x7fcd440b0680 2023-10-21T16:15:31.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:31.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43632/0 (socket says 172.21.15.53:43632) 2023-10-21T16:15:31.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 -- 172.21.15.53:0/2384568751 learned_addr learned my addr 172.21.15.53:0/2384568751 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:15:31.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd49fb1640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd440b0680 0x7fcd440cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:31.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd4a7b2640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd4415a820 0x7fcd440ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 -- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd440b0680 msgr2=0x7fcd440cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd440b0680 0x7fcd440cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 -- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd4415a820 msgr2=0x7fcd440ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd4415a820 0x7fcd440ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.461+0000 7fcd497b0640 1 -- 172.21.15.53:0/2384568751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c009b30 con 0x7fcd440ca680 2023-10-21T16:15:31.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd497b0640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c480 tx=0x7fcd3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:31.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340052d0 con 0x7fcd440ca680 2023-10-21T16:15:31.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34005430 con 0x7fcd440ca680 2023-10-21T16:15:31.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34012870 con 0x7fcd440ca680 2023-10-21T16:15:31.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd440c7d00 con 0x7fcd440ca680 2023-10-21T16:15:31.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.462+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd440aca40 con 0x7fcd440ca680 2023-10-21T16:15:31.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.463+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd44074150 con 0x7fcd440ca680 2023-10-21T16:15:31.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.464+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fcd340129d0 con 0x7fcd440ca680 2023-10-21T16:15:31.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.465+0000 7fcd3a7fc640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 0x7fcd2806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:15:31.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.465+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fcd340891c0 con 0x7fcd440ca680 2023-10-21T16:15:31.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.466+0000 7fcd49fb1640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 0x7fcd2806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:15:31.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.467+0000 7fcd49fb1640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 0x7fcd2806d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000f60 tx=0x7fcd40009040 comp rx=0 tx=0).ready entity=mgr.5644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:15:31.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.470+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd34054210 con 0x7fcd440ca680 2023-10-21T16:15:31.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.687+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 --> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd440cb310 con 0x7fcd2806ad10 2023-10-21T16:15:31.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.705+0000 7fcd3a7fc640 1 -- 172.21.15.53:0/2384568751 <== mgr.5644 v2:172.21.15.124:6832/3738768716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fcd400012c0 con 0x7fcd2806ad10 2023-10-21T16:15:31.706 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:15:31.707 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-21T16:15:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 msgr2=0x7fcd2806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 0x7fcd2806d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000f60 tx=0x7fcd40009040 comp rx=0 tx=0).stop 2023-10-21T16:15:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 msgr2=0x7fcd4414f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:15:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c480 tx=0x7fcd3400c940 comp rx=0 tx=0).stop 2023-10-21T16:15:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 shutdown_connections 2023-10-21T16:15:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fcd2806ad10 0x7fcd2806d1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd4415a820 0x7fcd440ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd440ca680 0x7fcd4414f2e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 --2- 172.21.15.53:0/2384568751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd440b0680 0x7fcd440cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:15:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.707+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 >> 172.21.15.53:0/2384568751 conn(0x7fcd4406c050 msgr2=0x7fcd440af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:15:31.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.708+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 shutdown_connections 2023-10-21T16:15:31.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:15:31.708+0000 7fcd4afb3640 1 -- 172.21.15.53:0/2384568751 wait complete. 2023-10-21T16:15:31.728 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-21T16:15:31.729 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2023-10-21T16:15:31.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:31.730 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:31.785 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:31.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:31.786 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:32.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:32.788 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:32.855 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:32.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:32.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:33.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:33.858 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:33.927 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:33.927 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:33.928 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:34.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:34.930 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:34.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:34.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:34.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:36.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:36.001 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:36.067 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:36.068 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:36.068 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:37.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:37.069 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:37.137 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:37.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:37.138 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:38.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:38.139 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:38.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:38.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:38.206 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:39.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:39.208 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:39.275 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:39.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:39.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:40.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:40.277 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:40.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:40.347 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:40.347 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:41.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:41.349 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:41.415 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:41.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:41.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:42.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:42.418 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:42.485 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:42.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:42.486 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:43.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:43.488 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:43.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:43.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:43.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:44.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:44.557 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:44.625 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:44.626 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:44.626 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:45.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:45.628 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:45.696 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:45.696 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:45.697 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:46.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:46.698 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:46.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:46.766 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:46.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:47.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:47.768 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:47.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:47.834 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:47.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:48.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:48.836 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:48.904 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:48.904 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:48.905 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:49.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:49.906 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:49.972 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:49.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:49.973 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:50.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:15:50.339 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:15:50.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:50.975 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:51.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:51.043 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:51.044 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:52.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:52.045 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:52.114 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:52.114 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:52.115 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:53.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:53.116 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:53.182 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:53.183 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:53.183 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:54.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:54.184 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:54.250 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:54.251 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:54.251 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:55.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:55.254 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:55.323 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:55.324 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:55.324 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:56.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:56.327 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:56.396 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:56.397 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:56.397 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:57.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:57.398 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:57.465 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:57.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:57.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:58.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:58.467 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:58.536 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:58.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:58.536 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:15:59.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:15:59.538 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:15:59.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:15:59.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:15:59.606 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:00.221 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:16:00.219+0000 7f4affebb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:16:00.221 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:16:00.219+0000 7f4affebb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:16:00.221 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:16:00.219+0000 7f4affebb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:16:00.221 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:16:00.219+0000 7f4affebb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:16:00.222 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:16:00.219+0000 7f4affebb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:16:00.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:00.609 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:00.675 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:00.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:00.677 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:00.677 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:00.677 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-21-16_16_00_UTC'] 2023-10-21T16:16:00.678 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697904931.2247927, now = 1697904960.6782997, timo = 29 2023-10-21T16:16:01.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:01.680 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:01.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:01.747 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:01.748 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:01.748 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:02.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:02.750 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:02.817 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:02.818 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:02.818 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:02.819 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:03.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:03.821 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:03.888 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:03.890 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:03.890 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:03.891 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:04.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:04.893 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:04.959 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:04.960 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:04.961 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:04.961 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:05.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:05.963 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:06.029 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:06.031 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:06.031 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:06.031 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:07.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:07.033 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:07.097 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:07.099 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:07.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:07.100 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:08.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:08.101 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:08.168 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:08.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:08.170 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:08.171 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:09.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:09.172 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:09.238 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:09.240 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:09.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:09.241 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:10.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:10.243 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:10.308 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:10.310 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:10.310 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:10.311 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:11.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:11.313 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:11.378 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:11.379 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:11.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:11.380 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:12.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:12.382 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:12.447 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:12.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:12.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:12.449 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:13.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:13.451 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:13.522 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:13.523 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:13.524 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:13.524 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:14.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:14.526 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:14.592 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:14.593 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:14.594 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:14.594 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:15.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:15.599 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:15.663 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:15.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:15.665 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:15.665 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:16.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:16.667 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:16.734 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:16.736 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:16.736 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:16.736 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:17.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:17.738 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:17.804 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:17.805 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:17.805 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:17.805 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:18.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:18.808 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:18.873 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:18.874 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:18.875 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:18.875 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:19.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:19.877 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:19.941 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:19.943 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:19.943 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:19.943 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:20.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:16:20.512 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:16:20.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:20.945 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:21.009 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:21.011 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:21.011 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:21.012 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:22.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:22.014 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:22.079 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:22.080 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:22.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:22.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:23.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:23.083 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:23.150 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:23.151 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:23.152 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:23.152 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:24.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:24.154 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:24.222 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:24.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:24.223 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:24.223 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:25.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:25.225 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:25.291 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:25.292 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:25.293 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:25.293 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:26.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:26.295 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:26.360 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:26.361 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:26.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:26.362 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:27.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:27.363 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:27.429 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:27.431 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:27.431 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:27.431 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:28.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:28.433 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:28.499 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:28.501 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:28.501 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:28.502 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:29.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:29.503 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:29.568 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:29.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:29.570 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:29.570 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:30.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:30.572 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:30.638 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:30.639 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:30.640 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:30.640 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:31.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:31.642 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:31.710 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:31.711 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:31.712 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:31.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:32.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:32.713 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:32.780 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:32.781 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:32.781 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:32.782 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:33.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:33.783 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:33.849 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:33.850 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:33.851 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:33.851 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:34.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:34.853 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:34.920 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:34.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:34.922 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:34.922 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:35.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:35.925 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:35.992 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:35.993 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:35.993 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:35.994 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:36.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:36.996 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:37.063 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:37.064 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:37.064 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:37.065 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:38.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:38.067 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:38.130 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:38.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:38.132 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:38.132 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:39.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:39.135 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:39.202 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:39.203 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:39.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:39.204 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:40.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:40.205 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:40.271 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:40.272 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:40.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:40.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:41.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:41.274 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:41.341 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:41.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:41.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:41.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:42.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:42.345 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:42.413 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:42.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:42.414 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:42.415 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:43.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:43.416 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:43.481 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:43.483 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:43.484 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:43.485 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:44.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:44.487 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:44.553 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:44.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:44.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:44.556 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:45.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:45.558 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:45.624 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:45.625 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:45.625 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:45.626 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:46.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:46.628 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:46.694 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:46.696 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:46.696 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:46.696 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:47.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:47.698 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:47.763 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:47.764 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:47.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:47.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:48.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:48.766 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:48.831 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:48.833 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:48.833 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:48.833 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:49.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:49.835 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:49.902 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:49.903 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:49.903 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:49.904 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:50.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:16:50.652 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:16:50.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:50.906 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:50.971 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:50.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:50.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:50.974 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:51.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:51.975 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:52.040 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:52.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:52.042 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:52.042 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:53.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:53.044 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:53.111 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:53.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:53.113 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:53.113 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:54.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:54.115 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:54.182 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:54.183 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:54.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:54.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:55.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:55.186 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:55.253 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:55.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:55.255 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:55.255 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:56.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:56.256 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:56.322 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:56.324 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:56.324 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:56.324 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:57.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:57.326 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:57.390 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:57.392 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:57.392 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:57.393 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:58.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:58.394 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:58.459 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:58.461 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:58.462 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:58.462 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:16:59.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:16:59.464 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:16:59.529 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:16:59.531 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC'} 2023-10-21T16:16:59.531 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:16:59.531 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:17:00.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:00.533 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:17:00.596 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:17:00.597 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_17_00_UTC 2023-10-21T16:17:00.598 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_16_00_UTC', 'scheduled-2023-10-21-16_17_00_UTC'} 2023-10-21T16:17:00.598 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-21-16_17_00_UTC'} 2023-10-21T16:17:00.598 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:17:00.599 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-21-16_17_00_UTC'] 2023-10-21T16:17:00.599 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697904931.2247927, now = 1697905020.5995314, timo = 89 2023-10-21T16:17:01.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-21T16:17:01.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.840+0000 7f3549789640 1 Processor -- start 2023-10-21T16:17:01.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.840+0000 7f3549789640 1 -- start start 2023-10-21T16:17:01.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f3549789640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 0x7f35440b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f3549789640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35440cb130 0x7f35440cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f3549789640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35440b2620 con 0x7f354406dfc0 2023-10-21T16:17:01.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f3549789640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35440cbae0 con 0x7f35440cb130 2023-10-21T16:17:01.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f3549789640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35440b2060 con 0x7f35440afbe0 2023-10-21T16:17:01.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.842+0000 7f35437fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35440cb130 0x7f35440cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35440cb130 0x7f35440cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59180/0 (socket says 172.21.15.53:59180) 2023-10-21T16:17:01.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 -- 172.21.15.53:0/3734143339 learned_addr learned my addr 172.21.15.53:0/3734143339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:01.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f3542ffd640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 0x7f35440b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 -- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 msgr2=0x7f35440b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:01.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 0x7f35440b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:01.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 -- 172.21.15.53:0/3734143339 >> v1:172.21.15.124:6789/0 conn(0x7f354406dfc0 legacy=0x7f3544070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:01.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f35437fe640 1 -- 172.21.15.53:0/3734143339 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35440b2340 con 0x7f35440cb130 2023-10-21T16:17:01.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f3541ffb640 1 -- 172.21.15.53:0/3734143339 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190560249 0 0) 0x7f35440b2620 con 0x7f354406dfc0 2023-10-21T16:17:01.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.843+0000 7f3542ffd640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 0x7f35440b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:01.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.845+0000 7f35437fe640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440cb130 0x7f35440cb5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e50 tx=0x7f353c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfc276d07105dba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:01.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.845+0000 7f3541ffb640 1 -- 172.21.15.53:0/3734143339 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c02cb00 con 0x7f35440cb130 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.845+0000 7f3541ffb640 1 -- 172.21.15.53:0/3734143339 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c02cc60 con 0x7f35440cb130 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.845+0000 7f3541ffb640 1 -- 172.21.15.53:0/3734143339 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0343f0 con 0x7f35440cb130 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 -- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440cb130 msgr2=0x7f35440cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440cb130 0x7f35440cb5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e50 tx=0x7f353c02bf60 comp rx=0 tx=0).stop 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 -- 172.21.15.53:0/3734143339 shutdown_connections 2023-10-21T16:17:01.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440cb130 0x7f35440cb5a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:01.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 --2- 172.21.15.53:0/3734143339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35440afbe0 0x7f35440b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:01.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 -- 172.21.15.53:0/3734143339 >> 172.21.15.53:0/3734143339 conn(0x7f354406a5d0 msgr2=0x7f35440b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:01.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 -- 172.21.15.53:0/3734143339 shutdown_connections 2023-10-21T16:17:01.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.846+0000 7f3549789640 1 -- 172.21.15.53:0/3734143339 wait complete. 2023-10-21T16:17:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.848+0000 7f3549789640 1 Processor -- start 2023-10-21T16:17:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.848+0000 7f3549789640 1 -- start start 2023-10-21T16:17:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f354406dfc0 0x7f35440ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440afbe0 0x7f3544155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35440cdcd0 con 0x7f35440cb130 2023-10-21T16:17:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3544156300 con 0x7f35440afbe0 2023-10-21T16:17:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.849+0000 7f3549789640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35440cd0c0 con 0x7f354406dfc0 2023-10-21T16:17:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33628/0 (socket says 172.21.15.53:33628) 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 -- 172.21.15.53:0/1218622926 learned_addr learned my addr 172.21.15.53:0/1218622926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35427fc640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440afbe0 0x7f3544155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f3542ffd640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f354406dfc0 0x7f35440ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 -- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f354406dfc0 msgr2=0x7f35440ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f354406dfc0 0x7f35440ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:01.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 -- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440afbe0 msgr2=0x7f3544155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:01.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440afbe0 0x7f3544155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:01.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.850+0000 7f35437fe640 1 -- 172.21.15.53:0/1218622926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534009400 con 0x7f35440cb130 2023-10-21T16:17:01.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f35437fe640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e50 tx=0x7f353c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:01.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c009510 con 0x7f35440cb130 2023-10-21T16:17:01.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c03e450 con 0x7f35440cb130 2023-10-21T16:17:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c009b30 con 0x7f35440cb130 2023-10-21T16:17:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c03e800 con 0x7f35440cb130 2023-10-21T16:17:01.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.851+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35440cd680 con 0x7f35440cb130 2023-10-21T16:17:01.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.852+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35440cde40 con 0x7f35440cb130 2023-10-21T16:17:01.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.853+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f353c03e960 con 0x7f35440cb130 2023-10-21T16:17:01.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.854+0000 7f352b7fe640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 0x7f353006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:01.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.854+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f353c0acad0 con 0x7f35440cb130 2023-10-21T16:17:01.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.855+0000 7f3542ffd640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 0x7f353006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:01.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.856+0000 7f3542ffd640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 0x7f353006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3534009bc0 tx=0x7f3534008040 comp rx=0 tx=0).ready entity=mgr.5644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:01.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:01.859+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f353c077510 con 0x7f35440cb130 2023-10-21T16:17:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.072+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 --> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3544070630 con 0x7f353006b100 2023-10-21T16:17:02.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.236+0000 7f352b7fe640 1 -- 172.21.15.53:0/1218622926 <== mgr.5644 v2:172.21.15.124:6832/3738768716 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3534011ac0 con 0x7f353006b100 2023-10-21T16:17:02.239 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path snap_test_dir1 2023-10-21T16:17:02.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 msgr2=0x7f353006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 0x7f353006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3534009bc0 tx=0x7f3534008040 comp rx=0 tx=0).stop 2023-10-21T16:17:02.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 msgr2=0x7f35440cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f353c009e50 tx=0x7f353c004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:02.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 shutdown_connections 2023-10-21T16:17:02.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f353006b100 0x7f353006d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35440cb130 0x7f35440cc640 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35440afbe0 0x7f3544155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 --2- 172.21.15.53:0/1218622926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f354406dfc0 0x7f35440ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 >> 172.21.15.53:0/1218622926 conn(0x7f354406a5d0 msgr2=0x7f35440c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:02.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 shutdown_connections 2023-10-21T16:17:02.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.240+0000 7f3549789640 1 -- 172.21.15.53:0/1218622926 wait complete. 2023-10-21T16:17:02.265 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-21T16:17:02.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:02.266 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:17:02.322 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:17:02.323 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_17_00_UTC 2023-10-21T16:17:02.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_16_00_UTC 2023-10-21T16:17:02.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:02.325 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_16_00_UTC) 2023-10-21T16:17:02.384 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_17_00_UTC 2023-10-21T16:17:02.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:02.385 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_17_00_UTC) 2023-10-21T16:17:02.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:02.454 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T16:17:02.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:17:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.752+0000 7f91ffd83640 1 Processor -- start 2023-10-21T16:17:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.752+0000 7f91ffd83640 1 -- start start 2023-10-21T16:17:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.753+0000 7f91ffd83640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91f80afbe0 0x7f91f80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91ffd83640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f80b2620 con 0x7f91f806dfc0 2023-10-21T16:17:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91ffd83640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f80b2060 con 0x7f91f80cb130 2023-10-21T16:17:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91ffd83640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f80b2340 con 0x7f91f80afbe0 2023-10-21T16:17:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91f80afbe0 0x7f91f80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91f80afbe0 0x7f91f80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33630/0 (socket says 172.21.15.53:33630) 2023-10-21T16:17:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 -- 172.21.15.53:0/3185491353 learned_addr learned my addr 172.21.15.53:0/3185491353 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 -- 172.21.15.53:0/3185491353 >> v1:172.21.15.124:6789/0 conn(0x7f91f80cb130 legacy=0x7f91f806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 -- 172.21.15.53:0/3185491353 >> v1:172.21.15.53:6789/0 conn(0x7f91f806dfc0 legacy=0x7f91f8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.754+0000 7f91fdb1f640 1 -- 172.21.15.53:0/3185491353 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8070350 con 0x7f91f80afbe0 2023-10-21T16:17:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.755+0000 7f91fdb1f640 1 --2- 172.21.15.53:0/3185491353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f80afbe0 0x7f91f80b15b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396dcb57c8cb8da5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91fcb1d640 1 -- 172.21.15.53:0/3185491353 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4030850 con 0x7f91f80afbe0 2023-10-21T16:17:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91fcb1d640 1 -- 172.21.15.53:0/3185491353 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f40309b0 con 0x7f91f80afbe0 2023-10-21T16:17:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91fcb1d640 1 -- 172.21.15.53:0/3185491353 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4030d40 con 0x7f91f80afbe0 2023-10-21T16:17:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91ffd83640 1 -- 172.21.15.53:0/3185491353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f80afbe0 msgr2=0x7f91f80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91ffd83640 1 --2- 172.21.15.53:0/3185491353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f80afbe0 0x7f91f80b15b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f402b990 comp rx=0 tx=0).stop 2023-10-21T16:17:02.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91ffd83640 1 -- 172.21.15.53:0/3185491353 shutdown_connections 2023-10-21T16:17:02.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.756+0000 7f91ffd83640 1 --2- 172.21.15.53:0/3185491353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f80afbe0 0x7f91f80b15b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.757+0000 7f91ffd83640 1 -- 172.21.15.53:0/3185491353 >> 172.21.15.53:0/3185491353 conn(0x7f91f806a5d0 msgr2=0x7f91f80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:02.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.757+0000 7f91ffd83640 1 -- 172.21.15.53:0/3185491353 shutdown_connections 2023-10-21T16:17:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.757+0000 7f91ffd83640 1 -- 172.21.15.53:0/3185491353 wait complete. 2023-10-21T16:17:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.758+0000 7f91ffd83640 1 Processor -- start 2023-10-21T16:17:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.758+0000 7f91ffd83640 1 -- start start 2023-10-21T16:17:02.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:02.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91f80afbe0 0x7f91f814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:02.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91f80cb130 0x7f91f81f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:02.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81f4060 con 0x7f91f806dfc0 2023-10-21T16:17:02.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f814d100 con 0x7f91f80afbe0 2023-10-21T16:17:02.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.759+0000 7f91ffd83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f81f33d0 con 0x7f91f80cb130 2023-10-21T16:17:02.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:02.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33636/0 (socket says 172.21.15.53:33636) 2023-10-21T16:17:02.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 -- 172.21.15.53:0/2288774025 learned_addr learned my addr 172.21.15.53:0/2288774025 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:02.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 -- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91f80cb130 msgr2=0x7f91f81f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91f80cb130 0x7f91f81f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 -- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91f80afbe0 msgr2=0x7f91f814cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:02.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91f80afbe0 0x7f91f814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:02.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.760+0000 7f91fdb1f640 1 -- 172.21.15.53:0/2288774025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40094d0 con 0x7f91f806dfc0 2023-10-21T16:17:02.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91fdb1f640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:02.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4039d60 con 0x7f91f806dfc0 2023-10-21T16:17:02.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4043400 con 0x7f91f806dfc0 2023-10-21T16:17:02.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f81f36b0 con 0x7f91f806dfc0 2023-10-21T16:17:02.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40437b0 con 0x7f91f806dfc0 2023-10-21T16:17:02.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.761+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f81f3cf0 con 0x7f91f806dfc0 2023-10-21T16:17:02.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.762+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f81f3e80 con 0x7f91f806dfc0 2023-10-21T16:17:02.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.763+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f91f402e070 con 0x7f91f806dfc0 2023-10-21T16:17:02.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.764+0000 7f91ee7fc640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 0x7f91dc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:02.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.764+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f91f403e070 con 0x7f91f806dfc0 2023-10-21T16:17:02.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.765+0000 7f91fd31e640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 0x7f91dc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:02.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.766+0000 7f91fd31e640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 0x7f91dc06d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000f60 tx=0x7f91e80079b0 comp rx=0 tx=0).ready entity=mgr.5644 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:02.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:02.768+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91f4077720 con 0x7f91f806dfc0 2023-10-21T16:17:03.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.010+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f91f804e160 con 0x7f91f806dfc0 2023-10-21T16:17:03.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.316+0000 7f91ee7fc640 1 -- 172.21.15.53:0/2288774025 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) v1 ==== 93+0+0 (secure 0 0 0) 0x7f91f40340a0 con 0x7f91f806dfc0 2023-10-21T16:17:03.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.320+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 msgr2=0x7f91dc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.320+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 0x7f91dc06d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f91e8000f60 tx=0x7f91e80079b0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.320+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 msgr2=0x7f91f80ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:03.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.320+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009870 tx=0x7f91f4004730 comp rx=0 tx=0).stop 2023-10-21T16:17:03.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 shutdown_connections 2023-10-21T16:17:03.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f91dc06afe0 0x7f91dc06d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91f80cb130 0x7f91f81f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91f80afbe0 0x7f91f814cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 --2- 172.21.15.53:0/2288774025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91f806dfc0 0x7f91f80ceec0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 >> 172.21.15.53:0/2288774025 conn(0x7f91f806a5d0 msgr2=0x7f91f806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:03.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.321+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 shutdown_connections 2023-10-21T16:17:03.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.322+0000 7f91ffd83640 1 -- 172.21.15.53:0/2288774025 wait complete. 2023-10-21T16:17:03.346 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:17:03.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:03.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:03.557 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:17:03.557 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:03.558 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:03.612 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:03.612 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:03.658 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:03.858 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:03.856+0000 7f8358a45040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:03.875 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:03.873+0000 7f1206e4a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:03.910 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:03.908+0000 7f8358a45040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:03.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.915+0000 7f70e8d33640 1 Processor -- start 2023-10-21T16:17:03.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.916+0000 7f70e8d33640 1 -- start start 2023-10-21T16:17:03.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.917+0000 7f70e8d33640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70e4160460 0x7f70e40cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.917+0000 7f70e8d33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e40cd730 0x7f70e40cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.917+0000 7f70e8d33640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e40cefe0 con 0x7f70e41612d0 2023-10-21T16:17:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.917+0000 7f70e8d33640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e40ce100 con 0x7f70e4160460 2023-10-21T16:17:03.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.917+0000 7f70e8d33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cf2c0 con 0x7f70e40cd730 2023-10-21T16:17:03.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70e4160460 0x7f70e40cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:03.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70e4160460 0x7f70e40cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33684/0 (socket says 172.21.15.53:33684) 2023-10-21T16:17:03.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 -- 172.21.15.53:0/454628566 learned_addr learned my addr 172.21.15.53:0/454628566 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:03.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e2d9d640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e40cd730 0x7f70e40cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:03.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 -- 172.21.15.53:0/454628566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e40cd730 msgr2=0x7f70e40cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:03.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e40cd730 0x7f70e40cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 -- 172.21.15.53:0/454628566 >> v1:172.21.15.124:6789/0 conn(0x7f70e41612d0 legacy=0x7f70e406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:03.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.918+0000 7f70e259c640 1 -- 172.21.15.53:0/454628566 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40cf5a0 con 0x7f70e4160460 2023-10-21T16:17:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.919+0000 7f70e259c640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e4160460 0x7f70e40cced0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009870 tx=0x7f70d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6c95931bf18e40 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e1d9b640 1 -- 172.21.15.53:0/454628566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4031640 con 0x7f70e4160460 2023-10-21T16:17:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e1d9b640 1 -- 172.21.15.53:0/454628566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d40317a0 con 0x7f70e4160460 2023-10-21T16:17:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e1d9b640 1 -- 172.21.15.53:0/454628566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4031b30 con 0x7f70e4160460 2023-10-21T16:17:03.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 -- 172.21.15.53:0/454628566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e4160460 msgr2=0x7f70e40cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:03.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e4160460 0x7f70e40cced0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009870 tx=0x7f70d402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 -- 172.21.15.53:0/454628566 shutdown_connections 2023-10-21T16:17:03.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e40cd730 0x7f70e40cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 --2- 172.21.15.53:0/454628566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e4160460 0x7f70e40cced0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.920+0000 7f70e8d33640 1 -- 172.21.15.53:0/454628566 >> 172.21.15.53:0/454628566 conn(0x7f70e4069f30 msgr2=0x7f70e406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:03.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.921+0000 7f70e8d33640 1 -- 172.21.15.53:0/454628566 shutdown_connections 2023-10-21T16:17:03.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.921+0000 7f70e8d33640 1 -- 172.21.15.53:0/454628566 wait complete. 2023-10-21T16:17:03.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.923+0000 7f70e8d33640 1 Processor -- start 2023-10-21T16:17:03.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.923+0000 7f70e8d33640 1 -- start start 2023-10-21T16:17:03.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.924+0000 7f70e8d33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e8d33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 0x7f70e40b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e8d33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e41612d0 0x7f70e40b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e8d33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40b2da0 con 0x7f70e40cd730 2023-10-21T16:17:03.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e8d33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40b3b40 con 0x7f70e4160460 2023-10-21T16:17:03.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e8d33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40ca550 con 0x7f70e41612d0 2023-10-21T16:17:03.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:03.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33688/0 (socket says 172.21.15.53:33688) 2023-10-21T16:17:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 -- 172.21.15.53:0/576871545 learned_addr learned my addr 172.21.15.53:0/576871545 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e2d9d640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e41612d0 0x7f70e40b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70dbfff640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 0x7f70e40b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e41612d0 msgr2=0x7f70e40b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:03.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e41612d0 0x7f70e40b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 msgr2=0x7f70e40b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:03.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 0x7f70e40b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:03.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70e259c640 1 -- 172.21.15.53:0/576871545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40094d0 con 0x7f70e40cd730 2023-10-21T16:17:03.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.925+0000 7f70dbfff640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 0x7f70e40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:03.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70e259c640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009870 tx=0x7f70d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:03.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d403ac60 con 0x7f70e40cd730 2023-10-21T16:17:03.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d403adc0 con 0x7f70e40cd730 2023-10-21T16:17:03.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4044630 con 0x7f70e40cd730 2023-10-21T16:17:03.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e40ca830 con 0x7f70e40cd730 2023-10-21T16:17:03.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.926+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e415fa60 con 0x7f70e40cd730 2023-10-21T16:17:03.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.928+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e4161740 con 0x7f70e40cd730 2023-10-21T16:17:03.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.929+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f70d402f050 con 0x7f70e40cd730 2023-10-21T16:17:03.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.929+0000 7f70daffd640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 0x7f70c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:03.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.930+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f70d40abac0 con 0x7f70e40cd730 2023-10-21T16:17:03.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.930+0000 7f70dbfff640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 msgr2=0x7f70c406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:03.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.930+0000 7f70dbfff640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 0x7f70c406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:03.936 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:03.934+0000 7f1206e4a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:03.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:03.935+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70e4161740 con 0x7f70e40cd730 2023-10-21T16:17:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.127+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70e4161740 con 0x7f70e40cd730 2023-10-21T16:17:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.129+0000 7f70daffd640 1 -- 172.21.15.53:0/576871545 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17435 (secure 0 0 0) 0x7f70e4161740 con 0x7f70e40cd730 2023-10-21T16:17:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:04.131 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":75,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:16:31.299983+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.32","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/473879295":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/3508742564":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/2309645985":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/4037905816":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/1978958043":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/102098566":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6833/3288323666":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6832/3288323666":"2023-10-22T16:15:29.377682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.131+0000 7f70dbfff640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 msgr2=0x7f70c406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.131+0000 7f70dbfff640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 0x7f70c406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 msgr2=0x7f70c406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 0x7f70c406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 msgr2=0x7f70e4150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009870 tx=0x7f70d40046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 shutdown_connections 2023-10-21T16:17:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f70c406afc0 0x7f70c406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70e41612d0 0x7f70e40b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70e4160460 0x7f70e40b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 --2- 172.21.15.53:0/576871545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70e40cd730 0x7f70e4150f70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 >> 172.21.15.53:0/576871545 conn(0x7f70e4069f30 msgr2=0x7f70e406c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.132+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 shutdown_connections 2023-10-21T16:17:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.133+0000 7f70e8d33640 1 -- 172.21.15.53:0/576871545 wait complete. 2023-10-21T16:17:04.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:15:12.546220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:15:13.608148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:04.152 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:04.354 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.352+0000 7f8358a45040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.362+0000 7fbbc2dfb640 1 Processor -- start 2023-10-21T16:17:04.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.362+0000 7fbbc2dfb640 1 -- start start 2023-10-21T16:17:04.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc2dfb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbbbc0d10c0 0x7fbbbc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc2dfb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbbc0b0040 0x7fbbbc0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc2dfb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc0cdce0 con 0x7fbbbc1605c0 2023-10-21T16:17:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc2dfb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc06bc80 con 0x7fbbbc0b0040 2023-10-21T16:17:04.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc2dfb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc0b04b0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc0b97640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbbbc1605c0 0x7fbbbc0d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43470/0 (socket says 172.21.15.53:43470) 2023-10-21T16:17:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc0b97640 1 -- 172.21.15.53:0/336082841 learned_addr learned my addr 172.21.15.53:0/336082841 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:04.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.364+0000 7fbbc1398640 1 --2- 172.21.15.53:0/336082841 >> v2:172.21.15.53:3300/0 conn(0x7fbbbc0b0040 0x7fbbbc0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbbbfff640 1 --2- 172.21.15.53:0/336082841 >> v2:172.21.15.124:3300/0 conn(0x7fbbbc0d10c0 0x7fbbbc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbbb7fe640 1 -- 172.21.15.53:0/336082841 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724207575 0 0) 0x7fbbbc0cdce0 con 0x7fbbbc1605c0 2023-10-21T16:17:04.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbbb7fe640 1 -- 172.21.15.53:0/336082841 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac003520 con 0x7fbbbc1605c0 2023-10-21T16:17:04.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbc1398640 1 -- 172.21.15.53:0/336082841 >> v2:172.21.15.124:3300/0 conn(0x7fbbbc0d10c0 msgr2=0x7fbbbc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:04.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbc1398640 1 --2- 172.21.15.53:0/336082841 >> v2:172.21.15.124:3300/0 conn(0x7fbbbc0d10c0 0x7fbbbc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbc1398640 1 -- 172.21.15.53:0/336082841 >> v1:172.21.15.53:6789/0 conn(0x7fbbbc1605c0 legacy=0x7fbbbc0d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.365+0000 7fbbc1398640 1 -- 172.21.15.53:0/336082841 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0cdf30 con 0x7fbbbc0b0040 2023-10-21T16:17:04.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.366+0000 7fbbc1398640 1 --2- 172.21.15.53:0/336082841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0b0040 0x7fbbbc0b0980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0c7a50 tx=0x7fbbb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5339208b4a2c26ca server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.366+0000 7fbbbb7fe640 1 -- 172.21.15.53:0/336082841 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb402cc80 con 0x7fbbbc0b0040 2023-10-21T16:17:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.366+0000 7fbbbb7fe640 1 -- 172.21.15.53:0/336082841 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb402cde0 con 0x7fbbbc0b0040 2023-10-21T16:17:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbbb7fe640 1 -- 172.21.15.53:0/336082841 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4034660 con 0x7fbbbc0b0040 2023-10-21T16:17:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/336082841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0b0040 msgr2=0x7fbbbc0b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/336082841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0b0040 0x7fbbbc0b0980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0c7a50 tx=0x7fbbb402bed0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/336082841 shutdown_connections 2023-10-21T16:17:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/336082841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0b0040 0x7fbbbc0b0980 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/336082841 >> v2:172.21.15.124:3300/0 conn(0x7fbbbc0d10c0 0x7fbbbc06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/336082841 >> 172.21.15.53:0/336082841 conn(0x7fbbbc089fe0 msgr2=0x7fbbbc099d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/336082841 shutdown_connections 2023-10-21T16:17:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.367+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/336082841 wait complete. 2023-10-21T16:17:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.369+0000 7fbbc2dfb640 1 Processor -- start 2023-10-21T16:17:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.369+0000 7fbbc2dfb640 1 -- start start 2023-10-21T16:17:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbbc0b0040 0x7fbbbc14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbbc1605c0 0x7fbbbc159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0b3d20 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0c4580 con 0x7fbbbc1605c0 2023-10-21T16:17:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbc2dfb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc0c47d0 con 0x7fbbbc0b0040 2023-10-21T16:17:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.370+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47360/0 (socket says 172.21.15.53:47360) 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 -- 172.21.15.53:0/3296726562 learned_addr learned my addr 172.21.15.53:0/3296726562 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbc0b97640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbbc0b0040 0x7fbbbc14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbc1398640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbbc1605c0 0x7fbbbc159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbbc0b0040 msgr2=0x7fbbbc14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbbc0b0040 0x7fbbbc14dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbbc1605c0 msgr2=0x7fbbbc159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbbc1605c0 0x7fbbbc159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 -- 172.21.15.53:0/3296726562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4009a70 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.371+0000 7fbbbbfff640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbba800d800 tx=0x7fbba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.372+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800b6c0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.372+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800b820 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.372+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800bbb0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.372+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc0c4ab0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.372+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc0c5070 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.373+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc0c5200 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.374+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fbba800bd10 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.375+0000 7fbbb97fa640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.375+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7fbba808a010 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.375+0000 7fbbc0b97640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 msgr2=0x7fbba006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:04.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.375+0000 7fbbc0b97640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.383+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbba8054cb0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.408 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.406+0000 7f8358a45040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:04.422 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.421+0000 7f1206e4a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:04.457 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.455+0000 7f8358a45040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:04.477 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.476+0000 7f1206e4a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:04.517 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.515+0000 7f8358a45040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:04.530 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.529+0000 7f1206e4a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:04.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.564+0000 7f8358a45040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:04.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.575+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbbc08b0a0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.576+0000 7fbbc0b97640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 msgr2=0x7fbba006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:04.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.576+0000 7fbbc0b97640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:04.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.576+0000 7fbbb97fa640 1 -- 172.21.15.53:0/3296726562 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbba80582d0 con 0x7fbbbc0d10c0 2023-10-21T16:17:04.578 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 msgr2=0x7fbba006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 msgr2=0x7fbbbc14f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbba800d800 tx=0x7fbba800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 shutdown_connections 2023-10-21T16:17:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7fbba006af90 0x7fbba006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbbc1605c0 0x7fbbbc159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbbc0d10c0 0x7fbbbc14f6c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 --2- 172.21.15.53:0/3296726562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbbc0b0040 0x7fbbbc14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 >> 172.21.15.53:0/3296726562 conn(0x7fbbbc089fe0 msgr2=0x7fbbbc151130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 shutdown_connections 2023-10-21T16:17:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.579+0000 7fbbc2dfb640 1 -- 172.21.15.53:0/3296726562 wait complete. 2023-10-21T16:17:04.595 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.594+0000 7f1206e4a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:04.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:17:04.648 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.647+0000 7f1206e4a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:04.677 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.675+0000 7f8358a45040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:04.739 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.737+0000 7f8358a45040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:04.768 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.766+0000 7f1206e4a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:04.789 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.786+0000 7f8358a45040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:04.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.809+0000 7f68b671d640 1 Processor -- start 2023-10-21T16:17:04.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.809+0000 7f68b671d640 1 -- start start 2023-10-21T16:17:04.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.810+0000 7f68b671d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68b015cd80 0x7f68b00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.810+0000 7f68b671d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 0x7f68b00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.810+0000 7f68b671d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68b00cdda0 0x7f68b00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b671d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00ce230 con 0x7f68b00cdda0 2023-10-21T16:17:04.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b671d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b015d160 con 0x7f68b015cd80 2023-10-21T16:17:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b671d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00b7b90 con 0x7f68b00b6730 2023-10-21T16:17:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68b00cdda0 0x7f68b00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68b00cdda0 0x7f68b00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47376/0 (socket says 172.21.15.53:47376) 2023-10-21T16:17:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 -- 172.21.15.53:0/3986490749 learned_addr learned my addr 172.21.15.53:0/3986490749 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:04.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68affff640 1 --2- 172.21.15.53:0/3986490749 >> v2:172.21.15.124:3300/0 conn(0x7f68b015cd80 0x7f68b00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68af7fe640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 0x7f68b00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 -- 172.21.15.53:0/3986490749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 msgr2=0x7f68b00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:04.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 0x7f68b00cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 -- 172.21.15.53:0/3986490749 >> v2:172.21.15.124:3300/0 conn(0x7f68b015cd80 msgr2=0x7f68b00b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:04.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 --2- 172.21.15.53:0/3986490749 >> v2:172.21.15.124:3300/0 conn(0x7f68b015cd80 0x7f68b00b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68b4cba640 1 -- 172.21.15.53:0/3986490749 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00b8020 con 0x7f68b00cdda0 2023-10-21T16:17:04.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.811+0000 7f68af7fe640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 0x7f68b00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:04.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.812+0000 7f68b4cba640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b00b4610 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b5710 tx=0x7f68a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a5800d727abb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68aeffd640 1 -- 172.21.15.53:0/3986490749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a802ca80 con 0x7f68b00cdda0 2023-10-21T16:17:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68aeffd640 1 -- 172.21.15.53:0/3986490749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a802cbe0 con 0x7f68b00cdda0 2023-10-21T16:17:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68aeffd640 1 -- 172.21.15.53:0/3986490749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80343f0 con 0x7f68b00cdda0 2023-10-21T16:17:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 -- 172.21.15.53:0/3986490749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 msgr2=0x7f68b00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b00b4610 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b5710 tx=0x7f68a802bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 -- 172.21.15.53:0/3986490749 shutdown_connections 2023-10-21T16:17:04.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b00b6730 0x7f68b00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 --2- 172.21.15.53:0/3986490749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b00b4610 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 --2- 172.21.15.53:0/3986490749 >> v2:172.21.15.124:3300/0 conn(0x7f68b015cd80 0x7f68b00b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 -- 172.21.15.53:0/3986490749 >> 172.21.15.53:0/3986490749 conn(0x7f68b006b5d0 msgr2=0x7f68b006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 -- 172.21.15.53:0/3986490749 shutdown_connections 2023-10-21T16:17:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.813+0000 7f68b671d640 1 -- 172.21.15.53:0/3986490749 wait complete. 2023-10-21T16:17:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.815+0000 7f68b671d640 1 Processor -- start 2023-10-21T16:17:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.815+0000 7f68b671d640 1 -- start start 2023-10-21T16:17:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68b00b6730 0x7f68b01f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b015cd80 0x7f68b0148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f62e0 con 0x7f68b00cdda0 2023-10-21T16:17:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f23d0 con 0x7f68b00b6730 2023-10-21T16:17:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68b671d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01f26b0 con 0x7f68b015cd80 2023-10-21T16:17:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47392/0 (socket says 172.21.15.53:47392) 2023-10-21T16:17:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.816+0000 7f68af7fe640 1 -- 172.21.15.53:0/2839812489 learned_addr learned my addr 172.21.15.53:0/2839812489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68affff640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68b00b6730 0x7f68b01f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68b4cba640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b015cd80 0x7f68b0148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b015cd80 msgr2=0x7f68b0148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b015cd80 0x7f68b0148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68b00b6730 msgr2=0x7f68b01f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68b00b6730 0x7f68b01f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 -- 172.21.15.53:0/2839812489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8009b30 con 0x7f68b00cdda0 2023-10-21T16:17:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68af7fe640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c7f0 tx=0x7f68a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.817+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4007a30 con 0x7f68b00cdda0 2023-10-21T16:17:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.818+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4007b90 con 0x7f68b00cdda0 2023-10-21T16:17:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.818+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400ce40 con 0x7f68b00cdda0 2023-10-21T16:17:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.818+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b01f2990 con 0x7f68b00cdda0 2023-10-21T16:17:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.818+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b01f2c70 con 0x7f68b00cdda0 2023-10-21T16:17:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.819+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b004e160 con 0x7f68b00cdda0 2023-10-21T16:17:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.820+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f68a40113f0 con 0x7f68b00cdda0 2023-10-21T16:17:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.821+0000 7f68acff9640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 0x7f689c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.821+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f68a4089d00 con 0x7f68b00cdda0 2023-10-21T16:17:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.821+0000 7f68affff640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 msgr2=0x7f689c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.821+0000 7f68affff640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 0x7f689c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:04.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:04.826+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68a4054880 con 0x7f68b00cdda0 2023-10-21T16:17:04.845 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.843+0000 7f1206e4a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:04.846 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.844+0000 7f8358a45040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:04.896 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:04.894+0000 7f8358a45040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:04.905 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.903+0000 7f1206e4a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:04.965 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:04.964+0000 7f1206e4a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:05.018 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.016+0000 7f1206e4a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:05.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.022+0000 7f68affff640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 msgr2=0x7f689c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:05.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.022+0000 7f68affff640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 0x7f689c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.035+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68b00b86c0 con 0x7f68b00cdda0 2023-10-21T16:17:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.035+0000 7f68acff9640 1 -- 172.21.15.53:0/2839812489 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68a4057ea0 con 0x7f68b00cdda0 2023-10-21T16:17:05.037 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:17:05.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 msgr2=0x7f689c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 0x7f689c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 msgr2=0x7f68b01f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c7f0 tx=0x7f68a400cbf0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 shutdown_connections 2023-10-21T16:17:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f689c06af90 0x7f689c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68b015cd80 0x7f68b0148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68b00cdda0 0x7f68b01f5da0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 --2- 172.21.15.53:0/2839812489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68b00b6730 0x7f68b01f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 >> 172.21.15.53:0/2839812489 conn(0x7f68b006b5d0 msgr2=0x7f68b015ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 shutdown_connections 2023-10-21T16:17:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.038+0000 7f68b671d640 1 -- 172.21.15.53:0/2839812489 wait complete. 2023-10-21T16:17:05.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:17:05.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.271+0000 7f2bdc15a640 1 Processor -- start 2023-10-21T16:17:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.271+0000 7f2bdc15a640 1 -- start start 2023-10-21T16:17:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.272+0000 7f2bdc15a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.272+0000 7f2bdc15a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2bd40cb620 0x7f2bd40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.272+0000 7f2bdc15a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd406e150 con 0x7f2bd40c9910 2023-10-21T16:17:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.272+0000 7f2bdc15a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd406de70 con 0x7f2bd40cb620 2023-10-21T16:17:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.272+0000 7f2bdc15a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c8e30 con 0x7f2bd4160660 2023-10-21T16:17:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bd96f5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2bd40c9910 0x7f2bd40caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43486/0 (socket says 172.21.15.53:43486) 2023-10-21T16:17:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bd96f5640 1 -- 172.21.15.53:0/2768624887 learned_addr learned my addr 172.21.15.53:0/2768624887 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bda6f7640 1 --2- 172.21.15.53:0/2768624887 >> v2:172.21.15.53:3300/0 conn(0x7f2bd40cb620 0x7f2bd40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.276 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.272+0000 7f8358a45040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:05.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bd9ef6640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bda6f7640 1 -- 172.21.15.53:0/2768624887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 msgr2=0x7f2bd4070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:05.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bda6f7640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd4070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bda6f7640 1 -- 172.21.15.53:0/2768624887 >> v1:172.21.15.53:6789/0 conn(0x7f2bd40c9910 legacy=0x7f2bd40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bda6f7640 1 -- 172.21.15.53:0/2768624887 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40c9110 con 0x7f2bd40cb620 2023-10-21T16:17:05.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.273+0000 7f2bd8ef4640 1 -- 172.21.15.53:0/2768624887 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616036098 0 0) 0x7f2bd406e150 con 0x7f2bd40c9910 2023-10-21T16:17:05.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.275+0000 7f2bda6f7640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40cb620 0x7f2bd40b1990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009d90 tx=0x7f2bcc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8bafd7bb5dc1f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:05.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.275+0000 7f2bd8ef4640 1 -- 172.21.15.53:0/2768624887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc02cc50 con 0x7f2bd40cb620 2023-10-21T16:17:05.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.275+0000 7f2bd8ef4640 1 -- 172.21.15.53:0/2768624887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc02cdb0 con 0x7f2bd40cb620 2023-10-21T16:17:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.275+0000 7f2bd8ef4640 1 -- 172.21.15.53:0/2768624887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc034660 con 0x7f2bd40cb620 2023-10-21T16:17:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 -- 172.21.15.53:0/2768624887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40cb620 msgr2=0x7f2bd40b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40cb620 0x7f2bd40b1990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009d90 tx=0x7f2bcc02bea0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 -- 172.21.15.53:0/2768624887 shutdown_connections 2023-10-21T16:17:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40cb620 0x7f2bd40b1990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/2768624887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd4070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 -- 172.21.15.53:0/2768624887 >> 172.21.15.53:0/2768624887 conn(0x7f2bd406a5d0 msgr2=0x7f2bd40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 -- 172.21.15.53:0/2768624887 shutdown_connections 2023-10-21T16:17:05.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.276+0000 7f2bdc15a640 1 -- 172.21.15.53:0/2768624887 wait complete. 2023-10-21T16:17:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.278+0000 7f2bdc15a640 1 Processor -- start 2023-10-21T16:17:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.278+0000 7f2bdc15a640 1 -- start start 2023-10-21T16:17:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd40cb620 0x7f2bd40c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd40c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c28e0 con 0x7f2bd40c9910 2023-10-21T16:17:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c2bc0 con 0x7f2bd40cb620 2023-10-21T16:17:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bdc15a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c2ea0 con 0x7f2bd4160660 2023-10-21T16:17:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bd9ef6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bd9ef6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47404/0 (socket says 172.21.15.53:47404) 2023-10-21T16:17:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.279+0000 7f2bd9ef6640 1 -- 172.21.15.53:0/3102561179 learned_addr learned my addr 172.21.15.53:0/3102561179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd96f5640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd40cb620 0x7f2bd40c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bda6f7640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd40c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 msgr2=0x7f2bd40c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd40c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd40cb620 msgr2=0x7f2bd40c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd40cb620 0x7f2bd40c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 -- 172.21.15.53:0/3102561179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc009a70 con 0x7f2bd40c9910 2023-10-21T16:17:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bda6f7640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd40c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bd9ef6640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d5f0 tx=0x7f2bd000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00125f0 con 0x7f2bd40c9910 2023-10-21T16:17:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0012750 con 0x7f2bd40c9910 2023-10-21T16:17:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.280+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0012b00 con 0x7f2bd40c9910 2023-10-21T16:17:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.281+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd40c3180 con 0x7f2bd40c9910 2023-10-21T16:17:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.281+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd40c3980 con 0x7f2bd40c9910 2023-10-21T16:17:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.282+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd404e160 con 0x7f2bd40c9910 2023-10-21T16:17:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.283+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f2bd0012c60 con 0x7f2bd40c9910 2023-10-21T16:17:05.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.284+0000 7f2bca7fc640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 0x7f2bb806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.284+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f2bd0089fe0 con 0x7f2bd40c9910 2023-10-21T16:17:05.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.284+0000 7f2bd96f5640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 msgr2=0x7f2bb806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:05.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.284+0000 7f2bd96f5640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 0x7f2bb806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:05.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.292+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bd0054c00 con 0x7f2bd40c9910 2023-10-21T16:17:05.438 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.437+0000 7f1206e4a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:05.440 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.439+0000 7f8358a45040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.485+0000 7f2bd96f5640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 msgr2=0x7f2bb806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:05.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.485+0000 7f2bd96f5640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 0x7f2bb806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:05.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.494+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bd40ca130 con 0x7f2bd40c9910 2023-10-21T16:17:05.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.495+0000 7f2bca7fc640 1 -- 172.21.15.53:0/3102561179 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bd0058220 con 0x7f2bd40c9910 2023-10-21T16:17:05.496 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:17:05.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 msgr2=0x7f2bb806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:05.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 0x7f2bb806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 msgr2=0x7f2bd40c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d5f0 tx=0x7f2bd000dab0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 shutdown_connections 2023-10-21T16:17:05.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f2bb806af90 0x7f2bb806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bd4160660 0x7f2bd40c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bd40cb620 0x7f2bd40c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 --2- 172.21.15.53:0/3102561179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bd40c9910 0x7f2bd40c2250 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 >> 172.21.15.53:0/3102561179 conn(0x7f2bd406a5d0 msgr2=0x7f2bd40caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:05.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 shutdown_connections 2023-10-21T16:17:05.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.497+0000 7f2bdc15a640 1 -- 172.21.15.53:0/3102561179 wait complete. 2023-10-21T16:17:05.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:17:05.548 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.546+0000 7f8358a45040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:05.598 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.596+0000 7f8358a45040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:05.610 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.608+0000 7f1206e4a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:05.705 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.703+0000 7f8358a45040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:05.724 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.723+0000 7f1206e4a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:05.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.724+0000 7f15bd626640 1 Processor -- start 2023-10-21T16:17:05.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.724+0000 7f15bd626640 1 -- start start 2023-10-21T16:17:05.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.725+0000 7f15bd626640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f15b806dfc0 0x7f15b8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.725+0000 7f15bd626640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b80b1cc0 con 0x7f15b80afbe0 2023-10-21T16:17:05.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.725+0000 7f15bd626640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b80b2620 con 0x7f15b8071070 2023-10-21T16:17:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.725+0000 7f15bd626640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80b2060 con 0x7f15b806dfc0 2023-10-21T16:17:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.726+0000 7f15b6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f15b80afbe0 0x7f15b80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43488/0 (socket says 172.21.15.53:43488) 2023-10-21T16:17:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.726+0000 7f15b6ffd640 1 -- 172.21.15.53:0/4176418016 learned_addr learned my addr 172.21.15.53:0/4176418016 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:05.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.726+0000 7f15b67fc640 1 --2- 172.21.15.53:0/4176418016 >> v2:172.21.15.124:3300/0 conn(0x7f15b806dfc0 0x7f15b8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.726+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514306758 0 0) 0x7f15b80b1cc0 con 0x7f15b80afbe0 2023-10-21T16:17:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.726+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a0003520 con 0x7f15b80afbe0 2023-10-21T16:17:05.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877426400 0 0) 0x7f15a0003520 con 0x7f15b80afbe0 2023-10-21T16:17:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 >> v2:172.21.15.124:3300/0 conn(0x7f15b806dfc0 msgr2=0x7f15b8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 --2- 172.21.15.53:0/4176418016 >> v2:172.21.15.124:3300/0 conn(0x7f15b806dfc0 0x7f15b8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 >> v1:172.21.15.124:6789/0 conn(0x7f15b8071070 legacy=0x7f15b80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b80b2340 con 0x7f15b80afbe0 2023-10-21T16:17:05.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f15a80019f0 con 0x7f15b80afbe0 2023-10-21T16:17:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 >> v1:172.21.15.53:6789/0 conn(0x7f15b80afbe0 legacy=0x7f15b80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15a0003970 0x7f15a0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15a0005180 0x7f15a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80b2060 con 0x7f15a00047c0 2023-10-21T16:17:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000c050 con 0x7f15a0003970 2023-10-21T16:17:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000c210 con 0x7f15a0005180 2023-10-21T16:17:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b67fc640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.727+0000 7f15b6ffd640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15a0003970 0x7f15a0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b77fe640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15a0005180 0x7f15a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b67fc640 1 -- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15a0005180 msgr2=0x7f15a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b67fc640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15a0005180 0x7f15a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b67fc640 1 -- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15a0003970 msgr2=0x7f15a0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b67fc640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15a0003970 0x7f15a0003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b67fc640 1 -- 172.21.15.53:0/4176418016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0003520 con 0x7f15a00047c0 2023-10-21T16:17:05.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b6ffd640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15a0003970 0x7f15a0003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:05.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b77fe640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15a0005180 0x7f15a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:05.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.728+0000 7f15b77fe640 1 -- 172.21.15.53:0/4176418016 reap_dead start 2023-10-21T16:17:05.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.729+0000 7f15b67fc640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15a0004c40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009a90 tx=0x7f15a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc3f9b303708cc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:05.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.729+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac02c680 con 0x7f15a00047c0 2023-10-21T16:17:05.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.729+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac02c7e0 con 0x7f15a00047c0 2023-10-21T16:17:05.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.729+0000 7f15b5ffb640 1 -- 172.21.15.53:0/4176418016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac02cb90 con 0x7f15a00047c0 2023-10-21T16:17:05.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 -- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 msgr2=0x7f15a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15a0004c40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009a90 tx=0x7f15a00057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 -- 172.21.15.53:0/4176418016 shutdown_connections 2023-10-21T16:17:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 --2- 172.21.15.53:0/4176418016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15a0004c40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 -- 172.21.15.53:0/4176418016 >> 172.21.15.53:0/4176418016 conn(0x7f15b806a5d0 msgr2=0x7f15b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 -- 172.21.15.53:0/4176418016 shutdown_connections 2023-10-21T16:17:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.730+0000 7f15bd626640 1 -- 172.21.15.53:0/4176418016 wait complete. 2023-10-21T16:17:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.732+0000 7f15bd626640 1 Processor -- start 2023-10-21T16:17:05.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.732+0000 7f15bd626640 1 -- start start 2023-10-21T16:17:05.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 0x7f15b814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 0x7f15b80cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b814f230 con 0x7f15a00047c0 2023-10-21T16:17:05.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815ca80 con 0x7f15b815bf20 2023-10-21T16:17:05.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15bd626640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80ce290 con 0x7f15b814eda0 2023-10-21T16:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47420/0 (socket says 172.21.15.53:47420) 2023-10-21T16:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.733+0000 7f15b6ffd640 1 -- 172.21.15.53:0/669121938 learned_addr learned my addr 172.21.15.53:0/669121938 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b67fc640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 0x7f15b814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 msgr2=0x7f15b80cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b77fe640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 0x7f15b80cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:05.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 0x7f15b80cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 msgr2=0x7f15b814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 0x7f15b814e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 -- 172.21.15.53:0/669121938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b80ce570 con 0x7f15a00047c0 2023-10-21T16:17:05.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b67fc640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 0x7f15b814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:05.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b77fe640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 0x7f15b80cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:05.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15b6ffd640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f15a8007ac0 tx=0x7f15a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:05.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800ee50 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8013420 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8013790 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b80ce850 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.734+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b80cee10 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.736+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b80cf030 con 0x7f15a00047c0 2023-10-21T16:17:05.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.737+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f15a80138f0 con 0x7f15a00047c0 2023-10-21T16:17:05.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.738+0000 7f159f7fe640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 0x7f15a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:05.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.738+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f15a808a490 con 0x7f15a00047c0 2023-10-21T16:17:05.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.738+0000 7f15b67fc640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 msgr2=0x7f15a406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:05.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.738+0000 7f15b67fc640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 0x7f15a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:05.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.742+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15a8054fa0 con 0x7f15a00047c0 2023-10-21T16:17:05.784 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.782+0000 7f1206e4a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:05.815 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.813+0000 7f8358a45040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:05.898 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:05.897+0000 7f1206e4a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:05.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.939+0000 7f15b67fc640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 msgr2=0x7f15a406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:05.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.939+0000 7f15b67fc640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 0x7f15a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:05.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.956+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15b804e160 con 0x7f15a00047c0 2023-10-21T16:17:05.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.956+0000 7f159f7fe640 1 -- 172.21.15.53:0/669121938 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v76) v1 ==== 80+0+476 (secure 0 0 0) 0x7f15a80585c0 con 0x7f15a00047c0 2023-10-21T16:17:05.958 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/473879295 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.958 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3508742564 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.958 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2309645985 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.958 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4037905816 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.959 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1978958043 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.959 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/102098566 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.959 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3288323666 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.959 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3288323666 2023-10-22T16:15:29.377682+0000 2023-10-21T16:17:05.959 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T16:17:05.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 msgr2=0x7f15a406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 0x7f15a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 msgr2=0x7f15b814d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f15a8007ac0 tx=0x7f15a800e530 comp rx=0 tx=0).stop 2023-10-21T16:17:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 shutdown_connections 2023-10-21T16:17:05.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f15a406af90 0x7f15a406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15b814eda0 0x7f15b80cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15b815bf20 0x7f15b814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 --2- 172.21.15.53:0/669121938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a00047c0 0x7f15b814d1a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:05.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 >> 172.21.15.53:0/669121938 conn(0x7f15b806a5d0 msgr2=0x7f15b80b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 shutdown_connections 2023-10-21T16:17:05.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:05.959+0000 7f15bd626640 1 -- 172.21.15.53:0/669121938 wait complete. 2023-10-21T16:17:05.971 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:05.969+0000 7f8358a45040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:05.979 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:17:05.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:05.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:06.015 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:06.014+0000 7f1206e4a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:06.023 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:06.021+0000 7f8358a45040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:06.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:17:06.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:06.087 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:17:06.154 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:06.151+0000 7f8358a45040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:06.179 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:06.179 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:06.194 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:06.193+0000 7f1206e4a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:06.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:06.233 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:06.233 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:06.247 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:06.246+0000 7f1206e4a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:06.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:06.291 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:06.291 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:06.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:06.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:06.381 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:06.380+0000 7f1206e4a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.553+0000 7ff9b759e640 1 Processor -- start 2023-10-21T16:17:06.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.553+0000 7ff9b759e640 1 -- start start 2023-10-21T16:17:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.554+0000 7ff9b759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b815a9a0 0x7ff9b80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.554+0000 7ff9b759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff9b80b54b0 0x7ff9b8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.554+0000 7ff9b759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b815b070 con 0x7ff9b80b5050 2023-10-21T16:17:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.554+0000 7ff9b759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b80cfd00 con 0x7ff9b80b54b0 2023-10-21T16:17:06.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.554+0000 7ff9b759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80cc590 con 0x7ff9b815a9a0 2023-10-21T16:17:06.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b6d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff9b80b54b0 0x7ff9b8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b5d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff9b80b5050 0x7ff9b80cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43502/0 (socket says 172.21.15.53:43502) 2023-10-21T16:17:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b5d9b640 1 -- 172.21.15.53:0/3028543267 learned_addr learned my addr 172.21.15.53:0/3028543267 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b659c640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b815a9a0 0x7ff9b80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/3028543267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b815a9a0 msgr2=0x7ff9b80cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b6d9d640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b815a9a0 0x7ff9b80cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/3028543267 >> v1:172.21.15.53:6789/0 conn(0x7ff9b80b5050 legacy=0x7ff9b80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/3028543267 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80cc870 con 0x7ff9b80b54b0 2023-10-21T16:17:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.555+0000 7ff9b559a640 1 -- 172.21.15.53:0/3028543267 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908939859 0 0) 0x7ff9b815b070 con 0x7ff9b80b5050 2023-10-21T16:17:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.556+0000 7ff9b6d9d640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b80b54b0 0x7ff9b8069d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566b1c474a702269 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b559a640 1 -- 172.21.15.53:0/3028543267 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b002cc50 con 0x7ff9b80b54b0 2023-10-21T16:17:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b559a640 1 -- 172.21.15.53:0/3028543267 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b002cdb0 con 0x7ff9b80b54b0 2023-10-21T16:17:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b559a640 1 -- 172.21.15.53:0/3028543267 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0034660 con 0x7ff9b80b54b0 2023-10-21T16:17:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 -- 172.21.15.53:0/3028543267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b80b54b0 msgr2=0x7ff9b8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b80b54b0 0x7ff9b8069d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b002bea0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 -- 172.21.15.53:0/3028543267 shutdown_connections 2023-10-21T16:17:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b80b54b0 0x7ff9b8069d50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 --2- 172.21.15.53:0/3028543267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b815a9a0 0x7ff9b80cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 -- 172.21.15.53:0/3028543267 >> 172.21.15.53:0/3028543267 conn(0x7ff9b806ae60 msgr2=0x7ff9b806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.557+0000 7ff9b759e640 1 -- 172.21.15.53:0/3028543267 shutdown_connections 2023-10-21T16:17:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.558+0000 7ff9b759e640 1 -- 172.21.15.53:0/3028543267 wait complete. 2023-10-21T16:17:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.559+0000 7ff9b759e640 1 Processor -- start 2023-10-21T16:17:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.559+0000 7ff9b759e640 1 -- start start 2023-10-21T16:17:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9b80b5050 0x7ff9b80ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b80b54b0 0x7ff9b80aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80ad290 con 0x7ff9b815a9a0 2023-10-21T16:17:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80ace00 con 0x7ff9b80b5050 2023-10-21T16:17:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.560+0000 7ff9b759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80ac1f0 con 0x7ff9b80b54b0 2023-10-21T16:17:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47430/0 (socket says 172.21.15.53:47430) 2023-10-21T16:17:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/1618979948 learned_addr learned my addr 172.21.15.53:0/1618979948 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b5d9b640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b80b54b0 0x7ff9b80aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b659c640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9b80b5050 0x7ff9b80ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b80b54b0 msgr2=0x7ff9b80aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b80b54b0 0x7ff9b80aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9b80b5050 msgr2=0x7ff9b80ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:06.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9b80b5050 0x7ff9b80ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 -- 172.21.15.53:0/1618979948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0009a70 con 0x7ff9b815a9a0 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.561+0000 7ff9b6d9d640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.562+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0040650 con 0x7ff9b815a9a0 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.562+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b00407d0 con 0x7ff9b815a9a0 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.562+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0040b80 con 0x7ff9b815a9a0 2023-10-21T16:17:06.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.562+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b80ac4d0 con 0x7ff9b815a9a0 2023-10-21T16:17:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.562+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b80aca90 con 0x7ff9b815a9a0 2023-10-21T16:17:06.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.563+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b80acc50 con 0x7ff9b815a9a0 2023-10-21T16:17:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.564+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff9b0040ce0 con 0x7ff9b815a9a0 2023-10-21T16:17:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.565+0000 7ff9a6ffd640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 0x7ff99806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.565+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9b00ac8b0 con 0x7ff9b815a9a0 2023-10-21T16:17:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.566+0000 7ff9b659c640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 msgr2=0x7ff99806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.566+0000 7ff9b659c640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 0x7ff99806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.570+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9b0077500 con 0x7ff9b815a9a0 2023-10-21T16:17:06.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.767+0000 7ff9b659c640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 msgr2=0x7ff99806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:06.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.767+0000 7ff9b659c640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 0x7ff99806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.809+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9b80acf70 con 0x7ff9b815a9a0 2023-10-21T16:17:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.811+0000 7ff9a6ffd640 1 -- 172.21.15.53:0/1618979948 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7ff9b007ab20 con 0x7ff9b815a9a0 2023-10-21T16:17:06.813 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:06.813 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:17:06.813 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:17:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 msgr2=0x7ff99806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 0x7ff99806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 msgr2=0x7ff9b81f25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009d90 tx=0x7ff9b0004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 shutdown_connections 2023-10-21T16:17:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7ff99806afc0 0x7ff99806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9b815a9a0 0x7ff9b81f25c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9b80b54b0 0x7ff9b80aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 --2- 172.21.15.53:0/1618979948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9b80b5050 0x7ff9b80ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 >> 172.21.15.53:0/1618979948 conn(0x7ff9b806ae60 msgr2=0x7ff9b80b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.814+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 shutdown_connections 2023-10-21T16:17:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:06.815+0000 7ff9b759e640 1 -- 172.21.15.53:0/1618979948 wait complete. 2023-10-21T16:17:06.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:07.012 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.010+0000 7f8358a45040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:07.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.060+0000 7f42f30ea640 1 Processor -- start 2023-10-21T16:17:07.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.060+0000 7f42f30ea640 1 -- start start 2023-10-21T16:17:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f30ea640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42ec1599d0 0x7f42ec0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f30ea640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec0cacf0 con 0x7f42ec0cc210 2023-10-21T16:17:07.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f30ea640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec0ca3c0 con 0x7f42ec0b07d0 2023-10-21T16:17:07.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f30ea640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0b10f0 con 0x7f42ec1599d0 2023-10-21T16:17:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f18e7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f42ec0cc210 0x7f42ec0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43506/0 (socket says 172.21.15.53:43506) 2023-10-21T16:17:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f18e7640 1 -- 172.21.15.53:0/2233476124 learned_addr learned my addr 172.21.15.53:0/2233476124 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.062+0000 7f42f20e8640 1 --2- 172.21.15.53:0/2233476124 >> v2:172.21.15.124:3300/0 conn(0x7f42ec1599d0 0x7f42ec0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288403065 0 0) 0x7f42ec0cacf0 con 0x7f42ec0cc210 2023-10-21T16:17:07.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8003520 con 0x7f42ec0cc210 2023-10-21T16:17:07.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312471237 0 0) 0x7f42d8003520 con 0x7f42ec0cc210 2023-10-21T16:17:07.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 >> v2:172.21.15.124:3300/0 conn(0x7f42ec1599d0 msgr2=0x7f42ec0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:07.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 --2- 172.21.15.53:0/2233476124 >> v2:172.21.15.124:3300/0 conn(0x7f42ec1599d0 0x7f42ec0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 >> v1:172.21.15.124:6789/0 conn(0x7f42ec0b07d0 legacy=0x7f42ec06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0ca6a0 con 0x7f42ec0cc210 2023-10-21T16:17:07.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f42dc001670 con 0x7f42ec0cc210 2023-10-21T16:17:07.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 >> v1:172.21.15.53:6789/0 conn(0x7f42ec0cc210 legacy=0x7f42ec0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d8003970 0x7f42d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80047c0 0x7f42d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0b10f0 con 0x7f42d8005180 2023-10-21T16:17:07.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d800c020 con 0x7f42d8003970 2023-10-21T16:17:07.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.063+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d800c1e0 con 0x7f42d80047c0 2023-10-21T16:17:07.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f18e7640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80047c0 0x7f42d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f20e8640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d8003970 0x7f42d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 -- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80047c0 msgr2=0x7f42d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:07.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80047c0 0x7f42d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 -- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d8003970 msgr2=0x7f42d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:07.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d8003970 0x7f42d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.064+0000 7f42f28e9640 1 -- 172.21.15.53:0/2233476124 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8003520 con 0x7f42d8005180 2023-10-21T16:17:07.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.065+0000 7f42f28e9640 1 -- 172.21.15.53:0/2233476124 reap_dead start 2023-10-21T16:17:07.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.065+0000 7f42f28e9640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42d800b980 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f42e4001e20 tx=0x7f42d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c686188c04a8f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.065+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4003b70 con 0x7f42d8005180 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.065+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e4003cd0 con 0x7f42d8005180 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.065+0000 7f42f10e6640 1 -- 172.21.15.53:0/2233476124 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4005310 con 0x7f42d8005180 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 -- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 msgr2=0x7f42d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42d800b980 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f42e4001e20 tx=0x7f42d80057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 -- 172.21.15.53:0/2233476124 shutdown_connections 2023-10-21T16:17:07.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 --2- 172.21.15.53:0/2233476124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42d800b980 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 -- 172.21.15.53:0/2233476124 >> 172.21.15.53:0/2233476124 conn(0x7f42ec0cb0c0 msgr2=0x7f42ec0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:07.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 -- 172.21.15.53:0/2233476124 shutdown_connections 2023-10-21T16:17:07.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.066+0000 7f42f30ea640 1 -- 172.21.15.53:0/2233476124 wait complete. 2023-10-21T16:17:07.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.068+0000 7f42f30ea640 1 Processor -- start 2023-10-21T16:17:07.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.068+0000 7f42f30ea640 1 -- start start 2023-10-21T16:17:07.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42ec0b3a70 0x7f42ec1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42ec1edb30 0x7f42ec1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec150130 con 0x7f42d8005180 2023-10-21T16:17:07.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0b3ef0 con 0x7f42ec0b3a70 2023-10-21T16:17:07.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f30ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec14fe50 con 0x7f42ec1edb30 2023-10-21T16:17:07.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f20e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f20e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47444/0 (socket says 172.21.15.53:47444) 2023-10-21T16:17:07.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.069+0000 7f42f20e8640 1 -- 172.21.15.53:0/3723480598 learned_addr learned my addr 172.21.15.53:0/3723480598 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:07.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f28e9640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42ec1edb30 0x7f42ec1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f18e7640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42ec0b3a70 0x7f42ec1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42ec1edb30 msgr2=0x7f42ec1f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42ec1edb30 0x7f42ec1f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42ec0b3a70 msgr2=0x7f42ec1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42ec0b3a70 0x7f42ec1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 -- 172.21.15.53:0/3723480598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec150700 con 0x7f42d8005180 2023-10-21T16:17:07.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f20e8640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f42e8011540 tx=0x7f42e8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8015930 con 0x7f42d8005180 2023-10-21T16:17:07.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8015a90 con 0x7f42d8005180 2023-10-21T16:17:07.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8015e60 con 0x7f42d8005180 2023-10-21T16:17:07.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.070+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec1509e0 con 0x7f42d8005180 2023-10-21T16:17:07.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.071+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec151720 con 0x7f42d8005180 2023-10-21T16:17:07.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.072+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec150560 con 0x7f42d8005180 2023-10-21T16:17:07.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.073+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f42e800f460 con 0x7f42d8005180 2023-10-21T16:17:07.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.073+0000 7f42e2ffd640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 0x7f42d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.074+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f42e808a2f0 con 0x7f42d8005180 2023-10-21T16:17:07.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.074+0000 7f42f18e7640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 msgr2=0x7f42d006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:07.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.074+0000 7f42f18e7640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 0x7f42d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:07.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.078+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42ec150560 con 0x7f42d8005180 2023-10-21T16:17:07.235 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:07.234+0000 7f1206e4a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:07.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.275+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42ec150560 con 0x7f42d8005180 2023-10-21T16:17:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.275+0000 7f42f18e7640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 msgr2=0x7f42d006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3738768716 2023-10-21T16:17:07.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.275+0000 7f42f18e7640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 0x7f42d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:07.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.276+0000 7f42e2ffd640 1 -- 172.21.15.53:0/3723480598 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17567 (secure 0 0 0) 0x7f42ec150560 con 0x7f42d8005180 2023-10-21T16:17:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:07.279 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":77,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:06.329816+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.32","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/473879295":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/3508742564":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/2309645985":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/4037905816":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/1978958043":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/102098566":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6833/3288323666":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6832/3288323666":"2023-10-22T16:15:29.377682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:07.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 msgr2=0x7f42d006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:07.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 0x7f42d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 msgr2=0x7f42ec14f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f42e8011540 tx=0x7f42e8011a00 comp rx=0 tx=0).stop 2023-10-21T16:17:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 shutdown_connections 2023-10-21T16:17:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:6832/3738768716,v1:172.21.15.124:6833/3738768716] conn(0x7f42d006af90 0x7f42d006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42ec1edb30 0x7f42ec1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42ec0b3a70 0x7f42ec1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 --2- 172.21.15.53:0/3723480598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d8005180 0x7f42ec14f7c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 >> 172.21.15.53:0/3723480598 conn(0x7f42ec0cb0c0 msgr2=0x7f42ec156730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 shutdown_connections 2023-10-21T16:17:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.279+0000 7f42f30ea640 1 -- 172.21.15.53:0/3723480598 wait complete. 2023-10-21T16:17:07.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:15:12.546220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:15:13.608148+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:07.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:07.311 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.309+0000 7f8358a45040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:07.370 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.368+0000 7f8358a45040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:07.483 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.481+0000 7f8358a45040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.510+0000 7ff8e9090640 1 Processor -- start 2023-10-21T16:17:07.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.511+0000 7ff8e9090640 1 -- start start 2023-10-21T16:17:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.512+0000 7ff8e9090640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 0x7ff8e40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.512+0000 7ff8e9090640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40ce290 con 0x7ff8e40d1120 2023-10-21T16:17:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e9090640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40d0a80 con 0x7ff8e415dcd0 2023-10-21T16:17:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e9090640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40b1fe0 con 0x7ff8e40d1500 2023-10-21T16:17:07.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff8e40d1120 0x7ff8e40cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43512/0 (socket says 172.21.15.53:43512) 2023-10-21T16:17:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e3fff640 1 -- 172.21.15.53:0/1710675547 learned_addr learned my addr 172.21.15.53:0/1710675547 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e37fe640 1 --2- 172.21.15.53:0/1710675547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 0x7ff8e40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e2ffd640 1 -- 172.21.15.53:0/1710675547 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665986078 0 0) 0x7ff8e40ce290 con 0x7ff8e40d1120 2023-10-21T16:17:07.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e2ffd640 1 -- 172.21.15.53:0/1710675547 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8cc003520 con 0x7ff8e40d1120 2023-10-21T16:17:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e37fe640 1 -- 172.21.15.53:0/1710675547 >> v1:172.21.15.124:6789/0 conn(0x7ff8e415dcd0 legacy=0x7ff8e40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e37fe640 1 -- 172.21.15.53:0/1710675547 >> v1:172.21.15.53:6789/0 conn(0x7ff8e40d1120 legacy=0x7ff8e40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.513+0000 7ff8e37fe640 1 -- 172.21.15.53:0/1710675547 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40b4580 con 0x7ff8e40d1500 2023-10-21T16:17:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.515+0000 7ff8e37fe640 1 --2- 172.21.15.53:0/1710675547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 0x7ff8e40d04a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80097d0 tx=0x7ff8d802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c67a29b1e542b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.515+0000 7ff8e2ffd640 1 -- 172.21.15.53:0/1710675547 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8031800 con 0x7ff8e40d1500 2023-10-21T16:17:07.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.515+0000 7ff8e2ffd640 1 -- 172.21.15.53:0/1710675547 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8031960 con 0x7ff8e40d1500 2023-10-21T16:17:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.515+0000 7ff8e2ffd640 1 -- 172.21.15.53:0/1710675547 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8031cf0 con 0x7ff8e40d1500 2023-10-21T16:17:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 -- 172.21.15.53:0/1710675547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 msgr2=0x7ff8e40d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 --2- 172.21.15.53:0/1710675547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 0x7ff8e40d04a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80097d0 tx=0x7ff8d802b8a0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 -- 172.21.15.53:0/1710675547 shutdown_connections 2023-10-21T16:17:07.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 --2- 172.21.15.53:0/1710675547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1500 0x7ff8e40d04a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 -- 172.21.15.53:0/1710675547 >> 172.21.15.53:0/1710675547 conn(0x7ff8e40d2960 msgr2=0x7ff8e40d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 -- 172.21.15.53:0/1710675547 shutdown_connections 2023-10-21T16:17:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.516+0000 7ff8e9090640 1 -- 172.21.15.53:0/1710675547 wait complete. 2023-10-21T16:17:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.518+0000 7ff8e9090640 1 Processor -- start 2023-10-21T16:17:07.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.518+0000 7ff8e9090640 1 -- start start 2023-10-21T16:17:07.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1120 0x7ff8e415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8e415dcd0 0x7ff8e415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40c6b40 con 0x7ff8e40d1500 2023-10-21T16:17:07.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e415c360 con 0x7ff8e415dcd0 2023-10-21T16:17:07.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e9090640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40cbfb0 con 0x7ff8e40d1120 2023-10-21T16:17:07.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47450/0 (socket says 172.21.15.53:47450) 2023-10-21T16:17:07.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e37fe640 1 -- 172.21.15.53:0/705379181 learned_addr learned my addr 172.21.15.53:0/705379181 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:07.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e888f640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8e415dcd0 0x7ff8e415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e3fff640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1120 0x7ff8e415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e37fe640 1 -- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1120 msgr2=0x7ff8e415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.519+0000 7ff8e37fe640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1120 0x7ff8e415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e37fe640 1 -- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8e415dcd0 msgr2=0x7ff8e415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e37fe640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8e415dcd0 0x7ff8e415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e37fe640 1 -- 172.21.15.53:0/705379181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d8009400 con 0x7ff8e40d1500 2023-10-21T16:17:07.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e37fe640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8d802beb0 tx=0x7ff8d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d803bc70 con 0x7ff8e40d1500 2023-10-21T16:17:07.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d803bdf0 con 0x7ff8e40d1500 2023-10-21T16:17:07.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e40cc290 con 0x7ff8e40d1500 2023-10-21T16:17:07.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8045630 con 0x7ff8e40d1500 2023-10-21T16:17:07.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.520+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40c6760 con 0x7ff8e40d1500 2023-10-21T16:17:07.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.522+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e4074150 con 0x7ff8e40d1500 2023-10-21T16:17:07.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.522+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff8d8045790 con 0x7ff8e40d1500 2023-10-21T16:17:07.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.524+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7ff8d802e070 con 0x7ff8e40d1500 2023-10-21T16:17:07.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.528+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8d8076b60 con 0x7ff8e40d1500 2023-10-21T16:17:07.569 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:07.568+0000 7f1206e4a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:07.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.571+0000 7f82dad99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.571+0000 7f82dad99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.573 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.571+0000 7f82dad99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.571+0000 7f82dad99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.571+0000 7f82dad99640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.572+0000 7f82ddd9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.574 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.572+0000 7f82ddd9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.572+0000 7f82ddd9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.572+0000 7f82ddd9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:07.572+0000 7f82ddd9f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:07.632 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:07.630+0000 7f1206e4a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:07.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.725+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8e40ccc90 con 0x7ff8e40d1500 2023-10-21T16:17:07.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.727+0000 7ff8e0ff9640 1 -- 172.21.15.53:0/705379181 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8d8035020 con 0x7ff8e40d1500 2023-10-21T16:17:07.728 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:07.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.729+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 msgr2=0x7ff8e40cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.729+0000 7ff8e9090640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8d802beb0 tx=0x7ff8d80046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 shutdown_connections 2023-10-21T16:17:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8e415dcd0 0x7ff8e415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8e40d1500 0x7ff8e40cb6b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 --2- 172.21.15.53:0/705379181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8e40d1120 0x7ff8e415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 >> 172.21.15.53:0/705379181 conn(0x7ff8e40d2960 msgr2=0x7ff8e40cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:07.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 shutdown_connections 2023-10-21T16:17:07.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.730+0000 7ff8e9090640 1 -- 172.21.15.53:0/705379181 wait complete. 2023-10-21T16:17:07.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:17:07.758 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:07.756+0000 7f1206e4a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:07.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.958+0000 7fb330814640 1 Processor -- start 2023-10-21T16:17:07.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.958+0000 7fb330814640 1 -- start start 2023-10-21T16:17:07.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.960+0000 7fb330814640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.960+0000 7fb330814640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 0x7fb32c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.960+0000 7fb330814640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c0cab60 con 0x7fb32c15ac00 2023-10-21T16:17:07.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.960+0000 7fb330814640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0cb960 con 0x7fb32c15a820 2023-10-21T16:17:07.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.960+0000 7fb330814640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0cbdf0 con 0x7fb32c0ca680 2023-10-21T16:17:07.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.961+0000 7fb32b7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.961+0000 7fb32affd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 0x7fb32c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.961+0000 7fb32b7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59128/0 (socket says 172.21.15.53:59128) 2023-10-21T16:17:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.961+0000 7fb32b7fe640 1 -- 172.21.15.53:0/583900077 learned_addr learned my addr 172.21.15.53:0/583900077 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.962+0000 7fb32affd640 1 -- 172.21.15.53:0/583900077 >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 msgr2=0x7fb32c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.962+0000 7fb32affd640 1 --2- 172.21.15.53:0/583900077 >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.962+0000 7fb32affd640 1 -- 172.21.15.53:0/583900077 >> v1:172.21.15.124:6789/0 conn(0x7fb32c15ac00 legacy=0x7fb32c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.962+0000 7fb32affd640 1 -- 172.21.15.53:0/583900077 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0cb030 con 0x7fb32c0ca680 2023-10-21T16:17:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.962+0000 7fb32b7fe640 1 --2- 172.21.15.53:0/583900077 >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:07.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.963+0000 7fb32affd640 1 --2- 172.21.15.53:0/583900077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 0x7fb32c0b04b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb3200097a0 tx=0x7fb32002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e8244673ac531e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.964+0000 7fb32a7fc640 1 -- 172.21.15.53:0/583900077 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320031780 con 0x7fb32c0ca680 2023-10-21T16:17:07.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.964+0000 7fb32a7fc640 1 -- 172.21.15.53:0/583900077 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3200318e0 con 0x7fb32c0ca680 2023-10-21T16:17:07.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.964+0000 7fb32a7fc640 1 -- 172.21.15.53:0/583900077 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320031c70 con 0x7fb32c0ca680 2023-10-21T16:17:07.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 -- 172.21.15.53:0/583900077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 msgr2=0x7fb32c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 --2- 172.21.15.53:0/583900077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 0x7fb32c0b04b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb3200097a0 tx=0x7fb32002b820 comp rx=0 tx=0).stop 2023-10-21T16:17:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 -- 172.21.15.53:0/583900077 shutdown_connections 2023-10-21T16:17:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 --2- 172.21.15.53:0/583900077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0ca680 0x7fb32c0b04b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 --2- 172.21.15.53:0/583900077 >> v2:172.21.15.124:3300/0 conn(0x7fb32c15a820 0x7fb32c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 -- 172.21.15.53:0/583900077 >> 172.21.15.53:0/583900077 conn(0x7fb32c06c050 msgr2=0x7fb32c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:07.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 -- 172.21.15.53:0/583900077 shutdown_connections 2023-10-21T16:17:07.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.965+0000 7fb330814640 1 -- 172.21.15.53:0/583900077 wait complete. 2023-10-21T16:17:07.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.967+0000 7fb330814640 1 Processor -- start 2023-10-21T16:17:07.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.967+0000 7fb330814640 1 -- start start 2023-10-21T16:17:07.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c0ca680 0x7fb32c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c15a820 0x7fb32c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:07.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c152070 con 0x7fb32c15ac00 2023-10-21T16:17:07.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c15d360 con 0x7fb32c0ca680 2023-10-21T16:17:07.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb330814640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c15c750 con 0x7fb32c15a820 2023-10-21T16:17:07.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb32bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb32bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47466/0 (socket says 172.21.15.53:47466) 2023-10-21T16:17:07.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb32bfff640 1 -- 172.21.15.53:0/3220433409 learned_addr learned my addr 172.21.15.53:0/3220433409 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:07.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb32b7fe640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c0ca680 0x7fb32c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.968+0000 7fb32affd640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c15a820 0x7fb32c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:07.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 -- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c15a820 msgr2=0x7fb32c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c15a820 0x7fb32c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 -- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c0ca680 msgr2=0x7fb32c154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:07.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c0ca680 0x7fb32c154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:07.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 -- 172.21.15.53:0/3220433409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0094d0 con 0x7fb32c15ac00 2023-10-21T16:17:07.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb32bfff640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c730 tx=0x7fb32400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:07.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324003b70 con 0x7fb32c15ac00 2023-10-21T16:17:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324003cd0 con 0x7fb32c15ac00 2023-10-21T16:17:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb320009400 con 0x7fb32c15ac00 2023-10-21T16:17:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400aa90 con 0x7fb32c15ac00 2023-10-21T16:17:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.969+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c15cd10 con 0x7fb32c15ac00 2023-10-21T16:17:07.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.971+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c15d4d0 con 0x7fb32c15ac00 2023-10-21T16:17:07.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.971+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fb32400abf0 con 0x7fb32c15ac00 2023-10-21T16:17:07.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.972+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fb3240892f0 con 0x7fb32c15ac00 2023-10-21T16:17:07.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:07.977+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb324054380 con 0x7fb32c15ac00 2023-10-21T16:17:08.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.168+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb32c074150 con 0x7fb32c15ac00 2023-10-21T16:17:08.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.169+0000 7fb313fff640 1 -- 172.21.15.53:0/3220433409 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3240579a0 con 0x7fb32c15ac00 2023-10-21T16:17:08.170 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:17:08.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.171+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 msgr2=0x7fb32c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.171+0000 7fb330814640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c730 tx=0x7fb32400cbf0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 shutdown_connections 2023-10-21T16:17:08.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c15ac00 0x7fb32c0ce990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c15a820 0x7fb32c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 --2- 172.21.15.53:0/3220433409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c0ca680 0x7fb32c154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 >> 172.21.15.53:0/3220433409 conn(0x7fb32c06c050 msgr2=0x7fb32c0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:08.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 shutdown_connections 2023-10-21T16:17:08.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.172+0000 7fb330814640 1 -- 172.21.15.53:0/3220433409 wait complete. 2023-10-21T16:17:08.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:17:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.470+0000 7f079b59e640 1 Processor -- start 2023-10-21T16:17:08.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.471+0000 7f079b59e640 1 -- start start 2023-10-21T16:17:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.472+0000 7f079b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.472+0000 7f079b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f079c0ce510 0x7f079c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.472+0000 7f079b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0cd520 con 0x7f079c0cf690 2023-10-21T16:17:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.472+0000 7f079b59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c0d1910 con 0x7f079c0ce510 2023-10-21T16:17:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.472+0000 7f079b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0ce080 con 0x7f079c0cf2b0 2023-10-21T16:17:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f0793fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f079c0cf690 0x7f079c0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43524/0 (socket says 172.21.15.53:43524) 2023-10-21T16:17:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f0793fff640 1 -- 172.21.15.53:0/2388588399 learned_addr learned my addr 172.21.15.53:0/2388588399 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:08.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f079ad9d640 1 --2- 172.21.15.53:0/2388588399 >> v2:172.21.15.53:3300/0 conn(0x7f079c0ce510 0x7f079c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f079a59c640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050117934 0 0) 0x7f079c0cd520 con 0x7f079c0cf690 2023-10-21T16:17:08.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0788003520 con 0x7f079c0cf690 2023-10-21T16:17:08.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f079ad9d640 1 -- 172.21.15.53:0/2388588399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 msgr2=0x7f079c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.473+0000 7f079ad9d640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.474+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728797208 0 0) 0x7f0788003520 con 0x7f079c0cf690 2023-10-21T16:17:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.474+0000 7f079ad9d640 1 -- 172.21.15.53:0/2388588399 >> v1:172.21.15.53:6789/0 conn(0x7f079c0cf690 legacy=0x7f079c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.474+0000 7f079ad9d640 1 -- 172.21.15.53:0/2388588399 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0cd750 con 0x7f079c0ce510 2023-10-21T16:17:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.475+0000 7f079ad9d640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15c820 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079402bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc3e1c14328b859 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.475+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784001670 con 0x7f079c0ce510 2023-10-21T16:17:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.475+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079402c810 con 0x7f079c0ce510 2023-10-21T16:17:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f0799d9b640 1 -- 172.21.15.53:0/2388588399 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079402cb80 con 0x7f079c0ce510 2023-10-21T16:17:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 -- 172.21.15.53:0/2388588399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 msgr2=0x7f079c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15c820 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0794009d90 tx=0x7f079402bce0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 -- 172.21.15.53:0/2388588399 shutdown_connections 2023-10-21T16:17:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15c820 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 --2- 172.21.15.53:0/2388588399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 -- 172.21.15.53:0/2388588399 >> 172.21.15.53:0/2388588399 conn(0x7f079c09bc40 msgr2=0x7f079c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 -- 172.21.15.53:0/2388588399 shutdown_connections 2023-10-21T16:17:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.476+0000 7f079b59e640 1 -- 172.21.15.53:0/2388588399 wait complete. 2023-10-21T16:17:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.478+0000 7f079b59e640 1 Processor -- start 2023-10-21T16:17:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.478+0000 7f079b59e640 1 -- start start 2023-10-21T16:17:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c0cf690 0x7f079c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15e510 con 0x7f079c0ce510 2023-10-21T16:17:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15e7b0 con 0x7f079c0cf690 2023-10-21T16:17:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.479+0000 7f079b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15ea90 con 0x7f079c0cf2b0 2023-10-21T16:17:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47498/0 (socket says 172.21.15.53:47498) 2023-10-21T16:17:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 -- 172.21.15.53:0/3282377931 learned_addr learned my addr 172.21.15.53:0/3282377931 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079ad9d640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c0cf690 0x7f079c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 -- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 msgr2=0x7f079c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 -- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c0cf690 msgr2=0x7f079c1f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c0cf690 0x7f079c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 -- 172.21.15.53:0/3282377931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009a70 con 0x7f079c0ce510 2023-10-21T16:17:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.480+0000 7f079a59c640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f078c00e9b0 tx=0x7f078c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.481+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c009670 con 0x7f079c0ce510 2023-10-21T16:17:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.481+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00c9a0 con 0x7f079c0ce510 2023-10-21T16:17:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.481+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00cd50 con 0x7f079c0ce510 2023-10-21T16:17:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.481+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c15ed70 con 0x7f079c0ce510 2023-10-21T16:17:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.481+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c1f2400 con 0x7f079c0ce510 2023-10-21T16:17:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.482+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c15ef00 con 0x7f079c0ce510 2023-10-21T16:17:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.483+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f078c013470 con 0x7f079c0ce510 2023-10-21T16:17:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.484+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f078c0897e0 con 0x7f079c0ce510 2023-10-21T16:17:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.488+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f078c054780 con 0x7f079c0ce510 2023-10-21T16:17:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.510+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f078c054230 con 0x7f079c0ce510 2023-10-21T16:17:08.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.510+0000 7f0792ffd640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 0x7f077c073100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.512+0000 7f0793fff640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 0x7f077c073100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.526+0000 7f0793fff640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 0x7f077c073100 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0784005880 tx=0x7f0784001080 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.686+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f079c074150 con 0x7f079c0ce510 2023-10-21T16:17:08.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.687+0000 7f0792ffd640 1 -- 172.21.15.53:0/3282377931 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f078c087850 con 0x7f079c0ce510 2023-10-21T16:17:08.689 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:17:08.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 msgr2=0x7f077c073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 0x7f077c073100 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0784005880 tx=0x7f0784001080 comp rx=0 tx=0).stop 2023-10-21T16:17:08.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 msgr2=0x7f079c15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f078c00e9b0 tx=0x7f078c00ee70 comp rx=0 tx=0).stop 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 shutdown_connections 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f077c070d10 0x7f077c073100 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c0cf690 0x7f079c1f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0cf2b0 0x7f079c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 --2- 172.21.15.53:0/3282377931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0ce510 0x7f079c15f290 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 >> 172.21.15.53:0/3282377931 conn(0x7f079c09bc40 msgr2=0x7f079c0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:08.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 shutdown_connections 2023-10-21T16:17:08.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.690+0000 7f079b59e640 1 -- 172.21.15.53:0/3282377931 wait complete. 2023-10-21T16:17:08.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:08.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.943+0000 7ff008c3f640 1 Processor -- start 2023-10-21T16:17:08.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.943+0000 7ff008c3f640 1 -- start start 2023-10-21T16:17:08.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.945+0000 7ff008c3f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 0x7ff0040ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.945+0000 7ff008c3f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff0040c90b0 0x7ff0040adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.945+0000 7ff008c3f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00415b2b0 con 0x7ff0040cae70 2023-10-21T16:17:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.945+0000 7ff008c3f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00415bf30 con 0x7ff0040c90b0 2023-10-21T16:17:08.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.945+0000 7ff008c3f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00415f7a0 con 0x7ff00415ab40 2023-10-21T16:17:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.946+0000 7ff00259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 0x7ff0040ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.946+0000 7ff00259c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 0x7ff0040ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:55780/0 (socket says 172.21.15.53:55780) 2023-10-21T16:17:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.946+0000 7ff00259c640 1 -- 172.21.15.53:0/2237749050 learned_addr learned my addr 172.21.15.53:0/2237749050 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.946+0000 7ff001d9b640 1 --2- 172.21.15.53:0/2237749050 >> v2:172.21.15.124:3300/0 conn(0x7ff0040c90b0 0x7ff0040adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.947+0000 7ff001d9b640 1 -- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 msgr2=0x7ff0040ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.947+0000 7ff001d9b640 1 --2- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 0x7ff0040ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.947+0000 7ff001d9b640 1 -- 172.21.15.53:0/2237749050 >> v1:172.21.15.124:6789/0 conn(0x7ff0040cae70 legacy=0x7ff0040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.947+0000 7ff001d9b640 1 -- 172.21.15.53:0/2237749050 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00415b590 con 0x7ff0040c90b0 2023-10-21T16:17:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.948+0000 7ff001d9b640 1 --2- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0040c90b0 0x7ff0040adc60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feff80097a0 tx=0x7feff802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4413a365ba831e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff00159a640 1 -- 172.21.15.53:0/2237749050 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff802d040 con 0x7ff0040c90b0 2023-10-21T16:17:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff00159a640 1 -- 172.21.15.53:0/2237749050 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8031810 con 0x7ff0040c90b0 2023-10-21T16:17:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff00159a640 1 -- 172.21.15.53:0/2237749050 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8031ba0 con 0x7ff0040c90b0 2023-10-21T16:17:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 -- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0040c90b0 msgr2=0x7ff0040adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 --2- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0040c90b0 0x7ff0040adc60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feff80097a0 tx=0x7feff802b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 -- 172.21.15.53:0/2237749050 shutdown_connections 2023-10-21T16:17:08.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 --2- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0040c90b0 0x7ff0040adc60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 --2- 172.21.15.53:0/2237749050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff00415ab40 0x7ff0040ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 -- 172.21.15.53:0/2237749050 >> 172.21.15.53:0/2237749050 conn(0x7ff00406a380 msgr2=0x7ff0040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:08.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 -- 172.21.15.53:0/2237749050 shutdown_connections 2023-10-21T16:17:08.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.949+0000 7ff008c3f640 1 -- 172.21.15.53:0/2237749050 wait complete. 2023-10-21T16:17:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.951+0000 7ff008c3f640 1 Processor -- start 2023-10-21T16:17:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.951+0000 7ff008c3f640 1 -- start start 2023-10-21T16:17:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.952+0000 7ff008c3f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0040c90b0 0x7ff0041502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.952+0000 7ff008c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.952+0000 7ff008c3f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 0x7ff00415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff008c3f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004159800 con 0x7ff0040cae70 2023-10-21T16:17:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff008c3f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c58f0 con 0x7ff00415ab40 2023-10-21T16:17:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff008c3f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c5bd0 con 0x7ff0040c90b0 2023-10-21T16:17:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47514/0 (socket says 172.21.15.53:47514) 2023-10-21T16:17:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 -- 172.21.15.53:0/3543484505 learned_addr learned my addr 172.21.15.53:0/3543484505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff00259c640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0040c90b0 0x7ff0041502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff002d9d640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 0x7ff00415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 -- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0040c90b0 msgr2=0x7ff0041502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0040c90b0 0x7ff0041502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 -- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 msgr2=0x7ff00415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:08.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 0x7ff00415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:08.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff001d9b640 1 -- 172.21.15.53:0/3543484505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009400 con 0x7ff0040cae70 2023-10-21T16:17:08.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.953+0000 7ff002d9d640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 0x7ff00415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:08.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7ff001d9b640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8006e10 con 0x7ff0040cae70 2023-10-21T16:17:08.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff803ad60 con 0x7ff0040cae70 2023-10-21T16:17:08.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040c5eb0 con 0x7ff0040cae70 2023-10-21T16:17:08.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8044630 con 0x7ff0040cae70 2023-10-21T16:17:08.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.954+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff004150800 con 0x7ff0040cae70 2023-10-21T16:17:08.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.955+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0040c63d0 con 0x7ff0040cae70 2023-10-21T16:17:08.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.957+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7feff804d450 con 0x7ff0040cae70 2023-10-21T16:17:08.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.958+0000 7feff2ffd640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 0x7fefe406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.958+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7feff80ab950 con 0x7ff0040cae70 2023-10-21T16:17:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.958+0000 7ff00259c640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 0x7fefe406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:08.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.959+0000 7ff00259c640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 0x7fefe406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff0040aedc0 tx=0x7feff4008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:08.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:08.962+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0040c63d0 con 0x7ff0040cae70 2023-10-21T16:17:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.195+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0040c63d0 con 0x7ff0040cae70 2023-10-21T16:17:09.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.196+0000 7feff2ffd640 1 -- 172.21.15.53:0/3543484505 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7ff0040c63d0 con 0x7ff0040cae70 2023-10-21T16:17:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:09.199 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:17:09.199 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:17:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 msgr2=0x7fefe406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 0x7fefe406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff0040aedc0 tx=0x7feff4008040 comp rx=0 tx=0).stop 2023-10-21T16:17:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 msgr2=0x7ff00415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff80046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 shutdown_connections 2023-10-21T16:17:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fefe406acc0 0x7fefe406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff00415ab40 0x7ff00415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff0040cae70 0x7ff00415d4c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 --2- 172.21.15.53:0/3543484505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0040c90b0 0x7ff0041502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 >> 172.21.15.53:0/3543484505 conn(0x7ff00406a380 msgr2=0x7ff0040cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 shutdown_connections 2023-10-21T16:17:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.199+0000 7ff008c3f640 1 -- 172.21.15.53:0/3543484505 wait complete. 2023-10-21T16:17:09.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:09.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.448+0000 7fdd74000640 1 Processor -- start 2023-10-21T16:17:09.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.449+0000 7fdd74000640 1 -- start start 2023-10-21T16:17:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.450+0000 7fdd74000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.450+0000 7fdd74000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.450+0000 7fdd74000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c15b2b0 con 0x7fdd6c0cae70 2023-10-21T16:17:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.450+0000 7fdd74000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c15bf30 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.450+0000 7fdd74000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c15f7a0 con 0x7fdd6c15ab40 2023-10-21T16:17:09.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47526/0 (socket says 172.21.15.53:47526) 2023-10-21T16:17:09.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 -- 172.21.15.53:0/323782771 learned_addr learned my addr 172.21.15.53:0/323782771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:09.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd71d9c640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 -- 172.21.15.53:0/323782771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 msgr2=0x7fdd6c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:09.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 -- 172.21.15.53:0/323782771 >> v1:172.21.15.53:6789/0 conn(0x7fdd6c0cae70 legacy=0x7fdd6c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd7159b640 1 -- 172.21.15.53:0/323782771 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c15b590 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.451+0000 7fdd71d9c640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:09.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.452+0000 7fdd7159b640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009870 tx=0x7fdd5c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d784302470dc059b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd70d9a640 1 -- 172.21.15.53:0/323782771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c030840 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd70d9a640 1 -- 172.21.15.53:0/323782771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c0309a0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd70d9a640 1 -- 172.21.15.53:0/323782771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c030d30 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd74000640 1 -- 172.21.15.53:0/323782771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 msgr2=0x7fdd6c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd74000640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009870 tx=0x7fdd5c02b980 comp rx=0 tx=0).stop 2023-10-21T16:17:09.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.453+0000 7fdd74000640 1 -- 172.21.15.53:0/323782771 shutdown_connections 2023-10-21T16:17:09.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.454+0000 7fdd74000640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c0adc60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.454+0000 7fdd74000640 1 --2- 172.21.15.53:0/323782771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.454+0000 7fdd74000640 1 -- 172.21.15.53:0/323782771 >> 172.21.15.53:0/323782771 conn(0x7fdd6c06a380 msgr2=0x7fdd6c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:09.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.454+0000 7fdd74000640 1 -- 172.21.15.53:0/323782771 shutdown_connections 2023-10-21T16:17:09.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.454+0000 7fdd74000640 1 -- 172.21.15.53:0/323782771 wait complete. 2023-10-21T16:17:09.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.455+0000 7fdd74000640 1 Processor -- start 2023-10-21T16:17:09.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.456+0000 7fdd74000640 1 -- start start 2023-10-21T16:17:09.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.456+0000 7fdd74000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd74000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 0x7fdd6c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd74000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd74000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c151b60 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd74000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0c5850 con 0x7fdd6c0cae70 2023-10-21T16:17:09.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd74000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c152300 con 0x7fdd6c15ab40 2023-10-21T16:17:09.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47528/0 (socket says 172.21.15.53:47528) 2023-10-21T16:17:09.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 -- 172.21.15.53:0/4216229313 learned_addr learned my addr 172.21.15.53:0/4216229313 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:09.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd7259d640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd7159b640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 0x7fdd6c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 -- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 msgr2=0x7fdd6c1503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c1503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 -- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 msgr2=0x7fdd6c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 0x7fdd6c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd71d9c640 1 -- 172.21.15.53:0/4216229313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0094d0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd7259d640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:09.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.457+0000 7fdd7159b640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 0x7fdd6c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd71d9c640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800cf60 tx=0x7fdd6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800d840 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6800d9a0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800dd30 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c150900 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.458+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c1589f0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.459+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c067490 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.460+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fdd68015440 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.462+0000 7fdd627fc640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 0x7fdd5006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.462+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fdd6808a0c0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.463+0000 7fdd7159b640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 0x7fdd5006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.464+0000 7fdd7159b640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 0x7fdd5006d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c036000 tx=0x7fdd5c03b040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.466+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd68055020 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.524+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdd680553f0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.667+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd6c0b0ef0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.671+0000 7fdd627fc640 1 -- 172.21.15.53:0/4216229313 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18010 (secure 0 0 0) 0x7fdd68072ae0 con 0x7fdd6c0c90b0 2023-10-21T16:17:09.673 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:09.673 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:07.486639+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.32","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1823284108":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/439129464":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6833/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/2911894415":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1304233815":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6832/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/3052410171":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1876804894":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/473879295":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/3508742564":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/2309645985":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/4037905816":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/1978958043":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/102098566":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6833/3288323666":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6832/3288323666":"2023-10-22T16:15:29.377682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 msgr2=0x7fdd5006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 0x7fdd5006d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c036000 tx=0x7fdd5c03b040 comp rx=0 tx=0).stop 2023-10-21T16:17:09.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 msgr2=0x7fdd6c158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800cf60 tx=0x7fdd6800cf90 comp rx=0 tx=0).stop 2023-10-21T16:17:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 shutdown_connections 2023-10-21T16:17:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdd5006acf0 0x7fdd5006d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd6c15ab40 0x7fdd6c1503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd6c0cae70 0x7fdd6c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 --2- 172.21.15.53:0/4216229313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd6c0c90b0 0x7fdd6c158130 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.673+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 >> 172.21.15.53:0/4216229313 conn(0x7fdd6c06a380 msgr2=0x7fdd6c153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.674+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 shutdown_connections 2023-10-21T16:17:09.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.674+0000 7fdd74000640 1 -- 172.21.15.53:0/4216229313 wait complete. 2023-10-21T16:17:09.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:17:09.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:09.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.918+0000 7fe52ef27640 1 Processor -- start 2023-10-21T16:17:09.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.918+0000 7fe52ef27640 1 -- start start 2023-10-21T16:17:09.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52ef27640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5280d16f0 0x7fe5280cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52ef27640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5280cfa50 0x7fe5280ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52ef27640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280cea50 con 0x7fe52815cc60 2023-10-21T16:17:09.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52ef27640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52815d040 con 0x7fe5280cfa50 2023-10-21T16:17:09.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52ef27640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5280d0650 con 0x7fe5280d16f0 2023-10-21T16:17:09.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52e726640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5280cfa50 0x7fe5280ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52e726640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5280cfa50 0x7fe5280ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47530/0 (socket says 172.21.15.53:47530) 2023-10-21T16:17:09.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.919+0000 7fe52e726640 1 -- 172.21.15.53:0/1021831828 learned_addr learned my addr 172.21.15.53:0/1021831828 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:09.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.920+0000 7fe52e726640 1 -- 172.21.15.53:0/1021831828 >> v2:172.21.15.124:3300/0 conn(0x7fe5280d16f0 msgr2=0x7fe5280cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:09.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.920+0000 7fe52e726640 1 --2- 172.21.15.53:0/1021831828 >> v2:172.21.15.124:3300/0 conn(0x7fe5280d16f0 0x7fe5280cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.920+0000 7fe52e726640 1 -- 172.21.15.53:0/1021831828 >> v1:172.21.15.124:6789/0 conn(0x7fe52815cc60 legacy=0x7fe5280cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:09.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.920+0000 7fe52e726640 1 -- 172.21.15.53:0/1021831828 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280d0930 con 0x7fe5280cfa50 2023-10-21T16:17:09.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.921+0000 7fe52e726640 1 --2- 172.21.15.53:0/1021831828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280ce510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b5560 tx=0x7fe52002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6fea66dd4dcbc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.921+0000 7fe52cf23640 1 -- 172.21.15.53:0/1021831828 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52002ca80 con 0x7fe5280cfa50 2023-10-21T16:17:09.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.921+0000 7fe52cf23640 1 -- 172.21.15.53:0/1021831828 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52002cbe0 con 0x7fe5280cfa50 2023-10-21T16:17:09.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.921+0000 7fe52cf23640 1 -- 172.21.15.53:0/1021831828 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200343f0 con 0x7fe5280cfa50 2023-10-21T16:17:09.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 -- 172.21.15.53:0/1021831828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 msgr2=0x7fe5280ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 --2- 172.21.15.53:0/1021831828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280ce510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b5560 tx=0x7fe52002bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 -- 172.21.15.53:0/1021831828 shutdown_connections 2023-10-21T16:17:09.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 --2- 172.21.15.53:0/1021831828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280ce510 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 --2- 172.21.15.53:0/1021831828 >> v2:172.21.15.124:3300/0 conn(0x7fe5280d16f0 0x7fe5280cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 -- 172.21.15.53:0/1021831828 >> 172.21.15.53:0/1021831828 conn(0x7fe52809bcf0 msgr2=0x7fe5280d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:09.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 -- 172.21.15.53:0/1021831828 shutdown_connections 2023-10-21T16:17:09.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.922+0000 7fe52ef27640 1 -- 172.21.15.53:0/1021831828 wait complete. 2023-10-21T16:17:09.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.924+0000 7fe52ef27640 1 Processor -- start 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.924+0000 7fe52ef27640 1 -- start start 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5280d16f0 0x7fe5280cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe52815cc60 0x7fe5281519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280b6280 con 0x7fe5280cfa50 2023-10-21T16:17:09.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280b5600 con 0x7fe5280d16f0 2023-10-21T16:17:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52ef27640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280b58e0 con 0x7fe52815cc60 2023-10-21T16:17:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52df25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52df25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47536/0 (socket says 172.21.15.53:47536) 2023-10-21T16:17:09.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.925+0000 7fe52df25640 1 -- 172.21.15.53:0/2387740476 learned_addr learned my addr 172.21.15.53:0/2387740476 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 -- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe52815cc60 msgr2=0x7fe5281519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe52815cc60 0x7fe5281519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 -- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5280d16f0 msgr2=0x7fe5280cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5280d16f0 0x7fe5280cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:09.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 -- 172.21.15.53:0/2387740476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009b30 con 0x7fe5280cfa50 2023-10-21T16:17:09.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52df25640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe524004640 tx=0x7fe52400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524011790 con 0x7fe5280cfa50 2023-10-21T16:17:09.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5240118f0 con 0x7fe5280cfa50 2023-10-21T16:17:09.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524011ce0 con 0x7fe5280cfa50 2023-10-21T16:17:09.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.926+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5280b5bc0 con 0x7fe5280cfa50 2023-10-21T16:17:09.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.927+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5280b4e70 con 0x7fe5280cfa50 2023-10-21T16:17:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.928+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5280b63c0 con 0x7fe5280cfa50 2023-10-21T16:17:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.929+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe52401c460 con 0x7fe5280cfa50 2023-10-21T16:17:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.930+0000 7fe51e7fc640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 0x7fe50c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:09.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.930+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fe5240898d0 con 0x7fe5280cfa50 2023-10-21T16:17:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.930+0000 7fe52d724640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 0x7fe50c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.932+0000 7fe52d724640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 0x7fe50c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b6dc0 tx=0x7fe518006bf0 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:09.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:09.934+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5280b63c0 con 0x7fe5280cfa50 2023-10-21T16:17:10.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.163+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5280b63c0 con 0x7fe5280cfa50 2023-10-21T16:17:10.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.165+0000 7fe51e7fc640 1 -- 172.21.15.53:0/2387740476 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe5240545b0 con 0x7fe5280cfa50 2023-10-21T16:17:10.167 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:10.167 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:17:10.167 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:17:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.167+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 msgr2=0x7fe50c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.167+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 0x7fe50c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b6dc0 tx=0x7fe518006bf0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.167+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 msgr2=0x7fe5280b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.167+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe524004640 tx=0x7fe52400c8e0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 shutdown_connections 2023-10-21T16:17:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe50c06af00 0x7fe50c06d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe52815cc60 0x7fe5281519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5280d16f0 0x7fe5280cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 --2- 172.21.15.53:0/2387740476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5280cfa50 0x7fe5280b4930 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 >> 172.21.15.53:0/2387740476 conn(0x7fe52809bcf0 msgr2=0x7fe5280b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 shutdown_connections 2023-10-21T16:17:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.168+0000 7fe52ef27640 1 -- 172.21.15.53:0/2387740476 wait complete. 2023-10-21T16:17:10.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.415+0000 7faa28578640 1 Processor -- start 2023-10-21T16:17:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.415+0000 7faa28578640 1 -- start start 2023-10-21T16:17:10.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa28578640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 0x7faa200702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa28578640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faa2015e330 0x7faa20070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa28578640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006dee0 con 0x7faa20159ac0 2023-10-21T16:17:10.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa28578640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2006dcf0 con 0x7faa2015e330 2023-10-21T16:17:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa28578640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2015d850 con 0x7faa200ad790 2023-10-21T16:17:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.417+0000 7faa26b15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faa2015e330 0x7faa20070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faa2015e330 0x7faa20070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47552/0 (socket says 172.21.15.53:47552) 2023-10-21T16:17:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 -- 172.21.15.53:0/1185123611 learned_addr learned my addr 172.21.15.53:0/1185123611 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa25b13640 1 --2- 172.21.15.53:0/1185123611 >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 0x7faa200702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 -- 172.21.15.53:0/1185123611 >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 msgr2=0x7faa200702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 --2- 172.21.15.53:0/1185123611 >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 0x7faa200702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 -- 172.21.15.53:0/1185123611 >> v1:172.21.15.124:6789/0 conn(0x7faa20159ac0 legacy=0x7faa2006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa26b15640 1 -- 172.21.15.53:0/1185123611 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2015db30 con 0x7faa2015e330 2023-10-21T16:17:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.418+0000 7faa25b13640 1 --2- 172.21.15.53:0/1185123611 >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 0x7faa200702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:10.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.419+0000 7faa26b15640 1 --2- 172.21.15.53:0/1185123611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa2015e330 0x7faa20070810 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faa200b4970 tx=0x7faa1802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abee311195f8e11b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.420+0000 7faa25312640 1 -- 172.21.15.53:0/1185123611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1802ca80 con 0x7faa2015e330 2023-10-21T16:17:10.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.420+0000 7faa25312640 1 -- 172.21.15.53:0/1185123611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1802cbe0 con 0x7faa2015e330 2023-10-21T16:17:10.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.420+0000 7faa25312640 1 -- 172.21.15.53:0/1185123611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180343f0 con 0x7faa2015e330 2023-10-21T16:17:10.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.420+0000 7faa28578640 1 -- 172.21.15.53:0/1185123611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa2015e330 msgr2=0x7faa20070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.420+0000 7faa28578640 1 --2- 172.21.15.53:0/1185123611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa2015e330 0x7faa20070810 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7faa200b4970 tx=0x7faa1802bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 -- 172.21.15.53:0/1185123611 shutdown_connections 2023-10-21T16:17:10.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 --2- 172.21.15.53:0/1185123611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa2015e330 0x7faa20070810 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 --2- 172.21.15.53:0/1185123611 >> v2:172.21.15.124:3300/0 conn(0x7faa200ad790 0x7faa200702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 -- 172.21.15.53:0/1185123611 >> 172.21.15.53:0/1185123611 conn(0x7faa2006a380 msgr2=0x7faa200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:10.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 -- 172.21.15.53:0/1185123611 shutdown_connections 2023-10-21T16:17:10.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.421+0000 7faa28578640 1 -- 172.21.15.53:0/1185123611 wait complete. 2023-10-21T16:17:10.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.423+0000 7faa28578640 1 Processor -- start 2023-10-21T16:17:10.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.423+0000 7faa28578640 1 -- start start 2023-10-21T16:17:10.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 0x7faa200ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 0x7faa200af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20158bc0 con 0x7faa20159ac0 2023-10-21T16:17:10.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2014c030 con 0x7faa2015e330 2023-10-21T16:17:10.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa28578640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20158600 con 0x7faa200ad790 2023-10-21T16:17:10.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa25b13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa25b13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47566/0 (socket says 172.21.15.53:47566) 2023-10-21T16:17:10.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa25b13640 1 -- 172.21.15.53:0/804324982 learned_addr learned my addr 172.21.15.53:0/804324982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:10.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa26314640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 0x7faa200ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.424+0000 7faa26b15640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 0x7faa200af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 -- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 msgr2=0x7faa200ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 0x7faa200ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 -- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 msgr2=0x7faa200af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 0x7faa200af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 -- 172.21.15.53:0/804324982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18009b30 con 0x7faa20159ac0 2023-10-21T16:17:10.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa26b15640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 0x7faa200af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:10.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa26314640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 0x7faa200ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:10.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa25b13640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7f0 tx=0x7faa1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10007a30 con 0x7faa20159ac0 2023-10-21T16:17:10.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa10007b90 con 0x7faa20159ac0 2023-10-21T16:17:10.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201588e0 con 0x7faa20159ac0 2023-10-21T16:17:10.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000ce40 con 0x7faa20159ac0 2023-10-21T16:17:10.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.425+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200cd4f0 con 0x7faa20159ac0 2023-10-21T16:17:10.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.426+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2014d680 con 0x7faa20159ac0 2023-10-21T16:17:10.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.428+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7faa100113f0 con 0x7faa20159ac0 2023-10-21T16:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.428+0000 7faa16ffd640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 0x7faa0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.429+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7faa10089ee0 con 0x7faa20159ac0 2023-10-21T16:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.429+0000 7faa26314640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 0x7faa0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.430+0000 7faa26314640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 0x7faa0406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa1c003eb0 tx=0x7faa1c009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.433+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa2014d680 con 0x7faa20159ac0 2023-10-21T16:17:10.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.670+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa2014da60 con 0x7faa20159ac0 2023-10-21T16:17:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.671+0000 7faa16ffd640 1 -- 172.21.15.53:0/804324982 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7faa2014da60 con 0x7faa20159ac0 2023-10-21T16:17:10.673 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:10.673 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:15:14.581274+0000","modified":"2023-10-21T16:15:17.733529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6126},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6126":{"gid":6126,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3351749776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3351749776},{"type":"v1","addr":"172.21.15.53:6835","nonce":3351749776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-21T16:17:10.674 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 27 2023-10-21T16:17:10.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.674+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 msgr2=0x7faa0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.674+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 0x7faa0406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa1c003eb0 tx=0x7faa1c009040 comp rx=0 tx=0).stop 2023-10-21T16:17:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 msgr2=0x7faa2013fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c7f0 tx=0x7faa1000cbf0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 shutdown_connections 2023-10-21T16:17:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faa0406af00 0x7faa0406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa2015e330 0x7faa200af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa20159ac0 0x7faa2013fd70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 --2- 172.21.15.53:0/804324982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa200ad790 0x7faa200ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 >> 172.21.15.53:0/804324982 conn(0x7faa2006a380 msgr2=0x7faa200cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 shutdown_connections 2023-10-21T16:17:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.675+0000 7faa28578640 1 -- 172.21.15.53:0/804324982 wait complete. 2023-10-21T16:17:10.694 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.925+0000 7fe828da5640 1 Processor -- start 2023-10-21T16:17:10.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.925+0000 7fe828da5640 1 -- start start 2023-10-21T16:17:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe828da5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 0x7fe82415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe828da5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8240adc70 0x7fe8240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe828da5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82415f7d0 con 0x7fe82415af10 2023-10-21T16:17:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe828da5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82415f210 con 0x7fe8240adc70 2023-10-21T16:17:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe828da5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82415f4f0 con 0x7fe82415ab30 2023-10-21T16:17:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe81bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8240adc70 0x7fe8240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe81bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8240adc70 0x7fe8240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47582/0 (socket says 172.21.15.53:47582) 2023-10-21T16:17:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.927+0000 7fe81bfff640 1 -- 172.21.15.53:0/2218328653 learned_addr learned my addr 172.21.15.53:0/2218328653 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe8237fe640 1 --2- 172.21.15.53:0/2218328653 >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 0x7fe82415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe81bfff640 1 -- 172.21.15.53:0/2218328653 >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 msgr2=0x7fe82415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe81bfff640 1 --2- 172.21.15.53:0/2218328653 >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 0x7fe82415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe81bfff640 1 -- 172.21.15.53:0/2218328653 >> v1:172.21.15.124:6789/0 conn(0x7fe82415af10 legacy=0x7fe8240701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe81bfff640 1 -- 172.21.15.53:0/2218328653 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82406c550 con 0x7fe8240adc70 2023-10-21T16:17:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.928+0000 7fe8237fe640 1 --2- 172.21.15.53:0/2218328653 >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 0x7fe82415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.929+0000 7fe81bfff640 1 --2- 172.21.15.53:0/2218328653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8240adc70 0x7fe8240b00c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe8240c56c0 tx=0x7fe80c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f923555f194319 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.929+0000 7fe822ffd640 1 -- 172.21.15.53:0/2218328653 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c031640 con 0x7fe8240adc70 2023-10-21T16:17:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.929+0000 7fe822ffd640 1 -- 172.21.15.53:0/2218328653 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c0317a0 con 0x7fe8240adc70 2023-10-21T16:17:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.929+0000 7fe822ffd640 1 -- 172.21.15.53:0/2218328653 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c031b30 con 0x7fe8240adc70 2023-10-21T16:17:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 -- 172.21.15.53:0/2218328653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8240adc70 msgr2=0x7fe8240b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 --2- 172.21.15.53:0/2218328653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8240adc70 0x7fe8240b00c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe8240c56c0 tx=0x7fe80c02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 -- 172.21.15.53:0/2218328653 shutdown_connections 2023-10-21T16:17:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 --2- 172.21.15.53:0/2218328653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8240adc70 0x7fe8240b00c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 --2- 172.21.15.53:0/2218328653 >> v2:172.21.15.124:3300/0 conn(0x7fe82415ab30 0x7fe82415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 -- 172.21.15.53:0/2218328653 >> 172.21.15.53:0/2218328653 conn(0x7fe8240cb0c0 msgr2=0x7fe8240cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 -- 172.21.15.53:0/2218328653 shutdown_connections 2023-10-21T16:17:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.930+0000 7fe828da5640 1 -- 172.21.15.53:0/2218328653 wait complete. 2023-10-21T16:17:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.932+0000 7fe828da5640 1 Processor -- start 2023-10-21T16:17:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.932+0000 7fe828da5640 1 -- start start 2023-10-21T16:17:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 0x7fe8240c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe82415ab30 0x7fe82415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c6e20 con 0x7fe82415af10 2023-10-21T16:17:10.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c7100 con 0x7fe8240adc70 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.933+0000 7fe828da5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c73e0 con 0x7fe82415ab30 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47588/0 (socket says 172.21.15.53:47588) 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 -- 172.21.15.53:0/61172885 learned_addr learned my addr 172.21.15.53:0/61172885 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe8237fe640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 0x7fe8240c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe81bfff640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe82415ab30 0x7fe82415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 -- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe82415ab30 msgr2=0x7fe82415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe82415ab30 0x7fe82415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 -- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 msgr2=0x7fe8240c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 0x7fe8240c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 -- 172.21.15.53:0/61172885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0094d0 con 0x7fe82415af10 2023-10-21T16:17:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe8237fe640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 0x7fe8240c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.934+0000 7fe823fff640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00b970 tx=0x7fe81c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.935+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c00cb40 con 0x7fe82415af10 2023-10-21T16:17:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.935+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c00cca0 con 0x7fe82415af10 2023-10-21T16:17:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.935+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8240c76c0 con 0x7fe82415af10 2023-10-21T16:17:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.935+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c014680 con 0x7fe82415af10 2023-10-21T16:17:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.935+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82415e110 con 0x7fe82415af10 2023-10-21T16:17:10.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.936+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8240c7be0 con 0x7fe82415af10 2023-10-21T16:17:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.937+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe81c01d3b0 con 0x7fe82415af10 2023-10-21T16:17:10.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.938+0000 7fe820ff9640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 0x7fe80406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.938+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fe81c0899e0 con 0x7fe82415af10 2023-10-21T16:17:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.939+0000 7fe8237fe640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 0x7fe80406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.940+0000 7fe8237fe640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 0x7fe80406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8240cc780 tx=0x7fe814008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:10.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:10.942+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8240c7be0 con 0x7fe82415af10 2023-10-21T16:17:11.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.137+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8240c7be0 con 0x7fe82415af10 2023-10-21T16:17:11.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.140+0000 7fe820ff9640 1 -- 172.21.15.53:0/61172885 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18010 (secure 0 0 0) 0x7fe8240c7be0 con 0x7fe82415af10 2023-10-21T16:17:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:11.142 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:07.486639+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:15:12.546220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-21T16:15:13.608148+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.32","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1823284108":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/439129464":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6833/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/2911894415":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1304233815":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6832/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/3052410171":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1876804894":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/473879295":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/3508742564":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/2309645985":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/4037905816":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/1978958043":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/102098566":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6833/3288323666":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6832/3288323666":"2023-10-22T16:15:29.377682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.142+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 msgr2=0x7fe80406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.142+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 0x7fe80406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8240cc780 tx=0x7fe814008040 comp rx=0 tx=0).stop 2023-10-21T16:17:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.142+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 msgr2=0x7fe82415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.142+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00b970 tx=0x7fe81c00be30 comp rx=0 tx=0).stop 2023-10-21T16:17:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 shutdown_connections 2023-10-21T16:17:11.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe80406af00 0x7fe80406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe82415af10 0x7fe82415c360 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe82415ab30 0x7fe82415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 --2- 172.21.15.53:0/61172885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8240adc70 0x7fe8240c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 >> 172.21.15.53:0/61172885 conn(0x7fe8240cb0c0 msgr2=0x7fe8240acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 shutdown_connections 2023-10-21T16:17:11.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.143+0000 7fe828da5640 1 -- 172.21.15.53:0/61172885 wait complete. 2023-10-21T16:17:11.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:17:11.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.403+0000 7f45c4821640 1 Processor -- start 2023-10-21T16:17:11.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.403+0000 7f45c4821640 1 -- start start 2023-10-21T16:17:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.404+0000 7f45c4821640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f45c00cc9b0 0x7f45c006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.404+0000 7f45c4821640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.404+0000 7f45c4821640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c006b6f0 con 0x7f45c006d590 2023-10-21T16:17:11.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.404+0000 7f45c4821640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c006d100 con 0x7f45c00cc9b0 2023-10-21T16:17:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.404+0000 7f45c4821640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c015a690 con 0x7f45c015a1e0 2023-10-21T16:17:11.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.405+0000 7f45bffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.405+0000 7f45bffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:55846/0 (socket says 172.21.15.53:55846) 2023-10-21T16:17:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.405+0000 7f45bffff640 1 -- 172.21.15.53:0/1804124539 learned_addr learned my addr 172.21.15.53:0/1804124539 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.405+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/1804124539 >> v2:172.21.15.124:3300/0 conn(0x7f45c00cc9b0 0x7f45c006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.406+0000 7f45bf7fe640 1 -- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 msgr2=0x7f45c0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:11.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.406+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.406+0000 7f45bf7fe640 1 -- 172.21.15.53:0/1804124539 >> v1:172.21.15.124:6789/0 conn(0x7f45c006d590 legacy=0x7f45c00ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.406+0000 7f45bf7fe640 1 -- 172.21.15.53:0/1804124539 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0159550 con 0x7f45c00cc9b0 2023-10-21T16:17:11.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.406+0000 7f45bffff640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:11.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.408+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c00cc9b0 0x7f45c006dee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009d90 tx=0x7f45b002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163e7f34e9293cca server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.408+0000 7f45be7fc640 1 -- 172.21.15.53:0/1804124539 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b002cc60 con 0x7f45c00cc9b0 2023-10-21T16:17:11.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.408+0000 7f45be7fc640 1 -- 172.21.15.53:0/1804124539 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b002cdc0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.408+0000 7f45be7fc640 1 -- 172.21.15.53:0/1804124539 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0034660 con 0x7f45c00cc9b0 2023-10-21T16:17:11.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 -- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c00cc9b0 msgr2=0x7f45c006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c00cc9b0 0x7f45c006dee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009d90 tx=0x7f45b002beb0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 -- 172.21.15.53:0/1804124539 shutdown_connections 2023-10-21T16:17:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c015a1e0 0x7f45c0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 --2- 172.21.15.53:0/1804124539 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c00cc9b0 0x7f45c006dee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 -- 172.21.15.53:0/1804124539 >> 172.21.15.53:0/1804124539 conn(0x7f45c00b08d0 msgr2=0x7f45c00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:11.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 -- 172.21.15.53:0/1804124539 shutdown_connections 2023-10-21T16:17:11.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.409+0000 7f45c4821640 1 -- 172.21.15.53:0/1804124539 wait complete. 2023-10-21T16:17:11.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.411+0000 7f45c4821640 1 Processor -- start 2023-10-21T16:17:11.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.412+0000 7f45c4821640 1 -- start start 2023-10-21T16:17:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c006d590 0x7f45c00afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 0x7f45c0166800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00cae40 con 0x7f45c00cc9b0 2023-10-21T16:17:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00ca2c0 con 0x7f45c015a1e0 2023-10-21T16:17:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45c4821640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00ca5a0 con 0x7f45c006d590 2023-10-21T16:17:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45beffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47594/0 (socket says 172.21.15.53:47594) 2023-10-21T16:17:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.413+0000 7f45beffd640 1 -- 172.21.15.53:0/2711973293 learned_addr learned my addr 172.21.15.53:0/2711973293 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c006d590 0x7f45c00afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45bffff640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 0x7f45c0166800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 -- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c006d590 msgr2=0x7f45c00afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c006d590 0x7f45c00afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 -- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 msgr2=0x7f45c0166800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 0x7f45c0166800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 -- 172.21.15.53:0/2711973293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80094d0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45bffff640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 0x7f45c0166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45beffd640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c440 tx=0x7f45a800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.414+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80052d0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.415+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8005430 con 0x7f45c00cc9b0 2023-10-21T16:17:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.415+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8007260 con 0x7f45c00cc9b0 2023-10-21T16:17:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.415+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b0009a70 con 0x7f45c00cc9b0 2023-10-21T16:17:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.415+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c00cab60 con 0x7f45c00cc9b0 2023-10-21T16:17:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.416+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c0074150 con 0x7f45c00cc9b0 2023-10-21T16:17:11.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.417+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f45a80073c0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.418+0000 7f45b7fff640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 0x7f45a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.419+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f45a8088d50 con 0x7f45c00cc9b0 2023-10-21T16:17:11.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.419+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 0x7f45a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.420+0000 7f45bf7fe640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 0x7f45a006d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45b0036000 tx=0x7f45b0030000 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.424+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45a80538a0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.660+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f45c006b6f0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.663 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:17:11.662+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:17:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.665+0000 7f45b7fff640 1 -- 172.21.15.53:0/2711973293 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f45c006b6f0 con 0x7f45c00cc9b0 2023-10-21T16:17:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:17:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 msgr2=0x7f45a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 0x7f45a006d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45b0036000 tx=0x7f45b0030000 comp rx=0 tx=0).stop 2023-10-21T16:17:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 msgr2=0x7f45c01662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c440 tx=0x7f45a800c900 comp rx=0 tx=0).stop 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 shutdown_connections 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f45a006af00 0x7f45a006d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f45c015a1e0 0x7f45c0166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45c00cc9b0 0x7f45c01662c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 --2- 172.21.15.53:0/2711973293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45c006d590 0x7f45c00afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.669+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 >> 172.21.15.53:0/2711973293 conn(0x7f45c00b08d0 msgr2=0x7f45c00b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.670+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 shutdown_connections 2023-10-21T16:17:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.670+0000 7f45c4821640 1 -- 172.21.15.53:0/2711973293 wait complete. 2023-10-21T16:17:11.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:17:11.761 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:17:11.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.889+0000 7f6adbfff640 1 Processor -- start 2023-10-21T16:17:11.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.889+0000 7f6adbfff640 1 -- start start 2023-10-21T16:17:11.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6adc0d00e0 0x7f6adc0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adbfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc0d2870 con 0x7f6adc0d0b90 2023-10-21T16:17:11.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adbfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0d1150 con 0x7f6adc15ccb0 2023-10-21T16:17:11.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adbfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0d1430 con 0x7f6adc0d00e0 2023-10-21T16:17:11.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.891+0000 7f6adaffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47622/0 (socket says 172.21.15.53:47622) 2023-10-21T16:17:11.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 -- 172.21.15.53:0/3799715682 learned_addr learned my addr 172.21.15.53:0/3799715682 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:11.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adb7fe640 1 --2- 172.21.15.53:0/3799715682 >> v2:172.21.15.124:3300/0 conn(0x7f6adc0d00e0 0x7f6adc0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 -- 172.21.15.53:0/3799715682 >> v2:172.21.15.124:3300/0 conn(0x7f6adc0d00e0 msgr2=0x7f6adc0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:11.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 --2- 172.21.15.53:0/3799715682 >> v2:172.21.15.124:3300/0 conn(0x7f6adc0d00e0 0x7f6adc0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 -- 172.21.15.53:0/3799715682 >> v1:172.21.15.124:6789/0 conn(0x7f6adc0d0b90 legacy=0x7f6adc0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.892+0000 7f6adaffd640 1 -- 172.21.15.53:0/3799715682 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0d1710 con 0x7f6adc15ccb0 2023-10-21T16:17:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.893+0000 7f6adaffd640 1 --2- 172.21.15.53:0/3799715682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0cd1c0 tx=0x7f6acc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedb2a9452271d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.893+0000 7f6ada7fc640 1 -- 172.21.15.53:0/3799715682 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0317e0 con 0x7f6adc15ccb0 2023-10-21T16:17:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6ada7fc640 1 -- 172.21.15.53:0/3799715682 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc031940 con 0x7f6adc15ccb0 2023-10-21T16:17:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6ada7fc640 1 -- 172.21.15.53:0/3799715682 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc031cd0 con 0x7f6adc15ccb0 2023-10-21T16:17:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 -- 172.21.15.53:0/3799715682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc15ccb0 msgr2=0x7f6adc0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 --2- 172.21.15.53:0/3799715682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0cd1c0 tx=0x7f6acc02b880 comp rx=0 tx=0).stop 2023-10-21T16:17:11.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 -- 172.21.15.53:0/3799715682 shutdown_connections 2023-10-21T16:17:11.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 --2- 172.21.15.53:0/3799715682 >> v2:172.21.15.124:3300/0 conn(0x7f6adc0d00e0 0x7f6adc0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 --2- 172.21.15.53:0/3799715682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc15ccb0 0x7f6adc0d1fb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 -- 172.21.15.53:0/3799715682 >> 172.21.15.53:0/3799715682 conn(0x7f6adc071660 msgr2=0x7f6adc071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 -- 172.21.15.53:0/3799715682 shutdown_connections 2023-10-21T16:17:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.894+0000 7f6adbfff640 1 -- 172.21.15.53:0/3799715682 wait complete. 2023-10-21T16:17:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.896+0000 7f6adbfff640 1 Processor -- start 2023-10-21T16:17:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.896+0000 7f6adbfff640 1 -- start start 2023-10-21T16:17:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 0x7f6adc0c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6adc15ccb0 0x7f6adc0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0c7c20 con 0x7f6adc0d00e0 2023-10-21T16:17:11.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0c7790 con 0x7f6adc0d0b90 2023-10-21T16:17:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0c6b80 con 0x7f6adc15ccb0 2023-10-21T16:17:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47628/0 (socket says 172.21.15.53:47628) 2023-10-21T16:17:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.897+0000 7f6adaffd640 1 -- 172.21.15.53:0/1794162184 learned_addr learned my addr 172.21.15.53:0/1794162184 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6ad3fff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 0x7f6adc0c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adb7fe640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6adc15ccb0 0x7f6adc0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 -- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6adc15ccb0 msgr2=0x7f6adc0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6adc15ccb0 0x7f6adc0b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 -- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 msgr2=0x7f6adc0c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 0x7f6adc0c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 -- 172.21.15.53:0/1794162184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0094d0 con 0x7f6adc0d00e0 2023-10-21T16:17:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6ad3fff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 0x7f6adc0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.898+0000 7f6adaffd640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6acc02b8c0 tx=0x7f6acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.899+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0329c0 con 0x7f6adc0d00e0 2023-10-21T16:17:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.899+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc032b40 con 0x7f6adc0d00e0 2023-10-21T16:17:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.899+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0453f0 con 0x7f6adc0d00e0 2023-10-21T16:17:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.899+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc0c6e60 con 0x7f6adc0d00e0 2023-10-21T16:17:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.899+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc0c7420 con 0x7f6adc0d00e0 2023-10-21T16:17:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.900+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc074150 con 0x7f6adc0d00e0 2023-10-21T16:17:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.901+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6acc04d3f0 con 0x7f6adc0d00e0 2023-10-21T16:17:11.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.902+0000 7f6ad8ff9640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 0x7f6abc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:11.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.902+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f6acc0abb80 con 0x7f6adc0d00e0 2023-10-21T16:17:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.903+0000 7f6ad3fff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 0x7f6abc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.904+0000 7f6ad3fff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 0x7f6abc06d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6adc06ba70 tx=0x7f6ac4009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:11.906+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6acc0766c0 con 0x7f6adc0d00e0 2023-10-21T16:17:12.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.135+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6adc06bda0 con 0x7f6adc0d00e0 2023-10-21T16:17:12.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.299+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6acc076a90 con 0x7f6adc0d00e0 2023-10-21T16:17:12.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.668+0000 7f6ad8ff9640 1 -- 172.21.15.53:0/1794162184 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6acc090210 con 0x7f6adc0d00e0 2023-10-21T16:17:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 msgr2=0x7f6abc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 0x7f6abc06d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6adc06ba70 tx=0x7f6ac4009040 comp rx=0 tx=0).stop 2023-10-21T16:17:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 msgr2=0x7f6adc159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6acc02b8c0 tx=0x7f6acc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 shutdown_connections 2023-10-21T16:17:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f6abc06af30 0x7f6abc06d3f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6adc15ccb0 0x7f6adc0b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6adc0d0b90 0x7f6adc0c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 --2- 172.21.15.53:0/1794162184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6adc0d00e0 0x7f6adc159470 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 >> 172.21.15.53:0/1794162184 conn(0x7f6adc071660 msgr2=0x7f6adc0cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:12.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 shutdown_connections 2023-10-21T16:17:12.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.672+0000 7f6adbfff640 1 -- 172.21.15.53:0/1794162184 wait complete. 2023-10-21T16:17:12.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:17:12.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.923+0000 7fdfef59e640 1 Processor -- start 2023-10-21T16:17:12.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.923+0000 7fdfef59e640 1 -- start start 2023-10-21T16:17:12.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfef59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdff01599d0 0x7fdff00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfef59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdff00cb5e0 0x7fdff00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfef59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff0159db0 con 0x7fdff00b3c50 2023-10-21T16:17:12.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfef59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff0071fe0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfef59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00b44c0 con 0x7fdff01599d0 2023-10-21T16:17:12.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfe7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdff00b3c50 0x7fdff00caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43556/0 (socket says 172.21.15.53:43556) 2023-10-21T16:17:12.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfe7fff640 1 -- 172.21.15.53:0/531577935 learned_addr learned my addr 172.21.15.53:0/531577935 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:12.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.925+0000 7fdfeed9d640 1 --2- 172.21.15.53:0/531577935 >> v2:172.21.15.53:3300/0 conn(0x7fdff00cb5e0 0x7fdff00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:12.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.926+0000 7fdfee59c640 1 --2- 172.21.15.53:0/531577935 >> v2:172.21.15.124:3300/0 conn(0x7fdff01599d0 0x7fdff00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:12.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.926+0000 7fdfeed9d640 1 -- 172.21.15.53:0/531577935 >> v2:172.21.15.124:3300/0 conn(0x7fdff01599d0 msgr2=0x7fdff00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:12.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.926+0000 7fdfeed9d640 1 --2- 172.21.15.53:0/531577935 >> v2:172.21.15.124:3300/0 conn(0x7fdff01599d0 0x7fdff00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.926+0000 7fdfeed9d640 1 -- 172.21.15.53:0/531577935 >> v1:172.21.15.53:6789/0 conn(0x7fdff00b3c50 legacy=0x7fdff00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.926+0000 7fdfeed9d640 1 -- 172.21.15.53:0/531577935 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00713d0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.927+0000 7fdfeed9d640 1 --2- 172.21.15.53:0/531577935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00cc0c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdff006e150 tx=0x7fdfe802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9ae07859a45671 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfedd9b640 1 -- 172.21.15.53:0/531577935 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe802cc80 con 0x7fdff00cb5e0 2023-10-21T16:17:12.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfedd9b640 1 -- 172.21.15.53:0/531577935 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe802cde0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfedd9b640 1 -- 172.21.15.53:0/531577935 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8034660 con 0x7fdff00cb5e0 2023-10-21T16:17:12.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 -- 172.21.15.53:0/531577935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 msgr2=0x7fdff00cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 --2- 172.21.15.53:0/531577935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00cc0c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdff006e150 tx=0x7fdfe802bed0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 -- 172.21.15.53:0/531577935 shutdown_connections 2023-10-21T16:17:12.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 --2- 172.21.15.53:0/531577935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00cc0c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 --2- 172.21.15.53:0/531577935 >> v2:172.21.15.124:3300/0 conn(0x7fdff01599d0 0x7fdff00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 -- 172.21.15.53:0/531577935 >> 172.21.15.53:0/531577935 conn(0x7fdff006c050 msgr2=0x7fdff006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.928+0000 7fdfef59e640 1 -- 172.21.15.53:0/531577935 shutdown_connections 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.929+0000 7fdfef59e640 1 -- 172.21.15.53:0/531577935 wait complete. 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.930+0000 7fdfef59e640 1 Processor -- start 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.930+0000 7fdfef59e640 1 -- start start 2023-10-21T16:17:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00b3c50 0x7fdff00c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 0x7fdff00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00c8d60 con 0x7fdff00cb5e0 2023-10-21T16:17:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b3580 con 0x7fdff01599d0 2023-10-21T16:17:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.931+0000 7fdfef59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff014ea80 con 0x7fdff00b3c50 2023-10-21T16:17:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47658/0 (socket says 172.21.15.53:47658) 2023-10-21T16:17:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2821393015 learned_addr learned my addr 172.21.15.53:0/2821393015 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfeed9d640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 0x7fdff00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00b3c50 msgr2=0x7fdff00c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00b3c50 0x7fdff00c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 msgr2=0x7fdff00c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 0x7fdff00c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2821393015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009a70 con 0x7fdff00cb5e0 2023-10-21T16:17:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfeed9d640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 0x7fdff00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.932+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c4a0 tx=0x7fdfd800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.933+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8005ce0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.933+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8005e40 con 0x7fdff00cb5e0 2023-10-21T16:17:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.933+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff00b2fc0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.933+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8007270 con 0x7fdff00cb5e0 2023-10-21T16:17:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.933+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff00c5760 con 0x7fdff00cb5e0 2023-10-21T16:17:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.934+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff0074150 con 0x7fdff00cb5e0 2023-10-21T16:17:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.935+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdfd8007490 con 0x7fdff00cb5e0 2023-10-21T16:17:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.936+0000 7fdfe6ffd640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 0x7fdfd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.936+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fdfd8088fc0 con 0x7fdff00cb5e0 2023-10-21T16:17:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.937+0000 7fdfee59c640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 0x7fdfd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.938+0000 7fdfee59c640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 0x7fdfd006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006a90 tx=0x7fdfe0006a20 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:12.940+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfd8053ad0 con 0x7fdff00cb5e0 2023-10-21T16:17:13.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.126+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdff0159db0 con 0x7fdff00cb5e0 2023-10-21T16:17:13.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.672+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/2821393015 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdff0159db0 con 0x7fdff00cb5e0 2023-10-21T16:17:13.675 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:17:13.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 msgr2=0x7fdfd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:13.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 0x7fdfd006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0006a90 tx=0x7fdfe0006a20 comp rx=0 tx=0).stop 2023-10-21T16:17:13.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 msgr2=0x7fdff00b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:13.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c4a0 tx=0x7fdfd800c960 comp rx=0 tx=0).stop 2023-10-21T16:17:13.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 shutdown_connections 2023-10-21T16:17:13.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fdfd006af90 0x7fdfd006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdff01599d0 0x7fdff00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdff00cb5e0 0x7fdff00b2950 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 --2- 172.21.15.53:0/2821393015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdff00b3c50 0x7fdff00c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.676+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 >> 172.21.15.53:0/2821393015 conn(0x7fdff006c050 msgr2=0x7fdff015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:13.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.677+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 shutdown_connections 2023-10-21T16:17:13.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.677+0000 7fdfef59e640 1 -- 172.21.15.53:0/2821393015 wait complete. 2023-10-21T16:17:13.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:17:13.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.935+0000 7f84c5b1d640 1 Processor -- start 2023-10-21T16:17:13.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.936+0000 7f84c5b1d640 1 -- start start 2023-10-21T16:17:13.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.937+0000 7f84c5b1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 0x7f84c0161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.937+0000 7f84c5b1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f84c00cd3f0 0x7f84c00d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.937+0000 7f84c5b1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c00cdd90 con 0x7f84c015cc60 2023-10-21T16:17:13.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.937+0000 7f84c5b1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00ce070 con 0x7f84c00cd3f0 2023-10-21T16:17:13.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.937+0000 7f84c5b1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c00d1030 con 0x7f84c00b4d20 2023-10-21T16:17:13.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f84c00cd3f0 0x7f84c00d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f84c00cd3f0 0x7f84c00d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47674/0 (socket says 172.21.15.53:47674) 2023-10-21T16:17:13.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 -- 172.21.15.53:0/259662075 learned_addr learned my addr 172.21.15.53:0/259662075 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:13.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84b7fff640 1 --2- 172.21.15.53:0/259662075 >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 0x7f84c0161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 -- 172.21.15.53:0/259662075 >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 msgr2=0x7f84c0161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:13.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 --2- 172.21.15.53:0/259662075 >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 0x7f84c0161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 -- 172.21.15.53:0/259662075 >> v1:172.21.15.124:6789/0 conn(0x7f84c015cc60 legacy=0x7f84c00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:13.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84bffff640 1 -- 172.21.15.53:0/259662075 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00d1310 con 0x7f84c00cd3f0 2023-10-21T16:17:13.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.938+0000 7f84b7fff640 1 --2- 172.21.15.53:0/259662075 >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 0x7f84c0161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:13.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84bffff640 1 --2- 172.21.15.53:0/259662075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c00d1830 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f84c00cf0a0 tx=0x7f84b802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207758fa75216c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:13.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84beffd640 1 -- 172.21.15.53:0/259662075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b802cbb0 con 0x7f84c00cd3f0 2023-10-21T16:17:13.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84beffd640 1 -- 172.21.15.53:0/259662075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b802cd10 con 0x7f84c00cd3f0 2023-10-21T16:17:13.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84beffd640 1 -- 172.21.15.53:0/259662075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8034610 con 0x7f84c00cd3f0 2023-10-21T16:17:13.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84c5b1d640 1 -- 172.21.15.53:0/259662075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 msgr2=0x7f84c00d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:13.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.940+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/259662075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c00d1830 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f84c00cf0a0 tx=0x7f84b802c010 comp rx=0 tx=0).stop 2023-10-21T16:17:13.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 -- 172.21.15.53:0/259662075 shutdown_connections 2023-10-21T16:17:13.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/259662075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c00d1830 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/259662075 >> v2:172.21.15.124:3300/0 conn(0x7f84c00b4d20 0x7f84c0161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 -- 172.21.15.53:0/259662075 >> 172.21.15.53:0/259662075 conn(0x7f84c00cf610 msgr2=0x7f84c00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:13.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 -- 172.21.15.53:0/259662075 shutdown_connections 2023-10-21T16:17:13.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.941+0000 7f84c5b1d640 1 -- 172.21.15.53:0/259662075 wait complete. 2023-10-21T16:17:13.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.942+0000 7f84c5b1d640 1 Processor -- start 2023-10-21T16:17:13.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.943+0000 7f84c5b1d640 1 -- start start 2023-10-21T16:17:13.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.943+0000 7f84c5b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 0x7f84c015d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.943+0000 7f84c5b1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.943+0000 7f84c5b1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c015cc60 0x7f84c00c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84c5b1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00cb0d0 con 0x7f84c00cd3f0 2023-10-21T16:17:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84c5b1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00cb970 con 0x7f84c015cc60 2023-10-21T16:17:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84c5b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00c7e70 con 0x7f84c00b4d20 2023-10-21T16:17:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47684/0 (socket says 172.21.15.53:47684) 2023-10-21T16:17:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 -- 172.21.15.53:0/1491696494 learned_addr learned my addr 172.21.15.53:0/1491696494 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 -- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 msgr2=0x7f84c015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84bf7fe640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 0x7f84c015d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84bffff640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c015cc60 0x7f84c00c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 0x7f84c015d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 -- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c015cc60 msgr2=0x7f84c00c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c015cc60 0x7f84c00c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84b7fff640 1 -- 172.21.15.53:0/1491696494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8009b30 con 0x7f84c00cd3f0 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.944+0000 7f84bf7fe640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 0x7f84c015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84b7fff640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c820 tx=0x7f84a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004230 con 0x7f84c00cd3f0 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a8004390 con 0x7f84c00cd3f0 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c00cb320 con 0x7f84c00cd3f0 2023-10-21T16:17:13.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8007a70 con 0x7f84c00cd3f0 2023-10-21T16:17:13.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.945+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c01f1f70 con 0x7f84c00cd3f0 2023-10-21T16:17:13.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.946+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c004e160 con 0x7f84c00cd3f0 2023-10-21T16:17:13.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.947+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84a8007bd0 con 0x7f84c00cd3f0 2023-10-21T16:17:13.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.948+0000 7f84bcff9640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 0x7f84a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:13.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.948+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f84a8089c20 con 0x7f84c00cd3f0 2023-10-21T16:17:13.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.949+0000 7f84bf7fe640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 0x7f84a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:13.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.950+0000 7f84bf7fe640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 0x7f84a006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000f60 tx=0x7f84b0009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:13.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:13.952+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84a8054710 con 0x7f84c00cd3f0 2023-10-21T16:17:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.151+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84c00cbab0 con 0x7f84c00cd3f0 2023-10-21T16:17:14.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.692+0000 7f84bcff9640 1 -- 172.21.15.53:0/1491696494 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7f84a801e0d0 con 0x7f84c00cd3f0 2023-10-21T16:17:14.694 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:17:14.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.697+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 msgr2=0x7f84a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.697+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 0x7f84a006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f84b0000f60 tx=0x7f84b0009040 comp rx=0 tx=0).stop 2023-10-21T16:17:14.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.697+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 msgr2=0x7f84c01f1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.697+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f84a800c820 tx=0x7f84a800cce0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 shutdown_connections 2023-10-21T16:17:14.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f84a006af90 0x7f84a006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84c015cc60 0x7f84c00c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84c00cd3f0 0x7f84c01f1a30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 --2- 172.21.15.53:0/1491696494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84c00b4d20 0x7f84c015d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 >> 172.21.15.53:0/1491696494 conn(0x7f84c00cf610 msgr2=0x7f84c00b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:14.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 shutdown_connections 2023-10-21T16:17:14.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.698+0000 7f84c5b1d640 1 -- 172.21.15.53:0/1491696494 wait complete. 2023-10-21T16:17:14.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:17:14.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.934+0000 7f1620843640 1 Processor -- start 2023-10-21T16:17:14.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.934+0000 7f1620843640 1 -- start start 2023-10-21T16:17:14.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.936+0000 7f1620843640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 0x7f161c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.936+0000 7f1620843640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.936+0000 7f1620843640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c161590 con 0x7f161c0aff70 2023-10-21T16:17:14.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.936+0000 7f1620843640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0cc570 con 0x7f161c0c18d0 2023-10-21T16:17:14.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.936+0000 7f1620843640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0c94b0 con 0x7f161c164d10 2023-10-21T16:17:14.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.937+0000 7f161b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.937+0000 7f161b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:50940/0 (socket says 172.21.15.53:50940) 2023-10-21T16:17:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.937+0000 7f161b59e640 1 -- 172.21.15.53:0/3674045925 learned_addr learned my addr 172.21.15.53:0/3674045925 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.937+0000 7f1613fff640 1 --2- 172.21.15.53:0/3674045925 >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 0x7f161c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.938+0000 7f161b59e640 1 -- 172.21.15.53:0/3674045925 >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 msgr2=0x7f161c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.938+0000 7f161b59e640 1 --2- 172.21.15.53:0/3674045925 >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 0x7f161c0c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.938+0000 7f161b59e640 1 -- 172.21.15.53:0/3674045925 >> v1:172.21.15.124:6789/0 conn(0x7f161c0aff70 legacy=0x7f161c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.938+0000 7f161b59e640 1 -- 172.21.15.53:0/3674045925 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0c9790 con 0x7f161c164d10 2023-10-21T16:17:14.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.938+0000 7f1613fff640 1 --2- 172.21.15.53:0/3674045925 >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 0x7f161c0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:14.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.939+0000 7f161b59e640 1 --2- 172.21.15.53:0/3674045925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161402be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac449edee94f590 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:14.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.940+0000 7f161a59c640 1 -- 172.21.15.53:0/3674045925 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161402c9f0 con 0x7f161c164d10 2023-10-21T16:17:14.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.940+0000 7f161a59c640 1 -- 172.21.15.53:0/3674045925 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161402cb50 con 0x7f161c164d10 2023-10-21T16:17:14.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.940+0000 7f161a59c640 1 -- 172.21.15.53:0/3674045925 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140343f0 con 0x7f161c164d10 2023-10-21T16:17:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 -- 172.21.15.53:0/3674045925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 msgr2=0x7f161c0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 --2- 172.21.15.53:0/3674045925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161402be50 comp rx=0 tx=0).stop 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 -- 172.21.15.53:0/3674045925 shutdown_connections 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 --2- 172.21.15.53:0/3674045925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c164d10 0x7f161c0ccd30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 --2- 172.21.15.53:0/3674045925 >> v2:172.21.15.124:3300/0 conn(0x7f161c0c18d0 0x7f161c0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 -- 172.21.15.53:0/3674045925 >> 172.21.15.53:0/3674045925 conn(0x7f161c06c050 msgr2=0x7f161c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 -- 172.21.15.53:0/3674045925 shutdown_connections 2023-10-21T16:17:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.941+0000 7f1620843640 1 -- 172.21.15.53:0/3674045925 wait complete. 2023-10-21T16:17:14.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.943+0000 7f1620843640 1 Processor -- start 2023-10-21T16:17:14.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.943+0000 7f1620843640 1 -- start start 2023-10-21T16:17:14.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161c0aff70 0x7f161c162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c0c18d0 0x7f161c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c1626a0 con 0x7f161c164d10 2023-10-21T16:17:14.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c1628c0 con 0x7f161c0aff70 2023-10-21T16:17:14.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1620843640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c162ba0 con 0x7f161c0c18d0 2023-10-21T16:17:14.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f161b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f161b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39516/0 (socket says 172.21.15.53:39516) 2023-10-21T16:17:14.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f161b59e640 1 -- 172.21.15.53:0/1697268941 learned_addr learned my addr 172.21.15.53:0/1697268941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:14.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f161ad9d640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161c0aff70 0x7f161c162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.945+0000 7f1613fff640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c0c18d0 0x7f161c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 -- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c0c18d0 msgr2=0x7f161c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c0c18d0 0x7f161c1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 -- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161c0aff70 msgr2=0x7f161c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:14.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161c0aff70 0x7f161c162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:14.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 -- 172.21.15.53:0/1697268941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604009400 con 0x7f161c164d10 2023-10-21T16:17:14.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f161b59e640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f1614004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:14.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161403e450 con 0x7f161c164d10 2023-10-21T16:17:14.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161403e5f0 con 0x7f161c164d10 2023-10-21T16:17:14.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161403e9a0 con 0x7f161c164d10 2023-10-21T16:17:14.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.946+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1614009b30 con 0x7f161c164d10 2023-10-21T16:17:14.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.947+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c163160 con 0x7f161c164d10 2023-10-21T16:17:14.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.948+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c074150 con 0x7f161c164d10 2023-10-21T16:17:14.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.949+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f161403eb00 con 0x7f161c164d10 2023-10-21T16:17:14.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.949+0000 7f16137fe640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 0x7f15fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:14.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.950+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f16140ac650 con 0x7f161c164d10 2023-10-21T16:17:14.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.950+0000 7f161ad9d640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 0x7f15fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:14.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.951+0000 7f161ad9d640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 0x7f15fc06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f160c000f60 tx=0x7f160c009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:14.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:14.954+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1614077430 con 0x7f161c164d10 2023-10-21T16:17:15.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.148+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f161c0ad9e0 con 0x7f161c164d10 2023-10-21T16:17:15.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.154+0000 7f16137fe640 1 -- 172.21.15.53:0/1697268941 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f161407aa50 con 0x7f161c164d10 2023-10-21T16:17:15.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.156+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 msgr2=0x7f15fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.156+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 0x7f15fc06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f160c000f60 tx=0x7f160c009040 comp rx=0 tx=0).stop 2023-10-21T16:17:15.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.156+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 msgr2=0x7f161c0c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.156+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f1614004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:15.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 shutdown_connections 2023-10-21T16:17:15.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f15fc06af90 0x7f15fc06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161c164d10 0x7f161c0c31d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161c0c18d0 0x7f161c1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 --2- 172.21.15.53:0/1697268941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161c0aff70 0x7f161c162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 >> 172.21.15.53:0/1697268941 conn(0x7f161c06c050 msgr2=0x7f161c0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:15.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 shutdown_connections 2023-10-21T16:17:15.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.157+0000 7f1620843640 1 -- 172.21.15.53:0/1697268941 wait complete. 2023-10-21T16:17:15.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:17:15.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.411+0000 7f426127e640 1 Processor -- start 2023-10-21T16:17:15.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.411+0000 7f426127e640 1 -- start start 2023-10-21T16:17:15.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f426127e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f425c0cdc30 0x7f425c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f426127e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15ac90 con 0x7f425c0cce90 2023-10-21T16:17:15.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f426127e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15a6d0 con 0x7f425c0caef0 2023-10-21T16:17:15.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f426127e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c15a9b0 con 0x7f425c0cdc30 2023-10-21T16:17:15.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f4260a7d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f425c0cce90 0x7f425c159bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52062/0 (socket says 172.21.15.53:52062) 2023-10-21T16:17:15.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.412+0000 7f4260a7d640 1 -- 172.21.15.53:0/1961073705 learned_addr learned my addr 172.21.15.53:0/1961073705 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:15.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425b7fe640 1 --2- 172.21.15.53:0/1961073705 >> v2:172.21.15.124:3300/0 conn(0x7f425c0cdc30 0x7f425c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179476902 0 0) 0x7f425c15ac90 con 0x7f425c0cce90 2023-10-21T16:17:15.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4244003520 con 0x7f425c0cce90 2023-10-21T16:17:15.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 282503791 0 0) 0x7f4244003520 con 0x7f425c0cce90 2023-10-21T16:17:15.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 >> v2:172.21.15.124:3300/0 conn(0x7f425c0cdc30 msgr2=0x7f425c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 --2- 172.21.15.53:0/1961073705 >> v2:172.21.15.124:3300/0 conn(0x7f425c0cdc30 0x7f425c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 >> v1:172.21.15.124:6789/0 conn(0x7f425c0caef0 legacy=0x7f425c0cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.413+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0cc150 con 0x7f425c0cce90 2023-10-21T16:17:15.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4254001670 con 0x7f425c0cce90 2023-10-21T16:17:15.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 >> v1:172.21.15.53:6789/0 conn(0x7f425c0cce90 legacy=0x7f425c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 0x7f4244003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440047c0 0x7f4244004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244005180 0x7f424400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c15a9b0 con 0x7f42440039c0 2023-10-21T16:17:15.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424400c080 con 0x7f4244005180 2023-10-21T16:17:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424400c240 con 0x7f42440047c0 2023-10-21T16:17:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425bfff640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 0x7f4244003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425b7fe640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440047c0 0x7f4244004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f4260a7d640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244005180 0x7f424400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425bfff640 1 -- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440047c0 msgr2=0x7f4244004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425bfff640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440047c0 0x7f4244004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425bfff640 1 -- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244005180 msgr2=0x7f424400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.414+0000 7f425bfff640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244005180 0x7f424400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.415+0000 7f425bfff640 1 -- 172.21.15.53:0/1961073705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244003520 con 0x7f42440039c0 2023-10-21T16:17:15.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.415+0000 7f425b7fe640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440047c0 0x7f4244004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:15.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.415+0000 7f4260a7d640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4244005180 0x7f424400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:15.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.415+0000 7f4260a7d640 1 -- 172.21.15.53:0/1961073705 reap_dead start 2023-10-21T16:17:15.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.416+0000 7f425bfff640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 0x7f4244003de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f424c001a30 tx=0x7f42440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95cff4f0b8afb58 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.416+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c003b70 con 0x7f42440039c0 2023-10-21T16:17:15.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.416+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c003cd0 con 0x7f42440039c0 2023-10-21T16:17:15.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.416+0000 7f425affd640 1 -- 172.21.15.53:0/1961073705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c005310 con 0x7f42440039c0 2023-10-21T16:17:15.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 -- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 msgr2=0x7f4244003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 0x7f4244003de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f424c001a30 tx=0x7f42440057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 -- 172.21.15.53:0/1961073705 shutdown_connections 2023-10-21T16:17:15.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 --2- 172.21.15.53:0/1961073705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42440039c0 0x7f4244003de0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 -- 172.21.15.53:0/1961073705 >> 172.21.15.53:0/1961073705 conn(0x7f425c06c010 msgr2=0x7f425c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:15.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 -- 172.21.15.53:0/1961073705 shutdown_connections 2023-10-21T16:17:15.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.417+0000 7f426127e640 1 -- 172.21.15.53:0/1961073705 wait complete. 2023-10-21T16:17:15.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.419+0000 7f426127e640 1 Processor -- start 2023-10-21T16:17:15.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.419+0000 7f426127e640 1 -- start start 2023-10-21T16:17:15.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440039c0 0x7f425c146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f425c0b39d0 0x7f425c0ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0adf20 con 0x7f425c156de0 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0ae170 con 0x7f425c0b39d0 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f426127e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0ae450 con 0x7f42440039c0 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39538/0 (socket says 172.21.15.53:39538) 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 -- 172.21.15.53:0/913744178 learned_addr learned my addr 172.21.15.53:0/913744178 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:15.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425bfff640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440039c0 0x7f425c146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 -- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440039c0 msgr2=0x7f425c146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440039c0 0x7f425c146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 -- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f425c0b39d0 msgr2=0x7f425c0ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f425c0b39d0 0x7f425c0ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.420+0000 7f425b7fe640 1 -- 172.21.15.53:0/913744178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0ae730 con 0x7f425c156de0 2023-10-21T16:17:15.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f425b7fe640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f42500107c0 tx=0x7f4250010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000eb60 con 0x7f425c156de0 2023-10-21T16:17:15.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4250010ed0 con 0x7f425c156de0 2023-10-21T16:17:15.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425001c400 con 0x7f425c156de0 2023-10-21T16:17:15.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c0aea10 con 0x7f425c156de0 2023-10-21T16:17:15.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.421+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c0ab820 con 0x7f425c156de0 2023-10-21T16:17:15.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.422+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c074150 con 0x7f425c156de0 2023-10-21T16:17:15.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.424+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f425001c560 con 0x7f425c156de0 2023-10-21T16:17:15.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.424+0000 7f4258ff9640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 0x7f424806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.425+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f4250089d70 con 0x7f425c156de0 2023-10-21T16:17:15.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.425+0000 7f425bfff640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 0x7f424806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.426+0000 7f425bfff640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 0x7f424806d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f424c036000 tx=0x7f424c032000 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.429+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4250054c00 con 0x7f425c156de0 2023-10-21T16:17:15.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.623+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f425c0ac5d0 con 0x7f425c156de0 2023-10-21T16:17:15.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.628+0000 7f4258ff9640 1 -- 172.21.15.53:0/913744178 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4250058220 con 0x7f425c156de0 2023-10-21T16:17:15.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 msgr2=0x7f424806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 0x7f424806d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f424c036000 tx=0x7f424c032000 comp rx=0 tx=0).stop 2023-10-21T16:17:15.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 msgr2=0x7f425c0b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f42500107c0 tx=0x7f4250010c80 comp rx=0 tx=0).stop 2023-10-21T16:17:15.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 shutdown_connections 2023-10-21T16:17:15.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f424806afc0 0x7f424806d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f425c0b39d0 0x7f425c0ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f425c156de0 0x7f425c0b3490 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 --2- 172.21.15.53:0/913744178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42440039c0 0x7f425c146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 >> 172.21.15.53:0/913744178 conn(0x7f425c06c010 msgr2=0x7f425c0a74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:15.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 shutdown_connections 2023-10-21T16:17:15.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.632+0000 7f426127e640 1 -- 172.21.15.53:0/913744178 wait complete. 2023-10-21T16:17:15.651 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:17:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.876+0000 7fa00a748640 1 Processor -- start 2023-10-21T16:17:15.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.876+0000 7fa00a748640 1 -- start start 2023-10-21T16:17:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa00a748640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa0040cdb20 0x7fa0040b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa00a748640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00415b1d0 con 0x7fa00415adf0 2023-10-21T16:17:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa00a748640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0040cf510 con 0x7fa0040d05b0 2023-10-21T16:17:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa00a748640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0040cf7f0 con 0x7fa0040cdb20 2023-10-21T16:17:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa003fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa00415adf0 0x7fa0040ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52076/0 (socket says 172.21.15.53:52076) 2023-10-21T16:17:15.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa003fff640 1 -- 172.21.15.53:0/1211096786 learned_addr learned my addr 172.21.15.53:0/1211096786 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:15.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343792655 0 0) 0x7fa00415b1d0 con 0x7fa00415adf0 2023-10-21T16:17:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.878+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fec003520 con 0x7fa00415adf0 2023-10-21T16:17:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa008ce5640 1 --2- 172.21.15.53:0/1211096786 >> v2:172.21.15.124:3300/0 conn(0x7fa0040cdb20 0x7fa0040b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2875561952 0 0) 0x7f9fec003520 con 0x7fa00415adf0 2023-10-21T16:17:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 >> v2:172.21.15.124:3300/0 conn(0x7fa0040cdb20 msgr2=0x7fa0040b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 --2- 172.21.15.53:0/1211096786 >> v2:172.21.15.124:3300/0 conn(0x7fa0040cdb20 0x7fa0040b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 >> v1:172.21.15.124:6789/0 conn(0x7fa0040d05b0 legacy=0x7fa0040cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040cfad0 con 0x7fa00415adf0 2023-10-21T16:17:15.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9ff40019f0 con 0x7fa00415adf0 2023-10-21T16:17:15.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 >> v1:172.21.15.53:6789/0 conn(0x7fa00415adf0 legacy=0x7fa0040ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0039c0 0x7f9fec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fec0047c0 0x7f9fec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 0x7f9fec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040cf7f0 con 0x7f9fec0051d0 2023-10-21T16:17:15.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec00c100 con 0x7f9fec0039c0 2023-10-21T16:17:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec00c2c0 con 0x7f9fec0047c0 2023-10-21T16:17:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.879+0000 7fa008ce5640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 0x7f9fec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa003fff640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0039c0 0x7f9fec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa0037fe640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fec0047c0 0x7f9fec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa008ce5640 1 -- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fec0047c0 msgr2=0x7f9fec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa008ce5640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fec0047c0 0x7f9fec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa008ce5640 1 -- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0039c0 msgr2=0x7f9fec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:15.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa008ce5640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0039c0 0x7f9fec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa008ce5640 1 -- 172.21.15.53:0/1211096786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec003520 con 0x7f9fec0051d0 2023-10-21T16:17:15.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa0037fe640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fec0047c0 0x7f9fec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.880+0000 7fa003fff640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0039c0 0x7f9fec003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.881+0000 7fa008ce5640 1 -- 172.21.15.53:0/1211096786 reap_dead start 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.881+0000 7fa008ce5640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 0x7f9fec00b980 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00ba00 tx=0x7f9fec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2bc3df35e2bc13 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.881+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc004020 con 0x7f9fec0051d0 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.881+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc004180 con 0x7f9fec0051d0 2023-10-21T16:17:15.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.881+0000 7fa002ffd640 1 -- 172.21.15.53:0/1211096786 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc003610 con 0x7f9fec0051d0 2023-10-21T16:17:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 -- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 msgr2=0x7f9fec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 0x7f9fec00b980 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00ba00 tx=0x7f9fec0057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 -- 172.21.15.53:0/1211096786 shutdown_connections 2023-10-21T16:17:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 --2- 172.21.15.53:0/1211096786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fec0051d0 0x7f9fec00b980 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 -- 172.21.15.53:0/1211096786 >> 172.21.15.53:0/1211096786 conn(0x7fa00406e630 msgr2=0x7fa00406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:15.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 -- 172.21.15.53:0/1211096786 shutdown_connections 2023-10-21T16:17:15.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.882+0000 7fa00a748640 1 -- 172.21.15.53:0/1211096786 wait complete. 2023-10-21T16:17:15.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.884+0000 7fa00a748640 1 Processor -- start 2023-10-21T16:17:15.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.884+0000 7fa00a748640 1 -- start start 2023-10-21T16:17:15.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0051d0 0x7fa00414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0041f3b00 0x7fa0040c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0041f3f80 con 0x7fa0040c7b20 2023-10-21T16:17:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040ca390 con 0x7f9fec0051d0 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa00a748640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0040ca670 con 0x7fa0041f3b00 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa0037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa0037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39554/0 (socket says 172.21.15.53:39554) 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.885+0000 7fa0037fe640 1 -- 172.21.15.53:0/3042710740 learned_addr learned my addr 172.21.15.53:0/3042710740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 -- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0041f3b00 msgr2=0x7fa0040c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0041f3b00 0x7fa0040c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 -- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0051d0 msgr2=0x7fa00414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0051d0 0x7fa00414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 -- 172.21.15.53:0/3042710740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0041563b0 con 0x7fa0040c7b20 2023-10-21T16:17:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa0037fe640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800d590 tx=0x7f9ff800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8007220 con 0x7fa0040c7b20 2023-10-21T16:17:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff80073a0 con 0x7fa0040c7b20 2023-10-21T16:17:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80067d0 con 0x7fa0040c7b20 2023-10-21T16:17:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa004156690 con 0x7fa0040c7b20 2023-10-21T16:17:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.886+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0040c4630 con 0x7fa0040c7b20 2023-10-21T16:17:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.887+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00404e160 con 0x7fa0040c7b20 2023-10-21T16:17:15.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.888+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ff8006930 con 0x7fa0040c7b20 2023-10-21T16:17:15.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.889+0000 7fa000ff9640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 0x7f9ff006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:15.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.889+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f9ff80897b0 con 0x7fa0040c7b20 2023-10-21T16:17:15.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.890+0000 7fa003fff640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 0x7f9ff006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.891+0000 7fa003fff640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 0x7f9ff006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4001b20 tx=0x7f9ff4009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:15.893+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ff8054480 con 0x7fa0040c7b20 2023-10-21T16:17:16.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.090+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa00415cb60 con 0x7fa0040c7b20 2023-10-21T16:17:16.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.098+0000 7fa000ff9640 1 -- 172.21.15.53:0/3042710740 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9ff8057aa0 con 0x7fa0040c7b20 2023-10-21T16:17:16.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 msgr2=0x7f9ff006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 0x7f9ff006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4001b20 tx=0x7f9ff4009040 comp rx=0 tx=0).stop 2023-10-21T16:17:16.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 msgr2=0x7fa0041f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800d590 tx=0x7f9ff800da50 comp rx=0 tx=0).stop 2023-10-21T16:17:16.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 shutdown_connections 2023-10-21T16:17:16.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9ff006af90 0x7f9ff006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0041f3b00 0x7fa0040c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0040c7b20 0x7fa0041f35c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 --2- 172.21.15.53:0/3042710740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fec0051d0 0x7fa00414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 >> 172.21.15.53:0/3042710740 conn(0x7fa00406e630 msgr2=0x7fa00415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:16.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 shutdown_connections 2023-10-21T16:17:16.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.101+0000 7fa00a748640 1 -- 172.21.15.53:0/3042710740 wait complete. 2023-10-21T16:17:16.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-21T16:17:16.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.337+0000 7f307a8a8640 1 Processor -- start 2023-10-21T16:17:16.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.337+0000 7f307a8a8640 1 -- start start 2023-10-21T16:17:16.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.338+0000 7f307a8a8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30740ca620 0x7f30740b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.338+0000 7f307a8a8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740cade0 con 0x7f30740ada10 2023-10-21T16:17:16.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.338+0000 7f307a8a8640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740addf0 con 0x7f30740cbb60 2023-10-21T16:17:16.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.338+0000 7f307a8a8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307406c270 con 0x7f30740ca620 2023-10-21T16:17:16.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30740ca620 0x7f30740b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30740ca620 0x7f30740b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39562/0 (socket says 172.21.15.53:39562) 2023-10-21T16:17:16.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 -- 172.21.15.53:0/39114211 learned_addr learned my addr 172.21.15.53:0/39114211 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 -- 172.21.15.53:0/39114211 >> v1:172.21.15.124:6789/0 conn(0x7f30740cbb60 legacy=0x7f307415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 -- 172.21.15.53:0/39114211 >> v1:172.21.15.53:6789/0 conn(0x7f30740ada10 legacy=0x7f30740cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.339+0000 7f307a0a7640 1 -- 172.21.15.53:0/39114211 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307406c550 con 0x7f30740ca620 2023-10-21T16:17:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.340+0000 7f307a0a7640 1 --2- 172.21.15.53:0/39114211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f30740b3080 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f306c009e50 tx=0x7f306c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfc21b9662d46e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f30788a4640 1 -- 172.21.15.53:0/39114211 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c02ca80 con 0x7f30740ca620 2023-10-21T16:17:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f30788a4640 1 -- 172.21.15.53:0/39114211 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c02cbe0 con 0x7f30740ca620 2023-10-21T16:17:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f30788a4640 1 -- 172.21.15.53:0/39114211 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0343f0 con 0x7f30740ca620 2023-10-21T16:17:16.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 -- 172.21.15.53:0/39114211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 msgr2=0x7f30740b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 --2- 172.21.15.53:0/39114211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f30740b3080 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f306c009e50 tx=0x7f306c02bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 -- 172.21.15.53:0/39114211 shutdown_connections 2023-10-21T16:17:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 --2- 172.21.15.53:0/39114211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f30740b3080 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 -- 172.21.15.53:0/39114211 >> 172.21.15.53:0/39114211 conn(0x7f30740ce9d0 msgr2=0x7f30740cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.341+0000 7f307a8a8640 1 -- 172.21.15.53:0/39114211 shutdown_connections 2023-10-21T16:17:16.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.342+0000 7f307a8a8640 1 -- 172.21.15.53:0/39114211 wait complete. 2023-10-21T16:17:16.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.343+0000 7f307a8a8640 1 Processor -- start 2023-10-21T16:17:16.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.343+0000 7f307a8a8640 1 -- start start 2023-10-21T16:17:16.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30740ada10 0x7f30741588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 0x7f307415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741597a0 con 0x7f30740ca620 2023-10-21T16:17:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307415a430 con 0x7f30740cbb60 2023-10-21T16:17:16.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.344+0000 7f307a8a8640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307415a710 con 0x7f30740ada10 2023-10-21T16:17:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39576/0 (socket says 172.21.15.53:39576) 2023-10-21T16:17:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 -- 172.21.15.53:0/2635033106 learned_addr learned my addr 172.21.15.53:0/2635033106 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f307a0a7640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 0x7f307415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 -- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30740ada10 msgr2=0x7f30741588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30798a6640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30740ada10 0x7f30741588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30740ada10 0x7f30741588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 -- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 msgr2=0x7f307415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 0x7f307415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f30790a5640 1 -- 172.21.15.53:0/2635033106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009b30 con 0x7f30740ca620 2023-10-21T16:17:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.345+0000 7f307a0a7640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 0x7f307415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f30790a5640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3064009fd0 tx=0x7f306400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011830 con 0x7f30740ca620 2023-10-21T16:17:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064011990 con 0x7f30740ca620 2023-10-21T16:17:16.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3064011d20 con 0x7f30740ca620 2023-10-21T16:17:16.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307415a9f0 con 0x7f30740ca620 2023-10-21T16:17:16.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.346+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307415afb0 con 0x7f30740ca620 2023-10-21T16:17:16.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.347+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307404e160 con 0x7f30740ca620 2023-10-21T16:17:16.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.348+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f306401a400 con 0x7f30740ca620 2023-10-21T16:17:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.349+0000 7f306a7fc640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 0x7f305806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.350+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f3064089a70 con 0x7f30740ca620 2023-10-21T16:17:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.350+0000 7f30798a6640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 0x7f305806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.351+0000 7f30798a6640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 0x7f305806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3070000f60 tx=0x7f3070009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.354+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30640548e0 con 0x7f30740ca620 2023-10-21T16:17:16.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.620+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f307406de10 con 0x7f30740ca620 2023-10-21T16:17:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.697+0000 7f306a7fc640 1 -- 172.21.15.53:0/2635033106 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v565) v1 ==== 154+0+0 (secure 0 0 0) 0x7f3064057f00 con 0x7f30740ca620 2023-10-21T16:17:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 msgr2=0x7f305806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 0x7f305806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3070000f60 tx=0x7f3070009040 comp rx=0 tx=0).stop 2023-10-21T16:17:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 msgr2=0x7f3074159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3064009fd0 tx=0x7f306400b870 comp rx=0 tx=0).stop 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 shutdown_connections 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f305806af90 0x7f305806d450 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3070000f60 tx=0x7f3070009040 comp rx=0 tx=0).stop 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30740cbb60 0x7f307415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30740ca620 0x7f3074159d30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 --2- 172.21.15.53:0/2635033106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30740ada10 0x7f30741588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.700+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 >> 172.21.15.53:0/2635033106 conn(0x7f30740ce9d0 msgr2=0x7f30740ab1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.701+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 shutdown_connections 2023-10-21T16:17:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.701+0000 7f307a8a8640 1 -- 172.21.15.53:0/2635033106 wait complete. 2023-10-21T16:17:16.723 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-21T16:17:16.724 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-21T16:17:16.724 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:17:16.725 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:16.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.952+0000 7fe46ae52640 1 Processor -- start 2023-10-21T16:17:16.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.952+0000 7fe46ae52640 1 -- start start 2023-10-21T16:17:16.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe46415caa0 0x7fe464069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4640cc460 0x7fe46406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640d0230 0x7fe464160690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4640ccbd0 con 0x7fe4640cc460 2023-10-21T16:17:16.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4640cc8f0 con 0x7fe46415caa0 2023-10-21T16:17:16.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46ae52640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640cf750 con 0x7fe4640d0230 2023-10-21T16:17:16.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.954+0000 7fe46964f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4640cc460 0x7fe46406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4640cc460 0x7fe46406a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39580/0 (socket says 172.21.15.53:39580) 2023-10-21T16:17:16.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 -- 172.21.15.53:0/2831292476 learned_addr learned my addr 172.21.15.53:0/2831292476 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:16.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 -- 172.21.15.53:0/2831292476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640d0230 msgr2=0x7fe464160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:16.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 --2- 172.21.15.53:0/2831292476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640d0230 0x7fe464160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 -- 172.21.15.53:0/2831292476 >> v2:172.21.15.124:3300/0 conn(0x7fe46415caa0 msgr2=0x7fe464069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:16.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 --2- 172.21.15.53:0/2831292476 >> v2:172.21.15.124:3300/0 conn(0x7fe46415caa0 0x7fe464069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.955+0000 7fe46964f640 1 -- 172.21.15.53:0/2831292476 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640cfa30 con 0x7fe4640cc460 2023-10-21T16:17:16.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.956+0000 7fe46964f640 1 --2- 172.21.15.53:0/2831292476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640cc460 0x7fe46406a290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe4640cd530 tx=0x7fe45402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d64fc332c62817 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.956+0000 7fe468e4e640 1 -- 172.21.15.53:0/2831292476 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454030860 con 0x7fe4640cc460 2023-10-21T16:17:16.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.956+0000 7fe468e4e640 1 -- 172.21.15.53:0/2831292476 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4540309c0 con 0x7fe4640cc460 2023-10-21T16:17:16.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.956+0000 7fe468e4e640 1 -- 172.21.15.53:0/2831292476 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454030d50 con 0x7fe4640cc460 2023-10-21T16:17:16.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 -- 172.21.15.53:0/2831292476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640cc460 msgr2=0x7fe46406a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2831292476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640cc460 0x7fe46406a290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe4640cd530 tx=0x7fe45402b9a0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 -- 172.21.15.53:0/2831292476 shutdown_connections 2023-10-21T16:17:16.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2831292476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640d0230 0x7fe464160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2831292476 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640cc460 0x7fe46406a290 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2831292476 >> v2:172.21.15.124:3300/0 conn(0x7fe46415caa0 0x7fe464069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 -- 172.21.15.53:0/2831292476 >> 172.21.15.53:0/2831292476 conn(0x7fe46406ae60 msgr2=0x7fe46406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:16.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.957+0000 7fe46ae52640 1 -- 172.21.15.53:0/2831292476 shutdown_connections 2023-10-21T16:17:16.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.958+0000 7fe46ae52640 1 -- 172.21.15.53:0/2831292476 wait complete. 2023-10-21T16:17:16.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.960+0000 7fe46ae52640 1 Processor -- start 2023-10-21T16:17:16.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.960+0000 7fe46ae52640 1 -- start start 2023-10-21T16:17:16.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe46415caa0 0x7fe4640c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640c55e0 0x7fe4640c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464147670 con 0x7fe4640d0230 2023-10-21T16:17:16.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640c3430 con 0x7fe46415caa0 2023-10-21T16:17:16.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe46ae52640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640c4880 con 0x7fe4640c55e0 2023-10-21T16:17:16.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe469e50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe469e50640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39584/0 (socket says 172.21.15.53:39584) 2023-10-21T16:17:16.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.961+0000 7fe469e50640 1 -- 172.21.15.53:0/2769260513 learned_addr learned my addr 172.21.15.53:0/2769260513 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:16.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 -- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640c55e0 msgr2=0x7fe4640c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe46964f640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe46415caa0 0x7fe4640c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640c55e0 0x7fe4640c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 -- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe46415caa0 msgr2=0x7fe4640c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:16.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe46415caa0 0x7fe4640c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:16.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 -- 172.21.15.53:0/2769260513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540094d0 con 0x7fe4640d0230 2023-10-21T16:17:16.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe469e50640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b4a0 tx=0x7fe46000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.962+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460010610 con 0x7fe4640d0230 2023-10-21T16:17:16.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.963+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460010770 con 0x7fe4640d0230 2023-10-21T16:17:16.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.963+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460010b20 con 0x7fe4640d0230 2023-10-21T16:17:16.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.963+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4640c3150 con 0x7fe4640d0230 2023-10-21T16:17:16.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.963+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4640c4de0 con 0x7fe4640d0230 2023-10-21T16:17:16.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.964+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46404e160 con 0x7fe4640d0230 2023-10-21T16:17:16.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.965+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe460010c80 con 0x7fe4640d0230 2023-10-21T16:17:16.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.966+0000 7fe45a7fc640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 0x7fe44806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:16.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.966+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fe4600895b0 con 0x7fe4640d0230 2023-10-21T16:17:16.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.967+0000 7fe46964f640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 0x7fe44806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:16.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.968+0000 7fe46964f640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 0x7fe44806d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe454036000 tx=0x7fe45403b040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:16.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:16.970+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe460054390 con 0x7fe4640d0230 2023-10-21T16:17:17.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.167+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4640abe50 con 0x7fe4640d0230 2023-10-21T16:17:17.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.168+0000 7fe45a7fc640 1 -- 172.21.15.53:0/2769260513 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+14266 (secure 0 0 0) 0x7fe4600579b0 con 0x7fe4640d0230 2023-10-21T16:17:17.170 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:17.170 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":82,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:14.688982+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/3351749776":"2023-10-22T16:17:11.662276+0000","172.21.15.53:6833/3351749776":"2023-10-22T16:17:11.662276+0000","172.21.15.124:0/1823284108":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/439129464":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6833/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/2911894415":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1304233815":"2023-10-22T16:17:07.486562+0000","172.21.15.124:6832/3738768716":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/3052410171":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/1876804894":"2023-10-22T16:17:07.486562+0000","172.21.15.124:0/473879295":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/3508742564":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/2309645985":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/4037905816":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/1978958043":"2023-10-22T16:15:29.377682+0000","172.21.15.124:0/102098566":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6833/3288323666":"2023-10-22T16:15:29.377682+0000","172.21.15.124:6832/3288323666":"2023-10-22T16:15:29.377682+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 msgr2=0x7fe44806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 0x7fe44806d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe454036000 tx=0x7fe45403b040 comp rx=0 tx=0).stop 2023-10-21T16:17:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 msgr2=0x7fe4640c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b4a0 tx=0x7fe46000b960 comp rx=0 tx=0).stop 2023-10-21T16:17:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 shutdown_connections 2023-10-21T16:17:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe44806afc0 0x7fe44806d480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4640c55e0 0x7fe4640c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe46415caa0 0x7fe4640c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 --2- 172.21.15.53:0/2769260513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4640d0230 0x7fe4640c7a80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.171+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 >> 172.21.15.53:0/2769260513 conn(0x7fe46406ae60 msgr2=0x7fe46415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:17.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.172+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 shutdown_connections 2023-10-21T16:17:17.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.172+0000 7fe46ae52640 1 -- 172.21.15.53:0/2769260513 wait complete. 2023-10-21T16:17:17.194 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:17.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:17.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.416+0000 7f0562b1d640 1 Processor -- start 2023-10-21T16:17:17.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.416+0000 7f0562b1d640 1 -- start start 2023-10-21T16:17:17.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f0562b1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f055c0d0190 0x7f055c0b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f0562b1d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0b71c0 con 0x7f055c0d1f70 2023-10-21T16:17:17.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f0562b1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0b74a0 con 0x7f055c15cc50 2023-10-21T16:17:17.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f0562b1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c0b7780 con 0x7f055c0d0190 2023-10-21T16:17:17.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f055bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f055c0d1f70 0x7f055c0cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52088/0 (socket says 172.21.15.53:52088) 2023-10-21T16:17:17.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.417+0000 7f055bfff640 1 -- 172.21.15.53:0/2704626705 learned_addr learned my addr 172.21.15.53:0/2704626705 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:17.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f05610ba640 1 --2- 172.21.15.53:0/2704626705 >> v2:172.21.15.124:3300/0 conn(0x7f055c0d0190 0x7f055c0b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070882073 0 0) 0x7f055c0b71c0 con 0x7f055c0d1f70 2023-10-21T16:17:17.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0548003520 con 0x7f055c0d1f70 2023-10-21T16:17:17.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610041182 0 0) 0x7f0548003520 con 0x7f055c0d1f70 2023-10-21T16:17:17.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 >> v2:172.21.15.124:3300/0 conn(0x7f055c0d0190 msgr2=0x7f055c0b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:17.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 --2- 172.21.15.53:0/2704626705 >> v2:172.21.15.124:3300/0 conn(0x7f055c0d0190 0x7f055c0b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 >> v1:172.21.15.124:6789/0 conn(0x7f055c15cc50 legacy=0x7f055c0b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0b7a60 con 0x7f055c0d1f70 2023-10-21T16:17:17.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.418+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0544001670 con 0x7f055c0d1f70 2023-10-21T16:17:17.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 >> v1:172.21.15.53:6789/0 conn(0x7f055c0d1f70 legacy=0x7f055c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0548003970 0x7f0548003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 0x7f0548004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0548005180 0x7f054800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0b7780 con 0x7f05480047c0 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054800bf40 con 0x7f0548003970 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054800c100 con 0x7f0548005180 2023-10-21T16:17:17.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 0x7f0548004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f05610ba640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0548005180 0x7f054800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 -- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0548005180 msgr2=0x7f054800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0548005180 0x7f054800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 -- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0548003970 msgr2=0x7f0548003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:17.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f05608b9640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0548003970 0x7f0548003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0548003970 0x7f0548003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f055bfff640 1 -- 172.21.15.53:0/2704626705 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0548003520 con 0x7f05480047c0 2023-10-21T16:17:17.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f05610ba640 1 -- 172.21.15.53:0/2704626705 reap_dead start 2023-10-21T16:17:17.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.419+0000 7f05608b9640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0548003970 0x7f0548003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:17.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f055bfff640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 0x7f0548004c40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f054400c160 tx=0x7f05480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480e1d528d1701db server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544005ad0 con 0x7f05480047c0 2023-10-21T16:17:17.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544005c30 con 0x7f05480047c0 2023-10-21T16:17:17.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f055b7fe640 1 -- 172.21.15.53:0/2704626705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544007070 con 0x7f05480047c0 2023-10-21T16:17:17.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f0562b1d640 1 -- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 msgr2=0x7f0548004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.421+0000 7f0562b1d640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 0x7f0548004c40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f054400c160 tx=0x7f05480057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.422+0000 7f0562b1d640 1 -- 172.21.15.53:0/2704626705 shutdown_connections 2023-10-21T16:17:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.422+0000 7f0562b1d640 1 --2- 172.21.15.53:0/2704626705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05480047c0 0x7f0548004c40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.422+0000 7f0562b1d640 1 -- 172.21.15.53:0/2704626705 >> 172.21.15.53:0/2704626705 conn(0x7f055c06b5e0 msgr2=0x7f055c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.422+0000 7f0562b1d640 1 -- 172.21.15.53:0/2704626705 shutdown_connections 2023-10-21T16:17:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.422+0000 7f0562b1d640 1 -- 172.21.15.53:0/2704626705 wait complete. 2023-10-21T16:17:17.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.423+0000 7f0562b1d640 1 Processor -- start 2023-10-21T16:17:17.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.424+0000 7f0562b1d640 1 -- start start 2023-10-21T16:17:17.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 0x7f055c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 0x7f055c1f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c152ec0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1f2a40 con 0x7f05480047c0 2023-10-21T16:17:17.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f0562b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1f2d20 con 0x7f055c15dbc0 2023-10-21T16:17:17.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39610/0 (socket says 172.21.15.53:39610) 2023-10-21T16:17:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 -- 172.21.15.53:0/3137089466 learned_addr learned my addr 172.21.15.53:0/3137089466 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f055bfff640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 0x7f055c1f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05608b9640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 0x7f055c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 -- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 msgr2=0x7f055c1f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 0x7f055c1f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.425+0000 7f05610ba640 1 -- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 msgr2=0x7f055c152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05610ba640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 0x7f055c152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05610ba640 1 -- 172.21.15.53:0/3137089466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c1f5810 con 0x7f055c1f1fc0 2023-10-21T16:17:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f055bfff640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 0x7f055c1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05608b9640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 0x7f055c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05610ba640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f055c0d23e0 tx=0x7f0554010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400e8a0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055400ea00 con 0x7f055c1f1fc0 2023-10-21T16:17:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400edb0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c1f5af0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.426+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c1f6550 con 0x7f055c1f1fc0 2023-10-21T16:17:17.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.428+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c1f5ce0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.429+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f05540143f0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.430+0000 7f05597fa640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 0x7f053c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.430+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f0554089440 con 0x7f055c1f1fc0 2023-10-21T16:17:17.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.431+0000 7f05608b9640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 0x7f053c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.432+0000 7f05608b9640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 0x7f053c06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f054c007fd0 tx=0x7f054c009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.435+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05540542b0 con 0x7f055c1f1fc0 2023-10-21T16:17:17.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.631+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f055c04e160 con 0x7f055c1f1fc0 2023-10-21T16:17:17.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.631+0000 7f05597fa640 1 -- 172.21.15.53:0/3137089466 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0554000e60 con 0x7f055c1f1fc0 2023-10-21T16:17:17.633 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:17.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 msgr2=0x7f053c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 0x7f053c06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f054c007fd0 tx=0x7f054c009040 comp rx=0 tx=0).stop 2023-10-21T16:17:17.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 msgr2=0x7f055c1f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f055c0d23e0 tx=0x7f0554010c20 comp rx=0 tx=0).stop 2023-10-21T16:17:17.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 shutdown_connections 2023-10-21T16:17:17.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f053c06afc0 0x7f053c06d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f055c1f1fc0 0x7f055c1f2440 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f055c15dbc0 0x7f055c1f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 --2- 172.21.15.53:0/3137089466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05480047c0 0x7f055c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 >> 172.21.15.53:0/3137089466 conn(0x7f055c06b5e0 msgr2=0x7f055c15c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:17.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 shutdown_connections 2023-10-21T16:17:17.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.634+0000 7f0562b1d640 1 -- 172.21.15.53:0/3137089466 wait complete. 2023-10-21T16:17:17.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-21T16:17:17.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.885+0000 7f07bf802640 1 Processor -- start 2023-10-21T16:17:17.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.885+0000 7f07bf802640 1 -- start start 2023-10-21T16:17:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bf802640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07b8159ac0 0x7f07b806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bf802640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b806dee0 con 0x7f07b80ad790 2023-10-21T16:17:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bf802640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b815d850 con 0x7f07b80709e0 2023-10-21T16:17:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bf802640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b815db30 con 0x7f07b8159ac0 2023-10-21T16:17:17.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bd59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07b8159ac0 0x7f07b806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bd59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07b8159ac0 0x7f07b806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39624/0 (socket says 172.21.15.53:39624) 2023-10-21T16:17:17.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.887+0000 7f07bd59e640 1 -- 172.21.15.53:0/642199455 learned_addr learned my addr 172.21.15.53:0/642199455 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:17.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.888+0000 7f07bd59e640 1 -- 172.21.15.53:0/642199455 >> v1:172.21.15.124:6789/0 conn(0x7f07b80709e0 legacy=0x7f07b80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:17.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.888+0000 7f07bd59e640 1 -- 172.21.15.53:0/642199455 >> v1:172.21.15.53:6789/0 conn(0x7f07b80ad790 legacy=0x7f07b80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.888+0000 7f07bd59e640 1 -- 172.21.15.53:0/642199455 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b815de10 con 0x7f07b8159ac0 2023-10-21T16:17:17.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.889+0000 7f07bd59e640 1 --2- 172.21.15.53:0/642199455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b8159ac0 0x7f07b806d5e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be482178e9fd6859 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.889+0000 7f07affff640 1 -- 172.21.15.53:0/642199455 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b402e960 con 0x7f07b8159ac0 2023-10-21T16:17:17.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07affff640 1 -- 172.21.15.53:0/642199455 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b402eac0 con 0x7f07b8159ac0 2023-10-21T16:17:17.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07affff640 1 -- 172.21.15.53:0/642199455 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b402ee50 con 0x7f07b8159ac0 2023-10-21T16:17:17.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07bf802640 1 -- 172.21.15.53:0/642199455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b8159ac0 msgr2=0x7f07b806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07bf802640 1 --2- 172.21.15.53:0/642199455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b8159ac0 0x7f07b806d5e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b402dbb0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07bf802640 1 -- 172.21.15.53:0/642199455 shutdown_connections 2023-10-21T16:17:17.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07bf802640 1 --2- 172.21.15.53:0/642199455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b8159ac0 0x7f07b806d5e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.890+0000 7f07bf802640 1 -- 172.21.15.53:0/642199455 >> 172.21.15.53:0/642199455 conn(0x7f07b806a380 msgr2=0x7f07b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:17.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.891+0000 7f07bf802640 1 -- 172.21.15.53:0/642199455 shutdown_connections 2023-10-21T16:17:17.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.891+0000 7f07bf802640 1 -- 172.21.15.53:0/642199455 wait complete. 2023-10-21T16:17:17.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.892+0000 7f07bf802640 1 Processor -- start 2023-10-21T16:17:17.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.892+0000 7f07bf802640 1 -- start start 2023-10-21T16:17:17.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07b80ad790 0x7f07b80ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07b8159ac0 0x7f07b814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80ac810 con 0x7f07b80709e0 2023-10-21T16:17:17.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80acab0 con 0x7f07b8159ac0 2023-10-21T16:17:17.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.893+0000 7f07bf802640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80acd90 con 0x7f07b80ad790 2023-10-21T16:17:17.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39630/0 (socket says 172.21.15.53:39630) 2023-10-21T16:17:17.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 -- 172.21.15.53:0/2809556826 learned_addr learned my addr 172.21.15.53:0/2809556826 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:17.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 -- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07b80ad790 msgr2=0x7f07b80ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07b80ad790 0x7f07b80ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 -- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07b8159ac0 msgr2=0x7f07b814c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:17.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07b8159ac0 0x7f07b814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:17.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 -- 172.21.15.53:0/2809556826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40094d0 con 0x7f07b80709e0 2023-10-21T16:17:17.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.894+0000 7f07bd59e640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.895+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4041650 con 0x7f07b80709e0 2023-10-21T16:17:17.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.895+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b40417b0 con 0x7f07b80709e0 2023-10-21T16:17:17.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.895+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4041b60 con 0x7f07b80709e0 2023-10-21T16:17:17.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.895+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b80ad070 con 0x7f07b80709e0 2023-10-21T16:17:17.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.895+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b80b1e80 con 0x7f07b80709e0 2023-10-21T16:17:17.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.896+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b80c4ad0 con 0x7f07b80709e0 2023-10-21T16:17:17.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.897+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f07b403d070 con 0x7f07b80709e0 2023-10-21T16:17:17.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.898+0000 7f07adffb640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 0x7f079c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:17.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.898+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f07b40ac630 con 0x7f07b80709e0 2023-10-21T16:17:17.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.899+0000 7f07bcd9d640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 0x7f079c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:17.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.900+0000 7f07bcd9d640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 0x7f079c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f60 tx=0x7f07a80079d0 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:17.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:17.902+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07b4077380 con 0x7f07b80709e0 2023-10-21T16:17:18.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.167+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f07b8067490 con 0x7f07b80709e0 2023-10-21T16:17:18.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.709+0000 7f07adffb640 1 -- 172.21.15.53:0/2809556826 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v567) v1 ==== 159+0+0 (secure 0 0 0) 0x7f07b407a9a0 con 0x7f07b80709e0 2023-10-21T16:17:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 msgr2=0x7f079c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 0x7f079c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f60 tx=0x7f07a80079d0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 msgr2=0x7f07b815c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b40046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 shutdown_connections 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f079c06af90 0x7f079c06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07b8159ac0 0x7f07b814c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07b80ad790 0x7f07b80ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 --2- 172.21.15.53:0/2809556826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07b80709e0 0x7f07b815c330 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 >> 172.21.15.53:0/2809556826 conn(0x7f07b806a380 msgr2=0x7f07b80ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:18.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 shutdown_connections 2023-10-21T16:17:18.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.713+0000 7f07bf802640 1 -- 172.21.15.53:0/2809556826 wait complete. 2023-10-21T16:17:18.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:17:18.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.964+0000 7fb7aeb73640 1 Processor -- start 2023-10-21T16:17:18.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.964+0000 7fb7aeb73640 1 -- start start 2023-10-21T16:17:18.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.965+0000 7fb7aeb73640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7a80b0680 0x7fb7a81594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:18.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.965+0000 7fb7aeb73640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a8071d90 con 0x7fb7a8156a10 2023-10-21T16:17:18.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7aeb73640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a8070ef0 con 0x7fb7a8070ac0 2023-10-21T16:17:18.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7aeb73640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a8071180 con 0x7fb7a80b0680 2023-10-21T16:17:18.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7adb71640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb7a8156a10 0x7fb7a80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52108/0 (socket says 172.21.15.53:52108) 2023-10-21T16:17:18.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7ad370640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7a80b0680 0x7fb7a81594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:18.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7adb71640 1 -- 172.21.15.53:0/2677973497 learned_addr learned my addr 172.21.15.53:0/2677973497 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:18.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7ad370640 1 -- 172.21.15.53:0/2677973497 >> v1:172.21.15.124:6789/0 conn(0x7fb7a8070ac0 legacy=0x7fb7a80c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:18.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7ad370640 1 -- 172.21.15.53:0/2677973497 >> v1:172.21.15.53:6789/0 conn(0x7fb7a8156a10 legacy=0x7fb7a80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.966+0000 7fb7ad370640 1 -- 172.21.15.53:0/2677973497 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8071460 con 0x7fb7a80b0680 2023-10-21T16:17:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.967+0000 7fb7ad370640 1 --2- 172.21.15.53:0/2677973497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a80b0680 0x7fb7a81594e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7980097a0 tx=0x7fb79802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e028780f8139551 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7acb6f640 1 -- 172.21.15.53:0/2677973497 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798031830 con 0x7fb7a80b0680 2023-10-21T16:17:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7acb6f640 1 -- 172.21.15.53:0/2677973497 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798031990 con 0x7fb7a80b0680 2023-10-21T16:17:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7acb6f640 1 -- 172.21.15.53:0/2677973497 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798031d20 con 0x7fb7a80b0680 2023-10-21T16:17:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2677973497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a80b0680 msgr2=0x7fb7a81594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2677973497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a80b0680 0x7fb7a81594e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7980097a0 tx=0x7fb79802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2677973497 shutdown_connections 2023-10-21T16:17:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.968+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2677973497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a80b0680 0x7fb7a81594e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.969+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2677973497 >> 172.21.15.53:0/2677973497 conn(0x7fb7a806c050 msgr2=0x7fb7a806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.969+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2677973497 shutdown_connections 2023-10-21T16:17:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.969+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2677973497 wait complete. 2023-10-21T16:17:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.970+0000 7fb7aeb73640 1 Processor -- start 2023-10-21T16:17:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 -- start start 2023-10-21T16:17:18.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:18.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 0x7fb7a80b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:18.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 0x7fb7a80b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:18.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80ad240 con 0x7fb7a8070ac0 2023-10-21T16:17:18.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80acdb0 con 0x7fb7a80b0680 2023-10-21T16:17:18.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.971+0000 7fb7aeb73640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80ac480 con 0x7fb7a8156a10 2023-10-21T16:17:18.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:18.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39646/0 (socket says 172.21.15.53:39646) 2023-10-21T16:17:18.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 -- 172.21.15.53:0/2145886850 learned_addr learned my addr 172.21.15.53:0/2145886850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:18.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7ad370640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 0x7fb7a80b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:18.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 -- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 msgr2=0x7fb7a80b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7ae372640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 0x7fb7a80b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:18.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 0x7fb7a80b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 -- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 msgr2=0x7fb7a80b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:18.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 0x7fb7a80b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:18.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7adb71640 1 -- 172.21.15.53:0/2145886850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798009400 con 0x7fb7a8070ac0 2023-10-21T16:17:18.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7ad370640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 0x7fb7a80b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:18.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.972+0000 7fb7ae372640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 0x7fb7a80b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:18.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb7adb71640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c480 tx=0x7fb7a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:18.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4005ce0 con 0x7fb7a8070ac0 2023-10-21T16:17:18.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a4005e40 con 0x7fb7a8070ac0 2023-10-21T16:17:18.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40072f0 con 0x7fb7a8070ac0 2023-10-21T16:17:18.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a80ac760 con 0x7fb7a8070ac0 2023-10-21T16:17:18.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.973+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a8156160 con 0x7fb7a8070ac0 2023-10-21T16:17:18.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.975+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a8074150 con 0x7fb7a8070ac0 2023-10-21T16:17:18.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.975+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb7a4007510 con 0x7fb7a8070ac0 2023-10-21T16:17:18.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.977+0000 7fb79e7fc640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 0x7fb78c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:18.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.977+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fb7a4088de0 con 0x7fb7a8070ac0 2023-10-21T16:17:18.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.978+0000 7fb7ad370640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 0x7fb78c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:18.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.979+0000 7fb7ad370640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 0x7fb78c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:18.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:18.982+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7a4053d00 con 0x7fb7a8070ac0 2023-10-21T16:17:19.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.176+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7a815c560 con 0x7fb7a8070ac0 2023-10-21T16:17:19.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.182+0000 7fb79e7fc640 1 -- 172.21.15.53:0/2145886850 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7a4017020 con 0x7fb7a8070ac0 2023-10-21T16:17:19.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 msgr2=0x7fb78c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:19.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 0x7fb78c06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).stop 2023-10-21T16:17:19.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 msgr2=0x7fb7a8155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:19.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c480 tx=0x7fb7a400c940 comp rx=0 tx=0).stop 2023-10-21T16:17:19.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 shutdown_connections 2023-10-21T16:17:19.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fb78c06af90 0x7fb78c06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7a8156a10 0x7fb7a80b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7a80b0680 0x7fb7a80b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 --2- 172.21.15.53:0/2145886850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7a8070ac0 0x7fb7a8155c20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.184+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 >> 172.21.15.53:0/2145886850 conn(0x7fb7a806c050 msgr2=0x7fb7a80cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:19.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.185+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 shutdown_connections 2023-10-21T16:17:19.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.185+0000 7fb7aeb73640 1 -- 172.21.15.53:0/2145886850 wait complete. 2023-10-21T16:17:19.204 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:17:19.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:19.319 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:19.445 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:17:19.446 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:19.446 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:19.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:19.511 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:19.511 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:19.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:19.571 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:19.571 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:19.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:19.636 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:19.636 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:19.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:19.697 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:17:19.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:19.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.934+0000 7ff424ff4640 1 Processor -- start 2023-10-21T16:17:19.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.934+0000 7ff424ff4640 1 -- start start 2023-10-21T16:17:19.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff424ff4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 0x7ff42015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff424ff4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4200b07d0 0x7ff42006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff424ff4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42015f7d0 con 0x7ff4200adc70 2023-10-21T16:17:19.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff424ff4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42015f210 con 0x7ff4200b07d0 2023-10-21T16:17:19.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff424ff4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015f4f0 con 0x7ff42015ab30 2023-10-21T16:17:19.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff41ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4200b07d0 0x7ff42006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff41ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff4200b07d0 0x7ff42006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39662/0 (socket says 172.21.15.53:39662) 2023-10-21T16:17:19.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.935+0000 7ff41ffff640 1 -- 172.21.15.53:0/3642873520 learned_addr learned my addr 172.21.15.53:0/3642873520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:19.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41f7fe640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 0x7ff42015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41ffff640 1 -- 172.21.15.53:0/3642873520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 msgr2=0x7ff42015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:19.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41ffff640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 0x7ff42015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41ffff640 1 -- 172.21.15.53:0/3642873520 >> v1:172.21.15.124:6789/0 conn(0x7ff4200adc70 legacy=0x7ff4200b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:19.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41ffff640 1 -- 172.21.15.53:0/3642873520 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42006c550 con 0x7ff4200b07d0 2023-10-21T16:17:19.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.936+0000 7ff41f7fe640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 0x7ff42015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:19.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.937+0000 7ff41ffff640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff42006cd50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e50 tx=0x7ff41802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca6e410cef6a68 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:19.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.937+0000 7ff41e7fc640 1 -- 172.21.15.53:0/3642873520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41802ca80 con 0x7ff4200b07d0 2023-10-21T16:17:19.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.937+0000 7ff41e7fc640 1 -- 172.21.15.53:0/3642873520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41802cbe0 con 0x7ff4200b07d0 2023-10-21T16:17:19.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.937+0000 7ff41e7fc640 1 -- 172.21.15.53:0/3642873520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4180343f0 con 0x7ff4200b07d0 2023-10-21T16:17:19.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 -- 172.21.15.53:0/3642873520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 msgr2=0x7ff42006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:19.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff42006cd50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff418009e50 tx=0x7ff41802bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 -- 172.21.15.53:0/3642873520 shutdown_connections 2023-10-21T16:17:19.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff42006cd50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3642873520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff42015ab30 0x7ff42015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 -- 172.21.15.53:0/3642873520 >> 172.21.15.53:0/3642873520 conn(0x7ff4200cb0c0 msgr2=0x7ff4200cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:19.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 -- 172.21.15.53:0/3642873520 shutdown_connections 2023-10-21T16:17:19.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.938+0000 7ff424ff4640 1 -- 172.21.15.53:0/3642873520 wait complete. 2023-10-21T16:17:19.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.940+0000 7ff424ff4640 1 Processor -- start 2023-10-21T16:17:19.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.940+0000 7ff424ff4640 1 -- start start 2023-10-21T16:17:19.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200adc70 0x7ff4200c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42015ab30 0x7ff4201594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200ab8d0 con 0x7ff4200b07d0 2023-10-21T16:17:19.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200abb20 con 0x7ff42015ab30 2023-10-21T16:17:19.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff424ff4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200abe00 con 0x7ff4200adc70 2023-10-21T16:17:19.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff41effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff41effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39670/0 (socket says 172.21.15.53:39670) 2023-10-21T16:17:19.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.941+0000 7ff41effd640 1 -- 172.21.15.53:0/3807345777 learned_addr learned my addr 172.21.15.53:0/3807345777 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:19.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41f7fe640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200adc70 0x7ff4200c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41ffff640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42015ab30 0x7ff4201594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 -- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200adc70 msgr2=0x7ff4200c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200adc70 0x7ff4200c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 -- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42015ab30 msgr2=0x7ff4201594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42015ab30 0x7ff4201594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:19.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 -- 172.21.15.53:0/3807345777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418009b30 con 0x7ff4200b07d0 2023-10-21T16:17:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff41effd640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c430 tx=0x7ff41400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414011790 con 0x7ff4200b07d0 2023-10-21T16:17:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140118f0 con 0x7ff4200b07d0 2023-10-21T16:17:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.942+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414011ce0 con 0x7ff4200b07d0 2023-10-21T16:17:19.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.943+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4200ac0e0 con 0x7ff4200b07d0 2023-10-21T16:17:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.943+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4200ac8e0 con 0x7ff4200b07d0 2023-10-21T16:17:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.944+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42004e160 con 0x7ff4200b07d0 2023-10-21T16:17:19.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.945+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff41401c460 con 0x7ff4200b07d0 2023-10-21T16:17:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.946+0000 7ff407fff640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 0x7ff40c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:19.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.946+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7ff4140898a0 con 0x7ff4200b07d0 2023-10-21T16:17:19.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.947+0000 7ff41f7fe640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 0x7ff40c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:19.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.948+0000 7ff41f7fe640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 0x7ff40c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff4200ce3b0 tx=0x7ff410008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:19.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:19.950+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff414054680 con 0x7ff4200b07d0 2023-10-21T16:17:20.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.184+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4200ada50 con 0x7ff4200b07d0 2023-10-21T16:17:20.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.187+0000 7ff407fff640 1 -- 172.21.15.53:0/3807345777 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff414057ca0 con 0x7ff4200b07d0 2023-10-21T16:17:20.189 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:20.189 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/857072042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":857072042},{"type":"v1","addr":"172.21.15.124:6835","nonce":857072042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3965024480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3965024480},{"type":"v1","addr":"172.21.15.124:6837","nonce":3965024480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/598430945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":598430945},{"type":"v1","addr":"172.21.15.53:6834","nonce":598430945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/4018544171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":4018544171},{"type":"v1","addr":"172.21.15.53:6835","nonce":4018544171}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2023-10-21T16:17:20.190 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 29 2023-10-21T16:17:20.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 msgr2=0x7ff40c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 0x7ff40c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff4200ce3b0 tx=0x7ff410008040 comp rx=0 tx=0).stop 2023-10-21T16:17:20.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 msgr2=0x7ff420158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c430 tx=0x7ff41400c8f0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 shutdown_connections 2023-10-21T16:17:20.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7ff40c06af90 0x7ff40c06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff42015ab30 0x7ff4201594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.190+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4200b07d0 0x7ff420158fb0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.191+0000 7ff424ff4640 1 --2- 172.21.15.53:0/3807345777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200adc70 0x7ff4200c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.191+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 >> 172.21.15.53:0/3807345777 conn(0x7ff4200cb0c0 msgr2=0x7ff4200cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:20.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.191+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 shutdown_connections 2023-10-21T16:17:20.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.191+0000 7ff424ff4640 1 -- 172.21.15.53:0/3807345777 wait complete. 2023-10-21T16:17:20.210 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:17:20.211 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:17:20.211 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:17:20.211 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:20.212 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:17:20.213 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:17:20.213 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:17:20.213 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:20.213 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:17:20.214 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:17:20.214 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:17:20.214 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:20.215 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:17:20.215 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:17:20.215 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:17:20.215 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:20.230 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:17:20.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:17:20.232 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:17:20.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:17:20.235 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:17:20.236 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:17:20.238 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:17:20.239 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:17:20.278 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:17:20.279 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:20.285 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:17:20.286 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:20.287 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:17:20.287 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:20.295 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:17:20.295 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:20.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:17:20.448 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:17:20.449 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:17:20.450 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:17:20.450 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:17:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.490+0000 7f19c9821640 1 Processor -- start 2023-10-21T16:17:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.490+0000 7f19c9821640 1 -- start start 2023-10-21T16:17:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.491+0000 7f19c9821640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c40ce2d0 0x7f19c406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.491+0000 7f19c9821640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19c40b2180 0x7f19c40b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.491+0000 7f19c9821640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c406c1f0 con 0x7f19c415ccd0 2023-10-21T16:17:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.491+0000 7f19c9821640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40b2b50 con 0x7f19c40b2180 2023-10-21T16:17:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.491+0000 7f19c9821640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40ce730 con 0x7f19c40ce2d0 2023-10-21T16:17:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19c40b2180 0x7f19c40b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19c40b2180 0x7f19c40b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39700/0 (socket says 172.21.15.53:39700) 2023-10-21T16:17:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 -- 172.21.15.53:0/4126948290 learned_addr learned my addr 172.21.15.53:0/4126948290 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c3fff640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c40ce2d0 0x7f19c406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 -- 172.21.15.53:0/4126948290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c40ce2d0 msgr2=0x7f19c406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c40ce2d0 0x7f19c406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 -- 172.21.15.53:0/4126948290 >> v1:172.21.15.124:6789/0 conn(0x7f19c415ccd0 legacy=0x7f19c4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.492+0000 7f19c9020640 1 -- 172.21.15.53:0/4126948290 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40ce9b0 con 0x7f19c40b2180 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.493+0000 7f19c9020640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40b2180 0x7f19c40b2610 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f19bc009e50 tx=0x7f19bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ff3f0b450c2686 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c37fe640 1 -- 172.21.15.53:0/4126948290 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc02cb00 con 0x7f19c40b2180 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c37fe640 1 -- 172.21.15.53:0/4126948290 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc02cc60 con 0x7f19c40b2180 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c37fe640 1 -- 172.21.15.53:0/4126948290 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc0343f0 con 0x7f19c40b2180 2023-10-21T16:17:20.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 -- 172.21.15.53:0/4126948290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40b2180 msgr2=0x7f19c40b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40b2180 0x7f19c40b2610 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f19bc009e50 tx=0x7f19bc02bf60 comp rx=0 tx=0).stop 2023-10-21T16:17:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 -- 172.21.15.53:0/4126948290 shutdown_connections 2023-10-21T16:17:20.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c40ce2d0 0x7f19c406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 --2- 172.21.15.53:0/4126948290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40b2180 0x7f19c40b2610 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 -- 172.21.15.53:0/4126948290 >> 172.21.15.53:0/4126948290 conn(0x7f19c40cd180 msgr2=0x7f19c40cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 -- 172.21.15.53:0/4126948290 shutdown_connections 2023-10-21T16:17:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.494+0000 7f19c9821640 1 -- 172.21.15.53:0/4126948290 wait complete. 2023-10-21T16:17:20.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.496+0000 7f19c9821640 1 Processor -- start 2023-10-21T16:17:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.496+0000 7f19c9821640 1 -- start start 2023-10-21T16:17:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 0x7f19c4151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c415ccd0 0x7f19c415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c415e710 con 0x7f19c40ce2d0 2023-10-21T16:17:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c415db00 con 0x7f19c40b2180 2023-10-21T16:17:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c9821640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c415dde0 con 0x7f19c415ccd0 2023-10-21T16:17:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.497+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39704/0 (socket says 172.21.15.53:39704) 2023-10-21T16:17:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 -- 172.21.15.53:0/602213602 learned_addr learned my addr 172.21.15.53:0/602213602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c881f640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 0x7f19c4151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c9020640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c415ccd0 0x7f19c415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 -- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c415ccd0 msgr2=0x7f19c415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c415ccd0 0x7f19c415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 -- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 msgr2=0x7f19c4151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:20.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 0x7f19c4151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:20.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 -- 172.21.15.53:0/602213602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc009b30 con 0x7f19c40ce2d0 2023-10-21T16:17:20.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c881f640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 0x7f19c4151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:20.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.498+0000 7f19c3fff640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d800 tx=0x7f19b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:20.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.499+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000b6c0 con 0x7f19c40ce2d0 2023-10-21T16:17:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.499+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000b820 con 0x7f19c40ce2d0 2023-10-21T16:17:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.499+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c415e0c0 con 0x7f19c40ce2d0 2023-10-21T16:17:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.499+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000bbb0 con 0x7f19c40ce2d0 2023-10-21T16:17:20.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.499+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c41521f0 con 0x7f19c40ce2d0 2023-10-21T16:17:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.500+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c415e880 con 0x7f19c40ce2d0 2023-10-21T16:17:20.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.501+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f19b000bd10 con 0x7f19c40ce2d0 2023-10-21T16:17:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.502+0000 7f19c17fa640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 0x7f19a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.503+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f19b0089d60 con 0x7f19c40ce2d0 2023-10-21T16:17:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.503+0000 7f19c881f640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 0x7f19a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.504+0000 7f19c881f640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 0x7f19a806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19b8000f60 tx=0x7f19b8009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:20.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.507+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19b0054c00 con 0x7f19c40ce2d0 2023-10-21T16:17:20.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:20.712+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f19c4074150 con 0x7f19c40ce2d0 2023-10-21T16:17:20.793 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:17:20.803 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:17:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.209+0000 7f19c17fa640 1 -- 172.21.15.53:0/602213602 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7f19b0058220 con 0x7f19c40ce2d0 2023-10-21T16:17:21.211 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:17:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.213+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 msgr2=0x7f19a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.213+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 0x7f19a806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f19b8000f60 tx=0x7f19b8009040 comp rx=0 tx=0).stop 2023-10-21T16:17:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.213+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 msgr2=0x7f19c415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.213+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f19b000d800 tx=0x7f19b000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 shutdown_connections 2023-10-21T16:17:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f19a806af90 0x7f19a806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19c415ccd0 0x7f19c415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c40ce2d0 0x7f19c415eba0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 --2- 172.21.15.53:0/602213602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c40b2180 0x7f19c4151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 >> 172.21.15.53:0/602213602 conn(0x7f19c40cd180 msgr2=0x7f19c40cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 shutdown_connections 2023-10-21T16:17:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.214+0000 7f19c9821640 1 -- 172.21.15.53:0/602213602 wait complete. 2023-10-21T16:17:21.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:17:21.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.461+0000 7f7e8bd2c640 1 Processor -- start 2023-10-21T16:17:21.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.461+0000 7f7e8bd2c640 1 -- start start 2023-10-21T16:17:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8bd2c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 0x7f7e840709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8bd2c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e8406c890 0x7f7e8406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8bd2c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e840c48f0 con 0x7f7e8415cd90 2023-10-21T16:17:21.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8bd2c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e840c4d80 con 0x7f7e8406c890 2023-10-21T16:17:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8bd2c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e840c3ce0 con 0x7f7e84160940 2023-10-21T16:17:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.463+0000 7f7e8a2c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e8406c890 0x7f7e8406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e8406c890 0x7f7e8406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39718/0 (socket says 172.21.15.53:39718) 2023-10-21T16:17:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 -- 172.21.15.53:0/1736439850 learned_addr learned my addr 172.21.15.53:0/1736439850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:21.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e892c7640 1 --2- 172.21.15.53:0/1736439850 >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 0x7f7e840709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 -- 172.21.15.53:0/1736439850 >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 msgr2=0x7f7e840709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:21.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 --2- 172.21.15.53:0/1736439850 >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 0x7f7e840709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 -- 172.21.15.53:0/1736439850 >> v1:172.21.15.124:6789/0 conn(0x7f7e8415cd90 legacy=0x7f7e840702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e8a2c9640 1 -- 172.21.15.53:0/1736439850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840c3fc0 con 0x7f7e8406c890 2023-10-21T16:17:21.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.464+0000 7f7e892c7640 1 --2- 172.21.15.53:0/1736439850 >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 0x7f7e840709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:21.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.465+0000 7f7e8a2c9640 1 --2- 172.21.15.53:0/1736439850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8406c890 0x7f7e8406cd20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7e840cc090 tx=0x7f7e7c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80110695bb4c336 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:21.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e88ac6640 1 -- 172.21.15.53:0/1736439850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c02cb10 con 0x7f7e8406c890 2023-10-21T16:17:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e88ac6640 1 -- 172.21.15.53:0/1736439850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c02cc70 con 0x7f7e8406c890 2023-10-21T16:17:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e88ac6640 1 -- 172.21.15.53:0/1736439850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0343f0 con 0x7f7e8406c890 2023-10-21T16:17:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/1736439850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8406c890 msgr2=0x7f7e8406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/1736439850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8406c890 0x7f7e8406cd20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7e840cc090 tx=0x7f7e7c02bf70 comp rx=0 tx=0).stop 2023-10-21T16:17:21.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/1736439850 shutdown_connections 2023-10-21T16:17:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/1736439850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8406c890 0x7f7e8406cd20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/1736439850 >> v2:172.21.15.124:3300/0 conn(0x7f7e84160940 0x7f7e840709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.466+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/1736439850 >> 172.21.15.53:0/1736439850 conn(0x7f7e8406a380 msgr2=0x7f7e840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.467+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/1736439850 shutdown_connections 2023-10-21T16:17:21.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.467+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/1736439850 wait complete. 2023-10-21T16:17:21.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.468+0000 7f7e8bd2c640 1 Processor -- start 2023-10-21T16:17:21.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.468+0000 7f7e8bd2c640 1 -- start start 2023-10-21T16:17:21.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e8406c890 0x7f7e840b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 0x7f7e8415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840b3900 con 0x7f7e8415cd90 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840b2470 con 0x7f7e84160940 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.469+0000 7f7e8bd2c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840b2750 con 0x7f7e8406c890 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39726/0 (socket says 172.21.15.53:39726) 2023-10-21T16:17:21.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 -- 172.21.15.53:0/2190189860 learned_addr learned my addr 172.21.15.53:0/2190189860 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:21.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e8a2c9640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 0x7f7e8415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e89ac8640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e8406c890 0x7f7e840b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 -- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e8406c890 msgr2=0x7f7e840b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e8406c890 0x7f7e840b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 -- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 msgr2=0x7f7e8415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 0x7f7e8415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 -- 172.21.15.53:0/2190189860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c009b30 con 0x7f7e8415cd90 2023-10-21T16:17:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e8a2c9640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 0x7f7e8415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.470+0000 7f7e892c7640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e7e0 tx=0x7f7e7400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.471+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74004030 con 0x7f7e8415cd90 2023-10-21T16:17:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.471+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e74004190 con 0x7f7e8415cd90 2023-10-21T16:17:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.471+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e840b2a30 con 0x7f7e8415cd90 2023-10-21T16:17:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.471+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74004520 con 0x7f7e8415cd90 2023-10-21T16:17:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.471+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e840c94b0 con 0x7f7e8415cd90 2023-10-21T16:17:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.472+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e840c9670 con 0x7f7e8415cd90 2023-10-21T16:17:21.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.474+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e7400c6d0 con 0x7f7e8415cd90 2023-10-21T16:17:21.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.475+0000 7f7e7a7fc640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 0x7f7e6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:21.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.475+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f7e740893b0 con 0x7f7e8415cd90 2023-10-21T16:17:21.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.476+0000 7f7e89ac8640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 0x7f7e6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:21.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.477+0000 7f7e89ac8640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 0x7f7e6806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e800015a0 tx=0x7f7e80009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:21.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.480+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e74054420 con 0x7f7e8415cd90 2023-10-21T16:17:21.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.745+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e84067490 con 0x7f7e8415cd90 2023-10-21T16:17:21.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.747+0000 7f7e7a7fc640 1 -- 172.21.15.53:0/2190189860 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7e74057a40 con 0x7f7e8415cd90 2023-10-21T16:17:21.748 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:21.749 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:17:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:17:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:17:21.749 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:17:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:17:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:17:21.750 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:17:21.751 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:17:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:17:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:17:21.752 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:17:21.753 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:17:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:17:21.755 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:17:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:17:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.757 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:17:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:17:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.759 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:17:21.760 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:17:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:17:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:17:21.762 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:17:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:17:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.764 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:17:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:17:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.766 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.767 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:17:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:17:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.768 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:17:21.769 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:17:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:17:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:17:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:17:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.771 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:17:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:17:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:17:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:17:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:17:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:17:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:17:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:17:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:17:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:17:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:17:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:17:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:17:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:17:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:17:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:17:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:17:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:17:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:17:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:17:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:17:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:17:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:17:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:17:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:17:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:17:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:17:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:17:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:17:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:17:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:17:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:17:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:17:21.784 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:17:21.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.749+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 msgr2=0x7f7e6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 0x7f7e6806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e800015a0 tx=0x7f7e80009040 comp rx=0 tx=0).stop 2023-10-21T16:17:21.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 msgr2=0x7f7e840b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:21.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400e7e0 tx=0x7f7e7400eca0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 shutdown_connections 2023-10-21T16:17:21.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f7e6806af90 0x7f7e6806d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e84160940 0x7f7e8415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e8415cd90 0x7f7e840b33c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 --2- 172.21.15.53:0/2190189860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e8406c890 0x7f7e840b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:21.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 >> 172.21.15.53:0/2190189860 conn(0x7f7e8406a380 msgr2=0x7f7e84160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:21.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 shutdown_connections 2023-10-21T16:17:21.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:21.750+0000 7f7e8bd2c640 1 -- 172.21.15.53:0/2190189860 wait complete. 2023-10-21T16:17:21.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.010+0000 7f1db04bb640 1 Processor -- start 2023-10-21T16:17:22.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.011+0000 7f1db04bb640 1 -- start start 2023-10-21T16:17:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1db04bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1da8159ac0 0x7f1da806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1db04bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 0x7f1da80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1db04bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da806dee0 con 0x7f1da80ad790 2023-10-21T16:17:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1db04bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da815d850 con 0x7f1da8159ac0 2023-10-21T16:17:22.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1db04bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815db30 con 0x7f1da80709e0 2023-10-21T16:17:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1dada56640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1da80ad790 0x7f1da80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52130/0 (socket says 172.21.15.53:52130) 2023-10-21T16:17:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.012+0000 7f1dada56640 1 -- 172.21.15.53:0/1810493811 learned_addr learned my addr 172.21.15.53:0/1810493811 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dae257640 1 --2- 172.21.15.53:0/1810493811 >> v2:172.21.15.124:3300/0 conn(0x7f1da8159ac0 0x7f1da806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1daea58640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 0x7f1da80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846721294 0 0) 0x7f1da806dee0 con 0x7f1da80ad790 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d94003520 con 0x7f1da80ad790 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756460435 0 0) 0x7f1d94003520 con 0x7f1da80ad790 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 msgr2=0x7f1da80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 0x7f1da80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 >> v2:172.21.15.124:3300/0 conn(0x7f1da8159ac0 msgr2=0x7f1da806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 --2- 172.21.15.53:0/1810493811 >> v2:172.21.15.124:3300/0 conn(0x7f1da8159ac0 0x7f1da806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da815de10 con 0x7f1da80ad790 2023-10-21T16:17:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1d98001690 con 0x7f1da80ad790 2023-10-21T16:17:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 >> v1:172.21.15.53:6789/0 conn(0x7f1da80ad790 legacy=0x7f1da80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1d94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.013+0000 7f1dad255640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d940047c0 0x7f1d94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dad255640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 0x7f1d94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815d850 con 0x7f1d94003970 2023-10-21T16:17:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815db30 con 0x7f1d940047c0 2023-10-21T16:17:22.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9400b980 con 0x7f1da80709e0 2023-10-21T16:17:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1d94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 -- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 msgr2=0x7f1d94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da80709e0 0x7f1d94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 -- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d940047c0 msgr2=0x7f1d94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d940047c0 0x7f1d94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.014+0000 7f1dae257640 1 -- 172.21.15.53:0/1810493811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94003520 con 0x7f1d94003970 2023-10-21T16:17:22.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.015+0000 7f1dae257640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1d94003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009ac0 tx=0x7f1d940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcd1902fd1728a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.015+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da402c710 con 0x7f1d94003970 2023-10-21T16:17:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.015+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da402c870 con 0x7f1d94003970 2023-10-21T16:17:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.015+0000 7f1dad255640 1 -- 172.21.15.53:0/1810493811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da402cc00 con 0x7f1d94003970 2023-10-21T16:17:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 -- 172.21.15.53:0/1810493811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 msgr2=0x7f1d94003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 --2- 172.21.15.53:0/1810493811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1d94003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009ac0 tx=0x7f1d940057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1daea58640 1 -- 172.21.15.53:0/1810493811 reap_dead start 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 -- 172.21.15.53:0/1810493811 shutdown_connections 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 -- 172.21.15.53:0/1810493811 >> 172.21.15.53:0/1810493811 conn(0x7f1da806a380 msgr2=0x7f1da80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 -- 172.21.15.53:0/1810493811 shutdown_connections 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.016+0000 7f1db04bb640 1 -- 172.21.15.53:0/1810493811 wait complete. 2023-10-21T16:17:22.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.018+0000 7f1db04bb640 1 Processor -- start 2023-10-21T16:17:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.018+0000 7f1db04bb640 1 -- start start 2023-10-21T16:17:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 0x7f1da814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da8149040 0x7f1da814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8158920 con 0x7f1d94003970 2023-10-21T16:17:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da81497c0 con 0x7f1da81559b0 2023-10-21T16:17:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1db04bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8165a70 con 0x7f1da8149040 2023-10-21T16:17:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.019+0000 7f1dae257640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39746/0 (socket says 172.21.15.53:39746) 2023-10-21T16:17:22.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 -- 172.21.15.53:0/174182516 learned_addr learned my addr 172.21.15.53:0/174182516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dada56640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 0x7f1da814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1daea58640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da8149040 0x7f1da814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 -- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da8149040 msgr2=0x7f1da814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da8149040 0x7f1da814d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 -- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 msgr2=0x7f1da814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 0x7f1da814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 -- 172.21.15.53:0/174182516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8158640 con 0x7f1d94003970 2023-10-21T16:17:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dada56640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 0x7f1da814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.020+0000 7f1dae257640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1da402c2b0 tx=0x7f1da4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.021+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4003810 con 0x7f1d94003970 2023-10-21T16:17:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.021+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da402de90 con 0x7f1d94003970 2023-10-21T16:17:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.021+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4003b70 con 0x7f1d94003970 2023-10-21T16:17:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.021+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8158e10 con 0x7f1d94003970 2023-10-21T16:17:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.021+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da81573e0 con 0x7f1d94003970 2023-10-21T16:17:22.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.022+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da8158c70 con 0x7f1d94003970 2023-10-21T16:17:22.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.023+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1da4003cd0 con 0x7f1d94003970 2023-10-21T16:17:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.024+0000 7f1d9effd640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 0x7f1d8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.024+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f1da40ab3c0 con 0x7f1d94003970 2023-10-21T16:17:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.024+0000 7f1dada56640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 0x7f1d8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.026+0000 7f1dada56640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 0x7f1d8c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d980017c0 tx=0x7f1d98009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.028+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1da8158c70 con 0x7f1d94003970 2023-10-21T16:17:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.224+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1da8158c70 con 0x7f1d94003970 2023-10-21T16:17:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.225+0000 7f1d9effd640 1 -- 172.21.15.53:0/174182516 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13120 (secure 0 0 0) 0x7f1da8158c70 con 0x7f1d94003970 2023-10-21T16:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:22.227 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:21.222395+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 msgr2=0x7f1d8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 0x7f1d8c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1d980017c0 tx=0x7f1d98009040 comp rx=0 tx=0).stop 2023-10-21T16:17:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 msgr2=0x7f1da81489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1da402c2b0 tx=0x7f1da4004730 comp rx=0 tx=0).stop 2023-10-21T16:17:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 shutdown_connections 2023-10-21T16:17:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f1d8c06af90 0x7f1d8c06d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1da8149040 0x7f1da814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1da81559b0 0x7f1da814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 --2- 172.21.15.53:0/174182516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d94003970 0x7f1da81489b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 >> 172.21.15.53:0/174182516 conn(0x7f1da806a380 msgr2=0x7f1da80ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 shutdown_connections 2023-10-21T16:17:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.228+0000 7f1db04bb640 1 -- 172.21.15.53:0/174182516 wait complete. 2023-10-21T16:17:22.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:22.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:22.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.464+0000 7fd1fc3e7640 1 Processor -- start 2023-10-21T16:17:22.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.464+0000 7fd1fc3e7640 1 -- start start 2023-10-21T16:17:22.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fc3e7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1f415aad0 0x7fd1f40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fc3e7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1f40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fc3e7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f415bf30 con 0x7fd1f40c82b0 2023-10-21T16:17:22.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fc3e7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f415b320 con 0x7fd1f415aad0 2023-10-21T16:17:22.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fc3e7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f415b600 con 0x7fd1f406d410 2023-10-21T16:17:22.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1f9982640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd1f40c82b0 0x7fd1f406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52132/0 (socket says 172.21.15.53:52132) 2023-10-21T16:17:22.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1f9982640 1 -- 172.21.15.53:0/3176694621 learned_addr learned my addr 172.21.15.53:0/3176694621 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:22.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.466+0000 7fd1fa183640 1 --2- 172.21.15.53:0/3176694621 >> v2:172.21.15.124:3300/0 conn(0x7fd1f415aad0 0x7fd1f40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1fa984640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1f40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747135896 0 0) 0x7fd1f415bf30 con 0x7fd1f40c82b0 2023-10-21T16:17:22.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e0003520 con 0x7fd1f40c82b0 2023-10-21T16:17:22.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964592930 0 0) 0x7fd1e0003520 con 0x7fd1f40c82b0 2023-10-21T16:17:22.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 msgr2=0x7fd1f40c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1f40c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 >> v2:172.21.15.124:3300/0 conn(0x7fd1f415aad0 msgr2=0x7fd1f40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 --2- 172.21.15.53:0/3176694621 >> v2:172.21.15.124:3300/0 conn(0x7fd1f415aad0 0x7fd1f40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f415b8e0 con 0x7fd1f40c82b0 2023-10-21T16:17:22.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd1e4001670 con 0x7fd1f40c82b0 2023-10-21T16:17:22.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 >> v1:172.21.15.53:6789/0 conn(0x7fd1f40c82b0 legacy=0x7fd1f406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1e0003f40 0x7fd1e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1e0004d00 0x7fd1e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f415b320 con 0x7fd1e0004d00 2023-10-21T16:17:22.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f415b600 con 0x7fd1e0003f40 2023-10-21T16:17:22.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.467+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e000b980 con 0x7fd1f406d410 2023-10-21T16:17:22.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1e0004d00 0x7fd1e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1f9982640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1e0003f40 0x7fd1e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa183640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 -- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 msgr2=0x7fd1e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1f406d410 0x7fd1e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 -- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1e0003f40 msgr2=0x7fd1e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1e0003f40 0x7fd1e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.468+0000 7fd1fa984640 1 -- 172.21.15.53:0/3176694621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0003520 con 0x7fd1e0004d00 2023-10-21T16:17:22.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.469+0000 7fd1fa984640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1e0004d00 0x7fd1e0005180 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009f90 tx=0x7fd1e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3cafb3557675bd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.469+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec02cb90 con 0x7fd1e0004d00 2023-10-21T16:17:22.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec00f040 con 0x7fd1e0004d00 2023-10-21T16:17:22.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1f9181640 1 -- 172.21.15.53:0/3176694621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec034440 con 0x7fd1e0004d00 2023-10-21T16:17:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/3176694621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1e0004d00 msgr2=0x7fd1e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/3176694621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1e0004d00 0x7fd1e0005180 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009f90 tx=0x7fd1e00057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fa984640 1 -- 172.21.15.53:0/3176694621 reap_dead start 2023-10-21T16:17:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/3176694621 shutdown_connections 2023-10-21T16:17:22.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/3176694621 >> 172.21.15.53:0/3176694621 conn(0x7fd1f4069d50 msgr2=0x7fd1f406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/3176694621 shutdown_connections 2023-10-21T16:17:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.470+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/3176694621 wait complete. 2023-10-21T16:17:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.472+0000 7fd1fc3e7640 1 Processor -- start 2023-10-21T16:17:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.473+0000 7fd1fc3e7640 1 -- start start 2023-10-21T16:17:22.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1e0004d00 0x7fd1f41517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 0x7fd1f414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f414ee40 con 0x7fd1f414f770 2023-10-21T16:17:22.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4153da0 con 0x7fd1f414e8a0 2023-10-21T16:17:22.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fc3e7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4154080 con 0x7fd1e0004d00 2023-10-21T16:17:22.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39756/0 (socket says 172.21.15.53:39756) 2023-10-21T16:17:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 -- 172.21.15.53:0/2438820941 learned_addr learned my addr 172.21.15.53:0/2438820941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa183640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1e0004d00 0x7fd1f41517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1f9982640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 0x7fd1f414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 -- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1e0004d00 msgr2=0x7fd1f41517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1e0004d00 0x7fd1f41517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 -- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 msgr2=0x7fd1f414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.474+0000 7fd1fa984640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 0x7fd1f414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1fa984640 1 -- 172.21.15.53:0/2438820941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4154360 con 0x7fd1f414f770 2023-10-21T16:17:22.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1f9982640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 0x7fd1f414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1fa984640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec037040 tx=0x7fd1ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec034cd0 con 0x7fd1f414f770 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec034e30 con 0x7fd1f414f770 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec03e660 con 0x7fd1f414f770 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.475+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f4154640 con 0x7fd1f414f770 2023-10-21T16:17:22.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.476+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f40cbf30 con 0x7fd1f414f770 2023-10-21T16:17:22.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.477+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f41541c0 con 0x7fd1f414f770 2023-10-21T16:17:22.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.478+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd1ec03e7c0 con 0x7fd1f414f770 2023-10-21T16:17:22.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.479+0000 7fd1eaffd640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 0x7fd1d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.479+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fd1ec0ab840 con 0x7fd1f414f770 2023-10-21T16:17:22.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.479+0000 7fd1fa183640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 0x7fd1d806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.481+0000 7fd1fa183640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 0x7fd1d806d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009a90 tx=0x7fd1f0008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.483+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1f41541c0 con 0x7fd1f414f770 2023-10-21T16:17:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.677+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1f41541c0 con 0x7fd1f414f770 2023-10-21T16:17:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.678+0000 7fd1eaffd640 1 -- 172.21.15.53:0/2438820941 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1f41541c0 con 0x7fd1f414f770 2023-10-21T16:17:22.680 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.681+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 msgr2=0x7fd1d806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 0x7fd1d806d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009a90 tx=0x7fd1f0008040 comp rx=0 tx=0).stop 2023-10-21T16:17:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 msgr2=0x7fd1f414fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec037040 tx=0x7fd1ec009750 comp rx=0 tx=0).stop 2023-10-21T16:17:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 shutdown_connections 2023-10-21T16:17:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fd1d806afc0 0x7fd1d806d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1f414f770 0x7fd1f414fc20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1f414e8a0 0x7fd1f414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 --2- 172.21.15.53:0/2438820941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1e0004d00 0x7fd1f41517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 >> 172.21.15.53:0/2438820941 conn(0x7fd1f4069d50 msgr2=0x7fd1f4070400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 shutdown_connections 2023-10-21T16:17:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.682+0000 7fd1fc3e7640 1 -- 172.21.15.53:0/2438820941 wait complete. 2023-10-21T16:17:22.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:17:22.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.926+0000 7f4fc1ef8640 1 Processor -- start 2023-10-21T16:17:22.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.927+0000 7f4fc1ef8640 1 -- start start 2023-10-21T16:17:22.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.928+0000 7f4fc1ef8640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4fbc0adc70 0x7f4fbc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.928+0000 7f4fc1ef8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 0x7f4fbc0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.928+0000 7f4fc1ef8640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc15f7d0 con 0x7f4fbc15ab30 2023-10-21T16:17:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.928+0000 7f4fc1ef8640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc15efb0 con 0x7f4fbc0adc70 2023-10-21T16:17:22.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.928+0000 7f4fc1ef8640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15f210 con 0x7f4fbc15af10 2023-10-21T16:17:22.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.929+0000 7f4fc16f7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 0x7f4fbc0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.929+0000 7f4fbbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4fbc0adc70 0x7f4fbc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.929+0000 7f4fc16f7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 0x7f4fbc0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:51192/0 (socket says 172.21.15.53:51192) 2023-10-21T16:17:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.929+0000 7f4fc16f7640 1 -- 172.21.15.53:0/718439891 learned_addr learned my addr 172.21.15.53:0/718439891 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.930+0000 7f4fbbfff640 1 -- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 msgr2=0x7f4fbc0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:22.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.930+0000 7f4fbbfff640 1 --2- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 0x7f4fbc0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.930+0000 7f4fbbfff640 1 -- 172.21.15.53:0/718439891 >> v1:172.21.15.124:6789/0 conn(0x7f4fbc15ab30 legacy=0x7f4fbc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.930+0000 7f4fbbfff640 1 -- 172.21.15.53:0/718439891 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc15f4f0 con 0x7f4fbc0adc70 2023-10-21T16:17:22.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.931+0000 7f4fbbfff640 1 --2- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc0adc70 0x7f4fbc0b00c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097a0 tx=0x7f4fa402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3755163c892261e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.931+0000 7f4fbb7fe640 1 -- 172.21.15.53:0/718439891 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa402d040 con 0x7f4fbc0adc70 2023-10-21T16:17:22.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.931+0000 7f4fbb7fe640 1 -- 172.21.15.53:0/718439891 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4031810 con 0x7f4fbc0adc70 2023-10-21T16:17:22.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fbb7fe640 1 -- 172.21.15.53:0/718439891 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4031ba0 con 0x7f4fbc0adc70 2023-10-21T16:17:22.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc0adc70 msgr2=0x7f4fbc0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc0adc70 0x7f4fbc0b00c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097a0 tx=0x7f4fa402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/718439891 shutdown_connections 2023-10-21T16:17:22.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc15af10 0x7f4fbc0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/718439891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc0adc70 0x7f4fbc0b00c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/718439891 >> 172.21.15.53:0/718439891 conn(0x7f4fbc0cb0c0 msgr2=0x7f4fbc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:22.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/718439891 shutdown_connections 2023-10-21T16:17:22.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.932+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/718439891 wait complete. 2023-10-21T16:17:22.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.934+0000 7f4fc1ef8640 1 Processor -- start 2023-10-21T16:17:22.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.934+0000 7f4fc1ef8640 1 -- start start 2023-10-21T16:17:22.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc0adc70 0x7f4fbc1e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 0x7f4fbc14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15b380 con 0x7f4fbc15ab30 2023-10-21T16:17:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1e4bd0 con 0x7f4fbc15af10 2023-10-21T16:17:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.935+0000 7f4fc1ef8640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1e4eb0 con 0x7f4fbc0adc70 2023-10-21T16:17:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39762/0 (socket says 172.21.15.53:39762) 2023-10-21T16:17:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 -- 172.21.15.53:0/386256805 learned_addr learned my addr 172.21.15.53:0/386256805 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fc0ef6640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc0adc70 0x7f4fbc1e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fc16f7640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 0x7f4fbc14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 -- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc0adc70 msgr2=0x7f4fbc1e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc0adc70 0x7f4fbc1e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 -- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 msgr2=0x7f4fbc14e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:22.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 0x7f4fbc14e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:22.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fbbfff640 1 -- 172.21.15.53:0/386256805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4009400 con 0x7f4fbc15ab30 2023-10-21T16:17:22.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.936+0000 7f4fc16f7640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 0x7f4fbc14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fbbfff640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098c0 tx=0x7f4fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4006e10 con 0x7f4fbc15ab30 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa403ad60 con 0x7f4fbc15ab30 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc1e5190 con 0x7f4fbc15ab30 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4044620 con 0x7f4fbc15ab30 2023-10-21T16:17:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.937+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc1e63e0 con 0x7f4fbc15ab30 2023-10-21T16:17:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.938+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc1e56b0 con 0x7f4fbc15ab30 2023-10-21T16:17:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.939+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fa404d440 con 0x7f4fbc15ab30 2023-10-21T16:17:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.940+0000 7f4fb97fa640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.940+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f4fa40abab0 con 0x7f4fbc15ab30 2023-10-21T16:17:22.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.941+0000 7f4fc0ef6640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:22.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.942+0000 7f4fc0ef6640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 0x7f4f9c06d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f60 tx=0x7f4fac009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:22.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:22.945+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fa4076a80 con 0x7f4fbc15ab30 2023-10-21T16:17:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.140+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4fbc04e160 con 0x7f4fbc15ab30 2023-10-21T16:17:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.141+0000 7f4fb97fa640 1 -- 172.21.15.53:0/386256805 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4fa4035090 con 0x7f4fbc15ab30 2023-10-21T16:17:23.143 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:23.143 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:17:23.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:23.143 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:17:23.144 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:17:23.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:23.144 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:17:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.143+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 msgr2=0x7f4f9c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 0x7f4f9c06d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f60 tx=0x7f4fac009040 comp rx=0 tx=0).stop 2023-10-21T16:17:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 msgr2=0x7f4fbc159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40098c0 tx=0x7f4fa40046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 shutdown_connections 2023-10-21T16:17:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fbc15af10 0x7f4fbc14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4fbc15ab30 0x7f4fbc159420 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 --2- 172.21.15.53:0/386256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4fbc0adc70 0x7f4fbc1e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 >> 172.21.15.53:0/386256805 conn(0x7f4fbc0cb0c0 msgr2=0x7f4fbc0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:23.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 shutdown_connections 2023-10-21T16:17:23.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.144+0000 7f4fc1ef8640 1 -- 172.21.15.53:0/386256805 wait complete. 2023-10-21T16:17:23.164 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:17:23.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:17:23.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.400+0000 7f9e468ec640 1 Processor -- start 2023-10-21T16:17:23.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.401+0000 7f9e468ec640 1 -- start start 2023-10-21T16:17:23.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.402+0000 7f9e468ec640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 0x7f9e400cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.402+0000 7f9e468ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 0x7f9e400cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.402+0000 7f9e468ec640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e400cb810 con 0x7f9e400cce90 2023-10-21T16:17:23.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.402+0000 7f9e468ec640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4015ac90 con 0x7f9e400caef0 2023-10-21T16:17:23.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.402+0000 7f9e468ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4015a6d0 con 0x7f9e400cdc30 2023-10-21T16:17:23.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.403+0000 7f9e458ea640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 0x7f9e400cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.403+0000 7f9e450e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 0x7f9e400cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.403+0000 7f9e458ea640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 0x7f9e400cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:41764/0 (socket says 172.21.15.53:41764) 2023-10-21T16:17:23.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.403+0000 7f9e458ea640 1 -- 172.21.15.53:0/1996248897 learned_addr learned my addr 172.21.15.53:0/1996248897 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:23.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.404+0000 7f9e450e9640 1 -- 172.21.15.53:0/1996248897 >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 msgr2=0x7f9e400cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:23.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.404+0000 7f9e450e9640 1 --2- 172.21.15.53:0/1996248897 >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 0x7f9e400cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.404+0000 7f9e450e9640 1 -- 172.21.15.53:0/1996248897 >> v1:172.21.15.124:6789/0 conn(0x7f9e400cce90 legacy=0x7f9e40159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.404+0000 7f9e450e9640 1 -- 172.21.15.53:0/1996248897 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4015a9b0 con 0x7f9e400cdc30 2023-10-21T16:17:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.405+0000 7f9e450e9640 1 --2- 172.21.15.53:0/1996248897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 0x7f9e400cc950 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097a0 tx=0x7f9e3002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf55357fe927a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.405+0000 7f9e448e8640 1 -- 172.21.15.53:0/1996248897 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3002d040 con 0x7f9e400cdc30 2023-10-21T16:17:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e448e8640 1 -- 172.21.15.53:0/1996248897 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30031810 con 0x7f9e400cdc30 2023-10-21T16:17:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e448e8640 1 -- 172.21.15.53:0/1996248897 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30031ba0 con 0x7f9e400cdc30 2023-10-21T16:17:23.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 -- 172.21.15.53:0/1996248897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 msgr2=0x7f9e400cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1996248897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 0x7f9e400cc950 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9e300097a0 tx=0x7f9e3002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 -- 172.21.15.53:0/1996248897 shutdown_connections 2023-10-21T16:17:23.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1996248897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cdc30 0x7f9e400cc950 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1996248897 >> v2:172.21.15.124:3300/0 conn(0x7f9e400caef0 0x7f9e400cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 -- 172.21.15.53:0/1996248897 >> 172.21.15.53:0/1996248897 conn(0x7f9e4006c010 msgr2=0x7f9e4006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:23.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.406+0000 7f9e468ec640 1 -- 172.21.15.53:0/1996248897 shutdown_connections 2023-10-21T16:17:23.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.407+0000 7f9e468ec640 1 -- 172.21.15.53:0/1996248897 wait complete. 2023-10-21T16:17:23.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.409+0000 7f9e468ec640 1 Processor -- start 2023-10-21T16:17:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.409+0000 7f9e468ec640 1 -- start start 2023-10-21T16:17:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e400caef0 0x7f9e400c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cce90 0x7f9e4014c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c78a0 con 0x7f9e400cdc30 2023-10-21T16:17:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c6c90 con 0x7f9e400caef0 2023-10-21T16:17:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e468ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c6f70 con 0x7f9e400cce90 2023-10-21T16:17:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.410+0000 7f9e460eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39778/0 (socket says 172.21.15.53:39778) 2023-10-21T16:17:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 -- 172.21.15.53:0/1524557578 learned_addr learned my addr 172.21.15.53:0/1524557578 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e458ea640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e400caef0 0x7f9e400c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e450e9640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cce90 0x7f9e4014c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 -- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cce90 msgr2=0x7f9e4014c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cce90 0x7f9e4014c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 -- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e400caef0 msgr2=0x7f9e400c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:23.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e400caef0 0x7f9e400c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:23.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 -- 172.21.15.53:0/1524557578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009400 con 0x7f9e400cdc30 2023-10-21T16:17:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.411+0000 7f9e460eb640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c730 tx=0x7f9e3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.412+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38003b70 con 0x7f9e400cdc30 2023-10-21T16:17:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.412+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38003cd0 con 0x7f9e400cdc30 2023-10-21T16:17:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.412+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800aa90 con 0x7f9e400cdc30 2023-10-21T16:17:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.412+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e400c7250 con 0x7f9e400cdc30 2023-10-21T16:17:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.412+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4014cee0 con 0x7f9e400cdc30 2023-10-21T16:17:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.413+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4014d070 con 0x7f9e400cdc30 2023-10-21T16:17:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.414+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e3800abf0 con 0x7f9e400cdc30 2023-10-21T16:17:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.415+0000 7f9e367fc640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 0x7f9e2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.415+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f9e380893a0 con 0x7f9e400cdc30 2023-10-21T16:17:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.415+0000 7f9e458ea640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 0x7f9e2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.417+0000 7f9e458ea640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 0x7f9e2406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009b60 tx=0x7f9e3c008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.419+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e38054510 con 0x7f9e400cdc30 2023-10-21T16:17:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:23.613+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9e40074150 con 0x7f9e400cdc30 2023-10-21T16:17:24.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.229+0000 7f9e367fc640 1 -- 172.21.15.53:0/1524557578 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9e38057b30 con 0x7f9e400cdc30 2023-10-21T16:17:24.231 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:17:24.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 msgr2=0x7f9e2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 0x7f9e2406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009b60 tx=0x7f9e3c008040 comp rx=0 tx=0).stop 2023-10-21T16:17:24.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 msgr2=0x7f9e401f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c730 tx=0x7f9e3800cbf0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 shutdown_connections 2023-10-21T16:17:24.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9e2406af90 0x7f9e2406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e400cdc30 0x7f9e401f3750 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e400cce90 0x7f9e4014c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 --2- 172.21.15.53:0/1524557578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e400caef0 0x7f9e400c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.233+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 >> 172.21.15.53:0/1524557578 conn(0x7f9e4006c010 msgr2=0x7f9e400711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:24.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.234+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 shutdown_connections 2023-10-21T16:17:24.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.234+0000 7f9e468ec640 1 -- 172.21.15.53:0/1524557578 wait complete. 2023-10-21T16:17:24.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:17:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.490+0000 7f988c6ec640 1 Processor -- start 2023-10-21T16:17:24.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.491+0000 7f988c6ec640 1 -- start start 2023-10-21T16:17:24.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.492+0000 7f988c6ec640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f988406cfa0 0x7f988415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.492+0000 7f988c6ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.492+0000 7f988c6ec640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98840c8c60 con 0x7f98840cb500 2023-10-21T16:17:24.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.492+0000 7f988c6ec640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840b1ac0 con 0x7f988406cfa0 2023-10-21T16:17:24.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.492+0000 7f988c6ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840b1da0 con 0x7f988415b630 2023-10-21T16:17:24.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f988406cfa0 0x7f988415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f988406cfa0 0x7f988415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49450/0 (socket says 172.21.15.53:49450) 2023-10-21T16:17:24.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 -- 172.21.15.53:0/600645133 learned_addr learned my addr 172.21.15.53:0/600645133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:24.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f988ac89640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 -- 172.21.15.53:0/600645133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 msgr2=0x7f9884070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 -- 172.21.15.53:0/600645133 >> v1:172.21.15.53:6789/0 conn(0x7f98840cb500 legacy=0x7f98840cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.493+0000 7f9889c87640 1 -- 172.21.15.53:0/600645133 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840b2080 con 0x7f988406cfa0 2023-10-21T16:17:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.495+0000 7f9889c87640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f988415b0f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9874009870 tx=0x7f987402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3426ec467e2918ac server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.495+0000 7f9889486640 1 -- 172.21.15.53:0/600645133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874031640 con 0x7f988406cfa0 2023-10-21T16:17:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.495+0000 7f9889486640 1 -- 172.21.15.53:0/600645133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98740317a0 con 0x7f988406cfa0 2023-10-21T16:17:24.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.495+0000 7f9889486640 1 -- 172.21.15.53:0/600645133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874031b30 con 0x7f988406cfa0 2023-10-21T16:17:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 -- 172.21.15.53:0/600645133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 msgr2=0x7f988415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f988415b0f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9874009870 tx=0x7f987402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 -- 172.21.15.53:0/600645133 shutdown_connections 2023-10-21T16:17:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 --2- 172.21.15.53:0/600645133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f988415b0f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 -- 172.21.15.53:0/600645133 >> 172.21.15.53:0/600645133 conn(0x7f988406a140 msgr2=0x7f988406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 -- 172.21.15.53:0/600645133 shutdown_connections 2023-10-21T16:17:24.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.496+0000 7f988c6ec640 1 -- 172.21.15.53:0/600645133 wait complete. 2023-10-21T16:17:24.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.498+0000 7f988c6ec640 1 Processor -- start 2023-10-21T16:17:24.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.498+0000 7f988c6ec640 1 -- start start 2023-10-21T16:17:24.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98840cb500 0x7f98840ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98840ad090 con 0x7f988406cfa0 2023-10-21T16:17:24.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98840ad370 con 0x7f98840cb500 2023-10-21T16:17:24.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988c6ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840ad650 con 0x7f988415b630 2023-10-21T16:17:24.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988a488640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988a488640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49454/0 (socket says 172.21.15.53:49454) 2023-10-21T16:17:24.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.499+0000 7f988a488640 1 -- 172.21.15.53:0/3523393005 learned_addr learned my addr 172.21.15.53:0/3523393005 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:24.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f9889c87640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98840cb500 0x7f98840ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988ac89640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 -- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 msgr2=0x7f9884159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 -- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98840cb500 msgr2=0x7f98840ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:24.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98840cb500 0x7f98840ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:24.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 -- 172.21.15.53:0/3523393005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740094d0 con 0x7f988406cfa0 2023-10-21T16:17:24.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988a488640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f988000c400 tx=0x7f988000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:24.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800052d0 con 0x7f988406cfa0 2023-10-21T16:17:24.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880005450 con 0x7f988406cfa0 2023-10-21T16:17:24.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880007260 con 0x7f988406cfa0 2023-10-21T16:17:24.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.500+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98840ad930 con 0x7f988406cfa0 2023-10-21T16:17:24.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.501+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98840ac920 con 0x7f988406cfa0 2023-10-21T16:17:24.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.502+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98840ade50 con 0x7f988406cfa0 2023-10-21T16:17:24.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.503+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f98800073c0 con 0x7f988406cfa0 2023-10-21T16:17:24.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.504+0000 7f987affd640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 0x7f986806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:24.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.504+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f98800892b0 con 0x7f988406cfa0 2023-10-21T16:17:24.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.505+0000 7f9889c87640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 0x7f986806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:24.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.506+0000 7f9889c87640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 0x7f986806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9874036000 tx=0x7f987403c040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:24.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.508+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9880054430 con 0x7f988406cfa0 2023-10-21T16:17:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:24.704+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9884071cb0 con 0x7f988406cfa0 2023-10-21T16:17:25.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.247+0000 7f987affd640 1 -- 172.21.15.53:0/3523393005 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9880057a50 con 0x7f988406cfa0 2023-10-21T16:17:25.249 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:17:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.254+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 msgr2=0x7f986806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.254+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 0x7f986806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9874036000 tx=0x7f987403c040 comp rx=0 tx=0).stop 2023-10-21T16:17:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.254+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 msgr2=0x7f98840aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.254+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f988000c400 tx=0x7f988000c8c0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 shutdown_connections 2023-10-21T16:17:25.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f986806af90 0x7f986806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988415b630 0x7f9884159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98840cb500 0x7f98840ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 --2- 172.21.15.53:0/3523393005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f988406cfa0 0x7f98840aee00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.255+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 >> 172.21.15.53:0/3523393005 conn(0x7f988406a140 msgr2=0x7f98840cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:25.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.258+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 shutdown_connections 2023-10-21T16:17:25.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.258+0000 7f988c6ec640 1 -- 172.21.15.53:0/3523393005 wait complete. 2023-10-21T16:17:25.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:17:25.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.484+0000 7fe068a4d640 1 Processor -- start 2023-10-21T16:17:25.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.484+0000 7fe068a4d640 1 -- start start 2023-10-21T16:17:25.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe068a4d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe064159db0 0x7fe0640cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe068a4d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 0x7fe0640cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe068a4d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe064072250 con 0x7fe0640cbbe0 2023-10-21T16:17:25.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe068a4d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06415e200 con 0x7fe064159db0 2023-10-21T16:17:25.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe068a4d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06415e4e0 con 0x7fe064071dc0 2023-10-21T16:17:25.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe063fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe0640cbbe0 0x7fe0640b1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40862/0 (socket says 172.21.15.53:40862) 2023-10-21T16:17:25.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe063fff640 1 -- 172.21.15.53:0/3549376251 learned_addr learned my addr 172.21.15.53:0/3549376251 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:25.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe05affd640 1 --2- 172.21.15.53:0/3549376251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 0x7fe0640cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.486+0000 7fe0637fe640 1 --2- 172.21.15.53:0/3549376251 >> v2:172.21.15.124:3300/0 conn(0x7fe064159db0 0x7fe0640cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe062ffd640 1 -- 172.21.15.53:0/3549376251 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697123692 0 0) 0x7fe064072250 con 0x7fe0640cbbe0 2023-10-21T16:17:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe062ffd640 1 -- 172.21.15.53:0/3549376251 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003520 con 0x7fe0640cbbe0 2023-10-21T16:17:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe05affd640 1 -- 172.21.15.53:0/3549376251 >> v2:172.21.15.124:3300/0 conn(0x7fe064159db0 msgr2=0x7fe0640cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe05affd640 1 --2- 172.21.15.53:0/3549376251 >> v2:172.21.15.124:3300/0 conn(0x7fe064159db0 0x7fe0640cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe05affd640 1 -- 172.21.15.53:0/3549376251 >> v1:172.21.15.53:6789/0 conn(0x7fe0640cbbe0 legacy=0x7fe0640b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.487+0000 7fe05affd640 1 -- 172.21.15.53:0/3549376251 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0640cb180 con 0x7fe064071dc0 2023-10-21T16:17:25.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.489+0000 7fe05affd640 1 --2- 172.21.15.53:0/3549376251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 0x7fe0640cd130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0097a0 tx=0x7fe04c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61329200f9562196 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:25.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.489+0000 7fe062ffd640 1 -- 172.21.15.53:0/3549376251 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c02d040 con 0x7fe064071dc0 2023-10-21T16:17:25.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.489+0000 7fe062ffd640 1 -- 172.21.15.53:0/3549376251 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c031810 con 0x7fe064071dc0 2023-10-21T16:17:25.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.489+0000 7fe062ffd640 1 -- 172.21.15.53:0/3549376251 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c031b80 con 0x7fe064071dc0 2023-10-21T16:17:25.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 -- 172.21.15.53:0/3549376251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 msgr2=0x7fe0640cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 --2- 172.21.15.53:0/3549376251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 0x7fe0640cd130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0097a0 tx=0x7fe04c02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 -- 172.21.15.53:0/3549376251 shutdown_connections 2023-10-21T16:17:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 --2- 172.21.15.53:0/3549376251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064071dc0 0x7fe0640cd130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 --2- 172.21.15.53:0/3549376251 >> v2:172.21.15.124:3300/0 conn(0x7fe064159db0 0x7fe0640cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 -- 172.21.15.53:0/3549376251 >> 172.21.15.53:0/3549376251 conn(0x7fe0640cf5f0 msgr2=0x7fe0640cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 -- 172.21.15.53:0/3549376251 shutdown_connections 2023-10-21T16:17:25.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.490+0000 7fe068a4d640 1 -- 172.21.15.53:0/3549376251 wait complete. 2023-10-21T16:17:25.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.492+0000 7fe068a4d640 1 Processor -- start 2023-10-21T16:17:25.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.492+0000 7fe068a4d640 1 -- start start 2023-10-21T16:17:25.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064159db0 0x7fe064151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0640c3bb0 0x7fe064153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06415b170 con 0x7fe0640cbbe0 2023-10-21T16:17:25.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640c3990 con 0x7fe0640c3bb0 2023-10-21T16:17:25.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe068a4d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640c4040 con 0x7fe064159db0 2023-10-21T16:17:25.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:25.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe0637fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49464/0 (socket says 172.21.15.53:49464) 2023-10-21T16:17:25.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.493+0000 7fe0637fe640 1 -- 172.21.15.53:0/1515764483 learned_addr learned my addr 172.21.15.53:0/1515764483 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:25.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe05affd640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064159db0 0x7fe064151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:25.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 -- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064159db0 msgr2=0x7fe064151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064159db0 0x7fe064151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 -- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0640c3bb0 msgr2=0x7fe064153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:25.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0640c3bb0 0x7fe064153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:25.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 -- 172.21.15.53:0/1515764483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c009400 con 0x7fe0640cbbe0 2023-10-21T16:17:25.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe0637fe640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe05400efc0 tx=0x7fe05400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:25.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054014480 con 0x7fe0640cbbe0 2023-10-21T16:17:25.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05400f040 con 0x7fe0640cbbe0 2023-10-21T16:17:25.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0640c4320 con 0x7fe0640cbbe0 2023-10-21T16:17:25.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.494+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054014800 con 0x7fe0640cbbe0 2023-10-21T16:17:25.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.495+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06415b6d0 con 0x7fe0640cbbe0 2023-10-21T16:17:25.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.496+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06415b8c0 con 0x7fe0640cbbe0 2023-10-21T16:17:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.497+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe054014960 con 0x7fe0640cbbe0 2023-10-21T16:17:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.498+0000 7fe060ff9640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 0x7fe04406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.498+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5973+0+0 (secure 0 0 0) 0x7fe05408ade0 con 0x7fe0640cbbe0 2023-10-21T16:17:25.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.499+0000 7fe05affd640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 0x7fe04406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:25.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.500+0000 7fe05affd640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 0x7fe04406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c008030 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:25.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.502+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe054055b40 con 0x7fe0640cbbe0 2023-10-21T16:17:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:25.735+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe064074150 con 0x7fe0640cbbe0 2023-10-21T16:17:26.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.284+0000 7fe060ff9640 1 -- 172.21.15.53:0/1515764483 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) v1 ==== 173+0+0 (secure 0 0 0) 0x7fe054059160 con 0x7fe0640cbbe0 2023-10-21T16:17:26.286 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 9 and data pool 10 2023-10-21T16:17:26.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 msgr2=0x7fe04406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 0x7fe04406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe04c036000 tx=0x7fe04c008030 comp rx=0 tx=0).stop 2023-10-21T16:17:26.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 msgr2=0x7fe06415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe05400efc0 tx=0x7fe05400c490 comp rx=0 tx=0).stop 2023-10-21T16:17:26.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 shutdown_connections 2023-10-21T16:17:26.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe04406af90 0x7fe04406d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0640c3bb0 0x7fe064153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe064159db0 0x7fe064151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 --2- 172.21.15.53:0/1515764483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0640cbbe0 0x7fe06415ac30 secure :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe05400efc0 tx=0x7fe05400c490 comp rx=0 tx=0).stop 2023-10-21T16:17:26.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.294+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 >> 172.21.15.53:0/1515764483 conn(0x7fe0640cf5f0 msgr2=0x7fe0640b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:26.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.296+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 shutdown_connections 2023-10-21T16:17:26.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.297+0000 7fe068a4d640 1 -- 172.21.15.53:0/1515764483 wait complete. 2023-10-21T16:17:26.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:26.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.512+0000 7faf05283640 1 Processor -- start 2023-10-21T16:17:26.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.512+0000 7faf05283640 1 -- start start 2023-10-21T16:17:26.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.513+0000 7faf05283640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf000b1840 0x7faf000d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.513+0000 7faf05283640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.513+0000 7faf05283640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000b8bf0 con 0x7faf000cd8d0 2023-10-21T16:17:26.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.513+0000 7faf05283640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000b8630 con 0x7faf000b1840 2023-10-21T16:17:26.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.513+0000 7faf05283640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000b8910 con 0x7faf000b1d90 2023-10-21T16:17:26.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.514+0000 7faeff7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.514+0000 7faefe7fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf000b1840 0x7faf000d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.514+0000 7faeff7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:33666/0 (socket says 172.21.15.53:33666) 2023-10-21T16:17:26.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.514+0000 7faeff7fe640 1 -- 172.21.15.53:0/4107993151 learned_addr learned my addr 172.21.15.53:0/4107993151 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:26.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.515+0000 7faeff7fe640 1 -- 172.21.15.53:0/4107993151 >> v2:172.21.15.124:3300/0 conn(0x7faf000b1840 msgr2=0x7faf000d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:26.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.515+0000 7faeff7fe640 1 --2- 172.21.15.53:0/4107993151 >> v2:172.21.15.124:3300/0 conn(0x7faf000b1840 0x7faf000d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.515+0000 7faeff7fe640 1 -- 172.21.15.53:0/4107993151 >> v1:172.21.15.124:6789/0 conn(0x7faf000cd8d0 legacy=0x7faf000cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.515+0000 7faeff7fe640 1 -- 172.21.15.53:0/4107993151 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000d1100 con 0x7faf000b1d90 2023-10-21T16:17:26.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.516+0000 7faeff7fe640 1 --2- 172.21.15.53:0/4107993151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef8009870 tx=0x7faef802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7421f9b5749bce55 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.516+0000 7faefdffb640 1 -- 172.21.15.53:0/4107993151 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8030840 con 0x7faf000b1d90 2023-10-21T16:17:26.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faefdffb640 1 -- 172.21.15.53:0/4107993151 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef80309a0 con 0x7faf000b1d90 2023-10-21T16:17:26.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faefdffb640 1 -- 172.21.15.53:0/4107993151 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8030d30 con 0x7faf000b1d90 2023-10-21T16:17:26.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 -- 172.21.15.53:0/4107993151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 msgr2=0x7faf000d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 --2- 172.21.15.53:0/4107993151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faef8009870 tx=0x7faef802b980 comp rx=0 tx=0).stop 2023-10-21T16:17:26.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 -- 172.21.15.53:0/4107993151 shutdown_connections 2023-10-21T16:17:26.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 --2- 172.21.15.53:0/4107993151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000d1e40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 --2- 172.21.15.53:0/4107993151 >> v2:172.21.15.124:3300/0 conn(0x7faf000b1840 0x7faf000d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 -- 172.21.15.53:0/4107993151 >> 172.21.15.53:0/4107993151 conn(0x7faf0006b6a0 msgr2=0x7faf0006bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:26.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 -- 172.21.15.53:0/4107993151 shutdown_connections 2023-10-21T16:17:26.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.517+0000 7faf05283640 1 -- 172.21.15.53:0/4107993151 wait complete. 2023-10-21T16:17:26.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.519+0000 7faf05283640 1 Processor -- start 2023-10-21T16:17:26.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.519+0000 7faf05283640 1 -- start start 2023-10-21T16:17:26.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 0x7faf000c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf00157ea0 con 0x7faf000cd8d0 2023-10-21T16:17:26.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000cac90 con 0x7faf000b1840 2023-10-21T16:17:26.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.520+0000 7faf05283640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf00154880 con 0x7faf000b1d90 2023-10-21T16:17:26.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49466/0 (socket says 172.21.15.53:49466) 2023-10-21T16:17:26.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 -- 172.21.15.53:0/315911014 learned_addr learned my addr 172.21.15.53:0/315911014 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:26.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faefeffd640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 0x7faf000c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faefe7fc640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 -- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 msgr2=0x7faf000ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 -- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 msgr2=0x7faf000c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 0x7faf000c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faeff7fe640 1 -- 172.21.15.53:0/315911014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80094d0 con 0x7faf000cd8d0 2023-10-21T16:17:26.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faefeffd640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 0x7faf000c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:26.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.521+0000 7faefe7fc640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:26.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faeff7fe640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7faef8009870 tx=0x7faef8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8039c90 con 0x7faf000cd8d0 2023-10-21T16:17:26.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8039e10 con 0x7faf000cd8d0 2023-10-21T16:17:26.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8043640 con 0x7faf000cd8d0 2023-10-21T16:17:26.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000c7cb0 con 0x7faf000cd8d0 2023-10-21T16:17:26.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.522+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf000c8270 con 0x7faf000cd8d0 2023-10-21T16:17:26.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.523+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf00066d80 con 0x7faf000cd8d0 2023-10-21T16:17:26.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.524+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faef80437a0 con 0x7faf000cd8d0 2023-10-21T16:17:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.525+0000 7faee77fe640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 0x7faeec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.525+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faef803e070 con 0x7faf000cd8d0 2023-10-21T16:17:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.526+0000 7faefeffd640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 0x7faeec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.527+0000 7faefeffd640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 0x7faeec06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faef0007fd0 tx=0x7faef0009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.530+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faef8077720 con 0x7faf000cd8d0 2023-10-21T16:17:26.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.721+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf000c9a90 con 0x7faf000cd8d0 2023-10-21T16:17:26.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.724+0000 7faee77fe640 1 -- 172.21.15.53:0/315911014 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16787 (secure 0 0 0) 0x7faef807ad40 con 0x7faf000cd8d0 2023-10-21T16:17:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:26.726 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":89,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:25.737138+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:26.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 msgr2=0x7faeec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 0x7faeec06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faef0007fd0 tx=0x7faef0009040 comp rx=0 tx=0).stop 2023-10-21T16:17:26.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 msgr2=0x7faf00154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7faef8009870 tx=0x7faef8004730 comp rx=0 tx=0).stop 2023-10-21T16:17:26.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 shutdown_connections 2023-10-21T16:17:26.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7faeec06af90 0x7faeec06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf000cd8d0 0x7faf00154340 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf000b1d90 0x7faf000ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 --2- 172.21.15.53:0/315911014 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf000b1840 0x7faf000c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 >> 172.21.15.53:0/315911014 conn(0x7faf0006b6a0 msgr2=0x7faf000b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:26.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 shutdown_connections 2023-10-21T16:17:26.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.726+0000 7faf05283640 1 -- 172.21.15.53:0/315911014 wait complete. 2023-10-21T16:17:26.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.969+0000 7fa222abb640 1 Processor -- start 2023-10-21T16:17:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.969+0000 7fa222abb640 1 -- start start 2023-10-21T16:17:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.971+0000 7fa222abb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa21c15cca0 0x7fa21c1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.971+0000 7fa222abb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 0x7fa21c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.971+0000 7fa222abb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c0c9020 con 0x7fa21c15d080 2023-10-21T16:17:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.971+0000 7fa222abb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0ccc20 con 0x7fa21c15cca0 2023-10-21T16:17:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.971+0000 7fa222abb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0c92e0 con 0x7fa21c0cab40 2023-10-21T16:17:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa21c15cca0 0x7fa21c1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa21c15cca0 0x7fa21c1614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49468/0 (socket says 172.21.15.53:49468) 2023-10-21T16:17:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 -- 172.21.15.53:0/631038731 learned_addr learned my addr 172.21.15.53:0/631038731 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa2212b8640 1 --2- 172.21.15.53:0/631038731 >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 0x7fa21c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 -- 172.21.15.53:0/631038731 >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 msgr2=0x7fa21c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 --2- 172.21.15.53:0/631038731 >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 0x7fa21c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 -- 172.21.15.53:0/631038731 >> v1:172.21.15.124:6789/0 conn(0x7fa21c15d080 legacy=0x7fa21c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa221ab9640 1 -- 172.21.15.53:0/631038731 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0cc7b0 con 0x7fa21c15cca0 2023-10-21T16:17:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.972+0000 7fa2212b8640 1 --2- 172.21.15.53:0/631038731 >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 0x7fa21c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.973+0000 7fa221ab9640 1 --2- 172.21.15.53:0/631038731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c1614b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cc2c0 tx=0x7fa21802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ba1d8586ee038 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa220ab7640 1 -- 172.21.15.53:0/631038731 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180317e0 con 0x7fa21c15cca0 2023-10-21T16:17:26.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa220ab7640 1 -- 172.21.15.53:0/631038731 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218031940 con 0x7fa21c15cca0 2023-10-21T16:17:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa220ab7640 1 -- 172.21.15.53:0/631038731 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218031cd0 con 0x7fa21c15cca0 2023-10-21T16:17:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 -- 172.21.15.53:0/631038731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 msgr2=0x7fa21c1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 --2- 172.21.15.53:0/631038731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c1614b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cc2c0 tx=0x7fa21802b880 comp rx=0 tx=0).stop 2023-10-21T16:17:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 -- 172.21.15.53:0/631038731 shutdown_connections 2023-10-21T16:17:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 --2- 172.21.15.53:0/631038731 >> v2:172.21.15.124:3300/0 conn(0x7fa21c0cab40 0x7fa21c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 --2- 172.21.15.53:0/631038731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c1614b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.974+0000 7fa222abb640 1 -- 172.21.15.53:0/631038731 >> 172.21.15.53:0/631038731 conn(0x7fa21c0cb0c0 msgr2=0x7fa21c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.975+0000 7fa222abb640 1 -- 172.21.15.53:0/631038731 shutdown_connections 2023-10-21T16:17:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.975+0000 7fa222abb640 1 -- 172.21.15.53:0/631038731 wait complete. 2023-10-21T16:17:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.976+0000 7fa222abb640 1 Processor -- start 2023-10-21T16:17:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.976+0000 7fa222abb640 1 -- start start 2023-10-21T16:17:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa21c0cab40 0x7fa21c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 0x7fa21c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c15d490 con 0x7fa21c15cca0 2023-10-21T16:17:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0c8670 con 0x7fa21c15d080 2023-10-21T16:17:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.977+0000 7fa222abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c15d730 con 0x7fa21c0cab40 2023-10-21T16:17:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49478/0 (socket says 172.21.15.53:49478) 2023-10-21T16:17:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 -- 172.21.15.53:0/3610238425 learned_addr learned my addr 172.21.15.53:0/3610238425 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 -- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa21c0cab40 msgr2=0x7fa21c0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa21c0cab40 0x7fa21c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2222ba640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 0x7fa21c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 -- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 msgr2=0x7fa21c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 0x7fa21c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2212b8640 1 -- 172.21.15.53:0/3610238425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180094d0 con 0x7fa21c15cca0 2023-10-21T16:17:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.978+0000 7fa2222ba640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 0x7fa21c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa2212b8640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00d7d0 tx=0x7fa20c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004030 con 0x7fa21c15cca0 2023-10-21T16:17:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20c004190 con 0x7fa21c15cca0 2023-10-21T16:17:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004520 con 0x7fa21c15cca0 2023-10-21T16:17:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c15da10 con 0x7fa21c15cca0 2023-10-21T16:17:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.979+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c15dfd0 con 0x7fa21c15cca0 2023-10-21T16:17:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.980+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c04e160 con 0x7fa21c15cca0 2023-10-21T16:17:26.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.981+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa20c00b6c0 con 0x7fa21c15cca0 2023-10-21T16:17:26.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.982+0000 7fa2127fc640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 0x7fa20006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:26.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.982+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa20c089af0 con 0x7fa21c15cca0 2023-10-21T16:17:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.983+0000 7fa221ab9640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 0x7fa20006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.984+0000 7fa221ab9640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 0x7fa20006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa218036000 tx=0x7fa218030000 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:26.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:26.986+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa20c054880 con 0x7fa21c15cca0 2023-10-21T16:17:27.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.181+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa21c0c8ae0 con 0x7fa21c15cca0 2023-10-21T16:17:27.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.184+0000 7fa2127fc640 1 -- 172.21.15.53:0/3610238425 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fa20c057ea0 con 0x7fa21c15cca0 2023-10-21T16:17:27.186 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:27.186 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":89,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:25.737138+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:27.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 msgr2=0x7fa20006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 0x7fa20006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa218036000 tx=0x7fa218030000 comp rx=0 tx=0).stop 2023-10-21T16:17:27.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 msgr2=0x7fa21c15e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00d7d0 tx=0x7fa20c00dc90 comp rx=0 tx=0).stop 2023-10-21T16:17:27.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 shutdown_connections 2023-10-21T16:17:27.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fa20006af90 0x7fa20006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa21c15d080 0x7fa21c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa21c15cca0 0x7fa21c15e4f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 --2- 172.21.15.53:0/3610238425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa21c0cab40 0x7fa21c0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 >> 172.21.15.53:0/3610238425 conn(0x7fa21c0cb0c0 msgr2=0x7fa21c0c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:27.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 shutdown_connections 2023-10-21T16:17:27.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.187+0000 7fa222abb640 1 -- 172.21.15.53:0/3610238425 wait complete. 2023-10-21T16:17:27.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:17:27.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.417+0000 7f5d7f415640 1 Processor -- start 2023-10-21T16:17:27.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.417+0000 7f5d7f415640 1 -- start start 2023-10-21T16:17:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7f415640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d780cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7f415640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780b1cc0 con 0x7f5d780afbe0 2023-10-21T16:17:27.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7f415640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780cbad0 con 0x7f5d7806dfc0 2023-10-21T16:17:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7f415640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780b2620 con 0x7f5d780cb130 2023-10-21T16:17:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7d1b1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5d780afbe0 0x7f5d780b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40868/0 (socket says 172.21.15.53:40868) 2023-10-21T16:17:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.419+0000 7f5d7d1b1640 1 -- 172.21.15.53:0/1912987567 learned_addr learned my addr 172.21.15.53:0/1912987567 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:27.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d780cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322310489 0 0) 0x7f5d780b1cc0 con 0x7f5d780afbe0 2023-10-21T16:17:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d64003520 con 0x7f5d780afbe0 2023-10-21T16:17:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857656661 0 0) 0x7f5d64003520 con 0x7f5d780afbe0 2023-10-21T16:17:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 msgr2=0x7f5d780cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:27.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d780cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 >> v1:172.21.15.124:6789/0 conn(0x7f5d7806dfc0 legacy=0x7f5d78070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780b2060 con 0x7f5d780afbe0 2023-10-21T16:17:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5d74001670 con 0x7f5d780afbe0 2023-10-21T16:17:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d780cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:27.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 >> v1:172.21.15.53:6789/0 conn(0x7f5d780afbe0 legacy=0x7f5d780b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d64003f90 0x7f5d640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.420+0000 7f5d6ffff640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d64004d00 0x7f5d64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780b2620 con 0x7f5d64003f90 2023-10-21T16:17:27.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6400bae0 con 0x7f5d64004d00 2023-10-21T16:17:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6400bca0 con 0x7f5d780cb130 2023-10-21T16:17:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d64003f90 0x7f5d640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7d1b1640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d64004d00 0x7f5d64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 -- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 msgr2=0x7f5d640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d780cb130 0x7f5d640039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 -- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d64004d00 msgr2=0x7f5d64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d64004d00 0x7f5d64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.421+0000 7f5d7c9b0640 1 -- 172.21.15.53:0/1912987567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64003520 con 0x7f5d64003f90 2023-10-21T16:17:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d64003f90 0x7f5d640047c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001720 tx=0x7f5d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acae0412f665a4cf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680052d0 con 0x7f5d64003f90 2023-10-21T16:17:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802e050 con 0x7f5d64003f90 2023-10-21T16:17:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d6ffff640 1 -- 172.21.15.53:0/1912987567 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68003830 con 0x7f5d64003f90 2023-10-21T16:17:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d7f415640 1 -- 172.21.15.53:0/1912987567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d64003f90 msgr2=0x7f5d640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d7f415640 1 --2- 172.21.15.53:0/1912987567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d64003f90 0x7f5d640047c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001720 tx=0x7f5d640057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.423+0000 7f5d7d9b2640 1 -- 172.21.15.53:0/1912987567 reap_dead start 2023-10-21T16:17:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.424+0000 7f5d7f415640 1 -- 172.21.15.53:0/1912987567 shutdown_connections 2023-10-21T16:17:27.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.424+0000 7f5d7f415640 1 -- 172.21.15.53:0/1912987567 >> 172.21.15.53:0/1912987567 conn(0x7f5d7806a5d0 msgr2=0x7f5d780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.424+0000 7f5d7f415640 1 -- 172.21.15.53:0/1912987567 shutdown_connections 2023-10-21T16:17:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.424+0000 7f5d7f415640 1 -- 172.21.15.53:0/1912987567 wait complete. 2023-10-21T16:17:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.426+0000 7f5d7f415640 1 Processor -- start 2023-10-21T16:17:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.426+0000 7f5d7f415640 1 -- start start 2023-10-21T16:17:27.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 0x7f5d781f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 0x7f5d7806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c2c80 con 0x7f5d780c26d0 2023-10-21T16:17:27.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c2f60 con 0x7f5d780c2250 2023-10-21T16:17:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7f415640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c3240 con 0x7f5d64003f90 2023-10-21T16:17:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7d9b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7d9b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49496/0 (socket says 172.21.15.53:49496) 2023-10-21T16:17:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7d9b2640 1 -- 172.21.15.53:0/3021331399 learned_addr learned my addr 172.21.15.53:0/3021331399 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 0x7f5d7806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.427+0000 7f5d7d1b1640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 0x7f5d781f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 -- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 msgr2=0x7f5d781f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 0x7f5d781f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 -- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 msgr2=0x7f5d7806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 0x7f5d7806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 -- 172.21.15.53:0/3021331399 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780c3520 con 0x7f5d780c26d0 2023-10-21T16:17:27.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7c9b0640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 0x7f5d7806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d1b1640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 0x7f5d781f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7d9b2640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5d780cb080 tx=0x7f5d70004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70007cf0 con 0x7f5d780c26d0 2023-10-21T16:17:27.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000f050 con 0x7f5d780c26d0 2023-10-21T16:17:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70014650 con 0x7f5d780c26d0 2023-10-21T16:17:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780c3800 con 0x7f5d780c26d0 2023-10-21T16:17:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.428+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7815d500 con 0x7f5d780c26d0 2023-10-21T16:17:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.430+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d781f0730 con 0x7f5d780c26d0 2023-10-21T16:17:27.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.431+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d700147b0 con 0x7f5d780c26d0 2023-10-21T16:17:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.431+0000 7f5d6dffb640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 0x7f5d5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.432+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5d70089fa0 con 0x7f5d780c26d0 2023-10-21T16:17:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.432+0000 7f5d7d1b1640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 0x7f5d5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.433+0000 7f5d7d1b1640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 0x7f5d5c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d74007c60 tx=0x7f5d74009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:27.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.436+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d70054cb0 con 0x7f5d780c26d0 2023-10-21T16:17:27.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:27.669+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5d7804e160 con 0x7f5d780c26d0 2023-10-21T16:17:28.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.314+0000 7f5d6dffb640 1 -- 172.21.15.53:0/3021331399 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5d700582d0 con 0x7f5d780c26d0 2023-10-21T16:17:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.317+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 msgr2=0x7f5d5c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.317+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 0x7f5d5c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5d74007c60 tx=0x7f5d74009040 comp rx=0 tx=0).stop 2023-10-21T16:17:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.317+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 msgr2=0x7f5d7806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.317+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5d780cb080 tx=0x7f5d70004780 comp rx=0 tx=0).stop 2023-10-21T16:17:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 shutdown_connections 2023-10-21T16:17:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f5d5c06af90 0x7f5d5c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d780c26d0 0x7f5d7806d7d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d780c2250 0x7f5d7806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 --2- 172.21.15.53:0/3021331399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d64003f90 0x7f5d781f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 >> 172.21.15.53:0/3021331399 conn(0x7f5d7806a5d0 msgr2=0x7f5d780cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 shutdown_connections 2023-10-21T16:17:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.318+0000 7f5d7f415640 1 -- 172.21.15.53:0/3021331399 wait complete. 2023-10-21T16:17:28.340 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:17:28.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:17:28.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.566+0000 7fe9c070c640 1 Processor -- start 2023-10-21T16:17:28.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.566+0000 7fe9c070c640 1 -- start start 2023-10-21T16:17:28.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9c070c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9b80c9cf0 0x7fe9b806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9c070c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b815aa00 0x7fe9b80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9c070c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b815b310 con 0x7fe9b806c3a0 2023-10-21T16:17:28.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9c070c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b806e0a0 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9c070c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806d290 con 0x7fe9b815aa00 2023-10-21T16:17:28.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9bdca7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9b80c9cf0 0x7fe9b806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:28.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9bdca7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe9b80c9cf0 0x7fe9b806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49512/0 (socket says 172.21.15.53:49512) 2023-10-21T16:17:28.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.568+0000 7fe9bdca7640 1 -- 172.21.15.53:0/3049319050 learned_addr learned my addr 172.21.15.53:0/3049319050 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:28.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.569+0000 7fe9beca9640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b815aa00 0x7fe9b80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:28.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.569+0000 7fe9bdca7640 1 -- 172.21.15.53:0/3049319050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b815aa00 msgr2=0x7fe9b80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:28.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.569+0000 7fe9bdca7640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b815aa00 0x7fe9b80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.569+0000 7fe9bdca7640 1 -- 172.21.15.53:0/3049319050 >> v1:172.21.15.53:6789/0 conn(0x7fe9b806c3a0 legacy=0x7fe9b806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.569+0000 7fe9bdca7640 1 -- 172.21.15.53:0/3049319050 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80c9860 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.570+0000 7fe9bdca7640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b80c9cf0 0x7fe9b806db60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009870 tx=0x7fe9a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3cd1a67ce09423 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:28.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.570+0000 7fe9bd4a6640 1 -- 172.21.15.53:0/3049319050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8030840 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.570+0000 7fe9bd4a6640 1 -- 172.21.15.53:0/3049319050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a80309a0 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.570+0000 7fe9bd4a6640 1 -- 172.21.15.53:0/3049319050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8030d30 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 -- 172.21.15.53:0/3049319050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b80c9cf0 msgr2=0x7fe9b806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b80c9cf0 0x7fe9b806db60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009870 tx=0x7fe9a802b980 comp rx=0 tx=0).stop 2023-10-21T16:17:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 -- 172.21.15.53:0/3049319050 shutdown_connections 2023-10-21T16:17:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b815aa00 0x7fe9b80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 --2- 172.21.15.53:0/3049319050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b80c9cf0 0x7fe9b806db60 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 -- 172.21.15.53:0/3049319050 >> 172.21.15.53:0/3049319050 conn(0x7fe9b806a5d0 msgr2=0x7fe9b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:28.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 -- 172.21.15.53:0/3049319050 shutdown_connections 2023-10-21T16:17:28.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.571+0000 7fe9c070c640 1 -- 172.21.15.53:0/3049319050 wait complete. 2023-10-21T16:17:28.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.573+0000 7fe9c070c640 1 Processor -- start 2023-10-21T16:17:28.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.573+0000 7fe9c070c640 1 -- start start 2023-10-21T16:17:28.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b806c3a0 0x7fe9b80ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 0x7fe9b80cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80c6f50 con 0x7fe9b815aa00 2023-10-21T16:17:28.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b81f34c0 con 0x7fe9b806c3a0 2023-10-21T16:17:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9c070c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80c6900 con 0x7fe9b80c9cf0 2023-10-21T16:17:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9beca9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9beca9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49528/0 (socket says 172.21.15.53:49528) 2023-10-21T16:17:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.574+0000 7fe9beca9640 1 -- 172.21.15.53:0/1443598911 learned_addr learned my addr 172.21.15.53:0/1443598911 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9bdca7640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 0x7fe9b80cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 -- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 msgr2=0x7fe9b80cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 0x7fe9b80cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 -- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b806c3a0 msgr2=0x7fe9b80ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b806c3a0 0x7fe9b80ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 -- 172.21.15.53:0/1443598911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80094d0 con 0x7fe9b815aa00 2023-10-21T16:17:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9bdca7640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 0x7fe9b80cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9beca9640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000cc30 tx=0x7fe9b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000ce40 con 0x7fe9b815aa00 2023-10-21T16:17:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0003990 con 0x7fe9b815aa00 2023-10-21T16:17:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0003d20 con 0x7fe9b815aa00 2023-10-21T16:17:28.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b80c6be0 con 0x7fe9b815aa00 2023-10-21T16:17:28.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.575+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b81f3ac0 con 0x7fe9b815aa00 2023-10-21T16:17:28.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.577+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b80c6d70 con 0x7fe9b815aa00 2023-10-21T16:17:28.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.578+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe9b0003e80 con 0x7fe9b815aa00 2023-10-21T16:17:28.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.579+0000 7fe9aeffd640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 0x7fe99c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:28.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.579+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9b0089ee0 con 0x7fe9b815aa00 2023-10-21T16:17:28.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.579+0000 7fe9be4a8640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 0x7fe99c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:28.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.580+0000 7fe9be4a8640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 0x7fe99c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f60 tx=0x7fe9b40079b0 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:28.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.583+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9b0054b60 con 0x7fe9b815aa00 2023-10-21T16:17:28.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:28.813+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9b804e160 con 0x7fe9b815aa00 2023-10-21T16:17:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.324+0000 7fe9aeffd640 1 -- 172.21.15.53:0/1443598911 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9b001d070 con 0x7fe9b815aa00 2023-10-21T16:17:29.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 msgr2=0x7fe99c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 0x7fe99c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f60 tx=0x7fe9b40079b0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 msgr2=0x7fe9b80c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000cc30 tx=0x7fe9b000a5c0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 shutdown_connections 2023-10-21T16:17:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe99c06af90 0x7fe99c06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9b815aa00 0x7fe9b80c73e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9b80c9cf0 0x7fe9b80cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 --2- 172.21.15.53:0/1443598911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9b806c3a0 0x7fe9b80ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.328+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 >> 172.21.15.53:0/1443598911 conn(0x7fe9b806a5d0 msgr2=0x7fe9b81572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:29.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.329+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 shutdown_connections 2023-10-21T16:17:29.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.329+0000 7fe9c070c640 1 -- 172.21.15.53:0/1443598911 wait complete. 2023-10-21T16:17:29.351 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:17:29.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:29.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.579+0000 7f9a5a027640 1 Processor -- start 2023-10-21T16:17:29.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.579+0000 7f9a5a027640 1 -- start start 2023-10-21T16:17:29.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a5415ca60 0x7f9a540af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a540ad400 0x7f9a540cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a540ad8b0 0x7f9a540cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540d0030 con 0x7f9a540ad400 2023-10-21T16:17:29.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a540ceba0 con 0x7f9a5415ca60 2023-10-21T16:17:29.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a5a027640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5406ba80 con 0x7f9a540ad8b0 2023-10-21T16:17:29.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a58824640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a540ad400 0x7f9a540cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:29.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a58824640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a540ad400 0x7f9a540cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49540/0 (socket says 172.21.15.53:49540) 2023-10-21T16:17:29.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a58824640 1 -- 172.21.15.53:0/3497841821 learned_addr learned my addr 172.21.15.53:0/3497841821 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:29.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a59025640 1 --2- 172.21.15.53:0/3497841821 >> v2:172.21.15.124:3300/0 conn(0x7f9a5415ca60 0x7f9a540af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:29.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.581+0000 7f9a59826640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a540ad8b0 0x7f9a540cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:29.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.582+0000 7f9a58824640 1 -- 172.21.15.53:0/3497841821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a540ad8b0 msgr2=0x7f9a540cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:29.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.582+0000 7f9a58824640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a540ad8b0 0x7f9a540cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.582+0000 7f9a58824640 1 -- 172.21.15.53:0/3497841821 >> v2:172.21.15.124:3300/0 conn(0x7f9a5415ca60 msgr2=0x7f9a540af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:29.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.582+0000 7f9a58824640 1 --2- 172.21.15.53:0/3497841821 >> v2:172.21.15.124:3300/0 conn(0x7f9a5415ca60 0x7f9a540af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.582+0000 7f9a58824640 1 -- 172.21.15.53:0/3497841821 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5406bcd0 con 0x7f9a540ad400 2023-10-21T16:17:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.583+0000 7f9a58824640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a540cf5b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9a54071f50 tx=0x7f9a4402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621a4456cc91b160 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a4bfff640 1 -- 172.21.15.53:0/3497841821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440317d0 con 0x7f9a540ad400 2023-10-21T16:17:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a4bfff640 1 -- 172.21.15.53:0/3497841821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a44031930 con 0x7f9a540ad400 2023-10-21T16:17:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a4bfff640 1 -- 172.21.15.53:0/3497841821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44031cc0 con 0x7f9a540ad400 2023-10-21T16:17:29.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 -- 172.21.15.53:0/3497841821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 msgr2=0x7f9a540cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a540cf5b0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9a54071f50 tx=0x7f9a4402b870 comp rx=0 tx=0).stop 2023-10-21T16:17:29.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 -- 172.21.15.53:0/3497841821 shutdown_connections 2023-10-21T16:17:29.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a540ad8b0 0x7f9a540cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 --2- 172.21.15.53:0/3497841821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a540cf5b0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 --2- 172.21.15.53:0/3497841821 >> v2:172.21.15.124:3300/0 conn(0x7f9a5415ca60 0x7f9a540af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.584+0000 7f9a5a027640 1 -- 172.21.15.53:0/3497841821 >> 172.21.15.53:0/3497841821 conn(0x7f9a540d0830 msgr2=0x7f9a540d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:29.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.585+0000 7f9a5a027640 1 -- 172.21.15.53:0/3497841821 shutdown_connections 2023-10-21T16:17:29.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.585+0000 7f9a5a027640 1 -- 172.21.15.53:0/3497841821 wait complete. 2023-10-21T16:17:29.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.586+0000 7f9a5a027640 1 Processor -- start 2023-10-21T16:17:29.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 -- start start 2023-10-21T16:17:29.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a540ad8b0 0x7f9a540b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a5415ca60 0x7f9a5406f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54151010 con 0x7f9a540ad400 2023-10-21T16:17:29.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54150a00 con 0x7f9a540ad8b0 2023-10-21T16:17:29.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.587+0000 7f9a5a027640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a541f2080 con 0x7f9a5415ca60 2023-10-21T16:17:29.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:29.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49548/0 (socket says 172.21.15.53:49548) 2023-10-21T16:17:29.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 -- 172.21.15.53:0/1835517602 learned_addr learned my addr 172.21.15.53:0/1835517602 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:29.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 -- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a5415ca60 msgr2=0x7f9a5406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a5415ca60 0x7f9a5406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 -- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a540ad8b0 msgr2=0x7f9a540b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a540ad8b0 0x7f9a540b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.588+0000 7f9a59025640 1 -- 172.21.15.53:0/1835517602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440094d0 con 0x7f9a540ad400 2023-10-21T16:17:29.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a59025640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d800 tx=0x7f9a5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:29.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000b6c0 con 0x7f9a540ad400 2023-10-21T16:17:29.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5000b820 con 0x7f9a540ad400 2023-10-21T16:17:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5000bbd0 con 0x7f9a540ad400 2023-10-21T16:17:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a541f2360 con 0x7f9a540ad400 2023-10-21T16:17:29.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.589+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a541f2920 con 0x7f9a540ad400 2023-10-21T16:17:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.590+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5404e160 con 0x7f9a540ad400 2023-10-21T16:17:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.591+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9a5000bd30 con 0x7f9a540ad400 2023-10-21T16:17:29.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.592+0000 7f9a49ffb640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 0x7f9a3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.592+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9a50089f10 con 0x7f9a540ad400 2023-10-21T16:17:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.593+0000 7f9a58824640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 0x7f9a3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.594+0000 7f9a58824640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 0x7f9a3806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a44036000 tx=0x7f9a44030000 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:29.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.596+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a50054ca0 con 0x7f9a540ad400 2023-10-21T16:17:29.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.828+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a540cdde0 con 0x7f9a540ad400 2023-10-21T16:17:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.829+0000 7f9a49ffb640 1 -- 172.21.15.53:0/1835517602 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f9a500582c0 con 0x7f9a540ad400 2023-10-21T16:17:29.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:29.831 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 msgr2=0x7f9a3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 0x7f9a3806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a44036000 tx=0x7f9a44030000 comp rx=0 tx=0).stop 2023-10-21T16:17:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 msgr2=0x7f9a541504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9a5000d800 tx=0x7f9a5000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 shutdown_connections 2023-10-21T16:17:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f9a3806af90 0x7f9a3806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a5415ca60 0x7f9a5406f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a540ad8b0 0x7f9a540b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 --2- 172.21.15.53:0/1835517602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a540ad400 0x7f9a541504c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.832+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 >> 172.21.15.53:0/1835517602 conn(0x7f9a540d0830 msgr2=0x7f9a540add00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.833+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 shutdown_connections 2023-10-21T16:17:29.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:29.833+0000 7f9a5a027640 1 -- 172.21.15.53:0/1835517602 wait complete. 2023-10-21T16:17:29.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.073+0000 7f2c520ad640 1 Processor -- start 2023-10-21T16:17:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.073+0000 7f2c520ad640 1 -- start start 2023-10-21T16:17:30.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.074+0000 7f2c520ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.074+0000 7f2c520ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2c4c0cf5d0 0x7f2c4c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.074+0000 7f2c520ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c15f480 con 0x7f2c4c15a9f0 2023-10-21T16:17:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c520ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cb760 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c520ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cba40 con 0x7f2c4c15add0 2023-10-21T16:17:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c510ab640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f2c4c15a9f0 0x7f2c4c0ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:33700/0 (socket says 172.21.15.53:33700) 2023-10-21T16:17:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c510ab640 1 -- 172.21.15.53:0/1090379710 learned_addr learned my addr 172.21.15.53:0/1090379710 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1090379710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.075+0000 7f2c518ac640 1 --2- 172.21.15.53:0/1090379710 >> v2:172.21.15.124:3300/0 conn(0x7f2c4c0cf5d0 0x7f2c4c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c43fff640 1 -- 172.21.15.53:0/1090379710 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849958994 0 0) 0x7f2c4c15f480 con 0x7f2c4c15a9f0 2023-10-21T16:17:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c43fff640 1 -- 172.21.15.53:0/1090379710 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c38003520 con 0x7f2c4c15a9f0 2023-10-21T16:17:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c508aa640 1 -- 172.21.15.53:0/1090379710 >> v2:172.21.15.124:3300/0 conn(0x7f2c4c0cf5d0 msgr2=0x7f2c4c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1090379710 >> v2:172.21.15.124:3300/0 conn(0x7f2c4c0cf5d0 0x7f2c4c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c508aa640 1 -- 172.21.15.53:0/1090379710 >> v1:172.21.15.124:6789/0 conn(0x7f2c4c15a9f0 legacy=0x7f2c4c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.076+0000 7f2c508aa640 1 -- 172.21.15.53:0/1090379710 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0cbd20 con 0x7f2c4c15add0 2023-10-21T16:17:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.078+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1090379710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0c4450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097a0 tx=0x7f2c3c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6cb2e3e0f72436 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.078+0000 7f2c43fff640 1 -- 172.21.15.53:0/1090379710 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c02d040 con 0x7f2c4c15add0 2023-10-21T16:17:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.078+0000 7f2c43fff640 1 -- 172.21.15.53:0/1090379710 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c031810 con 0x7f2c4c15add0 2023-10-21T16:17:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.078+0000 7f2c43fff640 1 -- 172.21.15.53:0/1090379710 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c031ba0 con 0x7f2c4c15add0 2023-10-21T16:17:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 -- 172.21.15.53:0/1090379710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 msgr2=0x7f2c4c0c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1090379710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0c4450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0097a0 tx=0x7f2c3c02b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 -- 172.21.15.53:0/1090379710 shutdown_connections 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1090379710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0c4450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1090379710 >> v2:172.21.15.124:3300/0 conn(0x7f2c4c0cf5d0 0x7f2c4c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 -- 172.21.15.53:0/1090379710 >> 172.21.15.53:0/1090379710 conn(0x7f2c4c06a780 msgr2=0x7f2c4c099bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 -- 172.21.15.53:0/1090379710 shutdown_connections 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.079+0000 7f2c520ad640 1 -- 172.21.15.53:0/1090379710 wait complete. 2023-10-21T16:17:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.081+0000 7f2c520ad640 1 Processor -- start 2023-10-21T16:17:30.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.082+0000 7f2c520ad640 1 -- start start 2023-10-21T16:17:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c4c15a9f0 0x7f2c4c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c14a4e0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c14b260 con 0x7f2c4c15a9f0 2023-10-21T16:17:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c520ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c14b540 con 0x7f2c4c15add0 2023-10-21T16:17:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c510ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c510ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49554/0 (socket says 172.21.15.53:49554) 2023-10-21T16:17:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.083+0000 7f2c510ab640 1 -- 172.21.15.53:0/1934900246 learned_addr learned my addr 172.21.15.53:0/1934900246 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:30.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c4c15a9f0 0x7f2c4c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c518ac640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 -- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 msgr2=0x7f2c4c0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 -- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c4c15a9f0 msgr2=0x7f2c4c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c4c15a9f0 0x7f2c4c14a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 -- 172.21.15.53:0/1934900246 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c009400 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c518ac640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c510ab640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800c500 tx=0x7f2c4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.084+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48005ce0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.085+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c48005e40 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.085+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c14b820 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.085+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c48007330 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.085+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c14bde0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.086+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c14bfd0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.087+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2c48007550 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.089+0000 7f2c41ffb640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 0x7f2c3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.089+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c48088fb0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.090+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 0x7f2c3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.091+0000 7f2c508aa640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 0x7f2c3006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c036000 tx=0x7f2c3c03c040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.093+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c48053e90 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.286+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4c04e160 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.288+0000 7f2c41ffb640 1 -- 172.21.15.53:0/1934900246 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f2c480861a0 con 0x7f2c4c0cf5d0 2023-10-21T16:17:30.290 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:30.290 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":90,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:27.281249+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 msgr2=0x7f2c3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 0x7f2c3006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c036000 tx=0x7f2c3c03c040 comp rx=0 tx=0).stop 2023-10-21T16:17:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 msgr2=0x7f2c4c149fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800c500 tx=0x7f2c4800c9c0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 shutdown_connections 2023-10-21T16:17:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f2c3006af90 0x7f2c3006d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c4c15add0 0x7f2c4c0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c4c15a9f0 0x7f2c4c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 --2- 172.21.15.53:0/1934900246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4c0cf5d0 0x7f2c4c149fa0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 >> 172.21.15.53:0/1934900246 conn(0x7f2c4c06a780 msgr2=0x7f2c4c0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 shutdown_connections 2023-10-21T16:17:30.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.290+0000 7f2c520ad640 1 -- 172.21.15.53:0/1934900246 wait complete. 2023-10-21T16:17:30.312 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:17:30.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:17:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.540+0000 7efdaf59e640 1 Processor -- start 2023-10-21T16:17:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.540+0000 7efdaf59e640 1 -- start start 2023-10-21T16:17:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.542+0000 7efdaf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 0x7efdb00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.542+0000 7efdaf59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efdb00b07d0 0x7efdb00c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.542+0000 7efdaf59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb0165470 con 0x7efdb0164f70 2023-10-21T16:17:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.542+0000 7efdaf59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb0165750 con 0x7efdb00b07d0 2023-10-21T16:17:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.542+0000 7efdaf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0165a30 con 0x7efdb0161320 2023-10-21T16:17:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.543+0000 7efdae59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 0x7efdb00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.543+0000 7efdaed9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efdb00b07d0 0x7efdb00c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.543+0000 7efdae59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 0x7efdb00cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:33790/0 (socket says 172.21.15.53:33790) 2023-10-21T16:17:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.543+0000 7efdae59c640 1 -- 172.21.15.53:0/807413796 learned_addr learned my addr 172.21.15.53:0/807413796 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.544+0000 7efdaed9d640 1 -- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 msgr2=0x7efdb00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.544+0000 7efdaed9d640 1 --2- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 0x7efdb00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.544+0000 7efdaed9d640 1 -- 172.21.15.53:0/807413796 >> v1:172.21.15.124:6789/0 conn(0x7efdb0164f70 legacy=0x7efdb00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.544+0000 7efdaed9d640 1 -- 172.21.15.53:0/807413796 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0165d10 con 0x7efdb00b07d0 2023-10-21T16:17:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.545+0000 7efdaed9d640 1 --2- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb00b07d0 0x7efdb00c95b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efda8009d90 tx=0x7efda802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1efed936ded25058 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdad59a640 1 -- 172.21.15.53:0/807413796 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda802cce0 con 0x7efdb00b07d0 2023-10-21T16:17:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdad59a640 1 -- 172.21.15.53:0/807413796 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda802ce40 con 0x7efdb00b07d0 2023-10-21T16:17:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdad59a640 1 -- 172.21.15.53:0/807413796 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8034660 con 0x7efdb00b07d0 2023-10-21T16:17:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 -- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb00b07d0 msgr2=0x7efdb00c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 --2- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb00b07d0 0x7efdb00c95b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efda8009d90 tx=0x7efda802bf30 comp rx=0 tx=0).stop 2023-10-21T16:17:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 -- 172.21.15.53:0/807413796 shutdown_connections 2023-10-21T16:17:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 --2- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb00b07d0 0x7efdb00c95b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 --2- 172.21.15.53:0/807413796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb0161320 0x7efdb00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.546+0000 7efdaf59e640 1 -- 172.21.15.53:0/807413796 >> 172.21.15.53:0/807413796 conn(0x7efdb00cb0c0 msgr2=0x7efdb00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.547+0000 7efdaf59e640 1 -- 172.21.15.53:0/807413796 shutdown_connections 2023-10-21T16:17:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.547+0000 7efdaf59e640 1 -- 172.21.15.53:0/807413796 wait complete. 2023-10-21T16:17:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.548+0000 7efdaf59e640 1 Processor -- start 2023-10-21T16:17:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 -- start start 2023-10-21T16:17:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb00b07d0 0x7efdb00c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 0x7efdb00b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb015e6d0 con 0x7efdb0164f70 2023-10-21T16:17:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00c2740 con 0x7efdb0161320 2023-10-21T16:17:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.549+0000 7efdaf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00b2660 con 0x7efdb00b07d0 2023-10-21T16:17:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49556/0 (socket says 172.21.15.53:49556) 2023-10-21T16:17:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 -- 172.21.15.53:0/4209248740 learned_addr learned my addr 172.21.15.53:0/4209248740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdae59c640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb00b07d0 0x7efdb00c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdadd9b640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 0x7efdb00b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 -- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb00b07d0 msgr2=0x7efdb00c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb00b07d0 0x7efdb00c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 -- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 msgr2=0x7efdb00b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 0x7efdb00b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdaed9d640 1 -- 172.21.15.53:0/4209248740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda8009a70 con 0x7efdb0164f70 2023-10-21T16:17:30.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.550+0000 7efdadd9b640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 0x7efdb00b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efdaed9d640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efda8009d90 tx=0x7efda8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda803d600 con 0x7efdb0164f70 2023-10-21T16:17:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda803d780 con 0x7efdb0164f70 2023-10-21T16:17:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb00b2940 con 0x7efdb0164f70 2023-10-21T16:17:30.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda803db30 con 0x7efdb0164f70 2023-10-21T16:17:30.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.551+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb00b2f00 con 0x7efdb0164f70 2023-10-21T16:17:30.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.553+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb004e160 con 0x7efdb0164f70 2023-10-21T16:17:30.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.553+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efda803b040 con 0x7efdb0164f70 2023-10-21T16:17:30.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.554+0000 7efd9effd640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 0x7efd9006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:30.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.554+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efda80abdd0 con 0x7efdb0164f70 2023-10-21T16:17:30.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.555+0000 7efdae59c640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 0x7efd9006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:30.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.556+0000 7efdae59c640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 0x7efd9006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efda0009b60 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:30.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.559+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efda8076c00 con 0x7efdb0164f70 2023-10-21T16:17:30.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.751+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 --> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdb00c9020 con 0x7efd9006afc0 2023-10-21T16:17:30.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.779+0000 7efd9effd640 1 -- 172.21.15.53:0/4209248740 <== mgr.5734 v2:172.21.15.124:6832/1636058776 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7efda0011a60 con 0x7efd9006afc0 2023-10-21T16:17:30.787 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:30.790 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:17:30.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 msgr2=0x7efd9006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 0x7efd9006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efda0009b60 tx=0x7efda0008040 comp rx=0 tx=0).stop 2023-10-21T16:17:30.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 msgr2=0x7efdb015e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:30.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efda8009d90 tx=0x7efda8004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:30.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 shutdown_connections 2023-10-21T16:17:30.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7efd9006afc0 0x7efd9006d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efdb0164f70 0x7efdb015e190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efdb0161320 0x7efdb00b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 --2- 172.21.15.53:0/4209248740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efdb00b07d0 0x7efdb00c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 >> 172.21.15.53:0/4209248740 conn(0x7efdb00cb0c0 msgr2=0x7efdb015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 shutdown_connections 2023-10-21T16:17:30.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:30.785+0000 7efdaf59e640 1 -- 172.21.15.53:0/4209248740 wait complete. 2023-10-21T16:17:30.807 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-21T16:17:29.528109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324444,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754650276,"statfs":{"total":773094113280,"available":772761882624,"internally_reserved":0,"allocated":10002432,"data_stored":5602116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":322096697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.295227"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367084+0000","last_change":"2023-10-21T16:17:26.291685+0000","last_active":"2023-10-21T16:17:27.367084+0000","last_peered":"2023-10-21T16:17:27.367084+0000","last_clean":"2023-10-21T16:17:27.367084+0000","last_became_active":"2023-10-21T16:17:26.290210+0000","last_became_peered":"2023-10-21T16:17:26.290210+0000","last_unstale":"2023-10-21T16:17:27.367084+0000","last_undegraded":"2023-10-21T16:17:27.367084+0000","last_fullsized":"2023-10-21T16:17:27.367084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:58:05.811035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367423+0000","last_change":"2023-10-21T16:17:26.290788+0000","last_active":"2023-10-21T16:17:27.367423+0000","last_peered":"2023-10-21T16:17:27.367423+0000","last_clean":"2023-10-21T16:17:27.367423+0000","last_became_active":"2023-10-21T16:17:26.290479+0000","last_became_peered":"2023-10-21T16:17:26.290479+0000","last_unstale":"2023-10-21T16:17:27.367423+0000","last_undegraded":"2023-10-21T16:17:27.367423+0000","last_fullsized":"2023-10-21T16:17:27.367423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:54.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365988+0000","last_change":"2023-10-21T16:17:26.289047+0000","last_active":"2023-10-21T16:17:27.365988+0000","last_peered":"2023-10-21T16:17:27.365988+0000","last_clean":"2023-10-21T16:17:27.365988+0000","last_became_active":"2023-10-21T16:17:26.288749+0000","last_became_peered":"2023-10-21T16:17:26.288749+0000","last_unstale":"2023-10-21T16:17:27.365988+0000","last_undegraded":"2023-10-21T16:17:27.365988+0000","last_fullsized":"2023-10-21T16:17:27.365988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:34.472760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301974+0000","last_change":"2023-10-21T16:17:26.282142+0000","last_active":"2023-10-21T16:17:27.301974+0000","last_peered":"2023-10-21T16:17:27.301974+0000","last_clean":"2023-10-21T16:17:27.301974+0000","last_became_active":"2023-10-21T16:17:26.281673+0000","last_became_peered":"2023-10-21T16:17:26.281673+0000","last_unstale":"2023-10-21T16:17:27.301974+0000","last_undegraded":"2023-10-21T16:17:27.301974+0000","last_fullsized":"2023-10-21T16:17:27.301974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:58.531464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368525+0000","last_change":"2023-10-21T16:17:26.288820+0000","last_active":"2023-10-21T16:17:27.368525+0000","last_peered":"2023-10-21T16:17:27.368525+0000","last_clean":"2023-10-21T16:17:27.368525+0000","last_became_active":"2023-10-21T16:17:26.288533+0000","last_became_peered":"2023-10-21T16:17:26.288533+0000","last_unstale":"2023-10-21T16:17:27.368525+0000","last_undegraded":"2023-10-21T16:17:27.368525+0000","last_fullsized":"2023-10-21T16:17:27.368525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:21:20.340682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365278+0000","last_change":"2023-10-21T16:17:26.287137+0000","last_active":"2023-10-21T16:17:27.365278+0000","last_peered":"2023-10-21T16:17:27.365278+0000","last_clean":"2023-10-21T16:17:27.365278+0000","last_became_active":"2023-10-21T16:17:26.286792+0000","last_became_peered":"2023-10-21T16:17:26.286792+0000","last_unstale":"2023-10-21T16:17:27.365278+0000","last_undegraded":"2023-10-21T16:17:27.365278+0000","last_fullsized":"2023-10-21T16:17:27.365278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:29.105875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299165+0000","last_change":"2023-10-21T16:17:26.288485+0000","last_active":"2023-10-21T16:17:27.299165+0000","last_peered":"2023-10-21T16:17:27.299165+0000","last_clean":"2023-10-21T16:17:27.299165+0000","last_became_active":"2023-10-21T16:17:26.285435+0000","last_became_peered":"2023-10-21T16:17:26.285435+0000","last_unstale":"2023-10-21T16:17:27.299165+0000","last_undegraded":"2023-10-21T16:17:27.299165+0000","last_fullsized":"2023-10-21T16:17:27.299165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:40.043572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367477+0000","last_change":"2023-10-21T16:17:26.292237+0000","last_active":"2023-10-21T16:17:27.367477+0000","last_peered":"2023-10-21T16:17:27.367477+0000","last_clean":"2023-10-21T16:17:27.367477+0000","last_became_active":"2023-10-21T16:17:26.291846+0000","last_became_peered":"2023-10-21T16:17:26.291846+0000","last_unstale":"2023-10-21T16:17:27.367477+0000","last_undegraded":"2023-10-21T16:17:27.367477+0000","last_fullsized":"2023-10-21T16:17:27.367477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:53.045552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367435+0000","last_change":"2023-10-21T16:17:26.287211+0000","last_active":"2023-10-21T16:17:27.367435+0000","last_peered":"2023-10-21T16:17:27.367435+0000","last_clean":"2023-10-21T16:17:27.367435+0000","last_became_active":"2023-10-21T16:17:26.286866+0000","last_became_peered":"2023-10-21T16:17:26.286866+0000","last_unstale":"2023-10-21T16:17:27.367435+0000","last_undegraded":"2023-10-21T16:17:27.367435+0000","last_fullsized":"2023-10-21T16:17:27.367435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:48.229280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366958+0000","last_change":"2023-10-21T16:17:26.293259+0000","last_active":"2023-10-21T16:17:27.366958+0000","last_peered":"2023-10-21T16:17:27.366958+0000","last_clean":"2023-10-21T16:17:27.366958+0000","last_became_active":"2023-10-21T16:17:26.292984+0000","last_became_peered":"2023-10-21T16:17:26.292984+0000","last_unstale":"2023-10-21T16:17:27.366958+0000","last_undegraded":"2023-10-21T16:17:27.366958+0000","last_fullsized":"2023-10-21T16:17:27.366958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:26.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302373+0000","last_change":"2023-10-21T16:17:26.294540+0000","last_active":"2023-10-21T16:17:27.302373+0000","last_peered":"2023-10-21T16:17:27.302373+0000","last_clean":"2023-10-21T16:17:27.302373+0000","last_became_active":"2023-10-21T16:17:26.294272+0000","last_became_peered":"2023-10-21T16:17:26.294272+0000","last_unstale":"2023-10-21T16:17:27.302373+0000","last_undegraded":"2023-10-21T16:17:27.302373+0000","last_fullsized":"2023-10-21T16:17:27.302373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:38.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369282+0000","last_change":"2023-10-21T16:17:26.286271+0000","last_active":"2023-10-21T16:17:27.369282+0000","last_peered":"2023-10-21T16:17:27.369282+0000","last_clean":"2023-10-21T16:17:27.369282+0000","last_became_active":"2023-10-21T16:17:26.285943+0000","last_became_peered":"2023-10-21T16:17:26.285943+0000","last_unstale":"2023-10-21T16:17:27.369282+0000","last_undegraded":"2023-10-21T16:17:27.369282+0000","last_fullsized":"2023-10-21T16:17:27.369282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:28.008873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369272+0000","last_change":"2023-10-21T16:17:26.285026+0000","last_active":"2023-10-21T16:17:27.369272+0000","last_peered":"2023-10-21T16:17:27.369272+0000","last_clean":"2023-10-21T16:17:27.369272+0000","last_became_active":"2023-10-21T16:17:26.284749+0000","last_became_peered":"2023-10-21T16:17:26.284749+0000","last_unstale":"2023-10-21T16:17:27.369272+0000","last_undegraded":"2023-10-21T16:17:27.369272+0000","last_fullsized":"2023-10-21T16:17:27.369272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:38:53.817692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366531+0000","last_change":"2023-10-21T16:17:26.288357+0000","last_active":"2023-10-21T16:17:27.366531+0000","last_peered":"2023-10-21T16:17:27.366531+0000","last_clean":"2023-10-21T16:17:27.366531+0000","last_became_active":"2023-10-21T16:17:26.286831+0000","last_became_peered":"2023-10-21T16:17:26.286831+0000","last_unstale":"2023-10-21T16:17:27.366531+0000","last_undegraded":"2023-10-21T16:17:27.366531+0000","last_fullsized":"2023-10-21T16:17:27.366531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365831+0000","last_change":"2023-10-21T16:17:26.294624+0000","last_active":"2023-10-21T16:17:27.365831+0000","last_peered":"2023-10-21T16:17:27.365831+0000","last_clean":"2023-10-21T16:17:27.365831+0000","last_became_active":"2023-10-21T16:17:26.294359+0000","last_became_peered":"2023-10-21T16:17:26.294359+0000","last_unstale":"2023-10-21T16:17:27.365831+0000","last_undegraded":"2023-10-21T16:17:27.365831+0000","last_fullsized":"2023-10-21T16:17:27.365831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:09:01.212669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299079+0000","last_change":"2023-10-21T16:17:26.287669+0000","last_active":"2023-10-21T16:17:27.299079+0000","last_peered":"2023-10-21T16:17:27.299079+0000","last_clean":"2023-10-21T16:17:27.299079+0000","last_became_active":"2023-10-21T16:17:26.287327+0000","last_became_peered":"2023-10-21T16:17:26.287327+0000","last_unstale":"2023-10-21T16:17:27.299079+0000","last_undegraded":"2023-10-21T16:17:27.299079+0000","last_fullsized":"2023-10-21T16:17:27.299079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:06.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367428+0000","last_change":"2023-10-21T16:17:26.286640+0000","last_active":"2023-10-21T16:17:27.367428+0000","last_peered":"2023-10-21T16:17:27.367428+0000","last_clean":"2023-10-21T16:17:27.367428+0000","last_became_active":"2023-10-21T16:17:26.286318+0000","last_became_peered":"2023-10-21T16:17:26.286318+0000","last_unstale":"2023-10-21T16:17:27.367428+0000","last_undegraded":"2023-10-21T16:17:27.367428+0000","last_fullsized":"2023-10-21T16:17:27.367428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:51.768527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302760+0000","last_change":"2023-10-21T16:17:26.284296+0000","last_active":"2023-10-21T16:17:27.302760+0000","last_peered":"2023-10-21T16:17:27.302760+0000","last_clean":"2023-10-21T16:17:27.302760+0000","last_became_active":"2023-10-21T16:17:26.284022+0000","last_became_peered":"2023-10-21T16:17:26.284022+0000","last_unstale":"2023-10-21T16:17:27.302760+0000","last_undegraded":"2023-10-21T16:17:27.302760+0000","last_fullsized":"2023-10-21T16:17:27.302760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:00.484233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368830+0000","last_change":"2023-10-21T16:17:26.289182+0000","last_active":"2023-10-21T16:17:27.368830+0000","last_peered":"2023-10-21T16:17:27.368830+0000","last_clean":"2023-10-21T16:17:27.368830+0000","last_became_active":"2023-10-21T16:17:26.288813+0000","last_became_peered":"2023-10-21T16:17:26.288813+0000","last_unstale":"2023-10-21T16:17:27.368830+0000","last_undegraded":"2023-10-21T16:17:27.368830+0000","last_fullsized":"2023-10-21T16:17:27.368830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:07.851912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297606+0000","last_change":"2023-10-21T16:17:26.290218+0000","last_active":"2023-10-21T16:17:27.297606+0000","last_peered":"2023-10-21T16:17:27.297606+0000","last_clean":"2023-10-21T16:17:27.297606+0000","last_became_active":"2023-10-21T16:17:26.289914+0000","last_became_peered":"2023-10-21T16:17:26.289914+0000","last_unstale":"2023-10-21T16:17:27.297606+0000","last_undegraded":"2023-10-21T16:17:27.297606+0000","last_fullsized":"2023-10-21T16:17:27.297606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:02.997864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369642+0000","last_change":"2023-10-21T16:17:26.296790+0000","last_active":"2023-10-21T16:17:27.369642+0000","last_peered":"2023-10-21T16:17:27.369642+0000","last_clean":"2023-10-21T16:17:27.369642+0000","last_became_active":"2023-10-21T16:17:26.294405+0000","last_became_peered":"2023-10-21T16:17:26.294405+0000","last_unstale":"2023-10-21T16:17:27.369642+0000","last_undegraded":"2023-10-21T16:17:27.369642+0000","last_fullsized":"2023-10-21T16:17:27.369642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:37.543660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297094+0000","last_change":"2023-10-21T16:17:26.284929+0000","last_active":"2023-10-21T16:17:27.297094+0000","last_peered":"2023-10-21T16:17:27.297094+0000","last_clean":"2023-10-21T16:17:27.297094+0000","last_became_active":"2023-10-21T16:17:26.284598+0000","last_became_peered":"2023-10-21T16:17:26.284598+0000","last_unstale":"2023-10-21T16:17:27.297094+0000","last_undegraded":"2023-10-21T16:17:27.297094+0000","last_fullsized":"2023-10-21T16:17:27.297094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:15.797402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297043+0000","last_change":"2023-10-21T16:17:26.284000+0000","last_active":"2023-10-21T16:17:27.297043+0000","last_peered":"2023-10-21T16:17:27.297043+0000","last_clean":"2023-10-21T16:17:27.297043+0000","last_became_active":"2023-10-21T16:17:26.283650+0000","last_became_peered":"2023-10-21T16:17:26.283650+0000","last_unstale":"2023-10-21T16:17:27.297043+0000","last_undegraded":"2023-10-21T16:17:27.297043+0000","last_fullsized":"2023-10-21T16:17:27.297043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:16.143933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367090+0000","last_change":"2023-10-21T16:17:26.301552+0000","last_active":"2023-10-21T16:17:27.367090+0000","last_peered":"2023-10-21T16:17:27.367090+0000","last_clean":"2023-10-21T16:17:27.367090+0000","last_became_active":"2023-10-21T16:17:26.299042+0000","last_became_peered":"2023-10-21T16:17:26.299042+0000","last_unstale":"2023-10-21T16:17:27.367090+0000","last_undegraded":"2023-10-21T16:17:27.367090+0000","last_fullsized":"2023-10-21T16:17:27.367090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:23.269157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369861+0000","last_change":"2023-10-21T16:17:26.290031+0000","last_active":"2023-10-21T16:17:27.369861+0000","last_peered":"2023-10-21T16:17:27.369861+0000","last_clean":"2023-10-21T16:17:27.369861+0000","last_became_active":"2023-10-21T16:17:26.289729+0000","last_became_peered":"2023-10-21T16:17:26.289729+0000","last_unstale":"2023-10-21T16:17:27.369861+0000","last_undegraded":"2023-10-21T16:17:27.369861+0000","last_fullsized":"2023-10-21T16:17:27.369861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:27.688997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369406+0000","last_change":"2023-10-21T16:17:26.287319+0000","last_active":"2023-10-21T16:17:27.369406+0000","last_peered":"2023-10-21T16:17:27.369406+0000","last_clean":"2023-10-21T16:17:27.369406+0000","last_became_active":"2023-10-21T16:17:26.286998+0000","last_became_peered":"2023-10-21T16:17:26.286998+0000","last_unstale":"2023-10-21T16:17:27.369406+0000","last_undegraded":"2023-10-21T16:17:27.369406+0000","last_fullsized":"2023-10-21T16:17:27.369406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:57.862665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302726+0000","last_change":"2023-10-21T16:17:26.275379+0000","last_active":"2023-10-21T16:17:27.302726+0000","last_peered":"2023-10-21T16:17:27.302726+0000","last_clean":"2023-10-21T16:17:27.302726+0000","last_became_active":"2023-10-21T16:17:26.275027+0000","last_became_peered":"2023-10-21T16:17:26.275027+0000","last_unstale":"2023-10-21T16:17:27.302726+0000","last_undegraded":"2023-10-21T16:17:27.302726+0000","last_fullsized":"2023-10-21T16:17:27.302726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:10.163662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367026+0000","last_change":"2023-10-21T16:17:26.291226+0000","last_active":"2023-10-21T16:17:27.367026+0000","last_peered":"2023-10-21T16:17:27.367026+0000","last_clean":"2023-10-21T16:17:27.367026+0000","last_became_active":"2023-10-21T16:17:26.290932+0000","last_became_peered":"2023-10-21T16:17:26.290932+0000","last_unstale":"2023-10-21T16:17:27.367026+0000","last_undegraded":"2023-10-21T16:17:27.367026+0000","last_fullsized":"2023-10-21T16:17:27.367026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:15:20.884000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365898+0000","last_change":"2023-10-21T16:17:26.291057+0000","last_active":"2023-10-21T16:17:27.365898+0000","last_peered":"2023-10-21T16:17:27.365898+0000","last_clean":"2023-10-21T16:17:27.365898+0000","last_became_active":"2023-10-21T16:17:26.290005+0000","last_became_peered":"2023-10-21T16:17:26.290005+0000","last_unstale":"2023-10-21T16:17:27.365898+0000","last_undegraded":"2023-10-21T16:17:27.365898+0000","last_fullsized":"2023-10-21T16:17:27.365898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:19:42.094980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367125+0000","last_change":"2023-10-21T16:17:26.287655+0000","last_active":"2023-10-21T16:17:27.367125+0000","last_peered":"2023-10-21T16:17:27.367125+0000","last_clean":"2023-10-21T16:17:27.367125+0000","last_became_active":"2023-10-21T16:17:26.286481+0000","last_became_peered":"2023-10-21T16:17:26.286481+0000","last_unstale":"2023-10-21T16:17:27.367125+0000","last_undegraded":"2023-10-21T16:17:27.367125+0000","last_fullsized":"2023-10-21T16:17:27.367125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:50:03.302733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301250+0000","last_change":"2023-10-21T16:17:26.274895+0000","last_active":"2023-10-21T16:17:27.301250+0000","last_peered":"2023-10-21T16:17:27.301250+0000","last_clean":"2023-10-21T16:17:27.301250+0000","last_became_active":"2023-10-21T16:17:26.274437+0000","last_became_peered":"2023-10-21T16:17:26.274437+0000","last_unstale":"2023-10-21T16:17:27.301250+0000","last_undegraded":"2023-10-21T16:17:27.301250+0000","last_fullsized":"2023-10-21T16:17:27.301250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:54.820486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365352+0000","last_change":"2023-10-21T16:17:26.295127+0000","last_active":"2023-10-21T16:17:27.365352+0000","last_peered":"2023-10-21T16:17:27.365352+0000","last_clean":"2023-10-21T16:17:27.365352+0000","last_became_active":"2023-10-21T16:17:26.289019+0000","last_became_peered":"2023-10-21T16:17:26.289019+0000","last_unstale":"2023-10-21T16:17:27.365352+0000","last_undegraded":"2023-10-21T16:17:27.365352+0000","last_fullsized":"2023-10-21T16:17:27.365352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:03.828384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369142+0000","last_change":"2023-10-21T16:17:26.297339+0000","last_active":"2023-10-21T16:17:27.369142+0000","last_peered":"2023-10-21T16:17:27.369142+0000","last_clean":"2023-10-21T16:17:27.369142+0000","last_became_active":"2023-10-21T16:17:26.297016+0000","last_became_peered":"2023-10-21T16:17:26.297016+0000","last_unstale":"2023-10-21T16:17:27.369142+0000","last_undegraded":"2023-10-21T16:17:27.369142+0000","last_fullsized":"2023-10-21T16:17:27.369142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:07.189884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"89'4","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367489+0000","last_change":"2023-10-21T16:17:25.286504+0000","last_active":"2023-10-21T16:17:27.367489+0000","last_peered":"2023-10-21T16:17:27.367489+0000","last_clean":"2023-10-21T16:17:27.367489+0000","last_became_active":"2023-10-21T16:17:25.286181+0000","last_became_peered":"2023-10-21T16:17:25.286181+0000","last_unstale":"2023-10-21T16:17:27.367489+0000","last_undegraded":"2023-10-21T16:17:27.367489+0000","last_fullsized":"2023-10-21T16:17:27.367489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:11.332814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302750+0000","last_change":"2023-10-21T16:17:26.296899+0000","last_active":"2023-10-21T16:17:27.302750+0000","last_peered":"2023-10-21T16:17:27.302750+0000","last_clean":"2023-10-21T16:17:27.302750+0000","last_became_active":"2023-10-21T16:17:26.296644+0000","last_became_peered":"2023-10-21T16:17:26.296644+0000","last_unstale":"2023-10-21T16:17:27.302750+0000","last_undegraded":"2023-10-21T16:17:27.302750+0000","last_fullsized":"2023-10-21T16:17:27.302750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:38:41.769284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367052+0000","last_change":"2023-10-21T16:17:25.278363+0000","last_active":"2023-10-21T16:17:27.367052+0000","last_peered":"2023-10-21T16:17:27.367052+0000","last_clean":"2023-10-21T16:17:27.367052+0000","last_became_active":"2023-10-21T16:17:25.278046+0000","last_became_peered":"2023-10-21T16:17:25.278046+0000","last_unstale":"2023-10-21T16:17:27.367052+0000","last_undegraded":"2023-10-21T16:17:27.367052+0000","last_fullsized":"2023-10-21T16:17:27.367052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:23.896347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367115+0000","last_change":"2023-10-21T16:17:26.297169+0000","last_active":"2023-10-21T16:17:27.367115+0000","last_peered":"2023-10-21T16:17:27.367115+0000","last_clean":"2023-10-21T16:17:27.367115+0000","last_became_active":"2023-10-21T16:17:26.296022+0000","last_became_peered":"2023-10-21T16:17:26.296022+0000","last_unstale":"2023-10-21T16:17:27.367115+0000","last_undegraded":"2023-10-21T16:17:27.367115+0000","last_fullsized":"2023-10-21T16:17:27.367115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:11.774948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365803+0000","last_change":"2023-10-21T16:17:25.282374+0000","last_active":"2023-10-21T16:17:27.365803+0000","last_peered":"2023-10-21T16:17:27.365803+0000","last_clean":"2023-10-21T16:17:27.365803+0000","last_became_active":"2023-10-21T16:17:25.281843+0000","last_became_peered":"2023-10-21T16:17:25.281843+0000","last_unstale":"2023-10-21T16:17:27.365803+0000","last_undegraded":"2023-10-21T16:17:27.365803+0000","last_fullsized":"2023-10-21T16:17:27.365803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:05.347647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366935+0000","last_change":"2023-10-21T16:17:26.295334+0000","last_active":"2023-10-21T16:17:27.366935+0000","last_peered":"2023-10-21T16:17:27.366935+0000","last_clean":"2023-10-21T16:17:27.366935+0000","last_became_active":"2023-10-21T16:17:26.295059+0000","last_became_peered":"2023-10-21T16:17:26.295059+0000","last_unstale":"2023-10-21T16:17:27.366935+0000","last_undegraded":"2023-10-21T16:17:27.366935+0000","last_fullsized":"2023-10-21T16:17:27.366935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:00.941402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365312+0000","last_change":"2023-10-21T16:17:25.274273+0000","last_active":"2023-10-21T16:17:27.365312+0000","last_peered":"2023-10-21T16:17:27.365312+0000","last_clean":"2023-10-21T16:17:27.365312+0000","last_became_active":"2023-10-21T16:17:25.273950+0000","last_became_peered":"2023-10-21T16:17:25.273950+0000","last_unstale":"2023-10-21T16:17:27.365312+0000","last_undegraded":"2023-10-21T16:17:27.365312+0000","last_fullsized":"2023-10-21T16:17:27.365312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:19:40.938948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299289+0000","last_change":"2023-10-21T16:17:26.296527+0000","last_active":"2023-10-21T16:17:27.299289+0000","last_peered":"2023-10-21T16:17:27.299289+0000","last_clean":"2023-10-21T16:17:27.299289+0000","last_became_active":"2023-10-21T16:17:26.294932+0000","last_became_peered":"2023-10-21T16:17:26.294932+0000","last_unstale":"2023-10-21T16:17:27.299289+0000","last_undegraded":"2023-10-21T16:17:27.299289+0000","last_fullsized":"2023-10-21T16:17:27.299289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:32.534067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297540+0000","last_change":"2023-10-21T16:17:25.269792+0000","last_active":"2023-10-21T16:17:27.297540+0000","last_peered":"2023-10-21T16:17:27.297540+0000","last_clean":"2023-10-21T16:17:27.297540+0000","last_became_active":"2023-10-21T16:17:25.269492+0000","last_became_peered":"2023-10-21T16:17:25.269492+0000","last_unstale":"2023-10-21T16:17:27.297540+0000","last_undegraded":"2023-10-21T16:17:27.297540+0000","last_fullsized":"2023-10-21T16:17:27.297540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:32.180154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365793+0000","last_change":"2023-10-21T16:17:26.292080+0000","last_active":"2023-10-21T16:17:27.365793+0000","last_peered":"2023-10-21T16:17:27.365793+0000","last_clean":"2023-10-21T16:17:27.365793+0000","last_became_active":"2023-10-21T16:17:26.290984+0000","last_became_peered":"2023-10-21T16:17:26.290984+0000","last_unstale":"2023-10-21T16:17:27.365793+0000","last_undegraded":"2023-10-21T16:17:27.365793+0000","last_fullsized":"2023-10-21T16:17:27.365793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:29.377919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297218+0000","last_change":"2023-10-21T16:17:25.267715+0000","last_active":"2023-10-21T16:17:27.297218+0000","last_peered":"2023-10-21T16:17:27.297218+0000","last_clean":"2023-10-21T16:17:27.297218+0000","last_became_active":"2023-10-21T16:17:25.267382+0000","last_became_peered":"2023-10-21T16:17:25.267382+0000","last_unstale":"2023-10-21T16:17:27.297218+0000","last_undegraded":"2023-10-21T16:17:27.297218+0000","last_fullsized":"2023-10-21T16:17:27.297218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:43:33.147295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366244+0000","last_change":"2023-10-21T16:17:26.296805+0000","last_active":"2023-10-21T16:17:27.366244+0000","last_peered":"2023-10-21T16:17:27.366244+0000","last_clean":"2023-10-21T16:17:27.366244+0000","last_became_active":"2023-10-21T16:17:26.295768+0000","last_became_peered":"2023-10-21T16:17:26.295768+0000","last_unstale":"2023-10-21T16:17:27.366244+0000","last_undegraded":"2023-10-21T16:17:27.366244+0000","last_fullsized":"2023-10-21T16:17:27.366244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:10:51.308191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297252+0000","last_change":"2023-10-21T16:17:25.285931+0000","last_active":"2023-10-21T16:17:27.297252+0000","last_peered":"2023-10-21T16:17:27.297252+0000","last_clean":"2023-10-21T16:17:27.297252+0000","last_became_active":"2023-10-21T16:17:25.284775+0000","last_became_peered":"2023-10-21T16:17:25.284775+0000","last_unstale":"2023-10-21T16:17:27.297252+0000","last_undegraded":"2023-10-21T16:17:27.297252+0000","last_fullsized":"2023-10-21T16:17:27.297252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:17.712588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297307+0000","last_change":"2023-10-21T16:17:26.275150+0000","last_active":"2023-10-21T16:17:27.297307+0000","last_peered":"2023-10-21T16:17:27.297307+0000","last_clean":"2023-10-21T16:17:27.297307+0000","last_became_active":"2023-10-21T16:17:26.273956+0000","last_became_peered":"2023-10-21T16:17:26.273956+0000","last_unstale":"2023-10-21T16:17:27.297307+0000","last_undegraded":"2023-10-21T16:17:27.297307+0000","last_fullsized":"2023-10-21T16:17:27.297307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:05:56.193212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366469+0000","last_change":"2023-10-21T16:17:25.284798+0000","last_active":"2023-10-21T16:17:27.366469+0000","last_peered":"2023-10-21T16:17:27.366469+0000","last_clean":"2023-10-21T16:17:27.366469+0000","last_became_active":"2023-10-21T16:17:25.284489+0000","last_became_peered":"2023-10-21T16:17:25.284489+0000","last_unstale":"2023-10-21T16:17:27.366469+0000","last_undegraded":"2023-10-21T16:17:27.366469+0000","last_fullsized":"2023-10-21T16:17:27.366469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:15.337252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366478+0000","last_change":"2023-10-21T16:17:26.286690+0000","last_active":"2023-10-21T16:17:27.366478+0000","last_peered":"2023-10-21T16:17:27.366478+0000","last_clean":"2023-10-21T16:17:27.366478+0000","last_became_active":"2023-10-21T16:17:26.286372+0000","last_became_peered":"2023-10-21T16:17:26.286372+0000","last_unstale":"2023-10-21T16:17:27.366478+0000","last_undegraded":"2023-10-21T16:17:27.366478+0000","last_fullsized":"2023-10-21T16:17:27.366478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:50:49.380985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367776+0000","last_change":"2023-10-21T16:17:25.288345+0000","last_active":"2023-10-21T16:17:27.367776+0000","last_peered":"2023-10-21T16:17:27.367776+0000","last_clean":"2023-10-21T16:17:27.367776+0000","last_became_active":"2023-10-21T16:17:25.287827+0000","last_became_peered":"2023-10-21T16:17:25.287827+0000","last_unstale":"2023-10-21T16:17:27.367776+0000","last_undegraded":"2023-10-21T16:17:27.367776+0000","last_fullsized":"2023-10-21T16:17:27.367776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:48.696560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297583+0000","last_change":"2023-10-21T16:17:26.285078+0000","last_active":"2023-10-21T16:17:27.297583+0000","last_peered":"2023-10-21T16:17:27.297583+0000","last_clean":"2023-10-21T16:17:27.297583+0000","last_became_active":"2023-10-21T16:17:26.284773+0000","last_became_peered":"2023-10-21T16:17:26.284773+0000","last_unstale":"2023-10-21T16:17:27.297583+0000","last_undegraded":"2023-10-21T16:17:27.297583+0000","last_fullsized":"2023-10-21T16:17:27.297583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:05.933204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302901+0000","last_change":"2023-10-21T16:17:25.277413+0000","last_active":"2023-10-21T16:17:27.302901+0000","last_peered":"2023-10-21T16:17:27.302901+0000","last_clean":"2023-10-21T16:17:27.302901+0000","last_became_active":"2023-10-21T16:17:25.276613+0000","last_became_peered":"2023-10-21T16:17:25.276613+0000","last_unstale":"2023-10-21T16:17:27.302901+0000","last_undegraded":"2023-10-21T16:17:27.302901+0000","last_fullsized":"2023-10-21T16:17:27.302901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:40:21.018413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368789+0000","last_change":"2023-10-21T16:17:26.293676+0000","last_active":"2023-10-21T16:17:27.368789+0000","last_peered":"2023-10-21T16:17:27.368789+0000","last_clean":"2023-10-21T16:17:27.368789+0000","last_became_active":"2023-10-21T16:17:26.292144+0000","last_became_peered":"2023-10-21T16:17:26.292144+0000","last_unstale":"2023-10-21T16:17:27.368789+0000","last_undegraded":"2023-10-21T16:17:27.368789+0000","last_fullsized":"2023-10-21T16:17:27.368789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:54:23.901665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"89'3","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366557+0000","last_change":"2023-10-21T16:17:25.288121+0000","last_active":"2023-10-21T16:17:27.366557+0000","last_peered":"2023-10-21T16:17:27.366557+0000","last_clean":"2023-10-21T16:17:27.366557+0000","last_became_active":"2023-10-21T16:17:25.287822+0000","last_became_peered":"2023-10-21T16:17:25.287822+0000","last_unstale":"2023-10-21T16:17:27.366557+0000","last_undegraded":"2023-10-21T16:17:27.366557+0000","last_fullsized":"2023-10-21T16:17:27.366557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:11.234181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.303317+0000","last_change":"2023-10-21T16:17:26.273598+0000","last_active":"2023-10-21T16:17:27.303317+0000","last_peered":"2023-10-21T16:17:27.303317+0000","last_clean":"2023-10-21T16:17:27.303317+0000","last_became_active":"2023-10-21T16:17:26.272982+0000","last_became_peered":"2023-10-21T16:17:26.272982+0000","last_unstale":"2023-10-21T16:17:27.303317+0000","last_undegraded":"2023-10-21T16:17:27.303317+0000","last_fullsized":"2023-10-21T16:17:27.303317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:12.347820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"89'6","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366577+0000","last_change":"2023-10-21T16:17:25.280566+0000","last_active":"2023-10-21T16:17:27.366577+0000","last_peered":"2023-10-21T16:17:27.366577+0000","last_clean":"2023-10-21T16:17:27.366577+0000","last_became_active":"2023-10-21T16:17:25.280253+0000","last_became_peered":"2023-10-21T16:17:25.280253+0000","last_unstale":"2023-10-21T16:17:27.366577+0000","last_undegraded":"2023-10-21T16:17:27.366577+0000","last_fullsized":"2023-10-21T16:17:27.366577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:59.469227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301192+0000","last_change":"2023-10-21T16:17:26.292447+0000","last_active":"2023-10-21T16:17:27.301192+0000","last_peered":"2023-10-21T16:17:27.301192+0000","last_clean":"2023-10-21T16:17:27.301192+0000","last_became_active":"2023-10-21T16:17:26.292164+0000","last_became_peered":"2023-10-21T16:17:26.292164+0000","last_unstale":"2023-10-21T16:17:27.301192+0000","last_undegraded":"2023-10-21T16:17:27.301192+0000","last_fullsized":"2023-10-21T16:17:27.301192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:58.276482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366146+0000","last_change":"2023-10-21T16:17:25.288172+0000","last_active":"2023-10-21T16:17:27.366146+0000","last_peered":"2023-10-21T16:17:27.366146+0000","last_clean":"2023-10-21T16:17:27.366146+0000","last_became_active":"2023-10-21T16:17:25.287777+0000","last_became_peered":"2023-10-21T16:17:25.287777+0000","last_unstale":"2023-10-21T16:17:27.366146+0000","last_undegraded":"2023-10-21T16:17:27.366146+0000","last_fullsized":"2023-10-21T16:17:27.366146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:05.347647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367131+0000","last_change":"2023-10-21T16:17:26.285481+0000","last_active":"2023-10-21T16:17:27.367131+0000","last_peered":"2023-10-21T16:17:27.367131+0000","last_clean":"2023-10-21T16:17:27.367131+0000","last_became_active":"2023-10-21T16:17:26.285137+0000","last_became_peered":"2023-10-21T16:17:26.285137+0000","last_unstale":"2023-10-21T16:17:27.367131+0000","last_undegraded":"2023-10-21T16:17:27.367131+0000","last_fullsized":"2023-10-21T16:17:27.367131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:36:37.094947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367934+0000","last_change":"2023-10-21T16:17:25.275674+0000","last_active":"2023-10-21T16:17:27.367934+0000","last_peered":"2023-10-21T16:17:27.367934+0000","last_clean":"2023-10-21T16:17:27.367934+0000","last_became_active":"2023-10-21T16:17:25.274886+0000","last_became_peered":"2023-10-21T16:17:25.274886+0000","last_unstale":"2023-10-21T16:17:27.367934+0000","last_undegraded":"2023-10-21T16:17:27.367934+0000","last_fullsized":"2023-10-21T16:17:27.367934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:11.416665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366620+0000","last_change":"2023-10-21T16:17:26.290817+0000","last_active":"2023-10-21T16:17:27.366620+0000","last_peered":"2023-10-21T16:17:27.366620+0000","last_clean":"2023-10-21T16:17:27.366620+0000","last_became_active":"2023-10-21T16:17:26.290537+0000","last_became_peered":"2023-10-21T16:17:26.290537+0000","last_unstale":"2023-10-21T16:17:27.366620+0000","last_undegraded":"2023-10-21T16:17:27.366620+0000","last_fullsized":"2023-10-21T16:17:27.366620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:53.559520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366480+0000","last_change":"2023-10-21T16:17:25.284832+0000","last_active":"2023-10-21T16:17:27.366480+0000","last_peered":"2023-10-21T16:17:27.366480+0000","last_clean":"2023-10-21T16:17:27.366480+0000","last_became_active":"2023-10-21T16:17:25.284544+0000","last_became_peered":"2023-10-21T16:17:25.284544+0000","last_unstale":"2023-10-21T16:17:27.366480+0000","last_undegraded":"2023-10-21T16:17:27.366480+0000","last_fullsized":"2023-10-21T16:17:27.366480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:15.254800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366598+0000","last_change":"2023-10-21T16:17:26.300244+0000","last_active":"2023-10-21T16:17:27.366598+0000","last_peered":"2023-10-21T16:17:27.366598+0000","last_clean":"2023-10-21T16:17:27.366598+0000","last_became_active":"2023-10-21T16:17:26.299962+0000","last_became_peered":"2023-10-21T16:17:26.299962+0000","last_unstale":"2023-10-21T16:17:27.366598+0000","last_undegraded":"2023-10-21T16:17:27.366598+0000","last_fullsized":"2023-10-21T16:17:27.366598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:52.423857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"89'8","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301050+0000","last_change":"2023-10-21T16:17:25.287009+0000","last_active":"2023-10-21T16:17:27.301050+0000","last_peered":"2023-10-21T16:17:27.301050+0000","last_clean":"2023-10-21T16:17:27.301050+0000","last_became_active":"2023-10-21T16:17:25.285744+0000","last_became_peered":"2023-10-21T16:17:25.285744+0000","last_unstale":"2023-10-21T16:17:27.301050+0000","last_undegraded":"2023-10-21T16:17:27.301050+0000","last_fullsized":"2023-10-21T16:17:27.301050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:16.827311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365759+0000","last_change":"2023-10-21T16:17:26.284856+0000","last_active":"2023-10-21T16:17:27.365759+0000","last_peered":"2023-10-21T16:17:27.365759+0000","last_clean":"2023-10-21T16:17:27.365759+0000","last_became_active":"2023-10-21T16:17:26.284552+0000","last_became_peered":"2023-10-21T16:17:26.284552+0000","last_unstale":"2023-10-21T16:17:27.365759+0000","last_undegraded":"2023-10-21T16:17:27.365759+0000","last_fullsized":"2023-10-21T16:17:27.365759+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:11.774948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369915+0000","last_change":"2023-10-21T16:17:25.278013+0000","last_active":"2023-10-21T16:17:27.369915+0000","last_peered":"2023-10-21T16:17:27.369915+0000","last_clean":"2023-10-21T16:17:27.369915+0000","last_became_active":"2023-10-21T16:17:25.277678+0000","last_became_peered":"2023-10-21T16:17:25.277678+0000","last_unstale":"2023-10-21T16:17:27.369915+0000","last_undegraded":"2023-10-21T16:17:27.369915+0000","last_fullsized":"2023-10-21T16:17:27.369915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:19:41.079975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369161+0000","last_change":"2023-10-21T16:17:26.298784+0000","last_active":"2023-10-21T16:17:27.369161+0000","last_peered":"2023-10-21T16:17:27.369161+0000","last_clean":"2023-10-21T16:17:27.369161+0000","last_became_active":"2023-10-21T16:17:26.298301+0000","last_became_peered":"2023-10-21T16:17:26.298301+0000","last_unstale":"2023-10-21T16:17:27.369161+0000","last_undegraded":"2023-10-21T16:17:27.369161+0000","last_fullsized":"2023-10-21T16:17:27.369161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:40:22.033419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367318+0000","last_change":"2023-10-21T16:17:25.289816+0000","last_active":"2023-10-21T16:17:27.367318+0000","last_peered":"2023-10-21T16:17:27.367318+0000","last_clean":"2023-10-21T16:17:27.367318+0000","last_became_active":"2023-10-21T16:17:25.289595+0000","last_became_peered":"2023-10-21T16:17:25.289595+0000","last_unstale":"2023-10-21T16:17:27.367318+0000","last_undegraded":"2023-10-21T16:17:27.367318+0000","last_fullsized":"2023-10-21T16:17:27.367318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:23.896347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302942+0000","last_change":"2023-10-21T16:17:26.299750+0000","last_active":"2023-10-21T16:17:27.302942+0000","last_peered":"2023-10-21T16:17:27.302942+0000","last_clean":"2023-10-21T16:17:27.302942+0000","last_became_active":"2023-10-21T16:17:26.299483+0000","last_became_peered":"2023-10-21T16:17:26.299483+0000","last_unstale":"2023-10-21T16:17:27.302942+0000","last_undegraded":"2023-10-21T16:17:27.302942+0000","last_fullsized":"2023-10-21T16:17:27.302942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:20.551598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366267+0000","last_change":"2023-10-21T16:17:25.286921+0000","last_active":"2023-10-21T16:17:27.366267+0000","last_peered":"2023-10-21T16:17:27.366267+0000","last_clean":"2023-10-21T16:17:27.366267+0000","last_became_active":"2023-10-21T16:17:25.286653+0000","last_became_peered":"2023-10-21T16:17:25.286653+0000","last_unstale":"2023-10-21T16:17:27.366267+0000","last_undegraded":"2023-10-21T16:17:27.366267+0000","last_fullsized":"2023-10-21T16:17:27.366267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:34.072286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368575+0000","last_change":"2023-10-21T16:17:26.295660+0000","last_active":"2023-10-21T16:17:27.368575+0000","last_peered":"2023-10-21T16:17:27.368575+0000","last_clean":"2023-10-21T16:17:27.368575+0000","last_became_active":"2023-10-21T16:17:26.295312+0000","last_became_peered":"2023-10-21T16:17:26.295312+0000","last_unstale":"2023-10-21T16:17:27.368575+0000","last_undegraded":"2023-10-21T16:17:27.368575+0000","last_fullsized":"2023-10-21T16:17:27.368575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:06:08.557188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297680+0000","last_change":"2023-10-21T16:17:25.276983+0000","last_active":"2023-10-21T16:17:27.297680+0000","last_peered":"2023-10-21T16:17:27.297680+0000","last_clean":"2023-10-21T16:17:27.297680+0000","last_became_active":"2023-10-21T16:17:25.276601+0000","last_became_peered":"2023-10-21T16:17:25.276601+0000","last_unstale":"2023-10-21T16:17:27.297680+0000","last_undegraded":"2023-10-21T16:17:27.297680+0000","last_fullsized":"2023-10-21T16:17:27.297680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:16.744447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.364811+0000","last_change":"2023-10-21T16:17:26.292644+0000","last_active":"2023-10-21T16:17:27.364811+0000","last_peered":"2023-10-21T16:17:27.364811+0000","last_clean":"2023-10-21T16:17:27.364811+0000","last_became_active":"2023-10-21T16:17:26.292322+0000","last_became_peered":"2023-10-21T16:17:26.292322+0000","last_unstale":"2023-10-21T16:17:27.364811+0000","last_undegraded":"2023-10-21T16:17:27.364811+0000","last_fullsized":"2023-10-21T16:17:27.364811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:49.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.364963+0000","last_change":"2023-10-21T16:17:25.284830+0000","last_active":"2023-10-21T16:17:27.364963+0000","last_peered":"2023-10-21T16:17:27.364963+0000","last_clean":"2023-10-21T16:17:27.364963+0000","last_became_active":"2023-10-21T16:17:25.284537+0000","last_became_peered":"2023-10-21T16:17:25.284537+0000","last_unstale":"2023-10-21T16:17:27.364963+0000","last_undegraded":"2023-10-21T16:17:27.364963+0000","last_fullsized":"2023-10-21T16:17:27.364963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:21.696353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367496+0000","last_change":"2023-10-21T16:17:26.282406+0000","last_active":"2023-10-21T16:17:27.367496+0000","last_peered":"2023-10-21T16:17:27.367496+0000","last_clean":"2023-10-21T16:17:27.367496+0000","last_became_active":"2023-10-21T16:17:26.281920+0000","last_became_peered":"2023-10-21T16:17:26.281920+0000","last_unstale":"2023-10-21T16:17:27.367496+0000","last_undegraded":"2023-10-21T16:17:27.367496+0000","last_fullsized":"2023-10-21T16:17:27.367496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:44.434177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369546+0000","last_change":"2023-10-21T16:17:25.273758+0000","last_active":"2023-10-21T16:17:27.369546+0000","last_peered":"2023-10-21T16:17:27.369546+0000","last_clean":"2023-10-21T16:17:27.369546+0000","last_became_active":"2023-10-21T16:17:25.273285+0000","last_became_peered":"2023-10-21T16:17:25.273285+0000","last_unstale":"2023-10-21T16:17:27.369546+0000","last_undegraded":"2023-10-21T16:17:27.369546+0000","last_fullsized":"2023-10-21T16:17:27.369546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:38.570991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368005+0000","last_change":"2023-10-21T16:17:26.290724+0000","last_active":"2023-10-21T16:17:27.368005+0000","last_peered":"2023-10-21T16:17:27.368005+0000","last_clean":"2023-10-21T16:17:27.368005+0000","last_became_active":"2023-10-21T16:17:26.290421+0000","last_became_peered":"2023-10-21T16:17:26.290421+0000","last_unstale":"2023-10-21T16:17:27.368005+0000","last_undegraded":"2023-10-21T16:17:27.368005+0000","last_fullsized":"2023-10-21T16:17:27.368005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:00.858570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365946+0000","last_change":"2023-10-21T16:17:25.280584+0000","last_active":"2023-10-21T16:17:27.365946+0000","last_peered":"2023-10-21T16:17:27.365946+0000","last_clean":"2023-10-21T16:17:27.365946+0000","last_became_active":"2023-10-21T16:17:25.280267+0000","last_became_peered":"2023-10-21T16:17:25.280267+0000","last_unstale":"2023-10-21T16:17:27.365946+0000","last_undegraded":"2023-10-21T16:17:27.365946+0000","last_fullsized":"2023-10-21T16:17:27.365946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:07.790899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365219+0000","last_change":"2023-10-21T16:17:26.289215+0000","last_active":"2023-10-21T16:17:27.365219+0000","last_peered":"2023-10-21T16:17:27.365219+0000","last_clean":"2023-10-21T16:17:27.365219+0000","last_became_active":"2023-10-21T16:17:26.288885+0000","last_became_peered":"2023-10-21T16:17:26.288885+0000","last_unstale":"2023-10-21T16:17:27.365219+0000","last_undegraded":"2023-10-21T16:17:27.365219+0000","last_fullsized":"2023-10-21T16:17:27.365219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:16.269805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.370044+0000","last_change":"2023-10-21T16:17:25.286175+0000","last_active":"2023-10-21T16:17:27.370044+0000","last_peered":"2023-10-21T16:17:27.370044+0000","last_clean":"2023-10-21T16:17:27.370044+0000","last_became_active":"2023-10-21T16:17:25.285793+0000","last_became_peered":"2023-10-21T16:17:25.285793+0000","last_unstale":"2023-10-21T16:17:27.370044+0000","last_undegraded":"2023-10-21T16:17:27.370044+0000","last_fullsized":"2023-10-21T16:17:27.370044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:27.957248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299250+0000","last_change":"2023-10-21T16:17:26.281191+0000","last_active":"2023-10-21T16:17:27.299250+0000","last_peered":"2023-10-21T16:17:27.299250+0000","last_clean":"2023-10-21T16:17:27.299250+0000","last_became_active":"2023-10-21T16:17:26.280828+0000","last_became_peered":"2023-10-21T16:17:26.280828+0000","last_unstale":"2023-10-21T16:17:27.299250+0000","last_undegraded":"2023-10-21T16:17:27.299250+0000","last_fullsized":"2023-10-21T16:17:27.299250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:20:12.571924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369867+0000","last_change":"2023-10-21T16:17:25.278680+0000","last_active":"2023-10-21T16:17:27.369867+0000","last_peered":"2023-10-21T16:17:27.369867+0000","last_clean":"2023-10-21T16:17:27.369867+0000","last_became_active":"2023-10-21T16:17:25.276902+0000","last_became_peered":"2023-10-21T16:17:25.276902+0000","last_unstale":"2023-10-21T16:17:27.369867+0000","last_undegraded":"2023-10-21T16:17:27.369867+0000","last_fullsized":"2023-10-21T16:17:27.369867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:39.891079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369281+0000","last_change":"2023-10-21T16:17:26.287903+0000","last_active":"2023-10-21T16:17:27.369281+0000","last_peered":"2023-10-21T16:17:27.369281+0000","last_clean":"2023-10-21T16:17:27.369281+0000","last_became_active":"2023-10-21T16:17:26.287333+0000","last_became_peered":"2023-10-21T16:17:26.287333+0000","last_unstale":"2023-10-21T16:17:27.369281+0000","last_undegraded":"2023-10-21T16:17:27.369281+0000","last_fullsized":"2023-10-21T16:17:27.369281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:40.063531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365829+0000","last_change":"2023-10-21T16:17:25.279368+0000","last_active":"2023-10-21T16:17:27.365829+0000","last_peered":"2023-10-21T16:17:27.365829+0000","last_clean":"2023-10-21T16:17:27.365829+0000","last_became_active":"2023-10-21T16:17:25.278597+0000","last_became_peered":"2023-10-21T16:17:25.278597+0000","last_unstale":"2023-10-21T16:17:27.365829+0000","last_undegraded":"2023-10-21T16:17:27.365829+0000","last_fullsized":"2023-10-21T16:17:27.365829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:11.234181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368985+0000","last_change":"2023-10-21T16:17:26.297310+0000","last_active":"2023-10-21T16:17:27.368985+0000","last_peered":"2023-10-21T16:17:27.368985+0000","last_clean":"2023-10-21T16:17:27.368985+0000","last_became_active":"2023-10-21T16:17:26.296995+0000","last_became_peered":"2023-10-21T16:17:26.296995+0000","last_unstale":"2023-10-21T16:17:27.368985+0000","last_undegraded":"2023-10-21T16:17:27.368985+0000","last_fullsized":"2023-10-21T16:17:27.368985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:56.406047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366671+0000","last_change":"2023-10-21T16:17:25.282036+0000","last_active":"2023-10-21T16:17:27.366671+0000","last_peered":"2023-10-21T16:17:27.366671+0000","last_clean":"2023-10-21T16:17:27.366671+0000","last_became_active":"2023-10-21T16:17:25.281693+0000","last_became_peered":"2023-10-21T16:17:25.281693+0000","last_unstale":"2023-10-21T16:17:27.366671+0000","last_undegraded":"2023-10-21T16:17:27.366671+0000","last_fullsized":"2023-10-21T16:17:27.366671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:19.536592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366210+0000","last_change":"2023-10-21T16:17:26.295607+0000","last_active":"2023-10-21T16:17:27.366210+0000","last_peered":"2023-10-21T16:17:27.366210+0000","last_clean":"2023-10-21T16:17:27.366210+0000","last_became_active":"2023-10-21T16:17:26.295195+0000","last_became_peered":"2023-10-21T16:17:26.295195+0000","last_unstale":"2023-10-21T16:17:27.366210+0000","last_undegraded":"2023-10-21T16:17:27.366210+0000","last_fullsized":"2023-10-21T16:17:27.366210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:13.092858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366421+0000","last_change":"2023-10-21T16:17:25.283721+0000","last_active":"2023-10-21T16:17:27.366421+0000","last_peered":"2023-10-21T16:17:27.366421+0000","last_clean":"2023-10-21T16:17:27.366421+0000","last_became_active":"2023-10-21T16:17:25.283420+0000","last_became_peered":"2023-10-21T16:17:25.283420+0000","last_unstale":"2023-10-21T16:17:27.366421+0000","last_undegraded":"2023-10-21T16:17:27.366421+0000","last_fullsized":"2023-10-21T16:17:27.366421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:07.790899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301391+0000","last_change":"2023-10-21T16:17:26.283797+0000","last_active":"2023-10-21T16:17:27.301391+0000","last_peered":"2023-10-21T16:17:27.301391+0000","last_clean":"2023-10-21T16:17:27.301391+0000","last_became_active":"2023-10-21T16:17:26.283482+0000","last_became_peered":"2023-10-21T16:17:26.283482+0000","last_unstale":"2023-10-21T16:17:27.301391+0000","last_undegraded":"2023-10-21T16:17:27.301391+0000","last_fullsized":"2023-10-21T16:17:27.301391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:14.588110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297184+0000","last_change":"2023-10-21T16:17:25.271206+0000","last_active":"2023-10-21T16:17:27.297184+0000","last_peered":"2023-10-21T16:17:27.297184+0000","last_clean":"2023-10-21T16:17:27.297184+0000","last_became_active":"2023-10-21T16:17:25.270855+0000","last_became_peered":"2023-10-21T16:17:25.270855+0000","last_unstale":"2023-10-21T16:17:27.297184+0000","last_undegraded":"2023-10-21T16:17:27.297184+0000","last_fullsized":"2023-10-21T16:17:27.297184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:37.021942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365815+0000","last_change":"2023-10-21T16:17:26.286806+0000","last_active":"2023-10-21T16:17:27.365815+0000","last_peered":"2023-10-21T16:17:27.365815+0000","last_clean":"2023-10-21T16:17:27.365815+0000","last_became_active":"2023-10-21T16:17:26.286500+0000","last_became_peered":"2023-10-21T16:17:26.286500+0000","last_unstale":"2023-10-21T16:17:27.365815+0000","last_undegraded":"2023-10-21T16:17:27.365815+0000","last_fullsized":"2023-10-21T16:17:27.365815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:04:29.762951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297136+0000","last_change":"2023-10-21T16:17:25.285562+0000","last_active":"2023-10-21T16:17:27.297136+0000","last_peered":"2023-10-21T16:17:27.297136+0000","last_clean":"2023-10-21T16:17:27.297136+0000","last_became_active":"2023-10-21T16:17:25.284965+0000","last_became_peered":"2023-10-21T16:17:25.284965+0000","last_unstale":"2023-10-21T16:17:27.297136+0000","last_undegraded":"2023-10-21T16:17:27.297136+0000","last_fullsized":"2023-10-21T16:17:27.297136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:04.256308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299300+0000","last_change":"2023-10-21T16:17:26.293843+0000","last_active":"2023-10-21T16:17:27.299300+0000","last_peered":"2023-10-21T16:17:27.299300+0000","last_clean":"2023-10-21T16:17:27.299300+0000","last_became_active":"2023-10-21T16:17:26.286543+0000","last_became_peered":"2023-10-21T16:17:26.286543+0000","last_unstale":"2023-10-21T16:17:27.299300+0000","last_undegraded":"2023-10-21T16:17:27.299300+0000","last_fullsized":"2023-10-21T16:17:27.299300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:24.054686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365324+0000","last_change":"2023-10-21T16:17:25.287677+0000","last_active":"2023-10-21T16:17:27.365324+0000","last_peered":"2023-10-21T16:17:27.365324+0000","last_clean":"2023-10-21T16:17:27.365324+0000","last_became_active":"2023-10-21T16:17:25.287459+0000","last_became_peered":"2023-10-21T16:17:25.287459+0000","last_unstale":"2023-10-21T16:17:27.365324+0000","last_undegraded":"2023-10-21T16:17:27.365324+0000","last_fullsized":"2023-10-21T16:17:27.365324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:37.878379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368876+0000","last_change":"2023-10-21T16:17:26.298361+0000","last_active":"2023-10-21T16:17:27.368876+0000","last_peered":"2023-10-21T16:17:27.368876+0000","last_clean":"2023-10-21T16:17:27.368876+0000","last_became_active":"2023-10-21T16:17:26.296172+0000","last_became_peered":"2023-10-21T16:17:26.296172+0000","last_unstale":"2023-10-21T16:17:27.368876+0000","last_undegraded":"2023-10-21T16:17:27.368876+0000","last_fullsized":"2023-10-21T16:17:27.368876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:17.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.296910+0000","last_change":"2023-10-21T16:17:25.271022+0000","last_active":"2023-10-21T16:17:27.296910+0000","last_peered":"2023-10-21T16:17:27.296910+0000","last_clean":"2023-10-21T16:17:27.296910+0000","last_became_active":"2023-10-21T16:17:25.270663+0000","last_became_peered":"2023-10-21T16:17:25.270663+0000","last_unstale":"2023-10-21T16:17:27.296910+0000","last_undegraded":"2023-10-21T16:17:27.296910+0000","last_fullsized":"2023-10-21T16:17:27.296910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:26.673991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"78'161","reported_seq":387,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-21T16:17:26.277541+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:17:26.277541+0000","last_peered":"2023-10-21T16:17:26.277541+0000","last_clean":"2023-10-21T16:17:26.277541+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:17:26.277541+0000","last_undegraded":"2023-10-21T16:17:26.277541+0000","last_fullsized":"2023-10-21T16:17:26.277541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1703936,"data_stored":1143214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.65000000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5609999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.171},"min":{"1min":0.76200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4019999999999999,"5min":2.448,"15min":2.448},"last":0.98899999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78300000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5389999999999999,"5min":1.915,"15min":1.915},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.583,"5min":1.778,"15min":1.778},"last":0.875}]},{"osd":2,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.27,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5880000000000001,"5min":1.748,"15min":2.085},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2849999999999999,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.556,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.262}]},{"osd":3,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.2529999999999999},"min":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6040000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.85999999999999999,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.9570000000000001,"15min":2.0099999999999998},"last":1.3169999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3360000000000001,"15min":1.355},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":1.7350000000000001,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.423,"15min":1.4450000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70399999999999996},"max":{"1min":1.8360000000000001,"5min":2.355,"15min":2.355},"last":1.3540000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.391,"15min":1.3999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.716,"5min":1.865,"15min":2.258},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.4350000000000001,"15min":1.4379999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.60899999999999999},"max":{"1min":1.9079999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.4019999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.53,"15min":1.548},"min":{"1min":1.0580000000000001,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":1.706,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.506},{"interface":"front","average":{"1min":1.47,"5min":1.4790000000000001,"15min":1.466},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7709999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.5940000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1097728,"data_stored":552104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.68799999999999994,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.514,"5min":1.895,"15min":2.4350000000000001},"last":1.4330000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.417,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.3480000000000001,"15min":1.306},"min":{"1min":0.90700000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5960000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.92700000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.895,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.099},{"interface":"front","average":{"1min":1.24,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6839999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.2470000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.377,"15min":1.3740000000000001},"min":{"1min":1.1200000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7490000000000001,"5min":2.2000000000000002,"15min":3.46},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.446,"15min":1.4299999999999999},"min":{"1min":1.204,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":1.9970000000000001,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.335}]},{"osd":4,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.393,"15min":1.365},"min":{"1min":0.94599999999999995,"5min":0.82999999999999996,"15min":0.76300000000000001},"max":{"1min":1.9379999999999999,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.401,"15min":1.397},"min":{"1min":1.2110000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1299999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":0.96499999999999997}]},{"osd":5,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":1.1120000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.246,"5min":2.246,"15min":2.6909999999999998},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.99199999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8240000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.163}]},{"osd":7,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.546,"15min":1.5149999999999999},"min":{"1min":1.0429999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.2050000000000001,"5min":2.3570000000000002,"15min":3.71},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.631,"15min":1.623},"min":{"1min":1.4710000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.7650000000000001},"last":1.4910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":552092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0900000000000001,"15min":1.1160000000000001},"min":{"1min":0.81100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4399999999999999,"5min":1.554,"15min":2.0070000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.133,"5min":1.1850000000000001,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.51,"5min":1.712,"15min":1.9530000000000001},"last":1.599}]},{"osd":1,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6339999999999999,"5min":1.877,"15min":1.944},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.91200000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.907},"last":1.123}]},{"osd":2,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.83599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.8,"5min":1.8,"15min":1.95},"last":1.02},{"interface":"front","average":{"1min":1.24,"5min":1.206,"15min":1.242},"min":{"1min":0.85099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.0249999999999999},"last":1.5209999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0880000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.573,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":1.153},{"interface":"front","average":{"1min":1.444,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":1.0009999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.79,"5min":1.79,"15min":1.8959999999999999},"last":1.333}]},{"osd":4,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2050000000000001,"15min":1.216},"min":{"1min":0.93999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.919,"5min":1.919,"15min":2.0960000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.349,"15min":1.339},"min":{"1min":0.96499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.76,"5min":2.0529999999999999,"15min":2.121},"last":1.837}]},{"osd":6,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.4550000000000001,"15min":1.45},"min":{"1min":1.3819999999999999,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1360000000000001},"last":1.23},{"interface":"front","average":{"1min":1.552,"5min":1.4790000000000001,"15min":1.4870000000000001},"min":{"1min":1.0860000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":1.6699999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.516,"15min":1.504},"min":{"1min":1.1699999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8300000000000001,"5min":2.218,"15min":2.218},"last":1.9350000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.534,"15min":1.556},"min":{"1min":1.371,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.286,"15min":2.286},"last":1.4319999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":553682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1830000000000001,"15min":1.1379999999999999},"min":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.55400000000000005},"max":{"1min":1.5509999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.194,"15min":1.21},"min":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7230000000000001,"5min":1.758,"15min":3.6080000000000001},"last":1.452}]},{"osd":1,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.252,"15min":1.2629999999999999},"min":{"1min":0.78200000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.663,"5min":1.885,"15min":2.0699999999999998},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2190000000000001,"15min":1.228},"min":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5609999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.179}]},{"osd":2,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.637,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.298,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.698,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.02}]},{"osd":3,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.946,"5min":2.5099999999999998,"15min":3.8109999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.409,"15min":1.3779999999999999},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.8340000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2889999999999999,"15min":1.3080000000000001},"min":{"1min":0.874,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7889999999999999,"5min":3.9199999999999999,"15min":3.9199999999999999},"last":1.607},{"interface":"front","average":{"1min":1.343,"5min":1.335,"15min":1.3340000000000001},"min":{"1min":0.89500000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0670000000000002,"5min":3.0699999999999998,"15min":3.9300000000000002},"last":1.885}]},{"osd":6,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.8080000000000001,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.583,"15min":1.5800000000000001},"min":{"1min":1.2470000000000001,"5min":0.878,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.4500000000000002,"15min":3.9609999999999999},"last":1.9710000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5009999999999999,"15min":1.492},"min":{"1min":0.91800000000000004,"5min":0.77200000000000002,"15min":0.69599999999999995},"max":{"1min":2.2170000000000001,"5min":3.1859999999999999,"15min":4.0190000000000001},"last":2.2400000000000002},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6120000000000001,"15min":1.5609999999999999},"min":{"1min":1.3320000000000001,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9490000000000001,"5min":3.968,"15min":3.968},"last":1.9239999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":552070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98199999999999998,"15min":0.94999999999999996},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.526,"5min":1.746,"15min":1.746},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.1779999999999999,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.365,"5min":2.04,"15min":2.5289999999999999},"last":0.72899999999999998}]},{"osd":1,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99399999999999999,"15min":1.0029999999999999},"min":{"1min":0.71599999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.53,"5min":2.173,"15min":2.2519999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":1.0069999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.8100000000000001,"15min":3.827},"last":0.62}]},{"osd":2,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.3360000000000001,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.206,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.663,"5min":2.847,"15min":2.847},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.403},"min":{"1min":1.101,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6970000000000001,"5min":2.1040000000000001,"15min":2.2349999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.1180000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.8160000000000001,"5min":2.919,"15min":2.919},"last":0.88800000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.415,"15min":1.427},"min":{"1min":1.167,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.619,"5min":2.02,"15min":3.7250000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4930000000000001,"15min":1.4990000000000001},"min":{"1min":1.129,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.841,"5min":2.2629999999999999,"15min":2.423},"last":1.0660000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.1399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.054},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.2909999999999999,"5min":0.871,"15min":0.70299999999999996},"max":{"1min":1.7689999999999999,"5min":2.2290000000000001,"15min":3.573},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5740000000000001,"15min":1.591},"min":{"1min":1.2669999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.915,"5min":2.302,"15min":2.3759999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5780000000000001,"15min":1.6100000000000001},"min":{"1min":1.304,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.1280000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.0960000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1093632,"data_stored":552070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.85999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.268,"5min":1.859,"15min":1.859},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.42699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.331,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.69599999999999995}]},{"osd":1,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.093,"15min":1.129},"min":{"1min":0.63400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.528,"5min":1.907,"15min":1.9390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.206,"15min":1.2549999999999999},"min":{"1min":0.64900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.657,"5min":1.881,"15min":1.881},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.0580000000000001,"15min":1.1240000000000001},"min":{"1min":0.60999999999999999,"5min":0.438,"15min":0.379},"max":{"1min":1.7230000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3240000000000001,"15min":1.3640000000000001},"min":{"1min":0.69799999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.6919999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.345}]},{"osd":4,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3600000000000001,"15min":1.369},"min":{"1min":0.94499999999999995,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.958,"5min":1.958,"15min":2.206},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.825,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.343},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":2.024,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1099999999999999,"5min":2.1349999999999998,"15min":2.4870000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4470000000000001,"15min":1.462},"min":{"1min":0.93000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7589999999999999,"5min":2.6819999999999999,"15min":2.698},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.4359999999999999},"min":{"1min":0.84999999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8200000000000001,"5min":2.552,"15min":2.6179999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.494,"15min":1.5309999999999999},"min":{"1min":0.88,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.863,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4590000000000001,"15min":1.5129999999999999},"min":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.865,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.1439999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1122304,"data_stored":554446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88500000000000001,"15min":0.91000000000000003},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.323,"5min":1.905,"15min":1.905},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.071,"15min":1.0740000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6930000000000001,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":1.006}]},{"osd":2,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":0.997},"min":{"1min":0.53500000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4419999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.133,"5min":1.157,"15min":1.1850000000000001},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.974,"15min":2.29},"last":1.196}]},{"osd":3,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.244,"15min":1.3},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7350000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.323},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5489999999999999,"5min":1.877,"15min":2.3620000000000001},"last":1.1160000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2589999999999999,"15min":1.276},"min":{"1min":0.88500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.71,"5min":1.944,"15min":1.944},"last":1.157},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.833,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.468}]},{"osd":5,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.4610000000000001,"15min":1.5049999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.75900000000000001},"max":{"1min":1.9099999999999999,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.458,"15min":1.4790000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.52}]},{"osd":6,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4790000000000001,"15min":1.5109999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.984,"5min":2.2469999999999999,"15min":4.1260000000000003},"last":1.377},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4730000000000001},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9510000000000001,"5min":2.2090000000000001,"15min":3.101},"last":1.2689999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5820000000000001,"15min":1.609},"min":{"1min":0.872,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2530000000000001,"5min":2.2679999999999998,"15min":2.8180000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5569999999999999,"15min":1.613},"min":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.901,"5min":2.1619999999999999,"15min":3.952},"last":1.659}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1687552,"data_stored":1142438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.583,"5min":1.782,"15min":2.9260000000000002},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0569999999999999},"min":{"1min":0.66000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":1.5329999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.075,"15min":1.123},"min":{"1min":0.80600000000000005,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":1.6559999999999999,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":1.234},{"interface":"front","average":{"1min":1.131,"5min":1.125,"15min":1.147},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.42999999999999999},"max":{"1min":1.603,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.194}]},{"osd":3,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.093},"min":{"1min":0.67200000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.702,"5min":1.8660000000000001,"15min":2.1429999999999998},"last":1.121},{"interface":"front","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.29},"min":{"1min":0.91400000000000003,"5min":0.48599999999999999,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8109999999999999,"15min":3.052},"last":1.4990000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3939999999999999,"15min":1.417},"min":{"1min":0.98699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.756,"5min":2.0339999999999998,"15min":3.3580000000000001},"last":1.702},{"interface":"front","average":{"1min":1.546,"5min":1.4950000000000001,"15min":1.5309999999999999},"min":{"1min":1.3069999999999999,"5min":0.872,"15min":0.872},"max":{"1min":1.863,"5min":2.246,"15min":3.306},"last":1.448}]},{"osd":5,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3859999999999999},"min":{"1min":1.0129999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6279999999999999,"5min":1.9419999999999999,"15min":3.8849999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.454,"15min":1.504},"min":{"1min":1.046,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8089999999999999,"5min":2.137,"15min":3.1789999999999998},"last":1.3819999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4419999999999999,"15min":1.4850000000000001},"min":{"1min":0.95799999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.8069999999999999,"5min":2.5379999999999998,"15min":4.3019999999999996},"last":1.579},{"interface":"front","average":{"1min":1.526,"5min":1.5489999999999999,"15min":1.595},"min":{"1min":1.2030000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8069999999999999,"5min":2.355,"15min":3.5139999999999998},"last":1.5609999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4330000000000001,"15min":1.4610000000000001},"min":{"1min":1.0389999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.637,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.417},{"interface":"front","average":{"1min":1.476,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":1.0960000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.802,"5min":2.077,"15min":4.6219999999999999},"last":1.4630000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:17:30.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:17:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.079+0000 7f47a8ba9640 1 Processor -- start 2023-10-21T16:17:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.080+0000 7f47a8ba9640 1 -- start start 2023-10-21T16:17:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.081+0000 7f47a8ba9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.081+0000 7f47a8ba9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f47a40cb580 0x7f47a40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.081+0000 7f47a8ba9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a415b2b0 con 0x7f47a40c90b0 2023-10-21T16:17:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.081+0000 7f47a8ba9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a415bf30 con 0x7f47a40cb580 2023-10-21T16:17:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.081+0000 7f47a8ba9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a415f7a0 con 0x7f47a415ab40 2023-10-21T16:17:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a1d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f47a40c90b0 0x7f47a40cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40888/0 (socket says 172.21.15.53:40888) 2023-10-21T16:17:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a1d9b640 1 -- 172.21.15.53:0/1468820862 learned_addr learned my addr 172.21.15.53:0/1468820862 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a2d9d640 1 --2- 172.21.15.53:0/1468820862 >> v2:172.21.15.53:3300/0 conn(0x7f47a40cb580 0x7f47a40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a259c640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a159a640 1 -- 172.21.15.53:0/1468820862 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212187359 0 0) 0x7f47a415b2b0 con 0x7f47a40c90b0 2023-10-21T16:17:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a159a640 1 -- 172.21.15.53:0/1468820862 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4790003520 con 0x7f47a40c90b0 2023-10-21T16:17:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a2d9d640 1 -- 172.21.15.53:0/1468820862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 msgr2=0x7f47a40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.082+0000 7f47a2d9d640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a40ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.083+0000 7f47a2d9d640 1 -- 172.21.15.53:0/1468820862 >> v1:172.21.15.53:6789/0 conn(0x7f47a40c90b0 legacy=0x7f47a40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.083+0000 7f47a2d9d640 1 -- 172.21.15.53:0/1468820862 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a415b590 con 0x7f47a40cb580 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.084+0000 7f47a2d9d640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a40adc60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f479c0097a0 tx=0x7f479c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4cf432fb350c5f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.084+0000 7f47a159a640 1 -- 172.21.15.53:0/1468820862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c031830 con 0x7f47a40cb580 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.084+0000 7f47a159a640 1 -- 172.21.15.53:0/1468820862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c031990 con 0x7f47a40cb580 2023-10-21T16:17:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.084+0000 7f47a159a640 1 -- 172.21.15.53:0/1468820862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c031d20 con 0x7f47a40cb580 2023-10-21T16:17:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 -- 172.21.15.53:0/1468820862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 msgr2=0x7f47a40adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a40adc60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f479c0097a0 tx=0x7f479c02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 -- 172.21.15.53:0/1468820862 shutdown_connections 2023-10-21T16:17:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a40adc60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/1468820862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 -- 172.21.15.53:0/1468820862 >> 172.21.15.53:0/1468820862 conn(0x7f47a406a380 msgr2=0x7f47a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:31.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 -- 172.21.15.53:0/1468820862 shutdown_connections 2023-10-21T16:17:31.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.085+0000 7f47a8ba9640 1 -- 172.21.15.53:0/1468820862 wait complete. 2023-10-21T16:17:31.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.087+0000 7f47a8ba9640 1 Processor -- start 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.087+0000 7f47a8ba9640 1 -- start start 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47a40c90b0 0x7f47a40ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40ac7e0 con 0x7f47a40cb580 2023-10-21T16:17:31.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40ac9d0 con 0x7f47a40c90b0 2023-10-21T16:17:31.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a8ba9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a4159ed0 con 0x7f47a415ab40 2023-10-21T16:17:31.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49582/0 (socket says 172.21.15.53:49582) 2023-10-21T16:17:31.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a1d9b640 1 -- 172.21.15.53:0/214365968 learned_addr learned my addr 172.21.15.53:0/214365968 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a2d9d640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a259c640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47a40c90b0 0x7f47a40ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a1d9b640 1 -- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 msgr2=0x7f47a4159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.088+0000 7f47a1d9b640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a4159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a1d9b640 1 -- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47a40c90b0 msgr2=0x7f47a40ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a1d9b640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47a40c90b0 0x7f47a40ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a1d9b640 1 -- 172.21.15.53:0/214365968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009400 con 0x7f47a40cb580 2023-10-21T16:17:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a1d9b640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c440 tx=0x7f478c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c005ce0 con 0x7f47a40cb580 2023-10-21T16:17:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c005e40 con 0x7f47a40cb580 2023-10-21T16:17:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c0072b0 con 0x7f47a40cb580 2023-10-21T16:17:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a415a3b0 con 0x7f47a40cb580 2023-10-21T16:17:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.089+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4153000 con 0x7f47a40cb580 2023-10-21T16:17:31.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.091+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a4067490 con 0x7f47a40cb580 2023-10-21T16:17:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.092+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f478c007410 con 0x7f47a40cb580 2023-10-21T16:17:31.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.092+0000 7f479affd640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 0x7f478406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.093+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f478c088ee0 con 0x7f47a40cb580 2023-10-21T16:17:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.093+0000 7f47a259c640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 0x7f478406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.094+0000 7f47a259c640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 0x7f478406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47940074a0 tx=0x7f479400b040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.097+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f478c053dc0 con 0x7f47a40cb580 2023-10-21T16:17:31.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.288+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 --> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47a415bb50 con 0x7f478406af90 2023-10-21T16:17:31.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.310+0000 7f479affd640 1 -- 172.21.15.53:0/214365968 <== mgr.5734 v2:172.21.15.124:6832/1636058776 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286772 (secure 0 0 0) 0x7f479400f050 con 0x7f478406af90 2023-10-21T16:17:31.316 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:31.317 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:17:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.316+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 msgr2=0x7f478406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.316+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 0x7f478406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47940074a0 tx=0x7f479400b040 comp rx=0 tx=0).stop 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.316+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 msgr2=0x7f47a4159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.316+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f478c00c440 tx=0x7f478c00c900 comp rx=0 tx=0).stop 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 shutdown_connections 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f478406af90 0x7f478406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47a415ab40 0x7f47a4159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47a40cb580 0x7f47a4159450 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 --2- 172.21.15.53:0/214365968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47a40c90b0 0x7f47a40ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 >> 172.21.15.53:0/214365968 conn(0x7f47a406a380 msgr2=0x7f47a40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:31.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 shutdown_connections 2023-10-21T16:17:31.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.317+0000 7f47a8ba9640 1 -- 172.21.15.53:0/214365968 wait complete. 2023-10-21T16:17:31.339 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-21T16:17:29.528109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324444,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754650276,"statfs":{"total":773094113280,"available":772761882624,"internally_reserved":0,"allocated":10002432,"data_stored":5602116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":322096697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.295227"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367084+0000","last_change":"2023-10-21T16:17:26.291685+0000","last_active":"2023-10-21T16:17:27.367084+0000","last_peered":"2023-10-21T16:17:27.367084+0000","last_clean":"2023-10-21T16:17:27.367084+0000","last_became_active":"2023-10-21T16:17:26.290210+0000","last_became_peered":"2023-10-21T16:17:26.290210+0000","last_unstale":"2023-10-21T16:17:27.367084+0000","last_undegraded":"2023-10-21T16:17:27.367084+0000","last_fullsized":"2023-10-21T16:17:27.367084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:58:05.811035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367423+0000","last_change":"2023-10-21T16:17:26.290788+0000","last_active":"2023-10-21T16:17:27.367423+0000","last_peered":"2023-10-21T16:17:27.367423+0000","last_clean":"2023-10-21T16:17:27.367423+0000","last_became_active":"2023-10-21T16:17:26.290479+0000","last_became_peered":"2023-10-21T16:17:26.290479+0000","last_unstale":"2023-10-21T16:17:27.367423+0000","last_undegraded":"2023-10-21T16:17:27.367423+0000","last_fullsized":"2023-10-21T16:17:27.367423+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:54.922933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365988+0000","last_change":"2023-10-21T16:17:26.289047+0000","last_active":"2023-10-21T16:17:27.365988+0000","last_peered":"2023-10-21T16:17:27.365988+0000","last_clean":"2023-10-21T16:17:27.365988+0000","last_became_active":"2023-10-21T16:17:26.288749+0000","last_became_peered":"2023-10-21T16:17:26.288749+0000","last_unstale":"2023-10-21T16:17:27.365988+0000","last_undegraded":"2023-10-21T16:17:27.365988+0000","last_fullsized":"2023-10-21T16:17:27.365988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:34.472760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301974+0000","last_change":"2023-10-21T16:17:26.282142+0000","last_active":"2023-10-21T16:17:27.301974+0000","last_peered":"2023-10-21T16:17:27.301974+0000","last_clean":"2023-10-21T16:17:27.301974+0000","last_became_active":"2023-10-21T16:17:26.281673+0000","last_became_peered":"2023-10-21T16:17:26.281673+0000","last_unstale":"2023-10-21T16:17:27.301974+0000","last_undegraded":"2023-10-21T16:17:27.301974+0000","last_fullsized":"2023-10-21T16:17:27.301974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:58.531464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368525+0000","last_change":"2023-10-21T16:17:26.288820+0000","last_active":"2023-10-21T16:17:27.368525+0000","last_peered":"2023-10-21T16:17:27.368525+0000","last_clean":"2023-10-21T16:17:27.368525+0000","last_became_active":"2023-10-21T16:17:26.288533+0000","last_became_peered":"2023-10-21T16:17:26.288533+0000","last_unstale":"2023-10-21T16:17:27.368525+0000","last_undegraded":"2023-10-21T16:17:27.368525+0000","last_fullsized":"2023-10-21T16:17:27.368525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:21:20.340682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365278+0000","last_change":"2023-10-21T16:17:26.287137+0000","last_active":"2023-10-21T16:17:27.365278+0000","last_peered":"2023-10-21T16:17:27.365278+0000","last_clean":"2023-10-21T16:17:27.365278+0000","last_became_active":"2023-10-21T16:17:26.286792+0000","last_became_peered":"2023-10-21T16:17:26.286792+0000","last_unstale":"2023-10-21T16:17:27.365278+0000","last_undegraded":"2023-10-21T16:17:27.365278+0000","last_fullsized":"2023-10-21T16:17:27.365278+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:29.105875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299165+0000","last_change":"2023-10-21T16:17:26.288485+0000","last_active":"2023-10-21T16:17:27.299165+0000","last_peered":"2023-10-21T16:17:27.299165+0000","last_clean":"2023-10-21T16:17:27.299165+0000","last_became_active":"2023-10-21T16:17:26.285435+0000","last_became_peered":"2023-10-21T16:17:26.285435+0000","last_unstale":"2023-10-21T16:17:27.299165+0000","last_undegraded":"2023-10-21T16:17:27.299165+0000","last_fullsized":"2023-10-21T16:17:27.299165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:40.043572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367477+0000","last_change":"2023-10-21T16:17:26.292237+0000","last_active":"2023-10-21T16:17:27.367477+0000","last_peered":"2023-10-21T16:17:27.367477+0000","last_clean":"2023-10-21T16:17:27.367477+0000","last_became_active":"2023-10-21T16:17:26.291846+0000","last_became_peered":"2023-10-21T16:17:26.291846+0000","last_unstale":"2023-10-21T16:17:27.367477+0000","last_undegraded":"2023-10-21T16:17:27.367477+0000","last_fullsized":"2023-10-21T16:17:27.367477+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:53.045552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367435+0000","last_change":"2023-10-21T16:17:26.287211+0000","last_active":"2023-10-21T16:17:27.367435+0000","last_peered":"2023-10-21T16:17:27.367435+0000","last_clean":"2023-10-21T16:17:27.367435+0000","last_became_active":"2023-10-21T16:17:26.286866+0000","last_became_peered":"2023-10-21T16:17:26.286866+0000","last_unstale":"2023-10-21T16:17:27.367435+0000","last_undegraded":"2023-10-21T16:17:27.367435+0000","last_fullsized":"2023-10-21T16:17:27.367435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:48.229280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366958+0000","last_change":"2023-10-21T16:17:26.293259+0000","last_active":"2023-10-21T16:17:27.366958+0000","last_peered":"2023-10-21T16:17:27.366958+0000","last_clean":"2023-10-21T16:17:27.366958+0000","last_became_active":"2023-10-21T16:17:26.292984+0000","last_became_peered":"2023-10-21T16:17:26.292984+0000","last_unstale":"2023-10-21T16:17:27.366958+0000","last_undegraded":"2023-10-21T16:17:27.366958+0000","last_fullsized":"2023-10-21T16:17:27.366958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:26.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302373+0000","last_change":"2023-10-21T16:17:26.294540+0000","last_active":"2023-10-21T16:17:27.302373+0000","last_peered":"2023-10-21T16:17:27.302373+0000","last_clean":"2023-10-21T16:17:27.302373+0000","last_became_active":"2023-10-21T16:17:26.294272+0000","last_became_peered":"2023-10-21T16:17:26.294272+0000","last_unstale":"2023-10-21T16:17:27.302373+0000","last_undegraded":"2023-10-21T16:17:27.302373+0000","last_fullsized":"2023-10-21T16:17:27.302373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:38.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369282+0000","last_change":"2023-10-21T16:17:26.286271+0000","last_active":"2023-10-21T16:17:27.369282+0000","last_peered":"2023-10-21T16:17:27.369282+0000","last_clean":"2023-10-21T16:17:27.369282+0000","last_became_active":"2023-10-21T16:17:26.285943+0000","last_became_peered":"2023-10-21T16:17:26.285943+0000","last_unstale":"2023-10-21T16:17:27.369282+0000","last_undegraded":"2023-10-21T16:17:27.369282+0000","last_fullsized":"2023-10-21T16:17:27.369282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:28.008873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369272+0000","last_change":"2023-10-21T16:17:26.285026+0000","last_active":"2023-10-21T16:17:27.369272+0000","last_peered":"2023-10-21T16:17:27.369272+0000","last_clean":"2023-10-21T16:17:27.369272+0000","last_became_active":"2023-10-21T16:17:26.284749+0000","last_became_peered":"2023-10-21T16:17:26.284749+0000","last_unstale":"2023-10-21T16:17:27.369272+0000","last_undegraded":"2023-10-21T16:17:27.369272+0000","last_fullsized":"2023-10-21T16:17:27.369272+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:38:53.817692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366531+0000","last_change":"2023-10-21T16:17:26.288357+0000","last_active":"2023-10-21T16:17:27.366531+0000","last_peered":"2023-10-21T16:17:27.366531+0000","last_clean":"2023-10-21T16:17:27.366531+0000","last_became_active":"2023-10-21T16:17:26.286831+0000","last_became_peered":"2023-10-21T16:17:26.286831+0000","last_unstale":"2023-10-21T16:17:27.366531+0000","last_undegraded":"2023-10-21T16:17:27.366531+0000","last_fullsized":"2023-10-21T16:17:27.366531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.972254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365831+0000","last_change":"2023-10-21T16:17:26.294624+0000","last_active":"2023-10-21T16:17:27.365831+0000","last_peered":"2023-10-21T16:17:27.365831+0000","last_clean":"2023-10-21T16:17:27.365831+0000","last_became_active":"2023-10-21T16:17:26.294359+0000","last_became_peered":"2023-10-21T16:17:26.294359+0000","last_unstale":"2023-10-21T16:17:27.365831+0000","last_undegraded":"2023-10-21T16:17:27.365831+0000","last_fullsized":"2023-10-21T16:17:27.365831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:09:01.212669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299079+0000","last_change":"2023-10-21T16:17:26.287669+0000","last_active":"2023-10-21T16:17:27.299079+0000","last_peered":"2023-10-21T16:17:27.299079+0000","last_clean":"2023-10-21T16:17:27.299079+0000","last_became_active":"2023-10-21T16:17:26.287327+0000","last_became_peered":"2023-10-21T16:17:26.287327+0000","last_unstale":"2023-10-21T16:17:27.299079+0000","last_undegraded":"2023-10-21T16:17:27.299079+0000","last_fullsized":"2023-10-21T16:17:27.299079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:06.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367428+0000","last_change":"2023-10-21T16:17:26.286640+0000","last_active":"2023-10-21T16:17:27.367428+0000","last_peered":"2023-10-21T16:17:27.367428+0000","last_clean":"2023-10-21T16:17:27.367428+0000","last_became_active":"2023-10-21T16:17:26.286318+0000","last_became_peered":"2023-10-21T16:17:26.286318+0000","last_unstale":"2023-10-21T16:17:27.367428+0000","last_undegraded":"2023-10-21T16:17:27.367428+0000","last_fullsized":"2023-10-21T16:17:27.367428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:51.768527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302760+0000","last_change":"2023-10-21T16:17:26.284296+0000","last_active":"2023-10-21T16:17:27.302760+0000","last_peered":"2023-10-21T16:17:27.302760+0000","last_clean":"2023-10-21T16:17:27.302760+0000","last_became_active":"2023-10-21T16:17:26.284022+0000","last_became_peered":"2023-10-21T16:17:26.284022+0000","last_unstale":"2023-10-21T16:17:27.302760+0000","last_undegraded":"2023-10-21T16:17:27.302760+0000","last_fullsized":"2023-10-21T16:17:27.302760+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:00.484233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368830+0000","last_change":"2023-10-21T16:17:26.289182+0000","last_active":"2023-10-21T16:17:27.368830+0000","last_peered":"2023-10-21T16:17:27.368830+0000","last_clean":"2023-10-21T16:17:27.368830+0000","last_became_active":"2023-10-21T16:17:26.288813+0000","last_became_peered":"2023-10-21T16:17:26.288813+0000","last_unstale":"2023-10-21T16:17:27.368830+0000","last_undegraded":"2023-10-21T16:17:27.368830+0000","last_fullsized":"2023-10-21T16:17:27.368830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:07.851912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297606+0000","last_change":"2023-10-21T16:17:26.290218+0000","last_active":"2023-10-21T16:17:27.297606+0000","last_peered":"2023-10-21T16:17:27.297606+0000","last_clean":"2023-10-21T16:17:27.297606+0000","last_became_active":"2023-10-21T16:17:26.289914+0000","last_became_peered":"2023-10-21T16:17:26.289914+0000","last_unstale":"2023-10-21T16:17:27.297606+0000","last_undegraded":"2023-10-21T16:17:27.297606+0000","last_fullsized":"2023-10-21T16:17:27.297606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:02.997864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369642+0000","last_change":"2023-10-21T16:17:26.296790+0000","last_active":"2023-10-21T16:17:27.369642+0000","last_peered":"2023-10-21T16:17:27.369642+0000","last_clean":"2023-10-21T16:17:27.369642+0000","last_became_active":"2023-10-21T16:17:26.294405+0000","last_became_peered":"2023-10-21T16:17:26.294405+0000","last_unstale":"2023-10-21T16:17:27.369642+0000","last_undegraded":"2023-10-21T16:17:27.369642+0000","last_fullsized":"2023-10-21T16:17:27.369642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:37.543660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297094+0000","last_change":"2023-10-21T16:17:26.284929+0000","last_active":"2023-10-21T16:17:27.297094+0000","last_peered":"2023-10-21T16:17:27.297094+0000","last_clean":"2023-10-21T16:17:27.297094+0000","last_became_active":"2023-10-21T16:17:26.284598+0000","last_became_peered":"2023-10-21T16:17:26.284598+0000","last_unstale":"2023-10-21T16:17:27.297094+0000","last_undegraded":"2023-10-21T16:17:27.297094+0000","last_fullsized":"2023-10-21T16:17:27.297094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:15.797402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297043+0000","last_change":"2023-10-21T16:17:26.284000+0000","last_active":"2023-10-21T16:17:27.297043+0000","last_peered":"2023-10-21T16:17:27.297043+0000","last_clean":"2023-10-21T16:17:27.297043+0000","last_became_active":"2023-10-21T16:17:26.283650+0000","last_became_peered":"2023-10-21T16:17:26.283650+0000","last_unstale":"2023-10-21T16:17:27.297043+0000","last_undegraded":"2023-10-21T16:17:27.297043+0000","last_fullsized":"2023-10-21T16:17:27.297043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:16.143933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367090+0000","last_change":"2023-10-21T16:17:26.301552+0000","last_active":"2023-10-21T16:17:27.367090+0000","last_peered":"2023-10-21T16:17:27.367090+0000","last_clean":"2023-10-21T16:17:27.367090+0000","last_became_active":"2023-10-21T16:17:26.299042+0000","last_became_peered":"2023-10-21T16:17:26.299042+0000","last_unstale":"2023-10-21T16:17:27.367090+0000","last_undegraded":"2023-10-21T16:17:27.367090+0000","last_fullsized":"2023-10-21T16:17:27.367090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:23.269157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369861+0000","last_change":"2023-10-21T16:17:26.290031+0000","last_active":"2023-10-21T16:17:27.369861+0000","last_peered":"2023-10-21T16:17:27.369861+0000","last_clean":"2023-10-21T16:17:27.369861+0000","last_became_active":"2023-10-21T16:17:26.289729+0000","last_became_peered":"2023-10-21T16:17:26.289729+0000","last_unstale":"2023-10-21T16:17:27.369861+0000","last_undegraded":"2023-10-21T16:17:27.369861+0000","last_fullsized":"2023-10-21T16:17:27.369861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:27.688997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369406+0000","last_change":"2023-10-21T16:17:26.287319+0000","last_active":"2023-10-21T16:17:27.369406+0000","last_peered":"2023-10-21T16:17:27.369406+0000","last_clean":"2023-10-21T16:17:27.369406+0000","last_became_active":"2023-10-21T16:17:26.286998+0000","last_became_peered":"2023-10-21T16:17:26.286998+0000","last_unstale":"2023-10-21T16:17:27.369406+0000","last_undegraded":"2023-10-21T16:17:27.369406+0000","last_fullsized":"2023-10-21T16:17:27.369406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:57.862665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302726+0000","last_change":"2023-10-21T16:17:26.275379+0000","last_active":"2023-10-21T16:17:27.302726+0000","last_peered":"2023-10-21T16:17:27.302726+0000","last_clean":"2023-10-21T16:17:27.302726+0000","last_became_active":"2023-10-21T16:17:26.275027+0000","last_became_peered":"2023-10-21T16:17:26.275027+0000","last_unstale":"2023-10-21T16:17:27.302726+0000","last_undegraded":"2023-10-21T16:17:27.302726+0000","last_fullsized":"2023-10-21T16:17:27.302726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:10.163662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367026+0000","last_change":"2023-10-21T16:17:26.291226+0000","last_active":"2023-10-21T16:17:27.367026+0000","last_peered":"2023-10-21T16:17:27.367026+0000","last_clean":"2023-10-21T16:17:27.367026+0000","last_became_active":"2023-10-21T16:17:26.290932+0000","last_became_peered":"2023-10-21T16:17:26.290932+0000","last_unstale":"2023-10-21T16:17:27.367026+0000","last_undegraded":"2023-10-21T16:17:27.367026+0000","last_fullsized":"2023-10-21T16:17:27.367026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:15:20.884000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365898+0000","last_change":"2023-10-21T16:17:26.291057+0000","last_active":"2023-10-21T16:17:27.365898+0000","last_peered":"2023-10-21T16:17:27.365898+0000","last_clean":"2023-10-21T16:17:27.365898+0000","last_became_active":"2023-10-21T16:17:26.290005+0000","last_became_peered":"2023-10-21T16:17:26.290005+0000","last_unstale":"2023-10-21T16:17:27.365898+0000","last_undegraded":"2023-10-21T16:17:27.365898+0000","last_fullsized":"2023-10-21T16:17:27.365898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:19:42.094980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367125+0000","last_change":"2023-10-21T16:17:26.287655+0000","last_active":"2023-10-21T16:17:27.367125+0000","last_peered":"2023-10-21T16:17:27.367125+0000","last_clean":"2023-10-21T16:17:27.367125+0000","last_became_active":"2023-10-21T16:17:26.286481+0000","last_became_peered":"2023-10-21T16:17:26.286481+0000","last_unstale":"2023-10-21T16:17:27.367125+0000","last_undegraded":"2023-10-21T16:17:27.367125+0000","last_fullsized":"2023-10-21T16:17:27.367125+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:50:03.302733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301250+0000","last_change":"2023-10-21T16:17:26.274895+0000","last_active":"2023-10-21T16:17:27.301250+0000","last_peered":"2023-10-21T16:17:27.301250+0000","last_clean":"2023-10-21T16:17:27.301250+0000","last_became_active":"2023-10-21T16:17:26.274437+0000","last_became_peered":"2023-10-21T16:17:26.274437+0000","last_unstale":"2023-10-21T16:17:27.301250+0000","last_undegraded":"2023-10-21T16:17:27.301250+0000","last_fullsized":"2023-10-21T16:17:27.301250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:54.820486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365352+0000","last_change":"2023-10-21T16:17:26.295127+0000","last_active":"2023-10-21T16:17:27.365352+0000","last_peered":"2023-10-21T16:17:27.365352+0000","last_clean":"2023-10-21T16:17:27.365352+0000","last_became_active":"2023-10-21T16:17:26.289019+0000","last_became_peered":"2023-10-21T16:17:26.289019+0000","last_unstale":"2023-10-21T16:17:27.365352+0000","last_undegraded":"2023-10-21T16:17:27.365352+0000","last_fullsized":"2023-10-21T16:17:27.365352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:03.828384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369142+0000","last_change":"2023-10-21T16:17:26.297339+0000","last_active":"2023-10-21T16:17:27.369142+0000","last_peered":"2023-10-21T16:17:27.369142+0000","last_clean":"2023-10-21T16:17:27.369142+0000","last_became_active":"2023-10-21T16:17:26.297016+0000","last_became_peered":"2023-10-21T16:17:26.297016+0000","last_unstale":"2023-10-21T16:17:27.369142+0000","last_undegraded":"2023-10-21T16:17:27.369142+0000","last_fullsized":"2023-10-21T16:17:27.369142+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:07.189884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"89'4","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367489+0000","last_change":"2023-10-21T16:17:25.286504+0000","last_active":"2023-10-21T16:17:27.367489+0000","last_peered":"2023-10-21T16:17:27.367489+0000","last_clean":"2023-10-21T16:17:27.367489+0000","last_became_active":"2023-10-21T16:17:25.286181+0000","last_became_peered":"2023-10-21T16:17:25.286181+0000","last_unstale":"2023-10-21T16:17:27.367489+0000","last_undegraded":"2023-10-21T16:17:27.367489+0000","last_fullsized":"2023-10-21T16:17:27.367489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:11.332814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302750+0000","last_change":"2023-10-21T16:17:26.296899+0000","last_active":"2023-10-21T16:17:27.302750+0000","last_peered":"2023-10-21T16:17:27.302750+0000","last_clean":"2023-10-21T16:17:27.302750+0000","last_became_active":"2023-10-21T16:17:26.296644+0000","last_became_peered":"2023-10-21T16:17:26.296644+0000","last_unstale":"2023-10-21T16:17:27.302750+0000","last_undegraded":"2023-10-21T16:17:27.302750+0000","last_fullsized":"2023-10-21T16:17:27.302750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:38:41.769284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367052+0000","last_change":"2023-10-21T16:17:25.278363+0000","last_active":"2023-10-21T16:17:27.367052+0000","last_peered":"2023-10-21T16:17:27.367052+0000","last_clean":"2023-10-21T16:17:27.367052+0000","last_became_active":"2023-10-21T16:17:25.278046+0000","last_became_peered":"2023-10-21T16:17:25.278046+0000","last_unstale":"2023-10-21T16:17:27.367052+0000","last_undegraded":"2023-10-21T16:17:27.367052+0000","last_fullsized":"2023-10-21T16:17:27.367052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:23.896347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367115+0000","last_change":"2023-10-21T16:17:26.297169+0000","last_active":"2023-10-21T16:17:27.367115+0000","last_peered":"2023-10-21T16:17:27.367115+0000","last_clean":"2023-10-21T16:17:27.367115+0000","last_became_active":"2023-10-21T16:17:26.296022+0000","last_became_peered":"2023-10-21T16:17:26.296022+0000","last_unstale":"2023-10-21T16:17:27.367115+0000","last_undegraded":"2023-10-21T16:17:27.367115+0000","last_fullsized":"2023-10-21T16:17:27.367115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:11.774948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365803+0000","last_change":"2023-10-21T16:17:25.282374+0000","last_active":"2023-10-21T16:17:27.365803+0000","last_peered":"2023-10-21T16:17:27.365803+0000","last_clean":"2023-10-21T16:17:27.365803+0000","last_became_active":"2023-10-21T16:17:25.281843+0000","last_became_peered":"2023-10-21T16:17:25.281843+0000","last_unstale":"2023-10-21T16:17:27.365803+0000","last_undegraded":"2023-10-21T16:17:27.365803+0000","last_fullsized":"2023-10-21T16:17:27.365803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:05.347647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366935+0000","last_change":"2023-10-21T16:17:26.295334+0000","last_active":"2023-10-21T16:17:27.366935+0000","last_peered":"2023-10-21T16:17:27.366935+0000","last_clean":"2023-10-21T16:17:27.366935+0000","last_became_active":"2023-10-21T16:17:26.295059+0000","last_became_peered":"2023-10-21T16:17:26.295059+0000","last_unstale":"2023-10-21T16:17:27.366935+0000","last_undegraded":"2023-10-21T16:17:27.366935+0000","last_fullsized":"2023-10-21T16:17:27.366935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:00.941402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365312+0000","last_change":"2023-10-21T16:17:25.274273+0000","last_active":"2023-10-21T16:17:27.365312+0000","last_peered":"2023-10-21T16:17:27.365312+0000","last_clean":"2023-10-21T16:17:27.365312+0000","last_became_active":"2023-10-21T16:17:25.273950+0000","last_became_peered":"2023-10-21T16:17:25.273950+0000","last_unstale":"2023-10-21T16:17:27.365312+0000","last_undegraded":"2023-10-21T16:17:27.365312+0000","last_fullsized":"2023-10-21T16:17:27.365312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:19:40.938948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299289+0000","last_change":"2023-10-21T16:17:26.296527+0000","last_active":"2023-10-21T16:17:27.299289+0000","last_peered":"2023-10-21T16:17:27.299289+0000","last_clean":"2023-10-21T16:17:27.299289+0000","last_became_active":"2023-10-21T16:17:26.294932+0000","last_became_peered":"2023-10-21T16:17:26.294932+0000","last_unstale":"2023-10-21T16:17:27.299289+0000","last_undegraded":"2023-10-21T16:17:27.299289+0000","last_fullsized":"2023-10-21T16:17:27.299289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:32.534067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297540+0000","last_change":"2023-10-21T16:17:25.269792+0000","last_active":"2023-10-21T16:17:27.297540+0000","last_peered":"2023-10-21T16:17:27.297540+0000","last_clean":"2023-10-21T16:17:27.297540+0000","last_became_active":"2023-10-21T16:17:25.269492+0000","last_became_peered":"2023-10-21T16:17:25.269492+0000","last_unstale":"2023-10-21T16:17:27.297540+0000","last_undegraded":"2023-10-21T16:17:27.297540+0000","last_fullsized":"2023-10-21T16:17:27.297540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:32.180154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365793+0000","last_change":"2023-10-21T16:17:26.292080+0000","last_active":"2023-10-21T16:17:27.365793+0000","last_peered":"2023-10-21T16:17:27.365793+0000","last_clean":"2023-10-21T16:17:27.365793+0000","last_became_active":"2023-10-21T16:17:26.290984+0000","last_became_peered":"2023-10-21T16:17:26.290984+0000","last_unstale":"2023-10-21T16:17:27.365793+0000","last_undegraded":"2023-10-21T16:17:27.365793+0000","last_fullsized":"2023-10-21T16:17:27.365793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:29.377919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297218+0000","last_change":"2023-10-21T16:17:25.267715+0000","last_active":"2023-10-21T16:17:27.297218+0000","last_peered":"2023-10-21T16:17:27.297218+0000","last_clean":"2023-10-21T16:17:27.297218+0000","last_became_active":"2023-10-21T16:17:25.267382+0000","last_became_peered":"2023-10-21T16:17:25.267382+0000","last_unstale":"2023-10-21T16:17:27.297218+0000","last_undegraded":"2023-10-21T16:17:27.297218+0000","last_fullsized":"2023-10-21T16:17:27.297218+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:43:33.147295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366244+0000","last_change":"2023-10-21T16:17:26.296805+0000","last_active":"2023-10-21T16:17:27.366244+0000","last_peered":"2023-10-21T16:17:27.366244+0000","last_clean":"2023-10-21T16:17:27.366244+0000","last_became_active":"2023-10-21T16:17:26.295768+0000","last_became_peered":"2023-10-21T16:17:26.295768+0000","last_unstale":"2023-10-21T16:17:27.366244+0000","last_undegraded":"2023-10-21T16:17:27.366244+0000","last_fullsized":"2023-10-21T16:17:27.366244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:10:51.308191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297252+0000","last_change":"2023-10-21T16:17:25.285931+0000","last_active":"2023-10-21T16:17:27.297252+0000","last_peered":"2023-10-21T16:17:27.297252+0000","last_clean":"2023-10-21T16:17:27.297252+0000","last_became_active":"2023-10-21T16:17:25.284775+0000","last_became_peered":"2023-10-21T16:17:25.284775+0000","last_unstale":"2023-10-21T16:17:27.297252+0000","last_undegraded":"2023-10-21T16:17:27.297252+0000","last_fullsized":"2023-10-21T16:17:27.297252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:17.712588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297307+0000","last_change":"2023-10-21T16:17:26.275150+0000","last_active":"2023-10-21T16:17:27.297307+0000","last_peered":"2023-10-21T16:17:27.297307+0000","last_clean":"2023-10-21T16:17:27.297307+0000","last_became_active":"2023-10-21T16:17:26.273956+0000","last_became_peered":"2023-10-21T16:17:26.273956+0000","last_unstale":"2023-10-21T16:17:27.297307+0000","last_undegraded":"2023-10-21T16:17:27.297307+0000","last_fullsized":"2023-10-21T16:17:27.297307+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:05:56.193212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366469+0000","last_change":"2023-10-21T16:17:25.284798+0000","last_active":"2023-10-21T16:17:27.366469+0000","last_peered":"2023-10-21T16:17:27.366469+0000","last_clean":"2023-10-21T16:17:27.366469+0000","last_became_active":"2023-10-21T16:17:25.284489+0000","last_became_peered":"2023-10-21T16:17:25.284489+0000","last_unstale":"2023-10-21T16:17:27.366469+0000","last_undegraded":"2023-10-21T16:17:27.366469+0000","last_fullsized":"2023-10-21T16:17:27.366469+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:15.337252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366478+0000","last_change":"2023-10-21T16:17:26.286690+0000","last_active":"2023-10-21T16:17:27.366478+0000","last_peered":"2023-10-21T16:17:27.366478+0000","last_clean":"2023-10-21T16:17:27.366478+0000","last_became_active":"2023-10-21T16:17:26.286372+0000","last_became_peered":"2023-10-21T16:17:26.286372+0000","last_unstale":"2023-10-21T16:17:27.366478+0000","last_undegraded":"2023-10-21T16:17:27.366478+0000","last_fullsized":"2023-10-21T16:17:27.366478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:50:49.380985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367776+0000","last_change":"2023-10-21T16:17:25.288345+0000","last_active":"2023-10-21T16:17:27.367776+0000","last_peered":"2023-10-21T16:17:27.367776+0000","last_clean":"2023-10-21T16:17:27.367776+0000","last_became_active":"2023-10-21T16:17:25.287827+0000","last_became_peered":"2023-10-21T16:17:25.287827+0000","last_unstale":"2023-10-21T16:17:27.367776+0000","last_undegraded":"2023-10-21T16:17:27.367776+0000","last_fullsized":"2023-10-21T16:17:27.367776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:48.696560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297583+0000","last_change":"2023-10-21T16:17:26.285078+0000","last_active":"2023-10-21T16:17:27.297583+0000","last_peered":"2023-10-21T16:17:27.297583+0000","last_clean":"2023-10-21T16:17:27.297583+0000","last_became_active":"2023-10-21T16:17:26.284773+0000","last_became_peered":"2023-10-21T16:17:26.284773+0000","last_unstale":"2023-10-21T16:17:27.297583+0000","last_undegraded":"2023-10-21T16:17:27.297583+0000","last_fullsized":"2023-10-21T16:17:27.297583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:05.933204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302901+0000","last_change":"2023-10-21T16:17:25.277413+0000","last_active":"2023-10-21T16:17:27.302901+0000","last_peered":"2023-10-21T16:17:27.302901+0000","last_clean":"2023-10-21T16:17:27.302901+0000","last_became_active":"2023-10-21T16:17:25.276613+0000","last_became_peered":"2023-10-21T16:17:25.276613+0000","last_unstale":"2023-10-21T16:17:27.302901+0000","last_undegraded":"2023-10-21T16:17:27.302901+0000","last_fullsized":"2023-10-21T16:17:27.302901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:40:21.018413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368789+0000","last_change":"2023-10-21T16:17:26.293676+0000","last_active":"2023-10-21T16:17:27.368789+0000","last_peered":"2023-10-21T16:17:27.368789+0000","last_clean":"2023-10-21T16:17:27.368789+0000","last_became_active":"2023-10-21T16:17:26.292144+0000","last_became_peered":"2023-10-21T16:17:26.292144+0000","last_unstale":"2023-10-21T16:17:27.368789+0000","last_undegraded":"2023-10-21T16:17:27.368789+0000","last_fullsized":"2023-10-21T16:17:27.368789+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:54:23.901665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"89'3","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366557+0000","last_change":"2023-10-21T16:17:25.288121+0000","last_active":"2023-10-21T16:17:27.366557+0000","last_peered":"2023-10-21T16:17:27.366557+0000","last_clean":"2023-10-21T16:17:27.366557+0000","last_became_active":"2023-10-21T16:17:25.287822+0000","last_became_peered":"2023-10-21T16:17:25.287822+0000","last_unstale":"2023-10-21T16:17:27.366557+0000","last_undegraded":"2023-10-21T16:17:27.366557+0000","last_fullsized":"2023-10-21T16:17:27.366557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:11.234181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.303317+0000","last_change":"2023-10-21T16:17:26.273598+0000","last_active":"2023-10-21T16:17:27.303317+0000","last_peered":"2023-10-21T16:17:27.303317+0000","last_clean":"2023-10-21T16:17:27.303317+0000","last_became_active":"2023-10-21T16:17:26.272982+0000","last_became_peered":"2023-10-21T16:17:26.272982+0000","last_unstale":"2023-10-21T16:17:27.303317+0000","last_undegraded":"2023-10-21T16:17:27.303317+0000","last_fullsized":"2023-10-21T16:17:27.303317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:12.347820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"89'6","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366577+0000","last_change":"2023-10-21T16:17:25.280566+0000","last_active":"2023-10-21T16:17:27.366577+0000","last_peered":"2023-10-21T16:17:27.366577+0000","last_clean":"2023-10-21T16:17:27.366577+0000","last_became_active":"2023-10-21T16:17:25.280253+0000","last_became_peered":"2023-10-21T16:17:25.280253+0000","last_unstale":"2023-10-21T16:17:27.366577+0000","last_undegraded":"2023-10-21T16:17:27.366577+0000","last_fullsized":"2023-10-21T16:17:27.366577+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:59.469227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301192+0000","last_change":"2023-10-21T16:17:26.292447+0000","last_active":"2023-10-21T16:17:27.301192+0000","last_peered":"2023-10-21T16:17:27.301192+0000","last_clean":"2023-10-21T16:17:27.301192+0000","last_became_active":"2023-10-21T16:17:26.292164+0000","last_became_peered":"2023-10-21T16:17:26.292164+0000","last_unstale":"2023-10-21T16:17:27.301192+0000","last_undegraded":"2023-10-21T16:17:27.301192+0000","last_fullsized":"2023-10-21T16:17:27.301192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:58.276482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366146+0000","last_change":"2023-10-21T16:17:25.288172+0000","last_active":"2023-10-21T16:17:27.366146+0000","last_peered":"2023-10-21T16:17:27.366146+0000","last_clean":"2023-10-21T16:17:27.366146+0000","last_became_active":"2023-10-21T16:17:25.287777+0000","last_became_peered":"2023-10-21T16:17:25.287777+0000","last_unstale":"2023-10-21T16:17:27.366146+0000","last_undegraded":"2023-10-21T16:17:27.366146+0000","last_fullsized":"2023-10-21T16:17:27.366146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:05.347647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367131+0000","last_change":"2023-10-21T16:17:26.285481+0000","last_active":"2023-10-21T16:17:27.367131+0000","last_peered":"2023-10-21T16:17:27.367131+0000","last_clean":"2023-10-21T16:17:27.367131+0000","last_became_active":"2023-10-21T16:17:26.285137+0000","last_became_peered":"2023-10-21T16:17:26.285137+0000","last_unstale":"2023-10-21T16:17:27.367131+0000","last_undegraded":"2023-10-21T16:17:27.367131+0000","last_fullsized":"2023-10-21T16:17:27.367131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:36:37.094947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367934+0000","last_change":"2023-10-21T16:17:25.275674+0000","last_active":"2023-10-21T16:17:27.367934+0000","last_peered":"2023-10-21T16:17:27.367934+0000","last_clean":"2023-10-21T16:17:27.367934+0000","last_became_active":"2023-10-21T16:17:25.274886+0000","last_became_peered":"2023-10-21T16:17:25.274886+0000","last_unstale":"2023-10-21T16:17:27.367934+0000","last_undegraded":"2023-10-21T16:17:27.367934+0000","last_fullsized":"2023-10-21T16:17:27.367934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:11.416665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366620+0000","last_change":"2023-10-21T16:17:26.290817+0000","last_active":"2023-10-21T16:17:27.366620+0000","last_peered":"2023-10-21T16:17:27.366620+0000","last_clean":"2023-10-21T16:17:27.366620+0000","last_became_active":"2023-10-21T16:17:26.290537+0000","last_became_peered":"2023-10-21T16:17:26.290537+0000","last_unstale":"2023-10-21T16:17:27.366620+0000","last_undegraded":"2023-10-21T16:17:27.366620+0000","last_fullsized":"2023-10-21T16:17:27.366620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:53.559520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366480+0000","last_change":"2023-10-21T16:17:25.284832+0000","last_active":"2023-10-21T16:17:27.366480+0000","last_peered":"2023-10-21T16:17:27.366480+0000","last_clean":"2023-10-21T16:17:27.366480+0000","last_became_active":"2023-10-21T16:17:25.284544+0000","last_became_peered":"2023-10-21T16:17:25.284544+0000","last_unstale":"2023-10-21T16:17:27.366480+0000","last_undegraded":"2023-10-21T16:17:27.366480+0000","last_fullsized":"2023-10-21T16:17:27.366480+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:15.254800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366598+0000","last_change":"2023-10-21T16:17:26.300244+0000","last_active":"2023-10-21T16:17:27.366598+0000","last_peered":"2023-10-21T16:17:27.366598+0000","last_clean":"2023-10-21T16:17:27.366598+0000","last_became_active":"2023-10-21T16:17:26.299962+0000","last_became_peered":"2023-10-21T16:17:26.299962+0000","last_unstale":"2023-10-21T16:17:27.366598+0000","last_undegraded":"2023-10-21T16:17:27.366598+0000","last_fullsized":"2023-10-21T16:17:27.366598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:52.423857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"89'8","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301050+0000","last_change":"2023-10-21T16:17:25.287009+0000","last_active":"2023-10-21T16:17:27.301050+0000","last_peered":"2023-10-21T16:17:27.301050+0000","last_clean":"2023-10-21T16:17:27.301050+0000","last_became_active":"2023-10-21T16:17:25.285744+0000","last_became_peered":"2023-10-21T16:17:25.285744+0000","last_unstale":"2023-10-21T16:17:27.301050+0000","last_undegraded":"2023-10-21T16:17:27.301050+0000","last_fullsized":"2023-10-21T16:17:27.301050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:16.827311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365759+0000","last_change":"2023-10-21T16:17:26.284856+0000","last_active":"2023-10-21T16:17:27.365759+0000","last_peered":"2023-10-21T16:17:27.365759+0000","last_clean":"2023-10-21T16:17:27.365759+0000","last_became_active":"2023-10-21T16:17:26.284552+0000","last_became_peered":"2023-10-21T16:17:26.284552+0000","last_unstale":"2023-10-21T16:17:27.365759+0000","last_undegraded":"2023-10-21T16:17:27.365759+0000","last_fullsized":"2023-10-21T16:17:27.365759+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:11.774948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369915+0000","last_change":"2023-10-21T16:17:25.278013+0000","last_active":"2023-10-21T16:17:27.369915+0000","last_peered":"2023-10-21T16:17:27.369915+0000","last_clean":"2023-10-21T16:17:27.369915+0000","last_became_active":"2023-10-21T16:17:25.277678+0000","last_became_peered":"2023-10-21T16:17:25.277678+0000","last_unstale":"2023-10-21T16:17:27.369915+0000","last_undegraded":"2023-10-21T16:17:27.369915+0000","last_fullsized":"2023-10-21T16:17:27.369915+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:19:41.079975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369161+0000","last_change":"2023-10-21T16:17:26.298784+0000","last_active":"2023-10-21T16:17:27.369161+0000","last_peered":"2023-10-21T16:17:27.369161+0000","last_clean":"2023-10-21T16:17:27.369161+0000","last_became_active":"2023-10-21T16:17:26.298301+0000","last_became_peered":"2023-10-21T16:17:26.298301+0000","last_unstale":"2023-10-21T16:17:27.369161+0000","last_undegraded":"2023-10-21T16:17:27.369161+0000","last_fullsized":"2023-10-21T16:17:27.369161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:40:22.033419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367318+0000","last_change":"2023-10-21T16:17:25.289816+0000","last_active":"2023-10-21T16:17:27.367318+0000","last_peered":"2023-10-21T16:17:27.367318+0000","last_clean":"2023-10-21T16:17:27.367318+0000","last_became_active":"2023-10-21T16:17:25.289595+0000","last_became_peered":"2023-10-21T16:17:25.289595+0000","last_unstale":"2023-10-21T16:17:27.367318+0000","last_undegraded":"2023-10-21T16:17:27.367318+0000","last_fullsized":"2023-10-21T16:17:27.367318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:23.896347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.302942+0000","last_change":"2023-10-21T16:17:26.299750+0000","last_active":"2023-10-21T16:17:27.302942+0000","last_peered":"2023-10-21T16:17:27.302942+0000","last_clean":"2023-10-21T16:17:27.302942+0000","last_became_active":"2023-10-21T16:17:26.299483+0000","last_became_peered":"2023-10-21T16:17:26.299483+0000","last_unstale":"2023-10-21T16:17:27.302942+0000","last_undegraded":"2023-10-21T16:17:27.302942+0000","last_fullsized":"2023-10-21T16:17:27.302942+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:20.551598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"89'1","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366267+0000","last_change":"2023-10-21T16:17:25.286921+0000","last_active":"2023-10-21T16:17:27.366267+0000","last_peered":"2023-10-21T16:17:27.366267+0000","last_clean":"2023-10-21T16:17:27.366267+0000","last_became_active":"2023-10-21T16:17:25.286653+0000","last_became_peered":"2023-10-21T16:17:25.286653+0000","last_unstale":"2023-10-21T16:17:27.366267+0000","last_undegraded":"2023-10-21T16:17:27.366267+0000","last_fullsized":"2023-10-21T16:17:27.366267+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:34.072286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368575+0000","last_change":"2023-10-21T16:17:26.295660+0000","last_active":"2023-10-21T16:17:27.368575+0000","last_peered":"2023-10-21T16:17:27.368575+0000","last_clean":"2023-10-21T16:17:27.368575+0000","last_became_active":"2023-10-21T16:17:26.295312+0000","last_became_peered":"2023-10-21T16:17:26.295312+0000","last_unstale":"2023-10-21T16:17:27.368575+0000","last_undegraded":"2023-10-21T16:17:27.368575+0000","last_fullsized":"2023-10-21T16:17:27.368575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:06:08.557188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297680+0000","last_change":"2023-10-21T16:17:25.276983+0000","last_active":"2023-10-21T16:17:27.297680+0000","last_peered":"2023-10-21T16:17:27.297680+0000","last_clean":"2023-10-21T16:17:27.297680+0000","last_became_active":"2023-10-21T16:17:25.276601+0000","last_became_peered":"2023-10-21T16:17:25.276601+0000","last_unstale":"2023-10-21T16:17:27.297680+0000","last_undegraded":"2023-10-21T16:17:27.297680+0000","last_fullsized":"2023-10-21T16:17:27.297680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:16.744447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.364811+0000","last_change":"2023-10-21T16:17:26.292644+0000","last_active":"2023-10-21T16:17:27.364811+0000","last_peered":"2023-10-21T16:17:27.364811+0000","last_clean":"2023-10-21T16:17:27.364811+0000","last_became_active":"2023-10-21T16:17:26.292322+0000","last_became_peered":"2023-10-21T16:17:26.292322+0000","last_unstale":"2023-10-21T16:17:27.364811+0000","last_undegraded":"2023-10-21T16:17:27.364811+0000","last_fullsized":"2023-10-21T16:17:27.364811+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:49.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.364963+0000","last_change":"2023-10-21T16:17:25.284830+0000","last_active":"2023-10-21T16:17:27.364963+0000","last_peered":"2023-10-21T16:17:27.364963+0000","last_clean":"2023-10-21T16:17:27.364963+0000","last_became_active":"2023-10-21T16:17:25.284537+0000","last_became_peered":"2023-10-21T16:17:25.284537+0000","last_unstale":"2023-10-21T16:17:27.364963+0000","last_undegraded":"2023-10-21T16:17:27.364963+0000","last_fullsized":"2023-10-21T16:17:27.364963+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:52:21.696353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.367496+0000","last_change":"2023-10-21T16:17:26.282406+0000","last_active":"2023-10-21T16:17:27.367496+0000","last_peered":"2023-10-21T16:17:27.367496+0000","last_clean":"2023-10-21T16:17:27.367496+0000","last_became_active":"2023-10-21T16:17:26.281920+0000","last_became_peered":"2023-10-21T16:17:26.281920+0000","last_unstale":"2023-10-21T16:17:27.367496+0000","last_undegraded":"2023-10-21T16:17:27.367496+0000","last_fullsized":"2023-10-21T16:17:27.367496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:44.434177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369546+0000","last_change":"2023-10-21T16:17:25.273758+0000","last_active":"2023-10-21T16:17:27.369546+0000","last_peered":"2023-10-21T16:17:27.369546+0000","last_clean":"2023-10-21T16:17:27.369546+0000","last_became_active":"2023-10-21T16:17:25.273285+0000","last_became_peered":"2023-10-21T16:17:25.273285+0000","last_unstale":"2023-10-21T16:17:27.369546+0000","last_undegraded":"2023-10-21T16:17:27.369546+0000","last_fullsized":"2023-10-21T16:17:27.369546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:38.570991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368005+0000","last_change":"2023-10-21T16:17:26.290724+0000","last_active":"2023-10-21T16:17:27.368005+0000","last_peered":"2023-10-21T16:17:27.368005+0000","last_clean":"2023-10-21T16:17:27.368005+0000","last_became_active":"2023-10-21T16:17:26.290421+0000","last_became_peered":"2023-10-21T16:17:26.290421+0000","last_unstale":"2023-10-21T16:17:27.368005+0000","last_undegraded":"2023-10-21T16:17:27.368005+0000","last_fullsized":"2023-10-21T16:17:27.368005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:00.858570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365946+0000","last_change":"2023-10-21T16:17:25.280584+0000","last_active":"2023-10-21T16:17:27.365946+0000","last_peered":"2023-10-21T16:17:27.365946+0000","last_clean":"2023-10-21T16:17:27.365946+0000","last_became_active":"2023-10-21T16:17:25.280267+0000","last_became_peered":"2023-10-21T16:17:25.280267+0000","last_unstale":"2023-10-21T16:17:27.365946+0000","last_undegraded":"2023-10-21T16:17:27.365946+0000","last_fullsized":"2023-10-21T16:17:27.365946+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:07.790899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365219+0000","last_change":"2023-10-21T16:17:26.289215+0000","last_active":"2023-10-21T16:17:27.365219+0000","last_peered":"2023-10-21T16:17:27.365219+0000","last_clean":"2023-10-21T16:17:27.365219+0000","last_became_active":"2023-10-21T16:17:26.288885+0000","last_became_peered":"2023-10-21T16:17:26.288885+0000","last_unstale":"2023-10-21T16:17:27.365219+0000","last_undegraded":"2023-10-21T16:17:27.365219+0000","last_fullsized":"2023-10-21T16:17:27.365219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:16.269805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"89'2","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.370044+0000","last_change":"2023-10-21T16:17:25.286175+0000","last_active":"2023-10-21T16:17:27.370044+0000","last_peered":"2023-10-21T16:17:27.370044+0000","last_clean":"2023-10-21T16:17:27.370044+0000","last_became_active":"2023-10-21T16:17:25.285793+0000","last_became_peered":"2023-10-21T16:17:25.285793+0000","last_unstale":"2023-10-21T16:17:27.370044+0000","last_undegraded":"2023-10-21T16:17:27.370044+0000","last_fullsized":"2023-10-21T16:17:27.370044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:27.957248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299250+0000","last_change":"2023-10-21T16:17:26.281191+0000","last_active":"2023-10-21T16:17:27.299250+0000","last_peered":"2023-10-21T16:17:27.299250+0000","last_clean":"2023-10-21T16:17:27.299250+0000","last_became_active":"2023-10-21T16:17:26.280828+0000","last_became_peered":"2023-10-21T16:17:26.280828+0000","last_unstale":"2023-10-21T16:17:27.299250+0000","last_undegraded":"2023-10-21T16:17:27.299250+0000","last_fullsized":"2023-10-21T16:17:27.299250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:20:12.571924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369867+0000","last_change":"2023-10-21T16:17:25.278680+0000","last_active":"2023-10-21T16:17:27.369867+0000","last_peered":"2023-10-21T16:17:27.369867+0000","last_clean":"2023-10-21T16:17:27.369867+0000","last_became_active":"2023-10-21T16:17:25.276902+0000","last_became_peered":"2023-10-21T16:17:25.276902+0000","last_unstale":"2023-10-21T16:17:27.369867+0000","last_undegraded":"2023-10-21T16:17:27.369867+0000","last_fullsized":"2023-10-21T16:17:27.369867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:39.891079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.369281+0000","last_change":"2023-10-21T16:17:26.287903+0000","last_active":"2023-10-21T16:17:27.369281+0000","last_peered":"2023-10-21T16:17:27.369281+0000","last_clean":"2023-10-21T16:17:27.369281+0000","last_became_active":"2023-10-21T16:17:26.287333+0000","last_became_peered":"2023-10-21T16:17:26.287333+0000","last_unstale":"2023-10-21T16:17:27.369281+0000","last_undegraded":"2023-10-21T16:17:27.369281+0000","last_fullsized":"2023-10-21T16:17:27.369281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:40.063531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365829+0000","last_change":"2023-10-21T16:17:25.279368+0000","last_active":"2023-10-21T16:17:27.365829+0000","last_peered":"2023-10-21T16:17:27.365829+0000","last_clean":"2023-10-21T16:17:27.365829+0000","last_became_active":"2023-10-21T16:17:25.278597+0000","last_became_peered":"2023-10-21T16:17:25.278597+0000","last_unstale":"2023-10-21T16:17:27.365829+0000","last_undegraded":"2023-10-21T16:17:27.365829+0000","last_fullsized":"2023-10-21T16:17:27.365829+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:11.234181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368985+0000","last_change":"2023-10-21T16:17:26.297310+0000","last_active":"2023-10-21T16:17:27.368985+0000","last_peered":"2023-10-21T16:17:27.368985+0000","last_clean":"2023-10-21T16:17:27.368985+0000","last_became_active":"2023-10-21T16:17:26.296995+0000","last_became_peered":"2023-10-21T16:17:26.296995+0000","last_unstale":"2023-10-21T16:17:27.368985+0000","last_undegraded":"2023-10-21T16:17:27.368985+0000","last_fullsized":"2023-10-21T16:17:27.368985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:56.406047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366671+0000","last_change":"2023-10-21T16:17:25.282036+0000","last_active":"2023-10-21T16:17:27.366671+0000","last_peered":"2023-10-21T16:17:27.366671+0000","last_clean":"2023-10-21T16:17:27.366671+0000","last_became_active":"2023-10-21T16:17:25.281693+0000","last_became_peered":"2023-10-21T16:17:25.281693+0000","last_unstale":"2023-10-21T16:17:27.366671+0000","last_undegraded":"2023-10-21T16:17:27.366671+0000","last_fullsized":"2023-10-21T16:17:27.366671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:19.536592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366210+0000","last_change":"2023-10-21T16:17:26.295607+0000","last_active":"2023-10-21T16:17:27.366210+0000","last_peered":"2023-10-21T16:17:27.366210+0000","last_clean":"2023-10-21T16:17:27.366210+0000","last_became_active":"2023-10-21T16:17:26.295195+0000","last_became_peered":"2023-10-21T16:17:26.295195+0000","last_unstale":"2023-10-21T16:17:27.366210+0000","last_undegraded":"2023-10-21T16:17:27.366210+0000","last_fullsized":"2023-10-21T16:17:27.366210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:13.092858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.366421+0000","last_change":"2023-10-21T16:17:25.283721+0000","last_active":"2023-10-21T16:17:27.366421+0000","last_peered":"2023-10-21T16:17:27.366421+0000","last_clean":"2023-10-21T16:17:27.366421+0000","last_became_active":"2023-10-21T16:17:25.283420+0000","last_became_peered":"2023-10-21T16:17:25.283420+0000","last_unstale":"2023-10-21T16:17:27.366421+0000","last_undegraded":"2023-10-21T16:17:27.366421+0000","last_fullsized":"2023-10-21T16:17:27.366421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:07.790899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.301391+0000","last_change":"2023-10-21T16:17:26.283797+0000","last_active":"2023-10-21T16:17:27.301391+0000","last_peered":"2023-10-21T16:17:27.301391+0000","last_clean":"2023-10-21T16:17:27.301391+0000","last_became_active":"2023-10-21T16:17:26.283482+0000","last_became_peered":"2023-10-21T16:17:26.283482+0000","last_unstale":"2023-10-21T16:17:27.301391+0000","last_undegraded":"2023-10-21T16:17:27.301391+0000","last_fullsized":"2023-10-21T16:17:27.301391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:14.588110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297184+0000","last_change":"2023-10-21T16:17:25.271206+0000","last_active":"2023-10-21T16:17:27.297184+0000","last_peered":"2023-10-21T16:17:27.297184+0000","last_clean":"2023-10-21T16:17:27.297184+0000","last_became_active":"2023-10-21T16:17:25.270855+0000","last_became_peered":"2023-10-21T16:17:25.270855+0000","last_unstale":"2023-10-21T16:17:27.297184+0000","last_undegraded":"2023-10-21T16:17:27.297184+0000","last_fullsized":"2023-10-21T16:17:27.297184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:37.021942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365815+0000","last_change":"2023-10-21T16:17:26.286806+0000","last_active":"2023-10-21T16:17:27.365815+0000","last_peered":"2023-10-21T16:17:27.365815+0000","last_clean":"2023-10-21T16:17:27.365815+0000","last_became_active":"2023-10-21T16:17:26.286500+0000","last_became_peered":"2023-10-21T16:17:26.286500+0000","last_unstale":"2023-10-21T16:17:27.365815+0000","last_undegraded":"2023-10-21T16:17:27.365815+0000","last_fullsized":"2023-10-21T16:17:27.365815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:04:29.762951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.297136+0000","last_change":"2023-10-21T16:17:25.285562+0000","last_active":"2023-10-21T16:17:27.297136+0000","last_peered":"2023-10-21T16:17:27.297136+0000","last_clean":"2023-10-21T16:17:27.297136+0000","last_became_active":"2023-10-21T16:17:25.284965+0000","last_became_peered":"2023-10-21T16:17:25.284965+0000","last_unstale":"2023-10-21T16:17:27.297136+0000","last_undegraded":"2023-10-21T16:17:27.297136+0000","last_fullsized":"2023-10-21T16:17:27.297136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:04.256308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.299300+0000","last_change":"2023-10-21T16:17:26.293843+0000","last_active":"2023-10-21T16:17:27.299300+0000","last_peered":"2023-10-21T16:17:27.299300+0000","last_clean":"2023-10-21T16:17:27.299300+0000","last_became_active":"2023-10-21T16:17:26.286543+0000","last_became_peered":"2023-10-21T16:17:26.286543+0000","last_unstale":"2023-10-21T16:17:27.299300+0000","last_undegraded":"2023-10-21T16:17:27.299300+0000","last_fullsized":"2023-10-21T16:17:27.299300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:24.054686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.365324+0000","last_change":"2023-10-21T16:17:25.287677+0000","last_active":"2023-10-21T16:17:27.365324+0000","last_peered":"2023-10-21T16:17:27.365324+0000","last_clean":"2023-10-21T16:17:27.365324+0000","last_became_active":"2023-10-21T16:17:25.287459+0000","last_became_peered":"2023-10-21T16:17:25.287459+0000","last_unstale":"2023-10-21T16:17:27.365324+0000","last_undegraded":"2023-10-21T16:17:27.365324+0000","last_fullsized":"2023-10-21T16:17:27.365324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:37.878379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.368876+0000","last_change":"2023-10-21T16:17:26.298361+0000","last_active":"2023-10-21T16:17:27.368876+0000","last_peered":"2023-10-21T16:17:27.368876+0000","last_clean":"2023-10-21T16:17:27.368876+0000","last_became_active":"2023-10-21T16:17:26.296172+0000","last_became_peered":"2023-10-21T16:17:26.296172+0000","last_unstale":"2023-10-21T16:17:27.368876+0000","last_undegraded":"2023-10-21T16:17:27.368876+0000","last_fullsized":"2023-10-21T16:17:27.368876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:25.231208+0000","last_clean_scrub_stamp":"2023-10-21T16:17:25.231208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:17.842316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-21T16:17:27.296910+0000","last_change":"2023-10-21T16:17:25.271022+0000","last_active":"2023-10-21T16:17:27.296910+0000","last_peered":"2023-10-21T16:17:27.296910+0000","last_clean":"2023-10-21T16:17:27.296910+0000","last_became_active":"2023-10-21T16:17:25.270663+0000","last_became_peered":"2023-10-21T16:17:25.270663+0000","last_unstale":"2023-10-21T16:17:27.296910+0000","last_undegraded":"2023-10-21T16:17:27.296910+0000","last_fullsized":"2023-10-21T16:17:27.296910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:17:24.216203+0000","last_clean_scrub_stamp":"2023-10-21T16:17:24.216203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:26.673991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"78'161","reported_seq":387,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-21T16:17:26.277541+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:17:26.277541+0000","last_peered":"2023-10-21T16:17:26.277541+0000","last_clean":"2023-10-21T16:17:26.277541+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:17:26.277541+0000","last_undegraded":"2023-10-21T16:17:26.277541+0000","last_fullsized":"2023-10-21T16:17:26.277541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1703936,"data_stored":1143214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.65000000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5609999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.171},"min":{"1min":0.76200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4019999999999999,"5min":2.448,"15min":2.448},"last":0.98899999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78300000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5389999999999999,"5min":1.915,"15min":1.915},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.583,"5min":1.778,"15min":1.778},"last":0.875}]},{"osd":2,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.27,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5880000000000001,"5min":1.748,"15min":2.085},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2849999999999999,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.556,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.262}]},{"osd":3,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.2529999999999999},"min":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6040000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.85999999999999999,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.9570000000000001,"15min":2.0099999999999998},"last":1.3169999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3360000000000001,"15min":1.355},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":1.7350000000000001,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.423,"15min":1.4450000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70399999999999996},"max":{"1min":1.8360000000000001,"5min":2.355,"15min":2.355},"last":1.3540000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.391,"15min":1.3999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.716,"5min":1.865,"15min":2.258},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.4350000000000001,"15min":1.4379999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.60899999999999999},"max":{"1min":1.9079999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.4019999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.53,"15min":1.548},"min":{"1min":1.0580000000000001,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":1.706,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.506},{"interface":"front","average":{"1min":1.47,"5min":1.4790000000000001,"15min":1.466},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7709999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.5940000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1097728,"data_stored":552104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.68799999999999994,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.514,"5min":1.895,"15min":2.4350000000000001},"last":1.4330000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.417,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.3480000000000001,"15min":1.306},"min":{"1min":0.90700000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5960000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.92700000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.895,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.099},{"interface":"front","average":{"1min":1.24,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6839999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.2470000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.377,"15min":1.3740000000000001},"min":{"1min":1.1200000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7490000000000001,"5min":2.2000000000000002,"15min":3.46},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.446,"15min":1.4299999999999999},"min":{"1min":1.204,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":1.9970000000000001,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.335}]},{"osd":4,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.393,"15min":1.365},"min":{"1min":0.94599999999999995,"5min":0.82999999999999996,"15min":0.76300000000000001},"max":{"1min":1.9379999999999999,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.401,"15min":1.397},"min":{"1min":1.2110000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1299999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":0.96499999999999997}]},{"osd":5,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":1.1120000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.246,"5min":2.246,"15min":2.6909999999999998},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.99199999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8240000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.163}]},{"osd":7,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.546,"15min":1.5149999999999999},"min":{"1min":1.0429999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.2050000000000001,"5min":2.3570000000000002,"15min":3.71},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.631,"15min":1.623},"min":{"1min":1.4710000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.7650000000000001},"last":1.4910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":552092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0900000000000001,"15min":1.1160000000000001},"min":{"1min":0.81100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4399999999999999,"5min":1.554,"15min":2.0070000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.133,"5min":1.1850000000000001,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.51,"5min":1.712,"15min":1.9530000000000001},"last":1.599}]},{"osd":1,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6339999999999999,"5min":1.877,"15min":1.944},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.91200000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.907},"last":1.123}]},{"osd":2,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.83599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.8,"5min":1.8,"15min":1.95},"last":1.02},{"interface":"front","average":{"1min":1.24,"5min":1.206,"15min":1.242},"min":{"1min":0.85099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.0249999999999999},"last":1.5209999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0880000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.573,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":1.153},{"interface":"front","average":{"1min":1.444,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":1.0009999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.79,"5min":1.79,"15min":1.8959999999999999},"last":1.333}]},{"osd":4,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2050000000000001,"15min":1.216},"min":{"1min":0.93999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.919,"5min":1.919,"15min":2.0960000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.349,"15min":1.339},"min":{"1min":0.96499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.76,"5min":2.0529999999999999,"15min":2.121},"last":1.837}]},{"osd":6,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.4550000000000001,"15min":1.45},"min":{"1min":1.3819999999999999,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1360000000000001},"last":1.23},{"interface":"front","average":{"1min":1.552,"5min":1.4790000000000001,"15min":1.4870000000000001},"min":{"1min":1.0860000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":1.6699999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.516,"15min":1.504},"min":{"1min":1.1699999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8300000000000001,"5min":2.218,"15min":2.218},"last":1.9350000000000001},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.534,"15min":1.556},"min":{"1min":1.371,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.286,"15min":2.286},"last":1.4319999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":553682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1830000000000001,"15min":1.1379999999999999},"min":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.55400000000000005},"max":{"1min":1.5509999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.194,"15min":1.21},"min":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7230000000000001,"5min":1.758,"15min":3.6080000000000001},"last":1.452}]},{"osd":1,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.252,"15min":1.2629999999999999},"min":{"1min":0.78200000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.663,"5min":1.885,"15min":2.0699999999999998},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2190000000000001,"15min":1.228},"min":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5609999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.179}]},{"osd":2,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.637,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.298,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.698,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.02}]},{"osd":3,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.946,"5min":2.5099999999999998,"15min":3.8109999999999999},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.409,"15min":1.3779999999999999},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.8340000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2889999999999999,"15min":1.3080000000000001},"min":{"1min":0.874,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7889999999999999,"5min":3.9199999999999999,"15min":3.9199999999999999},"last":1.607},{"interface":"front","average":{"1min":1.343,"5min":1.335,"15min":1.3340000000000001},"min":{"1min":0.89500000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0670000000000002,"5min":3.0699999999999998,"15min":3.9300000000000002},"last":1.885}]},{"osd":6,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.8080000000000001,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.583,"15min":1.5800000000000001},"min":{"1min":1.2470000000000001,"5min":0.878,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.4500000000000002,"15min":3.9609999999999999},"last":1.9710000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5009999999999999,"15min":1.492},"min":{"1min":0.91800000000000004,"5min":0.77200000000000002,"15min":0.69599999999999995},"max":{"1min":2.2170000000000001,"5min":3.1859999999999999,"15min":4.0190000000000001},"last":2.2400000000000002},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6120000000000001,"15min":1.5609999999999999},"min":{"1min":1.3320000000000001,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9490000000000001,"5min":3.968,"15min":3.968},"last":1.9239999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":552070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98199999999999998,"15min":0.94999999999999996},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.526,"5min":1.746,"15min":1.746},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.147,"5min":1.1779999999999999,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.365,"5min":2.04,"15min":2.5289999999999999},"last":0.72899999999999998}]},{"osd":1,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99399999999999999,"15min":1.0029999999999999},"min":{"1min":0.71599999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.53,"5min":2.173,"15min":2.2519999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":1.0069999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.8100000000000001,"15min":3.827},"last":0.62}]},{"osd":2,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.3360000000000001,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.206,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.663,"5min":2.847,"15min":2.847},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.403},"min":{"1min":1.101,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6970000000000001,"5min":2.1040000000000001,"15min":2.2349999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.1180000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.8160000000000001,"5min":2.919,"15min":2.919},"last":0.88800000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.415,"15min":1.427},"min":{"1min":1.167,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.619,"5min":2.02,"15min":3.7250000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4930000000000001,"15min":1.4990000000000001},"min":{"1min":1.129,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.841,"5min":2.2629999999999999,"15min":2.423},"last":1.0660000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.1399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.054},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.2909999999999999,"5min":0.871,"15min":0.70299999999999996},"max":{"1min":1.7689999999999999,"5min":2.2290000000000001,"15min":3.573},"last":0.76500000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5740000000000001,"15min":1.591},"min":{"1min":1.2669999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.915,"5min":2.302,"15min":2.3759999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5780000000000001,"15min":1.6100000000000001},"min":{"1min":1.304,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.1280000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.0960000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1093632,"data_stored":552070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.85999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.268,"5min":1.859,"15min":1.859},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.42699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.331,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.69599999999999995}]},{"osd":1,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.093,"15min":1.129},"min":{"1min":0.63400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.528,"5min":1.907,"15min":1.9390000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.206,"15min":1.2549999999999999},"min":{"1min":0.64900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.657,"5min":1.881,"15min":1.881},"last":0.64500000000000002}]},{"osd":3,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.0580000000000001,"15min":1.1240000000000001},"min":{"1min":0.60999999999999999,"5min":0.438,"15min":0.379},"max":{"1min":1.7230000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3240000000000001,"15min":1.3640000000000001},"min":{"1min":0.69799999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.6919999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.345}]},{"osd":4,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3600000000000001,"15min":1.369},"min":{"1min":0.94499999999999995,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.958,"5min":1.958,"15min":2.206},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.825,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.343},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":2.024,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1099999999999999,"5min":2.1349999999999998,"15min":2.4870000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4470000000000001,"15min":1.462},"min":{"1min":0.93000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7589999999999999,"5min":2.6819999999999999,"15min":2.698},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.4359999999999999},"min":{"1min":0.84999999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8200000000000001,"5min":2.552,"15min":2.6179999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.494,"15min":1.5309999999999999},"min":{"1min":0.88,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.863,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4590000000000001,"15min":1.5129999999999999},"min":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.865,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.1439999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1122304,"data_stored":554446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88500000000000001,"15min":0.91000000000000003},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.323,"5min":1.905,"15min":1.905},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.071,"15min":1.0740000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6930000000000001,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":1.006}]},{"osd":2,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":0.997},"min":{"1min":0.53500000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4419999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.133,"5min":1.157,"15min":1.1850000000000001},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.974,"15min":2.29},"last":1.196}]},{"osd":3,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.244,"15min":1.3},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7350000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.323},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5489999999999999,"5min":1.877,"15min":2.3620000000000001},"last":1.1160000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2589999999999999,"15min":1.276},"min":{"1min":0.88500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.71,"5min":1.944,"15min":1.944},"last":1.157},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.833,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.468}]},{"osd":5,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.4610000000000001,"15min":1.5049999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.75900000000000001},"max":{"1min":1.9099999999999999,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.458,"15min":1.4790000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.52}]},{"osd":6,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4790000000000001,"15min":1.5109999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.984,"5min":2.2469999999999999,"15min":4.1260000000000003},"last":1.377},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4730000000000001},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9510000000000001,"5min":2.2090000000000001,"15min":3.101},"last":1.2689999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5820000000000001,"15min":1.609},"min":{"1min":0.872,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2530000000000001,"5min":2.2679999999999998,"15min":2.8180000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5569999999999999,"15min":1.613},"min":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.901,"5min":2.1619999999999999,"15min":3.952},"last":1.659}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1687552,"data_stored":1142438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.583,"5min":1.782,"15min":2.9260000000000002},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0569999999999999},"min":{"1min":0.66000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":1.5329999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.075,"15min":1.123},"min":{"1min":0.80600000000000005,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":1.6559999999999999,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":1.234},{"interface":"front","average":{"1min":1.131,"5min":1.125,"15min":1.147},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.42999999999999999},"max":{"1min":1.603,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.194}]},{"osd":3,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.093},"min":{"1min":0.67200000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.702,"5min":1.8660000000000001,"15min":2.1429999999999998},"last":1.121},{"interface":"front","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.29},"min":{"1min":0.91400000000000003,"5min":0.48599999999999999,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8109999999999999,"15min":3.052},"last":1.4990000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3939999999999999,"15min":1.417},"min":{"1min":0.98699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.756,"5min":2.0339999999999998,"15min":3.3580000000000001},"last":1.702},{"interface":"front","average":{"1min":1.546,"5min":1.4950000000000001,"15min":1.5309999999999999},"min":{"1min":1.3069999999999999,"5min":0.872,"15min":0.872},"max":{"1min":1.863,"5min":2.246,"15min":3.306},"last":1.448}]},{"osd":5,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3859999999999999},"min":{"1min":1.0129999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6279999999999999,"5min":1.9419999999999999,"15min":3.8849999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.454,"15min":1.504},"min":{"1min":1.046,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8089999999999999,"5min":2.137,"15min":3.1789999999999998},"last":1.3819999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4419999999999999,"15min":1.4850000000000001},"min":{"1min":0.95799999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.8069999999999999,"5min":2.5379999999999998,"15min":4.3019999999999996},"last":1.579},{"interface":"front","average":{"1min":1.526,"5min":1.5489999999999999,"15min":1.595},"min":{"1min":1.2030000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8069999999999999,"5min":2.355,"15min":3.5139999999999998},"last":1.5609999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4330000000000001,"15min":1.4610000000000001},"min":{"1min":1.0389999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.637,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.417},{"interface":"front","average":{"1min":1.476,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":1.0960000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.802,"5min":2.077,"15min":4.6219999999999999},"last":1.4630000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:17:31.345 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:17:31.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:17:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.622+0000 7f84bda67640 1 Processor -- start 2023-10-21T16:17:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.623+0000 7f84bda67640 1 -- start start 2023-10-21T16:17:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.624+0000 7f84bda67640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f84b815cb40 0x7f84b806d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.624+0000 7f84bda67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84b80b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.624+0000 7f84bda67640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b80cd2e0 con 0x7f84b806da60 2023-10-21T16:17:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.624+0000 7f84bda67640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b815cf20 con 0x7f84b815cb40 2023-10-21T16:17:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.624+0000 7f84bda67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cd510 con 0x7f84b80cb970 2023-10-21T16:17:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84b7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f84b806da60 0x7f84b80cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40894/0 (socket says 172.21.15.53:40894) 2023-10-21T16:17:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84b7fff640 1 -- 172.21.15.53:0/3457535439 learned_addr learned my addr 172.21.15.53:0/3457535439 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84bca65640 1 --2- 172.21.15.53:0/3457535439 >> v2:172.21.15.124:3300/0 conn(0x7f84b815cb40 0x7f84b806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84bd266640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84b80b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050457112 0 0) 0x7f84b80cd2e0 con 0x7f84b806da60 2023-10-21T16:17:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.625+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a8003520 con 0x7f84b806da60 2023-10-21T16:17:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017002767 0 0) 0x7f84a8003520 con 0x7f84b806da60 2023-10-21T16:17:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 msgr2=0x7f84b80b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84b80b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 >> v2:172.21.15.124:3300/0 conn(0x7f84b815cb40 msgr2=0x7f84b806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 --2- 172.21.15.53:0/3457535439 >> v2:172.21.15.124:3300/0 conn(0x7f84b815cb40 0x7f84b806d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80ce120 con 0x7f84b806da60 2023-10-21T16:17:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f84a4001670 con 0x7f84b806da60 2023-10-21T16:17:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 >> v1:172.21.15.53:6789/0 conn(0x7f84b806da60 legacy=0x7f84b80cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b815cf20 con 0x7f84a80047c0 2023-10-21T16:17:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cd510 con 0x7f84a8003970 2023-10-21T16:17:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800b980 con 0x7f84b80cb970 2023-10-21T16:17:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84bca65640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.626+0000 7f84bd266640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84b7fff640 1 -- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 msgr2=0x7f84a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84b7fff640 1 -- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84a8003970 msgr2=0x7f84a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84b7fff640 1 -- 172.21.15.53:0/3457535439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8003520 con 0x7f84a80047c0 2023-10-21T16:17:31.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84bd266640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84b80cb970 0x7f84a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.627+0000 7f84bca65640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.628+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84a400c160 tx=0x7f84a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc9f6c2c1a8300f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.628+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4005ad0 con 0x7f84a80047c0 2023-10-21T16:17:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.628+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4005c30 con 0x7f84a80047c0 2023-10-21T16:17:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.628+0000 7f84b77fe640 1 -- 172.21.15.53:0/3457535439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a402c830 con 0x7f84a80047c0 2023-10-21T16:17:31.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 -- 172.21.15.53:0/3457535439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84a80047c0 msgr2=0x7f84a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 --2- 172.21.15.53:0/3457535439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f84a400c160 tx=0x7f84a80057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bd266640 1 -- 172.21.15.53:0/3457535439 reap_dead start 2023-10-21T16:17:31.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 -- 172.21.15.53:0/3457535439 shutdown_connections 2023-10-21T16:17:31.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 -- 172.21.15.53:0/3457535439 >> 172.21.15.53:0/3457535439 conn(0x7f84b80ce9f0 msgr2=0x7f84b80cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:31.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 -- 172.21.15.53:0/3457535439 shutdown_connections 2023-10-21T16:17:31.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.629+0000 7f84bda67640 1 -- 172.21.15.53:0/3457535439 wait complete. 2023-10-21T16:17:31.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.631+0000 7f84bda67640 1 Processor -- start 2023-10-21T16:17:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.631+0000 7f84bda67640 1 -- start start 2023-10-21T16:17:31.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84a80047c0 0x7f84b8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84b80c7230 0x7f84b80c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8151320 con 0x7f84b80c7f70 2023-10-21T16:17:31.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80c7c20 con 0x7f84b80c7230 2023-10-21T16:17:31.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84bda67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b81f2ee0 con 0x7f84a80047c0 2023-10-21T16:17:31.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49592/0 (socket says 172.21.15.53:49592) 2023-10-21T16:17:31.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.632+0000 7f84b7fff640 1 -- 172.21.15.53:0/3363874084 learned_addr learned my addr 172.21.15.53:0/3363874084 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:31.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84bca65640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84a80047c0 0x7f84b8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84bd266640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84b80c7230 0x7f84b80c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 -- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84a80047c0 msgr2=0x7f84b8150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84a80047c0 0x7f84b8150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 -- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84b80c7230 msgr2=0x7f84b80c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84b80c7230 0x7f84b80c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 -- 172.21.15.53:0/3363874084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b81f2540 con 0x7f84b80c7f70 2023-10-21T16:17:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.633+0000 7f84b7fff640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84a402c3d0 tx=0x7f84a4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.634+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4039070 con 0x7f84b80c7f70 2023-10-21T16:17:31.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.634+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a402cc40 con 0x7f84b80c7f70 2023-10-21T16:17:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.634+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4007070 con 0x7f84b80c7f70 2023-10-21T16:17:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.634+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b81f2820 con 0x7f84b80c7f70 2023-10-21T16:17:31.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.634+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b80ca1b0 con 0x7f84b80c7f70 2023-10-21T16:17:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.635+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b80ca340 con 0x7f84b80c7f70 2023-10-21T16:17:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.636+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84a4032070 con 0x7f84b80c7f70 2023-10-21T16:17:31.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.637+0000 7f84b57fa640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 0x7f849c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.638+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84a403c040 con 0x7f84b80c7f70 2023-10-21T16:17:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.638+0000 7f84bca65640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 0x7f849c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:31.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.639+0000 7f84bca65640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 0x7f849c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009b20 tx=0x7f84ac008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:31.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.642+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84a4075840 con 0x7f84b80c7f70 2023-10-21T16:17:31.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.919+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f84b80ad290 con 0x7f84b80c7f70 2023-10-21T16:17:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.923+0000 7f84b57fa640 1 -- 172.21.15.53:0/3363874084 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f84a4078e60 con 0x7f84b80c7f70 2023-10-21T16:17:31.925 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:17:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:17:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:17:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:17:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:17:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:17:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 msgr2=0x7f849c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 0x7f849c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009b20 tx=0x7f84ac008040 comp rx=0 tx=0).stop 2023-10-21T16:17:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 msgr2=0x7f84b81f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f84a402c3d0 tx=0x7f84a4000fe0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 shutdown_connections 2023-10-21T16:17:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f849c06afc0 0x7f849c06d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84b80c7230 0x7f84b80c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b80c7f70 0x7f84b81f1e00 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 --2- 172.21.15.53:0/3363874084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84a80047c0 0x7f84b8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.926+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 >> 172.21.15.53:0/3363874084 conn(0x7f84b80ce9f0 msgr2=0x7f84b8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.927+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 shutdown_connections 2023-10-21T16:17:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:31.927+0000 7f84bda67640 1 -- 172.21.15.53:0/3363874084 wait complete. 2023-10-21T16:17:31.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.176+0000 7f093359e640 1 Processor -- start 2023-10-21T16:17:32.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.177+0000 7f093359e640 1 -- start start 2023-10-21T16:17:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0934157480 0x7f093406af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f093406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f093415d550 0x7f093415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340cde30 con 0x7f093415d550 2023-10-21T16:17:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0934157860 con 0x7f0934157480 2023-10-21T16:17:32.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.178+0000 7f093359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093415e2c0 con 0x7f09340ce2c0 2023-10-21T16:17:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f093415d550 0x7f093415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f093415d550 0x7f093415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49598/0 (socket says 172.21.15.53:49598) 2023-10-21T16:17:32.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 -- 172.21.15.53:0/1485223109 learned_addr learned my addr 172.21.15.53:0/1485223109 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f092bfff640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f093406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f093259c640 1 --2- 172.21.15.53:0/1485223109 >> v2:172.21.15.124:3300/0 conn(0x7f0934157480 0x7f093406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 -- 172.21.15.53:0/1485223109 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 msgr2=0x7f093406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:32.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f093406b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 -- 172.21.15.53:0/1485223109 >> v2:172.21.15.124:3300/0 conn(0x7f0934157480 msgr2=0x7f093406af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 --2- 172.21.15.53:0/1485223109 >> v2:172.21.15.124:3300/0 conn(0x7f0934157480 0x7f093406af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.179+0000 7f0932d9d640 1 -- 172.21.15.53:0/1485223109 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340cd500 con 0x7f093415d550 2023-10-21T16:17:32.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.180+0000 7f0932d9d640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f093415d9a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0934066ff0 tx=0x7f092c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc5752cdc16945 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f0931d9b640 1 -- 172.21.15.53:0/1485223109 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c02c9f0 con 0x7f093415d550 2023-10-21T16:17:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f0931d9b640 1 -- 172.21.15.53:0/1485223109 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c02cb50 con 0x7f093415d550 2023-10-21T16:17:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f0931d9b640 1 -- 172.21.15.53:0/1485223109 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0343f0 con 0x7f093415d550 2023-10-21T16:17:32.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 -- 172.21.15.53:0/1485223109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 msgr2=0x7f093415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:32.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f093415d9a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0934066ff0 tx=0x7f092c02be50 comp rx=0 tx=0).stop 2023-10-21T16:17:32.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 -- 172.21.15.53:0/1485223109 shutdown_connections 2023-10-21T16:17:32.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f093406b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 --2- 172.21.15.53:0/1485223109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f093415d9a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 --2- 172.21.15.53:0/1485223109 >> v2:172.21.15.124:3300/0 conn(0x7f0934157480 0x7f093406af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.181+0000 7f093359e640 1 -- 172.21.15.53:0/1485223109 >> 172.21.15.53:0/1485223109 conn(0x7f09340d2ae0 msgr2=0x7f09340d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.182+0000 7f093359e640 1 -- 172.21.15.53:0/1485223109 shutdown_connections 2023-10-21T16:17:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.182+0000 7f093359e640 1 -- 172.21.15.53:0/1485223109 wait complete. 2023-10-21T16:17:32.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.183+0000 7f093359e640 1 Processor -- start 2023-10-21T16:17:32.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.184+0000 7f093359e640 1 -- start start 2023-10-21T16:17:32.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.184+0000 7f093359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f09341f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.184+0000 7f093359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0934157480 0x7f0934141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f093359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f093359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934155d40 con 0x7f093415d550 2023-10-21T16:17:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f093359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934156020 con 0x7f0934157480 2023-10-21T16:17:32.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f093359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0934156300 con 0x7f09340ce2c0 2023-10-21T16:17:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49604/0 (socket says 172.21.15.53:49604) 2023-10-21T16:17:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 -- 172.21.15.53:0/2114379694 learned_addr learned my addr 172.21.15.53:0/2114379694 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:32.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f092bfff640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0934157480 0x7f0934141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f093259c640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f09341f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 -- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 msgr2=0x7f09341f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f09341f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 -- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0934157480 msgr2=0x7f0934141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:32.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0934157480 0x7f0934141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.185+0000 7f0932d9d640 1 -- 172.21.15.53:0/2114379694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009b30 con 0x7f093415d550 2023-10-21T16:17:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f0932d9d640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f092c036040 tx=0x7f092c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c009510 con 0x7f093415d550 2023-10-21T16:17:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c03e450 con 0x7f093415d550 2023-10-21T16:17:32.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c03e800 con 0x7f093415d550 2023-10-21T16:17:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09341565e0 con 0x7f093415d550 2023-10-21T16:17:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.186+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0934154f80 con 0x7f093415d550 2023-10-21T16:17:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.187+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934156aa0 con 0x7f093415d550 2023-10-21T16:17:32.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.188+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f092c03e960 con 0x7f093415d550 2023-10-21T16:17:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.189+0000 7f092affd640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 0x7f091406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.189+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f092c0ac8e0 con 0x7f093415d550 2023-10-21T16:17:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.190+0000 7f093259c640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 0x7f091406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:32.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.191+0000 7f093259c640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 0x7f091406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f092400b7c0 tx=0x7f0924008040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:32.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.194+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f092c0777b0 con 0x7f093415d550 2023-10-21T16:17:32.226 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-10-21T16:17:32.224+0000 7f0bb4d6c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:17:32.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.429+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09340cd7b0 con 0x7f093415d550 2023-10-21T16:17:32.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.431+0000 7f092affd640 1 -- 172.21.15.53:0/2114379694 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f092c043090 con 0x7f093415d550 2023-10-21T16:17:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:32.433 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:32.433 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 msgr2=0x7f091406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 0x7f091406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f092400b7c0 tx=0x7f0924008040 comp rx=0 tx=0).stop 2023-10-21T16:17:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 msgr2=0x7f0934141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f092c036040 tx=0x7f092c004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 shutdown_connections 2023-10-21T16:17:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f091406af90 0x7f091406d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f093415d550 0x7f0934141d60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0934157480 0x7f0934141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 --2- 172.21.15.53:0/2114379694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09340ce2c0 0x7f09341f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 >> 172.21.15.53:0/2114379694 conn(0x7f09340d2ae0 msgr2=0x7f09340b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:32.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 shutdown_connections 2023-10-21T16:17:32.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:32.434+0000 7f093359e640 1 -- 172.21.15.53:0/2114379694 wait complete. 2023-10-21T16:17:32.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:17:25.738039+0000', 'modified': '2023-10-21T16:17:28.815415+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35345}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35345': {'gid': 35345, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/2020660539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 2020660539}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2020660539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:17:32.857 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:17:32.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-21T16:17:33.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.098+0000 7f2607461640 1 Processor -- start 2023-10-21T16:17:33.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.098+0000 7f2607461640 1 -- start start 2023-10-21T16:17:33.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f2607461640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f26000aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f2607461640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f260015e5f0 0x7f260015b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f2607461640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26000af0a0 con 0x7f26001518d0 2023-10-21T16:17:33.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f2607461640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260015ebb0 con 0x7f260015e5f0 2023-10-21T16:17:33.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f2607461640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260015ee90 con 0x7f2600151cb0 2023-10-21T16:17:33.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f26051fd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f26001518d0 0x7f260006ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40904/0 (socket says 172.21.15.53:40904) 2023-10-21T16:17:33.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f26051fd640 1 -- 172.21.15.53:0/3429362269 learned_addr learned my addr 172.21.15.53:0/3429362269 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:33.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f26059fe640 1 --2- 172.21.15.53:0/3429362269 >> v2:172.21.15.124:3300/0 conn(0x7f260015e5f0 0x7f260015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.100+0000 7f26049fc640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f26000aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050088692 0 0) 0x7f25fc001490 con 0x7f26001518d0 2023-10-21T16:17:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec003520 con 0x7f26001518d0 2023-10-21T16:17:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272448305 0 0) 0x7f25ec003520 con 0x7f26001518d0 2023-10-21T16:17:33.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 msgr2=0x7f26000aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f26000aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 >> v2:172.21.15.124:3300/0 conn(0x7f260015e5f0 msgr2=0x7f260015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 --2- 172.21.15.53:0/3429362269 >> v2:172.21.15.124:3300/0 conn(0x7f260015e5f0 0x7f260015b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260015f170 con 0x7f26001518d0 2023-10-21T16:17:33.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f25fc001b80 con 0x7f26001518d0 2023-10-21T16:17:33.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 >> v1:172.21.15.53:6789/0 conn(0x7f26001518d0 legacy=0x7f260006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ec003970 0x7f25ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f25ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec004d00 0x7f25ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260015ebb0 con 0x7f25ec003970 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260015ee90 con 0x7f25ec004d00 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.101+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec00b980 con 0x7f2600151cb0 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ec003970 0x7f25ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26059fe640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec004d00 0x7f25ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26049fc640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f25ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 -- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 msgr2=0x7f25ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2600151cb0 0x7f25ec0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 -- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec004d00 msgr2=0x7f25ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec004d00 0x7f25ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.102+0000 7f26051fd640 1 -- 172.21.15.53:0/3429362269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec003520 con 0x7f25ec003970 2023-10-21T16:17:33.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.103+0000 7f26051fd640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ec003970 0x7f25ec003d90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a830 tx=0x7f25ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f2e02f16ed9432 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.103+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc034630 con 0x7f25ec003970 2023-10-21T16:17:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.103+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc02f070 con 0x7f25ec003970 2023-10-21T16:17:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.103+0000 7f25f7fff640 1 -- 172.21.15.53:0/3429362269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0349f0 con 0x7f25ec003970 2023-10-21T16:17:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 -- 172.21.15.53:0/3429362269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ec003970 msgr2=0x7f25ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 --2- 172.21.15.53:0/3429362269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25ec003970 0x7f25ec003d90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a830 tx=0x7f25ec0057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f26059fe640 1 -- 172.21.15.53:0/3429362269 reap_dead start 2023-10-21T16:17:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 -- 172.21.15.53:0/3429362269 shutdown_connections 2023-10-21T16:17:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 -- 172.21.15.53:0/3429362269 >> 172.21.15.53:0/3429362269 conn(0x7f26000cc2e0 msgr2=0x7f26000cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 -- 172.21.15.53:0/3429362269 shutdown_connections 2023-10-21T16:17:33.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.104+0000 7f2607461640 1 -- 172.21.15.53:0/3429362269 wait complete. 2023-10-21T16:17:33.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.106+0000 7f2607461640 1 Processor -- start 2023-10-21T16:17:33.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.106+0000 7f2607461640 1 -- start start 2023-10-21T16:17:33.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec003970 0x7f2600150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26000c43e0 0x7f26000c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000c4da0 con 0x7f26000b0b40 2023-10-21T16:17:33.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26001512c0 con 0x7f25ec003970 2023-10-21T16:17:33.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f2607461640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26000b16b0 con 0x7f26000c43e0 2023-10-21T16:17:33.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f26049fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f26049fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49620/0 (socket says 172.21.15.53:49620) 2023-10-21T16:17:33.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.107+0000 7f26049fc640 1 -- 172.21.15.53:0/1888876505 learned_addr learned my addr 172.21.15.53:0/1888876505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26051fd640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec003970 0x7f2600150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26059fe640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26000c43e0 0x7f26000c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 -- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26000c43e0 msgr2=0x7f26000c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26000c43e0 0x7f26000c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 -- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec003970 msgr2=0x7f2600150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec003970 0x7f2600150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 -- 172.21.15.53:0/1888876505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000b1990 con 0x7f26000b0b40 2023-10-21T16:17:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f26049fc640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f25f0010890 tx=0x7f25f0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1888876505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0009610 con 0x7f26000b0b40 2023-10-21T16:17:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1888876505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f000eb60 con 0x7f26000b0b40 2023-10-21T16:17:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.108+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1888876505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f001c400 con 0x7f26000b0b40 2023-10-21T16:17:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.109+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000b1c70 con 0x7f26000b0b40 2023-10-21T16:17:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.109+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26000b05b0 con 0x7f26000b0b40 2023-10-21T16:17:33.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.112+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1888876505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f25f001c560 con 0x7f26000b0b40 2023-10-21T16:17:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.113+0000 7f25f5ffb640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 0x7f25e406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.113+0000 7f25f5ffb640 1 -- 172.21.15.53:0/1888876505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25f0089f00 con 0x7f26000b0b40 2023-10-21T16:17:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.113+0000 7f2609ffa740 1 Processor -- start 2023-10-21T16:17:33.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.114+0000 7f26051fd640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 0x7f25e406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.114+0000 7f2609ffa740 1 -- start start 2023-10-21T16:17:33.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 0x564b1d96de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d99dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d88e7a0 con 0x564b1d96e3c0 2023-10-21T16:17:33.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d8e0740 con 0x564b1d99b360 2023-10-21T16:17:33.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1d8dde30 con 0x564b1d99af80 2023-10-21T16:17:33.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26051fd640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 0x7f25e406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25fc034000 tx=0x7f25fc009030 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26059fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26059fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49626/0 (socket says 172.21.15.53:49626) 2023-10-21T16:17:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26059fe640 1 -- 172.21.15.53:0/1920257001 learned_addr learned my addr 172.21.15.53:0/1920257001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26049fc640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d99dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.115+0000 7f26051fd640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 0x564b1d96de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 -- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 msgr2=0x564b1d96de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 0x564b1d96de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 -- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 msgr2=0x564b1d99dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d99dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 -- 172.21.15.53:0/1920257001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1d89d910 con 0x564b1d96e3c0 2023-10-21T16:17:33.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26051fd640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 0x564b1d96de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:33.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f26059fe640 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f260015eab0 tx=0x7f25f8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45a916e7a9c5ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f25e2ffd640 1 -- 172.21.15.53:0/1920257001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004230 con 0x564b1d96e3c0 2023-10-21T16:17:33.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f25e2ffd640 1 -- 172.21.15.53:0/1920257001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f80043b0 con 0x564b1d96e3c0 2023-10-21T16:17:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.116+0000 7f25e2ffd640 1 -- 172.21.15.53:0/1920257001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004520 con 0x564b1d96e3c0 2023-10-21T16:17:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 -- 172.21.15.53:0/1920257001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 msgr2=0x564b1d99e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f260015eab0 tx=0x7f25f8009190 comp rx=0 tx=0).stop 2023-10-21T16:17:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 -- 172.21.15.53:0/1920257001 shutdown_connections 2023-10-21T16:17:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d96e3c0 0x564b1d99e250 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d99dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 --2- 172.21.15.53:0/1920257001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99af80 0x564b1d96de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 -- 172.21.15.53:0/1920257001 >> 172.21.15.53:0/1920257001 conn(0x564b1d970190 msgr2=0x564b1d9b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 -- 172.21.15.53:0/1920257001 shutdown_connections 2023-10-21T16:17:33.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.117+0000 7f2609ffa740 1 -- 172.21.15.53:0/1920257001 wait complete. 2023-10-21T16:17:33.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.118+0000 7f2609ffa740 1 -- start start 2023-10-21T16:17:33.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.118+0000 7f2609ffa740 1 Processor -- start 2023-10-21T16:17:33.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d98b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 0x564b1d9bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d8d9720 con 0x564b1d99af80 2023-10-21T16:17:33.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d877970 con 0x564b1d99b360 2023-10-21T16:17:33.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1d877610 con 0x564b1d988c90 2023-10-21T16:17:33.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f2609ffa740 10 client.0 _ll_register_callbacks cb 0x564b1d8999d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:17:33.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49640/0 (socket says 172.21.15.53:49640) 2023-10-21T16:17:33.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.119+0000 7f26051fd640 1 -- 172.21.15.53:0/819952251 learned_addr learned my addr 172.21.15.53:0/819952251 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26059fe640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 0x564b1d9bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26049fc640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d98b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 -- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 msgr2=0x564b1d9bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 0x564b1d9bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 -- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 msgr2=0x564b1d98b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d98b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 -- 172.21.15.53:0/819952251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f8009600 con 0x564b1d99af80 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26059fe640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 0x564b1d9bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:33.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f25dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:17:33.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f26051fd640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a920 tx=0x7f25fc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ffc6bdbeafd79 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.120+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc041e20 con 0x564b1d99af80 2023-10-21T16:17:33.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25fc034430 con 0x564b1d99af80 2023-10-21T16:17:33.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b1d900160 con 0x564b1d99af80 2023-10-21T16:17:33.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25fc0345f0 con 0x564b1d99af80 2023-10-21T16:17:33.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 10 client.6777 fetch_fsmap learned FSMap version 36 2023-10-21T16:17:33.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x564b1d8d7da0 con 0x564b1d99af80 2023-10-21T16:17:33.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b1d900160 con 0x564b1d99af80 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f25fc034750 con 0x564b1d99af80 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 10 client.6777 fetch_fsmap finished waiting for FSMap version 36 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 10 client.6777 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 20 client.6777 populate_metadata read hostname 'smithi053' 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1d9b1860 0x564b1d989e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 4 client.6777 mds_command: new command op to 35345 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:17:33.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.121+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 --> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b1d8abcc0 con 0x564b1d9b1860 2023-10-21T16:17:33.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.122+0000 7f26049fc640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1d9b1860 0x564b1d989e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.122+0000 7f26049fc640 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1d9b1860 0x564b1d989e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bdef4606581ccb3 server_cookie=9accc34d8723873b in_seq=0 out_seq=0 2023-10-21T16:17:33.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.122+0000 7f25dbfff640 10 client.6777 ms_handle_connect on v2:172.21.15.53:6833/2020660539 2023-10-21T16:17:33.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f25dbfff640 1 -- 172.21.15.53:0/819952251 <== mds.0 v2:172.21.15.53:6833/2020660539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b1d8abcc0 con 0x564b1d9b1860 2023-10-21T16:17:33.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f25dbfff640 10 client.6777 handle_command_reply: tid=0 2023-10-21T16:17:33.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 client.6777 shutdown 2023-10-21T16:17:33.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 msgr2=0x564b1d98b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a920 tx=0x7f25fc02e000 comp rx=0 tx=0).stop 2023-10-21T16:17:33.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 shutdown_connections 2023-10-21T16:17:33.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1d9b1860 0x564b1d989e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d988c90 0x564b1d9bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d98b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f25dbfff640 0 client.6777 ms_handle_reset on v2:172.21.15.53:6833/2020660539 2023-10-21T16:17:33.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 --2- 172.21.15.53:0/819952251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d99af80 0x564b1d98b250 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.123+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 >> 172.21.15.53:0/819952251 conn(0x564b1d96e3c0 msgr2=0x564b1d971d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.124+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 shutdown_connections 2023-10-21T16:17:33.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.124+0000 7f2609ffa740 1 -- 172.21.15.53:0/819952251 wait complete. 2023-10-21T16:17:33.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.124+0000 7f2609ffa740 20 client.6777 trim_cache size 0 max 16384 2023-10-21T16:17:33.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.141+0000 7f2609ffa740 1 Processor -- start 2023-10-21T16:17:33.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.141+0000 7f2609ffa740 1 -- start start 2023-10-21T16:17:33.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d9b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99d810 0x564b1d98af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d63df20 con 0x564b1d988c90 2023-10-21T16:17:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d988aa0 con 0x564b1d99b360 2023-10-21T16:17:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1d9bb970 con 0x564b1d99d810 2023-10-21T16:17:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49650/0 (socket says 172.21.15.53:49650) 2023-10-21T16:17:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.142+0000 7f26051fd640 1 -- 172.21.15.53:0/2770655738 learned_addr learned my addr 172.21.15.53:0/2770655738 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26059fe640 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99d810 0x564b1d98af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26049fc640 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d9b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 -- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99d810 msgr2=0x564b1d98af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99d810 0x564b1d98af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 -- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 msgr2=0x564b1d9b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d9b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 -- 172.21.15.53:0/2770655738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1d876b80 con 0x564b1d988c90 2023-10-21T16:17:33.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f26051fd640 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02cb80 tx=0x7f25fc04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f04fc03c49f908 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f25e37fe640 1 -- 172.21.15.53:0/2770655738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc034630 con 0x564b1d988c90 2023-10-21T16:17:33.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f25e37fe640 1 -- 172.21.15.53:0/2770655738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc047050 con 0x564b1d988c90 2023-10-21T16:17:33.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.143+0000 7f25e37fe640 1 -- 172.21.15.53:0/2770655738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc041860 con 0x564b1d988c90 2023-10-21T16:17:33.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 -- 172.21.15.53:0/2770655738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 msgr2=0x564b1d9b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f25fc02cb80 tx=0x7f25fc04a000 comp rx=0 tx=0).stop 2023-10-21T16:17:33.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 -- 172.21.15.53:0/2770655738 shutdown_connections 2023-10-21T16:17:33.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99d810 0x564b1d98af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99b360 0x564b1d9b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2770655738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1d9b9190 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 -- 172.21.15.53:0/2770655738 >> 172.21.15.53:0/2770655738 conn(0x564b1d96e3c0 msgr2=0x564b1d9cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 -- 172.21.15.53:0/2770655738 shutdown_connections 2023-10-21T16:17:33.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.144+0000 7f2609ffa740 1 -- 172.21.15.53:0/2770655738 wait complete. 2023-10-21T16:17:33.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.145+0000 7f2609ffa740 1 -- start start 2023-10-21T16:17:33.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.145+0000 7f2609ffa740 1 Processor -- start 2023-10-21T16:17:33.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99b360 0x564b1d9b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99d810 0x564b1d9b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d9a11f0 con 0x564b1d988c90 2023-10-21T16:17:33.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1d999650 con 0x564b1d99d810 2023-10-21T16:17:33.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1d9db4d0 con 0x564b1d99b360 2023-10-21T16:17:33.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f2609ffa740 10 client.0 _ll_register_callbacks cb 0x7f25ec0036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:17:33.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f26051fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49664/0 (socket says 172.21.15.53:49664) 2023-10-21T16:17:33.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.146+0000 7f26051fd640 1 -- 172.21.15.53:0/2725001567 learned_addr learned my addr 172.21.15.53:0/2725001567 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 -- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99b360 msgr2=0x564b1d9b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26059fe640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99d810 0x564b1d9b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26049fc640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99b360 0x564b1d9b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99b360 0x564b1d9b9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 -- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99d810 msgr2=0x564b1d9b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99d810 0x564b1d9b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 -- 172.21.15.53:0/2725001567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1d8d7da0 con 0x564b1d988c90 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f26051fd640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25fc05e040 tx=0x7f25fc001f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb56c40b6d7f589 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f25dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.147+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc064460 con 0x564b1d988c90 2023-10-21T16:17:33.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25fc0645c0 con 0x564b1d988c90 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25f80043b0 con 0x564b1d988c90 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25fc064780 con 0x564b1d988c90 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 10 client.6783 fetch_fsmap learned FSMap version 36 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x564b1d89c170 con 0x564b1d988c90 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f80043b0 con 0x564b1d988c90 2023-10-21T16:17:33.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f25fc0648e0 con 0x564b1d988c90 2023-10-21T16:17:33.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 10 client.6783 fetch_fsmap finished waiting for FSMap version 36 2023-10-21T16:17:33.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 10 client.6783 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T16:17:33.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 20 client.6783 populate_metadata read hostname 'smithi053' 2023-10-21T16:17:33.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1da23ec0 0x564b1da242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 4 client.6783 mds_command: new command op to 35345 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:17:33.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.148+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 --> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b1d8d2700 con 0x564b1da23ec0 2023-10-21T16:17:33.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f26049fc640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1da23ec0 0x564b1da242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f26049fc640 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1da23ec0 0x564b1da242a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8da3357b796328 server_cookie=3eaaafedf55e6860 in_seq=0 out_seq=0 2023-10-21T16:17:33.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f25dbfff640 10 client.6783 ms_handle_connect on v2:172.21.15.53:6833/2020660539 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f25dbfff640 1 -- 172.21.15.53:0/2725001567 <== mds.0 v2:172.21.15.53:6833/2020660539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564b1d8d2700 con 0x564b1da23ec0 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f25dbfff640 10 client.6783 handle_command_reply: tid=0 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f2609ffa740 1 client.6783 shutdown 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 msgr2=0x564b1da20350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.149+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25fc05e040 tx=0x7f25fc001f80 comp rx=0 tx=0).stop 2023-10-21T16:17:33.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 shutdown_connections 2023-10-21T16:17:33.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:6833/2020660539,v1:172.21.15.53:6835/2020660539] conn(0x564b1da23ec0 0x564b1da242a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b1d99d810 0x564b1d9b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b1d99b360 0x564b1d9b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 --2- 172.21.15.53:0/2725001567 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b1d988c90 0x564b1da20350 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f25dbfff640 0 client.6783 ms_handle_reset on v2:172.21.15.53:6833/2020660539 2023-10-21T16:17:33.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 >> 172.21.15.53:0/2725001567 conn(0x564b1d96e3c0 msgr2=0x564b1d9a1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 shutdown_connections 2023-10-21T16:17:33.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 1 -- 172.21.15.53:0/2725001567 wait complete. 2023-10-21T16:17:33.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.150+0000 7f2609ffa740 20 client.6783 trim_cache size 0 max 16384 2023-10-21T16:17:33.170 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:17:33.170 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:17:33.170 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:17:33.170 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 35345, 2023-10-21T16:17:33.170 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:17:33.171 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:17:33.171 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:17:33.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8635405629999999, 2023-10-21T16:17:33.171 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 36, 2023-10-21T16:17:33.171 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 90, 2023-10-21T16:17:33.172 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 90, 2023-10-21T16:17:33.172 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.69751037 2023-10-21T16:17:33.172 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:17:33.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 msgr2=0x7f25e406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 0x7f25e406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25fc034000 tx=0x7f25fc009030 comp rx=0 tx=0).stop 2023-10-21T16:17:33.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 msgr2=0x7f26000b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f25f0010890 tx=0x7f25f0010d50 comp rx=0 tx=0).stop 2023-10-21T16:17:33.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 shutdown_connections 2023-10-21T16:17:33.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f25e406afc0 0x7f25e406d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26000c43e0 0x7f26000c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26000b0b40 0x7f26000b0fc0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 --2- 172.21.15.53:0/1888876505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25ec003970 0x7f2600150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 >> 172.21.15.53:0/1888876505 conn(0x7f26000cc2e0 msgr2=0x7f260015d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 shutdown_connections 2023-10-21T16:17:33.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.151+0000 7f2607461640 1 -- 172.21.15.53:0/1888876505 wait complete. 2023-10-21T16:17:33.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:17:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.402+0000 7f30af4d7640 1 Processor -- start 2023-10-21T16:17:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.402+0000 7f30af4d7640 1 -- start start 2023-10-21T16:17:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30af4d7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30a8159ac0 0x7f30a806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30af4d7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f30a80709e0 0x7f30a80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30af4d7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a806dee0 con 0x7f30a80ad790 2023-10-21T16:17:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30af4d7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a815d850 con 0x7f30a8159ac0 2023-10-21T16:17:33.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30af4d7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a815db30 con 0x7f30a80709e0 2023-10-21T16:17:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30ad273640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30a8159ac0 0x7f30a806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30ad273640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30a8159ac0 0x7f30a806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49668/0 (socket says 172.21.15.53:49668) 2023-10-21T16:17:33.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.404+0000 7f30ad273640 1 -- 172.21.15.53:0/3433360144 learned_addr learned my addr 172.21.15.53:0/3433360144 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.405+0000 7f30ada74640 1 --2- 172.21.15.53:0/3433360144 >> v2:172.21.15.124:3300/0 conn(0x7f30a80709e0 0x7f30a80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.405+0000 7f30ad273640 1 -- 172.21.15.53:0/3433360144 >> v2:172.21.15.124:3300/0 conn(0x7f30a80709e0 msgr2=0x7f30a80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.405+0000 7f30ad273640 1 --2- 172.21.15.53:0/3433360144 >> v2:172.21.15.124:3300/0 conn(0x7f30a80709e0 0x7f30a80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.405+0000 7f30ad273640 1 -- 172.21.15.53:0/3433360144 >> v1:172.21.15.53:6789/0 conn(0x7f30a80ad790 legacy=0x7f30a80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.405+0000 7f30ad273640 1 -- 172.21.15.53:0/3433360144 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a815de10 con 0x7f30a8159ac0 2023-10-21T16:17:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.406+0000 7f30ad273640 1 --2- 172.21.15.53:0/3433360144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8159ac0 0x7f30a806d5e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f30a80caab0 tx=0x7f30a402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86bc2da93df80bb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.406+0000 7f309ffff640 1 -- 172.21.15.53:0/3433360144 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a402e960 con 0x7f30a8159ac0 2023-10-21T16:17:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f309ffff640 1 -- 172.21.15.53:0/3433360144 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a402eac0 con 0x7f30a8159ac0 2023-10-21T16:17:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f309ffff640 1 -- 172.21.15.53:0/3433360144 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a402ee50 con 0x7f30a8159ac0 2023-10-21T16:17:33.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 -- 172.21.15.53:0/3433360144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8159ac0 msgr2=0x7f30a806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 --2- 172.21.15.53:0/3433360144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8159ac0 0x7f30a806d5e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f30a80caab0 tx=0x7f30a402dbb0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 -- 172.21.15.53:0/3433360144 shutdown_connections 2023-10-21T16:17:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 --2- 172.21.15.53:0/3433360144 >> v2:172.21.15.124:3300/0 conn(0x7f30a80709e0 0x7f30a80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 --2- 172.21.15.53:0/3433360144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a8159ac0 0x7f30a806d5e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 -- 172.21.15.53:0/3433360144 >> 172.21.15.53:0/3433360144 conn(0x7f30a806a380 msgr2=0x7f30a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 -- 172.21.15.53:0/3433360144 shutdown_connections 2023-10-21T16:17:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.407+0000 7f30af4d7640 1 -- 172.21.15.53:0/3433360144 wait complete. 2023-10-21T16:17:33.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.409+0000 7f30af4d7640 1 Processor -- start 2023-10-21T16:17:33.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.409+0000 7f30af4d7640 1 -- start start 2023-10-21T16:17:33.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30a80ad790 0x7f30a80af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30a8159ac0 0x7f30a80afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b4240 con 0x7f30a80709e0 2023-10-21T16:17:33.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b1f50 con 0x7f30a80ad790 2023-10-21T16:17:33.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30af4d7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b1b00 con 0x7f30a8159ac0 2023-10-21T16:17:33.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30ad273640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30ad273640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49678/0 (socket says 172.21.15.53:49678) 2023-10-21T16:17:33.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.410+0000 7f30ad273640 1 -- 172.21.15.53:0/564041862 learned_addr learned my addr 172.21.15.53:0/564041862 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:33.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 -- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30a8159ac0 msgr2=0x7f30a80afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30aca72640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30a80ad790 0x7f30a80af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ada74640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30a8159ac0 0x7f30a80afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30a8159ac0 0x7f30a80afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 -- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30a80ad790 msgr2=0x7f30a80af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30a80ad790 0x7f30a80af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 -- 172.21.15.53:0/564041862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40094d0 con 0x7f30a80709e0 2023-10-21T16:17:33.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.411+0000 7f30ad273640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f30a402dbf0 tx=0x7f30a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.412+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4041640 con 0x7f30a80709e0 2023-10-21T16:17:33.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.412+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a40417c0 con 0x7f30a80709e0 2023-10-21T16:17:33.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.412+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4041b70 con 0x7f30a80709e0 2023-10-21T16:17:33.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.412+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a815c9c0 con 0x7f30a80709e0 2023-10-21T16:17:33.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.412+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a815cf80 con 0x7f30a80709e0 2023-10-21T16:17:33.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.413+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a8067490 con 0x7f30a80709e0 2023-10-21T16:17:33.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.414+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30a403e3f0 con 0x7f30a80709e0 2023-10-21T16:17:33.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.415+0000 7f309dffb640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 0x7f308c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:33.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.415+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f30a40abc70 con 0x7f30a80709e0 2023-10-21T16:17:33.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.416+0000 7f30aca72640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 0x7f308c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:33.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.417+0000 7f30aca72640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 0x7f308c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f30a80c4d50 tx=0x7f3098009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:33.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.424+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30a4076a00 con 0x7f30a80709e0 2023-10-21T16:17:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.659+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f30a80b0ff0 con 0x7f30a80709e0 2023-10-21T16:17:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.660+0000 7f309dffb640 1 -- 172.21.15.53:0/564041862 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7f30a403b020 con 0x7f30a80709e0 2023-10-21T16:17:33.661 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:17:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 msgr2=0x7f308c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 0x7f308c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f30a80c4d50 tx=0x7f3098009040 comp rx=0 tx=0).stop 2023-10-21T16:17:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 msgr2=0x7f30a80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f30a402dbf0 tx=0x7f30a40046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 shutdown_connections 2023-10-21T16:17:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f308c06afc0 0x7f308c06d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30a8159ac0 0x7f30a80afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30a80ad790 0x7f30a80af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 --2- 172.21.15.53:0/564041862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30a80709e0 0x7f30a80b15c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.662+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 >> 172.21.15.53:0/564041862 conn(0x7f30a806a380 msgr2=0x7f30a8156560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.663+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 shutdown_connections 2023-10-21T16:17:33.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:33.663+0000 7f30af4d7640 1 -- 172.21.15.53:0/564041862 wait complete. 2023-10-21T16:17:33.684 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:17:33.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:17:33.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:17:33.685 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:17:33.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:33.686 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:17:33.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:17:33.687 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:17:33.687 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:17:33.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:33.688 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:17:33.746 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:17:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:17:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:17:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:17:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.747 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:17:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:17:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:17:33.748 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:17:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:17:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:17:33.749 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:17:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:17:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:17:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:17:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41439sec preferred_lft 41439sec 2023-10-21T16:17:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:17:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.750 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:17:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:17:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:17:33.751 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:17:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:17:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:17:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:17:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.752 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:17:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:17:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:17:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.753 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:17:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:17:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:17:33.754 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:17:33.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:33.755 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:17:33.859 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:17:33.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:33.863 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:17:33.921 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:17:33.921 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:17:33.921 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:17:33.922 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:17:33.922 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:17:33.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:33.924 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:17:33.924 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:17:33.924 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:17:33.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:17:33.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:17:33.927 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:17:34.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:34.128 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:17:34.183 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:17:34.183 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:17:34.184 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:17:34.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:34.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:17:34.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:17:34.291 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:34.351 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:17:34.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:17:34.352 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:34.406 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:17:34.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:34.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:17:34.551 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:17:34.551 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:17:34.585 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:17:34.585 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:17:34.585 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:17:34.585 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:17:34.586 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:17:34.588 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:17:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:17:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:17:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:17:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:17:34.592 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:17:34.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:34.593 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:34.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:34.671 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:17:34.731 INFO:teuthology.orchestra.run.smithi053.stdout:client6795 2023-10-21T16:17:34.732 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:34.732 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:34.786 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:17:35.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.014+0000 7fe88ca76640 1 Processor -- start 2023-10-21T16:17:35.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.014+0000 7fe88ca76640 1 -- start start 2023-10-21T16:17:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe88ca76640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe8880cb130 0x7fe8880cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe88ca76640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8880b1cc0 con 0x7fe88806dfc0 2023-10-21T16:17:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe88ca76640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8880cbad0 con 0x7fe8880afbe0 2023-10-21T16:17:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe88ca76640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8880b2620 con 0x7fe8880cb130 2023-10-21T16:17:35.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe885d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe88806dfc0 0x7fe888070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34284/0 (socket says 172.21.15.53:34284) 2023-10-21T16:17:35.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.016+0000 7fe885d9b640 1 -- 172.21.15.53:0/629247193 learned_addr learned my addr 172.21.15.53:0/629247193 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:35.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe886d9d640 1 --2- 172.21.15.53:0/629247193 >> v2:172.21.15.124:3300/0 conn(0x7fe8880cb130 0x7fe8880cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253130740 0 0) 0x7fe8880b1cc0 con 0x7fe88806dfc0 2023-10-21T16:17:35.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe874003520 con 0x7fe88806dfc0 2023-10-21T16:17:35.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028875831 0 0) 0x7fe8880cbad0 con 0x7fe8880afbe0 2023-10-21T16:17:35.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8880b1cc0 con 0x7fe8880afbe0 2023-10-21T16:17:35.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe886d9d640 1 -- 172.21.15.53:0/629247193 >> v1:172.21.15.124:6789/0 conn(0x7fe8880afbe0 legacy=0x7fe8880b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe886d9d640 1 -- 172.21.15.53:0/629247193 >> v1:172.21.15.53:6789/0 conn(0x7fe88806dfc0 legacy=0x7fe888070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2796372976 0 0) 0x7fe874003520 con 0x7fe88806dfc0 2023-10-21T16:17:35.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.017+0000 7fe886d9d640 1 -- 172.21.15.53:0/629247193 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8880b2060 con 0x7fe8880cb130 2023-10-21T16:17:35.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.019+0000 7fe886d9d640 1 --2- 172.21.15.53:0/629247193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8880cb130 0x7fe8880cb590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8800097a0 tx=0x7fe88002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ecbffa50af77c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.019+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88002f050 con 0x7fe8880cb130 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.019+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880031500 con 0x7fe8880cb130 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.019+0000 7fe88559a640 1 -- 172.21.15.53:0/629247193 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870001670 con 0x7fe8880cb130 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 -- 172.21.15.53:0/629247193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8880cb130 msgr2=0x7fe8880cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 --2- 172.21.15.53:0/629247193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8880cb130 0x7fe8880cb590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8800097a0 tx=0x7fe88002b820 comp rx=0 tx=0).stop 2023-10-21T16:17:35.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 -- 172.21.15.53:0/629247193 shutdown_connections 2023-10-21T16:17:35.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 --2- 172.21.15.53:0/629247193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8880cb130 0x7fe8880cb590 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 -- 172.21.15.53:0/629247193 >> 172.21.15.53:0/629247193 conn(0x7fe88806a5d0 msgr2=0x7fe8880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:35.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 -- 172.21.15.53:0/629247193 shutdown_connections 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.020+0000 7fe88ca76640 1 -- 172.21.15.53:0/629247193 wait complete. 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.022+0000 7fe88ca76640 1 Processor -- start 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.022+0000 7fe88ca76640 1 -- start start 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88806dfc0 0x7fe8881f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8880afbe0 0x7fe8880c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881f2f00 con 0x7fe8880cb130 2023-10-21T16:17:35.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881f2860 con 0x7fe88806dfc0 2023-10-21T16:17:35.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe88ca76640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881f68a0 con 0x7fe8880afbe0 2023-10-21T16:17:35.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.023+0000 7fe886d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53724/0 (socket says 172.21.15.53:53724) 2023-10-21T16:17:35.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 -- 172.21.15.53:0/1589716198 learned_addr learned my addr 172.21.15.53:0/1589716198 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe88659c640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88806dfc0 0x7fe8881f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe885d9b640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8880afbe0 0x7fe8880c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 -- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8880afbe0 msgr2=0x7fe8880c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8880afbe0 0x7fe8880c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 -- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88806dfc0 msgr2=0x7fe8881f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88806dfc0 0x7fe8881f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 -- 172.21.15.53:0/1589716198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880009400 con 0x7fe8880cb130 2023-10-21T16:17:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.024+0000 7fe886d9d640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe880036040 tx=0x7fe8800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.025+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880004230 con 0x7fe8880cb130 2023-10-21T16:17:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.025+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880004390 con 0x7fe8880cb130 2023-10-21T16:17:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.025+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8881f6b80 con 0x7fe8880cb130 2023-10-21T16:17:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.025+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880003850 con 0x7fe8880cb130 2023-10-21T16:17:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.025+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8880c5200 con 0x7fe8880cb130 2023-10-21T16:17:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.026+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8881f3040 con 0x7fe8880cb130 2023-10-21T16:17:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.027+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe88003a040 con 0x7fe8880cb130 2023-10-21T16:17:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.028+0000 7fe87effd640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 0x7fe86806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.028+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe8800aaf10 con 0x7fe8880cb130 2023-10-21T16:17:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.029+0000 7fe88659c640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 0x7fe86806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.030+0000 7fe88659c640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 0x7fe86806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe878001560 tx=0x7fe878009040 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.032+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8881f3040 con 0x7fe8880cb130 2023-10-21T16:17:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.250+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 --> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe888070350 con 0x7fe86806af90 2023-10-21T16:17:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.254+0000 7fe87effd640 1 -- 172.21.15.53:0/1589716198 <== mgr.5734 v2:172.21.15.124:6832/1636058776 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe878001ab0 con 0x7fe86806af90 2023-10-21T16:17:35.255 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:17:35.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:17:35.256 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:17:35.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:17:35.256 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:17:35.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 msgr2=0x7fe86806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 0x7fe86806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe878001560 tx=0x7fe878009040 comp rx=0 tx=0).stop 2023-10-21T16:17:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 msgr2=0x7fe8881f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe880036040 tx=0x7fe8800046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 shutdown_connections 2023-10-21T16:17:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fe86806af90 0x7fe86806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8880cb130 0x7fe8881f5e50 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8880afbe0 0x7fe8880c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 --2- 172.21.15.53:0/1589716198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88806dfc0 0x7fe8881f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 >> 172.21.15.53:0/1589716198 conn(0x7fe88806a5d0 msgr2=0x7fe8880c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.256+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 shutdown_connections 2023-10-21T16:17:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.257+0000 7fe88ca76640 1 -- 172.21.15.53:0/1589716198 wait complete. 2023-10-21T16:17:35.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:17:35.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.501+0000 7f144da83640 1 Processor -- start 2023-10-21T16:17:35.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.501+0000 7f144da83640 1 -- start start 2023-10-21T16:17:35.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f144da83640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f144806dfc0 0x7f1448070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f144da83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14480cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f144da83640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480b1cc0 con 0x7f14480afbe0 2023-10-21T16:17:35.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f144da83640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14480b2620 con 0x7f144806dfc0 2023-10-21T16:17:35.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f144da83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480b2060 con 0x7f1448071070 2023-10-21T16:17:35.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f1446ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f144806dfc0 0x7f1448070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.503+0000 7f1447fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14480cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.504+0000 7f1446ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f144806dfc0 0x7f1448070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:35706/0 (socket says 172.21.15.53:35706) 2023-10-21T16:17:35.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.504+0000 7f1446ffd640 1 -- 172.21.15.53:0/2606380928 learned_addr learned my addr 172.21.15.53:0/2606380928 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:35.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.504+0000 7f1446ffd640 1 -- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 msgr2=0x7f14480cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:35.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.504+0000 7f1446ffd640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14480cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.504+0000 7f1446ffd640 1 -- 172.21.15.53:0/2606380928 >> v1:172.21.15.124:6789/0 conn(0x7f14480afbe0 legacy=0x7f14480b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.505+0000 7f1446ffd640 1 -- 172.21.15.53:0/2606380928 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480b2340 con 0x7f144806dfc0 2023-10-21T16:17:35.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.505+0000 7f1447fff640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14480cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.506+0000 7f1446ffd640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f144806dfc0 0x7f1448070b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f143c009870 tx=0x7f143c02b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609d4cf5ebdb8acf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.506+0000 7f14467fc640 1 -- 172.21.15.53:0/2606380928 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c030710 con 0x7f144806dfc0 2023-10-21T16:17:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f14467fc640 1 -- 172.21.15.53:0/2606380928 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c02e050 con 0x7f144806dfc0 2023-10-21T16:17:35.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f14467fc640 1 -- 172.21.15.53:0/2606380928 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c030a90 con 0x7f144806dfc0 2023-10-21T16:17:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 -- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f144806dfc0 msgr2=0x7f1448070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f144806dfc0 0x7f1448070b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f143c009870 tx=0x7f143c02b7d0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 -- 172.21.15.53:0/2606380928 shutdown_connections 2023-10-21T16:17:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14480cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 --2- 172.21.15.53:0/2606380928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f144806dfc0 0x7f1448070b30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 -- 172.21.15.53:0/2606380928 >> 172.21.15.53:0/2606380928 conn(0x7f144806a5d0 msgr2=0x7f14480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:35.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 -- 172.21.15.53:0/2606380928 shutdown_connections 2023-10-21T16:17:35.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.507+0000 7f144da83640 1 -- 172.21.15.53:0/2606380928 wait complete. 2023-10-21T16:17:35.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.509+0000 7f144da83640 1 Processor -- start 2023-10-21T16:17:35.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.509+0000 7f144da83640 1 -- start start 2023-10-21T16:17:35.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14481f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14480afbe0 0x7f1448159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481fb450 con 0x7f144806dfc0 2023-10-21T16:17:35.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14481f7400 con 0x7f14480afbe0 2023-10-21T16:17:35.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.510+0000 7f144da83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14481f7680 con 0x7f1448071070 2023-10-21T16:17:35.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53734/0 (socket says 172.21.15.53:53734) 2023-10-21T16:17:35.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 -- 172.21.15.53:0/2809717104 learned_addr learned my addr 172.21.15.53:0/2809717104 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:35.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f1447fff640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14480afbe0 0x7f1448159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f1446ffd640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14481f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 -- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 msgr2=0x7f14481f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14481f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 -- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14480afbe0 msgr2=0x7f1448159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:35.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14480afbe0 0x7f1448159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:35.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.511+0000 7f14477fe640 1 -- 172.21.15.53:0/2809717104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009400 con 0x7f144806dfc0 2023-10-21T16:17:35.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f14477fe640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f143800cb20 tx=0x7f14380020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438003da0 con 0x7f144806dfc0 2023-10-21T16:17:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800a6e0 con 0x7f144806dfc0 2023-10-21T16:17:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800aa50 con 0x7f144806dfc0 2023-10-21T16:17:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c0094d0 con 0x7f144806dfc0 2023-10-21T16:17:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.512+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14481f7c40 con 0x7f144806dfc0 2023-10-21T16:17:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.513+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14481f7e30 con 0x7f144806dfc0 2023-10-21T16:17:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.514+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f143800abb0 con 0x7f144806dfc0 2023-10-21T16:17:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.515+0000 7f142ffff640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 0x7f143406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.515+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f14380891d0 con 0x7f144806dfc0 2023-10-21T16:17:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.516+0000 7f1446ffd640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 0x7f143406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.517+0000 7f1446ffd640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 0x7f143406d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f144806e640 tx=0x7f143c008030 comp rx=0 tx=0).ready entity=mgr.5734 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.519+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14380540a0 con 0x7f144806dfc0 2023-10-21T16:17:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:35.760+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1448070320 con 0x7f144806dfc0 2023-10-21T16:17:36.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.379+0000 7f142ffff640 1 -- 172.21.15.53:0/2809717104 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) v1 ==== 92+0+0 (secure 0 0 0) 0x7f143800ae60 con 0x7f144806dfc0 2023-10-21T16:17:36.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 msgr2=0x7f143406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 0x7f143406d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f144806e640 tx=0x7f143c008030 comp rx=0 tx=0).stop 2023-10-21T16:17:36.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 msgr2=0x7f14481faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f143800cb20 tx=0x7f14380020b0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 shutdown_connections 2023-10-21T16:17:36.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f143406af90 0x7f143406d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14480afbe0 0x7f1448159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1448071070 0x7f14481f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 --2- 172.21.15.53:0/2809717104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f144806dfc0 0x7f14481faf10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.383+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 >> 172.21.15.53:0/2809717104 conn(0x7f144806a5d0 msgr2=0x7f14480c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:36.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.384+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 shutdown_connections 2023-10-21T16:17:36.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.384+0000 7f144da83640 1 -- 172.21.15.53:0/2809717104 wait complete. 2023-10-21T16:17:36.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:17:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.606+0000 7f110630a640 1 Processor -- start 2023-10-21T16:17:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.606+0000 7f110630a640 1 -- start start 2023-10-21T16:17:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.608+0000 7f110630a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.608+0000 7f110630a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 0x7f110006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.608+0000 7f110630a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110015b310 con 0x7f110015aa00 2023-10-21T16:17:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.608+0000 7f110630a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000ca230 con 0x7f110006db60 2023-10-21T16:17:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.608+0000 7f110630a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110006d290 con 0x7f110006c3a0 2023-10-21T16:17:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10ff7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 0x7f110006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10ff7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 0x7f110006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:35736/0 (socket says 172.21.15.53:35736) 2023-10-21T16:17:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10ff7fe640 1 -- 172.21.15.53:0/3846373335 learned_addr learned my addr 172.21.15.53:0/3846373335 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 --2- 172.21.15.53:0/3846373335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:42872/0 (socket says 172.21.15.53:42872) 2023-10-21T16:17:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 -- 172.21.15.53:0/3846373335 >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 msgr2=0x7f110006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 --2- 172.21.15.53:0/3846373335 >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 0x7f110006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 -- 172.21.15.53:0/3846373335 >> v1:172.21.15.124:6789/0 conn(0x7f110015aa00 legacy=0x7f11000b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.609+0000 7f10fffff640 1 -- 172.21.15.53:0/3846373335 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000c9860 con 0x7f110006c3a0 2023-10-21T16:17:36.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.611+0000 7f10fffff640 1 --2- 172.21.15.53:0/3846373335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4739eeb93c1dcfc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:36.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.611+0000 7f10feffd640 1 -- 172.21.15.53:0/3846373335 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0031830 con 0x7f110006c3a0 2023-10-21T16:17:36.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.611+0000 7f10feffd640 1 -- 172.21.15.53:0/3846373335 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f0031990 con 0x7f110006c3a0 2023-10-21T16:17:36.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.611+0000 7f10feffd640 1 -- 172.21.15.53:0/3846373335 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0031d20 con 0x7f110006c3a0 2023-10-21T16:17:36.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 -- 172.21.15.53:0/3846373335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 msgr2=0x7f11000c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 --2- 172.21.15.53:0/3846373335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097a0 tx=0x7f10f002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 -- 172.21.15.53:0/3846373335 shutdown_connections 2023-10-21T16:17:36.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 --2- 172.21.15.53:0/3846373335 >> v2:172.21.15.124:3300/0 conn(0x7f110006db60 0x7f110006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 --2- 172.21.15.53:0/3846373335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c9cf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 -- 172.21.15.53:0/3846373335 >> 172.21.15.53:0/3846373335 conn(0x7f110006a5d0 msgr2=0x7f11000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:36.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 -- 172.21.15.53:0/3846373335 shutdown_connections 2023-10-21T16:17:36.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.612+0000 7f110630a640 1 -- 172.21.15.53:0/3846373335 wait complete. 2023-10-21T16:17:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.614+0000 7f110630a640 1 Processor -- start 2023-10-21T16:17:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.614+0000 7f110630a640 1 -- start start 2023-10-21T16:17:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f110006db60 0x7f11000c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000c5140 con 0x7f110015aa00 2023-10-21T16:17:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000c2820 con 0x7f110006db60 2023-10-21T16:17:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.615+0000 7f110630a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000c6950 con 0x7f110006c3a0 2023-10-21T16:17:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53742/0 (socket says 172.21.15.53:53742) 2023-10-21T16:17:36.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 -- 172.21.15.53:0/2260523300 learned_addr learned my addr 172.21.15.53:0/2260523300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:36.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f10fffff640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:36.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f10ff7fe640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f110006db60 0x7f11000c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:36.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 msgr2=0x7f11000c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f110006db60 msgr2=0x7f11000c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f110006db60 0x7f11000c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 -- 172.21.15.53:0/2260523300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009400 con 0x7f110015aa00 2023-10-21T16:17:36.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.616+0000 7f11048a7640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c6b0 tx=0x7f10f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:36.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.617+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003b70 con 0x7f110015aa00 2023-10-21T16:17:36.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.617+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f8003cf0 con 0x7f110015aa00 2023-10-21T16:17:36.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.617+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003ea0 con 0x7f110015aa00 2023-10-21T16:17:36.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.617+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11000c6c30 con 0x7f110015aa00 2023-10-21T16:17:36.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.617+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11000c71f0 con 0x7f110015aa00 2023-10-21T16:17:36.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.618+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110004e160 con 0x7f110015aa00 2023-10-21T16:17:36.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.619+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f10f8005310 con 0x7f110015aa00 2023-10-21T16:17:36.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.620+0000 7f10fcff9640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 0x7f10ec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:36.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.620+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10f8089610 con 0x7f110015aa00 2023-10-21T16:17:36.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.620+0000 7f10fffff640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 msgr2=0x7f10ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:36.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.620+0000 7f10fffff640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 0x7f10ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:36.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.624+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10f80543c0 con 0x7f110015aa00 2023-10-21T16:17:36.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.819+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f110015b310 con 0x7f110015aa00 2023-10-21T16:17:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.821+0000 7f10fffff640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 msgr2=0x7f10ec06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:36.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.821+0000 7f10fffff640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 0x7f10ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:36.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.824+0000 7f10fcff9640 1 -- 172.21.15.53:0/2260523300 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f110015b310 con 0x7f110015aa00 2023-10-21T16:17:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.826+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 msgr2=0x7f10ec06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:36.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.826+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 0x7f10ec06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.826+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 msgr2=0x7f1100157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.826+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c6b0 tx=0x7f10f800cb70 comp rx=0 tx=0).stop 2023-10-21T16:17:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 shutdown_connections 2023-10-21T16:17:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f10ec06afe0 0x7f10ec06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f110015aa00 0x7f1100157170 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f110006db60 0x7f11000c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 --2- 172.21.15.53:0/2260523300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f110006c3a0 0x7f11000c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 >> 172.21.15.53:0/2260523300 conn(0x7f110006a5d0 msgr2=0x7f11000b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 shutdown_connections 2023-10-21T16:17:36.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:36.827+0000 7f110630a640 1 -- 172.21.15.53:0/2260523300 wait complete. 2023-10-21T16:17:36.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:17:36.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:36.903+0000 7f6204232040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:36.917 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:36.916+0000 7f29d362b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:36.957 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:36.956+0000 7f6204232040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:36.974 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:36.972+0000 7f29d362b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:37.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.054+0000 7f42d759e640 1 Processor -- start 2023-10-21T16:17:37.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.054+0000 7f42d759e640 1 -- start start 2023-10-21T16:17:37.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42d80cf690 0x7f42d80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42d80d1100 0x7f42d815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80cd520 con 0x7f42d80cf690 2023-10-21T16:17:37.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80d1910 con 0x7f42d80d1100 2023-10-21T16:17:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42d759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80ce080 con 0x7f42d80cf2b0 2023-10-21T16:17:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42cffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42d80cf690 0x7f42d80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42cffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42d80cf690 0x7f42d80ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53750/0 (socket says 172.21.15.53:53750) 2023-10-21T16:17:37.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.056+0000 7f42cffff640 1 -- 172.21.15.53:0/2896977894 learned_addr learned my addr 172.21.15.53:0/2896977894 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:37.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42d6d9d640 1 --2- 172.21.15.53:0/2896977894 >> v2:172.21.15.124:3300/0 conn(0x7f42d80d1100 0x7f42d815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42d659c640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42cffff640 1 -- 172.21.15.53:0/2896977894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 msgr2=0x7f42d80d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42cffff640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d80d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42cffff640 1 -- 172.21.15.53:0/2896977894 >> v2:172.21.15.124:3300/0 conn(0x7f42d80d1100 msgr2=0x7f42d815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42cffff640 1 --2- 172.21.15.53:0/2896977894 >> v2:172.21.15.124:3300/0 conn(0x7f42d80d1100 0x7f42d815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.057+0000 7f42cffff640 1 -- 172.21.15.53:0/2896977894 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80cd750 con 0x7f42d80cf690 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.059+0000 7f42cffff640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d80ce510 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f42d80d0020 tx=0x7f42c002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341ea715405d057a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.059+0000 7f42d5d9b640 1 -- 172.21.15.53:0/2896977894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00317e0 con 0x7f42d80cf690 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.059+0000 7f42d5d9b640 1 -- 172.21.15.53:0/2896977894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0031940 con 0x7f42d80cf690 2023-10-21T16:17:37.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.059+0000 7f42d5d9b640 1 -- 172.21.15.53:0/2896977894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0031cd0 con 0x7f42d80cf690 2023-10-21T16:17:37.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 -- 172.21.15.53:0/2896977894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 msgr2=0x7f42d80ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d80ce510 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f42d80d0020 tx=0x7f42c002b880 comp rx=0 tx=0).stop 2023-10-21T16:17:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 -- 172.21.15.53:0/2896977894 shutdown_connections 2023-10-21T16:17:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 --2- 172.21.15.53:0/2896977894 >> v2:172.21.15.124:3300/0 conn(0x7f42d80d1100 0x7f42d815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d80ce510 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 --2- 172.21.15.53:0/2896977894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d80d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 -- 172.21.15.53:0/2896977894 >> 172.21.15.53:0/2896977894 conn(0x7f42d809bc40 msgr2=0x7f42d80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 -- 172.21.15.53:0/2896977894 shutdown_connections 2023-10-21T16:17:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.060+0000 7f42d759e640 1 -- 172.21.15.53:0/2896977894 wait complete. 2023-10-21T16:17:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.062+0000 7f42d759e640 1 Processor -- start 2023-10-21T16:17:37.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.062+0000 7f42d759e640 1 -- start start 2023-10-21T16:17:37.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d815f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d80d1100 0x7f42d8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d815c5f0 con 0x7f42d80cf690 2023-10-21T16:17:37.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d815cbb0 con 0x7f42d80d1100 2023-10-21T16:17:37.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42d759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8158900 con 0x7f42d80cf2b0 2023-10-21T16:17:37.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42cffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53754/0 (socket says 172.21.15.53:53754) 2023-10-21T16:17:37.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.063+0000 7f42cffff640 1 -- 172.21.15.53:0/878931367 learned_addr learned my addr 172.21.15.53:0/878931367 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:37.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42d659c640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d815f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42d6d9d640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d80d1100 0x7f42d8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 msgr2=0x7f42d815f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d815f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d80d1100 msgr2=0x7f42d8159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d80d1100 0x7f42d8159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 -- 172.21.15.53:0/878931367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00094d0 con 0x7f42d80cf690 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42d659c640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.064+0000 7f42cffff640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f42c002b8c0 tx=0x7f42c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.065+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0032880 con 0x7f42d80cf690 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.065+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0032a20 con 0x7f42d80cf690 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.065+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0032dd0 con 0x7f42d80cf690 2023-10-21T16:17:37.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.065+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d8158be0 con 0x7f42d80cf690 2023-10-21T16:17:37.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.065+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d815f690 con 0x7f42d80cf690 2023-10-21T16:17:37.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.066+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d8158da0 con 0x7f42d80cf690 2023-10-21T16:17:37.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.067+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f42c004e450 con 0x7f42d80cf690 2023-10-21T16:17:37.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.068+0000 7f42ceffd640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 0x7f42b806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.068+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f42c00abcb0 con 0x7f42d80cf690 2023-10-21T16:17:37.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.068+0000 7f42d659c640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 msgr2=0x7f42b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:37.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.068+0000 7f42d659c640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 0x7f42b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:37.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.076+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42c0076930 con 0x7f42d80cf690 2023-10-21T16:17:37.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.267+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f42d8074150 con 0x7f42d80cf690 2023-10-21T16:17:37.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.269+0000 7f42d659c640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 msgr2=0x7f42b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:37.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.269+0000 7f42d659c640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 0x7f42b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:37.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.273+0000 7f42ceffd640 1 -- 172.21.15.53:0/878931367 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f42c0079f50 con 0x7f42d80cf690 2023-10-21T16:17:37.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.275+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 msgr2=0x7f42b806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:37.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.275+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 0x7f42b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.275+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 msgr2=0x7f42d8159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.275+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f42c002b8c0 tx=0x7f42c00046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 shutdown_connections 2023-10-21T16:17:37.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7f42b806afe0 0x7f42b806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42d80d1100 0x7f42d8159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42d80cf690 0x7f42d8159520 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 --2- 172.21.15.53:0/878931367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42d80cf2b0 0x7f42d815f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 >> 172.21.15.53:0/878931367 conn(0x7f42d809bc40 msgr2=0x7f42d80c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:37.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 shutdown_connections 2023-10-21T16:17:37.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.276+0000 7f42d759e640 1 -- 172.21.15.53:0/878931367 wait complete. 2023-10-21T16:17:37.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:37.296 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T16:17:37.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-21T16:17:37.407 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.406+0000 7f6204232040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:37.458 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.456+0000 7f29d362b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:37.459 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.458+0000 7f6204232040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:37.508 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.507+0000 7f6204232040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:37.513 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.511+0000 7f29d362b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:37.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.558+0000 7fef4b090640 1 Processor -- start 2023-10-21T16:17:37.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.558+0000 7fef4b090640 1 -- start start 2023-10-21T16:17:37.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4b090640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fef4415cf30 0x7fef440d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4b090640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef440cd520 con 0x7fef440cf2b0 2023-10-21T16:17:37.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4b090640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef440d1910 con 0x7fef440cf690 2023-10-21T16:17:37.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4b090640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef440ce080 con 0x7fef4415cf30 2023-10-21T16:17:37.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4a08e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fef440cf2b0 0x7fef440d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34288/0 (socket says 172.21.15.53:34288) 2023-10-21T16:17:37.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4a08e640 1 -- 172.21.15.53:0/3263509425 learned_addr learned my addr 172.21.15.53:0/3263509425 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:37.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.560+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> v2:172.21.15.124:3300/0 conn(0x7fef4415cf30 0x7fef440d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695683336 0 0) 0x7fef440cd520 con 0x7fef440cf2b0 2023-10-21T16:17:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef30003520 con 0x7fef440cf2b0 2023-10-21T16:17:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635383086 0 0) 0x7fef30003520 con 0x7fef440cf2b0 2023-10-21T16:17:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 >> v2:172.21.15.124:3300/0 conn(0x7fef4415cf30 msgr2=0x7fef440d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 --2- 172.21.15.53:0/3263509425 >> v2:172.21.15.124:3300/0 conn(0x7fef4415cf30 0x7fef440d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 >> v1:172.21.15.124:6789/0 conn(0x7fef440cf690 legacy=0x7fef4415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440cd750 con 0x7fef440cf2b0 2023-10-21T16:17:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fef40001670 con 0x7fef440cf2b0 2023-10-21T16:17:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 >> v1:172.21.15.53:6789/0 conn(0x7fef440cf2b0 legacy=0x7fef440d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300047c0 0x7fef30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 0x7fef3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440ce080 con 0x7fef300051d0 2023-10-21T16:17:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3000c120 con 0x7fef300039c0 2023-10-21T16:17:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3000c2e0 con 0x7fef300047c0 2023-10-21T16:17:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.561+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> v2:172.21.15.124:3300/0 conn(0x7fef4415cf30 0x7fef440d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 0x7fef3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4988d640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300047c0 0x7fef30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a08e640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 -- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300047c0 msgr2=0x7fef30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300047c0 0x7fef30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 -- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300039c0 msgr2=0x7fef30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300039c0 0x7fef30003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.562+0000 7fef4a88f640 1 -- 172.21.15.53:0/3263509425 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30003520 con 0x7fef300051d0 2023-10-21T16:17:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.563+0000 7fef4a88f640 1 -- 172.21.15.53:0/3263509425 reap_dead start 2023-10-21T16:17:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.564+0000 7fef4a88f640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 0x7fef3000b980 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fef4415d3a0 tx=0x7fef300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5f6998c49f293e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.564+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c02cd70 con 0x7fef300051d0 2023-10-21T16:17:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.564+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00f040 con 0x7fef300051d0 2023-10-21T16:17:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.564+0000 7fef4908c640 1 -- 172.21.15.53:0/3263509425 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c003bb0 con 0x7fef300051d0 2023-10-21T16:17:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 -- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 msgr2=0x7fef3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 0x7fef3000b980 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fef4415d3a0 tx=0x7fef300057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 -- 172.21.15.53:0/3263509425 shutdown_connections 2023-10-21T16:17:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 --2- 172.21.15.53:0/3263509425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef300051d0 0x7fef3000b980 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 -- 172.21.15.53:0/3263509425 >> 172.21.15.53:0/3263509425 conn(0x7fef4409bc40 msgr2=0x7fef440d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 -- 172.21.15.53:0/3263509425 shutdown_connections 2023-10-21T16:17:37.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.565+0000 7fef4b090640 1 -- 172.21.15.53:0/3263509425 wait complete. 2023-10-21T16:17:37.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.568+0000 7f6204232040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:37.576 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.566+0000 7f29d362b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.567+0000 7fef4b090640 1 Processor -- start 2023-10-21T16:17:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.567+0000 7fef4b090640 1 -- start start 2023-10-21T16:17:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300051d0 0x7fef440b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef441f6960 0x7fef441f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef441f62e0 0x7fef4415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef441f6760 con 0x7fef441f6960 2023-10-21T16:17:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4415dc90 con 0x7fef300051d0 2023-10-21T16:17:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4b090640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef440b4960 con 0x7fef441f62e0 2023-10-21T16:17:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4988d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef441f6960 0x7fef441f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4988d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef441f6960 0x7fef441f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53776/0 (socket says 172.21.15.53:53776) 2023-10-21T16:17:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.568+0000 7fef4988d640 1 -- 172.21.15.53:0/2736786177 learned_addr learned my addr 172.21.15.53:0/2736786177 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:37.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4a88f640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef441f62e0 0x7fef4415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300051d0 0x7fef440b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:37.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef441f62e0 msgr2=0x7fef4415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef441f62e0 0x7fef4415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300051d0 msgr2=0x7fef440b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300051d0 0x7fef440b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440b4c40 con 0x7fef441f6960 2023-10-21T16:17:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300051d0 0x7fef440b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4988d640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef441f6960 0x7fef441f5da0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e4b0 tx=0x7fef3400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340052e0 con 0x7fef441f6960 2023-10-21T16:17:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef34005440 con 0x7fef441f6960 2023-10-21T16:17:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef4b090640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef440b4f20 con 0x7fef441f6960 2023-10-21T16:17:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.569+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34003830 con 0x7fef441f6960 2023-10-21T16:17:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.570+0000 7fef4b090640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef440b5200 con 0x7fef441f6960 2023-10-21T16:17:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.571+0000 7fef4b090640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef44074150 con 0x7fef441f6960 2023-10-21T16:17:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.572+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fef34003990 con 0x7fef441f6960 2023-10-21T16:17:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.573+0000 7fef3affd640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.573+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fef340895b0 con 0x7fef441f6960 2023-10-21T16:17:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.573+0000 7fef4a08e640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 msgr2=0x7fef2806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.573+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.577+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef34054470 con 0x7fef441f6960 2023-10-21T16:17:37.619 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.618+0000 7f6204232040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:37.639 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.637+0000 7f29d362b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:37.691 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.689+0000 7f29d362b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:37.730 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.728+0000 7f6204232040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:37.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.774+0000 7fef4a08e640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 msgr2=0x7fef2806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:37.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.774+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:37.792 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.791+0000 7f6204232040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:37.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:37.798+0000 7fef4b090640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fef440cdd10 con 0x7fef2806afb0 2023-10-21T16:17:37.809 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.807+0000 7f29d362b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:37.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.840+0000 7f6204232040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:37.876 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.873+0000 7f29d362b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:37.898 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.897+0000 7f6204232040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:37.927 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.925+0000 7f29d362b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:37.948 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:37.947+0000 7f6204232040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:37.986 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:37.984+0000 7f29d362b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:38.039 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.036+0000 7f29d362b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:38.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:38.175+0000 7fef4a08e640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 msgr2=0x7fef2806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:38.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:38.175+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:17:38.324 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.323+0000 7f6204232040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:38.438 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.436+0000 7f29d362b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:38.485 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.483+0000 7f6204232040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:38.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.593+0000 7f6204232040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:38.606 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.604+0000 7f29d362b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:38.644 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.643+0000 7f6204232040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:38.719 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.716+0000 7f29d362b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:38.751 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.750+0000 7f6204232040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:38.771 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.769+0000 7f29d362b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:38.857 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:38.856+0000 7f6204232040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:38.884 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.882+0000 7f29d362b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:38.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:38.976+0000 7fef4a08e640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 msgr2=0x7fef2806d470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1636058776 2023-10-21T16:17:38.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:38.976+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:17:38.997 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:38.995+0000 7f29d362b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:39.013 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:39.011+0000 7f6204232040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:39.061 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:39.060+0000 7f6204232040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:39.163 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:39.160+0000 7f29d362b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:39.181 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:39.179+0000 7f6204232040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:39.214 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:39.212+0000 7f29d362b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:39.339 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:39.336+0000 7f29d362b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:40.001 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:39.999+0000 7f6204232040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:40.181 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:40.178+0000 7f29d362b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:40.293 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.292+0000 7f6204232040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:40.349 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.348+0000 7f6204232040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:40.463 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.462+0000 7f6204232040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:40.479+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 32) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fef34010db0 con 0x7fef441f6960 2023-10-21T16:17:40.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:40.480+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 msgr2=0x7fef2806d470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:40.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:40.480+0000 7fef3affd640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/1636058776,v1:172.21.15.124:6833/1636058776] conn(0x7fef2806afb0 0x7fef2806d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:40.499 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:40.497+0000 7f29d362b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:40.566 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:40.564+0000 7f29d362b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:40.641 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f61835be640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.641 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f61835be640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.641 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f61835be640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f61835be640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f61835be640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f6180578640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f6180578640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f6180578640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.643 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f6180578640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.643 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:40.640+0000 7f6180578640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:40.691 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:40.689+0000 7f29d362b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:41.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.493+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 33) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fef34054840 con 0x7fef441f6960 2023-10-21T16:17:41.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.494+0000 7fef3affd640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fef28035850 0x7fef28075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.494+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 --> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fef28035c30 con 0x7fef28035850 2023-10-21T16:17:41.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.500+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fef28035850 0x7fef28075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.510+0000 7fef4a08e640 1 --2- 172.21.15.53:0/2736786177 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fef28035850 0x7fef28075410 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef40001eb0 tx=0x7fef40009040 comp rx=0 tx=0).ready entity=mgr.35444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:41.530 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:41.529+0000 7f618a5cc640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-21T16:17:41.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.528+0000 7fef3affd640 1 -- 172.21.15.53:0/2736786177 <== mgr.35444 v2:172.21.15.124:6832/2480922944 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fef28035c30 con 0x7fef28035850 2023-10-21T16:17:41.531 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-21T16:17:41.549 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:17:41.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:41.550 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T16:17:41.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:17:41.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.827+0000 7f7e98b28640 1 Processor -- start 2023-10-21T16:17:41.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.827+0000 7f7e98b28640 1 -- start start 2023-10-21T16:17:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.829+0000 7f7e98b28640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 0x7f7e940cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.829+0000 7f7e98b28640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7e940c8b40 0x7f7e940c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.829+0000 7f7e98b28640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e940c94e0 con 0x7f7e940aff70 2023-10-21T16:17:41.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.829+0000 7f7e98b28640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e940ccbc0 con 0x7f7e940c8b40 2023-10-21T16:17:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.829+0000 7f7e98b28640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940ce1f0 con 0x7f7e9415cae0 2023-10-21T16:17:41.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.830+0000 7f7e8bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7e940c8b40 0x7f7e940c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.830+0000 7f7e937fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 0x7f7e940cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.830+0000 7f7e8bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7e940c8b40 0x7f7e940c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:35776/0 (socket says 172.21.15.53:35776) 2023-10-21T16:17:41.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.830+0000 7f7e8bfff640 1 -- 172.21.15.53:0/842652009 learned_addr learned my addr 172.21.15.53:0/842652009 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:41.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.831+0000 7f7e8bfff640 1 -- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 msgr2=0x7f7e940cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.831+0000 7f7e8bfff640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 0x7f7e940cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.831+0000 7f7e8bfff640 1 -- 172.21.15.53:0/842652009 >> v1:172.21.15.124:6789/0 conn(0x7f7e940aff70 legacy=0x7f7e94070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.831+0000 7f7e8bfff640 1 -- 172.21.15.53:0/842652009 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e940cd590 con 0x7f7e940c8b40 2023-10-21T16:17:41.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.831+0000 7f7e937fe640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 0x7f7e940cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e8bfff640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940c8b40 0x7f7e940c8fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009870 tx=0x7f7e7c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53ec3e402651302 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:41.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e92ffd640 1 -- 172.21.15.53:0/842652009 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c030840 con 0x7f7e940c8b40 2023-10-21T16:17:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e92ffd640 1 -- 172.21.15.53:0/842652009 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c0309a0 con 0x7f7e940c8b40 2023-10-21T16:17:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e92ffd640 1 -- 172.21.15.53:0/842652009 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c030d30 con 0x7f7e940c8b40 2023-10-21T16:17:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 -- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940c8b40 msgr2=0x7f7e940c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940c8b40 0x7f7e940c8fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009870 tx=0x7f7e7c02b980 comp rx=0 tx=0).stop 2023-10-21T16:17:41.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 -- 172.21.15.53:0/842652009 shutdown_connections 2023-10-21T16:17:41.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940c8b40 0x7f7e940c8fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:41.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 --2- 172.21.15.53:0/842652009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e9415cae0 0x7f7e940cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:41.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 -- 172.21.15.53:0/842652009 >> 172.21.15.53:0/842652009 conn(0x7f7e9406c050 msgr2=0x7f7e9406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:41.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 -- 172.21.15.53:0/842652009 shutdown_connections 2023-10-21T16:17:41.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.834+0000 7f7e98b28640 1 -- 172.21.15.53:0/842652009 wait complete. 2023-10-21T16:17:41.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.836+0000 7f7e98b28640 1 Processor -- start 2023-10-21T16:17:41.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.837+0000 7f7e98b28640 1 -- start start 2023-10-21T16:17:41.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 0x7f7e9414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e940c8b40 0x7f7e940c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415cec0 con 0x7f7e9415cae0 2023-10-21T16:17:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415aab0 con 0x7f7e940aff70 2023-10-21T16:17:41.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e98b28640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415ad90 con 0x7f7e940c8b40 2023-10-21T16:17:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53778/0 (socket says 172.21.15.53:53778) 2023-10-21T16:17:41.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.838+0000 7f7e93fff640 1 -- 172.21.15.53:0/2534594489 learned_addr learned my addr 172.21.15.53:0/2534594489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e937fe640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 0x7f7e9414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e8bfff640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e940c8b40 0x7f7e940c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 -- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e940c8b40 msgr2=0x7f7e940c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e940c8b40 0x7f7e940c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:41.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 -- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 msgr2=0x7f7e9414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 0x7f7e9414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 -- 172.21.15.53:0/2534594489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84009400 con 0x7f7e9415cae0 2023-10-21T16:17:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e937fe640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 0x7f7e9414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.839+0000 7f7e93fff640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c7a0 tx=0x7f7e8c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:41.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.840+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c003b70 con 0x7f7e9415cae0 2023-10-21T16:17:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.840+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c003cf0 con 0x7f7e9415cae0 2023-10-21T16:17:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.840+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c003ea0 con 0x7f7e9415cae0 2023-10-21T16:17:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.840+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c0094d0 con 0x7f7e9415cae0 2023-10-21T16:17:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.840+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9415d110 con 0x7f7e9415cae0 2023-10-21T16:17:41.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.841+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9415b2b0 con 0x7f7e9415cae0 2023-10-21T16:17:41.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.842+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 33) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f7e8c00aa90 con 0x7f7e9415cae0 2023-10-21T16:17:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.843+0000 7f7e90ff9640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 0x7f7e7406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.843+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f7e8c089210 con 0x7f7e9415cae0 2023-10-21T16:17:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.844+0000 7f7e937fe640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 0x7f7e7406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:41.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.846+0000 7f7e937fe640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 0x7f7e7406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009c80 tx=0x7f7e84008040 comp rx=0 tx=0).ready entity=mgr.35444 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:41.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:41.849+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e8c054330 con 0x7f7e9415cae0 2023-10-21T16:17:42.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.093+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f7e940c94e0 con 0x7f7e9415cae0 2023-10-21T16:17:42.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.539+0000 7f7e90ff9640 1 -- 172.21.15.53:0/2534594489 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7e940c94e0 con 0x7f7e9415cae0 2023-10-21T16:17:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.544+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 msgr2=0x7f7e7406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.544+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 0x7f7e7406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009c80 tx=0x7f7e84008040 comp rx=0 tx=0).stop 2023-10-21T16:17:42.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.544+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 msgr2=0x7f7e941f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:42.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.544+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00c7a0 tx=0x7f7e8c00cc60 comp rx=0 tx=0).stop 2023-10-21T16:17:42.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 shutdown_connections 2023-10-21T16:17:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7e7406ad10 0x7f7e7406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e9415cae0 0x7f7e941f5c30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e940c8b40 0x7f7e940c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 --2- 172.21.15.53:0/2534594489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e940aff70 0x7f7e9414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:42.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 >> 172.21.15.53:0/2534594489 conn(0x7f7e9406c050 msgr2=0x7f7e94071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 shutdown_connections 2023-10-21T16:17:42.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:42.545+0000 7f7e98b28640 1 -- 172.21.15.53:0/2534594489 wait complete. 2023-10-21T16:17:42.569 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:17:42.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:42.686 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:42.780 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:17:42.781 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:42.781 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:42.836 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:42.837 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:42.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:43.062 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.061+0000 7fc2bf632040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:43.099 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.098+0000 7fae03e37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:17:43.115 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.114+0000 7fc2bf632040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:43.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.147+0000 7f7d649d5640 1 Processor -- start 2023-10-21T16:17:43.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.148+0000 7f7d649d5640 1 -- start start 2023-10-21T16:17:43.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.149+0000 7f7d649d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d600cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.149+0000 7f7d649d5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6015b2b0 con 0x7f7d6015ab40 2023-10-21T16:17:43.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.149+0000 7f7d649d5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6015b590 con 0x7f7d600cb3b0 2023-10-21T16:17:43.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.149+0000 7f7d649d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6015bf30 con 0x7f7d600c90b0 2023-10-21T16:17:43.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.150+0000 7f7d5e59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d6015ab40 0x7f7d600ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34294/0 (socket says 172.21.15.53:34294) 2023-10-21T16:17:43.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.150+0000 7f7d5e59c640 1 -- 172.21.15.53:0/3431773796 learned_addr learned my addr 172.21.15.53:0/3431773796 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:43.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.150+0000 7f7d57fff640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d600cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.150+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256455622 0 0) 0x7f7d6015b2b0 con 0x7f7d6015ab40 2023-10-21T16:17:43.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.150+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d4c003520 con 0x7f7d6015ab40 2023-10-21T16:17:43.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794571621 0 0) 0x7f7d4c003520 con 0x7f7d6015ab40 2023-10-21T16:17:43.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 msgr2=0x7f7d600cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d600cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 >> v1:172.21.15.124:6789/0 conn(0x7f7d600cb3b0 legacy=0x7f7d6015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6015b870 con 0x7f7d6015ab40 2023-10-21T16:17:43.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7d50001670 con 0x7f7d6015ab40 2023-10-21T16:17:43.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 >> v1:172.21.15.53:6789/0 conn(0x7f7d6015ab40 legacy=0x7f7d600ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d4c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d4c003ee0 0x7f7d4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d4c004d00 0x7f7d4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6015bf30 con 0x7f7d4c004d00 2023-10-21T16:17:43.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c00b9e0 con 0x7f7d4c003ee0 2023-10-21T16:17:43.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c00bba0 con 0x7f7d600c90b0 2023-10-21T16:17:43.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5ed9d640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d4c004d00 0x7f7d4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.151+0000 7f7d5e59c640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d4c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d57fff640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d4c003ee0 0x7f7d4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d5ed9d640 1 -- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 msgr2=0x7f7d4c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d5ed9d640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d600c90b0 0x7f7d4c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d5ed9d640 1 -- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d4c003ee0 msgr2=0x7f7d4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d5ed9d640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d4c003ee0 0x7f7d4c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.152+0000 7f7d5ed9d640 1 -- 172.21.15.53:0/3431773796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c003520 con 0x7f7d4c004d00 2023-10-21T16:17:43.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.153+0000 7f7d5ed9d640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d4c004d00 0x7f7d4c005180 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7d58001f60 tx=0x7f7d4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fe967ae29c3f05 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:43.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.153+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802d6f0 con 0x7f7d4c004d00 2023-10-21T16:17:43.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.153+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5802d870 con 0x7f7d4c004d00 2023-10-21T16:17:43.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d5dd9b640 1 -- 172.21.15.53:0/3431773796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802dc20 con 0x7f7d4c004d00 2023-10-21T16:17:43.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d649d5640 1 -- 172.21.15.53:0/3431773796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d4c004d00 msgr2=0x7f7d4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d649d5640 1 --2- 172.21.15.53:0/3431773796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d4c004d00 0x7f7d4c005180 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7d58001f60 tx=0x7f7d4c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d5ed9d640 1 -- 172.21.15.53:0/3431773796 reap_dead start 2023-10-21T16:17:43.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d649d5640 1 -- 172.21.15.53:0/3431773796 shutdown_connections 2023-10-21T16:17:43.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.154+0000 7f7d649d5640 1 -- 172.21.15.53:0/3431773796 >> 172.21.15.53:0/3431773796 conn(0x7f7d6006a380 msgr2=0x7f7d600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:43.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.155+0000 7f7d649d5640 1 -- 172.21.15.53:0/3431773796 shutdown_connections 2023-10-21T16:17:43.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.155+0000 7f7d649d5640 1 -- 172.21.15.53:0/3431773796 wait complete. 2023-10-21T16:17:43.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.156+0000 7f7d649d5640 1 Processor -- start 2023-10-21T16:17:43.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 -- start start 2023-10-21T16:17:43.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6014b370 0x7f7d600c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d6014e220 0x7f7d6014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d600c7e80 con 0x7f7d6015d8d0 2023-10-21T16:17:43.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014ebe0 con 0x7f7d6014e220 2023-10-21T16:17:43.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.157+0000 7f7d649d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014b750 con 0x7f7d6014b370 2023-10-21T16:17:43.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53810/0 (socket says 172.21.15.53:53810) 2023-10-21T16:17:43.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 -- 172.21.15.53:0/2729449596 learned_addr learned my addr 172.21.15.53:0/2729449596 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d5ed9d640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d6014e220 0x7f7d6014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6014b370 msgr2=0x7f7d600c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d5e59c640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6014b370 0x7f7d600c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6014b370 0x7f7d600c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d6014e220 msgr2=0x7f7d6014e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d6014e220 0x7f7d6014e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.158+0000 7f7d57fff640 1 -- 172.21.15.53:0/2729449596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600c74e0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d57fff640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7d480119a0 tx=0x7f7d48011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:43.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800faf0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4800fc50 con 0x7f7d6015d8d0 2023-10-21T16:17:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d48014440 con 0x7f7d6015d8d0 2023-10-21T16:17:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d600c77c0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.159+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d600c5b10 con 0x7f7d6015d8d0 2023-10-21T16:17:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.160+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d60067490 con 0x7f7d6015d8d0 2023-10-21T16:17:43.170 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.161+0000 7fae03e37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:17:43.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.161+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7d480145a0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.163+0000 7f7d56ffd640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 0x7f7d4006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.163+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f7d4808a240 con 0x7f7d6015d8d0 2023-10-21T16:17:43.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.163+0000 7f7d5e59c640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 msgr2=0x7f7d4006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:43.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.163+0000 7f7d5e59c640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 0x7f7d4006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:43.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.167+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d48054fa0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.362+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d600b4780 con 0x7f7d6015d8d0 2023-10-21T16:17:43.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.364+0000 7f7d56ffd640 1 -- 172.21.15.53:0/2729449596 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17282 (secure 0 0 0) 0x7f7d480585c0 con 0x7f7d6015d8d0 2023-10-21T16:17:43.366 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:43.366 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":91,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:40.466887+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1810216649":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6833/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6832/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/2018576779":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3111202668":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/356555339":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3273170228":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3109992220":"2023-10-22T16:17:40.466814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:43.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.364+0000 7f7d5e59c640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 msgr2=0x7f7d4006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:43.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.364+0000 7f7d5e59c640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 0x7f7d4006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:43.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.366+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 msgr2=0x7f7d4006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:43.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.366+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 0x7f7d4006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.366+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 msgr2=0x7f7d6015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.366+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7d480119a0 tx=0x7f7d48011e60 comp rx=0 tx=0).stop 2023-10-21T16:17:43.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 shutdown_connections 2023-10-21T16:17:43.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f7d4006afc0 0x7f7d4006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d6014e220 0x7f7d6014e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d6015d8d0 0x7f7d6015dd50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 --2- 172.21.15.53:0/2729449596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6014b370 0x7f7d600c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 >> 172.21.15.53:0/2729449596 conn(0x7f7d6006a380 msgr2=0x7f7d600cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:43.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 shutdown_connections 2023-10-21T16:17:43.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.367+0000 7f7d649d5640 1 -- 172.21.15.53:0/2729449596 wait complete. 2023-10-21T16:17:43.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:17:23.670145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:17:24.763686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:43.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:43.561 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.560+0000 7fc2bf632040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:43.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.603+0000 7f663bfff640 1 Processor -- start 2023-10-21T16:17:43.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.603+0000 7f663bfff640 1 -- start start 2023-10-21T16:17:43.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.604+0000 7f663bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f663c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.604+0000 7f663bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0ce290 con 0x7f663c0d1120 2023-10-21T16:17:43.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.604+0000 7f663bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c0d0a80 con 0x7f663c0d1500 2023-10-21T16:17:43.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.604+0000 7f663bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b1fe0 con 0x7f663c15dcd0 2023-10-21T16:17:43.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f663affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f663c0d1120 0x7f663c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34300/0 (socket says 172.21.15.53:34300) 2023-10-21T16:17:43.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f663affd640 1 -- 172.21.15.53:0/277510932 learned_addr learned my addr 172.21.15.53:0/277510932 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f663b7fe640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f663c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7152707 0 0) 0x7f663c0ce290 con 0x7f663c0d1120 2023-10-21T16:17:43.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624003520 con 0x7f663c0d1120 2023-10-21T16:17:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 235287721 0 0) 0x7f6624003520 con 0x7f663c0d1120 2023-10-21T16:17:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 msgr2=0x7f663c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f663c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 >> v1:172.21.15.124:6789/0 conn(0x7f663c0d1500 legacy=0x7f663c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.605+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0b4580 con 0x7f663c0d1120 2023-10-21T16:17:43.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f662c0017f0 con 0x7f663c0d1120 2023-10-21T16:17:43.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 >> v1:172.21.15.53:6789/0 conn(0x7f663c0d1120 legacy=0x7f663c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66240039a0 0x7f6624003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f66240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6624004d00 0x7f6624005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b1fe0 con 0x7f66240039a0 2023-10-21T16:17:43.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662400bc30 con 0x7f6624004d00 2023-10-21T16:17:43.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662400bdf0 con 0x7f663c15dcd0 2023-10-21T16:17:43.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66240039a0 0x7f6624003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663b7fe640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6624004d00 0x7f6624005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663a7fc640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f66240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 -- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 msgr2=0x7f66240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f663c15dcd0 0x7f66240047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 -- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6624004d00 msgr2=0x7f6624005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:43.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6624004d00 0x7f6624005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.606+0000 7f663affd640 1 -- 172.21.15.53:0/277510932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624003520 con 0x7f66240039a0 2023-10-21T16:17:43.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f663affd640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66240039a0 0x7f6624003dc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f662c00c0e0 tx=0x7f66240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b0c5a3d23dca45 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:43.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c007270 con 0x7f66240039a0 2023-10-21T16:17:43.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c0073d0 con 0x7f66240039a0 2023-10-21T16:17:43.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f6639ffb640 1 -- 172.21.15.53:0/277510932 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c02c7e0 con 0x7f66240039a0 2023-10-21T16:17:43.618 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.613+0000 7fc2bf632040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:43.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f663bfff640 1 -- 172.21.15.53:0/277510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66240039a0 msgr2=0x7f6624003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f663bfff640 1 --2- 172.21.15.53:0/277510932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66240039a0 0x7f6624003dc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f662c00c0e0 tx=0x7f66240057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.608+0000 7f663b7fe640 1 -- 172.21.15.53:0/277510932 reap_dead start 2023-10-21T16:17:43.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.609+0000 7f663bfff640 1 -- 172.21.15.53:0/277510932 shutdown_connections 2023-10-21T16:17:43.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.609+0000 7f663bfff640 1 -- 172.21.15.53:0/277510932 >> 172.21.15.53:0/277510932 conn(0x7f663c0d2960 msgr2=0x7f663c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:43.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.609+0000 7f663bfff640 1 -- 172.21.15.53:0/277510932 shutdown_connections 2023-10-21T16:17:43.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.609+0000 7f663bfff640 1 -- 172.21.15.53:0/277510932 wait complete. 2023-10-21T16:17:43.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.610+0000 7f663bfff640 1 Processor -- start 2023-10-21T16:17:43.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 -- start start 2023-10-21T16:17:43.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66240039a0 0x7f663c15e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 0x7f663c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0caf10 con 0x7f663c15f2b0 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b0c30 con 0x7f663c0ca550 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.611+0000 7f663bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b0060 con 0x7f66240039a0 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53828/0 (socket says 172.21.15.53:53828) 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 -- 172.21.15.53:0/2976532728 learned_addr learned my addr 172.21.15.53:0/2976532728 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663a7fc640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 0x7f663c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663affd640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66240039a0 0x7f663c15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 -- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66240039a0 msgr2=0x7f663c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66240039a0 0x7f663c15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 -- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 msgr2=0x7f663c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 0x7f663c0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663b7fe640 1 -- 172.21.15.53:0/2976532728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0b0300 con 0x7f663c15f2b0 2023-10-21T16:17:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.612+0000 7f663a7fc640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 0x7f663c0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f663b7fe640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f663c15e0b0 tx=0x7f6634009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634007db0 con 0x7f663c15f2b0 2023-10-21T16:17:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663400f050 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c0b05e0 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634014650 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.613+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c15ee60 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.614+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c074150 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.615+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f66340147b0 con 0x7f663c15f2b0 2023-10-21T16:17:43.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.616+0000 7f66237fe640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 0x7f662806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:43.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.616+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f663408a070 con 0x7f663c15f2b0 2023-10-21T16:17:43.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.617+0000 7f663affd640 1 -- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 msgr2=0x7f662806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:43.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.617+0000 7f663affd640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 0x7f662806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:43.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.621+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6634054cc0 con 0x7f663c15f2b0 2023-10-21T16:17:43.646 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.645+0000 7fae03e37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:17:43.664 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.663+0000 7fc2bf632040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:43.702 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.700+0000 7fae03e37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:17:43.725 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.724+0000 7fc2bf632040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:43.755 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.753+0000 7fae03e37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:17:43.774 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.773+0000 7fc2bf632040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:43.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.810+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f663c160d30 con 0x7f663c15f2b0 2023-10-21T16:17:43.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.811+0000 7f66237fe640 1 -- 172.21.15.53:0/2976532728 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66340582e0 con 0x7f663c15f2b0 2023-10-21T16:17:43.812 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:43.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.813+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 msgr2=0x7f662806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:43.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.813+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 0x7f662806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.813+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 msgr2=0x7f663c15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.813+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f663c15e0b0 tx=0x7f6634009da0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 shutdown_connections 2023-10-21T16:17:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f662806af90 0x7f662806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f663c15f2b0 0x7f663c15f730 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f663c0ca550 0x7f663c0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 --2- 172.21.15.53:0/2976532728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66240039a0 0x7f663c15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:43.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 >> 172.21.15.53:0/2976532728 conn(0x7f663c0d2960 msgr2=0x7f663c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:43.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 shutdown_connections 2023-10-21T16:17:43.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:43.814+0000 7f663bfff640 1 -- 172.21.15.53:0/2976532728 wait complete. 2023-10-21T16:17:43.820 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.818+0000 7fae03e37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:17:43.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:17:43.873 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.871+0000 7fae03e37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:17:43.885 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.884+0000 7fc2bf632040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:43.946 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.946+0000 7fc2bf632040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:43.991 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:43.990+0000 7fae03e37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:17:43.995 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:43.994+0000 7fc2bf632040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:44.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.031+0000 7fd03554e640 1 Processor -- start 2023-10-21T16:17:44.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.031+0000 7fd03554e640 1 -- start start 2023-10-21T16:17:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0300ce290 0x7fd0300ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd030065ab0 0x7fd0301608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 0x7fd030160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03006e6c0 con 0x7fd0300ce290 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd030069d50 con 0x7fd03015dde0 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd03554e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03006c750 con 0x7fd030065ab0 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd02ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0300ce290 0x7fd0300ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd02ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0300ce290 0x7fd0300ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56578/0 (socket says 172.21.15.53:56578) 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.033+0000 7fd02ffff640 1 -- 172.21.15.53:0/2857993876 learned_addr learned my addr 172.21.15.53:0/2857993876 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:44.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd034d4d640 1 --2- 172.21.15.53:0/2857993876 >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 0x7fd030160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02f7fe640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd030065ab0 0x7fd0301608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02ffff640 1 -- 172.21.15.53:0/2857993876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd030065ab0 msgr2=0x7fd0301608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:44.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02ffff640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd030065ab0 0x7fd0301608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02ffff640 1 -- 172.21.15.53:0/2857993876 >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 msgr2=0x7fd030160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:44.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02ffff640 1 --2- 172.21.15.53:0/2857993876 >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 0x7fd030160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd02ffff640 1 -- 172.21.15.53:0/2857993876 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0300cebb0 con 0x7fd0300ce290 2023-10-21T16:17:44.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.034+0000 7fd034d4d640 1 --2- 172.21.15.53:0/2857993876 >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 0x7fd030160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:44.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.035+0000 7fd02ffff640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0300ce290 0x7fd0300ce670 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ae2d0 tx=0x7fd02002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1ccfa1fc97c2a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.036+0000 7fd02effd640 1 -- 172.21.15.53:0/2857993876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020031860 con 0x7fd0300ce290 2023-10-21T16:17:44.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.036+0000 7fd02effd640 1 -- 172.21.15.53:0/2857993876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0200319c0 con 0x7fd0300ce290 2023-10-21T16:17:44.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.036+0000 7fd02effd640 1 -- 172.21.15.53:0/2857993876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020031d50 con 0x7fd0300ce290 2023-10-21T16:17:44.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.036+0000 7fd03554e640 1 -- 172.21.15.53:0/2857993876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0300ce290 msgr2=0x7fd0300ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0300ce290 0x7fd0300ce670 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ae2d0 tx=0x7fd02002b900 comp rx=0 tx=0).stop 2023-10-21T16:17:44.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 -- 172.21.15.53:0/2857993876 shutdown_connections 2023-10-21T16:17:44.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd030065ab0 0x7fd0301608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 --2- 172.21.15.53:0/2857993876 >> v2:172.21.15.124:3300/0 conn(0x7fd03015dde0 0x7fd030160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 --2- 172.21.15.53:0/2857993876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0300ce290 0x7fd0300ce670 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 -- 172.21.15.53:0/2857993876 >> 172.21.15.53:0/2857993876 conn(0x7fd0300b08d0 msgr2=0x7fd0300b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:44.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 -- 172.21.15.53:0/2857993876 shutdown_connections 2023-10-21T16:17:44.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.037+0000 7fd03554e640 1 -- 172.21.15.53:0/2857993876 wait complete. 2023-10-21T16:17:44.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.039+0000 7fd03554e640 1 Processor -- start 2023-10-21T16:17:44.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.039+0000 7fd03554e640 1 -- start start 2023-10-21T16:17:44.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 0x7fd03009f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd03015dde0 0x7fd0300b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300c6220 con 0x7fd030065ab0 2023-10-21T16:17:44.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300c6500 con 0x7fd0300ce290 2023-10-21T16:17:44.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd03554e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300c67e0 con 0x7fd03015dde0 2023-10-21T16:17:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56584/0 (socket says 172.21.15.53:56584) 2023-10-21T16:17:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.040+0000 7fd02ffff640 1 -- 172.21.15.53:0/3848039970 learned_addr learned my addr 172.21.15.53:0/3848039970 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd034d4d640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd03015dde0 0x7fd0300b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02f7fe640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 0x7fd03009f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd03015dde0 msgr2=0x7fd0300b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd03015dde0 0x7fd0300b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 msgr2=0x7fd03009f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 0x7fd03009f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 -- 172.21.15.53:0/3848039970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200094d0 con 0x7fd030065ab0 2023-10-21T16:17:44.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02f7fe640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 0x7fd03009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:44.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02ffff640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd0200097c0 tx=0x7fd0200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02003ac60 con 0x7fd030065ab0 2023-10-21T16:17:44.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02003adc0 con 0x7fd030065ab0 2023-10-21T16:17:44.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0300c6ac0 con 0x7fd030065ab0 2023-10-21T16:17:44.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.041+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020044610 con 0x7fd030065ab0 2023-10-21T16:17:44.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.042+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0300c5eb0 con 0x7fd030065ab0 2023-10-21T16:17:44.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.043+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03006b6b0 con 0x7fd030065ab0 2023-10-21T16:17:44.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.043+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd02004d450 con 0x7fd030065ab0 2023-10-21T16:17:44.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.044+0000 7fd02cff9640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 0x7fd01806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.044+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fd0200ac0e0 con 0x7fd030065ab0 2023-10-21T16:17:44.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.045+0000 7fd02f7fe640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 msgr2=0x7fd01806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:44.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.045+0000 7fd02f7fe640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 0x7fd01806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:44.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.048+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd020076c00 con 0x7fd030065ab0 2023-10-21T16:17:44.053 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.052+0000 7fc2bf632040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:44.070 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.069+0000 7fae03e37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:44.103 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.102+0000 7fc2bf632040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:44.131 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.129+0000 7fae03e37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:17:44.192 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.191+0000 7fae03e37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:17:44.246 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.245+0000 7fae03e37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:17:44.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.246+0000 7fd02f7fe640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 msgr2=0x7fd01806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:44.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.246+0000 7fd02f7fe640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 0x7fd01806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:44.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.252+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd030074150 con 0x7fd030065ab0 2023-10-21T16:17:44.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.253+0000 7fd02cff9640 1 -- 172.21.15.53:0/3848039970 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0200350a0 con 0x7fd030065ab0 2023-10-21T16:17:44.255 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:17:44.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 msgr2=0x7fd01806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:44.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 0x7fd01806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 msgr2=0x7fd03015f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd0200097c0 tx=0x7fd0200046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 shutdown_connections 2023-10-21T16:17:44.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fd01806af90 0x7fd01806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd03015dde0 0x7fd0300b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0300ce290 0x7fd03009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 --2- 172.21.15.53:0/3848039970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd030065ab0 0x7fd03015f660 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 >> 172.21.15.53:0/3848039970 conn(0x7fd0300b08d0 msgr2=0x7fd0300ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:44.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 shutdown_connections 2023-10-21T16:17:44.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.256+0000 7fd03554e640 1 -- 172.21.15.53:0/3848039970 wait complete. 2023-10-21T16:17:44.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:17:44.483 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.482+0000 7fc2bf632040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.535+0000 7fed1d6ec640 1 Processor -- start 2023-10-21T16:17:44.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.535+0000 7fed1d6ec640 1 -- start start 2023-10-21T16:17:44.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed180b1600 0x7fed180cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fed1806ae60 0x7fed180cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180cdf10 con 0x7fed180b1600 2023-10-21T16:17:44.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180cd1c0 con 0x7fed1806ae60 2023-10-21T16:17:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.536+0000 7fed1d6ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cc9b0 con 0x7fed1815ce70 2023-10-21T16:17:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed180b1600 0x7fed180cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed180b1600 0x7fed180cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56598/0 (socket says 172.21.15.53:56598) 2023-10-21T16:17:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 -- 172.21.15.53:0/2615384736 learned_addr learned my addr 172.21.15.53:0/2615384736 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:44.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed1ceeb640 1 --2- 172.21.15.53:0/2615384736 >> v2:172.21.15.124:3300/0 conn(0x7fed1806ae60 0x7fed180cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed17fff640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 -- 172.21.15.53:0/2615384736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 msgr2=0x7fed1806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1806c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 -- 172.21.15.53:0/2615384736 >> v2:172.21.15.124:3300/0 conn(0x7fed1806ae60 msgr2=0x7fed180cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:44.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 --2- 172.21.15.53:0/2615384736 >> v2:172.21.15.124:3300/0 conn(0x7fed1806ae60 0x7fed180cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.537+0000 7fed0ffff640 1 -- 172.21.15.53:0/2615384736 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180ca6c0 con 0x7fed180b1600 2023-10-21T16:17:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.538+0000 7fed0ffff640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed180cd330 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fed1809fcc0 tx=0x7fed0002e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1b64edfa9aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed177fe640 1 -- 172.21.15.53:0/2615384736 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0002ee80 con 0x7fed180b1600 2023-10-21T16:17:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed177fe640 1 -- 172.21.15.53:0/2615384736 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00036440 con 0x7fed180b1600 2023-10-21T16:17:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed177fe640 1 -- 172.21.15.53:0/2615384736 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000367b0 con 0x7fed180b1600 2023-10-21T16:17:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 -- 172.21.15.53:0/2615384736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 msgr2=0x7fed180cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed180cd330 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fed1809fcc0 tx=0x7fed0002e0d0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 -- 172.21.15.53:0/2615384736 shutdown_connections 2023-10-21T16:17:44.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/2615384736 >> v2:172.21.15.124:3300/0 conn(0x7fed1806ae60 0x7fed180cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed180cd330 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/2615384736 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1806c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.539+0000 7fed1d6ec640 1 -- 172.21.15.53:0/2615384736 >> 172.21.15.53:0/2615384736 conn(0x7fed180b08d0 msgr2=0x7fed180b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.540+0000 7fed1d6ec640 1 -- 172.21.15.53:0/2615384736 shutdown_connections 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.540+0000 7fed1d6ec640 1 -- 172.21.15.53:0/2615384736 wait complete. 2023-10-21T16:17:44.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.541+0000 7fed1d6ec640 1 Processor -- start 2023-10-21T16:17:44.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.541+0000 7fed1d6ec640 1 -- start start 2023-10-21T16:17:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed1806ae60 0x7fed1809fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1809ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b4580 con 0x7fed180b1600 2023-10-21T16:17:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c6c80 con 0x7fed1806ae60 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed1d6ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b42a0 con 0x7fed1815ce70 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56612/0 (socket says 172.21.15.53:56612) 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed0ffff640 1 -- 172.21.15.53:0/3287319464 learned_addr learned my addr 172.21.15.53:0/3287319464 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.543+0000 7fed17fff640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed1806ae60 0x7fed1809fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 -- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 msgr2=0x7fed1809ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1809ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 -- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed1806ae60 msgr2=0x7fed1809fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed1806ae60 0x7fed1809fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 -- 172.21.15.53:0/3287319464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0000b770 con 0x7fed180b1600 2023-10-21T16:17:44.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.544+0000 7fed0ffff640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fed0002e110 tx=0x7fed00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.545+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000348d0 con 0x7fed180b1600 2023-10-21T16:17:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.545+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00034a30 con 0x7fed180b1600 2023-10-21T16:17:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.545+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00034de0 con 0x7fed180b1600 2023-10-21T16:17:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.545+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180c6630 con 0x7fed180b1600 2023-10-21T16:17:44.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.545+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed18157d80 con 0x7fed180b1600 2023-10-21T16:17:44.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.546+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180c6490 con 0x7fed180b1600 2023-10-21T16:17:44.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.547+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fed00041480 con 0x7fed180b1600 2023-10-21T16:17:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.548+0000 7fed157fa640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 0x7fecf806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.548+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fed000acc90 con 0x7fed180b1600 2023-10-21T16:17:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.548+0000 7fed17fff640 1 -- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 msgr2=0x7fecf806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:44.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.548+0000 7fed17fff640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 0x7fecf806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:44.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.553+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed180c6490 con 0x7fed180b1600 2023-10-21T16:17:44.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.641+0000 7fc2bf632040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:44.660 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.658+0000 7fae03e37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:17:44.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.744+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed180c6490 con 0x7fed180b1600 2023-10-21T16:17:44.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.745+0000 7fed157fa640 1 -- 172.21.15.53:0/3287319464 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed180c6490 con 0x7fed180b1600 2023-10-21T16:17:44.747 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:17:44.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 msgr2=0x7fecf806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 0x7fecf806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 msgr2=0x7fed18157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fed0002e110 tx=0x7fed00004730 comp rx=0 tx=0).stop 2023-10-21T16:17:44.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 shutdown_connections 2023-10-21T16:17:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7fecf806af90 0x7fecf806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed1815ce70 0x7fed1809ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed180b1600 0x7fed18157840 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 --2- 172.21.15.53:0/3287319464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed1806ae60 0x7fed1809fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 >> 172.21.15.53:0/3287319464 conn(0x7fed180b08d0 msgr2=0x7fed1815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 shutdown_connections 2023-10-21T16:17:44.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.748+0000 7fed1d6ec640 1 -- 172.21.15.53:0/3287319464 wait complete. 2023-10-21T16:17:44.752 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.749+0000 7fc2bf632040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:44.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:17:44.799 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.798+0000 7fc2bf632040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:44.830 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.829+0000 7fae03e37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:17:44.905 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:44.904+0000 7fc2bf632040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:44.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:44.945+0000 7fae03e37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:17:44.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.980+0000 7f3bd2ebb640 1 Processor -- start 2023-10-21T16:17:44.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.980+0000 7f3bd2ebb640 1 -- start start 2023-10-21T16:17:44.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd2ebb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3bcc06c3a0 0x7f3bcc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd2ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd2ebb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc15b310 con 0x7f3bcc06db60 2023-10-21T16:17:44.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd2ebb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0ca230 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd2ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc06d290 con 0x7f3bcc15aa00 2023-10-21T16:17:44.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bcbfff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f3bcc06db60 0x7f3bcc06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:59140/0 (socket says 172.21.15.53:59140) 2023-10-21T16:17:44.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bcbfff640 1 -- 172.21.15.53:0/296916841 learned_addr learned my addr 172.21.15.53:0/296916841 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:44.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.982+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/296916841 >> v2:172.21.15.124:3300/0 conn(0x7f3bcc06c3a0 0x7f3bcc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bd1458640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bcb7fe640 1 -- 172.21.15.53:0/296916841 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924666904 0 0) 0x7f3bb8001110 con 0x7f3bcc06db60 2023-10-21T16:17:44.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bcb7fe640 1 -- 172.21.15.53:0/296916841 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bbc005730 con 0x7f3bcc06db60 2023-10-21T16:17:44.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bd0c57640 1 -- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 msgr2=0x7f3bcc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:44.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bd0c57640 1 -- 172.21.15.53:0/296916841 >> v1:172.21.15.124:6789/0 conn(0x7f3bcc06db60 legacy=0x7f3bcc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.983+0000 7f3bd0c57640 1 -- 172.21.15.53:0/296916841 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0c9860 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.984+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0c9cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097a0 tx=0x7f3bc002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46dfdbd10a97d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.985+0000 7f3bcb7fe640 1 -- 172.21.15.53:0/296916841 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0031830 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.985+0000 7f3bcb7fe640 1 -- 172.21.15.53:0/296916841 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0031990 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.985+0000 7f3bcb7fe640 1 -- 172.21.15.53:0/296916841 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0031d20 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.985+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 msgr2=0x7f3bcc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.985+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0c9cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00097a0 tx=0x7f3bc002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/296916841 shutdown_connections 2023-10-21T16:17:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/296916841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0c9cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/296916841 >> 172.21.15.53:0/296916841 conn(0x7f3bcc06a5d0 msgr2=0x7f3bcc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:44.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/296916841 shutdown_connections 2023-10-21T16:17:44.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.986+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/296916841 wait complete. 2023-10-21T16:17:44.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.987+0000 7f3bd2ebb640 1 Processor -- start 2023-10-21T16:17:44.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.988+0000 7f3bd2ebb640 1 -- start start 2023-10-21T16:17:44.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f6560 con 0x7f3bcc06db60 2023-10-21T16:17:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f33f0 con 0x7f3bcc06c3a0 2023-10-21T16:17:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd2ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f6980 con 0x7f3bcc15aa00 2023-10-21T16:17:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56620/0 (socket says 172.21.15.53:56620) 2023-10-21T16:17:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 -- 172.21.15.53:0/1568655083 learned_addr learned my addr 172.21.15.53:0/1568655083 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd1458640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 msgr2=0x7f3bcc1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc1551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 msgr2=0x7f3bcc0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bcbfff640 1 -- 172.21.15.53:0/1568655083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc0009400 con 0x7f3bcc06db60 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.989+0000 7f3bd1458640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bcbfff640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800c700 tx=0x7f3bb800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:44.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8005ce0 con 0x7f3bcc06db60 2023-10-21T16:17:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8005e40 con 0x7f3bcc06db60 2023-10-21T16:17:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc1f5bc0 con 0x7f3bcc06db60 2023-10-21T16:17:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8007290 con 0x7f3bcc06db60 2023-10-21T16:17:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.990+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1f6180 con 0x7f3bcc06db60 2023-10-21T16:17:44.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.991+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc0c94d0 con 0x7f3bcc06db60 2023-10-21T16:17:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.992+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3bb80073f0 con 0x7f3bcc06db60 2023-10-21T16:17:44.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.993+0000 7f3bc97fa640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 0x7f3bb006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.993+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f3bb8088eb0 con 0x7f3bcc06db60 2023-10-21T16:17:45.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.993+0000 7f3bd0c57640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 msgr2=0x7f3bb006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:45.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:44.994+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 0x7f3bb006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:45.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.001+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3bcc0c94d0 con 0x7f3bcc06db60 2023-10-21T16:17:45.005 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.004+0000 7fae03e37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:17:45.012 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:45.011+0000 7fc2bf632040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:45.120 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.118+0000 7fae03e37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:45.167 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:45.165+0000 7fc2bf632040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.195+0000 7f3bd0c57640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 msgr2=0x7f3bb006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.195+0000 7f3bd0c57640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 0x7f3bb006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:45.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.208+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3bcc1f66a0 con 0x7f3bcc06db60 2023-10-21T16:17:45.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.208+0000 7f3bc97fa640 1 -- 172.21.15.53:0/1568655083 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v91) v1 ==== 80+0+477 (secure 0 0 0) 0x7f3bcc1f66a0 con 0x7f3bcc06db60 2023-10-21T16:17:45.210 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1810216649 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.210 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/1636058776 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.210 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/1636058776 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2018576779 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3111202668 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/356555339 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3273170228 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3109992220 2023-10-22T16:17:40.466814+0000 2023-10-21T16:17:45.212 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 msgr2=0x7f3bb006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 0x7f3bb006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 msgr2=0x7f3bcc154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800c700 tx=0x7f3bb800cb00 comp rx=0 tx=0).stop 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 shutdown_connections 2023-10-21T16:17:45.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f3bb006af90 0x7f3bb006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3bcc15aa00 0x7f3bcc1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bcc06db60 0x7f3bcc154ca0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 --2- 172.21.15.53:0/1568655083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3bcc06c3a0 0x7f3bcc0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 >> 172.21.15.53:0/1568655083 conn(0x7f3bcc06a5d0 msgr2=0x7f3bcc070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:45.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 shutdown_connections 2023-10-21T16:17:45.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.211+0000 7f3bd2ebb640 1 -- 172.21.15.53:0/1568655083 wait complete. 2023-10-21T16:17:45.215 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:45.214+0000 7fc2bf632040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:45.231 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:17:45.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:45.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:17:45.237 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.233+0000 7fae03e37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:17:45.334 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:45.333+0000 7fc2bf632040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:45.339 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:17:45.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:17:45.340 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:17:45.389 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:45.389 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:45.409 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.408+0000 7fae03e37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:17:45.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:45.443 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:45.444 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:45.465 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.462+0000 7fae03e37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:17:45.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:45.498 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:45.499 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:45.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:45.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:45.594 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:45.593+0000 7fae03e37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:17:45.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.769+0000 7f16498ff640 1 Processor -- start 2023-10-21T16:17:45.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.769+0000 7f16498ff640 1 -- start start 2023-10-21T16:17:45.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f164415ab30 0x7f164415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f16440adc70 0x7f16440b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 0x7f16440b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164415f7d0 con 0x7f164415ab30 2023-10-21T16:17:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164415f210 con 0x7f16440adc70 2023-10-21T16:17:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.770+0000 7f16498ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164415f4f0 con 0x7f164415af10 2023-10-21T16:17:45.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f164415ab30 0x7f164415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f164415ab30 0x7f164415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56636/0 (socket says 172.21.15.53:56636) 2023-10-21T16:17:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 -- 172.21.15.53:0/2723099430 learned_addr learned my addr 172.21.15.53:0/2723099430 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16490fe640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 0x7f16440b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f1643fff640 1 --2- 172.21.15.53:0/2723099430 >> v2:172.21.15.124:3300/0 conn(0x7f16440adc70 0x7f16440b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 -- 172.21.15.53:0/2723099430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 msgr2=0x7f16440b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:45.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 0x7f16440b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.771+0000 7f16488fd640 1 -- 172.21.15.53:0/2723099430 >> v2:172.21.15.124:3300/0 conn(0x7f16440adc70 msgr2=0x7f16440b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:45.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.772+0000 7f16488fd640 1 --2- 172.21.15.53:0/2723099430 >> v2:172.21.15.124:3300/0 conn(0x7f16440adc70 0x7f16440b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.772+0000 7f16488fd640 1 -- 172.21.15.53:0/2723099430 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164406c550 con 0x7f164415ab30 2023-10-21T16:17:45.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.772+0000 7f16490fe640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 0x7f16440b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:45.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16488fd640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415ab30 0x7f164415e8a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f16440c56c0 tx=0x7f163802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782974adf2b287cc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:45.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16437fe640 1 -- 172.21.15.53:0/2723099430 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638031870 con 0x7f164415ab30 2023-10-21T16:17:45.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16437fe640 1 -- 172.21.15.53:0/2723099430 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16380319d0 con 0x7f164415ab30 2023-10-21T16:17:45.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16437fe640 1 -- 172.21.15.53:0/2723099430 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638031d60 con 0x7f164415ab30 2023-10-21T16:17:45.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16498ff640 1 -- 172.21.15.53:0/2723099430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415ab30 msgr2=0x7f164415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:45.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.773+0000 7f16498ff640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415ab30 0x7f164415e8a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f16440c56c0 tx=0x7f163802b910 comp rx=0 tx=0).stop 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 -- 172.21.15.53:0/2723099430 shutdown_connections 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f164415af10 0x7f16440b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 --2- 172.21.15.53:0/2723099430 >> v2:172.21.15.124:3300/0 conn(0x7f16440adc70 0x7f16440b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 --2- 172.21.15.53:0/2723099430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415ab30 0x7f164415e8a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 -- 172.21.15.53:0/2723099430 >> 172.21.15.53:0/2723099430 conn(0x7f16440cb0c0 msgr2=0x7f16440cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:45.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 -- 172.21.15.53:0/2723099430 shutdown_connections 2023-10-21T16:17:45.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.774+0000 7f16498ff640 1 -- 172.21.15.53:0/2723099430 wait complete. 2023-10-21T16:17:45.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.776+0000 7f16498ff640 1 Processor -- start 2023-10-21T16:17:45.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.776+0000 7f16498ff640 1 -- start start 2023-10-21T16:17:45.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16440adc70 0x7f1644148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f164415ab30 0x7f16440af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644149010 con 0x7f164415af10 2023-10-21T16:17:45.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164415e1e0 con 0x7f164415ab30 2023-10-21T16:17:45.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16498ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164415df00 con 0x7f16440adc70 2023-10-21T16:17:45.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56646/0 (socket says 172.21.15.53:56646) 2023-10-21T16:17:45.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 -- 172.21.15.53:0/277924659 learned_addr learned my addr 172.21.15.53:0/277924659 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:45.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16488fd640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16440adc70 0x7f1644148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f1643fff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f164415ab30 0x7f16440af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:45.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16440adc70 msgr2=0x7f1644148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:45.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16440adc70 0x7f1644148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f164415ab30 msgr2=0x7f16440af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:45.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f164415ab30 0x7f16440af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:45.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.777+0000 7f16490fe640 1 -- 172.21.15.53:0/277924659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009a70 con 0x7f164415af10 2023-10-21T16:17:45.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16490fe640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f163c00efc0 tx=0x7f163c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:45.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c009cd0 con 0x7f164415af10 2023-10-21T16:17:45.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00f040 con 0x7f164415af10 2023-10-21T16:17:45.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16380094d0 con 0x7f164415af10 2023-10-21T16:17:45.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c003990 con 0x7f164415af10 2023-10-21T16:17:45.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.778+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16440c6b30 con 0x7f164415af10 2023-10-21T16:17:45.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.779+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164404e160 con 0x7f164415af10 2023-10-21T16:17:45.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.780+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f163c014480 con 0x7f164415af10 2023-10-21T16:17:45.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.782+0000 7f16417fa640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 0x7f162806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:45.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.782+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f163c08a870 con 0x7f164415af10 2023-10-21T16:17:45.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.782+0000 7f16488fd640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 msgr2=0x7f162806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:45.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.782+0000 7f16488fd640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 0x7f162806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:45.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.786+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f163c055300 con 0x7f164415af10 2023-10-21T16:17:45.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.983+0000 7f16488fd640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 msgr2=0x7f162806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:45.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:45.983+0000 7f16488fd640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 0x7f162806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:46.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.015+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16440ada50 con 0x7f164415af10 2023-10-21T16:17:46.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.016+0000 7f16417fa640 1 -- 172.21.15.53:0/277924659 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f163c058920 con 0x7f164415af10 2023-10-21T16:17:46.018 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:46.018 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:46.019 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:46.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 msgr2=0x7f162806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:46.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 0x7f162806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 msgr2=0x7f16440af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f163c00efc0 tx=0x7f163c00c490 comp rx=0 tx=0).stop 2023-10-21T16:17:46.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 shutdown_connections 2023-10-21T16:17:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f162806af90 0x7f162806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f164415af10 0x7f16440af5e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f164415ab30 0x7f16440af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 --2- 172.21.15.53:0/277924659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16440adc70 0x7f1644148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 >> 172.21.15.53:0/277924659 conn(0x7f16440cb0c0 msgr2=0x7f16440701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:46.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 shutdown_connections 2023-10-21T16:17:46.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.019+0000 7f16498ff640 1 -- 172.21.15.53:0/277924659 wait complete. 2023-10-21T16:17:46.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:46.170 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.169+0000 7fc2bf632040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:46.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.280+0000 7f4f9f0ea640 1 Processor -- start 2023-10-21T16:17:46.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.280+0000 7f4f9f0ea640 1 -- start start 2023-10-21T16:17:46.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.281+0000 7f4f9f0ea640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f9815cc60 0x7f4f980cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.281+0000 7f4f9f0ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.281+0000 7f4f9f0ea640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9815d040 con 0x7f4f980cdfd0 2023-10-21T16:17:46.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.281+0000 7f4f9f0ea640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980d0650 con 0x7f4f9815cc60 2023-10-21T16:17:46.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.281+0000 7f4f9f0ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980d0930 con 0x7f4f980d16f0 2023-10-21T16:17:46.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.282+0000 7f4f9e0e8640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f9815cc60 0x7f4f980cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.282+0000 7f4f9e0e8640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f9815cc60 0x7f4f980cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:47256/0 (socket says 172.21.15.53:47256) 2023-10-21T16:17:46.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.282+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/1036682314 learned_addr learned my addr 172.21.15.53:0/1036682314 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:46.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.282+0000 7f4f9d8e7640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.283+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 msgr2=0x7f4f980cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:46.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.283+0000 7f4f9e0e8640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.283+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/1036682314 >> v1:172.21.15.124:6789/0 conn(0x7f4f980cdfd0 legacy=0x7f4f980ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.283+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/1036682314 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980d0c10 con 0x7f4f9815cc60 2023-10-21T16:17:46.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.283+0000 7f4f9d8e7640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:46.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.284+0000 7f4f9e0e8640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980cf340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ea0 tx=0x7f4f8802bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9cc2509ac062f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.285+0000 7f4f9d0e6640 1 -- 172.21.15.53:0/1036682314 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8802cb40 con 0x7f4f9815cc60 2023-10-21T16:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.285+0000 7f4f9d0e6640 1 -- 172.21.15.53:0/1036682314 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8802cca0 con 0x7f4f9815cc60 2023-10-21T16:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.285+0000 7f4f9d0e6640 1 -- 172.21.15.53:0/1036682314 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88034610 con 0x7f4f9815cc60 2023-10-21T16:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.285+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 msgr2=0x7f4f980cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.285+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980cf340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ea0 tx=0x7f4f8802bfa0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/1036682314 shutdown_connections 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/1036682314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980cf340 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/1036682314 >> 172.21.15.53:0/1036682314 conn(0x7f4f9809bcf0 msgr2=0x7f4f980d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/1036682314 shutdown_connections 2023-10-21T16:17:46.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.286+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/1036682314 wait complete. 2023-10-21T16:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.287+0000 7f4f9f0ea640 1 Processor -- start 2023-10-21T16:17:46.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.288+0000 7f4f9f0ea640 1 -- start start 2023-10-21T16:17:46.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9815a920 con 0x7f4f980cdfd0 2023-10-21T16:17:46.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98159d10 con 0x7f4f9815cc60 2023-10-21T16:17:46.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9f0ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98159ff0 con 0x7f4f980d16f0 2023-10-21T16:17:46.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56656/0 (socket says 172.21.15.53:56656) 2023-10-21T16:17:46.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/2851448947 learned_addr learned my addr 172.21.15.53:0/2851448947 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:46.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 msgr2=0x7f4f980c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 msgr2=0x7f4f980c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.289+0000 7f4f9e0e8640 1 -- 172.21.15.53:0/2851448947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c009400 con 0x7f4f980cdfd0 2023-10-21T16:17:46.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.290+0000 7f4f9e0e8640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ea0 tx=0x7f4f88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:46.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.291+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88009510 con 0x7f4f980cdfd0 2023-10-21T16:17:46.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.291+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8803e450 con 0x7f4f980cdfd0 2023-10-21T16:17:46.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.291+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8803e820 con 0x7f4f980cdfd0 2023-10-21T16:17:46.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.291+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f88009b30 con 0x7f4f980cdfd0 2023-10-21T16:17:46.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.292+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9815a5b0 con 0x7f4f980cdfd0 2023-10-21T16:17:46.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.293+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9815aad0 con 0x7f4f980cdfd0 2023-10-21T16:17:46.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.294+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4f8803e980 con 0x7f4f980cdfd0 2023-10-21T16:17:46.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.295+0000 7f4f96ffd640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 0x7f4f8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.295+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f4f880ac890 con 0x7f4f980cdfd0 2023-10-21T16:17:46.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.295+0000 7f4f9d8e7640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 msgr2=0x7f4f8406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:46.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.295+0000 7f4f9d8e7640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 0x7f4f8406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:17:46.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.299+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f9815aad0 con 0x7f4f980cdfd0 2023-10-21T16:17:46.443 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:46.442+0000 7fae03e37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:17:46.469 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.468+0000 7fc2bf632040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:46.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.496+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f9804e160 con 0x7f4f980cdfd0 2023-10-21T16:17:46.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.496+0000 7f4f9d8e7640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 msgr2=0x7f4f8406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2480922944 2023-10-21T16:17:46.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.496+0000 7f4f9d8e7640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 0x7f4f8406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:17:46.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.497+0000 7f4f96ffd640 1 -- 172.21.15.53:0/2851448947 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17282 (secure 0 0 0) 0x7f4f880774c0 con 0x7f4f980cdfd0 2023-10-21T16:17:46.499 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:46.499 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":91,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:40.466887+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1810216649":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6833/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6832/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/2018576779":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3111202668":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/356555339":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3273170228":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3109992220":"2023-10-22T16:17:40.466814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:46.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.499+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 msgr2=0x7f4f8406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:17:46.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.499+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 0x7f4f8406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.499+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 msgr2=0x7f4f98154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ea0 tx=0x7f4f88004a40 comp rx=0 tx=0).stop 2023-10-21T16:17:46.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 shutdown_connections 2023-10-21T16:17:46.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:6832/2480922944,v1:172.21.15.124:6833/2480922944] conn(0x7f4f8406af90 0x7f4f8406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f9815cc60 0x7f4f980c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f980d16f0 0x7f4f980c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 --2- 172.21.15.53:0/2851448947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f980cdfd0 0x7f4f98154ae0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 >> 172.21.15.53:0/2851448947 conn(0x7f4f9809bcf0 msgr2=0x7f4f9815e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:46.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 shutdown_connections 2023-10-21T16:17:46.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.500+0000 7f4f9f0ea640 1 -- 172.21.15.53:0/2851448947 wait complete. 2023-10-21T16:17:46.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:17:23.670145+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:17:24.763686+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:46.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:46.526 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.525+0000 7fc2bf632040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:46.641 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.639+0000 7fc2bf632040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.732+0000 7f8d5aabb640 1 Processor -- start 2023-10-21T16:17:46.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.732+0000 7f8d5aabb640 1 -- start start 2023-10-21T16:17:46.735 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.733+0000 7fc2441c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.735 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.733+0000 7fc2441c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.735 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.733+0000 7fc2441c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.735 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.733+0000 7fc2441c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.736 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.733+0000 7fc2441c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d5aabb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8d540b1750 0x7f8d5406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d5aabb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 0x7f8d54070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d5aabb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5406d670 con 0x7f8d5415aad0 2023-10-21T16:17:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d5aabb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5415bf30 con 0x7f8d540b1750 2023-10-21T16:17:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d5aabb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540b1bb0 con 0x7f8d5406d240 2023-10-21T16:17:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d53fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8d540b1750 0x7f8d5406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d58857640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8d5415aad0 0x7f8d540c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:59160/0 (socket says 172.21.15.53:59160) 2023-10-21T16:17:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d58857640 1 -- 172.21.15.53:0/3629447939 learned_addr learned my addr 172.21.15.53:0/3629447939 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.734+0000 7f8d59058640 1 --2- 172.21.15.53:0/3629447939 >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 0x7f8d54070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.735+0000 7f8d53fff640 1 -- 172.21.15.53:0/3629447939 >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 msgr2=0x7f8d54070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.735+0000 7f8d53fff640 1 --2- 172.21.15.53:0/3629447939 >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 0x7f8d54070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.735+0000 7f8d53fff640 1 -- 172.21.15.53:0/3629447939 >> v1:172.21.15.124:6789/0 conn(0x7f8d5415aad0 legacy=0x7f8d540c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.735+0000 7f8d53fff640 1 -- 172.21.15.53:0/3629447939 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5415b320 con 0x7f8d540b1750 2023-10-21T16:17:46.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.735+0000 7f8d59058640 1 --2- 172.21.15.53:0/3629447939 >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 0x7f8d54070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:46.741 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.735+0000 7fc2411c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.735+0000 7fc2411c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.735+0000 7fc2411c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.735+0000 7fc2411c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:46.735+0000 7fc2411c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:17:46.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.736+0000 7f8d53fff640 1 --2- 172.21.15.53:0/3629447939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5406cd00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8d5406daa0 tx=0x7f8d4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db071b4af308b5d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:46.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.737+0000 7f8d537fe640 1 -- 172.21.15.53:0/3629447939 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40031830 con 0x7f8d540b1750 2023-10-21T16:17:46.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.737+0000 7f8d537fe640 1 -- 172.21.15.53:0/3629447939 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40031990 con 0x7f8d540b1750 2023-10-21T16:17:46.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.737+0000 7f8d537fe640 1 -- 172.21.15.53:0/3629447939 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40031d20 con 0x7f8d540b1750 2023-10-21T16:17:46.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.737+0000 7f8d5aabb640 1 -- 172.21.15.53:0/3629447939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 msgr2=0x7f8d5406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.737+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/3629447939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5406cd00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8d5406daa0 tx=0x7f8d4002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 -- 172.21.15.53:0/3629447939 shutdown_connections 2023-10-21T16:17:46.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/3629447939 >> v2:172.21.15.124:3300/0 conn(0x7f8d5406d240 0x7f8d54070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/3629447939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5406cd00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 -- 172.21.15.53:0/3629447939 >> 172.21.15.53:0/3629447939 conn(0x7f8d54069d50 msgr2=0x7f8d5406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 -- 172.21.15.53:0/3629447939 shutdown_connections 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.738+0000 7f8d5aabb640 1 -- 172.21.15.53:0/3629447939 wait complete. 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.740+0000 7f8d5aabb640 1 Processor -- start 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.740+0000 7f8d5aabb640 1 -- start start 2023-10-21T16:17:46.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5406d240 0x7f8d540cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5414ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 0x7f8d5414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:46.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5414e820 con 0x7f8d540b1750 2023-10-21T16:17:46.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5414b250 con 0x7f8d5415aad0 2023-10-21T16:17:46.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d5aabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5414e540 con 0x7f8d5406d240 2023-10-21T16:17:46.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d58857640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5406d240 0x7f8d540cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.741+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5414ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5414ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56678/0 (socket says 172.21.15.53:56678) 2023-10-21T16:17:46.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d53fff640 1 -- 172.21.15.53:0/2300767462 learned_addr learned my addr 172.21.15.53:0/2300767462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:46.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 0x7f8d5414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:46.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 -- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5406d240 msgr2=0x7f8d540cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5406d240 0x7f8d540cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 -- 172.21.15.53:0/2300767462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 msgr2=0x7f8d5414ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5414ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 -- 172.21.15.53:0/2300767462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c009a70 con 0x7f8d5415aad0 2023-10-21T16:17:46.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.742+0000 7f8d59058640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 0x7f8d5414dbd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00efc0 tx=0x7f8d4c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:46.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.743+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00cd30 con 0x7f8d5415aad0 2023-10-21T16:17:46.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.743+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c00f040 con 0x7f8d5415aad0 2023-10-21T16:17:46.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.743+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c003bd0 con 0x7f8d5415aad0 2023-10-21T16:17:46.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.743+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40009400 con 0x7f8d5415aad0 2023-10-21T16:17:46.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.743+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540cb5a0 con 0x7f8d5415aad0 2023-10-21T16:17:46.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.744+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5406c480 con 0x7f8d5415aad0 2023-10-21T16:17:46.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.745+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8d4c014480 con 0x7f8d5415aad0 2023-10-21T16:17:46.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.746+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f8d4c089f40 con 0x7f8d5415aad0 2023-10-21T16:17:46.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.751+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d4c0bcca0 con 0x7f8d5415aad0 2023-10-21T16:17:46.772 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:46.771+0000 7fae03e37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:17:46.833 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:46.832+0000 7fae03e37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:17:46.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.942+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d5404e160 con 0x7f8d5415aad0 2023-10-21T16:17:46.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.943+0000 7f8d517fa640 1 -- 172.21.15.53:0/2300767462 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d4c0550d0 con 0x7f8d5415aad0 2023-10-21T16:17:46.945 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:46.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 msgr2=0x7f8d5414dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:46.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 0x7f8d5414dbd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00efc0 tx=0x7f8d4c00c490 comp rx=0 tx=0).stop 2023-10-21T16:17:46.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 shutdown_connections 2023-10-21T16:17:46.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415aad0 0x7f8d5414dbd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d540b1750 0x7f8d5414ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 --2- 172.21.15.53:0/2300767462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5406d240 0x7f8d540cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:46.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 >> 172.21.15.53:0/2300767462 conn(0x7f8d54069d50 msgr2=0x7f8d5415e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:46.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 shutdown_connections 2023-10-21T16:17:46.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:46.946+0000 7f8d5aabb640 1 -- 172.21.15.53:0/2300767462 wait complete. 2023-10-21T16:17:46.953 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:17:46.952+0000 7fae03e37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:17:46.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:17:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.175+0000 7f0c7a9ff640 1 Processor -- start 2023-10-21T16:17:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.175+0000 7f0c7a9ff640 1 -- start start 2023-10-21T16:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.176+0000 7f0c7a9ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c7415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.176+0000 7f0c7a9ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c740cab60 con 0x7f0c7415a820 2023-10-21T16:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.176+0000 7f0c7a9ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c740cb960 con 0x7f0c740ca680 2023-10-21T16:17:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.176+0000 7f0c7a9ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740cbdf0 con 0x7f0c740b0680 2023-10-21T16:17:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c799fd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0c7415a820 0x7f0c740aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48922/0 (socket says 172.21.15.53:48922) 2023-10-21T16:17:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c799fd640 1 -- 172.21.15.53:0/1775094390 learned_addr learned my addr 172.21.15.53:0/1775094390 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845620563 0 0) 0x7f0c740cab60 con 0x7f0c7415a820 2023-10-21T16:17:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c60003520 con 0x7f0c7415a820 2023-10-21T16:17:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c791fc640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c7415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.177+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281740241 0 0) 0x7f0c60003520 con 0x7f0c7415a820 2023-10-21T16:17:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 msgr2=0x7f0c7415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c7415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 >> v1:172.21.15.124:6789/0 conn(0x7f0c740ca680 legacy=0x7f0c7415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740cb030 con 0x7f0c7415a820 2023-10-21T16:17:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0c70001670 con 0x7f0c7415a820 2023-10-21T16:17:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 >> v1:172.21.15.53:6789/0 conn(0x7f0c7415a820 legacy=0x7f0c740aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c600039a0 0x7f0c60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c600047c0 0x7f0c60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c6000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740cbdf0 con 0x7f0c600039a0 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600054e0 con 0x7f0c600047c0 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6000bec0 con 0x7f0c740b0680 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c799fd640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c600039a0 0x7f0c60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c791fc640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c600047c0 0x7f0c60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.178+0000 7f0c7a1fe640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c6000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c799fd640 1 -- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 msgr2=0x7f0c6000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c799fd640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c6000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c799fd640 1 -- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c600047c0 msgr2=0x7f0c60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c799fd640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c600047c0 0x7f0c60004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c799fd640 1 -- 172.21.15.53:0/1775094390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60003520 con 0x7f0c600039a0 2023-10-21T16:17:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.179+0000 7f0c7a1fe640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740b0680 0x7f0c6000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.180+0000 7f0c799fd640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c600039a0 0x7f0c60003dc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c240 tx=0x7f0c600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68432fe28526599 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.180+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70005ad0 con 0x7f0c600039a0 2023-10-21T16:17:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.180+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70005c50 con 0x7f0c600039a0 2023-10-21T16:17:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.180+0000 7f0c789fb640 1 -- 172.21.15.53:0/1775094390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7002c8b0 con 0x7f0c600039a0 2023-10-21T16:17:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/1775094390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c600039a0 msgr2=0x7f0c60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 --2- 172.21.15.53:0/1775094390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c600039a0 0x7f0c60003dc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c240 tx=0x7f0c600057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a1fe640 1 -- 172.21.15.53:0/1775094390 reap_dead start 2023-10-21T16:17:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/1775094390 shutdown_connections 2023-10-21T16:17:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/1775094390 >> 172.21.15.53:0/1775094390 conn(0x7f0c7406c050 msgr2=0x7f0c7406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/1775094390 shutdown_connections 2023-10-21T16:17:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.181+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/1775094390 wait complete. 2023-10-21T16:17:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.183+0000 7f0c7a9ff640 1 Processor -- start 2023-10-21T16:17:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.183+0000 7f0c7a9ff640 1 -- start start 2023-10-21T16:17:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c600039a0 0x7f0c740c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740cd210 0x7f0c740c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740c7440 con 0x7f0c7415d260 2023-10-21T16:17:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740c78d0 con 0x7f0c740cd210 2023-10-21T16:17:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a9ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c740c7bb0 con 0x7f0c600039a0 2023-10-21T16:17:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a1fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a1fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56704/0 (socket says 172.21.15.53:56704) 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c7a1fe640 1 -- 172.21.15.53:0/516695796 learned_addr learned my addr 172.21.15.53:0/516695796 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.184+0000 7f0c791fc640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740cd210 0x7f0c740c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c799fd640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c600039a0 0x7f0c740c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 -- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c600039a0 msgr2=0x7f0c740c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c600039a0 0x7f0c740c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 -- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740cd210 msgr2=0x7f0c740c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740cd210 0x7f0c740c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 -- 172.21.15.53:0/516695796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740c7e90 con 0x7f0c7415d260 2023-10-21T16:17:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a1fe640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f80 tx=0x7f0c6c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c006d50 con 0x7f0c7415d260 2023-10-21T16:17:47.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c003970 con 0x7f0c7415d260 2023-10-21T16:17:47.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c740c8170 con 0x7f0c7415d260 2023-10-21T16:17:47.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c003d20 con 0x7f0c7415d260 2023-10-21T16:17:47.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.185+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7415dc30 con 0x7f0c7415d260 2023-10-21T16:17:47.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.187+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c740c8690 con 0x7f0c7415d260 2023-10-21T16:17:47.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.188+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0c6c005070 con 0x7f0c7415d260 2023-10-21T16:17:47.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.189+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f0c6c0892f0 con 0x7f0c7415d260 2023-10-21T16:17:47.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.193+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c740c8690 con 0x7f0c7415d260 2023-10-21T16:17:47.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.385+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c740c8690 con 0x7f0c7415d260 2023-10-21T16:17:47.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.386+0000 7f0c6a7fc640 1 -- 172.21.15.53:0/516695796 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c740c8690 con 0x7f0c7415d260 2023-10-21T16:17:47.388 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:17:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 msgr2=0x7f0c7415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f80 tx=0x7f0c6c00dc00 comp rx=0 tx=0).stop 2023-10-21T16:17:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 shutdown_connections 2023-10-21T16:17:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c7415d260 0x7f0c7415d6f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740cd210 0x7f0c740c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 --2- 172.21.15.53:0/516695796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c600039a0 0x7f0c740c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 >> 172.21.15.53:0/516695796 conn(0x7f0c7406c050 msgr2=0x7f0c740ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:47.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 shutdown_connections 2023-10-21T16:17:47.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.389+0000 7f0c7a9ff640 1 -- 172.21.15.53:0/516695796 wait complete. 2023-10-21T16:17:47.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:17:47.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.623+0000 7fccbece0640 1 Processor -- start 2023-10-21T16:17:47.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.623+0000 7fccbece0640 1 -- start start 2023-10-21T16:17:47.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.624+0000 7fccbece0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fccb815adf0 0x7fccb80b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.624+0000 7fccbece0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 0x7fccb80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbece0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb815b1d0 con 0x7fccb80cd410 2023-10-21T16:17:47.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbece0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80cf270 con 0x7fccb815adf0 2023-10-21T16:17:47.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbece0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80cf510 con 0x7fccb80d05b0 2023-10-21T16:17:47.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fccb815adf0 0x7fccb80b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fccb815adf0 0x7fccb80b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56710/0 (socket says 172.21.15.53:56710) 2023-10-21T16:17:47.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 -- 172.21.15.53:0/689830978 learned_addr learned my addr 172.21.15.53:0/689830978 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:47.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccb7fff640 1 --2- 172.21.15.53:0/689830978 >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 0x7fccb80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 -- 172.21.15.53:0/689830978 >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 msgr2=0x7fccb80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:47.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 --2- 172.21.15.53:0/689830978 >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 0x7fccb80ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.625+0000 7fccbca7c640 1 -- 172.21.15.53:0/689830978 >> v1:172.21.15.124:6789/0 conn(0x7fccb80cd410 legacy=0x7fccb80b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:47.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.626+0000 7fccbca7c640 1 -- 172.21.15.53:0/689830978 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80cf7f0 con 0x7fccb815adf0 2023-10-21T16:17:47.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.626+0000 7fccb7fff640 1 --2- 172.21.15.53:0/689830978 >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 0x7fccb80ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:47.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccbca7c640 1 --2- 172.21.15.53:0/689830978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb80b65a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fccb815e820 tx=0x7fcca802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b4a2b2ad7904f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:47.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccb77fe640 1 -- 172.21.15.53:0/689830978 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca802ca80 con 0x7fccb815adf0 2023-10-21T16:17:47.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccb77fe640 1 -- 172.21.15.53:0/689830978 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca802cbe0 con 0x7fccb815adf0 2023-10-21T16:17:47.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccb77fe640 1 -- 172.21.15.53:0/689830978 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80343f0 con 0x7fccb815adf0 2023-10-21T16:17:47.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccbece0640 1 -- 172.21.15.53:0/689830978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 msgr2=0x7fccb80b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.627+0000 7fccbece0640 1 --2- 172.21.15.53:0/689830978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb80b65a0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fccb815e820 tx=0x7fcca802bee0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 -- 172.21.15.53:0/689830978 shutdown_connections 2023-10-21T16:17:47.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 --2- 172.21.15.53:0/689830978 >> v2:172.21.15.124:3300/0 conn(0x7fccb80d05b0 0x7fccb80ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 --2- 172.21.15.53:0/689830978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb80b65a0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 -- 172.21.15.53:0/689830978 >> 172.21.15.53:0/689830978 conn(0x7fccb806e630 msgr2=0x7fccb806ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:47.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 -- 172.21.15.53:0/689830978 shutdown_connections 2023-10-21T16:17:47.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.628+0000 7fccbece0640 1 -- 172.21.15.53:0/689830978 wait complete. 2023-10-21T16:17:47.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.629+0000 7fccbece0640 1 Processor -- start 2023-10-21T16:17:47.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.629+0000 7fccbece0640 1 -- start start 2023-10-21T16:17:47.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 0x7fccb80c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fccb80d05b0 0x7fccb81f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80b06d0 con 0x7fccb815adf0 2023-10-21T16:17:47.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80c7120 con 0x7fccb80d05b0 2023-10-21T16:17:47.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.630+0000 7fccbece0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80c73c0 con 0x7fccb80cd410 2023-10-21T16:17:47.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56712/0 (socket says 172.21.15.53:56712) 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 -- 172.21.15.53:0/1842912380 learned_addr learned my addr 172.21.15.53:0/1842912380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbca7c640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 0x7fccb80c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccb7fff640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fccb80d05b0 0x7fccb81f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 -- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 msgr2=0x7fccb80c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 0x7fccb80c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 -- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fccb80d05b0 msgr2=0x7fccb81f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fccb80d05b0 0x7fccb81f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 -- 172.21.15.53:0/1842912380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009b30 con 0x7fccb815adf0 2023-10-21T16:17:47.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbca7c640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 0x7fccb80c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:47.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.631+0000 7fccbd27d640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fccb000c410 tx=0x7fccb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:47.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.632+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0011790 con 0x7fccb815adf0 2023-10-21T16:17:47.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.632+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb00118f0 con 0x7fccb815adf0 2023-10-21T16:17:47.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.632+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb0011ce0 con 0x7fccb815adf0 2023-10-21T16:17:47.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.632+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb80b2720 con 0x7fccb815adf0 2023-10-21T16:17:47.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.632+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb80b2ce0 con 0x7fccb815adf0 2023-10-21T16:17:47.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.633+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb804e160 con 0x7fccb815adf0 2023-10-21T16:17:47.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.634+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fccb0007020 con 0x7fccb815adf0 2023-10-21T16:17:47.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.636+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fccb0089b70 con 0x7fccb815adf0 2023-10-21T16:17:47.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.639+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fccb0054be0 con 0x7fccb815adf0 2023-10-21T16:17:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.660+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fccb0054690 con 0x7fccb815adf0 2023-10-21T16:17:47.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.661+0000 7fccb57fa640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 0x7fcc98072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:47.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.662+0000 7fccbca7c640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 0x7fcc98072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:47.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.665+0000 7fccbca7c640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 0x7fcc98072fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcca8035000 tx=0x7fcca8030000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:47.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.837+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fccb80d0060 con 0x7fccb815adf0 2023-10-21T16:17:47.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.838+0000 7fccb57fa640 1 -- 172.21.15.53:0/1842912380 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccb0075280 con 0x7fccb815adf0 2023-10-21T16:17:47.840 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:17:47.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.840+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 msgr2=0x7fcc98072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.840+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 0x7fcc98072fa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcca8035000 tx=0x7fcca8030000 comp rx=0 tx=0).stop 2023-10-21T16:17:47.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.840+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 msgr2=0x7fccb81f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:47.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.840+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fccb000c410 tx=0x7fccb000c8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 shutdown_connections 2023-10-21T16:17:47.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc98070bb0 0x7fcc98072fa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccb815adf0 0x7fccb81f6000 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fccb80d05b0 0x7fccb81f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 --2- 172.21.15.53:0/1842912380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fccb80cd410 0x7fccb80c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:47.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 >> 172.21.15.53:0/1842912380 conn(0x7fccb806e630 msgr2=0x7fccb815d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:47.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 shutdown_connections 2023-10-21T16:17:47.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:47.841+0000 7fccbece0640 1 -- 172.21.15.53:0/1842912380 wait complete. 2023-10-21T16:17:47.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.076+0000 7f3a03fff640 1 Processor -- start 2023-10-21T16:17:48.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.077+0000 7f3a03fff640 1 -- start start 2023-10-21T16:17:48.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3a040cc210 0x7f3a040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a04159db0 0x7f3a040b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040b10f0 con 0x7f3a04159db0 2023-10-21T16:17:48.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a040cacf0 con 0x7f3a040cc210 2023-10-21T16:17:48.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.078+0000 7f3a03fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040ca3c0 con 0x7f3a041599d0 2023-10-21T16:17:48.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a04159db0 0x7f3a040b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a04159db0 0x7f3a040b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56716/0 (socket says 172.21.15.53:56716) 2023-10-21T16:17:48.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 -- 172.21.15.53:0/1798407159 learned_addr learned my addr 172.21.15.53:0/1798407159 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:48.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a02ffd640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a027fc640 1 --2- 172.21.15.53:0/1798407159 >> v2:172.21.15.124:3300/0 conn(0x7f3a040cc210 0x7f3a040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 -- 172.21.15.53:0/1798407159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 msgr2=0x7f3a040ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:48.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 -- 172.21.15.53:0/1798407159 >> v2:172.21.15.124:3300/0 conn(0x7f3a040cc210 msgr2=0x7f3a040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:48.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 --2- 172.21.15.53:0/1798407159 >> v2:172.21.15.124:3300/0 conn(0x7f3a040cc210 0x7f3a040b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.079+0000 7f3a037fe640 1 -- 172.21.15.53:0/1798407159 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040ca6a0 con 0x7f3a04159db0 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.080+0000 7f3a037fe640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a040b0600 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cf740 tx=0x7f39fc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04a3a66d15b0d54 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a01ffb640 1 -- 172.21.15.53:0/1798407159 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc031860 con 0x7f3a04159db0 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a01ffb640 1 -- 172.21.15.53:0/1798407159 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc0319c0 con 0x7f3a04159db0 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a01ffb640 1 -- 172.21.15.53:0/1798407159 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc031d50 con 0x7f3a04159db0 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 -- 172.21.15.53:0/1798407159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 msgr2=0x7f3a040b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a040b0600 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3a040cf740 tx=0x7f39fc02b900 comp rx=0 tx=0).stop 2023-10-21T16:17:48.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 -- 172.21.15.53:0/1798407159 shutdown_connections 2023-10-21T16:17:48.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a040b0600 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 --2- 172.21.15.53:0/1798407159 >> v2:172.21.15.124:3300/0 conn(0x7f3a040cc210 0x7f3a040b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 --2- 172.21.15.53:0/1798407159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.081+0000 7f3a03fff640 1 -- 172.21.15.53:0/1798407159 >> 172.21.15.53:0/1798407159 conn(0x7f3a040cb0c0 msgr2=0x7f3a040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:48.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.082+0000 7f3a03fff640 1 -- 172.21.15.53:0/1798407159 shutdown_connections 2023-10-21T16:17:48.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.082+0000 7f3a03fff640 1 -- 172.21.15.53:0/1798407159 wait complete. 2023-10-21T16:17:48.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.083+0000 7f3a03fff640 1 Processor -- start 2023-10-21T16:17:48.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 -- start start 2023-10-21T16:17:48.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a040cc210 0x7f3a04157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040b0170 con 0x7f3a04159db0 2023-10-21T16:17:48.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.084+0000 7f3a03fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040aff40 con 0x7f3a040cc210 2023-10-21T16:17:48.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a03fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040ac5b0 con 0x7f3a041599d0 2023-10-21T16:17:48.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56720/0 (socket says 172.21.15.53:56720) 2023-10-21T16:17:48.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 -- 172.21.15.53:0/2797824829 learned_addr learned my addr 172.21.15.53:0/2797824829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:48.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a02ffd640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a040cc210 0x7f3a04157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a027fc640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 -- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 msgr2=0x7f3a040afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 -- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a040cc210 msgr2=0x7f3a04157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a040cc210 0x7f3a04157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.085+0000 7f3a037fe640 1 -- 172.21.15.53:0/2797824829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0094d0 con 0x7f3a04159db0 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f3a037fe640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b940 tx=0x7f39fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc03ac60 con 0x7f3a04159db0 2023-10-21T16:17:48.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc03adc0 con 0x7f3a04159db0 2023-10-21T16:17:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc044630 con 0x7f3a04159db0 2023-10-21T16:17:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040ac890 con 0x7f3a04159db0 2023-10-21T16:17:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.086+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a040ad050 con 0x7f3a04159db0 2023-10-21T16:17:48.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.087+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0415a550 con 0x7f3a04159db0 2023-10-21T16:17:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.088+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f39fc044790 con 0x7f3a04159db0 2023-10-21T16:17:48.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.089+0000 7f39ebfff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 0x7f39f006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.089+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f39fc0abcb0 con 0x7f3a04159db0 2023-10-21T16:17:48.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.090+0000 7f3a02ffd640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 0x7f39f006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.091+0000 7f3a02ffd640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 0x7f39f006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ce5f0 tx=0x7f39f400b040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.094+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39fc076cd0 con 0x7f3a04159db0 2023-10-21T16:17:48.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.325+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a0404e160 con 0x7f3a04159db0 2023-10-21T16:17:48.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.326+0000 7f39ebfff640 1 -- 172.21.15.53:0/2797824829 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f39fc035020 con 0x7f3a04159db0 2023-10-21T16:17:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:48.330 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:48.331 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:48.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.328+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 msgr2=0x7f39f006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 0x7f39f006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a040ce5f0 tx=0x7f39f400b040 comp rx=0 tx=0).stop 2023-10-21T16:17:48.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 msgr2=0x7f3a0406c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b940 tx=0x7f39fc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 shutdown_connections 2023-10-21T16:17:48.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f39f006acc0 0x7f39f006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a04159db0 0x7f3a0406c210 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a041599d0 0x7f3a040afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 --2- 172.21.15.53:0/2797824829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a040cc210 0x7f3a04157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 >> 172.21.15.53:0/2797824829 conn(0x7f3a040cb0c0 msgr2=0x7f3a04158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:48.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 shutdown_connections 2023-10-21T16:17:48.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.329+0000 7f3a03fff640 1 -- 172.21.15.53:0/2797824829 wait complete. 2023-10-21T16:17:48.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:48.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.571+0000 7fcc3b59e640 1 Processor -- start 2023-10-21T16:17:48.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.572+0000 7fcc3b59e640 1 -- start start 2023-10-21T16:17:48.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.573+0000 7fcc3b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc3c0cf720 0x7fcc3c06b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.573+0000 7fcc3b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3c0cb590 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.573+0000 7fcc3b59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3c0cfc90 con 0x7fcc3c06bd90 2023-10-21T16:17:48.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.573+0000 7fcc3b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3c06c220 con 0x7fcc3c0cf720 2023-10-21T16:17:48.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc3a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcc3c0cd8a0 0x7fcc3c06b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48926/0 (socket says 172.21.15.53:48926) 2023-10-21T16:17:48.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc3a59c640 1 -- 172.21.15.53:0/1725516493 learned_addr learned my addr 172.21.15.53:0/1725516493 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:48.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc33fff640 1 --2- 172.21.15.53:0/1725516493 >> v2:172.21.15.124:3300/0 conn(0x7fcc3c0cf720 0x7fcc3c06b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147496709 0 0) 0x7fcc3c0cb590 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc28003520 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369335648 0 0) 0x7fcc28003520 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 >> v2:172.21.15.124:3300/0 conn(0x7fcc3c0cf720 msgr2=0x7fcc3c06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:48.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.574+0000 7fcc39d9b640 1 --2- 172.21.15.53:0/1725516493 >> v2:172.21.15.124:3300/0 conn(0x7fcc3c0cf720 0x7fcc3c06b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 >> v1:172.21.15.124:6789/0 conn(0x7fcc3c06bd90 legacy=0x7fcc3c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0cff70 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcc2c001670 con 0x7fcc3c0cd8a0 2023-10-21T16:17:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 >> v1:172.21.15.53:6789/0 conn(0x7fcc3c0cd8a0 legacy=0x7fcc3c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280039c0 0x7fcc28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280047c0 0x7fcc28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 0x7fcc2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c06c220 con 0x7fcc28005180 2023-10-21T16:17:48.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2800c080 con 0x7fcc280047c0 2023-10-21T16:17:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2800c240 con 0x7fcc280039c0 2023-10-21T16:17:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc3ad9d640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 0x7fcc2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc3a59c640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280039c0 0x7fcc28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.575+0000 7fcc33fff640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280047c0 0x7fcc28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.576+0000 7fcc3ad9d640 1 -- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280039c0 msgr2=0x7fcc28003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.576+0000 7fcc3ad9d640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc280039c0 0x7fcc28003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.576+0000 7fcc3ad9d640 1 -- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280047c0 msgr2=0x7fcc28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.576+0000 7fcc3ad9d640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc280047c0 0x7fcc28004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.576+0000 7fcc3ad9d640 1 -- 172.21.15.53:0/1725516493 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28003520 con 0x7fcc28005180 2023-10-21T16:17:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.577+0000 7fcc3ad9d640 1 -- 172.21.15.53:0/1725516493 reap_dead start 2023-10-21T16:17:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.577+0000 7fcc3ad9d640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 0x7fcc2800b980 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcc34001e50 tx=0x7fcc280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f438155b73f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.577+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003b70 con 0x7fcc28005180 2023-10-21T16:17:48.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.577+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc34003d10 con 0x7fcc28005180 2023-10-21T16:17:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.577+0000 7fcc39d9b640 1 -- 172.21.15.53:0/1725516493 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34005310 con 0x7fcc28005180 2023-10-21T16:17:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 -- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 msgr2=0x7fcc2800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 0x7fcc2800b980 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fcc34001e50 tx=0x7fcc280057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 -- 172.21.15.53:0/1725516493 shutdown_connections 2023-10-21T16:17:48.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/1725516493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc28005180 0x7fcc2800b980 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 -- 172.21.15.53:0/1725516493 >> 172.21.15.53:0/1725516493 conn(0x7fcc3c099b60 msgr2=0x7fcc3c0d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:48.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 -- 172.21.15.53:0/1725516493 shutdown_connections 2023-10-21T16:17:48.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.578+0000 7fcc3b59e640 1 -- 172.21.15.53:0/1725516493 wait complete. 2023-10-21T16:17:48.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.580+0000 7fcc3b59e640 1 Processor -- start 2023-10-21T16:17:48.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.580+0000 7fcc3b59e640 1 -- start start 2023-10-21T16:17:48.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28005180 0x7fcc3c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc3c1f3770 0x7fcc3c1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0c98a0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0c4420 con 0x7fcc28005180 2023-10-21T16:17:48.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.581+0000 7fcc3b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c0c4700 con 0x7fcc3c1f3770 2023-10-21T16:17:48.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56736/0 (socket says 172.21.15.53:56736) 2023-10-21T16:17:48.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 -- 172.21.15.53:0/2389025878 learned_addr learned my addr 172.21.15.53:0/2389025878 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:48.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 -- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc3c1f3770 msgr2=0x7fcc3c1f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc3c1f3770 0x7fcc3c1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 -- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28005180 msgr2=0x7fcc3c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc3a59c640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28005180 0x7fcc3c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28005180 0x7fcc3c06f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.582+0000 7fcc33fff640 1 -- 172.21.15.53:0/2389025878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0c49e0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc33fff640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcc24011540 tx=0x7fcc24011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24015930 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24015a90 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24015e60 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3c0c4cc0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.583+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c06ff00 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.584+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3c04e160 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.585+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fcc2400f460 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.586+0000 7fcc32ffd640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 0x7fcc1c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:48.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.586+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fcc24089da0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.587+0000 7fcc3a59c640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 0x7fcc1c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:48.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.588+0000 7fcc3a59c640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 0x7fcc1c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0016d0 tx=0x7fcc2c000f30 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:48.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.591+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc24054ca0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.678+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcc24055070 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.717 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:48.714+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:48.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.797+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc3c06ff00 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.799+0000 7fcc32ffd640 1 -- 172.21.15.53:0/2389025878 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17791 (secure 0 0 0) 0x7fcc24084ec0 con 0x7fcc3c0c51e0 2023-10-21T16:17:48.801 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:48.801 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:46.645431+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/4222569834":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/3415687515":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6832/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/812907048":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/4157077207":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1100506891":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1810216649":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3437816425":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6832/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/2018576779":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3111202668":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/356555339":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3273170228":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3109992220":"2023-10-22T16:17:40.466814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.801+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 msgr2=0x7fcc1c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.801+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 0x7fcc1c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0016d0 tx=0x7fcc2c000f30 comp rx=0 tx=0).stop 2023-10-21T16:17:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 msgr2=0x7fcc3c0c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:48.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcc24011540 tx=0x7fcc24011a00 comp rx=0 tx=0).stop 2023-10-21T16:17:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 shutdown_connections 2023-10-21T16:17:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcc1c06acc0 0x7fcc1c06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc3c1f3770 0x7fcc3c1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc3c0c51e0 0x7fcc3c0c5640 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 --2- 172.21.15.53:0/2389025878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc28005180 0x7fcc3c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 >> 172.21.15.53:0/2389025878 conn(0x7fcc3c099b60 msgr2=0x7fcc3c0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 shutdown_connections 2023-10-21T16:17:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:48.802+0000 7fcc3b59e640 1 -- 172.21.15.53:0/2389025878 wait complete. 2023-10-21T16:17:48.823 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:17:48.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:49.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.035+0000 7f7a49b6a640 1 Processor -- start 2023-10-21T16:17:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.035+0000 7f7a49b6a640 1 -- start start 2023-10-21T16:17:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.036+0000 7f7a49b6a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a441599d0 0x7f7a440ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.036+0000 7f7a49b6a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440b07d0 0x7f7a4406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.036+0000 7f7a49b6a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a440cacf0 con 0x7f7a440cc210 2023-10-21T16:17:49.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.037+0000 7f7a49b6a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440b10f0 con 0x7f7a441599d0 2023-10-21T16:17:49.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.037+0000 7f7a49b6a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440ca3c0 con 0x7f7a440b07d0 2023-10-21T16:17:49.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.037+0000 7f7a48b68640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a441599d0 0x7f7a440ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.037+0000 7f7a48b68640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a441599d0 0x7f7a440ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:47356/0 (socket says 172.21.15.53:47356) 2023-10-21T16:17:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.037+0000 7f7a48b68640 1 -- 172.21.15.53:0/1970460072 learned_addr learned my addr 172.21.15.53:0/1970460072 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.038+0000 7f7a49369640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440b07d0 0x7f7a4406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.038+0000 7f7a48b68640 1 -- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440b07d0 msgr2=0x7f7a4406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:49.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.038+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440b07d0 0x7f7a4406c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.038+0000 7f7a48b68640 1 -- 172.21.15.53:0/1970460072 >> v1:172.21.15.124:6789/0 conn(0x7f7a440cc210 legacy=0x7f7a440b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.038+0000 7f7a48b68640 1 -- 172.21.15.53:0/1970460072 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440ca6a0 con 0x7f7a441599d0 2023-10-21T16:17:49.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.040+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a441599d0 0x7f7a440ad730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bb10 tx=0x7f7a3802e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=549781057ad27975 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.040+0000 7f7a437fe640 1 -- 172.21.15.53:0/1970460072 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38036440 con 0x7f7a441599d0 2023-10-21T16:17:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.040+0000 7f7a437fe640 1 -- 172.21.15.53:0/1970460072 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a380365a0 con 0x7f7a441599d0 2023-10-21T16:17:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.040+0000 7f7a437fe640 1 -- 172.21.15.53:0/1970460072 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38036930 con 0x7f7a441599d0 2023-10-21T16:17:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a441599d0 msgr2=0x7f7a440ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a441599d0 0x7f7a440ad730 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bb10 tx=0x7f7a3802e150 comp rx=0 tx=0).stop 2023-10-21T16:17:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1970460072 shutdown_connections 2023-10-21T16:17:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440b07d0 0x7f7a4406c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1970460072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a441599d0 0x7f7a440ad730 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1970460072 >> 172.21.15.53:0/1970460072 conn(0x7f7a440cb0c0 msgr2=0x7f7a440cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:49.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1970460072 shutdown_connections 2023-10-21T16:17:49.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.041+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1970460072 wait complete. 2023-10-21T16:17:49.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.043+0000 7f7a49b6a640 1 Processor -- start 2023-10-21T16:17:49.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.043+0000 7f7a49b6a640 1 -- start start 2023-10-21T16:17:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 0x7f7a4414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440cc210 0x7f7a441e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441e6e10 con 0x7f7a441599d0 2023-10-21T16:17:49.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441e6190 con 0x7f7a440b07d0 2023-10-21T16:17:49.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49b6a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a441e6470 con 0x7f7a440cc210 2023-10-21T16:17:49.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49369640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49369640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56744/0 (socket says 172.21.15.53:56744) 2023-10-21T16:17:49.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a49369640 1 -- 172.21.15.53:0/1226086479 learned_addr learned my addr 172.21.15.53:0/1226086479 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:49.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.044+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 0x7f7a4414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a43fff640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440cc210 0x7f7a441e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 -- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440cc210 msgr2=0x7f7a441e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440cc210 0x7f7a441e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 -- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 msgr2=0x7f7a4414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 0x7f7a4414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 -- 172.21.15.53:0/1226086479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3800b770 con 0x7f7a441599d0 2023-10-21T16:17:49.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 0x7f7a4414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:17:49.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.045+0000 7f7a49369640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00efc0 tx=0x7f7a3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.046+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c009ca0 con 0x7f7a441599d0 2023-10-21T16:17:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.046+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00f040 con 0x7f7a441599d0 2023-10-21T16:17:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.046+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0146c0 con 0x7f7a441599d0 2023-10-21T16:17:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.046+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a441e6750 con 0x7f7a441599d0 2023-10-21T16:17:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.046+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a441e5e20 con 0x7f7a441599d0 2023-10-21T16:17:49.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.047+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a441e6f50 con 0x7f7a441599d0 2023-10-21T16:17:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.048+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7a3c014820 con 0x7f7a441599d0 2023-10-21T16:17:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.049+0000 7f7a417fa640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 0x7f7a2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.049+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f7a3c08ab30 con 0x7f7a441599d0 2023-10-21T16:17:49.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.050+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 0x7f7a2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.051+0000 7f7a48b68640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 0x7f7a2806d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a38038000 tx=0x7f7a38008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.054+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a441e6f50 con 0x7f7a441599d0 2023-10-21T16:17:49.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.299+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a441e6f50 con 0x7f7a441599d0 2023-10-21T16:17:49.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.301+0000 7f7a417fa640 1 -- 172.21.15.53:0/1226086479 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7a441e6f50 con 0x7f7a441599d0 2023-10-21T16:17:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:49.303 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:49.303 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:49.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 msgr2=0x7f7a2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 0x7f7a2806d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7a38038000 tx=0x7f7a38008040 comp rx=0 tx=0).stop 2023-10-21T16:17:49.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 msgr2=0x7f7a441e4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00efc0 tx=0x7f7a3c00c490 comp rx=0 tx=0).stop 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 shutdown_connections 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7a2806af00 0x7f7a2806d3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a441599d0 0x7f7a441e4fe0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a440cc210 0x7f7a441e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 --2- 172.21.15.53:0/1226086479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a440b07d0 0x7f7a4414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 >> 172.21.15.53:0/1226086479 conn(0x7f7a440cb0c0 msgr2=0x7f7a440aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:49.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 shutdown_connections 2023-10-21T16:17:49.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.304+0000 7f7a49b6a640 1 -- 172.21.15.53:0/1226086479 wait complete. 2023-10-21T16:17:49.325 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:49.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.543+0000 7fe20359e640 1 Processor -- start 2023-10-21T16:17:49.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.543+0000 7fe20359e640 1 -- start start 2023-10-21T16:17:49.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe20359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2040adc70 0x7fe2040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe20359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe20415af10 0x7fe2040b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe20359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20415f7d0 con 0x7fe20415ab30 2023-10-21T16:17:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe20359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20415efb0 con 0x7fe20415af10 2023-10-21T16:17:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe20359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20415f210 con 0x7fe2040adc70 2023-10-21T16:17:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe202d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe20415af10 0x7fe2040b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe202d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe20415af10 0x7fe2040b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56746/0 (socket says 172.21.15.53:56746) 2023-10-21T16:17:49.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.545+0000 7fe202d9d640 1 -- 172.21.15.53:0/2229348885 learned_addr learned my addr 172.21.15.53:0/2229348885 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:49.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.546+0000 7fe1f9d9b640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2040adc70 0x7fe2040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.546+0000 7fe202d9d640 1 -- 172.21.15.53:0/2229348885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2040adc70 msgr2=0x7fe2040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:49.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.546+0000 7fe202d9d640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2040adc70 0x7fe2040b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.546+0000 7fe202d9d640 1 -- 172.21.15.53:0/2229348885 >> v1:172.21.15.124:6789/0 conn(0x7fe20415ab30 legacy=0x7fe20415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:49.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.546+0000 7fe202d9d640 1 -- 172.21.15.53:0/2229348885 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20415f4f0 con 0x7fe20415af10 2023-10-21T16:17:49.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.547+0000 7fe202d9d640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415af10 0x7fe2040b0600 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cb000 tx=0x7fe1fc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6661e57a30bb7487 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.547+0000 7fe201d9b640 1 -- 172.21.15.53:0/2229348885 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc02ca30 con 0x7fe20415af10 2023-10-21T16:17:49.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe201d9b640 1 -- 172.21.15.53:0/2229348885 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc02cb90 con 0x7fe20415af10 2023-10-21T16:17:49.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe201d9b640 1 -- 172.21.15.53:0/2229348885 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0343f0 con 0x7fe20415af10 2023-10-21T16:17:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 -- 172.21.15.53:0/2229348885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415af10 msgr2=0x7fe2040b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415af10 0x7fe2040b0600 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cb000 tx=0x7fe1fc02be90 comp rx=0 tx=0).stop 2023-10-21T16:17:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 -- 172.21.15.53:0/2229348885 shutdown_connections 2023-10-21T16:17:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2040adc70 0x7fe2040b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 --2- 172.21.15.53:0/2229348885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415af10 0x7fe2040b0600 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 -- 172.21.15.53:0/2229348885 >> 172.21.15.53:0/2229348885 conn(0x7fe2040cb0c0 msgr2=0x7fe2040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:49.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 -- 172.21.15.53:0/2229348885 shutdown_connections 2023-10-21T16:17:49.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.548+0000 7fe20359e640 1 -- 172.21.15.53:0/2229348885 wait complete. 2023-10-21T16:17:49.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.550+0000 7fe20359e640 1 Processor -- start 2023-10-21T16:17:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.550+0000 7fe20359e640 1 -- start start 2023-10-21T16:17:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2040adc70 0x7fe204158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20415af10 0x7fe204147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20414a780 con 0x7fe20415ab30 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204149be0 con 0x7fe2040adc70 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.551+0000 7fe20359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe204149e50 con 0x7fe20415af10 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56762/0 (socket says 172.21.15.53:56762) 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 -- 172.21.15.53:0/577130027 learned_addr learned my addr 172.21.15.53:0/577130027 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe20259c640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2040adc70 0x7fe204158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe202d9d640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20415af10 0x7fe204147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 -- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20415af10 msgr2=0x7fe204147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20415af10 0x7fe204147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 -- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2040adc70 msgr2=0x7fe204158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2040adc70 0x7fe204158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.552+0000 7fe1f9d9b640 1 -- 172.21.15.53:0/577130027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009b30 con 0x7fe20415ab30 2023-10-21T16:17:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe1f9d9b640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d800 tx=0x7fe1ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00b6c0 con 0x7fe20415ab30 2023-10-21T16:17:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec00b820 con 0x7fe20415ab30 2023-10-21T16:17:49.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00bbb0 con 0x7fe20415ab30 2023-10-21T16:17:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20414a130 con 0x7fe20415ab30 2023-10-21T16:17:49.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.553+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe204158660 con 0x7fe20415ab30 2023-10-21T16:17:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.554+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe204158820 con 0x7fe20415ab30 2023-10-21T16:17:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.555+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe1ec00bd10 con 0x7fe20415ab30 2023-10-21T16:17:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.556+0000 7fe1fb7fe640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 0x7fe1e406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:49.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.556+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fe1ec08a040 con 0x7fe20415ab30 2023-10-21T16:17:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.557+0000 7fe20259c640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 0x7fe1e406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.558+0000 7fe20259c640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 0x7fe1e406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4004ea0 tx=0x7fe1f4009040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.560+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1ec054c50 con 0x7fe20415ab30 2023-10-21T16:17:49.575 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:49.572+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:49.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.799+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe20406c830 con 0x7fe20415ab30 2023-10-21T16:17:49.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.800+0000 7fe1fb7fe640 1 -- 172.21.15.53:0/577130027 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fe1ec058270 con 0x7fe20415ab30 2023-10-21T16:17:49.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:49.802 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:17:25.738039+0000","modified":"2023-10-21T16:17:28.815415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35345":{"gid":35345,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/2020660539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2020660539},{"type":"v1","addr":"172.21.15.53:6835","nonce":2020660539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-21T16:17:49.802 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 36 2023-10-21T16:17:49.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 msgr2=0x7fe1e406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 0x7fe1e406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4004ea0 tx=0x7fe1f4009040 comp rx=0 tx=0).stop 2023-10-21T16:17:49.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 msgr2=0x7fe204146e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:49.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00d800 tx=0x7fe1ec00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 shutdown_connections 2023-10-21T16:17:49.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe1e406af00 0x7fe1e406d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20415af10 0x7fe204147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20415ab30 0x7fe204146e40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 --2- 172.21.15.53:0/577130027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2040adc70 0x7fe204158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 >> 172.21.15.53:0/577130027 conn(0x7fe2040cb0c0 msgr2=0x7fe2040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:49.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 shutdown_connections 2023-10-21T16:17:49.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:49.803+0000 7fe20359e640 1 -- 172.21.15.53:0/577130027 wait complete. 2023-10-21T16:17:49.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:50.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.051+0000 7fc4f9832640 1 Processor -- start 2023-10-21T16:17:50.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.051+0000 7fc4f9832640 1 -- start start 2023-10-21T16:17:50.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4f40cc210 0x7fc4f40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc4f4159db0 0x7fc4f40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f40b10f0 con 0x7fc4f40cc210 2023-10-21T16:17:50.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f40cacf0 con 0x7fc4f4159db0 2023-10-21T16:17:50.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f9832640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f40ca3c0 con 0x7fc4f41599d0 2023-10-21T16:17:50.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4f40cc210 0x7fc4f40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4f40cc210 0x7fc4f40b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56772/0 (socket says 172.21.15.53:56772) 2023-10-21T16:17:50.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.053+0000 7fc4f3fff640 1 -- 172.21.15.53:0/688647562 learned_addr learned my addr 172.21.15.53:0/688647562 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:50.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f8830640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f9031640 1 --2- 172.21.15.53:0/688647562 >> v2:172.21.15.124:3300/0 conn(0x7fc4f4159db0 0x7fc4f40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f3fff640 1 -- 172.21.15.53:0/688647562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 msgr2=0x7fc4f40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:50.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f3fff640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f3fff640 1 -- 172.21.15.53:0/688647562 >> v2:172.21.15.124:3300/0 conn(0x7fc4f4159db0 msgr2=0x7fc4f40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:50.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f3fff640 1 --2- 172.21.15.53:0/688647562 >> v2:172.21.15.124:3300/0 conn(0x7fc4f4159db0 0x7fc4f40b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.054+0000 7fc4f3fff640 1 -- 172.21.15.53:0/688647562 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f40ca6a0 con 0x7fc4f40cc210 2023-10-21T16:17:50.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.055+0000 7fc4f3fff640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f40cc210 0x7fc4f40b00c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40b4430 tx=0x7fc4e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a358427261863fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.056+0000 7fc4f37fe640 1 -- 172.21.15.53:0/688647562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0031870 con 0x7fc4f40cc210 2023-10-21T16:17:50.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.056+0000 7fc4f37fe640 1 -- 172.21.15.53:0/688647562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e00319d0 con 0x7fc4f40cc210 2023-10-21T16:17:50.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.056+0000 7fc4f37fe640 1 -- 172.21.15.53:0/688647562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0031d60 con 0x7fc4f40cc210 2023-10-21T16:17:50.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.056+0000 7fc4f9832640 1 -- 172.21.15.53:0/688647562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f40cc210 msgr2=0x7fc4f40b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.056+0000 7fc4f9832640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f40cc210 0x7fc4f40b00c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40b4430 tx=0x7fc4e002b910 comp rx=0 tx=0).stop 2023-10-21T16:17:50.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 -- 172.21.15.53:0/688647562 shutdown_connections 2023-10-21T16:17:50.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 --2- 172.21.15.53:0/688647562 >> v2:172.21.15.124:3300/0 conn(0x7fc4f4159db0 0x7fc4f40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f40cc210 0x7fc4f40b00c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 --2- 172.21.15.53:0/688647562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 -- 172.21.15.53:0/688647562 >> 172.21.15.53:0/688647562 conn(0x7fc4f40cb0c0 msgr2=0x7fc4f40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:50.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 -- 172.21.15.53:0/688647562 shutdown_connections 2023-10-21T16:17:50.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.057+0000 7fc4f9832640 1 -- 172.21.15.53:0/688647562 wait complete. 2023-10-21T16:17:50.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.059+0000 7fc4f9832640 1 Processor -- start 2023-10-21T16:17:50.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.059+0000 7fc4f9832640 1 -- start start 2023-10-21T16:17:50.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 0x7fc4f415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f415be40 con 0x7fc4f4159db0 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f40b04f0 con 0x7fc4f40cc210 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9832640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f415b230 con 0x7fc4f41599d0 2023-10-21T16:17:50.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9031640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9031640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56776/0 (socket says 172.21.15.53:56776) 2023-10-21T16:17:50.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f9031640 1 -- 172.21.15.53:0/2498352898 learned_addr learned my addr 172.21.15.53:0/2498352898 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:50.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f8830640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 0x7fc4f415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.060+0000 7fc4f3fff640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 -- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 msgr2=0x7fc4f40affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 -- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 msgr2=0x7fc4f415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 0x7fc4f415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 -- 172.21.15.53:0/2498352898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00094d0 con 0x7fc4f4159db0 2023-10-21T16:17:50.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f3fff640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:50.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f8830640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 0x7fc4f415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:50.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9031640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e60 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014420 con 0x7fc4f4159db0 2023-10-21T16:17:50.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec0145a0 con 0x7fc4f4159db0 2023-10-21T16:17:50.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec014950 con 0x7fc4f4159db0 2023-10-21T16:17:50.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.061+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f415b510 con 0x7fc4f4159db0 2023-10-21T16:17:50.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.062+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f415bad0 con 0x7fc4f4159db0 2023-10-21T16:17:50.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.063+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f415bfb0 con 0x7fc4f4159db0 2023-10-21T16:17:50.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.064+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc4ec014ab0 con 0x7fc4f4159db0 2023-10-21T16:17:50.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.065+0000 7fc4f17fa640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 0x7fc4d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.065+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fc4ec08a450 con 0x7fc4f4159db0 2023-10-21T16:17:50.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.065+0000 7fc4f8830640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 0x7fc4d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.066+0000 7fc4f8830640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 0x7fc4d806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8006fd0 tx=0x7fc4e8008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.069+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4ec054ff0 con 0x7fc4f4159db0 2023-10-21T16:17:50.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.271+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4f404e160 con 0x7fc4f4159db0 2023-10-21T16:17:50.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.274+0000 7fc4f17fa640 1 -- 172.21.15.53:0/2498352898 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17791 (secure 0 0 0) 0x7fc4ec058610 con 0x7fc4f4159db0 2023-10-21T16:17:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:50.276 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":92,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:46.645431+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:17:23.670145+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-21T16:17:24.763686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/4222569834":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/3415687515":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6832/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/812907048":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/4157077207":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1100506891":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1810216649":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3437816425":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6832/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/2018576779":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3111202668":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/356555339":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3273170228":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3109992220":"2023-10-22T16:17:40.466814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:50.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 msgr2=0x7fc4d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 0x7fc4d806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8006fd0 tx=0x7fc4e8008040 comp rx=0 tx=0).stop 2023-10-21T16:17:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 msgr2=0x7fc4f4156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007e60 tx=0x7fc4ec00e4b0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 shutdown_connections 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fc4d806af30 0x7fc4d806d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4f4159db0 0x7fc4f4156830 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc4f41599d0 0x7fc4f40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 --2- 172.21.15.53:0/2498352898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc4f40cc210 0x7fc4f415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 >> 172.21.15.53:0/2498352898 conn(0x7fc4f40cb0c0 msgr2=0x7fc4f4158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:50.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 shutdown_connections 2023-10-21T16:17:50.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.277+0000 7fc4f9832640 1 -- 172.21.15.53:0/2498352898 wait complete. 2023-10-21T16:17:50.301 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:17:50.447 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:50.444+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:50.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.529+0000 7fb4d9dac640 1 Processor -- start 2023-10-21T16:17:50.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.529+0000 7fb4d9dac640 1 -- start start 2023-10-21T16:17:50.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.530+0000 7fb4d9dac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4d415a820 0x7fb4d40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d9dac640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40cab60 con 0x7fb4d415ac00 2023-10-21T16:17:50.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d9dac640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40cb960 con 0x7fb4d40ca680 2023-10-21T16:17:50.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d9dac640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d40cbdf0 con 0x7fb4d415a820 2023-10-21T16:17:50.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4d415a820 0x7fb4d40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4d415a820 0x7fb4d40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56786/0 (socket says 172.21.15.53:56786) 2023-10-21T16:17:50.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 -- 172.21.15.53:0/215636181 learned_addr learned my addr 172.21.15.53:0/215636181 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:50.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 -- 172.21.15.53:0/215636181 >> v1:172.21.15.124:6789/0 conn(0x7fb4d40ca680 legacy=0x7fb4d40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:50.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 -- 172.21.15.53:0/215636181 >> v1:172.21.15.53:6789/0 conn(0x7fb4d415ac00 legacy=0x7fb4d40c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.531+0000 7fb4d8daa640 1 -- 172.21.15.53:0/215636181 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d40c9f20 con 0x7fb4d415a820 2023-10-21T16:17:50.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.533+0000 7fb4d8daa640 1 --2- 172.21.15.53:0/215636181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d415a820 0x7fb4d40aff70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009870 tx=0x7fb4c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcc79401329e013 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.533+0000 7fb4d37fe640 1 -- 172.21.15.53:0/215636181 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8031870 con 0x7fb4d415a820 2023-10-21T16:17:50.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.533+0000 7fb4d37fe640 1 -- 172.21.15.53:0/215636181 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c80319d0 con 0x7fb4d415a820 2023-10-21T16:17:50.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.533+0000 7fb4d37fe640 1 -- 172.21.15.53:0/215636181 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8031d60 con 0x7fb4d415a820 2023-10-21T16:17:50.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 -- 172.21.15.53:0/215636181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d415a820 msgr2=0x7fb4d40aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/215636181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d415a820 0x7fb4d40aff70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009870 tx=0x7fb4c802b910 comp rx=0 tx=0).stop 2023-10-21T16:17:50.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 -- 172.21.15.53:0/215636181 shutdown_connections 2023-10-21T16:17:50.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/215636181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d415a820 0x7fb4d40aff70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 -- 172.21.15.53:0/215636181 >> 172.21.15.53:0/215636181 conn(0x7fb4d406c050 msgr2=0x7fb4d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:50.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 -- 172.21.15.53:0/215636181 shutdown_connections 2023-10-21T16:17:50.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.534+0000 7fb4d9dac640 1 -- 172.21.15.53:0/215636181 wait complete. 2023-10-21T16:17:50.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.536+0000 7fb4d9dac640 1 Processor -- start 2023-10-21T16:17:50.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.536+0000 7fb4d9dac640 1 -- start start 2023-10-21T16:17:50.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4d415a820 0x7fb4d40ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4d415ac00 0x7fb4d40cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40afdb0 con 0x7fb4d40ca680 2023-10-21T16:17:50.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40ae050 con 0x7fb4d415a820 2023-10-21T16:17:50.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d9dac640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40add70 con 0x7fb4d415ac00 2023-10-21T16:17:50.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d8daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d8daa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56794/0 (socket says 172.21.15.53:56794) 2023-10-21T16:17:50.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.537+0000 7fb4d8daa640 1 -- 172.21.15.53:0/2573850027 learned_addr learned my addr 172.21.15.53:0/2573850027 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:50.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 -- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4d415ac00 msgr2=0x7fb4d40cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4d415ac00 0x7fb4d40cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 -- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4d415a820 msgr2=0x7fb4d40ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4d415a820 0x7fb4d40ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 -- 172.21.15.53:0/2573850027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c80094d0 con 0x7fb4d40ca680 2023-10-21T16:17:50.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d8daa640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8031820 con 0x7fb4d40ca680 2023-10-21T16:17:50.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c80443f0 con 0x7fb4d40ca680 2023-10-21T16:17:50.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8044780 con 0x7fb4d40ca680 2023-10-21T16:17:50.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d40af480 con 0x7fb4d40ca680 2023-10-21T16:17:50.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.538+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d40afa40 con 0x7fb4d40ca680 2023-10-21T16:17:50.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.540+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d4157ad0 con 0x7fb4d40ca680 2023-10-21T16:17:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.541+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb4c804d450 con 0x7fb4d40ca680 2023-10-21T16:17:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.542+0000 7fb4d17fa640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 0x7fb4b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.542+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fb4c802d040 con 0x7fb4d40ca680 2023-10-21T16:17:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.542+0000 7fb4d3fff640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 0x7fb4b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:50.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.544+0000 7fb4d3fff640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 0x7fb4b806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40ce250 tx=0x7fb4c0007990 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:50.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.546+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4c8076e50 con 0x7fb4d40ca680 2023-10-21T16:17:50.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.781+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb4d4074150 con 0x7fb4d40ca680 2023-10-21T16:17:50.786 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:17:50.782+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:17:50.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.786+0000 7fb4d17fa640 1 -- 172.21.15.53:0/2573850027 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb4c807a470 con 0x7fb4d40ca680 2023-10-21T16:17:50.788 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:17:50.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 msgr2=0x7fb4b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 0x7fb4b806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40ce250 tx=0x7fb4c0007990 comp rx=0 tx=0).stop 2023-10-21T16:17:50.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 msgr2=0x7fb4d414edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c80046c0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 shutdown_connections 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fb4b806af30 0x7fb4b806d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4d415ac00 0x7fb4d40cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4d415a820 0x7fb4d40ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 --2- 172.21.15.53:0/2573850027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4d40ca680 0x7fb4d414edb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:50.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.789+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 >> 172.21.15.53:0/2573850027 conn(0x7fb4d406c050 msgr2=0x7fb4d41573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:50.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.791+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 shutdown_connections 2023-10-21T16:17:50.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:50.791+0000 7fb4d9dac640 1 -- 172.21.15.53:0/2573850027 wait complete. 2023-10-21T16:17:50.805 INFO:tasks.ceph.mds.c.smithi053.stderr: -3720> 2023-10-21T16:17:32.224+0000 7f0bb4d6c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:17:50.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:17:50.892 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:17:50.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:17:50.946 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:17:51.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.031+0000 7f78a6602640 1 Processor -- start 2023-10-21T16:17:51.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.032+0000 7f78a6602640 1 -- start start 2023-10-21T16:17:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a6602640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a6602640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78a015aa00 0x7f78a00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a6602640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a015b310 con 0x7f78a006c3a0 2023-10-21T16:17:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a6602640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a006e0a0 con 0x7f78a015aa00 2023-10-21T16:17:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a6602640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a006d290 con 0x7f78a00c9cf0 2023-10-21T16:17:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a4b9f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78a015aa00 0x7f78a00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a4b9f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78a015aa00 0x7f78a00b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56822/0 (socket says 172.21.15.53:56822) 2023-10-21T16:17:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.033+0000 7f78a4b9f640 1 -- 172.21.15.53:0/88748423 learned_addr learned my addr 172.21.15.53:0/88748423 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.034+0000 7f7897fff640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.034+0000 7f78a4b9f640 1 -- 172.21.15.53:0/88748423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 msgr2=0x7f78a006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.034+0000 7f78a4b9f640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.034+0000 7f78a4b9f640 1 -- 172.21.15.53:0/88748423 >> v1:172.21.15.124:6789/0 conn(0x7f78a006c3a0 legacy=0x7f78a006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.034+0000 7f78a4b9f640 1 -- 172.21.15.53:0/88748423 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a00c9860 con 0x7f78a015aa00 2023-10-21T16:17:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.035+0000 7f78a4b9f640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a015aa00 0x7f78a00b2a90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7898009e50 tx=0x7f789802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af36974a15355a4e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.035+0000 7f789f7fe640 1 -- 172.21.15.53:0/88748423 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789802cb00 con 0x7f78a015aa00 2023-10-21T16:17:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.035+0000 7f789f7fe640 1 -- 172.21.15.53:0/88748423 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789802cc60 con 0x7f78a015aa00 2023-10-21T16:17:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.035+0000 7f789f7fe640 1 -- 172.21.15.53:0/88748423 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980343f0 con 0x7f78a015aa00 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 -- 172.21.15.53:0/88748423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a015aa00 msgr2=0x7f78a00b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a015aa00 0x7f78a00b2a90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7898009e50 tx=0x7f789802bf60 comp rx=0 tx=0).stop 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 -- 172.21.15.53:0/88748423 shutdown_connections 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 --2- 172.21.15.53:0/88748423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a015aa00 0x7f78a00b2a90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 -- 172.21.15.53:0/88748423 >> 172.21.15.53:0/88748423 conn(0x7f78a006a5d0 msgr2=0x7f78a00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:51.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 -- 172.21.15.53:0/88748423 shutdown_connections 2023-10-21T16:17:51.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.036+0000 7f78a6602640 1 -- 172.21.15.53:0/88748423 wait complete. 2023-10-21T16:17:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.038+0000 7f78a6602640 1 Processor -- start 2023-10-21T16:17:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.038+0000 7f78a6602640 1 -- start start 2023-10-21T16:17:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a00c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78a015aa00 0x7f78a01545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a00c51b0 con 0x7f78a006c3a0 2023-10-21T16:17:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a015db30 con 0x7f78a015aa00 2023-10-21T16:17:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f78a6602640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a015de10 con 0x7f78a00c9cf0 2023-10-21T16:17:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.039+0000 7f789ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56824/0 (socket says 172.21.15.53:56824) 2023-10-21T16:17:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 -- 172.21.15.53:0/4197832749 learned_addr learned my addr 172.21.15.53:0/4197832749 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f78a4b9f640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78a015aa00 0x7f78a01545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f7897fff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a00c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 -- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 msgr2=0x7f78a00c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a00c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 -- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78a015aa00 msgr2=0x7f78a01545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78a015aa00 0x7f78a01545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 -- 172.21.15.53:0/4197832749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009b30 con 0x7f78a006c3a0 2023-10-21T16:17:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f7897fff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a00c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.040+0000 7f789ffff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f789000c450 tx=0x7f789000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.041+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900052d0 con 0x7f78a006c3a0 2023-10-21T16:17:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.041+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7890005430 con 0x7f78a006c3a0 2023-10-21T16:17:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.041+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890012870 con 0x7f78a006c3a0 2023-10-21T16:17:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.041+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a0153120 con 0x7f78a006c3a0 2023-10-21T16:17:51.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.041+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a01536e0 con 0x7f78a006c3a0 2023-10-21T16:17:51.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.042+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a004e160 con 0x7f78a006c3a0 2023-10-21T16:17:51.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.043+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f78900129d0 con 0x7f78a006c3a0 2023-10-21T16:17:51.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.044+0000 7f789d7fa640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 0x7f788006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:51.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.044+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f7890089450 con 0x7f78a006c3a0 2023-10-21T16:17:51.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.045+0000 7f7897fff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 0x7f788006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:51.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.046+0000 7f7897fff640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 0x7f788006d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f78a0155710 tx=0x7f7888008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:51.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.048+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7890054030 con 0x7f78a006c3a0 2023-10-21T16:17:51.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.297+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f78a00cd1d0 con 0x7f78a006c3a0 2023-10-21T16:17:51.305 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:51.302+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:51.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.789+0000 7f789d7fa640 1 -- 172.21.15.53:0/4197832749 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7890057650 con 0x7f78a006c3a0 2023-10-21T16:17:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 msgr2=0x7f788006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 0x7f788006d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f78a0155710 tx=0x7f7888008040 comp rx=0 tx=0).stop 2023-10-21T16:17:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 msgr2=0x7f78a0151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f789000c450 tx=0x7f789000c910 comp rx=0 tx=0).stop 2023-10-21T16:17:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 shutdown_connections 2023-10-21T16:17:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f788006af30 0x7f788006d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78a015aa00 0x7f78a01545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78a00c9cf0 0x7f78a00c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 --2- 172.21.15.53:0/4197832749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78a006c3a0 0x7f78a0151f10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 >> 172.21.15.53:0/4197832749 conn(0x7f78a006a5d0 msgr2=0x7f78a00c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 shutdown_connections 2023-10-21T16:17:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:51.794+0000 7f78a6602640 1 -- 172.21.15.53:0/4197832749 wait complete. 2023-10-21T16:17:51.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:17:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.044+0000 7f18b60b4640 1 Processor -- start 2023-10-21T16:17:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.045+0000 7f18b60b4640 1 -- start start 2023-10-21T16:17:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.046+0000 7f18b60b4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f18b015cae0 0x7f18b00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.046+0000 7f18b60b4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18b00b0680 0x7f18b00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.046+0000 7f18b60b4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b00ccbc0 con 0x7f18b00c8b40 2023-10-21T16:17:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.046+0000 7f18b60b4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00c9510 con 0x7f18b00b0680 2023-10-21T16:17:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.046+0000 7f18b60b4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00ce1f0 con 0x7f18b015cae0 2023-10-21T16:17:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18b00b0680 0x7f18b00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18b00b0680 0x7f18b00c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56836/0 (socket says 172.21.15.53:56836) 2023-10-21T16:17:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 -- 172.21.15.53:0/2243370403 learned_addr learned my addr 172.21.15.53:0/2243370403 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b50b2640 1 --2- 172.21.15.53:0/2243370403 >> v2:172.21.15.124:3300/0 conn(0x7f18b015cae0 0x7f18b00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 -- 172.21.15.53:0/2243370403 >> v2:172.21.15.124:3300/0 conn(0x7f18b015cae0 msgr2=0x7f18b00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 --2- 172.21.15.53:0/2243370403 >> v2:172.21.15.124:3300/0 conn(0x7f18b015cae0 0x7f18b00cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 -- 172.21.15.53:0/2243370403 >> v1:172.21.15.124:6789/0 conn(0x7f18b00c8b40 legacy=0x7f18b00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.047+0000 7f18b58b3640 1 -- 172.21.15.53:0/2243370403 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00cd590 con 0x7f18b00b0680 2023-10-21T16:17:52.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.048+0000 7f18b58b3640 1 --2- 172.21.15.53:0/2243370403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00b0680 0x7f18b00c8fd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f18b00caed0 tx=0x7f18a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce4d71174347ed9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:52.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.049+0000 7f18a7fff640 1 -- 172.21.15.53:0/2243370403 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8031870 con 0x7f18b00b0680 2023-10-21T16:17:52.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.049+0000 7f18a7fff640 1 -- 172.21.15.53:0/2243370403 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80319d0 con 0x7f18b00b0680 2023-10-21T16:17:52.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.049+0000 7f18a7fff640 1 -- 172.21.15.53:0/2243370403 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8031d60 con 0x7f18b00b0680 2023-10-21T16:17:52.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 -- 172.21.15.53:0/2243370403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00b0680 msgr2=0x7f18b00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:52.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 --2- 172.21.15.53:0/2243370403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00b0680 0x7f18b00c8fd0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f18b00caed0 tx=0x7f18a802b910 comp rx=0 tx=0).stop 2023-10-21T16:17:52.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 -- 172.21.15.53:0/2243370403 shutdown_connections 2023-10-21T16:17:52.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 --2- 172.21.15.53:0/2243370403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00b0680 0x7f18b00c8fd0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 --2- 172.21.15.53:0/2243370403 >> v2:172.21.15.124:3300/0 conn(0x7f18b015cae0 0x7f18b00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 -- 172.21.15.53:0/2243370403 >> 172.21.15.53:0/2243370403 conn(0x7f18b006c050 msgr2=0x7f18b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:52.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 -- 172.21.15.53:0/2243370403 shutdown_connections 2023-10-21T16:17:52.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.050+0000 7f18b60b4640 1 -- 172.21.15.53:0/2243370403 wait complete. 2023-10-21T16:17:52.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.052+0000 7f18b60b4640 1 Processor -- start 2023-10-21T16:17:52.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.052+0000 7f18b60b4640 1 -- start start 2023-10-21T16:17:52.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b00b0680 0x7f18b0152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 0x7f18b00c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01f3db0 con 0x7f18b00c8b40 2023-10-21T16:17:52.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01f3710 con 0x7f18b015cae0 2023-10-21T16:17:52.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b60b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00c7b90 con 0x7f18b00b0680 2023-10-21T16:17:52.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b48b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b48b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56840/0 (socket says 172.21.15.53:56840) 2023-10-21T16:17:52.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b48b1640 1 -- 172.21.15.53:0/3447920971 learned_addr learned my addr 172.21.15.53:0/3447920971 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:52.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.053+0000 7f18b50b2640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b00b0680 0x7f18b0152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b58b3640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 0x7f18b00c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 -- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b00b0680 msgr2=0x7f18b0152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:52.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b00b0680 0x7f18b0152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 -- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 msgr2=0x7f18b00c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:52.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 0x7f18b00c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 -- 172.21.15.53:0/3447920971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80094d0 con 0x7f18b00c8b40 2023-10-21T16:17:52.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b58b3640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 0x7f18b00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:52.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b48b1640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c7a0 tx=0x7f18a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0003d80 con 0x7f18b00c8b40 2023-10-21T16:17:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0005070 con 0x7f18b00c8b40 2023-10-21T16:17:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0005460 con 0x7f18b00c8b40 2023-10-21T16:17:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.054+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b00c7e70 con 0x7f18b00c8b40 2023-10-21T16:17:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.055+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b015d940 con 0x7f18b00c8b40 2023-10-21T16:17:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.056+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b01f3ef0 con 0x7f18b00c8b40 2023-10-21T16:17:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.057+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f18a00055c0 con 0x7f18b00c8b40 2023-10-21T16:17:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.058+0000 7f18a5ffb640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 0x7f189406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.058+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f18a0088eb0 con 0x7f18b00c8b40 2023-10-21T16:17:52.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.059+0000 7f18b50b2640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 0x7f189406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:52.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.060+0000 7f18b50b2640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 0x7f189406d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:52.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.063+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18b01f3ef0 con 0x7f18b00c8b40 2023-10-21T16:17:52.216 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:52.213+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:52.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.260+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18b01f3ef0 con 0x7f18b00c8b40 2023-10-21T16:17:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.793+0000 7f18a5ffb640 1 -- 172.21.15.53:0/3447920971 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f18b01f3ef0 con 0x7f18b00c8b40 2023-10-21T16:17:52.795 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:17:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.796+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 msgr2=0x7f189406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.796+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 0x7f189406d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac008040 comp rx=0 tx=0).stop 2023-10-21T16:17:52.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 msgr2=0x7f18b00c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c7a0 tx=0x7f18a000cc60 comp rx=0 tx=0).stop 2023-10-21T16:17:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 shutdown_connections 2023-10-21T16:17:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f189406af00 0x7f189406d3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b015cae0 0x7f18b00c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18b00c8b40 0x7f18b00c6fa0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 --2- 172.21.15.53:0/3447920971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b00b0680 0x7f18b0152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:52.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.797+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 >> 172.21.15.53:0/3447920971 conn(0x7f18b006c050 msgr2=0x7f18b00acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:52.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.799+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 shutdown_connections 2023-10-21T16:17:52.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:52.799+0000 7f18b60b4640 1 -- 172.21.15.53:0/3447920971 wait complete. 2023-10-21T16:17:52.826 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:17:53.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.058+0000 7f279f202640 1 Processor -- start 2023-10-21T16:17:53.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.058+0000 7f279f202640 1 -- start start 2023-10-21T16:17:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27980702b0 0x7f2798070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980b21f0 0x7f27980c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f27980ca1a0 0x7f279806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279806c3a0 con 0x7f27980702b0 2023-10-21T16:17:53.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2798070bd0 con 0x7f27980ca1a0 2023-10-21T16:17:53.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.059+0000 7f279f202640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279806c640 con 0x7f27980b21f0 2023-10-21T16:17:53.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27980702b0 0x7f2798070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27980702b0 0x7f2798070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56848/0 (socket says 172.21.15.53:56848) 2023-10-21T16:17:53.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 -- 172.21.15.53:0/720357073 learned_addr learned my addr 172.21.15.53:0/720357073 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:53.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279d79f640 1 --2- 172.21.15.53:0/720357073 >> v2:172.21.15.124:3300/0 conn(0x7f27980ca1a0 0x7f279806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 -- 172.21.15.53:0/720357073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980b21f0 msgr2=0x7f27980c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 --2- 172.21.15.53:0/720357073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980b21f0 0x7f27980c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 -- 172.21.15.53:0/720357073 >> v2:172.21.15.124:3300/0 conn(0x7f27980ca1a0 msgr2=0x7f279806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 --2- 172.21.15.53:0/720357073 >> v2:172.21.15.124:3300/0 conn(0x7f27980ca1a0 0x7f279806d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.060+0000 7f279cf9e640 1 -- 172.21.15.53:0/720357073 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279806c920 con 0x7f27980702b0 2023-10-21T16:17:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.061+0000 7f279cf9e640 1 --2- 172.21.15.53:0/720357073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980702b0 0x7f2798070690 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f279806e310 tx=0x7f278c02dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c4bec8048232fd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:53.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.062+0000 7f27977fe640 1 -- 172.21.15.53:0/720357073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c02ec50 con 0x7f27980702b0 2023-10-21T16:17:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.062+0000 7f27977fe640 1 -- 172.21.15.53:0/720357073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c032070 con 0x7f27980702b0 2023-10-21T16:17:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.062+0000 7f27977fe640 1 -- 172.21.15.53:0/720357073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c038400 con 0x7f27980702b0 2023-10-21T16:17:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.062+0000 7f279f202640 1 -- 172.21.15.53:0/720357073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980702b0 msgr2=0x7f2798070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.062+0000 7f279f202640 1 --2- 172.21.15.53:0/720357073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980702b0 0x7f2798070690 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f279806e310 tx=0x7f278c02dea0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 -- 172.21.15.53:0/720357073 shutdown_connections 2023-10-21T16:17:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 --2- 172.21.15.53:0/720357073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980b21f0 0x7f27980c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 --2- 172.21.15.53:0/720357073 >> v2:172.21.15.124:3300/0 conn(0x7f27980ca1a0 0x7f279806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 --2- 172.21.15.53:0/720357073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980702b0 0x7f2798070690 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 -- 172.21.15.53:0/720357073 >> 172.21.15.53:0/720357073 conn(0x7f279806a5d0 msgr2=0x7f27980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 -- 172.21.15.53:0/720357073 shutdown_connections 2023-10-21T16:17:53.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.063+0000 7f279f202640 1 -- 172.21.15.53:0/720357073 wait complete. 2023-10-21T16:17:53.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.065+0000 7f279f202640 1 Processor -- start 2023-10-21T16:17:53.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.065+0000 7f279f202640 1 -- start start 2023-10-21T16:17:53.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 0x7f2798143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27980ca1a0 0x7f2798155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279815b630 con 0x7f27980b21f0 2023-10-21T16:17:53.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279815aa80 con 0x7f27980ca1a0 2023-10-21T16:17:53.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279f202640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279815ac90 con 0x7f27980702b0 2023-10-21T16:17:53.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f2797fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f2797fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56864/0 (socket says 172.21.15.53:56864) 2023-10-21T16:17:53.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f2797fff640 1 -- 172.21.15.53:0/3016900865 learned_addr learned my addr 172.21.15.53:0/3016900865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:53.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279cf9e640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 0x7f2798143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f279d79f640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27980ca1a0 0x7f2798155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f2797fff640 1 -- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 msgr2=0x7f2798143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:53.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.066+0000 7f2797fff640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 0x7f2798143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f2797fff640 1 -- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27980ca1a0 msgr2=0x7f2798155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:53.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f2797fff640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27980ca1a0 0x7f2798155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f2797fff640 1 -- 172.21.15.53:0/3016900865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c00b760 con 0x7f27980b21f0 2023-10-21T16:17:53.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f279cf9e640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 0x7f2798143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:53.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f2797fff640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f278800b4a0 tx=0x7f278800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:53.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788010610 con 0x7f27980b21f0 2023-10-21T16:17:53.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788010770 con 0x7f27980b21f0 2023-10-21T16:17:53.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788010b20 con 0x7f27980b21f0 2023-10-21T16:17:53.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279815af70 con 0x7f27980b21f0 2023-10-21T16:17:53.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.067+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980c4770 con 0x7f27980b21f0 2023-10-21T16:17:53.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.068+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279815b770 con 0x7f27980b21f0 2023-10-21T16:17:53.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.070+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2788010c80 con 0x7f27980b21f0 2023-10-21T16:17:53.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.070+0000 7f27957fa640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 0x7f278406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:53.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.071+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f2788089510 con 0x7f27980b21f0 2023-10-21T16:17:53.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.071+0000 7f279cf9e640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 0x7f278406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.072+0000 7f279cf9e640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 0x7f278406d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f278c039000 tx=0x7f278c034000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:53.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.075+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2788054310 con 0x7f27980b21f0 2023-10-21T16:17:53.159 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:53.153+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:53.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.274+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f279804e160 con 0x7f27980b21f0 2023-10-21T16:17:53.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.805+0000 7f27957fa640 1 -- 172.21.15.53:0/3016900865 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f278805ce60 con 0x7f27980b21f0 2023-10-21T16:17:53.807 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:17:53.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.811+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 msgr2=0x7f278406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:53.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.811+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 0x7f278406d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f278c039000 tx=0x7f278c034000 comp rx=0 tx=0).stop 2023-10-21T16:17:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.811+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 msgr2=0x7f27980c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.811+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f278800b4a0 tx=0x7f278800b960 comp rx=0 tx=0).stop 2023-10-21T16:17:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 shutdown_connections 2023-10-21T16:17:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f278406af00 0x7f278406d3c0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f278c039000 tx=0x7f278c034000 comp rx=0 tx=0).stop 2023-10-21T16:17:53.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27980ca1a0 0x7f2798155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27980b21f0 0x7f27980c4230 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 --2- 172.21.15.53:0/3016900865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27980702b0 0x7f2798143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:53.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.812+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 >> 172.21.15.53:0/3016900865 conn(0x7f279806a5d0 msgr2=0x7f27980cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:53.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.814+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 shutdown_connections 2023-10-21T16:17:53.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:53.815+0000 7f279f202640 1 -- 172.21.15.53:0/3016900865 wait complete. 2023-10-21T16:17:53.840 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:17:54.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.048+0000 7f56511b0640 1 Processor -- start 2023-10-21T16:17:54.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.048+0000 7f56511b0640 1 -- start start 2023-10-21T16:17:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.050+0000 7f56511b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 0x7f564c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.050+0000 7f56511b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c071140 con 0x7f564c0cb450 2023-10-21T16:17:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.050+0000 7f56511b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c06e150 con 0x7f564c160660 2023-10-21T16:17:54.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.050+0000 7f56511b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c0cb8c0 con 0x7f564c070cc0 2023-10-21T16:17:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.050+0000 7f564b59e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f564c0cb450 0x7f564c0b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49064/0 (socket says 172.21.15.53:49064) 2023-10-21T16:17:54.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564b59e640 1 -- 172.21.15.53:0/2439605059 learned_addr learned my addr 172.21.15.53:0/2439605059 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f5643fff640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 0x7f564c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649048759 0 0) 0x7f564c071140 con 0x7f564c0cb450 2023-10-21T16:17:54.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5638003520 con 0x7f564c0cb450 2023-10-21T16:17:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597556163 0 0) 0x7f5638003520 con 0x7f564c0cb450 2023-10-21T16:17:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 msgr2=0x7f564c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 0x7f564c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.051+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 >> v1:172.21.15.124:6789/0 conn(0x7f564c160660 legacy=0x7f564c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c06de70 con 0x7f564c0cb450 2023-10-21T16:17:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f56440019f0 con 0x7f564c0cb450 2023-10-21T16:17:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 >> v1:172.21.15.53:6789/0 conn(0x7f564c0cb450 legacy=0x7f564c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56380039a0 0x7f5638003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f5638004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 0x7f5638005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c0cb8c0 con 0x7f56380047c0 2023-10-21T16:17:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563800ba30 con 0x7f56380039a0 2023-10-21T16:17:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563800bba0 con 0x7f564c070cc0 2023-10-21T16:17:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f5638004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f564ad9d640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56380039a0 0x7f5638003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 -- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 msgr2=0x7f5638005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c070cc0 0x7f5638005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 -- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56380039a0 msgr2=0x7f5638003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56380039a0 0x7f5638003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.052+0000 7f5643fff640 1 -- 172.21.15.53:0/2439605059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638003520 con 0x7f56380047c0 2023-10-21T16:17:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.053+0000 7f564ad9d640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56380039a0 0x7f5638003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:54.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.054+0000 7f5643fff640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f5638004c40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5634009a90 tx=0x7f56380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5cc749c0cde0c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.054+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563402c6b0 con 0x7f56380047c0 2023-10-21T16:17:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.054+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563402c810 con 0x7f56380047c0 2023-10-21T16:17:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.054+0000 7f564a59c640 1 -- 172.21.15.53:0/2439605059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563402cba0 con 0x7f56380047c0 2023-10-21T16:17:54.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 -- 172.21.15.53:0/2439605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 msgr2=0x7f5638004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 --2- 172.21.15.53:0/2439605059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f5638004c40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5634009a90 tx=0x7f56380057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f564b59e640 1 -- 172.21.15.53:0/2439605059 reap_dead start 2023-10-21T16:17:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 -- 172.21.15.53:0/2439605059 shutdown_connections 2023-10-21T16:17:54.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 -- 172.21.15.53:0/2439605059 >> 172.21.15.53:0/2439605059 conn(0x7f564c06a5d0 msgr2=0x7f564c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 -- 172.21.15.53:0/2439605059 shutdown_connections 2023-10-21T16:17:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.055+0000 7f56511b0640 1 -- 172.21.15.53:0/2439605059 wait complete. 2023-10-21T16:17:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.057+0000 7f56511b0640 1 Processor -- start 2023-10-21T16:17:54.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.057+0000 7f56511b0640 1 -- start start 2023-10-21T16:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c15ed20 0x7f564c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c0c1ca0 0x7f564c0ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c14d170 con 0x7f56380047c0 2023-10-21T16:17:54.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c0ab8d0 con 0x7f564c0c1ca0 2023-10-21T16:17:54.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f56511b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c14f500 con 0x7f564c15ed20 2023-10-21T16:17:54.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33118/0 (socket says 172.21.15.53:33118) 2023-10-21T16:17:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 -- 172.21.15.53:0/2888560487 learned_addr learned my addr 172.21.15.53:0/2888560487 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 -- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c15ed20 msgr2=0x7f564c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c15ed20 0x7f564c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564b59e640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c0c1ca0 0x7f564c0ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 -- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c0c1ca0 msgr2=0x7f564c0ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c0c1ca0 0x7f564c0ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.058+0000 7f564ad9d640 1 -- 172.21.15.53:0/2888560487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0abb20 con 0x7f56380047c0 2023-10-21T16:17:54.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f564ad9d640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d710 tx=0x7f563c00dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c005480 con 0x7f56380047c0 2023-10-21T16:17:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c00de20 con 0x7f56380047c0 2023-10-21T16:17:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00b8f0 con 0x7f56380047c0 2023-10-21T16:17:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c0aed90 con 0x7f56380047c0 2023-10-21T16:17:54.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.059+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c0af350 con 0x7f56380047c0 2023-10-21T16:17:54.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.061+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c0af510 con 0x7f56380047c0 2023-10-21T16:17:54.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.062+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f563c00ba50 con 0x7f56380047c0 2023-10-21T16:17:54.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.063+0000 7f56437fe640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 0x7f562c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.063+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f563c0899f0 con 0x7f56380047c0 2023-10-21T16:17:54.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.064+0000 7f5643fff640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 0x7f562c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.065+0000 7f5643fff640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 0x7f562c06d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5634036000 tx=0x7f563402e000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.068+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f563c054810 con 0x7f56380047c0 2023-10-21T16:17:54.118 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:54.115+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:54.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.276+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f564c04e160 con 0x7f56380047c0 2023-10-21T16:17:54.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.281+0000 7f56437fe640 1 -- 172.21.15.53:0/2888560487 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f563c087e60 con 0x7f56380047c0 2023-10-21T16:17:54.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 msgr2=0x7f562c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 0x7f562c06d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5634036000 tx=0x7f563402e000 comp rx=0 tx=0).stop 2023-10-21T16:17:54.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 msgr2=0x7f564c14cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d710 tx=0x7f563c00dbd0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 shutdown_connections 2023-10-21T16:17:54.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f562c06af00 0x7f562c06d3c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c0c1ca0 0x7f564c0ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c15ed20 0x7f564c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 --2- 172.21.15.53:0/2888560487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56380047c0 0x7f564c14cc30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 >> 172.21.15.53:0/2888560487 conn(0x7f564c06a5d0 msgr2=0x7f564c15f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:54.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 shutdown_connections 2023-10-21T16:17:54.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.284+0000 7f56511b0640 1 -- 172.21.15.53:0/2888560487 wait complete. 2023-10-21T16:17:54.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:17:54.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.532+0000 7f5ee2335640 1 Processor -- start 2023-10-21T16:17:54.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.532+0000 7f5ee2335640 1 -- start start 2023-10-21T16:17:54.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee2335640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5edc0b3f40 0x7f5edc0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee2335640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc071690 con 0x7f5edc1597a0 2023-10-21T16:17:54.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee2335640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5edc071380 con 0x7f5edc15d400 2023-10-21T16:17:54.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee2335640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5edc0ca630 con 0x7f5edc0b3f40 2023-10-21T16:17:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee1333640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5edc1597a0 0x7f5edc071b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49076/0 (socket says 172.21.15.53:49076) 2023-10-21T16:17:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee1333640 1 -- 172.21.15.53:0/2790809822 learned_addr learned my addr 172.21.15.53:0/2790809822 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.533+0000 7f5ee0b32640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5edc0b3f40 0x7f5edc0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.534+0000 7f5ee0b32640 1 -- 172.21.15.53:0/2790809822 >> v1:172.21.15.124:6789/0 conn(0x7f5edc15d400 legacy=0x7f5edc0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:54.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.534+0000 7f5ee0b32640 1 -- 172.21.15.53:0/2790809822 >> v1:172.21.15.53:6789/0 conn(0x7f5edc1597a0 legacy=0x7f5edc071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.534+0000 7f5ee0b32640 1 -- 172.21.15.53:0/2790809822 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0ca910 con 0x7f5edc0b3f40 2023-10-21T16:17:54.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.534+0000 7f5ed3fff640 1 -- 172.21.15.53:0/2790809822 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892624406 0 0) 0x7f5edc071690 con 0x7f5edc1597a0 2023-10-21T16:17:54.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.535+0000 7f5ee0b32640 1 --2- 172.21.15.53:0/2790809822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0ca020 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0097a0 tx=0x7f5ecc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d169e555ba6e8b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.535+0000 7f5ed3fff640 1 -- 172.21.15.53:0/2790809822 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc02e070 con 0x7f5edc0b3f40 2023-10-21T16:17:54.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ed3fff640 1 -- 172.21.15.53:0/2790809822 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc0315a0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ed3fff640 1 -- 172.21.15.53:0/2790809822 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc031930 con 0x7f5edc0b3f40 2023-10-21T16:17:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 -- 172.21.15.53:0/2790809822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 msgr2=0x7f5edc0ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 --2- 172.21.15.53:0/2790809822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0ca020 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0097a0 tx=0x7f5ecc02b840 comp rx=0 tx=0).stop 2023-10-21T16:17:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 -- 172.21.15.53:0/2790809822 shutdown_connections 2023-10-21T16:17:54.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 --2- 172.21.15.53:0/2790809822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0ca020 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 -- 172.21.15.53:0/2790809822 >> 172.21.15.53:0/2790809822 conn(0x7f5edc06a430 msgr2=0x7f5edc06c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:54.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 -- 172.21.15.53:0/2790809822 shutdown_connections 2023-10-21T16:17:54.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.536+0000 7f5ee2335640 1 -- 172.21.15.53:0/2790809822 wait complete. 2023-10-21T16:17:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.538+0000 7f5ee2335640 1 Processor -- start 2023-10-21T16:17:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.538+0000 7f5ee2335640 1 -- start start 2023-10-21T16:17:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5edc1597a0 0x7f5edc14ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 0x7f5edc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc14bfc0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0c7ba0 con 0x7f5edc15d400 2023-10-21T16:17:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.539+0000 7f5ee2335640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc0c7e40 con 0x7f5edc1597a0 2023-10-21T16:17:54.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33144/0 (socket says 172.21.15.53:33144) 2023-10-21T16:17:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 -- 172.21.15.53:0/3109848549 learned_addr learned my addr 172.21.15.53:0/3109848549 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1b34640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 0x7f5edc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 -- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5edc1597a0 msgr2=0x7f5edc14ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee0b32640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5edc1597a0 0x7f5edc14ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5edc1597a0 0x7f5edc14ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 -- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 msgr2=0x7f5edc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 0x7f5edc0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1333640 1 -- 172.21.15.53:0/3109848549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc009400 con 0x7f5edc0b3f40 2023-10-21T16:17:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.540+0000 7f5ee1b34640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 0x7f5edc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ee1333640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c7c0 tx=0x7f5ed800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800a640 con 0x7f5edc0b3f40 2023-10-21T16:17:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed800a7c0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800ab70 con 0x7f5edc0b3f40 2023-10-21T16:17:54.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5edc0c8120 con 0x7f5edc0b3f40 2023-10-21T16:17:54.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.541+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5edc0c86e0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.543+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5edc0740e0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.544+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5ed800acd0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.544+0000 7f5ed1ffb640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 0x7f5ec006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.545+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f5ed8088d30 con 0x7f5edc0b3f40 2023-10-21T16:17:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.545+0000 7f5ee0b32640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 0x7f5ec006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.547+0000 7f5ee0b32640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 0x7f5ec006d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc036000 tx=0x7f5ecc008030 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:54.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.549+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ed8053cd0 con 0x7f5edc0b3f40 2023-10-21T16:17:54.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.745+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f5edc0ca370 con 0x7f5edc0b3f40 2023-10-21T16:17:54.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.753+0000 7f5ed1ffb640 1 -- 172.21.15.53:0/3109848549 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5ed8018070 con 0x7f5edc0b3f40 2023-10-21T16:17:54.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 msgr2=0x7f5ec006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 0x7f5ec006d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc036000 tx=0x7f5ecc008030 comp rx=0 tx=0).stop 2023-10-21T16:17:54.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 msgr2=0x7f5edc0a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c7c0 tx=0x7f5ed800cc80 comp rx=0 tx=0).stop 2023-10-21T16:17:54.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 shutdown_connections 2023-10-21T16:17:54.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5ec006af00 0x7f5ec006d3c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5edc15d400 0x7f5edc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5edc1597a0 0x7f5edc14ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 --2- 172.21.15.53:0/3109848549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5edc0b3f40 0x7f5edc0a8660 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 >> 172.21.15.53:0/3109848549 conn(0x7f5edc06a430 msgr2=0x7f5edc071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:54.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 shutdown_connections 2023-10-21T16:17:54.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.756+0000 7f5ee2335640 1 -- 172.21.15.53:0/3109848549 wait complete. 2023-10-21T16:17:54.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:17:54.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.989+0000 7f1901290640 1 Processor -- start 2023-10-21T16:17:54.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.989+0000 7f1901290640 1 -- start start 2023-10-21T16:17:54.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.991+0000 7f1901290640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18fc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.991+0000 7f1901290640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0b1cc0 con 0x7f18fc071070 2023-10-21T16:17:54.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f1901290640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc0b2620 con 0x7f18fc0afbe0 2023-10-21T16:17:54.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f1901290640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0b2060 con 0x7f18fc06dfc0 2023-10-21T16:17:54.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f18fb7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f18fc071070 0x7f18fc0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49078/0 (socket says 172.21.15.53:49078) 2023-10-21T16:17:54.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f18fb7fe640 1 -- 172.21.15.53:0/2421396826 learned_addr learned my addr 172.21.15.53:0/2421396826 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:54.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f18f3fff640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18fc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964500615 0 0) 0x7f18fc0b1cc0 con 0x7f18fc071070 2023-10-21T16:17:54.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.992+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003520 con 0x7f18fc071070 2023-10-21T16:17:54.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926908180 0 0) 0x7f18e8003520 con 0x7f18fc071070 2023-10-21T16:17:54.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 msgr2=0x7f18fc070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:54.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18fc070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:54.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 >> v1:172.21.15.124:6789/0 conn(0x7f18fc0afbe0 legacy=0x7f18fc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0b2340 con 0x7f18fc071070 2023-10-21T16:17:54.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f18f40019f0 con 0x7f18fc071070 2023-10-21T16:17:54.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 >> v1:172.21.15.53:6789/0 conn(0x7f18fc071070 legacy=0x7f18fc0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:54.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0b2060 con 0x7f18e8003f70 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800bac0 con 0x7f18e8004d00 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800bc80 con 0x7f18fc06dfc0 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.993+0000 7f18f3fff640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:54.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18fb7fe640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18faffd640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18f3fff640 1 -- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 msgr2=0x7f18e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:55.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18f3fff640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18e80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18f3fff640 1 -- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e8004d00 msgr2=0x7f18e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:55.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18f3fff640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e8004d00 0x7f18e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18f3fff640 1 -- 172.21.15.53:0/2421396826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8003520 con 0x7f18e8003f70 2023-10-21T16:17:55.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.994+0000 7f18faffd640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc06dfc0 0x7f18e80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:55.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.995+0000 7f18f3fff640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18e80047c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009a90 tx=0x7f18e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d976f3893b0531cd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.995+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e402c5c0 con 0x7f18e8003f70 2023-10-21T16:17:55.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e402c720 con 0x7f18e8003f70 2023-10-21T16:17:55.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f18fa7fc640 1 -- 172.21.15.53:0/2421396826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e402cad0 con 0x7f18e8003f70 2023-10-21T16:17:55.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 -- 172.21.15.53:0/2421396826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 msgr2=0x7f18e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 --2- 172.21.15.53:0/2421396826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18e80047c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009a90 tx=0x7f18e80057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f18fb7fe640 1 -- 172.21.15.53:0/2421396826 reap_dead start 2023-10-21T16:17:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 -- 172.21.15.53:0/2421396826 shutdown_connections 2023-10-21T16:17:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 -- 172.21.15.53:0/2421396826 >> 172.21.15.53:0/2421396826 conn(0x7f18fc06a5d0 msgr2=0x7f18fc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:55.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 -- 172.21.15.53:0/2421396826 shutdown_connections 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.996+0000 7f1901290640 1 -- 172.21.15.53:0/2421396826 wait complete. 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.998+0000 7f1901290640 1 Processor -- start 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.998+0000 7f1901290640 1 -- start start 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc158370 0x7f18fc1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 0x7f18fc0c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0c4c70 con 0x7f18e8003f70 2023-10-21T16:17:55.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc14f520 con 0x7f18fc0c42d0 2023-10-21T16:17:55.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f1901290640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc158e10 con 0x7f18fc158370 2023-10-21T16:17:55.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:54.999+0000 7f18faffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33156/0 (socket says 172.21.15.53:33156) 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 -- 172.21.15.53:0/3302144757 learned_addr learned my addr 172.21.15.53:0/3302144757 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18fb7fe640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 0x7f18fc0c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18f3fff640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc158370 0x7f18fc1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 -- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc158370 msgr2=0x7f18fc1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc158370 0x7f18fc1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 -- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 msgr2=0x7f18fc0c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 0x7f18fc0c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 -- 172.21.15.53:0/3302144757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc1590f0 con 0x7f18e8003f70 2023-10-21T16:17:55.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18fb7fe640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 0x7f18fc0c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:55.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.000+0000 7f18faffd640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00d770 tx=0x7f18ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.001+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec005270 con 0x7f18e8003f70 2023-10-21T16:17:55.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.001+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec0053d0 con 0x7f18e8003f70 2023-10-21T16:17:55.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.001+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00b8f0 con 0x7f18e8003f70 2023-10-21T16:17:55.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.001+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc1593d0 con 0x7f18e8003f70 2023-10-21T16:17:55.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.001+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc1565f0 con 0x7f18e8003f70 2023-10-21T16:17:55.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.002+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc06ad20 con 0x7f18e8003f70 2023-10-21T16:17:55.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.003+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f18ec00ba50 con 0x7f18e8003f70 2023-10-21T16:17:55.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.004+0000 7f18f37fe640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 0x7f18dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.004+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f18ec0898b0 con 0x7f18e8003f70 2023-10-21T16:17:55.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.005+0000 7f18f3fff640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 0x7f18dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.006+0000 7f18f3fff640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 0x7f18dc06d3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f18e4036000 tx=0x7f18e4030000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.008+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18ec054640 con 0x7f18e8003f70 2023-10-21T16:17:55.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:55.022+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:55.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.211+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f18fc04e160 con 0x7f18e8003f70 2023-10-21T16:17:55.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.219+0000 7f18f37fe640 1 -- 172.21.15.53:0/3302144757 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f18ec019310 con 0x7f18e8003f70 2023-10-21T16:17:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 msgr2=0x7f18dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 0x7f18dc06d3f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f18e4036000 tx=0x7f18e4030000 comp rx=0 tx=0).stop 2023-10-21T16:17:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 msgr2=0x7f18fc1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00d770 tx=0x7f18ec00dc30 comp rx=0 tx=0).stop 2023-10-21T16:17:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 shutdown_connections 2023-10-21T16:17:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f18dc06af30 0x7f18dc06d3f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18fc0c42d0 0x7f18fc0c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18fc158370 0x7f18fc1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.221+0000 7f1901290640 1 --2- 172.21.15.53:0/3302144757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18e8003f70 0x7f18fc1560b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.222+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 >> 172.21.15.53:0/3302144757 conn(0x7f18fc06a5d0 msgr2=0x7f18fc0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.222+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 shutdown_connections 2023-10-21T16:17:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.222+0000 7f1901290640 1 -- 172.21.15.53:0/3302144757 wait complete. 2023-10-21T16:17:55.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-21T16:17:55.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.474+0000 7f3f0c851640 1 Processor -- start 2023-10-21T16:17:55.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.474+0000 7f3f0c851640 1 -- start start 2023-10-21T16:17:55.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.475+0000 7f3f0c851640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3f080ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3f0c851640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0815b2b0 con 0x7f3f080c90b0 2023-10-21T16:17:55.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3f0c851640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0815b590 con 0x7f3f080cb580 2023-10-21T16:17:55.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3f0c851640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0815bf30 con 0x7f3f0815ab40 2023-10-21T16:17:55.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3efffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3f080c90b0 0x7f3f080cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49092/0 (socket says 172.21.15.53:49092) 2023-10-21T16:17:55.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3efffff640 1 -- 172.21.15.53:0/2810668034 learned_addr learned my addr 172.21.15.53:0/2810668034 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:55.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3f0659c640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3f080ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.476+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485554189 0 0) 0x7f3f0815b2b0 con 0x7f3f080c90b0 2023-10-21T16:17:55.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4003520 con 0x7f3f080c90b0 2023-10-21T16:17:55.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430599941 0 0) 0x7f3ef4003520 con 0x7f3f080c90b0 2023-10-21T16:17:55.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 msgr2=0x7f3f080ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:55.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3f080ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 >> v1:172.21.15.124:6789/0 conn(0x7f3f080cb580 legacy=0x7f3f0815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0815b870 con 0x7f3f080c90b0 2023-10-21T16:17:55.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3ef0001670 con 0x7f3f080c90b0 2023-10-21T16:17:55.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 >> v1:172.21.15.53:6789/0 conn(0x7f3f080c90b0 legacy=0x7f3f080cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3ef40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef4003f70 0x7f3ef40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4004d00 0x7f3ef4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0815bf30 con 0x7f3ef4003f70 2023-10-21T16:17:55.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400bac0 con 0x7f3ef4004d00 2023-10-21T16:17:55.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.477+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400bc80 con 0x7f3f0815ab40 2023-10-21T16:17:55.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef4003f70 0x7f3ef40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3f06d9d640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4004d00 0x7f3ef4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 -- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 msgr2=0x7f3ef40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3f0659c640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3ef40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0815ab40 0x7f3ef40039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 -- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4004d00 msgr2=0x7f3ef4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4004d00 0x7f3ef4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.478+0000 7f3efffff640 1 -- 172.21.15.53:0/2810668034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4003520 con 0x7f3ef4003f70 2023-10-21T16:17:55.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.479+0000 7f3efffff640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef4003f70 0x7f3ef40047c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c0e0 tx=0x7f3ef40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db71948a8debec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0005ad0 con 0x7f3ef4003f70 2023-10-21T16:17:55.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0005c50 con 0x7f3ef4003f70 2023-10-21T16:17:55.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f05d9b640 1 -- 172.21.15.53:0/2810668034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0007070 con 0x7f3ef4003f70 2023-10-21T16:17:55.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f0c851640 1 -- 172.21.15.53:0/2810668034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef4003f70 msgr2=0x7f3ef40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f0c851640 1 --2- 172.21.15.53:0/2810668034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef4003f70 0x7f3ef40047c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c0e0 tx=0x7f3ef40057f0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.480+0000 7f3f06d9d640 1 -- 172.21.15.53:0/2810668034 reap_dead start 2023-10-21T16:17:55.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.481+0000 7f3f0c851640 1 -- 172.21.15.53:0/2810668034 shutdown_connections 2023-10-21T16:17:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.481+0000 7f3f0c851640 1 -- 172.21.15.53:0/2810668034 >> 172.21.15.53:0/2810668034 conn(0x7f3f0806a380 msgr2=0x7f3f080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.481+0000 7f3f0c851640 1 -- 172.21.15.53:0/2810668034 shutdown_connections 2023-10-21T16:17:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.481+0000 7f3f0c851640 1 -- 172.21.15.53:0/2810668034 wait complete. 2023-10-21T16:17:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.482+0000 7f3f0c851640 1 Processor -- start 2023-10-21T16:17:55.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.483+0000 7f3f0c851640 1 -- start start 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4003f70 0x7f3f08147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0806d5d0 0x7f3f0814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080b3e20 con 0x7f3f080b4240 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806da50 con 0x7f3ef4003f70 2023-10-21T16:17:55.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0c851640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080b2010 con 0x7f3f0806d5d0 2023-10-21T16:17:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33174/0 (socket says 172.21.15.53:33174) 2023-10-21T16:17:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 -- 172.21.15.53:0/3989589507 learned_addr learned my addr 172.21.15.53:0/3989589507 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 -- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0806d5d0 msgr2=0x7f3f0814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0806d5d0 0x7f3f0814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3efffff640 1 -- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4003f70 msgr2=0x7f3f08147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.484+0000 7f3f0659c640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4003f70 0x7f3f08147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efffff640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4003f70 0x7f3f08147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efffff640 1 -- 172.21.15.53:0/3989589507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080b46c0 con 0x7f3f080b4240 2023-10-21T16:17:55.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efffff640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c140 tx=0x7f3ef0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef003c040 con 0x7f3f080b4240 2023-10-21T16:17:55.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0007580 con 0x7f3f080b4240 2023-10-21T16:17:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef002c740 con 0x7f3f080b4240 2023-10-21T16:17:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0814b0b0 con 0x7f3f080b4240 2023-10-21T16:17:55.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.485+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f080b3a40 con 0x7f3f080b4240 2023-10-21T16:17:55.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.487+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f080b3c00 con 0x7f3f080b4240 2023-10-21T16:17:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.488+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3ef0032070 con 0x7f3f080b4240 2023-10-21T16:17:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.489+0000 7f3efeffd640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 0x7f3ee806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:55.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.489+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f3ef00aac00 con 0x7f3f080b4240 2023-10-21T16:17:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.489+0000 7f3f0659c640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 0x7f3ee806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:55.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.490+0000 7f3f0659c640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 0x7f3ee806d3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009910 tx=0x7f3ef8008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:55.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.493+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ef0075be0 con 0x7f3f080b4240 2023-10-21T16:17:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.766+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f3f08067490 con 0x7f3f080b4240 2023-10-21T16:17:55.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.809+0000 7f3efeffd640 1 -- 172.21.15.53:0/3989589507 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v601) v1 ==== 156+0+0 (secure 0 0 0) 0x7f3ef0046020 con 0x7f3f080b4240 2023-10-21T16:17:55.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.812+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 msgr2=0x7f3ee806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.812+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 0x7f3ee806d3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009910 tx=0x7f3ef8008040 comp rx=0 tx=0).stop 2023-10-21T16:17:55.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.812+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 msgr2=0x7f3f0806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:55.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.812+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c140 tx=0x7f3ef0001930 comp rx=0 tx=0).stop 2023-10-21T16:17:55.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 shutdown_connections 2023-10-21T16:17:55.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f3ee806af00 0x7f3ee806d3c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0806d5d0 0x7f3f0814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f080b4240 0x7f3f0806d090 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 --2- 172.21.15.53:0/3989589507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ef4003f70 0x7f3f08147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:55.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 >> 172.21.15.53:0/3989589507 conn(0x7f3f0806a380 msgr2=0x7f3f080af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:55.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 shutdown_connections 2023-10-21T16:17:55.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:55.813+0000 7f3f0c851640 1 -- 172.21.15.53:0/3989589507 wait complete. 2023-10-21T16:17:55.835 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-21T16:17:55.835 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-21T16:17:55.836 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:17:55.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:17:55.955 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:55.952+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:56.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.066+0000 7fa4de72c640 1 Processor -- start 2023-10-21T16:17:56.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.066+0000 7fa4de72c640 1 -- start start 2023-10-21T16:17:56.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4de72c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa4d815cd90 0x7fa4d80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4de72c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4d8160940 0x7fa4d8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4de72c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d80c48f0 con 0x7fa4d8070d50 2023-10-21T16:17:56.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4de72c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d80c4d80 con 0x7fa4d8160940 2023-10-21T16:17:56.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4de72c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d80c3ce0 con 0x7fa4d815cd90 2023-10-21T16:17:56.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4d77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa4d8160940 0x7fa4d8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4dccc9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa4d8070d50 0x7fa4d80cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49094/0 (socket says 172.21.15.53:49094) 2023-10-21T16:17:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.068+0000 7fa4dccc9640 1 -- 172.21.15.53:0/3795126838 learned_addr learned my addr 172.21.15.53:0/3795126838 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:56.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.069+0000 7fa4d7fff640 1 --2- 172.21.15.53:0/3795126838 >> v2:172.21.15.124:3300/0 conn(0x7fa4d815cd90 0x7fa4d80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.069+0000 7fa4d77fe640 1 -- 172.21.15.53:0/3795126838 >> v2:172.21.15.124:3300/0 conn(0x7fa4d815cd90 msgr2=0x7fa4d80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:56.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.069+0000 7fa4d77fe640 1 --2- 172.21.15.53:0/3795126838 >> v2:172.21.15.124:3300/0 conn(0x7fa4d815cd90 0x7fa4d80702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.069+0000 7fa4d77fe640 1 -- 172.21.15.53:0/3795126838 >> v1:172.21.15.53:6789/0 conn(0x7fa4d8070d50 legacy=0x7fa4d80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.069+0000 7fa4d77fe640 1 -- 172.21.15.53:0/3795126838 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80c3fc0 con 0x7fa4d8160940 2023-10-21T16:17:56.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.070+0000 7fa4d77fe640 1 --2- 172.21.15.53:0/3795126838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8160940 0x7fa4d8070810 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa4d806e440 tx=0x7fa4c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805635813650951 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.070+0000 7fa4d6ffd640 1 -- 172.21.15.53:0/3795126838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0031830 con 0x7fa4d8160940 2023-10-21T16:17:56.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.070+0000 7fa4d6ffd640 1 -- 172.21.15.53:0/3795126838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0031990 con 0x7fa4d8160940 2023-10-21T16:17:56.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.070+0000 7fa4d6ffd640 1 -- 172.21.15.53:0/3795126838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0031d20 con 0x7fa4d8160940 2023-10-21T16:17:56.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 -- 172.21.15.53:0/3795126838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8160940 msgr2=0x7fa4d8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 --2- 172.21.15.53:0/3795126838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8160940 0x7fa4d8070810 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa4d806e440 tx=0x7fa4c002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 -- 172.21.15.53:0/3795126838 shutdown_connections 2023-10-21T16:17:56.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 --2- 172.21.15.53:0/3795126838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8160940 0x7fa4d8070810 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 --2- 172.21.15.53:0/3795126838 >> v2:172.21.15.124:3300/0 conn(0x7fa4d815cd90 0x7fa4d80702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 -- 172.21.15.53:0/3795126838 >> 172.21.15.53:0/3795126838 conn(0x7fa4d806a380 msgr2=0x7fa4d80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 -- 172.21.15.53:0/3795126838 shutdown_connections 2023-10-21T16:17:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.071+0000 7fa4de72c640 1 -- 172.21.15.53:0/3795126838 wait complete. 2023-10-21T16:17:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.073+0000 7fa4de72c640 1 Processor -- start 2023-10-21T16:17:56.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.073+0000 7fa4de72c640 1 -- start start 2023-10-21T16:17:56.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d815cd90 0x7fa4d814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 0x7fa4d80b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d814a670 con 0x7fa4d8070d50 2023-10-21T16:17:56.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80ca290 con 0x7fa4d8160940 2023-10-21T16:17:56.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4de72c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80ca570 con 0x7fa4d815cd90 2023-10-21T16:17:56.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.074+0000 7fa4d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33188/0 (socket says 172.21.15.53:33188) 2023-10-21T16:17:56.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 -- 172.21.15.53:0/2524552547 learned_addr learned my addr 172.21.15.53:0/2524552547 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:56.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 -- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d815cd90 msgr2=0x7fa4d814a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4dccc9640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 0x7fa4d80b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d77fe640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d815cd90 0x7fa4d814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d815cd90 0x7fa4d814a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 -- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 msgr2=0x7fa4d80b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 0x7fa4d80b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 -- 172.21.15.53:0/2524552547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009400 con 0x7fa4d8070d50 2023-10-21T16:17:56.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4dccc9640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 0x7fa4d80b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:56.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.075+0000 7fa4d7fff640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8007ef0 tx=0x7fa4c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.076+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8014620 con 0x7fa4d8070d50 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.076+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8014780 con 0x7fa4d8070d50 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.076+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8014b30 con 0x7fa4d8070d50 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.076+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d80ca850 con 0x7fa4d8070d50 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.076+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d80cae10 con 0x7fa4d8070d50 2023-10-21T16:17:56.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.077+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d80cb000 con 0x7fa4d8070d50 2023-10-21T16:17:56.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.078+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa4c8014c90 con 0x7fa4d8070d50 2023-10-21T16:17:56.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.079+0000 7fa4d4ff9640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.079+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa4c808a2e0 con 0x7fa4d8070d50 2023-10-21T16:17:56.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.080+0000 7fa4d77fe640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.081+0000 7fa4d77fe640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 0x7fa4b806d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0036000 tx=0x7fa4c003c040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.083+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4c8055180 con 0x7fa4d8070d50 2023-10-21T16:17:56.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.282+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4d8067490 con 0x7fa4d8070d50 2023-10-21T16:17:56.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.285+0000 7fa4d4ff9640 1 -- 172.21.15.53:0/2524552547 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+14268 (secure 0 0 0) 0x7fa4c80587a0 con 0x7fa4d8070d50 2023-10-21T16:17:56.286 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:56.287 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:17:53.802551+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/2020660539":"2023-10-22T16:17:50.783070+0000","172.21.15.53:6833/2020660539":"2023-10-22T16:17:50.783070+0000","172.21.15.124:0/4222569834":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/3415687515":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6832/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/812907048":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/4157077207":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1100506891":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/2480922944":"2023-10-22T16:17:46.645364+0000","172.21.15.124:0/1810216649":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3437816425":"2023-10-22T16:17:46.645364+0000","172.21.15.124:6833/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:6832/1636058776":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/2018576779":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3111202668":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/356555339":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3273170228":"2023-10-22T16:17:40.466814+0000","172.21.15.124:0/3109992220":"2023-10-22T16:17:40.466814+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:17:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.287+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 msgr2=0x7fa4b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.287+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 0x7fa4b806d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0036000 tx=0x7fa4c003c040 comp rx=0 tx=0).stop 2023-10-21T16:17:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.287+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 msgr2=0x7fa4d80c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.287+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8007ef0 tx=0x7fa4c800e4b0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 shutdown_connections 2023-10-21T16:17:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa4b806af00 0x7fa4b806d3c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d8160940 0x7fa4d80b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d815cd90 0x7fa4d814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 --2- 172.21.15.53:0/2524552547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d8070d50 0x7fa4d80c9ca0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 >> 172.21.15.53:0/2524552547 conn(0x7fa4d806a380 msgr2=0x7fa4d80af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 shutdown_connections 2023-10-21T16:17:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.288+0000 7fa4de72c640 1 -- 172.21.15.53:0/2524552547 wait complete. 2023-10-21T16:17:56.310 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:17:56.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:17:56.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.525+0000 7f5a4ee02640 1 Processor -- start 2023-10-21T16:17:56.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.525+0000 7f5a4ee02640 1 -- start start 2023-10-21T16:17:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4ee02640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5a480cb880 0x7f5a480b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4ee02640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a480b4070 0x7f5a48159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4ee02640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4815d440 con 0x7f5a481432f0 2023-10-21T16:17:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4ee02640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a481436d0 con 0x7f5a480b4070 2023-10-21T16:17:56.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4ee02640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4815d690 con 0x7f5a480cb880 2023-10-21T16:17:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4d39f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a480b4070 0x7f5a48159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4d39f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5a480b4070 0x7f5a48159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33194/0 (socket says 172.21.15.53:33194) 2023-10-21T16:17:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.527+0000 7f5a4d39f640 1 -- 172.21.15.53:0/2791750238 learned_addr learned my addr 172.21.15.53:0/2791750238 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:56.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.528+0000 7f5a47fff640 1 --2- 172.21.15.53:0/2791750238 >> v2:172.21.15.124:3300/0 conn(0x7f5a480cb880 0x7f5a480b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.528+0000 7f5a4d39f640 1 -- 172.21.15.53:0/2791750238 >> v2:172.21.15.124:3300/0 conn(0x7f5a480cb880 msgr2=0x7f5a480b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:56.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.528+0000 7f5a4d39f640 1 --2- 172.21.15.53:0/2791750238 >> v2:172.21.15.124:3300/0 conn(0x7f5a480cb880 0x7f5a480b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.528+0000 7f5a4d39f640 1 -- 172.21.15.53:0/2791750238 >> v1:172.21.15.124:6789/0 conn(0x7f5a481432f0 legacy=0x7f5a480cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:56.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.528+0000 7f5a4d39f640 1 -- 172.21.15.53:0/2791750238 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4815d970 con 0x7f5a480b4070 2023-10-21T16:17:56.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.529+0000 7f5a4d39f640 1 --2- 172.21.15.53:0/2791750238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480b4070 0x7f5a48159e50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5a4806ab90 tx=0x7f5a4002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692ccc55538443b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.529+0000 7f5a477fe640 1 -- 172.21.15.53:0/2791750238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4002c9f0 con 0x7f5a480b4070 2023-10-21T16:17:56.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a477fe640 1 -- 172.21.15.53:0/2791750238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4002cb50 con 0x7f5a480b4070 2023-10-21T16:17:56.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a477fe640 1 -- 172.21.15.53:0/2791750238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400343f0 con 0x7f5a480b4070 2023-10-21T16:17:56.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2791750238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480b4070 msgr2=0x7f5a48159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2791750238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480b4070 0x7f5a48159e50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5a4806ab90 tx=0x7f5a4002be50 comp rx=0 tx=0).stop 2023-10-21T16:17:56.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2791750238 shutdown_connections 2023-10-21T16:17:56.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2791750238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480b4070 0x7f5a48159e50 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2791750238 >> v2:172.21.15.124:3300/0 conn(0x7f5a480cb880 0x7f5a480b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2791750238 >> 172.21.15.53:0/2791750238 conn(0x7f5a48069d50 msgr2=0x7f5a4806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:56.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2791750238 shutdown_connections 2023-10-21T16:17:56.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.530+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2791750238 wait complete. 2023-10-21T16:17:56.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.532+0000 7f5a4ee02640 1 Processor -- start 2023-10-21T16:17:56.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.532+0000 7f5a4ee02640 1 -- start start 2023-10-21T16:17:56.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480b4070 0x7f5a4813f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a481432f0 0x7f5a4806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481f17a0 con 0x7f5a480cb880 2023-10-21T16:17:56.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4813dce0 con 0x7f5a481432f0 2023-10-21T16:17:56.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.533+0000 7f5a4ee02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4813dfc0 con 0x7f5a480b4070 2023-10-21T16:17:56.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33210/0 (socket says 172.21.15.53:33210) 2023-10-21T16:17:56.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 -- 172.21.15.53:0/2254061998 learned_addr learned my addr 172.21.15.53:0/2254061998 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:56.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a4cb9e640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480b4070 0x7f5a4813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a4d39f640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a481432f0 0x7f5a4806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 -- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480b4070 msgr2=0x7f5a4813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480b4070 0x7f5a4813f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 -- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a481432f0 msgr2=0x7f5a4806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a481432f0 0x7f5a4806cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 -- 172.21.15.53:0/2254061998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009b30 con 0x7f5a480cb880 2023-10-21T16:17:56.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.534+0000 7f5a47fff640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d800 tx=0x7f5a3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.535+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400b6c0 con 0x7f5a480cb880 2023-10-21T16:17:56.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.535+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3400b820 con 0x7f5a480cb880 2023-10-21T16:17:56.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.535+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400bbb0 con 0x7f5a480cb880 2023-10-21T16:17:56.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.535+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4813e2a0 con 0x7f5a480cb880 2023-10-21T16:17:56.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.535+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4813f740 con 0x7f5a480cb880 2023-10-21T16:17:56.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.536+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4804e160 con 0x7f5a480cb880 2023-10-21T16:17:56.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.538+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5a3400bd10 con 0x7f5a480cb880 2023-10-21T16:17:56.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.539+0000 7f5a457fa640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.539+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f5a34089c60 con 0x7f5a480cb880 2023-10-21T16:17:56.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.540+0000 7f5a4cb9e640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:56.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.541+0000 7f5a4cb9e640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006fd0 tx=0x7f5a3c008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:56.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.546+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a34054c90 con 0x7f5a480cb880 2023-10-21T16:17:56.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.747+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a4806d660 con 0x7f5a480cb880 2023-10-21T16:17:56.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.748+0000 7f5a457fa640 1 -- 172.21.15.53:0/2254061998 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a340582b0 con 0x7f5a480cb880 2023-10-21T16:17:56.750 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:17:56.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 msgr2=0x7f5a2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006fd0 tx=0x7f5a3c008040 comp rx=0 tx=0).stop 2023-10-21T16:17:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 msgr2=0x7f5a4806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d800 tx=0x7f5a3400dcc0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 shutdown_connections 2023-10-21T16:17:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a481432f0 0x7f5a4806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a480cb880 0x7f5a4806c6b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.751+0000 7f5a4ee02640 1 --2- 172.21.15.53:0/2254061998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480b4070 0x7f5a4813f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.752+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 >> 172.21.15.53:0/2254061998 conn(0x7f5a48069d50 msgr2=0x7f5a480c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.752+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 shutdown_connections 2023-10-21T16:17:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.752+0000 7f5a4ee02640 1 -- 172.21.15.53:0/2254061998 wait complete. 2023-10-21T16:17:56.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-21T16:17:56.885 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:56.882+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:56.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.994+0000 7ffadb101640 1 Processor -- start 2023-10-21T16:17:56.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.994+0000 7ffadb101640 1 -- start start 2023-10-21T16:17:56.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffadb101640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffad40ad790 0x7ffad40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffadb101640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad406dee0 con 0x7ffad4159ac0 2023-10-21T16:17:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffadb101640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad415d850 con 0x7ffad415e330 2023-10-21T16:17:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffadb101640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad406dcf0 con 0x7ffad40ad790 2023-10-21T16:17:56.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffad8e9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ffad4159ac0 0x7ffad406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49100/0 (socket says 172.21.15.53:49100) 2023-10-21T16:17:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffad8e9d640 1 -- 172.21.15.53:0/1169551984 learned_addr learned my addr 172.21.15.53:0/1169551984 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:56.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.996+0000 7ffad3fff640 1 --2- 172.21.15.53:0/1169551984 >> v2:172.21.15.53:3300/0 conn(0x7ffad40ad790 0x7ffad40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.997+0000 7ffad37fe640 1 -- 172.21.15.53:0/1169551984 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333720663 0 0) 0x7ffad406dee0 con 0x7ffad4159ac0 2023-10-21T16:17:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.997+0000 7ffad3fff640 1 -- 172.21.15.53:0/1169551984 >> v1:172.21.15.124:6789/0 conn(0x7ffad415e330 legacy=0x7ffad40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.997+0000 7ffad3fff640 1 -- 172.21.15.53:0/1169551984 >> v1:172.21.15.53:6789/0 conn(0x7ffad4159ac0 legacy=0x7ffad406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.997+0000 7ffad3fff640 1 -- 172.21.15.53:0/1169551984 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad415db30 con 0x7ffad40ad790 2023-10-21T16:17:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.998+0000 7ffad3fff640 1 --2- 172.21.15.53:0/1169551984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad40ad790 0x7ffad40702d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097a0 tx=0x7ffac002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ce3ef995db3f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.998+0000 7ffad37fe640 1 -- 172.21.15.53:0/1169551984 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0031830 con 0x7ffad40ad790 2023-10-21T16:17:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.998+0000 7ffad37fe640 1 -- 172.21.15.53:0/1169551984 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0031990 con 0x7ffad40ad790 2023-10-21T16:17:57.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.998+0000 7ffad37fe640 1 -- 172.21.15.53:0/1169551984 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0031d20 con 0x7ffad40ad790 2023-10-21T16:17:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.999+0000 7ffadb101640 1 -- 172.21.15.53:0/1169551984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad40ad790 msgr2=0x7ffad40702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.999+0000 7ffadb101640 1 --2- 172.21.15.53:0/1169551984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad40ad790 0x7ffad40702d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ffac00097a0 tx=0x7ffac002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.999+0000 7ffadb101640 1 -- 172.21.15.53:0/1169551984 shutdown_connections 2023-10-21T16:17:57.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.999+0000 7ffadb101640 1 --2- 172.21.15.53:0/1169551984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad40ad790 0x7ffad40702d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:56.999+0000 7ffadb101640 1 -- 172.21.15.53:0/1169551984 >> 172.21.15.53:0/1169551984 conn(0x7ffad406a380 msgr2=0x7ffad40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:57.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.000+0000 7ffadb101640 1 -- 172.21.15.53:0/1169551984 shutdown_connections 2023-10-21T16:17:57.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.000+0000 7ffadb101640 1 -- 172.21.15.53:0/1169551984 wait complete. 2023-10-21T16:17:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.001+0000 7ffadb101640 1 Processor -- start 2023-10-21T16:17:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.002+0000 7ffadb101640 1 -- start start 2023-10-21T16:17:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.002+0000 7ffadb101640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad40ad790 0x7ffad4144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:57.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffadb101640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad4159ac0 0x7ffad41565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffadb101640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffadb101640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad415c2e0 con 0x7ffad415e330 2023-10-21T16:17:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffadb101640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad415b660 con 0x7ffad40ad790 2023-10-21T16:17:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffadb101640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad415b940 con 0x7ffad4159ac0 2023-10-21T16:17:57.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffad969e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffad969e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33234/0 (socket says 172.21.15.53:33234) 2023-10-21T16:17:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffad969e640 1 -- 172.21.15.53:0/292232985 learned_addr learned my addr 172.21.15.53:0/292232985 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffad8e9d640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad40ad790 0x7ffad4144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:57.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.003+0000 7ffad3fff640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad4159ac0 0x7ffad41565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 -- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad4159ac0 msgr2=0x7ffad41565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad4159ac0 0x7ffad41565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 -- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad40ad790 msgr2=0x7ffad4144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad40ad790 0x7ffad4144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 -- 172.21.15.53:0/292232985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0009400 con 0x7ffad415e330 2023-10-21T16:17:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad969e640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b970 tx=0x7ffacc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00cb40 con 0x7ffad415e330 2023-10-21T16:17:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc00cca0 con 0x7ffad415e330 2023-10-21T16:17:57.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc014680 con 0x7ffad415e330 2023-10-21T16:17:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.004+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad415bc20 con 0x7ffad415e330 2023-10-21T16:17:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.005+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4145500 con 0x7ffad415e330 2023-10-21T16:17:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.006+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad415c420 con 0x7ffad415e330 2023-10-21T16:17:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.007+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ffacc01d3b0 con 0x7ffad415e330 2023-10-21T16:17:57.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.007+0000 7ffad17fa640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 0x7ffab806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.008+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7ffacc0895a0 con 0x7ffad415e330 2023-10-21T16:17:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.008+0000 7ffad8e9d640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 0x7ffab806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:57.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.009+0000 7ffad8e9d640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 0x7ffab806d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffac8000f60 tx=0x7ffac8007b50 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:57.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.012+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffad415c420 con 0x7ffad415e330 2023-10-21T16:17:57.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.286+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7ffad415c420 con 0x7ffad415e330 2023-10-21T16:17:57.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.815+0000 7ffad17fa640 1 -- 172.21.15.53:0/292232985 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v603) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffad415c420 con 0x7ffad415e330 2023-10-21T16:17:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.818+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 msgr2=0x7ffab806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.818+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 0x7ffab806d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffac8000f60 tx=0x7ffac8007b50 comp rx=0 tx=0).stop 2023-10-21T16:17:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 msgr2=0x7ffad41f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b970 tx=0x7ffacc00be30 comp rx=0 tx=0).stop 2023-10-21T16:17:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 shutdown_connections 2023-10-21T16:17:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ffab806af00 0x7ffab806d3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad415e330 0x7ffad41f2d30 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad4159ac0 0x7ffad41565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 --2- 172.21.15.53:0/292232985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad40ad790 0x7ffad4144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 >> 172.21.15.53:0/292232985 conn(0x7ffad406a380 msgr2=0x7ffad40ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 shutdown_connections 2023-10-21T16:17:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:57.819+0000 7ffadb101640 1 -- 172.21.15.53:0/292232985 wait complete. 2023-10-21T16:17:57.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:17:57.851 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:57.848+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:58.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.066+0000 7fa082301640 1 Processor -- start 2023-10-21T16:17:58.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.066+0000 7fa082301640 1 -- start start 2023-10-21T16:17:58.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa082301640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa07c15cd80 0x7fa07c0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa082301640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa07c0b6730 0x7fa07c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa082301640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c15d160 con 0x7fa07c0cdda0 2023-10-21T16:17:58.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa082301640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0b7b90 con 0x7fa07c0b6730 2023-10-21T16:17:58.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa082301640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0ce1c0 con 0x7fa07c15cd80 2023-10-21T16:17:58.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa07b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa07c0b6730 0x7fa07c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa08089e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa07c0cdda0 0x7fa07c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49112/0 (socket says 172.21.15.53:49112) 2023-10-21T16:17:58.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.068+0000 7fa08089e640 1 -- 172.21.15.53:0/2480115409 learned_addr learned my addr 172.21.15.53:0/2480115409 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:58.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07bfff640 1 --2- 172.21.15.53:0/2480115409 >> v2:172.21.15.124:3300/0 conn(0x7fa07c15cd80 0x7fa07c0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07b7fe640 1 -- 172.21.15.53:0/2480115409 >> v2:172.21.15.124:3300/0 conn(0x7fa07c15cd80 msgr2=0x7fa07c0b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/2480115409 >> v2:172.21.15.124:3300/0 conn(0x7fa07c15cd80 0x7fa07c0b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07b7fe640 1 -- 172.21.15.53:0/2480115409 >> v1:172.21.15.53:6789/0 conn(0x7fa07c0cdda0 legacy=0x7fa07c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07b7fe640 1 -- 172.21.15.53:0/2480115409 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0b8020 con 0x7fa07c0b6730 2023-10-21T16:17:58.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.069+0000 7fa07affd640 1 -- 172.21.15.53:0/2480115409 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575363108 0 0) 0x7fa07c15d160 con 0x7fa07c0cdda0 2023-10-21T16:17:58.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.070+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/2480115409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0b6730 0x7fa07c0cd860 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0b5710 tx=0x7fa06402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de6e3c24b4ccb3c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:58.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa07affd640 1 -- 172.21.15.53:0/2480115409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064031830 con 0x7fa07c0b6730 2023-10-21T16:17:58.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa07affd640 1 -- 172.21.15.53:0/2480115409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa064031990 con 0x7fa07c0b6730 2023-10-21T16:17:58.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa07affd640 1 -- 172.21.15.53:0/2480115409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064031d20 con 0x7fa07c0b6730 2023-10-21T16:17:58.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 -- 172.21.15.53:0/2480115409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0b6730 msgr2=0x7fa07c0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 --2- 172.21.15.53:0/2480115409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0b6730 0x7fa07c0cd860 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0b5710 tx=0x7fa06402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 -- 172.21.15.53:0/2480115409 shutdown_connections 2023-10-21T16:17:58.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 --2- 172.21.15.53:0/2480115409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c0b6730 0x7fa07c0cd860 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 --2- 172.21.15.53:0/2480115409 >> v2:172.21.15.124:3300/0 conn(0x7fa07c15cd80 0x7fa07c0b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.071+0000 7fa082301640 1 -- 172.21.15.53:0/2480115409 >> 172.21.15.53:0/2480115409 conn(0x7fa07c06b5d0 msgr2=0x7fa07c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:58.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.072+0000 7fa082301640 1 -- 172.21.15.53:0/2480115409 shutdown_connections 2023-10-21T16:17:58.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.072+0000 7fa082301640 1 -- 172.21.15.53:0/2480115409 wait complete. 2023-10-21T16:17:58.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.073+0000 7fa082301640 1 Processor -- start 2023-10-21T16:17:58.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.074+0000 7fa082301640 1 -- start start 2023-10-21T16:17:58.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07c0b6730 0x7fa07c1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 0x7fa07c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0cb900 con 0x7fa07c15cd80 2023-10-21T16:17:58.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0cbbe0 con 0x7fa07c0cdda0 2023-10-21T16:17:58.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa082301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0cbec0 con 0x7fa07c0b6730 2023-10-21T16:17:58.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa08089e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa08089e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33264/0 (socket says 172.21.15.53:33264) 2023-10-21T16:17:58.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa08089e640 1 -- 172.21.15.53:0/3074113492 learned_addr learned my addr 172.21.15.53:0/3074113492 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:58.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa07bfff640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07c0b6730 0x7fa07c1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.075+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 0x7fa07c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 -- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07c0b6730 msgr2=0x7fa07c1f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07c0b6730 0x7fa07c1f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 -- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 msgr2=0x7fa07c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 0x7fa07c1f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 -- 172.21.15.53:0/3074113492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009400 con 0x7fa07c15cd80 2023-10-21T16:17:58.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa07b7fe640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 0x7fa07c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:58.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa08089e640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c450 tx=0x7fa07400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:58.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740074e0 con 0x7fa07c15cd80 2023-10-21T16:17:58.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074007660 con 0x7fa07c15cd80 2023-10-21T16:17:58.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074004a20 con 0x7fa07c15cd80 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c0cc1a0 con 0x7fa07c15cd80 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.076+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c0cb1f0 con 0x7fa07c15cd80 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.078+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c0cc6c0 con 0x7fa07c15cd80 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.079+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa074004b80 con 0x7fa07c15cd80 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.079+0000 7fa078ff9640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 0x7fa05c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:58.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.080+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa074088910 con 0x7fa07c15cd80 2023-10-21T16:17:58.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.080+0000 7fa07bfff640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 0x7fa05c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:58.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.081+0000 7fa07bfff640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 0x7fa05c06d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0099c0 tx=0x7fa06c003990 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:58.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.084+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa074053900 con 0x7fa07c15cd80 2023-10-21T16:17:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.279+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa07c04e160 con 0x7fa07c15cd80 2023-10-21T16:17:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.285+0000 7fa078ff9640 1 -- 172.21.15.53:0/3074113492 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0740180c0 con 0x7fa07c15cd80 2023-10-21T16:17:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 msgr2=0x7fa05c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 0x7fa05c06d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0099c0 tx=0x7fa06c003990 comp rx=0 tx=0).stop 2023-10-21T16:17:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 msgr2=0x7fa07c0cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa07400c450 tx=0x7fa07400c910 comp rx=0 tx=0).stop 2023-10-21T16:17:58.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 shutdown_connections 2023-10-21T16:17:58.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa05c06af00 0x7fa05c06d3c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa07c15cd80 0x7fa07c0cacb0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa07c0cdda0 0x7fa07c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 --2- 172.21.15.53:0/3074113492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07c0b6730 0x7fa07c1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:58.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 >> 172.21.15.53:0/3074113492 conn(0x7fa07c06b5d0 msgr2=0x7fa07c0c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:58.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 shutdown_connections 2023-10-21T16:17:58.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:58.288+0000 7fa082301640 1 -- 172.21.15.53:0/3074113492 wait complete. 2023-10-21T16:17:58.309 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:17:58.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:58.425 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:17:58.613 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:17:58.614 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:58.615 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:58.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:58.681 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:58.681 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:58.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:58.740 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:17:58.740 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:58.780 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:58.777+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:58.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:17:58.806 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:17:58.806 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:17:58.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:17:58.864 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:17:58.865 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:17:59.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.103+0000 7ff85a456640 1 Processor -- start 2023-10-21T16:17:59.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.104+0000 7ff85a456640 1 -- start start 2023-10-21T16:17:59.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff85a456640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff85415ab40 0x7ff8540ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff85a456640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff8540cb580 0x7ff8540adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff85a456640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85415b2b0 con 0x7ff8540c90b0 2023-10-21T16:17:59.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff85a456640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85415bf30 con 0x7ff85415ab40 2023-10-21T16:17:59.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff85a456640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85415f7a0 con 0x7ff8540cb580 2023-10-21T16:17:59.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff853fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff85415ab40 0x7ff8540ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff853fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff85415ab40 0x7ff8540ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33272/0 (socket says 172.21.15.53:33272) 2023-10-21T16:17:59.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff853fff640 1 -- 172.21.15.53:0/2646540578 learned_addr learned my addr 172.21.15.53:0/2646540578 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:59.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.106+0000 7ff853fff640 1 -- 172.21.15.53:0/2646540578 >> v2:172.21.15.124:3300/0 conn(0x7ff8540cb580 msgr2=0x7ff8540adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:59.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.107+0000 7ff8589f3640 1 --2- 172.21.15.53:0/2646540578 >> v2:172.21.15.124:3300/0 conn(0x7ff8540cb580 0x7ff8540adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.107+0000 7ff853fff640 1 --2- 172.21.15.53:0/2646540578 >> v2:172.21.15.124:3300/0 conn(0x7ff8540cb580 0x7ff8540adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.107+0000 7ff853fff640 1 -- 172.21.15.53:0/2646540578 >> v1:172.21.15.124:6789/0 conn(0x7ff8540c90b0 legacy=0x7ff8540cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.107+0000 7ff853fff640 1 -- 172.21.15.53:0/2646540578 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85415b590 con 0x7ff85415ab40 2023-10-21T16:17:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.108+0000 7ff853fff640 1 --2- 172.21.15.53:0/2646540578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85415ab40 0x7ff8540ad720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff8540b0ec0 tx=0x7ff84402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd64a8c66179c00d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.108+0000 7ff852ffd640 1 -- 172.21.15.53:0/2646540578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844031890 con 0x7ff85415ab40 2023-10-21T16:17:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.108+0000 7ff852ffd640 1 -- 172.21.15.53:0/2646540578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440319f0 con 0x7ff85415ab40 2023-10-21T16:17:59.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.108+0000 7ff852ffd640 1 -- 172.21.15.53:0/2646540578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844031d80 con 0x7ff85415ab40 2023-10-21T16:17:59.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 -- 172.21.15.53:0/2646540578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85415ab40 msgr2=0x7ff8540ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 --2- 172.21.15.53:0/2646540578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85415ab40 0x7ff8540ad720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff8540b0ec0 tx=0x7ff84402b930 comp rx=0 tx=0).stop 2023-10-21T16:17:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 -- 172.21.15.53:0/2646540578 shutdown_connections 2023-10-21T16:17:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 --2- 172.21.15.53:0/2646540578 >> v2:172.21.15.124:3300/0 conn(0x7ff8540cb580 0x7ff8540adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 --2- 172.21.15.53:0/2646540578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff85415ab40 0x7ff8540ad720 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 -- 172.21.15.53:0/2646540578 >> 172.21.15.53:0/2646540578 conn(0x7ff85406a380 msgr2=0x7ff8540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 -- 172.21.15.53:0/2646540578 shutdown_connections 2023-10-21T16:17:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.109+0000 7ff85a456640 1 -- 172.21.15.53:0/2646540578 wait complete. 2023-10-21T16:17:59.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.111+0000 7ff85a456640 1 Processor -- start 2023-10-21T16:17:59.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.111+0000 7ff85a456640 1 -- start start 2023-10-21T16:17:59.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 0x7ff8541502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 0x7ff854155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540b3a90 con 0x7ff8540cb580 2023-10-21T16:17:59.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540c84a0 con 0x7ff8540c90b0 2023-10-21T16:17:59.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff85a456640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540c7890 con 0x7ff85415ab40 2023-10-21T16:17:59.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff8537fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff8537fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33280/0 (socket says 172.21.15.53:33280) 2023-10-21T16:17:59.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.112+0000 7ff8537fe640 1 -- 172.21.15.53:0/1306287536 learned_addr learned my addr 172.21.15.53:0/1306287536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:59.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff853fff640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 0x7ff8541502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8589f3640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 0x7ff854155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 -- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 msgr2=0x7ff854155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 0x7ff854155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 -- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 msgr2=0x7ff8541502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 0x7ff8541502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 -- 172.21.15.53:0/1306287536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440094d0 con 0x7ff8540cb580 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff853fff640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 0x7ff8541502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8589f3640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 0x7ff854155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:17:59.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff8537fe640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c400 tx=0x7ff84800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480035f0 con 0x7ff8540cb580 2023-10-21T16:17:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848003750 con 0x7ff8540cb580 2023-10-21T16:17:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.113+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848003b00 con 0x7ff8540cb580 2023-10-21T16:17:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.114+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8540c7b70 con 0x7ff8540cb580 2023-10-21T16:17:59.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.114+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8540c8130 con 0x7ff8540cb580 2023-10-21T16:17:59.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.115+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8540c82f0 con 0x7ff8540cb580 2023-10-21T16:17:59.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.116+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff848003c60 con 0x7ff8540cb580 2023-10-21T16:17:59.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.117+0000 7ff850ff9640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 0x7ff84006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.117+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7ff8480888f0 con 0x7ff8540cb580 2023-10-21T16:17:59.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.117+0000 7ff853fff640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 0x7ff84006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.119+0000 7ff853fff640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 0x7ff84006d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff844036000 tx=0x7ff84403c040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.122+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff848053940 con 0x7ff8540cb580 2023-10-21T16:17:59.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.358+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8540cc680 con 0x7ff8540cb580 2023-10-21T16:17:59.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.360+0000 7ff850ff9640 1 -- 172.21.15.53:0/1306287536 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3703 (secure 0 0 0) 0x7ff848017e90 con 0x7ff8540cb580 2023-10-21T16:17:59.362 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:17:59.362 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3176965925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3176965925},{"type":"v1","addr":"172.21.15.124:6836","nonce":3176965925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1936634918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1936634918},{"type":"v1","addr":"172.21.15.124:6837","nonce":1936634918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2666912142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2666912142},{"type":"v1","addr":"172.21.15.53:6835","nonce":2666912142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":35342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/523533946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":523533946},{"type":"v1","addr":"172.21.15.53:6834","nonce":523533946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[]} 2023-10-21T16:17:59.362 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 38 2023-10-21T16:17:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 msgr2=0x7ff84006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 0x7ff84006d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff844036000 tx=0x7ff84403c040 comp rx=0 tx=0).stop 2023-10-21T16:17:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 msgr2=0x7ff8541550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c400 tx=0x7ff84800c8c0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 shutdown_connections 2023-10-21T16:17:59.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7ff84006af00 0x7ff84006d3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff85415ab40 0x7ff854155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8540cb580 0x7ff8541550d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 --2- 172.21.15.53:0/1306287536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8540c90b0 0x7ff8541502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.362+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 >> 172.21.15.53:0/1306287536 conn(0x7ff85406a380 msgr2=0x7ff8540701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:59.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.363+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 shutdown_connections 2023-10-21T16:17:59.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.363+0000 7ff85a456640 1 -- 172.21.15.53:0/1306287536 wait complete. 2023-10-21T16:17:59.385 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:17:59.385 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:17:59.386 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:17:59.386 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:59.389 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:17:59.389 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:17:59.390 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:17:59.390 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:59.390 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:17:59.391 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:17:59.391 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:17:59.391 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:59.391 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:17:59.392 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:17:59.392 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:17:59.392 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:17:59.404 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:17:59.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:17:59.413 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:17:59.413 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:17:59.415 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:17:59.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:17:59.419 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:17:59.419 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:17:59.464 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:17:59.464 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:59.465 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:17:59.466 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:59.472 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:17:59.472 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:59.473 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:17:59.474 DEBUG:teuthology.parallel:result is None 2023-10-21T16:17:59.474 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:17:59.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-21T16:17:59.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-21T16:17:59.635 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:17:59.637 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:17:59.639 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:17:59.641 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:17:59.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.673+0000 7fa7f4df0640 1 Processor -- start 2023-10-21T16:17:59.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.674+0000 7fa7f4df0640 1 -- start start 2023-10-21T16:17:59.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7f4df0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7f006c3a0 0x7fa7f00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7f4df0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7f4df0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f015b310 con 0x7fa7f015aa00 2023-10-21T16:17:59.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7f4df0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f00ca230 con 0x7fa7f006c3a0 2023-10-21T16:17:59.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7f4df0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f006d290 con 0x7fa7f006db60 2023-10-21T16:17:59.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7ee59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7f006c3a0 0x7fa7f00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7eed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa7f015aa00 0x7fa7f00b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49138/0 (socket says 172.21.15.53:49138) 2023-10-21T16:17:59.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7eed9d640 1 -- 172.21.15.53:0/1180184175 learned_addr learned my addr 172.21.15.53:0/1180184175 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:17:59.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.675+0000 7fa7e7fff640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7ee59c640 1 -- 172.21.15.53:0/1180184175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 msgr2=0x7fa7f006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:17:59.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7ee59c640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7ee59c640 1 -- 172.21.15.53:0/1180184175 >> v1:172.21.15.53:6789/0 conn(0x7fa7f015aa00 legacy=0x7fa7f00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7ee59c640 1 -- 172.21.15.53:0/1180184175 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f00c9860 con 0x7fa7f006c3a0 2023-10-21T16:17:59.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7edd9b640 1 -- 172.21.15.53:0/1180184175 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148558549 0 0) 0x7fa7e8001490 con 0x7fa7f015aa00 2023-10-21T16:17:59.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.676+0000 7fa7e7fff640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:17:59.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.677+0000 7fa7ee59c640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f00c9cf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00097a0 tx=0x7fa7e002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930a829de1e2fe0b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.677+0000 7fa7edd9b640 1 -- 172.21.15.53:0/1180184175 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e002d040 con 0x7fa7f006c3a0 2023-10-21T16:17:59.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.677+0000 7fa7edd9b640 1 -- 172.21.15.53:0/1180184175 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e0031810 con 0x7fa7f006c3a0 2023-10-21T16:17:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.677+0000 7fa7edd9b640 1 -- 172.21.15.53:0/1180184175 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0031ba0 con 0x7fa7f006c3a0 2023-10-21T16:17:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1180184175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 msgr2=0x7fa7f00c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f00c9cf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00097a0 tx=0x7fa7e002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1180184175 shutdown_connections 2023-10-21T16:17:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1180184175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f00c9cf0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1180184175 >> 172.21.15.53:0/1180184175 conn(0x7fa7f006a5d0 msgr2=0x7fa7f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:17:59.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1180184175 shutdown_connections 2023-10-21T16:17:59.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.678+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1180184175 wait complete. 2023-10-21T16:17:59.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.680+0000 7fa7f4df0640 1 Processor -- start 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.680+0000 7fa7f4df0640 1 -- start start 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f0150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7f015aa00 0x7fa7f0158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c7cf0 con 0x7fa7f006c3a0 2023-10-21T16:17:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00c7fd0 con 0x7fa7f015aa00 2023-10-21T16:17:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7f4df0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0166600 con 0x7fa7f006db60 2023-10-21T16:17:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f0150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f0150410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33328/0 (socket says 172.21.15.53:33328) 2023-10-21T16:17:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7e7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.681+0000 7fa7ee59c640 1 -- 172.21.15.53:0/1001317185 learned_addr learned my addr 172.21.15.53:0/1001317185 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:17:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7eed9d640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7f015aa00 0x7fa7f0158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 -- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7f015aa00 msgr2=0x7fa7f0158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7f015aa00 0x7fa7f0158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 -- 172.21.15.53:0/1001317185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 msgr2=0x7fa7f0150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:17:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f0150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:17:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 -- 172.21.15.53:0/1001317185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009400 con 0x7fa7f006db60 2023-10-21T16:17:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.682+0000 7fa7e7fff640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f014fad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e900 tx=0x7fa7d800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.683+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800ca00 con 0x7fa7f006db60 2023-10-21T16:17:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.683+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e0009400 con 0x7fa7f006db60 2023-10-21T16:17:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.683+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f00c85d0 con 0x7fa7f006db60 2023-10-21T16:17:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.683+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d800cb60 con 0x7fa7f006db60 2023-10-21T16:17:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.683+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8013470 con 0x7fa7f006db60 2023-10-21T16:17:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.684+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d4005250 con 0x7fa7f006db60 2023-10-21T16:17:59.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:17:59.682+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:17:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.685+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa7d80135d0 con 0x7fa7f006db60 2023-10-21T16:17:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.686+0000 7fa7e6ffd640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 0x7fa7d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:17:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.686+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa7d80898e0 con 0x7fa7f006db60 2023-10-21T16:17:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.687+0000 7fa7ee59c640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 0x7fa7d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:17:59.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.687+0000 7fa7ee59c640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 0x7fa7d006d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00c7b00 tx=0x7fa7e003d040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:17:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.693+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7d8054930 con 0x7fa7f006db60 2023-10-21T16:17:59.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:17:59.908+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa7d4005500 con 0x7fa7f006db60 2023-10-21T16:18:00.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.312+0000 7fa7e6ffd640 1 -- 172.21.15.53:0/1001317185 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa7d8057f50 con 0x7fa7f006db60 2023-10-21T16:18:00.314 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:18:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.315+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 msgr2=0x7fa7d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.315+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 0x7fa7d006d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00c7b00 tx=0x7fa7e003d040 comp rx=0 tx=0).stop 2023-10-21T16:18:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.315+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 msgr2=0x7fa7f014fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.315+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f014fad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800e900 tx=0x7fa7d800edc0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 shutdown_connections 2023-10-21T16:18:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fa7d006af00 0x7fa7d006d3c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7f015aa00 0x7fa7f0158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7f006db60 0x7fa7f014fad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 --2- 172.21.15.53:0/1001317185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7f006c3a0 0x7fa7f0150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 >> 172.21.15.53:0/1001317185 conn(0x7fa7f006a5d0 msgr2=0x7fa7f00b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 shutdown_connections 2023-10-21T16:18:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.316+0000 7fa7f4df0640 1 -- 172.21.15.53:0/1001317185 wait complete. 2023-10-21T16:18:00.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:18:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.558+0000 7f69d2b0f640 1 Processor -- start 2023-10-21T16:18:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.558+0000 7f69d2b0f640 1 -- start start 2023-10-21T16:18:00.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.560+0000 7f69d2b0f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.560+0000 7f69d2b0f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69cc0ad790 0x7f69cc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.560+0000 7f69d2b0f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc06dee0 con 0x7f69cc15e330 2023-10-21T16:18:00.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.560+0000 7f69d2b0f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc15d850 con 0x7f69cc0ad790 2023-10-21T16:18:00.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.560+0000 7f69d2b0f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc15db30 con 0x7f69cc159ac0 2023-10-21T16:18:00.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.561+0000 7f69d08ab640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.561+0000 7f69d08ab640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:38448/0 (socket says 172.21.15.53:38448) 2023-10-21T16:18:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.561+0000 7f69d08ab640 1 -- 172.21.15.53:0/3717620609 learned_addr learned my addr 172.21.15.53:0/3717620609 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.561+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3717620609 >> v2:172.21.15.124:3300/0 conn(0x7f69cc0ad790 0x7f69cc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.562+0000 7f69cbfff640 1 -- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 msgr2=0x7f69cc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.562+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.562+0000 7f69cbfff640 1 -- 172.21.15.53:0/3717620609 >> v1:172.21.15.124:6789/0 conn(0x7f69cc15e330 legacy=0x7f69cc06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.562+0000 7f69cbfff640 1 -- 172.21.15.53:0/3717620609 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc15de10 con 0x7f69cc0ad790 2023-10-21T16:18:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.562+0000 7f69d08ab640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:00.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.563+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc0ad790 0x7f69cc0702d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332bfda0cd816b54 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:00.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69cb7fe640 1 -- 172.21.15.53:0/3717620609 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b402cc50 con 0x7f69cc0ad790 2023-10-21T16:18:00.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69cb7fe640 1 -- 172.21.15.53:0/3717620609 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b402cdb0 con 0x7f69cc0ad790 2023-10-21T16:18:00.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69cb7fe640 1 -- 172.21.15.53:0/3717620609 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4034660 con 0x7f69cc0ad790 2023-10-21T16:18:00.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc0ad790 msgr2=0x7f69cc0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc0ad790 0x7f69cc0702d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b402bea0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3717620609 shutdown_connections 2023-10-21T16:18:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc0ad790 0x7f69cc0702d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3717620609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.564+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3717620609 >> 172.21.15.53:0/3717620609 conn(0x7f69cc06a380 msgr2=0x7f69cc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.565+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3717620609 shutdown_connections 2023-10-21T16:18:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.565+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3717620609 wait complete. 2023-10-21T16:18:00.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.566+0000 7f69d2b0f640 1 Processor -- start 2023-10-21T16:18:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.567+0000 7f69d2b0f640 1 -- start start 2023-10-21T16:18:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.567+0000 7f69d2b0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.567+0000 7f69d2b0f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d2b0f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 0x7f69cc15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d2b0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b15d0 con 0x7f69cc0ad790 2023-10-21T16:18:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d2b0f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc148eb0 con 0x7f69cc15e330 2023-10-21T16:18:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d2b0f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc149190 con 0x7f69cc159ac0 2023-10-21T16:18:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33330/0 (socket says 172.21.15.53:33330) 2023-10-21T16:18:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 -- 172.21.15.53:0/3855859895 learned_addr learned my addr 172.21.15.53:0/3855859895 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d10ac640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 0x7f69cc15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 -- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 msgr2=0x7f69cc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 -- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 msgr2=0x7f69cc15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 0x7f69cc15bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d08ab640 1 -- 172.21.15.53:0/3855859895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40094d0 con 0x7f69cc0ad790 2023-10-21T16:18:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.568+0000 7f69d10ac640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 0x7f69cc15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69d08ab640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e9d0 tx=0x7f69c400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400ca60 con 0x7f69cc0ad790 2023-10-21T16:18:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c400cbc0 con 0x7f69cc0ad790 2023-10-21T16:18:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4013470 con 0x7f69cc0ad790 2023-10-21T16:18:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b4009a70 con 0x7f69cc0ad790 2023-10-21T16:18:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.569+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0b41b0 con 0x7f69cc0ad790 2023-10-21T16:18:00.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.570+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc0b1420 con 0x7f69cc0ad790 2023-10-21T16:18:00.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.571+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f69c40135d0 con 0x7f69cc0ad790 2023-10-21T16:18:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.572+0000 7f69c97fa640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 0x7f69b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.572+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f69c4089b40 con 0x7f69cc0ad790 2023-10-21T16:18:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.573+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 0x7f69b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.574+0000 7f69cbfff640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 0x7f69b006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b4036000 tx=0x7f69b4030000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:00.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.577+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69c4054d70 con 0x7f69cc0ad790 2023-10-21T16:18:00.668 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:00.664+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.855+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f69cc067490 con 0x7f69cc0ad790 2023-10-21T16:18:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.857+0000 7f69c97fa640 1 -- 172.21.15.53:0/3855859895 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f69c4017090 con 0x7f69cc0ad790 2023-10-21T16:18:00.858 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:00.859 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:18:00.859 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:18:00.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.859 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:00.860 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.861 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:18:00.861 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:18:00.862 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.862 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.862 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:00.862 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:00.862 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:18:00.863 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.867 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:18:00.867 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:18:00.867 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.867 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.868 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:18:00.869 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:18:00.869 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.869 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.869 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.869 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.870 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:18:00.870 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:18:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:18:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.872 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.873 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.873 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.875 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:18:00.875 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:18:00.876 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.876 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.876 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.876 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.877 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.878 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.878 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.879 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:18:00.879 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:18:00.879 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.879 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:00.879 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:18:00.880 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:18:00.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.882 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:18:00.882 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:18:00.882 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.883 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.883 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:00.883 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:00.883 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:00.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.884 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:18:00.884 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:18:00.884 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.886 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:18:00.886 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:18:00.886 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.886 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.888 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:18:00.888 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:18:00.888 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.888 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.890 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:18:00.890 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:18:00.890 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.890 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:00.890 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:18:00.891 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:00.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:00.891 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:00.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:00.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:18:00.893 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:18:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 msgr2=0x7f69b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 0x7f69b006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b4036000 tx=0x7f69b4030000 comp rx=0 tx=0).stop 2023-10-21T16:18:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 msgr2=0x7f69cc15c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f69c400e9d0 tx=0x7f69c400ee90 comp rx=0 tx=0).stop 2023-10-21T16:18:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 shutdown_connections 2023-10-21T16:18:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f69b006af00 0x7f69b006d3c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69cc15e330 0x7f69cc15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69cc159ac0 0x7f69cc0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 --2- 172.21.15.53:0/3855859895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69cc0ad790 0x7f69cc15c990 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 >> 172.21.15.53:0/3855859895 conn(0x7f69cc06a380 msgr2=0x7f69cc0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 shutdown_connections 2023-10-21T16:18:00.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:00.860+0000 7f69d2b0f640 1 -- 172.21.15.53:0/3855859895 wait complete. 2023-10-21T16:18:00.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:01.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.128+0000 7f7b02373640 1 Processor -- start 2023-10-21T16:18:01.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.128+0000 7f7b02373640 1 -- start start 2023-10-21T16:18:01.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.129+0000 7f7b02373640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7afc0cb880 0x7f7afc0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7b02373640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7afc0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7b02373640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc0ce550 con 0x7f7afc069d50 2023-10-21T16:18:01.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7b02373640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc1609d0 con 0x7f7afc0cb880 2023-10-21T16:18:01.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7b02373640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0cdcf0 con 0x7f7afc0cbce0 2023-10-21T16:18:01.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7afbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7afc069d50 0x7f7afc06a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49148/0 (socket says 172.21.15.53:49148) 2023-10-21T16:18:01.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7afbfff640 1 -- 172.21.15.53:0/910329905 learned_addr learned my addr 172.21.15.53:0/910329905 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:01.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7b00910640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7afc0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.130+0000 7f7af3fff640 1 --2- 172.21.15.53:0/910329905 >> v2:172.21.15.124:3300/0 conn(0x7f7afc0cb880 0x7f7afc0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726242860 0 0) 0x7f7afc0ce550 con 0x7f7afc069d50 2023-10-21T16:18:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae8003520 con 0x7f7afc069d50 2023-10-21T16:18:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355751502 0 0) 0x7f7ae8003520 con 0x7f7afc069d50 2023-10-21T16:18:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 msgr2=0x7f7afc0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7afc0b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 >> v2:172.21.15.124:3300/0 conn(0x7f7afc0cb880 msgr2=0x7f7afc0b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 --2- 172.21.15.53:0/910329905 >> v2:172.21.15.124:3300/0 conn(0x7f7afc0cb880 0x7f7afc0b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0cdf90 con 0x7f7afc069d50 2023-10-21T16:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7aec001670 con 0x7f7afc069d50 2023-10-21T16:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 >> v1:172.21.15.53:6789/0 conn(0x7f7afc069d50 legacy=0x7f7afc06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7ae80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1609d0 con 0x7f7ae8003970 2023-10-21T16:18:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0cdcf0 con 0x7f7ae8004d00 2023-10-21T16:18:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae800b980 con 0x7f7afc0cbce0 2023-10-21T16:18:01.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.131+0000 7f7afbfff640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7af3fff640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7ae80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7b00910640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7afbfff640 1 -- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 msgr2=0x7f7ae80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:01.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7afbfff640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc0cbce0 0x7f7ae80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7afbfff640 1 -- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8004d00 msgr2=0x7f7ae8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7afbfff640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8004d00 0x7f7ae8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.132+0000 7f7afbfff640 1 -- 172.21.15.53:0/910329905 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8003520 con 0x7f7ae8003970 2023-10-21T16:18:01.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.133+0000 7f7afbfff640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c270 tx=0x7f7ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3460d6d031eb82cc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.134+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02cae0 con 0x7f7ae8003970 2023-10-21T16:18:01.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.134+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec02cc40 con 0x7f7ae8003970 2023-10-21T16:18:01.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.134+0000 7f7afb7fe640 1 -- 172.21.15.53:0/910329905 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0058d0 con 0x7f7ae8003970 2023-10-21T16:18:01.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.134+0000 7f7b02373640 1 -- 172.21.15.53:0/910329905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ae8003970 msgr2=0x7f7ae8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.134+0000 7f7b02373640 1 --2- 172.21.15.53:0/910329905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ae8003970 0x7f7ae8003d90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00c270 tx=0x7f7ae80057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.135+0000 7f7b00910640 1 -- 172.21.15.53:0/910329905 reap_dead start 2023-10-21T16:18:01.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.135+0000 7f7b02373640 1 -- 172.21.15.53:0/910329905 shutdown_connections 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.135+0000 7f7b02373640 1 -- 172.21.15.53:0/910329905 >> 172.21.15.53:0/910329905 conn(0x7f7afc06d450 msgr2=0x7f7afc06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.135+0000 7f7b02373640 1 -- 172.21.15.53:0/910329905 shutdown_connections 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.135+0000 7f7b02373640 1 -- 172.21.15.53:0/910329905 wait complete. 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.137+0000 7f7b02373640 1 Processor -- start 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.138+0000 7f7b02373640 1 -- start start 2023-10-21T16:18:01.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8003970 0x7f7afc14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc071700 0x7f7afc071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0720c0 con 0x7f7afc0b4970 2023-10-21T16:18:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0b5410 con 0x7f7ae8003970 2023-10-21T16:18:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7b02373640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0aba20 con 0x7f7afc071700 2023-10-21T16:18:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7af3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7af3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33354/0 (socket says 172.21.15.53:33354) 2023-10-21T16:18:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.139+0000 7f7af3fff640 1 -- 172.21.15.53:0/892695354 learned_addr learned my addr 172.21.15.53:0/892695354 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7afbfff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8003970 0x7f7afc14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7b00910640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc071700 0x7f7afc071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 -- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc071700 msgr2=0x7f7afc071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc071700 0x7f7afc071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 -- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8003970 msgr2=0x7f7afc14ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8003970 0x7f7afc14ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 -- 172.21.15.53:0/892695354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0abd00 con 0x7f7afc0b4970 2023-10-21T16:18:01.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af3fff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4011630 tx=0x7f7ae4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.140+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400f920 con 0x7f7afc0b4970 2023-10-21T16:18:01.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.141+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae400fa80 con 0x7f7afc0b4970 2023-10-21T16:18:01.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.141+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400fdf0 con 0x7f7afc0b4970 2023-10-21T16:18:01.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.141+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc0abfe0 con 0x7f7afc0b4970 2023-10-21T16:18:01.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.141+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc0b26e0 con 0x7f7afc0b4970 2023-10-21T16:18:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.142+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc067490 con 0x7f7afc0b4970 2023-10-21T16:18:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.143+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7ae4015460 con 0x7f7afc0b4970 2023-10-21T16:18:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.144+0000 7f7af97fa640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 0x7f7adc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.144+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f7ae40899f0 con 0x7f7afc0b4970 2023-10-21T16:18:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.145+0000 7f7afbfff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 0x7f7adc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.146+0000 7f7afbfff640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 0x7f7adc06d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7aec034000 tx=0x7f7aec007000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.149+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ae4054b00 con 0x7f7afc0b4970 2023-10-21T16:18:01.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.358+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7afc0b5810 con 0x7f7afc0b4970 2023-10-21T16:18:01.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.360+0000 7f7af97fa640 1 -- 172.21.15.53:0/892695354 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13121 (secure 0 0 0) 0x7f7ae4022020 con 0x7f7afc0b4970 2023-10-21T16:18:01.362 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:01.362 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":99,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:00.322514+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 msgr2=0x7f7adc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 0x7f7adc06d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7aec034000 tx=0x7f7aec007000 comp rx=0 tx=0).stop 2023-10-21T16:18:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 msgr2=0x7f7afc0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4011630 tx=0x7f7ae4011af0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 shutdown_connections 2023-10-21T16:18:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f7adc06af30 0x7f7adc06d3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7afc071700 0x7f7afc071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7afc0b4970 0x7f7afc0b4df0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 --2- 172.21.15.53:0/892695354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ae8003970 0x7f7afc14ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 >> 172.21.15.53:0/892695354 conn(0x7f7afc06d450 msgr2=0x7f7afc15ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 shutdown_connections 2023-10-21T16:18:01.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.363+0000 7f7b02373640 1 -- 172.21.15.53:0/892695354 wait complete. 2023-10-21T16:18:01.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:18:01.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:18:01.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.605+0000 7f797bfb0640 1 Processor -- start 2023-10-21T16:18:01.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.605+0000 7f797bfb0640 1 -- start start 2023-10-21T16:18:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797bfb0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f797415aad0 0x7f79740b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797bfb0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 0x7f79740c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797bfb0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797415bf30 con 0x7f797406cd00 2023-10-21T16:18:01.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797bfb0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797415b320 con 0x7f797415aad0 2023-10-21T16:18:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797bfb0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797415b600 con 0x7f79740c82b0 2023-10-21T16:18:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f7979d4c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f797415aad0 0x7f79740b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797a54d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f797406cd00 0x7f7974070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49164/0 (socket says 172.21.15.53:49164) 2023-10-21T16:18:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f797a54d640 1 -- 172.21.15.53:0/1497031203 learned_addr learned my addr 172.21.15.53:0/1497031203 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:01.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.607+0000 7f7979d4c640 1 --2- 172.21.15.53:0/1497031203 >> v2:172.21.15.53:3300/0 conn(0x7f797415aad0 0x7f79740b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33360/0 (socket says 172.21.15.53:33360) 2023-10-21T16:18:01.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f7979d4c640 1 -- 172.21.15.53:0/1497031203 >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 msgr2=0x7f79740c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:01.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f797954b640 1 --2- 172.21.15.53:0/1497031203 >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 0x7f79740c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f7979d4c640 1 --2- 172.21.15.53:0/1497031203 >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 0x7f79740c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f7979d4c640 1 -- 172.21.15.53:0/1497031203 >> v1:172.21.15.53:6789/0 conn(0x7f797406cd00 legacy=0x7f7974070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f7979d4c640 1 -- 172.21.15.53:0/1497031203 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797415b8e0 con 0x7f797415aad0 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f797954b640 1 --2- 172.21.15.53:0/1497031203 >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 0x7f79740c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.608+0000 7f7978d4a640 1 -- 172.21.15.53:0/1497031203 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582852932 0 0) 0x7f797415bf30 con 0x7f797406cd00 2023-10-21T16:18:01.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.609+0000 7f7979d4c640 1 --2- 172.21.15.53:0/1497031203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797415aad0 0x7f79740b1750 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f797406e330 tx=0x7f797002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6861fdac514bca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f7978d4a640 1 -- 172.21.15.53:0/1497031203 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797002cc80 con 0x7f797415aad0 2023-10-21T16:18:01.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f7978d4a640 1 -- 172.21.15.53:0/1497031203 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797002cde0 con 0x7f797415aad0 2023-10-21T16:18:01.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f7978d4a640 1 -- 172.21.15.53:0/1497031203 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970034660 con 0x7f797415aad0 2023-10-21T16:18:01.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 -- 172.21.15.53:0/1497031203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797415aad0 msgr2=0x7f79740b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 --2- 172.21.15.53:0/1497031203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797415aad0 0x7f79740b1750 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f797406e330 tx=0x7f797002bed0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 -- 172.21.15.53:0/1497031203 shutdown_connections 2023-10-21T16:18:01.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 --2- 172.21.15.53:0/1497031203 >> v2:172.21.15.124:3300/0 conn(0x7f79740c82b0 0x7f79740c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 --2- 172.21.15.53:0/1497031203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797415aad0 0x7f79740b1750 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 -- 172.21.15.53:0/1497031203 >> 172.21.15.53:0/1497031203 conn(0x7f7974069d50 msgr2=0x7f797406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:01.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.610+0000 7f797bfb0640 1 -- 172.21.15.53:0/1497031203 shutdown_connections 2023-10-21T16:18:01.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.611+0000 7f797bfb0640 1 -- 172.21.15.53:0/1497031203 wait complete. 2023-10-21T16:18:01.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.613+0000 7f797bfb0640 1 Processor -- start 2023-10-21T16:18:01.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.613+0000 7f797bfb0640 1 -- start start 2023-10-21T16:18:01.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79740c82b0 0x7f797414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 0x7f79740cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741e5840 con 0x7f797406cd00 2023-10-21T16:18:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741e4bc0 con 0x7f79740c82b0 2023-10-21T16:18:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.614+0000 7f797bfb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79741e4ea0 con 0x7f797415aad0 2023-10-21T16:18:01.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33366/0 (socket says 172.21.15.53:33366) 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 -- 172.21.15.53:0/2029804251 learned_addr learned my addr 172.21.15.53:0/2029804251 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f797a54d640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 0x7f79740cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f797954b640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79740c82b0 0x7f797414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 -- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 msgr2=0x7f79740cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 0x7f79740cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 -- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79740c82b0 msgr2=0x7f797414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79740c82b0 0x7f797414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f7979d4c640 1 -- 172.21.15.53:0/2029804251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970009a70 con 0x7f797406cd00 2023-10-21T16:18:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.615+0000 7f797a54d640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 0x7f79740cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f7979d4c640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7970009e80 tx=0x7f7970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970003b90 con 0x7f797406cd00 2023-10-21T16:18:01.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970003cf0 con 0x7f797406cd00 2023-10-21T16:18:01.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970004260 con 0x7f797406cd00 2023-10-21T16:18:01.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79741e5180 con 0x7f797406cd00 2023-10-21T16:18:01.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.616+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79741e48d0 con 0x7f797406cd00 2023-10-21T16:18:01.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.617+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797404e160 con 0x7f797406cd00 2023-10-21T16:18:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.618+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f797003b050 con 0x7f797406cd00 2023-10-21T16:18:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.619+0000 7f796a7fc640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 0x7f795806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.619+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f79700abca0 con 0x7f797406cd00 2023-10-21T16:18:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.620+0000 7f797954b640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 0x7f795806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.621+0000 7f797954b640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 0x7f795806d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79741f51f0 tx=0x7f7964008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.625+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7970076e40 con 0x7f797406cd00 2023-10-21T16:18:01.640 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:01.637+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:01.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.828+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79740b3a60 con 0x7f797406cd00 2023-10-21T16:18:01.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.828+0000 7f796a7fc640 1 -- 172.21.15.53:0/2029804251 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7970039020 con 0x7f797406cd00 2023-10-21T16:18:01.830 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:18:01.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 msgr2=0x7f795806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 0x7f795806d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f79741f51f0 tx=0x7f7964008040 comp rx=0 tx=0).stop 2023-10-21T16:18:01.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 msgr2=0x7f79741e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:01.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7970009e80 tx=0x7f7970004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 shutdown_connections 2023-10-21T16:18:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f795806af30 0x7f795806d3f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f797415aad0 0x7f79740cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79740c82b0 0x7f797414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 --2- 172.21.15.53:0/2029804251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797406cd00 0x7f79741e4390 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:01.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 >> 172.21.15.53:0/2029804251 conn(0x7f7974069d50 msgr2=0x7f7974150b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:01.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 shutdown_connections 2023-10-21T16:18:01.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:01.831+0000 7f797bfb0640 1 -- 172.21.15.53:0/2029804251 wait complete. 2023-10-21T16:18:01.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:18:02.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.072+0000 7fcb696bb640 1 Processor -- start 2023-10-21T16:18:02.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.072+0000 7fcb696bb640 1 -- start start 2023-10-21T16:18:02.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb696bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcb640b07d0 0x7fcb640cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb696bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb640c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb696bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb64165470 con 0x7fcb64161320 2023-10-21T16:18:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb696bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb64165750 con 0x7fcb640b07d0 2023-10-21T16:18:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb696bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64165a30 con 0x7fcb64164f70 2023-10-21T16:18:02.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb63fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcb64161320 0x7fcb640b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49176/0 (socket says 172.21.15.53:49176) 2023-10-21T16:18:02.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.074+0000 7fcb63fff640 1 -- 172.21.15.53:0/2667299861 learned_addr learned my addr 172.21.15.53:0/2667299861 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb68eba640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb640c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2667299861 >> v2:172.21.15.124:3300/0 conn(0x7fcb640b07d0 0x7fcb640cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293793754 0 0) 0x7fcb64165470 con 0x7fcb64161320 2023-10-21T16:18:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb50003520 con 0x7fcb64161320 2023-10-21T16:18:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 878082490 0 0) 0x7fcb50003520 con 0x7fcb64161320 2023-10-21T16:18:02.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 msgr2=0x7fcb640c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb640c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 >> v2:172.21.15.124:3300/0 conn(0x7fcb640b07d0 msgr2=0x7fcb640cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 --2- 172.21.15.53:0/2667299861 >> v2:172.21.15.124:3300/0 conn(0x7fcb640b07d0 0x7fcb640cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64165d10 con 0x7fcb64161320 2023-10-21T16:18:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcb540019f0 con 0x7fcb64161320 2023-10-21T16:18:02.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 >> v1:172.21.15.53:6789/0 conn(0x7fcb64161320 legacy=0x7fcb640b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb50003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50003f40 0x7fcb500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50004d00 0x7fcb50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64165750 con 0x7fcb50003f40 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64165a30 con 0x7fcb50004d00 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.075+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5000ba00 con 0x7fcb64164f70 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50003f40 0x7fcb500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb63fff640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb50003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 msgr2=0x7fcb50003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb64164f70 0x7fcb50003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50004d00 msgr2=0x7fcb50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50004d00 0x7fcb50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.076+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2667299861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50003520 con 0x7fcb50003f40 2023-10-21T16:18:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.077+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50003f40 0x7fcb500047c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009ac0 tx=0x7fcb500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f3ac39a7960d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c02c710 con 0x7fcb50003f40 2023-10-21T16:18:02.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c02c870 con 0x7fcb50003f40 2023-10-21T16:18:02.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb637fe640 1 -- 172.21.15.53:0/2667299861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c02cc00 con 0x7fcb50003f40 2023-10-21T16:18:02.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 -- 172.21.15.53:0/2667299861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50003f40 msgr2=0x7fcb500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2667299861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb50003f40 0x7fcb500047c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009ac0 tx=0x7fcb500057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb68eba640 1 -- 172.21.15.53:0/2667299861 reap_dead start 2023-10-21T16:18:02.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 -- 172.21.15.53:0/2667299861 shutdown_connections 2023-10-21T16:18:02.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 -- 172.21.15.53:0/2667299861 >> 172.21.15.53:0/2667299861 conn(0x7fcb640cb0c0 msgr2=0x7fcb640cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:02.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 -- 172.21.15.53:0/2667299861 shutdown_connections 2023-10-21T16:18:02.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.078+0000 7fcb696bb640 1 -- 172.21.15.53:0/2667299861 wait complete. 2023-10-21T16:18:02.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.080+0000 7fcb696bb640 1 Processor -- start 2023-10-21T16:18:02.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.080+0000 7fcb696bb640 1 -- start start 2023-10-21T16:18:02.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50003f40 0x7fcb641f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb640af2a0 0x7fcb640af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640afc60 con 0x7fcb641f74f0 2023-10-21T16:18:02.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b25c0 con 0x7fcb50003f40 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.081+0000 7fcb696bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b28a0 con 0x7fcb640af2a0 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33376/0 (socket says 172.21.15.53:33376) 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2755646368 learned_addr learned my addr 172.21.15.53:0/2755646368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb63fff640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50003f40 0x7fcb641f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb640af2a0 msgr2=0x7fcb640af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb640af2a0 0x7fcb640af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50003f40 msgr2=0x7fcb641f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50003f40 0x7fcb641f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 -- 172.21.15.53:0/2755646368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640b2b80 con 0x7fcb641f74f0 2023-10-21T16:18:02.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.082+0000 7fcb5b7fe640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02c2b0 tx=0x7fcb4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.083+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003810 con 0x7fcb641f74f0 2023-10-21T16:18:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.083+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c02de90 con 0x7fcb641f74f0 2023-10-21T16:18:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.083+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003b70 con 0x7fcb641f74f0 2023-10-21T16:18:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.083+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640b2e60 con 0x7fcb641f74f0 2023-10-21T16:18:02.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.083+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640b22b0 con 0x7fcb641f74f0 2023-10-21T16:18:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.084+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64162e00 con 0x7fcb641f74f0 2023-10-21T16:18:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.085+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcb4c003cd0 con 0x7fcb641f74f0 2023-10-21T16:18:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.086+0000 7fcb617fa640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 0x7fcb4406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.086+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fcb4c0ab350 con 0x7fcb641f74f0 2023-10-21T16:18:02.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.087+0000 7fcb63fff640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 0x7fcb4406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.088+0000 7fcb63fff640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 0x7fcb4406d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb54007c40 tx=0x7fcb54009040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.090+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb64162e00 con 0x7fcb641f74f0 2023-10-21T16:18:02.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.291+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcb64162e00 con 0x7fcb641f74f0 2023-10-21T16:18:02.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.292+0000 7fcb617fa640 1 -- 172.21.15.53:0/2755646368 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcb4c079f80 con 0x7fcb641f74f0 2023-10-21T16:18:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:02.294 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:18:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:18:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:18:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:02.295 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:18:02.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 msgr2=0x7fcb4406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 0x7fcb4406d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb54007c40 tx=0x7fcb54009040 comp rx=0 tx=0).stop 2023-10-21T16:18:02.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 msgr2=0x7fcb641f98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c02c2b0 tx=0x7fcb4c004730 comp rx=0 tx=0).stop 2023-10-21T16:18:02.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 shutdown_connections 2023-10-21T16:18:02.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fcb4406af00 0x7fcb4406d3c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb640af2a0 0x7fcb640af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb641f74f0 0x7fcb641f98a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 --2- 172.21.15.53:0/2755646368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb50003f40 0x7fcb641f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.295+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 >> 172.21.15.53:0/2755646368 conn(0x7fcb640cb0c0 msgr2=0x7fcb6406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.296+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 shutdown_connections 2023-10-21T16:18:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.296+0000 7fcb696bb640 1 -- 172.21.15.53:0/2755646368 wait complete. 2023-10-21T16:18:02.318 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:18:02.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:18:02.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.547+0000 7f3918468640 1 Processor -- start 2023-10-21T16:18:02.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.547+0000 7f3918468640 1 -- start start 2023-10-21T16:18:02.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3918468640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 0x7f39100d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3918468640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39100d1630 con 0x7f39100d26d0 2023-10-21T16:18:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3918468640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39100d1910 con 0x7f39100d32e0 2023-10-21T16:18:02.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3918468640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100d1bf0 con 0x7f391015cb80 2023-10-21T16:18:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3915a03640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39100d26d0 0x7f39100d2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49188/0 (socket says 172.21.15.53:49188) 2023-10-21T16:18:02.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.549+0000 7f3915a03640 1 -- 172.21.15.53:0/3435787562 learned_addr learned my addr 172.21.15.53:0/3435787562 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:02.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.550+0000 7f3916204640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 0x7f39100d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.550+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585315195 0 0) 0x7f39100d1630 con 0x7f39100d26d0 2023-10-21T16:18:02.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.550+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc003520 con 0x7f39100d26d0 2023-10-21T16:18:02.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.550+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970339847 0 0) 0x7f39100d1910 con 0x7f39100d32e0 2023-10-21T16:18:02.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39100d1630 con 0x7f39100d32e0 2023-10-21T16:18:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3135472509 0 0) 0x7f38fc003520 con 0x7f39100d26d0 2023-10-21T16:18:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 msgr2=0x7f39100d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 0x7f39100d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 >> v1:172.21.15.124:6789/0 conn(0x7f39100d32e0 legacy=0x7f39100b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100d1ed0 con 0x7f39100d26d0 2023-10-21T16:18:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3900001670 con 0x7f39100d26d0 2023-10-21T16:18:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 >> v1:172.21.15.53:6789/0 conn(0x7f39100d26d0 legacy=0x7f39100d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38fc003b30 0x7f38fc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f38fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 0x7f38fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100d1bf0 con 0x7f38fc0047c0 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc00ba30 con 0x7f38fc003b30 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc00bbf0 con 0x7f391015cb80 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3915a03640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f38fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.551+0000 7f3916204640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38fc003b30 0x7f38fc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.552+0000 7f3915a03640 1 -- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 msgr2=0x7f38fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.552+0000 7f3915a03640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015cb80 0x7f38fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.552+0000 7f3915a03640 1 -- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38fc003b30 msgr2=0x7f38fc003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:02.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.552+0000 7f3915a03640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38fc003b30 0x7f38fc003f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.552+0000 7f3915a03640 1 -- 172.21.15.53:0/3435787562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc003520 con 0x7f38fc0047c0 2023-10-21T16:18:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3915a03640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f38fc004c40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3900007f90 tx=0x7f38fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bde3258a2333a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900005ad0 con 0x7f38fc0047c0 2023-10-21T16:18:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900005c30 con 0x7f38fc0047c0 2023-10-21T16:18:02.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3915202640 1 -- 172.21.15.53:0/3435787562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390002c790 con 0x7f38fc0047c0 2023-10-21T16:18:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3918468640 1 -- 172.21.15.53:0/3435787562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 msgr2=0x7f38fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3918468640 1 --2- 172.21.15.53:0/3435787562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f38fc004c40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3900007f90 tx=0x7f38fc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.553+0000 7f3916a05640 1 -- 172.21.15.53:0/3435787562 reap_dead start 2023-10-21T16:18:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.554+0000 7f3918468640 1 -- 172.21.15.53:0/3435787562 shutdown_connections 2023-10-21T16:18:02.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.554+0000 7f3918468640 1 -- 172.21.15.53:0/3435787562 >> 172.21.15.53:0/3435787562 conn(0x7f39100718a0 msgr2=0x7f3910071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.554+0000 7f3918468640 1 -- 172.21.15.53:0/3435787562 shutdown_connections 2023-10-21T16:18:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.554+0000 7f3918468640 1 -- 172.21.15.53:0/3435787562 wait complete. 2023-10-21T16:18:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.555+0000 7f3918468640 1 Processor -- start 2023-10-21T16:18:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.556+0000 7f3918468640 1 -- start start 2023-10-21T16:18:02.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.556+0000 7f3918468640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3918468640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015ecc0 0x7f39100c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3918468640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39100caad0 0x7f39100caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3918468640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100ca6b0 con 0x7f38fc0047c0 2023-10-21T16:18:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3918468640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100ccc90 con 0x7f39100caad0 2023-10-21T16:18:02.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3918468640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391015f430 con 0x7f391015ecc0 2023-10-21T16:18:02.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33388/0 (socket says 172.21.15.53:33388) 2023-10-21T16:18:02.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 -- 172.21.15.53:0/1002958963 learned_addr learned my addr 172.21.15.53:0/1002958963 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:02.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 -- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015ecc0 msgr2=0x7f39100c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015ecc0 0x7f39100c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 -- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39100caad0 msgr2=0x7f39100caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39100caad0 0x7f39100caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.557+0000 7f3916204640 1 -- 172.21.15.53:0/1002958963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100c9d10 con 0x7f38fc0047c0 2023-10-21T16:18:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3916204640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f390c010890 tx=0x7f390c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c009550 con 0x7f38fc0047c0 2023-10-21T16:18:02.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c00eb00 con 0x7f38fc0047c0 2023-10-21T16:18:02.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00ee90 con 0x7f38fc0047c0 2023-10-21T16:18:02.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39100c9ff0 con 0x7f38fc0047c0 2023-10-21T16:18:02.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.558+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391015f140 con 0x7f38fc0047c0 2023-10-21T16:18:02.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.559+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391004e160 con 0x7f38fc0047c0 2023-10-21T16:18:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.560+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f390c01c400 con 0x7f38fc0047c0 2023-10-21T16:18:02.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.561+0000 7f3906ffd640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 0x7f38f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.561+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f390c0897c0 con 0x7f38fc0047c0 2023-10-21T16:18:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.562+0000 7f3915a03640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 0x7f38f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.563+0000 7f3915a03640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 0x7f38f406d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3900034000 tx=0x7f3900007000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:02.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.566+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f390c054af0 con 0x7f38fc0047c0 2023-10-21T16:18:02.619 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:02.616+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:02.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:02.761+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f39100cf160 con 0x7f38fc0047c0 2023-10-21T16:18:03.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.333+0000 7f3906ffd640 1 -- 172.21.15.53:0/1002958963 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7f390c058110 con 0x7f38fc0047c0 2023-10-21T16:18:03.335 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:18:03.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.337+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 msgr2=0x7f38f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.337+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 0x7f38f406d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3900034000 tx=0x7f3900007000 comp rx=0 tx=0).stop 2023-10-21T16:18:03.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 msgr2=0x7f39100cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f390c010890 tx=0x7f390c010d50 comp rx=0 tx=0).stop 2023-10-21T16:18:03.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 shutdown_connections 2023-10-21T16:18:03.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f38f406af00 0x7f38f406d3c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39100caad0 0x7f39100caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f391015ecc0 0x7f39100c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 --2- 172.21.15.53:0/1002958963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38fc0047c0 0x7f39100cc750 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 >> 172.21.15.53:0/1002958963 conn(0x7f39100718a0 msgr2=0x7f391015b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:03.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 shutdown_connections 2023-10-21T16:18:03.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.338+0000 7f3918468640 1 -- 172.21.15.53:0/1002958963 wait complete. 2023-10-21T16:18:03.367 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:18:03.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.593+0000 7f8f969a5640 1 Processor -- start 2023-10-21T16:18:03.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.594+0000 7f8f969a5640 1 -- start start 2023-10-21T16:18:03.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f969a5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f900cb500 0x7f8f900cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:03.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f969a5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900c8c60 con 0x7f8f90070690 2023-10-21T16:18:03.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f969a5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f90071250 con 0x7f8f9006cfa0 2023-10-21T16:18:03.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f969a5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f900cbe20 con 0x7f8f900cb500 2023-10-21T16:18:03.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f94f42640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8f90070690 0x7f8f90070b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49200/0 (socket says 172.21.15.53:49200) 2023-10-21T16:18:03.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f94f42640 1 -- 172.21.15.53:0/3854698887 learned_addr learned my addr 172.21.15.53:0/3854698887 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:03.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.595+0000 7f8f8ffff640 1 --2- 172.21.15.53:0/3854698887 >> v2:172.21.15.53:3300/0 conn(0x7f8f900cb500 0x7f8f900cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:03.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.596+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/3854698887 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023795882 0 0) 0x7f8f900c8c60 con 0x7f8f90070690 2023-10-21T16:18:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.596+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/3854698887 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c003520 con 0x7f8f90070690 2023-10-21T16:18:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.596+0000 7f8f8ffff640 1 -- 172.21.15.53:0/3854698887 >> v1:172.21.15.124:6789/0 conn(0x7f8f9006cfa0 legacy=0x7f8f9015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.596+0000 7f8f8ffff640 1 -- 172.21.15.53:0/3854698887 >> v1:172.21.15.53:6789/0 conn(0x7f8f90070690 legacy=0x7f8f90070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.596+0000 7f8f8ffff640 1 -- 172.21.15.53:0/3854698887 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900b1ac0 con 0x7f8f900cb500 2023-10-21T16:18:03.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.598+0000 7f8f8ffff640 1 --2- 172.21.15.53:0/3854698887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cb8e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8f800097a0 tx=0x7f8f8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7558e83cea8c193f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:03.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.598+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/3854698887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80031830 con 0x7f8f900cb500 2023-10-21T16:18:03.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.598+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/3854698887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80031990 con 0x7f8f900cb500 2023-10-21T16:18:03.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.598+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/3854698887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80031d00 con 0x7f8f900cb500 2023-10-21T16:18:03.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 -- 172.21.15.53:0/3854698887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 msgr2=0x7f8f900cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 --2- 172.21.15.53:0/3854698887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cb8e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8f800097a0 tx=0x7f8f8002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 -- 172.21.15.53:0/3854698887 shutdown_connections 2023-10-21T16:18:03.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 --2- 172.21.15.53:0/3854698887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cb8e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 -- 172.21.15.53:0/3854698887 >> 172.21.15.53:0/3854698887 conn(0x7f8f9006a140 msgr2=0x7f8f9006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:03.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 -- 172.21.15.53:0/3854698887 shutdown_connections 2023-10-21T16:18:03.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.599+0000 7f8f969a5640 1 -- 172.21.15.53:0/3854698887 wait complete. 2023-10-21T16:18:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.601+0000 7f8f969a5640 1 Processor -- start 2023-10-21T16:18:03.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.601+0000 7f8f969a5640 1 -- start start 2023-10-21T16:18:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9006cfa0 0x7f8f90159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90070690 0x7f8f9015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901f3260 con 0x7f8f900cb500 2023-10-21T16:18:03.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901f2650 con 0x7f8f9006cfa0 2023-10-21T16:18:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f969a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901f2930 con 0x7f8f90070690 2023-10-21T16:18:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f94f42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f94f42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:33402/0 (socket says 172.21.15.53:33402) 2023-10-21T16:18:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.602+0000 7f8f94f42640 1 -- 172.21.15.53:0/2368024210 learned_addr learned my addr 172.21.15.53:0/2368024210 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:03.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f8ffff640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9006cfa0 0x7f8f90159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f877fe640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90070690 0x7f8f9015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 -- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90070690 msgr2=0x7f8f9015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90070690 0x7f8f9015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 -- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9006cfa0 msgr2=0x7f8f90159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9006cfa0 0x7f8f90159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 -- 172.21.15.53:0/2368024210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80009400 con 0x7f8f900cb500 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f94f42640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c390 tx=0x7f8f8800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.603+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88005ce0 con 0x7f8f900cb500 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.604+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88005e40 con 0x7f8f900cb500 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.604+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88007310 con 0x7f8f900cb500 2023-10-21T16:18:03.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.604+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f901f2c10 con 0x7f8f900cb500 2023-10-21T16:18:03.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.604+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9015c3b0 con 0x7f8f900cb500 2023-10-21T16:18:03.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.605+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9006d990 con 0x7f8f900cb500 2023-10-21T16:18:03.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.608+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8f88007470 con 0x7f8f900cb500 2023-10-21T16:18:03.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.609+0000 7f8f8d7fa640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 0x7f8f7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:03.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.609+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f8f88088fc0 con 0x7f8f900cb500 2023-10-21T16:18:03.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.609+0000 7f8f8ffff640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 0x7f8f7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:03.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.611+0000 7f8f8ffff640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 0x7f8f7006d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f80036000 tx=0x7f8f80008030 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.612+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f88054170 con 0x7f8f900cb500 2023-10-21T16:18:03.628 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:03.625+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:03.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:03.807+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8f90071cb0 con 0x7f8f900cb500 2023-10-21T16:18:04.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.353+0000 7f8f8d7fa640 1 -- 172.21.15.53:0/2368024210 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8f88057790 con 0x7f8f900cb500 2023-10-21T16:18:04.355 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:18:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.358+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 msgr2=0x7f8f7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.358+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 0x7f8f7006d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f80036000 tx=0x7f8f80008030 comp rx=0 tx=0).stop 2023-10-21T16:18:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.358+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 msgr2=0x7f8f900cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.358+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c390 tx=0x7f8f8800c850 comp rx=0 tx=0).stop 2023-10-21T16:18:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 shutdown_connections 2023-10-21T16:18:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f7006af00 0x7f8f7006d3c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f900cb500 0x7f8f900cd4e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90070690 0x7f8f9015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 --2- 172.21.15.53:0/2368024210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9006cfa0 0x7f8f90159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.359+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 >> 172.21.15.53:0/2368024210 conn(0x7f8f9006a140 msgr2=0x7f8f900cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:04.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.360+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 shutdown_connections 2023-10-21T16:18:04.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.362+0000 7f8f969a5640 1 -- 172.21.15.53:0/2368024210 wait complete. 2023-10-21T16:18:04.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:18:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.594+0000 7f9719985640 1 Processor -- start 2023-10-21T16:18:04.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.594+0000 7f9719985640 1 -- start start 2023-10-21T16:18:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9719985640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 0x7f97141594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9719985640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9714070ac0 0x7f97140aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9719985640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9714071d90 con 0x7f97140b04b0 2023-10-21T16:18:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9719985640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97140b0900 con 0x7f9714070ac0 2023-10-21T16:18:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9719985640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9714070f20 con 0x7f9714156a10 2023-10-21T16:18:04.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9713fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9714070ac0 0x7f97140aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9713fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9714070ac0 0x7f97140aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53240/0 (socket says 172.21.15.53:53240) 2023-10-21T16:18:04.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.596+0000 7f9713fff640 1 -- 172.21.15.53:0/3773107575 learned_addr learned my addr 172.21.15.53:0/3773107575 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:04.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9713fff640 1 -- 172.21.15.53:0/3773107575 >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 msgr2=0x7f97141594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:04.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9718983640 1 --2- 172.21.15.53:0/3773107575 >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 0x7f97141594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9713fff640 1 --2- 172.21.15.53:0/3773107575 >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 0x7f97141594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9713fff640 1 -- 172.21.15.53:0/3773107575 >> v1:172.21.15.124:6789/0 conn(0x7f97140b04b0 legacy=0x7f971415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:04.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9713fff640 1 -- 172.21.15.53:0/3773107575 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714071180 con 0x7f9714070ac0 2023-10-21T16:18:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.597+0000 7f9718983640 1 --2- 172.21.15.53:0/3773107575 >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 0x7f97141594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.598+0000 7f9713fff640 1 --2- 172.21.15.53:0/3773107575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714070ac0 0x7f97140aff70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f971406e2e0 tx=0x7f970802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e448e44362134 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.598+0000 7f97137fe640 1 -- 172.21.15.53:0/3773107575 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708031870 con 0x7f9714070ac0 2023-10-21T16:18:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.598+0000 7f97137fe640 1 -- 172.21.15.53:0/3773107575 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97080319d0 con 0x7f9714070ac0 2023-10-21T16:18:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.598+0000 7f97137fe640 1 -- 172.21.15.53:0/3773107575 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708031d60 con 0x7f9714070ac0 2023-10-21T16:18:04.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 -- 172.21.15.53:0/3773107575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714070ac0 msgr2=0x7f97140aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:04.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 --2- 172.21.15.53:0/3773107575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714070ac0 0x7f97140aff70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f971406e2e0 tx=0x7f970802b910 comp rx=0 tx=0).stop 2023-10-21T16:18:04.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 -- 172.21.15.53:0/3773107575 shutdown_connections 2023-10-21T16:18:04.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 --2- 172.21.15.53:0/3773107575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714070ac0 0x7f97140aff70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 --2- 172.21.15.53:0/3773107575 >> v2:172.21.15.124:3300/0 conn(0x7f9714156a10 0x7f97141594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 -- 172.21.15.53:0/3773107575 >> 172.21.15.53:0/3773107575 conn(0x7f971406c050 msgr2=0x7f971406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:04.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 -- 172.21.15.53:0/3773107575 shutdown_connections 2023-10-21T16:18:04.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.599+0000 7f9719985640 1 -- 172.21.15.53:0/3773107575 wait complete. 2023-10-21T16:18:04.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.601+0000 7f9719985640 1 Processor -- start 2023-10-21T16:18:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.602+0000 7f9719985640 1 -- start start 2023-10-21T16:18:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9714070ac0 0x7f97141f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97140b04b0 0x7f9714156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141f3f70 con 0x7f9714156a10 2023-10-21T16:18:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97141f3990 con 0x7f97140b04b0 2023-10-21T16:18:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719985640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97141578f0 con 0x7f9714070ac0 2023-10-21T16:18:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719184640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719184640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53254/0 (socket says 172.21.15.53:53254) 2023-10-21T16:18:04.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.603+0000 7f9719184640 1 -- 172.21.15.53:0/1278092597 learned_addr learned my addr 172.21.15.53:0/1278092597 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9713fff640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97140b04b0 0x7f9714156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9718983640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9714070ac0 0x7f97141f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 -- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9714070ac0 msgr2=0x7f97141f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9714070ac0 0x7f97141f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 -- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97140b04b0 msgr2=0x7f9714156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:04.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97140b04b0 0x7f9714156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 -- 172.21.15.53:0/1278092597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080094d0 con 0x7f9714156a10 2023-10-21T16:18:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.604+0000 7f9719184640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b410 tx=0x7f970c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.605+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c004020 con 0x7f9714156a10 2023-10-21T16:18:04.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.605+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c004180 con 0x7f9714156a10 2023-10-21T16:18:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.605+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c011630 con 0x7f9714156a10 2023-10-21T16:18:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.605+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97141f3c00 con 0x7f9714156a10 2023-10-21T16:18:04.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.605+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9714157bd0 con 0x7f9714156a10 2023-10-21T16:18:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.606+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97141f40e0 con 0x7f9714156a10 2023-10-21T16:18:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.607+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f970c011790 con 0x7f9714156a10 2023-10-21T16:18:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.608+0000 7f97117fa640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 0x7f97000717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:04.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.608+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f970c089c30 con 0x7f9714156a10 2023-10-21T16:18:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.609+0000 7f9718983640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 0x7f97000717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.610+0000 7f9718983640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 0x7f97000717d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9704007510 tx=0x7f97040074a0 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:04.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.612+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f970c054c60 con 0x7f9714156a10 2023-10-21T16:18:04.675 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:04.672+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:04.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:04.861+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9714074150 con 0x7f9714156a10 2023-10-21T16:18:05.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.383+0000 7f97117fa640 1 -- 172.21.15.53:0/1278092597 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v41) v1 ==== 174+0+0 (secure 0 0 0) 0x7f970c015020 con 0x7f9714156a10 2023-10-21T16:18:05.387 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 11 and data pool 12 2023-10-21T16:18:05.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.390+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 msgr2=0x7f97000717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.390+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 0x7f97000717d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9704007510 tx=0x7f97040074a0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.390+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 msgr2=0x7f9714142ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.390+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b410 tx=0x7f970c00b8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 shutdown_connections 2023-10-21T16:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f970006f310 0x7f97000717d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9714156a10 0x7f9714142ff0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97140b04b0 0x7f9714156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 --2- 172.21.15.53:0/1278092597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9714070ac0 0x7f97141f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.391+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 >> 172.21.15.53:0/1278092597 conn(0x7f971406c050 msgr2=0x7f971415a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.392+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 shutdown_connections 2023-10-21T16:18:05.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.393+0000 7f9719985640 1 -- 172.21.15.53:0/1278092597 wait complete. 2023-10-21T16:18:05.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:05.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.619+0000 7f4150901640 1 Processor -- start 2023-10-21T16:18:05.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.619+0000 7f4150901640 1 -- start start 2023-10-21T16:18:05.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f4150901640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f414c15ab40 0x7f414c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f4150901640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f414c0c90b0 0x7f414c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f4150901640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414c15b2b0 con 0x7f414c0cae70 2023-10-21T16:18:05.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f4150901640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c15bf30 con 0x7f414c15ab40 2023-10-21T16:18:05.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f4150901640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414c15f7a0 con 0x7f414c0c90b0 2023-10-21T16:18:05.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f414a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f414c15ab40 0x7f414c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:05.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f414a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f414c15ab40 0x7f414c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53262/0 (socket says 172.21.15.53:53262) 2023-10-21T16:18:05.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.621+0000 7f414a59c640 1 -- 172.21.15.53:0/1532011700 learned_addr learned my addr 172.21.15.53:0/1532011700 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.622+0000 7f414a59c640 1 -- 172.21.15.53:0/1532011700 >> v2:172.21.15.124:3300/0 conn(0x7f414c0c90b0 msgr2=0x7f414c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.622+0000 7f414a59c640 1 --2- 172.21.15.53:0/1532011700 >> v2:172.21.15.124:3300/0 conn(0x7f414c0c90b0 0x7f414c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.622+0000 7f414a59c640 1 -- 172.21.15.53:0/1532011700 >> v1:172.21.15.124:6789/0 conn(0x7f414c0cae70 legacy=0x7f414c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.622+0000 7f414a59c640 1 -- 172.21.15.53:0/1532011700 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c15b590 con 0x7f414c15ab40 2023-10-21T16:18:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.623+0000 7f414a59c640 1 --2- 172.21.15.53:0/1532011700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0ad720 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f414c06e560 tx=0x7f413c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a166c6ca3f78b55 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.623+0000 7f414959a640 1 -- 172.21.15.53:0/1532011700 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c02ca80 con 0x7f414c15ab40 2023-10-21T16:18:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.623+0000 7f414959a640 1 -- 172.21.15.53:0/1532011700 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c02cbe0 con 0x7f414c15ab40 2023-10-21T16:18:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.623+0000 7f414959a640 1 -- 172.21.15.53:0/1532011700 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0343f0 con 0x7f414c15ab40 2023-10-21T16:18:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 -- 172.21.15.53:0/1532011700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 msgr2=0x7f414c0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 --2- 172.21.15.53:0/1532011700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0ad720 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f414c06e560 tx=0x7f413c02bee0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 -- 172.21.15.53:0/1532011700 shutdown_connections 2023-10-21T16:18:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 --2- 172.21.15.53:0/1532011700 >> v2:172.21.15.124:3300/0 conn(0x7f414c0c90b0 0x7f414c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 --2- 172.21.15.53:0/1532011700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0ad720 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 -- 172.21.15.53:0/1532011700 >> 172.21.15.53:0/1532011700 conn(0x7f414c06a380 msgr2=0x7f414c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 -- 172.21.15.53:0/1532011700 shutdown_connections 2023-10-21T16:18:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.624+0000 7f4150901640 1 -- 172.21.15.53:0/1532011700 wait complete. 2023-10-21T16:18:05.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.625+0000 7f4150901640 1 Processor -- start 2023-10-21T16:18:05.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.626+0000 7f4150901640 1 -- start start 2023-10-21T16:18:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.626+0000 7f4150901640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 0x7f414c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.626+0000 7f4150901640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 0x7f414c0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4150901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4150901640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15d510 con 0x7f414c15ab40 2023-10-21T16:18:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4150901640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c14e750 con 0x7f414c0c90b0 2023-10-21T16:18:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4150901640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c15c900 con 0x7f414c0cae70 2023-10-21T16:18:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53276/0 (socket says 172.21.15.53:53276) 2023-10-21T16:18:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 -- 172.21.15.53:0/2204577717 learned_addr learned my addr 172.21.15.53:0/2204577717 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4149d9b640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 0x7f414c0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414a59c640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 0x7f414c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 -- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 msgr2=0x7f414c0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 0x7f414c0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 -- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 msgr2=0x7f414c14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 0x7f414c14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414ad9d640 1 -- 172.21.15.53:0/2204577717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c009b30 con 0x7f414c15ab40 2023-10-21T16:18:05.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f4149d9b640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 0x7f414c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:05.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.627+0000 7f414a59c640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 0x7f414c14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:05.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f414ad9d640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f414400c3c0 tx=0x7f414400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:05.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440035f0 con 0x7f414c15ab40 2023-10-21T16:18:05.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144003750 con 0x7f414c15ab40 2023-10-21T16:18:05.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144003b00 con 0x7f414c15ab40 2023-10-21T16:18:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414c15cbe0 con 0x7f414c15ab40 2023-10-21T16:18:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.628+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414c15d1a0 con 0x7f414c15ab40 2023-10-21T16:18:05.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.630+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414c15d6c0 con 0x7f414c15ab40 2023-10-21T16:18:05.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.631+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4144003c60 con 0x7f414c15ab40 2023-10-21T16:18:05.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.631+0000 7f413affd640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 0x7f412c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:05.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.632+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4144088cd0 con 0x7f414c15ab40 2023-10-21T16:18:05.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.632+0000 7f414a59c640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 0x7f412c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:05.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.634+0000 7f414a59c640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 0x7f412c06d410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f413c035000 tx=0x7f413c030000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:05.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.636+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f414c15d6c0 con 0x7f414c15ab40 2023-10-21T16:18:05.715 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:05.712+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:05.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.831+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f414c0cd090 con 0x7f414c15ab40 2023-10-21T16:18:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.834+0000 7f413affd640 1 -- 172.21.15.53:0/2204577717 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f4144017070 con 0x7f414c15ab40 2023-10-21T16:18:05.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:05.836 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":102,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:04.863070+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:05.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 msgr2=0x7f412c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 0x7f412c06d410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f413c035000 tx=0x7f413c030000 comp rx=0 tx=0).stop 2023-10-21T16:18:05.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 msgr2=0x7f414c0c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:05.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f414400c3c0 tx=0x7f414400c880 comp rx=0 tx=0).stop 2023-10-21T16:18:05.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 shutdown_connections 2023-10-21T16:18:05.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f412c06af50 0x7f412c06d410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f414c15ab40 0x7f414c0c5d90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f414c0cae70 0x7f414c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 --2- 172.21.15.53:0/2204577717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f414c0c90b0 0x7f414c14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:05.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.836+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 >> 172.21.15.53:0/2204577717 conn(0x7f414c06a380 msgr2=0x7f414c0aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:05.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.837+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 shutdown_connections 2023-10-21T16:18:05.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:05.837+0000 7f4150901640 1 -- 172.21.15.53:0/2204577717 wait complete. 2023-10-21T16:18:05.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:06.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.077+0000 7f5a501ee640 1 Processor -- start 2023-10-21T16:18:06.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.077+0000 7f5a501ee640 1 -- start start 2023-10-21T16:18:06.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.079+0000 7f5a501ee640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5a480ad790 0x7f5a480702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.079+0000 7f5a501ee640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 0x7f5a48070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.079+0000 7f5a501ee640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4806dee0 con 0x7f5a48159ac0 2023-10-21T16:18:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.079+0000 7f5a501ee640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4806dcf0 con 0x7f5a480ad790 2023-10-21T16:18:06.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.079+0000 7f5a501ee640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815d850 con 0x7f5a4815e330 2023-10-21T16:18:06.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.080+0000 7f5a4e78b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 0x7f5a48070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.080+0000 7f5a4df8a640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5a48159ac0 0x7f5a4806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:54424/0 (socket says 172.21.15.53:54424) 2023-10-21T16:18:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.080+0000 7f5a4df8a640 1 -- 172.21.15.53:0/3912392913 learned_addr learned my addr 172.21.15.53:0/3912392913 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.080+0000 7f5a4d789640 1 --2- 172.21.15.53:0/3912392913 >> v2:172.21.15.124:3300/0 conn(0x7f5a480ad790 0x7f5a480702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.081+0000 7f5a4e78b640 1 -- 172.21.15.53:0/3912392913 >> v2:172.21.15.124:3300/0 conn(0x7f5a480ad790 msgr2=0x7f5a480702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:06.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.081+0000 7f5a4e78b640 1 --2- 172.21.15.53:0/3912392913 >> v2:172.21.15.124:3300/0 conn(0x7f5a480ad790 0x7f5a480702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.081+0000 7f5a4cf88640 1 -- 172.21.15.53:0/3912392913 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008366768 0 0) 0x7f5a4806dee0 con 0x7f5a48159ac0 2023-10-21T16:18:06.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.081+0000 7f5a4e78b640 1 -- 172.21.15.53:0/3912392913 >> v1:172.21.15.124:6789/0 conn(0x7f5a48159ac0 legacy=0x7f5a4806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.081+0000 7f5a4e78b640 1 -- 172.21.15.53:0/3912392913 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4815db30 con 0x7f5a4815e330 2023-10-21T16:18:06.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.082+0000 7f5a4e78b640 1 --2- 172.21.15.53:0/3912392913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 0x7f5a48070810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009d90 tx=0x7f5a4002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d0697012369bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.082+0000 7f5a4cf88640 1 -- 172.21.15.53:0/3912392913 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4002cc50 con 0x7f5a4815e330 2023-10-21T16:18:06.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a4cf88640 1 -- 172.21.15.53:0/3912392913 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4002cdb0 con 0x7f5a4815e330 2023-10-21T16:18:06.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a4cf88640 1 -- 172.21.15.53:0/3912392913 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40034660 con 0x7f5a4815e330 2023-10-21T16:18:06.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 -- 172.21.15.53:0/3912392913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 msgr2=0x7f5a48070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 --2- 172.21.15.53:0/3912392913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 0x7f5a48070810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009d90 tx=0x7f5a4002bea0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 -- 172.21.15.53:0/3912392913 shutdown_connections 2023-10-21T16:18:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 --2- 172.21.15.53:0/3912392913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815e330 0x7f5a48070810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 --2- 172.21.15.53:0/3912392913 >> v2:172.21.15.124:3300/0 conn(0x7f5a480ad790 0x7f5a480702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 -- 172.21.15.53:0/3912392913 >> 172.21.15.53:0/3912392913 conn(0x7f5a4806a380 msgr2=0x7f5a480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 -- 172.21.15.53:0/3912392913 shutdown_connections 2023-10-21T16:18:06.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.083+0000 7f5a501ee640 1 -- 172.21.15.53:0/3912392913 wait complete. 2023-10-21T16:18:06.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.085+0000 7f5a501ee640 1 Processor -- start 2023-10-21T16:18:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.085+0000 7f5a501ee640 1 -- start start 2023-10-21T16:18:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480ad790 0x7f5a48151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4815e330 0x7f5a480ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480ccfe0 con 0x7f5a48159ac0 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cc720 con 0x7f5a4815e330 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a501ee640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480cc990 con 0x7f5a480ad790 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.086+0000 7f5a4d789640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53282/0 (socket says 172.21.15.53:53282) 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 -- 172.21.15.53:0/2725738847 learned_addr learned my addr 172.21.15.53:0/2725738847 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:06.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4df8a640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480ad790 0x7f5a48151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4e78b640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4815e330 0x7f5a480ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 -- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480ad790 msgr2=0x7f5a48151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480ad790 0x7f5a48151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 -- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4815e330 msgr2=0x7f5a480ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4815e330 0x7f5a480ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.087+0000 7f5a4d789640 1 -- 172.21.15.53:0/2725738847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009a70 con 0x7f5a48159ac0 2023-10-21T16:18:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a4d789640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9b0 tx=0x7f5a3800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38009670 con 0x7f5a48159ac0 2023-10-21T16:18:06.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800c9a0 con 0x7f5a48159ac0 2023-10-21T16:18:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800cd50 con 0x7f5a48159ac0 2023-10-21T16:18:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a480ccc70 con 0x7f5a48159ac0 2023-10-21T16:18:06.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.088+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48151b70 con 0x7f5a48159ac0 2023-10-21T16:18:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.090+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a480cd190 con 0x7f5a48159ac0 2023-10-21T16:18:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.090+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5a38013470 con 0x7f5a48159ac0 2023-10-21T16:18:06.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.092+0000 7f5a3e7fc640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.092+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a38089a70 con 0x7f5a48159ac0 2023-10-21T16:18:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.093+0000 7f5a4df8a640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.094+0000 7f5a4df8a640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000f60 tx=0x7f5a44007b50 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.096+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a480cd190 con 0x7f5a48159ac0 2023-10-21T16:18:06.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.297+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a480c8bb0 con 0x7f5a48159ac0 2023-10-21T16:18:06.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.300+0000 7f5a3e7fc640 1 -- 172.21.15.53:0/2725738847 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f5a38054940 con 0x7f5a48159ac0 2023-10-21T16:18:06.302 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:06.302 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":102,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:04.863070+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.302+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 msgr2=0x7f5a2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.302+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a44000f60 tx=0x7f5a44007b50 comp rx=0 tx=0).stop 2023-10-21T16:18:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 msgr2=0x7f5a480cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9b0 tx=0x7f5a3800ee70 comp rx=0 tx=0).stop 2023-10-21T16:18:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 shutdown_connections 2023-10-21T16:18:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f5a2c06af00 0x7f5a2c06d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4815e330 0x7f5a480ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a48159ac0 0x7f5a480cdb60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 --2- 172.21.15.53:0/2725738847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a480ad790 0x7f5a48151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 >> 172.21.15.53:0/2725738847 conn(0x7f5a4806a380 msgr2=0x7f5a4806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 shutdown_connections 2023-10-21T16:18:06.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.303+0000 7f5a501ee640 1 -- 172.21.15.53:0/2725738847 wait complete. 2023-10-21T16:18:06.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:18:06.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-21T16:18:06.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-21T16:18:06.334 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-10-21T16:18:06.329+0000 7fa0d78d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:18:06.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.545+0000 7f188a472640 1 Processor -- start 2023-10-21T16:18:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.545+0000 7f188a472640 1 -- start start 2023-10-21T16:18:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f188a472640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f188415ccd0 0x7f18840ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f188a472640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f188a472640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18840ce9b0 con 0x7f18841612e0 2023-10-21T16:18:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f188a472640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18840cec90 con 0x7f188415ccd0 2023-10-21T16:18:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f188a472640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18840cef70 con 0x7f188406bcb0 2023-10-21T16:18:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f1889470640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f188415ccd0 0x7f18840ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f1889470640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f188415ccd0 0x7f18840ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53284/0 (socket says 172.21.15.53:53284) 2023-10-21T16:18:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.547+0000 7f1889470640 1 -- 172.21.15.53:0/1807225937 learned_addr learned my addr 172.21.15.53:0/1807225937 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:06.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.548+0000 7f1888c6f640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.548+0000 7f1889470640 1 -- 172.21.15.53:0/1807225937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 msgr2=0x7f1884160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.548+0000 7f1889470640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.548+0000 7f1889470640 1 -- 172.21.15.53:0/1807225937 >> v1:172.21.15.124:6789/0 conn(0x7f18841612e0 legacy=0x7f18840b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.548+0000 7f1889470640 1 -- 172.21.15.53:0/1807225937 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884160880 con 0x7f188415ccd0 2023-10-21T16:18:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.549+0000 7f1889470640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ce2d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1880009870 tx=0x7f188002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a067d56398704 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.550+0000 7f187bfff640 1 -- 172.21.15.53:0/1807225937 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880031640 con 0x7f188415ccd0 2023-10-21T16:18:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.550+0000 7f187bfff640 1 -- 172.21.15.53:0/1807225937 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18800317a0 con 0x7f188415ccd0 2023-10-21T16:18:06.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.550+0000 7f187bfff640 1 -- 172.21.15.53:0/1807225937 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880031b30 con 0x7f188415ccd0 2023-10-21T16:18:06.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.550+0000 7f188a472640 1 -- 172.21.15.53:0/1807225937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 msgr2=0x7f18840ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.550+0000 7f188a472640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ce2d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1880009870 tx=0x7f188002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 -- 172.21.15.53:0/1807225937 shutdown_connections 2023-10-21T16:18:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 --2- 172.21.15.53:0/1807225937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ce2d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 -- 172.21.15.53:0/1807225937 >> 172.21.15.53:0/1807225937 conn(0x7f18840cd180 msgr2=0x7f18840cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:06.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 -- 172.21.15.53:0/1807225937 shutdown_connections 2023-10-21T16:18:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.551+0000 7f188a472640 1 -- 172.21.15.53:0/1807225937 wait complete. 2023-10-21T16:18:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.553+0000 7f188a472640 1 Processor -- start 2023-10-21T16:18:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.553+0000 7f188a472640 1 -- start start 2023-10-21T16:18:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18841612e0 0x7f188415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1884147950 con 0x7f188415ccd0 2023-10-21T16:18:06.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841559e0 con 0x7f18841612e0 2023-10-21T16:18:06.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f188a472640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1884144560 con 0x7f188406bcb0 2023-10-21T16:18:06.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f1888c6f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f1888c6f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53296/0 (socket says 172.21.15.53:53296) 2023-10-21T16:18:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f1888c6f640 1 -- 172.21.15.53:0/2302158289 learned_addr learned my addr 172.21.15.53:0/2302158289 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f1889470640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.554+0000 7f1889c71640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18841612e0 0x7f188415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 -- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 msgr2=0x7f1884144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 -- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18841612e0 msgr2=0x7f188415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18841612e0 0x7f188415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 -- 172.21.15.53:0/2302158289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18800094d0 con 0x7f188415ccd0 2023-10-21T16:18:06.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1888c6f640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f187400d830 tx=0x7f187400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400b6c0 con 0x7f188415ccd0 2023-10-21T16:18:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187400b820 con 0x7f188415ccd0 2023-10-21T16:18:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400bbb0 con 0x7f188415ccd0 2023-10-21T16:18:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.555+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18841f5fb0 con 0x7f188415ccd0 2023-10-21T16:18:06.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.556+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18841f6570 con 0x7f188415ccd0 2023-10-21T16:18:06.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.557+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1884074150 con 0x7f188415ccd0 2023-10-21T16:18:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.558+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f187401e460 con 0x7f188415ccd0 2023-10-21T16:18:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.559+0000 7f1879ffb640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 0x7f186806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.559+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1874089f60 con 0x7f188415ccd0 2023-10-21T16:18:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.560+0000 7f1889470640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 0x7f186806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:06.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.561+0000 7f1889470640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 0x7f186806d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1880036000 tx=0x7f188003c040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:06.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.564+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1874054e40 con 0x7f188415ccd0 2023-10-21T16:18:06.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:06.739+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:06.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:06.802+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18840cdf40 con 0x7f188415ccd0 2023-10-21T16:18:07.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.373+0000 7f1879ffb640 1 -- 172.21.15.53:0/2302158289 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1874058460 con 0x7f188415ccd0 2023-10-21T16:18:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 msgr2=0x7f186806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 0x7f186806d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1880036000 tx=0x7f188003c040 comp rx=0 tx=0).stop 2023-10-21T16:18:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 msgr2=0x7f18840ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f187400d830 tx=0x7f187400dcf0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 shutdown_connections 2023-10-21T16:18:07.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f186806af00 0x7f186806d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18841612e0 0x7f188415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f188415ccd0 0x7f18840ca080 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 --2- 172.21.15.53:0/2302158289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188406bcb0 0x7f1884144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 >> 172.21.15.53:0/2302158289 conn(0x7f18840cd180 msgr2=0x7f18840b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:07.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 shutdown_connections 2023-10-21T16:18:07.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.377+0000 7f188a472640 1 -- 172.21.15.53:0/2302158289 wait complete. 2023-10-21T16:18:07.402 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:18:07.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:18:07.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.634+0000 7f0ed796c640 1 Processor -- start 2023-10-21T16:18:07.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.634+0000 7f0ed796c640 1 -- start start 2023-10-21T16:18:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006c3a0 0x7f0ed00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ed006db60 0x7f0ed006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0ed015aa00 0x7f0ed00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed00ca230 con 0x7f0ed006db60 2023-10-21T16:18:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed015b310 con 0x7f0ed015aa00 2023-10-21T16:18:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed796c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed006d290 con 0x7f0ed006c3a0 2023-10-21T16:18:07.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed4f07640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ed006db60 0x7f0ed006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed4f07640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ed006db60 0x7f0ed006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53300/0 (socket says 172.21.15.53:53300) 2023-10-21T16:18:07.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed4f07640 1 -- 172.21.15.53:0/3373670979 learned_addr learned my addr 172.21.15.53:0/3373670979 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:07.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed5708640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006c3a0 0x7f0ed00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.636+0000 7f0ed5f09640 1 --2- 172.21.15.53:0/3373670979 >> v2:172.21.15.124:3300/0 conn(0x7f0ed015aa00 0x7f0ed00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.637+0000 7f0ed4f07640 1 -- 172.21.15.53:0/3373670979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006c3a0 msgr2=0x7f0ed00c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:07.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.637+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006c3a0 0x7f0ed00c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.637+0000 7f0ed4f07640 1 -- 172.21.15.53:0/3373670979 >> v2:172.21.15.124:3300/0 conn(0x7f0ed015aa00 msgr2=0x7f0ed00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:07.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.637+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/3373670979 >> v2:172.21.15.124:3300/0 conn(0x7f0ed015aa00 0x7f0ed00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.637+0000 7f0ed4f07640 1 -- 172.21.15.53:0/3373670979 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00c9860 con 0x7f0ed006db60 2023-10-21T16:18:07.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.638+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006db60 0x7f0ed006c780 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0070240 tx=0x7f0ec002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa5046716f5a520 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:07.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.639+0000 7f0ec7fff640 1 -- 172.21.15.53:0/3373670979 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0031870 con 0x7f0ed006db60 2023-10-21T16:18:07.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.639+0000 7f0ec7fff640 1 -- 172.21.15.53:0/3373670979 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00319d0 con 0x7f0ed006db60 2023-10-21T16:18:07.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.639+0000 7f0ec7fff640 1 -- 172.21.15.53:0/3373670979 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0031d60 con 0x7f0ed006db60 2023-10-21T16:18:07.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.639+0000 7f0ed796c640 1 -- 172.21.15.53:0/3373670979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006db60 msgr2=0x7f0ed006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.639+0000 7f0ed796c640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006db60 0x7f0ed006c780 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0070240 tx=0x7f0ec002b910 comp rx=0 tx=0).stop 2023-10-21T16:18:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 -- 172.21.15.53:0/3373670979 shutdown_connections 2023-10-21T16:18:07.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 --2- 172.21.15.53:0/3373670979 >> v2:172.21.15.124:3300/0 conn(0x7f0ed015aa00 0x7f0ed00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006db60 0x7f0ed006c780 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 --2- 172.21.15.53:0/3373670979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006c3a0 0x7f0ed00c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 -- 172.21.15.53:0/3373670979 >> 172.21.15.53:0/3373670979 conn(0x7f0ed006a5d0 msgr2=0x7f0ed00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 -- 172.21.15.53:0/3373670979 shutdown_connections 2023-10-21T16:18:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.640+0000 7f0ed796c640 1 -- 172.21.15.53:0/3373670979 wait complete. 2023-10-21T16:18:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.642+0000 7f0ed796c640 1 Processor -- start 2023-10-21T16:18:07.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.642+0000 7f0ed796c640 1 -- start start 2023-10-21T16:18:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006db60 0x7f0ed014d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 0x7f0ed014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed014c7b0 con 0x7f0ed006c3a0 2023-10-21T16:18:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed014cd70 con 0x7f0ed015aa00 2023-10-21T16:18:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed796c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0151870 con 0x7f0ed006db60 2023-10-21T16:18:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed5708640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed5708640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53302/0 (socket says 172.21.15.53:53302) 2023-10-21T16:18:07.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.643+0000 7f0ed5708640 1 -- 172.21.15.53:0/1719086886 learned_addr learned my addr 172.21.15.53:0/1719086886 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006db60 0x7f0ed014d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5f09640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 0x7f0ed014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 -- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006db60 msgr2=0x7f0ed014d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006db60 0x7f0ed014d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 -- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 msgr2=0x7f0ed014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:07.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 0x7f0ed014ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 -- 172.21.15.53:0/1719086886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00094d0 con 0x7f0ed006c3a0 2023-10-21T16:18:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5f09640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 0x7f0ed014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed5708640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e8c0 tx=0x7f0ecc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00c940 con 0x7f0ed006c3a0 2023-10-21T16:18:07.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc00caa0 con 0x7f0ed006c3a0 2023-10-21T16:18:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed0151b40 con 0x7f0ed006c3a0 2023-10-21T16:18:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.644+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00ce30 con 0x7f0ed006c3a0 2023-10-21T16:18:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.645+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed014ca00 con 0x7f0ed006c3a0 2023-10-21T16:18:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.646+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed006cca0 con 0x7f0ed006c3a0 2023-10-21T16:18:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.647+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0ecc013470 con 0x7f0ed006c3a0 2023-10-21T16:18:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.648+0000 7f0ec5ffb640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 0x7f0eb406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.648+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ecc089a50 con 0x7f0ed006c3a0 2023-10-21T16:18:07.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.649+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 0x7f0eb406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:07.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.650+0000 7f0ed4f07640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 0x7f0eb406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0036000 tx=0x7f0ec003c040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:07.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.652+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ed006cca0 con 0x7f0ed006c3a0 2023-10-21T16:18:07.784 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:07.780+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:07.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:07.894+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0ed006cca0 con 0x7f0ed006c3a0 2023-10-21T16:18:08.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.394+0000 7f0ec5ffb640 1 -- 172.21.15.53:0/1719086886 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ed006cca0 con 0x7f0ed006c3a0 2023-10-21T16:18:08.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 msgr2=0x7f0eb406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 0x7f0eb406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0036000 tx=0x7f0ec003c040 comp rx=0 tx=0).stop 2023-10-21T16:18:08.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 msgr2=0x7f0ed0151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e8c0 tx=0x7f0ecc00ed80 comp rx=0 tx=0).stop 2023-10-21T16:18:08.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 shutdown_connections 2023-10-21T16:18:08.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f0eb406af00 0x7f0eb406d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ed015aa00 0x7f0ed014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ed006db60 0x7f0ed014d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 --2- 172.21.15.53:0/1719086886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ed006c3a0 0x7f0ed0151330 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.398+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 >> 172.21.15.53:0/1719086886 conn(0x7f0ed006a5d0 msgr2=0x7f0ed0159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:08.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.399+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 shutdown_connections 2023-10-21T16:18:08.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.399+0000 7f0ed796c640 1 -- 172.21.15.53:0/1719086886 wait complete. 2023-10-21T16:18:08.423 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:18:08.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:08.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.654+0000 7f05922a2640 1 Processor -- start 2023-10-21T16:18:08.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.655+0000 7f05922a2640 1 -- start start 2023-10-21T16:18:08.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f05922a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 0x7f058c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f05922a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058c0b1840 0x7f058c0d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f05922a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c0b8bf0 con 0x7f058c0b1d90 2023-10-21T16:18:08.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f05922a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c0d2910 con 0x7f058c0b1840 2023-10-21T16:18:08.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f05922a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0ce1f0 con 0x7f058c0cd8d0 2023-10-21T16:18:08.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.656+0000 7f0583fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058c0b1840 0x7f058c0d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f058c0b1840 0x7f058c0d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53306/0 (socket says 172.21.15.53:53306) 2023-10-21T16:18:08.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 -- 172.21.15.53:0/1293968593 learned_addr learned my addr 172.21.15.53:0/1293968593 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:08.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f059083f640 1 --1- 172.21.15.53:0/1293968593 >> v1:172.21.15.53:6789/0 conn(0x7f058c0b1d90 0x7f058c0d30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60134/0 (socket says 172.21.15.53:60134) 2023-10-21T16:18:08.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f058bfff640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 0x7f058c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 -- 172.21.15.53:0/1293968593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 msgr2=0x7f058c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:08.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 0x7f058c0cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 -- 172.21.15.53:0/1293968593 >> v1:172.21.15.53:6789/0 conn(0x7f058c0b1d90 legacy=0x7f058c0d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f0583fff640 1 -- 172.21.15.53:0/1293968593 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0b8630 con 0x7f058c0b1840 2023-10-21T16:18:08.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.657+0000 7f058bfff640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 0x7f058c0cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:08.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.658+0000 7f0583fff640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0b1840 0x7f058c0d2b90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0574009870 tx=0x7f057402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119a4404076608dd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:08.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f058b7fe640 1 -- 172.21.15.53:0/1293968593 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574030840 con 0x7f058c0b1840 2023-10-21T16:18:08.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f058b7fe640 1 -- 172.21.15.53:0/1293968593 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05740309a0 con 0x7f058c0b1840 2023-10-21T16:18:08.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f058b7fe640 1 -- 172.21.15.53:0/1293968593 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574030d30 con 0x7f058c0b1840 2023-10-21T16:18:08.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 -- 172.21.15.53:0/1293968593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0b1840 msgr2=0x7f058c0d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0b1840 0x7f058c0d2b90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0574009870 tx=0x7f057402b980 comp rx=0 tx=0).stop 2023-10-21T16:18:08.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 -- 172.21.15.53:0/1293968593 shutdown_connections 2023-10-21T16:18:08.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0b1840 0x7f058c0d2b90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 --2- 172.21.15.53:0/1293968593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0cd8d0 0x7f058c0cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 -- 172.21.15.53:0/1293968593 >> 172.21.15.53:0/1293968593 conn(0x7f058c06b6a0 msgr2=0x7f058c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:08.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.659+0000 7f05922a2640 1 -- 172.21.15.53:0/1293968593 shutdown_connections 2023-10-21T16:18:08.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.660+0000 7f05922a2640 1 -- 172.21.15.53:0/1293968593 wait complete. 2023-10-21T16:18:08.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.661+0000 7f05922a2640 1 Processor -- start 2023-10-21T16:18:08.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.661+0000 7f05922a2640 1 -- start start 2023-10-21T16:18:08.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.662+0000 7f05922a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0b1840 0x7f058c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.662+0000 7f05922a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f058c0b1d90 0x7f058c1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.662+0000 7f05922a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f05922a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c1598d0 con 0x7f058c0cd8d0 2023-10-21T16:18:08.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f05922a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0c91c0 con 0x7f058c0b1d90 2023-10-21T16:18:08.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f05922a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c158c50 con 0x7f058c0b1840 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53310/0 (socket says 172.21.15.53:53310) 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 -- 172.21.15.53:0/564893848 learned_addr learned my addr 172.21.15.53:0/564893848 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f058bfff640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0b1840 0x7f058c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f0583fff640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f058c0b1d90 0x7f058c1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 -- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0b1840 msgr2=0x7f058c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0b1840 0x7f058c0c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 -- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f058c0b1d90 msgr2=0x7f058c1f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f058c0b1d90 0x7f058c1f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.663+0000 7f059083f640 1 -- 172.21.15.53:0/564893848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740094d0 con 0x7f058c0cd8d0 2023-10-21T16:18:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f059083f640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f058400c7b0 tx=0x7f058400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400a640 con 0x7f058c0cd8d0 2023-10-21T16:18:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400a7a0 con 0x7f058c0cd8d0 2023-10-21T16:18:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c158f30 con 0x7f058c0cd8d0 2023-10-21T16:18:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058400ab10 con 0x7f058c0cd8d0 2023-10-21T16:18:08.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.664+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c1594f0 con 0x7f058c0cd8d0 2023-10-21T16:18:08.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.665+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c159a10 con 0x7f058c0cd8d0 2023-10-21T16:18:08.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.666+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f058400ac70 con 0x7f058c0cd8d0 2023-10-21T16:18:08.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.667+0000 7f05897fa640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 0x7f056c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:08.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.667+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0584088e80 con 0x7f058c0cd8d0 2023-10-21T16:18:08.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.668+0000 7f058bfff640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 0x7f056c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:08.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.669+0000 7f058bfff640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 0x7f056c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a990 tx=0x7f057c008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:08.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.672+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f058c159a10 con 0x7f058c0cd8d0 2023-10-21T16:18:08.854 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:08.851+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:08.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.911+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f058c159a10 con 0x7f058c0cd8d0 2023-10-21T16:18:08.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.913+0000 7f05897fa640 1 -- 172.21.15.53:0/564893848 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f058c159a10 con 0x7f058c0cd8d0 2023-10-21T16:18:08.916 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:08.916 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:08.916 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:08.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.915+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 msgr2=0x7f056c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.915+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 0x7f056c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a990 tx=0x7f057c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.915+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 msgr2=0x7f058c0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.915+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f058400c7b0 tx=0x7f058400cc70 comp rx=0 tx=0).stop 2023-10-21T16:18:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 shutdown_connections 2023-10-21T16:18:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f056c06af00 0x7f056c06d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f058c0cd8d0 0x7f058c0cc320 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f058c0b1d90 0x7f058c1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 --2- 172.21.15.53:0/564893848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f058c0b1840 0x7f058c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:08.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 >> 172.21.15.53:0/564893848 conn(0x7f058c06b6a0 msgr2=0x7f058c0b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:08.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 shutdown_connections 2023-10-21T16:18:08.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:08.916+0000 7f05922a2640 1 -- 172.21.15.53:0/564893848 wait complete. 2023-10-21T16:18:08.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:09.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.162+0000 7f1d22573640 1 Processor -- start 2023-10-21T16:18:09.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.162+0000 7f1d22573640 1 -- start start 2023-10-21T16:18:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.163+0000 7f1d22573640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.163+0000 7f1d22573640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c15f7d0 con 0x7f1d1c15af10 2023-10-21T16:18:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.163+0000 7f1d22573640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c15efb0 con 0x7f1d1c15ab30 2023-10-21T16:18:09.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.163+0000 7f1d22573640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c15f210 con 0x7f1d1c0adc70 2023-10-21T16:18:09.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53324/0 (socket says 172.21.15.53:53324) 2023-10-21T16:18:09.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 -- 172.21.15.53:0/2850154105 learned_addr learned my addr 172.21.15.53:0/2850154105 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:09.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d21d72640 1 --1- 172.21.15.53:0/2850154105 >> v1:172.21.15.53:6789/0 conn(0x7f1d1c15af10 0x7f1d1c06cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60138/0 (socket says 172.21.15.53:60138) 2023-10-21T16:18:09.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 -- 172.21.15.53:0/2850154105 >> v1:172.21.15.124:6789/0 conn(0x7f1d1c15ab30 legacy=0x7f1d1c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 -- 172.21.15.53:0/2850154105 >> v1:172.21.15.53:6789/0 conn(0x7f1d1c15af10 legacy=0x7f1d1c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.164+0000 7f1d20d70640 1 -- 172.21.15.53:0/2850154105 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c15f4f0 con 0x7f1d1c0adc70 2023-10-21T16:18:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d20d70640 1 --2- 172.21.15.53:0/2850154105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009e50 tx=0x7f1d1802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22654dff4832af4a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d13fff640 1 -- 172.21.15.53:0/2850154105 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1802cbb0 con 0x7f1d1c0adc70 2023-10-21T16:18:09.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d13fff640 1 -- 172.21.15.53:0/2850154105 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1802cd10 con 0x7f1d1c0adc70 2023-10-21T16:18:09.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d13fff640 1 -- 172.21.15.53:0/2850154105 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18034610 con 0x7f1d1c0adc70 2023-10-21T16:18:09.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d22573640 1 -- 172.21.15.53:0/2850154105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c0adc70 msgr2=0x7f1d1c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.166+0000 7f1d22573640 1 --2- 172.21.15.53:0/2850154105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009e50 tx=0x7f1d1802c010 comp rx=0 tx=0).stop 2023-10-21T16:18:09.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.167+0000 7f1d22573640 1 -- 172.21.15.53:0/2850154105 shutdown_connections 2023-10-21T16:18:09.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.167+0000 7f1d22573640 1 --2- 172.21.15.53:0/2850154105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c0adc70 0x7f1d1c0b00c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.167+0000 7f1d22573640 1 -- 172.21.15.53:0/2850154105 >> 172.21.15.53:0/2850154105 conn(0x7f1d1c0cb0c0 msgr2=0x7f1d1c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:09.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.167+0000 7f1d22573640 1 -- 172.21.15.53:0/2850154105 shutdown_connections 2023-10-21T16:18:09.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.167+0000 7f1d22573640 1 -- 172.21.15.53:0/2850154105 wait complete. 2023-10-21T16:18:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.168+0000 7f1d22573640 1 Processor -- start 2023-10-21T16:18:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.169+0000 7f1d22573640 1 -- start start 2023-10-21T16:18:09.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.169+0000 7f1d22573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 0x7f1d1c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.169+0000 7f1d22573640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d1c15ab30 0x7f1d1c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.169+0000 7f1d22573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d22573640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c15dc90 con 0x7f1d1c15af10 2023-10-21T16:18:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d22573640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1f4870 con 0x7f1d1c15ab30 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d22573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1f4b50 con 0x7f1d1c0adc70 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53340/0 (socket says 172.21.15.53:53340) 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 -- 172.21.15.53:0/2653317102 learned_addr learned my addr 172.21.15.53:0/2653317102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 -- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 msgr2=0x7f1d1c1f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21571640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 0x7f1d1c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d20d70640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d1c15ab30 0x7f1d1c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 0x7f1d1c1f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 -- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d1c15ab30 msgr2=0x7f1d1c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d1c15ab30 0x7f1d1c15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21d72640 1 -- 172.21.15.53:0/2653317102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009b30 con 0x7f1d1c15af10 2023-10-21T16:18:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.170+0000 7f1d21571640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 0x7f1d1c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d21d72640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c3c0 tx=0x7f1d1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d140074e0 con 0x7f1d1c15af10 2023-10-21T16:18:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d14007640 con 0x7f1d1c15af10 2023-10-21T16:18:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c1f4e30 con 0x7f1d1c15af10 2023-10-21T16:18:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14004a20 con 0x7f1d1c15af10 2023-10-21T16:18:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.171+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c1554a0 con 0x7f1d1c15af10 2023-10-21T16:18:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.172+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c04e160 con 0x7f1d1c15af10 2023-10-21T16:18:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.173+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1d14004b80 con 0x7f1d1c15af10 2023-10-21T16:18:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.174+0000 7f1d11ffb640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 0x7f1d0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.175+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d14089200 con 0x7f1d1c15af10 2023-10-21T16:18:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.175+0000 7f1d21571640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 0x7f1d0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.176+0000 7f1d21571640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 0x7f1d0806d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000f60 tx=0x7f1d0c009040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.179+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d14054110 con 0x7f1d1c15af10 2023-10-21T16:18:09.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.380+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d1c1f5020 con 0x7f1d1c15af10 2023-10-21T16:18:09.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.383+0000 7f1d11ffb640 1 -- 172.21.15.53:0/2653317102 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16846 (secure 0 0 0) 0x7f1d14057730 con 0x7f1d1c15af10 2023-10-21T16:18:09.385 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:09.385 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:06.357598+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:09.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 msgr2=0x7f1d0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 0x7f1d0806d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000f60 tx=0x7f1d0c009040 comp rx=0 tx=0).stop 2023-10-21T16:18:09.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 msgr2=0x7f1d1c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c3c0 tx=0x7f1d1400c880 comp rx=0 tx=0).stop 2023-10-21T16:18:09.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 shutdown_connections 2023-10-21T16:18:09.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f1d0806af00 0x7f1d0806d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d1c15af10 0x7f1d1c154f60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d1c15ab30 0x7f1d1c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 --2- 172.21.15.53:0/2653317102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d1c0adc70 0x7f1d1c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 >> 172.21.15.53:0/2653317102 conn(0x7f1d1c0cb0c0 msgr2=0x7f1d1c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:09.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 shutdown_connections 2023-10-21T16:18:09.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.386+0000 7f1d22573640 1 -- 172.21.15.53:0/2653317102 wait complete. 2023-10-21T16:18:09.411 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:18:09.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:18:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.632+0000 7f7827421640 1 Processor -- start 2023-10-21T16:18:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.632+0000 7f7827421640 1 -- start start 2023-10-21T16:18:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f7827421640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78200ad790 0x7f78200702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f7827421640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f782015e330 0x7f7820070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f7827421640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782006dee0 con 0x7f7820159ac0 2023-10-21T16:18:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f7827421640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782006dcf0 con 0x7f782015e330 2023-10-21T16:18:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f7827421640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782015d850 con 0x7f78200ad790 2023-10-21T16:18:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.634+0000 7f78259be640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f782015e330 0x7f7820070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f782015e330 0x7f7820070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53344/0 (socket says 172.21.15.53:53344) 2023-10-21T16:18:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 -- 172.21.15.53:0/3455958570 learned_addr learned my addr 172.21.15.53:0/3455958570 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78249bc640 1 --2- 172.21.15.53:0/3455958570 >> v2:172.21.15.124:3300/0 conn(0x7f78200ad790 0x7f78200702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 -- 172.21.15.53:0/3455958570 >> v2:172.21.15.124:3300/0 conn(0x7f78200ad790 msgr2=0x7f78200702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 --2- 172.21.15.53:0/3455958570 >> v2:172.21.15.124:3300/0 conn(0x7f78200ad790 0x7f78200702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 -- 172.21.15.53:0/3455958570 >> v1:172.21.15.124:6789/0 conn(0x7f7820159ac0 legacy=0x7f782006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.635+0000 7f78259be640 1 -- 172.21.15.53:0/3455958570 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782015db30 con 0x7f782015e330 2023-10-21T16:18:09.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.636+0000 7f78259be640 1 --2- 172.21.15.53:0/3455958570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f782015e330 0x7f7820070810 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f78200b4970 tx=0x7f781802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eebacde89977d6d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7817fff640 1 -- 172.21.15.53:0/3455958570 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781800b0d0 con 0x7f782015e330 2023-10-21T16:18:09.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7817fff640 1 -- 172.21.15.53:0/3455958570 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781800b230 con 0x7f782015e330 2023-10-21T16:18:09.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7817fff640 1 -- 172.21.15.53:0/3455958570 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818009600 con 0x7f782015e330 2023-10-21T16:18:09.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 -- 172.21.15.53:0/3455958570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f782015e330 msgr2=0x7f7820070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 --2- 172.21.15.53:0/3455958570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f782015e330 0x7f7820070810 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f78200b4970 tx=0x7f781802e180 comp rx=0 tx=0).stop 2023-10-21T16:18:09.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 -- 172.21.15.53:0/3455958570 shutdown_connections 2023-10-21T16:18:09.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 --2- 172.21.15.53:0/3455958570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f782015e330 0x7f7820070810 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 --2- 172.21.15.53:0/3455958570 >> v2:172.21.15.124:3300/0 conn(0x7f78200ad790 0x7f78200702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.637+0000 7f7827421640 1 -- 172.21.15.53:0/3455958570 >> 172.21.15.53:0/3455958570 conn(0x7f782006a380 msgr2=0x7f78200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.638+0000 7f7827421640 1 -- 172.21.15.53:0/3455958570 shutdown_connections 2023-10-21T16:18:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.638+0000 7f7827421640 1 -- 172.21.15.53:0/3455958570 wait complete. 2023-10-21T16:18:09.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.639+0000 7f7827421640 1 Processor -- start 2023-10-21T16:18:09.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.640+0000 7f7827421640 1 -- start start 2023-10-21T16:18:09.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.640+0000 7f7827421640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 0x7f782014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.640+0000 7f7827421640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f7827421640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782015e330 0x7f78200701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f7827421640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782014f490 con 0x7f7820159ac0 2023-10-21T16:18:09.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f7827421640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820158330 con 0x7f78200ad790 2023-10-21T16:18:09.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f7827421640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782014f730 con 0x7f782015e330 2023-10-21T16:18:09.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53348/0 (socket says 172.21.15.53:53348) 2023-10-21T16:18:09.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 -- 172.21.15.53:0/3188422786 learned_addr learned my addr 172.21.15.53:0/3188422786 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:09.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78251bd640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 0x7f782014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78259be640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782015e330 0x7f78200701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 -- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782015e330 msgr2=0x7f78200701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782015e330 0x7f78200701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 -- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 msgr2=0x7f782014b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 0x7f782014b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78249bc640 1 -- 172.21.15.53:0/3188422786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781800bd40 con 0x7f7820159ac0 2023-10-21T16:18:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.641+0000 7f78251bd640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 0x7f782014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f78249bc640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f781000d800 tx=0x7f781000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000b6c0 con 0x7f7820159ac0 2023-10-21T16:18:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781000b820 con 0x7f7820159ac0 2023-10-21T16:18:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782014fa10 con 0x7f7820159ac0 2023-10-21T16:18:09.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000bbb0 con 0x7f7820159ac0 2023-10-21T16:18:09.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.642+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782014ffd0 con 0x7f7820159ac0 2023-10-21T16:18:09.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.643+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78200c5780 con 0x7f7820159ac0 2023-10-21T16:18:09.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.645+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f781000bd10 con 0x7f7820159ac0 2023-10-21T16:18:09.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.646+0000 7f7815ffb640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 0x7f780406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:09.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.646+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7810089e60 con 0x7f7820159ac0 2023-10-21T16:18:09.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.647+0000 7f78251bd640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 0x7f780406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:09.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.648+0000 7f78251bd640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 0x7f780406d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f781c000f60 tx=0x7f781c009040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:09.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.651+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7810054cb0 con 0x7f7820159ac0 2023-10-21T16:18:09.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.850+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 --> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7820159ea0 con 0x7f780406af00 2023-10-21T16:18:09.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.880+0000 7f7815ffb640 1 -- 172.21.15.53:0/3188422786 <== mgr.6082 v2:172.21.15.124:6832/3674677134 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287144 (secure 0 0 0) 0x7f781c0013e0 con 0x7f780406af00 2023-10-21T16:18:09.883 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:09.888 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:18:09.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 msgr2=0x7f780406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 0x7f780406d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f781c000f60 tx=0x7f781c009040 comp rx=0 tx=0).stop 2023-10-21T16:18:09.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 msgr2=0x7f7820157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:09.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f781000d800 tx=0x7f781000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 shutdown_connections 2023-10-21T16:18:09.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f780406af00 0x7f780406d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782015e330 0x7f78200701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7820159ac0 0x7f7820157df0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 --2- 172.21.15.53:0/3188422786 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78200ad790 0x7f782014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 >> 172.21.15.53:0/3188422786 conn(0x7f782006a380 msgr2=0x7f78201567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.888+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 shutdown_connections 2023-10-21T16:18:09.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:09.889+0000 7f7827421640 1 -- 172.21.15.53:0/3188422786 wait complete. 2023-10-21T16:18:09.896 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:09.893+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:09.909 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:18:08.690246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":285,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2194,"ondisk_log_size":2194,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317416,"kb_used_data":11056,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754657304,"statfs":{"total":773094113280,"available":772769079296,"internally_reserved":0,"allocated":11321344,"data_stored":6267044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.352037"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376046+0000","last_change":"2023-10-21T16:18:05.396808+0000","last_active":"2023-10-21T16:18:06.376046+0000","last_peered":"2023-10-21T16:18:06.376046+0000","last_clean":"2023-10-21T16:18:06.376046+0000","last_became_active":"2023-10-21T16:18:05.396507+0000","last_became_peered":"2023-10-21T16:18:05.396507+0000","last_unstale":"2023-10-21T16:18:06.376046+0000","last_undegraded":"2023-10-21T16:18:06.376046+0000","last_fullsized":"2023-10-21T16:18:06.376046+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:08.440136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365373+0000","last_change":"2023-10-21T16:18:05.379047+0000","last_active":"2023-10-21T16:18:06.365373+0000","last_peered":"2023-10-21T16:18:06.365373+0000","last_clean":"2023-10-21T16:18:06.365373+0000","last_became_active":"2023-10-21T16:18:05.377590+0000","last_became_peered":"2023-10-21T16:18:05.377590+0000","last_unstale":"2023-10-21T16:18:06.365373+0000","last_undegraded":"2023-10-21T16:18:06.365373+0000","last_fullsized":"2023-10-21T16:18:06.365373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:55.248105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093191+0000","last_change":"2023-10-21T16:18:05.398947+0000","last_active":"2023-10-21T16:18:07.093191+0000","last_peered":"2023-10-21T16:18:07.093191+0000","last_clean":"2023-10-21T16:18:07.093191+0000","last_became_active":"2023-10-21T16:18:05.398585+0000","last_became_peered":"2023-10-21T16:18:05.398585+0000","last_unstale":"2023-10-21T16:18:07.093191+0000","last_undegraded":"2023-10-21T16:18:07.093191+0000","last_fullsized":"2023-10-21T16:18:07.093191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:36.442174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093833+0000","last_change":"2023-10-21T16:18:05.396769+0000","last_active":"2023-10-21T16:18:07.093833+0000","last_peered":"2023-10-21T16:18:07.093833+0000","last_clean":"2023-10-21T16:18:07.093833+0000","last_became_active":"2023-10-21T16:18:05.396234+0000","last_became_peered":"2023-10-21T16:18:05.396234+0000","last_unstale":"2023-10-21T16:18:07.093833+0000","last_undegraded":"2023-10-21T16:18:07.093833+0000","last_fullsized":"2023-10-21T16:18:07.093833+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:34:13.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366717+0000","last_change":"2023-10-21T16:18:05.382529+0000","last_active":"2023-10-21T16:18:06.366717+0000","last_peered":"2023-10-21T16:18:06.366717+0000","last_clean":"2023-10-21T16:18:06.366717+0000","last_became_active":"2023-10-21T16:18:05.382111+0000","last_became_peered":"2023-10-21T16:18:05.382111+0000","last_unstale":"2023-10-21T16:18:06.366717+0000","last_undegraded":"2023-10-21T16:18:06.366717+0000","last_fullsized":"2023-10-21T16:18:06.366717+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:06.834456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093841+0000","last_change":"2023-10-21T16:18:05.399970+0000","last_active":"2023-10-21T16:18:07.093841+0000","last_peered":"2023-10-21T16:18:07.093841+0000","last_clean":"2023-10-21T16:18:07.093841+0000","last_became_active":"2023-10-21T16:18:05.399636+0000","last_became_peered":"2023-10-21T16:18:05.399636+0000","last_unstale":"2023-10-21T16:18:07.093841+0000","last_undegraded":"2023-10-21T16:18:07.093841+0000","last_fullsized":"2023-10-21T16:18:07.093841+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:49.458598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094127+0000","last_change":"2023-10-21T16:18:05.399901+0000","last_active":"2023-10-21T16:18:07.094127+0000","last_peered":"2023-10-21T16:18:07.094127+0000","last_clean":"2023-10-21T16:18:07.094127+0000","last_became_active":"2023-10-21T16:18:05.399626+0000","last_became_peered":"2023-10-21T16:18:05.399626+0000","last_unstale":"2023-10-21T16:18:07.094127+0000","last_undegraded":"2023-10-21T16:18:07.094127+0000","last_fullsized":"2023-10-21T16:18:07.094127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:51.657609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366068+0000","last_change":"2023-10-21T16:18:05.387836+0000","last_active":"2023-10-21T16:18:06.366068+0000","last_peered":"2023-10-21T16:18:06.366068+0000","last_clean":"2023-10-21T16:18:06.366068+0000","last_became_active":"2023-10-21T16:18:05.385582+0000","last_became_peered":"2023-10-21T16:18:05.385582+0000","last_unstale":"2023-10-21T16:18:06.366068+0000","last_undegraded":"2023-10-21T16:18:06.366068+0000","last_fullsized":"2023-10-21T16:18:06.366068+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:55.456430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369054+0000","last_change":"2023-10-21T16:18:05.391819+0000","last_active":"2023-10-21T16:18:06.369054+0000","last_peered":"2023-10-21T16:18:06.369054+0000","last_clean":"2023-10-21T16:18:06.369054+0000","last_became_active":"2023-10-21T16:18:05.391540+0000","last_became_peered":"2023-10-21T16:18:05.391540+0000","last_unstale":"2023-10-21T16:18:06.369054+0000","last_undegraded":"2023-10-21T16:18:06.369054+0000","last_fullsized":"2023-10-21T16:18:06.369054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:18.452219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093281+0000","last_change":"2023-10-21T16:18:05.392217+0000","last_active":"2023-10-21T16:18:07.093281+0000","last_peered":"2023-10-21T16:18:07.093281+0000","last_clean":"2023-10-21T16:18:07.093281+0000","last_became_active":"2023-10-21T16:18:05.391830+0000","last_became_peered":"2023-10-21T16:18:05.391830+0000","last_unstale":"2023-10-21T16:18:07.093281+0000","last_undegraded":"2023-10-21T16:18:07.093281+0000","last_fullsized":"2023-10-21T16:18:07.093281+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:39:08.492274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365373+0000","last_change":"2023-10-21T16:18:05.382643+0000","last_active":"2023-10-21T16:18:06.365373+0000","last_peered":"2023-10-21T16:18:06.365373+0000","last_clean":"2023-10-21T16:18:06.365373+0000","last_became_active":"2023-10-21T16:18:05.382327+0000","last_became_peered":"2023-10-21T16:18:05.382327+0000","last_unstale":"2023-10-21T16:18:06.365373+0000","last_undegraded":"2023-10-21T16:18:06.365373+0000","last_fullsized":"2023-10-21T16:18:06.365373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:54.901574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094175+0000","last_change":"2023-10-21T16:18:05.404568+0000","last_active":"2023-10-21T16:18:07.094175+0000","last_peered":"2023-10-21T16:18:07.094175+0000","last_clean":"2023-10-21T16:18:07.094175+0000","last_became_active":"2023-10-21T16:18:05.404165+0000","last_became_peered":"2023-10-21T16:18:05.404165+0000","last_unstale":"2023-10-21T16:18:07.094175+0000","last_undegraded":"2023-10-21T16:18:07.094175+0000","last_fullsized":"2023-10-21T16:18:07.094175+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:10.950082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366876+0000","last_change":"2023-10-21T16:18:05.392101+0000","last_active":"2023-10-21T16:18:06.366876+0000","last_peered":"2023-10-21T16:18:06.366876+0000","last_clean":"2023-10-21T16:18:06.366876+0000","last_became_active":"2023-10-21T16:18:05.391799+0000","last_became_peered":"2023-10-21T16:18:05.391799+0000","last_unstale":"2023-10-21T16:18:06.366876+0000","last_undegraded":"2023-10-21T16:18:06.366876+0000","last_fullsized":"2023-10-21T16:18:06.366876+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:11:30.412362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094248+0000","last_change":"2023-10-21T16:18:05.398971+0000","last_active":"2023-10-21T16:18:07.094248+0000","last_peered":"2023-10-21T16:18:07.094248+0000","last_clean":"2023-10-21T16:18:07.094248+0000","last_became_active":"2023-10-21T16:18:05.398695+0000","last_became_peered":"2023-10-21T16:18:05.398695+0000","last_unstale":"2023-10-21T16:18:07.094248+0000","last_undegraded":"2023-10-21T16:18:07.094248+0000","last_fullsized":"2023-10-21T16:18:07.094248+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:31:05.382950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363850+0000","last_change":"2023-10-21T16:18:05.397744+0000","last_active":"2023-10-21T16:18:06.363850+0000","last_peered":"2023-10-21T16:18:06.363850+0000","last_clean":"2023-10-21T16:18:06.363850+0000","last_became_active":"2023-10-21T16:18:05.396547+0000","last_became_peered":"2023-10-21T16:18:05.396547+0000","last_unstale":"2023-10-21T16:18:06.363850+0000","last_undegraded":"2023-10-21T16:18:06.363850+0000","last_fullsized":"2023-10-21T16:18:06.363850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:12:53.122599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375002+0000","last_change":"2023-10-21T16:18:05.391234+0000","last_active":"2023-10-21T16:18:06.375002+0000","last_peered":"2023-10-21T16:18:06.375002+0000","last_clean":"2023-10-21T16:18:06.375002+0000","last_became_active":"2023-10-21T16:18:05.390851+0000","last_became_peered":"2023-10-21T16:18:05.390851+0000","last_unstale":"2023-10-21T16:18:06.375002+0000","last_undegraded":"2023-10-21T16:18:06.375002+0000","last_fullsized":"2023-10-21T16:18:06.375002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:42.102036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367153+0000","last_change":"2023-10-21T16:18:05.402479+0000","last_active":"2023-10-21T16:18:06.367153+0000","last_peered":"2023-10-21T16:18:06.367153+0000","last_clean":"2023-10-21T16:18:06.367153+0000","last_became_active":"2023-10-21T16:18:05.402177+0000","last_became_peered":"2023-10-21T16:18:05.402177+0000","last_unstale":"2023-10-21T16:18:06.367153+0000","last_undegraded":"2023-10-21T16:18:06.367153+0000","last_fullsized":"2023-10-21T16:18:06.367153+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:39.962742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095010+0000","last_change":"2023-10-21T16:18:05.402143+0000","last_active":"2023-10-21T16:18:07.095010+0000","last_peered":"2023-10-21T16:18:07.095010+0000","last_clean":"2023-10-21T16:18:07.095010+0000","last_became_active":"2023-10-21T16:18:05.401838+0000","last_became_peered":"2023-10-21T16:18:05.401838+0000","last_unstale":"2023-10-21T16:18:07.095010+0000","last_undegraded":"2023-10-21T16:18:07.095010+0000","last_fullsized":"2023-10-21T16:18:07.095010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:32.094965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094191+0000","last_change":"2023-10-21T16:18:05.398158+0000","last_active":"2023-10-21T16:18:07.094191+0000","last_peered":"2023-10-21T16:18:07.094191+0000","last_clean":"2023-10-21T16:18:07.094191+0000","last_became_active":"2023-10-21T16:18:05.397845+0000","last_became_peered":"2023-10-21T16:18:05.397845+0000","last_unstale":"2023-10-21T16:18:07.094191+0000","last_undegraded":"2023-10-21T16:18:07.094191+0000","last_fullsized":"2023-10-21T16:18:07.094191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:16.992159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367335+0000","last_change":"2023-10-21T16:18:05.406280+0000","last_active":"2023-10-21T16:18:06.367335+0000","last_peered":"2023-10-21T16:18:06.367335+0000","last_clean":"2023-10-21T16:18:06.367335+0000","last_became_active":"2023-10-21T16:18:05.405701+0000","last_became_peered":"2023-10-21T16:18:05.405701+0000","last_unstale":"2023-10-21T16:18:06.367335+0000","last_undegraded":"2023-10-21T16:18:06.367335+0000","last_fullsized":"2023-10-21T16:18:06.367335+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:46.294056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367941+0000","last_change":"2023-10-21T16:18:05.400681+0000","last_active":"2023-10-21T16:18:06.367941+0000","last_peered":"2023-10-21T16:18:06.367941+0000","last_clean":"2023-10-21T16:18:06.367941+0000","last_became_active":"2023-10-21T16:18:05.400385+0000","last_became_peered":"2023-10-21T16:18:05.400385+0000","last_unstale":"2023-10-21T16:18:06.367941+0000","last_undegraded":"2023-10-21T16:18:06.367941+0000","last_fullsized":"2023-10-21T16:18:06.367941+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:36.590042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366782+0000","last_change":"2023-10-21T16:18:05.393511+0000","last_active":"2023-10-21T16:18:06.366782+0000","last_peered":"2023-10-21T16:18:06.366782+0000","last_clean":"2023-10-21T16:18:06.366782+0000","last_became_active":"2023-10-21T16:18:05.393194+0000","last_became_peered":"2023-10-21T16:18:05.393194+0000","last_unstale":"2023-10-21T16:18:06.366782+0000","last_undegraded":"2023-10-21T16:18:06.366782+0000","last_fullsized":"2023-10-21T16:18:06.366782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:35.219927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367619+0000","last_change":"2023-10-21T16:18:05.393172+0000","last_active":"2023-10-21T16:18:06.367619+0000","last_peered":"2023-10-21T16:18:06.367619+0000","last_clean":"2023-10-21T16:18:06.367619+0000","last_became_active":"2023-10-21T16:18:05.392885+0000","last_became_peered":"2023-10-21T16:18:05.392885+0000","last_unstale":"2023-10-21T16:18:06.367619+0000","last_undegraded":"2023-10-21T16:18:06.367619+0000","last_fullsized":"2023-10-21T16:18:06.367619+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:48:14.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363795+0000","last_change":"2023-10-21T16:18:05.400615+0000","last_active":"2023-10-21T16:18:06.363795+0000","last_peered":"2023-10-21T16:18:06.363795+0000","last_clean":"2023-10-21T16:18:06.363795+0000","last_became_active":"2023-10-21T16:18:05.395354+0000","last_became_peered":"2023-10-21T16:18:05.395354+0000","last_unstale":"2023-10-21T16:18:06.363795+0000","last_undegraded":"2023-10-21T16:18:06.363795+0000","last_fullsized":"2023-10-21T16:18:06.363795+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:44.375485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365876+0000","last_change":"2023-10-21T16:18:05.400022+0000","last_active":"2023-10-21T16:18:06.365876+0000","last_peered":"2023-10-21T16:18:06.365876+0000","last_clean":"2023-10-21T16:18:06.365876+0000","last_became_active":"2023-10-21T16:18:05.394831+0000","last_became_peered":"2023-10-21T16:18:05.394831+0000","last_unstale":"2023-10-21T16:18:06.365876+0000","last_undegraded":"2023-10-21T16:18:06.365876+0000","last_fullsized":"2023-10-21T16:18:06.365876+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:00.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093674+0000","last_change":"2023-10-21T16:18:05.399255+0000","last_active":"2023-10-21T16:18:07.093674+0000","last_peered":"2023-10-21T16:18:07.093674+0000","last_clean":"2023-10-21T16:18:07.093674+0000","last_became_active":"2023-10-21T16:18:05.398954+0000","last_became_peered":"2023-10-21T16:18:05.398954+0000","last_unstale":"2023-10-21T16:18:07.093674+0000","last_undegraded":"2023-10-21T16:18:07.093674+0000","last_fullsized":"2023-10-21T16:18:07.093674+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:24:55.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375912+0000","last_change":"2023-10-21T16:18:05.391594+0000","last_active":"2023-10-21T16:18:06.375912+0000","last_peered":"2023-10-21T16:18:06.375912+0000","last_clean":"2023-10-21T16:18:06.375912+0000","last_became_active":"2023-10-21T16:18:05.390756+0000","last_became_peered":"2023-10-21T16:18:05.390756+0000","last_unstale":"2023-10-21T16:18:06.375912+0000","last_undegraded":"2023-10-21T16:18:06.375912+0000","last_fullsized":"2023-10-21T16:18:06.375912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:00.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366992+0000","last_change":"2023-10-21T16:18:05.392153+0000","last_active":"2023-10-21T16:18:06.366992+0000","last_peered":"2023-10-21T16:18:06.366992+0000","last_clean":"2023-10-21T16:18:06.366992+0000","last_became_active":"2023-10-21T16:18:05.391727+0000","last_became_peered":"2023-10-21T16:18:05.391727+0000","last_unstale":"2023-10-21T16:18:06.366992+0000","last_undegraded":"2023-10-21T16:18:06.366992+0000","last_fullsized":"2023-10-21T16:18:06.366992+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:32.663692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365427+0000","last_change":"2023-10-21T16:18:05.402166+0000","last_active":"2023-10-21T16:18:06.365427+0000","last_peered":"2023-10-21T16:18:06.365427+0000","last_clean":"2023-10-21T16:18:06.365427+0000","last_became_active":"2023-10-21T16:18:05.401860+0000","last_became_peered":"2023-10-21T16:18:05.401860+0000","last_unstale":"2023-10-21T16:18:06.365427+0000","last_undegraded":"2023-10-21T16:18:06.365427+0000","last_fullsized":"2023-10-21T16:18:06.365427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:45.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375128+0000","last_change":"2023-10-21T16:18:05.397381+0000","last_active":"2023-10-21T16:18:06.375128+0000","last_peered":"2023-10-21T16:18:06.375128+0000","last_clean":"2023-10-21T16:18:06.375128+0000","last_became_active":"2023-10-21T16:18:05.397087+0000","last_became_peered":"2023-10-21T16:18:05.397087+0000","last_unstale":"2023-10-21T16:18:06.375128+0000","last_undegraded":"2023-10-21T16:18:06.375128+0000","last_fullsized":"2023-10-21T16:18:06.375128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:49.652191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092476+0000","last_change":"2023-10-21T16:18:05.400010+0000","last_active":"2023-10-21T16:18:07.092476+0000","last_peered":"2023-10-21T16:18:07.092476+0000","last_clean":"2023-10-21T16:18:07.092476+0000","last_became_active":"2023-10-21T16:18:05.399701+0000","last_became_peered":"2023-10-21T16:18:05.399701+0000","last_unstale":"2023-10-21T16:18:07.092476+0000","last_undegraded":"2023-10-21T16:18:07.092476+0000","last_fullsized":"2023-10-21T16:18:07.092476+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:31.910411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375003+0000","last_change":"2023-10-21T16:18:05.397565+0000","last_active":"2023-10-21T16:18:06.375003+0000","last_peered":"2023-10-21T16:18:06.375003+0000","last_clean":"2023-10-21T16:18:06.375003+0000","last_became_active":"2023-10-21T16:18:05.397266+0000","last_became_peered":"2023-10-21T16:18:05.397266+0000","last_unstale":"2023-10-21T16:18:06.375003+0000","last_undegraded":"2023-10-21T16:18:06.375003+0000","last_fullsized":"2023-10-21T16:18:06.375003+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:45.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092677+0000","last_change":"2023-10-21T16:18:05.397310+0000","last_active":"2023-10-21T16:18:07.092677+0000","last_peered":"2023-10-21T16:18:07.092677+0000","last_clean":"2023-10-21T16:18:07.092677+0000","last_became_active":"2023-10-21T16:18:05.397035+0000","last_became_peered":"2023-10-21T16:18:05.397035+0000","last_unstale":"2023-10-21T16:18:07.092677+0000","last_undegraded":"2023-10-21T16:18:07.092677+0000","last_fullsized":"2023-10-21T16:18:07.092677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:18.452219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367202+0000","last_change":"2023-10-21T16:18:04.378787+0000","last_active":"2023-10-21T16:18:06.367202+0000","last_peered":"2023-10-21T16:18:06.367202+0000","last_clean":"2023-10-21T16:18:06.367202+0000","last_became_active":"2023-10-21T16:18:04.377509+0000","last_became_peered":"2023-10-21T16:18:04.377509+0000","last_unstale":"2023-10-21T16:18:06.367202+0000","last_undegraded":"2023-10-21T16:18:06.367202+0000","last_fullsized":"2023-10-21T16:18:06.367202+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:32.211922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365052+0000","last_change":"2023-10-21T16:18:05.389427+0000","last_active":"2023-10-21T16:18:06.365052+0000","last_peered":"2023-10-21T16:18:06.365052+0000","last_clean":"2023-10-21T16:18:06.365052+0000","last_became_active":"2023-10-21T16:18:05.387656+0000","last_became_peered":"2023-10-21T16:18:05.387656+0000","last_unstale":"2023-10-21T16:18:06.365052+0000","last_undegraded":"2023-10-21T16:18:06.365052+0000","last_fullsized":"2023-10-21T16:18:06.365052+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:17:12.299332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375859+0000","last_change":"2023-10-21T16:18:04.384992+0000","last_active":"2023-10-21T16:18:06.375859+0000","last_peered":"2023-10-21T16:18:06.375859+0000","last_clean":"2023-10-21T16:18:06.375859+0000","last_became_active":"2023-10-21T16:18:04.384727+0000","last_became_peered":"2023-10-21T16:18:04.384727+0000","last_unstale":"2023-10-21T16:18:06.375859+0000","last_undegraded":"2023-10-21T16:18:06.375859+0000","last_fullsized":"2023-10-21T16:18:06.375859+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:12:52.105434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368148+0000","last_change":"2023-10-21T16:18:05.395393+0000","last_active":"2023-10-21T16:18:06.368148+0000","last_peered":"2023-10-21T16:18:06.368148+0000","last_clean":"2023-10-21T16:18:06.368148+0000","last_became_active":"2023-10-21T16:18:05.395081+0000","last_became_peered":"2023-10-21T16:18:05.395081+0000","last_unstale":"2023-10-21T16:18:06.368148+0000","last_undegraded":"2023-10-21T16:18:06.368148+0000","last_fullsized":"2023-10-21T16:18:06.368148+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:31.335608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375833+0000","last_change":"2023-10-21T16:18:04.387051+0000","last_active":"2023-10-21T16:18:06.375833+0000","last_peered":"2023-10-21T16:18:06.375833+0000","last_clean":"2023-10-21T16:18:06.375833+0000","last_became_active":"2023-10-21T16:18:04.386709+0000","last_became_peered":"2023-10-21T16:18:04.386709+0000","last_unstale":"2023-10-21T16:18:06.375833+0000","last_undegraded":"2023-10-21T16:18:06.375833+0000","last_fullsized":"2023-10-21T16:18:06.375833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:52.599872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368294+0000","last_change":"2023-10-21T16:18:05.402389+0000","last_active":"2023-10-21T16:18:06.368294+0000","last_peered":"2023-10-21T16:18:06.368294+0000","last_clean":"2023-10-21T16:18:06.368294+0000","last_became_active":"2023-10-21T16:18:05.401075+0000","last_became_peered":"2023-10-21T16:18:05.401075+0000","last_unstale":"2023-10-21T16:18:06.368294+0000","last_undegraded":"2023-10-21T16:18:06.368294+0000","last_fullsized":"2023-10-21T16:18:06.368294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:32:55.693833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093505+0000","last_change":"2023-10-21T16:18:04.388551+0000","last_active":"2023-10-21T16:18:07.093505+0000","last_peered":"2023-10-21T16:18:07.093505+0000","last_clean":"2023-10-21T16:18:07.093505+0000","last_became_active":"2023-10-21T16:18:04.388062+0000","last_became_peered":"2023-10-21T16:18:04.388062+0000","last_unstale":"2023-10-21T16:18:07.093505+0000","last_undegraded":"2023-10-21T16:18:07.093505+0000","last_fullsized":"2023-10-21T16:18:07.093505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:39.716055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364644+0000","last_change":"2023-10-21T16:18:05.391581+0000","last_active":"2023-10-21T16:18:06.364644+0000","last_peered":"2023-10-21T16:18:06.364644+0000","last_clean":"2023-10-21T16:18:06.364644+0000","last_became_active":"2023-10-21T16:18:05.390834+0000","last_became_peered":"2023-10-21T16:18:05.390834+0000","last_unstale":"2023-10-21T16:18:06.364644+0000","last_undegraded":"2023-10-21T16:18:06.364644+0000","last_fullsized":"2023-10-21T16:18:06.364644+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:53.617037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368553+0000","last_change":"2023-10-21T16:18:04.386427+0000","last_active":"2023-10-21T16:18:06.368553+0000","last_peered":"2023-10-21T16:18:06.368553+0000","last_clean":"2023-10-21T16:18:06.368553+0000","last_became_active":"2023-10-21T16:18:04.385256+0000","last_became_peered":"2023-10-21T16:18:04.385256+0000","last_unstale":"2023-10-21T16:18:06.368553+0000","last_undegraded":"2023-10-21T16:18:06.368553+0000","last_fullsized":"2023-10-21T16:18:06.368553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:05.761467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365561+0000","last_change":"2023-10-21T16:18:05.387666+0000","last_active":"2023-10-21T16:18:06.365561+0000","last_peered":"2023-10-21T16:18:06.365561+0000","last_clean":"2023-10-21T16:18:06.365561+0000","last_became_active":"2023-10-21T16:18:05.384640+0000","last_became_peered":"2023-10-21T16:18:05.384640+0000","last_unstale":"2023-10-21T16:18:06.365561+0000","last_undegraded":"2023-10-21T16:18:06.365561+0000","last_fullsized":"2023-10-21T16:18:06.365561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:42.102036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093789+0000","last_change":"2023-10-21T16:18:04.383384+0000","last_active":"2023-10-21T16:18:07.093789+0000","last_peered":"2023-10-21T16:18:07.093789+0000","last_clean":"2023-10-21T16:18:07.093789+0000","last_became_active":"2023-10-21T16:18:04.383031+0000","last_became_peered":"2023-10-21T16:18:04.383031+0000","last_unstale":"2023-10-21T16:18:07.093789+0000","last_undegraded":"2023-10-21T16:18:07.093789+0000","last_fullsized":"2023-10-21T16:18:07.093789+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:07:59.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369438+0000","last_change":"2023-10-21T16:18:05.393645+0000","last_active":"2023-10-21T16:18:06.369438+0000","last_peered":"2023-10-21T16:18:06.369438+0000","last_clean":"2023-10-21T16:18:06.369438+0000","last_became_active":"2023-10-21T16:18:05.393310+0000","last_became_peered":"2023-10-21T16:18:05.393310+0000","last_unstale":"2023-10-21T16:18:06.369438+0000","last_undegraded":"2023-10-21T16:18:06.369438+0000","last_fullsized":"2023-10-21T16:18:06.369438+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:59.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364774+0000","last_change":"2023-10-21T16:18:04.386037+0000","last_active":"2023-10-21T16:18:06.364774+0000","last_peered":"2023-10-21T16:18:06.364774+0000","last_clean":"2023-10-21T16:18:06.364774+0000","last_became_active":"2023-10-21T16:18:04.385281+0000","last_became_peered":"2023-10-21T16:18:04.385281+0000","last_unstale":"2023-10-21T16:18:06.364774+0000","last_undegraded":"2023-10-21T16:18:06.364774+0000","last_fullsized":"2023-10-21T16:18:06.364774+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:39:31.904699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093966+0000","last_change":"2023-10-21T16:18:05.400829+0000","last_active":"2023-10-21T16:18:07.093966+0000","last_peered":"2023-10-21T16:18:07.093966+0000","last_clean":"2023-10-21T16:18:07.093966+0000","last_became_active":"2023-10-21T16:18:05.400532+0000","last_became_peered":"2023-10-21T16:18:05.400532+0000","last_unstale":"2023-10-21T16:18:07.093966+0000","last_undegraded":"2023-10-21T16:18:07.093966+0000","last_fullsized":"2023-10-21T16:18:07.093966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:41.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366158+0000","last_change":"2023-10-21T16:18:04.383639+0000","last_active":"2023-10-21T16:18:06.366158+0000","last_peered":"2023-10-21T16:18:06.366158+0000","last_clean":"2023-10-21T16:18:06.366158+0000","last_became_active":"2023-10-21T16:18:04.383170+0000","last_became_peered":"2023-10-21T16:18:04.383170+0000","last_unstale":"2023-10-21T16:18:06.366158+0000","last_undegraded":"2023-10-21T16:18:06.366158+0000","last_fullsized":"2023-10-21T16:18:06.366158+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:52:12.529588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363880+0000","last_change":"2023-10-21T16:18:05.381838+0000","last_active":"2023-10-21T16:18:06.363880+0000","last_peered":"2023-10-21T16:18:06.363880+0000","last_clean":"2023-10-21T16:18:06.363880+0000","last_became_active":"2023-10-21T16:18:05.380561+0000","last_became_peered":"2023-10-21T16:18:05.380561+0000","last_unstale":"2023-10-21T16:18:06.363880+0000","last_undegraded":"2023-10-21T16:18:06.363880+0000","last_fullsized":"2023-10-21T16:18:06.363880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:36.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376463+0000","last_change":"2023-10-21T16:18:04.393918+0000","last_active":"2023-10-21T16:18:06.376463+0000","last_peered":"2023-10-21T16:18:06.376463+0000","last_clean":"2023-10-21T16:18:06.376463+0000","last_became_active":"2023-10-21T16:18:04.393297+0000","last_became_peered":"2023-10-21T16:18:04.393297+0000","last_unstale":"2023-10-21T16:18:06.376463+0000","last_undegraded":"2023-10-21T16:18:06.376463+0000","last_fullsized":"2023-10-21T16:18:06.376463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:53.884409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094262+0000","last_change":"2023-10-21T16:18:05.395917+0000","last_active":"2023-10-21T16:18:07.094262+0000","last_peered":"2023-10-21T16:18:07.094262+0000","last_clean":"2023-10-21T16:18:07.094262+0000","last_became_active":"2023-10-21T16:18:05.395605+0000","last_became_peered":"2023-10-21T16:18:05.395605+0000","last_unstale":"2023-10-21T16:18:07.094262+0000","last_undegraded":"2023-10-21T16:18:07.094262+0000","last_fullsized":"2023-10-21T16:18:07.094262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:04.771243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095189+0000","last_change":"2023-10-21T16:18:04.389454+0000","last_active":"2023-10-21T16:18:07.095189+0000","last_peered":"2023-10-21T16:18:07.095189+0000","last_clean":"2023-10-21T16:18:07.095189+0000","last_became_active":"2023-10-21T16:18:04.389121+0000","last_became_peered":"2023-10-21T16:18:04.389121+0000","last_unstale":"2023-10-21T16:18:07.095189+0000","last_undegraded":"2023-10-21T16:18:07.095189+0000","last_fullsized":"2023-10-21T16:18:07.095189+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:22.191569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094823+0000","last_change":"2023-10-21T16:18:05.408329+0000","last_active":"2023-10-21T16:18:07.094823+0000","last_peered":"2023-10-21T16:18:07.094823+0000","last_clean":"2023-10-21T16:18:07.094823+0000","last_became_active":"2023-10-21T16:18:05.408053+0000","last_became_peered":"2023-10-21T16:18:05.408053+0000","last_unstale":"2023-10-21T16:18:07.094823+0000","last_undegraded":"2023-10-21T16:18:07.094823+0000","last_fullsized":"2023-10-21T16:18:07.094823+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:31:02.308970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367213+0000","last_change":"2023-10-21T16:18:04.382155+0000","last_active":"2023-10-21T16:18:06.367213+0000","last_peered":"2023-10-21T16:18:06.367213+0000","last_clean":"2023-10-21T16:18:06.367213+0000","last_became_active":"2023-10-21T16:18:04.380887+0000","last_became_peered":"2023-10-21T16:18:04.380887+0000","last_unstale":"2023-10-21T16:18:06.367213+0000","last_undegraded":"2023-10-21T16:18:06.367213+0000","last_fullsized":"2023-10-21T16:18:06.367213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:01.732635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093042+0000","last_change":"2023-10-21T16:18:05.398856+0000","last_active":"2023-10-21T16:18:07.093042+0000","last_peered":"2023-10-21T16:18:07.093042+0000","last_clean":"2023-10-21T16:18:07.093042+0000","last_became_active":"2023-10-21T16:18:05.398567+0000","last_became_peered":"2023-10-21T16:18:05.398567+0000","last_unstale":"2023-10-21T16:18:07.093042+0000","last_undegraded":"2023-10-21T16:18:07.093042+0000","last_fullsized":"2023-10-21T16:18:07.093042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:59.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093581+0000","last_change":"2023-10-21T16:18:04.390512+0000","last_active":"2023-10-21T16:18:07.093581+0000","last_peered":"2023-10-21T16:18:07.093581+0000","last_clean":"2023-10-21T16:18:07.093581+0000","last_became_active":"2023-10-21T16:18:04.389892+0000","last_became_peered":"2023-10-21T16:18:04.389892+0000","last_unstale":"2023-10-21T16:18:07.093581+0000","last_undegraded":"2023-10-21T16:18:07.093581+0000","last_fullsized":"2023-10-21T16:18:07.093581+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:37:35.914023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094536+0000","last_change":"2023-10-21T16:18:05.397866+0000","last_active":"2023-10-21T16:18:07.094536+0000","last_peered":"2023-10-21T16:18:07.094536+0000","last_clean":"2023-10-21T16:18:07.094536+0000","last_became_active":"2023-10-21T16:18:05.397458+0000","last_became_peered":"2023-10-21T16:18:05.397458+0000","last_unstale":"2023-10-21T16:18:07.094536+0000","last_undegraded":"2023-10-21T16:18:07.094536+0000","last_fullsized":"2023-10-21T16:18:07.094536+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:50.422461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094106+0000","last_change":"2023-10-21T16:18:04.384460+0000","last_active":"2023-10-21T16:18:07.094106+0000","last_peered":"2023-10-21T16:18:07.094106+0000","last_clean":"2023-10-21T16:18:07.094106+0000","last_became_active":"2023-10-21T16:18:04.384110+0000","last_became_peered":"2023-10-21T16:18:04.384110+0000","last_unstale":"2023-10-21T16:18:07.094106+0000","last_undegraded":"2023-10-21T16:18:07.094106+0000","last_fullsized":"2023-10-21T16:18:07.094106+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094124+0000","last_change":"2023-10-21T16:18:05.401680+0000","last_active":"2023-10-21T16:18:07.094124+0000","last_peered":"2023-10-21T16:18:07.094124+0000","last_clean":"2023-10-21T16:18:07.094124+0000","last_became_active":"2023-10-21T16:18:05.401209+0000","last_became_peered":"2023-10-21T16:18:05.401209+0000","last_unstale":"2023-10-21T16:18:07.094124+0000","last_undegraded":"2023-10-21T16:18:07.094124+0000","last_fullsized":"2023-10-21T16:18:07.094124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:06.778632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375392+0000","last_change":"2023-10-21T16:18:04.378706+0000","last_active":"2023-10-21T16:18:06.375392+0000","last_peered":"2023-10-21T16:18:06.375392+0000","last_clean":"2023-10-21T16:18:06.375392+0000","last_became_active":"2023-10-21T16:18:04.376187+0000","last_became_peered":"2023-10-21T16:18:04.376187+0000","last_unstale":"2023-10-21T16:18:06.375392+0000","last_undegraded":"2023-10-21T16:18:06.375392+0000","last_fullsized":"2023-10-21T16:18:06.375392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:16.123954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093740+0000","last_change":"2023-10-21T16:18:05.398595+0000","last_active":"2023-10-21T16:18:07.093740+0000","last_peered":"2023-10-21T16:18:07.093740+0000","last_clean":"2023-10-21T16:18:07.093740+0000","last_became_active":"2023-10-21T16:18:05.398294+0000","last_became_peered":"2023-10-21T16:18:05.398294+0000","last_unstale":"2023-10-21T16:18:07.093740+0000","last_undegraded":"2023-10-21T16:18:07.093740+0000","last_fullsized":"2023-10-21T16:18:07.093740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:34:02.300707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367166+0000","last_change":"2023-10-21T16:18:04.379131+0000","last_active":"2023-10-21T16:18:06.367166+0000","last_peered":"2023-10-21T16:18:06.367166+0000","last_clean":"2023-10-21T16:18:06.367166+0000","last_became_active":"2023-10-21T16:18:04.378830+0000","last_became_peered":"2023-10-21T16:18:04.378830+0000","last_unstale":"2023-10-21T16:18:06.367166+0000","last_undegraded":"2023-10-21T16:18:06.367166+0000","last_fullsized":"2023-10-21T16:18:06.367166+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:43.935138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094276+0000","last_change":"2023-10-21T16:18:05.398942+0000","last_active":"2023-10-21T16:18:07.094276+0000","last_peered":"2023-10-21T16:18:07.094276+0000","last_clean":"2023-10-21T16:18:07.094276+0000","last_became_active":"2023-10-21T16:18:05.398463+0000","last_became_peered":"2023-10-21T16:18:05.398463+0000","last_unstale":"2023-10-21T16:18:07.094276+0000","last_undegraded":"2023-10-21T16:18:07.094276+0000","last_fullsized":"2023-10-21T16:18:07.094276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:20.440829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"102'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364085+0000","last_change":"2023-10-21T16:18:04.387404+0000","last_active":"2023-10-21T16:18:06.364085+0000","last_peered":"2023-10-21T16:18:06.364085+0000","last_clean":"2023-10-21T16:18:06.364085+0000","last_became_active":"2023-10-21T16:18:04.387111+0000","last_became_peered":"2023-10-21T16:18:04.387111+0000","last_unstale":"2023-10-21T16:18:06.364085+0000","last_undegraded":"2023-10-21T16:18:06.364085+0000","last_fullsized":"2023-10-21T16:18:06.364085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:15.630667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092995+0000","last_change":"2023-10-21T16:18:05.400133+0000","last_active":"2023-10-21T16:18:07.092995+0000","last_peered":"2023-10-21T16:18:07.092995+0000","last_clean":"2023-10-21T16:18:07.092995+0000","last_became_active":"2023-10-21T16:18:05.399831+0000","last_became_peered":"2023-10-21T16:18:05.399831+0000","last_unstale":"2023-10-21T16:18:07.092995+0000","last_undegraded":"2023-10-21T16:18:07.092995+0000","last_fullsized":"2023-10-21T16:18:07.092995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:24.680934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376569+0000","last_change":"2023-10-21T16:18:04.388088+0000","last_active":"2023-10-21T16:18:06.376569+0000","last_peered":"2023-10-21T16:18:06.376569+0000","last_clean":"2023-10-21T16:18:06.376569+0000","last_became_active":"2023-10-21T16:18:04.384451+0000","last_became_peered":"2023-10-21T16:18:04.384451+0000","last_unstale":"2023-10-21T16:18:06.376569+0000","last_undegraded":"2023-10-21T16:18:06.376569+0000","last_fullsized":"2023-10-21T16:18:06.376569+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:34.280219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366558+0000","last_change":"2023-10-21T16:18:05.402635+0000","last_active":"2023-10-21T16:18:06.366558+0000","last_peered":"2023-10-21T16:18:06.366558+0000","last_clean":"2023-10-21T16:18:06.366558+0000","last_became_active":"2023-10-21T16:18:05.402361+0000","last_became_peered":"2023-10-21T16:18:05.402361+0000","last_unstale":"2023-10-21T16:18:06.366558+0000","last_undegraded":"2023-10-21T16:18:06.366558+0000","last_fullsized":"2023-10-21T16:18:06.366558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:20.765430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369270+0000","last_change":"2023-10-21T16:18:04.386426+0000","last_active":"2023-10-21T16:18:06.369270+0000","last_peered":"2023-10-21T16:18:06.369270+0000","last_clean":"2023-10-21T16:18:06.369270+0000","last_became_active":"2023-10-21T16:18:04.386104+0000","last_became_peered":"2023-10-21T16:18:04.386104+0000","last_unstale":"2023-10-21T16:18:06.369270+0000","last_undegraded":"2023-10-21T16:18:06.369270+0000","last_fullsized":"2023-10-21T16:18:06.369270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:50.640444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366848+0000","last_change":"2023-10-21T16:18:05.386758+0000","last_active":"2023-10-21T16:18:06.366848+0000","last_peered":"2023-10-21T16:18:06.366848+0000","last_clean":"2023-10-21T16:18:06.366848+0000","last_became_active":"2023-10-21T16:18:05.386463+0000","last_became_peered":"2023-10-21T16:18:05.386463+0000","last_unstale":"2023-10-21T16:18:06.366848+0000","last_undegraded":"2023-10-21T16:18:06.366848+0000","last_fullsized":"2023-10-21T16:18:06.366848+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:52.197030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367337+0000","last_change":"2023-10-21T16:18:04.385522+0000","last_active":"2023-10-21T16:18:06.367337+0000","last_peered":"2023-10-21T16:18:06.367337+0000","last_clean":"2023-10-21T16:18:06.367337+0000","last_became_active":"2023-10-21T16:18:04.385129+0000","last_became_peered":"2023-10-21T16:18:04.385129+0000","last_unstale":"2023-10-21T16:18:06.367337+0000","last_undegraded":"2023-10-21T16:18:06.367337+0000","last_fullsized":"2023-10-21T16:18:06.367337+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:39.561258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094549+0000","last_change":"2023-10-21T16:18:05.407453+0000","last_active":"2023-10-21T16:18:07.094549+0000","last_peered":"2023-10-21T16:18:07.094549+0000","last_clean":"2023-10-21T16:18:07.094549+0000","last_became_active":"2023-10-21T16:18:05.407174+0000","last_became_peered":"2023-10-21T16:18:05.407174+0000","last_unstale":"2023-10-21T16:18:07.094549+0000","last_undegraded":"2023-10-21T16:18:07.094549+0000","last_fullsized":"2023-10-21T16:18:07.094549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:15:27.225254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093302+0000","last_change":"2023-10-21T16:18:04.374973+0000","last_active":"2023-10-21T16:18:07.093302+0000","last_peered":"2023-10-21T16:18:07.093302+0000","last_clean":"2023-10-21T16:18:07.093302+0000","last_became_active":"2023-10-21T16:18:04.374649+0000","last_became_peered":"2023-10-21T16:18:04.374649+0000","last_unstale":"2023-10-21T16:18:07.093302+0000","last_undegraded":"2023-10-21T16:18:07.093302+0000","last_fullsized":"2023-10-21T16:18:07.093302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:32:54.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364832+0000","last_change":"2023-10-21T16:18:05.399485+0000","last_active":"2023-10-21T16:18:06.364832+0000","last_peered":"2023-10-21T16:18:06.364832+0000","last_clean":"2023-10-21T16:18:06.364832+0000","last_became_active":"2023-10-21T16:18:05.396459+0000","last_became_peered":"2023-10-21T16:18:05.396459+0000","last_unstale":"2023-10-21T16:18:06.364832+0000","last_undegraded":"2023-10-21T16:18:06.364832+0000","last_fullsized":"2023-10-21T16:18:06.364832+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:56.863624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094256+0000","last_change":"2023-10-21T16:18:04.378376+0000","last_active":"2023-10-21T16:18:07.094256+0000","last_peered":"2023-10-21T16:18:07.094256+0000","last_clean":"2023-10-21T16:18:07.094256+0000","last_became_active":"2023-10-21T16:18:04.378024+0000","last_became_peered":"2023-10-21T16:18:04.378024+0000","last_unstale":"2023-10-21T16:18:07.094256+0000","last_undegraded":"2023-10-21T16:18:07.094256+0000","last_fullsized":"2023-10-21T16:18:07.094256+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:47:57.875260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365702+0000","last_change":"2023-10-21T16:18:05.380394+0000","last_active":"2023-10-21T16:18:06.365702+0000","last_peered":"2023-10-21T16:18:06.365702+0000","last_clean":"2023-10-21T16:18:06.365702+0000","last_became_active":"2023-10-21T16:18:05.379976+0000","last_became_peered":"2023-10-21T16:18:05.379976+0000","last_unstale":"2023-10-21T16:18:06.365702+0000","last_undegraded":"2023-10-21T16:18:06.365702+0000","last_fullsized":"2023-10-21T16:18:06.365702+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:49.652191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093764+0000","last_change":"2023-10-21T16:18:04.392725+0000","last_active":"2023-10-21T16:18:07.093764+0000","last_peered":"2023-10-21T16:18:07.093764+0000","last_clean":"2023-10-21T16:18:07.093764+0000","last_became_active":"2023-10-21T16:18:04.392112+0000","last_became_peered":"2023-10-21T16:18:04.392112+0000","last_unstale":"2023-10-21T16:18:07.093764+0000","last_undegraded":"2023-10-21T16:18:07.093764+0000","last_fullsized":"2023-10-21T16:18:07.093764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:55:04.795282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094641+0000","last_change":"2023-10-21T16:18:05.397703+0000","last_active":"2023-10-21T16:18:07.094641+0000","last_peered":"2023-10-21T16:18:07.094641+0000","last_clean":"2023-10-21T16:18:07.094641+0000","last_became_active":"2023-10-21T16:18:05.390756+0000","last_became_peered":"2023-10-21T16:18:05.390756+0000","last_unstale":"2023-10-21T16:18:07.094641+0000","last_undegraded":"2023-10-21T16:18:07.094641+0000","last_fullsized":"2023-10-21T16:18:07.094641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:31.335608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093614+0000","last_change":"2023-10-21T16:18:04.392499+0000","last_active":"2023-10-21T16:18:07.093614+0000","last_peered":"2023-10-21T16:18:07.093614+0000","last_clean":"2023-10-21T16:18:07.093614+0000","last_became_active":"2023-10-21T16:18:04.392158+0000","last_became_peered":"2023-10-21T16:18:04.392158+0000","last_unstale":"2023-10-21T16:18:07.093614+0000","last_undegraded":"2023-10-21T16:18:07.093614+0000","last_fullsized":"2023-10-21T16:18:07.093614+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:54:16.559817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367728+0000","last_change":"2023-10-21T16:18:05.403273+0000","last_active":"2023-10-21T16:18:06.367728+0000","last_peered":"2023-10-21T16:18:06.367728+0000","last_clean":"2023-10-21T16:18:06.367728+0000","last_became_active":"2023-10-21T16:18:05.402953+0000","last_became_peered":"2023-10-21T16:18:05.402953+0000","last_unstale":"2023-10-21T16:18:06.367728+0000","last_undegraded":"2023-10-21T16:18:06.367728+0000","last_fullsized":"2023-10-21T16:18:06.367728+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:20.440829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095254+0000","last_change":"2023-10-21T16:18:04.383743+0000","last_active":"2023-10-21T16:18:07.095254+0000","last_peered":"2023-10-21T16:18:07.095254+0000","last_clean":"2023-10-21T16:18:07.095254+0000","last_became_active":"2023-10-21T16:18:04.383406+0000","last_became_peered":"2023-10-21T16:18:04.383406+0000","last_unstale":"2023-10-21T16:18:07.095254+0000","last_undegraded":"2023-10-21T16:18:07.095254+0000","last_fullsized":"2023-10-21T16:18:07.095254+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:49:00.300035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094496+0000","last_change":"2023-10-21T16:18:05.395186+0000","last_active":"2023-10-21T16:18:07.094496+0000","last_peered":"2023-10-21T16:18:07.094496+0000","last_clean":"2023-10-21T16:18:07.094496+0000","last_became_active":"2023-10-21T16:18:05.394918+0000","last_became_peered":"2023-10-21T16:18:05.394918+0000","last_unstale":"2023-10-21T16:18:07.094496+0000","last_undegraded":"2023-10-21T16:18:07.094496+0000","last_fullsized":"2023-10-21T16:18:07.094496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:36.590042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365955+0000","last_change":"2023-10-21T16:18:04.385072+0000","last_active":"2023-10-21T16:18:06.365955+0000","last_peered":"2023-10-21T16:18:06.365955+0000","last_clean":"2023-10-21T16:18:06.365955+0000","last_became_active":"2023-10-21T16:18:04.384675+0000","last_became_peered":"2023-10-21T16:18:04.384675+0000","last_unstale":"2023-10-21T16:18:06.365955+0000","last_undegraded":"2023-10-21T16:18:06.365955+0000","last_fullsized":"2023-10-21T16:18:06.365955+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:27:18.708201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368319+0000","last_change":"2023-10-21T16:18:05.393503+0000","last_active":"2023-10-21T16:18:06.368319+0000","last_peered":"2023-10-21T16:18:06.368319+0000","last_clean":"2023-10-21T16:18:06.368319+0000","last_became_active":"2023-10-21T16:18:05.393187+0000","last_became_peered":"2023-10-21T16:18:05.393187+0000","last_unstale":"2023-10-21T16:18:06.368319+0000","last_undegraded":"2023-10-21T16:18:06.368319+0000","last_fullsized":"2023-10-21T16:18:06.368319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:31.910411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376135+0000","last_change":"2023-10-21T16:18:04.382536+0000","last_active":"2023-10-21T16:18:06.376135+0000","last_peered":"2023-10-21T16:18:06.376135+0000","last_clean":"2023-10-21T16:18:06.376135+0000","last_became_active":"2023-10-21T16:18:04.382231+0000","last_became_peered":"2023-10-21T16:18:04.382231+0000","last_unstale":"2023-10-21T16:18:06.376135+0000","last_undegraded":"2023-10-21T16:18:06.376135+0000","last_fullsized":"2023-10-21T16:18:06.376135+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:54.230940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365375+0000","last_change":"2023-10-21T16:18:05.381505+0000","last_active":"2023-10-21T16:18:06.365375+0000","last_peered":"2023-10-21T16:18:06.365375+0000","last_clean":"2023-10-21T16:18:06.365375+0000","last_became_active":"2023-10-21T16:18:05.381024+0000","last_became_peered":"2023-10-21T16:18:05.381024+0000","last_unstale":"2023-10-21T16:18:06.365375+0000","last_undegraded":"2023-10-21T16:18:06.365375+0000","last_fullsized":"2023-10-21T16:18:06.365375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:35.297384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375888+0000","last_change":"2023-10-21T16:18:04.392811+0000","last_active":"2023-10-21T16:18:06.375888+0000","last_peered":"2023-10-21T16:18:06.375888+0000","last_clean":"2023-10-21T16:18:06.375888+0000","last_became_active":"2023-10-21T16:18:04.392525+0000","last_became_peered":"2023-10-21T16:18:04.392525+0000","last_unstale":"2023-10-21T16:18:06.375888+0000","last_undegraded":"2023-10-21T16:18:06.375888+0000","last_fullsized":"2023-10-21T16:18:06.375888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:43.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093296+0000","last_change":"2023-10-21T16:18:05.407152+0000","last_active":"2023-10-21T16:18:07.093296+0000","last_peered":"2023-10-21T16:18:07.093296+0000","last_clean":"2023-10-21T16:18:07.093296+0000","last_became_active":"2023-10-21T16:18:05.406858+0000","last_became_peered":"2023-10-21T16:18:05.406858+0000","last_unstale":"2023-10-21T16:18:07.093296+0000","last_undegraded":"2023-10-21T16:18:07.093296+0000","last_fullsized":"2023-10-21T16:18:07.093296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:33.733379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094839+0000","last_change":"2023-10-21T16:18:04.386344+0000","last_active":"2023-10-21T16:18:07.094839+0000","last_peered":"2023-10-21T16:18:07.094839+0000","last_clean":"2023-10-21T16:18:07.094839+0000","last_became_active":"2023-10-21T16:18:04.386028+0000","last_became_peered":"2023-10-21T16:18:04.386028+0000","last_unstale":"2023-10-21T16:18:07.094839+0000","last_undegraded":"2023-10-21T16:18:07.094839+0000","last_fullsized":"2023-10-21T16:18:07.094839+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:47:53.293319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094207+0000","last_change":"2023-10-21T16:18:05.400979+0000","last_active":"2023-10-21T16:18:07.094207+0000","last_peered":"2023-10-21T16:18:07.094207+0000","last_clean":"2023-10-21T16:18:07.094207+0000","last_became_active":"2023-10-21T16:18:05.400673+0000","last_became_peered":"2023-10-21T16:18:05.400673+0000","last_unstale":"2023-10-21T16:18:07.094207+0000","last_undegraded":"2023-10-21T16:18:07.094207+0000","last_fullsized":"2023-10-21T16:18:07.094207+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:48:14.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366885+0000","last_change":"2023-10-21T16:18:04.383192+0000","last_active":"2023-10-21T16:18:06.366885+0000","last_peered":"2023-10-21T16:18:06.366885+0000","last_clean":"2023-10-21T16:18:06.366885+0000","last_became_active":"2023-10-21T16:18:04.382166+0000","last_became_peered":"2023-10-21T16:18:04.382166+0000","last_unstale":"2023-10-21T16:18:06.366885+0000","last_undegraded":"2023-10-21T16:18:06.366885+0000","last_fullsized":"2023-10-21T16:18:06.366885+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:26.364173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094340+0000","last_change":"2023-10-21T16:18:05.398878+0000","last_active":"2023-10-21T16:18:07.094340+0000","last_peered":"2023-10-21T16:18:07.094340+0000","last_clean":"2023-10-21T16:18:07.094340+0000","last_became_active":"2023-10-21T16:18:05.398481+0000","last_became_peered":"2023-10-21T16:18:05.398481+0000","last_unstale":"2023-10-21T16:18:07.094340+0000","last_undegraded":"2023-10-21T16:18:07.094340+0000","last_fullsized":"2023-10-21T16:18:07.094340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:42.601976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094707+0000","last_change":"2023-10-21T16:18:04.392386+0000","last_active":"2023-10-21T16:18:07.094707+0000","last_peered":"2023-10-21T16:18:07.094707+0000","last_clean":"2023-10-21T16:18:07.094707+0000","last_became_active":"2023-10-21T16:18:04.392039+0000","last_became_peered":"2023-10-21T16:18:04.392039+0000","last_unstale":"2023-10-21T16:18:07.094707+0000","last_undegraded":"2023-10-21T16:18:07.094707+0000","last_fullsized":"2023-10-21T16:18:07.094707+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:41.045578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094747+0000","last_change":"2023-10-21T16:18:05.404187+0000","last_active":"2023-10-21T16:18:07.094747+0000","last_peered":"2023-10-21T16:18:07.094747+0000","last_clean":"2023-10-21T16:18:07.094747+0000","last_became_active":"2023-10-21T16:18:05.402262+0000","last_became_peered":"2023-10-21T16:18:05.402262+0000","last_unstale":"2023-10-21T16:18:07.094747+0000","last_undegraded":"2023-10-21T16:18:07.094747+0000","last_fullsized":"2023-10-21T16:18:07.094747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:27.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092298+0000","last_change":"2023-10-21T16:18:04.375387+0000","last_active":"2023-10-21T16:18:07.092298+0000","last_peered":"2023-10-21T16:18:07.092298+0000","last_clean":"2023-10-21T16:18:07.092298+0000","last_became_active":"2023-10-21T16:18:04.375101+0000","last_became_peered":"2023-10-21T16:18:04.375101+0000","last_unstale":"2023-10-21T16:18:07.092298+0000","last_undegraded":"2023-10-21T16:18:07.092298+0000","last_fullsized":"2023-10-21T16:18:07.092298+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:30.318443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376198+0000","last_change":"2023-10-21T16:18:05.405075+0000","last_active":"2023-10-21T16:18:06.376198+0000","last_peered":"2023-10-21T16:18:06.376198+0000","last_clean":"2023-10-21T16:18:06.376198+0000","last_became_active":"2023-10-21T16:18:05.404698+0000","last_became_peered":"2023-10-21T16:18:05.404698+0000","last_unstale":"2023-10-21T16:18:06.376198+0000","last_undegraded":"2023-10-21T16:18:06.376198+0000","last_fullsized":"2023-10-21T16:18:06.376198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:27:42.758492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095590+0000","last_change":"2023-10-21T16:18:04.392526+0000","last_active":"2023-10-21T16:18:07.095590+0000","last_peered":"2023-10-21T16:18:07.095590+0000","last_clean":"2023-10-21T16:18:07.095590+0000","last_became_active":"2023-10-21T16:18:04.392106+0000","last_became_peered":"2023-10-21T16:18:04.392106+0000","last_unstale":"2023-10-21T16:18:07.095590+0000","last_undegraded":"2023-10-21T16:18:07.095590+0000","last_fullsized":"2023-10-21T16:18:07.095590+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:45.012971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"103'2156","reported_seq":2424,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.677720+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:18:07.677720+0000","last_peered":"2023-10-21T16:18:07.677720+0000","last_clean":"2023-10-21T16:18:07.677720+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:18:07.677720+0000","last_undegraded":"2023-10-21T16:18:07.677720+0000","last_fullsized":"2023-10-21T16:18:07.677720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":2156,"log_dups_size":0,"ondisk_log_size":2156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2156,"ondisk_log_size":2156,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":1227242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.65000000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5609999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.171},"min":{"1min":0.76200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4019999999999999,"5min":2.448,"15min":2.448},"last":0.89000000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78300000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5389999999999999,"5min":1.915,"15min":1.915},"last":1.012},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.583,"5min":1.778,"15min":1.778},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.27,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5880000000000001,"5min":1.748,"15min":2.085},"last":1.256},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2849999999999999,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.556,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.6519999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.2529999999999999},"min":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6040000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.85999999999999999,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.9570000000000001,"15min":2.0099999999999998},"last":1.4379999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3360000000000001,"15min":1.355},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":1.7350000000000001,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.974},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.423,"15min":1.4450000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70399999999999996},"max":{"1min":1.8360000000000001,"5min":2.355,"15min":2.355},"last":1.915}]},{"osd":5,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.391,"15min":1.3999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.716,"5min":1.865,"15min":2.258},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.4350000000000001,"15min":1.4379999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.60899999999999999},"max":{"1min":1.9079999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":2.1200000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.53,"15min":1.548},"min":{"1min":1.0580000000000001,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":1.706,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":2.1859999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4790000000000001,"15min":1.466},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7709999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":2.0449999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":631800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.68799999999999994,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.514,"5min":1.895,"15min":2.4350000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.417,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.3480000000000001,"15min":1.306},"min":{"1min":0.90700000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5960000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.895,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.121},{"interface":"front","average":{"1min":1.24,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6839999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.3120000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.377,"15min":1.3740000000000001},"min":{"1min":1.1200000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7490000000000001,"5min":2.2000000000000002,"15min":3.46},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.446,"15min":1.4299999999999999},"min":{"1min":1.204,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":1.9970000000000001,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.4359999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.393,"15min":1.365},"min":{"1min":0.94599999999999995,"5min":0.82999999999999996,"15min":0.76300000000000001},"max":{"1min":1.9379999999999999,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":1.167},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.401,"15min":1.397},"min":{"1min":1.2110000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1299999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":1.536}]},{"osd":5,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":1.1120000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.246,"5min":2.246,"15min":2.6909999999999998},"last":1.627},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.99199999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8240000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.546,"15min":1.5149999999999999},"min":{"1min":1.0429999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.2050000000000001,"5min":2.3570000000000002,"15min":3.71},"last":1.363},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.631,"15min":1.623},"min":{"1min":1.4710000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.7650000000000001},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":636886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0900000000000001,"15min":1.1160000000000001},"min":{"1min":0.81100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4399999999999999,"5min":1.554,"15min":2.0070000000000001},"last":1.089},{"interface":"front","average":{"1min":1.133,"5min":1.1850000000000001,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.51,"5min":1.712,"15min":1.9530000000000001},"last":1.446}]},{"osd":1,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6339999999999999,"5min":1.877,"15min":1.944},"last":1.351},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.91200000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.907},"last":1.24}]},{"osd":2,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.83599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.8,"5min":1.8,"15min":1.95},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.206,"15min":1.242},"min":{"1min":0.85099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.0249999999999999},"last":1.288}]},{"osd":3,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0880000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.573,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":1.0009999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.79,"5min":1.79,"15min":1.8959999999999999},"last":1.522}]},{"osd":4,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2050000000000001,"15min":1.216},"min":{"1min":0.93999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.919,"5min":1.919,"15min":2.0960000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.349,"15min":1.339},"min":{"1min":0.96499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.76,"5min":2.0529999999999999,"15min":2.121},"last":1.5900000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.4550000000000001,"15min":1.45},"min":{"1min":1.3819999999999999,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1360000000000001},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.4790000000000001,"15min":1.4870000000000001},"min":{"1min":1.0860000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":2.097}]},{"osd":7,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.516,"15min":1.504},"min":{"1min":1.1699999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8300000000000001,"5min":2.218,"15min":2.218},"last":1.679},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.534,"15min":1.556},"min":{"1min":1.371,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.286,"15min":2.286},"last":1.8240000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":638568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1830000000000001,"15min":1.1379999999999999},"min":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.55400000000000005},"max":{"1min":1.5509999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.048},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.194,"15min":1.21},"min":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7230000000000001,"5min":1.758,"15min":3.6080000000000001},"last":1.385}]},{"osd":1,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.252,"15min":1.2629999999999999},"min":{"1min":0.78200000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.663,"5min":1.885,"15min":2.0699999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2190000000000001,"15min":1.228},"min":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5609999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.96799999999999997}]},{"osd":2,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.637,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.325},{"interface":"front","average":{"1min":1.304,"5min":1.298,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.698,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.89000000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.946,"5min":2.5099999999999998,"15min":3.8109999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.409,"15min":1.3779999999999999},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.5649999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2889999999999999,"15min":1.3080000000000001},"min":{"1min":0.874,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7889999999999999,"5min":3.9199999999999999,"15min":3.9199999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.335,"15min":1.3340000000000001},"min":{"1min":0.89500000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0670000000000002,"5min":3.0699999999999998,"15min":3.9300000000000002},"last":1.6240000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.8080000000000001,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.583,"15min":1.5800000000000001},"min":{"1min":1.2470000000000001,"5min":0.878,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.4500000000000002,"15min":3.9609999999999999},"last":1.48}]},{"osd":7,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5009999999999999,"15min":1.492},"min":{"1min":0.91800000000000004,"5min":0.77200000000000002,"15min":0.69599999999999995},"max":{"1min":2.2170000000000001,"5min":3.1859999999999999,"15min":4.0190000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6120000000000001,"15min":1.5609999999999999},"min":{"1min":1.3320000000000001,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9490000000000001,"5min":3.968,"15min":3.968},"last":1.7669999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1253376,"data_stored":630770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98199999999999998,"15min":0.94999999999999996},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.526,"5min":1.746,"15min":1.746},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1779999999999999,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.365,"5min":2.04,"15min":2.5289999999999999},"last":0.89000000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99399999999999999,"15min":1.0029999999999999},"min":{"1min":0.71599999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.53,"5min":2.173,"15min":2.2519999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":1.0069999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.8100000000000001,"15min":3.827},"last":0.97299999999999998}]},{"osd":2,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.3360000000000001,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.206,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.663,"5min":2.847,"15min":2.847},"last":1.0049999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.403},"min":{"1min":1.101,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6970000000000001,"5min":2.1040000000000001,"15min":2.2349999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.1180000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.8160000000000001,"5min":2.919,"15min":2.919},"last":1.1140000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.415,"15min":1.427},"min":{"1min":1.167,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.619,"5min":2.02,"15min":3.7250000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4930000000000001,"15min":1.4990000000000001},"min":{"1min":1.129,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.841,"5min":2.2629999999999999,"15min":2.423},"last":1.0349999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.1399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.2909999999999999,"5min":0.871,"15min":0.70299999999999996},"max":{"1min":1.7689999999999999,"5min":2.2290000000000001,"15min":3.573},"last":1.1619999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5740000000000001,"15min":1.591},"min":{"1min":1.2669999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.915,"5min":2.302,"15min":2.3759999999999999},"last":1.375},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5780000000000001,"15min":1.6100000000000001},"min":{"1min":1.304,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.1280000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.2789999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62952,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308888,"statfs":{"total":96636764160,"available":96572301312,"internally_reserved":0,"allocated":1273856,"data_stored":637594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.85999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.268,"5min":1.859,"15min":1.859},"last":0.374},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.42699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.331,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.093,"15min":1.129},"min":{"1min":0.63400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.528,"5min":1.907,"15min":1.9390000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.206,"15min":1.2549999999999999},"min":{"1min":0.64900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.657,"5min":1.881,"15min":1.881},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.0580000000000001,"15min":1.1240000000000001},"min":{"1min":0.60999999999999999,"5min":0.438,"15min":0.379},"max":{"1min":1.7230000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3240000000000001,"15min":1.3640000000000001},"min":{"1min":0.69799999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.6919999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3600000000000001,"15min":1.369},"min":{"1min":0.94499999999999995,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.958,"5min":1.958,"15min":2.206},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.415,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.825,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.034}]},{"osd":5,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.343},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":2.024,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1099999999999999,"5min":2.1349999999999998,"15min":2.4870000000000001},"last":1.121}]},{"osd":6,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4470000000000001,"15min":1.462},"min":{"1min":0.93000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7589999999999999,"5min":2.6819999999999999,"15min":2.698},"last":1.131},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.4359999999999999},"min":{"1min":0.84999999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8200000000000001,"5min":2.552,"15min":2.6179999999999999},"last":0.83399999999999996}]},{"osd":7,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.494,"15min":1.5309999999999999},"min":{"1min":0.88,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.863,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.395},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4590000000000001,"15min":1.5129999999999999},"min":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.865,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.325}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1269760,"data_stored":636964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88500000000000001,"15min":0.91000000000000003},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.323,"5min":1.905,"15min":1.905},"last":1.032},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.071,"15min":1.0740000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6930000000000001,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":1.3169999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":0.997},"min":{"1min":0.53500000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4419999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":1.121},{"interface":"front","average":{"1min":1.133,"5min":1.157,"15min":1.1850000000000001},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.974,"15min":2.29},"last":1.2450000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.244,"15min":1.3},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7350000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5489999999999999,"5min":1.877,"15min":2.3620000000000001},"last":0.96099999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2589999999999999,"15min":1.276},"min":{"1min":0.88500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.71,"5min":1.944,"15min":1.944},"last":1.377},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.833,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.4319999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.4610000000000001,"15min":1.5049999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.75900000000000001},"max":{"1min":1.9099999999999999,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.458,"15min":1.4790000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.653}]},{"osd":6,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4790000000000001,"15min":1.5109999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.984,"5min":2.2469999999999999,"15min":4.1260000000000003},"last":1.48},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4730000000000001},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9510000000000001,"5min":2.2090000000000001,"15min":3.101},"last":1.825}]},{"osd":7,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5820000000000001,"15min":1.609},"min":{"1min":0.872,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2530000000000001,"5min":2.2679999999999998,"15min":2.8180000000000001},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5569999999999999,"15min":1.613},"min":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.901,"5min":2.1619999999999999,"15min":3.952},"last":1.5660000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28836,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1855488,"data_stored":1227220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.583,"5min":1.782,"15min":2.9260000000000002},"last":0.87},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0569999999999999},"min":{"1min":0.66000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.075,"15min":1.123},"min":{"1min":0.80600000000000005,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":1.6559999999999999,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.125,"15min":1.147},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.42999999999999999},"max":{"1min":1.603,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.198}]},{"osd":3,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.093},"min":{"1min":0.67200000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.702,"5min":1.8660000000000001,"15min":2.1429999999999998},"last":1.234},{"interface":"front","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.29},"min":{"1min":0.91400000000000003,"5min":0.48599999999999999,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8109999999999999,"15min":3.052},"last":1.278}]},{"osd":4,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3939999999999999,"15min":1.417},"min":{"1min":0.98699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.756,"5min":2.0339999999999998,"15min":3.3580000000000001},"last":1.405},{"interface":"front","average":{"1min":1.546,"5min":1.4950000000000001,"15min":1.5309999999999999},"min":{"1min":1.3069999999999999,"5min":0.872,"15min":0.872},"max":{"1min":1.863,"5min":2.246,"15min":3.306},"last":1.778}]},{"osd":5,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3859999999999999},"min":{"1min":1.0129999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6279999999999999,"5min":1.9419999999999999,"15min":3.8849999999999998},"last":1.329},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.454,"15min":1.504},"min":{"1min":1.046,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8089999999999999,"5min":2.137,"15min":3.1789999999999998},"last":1.635}]},{"osd":6,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4419999999999999,"15min":1.4850000000000001},"min":{"1min":0.95799999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.8069999999999999,"5min":2.5379999999999998,"15min":4.3019999999999996},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5489999999999999,"15min":1.595},"min":{"1min":1.2030000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8069999999999999,"5min":2.355,"15min":3.5139999999999998},"last":1.5329999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4330000000000001,"15min":1.4610000000000001},"min":{"1min":1.0389999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.637,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.71},{"interface":"front","average":{"1min":1.476,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":1.0960000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.802,"5min":2.077,"15min":4.6219999999999999},"last":1.4630000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:18:09.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:18:10.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.170+0000 7fef366e0640 1 Processor -- start 2023-10-21T16:18:10.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.170+0000 7fef366e0640 1 -- start start 2023-10-21T16:18:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef3015ab40 0x7fef300ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fef300c90b0 0x7fef300adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300cae70 0x7fef300cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3015bf30 con 0x7fef3015ab40 2023-10-21T16:18:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef300cb840 con 0x7fef300c90b0 2023-10-21T16:18:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.171+0000 7fef366e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3015b2b0 con 0x7fef300cae70 2023-10-21T16:18:10.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef3015ab40 0x7fef300ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef3015ab40 0x7fef300ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53352/0 (socket says 172.21.15.53:53352) 2023-10-21T16:18:10.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 -- 172.21.15.53:0/914928775 learned_addr learned my addr 172.21.15.53:0/914928775 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:10.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef34c7d640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300cae70 0x7fef300cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 -- 172.21.15.53:0/914928775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300cae70 msgr2=0x7fef300cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:10.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300cae70 0x7fef300cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2ffff640 1 -- 172.21.15.53:0/914928775 >> v2:172.21.15.124:3300/0 conn(0x7fef300c90b0 msgr2=0x7fef300adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.172+0000 7fef2f7fe640 1 --2- 172.21.15.53:0/914928775 >> v2:172.21.15.124:3300/0 conn(0x7fef300c90b0 0x7fef300adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.173+0000 7fef2ffff640 1 --2- 172.21.15.53:0/914928775 >> v2:172.21.15.124:3300/0 conn(0x7fef300c90b0 0x7fef300adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.173+0000 7fef2ffff640 1 -- 172.21.15.53:0/914928775 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3015b590 con 0x7fef3015ab40 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.174+0000 7fef2ffff640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef300ad720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fef3006e560 tx=0x7fef2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea37f840585117a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.174+0000 7fef2effd640 1 -- 172.21.15.53:0/914928775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20031870 con 0x7fef3015ab40 2023-10-21T16:18:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.175+0000 7fef2effd640 1 -- 172.21.15.53:0/914928775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef200319d0 con 0x7fef3015ab40 2023-10-21T16:18:10.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.175+0000 7fef2effd640 1 -- 172.21.15.53:0/914928775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20031d60 con 0x7fef3015ab40 2023-10-21T16:18:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.175+0000 7fef366e0640 1 -- 172.21.15.53:0/914928775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 msgr2=0x7fef300ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.175+0000 7fef366e0640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef300ad720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fef3006e560 tx=0x7fef2002b910 comp rx=0 tx=0).stop 2023-10-21T16:18:10.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 -- 172.21.15.53:0/914928775 shutdown_connections 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300cae70 0x7fef300cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 --2- 172.21.15.53:0/914928775 >> v2:172.21.15.124:3300/0 conn(0x7fef300c90b0 0x7fef300adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 --2- 172.21.15.53:0/914928775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef300ad720 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 -- 172.21.15.53:0/914928775 >> 172.21.15.53:0/914928775 conn(0x7fef3006a380 msgr2=0x7fef300b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 -- 172.21.15.53:0/914928775 shutdown_connections 2023-10-21T16:18:10.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.176+0000 7fef366e0640 1 -- 172.21.15.53:0/914928775 wait complete. 2023-10-21T16:18:10.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.178+0000 7fef366e0640 1 Processor -- start 2023-10-21T16:18:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.178+0000 7fef366e0640 1 -- start start 2023-10-21T16:18:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300c90b0 0x7fef300c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 0x7fef301480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef300c5f70 con 0x7fef3015ab40 2023-10-21T16:18:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30150a80 con 0x7fef300cae70 2023-10-21T16:18:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.179+0000 7fef366e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef30150d60 con 0x7fef300c90b0 2023-10-21T16:18:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef34c7d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef34c7d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53358/0 (socket says 172.21.15.53:53358) 2023-10-21T16:18:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef34c7d640 1 -- 172.21.15.53:0/3345731814 learned_addr learned my addr 172.21.15.53:0/3345731814 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef2ffff640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300c90b0 0x7fef300c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef2f7fe640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 0x7fef301480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef34c7d640 1 -- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300c90b0 msgr2=0x7fef300c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.180+0000 7fef34c7d640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300c90b0 0x7fef300c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef34c7d640 1 -- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 msgr2=0x7fef301480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef34c7d640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 0x7fef301480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef34c7d640 1 -- 172.21.15.53:0/3345731814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200094d0 con 0x7fef3015ab40 2023-10-21T16:18:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef2f7fe640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 0x7fef301480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:10.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef34c7d640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fef2800cc30 tx=0x7fef2800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800ce40 con 0x7fef3015ab40 2023-10-21T16:18:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef28003990 con 0x7fef3015ab40 2023-10-21T16:18:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef30151040 con 0x7fef3015ab40 2023-10-21T16:18:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28003d20 con 0x7fef3015ab40 2023-10-21T16:18:10.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.181+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef30151600 con 0x7fef3015ab40 2023-10-21T16:18:10.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.183+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef30151820 con 0x7fef3015ab40 2023-10-21T16:18:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.184+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fef28003e80 con 0x7fef3015ab40 2023-10-21T16:18:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.185+0000 7fef2cff9640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 0x7fef1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.185+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fef28089cd0 con 0x7fef3015ab40 2023-10-21T16:18:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.185+0000 7fef2ffff640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 0x7fef1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.187+0000 7fef2ffff640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 0x7fef1c06d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef20036000 tx=0x7fef2003c040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.189+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef28054ac0 con 0x7fef3015ab40 2023-10-21T16:18:10.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.385+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 --> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef3015bb50 con 0x7fef1c06af00 2023-10-21T16:18:10.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.411+0000 7fef2cff9640 1 -- 172.21.15.53:0/3345731814 <== mgr.6082 v2:172.21.15.124:6832/3674677134 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287144 (secure 0 0 0) 0x7fef20031d10 con 0x7fef1c06af00 2023-10-21T16:18:10.416 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:10.418 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:18:10.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.416+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 msgr2=0x7fef1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.416+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 0x7fef1c06d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef20036000 tx=0x7fef2003c040 comp rx=0 tx=0).stop 2023-10-21T16:18:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.416+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 msgr2=0x7fef30150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.416+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fef2800cc30 tx=0x7fef2800a5c0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 shutdown_connections 2023-10-21T16:18:10.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fef1c06af00 0x7fef1c06d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef3015ab40 0x7fef30150010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef300cae70 0x7fef301480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 --2- 172.21.15.53:0/3345731814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef300c90b0 0x7fef300c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 >> 172.21.15.53:0/3345731814 conn(0x7fef3006a380 msgr2=0x7fef300ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:10.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 shutdown_connections 2023-10-21T16:18:10.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.417+0000 7fef366e0640 1 -- 172.21.15.53:0/3345731814 wait complete. 2023-10-21T16:18:10.439 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:18:08.690246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":285,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2194,"ondisk_log_size":2194,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":317416,"kb_used_data":11056,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754657304,"statfs":{"total":773094113280,"available":772769079296,"internally_reserved":0,"allocated":11321344,"data_stored":6267044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":313577017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.352037"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376046+0000","last_change":"2023-10-21T16:18:05.396808+0000","last_active":"2023-10-21T16:18:06.376046+0000","last_peered":"2023-10-21T16:18:06.376046+0000","last_clean":"2023-10-21T16:18:06.376046+0000","last_became_active":"2023-10-21T16:18:05.396507+0000","last_became_peered":"2023-10-21T16:18:05.396507+0000","last_unstale":"2023-10-21T16:18:06.376046+0000","last_undegraded":"2023-10-21T16:18:06.376046+0000","last_fullsized":"2023-10-21T16:18:06.376046+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:08.440136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365373+0000","last_change":"2023-10-21T16:18:05.379047+0000","last_active":"2023-10-21T16:18:06.365373+0000","last_peered":"2023-10-21T16:18:06.365373+0000","last_clean":"2023-10-21T16:18:06.365373+0000","last_became_active":"2023-10-21T16:18:05.377590+0000","last_became_peered":"2023-10-21T16:18:05.377590+0000","last_unstale":"2023-10-21T16:18:06.365373+0000","last_undegraded":"2023-10-21T16:18:06.365373+0000","last_fullsized":"2023-10-21T16:18:06.365373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:55.248105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093191+0000","last_change":"2023-10-21T16:18:05.398947+0000","last_active":"2023-10-21T16:18:07.093191+0000","last_peered":"2023-10-21T16:18:07.093191+0000","last_clean":"2023-10-21T16:18:07.093191+0000","last_became_active":"2023-10-21T16:18:05.398585+0000","last_became_peered":"2023-10-21T16:18:05.398585+0000","last_unstale":"2023-10-21T16:18:07.093191+0000","last_undegraded":"2023-10-21T16:18:07.093191+0000","last_fullsized":"2023-10-21T16:18:07.093191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:36.442174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093833+0000","last_change":"2023-10-21T16:18:05.396769+0000","last_active":"2023-10-21T16:18:07.093833+0000","last_peered":"2023-10-21T16:18:07.093833+0000","last_clean":"2023-10-21T16:18:07.093833+0000","last_became_active":"2023-10-21T16:18:05.396234+0000","last_became_peered":"2023-10-21T16:18:05.396234+0000","last_unstale":"2023-10-21T16:18:07.093833+0000","last_undegraded":"2023-10-21T16:18:07.093833+0000","last_fullsized":"2023-10-21T16:18:07.093833+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:34:13.629672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366717+0000","last_change":"2023-10-21T16:18:05.382529+0000","last_active":"2023-10-21T16:18:06.366717+0000","last_peered":"2023-10-21T16:18:06.366717+0000","last_clean":"2023-10-21T16:18:06.366717+0000","last_became_active":"2023-10-21T16:18:05.382111+0000","last_became_peered":"2023-10-21T16:18:05.382111+0000","last_unstale":"2023-10-21T16:18:06.366717+0000","last_undegraded":"2023-10-21T16:18:06.366717+0000","last_fullsized":"2023-10-21T16:18:06.366717+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:06.834456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093841+0000","last_change":"2023-10-21T16:18:05.399970+0000","last_active":"2023-10-21T16:18:07.093841+0000","last_peered":"2023-10-21T16:18:07.093841+0000","last_clean":"2023-10-21T16:18:07.093841+0000","last_became_active":"2023-10-21T16:18:05.399636+0000","last_became_peered":"2023-10-21T16:18:05.399636+0000","last_unstale":"2023-10-21T16:18:07.093841+0000","last_undegraded":"2023-10-21T16:18:07.093841+0000","last_fullsized":"2023-10-21T16:18:07.093841+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:49.458598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094127+0000","last_change":"2023-10-21T16:18:05.399901+0000","last_active":"2023-10-21T16:18:07.094127+0000","last_peered":"2023-10-21T16:18:07.094127+0000","last_clean":"2023-10-21T16:18:07.094127+0000","last_became_active":"2023-10-21T16:18:05.399626+0000","last_became_peered":"2023-10-21T16:18:05.399626+0000","last_unstale":"2023-10-21T16:18:07.094127+0000","last_undegraded":"2023-10-21T16:18:07.094127+0000","last_fullsized":"2023-10-21T16:18:07.094127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:51.657609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366068+0000","last_change":"2023-10-21T16:18:05.387836+0000","last_active":"2023-10-21T16:18:06.366068+0000","last_peered":"2023-10-21T16:18:06.366068+0000","last_clean":"2023-10-21T16:18:06.366068+0000","last_became_active":"2023-10-21T16:18:05.385582+0000","last_became_peered":"2023-10-21T16:18:05.385582+0000","last_unstale":"2023-10-21T16:18:06.366068+0000","last_undegraded":"2023-10-21T16:18:06.366068+0000","last_fullsized":"2023-10-21T16:18:06.366068+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:28:55.456430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369054+0000","last_change":"2023-10-21T16:18:05.391819+0000","last_active":"2023-10-21T16:18:06.369054+0000","last_peered":"2023-10-21T16:18:06.369054+0000","last_clean":"2023-10-21T16:18:06.369054+0000","last_became_active":"2023-10-21T16:18:05.391540+0000","last_became_peered":"2023-10-21T16:18:05.391540+0000","last_unstale":"2023-10-21T16:18:06.369054+0000","last_undegraded":"2023-10-21T16:18:06.369054+0000","last_fullsized":"2023-10-21T16:18:06.369054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:18.452219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093281+0000","last_change":"2023-10-21T16:18:05.392217+0000","last_active":"2023-10-21T16:18:07.093281+0000","last_peered":"2023-10-21T16:18:07.093281+0000","last_clean":"2023-10-21T16:18:07.093281+0000","last_became_active":"2023-10-21T16:18:05.391830+0000","last_became_peered":"2023-10-21T16:18:05.391830+0000","last_unstale":"2023-10-21T16:18:07.093281+0000","last_undegraded":"2023-10-21T16:18:07.093281+0000","last_fullsized":"2023-10-21T16:18:07.093281+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:39:08.492274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365373+0000","last_change":"2023-10-21T16:18:05.382643+0000","last_active":"2023-10-21T16:18:06.365373+0000","last_peered":"2023-10-21T16:18:06.365373+0000","last_clean":"2023-10-21T16:18:06.365373+0000","last_became_active":"2023-10-21T16:18:05.382327+0000","last_became_peered":"2023-10-21T16:18:05.382327+0000","last_unstale":"2023-10-21T16:18:06.365373+0000","last_undegraded":"2023-10-21T16:18:06.365373+0000","last_fullsized":"2023-10-21T16:18:06.365373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:54.901574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094175+0000","last_change":"2023-10-21T16:18:05.404568+0000","last_active":"2023-10-21T16:18:07.094175+0000","last_peered":"2023-10-21T16:18:07.094175+0000","last_clean":"2023-10-21T16:18:07.094175+0000","last_became_active":"2023-10-21T16:18:05.404165+0000","last_became_peered":"2023-10-21T16:18:05.404165+0000","last_unstale":"2023-10-21T16:18:07.094175+0000","last_undegraded":"2023-10-21T16:18:07.094175+0000","last_fullsized":"2023-10-21T16:18:07.094175+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:10.950082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366876+0000","last_change":"2023-10-21T16:18:05.392101+0000","last_active":"2023-10-21T16:18:06.366876+0000","last_peered":"2023-10-21T16:18:06.366876+0000","last_clean":"2023-10-21T16:18:06.366876+0000","last_became_active":"2023-10-21T16:18:05.391799+0000","last_became_peered":"2023-10-21T16:18:05.391799+0000","last_unstale":"2023-10-21T16:18:06.366876+0000","last_undegraded":"2023-10-21T16:18:06.366876+0000","last_fullsized":"2023-10-21T16:18:06.366876+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:11:30.412362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094248+0000","last_change":"2023-10-21T16:18:05.398971+0000","last_active":"2023-10-21T16:18:07.094248+0000","last_peered":"2023-10-21T16:18:07.094248+0000","last_clean":"2023-10-21T16:18:07.094248+0000","last_became_active":"2023-10-21T16:18:05.398695+0000","last_became_peered":"2023-10-21T16:18:05.398695+0000","last_unstale":"2023-10-21T16:18:07.094248+0000","last_undegraded":"2023-10-21T16:18:07.094248+0000","last_fullsized":"2023-10-21T16:18:07.094248+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:31:05.382950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363850+0000","last_change":"2023-10-21T16:18:05.397744+0000","last_active":"2023-10-21T16:18:06.363850+0000","last_peered":"2023-10-21T16:18:06.363850+0000","last_clean":"2023-10-21T16:18:06.363850+0000","last_became_active":"2023-10-21T16:18:05.396547+0000","last_became_peered":"2023-10-21T16:18:05.396547+0000","last_unstale":"2023-10-21T16:18:06.363850+0000","last_undegraded":"2023-10-21T16:18:06.363850+0000","last_fullsized":"2023-10-21T16:18:06.363850+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:12:53.122599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375002+0000","last_change":"2023-10-21T16:18:05.391234+0000","last_active":"2023-10-21T16:18:06.375002+0000","last_peered":"2023-10-21T16:18:06.375002+0000","last_clean":"2023-10-21T16:18:06.375002+0000","last_became_active":"2023-10-21T16:18:05.390851+0000","last_became_peered":"2023-10-21T16:18:05.390851+0000","last_unstale":"2023-10-21T16:18:06.375002+0000","last_undegraded":"2023-10-21T16:18:06.375002+0000","last_fullsized":"2023-10-21T16:18:06.375002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:42.102036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367153+0000","last_change":"2023-10-21T16:18:05.402479+0000","last_active":"2023-10-21T16:18:06.367153+0000","last_peered":"2023-10-21T16:18:06.367153+0000","last_clean":"2023-10-21T16:18:06.367153+0000","last_became_active":"2023-10-21T16:18:05.402177+0000","last_became_peered":"2023-10-21T16:18:05.402177+0000","last_unstale":"2023-10-21T16:18:06.367153+0000","last_undegraded":"2023-10-21T16:18:06.367153+0000","last_fullsized":"2023-10-21T16:18:06.367153+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:39.962742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095010+0000","last_change":"2023-10-21T16:18:05.402143+0000","last_active":"2023-10-21T16:18:07.095010+0000","last_peered":"2023-10-21T16:18:07.095010+0000","last_clean":"2023-10-21T16:18:07.095010+0000","last_became_active":"2023-10-21T16:18:05.401838+0000","last_became_peered":"2023-10-21T16:18:05.401838+0000","last_unstale":"2023-10-21T16:18:07.095010+0000","last_undegraded":"2023-10-21T16:18:07.095010+0000","last_fullsized":"2023-10-21T16:18:07.095010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:32.094965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094191+0000","last_change":"2023-10-21T16:18:05.398158+0000","last_active":"2023-10-21T16:18:07.094191+0000","last_peered":"2023-10-21T16:18:07.094191+0000","last_clean":"2023-10-21T16:18:07.094191+0000","last_became_active":"2023-10-21T16:18:05.397845+0000","last_became_peered":"2023-10-21T16:18:05.397845+0000","last_unstale":"2023-10-21T16:18:07.094191+0000","last_undegraded":"2023-10-21T16:18:07.094191+0000","last_fullsized":"2023-10-21T16:18:07.094191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:16.992159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367335+0000","last_change":"2023-10-21T16:18:05.406280+0000","last_active":"2023-10-21T16:18:06.367335+0000","last_peered":"2023-10-21T16:18:06.367335+0000","last_clean":"2023-10-21T16:18:06.367335+0000","last_became_active":"2023-10-21T16:18:05.405701+0000","last_became_peered":"2023-10-21T16:18:05.405701+0000","last_unstale":"2023-10-21T16:18:06.367335+0000","last_undegraded":"2023-10-21T16:18:06.367335+0000","last_fullsized":"2023-10-21T16:18:06.367335+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:46.294056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367941+0000","last_change":"2023-10-21T16:18:05.400681+0000","last_active":"2023-10-21T16:18:06.367941+0000","last_peered":"2023-10-21T16:18:06.367941+0000","last_clean":"2023-10-21T16:18:06.367941+0000","last_became_active":"2023-10-21T16:18:05.400385+0000","last_became_peered":"2023-10-21T16:18:05.400385+0000","last_unstale":"2023-10-21T16:18:06.367941+0000","last_undegraded":"2023-10-21T16:18:06.367941+0000","last_fullsized":"2023-10-21T16:18:06.367941+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:36.590042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366782+0000","last_change":"2023-10-21T16:18:05.393511+0000","last_active":"2023-10-21T16:18:06.366782+0000","last_peered":"2023-10-21T16:18:06.366782+0000","last_clean":"2023-10-21T16:18:06.366782+0000","last_became_active":"2023-10-21T16:18:05.393194+0000","last_became_peered":"2023-10-21T16:18:05.393194+0000","last_unstale":"2023-10-21T16:18:06.366782+0000","last_undegraded":"2023-10-21T16:18:06.366782+0000","last_fullsized":"2023-10-21T16:18:06.366782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:35.219927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367619+0000","last_change":"2023-10-21T16:18:05.393172+0000","last_active":"2023-10-21T16:18:06.367619+0000","last_peered":"2023-10-21T16:18:06.367619+0000","last_clean":"2023-10-21T16:18:06.367619+0000","last_became_active":"2023-10-21T16:18:05.392885+0000","last_became_peered":"2023-10-21T16:18:05.392885+0000","last_unstale":"2023-10-21T16:18:06.367619+0000","last_undegraded":"2023-10-21T16:18:06.367619+0000","last_fullsized":"2023-10-21T16:18:06.367619+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:48:14.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363795+0000","last_change":"2023-10-21T16:18:05.400615+0000","last_active":"2023-10-21T16:18:06.363795+0000","last_peered":"2023-10-21T16:18:06.363795+0000","last_clean":"2023-10-21T16:18:06.363795+0000","last_became_active":"2023-10-21T16:18:05.395354+0000","last_became_peered":"2023-10-21T16:18:05.395354+0000","last_unstale":"2023-10-21T16:18:06.363795+0000","last_undegraded":"2023-10-21T16:18:06.363795+0000","last_fullsized":"2023-10-21T16:18:06.363795+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:44.375485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365876+0000","last_change":"2023-10-21T16:18:05.400022+0000","last_active":"2023-10-21T16:18:06.365876+0000","last_peered":"2023-10-21T16:18:06.365876+0000","last_clean":"2023-10-21T16:18:06.365876+0000","last_became_active":"2023-10-21T16:18:05.394831+0000","last_became_peered":"2023-10-21T16:18:05.394831+0000","last_unstale":"2023-10-21T16:18:06.365876+0000","last_undegraded":"2023-10-21T16:18:06.365876+0000","last_fullsized":"2023-10-21T16:18:06.365876+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:00.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093674+0000","last_change":"2023-10-21T16:18:05.399255+0000","last_active":"2023-10-21T16:18:07.093674+0000","last_peered":"2023-10-21T16:18:07.093674+0000","last_clean":"2023-10-21T16:18:07.093674+0000","last_became_active":"2023-10-21T16:18:05.398954+0000","last_became_peered":"2023-10-21T16:18:05.398954+0000","last_unstale":"2023-10-21T16:18:07.093674+0000","last_undegraded":"2023-10-21T16:18:07.093674+0000","last_fullsized":"2023-10-21T16:18:07.093674+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:24:55.840171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375912+0000","last_change":"2023-10-21T16:18:05.391594+0000","last_active":"2023-10-21T16:18:06.375912+0000","last_peered":"2023-10-21T16:18:06.375912+0000","last_clean":"2023-10-21T16:18:06.375912+0000","last_became_active":"2023-10-21T16:18:05.390756+0000","last_became_peered":"2023-10-21T16:18:05.390756+0000","last_unstale":"2023-10-21T16:18:06.375912+0000","last_undegraded":"2023-10-21T16:18:06.375912+0000","last_fullsized":"2023-10-21T16:18:06.375912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:08:00.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366992+0000","last_change":"2023-10-21T16:18:05.392153+0000","last_active":"2023-10-21T16:18:06.366992+0000","last_peered":"2023-10-21T16:18:06.366992+0000","last_clean":"2023-10-21T16:18:06.366992+0000","last_became_active":"2023-10-21T16:18:05.391727+0000","last_became_peered":"2023-10-21T16:18:05.391727+0000","last_unstale":"2023-10-21T16:18:06.366992+0000","last_undegraded":"2023-10-21T16:18:06.366992+0000","last_fullsized":"2023-10-21T16:18:06.366992+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:32.663692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365427+0000","last_change":"2023-10-21T16:18:05.402166+0000","last_active":"2023-10-21T16:18:06.365427+0000","last_peered":"2023-10-21T16:18:06.365427+0000","last_clean":"2023-10-21T16:18:06.365427+0000","last_became_active":"2023-10-21T16:18:05.401860+0000","last_became_peered":"2023-10-21T16:18:05.401860+0000","last_unstale":"2023-10-21T16:18:06.365427+0000","last_undegraded":"2023-10-21T16:18:06.365427+0000","last_fullsized":"2023-10-21T16:18:06.365427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:45.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375128+0000","last_change":"2023-10-21T16:18:05.397381+0000","last_active":"2023-10-21T16:18:06.375128+0000","last_peered":"2023-10-21T16:18:06.375128+0000","last_clean":"2023-10-21T16:18:06.375128+0000","last_became_active":"2023-10-21T16:18:05.397087+0000","last_became_peered":"2023-10-21T16:18:05.397087+0000","last_unstale":"2023-10-21T16:18:06.375128+0000","last_undegraded":"2023-10-21T16:18:06.375128+0000","last_fullsized":"2023-10-21T16:18:06.375128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:49.652191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092476+0000","last_change":"2023-10-21T16:18:05.400010+0000","last_active":"2023-10-21T16:18:07.092476+0000","last_peered":"2023-10-21T16:18:07.092476+0000","last_clean":"2023-10-21T16:18:07.092476+0000","last_became_active":"2023-10-21T16:18:05.399701+0000","last_became_peered":"2023-10-21T16:18:05.399701+0000","last_unstale":"2023-10-21T16:18:07.092476+0000","last_undegraded":"2023-10-21T16:18:07.092476+0000","last_fullsized":"2023-10-21T16:18:07.092476+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:31.910411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375003+0000","last_change":"2023-10-21T16:18:05.397565+0000","last_active":"2023-10-21T16:18:06.375003+0000","last_peered":"2023-10-21T16:18:06.375003+0000","last_clean":"2023-10-21T16:18:06.375003+0000","last_became_active":"2023-10-21T16:18:05.397266+0000","last_became_peered":"2023-10-21T16:18:05.397266+0000","last_unstale":"2023-10-21T16:18:06.375003+0000","last_undegraded":"2023-10-21T16:18:06.375003+0000","last_fullsized":"2023-10-21T16:18:06.375003+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:45.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092677+0000","last_change":"2023-10-21T16:18:05.397310+0000","last_active":"2023-10-21T16:18:07.092677+0000","last_peered":"2023-10-21T16:18:07.092677+0000","last_clean":"2023-10-21T16:18:07.092677+0000","last_became_active":"2023-10-21T16:18:05.397035+0000","last_became_peered":"2023-10-21T16:18:05.397035+0000","last_unstale":"2023-10-21T16:18:07.092677+0000","last_undegraded":"2023-10-21T16:18:07.092677+0000","last_fullsized":"2023-10-21T16:18:07.092677+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:49:18.452219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367202+0000","last_change":"2023-10-21T16:18:04.378787+0000","last_active":"2023-10-21T16:18:06.367202+0000","last_peered":"2023-10-21T16:18:06.367202+0000","last_clean":"2023-10-21T16:18:06.367202+0000","last_became_active":"2023-10-21T16:18:04.377509+0000","last_became_peered":"2023-10-21T16:18:04.377509+0000","last_unstale":"2023-10-21T16:18:06.367202+0000","last_undegraded":"2023-10-21T16:18:06.367202+0000","last_fullsized":"2023-10-21T16:18:06.367202+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:32.211922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365052+0000","last_change":"2023-10-21T16:18:05.389427+0000","last_active":"2023-10-21T16:18:06.365052+0000","last_peered":"2023-10-21T16:18:06.365052+0000","last_clean":"2023-10-21T16:18:06.365052+0000","last_became_active":"2023-10-21T16:18:05.387656+0000","last_became_peered":"2023-10-21T16:18:05.387656+0000","last_unstale":"2023-10-21T16:18:06.365052+0000","last_undegraded":"2023-10-21T16:18:06.365052+0000","last_fullsized":"2023-10-21T16:18:06.365052+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:17:12.299332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375859+0000","last_change":"2023-10-21T16:18:04.384992+0000","last_active":"2023-10-21T16:18:06.375859+0000","last_peered":"2023-10-21T16:18:06.375859+0000","last_clean":"2023-10-21T16:18:06.375859+0000","last_became_active":"2023-10-21T16:18:04.384727+0000","last_became_peered":"2023-10-21T16:18:04.384727+0000","last_unstale":"2023-10-21T16:18:06.375859+0000","last_undegraded":"2023-10-21T16:18:06.375859+0000","last_fullsized":"2023-10-21T16:18:06.375859+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:12:52.105434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368148+0000","last_change":"2023-10-21T16:18:05.395393+0000","last_active":"2023-10-21T16:18:06.368148+0000","last_peered":"2023-10-21T16:18:06.368148+0000","last_clean":"2023-10-21T16:18:06.368148+0000","last_became_active":"2023-10-21T16:18:05.395081+0000","last_became_peered":"2023-10-21T16:18:05.395081+0000","last_unstale":"2023-10-21T16:18:06.368148+0000","last_undegraded":"2023-10-21T16:18:06.368148+0000","last_fullsized":"2023-10-21T16:18:06.368148+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:31.335608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375833+0000","last_change":"2023-10-21T16:18:04.387051+0000","last_active":"2023-10-21T16:18:06.375833+0000","last_peered":"2023-10-21T16:18:06.375833+0000","last_clean":"2023-10-21T16:18:06.375833+0000","last_became_active":"2023-10-21T16:18:04.386709+0000","last_became_peered":"2023-10-21T16:18:04.386709+0000","last_unstale":"2023-10-21T16:18:06.375833+0000","last_undegraded":"2023-10-21T16:18:06.375833+0000","last_fullsized":"2023-10-21T16:18:06.375833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:52.599872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368294+0000","last_change":"2023-10-21T16:18:05.402389+0000","last_active":"2023-10-21T16:18:06.368294+0000","last_peered":"2023-10-21T16:18:06.368294+0000","last_clean":"2023-10-21T16:18:06.368294+0000","last_became_active":"2023-10-21T16:18:05.401075+0000","last_became_peered":"2023-10-21T16:18:05.401075+0000","last_unstale":"2023-10-21T16:18:06.368294+0000","last_undegraded":"2023-10-21T16:18:06.368294+0000","last_fullsized":"2023-10-21T16:18:06.368294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:32:55.693833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093505+0000","last_change":"2023-10-21T16:18:04.388551+0000","last_active":"2023-10-21T16:18:07.093505+0000","last_peered":"2023-10-21T16:18:07.093505+0000","last_clean":"2023-10-21T16:18:07.093505+0000","last_became_active":"2023-10-21T16:18:04.388062+0000","last_became_peered":"2023-10-21T16:18:04.388062+0000","last_unstale":"2023-10-21T16:18:07.093505+0000","last_undegraded":"2023-10-21T16:18:07.093505+0000","last_fullsized":"2023-10-21T16:18:07.093505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:39.716055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364644+0000","last_change":"2023-10-21T16:18:05.391581+0000","last_active":"2023-10-21T16:18:06.364644+0000","last_peered":"2023-10-21T16:18:06.364644+0000","last_clean":"2023-10-21T16:18:06.364644+0000","last_became_active":"2023-10-21T16:18:05.390834+0000","last_became_peered":"2023-10-21T16:18:05.390834+0000","last_unstale":"2023-10-21T16:18:06.364644+0000","last_undegraded":"2023-10-21T16:18:06.364644+0000","last_fullsized":"2023-10-21T16:18:06.364644+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:53.617037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368553+0000","last_change":"2023-10-21T16:18:04.386427+0000","last_active":"2023-10-21T16:18:06.368553+0000","last_peered":"2023-10-21T16:18:06.368553+0000","last_clean":"2023-10-21T16:18:06.368553+0000","last_became_active":"2023-10-21T16:18:04.385256+0000","last_became_peered":"2023-10-21T16:18:04.385256+0000","last_unstale":"2023-10-21T16:18:06.368553+0000","last_undegraded":"2023-10-21T16:18:06.368553+0000","last_fullsized":"2023-10-21T16:18:06.368553+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:05.761467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365561+0000","last_change":"2023-10-21T16:18:05.387666+0000","last_active":"2023-10-21T16:18:06.365561+0000","last_peered":"2023-10-21T16:18:06.365561+0000","last_clean":"2023-10-21T16:18:06.365561+0000","last_became_active":"2023-10-21T16:18:05.384640+0000","last_became_peered":"2023-10-21T16:18:05.384640+0000","last_unstale":"2023-10-21T16:18:06.365561+0000","last_undegraded":"2023-10-21T16:18:06.365561+0000","last_fullsized":"2023-10-21T16:18:06.365561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:29:42.102036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093789+0000","last_change":"2023-10-21T16:18:04.383384+0000","last_active":"2023-10-21T16:18:07.093789+0000","last_peered":"2023-10-21T16:18:07.093789+0000","last_clean":"2023-10-21T16:18:07.093789+0000","last_became_active":"2023-10-21T16:18:04.383031+0000","last_became_peered":"2023-10-21T16:18:04.383031+0000","last_unstale":"2023-10-21T16:18:07.093789+0000","last_undegraded":"2023-10-21T16:18:07.093789+0000","last_fullsized":"2023-10-21T16:18:07.093789+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:07:59.695108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369438+0000","last_change":"2023-10-21T16:18:05.393645+0000","last_active":"2023-10-21T16:18:06.369438+0000","last_peered":"2023-10-21T16:18:06.369438+0000","last_clean":"2023-10-21T16:18:06.369438+0000","last_became_active":"2023-10-21T16:18:05.393310+0000","last_became_peered":"2023-10-21T16:18:05.393310+0000","last_unstale":"2023-10-21T16:18:06.369438+0000","last_undegraded":"2023-10-21T16:18:06.369438+0000","last_fullsized":"2023-10-21T16:18:06.369438+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:59.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364774+0000","last_change":"2023-10-21T16:18:04.386037+0000","last_active":"2023-10-21T16:18:06.364774+0000","last_peered":"2023-10-21T16:18:06.364774+0000","last_clean":"2023-10-21T16:18:06.364774+0000","last_became_active":"2023-10-21T16:18:04.385281+0000","last_became_peered":"2023-10-21T16:18:04.385281+0000","last_unstale":"2023-10-21T16:18:06.364774+0000","last_undegraded":"2023-10-21T16:18:06.364774+0000","last_fullsized":"2023-10-21T16:18:06.364774+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:39:31.904699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093966+0000","last_change":"2023-10-21T16:18:05.400829+0000","last_active":"2023-10-21T16:18:07.093966+0000","last_peered":"2023-10-21T16:18:07.093966+0000","last_clean":"2023-10-21T16:18:07.093966+0000","last_became_active":"2023-10-21T16:18:05.400532+0000","last_became_peered":"2023-10-21T16:18:05.400532+0000","last_unstale":"2023-10-21T16:18:07.093966+0000","last_undegraded":"2023-10-21T16:18:07.093966+0000","last_fullsized":"2023-10-21T16:18:07.093966+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:41.497296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366158+0000","last_change":"2023-10-21T16:18:04.383639+0000","last_active":"2023-10-21T16:18:06.366158+0000","last_peered":"2023-10-21T16:18:06.366158+0000","last_clean":"2023-10-21T16:18:06.366158+0000","last_became_active":"2023-10-21T16:18:04.383170+0000","last_became_peered":"2023-10-21T16:18:04.383170+0000","last_unstale":"2023-10-21T16:18:06.366158+0000","last_undegraded":"2023-10-21T16:18:06.366158+0000","last_fullsized":"2023-10-21T16:18:06.366158+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:52:12.529588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.363880+0000","last_change":"2023-10-21T16:18:05.381838+0000","last_active":"2023-10-21T16:18:06.363880+0000","last_peered":"2023-10-21T16:18:06.363880+0000","last_clean":"2023-10-21T16:18:06.363880+0000","last_became_active":"2023-10-21T16:18:05.380561+0000","last_became_peered":"2023-10-21T16:18:05.380561+0000","last_unstale":"2023-10-21T16:18:06.363880+0000","last_undegraded":"2023-10-21T16:18:06.363880+0000","last_fullsized":"2023-10-21T16:18:06.363880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:36.589138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376463+0000","last_change":"2023-10-21T16:18:04.393918+0000","last_active":"2023-10-21T16:18:06.376463+0000","last_peered":"2023-10-21T16:18:06.376463+0000","last_clean":"2023-10-21T16:18:06.376463+0000","last_became_active":"2023-10-21T16:18:04.393297+0000","last_became_peered":"2023-10-21T16:18:04.393297+0000","last_unstale":"2023-10-21T16:18:06.376463+0000","last_undegraded":"2023-10-21T16:18:06.376463+0000","last_fullsized":"2023-10-21T16:18:06.376463+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:33:53.884409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094262+0000","last_change":"2023-10-21T16:18:05.395917+0000","last_active":"2023-10-21T16:18:07.094262+0000","last_peered":"2023-10-21T16:18:07.094262+0000","last_clean":"2023-10-21T16:18:07.094262+0000","last_became_active":"2023-10-21T16:18:05.395605+0000","last_became_peered":"2023-10-21T16:18:05.395605+0000","last_unstale":"2023-10-21T16:18:07.094262+0000","last_undegraded":"2023-10-21T16:18:07.094262+0000","last_fullsized":"2023-10-21T16:18:07.094262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:04.771243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095189+0000","last_change":"2023-10-21T16:18:04.389454+0000","last_active":"2023-10-21T16:18:07.095189+0000","last_peered":"2023-10-21T16:18:07.095189+0000","last_clean":"2023-10-21T16:18:07.095189+0000","last_became_active":"2023-10-21T16:18:04.389121+0000","last_became_peered":"2023-10-21T16:18:04.389121+0000","last_unstale":"2023-10-21T16:18:07.095189+0000","last_undegraded":"2023-10-21T16:18:07.095189+0000","last_fullsized":"2023-10-21T16:18:07.095189+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:22.191569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094823+0000","last_change":"2023-10-21T16:18:05.408329+0000","last_active":"2023-10-21T16:18:07.094823+0000","last_peered":"2023-10-21T16:18:07.094823+0000","last_clean":"2023-10-21T16:18:07.094823+0000","last_became_active":"2023-10-21T16:18:05.408053+0000","last_became_peered":"2023-10-21T16:18:05.408053+0000","last_unstale":"2023-10-21T16:18:07.094823+0000","last_undegraded":"2023-10-21T16:18:07.094823+0000","last_fullsized":"2023-10-21T16:18:07.094823+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:31:02.308970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367213+0000","last_change":"2023-10-21T16:18:04.382155+0000","last_active":"2023-10-21T16:18:06.367213+0000","last_peered":"2023-10-21T16:18:06.367213+0000","last_clean":"2023-10-21T16:18:06.367213+0000","last_became_active":"2023-10-21T16:18:04.380887+0000","last_became_peered":"2023-10-21T16:18:04.380887+0000","last_unstale":"2023-10-21T16:18:06.367213+0000","last_undegraded":"2023-10-21T16:18:06.367213+0000","last_fullsized":"2023-10-21T16:18:06.367213+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:01.732635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093042+0000","last_change":"2023-10-21T16:18:05.398856+0000","last_active":"2023-10-21T16:18:07.093042+0000","last_peered":"2023-10-21T16:18:07.093042+0000","last_clean":"2023-10-21T16:18:07.093042+0000","last_became_active":"2023-10-21T16:18:05.398567+0000","last_became_peered":"2023-10-21T16:18:05.398567+0000","last_unstale":"2023-10-21T16:18:07.093042+0000","last_undegraded":"2023-10-21T16:18:07.093042+0000","last_fullsized":"2023-10-21T16:18:07.093042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:59.840543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093581+0000","last_change":"2023-10-21T16:18:04.390512+0000","last_active":"2023-10-21T16:18:07.093581+0000","last_peered":"2023-10-21T16:18:07.093581+0000","last_clean":"2023-10-21T16:18:07.093581+0000","last_became_active":"2023-10-21T16:18:04.389892+0000","last_became_peered":"2023-10-21T16:18:04.389892+0000","last_unstale":"2023-10-21T16:18:07.093581+0000","last_undegraded":"2023-10-21T16:18:07.093581+0000","last_fullsized":"2023-10-21T16:18:07.093581+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:37:35.914023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094536+0000","last_change":"2023-10-21T16:18:05.397866+0000","last_active":"2023-10-21T16:18:07.094536+0000","last_peered":"2023-10-21T16:18:07.094536+0000","last_clean":"2023-10-21T16:18:07.094536+0000","last_became_active":"2023-10-21T16:18:05.397458+0000","last_became_peered":"2023-10-21T16:18:05.397458+0000","last_unstale":"2023-10-21T16:18:07.094536+0000","last_undegraded":"2023-10-21T16:18:07.094536+0000","last_fullsized":"2023-10-21T16:18:07.094536+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:50.422461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094106+0000","last_change":"2023-10-21T16:18:04.384460+0000","last_active":"2023-10-21T16:18:07.094106+0000","last_peered":"2023-10-21T16:18:07.094106+0000","last_clean":"2023-10-21T16:18:07.094106+0000","last_became_active":"2023-10-21T16:18:04.384110+0000","last_became_peered":"2023-10-21T16:18:04.384110+0000","last_unstale":"2023-10-21T16:18:07.094106+0000","last_undegraded":"2023-10-21T16:18:07.094106+0000","last_fullsized":"2023-10-21T16:18:07.094106+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:46:53.605066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094124+0000","last_change":"2023-10-21T16:18:05.401680+0000","last_active":"2023-10-21T16:18:07.094124+0000","last_peered":"2023-10-21T16:18:07.094124+0000","last_clean":"2023-10-21T16:18:07.094124+0000","last_became_active":"2023-10-21T16:18:05.401209+0000","last_became_peered":"2023-10-21T16:18:05.401209+0000","last_unstale":"2023-10-21T16:18:07.094124+0000","last_undegraded":"2023-10-21T16:18:07.094124+0000","last_fullsized":"2023-10-21T16:18:07.094124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:06.778632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375392+0000","last_change":"2023-10-21T16:18:04.378706+0000","last_active":"2023-10-21T16:18:06.375392+0000","last_peered":"2023-10-21T16:18:06.375392+0000","last_clean":"2023-10-21T16:18:06.375392+0000","last_became_active":"2023-10-21T16:18:04.376187+0000","last_became_peered":"2023-10-21T16:18:04.376187+0000","last_unstale":"2023-10-21T16:18:06.375392+0000","last_undegraded":"2023-10-21T16:18:06.375392+0000","last_fullsized":"2023-10-21T16:18:06.375392+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:16.123954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093740+0000","last_change":"2023-10-21T16:18:05.398595+0000","last_active":"2023-10-21T16:18:07.093740+0000","last_peered":"2023-10-21T16:18:07.093740+0000","last_clean":"2023-10-21T16:18:07.093740+0000","last_became_active":"2023-10-21T16:18:05.398294+0000","last_became_peered":"2023-10-21T16:18:05.398294+0000","last_unstale":"2023-10-21T16:18:07.093740+0000","last_undegraded":"2023-10-21T16:18:07.093740+0000","last_fullsized":"2023-10-21T16:18:07.093740+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:34:02.300707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367166+0000","last_change":"2023-10-21T16:18:04.379131+0000","last_active":"2023-10-21T16:18:06.367166+0000","last_peered":"2023-10-21T16:18:06.367166+0000","last_clean":"2023-10-21T16:18:06.367166+0000","last_became_active":"2023-10-21T16:18:04.378830+0000","last_became_peered":"2023-10-21T16:18:04.378830+0000","last_unstale":"2023-10-21T16:18:06.367166+0000","last_undegraded":"2023-10-21T16:18:06.367166+0000","last_fullsized":"2023-10-21T16:18:06.367166+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:43.935138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094276+0000","last_change":"2023-10-21T16:18:05.398942+0000","last_active":"2023-10-21T16:18:07.094276+0000","last_peered":"2023-10-21T16:18:07.094276+0000","last_clean":"2023-10-21T16:18:07.094276+0000","last_became_active":"2023-10-21T16:18:05.398463+0000","last_became_peered":"2023-10-21T16:18:05.398463+0000","last_unstale":"2023-10-21T16:18:07.094276+0000","last_undegraded":"2023-10-21T16:18:07.094276+0000","last_fullsized":"2023-10-21T16:18:07.094276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:20.440829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"102'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364085+0000","last_change":"2023-10-21T16:18:04.387404+0000","last_active":"2023-10-21T16:18:06.364085+0000","last_peered":"2023-10-21T16:18:06.364085+0000","last_clean":"2023-10-21T16:18:06.364085+0000","last_became_active":"2023-10-21T16:18:04.387111+0000","last_became_peered":"2023-10-21T16:18:04.387111+0000","last_unstale":"2023-10-21T16:18:06.364085+0000","last_undegraded":"2023-10-21T16:18:06.364085+0000","last_fullsized":"2023-10-21T16:18:06.364085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:15.630667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092995+0000","last_change":"2023-10-21T16:18:05.400133+0000","last_active":"2023-10-21T16:18:07.092995+0000","last_peered":"2023-10-21T16:18:07.092995+0000","last_clean":"2023-10-21T16:18:07.092995+0000","last_became_active":"2023-10-21T16:18:05.399831+0000","last_became_peered":"2023-10-21T16:18:05.399831+0000","last_unstale":"2023-10-21T16:18:07.092995+0000","last_undegraded":"2023-10-21T16:18:07.092995+0000","last_fullsized":"2023-10-21T16:18:07.092995+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:24.680934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376569+0000","last_change":"2023-10-21T16:18:04.388088+0000","last_active":"2023-10-21T16:18:06.376569+0000","last_peered":"2023-10-21T16:18:06.376569+0000","last_clean":"2023-10-21T16:18:06.376569+0000","last_became_active":"2023-10-21T16:18:04.384451+0000","last_became_peered":"2023-10-21T16:18:04.384451+0000","last_unstale":"2023-10-21T16:18:06.376569+0000","last_undegraded":"2023-10-21T16:18:06.376569+0000","last_fullsized":"2023-10-21T16:18:06.376569+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:34.280219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366558+0000","last_change":"2023-10-21T16:18:05.402635+0000","last_active":"2023-10-21T16:18:06.366558+0000","last_peered":"2023-10-21T16:18:06.366558+0000","last_clean":"2023-10-21T16:18:06.366558+0000","last_became_active":"2023-10-21T16:18:05.402361+0000","last_became_peered":"2023-10-21T16:18:05.402361+0000","last_unstale":"2023-10-21T16:18:06.366558+0000","last_undegraded":"2023-10-21T16:18:06.366558+0000","last_fullsized":"2023-10-21T16:18:06.366558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:20.765430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.369270+0000","last_change":"2023-10-21T16:18:04.386426+0000","last_active":"2023-10-21T16:18:06.369270+0000","last_peered":"2023-10-21T16:18:06.369270+0000","last_clean":"2023-10-21T16:18:06.369270+0000","last_became_active":"2023-10-21T16:18:04.386104+0000","last_became_peered":"2023-10-21T16:18:04.386104+0000","last_unstale":"2023-10-21T16:18:06.369270+0000","last_undegraded":"2023-10-21T16:18:06.369270+0000","last_fullsized":"2023-10-21T16:18:06.369270+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:50.640444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366848+0000","last_change":"2023-10-21T16:18:05.386758+0000","last_active":"2023-10-21T16:18:06.366848+0000","last_peered":"2023-10-21T16:18:06.366848+0000","last_clean":"2023-10-21T16:18:06.366848+0000","last_became_active":"2023-10-21T16:18:05.386463+0000","last_became_peered":"2023-10-21T16:18:05.386463+0000","last_unstale":"2023-10-21T16:18:06.366848+0000","last_undegraded":"2023-10-21T16:18:06.366848+0000","last_fullsized":"2023-10-21T16:18:06.366848+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:52.197030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367337+0000","last_change":"2023-10-21T16:18:04.385522+0000","last_active":"2023-10-21T16:18:06.367337+0000","last_peered":"2023-10-21T16:18:06.367337+0000","last_clean":"2023-10-21T16:18:06.367337+0000","last_became_active":"2023-10-21T16:18:04.385129+0000","last_became_peered":"2023-10-21T16:18:04.385129+0000","last_unstale":"2023-10-21T16:18:06.367337+0000","last_undegraded":"2023-10-21T16:18:06.367337+0000","last_fullsized":"2023-10-21T16:18:06.367337+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:39.561258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094549+0000","last_change":"2023-10-21T16:18:05.407453+0000","last_active":"2023-10-21T16:18:07.094549+0000","last_peered":"2023-10-21T16:18:07.094549+0000","last_clean":"2023-10-21T16:18:07.094549+0000","last_became_active":"2023-10-21T16:18:05.407174+0000","last_became_peered":"2023-10-21T16:18:05.407174+0000","last_unstale":"2023-10-21T16:18:07.094549+0000","last_undegraded":"2023-10-21T16:18:07.094549+0000","last_fullsized":"2023-10-21T16:18:07.094549+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:15:27.225254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093302+0000","last_change":"2023-10-21T16:18:04.374973+0000","last_active":"2023-10-21T16:18:07.093302+0000","last_peered":"2023-10-21T16:18:07.093302+0000","last_clean":"2023-10-21T16:18:07.093302+0000","last_became_active":"2023-10-21T16:18:04.374649+0000","last_became_peered":"2023-10-21T16:18:04.374649+0000","last_unstale":"2023-10-21T16:18:07.093302+0000","last_undegraded":"2023-10-21T16:18:07.093302+0000","last_fullsized":"2023-10-21T16:18:07.093302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:32:54.676668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.364832+0000","last_change":"2023-10-21T16:18:05.399485+0000","last_active":"2023-10-21T16:18:06.364832+0000","last_peered":"2023-10-21T16:18:06.364832+0000","last_clean":"2023-10-21T16:18:06.364832+0000","last_became_active":"2023-10-21T16:18:05.396459+0000","last_became_peered":"2023-10-21T16:18:05.396459+0000","last_unstale":"2023-10-21T16:18:06.364832+0000","last_undegraded":"2023-10-21T16:18:06.364832+0000","last_fullsized":"2023-10-21T16:18:06.364832+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:56.863624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094256+0000","last_change":"2023-10-21T16:18:04.378376+0000","last_active":"2023-10-21T16:18:07.094256+0000","last_peered":"2023-10-21T16:18:07.094256+0000","last_clean":"2023-10-21T16:18:07.094256+0000","last_became_active":"2023-10-21T16:18:04.378024+0000","last_became_peered":"2023-10-21T16:18:04.378024+0000","last_unstale":"2023-10-21T16:18:07.094256+0000","last_undegraded":"2023-10-21T16:18:07.094256+0000","last_fullsized":"2023-10-21T16:18:07.094256+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:47:57.875260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365702+0000","last_change":"2023-10-21T16:18:05.380394+0000","last_active":"2023-10-21T16:18:06.365702+0000","last_peered":"2023-10-21T16:18:06.365702+0000","last_clean":"2023-10-21T16:18:06.365702+0000","last_became_active":"2023-10-21T16:18:05.379976+0000","last_became_peered":"2023-10-21T16:18:05.379976+0000","last_unstale":"2023-10-21T16:18:06.365702+0000","last_undegraded":"2023-10-21T16:18:06.365702+0000","last_fullsized":"2023-10-21T16:18:06.365702+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:49.652191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093764+0000","last_change":"2023-10-21T16:18:04.392725+0000","last_active":"2023-10-21T16:18:07.093764+0000","last_peered":"2023-10-21T16:18:07.093764+0000","last_clean":"2023-10-21T16:18:07.093764+0000","last_became_active":"2023-10-21T16:18:04.392112+0000","last_became_peered":"2023-10-21T16:18:04.392112+0000","last_unstale":"2023-10-21T16:18:07.093764+0000","last_undegraded":"2023-10-21T16:18:07.093764+0000","last_fullsized":"2023-10-21T16:18:07.093764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:55:04.795282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094641+0000","last_change":"2023-10-21T16:18:05.397703+0000","last_active":"2023-10-21T16:18:07.094641+0000","last_peered":"2023-10-21T16:18:07.094641+0000","last_clean":"2023-10-21T16:18:07.094641+0000","last_became_active":"2023-10-21T16:18:05.390756+0000","last_became_peered":"2023-10-21T16:18:05.390756+0000","last_unstale":"2023-10-21T16:18:07.094641+0000","last_undegraded":"2023-10-21T16:18:07.094641+0000","last_fullsized":"2023-10-21T16:18:07.094641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:31.335608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093614+0000","last_change":"2023-10-21T16:18:04.392499+0000","last_active":"2023-10-21T16:18:07.093614+0000","last_peered":"2023-10-21T16:18:07.093614+0000","last_clean":"2023-10-21T16:18:07.093614+0000","last_became_active":"2023-10-21T16:18:04.392158+0000","last_became_peered":"2023-10-21T16:18:04.392158+0000","last_unstale":"2023-10-21T16:18:07.093614+0000","last_undegraded":"2023-10-21T16:18:07.093614+0000","last_fullsized":"2023-10-21T16:18:07.093614+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:54:16.559817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.367728+0000","last_change":"2023-10-21T16:18:05.403273+0000","last_active":"2023-10-21T16:18:06.367728+0000","last_peered":"2023-10-21T16:18:06.367728+0000","last_clean":"2023-10-21T16:18:06.367728+0000","last_became_active":"2023-10-21T16:18:05.402953+0000","last_became_peered":"2023-10-21T16:18:05.402953+0000","last_unstale":"2023-10-21T16:18:06.367728+0000","last_undegraded":"2023-10-21T16:18:06.367728+0000","last_fullsized":"2023-10-21T16:18:06.367728+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:20.440829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095254+0000","last_change":"2023-10-21T16:18:04.383743+0000","last_active":"2023-10-21T16:18:07.095254+0000","last_peered":"2023-10-21T16:18:07.095254+0000","last_clean":"2023-10-21T16:18:07.095254+0000","last_became_active":"2023-10-21T16:18:04.383406+0000","last_became_peered":"2023-10-21T16:18:04.383406+0000","last_unstale":"2023-10-21T16:18:07.095254+0000","last_undegraded":"2023-10-21T16:18:07.095254+0000","last_fullsized":"2023-10-21T16:18:07.095254+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:49:00.300035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094496+0000","last_change":"2023-10-21T16:18:05.395186+0000","last_active":"2023-10-21T16:18:07.094496+0000","last_peered":"2023-10-21T16:18:07.094496+0000","last_clean":"2023-10-21T16:18:07.094496+0000","last_became_active":"2023-10-21T16:18:05.394918+0000","last_became_peered":"2023-10-21T16:18:05.394918+0000","last_unstale":"2023-10-21T16:18:07.094496+0000","last_undegraded":"2023-10-21T16:18:07.094496+0000","last_fullsized":"2023-10-21T16:18:07.094496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:36.590042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365955+0000","last_change":"2023-10-21T16:18:04.385072+0000","last_active":"2023-10-21T16:18:06.365955+0000","last_peered":"2023-10-21T16:18:06.365955+0000","last_clean":"2023-10-21T16:18:06.365955+0000","last_became_active":"2023-10-21T16:18:04.384675+0000","last_became_peered":"2023-10-21T16:18:04.384675+0000","last_unstale":"2023-10-21T16:18:06.365955+0000","last_undegraded":"2023-10-21T16:18:06.365955+0000","last_fullsized":"2023-10-21T16:18:06.365955+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:27:18.708201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.368319+0000","last_change":"2023-10-21T16:18:05.393503+0000","last_active":"2023-10-21T16:18:06.368319+0000","last_peered":"2023-10-21T16:18:06.368319+0000","last_clean":"2023-10-21T16:18:06.368319+0000","last_became_active":"2023-10-21T16:18:05.393187+0000","last_became_peered":"2023-10-21T16:18:05.393187+0000","last_unstale":"2023-10-21T16:18:06.368319+0000","last_undegraded":"2023-10-21T16:18:06.368319+0000","last_fullsized":"2023-10-21T16:18:06.368319+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:45:31.910411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376135+0000","last_change":"2023-10-21T16:18:04.382536+0000","last_active":"2023-10-21T16:18:06.376135+0000","last_peered":"2023-10-21T16:18:06.376135+0000","last_clean":"2023-10-21T16:18:06.376135+0000","last_became_active":"2023-10-21T16:18:04.382231+0000","last_became_peered":"2023-10-21T16:18:04.382231+0000","last_unstale":"2023-10-21T16:18:06.376135+0000","last_undegraded":"2023-10-21T16:18:06.376135+0000","last_fullsized":"2023-10-21T16:18:06.376135+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:24:54.230940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.365375+0000","last_change":"2023-10-21T16:18:05.381505+0000","last_active":"2023-10-21T16:18:06.365375+0000","last_peered":"2023-10-21T16:18:06.365375+0000","last_clean":"2023-10-21T16:18:06.365375+0000","last_became_active":"2023-10-21T16:18:05.381024+0000","last_became_peered":"2023-10-21T16:18:05.381024+0000","last_unstale":"2023-10-21T16:18:06.365375+0000","last_undegraded":"2023-10-21T16:18:06.365375+0000","last_fullsized":"2023-10-21T16:18:06.365375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:06:35.297384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.375888+0000","last_change":"2023-10-21T16:18:04.392811+0000","last_active":"2023-10-21T16:18:06.375888+0000","last_peered":"2023-10-21T16:18:06.375888+0000","last_clean":"2023-10-21T16:18:06.375888+0000","last_became_active":"2023-10-21T16:18:04.392525+0000","last_became_peered":"2023-10-21T16:18:04.392525+0000","last_unstale":"2023-10-21T16:18:06.375888+0000","last_undegraded":"2023-10-21T16:18:06.375888+0000","last_fullsized":"2023-10-21T16:18:06.375888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:01:43.358320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.093296+0000","last_change":"2023-10-21T16:18:05.407152+0000","last_active":"2023-10-21T16:18:07.093296+0000","last_peered":"2023-10-21T16:18:07.093296+0000","last_clean":"2023-10-21T16:18:07.093296+0000","last_became_active":"2023-10-21T16:18:05.406858+0000","last_became_peered":"2023-10-21T16:18:05.406858+0000","last_unstale":"2023-10-21T16:18:07.093296+0000","last_undegraded":"2023-10-21T16:18:07.093296+0000","last_fullsized":"2023-10-21T16:18:07.093296+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:33.733379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094839+0000","last_change":"2023-10-21T16:18:04.386344+0000","last_active":"2023-10-21T16:18:07.094839+0000","last_peered":"2023-10-21T16:18:07.094839+0000","last_clean":"2023-10-21T16:18:07.094839+0000","last_became_active":"2023-10-21T16:18:04.386028+0000","last_became_peered":"2023-10-21T16:18:04.386028+0000","last_unstale":"2023-10-21T16:18:07.094839+0000","last_undegraded":"2023-10-21T16:18:07.094839+0000","last_fullsized":"2023-10-21T16:18:07.094839+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:47:53.293319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094207+0000","last_change":"2023-10-21T16:18:05.400979+0000","last_active":"2023-10-21T16:18:07.094207+0000","last_peered":"2023-10-21T16:18:07.094207+0000","last_clean":"2023-10-21T16:18:07.094207+0000","last_became_active":"2023-10-21T16:18:05.400673+0000","last_became_peered":"2023-10-21T16:18:05.400673+0000","last_unstale":"2023-10-21T16:18:07.094207+0000","last_undegraded":"2023-10-21T16:18:07.094207+0000","last_fullsized":"2023-10-21T16:18:07.094207+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:48:14.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.366885+0000","last_change":"2023-10-21T16:18:04.383192+0000","last_active":"2023-10-21T16:18:06.366885+0000","last_peered":"2023-10-21T16:18:06.366885+0000","last_clean":"2023-10-21T16:18:06.366885+0000","last_became_active":"2023-10-21T16:18:04.382166+0000","last_became_peered":"2023-10-21T16:18:04.382166+0000","last_unstale":"2023-10-21T16:18:06.366885+0000","last_undegraded":"2023-10-21T16:18:06.366885+0000","last_fullsized":"2023-10-21T16:18:06.366885+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:26.364173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094340+0000","last_change":"2023-10-21T16:18:05.398878+0000","last_active":"2023-10-21T16:18:07.094340+0000","last_peered":"2023-10-21T16:18:07.094340+0000","last_clean":"2023-10-21T16:18:07.094340+0000","last_became_active":"2023-10-21T16:18:05.398481+0000","last_became_peered":"2023-10-21T16:18:05.398481+0000","last_unstale":"2023-10-21T16:18:07.094340+0000","last_undegraded":"2023-10-21T16:18:07.094340+0000","last_fullsized":"2023-10-21T16:18:07.094340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:42.601976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094707+0000","last_change":"2023-10-21T16:18:04.392386+0000","last_active":"2023-10-21T16:18:07.094707+0000","last_peered":"2023-10-21T16:18:07.094707+0000","last_clean":"2023-10-21T16:18:07.094707+0000","last_became_active":"2023-10-21T16:18:04.392039+0000","last_became_peered":"2023-10-21T16:18:04.392039+0000","last_unstale":"2023-10-21T16:18:07.094707+0000","last_undegraded":"2023-10-21T16:18:07.094707+0000","last_fullsized":"2023-10-21T16:18:07.094707+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:33:41.045578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.094747+0000","last_change":"2023-10-21T16:18:05.404187+0000","last_active":"2023-10-21T16:18:07.094747+0000","last_peered":"2023-10-21T16:18:07.094747+0000","last_clean":"2023-10-21T16:18:07.094747+0000","last_became_active":"2023-10-21T16:18:05.402262+0000","last_became_peered":"2023-10-21T16:18:05.402262+0000","last_unstale":"2023-10-21T16:18:07.094747+0000","last_undegraded":"2023-10-21T16:18:07.094747+0000","last_fullsized":"2023-10-21T16:18:07.094747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:17:27.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.092298+0000","last_change":"2023-10-21T16:18:04.375387+0000","last_active":"2023-10-21T16:18:07.092298+0000","last_peered":"2023-10-21T16:18:07.092298+0000","last_clean":"2023-10-21T16:18:07.092298+0000","last_became_active":"2023-10-21T16:18:04.375101+0000","last_became_peered":"2023-10-21T16:18:04.375101+0000","last_unstale":"2023-10-21T16:18:07.092298+0000","last_undegraded":"2023-10-21T16:18:07.092298+0000","last_fullsized":"2023-10-21T16:18:07.092298+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:30.318443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:06.376198+0000","last_change":"2023-10-21T16:18:05.405075+0000","last_active":"2023-10-21T16:18:06.376198+0000","last_peered":"2023-10-21T16:18:06.376198+0000","last_clean":"2023-10-21T16:18:06.376198+0000","last_became_active":"2023-10-21T16:18:05.404698+0000","last_became_peered":"2023-10-21T16:18:05.404698+0000","last_unstale":"2023-10-21T16:18:06.376198+0000","last_undegraded":"2023-10-21T16:18:06.376198+0000","last_fullsized":"2023-10-21T16:18:06.376198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:04.335380+0000","last_clean_scrub_stamp":"2023-10-21T16:18:04.335380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:27:42.758492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.095590+0000","last_change":"2023-10-21T16:18:04.392526+0000","last_active":"2023-10-21T16:18:07.095590+0000","last_peered":"2023-10-21T16:18:07.095590+0000","last_clean":"2023-10-21T16:18:07.095590+0000","last_became_active":"2023-10-21T16:18:04.392106+0000","last_became_peered":"2023-10-21T16:18:04.392106+0000","last_unstale":"2023-10-21T16:18:07.095590+0000","last_undegraded":"2023-10-21T16:18:07.095590+0000","last_fullsized":"2023-10-21T16:18:07.095590+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:03.318215+0000","last_clean_scrub_stamp":"2023-10-21T16:18:03.318215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:45.012971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"103'2156","reported_seq":2424,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-21T16:18:07.677720+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:18:07.677720+0000","last_peered":"2023-10-21T16:18:07.677720+0000","last_clean":"2023-10-21T16:18:07.677720+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:18:07.677720+0000","last_undegraded":"2023-10-21T16:18:07.677720+0000","last_fullsized":"2023-10-21T16:18:07.677720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":2156,"log_dups_size":0,"ondisk_log_size":2156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":546,"num_read_kb":1029,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2156,"ondisk_log_size":2156,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575043,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":1227242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.145},"min":{"1min":0.65000000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5609999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1950000000000001,"15min":1.171},"min":{"1min":0.76200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4019999999999999,"5min":2.448,"15min":2.448},"last":0.89000000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78300000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5389999999999999,"5min":1.915,"15min":1.915},"last":1.012},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.224},"min":{"1min":0.51600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.583,"5min":1.778,"15min":1.778},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.27,"15min":1.2929999999999999},"min":{"1min":0.753,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5880000000000001,"5min":1.748,"15min":2.085},"last":1.256},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2849999999999999,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.556,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.6519999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.2529999999999999},"min":{"1min":0.626,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.6040000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.85999999999999999,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.9570000000000001,"15min":2.0099999999999998},"last":1.4379999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3360000000000001,"15min":1.355},"min":{"1min":0.81599999999999995,"5min":0.76700000000000002,"15min":0.54400000000000004},"max":{"1min":1.7350000000000001,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.974},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.423,"15min":1.4450000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.70399999999999996},"max":{"1min":1.8360000000000001,"5min":2.355,"15min":2.355},"last":1.915}]},{"osd":5,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.391,"15min":1.3999999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.716,"5min":1.865,"15min":2.258},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.4350000000000001,"15min":1.4379999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.60899999999999999},"max":{"1min":1.9079999999999999,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":2.1200000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.53,"15min":1.548},"min":{"1min":1.0580000000000001,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":1.706,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":2.1859999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4790000000000001,"15min":1.466},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7709999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":2.0449999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1261568,"data_stored":631800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.68799999999999994,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2210000000000001,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.514,"5min":1.895,"15min":2.4350000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.72599999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.417,"5min":2.2309999999999999,"15min":3.7530000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.3480000000000001,"15min":1.306},"min":{"1min":0.90700000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5960000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.25,"15min":1.264},"min":{"1min":0.85599999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.895,"5min":2.5339999999999998,"15min":2.7250000000000001},"last":1.121},{"interface":"front","average":{"1min":1.24,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.6839999999999999,"5min":1.7789999999999999,"15min":1.8169999999999999},"last":1.3120000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.377,"15min":1.3740000000000001},"min":{"1min":1.1200000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.7490000000000001,"5min":2.2000000000000002,"15min":3.46},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.446,"15min":1.4299999999999999},"min":{"1min":1.204,"5min":0.83699999999999997,"15min":0.80000000000000004},"max":{"1min":1.9970000000000001,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":1.4359999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.393,"15min":1.365},"min":{"1min":0.94599999999999995,"5min":0.82999999999999996,"15min":0.76300000000000001},"max":{"1min":1.9379999999999999,"5min":2.1930000000000001,"15min":3.6150000000000002},"last":1.167},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.401,"15min":1.397},"min":{"1min":1.2110000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1299999999999999,"5min":2.2269999999999999,"15min":2.5470000000000002},"last":1.536}]},{"osd":5,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.5029999999999999,"15min":1.5129999999999999},"min":{"1min":1.1120000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.246,"5min":2.246,"15min":2.6909999999999998},"last":1.627},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.99199999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8240000000000001,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:14 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.546,"15min":1.5149999999999999},"min":{"1min":1.0429999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.2050000000000001,"5min":2.3570000000000002,"15min":3.71},"last":1.363},{"interface":"front","average":{"1min":1.7090000000000001,"5min":1.631,"15min":1.623},"min":{"1min":1.4710000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.7650000000000001},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1265664,"data_stored":636886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0900000000000001,"15min":1.1160000000000001},"min":{"1min":0.81100000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4399999999999999,"5min":1.554,"15min":2.0070000000000001},"last":1.089},{"interface":"front","average":{"1min":1.133,"5min":1.1850000000000001,"15min":1.1839999999999999},"min":{"1min":0.72699999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.51,"5min":1.712,"15min":1.9530000000000001},"last":1.446}]},{"osd":1,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.64200000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6339999999999999,"5min":1.877,"15min":1.944},"last":1.351},{"interface":"front","average":{"1min":1.232,"5min":1.2210000000000001,"15min":1.2270000000000001},"min":{"1min":0.91200000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.907},"last":1.24}]},{"osd":2,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.2410000000000001,"15min":1.2929999999999999},"min":{"1min":0.83599999999999997,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.8,"5min":1.8,"15min":1.95},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.206,"15min":1.242},"min":{"1min":0.85099999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":2.0249999999999999},"last":1.288}]},{"osd":3,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.3129999999999999},"min":{"1min":1.0880000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.573,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.3620000000000001,"15min":1.381},"min":{"1min":1.0009999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.79,"5min":1.79,"15min":1.8959999999999999},"last":1.522}]},{"osd":4,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2050000000000001,"15min":1.216},"min":{"1min":0.93999999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.919,"5min":1.919,"15min":2.0960000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.349,"15min":1.339},"min":{"1min":0.96499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.76,"5min":2.0529999999999999,"15min":2.121},"last":1.5900000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.4550000000000001,"15min":1.45},"min":{"1min":1.3819999999999999,"5min":0.66900000000000004,"15min":0.61099999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1360000000000001},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.4790000000000001,"15min":1.4870000000000001},"min":{"1min":1.0860000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.855,"5min":1.9450000000000001,"15min":2.1859999999999999},"last":2.097}]},{"osd":7,"last update":"Sat Oct 21 16:17:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.516,"15min":1.504},"min":{"1min":1.1699999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8300000000000001,"5min":2.218,"15min":2.218},"last":1.679},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.534,"15min":1.556},"min":{"1min":1.371,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.286,"15min":2.286},"last":1.8240000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":638568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1830000000000001,"15min":1.1379999999999999},"min":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.55400000000000005},"max":{"1min":1.5509999999999999,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.048},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.194,"15min":1.21},"min":{"1min":0.77800000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7230000000000001,"5min":1.758,"15min":3.6080000000000001},"last":1.385}]},{"osd":1,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.252,"15min":1.2629999999999999},"min":{"1min":0.78200000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.663,"5min":1.885,"15min":2.0699999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2190000000000001,"15min":1.228},"min":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5609999999999999,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.96799999999999997}]},{"osd":2,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.26,"15min":1.2470000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.637,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.325},{"interface":"front","average":{"1min":1.304,"5min":1.298,"15min":1.331},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.698,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.89000000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3140000000000001},"min":{"1min":0.64800000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.946,"5min":2.5099999999999998,"15min":3.8109999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.409,"15min":1.3779999999999999},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":1.5649999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2889999999999999,"15min":1.3080000000000001},"min":{"1min":0.874,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7889999999999999,"5min":3.9199999999999999,"15min":3.9199999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.335,"15min":1.3340000000000001},"min":{"1min":0.89500000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0670000000000002,"5min":3.0699999999999998,"15min":3.9300000000000002},"last":1.6240000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4750000000000001,"15min":1.415},"min":{"1min":0.85199999999999998,"5min":0.79700000000000004,"15min":0.77700000000000002},"max":{"1min":1.8080000000000001,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.583,"15min":1.5800000000000001},"min":{"1min":1.2470000000000001,"5min":0.878,"15min":0.86899999999999999},"max":{"1min":1.8919999999999999,"5min":2.4500000000000002,"15min":3.9609999999999999},"last":1.48}]},{"osd":7,"last update":"Sat Oct 21 16:17:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5009999999999999,"15min":1.492},"min":{"1min":0.91800000000000004,"5min":0.77200000000000002,"15min":0.69599999999999995},"max":{"1min":2.2170000000000001,"5min":3.1859999999999999,"15min":4.0190000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.6120000000000001,"15min":1.5609999999999999},"min":{"1min":1.3320000000000001,"5min":0.90200000000000002,"15min":0.84699999999999998},"max":{"1min":1.9490000000000001,"5min":3.968,"15min":3.968},"last":1.7669999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1253376,"data_stored":630770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98199999999999998,"15min":0.94999999999999996},"min":{"1min":0.61199999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.526,"5min":1.746,"15min":1.746},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.1779999999999999,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.365,"5min":2.04,"15min":2.5289999999999999},"last":0.89000000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.99399999999999999,"15min":1.0029999999999999},"min":{"1min":0.71599999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.53,"5min":2.173,"15min":2.2519999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":1.0069999999999999,"5min":0.60999999999999999,"15min":0.498},"max":{"1min":1.6180000000000001,"5min":1.8100000000000001,"15min":3.827},"last":0.97299999999999998}]},{"osd":2,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0620000000000001,"15min":1.0589999999999999},"min":{"1min":0.82699999999999996,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.3360000000000001,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.206,"15min":1.224},"min":{"1min":0.73699999999999999,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.663,"5min":2.847,"15min":2.847},"last":1.0049999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.379,"15min":1.403},"min":{"1min":1.101,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.6970000000000001,"5min":2.1040000000000001,"15min":2.2349999999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.391,"15min":1.4059999999999999},"min":{"1min":1.1180000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.8160000000000001,"5min":2.919,"15min":2.919},"last":1.1140000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.415,"15min":1.427},"min":{"1min":1.167,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.619,"5min":2.02,"15min":3.7250000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4930000000000001,"15min":1.4990000000000001},"min":{"1min":1.129,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.841,"5min":2.2629999999999999,"15min":2.423},"last":1.0349999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.474,"15min":1.5109999999999999},"min":{"1min":1.1399999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.2909999999999999,"5min":0.871,"15min":0.70299999999999996},"max":{"1min":1.7689999999999999,"5min":2.2290000000000001,"15min":3.573},"last":1.1619999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5740000000000001,"15min":1.591},"min":{"1min":1.2669999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.915,"5min":2.302,"15min":2.3759999999999999},"last":1.375},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.5780000000000001,"15min":1.6100000000000001},"min":{"1min":1.304,"5min":0.93000000000000005,"15min":0.68899999999999995},"max":{"1min":2.1280000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.2789999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62952,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308888,"statfs":{"total":96636764160,"available":96572301312,"internally_reserved":0,"allocated":1273856,"data_stored":637594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.85999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.268,"5min":1.859,"15min":1.859},"last":0.374},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.97599999999999998,"15min":0.98199999999999998},"min":{"1min":0.42699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.331,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.093,"15min":1.129},"min":{"1min":0.63400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.528,"5min":1.907,"15min":1.9390000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.206,"15min":1.2549999999999999},"min":{"1min":0.64900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.657,"5min":1.881,"15min":1.881},"last":0.70199999999999996}]},{"osd":3,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.0580000000000001,"15min":1.1240000000000001},"min":{"1min":0.60999999999999999,"5min":0.438,"15min":0.379},"max":{"1min":1.7230000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3240000000000001,"15min":1.3640000000000001},"min":{"1min":0.69799999999999995,"5min":0.56699999999999995,"15min":0.56599999999999995},"max":{"1min":1.6919999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3600000000000001,"15min":1.369},"min":{"1min":0.94499999999999995,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.958,"5min":1.958,"15min":2.206},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.415,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.825,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.034}]},{"osd":5,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.343},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":2.024,"5min":2.2730000000000001,"15min":2.7679999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5069999999999999,"15min":1.5209999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1099999999999999,"5min":2.1349999999999998,"15min":2.4870000000000001},"last":1.121}]},{"osd":6,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4470000000000001,"15min":1.462},"min":{"1min":0.93000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7589999999999999,"5min":2.6819999999999999,"15min":2.698},"last":1.131},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.4359999999999999},"min":{"1min":0.84999999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8200000000000001,"5min":2.552,"15min":2.6179999999999999},"last":0.83399999999999996}]},{"osd":7,"last update":"Sat Oct 21 16:17:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.494,"15min":1.5309999999999999},"min":{"1min":0.88,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.863,"5min":2.4969999999999999,"15min":2.8769999999999998},"last":1.395},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4590000000000001,"15min":1.5129999999999999},"min":{"1min":0.81599999999999995,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.865,"5min":2.3999999999999999,"15min":3.6150000000000002},"last":1.325}]}]},{"osd":1,"up_from":13,"seq":55834575043,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1269760,"data_stored":636964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88500000000000001,"15min":0.91000000000000003},"min":{"1min":0.45900000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.323,"5min":1.905,"15min":1.905},"last":1.032},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.071,"15min":1.0740000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.6930000000000001,"5min":1.8759999999999999,"15min":2.5089999999999999},"last":1.3169999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":0.997},"min":{"1min":0.53500000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.4419999999999999,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":1.121},{"interface":"front","average":{"1min":1.133,"5min":1.157,"15min":1.1850000000000001},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.974,"15min":2.29},"last":1.2450000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.244,"15min":1.3},"min":{"1min":0.54800000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7350000000000001,"5min":1.8420000000000001,"15min":1.9510000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.52200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5489999999999999,"5min":1.877,"15min":2.3620000000000001},"last":0.96099999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2589999999999999,"15min":1.276},"min":{"1min":0.88500000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.71,"5min":1.944,"15min":1.944},"last":1.377},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4159999999999999,"15min":1.4219999999999999},"min":{"1min":0.68700000000000006,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.833,"5min":2.2719999999999998,"15min":2.4409999999999998},"last":1.4319999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.4610000000000001,"15min":1.5049999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.75900000000000001},"max":{"1min":1.9099999999999999,"5min":2.2410000000000001,"15min":2.6030000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.458,"15min":1.4790000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.855,"5min":2.0880000000000001,"15min":2.2570000000000001},"last":1.653}]},{"osd":6,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4790000000000001,"15min":1.5109999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.984,"5min":2.2469999999999999,"15min":4.1260000000000003},"last":1.48},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4219999999999999,"15min":1.4730000000000001},"min":{"1min":0.63800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.9510000000000001,"5min":2.2090000000000001,"15min":3.101},"last":1.825}]},{"osd":7,"last update":"Sat Oct 21 16:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.5820000000000001,"15min":1.609},"min":{"1min":0.872,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.2530000000000001,"5min":2.2679999999999998,"15min":2.8180000000000001},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5569999999999999,"15min":1.613},"min":{"1min":0.74399999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.901,"5min":2.1619999999999999,"15min":3.952},"last":1.5660000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575043,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28836,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343004,"statfs":{"total":96636764160,"available":96607236096,"internally_reserved":0,"allocated":1855488,"data_stored":1227220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.89400000000000002,"15min":0.92200000000000004},"min":{"1min":0.67600000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.583,"5min":1.782,"15min":2.9260000000000002},"last":0.87},{"interface":"front","average":{"1min":1.028,"5min":1.0049999999999999,"15min":1.0569999999999999},"min":{"1min":0.66000000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.5940000000000001,"15min":2.4860000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.075,"15min":1.123},"min":{"1min":0.80600000000000005,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":1.6559999999999999,"5min":1.7410000000000001,"15min":2.0329999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.125,"15min":1.147},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.42999999999999999},"max":{"1min":1.603,"5min":1.7210000000000001,"15min":3.0939999999999999},"last":1.198}]},{"osd":3,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0820000000000001,"15min":1.093},"min":{"1min":0.67200000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.702,"5min":1.8660000000000001,"15min":2.1429999999999998},"last":1.234},{"interface":"front","average":{"1min":1.272,"5min":1.2410000000000001,"15min":1.29},"min":{"1min":0.91400000000000003,"5min":0.48599999999999999,"15min":0.44900000000000001},"max":{"1min":1.8100000000000001,"5min":1.8109999999999999,"15min":3.052},"last":1.278}]},{"osd":4,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3939999999999999,"15min":1.417},"min":{"1min":0.98699999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.756,"5min":2.0339999999999998,"15min":3.3580000000000001},"last":1.405},{"interface":"front","average":{"1min":1.546,"5min":1.4950000000000001,"15min":1.5309999999999999},"min":{"1min":1.3069999999999999,"5min":0.872,"15min":0.872},"max":{"1min":1.863,"5min":2.246,"15min":3.306},"last":1.778}]},{"osd":5,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3759999999999999,"15min":1.3859999999999999},"min":{"1min":1.0129999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6279999999999999,"5min":1.9419999999999999,"15min":3.8849999999999998},"last":1.329},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.454,"15min":1.504},"min":{"1min":1.046,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8089999999999999,"5min":2.137,"15min":3.1789999999999998},"last":1.635}]},{"osd":6,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4419999999999999,"15min":1.4850000000000001},"min":{"1min":0.95799999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.8069999999999999,"5min":2.5379999999999998,"15min":4.3019999999999996},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5489999999999999,"15min":1.595},"min":{"1min":1.2030000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8069999999999999,"5min":2.355,"15min":3.5139999999999998},"last":1.5329999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:17:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4330000000000001,"15min":1.4610000000000001},"min":{"1min":1.0389999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.637,"5min":2.2959999999999998,"15min":3.8140000000000001},"last":1.71},{"interface":"front","average":{"1min":1.476,"5min":1.532,"15min":1.5800000000000001},"min":{"1min":1.0960000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.802,"5min":2.077,"15min":4.6219999999999999},"last":1.4630000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:18:10.445 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:18:10.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:18:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.726+0000 7f474907e640 1 Processor -- start 2023-10-21T16:18:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.726+0000 7f474907e640 1 -- start start 2023-10-21T16:18:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f474907e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f474415de70 0x7f47440ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f474907e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47440c9c90 con 0x7f474415eba0 2023-10-21T16:18:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f474907e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474415e250 con 0x7f47440cb260 2023-10-21T16:18:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f474907e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474415f480 con 0x7f474415de70 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f4743fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f474415de70 0x7f47440ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f4743fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f474415de70 0x7f47440ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53360/0 (socket says 172.21.15.53:53360) 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.728+0000 7f4743fff640 1 -- 172.21.15.53:0/4037947791 learned_addr learned my addr 172.21.15.53:0/4037947791 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.729+0000 7f4743fff640 1 -- 172.21.15.53:0/4037947791 >> v1:172.21.15.124:6789/0 conn(0x7f47440cb260 legacy=0x7f474415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.729+0000 7f4743fff640 1 -- 172.21.15.53:0/4037947791 >> v1:172.21.15.53:6789/0 conn(0x7f474415eba0 legacy=0x7f47440b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.729+0000 7f4743fff640 1 -- 172.21.15.53:0/4037947791 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474415f1a0 con 0x7f474415de70 2023-10-21T16:18:10.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.730+0000 7f4743fff640 1 --2- 172.21.15.53:0/4037947791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415de70 0x7f47440ca9a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4734009870 tx=0x7f473402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f9209f1dd16d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.730+0000 7f4742ffd640 1 -- 172.21.15.53:0/4037947791 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734030860 con 0x7f474415de70 2023-10-21T16:18:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.730+0000 7f4742ffd640 1 -- 172.21.15.53:0/4037947791 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47340309c0 con 0x7f474415de70 2023-10-21T16:18:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.730+0000 7f4742ffd640 1 -- 172.21.15.53:0/4037947791 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734030d50 con 0x7f474415de70 2023-10-21T16:18:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 -- 172.21.15.53:0/4037947791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415de70 msgr2=0x7f47440ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 --2- 172.21.15.53:0/4037947791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415de70 0x7f47440ca9a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4734009870 tx=0x7f473402b9a0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 -- 172.21.15.53:0/4037947791 shutdown_connections 2023-10-21T16:18:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 --2- 172.21.15.53:0/4037947791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415de70 0x7f47440ca9a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 -- 172.21.15.53:0/4037947791 >> 172.21.15.53:0/4037947791 conn(0x7f47440ce9f0 msgr2=0x7f47440cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 -- 172.21.15.53:0/4037947791 shutdown_connections 2023-10-21T16:18:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.731+0000 7f474907e640 1 -- 172.21.15.53:0/4037947791 wait complete. 2023-10-21T16:18:10.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.733+0000 7f474907e640 1 Processor -- start 2023-10-21T16:18:10.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.733+0000 7f474907e640 1 -- start start 2023-10-21T16:18:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47440cb260 0x7f474415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f474415de70 0x7f4744099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47440c3b30 con 0x7f474415eba0 2023-10-21T16:18:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474415ada0 con 0x7f47440cb260 2023-10-21T16:18:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474907e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474409bd90 con 0x7f474415de70 2023-10-21T16:18:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474887d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474887d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53372/0 (socket says 172.21.15.53:53372) 2023-10-21T16:18:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.735+0000 7f474887d640 1 -- 172.21.15.53:0/893477317 learned_addr learned my addr 172.21.15.53:0/893477317 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f4743fff640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47440cb260 0x7f474415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f47437fe640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f474415de70 0x7f4744099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 -- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f474415de70 msgr2=0x7f4744099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f474415de70 0x7f4744099290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 -- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47440cb260 msgr2=0x7f474415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47440cb260 0x7f474415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 -- 172.21.15.53:0/893477317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340094d0 con 0x7f474415eba0 2023-10-21T16:18:10.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f474887d640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c7a0 tx=0x7f473c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.736+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c003b70 con 0x7f474415eba0 2023-10-21T16:18:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.737+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c003cf0 con 0x7f474415eba0 2023-10-21T16:18:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.737+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474409b460 con 0x7f474415eba0 2023-10-21T16:18:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.737+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c00aa90 con 0x7f474415eba0 2023-10-21T16:18:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.737+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474409ba20 con 0x7f474415eba0 2023-10-21T16:18:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.738+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474404e160 con 0x7f474415eba0 2023-10-21T16:18:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.739+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f473c00abf0 con 0x7f474415eba0 2023-10-21T16:18:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.740+0000 7f4740ff9640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 0x7f473006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.740+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f473c0896e0 con 0x7f474415eba0 2023-10-21T16:18:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.741+0000 7f4743fff640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 0x7f473006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.742+0000 7f4743fff640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 0x7f473006d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4734036000 tx=0x7f4734008030 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:10.744+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f473c0545b0 con 0x7f474415eba0 2023-10-21T16:18:10.945 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:10.942+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:11.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.012+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f474406cab0 con 0x7f474415eba0 2023-10-21T16:18:11.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.016+0000 7f4740ff9640 1 -- 172.21.15.53:0/893477317 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f473c003e60 con 0x7f474415eba0 2023-10-21T16:18:11.018 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:18:11.018 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:18:11.019 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:18:11.019 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:18:11.019 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:18:11.019 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:18:11.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 msgr2=0x7f473006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 0x7f473006d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4734036000 tx=0x7f4734008030 comp rx=0 tx=0).stop 2023-10-21T16:18:11.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 msgr2=0x7f474415c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c7a0 tx=0x7f473c00cc60 comp rx=0 tx=0).stop 2023-10-21T16:18:11.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 shutdown_connections 2023-10-21T16:18:11.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f473006af50 0x7f473006d410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f474415eba0 0x7f474415c170 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f474415de70 0x7f4744099290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 --2- 172.21.15.53:0/893477317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47440cb260 0x7f474415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 >> 172.21.15.53:0/893477317 conn(0x7f47440ce9f0 msgr2=0x7f47440ab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:11.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 shutdown_connections 2023-10-21T16:18:11.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.019+0000 7f474907e640 1 -- 172.21.15.53:0/893477317 wait complete. 2023-10-21T16:18:11.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:11.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.269+0000 7faf9c708640 1 Processor -- start 2023-10-21T16:18:11.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.269+0000 7faf9c708640 1 -- start start 2023-10-21T16:18:11.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.270+0000 7faf9c708640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9c708640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf940709e0 0x7faf940c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9c708640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9406dee0 con 0x7faf940ad790 2023-10-21T16:18:11.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9c708640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9415d850 con 0x7faf940709e0 2023-10-21T16:18:11.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9c708640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9415db30 con 0x7faf94159ac0 2023-10-21T16:18:11.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf940709e0 0x7faf940c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:11.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf940709e0 0x7faf940c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53388/0 (socket says 172.21.15.53:53388) 2023-10-21T16:18:11.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 -- 172.21.15.53:0/4249456454 learned_addr learned my addr 172.21.15.53:0/4249456454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:11.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 -- 172.21.15.53:0/4249456454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 msgr2=0x7faf9406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:11.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 --2- 172.21.15.53:0/4249456454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.271+0000 7faf9aca5640 1 -- 172.21.15.53:0/4249456454 >> v1:172.21.15.124:6789/0 conn(0x7faf940ad790 legacy=0x7faf940702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:11.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.272+0000 7faf9aca5640 1 -- 172.21.15.53:0/4249456454 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9415de10 con 0x7faf940709e0 2023-10-21T16:18:11.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.273+0000 7faf9aca5640 1 --2- 172.21.15.53:0/4249456454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940709e0 0x7faf940c8dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009870 tx=0x7faf8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6983ad3ae1b6f888 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:11.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.273+0000 7faf994a2640 1 -- 172.21.15.53:0/4249456454 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031640 con 0x7faf940709e0 2023-10-21T16:18:11.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.273+0000 7faf994a2640 1 -- 172.21.15.53:0/4249456454 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0317a0 con 0x7faf940709e0 2023-10-21T16:18:11.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.273+0000 7faf994a2640 1 -- 172.21.15.53:0/4249456454 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c031b30 con 0x7faf940709e0 2023-10-21T16:18:11.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 -- 172.21.15.53:0/4249456454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940709e0 msgr2=0x7faf940c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 --2- 172.21.15.53:0/4249456454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940709e0 0x7faf940c8dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009870 tx=0x7faf8c02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 -- 172.21.15.53:0/4249456454 shutdown_connections 2023-10-21T16:18:11.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 --2- 172.21.15.53:0/4249456454 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940709e0 0x7faf940c8dd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 --2- 172.21.15.53:0/4249456454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 -- 172.21.15.53:0/4249456454 >> 172.21.15.53:0/4249456454 conn(0x7faf9406a380 msgr2=0x7faf940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:11.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 -- 172.21.15.53:0/4249456454 shutdown_connections 2023-10-21T16:18:11.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.274+0000 7faf9c708640 1 -- 172.21.15.53:0/4249456454 wait complete. 2023-10-21T16:18:11.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.276+0000 7faf9c708640 1 Processor -- start 2023-10-21T16:18:11.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.276+0000 7faf9c708640 1 -- start start 2023-10-21T16:18:11.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940709e0 0x7faf9414a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9414f0b0 con 0x7faf940ad790 2023-10-21T16:18:11.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940b3950 con 0x7faf940709e0 2023-10-21T16:18:11.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf9c708640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9414ac80 con 0x7faf94159ac0 2023-10-21T16:18:11.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf99ca3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:11.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf99ca3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53400/0 (socket says 172.21.15.53:53400) 2023-10-21T16:18:11.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.277+0000 7faf99ca3640 1 -- 172.21.15.53:0/3804314722 learned_addr learned my addr 172.21.15.53:0/3804314722 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:11.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf9aca5640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf9a4a4640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940709e0 0x7faf9414a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 -- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 msgr2=0x7faf9415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 -- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940709e0 msgr2=0x7faf9414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940709e0 0x7faf9414a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 -- 172.21.15.53:0/3804314722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0094d0 con 0x7faf940ad790 2023-10-21T16:18:11.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf9aca5640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:11.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf99ca3640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c400 tx=0x7faf8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:11.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88011790 con 0x7faf940ad790 2023-10-21T16:18:11.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880118f0 con 0x7faf940ad790 2023-10-21T16:18:11.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9415b500 con 0x7faf940ad790 2023-10-21T16:18:11.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.278+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88011ce0 con 0x7faf940ad790 2023-10-21T16:18:11.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.279+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9415aeb0 con 0x7faf940ad790 2023-10-21T16:18:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.280+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9415b690 con 0x7faf940ad790 2023-10-21T16:18:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.281+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7faf8801c460 con 0x7faf940ad790 2023-10-21T16:18:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.282+0000 7faf92ffd640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 0x7faf8006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:11.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.282+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faf88089840 con 0x7faf940ad790 2023-10-21T16:18:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.282+0000 7faf9a4a4640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 0x7faf8006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.284+0000 7faf9a4a4640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 0x7faf8006d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf84007dc0 tx=0x7faf840072f0 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:11.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.286+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf88054600 con 0x7faf940ad790 2023-10-21T16:18:11.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.528+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf94067490 con 0x7faf940ad790 2023-10-21T16:18:11.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.529+0000 7faf92ffd640 1 -- 172.21.15.53:0/3804314722 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7faf88057c20 con 0x7faf940ad790 2023-10-21T16:18:11.531 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:11.532 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:11.532 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 msgr2=0x7faf8006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 0x7faf8006d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf84007dc0 tx=0x7faf840072f0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 msgr2=0x7faf940b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:11.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c400 tx=0x7faf8800c8c0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 shutdown_connections 2023-10-21T16:18:11.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7faf8006af00 0x7faf8006d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf94159ac0 0x7faf9415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf940ad790 0x7faf940b3120 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 --2- 172.21.15.53:0/3804314722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf940709e0 0x7faf9414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:11.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.532+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 >> 172.21.15.53:0/3804314722 conn(0x7faf9406a380 msgr2=0x7faf94073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:11.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.533+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 shutdown_connections 2023-10-21T16:18:11.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:11.533+0000 7faf9c708640 1 -- 172.21.15.53:0/3804314722 wait complete. 2023-10-21T16:18:11.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:18:04.863937+0000', 'modified': '2023-10-21T16:18:07.895846+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35624}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35624': {'gid': 35624, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/797303097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 797303097}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 797303097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:18:11.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:18:11.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-21T16:18:12.007 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:12.004+0000 7fc258a72640 -1 client.35537: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35465:8e0c5166-c218-4c15-b84d-d0ba6d7bc017:v1:172.21.15.124:0/3415687515] 2023-10-21T16:18:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.200+0000 7f161d64b640 1 Processor -- start 2023-10-21T16:18:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.200+0000 7f161d64b640 1 -- start start 2023-10-21T16:18:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f161d64b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f161806a460 0x7f16180ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f161d64b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1618071460 con 0x7f161815ac80 2023-10-21T16:18:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f161d64b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1618071740 con 0x7f1618072220 2023-10-21T16:18:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f161d64b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16180aefd0 con 0x7f161806a460 2023-10-21T16:18:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f1616ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f161815ac80 0x7f1618069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60160/0 (socket says 172.21.15.53:60160) 2023-10-21T16:18:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f160e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f161806a460 0x7f16180ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.202+0000 7f1616ffd640 1 -- 172.21.15.53:0/3986839502 learned_addr learned my addr 172.21.15.53:0/3986839502 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.203+0000 7f16167fc640 1 -- 172.21.15.53:0/3986839502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824432844 0 0) 0x7f1618071460 con 0x7f161815ac80 2023-10-21T16:18:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.203+0000 7f16167fc640 1 -- 172.21.15.53:0/3986839502 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1604003520 con 0x7f161815ac80 2023-10-21T16:18:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.203+0000 7f160e7fc640 1 -- 172.21.15.53:0/3986839502 >> v1:172.21.15.124:6789/0 conn(0x7f1618072220 legacy=0x7f16180b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.203+0000 7f160e7fc640 1 -- 172.21.15.53:0/3986839502 >> v1:172.21.15.53:6789/0 conn(0x7f161815ac80 legacy=0x7f1618069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.203+0000 7f160e7fc640 1 -- 172.21.15.53:0/3986839502 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618071a20 con 0x7f161806a460 2023-10-21T16:18:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.204+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3986839502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161806a460 0x7f16180ca0c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c2620dae5e9cea server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.204+0000 7f16167fc640 1 -- 172.21.15.53:0/3986839502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160002e070 con 0x7f161806a460 2023-10-21T16:18:12.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.204+0000 7f16167fc640 1 -- 172.21.15.53:0/3986839502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000315a0 con 0x7f161806a460 2023-10-21T16:18:12.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f16167fc640 1 -- 172.21.15.53:0/3986839502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600031930 con 0x7f161806a460 2023-10-21T16:18:12.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 -- 172.21.15.53:0/3986839502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161806a460 msgr2=0x7f16180ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 --2- 172.21.15.53:0/3986839502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161806a460 0x7f16180ca0c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b840 comp rx=0 tx=0).stop 2023-10-21T16:18:12.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 -- 172.21.15.53:0/3986839502 shutdown_connections 2023-10-21T16:18:12.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 --2- 172.21.15.53:0/3986839502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161806a460 0x7f16180ca0c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 -- 172.21.15.53:0/3986839502 >> 172.21.15.53:0/3986839502 conn(0x7f161806d430 msgr2=0x7f161806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 -- 172.21.15.53:0/3986839502 shutdown_connections 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.205+0000 7f161d64b640 1 -- 172.21.15.53:0/3986839502 wait complete. 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.207+0000 7f161d64b640 1 Processor -- start 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.207+0000 7f161d64b640 1 -- start start 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161806a460 0x7f1618158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161815ac80 0x7f16180ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618158bb0 con 0x7f1618072220 2023-10-21T16:18:12.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180c40a0 con 0x7f161806a460 2023-10-21T16:18:12.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.208+0000 7f161d64b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16180c6ad0 con 0x7f161815ac80 2023-10-21T16:18:12.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53430/0 (socket says 172.21.15.53:53430) 2023-10-21T16:18:12.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 -- 172.21.15.53:0/245921207 learned_addr learned my addr 172.21.15.53:0/245921207 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f1616ffd640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161806a460 0x7f1618158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f16177fe640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161815ac80 0x7f16180ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 -- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161815ac80 msgr2=0x7f16180ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161815ac80 0x7f16180ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 -- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161806a460 msgr2=0x7f1618158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161806a460 0x7f1618158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 -- 172.21.15.53:0/245921207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009400 con 0x7f1618072220 2023-10-21T16:18:12.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.209+0000 7f160e7fc640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1600034040 tx=0x7f16000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.210+0000 7f160ffff640 1 -- 172.21.15.53:0/245921207 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600032840 con 0x7f1618072220 2023-10-21T16:18:12.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.210+0000 7f160ffff640 1 -- 172.21.15.53:0/245921207 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000329c0 con 0x7f1618072220 2023-10-21T16:18:12.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.210+0000 7f160ffff640 1 -- 172.21.15.53:0/245921207 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600032d70 con 0x7f1618072220 2023-10-21T16:18:12.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.210+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16180c6db0 con 0x7f1618072220 2023-10-21T16:18:12.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.210+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181f2340 con 0x7f1618072220 2023-10-21T16:18:12.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.214+0000 7f160ffff640 1 -- 172.21.15.53:0/245921207 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f160003b3f0 con 0x7f1618072220 2023-10-21T16:18:12.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.215+0000 7f160ffff640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.215+0000 7f160ffff640 1 -- 172.21.15.53:0/245921207 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f16000ab800 con 0x7f1618072220 2023-10-21T16:18:12.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.215+0000 7f16201dc740 1 Processor -- start 2023-10-21T16:18:12.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.215+0000 7f1616ffd640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.216+0000 7f16201dc740 1 -- start start 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc053e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 0x5636fc083d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fbf747a0 con 0x5636fc0543c0 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fbfc6740 con 0x5636fc081360 2023-10-21T16:18:12.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636fbfc3e30 con 0x5636fc080f80 2023-10-21T16:18:12.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f1616ffd640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 0x7f15f806d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1608005880 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53440/0 (socket says 172.21.15.53:53440) 2023-10-21T16:18:12.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f16177fe640 1 -- 172.21.15.53:0/3195772825 learned_addr learned my addr 172.21.15.53:0/3195772825 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f1616ffd640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc053e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.217+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 0x5636fc083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 -- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 msgr2=0x5636fc053e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc053e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 -- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 msgr2=0x5636fc083d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 0x5636fc083d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 -- 172.21.15.53:0/3195772825 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636fbf83910 con 0x5636fc0543c0 2023-10-21T16:18:12.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 0x5636fc083d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:12.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f16177fe640 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f16100044b0 tx=0x7f161000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec9c7b5c787967f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.218+0000 7f160cff9640 1 -- 172.21.15.53:0/3195772825 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000dd60 con 0x5636fc0543c0 2023-10-21T16:18:12.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f160cff9640 1 -- 172.21.15.53:0/3195772825 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100123f0 con 0x5636fc0543c0 2023-10-21T16:18:12.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f160cff9640 1 -- 172.21.15.53:0/3195772825 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100127a0 con 0x5636fc0543c0 2023-10-21T16:18:12.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 -- 172.21.15.53:0/3195772825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 msgr2=0x5636fc084250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f16100044b0 tx=0x7f161000d150 comp rx=0 tx=0).stop 2023-10-21T16:18:12.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 -- 172.21.15.53:0/3195772825 shutdown_connections 2023-10-21T16:18:12.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc0543c0 0x5636fc084250 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc081360 0x5636fc083d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 --2- 172.21.15.53:0/3195772825 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc053e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.219+0000 7f16201dc740 1 -- 172.21.15.53:0/3195772825 >> 172.21.15.53:0/3195772825 conn(0x5636fc056190 msgr2=0x5636fc09ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.220+0000 7f16201dc740 1 -- 172.21.15.53:0/3195772825 shutdown_connections 2023-10-21T16:18:12.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.220+0000 7f16201dc740 1 -- 172.21.15.53:0/3195772825 wait complete. 2023-10-21T16:18:12.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.220+0000 7f16201dc740 1 -- start start 2023-10-21T16:18:12.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.221+0000 7f16201dc740 1 Processor -- start 2023-10-21T16:18:12.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc083860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fbfbf720 con 0x5636fc081360 2023-10-21T16:18:12.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fbf5d970 con 0x5636fc0b2af0 2023-10-21T16:18:12.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636fbf5d610 con 0x5636fc080f80 2023-10-21T16:18:12.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16201dc740 10 client.0 _ll_register_callbacks cb 0x5636fbf7f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:18:12.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53450/0 (socket says 172.21.15.53:53450) 2023-10-21T16:18:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 -- 172.21.15.53:0/2306043639 learned_addr learned my addr 172.21.15.53:0/2306043639 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc083860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f16177fe640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 -- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 msgr2=0x5636fc083860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc083860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 -- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 msgr2=0x5636fc0856d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.222+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0856d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f160e7fc640 1 -- 172.21.15.53:0/2306043639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100040b0 con 0x5636fc081360 2023-10-21T16:18:12.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f160007a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7cbe61a57c95d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f15f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:18:12.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160007e3e0 con 0x5636fc081360 2023-10-21T16:18:12.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1600082590 con 0x5636fc081360 2023-10-21T16:18:12.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636fbfe6160 con 0x5636fc081360 2023-10-21T16:18:12.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16000769a0 con 0x5636fc081360 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f16201dc740 10 client.7236 fetch_fsmap learned FSMap version 44 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.223+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5636fbfbdda0 con 0x5636fc081360 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636fbfe6160 con 0x5636fc081360 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f160007a4f0 con 0x5636fc081360 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 10 client.7236 fetch_fsmap finished waiting for FSMap version 44 2023-10-21T16:18:12.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 10 client.7236 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:18:12.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 20 client.7236 populate_metadata read hostname 'smithi053' 2023-10-21T16:18:12.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc07a2b0 0x5636fc07a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 4 client.7236 mds_command: new command op to 35624 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:18:12.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 --> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636fbf91cc0 con 0x5636fc07a2b0 2023-10-21T16:18:12.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.224+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc07a2b0 0x5636fc07a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.225+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc07a2b0 0x5636fc07a690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd27b99d217a614f server_cookie=b167312895feb25e in_seq=0 out_seq=0 2023-10-21T16:18:12.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.225+0000 7f15f5ffb640 10 client.7236 ms_handle_connect on v2:172.21.15.53:6833/797303097 2023-10-21T16:18:12.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2306043639 <== mds.0 v2:172.21.15.53:6833/797303097 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5636fbf91cc0 con 0x5636fc07a2b0 2023-10-21T16:18:12.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f15f5ffb640 10 client.7236 handle_command_reply: tid=0 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 client.7236 shutdown 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 msgr2=0x5636fc09f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1600000c00 tx=0x7f160007a990 comp rx=0 tx=0).stop 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 shutdown_connections 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc07a2b0 0x5636fc07a690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0856d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc09f480 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f15f5ffb640 0 client.7236 ms_handle_reset on v2:172.21.15.53:6833/797303097 2023-10-21T16:18:12.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 --2- 172.21.15.53:0/2306043639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc083860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 >> 172.21.15.53:0/2306043639 conn(0x5636fc0543c0 msgr2=0x5636fc0a2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 shutdown_connections 2023-10-21T16:18:12.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 1 -- 172.21.15.53:0/2306043639 wait complete. 2023-10-21T16:18:12.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.226+0000 7f16201dc740 20 client.7236 trim_cache size 0 max 16384 2023-10-21T16:18:12.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.243+0000 7f16201dc740 1 Processor -- start 2023-10-21T16:18:12.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.243+0000 7f16201dc740 1 -- start start 2023-10-21T16:18:12.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc0a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc057b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fbd23f20 con 0x5636fc081360 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fc09eec0 con 0x5636fc0b2af0 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.244+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636fc07bc10 con 0x5636fc080f80 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53462/0 (socket says 172.21.15.53:53462) 2023-10-21T16:18:12.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 -- 172.21.15.53:0/3383545010 learned_addr learned my addr 172.21.15.53:0/3383545010 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f1616ffd640 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc0a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f16177fe640 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc057b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 -- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 msgr2=0x5636fc0a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc0a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 -- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 msgr2=0x5636fc057b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc057b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.245+0000 7f160e7fc640 1 -- 172.21.15.53:0/3383545010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636fbf5cb80 con 0x5636fc081360 2023-10-21T16:18:12.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f160e7fc640 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1600076cb0 tx=0x7f160008e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869e3f1ae40b22c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f160d7fa640 1 -- 172.21.15.53:0/3383545010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16000769a0 con 0x5636fc081360 2023-10-21T16:18:12.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f160d7fa640 1 -- 172.21.15.53:0/3383545010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600082590 con 0x5636fc081360 2023-10-21T16:18:12.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f160d7fa640 1 -- 172.21.15.53:0/3383545010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160007e3e0 con 0x5636fc081360 2023-10-21T16:18:12.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f16201dc740 1 -- 172.21.15.53:0/3383545010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 msgr2=0x5636fc0a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.246+0000 7f16201dc740 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1600076cb0 tx=0x7f160008e000 comp rx=0 tx=0).stop 2023-10-21T16:18:12.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 -- 172.21.15.53:0/3383545010 shutdown_connections 2023-10-21T16:18:12.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc057b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc0a2320 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 --2- 172.21.15.53:0/3383545010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc0a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 -- 172.21.15.53:0/3383545010 >> 172.21.15.53:0/3383545010 conn(0x5636fc0543c0 msgr2=0x5636fc0c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 -- 172.21.15.53:0/3383545010 shutdown_connections 2023-10-21T16:18:12.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 -- 172.21.15.53:0/3383545010 wait complete. 2023-10-21T16:18:12.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.247+0000 7f16201dc740 1 -- start start 2023-10-21T16:18:12.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.248+0000 7f16201dc740 1 Processor -- start 2023-10-21T16:18:12.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc09e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fc1066b0 con 0x5636fc081360 2023-10-21T16:18:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636fc052e40 con 0x5636fc0b2af0 2023-10-21T16:18:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636fc085950 con 0x5636fc080f80 2023-10-21T16:18:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16201dc740 10 client.0 _ll_register_callbacks cb 0x7f1600032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:18:12.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53478/0 (socket says 172.21.15.53:53478) 2023-10-21T16:18:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 -- 172.21.15.53:0/2524648256 learned_addr learned my addr 172.21.15.53:0/2524648256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc09e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 -- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 msgr2=0x5636fc09e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc09e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 -- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 msgr2=0x5636fc0ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f16177fe640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.249+0000 7f160e7fc640 1 -- 172.21.15.53:0/2524648256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100040b0 con 0x5636fc081360 2023-10-21T16:18:12.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f160e7fc640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f16000785e0 tx=0x7f160007c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8eaebeeeab965d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:18:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160007e3e0 con 0x5636fc081360 2023-10-21T16:18:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1600079980 con 0x5636fc081360 2023-10-21T16:18:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16100123f0 con 0x5636fc081360 2023-10-21T16:18:12.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16000769a0 con 0x5636fc081360 2023-10-21T16:18:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f16201dc740 10 client.7242 fetch_fsmap learned FSMap version 44 2023-10-21T16:18:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5636fbf82170 con 0x5636fc081360 2023-10-21T16:18:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100123f0 con 0x5636fc081360 2023-10-21T16:18:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.250+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1600082100 con 0x5636fc081360 2023-10-21T16:18:12.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 10 client.7242 fetch_fsmap finished waiting for FSMap version 44 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 10 client.7242 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 20 client.7242 populate_metadata read hostname 'smithi053' 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc0b0cd0 0x5636fc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 4 client.7242 mds_command: new command op to 35624 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 --> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5636fbfb8700 con 0x5636fc0b0cd0 2023-10-21T16:18:12.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc0b0cd0 0x5636fc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f1616ffd640 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc0b0cd0 0x5636fc0b10b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2537616018d6ef39 server_cookie=925e0aadeea881fa in_seq=0 out_seq=0 2023-10-21T16:18:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.251+0000 7f15f5ffb640 10 client.7242 ms_handle_connect on v2:172.21.15.53:6833/797303097 2023-10-21T16:18:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f15f5ffb640 1 -- 172.21.15.53:0/2524648256 <== mds.0 v2:172.21.15.53:6833/797303097 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5636fbfb8700 con 0x5636fc0b0cd0 2023-10-21T16:18:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f15f5ffb640 10 client.7242 handle_command_reply: tid=0 2023-10-21T16:18:12.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 client.7242 shutdown 2023-10-21T16:18:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 msgr2=0x5636fc085410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f16000785e0 tx=0x7f160007c000 comp rx=0 tx=0).stop 2023-10-21T16:18:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 shutdown_connections 2023-10-21T16:18:12.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:6833/797303097,v1:172.21.15.53:6835/797303097] conn(0x5636fc0b0cd0 0x5636fc0b10b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5636fc0b2af0 0x5636fc0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636fc081360 0x5636fc085410 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 --2- 172.21.15.53:0/2524648256 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5636fc080f80 0x5636fc09e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 >> 172.21.15.53:0/2524648256 conn(0x5636fc0543c0 msgr2=0x5636fc057b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f15f5ffb640 0 client.7242 ms_handle_reset on v2:172.21.15.53:6833/797303097 2023-10-21T16:18:12.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 shutdown_connections 2023-10-21T16:18:12.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.252+0000 7f16201dc740 1 -- 172.21.15.53:0/2524648256 wait complete. 2023-10-21T16:18:12.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f16201dc740 20 client.7242 trim_cache size 0 max 16384 2023-10-21T16:18:12.267 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:18:12.268 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:18:12.268 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:18:12.268 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 35624, 2023-10-21T16:18:12.268 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:18:12.268 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:18:12.269 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:18:12.269 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8606000309999997, 2023-10-21T16:18:12.269 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 44, 2023-10-21T16:18:12.269 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 103, 2023-10-21T16:18:12.269 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 103, 2023-10-21T16:18:12.270 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.607273359000001 2023-10-21T16:18:12.270 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:18:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 msgr2=0x7f15f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 0x7f15f806d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1608005880 tx=0x7f1608009040 comp rx=0 tx=0).stop 2023-10-21T16:18:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 msgr2=0x7f16180ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1600034040 tx=0x7f16000046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 shutdown_connections 2023-10-21T16:18:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f15f806af00 0x7f15f806d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161815ac80 0x7f16180ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1618072220 0x7f16180ac920 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 --2- 172.21.15.53:0/245921207 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f161806a460 0x7f1618158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 >> 172.21.15.53:0/245921207 conn(0x7f161806d430 msgr2=0x7f16180b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 shutdown_connections 2023-10-21T16:18:12.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.253+0000 7f161d64b640 1 -- 172.21.15.53:0/245921207 wait complete. 2023-10-21T16:18:12.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:18:12.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.482+0000 7f6e7e626640 1 Processor -- start 2023-10-21T16:18:12.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.483+0000 7f6e7e626640 1 -- start start 2023-10-21T16:18:12.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e7e626640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7806d4c0 0x7f6e78162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e7e626640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e780cb150 0x7f6e780cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e7e626640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780ae140 con 0x7f6e780add60 2023-10-21T16:18:12.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e7e626640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e78161f10 con 0x7f6e7806d4c0 2023-10-21T16:18:12.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e7e626640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780cbb20 con 0x7f6e780cb150 2023-10-21T16:18:12.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7806d4c0 0x7f6e78162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6e7806d4c0 0x7f6e78162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53482/0 (socket says 172.21.15.53:53482) 2023-10-21T16:18:12.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 -- 172.21.15.53:0/1003641744 learned_addr learned my addr 172.21.15.53:0/1003641744 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 -- 172.21.15.53:0/1003641744 >> v2:172.21.15.124:3300/0 conn(0x7f6e780cb150 msgr2=0x7f6e780cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1003641744 >> v2:172.21.15.124:3300/0 conn(0x7f6e780cb150 0x7f6e780cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 -- 172.21.15.53:0/1003641744 >> v1:172.21.15.124:6789/0 conn(0x7f6e780add60 legacy=0x7f6e780cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:12.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.485+0000 7f6e6ffff640 1 -- 172.21.15.53:0/1003641744 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780ccf20 con 0x7f6e7806d4c0 2023-10-21T16:18:12.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1003641744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e78162330 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6e780cb0b0 tx=0x7f6e6002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6ef6e6c06e6240 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e777fe640 1 -- 172.21.15.53:0/1003641744 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e600317e0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e777fe640 1 -- 172.21.15.53:0/1003641744 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60031940 con 0x7f6e7806d4c0 2023-10-21T16:18:12.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e777fe640 1 -- 172.21.15.53:0/1003641744 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60031cd0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e7e626640 1 -- 172.21.15.53:0/1003641744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 msgr2=0x7f6e78162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.487+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1003641744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e78162330 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6e780cb0b0 tx=0x7f6e6002b880 comp rx=0 tx=0).stop 2023-10-21T16:18:12.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 -- 172.21.15.53:0/1003641744 shutdown_connections 2023-10-21T16:18:12.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1003641744 >> v2:172.21.15.124:3300/0 conn(0x7f6e780cb150 0x7f6e780cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1003641744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e78162330 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 -- 172.21.15.53:0/1003641744 >> 172.21.15.53:0/1003641744 conn(0x7f6e7806a380 msgr2=0x7f6e780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 -- 172.21.15.53:0/1003641744 shutdown_connections 2023-10-21T16:18:12.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.488+0000 7f6e7e626640 1 -- 172.21.15.53:0/1003641744 wait complete. 2023-10-21T16:18:12.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.489+0000 7f6e7e626640 1 Processor -- start 2023-10-21T16:18:12.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.490+0000 7f6e7e626640 1 -- start start 2023-10-21T16:18:12.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 0x7f6e78159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e780cb150 0x7f6e7815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780aed00 con 0x7f6e7806d4c0 2023-10-21T16:18:12.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780aefe0 con 0x7f6e780add60 2023-10-21T16:18:12.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7e626640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780af2c0 con 0x7f6e780cb150 2023-10-21T16:18:12.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53494/0 (socket says 172.21.15.53:53494) 2023-10-21T16:18:12.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 -- 172.21.15.53:0/1739189303 learned_addr learned my addr 172.21.15.53:0/1739189303 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:12.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e7cbc3640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e780cb150 0x7f6e7815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 -- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e780cb150 msgr2=0x7f6e7815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e780cb150 0x7f6e7815a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 -- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 msgr2=0x7f6e78159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 0x7f6e78159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.491+0000 7f6e77fff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 0x7f6e78159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e77fff640 1 -- 172.21.15.53:0/1739189303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600094d0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 0x7f6e78159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:12.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e77fff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c410 tx=0x7f6e6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680035f0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68003750 con 0x7f6e7806d4c0 2023-10-21T16:18:12.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003b40 con 0x7f6e7806d4c0 2023-10-21T16:18:12.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e780af5a0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.492+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7815bbb0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.494+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e780cb7f0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.495+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6e68003ca0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.496+0000 7f6e757fa640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 0x7f6e5806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:12.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.496+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e68088a00 con 0x7f6e7806d4c0 2023-10-21T16:18:12.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.496+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 0x7f6e5806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:12.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.497+0000 7f6e6ffff640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 0x7f6e5806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e60036000 tx=0x7f6e6003d040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:12.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.500+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e680538e0 con 0x7f6e7806d4c0 2023-10-21T16:18:12.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.740+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e78067490 con 0x7f6e7806d4c0 2023-10-21T16:18:12.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.741+0000 7f6e757fa640 1 -- 172.21.15.53:0/1739189303 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e68056f00 con 0x7f6e7806d4c0 2023-10-21T16:18:12.743 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:18:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 msgr2=0x7f6e5806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 0x7f6e5806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6e60036000 tx=0x7f6e6003d040 comp rx=0 tx=0).stop 2023-10-21T16:18:12.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 msgr2=0x7f6e780ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:12.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c410 tx=0x7f6e6800c8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 shutdown_connections 2023-10-21T16:18:12.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f6e5806af30 0x7f6e5806d3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e780cb150 0x7f6e7815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e780add60 0x7f6e78159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 --2- 172.21.15.53:0/1739189303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e7806d4c0 0x7f6e780ae320 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:12.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 >> 172.21.15.53:0/1739189303 conn(0x7f6e7806a380 msgr2=0x7f6e78160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:12.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 shutdown_connections 2023-10-21T16:18:12.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:12.744+0000 7f6e7e626640 1 -- 172.21.15.53:0/1739189303 wait complete. 2023-10-21T16:18:12.764 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:18:12.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:18:12.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:18:12.765 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:18:12.765 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:12.765 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:18:12.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:18:12.766 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:18:12.766 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:12.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:12.767 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:18:12.823 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:18:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:18:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:18:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:18:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.824 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:18:12.825 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:18:12.825 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:18:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:18:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:18:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41400sec preferred_lft 41400sec 2023-10-21T16:18:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:18:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.827 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:18:12.827 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:18:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:18:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:18:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.829 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:18:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:18:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:18:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.829 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:18:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:18:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:18:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:12.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:12.832 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:18:12.933 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:18:12.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:12.937 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:18:12.994 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:18:12.995 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:18:12.995 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:18:12.995 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:18:12.995 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:18:12.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:12.997 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:18:12.997 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:18:12.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:18:12.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:18:12.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:18:12.999 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:18:13.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:13.196 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:18:13.252 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:18:13.253 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:18:13.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:13.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:13.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:18:13.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:18:13.366 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:13.431 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:13.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:18:13.432 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:13.483 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:18:13.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:13.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:13.619 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:13.619 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:18:13.660 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:13.660 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:18:13.661 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:18:13.661 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:18:13.661 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:18:13.665 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:18:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:18:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:18:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:18:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:18:13.670 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:18:13.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:13.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:13.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:13.766 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:18:13.840 INFO:teuthology.orchestra.run.smithi053.stdout:client35693 2023-10-21T16:18:13.840 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:13.841 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:13.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:18:14.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.127+0000 7feb8ffff640 1 Processor -- start 2023-10-21T16:18:14.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.128+0000 7feb8ffff640 1 -- start start 2023-10-21T16:18:14.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.129+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 0x7feb901594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.129+0000 7feb8ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb90071d90 con 0x7feb90070ac0 2023-10-21T16:18:14.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb8ffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb90070f20 con 0x7feb900b0680 2023-10-21T16:18:14.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb8ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb90071180 con 0x7feb90156a10 2023-10-21T16:18:14.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb867fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb90070ac0 0x7feb900aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42822/0 (socket says 172.21.15.53:42822) 2023-10-21T16:18:14.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb867fc640 1 -- 172.21.15.53:0/474817283 learned_addr learned my addr 172.21.15.53:0/474817283 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:14.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363609937 0 0) 0x7feb90071d90 con 0x7feb90070ac0 2023-10-21T16:18:14.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb8effd640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 0x7feb901594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.130+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb7c003520 con 0x7feb90070ac0 2023-10-21T16:18:14.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1660954759 0 0) 0x7feb7c003520 con 0x7feb90070ac0 2023-10-21T16:18:14.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 msgr2=0x7feb901594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:14.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 0x7feb901594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 >> v1:172.21.15.124:6789/0 conn(0x7feb900b0680 legacy=0x7feb900c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90071460 con 0x7feb90070ac0 2023-10-21T16:18:14.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7feb78001670 con 0x7feb90070ac0 2023-10-21T16:18:14.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 >> v1:172.21.15.53:6789/0 conn(0x7feb90070ac0 legacy=0x7feb900aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0039a0 0x7feb7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 0x7feb7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c004d00 0x7feb7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90071180 con 0x7feb7c0039a0 2023-10-21T16:18:14.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c00ba30 con 0x7feb7c004d00 2023-10-21T16:18:14.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c00de00 con 0x7feb90156a10 2023-10-21T16:18:14.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.131+0000 7feb8effd640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0039a0 0x7feb7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8effd640 1 -- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 msgr2=0x7feb7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:14.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8effd640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb90156a10 0x7feb7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8effd640 1 -- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c004d00 msgr2=0x7feb7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:14.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8f7fe640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c004d00 0x7feb7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8effd640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c004d00 0x7feb7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.132+0000 7feb8effd640 1 -- 172.21.15.53:0/474817283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c003520 con 0x7feb7c0039a0 2023-10-21T16:18:14.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.133+0000 7feb8effd640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0039a0 0x7feb7c003dc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feb80009a90 tx=0x7feb7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbc08c704db1ff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.133+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8002c6b0 con 0x7feb7c0039a0 2023-10-21T16:18:14.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.133+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8002c810 con 0x7feb7c0039a0 2023-10-21T16:18:14.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.133+0000 7feb8e7fc640 1 -- 172.21.15.53:0/474817283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8002cba0 con 0x7feb7c0039a0 2023-10-21T16:18:14.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 -- 172.21.15.53:0/474817283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0039a0 msgr2=0x7feb7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 --2- 172.21.15.53:0/474817283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb7c0039a0 0x7feb7c003dc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feb80009a90 tx=0x7feb7c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8f7fe640 1 -- 172.21.15.53:0/474817283 reap_dead start 2023-10-21T16:18:14.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 -- 172.21.15.53:0/474817283 shutdown_connections 2023-10-21T16:18:14.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 -- 172.21.15.53:0/474817283 >> 172.21.15.53:0/474817283 conn(0x7feb9006c050 msgr2=0x7feb9006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:14.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 -- 172.21.15.53:0/474817283 shutdown_connections 2023-10-21T16:18:14.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.134+0000 7feb8ffff640 1 -- 172.21.15.53:0/474817283 wait complete. 2023-10-21T16:18:14.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.136+0000 7feb8ffff640 1 Processor -- start 2023-10-21T16:18:14.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.136+0000 7feb8ffff640 1 -- start start 2023-10-21T16:18:14.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c0039a0 0x7feb901f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb900c62f0 0x7feb900c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900cdae0 con 0x7feb900cd0f0 2023-10-21T16:18:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9014f2d0 con 0x7feb7c0039a0 2023-10-21T16:18:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c5b30 con 0x7feb900c62f0 2023-10-21T16:18:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47822/0 (socket says 172.21.15.53:47822) 2023-10-21T16:18:14.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.137+0000 7feb8f7fe640 1 -- 172.21.15.53:0/1226971957 learned_addr learned my addr 172.21.15.53:0/1226971957 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8effd640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c0039a0 0x7feb901f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb867fc640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb900c62f0 0x7feb900c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 -- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb900c62f0 msgr2=0x7feb900c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb900c62f0 0x7feb900c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 -- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c0039a0 msgr2=0x7feb901f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c0039a0 0x7feb901f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 -- 172.21.15.53:0/1226971957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb901f2b60 con 0x7feb900cd0f0 2023-10-21T16:18:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8f7fe640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7feb900cf5f0 tx=0x7feb8800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88003b70 con 0x7feb900cd0f0 2023-10-21T16:18:14.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88003cd0 con 0x7feb900cd0f0 2023-10-21T16:18:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb880052b0 con 0x7feb900cd0f0 2023-10-21T16:18:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.138+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb901f2e40 con 0x7feb900cd0f0 2023-10-21T16:18:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.139+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900c5dd0 con 0x7feb900cd0f0 2023-10-21T16:18:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.140+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb900c5ff0 con 0x7feb900cd0f0 2023-10-21T16:18:14.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.141+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7feb88005410 con 0x7feb900cd0f0 2023-10-21T16:18:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.142+0000 7feb87fff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 0x7feb7006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.142+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6116+0+0 (secure 0 0 0) 0x7feb88089380 con 0x7feb900cd0f0 2023-10-21T16:18:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.142+0000 7feb8effd640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 0x7feb7006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.144+0000 7feb8effd640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 0x7feb7006d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb80036000 tx=0x7feb80030000 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.146+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb88054120 con 0x7feb900cd0f0 2023-10-21T16:18:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.368+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 --> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feb90071a20 con 0x7feb7006af30 2023-10-21T16:18:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.372+0000 7feb87fff640 1 -- 172.21.15.53:0/1226971957 <== mgr.6082 v2:172.21.15.124:6832/3674677134 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7feb80004040 con 0x7feb7006af30 2023-10-21T16:18:14.374 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:18:14.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:14.374 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:18:14.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:14.375 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:18:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 msgr2=0x7feb7006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 0x7feb7006d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb80036000 tx=0x7feb80030000 comp rx=0 tx=0).stop 2023-10-21T16:18:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 msgr2=0x7feb900cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7feb900cf5f0 tx=0x7feb8800dc00 comp rx=0 tx=0).stop 2023-10-21T16:18:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 shutdown_connections 2023-10-21T16:18:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7feb7006af30 0x7feb7006d3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb900cd0f0 0x7feb900cd5a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb900c62f0 0x7feb900c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 --2- 172.21.15.53:0/1226971957 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb7c0039a0 0x7feb901f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 >> 172.21.15.53:0/1226971957 conn(0x7feb9006c050 msgr2=0x7feb9015b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.375+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 shutdown_connections 2023-10-21T16:18:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.376+0000 7feb8ffff640 1 -- 172.21.15.53:0/1226971957 wait complete. 2023-10-21T16:18:14.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:18:14.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.623+0000 7fe80b1a9640 1 Processor -- start 2023-10-21T16:18:14.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.623+0000 7fe80b1a9640 1 -- start start 2023-10-21T16:18:14.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.624+0000 7fe80b1a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8040afbe0 0x7fe8040b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe80b1a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 0x7fe804070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe80b1a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe804071070 0x7fe8040cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe80b1a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8040b2620 con 0x7fe804071070 2023-10-21T16:18:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe80b1a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8040b2060 con 0x7fe80406dfc0 2023-10-21T16:18:14.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe80b1a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8040b2340 con 0x7fe8040afbe0 2023-10-21T16:18:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe809746640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe804071070 0x7fe8040cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe809746640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe804071070 0x7fe8040cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47830/0 (socket says 172.21.15.53:47830) 2023-10-21T16:18:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe809746640 1 -- 172.21.15.53:0/3150867822 learned_addr learned my addr 172.21.15.53:0/3150867822 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe803fff640 1 --2- 172.21.15.53:0/3150867822 >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 0x7fe804070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.625+0000 7fe808f45640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8040afbe0 0x7fe8040b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe809746640 1 -- 172.21.15.53:0/3150867822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8040afbe0 msgr2=0x7fe8040b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe809746640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8040afbe0 0x7fe8040b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe809746640 1 -- 172.21.15.53:0/3150867822 >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 msgr2=0x7fe804070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:14.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe809746640 1 --2- 172.21.15.53:0/3150867822 >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 0x7fe804070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe809746640 1 -- 172.21.15.53:0/3150867822 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804070350 con 0x7fe804071070 2023-10-21T16:18:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.626+0000 7fe803fff640 1 --2- 172.21.15.53:0/3150867822 >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 0x7fe804070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.627+0000 7fe809746640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe804071070 0x7fe8040cb130 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe8040cb0b0 tx=0x7fe7fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c1f6385910cb25 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.627+0000 7fe8037fe640 1 -- 172.21.15.53:0/3150867822 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc02ca80 con 0x7fe804071070 2023-10-21T16:18:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.627+0000 7fe8037fe640 1 -- 172.21.15.53:0/3150867822 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc02cbe0 con 0x7fe804071070 2023-10-21T16:18:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.627+0000 7fe8037fe640 1 -- 172.21.15.53:0/3150867822 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0343f0 con 0x7fe804071070 2023-10-21T16:18:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 -- 172.21.15.53:0/3150867822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe804071070 msgr2=0x7fe8040cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe804071070 0x7fe8040cb130 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fe8040cb0b0 tx=0x7fe7fc02bee0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 -- 172.21.15.53:0/3150867822 shutdown_connections 2023-10-21T16:18:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe804071070 0x7fe8040cb130 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/3150867822 >> v2:172.21.15.124:3300/0 conn(0x7fe80406dfc0 0x7fe804070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/3150867822 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8040afbe0 0x7fe8040b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 -- 172.21.15.53:0/3150867822 >> 172.21.15.53:0/3150867822 conn(0x7fe80406a5d0 msgr2=0x7fe8040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:14.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 -- 172.21.15.53:0/3150867822 shutdown_connections 2023-10-21T16:18:14.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.628+0000 7fe80b1a9640 1 -- 172.21.15.53:0/3150867822 wait complete. 2023-10-21T16:18:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.630+0000 7fe80b1a9640 1 Processor -- start 2023-10-21T16:18:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.630+0000 7fe80b1a9640 1 -- start start 2023-10-21T16:18:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe80406dfc0 0x7fe8040c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe804071070 0x7fe80414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804150400 con 0x7fe8040afbe0 2023-10-21T16:18:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe804155130 con 0x7fe804071070 2023-10-21T16:18:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe80b1a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80414b100 con 0x7fe80406dfc0 2023-10-21T16:18:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47846/0 (socket says 172.21.15.53:47846) 2023-10-21T16:18:14.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 -- 172.21.15.53:0/1959158089 learned_addr learned my addr 172.21.15.53:0/1959158089 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:14.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe808f45640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe80406dfc0 0x7fe8040c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 -- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe80406dfc0 msgr2=0x7fe8040c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe80406dfc0 0x7fe8040c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 -- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe804071070 msgr2=0x7fe80414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe804071070 0x7fe80414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.631+0000 7fe809746640 1 -- 172.21.15.53:0/1959158089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc009b30 con 0x7fe8040afbe0 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe809746640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009f40 tx=0x7fe7fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009510 con 0x7fe8040afbe0 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc03e450 con 0x7fe8040afbe0 2023-10-21T16:18:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc03e800 con 0x7fe8040afbe0 2023-10-21T16:18:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe804154800 con 0x7fe8040afbe0 2023-10-21T16:18:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.632+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe804154dc0 con 0x7fe8040afbe0 2023-10-21T16:18:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.633+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8041552c0 con 0x7fe8040afbe0 2023-10-21T16:18:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.634+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe7fc02cd50 con 0x7fe8040afbe0 2023-10-21T16:18:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.636+0000 7fe8017fa640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 0x7fe7e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.636+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe7fc0ac710 con 0x7fe8040afbe0 2023-10-21T16:18:14.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.637+0000 7fe808f45640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 0x7fe7e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:14.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.638+0000 7fe808f45640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 0x7fe7e806d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8006fd0 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:14.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.640+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7fc077530 con 0x7fe8040afbe0 2023-10-21T16:18:14.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:14.889+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe804070600 con 0x7fe8040afbe0 2023-10-21T16:18:15.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.443+0000 7fe8017fa640 1 -- 172.21.15.53:0/1959158089 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v38) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe7fc07ab50 con 0x7fe8040afbe0 2023-10-21T16:18:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 msgr2=0x7fe7e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 0x7fe7e806d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8006fd0 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2023-10-21T16:18:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 msgr2=0x7fe804145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009f40 tx=0x7fe7fc004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 shutdown_connections 2023-10-21T16:18:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7fe7e806af00 0x7fe7e806d3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8040afbe0 0x7fe804145be0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe804071070 0x7fe80414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 --2- 172.21.15.53:0/1959158089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe80406dfc0 0x7fe8040c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 >> 172.21.15.53:0/1959158089 conn(0x7fe80406a5d0 msgr2=0x7fe804151a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.447+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 shutdown_connections 2023-10-21T16:18:15.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.449+0000 7fe80b1a9640 1 -- 172.21.15.53:0/1959158089 wait complete. 2023-10-21T16:18:15.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:18:15.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.679+0000 7f289e42b640 1 Processor -- start 2023-10-21T16:18:15.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.679+0000 7f289e42b640 1 -- start start 2023-10-21T16:18:15.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.680+0000 7f289e42b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28980afbe0 0x7f28980b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f289e42b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980cb130 0x7f28980cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f289e42b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28980cbad0 con 0x7f289806dfc0 2023-10-21T16:18:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f289e42b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28980b2620 con 0x7f28980afbe0 2023-10-21T16:18:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f289e42b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980b2060 con 0x7f28980cb130 2023-10-21T16:18:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f2897fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28980afbe0 0x7f28980b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f2897fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28980afbe0 0x7f28980b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47870/0 (socket says 172.21.15.53:47870) 2023-10-21T16:18:15.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f2897fff640 1 -- 172.21.15.53:0/1870177819 learned_addr learned my addr 172.21.15.53:0/1870177819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.681+0000 7f289c9c8640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980cb130 0x7f28980cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.682+0000 7f2897fff640 1 -- 172.21.15.53:0/1870177819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980cb130 msgr2=0x7f28980cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.682+0000 7f2897fff640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980cb130 0x7f28980cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.682+0000 7f2897fff640 1 -- 172.21.15.53:0/1870177819 >> v1:172.21.15.53:6789/0 conn(0x7f289806dfc0 legacy=0x7f2898070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.682+0000 7f2897fff640 1 -- 172.21.15.53:0/1870177819 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980b2340 con 0x7f28980afbe0 2023-10-21T16:18:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.683+0000 7f2897fff640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980afbe0 0x7f28980b15b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2888009870 tx=0x7f288802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de14244e0069dcd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.683+0000 7f2896ffd640 1 -- 172.21.15.53:0/1870177819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888030840 con 0x7f28980afbe0 2023-10-21T16:18:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.683+0000 7f2896ffd640 1 -- 172.21.15.53:0/1870177819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28880309a0 con 0x7f28980afbe0 2023-10-21T16:18:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.683+0000 7f2896ffd640 1 -- 172.21.15.53:0/1870177819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888030d30 con 0x7f28980afbe0 2023-10-21T16:18:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 -- 172.21.15.53:0/1870177819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980afbe0 msgr2=0x7f28980b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980afbe0 0x7f28980b15b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2888009870 tx=0x7f288802b980 comp rx=0 tx=0).stop 2023-10-21T16:18:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 -- 172.21.15.53:0/1870177819 shutdown_connections 2023-10-21T16:18:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980cb130 0x7f28980cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 --2- 172.21.15.53:0/1870177819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980afbe0 0x7f28980b15b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 -- 172.21.15.53:0/1870177819 >> 172.21.15.53:0/1870177819 conn(0x7f289806a5d0 msgr2=0x7f28980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 -- 172.21.15.53:0/1870177819 shutdown_connections 2023-10-21T16:18:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.684+0000 7f289e42b640 1 -- 172.21.15.53:0/1870177819 wait complete. 2023-10-21T16:18:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.686+0000 7f289e42b640 1 Processor -- start 2023-10-21T16:18:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.686+0000 7f289e42b640 1 -- start start 2023-10-21T16:18:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 0x7f28980af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980afbe0 0x7f28981f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980cac40 con 0x7f28980cb130 2023-10-21T16:18:15.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980ca3a0 con 0x7f289806dfc0 2023-10-21T16:18:15.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289e42b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980ca680 con 0x7f28980afbe0 2023-10-21T16:18:15.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289c9c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:15.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289c9c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47872/0 (socket says 172.21.15.53:47872) 2023-10-21T16:18:15.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f289c9c8640 1 -- 172.21.15.53:0/2573248651 learned_addr learned my addr 172.21.15.53:0/2573248651 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.687+0000 7f28977fe640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980afbe0 0x7f28981f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f2897fff640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 0x7f28980af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980afbe0 msgr2=0x7f28981f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980afbe0 0x7f28981f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 msgr2=0x7f28980af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 0x7f28980af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 -- 172.21.15.53:0/2573248651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880094d0 con 0x7f28980cb130 2023-10-21T16:18:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f2897fff640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 0x7f28980af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f289c9c8640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2890007e40 tx=0x7f289000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890014620 con 0x7f28980cb130 2023-10-21T16:18:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289000ee90 con 0x7f28980cb130 2023-10-21T16:18:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.688+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28900149e0 con 0x7f28980cb130 2023-10-21T16:18:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.689+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28980ca960 con 0x7f28980cb130 2023-10-21T16:18:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.689+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28980c1ee0 con 0x7f28980cb130 2023-10-21T16:18:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.690+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289804e160 con 0x7f28980cb130 2023-10-21T16:18:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.691+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f2890014b40 con 0x7f28980cb130 2023-10-21T16:18:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.692+0000 7f2894ff9640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 0x7f288406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.692+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f289008a380 con 0x7f28980cb130 2023-10-21T16:18:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.692+0000 7f2897fff640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 msgr2=0x7f288406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.692+0000 7f2897fff640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 0x7f288406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.696+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28900550e0 con 0x7f28980cb130 2023-10-21T16:18:15.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.890+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f28980735d0 con 0x7f28980cb130 2023-10-21T16:18:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.893+0000 7f2897fff640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 msgr2=0x7f288406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:15.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.893+0000 7f2897fff640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 0x7f288406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:15.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.896+0000 7f2894ff9640 1 -- 172.21.15.53:0/2573248651 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2890058700 con 0x7f28980cb130 2023-10-21T16:18:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.898+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 msgr2=0x7f288406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 0x7f288406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 msgr2=0x7f289815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2890007e40 tx=0x7f289000e4b0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 shutdown_connections 2023-10-21T16:18:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f288406afe0 0x7f288406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28980cb130 0x7f289815d920 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28980afbe0 0x7f28981f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 --2- 172.21.15.53:0/2573248651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289806dfc0 0x7f28980af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:15.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 >> 172.21.15.53:0/2573248651 conn(0x7f289806a5d0 msgr2=0x7f28980c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:15.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 shutdown_connections 2023-10-21T16:18:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:15.899+0000 7f289e42b640 1 -- 172.21.15.53:0/2573248651 wait complete. 2023-10-21T16:18:15.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:18:15.964 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:15.962+0000 7fd64d445040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:15.986 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:15.985+0000 7fd10202d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:16.016 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.014+0000 7fd64d445040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:16.042 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.041+0000 7fd10202d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:16.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.137+0000 7f92ceb8a640 1 Processor -- start 2023-10-21T16:18:16.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.138+0000 7f92ceb8a640 1 -- start start 2023-10-21T16:18:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.139+0000 7f92ceb8a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92c80b1910 0x7f92c815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.139+0000 7f92ceb8a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c80b12b0 con 0x7f92c815f6d0 2023-10-21T16:18:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.139+0000 7f92ceb8a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c80b1500 con 0x7f92c815e3c0 2023-10-21T16:18:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.139+0000 7f92ceb8a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c80cf1d0 con 0x7f92c80b1910 2023-10-21T16:18:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92c80b1910 0x7f92c815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92c80b1910 0x7f92c815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47882/0 (socket says 172.21.15.53:47882) 2023-10-21T16:18:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 -- 172.21.15.53:0/1990837812 learned_addr learned my addr 172.21.15.53:0/1990837812 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92cd127640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f92c815f6d0 0x7f92c815c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42838/0 (socket says 172.21.15.53:42838) 2023-10-21T16:18:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 -- 172.21.15.53:0/1990837812 >> v1:172.21.15.124:6789/0 conn(0x7f92c815e3c0 legacy=0x7f92c815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 -- 172.21.15.53:0/1990837812 >> v1:172.21.15.53:6789/0 conn(0x7f92c815f6d0 legacy=0x7f92c815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.140+0000 7f92c7fff640 1 -- 172.21.15.53:0/1990837812 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c80cf4b0 con 0x7f92c80b1910 2023-10-21T16:18:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.141+0000 7f92c7fff640 1 --2- 172.21.15.53:0/1990837812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c80b1910 0x7f92c815f190 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009870 tx=0x7f92b402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bc70e1843cbaf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92c77fe640 1 -- 172.21.15.53:0/1990837812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b402f850 con 0x7f92c80b1910 2023-10-21T16:18:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92c77fe640 1 -- 172.21.15.53:0/1990837812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b402f9b0 con 0x7f92c80b1910 2023-10-21T16:18:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92c77fe640 1 -- 172.21.15.53:0/1990837812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b402fd40 con 0x7f92c80b1910 2023-10-21T16:18:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1990837812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c80b1910 msgr2=0x7f92c815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1990837812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c80b1910 0x7f92c815f190 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009870 tx=0x7f92b402ba30 comp rx=0 tx=0).stop 2023-10-21T16:18:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1990837812 shutdown_connections 2023-10-21T16:18:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1990837812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c80b1910 0x7f92c815f190 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.142+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1990837812 >> 172.21.15.53:0/1990837812 conn(0x7f92c8099d40 msgr2=0x7f92c80d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.143+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1990837812 shutdown_connections 2023-10-21T16:18:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.143+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1990837812 wait complete. 2023-10-21T16:18:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.144+0000 7f92ceb8a640 1 Processor -- start 2023-10-21T16:18:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.144+0000 7f92ceb8a640 1 -- start start 2023-10-21T16:18:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92c80b1910 0x7f92c80c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 0x7f92c80cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81f3200 con 0x7f92c815f6d0 2023-10-21T16:18:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81f28f0 con 0x7f92c815e3c0 2023-10-21T16:18:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.145+0000 7f92ceb8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c806cc50 con 0x7f92c80b1910 2023-10-21T16:18:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47884/0 (socket says 172.21.15.53:47884) 2023-10-21T16:18:16.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 -- 172.21.15.53:0/1757203771 learned_addr learned my addr 172.21.15.53:0/1757203771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:16.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cc926640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92c80b1910 0x7f92c80c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92c7fff640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 0x7f92c80cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92c80b1910 msgr2=0x7f92c80c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92c80b1910 0x7f92c80c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 msgr2=0x7f92c80cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 0x7f92c80cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92cd127640 1 -- 172.21.15.53:0/1757203771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40094d0 con 0x7f92c815f6d0 2023-10-21T16:18:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.146+0000 7f92c7fff640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 0x7f92c80cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92cd127640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f92c000c730 tx=0x7f92c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000ce40 con 0x7f92c815f6d0 2023-10-21T16:18:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c000a640 con 0x7f92c815f6d0 2023-10-21T16:18:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c81f2bb0 con 0x7f92c815f6d0 2023-10-21T16:18:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000a9f0 con 0x7f92c815f6d0 2023-10-21T16:18:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.147+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c80c9db0 con 0x7f92c815f6d0 2023-10-21T16:18:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.149+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c81f3370 con 0x7f92c815f6d0 2023-10-21T16:18:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.150+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f92c000ab50 con 0x7f92c815f6d0 2023-10-21T16:18:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.152+0000 7f92c57fa640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 0x7f92ac06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.152+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f92c0089090 con 0x7f92c815f6d0 2023-10-21T16:18:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.152+0000 7f92cc926640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 msgr2=0x7f92ac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.152+0000 7f92cc926640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 0x7f92ac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.156+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92c0053ea0 con 0x7f92c815f6d0 2023-10-21T16:18:16.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.348+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f92c804e160 con 0x7f92c815f6d0 2023-10-21T16:18:16.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.353+0000 7f92cc926640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 msgr2=0x7f92ac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:16.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.353+0000 7f92cc926640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 0x7f92ac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:16.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.355+0000 7f92c57fa640 1 -- 172.21.15.53:0/1757203771 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f92c00170c0 con 0x7f92c815f6d0 2023-10-21T16:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.357+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 msgr2=0x7f92ac06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.357+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 0x7f92ac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 msgr2=0x7f92c80ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f92c000c730 tx=0x7f92c000cbf0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 shutdown_connections 2023-10-21T16:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f92ac06afe0 0x7f92ac06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92c815f6d0 0x7f92c80ccdb0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92c815e3c0 0x7f92c80cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 --2- 172.21.15.53:0/1757203771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92c80b1910 0x7f92c80c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 >> 172.21.15.53:0/1757203771 conn(0x7f92c8099d40 msgr2=0x7f92c815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 shutdown_connections 2023-10-21T16:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.358+0000 7f92ceb8a640 1 -- 172.21.15.53:0/1757203771 wait complete. 2023-10-21T16:18:16.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-21T16:18:16.466 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.463+0000 7fd64d445040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:16.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.517+0000 7fd64d445040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:16.520 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.519+0000 7fd10202d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:16.569 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.567+0000 7fd64d445040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:16.575 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.574+0000 7fd10202d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:16.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.589+0000 7f8f64e83640 1 Processor -- start 2023-10-21T16:18:16.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.589+0000 7f8f64e83640 1 -- start start 2023-10-21T16:18:16.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f64e83640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f6015cba0 0x7f8f600b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f64e83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f64e83640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600b54a0 con 0x7f8f600b4b40 2023-10-21T16:18:16.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f64e83640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600b4fb0 con 0x7f8f6015cba0 2023-10-21T16:18:16.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f64e83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b7130 con 0x7f8f6015cf80 2023-10-21T16:18:16.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f6015cba0 0x7f8f600b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f6015cba0 0x7f8f600b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47900/0 (socket says 172.21.15.53:47900) 2023-10-21T16:18:16.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 -- 172.21.15.53:0/2032969370 learned_addr learned my addr 172.21.15.53:0/2032969370 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:16.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f57fff640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 -- 172.21.15.53:0/2032969370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 msgr2=0x7f8f600b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:16.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.591+0000 7f8f5e59c640 1 -- 172.21.15.53:0/2032969370 >> v1:172.21.15.124:6789/0 conn(0x7f8f600b4b40 legacy=0x7f8f600b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:16.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.592+0000 7f8f5e59c640 1 -- 172.21.15.53:0/2032969370 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600b7410 con 0x7f8f6015cba0 2023-10-21T16:18:16.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.593+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f600b84a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f491a82cbb2d7103 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:16.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.593+0000 7f8f5dd9b640 1 -- 172.21.15.53:0/2032969370 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50031640 con 0x7f8f6015cba0 2023-10-21T16:18:16.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.593+0000 7f8f5dd9b640 1 -- 172.21.15.53:0/2032969370 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f500317a0 con 0x7f8f6015cba0 2023-10-21T16:18:16.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.593+0000 7f8f5dd9b640 1 -- 172.21.15.53:0/2032969370 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50031b30 con 0x7f8f6015cba0 2023-10-21T16:18:16.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 -- 172.21.15.53:0/2032969370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 msgr2=0x7f8f600b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f600b84a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 -- 172.21.15.53:0/2032969370 shutdown_connections 2023-10-21T16:18:16.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 --2- 172.21.15.53:0/2032969370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f600b84a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 -- 172.21.15.53:0/2032969370 >> 172.21.15.53:0/2032969370 conn(0x7f8f600718a0 msgr2=0x7f8f60071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:16.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 -- 172.21.15.53:0/2032969370 shutdown_connections 2023-10-21T16:18:16.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.594+0000 7f8f64e83640 1 -- 172.21.15.53:0/2032969370 wait complete. 2023-10-21T16:18:16.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.595+0000 7f8f64e83640 1 Processor -- start 2023-10-21T16:18:16.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.596+0000 7f8f64e83640 1 -- start start 2023-10-21T16:18:16.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f600b4b40 0x7f8f60154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f601e2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600c9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601e5020 con 0x7f8f6015cba0 2023-10-21T16:18:16.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601e2cd0 con 0x7f8f600b4b40 2023-10-21T16:18:16.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f64e83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601e2fb0 con 0x7f8f6015cf80 2023-10-21T16:18:16.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f601e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f601e2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47904/0 (socket says 172.21.15.53:47904) 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 -- 172.21.15.53:0/687781188 learned_addr learned my addr 172.21.15.53:0/687781188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f5ed9d640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f600b4b40 0x7f8f60154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 msgr2=0x7f8f600c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.597+0000 7f8f57fff640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f600b4b40 msgr2=0x7f8f60154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:16.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f57fff640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f600b4b40 0x7f8f60154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:16.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f57fff640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500094d0 con 0x7f8f6015cba0 2023-10-21T16:18:16.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f5ed9d640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f6015cf80 0x7f8f600c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:16.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f57fff640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f6015cba0 0x7f8f601e2640 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800d5f0 tx=0x7f8f4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f480125f0 con 0x7f8f6015cba0 2023-10-21T16:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f48012750 con 0x7f8f6015cba0 2023-10-21T16:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48012b00 con 0x7f8f6015cba0 2023-10-21T16:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f64e83640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f601e3290 con 0x7f8f6015cba0 2023-10-21T16:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.598+0000 7f8f64e83640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601e3850 con 0x7f8f6015cba0 2023-10-21T16:18:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.599+0000 7f8f64e83640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6004e160 con 0x7f8f6015cba0 2023-10-21T16:18:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.601+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8f48012c60 con 0x7f8f6015cba0 2023-10-21T16:18:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.601+0000 7f8f56ffd640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.602+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f8f48089f00 con 0x7f8f6015cba0 2023-10-21T16:18:16.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.602+0000 7f8f5e59c640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 msgr2=0x7f8f4006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:16.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.602+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:16.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.606+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f48054c60 con 0x7f8f6015cba0 2023-10-21T16:18:16.630 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.629+0000 7fd10202d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:16.631 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.629+0000 7fd64d445040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:16.680 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.678+0000 7fd64d445040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:16.695 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.693+0000 7fd10202d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:16.746 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.745+0000 7fd10202d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:16.790 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.788+0000 7fd64d445040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:16.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.803+0000 7f8f5e59c640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 msgr2=0x7f8f4006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:16.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.803+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:16.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:16.817+0000 7f8f64e83640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f600b79d0 con 0x7f8f4006afe0 2023-10-21T16:18:16.853 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.851+0000 7fd64d445040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:16.865 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.863+0000 7fd10202d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:16.901 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.899+0000 7fd64d445040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:16.931 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.929+0000 7fd10202d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:16.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:16.956+0000 7fd64d445040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:16.983 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:16.981+0000 7fd10202d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:17.009 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.006+0000 7fd64d445040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:17.042 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.040+0000 7fd10202d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:17.094 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.093+0000 7fd10202d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:17.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:17.204+0000 7f8f5e59c640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 msgr2=0x7f8f4006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:17.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:17.204+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:18:17.389 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.387+0000 7fd64d445040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:17.495 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.494+0000 7fd10202d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:17.548 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.546+0000 7fd64d445040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:17.657 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.655+0000 7fd64d445040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:17.663 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.662+0000 7fd10202d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:17.707 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.704+0000 7fd64d445040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:17.776 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.775+0000 7fd10202d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:17.814 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.812+0000 7fd64d445040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:17.829 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.827+0000 7fd10202d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:17.920 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:17.918+0000 7fd64d445040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:17.941 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:17.940+0000 7fd10202d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:18.005+0000 7f8f5e59c640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 msgr2=0x7f8f4006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3674677134 2023-10-21T16:18:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:18.005+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:18:18.053 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:18.052+0000 7fd10202d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:18.076 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:18.073+0000 7fd64d445040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:18.124 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:18.122+0000 7fd64d445040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:18.217 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:18.215+0000 7fd10202d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:18.243 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:18.240+0000 7fd64d445040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:18.268 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:18.267+0000 7fd10202d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:18.393 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:18.391+0000 7fd10202d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:19.056 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.051+0000 7fd64d445040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:19.264 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:19.262+0000 7fd10202d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:19.343 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.341+0000 7fd64d445040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:19.400 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.398+0000 7fd64d445040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:19.513 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.511+0000 7fd64d445040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:19.537+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 39) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f8f48055030 con 0x7f8f6015cba0 2023-10-21T16:18:19.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:19.538+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 msgr2=0x7f8f4006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:19.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:19.538+0000 7f8f56ffd640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/3674677134,v1:172.21.15.124:6833/3674677134] conn(0x7f8f4006afe0 0x7f8f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:19.584 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:19.582+0000 7fd10202d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:19.654 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:19.652+0000 7fd10202d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:19.721 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5cc793640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.722 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5cc793640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.722 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5cc793640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.722 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5cc793640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.723 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5cc793640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.723 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5c9f4e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.723 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5c9f4e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.723 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5c9f4e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.724 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5c9f4e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.724 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:19.720+0000 7fd5c9f4e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:19.781 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:19.780+0000 7fd10202d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:20.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.546+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 40) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8f48070440 con 0x7f8f6015cba0 2023-10-21T16:18:20.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.547+0000 7f8f56ffd640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f8f40035880 0x7f8f40075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.547+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 --> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f40035c60 con 0x7f8f40035880 2023-10-21T16:18:20.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.553+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f8f40035880 0x7f8f40075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.561+0000 7f8f5e59c640 1 --2- 172.21.15.53:0/687781188 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f8f40035880 0x7f8f40075500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f50036000 tx=0x7f8f50008030 comp rx=0 tx=0).ready entity=mgr.6343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:20.589 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:20.587+0000 7fd5d3fa2640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-21T16:18:20.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.588+0000 7f8f56ffd640 1 -- 172.21.15.53:0/687781188 <== mgr.6343 v2:172.21.15.124:6832/1331540020 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f8f40035c60 con 0x7f8f40035880 2023-10-21T16:18:20.590 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-21T16:18:20.608 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:18:20.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:18:20.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.839+0000 7fd4d88d1640 1 Processor -- start 2023-10-21T16:18:20.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.840+0000 7fd4d88d1640 1 -- start start 2023-10-21T16:18:20.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.841+0000 7fd4d88d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d415a820 0x7fd4d40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.841+0000 7fd4d88d1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4d40ca680 0x7fd4d40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.841+0000 7fd4d88d1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40cab60 con 0x7fd4d415ac00 2023-10-21T16:18:20.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.841+0000 7fd4d88d1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40cb960 con 0x7fd4d40ca680 2023-10-21T16:18:20.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.841+0000 7fd4d88d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40cbdf0 con 0x7fd4d415a820 2023-10-21T16:18:20.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4d40ca680 0x7fd4d40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4d3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd4d415ac00 0x7fd4d40c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42844/0 (socket says 172.21.15.53:42844) 2023-10-21T16:18:20.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4d40ca680 0x7fd4d40b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47914/0 (socket says 172.21.15.53:47914) 2023-10-21T16:18:20.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4d3fff640 1 -- 172.21.15.53:0/2867091911 learned_addr learned my addr 172.21.15.53:0/2867091911 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:20.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4d37fe640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d415a820 0x7fd4d40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2867091911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d415a820 msgr2=0x7fd4d40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d415a820 0x7fd4d40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2867091911 >> v1:172.21.15.53:6789/0 conn(0x7fd4d415ac00 legacy=0x7fd4d40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2867091911 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40cb030 con 0x7fd4d40ca680 2023-10-21T16:18:20.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.842+0000 7fd4d2ffd640 1 -- 172.21.15.53:0/2867091911 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158259607 0 0) 0x7fd4d40cab60 con 0x7fd4d415ac00 2023-10-21T16:18:20.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4cbfff640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40ca680 0x7fd4d40b04b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097a0 tx=0x7fd4bc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8101b6d7ab1170aa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:20.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4d2ffd640 1 -- 172.21.15.53:0/2867091911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc031780 con 0x7fd4d40ca680 2023-10-21T16:18:20.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4d2ffd640 1 -- 172.21.15.53:0/2867091911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0318e0 con 0x7fd4d40ca680 2023-10-21T16:18:20.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4d2ffd640 1 -- 172.21.15.53:0/2867091911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc031c70 con 0x7fd4d40ca680 2023-10-21T16:18:20.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2867091911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40ca680 msgr2=0x7fd4d40b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.844+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40ca680 0x7fd4d40b04b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097a0 tx=0x7fd4bc02b820 comp rx=0 tx=0).stop 2023-10-21T16:18:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2867091911 shutdown_connections 2023-10-21T16:18:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d40ca680 0x7fd4d40b04b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2867091911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d415a820 0x7fd4d40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:20.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2867091911 >> 172.21.15.53:0/2867091911 conn(0x7fd4d406c050 msgr2=0x7fd4d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2867091911 shutdown_connections 2023-10-21T16:18:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.845+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2867091911 wait complete. 2023-10-21T16:18:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.846+0000 7fd4d88d1640 1 Processor -- start 2023-10-21T16:18:20.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.847+0000 7fd4d88d1640 1 -- start start 2023-10-21T16:18:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.847+0000 7fd4d88d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 0x7fd4d41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d88d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d88d1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d415ac00 0x7fd4d4157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d88d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415ded0 con 0x7fd4d415a820 2023-10-21T16:18:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d88d1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415d350 con 0x7fd4d415ac00 2023-10-21T16:18:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d88d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415d630 con 0x7fd4d40ca680 2023-10-21T16:18:20.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47926/0 (socket says 172.21.15.53:47926) 2023-10-21T16:18:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2424693621 learned_addr learned my addr 172.21.15.53:0/2424693621 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d37fe640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 0x7fd4d41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d3fff640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d415ac00 0x7fd4d4157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 msgr2=0x7fd4d41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 0x7fd4d41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d415ac00 msgr2=0x7fd4d4157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d415ac00 0x7fd4d4157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4cbfff640 1 -- 172.21.15.53:0/2424693621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009400 con 0x7fd4d415a820 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.848+0000 7fd4d37fe640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 0x7fd4d41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:20.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4cbfff640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0328e0 con 0x7fd4d415a820 2023-10-21T16:18:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc032a40 con 0x7fd4d415a820 2023-10-21T16:18:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc032df0 con 0x7fd4d415a820 2023-10-21T16:18:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d415d910 con 0x7fd4d415a820 2023-10-21T16:18:20.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.849+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4152620 con 0x7fd4d415a820 2023-10-21T16:18:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.850+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4074150 con 0x7fd4d415a820 2023-10-21T16:18:20.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.851+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 40) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd4bc04e450 con 0x7fd4d415a820 2023-10-21T16:18:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.852+0000 7fd4d0ff9640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 0x7fd4b406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.852+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4bc0ab740 con 0x7fd4d415a820 2023-10-21T16:18:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.853+0000 7fd4d37fe640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 0x7fd4b406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:20.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.854+0000 7fd4d37fe640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 0x7fd4b406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4006fd0 tx=0x7fd4c4008040 comp rx=0 tx=0).ready entity=mgr.6343 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:20.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:20.857+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4bc076770 con 0x7fd4d415a820 2023-10-21T16:18:21.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:18:21.096 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:18:21.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.089+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd4d40b1310 con 0x7fd4d415a820 2023-10-21T16:18:21.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.596+0000 7fd4d0ff9640 1 -- 172.21.15.53:0/2424693621 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v41) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd4bc079d90 con 0x7fd4d415a820 2023-10-21T16:18:21.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.599+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 msgr2=0x7fd4b406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:21.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.599+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 0x7fd4b406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4006fd0 tx=0x7fd4c4008040 comp rx=0 tx=0).stop 2023-10-21T16:18:21.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.599+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 msgr2=0x7fd4d41570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:21.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.599+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:21.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 shutdown_connections 2023-10-21T16:18:21.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fd4b406ad10 0x7fd4b406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:21.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4d415ac00 0x7fd4d4157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:21.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4d415a820 0x7fd4d41570e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:21.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 --2- 172.21.15.53:0/2424693621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4d40ca680 0x7fd4d41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:21.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.600+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 >> 172.21.15.53:0/2424693621 conn(0x7fd4d406c050 msgr2=0x7fd4d4159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:21.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.601+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 shutdown_connections 2023-10-21T16:18:21.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:21.601+0000 7fd4d88d1640 1 -- 172.21.15.53:0/2424693621 wait complete. 2023-10-21T16:18:21.630 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:18:21.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:18:21.754 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:18:21.856 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:18:21.856 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:21.857 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:21.909 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:21.910 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:22.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:22.134 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.132+0000 7f56e2c41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:22.164 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.163+0000 7f2bd9a13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:22.188 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.186+0000 7f56e2c41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:22.225 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.223+0000 7f2bd9a13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.224+0000 7fe8c3d33640 1 Processor -- start 2023-10-21T16:18:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.224+0000 7fe8c3d33640 1 -- start start 2023-10-21T16:18:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.226+0000 7fe8c3d33640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe8bc0ad790 0x7fe8bc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.226+0000 7fe8c3d33640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8bc15e330 0x7fe8bc070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.226+0000 7fe8c3d33640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc06dee0 con 0x7fe8bc159ac0 2023-10-21T16:18:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.226+0000 7fe8c3d33640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc06dcf0 con 0x7fe8bc15e330 2023-10-21T16:18:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.226+0000 7fe8c3d33640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc15d850 con 0x7fe8bc0ad790 2023-10-21T16:18:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8bc15e330 0x7fe8bc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe8bc15e330 0x7fe8bc070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47954/0 (socket says 172.21.15.53:47954) 2023-10-21T16:18:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 -- 172.21.15.53:0/4218982188 learned_addr learned my addr 172.21.15.53:0/4218982188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c12ce640 1 --2- 172.21.15.53:0/4218982188 >> v2:172.21.15.124:3300/0 conn(0x7fe8bc0ad790 0x7fe8bc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 -- 172.21.15.53:0/4218982188 >> v2:172.21.15.124:3300/0 conn(0x7fe8bc0ad790 msgr2=0x7fe8bc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 --2- 172.21.15.53:0/4218982188 >> v2:172.21.15.124:3300/0 conn(0x7fe8bc0ad790 0x7fe8bc0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 -- 172.21.15.53:0/4218982188 >> v1:172.21.15.124:6789/0 conn(0x7fe8bc159ac0 legacy=0x7fe8bc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.227+0000 7fe8c22d0640 1 -- 172.21.15.53:0/4218982188 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc15db30 con 0x7fe8bc15e330 2023-10-21T16:18:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.228+0000 7fe8c22d0640 1 --2- 172.21.15.53:0/4218982188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc070810 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b4970 tx=0x7fe8b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ededab19654bf2f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c0acd640 1 -- 172.21.15.53:0/4218982188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b402ca80 con 0x7fe8bc15e330 2023-10-21T16:18:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c0acd640 1 -- 172.21.15.53:0/4218982188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b402cbe0 con 0x7fe8bc15e330 2023-10-21T16:18:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c0acd640 1 -- 172.21.15.53:0/4218982188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40343f0 con 0x7fe8bc15e330 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 -- 172.21.15.53:0/4218982188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 msgr2=0x7fe8bc070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/4218982188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc070810 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b4970 tx=0x7fe8b402bee0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 -- 172.21.15.53:0/4218982188 shutdown_connections 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/4218982188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc070810 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/4218982188 >> v2:172.21.15.124:3300/0 conn(0x7fe8bc0ad790 0x7fe8bc0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.229+0000 7fe8c3d33640 1 -- 172.21.15.53:0/4218982188 >> 172.21.15.53:0/4218982188 conn(0x7fe8bc06a380 msgr2=0x7fe8bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.230+0000 7fe8c3d33640 1 -- 172.21.15.53:0/4218982188 shutdown_connections 2023-10-21T16:18:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.230+0000 7fe8c3d33640 1 -- 172.21.15.53:0/4218982188 wait complete. 2023-10-21T16:18:22.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.231+0000 7fe8c3d33640 1 Processor -- start 2023-10-21T16:18:22.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.231+0000 7fe8c3d33640 1 -- start start 2023-10-21T16:18:22.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 0x7fe8bc158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8bc159ac0 0x7fe8bc158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc152960 con 0x7fe8bc15e330 2023-10-21T16:18:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc151ef0 con 0x7fe8bc0ad790 2023-10-21T16:18:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.232+0000 7fe8c3d33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc158cb0 con 0x7fe8bc159ac0 2023-10-21T16:18:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47960/0 (socket says 172.21.15.53:47960) 2023-10-21T16:18:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 -- 172.21.15.53:0/563797129 learned_addr learned my addr 172.21.15.53:0/563797129 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c1acf640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 0x7fe8bc158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c12ce640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8bc159ac0 0x7fe8bc158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8bc159ac0 msgr2=0x7fe8bc158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8bc159ac0 0x7fe8bc158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 msgr2=0x7fe8bc158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 0x7fe8bc158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c22d0640 1 -- 172.21.15.53:0/563797129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4009b30 con 0x7fe8bc15e330 2023-10-21T16:18:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.233+0000 7fe8c1acf640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 0x7fe8bc158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8c22d0640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4036040 tx=0x7fe8b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4009510 con 0x7fe8bc15e330 2023-10-21T16:18:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b403e450 con 0x7fe8bc15e330 2023-10-21T16:18:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b403e7e0 con 0x7fe8bc15e330 2023-10-21T16:18:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc154670 con 0x7fe8bc15e330 2023-10-21T16:18:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.234+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0b2a20 con 0x7fe8bc15e330 2023-10-21T16:18:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.236+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc0b2880 con 0x7fe8bc15e330 2023-10-21T16:18:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.237+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe8b402cd50 con 0x7fe8bc15e330 2023-10-21T16:18:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.238+0000 7fe8b27fc640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 0x7fe8a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.238+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8b40ac5f0 con 0x7fe8bc15e330 2023-10-21T16:18:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.238+0000 7fe8c1acf640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 msgr2=0x7fe8a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.238+0000 7fe8c1acf640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 0x7fe8a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.242+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8bc0b2880 con 0x7fe8bc15e330 2023-10-21T16:18:22.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.438+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8bc0c8db0 con 0x7fe8bc15e330 2023-10-21T16:18:22.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.439+0000 7fe8c1acf640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 msgr2=0x7fe8a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:22.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.439+0000 7fe8c1acf640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 0x7fe8a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:22.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.441+0000 7fe8b27fc640 1 -- 172.21.15.53:0/563797129 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+17416 (secure 0 0 0) 0x7fe8bc0c8db0 con 0x7fe8bc15e330 2023-10-21T16:18:22.443 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:22.443 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":105,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:19.516665+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/824950480":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6833/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/2324608475":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3280078422":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6832/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3826369357":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3225172013":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/798725635":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3415687515":"2023-10-21T17:18:12.553813+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 msgr2=0x7fe8a006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 0x7fe8a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 msgr2=0x7fe8bc1519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4036040 tx=0x7fe8b4004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:22.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 shutdown_connections 2023-10-21T16:18:22.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fe8a006af90 0x7fe8a006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8bc15e330 0x7fe8bc1519b0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8bc159ac0 0x7fe8bc158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 --2- 172.21.15.53:0/563797129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8bc0ad790 0x7fe8bc158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 >> 172.21.15.53:0/563797129 conn(0x7fe8bc06a380 msgr2=0x7fe8bc06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:22.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.443+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 shutdown_connections 2023-10-21T16:18:22.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.444+0000 7fe8c3d33640 1 -- 172.21.15.53:0/563797129 wait complete. 2023-10-21T16:18:22.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:18:02.821177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:18:03.863098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:18:22.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:18:22.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.630+0000 7f56e2c41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:22.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.673+0000 7fc770aa2640 1 Processor -- start 2023-10-21T16:18:22.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.673+0000 7fc770aa2640 1 -- start start 2023-10-21T16:18:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.675+0000 7fc770aa2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc76c15cc50 0x7fc76c0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.675+0000 7fc770aa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0b6b50 0x7fc76c0cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.675+0000 7fc770aa2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c0d0400 con 0x7fc76c0cfa80 2023-10-21T16:18:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.675+0000 7fc770aa2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0b71c0 con 0x7fc76c15cc50 2023-10-21T16:18:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.675+0000 7fc770aa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0b74a0 con 0x7fc76c0b6b50 2023-10-21T16:18:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc76c15cc50 0x7fc76c0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0b6b50 0x7fc76c0cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc76c15cc50 0x7fc76c0b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:48078/0 (socket says 172.21.15.53:48078) 2023-10-21T16:18:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 -- 172.21.15.53:0/1016200233 learned_addr learned my addr 172.21.15.53:0/1016200233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 -- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0b6b50 msgr2=0x7fc76c0cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 --2- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0b6b50 0x7fc76c0cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 -- 172.21.15.53:0/1016200233 >> v1:172.21.15.124:6789/0 conn(0x7fc76c0cfa80 legacy=0x7fc76c0d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.676+0000 7fc76a59c640 1 -- 172.21.15.53:0/1016200233 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c0b7780 con 0x7fc76c15cc50 2023-10-21T16:18:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.677+0000 7fc76a59c640 1 --2- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c15cc50 0x7fc76c0b8480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813dc35576d145ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.678+0000 7fc769d9b640 1 -- 172.21.15.53:0/1016200233 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c02f830 con 0x7fc76c15cc50 2023-10-21T16:18:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.678+0000 7fc769d9b640 1 -- 172.21.15.53:0/1016200233 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c02f990 con 0x7fc76c15cc50 2023-10-21T16:18:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.678+0000 7fc769d9b640 1 -- 172.21.15.53:0/1016200233 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c02fd20 con 0x7fc76c15cc50 2023-10-21T16:18:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 -- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c15cc50 msgr2=0x7fc76c0b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 --2- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c15cc50 0x7fc76c0b8480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02ba10 comp rx=0 tx=0).stop 2023-10-21T16:18:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 -- 172.21.15.53:0/1016200233 shutdown_connections 2023-10-21T16:18:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 --2- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0b6b50 0x7fc76c0cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 --2- 172.21.15.53:0/1016200233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c15cc50 0x7fc76c0b8480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 -- 172.21.15.53:0/1016200233 >> 172.21.15.53:0/1016200233 conn(0x7fc76c06b5e0 msgr2=0x7fc76c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 -- 172.21.15.53:0/1016200233 shutdown_connections 2023-10-21T16:18:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.679+0000 7fc770aa2640 1 -- 172.21.15.53:0/1016200233 wait complete. 2023-10-21T16:18:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.682+0000 7fc770aa2640 1 Processor -- start 2023-10-21T16:18:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.682+0000 7fc770aa2640 1 -- start start 2023-10-21T16:18:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 0x7fc76c15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0cfa80 0x7fc76c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c160170 con 0x7fc76c15cc50 2023-10-21T16:18:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c15f230 con 0x7fc76c0b6b50 2023-10-21T16:18:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.683+0000 7fc770aa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c15f960 con 0x7fc76c0cfa80 2023-10-21T16:18:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47972/0 (socket says 172.21.15.53:47972) 2023-10-21T16:18:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 -- 172.21.15.53:0/3538545492 learned_addr learned my addr 172.21.15.53:0/3538545492 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76a59c640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 0x7fc76c15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc763fff640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0cfa80 0x7fc76c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0cfa80 msgr2=0x7fc76c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0cfa80 0x7fc76c0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 msgr2=0x7fc76c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 0x7fc76c15f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76ad9d640 1 -- 172.21.15.53:0/3538545492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0094d0 con 0x7fc76c15cc50 2023-10-21T16:18:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.684+0000 7fc76a59c640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 0x7fc76c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:22.689 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.683+0000 7f56e2c41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc76ad9d640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d30 tx=0x7fc76400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764009ba0 con 0x7fc76c15cc50 2023-10-21T16:18:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400f040 con 0x7fc76c15cc50 2023-10-21T16:18:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640146c0 con 0x7fc76c15cc50 2023-10-21T16:18:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c15fc00 con 0x7fc76c15cc50 2023-10-21T16:18:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.685+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c0cb7c0 con 0x7fc76c15cc50 2023-10-21T16:18:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.686+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c04e160 con 0x7fc76c15cc50 2023-10-21T16:18:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.687+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fc764014820 con 0x7fc76c15cc50 2023-10-21T16:18:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.689+0000 7fc762ffd640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 0x7fc74c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:22.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.689+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc76408a930 con 0x7fc76c15cc50 2023-10-21T16:18:22.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.689+0000 7fc76a59c640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 msgr2=0x7fc74c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:22.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.689+0000 7fc76a59c640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 0x7fc74c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:22.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.693+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7640554c0 con 0x7fc76c15cc50 2023-10-21T16:18:22.713 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.712+0000 7f2bd9a13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:22.735 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.733+0000 7f56e2c41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:22.770 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.766+0000 7f2bd9a13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:22.794 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.792+0000 7f56e2c41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:22.820 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.819+0000 7f2bd9a13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:22.842 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.840+0000 7f56e2c41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:22.884 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.883+0000 7f2bd9a13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:22.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.888+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc76c0cdc00 con 0x7fc76c15cc50 2023-10-21T16:18:22.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.888+0000 7fc762ffd640 1 -- 172.21.15.53:0/3538545492 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7640180a0 con 0x7fc76c15cc50 2023-10-21T16:18:22.890 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:18:22.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.890+0000 7fc76a59c640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 msgr2=0x7fc74c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:22.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.890+0000 7fc76a59c640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 0x7fc74c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:22.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 msgr2=0x7fc74c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 0x7fc74c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 msgr2=0x7fc76c0c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d30 tx=0x7fc76400c490 comp rx=0 tx=0).stop 2023-10-21T16:18:22.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 shutdown_connections 2023-10-21T16:18:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fc74c06afc0 0x7fc74c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc76c15cc50 0x7fc76c0c8730 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc76c0cfa80 0x7fc76c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 --2- 172.21.15.53:0/3538545492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc76c0b6b50 0x7fc76c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:22.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 >> 172.21.15.53:0/3538545492 conn(0x7fc76c06b5e0 msgr2=0x7fc76c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 shutdown_connections 2023-10-21T16:18:22.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:22.891+0000 7fc770aa2640 1 -- 172.21.15.53:0/3538545492 wait complete. 2023-10-21T16:18:22.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:18:22.937 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:22.936+0000 7f2bd9a13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:22.952 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:22.950+0000 7f56e2c41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:23.014 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.012+0000 7f56e2c41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:23.055 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.054+0000 7f2bd9a13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:23.062 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.060+0000 7f56e2c41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:23.119 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.117+0000 7f56e2c41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.118+0000 7fb7c647c640 1 Processor -- start 2023-10-21T16:18:23.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.118+0000 7fb7c647c640 1 -- start start 2023-10-21T16:18:23.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.120+0000 7fb7c647c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c00b4d50 0x7fb7c00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.120+0000 7fb7c647c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c0071ef0 0x7fb7c00ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.120+0000 7fb7c647c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 0x7fb7c00b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.120+0000 7fb7c647c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c00b5130 con 0x7fb7c0071ef0 2023-10-21T16:18:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7c647c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c00b1ca0 con 0x7fb7c00b12d0 2023-10-21T16:18:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7c647c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00ce5c0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c0071ef0 0x7fb7c00ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7c0071ef0 0x7fb7c00ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47988/0 (socket says 172.21.15.53:47988) 2023-10-21T16:18:23.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/2936367663 learned_addr learned my addr 172.21.15.53:0/2936367663 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bffff640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c00b4d50 0x7fb7c00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7c4a19640 1 --2- 172.21.15.53:0/2936367663 >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 0x7fb7c00b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/2936367663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c00b4d50 msgr2=0x7fb7c00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c00b4d50 0x7fb7c00cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/2936367663 >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 msgr2=0x7fb7c00b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/2936367663 >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 0x7fb7c00b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7bf7fe640 1 -- 172.21.15.53:0/2936367663 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00cefe0 con 0x7fb7c0071ef0 2023-10-21T16:18:23.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.121+0000 7fb7c4a19640 1 --2- 172.21.15.53:0/2936367663 >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 0x7fb7c00b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:23.126 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.122+0000 7f2bd9a13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:23.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.122+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c0071ef0 0x7fb7c00ce080 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00b2120 tx=0x7fb7b402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae0aaf6fc71a3df server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.123+0000 7fb7beffd640 1 -- 172.21.15.53:0/2936367663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40317d0 con 0x7fb7c0071ef0 2023-10-21T16:18:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.123+0000 7fb7beffd640 1 -- 172.21.15.53:0/2936367663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4031930 con 0x7fb7c0071ef0 2023-10-21T16:18:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.123+0000 7fb7beffd640 1 -- 172.21.15.53:0/2936367663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4031cc0 con 0x7fb7c0071ef0 2023-10-21T16:18:23.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.123+0000 7fb7c647c640 1 -- 172.21.15.53:0/2936367663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c0071ef0 msgr2=0x7fb7c00ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.123+0000 7fb7c647c640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c0071ef0 0x7fb7c00ce080 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00b2120 tx=0x7fb7b402b870 comp rx=0 tx=0).stop 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 -- 172.21.15.53:0/2936367663 shutdown_connections 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 --2- 172.21.15.53:0/2936367663 >> v2:172.21.15.124:3300/0 conn(0x7fb7c00b12d0 0x7fb7c00b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c0071ef0 0x7fb7c00ce080 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 --2- 172.21.15.53:0/2936367663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c00b4d50 0x7fb7c00cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 -- 172.21.15.53:0/2936367663 >> 172.21.15.53:0/2936367663 conn(0x7fb7c00cc2d0 msgr2=0x7fb7c00cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 -- 172.21.15.53:0/2936367663 shutdown_connections 2023-10-21T16:18:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.124+0000 7fb7c647c640 1 -- 172.21.15.53:0/2936367663 wait complete. 2023-10-21T16:18:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.125+0000 7fb7c647c640 1 Processor -- start 2023-10-21T16:18:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.126+0000 7fb7c647c640 1 -- start start 2023-10-21T16:18:23.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.126+0000 7fb7c647c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c0071ef0 0x7fb7c015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c647c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c00b12d0 0x7fb7c00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c647c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c647c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00abf10 con 0x7fb7c00b4d50 2023-10-21T16:18:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c647c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00a8580 con 0x7fb7c00b12d0 2023-10-21T16:18:23.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c647c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00a8860 con 0x7fb7c0071ef0 2023-10-21T16:18:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48004/0 (socket says 172.21.15.53:48004) 2023-10-21T16:18:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 -- 172.21.15.53:0/1610853073 learned_addr learned my addr 172.21.15.53:0/1610853073 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7bf7fe640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c00b12d0 0x7fb7c00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7bffff640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c0071ef0 0x7fb7c015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 -- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c0071ef0 msgr2=0x7fb7c015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c0071ef0 0x7fb7c015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 -- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c00b12d0 msgr2=0x7fb7c00cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c00b12d0 0x7fb7c00cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.127+0000 7fb7c4a19640 1 -- 172.21.15.53:0/1610853073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40094d0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7c4a19640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800cef0 tx=0x7fb7b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800d840 con 0x7fb7c00b4d50 2023-10-21T16:18:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b800d9c0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800dd70 con 0x7fb7c00b4d50 2023-10-21T16:18:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c00a8b40 con 0x7fb7c00b4d50 2023-10-21T16:18:23.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.128+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c015b7c0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.129+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c004e160 con 0x7fb7c00b4d50 2023-10-21T16:18:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.130+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb7b801e3b0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.132+0000 7fb7bcff9640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 0x7fb7ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.132+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb7b808a550 con 0x7fb7c00b4d50 2023-10-21T16:18:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.132+0000 7fb7bffff640 1 -- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 msgr2=0x7fb7ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.132+0000 7fb7bffff640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 0x7fb7ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:23.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.136+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7b80552d0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.170 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.168+0000 7f56e2c41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:23.177 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.176+0000 7f2bd9a13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:23.237 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.235+0000 7f2bd9a13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:23.290 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.288+0000 7f2bd9a13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.326+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7c00cdb30 con 0x7fb7c00b4d50 2023-10-21T16:18:23.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.327+0000 7fb7bcff9640 1 -- 172.21.15.53:0/1610853073 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7b80588f0 con 0x7fb7c00b4d50 2023-10-21T16:18:23.329 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:18:23.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 msgr2=0x7fb7ac06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:23.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 0x7fb7ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 msgr2=0x7fb7c00cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800cef0 tx=0x7fb7b800a710 comp rx=0 tx=0).stop 2023-10-21T16:18:23.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 shutdown_connections 2023-10-21T16:18:23.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb7ac06af90 0x7fb7ac06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c00b4d50 0x7fb7c00cb980 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7c00b12d0 0x7fb7c00cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 --2- 172.21.15.53:0/1610853073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7c0071ef0 0x7fb7c015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.330+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 >> 172.21.15.53:0/1610853073 conn(0x7fb7c00cc2d0 msgr2=0x7fb7c00af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:23.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.331+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 shutdown_connections 2023-10-21T16:18:23.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.331+0000 7fb7c647c640 1 -- 172.21.15.53:0/1610853073 wait complete. 2023-10-21T16:18:23.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:18:23.547 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.545+0000 7f56e2c41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.615+0000 7ff398d53640 1 Processor -- start 2023-10-21T16:18:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.615+0000 7ff398d53640 1 -- start start 2023-10-21T16:18:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.616+0000 7ff398d53640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff3941612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff398d53640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3940b8c30 con 0x7ff3940cd4e0 2023-10-21T16:18:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff398d53640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3940cdd80 con 0x7ff39415cd70 2023-10-21T16:18:23.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff398d53640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940ce060 con 0x7ff3940b4e10 2023-10-21T16:18:23.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff392d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3940cd4e0 0x7ff3940d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:42848/0 (socket says 172.21.15.53:42848) 2023-10-21T16:18:23.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff392d9d640 1 -- 172.21.15.53:0/3164480333 learned_addr learned my addr 172.21.15.53:0/3164480333 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff389d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff3941612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490050691 0 0) 0x7ff3940b8c30 con 0x7ff3940cd4e0 2023-10-21T16:18:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.617+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff380003520 con 0x7ff3940cd4e0 2023-10-21T16:18:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160283450 0 0) 0x7ff3940cdd80 con 0x7ff39415cd70 2023-10-21T16:18:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3940b8c30 con 0x7ff39415cd70 2023-10-21T16:18:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 863401977 0 0) 0x7ff380003520 con 0x7ff3940cd4e0 2023-10-21T16:18:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 msgr2=0x7ff3941612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff3941612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 >> v1:172.21.15.124:6789/0 conn(0x7ff39415cd70 legacy=0x7ff3940b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3940ce340 con 0x7ff3940cd4e0 2023-10-21T16:18:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff38c001670 con 0x7ff3940cd4e0 2023-10-21T16:18:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 >> v1:172.21.15.53:6789/0 conn(0x7ff3940cd4e0 legacy=0x7ff3940d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff380003b30 0x7ff380003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3800047c0 0x7ff380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940ce060 con 0x7ff3800047c0 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38000ba60 con 0x7ff380003b30 2023-10-21T16:18:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38000bc20 con 0x7ff3940b4e10 2023-10-21T16:18:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff389d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3800047c0 0x7ff380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.618+0000 7ff39259c640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff380003b30 0x7ff380003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff392d9d640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff389d9b640 1 -- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 msgr2=0x7ff380005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff389d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff380005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff389d9b640 1 -- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff380003b30 msgr2=0x7ff380003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff389d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff380003b30 0x7ff380003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff389d9b640 1 -- 172.21.15.53:0/3164480333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380003520 con 0x7ff3800047c0 2023-10-21T16:18:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.619+0000 7ff392d9d640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3940b4e10 0x7ff380005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.620+0000 7ff389d9b640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3800047c0 0x7ff380004c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009ac0 tx=0x7ff3800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b4e900858a67c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:23.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.620+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c02c6b0 con 0x7ff3800047c0 2023-10-21T16:18:23.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.620+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c02c810 con 0x7ff3800047c0 2023-10-21T16:18:23.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.620+0000 7ff391d9b640 1 -- 172.21.15.53:0/3164480333 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c02cba0 con 0x7ff3800047c0 2023-10-21T16:18:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 -- 172.21.15.53:0/3164480333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3800047c0 msgr2=0x7ff380004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 --2- 172.21.15.53:0/3164480333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3800047c0 0x7ff380004c40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009ac0 tx=0x7ff3800057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff392d9d640 1 -- 172.21.15.53:0/3164480333 reap_dead start 2023-10-21T16:18:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 -- 172.21.15.53:0/3164480333 shutdown_connections 2023-10-21T16:18:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 -- 172.21.15.53:0/3164480333 >> 172.21.15.53:0/3164480333 conn(0x7ff3940cf700 msgr2=0x7ff3940cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 -- 172.21.15.53:0/3164480333 shutdown_connections 2023-10-21T16:18:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.621+0000 7ff398d53640 1 -- 172.21.15.53:0/3164480333 wait complete. 2023-10-21T16:18:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.623+0000 7ff398d53640 1 Processor -- start 2023-10-21T16:18:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.623+0000 7ff398d53640 1 -- start start 2023-10-21T16:18:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 0x7ff39415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff39415e430 0x7ff3940c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940c93a0 con 0x7ff3940c8f20 2023-10-21T16:18:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39415fd80 con 0x7ff3800047c0 2023-10-21T16:18:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff398d53640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940a96a0 con 0x7ff39415e430 2023-10-21T16:18:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff392d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff392d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48026/0 (socket says 172.21.15.53:48026) 2023-10-21T16:18:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff392d9d640 1 -- 172.21.15.53:0/1137906363 learned_addr learned my addr 172.21.15.53:0/1137906363 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff389d9b640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff39415e430 0x7ff3940c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.624+0000 7ff39259c640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 0x7ff39415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 -- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff39415e430 msgr2=0x7ff3940c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff39415e430 0x7ff3940c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 -- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 msgr2=0x7ff39415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 0x7ff39415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 -- 172.21.15.53:0/1137906363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39415f7c0 con 0x7ff3940c8f20 2023-10-21T16:18:23.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff39259c640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 0x7ff39415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:23.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff392d9d640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b8440 tx=0x7ff38c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:23.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0074e0 con 0x7ff3940c8f20 2023-10-21T16:18:23.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c007660 con 0x7ff3940c8f20 2023-10-21T16:18:23.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c004a20 con 0x7ff3940c8f20 2023-10-21T16:18:23.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39415faa0 con 0x7ff3940c8f20 2023-10-21T16:18:23.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.625+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3940b2d90 con 0x7ff3940c8f20 2023-10-21T16:18:23.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.627+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3940b2f50 con 0x7ff3940c8f20 2023-10-21T16:18:23.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.629+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7ff38c004b80 con 0x7ff3940c8f20 2023-10-21T16:18:23.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.629+0000 7ff38b7fe640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 0x7ff37406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:23.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.630+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff38c089850 con 0x7ff3940c8f20 2023-10-21T16:18:23.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.630+0000 7ff39259c640 1 -- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 msgr2=0x7ff37406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:23.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.630+0000 7ff39259c640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 0x7ff37406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:23.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.634+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3940b2f50 con 0x7ff3940c8f20 2023-10-21T16:18:23.702 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.701+0000 7f2bd9a13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:23.709 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.707+0000 7f56e2c41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:23.816 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.814+0000 7f56e2c41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:23.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.825+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3940b2f50 con 0x7ff3940c8f20 2023-10-21T16:18:23.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.825+0000 7ff38b7fe640 1 -- 172.21.15.53:0/1137906363 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3940b2f50 con 0x7ff3940c8f20 2023-10-21T16:18:23.827 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:18:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 msgr2=0x7ff37406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 0x7ff37406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 msgr2=0x7ff3940c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:23.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff3940b8440 tx=0x7ff38c00d9d0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 shutdown_connections 2023-10-21T16:18:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7ff37406afc0 0x7ff37406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3940c8f20 0x7ff3940c6de0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff39415e430 0x7ff3940c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 --2- 172.21.15.53:0/1137906363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3800047c0 0x7ff39415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:23.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 >> 172.21.15.53:0/1137906363 conn(0x7ff3940cf700 msgr2=0x7ff3940b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:23.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 shutdown_connections 2023-10-21T16:18:23.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:23.828+0000 7ff398d53640 1 -- 172.21.15.53:0/1137906363 wait complete. 2023-10-21T16:18:23.847 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:18:23.865 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.863+0000 7f56e2c41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:23.871 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.870+0000 7f2bd9a13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:23.972 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:23.970+0000 7f56e2c41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:23.986 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:23.984+0000 7f2bd9a13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:24.039 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.037+0000 7f2bd9a13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:24.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.054+0000 7f9ba4c75640 1 Processor -- start 2023-10-21T16:18:24.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.055+0000 7f9ba4c75640 1 -- start start 2023-10-21T16:18:24.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.056+0000 7f9ba4c75640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ba00cf690 0x7f9ba00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.056+0000 7f9ba4c75640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba00ce510 0x7f9ba015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.056+0000 7f9ba4c75640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba00cd520 con 0x7f9ba00cf2b0 2023-10-21T16:18:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.056+0000 7f9ba4c75640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00d1910 con 0x7f9ba00ce510 2023-10-21T16:18:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.056+0000 7f9ba4c75640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba00ce080 con 0x7f9ba00cf690 2023-10-21T16:18:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba00ce510 0x7f9ba015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba00ce510 0x7f9ba015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50952/0 (socket says 172.21.15.53:50952) 2023-10-21T16:18:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 -- 172.21.15.53:0/1435175850 learned_addr learned my addr 172.21.15.53:0/1435175850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:24.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9effd640 1 --2- 172.21.15.53:0/1435175850 >> v2:172.21.15.124:3300/0 conn(0x7f9ba00cf690 0x7f9ba00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 -- 172.21.15.53:0/1435175850 >> v2:172.21.15.124:3300/0 conn(0x7f9ba00cf690 msgr2=0x7f9ba00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 --2- 172.21.15.53:0/1435175850 >> v2:172.21.15.124:3300/0 conn(0x7f9ba00cf690 0x7f9ba00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 -- 172.21.15.53:0/1435175850 >> v1:172.21.15.124:6789/0 conn(0x7f9ba00cf2b0 legacy=0x7f9ba00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.057+0000 7f9b9ffff640 1 -- 172.21.15.53:0/1435175850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba00cd750 con 0x7f9ba00ce510 2023-10-21T16:18:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.058+0000 7f9b9ffff640 1 --2- 172.21.15.53:0/1435175850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba015c820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9ba008baa0 tx=0x7f9b9802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1533ee41b67b1d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/1435175850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9802ca80 con 0x7f9ba00ce510 2023-10-21T16:18:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/1435175850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9802cbe0 con 0x7f9ba00ce510 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/1435175850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980343f0 con 0x7f9ba00ce510 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 -- 172.21.15.53:0/1435175850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 msgr2=0x7f9ba015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/1435175850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba015c820 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9ba008baa0 tx=0x7f9b9802bee0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 -- 172.21.15.53:0/1435175850 shutdown_connections 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/1435175850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba015c820 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/1435175850 >> v2:172.21.15.124:3300/0 conn(0x7f9ba00cf690 0x7f9ba00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 -- 172.21.15.53:0/1435175850 >> 172.21.15.53:0/1435175850 conn(0x7f9ba009bc40 msgr2=0x7f9ba00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:24.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 -- 172.21.15.53:0/1435175850 shutdown_connections 2023-10-21T16:18:24.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.059+0000 7f9ba4c75640 1 -- 172.21.15.53:0/1435175850 wait complete. 2023-10-21T16:18:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.061+0000 7f9ba4c75640 1 Processor -- start 2023-10-21T16:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.061+0000 7f9ba4c75640 1 -- start start 2023-10-21T16:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 0x7f9ba009d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba00cf690 0x7f9ba015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00c9ee0 con 0x7f9ba00ce510 2023-10-21T16:18:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00ca1c0 con 0x7f9ba00cf690 2023-10-21T16:18:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9ba4c75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba00ca4a0 con 0x7f9ba00cf2b0 2023-10-21T16:18:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50968/0 (socket says 172.21.15.53:50968) 2023-10-21T16:18:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.062+0000 7f9b9f7fe640 1 -- 172.21.15.53:0/26089236 learned_addr learned my addr 172.21.15.53:0/26089236 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9effd640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 0x7f9ba009d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9ffff640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba00cf690 0x7f9ba015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 msgr2=0x7f9ba009d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 0x7f9ba009d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba00cf690 msgr2=0x7f9ba015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba00cf690 0x7f9ba015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 -- 172.21.15.53:0/26089236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98009b30 con 0x7f9ba00ce510 2023-10-21T16:18:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9effd640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 0x7f9ba009d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b9f7fe640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c450 tx=0x7f9b9000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900052d0 con 0x7f9ba00ce510 2023-10-21T16:18:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90005430 con 0x7f9ba00ce510 2023-10-21T16:18:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.063+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90012870 con 0x7f9ba00ce510 2023-10-21T16:18:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.064+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba00ca780 con 0x7f9ba00ce510 2023-10-21T16:18:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.064+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba009db00 con 0x7f9ba00ce510 2023-10-21T16:18:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.065+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba00caca0 con 0x7f9ba00ce510 2023-10-21T16:18:24.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.066+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9b900129d0 con 0x7f9ba00ce510 2023-10-21T16:18:24.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.067+0000 7f9b87fff640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 0x7f9b8c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.067+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9b90089590 con 0x7f9ba00ce510 2023-10-21T16:18:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.068+0000 7f9b9effd640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 msgr2=0x7f9b8c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.068+0000 7f9b9effd640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 0x7f9b8c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.072+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ba00caca0 con 0x7f9ba00ce510 2023-10-21T16:18:24.080 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:24.078+0000 7f56e2c41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:24.154 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.153+0000 7f2bd9a13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:24.234 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:24.232+0000 7f56e2c41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:24.267 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.265+0000 7f2bd9a13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:24.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.269+0000 7f9b9effd640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 msgr2=0x7f9b8c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:24.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.269+0000 7f9b9effd640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 0x7f9b8c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:24.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.276+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ba00caca0 con 0x7f9ba00ce510 2023-10-21T16:18:24.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.277+0000 7f9b87fff640 1 -- 172.21.15.53:0/26089236 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v105) v1 ==== 80+0+535 (secure 0 0 0) 0x7f9ba00caca0 con 0x7f9ba00ce510 2023-10-21T16:18:24.278 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/824950480 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3674677134 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2324608475 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3280078422 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3674677134 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3826369357 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.279 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3225172013 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.280 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/798725635 2023-10-22T16:18:19.516545+0000 2023-10-21T16:18:24.280 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3415687515 2023-10-21T17:18:12.553813+0000 2023-10-21T16:18:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:listed 9 entries 2023-10-21T16:18:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 msgr2=0x7f9b8c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:24.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 0x7f9b8c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 msgr2=0x7f9ba00caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c450 tx=0x7f9b9000c910 comp rx=0 tx=0).stop 2023-10-21T16:18:24.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 shutdown_connections 2023-10-21T16:18:24.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f9b8c06afc0 0x7f9b8c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba00cf690 0x7f9ba015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba00cf2b0 0x7f9ba009d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 --2- 172.21.15.53:0/26089236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba00ce510 0x7f9ba00caf80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 >> 172.21.15.53:0/26089236 conn(0x7f9ba009bc40 msgr2=0x7f9ba00b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:24.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 shutdown_connections 2023-10-21T16:18:24.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.279+0000 7f9ba4c75640 1 -- 172.21.15.53:0/26089236 wait complete. 2023-10-21T16:18:24.284 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:24.281+0000 7f56e2c41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:24.299 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:18:24.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:24.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:24.401 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:24.398+0000 7f56e2c41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:24.435 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.432+0000 7f2bd9a13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:24.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:18:24.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:24.446 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:18:24.488 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.487+0000 7f2bd9a13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:24.490 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:18:24.491 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:24.547 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:18:24.547 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:24.548 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:24.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:18:24.607 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:18:24.608 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:24.614 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:24.613+0000 7f2bd9a13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:24.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:18:24.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:24.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.871+0000 7fb046573640 1 Processor -- start 2023-10-21T16:18:24.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.871+0000 7fb046573640 1 -- start start 2023-10-21T16:18:24.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb046573640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb040157480 0x7fb04006af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb046573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 0x7fb04006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb046573640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0400cde30 con 0x7fb04015d550 2023-10-21T16:18:24.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb046573640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04015e0b0 con 0x7fb040157480 2023-10-21T16:18:24.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb046573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb040157860 con 0x7fb0400ce2c0 2023-10-21T16:18:24.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb044d70640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 0x7fb04006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb045571640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb040157480 0x7fb04006af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb044d70640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 0x7fb04006b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:38728/0 (socket says 172.21.15.53:38728) 2023-10-21T16:18:24.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.873+0000 7fb044d70640 1 -- 172.21.15.53:0/3539575105 learned_addr learned my addr 172.21.15.53:0/3539575105 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:24.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.874+0000 7fb045571640 1 -- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 msgr2=0x7fb04006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:24.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.874+0000 7fb045571640 1 --2- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 0x7fb04006b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.874+0000 7fb045571640 1 -- 172.21.15.53:0/3539575105 >> v1:172.21.15.124:6789/0 conn(0x7fb04015d550 legacy=0x7fb04015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.874+0000 7fb045571640 1 -- 172.21.15.53:0/3539575105 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04015e2c0 con 0x7fb040157480 2023-10-21T16:18:24.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.875+0000 7fb045571640 1 --2- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb040157480 0x7fb04006af70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab38f315a318c1a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:24.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.875+0000 7fb037fff640 1 -- 172.21.15.53:0/3539575105 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c02cbc0 con 0x7fb040157480 2023-10-21T16:18:24.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.875+0000 7fb037fff640 1 -- 172.21.15.53:0/3539575105 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c02cd20 con 0x7fb040157480 2023-10-21T16:18:24.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.875+0000 7fb037fff640 1 -- 172.21.15.53:0/3539575105 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c034660 con 0x7fb040157480 2023-10-21T16:18:24.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 -- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb040157480 msgr2=0x7fb04006af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 --2- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb040157480 0x7fb04006af70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c02be10 comp rx=0 tx=0).stop 2023-10-21T16:18:24.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 -- 172.21.15.53:0/3539575105 shutdown_connections 2023-10-21T16:18:24.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 --2- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0400ce2c0 0x7fb04006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 --2- 172.21.15.53:0/3539575105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb040157480 0x7fb04006af70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 -- 172.21.15.53:0/3539575105 >> 172.21.15.53:0/3539575105 conn(0x7fb0400d2ae0 msgr2=0x7fb0400d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:24.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 -- 172.21.15.53:0/3539575105 shutdown_connections 2023-10-21T16:18:24.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.876+0000 7fb046573640 1 -- 172.21.15.53:0/3539575105 wait complete. 2023-10-21T16:18:24.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.878+0000 7fb046573640 1 Processor -- start 2023-10-21T16:18:24.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.878+0000 7fb046573640 1 -- start start 2023-10-21T16:18:24.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04015d550 0x7fb0400c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb040147750 con 0x7fb04015d550 2023-10-21T16:18:24.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400c7510 con 0x7fb0400ce2c0 2023-10-21T16:18:24.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb046573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400c77f0 con 0x7fb040157480 2023-10-21T16:18:24.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb045d72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04015d550 0x7fb0400c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb044d70640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb045571640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:24.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb044d70640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:38736/0 (socket says 172.21.15.53:38736) 2023-10-21T16:18:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb044d70640 1 -- 172.21.15.53:0/3488849779 learned_addr learned my addr 172.21.15.53:0/3488849779 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.880+0000 7fb045571640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:60638/0 (socket says 172.21.15.53:60638) 2023-10-21T16:18:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 msgr2=0x7fb040146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04015d550 msgr2=0x7fb0400c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04015d550 0x7fb0400c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 -- 172.21.15.53:0/3488849779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009a70 con 0x7fb040157480 2023-10-21T16:18:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb045571640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb044d70640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e870 tx=0x7fb03000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:24.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000c8e0 con 0x7fb040157480 2023-10-21T16:18:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000ca40 con 0x7fb040157480 2023-10-21T16:18:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000cdf0 con 0x7fb040157480 2023-10-21T16:18:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.881+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0400c7ad0 con 0x7fb040157480 2023-10-21T16:18:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.882+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb040153730 con 0x7fb040157480 2023-10-21T16:18:24.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.883+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04004e160 con 0x7fb040157480 2023-10-21T16:18:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.884+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb030013470 con 0x7fb040157480 2023-10-21T16:18:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.885+0000 7fb035ffb640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 0x7fb02406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.885+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb030089a90 con 0x7fb040157480 2023-10-21T16:18:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.885+0000 7fb045571640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 msgr2=0x7fb02406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:24.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.885+0000 7fb045571640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 0x7fb02406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:24.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:24.895+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0300548a0 con 0x7fb040157480 2023-10-21T16:18:25.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.086+0000 7fb045571640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 msgr2=0x7fb02406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:25.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.086+0000 7fb045571640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 0x7fb02406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.122+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb040160330 con 0x7fb040157480 2023-10-21T16:18:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.124+0000 7fb035ffb640 1 -- 172.21.15.53:0/3488849779 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fb030017090 con 0x7fb040157480 2023-10-21T16:18:25.125 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:25.126 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 msgr2=0x7fb02406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 0x7fb02406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 msgr2=0x7fb0400c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e870 tx=0x7fb03000ed30 comp rx=0 tx=0).stop 2023-10-21T16:18:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 shutdown_connections 2023-10-21T16:18:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7fb02406af90 0x7fb02406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb04015d550 0x7fb0400c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb040157480 0x7fb0400c82f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 --2- 172.21.15.53:0/3488849779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0400ce2c0 0x7fb040146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 >> 172.21.15.53:0/3488849779 conn(0x7fb0400d2ae0 msgr2=0x7fb0400b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 shutdown_connections 2023-10-21T16:18:25.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.126+0000 7fb046573640 1 -- 172.21.15.53:0/3488849779 wait complete. 2023-10-21T16:18:25.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:25.207 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.206+0000 7f56e2c41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:25.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.334+0000 7f367ffff640 1 Processor -- start 2023-10-21T16:18:25.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.335+0000 7f367ffff640 1 -- start start 2023-10-21T16:18:25.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.336+0000 7f367ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 0x7f368015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.336+0000 7f367ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800cde30 con 0x7f3680157480 2023-10-21T16:18:25.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f367ffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3680157860 con 0x7f36800ce2c0 2023-10-21T16:18:25.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f367ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368015e2c0 con 0x7f368006b680 2023-10-21T16:18:25.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f367effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3680157480 0x7f368006af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41888/0 (socket says 172.21.15.53:41888) 2023-10-21T16:18:25.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f367effd640 1 -- 172.21.15.53:0/2189413864 learned_addr learned my addr 172.21.15.53:0/2189413864 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f3677fff640 1 --2- 172.21.15.53:0/2189413864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 0x7f368015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.337+0000 7f367e7fc640 1 -- 172.21.15.53:0/2189413864 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480318148 0 0) 0x7f36800cde30 con 0x7f3680157480 2023-10-21T16:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.338+0000 7f367e7fc640 1 -- 172.21.15.53:0/2189413864 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c003520 con 0x7f3680157480 2023-10-21T16:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.338+0000 7f3677fff640 1 -- 172.21.15.53:0/2189413864 >> v1:172.21.15.124:6789/0 conn(0x7f36800ce2c0 legacy=0x7f368015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.338+0000 7f3677fff640 1 -- 172.21.15.53:0/2189413864 >> v1:172.21.15.53:6789/0 conn(0x7f3680157480 legacy=0x7f368006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.338+0000 7f3677fff640 1 -- 172.21.15.53:0/2189413864 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800cd500 con 0x7f368006b680 2023-10-21T16:18:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.339+0000 7f3677fff640 1 --2- 172.21.15.53:0/2189413864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 0x7f368015d550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36680097d0 tx=0x7f366802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d723d9098876f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.339+0000 7f367e7fc640 1 -- 172.21.15.53:0/2189413864 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366802d040 con 0x7f368006b680 2023-10-21T16:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.339+0000 7f367e7fc640 1 -- 172.21.15.53:0/2189413864 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668031810 con 0x7f368006b680 2023-10-21T16:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.339+0000 7f367e7fc640 1 -- 172.21.15.53:0/2189413864 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668031b80 con 0x7f368006b680 2023-10-21T16:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 -- 172.21.15.53:0/2189413864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 msgr2=0x7f368015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 --2- 172.21.15.53:0/2189413864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 0x7f368015d550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36680097d0 tx=0x7f366802b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 -- 172.21.15.53:0/2189413864 shutdown_connections 2023-10-21T16:18:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 --2- 172.21.15.53:0/2189413864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f368006b680 0x7f368015d550 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 -- 172.21.15.53:0/2189413864 >> 172.21.15.53:0/2189413864 conn(0x7f36800d2ae0 msgr2=0x7f36800d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 -- 172.21.15.53:0/2189413864 shutdown_connections 2023-10-21T16:18:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.340+0000 7f367ffff640 1 -- 172.21.15.53:0/2189413864 wait complete. 2023-10-21T16:18:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.342+0000 7f367ffff640 1 Processor -- start 2023-10-21T16:18:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.342+0000 7f367ffff640 1 -- start start 2023-10-21T16:18:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800ce2c0 0x7f36800cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3680157480 0x7f36800cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800cba80 con 0x7f368006b680 2023-10-21T16:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36801507a0 con 0x7f3680157480 2023-10-21T16:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36801f3710 con 0x7f36800ce2c0 2023-10-21T16:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50976/0 (socket says 172.21.15.53:50976) 2023-10-21T16:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f367effd640 1 -- 172.21.15.53:0/1175786281 learned_addr learned my addr 172.21.15.53:0/1175786281 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.343+0000 7f3677fff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800ce2c0 0x7f36800cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367f7fe640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3680157480 0x7f36800cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800ce2c0 msgr2=0x7f36800cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800ce2c0 0x7f36800cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3680157480 msgr2=0x7f36800cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3680157480 0x7f36800cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 -- 172.21.15.53:0/1175786281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668009400 con 0x7f368006b680 2023-10-21T16:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f367effd640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f367000c500 tx=0x7f367000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.344+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670005ce0 con 0x7f368006b680 2023-10-21T16:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.345+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670005e40 con 0x7f368006b680 2023-10-21T16:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.345+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670007330 con 0x7f368006b680 2023-10-21T16:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.345+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3680150470 con 0x7f368006b680 2023-10-21T16:18:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.345+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36800cb7a0 con 0x7f368006b680 2023-10-21T16:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.346+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36801502d0 con 0x7f368006b680 2023-10-21T16:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.347+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3670007490 con 0x7f368006b680 2023-10-21T16:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.348+0000 7f36777fe640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 0x7f366006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.348+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3670088ff0 con 0x7f368006b680 2023-10-21T16:18:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.348+0000 7f3677fff640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 msgr2=0x7f366006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.348+0000 7f3677fff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 0x7f366006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.352+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36801502d0 con 0x7f368006b680 2023-10-21T16:18:25.465 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:25.464+0000 7f2bd9a13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:25.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.500+0000 7f56e2c41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:25.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.545+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36801502d0 con 0x7f368006b680 2023-10-21T16:18:25.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.548+0000 7f36777fe640 1 -- 172.21.15.53:0/1175786281 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+17416 (secure 0 0 0) 0x7f36801502d0 con 0x7f368006b680 2023-10-21T16:18:25.550 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:25.550 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":105,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:19.516665+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/824950480":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6833/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/2324608475":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3280078422":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6832/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3826369357":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3225172013":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/798725635":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3415687515":"2023-10-21T17:18:12.553813+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:25.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.549+0000 7f3677fff640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 msgr2=0x7f366006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1331540020 2023-10-21T16:18:25.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.549+0000 7f3677fff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 0x7f366006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 msgr2=0x7f366006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 0x7f366006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 msgr2=0x7f36800caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f367000c500 tx=0x7f367000c9c0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 shutdown_connections 2023-10-21T16:18:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:6832/1331540020,v1:172.21.15.124:6833/1331540020] conn(0x7f366006af90 0x7f366006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3680157480 0x7f36800cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36800ce2c0 0x7f36800cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 --2- 172.21.15.53:0/1175786281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f368006b680 0x7f36800caa00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.550+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 >> 172.21.15.53:0/1175786281 conn(0x7f36800d2ae0 msgr2=0x7f36800b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.551+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 shutdown_connections 2023-10-21T16:18:25.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.551+0000 7f367ffff640 1 -- 172.21.15.53:0/1175786281 wait complete. 2023-10-21T16:18:25.557 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.557+0000 7f56e2c41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:25.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:18:02.821177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:18:03.863098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:18:25.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:18:25.673 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.672+0000 7f56e2c41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:25.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.761+0000 7f5667fba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.763 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.761+0000 7f5667fba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.763 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.761+0000 7f5667fba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.763 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.761+0000 7f5667fba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.764 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.761+0000 7f5667fba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.765 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.762+0000 7f5664fb4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.765 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.762+0000 7f5664fb4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.765 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.762+0000 7f5664fb4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.766 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.762+0000 7f5664fb4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.766 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:25.762+0000 7f5664fb4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.778+0000 7f61e939e640 1 Processor -- start 2023-10-21T16:18:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.778+0000 7f61e939e640 1 -- start start 2023-10-21T16:18:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61e40b45e0 0x7f61e40b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61e40b83a0 0x7f61e40b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b8830 0x7f61e40b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40b6b40 con 0x7f61e40b45e0 2023-10-21T16:18:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40b4f00 con 0x7f61e40b83a0 2023-10-21T16:18:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e939e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40b5ec0 con 0x7f61e40b8830 2023-10-21T16:18:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e2ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61e40b45e0 0x7f61e40b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e2ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f61e40b45e0 0x7f61e40b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50984/0 (socket says 172.21.15.53:50984) 2023-10-21T16:18:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.780+0000 7f61e2ffd640 1 -- 172.21.15.53:0/4102234582 learned_addr learned my addr 172.21.15.53:0/4102234582 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e37fe640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b8830 0x7f61e40b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61dbfff640 1 --2- 172.21.15.53:0/4102234582 >> v2:172.21.15.124:3300/0 conn(0x7f61e40b83a0 0x7f61e40b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e2ffd640 1 -- 172.21.15.53:0/4102234582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b8830 msgr2=0x7f61e40b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:25.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e2ffd640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b8830 0x7f61e40b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e2ffd640 1 -- 172.21.15.53:0/4102234582 >> v2:172.21.15.124:3300/0 conn(0x7f61e40b83a0 msgr2=0x7f61e40b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:25.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e2ffd640 1 --2- 172.21.15.53:0/4102234582 >> v2:172.21.15.124:3300/0 conn(0x7f61e40b83a0 0x7f61e40b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.781+0000 7f61e2ffd640 1 -- 172.21.15.53:0/4102234582 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40b61a0 con 0x7f61e40b45e0 2023-10-21T16:18:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.782+0000 7f61e2ffd640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b45e0 0x7f61e40b49c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f61e40d3c40 tx=0x7f61d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516e5a9a024458a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.782+0000 7f61e27fc640 1 -- 172.21.15.53:0/4102234582 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4031870 con 0x7f61e40b45e0 2023-10-21T16:18:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.782+0000 7f61e27fc640 1 -- 172.21.15.53:0/4102234582 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d40319d0 con 0x7f61e40b45e0 2023-10-21T16:18:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.782+0000 7f61e27fc640 1 -- 172.21.15.53:0/4102234582 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4031d60 con 0x7f61e40b45e0 2023-10-21T16:18:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 -- 172.21.15.53:0/4102234582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b45e0 msgr2=0x7f61e40b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b45e0 0x7f61e40b49c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f61e40d3c40 tx=0x7f61d402b910 comp rx=0 tx=0).stop 2023-10-21T16:18:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 -- 172.21.15.53:0/4102234582 shutdown_connections 2023-10-21T16:18:25.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b8830 0x7f61e40b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 --2- 172.21.15.53:0/4102234582 >> v2:172.21.15.124:3300/0 conn(0x7f61e40b83a0 0x7f61e40b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 --2- 172.21.15.53:0/4102234582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b45e0 0x7f61e40b49c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 -- 172.21.15.53:0/4102234582 >> 172.21.15.53:0/4102234582 conn(0x7f61e40718a0 msgr2=0x7f61e4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 -- 172.21.15.53:0/4102234582 shutdown_connections 2023-10-21T16:18:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.783+0000 7f61e939e640 1 -- 172.21.15.53:0/4102234582 wait complete. 2023-10-21T16:18:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.785+0000 7f61e939e640 1 Processor -- start 2023-10-21T16:18:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.785+0000 7f61e939e640 1 -- start start 2023-10-21T16:18:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 0x7f61e40c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 0x7f61e40cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40cb7f0 con 0x7f61e40b8830 2023-10-21T16:18:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40ca210 con 0x7f61e40b83a0 2023-10-21T16:18:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e939e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40ca4f0 con 0x7f61e40b45e0 2023-10-21T16:18:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e2ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 0x7f61e40c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.786+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 0x7f61e40cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e2ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 0x7f61e40c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:38760/0 (socket says 172.21.15.53:38760) 2023-10-21T16:18:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50990/0 (socket says 172.21.15.53:50990) 2023-10-21T16:18:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e2ffd640 1 -- 172.21.15.53:0/62913035 learned_addr learned my addr 172.21.15.53:0/62913035 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:25.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 0x7f61e40cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:60680/0 (socket says 172.21.15.53:60680) 2023-10-21T16:18:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 -- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 msgr2=0x7f61e40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 0x7f61e40c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 -- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 msgr2=0x7f61e40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 0x7f61e40cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 -- 172.21.15.53:0/62913035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40094d0 con 0x7f61e40b8830 2023-10-21T16:18:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.787+0000 7f61e37fe640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eba0 tx=0x7f61dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.788+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00cd20 con 0x7f61e40b8830 2023-10-21T16:18:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.788+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc00f040 con 0x7f61e40b8830 2023-10-21T16:18:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.788+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0146c0 con 0x7f61e40b8830 2023-10-21T16:18:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.788+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e40ca7d0 con 0x7f61e40b8830 2023-10-21T16:18:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.788+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e40cad90 con 0x7f61e40b8830 2023-10-21T16:18:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.789+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e40cafb0 con 0x7f61e40b8830 2023-10-21T16:18:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.790+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 42) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f61dc014820 con 0x7f61e40b8830 2023-10-21T16:18:25.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.791+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f61dc08a200 con 0x7f61e40b8830 2023-10-21T16:18:25.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.795+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61dc055270 con 0x7f61e40b8830 2023-10-21T16:18:25.803 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:25.802+0000 7f2bd9a13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:25.867 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:25.865+0000 7f2bd9a13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:25.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.986+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61e40b6450 con 0x7f61e40b8830 2023-10-21T16:18:25.988 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:25.986+0000 7f2bd9a13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:25.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.986+0000 7f61db7fe640 1 -- 172.21.15.53:0/62913035 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61dc058890 con 0x7f61e40b8830 2023-10-21T16:18:25.989 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:18:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 msgr2=0x7f61e41f1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:25.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00eba0 tx=0x7f61dc00c490 comp rx=0 tx=0).stop 2023-10-21T16:18:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 shutdown_connections 2023-10-21T16:18:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e40b8830 0x7f61e41f1e20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e40b83a0 0x7f61e40cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 --2- 172.21.15.53:0/62913035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e40b45e0 0x7f61e40c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 >> 172.21.15.53:0/62913035 conn(0x7f61e40718a0 msgr2=0x7f61e40d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:25.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 shutdown_connections 2023-10-21T16:18:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:25.990+0000 7f61e939e640 1 -- 172.21.15.53:0/62913035 wait complete. 2023-10-21T16:18:26.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:18:26.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.223+0000 7f4ebfdb0640 1 Processor -- start 2023-10-21T16:18:26.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.223+0000 7f4ebfdb0640 1 -- start start 2023-10-21T16:18:26.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.224+0000 7f4ebfdb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4eb80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebfdb0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb815b310 con 0x7f4eb806db60 2023-10-21T16:18:26.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebfdb0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb806d290 con 0x7f4eb815aa00 2023-10-21T16:18:26.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebfdb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80ca230 con 0x7f4eb806c3a0 2023-10-21T16:18:26.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebd34b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4eb806db60 0x7f4eb806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41892/0 (socket says 172.21.15.53:41892) 2023-10-21T16:18:26.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebd34b640 1 -- 172.21.15.53:0/1403373867 learned_addr learned my addr 172.21.15.53:0/1403373867 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:26.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebdb4c640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4eb80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416750204 0 0) 0x7f4ea8001110 con 0x7f4eb806db60 2023-10-21T16:18:26.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.225+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea4003520 con 0x7f4eb806db60 2023-10-21T16:18:26.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590788539 0 0) 0x7f4ea4003520 con 0x7f4eb806db60 2023-10-21T16:18:26.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 msgr2=0x7f4eb80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:26.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4eb80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 >> v1:172.21.15.124:6789/0 conn(0x7f4eb815aa00 legacy=0x7f4eb80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80c9860 con 0x7f4eb806db60 2023-10-21T16:18:26.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4ea8001800 con 0x7f4eb806db60 2023-10-21T16:18:26.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 >> v1:172.21.15.53:6789/0 conn(0x7f4eb806db60 legacy=0x7f4eb806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4ea40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ea4003f70 0x7f4ea40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4004d00 0x7f4ea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea400ba90 con 0x7f4ea4003f70 2023-10-21T16:18:26.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea400bc00 con 0x7f4ea4004d00 2023-10-21T16:18:26.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea400dfd0 con 0x7f4eb806c3a0 2023-10-21T16:18:26.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.226+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ea4003f70 0x7f4ea40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebe34d640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4004d00 0x7f4ea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebdb4c640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4ea40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebd34b640 1 -- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 msgr2=0x7f4ea40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:26.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb806c3a0 0x7f4ea40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebd34b640 1 -- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4004d00 msgr2=0x7f4ea4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:26.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4004d00 0x7f4ea4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.227+0000 7f4ebd34b640 1 -- 172.21.15.53:0/1403373867 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4003520 con 0x7f4ea4003f70 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.228+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ea4003f70 0x7f4ea40047c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8007fb0 tx=0x7f4ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0d52bc99326713 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.228+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8005ad0 con 0x7f4ea4003f70 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.228+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8005c50 con 0x7f4ea4003f70 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.228+0000 7f4ebcb4a640 1 -- 172.21.15.53:0/1403373867 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8007070 con 0x7f4ea4003f70 2023-10-21T16:18:26.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/1403373867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ea4003f70 msgr2=0x7f4ea40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 --2- 172.21.15.53:0/1403373867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ea4003f70 0x7f4ea40047c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8007fb0 tx=0x7f4ea40057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebe34d640 1 -- 172.21.15.53:0/1403373867 reap_dead start 2023-10-21T16:18:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/1403373867 shutdown_connections 2023-10-21T16:18:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/1403373867 >> 172.21.15.53:0/1403373867 conn(0x7f4eb806a5d0 msgr2=0x7f4eb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:26.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/1403373867 shutdown_connections 2023-10-21T16:18:26.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.229+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/1403373867 wait complete. 2023-10-21T16:18:26.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.231+0000 7f4ebfdb0640 1 Processor -- start 2023-10-21T16:18:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.231+0000 7f4ebfdb0640 1 -- start start 2023-10-21T16:18:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 0x7f4eb80c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 0x7f4eb80c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80c86c0 con 0x7f4eb8153590 2023-10-21T16:18:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb815cc40 con 0x7f4ea4003f70 2023-10-21T16:18:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebfdb0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80c2620 con 0x7f4eb80c6220 2023-10-21T16:18:26.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebd34b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebd34b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50998/0 (socket says 172.21.15.53:50998) 2023-10-21T16:18:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebd34b640 1 -- 172.21.15.53:0/812976082 learned_addr learned my addr 172.21.15.53:0/812976082 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.232+0000 7f4ebe34d640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 0x7f4eb80c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebdb4c640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 0x7f4eb80c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 -- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 msgr2=0x7f4eb80c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 0x7f4eb80c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 -- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 msgr2=0x7f4eb80c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 0x7f4eb80c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 -- 172.21.15.53:0/812976082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb815c610 con 0x7f4eb8153590 2023-10-21T16:18:26.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebe34d640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 0x7f4eb80c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:26.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebdb4c640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 0x7f4eb80c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:26.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebd34b640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4ea802c280 tx=0x7f4eb80c5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:26.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8039070 con 0x7f4eb8153590 2023-10-21T16:18:26.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80074a0 con 0x7f4eb8153590 2023-10-21T16:18:26.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea803c040 con 0x7f4eb8153590 2023-10-21T16:18:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb80c4d70 con 0x7f4eb8153590 2023-10-21T16:18:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.233+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb815c8d0 con 0x7f4eb8153590 2023-10-21T16:18:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.234+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb80c7ea0 con 0x7f4eb8153590 2023-10-21T16:18:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.235+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 42) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4ea8032070 con 0x7f4eb8153590 2023-10-21T16:18:26.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.236+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f4ea80aa6e0 con 0x7f4eb8153590 2023-10-21T16:18:26.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.241+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4eb80c7ea0 con 0x7f4eb8153590 2023-10-21T16:18:26.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.438+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4eb80c7ea0 con 0x7f4eb8153590 2023-10-21T16:18:26.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.439+0000 7f4eae7fc640 1 -- 172.21.15.53:0/812976082 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4eb80c7ea0 con 0x7f4eb8153590 2023-10-21T16:18:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:18:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 msgr2=0x7f4eb8153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4ea802c280 tx=0x7f4eb80c5560 comp rx=0 tx=0).stop 2023-10-21T16:18:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 shutdown_connections 2023-10-21T16:18:26.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4eb80c6220 0x7f4eb80c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4eb8153590 0x7f4eb8153a10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 --2- 172.21.15.53:0/812976082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ea4003f70 0x7f4eb80c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 >> 172.21.15.53:0/812976082 conn(0x7f4eb806a5d0 msgr2=0x7f4eb80caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 shutdown_connections 2023-10-21T16:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.442+0000 7f4ebfdb0640 1 -- 172.21.15.53:0/812976082 wait complete. 2023-10-21T16:18:26.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:18:26.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.696+0000 7f4284808640 1 Processor -- start 2023-10-21T16:18:26.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.696+0000 7f4284808640 1 -- start start 2023-10-21T16:18:26.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.698+0000 7f4284808640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42801599d0 0x7f42800cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.699+0000 7f4284808640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.699+0000 7f4284808640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4280159db0 con 0x7f42800b3c50 2023-10-21T16:18:26.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.699+0000 7f4284808640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4280071fe0 con 0x7f42801599d0 2023-10-21T16:18:26.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.699+0000 7f4284808640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42800b44c0 con 0x7f42800cb5e0 2023-10-21T16:18:26.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.700+0000 7f427e59c640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f42800b3c50 0x7f42800caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:49540/0 (socket says 172.21.15.53:49540) 2023-10-21T16:18:26.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.700+0000 7f427e59c640 1 -- 172.21.15.53:0/4213928046 learned_addr learned my addr 172.21.15.53:0/4213928046 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:26.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.701+0000 7f427ed9d640 1 --2- 172.21.15.53:0/4213928046 >> v2:172.21.15.124:3300/0 conn(0x7f42801599d0 0x7f42800cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.701+0000 7f427f59e640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.702+0000 7f427ed9d640 1 -- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 msgr2=0x7f42800cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:26.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.702+0000 7f427ed9d640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.702+0000 7f427ed9d640 1 -- 172.21.15.53:0/4213928046 >> v1:172.21.15.124:6789/0 conn(0x7f42800b3c50 legacy=0x7f42800caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:26.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.702+0000 7f427ed9d640 1 -- 172.21.15.53:0/4213928046 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800713d0 con 0x7f42801599d0 2023-10-21T16:18:26.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.704+0000 7f427ed9d640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42801599d0 0x7f42800cdcd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42700097d0 tx=0x7f427002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80a59e7df033391 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:26.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.707+0000 7f427dd9b640 1 -- 172.21.15.53:0/4213928046 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270031780 con 0x7f42801599d0 2023-10-21T16:18:26.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.707+0000 7f427dd9b640 1 -- 172.21.15.53:0/4213928046 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42700318e0 con 0x7f42801599d0 2023-10-21T16:18:26.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.707+0000 7f427dd9b640 1 -- 172.21.15.53:0/4213928046 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4270031c70 con 0x7f42801599d0 2023-10-21T16:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 -- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42801599d0 msgr2=0x7f42800cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42801599d0 0x7f42800cdcd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42700097d0 tx=0x7f427002b820 comp rx=0 tx=0).stop 2023-10-21T16:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 -- 172.21.15.53:0/4213928046 shutdown_connections 2023-10-21T16:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 --2- 172.21.15.53:0/4213928046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42801599d0 0x7f42800cdcd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 -- 172.21.15.53:0/4213928046 >> 172.21.15.53:0/4213928046 conn(0x7f428006c050 msgr2=0x7f428006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 -- 172.21.15.53:0/4213928046 shutdown_connections 2023-10-21T16:18:26.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.708+0000 7f4284808640 1 -- 172.21.15.53:0/4213928046 wait complete. 2023-10-21T16:18:26.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.710+0000 7f4284808640 1 Processor -- start 2023-10-21T16:18:26.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.710+0000 7f4284808640 1 -- start start 2023-10-21T16:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42800b3c50 0x7f42800c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42800c9460 con 0x7f42801599d0 2023-10-21T16:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42800c75e0 con 0x7f42800b3c50 2023-10-21T16:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.711+0000 7f4284808640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42800c69d0 con 0x7f42800cb5e0 2023-10-21T16:18:26.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51000/0 (socket says 172.21.15.53:51000) 2023-10-21T16:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 -- 172.21.15.53:0/3468786414 learned_addr learned my addr 172.21.15.53:0/3468786414 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:26.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427e59c640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 -- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 msgr2=0x7f42800c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 -- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42800b3c50 msgr2=0x7f42800c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42800b3c50 0x7f42800c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 -- 172.21.15.53:0/3468786414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009400 con 0x7f42801599d0 2023-10-21T16:18:26.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.712+0000 7f427f59e640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4278007e30 tx=0x7f427800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.713+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278014620 con 0x7f42801599d0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.713+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427800ee90 con 0x7f42801599d0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.713+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42800c6cb0 con 0x7f42801599d0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.713+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780149c0 con 0x7f42801599d0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.713+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42800c7270 con 0x7f42801599d0 2023-10-21T16:18:26.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.714+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4280074150 con 0x7f42801599d0 2023-10-21T16:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.715+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4278014b20 con 0x7f42801599d0 2023-10-21T16:18:26.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.716+0000 7f42677fe640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 0x7f426c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.716+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f427808a110 con 0x7f42801599d0 2023-10-21T16:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.717+0000 7f427ed9d640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 0x7f426c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:26.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.718+0000 7f427ed9d640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 0x7f426c06d180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4270036000 tx=0x7f4270030000 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:26.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.722+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4278054fe0 con 0x7f42801599d0 2023-10-21T16:18:26.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.917+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42800c7720 con 0x7f42801599d0 2023-10-21T16:18:26.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.918+0000 7f42677fe640 1 -- 172.21.15.53:0/3468786414 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4278058600 con 0x7f42801599d0 2023-10-21T16:18:26.919 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:18:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 msgr2=0x7f426c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 0x7f426c06d180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4270036000 tx=0x7f4270030000 comp rx=0 tx=0).stop 2023-10-21T16:18:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 msgr2=0x7f42800ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:26.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4278007e30 tx=0x7f427800e4b0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 shutdown_connections 2023-10-21T16:18:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f426c06acc0 0x7f426c06d180 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42801599d0 0x7f42800ae9f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42800cb5e0 0x7f42800c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 --2- 172.21.15.53:0/3468786414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42800b3c50 0x7f42800c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 >> 172.21.15.53:0/3468786414 conn(0x7f428006c050 msgr2=0x7f4280157780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 shutdown_connections 2023-10-21T16:18:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:26.920+0000 7f4284808640 1 -- 172.21.15.53:0/3468786414 wait complete. 2023-10-21T16:18:26.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.148+0000 7fa2ff173640 1 Processor -- start 2023-10-21T16:18:27.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.149+0000 7fa2ff173640 1 -- start start 2023-10-21T16:18:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.150+0000 7fa2ff173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f815cc60 0x7fa2f80cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.150+0000 7fa2ff173640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2f80d16f0 0x7fa2f80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.150+0000 7fa2ff173640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f815d040 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.150+0000 7fa2ff173640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f80d0650 con 0x7fa2f80d16f0 2023-10-21T16:18:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.150+0000 7fa2ff173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80d0930 con 0x7fa2f815cc60 2023-10-21T16:18:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2f80d16f0 0x7fa2f80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2f80d16f0 0x7fa2f80cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51016/0 (socket says 172.21.15.53:51016) 2023-10-21T16:18:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 -- 172.21.15.53:0/2381055681 learned_addr learned my addr 172.21.15.53:0/2381055681 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fe171640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f815cc60 0x7fa2f80cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 -- 172.21.15.53:0/2381055681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f815cc60 msgr2=0x7fa2f80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f815cc60 0x7fa2f80cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 -- 172.21.15.53:0/2381055681 >> v1:172.21.15.124:6789/0 conn(0x7fa2f80cdfd0 legacy=0x7fa2f80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:27.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.151+0000 7fa2fd970640 1 -- 172.21.15.53:0/2381055681 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80d0c10 con 0x7fa2f80d16f0 2023-10-21T16:18:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.152+0000 7fa2fd970640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80d16f0 0x7fa2f80cf880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009870 tx=0x7fa2e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3d6835a2142e97 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.152+0000 7fa2fd16f640 1 -- 172.21.15.53:0/2381055681 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8031640 con 0x7fa2f80d16f0 2023-10-21T16:18:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2fd16f640 1 -- 172.21.15.53:0/2381055681 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e80317a0 con 0x7fa2f80d16f0 2023-10-21T16:18:27.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2fd16f640 1 -- 172.21.15.53:0/2381055681 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8031b30 con 0x7fa2f80d16f0 2023-10-21T16:18:27.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 -- 172.21.15.53:0/2381055681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80d16f0 msgr2=0x7fa2f80cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80d16f0 0x7fa2f80cf880 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009870 tx=0x7fa2e802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 -- 172.21.15.53:0/2381055681 shutdown_connections 2023-10-21T16:18:27.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80d16f0 0x7fa2f80cf880 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 --2- 172.21.15.53:0/2381055681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f815cc60 0x7fa2f80cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 -- 172.21.15.53:0/2381055681 >> 172.21.15.53:0/2381055681 conn(0x7fa2f809bcf0 msgr2=0x7fa2f80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:27.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 -- 172.21.15.53:0/2381055681 shutdown_connections 2023-10-21T16:18:27.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.153+0000 7fa2ff173640 1 -- 172.21.15.53:0/2381055681 wait complete. 2023-10-21T16:18:27.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.155+0000 7fa2ff173640 1 Processor -- start 2023-10-21T16:18:27.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.155+0000 7fa2ff173640 1 -- start start 2023-10-21T16:18:27.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f80d16f0 0x7fa2f81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 0x7fa2f80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8154110 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81543f0 con 0x7fa2f815cc60 2023-10-21T16:18:27.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2ff173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81546d0 con 0x7fa2f80d16f0 2023-10-21T16:18:27.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2fe171640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2fe171640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51020/0 (socket says 172.21.15.53:51020) 2023-10-21T16:18:27.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.156+0000 7fa2fe171640 1 -- 172.21.15.53:0/1443623173 learned_addr learned my addr 172.21.15.53:0/1443623173 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:27.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fd970640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f80d16f0 0x7fa2f81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 -- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f80d16f0 msgr2=0x7fa2f81537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f80d16f0 0x7fa2f81537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe972640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 0x7fa2f80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 -- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 msgr2=0x7fa2f80b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 0x7fa2f80b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 -- 172.21.15.53:0/1443623173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e80094d0 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe972640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 0x7fa2f80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.157+0000 7fa2fe171640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c7f0 tx=0x7fa2f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.158+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4007820 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.158+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4007980 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.158+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f81549b0 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.158+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4007d30 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.158+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f8153450 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.159+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f804e160 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.160+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa2f4004020 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.160+0000 7fa2eeffd640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 0x7fa2dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.161+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fa2f4089ab0 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.161+0000 7fa2fd970640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 0x7fa2dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.163+0000 7fa2fd970640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 0x7fa2dc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8036000 tx=0x7fa2e803c040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.165+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2f4054980 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.397+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2f80d3080 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.398+0000 7fa2eeffd640 1 -- 172.21.15.53:0/1443623173 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fa2f401f090 con 0x7fa2f80cdfd0 2023-10-21T16:18:27.399 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:27.400 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:27.400 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.400+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 msgr2=0x7fa2dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.400+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 0x7fa2dc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8036000 tx=0x7fa2e803c040 comp rx=0 tx=0).stop 2023-10-21T16:18:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 msgr2=0x7fa2f8152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c7f0 tx=0x7fa2f400ccb0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 shutdown_connections 2023-10-21T16:18:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa2dc06acc0 0x7fa2dc06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2f815cc60 0x7fa2f80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2f80d16f0 0x7fa2f81537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 --2- 172.21.15.53:0/1443623173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2f80cdfd0 0x7fa2f8152f10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 >> 172.21.15.53:0/1443623173 conn(0x7fa2f809bcf0 msgr2=0x7fa2f80b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 shutdown_connections 2023-10-21T16:18:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.401+0000 7fa2ff173640 1 -- 172.21.15.53:0/1443623173 wait complete. 2023-10-21T16:18:27.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:27.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.632+0000 7ff6966ec640 1 Processor -- start 2023-10-21T16:18:27.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.632+0000 7ff6966ec640 1 -- start start 2023-10-21T16:18:27.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6901599d0 0x7ff6900ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6900cc210 0x7ff6900b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 0x7ff6900b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900b10f0 con 0x7ff6900cc210 2023-10-21T16:18:27.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900cacf0 con 0x7ff690159db0 2023-10-21T16:18:27.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff6966ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900ca3c0 con 0x7ff6901599d0 2023-10-21T16:18:27.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff694ee9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6900cc210 0x7ff6900b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff694ee9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6900cc210 0x7ff6900b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51028/0 (socket says 172.21.15.53:51028) 2023-10-21T16:18:27.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.634+0000 7ff694ee9640 1 -- 172.21.15.53:0/1465758540 learned_addr learned my addr 172.21.15.53:0/1465758540 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff6956ea640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6901599d0 0x7ff6900ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff695eeb640 1 --2- 172.21.15.53:0/1465758540 >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 0x7ff6900b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff694ee9640 1 -- 172.21.15.53:0/1465758540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6901599d0 msgr2=0x7ff6900ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff694ee9640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6901599d0 0x7ff6900ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff694ee9640 1 -- 172.21.15.53:0/1465758540 >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 msgr2=0x7ff6900b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff694ee9640 1 --2- 172.21.15.53:0/1465758540 >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 0x7ff6900b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff694ee9640 1 -- 172.21.15.53:0/1465758540 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900ca6a0 con 0x7ff6900cc210 2023-10-21T16:18:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.635+0000 7ff695eeb640 1 --2- 172.21.15.53:0/1465758540 >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 0x7ff6900b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:27.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.636+0000 7ff694ee9640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6900cc210 0x7ff6900b00c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff6900b4430 tx=0x7ff68002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb336c3ded5896b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.636+0000 7ff687fff640 1 -- 172.21.15.53:0/1465758540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68002ee90 con 0x7ff6900cc210 2023-10-21T16:18:27.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.636+0000 7ff687fff640 1 -- 172.21.15.53:0/1465758540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680036440 con 0x7ff6900cc210 2023-10-21T16:18:27.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.636+0000 7ff687fff640 1 -- 172.21.15.53:0/1465758540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800367d0 con 0x7ff6900cc210 2023-10-21T16:18:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 -- 172.21.15.53:0/1465758540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6900cc210 msgr2=0x7ff6900b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6900cc210 0x7ff6900b00c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff6900b4430 tx=0x7ff68002e0e0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 -- 172.21.15.53:0/1465758540 shutdown_connections 2023-10-21T16:18:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 --2- 172.21.15.53:0/1465758540 >> v2:172.21.15.124:3300/0 conn(0x7ff690159db0 0x7ff6900b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6900cc210 0x7ff6900b00c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 --2- 172.21.15.53:0/1465758540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6901599d0 0x7ff6900ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 -- 172.21.15.53:0/1465758540 >> 172.21.15.53:0/1465758540 conn(0x7ff6900cb0c0 msgr2=0x7ff6900cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 -- 172.21.15.53:0/1465758540 shutdown_connections 2023-10-21T16:18:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.637+0000 7ff6966ec640 1 -- 172.21.15.53:0/1465758540 wait complete. 2023-10-21T16:18:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.639+0000 7ff6966ec640 1 Processor -- start 2023-10-21T16:18:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.639+0000 7ff6966ec640 1 -- start start 2023-10-21T16:18:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6900cc210 0x7ff6901534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff690159db0 0x7ff69015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900c8630 con 0x7ff6901599d0 2023-10-21T16:18:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015bc50 con 0x7ff6900cc210 2023-10-21T16:18:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff6966ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900c8e40 con 0x7ff690159db0 2023-10-21T16:18:27.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff694ee9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff694ee9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51036/0 (socket says 172.21.15.53:51036) 2023-10-21T16:18:27.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.640+0000 7ff694ee9640 1 -- 172.21.15.53:0/2595566694 learned_addr learned my addr 172.21.15.53:0/2595566694 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:27.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 -- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff690159db0 msgr2=0x7ff69015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff690159db0 0x7ff69015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 -- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6900cc210 msgr2=0x7ff6901534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6900cc210 0x7ff6901534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 -- 172.21.15.53:0/2595566694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68000b770 con 0x7ff6901599d0 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff694ee9640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff68002e120 tx=0x7ff6800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680034b80 con 0x7ff6901599d0 2023-10-21T16:18:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680036e90 con 0x7ff6901599d0 2023-10-21T16:18:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680041460 con 0x7ff6901599d0 2023-10-21T16:18:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6900c5840 con 0x7ff6901599d0 2023-10-21T16:18:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.641+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6900c8b60 con 0x7ff6901599d0 2023-10-21T16:18:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.642+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69004e160 con 0x7ff6901599d0 2023-10-21T16:18:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.643+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff68004a3b0 con 0x7ff6901599d0 2023-10-21T16:18:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.644+0000 7ff685ffb640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 0x7ff67406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.644+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7ff6800abf10 con 0x7ff6901599d0 2023-10-21T16:18:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.645+0000 7ff6956ea640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 0x7ff67406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.646+0000 7ff6956ea640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 0x7ff67406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006b10 tx=0x7ff68c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.648+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff680076e60 con 0x7ff6901599d0 2023-10-21T16:18:27.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.712+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff680077230 con 0x7ff6901599d0 2023-10-21T16:18:27.744 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:27.742+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:27.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.856+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6901e7830 con 0x7ff6901599d0 2023-10-21T16:18:27.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.859+0000 7ff685ffb640 1 -- 172.21.15.53:0/2595566694 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17924 (secure 0 0 0) 0x7ff680090650 con 0x7ff6901599d0 2023-10-21T16:18:27.862 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:27.862 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":106,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:25.676361+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2451852009":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/600599962":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2629801108":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2272255357":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/1235374158":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6833/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6832/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/824950480":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6833/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/2324608475":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3280078422":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6832/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3826369357":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3225172013":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/539630491":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/798725635":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3415687515":"2023-10-21T17:18:12.553813+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:27.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 msgr2=0x7ff67406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 0x7ff67406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006b10 tx=0x7ff68c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:27.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 msgr2=0x7ff6900c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:27.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff68002e120 tx=0x7ff6800046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 shutdown_connections 2023-10-21T16:18:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7ff67406ad10 0x7ff67406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff690159db0 0x7ff69015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6901599d0 0x7ff6900c80f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 --2- 172.21.15.53:0/2595566694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6900cc210 0x7ff6901534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:27.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 >> 172.21.15.53:0/2595566694 conn(0x7ff6900cb0c0 msgr2=0x7ff690158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:27.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 shutdown_connections 2023-10-21T16:18:27.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:27.862+0000 7ff6966ec640 1 -- 172.21.15.53:0/2595566694 wait complete. 2023-10-21T16:18:27.883 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:18:27.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:28.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.105+0000 7fa130d65640 1 Processor -- start 2023-10-21T16:18:28.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.105+0000 7fa130d65640 1 -- start start 2023-10-21T16:18:28.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.106+0000 7fa130d65640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.106+0000 7fa130d65640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa12c0c90b0 0x7fa12c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.106+0000 7fa130d65640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12c15b2b0 con 0x7fa12c0cae70 2023-10-21T16:18:28.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.106+0000 7fa130d65640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c15bf30 con 0x7fa12c0c90b0 2023-10-21T16:18:28.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.106+0000 7fa130d65640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c15f7a0 con 0x7fa12c15ab40 2023-10-21T16:18:28.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa123fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa12c0c90b0 0x7fa12c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa12ad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa12c0cae70 0x7fa12c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41908/0 (socket says 172.21.15.53:41908) 2023-10-21T16:18:28.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa12ad9d640 1 -- 172.21.15.53:0/3214067140 learned_addr learned my addr 172.21.15.53:0/3214067140 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:28.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa12a59c640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa123fff640 1 -- 172.21.15.53:0/3214067140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 msgr2=0x7fa12c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:28.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.107+0000 7fa123fff640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.108+0000 7fa123fff640 1 -- 172.21.15.53:0/3214067140 >> v1:172.21.15.53:6789/0 conn(0x7fa12c0cae70 legacy=0x7fa12c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.108+0000 7fa123fff640 1 -- 172.21.15.53:0/3214067140 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c15b590 con 0x7fa12c0c90b0 2023-10-21T16:18:28.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.109+0000 7fa123fff640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0adc60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa1140097a0 tx=0x7fa11402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1e1163639a5320 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.109+0000 7fa129d9b640 1 -- 172.21.15.53:0/3214067140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11402d040 con 0x7fa12c0c90b0 2023-10-21T16:18:28.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.109+0000 7fa129d9b640 1 -- 172.21.15.53:0/3214067140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114031810 con 0x7fa12c0c90b0 2023-10-21T16:18:28.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.109+0000 7fa129d9b640 1 -- 172.21.15.53:0/3214067140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114031ba0 con 0x7fa12c0c90b0 2023-10-21T16:18:28.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 -- 172.21.15.53:0/3214067140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0c90b0 msgr2=0x7fa12c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0adc60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa1140097a0 tx=0x7fa11402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 -- 172.21.15.53:0/3214067140 shutdown_connections 2023-10-21T16:18:28.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0adc60 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 --2- 172.21.15.53:0/3214067140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 -- 172.21.15.53:0/3214067140 >> 172.21.15.53:0/3214067140 conn(0x7fa12c06a380 msgr2=0x7fa12c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:28.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 -- 172.21.15.53:0/3214067140 shutdown_connections 2023-10-21T16:18:28.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.110+0000 7fa130d65640 1 -- 172.21.15.53:0/3214067140 wait complete. 2023-10-21T16:18:28.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.112+0000 7fa130d65640 1 Processor -- start 2023-10-21T16:18:28.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.112+0000 7fa130d65640 1 -- start start 2023-10-21T16:18:28.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c56a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0c4160 con 0x7fa12c0c90b0 2023-10-21T16:18:28.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa130d65640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c14eb70 con 0x7fa12c15ab40 2023-10-21T16:18:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa123fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa123fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51052/0 (socket says 172.21.15.53:51052) 2023-10-21T16:18:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.113+0000 7fa123fff640 1 -- 172.21.15.53:0/2803885803 learned_addr learned my addr 172.21.15.53:0/2803885803 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa12ad9d640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 -- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 msgr2=0x7fa12c1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa12a59c640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c1596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 -- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 msgr2=0x7fa12c0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 -- 172.21.15.53:0/2803885803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009400 con 0x7fa12c0cae70 2023-10-21T16:18:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa12ad9d640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa12a59c640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa123fff640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa114000c00 tx=0x7fa1140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11403ad60 con 0x7fa12c0cae70 2023-10-21T16:18:28.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1140443f0 con 0x7fa12c0cae70 2023-10-21T16:18:28.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140447a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.114+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c0c37c0 con 0x7fa12c0cae70 2023-10-21T16:18:28.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.115+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c0c3d80 con 0x7fa12c0cae70 2023-10-21T16:18:28.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.116+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c0c42a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.117+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa11404d450 con 0x7fa12c0cae70 2023-10-21T16:18:28.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.118+0000 7fa122ffd640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.118+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fa1140abf80 con 0x7fa12c0cae70 2023-10-21T16:18:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.118+0000 7fa12a59c640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.120+0000 7fa12a59c640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 0x7fa10c06d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.122+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa12c0c42a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.369+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa12c0c42a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.371+0000 7fa122ffd640 1 -- 172.21.15.53:0/2803885803 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fa12c0c42a0 con 0x7fa12c0cae70 2023-10-21T16:18:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:28.373 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:28.374 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:28.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 msgr2=0x7fa10c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 0x7fa10c06d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa11c006fd0 tx=0x7fa11c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:28.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 msgr2=0x7fa12c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa114000c00 tx=0x7fa1140046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 shutdown_connections 2023-10-21T16:18:28.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa10c06af00 0x7fa10c06d3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa12c15ab40 0x7fa12c1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa12c0cae70 0x7fa12c159180 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 --2- 172.21.15.53:0/2803885803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa12c0c90b0 0x7fa12c0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.374+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 >> 172.21.15.53:0/2803885803 conn(0x7fa12c06a380 msgr2=0x7fa12c0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.375+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 shutdown_connections 2023-10-21T16:18:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.375+0000 7fa130d65640 1 -- 172.21.15.53:0/2803885803 wait complete. 2023-10-21T16:18:28.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:28.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.617+0000 7f0b003c9640 1 Processor -- start 2023-10-21T16:18:28.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.617+0000 7f0b003c9640 1 -- start start 2023-10-21T16:18:28.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0b003c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0b003c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0af80cb3b0 0x7f0af815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0b003c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af815b2b0 con 0x7f0af815ab40 2023-10-21T16:18:28.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0b003c9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af815bf30 con 0x7f0af80cb3b0 2023-10-21T16:18:28.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0b003c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cb840 con 0x7f0af80c90b0 2023-10-21T16:18:28.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0afe165640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0af815ab40 0x7f0af80ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41912/0 (socket says 172.21.15.53:41912) 2023-10-21T16:18:28.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0afe165640 1 -- 172.21.15.53:0/1567036597 learned_addr learned my addr 172.21.15.53:0/1567036597 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:28.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.619+0000 7f0afe966640 1 --2- 172.21.15.53:0/1567036597 >> v2:172.21.15.53:3300/0 conn(0x7f0af80cb3b0 0x7f0af815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afd964640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afe966640 1 -- 172.21.15.53:0/1567036597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 msgr2=0x7f0af80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:28.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afe966640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af80cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afe966640 1 -- 172.21.15.53:0/1567036597 >> v1:172.21.15.53:6789/0 conn(0x7f0af815ab40 legacy=0x7f0af80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afe966640 1 -- 172.21.15.53:0/1567036597 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af815b590 con 0x7f0af80cb3b0 2023-10-21T16:18:28.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.620+0000 7f0afd163640 1 -- 172.21.15.53:0/1567036597 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820303914 0 0) 0x7f0af815b2b0 con 0x7f0af815ab40 2023-10-21T16:18:28.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.621+0000 7f0afe966640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af815e790 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d90 tx=0x7f0af002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88edf0b0d6f43c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0afd163640 1 -- 172.21.15.53:0/1567036597 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af002cc60 con 0x7f0af80cb3b0 2023-10-21T16:18:28.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0afd163640 1 -- 172.21.15.53:0/1567036597 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af002cdc0 con 0x7f0af80cb3b0 2023-10-21T16:18:28.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0afd163640 1 -- 172.21.15.53:0/1567036597 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0034660 con 0x7f0af80cb3b0 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 -- 172.21.15.53:0/1567036597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 msgr2=0x7f0af815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af815e790 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009d90 tx=0x7f0af002beb0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 -- 172.21.15.53:0/1567036597 shutdown_connections 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af815e790 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 --2- 172.21.15.53:0/1567036597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af80cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 -- 172.21.15.53:0/1567036597 >> 172.21.15.53:0/1567036597 conn(0x7f0af806a380 msgr2=0x7f0af80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:28.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 -- 172.21.15.53:0/1567036597 shutdown_connections 2023-10-21T16:18:28.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.622+0000 7f0b003c9640 1 -- 172.21.15.53:0/1567036597 wait complete. 2023-10-21T16:18:28.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.624+0000 7f0b003c9640 1 Processor -- start 2023-10-21T16:18:28.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.624+0000 7f0b003c9640 1 -- start start 2023-10-21T16:18:28.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af8159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0af815ab40 0x7f0af815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af8155640 con 0x7f0af80cb3b0 2023-10-21T16:18:28.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815a100 con 0x7f0af815ab40 2023-10-21T16:18:28.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.625+0000 7f0b003c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af814bb60 con 0x7f0af80c90b0 2023-10-21T16:18:28.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51062/0 (socket says 172.21.15.53:51062) 2023-10-21T16:18:28.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 -- 172.21.15.53:0/502428509 learned_addr learned my addr 172.21.15.53:0/502428509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:28.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 -- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 msgr2=0x7f0af8159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af8159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 -- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0af815ab40 msgr2=0x7f0af815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afe966640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0af815ab40 0x7f0af815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0af815ab40 0x7f0af815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 -- 172.21.15.53:0/502428509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af0009a70 con 0x7f0af80cb3b0 2023-10-21T16:18:28.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.626+0000 7f0afd964640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800d800 tx=0x7f0ae800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.627+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800b6c0 con 0x7f0af80cb3b0 2023-10-21T16:18:28.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.627+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae800b820 con 0x7f0af80cb3b0 2023-10-21T16:18:28.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.627+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800bbb0 con 0x7f0af80cb3b0 2023-10-21T16:18:28.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.627+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af814c190 con 0x7f0af80cb3b0 2023-10-21T16:18:28.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.627+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af8154ff0 con 0x7f0af80cb3b0 2023-10-21T16:18:28.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.628+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8067490 con 0x7f0af80cb3b0 2023-10-21T16:18:28.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.629+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0ae800bd10 con 0x7f0af80cb3b0 2023-10-21T16:18:28.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.630+0000 7f0aeeffd640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:28.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.630+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f0ae808a0e0 con 0x7f0af80cb3b0 2023-10-21T16:18:28.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.631+0000 7f0afe165640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:28.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.632+0000 7f0afe165640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 0x7f0adc06d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0af4007fd0 tx=0x7f0af4007960 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:28.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.634+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ae8054c50 con 0x7f0af80cb3b0 2023-10-21T16:18:28.829 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:28.827+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:28.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.871+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0af8155780 con 0x7f0af80cb3b0 2023-10-21T16:18:28.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.872+0000 7f0aeeffd640 1 -- 172.21.15.53:0/502428509 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f0ae8058270 con 0x7f0af80cb3b0 2023-10-21T16:18:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:28.875 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:04.863937+0000","modified":"2023-10-21T16:18:07.895846+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35624":{"gid":35624,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/797303097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":797303097},{"type":"v1","addr":"172.21.15.53:6835","nonce":797303097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-21T16:18:28.875 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 44 2023-10-21T16:18:28.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 msgr2=0x7f0adc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 0x7f0adc06d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0af4007fd0 tx=0x7f0af4007960 comp rx=0 tx=0).stop 2023-10-21T16:18:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 msgr2=0x7f0af814b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800d800 tx=0x7f0ae800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 shutdown_connections 2023-10-21T16:18:28.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0af815ab40 0x7f0af815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af80cb3b0 0x7f0af814b620 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 --2- 172.21.15.53:0/502428509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0af80c90b0 0x7f0af8159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 >> 172.21.15.53:0/502428509 conn(0x7f0af806a380 msgr2=0x7f0af80ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:28.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 shutdown_connections 2023-10-21T16:18:28.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:28.875+0000 7f0b003c9640 1 -- 172.21.15.53:0/502428509 wait complete. 2023-10-21T16:18:28.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:29.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.126+0000 7fb6a255a640 1 Processor -- start 2023-10-21T16:18:29.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.126+0000 7fb6a255a640 1 -- start start 2023-10-21T16:18:29.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb69c071190 0x7fb69c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69c06ad00 0x7fb69c06b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0c9850 con 0x7fb69c06ad00 2023-10-21T16:18:29.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c06a870 con 0x7fb69c071190 2023-10-21T16:18:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a255a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c15e280 con 0x7fb69c070db0 2023-10-21T16:18:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a0af7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69c06ad00 0x7fb69c06b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a0af7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69c06ad00 0x7fb69c06b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51076/0 (socket says 172.21.15.53:51076) 2023-10-21T16:18:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.128+0000 7fb6a0af7640 1 -- 172.21.15.53:0/1329430237 learned_addr learned my addr 172.21.15.53:0/1329430237 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:29.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb693fff640 1 --2- 172.21.15.53:0/1329430237 >> v2:172.21.15.124:3300/0 conn(0x7fb69c071190 0x7fb69c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb69bfff640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb6a0af7640 1 -- 172.21.15.53:0/1329430237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 msgr2=0x7fb69c15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb6a0af7640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c15dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb6a0af7640 1 -- 172.21.15.53:0/1329430237 >> v2:172.21.15.124:3300/0 conn(0x7fb69c071190 msgr2=0x7fb69c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb6a0af7640 1 --2- 172.21.15.53:0/1329430237 >> v2:172.21.15.124:3300/0 conn(0x7fb69c071190 0x7fb69c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.129+0000 7fb6a0af7640 1 -- 172.21.15.53:0/1329430237 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c06b6b0 con 0x7fb69c06ad00 2023-10-21T16:18:29.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.130+0000 7fb6a0af7640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c06b170 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb69c071c00 tx=0x7fb69402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7c5c9d1f581f5f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb69b7fe640 1 -- 172.21.15.53:0/1329430237 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69402cb00 con 0x7fb69c06ad00 2023-10-21T16:18:29.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb69b7fe640 1 -- 172.21.15.53:0/1329430237 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69402cc60 con 0x7fb69c06ad00 2023-10-21T16:18:29.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb69b7fe640 1 -- 172.21.15.53:0/1329430237 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940343f0 con 0x7fb69c06ad00 2023-10-21T16:18:29.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 -- 172.21.15.53:0/1329430237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 msgr2=0x7fb69c06b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c06b170 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb69c071c00 tx=0x7fb69402bf60 comp rx=0 tx=0).stop 2023-10-21T16:18:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 -- 172.21.15.53:0/1329430237 shutdown_connections 2023-10-21T16:18:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c06b170 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 --2- 172.21.15.53:0/1329430237 >> v2:172.21.15.124:3300/0 conn(0x7fb69c071190 0x7fb69c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 --2- 172.21.15.53:0/1329430237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c15dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.131+0000 7fb6a255a640 1 -- 172.21.15.53:0/1329430237 >> 172.21.15.53:0/1329430237 conn(0x7fb69c0b1480 msgr2=0x7fb69c06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.132+0000 7fb6a255a640 1 -- 172.21.15.53:0/1329430237 shutdown_connections 2023-10-21T16:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.132+0000 7fb6a255a640 1 -- 172.21.15.53:0/1329430237 wait complete. 2023-10-21T16:18:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.134+0000 7fb6a255a640 1 Processor -- start 2023-10-21T16:18:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.134+0000 7fb6a255a640 1 -- start start 2023-10-21T16:18:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69c071190 0x7fb69c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c4700 con 0x7fb69c06ad00 2023-10-21T16:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1f5500 con 0x7fb69c071190 2023-10-21T16:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a255a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c3a80 con 0x7fb69c070db0 2023-10-21T16:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51092/0 (socket says 172.21.15.53:51092) 2023-10-21T16:18:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 -- 172.21.15.53:0/542338074 learned_addr learned my addr 172.21.15.53:0/542338074 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb693fff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 -- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 msgr2=0x7fb69c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c0b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb69bfff640 1 -- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69c071190 msgr2=0x7fb69c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.135+0000 7fb6a0af7640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69c071190 0x7fb69c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb69bfff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69c071190 0x7fb69c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb69bfff640 1 -- 172.21.15.53:0/542338074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009b30 con 0x7fb69c06ad00 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb69bfff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00d800 tx=0x7fb68c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0096a0 con 0x7fb69c06ad00 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c009800 con 0x7fb69c06ad00 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c009b90 con 0x7fb69c06ad00 2023-10-21T16:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.136+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c0c3d60 con 0x7fb69c06ad00 2023-10-21T16:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.137+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c0c4320 con 0x7fb69c06ad00 2023-10-21T16:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.138+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0c4970 con 0x7fb69c06ad00 2023-10-21T16:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.139+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb68c009cf0 con 0x7fb69c06ad00 2023-10-21T16:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.140+0000 7fb6997fa640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 0x7fb67c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.140+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fb68c089fe0 con 0x7fb69c06ad00 2023-10-21T16:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.141+0000 7fb693fff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 0x7fb67c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.142+0000 7fb693fff640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 0x7fb67c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb684006fd0 tx=0x7fb684008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.144+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb68c0bcca0 con 0x7fb69c06ad00 2023-10-21T16:18:29.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.344+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb69c0cd5c0 con 0x7fb69c06ad00 2023-10-21T16:18:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.347+0000 7fb6997fa640 1 -- 172.21.15.53:0/542338074 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17924 (secure 0 0 0) 0x7fb68c058640 con 0x7fb69c06ad00 2023-10-21T16:18:29.350 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:29.351 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":106,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:25.676361+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:02.821177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:03.863098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2451852009":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/600599962":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2629801108":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2272255357":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/1235374158":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6833/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6832/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/824950480":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6833/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/2324608475":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3280078422":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6832/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3826369357":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3225172013":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/539630491":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/798725635":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3415687515":"2023-10-21T17:18:12.553813+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 msgr2=0x7fb67c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 0x7fb67c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb684006fd0 tx=0x7fb684008040 comp rx=0 tx=0).stop 2023-10-21T16:18:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 msgr2=0x7fb69c1f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00d800 tx=0x7fb68c00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 shutdown_connections 2023-10-21T16:18:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb67c06af00 0x7fb67c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb69c071190 0x7fb69c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb69c070db0 0x7fb69c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 --2- 172.21.15.53:0/542338074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69c06ad00 0x7fb69c1f4fc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 >> 172.21.15.53:0/542338074 conn(0x7fb69c0b1480 msgr2=0x7fb69c0ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.350+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 shutdown_connections 2023-10-21T16:18:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.351+0000 7fb6a255a640 1 -- 172.21.15.53:0/542338074 wait complete. 2023-10-21T16:18:29.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:18:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.599+0000 7fb32c790640 1 Processor -- start 2023-10-21T16:18:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.599+0000 7fb32c790640 1 -- start start 2023-10-21T16:18:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32c790640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb3240c9cf0 0x7fb32406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32c790640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32415b310 con 0x7fb32415aa00 2023-10-21T16:18:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32c790640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32406d290 con 0x7fb32406c3a0 2023-10-21T16:18:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32c790640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32406e0a0 con 0x7fb3240c9cf0 2023-10-21T16:18:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32ad2d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb32415aa00 0x7fb3240b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41926/0 (socket says 172.21.15.53:41926) 2023-10-21T16:18:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.601+0000 7fb32ad2d640 1 -- 172.21.15.53:0/886385763 learned_addr learned my addr 172.21.15.53:0/886385763 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.602+0000 7fb329d2b640 1 --2- 172.21.15.53:0/886385763 >> v2:172.21.15.124:3300/0 conn(0x7fb3240c9cf0 0x7fb32406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.602+0000 7fb32952a640 1 -- 172.21.15.53:0/886385763 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939741915 0 0) 0x7fb31c001110 con 0x7fb32415aa00 2023-10-21T16:18:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.602+0000 7fb32952a640 1 -- 172.21.15.53:0/886385763 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb310003520 con 0x7fb32415aa00 2023-10-21T16:18:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.602+0000 7fb329d2b640 1 -- 172.21.15.53:0/886385763 >> v1:172.21.15.124:6789/0 conn(0x7fb32406c3a0 legacy=0x7fb32406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.603+0000 7fb329d2b640 1 -- 172.21.15.53:0/886385763 >> v1:172.21.15.53:6789/0 conn(0x7fb32415aa00 legacy=0x7fb3240b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.603+0000 7fb329d2b640 1 -- 172.21.15.53:0/886385763 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240c9860 con 0x7fb3240c9cf0 2023-10-21T16:18:29.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.604+0000 7fb329d2b640 1 --2- 172.21.15.53:0/886385763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3240c9cf0 0x7fb32406db60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb3140097a0 tx=0x7fb31402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f604178cb53fc43f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.604+0000 7fb32952a640 1 -- 172.21.15.53:0/886385763 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314030610 con 0x7fb3240c9cf0 2023-10-21T16:18:29.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.604+0000 7fb32952a640 1 -- 172.21.15.53:0/886385763 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314030770 con 0x7fb3240c9cf0 2023-10-21T16:18:29.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.604+0000 7fb32952a640 1 -- 172.21.15.53:0/886385763 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314030b00 con 0x7fb3240c9cf0 2023-10-21T16:18:29.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 -- 172.21.15.53:0/886385763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3240c9cf0 msgr2=0x7fb32406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 --2- 172.21.15.53:0/886385763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3240c9cf0 0x7fb32406db60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb3140097a0 tx=0x7fb31402b960 comp rx=0 tx=0).stop 2023-10-21T16:18:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 -- 172.21.15.53:0/886385763 shutdown_connections 2023-10-21T16:18:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 --2- 172.21.15.53:0/886385763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3240c9cf0 0x7fb32406db60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 -- 172.21.15.53:0/886385763 >> 172.21.15.53:0/886385763 conn(0x7fb32406a5d0 msgr2=0x7fb3240b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:29.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 -- 172.21.15.53:0/886385763 shutdown_connections 2023-10-21T16:18:29.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.605+0000 7fb32c790640 1 -- 172.21.15.53:0/886385763 wait complete. 2023-10-21T16:18:29.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.607+0000 7fb32c790640 1 Processor -- start 2023-10-21T16:18:29.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.607+0000 7fb32c790640 1 -- start start 2023-10-21T16:18:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240c9cf0 0x7fb3241f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 0x7fb3240c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241588b0 con 0x7fb32406c3a0 2023-10-21T16:18:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241f24c0 con 0x7fb32415aa00 2023-10-21T16:18:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.608+0000 7fb32c790640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241f6b10 con 0x7fb3240c9cf0 2023-10-21T16:18:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51094/0 (socket says 172.21.15.53:51094) 2023-10-21T16:18:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 -- 172.21.15.53:0/3713099782 learned_addr learned my addr 172.21.15.53:0/3713099782 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb329d2b640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240c9cf0 0x7fb3241f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32ad2d640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 0x7fb3240c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 -- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240c9cf0 msgr2=0x7fb3241f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240c9cf0 0x7fb3241f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 -- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 msgr2=0x7fb3240c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 0x7fb3240c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32a52c640 1 -- 172.21.15.53:0/3713099782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb314009400 con 0x7fb32406c3a0 2023-10-21T16:18:29.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.609+0000 7fb32ad2d640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 0x7fb3240c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:29.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb32a52c640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c870 tx=0x7fb32000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320003b70 con 0x7fb32406c3a0 2023-10-21T16:18:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320003cf0 con 0x7fb32406c3a0 2023-10-21T16:18:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3241f5d50 con 0x7fb32406c3a0 2023-10-21T16:18:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32000aa90 con 0x7fb32406c3a0 2023-10-21T16:18:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.610+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241f6310 con 0x7fb32406c3a0 2023-10-21T16:18:29.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.612+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32404e160 con 0x7fb32406c3a0 2023-10-21T16:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.613+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb32000abf0 con 0x7fb32406c3a0 2023-10-21T16:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.613+0000 7fb31affd640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 0x7fb30806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.614+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fb320089a40 con 0x7fb32406c3a0 2023-10-21T16:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.614+0000 7fb329d2b640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 0x7fb30806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.615+0000 7fb329d2b640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 0x7fb30806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb314036000 tx=0x7fb314008030 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.618+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3200545b0 con 0x7fb32406c3a0 2023-10-21T16:18:29.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.866+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb32415a0c0 con 0x7fb32406c3a0 2023-10-21T16:18:29.869 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:18:29.867+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:18:29.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.871+0000 7fb31affd640 1 -- 172.21.15.53:0/3713099782 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb320003e60 con 0x7fb32406c3a0 2023-10-21T16:18:29.873 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:18:29.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 msgr2=0x7fb30806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 0x7fb30806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb314036000 tx=0x7fb314008030 comp rx=0 tx=0).stop 2023-10-21T16:18:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 msgr2=0x7fb3240c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c870 tx=0x7fb32000cd30 comp rx=0 tx=0).stop 2023-10-21T16:18:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 shutdown_connections 2023-10-21T16:18:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb30806af00 0x7fb30806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415aa00 0x7fb3240c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240c9cf0 0x7fb3241f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 --2- 172.21.15.53:0/3713099782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32406c3a0 0x7fb3240c46c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:29.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 >> 172.21.15.53:0/3713099782 conn(0x7fb32406a5d0 msgr2=0x7fb32406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:29.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.874+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 shutdown_connections 2023-10-21T16:18:29.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:29.875+0000 7fb32c790640 1 -- 172.21.15.53:0/3713099782 wait complete. 2023-10-21T16:18:29.887 INFO:tasks.ceph.mds.a.smithi053.stderr: -2820> 2023-10-21T16:18:06.329+0000 7fa0d78d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:18:29.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:18:29.901 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:29.898+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:29.944 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:18:30.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.098+0000 7f907cc2d640 1 Processor -- start 2023-10-21T16:18:30.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.099+0000 7f907cc2d640 1 -- start start 2023-10-21T16:18:30.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f907cc2d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90780d03f0 0x7f90780c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f907cc2d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f90780cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f907cc2d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90781618b0 con 0x7f9078160280 2023-10-21T16:18:30.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f907cc2d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9078161350 con 0x7f90780d03f0 2023-10-21T16:18:30.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f907cc2d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b14c0 con 0x7f9078072170 2023-10-21T16:18:30.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f9075d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90780d03f0 0x7f90780c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f9075d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90780d03f0 0x7f90780c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51108/0 (socket says 172.21.15.53:51108) 2023-10-21T16:18:30.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.100+0000 7f9075d9b640 1 -- 172.21.15.53:0/748496771 learned_addr learned my addr 172.21.15.53:0/748496771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:30.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.101+0000 7f9076d9d640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f90780cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.101+0000 7f9075d9b640 1 -- 172.21.15.53:0/748496771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 msgr2=0x7f90780cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.101+0000 7f9075d9b640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f90780cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.101+0000 7f9075d9b640 1 -- 172.21.15.53:0/748496771 >> v1:172.21.15.124:6789/0 conn(0x7f9078160280 legacy=0x7f90780cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.101+0000 7f9075d9b640 1 -- 172.21.15.53:0/748496771 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9078161540 con 0x7f90780d03f0 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.102+0000 7f9075d9b640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90780d03f0 0x7f90780c9f90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9060009870 tx=0x7f906002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e2703cbce4643b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.102+0000 7f907559a640 1 -- 172.21.15.53:0/748496771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906002f830 con 0x7f90780d03f0 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.102+0000 7f907559a640 1 -- 172.21.15.53:0/748496771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906002f990 con 0x7f90780d03f0 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.102+0000 7f907559a640 1 -- 172.21.15.53:0/748496771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906002fd20 con 0x7f90780d03f0 2023-10-21T16:18:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 -- 172.21.15.53:0/748496771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90780d03f0 msgr2=0x7f90780c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90780d03f0 0x7f90780c9f90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9060009870 tx=0x7f906002ba10 comp rx=0 tx=0).stop 2023-10-21T16:18:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 -- 172.21.15.53:0/748496771 shutdown_connections 2023-10-21T16:18:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f90780cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 --2- 172.21.15.53:0/748496771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90780d03f0 0x7f90780c9f90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 -- 172.21.15.53:0/748496771 >> 172.21.15.53:0/748496771 conn(0x7f907806b250 msgr2=0x7f907806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 -- 172.21.15.53:0/748496771 shutdown_connections 2023-10-21T16:18:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.103+0000 7f907cc2d640 1 -- 172.21.15.53:0/748496771 wait complete. 2023-10-21T16:18:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.105+0000 7f907cc2d640 1 Processor -- start 2023-10-21T16:18:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.105+0000 7f907cc2d640 1 -- start start 2023-10-21T16:18:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f9078157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90780d03f0 0x7f90780ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078158320 con 0x7f9078160280 2023-10-21T16:18:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780ac1f0 con 0x7f90780d03f0 2023-10-21T16:18:30.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f907cc2d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780ac4d0 con 0x7f9078072170 2023-10-21T16:18:30.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.106+0000 7f9076d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51124/0 (socket says 172.21.15.53:51124) 2023-10-21T16:18:30.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 -- 172.21.15.53:0/600790918 learned_addr learned my addr 172.21.15.53:0/600790918 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:30.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f907659c640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f9078157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9075d9b640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90780d03f0 0x7f90780ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 -- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 msgr2=0x7f9078157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f9078157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 -- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90780d03f0 msgr2=0x7f90780ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90780d03f0 0x7f90780ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 -- 172.21.15.53:0/600790918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600094d0 con 0x7f9078160280 2023-10-21T16:18:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.107+0000 7f9076d9d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f907000d5d0 tx=0x7f907000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.108+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700125f0 con 0x7f9078160280 2023-10-21T16:18:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.108+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9070012750 con 0x7f9078160280 2023-10-21T16:18:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.108+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070012b00 con 0x7f9078160280 2023-10-21T16:18:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.108+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780ac7b0 con 0x7f9078160280 2023-10-21T16:18:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.108+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90780acd70 con 0x7f9078160280 2023-10-21T16:18:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.109+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90780acf60 con 0x7f9078160280 2023-10-21T16:18:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.110+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9070012c60 con 0x7f9078160280 2023-10-21T16:18:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.111+0000 7f906effd640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 0x7f905806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.111+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6890+0+0 (secure 0 0 0) 0x7f907008a0b0 con 0x7f9078160280 2023-10-21T16:18:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.112+0000 7f907659c640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 0x7f905806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.113+0000 7f907659c640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 0x7f905806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f90680012e0 tx=0x7f9068009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.115+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9070054c60 con 0x7f9078160280 2023-10-21T16:18:30.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.358+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f907804e160 con 0x7f9078160280 2023-10-21T16:18:30.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.874+0000 7f906effd640 1 -- 172.21.15.53:0/600790918 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9070058280 con 0x7f9078160280 2023-10-21T16:18:30.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 msgr2=0x7f905806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 0x7f905806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f90680012e0 tx=0x7f9068009040 comp rx=0 tx=0).stop 2023-10-21T16:18:30.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 msgr2=0x7f90780b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:30.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f907000d5d0 tx=0x7f907000da90 comp rx=0 tx=0).stop 2023-10-21T16:18:30.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 shutdown_connections 2023-10-21T16:18:30.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f905806af00 0x7f905806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9078160280 0x7f90780b24e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90780d03f0 0x7f90780ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 --2- 172.21.15.53:0/600790918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9078072170 0x7f9078157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:30.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 >> 172.21.15.53:0/600790918 conn(0x7f907806b250 msgr2=0x7f90780af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:30.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 shutdown_connections 2023-10-21T16:18:30.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:30.878+0000 7f907cc2d640 1 -- 172.21.15.53:0/600790918 wait complete. 2023-10-21T16:18:30.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:18:30.993 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:30.990+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:31.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.191+0000 7f2a13ecf640 1 Processor -- start 2023-10-21T16:18:31.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.192+0000 7f2a13ecf640 1 -- start start 2023-10-21T16:18:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.193+0000 7f2a13ecf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c160210 0x7f2a0c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.193+0000 7f2a13ecf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a0c1605f0 0x7f2a0c070790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.193+0000 7f2a13ecf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c06d1a0 con 0x7f2a0c0caa80 2023-10-21T16:18:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.193+0000 7f2a13ecf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c0cc7a0 con 0x7f2a0c1605f0 2023-10-21T16:18:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.193+0000 7f2a13ecf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0ccfb0 con 0x7f2a0c160210 2023-10-21T16:18:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1146a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2a0c0caa80 0x7f2a0c0cdbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41940/0 (socket says 172.21.15.53:41940) 2023-10-21T16:18:31.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1146a640 1 -- 172.21.15.53:0/1750568717 learned_addr learned my addr 172.21.15.53:0/1750568717 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1246c640 1 --2- 172.21.15.53:0/1750568717 >> v2:172.21.15.53:3300/0 conn(0x7f2a0c1605f0 0x7f2a0c070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a11c6b640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c160210 0x7f2a0c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a10c69640 1 -- 172.21.15.53:0/1750568717 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849298231 0 0) 0x7f2a0c06d1a0 con 0x7f2a0c0caa80 2023-10-21T16:18:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a10c69640 1 -- 172.21.15.53:0/1750568717 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f8003520 con 0x7f2a0c0caa80 2023-10-21T16:18:31.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1246c640 1 -- 172.21.15.53:0/1750568717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c160210 msgr2=0x7f2a0c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1246c640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c160210 0x7f2a0c0c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1246c640 1 -- 172.21.15.53:0/1750568717 >> v1:172.21.15.53:6789/0 conn(0x7f2a0c0caa80 legacy=0x7f2a0c0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.194+0000 7f2a1246c640 1 -- 172.21.15.53:0/1750568717 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0c9760 con 0x7f2a0c1605f0 2023-10-21T16:18:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.196+0000 7f2a1246c640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c1605f0 0x7f2a0c070790 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009d90 tx=0x7f2a0402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e5de27b322c5e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:31.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.196+0000 7f2a10c69640 1 -- 172.21.15.53:0/1750568717 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0402cc50 con 0x7f2a0c1605f0 2023-10-21T16:18:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.196+0000 7f2a10c69640 1 -- 172.21.15.53:0/1750568717 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0402cdb0 con 0x7f2a0c1605f0 2023-10-21T16:18:31.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.196+0000 7f2a10c69640 1 -- 172.21.15.53:0/1750568717 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04034660 con 0x7f2a0c1605f0 2023-10-21T16:18:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 -- 172.21.15.53:0/1750568717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c1605f0 msgr2=0x7f2a0c070790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c1605f0 0x7f2a0c070790 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009d90 tx=0x7f2a0402bea0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 -- 172.21.15.53:0/1750568717 shutdown_connections 2023-10-21T16:18:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c1605f0 0x7f2a0c070790 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/1750568717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c160210 0x7f2a0c0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 -- 172.21.15.53:0/1750568717 >> 172.21.15.53:0/1750568717 conn(0x7f2a0c0cebd0 msgr2=0x7f2a0c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 -- 172.21.15.53:0/1750568717 shutdown_connections 2023-10-21T16:18:31.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.197+0000 7f2a13ecf640 1 -- 172.21.15.53:0/1750568717 wait complete. 2023-10-21T16:18:31.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.199+0000 7f2a13ecf640 1 Processor -- start 2023-10-21T16:18:31.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.199+0000 7f2a13ecf640 1 -- start start 2023-10-21T16:18:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 0x7f2a0c1f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 0x7f2a0c1f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1f6520 con 0x7f2a0c0caa80 2023-10-21T16:18:31.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1f2a80 con 0x7f2a0c160210 2023-10-21T16:18:31.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.200+0000 7f2a13ecf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1f67c0 con 0x7f2a0c1605f0 2023-10-21T16:18:31.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51138/0 (socket says 172.21.15.53:51138) 2023-10-21T16:18:31.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 -- 172.21.15.53:0/4127721915 learned_addr learned my addr 172.21.15.53:0/4127721915 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a1246c640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 0x7f2a0c1f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a1146a640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 0x7f2a0c1f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 -- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 msgr2=0x7f2a0c1f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 0x7f2a0c1f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 -- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 msgr2=0x7f2a0c1f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 0x7f2a0c1f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 -- 172.21.15.53:0/4127721915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009a70 con 0x7f2a0c0caa80 2023-10-21T16:18:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a1246c640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 0x7f2a0c1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:31.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a1146a640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 0x7f2a0c1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.201+0000 7f2a11c6b640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800d800 tx=0x7f2a0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.202+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800b6c0 con 0x7f2a0c0caa80 2023-10-21T16:18:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.202+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0800b820 con 0x7f2a0c0caa80 2023-10-21T16:18:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.202+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c1f6aa0 con 0x7f2a0c0caa80 2023-10-21T16:18:31.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.202+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800bbb0 con 0x7f2a0c0caa80 2023-10-21T16:18:31.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.202+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c1f2d40 con 0x7f2a0c0caa80 2023-10-21T16:18:31.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.203+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c04e160 con 0x7f2a0c0caa80 2023-10-21T16:18:31.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.204+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2a0800bd10 con 0x7f2a0c0caa80 2023-10-21T16:18:31.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.205+0000 7f2a027fc640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 0x7f29f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:31.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.205+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6890+0+0 (secure 0 0 0) 0x7f2a0808a070 con 0x7f2a0c0caa80 2023-10-21T16:18:31.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.206+0000 7f2a1146a640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 0x7f29f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:31.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.207+0000 7f2a1146a640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 0x7f29f006d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005880 tx=0x7f29fc009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:31.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.209+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a08054c90 con 0x7f2a0c0caa80 2023-10-21T16:18:31.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:31.410+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a0c0b1280 con 0x7f2a0c0caa80 2023-10-21T16:18:32.112 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:32.110+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.326+0000 7f2a027fc640 1 -- 172.21.15.53:0/4127721915 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2a080582b0 con 0x7f2a0c0caa80 2023-10-21T16:18:32.329 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:18:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 msgr2=0x7f29f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 0x7f29f006d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005880 tx=0x7f29fc009040 comp rx=0 tx=0).stop 2023-10-21T16:18:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 msgr2=0x7f2a0c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800d800 tx=0x7f2a0800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 shutdown_connections 2023-10-21T16:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f29f006af00 0x7f29f006d3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a0c1605f0 0x7f2a0c1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a0c160210 0x7f2a0c1f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 --2- 172.21.15.53:0/4127721915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c0caa80 0x7f2a0c155d80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.330+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 >> 172.21.15.53:0/4127721915 conn(0x7f2a0c0cebd0 msgr2=0x7f2a0c1e71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:32.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.331+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 shutdown_connections 2023-10-21T16:18:32.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.331+0000 7f2a13ecf640 1 -- 172.21.15.53:0/4127721915 wait complete. 2023-10-21T16:18:32.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:18:32.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.575+0000 7f73c544d640 1 Processor -- start 2023-10-21T16:18:32.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.575+0000 7f73c544d640 1 -- start start 2023-10-21T16:18:32.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.577+0000 7f73c544d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73c01599d0 0x7f73c00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.577+0000 7f73c544d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.577+0000 7f73c544d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c0159db0 con 0x7f73c00b3c50 2023-10-21T16:18:32.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.577+0000 7f73c544d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c0071fe0 con 0x7f73c01599d0 2023-10-21T16:18:32.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.577+0000 7f73c544d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00b44c0 con 0x7f73c00cb5e0 2023-10-21T16:18:32.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.578+0000 7f73bffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73c01599d0 0x7f73c00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:32.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.578+0000 7f73c4c4c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:32.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.578+0000 7f73bffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f73c01599d0 0x7f73c00cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:60826/0 (socket says 172.21.15.53:60826) 2023-10-21T16:18:32.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.578+0000 7f73bffff640 1 -- 172.21.15.53:0/4178999280 learned_addr learned my addr 172.21.15.53:0/4178999280 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:32.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.578+0000 7f73bffff640 1 -- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 msgr2=0x7f73c00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:32.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.579+0000 7f73bffff640 1 --2- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c00cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.579+0000 7f73bffff640 1 -- 172.21.15.53:0/4178999280 >> v1:172.21.15.124:6789/0 conn(0x7f73c00b3c50 legacy=0x7f73c00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.579+0000 7f73bffff640 1 -- 172.21.15.53:0/4178999280 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c00713d0 con 0x7f73c01599d0 2023-10-21T16:18:32.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.580+0000 7f73bffff640 1 --2- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00cdcd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d150 tx=0x7f73b002e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46a36e080cb84a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:32.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73bf7fe640 1 -- 172.21.15.53:0/4178999280 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000b0d0 con 0x7f73c01599d0 2023-10-21T16:18:32.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73bf7fe640 1 -- 172.21.15.53:0/4178999280 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000b230 con 0x7f73c01599d0 2023-10-21T16:18:32.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73bf7fe640 1 -- 172.21.15.53:0/4178999280 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0009600 con 0x7f73c01599d0 2023-10-21T16:18:32.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 -- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 msgr2=0x7f73c00cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 --2- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00cdcd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d150 tx=0x7f73b002e160 comp rx=0 tx=0).stop 2023-10-21T16:18:32.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 -- 172.21.15.53:0/4178999280 shutdown_connections 2023-10-21T16:18:32.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 --2- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 --2- 172.21.15.53:0/4178999280 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00cdcd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 -- 172.21.15.53:0/4178999280 >> 172.21.15.53:0/4178999280 conn(0x7f73c006c050 msgr2=0x7f73c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:32.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.581+0000 7f73c544d640 1 -- 172.21.15.53:0/4178999280 shutdown_connections 2023-10-21T16:18:32.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.582+0000 7f73c544d640 1 -- 172.21.15.53:0/4178999280 wait complete. 2023-10-21T16:18:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.584+0000 7f73c544d640 1 Processor -- start 2023-10-21T16:18:32.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.584+0000 7f73c544d640 1 -- start start 2023-10-21T16:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.584+0000 7f73c544d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73c544d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c01f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73c544d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73c544d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c01f2fd0 con 0x7f73c00b3c50 2023-10-21T16:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73c544d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00a9c70 con 0x7f73c01599d0 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73c544d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00a9f50 con 0x7f73c00cb5e0 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51140/0 (socket says 172.21.15.53:51140) 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 -- 172.21.15.53:0/624716615 learned_addr learned my addr 172.21.15.53:0/624716615 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73b77fe640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c01f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 -- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 msgr2=0x7f73c01f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c01f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 -- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 msgr2=0x7f73c00add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73bffff640 1 -- 172.21.15.53:0/624716615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b000bd40 con 0x7f73c00b3c50 2023-10-21T16:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.585+0000 7f73b77fe640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c01f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73bffff640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f73b0000f80 tx=0x7f73b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0036800 con 0x7f73c00b3c50 2023-10-21T16:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b0036980 con 0x7f73c00b3c50 2023-10-21T16:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0036d30 con 0x7f73c00b3c50 2023-10-21T16:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c00aa230 con 0x7f73c00b3c50 2023-10-21T16:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.586+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c00aa7f0 con 0x7f73c00b3c50 2023-10-21T16:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.587+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c0074150 con 0x7f73c00b3c50 2023-10-21T16:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.588+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f73b004a460 con 0x7f73c00b3c50 2023-10-21T16:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.589+0000 7f73bd7fa640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 0x7f73a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.590+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6452+0+0 (secure 0 0 0) 0x7f73b00acc70 con 0x7f73c00b3c50 2023-10-21T16:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.590+0000 7f73b77fe640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 0x7f73a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.591+0000 7f73b77fe640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 0x7f73a006d3f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73c00c4d20 tx=0x7f73a8009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.595+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73b0077af0 con 0x7f73c00b3c50 2023-10-21T16:18:32.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:32.801+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73c00cb200 con 0x7f73c00b3c50 2023-10-21T16:18:33.239 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:33.237+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.331+0000 7f73bd7fa640 1 -- 172.21.15.53:0/624716615 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7f73b007b110 con 0x7f73c00b3c50 2023-10-21T16:18:33.333 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.335+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 msgr2=0x7f73a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.335+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 0x7f73a006d3f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f73c00c4d20 tx=0x7f73a8009040 comp rx=0 tx=0).stop 2023-10-21T16:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.335+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 msgr2=0x7f73c00a9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.335+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f73b0000f80 tx=0x7f73b0004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:33.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 shutdown_connections 2023-10-21T16:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f73a006af30 0x7f73a006d3f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73c01599d0 0x7f73c00add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73c00cb5e0 0x7f73c01f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 --2- 172.21.15.53:0/624716615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73c00b3c50 0x7f73c00a9690 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.337+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 >> 172.21.15.53:0/624716615 conn(0x7f73c006c050 msgr2=0x7f73c00b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:33.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.339+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 shutdown_connections 2023-10-21T16:18:33.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.339+0000 7f73c544d640 1 -- 172.21.15.53:0/624716615 wait complete. 2023-10-21T16:18:33.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:18:33.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.572+0000 7fc0d7fff640 1 Processor -- start 2023-10-21T16:18:33.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.572+0000 7fc0d7fff640 1 -- start start 2023-10-21T16:18:33.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc0d80cb5e0 0x7fc0d80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 0x7fc0d80cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d7fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d8159db0 con 0x7fc0d81599d0 2023-10-21T16:18:33.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d7fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d8071fe0 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80b44c0 con 0x7fc0d80b3c50 2023-10-21T16:18:33.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0cffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc0d80cb5e0 0x7fc0d80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:33.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc0d81599d0 0x7fc0d80caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41956/0 (socket says 172.21.15.53:41956) 2023-10-21T16:18:33.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.574+0000 7fc0d6ffd640 1 -- 172.21.15.53:0/2194616821 learned_addr learned my addr 172.21.15.53:0/2194616821 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:33.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0d77fe640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 0x7fc0d80cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:33.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0cffff640 1 -- 172.21.15.53:0/2194616821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 msgr2=0x7fc0d80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:33.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0cffff640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 0x7fc0d80cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0cffff640 1 -- 172.21.15.53:0/2194616821 >> v1:172.21.15.53:6789/0 conn(0x7fc0d81599d0 legacy=0x7fc0d80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0cffff640 1 -- 172.21.15.53:0/2194616821 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80713d0 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0d67fc640 1 -- 172.21.15.53:0/2194616821 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940252774 0 0) 0x7fc0c8001110 con 0x7fc0d81599d0 2023-10-21T16:18:33.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.575+0000 7fc0d77fe640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 0x7fc0d80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:33.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.576+0000 7fc0cffff640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d80cdcd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00097a0 tx=0x7fc0c002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5014a16eb4938e2e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:33.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.576+0000 7fc0d67fc640 1 -- 172.21.15.53:0/2194616821 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c002d040 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.576+0000 7fc0d67fc640 1 -- 172.21.15.53:0/2194616821 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0031810 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.576+0000 7fc0d67fc640 1 -- 172.21.15.53:0/2194616821 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0031ba0 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 -- 172.21.15.53:0/2194616821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80cb5e0 msgr2=0x7fc0d80cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d80cdcd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00097a0 tx=0x7fc0c002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 -- 172.21.15.53:0/2194616821 shutdown_connections 2023-10-21T16:18:33.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80b3c50 0x7fc0d80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/2194616821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d80cdcd0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 -- 172.21.15.53:0/2194616821 >> 172.21.15.53:0/2194616821 conn(0x7fc0d806c050 msgr2=0x7fc0d806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:33.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 -- 172.21.15.53:0/2194616821 shutdown_connections 2023-10-21T16:18:33.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.577+0000 7fc0d7fff640 1 -- 172.21.15.53:0/2194616821 wait complete. 2023-10-21T16:18:33.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.579+0000 7fc0d7fff640 1 Processor -- start 2023-10-21T16:18:33.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.579+0000 7fc0d7fff640 1 -- start start 2023-10-21T16:18:33.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d81f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d81599d0 0x7fc0d8164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8165430 con 0x7fc0d80b3c50 2023-10-21T16:18:33.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8164e70 con 0x7fc0d80cb5e0 2023-10-21T16:18:33.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8165150 con 0x7fc0d81599d0 2023-10-21T16:18:33.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:33.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51158/0 (socket says 172.21.15.53:51158) 2023-10-21T16:18:33.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 -- 172.21.15.53:0/1797674260 learned_addr learned my addr 172.21.15.53:0/1797674260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:33.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 -- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d81599d0 msgr2=0x7fc0d8164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d81599d0 0x7fc0d8164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 -- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80cb5e0 msgr2=0x7fc0d81f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.580+0000 7fc0d6ffd640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d81f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0d6ffd640 1 -- 172.21.15.53:0/1797674260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009400 con 0x7fc0d80b3c50 2023-10-21T16:18:33.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0d6ffd640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c430 tx=0x7fc0c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:33.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80074e0 con 0x7fc0d80b3c50 2023-10-21T16:18:33.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8007660 con 0x7fc0d80b3c50 2023-10-21T16:18:33.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8004a20 con 0x7fc0d80b3c50 2023-10-21T16:18:33.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d80c5760 con 0x7fc0d80b3c50 2023-10-21T16:18:33.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.581+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d81f2ca0 con 0x7fc0d80b3c50 2023-10-21T16:18:33.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.582+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d80c5c80 con 0x7fc0d80b3c50 2023-10-21T16:18:33.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.583+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc0c8004b80 con 0x7fc0d80b3c50 2023-10-21T16:18:33.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.584+0000 7fc0cf7fe640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 0x7fc0b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:33.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.584+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fc0c8089460 con 0x7fc0d80b3c50 2023-10-21T16:18:33.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.585+0000 7fc0cffff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 0x7fc0b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:33.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.586+0000 7fc0cffff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 0x7fc0b806d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0036000 tx=0x7fc0c0030000 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:33.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.588+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0d80c5c80 con 0x7fc0d80b3c50 2023-10-21T16:18:33.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.793+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc0d80c5c80 con 0x7fc0d80b3c50 2023-10-21T16:18:33.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.800+0000 7fc0cf7fe640 1 -- 172.21.15.53:0/1797674260 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0d80c5c80 con 0x7fc0d80b3c50 2023-10-21T16:18:33.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 msgr2=0x7fc0b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 0x7fc0b806d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0036000 tx=0x7fc0c0030000 comp rx=0 tx=0).stop 2023-10-21T16:18:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 msgr2=0x7fc0d814f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:33.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c430 tx=0x7fc0c800c8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 shutdown_connections 2023-10-21T16:18:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fc0b806af00 0x7fc0b806d3c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d81599d0 0x7fc0d8164490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80cb5e0 0x7fc0d81f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 --2- 172.21.15.53:0/1797674260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d80b3c50 0x7fc0d814f770 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.802+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 >> 172.21.15.53:0/1797674260 conn(0x7fc0d806c050 msgr2=0x7fc0d815c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.803+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 shutdown_connections 2023-10-21T16:18:33.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:33.803+0000 7fc0d7fff640 1 -- 172.21.15.53:0/1797674260 wait complete. 2023-10-21T16:18:33.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:18:34.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.055+0000 7fadff53c640 1 Processor -- start 2023-10-21T16:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.055+0000 7fadff53c640 1 -- start start 2023-10-21T16:18:34.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadff53c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb740 0x7fadf815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadff53c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fadf80ce2b0 0x7fadf80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadff53c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf815b040 con 0x7fadf80cb360 2023-10-21T16:18:34.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadff53c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf815adc0 con 0x7fadf80ce2b0 2023-10-21T16:18:34.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadff53c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80cbba0 con 0x7fadf80cb740 2023-10-21T16:18:34.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadfe53a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fadf80cb360 0x7fadf815b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38156/0 (socket says 172.21.15.53:38156) 2023-10-21T16:18:34.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadfe53a640 1 -- 172.21.15.53:0/1382230268 learned_addr learned my addr 172.21.15.53:0/1382230268 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:34.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.056+0000 7fadfed3b640 1 --2- 172.21.15.53:0/1382230268 >> v2:172.21.15.53:3300/0 conn(0x7fadf80ce2b0 0x7fadf80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfdd39640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb740 0x7fadf815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfd538640 1 -- 172.21.15.53:0/1382230268 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574251333 0 0) 0x7fadf815b040 con 0x7fadf80cb360 2023-10-21T16:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfd538640 1 -- 172.21.15.53:0/1382230268 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4003520 con 0x7fadf80cb360 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfed3b640 1 -- 172.21.15.53:0/1382230268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb740 msgr2=0x7fadf815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfed3b640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb740 0x7fadf815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfed3b640 1 -- 172.21.15.53:0/1382230268 >> v1:172.21.15.53:6789/0 conn(0x7fadf80cb360 legacy=0x7fadf815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.057+0000 7fadfed3b640 1 -- 172.21.15.53:0/1382230268 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf815e8f0 con 0x7fadf80ce2b0 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.058+0000 7fadfed3b640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80ce2b0 0x7fadf80ce720 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fadf00097a0 tx=0x7fadf002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9995634856ff358a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadfd538640 1 -- 172.21.15.53:0/1382230268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf002d040 con 0x7fadf80ce2b0 2023-10-21T16:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadfd538640 1 -- 172.21.15.53:0/1382230268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0031810 con 0x7fadf80ce2b0 2023-10-21T16:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadfd538640 1 -- 172.21.15.53:0/1382230268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0031ba0 con 0x7fadf80ce2b0 2023-10-21T16:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 -- 172.21.15.53:0/1382230268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80ce2b0 msgr2=0x7fadf80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80ce2b0 0x7fadf80ce720 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fadf00097a0 tx=0x7fadf002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 -- 172.21.15.53:0/1382230268 shutdown_connections 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80ce2b0 0x7fadf80ce720 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 --2- 172.21.15.53:0/1382230268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb740 0x7fadf815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.059+0000 7fadff53c640 1 -- 172.21.15.53:0/1382230268 >> 172.21.15.53:0/1382230268 conn(0x7fadf80cd160 msgr2=0x7fadf80cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.060+0000 7fadff53c640 1 -- 172.21.15.53:0/1382230268 shutdown_connections 2023-10-21T16:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.060+0000 7fadff53c640 1 -- 172.21.15.53:0/1382230268 wait complete. 2023-10-21T16:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.062+0000 7fadff53c640 1 Processor -- start 2023-10-21T16:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.062+0000 7fadff53c640 1 -- start start 2023-10-21T16:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 0x7fadf8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadf80ce2b0 0x7fadf80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80c6500 con 0x7fadf80cb740 2023-10-21T16:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80b33a0 con 0x7fadf80ce2b0 2023-10-21T16:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadff53c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80b3680 con 0x7fadf80cb360 2023-10-21T16:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadfdd39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadfdd39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42576/0 (socket says 172.21.15.53:42576) 2023-10-21T16:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadfdd39640 1 -- 172.21.15.53:0/904933755 learned_addr learned my addr 172.21.15.53:0/904933755 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.064+0000 7fadfe53a640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 0x7fadf8152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfed3b640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadf80ce2b0 0x7fadf80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 -- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 msgr2=0x7fadf8152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 0x7fadf8152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 -- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadf80ce2b0 msgr2=0x7fadf80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadf80ce2b0 0x7fadf80b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 -- 172.21.15.53:0/904933755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0009400 con 0x7fadf80cb740 2023-10-21T16:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfe53a640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 0x7fadf8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:34.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadfdd39640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fade800d800 tx=0x7fade800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800b6c0 con 0x7fadf80cb740 2023-10-21T16:18:34.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade800b820 con 0x7fadf80cb740 2023-10-21T16:18:34.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.065+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800bbb0 con 0x7fadf80cb740 2023-10-21T16:18:34.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.066+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf80b3960 con 0x7fadf80cb740 2023-10-21T16:18:34.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.066+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf80b4160 con 0x7fadf80cb740 2023-10-21T16:18:34.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.067+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8074150 con 0x7fadf80cb740 2023-10-21T16:18:34.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.068+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fade800bd10 con 0x7fadf80cb740 2023-10-21T16:18:34.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.069+0000 7fadeeffd640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 0x7faddc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.070+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fade8089de0 con 0x7fadf80cb740 2023-10-21T16:18:34.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.070+0000 7fadfe53a640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 0x7faddc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.071+0000 7fadfe53a640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 0x7faddc06d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fadf4001560 tx=0x7fadf4009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.074+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fade8054c50 con 0x7fadf80cb740 2023-10-21T16:18:34.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.278+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fadf80d0430 con 0x7fadf80cb740 2023-10-21T16:18:34.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.283+0000 7fadeeffd640 1 -- 172.21.15.53:0/904933755 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7fade8058270 con 0x7fadf80cb740 2023-10-21T16:18:34.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 msgr2=0x7faddc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 0x7faddc06d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fadf4001560 tx=0x7fadf4009040 comp rx=0 tx=0).stop 2023-10-21T16:18:34.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 msgr2=0x7fadf80c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fade800d800 tx=0x7fade800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 shutdown_connections 2023-10-21T16:18:34.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7faddc06af00 0x7faddc06d3c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadf80ce2b0 0x7fadf80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadf80cb740 0x7fadf80c5a80 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 --2- 172.21.15.53:0/904933755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadf80cb360 0x7fadf8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 >> 172.21.15.53:0/904933755 conn(0x7fadf80cd160 msgr2=0x7fadf81536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:34.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 shutdown_connections 2023-10-21T16:18:34.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.286+0000 7fadff53c640 1 -- 172.21.15.53:0/904933755 wait complete. 2023-10-21T16:18:34.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:18:34.337 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:34.335+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.541+0000 7f964ae46640 1 Processor -- start 2023-10-21T16:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.541+0000 7f964ae46640 1 -- start start 2023-10-21T16:18:34.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.543+0000 7f964ae46640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96440afbe0 0x7f96440b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.543+0000 7f964ae46640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96440cb130 0x7f96440cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.543+0000 7f964ae46640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440cbad0 con 0x7f964406dfc0 2023-10-21T16:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.543+0000 7f964ae46640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96440b2620 con 0x7f96440afbe0 2023-10-21T16:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.543+0000 7f964ae46640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440b2060 con 0x7f96440cb130 2023-10-21T16:18:34.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96440afbe0 0x7f96440b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f96493e3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96440cb130 0x7f96440cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96440afbe0 0x7f96440b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59608/0 (socket says 172.21.15.53:59608) 2023-10-21T16:18:34.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 -- 172.21.15.53:0/504690576 learned_addr learned my addr 172.21.15.53:0/504690576 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:34.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 -- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96440cb130 msgr2=0x7f96440cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:34.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 --2- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96440cb130 0x7f96440cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 -- 172.21.15.53:0/504690576 >> v1:172.21.15.124:6789/0 conn(0x7f964406dfc0 legacy=0x7f9644070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.544+0000 7f9648be2640 1 -- 172.21.15.53:0/504690576 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440b2340 con 0x7f96440afbe0 2023-10-21T16:18:34.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.546+0000 7f9648be2640 1 --2- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440b15b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9638009870 tx=0x7f963802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3cb54f1c541cdb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.546+0000 7f96437fe640 1 -- 172.21.15.53:0/504690576 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638030840 con 0x7f96440afbe0 2023-10-21T16:18:34.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.547+0000 7f96437fe640 1 -- 172.21.15.53:0/504690576 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96380309a0 con 0x7f96440afbe0 2023-10-21T16:18:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.547+0000 7f96437fe640 1 -- 172.21.15.53:0/504690576 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9638030d30 con 0x7f96440afbe0 2023-10-21T16:18:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.547+0000 7f964ae46640 1 -- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 msgr2=0x7f96440b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.547+0000 7f964ae46640 1 --2- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440b15b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9638009870 tx=0x7f963802b980 comp rx=0 tx=0).stop 2023-10-21T16:18:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 -- 172.21.15.53:0/504690576 shutdown_connections 2023-10-21T16:18:34.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 --2- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96440cb130 0x7f96440cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 --2- 172.21.15.53:0/504690576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440b15b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 -- 172.21.15.53:0/504690576 >> 172.21.15.53:0/504690576 conn(0x7f964406a5d0 msgr2=0x7f96440b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:34.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 -- 172.21.15.53:0/504690576 shutdown_connections 2023-10-21T16:18:34.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.548+0000 7f964ae46640 1 -- 172.21.15.53:0/504690576 wait complete. 2023-10-21T16:18:34.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.550+0000 7f964ae46640 1 Processor -- start 2023-10-21T16:18:34.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.550+0000 7f964ae46640 1 -- start start 2023-10-21T16:18:34.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f964406dfc0 0x7f96440c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c2c50 con 0x7f96440cb130 2023-10-21T16:18:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c22f0 con 0x7f96440afbe0 2023-10-21T16:18:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f964ae46640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c2590 con 0x7f964406dfc0 2023-10-21T16:18:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f96493e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f96493e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42580/0 (socket says 172.21.15.53:42580) 2023-10-21T16:18:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f96493e3640 1 -- 172.21.15.53:0/43939533 learned_addr learned my addr 172.21.15.53:0/43939533 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f9648be2640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f964406dfc0 0x7f96440c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.552+0000 7f9643fff640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 -- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f964406dfc0 msgr2=0x7f96440c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f964406dfc0 0x7f96440c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 -- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 msgr2=0x7f96440c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 -- 172.21.15.53:0/43939533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380094d0 con 0x7f96440cb130 2023-10-21T16:18:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.553+0000 7f96493e3640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f963c00efc0 tx=0x7f963c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.554+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c014480 con 0x7f96440cb130 2023-10-21T16:18:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.554+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c00f040 con 0x7f96440cb130 2023-10-21T16:18:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.554+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c014820 con 0x7f96440cb130 2023-10-21T16:18:34.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.554+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440c2870 con 0x7f96440cb130 2023-10-21T16:18:34.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.554+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440c35b0 con 0x7f96440cb130 2023-10-21T16:18:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.555+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96440c2d90 con 0x7f96440cb130 2023-10-21T16:18:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.556+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f963c014980 con 0x7f96440cb130 2023-10-21T16:18:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.557+0000 7f96417fa640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 0x7f962806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:34.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.557+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f963c08a6d0 con 0x7f96440cb130 2023-10-21T16:18:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.558+0000 7f9648be2640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 0x7f962806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:34.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.559+0000 7f9648be2640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 0x7f962806d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9638036000 tx=0x7f9638008030 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:34.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.564+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96440c2d90 con 0x7f96440cb130 2023-10-21T16:18:34.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.761+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f96440c2d90 con 0x7f96440cb130 2023-10-21T16:18:34.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.768+0000 7f96417fa640 1 -- 172.21.15.53:0/43939533 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f96440c2d90 con 0x7f96440cb130 2023-10-21T16:18:34.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.771+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 msgr2=0x7f962806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.771+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 0x7f962806d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9638036000 tx=0x7f9638008030 comp rx=0 tx=0).stop 2023-10-21T16:18:34.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.771+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 msgr2=0x7f96441f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:34.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.771+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f963c00efc0 tx=0x7f963c00c490 comp rx=0 tx=0).stop 2023-10-21T16:18:34.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 shutdown_connections 2023-10-21T16:18:34.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f962806af00 0x7f962806d3c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96440cb130 0x7f96441f3040 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96440afbe0 0x7f96440c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 --2- 172.21.15.53:0/43939533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f964406dfc0 0x7f96440c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:34.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 >> 172.21.15.53:0/43939533 conn(0x7f964406a5d0 msgr2=0x7f964406c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:34.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 shutdown_connections 2023-10-21T16:18:34.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:34.772+0000 7f964ae46640 1 -- 172.21.15.53:0/43939533 wait complete. 2023-10-21T16:18:34.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-21T16:18:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.025+0000 7f6a0c4a2640 1 Processor -- start 2023-10-21T16:18:35.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.026+0000 7f6a0c4a2640 1 -- start start 2023-10-21T16:18:35.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.027+0000 7f6a0c4a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a04160940 0x7f6a040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.027+0000 7f6a0c4a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0406c890 0x7f6a0406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.027+0000 7f6a0c4a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a040c48f0 con 0x7f6a0415cd90 2023-10-21T16:18:35.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.027+0000 7f6a0c4a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a040c4d80 con 0x7f6a04160940 2023-10-21T16:18:35.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.027+0000 7f6a0c4a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040c3ce0 con 0x7f6a0406c890 2023-10-21T16:18:35.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a0a23e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6a0415cd90 0x7f6a040702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38166/0 (socket says 172.21.15.53:38166) 2023-10-21T16:18:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a0a23e640 1 -- 172.21.15.53:0/3032240816 learned_addr learned my addr 172.21.15.53:0/3032240816 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/3032240816 >> v2:172.21.15.53:3300/0 conn(0x7f6a04160940 0x7f6a040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a0aa3f640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0406c890 0x7f6a0406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a0923c640 1 -- 172.21.15.53:0/3032240816 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106418244 0 0) 0x7f6a040c48f0 con 0x7f6a0415cd90 2023-10-21T16:18:35.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a0923c640 1 -- 172.21.15.53:0/3032240816 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f0003520 con 0x7f6a0415cd90 2023-10-21T16:18:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a09a3d640 1 -- 172.21.15.53:0/3032240816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0406c890 msgr2=0x7f6a0406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0406c890 0x7f6a0406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a09a3d640 1 -- 172.21.15.53:0/3032240816 >> v1:172.21.15.53:6789/0 conn(0x7f6a0415cd90 legacy=0x7f6a040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.028+0000 7f6a09a3d640 1 -- 172.21.15.53:0/3032240816 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a040c3fc0 con 0x7f6a04160940 2023-10-21T16:18:35.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.030+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a04160940 0x7f6a040709e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8b05d9ba426c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.030+0000 7f6a0923c640 1 -- 172.21.15.53:0/3032240816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f402d040 con 0x7f6a04160940 2023-10-21T16:18:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.030+0000 7f6a0923c640 1 -- 172.21.15.53:0/3032240816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4031810 con 0x7f6a04160940 2023-10-21T16:18:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.030+0000 7f6a0923c640 1 -- 172.21.15.53:0/3032240816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4031ba0 con 0x7f6a04160940 2023-10-21T16:18:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/3032240816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a04160940 msgr2=0x7f6a040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a04160940 0x7f6a040709e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/3032240816 shutdown_connections 2023-10-21T16:18:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0406c890 0x7f6a0406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/3032240816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a04160940 0x7f6a040709e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/3032240816 >> 172.21.15.53:0/3032240816 conn(0x7f6a0406a380 msgr2=0x7f6a040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:35.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/3032240816 shutdown_connections 2023-10-21T16:18:35.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.031+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/3032240816 wait complete. 2023-10-21T16:18:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.033+0000 7f6a0c4a2640 1 Processor -- start 2023-10-21T16:18:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.033+0000 7f6a0c4a2640 1 -- start start 2023-10-21T16:18:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0415cd90 0x7f6a040c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a04160940 0x7f6a0415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0414e8f0 con 0x7f6a0406c890 2023-10-21T16:18:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040c6b20 con 0x7f6a04160940 2023-10-21T16:18:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0c4a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040c6e00 con 0x7f6a0415cd90 2023-10-21T16:18:35.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42598/0 (socket says 172.21.15.53:42598) 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 -- 172.21.15.53:0/982201495 learned_addr learned my addr 172.21.15.53:0/982201495 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0415cd90 0x7f6a040c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 -- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0415cd90 msgr2=0x7f6a040c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0415cd90 0x7f6a040c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0aa3f640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a04160940 0x7f6a0415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 -- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a04160940 msgr2=0x7f6a0415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.034+0000 7f6a0a23e640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a04160940 0x7f6a0415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f6a0a23e640 1 -- 172.21.15.53:0/982201495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009400 con 0x7f6a0406c890 2023-10-21T16:18:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f6a0a23e640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c500 tx=0x7f6a0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00005ce0 con 0x7f6a0406c890 2023-10-21T16:18:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00005e40 con 0x7f6a0406c890 2023-10-21T16:18:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00007330 con 0x7f6a0406c890 2023-10-21T16:18:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a040c70e0 con 0x7f6a0406c890 2023-10-21T16:18:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.035+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0415ffc0 con 0x7f6a0406c890 2023-10-21T16:18:35.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.036+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a040c7600 con 0x7f6a0406c890 2023-10-21T16:18:35.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.037+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6a00007490 con 0x7f6a0406c890 2023-10-21T16:18:35.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.038+0000 7f69faffd640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 0x7f69e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.039+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f6a00088e60 con 0x7f6a0406c890 2023-10-21T16:18:35.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.039+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 0x7f69e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.040+0000 7f6a09a3d640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 0x7f69e806d3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69f4036000 tx=0x7f69f403c040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.043+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a040c7600 con 0x7f6a0406c890 2023-10-21T16:18:35.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.319+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f6a040c7600 con 0x7f6a0406c890 2023-10-21T16:18:35.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.358+0000 7f69faffd640 1 -- 172.21.15.53:0/982201495 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v637) v1 ==== 166+0+0 (secure 0 0 0) 0x7f6a040c7600 con 0x7f6a0406c890 2023-10-21T16:18:35.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.361+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 msgr2=0x7f69e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.361+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 0x7f69e806d3f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f69f4036000 tx=0x7f69f403c040 comp rx=0 tx=0).stop 2023-10-21T16:18:35.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.361+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 msgr2=0x7f6a0415e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c500 tx=0x7f6a0000c9c0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 shutdown_connections 2023-10-21T16:18:35.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f69e806af30 0x7f69e806d3f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a04160940 0x7f6a0415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a0415cd90 0x7f6a040c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 --2- 172.21.15.53:0/982201495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a0406c890 0x7f6a0415e230 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 >> 172.21.15.53:0/982201495 conn(0x7f6a0406a380 msgr2=0x7f6a040701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:35.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 shutdown_connections 2023-10-21T16:18:35.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.362+0000 7f6a0c4a2640 1 -- 172.21.15.53:0/982201495 wait complete. 2023-10-21T16:18:35.385 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-21T16:18:35.385 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-21T16:18:35.386 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:18:35.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:35.458 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:35.456+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:35.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.622+0000 7f6e4a533640 1 Processor -- start 2023-10-21T16:18:35.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.622+0000 7f6e4a533640 1 -- start start 2023-10-21T16:18:35.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.624+0000 7f6e4a533640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e44162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.624+0000 7f6e4a533640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e440cb150 0x7f6e440cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.624+0000 7f6e4a533640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ae140 con 0x7f6e440add60 2023-10-21T16:18:35.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.624+0000 7f6e4a533640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44161f10 con 0x7f6e440cb150 2023-10-21T16:18:35.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.624+0000 7f6e4a533640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cbb20 con 0x7f6e4406d4c0 2023-10-21T16:18:35.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.625+0000 7f6e48ad0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e440cb150 0x7f6e440cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.625+0000 7f6e48ad0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e440cb150 0x7f6e440cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59630/0 (socket says 172.21.15.53:59630) 2023-10-21T16:18:35.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.625+0000 7f6e48ad0640 1 -- 172.21.15.53:0/2380742873 learned_addr learned my addr 172.21.15.53:0/2380742873 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:35.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.625+0000 7f6e437fe640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e44162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.626+0000 7f6e48ad0640 1 -- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 msgr2=0x7f6e44162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:35.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.626+0000 7f6e48ad0640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e44162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.626+0000 7f6e48ad0640 1 -- 172.21.15.53:0/2380742873 >> v1:172.21.15.124:6789/0 conn(0x7f6e440add60 legacy=0x7f6e440cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.626+0000 7f6e48ad0640 1 -- 172.21.15.53:0/2380742873 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440ccf20 con 0x7f6e440cb150 2023-10-21T16:18:35.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.626+0000 7f6e437fe640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e44162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:35.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.627+0000 7f6e48ad0640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440cb150 0x7f6e440cb5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20205dc459570ae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e42ffd640 1 -- 172.21.15.53:0/2380742873 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c031640 con 0x7f6e440cb150 2023-10-21T16:18:35.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e42ffd640 1 -- 172.21.15.53:0/2380742873 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0317a0 con 0x7f6e440cb150 2023-10-21T16:18:35.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e42ffd640 1 -- 172.21.15.53:0/2380742873 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c031b30 con 0x7f6e440cb150 2023-10-21T16:18:35.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 -- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440cb150 msgr2=0x7f6e440cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440cb150 0x7f6e440cb5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c02b8f0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 -- 172.21.15.53:0/2380742873 shutdown_connections 2023-10-21T16:18:35.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e44162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 --2- 172.21.15.53:0/2380742873 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440cb150 0x7f6e440cb5e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 -- 172.21.15.53:0/2380742873 >> 172.21.15.53:0/2380742873 conn(0x7f6e4406a380 msgr2=0x7f6e440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:35.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.628+0000 7f6e4a533640 1 -- 172.21.15.53:0/2380742873 shutdown_connections 2023-10-21T16:18:35.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.629+0000 7f6e4a533640 1 -- 172.21.15.53:0/2380742873 wait complete. 2023-10-21T16:18:35.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.630+0000 7f6e4a533640 1 Processor -- start 2023-10-21T16:18:35.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.630+0000 7f6e4a533640 1 -- start start 2023-10-21T16:18:35.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.631+0000 7f6e4a533640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e440b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.631+0000 7f6e4a533640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440add60 0x7f6e440b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.631+0000 7f6e4a533640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e4a533640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b21c0 con 0x7f6e440cb150 2023-10-21T16:18:35.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e4a533640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b15b0 con 0x7f6e440add60 2023-10-21T16:18:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e4a533640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b1890 con 0x7f6e4406d4c0 2023-10-21T16:18:35.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42600/0 (socket says 172.21.15.53:42600) 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 -- 172.21.15.53:0/82075884 learned_addr learned my addr 172.21.15.53:0/82075884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e437fe640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440add60 0x7f6e440b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e43fff640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e440b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 -- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 msgr2=0x7f6e440b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e440b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 -- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440add60 msgr2=0x7f6e440b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440add60 0x7f6e440b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.632+0000 7f6e48ad0640 1 -- 172.21.15.53:0/82075884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c009400 con 0x7f6e440cb150 2023-10-21T16:18:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e48ad0640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c03a930 con 0x7f6e440cb150 2023-10-21T16:18:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c03aab0 con 0x7f6e440cb150 2023-10-21T16:18:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c03ae60 con 0x7f6e440cb150 2023-10-21T16:18:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c0094d0 con 0x7f6e440cb150 2023-10-21T16:18:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.633+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440b1e50 con 0x7f6e440cb150 2023-10-21T16:18:35.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.634+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440b2330 con 0x7f6e440cb150 2023-10-21T16:18:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.635+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6e3c02f050 con 0x7f6e440cb150 2023-10-21T16:18:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.636+0000 7f6e40ff9640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 0x7f6e2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.636+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f6e3c0ab800 con 0x7f6e440cb150 2023-10-21T16:18:35.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.637+0000 7f6e43fff640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 0x7f6e2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.638+0000 7f6e43fff640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 0x7f6e2406d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e340012e0 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:35.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.640+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e3c076710 con 0x7f6e440cb150 2023-10-21T16:18:35.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.849+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e44067490 con 0x7f6e440cb150 2023-10-21T16:18:35.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.851+0000 7f6e40ff9640 1 -- 172.21.15.53:0/82075884 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+14336 (secure 0 0 0) 0x7f6e3c079d30 con 0x7f6e440cb150 2023-10-21T16:18:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:35.853 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":109,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:33.328068+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2451852009":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/600599962":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2629801108":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/2272255357":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/1235374158":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6833/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.124:6832/1331540020":"2023-10-22T16:18:25.676317+0000","172.21.15.53:6835/797303097":"2023-10-22T16:18:29.868969+0000","172.21.15.124:0/824950480":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6833/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/2324608475":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3280078422":"2023-10-22T16:18:19.516545+0000","172.21.15.124:6832/3674677134":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3826369357":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3225172013":"2023-10-22T16:18:19.516545+0000","172.21.15.53:6833/797303097":"2023-10-22T16:18:29.868969+0000","172.21.15.124:0/539630491":"2023-10-22T16:18:25.676317+0000","172.21.15.124:0/798725635":"2023-10-22T16:18:19.516545+0000","172.21.15.124:0/3415687515":"2023-10-21T17:18:12.553813+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 msgr2=0x7f6e2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 0x7f6e2406d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e340012e0 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2023-10-21T16:18:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 msgr2=0x7f6e44151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 shutdown_connections 2023-10-21T16:18:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f6e2406af00 0x7f6e2406d3c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e440cb150 0x7f6e44151930 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e440add60 0x7f6e440b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 --2- 172.21.15.53:0/82075884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4406d4c0 0x7f6e440b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.854+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 >> 172.21.15.53:0/82075884 conn(0x7f6e4406a380 msgr2=0x7f6e4406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:35.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.855+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 shutdown_connections 2023-10-21T16:18:35.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:35.855+0000 7f6e4a533640 1 -- 172.21.15.53:0/82075884 wait complete. 2023-10-21T16:18:35.877 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:18:35.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:18:36.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.117+0000 7f1ba6f0f640 1 Processor -- start 2023-10-21T16:18:36.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.117+0000 7f1ba6f0f640 1 -- start start 2023-10-21T16:18:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba6f0f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ba00702d0 0x7f1ba015d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba6f0f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba00b30a0 con 0x7f1ba00b40e0 2023-10-21T16:18:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba6f0f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba00b3320 con 0x7f1ba00b1750 2023-10-21T16:18:36.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba6f0f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba015dcf0 con 0x7f1ba00702d0 2023-10-21T16:18:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba4cab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ba00702d0 0x7f1ba015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.119+0000 7f1ba54ac640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1ba00b40e0 0x7f1ba00c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38174/0 (socket says 172.21.15.53:38174) 2023-10-21T16:18:36.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.120+0000 7f1ba54ac640 1 -- 172.21.15.53:0/1164391678 learned_addr learned my addr 172.21.15.53:0/1164391678 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.120+0000 7f1ba4cab640 1 -- 172.21.15.53:0/1164391678 >> v1:172.21.15.124:6789/0 conn(0x7f1ba00b1750 legacy=0x7f1ba00706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.120+0000 7f1ba4cab640 1 -- 172.21.15.53:0/1164391678 >> v1:172.21.15.53:6789/0 conn(0x7f1ba00b40e0 legacy=0x7f1ba00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.120+0000 7f1ba4cab640 1 -- 172.21.15.53:0/1164391678 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00b3600 con 0x7f1ba00702d0 2023-10-21T16:18:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.121+0000 7f1ba4cab640 1 --2- 172.21.15.53:0/1164391678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00702d0 0x7f1ba015d710 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400ba40 tx=0x7f1b9402ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf99528017b5535 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1b9f7fe640 1 -- 172.21.15.53:0/1164391678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9402eba0 con 0x7f1ba00702d0 2023-10-21T16:18:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1b9f7fe640 1 -- 172.21.15.53:0/1164391678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94032070 con 0x7f1ba00702d0 2023-10-21T16:18:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1b9f7fe640 1 -- 172.21.15.53:0/1164391678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94038400 con 0x7f1ba00702d0 2023-10-21T16:18:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1164391678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00702d0 msgr2=0x7f1ba015d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1164391678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00702d0 0x7f1ba015d710 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1b9400ba40 tx=0x7f1b9402ddf0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1164391678 shutdown_connections 2023-10-21T16:18:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1164391678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00702d0 0x7f1ba015d710 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1164391678 >> 172.21.15.53:0/1164391678 conn(0x7f1ba0069d50 msgr2=0x7f1ba006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1164391678 shutdown_connections 2023-10-21T16:18:36.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.122+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1164391678 wait complete. 2023-10-21T16:18:36.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.124+0000 7f1ba6f0f640 1 Processor -- start 2023-10-21T16:18:36.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.124+0000 7f1ba6f0f640 1 -- start start 2023-10-21T16:18:36.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba00702d0 0x7f1ba0155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba00b40e0 0x7f1ba01f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014b500 con 0x7f1ba00b1750 2023-10-21T16:18:36.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014eb60 con 0x7f1ba00b40e0 2023-10-21T16:18:36.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.125+0000 7f1ba6f0f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014a8f0 con 0x7f1ba00702d0 2023-10-21T16:18:36.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42628/0 (socket says 172.21.15.53:42628) 2023-10-21T16:18:36.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 -- 172.21.15.53:0/1036748089 learned_addr learned my addr 172.21.15.53:0/1036748089 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:36.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1ba54ac640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba00b40e0 0x7f1ba01f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1ba4cab640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba00702d0 0x7f1ba0155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 -- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba00702d0 msgr2=0x7f1ba0155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba00702d0 0x7f1ba0155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 -- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba00b40e0 msgr2=0x7f1ba01f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba00b40e0 0x7f1ba01f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 -- 172.21.15.53:0/1036748089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9400b6a0 con 0x7f1ba00b1750 2023-10-21T16:18:36.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.126+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b3f0 tx=0x7f1b8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.127+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c011830 con 0x7f1ba00b1750 2023-10-21T16:18:36.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.127+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c011990 con 0x7f1ba00b1750 2023-10-21T16:18:36.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.127+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c011d40 con 0x7f1ba00b1750 2023-10-21T16:18:36.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.127+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba014abd0 con 0x7f1ba00b1750 2023-10-21T16:18:36.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.127+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba014b190 con 0x7f1ba00b1750 2023-10-21T16:18:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.128+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba004e160 con 0x7f1ba00b1750 2023-10-21T16:18:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.129+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1b8c01a400 con 0x7f1ba00b1750 2023-10-21T16:18:36.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.130+0000 7f1b9d7fa640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 0x7f1b8406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.130+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f1b8c089630 con 0x7f1ba00b1750 2023-10-21T16:18:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.131+0000 7f1ba4cab640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 0x7f1b8406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.132+0000 7f1ba4cab640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 0x7f1b8406d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b94039000 tx=0x7f1b94034000 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.134+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b8c0544a0 con 0x7f1ba00b1750 2023-10-21T16:18:36.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.331+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ba00710b0 con 0x7f1ba00b1750 2023-10-21T16:18:36.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.332+0000 7f1b9d7fa640 1 -- 172.21.15.53:0/1036748089 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b8c057ac0 con 0x7f1ba00b1750 2023-10-21T16:18:36.334 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:18:36.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 msgr2=0x7f1b8406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 0x7f1b8406d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b94039000 tx=0x7f1b94034000 comp rx=0 tx=0).stop 2023-10-21T16:18:36.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 msgr2=0x7f1ba014a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b3f0 tx=0x7f1b8c00b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 shutdown_connections 2023-10-21T16:18:36.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f1b8406af30 0x7f1b8406d3f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba00b40e0 0x7f1ba01f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba00b1750 0x7f1ba014a0a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 --2- 172.21.15.53:0/1036748089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba00702d0 0x7f1ba0155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.334+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 >> 172.21.15.53:0/1036748089 conn(0x7f1ba0069d50 msgr2=0x7f1ba00b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:36.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.335+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 shutdown_connections 2023-10-21T16:18:36.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.335+0000 7f1ba6f0f640 1 -- 172.21.15.53:0/1036748089 wait complete. 2023-10-21T16:18:36.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-21T16:18:36.571 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:36.569+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:36.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.585+0000 7f2843444640 1 Processor -- start 2023-10-21T16:18:36.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.585+0000 7f2843444640 1 -- start start 2023-10-21T16:18:36.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f2843444640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f283c165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f2843444640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283c15b2b0 con 0x7f283c15ab40 2023-10-21T16:18:36.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f2843444640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283c15b590 con 0x7f283c0c90b0 2023-10-21T16:18:36.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f2843444640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c15bf30 con 0x7f283c0cb580 2023-10-21T16:18:36.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f28411e0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f283c15ab40 0x7f283c0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38182/0 (socket says 172.21.15.53:38182) 2023-10-21T16:18:36.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.587+0000 7f28411e0640 1 -- 172.21.15.53:0/1584167408 learned_addr learned my addr 172.21.15.53:0/1584167408 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:36.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f28419e1640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f283c165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182148260 0 0) 0x7f283c15b2b0 con 0x7f283c15ab40 2023-10-21T16:18:36.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2828003520 con 0x7f283c15ab40 2023-10-21T16:18:36.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994031220 0 0) 0x7f283c15b590 con 0x7f283c0c90b0 2023-10-21T16:18:36.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f283c15b2b0 con 0x7f283c0c90b0 2023-10-21T16:18:36.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162583459 0 0) 0x7f2828003520 con 0x7f283c15ab40 2023-10-21T16:18:36.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 msgr2=0x7f283c165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:36.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f283c165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 >> v1:172.21.15.124:6789/0 conn(0x7f283c0c90b0 legacy=0x7f283c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c15b870 con 0x7f283c15ab40 2023-10-21T16:18:36.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.588+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2838001690 con 0x7f283c15ab40 2023-10-21T16:18:36.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 >> v1:172.21.15.53:6789/0 conn(0x7f283c15ab40 legacy=0x7f283c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f2828003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f28280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2828004d00 0x7f2828005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c15bf30 con 0x7f2828003b30 2023-10-21T16:18:36.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282800ba60 con 0x7f2828004d00 2023-10-21T16:18:36.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282800bc20 con 0x7f283c0cb580 2023-10-21T16:18:36.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f28411e0640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f2828003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f28409df640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f28280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f28419e1640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2828004d00 0x7f2828005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f28411e0640 1 -- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 msgr2=0x7f28280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:36.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.589+0000 7f28411e0640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0cb580 0x7f28280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.590+0000 7f28411e0640 1 -- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2828004d00 msgr2=0x7f2828005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:36.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.590+0000 7f28411e0640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2828004d00 0x7f2828005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.590+0000 7f28411e0640 1 -- 172.21.15.53:0/1584167408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828003520 con 0x7f2828003b30 2023-10-21T16:18:36.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.591+0000 7f28411e0640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f2828003f30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f283c06e240 tx=0x7f28280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a623b1904a38f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.591+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838005ad0 con 0x7f2828003b30 2023-10-21T16:18:36.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.591+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2838005c50 con 0x7f2828003b30 2023-10-21T16:18:36.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.591+0000 7f2833fff640 1 -- 172.21.15.53:0/1584167408 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283802c810 con 0x7f2828003b30 2023-10-21T16:18:36.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 -- 172.21.15.53:0/1584167408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 msgr2=0x7f2828003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 --2- 172.21.15.53:0/1584167408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f2828003f30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f283c06e240 tx=0x7f28280057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f28419e1640 1 -- 172.21.15.53:0/1584167408 reap_dead start 2023-10-21T16:18:36.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 -- 172.21.15.53:0/1584167408 shutdown_connections 2023-10-21T16:18:36.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 -- 172.21.15.53:0/1584167408 >> 172.21.15.53:0/1584167408 conn(0x7f283c06a380 msgr2=0x7f283c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:36.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 -- 172.21.15.53:0/1584167408 shutdown_connections 2023-10-21T16:18:36.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.592+0000 7f2843444640 1 -- 172.21.15.53:0/1584167408 wait complete. 2023-10-21T16:18:36.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.594+0000 7f2843444640 1 Processor -- start 2023-10-21T16:18:36.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.594+0000 7f2843444640 1 -- start start 2023-10-21T16:18:36.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0c7960 0x7f283c1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 0x7f283c0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c15f7a0 con 0x7f2828003b30 2023-10-21T16:18:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1f2580 con 0x7f283c0abf10 2023-10-21T16:18:36.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f2843444640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1f6a90 con 0x7f283c0c7960 2023-10-21T16:18:36.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f28411e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f28411e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42646/0 (socket says 172.21.15.53:42646) 2023-10-21T16:18:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.595+0000 7f28411e0640 1 -- 172.21.15.53:0/3646678774 learned_addr learned my addr 172.21.15.53:0/3646678774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28409df640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0c7960 0x7f283c1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 -- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0c7960 msgr2=0x7f283c1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0c7960 0x7f283c1f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 -- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 msgr2=0x7f283c0ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28419e1640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 0x7f283c0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 0x7f283c0ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 -- 172.21.15.53:0/3646678774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c0aba80 con 0x7f2828003b30 2023-10-21T16:18:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28419e1640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 0x7f283c0ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:36.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f28411e0640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f283800c2d0 tx=0x7f28380019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283802cc20 con 0x7f2828003b30 2023-10-21T16:18:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283802cd80 con 0x7f2828003b30 2023-10-21T16:18:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838007290 con 0x7f2828003b30 2023-10-21T16:18:36.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.596+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c1f61f0 con 0x7f2828003b30 2023-10-21T16:18:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.597+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c1f67b0 con 0x7f2828003b30 2023-10-21T16:18:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.598+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c067490 con 0x7f2828003b30 2023-10-21T16:18:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.599+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2838032070 con 0x7f2828003b30 2023-10-21T16:18:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.600+0000 7f2831ffb640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 0x7f282006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:36.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.600+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f283803c040 con 0x7f2828003b30 2023-10-21T16:18:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.600+0000 7f28409df640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 0x7f282006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.601+0000 7f28409df640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 0x7f282006d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f282c007fd0 tx=0x7f282c009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:36.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.605+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2838075b90 con 0x7f2828003b30 2023-10-21T16:18:36.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:36.880+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f283c0b0ef0 con 0x7f2828003b30 2023-10-21T16:18:37.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.376+0000 7f2831ffb640 1 -- 172.21.15.53:0/3646678774 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v639) v1 ==== 186+0+0 (secure 0 0 0) 0x7f2838045020 con 0x7f2828003b30 2023-10-21T16:18:37.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.379+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 msgr2=0x7f282006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 0x7f282006d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f282c007fd0 tx=0x7f282c009040 comp rx=0 tx=0).stop 2023-10-21T16:18:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 msgr2=0x7f283c1f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f283800c2d0 tx=0x7f28380019a0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 shutdown_connections 2023-10-21T16:18:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f282006af00 0x7f282006d3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f283c0abf10 0x7f283c0ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f283c0c7960 0x7f283c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 --2- 172.21.15.53:0/3646678774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2828003b30 0x7f283c1f5c10 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 >> 172.21.15.53:0/3646678774 conn(0x7f283c06a380 msgr2=0x7f283c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:37.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 shutdown_connections 2023-10-21T16:18:37.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.380+0000 7f2843444640 1 -- 172.21.15.53:0/3646678774 wait complete. 2023-10-21T16:18:37.406 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:18:37.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.631+0000 7fbe1a044640 1 Processor -- start 2023-10-21T16:18:37.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.631+0000 7fbe1a044640 1 -- start start 2023-10-21T16:18:37.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe1a044640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe140cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe1a044640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe140b1cc0 con 0x7fbe140afbe0 2023-10-21T16:18:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe1a044640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe140cbad0 con 0x7fbe1406dfc0 2023-10-21T16:18:37.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe1a044640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140b2620 con 0x7fbe140cb130 2023-10-21T16:18:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe137fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbe140afbe0 0x7fbe140b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38190/0 (socket says 172.21.15.53:38190) 2023-10-21T16:18:37.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.633+0000 7fbe137fe640 1 -- 172.21.15.53:0/1106934961 learned_addr learned my addr 172.21.15.53:0/1106934961 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:37.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe13fff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe140cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390768753 0 0) 0x7fbe140b1cc0 con 0x7fbe140afbe0 2023-10-21T16:18:37.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe00003520 con 0x7fbe140afbe0 2023-10-21T16:18:37.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 504489631 0 0) 0x7fbe00003520 con 0x7fbe140afbe0 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 msgr2=0x7fbe140cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe140cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 >> v1:172.21.15.124:6789/0 conn(0x7fbe1406dfc0 legacy=0x7fbe14070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140b2060 con 0x7fbe140afbe0 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fbe04001670 con 0x7fbe140afbe0 2023-10-21T16:18:37.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 >> v1:172.21.15.53:6789/0 conn(0x7fbe140afbe0 legacy=0x7fbe140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000039a0 0x7fbe00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.634+0000 7fbe12ffd640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe000047c0 0x7fbe00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe12ffd640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140b2620 con 0x7fbe000047c0 2023-10-21T16:18:37.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000ba60 con 0x7fbe000039a0 2023-10-21T16:18:37.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0000bc20 con 0x7fbe140cb130 2023-10-21T16:18:37.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe000047c0 0x7fbe00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe13fff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe137fe640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000039a0 0x7fbe00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 -- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 msgr2=0x7fbe00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:37.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe00005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 -- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000039a0 msgr2=0x7fbe00003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:37.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000039a0 0x7fbe00003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe0bfff640 1 -- 172.21.15.53:0/1106934961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00003520 con 0x7fbe000047c0 2023-10-21T16:18:37.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.635+0000 7fbe13fff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140cb130 0x7fbe00005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:37.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.636+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe000047c0 0x7fbe00004c40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbe1406e410 tx=0x7fbe000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdfb4c0a82a780a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0052d0 con 0x7fbe000047c0 2023-10-21T16:18:37.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc02e050 con 0x7fbe000047c0 2023-10-21T16:18:37.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe12ffd640 1 -- 172.21.15.53:0/1106934961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0035f0 con 0x7fbe000047c0 2023-10-21T16:18:37.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe1a044640 1 -- 172.21.15.53:0/1106934961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe000047c0 msgr2=0x7fbe00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe1a044640 1 --2- 172.21.15.53:0/1106934961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe000047c0 0x7fbe00004c40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbe1406e410 tx=0x7fbe000057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.637+0000 7fbe13fff640 1 -- 172.21.15.53:0/1106934961 reap_dead start 2023-10-21T16:18:37.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.638+0000 7fbe1a044640 1 -- 172.21.15.53:0/1106934961 shutdown_connections 2023-10-21T16:18:37.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.638+0000 7fbe1a044640 1 -- 172.21.15.53:0/1106934961 >> 172.21.15.53:0/1106934961 conn(0x7fbe1406a5d0 msgr2=0x7fbe140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:37.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.638+0000 7fbe1a044640 1 -- 172.21.15.53:0/1106934961 shutdown_connections 2023-10-21T16:18:37.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.638+0000 7fbe1a044640 1 -- 172.21.15.53:0/1106934961 wait complete. 2023-10-21T16:18:37.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.639+0000 7fbe1a044640 1 Processor -- start 2023-10-21T16:18:37.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.640+0000 7fbe1a044640 1 -- start start 2023-10-21T16:18:37.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000047c0 0x7fbe1415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 0x7fbe141f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140c9950 con 0x7fbe1414f450 2023-10-21T16:18:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140c4a40 con 0x7fbe000047c0 2023-10-21T16:18:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe1a044640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe141f6e90 con 0x7fbe140ca2b0 2023-10-21T16:18:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42664/0 (socket says 172.21.15.53:42664) 2023-10-21T16:18:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 -- 172.21.15.53:0/3315055471 learned_addr learned my addr 172.21.15.53:0/3315055471 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe137fe640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000047c0 0x7fbe1415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe13fff640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 0x7fbe141f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 -- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 msgr2=0x7fbe141f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 0x7fbe141f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 -- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000047c0 msgr2=0x7fbe1415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000047c0 0x7fbe1415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe0bfff640 1 -- 172.21.15.53:0/3315055471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140c4c90 con 0x7fbe1414f450 2023-10-21T16:18:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.641+0000 7fbe13fff640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 0x7fbe141f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe0bfff640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc008fd0 tx=0x7fbdfc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0076c0 con 0x7fbe1414f450 2023-10-21T16:18:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc007a30 con 0x7fbe1414f450 2023-10-21T16:18:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe140c4f70 con 0x7fbe1414f450 2023-10-21T16:18:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc006be0 con 0x7fbe1414f450 2023-10-21T16:18:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.642+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1414f8d0 con 0x7fbe1414f450 2023-10-21T16:18:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.644+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe140c9bc0 con 0x7fbe1414f450 2023-10-21T16:18:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.644+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbdfc006d40 con 0x7fbe1414f450 2023-10-21T16:18:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.645+0000 7fbe10ff9640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 0x7fbdf406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.645+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fbdfc0ab580 con 0x7fbe1414f450 2023-10-21T16:18:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.646+0000 7fbe137fe640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 0x7fbdf406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.647+0000 7fbe137fe640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 0x7fbdf406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe040018e0 tx=0x7fbe04009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.650+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdfc076370 con 0x7fbe1414f450 2023-10-21T16:18:37.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:37.735+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:37.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.854+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbe1404e160 con 0x7fbe1414f450 2023-10-21T16:18:37.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.861+0000 7fbe10ff9640 1 -- 172.21.15.53:0/3315055471 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbdfc079990 con 0x7fbe1414f450 2023-10-21T16:18:37.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.864+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 msgr2=0x7fbdf406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.864+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 0x7fbdf406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbe040018e0 tx=0x7fbe04009040 comp rx=0 tx=0).stop 2023-10-21T16:18:37.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.864+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 msgr2=0x7fbe140c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.864+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc008fd0 tx=0x7fbdfc001120 comp rx=0 tx=0).stop 2023-10-21T16:18:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 shutdown_connections 2023-10-21T16:18:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fbdf406af00 0x7fbdf406d3c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe140ca2b0 0x7fbe141f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe1414f450 0x7fbe140c9d70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 --2- 172.21.15.53:0/3315055471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe000047c0 0x7fbe1415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 >> 172.21.15.53:0/3315055471 conn(0x7fbe1406a5d0 msgr2=0x7fbe14070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 shutdown_connections 2023-10-21T16:18:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:37.865+0000 7fbe1a044640 1 -- 172.21.15.53:0/3315055471 wait complete. 2023-10-21T16:18:37.886 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:18:37.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:18:38.000 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:18:38.119 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:18:38.119 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:38.120 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:38.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:18:38.183 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:18:38.183 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:38.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:18:38.245 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:38.245 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:38.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:18:38.309 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:18:38.309 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:38.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:18:38.369 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:18:38.369 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.597+0000 7fdd8194e640 1 Processor -- start 2023-10-21T16:18:38.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.597+0000 7fdd8194e640 1 -- start start 2023-10-21T16:18:38.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.598+0000 7fdd8194e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 0x7fdd7c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.598+0000 7fdd8194e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd7c06c890 0x7fdd7c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.598+0000 7fdd8194e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c0c48f0 con 0x7fdd7c160940 2023-10-21T16:18:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.598+0000 7fdd8194e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0c4d80 con 0x7fdd7c06c890 2023-10-21T16:18:38.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.598+0000 7fdd8194e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0c3ce0 con 0x7fdd7c15cd90 2023-10-21T16:18:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd7c06c890 0x7fdd7c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd7c06c890 0x7fdd7c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42680/0 (socket says 172.21.15.53:42680) 2023-10-21T16:18:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 -- 172.21.15.53:0/595900245 learned_addr learned my addr 172.21.15.53:0/595900245 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:38.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7affd640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 0x7fdd7c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 -- 172.21.15.53:0/595900245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 msgr2=0x7fdd7c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 0x7fdd7c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 -- 172.21.15.53:0/595900245 >> v1:172.21.15.124:6789/0 conn(0x7fdd7c160940 legacy=0x7fdd7c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7b7fe640 1 -- 172.21.15.53:0/595900245 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c0c3fc0 con 0x7fdd7c06c890 2023-10-21T16:18:38.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.599+0000 7fdd7affd640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 0x7fdd7c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:38.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.600+0000 7fdd7b7fe640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c06c890 0x7fdd7c06cd20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009e50 tx=0x7fdd7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f068d52a2365e4e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:38.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.601+0000 7fdd79ffb640 1 -- 172.21.15.53:0/595900245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7402ca80 con 0x7fdd7c06c890 2023-10-21T16:18:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.601+0000 7fdd79ffb640 1 -- 172.21.15.53:0/595900245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7402cbe0 con 0x7fdd7c06c890 2023-10-21T16:18:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.601+0000 7fdd79ffb640 1 -- 172.21.15.53:0/595900245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740343f0 con 0x7fdd7c06c890 2023-10-21T16:18:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.601+0000 7fdd8194e640 1 -- 172.21.15.53:0/595900245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c06c890 msgr2=0x7fdd7c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:38.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.601+0000 7fdd8194e640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c06c890 0x7fdd7c06cd20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009e50 tx=0x7fdd7402bee0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 -- 172.21.15.53:0/595900245 shutdown_connections 2023-10-21T16:18:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c06c890 0x7fdd7c06cd20 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 --2- 172.21.15.53:0/595900245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c15cd90 0x7fdd7c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 -- 172.21.15.53:0/595900245 >> 172.21.15.53:0/595900245 conn(0x7fdd7c06a380 msgr2=0x7fdd7c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 -- 172.21.15.53:0/595900245 shutdown_connections 2023-10-21T16:18:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.602+0000 7fdd8194e640 1 -- 172.21.15.53:0/595900245 wait complete. 2023-10-21T16:18:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.603+0000 7fdd8194e640 1 Processor -- start 2023-10-21T16:18:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.604+0000 7fdd8194e640 1 -- start start 2023-10-21T16:18:38.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.604+0000 7fdd8194e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd7c06c890 0x7fdd7c157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.604+0000 7fdd8194e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.604+0000 7fdd8194e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c160940 0x7fdd7c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd8194e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c15e720 con 0x7fdd7c15cd90 2023-10-21T16:18:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd8194e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0b17b0 con 0x7fdd7c06c890 2023-10-21T16:18:38.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd8194e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c15e0d0 con 0x7fdd7c160940 2023-10-21T16:18:38.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:38.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42692/0 (socket says 172.21.15.53:42692) 2023-10-21T16:18:38.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 -- 172.21.15.53:0/2814011564 learned_addr learned my addr 172.21.15.53:0/2814011564 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:38.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 -- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c160940 msgr2=0x7fdd7c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c160940 0x7fdd7c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7affd640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd7c06c890 0x7fdd7c157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 -- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd7c06c890 msgr2=0x7fdd7c157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd7c06c890 0x7fdd7c157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.605+0000 7fdd7a7fc640 1 -- 172.21.15.53:0/2814011564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74009b30 con 0x7fdd7c15cd90 2023-10-21T16:18:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd7a7fc640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000b3f0 tx=0x7fdd7000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70011830 con 0x7fdd7c15cd90 2023-10-21T16:18:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70011990 con 0x7fdd7c15cd90 2023-10-21T16:18:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70011d40 con 0x7fdd7c15cd90 2023-10-21T16:18:38.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c15e3b0 con 0x7fdd7c15cd90 2023-10-21T16:18:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.606+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c13f260 con 0x7fdd7c15cd90 2023-10-21T16:18:38.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.607+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c067490 con 0x7fdd7c15cd90 2023-10-21T16:18:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.608+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdd7001a400 con 0x7fdd7c15cd90 2023-10-21T16:18:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.609+0000 7fdd63fff640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 0x7fdd6806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.609+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fdd70089680 con 0x7fdd7c15cd90 2023-10-21T16:18:38.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.610+0000 7fdd7affd640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 0x7fdd6806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.611+0000 7fdd7affd640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 0x7fdd6806d3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c008850 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:38.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.613+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd700544f0 con 0x7fdd7c15cd90 2023-10-21T16:18:38.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.851+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd7c0b0ef0 con 0x7fdd7c15cd90 2023-10-21T16:18:38.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.852+0000 7fdd63fff640 1 -- 172.21.15.53:0/2814011564 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3707 (secure 0 0 0) 0x7fdd70015e90 con 0x7fdd7c15cd90 2023-10-21T16:18:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:38.854 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/4203039490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":4203039490},{"type":"v1","addr":"172.21.15.124:6835","nonce":4203039490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7455,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2043750836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2043750836},{"type":"v1","addr":"172.21.15.53:6835","nonce":2043750836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":35618,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2036423079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2036423079},{"type":"v1","addr":"172.21.15.124:6837","nonce":2036423079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":35621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1697218527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1697218527},{"type":"v1","addr":"172.21.15.53:6834","nonce":1697218527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[]} 2023-10-21T16:18:38.855 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 46 2023-10-21T16:18:38.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 msgr2=0x7fdd6806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:38.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 0x7fdd6806d3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c008850 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:38.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 msgr2=0x7fdd7c155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:38.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000b3f0 tx=0x7fdd7000b8b0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 shutdown_connections 2023-10-21T16:18:38.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdd6806af30 0x7fdd6806d3f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd7c160940 0x7fdd7c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd7c15cd90 0x7fdd7c155c90 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 --2- 172.21.15.53:0/2814011564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd7c06c890 0x7fdd7c157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:38.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 >> 172.21.15.53:0/2814011564 conn(0x7fdd7c06a380 msgr2=0x7fdd7c0ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:38.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 shutdown_connections 2023-10-21T16:18:38.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:38.855+0000 7fdd8194e640 1 -- 172.21.15.53:0/2814011564 wait complete. 2023-10-21T16:18:38.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:38.864+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:38.878 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:18:38.878 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:18:38.879 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:18:38.879 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:18:38.879 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:18:38.879 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:18:38.880 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:18:38.880 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:18:38.880 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:18:38.880 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:18:38.881 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:18:38.881 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:18:38.881 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:18:38.882 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:18:38.882 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:18:38.882 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:18:38.900 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:18:38.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:18:38.902 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:18:38.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:18:38.906 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:18:38.906 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:18:38.907 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:18:38.907 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:18:38.950 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:18:38.951 DEBUG:teuthology.parallel:result is None 2023-10-21T16:18:38.955 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:18:38.955 DEBUG:teuthology.parallel:result is None 2023-10-21T16:18:38.958 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:18:38.958 DEBUG:teuthology.parallel:result is None 2023-10-21T16:18:38.961 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:18:38.962 DEBUG:teuthology.parallel:result is None 2023-10-21T16:18:38.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:18:39.120 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:18:39.122 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:18:39.129 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:18:39.129 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:18:39.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.162+0000 7fcc0ced7640 1 Processor -- start 2023-10-21T16:18:39.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.162+0000 7fcc0ced7640 1 -- start start 2023-10-21T16:18:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.163+0000 7fcc0ced7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.163+0000 7fcc0ced7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcc080ca370 0x7fcc0806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.163+0000 7fcc0ced7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0806c3a0 con 0x7fcc080b21f0 2023-10-21T16:18:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.163+0000 7fcc0ced7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc08070bd0 con 0x7fcc080ca370 2023-10-21T16:18:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.163+0000 7fcc0ced7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0806c640 con 0x7fcc080702b0 2023-10-21T16:18:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc05d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcc080b21f0 0x7fcc080c9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38198/0 (socket says 172.21.15.53:38198) 2023-10-21T16:18:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc05d9b640 1 -- 172.21.15.53:0/1588847665 learned_addr learned my addr 172.21.15.53:0/1588847665 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc06d9d640 1 --2- 172.21.15.53:0/1588847665 >> v2:172.21.15.53:3300/0 conn(0x7fcc080ca370 0x7fcc0806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc0659c640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc0559a640 1 -- 172.21.15.53:0/1588847665 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707665884 0 0) 0x7fcc0806c3a0 con 0x7fcc080b21f0 2023-10-21T16:18:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc0559a640 1 -- 172.21.15.53:0/1588847665 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf0003520 con 0x7fcc080b21f0 2023-10-21T16:18:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc06d9d640 1 -- 172.21.15.53:0/1588847665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 msgr2=0x7fcc08070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc06d9d640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc06d9d640 1 -- 172.21.15.53:0/1588847665 >> v1:172.21.15.53:6789/0 conn(0x7fcc080b21f0 legacy=0x7fcc080c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc06d9d640 1 -- 172.21.15.53:0/1588847665 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0806c920 con 0x7fcc080ca370 2023-10-21T16:18:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.164+0000 7fcc0659c640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.165+0000 7fcc06d9d640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080ca370 0x7fcc0806d400 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009d90 tx=0x7fcc0002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7992956226d25aec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0559a640 1 -- 172.21.15.53:0/1588847665 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0002cbc0 con 0x7fcc080ca370 2023-10-21T16:18:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0559a640 1 -- 172.21.15.53:0/1588847665 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0002cd20 con 0x7fcc080ca370 2023-10-21T16:18:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0559a640 1 -- 172.21.15.53:0/1588847665 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00034660 con 0x7fcc080ca370 2023-10-21T16:18:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 -- 172.21.15.53:0/1588847665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080ca370 msgr2=0x7fcc0806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080ca370 0x7fcc0806d400 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009d90 tx=0x7fcc0002be10 comp rx=0 tx=0).stop 2023-10-21T16:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 -- 172.21.15.53:0/1588847665 shutdown_connections 2023-10-21T16:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080ca370 0x7fcc0806d400 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/1588847665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 -- 172.21.15.53:0/1588847665 >> 172.21.15.53:0/1588847665 conn(0x7fcc0806a5d0 msgr2=0x7fcc080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:39.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 -- 172.21.15.53:0/1588847665 shutdown_connections 2023-10-21T16:18:39.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.166+0000 7fcc0ced7640 1 -- 172.21.15.53:0/1588847665 wait complete. 2023-10-21T16:18:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.168+0000 7fcc0ced7640 1 Processor -- start 2023-10-21T16:18:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.168+0000 7fcc0ced7640 1 -- start start 2023-10-21T16:18:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc080ca370 0x7fcc081f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc080cec70 con 0x7fcc080b21f0 2023-10-21T16:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0814c690 con 0x7fcc080ca370 2023-10-21T16:18:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.169+0000 7fcc0ced7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081529d0 con 0x7fcc080702b0 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42732/0 (socket says 172.21.15.53:42732) 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 -- 172.21.15.53:0/2834550807 learned_addr learned my addr 172.21.15.53:0/2834550807 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc0659c640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc06d9d640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc080ca370 0x7fcc081f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 -- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 msgr2=0x7fcc08157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 -- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc080ca370 msgr2=0x7fcc081f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc080ca370 0x7fcc081f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 -- 172.21.15.53:0/2834550807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8009400 con 0x7fcc080b21f0 2023-10-21T16:18:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.170+0000 7fcc05d9b640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c500 tx=0x7fcbfc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.171+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc005ce0 con 0x7fcc080b21f0 2023-10-21T16:18:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.171+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc005e40 con 0x7fcc080b21f0 2023-10-21T16:18:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.171+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc007330 con 0x7fcc080b21f0 2023-10-21T16:18:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.171+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00009a70 con 0x7fcc080b21f0 2023-10-21T16:18:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.171+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc080ce6b0 con 0x7fcc080b21f0 2023-10-21T16:18:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.172+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc08152b60 con 0x7fcc080b21f0 2023-10-21T16:18:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.173+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcbfc007550 con 0x7fcc080b21f0 2023-10-21T16:18:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.174+0000 7fcbf6ffd640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 0x7fcbe806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.174+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fcbfc088e90 con 0x7fcc080b21f0 2023-10-21T16:18:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.175+0000 7fcc0659c640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 0x7fcbe806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.175+0000 7fcc0659c640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 0x7fcbe806d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009c60 tx=0x7fcbf8008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.178+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbfc053e50 con 0x7fcc080b21f0 2023-10-21T16:18:39.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.398+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcc0804e160 con 0x7fcc080b21f0 2023-10-21T16:18:39.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.411+0000 7fcbf6ffd640 1 -- 172.21.15.53:0/2834550807 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcbfc08a080 con 0x7fcc080b21f0 2023-10-21T16:18:39.413 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:18:39.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.413+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 msgr2=0x7fcbe806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 0x7fcbe806d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009c60 tx=0x7fcbf8008040 comp rx=0 tx=0).stop 2023-10-21T16:18:39.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 msgr2=0x7fcc080cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c500 tx=0x7fcbfc00c9c0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 shutdown_connections 2023-10-21T16:18:39.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcbe806af00 0x7fcbe806d3c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc080ca370 0x7fcc081f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc080b21f0 0x7fcc080cdd30 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 --2- 172.21.15.53:0/2834550807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc080702b0 0x7fcc08157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 >> 172.21.15.53:0/2834550807 conn(0x7fcc0806a5d0 msgr2=0x7fcc080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:39.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 shutdown_connections 2023-10-21T16:18:39.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.414+0000 7fcc0ced7640 1 -- 172.21.15.53:0/2834550807 wait complete. 2023-10-21T16:18:39.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:18:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.649+0000 7f51eadff640 1 Processor -- start 2023-10-21T16:18:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.650+0000 7f51eadff640 1 -- start start 2023-10-21T16:18:39.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.651+0000 7f51eadff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51e40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.651+0000 7f51eadff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f51e40d16f0 0x7f51e40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.651+0000 7f51eadff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e415d040 con 0x7f51e40cdfd0 2023-10-21T16:18:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.651+0000 7f51eadff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e40d0650 con 0x7f51e40d16f0 2023-10-21T16:18:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.651+0000 7f51eadff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40d0930 con 0x7f51e415cc60 2023-10-21T16:18:39.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51ea5fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f51e40cdfd0 0x7f51e40ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38212/0 (socket says 172.21.15.53:38212) 2023-10-21T16:18:39.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51ea5fe640 1 -- 172.21.15.53:0/1843194048 learned_addr learned my addr 172.21.15.53:0/1843194048 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:39.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51e40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e95fc640 1 --2- 172.21.15.53:0/1843194048 >> v2:172.21.15.124:3300/0 conn(0x7f51e40d16f0 0x7f51e40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8843585 0 0) 0x7f51e415d040 con 0x7f51e40cdfd0 2023-10-21T16:18:39.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51cc003520 con 0x7f51e40cdfd0 2023-10-21T16:18:39.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814846045 0 0) 0x7f51cc003520 con 0x7f51e40cdfd0 2023-10-21T16:18:39.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 msgr2=0x7f51e40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:39.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.652+0000 7f51e8dfb640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51e40cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 >> v2:172.21.15.124:3300/0 conn(0x7f51e40d16f0 msgr2=0x7f51e40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 --2- 172.21.15.53:0/1843194048 >> v2:172.21.15.124:3300/0 conn(0x7f51e40d16f0 0x7f51e40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40d0c10 con 0x7f51e40cdfd0 2023-10-21T16:18:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f51dc001670 con 0x7f51e40cdfd0 2023-10-21T16:18:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 >> v1:172.21.15.53:6789/0 conn(0x7f51e40cdfd0 legacy=0x7f51e40ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc003f40 0x7f51cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc004d00 0x7f51cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40d0650 con 0x7f51cc004d00 2023-10-21T16:18:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40d0930 con 0x7f51cc003f40 2023-10-21T16:18:39.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc00b980 con 0x7f51e415cc60 2023-10-21T16:18:39.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc004d00 0x7f51cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 -- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 msgr2=0x7f51cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:39.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e415cc60 0x7f51cc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 -- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc003f40 msgr2=0x7f51cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:39.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc003f40 0x7f51cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.653+0000 7f51ea5fe640 1 -- 172.21.15.53:0/1843194048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc003520 con 0x7f51cc004d00 2023-10-21T16:18:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51ea5fe640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc004d00 0x7f51cc005180 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c190 tx=0x7f51cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d9098c96a105be server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc005ad0 con 0x7f51cc004d00 2023-10-21T16:18:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc005c30 con 0x7f51cc004d00 2023-10-21T16:18:39.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51e8dfb640 1 -- 172.21.15.53:0/1843194048 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc02c810 con 0x7f51cc004d00 2023-10-21T16:18:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51eadff640 1 -- 172.21.15.53:0/1843194048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc004d00 msgr2=0x7f51cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51eadff640 1 --2- 172.21.15.53:0/1843194048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51cc004d00 0x7f51cc005180 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c190 tx=0x7f51cc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.655+0000 7f51ea5fe640 1 -- 172.21.15.53:0/1843194048 reap_dead start 2023-10-21T16:18:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.656+0000 7f51eadff640 1 -- 172.21.15.53:0/1843194048 shutdown_connections 2023-10-21T16:18:39.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.656+0000 7f51eadff640 1 -- 172.21.15.53:0/1843194048 >> 172.21.15.53:0/1843194048 conn(0x7f51e409bcf0 msgr2=0x7f51e40d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:39.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.656+0000 7f51eadff640 1 -- 172.21.15.53:0/1843194048 shutdown_connections 2023-10-21T16:18:39.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.656+0000 7f51eadff640 1 -- 172.21.15.53:0/1843194048 wait complete. 2023-10-21T16:18:39.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.658+0000 7f51eadff640 1 Processor -- start 2023-10-21T16:18:39.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.658+0000 7f51eadff640 1 -- start start 2023-10-21T16:18:39.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 0x7f51e40b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e41f14b0 0x7f51e41f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e414c7c0 con 0x7f51e41f6020 2023-10-21T16:18:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41f1ea0 con 0x7f51cc004d00 2023-10-21T16:18:39.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.659+0000 7f51eadff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40b4030 con 0x7f51e41f14b0 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42740/0 (socket says 172.21.15.53:42740) 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 -- 172.21.15.53:0/467658133 learned_addr learned my addr 172.21.15.53:0/467658133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 0x7f51e40b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51ea5fe640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e41f14b0 0x7f51e41f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 -- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e41f14b0 msgr2=0x7f51e41f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e41f14b0 0x7f51e41f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 -- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 msgr2=0x7f51e40b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 0x7f51e40b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e95fc640 1 -- 172.21.15.53:0/467658133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40b4310 con 0x7f51e41f6020 2023-10-21T16:18:39.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.660+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 0x7f51e40b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51e95fc640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f51d800e7c0 tx=0x7f51d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800eed0 con 0x7f51e41f6020 2023-10-21T16:18:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800c940 con 0x7f51e41f6020 2023-10-21T16:18:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800ccf0 con 0x7f51e41f6020 2023-10-21T16:18:39.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e40b45f0 con 0x7f51e41f6020 2023-10-21T16:18:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.661+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e40b4bb0 con 0x7f51e41f6020 2023-10-21T16:18:39.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.662+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e404e160 con 0x7f51e41f6020 2023-10-21T16:18:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.663+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f51d8013470 con 0x7f51e41f6020 2023-10-21T16:18:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.664+0000 7f51e27fc640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 0x7f51d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.664+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f51d80896a0 con 0x7f51e41f6020 2023-10-21T16:18:39.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.665+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 0x7f51d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.666+0000 7f51e9dfd640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 0x7f51d006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51d4007520 tx=0x7f51d400b040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:39.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.670+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51d8054840 con 0x7f51e41f6020 2023-10-21T16:18:39.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.943+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f51e40b8590 con 0x7f51e41f6020 2023-10-21T16:18:39.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.944+0000 7f51e27fc640 1 -- 172.21.15.53:0/467658133 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f51d8057e60 con 0x7f51e41f6020 2023-10-21T16:18:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:39.950 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:18:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:18:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:18:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:18:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:18:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:18:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:18:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:18:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:18:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:18:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:18:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:18:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:18:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:18:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:18:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:18:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:18:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:18:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:18:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:18:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:18:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:18:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:18:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:18:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:18:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:18:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:18:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:18:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:18:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:18:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:18:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:18:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:18:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:18:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:18:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:18:39.985 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:18:39.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 msgr2=0x7f51d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 0x7f51d006d3c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f51d4007520 tx=0x7f51d400b040 comp rx=0 tx=0).stop 2023-10-21T16:18:39.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 msgr2=0x7f51e40afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:39.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f51d800e7c0 tx=0x7f51d800ec80 comp rx=0 tx=0).stop 2023-10-21T16:18:39.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 shutdown_connections 2023-10-21T16:18:39.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f51d006af00 0x7f51d006d3c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51e41f14b0 0x7f51e41f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51e41f6020 0x7f51e40afcf0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 --2- 172.21.15.53:0/467658133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51cc004d00 0x7f51e40b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.947+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 >> 172.21.15.53:0/467658133 conn(0x7f51e409bcf0 msgr2=0x7f51e415d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.948+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 shutdown_connections 2023-10-21T16:18:39.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:39.948+0000 7f51eadff640 1 -- 172.21.15.53:0/467658133 wait complete. 2023-10-21T16:18:39.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:39.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:39.977+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:40.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.224+0000 7f4f7166e640 1 Processor -- start 2023-10-21T16:18:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.224+0000 7f4f7166e640 1 -- start start 2023-10-21T16:18:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f7166e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f6c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f7166e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0cde30 con 0x7f4f6c157480 2023-10-21T16:18:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f7166e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c157860 con 0x7f4f6c0ce2c0 2023-10-21T16:18:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f7166e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c15e2c0 con 0x7f4f6c06b680 2023-10-21T16:18:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f6bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4f6c157480 0x7f4f6c06af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38226/0 (socket says 172.21.15.53:38226) 2023-10-21T16:18:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f6bfff640 1 -- 172.21.15.53:0/741391296 learned_addr learned my addr 172.21.15.53:0/741391296 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f63fff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f6c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.226+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610005285 0 0) 0x7f4f6c0cde30 con 0x7f4f6c157480 2023-10-21T16:18:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f58003520 con 0x7f4f6c157480 2023-10-21T16:18:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669371908 0 0) 0x7f4f58003520 con 0x7f4f6c157480 2023-10-21T16:18:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 msgr2=0x7f4f6c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f6c15d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 >> v1:172.21.15.124:6789/0 conn(0x7f4f6c0ce2c0 legacy=0x7f4f6c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0cd500 con 0x7f4f6c157480 2023-10-21T16:18:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4f5c001670 con 0x7f4f6c157480 2023-10-21T16:18:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 >> v1:172.21.15.53:6789/0 conn(0x7f4f6c157480 legacy=0x7f4f6c06af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f580047c0 0x7f4f58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f58005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c15e2c0 con 0x7f4f580039a0 2023-10-21T16:18:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800bb40 con 0x7f4f580047c0 2023-10-21T16:18:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.227+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5800bd00 con 0x7f4f6c06b680 2023-10-21T16:18:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f70e6d640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f58005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f63fff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f580047c0 0x7f4f58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 -- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 msgr2=0x7f4f58005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c06b680 0x7f4f58005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 -- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f580047c0 msgr2=0x7f4f58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f580047c0 0x7f4f58004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.228+0000 7f4f6bfff640 1 -- 172.21.15.53:0/741391296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58003520 con 0x7f4f580039a0 2023-10-21T16:18:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.229+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f58003dc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c0e0 tx=0x7f4f580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251cbb266e1375ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c005ad0 con 0x7f4f580039a0 2023-10-21T16:18:40.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c005c30 con 0x7f4f580039a0 2023-10-21T16:18:40.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f6b7fe640 1 -- 172.21.15.53:0/741391296 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c02c810 con 0x7f4f580039a0 2023-10-21T16:18:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 -- 172.21.15.53:0/741391296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 msgr2=0x7f4f58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 --2- 172.21.15.53:0/741391296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f58003dc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c0e0 tx=0x7f4f580057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f70e6d640 1 -- 172.21.15.53:0/741391296 reap_dead start 2023-10-21T16:18:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 -- 172.21.15.53:0/741391296 shutdown_connections 2023-10-21T16:18:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 -- 172.21.15.53:0/741391296 >> 172.21.15.53:0/741391296 conn(0x7f4f6c0d2ae0 msgr2=0x7f4f6c0d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 -- 172.21.15.53:0/741391296 shutdown_connections 2023-10-21T16:18:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.230+0000 7f4f7166e640 1 -- 172.21.15.53:0/741391296 wait complete. 2023-10-21T16:18:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.232+0000 7f4f7166e640 1 Processor -- start 2023-10-21T16:18:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.232+0000 7f4f7166e640 1 -- start start 2023-10-21T16:18:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c15c540 0x7f4f6c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f6c0cc000 0x7f4f6c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c15c0b0 con 0x7f4f580039a0 2023-10-21T16:18:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c15b4a0 con 0x7f4f6c0cc000 2023-10-21T16:18:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f7166e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c15b780 con 0x7f4f6c15c540 2023-10-21T16:18:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f6bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f6bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42752/0 (socket says 172.21.15.53:42752) 2023-10-21T16:18:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.234+0000 7f4f6bfff640 1 -- 172.21.15.53:0/2762147911 learned_addr learned my addr 172.21.15.53:0/2762147911 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 -- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c15c540 msgr2=0x7f4f6c0cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c15c540 0x7f4f6c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 -- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f6c0cc000 msgr2=0x7f4f6c0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f6c0cc000 0x7f4f6c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 -- 172.21.15.53:0/2762147911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c15ba60 con 0x7f4f580039a0 2023-10-21T16:18:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f6bfff640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c0e0 tx=0x7f4f5c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c03c040 con 0x7f4f580039a0 2023-10-21T16:18:40.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c02dcb0 con 0x7f4f580039a0 2023-10-21T16:18:40.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c15bd40 con 0x7f4f580039a0 2023-10-21T16:18:40.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.235+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c007270 con 0x7f4f580039a0 2023-10-21T16:18:40.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.236+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c0cfa30 con 0x7f4f580039a0 2023-10-21T16:18:40.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.237+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c04e160 con 0x7f4f580039a0 2023-10-21T16:18:40.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.238+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4f5c032070 con 0x7f4f580039a0 2023-10-21T16:18:40.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.239+0000 7f4f697fa640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 0x7f4f4c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.239+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f4f5c0aaf70 con 0x7f4f580039a0 2023-10-21T16:18:40.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.239+0000 7f4f63fff640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 0x7f4f4c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.240+0000 7f4f63fff640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 0x7f4f4c06d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0c9450 tx=0x7f4f54008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.243+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f5c0761a0 con 0x7f4f580039a0 2023-10-21T16:18:40.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.449+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f6c0d21e0 con 0x7f4f580039a0 2023-10-21T16:18:40.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.451+0000 7f4f697fa640 1 -- 172.21.15.53:0/2762147911 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13130 (secure 0 0 0) 0x7f4f5c0797c0 con 0x7f4f580039a0 2023-10-21T16:18:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:40.453 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":113,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:39.423124+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:40.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.453+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 msgr2=0x7f4f4c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.453+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 0x7f4f4c06d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0c9450 tx=0x7f4f54008040 comp rx=0 tx=0).stop 2023-10-21T16:18:40.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 msgr2=0x7f4f6c1524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c0e0 tx=0x7f4f5c001930 comp rx=0 tx=0).stop 2023-10-21T16:18:40.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 shutdown_connections 2023-10-21T16:18:40.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4f4c06af30 0x7f4f4c06d3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f6c0cc000 0x7f4f6c0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f6c15c540 0x7f4f6c0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 --2- 172.21.15.53:0/2762147911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f580039a0 0x7f4f6c1524a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 >> 172.21.15.53:0/2762147911 conn(0x7f4f6c0d2ae0 msgr2=0x7f4f6c0b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:40.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 shutdown_connections 2023-10-21T16:18:40.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.454+0000 7f4f7166e640 1 -- 172.21.15.53:0/2762147911 wait complete. 2023-10-21T16:18:40.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:18:40.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:18:40.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.707+0000 7f3dd3602640 1 Processor -- start 2023-10-21T16:18:40.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.707+0000 7f3dd3602640 1 -- start start 2023-10-21T16:18:40.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.708+0000 7f3dd3602640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3dcc15aad0 0x7f3dcc0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.708+0000 7f3dd3602640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd3602640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc15bf30 con 0x7f3dcc06cd00 2023-10-21T16:18:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd3602640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc15b320 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd3602640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc15b600 con 0x7f3dcc15aad0 2023-10-21T16:18:40.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd0b9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd0b9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42754/0 (socket says 172.21.15.53:42754) 2023-10-21T16:18:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd1b9f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3dcc06cd00 0x7f3dcc070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38234/0 (socket says 172.21.15.53:38234) 2023-10-21T16:18:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/3116031545 learned_addr learned my addr 172.21.15.53:0/3116031545 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:40.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd139e640 1 --2- 172.21.15.53:0/3116031545 >> v2:172.21.15.124:3300/0 conn(0x7f3dcc15aad0 0x7f3dcc0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/3116031545 >> v2:172.21.15.124:3300/0 conn(0x7f3dcc15aad0 msgr2=0x7f3dcc0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:40.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.709+0000 7f3dd0b9d640 1 --2- 172.21.15.53:0/3116031545 >> v2:172.21.15.124:3300/0 conn(0x7f3dcc15aad0 0x7f3dcc0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.710+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/3116031545 >> v1:172.21.15.53:6789/0 conn(0x7f3dcc06cd00 legacy=0x7f3dcc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.710+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/3116031545 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc15b8e0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.710+0000 7f3dc3fff640 1 -- 172.21.15.53:0/3116031545 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242328983 0 0) 0x7f3dcc15bf30 con 0x7f3dcc06cd00 2023-10-21T16:18:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.711+0000 7f3dd0b9d640 1 --2- 172.21.15.53:0/3116031545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc06e6a0 tx=0x7f3dbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaddf2a417909c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.711+0000 7f3dc3fff640 1 -- 172.21.15.53:0/3116031545 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc030860 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.711+0000 7f3dc3fff640 1 -- 172.21.15.53:0/3116031545 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0309c0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.711+0000 7f3dc3fff640 1 -- 172.21.15.53:0/3116031545 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc030d50 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 -- 172.21.15.53:0/3116031545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 msgr2=0x7f3dcc0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 --2- 172.21.15.53:0/3116031545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc06e6a0 tx=0x7f3dbc02b9a0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 -- 172.21.15.53:0/3116031545 shutdown_connections 2023-10-21T16:18:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 --2- 172.21.15.53:0/3116031545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc0c8710 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 --2- 172.21.15.53:0/3116031545 >> v2:172.21.15.124:3300/0 conn(0x7f3dcc15aad0 0x7f3dcc0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 -- 172.21.15.53:0/3116031545 >> 172.21.15.53:0/3116031545 conn(0x7f3dcc069d50 msgr2=0x7f3dcc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 -- 172.21.15.53:0/3116031545 shutdown_connections 2023-10-21T16:18:40.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.712+0000 7f3dd3602640 1 -- 172.21.15.53:0/3116031545 wait complete. 2023-10-21T16:18:40.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.714+0000 7f3dd3602640 1 Processor -- start 2023-10-21T16:18:40.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.714+0000 7f3dd3602640 1 -- start start 2023-10-21T16:18:40.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3dcc06cd00 0x7f3dcc1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3dcc15aad0 0x7f3dcc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc1f66a0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0cbe60 con 0x7f3dcc15aad0 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.715+0000 7f3dd3602640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0cb250 con 0x7f3dcc06cd00 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42758/0 (socket says 172.21.15.53:42758) 2023-10-21T16:18:40.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/419902515 learned_addr learned my addr 172.21.15.53:0/419902515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:40.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd1b9f640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3dcc15aad0 0x7f3dcc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3dcc06cd00 msgr2=0x7f3dcc1f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd139e640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3dcc06cd00 0x7f3dcc1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3dcc06cd00 0x7f3dcc1f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3dcc15aad0 msgr2=0x7f3dcc0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3dcc15aad0 0x7f3dcc0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.716+0000 7f3dd0b9d640 1 -- 172.21.15.53:0/419902515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0094d0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dd0b9d640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc02b9e0 tx=0x7f3dbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc039d60 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc043400 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0437b0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc0cb530 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.717+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc0cbaf0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.718+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc0cbc80 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.719+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3dbc043910 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.720+0000 7f3dc1ffb640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 0x7f3db006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:40.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.720+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f3dbc03e070 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.721+0000 7f3dd139e640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 0x7f3db006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:40.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.722+0000 7f3dd139e640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 0x7f3db006d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006fd0 tx=0x7f3dc8008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:40.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.725+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dbc077a80 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.924+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3dcc0cbfd0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.925+0000 7f3dc1ffb640 1 -- 172.21.15.53:0/419902515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dbc07b0a0 con 0x7f3dcc0c82b0 2023-10-21T16:18:40.927 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:18:40.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 msgr2=0x7f3db006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 0x7f3db006d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006fd0 tx=0x7f3dc8008040 comp rx=0 tx=0).stop 2023-10-21T16:18:40.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 msgr2=0x7f3dcc1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:40.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc02b9e0 tx=0x7f3dbc004730 comp rx=0 tx=0).stop 2023-10-21T16:18:40.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 shutdown_connections 2023-10-21T16:18:40.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f3db006af00 0x7f3db006d3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3dcc15aad0 0x7f3dcc0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3dcc0c82b0 0x7f3dcc1589f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 --2- 172.21.15.53:0/419902515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3dcc06cd00 0x7f3dcc1f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:40.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 >> 172.21.15.53:0/419902515 conn(0x7f3dcc069d50 msgr2=0x7f3dcc15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:40.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 shutdown_connections 2023-10-21T16:18:40.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:40.928+0000 7f3dd3602640 1 -- 172.21.15.53:0/419902515 wait complete. 2023-10-21T16:18:40.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:18:41.127 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:41.125+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:41.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.179+0000 7fda139a9640 1 Processor -- start 2023-10-21T16:18:41.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.179+0000 7fda139a9640 1 -- start start 2023-10-21T16:18:41.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.181+0000 7fda139a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.181+0000 7fda139a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda0c160940 0x7fda0c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.181+0000 7fda139a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c0c48f0 con 0x7fda0c070d50 2023-10-21T16:18:41.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.181+0000 7fda139a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0c0c4d80 con 0x7fda0c160940 2023-10-21T16:18:41.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.181+0000 7fda139a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0c3ce0 con 0x7fda0c15cd90 2023-10-21T16:18:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.182+0000 7fda11745640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.182+0000 7fda11745640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:56064/0 (socket says 172.21.15.53:56064) 2023-10-21T16:18:41.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.182+0000 7fda11745640 1 -- 172.21.15.53:0/2122677891 learned_addr learned my addr 172.21.15.53:0/2122677891 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.182+0000 7fda10f44640 1 --2- 172.21.15.53:0/2122677891 >> v2:172.21.15.124:3300/0 conn(0x7fda0c160940 0x7fda0c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.183+0000 7fda11745640 1 -- 172.21.15.53:0/2122677891 >> v2:172.21.15.124:3300/0 conn(0x7fda0c160940 msgr2=0x7fda0c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.183+0000 7fda11745640 1 --2- 172.21.15.53:0/2122677891 >> v2:172.21.15.124:3300/0 conn(0x7fda0c160940 0x7fda0c070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.183+0000 7fda11745640 1 -- 172.21.15.53:0/2122677891 >> v1:172.21.15.124:6789/0 conn(0x7fda0c070d50 legacy=0x7fda0c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.183+0000 7fda11745640 1 -- 172.21.15.53:0/2122677891 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0c3fc0 con 0x7fda0c15cd90 2023-10-21T16:18:41.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.184+0000 7fda11745640 1 --2- 172.21.15.53:0/2122677891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda08009870 tx=0x7fda0802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6813be233002921e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.185+0000 7fda03fff640 1 -- 172.21.15.53:0/2122677891 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08030840 con 0x7fda0c15cd90 2023-10-21T16:18:41.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.185+0000 7fda03fff640 1 -- 172.21.15.53:0/2122677891 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda080309a0 con 0x7fda0c15cd90 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.185+0000 7fda03fff640 1 -- 172.21.15.53:0/2122677891 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08030d30 con 0x7fda0c15cd90 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.185+0000 7fda139a9640 1 -- 172.21.15.53:0/2122677891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 msgr2=0x7fda0c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 --2- 172.21.15.53:0/2122677891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda08009870 tx=0x7fda0802b980 comp rx=0 tx=0).stop 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 -- 172.21.15.53:0/2122677891 shutdown_connections 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 --2- 172.21.15.53:0/2122677891 >> v2:172.21.15.124:3300/0 conn(0x7fda0c160940 0x7fda0c070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 --2- 172.21.15.53:0/2122677891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c15cd90 0x7fda0c0702d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 -- 172.21.15.53:0/2122677891 >> 172.21.15.53:0/2122677891 conn(0x7fda0c06a380 msgr2=0x7fda0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:41.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 -- 172.21.15.53:0/2122677891 shutdown_connections 2023-10-21T16:18:41.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.186+0000 7fda139a9640 1 -- 172.21.15.53:0/2122677891 wait complete. 2023-10-21T16:18:41.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.188+0000 7fda139a9640 1 Processor -- start 2023-10-21T16:18:41.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.188+0000 7fda139a9640 1 -- start start 2023-10-21T16:18:41.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 0x7fda0c156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c160940 0x7fda0c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c15e6d0 con 0x7fda0c070d50 2023-10-21T16:18:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c15e9b0 con 0x7fda0c15cd90 2023-10-21T16:18:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda139a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0b2970 con 0x7fda0c160940 2023-10-21T16:18:41.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda11745640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda11745640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42770/0 (socket says 172.21.15.53:42770) 2023-10-21T16:18:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.189+0000 7fda11745640 1 -- 172.21.15.53:0/2522410669 learned_addr learned my addr 172.21.15.53:0/2522410669 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda10f44640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 0x7fda0c156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 -- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c160940 msgr2=0x7fda0c156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c160940 0x7fda0c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 -- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 msgr2=0x7fda0c156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 0x7fda0c156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 -- 172.21.15.53:0/2522410669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080094d0 con 0x7fda0c070d50 2023-10-21T16:18:41.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda10f44640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 0x7fda0c156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:41.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.190+0000 7fda11745640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.191+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08039c90 con 0x7fda0c070d50 2023-10-21T16:18:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.191+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08039e10 con 0x7fda0c070d50 2023-10-21T16:18:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.191+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08043640 con 0x7fda0c070d50 2023-10-21T16:18:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.191+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c0b2c50 con 0x7fda0c070d50 2023-10-21T16:18:41.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.191+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c0c8db0 con 0x7fda0c070d50 2023-10-21T16:18:41.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.192+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c0b2e70 con 0x7fda0c070d50 2023-10-21T16:18:41.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.193+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fda080437a0 con 0x7fda0c070d50 2023-10-21T16:18:41.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.194+0000 7fda01ffb640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 0x7fd9f006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.194+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fda0803e070 con 0x7fda0c070d50 2023-10-21T16:18:41.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.195+0000 7fda10f44640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 0x7fd9f006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.196+0000 7fda10f44640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 0x7fd9f006d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0b27f0 tx=0x7fd9fc008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.198+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda08077720 con 0x7fda0c070d50 2023-10-21T16:18:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.412+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fda0c067490 con 0x7fda0c070d50 2023-10-21T16:18:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.413+0000 7fda01ffb640 1 -- 172.21.15.53:0/2522410669 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7fda0807ad40 con 0x7fda0c070d50 2023-10-21T16:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:18:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 msgr2=0x7fd9f006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 0x7fd9f006d3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0b27f0 tx=0x7fd9fc008040 comp rx=0 tx=0).stop 2023-10-21T16:18:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 msgr2=0x7fda0c14e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fda08000c00 tx=0x7fda08004730 comp rx=0 tx=0).stop 2023-10-21T16:18:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 shutdown_connections 2023-10-21T16:18:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fd9f006af30 0x7fd9f006d3f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda0c160940 0x7fda0c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c15cd90 0x7fda0c156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 --2- 172.21.15.53:0/2522410669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0c070d50 0x7fda0c14e5c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 >> 172.21.15.53:0/2522410669 conn(0x7fda0c06a380 msgr2=0x7fda0c0ab9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:41.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 shutdown_connections 2023-10-21T16:18:41.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.416+0000 7fda139a9640 1 -- 172.21.15.53:0/2522410669 wait complete. 2023-10-21T16:18:41.439 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:18:41.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:18:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.673+0000 7f0ced083640 1 Processor -- start 2023-10-21T16:18:41.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.673+0000 7f0ced083640 1 -- start start 2023-10-21T16:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.675+0000 7f0ced083640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ce80b1750 0x7f0ce806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0ced083640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce806d670 con 0x7f0ce815aad0 2023-10-21T16:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0ced083640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce815bf30 con 0x7f0ce806d240 2023-10-21T16:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0ced083640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce80b1bb0 con 0x7f0ce80b1750 2023-10-21T16:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0cde59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ce80b1750 0x7f0ce806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0cde59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0ce80b1750 0x7f0ce806cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42774/0 (socket says 172.21.15.53:42774) 2023-10-21T16:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0ce6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0ce815aad0 0x7f0ce80c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38250/0 (socket says 172.21.15.53:38250) 2023-10-21T16:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0cde59c640 1 -- 172.21.15.53:0/2119184137 learned_addr learned my addr 172.21.15.53:0/2119184137 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:41.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0cde59c640 1 -- 172.21.15.53:0/2119184137 >> v1:172.21.15.124:6789/0 conn(0x7f0ce806d240 legacy=0x7f0ce8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:41.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.676+0000 7f0cde59c640 1 -- 172.21.15.53:0/2119184137 >> v1:172.21.15.53:6789/0 conn(0x7f0ce815aad0 legacy=0x7f0ce80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.677+0000 7f0cde59c640 1 -- 172.21.15.53:0/2119184137 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce815b320 con 0x7f0ce80b1750 2023-10-21T16:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.677+0000 7f0ce659c640 1 -- 172.21.15.53:0/2119184137 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414065881 0 0) 0x7f0ce806d670 con 0x7f0ce815aad0 2023-10-21T16:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.678+0000 7f0cde59c640 1 --2- 172.21.15.53:0/2119184137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce80b1750 0x7f0ce806cd00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009e50 tx=0x7f0cd002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1833dcd789c463 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.678+0000 7f0ce659c640 1 -- 172.21.15.53:0/2119184137 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd002cb20 con 0x7f0ce80b1750 2023-10-21T16:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.678+0000 7f0ce659c640 1 -- 172.21.15.53:0/2119184137 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd002cc80 con 0x7f0ce80b1750 2023-10-21T16:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.678+0000 7f0ce659c640 1 -- 172.21.15.53:0/2119184137 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00343f0 con 0x7f0ce80b1750 2023-10-21T16:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 -- 172.21.15.53:0/2119184137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce80b1750 msgr2=0x7f0ce806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 --2- 172.21.15.53:0/2119184137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce80b1750 0x7f0ce806cd00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009e50 tx=0x7f0cd002bf80 comp rx=0 tx=0).stop 2023-10-21T16:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 -- 172.21.15.53:0/2119184137 shutdown_connections 2023-10-21T16:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 --2- 172.21.15.53:0/2119184137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce80b1750 0x7f0ce806cd00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 -- 172.21.15.53:0/2119184137 >> 172.21.15.53:0/2119184137 conn(0x7f0ce8069d50 msgr2=0x7f0ce806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 -- 172.21.15.53:0/2119184137 shutdown_connections 2023-10-21T16:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.679+0000 7f0ced083640 1 -- 172.21.15.53:0/2119184137 wait complete. 2023-10-21T16:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.680+0000 7f0ced083640 1 Processor -- start 2023-10-21T16:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.681+0000 7f0ced083640 1 -- start start 2023-10-21T16:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ce806d240 0x7f0ce8151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ce80b1750 0x7f0ce815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8157340 con 0x7f0ce815aad0 2023-10-21T16:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80c8dd0 con 0x7f0ce80b1750 2023-10-21T16:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ced083640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80c90b0 con 0x7f0ce806d240 2023-10-21T16:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42790/0 (socket says 172.21.15.53:42790) 2023-10-21T16:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 -- 172.21.15.53:0/4100640576 learned_addr learned my addr 172.21.15.53:0/4100640576 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0cde59c640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ce80b1750 0x7f0ce815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 -- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ce806d240 msgr2=0x7f0ce8151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ce806d240 0x7f0ce8151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 -- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ce80b1750 msgr2=0x7f0ce815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ce80b1750 0x7f0ce815c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.682+0000 7f0ce759e640 1 -- 172.21.15.53:0/4100640576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0009b30 con 0x7f0ce815aad0 2023-10-21T16:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0ce759e640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c410 tx=0x7f0ce000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0011790 con 0x7f0ce815aad0 2023-10-21T16:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00118f0 con 0x7f0ce815aad0 2023-10-21T16:18:41.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0011ce0 con 0x7f0ce815aad0 2023-10-21T16:18:41.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce80c9390 con 0x7f0ce815aad0 2023-10-21T16:18:41.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.683+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce8151b80 con 0x7f0ce815aad0 2023-10-21T16:18:41.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.685+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce80c98b0 con 0x7f0ce815aad0 2023-10-21T16:18:41.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.686+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0ce001c460 con 0x7f0ce815aad0 2023-10-21T16:18:41.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.687+0000 7f0cdffff640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 0x7f0cc806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:41.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.687+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0ce00893f0 con 0x7f0ce815aad0 2023-10-21T16:18:41.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.687+0000 7f0ce6d9d640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 0x7f0cc806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:41.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.688+0000 7f0ce6d9d640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 0x7f0cc806d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8007fd0 tx=0x7f0cd8007980 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:41.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.691+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ce80c98b0 con 0x7f0ce815aad0 2023-10-21T16:18:41.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:41.895+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0ce80c98b0 con 0x7f0ce815aad0 2023-10-21T16:18:42.284 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:42.282+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:42.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.435+0000 7f0cdffff640 1 -- 172.21.15.53:0/4100640576 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ce80c98b0 con 0x7f0ce815aad0 2023-10-21T16:18:42.437 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:18:42.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.440+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 msgr2=0x7f0cc806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.440+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 0x7f0cc806d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8007fd0 tx=0x7f0cd8007980 comp rx=0 tx=0).stop 2023-10-21T16:18:42.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.440+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 msgr2=0x7f0ce815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.440+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000c410 tx=0x7f0ce000c8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 shutdown_connections 2023-10-21T16:18:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cc806af30 0x7f0cc806d3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ce815aad0 0x7f0ce815c600 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ce80b1750 0x7f0ce815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 --2- 172.21.15.53:0/4100640576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ce806d240 0x7f0ce8151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 >> 172.21.15.53:0/4100640576 conn(0x7f0ce8069d50 msgr2=0x7f0ce8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 shutdown_connections 2023-10-21T16:18:42.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.441+0000 7f0ced083640 1 -- 172.21.15.53:0/4100640576 wait complete. 2023-10-21T16:18:42.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:18:42.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.688+0000 7f27afa7c640 1 Processor -- start 2023-10-21T16:18:42.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.688+0000 7f27afa7c640 1 -- start start 2023-10-21T16:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.689+0000 7f27afa7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27a80cb500 0x7f27a80cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.689+0000 7f27afa7c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a80c8c60 con 0x7f27a8070690 2023-10-21T16:18:42.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.689+0000 7f27afa7c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a8071250 con 0x7f27a806cfa0 2023-10-21T16:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.689+0000 7f27afa7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a80cbe20 con 0x7f27a80cb500 2023-10-21T16:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.689+0000 7f27ad818640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27a80cb500 0x7f27a80cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ad818640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27a80cb500 0x7f27a80cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42802/0 (socket says 172.21.15.53:42802) 2023-10-21T16:18:42.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ad818640 1 -- 172.21.15.53:0/608231586 learned_addr learned my addr 172.21.15.53:0/608231586 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:42.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ad818640 1 -- 172.21.15.53:0/608231586 >> v1:172.21.15.124:6789/0 conn(0x7f27a806cfa0 legacy=0x7f27a815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ad818640 1 -- 172.21.15.53:0/608231586 >> v1:172.21.15.53:6789/0 conn(0x7f27a8070690 legacy=0x7f27a8070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ad818640 1 -- 172.21.15.53:0/608231586 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80b1ac0 con 0x7f27a80cb500 2023-10-21T16:18:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.690+0000 7f27ac816640 1 -- 172.21.15.53:0/608231586 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466876381 0 0) 0x7f27a80c8c60 con 0x7f27a8070690 2023-10-21T16:18:42.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.692+0000 7f27ad818640 1 --2- 172.21.15.53:0/608231586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a80cb500 0x7f27a80cb8e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009870 tx=0x7f27a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552fdb241cfa2262 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.692+0000 7f27ac816640 1 -- 172.21.15.53:0/608231586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031870 con 0x7f27a80cb500 2023-10-21T16:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.692+0000 7f27ac816640 1 -- 172.21.15.53:0/608231586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a40319d0 con 0x7f27a80cb500 2023-10-21T16:18:42.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.692+0000 7f27ac816640 1 -- 172.21.15.53:0/608231586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4031d60 con 0x7f27a80cb500 2023-10-21T16:18:42.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 -- 172.21.15.53:0/608231586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a80cb500 msgr2=0x7f27a80cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 --2- 172.21.15.53:0/608231586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a80cb500 0x7f27a80cb8e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009870 tx=0x7f27a402b910 comp rx=0 tx=0).stop 2023-10-21T16:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 -- 172.21.15.53:0/608231586 shutdown_connections 2023-10-21T16:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 --2- 172.21.15.53:0/608231586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a80cb500 0x7f27a80cb8e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 -- 172.21.15.53:0/608231586 >> 172.21.15.53:0/608231586 conn(0x7f27a806a140 msgr2=0x7f27a806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 -- 172.21.15.53:0/608231586 shutdown_connections 2023-10-21T16:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.693+0000 7f27afa7c640 1 -- 172.21.15.53:0/608231586 wait complete. 2023-10-21T16:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.695+0000 7f27afa7c640 1 Processor -- start 2023-10-21T16:18:42.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.695+0000 7f27afa7c640 1 -- start start 2023-10-21T16:18:42.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27a806cfa0 0x7f27a81f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:42.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:42.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27a80cb500 0x7f27a81e1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:42.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a81592a0 con 0x7f27a8070690 2023-10-21T16:18:42.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80ac630 con 0x7f27a80cb500 2023-10-21T16:18:42.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27afa7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80ac910 con 0x7f27a806cfa0 2023-10-21T16:18:42.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42806/0 (socket says 172.21.15.53:42806) 2023-10-21T16:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 -- 172.21.15.53:0/1997699995 learned_addr learned my addr 172.21.15.53:0/1997699995 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ae019640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27a80cb500 0x7f27a81e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad818640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27a806cfa0 0x7f27a81f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:42.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 -- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27a806cfa0 msgr2=0x7f27a81f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27a806cfa0 0x7f27a81f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 -- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27a80cb500 msgr2=0x7f27a81e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:42.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27a80cb500 0x7f27a81e1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:42.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.696+0000 7f27ad017640 1 -- 172.21.15.53:0/1997699995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40094d0 con 0x7f27a8070690 2023-10-21T16:18:42.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f27ad017640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f279800b3c0 tx=0x7f279800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:42.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798011830 con 0x7f27a8070690 2023-10-21T16:18:42.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798011990 con 0x7f27a8070690 2023-10-21T16:18:42.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798011d80 con 0x7f27a8070690 2023-10-21T16:18:42.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a80acbf0 con 0x7f27a8070690 2023-10-21T16:18:42.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.697+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a80ad1b0 con 0x7f27a8070690 2023-10-21T16:18:42.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.698+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a80ad3d0 con 0x7f27a8070690 2023-10-21T16:18:42.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.699+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f279801b460 con 0x7f27a8070690 2023-10-21T16:18:42.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.700+0000 7f279e7fc640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 0x7f278c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:42.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.700+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f27980891d0 con 0x7f27a8070690 2023-10-21T16:18:42.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.701+0000 7f27ad818640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 0x7f278c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:42.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.702+0000 7f27ad818640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 0x7f278c06d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27a4036000 tx=0x7f27a403c040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:42.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.705+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2798054300 con 0x7f27a8070690 2023-10-21T16:18:42.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:42.919+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f27a8071cb0 con 0x7f27a8070690 2023-10-21T16:18:43.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.442+0000 7f279e7fc640 1 -- 172.21.15.53:0/1997699995 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2798057920 con 0x7f27a8070690 2023-10-21T16:18:43.444 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:18:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 msgr2=0x7f278c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 0x7f278c06d3c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27a4036000 tx=0x7f27a403c040 comp rx=0 tx=0).stop 2023-10-21T16:18:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 msgr2=0x7f27a80adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f279800b3c0 tx=0x7f279800b880 comp rx=0 tx=0).stop 2023-10-21T16:18:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 shutdown_connections 2023-10-21T16:18:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f278c06af00 0x7f278c06d3c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27a80cb500 0x7f27a81e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27a8070690 0x7f27a80adf50 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 --2- 172.21.15.53:0/1997699995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27a806cfa0 0x7f27a81f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.446+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 >> 172.21.15.53:0/1997699995 conn(0x7f27a806a140 msgr2=0x7f27a80cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.447+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 shutdown_connections 2023-10-21T16:18:43.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.447+0000 7f27afa7c640 1 -- 172.21.15.53:0/1997699995 wait complete. 2023-10-21T16:18:43.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:18:43.497 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:43.496+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.680+0000 7f2da78cf640 1 Processor -- start 2023-10-21T16:18:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.680+0000 7f2da78cf640 1 -- start start 2023-10-21T16:18:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.681+0000 7f2da78cf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2da01601b0 0x7f2da00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.681+0000 7f2da78cf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2da006a460 0x7f2da00ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.681+0000 7f2da78cf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00ce210 con 0x7f2da0071c30 2023-10-21T16:18:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.681+0000 7f2da78cf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00cdee0 con 0x7f2da006a460 2023-10-21T16:18:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.681+0000 7f2da78cf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da0161530 con 0x7f2da01601b0 2023-10-21T16:18:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2da006a460 0x7f2da00ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2da006a460 0x7f2da00ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42816/0 (socket says 172.21.15.53:42816) 2023-10-21T16:18:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 -- 172.21.15.53:0/685570342 learned_addr learned my addr 172.21.15.53:0/685570342 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da4e6a640 1 --1- 172.21.15.53:0/685570342 >> v1:172.21.15.53:6789/0 conn(0x7f2da0071c30 0x7f2da0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38270/0 (socket says 172.21.15.53:38270) 2023-10-21T16:18:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da566b640 1 --2- 172.21.15.53:0/685570342 >> v2:172.21.15.124:3300/0 conn(0x7f2da01601b0 0x7f2da00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 -- 172.21.15.53:0/685570342 >> v2:172.21.15.124:3300/0 conn(0x7f2da01601b0 msgr2=0x7f2da00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 --2- 172.21.15.53:0/685570342 >> v2:172.21.15.124:3300/0 conn(0x7f2da01601b0 0x7f2da00cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2da5e6c640 1 -- 172.21.15.53:0/685570342 >> v1:172.21.15.53:6789/0 conn(0x7f2da0071c30 legacy=0x7f2da0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.683+0000 7f2da5e6c640 1 -- 172.21.15.53:0/685570342 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0161810 con 0x7f2da006a460 2023-10-21T16:18:43.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.682+0000 7f2d97fff640 1 -- 172.21.15.53:0/685570342 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926433995 0 0) 0x7f2da00ce210 con 0x7f2da0071c30 2023-10-21T16:18:43.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.684+0000 7f2da5e6c640 1 --2- 172.21.15.53:0/685570342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da006a460 0x7f2da00ca090 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2da0067110 tx=0x7f2d9802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e88f26ca7a5a48c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:43.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.685+0000 7f2d97fff640 1 -- 172.21.15.53:0/685570342 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9802cb10 con 0x7f2da006a460 2023-10-21T16:18:43.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.685+0000 7f2d97fff640 1 -- 172.21.15.53:0/685570342 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9802cc70 con 0x7f2da006a460 2023-10-21T16:18:43.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.685+0000 7f2d97fff640 1 -- 172.21.15.53:0/685570342 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980343f0 con 0x7f2da006a460 2023-10-21T16:18:43.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.685+0000 7f2da78cf640 1 -- 172.21.15.53:0/685570342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da006a460 msgr2=0x7f2da00ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.685+0000 7f2da78cf640 1 --2- 172.21.15.53:0/685570342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da006a460 0x7f2da00ca090 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2da0067110 tx=0x7f2d9802bf70 comp rx=0 tx=0).stop 2023-10-21T16:18:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 -- 172.21.15.53:0/685570342 shutdown_connections 2023-10-21T16:18:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 --2- 172.21.15.53:0/685570342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da006a460 0x7f2da00ca090 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 --2- 172.21.15.53:0/685570342 >> v2:172.21.15.124:3300/0 conn(0x7f2da01601b0 0x7f2da00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 -- 172.21.15.53:0/685570342 >> 172.21.15.53:0/685570342 conn(0x7f2da006d430 msgr2=0x7f2da006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:43.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 -- 172.21.15.53:0/685570342 shutdown_connections 2023-10-21T16:18:43.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.686+0000 7f2da78cf640 1 -- 172.21.15.53:0/685570342 wait complete. 2023-10-21T16:18:43.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.688+0000 7f2da78cf640 1 Processor -- start 2023-10-21T16:18:43.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.688+0000 7f2da78cf640 1 -- start start 2023-10-21T16:18:43.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 0x7f2da01f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2da0071c30 0x7f2da00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00c3da0 con 0x7f2da01601b0 2023-10-21T16:18:43.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01566a0 con 0x7f2da0071c30 2023-10-21T16:18:43.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da78cf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00c48e0 con 0x7f2da006a460 2023-10-21T16:18:43.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42824/0 (socket says 172.21.15.53:42824) 2023-10-21T16:18:43.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 -- 172.21.15.53:0/2859159584 learned_addr learned my addr 172.21.15.53:0/2859159584 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:43.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da566b640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 0x7f2da01f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da4e6a640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2da0071c30 0x7f2da00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 -- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 msgr2=0x7f2da01f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 0x7f2da01f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 -- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2da0071c30 msgr2=0x7f2da00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:43.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2da0071c30 0x7f2da00c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:43.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da5e6c640 1 -- 172.21.15.53:0/2859159584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009b30 con 0x7f2da01601b0 2023-10-21T16:18:43.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.690+0000 7f2da566b640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 0x7f2da01f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:43.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2da5e6c640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009f40 tx=0x7f2d98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:43.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98009510 con 0x7f2da01601b0 2023-10-21T16:18:43.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9803e450 con 0x7f2da01601b0 2023-10-21T16:18:43.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9803e7e0 con 0x7f2da01601b0 2023-10-21T16:18:43.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00c3fb0 con 0x7f2da01601b0 2023-10-21T16:18:43.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.691+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00c4570 con 0x7f2da01601b0 2023-10-21T16:18:43.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.693+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da00cdcb0 con 0x7f2da01601b0 2023-10-21T16:18:43.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.694+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2d9803e940 con 0x7f2da01601b0 2023-10-21T16:18:43.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.695+0000 7f2d95ffb640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 0x7f2d8406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:43.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.695+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f2d980ac330 con 0x7f2da01601b0 2023-10-21T16:18:43.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.696+0000 7f2da566b640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 0x7f2d8406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:43.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.697+0000 7f2da566b640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 0x7f2d8406d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c006fd0 tx=0x7f2d9c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:43.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.700+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d980771e0 con 0x7f2da01601b0 2023-10-21T16:18:43.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:43.936+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2da00c4700 con 0x7f2da01601b0 2023-10-21T16:18:44.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.473+0000 7f2d95ffb640 1 -- 172.21.15.53:0/2859159584 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2d9807a800 con 0x7f2da01601b0 2023-10-21T16:18:44.475 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 13 and data pool 14 2023-10-21T16:18:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.477+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 msgr2=0x7f2d8406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.477+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 0x7f2d8406d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c006fd0 tx=0x7f2d9c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.477+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 msgr2=0x7f2da00c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.477+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009f40 tx=0x7f2d98004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 shutdown_connections 2023-10-21T16:18:44.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f2d8406af30 0x7f2d8406d3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2da01601b0 0x7f2da00c4d70 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2da0071c30 0x7f2da00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 --2- 172.21.15.53:0/2859159584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2da006a460 0x7f2da01f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 >> 172.21.15.53:0/2859159584 conn(0x7f2da006d430 msgr2=0x7f2da0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:44.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 shutdown_connections 2023-10-21T16:18:44.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.478+0000 7f2da78cf640 1 -- 172.21.15.53:0/2859159584 wait complete. 2023-10-21T16:18:44.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:44.691 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:44.688+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.714+0000 7f4c6fe90640 1 Processor -- start 2023-10-21T16:18:44.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.715+0000 7f4c6fe90640 1 -- start start 2023-10-21T16:18:44.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.716+0000 7f4c6fe90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c6815cc20 0x7f4c6806ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.716+0000 7f4c6fe90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4c6815d000 0x7f4c680cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.716+0000 7f4c6fe90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6806b3a0 con 0x7f4c680cf480 2023-10-21T16:18:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.716+0000 7f4c6fe90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c68161750 con 0x7f4c6815cc20 2023-10-21T16:18:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.716+0000 7f4c6fe90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680cb5c0 con 0x7f4c6815d000 2023-10-21T16:18:44.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c6815cc20 0x7f4c6806ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c6815cc20 0x7f4c6806ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47314/0 (socket says 172.21.15.53:47314) 2023-10-21T16:18:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6e42d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4c680cf480 0x7f4c680cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34564/0 (socket says 172.21.15.53:34564) 2023-10-21T16:18:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/3080490828 learned_addr learned my addr 172.21.15.53:0/3080490828 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6d42b640 1 --2- 172.21.15.53:0/3080490828 >> v2:172.21.15.124:3300/0 conn(0x7f4c6815d000 0x7f4c680cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/3080490828 >> v2:172.21.15.124:3300/0 conn(0x7f4c6815d000 msgr2=0x7f4c680cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 --2- 172.21.15.53:0/3080490828 >> v2:172.21.15.124:3300/0 conn(0x7f4c6815d000 0x7f4c680cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/3080490828 >> v1:172.21.15.53:6789/0 conn(0x7f4c680cf480 legacy=0x7f4c680cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6cc2a640 1 -- 172.21.15.53:0/3080490828 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095635924 0 0) 0x7f4c6806b3a0 con 0x7f4c680cf480 2023-10-21T16:18:44.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.717+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/3080490828 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680cb8a0 con 0x7f4c6815cc20 2023-10-21T16:18:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.718+0000 7f4c6dc2c640 1 --2- 172.21.15.53:0/3080490828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c6815cc20 0x7f4c6806ae60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4c680ce4b0 tx=0x7f4c6402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9821b8eb3b8efd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6cc2a640 1 -- 172.21.15.53:0/3080490828 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64030860 con 0x7f4c6815cc20 2023-10-21T16:18:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6cc2a640 1 -- 172.21.15.53:0/3080490828 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640309c0 con 0x7f4c6815cc20 2023-10-21T16:18:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6cc2a640 1 -- 172.21.15.53:0/3080490828 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64030d50 con 0x7f4c6815cc20 2023-10-21T16:18:44.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 -- 172.21.15.53:0/3080490828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c6815cc20 msgr2=0x7f4c6806ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/3080490828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c6815cc20 0x7f4c6806ae60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4c680ce4b0 tx=0x7f4c6402b9a0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 -- 172.21.15.53:0/3080490828 shutdown_connections 2023-10-21T16:18:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/3080490828 >> v2:172.21.15.124:3300/0 conn(0x7f4c6815d000 0x7f4c680cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/3080490828 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c6815cc20 0x7f4c6806ae60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 -- 172.21.15.53:0/3080490828 >> 172.21.15.53:0/3080490828 conn(0x7f4c68069f30 msgr2=0x7f4c6806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:44.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 -- 172.21.15.53:0/3080490828 shutdown_connections 2023-10-21T16:18:44.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.719+0000 7f4c6fe90640 1 -- 172.21.15.53:0/3080490828 wait complete. 2023-10-21T16:18:44.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.721+0000 7f4c6fe90640 1 Processor -- start 2023-10-21T16:18:44.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.721+0000 7f4c6fe90640 1 -- start start 2023-10-21T16:18:44.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 0x7f4c680c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 0x7f4c6815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680c4700 con 0x7f4c680cf480 2023-10-21T16:18:44.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680c4930 con 0x7f4c68150e00 2023-10-21T16:18:44.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.722+0000 7f4c6fe90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680c4c10 con 0x7f4c6815cc20 2023-10-21T16:18:44.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47330/0 (socket says 172.21.15.53:47330) 2023-10-21T16:18:44.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/1261353299 learned_addr learned my addr 172.21.15.53:0/1261353299 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:44.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6d42b640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 0x7f4c680c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6e42d640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 0x7f4c6815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 msgr2=0x7f4c680c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 0x7f4c680c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 msgr2=0x7f4c6815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 0x7f4c6815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6dc2c640 1 -- 172.21.15.53:0/1261353299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640094d0 con 0x7f4c680cf480 2023-10-21T16:18:44.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6d42b640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 0x7f4c680c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.723+0000 7f4c6e42d640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 0x7f4c6815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c6dc2c640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4c6402b9e0 tx=0x7f4c64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6403e040 con 0x7f4c680cf480 2023-10-21T16:18:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64039c90 con 0x7f4c680cf480 2023-10-21T16:18:44.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64004030 con 0x7f4c680cf480 2023-10-21T16:18:44.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680c4ef0 con 0x7f4c680cf480 2023-10-21T16:18:44.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.724+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680c54b0 con 0x7f4c680cf480 2023-10-21T16:18:44.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.726+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680c56a0 con 0x7f4c680cf480 2023-10-21T16:18:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.726+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4c64004190 con 0x7f4c680cf480 2023-10-21T16:18:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.727+0000 7f4c5e7fc640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 0x7f4c4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:44.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.727+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c640abd90 con 0x7f4c680cf480 2023-10-21T16:18:44.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.728+0000 7f4c6d42b640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 0x7f4c4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:44.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.729+0000 7f4c6d42b640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 0x7f4c4c06d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006fd0 tx=0x7f4c58008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:44.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.731+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c64076bf0 con 0x7f4c680cf480 2023-10-21T16:18:44.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.936+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c68066d80 con 0x7f4c680cf480 2023-10-21T16:18:44.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.938+0000 7f4c5e7fc640 1 -- 172.21.15.53:0/1261353299 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f4c6407a210 con 0x7f4c680cf480 2023-10-21T16:18:44.940 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:44.940 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":116,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:43.937666+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:44.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 msgr2=0x7f4c4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 0x7f4c4c06d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c58006fd0 tx=0x7f4c58008040 comp rx=0 tx=0).stop 2023-10-21T16:18:44.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 msgr2=0x7f4c6815f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:44.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4c6402b9e0 tx=0x7f4c64004730 comp rx=0 tx=0).stop 2023-10-21T16:18:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 shutdown_connections 2023-10-21T16:18:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f4c4c06af00 0x7f4c4c06d3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c68150e00 0x7f4c6815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c6815cc20 0x7f4c680c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 --2- 172.21.15.53:0/1261353299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c680cf480 0x7f4c6815f7e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 >> 172.21.15.53:0/1261353299 conn(0x7f4c68069f30 msgr2=0x7f4c680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 shutdown_connections 2023-10-21T16:18:44.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:44.941+0000 7f4c6fe90640 1 -- 172.21.15.53:0/1261353299 wait complete. 2023-10-21T16:18:44.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:45.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.184+0000 7f576840e640 1 Processor -- start 2023-10-21T16:18:45.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.184+0000 7f576840e640 1 -- start start 2023-10-21T16:18:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.186+0000 7f576840e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5760160940 0x7f57600709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.186+0000 7f576840e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600c48f0 con 0x7f576015cd90 2023-10-21T16:18:45.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.186+0000 7f576840e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600c3ce0 con 0x7f576006c890 2023-10-21T16:18:45.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.186+0000 7f576840e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600c4d80 con 0x7f5760160940 2023-10-21T16:18:45.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.187+0000 7f57661aa640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f576015cd90 0x7f57600702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34572/0 (socket says 172.21.15.53:34572) 2023-10-21T16:18:45.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.187+0000 7f57661aa640 1 -- 172.21.15.53:0/3179290606 learned_addr learned my addr 172.21.15.53:0/3179290606 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:45.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.187+0000 7f57659a9640 1 --2- 172.21.15.53:0/3179290606 >> v2:172.21.15.124:3300/0 conn(0x7f5760160940 0x7f57600709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.187+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275841394 0 0) 0x7f57600c48f0 con 0x7f576015cd90 2023-10-21T16:18:45.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.187+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c003520 con 0x7f576015cd90 2023-10-21T16:18:45.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132260486 0 0) 0x7f574c003520 con 0x7f576015cd90 2023-10-21T16:18:45.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 >> v2:172.21.15.124:3300/0 conn(0x7f5760160940 msgr2=0x7f57600709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:45.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 --2- 172.21.15.53:0/3179290606 >> v2:172.21.15.124:3300/0 conn(0x7f5760160940 0x7f57600709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 >> v1:172.21.15.124:6789/0 conn(0x7f576006c890 legacy=0x7f576006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600c3fc0 con 0x7f576015cd90 2023-10-21T16:18:45.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f575c001670 con 0x7f576015cd90 2023-10-21T16:18:45.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 >> v1:172.21.15.53:6789/0 conn(0x7f576015cd90 legacy=0x7f57600702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 0x7f574c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f574c0047c0 0x7f574c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c005180 0x7f574c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00bff0 con 0x7f574c003970 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00c160 con 0x7f574c0047c0 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c00c320 con 0x7f574c005180 2023-10-21T16:18:45.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.188+0000 7f57661aa640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 0x7f574c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57659a9640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f574c0047c0 0x7f574c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57661aa640 1 -- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c005180 msgr2=0x7f574c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:45.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57661aa640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c005180 0x7f574c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57661aa640 1 -- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f574c0047c0 msgr2=0x7f574c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57661aa640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f574c0047c0 0x7f574c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57661aa640 1 -- 172.21.15.53:0/3179290606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c003520 con 0x7f574c003970 2023-10-21T16:18:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.189+0000 7f57669ab640 1 -- 172.21.15.53:0/3179290606 reap_dead start 2023-10-21T16:18:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.190+0000 7f57661aa640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 0x7f574c003d90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c240 tx=0x7f574c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127e5ba8816e0cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.190+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c005ad0 con 0x7f574c003970 2023-10-21T16:18:45.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.190+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c005c30 con 0x7f574c003970 2023-10-21T16:18:45.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.190+0000 7f57651a8640 1 -- 172.21.15.53:0/3179290606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c02caf0 con 0x7f574c003970 2023-10-21T16:18:45.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 -- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 msgr2=0x7f574c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 0x7f574c003d90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f575c00c240 tx=0x7f574c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 -- 172.21.15.53:0/3179290606 shutdown_connections 2023-10-21T16:18:45.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 --2- 172.21.15.53:0/3179290606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f574c003970 0x7f574c003d90 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 -- 172.21.15.53:0/3179290606 >> 172.21.15.53:0/3179290606 conn(0x7f576006a380 msgr2=0x7f57600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:45.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 -- 172.21.15.53:0/3179290606 shutdown_connections 2023-10-21T16:18:45.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.191+0000 7f576840e640 1 -- 172.21.15.53:0/3179290606 wait complete. 2023-10-21T16:18:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.193+0000 7f576840e640 1 Processor -- start 2023-10-21T16:18:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.193+0000 7f576840e640 1 -- start start 2023-10-21T16:18:45.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c003970 0x7f57600c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57600ae590 0x7f57601f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57600c6ba0 con 0x7f57601f5c70 2023-10-21T16:18:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57600c9c80 con 0x7f57600ae590 2023-10-21T16:18:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f576840e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57600caf80 con 0x7f574c003970 2023-10-21T16:18:45.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f57659a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f57659a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47360/0 (socket says 172.21.15.53:47360) 2023-10-21T16:18:45.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.194+0000 7f57659a9640 1 -- 172.21.15.53:0/2929581323 learned_addr learned my addr 172.21.15.53:0/2929581323 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:45.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57661aa640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c003970 0x7f57600c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 -- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c003970 msgr2=0x7f57600c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57669ab640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57600ae590 0x7f57601f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c003970 0x7f57600c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 -- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57600ae590 msgr2=0x7f57601f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57600ae590 0x7f57601f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 -- 172.21.15.53:0/2929581323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57601f6230 con 0x7f57601f5c70 2023-10-21T16:18:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f57659a9640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f57500108e0 tx=0x7f5750010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.195+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000eb00 con 0x7f57601f5c70 2023-10-21T16:18:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.196+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575000ec60 con 0x7f57601f5c70 2023-10-21T16:18:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.196+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57601f6510 con 0x7f57601f5c70 2023-10-21T16:18:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.196+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575001c400 con 0x7f57601f5c70 2023-10-21T16:18:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.196+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57601f6ad0 con 0x7f57601f5c70 2023-10-21T16:18:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.197+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760067490 con 0x7f57601f5c70 2023-10-21T16:18:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.198+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f575001c560 con 0x7f57601f5c70 2023-10-21T16:18:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.199+0000 7f5756ffd640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 0x7f574406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.200+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5750089e30 con 0x7f57601f5c70 2023-10-21T16:18:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.200+0000 7f57661aa640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 0x7f574406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.201+0000 7f57661aa640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 0x7f574406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f575c034000 tx=0x7f575c009030 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.203+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5750054d30 con 0x7f57601f5c70 2023-10-21T16:18:45.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.404+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57600b0ef0 con 0x7f57601f5c70 2023-10-21T16:18:45.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.407+0000 7f5756ffd640 1 -- 172.21.15.53:0/2929581323 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f5750058350 con 0x7f57601f5c70 2023-10-21T16:18:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:45.409 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":116,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:43.937666+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:45.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 msgr2=0x7f574406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 0x7f574406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f575c034000 tx=0x7f575c009030 comp rx=0 tx=0).stop 2023-10-21T16:18:45.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 msgr2=0x7f576015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f57500108e0 tx=0x7f5750010da0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 shutdown_connections 2023-10-21T16:18:45.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f574406af00 0x7f574406d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57600ae590 0x7f57601f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57601f5c70 0x7f576015daf0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 --2- 172.21.15.53:0/2929581323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c003970 0x7f57600c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.410+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 >> 172.21.15.53:0/2929581323 conn(0x7f576006a380 msgr2=0x7f57601575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:45.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.411+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 shutdown_connections 2023-10-21T16:18:45.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.411+0000 7f576840e640 1 -- 172.21.15.53:0/2929581323 wait complete. 2023-10-21T16:18:45.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:18:45.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.655+0000 7fb676def640 1 Processor -- start 2023-10-21T16:18:45.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.655+0000 7fb676def640 1 -- start start 2023-10-21T16:18:45.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.656+0000 7fb676def640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb6700cb5e0 0x7fb6700cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.656+0000 7fb676def640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6700b3c50 0x7fb6700cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.656+0000 7fb676def640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb670159db0 con 0x7fb6701599d0 2023-10-21T16:18:45.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb676def640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb670071fe0 con 0x7fb6700b3c50 2023-10-21T16:18:45.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb676def640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700b44c0 con 0x7fb6700cb5e0 2023-10-21T16:18:45.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb675ded640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb6701599d0 0x7fb6700caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34576/0 (socket says 172.21.15.53:34576) 2023-10-21T16:18:45.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6765ee640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6700b3c50 0x7fb6700cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb675ded640 1 -- 172.21.15.53:0/3433223290 learned_addr learned my addr 172.21.15.53:0/3433223290 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:45.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6755ec640 1 --2- 172.21.15.53:0/3433223290 >> v2:172.21.15.124:3300/0 conn(0x7fb6700cb5e0 0x7fb6700cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6765ee640 1 -- 172.21.15.53:0/3433223290 >> v2:172.21.15.124:3300/0 conn(0x7fb6700cb5e0 msgr2=0x7fb6700cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:45.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6765ee640 1 --2- 172.21.15.53:0/3433223290 >> v2:172.21.15.124:3300/0 conn(0x7fb6700cb5e0 0x7fb6700cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6765ee640 1 -- 172.21.15.53:0/3433223290 >> v1:172.21.15.53:6789/0 conn(0x7fb6701599d0 legacy=0x7fb6700caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb6765ee640 1 -- 172.21.15.53:0/3433223290 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700713d0 con 0x7fb6700b3c50 2023-10-21T16:18:45.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.657+0000 7fb674deb640 1 -- 172.21.15.53:0/3433223290 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792417306 0 0) 0x7fb66c001110 con 0x7fb6701599d0 2023-10-21T16:18:45.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.659+0000 7fb6765ee640 1 --2- 172.21.15.53:0/3433223290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb6700cc0c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb6700709e0 tx=0x7fb66802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b64d6eae560057 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.659+0000 7fb674deb640 1 -- 172.21.15.53:0/3433223290 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668031830 con 0x7fb6700b3c50 2023-10-21T16:18:45.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.659+0000 7fb674deb640 1 -- 172.21.15.53:0/3433223290 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668031990 con 0x7fb6700b3c50 2023-10-21T16:18:45.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.659+0000 7fb674deb640 1 -- 172.21.15.53:0/3433223290 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668031d20 con 0x7fb6700b3c50 2023-10-21T16:18:45.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 -- 172.21.15.53:0/3433223290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 msgr2=0x7fb6700cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 --2- 172.21.15.53:0/3433223290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb6700cc0c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb6700709e0 tx=0x7fb66802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 -- 172.21.15.53:0/3433223290 shutdown_connections 2023-10-21T16:18:45.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 --2- 172.21.15.53:0/3433223290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb6700cc0c0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 --2- 172.21.15.53:0/3433223290 >> v2:172.21.15.124:3300/0 conn(0x7fb6700cb5e0 0x7fb6700cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 -- 172.21.15.53:0/3433223290 >> 172.21.15.53:0/3433223290 conn(0x7fb67006c050 msgr2=0x7fb67006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:45.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 -- 172.21.15.53:0/3433223290 shutdown_connections 2023-10-21T16:18:45.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.660+0000 7fb676def640 1 -- 172.21.15.53:0/3433223290 wait complete. 2023-10-21T16:18:45.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.662+0000 7fb676def640 1 Processor -- start 2023-10-21T16:18:45.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.662+0000 7fb676def640 1 -- start start 2023-10-21T16:18:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6700cb5e0 0x7fb6700b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6701599d0 0x7fb6700b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701581d0 con 0x7fb6700b3c50 2023-10-21T16:18:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700c5660 con 0x7fb6700cb5e0 2023-10-21T16:18:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb676def640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700c5900 con 0x7fb6701599d0 2023-10-21T16:18:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb675ded640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb675ded640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47368/0 (socket says 172.21.15.53:47368) 2023-10-21T16:18:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb675ded640 1 -- 172.21.15.53:0/302103993 learned_addr learned my addr 172.21.15.53:0/302103993 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.663+0000 7fb6755ec640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6700cb5e0 0x7fb6700b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 -- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6701599d0 msgr2=0x7fb6700b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6701599d0 0x7fb6700b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 -- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6700cb5e0 msgr2=0x7fb6700b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6700cb5e0 0x7fb6700b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 -- 172.21.15.53:0/302103993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009400 con 0x7fb6700b3c50 2023-10-21T16:18:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb675ded640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c780 tx=0x7fb66c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0072d0 con 0x7fb6700b3c50 2023-10-21T16:18:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c007450 con 0x7fb6700b3c50 2023-10-21T16:18:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6700c5be0 con 0x7fb6700b3c50 2023-10-21T16:18:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0047e0 con 0x7fb6700b3c50 2023-10-21T16:18:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.664+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6701f3260 con 0x7fb6700b3c50 2023-10-21T16:18:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.666+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb670074150 con 0x7fb6700b3c50 2023-10-21T16:18:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.667+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb66c004960 con 0x7fb6700b3c50 2023-10-21T16:18:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.668+0000 7fb6667fc640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 0x7fb65406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.668+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb66c088b80 con 0x7fb6700b3c50 2023-10-21T16:18:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.669+0000 7fb6755ec640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 0x7fb65406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.670+0000 7fb6755ec640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 0x7fb65406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6701f10b0 tx=0x7fb66000b040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.672+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb66c053ad0 con 0x7fb6700b3c50 2023-10-21T16:18:45.879 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:45.876+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:45.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:45.912+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb670159db0 con 0x7fb6700b3c50 2023-10-21T16:18:46.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.509+0000 7fb6667fc640 1 -- 172.21.15.53:0/302103993 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb670159db0 con 0x7fb6700b3c50 2023-10-21T16:18:46.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 msgr2=0x7fb65406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 0x7fb65406d3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6701f10b0 tx=0x7fb66000b040 comp rx=0 tx=0).stop 2023-10-21T16:18:46.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 msgr2=0x7fb670157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c780 tx=0x7fb66c00cc40 comp rx=0 tx=0).stop 2023-10-21T16:18:46.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 shutdown_connections 2023-10-21T16:18:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fb65406af00 0x7fb65406d3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6701599d0 0x7fb6700b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6700cb5e0 0x7fb6700b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 --2- 172.21.15.53:0/302103993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6700b3c50 0x7fb670157c90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 >> 172.21.15.53:0/302103993 conn(0x7fb67006c050 msgr2=0x7fb6700c4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 shutdown_connections 2023-10-21T16:18:46.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.513+0000 7fb676def640 1 -- 172.21.15.53:0/302103993 wait complete. 2023-10-21T16:18:46.538 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:18:46.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:18:46.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.775+0000 7f26fc2da640 1 Processor -- start 2023-10-21T16:18:46.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.775+0000 7f26fc2da640 1 -- start start 2023-10-21T16:18:46.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.776+0000 7f26fc2da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26f4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26fc2da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f406e150 con 0x7f26f40c9910 2023-10-21T16:18:46.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26fc2da640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f406de70 con 0x7f26f40cb620 2023-10-21T16:18:46.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26fc2da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40c8e30 con 0x7f26f4160660 2023-10-21T16:18:46.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26f9875640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f26f40c9910 0x7f26f40caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34586/0 (socket says 172.21.15.53:34586) 2023-10-21T16:18:46.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26f9875640 1 -- 172.21.15.53:0/2595874510 learned_addr learned my addr 172.21.15.53:0/2595874510 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:46.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26fa076640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26f4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.777+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471832174 0 0) 0x7f26f406e150 con 0x7f26f40c9910 2023-10-21T16:18:46.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e4003520 con 0x7f26f40c9910 2023-10-21T16:18:46.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747208161 0 0) 0x7f26e4003520 con 0x7f26f40c9910 2023-10-21T16:18:46.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 msgr2=0x7f26f4070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:46.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26f4070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 >> v1:172.21.15.124:6789/0 conn(0x7f26f40cb620 legacy=0x7f26f40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40c9110 con 0x7f26f40c9910 2023-10-21T16:18:46.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f26e0001670 con 0x7f26f40c9910 2023-10-21T16:18:46.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 >> v1:172.21.15.53:6789/0 conn(0x7f26f40c9910 legacy=0x7f26f40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26e40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e4003f20 0x7f26e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e4004d00 0x7f26e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40c8e30 con 0x7f26e4003f20 2023-10-21T16:18:46.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e400ba60 con 0x7f26e4004d00 2023-10-21T16:18:46.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e400bc20 con 0x7f26f4160660 2023-10-21T16:18:46.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.778+0000 7f26f9875640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e4003f20 0x7f26e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26fa076640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26e40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26fa877640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e4004d00 0x7f26e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26f9875640 1 -- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 msgr2=0x7f26e40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26f9875640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26f4160660 0x7f26e40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26f9875640 1 -- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e4004d00 msgr2=0x7f26e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26f9875640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26e4004d00 0x7f26e4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.779+0000 7f26f9875640 1 -- 172.21.15.53:0/2595874510 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4003520 con 0x7f26e4003f20 2023-10-21T16:18:46.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.780+0000 7f26f9875640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e4003f20 0x7f26e40047c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c240 tx=0x7f26e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d0b97ad1430ce server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:46.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0005ad0 con 0x7f26e4003f20 2023-10-21T16:18:46.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0005c30 con 0x7f26e4003f20 2023-10-21T16:18:46.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26f9074640 1 -- 172.21.15.53:0/2595874510 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e002caf0 con 0x7f26e4003f20 2023-10-21T16:18:46.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26fc2da640 1 -- 172.21.15.53:0/2595874510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e4003f20 msgr2=0x7f26e40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2595874510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26e4003f20 0x7f26e40047c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c240 tx=0x7f26e40057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26fa877640 1 -- 172.21.15.53:0/2595874510 reap_dead start 2023-10-21T16:18:46.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26fc2da640 1 -- 172.21.15.53:0/2595874510 shutdown_connections 2023-10-21T16:18:46.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.781+0000 7f26fc2da640 1 -- 172.21.15.53:0/2595874510 >> 172.21.15.53:0/2595874510 conn(0x7f26f406a5d0 msgr2=0x7f26f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:46.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.782+0000 7f26fc2da640 1 -- 172.21.15.53:0/2595874510 shutdown_connections 2023-10-21T16:18:46.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.782+0000 7f26fc2da640 1 -- 172.21.15.53:0/2595874510 wait complete. 2023-10-21T16:18:46.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.783+0000 7f26fc2da640 1 Processor -- start 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 -- start start 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 0x7f26f4146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26f41f1e40 0x7f26f40cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cd270 con 0x7f26f40ccdc0 2023-10-21T16:18:46.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4158970 con 0x7f26f41f1e40 2023-10-21T16:18:46.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.784+0000 7f26fc2da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4147370 con 0x7f26e4003f20 2023-10-21T16:18:46.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47390/0 (socket says 172.21.15.53:47390) 2023-10-21T16:18:46.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 -- 172.21.15.53:0/2613888639 learned_addr learned my addr 172.21.15.53:0/2613888639 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:46.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26f9875640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26f41f1e40 0x7f26f40cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa076640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 0x7f26f4146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 -- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 msgr2=0x7f26f4146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 0x7f26f4146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 -- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26f41f1e40 msgr2=0x7f26f40cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:46.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26f41f1e40 0x7f26f40cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:46.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa877640 1 -- 172.21.15.53:0/2613888639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f4157d60 con 0x7f26f40ccdc0 2023-10-21T16:18:46.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.785+0000 7f26fa076640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 0x7f26f4146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:46.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26fa877640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f26f4071310 tx=0x7f26ec00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:46.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00b070 con 0x7f26f40ccdc0 2023-10-21T16:18:46.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec0054e0 con 0x7f26f40ccdc0 2023-10-21T16:18:46.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec003810 con 0x7f26f40ccdc0 2023-10-21T16:18:46.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f4158040 con 0x7f26f40ccdc0 2023-10-21T16:18:46.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.786+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f4158600 con 0x7f26f40ccdc0 2023-10-21T16:18:46.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.787+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f4159540 con 0x7f26f40ccdc0 2023-10-21T16:18:46.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.788+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f26ec003970 con 0x7f26f40ccdc0 2023-10-21T16:18:46.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.789+0000 7f26f2ffd640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 0x7f26d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:46.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.789+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26ec0889e0 con 0x7f26f40ccdc0 2023-10-21T16:18:46.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.790+0000 7f26fa076640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 0x7f26d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:46.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.791+0000 7f26fa076640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 0x7f26d806d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26f414aaf0 tx=0x7f26e800b040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:46.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:46.794+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26ec053720 con 0x7f26f40ccdc0 2023-10-21T16:18:47.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.043+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f26f404e160 con 0x7f26f40ccdc0 2023-10-21T16:18:47.133 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:47.130+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:47.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.521+0000 7f26f2ffd640 1 -- 172.21.15.53:0/2613888639 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f26ec056d40 con 0x7f26f40ccdc0 2023-10-21T16:18:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.524+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 msgr2=0x7f26d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.524+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 0x7f26d806d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26f414aaf0 tx=0x7f26e800b040 comp rx=0 tx=0).stop 2023-10-21T16:18:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.524+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 msgr2=0x7f26f41f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.524+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f26f4071310 tx=0x7f26ec00d890 comp rx=0 tx=0).stop 2023-10-21T16:18:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 shutdown_connections 2023-10-21T16:18:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f26d806af30 0x7f26d806d3f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26f40ccdc0 0x7f26f41f5b10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26f41f1e40 0x7f26f40cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 --2- 172.21.15.53:0/2613888639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26e4003f20 0x7f26f4146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 >> 172.21.15.53:0/2613888639 conn(0x7f26f406a5d0 msgr2=0x7f26f415e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 shutdown_connections 2023-10-21T16:18:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.525+0000 7f26fc2da640 1 -- 172.21.15.53:0/2613888639 wait complete. 2023-10-21T16:18:47.551 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:18:47.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.780+0000 7f8072415640 1 Processor -- start 2023-10-21T16:18:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.781+0000 7f8072415640 1 -- start start 2023-10-21T16:18:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.782+0000 7f8072415640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f806c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.782+0000 7f8072415640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f806c0b1d90 0x7f806c0d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.782+0000 7f8072415640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806c0b8bf0 con 0x7f806c0b1840 2023-10-21T16:18:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.782+0000 7f8072415640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f806c0d2910 con 0x7f806c0b1d90 2023-10-21T16:18:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.782+0000 7f8072415640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0ce1f0 con 0x7f806c0cd8d0 2023-10-21T16:18:47.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f8063fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f806c0b1840 0x7f806c0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34598/0 (socket says 172.21.15.53:34598) 2023-10-21T16:18:47.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f8063fff640 1 -- 172.21.15.53:0/3125483854 learned_addr learned my addr 172.21.15.53:0/3125483854 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:47.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590996306 0 0) 0x7f806c0b8bf0 con 0x7f806c0b1840 2023-10-21T16:18:47.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8058003520 con 0x7f806c0b1840 2023-10-21T16:18:47.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f80709b2640 1 --2- 172.21.15.53:0/3125483854 >> v2:172.21.15.124:3300/0 conn(0x7f806c0b1d90 0x7f806c0d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.783+0000 7f806bfff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f806c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 982001907 0 0) 0x7f8058003520 con 0x7f806c0b1840 2023-10-21T16:18:47.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 msgr2=0x7f806c0cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:47.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f806c0cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 >> v2:172.21.15.124:3300/0 conn(0x7f806c0b1d90 msgr2=0x7f806c0d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:47.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 --2- 172.21.15.53:0/3125483854 >> v2:172.21.15.124:3300/0 conn(0x7f806c0b1d90 0x7f806c0d32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c0b8630 con 0x7f806c0b1840 2023-10-21T16:18:47.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8054001670 con 0x7f806c0b1840 2023-10-21T16:18:47.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 >> v1:172.21.15.53:6789/0 conn(0x7f806c0b1840 legacy=0x7f806c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8058003970 0x7f8058003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f80580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8058004d00 0x7f8058005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0d2910 con 0x7f8058003970 2023-10-21T16:18:47.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0ce1f0 con 0x7f8058004d00 2023-10-21T16:18:47.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805800bab0 con 0x7f806c0cd8d0 2023-10-21T16:18:47.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f80709b2640 1 --2- 172.21.15.53:0/3125483854 >> v2:172.21.15.124:3300/0 conn(0x7f806c0b1d90 0x7f806c0d32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:47.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f806bfff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8058003970 0x7f8058003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f8063fff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f80580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.784+0000 7f80709b2640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8058004d00 0x7f8058005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f806bfff640 1 -- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 msgr2=0x7f80580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:47.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f806bfff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0cd8d0 0x7f80580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f806bfff640 1 -- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8058004d00 msgr2=0x7f8058005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:47.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f806bfff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8058004d00 0x7f8058005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f806bfff640 1 -- 172.21.15.53:0/3125483854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8058003520 con 0x7f8058003970 2023-10-21T16:18:47.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.785+0000 7f80709b2640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8058004d00 0x7f8058005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:47.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.786+0000 7f806bfff640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8058003970 0x7f8058003d90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ba30 tx=0x7f80580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c85c665135a1cd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:47.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c032640 con 0x7f8058003970 2023-10-21T16:18:47.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c0327a0 con 0x7f8058003970 2023-10-21T16:18:47.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f806b7fe640 1 -- 172.21.15.53:0/3125483854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c032b30 con 0x7f8058003970 2023-10-21T16:18:47.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 -- 172.21.15.53:0/3125483854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8058003970 msgr2=0x7f8058003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 --2- 172.21.15.53:0/3125483854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8058003970 0x7f8058003d90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ba30 tx=0x7f80580057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f80709b2640 1 -- 172.21.15.53:0/3125483854 reap_dead start 2023-10-21T16:18:47.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 -- 172.21.15.53:0/3125483854 shutdown_connections 2023-10-21T16:18:47.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 -- 172.21.15.53:0/3125483854 >> 172.21.15.53:0/3125483854 conn(0x7f806c06b6a0 msgr2=0x7f806c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:47.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 -- 172.21.15.53:0/3125483854 shutdown_connections 2023-10-21T16:18:47.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.787+0000 7f8072415640 1 -- 172.21.15.53:0/3125483854 wait complete. 2023-10-21T16:18:47.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.789+0000 7f8072415640 1 Processor -- start 2023-10-21T16:18:47.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.789+0000 7f8072415640 1 -- start start 2023-10-21T16:18:47.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 0x7f806c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0c6a60 0x7f806c0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c157ec0 con 0x7f806c154c70 2023-10-21T16:18:47.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1581a0 con 0x7f806c0c8860 2023-10-21T16:18:47.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.790+0000 7f8072415640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c158480 con 0x7f806c0c6a60 2023-10-21T16:18:47.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47410/0 (socket says 172.21.15.53:47410) 2023-10-21T16:18:47.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 -- 172.21.15.53:0/2684844668 learned_addr learned my addr 172.21.15.53:0/2684844668 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:47.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f806bfff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 0x7f806c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 -- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0c6a60 msgr2=0x7f806c0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0c6a60 0x7f806c0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 -- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 msgr2=0x7f806c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 0x7f806c0c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 -- 172.21.15.53:0/2684844668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c158760 con 0x7f806c154c70 2023-10-21T16:18:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f806bfff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 0x7f806c0c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.791+0000 7f8063fff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8054004840 tx=0x7f805400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:47.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.792+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054005e10 con 0x7f806c154c70 2023-10-21T16:18:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.792+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805400c850 con 0x7f806c154c70 2023-10-21T16:18:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.792+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805400cc00 con 0x7f806c154c70 2023-10-21T16:18:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.792+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c158a40 con 0x7f806c154c70 2023-10-21T16:18:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.792+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c155610 con 0x7f806c154c70 2023-10-21T16:18:47.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.793+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c066d80 con 0x7f806c154c70 2023-10-21T16:18:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.794+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f805401d460 con 0x7f806c154c70 2023-10-21T16:18:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.795+0000 7f80697fa640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 0x7f804c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:47.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.795+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8054089800 con 0x7f806c154c70 2023-10-21T16:18:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.796+0000 7f806bfff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 0x7f804c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.797+0000 7f806bfff640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 0x7f804c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ba00 tx=0x7f805c030000 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:47.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:47.799+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80540545c0 con 0x7f806c154c70 2023-10-21T16:18:48.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.032+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f806c0ce960 con 0x7f806c154c70 2023-10-21T16:18:48.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.033+0000 7f80697fa640 1 -- 172.21.15.53:0/2684844668 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f8054057be0 con 0x7f806c154c70 2023-10-21T16:18:48.035 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:48.035 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:18:48.035 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:18:48.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 msgr2=0x7f804c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 0x7f804c06d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f805c00ba00 tx=0x7f805c030000 comp rx=0 tx=0).stop 2023-10-21T16:18:48.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 msgr2=0x7f806c1550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8054004840 tx=0x7f805400ef80 comp rx=0 tx=0).stop 2023-10-21T16:18:48.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 shutdown_connections 2023-10-21T16:18:48.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f804c06af00 0x7f804c06d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f806c0c6a60 0x7f806c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806c154c70 0x7f806c1550d0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 --2- 172.21.15.53:0/2684844668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f806c0c8860 0x7f806c0c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 >> 172.21.15.53:0/2684844668 conn(0x7f806c06b6a0 msgr2=0x7f806c0d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:48.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 shutdown_connections 2023-10-21T16:18:48.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.036+0000 7f8072415640 1 -- 172.21.15.53:0/2684844668 wait complete. 2023-10-21T16:18:48.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:18:48.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.288+0000 7fa365d35640 1 Processor -- start 2023-10-21T16:18:48.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.288+0000 7fa365d35640 1 -- start start 2023-10-21T16:18:48.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.289+0000 7fa365d35640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa3600cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.289+0000 7fa365d35640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36015d040 con 0x7fa3600cfa50 2023-10-21T16:18:48.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.289+0000 7fa365d35640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3600d0650 con 0x7fa36015cc60 2023-10-21T16:18:48.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.289+0000 7fa365d35640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600d0930 con 0x7fa3600d16f0 2023-10-21T16:18:48.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.290+0000 7fa365534640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa3600cfa50 0x7fa360166aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34600/0 (socket says 172.21.15.53:34600) 2023-10-21T16:18:48.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.290+0000 7fa365534640 1 -- 172.21.15.53:0/2677083450 learned_addr learned my addr 172.21.15.53:0/2677083450 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:48.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.290+0000 7fa35ffff640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa3600cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.290+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668275283 0 0) 0x7fa36015d040 con 0x7fa3600cfa50 2023-10-21T16:18:48.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.290+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa348003520 con 0x7fa3600cfa50 2023-10-21T16:18:48.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234881494 0 0) 0x7fa348003520 con 0x7fa3600cfa50 2023-10-21T16:18:48.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 msgr2=0x7fa3600cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa3600cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 >> v1:172.21.15.124:6789/0 conn(0x7fa36015cc60 legacy=0x7fa3600cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600d0c10 con 0x7fa3600cfa50 2023-10-21T16:18:48.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa358001670 con 0x7fa3600cfa50 2023-10-21T16:18:48.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 >> v1:172.21.15.53:6789/0 conn(0x7fa3600cfa50 legacy=0x7fa360166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480039a0 0x7fa348003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3480047c0 0x7fa348004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3600d0930 con 0x7fa3480047c0 2023-10-21T16:18:48.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800ba60 con 0x7fa3480039a0 2023-10-21T16:18:48.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800bc20 con 0x7fa3600d16f0 2023-10-21T16:18:48.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa35ffff640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3480047c0 0x7fa348004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa365534640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.291+0000 7fa364d33640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480039a0 0x7fa348003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa35ffff640 1 -- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 msgr2=0x7fa348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa35ffff640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa35ffff640 1 -- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480039a0 msgr2=0x7fa348003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa35ffff640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480039a0 0x7fa348003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa35ffff640 1 -- 172.21.15.53:0/2677083450 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348003520 con 0x7fa3480047c0 2023-10-21T16:18:48.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa364d33640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480039a0 0x7fa348003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:48.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.292+0000 7fa365534640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3600d16f0 0x7fa348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:48.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.293+0000 7fa35ffff640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3480047c0 0x7fa348004c40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa354009a90 tx=0x7fa3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98475cd88928151 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.293+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35402c660 con 0x7fa3480047c0 2023-10-21T16:18:48.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.293+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35402c7c0 con 0x7fa3480047c0 2023-10-21T16:18:48.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.293+0000 7fa35f7fe640 1 -- 172.21.15.53:0/2677083450 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35402cb50 con 0x7fa3480047c0 2023-10-21T16:18:48.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 -- 172.21.15.53:0/2677083450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3480047c0 msgr2=0x7fa348004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 --2- 172.21.15.53:0/2677083450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3480047c0 0x7fa348004c40 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa354009a90 tx=0x7fa3480057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365534640 1 -- 172.21.15.53:0/2677083450 reap_dead start 2023-10-21T16:18:48.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 -- 172.21.15.53:0/2677083450 shutdown_connections 2023-10-21T16:18:48.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 -- 172.21.15.53:0/2677083450 >> 172.21.15.53:0/2677083450 conn(0x7fa36009bcf0 msgr2=0x7fa3600d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:48.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 -- 172.21.15.53:0/2677083450 shutdown_connections 2023-10-21T16:18:48.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.294+0000 7fa365d35640 1 -- 172.21.15.53:0/2677083450 wait complete. 2023-10-21T16:18:48.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.296+0000 7fa365d35640 1 Processor -- start 2023-10-21T16:18:48.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.296+0000 7fa365d35640 1 -- start start 2023-10-21T16:18:48.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480047c0 0x7fa360159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa36006c680 0x7fa3600c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36006cb00 con 0x7fa3600cc250 2023-10-21T16:18:48.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36009f3f0 con 0x7fa3480047c0 2023-10-21T16:18:48.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.297+0000 7fa365d35640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36009f880 con 0x7fa36006c680 2023-10-21T16:18:48.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47434/0 (socket says 172.21.15.53:47434) 2023-10-21T16:18:48.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 -- 172.21.15.53:0/3250055793 learned_addr learned my addr 172.21.15.53:0/3250055793 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:48.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa364d33640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480047c0 0x7fa360159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa365534640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa36006c680 0x7fa3600c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 -- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa36006c680 msgr2=0x7fa3600c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa36006c680 0x7fa3600c64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 -- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480047c0 msgr2=0x7fa360159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480047c0 0x7fa360159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 -- 172.21.15.53:0/3250055793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36009e7e0 con 0x7fa3600cc250 2023-10-21T16:18:48.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.298+0000 7fa35ffff640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa35402c200 tx=0x7fa354004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.299+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354003810 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.299+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa354003970 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.299+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354003ce0 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.299+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36009eac0 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.299+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36009f080 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.300+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36009f5a0 con 0x7fa3600cc250 2023-10-21T16:18:48.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.301+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa354034440 con 0x7fa3600cc250 2023-10-21T16:18:48.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.302+0000 7fa35d7fa640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 0x7fa34c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.303+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa3540ab720 con 0x7fa3600cc250 2023-10-21T16:18:48.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.303+0000 7fa364d33640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 0x7fa34c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.304+0000 7fa364d33640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 0x7fa34c06d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa350001e80 tx=0x7fa350009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.307+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa36009f5a0 con 0x7fa3600cc250 2023-10-21T16:18:48.394 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:48.391+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:48.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.520+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa36009f560 con 0x7fa3600cc250 2023-10-21T16:18:48.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.523+0000 7fa35d7fa640 1 -- 172.21.15.53:0/3250055793 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16786 (secure 0 0 0) 0x7fa354076740 con 0x7fa3600cc250 2023-10-21T16:18:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:48.526 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":117,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:18:45.474116+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:18:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 msgr2=0x7fa34c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 0x7fa34c06d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa350001e80 tx=0x7fa350009040 comp rx=0 tx=0).stop 2023-10-21T16:18:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 msgr2=0x7fa36006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa35402c200 tx=0x7fa354004730 comp rx=0 tx=0).stop 2023-10-21T16:18:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 shutdown_connections 2023-10-21T16:18:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fa34c06af30 0x7fa34c06d3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa36006c680 0x7fa3600c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3600cc250 0x7fa36006c140 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 --2- 172.21.15.53:0/3250055793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3480047c0 0x7fa360159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.526+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 >> 172.21.15.53:0/3250055793 conn(0x7fa36009bcf0 msgr2=0x7fa3600ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.527+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 shutdown_connections 2023-10-21T16:18:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.527+0000 7fa365d35640 1 -- 172.21.15.53:0/3250055793 wait complete. 2023-10-21T16:18:48.548 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:18:48.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:18:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.782+0000 7f7b329bf640 1 Processor -- start 2023-10-21T16:18:48.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.782+0000 7f7b329bf640 1 -- start start 2023-10-21T16:18:48.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b329bf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 0x7f7b2c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b329bf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c0cc5b0 con 0x7f7b2c069d50 2023-10-21T16:18:48.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b329bf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c0cc890 con 0x7f7b2c0cbfb0 2023-10-21T16:18:48.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b329bf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b18a0 con 0x7f7b2c0b4fb0 2023-10-21T16:18:48.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b319bd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7b2c069d50 0x7f7b2c06a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34610/0 (socket says 172.21.15.53:34610) 2023-10-21T16:18:48.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.784+0000 7f7b319bd640 1 -- 172.21.15.53:0/3522906271 learned_addr learned my addr 172.21.15.53:0/3522906271 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:48.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b321be640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 0x7f7b2c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482860964 0 0) 0x7f7b2c0cc5b0 con 0x7f7b2c069d50 2023-10-21T16:18:48.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b18003520 con 0x7f7b2c069d50 2023-10-21T16:18:48.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475878779 0 0) 0x7f7b18003520 con 0x7f7b2c069d50 2023-10-21T16:18:48.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 msgr2=0x7f7b2c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 0x7f7b2c15b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 >> v1:172.21.15.124:6789/0 conn(0x7f7b2c0cbfb0 legacy=0x7f7b2c0b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0ccb70 con 0x7f7b2c069d50 2023-10-21T16:18:48.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7b28001670 con 0x7f7b2c069d50 2023-10-21T16:18:48.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 >> v1:172.21.15.53:6789/0 conn(0x7f7b2c069d50 legacy=0x7f7b2c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b180039a0 0x7f7b18003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 0x7f7b18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b18a0 con 0x7f7b180047c0 2023-10-21T16:18:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1800ba30 con 0x7f7b180039a0 2023-10-21T16:18:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.785+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1800bbf0 con 0x7f7b2c0b4fb0 2023-10-21T16:18:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b319bd640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b180039a0 0x7f7b18003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 -- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 msgr2=0x7f7b18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b4fb0 0x7f7b18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 -- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b180039a0 msgr2=0x7f7b18003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:48.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b180039a0 0x7f7b18003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.786+0000 7f7b311bc640 1 -- 172.21.15.53:0/3522906271 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18003520 con 0x7f7b180047c0 2023-10-21T16:18:48.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.787+0000 7f7b311bc640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b18004c40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c001720 tx=0x7f7b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6393c3e969d7ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0052d0 con 0x7f7b180047c0 2023-10-21T16:18:48.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c02e050 con 0x7f7b180047c0 2023-10-21T16:18:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b309bb640 1 -- 172.21.15.53:0/3522906271 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0035f0 con 0x7f7b180047c0 2023-10-21T16:18:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b329bf640 1 -- 172.21.15.53:0/3522906271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 msgr2=0x7f7b18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b329bf640 1 --2- 172.21.15.53:0/3522906271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b18004c40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c001720 tx=0x7f7b180057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b321be640 1 -- 172.21.15.53:0/3522906271 reap_dead start 2023-10-21T16:18:48.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b329bf640 1 -- 172.21.15.53:0/3522906271 shutdown_connections 2023-10-21T16:18:48.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b329bf640 1 -- 172.21.15.53:0/3522906271 >> 172.21.15.53:0/3522906271 conn(0x7f7b2c06ae60 msgr2=0x7f7b2c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.788+0000 7f7b329bf640 1 -- 172.21.15.53:0/3522906271 shutdown_connections 2023-10-21T16:18:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.789+0000 7f7b329bf640 1 -- 172.21.15.53:0/3522906271 wait complete. 2023-10-21T16:18:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.790+0000 7f7b329bf640 1 Processor -- start 2023-10-21T16:18:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.790+0000 7f7b329bf640 1 -- start start 2023-10-21T16:18:48.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b3220 0x7f7b2c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 0x7f7b2c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c14b6d0 con 0x7f7b180047c0 2023-10-21T16:18:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0cde90 con 0x7f7b2c0cf070 2023-10-21T16:18:48.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.791+0000 7f7b329bf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0ce2b0 con 0x7f7b2c0b3220 2023-10-21T16:18:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47450/0 (socket says 172.21.15.53:47450) 2023-10-21T16:18:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 -- 172.21.15.53:0/967757922 learned_addr learned my addr 172.21.15.53:0/967757922 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b321be640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 0x7f7b2c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b311bc640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b3220 0x7f7b2c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 -- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b3220 msgr2=0x7f7b2c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b3220 0x7f7b2c0b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 -- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 msgr2=0x7f7b2c0b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:48.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 0x7f7b2c0b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:48.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 -- 172.21.15.53:0/967757922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0ce590 con 0x7f7b180047c0 2023-10-21T16:18:48.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b321be640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 0x7f7b2c0b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:48.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.792+0000 7f7b319bd640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7b28000ff0 tx=0x7f7b2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.793+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280074e0 con 0x7f7b180047c0 2023-10-21T16:18:48.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.793+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28007640 con 0x7f7b180047c0 2023-10-21T16:18:48.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.793+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004a00 con 0x7f7b180047c0 2023-10-21T16:18:48.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.793+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c0ce870 con 0x7f7b180047c0 2023-10-21T16:18:48.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.793+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c0cd920 con 0x7f7b180047c0 2023-10-21T16:18:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.794+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c0afde0 con 0x7f7b180047c0 2023-10-21T16:18:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.795+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7b28004b60 con 0x7f7b180047c0 2023-10-21T16:18:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.796+0000 7f7b227fc640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 0x7f7b1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:48.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.796+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b280894f0 con 0x7f7b180047c0 2023-10-21T16:18:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.797+0000 7f7b311bc640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 0x7f7b1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.798+0000 7f7b311bc640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 0x7f7b1006d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c033000 tx=0x7f7b1c009030 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:48.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.800+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b280542e0 con 0x7f7b180047c0 2023-10-21T16:18:48.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:48.992+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 --> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b2c0ca9a0 con 0x7f7b1006af00 2023-10-21T16:18:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.023+0000 7f7b227fc640 1 -- 172.21.15.53:0/967757922 <== mgr.35759 v2:172.21.15.124:6832/1967358544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287376 (secure 0 0 0) 0x7f7b1c0053d0 con 0x7f7b1006af00 2023-10-21T16:18:49.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:18:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 msgr2=0x7f7b1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 0x7f7b1006d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c033000 tx=0x7f7b1c009030 comp rx=0 tx=0).stop 2023-10-21T16:18:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 msgr2=0x7f7b2c1f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7b28000ff0 tx=0x7f7b2800dc00 comp rx=0 tx=0).stop 2023-10-21T16:18:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 shutdown_connections 2023-10-21T16:18:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f7b1006af00 0x7f7b1006d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b2c0cf070 0x7f7b2c0b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b2c0b3220 0x7f7b2c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 --2- 172.21.15.53:0/967757922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b180047c0 0x7f7b2c1f1ad0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 >> 172.21.15.53:0/967757922 conn(0x7f7b2c06ae60 msgr2=0x7f7b2c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 shutdown_connections 2023-10-21T16:18:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.028+0000 7f7b329bf640 1 -- 172.21.15.53:0/967757922 wait complete. 2023-10-21T16:18:49.052 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:18:47.713533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":311,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4339,"ondisk_log_size":4339,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322952,"kb_used_data":12440,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754651768,"statfs":{"total":773094113280,"available":772763410432,"internally_reserved":0,"allocated":12738560,"data_stored":6985188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.275960"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480566+0000","last_change":"2023-10-21T16:18:44.506365+0000","last_active":"2023-10-21T16:18:45.480566+0000","last_peered":"2023-10-21T16:18:45.480566+0000","last_clean":"2023-10-21T16:18:45.480566+0000","last_became_active":"2023-10-21T16:18:44.506033+0000","last_became_peered":"2023-10-21T16:18:44.506033+0000","last_unstale":"2023-10-21T16:18:45.480566+0000","last_undegraded":"2023-10-21T16:18:45.480566+0000","last_fullsized":"2023-10-21T16:18:45.480566+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:01:59.267254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594075+0000","last_change":"2023-10-21T16:18:44.500375+0000","last_active":"2023-10-21T16:18:46.594075+0000","last_peered":"2023-10-21T16:18:46.594075+0000","last_clean":"2023-10-21T16:18:46.594075+0000","last_became_active":"2023-10-21T16:18:44.500040+0000","last_became_peered":"2023-10-21T16:18:44.500040+0000","last_unstale":"2023-10-21T16:18:46.594075+0000","last_undegraded":"2023-10-21T16:18:46.594075+0000","last_fullsized":"2023-10-21T16:18:46.594075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:40:20.933122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480112+0000","last_change":"2023-10-21T16:18:44.497948+0000","last_active":"2023-10-21T16:18:45.480112+0000","last_peered":"2023-10-21T16:18:45.480112+0000","last_clean":"2023-10-21T16:18:45.480112+0000","last_became_active":"2023-10-21T16:18:44.497667+0000","last_became_peered":"2023-10-21T16:18:44.497667+0000","last_unstale":"2023-10-21T16:18:45.480112+0000","last_undegraded":"2023-10-21T16:18:45.480112+0000","last_fullsized":"2023-10-21T16:18:45.480112+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:25:46.610475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152340+0000","last_change":"2023-10-21T16:18:44.517493+0000","last_active":"2023-10-21T16:18:46.152340+0000","last_peered":"2023-10-21T16:18:46.152340+0000","last_clean":"2023-10-21T16:18:46.152340+0000","last_became_active":"2023-10-21T16:18:44.516127+0000","last_became_peered":"2023-10-21T16:18:44.516127+0000","last_unstale":"2023-10-21T16:18:46.152340+0000","last_undegraded":"2023-10-21T16:18:46.152340+0000","last_fullsized":"2023-10-21T16:18:46.152340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:59.110459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484180+0000","last_change":"2023-10-21T16:18:44.501570+0000","last_active":"2023-10-21T16:18:45.484180+0000","last_peered":"2023-10-21T16:18:45.484180+0000","last_clean":"2023-10-21T16:18:45.484180+0000","last_became_active":"2023-10-21T16:18:44.501166+0000","last_became_peered":"2023-10-21T16:18:44.501166+0000","last_unstale":"2023-10-21T16:18:45.484180+0000","last_undegraded":"2023-10-21T16:18:45.484180+0000","last_fullsized":"2023-10-21T16:18:45.484180+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:30.636045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587778+0000","last_change":"2023-10-21T16:18:44.520659+0000","last_active":"2023-10-21T16:18:46.587778+0000","last_peered":"2023-10-21T16:18:46.587778+0000","last_clean":"2023-10-21T16:18:46.587778+0000","last_became_active":"2023-10-21T16:18:44.520306+0000","last_became_peered":"2023-10-21T16:18:44.520306+0000","last_unstale":"2023-10-21T16:18:46.587778+0000","last_undegraded":"2023-10-21T16:18:46.587778+0000","last_fullsized":"2023-10-21T16:18:46.587778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:10.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589549+0000","last_change":"2023-10-21T16:18:44.509299+0000","last_active":"2023-10-21T16:18:46.589549+0000","last_peered":"2023-10-21T16:18:46.589549+0000","last_clean":"2023-10-21T16:18:46.589549+0000","last_became_active":"2023-10-21T16:18:44.507862+0000","last_became_peered":"2023-10-21T16:18:44.507862+0000","last_unstale":"2023-10-21T16:18:46.589549+0000","last_undegraded":"2023-10-21T16:18:46.589549+0000","last_fullsized":"2023-10-21T16:18:46.589549+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:10.430211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151645+0000","last_change":"2023-10-21T16:18:44.524732+0000","last_active":"2023-10-21T16:18:46.151645+0000","last_peered":"2023-10-21T16:18:46.151645+0000","last_clean":"2023-10-21T16:18:46.151645+0000","last_became_active":"2023-10-21T16:18:44.522900+0000","last_became_peered":"2023-10-21T16:18:44.522900+0000","last_unstale":"2023-10-21T16:18:46.151645+0000","last_undegraded":"2023-10-21T16:18:46.151645+0000","last_fullsized":"2023-10-21T16:18:46.151645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:19.145776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485199+0000","last_change":"2023-10-21T16:18:44.498543+0000","last_active":"2023-10-21T16:18:45.485199+0000","last_peered":"2023-10-21T16:18:45.485199+0000","last_clean":"2023-10-21T16:18:45.485199+0000","last_became_active":"2023-10-21T16:18:44.498265+0000","last_became_peered":"2023-10-21T16:18:44.498265+0000","last_unstale":"2023-10-21T16:18:45.485199+0000","last_undegraded":"2023-10-21T16:18:45.485199+0000","last_fullsized":"2023-10-21T16:18:45.485199+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:55.299321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481480+0000","last_change":"2023-10-21T16:18:44.512214+0000","last_active":"2023-10-21T16:18:45.481480+0000","last_peered":"2023-10-21T16:18:45.481480+0000","last_clean":"2023-10-21T16:18:45.481480+0000","last_became_active":"2023-10-21T16:18:44.511936+0000","last_became_peered":"2023-10-21T16:18:44.511936+0000","last_unstale":"2023-10-21T16:18:45.481480+0000","last_undegraded":"2023-10-21T16:18:45.481480+0000","last_fullsized":"2023-10-21T16:18:45.481480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:47.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484917+0000","last_change":"2023-10-21T16:18:44.509839+0000","last_active":"2023-10-21T16:18:45.484917+0000","last_peered":"2023-10-21T16:18:45.484917+0000","last_clean":"2023-10-21T16:18:45.484917+0000","last_became_active":"2023-10-21T16:18:44.509582+0000","last_became_peered":"2023-10-21T16:18:44.509582+0000","last_unstale":"2023-10-21T16:18:45.484917+0000","last_undegraded":"2023-10-21T16:18:45.484917+0000","last_fullsized":"2023-10-21T16:18:45.484917+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:01.393863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481328+0000","last_change":"2023-10-21T16:18:44.492993+0000","last_active":"2023-10-21T16:18:45.481328+0000","last_peered":"2023-10-21T16:18:45.481328+0000","last_clean":"2023-10-21T16:18:45.481328+0000","last_became_active":"2023-10-21T16:18:44.492627+0000","last_became_peered":"2023-10-21T16:18:44.492627+0000","last_unstale":"2023-10-21T16:18:45.481328+0000","last_undegraded":"2023-10-21T16:18:45.481328+0000","last_fullsized":"2023-10-21T16:18:45.481328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:41:39.204289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151844+0000","last_change":"2023-10-21T16:18:44.519408+0000","last_active":"2023-10-21T16:18:46.151844+0000","last_peered":"2023-10-21T16:18:46.151844+0000","last_clean":"2023-10-21T16:18:46.151844+0000","last_became_active":"2023-10-21T16:18:44.519123+0000","last_became_peered":"2023-10-21T16:18:44.519123+0000","last_unstale":"2023-10-21T16:18:46.151844+0000","last_undegraded":"2023-10-21T16:18:46.151844+0000","last_fullsized":"2023-10-21T16:18:46.151844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:41.473531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483748+0000","last_change":"2023-10-21T16:18:44.511539+0000","last_active":"2023-10-21T16:18:45.483748+0000","last_peered":"2023-10-21T16:18:45.483748+0000","last_clean":"2023-10-21T16:18:45.483748+0000","last_became_active":"2023-10-21T16:18:44.511108+0000","last_became_peered":"2023-10-21T16:18:44.511108+0000","last_unstale":"2023-10-21T16:18:45.483748+0000","last_undegraded":"2023-10-21T16:18:45.483748+0000","last_fullsized":"2023-10-21T16:18:45.483748+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:41.473531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151861+0000","last_change":"2023-10-21T16:18:44.521659+0000","last_active":"2023-10-21T16:18:46.151861+0000","last_peered":"2023-10-21T16:18:46.151861+0000","last_clean":"2023-10-21T16:18:46.151861+0000","last_became_active":"2023-10-21T16:18:44.521166+0000","last_became_peered":"2023-10-21T16:18:44.521166+0000","last_unstale":"2023-10-21T16:18:46.151861+0000","last_undegraded":"2023-10-21T16:18:46.151861+0000","last_fullsized":"2023-10-21T16:18:46.151861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:06.433654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587680+0000","last_change":"2023-10-21T16:18:44.507106+0000","last_active":"2023-10-21T16:18:46.587680+0000","last_peered":"2023-10-21T16:18:46.587680+0000","last_clean":"2023-10-21T16:18:46.587680+0000","last_became_active":"2023-10-21T16:18:44.506800+0000","last_became_peered":"2023-10-21T16:18:44.506800+0000","last_unstale":"2023-10-21T16:18:46.587680+0000","last_undegraded":"2023-10-21T16:18:46.587680+0000","last_fullsized":"2023-10-21T16:18:46.587680+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:55.195177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588372+0000","last_change":"2023-10-21T16:18:44.505476+0000","last_active":"2023-10-21T16:18:46.588372+0000","last_peered":"2023-10-21T16:18:46.588372+0000","last_clean":"2023-10-21T16:18:46.588372+0000","last_became_active":"2023-10-21T16:18:44.505216+0000","last_became_peered":"2023-10-21T16:18:44.505216+0000","last_unstale":"2023-10-21T16:18:46.588372+0000","last_undegraded":"2023-10-21T16:18:46.588372+0000","last_fullsized":"2023-10-21T16:18:46.588372+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:43.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480335+0000","last_change":"2023-10-21T16:18:44.495870+0000","last_active":"2023-10-21T16:18:45.480335+0000","last_peered":"2023-10-21T16:18:45.480335+0000","last_clean":"2023-10-21T16:18:45.480335+0000","last_became_active":"2023-10-21T16:18:44.495576+0000","last_became_peered":"2023-10-21T16:18:44.495576+0000","last_unstale":"2023-10-21T16:18:45.480335+0000","last_undegraded":"2023-10-21T16:18:45.480335+0000","last_fullsized":"2023-10-21T16:18:45.480335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:34.797978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485149+0000","last_change":"2023-10-21T16:18:44.517339+0000","last_active":"2023-10-21T16:18:45.485149+0000","last_peered":"2023-10-21T16:18:45.485149+0000","last_clean":"2023-10-21T16:18:45.485149+0000","last_became_active":"2023-10-21T16:18:44.516966+0000","last_became_peered":"2023-10-21T16:18:44.516966+0000","last_unstale":"2023-10-21T16:18:45.485149+0000","last_undegraded":"2023-10-21T16:18:45.485149+0000","last_fullsized":"2023-10-21T16:18:45.485149+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:51:21.507107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152562+0000","last_change":"2023-10-21T16:18:44.521472+0000","last_active":"2023-10-21T16:18:46.152562+0000","last_peered":"2023-10-21T16:18:46.152562+0000","last_clean":"2023-10-21T16:18:46.152562+0000","last_became_active":"2023-10-21T16:18:44.521199+0000","last_became_peered":"2023-10-21T16:18:44.521199+0000","last_unstale":"2023-10-21T16:18:46.152562+0000","last_undegraded":"2023-10-21T16:18:46.152562+0000","last_fullsized":"2023-10-21T16:18:46.152562+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:09.773702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483880+0000","last_change":"2023-10-21T16:18:44.507052+0000","last_active":"2023-10-21T16:18:45.483880+0000","last_peered":"2023-10-21T16:18:45.483880+0000","last_clean":"2023-10-21T16:18:45.483880+0000","last_became_active":"2023-10-21T16:18:44.506744+0000","last_became_peered":"2023-10-21T16:18:44.506744+0000","last_unstale":"2023-10-21T16:18:45.483880+0000","last_undegraded":"2023-10-21T16:18:45.483880+0000","last_fullsized":"2023-10-21T16:18:45.483880+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:33.224873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483844+0000","last_change":"2023-10-21T16:18:44.511503+0000","last_active":"2023-10-21T16:18:45.483844+0000","last_peered":"2023-10-21T16:18:45.483844+0000","last_clean":"2023-10-21T16:18:45.483844+0000","last_became_active":"2023-10-21T16:18:44.511064+0000","last_became_peered":"2023-10-21T16:18:44.511064+0000","last_unstale":"2023-10-21T16:18:45.483844+0000","last_undegraded":"2023-10-21T16:18:45.483844+0000","last_fullsized":"2023-10-21T16:18:45.483844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:06.433654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590464+0000","last_change":"2023-10-21T16:18:44.521094+0000","last_active":"2023-10-21T16:18:46.590464+0000","last_peered":"2023-10-21T16:18:46.590464+0000","last_clean":"2023-10-21T16:18:46.590464+0000","last_became_active":"2023-10-21T16:18:44.520821+0000","last_became_peered":"2023-10-21T16:18:44.520821+0000","last_unstale":"2023-10-21T16:18:46.590464+0000","last_undegraded":"2023-10-21T16:18:46.590464+0000","last_fullsized":"2023-10-21T16:18:46.590464+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:47.480535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479480+0000","last_change":"2023-10-21T16:18:44.499756+0000","last_active":"2023-10-21T16:18:45.479480+0000","last_peered":"2023-10-21T16:18:45.479480+0000","last_clean":"2023-10-21T16:18:45.479480+0000","last_became_active":"2023-10-21T16:18:44.499379+0000","last_became_peered":"2023-10-21T16:18:44.499379+0000","last_unstale":"2023-10-21T16:18:45.479480+0000","last_undegraded":"2023-10-21T16:18:45.479480+0000","last_fullsized":"2023-10-21T16:18:45.479480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:14.452424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594414+0000","last_change":"2023-10-21T16:18:44.496921+0000","last_active":"2023-10-21T16:18:46.594414+0000","last_peered":"2023-10-21T16:18:46.594414+0000","last_clean":"2023-10-21T16:18:46.594414+0000","last_became_active":"2023-10-21T16:18:44.496630+0000","last_became_peered":"2023-10-21T16:18:44.496630+0000","last_unstale":"2023-10-21T16:18:46.594414+0000","last_undegraded":"2023-10-21T16:18:46.594414+0000","last_fullsized":"2023-10-21T16:18:46.594414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:29.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484676+0000","last_change":"2023-10-21T16:18:44.517337+0000","last_active":"2023-10-21T16:18:45.484676+0000","last_peered":"2023-10-21T16:18:45.484676+0000","last_clean":"2023-10-21T16:18:45.484676+0000","last_became_active":"2023-10-21T16:18:44.516975+0000","last_became_peered":"2023-10-21T16:18:44.516975+0000","last_unstale":"2023-10-21T16:18:45.484676+0000","last_undegraded":"2023-10-21T16:18:45.484676+0000","last_fullsized":"2023-10-21T16:18:45.484676+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:36.541903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481444+0000","last_change":"2023-10-21T16:18:44.498915+0000","last_active":"2023-10-21T16:18:45.481444+0000","last_peered":"2023-10-21T16:18:45.481444+0000","last_clean":"2023-10-21T16:18:45.481444+0000","last_became_active":"2023-10-21T16:18:44.498634+0000","last_became_peered":"2023-10-21T16:18:44.498634+0000","last_unstale":"2023-10-21T16:18:45.481444+0000","last_undegraded":"2023-10-21T16:18:45.481444+0000","last_fullsized":"2023-10-21T16:18:45.481444+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:38.301212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152699+0000","last_change":"2023-10-21T16:18:44.518091+0000","last_active":"2023-10-21T16:18:46.152699+0000","last_peered":"2023-10-21T16:18:46.152699+0000","last_clean":"2023-10-21T16:18:46.152699+0000","last_became_active":"2023-10-21T16:18:44.517782+0000","last_became_peered":"2023-10-21T16:18:44.517782+0000","last_unstale":"2023-10-21T16:18:46.152699+0000","last_undegraded":"2023-10-21T16:18:46.152699+0000","last_fullsized":"2023-10-21T16:18:46.152699+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:36.541903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590021+0000","last_change":"2023-10-21T16:18:44.513515+0000","last_active":"2023-10-21T16:18:46.590021+0000","last_peered":"2023-10-21T16:18:46.590021+0000","last_clean":"2023-10-21T16:18:46.590021+0000","last_became_active":"2023-10-21T16:18:44.513175+0000","last_became_peered":"2023-10-21T16:18:44.513175+0000","last_unstale":"2023-10-21T16:18:46.590021+0000","last_undegraded":"2023-10-21T16:18:46.590021+0000","last_fullsized":"2023-10-21T16:18:46.590021+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:55.195177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484218+0000","last_change":"2023-10-21T16:18:44.497774+0000","last_active":"2023-10-21T16:18:45.484218+0000","last_peered":"2023-10-21T16:18:45.484218+0000","last_clean":"2023-10-21T16:18:45.484218+0000","last_became_active":"2023-10-21T16:18:44.497510+0000","last_became_peered":"2023-10-21T16:18:44.497510+0000","last_unstale":"2023-10-21T16:18:45.484218+0000","last_undegraded":"2023-10-21T16:18:45.484218+0000","last_fullsized":"2023-10-21T16:18:45.484218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:57.790370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593283+0000","last_change":"2023-10-21T16:18:44.514907+0000","last_active":"2023-10-21T16:18:46.593283+0000","last_peered":"2023-10-21T16:18:46.593283+0000","last_clean":"2023-10-21T16:18:46.593283+0000","last_became_active":"2023-10-21T16:18:44.514612+0000","last_became_peered":"2023-10-21T16:18:44.514612+0000","last_unstale":"2023-10-21T16:18:46.593283+0000","last_undegraded":"2023-10-21T16:18:46.593283+0000","last_fullsized":"2023-10-21T16:18:46.593283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:20.497259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587745+0000","last_change":"2023-10-21T16:18:44.506531+0000","last_active":"2023-10-21T16:18:46.587745+0000","last_peered":"2023-10-21T16:18:46.587745+0000","last_clean":"2023-10-21T16:18:46.587745+0000","last_became_active":"2023-10-21T16:18:44.506266+0000","last_became_peered":"2023-10-21T16:18:44.506266+0000","last_unstale":"2023-10-21T16:18:46.587745+0000","last_undegraded":"2023-10-21T16:18:46.587745+0000","last_fullsized":"2023-10-21T16:18:46.587745+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:52.606492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588060+0000","last_change":"2023-10-21T16:18:44.509302+0000","last_active":"2023-10-21T16:18:46.588060+0000","last_peered":"2023-10-21T16:18:46.588060+0000","last_clean":"2023-10-21T16:18:46.588060+0000","last_became_active":"2023-10-21T16:18:44.508955+0000","last_became_peered":"2023-10-21T16:18:44.508955+0000","last_unstale":"2023-10-21T16:18:46.588060+0000","last_undegraded":"2023-10-21T16:18:46.588060+0000","last_fullsized":"2023-10-21T16:18:46.588060+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:28.992489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"116'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.592429+0000","last_change":"2023-10-21T16:18:43.472736+0000","last_active":"2023-10-21T16:18:46.592429+0000","last_peered":"2023-10-21T16:18:46.592429+0000","last_clean":"2023-10-21T16:18:46.592429+0000","last_became_active":"2023-10-21T16:18:43.471068+0000","last_became_peered":"2023-10-21T16:18:43.471068+0000","last_unstale":"2023-10-21T16:18:46.592429+0000","last_undegraded":"2023-10-21T16:18:46.592429+0000","last_fullsized":"2023-10-21T16:18:46.592429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:58:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588256+0000","last_change":"2023-10-21T16:18:44.503366+0000","last_active":"2023-10-21T16:18:46.588256+0000","last_peered":"2023-10-21T16:18:46.588256+0000","last_clean":"2023-10-21T16:18:46.588256+0000","last_became_active":"2023-10-21T16:18:44.503074+0000","last_became_peered":"2023-10-21T16:18:44.503074+0000","last_unstale":"2023-10-21T16:18:46.588256+0000","last_undegraded":"2023-10-21T16:18:46.588256+0000","last_fullsized":"2023-10-21T16:18:46.588256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:58.142635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484726+0000","last_change":"2023-10-21T16:18:43.476465+0000","last_active":"2023-10-21T16:18:45.484726+0000","last_peered":"2023-10-21T16:18:45.484726+0000","last_clean":"2023-10-21T16:18:45.484726+0000","last_became_active":"2023-10-21T16:18:43.476097+0000","last_became_peered":"2023-10-21T16:18:43.476097+0000","last_unstale":"2023-10-21T16:18:45.484726+0000","last_undegraded":"2023-10-21T16:18:45.484726+0000","last_fullsized":"2023-10-21T16:18:45.484726+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:46.162372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589716+0000","last_change":"2023-10-21T16:18:44.514052+0000","last_active":"2023-10-21T16:18:46.589716+0000","last_peered":"2023-10-21T16:18:46.589716+0000","last_clean":"2023-10-21T16:18:46.589716+0000","last_became_active":"2023-10-21T16:18:44.513686+0000","last_became_peered":"2023-10-21T16:18:44.513686+0000","last_unstale":"2023-10-21T16:18:46.589716+0000","last_undegraded":"2023-10-21T16:18:46.589716+0000","last_fullsized":"2023-10-21T16:18:46.589716+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:19.832284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590062+0000","last_change":"2023-10-21T16:18:43.477554+0000","last_active":"2023-10-21T16:18:46.590062+0000","last_peered":"2023-10-21T16:18:46.590062+0000","last_clean":"2023-10-21T16:18:46.590062+0000","last_became_active":"2023-10-21T16:18:43.477293+0000","last_became_peered":"2023-10-21T16:18:43.477293+0000","last_unstale":"2023-10-21T16:18:46.590062+0000","last_undegraded":"2023-10-21T16:18:46.590062+0000","last_fullsized":"2023-10-21T16:18:46.590062+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:31:40.394917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588450+0000","last_change":"2023-10-21T16:18:44.502663+0000","last_active":"2023-10-21T16:18:46.588450+0000","last_peered":"2023-10-21T16:18:46.588450+0000","last_clean":"2023-10-21T16:18:46.588450+0000","last_became_active":"2023-10-21T16:18:44.502306+0000","last_became_peered":"2023-10-21T16:18:44.502306+0000","last_unstale":"2023-10-21T16:18:46.588450+0000","last_undegraded":"2023-10-21T16:18:46.588450+0000","last_fullsized":"2023-10-21T16:18:46.588450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:40:20.933122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591928+0000","last_change":"2023-10-21T16:18:43.473558+0000","last_active":"2023-10-21T16:18:46.591928+0000","last_peered":"2023-10-21T16:18:46.591928+0000","last_clean":"2023-10-21T16:18:46.591928+0000","last_became_active":"2023-10-21T16:18:43.471659+0000","last_became_peered":"2023-10-21T16:18:43.471659+0000","last_unstale":"2023-10-21T16:18:46.591928+0000","last_undegraded":"2023-10-21T16:18:46.591928+0000","last_fullsized":"2023-10-21T16:18:46.591928+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:23:29.439305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151750+0000","last_change":"2023-10-21T16:18:44.525317+0000","last_active":"2023-10-21T16:18:46.151750+0000","last_peered":"2023-10-21T16:18:46.151750+0000","last_clean":"2023-10-21T16:18:46.151750+0000","last_became_active":"2023-10-21T16:18:44.524147+0000","last_became_peered":"2023-10-21T16:18:44.524147+0000","last_unstale":"2023-10-21T16:18:46.151750+0000","last_undegraded":"2023-10-21T16:18:46.151750+0000","last_fullsized":"2023-10-21T16:18:46.151750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:45.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484157+0000","last_change":"2023-10-21T16:18:43.472924+0000","last_active":"2023-10-21T16:18:45.484157+0000","last_peered":"2023-10-21T16:18:45.484157+0000","last_clean":"2023-10-21T16:18:45.484157+0000","last_became_active":"2023-10-21T16:18:43.468620+0000","last_became_peered":"2023-10-21T16:18:43.468620+0000","last_unstale":"2023-10-21T16:18:45.484157+0000","last_undegraded":"2023-10-21T16:18:45.484157+0000","last_fullsized":"2023-10-21T16:18:45.484157+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:28.965067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590794+0000","last_change":"2023-10-21T16:18:44.512280+0000","last_active":"2023-10-21T16:18:46.590794+0000","last_peered":"2023-10-21T16:18:46.590794+0000","last_clean":"2023-10-21T16:18:46.590794+0000","last_became_active":"2023-10-21T16:18:44.511655+0000","last_became_peered":"2023-10-21T16:18:44.511655+0000","last_unstale":"2023-10-21T16:18:46.590794+0000","last_undegraded":"2023-10-21T16:18:46.590794+0000","last_fullsized":"2023-10-21T16:18:46.590794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:53:40.915732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152629+0000","last_change":"2023-10-21T16:18:43.469665+0000","last_active":"2023-10-21T16:18:46.152629+0000","last_peered":"2023-10-21T16:18:46.152629+0000","last_clean":"2023-10-21T16:18:46.152629+0000","last_became_active":"2023-10-21T16:18:43.469386+0000","last_became_peered":"2023-10-21T16:18:43.469386+0000","last_unstale":"2023-10-21T16:18:46.152629+0000","last_undegraded":"2023-10-21T16:18:46.152629+0000","last_fullsized":"2023-10-21T16:18:46.152629+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:28.965067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587800+0000","last_change":"2023-10-21T16:18:44.520414+0000","last_active":"2023-10-21T16:18:46.587800+0000","last_peered":"2023-10-21T16:18:46.587800+0000","last_clean":"2023-10-21T16:18:46.587800+0000","last_became_active":"2023-10-21T16:18:44.520133+0000","last_became_peered":"2023-10-21T16:18:44.520133+0000","last_unstale":"2023-10-21T16:18:46.587800+0000","last_undegraded":"2023-10-21T16:18:46.587800+0000","last_fullsized":"2023-10-21T16:18:46.587800+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:57:37.344675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479779+0000","last_change":"2023-10-21T16:18:43.480874+0000","last_active":"2023-10-21T16:18:45.479779+0000","last_peered":"2023-10-21T16:18:45.479779+0000","last_clean":"2023-10-21T16:18:45.479779+0000","last_became_active":"2023-10-21T16:18:43.480577+0000","last_became_peered":"2023-10-21T16:18:43.480577+0000","last_unstale":"2023-10-21T16:18:45.479779+0000","last_undegraded":"2023-10-21T16:18:45.479779+0000","last_fullsized":"2023-10-21T16:18:45.479779+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:38:03.901484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152261+0000","last_change":"2023-10-21T16:18:44.516513+0000","last_active":"2023-10-21T16:18:46.152261+0000","last_peered":"2023-10-21T16:18:46.152261+0000","last_clean":"2023-10-21T16:18:46.152261+0000","last_became_active":"2023-10-21T16:18:44.514408+0000","last_became_peered":"2023-10-21T16:18:44.514408+0000","last_unstale":"2023-10-21T16:18:46.152261+0000","last_undegraded":"2023-10-21T16:18:46.152261+0000","last_fullsized":"2023-10-21T16:18:46.152261+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:30.636045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589149+0000","last_change":"2023-10-21T16:18:43.480360+0000","last_active":"2023-10-21T16:18:46.589149+0000","last_peered":"2023-10-21T16:18:46.589149+0000","last_clean":"2023-10-21T16:18:46.589149+0000","last_became_active":"2023-10-21T16:18:43.480083+0000","last_became_peered":"2023-10-21T16:18:43.480083+0000","last_unstale":"2023-10-21T16:18:46.589149+0000","last_undegraded":"2023-10-21T16:18:46.589149+0000","last_fullsized":"2023-10-21T16:18:46.589149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:07:16.152151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484511+0000","last_change":"2023-10-21T16:18:44.494929+0000","last_active":"2023-10-21T16:18:45.484511+0000","last_peered":"2023-10-21T16:18:45.484511+0000","last_clean":"2023-10-21T16:18:45.484511+0000","last_became_active":"2023-10-21T16:18:44.494446+0000","last_became_peered":"2023-10-21T16:18:44.494446+0000","last_unstale":"2023-10-21T16:18:45.484511+0000","last_undegraded":"2023-10-21T16:18:45.484511+0000","last_fullsized":"2023-10-21T16:18:45.484511+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:59.110459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594541+0000","last_change":"2023-10-21T16:18:43.458733+0000","last_active":"2023-10-21T16:18:46.594541+0000","last_peered":"2023-10-21T16:18:46.594541+0000","last_clean":"2023-10-21T16:18:46.594541+0000","last_became_active":"2023-10-21T16:18:43.458135+0000","last_became_peered":"2023-10-21T16:18:43.458135+0000","last_unstale":"2023-10-21T16:18:46.594541+0000","last_undegraded":"2023-10-21T16:18:46.594541+0000","last_fullsized":"2023-10-21T16:18:46.594541+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:57.128379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590071+0000","last_change":"2023-10-21T16:18:44.513109+0000","last_active":"2023-10-21T16:18:46.590071+0000","last_peered":"2023-10-21T16:18:46.590071+0000","last_clean":"2023-10-21T16:18:46.590071+0000","last_became_active":"2023-10-21T16:18:44.512856+0000","last_became_peered":"2023-10-21T16:18:44.512856+0000","last_unstale":"2023-10-21T16:18:46.590071+0000","last_undegraded":"2023-10-21T16:18:46.590071+0000","last_fullsized":"2023-10-21T16:18:46.590071+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:52.606492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480469+0000","last_change":"2023-10-21T16:18:43.484552+0000","last_active":"2023-10-21T16:18:45.480469+0000","last_peered":"2023-10-21T16:18:45.480469+0000","last_clean":"2023-10-21T16:18:45.480469+0000","last_became_active":"2023-10-21T16:18:43.484279+0000","last_became_peered":"2023-10-21T16:18:43.484279+0000","last_unstale":"2023-10-21T16:18:45.480469+0000","last_undegraded":"2023-10-21T16:18:45.480469+0000","last_fullsized":"2023-10-21T16:18:45.480469+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:49:30.170660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480444+0000","last_change":"2023-10-21T16:18:44.508213+0000","last_active":"2023-10-21T16:18:45.480444+0000","last_peered":"2023-10-21T16:18:45.480444+0000","last_clean":"2023-10-21T16:18:45.480444+0000","last_became_active":"2023-10-21T16:18:44.505331+0000","last_became_peered":"2023-10-21T16:18:44.505331+0000","last_unstale":"2023-10-21T16:18:45.480444+0000","last_undegraded":"2023-10-21T16:18:45.480444+0000","last_fullsized":"2023-10-21T16:18:45.480444+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:15.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"116'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485190+0000","last_change":"2023-10-21T16:18:43.475140+0000","last_active":"2023-10-21T16:18:45.485190+0000","last_peered":"2023-10-21T16:18:45.485190+0000","last_clean":"2023-10-21T16:18:45.485190+0000","last_became_active":"2023-10-21T16:18:43.473184+0000","last_became_peered":"2023-10-21T16:18:43.473184+0000","last_unstale":"2023-10-21T16:18:45.485190+0000","last_undegraded":"2023-10-21T16:18:45.485190+0000","last_fullsized":"2023-10-21T16:18:45.485190+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:18.402787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152298+0000","last_change":"2023-10-21T16:18:44.517097+0000","last_active":"2023-10-21T16:18:46.152298+0000","last_peered":"2023-10-21T16:18:46.152298+0000","last_clean":"2023-10-21T16:18:46.152298+0000","last_became_active":"2023-10-21T16:18:44.516793+0000","last_became_peered":"2023-10-21T16:18:44.516793+0000","last_unstale":"2023-10-21T16:18:46.152298+0000","last_undegraded":"2023-10-21T16:18:46.152298+0000","last_fullsized":"2023-10-21T16:18:46.152298+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:57.790370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"116'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588459+0000","last_change":"2023-10-21T16:18:43.478276+0000","last_active":"2023-10-21T16:18:46.588459+0000","last_peered":"2023-10-21T16:18:46.588459+0000","last_clean":"2023-10-21T16:18:46.588459+0000","last_became_active":"2023-10-21T16:18:43.477985+0000","last_became_peered":"2023-10-21T16:18:43.477985+0000","last_unstale":"2023-10-21T16:18:46.588459+0000","last_undegraded":"2023-10-21T16:18:46.588459+0000","last_fullsized":"2023-10-21T16:18:46.588459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:09.415956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152121+0000","last_change":"2023-10-21T16:18:44.505940+0000","last_active":"2023-10-21T16:18:46.152121+0000","last_peered":"2023-10-21T16:18:46.152121+0000","last_clean":"2023-10-21T16:18:46.152121+0000","last_became_active":"2023-10-21T16:18:44.505622+0000","last_became_peered":"2023-10-21T16:18:44.505622+0000","last_unstale":"2023-10-21T16:18:46.152121+0000","last_undegraded":"2023-10-21T16:18:46.152121+0000","last_fullsized":"2023-10-21T16:18:46.152121+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:33.224873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589695+0000","last_change":"2023-10-21T16:18:43.480810+0000","last_active":"2023-10-21T16:18:46.589695+0000","last_peered":"2023-10-21T16:18:46.589695+0000","last_clean":"2023-10-21T16:18:46.589695+0000","last_became_active":"2023-10-21T16:18:43.478976+0000","last_became_peered":"2023-10-21T16:18:43.478976+0000","last_unstale":"2023-10-21T16:18:46.589695+0000","last_undegraded":"2023-10-21T16:18:46.589695+0000","last_fullsized":"2023-10-21T16:18:46.589695+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:27.544544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483235+0000","last_change":"2023-10-21T16:18:44.502900+0000","last_active":"2023-10-21T16:18:45.483235+0000","last_peered":"2023-10-21T16:18:45.483235+0000","last_clean":"2023-10-21T16:18:45.483235+0000","last_became_active":"2023-10-21T16:18:44.502614+0000","last_became_peered":"2023-10-21T16:18:44.502614+0000","last_unstale":"2023-10-21T16:18:45.483235+0000","last_undegraded":"2023-10-21T16:18:45.483235+0000","last_fullsized":"2023-10-21T16:18:45.483235+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:45.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588201+0000","last_change":"2023-10-21T16:18:43.479657+0000","last_active":"2023-10-21T16:18:46.588201+0000","last_peered":"2023-10-21T16:18:46.588201+0000","last_clean":"2023-10-21T16:18:46.588201+0000","last_became_active":"2023-10-21T16:18:43.479363+0000","last_became_peered":"2023-10-21T16:18:43.479363+0000","last_unstale":"2023-10-21T16:18:46.588201+0000","last_undegraded":"2023-10-21T16:18:46.588201+0000","last_fullsized":"2023-10-21T16:18:46.588201+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:18.818029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481085+0000","last_change":"2023-10-21T16:18:44.513072+0000","last_active":"2023-10-21T16:18:45.481085+0000","last_peered":"2023-10-21T16:18:45.481085+0000","last_clean":"2023-10-21T16:18:45.481085+0000","last_became_active":"2023-10-21T16:18:44.508690+0000","last_became_peered":"2023-10-21T16:18:44.508690+0000","last_unstale":"2023-10-21T16:18:45.481085+0000","last_undegraded":"2023-10-21T16:18:45.481085+0000","last_fullsized":"2023-10-21T16:18:45.481085+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:19:09.972900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152149+0000","last_change":"2023-10-21T16:18:43.478821+0000","last_active":"2023-10-21T16:18:46.152149+0000","last_peered":"2023-10-21T16:18:46.152149+0000","last_clean":"2023-10-21T16:18:46.152149+0000","last_became_active":"2023-10-21T16:18:43.478552+0000","last_became_peered":"2023-10-21T16:18:43.478552+0000","last_unstale":"2023-10-21T16:18:46.152149+0000","last_undegraded":"2023-10-21T16:18:46.152149+0000","last_fullsized":"2023-10-21T16:18:46.152149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:51.662878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481286+0000","last_change":"2023-10-21T16:18:44.496779+0000","last_active":"2023-10-21T16:18:45.481286+0000","last_peered":"2023-10-21T16:18:45.481286+0000","last_clean":"2023-10-21T16:18:45.481286+0000","last_became_active":"2023-10-21T16:18:44.496535+0000","last_became_peered":"2023-10-21T16:18:44.496535+0000","last_unstale":"2023-10-21T16:18:45.481286+0000","last_undegraded":"2023-10-21T16:18:45.481286+0000","last_fullsized":"2023-10-21T16:18:45.481286+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:35:11.877058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"116'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152091+0000","last_change":"2023-10-21T16:18:43.457220+0000","last_active":"2023-10-21T16:18:46.152091+0000","last_peered":"2023-10-21T16:18:46.152091+0000","last_clean":"2023-10-21T16:18:46.152091+0000","last_became_active":"2023-10-21T16:18:43.456909+0000","last_became_peered":"2023-10-21T16:18:43.456909+0000","last_unstale":"2023-10-21T16:18:46.152091+0000","last_undegraded":"2023-10-21T16:18:46.152091+0000","last_fullsized":"2023-10-21T16:18:46.152091+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:20:59.285099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588312+0000","last_change":"2023-10-21T16:18:44.507652+0000","last_active":"2023-10-21T16:18:46.588312+0000","last_peered":"2023-10-21T16:18:46.588312+0000","last_clean":"2023-10-21T16:18:46.588312+0000","last_became_active":"2023-10-21T16:18:44.507276+0000","last_became_peered":"2023-10-21T16:18:44.507276+0000","last_unstale":"2023-10-21T16:18:46.588312+0000","last_undegraded":"2023-10-21T16:18:46.588312+0000","last_fullsized":"2023-10-21T16:18:46.588312+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:20.497259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.592618+0000","last_change":"2023-10-21T16:18:43.476440+0000","last_active":"2023-10-21T16:18:46.592618+0000","last_peered":"2023-10-21T16:18:46.592618+0000","last_clean":"2023-10-21T16:18:46.592618+0000","last_became_active":"2023-10-21T16:18:43.474936+0000","last_became_peered":"2023-10-21T16:18:43.474936+0000","last_unstale":"2023-10-21T16:18:46.592618+0000","last_undegraded":"2023-10-21T16:18:46.592618+0000","last_fullsized":"2023-10-21T16:18:46.592618+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:23.552770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591600+0000","last_change":"2023-10-21T16:18:44.499977+0000","last_active":"2023-10-21T16:18:46.591600+0000","last_peered":"2023-10-21T16:18:46.591600+0000","last_clean":"2023-10-21T16:18:46.591600+0000","last_became_active":"2023-10-21T16:18:44.499631+0000","last_became_peered":"2023-10-21T16:18:44.499631+0000","last_unstale":"2023-10-21T16:18:46.591600+0000","last_undegraded":"2023-10-21T16:18:46.591600+0000","last_fullsized":"2023-10-21T16:18:46.591600+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:19.735217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589805+0000","last_change":"2023-10-21T16:18:43.479934+0000","last_active":"2023-10-21T16:18:46.589805+0000","last_peered":"2023-10-21T16:18:46.589805+0000","last_clean":"2023-10-21T16:18:46.589805+0000","last_became_active":"2023-10-21T16:18:43.479664+0000","last_became_peered":"2023-10-21T16:18:43.479664+0000","last_unstale":"2023-10-21T16:18:46.589805+0000","last_undegraded":"2023-10-21T16:18:46.589805+0000","last_fullsized":"2023-10-21T16:18:46.589805+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:55.078106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484603+0000","last_change":"2023-10-21T16:18:44.495184+0000","last_active":"2023-10-21T16:18:45.484603+0000","last_peered":"2023-10-21T16:18:45.484603+0000","last_clean":"2023-10-21T16:18:45.484603+0000","last_became_active":"2023-10-21T16:18:44.493971+0000","last_became_peered":"2023-10-21T16:18:44.493971+0000","last_unstale":"2023-10-21T16:18:45.484603+0000","last_undegraded":"2023-10-21T16:18:45.484603+0000","last_fullsized":"2023-10-21T16:18:45.484603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:09.773702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485293+0000","last_change":"2023-10-21T16:18:43.470205+0000","last_active":"2023-10-21T16:18:45.485293+0000","last_peered":"2023-10-21T16:18:45.485293+0000","last_clean":"2023-10-21T16:18:45.485293+0000","last_became_active":"2023-10-21T16:18:43.469923+0000","last_became_peered":"2023-10-21T16:18:43.469923+0000","last_unstale":"2023-10-21T16:18:45.485293+0000","last_undegraded":"2023-10-21T16:18:45.485293+0000","last_fullsized":"2023-10-21T16:18:45.485293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:20:59.285099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587991+0000","last_change":"2023-10-21T16:18:44.514463+0000","last_active":"2023-10-21T16:18:46.587991+0000","last_peered":"2023-10-21T16:18:46.587991+0000","last_clean":"2023-10-21T16:18:46.587991+0000","last_became_active":"2023-10-21T16:18:44.514170+0000","last_became_peered":"2023-10-21T16:18:44.514170+0000","last_unstale":"2023-10-21T16:18:46.587991+0000","last_undegraded":"2023-10-21T16:18:46.587991+0000","last_fullsized":"2023-10-21T16:18:46.587991+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:53.985741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593434+0000","last_change":"2023-10-21T16:18:43.479995+0000","last_active":"2023-10-21T16:18:46.593434+0000","last_peered":"2023-10-21T16:18:46.593434+0000","last_clean":"2023-10-21T16:18:46.593434+0000","last_became_active":"2023-10-21T16:18:43.479704+0000","last_became_peered":"2023-10-21T16:18:43.479704+0000","last_unstale":"2023-10-21T16:18:46.593434+0000","last_undegraded":"2023-10-21T16:18:46.593434+0000","last_fullsized":"2023-10-21T16:18:46.593434+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:18.818029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593384+0000","last_change":"2023-10-21T16:18:44.492428+0000","last_active":"2023-10-21T16:18:46.593384+0000","last_peered":"2023-10-21T16:18:46.593384+0000","last_clean":"2023-10-21T16:18:46.593384+0000","last_became_active":"2023-10-21T16:18:44.492053+0000","last_became_peered":"2023-10-21T16:18:44.492053+0000","last_unstale":"2023-10-21T16:18:46.593384+0000","last_undegraded":"2023-10-21T16:18:46.593384+0000","last_fullsized":"2023-10-21T16:18:46.593384+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:43.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481109+0000","last_change":"2023-10-21T16:18:43.456861+0000","last_active":"2023-10-21T16:18:45.481109+0000","last_peered":"2023-10-21T16:18:45.481109+0000","last_clean":"2023-10-21T16:18:45.481109+0000","last_became_active":"2023-10-21T16:18:43.456383+0000","last_became_peered":"2023-10-21T16:18:43.456383+0000","last_unstale":"2023-10-21T16:18:45.481109+0000","last_undegraded":"2023-10-21T16:18:45.481109+0000","last_fullsized":"2023-10-21T16:18:45.481109+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:05:46.953069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593519+0000","last_change":"2023-10-21T16:18:44.508785+0000","last_active":"2023-10-21T16:18:46.593519+0000","last_peered":"2023-10-21T16:18:46.593519+0000","last_clean":"2023-10-21T16:18:46.593519+0000","last_became_active":"2023-10-21T16:18:44.508491+0000","last_became_peered":"2023-10-21T16:18:44.508491+0000","last_unstale":"2023-10-21T16:18:46.593519+0000","last_undegraded":"2023-10-21T16:18:46.593519+0000","last_fullsized":"2023-10-21T16:18:46.593519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:04.757337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593542+0000","last_change":"2023-10-21T16:18:43.472937+0000","last_active":"2023-10-21T16:18:46.593542+0000","last_peered":"2023-10-21T16:18:46.593542+0000","last_clean":"2023-10-21T16:18:46.593542+0000","last_became_active":"2023-10-21T16:18:43.472641+0000","last_became_peered":"2023-10-21T16:18:43.472641+0000","last_unstale":"2023-10-21T16:18:46.593542+0000","last_undegraded":"2023-10-21T16:18:46.593542+0000","last_fullsized":"2023-10-21T16:18:46.593542+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:27.978233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588692+0000","last_change":"2023-10-21T16:18:44.503119+0000","last_active":"2023-10-21T16:18:46.588692+0000","last_peered":"2023-10-21T16:18:46.588692+0000","last_clean":"2023-10-21T16:18:46.588692+0000","last_became_active":"2023-10-21T16:18:44.502662+0000","last_became_peered":"2023-10-21T16:18:44.502662+0000","last_unstale":"2023-10-21T16:18:46.588692+0000","last_undegraded":"2023-10-21T16:18:46.588692+0000","last_fullsized":"2023-10-21T16:18:46.588692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:04.868404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481012+0000","last_change":"2023-10-21T16:18:43.474737+0000","last_active":"2023-10-21T16:18:45.481012+0000","last_peered":"2023-10-21T16:18:45.481012+0000","last_clean":"2023-10-21T16:18:45.481012+0000","last_became_active":"2023-10-21T16:18:43.473487+0000","last_became_peered":"2023-10-21T16:18:43.473487+0000","last_unstale":"2023-10-21T16:18:45.481012+0000","last_undegraded":"2023-10-21T16:18:45.481012+0000","last_fullsized":"2023-10-21T16:18:45.481012+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:56:09.613975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591684+0000","last_change":"2023-10-21T16:18:44.501334+0000","last_active":"2023-10-21T16:18:46.591684+0000","last_peered":"2023-10-21T16:18:46.591684+0000","last_clean":"2023-10-21T16:18:46.591684+0000","last_became_active":"2023-10-21T16:18:44.501068+0000","last_became_peered":"2023-10-21T16:18:44.501068+0000","last_unstale":"2023-10-21T16:18:46.591684+0000","last_undegraded":"2023-10-21T16:18:46.591684+0000","last_fullsized":"2023-10-21T16:18:46.591684+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:07.915939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590421+0000","last_change":"2023-10-21T16:18:43.482899+0000","last_active":"2023-10-21T16:18:46.590421+0000","last_peered":"2023-10-21T16:18:46.590421+0000","last_clean":"2023-10-21T16:18:46.590421+0000","last_became_active":"2023-10-21T16:18:43.482645+0000","last_became_peered":"2023-10-21T16:18:43.482645+0000","last_unstale":"2023-10-21T16:18:46.590421+0000","last_undegraded":"2023-10-21T16:18:46.590421+0000","last_fullsized":"2023-10-21T16:18:46.590421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:39.845768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588535+0000","last_change":"2023-10-21T16:18:44.509101+0000","last_active":"2023-10-21T16:18:46.588535+0000","last_peered":"2023-10-21T16:18:46.588535+0000","last_clean":"2023-10-21T16:18:46.588535+0000","last_became_active":"2023-10-21T16:18:44.508801+0000","last_became_peered":"2023-10-21T16:18:44.508801+0000","last_unstale":"2023-10-21T16:18:46.588535+0000","last_undegraded":"2023-10-21T16:18:46.588535+0000","last_fullsized":"2023-10-21T16:18:46.588535+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:29.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484939+0000","last_change":"2023-10-21T16:18:43.471057+0000","last_active":"2023-10-21T16:18:45.484939+0000","last_peered":"2023-10-21T16:18:45.484939+0000","last_clean":"2023-10-21T16:18:45.484939+0000","last_became_active":"2023-10-21T16:18:43.469327+0000","last_became_peered":"2023-10-21T16:18:43.469327+0000","last_unstale":"2023-10-21T16:18:45.484939+0000","last_undegraded":"2023-10-21T16:18:45.484939+0000","last_fullsized":"2023-10-21T16:18:45.484939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:18.131521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588401+0000","last_change":"2023-10-21T16:18:44.500914+0000","last_active":"2023-10-21T16:18:46.588401+0000","last_peered":"2023-10-21T16:18:46.588401+0000","last_clean":"2023-10-21T16:18:46.588401+0000","last_became_active":"2023-10-21T16:18:44.500631+0000","last_became_peered":"2023-10-21T16:18:44.500631+0000","last_unstale":"2023-10-21T16:18:46.588401+0000","last_undegraded":"2023-10-21T16:18:46.588401+0000","last_fullsized":"2023-10-21T16:18:46.588401+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:04.757337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485313+0000","last_change":"2023-10-21T16:18:43.476754+0000","last_active":"2023-10-21T16:18:45.485313+0000","last_peered":"2023-10-21T16:18:45.485313+0000","last_clean":"2023-10-21T16:18:45.485313+0000","last_became_active":"2023-10-21T16:18:43.475367+0000","last_became_peered":"2023-10-21T16:18:43.475367+0000","last_unstale":"2023-10-21T16:18:45.485313+0000","last_undegraded":"2023-10-21T16:18:45.485313+0000","last_fullsized":"2023-10-21T16:18:45.485313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:51.662878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594570+0000","last_change":"2023-10-21T16:18:44.506782+0000","last_active":"2023-10-21T16:18:46.594570+0000","last_peered":"2023-10-21T16:18:46.594570+0000","last_clean":"2023-10-21T16:18:46.594570+0000","last_became_active":"2023-10-21T16:18:44.503178+0000","last_became_peered":"2023-10-21T16:18:44.503178+0000","last_unstale":"2023-10-21T16:18:46.594570+0000","last_undegraded":"2023-10-21T16:18:46.594570+0000","last_fullsized":"2023-10-21T16:18:46.594570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:04.868404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591918+0000","last_change":"2023-10-21T16:18:43.478809+0000","last_active":"2023-10-21T16:18:46.591918+0000","last_peered":"2023-10-21T16:18:46.591918+0000","last_clean":"2023-10-21T16:18:46.591918+0000","last_became_active":"2023-10-21T16:18:43.478515+0000","last_became_peered":"2023-10-21T16:18:43.478515+0000","last_unstale":"2023-10-21T16:18:46.591918+0000","last_undegraded":"2023-10-21T16:18:46.591918+0000","last_fullsized":"2023-10-21T16:18:46.591918+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:59.144071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480976+0000","last_change":"2023-10-21T16:18:44.513083+0000","last_active":"2023-10-21T16:18:45.480976+0000","last_peered":"2023-10-21T16:18:45.480976+0000","last_clean":"2023-10-21T16:18:45.480976+0000","last_became_active":"2023-10-21T16:18:44.512695+0000","last_became_peered":"2023-10-21T16:18:44.512695+0000","last_unstale":"2023-10-21T16:18:45.480976+0000","last_undegraded":"2023-10-21T16:18:45.480976+0000","last_fullsized":"2023-10-21T16:18:45.480976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:13.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588267+0000","last_change":"2023-10-21T16:18:43.479555+0000","last_active":"2023-10-21T16:18:46.588267+0000","last_peered":"2023-10-21T16:18:46.588267+0000","last_clean":"2023-10-21T16:18:46.588267+0000","last_became_active":"2023-10-21T16:18:43.479235+0000","last_became_peered":"2023-10-21T16:18:43.479235+0000","last_unstale":"2023-10-21T16:18:46.588267+0000","last_undegraded":"2023-10-21T16:18:46.588267+0000","last_fullsized":"2023-10-21T16:18:46.588267+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:39.845768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479621+0000","last_change":"2023-10-21T16:18:44.518321+0000","last_active":"2023-10-21T16:18:45.479621+0000","last_peered":"2023-10-21T16:18:45.479621+0000","last_clean":"2023-10-21T16:18:45.479621+0000","last_became_active":"2023-10-21T16:18:44.517603+0000","last_became_peered":"2023-10-21T16:18:44.517603+0000","last_unstale":"2023-10-21T16:18:45.479621+0000","last_undegraded":"2023-10-21T16:18:45.479621+0000","last_fullsized":"2023-10-21T16:18:45.479621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:17:30.760547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593497+0000","last_change":"2023-10-21T16:18:43.475615+0000","last_active":"2023-10-21T16:18:46.593497+0000","last_peered":"2023-10-21T16:18:46.593497+0000","last_clean":"2023-10-21T16:18:46.593497+0000","last_became_active":"2023-10-21T16:18:43.475334+0000","last_became_peered":"2023-10-21T16:18:43.475334+0000","last_unstale":"2023-10-21T16:18:46.593497+0000","last_undegraded":"2023-10-21T16:18:46.593497+0000","last_fullsized":"2023-10-21T16:18:46.593497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:51.592237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590871+0000","last_change":"2023-10-21T16:18:44.521812+0000","last_active":"2023-10-21T16:18:46.590871+0000","last_peered":"2023-10-21T16:18:46.590871+0000","last_clean":"2023-10-21T16:18:46.590871+0000","last_became_active":"2023-10-21T16:18:44.521494+0000","last_became_peered":"2023-10-21T16:18:44.521494+0000","last_unstale":"2023-10-21T16:18:46.590871+0000","last_undegraded":"2023-10-21T16:18:46.590871+0000","last_fullsized":"2023-10-21T16:18:46.590871+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:39:43.115726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591009+0000","last_change":"2023-10-21T16:18:43.478777+0000","last_active":"2023-10-21T16:18:46.591009+0000","last_peered":"2023-10-21T16:18:46.591009+0000","last_clean":"2023-10-21T16:18:46.591009+0000","last_became_active":"2023-10-21T16:18:43.478433+0000","last_became_peered":"2023-10-21T16:18:43.478433+0000","last_unstale":"2023-10-21T16:18:46.591009+0000","last_undegraded":"2023-10-21T16:18:46.591009+0000","last_fullsized":"2023-10-21T16:18:46.591009+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:46:54.154752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588109+0000","last_change":"2023-10-21T16:18:44.502939+0000","last_active":"2023-10-21T16:18:46.588109+0000","last_peered":"2023-10-21T16:18:46.588109+0000","last_clean":"2023-10-21T16:18:46.588109+0000","last_became_active":"2023-10-21T16:18:44.502339+0000","last_became_peered":"2023-10-21T16:18:44.502339+0000","last_unstale":"2023-10-21T16:18:46.588109+0000","last_undegraded":"2023-10-21T16:18:46.588109+0000","last_fullsized":"2023-10-21T16:18:46.588109+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:34.326495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589180+0000","last_change":"2023-10-21T16:18:43.475338+0000","last_active":"2023-10-21T16:18:46.589180+0000","last_peered":"2023-10-21T16:18:46.589180+0000","last_clean":"2023-10-21T16:18:46.589180+0000","last_became_active":"2023-10-21T16:18:43.475076+0000","last_became_peered":"2023-10-21T16:18:43.475076+0000","last_unstale":"2023-10-21T16:18:46.589180+0000","last_undegraded":"2023-10-21T16:18:46.589180+0000","last_fullsized":"2023-10-21T16:18:46.589180+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:00:11.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590273+0000","last_change":"2023-10-21T16:18:44.511539+0000","last_active":"2023-10-21T16:18:46.590273+0000","last_peered":"2023-10-21T16:18:46.590273+0000","last_clean":"2023-10-21T16:18:46.590273+0000","last_became_active":"2023-10-21T16:18:44.511172+0000","last_became_peered":"2023-10-21T16:18:44.511172+0000","last_unstale":"2023-10-21T16:18:46.590273+0000","last_undegraded":"2023-10-21T16:18:46.590273+0000","last_fullsized":"2023-10-21T16:18:46.590273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:10.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152951+0000","last_change":"2023-10-21T16:18:43.473411+0000","last_active":"2023-10-21T16:18:46.152951+0000","last_peered":"2023-10-21T16:18:46.152951+0000","last_clean":"2023-10-21T16:18:46.152951+0000","last_became_active":"2023-10-21T16:18:43.473147+0000","last_became_peered":"2023-10-21T16:18:43.473147+0000","last_unstale":"2023-10-21T16:18:46.152951+0000","last_undegraded":"2023-10-21T16:18:46.152951+0000","last_fullsized":"2023-10-21T16:18:46.152951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:18.402787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"117'4301","reported_seq":4639,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.706732+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:18:46.706732+0000","last_peered":"2023-10-21T16:18:46.706732+0000","last_clean":"2023-10-21T16:18:46.706732+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:18:46.706732+0000","last_undegraded":"2023-10-21T16:18:46.706732+0000","last_fullsized":"2023-10-21T16:18:46.706732+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4301,"log_dups_size":0,"ondisk_log_size":4301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":267,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":267,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4301,"ondisk_log_size":4301,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59528,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312312,"statfs":{"total":96636764160,"available":96575807488,"internally_reserved":0,"allocated":2031616,"data_stored":1312046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.0840000000000001,"15min":1.139},"min":{"1min":0.67600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.395,"5min":1.665,"15min":1.8400000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.167,"15min":1.1699999999999999},"min":{"1min":0.64200000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.5309999999999999,"5min":2.448,"15min":2.448},"last":1.0720000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.119,"15min":1.165},"min":{"1min":0.69099999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.468,"5min":1.915,"15min":1.915},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.84999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.5369999999999999,"5min":1.778,"15min":1.778},"last":1.331}]},{"osd":2,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.242,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.52,"5min":1.736,"15min":2.085},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.748,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6819999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.3700000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2509999999999999,"15min":1.254},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.659,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3859999999999999,"15min":1.4219999999999999},"min":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.581,"5min":1.9570000000000001,"15min":1.9830000000000001},"last":1.264}]},{"osd":4,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.83799999999999997,"5min":0.76700000000000002,"15min":0.61399999999999999},"max":{"1min":2.0579999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.915,"5min":2.355,"15min":2.355},"last":1.006}]},{"osd":5,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.417,"15min":1.395},"min":{"1min":1.004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8069999999999999,"5min":1.865,"15min":2.1059999999999999},"last":1.389},{"interface":"front","average":{"1min":1.55,"5min":1.4530000000000001,"15min":1.4390000000000001},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.2070000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.55,"15min":1.5469999999999999},"min":{"1min":1.028,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":2.1859999999999999,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4870000000000001,"15min":1.4650000000000001},"min":{"1min":0.873,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0449999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.1220000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575051,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1449984,"data_stored":723838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1240000000000001,"15min":1.135},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5309999999999999,"5min":1.869,"15min":1.869},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1890000000000001,"15min":1.2190000000000001},"min":{"1min":0.70299999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.839,"5min":1.895,"15min":1.895},"last":0.83799999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.1419999999999999,"15min":1.181},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.413,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3180000000000001,"15min":1.306},"min":{"1min":0.76100000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.748,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.21,"15min":1.2509999999999999},"min":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4750000000000001,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.234,"15min":1.2529999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5840000000000001,"5min":1.718,"15min":1.7789999999999999},"last":0.746}]},{"osd":3,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.331,"15min":1.355},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8360000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.423,"15min":1.417},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.589,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":0.63}]},{"osd":4,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3700000000000001,"15min":1.349},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.919,"5min":2.1930000000000001,"15min":2.2410000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.411,"15min":1.383},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5009999999999999,"15min":1.5},"min":{"1min":0.73099999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.956,"5min":2.246,"15min":2.6909999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.339,"15min":1.339},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6319999999999999,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5269999999999999,"15min":1.5029999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.98,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.623,"15min":1.6120000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8240000000000001,"5min":2.302,"15min":2.302},"last":0.76400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":721622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.054,"15min":1.1120000000000001},"min":{"1min":0.63300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.51,"5min":1.554,"15min":2.0070000000000001},"last":1.228},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1539999999999999,"15min":1.181},"min":{"1min":0.68100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.9530000000000001},"last":0.86199999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2090000000000001,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7410000000000001,"5min":1.877,"15min":1.944},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.1899999999999999,"15min":1.2170000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.619,"5min":1.6919999999999999,"15min":1.782},"last":1.0820000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2310000000000001,"15min":1.286},"min":{"1min":0.76200000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.663,"5min":1.8,"15min":1.95},"last":1.395},{"interface":"front","average":{"1min":1.121,"5min":1.1819999999999999,"15min":1.23},"min":{"1min":0.68200000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5209999999999999,"5min":1.7270000000000001,"15min":1.843},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3089999999999999,"15min":1.3169999999999999},"min":{"1min":0.91800000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8420000000000001,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.339,"15min":1.379},"min":{"1min":0.82199999999999995,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8959999999999999},"last":1.2}]},{"osd":4,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2050000000000001,"15min":1.2070000000000001},"min":{"1min":0.52600000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6819999999999999,"5min":1.919,"15min":2.0960000000000001},"last":0.747},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.345,"15min":1.343},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":2.0529999999999999,"15min":2.121},"last":1.0469999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4490000000000001,"15min":1.4430000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.1320000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.4790000000000001},"min":{"1min":1.0129999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.1859999999999999},"last":1.026}]},{"osd":7,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5269999999999999,"15min":1.5029999999999999},"min":{"1min":0.73099999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.2000000000000002,"5min":2.218,"15min":2.218},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.548},"min":{"1min":0.76700000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0329999999999999,"5min":2.286,"15min":2.286},"last":1.327}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":728446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2,"15min":1.155},"min":{"1min":0.73099999999999998,"5min":0.71599999999999997,"15min":0.624},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.236,"15min":1.2130000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":5.7309999999999999,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.272,"15min":1.2669999999999999},"min":{"1min":0.73999999999999999,"5min":0.621,"15min":0.621},"max":{"1min":5.8179999999999996,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.627},{"interface":"front","average":{"1min":1.579,"5min":1.292,"15min":1.254},"min":{"1min":0.91600000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.484,"5min":7.484,"15min":7.484},"last":0.95899999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2829999999999999,"15min":1.252},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.3200000000000001,"15min":1.3360000000000001},"min":{"1min":0.85899999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":5.6059999999999999,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.746}]},{"osd":3,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.349,"15min":1.3200000000000001},"min":{"1min":0.97099999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":7.5880000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4359999999999999,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.387}]},{"osd":5,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.3520000000000001,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":5.8929999999999998,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.9990000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.4330000000000001,"15min":1.355},"min":{"1min":0.97799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.6699999999999999,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.28}]},{"osd":6,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.5349999999999999,"15min":1.423},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.528},{"interface":"front","average":{"1min":1.827,"5min":1.635,"15min":1.5780000000000001},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":6.1630000000000003,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":2.1240000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":2.2149999999999999,"5min":1.657,"15min":1.5209999999999999},"min":{"1min":1.1559999999999999,"5min":0.81000000000000005,"15min":0.69599999999999995},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.778,"5min":1.6679999999999999,"15min":1.5600000000000001},"min":{"1min":1.2390000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":3.9510000000000001,"5min":3.968,"15min":3.968},"last":1.794}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40912,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330928,"statfs":{"total":96636764160,"available":96594870272,"internally_reserved":0,"allocated":1441792,"data_stored":728286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.94699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.276,"5min":1.746,"15min":1.746},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.101,"15min":1.171},"min":{"1min":0.53900000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.3680000000000001,"5min":1.8480000000000001,"15min":2.04},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96299999999999997,"15min":0.99099999999999999},"min":{"1min":0.51200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.22,"5min":2.173,"15min":2.173},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.2150000000000001,"15min":1.262},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.498},"max":{"1min":1.752,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.397,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.1739999999999999,"15min":1.2170000000000001},"min":{"1min":0.65200000000000002,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.5640000000000001,"5min":2.847,"15min":2.847},"last":0.61899999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3500000000000001,"15min":1.401},"min":{"1min":0.86399999999999999,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.96,"5min":2.0089999999999999,"15min":2.2349999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.292,"5min":1.375,"15min":1.397},"min":{"1min":0.88600000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.863,"5min":2.919,"15min":2.919},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.405,"15min":1.413},"min":{"1min":0.80700000000000005,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.8420000000000001,"5min":2.02,"15min":2.02},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.353,"5min":1.4550000000000001,"15min":1.49},"min":{"1min":0.86299999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8440000000000001,"5min":2.0390000000000001,"15min":2.423},"last":1.0309999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.92200000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7829999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.016},{"interface":"front","average":{"1min":1.321,"5min":1.5049999999999999,"15min":1.522},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5429999999999999,"15min":1.5880000000000001},"min":{"1min":0.93500000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8919999999999999,"5min":2.302,"15min":2.3759999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.548,"15min":1.595},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.68899999999999995},"max":{"1min":1.915,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46476,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325364,"statfs":{"total":96636764160,"available":96589172736,"internally_reserved":0,"allocated":1441792,"data_stored":728286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81999999999999995,"15min":0.84899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0660000000000001,"5min":1.859,"15min":1.859},"last":1.165},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.97199999999999998},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.357,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.69999999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0349999999999999,"15min":1.1120000000000001},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.476,"5min":1.907,"15min":1.9390000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1259999999999999,"15min":1.2390000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.516,"5min":1.881,"15min":1.881},"last":0.92600000000000005}]},{"osd":3,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.0289999999999999,"15min":1.1160000000000001},"min":{"1min":0.503,"5min":0.438,"15min":0.379},"max":{"1min":1.833,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.131},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2649999999999999,"15min":1.349},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8400000000000001,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.2410000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3160000000000001,"15min":1.3620000000000001},"min":{"1min":0.83599999999999997,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.331,"15min":1.3600000000000001},"min":{"1min":0.83099999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.7869999999999999,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.014}]},{"osd":5,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.286,"15min":1.3340000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":1.637,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.472,"15min":1.512},"min":{"1min":0.89500000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.958,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.5660000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4139999999999999,"15min":1.454},"min":{"1min":0.97599999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.8919999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.373,"15min":1.423},"min":{"1min":0.83399999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.6919999999999999,"5min":2.552,"15min":2.552},"last":1.4410000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.46,"15min":1.5169999999999999},"min":{"1min":0.88200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4339999999999999,"15min":1.496},"min":{"1min":0.92100000000000004,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.798,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1445888,"data_stored":728356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.90200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1220000000000001,"5min":1.3400000000000001,"15min":1.905},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.0660000000000001},"min":{"1min":0.59199999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.6719999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.93899999999999995,"15min":0.98499999999999999},"min":{"1min":0.503,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":2.222},{"interface":"front","average":{"1min":1.022,"5min":1.0960000000000001,"15min":1.1719999999999999},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.0510000000000002},"last":1.4550000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1839999999999999,"15min":1.2869999999999999},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6890000000000001,"5min":1.7350000000000001,"15min":1.9510000000000001},"last":1.911},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.67200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5900000000000001,"5min":1.877,"15min":1.9159999999999999},"last":1.516}]},{"osd":4,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2370000000000001,"15min":1.27},"min":{"1min":0.80700000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":1.403},{"interface":"front","average":{"1min":1.302,"5min":1.3660000000000001,"15min":1.4079999999999999},"min":{"1min":0.83599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7749999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.7769999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.429,"15min":1.49},"min":{"1min":0.90800000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4159999999999999,"15min":1.4630000000000001},"min":{"1min":0.875,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.25}]},{"osd":6,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.444,"15min":1.488},"min":{"1min":1.032,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.903,"5min":2.2469999999999999,"15min":2.4079999999999999},"last":1.821},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.381,"15min":1.4530000000000001},"min":{"1min":0.90500000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.925,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":2.0169999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.526,"15min":1.5920000000000001},"min":{"1min":0.93799999999999994,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8540000000000001,"5min":2.2679999999999998,"15min":2.3929999999999998},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.516,"15min":1.593},"min":{"1min":1.1080000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.885,"5min":2.1619999999999999,"15min":2.367},"last":1.958}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2048000,"data_stored":1314308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85299999999999998,"15min":0.90700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4259999999999999,"5min":1.649,"15min":1.855},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96799999999999997,"15min":1.052},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5329999999999999,"5min":1.5940000000000001,"15min":1.7509999999999999},"last":0.89900000000000002}]},{"osd":2,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.03,"15min":1.113},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5740000000000001,"5min":1.708,"15min":1.7749999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.081,"15min":1.135},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3420000000000001,"5min":1.694,"15min":1.732},"last":1.125}]},{"osd":3,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0309999999999999,"15min":1.0840000000000001},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.421,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1739999999999999,"15min":1.2749999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7050000000000001,"5min":1.8109999999999999,"15min":2.1619999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.355,"15min":1.403},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.863,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4330000000000001,"15min":1.5149999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.956,"5min":2.246,"15min":2.2890000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.329,"15min":1.3700000000000001},"min":{"1min":0.64000000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5620000000000001,"5min":1.9419999999999999,"15min":2.137},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.393,"15min":1.4790000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.0699999999999998,"5min":2.137,"15min":2.4670000000000001},"last":1.079}]},{"osd":6,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.4079999999999999,"15min":1.468},"min":{"1min":0.79600000000000004,"5min":0.753,"15min":0.753},"max":{"1min":1.8660000000000001,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.87},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.573},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.121,"5min":2.355,"15min":2.403},"last":1.0349999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0289999999999999,"5min":2.2959999999999998,"15min":2.3279999999999998},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4710000000000001,"15min":1.556},"min":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.978,"5min":2.077,"15min":2.3540000000000001},"last":1.1519999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:18:49.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:18:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.325+0000 7fdf3898c640 1 Processor -- start 2023-10-21T16:18:49.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.325+0000 7fdf3898c640 1 -- start start 2023-10-21T16:18:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf3415ab30 0x7fdf3415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 0x7fdf340b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 0x7fdf340b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3415f7d0 con 0x7fdf3415ab30 2023-10-21T16:18:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3415f210 con 0x7fdf3415af10 2023-10-21T16:18:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf3898c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3415f4f0 con 0x7fdf340adc70 2023-10-21T16:18:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf337fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf3415ab30 0x7fdf3415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf337fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdf3415ab30 0x7fdf3415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47464/0 (socket says 172.21.15.53:47464) 2023-10-21T16:18:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.327+0000 7fdf337fe640 1 -- 172.21.15.53:0/3483288275 learned_addr learned my addr 172.21.15.53:0/3483288275 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf33fff640 1 --2- 172.21.15.53:0/3483288275 >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 0x7fdf340b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf337fe640 1 -- 172.21.15.53:0/3483288275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 msgr2=0x7fdf340b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf2bfff640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 0x7fdf340b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf337fe640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 0x7fdf340b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf337fe640 1 -- 172.21.15.53:0/3483288275 >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 msgr2=0x7fdf340b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf337fe640 1 --2- 172.21.15.53:0/3483288275 >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 0x7fdf340b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf337fe640 1 -- 172.21.15.53:0/3483288275 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3406c550 con 0x7fdf3415ab30 2023-10-21T16:18:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf33fff640 1 --2- 172.21.15.53:0/3483288275 >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 0x7fdf340b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.328+0000 7fdf2bfff640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 0x7fdf340b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:49.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.329+0000 7fdf337fe640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf3415ab30 0x7fdf3415e8a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf340cb000 tx=0x7fdf2402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e87e8b0ff5d8fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.329+0000 7fdf32ffd640 1 -- 172.21.15.53:0/3483288275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031880 con 0x7fdf3415ab30 2023-10-21T16:18:49.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.329+0000 7fdf32ffd640 1 -- 172.21.15.53:0/3483288275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240319e0 con 0x7fdf3415ab30 2023-10-21T16:18:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.329+0000 7fdf32ffd640 1 -- 172.21.15.53:0/3483288275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031d70 con 0x7fdf3415ab30 2023-10-21T16:18:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 -- 172.21.15.53:0/3483288275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf3415ab30 msgr2=0x7fdf3415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf3415ab30 0x7fdf3415e8a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf340cb000 tx=0x7fdf2402b920 comp rx=0 tx=0).stop 2023-10-21T16:18:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 -- 172.21.15.53:0/3483288275 shutdown_connections 2023-10-21T16:18:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf340adc70 0x7fdf340b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 --2- 172.21.15.53:0/3483288275 >> v2:172.21.15.124:3300/0 conn(0x7fdf3415af10 0x7fdf340b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 --2- 172.21.15.53:0/3483288275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf3415ab30 0x7fdf3415e8a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 -- 172.21.15.53:0/3483288275 >> 172.21.15.53:0/3483288275 conn(0x7fdf340cb0c0 msgr2=0x7fdf340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 -- 172.21.15.53:0/3483288275 shutdown_connections 2023-10-21T16:18:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.330+0000 7fdf3898c640 1 -- 172.21.15.53:0/3483288275 wait complete. 2023-10-21T16:18:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.332+0000 7fdf3898c640 1 Processor -- start 2023-10-21T16:18:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.332+0000 7fdf3898c640 1 -- start start 2023-10-21T16:18:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3415ab30 0x7fdf340b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3415af10 0x7fdf3415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3415b3a0 con 0x7fdf340adc70 2023-10-21T16:18:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3415c170 con 0x7fdf3415af10 2023-10-21T16:18:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf3898c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3414be30 con 0x7fdf3415ab30 2023-10-21T16:18:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47466/0 (socket says 172.21.15.53:47466) 2023-10-21T16:18:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.333+0000 7fdf337fe640 1 -- 172.21.15.53:0/2185919225 learned_addr learned my addr 172.21.15.53:0/2185919225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf2bfff640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3415ab30 0x7fdf340b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 -- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3415ab30 msgr2=0x7fdf340b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3415ab30 0x7fdf340b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 -- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3415af10 msgr2=0x7fdf3415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3415af10 0x7fdf3415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 -- 172.21.15.53:0/2185919225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf240094d0 con 0x7fdf340adc70 2023-10-21T16:18:49.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.334+0000 7fdf337fe640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fdf2402b960 tx=0x7fdf240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.335+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031870 con 0x7fdf340adc70 2023-10-21T16:18:49.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.335+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240443f0 con 0x7fdf340adc70 2023-10-21T16:18:49.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.335+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3415b5f0 con 0x7fdf340adc70 2023-10-21T16:18:49.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.335+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240447a0 con 0x7fdf340adc70 2023-10-21T16:18:49.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.335+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3415bbb0 con 0x7fdf340adc70 2023-10-21T16:18:49.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.336+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3404e160 con 0x7fdf340adc70 2023-10-21T16:18:49.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.337+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdf2404d450 con 0x7fdf340adc70 2023-10-21T16:18:49.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.338+0000 7fdf30ff9640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 0x7fdf1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.338+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf240ac220 con 0x7fdf340adc70 2023-10-21T16:18:49.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.338+0000 7fdf2bfff640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 0x7fdf1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.340+0000 7fdf2bfff640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 0x7fdf1406d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf340c8090 tx=0x7fdf1c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.342+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf24076f60 con 0x7fdf340adc70 2023-10-21T16:18:49.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.540+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 --> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf3406c830 con 0x7fdf1406af00 2023-10-21T16:18:49.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.562+0000 7fdf30ff9640 1 -- 172.21.15.53:0/2185919225 <== mgr.35759 v2:172.21.15.124:6832/1967358544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287376 (secure 0 0 0) 0x7fdf1c00c050 con 0x7fdf1406af00 2023-10-21T16:18:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:49.570 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:18:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 msgr2=0x7fdf1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 0x7fdf1406d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf340c8090 tx=0x7fdf1c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 msgr2=0x7fdf3414b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fdf2402b960 tx=0x7fdf240046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 shutdown_connections 2023-10-21T16:18:49.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fdf1406af00 0x7fdf1406d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3415af10 0x7fdf3415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3415ab30 0x7fdf340b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 --2- 172.21.15.53:0/2185919225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf340adc70 0x7fdf3414b8f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 >> 172.21.15.53:0/2185919225 conn(0x7fdf340cb0c0 msgr2=0x7fdf340b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 shutdown_connections 2023-10-21T16:18:49.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.569+0000 7fdf3898c640 1 -- 172.21.15.53:0/2185919225 wait complete. 2023-10-21T16:18:49.592 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:18:47.713533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":311,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4339,"ondisk_log_size":4339,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322952,"kb_used_data":12440,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754651768,"statfs":{"total":773094113280,"available":772763410432,"internally_reserved":0,"allocated":12738560,"data_stored":6985188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":317836857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.275960"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480566+0000","last_change":"2023-10-21T16:18:44.506365+0000","last_active":"2023-10-21T16:18:45.480566+0000","last_peered":"2023-10-21T16:18:45.480566+0000","last_clean":"2023-10-21T16:18:45.480566+0000","last_became_active":"2023-10-21T16:18:44.506033+0000","last_became_peered":"2023-10-21T16:18:44.506033+0000","last_unstale":"2023-10-21T16:18:45.480566+0000","last_undegraded":"2023-10-21T16:18:45.480566+0000","last_fullsized":"2023-10-21T16:18:45.480566+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:01:59.267254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594075+0000","last_change":"2023-10-21T16:18:44.500375+0000","last_active":"2023-10-21T16:18:46.594075+0000","last_peered":"2023-10-21T16:18:46.594075+0000","last_clean":"2023-10-21T16:18:46.594075+0000","last_became_active":"2023-10-21T16:18:44.500040+0000","last_became_peered":"2023-10-21T16:18:44.500040+0000","last_unstale":"2023-10-21T16:18:46.594075+0000","last_undegraded":"2023-10-21T16:18:46.594075+0000","last_fullsized":"2023-10-21T16:18:46.594075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:40:20.933122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480112+0000","last_change":"2023-10-21T16:18:44.497948+0000","last_active":"2023-10-21T16:18:45.480112+0000","last_peered":"2023-10-21T16:18:45.480112+0000","last_clean":"2023-10-21T16:18:45.480112+0000","last_became_active":"2023-10-21T16:18:44.497667+0000","last_became_peered":"2023-10-21T16:18:44.497667+0000","last_unstale":"2023-10-21T16:18:45.480112+0000","last_undegraded":"2023-10-21T16:18:45.480112+0000","last_fullsized":"2023-10-21T16:18:45.480112+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:25:46.610475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152340+0000","last_change":"2023-10-21T16:18:44.517493+0000","last_active":"2023-10-21T16:18:46.152340+0000","last_peered":"2023-10-21T16:18:46.152340+0000","last_clean":"2023-10-21T16:18:46.152340+0000","last_became_active":"2023-10-21T16:18:44.516127+0000","last_became_peered":"2023-10-21T16:18:44.516127+0000","last_unstale":"2023-10-21T16:18:46.152340+0000","last_undegraded":"2023-10-21T16:18:46.152340+0000","last_fullsized":"2023-10-21T16:18:46.152340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:59.110459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484180+0000","last_change":"2023-10-21T16:18:44.501570+0000","last_active":"2023-10-21T16:18:45.484180+0000","last_peered":"2023-10-21T16:18:45.484180+0000","last_clean":"2023-10-21T16:18:45.484180+0000","last_became_active":"2023-10-21T16:18:44.501166+0000","last_became_peered":"2023-10-21T16:18:44.501166+0000","last_unstale":"2023-10-21T16:18:45.484180+0000","last_undegraded":"2023-10-21T16:18:45.484180+0000","last_fullsized":"2023-10-21T16:18:45.484180+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:30.636045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587778+0000","last_change":"2023-10-21T16:18:44.520659+0000","last_active":"2023-10-21T16:18:46.587778+0000","last_peered":"2023-10-21T16:18:46.587778+0000","last_clean":"2023-10-21T16:18:46.587778+0000","last_became_active":"2023-10-21T16:18:44.520306+0000","last_became_peered":"2023-10-21T16:18:44.520306+0000","last_unstale":"2023-10-21T16:18:46.587778+0000","last_undegraded":"2023-10-21T16:18:46.587778+0000","last_fullsized":"2023-10-21T16:18:46.587778+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:10.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589549+0000","last_change":"2023-10-21T16:18:44.509299+0000","last_active":"2023-10-21T16:18:46.589549+0000","last_peered":"2023-10-21T16:18:46.589549+0000","last_clean":"2023-10-21T16:18:46.589549+0000","last_became_active":"2023-10-21T16:18:44.507862+0000","last_became_peered":"2023-10-21T16:18:44.507862+0000","last_unstale":"2023-10-21T16:18:46.589549+0000","last_undegraded":"2023-10-21T16:18:46.589549+0000","last_fullsized":"2023-10-21T16:18:46.589549+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:10.430211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151645+0000","last_change":"2023-10-21T16:18:44.524732+0000","last_active":"2023-10-21T16:18:46.151645+0000","last_peered":"2023-10-21T16:18:46.151645+0000","last_clean":"2023-10-21T16:18:46.151645+0000","last_became_active":"2023-10-21T16:18:44.522900+0000","last_became_peered":"2023-10-21T16:18:44.522900+0000","last_unstale":"2023-10-21T16:18:46.151645+0000","last_undegraded":"2023-10-21T16:18:46.151645+0000","last_fullsized":"2023-10-21T16:18:46.151645+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:19.145776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485199+0000","last_change":"2023-10-21T16:18:44.498543+0000","last_active":"2023-10-21T16:18:45.485199+0000","last_peered":"2023-10-21T16:18:45.485199+0000","last_clean":"2023-10-21T16:18:45.485199+0000","last_became_active":"2023-10-21T16:18:44.498265+0000","last_became_peered":"2023-10-21T16:18:44.498265+0000","last_unstale":"2023-10-21T16:18:45.485199+0000","last_undegraded":"2023-10-21T16:18:45.485199+0000","last_fullsized":"2023-10-21T16:18:45.485199+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:37:55.299321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481480+0000","last_change":"2023-10-21T16:18:44.512214+0000","last_active":"2023-10-21T16:18:45.481480+0000","last_peered":"2023-10-21T16:18:45.481480+0000","last_clean":"2023-10-21T16:18:45.481480+0000","last_became_active":"2023-10-21T16:18:44.511936+0000","last_became_peered":"2023-10-21T16:18:44.511936+0000","last_unstale":"2023-10-21T16:18:45.481480+0000","last_undegraded":"2023-10-21T16:18:45.481480+0000","last_fullsized":"2023-10-21T16:18:45.481480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:47.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484917+0000","last_change":"2023-10-21T16:18:44.509839+0000","last_active":"2023-10-21T16:18:45.484917+0000","last_peered":"2023-10-21T16:18:45.484917+0000","last_clean":"2023-10-21T16:18:45.484917+0000","last_became_active":"2023-10-21T16:18:44.509582+0000","last_became_peered":"2023-10-21T16:18:44.509582+0000","last_unstale":"2023-10-21T16:18:45.484917+0000","last_undegraded":"2023-10-21T16:18:45.484917+0000","last_fullsized":"2023-10-21T16:18:45.484917+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:01.393863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481328+0000","last_change":"2023-10-21T16:18:44.492993+0000","last_active":"2023-10-21T16:18:45.481328+0000","last_peered":"2023-10-21T16:18:45.481328+0000","last_clean":"2023-10-21T16:18:45.481328+0000","last_became_active":"2023-10-21T16:18:44.492627+0000","last_became_peered":"2023-10-21T16:18:44.492627+0000","last_unstale":"2023-10-21T16:18:45.481328+0000","last_undegraded":"2023-10-21T16:18:45.481328+0000","last_fullsized":"2023-10-21T16:18:45.481328+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:41:39.204289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151844+0000","last_change":"2023-10-21T16:18:44.519408+0000","last_active":"2023-10-21T16:18:46.151844+0000","last_peered":"2023-10-21T16:18:46.151844+0000","last_clean":"2023-10-21T16:18:46.151844+0000","last_became_active":"2023-10-21T16:18:44.519123+0000","last_became_peered":"2023-10-21T16:18:44.519123+0000","last_unstale":"2023-10-21T16:18:46.151844+0000","last_undegraded":"2023-10-21T16:18:46.151844+0000","last_fullsized":"2023-10-21T16:18:46.151844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:41.473531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483748+0000","last_change":"2023-10-21T16:18:44.511539+0000","last_active":"2023-10-21T16:18:45.483748+0000","last_peered":"2023-10-21T16:18:45.483748+0000","last_clean":"2023-10-21T16:18:45.483748+0000","last_became_active":"2023-10-21T16:18:44.511108+0000","last_became_peered":"2023-10-21T16:18:44.511108+0000","last_unstale":"2023-10-21T16:18:45.483748+0000","last_undegraded":"2023-10-21T16:18:45.483748+0000","last_fullsized":"2023-10-21T16:18:45.483748+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:41.473531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151861+0000","last_change":"2023-10-21T16:18:44.521659+0000","last_active":"2023-10-21T16:18:46.151861+0000","last_peered":"2023-10-21T16:18:46.151861+0000","last_clean":"2023-10-21T16:18:46.151861+0000","last_became_active":"2023-10-21T16:18:44.521166+0000","last_became_peered":"2023-10-21T16:18:44.521166+0000","last_unstale":"2023-10-21T16:18:46.151861+0000","last_undegraded":"2023-10-21T16:18:46.151861+0000","last_fullsized":"2023-10-21T16:18:46.151861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:06.433654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587680+0000","last_change":"2023-10-21T16:18:44.507106+0000","last_active":"2023-10-21T16:18:46.587680+0000","last_peered":"2023-10-21T16:18:46.587680+0000","last_clean":"2023-10-21T16:18:46.587680+0000","last_became_active":"2023-10-21T16:18:44.506800+0000","last_became_peered":"2023-10-21T16:18:44.506800+0000","last_unstale":"2023-10-21T16:18:46.587680+0000","last_undegraded":"2023-10-21T16:18:46.587680+0000","last_fullsized":"2023-10-21T16:18:46.587680+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:55.195177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588372+0000","last_change":"2023-10-21T16:18:44.505476+0000","last_active":"2023-10-21T16:18:46.588372+0000","last_peered":"2023-10-21T16:18:46.588372+0000","last_clean":"2023-10-21T16:18:46.588372+0000","last_became_active":"2023-10-21T16:18:44.505216+0000","last_became_peered":"2023-10-21T16:18:44.505216+0000","last_unstale":"2023-10-21T16:18:46.588372+0000","last_undegraded":"2023-10-21T16:18:46.588372+0000","last_fullsized":"2023-10-21T16:18:46.588372+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:43.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480335+0000","last_change":"2023-10-21T16:18:44.495870+0000","last_active":"2023-10-21T16:18:45.480335+0000","last_peered":"2023-10-21T16:18:45.480335+0000","last_clean":"2023-10-21T16:18:45.480335+0000","last_became_active":"2023-10-21T16:18:44.495576+0000","last_became_peered":"2023-10-21T16:18:44.495576+0000","last_unstale":"2023-10-21T16:18:45.480335+0000","last_undegraded":"2023-10-21T16:18:45.480335+0000","last_fullsized":"2023-10-21T16:18:45.480335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:34.797978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485149+0000","last_change":"2023-10-21T16:18:44.517339+0000","last_active":"2023-10-21T16:18:45.485149+0000","last_peered":"2023-10-21T16:18:45.485149+0000","last_clean":"2023-10-21T16:18:45.485149+0000","last_became_active":"2023-10-21T16:18:44.516966+0000","last_became_peered":"2023-10-21T16:18:44.516966+0000","last_unstale":"2023-10-21T16:18:45.485149+0000","last_undegraded":"2023-10-21T16:18:45.485149+0000","last_fullsized":"2023-10-21T16:18:45.485149+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:51:21.507107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152562+0000","last_change":"2023-10-21T16:18:44.521472+0000","last_active":"2023-10-21T16:18:46.152562+0000","last_peered":"2023-10-21T16:18:46.152562+0000","last_clean":"2023-10-21T16:18:46.152562+0000","last_became_active":"2023-10-21T16:18:44.521199+0000","last_became_peered":"2023-10-21T16:18:44.521199+0000","last_unstale":"2023-10-21T16:18:46.152562+0000","last_undegraded":"2023-10-21T16:18:46.152562+0000","last_fullsized":"2023-10-21T16:18:46.152562+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:09.773702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483880+0000","last_change":"2023-10-21T16:18:44.507052+0000","last_active":"2023-10-21T16:18:45.483880+0000","last_peered":"2023-10-21T16:18:45.483880+0000","last_clean":"2023-10-21T16:18:45.483880+0000","last_became_active":"2023-10-21T16:18:44.506744+0000","last_became_peered":"2023-10-21T16:18:44.506744+0000","last_unstale":"2023-10-21T16:18:45.483880+0000","last_undegraded":"2023-10-21T16:18:45.483880+0000","last_fullsized":"2023-10-21T16:18:45.483880+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:33.224873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483844+0000","last_change":"2023-10-21T16:18:44.511503+0000","last_active":"2023-10-21T16:18:45.483844+0000","last_peered":"2023-10-21T16:18:45.483844+0000","last_clean":"2023-10-21T16:18:45.483844+0000","last_became_active":"2023-10-21T16:18:44.511064+0000","last_became_peered":"2023-10-21T16:18:44.511064+0000","last_unstale":"2023-10-21T16:18:45.483844+0000","last_undegraded":"2023-10-21T16:18:45.483844+0000","last_fullsized":"2023-10-21T16:18:45.483844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:08:06.433654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590464+0000","last_change":"2023-10-21T16:18:44.521094+0000","last_active":"2023-10-21T16:18:46.590464+0000","last_peered":"2023-10-21T16:18:46.590464+0000","last_clean":"2023-10-21T16:18:46.590464+0000","last_became_active":"2023-10-21T16:18:44.520821+0000","last_became_peered":"2023-10-21T16:18:44.520821+0000","last_unstale":"2023-10-21T16:18:46.590464+0000","last_undegraded":"2023-10-21T16:18:46.590464+0000","last_fullsized":"2023-10-21T16:18:46.590464+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:47.480535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479480+0000","last_change":"2023-10-21T16:18:44.499756+0000","last_active":"2023-10-21T16:18:45.479480+0000","last_peered":"2023-10-21T16:18:45.479480+0000","last_clean":"2023-10-21T16:18:45.479480+0000","last_became_active":"2023-10-21T16:18:44.499379+0000","last_became_peered":"2023-10-21T16:18:44.499379+0000","last_unstale":"2023-10-21T16:18:45.479480+0000","last_undegraded":"2023-10-21T16:18:45.479480+0000","last_fullsized":"2023-10-21T16:18:45.479480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:44:14.452424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594414+0000","last_change":"2023-10-21T16:18:44.496921+0000","last_active":"2023-10-21T16:18:46.594414+0000","last_peered":"2023-10-21T16:18:46.594414+0000","last_clean":"2023-10-21T16:18:46.594414+0000","last_became_active":"2023-10-21T16:18:44.496630+0000","last_became_peered":"2023-10-21T16:18:44.496630+0000","last_unstale":"2023-10-21T16:18:46.594414+0000","last_undegraded":"2023-10-21T16:18:46.594414+0000","last_fullsized":"2023-10-21T16:18:46.594414+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:29.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484676+0000","last_change":"2023-10-21T16:18:44.517337+0000","last_active":"2023-10-21T16:18:45.484676+0000","last_peered":"2023-10-21T16:18:45.484676+0000","last_clean":"2023-10-21T16:18:45.484676+0000","last_became_active":"2023-10-21T16:18:44.516975+0000","last_became_peered":"2023-10-21T16:18:44.516975+0000","last_unstale":"2023-10-21T16:18:45.484676+0000","last_undegraded":"2023-10-21T16:18:45.484676+0000","last_fullsized":"2023-10-21T16:18:45.484676+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:36.541903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481444+0000","last_change":"2023-10-21T16:18:44.498915+0000","last_active":"2023-10-21T16:18:45.481444+0000","last_peered":"2023-10-21T16:18:45.481444+0000","last_clean":"2023-10-21T16:18:45.481444+0000","last_became_active":"2023-10-21T16:18:44.498634+0000","last_became_peered":"2023-10-21T16:18:44.498634+0000","last_unstale":"2023-10-21T16:18:45.481444+0000","last_undegraded":"2023-10-21T16:18:45.481444+0000","last_fullsized":"2023-10-21T16:18:45.481444+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:38.301212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152699+0000","last_change":"2023-10-21T16:18:44.518091+0000","last_active":"2023-10-21T16:18:46.152699+0000","last_peered":"2023-10-21T16:18:46.152699+0000","last_clean":"2023-10-21T16:18:46.152699+0000","last_became_active":"2023-10-21T16:18:44.517782+0000","last_became_peered":"2023-10-21T16:18:44.517782+0000","last_unstale":"2023-10-21T16:18:46.152699+0000","last_undegraded":"2023-10-21T16:18:46.152699+0000","last_fullsized":"2023-10-21T16:18:46.152699+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:36.541903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590021+0000","last_change":"2023-10-21T16:18:44.513515+0000","last_active":"2023-10-21T16:18:46.590021+0000","last_peered":"2023-10-21T16:18:46.590021+0000","last_clean":"2023-10-21T16:18:46.590021+0000","last_became_active":"2023-10-21T16:18:44.513175+0000","last_became_peered":"2023-10-21T16:18:44.513175+0000","last_unstale":"2023-10-21T16:18:46.590021+0000","last_undegraded":"2023-10-21T16:18:46.590021+0000","last_fullsized":"2023-10-21T16:18:46.590021+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:55.195177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484218+0000","last_change":"2023-10-21T16:18:44.497774+0000","last_active":"2023-10-21T16:18:45.484218+0000","last_peered":"2023-10-21T16:18:45.484218+0000","last_clean":"2023-10-21T16:18:45.484218+0000","last_became_active":"2023-10-21T16:18:44.497510+0000","last_became_peered":"2023-10-21T16:18:44.497510+0000","last_unstale":"2023-10-21T16:18:45.484218+0000","last_undegraded":"2023-10-21T16:18:45.484218+0000","last_fullsized":"2023-10-21T16:18:45.484218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:57.790370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593283+0000","last_change":"2023-10-21T16:18:44.514907+0000","last_active":"2023-10-21T16:18:46.593283+0000","last_peered":"2023-10-21T16:18:46.593283+0000","last_clean":"2023-10-21T16:18:46.593283+0000","last_became_active":"2023-10-21T16:18:44.514612+0000","last_became_peered":"2023-10-21T16:18:44.514612+0000","last_unstale":"2023-10-21T16:18:46.593283+0000","last_undegraded":"2023-10-21T16:18:46.593283+0000","last_fullsized":"2023-10-21T16:18:46.593283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:20.497259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587745+0000","last_change":"2023-10-21T16:18:44.506531+0000","last_active":"2023-10-21T16:18:46.587745+0000","last_peered":"2023-10-21T16:18:46.587745+0000","last_clean":"2023-10-21T16:18:46.587745+0000","last_became_active":"2023-10-21T16:18:44.506266+0000","last_became_peered":"2023-10-21T16:18:44.506266+0000","last_unstale":"2023-10-21T16:18:46.587745+0000","last_undegraded":"2023-10-21T16:18:46.587745+0000","last_fullsized":"2023-10-21T16:18:46.587745+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:52.606492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588060+0000","last_change":"2023-10-21T16:18:44.509302+0000","last_active":"2023-10-21T16:18:46.588060+0000","last_peered":"2023-10-21T16:18:46.588060+0000","last_clean":"2023-10-21T16:18:46.588060+0000","last_became_active":"2023-10-21T16:18:44.508955+0000","last_became_peered":"2023-10-21T16:18:44.508955+0000","last_unstale":"2023-10-21T16:18:46.588060+0000","last_undegraded":"2023-10-21T16:18:46.588060+0000","last_fullsized":"2023-10-21T16:18:46.588060+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:28.992489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"116'4","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.592429+0000","last_change":"2023-10-21T16:18:43.472736+0000","last_active":"2023-10-21T16:18:46.592429+0000","last_peered":"2023-10-21T16:18:46.592429+0000","last_clean":"2023-10-21T16:18:46.592429+0000","last_became_active":"2023-10-21T16:18:43.471068+0000","last_became_peered":"2023-10-21T16:18:43.471068+0000","last_unstale":"2023-10-21T16:18:46.592429+0000","last_undegraded":"2023-10-21T16:18:46.592429+0000","last_fullsized":"2023-10-21T16:18:46.592429+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:58:52.277409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588256+0000","last_change":"2023-10-21T16:18:44.503366+0000","last_active":"2023-10-21T16:18:46.588256+0000","last_peered":"2023-10-21T16:18:46.588256+0000","last_clean":"2023-10-21T16:18:46.588256+0000","last_became_active":"2023-10-21T16:18:44.503074+0000","last_became_peered":"2023-10-21T16:18:44.503074+0000","last_unstale":"2023-10-21T16:18:46.588256+0000","last_undegraded":"2023-10-21T16:18:46.588256+0000","last_fullsized":"2023-10-21T16:18:46.588256+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:58.142635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484726+0000","last_change":"2023-10-21T16:18:43.476465+0000","last_active":"2023-10-21T16:18:45.484726+0000","last_peered":"2023-10-21T16:18:45.484726+0000","last_clean":"2023-10-21T16:18:45.484726+0000","last_became_active":"2023-10-21T16:18:43.476097+0000","last_became_peered":"2023-10-21T16:18:43.476097+0000","last_unstale":"2023-10-21T16:18:45.484726+0000","last_undegraded":"2023-10-21T16:18:45.484726+0000","last_fullsized":"2023-10-21T16:18:45.484726+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:46.162372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589716+0000","last_change":"2023-10-21T16:18:44.514052+0000","last_active":"2023-10-21T16:18:46.589716+0000","last_peered":"2023-10-21T16:18:46.589716+0000","last_clean":"2023-10-21T16:18:46.589716+0000","last_became_active":"2023-10-21T16:18:44.513686+0000","last_became_peered":"2023-10-21T16:18:44.513686+0000","last_unstale":"2023-10-21T16:18:46.589716+0000","last_undegraded":"2023-10-21T16:18:46.589716+0000","last_fullsized":"2023-10-21T16:18:46.589716+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:19.832284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590062+0000","last_change":"2023-10-21T16:18:43.477554+0000","last_active":"2023-10-21T16:18:46.590062+0000","last_peered":"2023-10-21T16:18:46.590062+0000","last_clean":"2023-10-21T16:18:46.590062+0000","last_became_active":"2023-10-21T16:18:43.477293+0000","last_became_peered":"2023-10-21T16:18:43.477293+0000","last_unstale":"2023-10-21T16:18:46.590062+0000","last_undegraded":"2023-10-21T16:18:46.590062+0000","last_fullsized":"2023-10-21T16:18:46.590062+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:31:40.394917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588450+0000","last_change":"2023-10-21T16:18:44.502663+0000","last_active":"2023-10-21T16:18:46.588450+0000","last_peered":"2023-10-21T16:18:46.588450+0000","last_clean":"2023-10-21T16:18:46.588450+0000","last_became_active":"2023-10-21T16:18:44.502306+0000","last_became_peered":"2023-10-21T16:18:44.502306+0000","last_unstale":"2023-10-21T16:18:46.588450+0000","last_undegraded":"2023-10-21T16:18:46.588450+0000","last_fullsized":"2023-10-21T16:18:46.588450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:40:20.933122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591928+0000","last_change":"2023-10-21T16:18:43.473558+0000","last_active":"2023-10-21T16:18:46.591928+0000","last_peered":"2023-10-21T16:18:46.591928+0000","last_clean":"2023-10-21T16:18:46.591928+0000","last_became_active":"2023-10-21T16:18:43.471659+0000","last_became_peered":"2023-10-21T16:18:43.471659+0000","last_unstale":"2023-10-21T16:18:46.591928+0000","last_undegraded":"2023-10-21T16:18:46.591928+0000","last_fullsized":"2023-10-21T16:18:46.591928+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:23:29.439305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.151750+0000","last_change":"2023-10-21T16:18:44.525317+0000","last_active":"2023-10-21T16:18:46.151750+0000","last_peered":"2023-10-21T16:18:46.151750+0000","last_clean":"2023-10-21T16:18:46.151750+0000","last_became_active":"2023-10-21T16:18:44.524147+0000","last_became_peered":"2023-10-21T16:18:44.524147+0000","last_unstale":"2023-10-21T16:18:46.151750+0000","last_undegraded":"2023-10-21T16:18:46.151750+0000","last_fullsized":"2023-10-21T16:18:46.151750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:45.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484157+0000","last_change":"2023-10-21T16:18:43.472924+0000","last_active":"2023-10-21T16:18:45.484157+0000","last_peered":"2023-10-21T16:18:45.484157+0000","last_clean":"2023-10-21T16:18:45.484157+0000","last_became_active":"2023-10-21T16:18:43.468620+0000","last_became_peered":"2023-10-21T16:18:43.468620+0000","last_unstale":"2023-10-21T16:18:45.484157+0000","last_undegraded":"2023-10-21T16:18:45.484157+0000","last_fullsized":"2023-10-21T16:18:45.484157+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:28.965067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590794+0000","last_change":"2023-10-21T16:18:44.512280+0000","last_active":"2023-10-21T16:18:46.590794+0000","last_peered":"2023-10-21T16:18:46.590794+0000","last_clean":"2023-10-21T16:18:46.590794+0000","last_became_active":"2023-10-21T16:18:44.511655+0000","last_became_peered":"2023-10-21T16:18:44.511655+0000","last_unstale":"2023-10-21T16:18:46.590794+0000","last_undegraded":"2023-10-21T16:18:46.590794+0000","last_fullsized":"2023-10-21T16:18:46.590794+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:53:40.915732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152629+0000","last_change":"2023-10-21T16:18:43.469665+0000","last_active":"2023-10-21T16:18:46.152629+0000","last_peered":"2023-10-21T16:18:46.152629+0000","last_clean":"2023-10-21T16:18:46.152629+0000","last_became_active":"2023-10-21T16:18:43.469386+0000","last_became_peered":"2023-10-21T16:18:43.469386+0000","last_unstale":"2023-10-21T16:18:46.152629+0000","last_undegraded":"2023-10-21T16:18:46.152629+0000","last_fullsized":"2023-10-21T16:18:46.152629+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:28.965067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587800+0000","last_change":"2023-10-21T16:18:44.520414+0000","last_active":"2023-10-21T16:18:46.587800+0000","last_peered":"2023-10-21T16:18:46.587800+0000","last_clean":"2023-10-21T16:18:46.587800+0000","last_became_active":"2023-10-21T16:18:44.520133+0000","last_became_peered":"2023-10-21T16:18:44.520133+0000","last_unstale":"2023-10-21T16:18:46.587800+0000","last_undegraded":"2023-10-21T16:18:46.587800+0000","last_fullsized":"2023-10-21T16:18:46.587800+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:57:37.344675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479779+0000","last_change":"2023-10-21T16:18:43.480874+0000","last_active":"2023-10-21T16:18:45.479779+0000","last_peered":"2023-10-21T16:18:45.479779+0000","last_clean":"2023-10-21T16:18:45.479779+0000","last_became_active":"2023-10-21T16:18:43.480577+0000","last_became_peered":"2023-10-21T16:18:43.480577+0000","last_unstale":"2023-10-21T16:18:45.479779+0000","last_undegraded":"2023-10-21T16:18:45.479779+0000","last_fullsized":"2023-10-21T16:18:45.479779+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:38:03.901484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152261+0000","last_change":"2023-10-21T16:18:44.516513+0000","last_active":"2023-10-21T16:18:46.152261+0000","last_peered":"2023-10-21T16:18:46.152261+0000","last_clean":"2023-10-21T16:18:46.152261+0000","last_became_active":"2023-10-21T16:18:44.514408+0000","last_became_peered":"2023-10-21T16:18:44.514408+0000","last_unstale":"2023-10-21T16:18:46.152261+0000","last_undegraded":"2023-10-21T16:18:46.152261+0000","last_fullsized":"2023-10-21T16:18:46.152261+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:30.636045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589149+0000","last_change":"2023-10-21T16:18:43.480360+0000","last_active":"2023-10-21T16:18:46.589149+0000","last_peered":"2023-10-21T16:18:46.589149+0000","last_clean":"2023-10-21T16:18:46.589149+0000","last_became_active":"2023-10-21T16:18:43.480083+0000","last_became_peered":"2023-10-21T16:18:43.480083+0000","last_unstale":"2023-10-21T16:18:46.589149+0000","last_undegraded":"2023-10-21T16:18:46.589149+0000","last_fullsized":"2023-10-21T16:18:46.589149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:07:16.152151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484511+0000","last_change":"2023-10-21T16:18:44.494929+0000","last_active":"2023-10-21T16:18:45.484511+0000","last_peered":"2023-10-21T16:18:45.484511+0000","last_clean":"2023-10-21T16:18:45.484511+0000","last_became_active":"2023-10-21T16:18:44.494446+0000","last_became_peered":"2023-10-21T16:18:44.494446+0000","last_unstale":"2023-10-21T16:18:45.484511+0000","last_undegraded":"2023-10-21T16:18:45.484511+0000","last_fullsized":"2023-10-21T16:18:45.484511+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:59.110459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594541+0000","last_change":"2023-10-21T16:18:43.458733+0000","last_active":"2023-10-21T16:18:46.594541+0000","last_peered":"2023-10-21T16:18:46.594541+0000","last_clean":"2023-10-21T16:18:46.594541+0000","last_became_active":"2023-10-21T16:18:43.458135+0000","last_became_peered":"2023-10-21T16:18:43.458135+0000","last_unstale":"2023-10-21T16:18:46.594541+0000","last_undegraded":"2023-10-21T16:18:46.594541+0000","last_fullsized":"2023-10-21T16:18:46.594541+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:57.128379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590071+0000","last_change":"2023-10-21T16:18:44.513109+0000","last_active":"2023-10-21T16:18:46.590071+0000","last_peered":"2023-10-21T16:18:46.590071+0000","last_clean":"2023-10-21T16:18:46.590071+0000","last_became_active":"2023-10-21T16:18:44.512856+0000","last_became_peered":"2023-10-21T16:18:44.512856+0000","last_unstale":"2023-10-21T16:18:46.590071+0000","last_undegraded":"2023-10-21T16:18:46.590071+0000","last_fullsized":"2023-10-21T16:18:46.590071+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:52.606492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480469+0000","last_change":"2023-10-21T16:18:43.484552+0000","last_active":"2023-10-21T16:18:45.480469+0000","last_peered":"2023-10-21T16:18:45.480469+0000","last_clean":"2023-10-21T16:18:45.480469+0000","last_became_active":"2023-10-21T16:18:43.484279+0000","last_became_peered":"2023-10-21T16:18:43.484279+0000","last_unstale":"2023-10-21T16:18:45.480469+0000","last_undegraded":"2023-10-21T16:18:45.480469+0000","last_fullsized":"2023-10-21T16:18:45.480469+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:49:30.170660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480444+0000","last_change":"2023-10-21T16:18:44.508213+0000","last_active":"2023-10-21T16:18:45.480444+0000","last_peered":"2023-10-21T16:18:45.480444+0000","last_clean":"2023-10-21T16:18:45.480444+0000","last_became_active":"2023-10-21T16:18:44.505331+0000","last_became_peered":"2023-10-21T16:18:44.505331+0000","last_unstale":"2023-10-21T16:18:45.480444+0000","last_undegraded":"2023-10-21T16:18:45.480444+0000","last_fullsized":"2023-10-21T16:18:45.480444+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:15.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"116'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485190+0000","last_change":"2023-10-21T16:18:43.475140+0000","last_active":"2023-10-21T16:18:45.485190+0000","last_peered":"2023-10-21T16:18:45.485190+0000","last_clean":"2023-10-21T16:18:45.485190+0000","last_became_active":"2023-10-21T16:18:43.473184+0000","last_became_peered":"2023-10-21T16:18:43.473184+0000","last_unstale":"2023-10-21T16:18:45.485190+0000","last_undegraded":"2023-10-21T16:18:45.485190+0000","last_fullsized":"2023-10-21T16:18:45.485190+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:18.402787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152298+0000","last_change":"2023-10-21T16:18:44.517097+0000","last_active":"2023-10-21T16:18:46.152298+0000","last_peered":"2023-10-21T16:18:46.152298+0000","last_clean":"2023-10-21T16:18:46.152298+0000","last_became_active":"2023-10-21T16:18:44.516793+0000","last_became_peered":"2023-10-21T16:18:44.516793+0000","last_unstale":"2023-10-21T16:18:46.152298+0000","last_undegraded":"2023-10-21T16:18:46.152298+0000","last_fullsized":"2023-10-21T16:18:46.152298+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:57.790370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"116'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588459+0000","last_change":"2023-10-21T16:18:43.478276+0000","last_active":"2023-10-21T16:18:46.588459+0000","last_peered":"2023-10-21T16:18:46.588459+0000","last_clean":"2023-10-21T16:18:46.588459+0000","last_became_active":"2023-10-21T16:18:43.477985+0000","last_became_peered":"2023-10-21T16:18:43.477985+0000","last_unstale":"2023-10-21T16:18:46.588459+0000","last_undegraded":"2023-10-21T16:18:46.588459+0000","last_fullsized":"2023-10-21T16:18:46.588459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:09.415956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152121+0000","last_change":"2023-10-21T16:18:44.505940+0000","last_active":"2023-10-21T16:18:46.152121+0000","last_peered":"2023-10-21T16:18:46.152121+0000","last_clean":"2023-10-21T16:18:46.152121+0000","last_became_active":"2023-10-21T16:18:44.505622+0000","last_became_peered":"2023-10-21T16:18:44.505622+0000","last_unstale":"2023-10-21T16:18:46.152121+0000","last_undegraded":"2023-10-21T16:18:46.152121+0000","last_fullsized":"2023-10-21T16:18:46.152121+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:33.224873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589695+0000","last_change":"2023-10-21T16:18:43.480810+0000","last_active":"2023-10-21T16:18:46.589695+0000","last_peered":"2023-10-21T16:18:46.589695+0000","last_clean":"2023-10-21T16:18:46.589695+0000","last_became_active":"2023-10-21T16:18:43.478976+0000","last_became_peered":"2023-10-21T16:18:43.478976+0000","last_unstale":"2023-10-21T16:18:46.589695+0000","last_undegraded":"2023-10-21T16:18:46.589695+0000","last_fullsized":"2023-10-21T16:18:46.589695+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:27.544544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.483235+0000","last_change":"2023-10-21T16:18:44.502900+0000","last_active":"2023-10-21T16:18:45.483235+0000","last_peered":"2023-10-21T16:18:45.483235+0000","last_clean":"2023-10-21T16:18:45.483235+0000","last_became_active":"2023-10-21T16:18:44.502614+0000","last_became_peered":"2023-10-21T16:18:44.502614+0000","last_unstale":"2023-10-21T16:18:45.483235+0000","last_undegraded":"2023-10-21T16:18:45.483235+0000","last_fullsized":"2023-10-21T16:18:45.483235+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:45.066650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588201+0000","last_change":"2023-10-21T16:18:43.479657+0000","last_active":"2023-10-21T16:18:46.588201+0000","last_peered":"2023-10-21T16:18:46.588201+0000","last_clean":"2023-10-21T16:18:46.588201+0000","last_became_active":"2023-10-21T16:18:43.479363+0000","last_became_peered":"2023-10-21T16:18:43.479363+0000","last_unstale":"2023-10-21T16:18:46.588201+0000","last_undegraded":"2023-10-21T16:18:46.588201+0000","last_fullsized":"2023-10-21T16:18:46.588201+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:18.818029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481085+0000","last_change":"2023-10-21T16:18:44.513072+0000","last_active":"2023-10-21T16:18:45.481085+0000","last_peered":"2023-10-21T16:18:45.481085+0000","last_clean":"2023-10-21T16:18:45.481085+0000","last_became_active":"2023-10-21T16:18:44.508690+0000","last_became_peered":"2023-10-21T16:18:44.508690+0000","last_unstale":"2023-10-21T16:18:45.481085+0000","last_undegraded":"2023-10-21T16:18:45.481085+0000","last_fullsized":"2023-10-21T16:18:45.481085+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:19:09.972900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152149+0000","last_change":"2023-10-21T16:18:43.478821+0000","last_active":"2023-10-21T16:18:46.152149+0000","last_peered":"2023-10-21T16:18:46.152149+0000","last_clean":"2023-10-21T16:18:46.152149+0000","last_became_active":"2023-10-21T16:18:43.478552+0000","last_became_peered":"2023-10-21T16:18:43.478552+0000","last_unstale":"2023-10-21T16:18:46.152149+0000","last_undegraded":"2023-10-21T16:18:46.152149+0000","last_fullsized":"2023-10-21T16:18:46.152149+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:51.662878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481286+0000","last_change":"2023-10-21T16:18:44.496779+0000","last_active":"2023-10-21T16:18:45.481286+0000","last_peered":"2023-10-21T16:18:45.481286+0000","last_clean":"2023-10-21T16:18:45.481286+0000","last_became_active":"2023-10-21T16:18:44.496535+0000","last_became_peered":"2023-10-21T16:18:44.496535+0000","last_unstale":"2023-10-21T16:18:45.481286+0000","last_undegraded":"2023-10-21T16:18:45.481286+0000","last_fullsized":"2023-10-21T16:18:45.481286+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:35:11.877058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"116'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152091+0000","last_change":"2023-10-21T16:18:43.457220+0000","last_active":"2023-10-21T16:18:46.152091+0000","last_peered":"2023-10-21T16:18:46.152091+0000","last_clean":"2023-10-21T16:18:46.152091+0000","last_became_active":"2023-10-21T16:18:43.456909+0000","last_became_peered":"2023-10-21T16:18:43.456909+0000","last_unstale":"2023-10-21T16:18:46.152091+0000","last_undegraded":"2023-10-21T16:18:46.152091+0000","last_fullsized":"2023-10-21T16:18:46.152091+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:20:59.285099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588312+0000","last_change":"2023-10-21T16:18:44.507652+0000","last_active":"2023-10-21T16:18:46.588312+0000","last_peered":"2023-10-21T16:18:46.588312+0000","last_clean":"2023-10-21T16:18:46.588312+0000","last_became_active":"2023-10-21T16:18:44.507276+0000","last_became_peered":"2023-10-21T16:18:44.507276+0000","last_unstale":"2023-10-21T16:18:46.588312+0000","last_undegraded":"2023-10-21T16:18:46.588312+0000","last_fullsized":"2023-10-21T16:18:46.588312+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:20.497259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.592618+0000","last_change":"2023-10-21T16:18:43.476440+0000","last_active":"2023-10-21T16:18:46.592618+0000","last_peered":"2023-10-21T16:18:46.592618+0000","last_clean":"2023-10-21T16:18:46.592618+0000","last_became_active":"2023-10-21T16:18:43.474936+0000","last_became_peered":"2023-10-21T16:18:43.474936+0000","last_unstale":"2023-10-21T16:18:46.592618+0000","last_undegraded":"2023-10-21T16:18:46.592618+0000","last_fullsized":"2023-10-21T16:18:46.592618+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:23.552770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591600+0000","last_change":"2023-10-21T16:18:44.499977+0000","last_active":"2023-10-21T16:18:46.591600+0000","last_peered":"2023-10-21T16:18:46.591600+0000","last_clean":"2023-10-21T16:18:46.591600+0000","last_became_active":"2023-10-21T16:18:44.499631+0000","last_became_peered":"2023-10-21T16:18:44.499631+0000","last_unstale":"2023-10-21T16:18:46.591600+0000","last_undegraded":"2023-10-21T16:18:46.591600+0000","last_fullsized":"2023-10-21T16:18:46.591600+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:19.735217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589805+0000","last_change":"2023-10-21T16:18:43.479934+0000","last_active":"2023-10-21T16:18:46.589805+0000","last_peered":"2023-10-21T16:18:46.589805+0000","last_clean":"2023-10-21T16:18:46.589805+0000","last_became_active":"2023-10-21T16:18:43.479664+0000","last_became_peered":"2023-10-21T16:18:43.479664+0000","last_unstale":"2023-10-21T16:18:46.589805+0000","last_undegraded":"2023-10-21T16:18:46.589805+0000","last_fullsized":"2023-10-21T16:18:46.589805+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:55.078106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484603+0000","last_change":"2023-10-21T16:18:44.495184+0000","last_active":"2023-10-21T16:18:45.484603+0000","last_peered":"2023-10-21T16:18:45.484603+0000","last_clean":"2023-10-21T16:18:45.484603+0000","last_became_active":"2023-10-21T16:18:44.493971+0000","last_became_peered":"2023-10-21T16:18:44.493971+0000","last_unstale":"2023-10-21T16:18:45.484603+0000","last_undegraded":"2023-10-21T16:18:45.484603+0000","last_fullsized":"2023-10-21T16:18:45.484603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:23:09.773702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485293+0000","last_change":"2023-10-21T16:18:43.470205+0000","last_active":"2023-10-21T16:18:45.485293+0000","last_peered":"2023-10-21T16:18:45.485293+0000","last_clean":"2023-10-21T16:18:45.485293+0000","last_became_active":"2023-10-21T16:18:43.469923+0000","last_became_peered":"2023-10-21T16:18:43.469923+0000","last_unstale":"2023-10-21T16:18:45.485293+0000","last_undegraded":"2023-10-21T16:18:45.485293+0000","last_fullsized":"2023-10-21T16:18:45.485293+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:20:59.285099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.587991+0000","last_change":"2023-10-21T16:18:44.514463+0000","last_active":"2023-10-21T16:18:46.587991+0000","last_peered":"2023-10-21T16:18:46.587991+0000","last_clean":"2023-10-21T16:18:46.587991+0000","last_became_active":"2023-10-21T16:18:44.514170+0000","last_became_peered":"2023-10-21T16:18:44.514170+0000","last_unstale":"2023-10-21T16:18:46.587991+0000","last_undegraded":"2023-10-21T16:18:46.587991+0000","last_fullsized":"2023-10-21T16:18:46.587991+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:53.985741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593434+0000","last_change":"2023-10-21T16:18:43.479995+0000","last_active":"2023-10-21T16:18:46.593434+0000","last_peered":"2023-10-21T16:18:46.593434+0000","last_clean":"2023-10-21T16:18:46.593434+0000","last_became_active":"2023-10-21T16:18:43.479704+0000","last_became_peered":"2023-10-21T16:18:43.479704+0000","last_unstale":"2023-10-21T16:18:46.593434+0000","last_undegraded":"2023-10-21T16:18:46.593434+0000","last_fullsized":"2023-10-21T16:18:46.593434+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:18.818029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593384+0000","last_change":"2023-10-21T16:18:44.492428+0000","last_active":"2023-10-21T16:18:46.593384+0000","last_peered":"2023-10-21T16:18:46.593384+0000","last_clean":"2023-10-21T16:18:46.593384+0000","last_became_active":"2023-10-21T16:18:44.492053+0000","last_became_peered":"2023-10-21T16:18:44.492053+0000","last_unstale":"2023-10-21T16:18:46.593384+0000","last_undegraded":"2023-10-21T16:18:46.593384+0000","last_fullsized":"2023-10-21T16:18:46.593384+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:43.620767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481109+0000","last_change":"2023-10-21T16:18:43.456861+0000","last_active":"2023-10-21T16:18:45.481109+0000","last_peered":"2023-10-21T16:18:45.481109+0000","last_clean":"2023-10-21T16:18:45.481109+0000","last_became_active":"2023-10-21T16:18:43.456383+0000","last_became_peered":"2023-10-21T16:18:43.456383+0000","last_unstale":"2023-10-21T16:18:45.481109+0000","last_undegraded":"2023-10-21T16:18:45.481109+0000","last_fullsized":"2023-10-21T16:18:45.481109+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:05:46.953069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593519+0000","last_change":"2023-10-21T16:18:44.508785+0000","last_active":"2023-10-21T16:18:46.593519+0000","last_peered":"2023-10-21T16:18:46.593519+0000","last_clean":"2023-10-21T16:18:46.593519+0000","last_became_active":"2023-10-21T16:18:44.508491+0000","last_became_peered":"2023-10-21T16:18:44.508491+0000","last_unstale":"2023-10-21T16:18:46.593519+0000","last_undegraded":"2023-10-21T16:18:46.593519+0000","last_fullsized":"2023-10-21T16:18:46.593519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:04.757337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593542+0000","last_change":"2023-10-21T16:18:43.472937+0000","last_active":"2023-10-21T16:18:46.593542+0000","last_peered":"2023-10-21T16:18:46.593542+0000","last_clean":"2023-10-21T16:18:46.593542+0000","last_became_active":"2023-10-21T16:18:43.472641+0000","last_became_peered":"2023-10-21T16:18:43.472641+0000","last_unstale":"2023-10-21T16:18:46.593542+0000","last_undegraded":"2023-10-21T16:18:46.593542+0000","last_fullsized":"2023-10-21T16:18:46.593542+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:27.978233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588692+0000","last_change":"2023-10-21T16:18:44.503119+0000","last_active":"2023-10-21T16:18:46.588692+0000","last_peered":"2023-10-21T16:18:46.588692+0000","last_clean":"2023-10-21T16:18:46.588692+0000","last_became_active":"2023-10-21T16:18:44.502662+0000","last_became_peered":"2023-10-21T16:18:44.502662+0000","last_unstale":"2023-10-21T16:18:46.588692+0000","last_undegraded":"2023-10-21T16:18:46.588692+0000","last_fullsized":"2023-10-21T16:18:46.588692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:04.868404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.481012+0000","last_change":"2023-10-21T16:18:43.474737+0000","last_active":"2023-10-21T16:18:45.481012+0000","last_peered":"2023-10-21T16:18:45.481012+0000","last_clean":"2023-10-21T16:18:45.481012+0000","last_became_active":"2023-10-21T16:18:43.473487+0000","last_became_peered":"2023-10-21T16:18:43.473487+0000","last_unstale":"2023-10-21T16:18:45.481012+0000","last_undegraded":"2023-10-21T16:18:45.481012+0000","last_fullsized":"2023-10-21T16:18:45.481012+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:56:09.613975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591684+0000","last_change":"2023-10-21T16:18:44.501334+0000","last_active":"2023-10-21T16:18:46.591684+0000","last_peered":"2023-10-21T16:18:46.591684+0000","last_clean":"2023-10-21T16:18:46.591684+0000","last_became_active":"2023-10-21T16:18:44.501068+0000","last_became_peered":"2023-10-21T16:18:44.501068+0000","last_unstale":"2023-10-21T16:18:46.591684+0000","last_undegraded":"2023-10-21T16:18:46.591684+0000","last_fullsized":"2023-10-21T16:18:46.591684+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:07.915939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"116'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590421+0000","last_change":"2023-10-21T16:18:43.482899+0000","last_active":"2023-10-21T16:18:46.590421+0000","last_peered":"2023-10-21T16:18:46.590421+0000","last_clean":"2023-10-21T16:18:46.590421+0000","last_became_active":"2023-10-21T16:18:43.482645+0000","last_became_peered":"2023-10-21T16:18:43.482645+0000","last_unstale":"2023-10-21T16:18:46.590421+0000","last_undegraded":"2023-10-21T16:18:46.590421+0000","last_fullsized":"2023-10-21T16:18:46.590421+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:39.845768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588535+0000","last_change":"2023-10-21T16:18:44.509101+0000","last_active":"2023-10-21T16:18:46.588535+0000","last_peered":"2023-10-21T16:18:46.588535+0000","last_clean":"2023-10-21T16:18:46.588535+0000","last_became_active":"2023-10-21T16:18:44.508801+0000","last_became_peered":"2023-10-21T16:18:44.508801+0000","last_unstale":"2023-10-21T16:18:46.588535+0000","last_undegraded":"2023-10-21T16:18:46.588535+0000","last_fullsized":"2023-10-21T16:18:46.588535+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:29.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.484939+0000","last_change":"2023-10-21T16:18:43.471057+0000","last_active":"2023-10-21T16:18:45.484939+0000","last_peered":"2023-10-21T16:18:45.484939+0000","last_clean":"2023-10-21T16:18:45.484939+0000","last_became_active":"2023-10-21T16:18:43.469327+0000","last_became_peered":"2023-10-21T16:18:43.469327+0000","last_unstale":"2023-10-21T16:18:45.484939+0000","last_undegraded":"2023-10-21T16:18:45.484939+0000","last_fullsized":"2023-10-21T16:18:45.484939+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:18.131521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588401+0000","last_change":"2023-10-21T16:18:44.500914+0000","last_active":"2023-10-21T16:18:46.588401+0000","last_peered":"2023-10-21T16:18:46.588401+0000","last_clean":"2023-10-21T16:18:46.588401+0000","last_became_active":"2023-10-21T16:18:44.500631+0000","last_became_peered":"2023-10-21T16:18:44.500631+0000","last_unstale":"2023-10-21T16:18:46.588401+0000","last_undegraded":"2023-10-21T16:18:46.588401+0000","last_fullsized":"2023-10-21T16:18:46.588401+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:04.757337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.485313+0000","last_change":"2023-10-21T16:18:43.476754+0000","last_active":"2023-10-21T16:18:45.485313+0000","last_peered":"2023-10-21T16:18:45.485313+0000","last_clean":"2023-10-21T16:18:45.485313+0000","last_became_active":"2023-10-21T16:18:43.475367+0000","last_became_peered":"2023-10-21T16:18:43.475367+0000","last_unstale":"2023-10-21T16:18:45.485313+0000","last_undegraded":"2023-10-21T16:18:45.485313+0000","last_fullsized":"2023-10-21T16:18:45.485313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:15:51.662878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.594570+0000","last_change":"2023-10-21T16:18:44.506782+0000","last_active":"2023-10-21T16:18:46.594570+0000","last_peered":"2023-10-21T16:18:46.594570+0000","last_clean":"2023-10-21T16:18:46.594570+0000","last_became_active":"2023-10-21T16:18:44.503178+0000","last_became_peered":"2023-10-21T16:18:44.503178+0000","last_unstale":"2023-10-21T16:18:46.594570+0000","last_undegraded":"2023-10-21T16:18:46.594570+0000","last_fullsized":"2023-10-21T16:18:46.594570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:02:04.868404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591918+0000","last_change":"2023-10-21T16:18:43.478809+0000","last_active":"2023-10-21T16:18:46.591918+0000","last_peered":"2023-10-21T16:18:46.591918+0000","last_clean":"2023-10-21T16:18:46.591918+0000","last_became_active":"2023-10-21T16:18:43.478515+0000","last_became_peered":"2023-10-21T16:18:43.478515+0000","last_unstale":"2023-10-21T16:18:46.591918+0000","last_undegraded":"2023-10-21T16:18:46.591918+0000","last_fullsized":"2023-10-21T16:18:46.591918+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:59.144071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.480976+0000","last_change":"2023-10-21T16:18:44.513083+0000","last_active":"2023-10-21T16:18:45.480976+0000","last_peered":"2023-10-21T16:18:45.480976+0000","last_clean":"2023-10-21T16:18:45.480976+0000","last_became_active":"2023-10-21T16:18:44.512695+0000","last_became_peered":"2023-10-21T16:18:44.512695+0000","last_unstale":"2023-10-21T16:18:45.480976+0000","last_undegraded":"2023-10-21T16:18:45.480976+0000","last_fullsized":"2023-10-21T16:18:45.480976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:13.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588267+0000","last_change":"2023-10-21T16:18:43.479555+0000","last_active":"2023-10-21T16:18:46.588267+0000","last_peered":"2023-10-21T16:18:46.588267+0000","last_clean":"2023-10-21T16:18:46.588267+0000","last_became_active":"2023-10-21T16:18:43.479235+0000","last_became_peered":"2023-10-21T16:18:43.479235+0000","last_unstale":"2023-10-21T16:18:46.588267+0000","last_undegraded":"2023-10-21T16:18:46.588267+0000","last_fullsized":"2023-10-21T16:18:46.588267+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:39.845768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:45.479621+0000","last_change":"2023-10-21T16:18:44.518321+0000","last_active":"2023-10-21T16:18:45.479621+0000","last_peered":"2023-10-21T16:18:45.479621+0000","last_clean":"2023-10-21T16:18:45.479621+0000","last_became_active":"2023-10-21T16:18:44.517603+0000","last_became_peered":"2023-10-21T16:18:44.517603+0000","last_unstale":"2023-10-21T16:18:45.479621+0000","last_undegraded":"2023-10-21T16:18:45.479621+0000","last_fullsized":"2023-10-21T16:18:45.479621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:17:30.760547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.593497+0000","last_change":"2023-10-21T16:18:43.475615+0000","last_active":"2023-10-21T16:18:46.593497+0000","last_peered":"2023-10-21T16:18:46.593497+0000","last_clean":"2023-10-21T16:18:46.593497+0000","last_became_active":"2023-10-21T16:18:43.475334+0000","last_became_peered":"2023-10-21T16:18:43.475334+0000","last_unstale":"2023-10-21T16:18:46.593497+0000","last_undegraded":"2023-10-21T16:18:46.593497+0000","last_fullsized":"2023-10-21T16:18:46.593497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:20:51.592237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590871+0000","last_change":"2023-10-21T16:18:44.521812+0000","last_active":"2023-10-21T16:18:46.590871+0000","last_peered":"2023-10-21T16:18:46.590871+0000","last_clean":"2023-10-21T16:18:46.590871+0000","last_became_active":"2023-10-21T16:18:44.521494+0000","last_became_peered":"2023-10-21T16:18:44.521494+0000","last_unstale":"2023-10-21T16:18:46.590871+0000","last_undegraded":"2023-10-21T16:18:46.590871+0000","last_fullsized":"2023-10-21T16:18:46.590871+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:39:43.115726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.591009+0000","last_change":"2023-10-21T16:18:43.478777+0000","last_active":"2023-10-21T16:18:46.591009+0000","last_peered":"2023-10-21T16:18:46.591009+0000","last_clean":"2023-10-21T16:18:46.591009+0000","last_became_active":"2023-10-21T16:18:43.478433+0000","last_became_peered":"2023-10-21T16:18:43.478433+0000","last_unstale":"2023-10-21T16:18:46.591009+0000","last_undegraded":"2023-10-21T16:18:46.591009+0000","last_fullsized":"2023-10-21T16:18:46.591009+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:46:54.154752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.588109+0000","last_change":"2023-10-21T16:18:44.502939+0000","last_active":"2023-10-21T16:18:46.588109+0000","last_peered":"2023-10-21T16:18:46.588109+0000","last_clean":"2023-10-21T16:18:46.588109+0000","last_became_active":"2023-10-21T16:18:44.502339+0000","last_became_peered":"2023-10-21T16:18:44.502339+0000","last_unstale":"2023-10-21T16:18:46.588109+0000","last_undegraded":"2023-10-21T16:18:46.588109+0000","last_fullsized":"2023-10-21T16:18:46.588109+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:34.326495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.589180+0000","last_change":"2023-10-21T16:18:43.475338+0000","last_active":"2023-10-21T16:18:46.589180+0000","last_peered":"2023-10-21T16:18:46.589180+0000","last_clean":"2023-10-21T16:18:46.589180+0000","last_became_active":"2023-10-21T16:18:43.475076+0000","last_became_peered":"2023-10-21T16:18:43.475076+0000","last_unstale":"2023-10-21T16:18:46.589180+0000","last_undegraded":"2023-10-21T16:18:46.589180+0000","last_fullsized":"2023-10-21T16:18:46.589180+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:00:11.854176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.590273+0000","last_change":"2023-10-21T16:18:44.511539+0000","last_active":"2023-10-21T16:18:46.590273+0000","last_peered":"2023-10-21T16:18:46.590273+0000","last_clean":"2023-10-21T16:18:46.590273+0000","last_became_active":"2023-10-21T16:18:44.511172+0000","last_became_peered":"2023-10-21T16:18:44.511172+0000","last_unstale":"2023-10-21T16:18:46.590273+0000","last_undegraded":"2023-10-21T16:18:46.590273+0000","last_fullsized":"2023-10-21T16:18:46.590273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:43.435582+0000","last_clean_scrub_stamp":"2023-10-21T16:18:43.435582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:10.119155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.152951+0000","last_change":"2023-10-21T16:18:43.473411+0000","last_active":"2023-10-21T16:18:46.152951+0000","last_peered":"2023-10-21T16:18:46.152951+0000","last_clean":"2023-10-21T16:18:46.152951+0000","last_became_active":"2023-10-21T16:18:43.473147+0000","last_became_peered":"2023-10-21T16:18:43.473147+0000","last_unstale":"2023-10-21T16:18:46.152951+0000","last_undegraded":"2023-10-21T16:18:46.152951+0000","last_fullsized":"2023-10-21T16:18:46.152951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:18:42.421327+0000","last_clean_scrub_stamp":"2023-10-21T16:18:42.421327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:18.402787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"117'4301","reported_seq":4639,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-21T16:18:46.706732+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:18:46.706732+0000","last_peered":"2023-10-21T16:18:46.706732+0000","last_clean":"2023-10-21T16:18:46.706732+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:18:46.706732+0000","last_undegraded":"2023-10-21T16:18:46.706732+0000","last_fullsized":"2023-10-21T16:18:46.706732+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4301,"log_dups_size":0,"ondisk_log_size":4301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":267,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":689,"num_read_kb":1342,"num_write":267,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4301,"ondisk_log_size":4301,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575050,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59528,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312312,"statfs":{"total":96636764160,"available":96575807488,"internally_reserved":0,"allocated":2031616,"data_stored":1312046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.0840000000000001,"15min":1.139},"min":{"1min":0.67600000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.395,"5min":1.665,"15min":1.8400000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.167,"15min":1.1699999999999999},"min":{"1min":0.64200000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.5309999999999999,"5min":2.448,"15min":2.448},"last":1.0720000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.119,"15min":1.165},"min":{"1min":0.69099999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.468,"5min":1.915,"15min":1.915},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.84999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.5369999999999999,"5min":1.778,"15min":1.778},"last":1.331}]},{"osd":2,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.242,"15min":1.29},"min":{"1min":0.80000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.52,"5min":1.736,"15min":2.085},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.288},"min":{"1min":0.748,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6819999999999999,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.3700000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2509999999999999,"15min":1.254},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.659,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.3859999999999999,"15min":1.4219999999999999},"min":{"1min":0.88500000000000001,"5min":0.82299999999999995,"15min":0.78700000000000003},"max":{"1min":1.581,"5min":1.9570000000000001,"15min":1.9830000000000001},"last":1.264}]},{"osd":4,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.83799999999999997,"5min":0.76700000000000002,"15min":0.61399999999999999},"max":{"1min":2.0579999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4199999999999999,"15min":1.4419999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.915,"5min":2.355,"15min":2.355},"last":1.006}]},{"osd":5,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.417,"15min":1.395},"min":{"1min":1.004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8069999999999999,"5min":1.865,"15min":2.1059999999999999},"last":1.389},{"interface":"front","average":{"1min":1.55,"5min":1.4530000000000001,"15min":1.4390000000000001},"min":{"1min":0.82099999999999995,"5min":0.67700000000000005,"15min":0.67400000000000004},"max":{"1min":2.1200000000000001,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":1.2070000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.55,"15min":1.5469999999999999},"min":{"1min":1.028,"5min":1.0009999999999999,"15min":0.80600000000000005},"max":{"1min":2.1859999999999999,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.4870000000000001,"15min":1.4650000000000001},"min":{"1min":0.873,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0449999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.1220000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575051,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1449984,"data_stored":723838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1240000000000001,"15min":1.135},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.5309999999999999,"5min":1.869,"15min":1.869},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1890000000000001,"15min":1.2190000000000001},"min":{"1min":0.70299999999999996,"5min":0.61299999999999999,"15min":0.60599999999999998},"max":{"1min":1.839,"5min":1.895,"15min":1.895},"last":0.83799999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.1419999999999999,"15min":1.181},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.413,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3180000000000001,"15min":1.306},"min":{"1min":0.76100000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.748,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.64600000000000002}]},{"osd":2,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.21,"15min":1.2509999999999999},"min":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4750000000000001,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.234,"15min":1.2529999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.5840000000000001,"5min":1.718,"15min":1.7789999999999999},"last":0.746}]},{"osd":3,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.331,"15min":1.355},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8360000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.423,"15min":1.417},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.589,"5min":4.6210000000000004,"15min":4.6210000000000004},"last":0.63}]},{"osd":4,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3700000000000001,"15min":1.349},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.919,"5min":2.1930000000000001,"15min":2.2410000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.411,"15min":1.383},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.72799999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5009999999999999,"15min":1.5},"min":{"1min":0.73099999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.956,"5min":2.246,"15min":2.6909999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.339,"15min":1.339},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.6319999999999999,"5min":2.4390000000000001,"15min":2.6379999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Sat Oct 21 16:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.5269999999999999,"15min":1.5029999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.98,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.623,"15min":1.6120000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.8240000000000001,"5min":2.302,"15min":2.302},"last":0.76400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":721622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.054,"15min":1.1120000000000001},"min":{"1min":0.63300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.51,"5min":1.554,"15min":2.0070000000000001},"last":1.228},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1539999999999999,"15min":1.181},"min":{"1min":0.68100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.9530000000000001},"last":0.86199999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2090000000000001,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7410000000000001,"5min":1.877,"15min":1.944},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.1899999999999999,"15min":1.2170000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.619,"5min":1.6919999999999999,"15min":1.782},"last":1.0820000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2310000000000001,"15min":1.286},"min":{"1min":0.76200000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.663,"5min":1.8,"15min":1.95},"last":1.395},{"interface":"front","average":{"1min":1.121,"5min":1.1819999999999999,"15min":1.23},"min":{"1min":0.68200000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5209999999999999,"5min":1.7270000000000001,"15min":1.843},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3089999999999999,"15min":1.3169999999999999},"min":{"1min":0.91800000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8420000000000001,"5min":1.9870000000000001,"15min":2.0630000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.339,"15min":1.379},"min":{"1min":0.82199999999999995,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8959999999999999},"last":1.2}]},{"osd":4,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2050000000000001,"15min":1.2070000000000001},"min":{"1min":0.52600000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6819999999999999,"5min":1.919,"15min":2.0960000000000001},"last":0.747},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.345,"15min":1.343},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.913,"5min":2.0529999999999999,"15min":2.121},"last":1.0469999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.4490000000000001,"15min":1.4430000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.1320000000000001},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.4790000000000001},"min":{"1min":1.0129999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.097,"5min":2.097,"15min":2.1859999999999999},"last":1.026}]},{"osd":7,"last update":"Sat Oct 21 16:18:26 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5269999999999999,"15min":1.5029999999999999},"min":{"1min":0.73099999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.2000000000000002,"5min":2.218,"15min":2.218},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.548},"min":{"1min":0.76700000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0329999999999999,"5min":2.286,"15min":2.286},"last":1.327}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":728446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2,"15min":1.155},"min":{"1min":0.73099999999999998,"5min":0.71599999999999997,"15min":0.624},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.236,"15min":1.2130000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":5.7309999999999999,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.272,"15min":1.2669999999999999},"min":{"1min":0.73999999999999999,"5min":0.621,"15min":0.621},"max":{"1min":5.8179999999999996,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.627},{"interface":"front","average":{"1min":1.579,"5min":1.292,"15min":1.254},"min":{"1min":0.91600000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.484,"5min":7.484,"15min":7.484},"last":0.95899999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.2829999999999999,"15min":1.252},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.3200000000000001,"15min":1.3360000000000001},"min":{"1min":0.85899999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":5.6059999999999999,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.746}]},{"osd":3,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.349,"15min":1.3200000000000001},"min":{"1min":0.97099999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":7.5880000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.4359999999999999,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.6280000000000001,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.387}]},{"osd":5,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.3520000000000001,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":5.8929999999999998,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.9990000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.4330000000000001,"15min":1.355},"min":{"1min":0.97799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.6699999999999999,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.28}]},{"osd":6,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.5349999999999999,"15min":1.423},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"max":{"1min":5.5099999999999998,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.528},{"interface":"front","average":{"1min":1.827,"5min":1.635,"15min":1.5780000000000001},"min":{"1min":1.018,"5min":0.878,"15min":0.878},"max":{"1min":6.1630000000000003,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":2.1240000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:18:31 2023","interfaces":[{"interface":"back","average":{"1min":2.2149999999999999,"5min":1.657,"15min":1.5209999999999999},"min":{"1min":1.1559999999999999,"5min":0.81000000000000005,"15min":0.69599999999999995},"max":{"1min":12.458,"5min":12.458,"15min":12.458},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.778,"5min":1.6679999999999999,"15min":1.5600000000000001},"min":{"1min":1.2390000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":3.9510000000000001,"5min":3.968,"15min":3.968},"last":1.794}]}]},{"osd":3,"up_from":13,"seq":55834575050,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40912,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330928,"statfs":{"total":96636764160,"available":96594870272,"internally_reserved":0,"allocated":1441792,"data_stored":728286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.94699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.276,"5min":1.746,"15min":1.746},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.101,"15min":1.171},"min":{"1min":0.53900000000000003,"5min":0.47799999999999998,"15min":0.45900000000000002},"max":{"1min":1.3680000000000001,"5min":1.8480000000000001,"15min":2.04},"last":0.58899999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96299999999999997,"15min":0.99099999999999999},"min":{"1min":0.51200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.22,"5min":2.173,"15min":2.173},"last":0.48499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.2150000000000001,"15min":1.262},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.498},"max":{"1min":1.752,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.54000000000000004}]},{"osd":2,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.40600000000000003},"max":{"1min":1.397,"5min":1.9830000000000001,"15min":2.0049999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.1739999999999999,"15min":1.2170000000000001},"min":{"1min":0.65200000000000002,"5min":0.46600000000000003,"15min":0.44500000000000001},"max":{"1min":1.5640000000000001,"5min":2.847,"15min":2.847},"last":0.61899999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3500000000000001,"15min":1.401},"min":{"1min":0.86399999999999999,"5min":0.70899999999999996,"15min":0.64900000000000002},"max":{"1min":1.96,"5min":2.0089999999999999,"15min":2.2349999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.292,"5min":1.375,"15min":1.397},"min":{"1min":0.88600000000000001,"5min":0.85199999999999998,"15min":0.56999999999999995},"max":{"1min":1.863,"5min":2.919,"15min":2.919},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.405,"15min":1.413},"min":{"1min":0.80700000000000005,"5min":0.77200000000000002,"15min":0.59399999999999997},"max":{"1min":1.8420000000000001,"5min":2.02,"15min":2.02},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.353,"5min":1.4550000000000001,"15min":1.49},"min":{"1min":0.86299999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8440000000000001,"5min":2.0390000000000001,"15min":2.423},"last":1.0309999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4570000000000001,"15min":1.4930000000000001},"min":{"1min":0.92200000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7829999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.016},{"interface":"front","average":{"1min":1.321,"5min":1.5049999999999999,"15min":1.522},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.70299999999999996},"max":{"1min":1.865,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.96499999999999997}]},{"osd":7,"last update":"Sat Oct 21 16:18:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5429999999999999,"15min":1.5880000000000001},"min":{"1min":0.93500000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8919999999999999,"5min":2.302,"15min":2.3759999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.548,"15min":1.595},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.68899999999999995},"max":{"1min":1.915,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46476,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325364,"statfs":{"total":96636764160,"available":96589172736,"internally_reserved":0,"allocated":1441792,"data_stored":728286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81999999999999995,"15min":0.84899999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0660000000000001,"5min":1.859,"15min":1.859},"last":1.165},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.91400000000000003,"15min":0.97199999999999998},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.357,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.69999999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.0349999999999999,"15min":1.1120000000000001},"min":{"1min":0.52000000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.476,"5min":1.907,"15min":1.9390000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1259999999999999,"15min":1.2390000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.516,"5min":1.881,"15min":1.881},"last":0.92600000000000005}]},{"osd":3,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.0289999999999999,"15min":1.1160000000000001},"min":{"1min":0.503,"5min":0.438,"15min":0.379},"max":{"1min":1.833,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.131},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2649999999999999,"15min":1.349},"min":{"1min":0.61699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8400000000000001,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.2410000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3160000000000001,"15min":1.3620000000000001},"min":{"1min":0.83599999999999997,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.331,"15min":1.3600000000000001},"min":{"1min":0.83099999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.7869999999999999,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":1.014}]},{"osd":5,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.286,"15min":1.3340000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.71099999999999997},"max":{"1min":1.637,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.472,"15min":1.512},"min":{"1min":0.89500000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.958,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.5660000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4139999999999999,"15min":1.454},"min":{"1min":0.97599999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.8919999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.373,"15min":1.423},"min":{"1min":0.83399999999999996,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.6919999999999999,"5min":2.552,"15min":2.552},"last":1.4410000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.46,"15min":1.5169999999999999},"min":{"1min":0.88200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4339999999999999,"15min":1.496},"min":{"1min":0.92100000000000004,"5min":0.78200000000000003,"15min":0.65400000000000003},"max":{"1min":1.798,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.2709999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":1445888,"data_stored":728356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85099999999999998,"15min":0.90200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1220000000000001,"5min":1.3400000000000001,"15min":1.905},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0349999999999999,"15min":1.0660000000000001},"min":{"1min":0.59199999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.6719999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.93899999999999995,"15min":0.98499999999999999},"min":{"1min":0.503,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.7450000000000001,"15min":2.2639999999999998},"last":2.222},{"interface":"front","average":{"1min":1.022,"5min":1.0960000000000001,"15min":1.1719999999999999},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":2.0510000000000002},"last":1.4550000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1839999999999999,"15min":1.2869999999999999},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6890000000000001,"5min":1.7350000000000001,"15min":1.9510000000000001},"last":1.911},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.67200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5900000000000001,"5min":1.877,"15min":1.9159999999999999},"last":1.516}]},{"osd":4,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2370000000000001,"15min":1.27},"min":{"1min":0.80700000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.702,"5min":1.944,"15min":1.944},"last":1.403},{"interface":"front","average":{"1min":1.302,"5min":1.3660000000000001,"15min":1.4079999999999999},"min":{"1min":0.83599999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7749999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.7769999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.429,"15min":1.49},"min":{"1min":0.90800000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4159999999999999,"15min":1.4630000000000001},"min":{"1min":0.875,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.25}]},{"osd":6,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.444,"15min":1.488},"min":{"1min":1.032,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.903,"5min":2.2469999999999999,"15min":2.4079999999999999},"last":1.821},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.381,"15min":1.4530000000000001},"min":{"1min":0.90500000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.925,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":2.0169999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:18:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.526,"15min":1.5920000000000001},"min":{"1min":0.93799999999999994,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8540000000000001,"5min":2.2679999999999998,"15min":2.3929999999999998},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.516,"15min":1.593},"min":{"1min":1.1080000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.885,"5min":2.1619999999999999,"15min":2.367},"last":1.958}]}]},{"osd":0,"up_from":13,"seq":55834575050,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2048000,"data_stored":1314308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.85299999999999998,"15min":0.90700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.4259999999999999,"5min":1.649,"15min":1.855},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.96799999999999997,"15min":1.052},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5329999999999999,"5min":1.5940000000000001,"15min":1.7509999999999999},"last":0.89900000000000002}]},{"osd":2,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.03,"15min":1.113},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5740000000000001,"5min":1.708,"15min":1.7749999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.081,"15min":1.135},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3420000000000001,"5min":1.694,"15min":1.732},"last":1.125}]},{"osd":3,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0309999999999999,"15min":1.0840000000000001},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.421,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1739999999999999,"15min":1.2749999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7050000000000001,"5min":1.8109999999999999,"15min":2.1619999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.355,"15min":1.403},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.863,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4330000000000001,"15min":1.5149999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.956,"5min":2.246,"15min":2.2890000000000001},"last":1.1100000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.329,"15min":1.3700000000000001},"min":{"1min":0.64000000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5620000000000001,"5min":1.9419999999999999,"15min":2.137},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.393,"15min":1.4790000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.0699999999999998,"5min":2.137,"15min":2.4670000000000001},"last":1.079}]},{"osd":6,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.4079999999999999,"15min":1.468},"min":{"1min":0.79600000000000004,"5min":0.753,"15min":0.753},"max":{"1min":1.8660000000000001,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":0.87},{"interface":"front","average":{"1min":1.333,"5min":1.4710000000000001,"15min":1.573},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.121,"5min":2.355,"15min":2.403},"last":1.0349999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:18:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0289999999999999,"5min":2.2959999999999998,"15min":2.3279999999999998},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.4710000000000001,"15min":1.556},"min":{"1min":0.65100000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.978,"5min":2.077,"15min":2.3540000000000001},"last":1.1519999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:18:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:18:49.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:18:49.616 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:49.613+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:49.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.885+0000 7f9f8d240640 1 Processor -- start 2023-10-21T16:18:49.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.885+0000 7f9f8d240640 1 -- start start 2023-10-21T16:18:49.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8d240640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f880cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8d240640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f880ca810 con 0x7f9f880b4900 2023-10-21T16:18:49.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8d240640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8815f190 con 0x7f9f8815a880 2023-10-21T16:18:49.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8d240640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815f470 con 0x7f9f880caca0 2023-10-21T16:18:49.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8ca3f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9f880b4900 0x7f9f8806e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34626/0 (socket says 172.21.15.53:34626) 2023-10-21T16:18:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.887+0000 7f9f8ca3f640 1 -- 172.21.15.53:0/3458321039 learned_addr learned my addr 172.21.15.53:0/3458321039 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f877fe640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f880cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537950948 0 0) 0x7f9f880ca810 con 0x7f9f880b4900 2023-10-21T16:18:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f70003520 con 0x7f9f880b4900 2023-10-21T16:18:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041113782 0 0) 0x7f9f70003520 con 0x7f9f880b4900 2023-10-21T16:18:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 msgr2=0x7f9f880cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f880cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 >> v1:172.21.15.124:6789/0 conn(0x7f9f8815a880 legacy=0x7f9f880b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8806cad0 con 0x7f9f880b4900 2023-10-21T16:18:49.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9f800019f0 con 0x7f9f880b4900 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 >> v1:172.21.15.53:6789/0 conn(0x7f9f880b4900 legacy=0x7f9f8806e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.888+0000 7f9f86ffd640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f700039a0 0x7f9f70003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f86ffd640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f86ffd640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815f470 con 0x7f9f70004d00 2023-10-21T16:18:49.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7000bb10 con 0x7f9f700039a0 2023-10-21T16:18:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7000bcd0 con 0x7f9f880caca0 2023-10-21T16:18:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f8ca3f640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f87fff640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f700039a0 0x7f9f70003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.889+0000 7f9f877fe640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.890+0000 7f9f8ca3f640 1 -- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 msgr2=0x7f9f700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.890+0000 7f9f8ca3f640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880caca0 0x7f9f700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.890+0000 7f9f8ca3f640 1 -- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f700039a0 msgr2=0x7f9f70003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.890+0000 7f9f8ca3f640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f700039a0 0x7f9f70003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.890+0000 7f9f8ca3f640 1 -- 172.21.15.53:0/3458321039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70003520 con 0x7f9f70004d00 2023-10-21T16:18:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.891+0000 7f9f8ca3f640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9f880b14b0 tx=0x7f9f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6856b75a8abf66 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.891+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80034630 con 0x7f9f70004d00 2023-10-21T16:18:49.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.891+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8002f070 con 0x7f9f70004d00 2023-10-21T16:18:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.891+0000 7f9f86ffd640 1 -- 172.21.15.53:0/3458321039 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80034a10 con 0x7f9f70004d00 2023-10-21T16:18:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 -- 172.21.15.53:0/3458321039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 msgr2=0x7f9f70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 --2- 172.21.15.53:0/3458321039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9f880b14b0 tx=0x7f9f700057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8ca3f640 1 -- 172.21.15.53:0/3458321039 reap_dead start 2023-10-21T16:18:49.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 -- 172.21.15.53:0/3458321039 shutdown_connections 2023-10-21T16:18:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 -- 172.21.15.53:0/3458321039 >> 172.21.15.53:0/3458321039 conn(0x7f9f880b08e0 msgr2=0x7f9f880b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 -- 172.21.15.53:0/3458321039 shutdown_connections 2023-10-21T16:18:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.892+0000 7f9f8d240640 1 -- 172.21.15.53:0/3458321039 wait complete. 2023-10-21T16:18:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.895+0000 7f9f8d240640 1 Processor -- start 2023-10-21T16:18:49.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.895+0000 7f9f8d240640 1 -- start start 2023-10-21T16:18:49.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8815ccf0 0x7f9f8815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880c3b40 0x7f9f880c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880c7de0 con 0x7f9f70004d00 2023-10-21T16:18:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880c2d80 con 0x7f9f8815ccf0 2023-10-21T16:18:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f8d240640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880c3060 con 0x7f9f880c3b40 2023-10-21T16:18:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f87fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47476/0 (socket says 172.21.15.53:47476) 2023-10-21T16:18:49.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f87fff640 1 -- 172.21.15.53:0/1895400193 learned_addr learned my addr 172.21.15.53:0/1895400193 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:49.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.896+0000 7f9f877fe640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8815ccf0 0x7f9f8815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f8ca3f640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880c3b40 0x7f9f880c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 -- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880c3b40 msgr2=0x7f9f880c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880c3b40 0x7f9f880c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 -- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8815ccf0 msgr2=0x7f9f8815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:49.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8815ccf0 0x7f9f8815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:49.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 -- 172.21.15.53:0/1895400193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880c3340 con 0x7f9f70004d00 2023-10-21T16:18:49.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f87fff640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800f820 tx=0x7f9f7800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f780050d0 con 0x7f9f70004d00 2023-10-21T16:18:49.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f78005230 con 0x7f9f70004d00 2023-10-21T16:18:49.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800d730 con 0x7f9f70004d00 2023-10-21T16:18:49.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.897+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f880c3620 con 0x7f9f70004d00 2023-10-21T16:18:49.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.898+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8815a060 con 0x7f9f70004d00 2023-10-21T16:18:49.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.899+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8804e160 con 0x7f9f70004d00 2023-10-21T16:18:49.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.900+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9f7800d890 con 0x7f9f70004d00 2023-10-21T16:18:49.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.901+0000 7f9f84ff9640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 0x7f9f7406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:49.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.901+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f78089730 con 0x7f9f70004d00 2023-10-21T16:18:49.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.901+0000 7f9f877fe640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 0x7f9f7406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:49.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.903+0000 7f9f877fe640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 0x7f9f7406d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f880c6320 tx=0x7f9f7c008040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:49.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:49.905+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f78054600 con 0x7f9f70004d00 2023-10-21T16:18:50.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.181+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9f8815a3f0 con 0x7f9f70004d00 2023-10-21T16:18:50.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.185+0000 7f9f84ff9640 1 -- 172.21.15.53:0/1895400193 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9f78057c20 con 0x7f9f70004d00 2023-10-21T16:18:50.187 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:18:50.187 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:18:50.188 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:18:50.188 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:18:50.188 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:18:50.188 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:18:50.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 msgr2=0x7f9f7406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 0x7f9f7406d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f880c6320 tx=0x7f9f7c008040 comp rx=0 tx=0).stop 2023-10-21T16:18:50.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 msgr2=0x7f9f88159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800f820 tx=0x7f9f7800fce0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 shutdown_connections 2023-10-21T16:18:50.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f9f7406af30 0x7f9f7406d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f880c3b40 0x7f9f880c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f8815ccf0 0x7f9f8815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 --2- 172.21.15.53:0/1895400193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f70004d00 0x7f9f88159900 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 >> 172.21.15.53:0/1895400193 conn(0x7f9f880b08e0 msgr2=0x7f9f88069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.188+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 shutdown_connections 2023-10-21T16:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.189+0000 7f9f8d240640 1 -- 172.21.15.53:0/1895400193 wait complete. 2023-10-21T16:18:50.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:18:50.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.421+0000 7f41defb3640 1 Processor -- start 2023-10-21T16:18:50.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.421+0000 7f41defb3640 1 -- start start 2023-10-21T16:18:50.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.423+0000 7f41defb3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.423+0000 7f41defb3640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 0x7f41d806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.423+0000 7f41defb3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d815f7d0 con 0x7f41d80adc70 2023-10-21T16:18:50.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.423+0000 7f41defb3640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d815f210 con 0x7f41d80b07d0 2023-10-21T16:18:50.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.423+0000 7f41defb3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d815f4f0 con 0x7f41d815ab30 2023-10-21T16:18:50.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41de7b2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 0x7f41d806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:50.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41ddfb1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:50.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41de7b2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 0x7f41d806cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:45300/0 (socket says 172.21.15.53:45300) 2023-10-21T16:18:50.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41ddfb1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:44506/0 (socket says 172.21.15.53:44506) 2023-10-21T16:18:50.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41de7b2640 1 -- 172.21.15.53:0/1365387365 learned_addr learned my addr 172.21.15.53:0/1365387365 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:50.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41dd7b0640 1 --1- 172.21.15.53:0/1365387365 >> v1:172.21.15.124:6789/0 conn(0x7f41d80adc70 0x7f41d80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:50118/0 (socket says 172.21.15.53:50118) 2023-10-21T16:18:50.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41ddfb1640 1 -- 172.21.15.53:0/1365387365 >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 msgr2=0x7f41d806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:50.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.424+0000 7f41ddfb1640 1 --2- 172.21.15.53:0/1365387365 >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 0x7f41d806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.425+0000 7f41ddfb1640 1 -- 172.21.15.53:0/1365387365 >> v1:172.21.15.124:6789/0 conn(0x7f41d80adc70 legacy=0x7f41d80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.425+0000 7f41ddfb1640 1 -- 172.21.15.53:0/1365387365 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d806c550 con 0x7f41d815ab30 2023-10-21T16:18:50.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.426+0000 7f41ddfb1640 1 --2- 172.21.15.53:0/1365387365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009d90 tx=0x7f41d402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1e26f538a7f785 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:50.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.426+0000 7f41dcfaf640 1 -- 172.21.15.53:0/1365387365 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d402cc50 con 0x7f41d815ab30 2023-10-21T16:18:50.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.426+0000 7f41dcfaf640 1 -- 172.21.15.53:0/1365387365 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d402cdb0 con 0x7f41d815ab30 2023-10-21T16:18:50.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.426+0000 7f41dcfaf640 1 -- 172.21.15.53:0/1365387365 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4034660 con 0x7f41d815ab30 2023-10-21T16:18:50.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 -- 172.21.15.53:0/1365387365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 msgr2=0x7f41d815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 --2- 172.21.15.53:0/1365387365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009d90 tx=0x7f41d402bea0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 -- 172.21.15.53:0/1365387365 shutdown_connections 2023-10-21T16:18:50.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 --2- 172.21.15.53:0/1365387365 >> v2:172.21.15.124:3300/0 conn(0x7f41d80b07d0 0x7f41d806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 --2- 172.21.15.53:0/1365387365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d815ab30 0x7f41d815e8a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 -- 172.21.15.53:0/1365387365 >> 172.21.15.53:0/1365387365 conn(0x7f41d80cb0c0 msgr2=0x7f41d80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:50.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 -- 172.21.15.53:0/1365387365 shutdown_connections 2023-10-21T16:18:50.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.427+0000 7f41defb3640 1 -- 172.21.15.53:0/1365387365 wait complete. 2023-10-21T16:18:50.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.429+0000 7f41defb3640 1 Processor -- start 2023-10-21T16:18:50.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.429+0000 7f41defb3640 1 -- start start 2023-10-21T16:18:50.436 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-21T16:18:50.430+0000 7f9ff0aaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d80b07d0 0x7f41d815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d815ab30 0x7f41d815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8154cc0 con 0x7f41d80adc70 2023-10-21T16:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80c2d40 con 0x7f41d815ab30 2023-10-21T16:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41defb3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80c2130 con 0x7f41d80b07d0 2023-10-21T16:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41ddfb1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41ddfb1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47480/0 (socket says 172.21.15.53:47480) 2023-10-21T16:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.430+0000 7f41ddfb1640 1 -- 172.21.15.53:0/587074634 learned_addr learned my addr 172.21.15.53:0/587074634 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41dd7b0640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d80b07d0 0x7f41d815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 -- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d80b07d0 msgr2=0x7f41d815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d80b07d0 0x7f41d815af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 -- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d815ab30 msgr2=0x7f41d815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d815ab30 0x7f41d815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 -- 172.21.15.53:0/587074634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d4009a70 con 0x7f41d80adc70 2023-10-21T16:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.431+0000 7f41ddfb1640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f41d402c4b0 tx=0x7f41d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.432+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d403e610 con 0x7f41d80adc70 2023-10-21T16:18:50.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.432+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d403e770 con 0x7f41d80adc70 2023-10-21T16:18:50.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.432+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d403eae0 con 0x7f41d80adc70 2023-10-21T16:18:50.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.432+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80c2410 con 0x7f41d80adc70 2023-10-21T16:18:50.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.432+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80c29d0 con 0x7f41d80adc70 2023-10-21T16:18:50.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.433+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d8066700 con 0x7f41d80adc70 2023-10-21T16:18:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.434+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f41d403b050 con 0x7f41d80adc70 2023-10-21T16:18:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.435+0000 7f41ce7fc640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 0x7f41bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.435+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f41d40ac110 con 0x7f41d80adc70 2023-10-21T16:18:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.436+0000 7f41dd7b0640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 0x7f41bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:50.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.437+0000 7f41dd7b0640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 0x7f41bc06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41d8158860 tx=0x7f41c8009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:50.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.439+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41d4077060 con 0x7f41d80adc70 2023-10-21T16:18:50.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.687+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41d80c2b60 con 0x7f41d80adc70 2023-10-21T16:18:50.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.688+0000 7f41ce7fc640 1 -- 172.21.15.53:0/587074634 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f41d4009450 con 0x7f41d80adc70 2023-10-21T16:18:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:18:50.690 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:18:50.690 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:18:50.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.690+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 msgr2=0x7f41bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.690+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 0x7f41bc06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41d8158860 tx=0x7f41c8009040 comp rx=0 tx=0).stop 2023-10-21T16:18:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.690+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 msgr2=0x7f41d8154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.690+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f41d402c4b0 tx=0x7f41d4004a40 comp rx=0 tx=0).stop 2023-10-21T16:18:50.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 shutdown_connections 2023-10-21T16:18:50.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f41bc06af00 0x7f41bc06d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d815ab30 0x7f41d815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d80b07d0 0x7f41d815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 --2- 172.21.15.53:0/587074634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d80adc70 0x7f41d8154780 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:50.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 >> 172.21.15.53:0/587074634 conn(0x7f41d80cb0c0 msgr2=0x7f41d80cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:50.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 shutdown_connections 2023-10-21T16:18:50.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:50.691+0000 7f41defb3640 1 -- 172.21.15.53:0/587074634 wait complete. 2023-10-21T16:18:50.858 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:50.855+0000 7f567c863640 -1 client.6454: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.35723:38f489f9-4793-4c2d-9289-616d8b6fcdf5:v1:172.21.15.124:0/2272255357] 2023-10-21T16:18:51.114 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:18:43.938045+0000', 'modified': '2023-10-21T16:18:47.045682+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35915}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35915': {'gid': 35915, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/3805349006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6835', 'nonce': 3805349006}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 3805349006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:18:51.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:18:51.116 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-21T16:18:51.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:18:51.334 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:18:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.349+0000 7f22effff640 1 Processor -- start 2023-10-21T16:18:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.349+0000 7f22effff640 1 -- start start 2023-10-21T16:18:51.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22effff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22f00cf690 0x7f22f00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22effff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f00cd520 con 0x7f22f00ce510 2023-10-21T16:18:51.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22effff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f00d1910 con 0x7f22f00cf2b0 2023-10-21T16:18:51.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22effff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f00ce080 con 0x7f22f00cf690 2023-10-21T16:18:51.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22e7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22f00cf690 0x7f22f00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22ef7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f22f00ce510 0x7f22f015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34634/0 (socket says 172.21.15.53:34634) 2023-10-21T16:18:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.351+0000 7f22ef7fe640 1 -- 172.21.15.53:0/4242253558 learned_addr learned my addr 172.21.15.53:0/4242253558 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.352+0000 7f22e7fff640 1 -- 172.21.15.53:0/4242253558 >> v1:172.21.15.124:6789/0 conn(0x7f22f00cf2b0 legacy=0x7f22f00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.352+0000 7f22e7fff640 1 -- 172.21.15.53:0/4242253558 >> v1:172.21.15.53:6789/0 conn(0x7f22f00ce510 legacy=0x7f22f015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.352+0000 7f22e7fff640 1 -- 172.21.15.53:0/4242253558 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f00cd750 con 0x7f22f00cf690 2023-10-21T16:18:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.353+0000 7f22e7fff640 1 --2- 172.21.15.53:0/4242253558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf690 0x7f22f00d34d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f22d80097a0 tx=0x7f22d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38358531a2cc651 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.353+0000 7f22ee7fc640 1 -- 172.21.15.53:0/4242253558 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8031830 con 0x7f22f00cf690 2023-10-21T16:18:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.353+0000 7f22ee7fc640 1 -- 172.21.15.53:0/4242253558 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8031990 con 0x7f22f00cf690 2023-10-21T16:18:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.353+0000 7f22ee7fc640 1 -- 172.21.15.53:0/4242253558 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8031d20 con 0x7f22f00cf690 2023-10-21T16:18:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 -- 172.21.15.53:0/4242253558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf690 msgr2=0x7f22f00d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 --2- 172.21.15.53:0/4242253558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf690 0x7f22f00d34d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f22d80097a0 tx=0x7f22d802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 -- 172.21.15.53:0/4242253558 shutdown_connections 2023-10-21T16:18:51.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 --2- 172.21.15.53:0/4242253558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf690 0x7f22f00d34d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 -- 172.21.15.53:0/4242253558 >> 172.21.15.53:0/4242253558 conn(0x7f22f009bc40 msgr2=0x7f22f00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 -- 172.21.15.53:0/4242253558 shutdown_connections 2023-10-21T16:18:51.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.354+0000 7f22effff640 1 -- 172.21.15.53:0/4242253558 wait complete. 2023-10-21T16:18:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.356+0000 7f22effff640 1 Processor -- start 2023-10-21T16:18:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.356+0000 7f22effff640 1 -- start start 2023-10-21T16:18:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22f00ce510 0x7f22f009cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 0x7f22f009e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0157e10 con 0x7f22f00cf2b0 2023-10-21T16:18:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f01580f0 con 0x7f22f00cf690 2023-10-21T16:18:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22effff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f01583d0 con 0x7f22f00ce510 2023-10-21T16:18:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47492/0 (socket says 172.21.15.53:47492) 2023-10-21T16:18:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22e7fff640 1 -- 172.21.15.53:0/411320955 learned_addr learned my addr 172.21.15.53:0/411320955 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 0x7f22f009e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.357+0000 7f22eeffd640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22f00ce510 0x7f22f009cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 -- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22f00ce510 msgr2=0x7f22f009cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22f00ce510 0x7f22f009cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 -- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 msgr2=0x7f22f009e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 0x7f22f009e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 -- 172.21.15.53:0/411320955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8009400 con 0x7f22f00cf2b0 2023-10-21T16:18:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 0x7f22f009e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22e7fff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.358+0000 7f22ecff9640 1 -- 172.21.15.53:0/411320955 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d803aa60 con 0x7f22f00cf2b0 2023-10-21T16:18:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.359+0000 7f22ecff9640 1 -- 172.21.15.53:0/411320955 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d803abe0 con 0x7f22f00cf2b0 2023-10-21T16:18:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.359+0000 7f22ecff9640 1 -- 172.21.15.53:0/411320955 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80443f0 con 0x7f22f00cf2b0 2023-10-21T16:18:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.359+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f01586b0 con 0x7f22f00cf2b0 2023-10-21T16:18:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.359+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0158c70 con 0x7f22f00cf2b0 2023-10-21T16:18:51.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.361+0000 7f22ecff9640 1 -- 172.21.15.53:0/411320955 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f22d804d450 con 0x7f22f00cf2b0 2023-10-21T16:18:51.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.362+0000 7f22ecff9640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 0x7f22d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.362+0000 7f22ecff9640 1 -- 172.21.15.53:0/411320955 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f22d80abb30 con 0x7f22f00cf2b0 2023-10-21T16:18:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.363+0000 7f22eeffd640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 0x7f22d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.363+0000 7f22f85ed740 1 Processor -- start 2023-10-21T16:18:51.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.363+0000 7f22f85ed740 1 -- start start 2023-10-21T16:18:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632429f65b0 0x563242a26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632429167a0 con 0x563242a23170 2023-10-21T16:18:51.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563242968740 con 0x563242a23550 2023-10-21T16:18:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563242965e30 con 0x5632429f65b0 2023-10-21T16:18:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22eeffd640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 0x7f22d006d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000f60 tx=0x7f22e0009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22eeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22eeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47496/0 (socket says 172.21.15.53:47496) 2023-10-21T16:18:51.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22eeffd640 1 -- 172.21.15.53:0/80759126 learned_addr learned my addr 172.21.15.53:0/80759126 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22e7fff640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.364+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632429f65b0 0x563242a26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 -- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632429f65b0 msgr2=0x563242a26440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632429f65b0 0x563242a26440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 -- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 msgr2=0x563242a25f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a25f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 -- 172.21.15.53:0/80759126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563242925910 con 0x563242a23170 2023-10-21T16:18:51.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22e7fff640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:51.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22eeffd640 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f22e0001eb0 tx=0x7f22e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea897e6f27751bcc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22e57fa640 1 -- 172.21.15.53:0/80759126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000ee20 con 0x563242a23170 2023-10-21T16:18:51.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22e57fa640 1 -- 172.21.15.53:0/80759126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0003800 con 0x563242a23170 2023-10-21T16:18:51.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.365+0000 7f22e57fa640 1 -- 172.21.15.53:0/80759126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003b70 con 0x563242a23170 2023-10-21T16:18:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 -- 172.21.15.53:0/80759126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 msgr2=0x5632429f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f22e0001eb0 tx=0x7f22e0011000 comp rx=0 tx=0).stop 2023-10-21T16:18:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 -- 172.21.15.53:0/80759126 shutdown_connections 2023-10-21T16:18:51.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5632429f65b0 0x563242a26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a25f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 --2- 172.21.15.53:0/80759126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23170 0x5632429f6070 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 -- 172.21.15.53:0/80759126 >> 172.21.15.53:0/80759126 conn(0x5632429f8380 msgr2=0x563242a40f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.366+0000 7f22f85ed740 1 -- 172.21.15.53:0/80759126 shutdown_connections 2023-10-21T16:18:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.367+0000 7f22f85ed740 1 -- 172.21.15.53:0/80759126 wait complete. 2023-10-21T16:18:51.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.369+0000 7f22f85ed740 1 -- start start 2023-10-21T16:18:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.369+0000 7f22f85ed740 1 Processor -- start 2023-10-21T16:18:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563242961720 con 0x563242a51800 2023-10-21T16:18:51.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632428ff970 con 0x563242a23550 2023-10-21T16:18:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632428ff610 con 0x563242a23170 2023-10-21T16:18:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22f85ed740 10 client.0 _ll_register_callbacks cb 0x5632429219d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:18:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47510/0 (socket says 172.21.15.53:47510) 2023-10-21T16:18:51.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22ef7fe640 1 -- 172.21.15.53:0/4005843132 learned_addr learned my addr 172.21.15.53:0/4005843132 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22eeffd640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.370+0000 7f22e7fff640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 -- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 msgr2=0x563242a11140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a11140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 -- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 msgr2=0x563242a512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 -- 172.21.15.53:0/4005843132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e000f040 con 0x563242a51800 2023-10-21T16:18:51.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22eeffd640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a11140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:51.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f22e8007670 tx=0x7f22e8007760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6c95d7e612a363 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:18:51.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e80106c0 con 0x563242a51800 2023-10-21T16:18:51.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563242988160 con 0x563242a51800 2023-10-21T16:18:51.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.371+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f22e8010820 con 0x563242a51800 2023-10-21T16:18:51.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22e80109e0 con 0x563242a51800 2023-10-21T16:18:51.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 10 client.7695 fetch_fsmap learned FSMap version 53 2023-10-21T16:18:51.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x56324295fda0 con 0x563242a51800 2023-10-21T16:18:51.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563242988160 con 0x563242a51800 2023-10-21T16:18:51.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22e8010b40 con 0x563242a51800 2023-10-21T16:18:51.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 10 client.7695 fetch_fsmap finished waiting for FSMap version 53 2023-10-21T16:18:51.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 10 client.7695 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T16:18:51.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 20 client.7695 populate_metadata read hostname 'smithi053' 2023-10-21T16:18:51.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a13810 0x563242a13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 4 client.7695 mds_command: new command op to 35915 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:18:51.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.372+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 --> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563242933cc0 con 0x563242a13810 2023-10-21T16:18:51.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.373+0000 7f22eeffd640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a13810 0x563242a13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.374+0000 7f22eeffd640 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a13810 0x563242a13bf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7601a2def2d2f2c0 server_cookie=de0efa9f5a9784df in_seq=0 out_seq=0 2023-10-21T16:18:51.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.374+0000 7f22ce7fc640 10 client.7695 ms_handle_connect on v2:172.21.15.124:6835/3805349006 2023-10-21T16:18:51.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22ce7fc640 1 -- 172.21.15.53:0/4005843132 <== mds.0 v2:172.21.15.124:6835/3805349006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563242933cc0 con 0x563242a13810 2023-10-21T16:18:51.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22ce7fc640 10 client.7695 handle_command_reply: tid=0 2023-10-21T16:18:51.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 client.7695 shutdown 2023-10-21T16:18:51.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 msgr2=0x563242a3b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f22e8007670 tx=0x7f22e8007760 comp rx=0 tx=0).stop 2023-10-21T16:18:51.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 shutdown_connections 2023-10-21T16:18:51.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a13810 0x563242a13bf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a3b530 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23550 0x563242a512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22ce7fc640 0 client.7695 ms_handle_reset on v2:172.21.15.124:6835/3805349006 2023-10-21T16:18:51.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 --2- 172.21.15.53:0/4005843132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a11140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 >> 172.21.15.53:0/4005843132 conn(0x5632429f65b0 msgr2=0x563242a25ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 shutdown_connections 2023-10-21T16:18:51.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.375+0000 7f22f85ed740 1 -- 172.21.15.53:0/4005843132 wait complete. 2023-10-21T16:18:51.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.376+0000 7f22f85ed740 20 client.7695 trim_cache size 0 max 16384 2023-10-21T16:18:51.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.393+0000 7f22f85ed740 1 Processor -- start 2023-10-21T16:18:51.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.393+0000 7f22f85ed740 1 -- start start 2023-10-21T16:18:51.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.393+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.393+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 0x563242a44950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22f85ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632426c5f20 con 0x563242a23550 2023-10-21T16:18:51.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563242920f10 con 0x563242a51800 2023-10-21T16:18:51.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563242a3c890 con 0x563242a23170 2023-10-21T16:18:51.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47516/0 (socket says 172.21.15.53:47516) 2023-10-21T16:18:51.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 -- 172.21.15.53:0/704236916 learned_addr learned my addr 172.21.15.53:0/704236916 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22eeffd640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 0x563242a44950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 -- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 msgr2=0x563242a3c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a3c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 -- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 msgr2=0x563242a44950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 0x563242a44950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22e7fff640 1 -- 172.21.15.53:0/704236916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632428feb80 con 0x563242a23550 2023-10-21T16:18:51.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.394+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 0x563242a44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:18:51.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.395+0000 7f22e7fff640 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f22d807a2c0 tx=0x7f22d8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8b8d9907ceea12 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.395+0000 7f22e5ffb640 1 -- 172.21.15.53:0/704236916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8094030 con 0x563242a23550 2023-10-21T16:18:51.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.395+0000 7f22e5ffb640 1 -- 172.21.15.53:0/704236916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d807e710 con 0x563242a23550 2023-10-21T16:18:51.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.395+0000 7f22e5ffb640 1 -- 172.21.15.53:0/704236916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80828c0 con 0x563242a23550 2023-10-21T16:18:51.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 -- 172.21.15.53:0/704236916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 msgr2=0x563242a44410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f22d807a2c0 tx=0x7f22d8009190 comp rx=0 tx=0).stop 2023-10-21T16:18:51.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 -- 172.21.15.53:0/704236916 shutdown_connections 2023-10-21T16:18:51.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a51800 0x563242a44950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a23550 0x563242a44410 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 --2- 172.21.15.53:0/704236916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23170 0x563242a3c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 -- 172.21.15.53:0/704236916 >> 172.21.15.53:0/704236916 conn(0x5632429f65b0 msgr2=0x563242a10f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 -- 172.21.15.53:0/704236916 shutdown_connections 2023-10-21T16:18:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.396+0000 7f22f85ed740 1 -- 172.21.15.53:0/704236916 wait complete. 2023-10-21T16:18:51.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.397+0000 7f22f85ed740 1 -- start start 2023-10-21T16:18:51.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.397+0000 7f22f85ed740 1 Processor -- start 2023-10-21T16:18:51.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 0x5632429fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23550 0x563242a29ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563242a3f380 con 0x563242a51800 2023-10-21T16:18:51.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632429fb640 con 0x563242a23170 2023-10-21T16:18:51.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563242a27cd0 con 0x563242a23550 2023-10-21T16:18:51.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22f85ed740 10 client.0 _ll_register_callbacks cb 0x7f22e8001b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:18:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22eeffd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 0x5632429fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47520/0 (socket says 172.21.15.53:47520) 2023-10-21T16:18:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22ef7fe640 1 -- 172.21.15.53:0/3784391366 learned_addr learned my addr 172.21.15.53:0/3784391366 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.399+0000 7f22e7fff640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23550 0x563242a29ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 -- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23550 msgr2=0x563242a29ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23550 0x563242a29ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 -- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 msgr2=0x5632429fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 0x5632429fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 -- 172.21.15.53:0/3784391366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f0158c70 con 0x563242a51800 2023-10-21T16:18:51.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22eeffd640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 0x5632429fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:51.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ef7fe640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f22e8007160 tx=0x7f22e8005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4976b75c7c26a831 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:18:51.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8014400 con 0x563242a51800 2023-10-21T16:18:51.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f22e8007190 con 0x563242a51800 2023-10-21T16:18:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.400+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22e0003800 con 0x563242a51800 2023-10-21T16:18:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22e8001c70 con 0x563242a51800 2023-10-21T16:18:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 10 client.7701 fetch_fsmap learned FSMap version 53 2023-10-21T16:18:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x563242924170 con 0x563242a51800 2023-10-21T16:18:51.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0003800 con 0x563242a51800 2023-10-21T16:18:51.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22e8014800 con 0x563242a51800 2023-10-21T16:18:51.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 10 client.7701 fetch_fsmap finished waiting for FSMap version 53 2023-10-21T16:18:51.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 10 client.7701 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T16:18:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 20 client.7701 populate_metadata read hostname 'smithi053' 2023-10-21T16:18:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a270e0 0x563242a274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 4 client.7701 mds_command: new command op to 35915 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:18:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.401+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 --> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56324295a700 con 0x563242a270e0 2023-10-21T16:18:51.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.402+0000 7f22eeffd640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a270e0 0x563242a274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.403+0000 7f22eeffd640 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a270e0 0x563242a274c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8ad39b4a6d3029 server_cookie=8e7a5caf4ccef6ec in_seq=0 out_seq=0 2023-10-21T16:18:51.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.403+0000 7f22ce7fc640 10 client.7701 ms_handle_connect on v2:172.21.15.124:6835/3805349006 2023-10-21T16:18:51.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.403+0000 7f22ce7fc640 1 -- 172.21.15.53:0/3784391366 <== mds.0 v2:172.21.15.124:6835/3805349006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x56324295a700 con 0x563242a270e0 2023-10-21T16:18:51.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.403+0000 7f22ce7fc640 10 client.7701 handle_command_reply: tid=0 2023-10-21T16:18:51.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.403+0000 7f22f85ed740 1 client.7701 shutdown 2023-10-21T16:18:51.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 msgr2=0x563242a2a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f22e8007160 tx=0x7f22e8005b70 comp rx=0 tx=0).stop 2023-10-21T16:18:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 shutdown_connections 2023-10-21T16:18:51.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:6835/3805349006,v1:172.21.15.124:6837/3805349006] conn(0x563242a270e0 0x563242a274c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563242a51800 0x563242a2a020 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563242a23550 0x563242a29ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 --2- 172.21.15.53:0/3784391366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563242a23170 0x5632429fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 >> 172.21.15.53:0/3784391366 conn(0x5632429f65b0 msgr2=0x5632429f1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22ce7fc640 0 client.7701 ms_handle_reset on v2:172.21.15.124:6835/3805349006 2023-10-21T16:18:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 shutdown_connections 2023-10-21T16:18:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 1 -- 172.21.15.53:0/3784391366 wait complete. 2023-10-21T16:18:51.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.404+0000 7f22f85ed740 20 client.7701 trim_cache size 0 max 16384 2023-10-21T16:18:51.422 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:18:51.422 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:18:51.422 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:18:51.422 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 35915, 2023-10-21T16:18:51.422 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:18:51.423 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:18:51.423 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:18:51.423 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.9266066940000002, 2023-10-21T16:18:51.424 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 53, 2023-10-21T16:18:51.424 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 117, 2023-10-21T16:18:51.424 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 117, 2023-10-21T16:18:51.424 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.270425380000001 2023-10-21T16:18:51.425 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:18:51.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 msgr2=0x7f22d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 0x7f22d006d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000f60 tx=0x7f22e0009040 comp rx=0 tx=0).stop 2023-10-21T16:18:51.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 msgr2=0x7f22f00b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000c00 tx=0x7f22d80046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 shutdown_connections 2023-10-21T16:18:51.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f22d006af00 0x7f22d006d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22f00cf690 0x7f22f009e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f00cf2b0 0x7f22f00b3b60 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 --2- 172.21.15.53:0/411320955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22f00ce510 0x7f22f009cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 >> 172.21.15.53:0/411320955 conn(0x7f22f009bc40 msgr2=0x7f22f015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 shutdown_connections 2023-10-21T16:18:51.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.405+0000 7f22effff640 1 -- 172.21.15.53:0/411320955 wait complete. 2023-10-21T16:18:51.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:18:51.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.684+0000 7f0cfa621640 1 Processor -- start 2023-10-21T16:18:51.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.685+0000 7f0cfa621640 1 -- start start 2023-10-21T16:18:51.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cfa621640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cf415a970 0x7f0cf40cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cfa621640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf406cfd0 con 0x7f0cf406dd00 2023-10-21T16:18:51.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cfa621640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf406d870 con 0x7f0cf40cb990 2023-10-21T16:18:51.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cfa621640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40cbe00 con 0x7f0cf415a970 2023-10-21T16:18:51.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cf961f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cf415a970 0x7f0cf40cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cf961f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cf415a970 0x7f0cf40cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47524/0 (socket says 172.21.15.53:47524) 2023-10-21T16:18:51.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.686+0000 7f0cf961f640 1 -- 172.21.15.53:0/1852121035 learned_addr learned my addr 172.21.15.53:0/1852121035 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.687+0000 7f0cf961f640 1 -- 172.21.15.53:0/1852121035 >> v1:172.21.15.124:6789/0 conn(0x7f0cf40cb990 legacy=0x7f0cf40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:51.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.687+0000 7f0cf961f640 1 -- 172.21.15.53:0/1852121035 >> v1:172.21.15.53:6789/0 conn(0x7f0cf406dd00 legacy=0x7f0cf40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.687+0000 7f0ce7fff640 1 -- 172.21.15.53:0/1852121035 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387113042 0 0) 0x7f0cf406cfd0 con 0x7f0cf406dd00 2023-10-21T16:18:51.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.687+0000 7f0cf961f640 1 -- 172.21.15.53:0/1852121035 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf415ad50 con 0x7f0cf415a970 2023-10-21T16:18:51.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.688+0000 7f0cf961f640 1 --2- 172.21.15.53:0/1852121035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf415a970 0x7f0cf40cb450 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387f3d28f114661a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0ce7fff640 1 -- 172.21.15.53:0/1852121035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00317e0 con 0x7f0cf415a970 2023-10-21T16:18:51.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0ce7fff640 1 -- 172.21.15.53:0/1852121035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0031940 con 0x7f0cf415a970 2023-10-21T16:18:51.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0ce7fff640 1 -- 172.21.15.53:0/1852121035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0031cd0 con 0x7f0cf415a970 2023-10-21T16:18:51.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 -- 172.21.15.53:0/1852121035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf415a970 msgr2=0x7f0cf40cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 --2- 172.21.15.53:0/1852121035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf415a970 0x7f0cf40cb450 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b880 comp rx=0 tx=0).stop 2023-10-21T16:18:51.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 -- 172.21.15.53:0/1852121035 shutdown_connections 2023-10-21T16:18:51.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 --2- 172.21.15.53:0/1852121035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf415a970 0x7f0cf40cb450 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 -- 172.21.15.53:0/1852121035 >> 172.21.15.53:0/1852121035 conn(0x7f0cf40ce9d0 msgr2=0x7f0cf40cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 -- 172.21.15.53:0/1852121035 shutdown_connections 2023-10-21T16:18:51.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.689+0000 7f0cfa621640 1 -- 172.21.15.53:0/1852121035 wait complete. 2023-10-21T16:18:51.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.691+0000 7f0cfa621640 1 Processor -- start 2023-10-21T16:18:51.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.691+0000 7f0cfa621640 1 -- start start 2023-10-21T16:18:51.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cf40cb990 0x7f0cf40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 0x7f0cf40c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41537c0 con 0x7f0cf406dd00 2023-10-21T16:18:51.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c82c0 con 0x7f0cf415a970 2023-10-21T16:18:51.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cfa621640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf40c7990 con 0x7f0cf40cb990 2023-10-21T16:18:51.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.692+0000 7f0cf961f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47528/0 (socket says 172.21.15.53:47528) 2023-10-21T16:18:51.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 -- 172.21.15.53:0/3425076464 learned_addr learned my addr 172.21.15.53:0/3425076464 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:51.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf8e1e640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cf40cb990 0x7f0cf40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf9e20640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 0x7f0cf40c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 -- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cf40cb990 msgr2=0x7f0cf40b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cf40cb990 0x7f0cf40b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 -- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 msgr2=0x7f0cf40c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 0x7f0cf40c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 -- 172.21.15.53:0/3425076464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00094d0 con 0x7f0cf406dd00 2023-10-21T16:18:51.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf9e20640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 0x7f0cf40c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:51.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.693+0000 7f0cf961f640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.694+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0032880 con 0x7f0cf406dd00 2023-10-21T16:18:51.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.694+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf00329e0 con 0x7f0cf406dd00 2023-10-21T16:18:51.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.694+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0032d70 con 0x7f0cf406dd00 2023-10-21T16:18:51.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.694+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf40c7c70 con 0x7f0cf406dd00 2023-10-21T16:18:51.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.694+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf40b0ba0 con 0x7f0cf406dd00 2023-10-21T16:18:51.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.695+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf404e160 con 0x7f0cf406dd00 2023-10-21T16:18:51.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.696+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0cf004d3f0 con 0x7f0cf406dd00 2023-10-21T16:18:51.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.697+0000 7f0ce5ffb640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 0x7f0cd806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:51.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.698+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0cf00ab890 con 0x7f0cf406dd00 2023-10-21T16:18:51.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.698+0000 7f0cf8e1e640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 0x7f0cd806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:51.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.699+0000 7f0cf8e1e640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 0x7f0cd806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cf41f3230 tx=0x7f0ce8009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:51.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.702+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cf00765f0 con 0x7f0cf406dd00 2023-10-21T16:18:51.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.929+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0cf4071090 con 0x7f0cf406dd00 2023-10-21T16:18:51.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.930+0000 7f0ce5ffb640 1 -- 172.21.15.53:0/3425076464 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0cf0079c10 con 0x7f0cf406dd00 2023-10-21T16:18:51.932 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:18:51.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 msgr2=0x7f0cd806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 0x7f0cd806d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0cf41f3230 tx=0x7f0ce8009040 comp rx=0 tx=0).stop 2023-10-21T16:18:51.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 msgr2=0x7f0cf4153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:51.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf00046c0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 shutdown_connections 2023-10-21T16:18:51.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f0cd806af30 0x7f0cd806d3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cf415a970 0x7f0cf40c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cf40cb990 0x7f0cf40b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 --2- 172.21.15.53:0/3425076464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cf406dd00 0x7f0cf4153280 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:51.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 >> 172.21.15.53:0/3425076464 conn(0x7f0cf40ce9d0 msgr2=0x7f0cf40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:51.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 shutdown_connections 2023-10-21T16:18:51.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:51.933+0000 7f0cfa621640 1 -- 172.21.15.53:0/3425076464 wait complete. 2023-10-21T16:18:51.951 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:18:51.952 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:18:51.952 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:18:51.953 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:18:51.953 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:51.953 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:18:51.953 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:18:51.954 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:18:51.954 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:51.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:51.955 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:18:52.008 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:18:52.008 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:18:52.008 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:18:52.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.009 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:18:52.009 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.009 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:52.009 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:52.010 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:18:52.010 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:52.010 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:52.010 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:18:52.010 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:18:52.011 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:52.011 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:18:52.011 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:18:52.011 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41361sec preferred_lft 41361sec 2023-10-21T16:18:52.011 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:18:52.012 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.012 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:18:52.012 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:52.013 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:18:52.013 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:18:52.014 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:18:52.014 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:18:52.014 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.014 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:18:52.014 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.015 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:18:52.015 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:18:52.015 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:18:52.015 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.016 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:18:52.016 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:18:52.016 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:18:52.016 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:18:52.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.017 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:18:52.111 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:18:52.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.116 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:18:52.175 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:18:52.176 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:18:52.176 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:18:52.176 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:18:52.176 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:18:52.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.177 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:18:52.178 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:18:52.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:18:52.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:18:52.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:18:52.179 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:18:52.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.369 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:18:52.422 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:18:52.422 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:18:52.423 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:52.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:18:52.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:18:52.525 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:52.576 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:18:52.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:18:52.577 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:52.625 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:18:52.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:52.768 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:18:52.804 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:18:52.804 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:18:52.804 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:18:52.804 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:18:52.805 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:18:52.807 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:18:52.807 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:18:52.807 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:18:52.808 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:18:52.808 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:18:52.812 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:18:52.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:18:52.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:18:52.904 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:18:52.982 INFO:teuthology.orchestra.run.smithi053.stdout:client7719 2023-10-21T16:18:52.983 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:18:52.983 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:18:53.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:18:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.276+0000 7f5df44f7640 1 Processor -- start 2023-10-21T16:18:53.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.276+0000 7f5df44f7640 1 -- start start 2023-10-21T16:18:53.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.277+0000 7f5df44f7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0afbe0 0x7f5dec0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.277+0000 7f5df44f7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5dec0cb130 0x7f5dec0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.277+0000 7f5df44f7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec0b2620 con 0x7f5dec06dfc0 2023-10-21T16:18:53.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.277+0000 7f5df44f7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec0cbae0 con 0x7f5dec0cb130 2023-10-21T16:18:53.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.277+0000 7f5df44f7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0b2060 con 0x7f5dec0afbe0 2023-10-21T16:18:53.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1a92640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5dec06dfc0 0x7f5dec070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34658/0 (socket says 172.21.15.53:34658) 2023-10-21T16:18:53.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1a92640 1 -- 172.21.15.53:0/3402796072 learned_addr learned my addr 172.21.15.53:0/3402796072 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:53.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0afbe0 0x7f5dec0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df2a94640 1 --2- 172.21.15.53:0/3402796072 >> v2:172.21.15.124:3300/0 conn(0x7f5dec0cb130 0x7f5dec0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638402223 0 0) 0x7f5dec0b2620 con 0x7f5dec06dfc0 2023-10-21T16:18:53.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ddc003520 con 0x7f5dec06dfc0 2023-10-21T16:18:53.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809866863 0 0) 0x7f5ddc003520 con 0x7f5dec06dfc0 2023-10-21T16:18:53.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0afbe0 msgr2=0x7f5dec0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0afbe0 0x7f5dec0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 >> v2:172.21.15.124:3300/0 conn(0x7f5dec0cb130 msgr2=0x7f5dec0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 --2- 172.21.15.53:0/3402796072 >> v2:172.21.15.124:3300/0 conn(0x7f5dec0cb130 0x7f5dec0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0b2340 con 0x7f5dec06dfc0 2023-10-21T16:18:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.278+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5dd8001670 con 0x7f5dec06dfc0 2023-10-21T16:18:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 >> v1:172.21.15.53:6789/0 conn(0x7f5dec06dfc0 legacy=0x7f5dec070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5ddc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ddc0047c0 0x7f5ddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ddc005180 0x7f5ddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0cbae0 con 0x7f5ddc003970 2023-10-21T16:18:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0b2060 con 0x7f5ddc0047c0 2023-10-21T16:18:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc00bec0 con 0x7f5ddc005180 2023-10-21T16:18:53.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0afbe0 0x7f5dec0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:18:53.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5ddc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df1a92640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ddc0047c0 0x7f5ddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2a94640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ddc005180 0x7f5ddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 -- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ddc005180 msgr2=0x7f5ddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ddc005180 0x7f5ddc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 -- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ddc0047c0 msgr2=0x7f5ddc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ddc0047c0 0x7f5ddc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.279+0000 7f5df2293640 1 -- 172.21.15.53:0/3402796072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc003520 con 0x7f5ddc003970 2023-10-21T16:18:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df2293640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5ddc003d90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e1f0 tx=0x7f5ddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f1331a1ce27b5a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de002cce0 con 0x7f5ddc003970 2023-10-21T16:18:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de000f040 con 0x7f5ddc003970 2023-10-21T16:18:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df1291640 1 -- 172.21.15.53:0/3402796072 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0003bb0 con 0x7f5ddc003970 2023-10-21T16:18:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df44f7640 1 -- 172.21.15.53:0/3402796072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 msgr2=0x7f5ddc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3402796072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5ddc003d90 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5de000e1f0 tx=0x7f5ddc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.281+0000 7f5df2a94640 1 -- 172.21.15.53:0/3402796072 reap_dead start 2023-10-21T16:18:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.282+0000 7f5df44f7640 1 -- 172.21.15.53:0/3402796072 shutdown_connections 2023-10-21T16:18:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.282+0000 7f5df44f7640 1 -- 172.21.15.53:0/3402796072 >> 172.21.15.53:0/3402796072 conn(0x7f5dec06a5d0 msgr2=0x7f5dec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.282+0000 7f5df44f7640 1 -- 172.21.15.53:0/3402796072 shutdown_connections 2023-10-21T16:18:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.282+0000 7f5df44f7640 1 -- 172.21.15.53:0/3402796072 wait complete. 2023-10-21T16:18:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.284+0000 7f5df44f7640 1 Processor -- start 2023-10-21T16:18:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.284+0000 7f5df44f7640 1 -- start start 2023-10-21T16:18:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dec1f33f0 0x7f5dec0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0c4790 0x7f5dec0c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0c5150 con 0x7f5ddc003970 2023-10-21T16:18:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0c92d0 con 0x7f5dec1f33f0 2023-10-21T16:18:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df44f7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0c9570 con 0x7f5dec0c4790 2023-10-21T16:18:53.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.285+0000 7f5df2293640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47550/0 (socket says 172.21.15.53:47550) 2023-10-21T16:18:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 -- 172.21.15.53:0/3847217518 learned_addr learned my addr 172.21.15.53:0/3847217518 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2a94640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0c4790 0x7f5dec0c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df1a92640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dec1f33f0 0x7f5dec0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 -- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0c4790 msgr2=0x7f5dec0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0c4790 0x7f5dec0c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 -- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dec1f33f0 msgr2=0x7f5dec0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dec1f33f0 0x7f5dec0c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 -- 172.21.15.53:0/3847217518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0c9850 con 0x7f5ddc003970 2023-10-21T16:18:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.286+0000 7f5df2293640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5de002c640 tx=0x7f5de0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.287+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de002cdb0 con 0x7f5ddc003970 2023-10-21T16:18:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.287+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0004130 con 0x7f5ddc003970 2023-10-21T16:18:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.287+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de00044e0 con 0x7f5ddc003970 2023-10-21T16:18:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.287+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec0c9b30 con 0x7f5ddc003970 2023-10-21T16:18:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.287+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec0ca0f0 con 0x7f5ddc003970 2023-10-21T16:18:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.288+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec1f37d0 con 0x7f5ddc003970 2023-10-21T16:18:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.289+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5de0004640 con 0x7f5ddc003970 2023-10-21T16:18:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.290+0000 7f5deaffd640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 0x7f5dd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.290+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f5de00ac640 con 0x7f5ddc003970 2023-10-21T16:18:53.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.291+0000 7f5df1a92640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 0x7f5dd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.292+0000 7f5df1a92640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 0x7f5dd006d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8001eb0 tx=0x7f5dd8009040 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.294+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dec1f37d0 con 0x7f5ddc003970 2023-10-21T16:18:53.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.516+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 --> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dec070630 con 0x7f5dd006af00 2023-10-21T16:18:53.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.521+0000 7f5deaffd640 1 -- 172.21.15.53:0/3847217518 <== mgr.35759 v2:172.21.15.124:6832/1967358544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5dd8001650 con 0x7f5dd006af00 2023-10-21T16:18:53.522 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:18:53.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:18:53.523 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:18:53.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:18:53.523 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:18:53.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.523+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 msgr2=0x7f5dd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.523+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 0x7f5dd006d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8001eb0 tx=0x7f5dd8009040 comp rx=0 tx=0).stop 2023-10-21T16:18:53.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.523+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 msgr2=0x7f5dec1f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.523+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5de002c640 tx=0x7f5de0009750 comp rx=0 tx=0).stop 2023-10-21T16:18:53.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 shutdown_connections 2023-10-21T16:18:53.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f5dd006af00 0x7f5dd006d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dec0c4790 0x7f5dec0c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dec1f33f0 0x7f5dec0c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 --2- 172.21.15.53:0/3847217518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ddc003970 0x7f5dec1f27d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 >> 172.21.15.53:0/3847217518 conn(0x7f5dec06a5d0 msgr2=0x7f5dec15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:53.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 shutdown_connections 2023-10-21T16:18:53.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.524+0000 7f5df44f7640 1 -- 172.21.15.53:0/3847217518 wait complete. 2023-10-21T16:18:53.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:18:53.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.773+0000 7f6062aa2640 1 Processor -- start 2023-10-21T16:18:53.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.773+0000 7f6062aa2640 1 -- start start 2023-10-21T16:18:53.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.774+0000 7f6062aa2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f605c0afbe0 0x7f605c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.774+0000 7f6062aa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f605c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.774+0000 7f6062aa2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c0cbad0 con 0x7f605c06dfc0 2023-10-21T16:18:53.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.774+0000 7f6062aa2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c0b2620 con 0x7f605c0afbe0 2023-10-21T16:18:53.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.774+0000 7f6062aa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c0b2060 con 0x7f605c0cb130 2023-10-21T16:18:53.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.775+0000 7f605bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f605c06dfc0 0x7f605c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:34674/0 (socket says 172.21.15.53:34674) 2023-10-21T16:18:53.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.775+0000 7f605bfff640 1 -- 172.21.15.53:0/213440906 learned_addr learned my addr 172.21.15.53:0/213440906 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:53.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.775+0000 7f606103f640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f605c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.775+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765716074 0 0) 0x7f605c0cbad0 con 0x7f605c06dfc0 2023-10-21T16:18:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.775+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f604c003520 con 0x7f605c06dfc0 2023-10-21T16:18:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f606083e640 1 --2- 172.21.15.53:0/213440906 >> v2:172.21.15.124:3300/0 conn(0x7f605c0afbe0 0x7f605c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974189435 0 0) 0x7f604c003520 con 0x7f605c06dfc0 2023-10-21T16:18:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 msgr2=0x7f605c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f605c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 >> v2:172.21.15.124:3300/0 conn(0x7f605c0afbe0 msgr2=0x7f605c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 --2- 172.21.15.53:0/213440906 >> v2:172.21.15.124:3300/0 conn(0x7f605c0afbe0 0x7f605c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0b2340 con 0x7f605c06dfc0 2023-10-21T16:18:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6048001670 con 0x7f605c06dfc0 2023-10-21T16:18:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 >> v1:172.21.15.53:6789/0 conn(0x7f605c06dfc0 legacy=0x7f605c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f604c003970 0x7f604c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f604c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f604c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c0b2620 con 0x7f604c004d00 2023-10-21T16:18:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c0b2060 con 0x7f604c003970 2023-10-21T16:18:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.776+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c00b9b0 con 0x7f605c0cb130 2023-10-21T16:18:53.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f604c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606083e640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f604c003970 0x7f604c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f605bfff640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f604c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 -- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 msgr2=0x7f604c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c0cb130 0x7f604c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 -- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f604c003970 msgr2=0x7f604c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f604c003970 0x7f604c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.777+0000 7f606103f640 1 -- 172.21.15.53:0/213440906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c003520 con 0x7f604c004d00 2023-10-21T16:18:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.778+0000 7f606103f640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f604c005180 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6054009fc0 tx=0x7f604c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2cf3484f7fcf68 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.778+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605402cc20 con 0x7f604c004d00 2023-10-21T16:18:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.778+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605400f040 con 0x7f604c004d00 2023-10-21T16:18:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.778+0000 7f605b7fe640 1 -- 172.21.15.53:0/213440906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054034440 con 0x7f604c004d00 2023-10-21T16:18:53.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 -- 172.21.15.53:0/213440906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 msgr2=0x7f604c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 --2- 172.21.15.53:0/213440906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f604c005180 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6054009fc0 tx=0x7f604c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f606103f640 1 -- 172.21.15.53:0/213440906 reap_dead start 2023-10-21T16:18:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 -- 172.21.15.53:0/213440906 shutdown_connections 2023-10-21T16:18:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 -- 172.21.15.53:0/213440906 >> 172.21.15.53:0/213440906 conn(0x7f605c06a5d0 msgr2=0x7f605c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:53.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 -- 172.21.15.53:0/213440906 shutdown_connections 2023-10-21T16:18:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.779+0000 7f6062aa2640 1 -- 172.21.15.53:0/213440906 wait complete. 2023-10-21T16:18:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.781+0000 7f6062aa2640 1 Processor -- start 2023-10-21T16:18:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.781+0000 7f6062aa2640 1 -- start start 2023-10-21T16:18:53.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f605c14e960 0x7f605c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 0x7f605c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1501f0 con 0x7f604c004d00 2023-10-21T16:18:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1f6a60 con 0x7f605c14e960 2023-10-21T16:18:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f6062aa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c14ee40 con 0x7f605c1f2090 2023-10-21T16:18:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f606083e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f606083e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47572/0 (socket says 172.21.15.53:47572) 2023-10-21T16:18:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.782+0000 7f606083e640 1 -- 172.21.15.53:0/30274953 learned_addr learned my addr 172.21.15.53:0/30274953 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606103f640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 0x7f605c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f605bfff640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f605c14e960 0x7f605c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 -- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 msgr2=0x7f605c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 0x7f605c06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 -- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f605c14e960 msgr2=0x7f605c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f605c14e960 0x7f605c1f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 -- 172.21.15.53:0/30274953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c14f120 con 0x7f604c004d00 2023-10-21T16:18:53.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606103f640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 0x7f605c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:53.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.783+0000 7f606083e640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f605000f5a0 tx=0x7f605000fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.784+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050014870 con 0x7f604c004d00 2023-10-21T16:18:53.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.784+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60500149d0 con 0x7f604c004d00 2023-10-21T16:18:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.784+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050014d40 con 0x7f604c004d00 2023-10-21T16:18:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.784+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c14f400 con 0x7f604c004d00 2023-10-21T16:18:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.784+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c14f9c0 con 0x7f604c004d00 2023-10-21T16:18:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.785+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c1f24a0 con 0x7f604c004d00 2023-10-21T16:18:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.786+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6050015410 con 0x7f604c004d00 2023-10-21T16:18:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.787+0000 7f60597fa640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 0x7f604006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.788+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f605008a1b0 con 0x7f604c004d00 2023-10-21T16:18:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.788+0000 7f605bfff640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 0x7f604006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.789+0000 7f605bfff640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 0x7f604006d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6048001050 tx=0x7f6048000fe0 comp rx=0 tx=0).ready entity=mgr.35759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:53.792+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f605c1f24a0 con 0x7f604c004d00 2023-10-21T16:18:54.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.036+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f605c1f24a0 con 0x7f604c004d00 2023-10-21T16:18:54.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.356+0000 7f60597fa640 1 -- 172.21.15.53:0/30274953 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v45) v1 ==== 92+0+0 (secure 0 0 0) 0x7f605c1f24a0 con 0x7f604c004d00 2023-10-21T16:18:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.359+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 msgr2=0x7f604006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.359+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 0x7f604006d3c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6048001050 tx=0x7f6048000fe0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.359+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 msgr2=0x7f605c1f6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.359+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f605000f5a0 tx=0x7f605000fa60 comp rx=0 tx=0).stop 2023-10-21T16:18:54.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 shutdown_connections 2023-10-21T16:18:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f604006af00 0x7f604006d3c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f605c1f2090 0x7f605c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f605c14e960 0x7f605c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 --2- 172.21.15.53:0/30274953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f604c004d00 0x7f605c1f6520 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.361+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 >> 172.21.15.53:0/30274953 conn(0x7f605c06a5d0 msgr2=0x7f605c0cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:54.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.362+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 shutdown_connections 2023-10-21T16:18:54.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.363+0000 7f6062aa2640 1 -- 172.21.15.53:0/30274953 wait complete. 2023-10-21T16:18:54.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:18:54.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.595+0000 7f15f0ecf640 1 Processor -- start 2023-10-21T16:18:54.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.595+0000 7f15f0ecf640 1 -- start start 2023-10-21T16:18:54.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f15ec06c3a0 0x7f15ec0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15ec06db60 0x7f15ec06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec15aa00 0x7f15ec0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec0ca230 con 0x7f15ec06db60 2023-10-21T16:18:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ec15b310 con 0x7f15ec06c3a0 2023-10-21T16:18:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15f0ecf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec06d290 con 0x7f15ec15aa00 2023-10-21T16:18:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15ec06db60 0x7f15ec06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15ec06db60 0x7f15ec06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43760/0 (socket says 172.21.15.53:43760) 2023-10-21T16:18:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15e3fff640 1 -- 172.21.15.53:0/2149515452 learned_addr learned my addr 172.21.15.53:0/2149515452 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15ea59c640 1 --2- 172.21.15.53:0/2149515452 >> v2:172.21.15.124:3300/0 conn(0x7f15ec06c3a0 0x7f15ec0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.597+0000 7f15ead9d640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec15aa00 0x7f15ec0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.598+0000 7f15e3fff640 1 -- 172.21.15.53:0/2149515452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec15aa00 msgr2=0x7f15ec0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.598+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec15aa00 0x7f15ec0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.598+0000 7f15e3fff640 1 -- 172.21.15.53:0/2149515452 >> v2:172.21.15.124:3300/0 conn(0x7f15ec06c3a0 msgr2=0x7f15ec0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.598+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2149515452 >> v2:172.21.15.124:3300/0 conn(0x7f15ec06c3a0 0x7f15ec0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.598+0000 7f15e3fff640 1 -- 172.21.15.53:0/2149515452 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0c9860 con 0x7f15ec06db60 2023-10-21T16:18:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.599+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06db60 0x7f15ec06c780 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0716b0 tx=0x7f15d402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78484cbfc431055 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:54.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.599+0000 7f15e9d9b640 1 -- 172.21.15.53:0/2149515452 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d402f8e0 con 0x7f15ec06db60 2023-10-21T16:18:54.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.599+0000 7f15e9d9b640 1 -- 172.21.15.53:0/2149515452 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d402fa40 con 0x7f15ec06db60 2023-10-21T16:18:54.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.599+0000 7f15e9d9b640 1 -- 172.21.15.53:0/2149515452 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d402fdd0 con 0x7f15ec06db60 2023-10-21T16:18:54.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2149515452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06db60 msgr2=0x7f15ec06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06db60 0x7f15ec06c780 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0716b0 tx=0x7f15d402db20 comp rx=0 tx=0).stop 2023-10-21T16:18:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2149515452 shutdown_connections 2023-10-21T16:18:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec15aa00 0x7f15ec0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2149515452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06db60 0x7f15ec06c780 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2149515452 >> v2:172.21.15.124:3300/0 conn(0x7f15ec06c3a0 0x7f15ec0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2149515452 >> 172.21.15.53:0/2149515452 conn(0x7f15ec06a5d0 msgr2=0x7f15ec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2149515452 shutdown_connections 2023-10-21T16:18:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.600+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2149515452 wait complete. 2023-10-21T16:18:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.602+0000 7f15f0ecf640 1 Processor -- start 2023-10-21T16:18:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.602+0000 7f15f0ecf640 1 -- start start 2023-10-21T16:18:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec06db60 0x7f15ec0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 0x7f15ec14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0c5170 con 0x7f15ec06c3a0 2023-10-21T16:18:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec14f090 con 0x7f15ec15aa00 2023-10-21T16:18:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15f0ecf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec15dd10 con 0x7f15ec06db60 2023-10-21T16:18:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.603+0000 7f15ea59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43762/0 (socket says 172.21.15.53:43762) 2023-10-21T16:18:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 -- 172.21.15.53:0/2968735564 learned_addr learned my addr 172.21.15.53:0/2968735564 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ead9d640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 0x7f15ec14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec06db60 0x7f15ec0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec06db60 msgr2=0x7f15ec0c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec06db60 0x7f15ec0c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 msgr2=0x7f15ec14eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 0x7f15ec14eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 -- 172.21.15.53:0/2968735564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40094d0 con 0x7f15ec06c3a0 2023-10-21T16:18:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ead9d640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 0x7f15ec14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.604+0000 7f15ea59c640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d800 tx=0x7f15dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.605+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0096a0 con 0x7f15ec06c3a0 2023-10-21T16:18:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.605+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc009800 con 0x7f15ec06c3a0 2023-10-21T16:18:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.605+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc009b90 con 0x7f15ec06c3a0 2023-10-21T16:18:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.605+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec15dff0 con 0x7f15ec06c3a0 2023-10-21T16:18:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.605+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec15e5b0 con 0x7f15ec06c3a0 2023-10-21T16:18:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.606+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec04e160 con 0x7f15ec06c3a0 2023-10-21T16:18:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.607+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 45) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f15dc009cf0 con 0x7f15ec06c3a0 2023-10-21T16:18:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.609+0000 7f15e2ffd640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 0x7f15cc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.609+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f15dc089e50 con 0x7f15ec06c3a0 2023-10-21T16:18:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.609+0000 7f15e3fff640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 msgr2=0x7f15cc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.609+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 0x7f15cc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.614+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15dc054c50 con 0x7f15ec06c3a0 2023-10-21T16:18:54.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.807+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f15ec0c2440 con 0x7f15ec06c3a0 2023-10-21T16:18:54.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.810+0000 7f15e3fff640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 msgr2=0x7f15cc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:54.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.810+0000 7f15e3fff640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 0x7f15cc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:54.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.812+0000 7f15e2ffd640 1 -- 172.21.15.53:0/2968735564 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15dc00b440 con 0x7f15ec06c3a0 2023-10-21T16:18:54.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.814+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 msgr2=0x7f15cc06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:54.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.814+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 0x7f15cc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.814+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 msgr2=0x7f15ec15d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:54.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.814+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d800 tx=0x7f15dc00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 shutdown_connections 2023-10-21T16:18:54.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f15cc06afe0 0x7f15cc06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15ec15aa00 0x7f15ec14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15ec06db60 0x7f15ec0c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 --2- 172.21.15.53:0/2968735564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15ec06c3a0 0x7f15ec15d680 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 >> 172.21.15.53:0/2968735564 conn(0x7f15ec06a5d0 msgr2=0x7f15ec0ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 shutdown_connections 2023-10-21T16:18:54.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:54.815+0000 7f15f0ecf640 1 -- 172.21.15.53:0/2968735564 wait complete. 2023-10-21T16:18:54.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:18:54.874 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:54.872+0000 7f4c878b6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:54.890 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:54.888+0000 7ffaede4c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:18:54.926 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:54.924+0000 7f4c878b6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:54.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:54.945+0000 7ffaede4c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:18:55.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.040+0000 7f07c3905640 1 Processor -- start 2023-10-21T16:18:55.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.041+0000 7f07c3905640 1 -- start start 2023-10-21T16:18:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.042+0000 7f07c3905640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07bc0b4e10 0x7f07bc1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.042+0000 7f07c3905640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cd4e0 0x7f07bc0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.042+0000 7f07c3905640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc0b8c30 con 0x7f07bc15cd70 2023-10-21T16:18:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.042+0000 7f07c3905640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0cdd80 con 0x7f07bc0b4e10 2023-10-21T16:18:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.042+0000 7f07c3905640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0ce060 con 0x7f07bc0cd4e0 2023-10-21T16:18:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c0ea0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f07bc0b4e10 0x7f07bc1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c16a1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f07bc15cd70 0x7f07bc0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60398/0 (socket says 172.21.15.53:60398) 2023-10-21T16:18:55.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c16a1640 1 -- 172.21.15.53:0/960205509 learned_addr learned my addr 172.21.15.53:0/960205509 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:55.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c1ea2640 1 --2- 172.21.15.53:0/960205509 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cd4e0 0x7f07bc0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c0ea0640 1 -- 172.21.15.53:0/960205509 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cd4e0 msgr2=0x7f07bc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:55.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c0ea0640 1 --2- 172.21.15.53:0/960205509 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cd4e0 0x7f07bc0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c0ea0640 1 -- 172.21.15.53:0/960205509 >> v1:172.21.15.53:6789/0 conn(0x7f07bc15cd70 legacy=0x7f07bc0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.043+0000 7f07c0ea0640 1 -- 172.21.15.53:0/960205509 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0ce340 con 0x7f07bc0b4e10 2023-10-21T16:18:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.044+0000 7f07c0ea0640 1 --2- 172.21.15.53:0/960205509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc0b4e10 0x7f07bc1612f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0d06c0 tx=0x7f07ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8215934ea829d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.044+0000 7f07b3fff640 1 -- 172.21.15.53:0/960205509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac031830 con 0x7f07bc0b4e10 2023-10-21T16:18:55.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.044+0000 7f07b3fff640 1 -- 172.21.15.53:0/960205509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac031990 con 0x7f07bc0b4e10 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.044+0000 7f07b3fff640 1 -- 172.21.15.53:0/960205509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac031d20 con 0x7f07bc0b4e10 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 -- 172.21.15.53:0/960205509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc0b4e10 msgr2=0x7f07bc1612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 --2- 172.21.15.53:0/960205509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc0b4e10 0x7f07bc1612f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0d06c0 tx=0x7f07ac02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 -- 172.21.15.53:0/960205509 shutdown_connections 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 --2- 172.21.15.53:0/960205509 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cd4e0 0x7f07bc0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 --2- 172.21.15.53:0/960205509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc0b4e10 0x7f07bc1612f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 -- 172.21.15.53:0/960205509 >> 172.21.15.53:0/960205509 conn(0x7f07bc0cf700 msgr2=0x7f07bc0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 -- 172.21.15.53:0/960205509 shutdown_connections 2023-10-21T16:18:55.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.045+0000 7f07c3905640 1 -- 172.21.15.53:0/960205509 wait complete. 2023-10-21T16:18:55.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.047+0000 7f07c3905640 1 Processor -- start 2023-10-21T16:18:55.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.047+0000 7f07c3905640 1 -- start start 2023-10-21T16:18:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 0x7f07bc0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc0cd4e0 0x7f07bc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1e7200 con 0x7f07bc15cd70 2023-10-21T16:18:55.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1e6ed0 con 0x7f07bc0cd4e0 2023-10-21T16:18:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c3905640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc15f4e0 con 0x7f07bc0b4e10 2023-10-21T16:18:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c1ea2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c1ea2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43778/0 (socket says 172.21.15.53:43778) 2023-10-21T16:18:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.048+0000 7f07c1ea2640 1 -- 172.21.15.53:0/1004504690 learned_addr learned my addr 172.21.15.53:0/1004504690 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:55.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c0ea0640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc0cd4e0 0x7f07bc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c16a1640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 0x7f07bc0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 msgr2=0x7f07bc0c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 0x7f07bc0c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc0cd4e0 msgr2=0x7f07bc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc0cd4e0 0x7f07bc15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 -- 172.21.15.53:0/1004504690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac009400 con 0x7f07bc15cd70 2023-10-21T16:18:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c16a1640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 0x7f07bc0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c1ea2640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f07b4007d90 tx=0x7f07b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4014420 con 0x7f07bc15cd70 2023-10-21T16:18:55.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b40145a0 con 0x7f07bc15cd70 2023-10-21T16:18:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4014950 con 0x7f07bc15cd70 2023-10-21T16:18:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.049+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc15f7c0 con 0x7f07bc15cd70 2023-10-21T16:18:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.050+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc15f1d0 con 0x7f07bc15cd70 2023-10-21T16:18:55.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.051+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc15fce0 con 0x7f07bc15cd70 2023-10-21T16:18:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.051+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 45) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f07b4014ab0 con 0x7f07bc15cd70 2023-10-21T16:18:55.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.053+0000 7f07b1ffb640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 0x7f07a006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.053+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f07b408a390 con 0x7f07bc15cd70 2023-10-21T16:18:55.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.053+0000 7f07c16a1640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 msgr2=0x7f07a006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:55.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.053+0000 7f07c16a1640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 0x7f07a006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:55.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.057+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07bc15fce0 con 0x7f07bc15cd70 2023-10-21T16:18:55.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.254+0000 7f07c16a1640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 msgr2=0x7f07a006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:55.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.254+0000 7f07c16a1640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 0x7f07a006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:55.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.269+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f07bc15fce0 con 0x7f07bc15cd70 2023-10-21T16:18:55.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.274+0000 7f07b1ffb640 1 -- 172.21.15.53:0/1004504690 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7f07bc15fce0 con 0x7f07bc15cd70 2023-10-21T16:18:55.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 msgr2=0x7f07a006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:55.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 0x7f07a006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 msgr2=0x7f07bc158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f07b4007d90 tx=0x7f07b400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 shutdown_connections 2023-10-21T16:18:55.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7f07a006afe0 0x7f07a006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07bc15cd70 0x7f07bc158290 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc0cd4e0 0x7f07bc15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 --2- 172.21.15.53:0/1004504690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc0b4e10 0x7f07bc0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 >> 172.21.15.53:0/1004504690 conn(0x7f07bc0cf700 msgr2=0x7f07bc0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:55.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 shutdown_connections 2023-10-21T16:18:55.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.277+0000 7f07c3905640 1 -- 172.21.15.53:0/1004504690 wait complete. 2023-10-21T16:18:55.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2023-10-21T16:18:55.371 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.369+0000 7f4c878b6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:55.425 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.422+0000 7f4c878b6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:55.439 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.438+0000 7ffaede4c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:18:55.474 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.472+0000 7f4c878b6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:55.494 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.492+0000 7ffaede4c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:18:55.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.512+0000 7fcd2e34e640 1 Processor -- start 2023-10-21T16:18:55.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.513+0000 7fcd2e34e640 1 -- start start 2023-10-21T16:18:55.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.514+0000 7fcd2e34e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd2815cd70 0x7fcd281612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.514+0000 7fcd2e34e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd280b8c30 con 0x7fcd280b4e10 2023-10-21T16:18:55.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.514+0000 7fcd2e34e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd280cdd80 con 0x7fcd280cd4e0 2023-10-21T16:18:55.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.514+0000 7fcd2e34e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd280ce060 con 0x7fcd2815cd70 2023-10-21T16:18:55.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd2815cd70 0x7fcd281612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd2815cd70 0x7fcd281612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43780/0 (socket says 172.21.15.53:43780) 2023-10-21T16:18:55.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 -- 172.21.15.53:0/3020151138 learned_addr learned my addr 172.21.15.53:0/3020151138 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:55.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 -- 172.21.15.53:0/3020151138 >> v1:172.21.15.124:6789/0 conn(0x7fcd280cd4e0 legacy=0x7fcd280b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 -- 172.21.15.53:0/3020151138 >> v1:172.21.15.53:6789/0 conn(0x7fcd280b4e10 legacy=0x7fcd280d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.515+0000 7fcd27fff640 1 -- 172.21.15.53:0/3020151138 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd280ce340 con 0x7fcd2815cd70 2023-10-21T16:18:55.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.517+0000 7fcd27fff640 1 --2- 172.21.15.53:0/3020151138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd281612f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009870 tx=0x7fcd1802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711d86734e57b429 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:55.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.517+0000 7fcd26ffd640 1 -- 172.21.15.53:0/3020151138 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1802f850 con 0x7fcd2815cd70 2023-10-21T16:18:55.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.517+0000 7fcd26ffd640 1 -- 172.21.15.53:0/3020151138 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1802f9b0 con 0x7fcd2815cd70 2023-10-21T16:18:55.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.517+0000 7fcd26ffd640 1 -- 172.21.15.53:0/3020151138 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1802fd40 con 0x7fcd2815cd70 2023-10-21T16:18:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 -- 172.21.15.53:0/3020151138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 msgr2=0x7fcd281612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 --2- 172.21.15.53:0/3020151138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd281612f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009870 tx=0x7fcd1802ba30 comp rx=0 tx=0).stop 2023-10-21T16:18:55.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 -- 172.21.15.53:0/3020151138 shutdown_connections 2023-10-21T16:18:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 --2- 172.21.15.53:0/3020151138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd281612f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 -- 172.21.15.53:0/3020151138 >> 172.21.15.53:0/3020151138 conn(0x7fcd280cf700 msgr2=0x7fcd280cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 -- 172.21.15.53:0/3020151138 shutdown_connections 2023-10-21T16:18:55.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.518+0000 7fcd2e34e640 1 -- 172.21.15.53:0/3020151138 wait complete. 2023-10-21T16:18:55.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.520+0000 7fcd2e34e640 1 Processor -- start 2023-10-21T16:18:55.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.520+0000 7fcd2e34e640 1 -- start start 2023-10-21T16:18:55.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd280b4e10 0x7fcd2815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd280cd4e0 0x7fcd281f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd2814a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd281e7fd0 con 0x7fcd2815cd70 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd281f21b0 con 0x7fcd280cd4e0 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2e34e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd280c7ff0 con 0x7fcd280b4e10 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2c8eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd2814a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2c8eb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd2814a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43784/0 (socket says 172.21.15.53:43784) 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.521+0000 7fcd2c8eb640 1 -- 172.21.15.53:0/1420224748 learned_addr learned my addr 172.21.15.53:0/1420224748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd280b4e10 0x7fcd2815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd277fe640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd280cd4e0 0x7fcd281f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd280b4e10 msgr2=0x7fcd2815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd280b4e10 0x7fcd2815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd280cd4e0 msgr2=0x7fcd281f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:55.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd280cd4e0 0x7fcd281f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180094d0 con 0x7fcd2815cd70 2023-10-21T16:18:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2c8eb640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd2814a820 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c730 tx=0x7fcd2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003b70 con 0x7fcd2815cd70 2023-10-21T16:18:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20003cf0 con 0x7fcd2815cd70 2023-10-21T16:18:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003ea0 con 0x7fcd2815cd70 2023-10-21T16:18:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.522+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd281e7980 con 0x7fcd2815cd70 2023-10-21T16:18:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.523+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd281e75c0 con 0x7fcd2815cd70 2023-10-21T16:18:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.524+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd281e8180 con 0x7fcd2815cd70 2023-10-21T16:18:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.525+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 45) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fcd20005310 con 0x7fcd2815cd70 2023-10-21T16:18:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.526+0000 7fcd24ff9640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.526+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fcd20089760 con 0x7fcd2815cd70 2023-10-21T16:18:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.526+0000 7fcd27fff640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 msgr2=0x7fcd1406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.526+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:18:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.531+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd281e8180 con 0x7fcd2815cd70 2023-10-21T16:18:55.535 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.534+0000 7f4c878b6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:55.548 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.546+0000 7ffaede4c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:18:55.584 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.582+0000 7f4c878b6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:55.612 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.610+0000 7ffaede4c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:18:55.663 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.662+0000 7ffaede4c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:18:55.695 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.693+0000 7f4c878b6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:55.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.728+0000 7fcd27fff640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 msgr2=0x7fcd1406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:55.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.728+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:18:55.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:55.755+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd280d1130 con 0x7fcd1406afe0 2023-10-21T16:18:55.757 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.755+0000 7f4c878b6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:55.779 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.778+0000 7ffaede4c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:18:55.806 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.804+0000 7f4c878b6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:55.845 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.843+0000 7ffaede4c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:55.863 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.861+0000 7f4c878b6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:55.896 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.894+0000 7ffaede4c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:18:55.913 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:55.911+0000 7f4c878b6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:55.954 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:55.953+0000 7ffaede4c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:18:56.007 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.005+0000 7ffaede4c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:18:56.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:56.129+0000 7fcd27fff640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 msgr2=0x7fcd1406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:56.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:56.129+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:18:56.292 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.290+0000 7f4c878b6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:56.403 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.402+0000 7ffaede4c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:18:56.451 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.449+0000 7f4c878b6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:56.559 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.556+0000 7f4c878b6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:56.570 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.568+0000 7ffaede4c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:18:56.608 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.606+0000 7f4c878b6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:56.682 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.680+0000 7ffaede4c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:18:56.715 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.713+0000 7f4c878b6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:56.734 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.732+0000 7ffaede4c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:18:56.821 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.819+0000 7f4c878b6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:56.846 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.845+0000 7ffaede4c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:56.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:56.930+0000 7fcd27fff640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 msgr2=0x7fcd1406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1967358544 2023-10-21T16:18:56.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:56.930+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:18:56.958 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:56.956+0000 7ffaede4c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:18:56.976 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:56.974+0000 7f4c878b6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:57.026 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:57.025+0000 7f4c878b6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:57.122 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:57.120+0000 7ffaede4c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:18:57.145 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:57.143+0000 7f4c878b6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:57.172 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:57.171+0000 7ffaede4c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:18:57.297 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:57.296+0000 7ffaede4c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:18:57.994 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:57.992+0000 7f4c878b6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:58.181 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:58.179+0000 7ffaede4c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:18:58.316 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.314+0000 7f4c878b6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:58.377 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.375+0000 7f4c878b6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:58.490 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.488+0000 7f4c878b6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:58.490 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:58.488+0000 7ffaede4c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:18:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:58.509+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 46) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fcd20087ae0 con 0x7fcd2815cd70 2023-10-21T16:18:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:58.510+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 msgr2=0x7fcd1406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:18:58.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:58.510+0000 7fcd24ff9640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/1967358544,v1:172.21.15.124:6833/1967358544] conn(0x7fcd1406afe0 0x7fcd1406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:58.556 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:58.555+0000 7ffaede4c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:18:58.648 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.646+0000 7f4c062fb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.648 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.646+0000 7f4c062fb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.649 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.646+0000 7f4c062fb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.649 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.646+0000 7f4c062fb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.649 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.646+0000 7f4c062fb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.649 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.647+0000 7f4c04af8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.650 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.647+0000 7f4c04af8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.650 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.647+0000 7f4c04af8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.650 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.647+0000 7f4c04af8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.650 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:18:58.647+0000 7f4c04af8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:18:58.681 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:18:58.680+0000 7ffaede4c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:18:59.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.522+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 47) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fcd20054930 con 0x7fcd2815cd70 2023-10-21T16:18:59.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.522+0000 7fcd24ff9640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fcd14035880 0x7fcd14075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.522+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 --> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd14035c60 con 0x7fcd14035880 2023-10-21T16:18:59.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.533+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fcd14035880 0x7fcd14075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.546+0000 7fcd27fff640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fcd14035880 0x7fcd14075500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd18008030 comp rx=0 tx=0).ready entity=mgr.6679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.686+0000 7fcd24ff9640 1 -- 172.21.15.53:0/1420224748 <== mgr.6679 v2:172.21.15.124:6832/2567187932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fcd14035c60 con 0x7fcd14035880 2023-10-21T16:18:59.688 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /bad-path 2023-10-21T16:18:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fcd14035880 msgr2=0x7fcd14075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fcd14035880 0x7fcd14075500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd18008030 comp rx=0 tx=0).stop 2023-10-21T16:18:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 msgr2=0x7fcd2814a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 --2- 172.21.15.53:0/1420224748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd2815cd70 0x7fcd2814a820 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c730 tx=0x7fcd2000cbf0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2c8eb640 1 -- 172.21.15.53:0/1420224748 reap_dead start 2023-10-21T16:18:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 shutdown_connections 2023-10-21T16:18:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 >> 172.21.15.53:0/1420224748 conn(0x7fcd280cf700 msgr2=0x7fcd280b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 shutdown_connections 2023-10-21T16:18:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.690+0000 7fcd2e34e640 1 -- 172.21.15.53:0/1420224748 wait complete. 2023-10-21T16:18:59.712 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-21T16:18:59.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-21T16:18:59.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.942+0000 7fd19bfff640 1 Processor -- start 2023-10-21T16:18:59.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.942+0000 7fd19bfff640 1 -- start start 2023-10-21T16:18:59.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd19c0cf2b0 0x7fd19c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0cd520 con 0x7fd19c0ce510 2023-10-21T16:18:59.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0d1910 con 0x7fd19c0cf690 2023-10-21T16:18:59.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c0ce080 con 0x7fd19c0cf2b0 2023-10-21T16:18:59.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd19c0ce510 0x7fd19c15c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60416/0 (socket says 172.21.15.53:60416) 2023-10-21T16:18:59.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.944+0000 7fd19b7fe640 1 -- 172.21.15.53:0/276689094 learned_addr learned my addr 172.21.15.53:0/276689094 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:18:59.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd19affd640 1 --2- 172.21.15.53:0/276689094 >> v2:172.21.15.124:3300/0 conn(0x7fd19c0cf2b0 0x7fd19c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443731038 0 0) 0x7fd19c0cd520 con 0x7fd19c0ce510 2023-10-21T16:18:59.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188003520 con 0x7fd19c0ce510 2023-10-21T16:18:59.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3824673786 0 0) 0x7fd188003520 con 0x7fd19c0ce510 2023-10-21T16:18:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 >> v2:172.21.15.124:3300/0 conn(0x7fd19c0cf2b0 msgr2=0x7fd19c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 --2- 172.21.15.53:0/276689094 >> v2:172.21.15.124:3300/0 conn(0x7fd19c0cf2b0 0x7fd19c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 >> v1:172.21.15.124:6789/0 conn(0x7fd19c0cf690 legacy=0x7fd19c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0cd750 con 0x7fd19c0ce510 2023-10-21T16:18:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd194001670 con 0x7fd19c0ce510 2023-10-21T16:18:59.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 >> v1:172.21.15.53:6789/0 conn(0x7fd19c0ce510 legacy=0x7fd19c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880039c0 0x7fd188003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880047c0 0x7fd188004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 0x7fd18800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0ce080 con 0x7fd1880051d0 2023-10-21T16:18:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18800c120 con 0x7fd1880039c0 2023-10-21T16:18:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.945+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18800c2e0 con 0x7fd1880047c0 2023-10-21T16:18:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 0x7fd18800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19affd640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880039c0 0x7fd188003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19a7fc640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880047c0 0x7fd188004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 -- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880047c0 msgr2=0x7fd188004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880047c0 0x7fd188004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 -- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880039c0 msgr2=0x7fd188003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:18:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880039c0 0x7fd188003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.946+0000 7fd19b7fe640 1 -- 172.21.15.53:0/276689094 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188003520 con 0x7fd1880051d0 2023-10-21T16:18:59.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.947+0000 7fd19b7fe640 1 -- 172.21.15.53:0/276689094 reap_dead start 2023-10-21T16:18:59.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.947+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 0x7fd18800b980 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd194001eb0 tx=0x7fd1880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74e428088e3352 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:59.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194005ad0 con 0x7fd1880051d0 2023-10-21T16:18:59.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194005c30 con 0x7fd1880051d0 2023-10-21T16:18:59.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd199ffb640 1 -- 172.21.15.53:0/276689094 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194007070 con 0x7fd1880051d0 2023-10-21T16:18:59.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 -- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 msgr2=0x7fd18800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 0x7fd18800b980 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fd194001eb0 tx=0x7fd1880057f0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 -- 172.21.15.53:0/276689094 shutdown_connections 2023-10-21T16:18:59.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 --2- 172.21.15.53:0/276689094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880051d0 0x7fd18800b980 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 -- 172.21.15.53:0/276689094 >> 172.21.15.53:0/276689094 conn(0x7fd19c09bc40 msgr2=0x7fd19c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:18:59.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 -- 172.21.15.53:0/276689094 shutdown_connections 2023-10-21T16:18:59.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.948+0000 7fd19bfff640 1 -- 172.21.15.53:0/276689094 wait complete. 2023-10-21T16:18:59.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.950+0000 7fd19bfff640 1 Processor -- start 2023-10-21T16:18:59.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.950+0000 7fd19bfff640 1 -- start start 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 0x7fd19c0b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 0x7fd19c0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b4940 con 0x7fd19c15c5e0 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b87b0 con 0x7fd1880051d0 2023-10-21T16:18:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c1581e0 con 0x7fd19c0b3f50 2023-10-21T16:18:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43802/0 (socket says 172.21.15.53:43802) 2023-10-21T16:18:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.951+0000 7fd19a7fc640 1 -- 172.21.15.53:0/3095055076 learned_addr learned my addr 172.21.15.53:0/3095055076 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:18:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 0x7fd19c0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19affd640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 0x7fd19c0b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 -- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 msgr2=0x7fd19c0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 0x7fd19c0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 -- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 msgr2=0x7fd19c0b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:18:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 0x7fd19c0b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:18:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 -- 172.21.15.53:0/3095055076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c158480 con 0x7fd19c15c5e0 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 0x7fd19c0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19affd640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 0x7fd19c0b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd19a7fc640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d8e0 tx=0x7fd18400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.952+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184003da0 con 0x7fd19c15c5e0 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.953+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18400b6b0 con 0x7fd19c15c5e0 2023-10-21T16:18:59.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.953+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400ba20 con 0x7fd19c15c5e0 2023-10-21T16:18:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.953+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c158760 con 0x7fd19c15c5e0 2023-10-21T16:18:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.953+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c158d20 con 0x7fd19c15c5e0 2023-10-21T16:18:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.954+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c158ee0 con 0x7fd19c15c5e0 2023-10-21T16:18:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.955+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 47) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd18400bb80 con 0x7fd19c15c5e0 2023-10-21T16:18:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.956+0000 7fd1937fe640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 0x7fd17c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:18:59.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.956+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd184089980 con 0x7fd19c15c5e0 2023-10-21T16:18:59.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.956+0000 7fd19affd640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 0x7fd17c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:18:59.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.957+0000 7fd19affd640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 0x7fd17c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009970 tx=0x7fd18c008040 comp rx=0 tx=0).ready entity=mgr.6679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:18:59.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:18:59.961+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1840549b0 con 0x7fd19c15c5e0 2023-10-21T16:19:00.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.178+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 --> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd19c0cdd10 con 0x7fd17c06ad40 2023-10-21T16:19:00.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.196+0000 7fd1937fe640 1 -- 172.21.15.53:0/3095055076 <== mgr.6679 v2:172.21.15.124:6832/2567187932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fd18c012950 con 0x7fd17c06ad40 2023-10-21T16:19:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:19:00.198 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:18:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-21T16:19:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.198+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 msgr2=0x7fd17c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.198+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 0x7fd17c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009970 tx=0x7fd18c008040 comp rx=0 tx=0).stop 2023-10-21T16:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.198+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 msgr2=0x7fd19c0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.198+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d8e0 tx=0x7fd18400dda0 comp rx=0 tx=0).stop 2023-10-21T16:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 shutdown_connections 2023-10-21T16:19:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd17c06ad40 0x7fd17c06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0b3f50 0x7fd19c0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c15c5e0 0x7fd19c0b5530 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 --2- 172.21.15.53:0/3095055076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1880051d0 0x7fd19c0b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 >> 172.21.15.53:0/3095055076 conn(0x7fd19c09bc40 msgr2=0x7fd19c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:19:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 shutdown_connections 2023-10-21T16:19:00.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:19:00.199+0000 7fd19bfff640 1 -- 172.21.15.53:0/3095055076 wait complete. 2023-10-21T16:19:00.217 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:18:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-21T16:19:00.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:19:00.706+0000 7f4bf72dd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:19:00.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:19:00.706+0000 7f4bf72dd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:19:00.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:19:00.706+0000 7f4bf72dd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:19:00.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:19:00.706+0000 7f4bf72dd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:19:00.708 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:19:00.706+0000 7f4bf72dd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:19:21.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:19:21.761 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:19:51.897 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:19:51.909 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:20:00.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-21T16:20:00.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.464+0000 7f320cba5640 1 Processor -- start 2023-10-21T16:20:00.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.464+0000 7f320cba5640 1 -- start start 2023-10-21T16:20:00.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.465+0000 7f320cba5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32080cb5e0 0x7f32080cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.465+0000 7f320cba5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3208159db0 con 0x7f32081599d0 2023-10-21T16:20:00.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f320cba5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3208071fe0 con 0x7f32080b3c50 2023-10-21T16:20:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f320cba5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080b44c0 con 0x7f32080cb5e0 2023-10-21T16:20:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f32077fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32081599d0 0x7f32080caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56880/0 (socket says 172.21.15.53:56880) 2023-10-21T16:20:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f32077fe640 1 -- 172.21.15.53:0/852445264 learned_addr learned my addr 172.21.15.53:0/852445264 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f31fffff640 1 --2- 172.21.15.53:0/852445264 >> v2:172.21.15.53:3300/0 conn(0x7f32080cb5e0 0x7f32080cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f3206ffd640 1 -- 172.21.15.53:0/852445264 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423206326 0 0) 0x7f31f8001110 con 0x7f32081599d0 2023-10-21T16:20:00.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f3206ffd640 1 -- 172.21.15.53:0/852445264 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31f4003520 con 0x7f32081599d0 2023-10-21T16:20:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f31fffff640 1 -- 172.21.15.53:0/852445264 >> v1:172.21.15.124:6789/0 conn(0x7f32080b3c50 legacy=0x7f32080cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.466+0000 7f31fffff640 1 -- 172.21.15.53:0/852445264 >> v1:172.21.15.53:6789/0 conn(0x7f32081599d0 legacy=0x7f32080caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.467+0000 7f31fffff640 1 -- 172.21.15.53:0/852445264 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080713d0 con 0x7f32080cb5e0 2023-10-21T16:20:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.468+0000 7f31fffff640 1 --2- 172.21.15.53:0/852445264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32080cb5e0 0x7f32080cdcd0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f31f00097a0 tx=0x7f31f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c38d641be2cbf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.468+0000 7f3206ffd640 1 -- 172.21.15.53:0/852445264 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0031830 con 0x7f32080cb5e0 2023-10-21T16:20:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.468+0000 7f3206ffd640 1 -- 172.21.15.53:0/852445264 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0031990 con 0x7f32080cb5e0 2023-10-21T16:20:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.468+0000 7f3206ffd640 1 -- 172.21.15.53:0/852445264 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0031d00 con 0x7f32080cb5e0 2023-10-21T16:20:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 -- 172.21.15.53:0/852445264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32080cb5e0 msgr2=0x7f32080cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 --2- 172.21.15.53:0/852445264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32080cb5e0 0x7f32080cdcd0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f31f00097a0 tx=0x7f31f002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 -- 172.21.15.53:0/852445264 shutdown_connections 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 --2- 172.21.15.53:0/852445264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32080cb5e0 0x7f32080cdcd0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 -- 172.21.15.53:0/852445264 >> 172.21.15.53:0/852445264 conn(0x7f320806c050 msgr2=0x7f320806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 -- 172.21.15.53:0/852445264 shutdown_connections 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.469+0000 7f320cba5640 1 -- 172.21.15.53:0/852445264 wait complete. 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.471+0000 7f320cba5640 1 Processor -- start 2023-10-21T16:20:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.471+0000 7f320cba5640 1 -- start start 2023-10-21T16:20:00.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 0x7f3208146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 0x7f32080ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208145180 con 0x7f32081599d0 2023-10-21T16:20:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3208145460 con 0x7f32080cb5e0 2023-10-21T16:20:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f320cba5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3208145740 con 0x7f32080b3c50 2023-10-21T16:20:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f3207fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f3207fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37474/0 (socket says 172.21.15.53:37474) 2023-10-21T16:20:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f3207fff640 1 -- 172.21.15.53:0/3870403202 learned_addr learned my addr 172.21.15.53:0/3870403202 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.472+0000 7f31fffff640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 0x7f32080ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f32077fe640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 0x7f3208146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 -- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 msgr2=0x7f3208146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 0x7f3208146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 -- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 msgr2=0x7f32080ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 0x7f32080ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 -- 172.21.15.53:0/3870403202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0009400 con 0x7f32081599d0 2023-10-21T16:20:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f32077fe640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 0x7f3208146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f31fffff640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 0x7f32080ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3207fff640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f320000c780 tx=0x7f320000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200004b60 con 0x7f32081599d0 2023-10-21T16:20:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320000ce90 con 0x7f32081599d0 2023-10-21T16:20:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320000a850 con 0x7f32081599d0 2023-10-21T16:20:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.473+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3208145a20 con 0x7f32081599d0 2023-10-21T16:20:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.474+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3208157f40 con 0x7f32081599d0 2023-10-21T16:20:00.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.475+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3208158160 con 0x7f32081599d0 2023-10-21T16:20:00.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.476+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 49) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f320000a9b0 con 0x7f32081599d0 2023-10-21T16:20:00.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.477+0000 7f3204ff9640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 0x7f31e806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.477+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f320008a050 con 0x7f32081599d0 2023-10-21T16:20:00.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.477+0000 7f32077fe640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 0x7f31e806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.478+0000 7f32077fe640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 0x7f31e806d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001270 tx=0x7f31f8009040 comp rx=0 tx=0).ready entity=mgr.6679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.481+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32000bcca0 con 0x7f32081599d0 2023-10-21T16:20:00.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.698+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 --> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3208071990 con 0x7f31e806b100 2023-10-21T16:20:00.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.716+0000 7f3204ff9640 1 -- 172.21.15.53:0/3870403202 <== mgr.6679 v2:172.21.15.124:6832/2567187932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f31f8001c50 con 0x7f31e806b100 2023-10-21T16:20:00.718 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:00.718 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:18:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-21T16:20:00.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.718+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 msgr2=0x7f31e806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.718+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 0x7f31e806d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001270 tx=0x7f31f8009040 comp rx=0 tx=0).stop 2023-10-21T16:20:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.718+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 msgr2=0x7f32081447a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.718+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f320000c780 tx=0x7f320000cc40 comp rx=0 tx=0).stop 2023-10-21T16:20:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.718+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 shutdown_connections 2023-10-21T16:20:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f31e806b100 0x7f31e806d5c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32081599d0 0x7f32081447a0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32080cb5e0 0x7f32080ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 --2- 172.21.15.53:0/3870403202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32080b3c50 0x7f3208146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 >> 172.21.15.53:0/3870403202 conn(0x7f320806c050 msgr2=0x7f32080af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 shutdown_connections 2023-10-21T16:20:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.719+0000 7f320cba5640 1 -- 172.21.15.53:0/3870403202 wait complete. 2023-10-21T16:20:00.741 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:18:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-21T16:20:00.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-10-21T16:20:00.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.971+0000 7f5e26173640 1 Processor -- start 2023-10-21T16:20:00.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.972+0000 7f5e26173640 1 -- start start 2023-10-21T16:20:00.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e26173640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5e20156a10 0x7f5e201594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e26173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e200aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e26173640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e20071d90 con 0x7f5e200b04b0 2023-10-21T16:20:00.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e26173640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200b0940 con 0x7f5e20156a10 2023-10-21T16:20:00.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e26173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20070f20 con 0x7f5e20070ac0 2023-10-21T16:20:00.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e25972640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5e200b04b0 0x7f5e2015cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56894/0 (socket says 172.21.15.53:56894) 2023-10-21T16:20:00.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.973+0000 7f5e25972640 1 -- 172.21.15.53:0/3927877379 learned_addr learned my addr 172.21.15.53:0/3927877379 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:00.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e25171640 1 --2- 172.21.15.53:0/3927877379 >> v2:172.21.15.124:3300/0 conn(0x7f5e20156a10 0x7f5e201594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e24970640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e200aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445876502 0 0) 0x7f5e20071d90 con 0x7f5e200b04b0 2023-10-21T16:20:00.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e0c003520 con 0x7f5e200b04b0 2023-10-21T16:20:00.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2515128950 0 0) 0x7f5e0c003520 con 0x7f5e200b04b0 2023-10-21T16:20:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 msgr2=0x7f5e200aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e200aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 >> v2:172.21.15.124:3300/0 conn(0x7f5e20156a10 msgr2=0x7f5e201594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 --2- 172.21.15.53:0/3927877379 >> v2:172.21.15.124:3300/0 conn(0x7f5e20156a10 0x7f5e201594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e20071180 con 0x7f5e200b04b0 2023-10-21T16:20:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5e180019f0 con 0x7f5e200b04b0 2023-10-21T16:20:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 >> v1:172.21.15.53:6789/0 conn(0x7f5e200b04b0 legacy=0x7f5e2015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.974+0000 7f5e17fff640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e0c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e17fff640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e17fff640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0c004d00 0x7f5e0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b0940 con 0x7f5e0c003ef0 2023-10-21T16:20:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20070f20 con 0x7f5e0c004d00 2023-10-21T16:20:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c00b980 con 0x7f5e20070ac0 2023-10-21T16:20:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e25972640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0c004d00 0x7f5e0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e25171640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e0c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 -- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 msgr2=0x7f5e0c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20070ac0 0x7f5e0c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 -- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0c004d00 msgr2=0x7f5e0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0c004d00 0x7f5e0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.975+0000 7f5e24970640 1 -- 172.21.15.53:0/3927877379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c003520 con 0x7f5e0c003ef0 2023-10-21T16:20:00.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.976+0000 7f5e24970640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e0c0047c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000aac0 tx=0x7f5e0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95d72ee46d5bdb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10035440 con 0x7f5e0c003ef0 2023-10-21T16:20:00.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100355a0 con 0x7f5e0c003ef0 2023-10-21T16:20:00.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e17fff640 1 -- 172.21.15.53:0/3927877379 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10035930 con 0x7f5e0c003ef0 2023-10-21T16:20:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e26173640 1 -- 172.21.15.53:0/3927877379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 msgr2=0x7f5e0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e26173640 1 --2- 172.21.15.53:0/3927877379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e0c0047c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000aac0 tx=0x7f5e0c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.977+0000 7f5e25972640 1 -- 172.21.15.53:0/3927877379 reap_dead start 2023-10-21T16:20:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.978+0000 7f5e26173640 1 -- 172.21.15.53:0/3927877379 shutdown_connections 2023-10-21T16:20:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.978+0000 7f5e26173640 1 -- 172.21.15.53:0/3927877379 >> 172.21.15.53:0/3927877379 conn(0x7f5e2006c050 msgr2=0x7f5e2006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.978+0000 7f5e26173640 1 -- 172.21.15.53:0/3927877379 shutdown_connections 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.978+0000 7f5e26173640 1 -- 172.21.15.53:0/3927877379 wait complete. 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.979+0000 7f5e26173640 1 Processor -- start 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.980+0000 7f5e26173640 1 -- start start 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.980+0000 7f5e26173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e26173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20157140 0x7f5e200cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e26173640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e2014f320 0x7f5e200b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e26173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200c66b0 con 0x7f5e0c003ef0 2023-10-21T16:20:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e26173640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200cd700 con 0x7f5e2014f320 2023-10-21T16:20:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e26173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b2100 con 0x7f5e20157140 2023-10-21T16:20:00.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37492/0 (socket says 172.21.15.53:37492) 2023-10-21T16:20:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 -- 172.21.15.53:0/408492407 learned_addr learned my addr 172.21.15.53:0/408492407 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 -- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20157140 msgr2=0x7f5e200cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20157140 0x7f5e200cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 -- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e2014f320 msgr2=0x7f5e200b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e2014f320 0x7f5e200b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.981+0000 7f5e25171640 1 -- 172.21.15.53:0/408492407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200c6920 con 0x7f5e0c003ef0 2023-10-21T16:20:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e25171640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0116d0 tx=0x7f5e1c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00f9c0 con 0x7f5e0c003ef0 2023-10-21T16:20:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00fb20 con 0x7f5e0c003ef0 2023-10-21T16:20:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00fe90 con 0x7f5e0c003ef0 2023-10-21T16:20:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200c6c00 con 0x7f5e0c003ef0 2023-10-21T16:20:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.982+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2014f7a0 con 0x7f5e0c003ef0 2023-10-21T16:20:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.983+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e20074150 con 0x7f5e0c003ef0 2023-10-21T16:20:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.984+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 49) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5e1c015460 con 0x7f5e0c003ef0 2023-10-21T16:20:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.985+0000 7f5e15ffb640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 0x7f5e0406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:00.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.985+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f5e1c08a370 con 0x7f5e0c003ef0 2023-10-21T16:20:00.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.986+0000 7f5e24970640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 0x7f5e0406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:00.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.987+0000 7f5e24970640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 0x7f5e0406d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e10037000 tx=0x7f5e1002f000 comp rx=0 tx=0).ready entity=mgr.6679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:00.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:00.990+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e1c054d60 con 0x7f5e0c003ef0 2023-10-21T16:20:01.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.203+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 --> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e20071740 con 0x7f5e0406b100 2023-10-21T16:20:01.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.362+0000 7f5e15ffb640 1 -- 172.21.15.53:0/408492407 <== mgr.6679 v2:172.21.15.124:6832/2567187932 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f5e1002ee60 con 0x7f5e0406b100 2023-10-21T16:20:01.364 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /bad-path 2023-10-21T16:20:01.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.366+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 msgr2=0x7f5e0406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.366+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 0x7f5e0406d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5e10037000 tx=0x7f5e1002f000 comp rx=0 tx=0).stop 2023-10-21T16:20:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.366+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 msgr2=0x7f5e200b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.366+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0116d0 tx=0x7f5e1c011b90 comp rx=0 tx=0).stop 2023-10-21T16:20:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 shutdown_connections 2023-10-21T16:20:01.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f5e0406b100 0x7f5e0406d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e2014f320 0x7f5e200b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e20157140 0x7f5e200cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 --2- 172.21.15.53:0/408492407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0c003ef0 0x7f5e200b1af0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 >> 172.21.15.53:0/408492407 conn(0x7f5e2006c050 msgr2=0x7f5e2015bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:01.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 shutdown_connections 2023-10-21T16:20:01.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.367+0000 7f5e26173640 1 -- 172.21.15.53:0/408492407 wait complete. 2023-10-21T16:20:01.391 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-21T16:20:01.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:20:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.619+0000 7fb02232e640 1 Processor -- start 2023-10-21T16:20:01.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.619+0000 7fb02232e640 1 -- start start 2023-10-21T16:20:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb02232e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb02232e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb01c0cb5e0 0x7fb01c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb02232e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c159db0 con 0x7fb01c0b3c50 2023-10-21T16:20:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb02232e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c071fe0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb02232e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b44c0 con 0x7fb01c1599d0 2023-10-21T16:20:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.621+0000 7fb021b2d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb01c0cb5e0 0x7fb01c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb020b2b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb01c0b3c50 0x7fb01c0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56900/0 (socket says 172.21.15.53:56900) 2023-10-21T16:20:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb020b2b640 1 -- 172.21.15.53:0/3284016724 learned_addr learned my addr 172.21.15.53:0/3284016724 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb02132c640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb021b2d640 1 -- 172.21.15.53:0/3284016724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 msgr2=0x7fb01c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb021b2d640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb021b2d640 1 -- 172.21.15.53:0/3284016724 >> v1:172.21.15.53:6789/0 conn(0x7fb01c0b3c50 legacy=0x7fb01c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb021b2d640 1 -- 172.21.15.53:0/3284016724 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0713d0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb013fff640 1 -- 172.21.15.53:0/3284016724 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239272896 0 0) 0x7fb00c001110 con 0x7fb01c0b3c50 2023-10-21T16:20:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.622+0000 7fb02132c640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:01.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.623+0000 7fb021b2d640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0cc0c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb014009d90 tx=0x7fb01402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2177bdfdf30e727 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:01.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb013fff640 1 -- 172.21.15.53:0/3284016724 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01402cc60 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb013fff640 1 -- 172.21.15.53:0/3284016724 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01402cdc0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb013fff640 1 -- 172.21.15.53:0/3284016724 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014034660 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 -- 172.21.15.53:0/3284016724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 msgr2=0x7fb01c0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0cc0c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb014009d90 tx=0x7fb01402beb0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 -- 172.21.15.53:0/3284016724 shutdown_connections 2023-10-21T16:20:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0cc0c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 --2- 172.21.15.53:0/3284016724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 -- 172.21.15.53:0/3284016724 >> 172.21.15.53:0/3284016724 conn(0x7fb01c06c050 msgr2=0x7fb01c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:01.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 -- 172.21.15.53:0/3284016724 shutdown_connections 2023-10-21T16:20:01.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.624+0000 7fb02232e640 1 -- 172.21.15.53:0/3284016724 wait complete. 2023-10-21T16:20:01.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.626+0000 7fb02232e640 1 Processor -- start 2023-10-21T16:20:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.626+0000 7fb02232e640 1 -- start start 2023-10-21T16:20:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb01c0b3c50 0x7fb01c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b12e0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b15c0 con 0x7fb01c0b3c50 2023-10-21T16:20:01.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb02232e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b18a0 con 0x7fb01c1599d0 2023-10-21T16:20:01.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.627+0000 7fb020b2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37508/0 (socket says 172.21.15.53:37508) 2023-10-21T16:20:01.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 -- 172.21.15.53:0/4147803557 learned_addr learned my addr 172.21.15.53:0/4147803557 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:01.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb021b2d640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb02132c640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb01c0b3c50 0x7fb01c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 -- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 msgr2=0x7fb01c0c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 -- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb01c0b3c50 msgr2=0x7fb01c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:01.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb01c0b3c50 0x7fb01c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:01.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 -- 172.21.15.53:0/4147803557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb014009a70 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.628+0000 7fb020b2b640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c780 tx=0x7fb00c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.629+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0072d0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.629+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c007450 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.629+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0047e0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.629+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c0b1b80 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.629+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c151b30 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.630+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c074150 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.631+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 49) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb00c004960 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.632+0000 7fb011ffb640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 0x7fb00006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.632+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fb00c088fe0 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.633+0000 7fb02132c640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 0x7fb00006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.634+0000 7fb02132c640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 0x7fb00006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).ready entity=mgr.6679 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.637+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb00c053990 con 0x7fb01c0cb5e0 2023-10-21T16:20:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:01.881+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb01c159db0 con 0x7fb01c0cb5e0 2023-10-21T16:20:02.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.653+0000 7fb011ffb640 1 -- 172.21.15.53:0/4147803557 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v50) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb01c159db0 con 0x7fb01c0cb5e0 2023-10-21T16:20:02.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.657+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 msgr2=0x7fb00006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:02.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.657+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 0x7fb00006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).stop 2023-10-21T16:20:02.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.657+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 msgr2=0x7fb01c0b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:02.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.657+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c780 tx=0x7fb00c00cc40 comp rx=0 tx=0).stop 2023-10-21T16:20:02.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.658+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 shutdown_connections 2023-10-21T16:20:02.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.658+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fb00006b100 0x7fb00006d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:02.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.658+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb01c1599d0 0x7fb01c0c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:02.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.658+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb01c0cb5e0 0x7fb01c0b2380 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:02.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.658+0000 7fb02232e640 1 --2- 172.21.15.53:0/4147803557 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb01c0b3c50 0x7fb01c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:02.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.659+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 >> 172.21.15.53:0/4147803557 conn(0x7fb01c06c050 msgr2=0x7fb01c0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:02.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.661+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 shutdown_connections 2023-10-21T16:20:02.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:02.662+0000 7fb02232e640 1 -- 172.21.15.53:0/4147803557 wait complete. 2023-10-21T16:20:02.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:20:02.688 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:20:02.797 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:20:02.891 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:20:02.892 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:02.892 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:02.949 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:02.949 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:03.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:03.184 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.182+0000 7f7c59e38040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:20:03.191 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.189+0000 7ff9aca2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:20:03.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.204+0000 7f3d6f91d640 1 Processor -- start 2023-10-21T16:20:03.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.204+0000 7f3d6f91d640 1 -- start start 2023-10-21T16:20:03.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6f91d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6815ab40 0x7f3d680ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6f91d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680c90b0 0x7f3d680adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6f91d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6815b2b0 con 0x7f3d680cae70 2023-10-21T16:20:03.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6f91d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6815bf30 con 0x7f3d6815ab40 2023-10-21T16:20:03.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6f91d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6815f7a0 con 0x7f3d680c90b0 2023-10-21T16:20:03.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6d6b9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6815ab40 0x7f3d680ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6d6b9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d6815ab40 0x7f3d680ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37536/0 (socket says 172.21.15.53:37536) 2023-10-21T16:20:03.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/3463217888 learned_addr learned my addr 172.21.15.53:0/3463217888 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:03.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6deba640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3d680cae70 0x7f3d680702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56920/0 (socket says 172.21.15.53:56920) 2023-10-21T16:20:03.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.206+0000 7f3d6ceb8640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680c90b0 0x7f3d680adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.207+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/3463217888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680c90b0 msgr2=0x7f3d680adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:03.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.207+0000 7f3d6d6b9640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680c90b0 0x7f3d680adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.207+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/3463217888 >> v1:172.21.15.53:6789/0 conn(0x7f3d680cae70 legacy=0x7f3d680702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.207+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/3463217888 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6815b590 con 0x7f3d6815ab40 2023-10-21T16:20:03.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.207+0000 7f3d5ffff640 1 -- 172.21.15.53:0/3463217888 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297448866 0 0) 0x7f3d6815b2b0 con 0x7f3d680cae70 2023-10-21T16:20:03.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.208+0000 7f3d6d6b9640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6815ab40 0x7f3d680ad720 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009870 tx=0x7f3d6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96faac996acad748 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:03.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.208+0000 7f3d5ffff640 1 -- 172.21.15.53:0/3463217888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64030840 con 0x7f3d6815ab40 2023-10-21T16:20:03.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.208+0000 7f3d5ffff640 1 -- 172.21.15.53:0/3463217888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d640309a0 con 0x7f3d6815ab40 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.208+0000 7f3d5ffff640 1 -- 172.21.15.53:0/3463217888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64030d30 con 0x7f3d6815ab40 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 -- 172.21.15.53:0/3463217888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6815ab40 msgr2=0x7f3d680ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6815ab40 0x7f3d680ad720 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009870 tx=0x7f3d6402b980 comp rx=0 tx=0).stop 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 -- 172.21.15.53:0/3463217888 shutdown_connections 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680c90b0 0x7f3d680adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/3463217888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d6815ab40 0x7f3d680ad720 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 -- 172.21.15.53:0/3463217888 >> 172.21.15.53:0/3463217888 conn(0x7f3d6806a380 msgr2=0x7f3d680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 -- 172.21.15.53:0/3463217888 shutdown_connections 2023-10-21T16:20:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.209+0000 7f3d6f91d640 1 -- 172.21.15.53:0/3463217888 wait complete. 2023-10-21T16:20:03.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.211+0000 7f3d6f91d640 1 Processor -- start 2023-10-21T16:20:03.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.211+0000 7f3d6f91d640 1 -- start start 2023-10-21T16:20:03.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680cae70 0x7f3d68154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 0x7f3d68153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68154880 con 0x7f3d680c90b0 2023-10-21T16:20:03.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68154ab0 con 0x7f3d6815ab40 2023-10-21T16:20:03.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6f91d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680ac2c0 con 0x7f3d680cae70 2023-10-21T16:20:03.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6d6b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6d6b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37546/0 (socket says 172.21.15.53:37546) 2023-10-21T16:20:03.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/105226536 learned_addr learned my addr 172.21.15.53:0/105226536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:03.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6ceb8640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680cae70 0x7f3d68154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.212+0000 7f3d6deba640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 0x7f3d68153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680cae70 msgr2=0x7f3d68154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680cae70 0x7f3d68154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 msgr2=0x7f3d68153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 0x7f3d68153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 -- 172.21.15.53:0/105226536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640094d0 con 0x7f3d680c90b0 2023-10-21T16:20:03.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6deba640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 0x7f3d68153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:03.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6d6b9640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000c00 tx=0x7f3d64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:03.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64039c90 con 0x7f3d680c90b0 2023-10-21T16:20:03.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d64039df0 con 0x7f3d680c90b0 2023-10-21T16:20:03.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64043640 con 0x7f3d680c90b0 2023-10-21T16:20:03.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d680ac5a0 con 0x7f3d680c90b0 2023-10-21T16:20:03.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.213+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d680acb60 con 0x7f3d680c90b0 2023-10-21T16:20:03.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.215+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68067490 con 0x7f3d680c90b0 2023-10-21T16:20:03.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.216+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3d640437a0 con 0x7f3d680c90b0 2023-10-21T16:20:03.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.216+0000 7f3d5dffb640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 0x7f3d4c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.217+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f3d6403e070 con 0x7f3d680c90b0 2023-10-21T16:20:03.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.217+0000 7f3d6ceb8640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 msgr2=0x7f3d4c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:03.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.217+0000 7f3d6ceb8640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 0x7f3d4c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:03.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.220+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d640778b0 con 0x7f3d680c90b0 2023-10-21T16:20:03.242 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.241+0000 7f7c59e38040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:20:03.243 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.242+0000 7ff9aca2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:20:03.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.415+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d680b0ef0 con 0x7f3d680c90b0 2023-10-21T16:20:03.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.417+0000 7f3d5dffb640 1 -- 172.21.15.53:0/105226536 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17549 (secure 0 0 0) 0x7f3d6407aed0 con 0x7f3d680c90b0 2023-10-21T16:20:03.419 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:03.420 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":121,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:19:59.623595+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3937751236":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2028143540":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/130280090":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/1496413032":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6833/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6832/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/3651970368":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/28232742":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2272255357":"2023-10-21T17:18:51.635575+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:03.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.418+0000 7f3d6ceb8640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 msgr2=0x7f3d4c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:03.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.418+0000 7f3d6ceb8640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 0x7f3d4c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:03.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.419+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 msgr2=0x7f3d4c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:03.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.419+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 0x7f3d4c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 msgr2=0x7f3d680b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000c00 tx=0x7f3d64004730 comp rx=0 tx=0).stop 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 shutdown_connections 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f3d4c06b0b0 0x7f3d4c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d6815ab40 0x7f3d68153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d680cae70 0x7f3d68154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 --2- 172.21.15.53:0/105226536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d680c90b0 0x7f3d680b1590 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 >> 172.21.15.53:0/105226536 conn(0x7f3d6806a380 msgr2=0x7f3d6806c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 shutdown_connections 2023-10-21T16:20:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.420+0000 7f3d6f91d640 1 -- 172.21.15.53:0/105226536 wait complete. 2023-10-21T16:20:03.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:18:41.952733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:18:42.974741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:20:03.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:20:03.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.648+0000 7f635359e640 1 Processor -- start 2023-10-21T16:20:03.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.649+0000 7f635359e640 1 -- start start 2023-10-21T16:20:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.650+0000 7f635359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f635415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.650+0000 7f635359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635415d6b0 con 0x7f635406b330 2023-10-21T16:20:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.650+0000 7f635359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635415d990 con 0x7f63540b27b0 2023-10-21T16:20:03.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.650+0000 7f635359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635415dc70 con 0x7f63540b23d0 2023-10-21T16:20:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6352d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f635406b330 0x7f6354160c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:56926/0 (socket says 172.21.15.53:56926) 2023-10-21T16:20:03.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6352d9d640 1 -- 172.21.15.53:0/3792814042 learned_addr learned my addr 172.21.15.53:0/3792814042 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:03.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f635259c640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f635415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100625048 0 0) 0x7f635415d6b0 con 0x7f635406b330 2023-10-21T16:20:03.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6340003520 con 0x7f635406b330 2023-10-21T16:20:03.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313039220 0 0) 0x7f6340003520 con 0x7f635406b330 2023-10-21T16:20:03.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 msgr2=0x7f635415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:03.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f635415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 >> v1:172.21.15.124:6789/0 conn(0x7f63540b27b0 legacy=0x7f635406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.651+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354160710 con 0x7f635406b330 2023-10-21T16:20:03.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f634c0019f0 con 0x7f635406b330 2023-10-21T16:20:03.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 >> v1:172.21.15.53:6789/0 conn(0x7f635406b330 legacy=0x7f6354160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400039a0 0x7f6340003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63400047c0 0x7f6340004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f6340005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635415dc70 con 0x7f63400047c0 2023-10-21T16:20:03.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634000bb10 con 0x7f63400039a0 2023-10-21T16:20:03.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634000bcd0 con 0x7f63540b23d0 2023-10-21T16:20:03.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f634bfff640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63400047c0 0x7f6340004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f6352d9d640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f6340005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.652+0000 7f635259c640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400039a0 0x7f6340003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f634bfff640 1 -- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 msgr2=0x7f6340005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f634bfff640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b23d0 0x7f6340005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f634bfff640 1 -- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400039a0 msgr2=0x7f6340003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:03.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f634bfff640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400039a0 0x7f6340003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f634bfff640 1 -- 172.21.15.53:0/3792814042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6340003520 con 0x7f63400047c0 2023-10-21T16:20:03.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.653+0000 7f635259c640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400039a0 0x7f6340003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:03.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.655+0000 7f634bfff640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63400047c0 0x7f6340004c40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f633c001750 tx=0x7f63400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8f02555989e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:03.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.656+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0052d0 con 0x7f63400047c0 2023-10-21T16:20:03.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.656+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c02e050 con 0x7f63400047c0 2023-10-21T16:20:03.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.656+0000 7f6351d9b640 1 -- 172.21.15.53:0/3792814042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0035f0 con 0x7f63400047c0 2023-10-21T16:20:03.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 -- 172.21.15.53:0/3792814042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63400047c0 msgr2=0x7f6340004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 --2- 172.21.15.53:0/3792814042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63400047c0 0x7f6340004c40 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f633c001750 tx=0x7f63400057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f6352d9d640 1 -- 172.21.15.53:0/3792814042 reap_dead start 2023-10-21T16:20:03.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 -- 172.21.15.53:0/3792814042 shutdown_connections 2023-10-21T16:20:03.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 -- 172.21.15.53:0/3792814042 >> 172.21.15.53:0/3792814042 conn(0x7f63540cd160 msgr2=0x7f63540cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:03.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 -- 172.21.15.53:0/3792814042 shutdown_connections 2023-10-21T16:20:03.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.657+0000 7f635359e640 1 -- 172.21.15.53:0/3792814042 wait complete. 2023-10-21T16:20:03.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.659+0000 7f635359e640 1 Processor -- start 2023-10-21T16:20:03.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.659+0000 7f635359e640 1 -- start start 2023-10-21T16:20:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400047c0 0x7f6354159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b50e0 0x7f6354158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540b55e0 con 0x7f6354159bc0 2023-10-21T16:20:03.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540c8210 con 0x7f63400047c0 2023-10-21T16:20:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.660+0000 7f635359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63540c78e0 con 0x7f63540b50e0 2023-10-21T16:20:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37556/0 (socket says 172.21.15.53:37556) 2023-10-21T16:20:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 -- 172.21.15.53:0/2540289763 learned_addr learned my addr 172.21.15.53:0/2540289763 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:03.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f635259c640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400047c0 0x7f6354159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f6352d9d640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b50e0 0x7f6354158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b50e0 msgr2=0x7f6354158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b50e0 0x7f6354158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400047c0 msgr2=0x7f6354159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400047c0 0x7f6354159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.661+0000 7f634bfff640 1 -- 172.21.15.53:0/2540289763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540c7bc0 con 0x7f6354159bc0 2023-10-21T16:20:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f634bfff640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f633c02bc30 tx=0x7f633c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c0076c0 con 0x7f6354159bc0 2023-10-21T16:20:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c007a30 con 0x7f6354159bc0 2023-10-21T16:20:03.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c006be0 con 0x7f6354159bc0 2023-10-21T16:20:03.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63540c7ea0 con 0x7f6354159bc0 2023-10-21T16:20:03.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.662+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63540b3220 con 0x7f6354159bc0 2023-10-21T16:20:03.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.664+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63540c8380 con 0x7f6354159bc0 2023-10-21T16:20:03.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.665+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f633c006d40 con 0x7f6354159bc0 2023-10-21T16:20:03.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.666+0000 7f634affd640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 0x7f633406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:03.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.666+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f633c0aba60 con 0x7f6354159bc0 2023-10-21T16:20:03.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.666+0000 7f635259c640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 msgr2=0x7f633406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:03.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.667+0000 7f635259c640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 0x7f633406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:03.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.671+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f633c076420 con 0x7f6354159bc0 2023-10-21T16:20:03.690 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.689+0000 7ff9aca2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:20:03.728 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.727+0000 7f7c59e38040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:20:03.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.740+0000 7ff9aca2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:20:03.783 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.782+0000 7f7c59e38040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:20:03.792 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.790+0000 7ff9aca2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:20:03.837 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.835+0000 7f7c59e38040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:20:03.853 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.851+0000 7ff9aca2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:20:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.864+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63540b55e0 con 0x7f6354159bc0 2023-10-21T16:20:03.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.865+0000 7f634affd640 1 -- 172.21.15.53:0/2540289763 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f633c079a40 con 0x7f6354159bc0 2023-10-21T16:20:03.867 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:20:03.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635259c640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 msgr2=0x7f633406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635259c640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 0x7f633406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 msgr2=0x7f633406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:03.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 0x7f633406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 msgr2=0x7f63540c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f633c02bc30 tx=0x7f633c001120 comp rx=0 tx=0).stop 2023-10-21T16:20:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 shutdown_connections 2023-10-21T16:20:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f633406b0b0 0x7f633406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63540b50e0 0x7f6354158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6354159bc0 0x7f63540c4210 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 --2- 172.21.15.53:0/2540289763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63400047c0 0x7f6354159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 >> 172.21.15.53:0/2540289763 conn(0x7f63540cd160 msgr2=0x7f63540cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 shutdown_connections 2023-10-21T16:20:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:03.868+0000 7f635359e640 1 -- 172.21.15.53:0/2540289763 wait complete. 2023-10-21T16:20:03.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:20:03.901 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:03.900+0000 7ff9aca2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:20:03.902 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.901+0000 7f7c59e38040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:20:03.955 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:03.953+0000 7f7c59e38040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:20:04.012 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.010+0000 7ff9aca2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:20:04.073 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.071+0000 7ff9aca2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:04.075 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.074+0000 7f7c59e38040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:20:04.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.087+0000 7f661b167640 1 Processor -- start 2023-10-21T16:20:04.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.088+0000 7f661b167640 1 -- start start 2023-10-21T16:20:04.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.089+0000 7f661b167640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140cf690 0x7f66140d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.089+0000 7f661b167640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66140ce510 0x7f661415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.089+0000 7f661b167640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66140cd520 con 0x7f66140cf2b0 2023-10-21T16:20:04.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.089+0000 7f661b167640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66140d1910 con 0x7f66140ce510 2023-10-21T16:20:04.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.089+0000 7f661b167640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140ce080 con 0x7f66140cf690 2023-10-21T16:20:04.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.090+0000 7f661a165640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f66140cf2b0 0x7f66140d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:43506/0 (socket says 172.21.15.53:43506) 2023-10-21T16:20:04.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.090+0000 7f661a165640 1 -- 172.21.15.53:0/1720969474 learned_addr learned my addr 172.21.15.53:0/1720969474 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:04.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.090+0000 7f661a966640 1 --2- 172.21.15.53:0/1720969474 >> v2:172.21.15.124:3300/0 conn(0x7f66140ce510 0x7f661415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.090+0000 7f6619964640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140cf690 0x7f66140d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f6619163640 1 -- 172.21.15.53:0/1720969474 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389192036 0 0) 0x7f66140cd520 con 0x7f66140cf2b0 2023-10-21T16:20:04.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f6619163640 1 -- 172.21.15.53:0/1720969474 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6600003520 con 0x7f66140cf2b0 2023-10-21T16:20:04.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f661a966640 1 -- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140cf690 msgr2=0x7f66140d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:04.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f661a966640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140cf690 0x7f66140d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f661a966640 1 -- 172.21.15.53:0/1720969474 >> v1:172.21.15.124:6789/0 conn(0x7f66140cf2b0 legacy=0x7f66140d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.091+0000 7f661a966640 1 -- 172.21.15.53:0/1720969474 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140cd750 con 0x7f66140ce510 2023-10-21T16:20:04.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.092+0000 7f661a966640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140ce510 0x7f661415c820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f660c009d90 tx=0x7f660c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92814ae682b5fbf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:04.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.092+0000 7f6619163640 1 -- 172.21.15.53:0/1720969474 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c02cbc0 con 0x7f66140ce510 2023-10-21T16:20:04.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.092+0000 7f6619163640 1 -- 172.21.15.53:0/1720969474 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c02cd20 con 0x7f66140ce510 2023-10-21T16:20:04.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.092+0000 7f6619163640 1 -- 172.21.15.53:0/1720969474 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c034660 con 0x7f66140ce510 2023-10-21T16:20:04.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 -- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140ce510 msgr2=0x7f661415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140ce510 0x7f661415c820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f660c009d90 tx=0x7f660c02be10 comp rx=0 tx=0).stop 2023-10-21T16:20:04.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 -- 172.21.15.53:0/1720969474 shutdown_connections 2023-10-21T16:20:04.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140cf690 0x7f66140d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 --2- 172.21.15.53:0/1720969474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140ce510 0x7f661415c820 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 -- 172.21.15.53:0/1720969474 >> 172.21.15.53:0/1720969474 conn(0x7f661409bc40 msgr2=0x7f66140d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:04.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 -- 172.21.15.53:0/1720969474 shutdown_connections 2023-10-21T16:20:04.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.093+0000 7f661b167640 1 -- 172.21.15.53:0/1720969474 wait complete. 2023-10-21T16:20:04.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.095+0000 7f661b167640 1 Processor -- start 2023-10-21T16:20:04.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.095+0000 7f661b167640 1 -- start start 2023-10-21T16:20:04.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140ce510 0x7f66140ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140cf690 0x7f661415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140caa00 con 0x7f66140cf2b0 2023-10-21T16:20:04.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661415fdc0 con 0x7f66140cf690 2023-10-21T16:20:04.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.096+0000 7f661b167640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66141600a0 con 0x7f66140ce510 2023-10-21T16:20:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47186/0 (socket says 172.21.15.53:47186) 2023-10-21T16:20:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 -- 172.21.15.53:0/12845424 learned_addr learned my addr 172.21.15.53:0/12845424 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f661a966640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140cf690 0x7f661415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f661a165640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140ce510 0x7f66140ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 -- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140ce510 msgr2=0x7f66140ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140ce510 0x7f66140ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 -- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140cf690 msgr2=0x7f661415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140cf690 0x7f661415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 -- 172.21.15.53:0/12845424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c009a70 con 0x7f66140cf2b0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.097+0000 7f6619964640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f660400e870 tx=0x7f660400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.098+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040094f0 con 0x7f66140cf2b0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.098+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660400c940 con 0x7f66140cf2b0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.098+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66140c99e0 con 0x7f66140cf2b0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.098+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400ccf0 con 0x7f66140cf2b0 2023-10-21T16:20:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.098+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66140c9fa0 con 0x7f66140cf2b0 2023-10-21T16:20:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.099+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6614074150 con 0x7f66140cf2b0 2023-10-21T16:20:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.100+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f6604013470 con 0x7f66140cf2b0 2023-10-21T16:20:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.101+0000 7f660affd640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.102+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f6604089f30 con 0x7f66140cf2b0 2023-10-21T16:20:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.102+0000 7f661a165640 1 -- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 msgr2=0x7f65f806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.102+0000 7f661a165640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.105+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66040548a0 con 0x7f66140cf2b0 2023-10-21T16:20:04.124 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.122+0000 7ff9aca2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:20:04.145 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.144+0000 7f7c59e38040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:04.181 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.179+0000 7ff9aca2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:20:04.198 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.196+0000 7f7c59e38040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:20:04.231 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.229+0000 7ff9aca2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:20:04.258 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.257+0000 7f7c59e38040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:20:04.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.296+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66140d0070 con 0x7f66140cf2b0 2023-10-21T16:20:04.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.297+0000 7f660affd640 1 -- 172.21.15.53:0/12845424 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6604021020 con 0x7f66140cf2b0 2023-10-21T16:20:04.299 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:20:04.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 msgr2=0x7f65f806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 msgr2=0x7f661415b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f660400e870 tx=0x7f660400ed30 comp rx=0 tx=0).stop 2023-10-21T16:20:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 shutdown_connections 2023-10-21T16:20:04.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66140cf690 0x7f661415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66140cf2b0 0x7f661415b660 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.299+0000 7f661b167640 1 --2- 172.21.15.53:0/12845424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66140ce510 0x7f66140ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.300+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 >> 172.21.15.53:0/12845424 conn(0x7f661409bc40 msgr2=0x7f661406c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.300+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 shutdown_connections 2023-10-21T16:20:04.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.300+0000 7f661b167640 1 -- 172.21.15.53:0/12845424 wait complete. 2023-10-21T16:20:04.313 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.312+0000 7f7c59e38040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:20:04.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:20:04.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.579+0000 7ff90d11d640 1 Processor -- start 2023-10-21T16:20:04.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.579+0000 7ff90d11d640 1 -- start start 2023-10-21T16:20:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90d11d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff90806c3a0 0x7ff9080c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90d11d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff90815aa00 0x7ff9080b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90d11d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90815b310 con 0x7ff90806db60 2023-10-21T16:20:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90d11d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9080ca230 con 0x7ff90815aa00 2023-10-21T16:20:04.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90d11d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90806d290 con 0x7ff90806c3a0 2023-10-21T16:20:04.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff90815aa00 0x7ff9080b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff90815aa00 0x7ff9080b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47202/0 (socket says 172.21.15.53:47202) 2023-10-21T16:20:04.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff90759e640 1 -- 172.21.15.53:0/2695628252 learned_addr learned my addr 172.21.15.53:0/2695628252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:04.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.581+0000 7ff906d9d640 1 --2- 172.21.15.53:0/2695628252 >> v2:172.21.15.124:3300/0 conn(0x7ff90806c3a0 0x7ff9080c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.582+0000 7ff90759e640 1 -- 172.21.15.53:0/2695628252 >> v2:172.21.15.124:3300/0 conn(0x7ff90806c3a0 msgr2=0x7ff9080c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:04.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.582+0000 7ff90759e640 1 --2- 172.21.15.53:0/2695628252 >> v2:172.21.15.124:3300/0 conn(0x7ff90806c3a0 0x7ff9080c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.582+0000 7ff90759e640 1 -- 172.21.15.53:0/2695628252 >> v1:172.21.15.124:6789/0 conn(0x7ff90806db60 legacy=0x7ff90806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.582+0000 7ff90759e640 1 -- 172.21.15.53:0/2695628252 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080c9860 con 0x7ff90815aa00 2023-10-21T16:20:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.583+0000 7ff90759e640 1 --2- 172.21.15.53:0/2695628252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90815aa00 0x7ff9080b2a90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff9080716b0 tx=0x7ff90002ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b02421b376fce server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.583+0000 7ff90659c640 1 -- 172.21.15.53:0/2695628252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90002ec80 con 0x7ff90815aa00 2023-10-21T16:20:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.583+0000 7ff90659c640 1 -- 172.21.15.53:0/2695628252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90002ede0 con 0x7ff90815aa00 2023-10-21T16:20:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.583+0000 7ff90659c640 1 -- 172.21.15.53:0/2695628252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff900036660 con 0x7ff90815aa00 2023-10-21T16:20:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 -- 172.21.15.53:0/2695628252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90815aa00 msgr2=0x7ff9080b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2695628252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90815aa00 0x7ff9080b2a90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff9080716b0 tx=0x7ff90002ded0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 -- 172.21.15.53:0/2695628252 shutdown_connections 2023-10-21T16:20:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2695628252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90815aa00 0x7ff9080b2a90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2695628252 >> v2:172.21.15.124:3300/0 conn(0x7ff90806c3a0 0x7ff9080c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 -- 172.21.15.53:0/2695628252 >> 172.21.15.53:0/2695628252 conn(0x7ff90806a5d0 msgr2=0x7ff9080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 -- 172.21.15.53:0/2695628252 shutdown_connections 2023-10-21T16:20:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.584+0000 7ff90d11d640 1 -- 172.21.15.53:0/2695628252 wait complete. 2023-10-21T16:20:04.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.586+0000 7ff90d11d640 1 Processor -- start 2023-10-21T16:20:04.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.586+0000 7ff90d11d640 1 -- start start 2023-10-21T16:20:04.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff90806c3a0 0x7ff9080c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff90815aa00 0x7ff908156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081f2a80 con 0x7ff90806db60 2023-10-21T16:20:04.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9081f25f0 con 0x7ff90806c3a0 2023-10-21T16:20:04.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff90d11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9080c27b0 con 0x7ff90815aa00 2023-10-21T16:20:04.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff8fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff8fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47216/0 (socket says 172.21.15.53:47216) 2023-10-21T16:20:04.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.587+0000 7ff8fe59c640 1 -- 172.21.15.53:0/2870889909 learned_addr learned my addr 172.21.15.53:0/2870889909 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:04.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff906d9d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff90806c3a0 0x7ff9080c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff90759e640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff90815aa00 0x7ff908156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:04.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 -- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff90815aa00 msgr2=0x7ff908156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff90815aa00 0x7ff908156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 -- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff90806c3a0 msgr2=0x7ff9080c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff90806c3a0 0x7ff9080c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 -- 172.21.15.53:0/2870889909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000094d0 con 0x7ff90806db60 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fe59c640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c450 tx=0x7ff8f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00052d0 con 0x7ff90806db60 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.588+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0005430 con 0x7ff90806db60 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.589+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0012870 con 0x7ff90806db60 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.589+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9081f1cc0 con 0x7ff90806db60 2023-10-21T16:20:04.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.589+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9081f2280 con 0x7ff90806db60 2023-10-21T16:20:04.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.590+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9080c7820 con 0x7ff90806db60 2023-10-21T16:20:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.591+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff8f00129d0 con 0x7ff90806db60 2023-10-21T16:20:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.592+0000 7ff8fffff640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 0x7ff8e806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.592+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7ff8f0089680 con 0x7ff90806db60 2023-10-21T16:20:04.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.592+0000 7ff906d9d640 1 -- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 msgr2=0x7ff8e806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.592+0000 7ff906d9d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 0x7ff8e806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:04.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.596+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8f0054100 con 0x7ff90806db60 2023-10-21T16:20:04.614 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.612+0000 7ff9aca2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:20:04.728 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.727+0000 7f7c59e38040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:20:04.775 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.773+0000 7ff9aca2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:20:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.788+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff90815b310 con 0x7ff90806db60 2023-10-21T16:20:04.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.789+0000 7ff8fffff640 1 -- 172.21.15.53:0/2870889909 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff90815b310 con 0x7ff90806db60 2023-10-21T16:20:04.791 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:20:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.791+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 msgr2=0x7ff8e806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.791+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 0x7ff8e806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 msgr2=0x7ff9080c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c450 tx=0x7ff8f000c910 comp rx=0 tx=0).stop 2023-10-21T16:20:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 shutdown_connections 2023-10-21T16:20:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7ff8e806b0b0 0x7ff8e806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff90815aa00 0x7ff908156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff90806db60 0x7ff9080c2270 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 --2- 172.21.15.53:0/2870889909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff90806c3a0 0x7ff9080c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 >> 172.21.15.53:0/2870889909 conn(0x7ff90806a5d0 msgr2=0x7ff90815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 shutdown_connections 2023-10-21T16:20:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:04.792+0000 7ff90d11d640 1 -- 172.21.15.53:0/2870889909 wait complete. 2023-10-21T16:20:04.812 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:20:04.882 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.880+0000 7ff9aca2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:20:04.899 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:04.898+0000 7f7c59e38040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:20:04.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:04.929+0000 7ff9aca2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:20:05.016 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.014+0000 7f7c59e38040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:20:05.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.018+0000 7fd413034640 1 Processor -- start 2023-10-21T16:20:05.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.019+0000 7fd413034640 1 -- start start 2023-10-21T16:20:05.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd413034640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd40c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd413034640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0b8b00 con 0x7fd40c15cc70 2023-10-21T16:20:05.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd413034640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c0b8540 con 0x7fd40c0b61f0 2023-10-21T16:20:05.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd413034640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b8820 con 0x7fd40c0d31a0 2023-10-21T16:20:05.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd410dd0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd40c15cc70 0x7fd40c0cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53530/0 (socket says 172.21.15.53:53530) 2023-10-21T16:20:05.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.020+0000 7fd410dd0640 1 -- 172.21.15.53:0/821288980 learned_addr learned my addr 172.21.15.53:0/821288980 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:05.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40bfff640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd40c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237579448 0 0) 0x7fd40c0b8b00 con 0x7fd40c15cc70 2023-10-21T16:20:05.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3fc003520 con 0x7fd40c15cc70 2023-10-21T16:20:05.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 949520980 0 0) 0x7fd3fc003520 con 0x7fd40c15cc70 2023-10-21T16:20:05.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 msgr2=0x7fd40c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:05.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd40c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 >> v1:172.21.15.124:6789/0 conn(0x7fd40c0b61f0 legacy=0x7fd40c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0cd270 con 0x7fd40c15cc70 2023-10-21T16:20:05.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd400001670 con 0x7fd40c15cc70 2023-10-21T16:20:05.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 >> v1:172.21.15.53:6789/0 conn(0x7fd40c15cc70 legacy=0x7fd40c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd3fc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3fc003f70 0x7fd3fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc004d00 0x7fd3fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b8820 con 0x7fd3fc004d00 2023-10-21T16:20:05.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc00bb70 con 0x7fd3fc003f70 2023-10-21T16:20:05.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.021+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc00bd30 con 0x7fd40c0d31a0 2023-10-21T16:20:05.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc004d00 0x7fd3fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd410dd0640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd3fc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd40bfff640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3fc003f70 0x7fd3fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 -- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 msgr2=0x7fd3fc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:05.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd40c0d31a0 0x7fd3fc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 -- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3fc003f70 msgr2=0x7fd3fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3fc003f70 0x7fd3fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.022+0000 7fd4115d1640 1 -- 172.21.15.53:0/821288980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc003520 con 0x7fd3fc004d00 2023-10-21T16:20:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.023+0000 7fd4115d1640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc004d00 0x7fd3fc005180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd404001e20 tx=0x7fd3fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f95aa38775fe8f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404003b70 con 0x7fd3fc004d00 2023-10-21T16:20:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd404003cd0 con 0x7fd3fc004d00 2023-10-21T16:20:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd40b7fe640 1 -- 172.21.15.53:0/821288980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404005310 con 0x7fd3fc004d00 2023-10-21T16:20:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 -- 172.21.15.53:0/821288980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc004d00 msgr2=0x7fd3fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 --2- 172.21.15.53:0/821288980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3fc004d00 0x7fd3fc005180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd404001e20 tx=0x7fd3fc0057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd4115d1640 1 -- 172.21.15.53:0/821288980 reap_dead start 2023-10-21T16:20:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 -- 172.21.15.53:0/821288980 shutdown_connections 2023-10-21T16:20:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 -- 172.21.15.53:0/821288980 >> 172.21.15.53:0/821288980 conn(0x7fd40c0718a0 msgr2=0x7fd40c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 -- 172.21.15.53:0/821288980 shutdown_connections 2023-10-21T16:20:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.024+0000 7fd413034640 1 -- 172.21.15.53:0/821288980 wait complete. 2023-10-21T16:20:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.026+0000 7fd413034640 1 Processor -- start 2023-10-21T16:20:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.026+0000 7fd413034640 1 -- start start 2023-10-21T16:20:05.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 0x7fd40c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd40c15a880 0x7fd40c15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c15b240 con 0x7fd40c0cb460 2023-10-21T16:20:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c15d350 con 0x7fd40c15a880 2023-10-21T16:20:05.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.027+0000 7fd413034640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c15d5f0 con 0x7fd3fc004d00 2023-10-21T16:20:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47228/0 (socket says 172.21.15.53:47228) 2023-10-21T16:20:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 -- 172.21.15.53:0/3800306669 learned_addr learned my addr 172.21.15.53:0/3800306669 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:05.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd4115d1640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd40c15a880 0x7fd40c15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd410dd0640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 0x7fd40c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 msgr2=0x7fd40c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 0x7fd40c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd40c15a880 msgr2=0x7fd40c15ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd40c15a880 0x7fd40c15ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 -- 172.21.15.53:0/3800306669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c15d8d0 con 0x7fd40c0cb460 2023-10-21T16:20:05.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd410dd0640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 0x7fd40c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:05.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.028+0000 7fd40bfff640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8010860 tx=0x7fd3f8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.029+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8009610 con 0x7fd40c0cb460 2023-10-21T16:20:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.029+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800eb60 con 0x7fd40c0cb460 2023-10-21T16:20:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.029+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f801c400 con 0x7fd40c0cb460 2023-10-21T16:20:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.029+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c15dbb0 con 0x7fd40c0cb460 2023-10-21T16:20:05.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.029+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c15e170 con 0x7fd40c0cb460 2023-10-21T16:20:05.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.030+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c04e160 con 0x7fd40c0cb460 2023-10-21T16:20:05.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.031+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fd3f801c560 con 0x7fd40c0cb460 2023-10-21T16:20:05.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.032+0000 7fd4097fa640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 0x7fd3f006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.032+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fd3f808a370 con 0x7fd40c0cb460 2023-10-21T16:20:05.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.032+0000 7fd410dd0640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 msgr2=0x7fd3f006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:05.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.032+0000 7fd410dd0640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 0x7fd3f006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:05.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.036+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3f8054d80 con 0x7fd40c0cb460 2023-10-21T16:20:05.048 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:05.039+0000 7ff9aca2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:05.072 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.070+0000 7f7c59e38040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:20:05.147 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:05.145+0000 7ff9aca2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:20:05.187 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.186+0000 7f7c59e38040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:05.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.233+0000 7fd410dd0640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 msgr2=0x7fd3f006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:05.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.233+0000 7fd410dd0640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 0x7fd3f006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:05.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.247+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd40c0b7570 con 0x7fd40c0cb460 2023-10-21T16:20:05.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.248+0000 7fd4097fa640 1 -- 172.21.15.53:0/3800306669 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v121) v1 ==== 80+0+534 (secure 0 0 0) 0x7fd3f80583a0 con 0x7fd40c0cb460 2023-10-21T16:20:05.250 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3937751236 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.250 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2028143540 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.251 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/130280090 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.251 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1496413032 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.251 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/1967358544 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.251 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/1967358544 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.251 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3651970368 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.252 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/28232742 2023-10-22T16:18:58.493629+0000 2023-10-21T16:20:05.252 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2272255357 2023-10-21T17:18:51.635575+0000 2023-10-21T16:20:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:listed 9 entries 2023-10-21T16:20:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 msgr2=0x7fd3f006d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 0x7fd3f006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 msgr2=0x7fd40c0cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8010860 tx=0x7fd3f8010d20 comp rx=0 tx=0).stop 2023-10-21T16:20:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 shutdown_connections 2023-10-21T16:20:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fd3f006b0e0 0x7fd3f006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd40c15a880 0x7fd40c15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd40c0cb460 0x7fd40c0cb8e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 --2- 172.21.15.53:0/3800306669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3fc004d00 0x7fd40c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 >> 172.21.15.53:0/3800306669 conn(0x7fd40c0718a0 msgr2=0x7fd40c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:05.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 shutdown_connections 2023-10-21T16:20:05.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.251+0000 7fd413034640 1 -- 172.21.15.53:0/3800306669 wait complete. 2023-10-21T16:20:05.271 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:20:05.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:05.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:20:05.301 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.300+0000 7f7c59e38040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:20:05.302 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:05.299+0000 7ff9aca2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:05.349 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:05.348+0000 7ff9aca2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:05.367 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:20:05.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:05.368 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:20:05.463 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:20:05.464 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:05.472 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.471+0000 7f7c59e38040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:05.473 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:05.466+0000 7ff9aca2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:20:05.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:20:05.520 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:05.520 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:05.530 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.523+0000 7f7c59e38040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:05.578 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:20:05.579 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:20:05.579 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:05.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:20:05.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:05.651 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:05.650+0000 7f7c59e38040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:20:05.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.845+0000 7f15102f2640 1 Processor -- start 2023-10-21T16:20:05.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.845+0000 7f15102f2640 1 -- start start 2023-10-21T16:20:05.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.846+0000 7f15102f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1508160940 0x7f15080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f15102f2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080c48f0 con 0x7f150806c890 2023-10-21T16:20:05.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f15102f2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080c3ce0 con 0x7f150815cd90 2023-10-21T16:20:05.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f15102f2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080c4d80 con 0x7f1508160940 2023-10-21T16:20:05.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f150d88d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1508160940 0x7f15080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f150d88d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1508160940 0x7f15080709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47242/0 (socket says 172.21.15.53:47242) 2023-10-21T16:20:05.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.847+0000 7f150d88d640 1 -- 172.21.15.53:0/351586327 learned_addr learned my addr 172.21.15.53:0/351586327 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:05.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.848+0000 7f150d88d640 1 -- 172.21.15.53:0/351586327 >> v1:172.21.15.124:6789/0 conn(0x7f150815cd90 legacy=0x7f15080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.848+0000 7f150d88d640 1 -- 172.21.15.53:0/351586327 >> v1:172.21.15.53:6789/0 conn(0x7f150806c890 legacy=0x7f150806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.848+0000 7f150d88d640 1 -- 172.21.15.53:0/351586327 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080c3fc0 con 0x7f1508160940 2023-10-21T16:20:05.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.849+0000 7f150d88d640 1 --2- 172.21.15.53:0/351586327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1508160940 0x7f15080709e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009870 tx=0x7f14f802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4deed57bc0b409 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:05.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.849+0000 7f150d08c640 1 -- 172.21.15.53:0/351586327 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f802f840 con 0x7f1508160940 2023-10-21T16:20:05.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.849+0000 7f150d08c640 1 -- 172.21.15.53:0/351586327 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f802f9a0 con 0x7f1508160940 2023-10-21T16:20:05.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.849+0000 7f150d08c640 1 -- 172.21.15.53:0/351586327 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f802fd30 con 0x7f1508160940 2023-10-21T16:20:05.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 -- 172.21.15.53:0/351586327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1508160940 msgr2=0x7f15080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 --2- 172.21.15.53:0/351586327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1508160940 0x7f15080709e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009870 tx=0x7f14f802ba20 comp rx=0 tx=0).stop 2023-10-21T16:20:05.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 -- 172.21.15.53:0/351586327 shutdown_connections 2023-10-21T16:20:05.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 --2- 172.21.15.53:0/351586327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1508160940 0x7f15080709e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 -- 172.21.15.53:0/351586327 >> 172.21.15.53:0/351586327 conn(0x7f150806a380 msgr2=0x7f15080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:05.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 -- 172.21.15.53:0/351586327 shutdown_connections 2023-10-21T16:20:05.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.850+0000 7f15102f2640 1 -- 172.21.15.53:0/351586327 wait complete. 2023-10-21T16:20:05.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.852+0000 7f15102f2640 1 Processor -- start 2023-10-21T16:20:05.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.852+0000 7f15102f2640 1 -- start start 2023-10-21T16:20:05.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150815cd90 0x7f15081e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 0x7f15081e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150815b180 con 0x7f150806c890 2023-10-21T16:20:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081e7950 con 0x7f1508160940 2023-10-21T16:20:05.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f15102f2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15081e7c30 con 0x7f150815cd90 2023-10-21T16:20:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47252/0 (socket says 172.21.15.53:47252) 2023-10-21T16:20:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 -- 172.21.15.53:0/898315818 learned_addr learned my addr 172.21.15.53:0/898315818 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150815cd90 msgr2=0x7f15081e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150815cd90 0x7f15081e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.853+0000 7f150e08e640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 msgr2=0x7f15081e6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f150e88f640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 0x7f15081e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f150e08e640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 0x7f15081e6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f150e08e640 1 -- 172.21.15.53:0/898315818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f80094d0 con 0x7f150806c890 2023-10-21T16:20:05.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f150e88f640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 0x7f15081e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:05.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f150e08e640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f150400c450 tx=0x7f150400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:05.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040052d0 con 0x7f150806c890 2023-10-21T16:20:05.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504005430 con 0x7f150806c890 2023-10-21T16:20:05.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504012870 con 0x7f150806c890 2023-10-21T16:20:05.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081e7f10 con 0x7f150806c890 2023-10-21T16:20:05.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.854+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150815bcd0 con 0x7f150806c890 2023-10-21T16:20:05.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.855+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15081e8150 con 0x7f150806c890 2023-10-21T16:20:05.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.857+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f15040129d0 con 0x7f150806c890 2023-10-21T16:20:05.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.857+0000 7f14feffd640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 0x7f14ec06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:05.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.858+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7f1504089860 con 0x7f150806c890 2023-10-21T16:20:05.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.858+0000 7f150d88d640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 msgr2=0x7f14ec06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:05.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.858+0000 7f150d88d640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 0x7f14ec06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:05.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:05.866+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15081e8150 con 0x7f150806c890 2023-10-21T16:20:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.059+0000 7f150d88d640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 msgr2=0x7f14ec06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.059+0000 7f150d88d640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 0x7f14ec06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.096+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15081e8150 con 0x7f150806c890 2023-10-21T16:20:06.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.097+0000 7f14feffd640 1 -- 172.21.15.53:0/898315818 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f15081e8150 con 0x7f150806c890 2023-10-21T16:20:06.098 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:06.099 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:20:06.099 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:20:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 msgr2=0x7f14ec06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 0x7f14ec06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 msgr2=0x7f15081507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f150400c450 tx=0x7f150400c910 comp rx=0 tx=0).stop 2023-10-21T16:20:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 shutdown_connections 2023-10-21T16:20:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7f14ec06b0b0 0x7f14ec06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1508160940 0x7f15081e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f150815cd90 0x7f15081e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 --2- 172.21.15.53:0/898315818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f150806c890 0x7f15081507a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.099+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 >> 172.21.15.53:0/898315818 conn(0x7f150806a380 msgr2=0x7f15080ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.100+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 shutdown_connections 2023-10-21T16:20:06.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.100+0000 7f15102f2640 1 -- 172.21.15.53:0/898315818 wait complete. 2023-10-21T16:20:06.119 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:06.305 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.303+0000 7ff9aca2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:20:06.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.355+0000 7fa90e8c9640 1 Processor -- start 2023-10-21T16:20:06.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.355+0000 7fa90e8c9640 1 -- start start 2023-10-21T16:20:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90e8c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 0x7fa9080cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90e8c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90815b2b0 con 0x7fa9080cb3b0 2023-10-21T16:20:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90e8c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90815b590 con 0x7fa90815ab40 2023-10-21T16:20:06.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90e8c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815bf30 con 0x7fa9080c90b0 2023-10-21T16:20:06.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90ce66640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa9080cb3b0 0x7fa90815e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53554/0 (socket says 172.21.15.53:53554) 2023-10-21T16:20:06.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.357+0000 7fa90ce66640 1 -- 172.21.15.53:0/966672501 learned_addr learned my addr 172.21.15.53:0/966672501 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:06.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa9077fe640 1 --2- 172.21.15.53:0/966672501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 0x7fa9080cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026573111 0 0) 0x7fa90815b2b0 con 0x7fa9080cb3b0 2023-10-21T16:20:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f0003520 con 0x7fa9080cb3b0 2023-10-21T16:20:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa9077fe640 1 -- 172.21.15.53:0/966672501 >> v1:172.21.15.124:6789/0 conn(0x7fa90815ab40 legacy=0x7fa9080ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa9077fe640 1 -- 172.21.15.53:0/966672501 >> v1:172.21.15.53:6789/0 conn(0x7fa9080cb3b0 legacy=0x7fa90815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551932257 0 0) 0x7fa90815b590 con 0x7fa90815ab40 2023-10-21T16:20:06.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.358+0000 7fa9077fe640 1 -- 172.21.15.53:0/966672501 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90815b870 con 0x7fa9080c90b0 2023-10-21T16:20:06.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.360+0000 7fa9077fe640 1 --2- 172.21.15.53:0/966672501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 0x7fa9080cae70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0097d0 tx=0x7fa8fc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aca53c9e75fd62 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:06.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.361+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc030800 con 0x7fa9080c90b0 2023-10-21T16:20:06.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.361+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc030960 con 0x7fa9080c90b0 2023-10-21T16:20:06.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.361+0000 7fa906ffd640 1 -- 172.21.15.53:0/966672501 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc030cd0 con 0x7fa9080c90b0 2023-10-21T16:20:06.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.361+0000 7fa90e8c9640 1 -- 172.21.15.53:0/966672501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 msgr2=0x7fa9080cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.361+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/966672501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 0x7fa9080cae70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0097d0 tx=0x7fa8fc02b940 comp rx=0 tx=0).stop 2023-10-21T16:20:06.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.362+0000 7fa90e8c9640 1 -- 172.21.15.53:0/966672501 shutdown_connections 2023-10-21T16:20:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.362+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/966672501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9080c90b0 0x7fa9080cae70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.362+0000 7fa90e8c9640 1 -- 172.21.15.53:0/966672501 >> 172.21.15.53:0/966672501 conn(0x7fa90806a380 msgr2=0x7fa9080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.362+0000 7fa90e8c9640 1 -- 172.21.15.53:0/966672501 shutdown_connections 2023-10-21T16:20:06.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.362+0000 7fa90e8c9640 1 -- 172.21.15.53:0/966672501 wait complete. 2023-10-21T16:20:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.364+0000 7fa90e8c9640 1 Processor -- start 2023-10-21T16:20:06.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.364+0000 7fa90e8c9640 1 -- start start 2023-10-21T16:20:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 0x7fa9080c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90815ab40 0x7fa9080c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9081f25b0 con 0x7fa9080c90b0 2023-10-21T16:20:06.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa908156010 con 0x7fa9080cb3b0 2023-10-21T16:20:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa90e8c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080adc90 con 0x7fa90815ab40 2023-10-21T16:20:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa907fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa907fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47258/0 (socket says 172.21.15.53:47258) 2023-10-21T16:20:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.365+0000 7fa907fff640 1 -- 172.21.15.53:0/137380846 learned_addr learned my addr 172.21.15.53:0/137380846 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa90ce66640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90815ab40 0x7fa9080c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa9077fe640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 0x7fa9080c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90815ab40 msgr2=0x7fa9080c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90815ab40 0x7fa9080c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 msgr2=0x7fa9080c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 0x7fa9080c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 -- 172.21.15.53:0/137380846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009400 con 0x7fa9080c90b0 2023-10-21T16:20:06.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa9077fe640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 0x7fa9080c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:06.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.366+0000 7fa907fff640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c4e0 tx=0x7fa8f800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.367+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8005480 con 0x7fa9080c90b0 2023-10-21T16:20:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.367+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80035f0 con 0x7fa9080c90b0 2023-10-21T16:20:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.367+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80039c0 con 0x7fa9080c90b0 2023-10-21T16:20:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.367+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080adec0 con 0x7fa9080c90b0 2023-10-21T16:20:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.367+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9081f2a80 con 0x7fa9080c90b0 2023-10-21T16:20:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.368+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908067490 con 0x7fa9080c90b0 2023-10-21T16:20:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.370+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 50) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa8f8003b20 con 0x7fa9080c90b0 2023-10-21T16:20:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.371+0000 7fa904ff9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 0x7fa8f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.371+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fa8f8089320 con 0x7fa9080c90b0 2023-10-21T16:20:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.371+0000 7fa9077fe640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 msgr2=0x7fa8f406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.371+0000 7fa9077fe640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 0x7fa8f406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:06.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.376+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8f8053d40 con 0x7fa9080c90b0 2023-10-21T16:20:06.530 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:06.529+0000 7f7c59e38040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:20:06.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.568+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9080b0ef0 con 0x7fa9080c90b0 2023-10-21T16:20:06.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.570+0000 7fa904ff9640 1 -- 172.21.15.53:0/137380846 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17549 (secure 0 0 0) 0x7fa8f8017090 con 0x7fa9080c90b0 2023-10-21T16:20:06.572 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:06.572 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":121,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:19:59.623595+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3937751236":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2028143540":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/130280090":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/1496413032":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6833/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6832/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/3651970368":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/28232742":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2272255357":"2023-10-21T17:18:51.635575+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:06.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.572+0000 7fa9077fe640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 msgr2=0x7fa8f406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2567187932 2023-10-21T16:20:06.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.572+0000 7fa9077fe640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 0x7fa8f406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:06.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.573+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 msgr2=0x7fa8f406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.573+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 0x7fa8f406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 msgr2=0x7fa9080ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c4e0 tx=0x7fa8f800c9a0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 shutdown_connections 2023-10-21T16:20:06.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:6832/2567187932,v1:172.21.15.124:6833/2567187932] conn(0x7fa8f406b0b0 0x7fa8f406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90815ab40 0x7fa9080c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9080cb3b0 0x7fa9080c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 --2- 172.21.15.53:0/137380846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9080c90b0 0x7fa9080ad750 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 >> 172.21.15.53:0/137380846 conn(0x7fa90806a380 msgr2=0x7fa90806d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:06.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 shutdown_connections 2023-10-21T16:20:06.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.574+0000 7fa90e8c9640 1 -- 172.21.15.53:0/137380846 wait complete. 2023-10-21T16:20:06.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:18:41.952733+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:18:42.974741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:20:06.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:20:06.601 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.599+0000 7ff9aca2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:20:06.658 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.656+0000 7ff9aca2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:20:06.772 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.770+0000 7ff9aca2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:20:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.803+0000 7fbd9bfff640 1 Processor -- start 2023-10-21T16:20:06.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.803+0000 7fbd9bfff640 1 -- start start 2023-10-21T16:20:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 0x7fbd9c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd9c15a020 0x7fbd9c0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0c9380 con 0x7fbd9c156840 2023-10-21T16:20:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9bfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c9660 con 0x7fbd9c15a020 2023-10-21T16:20:06.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c9940 con 0x7fbd9c156c20 2023-10-21T16:20:06.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbd9c15a020 0x7fbd9c0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbd9c156840 0x7fbd9c0c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53570/0 (socket says 172.21.15.53:53570) 2023-10-21T16:20:06.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd9affd640 1 -- 172.21.15.53:0/1341642678 learned_addr learned my addr 172.21.15.53:0/1341642678 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:06.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.805+0000 7fbd927fc640 1 --2- 172.21.15.53:0/1341642678 >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 0x7fbd9c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.806+0000 7fbd9b7fe640 1 -- 172.21.15.53:0/1341642678 >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 msgr2=0x7fbd9c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.806+0000 7fbd9b7fe640 1 --2- 172.21.15.53:0/1341642678 >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 0x7fbd9c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.806+0000 7fbd9b7fe640 1 -- 172.21.15.53:0/1341642678 >> v1:172.21.15.53:6789/0 conn(0x7fbd9c156840 legacy=0x7fbd9c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.806+0000 7fbd9b7fe640 1 -- 172.21.15.53:0/1341642678 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0c9c20 con 0x7fbd9c15a020 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.806+0000 7fbd927fc640 1 --2- 172.21.15.53:0/1341642678 >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 0x7fbd9c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.807+0000 7fbd9b7fe640 1 --2- 172.21.15.53:0/1341642678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0b04a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd9402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cffa2e71243a7f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.807+0000 7fbd9a7fc640 1 -- 172.21.15.53:0/1341642678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94030710 con 0x7fbd9c15a020 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.807+0000 7fbd9a7fc640 1 -- 172.21.15.53:0/1341642678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9402e050 con 0x7fbd9c15a020 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.807+0000 7fbd9a7fc640 1 -- 172.21.15.53:0/1341642678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94030a90 con 0x7fbd9c15a020 2023-10-21T16:20:06.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 -- 172.21.15.53:0/1341642678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 msgr2=0x7fbd9c0b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/1341642678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0b04a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd9402b710 comp rx=0 tx=0).stop 2023-10-21T16:20:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 -- 172.21.15.53:0/1341642678 shutdown_connections 2023-10-21T16:20:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/1341642678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0b04a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/1341642678 >> v2:172.21.15.124:3300/0 conn(0x7fbd9c156c20 0x7fbd9c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 -- 172.21.15.53:0/1341642678 >> 172.21.15.53:0/1341642678 conn(0x7fbd9c06c010 msgr2=0x7fbd9c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:06.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 -- 172.21.15.53:0/1341642678 shutdown_connections 2023-10-21T16:20:06.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.808+0000 7fbd9bfff640 1 -- 172.21.15.53:0/1341642678 wait complete. 2023-10-21T16:20:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.810+0000 7fbd9bfff640 1 Processor -- start 2023-10-21T16:20:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.810+0000 7fbd9bfff640 1 -- start start 2023-10-21T16:20:06.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd9c156840 0x7fbd9c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd9c0acc90 0x7fbd9c15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c156030 con 0x7fbd9c15a020 2023-10-21T16:20:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c67a0 con 0x7fbd9c156840 2023-10-21T16:20:06.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c0c6a80 con 0x7fbd9c0acc90 2023-10-21T16:20:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47280/0 (socket says 172.21.15.53:47280) 2023-10-21T16:20:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd927fc640 1 -- 172.21.15.53:0/3155913145 learned_addr learned my addr 172.21.15.53:0/3155913145 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9b7fe640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd9c0acc90 0x7fbd9c15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.811+0000 7fbd9affd640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd9c156840 0x7fbd9c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 -- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd9c0acc90 msgr2=0x7fbd9c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd9c0acc90 0x7fbd9c15b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 -- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd9c156840 msgr2=0x7fbd9c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd9c156840 0x7fbd9c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:06.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 -- 172.21.15.53:0/3155913145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94009400 con 0x7fbd9c15a020 2023-10-21T16:20:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd927fc640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e7e0 tx=0x7fbd8400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004030 con 0x7fbd9c15a020 2023-10-21T16:20:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84004190 con 0x7fbd9c15a020 2023-10-21T16:20:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004520 con 0x7fbd9c15a020 2023-10-21T16:20:06.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0c6d60 con 0x7fbd9c15a020 2023-10-21T16:20:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.812+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0c7320 con 0x7fbd9c15a020 2023-10-21T16:20:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.813+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c0c74e0 con 0x7fbd9c15a020 2023-10-21T16:20:06.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.815+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 51) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbd8400c6d0 con 0x7fbd9c15a020 2023-10-21T16:20:06.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.816+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7fbd84089280 con 0x7fbd9c15a020 2023-10-21T16:20:06.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:06.820+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd84054230 con 0x7fbd9c15a020 2023-10-21T16:20:06.863 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff9314fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.863 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff9314fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.863 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff9314fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.863 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff9314fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff9314fa640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff92dcf3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff92dcf3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff92dcf3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.864 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff92dcf3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.865 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:06.861+0000 7ff92dcf3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:06.865 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:06.864+0000 7f7c59e38040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:20:06.928 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:06.926+0000 7f7c59e38040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:20:07.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.007+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd9c074150 con 0x7fbd9c15a020 2023-10-21T16:20:07.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.007+0000 7fbd98ff9640 1 -- 172.21.15.53:0/3155913145 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd84057850 con 0x7fbd9c15a020 2023-10-21T16:20:07.009 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:20:07.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 msgr2=0x7fbd9c0c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e7e0 tx=0x7fbd8400eca0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 shutdown_connections 2023-10-21T16:20:07.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd9c0acc90 0x7fbd9c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd9c15a020 0x7fbd9c0c1fd0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 --2- 172.21.15.53:0/3155913145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd9c156840 0x7fbd9c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 >> 172.21.15.53:0/3155913145 conn(0x7fbd9c06c010 msgr2=0x7fbd9c157010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:07.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 shutdown_connections 2023-10-21T16:20:07.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.010+0000 7fbd9bfff640 1 -- 172.21.15.53:0/3155913145 wait complete. 2023-10-21T16:20:07.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:20:07.048 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:07.047+0000 7f7c59e38040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:20:07.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.250+0000 7f4f5b0bb640 1 Processor -- start 2023-10-21T16:20:07.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.250+0000 7f4f5b0bb640 1 -- start start 2023-10-21T16:20:07.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5b0bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4f540b0680 0x7f4f540c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5b0bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f540c94f0 con 0x7f4f5415cae0 2023-10-21T16:20:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5b0bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f540ccbc0 con 0x7f4f540c8b40 2023-10-21T16:20:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5b0bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f540ce1f0 con 0x7f4f540b0680 2023-10-21T16:20:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5a0b9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4f5415cae0 0x7f4f540cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53586/0 (socket says 172.21.15.53:53586) 2023-10-21T16:20:07.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5a0b9640 1 -- 172.21.15.53:0/2596861533 learned_addr learned my addr 172.21.15.53:0/2596861533 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:07.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.252+0000 7f4f5a8ba640 1 --2- 172.21.15.53:0/2596861533 >> v2:172.21.15.124:3300/0 conn(0x7f4f540b0680 0x7f4f540c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423182610 0 0) 0x7f4f540c94f0 con 0x7f4f5415cae0 2023-10-21T16:20:07.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f40003520 con 0x7f4f5415cae0 2023-10-21T16:20:07.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2553289434 0 0) 0x7f4f40003520 con 0x7f4f5415cae0 2023-10-21T16:20:07.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 >> v2:172.21.15.124:3300/0 conn(0x7f4f540b0680 msgr2=0x7f4f540c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:07.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 --2- 172.21.15.53:0/2596861533 >> v2:172.21.15.124:3300/0 conn(0x7f4f540b0680 0x7f4f540c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 >> v1:172.21.15.124:6789/0 conn(0x7f4f540c8b40 legacy=0x7f4f540aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540c9720 con 0x7f4f5415cae0 2023-10-21T16:20:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4f50001670 con 0x7f4f5415cae0 2023-10-21T16:20:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 >> v1:172.21.15.53:6789/0 conn(0x7f4f5415cae0 legacy=0x7f4f540cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f40003970 0x7f4f40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f40005180 0x7f4f4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540ce1f0 con 0x7f4f400047c0 2023-10-21T16:20:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4000c020 con 0x7f4f40003970 2023-10-21T16:20:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4000c1e0 con 0x7f4f40005180 2023-10-21T16:20:07.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.253+0000 7f4f598b8640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f5a0b9640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f40003970 0x7f4f40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f5a8ba640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f40005180 0x7f4f4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f598b8640 1 -- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f40005180 msgr2=0x7f4f4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f598b8640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f40005180 0x7f4f4000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f598b8640 1 -- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f40003970 msgr2=0x7f4f40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f598b8640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f40003970 0x7f4f40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f598b8640 1 -- 172.21.15.53:0/2596861533 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f40003520 con 0x7f4f400047c0 2023-10-21T16:20:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f5a0b9640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f40003970 0x7f4f40003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.254+0000 7f4f5a8ba640 1 -- 172.21.15.53:0/2596861533 reap_dead start 2023-10-21T16:20:07.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.255+0000 7f4f598b8640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f40004c40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4f44001750 tx=0x7f4f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e7c848707f63c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:07.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.255+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440052d0 con 0x7f4f400047c0 2023-10-21T16:20:07.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.255+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4402e050 con 0x7f4f400047c0 2023-10-21T16:20:07.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.255+0000 7f4f590b7640 1 -- 172.21.15.53:0/2596861533 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44003830 con 0x7f4f400047c0 2023-10-21T16:20:07.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 msgr2=0x7f4f40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f40004c40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4f44001750 tx=0x7f4f400057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/2596861533 shutdown_connections 2023-10-21T16:20:07.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/2596861533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f40004c40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/2596861533 >> 172.21.15.53:0/2596861533 conn(0x7f4f5406c050 msgr2=0x7f4f5406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:07.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/2596861533 shutdown_connections 2023-10-21T16:20:07.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.256+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/2596861533 wait complete. 2023-10-21T16:20:07.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.258+0000 7f4f5b0bb640 1 Processor -- start 2023-10-21T16:20:07.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.258+0000 7f4f5b0bb640 1 -- start start 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f5415f600 0x7f4f5415fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f540c5630 0x7f4f540c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415ffa0 con 0x7f4f400047c0 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540c63a0 con 0x7f4f5415f600 2023-10-21T16:20:07.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5b0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540c6680 con 0x7f4f540c5630 2023-10-21T16:20:07.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5a0b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.259+0000 7f4f5a0b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47298/0 (socket says 172.21.15.53:47298) 2023-10-21T16:20:07.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 -- 172.21.15.53:0/1679975803 learned_addr learned my addr 172.21.15.53:0/1679975803 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:07.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f598b8640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f5415f600 0x7f4f5415fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a8ba640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f540c5630 0x7f4f540c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 -- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f540c5630 msgr2=0x7f4f540c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f540c5630 0x7f4f540c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 -- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f5415f600 msgr2=0x7f4f5415fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f5415f600 0x7f4f5415fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 -- 172.21.15.53:0/1679975803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540c6960 con 0x7f4f400047c0 2023-10-21T16:20:07.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.260+0000 7f4f5a0b9640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000d7a0 tx=0x7f4f5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:07.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.261+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f500074e0 con 0x7f4f400047c0 2023-10-21T16:20:07.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.261+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f50007640 con 0x7f4f400047c0 2023-10-21T16:20:07.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.261+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f500049e0 con 0x7f4f400047c0 2023-10-21T16:20:07.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.261+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540c6c40 con 0x7f4f400047c0 2023-10-21T16:20:07.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.261+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f540c5ff0 con 0x7f4f400047c0 2023-10-21T16:20:07.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.262+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f54074150 con 0x7f4f400047c0 2023-10-21T16:20:07.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.263+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 51) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4f50004b40 con 0x7f4f400047c0 2023-10-21T16:20:07.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.264+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f4f50088fc0 con 0x7f4f400047c0 2023-10-21T16:20:07.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.268+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f50053cf0 con 0x7f4f400047c0 2023-10-21T16:20:07.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.451+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f540b4470 con 0x7f4f400047c0 2023-10-21T16:20:07.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.452+0000 7f4f4affd640 1 -- 172.21.15.53:0/1679975803 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f500190a0 con 0x7f4f400047c0 2023-10-21T16:20:07.454 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:20:07.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 msgr2=0x7f4f5415a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000d7a0 tx=0x7f4f5000dc60 comp rx=0 tx=0).stop 2023-10-21T16:20:07.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 shutdown_connections 2023-10-21T16:20:07.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f540c5630 0x7f4f540c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f5415f600 0x7f4f5415fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 --2- 172.21.15.53:0/1679975803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f400047c0 0x7f4f5415a3e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 >> 172.21.15.53:0/1679975803 conn(0x7f4f5406c050 msgr2=0x7f4f540caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:07.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 shutdown_connections 2023-10-21T16:20:07.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.454+0000 7f4f5b0bb640 1 -- 172.21.15.53:0/1679975803 wait complete. 2023-10-21T16:20:07.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:20:07.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.748+0000 7f026dbd2640 1 Processor -- start 2023-10-21T16:20:07.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.748+0000 7f026dbd2640 1 -- start start 2023-10-21T16:20:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f026dbd2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f02680b0680 0x7f026815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f026dbd2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680cab60 con 0x7f02680ca680 2023-10-21T16:20:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f026dbd2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680cb960 con 0x7f026815a820 2023-10-21T16:20:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f026dbd2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680cbdf0 con 0x7f02680b0680 2023-10-21T16:20:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f0267fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f02680b0680 0x7f026815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.750+0000 7f026d3d1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f02680ca680 0x7f026815ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53602/0 (socket says 172.21.15.53:53602) 2023-10-21T16:20:07.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f026d3d1640 1 -- 172.21.15.53:0/2215933516 learned_addr learned my addr 172.21.15.53:0/2215933516 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:07.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f02677fe640 1 -- 172.21.15.53:0/2215933516 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270992078 0 0) 0x7f02680cab60 con 0x7f02680ca680 2023-10-21T16:20:07.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f02677fe640 1 -- 172.21.15.53:0/2215933516 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0258003520 con 0x7f02680ca680 2023-10-21T16:20:07.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f0267fff640 1 -- 172.21.15.53:0/2215933516 >> v1:172.21.15.124:6789/0 conn(0x7f026815a820 legacy=0x7f02680aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f0267fff640 1 -- 172.21.15.53:0/2215933516 >> v1:172.21.15.53:6789/0 conn(0x7f02680ca680 legacy=0x7f026815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.751+0000 7f0267fff640 1 -- 172.21.15.53:0/2215933516 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680cb030 con 0x7f02680b0680 2023-10-21T16:20:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.752+0000 7f0267fff640 1 --2- 172.21.15.53:0/2215933516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680b0680 0x7f026815e540 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a6f0e70d8832b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.753+0000 7f02677fe640 1 -- 172.21.15.53:0/2215933516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254031830 con 0x7f02680b0680 2023-10-21T16:20:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.753+0000 7f02677fe640 1 -- 172.21.15.53:0/2215933516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0254031990 con 0x7f02680b0680 2023-10-21T16:20:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.753+0000 7f02677fe640 1 -- 172.21.15.53:0/2215933516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254031d20 con 0x7f02680b0680 2023-10-21T16:20:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.753+0000 7f026dbd2640 1 -- 172.21.15.53:0/2215933516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680b0680 msgr2=0x7f026815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.753+0000 7f026dbd2640 1 --2- 172.21.15.53:0/2215933516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680b0680 0x7f026815e540 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f02540097a0 tx=0x7f025402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.754+0000 7f026dbd2640 1 -- 172.21.15.53:0/2215933516 shutdown_connections 2023-10-21T16:20:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.754+0000 7f026dbd2640 1 --2- 172.21.15.53:0/2215933516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680b0680 0x7f026815e540 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.754+0000 7f026dbd2640 1 -- 172.21.15.53:0/2215933516 >> 172.21.15.53:0/2215933516 conn(0x7f026806c050 msgr2=0x7f026806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:07.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.754+0000 7f026dbd2640 1 -- 172.21.15.53:0/2215933516 shutdown_connections 2023-10-21T16:20:07.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.754+0000 7f026dbd2640 1 -- 172.21.15.53:0/2215933516 wait complete. 2023-10-21T16:20:07.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.755+0000 7f026dbd2640 1 Processor -- start 2023-10-21T16:20:07.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 -- start start 2023-10-21T16:20:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02680b0680 0x7f02680c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f026815a820 0x7f0268151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026815b650 con 0x7f02680ca680 2023-10-21T16:20:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026815b930 con 0x7f02680b0680 2023-10-21T16:20:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.756+0000 7f026dbd2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026815bc10 con 0x7f026815a820 2023-10-21T16:20:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47326/0 (socket says 172.21.15.53:47326) 2023-10-21T16:20:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 -- 172.21.15.53:0/3739796190 learned_addr learned my addr 172.21.15.53:0/3739796190 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f026cbd0640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02680b0680 0x7f02680c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 -- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f026815a820 msgr2=0x7f0268151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f026815a820 0x7f0268151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 -- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02680b0680 msgr2=0x7f02680c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02680b0680 0x7f02680c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.757+0000 7f0267fff640 1 -- 172.21.15.53:0/3739796190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254009400 con 0x7f02680ca680 2023-10-21T16:20:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f0267fff640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f02540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025403a6f0 con 0x7f02680ca680 2023-10-21T16:20:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025403a870 con 0x7f02680ca680 2023-10-21T16:20:07.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025403ac20 con 0x7f02680ca680 2023-10-21T16:20:07.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026815bef0 con 0x7f02680ca680 2023-10-21T16:20:07.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.758+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268158c90 con 0x7f02680ca680 2023-10-21T16:20:07.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.759+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02680c2300 con 0x7f02680ca680 2023-10-21T16:20:07.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.760+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 51) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0254030380 con 0x7f02680ca680 2023-10-21T16:20:07.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.761+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f02540ab060 con 0x7f02680ca680 2023-10-21T16:20:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.765+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02540ddca0 con 0x7f02680ca680 2023-10-21T16:20:07.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.795+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 52) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0254075a10 con 0x7f02680ca680 2023-10-21T16:20:07.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.796+0000 7f02657fa640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 0x7f024c073270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:07.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.799+0000 7f026cbd0640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 0x7f024c073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:07.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.811+0000 7f026cbd0640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 0x7f024c073270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f025c000f60 tx=0x7f025c009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:07.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.958+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02680c2300 con 0x7f02680ca680 2023-10-21T16:20:07.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.959+0000 7f02657fa640 1 -- 172.21.15.53:0/3739796190 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02680c2300 con 0x7f02680ca680 2023-10-21T16:20:07.960 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:20:07.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 msgr2=0x7f024c073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 0x7f024c073270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f025c000f60 tx=0x7f025c009040 comp rx=0 tx=0).stop 2023-10-21T16:20:07.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 msgr2=0x7f026815ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f02540098c0 tx=0x7f02540046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 shutdown_connections 2023-10-21T16:20:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f024c070e80 0x7f024c073270 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f026815a820 0x7f0268151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02680ca680 0x7f026815ac00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 --2- 172.21.15.53:0/3739796190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02680b0680 0x7f02680c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 >> 172.21.15.53:0/3739796190 conn(0x7f026806c050 msgr2=0x7f02680cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 shutdown_connections 2023-10-21T16:20:07.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:07.961+0000 7f026dbd2640 1 -- 172.21.15.53:0/3739796190 wait complete. 2023-10-21T16:20:07.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:08.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.203+0000 7f8a9b3ff640 1 Processor -- start 2023-10-21T16:20:08.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.204+0000 7f8a9b3ff640 1 -- start start 2023-10-21T16:20:08.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.205+0000 7f8a9b3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a9415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.205+0000 7f8a9b3ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a940adc70 0x7f8a940b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.205+0000 7f8a9b3ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9415f7d0 con 0x7f8a9415af10 2023-10-21T16:20:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.205+0000 7f8a9b3ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9415f210 con 0x7f8a940adc70 2023-10-21T16:20:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.205+0000 7f8a9b3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415f4f0 con 0x7f8a9415ab30 2023-10-21T16:20:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a940adc70 0x7f8a940b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8a940adc70 0x7f8a940b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47342/0 (socket says 172.21.15.53:47342) 2023-10-21T16:20:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 -- 172.21.15.53:0/1873817208 learned_addr learned my addr 172.21.15.53:0/1873817208 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a9a3fd640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a9415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 -- 172.21.15.53:0/1873817208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 msgr2=0x7f8a9415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a9415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 -- 172.21.15.53:0/1873817208 >> v1:172.21.15.124:6789/0 conn(0x7f8a9415af10 legacy=0x7f8a940701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.206+0000 7f8a99bfc640 1 -- 172.21.15.53:0/1873817208 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9406c550 con 0x7f8a940adc70 2023-10-21T16:20:08.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.207+0000 7f8a99bfc640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a940adc70 0x7f8a940b00c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009870 tx=0x7f8a8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615572365c2651b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a993fb640 1 -- 172.21.15.53:0/1873817208 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84030840 con 0x7f8a940adc70 2023-10-21T16:20:08.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a993fb640 1 -- 172.21.15.53:0/1873817208 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840309a0 con 0x7f8a940adc70 2023-10-21T16:20:08.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a993fb640 1 -- 172.21.15.53:0/1873817208 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84030d30 con 0x7f8a940adc70 2023-10-21T16:20:08.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/1873817208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a940adc70 msgr2=0x7f8a940b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a940adc70 0x7f8a940b00c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009870 tx=0x7f8a8402b980 comp rx=0 tx=0).stop 2023-10-21T16:20:08.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/1873817208 shutdown_connections 2023-10-21T16:20:08.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a940adc70 0x7f8a940b00c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/1873817208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a9415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.208+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/1873817208 >> 172.21.15.53:0/1873817208 conn(0x7f8a940cb0c0 msgr2=0x7f8a940cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:08.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.209+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/1873817208 shutdown_connections 2023-10-21T16:20:08.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.209+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/1873817208 wait complete. 2023-10-21T16:20:08.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.210+0000 7f8a9b3ff640 1 Processor -- start 2023-10-21T16:20:08.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.210+0000 7f8a9b3ff640 1 -- start start 2023-10-21T16:20:08.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 0x7f8a94155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a940cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940c59d0 con 0x7f8a9415af10 2023-10-21T16:20:08.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940c8530 con 0x7f8a940adc70 2023-10-21T16:20:08.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.211+0000 7f8a9b3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940cccd0 con 0x7f8a9415ab30 2023-10-21T16:20:08.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47350/0 (socket says 172.21.15.53:47350) 2023-10-21T16:20:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 -- 172.21.15.53:0/4023909877 learned_addr learned my addr 172.21.15.53:0/4023909877 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9a3fd640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 0x7f8a94155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a99bfc640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a940cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 -- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 msgr2=0x7f8a940cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a940cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 -- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 msgr2=0x7f8a94155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 0x7f8a94155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 -- 172.21.15.53:0/4023909877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840094d0 con 0x7f8a9415af10 2023-10-21T16:20:08.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9a3fd640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 0x7f8a94155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:08.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.212+0000 7f8a9abfe640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c730 tx=0x7f8a8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.213+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c003b90 con 0x7f8a9415af10 2023-10-21T16:20:08.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.213+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c003cf0 con 0x7f8a9415af10 2023-10-21T16:20:08.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.213+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a940ccfb0 con 0x7f8a9415af10 2023-10-21T16:20:08.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.213+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c003ea0 con 0x7f8a9415af10 2023-10-21T16:20:08.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.213+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a940cd570 con 0x7f8a9415af10 2023-10-21T16:20:08.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.214+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a94066700 con 0x7f8a9415af10 2023-10-21T16:20:08.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.215+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 52) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8a8c005520 con 0x7f8a9415af10 2023-10-21T16:20:08.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.216+0000 7f8a8affd640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 0x7f8a7806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.216+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f8a8c0897b0 con 0x7f8a9415af10 2023-10-21T16:20:08.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.217+0000 7f8a9a3fd640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 0x7f8a7806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.218+0000 7f8a9a3fd640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 0x7f8a7806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006fd0 tx=0x7f8a90008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.220+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a8c0545c0 con 0x7f8a9415af10 2023-10-21T16:20:08.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.461+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a9404e160 con 0x7f8a9415af10 2023-10-21T16:20:08.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.462+0000 7f8a8affd640 1 -- 172.21.15.53:0/4023909877 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f8a8c057be0 con 0x7f8a9415af10 2023-10-21T16:20:08.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:08.464 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:20:08.464 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:20:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.464+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 msgr2=0x7f8a7806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.464+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 0x7f8a7806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006fd0 tx=0x7f8a90008040 comp rx=0 tx=0).stop 2023-10-21T16:20:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.464+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 msgr2=0x7f8a940c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.464+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c730 tx=0x7f8a8c00cbf0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 shutdown_connections 2023-10-21T16:20:08.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8a7806acc0 0x7f8a7806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a9415af10 0x7f8a940c7ff0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a9415ab30 0x7f8a940cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 --2- 172.21.15.53:0/4023909877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a940adc70 0x7f8a94155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 >> 172.21.15.53:0/4023909877 conn(0x7f8a940cb0c0 msgr2=0x7f8a941597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:08.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 shutdown_connections 2023-10-21T16:20:08.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.465+0000 7f8a9b3ff640 1 -- 172.21.15.53:0/4023909877 wait complete. 2023-10-21T16:20:08.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:08.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.709+0000 7fc73dc40640 1 Processor -- start 2023-10-21T16:20:08.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.709+0000 7fc73dc40640 1 -- start start 2023-10-21T16:20:08.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc73dc40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cd8a0 0x7fc73806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc73dc40640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc7380cf720 0x7fc73806b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc73dc40640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380cb590 con 0x7fc73806bd90 2023-10-21T16:20:08.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc73dc40640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73806c220 con 0x7fc7380cf720 2023-10-21T16:20:08.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc73dc40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380cfc90 con 0x7fc7380cd8a0 2023-10-21T16:20:08.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc737fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc7380cf720 0x7fc73806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc737fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc7380cf720 0x7fc73806b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47364/0 (socket says 172.21.15.53:47364) 2023-10-21T16:20:08.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.711+0000 7fc737fff640 1 -- 172.21.15.53:0/3063091146 learned_addr learned my addr 172.21.15.53:0/3063091146 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:08.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.712+0000 7fc73cc3e640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cd8a0 0x7fc73806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.712+0000 7fc737fff640 1 -- 172.21.15.53:0/3063091146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cd8a0 msgr2=0x7fc73806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:08.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.712+0000 7fc737fff640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cd8a0 0x7fc73806b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.712+0000 7fc737fff640 1 -- 172.21.15.53:0/3063091146 >> v1:172.21.15.124:6789/0 conn(0x7fc73806bd90 legacy=0x7fc7381608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.712+0000 7fc737fff640 1 -- 172.21.15.53:0/3063091146 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380cff70 con 0x7fc7380cf720 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.713+0000 7fc737fff640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cf720 0x7fc73806b850 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744236157cd26236 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.713+0000 7fc7377fe640 1 -- 172.21.15.53:0/3063091146 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724031640 con 0x7fc7380cf720 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc7377fe640 1 -- 172.21.15.53:0/3063091146 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7240317a0 con 0x7fc7380cf720 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc7377fe640 1 -- 172.21.15.53:0/3063091146 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724031b30 con 0x7fc7380cf720 2023-10-21T16:20:08.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 -- 172.21.15.53:0/3063091146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cf720 msgr2=0x7fc73806b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cf720 0x7fc73806b850 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc72402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 -- 172.21.15.53:0/3063091146 shutdown_connections 2023-10-21T16:20:08.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cf720 0x7fc73806b850 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3063091146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cd8a0 0x7fc73806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 -- 172.21.15.53:0/3063091146 >> 172.21.15.53:0/3063091146 conn(0x7fc738099b60 msgr2=0x7fc7380d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:08.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 -- 172.21.15.53:0/3063091146 shutdown_connections 2023-10-21T16:20:08.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.714+0000 7fc73dc40640 1 -- 172.21.15.53:0/3063091146 wait complete. 2023-10-21T16:20:08.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.716+0000 7fc73dc40640 1 Processor -- start 2023-10-21T16:20:08.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.716+0000 7fc73dc40640 1 -- start start 2023-10-21T16:20:08.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc73806bd90 0x7fc73809b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 0x7fc73809bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809a4d0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809a6e0 con 0x7fc73806bd90 2023-10-21T16:20:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc73dc40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73809a9c0 con 0x7fc7380cf720 2023-10-21T16:20:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc737fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc737fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47376/0 (socket says 172.21.15.53:47376) 2023-10-21T16:20:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.717+0000 7fc737fff640 1 -- 172.21.15.53:0/3667062524 learned_addr learned my addr 172.21.15.53:0/3667062524 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc73d43f640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 0x7fc73809bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc73cc3e640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc73806bd90 0x7fc73809b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 -- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 msgr2=0x7fc73809bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 0x7fc73809bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 -- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc73806bd90 msgr2=0x7fc73809b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc73806bd90 0x7fc73809b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 -- 172.21.15.53:0/3667062524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240094d0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc73d43f640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 0x7fc73809bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.718+0000 7fc737fff640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc7240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.719+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72403ac60 con 0x7fc7380cd8a0 2023-10-21T16:20:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.719+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72403ade0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.719+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724044630 con 0x7fc7380cd8a0 2023-10-21T16:20:08.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.719+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73809aca0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.719+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73809b9e0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.720+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73809b1c0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.721+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 52) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc72402f050 con 0x7fc7380cd8a0 2023-10-21T16:20:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.722+0000 7fc7357fa640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 0x7fc71c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.722+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7fc7240ab990 con 0x7fc7380cd8a0 2023-10-21T16:20:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.722+0000 7fc73cc3e640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 0x7fc71c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.723+0000 7fc73cc3e640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 0x7fc71c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00a970 tx=0x7fc72c008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.726+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7240767a0 con 0x7fc7380cd8a0 2023-10-21T16:20:08.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.810+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 53) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc724076b70 con 0x7fc7380cd8a0 2023-10-21T16:20:08.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.933+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc73804e160 con 0x7fc7380cd8a0 2023-10-21T16:20:08.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.935+0000 7fc7357fa640 1 -- 172.21.15.53:0/3667062524 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18184 (secure 0 0 0) 0x7fc724090880 con 0x7fc7380cd8a0 2023-10-21T16:20:08.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:08.937 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":122,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:06.775839+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3147619579":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6833/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3354756863":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/2281736141":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1223499974":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3104959929":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/216550556":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/4133714254":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6832/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1757063465":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3937751236":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2028143540":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/130280090":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/1496413032":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6833/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6832/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/3651970368":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/28232742":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2272255357":"2023-10-21T17:18:51.635575+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:08.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 msgr2=0x7fc71c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 0x7fc71c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00a970 tx=0x7fc72c008040 comp rx=0 tx=0).stop 2023-10-21T16:20:08.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 msgr2=0x7fc73809dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:08.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc724009870 tx=0x7fc7240046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 shutdown_connections 2023-10-21T16:20:08.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fc71c06acc0 0x7fc71c06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7380cf720 0x7fc73809bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7380cd8a0 0x7fc73809dd90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 --2- 172.21.15.53:0/3667062524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc73806bd90 0x7fc73809b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:08.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 >> 172.21.15.53:0/3667062524 conn(0x7fc738099b60 msgr2=0x7fc7380ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:08.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 shutdown_connections 2023-10-21T16:20:08.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:08.938+0000 7fc73dc40640 1 -- 172.21.15.53:0/3667062524 wait complete. 2023-10-21T16:20:08.959 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:20:08.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.186+0000 7f10c9821640 1 Processor -- start 2023-10-21T16:20:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.186+0000 7f10c9821640 1 -- start start 2023-10-21T16:20:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c9821640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f10c40b1840 0x7f10c40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c9821640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f10c40b1d90 0x7f10c40d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c9821640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c40b8bf0 con 0x7f10c40cd8d0 2023-10-21T16:20:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c9821640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c40b8630 con 0x7f10c40b1d90 2023-10-21T16:20:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c9821640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c40b8910 con 0x7f10c40b1840 2023-10-21T16:20:09.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c2ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10c40cd8d0 0x7f10c40cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53604/0 (socket says 172.21.15.53:53604) 2023-10-21T16:20:09.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.188+0000 7f10c2ffd640 1 -- 172.21.15.53:0/3457337778 learned_addr learned my addr 172.21.15.53:0/3457337778 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:09.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c37fe640 1 --2- 172.21.15.53:0/3457337778 >> v2:172.21.15.53:3300/0 conn(0x7f10c40b1d90 0x7f10c40d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c27fc640 1 --2- 172.21.15.53:0/3457337778 >> v2:172.21.15.124:3300/0 conn(0x7f10c40b1840 0x7f10c40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c37fe640 1 -- 172.21.15.53:0/3457337778 >> v2:172.21.15.124:3300/0 conn(0x7f10c40b1840 msgr2=0x7f10c40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:09.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c37fe640 1 --2- 172.21.15.53:0/3457337778 >> v2:172.21.15.124:3300/0 conn(0x7f10c40b1840 0x7f10c40d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c37fe640 1 -- 172.21.15.53:0/3457337778 >> v1:172.21.15.53:6789/0 conn(0x7f10c40cd8d0 legacy=0x7f10c40cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.189+0000 7f10c37fe640 1 -- 172.21.15.53:0/3457337778 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40d1100 con 0x7f10c40b1d90 2023-10-21T16:20:09.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.190+0000 7f10c37fe640 1 --2- 172.21.15.53:0/3457337778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40d1e40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f10c40b8330 tx=0x7f10bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9a92bad1c57248 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.191+0000 7f10c1ffb640 1 -- 172.21.15.53:0/3457337778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc031830 con 0x7f10c40b1d90 2023-10-21T16:20:09.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.191+0000 7f10c1ffb640 1 -- 172.21.15.53:0/3457337778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc031990 con 0x7f10c40b1d90 2023-10-21T16:20:09.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.191+0000 7f10c1ffb640 1 -- 172.21.15.53:0/3457337778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc031d20 con 0x7f10c40b1d90 2023-10-21T16:20:09.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.191+0000 7f10c9821640 1 -- 172.21.15.53:0/3457337778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 msgr2=0x7f10c40d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.191+0000 7f10c9821640 1 --2- 172.21.15.53:0/3457337778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40d1e40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f10c40b8330 tx=0x7f10bc02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 -- 172.21.15.53:0/3457337778 shutdown_connections 2023-10-21T16:20:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 --2- 172.21.15.53:0/3457337778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40d1e40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 --2- 172.21.15.53:0/3457337778 >> v2:172.21.15.124:3300/0 conn(0x7f10c40b1840 0x7f10c40d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 -- 172.21.15.53:0/3457337778 >> 172.21.15.53:0/3457337778 conn(0x7f10c406b6a0 msgr2=0x7f10c406bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:09.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 -- 172.21.15.53:0/3457337778 shutdown_connections 2023-10-21T16:20:09.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.192+0000 7f10c9821640 1 -- 172.21.15.53:0/3457337778 wait complete. 2023-10-21T16:20:09.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.193+0000 7f10c9821640 1 Processor -- start 2023-10-21T16:20:09.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.194+0000 7f10c9821640 1 -- start start 2023-10-21T16:20:09.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c40b1840 0x7f10c41534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c40cd8d0 0x7f10c4159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4153a00 con 0x7f10c40b1d90 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40cbe20 con 0x7f10c40b1840 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c9821640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40cc100 con 0x7f10c40cd8d0 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47398/0 (socket says 172.21.15.53:47398) 2023-10-21T16:20:09.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 -- 172.21.15.53:0/3040695755 learned_addr learned my addr 172.21.15.53:0/3040695755 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 -- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c40cd8d0 msgr2=0x7f10c4159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c40cd8d0 0x7f10c4159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 -- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c40b1840 msgr2=0x7f10c41534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c40b1840 0x7f10c41534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.195+0000 7f10c27fc640 1 -- 172.21.15.53:0/3040695755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc009400 con 0x7f10c40b1d90 2023-10-21T16:20:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10c27fc640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e870 tx=0x7f10ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0094f0 con 0x7f10c40b1d90 2023-10-21T16:20:09.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac00c940 con 0x7f10c40b1d90 2023-10-21T16:20:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00ccf0 con 0x7f10c40b1d90 2023-10-21T16:20:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c4158e40 con 0x7f10c40b1d90 2023-10-21T16:20:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.196+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c40cbb10 con 0x7f10c40b1d90 2023-10-21T16:20:09.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.197+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c4158ca0 con 0x7f10c40b1d90 2023-10-21T16:20:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.198+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 53) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f10ac013470 con 0x7f10c40b1d90 2023-10-21T16:20:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.199+0000 7f10bbfff640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 0x7f10a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.199+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f10ac089cd0 con 0x7f10c40b1d90 2023-10-21T16:20:09.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.200+0000 7f10c2ffd640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 0x7f10a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.201+0000 7f10c2ffd640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 0x7f10a406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10c40cb390 tx=0x7f10b4001660 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.204+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10c4158ca0 con 0x7f10c40b1d90 2023-10-21T16:20:09.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.434+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10c4158ca0 con 0x7f10c40b1d90 2023-10-21T16:20:09.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.435+0000 7f10bbfff640 1 -- 172.21.15.53:0/3040695755 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f10c4158ca0 con 0x7f10c40b1d90 2023-10-21T16:20:09.437 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:09.438 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:20:09.438 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:20:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.437+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 msgr2=0x7f10a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.437+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 0x7f10a406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10c40cb390 tx=0x7f10b4001660 comp rx=0 tx=0).stop 2023-10-21T16:20:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 msgr2=0x7f10c40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e870 tx=0x7f10ac00ed30 comp rx=0 tx=0).stop 2023-10-21T16:20:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 shutdown_connections 2023-10-21T16:20:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f10a406af00 0x7f10a406d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c40cd8d0 0x7f10c4159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c40b1d90 0x7f10c40cb5d0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 --2- 172.21.15.53:0/3040695755 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c40b1840 0x7f10c41534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 >> 172.21.15.53:0/3040695755 conn(0x7f10c406b6a0 msgr2=0x7f10c40cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 shutdown_connections 2023-10-21T16:20:09.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.438+0000 7f10c9821640 1 -- 172.21.15.53:0/3040695755 wait complete. 2023-10-21T16:20:09.460 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.683+0000 7f51bd305640 1 Processor -- start 2023-10-21T16:20:09.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.683+0000 7f51bd305640 1 -- start start 2023-10-21T16:20:09.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51bd305640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51b80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51bd305640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f51b80cb3b0 0x7f51b815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51bd305640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b815b2b0 con 0x7f51b815ab40 2023-10-21T16:20:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51bd305640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b815bf30 con 0x7f51b80cb3b0 2023-10-21T16:20:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51bd305640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80cb840 con 0x7f51b80c90b0 2023-10-21T16:20:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51b6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f51b815ab40 0x7f51b80ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53614/0 (socket says 172.21.15.53:53614) 2023-10-21T16:20:09.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.685+0000 7f51b6ffd640 1 -- 172.21.15.53:0/3049700047 learned_addr learned my addr 172.21.15.53:0/3049700047 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51affff640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51b80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b77fe640 1 --2- 172.21.15.53:0/3049700047 >> v2:172.21.15.124:3300/0 conn(0x7f51b80cb3b0 0x7f51b815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057527864 0 0) 0x7f51b815b2b0 con 0x7f51b815ab40 2023-10-21T16:20:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a4003520 con 0x7f51b815ab40 2023-10-21T16:20:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945892866 0 0) 0x7f51a4003520 con 0x7f51b815ab40 2023-10-21T16:20:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 msgr2=0x7f51b80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51b80cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 >> v2:172.21.15.124:3300/0 conn(0x7f51b80cb3b0 msgr2=0x7f51b815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 --2- 172.21.15.53:0/3049700047 >> v2:172.21.15.124:3300/0 conn(0x7f51b80cb3b0 0x7f51b815e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b815b590 con 0x7f51b815ab40 2023-10-21T16:20:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f51a8001670 con 0x7f51b815ab40 2023-10-21T16:20:09.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 >> v1:172.21.15.53:6789/0 conn(0x7f51b815ab40 legacy=0x7f51b80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51a40047c0 0x7f51a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815bf30 con 0x7f51a4003970 2023-10-21T16:20:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b80cb840 con 0x7f51a40047c0 2023-10-21T16:20:09.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.686+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400b980 con 0x7f51b80c90b0 2023-10-21T16:20:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51affff640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51a40047c0 0x7f51a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b77fe640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 -- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 msgr2=0x7f51a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:09.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b80c90b0 0x7f51a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 -- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51a40047c0 msgr2=0x7f51a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:09.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51a40047c0 0x7f51a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.687+0000 7f51b6ffd640 1 -- 172.21.15.53:0/3049700047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4003520 con 0x7f51a4003970 2023-10-21T16:20:09.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.688+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51a4003d90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c240 tx=0x7f51a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84e2d3567becfeb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a802cb00 con 0x7f51a4003970 2023-10-21T16:20:09.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a802cc60 con 0x7f51a4003970 2023-10-21T16:20:09.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51b67fc640 1 -- 172.21.15.53:0/3049700047 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8005af0 con 0x7f51a4003970 2023-10-21T16:20:09.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51bd305640 1 -- 172.21.15.53:0/3049700047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 msgr2=0x7f51a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51bd305640 1 --2- 172.21.15.53:0/3049700047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51a4003d90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c240 tx=0x7f51a40057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.689+0000 7f51b77fe640 1 -- 172.21.15.53:0/3049700047 reap_dead start 2023-10-21T16:20:09.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.690+0000 7f51bd305640 1 -- 172.21.15.53:0/3049700047 shutdown_connections 2023-10-21T16:20:09.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.690+0000 7f51bd305640 1 -- 172.21.15.53:0/3049700047 >> 172.21.15.53:0/3049700047 conn(0x7f51b806a380 msgr2=0x7f51b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:09.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.690+0000 7f51bd305640 1 -- 172.21.15.53:0/3049700047 shutdown_connections 2023-10-21T16:20:09.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.690+0000 7f51bd305640 1 -- 172.21.15.53:0/3049700047 wait complete. 2023-10-21T16:20:09.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.692+0000 7f51bd305640 1 Processor -- start 2023-10-21T16:20:09.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.692+0000 7f51bd305640 1 -- start start 2023-10-21T16:20:09.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 0x7f51b81f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 0x7f51b815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f4460 con 0x7f51a4003970 2023-10-21T16:20:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8159520 con 0x7f51b81f3ff0 2023-10-21T16:20:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51bd305640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81599b0 con 0x7f51b815d630 2023-10-21T16:20:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47430/0 (socket says 172.21.15.53:47430) 2023-10-21T16:20:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.694+0000 7f51b6ffd640 1 -- 172.21.15.53:0/2315011096 learned_addr learned my addr 172.21.15.53:0/2315011096 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51affff640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 0x7f51b81f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b77fe640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 0x7f51b815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 -- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 msgr2=0x7f51b81f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 0x7f51b81f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 -- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 msgr2=0x7f51b815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 0x7f51b815c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 -- 172.21.15.53:0/2315011096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b81f4910 con 0x7f51a4003970 2023-10-21T16:20:09.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51affff640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 0x7f51b81f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:09.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b77fe640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 0x7f51b815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:09.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.695+0000 7f51b6ffd640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c360 tx=0x7f51a8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.696+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8039070 con 0x7f51a4003970 2023-10-21T16:20:09.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.696+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8004bf0 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.696+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a803c040 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.696+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b80c8330 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.696+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b81591b0 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.697+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b8067490 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.698+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 53) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f51a8032070 con 0x7f51a4003970 2023-10-21T16:20:09.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.699+0000 7f51af7fe640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 0x7f519806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:09.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.699+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f51a80ab030 con 0x7f51a4003970 2023-10-21T16:20:09.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.700+0000 7f51affff640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 0x7f519806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:09.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.701+0000 7f51affff640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 0x7f519806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009a50 tx=0x7f51a0008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:09.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.705+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51a8075c60 con 0x7f51a4003970 2023-10-21T16:20:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.944+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51b815d410 con 0x7f51a4003970 2023-10-21T16:20:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.946+0000 7f51af7fe640 1 -- 172.21.15.53:0/2315011096 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f51a8079280 con 0x7f51a4003970 2023-10-21T16:20:09.948 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:09.949 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:18:43.938045+0000","modified":"2023-10-21T16:18:47.045682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35915},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35915":{"gid":35915,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/3805349006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3805349006},{"type":"v1","addr":"172.21.15.124:6837","nonce":3805349006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-21T16:20:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 53 2023-10-21T16:20:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 msgr2=0x7f519806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 0x7f519806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009a50 tx=0x7f51a0008040 comp rx=0 tx=0).stop 2023-10-21T16:20:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 msgr2=0x7f51b8154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f51a800c360 tx=0x7f51a8001a00 comp rx=0 tx=0).stop 2023-10-21T16:20:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 shutdown_connections 2023-10-21T16:20:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f519806af00 0x7f519806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51b81f3ff0 0x7f51b815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51b815d630 0x7f51b81f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 --2- 172.21.15.53:0/2315011096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51a4003970 0x7f51b8154490 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 >> 172.21.15.53:0/2315011096 conn(0x7f51b806a380 msgr2=0x7f51b80c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.948+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 shutdown_connections 2023-10-21T16:20:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:09.949+0000 7f51bd305640 1 -- 172.21.15.53:0/2315011096 wait complete. 2023-10-21T16:20:09.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.193+0000 7f1f17fff640 1 Processor -- start 2023-10-21T16:20:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.193+0000 7f1f17fff640 1 -- start start 2023-10-21T16:20:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.195+0000 7f1f17fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f1815cd00 0x7f1f180b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.195+0000 7f1f17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.195+0000 7f1f17fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f180b17e0 con 0x7f1f18072140 2023-10-21T16:20:10.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.195+0000 7f1f17fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f18071380 con 0x7f1f1815cd00 2023-10-21T16:20:10.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.195+0000 7f1f17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18071660 con 0x7f1f180ccdd0 2023-10-21T16:20:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f1815cd00 0x7f1f180b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f177fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f1815cd00 0x7f1f180b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59296/0 (socket says 172.21.15.53:59296) 2023-10-21T16:20:10.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 -- 172.21.15.53:0/1207168171 learned_addr learned my addr 172.21.15.53:0/1207168171 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 -- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 msgr2=0x7f1f180c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 --2- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 -- 172.21.15.53:0/1207168171 >> v1:172.21.15.124:6789/0 conn(0x7f1f18072140 legacy=0x7f1f180cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.196+0000 7f1f16ffd640 1 -- 172.21.15.53:0/1207168171 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18071940 con 0x7f1f1815cd00 2023-10-21T16:20:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.198+0000 7f1f16ffd640 1 --2- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f180b1c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f516e9ad93cdd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.198+0000 7f1f167fc640 1 -- 172.21.15.53:0/1207168171 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08030840 con 0x7f1f1815cd00 2023-10-21T16:20:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.198+0000 7f1f167fc640 1 -- 172.21.15.53:0/1207168171 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080309a0 con 0x7f1f1815cd00 2023-10-21T16:20:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.198+0000 7f1f167fc640 1 -- 172.21.15.53:0/1207168171 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08030d30 con 0x7f1f1815cd00 2023-10-21T16:20:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 -- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 msgr2=0x7f1f180b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 --2- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f180b1c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0802b980 comp rx=0 tx=0).stop 2023-10-21T16:20:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 -- 172.21.15.53:0/1207168171 shutdown_connections 2023-10-21T16:20:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 --2- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 --2- 172.21.15.53:0/1207168171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f180b1c20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 -- 172.21.15.53:0/1207168171 >> 172.21.15.53:0/1207168171 conn(0x7f1f1806b280 msgr2=0x7f1f1806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 -- 172.21.15.53:0/1207168171 shutdown_connections 2023-10-21T16:20:10.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.199+0000 7f1f17fff640 1 -- 172.21.15.53:0/1207168171 wait complete. 2023-10-21T16:20:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.201+0000 7f1f17fff640 1 Processor -- start 2023-10-21T16:20:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.201+0000 7f1f17fff640 1 -- start start 2023-10-21T16:20:10.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f181f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180c8c50 con 0x7f1f18072140 2023-10-21T16:20:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180c8360 con 0x7f1f1815cd00 2023-10-21T16:20:10.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.202+0000 7f1f17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18160160 con 0x7f1f180ccdd0 2023-10-21T16:20:10.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47438/0 (socket says 172.21.15.53:47438) 2023-10-21T16:20:10.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 -- 172.21.15.53:0/2081286996 learned_addr learned my addr 172.21.15.53:0/2081286996 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:10.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f177fe640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f181f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f0ffff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 -- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 msgr2=0x7f1f180ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 -- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 msgr2=0x7f1f181f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f181f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.203+0000 7f1f16ffd640 1 -- 172.21.15.53:0/2081286996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080094d0 con 0x7f1f18072140 2023-10-21T16:20:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f16ffd640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08039a90 con 0x7f1f18072140 2023-10-21T16:20:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08039bf0 con 0x7f1f18072140 2023-10-21T16:20:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08043410 con 0x7f1f18072140 2023-10-21T16:20:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f180c8600 con 0x7f1f18072140 2023-10-21T16:20:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.204+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1815d7d0 con 0x7f1f18072140 2023-10-21T16:20:10.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.205+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18073700 con 0x7f1f18072140 2023-10-21T16:20:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.207+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 53) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1f0802f390 con 0x7f1f18072140 2023-10-21T16:20:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.207+0000 7f1f14ff9640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 0x7f1ef806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.208+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f1f080abe60 con 0x7f1f18072140 2023-10-21T16:20:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.208+0000 7f1f0ffff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 0x7f1ef806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.209+0000 7f1f0ffff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 0x7f1ef806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f18146ef0 tx=0x7f1f00009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.212+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f08076920 con 0x7f1f18072140 2023-10-21T16:20:10.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.406+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f1804e160 con 0x7f1f18072140 2023-10-21T16:20:10.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.409+0000 7f1f14ff9640 1 -- 172.21.15.53:0/2081286996 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18184 (secure 0 0 0) 0x7f1f08048090 con 0x7f1f18072140 2023-10-21T16:20:10.412 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:10.413 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":122,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:06.775839+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:18:41.952733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-21T16:18:42.974741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.31","mappings":[{"from":1,"to":2}]},{"pgid":"14.38","mappings":[{"from":1,"to":5}]},{"pgid":"14.3a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3147619579":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6833/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3354756863":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/2281736141":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1223499974":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3104959929":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/216550556":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/4133714254":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6832/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1757063465":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3937751236":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2028143540":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/130280090":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/1496413032":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6833/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6832/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/3651970368":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/28232742":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2272255357":"2023-10-21T17:18:51.635575+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.411+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 msgr2=0x7f1ef806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.411+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 0x7f1ef806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f18146ef0 tx=0x7f1f00009040 comp rx=0 tx=0).stop 2023-10-21T16:20:10.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.411+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 msgr2=0x7f1f180ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.411+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f08004730 comp rx=0 tx=0).stop 2023-10-21T16:20:10.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 shutdown_connections 2023-10-21T16:20:10.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f1ef806af00 0x7f1ef806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f1815cd00 0x7f1f181f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f180ccdd0 0x7f1f180ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 --2- 172.21.15.53:0/2081286996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f18072140 0x7f1f180ade70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 >> 172.21.15.53:0/2081286996 conn(0x7f1f1806b280 msgr2=0x7f1f1815c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:10.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 shutdown_connections 2023-10-21T16:20:10.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.412+0000 7f1f17fff640 1 -- 172.21.15.53:0/2081286996 wait complete. 2023-10-21T16:20:10.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:20:10.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.663+0000 7fb33710a640 1 Processor -- start 2023-10-21T16:20:10.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.663+0000 7fb33710a640 1 -- start start 2023-10-21T16:20:10.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.664+0000 7fb33710a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3300cc9b0 0x7fb33006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.664+0000 7fb33710a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb33006d590 0x7fb33015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.664+0000 7fb33710a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33006b6f0 con 0x7fb3300ccd90 2023-10-21T16:20:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb33710a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33006d100 con 0x7fb3300cc9b0 2023-10-21T16:20:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb33710a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330159550 con 0x7fb33006d590 2023-10-21T16:20:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3300cc9b0 0x7fb33006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3300cc9b0 0x7fb33006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47452/0 (socket says 172.21.15.53:47452) 2023-10-21T16:20:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 -- 172.21.15.53:0/1854674143 learned_addr learned my addr 172.21.15.53:0/1854674143 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:10.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb335907640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb33006d590 0x7fb33015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 -- 172.21.15.53:0/1854674143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb33006d590 msgr2=0x7fb33015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:10.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb33006d590 0x7fb33015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 -- 172.21.15.53:0/1854674143 >> v1:172.21.15.53:6789/0 conn(0x7fb3300ccd90 legacy=0x7fb33015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.665+0000 7fb336108640 1 -- 172.21.15.53:0/1854674143 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33006e5e0 con 0x7fb3300cc9b0 2023-10-21T16:20:10.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.667+0000 7fb336108640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3300cc9b0 0x7fb33006dee0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009870 tx=0x7fb32c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34854dc3658d6dd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.667+0000 7fb335106640 1 -- 172.21.15.53:0/1854674143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c030840 con 0x7fb3300cc9b0 2023-10-21T16:20:10.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.667+0000 7fb335106640 1 -- 172.21.15.53:0/1854674143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0309a0 con 0x7fb3300cc9b0 2023-10-21T16:20:10.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.667+0000 7fb335106640 1 -- 172.21.15.53:0/1854674143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c030d30 con 0x7fb3300cc9b0 2023-10-21T16:20:10.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 -- 172.21.15.53:0/1854674143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3300cc9b0 msgr2=0x7fb33006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3300cc9b0 0x7fb33006dee0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009870 tx=0x7fb32c02b980 comp rx=0 tx=0).stop 2023-10-21T16:20:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 -- 172.21.15.53:0/1854674143 shutdown_connections 2023-10-21T16:20:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb33006d590 0x7fb33015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 --2- 172.21.15.53:0/1854674143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3300cc9b0 0x7fb33006dee0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 -- 172.21.15.53:0/1854674143 >> 172.21.15.53:0/1854674143 conn(0x7fb3300b08d0 msgr2=0x7fb3300b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:10.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 -- 172.21.15.53:0/1854674143 shutdown_connections 2023-10-21T16:20:10.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.668+0000 7fb33710a640 1 -- 172.21.15.53:0/1854674143 wait complete. 2023-10-21T16:20:10.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.670+0000 7fb33710a640 1 Processor -- start 2023-10-21T16:20:10.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.670+0000 7fb33710a640 1 -- start start 2023-10-21T16:20:10.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3300cc9b0 0x7fb330157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 0x7fb3300c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330152f60 con 0x7fb33006d590 2023-10-21T16:20:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330153ae0 con 0x7fb3300ccd90 2023-10-21T16:20:10.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb33710a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330155720 con 0x7fb3300cc9b0 2023-10-21T16:20:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb336108640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb336108640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47462/0 (socket says 172.21.15.53:47462) 2023-10-21T16:20:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.671+0000 7fb336108640 1 -- 172.21.15.53:0/3934746056 learned_addr learned my addr 172.21.15.53:0/3934746056 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 -- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3300cc9b0 msgr2=0x7fb330157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3300cc9b0 0x7fb330157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336909640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 0x7fb3300c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 -- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 msgr2=0x7fb3300c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 0x7fb3300c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 -- 172.21.15.53:0/3934746056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0094d0 con 0x7fb33006d590 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336909640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 0x7fb3300c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb336108640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb32c000c00 tx=0x7fb32c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.672+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c039c90 con 0x7fb33006d590 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.673+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c039df0 con 0x7fb33006d590 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.673+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c043640 con 0x7fb33006d590 2023-10-21T16:20:10.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.673+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3301531b0 con 0x7fb33006d590 2023-10-21T16:20:10.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.673+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330153770 con 0x7fb33006d590 2023-10-21T16:20:10.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.674+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330074150 con 0x7fb33006d590 2023-10-21T16:20:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.675+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 53) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fb32c0437a0 con 0x7fb33006d590 2023-10-21T16:20:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.676+0000 7fb322ffd640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 0x7fb31406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.676+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7006+0+0 (secure 0 0 0) 0x7fb32c03e070 con 0x7fb33006d590 2023-10-21T16:20:10.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.676+0000 7fb335907640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 0x7fb31406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.678+0000 7fb335907640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 0x7fb31406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:10.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.680+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb32c0777e0 con 0x7fb33006d590 2023-10-21T16:20:10.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.825+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb32c077bb0 con 0x7fb33006d590 2023-10-21T16:20:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.918+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb33015c220 con 0x7fb33006d590 2023-10-21T16:20:10.922 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:20:10.919+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:20:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.923+0000 7fb322ffd640 1 -- 172.21.15.53:0/3934746056 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb32c04fb30 con 0x7fb33006d590 2023-10-21T16:20:10.926 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:20:10.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 msgr2=0x7fb31406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 0x7fb31406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).stop 2023-10-21T16:20:10.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 msgr2=0x7fb330158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:10.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb32c000c00 tx=0x7fb32c004730 comp rx=0 tx=0).stop 2023-10-21T16:20:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 shutdown_connections 2023-10-21T16:20:10.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fb31406af00 0x7fb31406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3300ccd90 0x7fb3300c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3300cc9b0 0x7fb330157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 --2- 172.21.15.53:0/3934746056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb33006d590 0x7fb330158d80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:10.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.926+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 >> 172.21.15.53:0/3934746056 conn(0x7fb3300b08d0 msgr2=0x7fb3300b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.927+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 shutdown_connections 2023-10-21T16:20:10.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:10.927+0000 7fb33710a640 1 -- 172.21.15.53:0/3934746056 wait complete. 2023-10-21T16:20:10.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:20:10.968 INFO:tasks.ceph.mds.b.smithi124.stderr: -7700> 2023-10-21T16:18:50.430+0000 7f9ff0aaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:20:11.048 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:20:11.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.168+0000 7fd88b37c640 1 Processor -- start 2023-10-21T16:20:11.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.168+0000 7fd88b37c640 1 -- start start 2023-10-21T16:20:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd88b37c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88406dfc0 0x7fd884070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd88b37c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd884071070 0x7fd8840cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd88b37c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8840b1cc0 con 0x7fd8840afbe0 2023-10-21T16:20:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd88b37c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840b2620 con 0x7fd88406dfc0 2023-10-21T16:20:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd88b37c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840b2060 con 0x7fd884071070 2023-10-21T16:20:11.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd888917640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88406dfc0 0x7fd884070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd888917640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88406dfc0 0x7fd884070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47478/0 (socket says 172.21.15.53:47478) 2023-10-21T16:20:11.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.170+0000 7fd888917640 1 -- 172.21.15.53:0/3677994321 learned_addr learned my addr 172.21.15.53:0/3677994321 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:11.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.171+0000 7fd889919640 1 --2- 172.21.15.53:0/3677994321 >> v2:172.21.15.124:3300/0 conn(0x7fd884071070 0x7fd8840cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.171+0000 7fd888917640 1 -- 172.21.15.53:0/3677994321 >> v2:172.21.15.124:3300/0 conn(0x7fd884071070 msgr2=0x7fd8840cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.171+0000 7fd888917640 1 --2- 172.21.15.53:0/3677994321 >> v2:172.21.15.124:3300/0 conn(0x7fd884071070 0x7fd8840cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.171+0000 7fd888917640 1 -- 172.21.15.53:0/3677994321 >> v1:172.21.15.124:6789/0 conn(0x7fd8840afbe0 legacy=0x7fd8840b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.171+0000 7fd888917640 1 -- 172.21.15.53:0/3677994321 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840b2340 con 0x7fd88406dfc0 2023-10-21T16:20:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.172+0000 7fd888917640 1 --2- 172.21.15.53:0/3677994321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88406dfc0 0x7fd884070b30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd8840c8d10 tx=0x7fd87402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b564a096f13a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.172+0000 7fd87bfff640 1 -- 172.21.15.53:0/3677994321 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874031890 con 0x7fd88406dfc0 2023-10-21T16:20:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.172+0000 7fd87bfff640 1 -- 172.21.15.53:0/3677994321 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8740319f0 con 0x7fd88406dfc0 2023-10-21T16:20:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.172+0000 7fd87bfff640 1 -- 172.21.15.53:0/3677994321 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874031d80 con 0x7fd88406dfc0 2023-10-21T16:20:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 -- 172.21.15.53:0/3677994321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88406dfc0 msgr2=0x7fd884070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 --2- 172.21.15.53:0/3677994321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88406dfc0 0x7fd884070b30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd8840c8d10 tx=0x7fd87402b930 comp rx=0 tx=0).stop 2023-10-21T16:20:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 -- 172.21.15.53:0/3677994321 shutdown_connections 2023-10-21T16:20:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 --2- 172.21.15.53:0/3677994321 >> v2:172.21.15.124:3300/0 conn(0x7fd884071070 0x7fd8840cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 --2- 172.21.15.53:0/3677994321 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88406dfc0 0x7fd884070b30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 -- 172.21.15.53:0/3677994321 >> 172.21.15.53:0/3677994321 conn(0x7fd88406a5d0 msgr2=0x7fd8840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:11.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 -- 172.21.15.53:0/3677994321 shutdown_connections 2023-10-21T16:20:11.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.173+0000 7fd88b37c640 1 -- 172.21.15.53:0/3677994321 wait complete. 2023-10-21T16:20:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.175+0000 7fd88b37c640 1 Processor -- start 2023-10-21T16:20:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.175+0000 7fd88b37c640 1 -- start start 2023-10-21T16:20:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88406dfc0 0x7fd8840c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8840afbe0 0x7fd88415c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c2250 con 0x7fd884071070 2023-10-21T16:20:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841f1320 con 0x7fd88406dfc0 2023-10-21T16:20:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.176+0000 7fd88b37c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8841f1600 con 0x7fd8840afbe0 2023-10-21T16:20:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47484/0 (socket says 172.21.15.53:47484) 2023-10-21T16:20:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 -- 172.21.15.53:0/1849031339 learned_addr learned my addr 172.21.15.53:0/1849031339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd889118640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88406dfc0 0x7fd8840c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd889919640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8840afbe0 0x7fd88415c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 -- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8840afbe0 msgr2=0x7fd88415c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8840afbe0 0x7fd88415c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 -- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88406dfc0 msgr2=0x7fd8840c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88406dfc0 0x7fd8840c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 -- 172.21.15.53:0/1849031339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740094d0 con 0x7fd884071070 2023-10-21T16:20:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.177+0000 7fd888917640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.178+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87403acb0 con 0x7fd884071070 2023-10-21T16:20:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.178+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87403ae10 con 0x7fd884071070 2023-10-21T16:20:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.178+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874044630 con 0x7fd884071070 2023-10-21T16:20:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.178+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8840c24a0 con 0x7fd884071070 2023-10-21T16:20:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.178+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8840cf0e0 con 0x7fd884071070 2023-10-21T16:20:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.179+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8841f17c0 con 0x7fd884071070 2023-10-21T16:20:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.180+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd87404d450 con 0x7fd884071070 2023-10-21T16:20:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.181+0000 7fd879ffb640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 0x7fd86806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.181+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fd8740ac5b0 con 0x7fd884071070 2023-10-21T16:20:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.182+0000 7fd889118640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 0x7fd86806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.183+0000 7fd889118640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 0x7fd86806d480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:11.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.185+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd874076f50 con 0x7fd884071070 2023-10-21T16:20:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.426+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd88404e160 con 0x7fd884071070 2023-10-21T16:20:11.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.927+0000 7fd879ffb640 1 -- 172.21.15.53:0/1849031339 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd87407a570 con 0x7fd884071070 2023-10-21T16:20:11.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.930+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 msgr2=0x7fd86806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:11.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.930+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 0x7fd86806d480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).stop 2023-10-21T16:20:11.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.930+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 msgr2=0x7fd8841f08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:11.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.930+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd8740046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 shutdown_connections 2023-10-21T16:20:11.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd86806afc0 0x7fd86806d480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8840afbe0 0x7fd88415c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd884071070 0x7fd8841f08a0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 --2- 172.21.15.53:0/1849031339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88406dfc0 0x7fd8840c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:11.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 >> 172.21.15.53:0/1849031339 conn(0x7fd88406a5d0 msgr2=0x7fd8841597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:11.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 shutdown_connections 2023-10-21T16:20:11.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:11.931+0000 7fd88b37c640 1 -- 172.21.15.53:0/1849031339 wait complete. 2023-10-21T16:20:11.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:20:12.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.180+0000 7f75037e9640 1 Processor -- start 2023-10-21T16:20:12.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.181+0000 7f75037e9640 1 -- start start 2023-10-21T16:20:12.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f75037e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 0x7f74fc164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f75037e9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc06dc10 con 0x7f74fc159a20 2023-10-21T16:20:12.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f75037e9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74fc06cb50 con 0x7f74fc06c6c0 2023-10-21T16:20:12.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f75037e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0cd6b0 con 0x7f74fc1628b0 2023-10-21T16:20:12.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f7501585640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f74fc159a20 0x7f74fc06cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53620/0 (socket says 172.21.15.53:53620) 2023-10-21T16:20:12.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.182+0000 7f7501585640 1 -- 172.21.15.53:0/4053836753 learned_addr learned my addr 172.21.15.53:0/4053836753 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:12.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f7501d86640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 0x7f74fc164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598774870 0 0) 0x7f74fc06dc10 con 0x7f74fc159a20 2023-10-21T16:20:12.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e8005730 con 0x7f74fc159a20 2023-10-21T16:20:12.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613800082 0 0) 0x7f74e8005730 con 0x7f74fc159a20 2023-10-21T16:20:12.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 msgr2=0x7f74fc164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:12.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 0x7f74fc164ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.183+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 >> v1:172.21.15.124:6789/0 conn(0x7f74fc06c6c0 legacy=0x7f74fc06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0cd990 con 0x7f74fc159a20 2023-10-21T16:20:12.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f74f80019f0 con 0x7f74fc159a20 2023-10-21T16:20:12.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 >> v1:172.21.15.53:6789/0 conn(0x7f74fc159a20 legacy=0x7f74fc06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e8005bb0 0x7f74e8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e80069d0 0x7f74e8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 0x7f74e80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0cd6b0 con 0x7f74e80069d0 2023-10-21T16:20:12.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e800dcc0 con 0x7f74e8005bb0 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e800de80 con 0x7f74fc1628b0 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e80069d0 0x7f74e8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 -- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 msgr2=0x7f74e80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc1628b0 0x7f74e80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 -- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e8005bb0 msgr2=0x7f74e8005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:12.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e8005bb0 0x7f74e8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.184+0000 7f7500d84640 1 -- 172.21.15.53:0/4053836753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8005730 con 0x7f74e80069d0 2023-10-21T16:20:12.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.186+0000 7f7500d84640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e80069d0 0x7f74e8006e50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f74ec001750 tx=0x7f74e8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69a0154b735fc21 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:12.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.186+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0052d0 con 0x7f74e80069d0 2023-10-21T16:20:12.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.186+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ec02e050 con 0x7f74e80069d0 2023-10-21T16:20:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.186+0000 7f74f3fff640 1 -- 172.21.15.53:0/4053836753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ec0035f0 con 0x7f74e80069d0 2023-10-21T16:20:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 -- 172.21.15.53:0/4053836753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e80069d0 msgr2=0x7f74e8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 --2- 172.21.15.53:0/4053836753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74e80069d0 0x7f74e8006e50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f74ec001750 tx=0x7f74e8007a00 comp rx=0 tx=0).stop 2023-10-21T16:20:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f7501d86640 1 -- 172.21.15.53:0/4053836753 reap_dead start 2023-10-21T16:20:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 -- 172.21.15.53:0/4053836753 shutdown_connections 2023-10-21T16:20:12.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 -- 172.21.15.53:0/4053836753 >> 172.21.15.53:0/4053836753 conn(0x7f74fc0cba20 msgr2=0x7f74fc0cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:12.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 -- 172.21.15.53:0/4053836753 shutdown_connections 2023-10-21T16:20:12.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.187+0000 7f75037e9640 1 -- 172.21.15.53:0/4053836753 wait complete. 2023-10-21T16:20:12.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.189+0000 7f75037e9640 1 Processor -- start 2023-10-21T16:20:12.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.190+0000 7f75037e9640 1 -- start start 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e80069d0 0x7f74fc0c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 0x7f74fc0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0c8c70 con 0x7f74fc0c87f0 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0b29a0 con 0x7f74e80069d0 2023-10-21T16:20:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f75037e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0c79e0 con 0x7f74fc15a910 2023-10-21T16:20:12.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f7501d86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.191+0000 7f7501d86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47498/0 (socket says 172.21.15.53:47498) 2023-10-21T16:20:12.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 -- 172.21.15.53:0/3917481838 learned_addr learned my addr 172.21.15.53:0/3917481838 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:12.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501585640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e80069d0 0x7f74fc0c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7500d84640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 0x7f74fc0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 -- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 msgr2=0x7f74fc0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 0x7f74fc0c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 -- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e80069d0 msgr2=0x7f74fc0c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e80069d0 0x7f74fc0c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 -- 172.21.15.53:0/3917481838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc0b2070 con 0x7f74fc0c87f0 2023-10-21T16:20:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7500d84640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 0x7f74fc0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.192+0000 7f7501d86640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f74fc070900 tx=0x7f74f400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.193+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f400c680 con 0x7f74fc0c87f0 2023-10-21T16:20:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.193+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f400c7e0 con 0x7f74fc0c87f0 2023-10-21T16:20:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.193+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f400cb90 con 0x7f74fc0c87f0 2023-10-21T16:20:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.193+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc0b2350 con 0x7f74fc0c87f0 2023-10-21T16:20:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.193+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc0c7160 con 0x7f74fc0c87f0 2023-10-21T16:20:12.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.194+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc0ce0d0 con 0x7f74fc0c87f0 2023-10-21T16:20:12.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.195+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f74f400ccf0 con 0x7f74fc0c87f0 2023-10-21T16:20:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.196+0000 7f74f1ffb640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 0x7f74e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.197+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f74f4089ae0 con 0x7f74fc0c87f0 2023-10-21T16:20:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.197+0000 7f7501585640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 0x7f74e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:12.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.198+0000 7f7501585640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 0x7f74e006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005fd0 tx=0x7f74f8001950 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:12.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.201+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74f4054500 con 0x7f74fc0c87f0 2023-10-21T16:20:12.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.395+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f74fc04e160 con 0x7f74fc0c87f0 2023-10-21T16:20:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.936+0000 7f74f1ffb640 1 -- 172.21.15.53:0/3917481838 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) v1 ==== 181+0+0 (secure 0 0 0) 0x7f74f4057b20 con 0x7f74fc0c87f0 2023-10-21T16:20:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:20:12.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 msgr2=0x7f74e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 0x7f74e006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005fd0 tx=0x7f74f8001950 comp rx=0 tx=0).stop 2023-10-21T16:20:12.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 msgr2=0x7f74fc0c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f74fc070900 tx=0x7f74f400ec00 comp rx=0 tx=0).stop 2023-10-21T16:20:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 shutdown_connections 2023-10-21T16:20:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f74e006af90 0x7f74e006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74fc0c87f0 0x7f74fc0c6c20 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74fc15a910 0x7f74fc0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 --2- 172.21.15.53:0/3917481838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74e80069d0 0x7f74fc0c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.941+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 >> 172.21.15.53:0/3917481838 conn(0x7f74fc0cba20 msgr2=0x7f74fc0ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.943+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 shutdown_connections 2023-10-21T16:20:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:12.943+0000 7f75037e9640 1 -- 172.21.15.53:0/3917481838 wait complete. 2023-10-21T16:20:12.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:20:13.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.176+0000 7f8af2f61640 1 Processor -- start 2023-10-21T16:20:13.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.176+0000 7f8af2f61640 1 -- start start 2023-10-21T16:20:13.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8aec0afbe0 0x7f8aec0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 0x7f8aec070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 0x7f8aec0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0b2620 con 0x7f8aec0afbe0 2023-10-21T16:20:13.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0b2060 con 0x7f8aec06dfc0 2023-10-21T16:20:13.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.178+0000 7f8af2f61640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0b2340 con 0x7f8aec071070 2023-10-21T16:20:13.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8aec0afbe0 0x7f8aec0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8aec0afbe0 0x7f8aec0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47500/0 (socket says 172.21.15.53:47500) 2023-10-21T16:20:13.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 -- 172.21.15.53:0/639837093 learned_addr learned my addr 172.21.15.53:0/639837093 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:13.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af14fe640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 0x7f8aec0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8aebfff640 1 --2- 172.21.15.53:0/639837093 >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 0x7f8aec070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 -- 172.21.15.53:0/639837093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 msgr2=0x7f8aec0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:13.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 0x7f8aec0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 -- 172.21.15.53:0/639837093 >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 msgr2=0x7f8aec070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/639837093 >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 0x7f8aec070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af0cfd640 1 -- 172.21.15.53:0/639837093 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec070350 con 0x7f8aec0afbe0 2023-10-21T16:20:13.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8af14fe640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 0x7f8aec0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:13.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.179+0000 7f8aebfff640 1 --2- 172.21.15.53:0/639837093 >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 0x7f8aec070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:13.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.180+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec0b15b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0b3980 tx=0x7f8ae002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0ddaf4130d7b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:13.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.181+0000 7f8aeb7fe640 1 -- 172.21.15.53:0/639837093 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0031880 con 0x7f8aec0afbe0 2023-10-21T16:20:13.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.181+0000 7f8aeb7fe640 1 -- 172.21.15.53:0/639837093 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae00319e0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.181+0000 7f8aeb7fe640 1 -- 172.21.15.53:0/639837093 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0031d70 con 0x7f8aec0afbe0 2023-10-21T16:20:13.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.181+0000 7f8af2f61640 1 -- 172.21.15.53:0/639837093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 msgr2=0x7f8aec0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:13.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.181+0000 7f8af2f61640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec0b15b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0b3980 tx=0x7f8ae002b920 comp rx=0 tx=0).stop 2023-10-21T16:20:13.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 -- 172.21.15.53:0/639837093 shutdown_connections 2023-10-21T16:20:13.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec071070 0x7f8aec0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 --2- 172.21.15.53:0/639837093 >> v2:172.21.15.124:3300/0 conn(0x7f8aec06dfc0 0x7f8aec070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 --2- 172.21.15.53:0/639837093 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec0b15b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 -- 172.21.15.53:0/639837093 >> 172.21.15.53:0/639837093 conn(0x7f8aec06a5d0 msgr2=0x7f8aec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:13.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 -- 172.21.15.53:0/639837093 shutdown_connections 2023-10-21T16:20:13.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.182+0000 7f8af2f61640 1 -- 172.21.15.53:0/639837093 wait complete. 2023-10-21T16:20:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.184+0000 7f8af2f61640 1 Processor -- start 2023-10-21T16:20:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.184+0000 7f8af2f61640 1 -- start start 2023-10-21T16:20:13.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec06dfc0 0x7f8aec1493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 0x7f8aec150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0c39c0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0c4500 con 0x7f8aec071070 2023-10-21T16:20:13.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af2f61640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec149930 con 0x7f8aec06dfc0 2023-10-21T16:20:13.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af14fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af14fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47516/0 (socket says 172.21.15.53:47516) 2023-10-21T16:20:13.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af14fe640 1 -- 172.21.15.53:0/222846147 learned_addr learned my addr 172.21.15.53:0/222846147 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:13.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.185+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec06dfc0 0x7f8aec1493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8aebfff640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 0x7f8aec150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 -- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec06dfc0 msgr2=0x7f8aec1493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:13.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec06dfc0 0x7f8aec1493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 -- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 msgr2=0x7f8aec150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:13.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 0x7f8aec150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 -- 172.21.15.53:0/222846147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00094d0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8aebfff640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 0x7f8aec150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:13.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af14fe640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400ceb0 tx=0x7f8ae400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:13.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400d840 con 0x7f8aec0afbe0 2023-10-21T16:20:13.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae400d9a0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec0c3bd0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400dd50 con 0x7f8aec0afbe0 2023-10-21T16:20:13.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.186+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec0c4190 con 0x7f8aec0afbe0 2023-10-21T16:20:13.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.188+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec0c4690 con 0x7f8aec0afbe0 2023-10-21T16:20:13.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.189+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8ae401e3b0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.190+0000 7f8ae97fa640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 0x7f8ad006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:13.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.190+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f8ae408a5e0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.190+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 0x7f8ad006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:13.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.192+0000 7f8af0cfd640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 0x7f8ad006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0036000 tx=0x7f8ae003c040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:13.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.194+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae40551d0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.394+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8aec04e160 con 0x7f8aec0afbe0 2023-10-21T16:20:13.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.940+0000 7f8ae97fa640 1 -- 172.21.15.53:0/222846147 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ae40587f0 con 0x7f8aec0afbe0 2023-10-21T16:20:13.943 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:20:13.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.946+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 msgr2=0x7f8ad006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:13.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.946+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 0x7f8ad006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0036000 tx=0x7f8ae003c040 comp rx=0 tx=0).stop 2023-10-21T16:20:13.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.946+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 msgr2=0x7f8aec1506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:13.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.946+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400ceb0 tx=0x7f8ae400a710 comp rx=0 tx=0).stop 2023-10-21T16:20:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 shutdown_connections 2023-10-21T16:20:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8ad006af90 0x7f8ad006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8aec0afbe0 0x7f8aec1506d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aec071070 0x7f8aec150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 --2- 172.21.15.53:0/222846147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aec06dfc0 0x7f8aec1493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:13.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.947+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 >> 172.21.15.53:0/222846147 conn(0x7f8aec06a5d0 msgr2=0x7f8aec1546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.950+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 shutdown_connections 2023-10-21T16:20:13.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:13.950+0000 7f8af2f61640 1 -- 172.21.15.53:0/222846147 wait complete. 2023-10-21T16:20:13.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:20:14.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.178+0000 7f3311b81640 1 Processor -- start 2023-10-21T16:20:14.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.178+0000 7f3311b81640 1 -- start start 2023-10-21T16:20:14.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311b81640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 0x7f330c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311b81640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f330c0ca680 0x7f330c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311b81640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c0cab60 con 0x7f330c15a820 2023-10-21T16:20:14.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311b81640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c0cb960 con 0x7f330c0ca680 2023-10-21T16:20:14.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311b81640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c0cbdf0 con 0x7f330c0b0680 2023-10-21T16:20:14.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3310b7f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f330c15a820 0x7f330c0aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39224/0 (socket says 172.21.15.53:39224) 2023-10-21T16:20:14.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3310b7f640 1 -- 172.21.15.53:0/3060136841 learned_addr learned my addr 172.21.15.53:0/3060136841 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:14.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311380640 1 --2- 172.21.15.53:0/3060136841 >> v2:172.21.15.53:3300/0 conn(0x7f330c0ca680 0x7f330c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f330bfff640 1 --2- 172.21.15.53:0/3060136841 >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 0x7f330c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311380640 1 -- 172.21.15.53:0/3060136841 >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 msgr2=0x7f330c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:14.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311380640 1 --2- 172.21.15.53:0/3060136841 >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 0x7f330c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311380640 1 -- 172.21.15.53:0/3060136841 >> v1:172.21.15.53:6789/0 conn(0x7f330c15a820 legacy=0x7f330c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f3311380640 1 -- 172.21.15.53:0/3060136841 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0cb030 con 0x7f330c0ca680 2023-10-21T16:20:14.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.180+0000 7f330bfff640 1 --2- 172.21.15.53:0/3060136841 >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 0x7f330c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:14.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.181+0000 7f3311380640 1 --2- 172.21.15.53:0/3060136841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0ca680 0x7f330c15ea80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f330c0718e0 tx=0x7f330402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757bb5ff43e61b1c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.181+0000 7f330b7fe640 1 -- 172.21.15.53:0/3060136841 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330402cbe0 con 0x7f330c0ca680 2023-10-21T16:20:14.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.181+0000 7f330b7fe640 1 -- 172.21.15.53:0/3060136841 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330402cd40 con 0x7f330c0ca680 2023-10-21T16:20:14.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.181+0000 7f330b7fe640 1 -- 172.21.15.53:0/3060136841 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304034660 con 0x7f330c0ca680 2023-10-21T16:20:14.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 -- 172.21.15.53:0/3060136841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0ca680 msgr2=0x7f330c15ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 --2- 172.21.15.53:0/3060136841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0ca680 0x7f330c15ea80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f330c0718e0 tx=0x7f330402be30 comp rx=0 tx=0).stop 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 -- 172.21.15.53:0/3060136841 shutdown_connections 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 --2- 172.21.15.53:0/3060136841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0ca680 0x7f330c15ea80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 --2- 172.21.15.53:0/3060136841 >> v2:172.21.15.124:3300/0 conn(0x7f330c0b0680 0x7f330c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 -- 172.21.15.53:0/3060136841 >> 172.21.15.53:0/3060136841 conn(0x7f330c06c050 msgr2=0x7f330c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:14.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 -- 172.21.15.53:0/3060136841 shutdown_connections 2023-10-21T16:20:14.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.182+0000 7f3311b81640 1 -- 172.21.15.53:0/3060136841 wait complete. 2023-10-21T16:20:14.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.184+0000 7f3311b81640 1 Processor -- start 2023-10-21T16:20:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.184+0000 7f3311b81640 1 -- start start 2023-10-21T16:20:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f330c0ca680 0x7f330c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330c15a820 0x7f330c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c7f90 con 0x7f330c0b0680 2023-10-21T16:20:14.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c14fff0 con 0x7f330c15a820 2023-10-21T16:20:14.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3311b81640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c69b0 con 0x7f330c0ca680 2023-10-21T16:20:14.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3310b7f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.185+0000 7f3310b7f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46022/0 (socket says 172.21.15.53:46022) 2023-10-21T16:20:14.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 -- 172.21.15.53:0/520209016 learned_addr learned my addr 172.21.15.53:0/520209016 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:14.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 -- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f330c0ca680 msgr2=0x7f330c15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f330c0ca680 0x7f330c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 -- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330c15a820 msgr2=0x7f330c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330c15a820 0x7f330c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 -- 172.21.15.53:0/520209016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304009a70 con 0x7f330c0b0680 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f3310b7f640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f330000c480 tx=0x7f330000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300005ce0 con 0x7f330c0b0680 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3300005e40 con 0x7f330c0b0680 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.186+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000072f0 con 0x7f330c0b0680 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.187+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c0c6c90 con 0x7f330c0b0680 2023-10-21T16:20:14.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.187+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c0c7250 con 0x7f330c0b0680 2023-10-21T16:20:14.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.189+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c0c1e90 con 0x7f330c0b0680 2023-10-21T16:20:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.189+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3300007510 con 0x7f330c0b0680 2023-10-21T16:20:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.190+0000 7f33097fa640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 0x7f32f006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.190+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f3300088d00 con 0x7f330c0b0680 2023-10-21T16:20:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.191+0000 7f330bfff640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 0x7f32f006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.192+0000 7f330bfff640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 0x7f32f006d4a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32f8006e80 tx=0x7f32f8008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.194+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f330008c030 con 0x7f330c0b0680 2023-10-21T16:20:14.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.388+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f330c0b1320 con 0x7f330c0b0680 2023-10-21T16:20:14.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.396+0000 7f33097fa640 1 -- 172.21.15.53:0/520209016 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f330c0b1320 con 0x7f330c0b0680 2023-10-21T16:20:14.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 msgr2=0x7f32f006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 0x7f32f006d4a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f32f8006e80 tx=0x7f32f8008040 comp rx=0 tx=0).stop 2023-10-21T16:20:14.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 msgr2=0x7f330c0c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f330000c480 tx=0x7f330000c940 comp rx=0 tx=0).stop 2023-10-21T16:20:14.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 shutdown_connections 2023-10-21T16:20:14.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f32f006afe0 0x7f32f006d4a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330c15a820 0x7f330c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f330c0ca680 0x7f330c15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 --2- 172.21.15.53:0/520209016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f330c0b0680 0x7f330c0c7a50 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 >> 172.21.15.53:0/520209016 conn(0x7f330c06c050 msgr2=0x7f330c0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:14.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 shutdown_connections 2023-10-21T16:20:14.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.398+0000 7f3311b81640 1 -- 172.21.15.53:0/520209016 wait complete. 2023-10-21T16:20:14.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:20:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.637+0000 7f033210a640 1 Processor -- start 2023-10-21T16:20:14.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.637+0000 7f033210a640 1 -- start start 2023-10-21T16:20:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f033210a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f032c15aab0 0x7f032c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f033210a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f032c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f033210a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032c15ae90 con 0x7f032c0cb350 2023-10-21T16:20:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f033210a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032c0cbdb0 con 0x7f032c15aab0 2023-10-21T16:20:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f033210a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cb7f0 con 0x7f032c0caef0 2023-10-21T16:20:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0331909640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f032c0cb350 0x7f032c0c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39236/0 (socket says 172.21.15.53:39236) 2023-10-21T16:20:14.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0331909640 1 -- 172.21.15.53:0/4169157850 learned_addr learned my addr 172.21.15.53:0/4169157850 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:14.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0331108640 1 --2- 172.21.15.53:0/4169157850 >> v2:172.21.15.124:3300/0 conn(0x7f032c15aab0 0x7f032c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0330907640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f032c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526209861 0 0) 0x7f0324001490 con 0x7f032c0cb350 2023-10-21T16:20:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0318003520 con 0x7f032c0cb350 2023-10-21T16:20:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2179154187 0 0) 0x7f0318003520 con 0x7f032c0cb350 2023-10-21T16:20:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 msgr2=0x7f032c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f032c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 >> v2:172.21.15.124:3300/0 conn(0x7f032c15aab0 msgr2=0x7f032c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 --2- 172.21.15.53:0/4169157850 >> v2:172.21.15.124:3300/0 conn(0x7f032c15aab0 0x7f032c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0cbad0 con 0x7f032c0cb350 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0324001b80 con 0x7f032c0cb350 2023-10-21T16:20:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 >> v1:172.21.15.53:6789/0 conn(0x7f032c0cb350 legacy=0x7f032c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0318003970 0x7f0318003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.639+0000 7f0323fff640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f03180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0323fff640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318004d00 0x7f0318005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cbdb0 con 0x7f0318003970 2023-10-21T16:20:14.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cb7f0 con 0x7f0318004d00 2023-10-21T16:20:14.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031800b9b0 con 0x7f032c0caef0 2023-10-21T16:20:14.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0318003970 0x7f0318003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331909640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318004d00 0x7f0318005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0330907640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f03180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 -- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 msgr2=0x7f03180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:14.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c0caef0 0x7f03180047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 -- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318004d00 msgr2=0x7f0318005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:14.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318004d00 0x7f0318005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 -- 172.21.15.53:0/4169157850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318003520 con 0x7f0318003970 2023-10-21T16:20:14.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.641+0000 7f0331108640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0318003970 0x7f0318003d90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ac0 tx=0x7f03180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6a2459bd4c0192 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032802c710 con 0x7f0318003970 2023-10-21T16:20:14.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032802c870 con 0x7f0318003970 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f0323fff640 1 -- 172.21.15.53:0/4169157850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032802cc00 con 0x7f0318003970 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f033210a640 1 -- 172.21.15.53:0/4169157850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0318003970 msgr2=0x7f0318003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f033210a640 1 --2- 172.21.15.53:0/4169157850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0318003970 0x7f0318003d90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ac0 tx=0x7f03180057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f0331909640 1 -- 172.21.15.53:0/4169157850 reap_dead start 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f033210a640 1 -- 172.21.15.53:0/4169157850 shutdown_connections 2023-10-21T16:20:14.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f033210a640 1 -- 172.21.15.53:0/4169157850 >> 172.21.15.53:0/4169157850 conn(0x7f032c06c010 msgr2=0x7f032c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:14.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.642+0000 7f033210a640 1 -- 172.21.15.53:0/4169157850 shutdown_connections 2023-10-21T16:20:14.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.643+0000 7f033210a640 1 -- 172.21.15.53:0/4169157850 wait complete. 2023-10-21T16:20:14.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 Processor -- start 2023-10-21T16:20:14.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 -- start start 2023-10-21T16:20:14.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318003970 0x7f032c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c1f3b50 0x7f032c0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c1f3fd0 con 0x7f032c0c9580 2023-10-21T16:20:14.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0c9d00 con 0x7f0318003970 2023-10-21T16:20:14.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.645+0000 7f033210a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0c9fe0 con 0x7f032c1f3b50 2023-10-21T16:20:14.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46032/0 (socket says 172.21.15.53:46032) 2023-10-21T16:20:14.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 -- 172.21.15.53:0/2618949656 learned_addr learned my addr 172.21.15.53:0/2618949656 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:14.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331108640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318003970 0x7f032c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 -- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c1f3b50 msgr2=0x7f032c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c1f3b50 0x7f032c0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 -- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318003970 msgr2=0x7f032c1f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318003970 0x7f032c1f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 -- 172.21.15.53:0/2618949656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0ca2c0 con 0x7f032c0c9580 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.646+0000 7f0331909640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f032c0cbfa0 tx=0x7f032400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.647+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324007ae0 con 0x7f032c0c9580 2023-10-21T16:20:14.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.647+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032400ee20 con 0x7f032c0c9580 2023-10-21T16:20:14.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.647+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324013680 con 0x7f032c0c9580 2023-10-21T16:20:14.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.647+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c0ca5a0 con 0x7f032c0c9580 2023-10-21T16:20:14.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.647+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c0c9a00 con 0x7f032c0c9580 2023-10-21T16:20:14.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.648+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032c074150 con 0x7f032c0c9580 2023-10-21T16:20:14.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.649+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03240137e0 con 0x7f032c0c9580 2023-10-21T16:20:14.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.650+0000 7f0321ffb640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 0x7f031006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:14.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.651+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f0324089cc0 con 0x7f032c0c9580 2023-10-21T16:20:14.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.651+0000 7f0331108640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 0x7f031006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:14.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.652+0000 7f0331108640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 0x7f031006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0328036000 tx=0x7f0328030000 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:14.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.655+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0324090080 con 0x7f032c0c9580 2023-10-21T16:20:14.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.848+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f032c15ae90 con 0x7f032c0c9580 2023-10-21T16:20:14.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.853+0000 7f0321ffb640 1 -- 172.21.15.53:0/2618949656 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7f032c15ae90 con 0x7f032c0c9580 2023-10-21T16:20:14.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 msgr2=0x7f031006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 0x7f031006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0328036000 tx=0x7f0328030000 comp rx=0 tx=0).stop 2023-10-21T16:20:14.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 msgr2=0x7f032c0b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:14.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f032c0cbfa0 tx=0x7f032400e530 comp rx=0 tx=0).stop 2023-10-21T16:20:14.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 shutdown_connections 2023-10-21T16:20:14.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f031006af90 0x7f031006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f032c0c9580 0x7f032c0b0310 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f032c1f3b50 0x7f032c0c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 --2- 172.21.15.53:0/2618949656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0318003970 0x7f032c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:14.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 >> 172.21.15.53:0/2618949656 conn(0x7f032c06c010 msgr2=0x7f032c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:14.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 shutdown_connections 2023-10-21T16:20:14.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:14.856+0000 7f033210a640 1 -- 172.21.15.53:0/2618949656 wait complete. 2023-10-21T16:20:14.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:20:15.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.105+0000 7fd41cf63640 1 Processor -- start 2023-10-21T16:20:15.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.105+0000 7fd41cf63640 1 -- start start 2023-10-21T16:20:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.106+0000 7fd41cf63640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd41815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.106+0000 7fd41cf63640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4180ca680 0x7fd41815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.106+0000 7fd41cf63640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180cab60 con 0x7fd41815a820 2023-10-21T16:20:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd41cf63640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180cb960 con 0x7fd4180ca680 2023-10-21T16:20:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd41cf63640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180cbdf0 con 0x7fd4180b0680 2023-10-21T16:20:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd4177fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd41815a820 0x7fd4180aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39250/0 (socket says 172.21.15.53:39250) 2023-10-21T16:20:15.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd4177fe640 1 -- 172.21.15.53:0/961318597 learned_addr learned my addr 172.21.15.53:0/961318597 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd417fff640 1 --2- 172.21.15.53:0/961318597 >> v2:172.21.15.124:3300/0 conn(0x7fd4180ca680 0x7fd41815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.107+0000 7fd416ffd640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd41815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238383921 0 0) 0x7fd4180cab60 con 0x7fd41815a820 2023-10-21T16:20:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd404003520 con 0x7fd41815a820 2023-10-21T16:20:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd417fff640 1 -- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 msgr2=0x7fd41815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd417fff640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd41815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd417fff640 1 -- 172.21.15.53:0/961318597 >> v1:172.21.15.53:6789/0 conn(0x7fd41815a820 legacy=0x7fd4180aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840390136 0 0) 0x7fd404003520 con 0x7fd41815a820 2023-10-21T16:20:15.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.108+0000 7fd417fff640 1 -- 172.21.15.53:0/961318597 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180cb030 con 0x7fd4180ca680 2023-10-21T16:20:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.110+0000 7fd417fff640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd41815ea80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4100097a0 tx=0x7fd41002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=536317dbcfdbd5ba server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.110+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080019f0 con 0x7fd4180ca680 2023-10-21T16:20:15.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.110+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41002e050 con 0x7fd4180ca680 2023-10-21T16:20:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.110+0000 7fd4167fc640 1 -- 172.21.15.53:0/961318597 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410030730 con 0x7fd4180ca680 2023-10-21T16:20:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 -- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 msgr2=0x7fd41815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd41815ea80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4100097a0 tx=0x7fd41002b710 comp rx=0 tx=0).stop 2023-10-21T16:20:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 -- 172.21.15.53:0/961318597 shutdown_connections 2023-10-21T16:20:15.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd41815ea80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 --2- 172.21.15.53:0/961318597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd41815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 -- 172.21.15.53:0/961318597 >> 172.21.15.53:0/961318597 conn(0x7fd41806c050 msgr2=0x7fd41806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 -- 172.21.15.53:0/961318597 shutdown_connections 2023-10-21T16:20:15.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.111+0000 7fd41cf63640 1 -- 172.21.15.53:0/961318597 wait complete. 2023-10-21T16:20:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.113+0000 7fd41cf63640 1 Processor -- start 2023-10-21T16:20:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.113+0000 7fd41cf63640 1 -- start start 2023-10-21T16:20:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd4180cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd4180b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180c2ce0 con 0x7fd41815a820 2023-10-21T16:20:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180cc160 con 0x7fd4180ca680 2023-10-21T16:20:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd41cf63640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180cc400 con 0x7fd4180b0680 2023-10-21T16:20:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.114+0000 7fd417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46046/0 (socket says 172.21.15.53:46046) 2023-10-21T16:20:15.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 -- 172.21.15.53:0/1422083177 learned_addr learned my addr 172.21.15.53:0/1422083177 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:15.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd416ffd640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd4180b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd4177fe640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd4180cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 -- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 msgr2=0x7fd4180cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd4180cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 -- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 msgr2=0x7fd4180b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd4180b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 -- 172.21.15.53:0/1422083177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410009400 con 0x7fd41815a820 2023-10-21T16:20:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.115+0000 7fd417fff640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4100098c0 tx=0x7fd4100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.116+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41003c820 con 0x7fd41815a820 2023-10-21T16:20:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.116+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41003c980 con 0x7fd41815a820 2023-10-21T16:20:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.116+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41003cd10 con 0x7fd41815a820 2023-10-21T16:20:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.116+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4180cc6e0 con 0x7fd41815a820 2023-10-21T16:20:15.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.116+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180ccca0 con 0x7fd41815a820 2023-10-21T16:20:15.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.117+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4180cce90 con 0x7fd41815a820 2023-10-21T16:20:15.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.118+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd410039070 con 0x7fd41815a820 2023-10-21T16:20:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.119+0000 7fd40ffff640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 0x7fd3f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.119+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7fd4100ab430 con 0x7fd41815a820 2023-10-21T16:20:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.120+0000 7fd4177fe640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 0x7fd3f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.121+0000 7fd4177fe640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 0x7fd3f806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd408001560 tx=0x7fd408009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.124+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd410076120 con 0x7fd41815a820 2023-10-21T16:20:15.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.318+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd418074150 con 0x7fd41815a820 2023-10-21T16:20:15.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.326+0000 7fd40ffff640 1 -- 172.21.15.53:0/1422083177 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd410079740 con 0x7fd41815a820 2023-10-21T16:20:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 msgr2=0x7fd3f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 0x7fd3f806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd408001560 tx=0x7fd408009040 comp rx=0 tx=0).stop 2023-10-21T16:20:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 msgr2=0x7fd4180b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4100098c0 tx=0x7fd4100046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 shutdown_connections 2023-10-21T16:20:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd3f806af90 0x7fd3f806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd41815a820 0x7fd4180b2b60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4180ca680 0x7fd4180b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 --2- 172.21.15.53:0/1422083177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4180b0680 0x7fd4180cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 >> 172.21.15.53:0/1422083177 conn(0x7fd41806c050 msgr2=0x7fd418070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 shutdown_connections 2023-10-21T16:20:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.329+0000 7fd41cf63640 1 -- 172.21.15.53:0/1422083177 wait complete. 2023-10-21T16:20:15.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-21T16:20:15.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.565+0000 7f55ccd41640 1 Processor -- start 2023-10-21T16:20:15.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.565+0000 7f55ccd41640 1 -- start start 2023-10-21T16:20:15.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55ccd41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55c80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55ccd41640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55c80cb580 0x7f55c80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55ccd41640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c815b2b0 con 0x7f55c80c90b0 2023-10-21T16:20:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55ccd41640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c815bf30 con 0x7f55c80cb580 2023-10-21T16:20:15.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55ccd41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c815f7a0 con 0x7f55c815ab40 2023-10-21T16:20:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55c5d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f55c80c90b0 0x7f55c80cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39262/0 (socket says 172.21.15.53:39262) 2023-10-21T16:20:15.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.566+0000 7f55c5d9b640 1 -- 172.21.15.53:0/1989779048 learned_addr learned my addr 172.21.15.53:0/1989779048 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:15.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c659c640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55c80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c6d9d640 1 --2- 172.21.15.53:0/1989779048 >> v2:172.21.15.124:3300/0 conn(0x7f55c80cb580 0x7f55c80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043086879 0 0) 0x7f55c815b2b0 con 0x7f55c80c90b0 2023-10-21T16:20:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b4003520 con 0x7f55c80c90b0 2023-10-21T16:20:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630205670 0 0) 0x7f55b4003520 con 0x7f55c80c90b0 2023-10-21T16:20:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 msgr2=0x7f55c80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:15.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55c80ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 >> v2:172.21.15.124:3300/0 conn(0x7f55c80cb580 msgr2=0x7f55c80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 --2- 172.21.15.53:0/1989779048 >> v2:172.21.15.124:3300/0 conn(0x7f55c80cb580 0x7f55c80adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c815b590 con 0x7f55c80c90b0 2023-10-21T16:20:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.567+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f55b0001670 con 0x7f55c80c90b0 2023-10-21T16:20:15.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 >> v1:172.21.15.53:6789/0 conn(0x7f55c80c90b0 legacy=0x7f55c80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55b4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b4003ef0 0x7f55b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b4004d00 0x7f55b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c815bf30 con 0x7f55b4003ef0 2023-10-21T16:20:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c815f7a0 con 0x7f55b4004d00 2023-10-21T16:20:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b400b980 con 0x7f55c815ab40 2023-10-21T16:20:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b4003ef0 0x7f55b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c659c640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55b4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c6d9d640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b4004d00 0x7f55b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 -- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 msgr2=0x7f55b4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55c815ab40 0x7f55b4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 -- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b4004d00 msgr2=0x7f55b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55b4004d00 0x7f55b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.568+0000 7f55c5d9b640 1 -- 172.21.15.53:0/1989779048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4003520 con 0x7f55b4003ef0 2023-10-21T16:20:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.570+0000 7f55c5d9b640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b4003ef0 0x7f55b40047c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c290 tx=0x7f55b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd164aff90fec69 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.570+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0005ad0 con 0x7f55b4003ef0 2023-10-21T16:20:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.570+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0005c30 con 0x7f55b4003ef0 2023-10-21T16:20:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.570+0000 7f55c559a640 1 -- 172.21.15.53:0/1989779048 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b002c8c0 con 0x7f55b4003ef0 2023-10-21T16:20:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 -- 172.21.15.53:0/1989779048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b4003ef0 msgr2=0x7f55b40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1989779048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55b4003ef0 0x7f55b40047c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c290 tx=0x7f55b40057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55c6d9d640 1 -- 172.21.15.53:0/1989779048 reap_dead start 2023-10-21T16:20:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 -- 172.21.15.53:0/1989779048 shutdown_connections 2023-10-21T16:20:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 -- 172.21.15.53:0/1989779048 >> 172.21.15.53:0/1989779048 conn(0x7f55c806a380 msgr2=0x7f55c80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 -- 172.21.15.53:0/1989779048 shutdown_connections 2023-10-21T16:20:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.571+0000 7f55ccd41640 1 -- 172.21.15.53:0/1989779048 wait complete. 2023-10-21T16:20:15.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.573+0000 7f55ccd41640 1 Processor -- start 2023-10-21T16:20:15.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.573+0000 7f55ccd41640 1 -- start start 2023-10-21T16:20:15.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b4003ef0 0x7f55c81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55c814e5a0 0x7f55c80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80c6210 con 0x7f55c80c5850 2023-10-21T16:20:15.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c814ea20 con 0x7f55c814e5a0 2023-10-21T16:20:15.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55ccd41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c80c7530 con 0x7f55b4003ef0 2023-10-21T16:20:15.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55c6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55c6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46064/0 (socket says 172.21.15.53:46064) 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.574+0000 7f55c6d9d640 1 -- 172.21.15.53:0/1576694079 learned_addr learned my addr 172.21.15.53:0/1576694079 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c659c640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b4003ef0 0x7f55c81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c5d9b640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55c814e5a0 0x7f55c80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 -- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b4003ef0 msgr2=0x7f55c81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b4003ef0 0x7f55c81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 -- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55c814e5a0 msgr2=0x7f55c80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55c814e5a0 0x7f55c80c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 -- 172.21.15.53:0/1576694079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80c7810 con 0x7f55c80c5850 2023-10-21T16:20:15.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55c6d9d640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f55c80c8d80 tx=0x7f55c0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004500 con 0x7f55c80c5850 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0003610 con 0x7f55c80c5850 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.575+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c00039e0 con 0x7f55c80c5850 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.576+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c80c7af0 con 0x7f55c80c5850 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.576+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c80c80b0 con 0x7f55c80c5850 2023-10-21T16:20:15.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.577+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c80c8270 con 0x7f55c80c5850 2023-10-21T16:20:15.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.578+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55c0003b40 con 0x7f55c80c5850 2023-10-21T16:20:15.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.579+0000 7f55beffd640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:15.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.579+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f55c0089a50 con 0x7f55c80c5850 2023-10-21T16:20:15.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.579+0000 7f55c659c640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:15.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.581+0000 7f55c659c640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 0x7f55a806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55c80b4680 tx=0x7f55b800b040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:15.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.583+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55c00547b0 con 0x7f55c80c5850 2023-10-21T16:20:15.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.849+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f55c80c97b0 con 0x7f55c80c5850 2023-10-21T16:20:15.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.961+0000 7f55beffd640 1 -- 172.21.15.53:0/1576694079 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v706) v1 ==== 183+0+0 (secure 0 0 0) 0x7f55c0057dd0 con 0x7f55c80c5850 2023-10-21T16:20:15.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 msgr2=0x7f55a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 0x7f55a806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55c80b4680 tx=0x7f55b800b040 comp rx=0 tx=0).stop 2023-10-21T16:20:15.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 msgr2=0x7f55c80c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:15.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f55c80c8d80 tx=0x7f55c0011920 comp rx=0 tx=0).stop 2023-10-21T16:20:15.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 shutdown_connections 2023-10-21T16:20:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55c80c5850 0x7f55c80c5cd0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55c814e5a0 0x7f55c80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 --2- 172.21.15.53:0/1576694079 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55b4003ef0 0x7f55c81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 >> 172.21.15.53:0/1576694079 conn(0x7f55c806a380 msgr2=0x7f55c8155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:15.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 shutdown_connections 2023-10-21T16:20:15.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:15.965+0000 7f55ccd41640 1 -- 172.21.15.53:0/1576694079 wait complete. 2023-10-21T16:20:15.987 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-21T16:20:15.988 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-21T16:20:15.988 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:20:15.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:16.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.213+0000 7f14d094d640 1 Processor -- start 2023-10-21T16:20:16.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.214+0000 7f14d094d640 1 -- start start 2023-10-21T16:20:16.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14d094d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14cc0adc70 0x7f14cc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14d094d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14cc15af10 0x7f14cc0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14d094d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14cc15f7d0 con 0x7f14cc15ab30 2023-10-21T16:20:16.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14d094d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc15efb0 con 0x7f14cc15af10 2023-10-21T16:20:16.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14d094d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14cc15f210 con 0x7f14cc0adc70 2023-10-21T16:20:16.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14cad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f14cc15ab30 0x7f14cc15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39278/0 (socket says 172.21.15.53:39278) 2023-10-21T16:20:16.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14cad9d640 1 -- 172.21.15.53:0/624168380 learned_addr learned my addr 172.21.15.53:0/624168380 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:16.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.215+0000 7f14cb59e640 1 --2- 172.21.15.53:0/624168380 >> v2:172.21.15.53:3300/0 conn(0x7f14cc15af10 0x7f14cc0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.216+0000 7f14c259c640 1 --2- 172.21.15.53:0/624168380 >> v2:172.21.15.124:3300/0 conn(0x7f14cc0adc70 0x7f14cc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.216+0000 7f14cb59e640 1 -- 172.21.15.53:0/624168380 >> v2:172.21.15.124:3300/0 conn(0x7f14cc0adc70 msgr2=0x7f14cc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:16.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.216+0000 7f14cb59e640 1 --2- 172.21.15.53:0/624168380 >> v2:172.21.15.124:3300/0 conn(0x7f14cc0adc70 0x7f14cc0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.216+0000 7f14cb59e640 1 -- 172.21.15.53:0/624168380 >> v1:172.21.15.53:6789/0 conn(0x7f14cc15ab30 legacy=0x7f14cc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.216+0000 7f14cb59e640 1 -- 172.21.15.53:0/624168380 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc15f4f0 con 0x7f14cc15af10 2023-10-21T16:20:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.217+0000 7f14cb59e640 1 --2- 172.21.15.53:0/624168380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15af10 0x7f14cc0b0600 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0c56c0 tx=0x7f14c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe62ff6c32c56b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.217+0000 7f14ca59c640 1 -- 172.21.15.53:0/624168380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c402cc80 con 0x7f14cc15af10 2023-10-21T16:20:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14ca59c640 1 -- 172.21.15.53:0/624168380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c402cde0 con 0x7f14cc15af10 2023-10-21T16:20:16.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14ca59c640 1 -- 172.21.15.53:0/624168380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4034660 con 0x7f14cc15af10 2023-10-21T16:20:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 -- 172.21.15.53:0/624168380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15af10 msgr2=0x7f14cc0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 --2- 172.21.15.53:0/624168380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15af10 0x7f14cc0b0600 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0c56c0 tx=0x7f14c402bed0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 -- 172.21.15.53:0/624168380 shutdown_connections 2023-10-21T16:20:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 --2- 172.21.15.53:0/624168380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15af10 0x7f14cc0b0600 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 --2- 172.21.15.53:0/624168380 >> v2:172.21.15.124:3300/0 conn(0x7f14cc0adc70 0x7f14cc0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 -- 172.21.15.53:0/624168380 >> 172.21.15.53:0/624168380 conn(0x7f14cc0cb0c0 msgr2=0x7f14cc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 -- 172.21.15.53:0/624168380 shutdown_connections 2023-10-21T16:20:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.218+0000 7f14d094d640 1 -- 172.21.15.53:0/624168380 wait complete. 2023-10-21T16:20:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.220+0000 7f14d094d640 1 Processor -- start 2023-10-21T16:20:16.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.220+0000 7f14d094d640 1 -- start start 2023-10-21T16:20:16.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14cc0adc70 0x7f14cc15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14cc15af10 0x7f14cc0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc15bc00 con 0x7f14cc15ab30 2023-10-21T16:20:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc1f2c20 con 0x7f14cc15af10 2023-10-21T16:20:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.221+0000 7f14d094d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc1f2ec0 con 0x7f14cc0adc70 2023-10-21T16:20:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46082/0 (socket says 172.21.15.53:46082) 2023-10-21T16:20:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 -- 172.21.15.53:0/2829951197 learned_addr learned my addr 172.21.15.53:0/2829951197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14cb59e640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14cc15af10 0x7f14cc0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14cad9d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14cc0adc70 0x7f14cc15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 -- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14cc0adc70 msgr2=0x7f14cc15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14cc0adc70 0x7f14cc15dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 -- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14cc15af10 msgr2=0x7f14cc0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14cc15af10 0x7f14cc0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.222+0000 7f14c259c640 1 -- 172.21.15.53:0/2829951197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4009a70 con 0x7f14cc15ab30 2023-10-21T16:20:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14c259c640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f14b400d800 tx=0x7f14b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b400b6c0 con 0x7f14cc15ab30 2023-10-21T16:20:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b400b820 con 0x7f14cc15ab30 2023-10-21T16:20:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b400bbb0 con 0x7f14cc15ab30 2023-10-21T16:20:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc1f31a0 con 0x7f14cc15ab30 2023-10-21T16:20:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.223+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc1f3a20 con 0x7f14cc15ab30 2023-10-21T16:20:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.224+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc1f3be0 con 0x7f14cc15ab30 2023-10-21T16:20:16.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.225+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f14b400bd10 con 0x7f14cc15ab30 2023-10-21T16:20:16.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.226+0000 7f14c3fff640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 0x7f14ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.226+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f14b4089f70 con 0x7f14cc15ab30 2023-10-21T16:20:16.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.227+0000 7f14cad9d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 0x7f14ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.228+0000 7f14cad9d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 0x7f14ac06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000f60 tx=0x7f14bc009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.231+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14b4054cd0 con 0x7f14cc15ab30 2023-10-21T16:20:16.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.425+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14cc1f3660 con 0x7f14cc15ab30 2023-10-21T16:20:16.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.427+0000 7f14c3fff640 1 -- 172.21.15.53:0/2829951197 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+14466 (secure 0 0 0) 0x7f14b40582f0 con 0x7f14cc15ab30 2023-10-21T16:20:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:16.429 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":125,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:13.938250+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6835/3805349006":"2023-10-22T16:20:10.921328+0000","172.21.15.124:0/3147619579":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6833/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3354756863":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/2281736141":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1223499974":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3104959929":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/216550556":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/4133714254":"2023-10-22T16:20:06.775731+0000","172.21.15.124:6832/2567187932":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/1757063465":"2023-10-22T16:20:06.775731+0000","172.21.15.124:0/3937751236":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2028143540":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6837/3805349006":"2023-10-22T16:20:10.921328+0000","172.21.15.124:0/130280090":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/1496413032":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6833/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:6832/1967358544":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/3651970368":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/28232742":"2023-10-22T16:18:58.493629+0000","172.21.15.124:0/2272255357":"2023-10-21T17:18:51.635575+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:16.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 msgr2=0x7f14ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 0x7f14ac06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000f60 tx=0x7f14bc009040 comp rx=0 tx=0).stop 2023-10-21T16:20:16.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 msgr2=0x7f14cc1f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f14b400d800 tx=0x7f14b400dcc0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 shutdown_connections 2023-10-21T16:20:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f14ac06af90 0x7f14ac06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14cc15af10 0x7f14cc0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14cc15ab30 0x7f14cc1f26e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 --2- 172.21.15.53:0/2829951197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14cc0adc70 0x7f14cc15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 >> 172.21.15.53:0/2829951197 conn(0x7f14cc0cb0c0 msgr2=0x7f14cc06d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:16.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 shutdown_connections 2023-10-21T16:20:16.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.430+0000 7f14d094d640 1 -- 172.21.15.53:0/2829951197 wait complete. 2023-10-21T16:20:16.450 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:20:16.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:20:16.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.682+0000 7f040543f640 1 Processor -- start 2023-10-21T16:20:16.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.683+0000 7f040543f640 1 -- start start 2023-10-21T16:20:16.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f040015aad0 0x7f04000b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f04000c82b0 0x7f04000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040006d6b0 con 0x7f040015aad0 2023-10-21T16:20:16.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040015bf30 con 0x7f04000c82b0 2023-10-21T16:20:16.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.684+0000 7f040543f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040015b320 con 0x7f040006cd00 2023-10-21T16:20:16.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.685+0000 7f03feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f040015aad0 0x7f04000b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.685+0000 7f03feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f040015aad0 0x7f04000b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46098/0 (socket says 172.21.15.53:46098) 2023-10-21T16:20:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.685+0000 7f03feffd640 1 -- 172.21.15.53:0/3732856429 learned_addr learned my addr 172.21.15.53:0/3732856429 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.685+0000 7f03fe7fc640 1 --2- 172.21.15.53:0/3732856429 >> v2:172.21.15.124:3300/0 conn(0x7f04000c82b0 0x7f04000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.685+0000 7f03ff7fe640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.686+0000 7f03feffd640 1 -- 172.21.15.53:0/3732856429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 msgr2=0x7f040006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:16.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.686+0000 7f03feffd640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.686+0000 7f03feffd640 1 -- 172.21.15.53:0/3732856429 >> v2:172.21.15.124:3300/0 conn(0x7f04000c82b0 msgr2=0x7f04000c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.686+0000 7f03feffd640 1 --2- 172.21.15.53:0/3732856429 >> v2:172.21.15.124:3300/0 conn(0x7f04000c82b0 0x7f04000c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.686+0000 7f03feffd640 1 -- 172.21.15.53:0/3732856429 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040015b600 con 0x7f040015aad0 2023-10-21T16:20:16.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.687+0000 7f03feffd640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f040015aad0 0x7f04000b1750 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f040006e330 tx=0x7f03f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff41e3e080ac9f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.687+0000 7f03fdffb640 1 -- 172.21.15.53:0/3732856429 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f002ca70 con 0x7f040015aad0 2023-10-21T16:20:16.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.687+0000 7f03fdffb640 1 -- 172.21.15.53:0/3732856429 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f002cbd0 con 0x7f040015aad0 2023-10-21T16:20:16.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.687+0000 7f03fdffb640 1 -- 172.21.15.53:0/3732856429 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f00343f0 con 0x7f040015aad0 2023-10-21T16:20:16.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 -- 172.21.15.53:0/3732856429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f040015aad0 msgr2=0x7f04000b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f040015aad0 0x7f04000b1750 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f040006e330 tx=0x7f03f002bed0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 -- 172.21.15.53:0/3732856429 shutdown_connections 2023-10-21T16:20:16.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 --2- 172.21.15.53:0/3732856429 >> v2:172.21.15.124:3300/0 conn(0x7f04000c82b0 0x7f04000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 --2- 172.21.15.53:0/3732856429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f040015aad0 0x7f04000b1750 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 -- 172.21.15.53:0/3732856429 >> 172.21.15.53:0/3732856429 conn(0x7f0400069d50 msgr2=0x7f040006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:16.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 -- 172.21.15.53:0/3732856429 shutdown_connections 2023-10-21T16:20:16.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.688+0000 7f040543f640 1 -- 172.21.15.53:0/3732856429 wait complete. 2023-10-21T16:20:16.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.690+0000 7f040543f640 1 Processor -- start 2023-10-21T16:20:16.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.690+0000 7f040543f640 1 -- start start 2023-10-21T16:20:16.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f040015aad0 0x7f04001401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040014f8a0 con 0x7f04000c82b0 2023-10-21T16:20:16.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04000b0c70 con 0x7f040015aad0 2023-10-21T16:20:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f040543f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04000b17f0 con 0x7f040006cd00 2023-10-21T16:20:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.691+0000 7f03fe7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46100/0 (socket says 172.21.15.53:46100) 2023-10-21T16:20:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 -- 172.21.15.53:0/2595694074 learned_addr learned my addr 172.21.15.53:0/2595694074 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03feffd640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03ff7fe640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f040015aad0 0x7f04001401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 -- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 msgr2=0x7f040014eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040014eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 -- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f040015aad0 msgr2=0x7f04001401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f040015aad0 0x7f04001401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 -- 172.21.15.53:0/2595694074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0009b30 con 0x7f04000c82b0 2023-10-21T16:20:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.692+0000 7f03fe7fc640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f03e800e9e0 tx=0x7f03e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.693+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8009670 con 0x7f04000c82b0 2023-10-21T16:20:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.693+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e800c970 con 0x7f04000c82b0 2023-10-21T16:20:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.693+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800cd20 con 0x7f04000c82b0 2023-10-21T16:20:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.693+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04000b0ec0 con 0x7f04000c82b0 2023-10-21T16:20:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.693+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04000b1480 con 0x7f04000c82b0 2023-10-21T16:20:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.694+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040004e160 con 0x7f04000c82b0 2023-10-21T16:20:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.695+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03e8013470 con 0x7f04000c82b0 2023-10-21T16:20:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.697+0000 7f03f77fe640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 0x7f03e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.697+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f03e8089e80 con 0x7f04000c82b0 2023-10-21T16:20:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.697+0000 7f03feffd640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 0x7f03e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.698+0000 7f03feffd640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 0x7f03e006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f03f0035000 tx=0x7f03f0030000 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.702+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03e8054c70 con 0x7f04000c82b0 2023-10-21T16:20:16.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.898+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f040015ee20 con 0x7f04000c82b0 2023-10-21T16:20:16.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.899+0000 7f03f77fe640 1 -- 172.21.15.53:0/2595694074 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03e80170a0 con 0x7f04000c82b0 2023-10-21T16:20:16.900 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:20:16.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.901+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 msgr2=0x7f03e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.901+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 0x7f03e006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f03f0035000 tx=0x7f03f0030000 comp rx=0 tx=0).stop 2023-10-21T16:20:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 msgr2=0x7f040014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f03e800e9e0 tx=0x7f03e800eea0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 shutdown_connections 2023-10-21T16:20:16.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f03e006af90 0x7f03e006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f040015aad0 0x7f04001401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04000c82b0 0x7f040014f360 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 --2- 172.21.15.53:0/2595694074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f040006cd00 0x7f040014eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:16.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 >> 172.21.15.53:0/2595694074 conn(0x7f0400069d50 msgr2=0x7f040015c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:16.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 shutdown_connections 2023-10-21T16:20:16.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:16.902+0000 7f040543f640 1 -- 172.21.15.53:0/2595694074 wait complete. 2023-10-21T16:20:16.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-21T16:20:17.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.146+0000 7f80826e6640 1 Processor -- start 2023-10-21T16:20:17.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.146+0000 7f80826e6640 1 -- start start 2023-10-21T16:20:17.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80826e6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f807c15a970 0x7f807c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:17.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80826e6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c06cfd0 con 0x7f807c0cb990 2023-10-21T16:20:17.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80826e6640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c06d870 con 0x7f807c06dd00 2023-10-21T16:20:17.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80826e6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0cbe00 con 0x7f807c15a970 2023-10-21T16:20:17.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80816e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f807c15a970 0x7f807c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:17.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f8081ee5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f807c0cb990 0x7f807c0b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39280/0 (socket says 172.21.15.53:39280) 2023-10-21T16:20:17.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f80816e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f807c15a970 0x7f807c0cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46102/0 (socket says 172.21.15.53:46102) 2023-10-21T16:20:17.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.148+0000 7f8081ee5640 1 -- 172.21.15.53:0/1736048510 learned_addr learned my addr 172.21.15.53:0/1736048510 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:17.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.149+0000 7f80816e4640 1 -- 172.21.15.53:0/1736048510 >> v1:172.21.15.124:6789/0 conn(0x7f807c06dd00 legacy=0x7f807c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:17.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.149+0000 7f80816e4640 1 -- 172.21.15.53:0/1736048510 >> v1:172.21.15.53:6789/0 conn(0x7f807c0cb990 legacy=0x7f807c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:17.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.149+0000 7f80816e4640 1 -- 172.21.15.53:0/1736048510 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c15ad50 con 0x7f807c15a970 2023-10-21T16:20:17.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.150+0000 7f80816e4640 1 --2- 172.21.15.53:0/1736048510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c0cb450 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f80780097a0 tx=0x7f807802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3c1ff359bc3a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:17.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.150+0000 7f8073fff640 1 -- 172.21.15.53:0/1736048510 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807802e070 con 0x7f807c15a970 2023-10-21T16:20:17.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.150+0000 7f8073fff640 1 -- 172.21.15.53:0/1736048510 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80780315a0 con 0x7f807c15a970 2023-10-21T16:20:17.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.150+0000 7f8073fff640 1 -- 172.21.15.53:0/1736048510 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078031930 con 0x7f807c15a970 2023-10-21T16:20:17.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 -- 172.21.15.53:0/1736048510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 msgr2=0x7f807c0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:17.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 --2- 172.21.15.53:0/1736048510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c0cb450 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f80780097a0 tx=0x7f807802b840 comp rx=0 tx=0).stop 2023-10-21T16:20:17.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 -- 172.21.15.53:0/1736048510 shutdown_connections 2023-10-21T16:20:17.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 --2- 172.21.15.53:0/1736048510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c0cb450 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:17.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 -- 172.21.15.53:0/1736048510 >> 172.21.15.53:0/1736048510 conn(0x7f807c0ce9d0 msgr2=0x7f807c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:17.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 -- 172.21.15.53:0/1736048510 shutdown_connections 2023-10-21T16:20:17.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.151+0000 7f80826e6640 1 -- 172.21.15.53:0/1736048510 wait complete. 2023-10-21T16:20:17.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.153+0000 7f80826e6640 1 Processor -- start 2023-10-21T16:20:17.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.153+0000 7f80826e6640 1 -- start start 2023-10-21T16:20:17.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c06dd00 0x7f807c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:17.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 0x7f807c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:17.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:17.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0ae0b0 con 0x7f807c15a970 2023-10-21T16:20:17.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0ae350 con 0x7f807c0cb990 2023-10-21T16:20:17.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f80826e6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0ae630 con 0x7f807c06dd00 2023-10-21T16:20:17.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f8081ee5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:17.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f8081ee5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46110/0 (socket says 172.21.15.53:46110) 2023-10-21T16:20:17.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.154+0000 7f8081ee5640 1 -- 172.21.15.53:0/357089714 learned_addr learned my addr 172.21.15.53:0/357089714 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:17.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8080ee3640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 0x7f807c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:17.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f80816e4640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c06dd00 0x7f807c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:17.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 -- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c06dd00 msgr2=0x7f807c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c06dd00 0x7f807c0aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 -- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 msgr2=0x7f807c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 0x7f807c0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 -- 172.21.15.53:0/357089714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078009400 con 0x7f807c15a970 2023-10-21T16:20:17.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8080ee3640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 0x7f807c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8081ee5640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f807400cbc0 tx=0x7f807400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.155+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400ad40 con 0x7f807c15a970 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.156+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074012400 con 0x7f807c15a970 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.156+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c0ae910 con 0x7f807c15a970 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.156+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400aea0 con 0x7f807c15a970 2023-10-21T16:20:17.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.156+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c0c4f70 con 0x7f807c15a970 2023-10-21T16:20:17.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.157+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c04e160 con 0x7f807c15a970 2023-10-21T16:20:17.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.158+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8074007070 con 0x7f807c15a970 2023-10-21T16:20:17.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.160+0000 7f8071ffb640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 0x7f806006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:17.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.160+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f8074089880 con 0x7f807c15a970 2023-10-21T16:20:17.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.161+0000 7f80816e4640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 0x7f806006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:17.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.162+0000 7f80816e4640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 0x7f806006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8078036000 tx=0x7f807803d040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:17.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.166+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8074054600 con 0x7f807c15a970 2023-10-21T16:20:17.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:17.433+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f807c06cb80 con 0x7f807c15a970 2023-10-21T16:20:18.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.372+0000 7f8071ffb640 1 -- 172.21.15.53:0/357089714 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v708) v1 ==== 151+0+0 (secure 0 0 0) 0x7f80740170d0 con 0x7f807c15a970 2023-10-21T16:20:18.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 msgr2=0x7f806006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 0x7f806006d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8078036000 tx=0x7f807803d040 comp rx=0 tx=0).stop 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 msgr2=0x7f807c15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f807400cbc0 tx=0x7f807400a430 comp rx=0 tx=0).stop 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 shutdown_connections 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f806006afc0 0x7f806006d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f807c15a970 0x7f807c15c8e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f807c0cb990 0x7f807c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 --2- 172.21.15.53:0/357089714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f807c06dd00 0x7f807c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.377+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 >> 172.21.15.53:0/357089714 conn(0x7f807c0ce9d0 msgr2=0x7f807c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.378+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 shutdown_connections 2023-10-21T16:20:18.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.378+0000 7f80826e6640 1 -- 172.21.15.53:0/357089714 wait complete. 2023-10-21T16:20:18.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:20:18.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.631+0000 7f5d16747640 1 Processor -- start 2023-10-21T16:20:18.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.631+0000 7f5d16747640 1 -- start start 2023-10-21T16:20:18.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d101599d0 0x7f5d100cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d100b3c50 0x7f5d100caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 0x7f5d100c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d100cb8a0 con 0x7f5d101599d0 2023-10-21T16:20:18.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d10159db0 con 0x7f5d100cb410 2023-10-21T16:20:18.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.633+0000 7f5d16747640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10071fe0 con 0x7f5d100b3c50 2023-10-21T16:20:18.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d101599d0 0x7f5d100cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d101599d0 0x7f5d100cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46126/0 (socket says 172.21.15.53:46126) 2023-10-21T16:20:18.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 -- 172.21.15.53:0/3085344031 learned_addr learned my addr 172.21.15.53:0/3085344031 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:18.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d14f44640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d100b3c50 0x7f5d100caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15f46640 1 --2- 172.21.15.53:0/3085344031 >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 0x7f5d100c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 -- 172.21.15.53:0/3085344031 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d100b3c50 msgr2=0x7f5d100caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:18.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d100b3c50 0x7f5d100caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 -- 172.21.15.53:0/3085344031 >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 msgr2=0x7f5d100c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:18.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 --2- 172.21.15.53:0/3085344031 >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 0x7f5d100c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15745640 1 -- 172.21.15.53:0/3085344031 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100b44c0 con 0x7f5d101599d0 2023-10-21T16:20:18.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.634+0000 7f5d15f46640 1 --2- 172.21.15.53:0/3085344031 >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 0x7f5d100c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.635+0000 7f5d15745640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101599d0 0x7f5d100cdcd0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5d100cbf80 tx=0x7f5d0c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed927954a1e04f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.636+0000 7f5d07fff640 1 -- 172.21.15.53:0/3085344031 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c031860 con 0x7f5d101599d0 2023-10-21T16:20:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.636+0000 7f5d07fff640 1 -- 172.21.15.53:0/3085344031 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0319c0 con 0x7f5d101599d0 2023-10-21T16:20:18.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.636+0000 7f5d07fff640 1 -- 172.21.15.53:0/3085344031 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c031d50 con 0x7f5d101599d0 2023-10-21T16:20:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.636+0000 7f5d16747640 1 -- 172.21.15.53:0/3085344031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101599d0 msgr2=0x7f5d100cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.636+0000 7f5d16747640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101599d0 0x7f5d100cdcd0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5d100cbf80 tx=0x7f5d0c02b900 comp rx=0 tx=0).stop 2023-10-21T16:20:18.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 -- 172.21.15.53:0/3085344031 shutdown_connections 2023-10-21T16:20:18.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d100b3c50 0x7f5d100caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 --2- 172.21.15.53:0/3085344031 >> v2:172.21.15.124:3300/0 conn(0x7f5d100cb410 0x7f5d100c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 --2- 172.21.15.53:0/3085344031 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d101599d0 0x7f5d100cdcd0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 -- 172.21.15.53:0/3085344031 >> 172.21.15.53:0/3085344031 conn(0x7f5d1006c050 msgr2=0x7f5d1006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:18.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 -- 172.21.15.53:0/3085344031 shutdown_connections 2023-10-21T16:20:18.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.637+0000 7f5d16747640 1 -- 172.21.15.53:0/3085344031 wait complete. 2023-10-21T16:20:18.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.639+0000 7f5d16747640 1 Processor -- start 2023-10-21T16:20:18.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.639+0000 7f5d16747640 1 -- start start 2023-10-21T16:20:18.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d100b3c50 0x7f5d100ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d101599d0 0x7f5d1014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d101f5670 con 0x7f5d100cb410 2023-10-21T16:20:18.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1014cc90 con 0x7f5d100b3c50 2023-10-21T16:20:18.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d16747640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1014cf70 con 0x7f5d101599d0 2023-10-21T16:20:18.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d14f44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d14f44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46132/0 (socket says 172.21.15.53:46132) 2023-10-21T16:20:18.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.641+0000 7f5d14f44640 1 -- 172.21.15.53:0/2394588036 learned_addr learned my addr 172.21.15.53:0/2394588036 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:18.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d15745640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d100b3c50 0x7f5d100ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d15f46640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d101599d0 0x7f5d1014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 -- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d101599d0 msgr2=0x7f5d1014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d101599d0 0x7f5d1014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 -- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d100b3c50 msgr2=0x7f5d100ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d100b3c50 0x7f5d100ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 -- 172.21.15.53:0/2394588036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0094d0 con 0x7f5d100cb410 2023-10-21T16:20:18.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d14f44640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000d800 tx=0x7f5d0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:18.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000096a0 con 0x7f5d100cb410 2023-10-21T16:20:18.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.642+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00009800 con 0x7f5d100cb410 2023-10-21T16:20:18.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.643+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00009b90 con 0x7f5d100cb410 2023-10-21T16:20:18.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.643+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1014d250 con 0x7f5d100cb410 2023-10-21T16:20:18.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.643+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d100ac090 con 0x7f5d100cb410 2023-10-21T16:20:18.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.644+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d101f58e0 con 0x7f5d100cb410 2023-10-21T16:20:18.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.645+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d00009cf0 con 0x7f5d100cb410 2023-10-21T16:20:18.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.646+0000 7f5d05ffb640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 0x7f5cf406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:18.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.646+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f5d0008a0f0 con 0x7f5d100cb410 2023-10-21T16:20:18.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.646+0000 7f5d15745640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 0x7f5cf406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:18.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.648+0000 7f5d15745640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 0x7f5cf406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c036000 tx=0x7f5d0c03c040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:18.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.652+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d00054e50 con 0x7f5d100cb410 2023-10-21T16:20:18.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.848+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5d10074150 con 0x7f5d100cb410 2023-10-21T16:20:18.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.855+0000 7f5d05ffb640 1 -- 172.21.15.53:0/2394588036 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5d0000b440 con 0x7f5d100cb410 2023-10-21T16:20:18.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 msgr2=0x7f5cf406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 0x7f5cf406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c036000 tx=0x7f5d0c03c040 comp rx=0 tx=0).stop 2023-10-21T16:20:18.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 msgr2=0x7f5d10149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:18.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000d800 tx=0x7f5d0000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 shutdown_connections 2023-10-21T16:20:18.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5cf406af90 0x7f5cf406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d101599d0 0x7f5d1014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d100cb410 0x7f5d10149d60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 --2- 172.21.15.53:0/2394588036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d100b3c50 0x7f5d100ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:18.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.858+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 >> 172.21.15.53:0/2394588036 conn(0x7f5d1006c050 msgr2=0x7f5d100b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:18.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.859+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 shutdown_connections 2023-10-21T16:20:18.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:18.859+0000 7f5d16747640 1 -- 172.21.15.53:0/2394588036 wait complete. 2023-10-21T16:20:18.881 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:20:18.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:20:19.009 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:20:19.129 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:20:19.130 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:19.130 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:19.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:20:19.196 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:20:19.196 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:19.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:20:19.259 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:19.259 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:19.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:20:19.323 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:20:19.323 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:19.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:20:19.387 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:20:19.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:19.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.615+0000 7f5a9d6ad640 1 Processor -- start 2023-10-21T16:20:19.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.615+0000 7f5a9d6ad640 1 -- start start 2023-10-21T16:20:19.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9d6ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 0x7f5a980b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9d6ad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9815f7d0 con 0x7f5a9815af10 2023-10-21T16:20:19.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9d6ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9815efb0 con 0x7f5a9815ab30 2023-10-21T16:20:19.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9d6ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815f210 con 0x7f5a980adc70 2023-10-21T16:20:19.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9ceac640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5a9815af10 0x7f5a9806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39290/0 (socket says 172.21.15.53:39290) 2023-10-21T16:20:19.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.617+0000 7f5a9ceac640 1 -- 172.21.15.53:0/317314826 learned_addr learned my addr 172.21.15.53:0/317314826 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:19.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a977fe640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 0x7f5a980b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807433171 0 0) 0x7f5a9815f7d0 con 0x7f5a9815af10 2023-10-21T16:20:19.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a80003520 con 0x7f5a9815af10 2023-10-21T16:20:19.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072522026 0 0) 0x7f5a80003520 con 0x7f5a9815af10 2023-10-21T16:20:19.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 msgr2=0x7f5a980b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:19.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 0x7f5a980b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 >> v1:172.21.15.124:6789/0 conn(0x7f5a9815ab30 legacy=0x7f5a9815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9815f4f0 con 0x7f5a9815af10 2023-10-21T16:20:19.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5a90001a10 con 0x7f5a9815af10 2023-10-21T16:20:19.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 >> v1:172.21.15.53:6789/0 conn(0x7f5a9815af10 legacy=0x7f5a9806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a800039a0 0x7f5a80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 0x7f5a800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a80004d00 0x7f5a80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815f210 con 0x7f5a800039a0 2023-10-21T16:20:19.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000ba90 con 0x7f5a80004d00 2023-10-21T16:20:19.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.618+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8000bc50 con 0x7f5a980adc70 2023-10-21T16:20:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a800039a0 0x7f5a80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a9ceac640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a80004d00 0x7f5a80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 -- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 msgr2=0x7f5a800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a980adc70 0x7f5a800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 -- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a80004d00 msgr2=0x7f5a80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:19.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a80004d00 0x7f5a80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.619+0000 7f5a97fff640 1 -- 172.21.15.53:0/317314826 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80003520 con 0x7f5a800039a0 2023-10-21T16:20:19.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.620+0000 7f5a97fff640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a800039a0 0x7f5a80003dc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f5a88001720 tx=0x7f5a800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab7f1b3f9ddb656 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:19.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880052d0 con 0x7f5a800039a0 2023-10-21T16:20:19.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8802e050 con 0x7f5a800039a0 2023-10-21T16:20:19.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a96ffd640 1 -- 172.21.15.53:0/317314826 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880035f0 con 0x7f5a800039a0 2023-10-21T16:20:19.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/317314826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a800039a0 msgr2=0x7f5a80003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/317314826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a800039a0 0x7f5a80003dc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f5a88001720 tx=0x7f5a800057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9ceac640 1 -- 172.21.15.53:0/317314826 reap_dead start 2023-10-21T16:20:19.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/317314826 shutdown_connections 2023-10-21T16:20:19.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/317314826 >> 172.21.15.53:0/317314826 conn(0x7f5a980cb0c0 msgr2=0x7f5a980cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:19.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/317314826 shutdown_connections 2023-10-21T16:20:19.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.621+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/317314826 wait complete. 2023-10-21T16:20:19.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.623+0000 7f5a9d6ad640 1 Processor -- start 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.623+0000 7f5a9d6ad640 1 -- start start 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a800039a0 0x7f5a98158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a9815d000 0x7f5a9815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815d9c0 con 0x7f5a980c7ca0 2023-10-21T16:20:19.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981593e0 con 0x7f5a800039a0 2023-10-21T16:20:19.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.624+0000 7f5a9d6ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815c090 con 0x7f5a9815d000 2023-10-21T16:20:19.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46144/0 (socket says 172.21.15.53:46144) 2023-10-21T16:20:19.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 -- 172.21.15.53:0/370244859 learned_addr learned my addr 172.21.15.53:0/370244859 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:19.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a97fff640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a800039a0 0x7f5a98158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 -- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a9815d000 msgr2=0x7f5a9815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a9815d000 0x7f5a9815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 -- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a800039a0 msgr2=0x7f5a98158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a800039a0 0x7f5a98158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 -- 172.21.15.53:0/370244859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9815c370 con 0x7f5a980c7ca0 2023-10-21T16:20:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.625+0000 7f5a977fe640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00e7c0 tx=0x7f5a8c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.626+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00eed0 con 0x7f5a980c7ca0 2023-10-21T16:20:19.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.626+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00c940 con 0x7f5a980c7ca0 2023-10-21T16:20:19.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.626+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00ccd0 con 0x7f5a980c7ca0 2023-10-21T16:20:19.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.626+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9815c650 con 0x7f5a980c7ca0 2023-10-21T16:20:19.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.626+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9815cc10 con 0x7f5a980c7ca0 2023-10-21T16:20:19.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.627+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9804e160 con 0x7f5a980c7ca0 2023-10-21T16:20:19.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.628+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5a8c013470 con 0x7f5a980c7ca0 2023-10-21T16:20:19.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.629+0000 7f5a94ff9640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 0x7f5a8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:19.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.630+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f5a8c0899e0 con 0x7f5a980c7ca0 2023-10-21T16:20:19.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.630+0000 7f5a97fff640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 0x7f5a8406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:19.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.631+0000 7f5a97fff640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 0x7f5a8406d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a88033000 tx=0x7f5a88009030 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:19.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.634+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a8c054740 con 0x7f5a980c7ca0 2023-10-21T16:20:19.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.864+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a980ada50 con 0x7f5a980c7ca0 2023-10-21T16:20:19.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.865+0000 7f5a94ff9640 1 -- 172.21.15.53:0/370244859 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3704 (secure 0 0 0) 0x7f5a8c021e90 con 0x7f5a980c7ca0 2023-10-21T16:20:19.867 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:19.867 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/659037109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":659037109},{"type":"v1","addr":"172.21.15.53:6834","nonce":659037109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3259791694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3259791694},{"type":"v1","addr":"172.21.15.53:6835","nonce":3259791694}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":35912,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3491547928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3491547928},{"type":"v1","addr":"172.21.15.124:6836","nonce":3491547928}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":36164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1492816269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1492816269},{"type":"v1","addr":"172.21.15.124:6837","nonce":1492816269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2023-10-21T16:20:19.867 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 55 2023-10-21T16:20:19.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 msgr2=0x7f5a8406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 0x7f5a8406d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a88033000 tx=0x7f5a88009030 comp rx=0 tx=0).stop 2023-10-21T16:20:19.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 msgr2=0x7f5a980c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:19.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00e7c0 tx=0x7f5a8c00ec80 comp rx=0 tx=0).stop 2023-10-21T16:20:19.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 shutdown_connections 2023-10-21T16:20:19.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f5a8406af90 0x7f5a8406d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a9815d000 0x7f5a9815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a980c7ca0 0x7f5a980c8120 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 --2- 172.21.15.53:0/370244859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a800039a0 0x7f5a98158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:19.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 >> 172.21.15.53:0/370244859 conn(0x7f5a980cb0c0 msgr2=0x7f5a98070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:19.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 shutdown_connections 2023-10-21T16:20:19.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:19.868+0000 7f5a9d6ad640 1 -- 172.21.15.53:0/370244859 wait complete. 2023-10-21T16:20:19.890 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:20:19.891 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:20:19.891 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:20:19.892 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:20:19.892 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:20:19.892 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:20:19.892 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:20:19.893 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:20:19.893 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:20:19.893 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:20:19.894 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:20:19.894 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:20:19.894 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:20:19.894 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:20:19.895 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:20:19.895 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:20:19.915 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:20:19.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:20:19.916 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:20:19.916 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:20:19.917 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:20:19.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:20:19.918 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:20:19.919 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:20:19.964 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:20:19.965 DEBUG:teuthology.parallel:result is None 2023-10-21T16:20:19.967 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:20:19.967 DEBUG:teuthology.parallel:result is None 2023-10-21T16:20:19.971 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:20:19.971 DEBUG:teuthology.parallel:result is None 2023-10-21T16:20:19.974 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:20:19.975 DEBUG:teuthology.parallel:result is None 2023-10-21T16:20:19.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:20:20.125 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:20:20.132 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:20:20.132 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:20:20.140 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:20:20.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.166+0000 7f8618446640 1 Processor -- start 2023-10-21T16:20:20.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.166+0000 7f8618446640 1 -- start start 2023-10-21T16:20:20.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f8618446640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8610160660 0x7f8610070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f8618446640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f8618446640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861006e150 con 0x7f86100cb450 2023-10-21T16:20:20.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f8618446640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86100cb8c0 con 0x7f8610160660 2023-10-21T16:20:20.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f8618446640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861006de70 con 0x7f86100c9910 2023-10-21T16:20:20.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f86159e1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f86161e2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8610160660 0x7f8610070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f86159e1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:54506/0 (socket says 172.21.15.53:54506) 2023-10-21T16:20:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f86159e1640 1 -- 172.21.15.53:0/619107067 learned_addr learned my addr 172.21.15.53:0/619107067 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.167+0000 7f86169e3640 1 --1- 172.21.15.53:0/619107067 >> v1:172.21.15.124:6789/0 conn(0x7f86100cb450 0x7f86100b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:60018/0 (socket says 172.21.15.53:60018) 2023-10-21T16:20:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.168+0000 7f86159e1640 1 -- 172.21.15.53:0/619107067 >> v2:172.21.15.124:3300/0 conn(0x7f8610160660 msgr2=0x7f8610070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:20.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.168+0000 7f86159e1640 1 --2- 172.21.15.53:0/619107067 >> v2:172.21.15.124:3300/0 conn(0x7f8610160660 0x7f8610070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.168+0000 7f86159e1640 1 -- 172.21.15.53:0/619107067 >> v1:172.21.15.124:6789/0 conn(0x7f86100cb450 legacy=0x7f86100b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.168+0000 7f86159e1640 1 -- 172.21.15.53:0/619107067 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86100c8e30 con 0x7f86100c9910 2023-10-21T16:20:20.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.169+0000 7f86159e1640 1 --2- 172.21.15.53:0/619107067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8600009870 tx=0x7f860002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210ba15380eb01d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.169+0000 7f86151e0640 1 -- 172.21.15.53:0/619107067 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600030840 con 0x7f86100c9910 2023-10-21T16:20:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.169+0000 7f86151e0640 1 -- 172.21.15.53:0/619107067 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86000309a0 con 0x7f86100c9910 2023-10-21T16:20:20.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.169+0000 7f86151e0640 1 -- 172.21.15.53:0/619107067 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600030d30 con 0x7f86100c9910 2023-10-21T16:20:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 -- 172.21.15.53:0/619107067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 msgr2=0x7f86100caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 --2- 172.21.15.53:0/619107067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8600009870 tx=0x7f860002b980 comp rx=0 tx=0).stop 2023-10-21T16:20:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 -- 172.21.15.53:0/619107067 shutdown_connections 2023-10-21T16:20:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 --2- 172.21.15.53:0/619107067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100caf10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 --2- 172.21.15.53:0/619107067 >> v2:172.21.15.124:3300/0 conn(0x7f8610160660 0x7f8610070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 -- 172.21.15.53:0/619107067 >> 172.21.15.53:0/619107067 conn(0x7f861006a5d0 msgr2=0x7f86100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:20.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 -- 172.21.15.53:0/619107067 shutdown_connections 2023-10-21T16:20:20.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.170+0000 7f8618446640 1 -- 172.21.15.53:0/619107067 wait complete. 2023-10-21T16:20:20.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.172+0000 7f8618446640 1 Processor -- start 2023-10-21T16:20:20.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.172+0000 7f8618446640 1 -- start start 2023-10-21T16:20:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8610160660 0x7f861015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86100c4e00 con 0x7f86100cb450 2023-10-21T16:20:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86101580c0 con 0x7f8610160660 2023-10-21T16:20:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f8618446640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86100c41f0 con 0x7f86100c9910 2023-10-21T16:20:20.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f86159e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f86159e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46198/0 (socket says 172.21.15.53:46198) 2023-10-21T16:20:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f86159e1640 1 -- 172.21.15.53:0/644513732 learned_addr learned my addr 172.21.15.53:0/644513732 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f86161e2640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.173+0000 7f86169e3640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8610160660 0x7f861015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 -- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 msgr2=0x7f86100c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 -- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8610160660 msgr2=0x7f861015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8610160660 0x7f861015f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 -- 172.21.15.53:0/644513732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000094d0 con 0x7f86100cb450 2023-10-21T16:20:20.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f86159e1640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f860002b9c0 tx=0x7f8600004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600039c90 con 0x7f86100cb450 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600039df0 con 0x7f86100cb450 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86100c44d0 con 0x7f86100cb450 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000446c0 con 0x7f86100cb450 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.174+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86100c4a90 con 0x7f86100cb450 2023-10-21T16:20:20.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.176+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861004e160 con 0x7f86100cb450 2023-10-21T16:20:20.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.177+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f860004d3b0 con 0x7f86100cb450 2023-10-21T16:20:20.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.178+0000 7f8606ffd640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 0x7f85f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.178+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6124+0+0 (secure 0 0 0) 0x7f860003e070 con 0x7f86100cb450 2023-10-21T16:20:20.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.179+0000 7f86161e2640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 0x7f85f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.180+0000 7f86161e2640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 0x7f85f406d480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f860c009aa0 tx=0x7f860c009190 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.183+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86000771e0 con 0x7f86100cb450 2023-10-21T16:20:20.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.387+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f86100b1410 con 0x7f86100cb450 2023-10-21T16:20:20.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.409+0000 7f8606ffd640 1 -- 172.21.15.53:0/644513732 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f860007a800 con 0x7f86100cb450 2023-10-21T16:20:20.411 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:20:20.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 msgr2=0x7f85f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 0x7f85f406d480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f860c009aa0 tx=0x7f860c009190 comp rx=0 tx=0).stop 2023-10-21T16:20:20.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 msgr2=0x7f86100aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f860002b9c0 tx=0x7f8600004730 comp rx=0 tx=0).stop 2023-10-21T16:20:20.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 shutdown_connections 2023-10-21T16:20:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f85f406afc0 0x7f85f406d480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8610160660 0x7f861015f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86100cb450 0x7f86100aed60 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 --2- 172.21.15.53:0/644513732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86100c9910 0x7f86100c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.412+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 >> 172.21.15.53:0/644513732 conn(0x7f861006a5d0 msgr2=0x7f86100cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.413+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 shutdown_connections 2023-10-21T16:20:20.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.413+0000 7f8618446640 1 -- 172.21.15.53:0/644513732 wait complete. 2023-10-21T16:20:20.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:20:20.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.640+0000 7f4ea0947640 1 Processor -- start 2023-10-21T16:20:20.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.641+0000 7f4ea0947640 1 -- start start 2023-10-21T16:20:20.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4ea0947640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4e9c0c9570 0x7f4e9c0c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4ea0947640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0718a0 con 0x7f4e9c071450 2023-10-21T16:20:20.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4ea0947640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0c9ec0 con 0x7f4e9c15cd70 2023-10-21T16:20:20.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4ea0947640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c15d150 con 0x7f4e9c0c9570 2023-10-21T16:20:20.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4e9bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4e9c0c9570 0x7f4e9c0c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4e9bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4e9c0c9570 0x7f4e9c0c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46200/0 (socket says 172.21.15.53:46200) 2023-10-21T16:20:20.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4e9bfff640 1 -- 172.21.15.53:0/1324281928 learned_addr learned my addr 172.21.15.53:0/1324281928 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:20.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.642+0000 7f4e9affd640 1 --1- 172.21.15.53:0/1324281928 >> v1:172.21.15.53:6789/0 conn(0x7f4e9c071450 0x7f4e9c06e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39308/0 (socket says 172.21.15.53:39308) 2023-10-21T16:20:20.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.643+0000 7f4e9bfff640 1 -- 172.21.15.53:0/1324281928 >> v1:172.21.15.124:6789/0 conn(0x7f4e9c15cd70 legacy=0x7f4e9c1607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.643+0000 7f4e9bfff640 1 -- 172.21.15.53:0/1324281928 >> v1:172.21.15.53:6789/0 conn(0x7f4e9c071450 legacy=0x7f4e9c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.643+0000 7f4e9a7fc640 1 -- 172.21.15.53:0/1324281928 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976147683 0 0) 0x7f4e9c0718a0 con 0x7f4e9c071450 2023-10-21T16:20:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.643+0000 7f4e9bfff640 1 -- 172.21.15.53:0/1324281928 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c06e000 con 0x7f4e9c0c9570 2023-10-21T16:20:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.644+0000 7f4e9bfff640 1 --2- 172.21.15.53:0/1324281928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c0c9570 0x7f4e9c0c9980 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e50 tx=0x7f4e9402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ddb73f5ef2b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.644+0000 7f4e9a7fc640 1 -- 172.21.15.53:0/1324281928 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9402cb20 con 0x7f4e9c0c9570 2023-10-21T16:20:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.644+0000 7f4e9a7fc640 1 -- 172.21.15.53:0/1324281928 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9402cc80 con 0x7f4e9c0c9570 2023-10-21T16:20:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.644+0000 7f4e9a7fc640 1 -- 172.21.15.53:0/1324281928 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940343f0 con 0x7f4e9c0c9570 2023-10-21T16:20:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 -- 172.21.15.53:0/1324281928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c0c9570 msgr2=0x7f4e9c0c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1324281928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c0c9570 0x7f4e9c0c9980 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e50 tx=0x7f4e9402bf80 comp rx=0 tx=0).stop 2023-10-21T16:20:20.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 -- 172.21.15.53:0/1324281928 shutdown_connections 2023-10-21T16:20:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1324281928 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c0c9570 0x7f4e9c0c9980 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 -- 172.21.15.53:0/1324281928 >> 172.21.15.53:0/1324281928 conn(0x7f4e9c0cb8f0 msgr2=0x7f4e9c0cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 -- 172.21.15.53:0/1324281928 shutdown_connections 2023-10-21T16:20:20.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.645+0000 7f4ea0947640 1 -- 172.21.15.53:0/1324281928 wait complete. 2023-10-21T16:20:20.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.647+0000 7f4ea0947640 1 Processor -- start 2023-10-21T16:20:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.647+0000 7f4ea0947640 1 -- start start 2023-10-21T16:20:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e9c0c9570 0x7f4e9c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9c15cd70 0x7f4e9c0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c150730 con 0x7f4e9c071450 2023-10-21T16:20:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c2b40 con 0x7f4e9c15cd70 2023-10-21T16:20:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4ea0947640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c2e20 con 0x7f4e9c0c9570 2023-10-21T16:20:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46214/0 (socket says 172.21.15.53:46214) 2023-10-21T16:20:20.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.648+0000 7f4e9b7fe640 1 -- 172.21.15.53:0/1276169090 learned_addr learned my addr 172.21.15.53:0/1276169090 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9affd640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e9c0c9570 0x7f4e9c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 -- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e9c0c9570 msgr2=0x7f4e9c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e9c0c9570 0x7f4e9c0afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 -- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9c15cd70 msgr2=0x7f4e9c0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9c15cd70 0x7f4e9c0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 -- 172.21.15.53:0/1276169090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009b30 con 0x7f4e9c071450 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e9b7fe640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00b3f0 tx=0x7f4e8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c011830 con 0x7f4e9c071450 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c011990 con 0x7f4e9c071450 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c011d40 con 0x7f4e9c071450 2023-10-21T16:20:20.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.649+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0c3100 con 0x7f4e9c071450 2023-10-21T16:20:20.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.650+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c0c36c0 con 0x7f4e9c071450 2023-10-21T16:20:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.651+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c04e160 con 0x7f4e9c071450 2023-10-21T16:20:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.652+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4e8c01a400 con 0x7f4e9c071450 2023-10-21T16:20:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.653+0000 7f4e83fff640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 0x7f4e8806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:20.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.653+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f4e8c089400 con 0x7f4e9c071450 2023-10-21T16:20:20.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.653+0000 7f4e9affd640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 0x7f4e8806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:20.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.655+0000 7f4e9affd640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 0x7f4e8806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e90007fd0 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:20.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.657+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e8c054580 con 0x7f4e9c071450 2023-10-21T16:20:20.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.922+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4e9c0cbe70 con 0x7f4e9c071450 2023-10-21T16:20:20.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.923+0000 7f4e83fff640 1 -- 172.21.15.53:0/1276169090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4e8c057ba0 con 0x7f4e9c071450 2023-10-21T16:20:20.925 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:20.925 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:20:20.925 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:20:20.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.925 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:20:20.926 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:20:20.926 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.927 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:20:20.928 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:20:20.928 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:20:20.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:20:20.930 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.931 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:20:20.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:20:20.932 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:20:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:20:20.933 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:20:20.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:20:20.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.936 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.936 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.937 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:20:20.937 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:20:20.937 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.937 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.938 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.938 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.940 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:20:20.940 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:20:20.940 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.942 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:20:20.942 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:20:20.942 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.943 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.943 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.944 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:20:20.944 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:20:20.944 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.944 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.945 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.945 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.946 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:20:20.946 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:20:20.946 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.946 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.947 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.947 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.947 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.948 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:20:20.948 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:20:20.948 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:20:20.948 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:20:20.949 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.949 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:20:20.950 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:20:20.950 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.950 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:20:20.950 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:20:20.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.951 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:20:20.951 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:20:20.952 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.952 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.952 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:20:20.952 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:20:20.953 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:20:20.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.953 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:20:20.954 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:20:20.954 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.954 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.954 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:20:20.954 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:20:20.955 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:20:20.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.955 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:20:20.956 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:20:20.957 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.958 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:20:20.958 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:20:20.958 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:20:20.958 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.959 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:20:20.959 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:20:20.959 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:20:20.960 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:20:20.960 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:20:20.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:20.961 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:20:20.961 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:20:20.961 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:20:20.961 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:20:20.962 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:20:20.962 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:20:20.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.962 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:20.962 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:20:20.963 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:20:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 msgr2=0x7f4e8806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 0x7f4e8806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e90007fd0 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2023-10-21T16:20:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 msgr2=0x7f4e9c1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00b3f0 tx=0x7f4e8c00b8b0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 shutdown_connections 2023-10-21T16:20:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f4e8806afc0 0x7f4e8806d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9c15cd70 0x7f4e9c0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e9c0c9570 0x7f4e9c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 --2- 172.21.15.53:0/1276169090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e9c071450 0x7f4e9c1501f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 >> 172.21.15.53:0/1276169090 conn(0x7f4e9c0cb8f0 msgr2=0x7f4e9c0ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 shutdown_connections 2023-10-21T16:20:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:20.926+0000 7f4ea0947640 1 -- 172.21.15.53:0/1276169090 wait complete. 2023-10-21T16:20:20.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:21.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.197+0000 7f8b4e459640 1 Processor -- start 2023-10-21T16:20:21.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.197+0000 7f8b4e459640 1 -- start start 2023-10-21T16:20:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4e459640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b481599d0 0x7f8b480ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4e459640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 0x7f8b480b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4e459640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480cacf0 con 0x7f8b48159db0 2023-10-21T16:20:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4e459640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b480b10f0 con 0x7f8b481599d0 2023-10-21T16:20:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4e459640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480ca3c0 con 0x7f8b480cc210 2023-10-21T16:20:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4dc58640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8b48159db0 0x7f8b4815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39310/0 (socket says 172.21.15.53:39310) 2023-10-21T16:20:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4dc58640 1 -- 172.21.15.53:0/1498599800 learned_addr learned my addr 172.21.15.53:0/1498599800 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.199+0000 7f8b4d457640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b481599d0 0x7f8b480ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4cc56640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 0x7f8b480b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b3ffff640 1 -- 172.21.15.53:0/1498599800 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535292571 0 0) 0x7f8b480cacf0 con 0x7f8b48159db0 2023-10-21T16:20:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b3ffff640 1 -- 172.21.15.53:0/1498599800 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b34003520 con 0x7f8b48159db0 2023-10-21T16:20:21.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4d457640 1 -- 172.21.15.53:0/1498599800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 msgr2=0x7f8b480b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:21.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4d457640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 0x7f8b480b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4d457640 1 -- 172.21.15.53:0/1498599800 >> v1:172.21.15.53:6789/0 conn(0x7f8b48159db0 legacy=0x7f8b4815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4d457640 1 -- 172.21.15.53:0/1498599800 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480ca6a0 con 0x7f8b481599d0 2023-10-21T16:20:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.200+0000 7f8b4cc56640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 0x7f8b480b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.202+0000 7f8b4d457640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481599d0 0x7f8b480ad730 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8b440097a0 tx=0x7f8b4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f8f4a8575effb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.202+0000 7f8b3ffff640 1 -- 172.21.15.53:0/1498599800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44031830 con 0x7f8b481599d0 2023-10-21T16:20:21.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.202+0000 7f8b3ffff640 1 -- 172.21.15.53:0/1498599800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44031990 con 0x7f8b481599d0 2023-10-21T16:20:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.202+0000 7f8b3ffff640 1 -- 172.21.15.53:0/1498599800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44031d00 con 0x7f8b481599d0 2023-10-21T16:20:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 -- 172.21.15.53:0/1498599800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481599d0 msgr2=0x7f8b480ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481599d0 0x7f8b480ad730 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8b440097a0 tx=0x7f8b4402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 -- 172.21.15.53:0/1498599800 shutdown_connections 2023-10-21T16:20:21.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b480cc210 0x7f8b480b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 --2- 172.21.15.53:0/1498599800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b481599d0 0x7f8b480ad730 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 -- 172.21.15.53:0/1498599800 >> 172.21.15.53:0/1498599800 conn(0x7f8b480cb0c0 msgr2=0x7f8b480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 -- 172.21.15.53:0/1498599800 shutdown_connections 2023-10-21T16:20:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.203+0000 7f8b4e459640 1 -- 172.21.15.53:0/1498599800 wait complete. 2023-10-21T16:20:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.205+0000 7f8b4e459640 1 Processor -- start 2023-10-21T16:20:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.205+0000 7f8b4e459640 1 -- start start 2023-10-21T16:20:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b481599d0 0x7f8b480c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b48159db0 0x7f8b481f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480c84f0 con 0x7f8b480cc210 2023-10-21T16:20:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480b0180 con 0x7f8b481599d0 2023-10-21T16:20:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4e459640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480c7870 con 0x7f8b48159db0 2023-10-21T16:20:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4d457640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4d457640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46234/0 (socket says 172.21.15.53:46234) 2023-10-21T16:20:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.206+0000 7f8b4d457640 1 -- 172.21.15.53:0/4074942931 learned_addr learned my addr 172.21.15.53:0/4074942931 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4cc56640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b481599d0 0x7f8b480c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 -- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b48159db0 msgr2=0x7f8b481f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b48159db0 0x7f8b481f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 -- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b481599d0 msgr2=0x7f8b480c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b481599d0 0x7f8b480c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 -- 172.21.15.53:0/4074942931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44009400 con 0x7f8b480cc210 2023-10-21T16:20:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4d457640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8b440098c0 tx=0x7f8b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4403a6f0 con 0x7f8b480cc210 2023-10-21T16:20:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4403a850 con 0x7f8b480cc210 2023-10-21T16:20:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b480c7b50 con 0x7f8b480cc210 2023-10-21T16:20:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.207+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4403ac00 con 0x7f8b480cc210 2023-10-21T16:20:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.208+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b480c8110 con 0x7f8b480cc210 2023-10-21T16:20:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.209+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4804e160 con 0x7f8b480cc210 2023-10-21T16:20:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.210+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8b4404c3b0 con 0x7f8b480cc210 2023-10-21T16:20:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.211+0000 7f8b3dffb640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 0x7f8b2c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.211+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f8b440ab250 con 0x7f8b480cc210 2023-10-21T16:20:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.211+0000 7f8b4cc56640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 0x7f8b2c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.213+0000 7f8b4cc56640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 0x7f8b2c06d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b480ad330 tx=0x7f8b38008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.215+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b440762b0 con 0x7f8b480cc210 2023-10-21T16:20:21.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.411+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b48070c50 con 0x7f8b480cc210 2023-10-21T16:20:21.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.415+0000 7f8b3dffb640 1 -- 172.21.15.53:0/4074942931 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+13130 (secure 0 0 0) 0x7f8b440798d0 con 0x7f8b480cc210 2023-10-21T16:20:21.417 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:21.417 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":129,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:20.423524+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:21.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.418+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 msgr2=0x7f8b2c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.418+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 0x7f8b2c06d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b480ad330 tx=0x7f8b38008040 comp rx=0 tx=0).stop 2023-10-21T16:20:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.418+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 msgr2=0x7f8b480c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8b440098c0 tx=0x7f8b440046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 shutdown_connections 2023-10-21T16:20:21.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f8b2c06afc0 0x7f8b2c06d480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b48159db0 0x7f8b481f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b481599d0 0x7f8b480c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 --2- 172.21.15.53:0/4074942931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b480cc210 0x7f8b480c7290 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 >> 172.21.15.53:0/4074942931 conn(0x7f8b480cb0c0 msgr2=0x7f8b480b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:21.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 shutdown_connections 2023-10-21T16:20:21.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.419+0000 7f8b4e459640 1 -- 172.21.15.53:0/4074942931 wait complete. 2023-10-21T16:20:21.441 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:20:21.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:20:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.663+0000 7f279501a640 1 Processor -- start 2023-10-21T16:20:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.663+0000 7f279501a640 1 -- start start 2023-10-21T16:20:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.664+0000 7f279501a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 0x7f27900d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.664+0000 7f279501a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f279015dcd0 0x7f27901610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.664+0000 7f279501a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27900d0a80 con 0x7f27900d1120 2023-10-21T16:20:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.664+0000 7f279501a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2790161620 con 0x7f279015dcd0 2023-10-21T16:20:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.664+0000 7f279501a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27900b1fe0 con 0x7f27900d1500 2023-10-21T16:20:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f278ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f27900d1120 0x7f27900cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39320/0 (socket says 172.21.15.53:39320) 2023-10-21T16:20:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f278ffff640 1 -- 172.21.15.53:0/2881901041 learned_addr learned my addr 172.21.15.53:0/2881901041 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:21.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f2794819640 1 --2- 172.21.15.53:0/2881901041 >> v2:172.21.15.124:3300/0 conn(0x7f279015dcd0 0x7f27901610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221488949 0 0) 0x7f27900d0a80 con 0x7f27900d1120 2023-10-21T16:20:21.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003520 con 0x7f27900d1120 2023-10-21T16:20:21.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.665+0000 7f2787fff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 0x7f27900d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023468243 0 0) 0x7f2770003520 con 0x7f27900d1120 2023-10-21T16:20:21.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 msgr2=0x7f27900d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:21.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 0x7f27900d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 >> v2:172.21.15.124:3300/0 conn(0x7f279015dcd0 msgr2=0x7f27901610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:21.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 --2- 172.21.15.53:0/2881901041 >> v2:172.21.15.124:3300/0 conn(0x7f279015dcd0 0x7f27901610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27900b4580 con 0x7f27900d1120 2023-10-21T16:20:21.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f27800017f0 con 0x7f27900d1120 2023-10-21T16:20:21.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 >> v1:172.21.15.53:6789/0 conn(0x7f27900d1120 legacy=0x7f27900cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2770003970 0x7f2770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27700047c0 0x7f2770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 0x7f2770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2790161620 con 0x7f2770003970 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27900b1fe0 con 0x7f27700047c0 2023-10-21T16:20:21.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277000b9d0 con 0x7f27900d1500 2023-10-21T16:20:21.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.666+0000 7f278ffff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2770003970 0x7f2770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f2787fff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27700047c0 0x7f2770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f278ffff640 1 -- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 msgr2=0x7f2770005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:21.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f278ffff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900d1500 0x7f2770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f278ffff640 1 -- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27700047c0 msgr2=0x7f2770004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:21.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f278ffff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27700047c0 0x7f2770004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.667+0000 7f278ffff640 1 -- 172.21.15.53:0/2881901041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770003520 con 0x7f2770003970 2023-10-21T16:20:21.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f278ffff640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2770003970 0x7f2770003d90 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f278000c140 tx=0x7f27700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75bada6aa6014ab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780007270 con 0x7f2770003970 2023-10-21T16:20:21.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27800073f0 con 0x7f2770003970 2023-10-21T16:20:21.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f278f7fe640 1 -- 172.21.15.53:0/2881901041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278002c810 con 0x7f2770003970 2023-10-21T16:20:21.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f279501a640 1 -- 172.21.15.53:0/2881901041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2770003970 msgr2=0x7f2770003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f279501a640 1 --2- 172.21.15.53:0/2881901041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2770003970 0x7f2770003d90 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f278000c140 tx=0x7f27700057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.668+0000 7f2794819640 1 -- 172.21.15.53:0/2881901041 reap_dead start 2023-10-21T16:20:21.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.669+0000 7f279501a640 1 -- 172.21.15.53:0/2881901041 shutdown_connections 2023-10-21T16:20:21.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.669+0000 7f279501a640 1 -- 172.21.15.53:0/2881901041 >> 172.21.15.53:0/2881901041 conn(0x7f27900d2960 msgr2=0x7f27900d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:21.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.669+0000 7f279501a640 1 -- 172.21.15.53:0/2881901041 shutdown_connections 2023-10-21T16:20:21.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.669+0000 7f279501a640 1 -- 172.21.15.53:0/2881901041 wait complete. 2023-10-21T16:20:21.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.670+0000 7f279501a640 1 Processor -- start 2023-10-21T16:20:21.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.671+0000 7f279501a640 1 -- start start 2023-10-21T16:20:21.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2770003970 0x7f27900ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900cc150 0x7f279015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279015fcd0 con 0x7f279015f850 2023-10-21T16:20:21.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27900c99a0 con 0x7f2770003970 2023-10-21T16:20:21.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f279501a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27900c9c80 con 0x7f27900cc150 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46244/0 (socket says 172.21.15.53:46244) 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 -- 172.21.15.53:0/2759359361 learned_addr learned my addr 172.21.15.53:0/2759359361 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 -- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900cc150 msgr2=0x7f279015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900cc150 0x7f279015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 -- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2770003970 msgr2=0x7f27900ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f278ffff640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2770003970 0x7f27900ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2770003970 0x7f27900ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.672+0000 7f2794819640 1 -- 172.21.15.53:0/2759359361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27900c9f60 con 0x7f279015f850 2023-10-21T16:20:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f2794819640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f27900d0430 tx=0x7f278800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788010040 con 0x7f279015f850 2023-10-21T16:20:21.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278800bce0 con 0x7f279015f850 2023-10-21T16:20:21.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788014640 con 0x7f279015f850 2023-10-21T16:20:21.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27900ca240 con 0x7f279015f850 2023-10-21T16:20:21.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.673+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27900b5600 con 0x7f279015f850 2023-10-21T16:20:21.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.674+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27900ca430 con 0x7f279015f850 2023-10-21T16:20:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.675+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f27880147a0 con 0x7f279015f850 2023-10-21T16:20:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.676+0000 7f278d7fa640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 0x7f277406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.677+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f2788089b10 con 0x7f279015f850 2023-10-21T16:20:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.677+0000 7f278ffff640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 0x7f277406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:21.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.678+0000 7f278ffff640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 0x7f277406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2780034000 tx=0x7f2780007000 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:21.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.681+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2788054ae0 con 0x7f279015f850 2023-10-21T16:20:21.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.879+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2790074150 con 0x7f279015f850 2023-10-21T16:20:21.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.880+0000 7f278d7fa640 1 -- 172.21.15.53:0/2759359361 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2788058100 con 0x7f279015f850 2023-10-21T16:20:21.881 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:20:21.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.882+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 msgr2=0x7f277406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.882+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 0x7f277406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2780034000 tx=0x7f2780007000 comp rx=0 tx=0).stop 2023-10-21T16:20:21.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.882+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 msgr2=0x7f27900b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:21.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.882+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f27900d0430 tx=0x7f278800b480 comp rx=0 tx=0).stop 2023-10-21T16:20:21.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 shutdown_connections 2023-10-21T16:20:21.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f277406af90 0x7f277406d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f279015f850 0x7f27900b5d30 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27900cc150 0x7f279015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 --2- 172.21.15.53:0/2759359361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2770003970 0x7f27900ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:21.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 >> 172.21.15.53:0/2759359361 conn(0x7f27900d2960 msgr2=0x7f27900b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:21.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 shutdown_connections 2023-10-21T16:20:21.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:21.883+0000 7f279501a640 1 -- 172.21.15.53:0/2759359361 wait complete. 2023-10-21T16:20:21.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:20:22.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:20:22.063 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:20:22.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.123+0000 7faaca933640 1 Processor -- start 2023-10-21T16:20:22.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.123+0000 7faaca933640 1 -- start start 2023-10-21T16:20:22.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faaca933640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faac4160940 0x7faac40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faaca933640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40c48f0 con 0x7faac415cd90 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faaca933640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40c3ce0 con 0x7faac406c890 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faaca933640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40c4d80 con 0x7faac4160940 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faabbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faac4160940 0x7faac40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faabbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faac4160940 0x7faac40709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46248/0 (socket says 172.21.15.53:46248) 2023-10-21T16:20:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faac3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faac415cd90 0x7faac40702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39336/0 (socket says 172.21.15.53:39336) 2023-10-21T16:20:22.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.125+0000 7faabbfff640 1 -- 172.21.15.53:0/3925175103 learned_addr learned my addr 172.21.15.53:0/3925175103 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.126+0000 7faabbfff640 1 -- 172.21.15.53:0/3925175103 >> v1:172.21.15.124:6789/0 conn(0x7faac406c890 legacy=0x7faac406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.126+0000 7faabbfff640 1 -- 172.21.15.53:0/3925175103 >> v1:172.21.15.53:6789/0 conn(0x7faac415cd90 legacy=0x7faac40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.126+0000 7faac37fe640 1 -- 172.21.15.53:0/3925175103 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096814616 0 0) 0x7faac40c48f0 con 0x7faac415cd90 2023-10-21T16:20:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.126+0000 7faabbfff640 1 -- 172.21.15.53:0/3925175103 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40c3fc0 con 0x7faac4160940 2023-10-21T16:20:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.127+0000 7faabbfff640 1 --2- 172.21.15.53:0/3925175103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac40709e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7faaac009870 tx=0x7faaac02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911649edd848892a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.127+0000 7faac37fe640 1 -- 172.21.15.53:0/3925175103 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac02f840 con 0x7faac4160940 2023-10-21T16:20:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.127+0000 7faac37fe640 1 -- 172.21.15.53:0/3925175103 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac02f9a0 con 0x7faac4160940 2023-10-21T16:20:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.127+0000 7faac37fe640 1 -- 172.21.15.53:0/3925175103 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac02fd30 con 0x7faac4160940 2023-10-21T16:20:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 -- 172.21.15.53:0/3925175103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 msgr2=0x7faac40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 --2- 172.21.15.53:0/3925175103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac40709e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7faaac009870 tx=0x7faaac02ba20 comp rx=0 tx=0).stop 2023-10-21T16:20:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 -- 172.21.15.53:0/3925175103 shutdown_connections 2023-10-21T16:20:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 --2- 172.21.15.53:0/3925175103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac40709e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 -- 172.21.15.53:0/3925175103 >> 172.21.15.53:0/3925175103 conn(0x7faac406a380 msgr2=0x7faac40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 -- 172.21.15.53:0/3925175103 shutdown_connections 2023-10-21T16:20:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.128+0000 7faaca933640 1 -- 172.21.15.53:0/3925175103 wait complete. 2023-10-21T16:20:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.130+0000 7faaca933640 1 Processor -- start 2023-10-21T16:20:22.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.130+0000 7faaca933640 1 -- start start 2023-10-21T16:20:22.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faac406c890 0x7faac4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faac415cd90 0x7faac40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40cb970 con 0x7faac4160940 2023-10-21T16:20:22.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40cada0 con 0x7faac415cd90 2023-10-21T16:20:22.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.131+0000 7faaca933640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40cb040 con 0x7faac406c890 2023-10-21T16:20:22.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46264/0 (socket says 172.21.15.53:46264) 2023-10-21T16:20:22.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 -- 172.21.15.53:0/86238436 learned_addr learned my addr 172.21.15.53:0/86238436 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:22.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faabbfff640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faac415cd90 0x7faac40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac3fff640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faac406c890 0x7faac4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 -- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faac406c890 msgr2=0x7faac4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faac406c890 0x7faac4154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 -- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faac415cd90 msgr2=0x7faac40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faac415cd90 0x7faac40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.132+0000 7faac8ed0640 1 -- 172.21.15.53:0/86238436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac0094d0 con 0x7faac4160940 2023-10-21T16:20:22.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faac8ed0640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c7a0 tx=0x7faabc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc003b70 con 0x7faac4160940 2023-10-21T16:20:22.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc003cf0 con 0x7faac4160940 2023-10-21T16:20:22.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc003ea0 con 0x7faac4160940 2023-10-21T16:20:22.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac40cb320 con 0x7faac4160940 2023-10-21T16:20:22.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.133+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac4154680 con 0x7faac4160940 2023-10-21T16:20:22.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.134+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac40cbae0 con 0x7faac4160940 2023-10-21T16:20:22.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.135+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faabc00aa90 con 0x7faac4160940 2023-10-21T16:20:22.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.136+0000 7faac17fa640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 0x7faaa406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.136+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5221+0+0 (secure 0 0 0) 0x7faabc089410 con 0x7faac4160940 2023-10-21T16:20:22.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.137+0000 7faac3fff640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 0x7faaa406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.138+0000 7faac3fff640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 0x7faaa406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.141+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faabc054590 con 0x7faac4160940 2023-10-21T16:20:22.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.337+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faac4067490 con 0x7faac4160940 2023-10-21T16:20:22.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.338+0000 7faac17fa640 1 -- 172.21.15.53:0/86238436 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v129) v1 ==== 84+0+65 (secure 0 0 0) 0x7faabc057bb0 con 0x7faac4160940 2023-10-21T16:20:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:22.340 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:20:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:20:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:20:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:22.341 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:20:22.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 msgr2=0x7faaa406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 0x7faaa406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4009040 comp rx=0 tx=0).stop 2023-10-21T16:20:22.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 msgr2=0x7faac4070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c7a0 tx=0x7faabc00cc60 comp rx=0 tx=0).stop 2023-10-21T16:20:22.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 shutdown_connections 2023-10-21T16:20:22.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faaa406af90 0x7faaa406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faac4160940 0x7faac4070be0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faac415cd90 0x7faac40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 --2- 172.21.15.53:0/86238436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faac406c890 0x7faac4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 >> 172.21.15.53:0/86238436 conn(0x7faac406a380 msgr2=0x7faac40cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:22.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 shutdown_connections 2023-10-21T16:20:22.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.341+0000 7faaca933640 1 -- 172.21.15.53:0/86238436 wait complete. 2023-10-21T16:20:22.363 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:20:22.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:20:22.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.598+0000 7fcda0933640 1 Processor -- start 2023-10-21T16:20:22.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.598+0000 7fcda0933640 1 -- start start 2023-10-21T16:20:22.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.599+0000 7fcda0933640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd9c15aad0 0x7fcd9c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcda0933640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c15bf30 con 0x7fcd9c0c82b0 2023-10-21T16:20:22.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcda0933640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c15b320 con 0x7fcd9c06d410 2023-10-21T16:20:22.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcda0933640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c15b600 con 0x7fcd9c15aad0 2023-10-21T16:20:22.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd9c15aad0 0x7fcd9c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd9c15aad0 0x7fcd9c0b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46266/0 (socket says 172.21.15.53:46266) 2023-10-21T16:20:22.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 -- 172.21.15.53:0/1106184856 learned_addr learned my addr 172.21.15.53:0/1106184856 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:22.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd99d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcd9c0c82b0 0x7fcd9c06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39346/0 (socket says 172.21.15.53:39346) 2023-10-21T16:20:22.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 -- 172.21.15.53:0/1106184856 >> v1:172.21.15.124:6789/0 conn(0x7fcd9c06d410 legacy=0x7fcd9c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:22.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 -- 172.21.15.53:0/1106184856 >> v1:172.21.15.53:6789/0 conn(0x7fcd9c0c82b0 legacy=0x7fcd9c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.600+0000 7fcd9a59c640 1 -- 172.21.15.53:0/1106184856 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c15b8e0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.602+0000 7fcd9a59c640 1 --2- 172.21.15.53:0/1106184856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c0b1750 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009e50 tx=0x7fcd8c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631fa2c2576fffff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.602+0000 7fcd9959a640 1 -- 172.21.15.53:0/1106184856 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c02cb10 con 0x7fcd9c15aad0 2023-10-21T16:20:22.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.602+0000 7fcd9959a640 1 -- 172.21.15.53:0/1106184856 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c02cc70 con 0x7fcd9c15aad0 2023-10-21T16:20:22.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.602+0000 7fcd9959a640 1 -- 172.21.15.53:0/1106184856 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c0343f0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 -- 172.21.15.53:0/1106184856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 msgr2=0x7fcd9c0b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 --2- 172.21.15.53:0/1106184856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c0b1750 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009e50 tx=0x7fcd8c02bf70 comp rx=0 tx=0).stop 2023-10-21T16:20:22.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 -- 172.21.15.53:0/1106184856 shutdown_connections 2023-10-21T16:20:22.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 --2- 172.21.15.53:0/1106184856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c0b1750 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 -- 172.21.15.53:0/1106184856 >> 172.21.15.53:0/1106184856 conn(0x7fcd9c069d50 msgr2=0x7fcd9c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:22.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 -- 172.21.15.53:0/1106184856 shutdown_connections 2023-10-21T16:20:22.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.603+0000 7fcda0933640 1 -- 172.21.15.53:0/1106184856 wait complete. 2023-10-21T16:20:22.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.605+0000 7fcda0933640 1 Processor -- start 2023-10-21T16:20:22.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.605+0000 7fcda0933640 1 -- start start 2023-10-21T16:20:22.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd9c06d410 0x7fcd9c1efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd9c0c82b0 0x7fcd9c1f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1f3520 con 0x7fcd9c15aad0 2023-10-21T16:20:22.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c153d70 con 0x7fcd9c06d410 2023-10-21T16:20:22.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcda0933640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c154050 con 0x7fcd9c0c82b0 2023-10-21T16:20:22.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcd9ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcd9ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46278/0 (socket says 172.21.15.53:46278) 2023-10-21T16:20:22.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.606+0000 7fcd9ad9d640 1 -- 172.21.15.53:0/3703761906 learned_addr learned my addr 172.21.15.53:0/3703761906 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:22.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9a59c640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd9c06d410 0x7fcd9c1efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd99d9b640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd9c0c82b0 0x7fcd9c1f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 -- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd9c0c82b0 msgr2=0x7fcd9c1f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd9c0c82b0 0x7fcd9c1f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 -- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd9c06d410 msgr2=0x7fcd9c1efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:22.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd9c06d410 0x7fcd9c1efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:22.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 -- 172.21.15.53:0/3703761906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009b30 con 0x7fcd9c15aad0 2023-10-21T16:20:22.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd9ad9d640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c3e0 tx=0x7fcd9400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.607+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94011790 con 0x7fcd9c15aad0 2023-10-21T16:20:22.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.608+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd940118f0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.608+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94011ce0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.608+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c154330 con 0x7fcd9c15aad0 2023-10-21T16:20:22.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.608+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1548f0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.609+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c04e160 con 0x7fcd9c15aad0 2023-10-21T16:20:22.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.610+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcd9401c460 con 0x7fcd9c15aad0 2023-10-21T16:20:22.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.611+0000 7fcd92ffd640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 0x7fcd7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:22.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.611+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fcd94089470 con 0x7fcd9c15aad0 2023-10-21T16:20:22.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.612+0000 7fcd9a59c640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 0x7fcd7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:22.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.613+0000 7fcd9a59c640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 0x7fcd7c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c035000 tx=0x7fcd8c030000 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:22.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.616+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd940545e0 con 0x7fcd9c15aad0 2023-10-21T16:20:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:22.810+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcd9c15c560 con 0x7fcd9c15aad0 2023-10-21T16:20:23.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.429+0000 7fcd92ffd640 1 -- 172.21.15.53:0/3703761906 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v130) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcd94057c00 con 0x7fcd9c15aad0 2023-10-21T16:20:23.432 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:20:23.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.433+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 msgr2=0x7fcd7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.433+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 0x7fcd7c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c035000 tx=0x7fcd8c030000 comp rx=0 tx=0).stop 2023-10-21T16:20:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.433+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 msgr2=0x7fcd9c151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.433+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400c3e0 tx=0x7fcd9400c8a0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 shutdown_connections 2023-10-21T16:20:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcd7c06af90 0x7fcd7c06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd9c15aad0 0x7fcd9c151950 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd9c0c82b0 0x7fcd9c1f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 --2- 172.21.15.53:0/3703761906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd9c06d410 0x7fcd9c1efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 >> 172.21.15.53:0/3703761906 conn(0x7fcd9c069d50 msgr2=0x7fcd9c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:23.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 shutdown_connections 2023-10-21T16:20:23.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.434+0000 7fcda0933640 1 -- 172.21.15.53:0/3703761906 wait complete. 2023-10-21T16:20:23.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:20:23.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.675+0000 7fa4d5593640 1 Processor -- start 2023-10-21T16:20:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.675+0000 7fa4d5593640 1 -- start start 2023-10-21T16:20:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4d5593640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4d00b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4d5593640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d006c880 con 0x7fa4d0160ea0 2023-10-21T16:20:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4d5593640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d006cb60 con 0x7fa4d00c1cb0 2023-10-21T16:20:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4d5593640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d006ce40 con 0x7fa4d0161280 2023-10-21T16:20:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4cffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa4d0160ea0 0x7fa4d00c15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39354/0 (socket says 172.21.15.53:39354) 2023-10-21T16:20:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.677+0000 7fa4cffff640 1 -- 172.21.15.53:0/4229995376 learned_addr learned my addr 172.21.15.53:0/4229995376 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:23.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4d00b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412755708 0 0) 0x7fa4d006c880 con 0x7fa4d0160ea0 2023-10-21T16:20:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b8003520 con 0x7fa4d0160ea0 2023-10-21T16:20:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642783532 0 0) 0x7fa4b8003520 con 0x7fa4d0160ea0 2023-10-21T16:20:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 msgr2=0x7fa4d00b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4d00b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 >> v1:172.21.15.124:6789/0 conn(0x7fa4d00c1cb0 legacy=0x7fa4d00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d006d120 con 0x7fa4d0160ea0 2023-10-21T16:20:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa4c0001670 con 0x7fa4d0160ea0 2023-10-21T16:20:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 >> v1:172.21.15.53:6789/0 conn(0x7fa4d0160ea0 legacy=0x7fa4d00c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b80039a0 0x7fa4b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4b8004d00 0x7fa4b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d006ce40 con 0x7fa4b8004d00 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800baf0 con 0x7fa4b80039a0 2023-10-21T16:20:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.678+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800bcb0 con 0x7fa4d0161280 2023-10-21T16:20:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4b8004d00 0x7fa4b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4cf7fe640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4cffff640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b80039a0 0x7fa4b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 -- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 msgr2=0x7fa4b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4d0161280 0x7fa4b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 -- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b80039a0 msgr2=0x7fa4b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b80039a0 0x7fa4b8003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.679+0000 7fa4d4d92640 1 -- 172.21.15.53:0/4229995376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8003520 con 0x7fa4b8004d00 2023-10-21T16:20:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.680+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4b8004d00 0x7fa4b8005180 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0161710 tx=0x7fa4b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10482a8ba2a2012d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.680+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c802cb90 con 0x7fa4b8004d00 2023-10-21T16:20:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c800f040 con 0x7fa4b8004d00 2023-10-21T16:20:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4ceffd640 1 -- 172.21.15.53:0/4229995376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8034440 con 0x7fa4b8004d00 2023-10-21T16:20:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 -- 172.21.15.53:0/4229995376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4b8004d00 msgr2=0x7fa4b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4229995376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4b8004d00 0x7fa4b8005180 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0161710 tx=0x7fa4b80057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d4d92640 1 -- 172.21.15.53:0/4229995376 reap_dead start 2023-10-21T16:20:23.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 -- 172.21.15.53:0/4229995376 shutdown_connections 2023-10-21T16:20:23.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 -- 172.21.15.53:0/4229995376 >> 172.21.15.53:0/4229995376 conn(0x7fa4d00ce9f0 msgr2=0x7fa4d00cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:23.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 -- 172.21.15.53:0/4229995376 shutdown_connections 2023-10-21T16:20:23.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.681+0000 7fa4d5593640 1 -- 172.21.15.53:0/4229995376 wait complete. 2023-10-21T16:20:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.683+0000 7fa4d5593640 1 Processor -- start 2023-10-21T16:20:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.683+0000 7fa4d5593640 1 -- start start 2023-10-21T16:20:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.684+0000 7fa4d5593640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 0x7fa4d015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.684+0000 7fa4d5593640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.684+0000 7fa4d5593640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 0x7fa4d0071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.684+0000 7fa4d5593640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00afde0 con 0x7fa4d015ea50 2023-10-21T16:20:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4d5593640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d015e7e0 con 0x7fa4d01f6310 2023-10-21T16:20:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4d5593640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00af160 con 0x7fa4b8004d00 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46288/0 (socket says 172.21.15.53:46288) 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 -- 172.21.15.53:0/4173762114 learned_addr learned my addr 172.21.15.53:0/4173762114 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 0x7fa4d0071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cffff640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 0x7fa4d015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 -- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 msgr2=0x7fa4d015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 0x7fa4d015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 -- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 msgr2=0x7fa4d0071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 0x7fa4d0071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cf7fe640 1 -- 172.21.15.53:0/4173762114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00af440 con 0x7fa4d015ea50 2023-10-21T16:20:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4d4d92640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 0x7fa4d0071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.685+0000 7fa4cffff640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 0x7fa4d015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4cf7fe640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e580 tx=0x7fa4c400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40052e0 con 0x7fa4d015ea50 2023-10-21T16:20:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4005440 con 0x7fa4d015ea50 2023-10-21T16:20:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4003810 con 0x7fa4d015ea50 2023-10-21T16:20:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d00af720 con 0x7fa4d015ea50 2023-10-21T16:20:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.686+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d0164150 con 0x7fa4d015ea50 2023-10-21T16:20:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.688+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d00aff20 con 0x7fa4d015ea50 2023-10-21T16:20:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.689+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa4c4003970 con 0x7fa4d015ea50 2023-10-21T16:20:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.690+0000 7fa4ccff9640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 0x7fa4bc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.690+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5591+0+0 (secure 0 0 0) 0x7fa4c4089530 con 0x7fa4d015ea50 2023-10-21T16:20:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.691+0000 7fa4cffff640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 0x7fa4bc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.692+0000 7fa4cffff640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 0x7fa4bc06d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0001200 tx=0x7fa4c0009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.695+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4d00aff20 con 0x7fa4d015ea50 2023-10-21T16:20:23.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:23.888+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa4d00aff20 con 0x7fa4d015ea50 2023-10-21T16:20:24.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.436+0000 7fa4ccff9640 1 -- 172.21.15.53:0/4173762114 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v131) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa4d00aff20 con 0x7fa4d015ea50 2023-10-21T16:20:24.438 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:20:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 msgr2=0x7fa4bc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 0x7fa4bc06d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0001200 tx=0x7fa4c0009040 comp rx=0 tx=0).stop 2023-10-21T16:20:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 msgr2=0x7fa4d0163c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e580 tx=0x7fa4c400ea40 comp rx=0 tx=0).stop 2023-10-21T16:20:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 shutdown_connections 2023-10-21T16:20:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa4bc06afc0 0x7fa4bc06d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4d01f6310 0x7fa4d0071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4d015ea50 0x7fa4d0163c10 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 --2- 172.21.15.53:0/4173762114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b8004d00 0x7fa4d015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 >> 172.21.15.53:0/4173762114 conn(0x7fa4d00ce9f0 msgr2=0x7fa4d0070820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.440+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 shutdown_connections 2023-10-21T16:20:24.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.441+0000 7fa4d5593640 1 -- 172.21.15.53:0/4173762114 wait complete. 2023-10-21T16:20:24.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:20:24.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.668+0000 7fcf8b5b7640 1 Processor -- start 2023-10-21T16:20:24.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.668+0000 7fcf8b5b7640 1 -- start start 2023-10-21T16:20:24.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf8b5b7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcf840c9cf0 0x7fcf8406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:24.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf8b5b7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8415b310 con 0x7fcf8406c3a0 2023-10-21T16:20:24.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf8b5b7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8406d290 con 0x7fcf8415aa00 2023-10-21T16:20:24.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf8b5b7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8406e0a0 con 0x7fcf840c9cf0 2023-10-21T16:20:24.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf88b52640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcf840c9cf0 0x7fcf8406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:24.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf88b52640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcf840c9cf0 0x7fcf8406db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37720/0 (socket says 172.21.15.53:37720) 2023-10-21T16:20:24.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf88b52640 1 -- 172.21.15.53:0/1669238996 learned_addr learned my addr 172.21.15.53:0/1669238996 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:24.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.670+0000 7fcf89353640 1 --1- 172.21.15.53:0/1669238996 >> v1:172.21.15.53:6789/0 conn(0x7fcf8406c3a0 0x7fcf8406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46058/0 (socket says 172.21.15.53:46058) 2023-10-21T16:20:24.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.671+0000 7fcf88b52640 1 -- 172.21.15.53:0/1669238996 >> v1:172.21.15.124:6789/0 conn(0x7fcf8415aa00 legacy=0x7fcf840b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:24.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.671+0000 7fcf88b52640 1 -- 172.21.15.53:0/1669238996 >> v1:172.21.15.53:6789/0 conn(0x7fcf8406c3a0 legacy=0x7fcf8406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.671+0000 7fcf88b52640 1 -- 172.21.15.53:0/1669238996 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf840c9860 con 0x7fcf840c9cf0 2023-10-21T16:20:24.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.671+0000 7fcf7bfff640 1 -- 172.21.15.53:0/1669238996 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513068611 0 0) 0x7fcf80001110 con 0x7fcf8406c3a0 2023-10-21T16:20:24.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.672+0000 7fcf88b52640 1 --2- 172.21.15.53:0/1669238996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf8406db60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bb10 tx=0x7fcf7402e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e8154fee00cf78 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:24.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.672+0000 7fcf7bfff640 1 -- 172.21.15.53:0/1669238996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74036440 con 0x7fcf840c9cf0 2023-10-21T16:20:24.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.672+0000 7fcf7bfff640 1 -- 172.21.15.53:0/1669238996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740365a0 con 0x7fcf840c9cf0 2023-10-21T16:20:24.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.672+0000 7fcf7bfff640 1 -- 172.21.15.53:0/1669238996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74036930 con 0x7fcf840c9cf0 2023-10-21T16:20:24.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/1669238996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 msgr2=0x7fcf8406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/1669238996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf8406db60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bb10 tx=0x7fcf7402e160 comp rx=0 tx=0).stop 2023-10-21T16:20:24.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/1669238996 shutdown_connections 2023-10-21T16:20:24.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/1669238996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf8406db60 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/1669238996 >> 172.21.15.53:0/1669238996 conn(0x7fcf8406a5d0 msgr2=0x7fcf840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:24.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/1669238996 shutdown_connections 2023-10-21T16:20:24.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.673+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/1669238996 wait complete. 2023-10-21T16:20:24.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.675+0000 7fcf8b5b7640 1 Processor -- start 2023-10-21T16:20:24.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.675+0000 7fcf8b5b7640 1 -- start start 2023-10-21T16:20:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 0x7fcf840c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf8415aa00 0x7fcf840c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840cd9f0 con 0x7fcf840c9cf0 2023-10-21T16:20:24.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c28c0 con 0x7fcf8406c3a0 2023-10-21T16:20:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf8b5b7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c2ba0 con 0x7fcf8415aa00 2023-10-21T16:20:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf88b52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf88b52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37730/0 (socket says 172.21.15.53:37730) 2023-10-21T16:20:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.676+0000 7fcf88b52640 1 -- 172.21.15.53:0/2473709216 learned_addr learned my addr 172.21.15.53:0/2473709216 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:24.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf89353640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 0x7fcf840c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:24.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 -- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf8415aa00 msgr2=0x7fcf840c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf8415aa00 0x7fcf840c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 -- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 msgr2=0x7fcf840c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:24.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 0x7fcf840c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:24.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 -- 172.21.15.53:0/2473709216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7400b770 con 0x7fcf840c9cf0 2023-10-21T16:20:24.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf89353640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 0x7fcf840c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:24.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf88b52640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bb10 tx=0x7fcf74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:24.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74036440 con 0x7fcf840c9cf0 2023-10-21T16:20:24.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf74004250 con 0x7fcf840c9cf0 2023-10-21T16:20:24.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74034980 con 0x7fcf840c9cf0 2023-10-21T16:20:24.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf840c2e80 con 0x7fcf840c9cf0 2023-10-21T16:20:24.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.677+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf840c3440 con 0x7fcf840c9cf0 2023-10-21T16:20:24.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.678+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8404e160 con 0x7fcf840c9cf0 2023-10-21T16:20:24.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.681+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf7403d040 con 0x7fcf840c9cf0 2023-10-21T16:20:24.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.682+0000 7fcf79ffb640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 0x7fcf6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:24.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.682+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5973+0+0 (secure 0 0 0) 0x7fcf740ad140 con 0x7fcf840c9cf0 2023-10-21T16:20:24.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.682+0000 7fcf89353640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 0x7fcf6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:24.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.684+0000 7fcf89353640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 0x7fcf6806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000f60 tx=0x7fcf80009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:24.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.685+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf74077f20 con 0x7fcf840c9cf0 2023-10-21T16:20:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:24.917+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcf8404e160 con 0x7fcf840c9cf0 2023-10-21T16:20:25.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.488+0000 7fcf79ffb640 1 -- 172.21.15.53:0/2473709216 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcf7407b540 con 0x7fcf840c9cf0 2023-10-21T16:20:25.494 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 15 and data pool 16 2023-10-21T16:20:25.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.501+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 msgr2=0x7fcf6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.501+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 0x7fcf6806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000f60 tx=0x7fcf80009040 comp rx=0 tx=0).stop 2023-10-21T16:20:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.501+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 msgr2=0x7fcf840cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.501+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400bb10 tx=0x7fcf74004730 comp rx=0 tx=0).stop 2023-10-21T16:20:25.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 shutdown_connections 2023-10-21T16:20:25.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fcf6806af90 0x7fcf6806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf8415aa00 0x7fcf840c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf840c9cf0 0x7fcf840cd4b0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 --2- 172.21.15.53:0/2473709216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf8406c3a0 0x7fcf840c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.502+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 >> 172.21.15.53:0/2473709216 conn(0x7fcf8406a5d0 msgr2=0x7fcf840b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:25.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.503+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 shutdown_connections 2023-10-21T16:20:25.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.504+0000 7fcf8b5b7640 1 -- 172.21.15.53:0/2473709216 wait complete. 2023-10-21T16:20:25.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:25.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.721+0000 7fa29dc7c640 1 Processor -- start 2023-10-21T16:20:25.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.721+0000 7fa29dc7c640 1 -- start start 2023-10-21T16:20:25.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa29dc7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa29815cd90 0x7fa2980702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa29dc7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29806c890 0x7fa29806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa29dc7c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2980c48f0 con 0x7fa298160940 2023-10-21T16:20:25.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa29dc7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2980c4d80 con 0x7fa29815cd90 2023-10-21T16:20:25.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa29dc7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2980c3ce0 con 0x7fa29806c890 2023-10-21T16:20:25.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa2977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa29815cd90 0x7fa2980702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa2977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa29815cd90 0x7fa2980702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37732/0 (socket says 172.21.15.53:37732) 2023-10-21T16:20:25.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.723+0000 7fa2977fe640 1 -- 172.21.15.53:0/3909477980 learned_addr learned my addr 172.21.15.53:0/3909477980 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:25.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.724+0000 7fa297fff640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29806c890 0x7fa29806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.724+0000 7fa2977fe640 1 -- 172.21.15.53:0/3909477980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29806c890 msgr2=0x7fa29806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:25.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.724+0000 7fa2977fe640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29806c890 0x7fa29806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.724+0000 7fa2977fe640 1 -- 172.21.15.53:0/3909477980 >> v1:172.21.15.124:6789/0 conn(0x7fa298160940 legacy=0x7fa298070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:25.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.724+0000 7fa2977fe640 1 -- 172.21.15.53:0/3909477980 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2980c3fc0 con 0x7fa29815cd90 2023-10-21T16:20:25.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.725+0000 7fa2977fe640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29815cd90 0x7fa2980702d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa288009870 tx=0x7fa28802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0992e978f225457 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:25.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.725+0000 7fa296ffd640 1 -- 172.21.15.53:0/3909477980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288031640 con 0x7fa29815cd90 2023-10-21T16:20:25.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.725+0000 7fa296ffd640 1 -- 172.21.15.53:0/3909477980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2880317a0 con 0x7fa29815cd90 2023-10-21T16:20:25.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.725+0000 7fa296ffd640 1 -- 172.21.15.53:0/3909477980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288031b30 con 0x7fa29815cd90 2023-10-21T16:20:25.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 -- 172.21.15.53:0/3909477980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29815cd90 msgr2=0x7fa2980702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29815cd90 0x7fa2980702d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa288009870 tx=0x7fa28802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 -- 172.21.15.53:0/3909477980 shutdown_connections 2023-10-21T16:20:25.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29806c890 0x7fa29806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/3909477980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa29815cd90 0x7fa2980702d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 -- 172.21.15.53:0/3909477980 >> 172.21.15.53:0/3909477980 conn(0x7fa29806a380 msgr2=0x7fa2980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:25.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 -- 172.21.15.53:0/3909477980 shutdown_connections 2023-10-21T16:20:25.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.726+0000 7fa29dc7c640 1 -- 172.21.15.53:0/3909477980 wait complete. 2023-10-21T16:20:25.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.728+0000 7fa29dc7c640 1 Processor -- start 2023-10-21T16:20:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.728+0000 7fa29dc7c640 1 -- start start 2023-10-21T16:20:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa29806c890 0x7fa29814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29815cd90 0x7fa29815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29814ef00 con 0x7fa298160940 2023-10-21T16:20:25.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2980b4370 con 0x7fa29806c890 2023-10-21T16:20:25.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa29dc7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2981f38c0 con 0x7fa29815cd90 2023-10-21T16:20:25.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa297fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa297fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37738/0 (socket says 172.21.15.53:37738) 2023-10-21T16:20:25.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.729+0000 7fa297fff640 1 -- 172.21.15.53:0/1438527099 learned_addr learned my addr 172.21.15.53:0/1438527099 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:25.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa2977fe640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa29806c890 0x7fa29814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa28ffff640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29815cd90 0x7fa29815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 -- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29815cd90 msgr2=0x7fa29815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29815cd90 0x7fa29815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 -- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa29806c890 msgr2=0x7fa29814e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa29806c890 0x7fa29814e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 -- 172.21.15.53:0/1438527099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2880094d0 con 0x7fa298160940 2023-10-21T16:20:25.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa297fff640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa290007e40 tx=0x7fa29000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:25.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.730+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290014620 con 0x7fa298160940 2023-10-21T16:20:25.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.731+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29000ee90 con 0x7fa298160940 2023-10-21T16:20:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.731+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900149e0 con 0x7fa298160940 2023-10-21T16:20:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.731+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2981f2cb0 con 0x7fa298160940 2023-10-21T16:20:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.731+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2981f3270 con 0x7fa298160940 2023-10-21T16:20:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.732+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa298067490 con 0x7fa298160940 2023-10-21T16:20:25.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.733+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa290014b40 con 0x7fa298160940 2023-10-21T16:20:25.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.734+0000 7fa294ff9640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 0x7fa27806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:25.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.734+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa29008a350 con 0x7fa298160940 2023-10-21T16:20:25.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.735+0000 7fa2977fe640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 0x7fa27806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:25.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.736+0000 7fa2977fe640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 0x7fa27806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa288036000 tx=0x7fa28803c040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:25.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.738+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2900550e0 con 0x7fa298160940 2023-10-21T16:20:25.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.931+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2980b0ef0 con 0x7fa298160940 2023-10-21T16:20:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.934+0000 7fa294ff9640 1 -- 172.21.15.53:0/1438527099 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fa290058700 con 0x7fa298160940 2023-10-21T16:20:25.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:25.936 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":132,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:24.919265+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:25.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 msgr2=0x7fa27806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 0x7fa27806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa288036000 tx=0x7fa28803c040 comp rx=0 tx=0).stop 2023-10-21T16:20:25.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 msgr2=0x7fa2981e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:25.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa290007e40 tx=0x7fa29000e4b0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 shutdown_connections 2023-10-21T16:20:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa27806af90 0x7fa27806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa298160940 0x7fa2981e6b60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa29815cd90 0x7fa29815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 --2- 172.21.15.53:0/1438527099 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa29806c890 0x7fa29814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.937+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 >> 172.21.15.53:0/1438527099 conn(0x7fa29806a380 msgr2=0x7fa2980cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.938+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 shutdown_connections 2023-10-21T16:20:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:25.938+0000 7fa29dc7c640 1 -- 172.21.15.53:0/1438527099 wait complete. 2023-10-21T16:20:25.958 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.176+0000 7f67367ac640 1 Processor -- start 2023-10-21T16:20:26.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.176+0000 7f67367ac640 1 -- start start 2023-10-21T16:20:26.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.178+0000 7f67367ac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67301599d0 0x7f67300ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.178+0000 7f67367ac640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67300cc210 0x7f67300b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.178+0000 7f67367ac640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67300cacf0 con 0x7f6730159db0 2023-10-21T16:20:26.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.178+0000 7f67367ac640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67300b10f0 con 0x7f67301599d0 2023-10-21T16:20:26.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.178+0000 7f67367ac640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67300ca3c0 con 0x7f67300cc210 2023-10-21T16:20:26.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67301599d0 0x7f67300ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67301599d0 0x7f67300ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37746/0 (socket says 172.21.15.53:37746) 2023-10-21T16:20:26.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 -- 172.21.15.53:0/2892076858 learned_addr learned my addr 172.21.15.53:0/2892076858 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:26.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f6734fa9640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67300cc210 0x7f67300b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 -- 172.21.15.53:0/2892076858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67300cc210 msgr2=0x7f67300b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:26.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67300cc210 0x7f67300b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 -- 172.21.15.53:0/2892076858 >> v1:172.21.15.124:6789/0 conn(0x7f6730159db0 legacy=0x7f673015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:26.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.179+0000 7f67357aa640 1 -- 172.21.15.53:0/2892076858 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67300ca6a0 con 0x7f67301599d0 2023-10-21T16:20:26.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.180+0000 7f67357aa640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67301599d0 0x7f67300ad730 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f672c00bb10 tx=0x7f672c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400cbd8c88aef869 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.181+0000 7f6727fff640 1 -- 172.21.15.53:0/2892076858 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c02ee70 con 0x7f67301599d0 2023-10-21T16:20:26.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.181+0000 7f6727fff640 1 -- 172.21.15.53:0/2892076858 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c036440 con 0x7f67301599d0 2023-10-21T16:20:26.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.181+0000 7f6727fff640 1 -- 172.21.15.53:0/2892076858 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0367d0 con 0x7f67301599d0 2023-10-21T16:20:26.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.181+0000 7f67367ac640 1 -- 172.21.15.53:0/2892076858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67301599d0 msgr2=0x7f67300ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.181+0000 7f67367ac640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67301599d0 0x7f67300ad730 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f672c00bb10 tx=0x7f672c02e0c0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 -- 172.21.15.53:0/2892076858 shutdown_connections 2023-10-21T16:20:26.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67300cc210 0x7f67300b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 --2- 172.21.15.53:0/2892076858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67301599d0 0x7f67300ad730 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 -- 172.21.15.53:0/2892076858 >> 172.21.15.53:0/2892076858 conn(0x7f67300cb0c0 msgr2=0x7f67300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:26.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 -- 172.21.15.53:0/2892076858 shutdown_connections 2023-10-21T16:20:26.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.182+0000 7f67367ac640 1 -- 172.21.15.53:0/2892076858 wait complete. 2023-10-21T16:20:26.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.183+0000 7f67367ac640 1 Processor -- start 2023-10-21T16:20:26.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.184+0000 7f67367ac640 1 -- start start 2023-10-21T16:20:26.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.184+0000 7f67367ac640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67300cc210 0x7f67300c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67367ac640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67301599d0 0x7f67300b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67367ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67367ac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b2730 con 0x7f6730159db0 2023-10-21T16:20:26.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67367ac640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b34d0 con 0x7f67300cc210 2023-10-21T16:20:26.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67367ac640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b29f0 con 0x7f67301599d0 2023-10-21T16:20:26.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37762/0 (socket says 172.21.15.53:37762) 2023-10-21T16:20:26.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 -- 172.21.15.53:0/2318140043 learned_addr learned my addr 172.21.15.53:0/2318140043 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:26.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f67357aa640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67300cc210 0x7f67300c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6734fa9640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67301599d0 0x7f67300b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 -- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67301599d0 msgr2=0x7f67300b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67301599d0 0x7f67300b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 -- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67300cc210 msgr2=0x7f67300c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67300cc210 0x7f67300c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.185+0000 7f6735fab640 1 -- 172.21.15.53:0/2318140043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c00b770 con 0x7f6730159db0 2023-10-21T16:20:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f6735fab640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f672800b970 tx=0x7f672800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800cb40 con 0x7f6730159db0 2023-10-21T16:20:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672800cca0 con 0x7f6730159db0 2023-10-21T16:20:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728014680 con 0x7f6730159db0 2023-10-21T16:20:26.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67300b2cd0 con 0x7f6730159db0 2023-10-21T16:20:26.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.186+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67300c7190 con 0x7f6730159db0 2023-10-21T16:20:26.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.187+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67300b31f0 con 0x7f6730159db0 2023-10-21T16:20:26.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.189+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6728003970 con 0x7f6730159db0 2023-10-21T16:20:26.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.190+0000 7f6725ffb640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 0x7f671406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.190+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67280897a0 con 0x7f6730159db0 2023-10-21T16:20:26.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.190+0000 7f67357aa640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 0x7f671406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.191+0000 7f67357aa640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 0x7f671406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f672c038000 tx=0x7f672c008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.193+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67280545b0 con 0x7f6730159db0 2023-10-21T16:20:26.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.387+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67300b31f0 con 0x7f6730159db0 2023-10-21T16:20:26.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.388+0000 7f6725ffb640 1 -- 172.21.15.53:0/2318140043 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f67300b31f0 con 0x7f6730159db0 2023-10-21T16:20:26.390 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:26.391 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":132,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:24.919265+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:26.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 msgr2=0x7f671406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 0x7f671406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f672c038000 tx=0x7f672c008040 comp rx=0 tx=0).stop 2023-10-21T16:20:26.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 msgr2=0x7f67300b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f672800b970 tx=0x7f672800be30 comp rx=0 tx=0).stop 2023-10-21T16:20:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 shutdown_connections 2023-10-21T16:20:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f671406af90 0x7f671406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6730159db0 0x7f67300b19b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67301599d0 0x7f67300b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 --2- 172.21.15.53:0/2318140043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67300cc210 0x7f67300c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 >> 172.21.15.53:0/2318140043 conn(0x7f67300cb0c0 msgr2=0x7f6730070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 shutdown_connections 2023-10-21T16:20:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.391+0000 7f67367ac640 1 -- 172.21.15.53:0/2318140043 wait complete. 2023-10-21T16:20:26.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:20:26.427 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-10-21T16:20:26.425+0000 7f127a69c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:20:26.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.634+0000 7fda50244640 1 Processor -- start 2023-10-21T16:20:26.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.634+0000 7fda50244640 1 -- start start 2023-10-21T16:20:26.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.635+0000 7fda50244640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda480cb600 0x7fda480b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.635+0000 7fda50244640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4806e1e0 con 0x7fda48161690 2023-10-21T16:20:26.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.635+0000 7fda50244640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda480cba50 con 0x7fda480cb150 2023-10-21T16:20:26.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.635+0000 7fda50244640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4806cc50 con 0x7fda480cb600 2023-10-21T16:20:26.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.636+0000 7fda4dfe0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fda48161690 0x7fda4806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46072/0 (socket says 172.21.15.53:46072) 2023-10-21T16:20:26.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.636+0000 7fda4dfe0640 1 -- 172.21.15.53:0/2925321813 learned_addr learned my addr 172.21.15.53:0/2925321813 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.636+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/2925321813 >> v2:172.21.15.124:3300/0 conn(0x7fda480cb600 0x7fda480b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.636+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032318388 0 0) 0x7fda4806e1e0 con 0x7fda48161690 2023-10-21T16:20:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda34003520 con 0x7fda48161690 2023-10-21T16:20:26.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126214232 0 0) 0x7fda480cba50 con 0x7fda480cb150 2023-10-21T16:20:26.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda4806e1e0 con 0x7fda480cb150 2023-10-21T16:20:26.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552901501 0 0) 0x7fda34003520 con 0x7fda48161690 2023-10-21T16:20:26.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 >> v2:172.21.15.124:3300/0 conn(0x7fda480cb600 msgr2=0x7fda480b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:26.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 --2- 172.21.15.53:0/2925321813 >> v2:172.21.15.124:3300/0 conn(0x7fda480cb600 0x7fda480b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 >> v1:172.21.15.124:6789/0 conn(0x7fda480cb150 legacy=0x7fda480b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4806cf30 con 0x7fda48161690 2023-10-21T16:20:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fda44001610 con 0x7fda48161690 2023-10-21T16:20:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 >> v1:172.21.15.53:6789/0 conn(0x7fda48161690 legacy=0x7fda4806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda34003b50 0x7fda34003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340047c0 0x7fda34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 0x7fda3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4806cc50 con 0x7fda34005180 2023-10-21T16:20:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400c050 con 0x7fda34003b50 2023-10-21T16:20:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.637+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3400c210 con 0x7fda340047c0 2023-10-21T16:20:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 0x7fda3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4dfe0640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda34003b50 0x7fda34003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4d7df640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340047c0 0x7fda34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 -- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340047c0 msgr2=0x7fda34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda340047c0 0x7fda34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 -- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda34003b50 msgr2=0x7fda34003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda34003b50 0x7fda34003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.638+0000 7fda4e7e1640 1 -- 172.21.15.53:0/2925321813 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34003520 con 0x7fda34005180 2023-10-21T16:20:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.639+0000 7fda4e7e1640 1 -- 172.21.15.53:0/2925321813 reap_dead start 2023-10-21T16:20:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.639+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 0x7fda3400b980 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fc0 tx=0x7fda340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b4bbf0506dcb97 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.640+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4002cc60 con 0x7fda34005180 2023-10-21T16:20:26.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.640+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4000f040 con 0x7fda34005180 2023-10-21T16:20:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.640+0000 7fda4cfde640 1 -- 172.21.15.53:0/2925321813 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40003990 con 0x7fda34005180 2023-10-21T16:20:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.640+0000 7fda50244640 1 -- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 msgr2=0x7fda3400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.640+0000 7fda50244640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 0x7fda3400b980 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fda40009fc0 tx=0x7fda340057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.641+0000 7fda50244640 1 -- 172.21.15.53:0/2925321813 shutdown_connections 2023-10-21T16:20:26.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.641+0000 7fda50244640 1 --2- 172.21.15.53:0/2925321813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda34005180 0x7fda3400b980 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.641+0000 7fda50244640 1 -- 172.21.15.53:0/2925321813 >> 172.21.15.53:0/2925321813 conn(0x7fda4806a5d0 msgr2=0x7fda480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.641+0000 7fda50244640 1 -- 172.21.15.53:0/2925321813 shutdown_connections 2023-10-21T16:20:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.641+0000 7fda50244640 1 -- 172.21.15.53:0/2925321813 wait complete. 2023-10-21T16:20:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.643+0000 7fda50244640 1 Processor -- start 2023-10-21T16:20:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.643+0000 7fda50244640 1 -- start start 2023-10-21T16:20:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 0x7fda481645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 0x7fda48161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda48162750 con 0x7fda48161fb0 2023-10-21T16:20:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda480c3990 con 0x7fda481f79b0 2023-10-21T16:20:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda50244640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda481f7830 con 0x7fda34005180 2023-10-21T16:20:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4e7e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4e7e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37788/0 (socket says 172.21.15.53:37788) 2023-10-21T16:20:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4e7e1640 1 -- 172.21.15.53:0/3130063136 learned_addr learned my addr 172.21.15.53:0/3130063136 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4dfe0640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 0x7fda481645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4d7df640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 0x7fda48161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4e7e1640 1 -- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 msgr2=0x7fda481645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.644+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 0x7fda481645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4e7e1640 1 -- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 msgr2=0x7fda48161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 0x7fda48161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4e7e1640 1 -- 172.21.15.53:0/3130063136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda481f7e30 con 0x7fda48161fb0 2023-10-21T16:20:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4dfe0640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 0x7fda481645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4d7df640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 0x7fda48161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda4e7e1640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e200 tx=0x7fda40009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40035df0 con 0x7fda48161fb0 2023-10-21T16:20:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda40003990 con 0x7fda48161fb0 2023-10-21T16:20:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda481f7550 con 0x7fda48161fb0 2023-10-21T16:20:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40003d40 con 0x7fda48161fb0 2023-10-21T16:20:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.645+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda480c35b0 con 0x7fda48161fb0 2023-10-21T16:20:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.647+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda480c3130 con 0x7fda48161fb0 2023-10-21T16:20:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.648+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda40004030 con 0x7fda48161fb0 2023-10-21T16:20:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.649+0000 7fda3e7fc640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 0x7fda2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.649+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda400ac5e0 con 0x7fda48161fb0 2023-10-21T16:20:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.650+0000 7fda4dfe0640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 0x7fda2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.651+0000 7fda4dfe0640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 0x7fda2c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca540 tx=0x7fda44009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.654+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda480c3130 con 0x7fda48161fb0 2023-10-21T16:20:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:26.887+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fda480c3ad0 con 0x7fda48161fb0 2023-10-21T16:20:27.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.481+0000 7fda3e7fc640 1 -- 172.21.15.53:0/3130063136 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7fda480c3ad0 con 0x7fda48161fb0 2023-10-21T16:20:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 msgr2=0x7fda2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 0x7fda2c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda480ca540 tx=0x7fda44009040 comp rx=0 tx=0).stop 2023-10-21T16:20:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 msgr2=0x7fda481f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e200 tx=0x7fda40009750 comp rx=0 tx=0).stop 2023-10-21T16:20:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 shutdown_connections 2023-10-21T16:20:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fda2c06af90 0x7fda2c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda48161fb0 0x7fda481f67a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda481f79b0 0x7fda48161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 --2- 172.21.15.53:0/3130063136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda34005180 0x7fda481645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 >> 172.21.15.53:0/3130063136 conn(0x7fda4806a5d0 msgr2=0x7fda480c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.485+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 shutdown_connections 2023-10-21T16:20:27.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.486+0000 7fda50244640 1 -- 172.21.15.53:0/3130063136 wait complete. 2023-10-21T16:20:27.508 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:20:27.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:20:27.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.782+0000 7effc43b0640 1 Processor -- start 2023-10-21T16:20:27.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.782+0000 7effc43b0640 1 -- start start 2023-10-21T16:20:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.784+0000 7effc43b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.784+0000 7effc43b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 0x7effbc0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.784+0000 7effc43b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc15b310 con 0x7effbc06db60 2023-10-21T16:20:27.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.784+0000 7effc43b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0ca230 con 0x7effbc15aa00 2023-10-21T16:20:27.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.784+0000 7effc43b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc06d290 con 0x7effbc06c3a0 2023-10-21T16:20:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc214c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc294d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 0x7effbc0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc214c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:45510/0 (socket says 172.21.15.53:45510) 2023-10-21T16:20:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc294d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 0x7effbc0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:58850/0 (socket says 172.21.15.53:58850) 2023-10-21T16:20:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc214c640 1 -- 172.21.15.53:0/1545577330 learned_addr learned my addr 172.21.15.53:0/1545577330 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc214c640 1 -- 172.21.15.53:0/1545577330 >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 msgr2=0x7effbc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.785+0000 7effc214c640 1 --2- 172.21.15.53:0/1545577330 >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 0x7effbc0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.786+0000 7effc214c640 1 -- 172.21.15.53:0/1545577330 >> v1:172.21.15.124:6789/0 conn(0x7effbc06db60 legacy=0x7effbc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.786+0000 7effc214c640 1 -- 172.21.15.53:0/1545577330 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc0c9860 con 0x7effbc06c3a0 2023-10-21T16:20:27.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.787+0000 7effc214c640 1 --2- 172.21.15.53:0/1545577330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7effb8009870 tx=0x7effb802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a81c77c12998147 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:27.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.787+0000 7effc114a640 1 -- 172.21.15.53:0/1545577330 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8030840 con 0x7effbc06c3a0 2023-10-21T16:20:27.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.787+0000 7effc114a640 1 -- 172.21.15.53:0/1545577330 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb80309a0 con 0x7effbc06c3a0 2023-10-21T16:20:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.787+0000 7effc114a640 1 -- 172.21.15.53:0/1545577330 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8030d30 con 0x7effbc06c3a0 2023-10-21T16:20:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 -- 172.21.15.53:0/1545577330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 msgr2=0x7effbc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 --2- 172.21.15.53:0/1545577330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7effb8009870 tx=0x7effb802b980 comp rx=0 tx=0).stop 2023-10-21T16:20:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 -- 172.21.15.53:0/1545577330 shutdown_connections 2023-10-21T16:20:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 --2- 172.21.15.53:0/1545577330 >> v2:172.21.15.124:3300/0 conn(0x7effbc15aa00 0x7effbc0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 --2- 172.21.15.53:0/1545577330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c9cf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 -- 172.21.15.53:0/1545577330 >> 172.21.15.53:0/1545577330 conn(0x7effbc06a5d0 msgr2=0x7effbc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 -- 172.21.15.53:0/1545577330 shutdown_connections 2023-10-21T16:20:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.788+0000 7effc43b0640 1 -- 172.21.15.53:0/1545577330 wait complete. 2023-10-21T16:20:27.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.790+0000 7effc43b0640 1 Processor -- start 2023-10-21T16:20:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.790+0000 7effc43b0640 1 -- start start 2023-10-21T16:20:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effbc06db60 0x7effbc1f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 0x7effbc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0af7c0 con 0x7effbc15aa00 2023-10-21T16:20:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0c6760 con 0x7effbc06db60 2023-10-21T16:20:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.791+0000 7effc43b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0c39e0 con 0x7effbc06c3a0 2023-10-21T16:20:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc214c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc214c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:45512/0 (socket says 172.21.15.53:45512) 2023-10-21T16:20:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc214c640 1 -- 172.21.15.53:0/4033741509 learned_addr learned my addr 172.21.15.53:0/4033741509 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 0x7effbc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc194b640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effbc06db60 0x7effbc1f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 -- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 msgr2=0x7effbc0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 -- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effbc06db60 msgr2=0x7effbc1f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effbc06db60 0x7effbc1f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.792+0000 7effc294d640 1 -- 172.21.15.53:0/4033741509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80094d0 con 0x7effbc15aa00 2023-10-21T16:20:27.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effc294d640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 0x7effbc0cd690 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7effb400eba0 tx=0x7effb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:27.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb400ccc0 con 0x7effbc15aa00 2023-10-21T16:20:27.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb400f040 con 0x7effbc15aa00 2023-10-21T16:20:27.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc0c2770 con 0x7effbc15aa00 2023-10-21T16:20:27.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40146c0 con 0x7effbc15aa00 2023-10-21T16:20:27.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.793+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc0c2d30 con 0x7effbc15aa00 2023-10-21T16:20:27.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.794+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc04e160 con 0x7effbc15aa00 2023-10-21T16:20:27.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.796+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7effb4014820 con 0x7effbc15aa00 2023-10-21T16:20:27.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.797+0000 7effb2ffd640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 0x7effa006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:27.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.797+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effb408a6f0 con 0x7effbc15aa00 2023-10-21T16:20:27.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.797+0000 7effc214c640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 0x7effa006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:27.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.799+0000 7effc214c640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 0x7effa006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effb8009870 tx=0x7effb8008030 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:27.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:27.801+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effb40553a0 con 0x7effbc15aa00 2023-10-21T16:20:28.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.034+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7effbc0c94c0 con 0x7effbc15aa00 2023-10-21T16:20:28.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.493+0000 7effb2ffd640 1 -- 172.21.15.53:0/4033741509 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7effb40589c0 con 0x7effbc15aa00 2023-10-21T16:20:28.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 msgr2=0x7effa006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 0x7effa006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effb8009870 tx=0x7effb8008030 comp rx=0 tx=0).stop 2023-10-21T16:20:28.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 msgr2=0x7effbc0cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 0x7effbc0cd690 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7effb400eba0 tx=0x7effb400c490 comp rx=0 tx=0).stop 2023-10-21T16:20:28.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 shutdown_connections 2023-10-21T16:20:28.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7effa006af90 0x7effa006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effbc15aa00 0x7effbc0cd690 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effbc06db60 0x7effbc1f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 --2- 172.21.15.53:0/4033741509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effbc06c3a0 0x7effbc0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 >> 172.21.15.53:0/4033741509 conn(0x7effbc06a5d0 msgr2=0x7effbc06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:28.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 shutdown_connections 2023-10-21T16:20:28.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.497+0000 7effc43b0640 1 -- 172.21.15.53:0/4033741509 wait complete. 2023-10-21T16:20:28.520 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:20:28.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:28.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.741+0000 7f54e7fff640 1 Processor -- start 2023-10-21T16:20:28.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.741+0000 7f54e7fff640 1 -- start start 2023-10-21T16:20:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f54e80b07d0 0x7f54e8165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e7fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e815f7d0 con 0x7f54e80adc70 2023-10-21T16:20:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e7fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e815efb0 con 0x7f54e815ab30 2023-10-21T16:20:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e7fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e815f210 con 0x7f54e80b07d0 2023-10-21T16:20:28.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e67fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f54e80adc70 0x7f54e80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46074/0 (socket says 172.21.15.53:46074) 2023-10-21T16:20:28.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.743+0000 7f54e67fc640 1 -- 172.21.15.53:0/3605778674 learned_addr learned my addr 172.21.15.53:0/3605778674 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:28.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e77fe640 1 --2- 172.21.15.53:0/3605778674 >> v2:172.21.15.124:3300/0 conn(0x7f54e80b07d0 0x7f54e8165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481557382 0 0) 0x7f54e815f7d0 con 0x7f54e80adc70 2023-10-21T16:20:28.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54d0003520 con 0x7f54e80adc70 2023-10-21T16:20:28.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112976330 0 0) 0x7f54d0003520 con 0x7f54e80adc70 2023-10-21T16:20:28.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 >> v2:172.21.15.124:3300/0 conn(0x7f54e80b07d0 msgr2=0x7f54e8165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:28.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 --2- 172.21.15.53:0/3605778674 >> v2:172.21.15.124:3300/0 conn(0x7f54e80b07d0 0x7f54e8165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 >> v1:172.21.15.124:6789/0 conn(0x7f54e815ab30 legacy=0x7f54e815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:28.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e815f4f0 con 0x7f54e80adc70 2023-10-21T16:20:28.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f54dc001670 con 0x7f54e80adc70 2023-10-21T16:20:28.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 >> v1:172.21.15.53:6789/0 conn(0x7f54e80adc70 legacy=0x7f54e80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54d0003970 0x7f54d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d00047c0 0x7f54d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.744+0000 7f54e5ffb640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 0x7f54d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e815f210 con 0x7f54d0005180 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d000c0d0 con 0x7f54d00047c0 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d000c290 con 0x7f54d0003970 2023-10-21T16:20:28.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 0x7f54d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e67fc640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d00047c0 0x7f54d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e6ffd640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54d0003970 0x7f54d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 -- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54d0003970 msgr2=0x7f54d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:28.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54d0003970 0x7f54d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 -- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d00047c0 msgr2=0x7f54d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:28.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d00047c0 0x7f54d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e77fe640 1 -- 172.21.15.53:0/3605778674 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d0003520 con 0x7f54d0005180 2023-10-21T16:20:28.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.745+0000 7f54e6ffd640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54d0003970 0x7f54d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:28.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.746+0000 7f54e77fe640 1 -- 172.21.15.53:0/3605778674 reap_dead start 2023-10-21T16:20:28.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.746+0000 7f54e77fe640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 0x7f54d000b980 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f54e0007360 tx=0x7f54d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f593d347a1dc7734 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:28.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.747+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e002ccf0 con 0x7f54d0005180 2023-10-21T16:20:28.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.747+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e002ce90 con 0x7f54d0005180 2023-10-21T16:20:28.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.747+0000 7f54e5ffb640 1 -- 172.21.15.53:0/3605778674 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0003bd0 con 0x7f54d0005180 2023-10-21T16:20:28.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.747+0000 7f54e7fff640 1 -- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 msgr2=0x7f54d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.747+0000 7f54e7fff640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 0x7f54d000b980 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f54e0007360 tx=0x7f54d00057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.748+0000 7f54e7fff640 1 -- 172.21.15.53:0/3605778674 shutdown_connections 2023-10-21T16:20:28.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.748+0000 7f54e7fff640 1 --2- 172.21.15.53:0/3605778674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54d0005180 0x7f54d000b980 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.748+0000 7f54e7fff640 1 -- 172.21.15.53:0/3605778674 >> 172.21.15.53:0/3605778674 conn(0x7f54e80cb0c0 msgr2=0x7f54e80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:28.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.748+0000 7f54e7fff640 1 -- 172.21.15.53:0/3605778674 shutdown_connections 2023-10-21T16:20:28.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.748+0000 7f54e7fff640 1 -- 172.21.15.53:0/3605778674 wait complete. 2023-10-21T16:20:28.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.749+0000 7f54e7fff640 1 Processor -- start 2023-10-21T16:20:28.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.750+0000 7f54e7fff640 1 -- start start 2023-10-21T16:20:28.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.750+0000 7f54e7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d0005180 0x7f54e80c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54e8155930 0x7f54e814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81549e0 con 0x7f54e81554b0 2023-10-21T16:20:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e80c7ff0 con 0x7f54d0005180 2023-10-21T16:20:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e80c2e90 con 0x7f54e8155930 2023-10-21T16:20:28.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37820/0 (socket says 172.21.15.53:37820) 2023-10-21T16:20:28.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e67fc640 1 -- 172.21.15.53:0/2160916401 learned_addr learned my addr 172.21.15.53:0/2160916401 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:28.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e6ffd640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d0005180 0x7f54e80c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.751+0000 7f54e77fe640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54e8155930 0x7f54e814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 -- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54e8155930 msgr2=0x7f54e814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54e8155930 0x7f54e814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 -- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d0005180 msgr2=0x7f54e80c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d0005180 0x7f54e80c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 -- 172.21.15.53:0/2160916401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8153e60 con 0x7f54e81554b0 2023-10-21T16:20:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e67fc640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e710 tx=0x7f54dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00ee20 con 0x7f54e81554b0 2023-10-21T16:20:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00c8d0 con 0x7f54e81554b0 2023-10-21T16:20:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e8154140 con 0x7f54e81554b0 2023-10-21T16:20:28.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00cc80 con 0x7f54e81554b0 2023-10-21T16:20:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.752+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e8154700 con 0x7f54e81554b0 2023-10-21T16:20:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.754+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e80c3090 con 0x7f54e81554b0 2023-10-21T16:20:28.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.755+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f54dc01d460 con 0x7f54e81554b0 2023-10-21T16:20:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.756+0000 7f54cf7fe640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 0x7f54d406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.757+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54dc0899b0 con 0x7f54e81554b0 2023-10-21T16:20:28.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.757+0000 7f54e6ffd640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 0x7f54d406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.758+0000 7f54e6ffd640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 0x7f54d406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54d8004e70 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:28.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.760+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54e80c3090 con 0x7f54e81554b0 2023-10-21T16:20:28.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.992+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54e80c9ca0 con 0x7f54e81554b0 2023-10-21T16:20:28.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.993+0000 7f54cf7fe640 1 -- 172.21.15.53:0/2160916401 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f54dc054740 con 0x7f54e81554b0 2023-10-21T16:20:28.995 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:28.996 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:20:28.996 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:20:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 msgr2=0x7f54d406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 0x7f54d406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54d8004e70 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2023-10-21T16:20:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 msgr2=0x7f54e814f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e710 tx=0x7f54dc00ebd0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 shutdown_connections 2023-10-21T16:20:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f54d406afc0 0x7f54d406d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54e8155930 0x7f54e814fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e81554b0 0x7f54e814f910 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 --2- 172.21.15.53:0/2160916401 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54d0005180 0x7f54e80c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:28.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 >> 172.21.15.53:0/2160916401 conn(0x7f54e80cb0c0 msgr2=0x7f54e80af750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 shutdown_connections 2023-10-21T16:20:29.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:28.996+0000 7f54e7fff640 1 -- 172.21.15.53:0/2160916401 wait complete. 2023-10-21T16:20:29.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:20:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.242+0000 7fa7d6106640 1 Processor -- start 2023-10-21T16:20:29.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.243+0000 7fa7d6106640 1 -- start start 2023-10-21T16:20:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.244+0000 7fa7d6106640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7d015ab30 0x7fa7d015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.244+0000 7fa7d6106640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d00adc70 0x7fa7d00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.244+0000 7fa7d6106640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d015f7d0 con 0x7fa7d015af10 2023-10-21T16:20:29.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.244+0000 7fa7d6106640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d015f210 con 0x7fa7d015ab30 2023-10-21T16:20:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.244+0000 7fa7d6106640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d015f4f0 con 0x7fa7d00adc70 2023-10-21T16:20:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7d015ab30 0x7fa7d015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa7d015ab30 0x7fa7d015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37830/0 (socket says 172.21.15.53:37830) 2023-10-21T16:20:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5905640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa7d015af10 0x7fa7d00701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46084/0 (socket says 172.21.15.53:46084) 2023-10-21T16:20:29.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 -- 172.21.15.53:0/271198167 learned_addr learned my addr 172.21.15.53:0/271198167 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d4903640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d00adc70 0x7fa7d00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 -- 172.21.15.53:0/271198167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d00adc70 msgr2=0x7fa7d00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d00adc70 0x7fa7d00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 -- 172.21.15.53:0/271198167 >> v1:172.21.15.53:6789/0 conn(0x7fa7d015af10 legacy=0x7fa7d00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.245+0000 7fa7d5104640 1 -- 172.21.15.53:0/271198167 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d006c550 con 0x7fa7d015ab30 2023-10-21T16:20:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7d5104640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d015ab30 0x7fa7d015e8a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009870 tx=0x7fa7cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b009d72e6bf7c0b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7c7fff640 1 -- 172.21.15.53:0/271198167 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc030840 con 0x7fa7d015ab30 2023-10-21T16:20:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7c7fff640 1 -- 172.21.15.53:0/271198167 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc0309a0 con 0x7fa7d015ab30 2023-10-21T16:20:29.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7c7fff640 1 -- 172.21.15.53:0/271198167 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc030d30 con 0x7fa7d015ab30 2023-10-21T16:20:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7d6106640 1 -- 172.21.15.53:0/271198167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d015ab30 msgr2=0x7fa7d015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.247+0000 7fa7d6106640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d015ab30 0x7fa7d015e8a0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009870 tx=0x7fa7cc02b980 comp rx=0 tx=0).stop 2023-10-21T16:20:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 -- 172.21.15.53:0/271198167 shutdown_connections 2023-10-21T16:20:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d00adc70 0x7fa7d00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 --2- 172.21.15.53:0/271198167 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d015ab30 0x7fa7d015e8a0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 -- 172.21.15.53:0/271198167 >> 172.21.15.53:0/271198167 conn(0x7fa7d00cb0c0 msgr2=0x7fa7d00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 -- 172.21.15.53:0/271198167 shutdown_connections 2023-10-21T16:20:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.248+0000 7fa7d6106640 1 -- 172.21.15.53:0/271198167 wait complete. 2023-10-21T16:20:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.249+0000 7fa7d6106640 1 Processor -- start 2023-10-21T16:20:29.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.250+0000 7fa7d6106640 1 -- start start 2023-10-21T16:20:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7d015ab30 0x7fa7d0148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d015af10 0x7fa7d0148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00b04a0 con 0x7fa7d00adc70 2023-10-21T16:20:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c58b0 con 0x7fa7d015ab30 2023-10-21T16:20:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d6106640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c5b90 con 0x7fa7d015af10 2023-10-21T16:20:29.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37846/0 (socket says 172.21.15.53:37846) 2023-10-21T16:20:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 -- 172.21.15.53:0/1763310629 learned_addr learned my addr 172.21.15.53:0/1763310629 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 -- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d015af10 msgr2=0x7fa7d0148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d015af10 0x7fa7d0148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 -- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7d015ab30 msgr2=0x7fa7d0148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7d015ab30 0x7fa7d0148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.251+0000 7fa7d5104640 1 -- 172.21.15.53:0/1763310629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0094d0 con 0x7fa7d00adc70 2023-10-21T16:20:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7d5104640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc039c90 con 0x7fa7d00adc70 2023-10-21T16:20:29.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc039df0 con 0x7fa7d00adc70 2023-10-21T16:20:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc043640 con 0x7fa7d00adc70 2023-10-21T16:20:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00c5e70 con 0x7fa7d00adc70 2023-10-21T16:20:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.252+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00b3000 con 0x7fa7d00adc70 2023-10-21T16:20:29.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.253+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d00c6390 con 0x7fa7d00adc70 2023-10-21T16:20:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.254+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa7cc0437a0 con 0x7fa7d00adc70 2023-10-21T16:20:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.256+0000 7fa7c5ffb640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 0x7fa7b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.256+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7cc03e070 con 0x7fa7d00adc70 2023-10-21T16:20:29.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.256+0000 7fa7d4903640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 0x7fa7b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.257+0000 7fa7d4903640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 0x7fa7b406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d01f6510 tx=0x7fa7c0008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.260+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7d00c6390 con 0x7fa7d00adc70 2023-10-21T16:20:29.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.458+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7d00c6390 con 0x7fa7d00adc70 2023-10-21T16:20:29.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.462+0000 7fa7c5ffb640 1 -- 172.21.15.53:0/1763310629 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16774 (secure 0 0 0) 0x7fa7cc034090 con 0x7fa7d00adc70 2023-10-21T16:20:29.464 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:29.464 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":133,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:26.463998+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:20:29.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.464+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 msgr2=0x7fa7b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.464+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 0x7fa7b406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa7d01f6510 tx=0x7fa7c0008040 comp rx=0 tx=0).stop 2023-10-21T16:20:29.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.464+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 msgr2=0x7fa7d00b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.464+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc004730 comp rx=0 tx=0).stop 2023-10-21T16:20:29.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 shutdown_connections 2023-10-21T16:20:29.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fa7b406af90 0x7fa7b406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7d015af10 0x7fa7d0148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7d015ab30 0x7fa7d0148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 --2- 172.21.15.53:0/1763310629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7d00adc70 0x7fa7d00b2ac0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 >> 172.21.15.53:0/1763310629 conn(0x7fa7d00cb0c0 msgr2=0x7fa7d00ac910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:29.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 shutdown_connections 2023-10-21T16:20:29.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.465+0000 7fa7d6106640 1 -- 172.21.15.53:0/1763310629 wait complete. 2023-10-21T16:20:29.486 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:20:29.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:20:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.710+0000 7f316a2af640 1 Processor -- start 2023-10-21T16:20:29.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.710+0000 7f316a2af640 1 -- start start 2023-10-21T16:20:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316a2af640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31640c90b0 0x7f31640cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316a2af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31640cb3b0 0x7f316415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316a2af640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316415b2b0 con 0x7f316415ab40 2023-10-21T16:20:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316a2af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316415bf30 con 0x7f31640cb3b0 2023-10-21T16:20:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316a2af640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640cb820 con 0x7f31640c90b0 2023-10-21T16:20:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316884c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31640cb3b0 0x7f316415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316884c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f31640cb3b0 0x7f316415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37848/0 (socket says 172.21.15.53:37848) 2023-10-21T16:20:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f316884c640 1 -- 172.21.15.53:0/2318646153 learned_addr learned my addr 172.21.15.53:0/2318646153 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.712+0000 7f31637fe640 1 --2- 172.21.15.53:0/2318646153 >> v2:172.21.15.124:3300/0 conn(0x7f31640c90b0 0x7f31640cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.713+0000 7f316884c640 1 -- 172.21.15.53:0/2318646153 >> v2:172.21.15.124:3300/0 conn(0x7f31640c90b0 msgr2=0x7f31640cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:29.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.713+0000 7f316884c640 1 --2- 172.21.15.53:0/2318646153 >> v2:172.21.15.124:3300/0 conn(0x7f31640c90b0 0x7f31640cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.713+0000 7f316884c640 1 -- 172.21.15.53:0/2318646153 >> v1:172.21.15.124:6789/0 conn(0x7f316415ab40 legacy=0x7f31640ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:29.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.713+0000 7f316884c640 1 -- 172.21.15.53:0/2318646153 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316415b590 con 0x7f31640cb3b0 2023-10-21T16:20:29.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.714+0000 7f316884c640 1 --2- 172.21.15.53:0/2318646153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640cb3b0 0x7f316415e790 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f316406e560 tx=0x7f315c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3fa56b700d8ce8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.714+0000 7f3162ffd640 1 -- 172.21.15.53:0/2318646153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c02ca80 con 0x7f31640cb3b0 2023-10-21T16:20:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.714+0000 7f3162ffd640 1 -- 172.21.15.53:0/2318646153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c02cbe0 con 0x7f31640cb3b0 2023-10-21T16:20:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.714+0000 7f3162ffd640 1 -- 172.21.15.53:0/2318646153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0343f0 con 0x7f31640cb3b0 2023-10-21T16:20:29.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 -- 172.21.15.53:0/2318646153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640cb3b0 msgr2=0x7f316415e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 --2- 172.21.15.53:0/2318646153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640cb3b0 0x7f316415e790 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f316406e560 tx=0x7f315c02bee0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 -- 172.21.15.53:0/2318646153 shutdown_connections 2023-10-21T16:20:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 --2- 172.21.15.53:0/2318646153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640cb3b0 0x7f316415e790 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 --2- 172.21.15.53:0/2318646153 >> v2:172.21.15.124:3300/0 conn(0x7f31640c90b0 0x7f31640cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 -- 172.21.15.53:0/2318646153 >> 172.21.15.53:0/2318646153 conn(0x7f316406a380 msgr2=0x7f31640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:29.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 -- 172.21.15.53:0/2318646153 shutdown_connections 2023-10-21T16:20:29.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.715+0000 7f316a2af640 1 -- 172.21.15.53:0/2318646153 wait complete. 2023-10-21T16:20:29.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.717+0000 7f316a2af640 1 Processor -- start 2023-10-21T16:20:29.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.717+0000 7f316a2af640 1 -- start start 2023-10-21T16:20:29.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 0x7f31640c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316415ab40 0x7f3164148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640c5f50 con 0x7f31640c90b0 2023-10-21T16:20:29.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31640c7f80 con 0x7f31640cb3b0 2023-10-21T16:20:29.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f316a2af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3164148780 con 0x7f316415ab40 2023-10-21T16:20:29.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.718+0000 7f3163fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37862/0 (socket says 172.21.15.53:37862) 2023-10-21T16:20:29.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 -- 172.21.15.53:0/2062029646 learned_addr learned my addr 172.21.15.53:0/2062029646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:29.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f31637fe640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 0x7f31640c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f316884c640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316415ab40 0x7f3164148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 -- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316415ab40 msgr2=0x7f3164148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316415ab40 0x7f3164148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 -- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 msgr2=0x7f31640c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 0x7f31640c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 -- 172.21.15.53:0/2062029646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c009b30 con 0x7f31640c90b0 2023-10-21T16:20:29.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f31637fe640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 0x7f31640c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:29.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.719+0000 7f3163fff640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f315400c460 tx=0x7f315400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.720+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154012830 con 0x7f31640c90b0 2023-10-21T16:20:29.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.720+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31540129d0 con 0x7f31640c90b0 2023-10-21T16:20:29.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.720+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31640c7650 con 0x7f31640c90b0 2023-10-21T16:20:29.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.720+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154012d80 con 0x7f31640c90b0 2023-10-21T16:20:29.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.720+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31640c7c10 con 0x7f31640c90b0 2023-10-21T16:20:29.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.721+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164067490 con 0x7f31640c90b0 2023-10-21T16:20:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.722+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f315401a3b0 con 0x7f31640c90b0 2023-10-21T16:20:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.723+0000 7f3160ff9640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 0x7f315006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.723+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3154089500 con 0x7f31640c90b0 2023-10-21T16:20:29.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.723+0000 7f31637fe640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 0x7f315006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:29.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.724+0000 7f31637fe640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 0x7f315006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3158006fd0 tx=0x7f3158008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:29.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.727+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31540543c0 con 0x7f31640c90b0 2023-10-21T16:20:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.915+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 --> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f316415bb50 con 0x7f315006af90 2023-10-21T16:20:29.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.945+0000 7f3160ff9640 1 -- 172.21.15.53:0/2062029646 <== mgr.6760 v2:172.21.15.124:6832/596536042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286860 (secure 0 0 0) 0x7f3158009530 con 0x7f315006af90 2023-10-21T16:20:29.950 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:29.952 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:20:29.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 msgr2=0x7f315006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 0x7f315006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3158006fd0 tx=0x7f3158008040 comp rx=0 tx=0).stop 2023-10-21T16:20:29.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 msgr2=0x7f31640c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:29.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f315400c460 tx=0x7f315400c920 comp rx=0 tx=0).stop 2023-10-21T16:20:29.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 shutdown_connections 2023-10-21T16:20:29.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f315006af90 0x7f315006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f316415ab40 0x7f3164148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640cb3b0 0x7f31640c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 --2- 172.21.15.53:0/2062029646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31640c90b0 0x7f31640c5270 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 >> 172.21.15.53:0/2062029646 conn(0x7f316406a380 msgr2=0x7f31640c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 shutdown_connections 2023-10-21T16:20:29.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:29.950+0000 7f316a2af640 1 -- 172.21.15.53:0/2062029646 wait complete. 2023-10-21T16:20:29.974 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:20:28.819056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":371,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4768,"ondisk_log_size":4768,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":332904,"kb_used_data":14052,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":754641816,"statfs":{"total":773094113280,"available":772753219584,"internally_reserved":0,"allocated":14389248,"data_stored":7844652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":326356537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.384646"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687907+0000","last_change":"2023-10-21T16:20:25.512333+0000","last_active":"2023-10-21T16:20:26.687907+0000","last_peered":"2023-10-21T16:20:26.687907+0000","last_clean":"2023-10-21T16:20:26.687907+0000","last_became_active":"2023-10-21T16:20:25.508973+0000","last_became_peered":"2023-10-21T16:20:25.508973+0000","last_unstale":"2023-10-21T16:20:26.687907+0000","last_undegraded":"2023-10-21T16:20:26.687907+0000","last_fullsized":"2023-10-21T16:20:26.687907+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:11:40.223441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218540+0000","last_change":"2023-10-21T16:20:25.497809+0000","last_active":"2023-10-21T16:20:27.218540+0000","last_peered":"2023-10-21T16:20:27.218540+0000","last_clean":"2023-10-21T16:20:27.218540+0000","last_became_active":"2023-10-21T16:20:25.497305+0000","last_became_peered":"2023-10-21T16:20:25.497305+0000","last_unstale":"2023-10-21T16:20:27.218540+0000","last_undegraded":"2023-10-21T16:20:27.218540+0000","last_fullsized":"2023-10-21T16:20:27.218540+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:36:33.724529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687731+0000","last_change":"2023-10-21T16:20:25.507106+0000","last_active":"2023-10-21T16:20:26.687731+0000","last_peered":"2023-10-21T16:20:26.687731+0000","last_clean":"2023-10-21T16:20:26.687731+0000","last_became_active":"2023-10-21T16:20:25.506334+0000","last_became_peered":"2023-10-21T16:20:25.506334+0000","last_unstale":"2023-10-21T16:20:26.687731+0000","last_undegraded":"2023-10-21T16:20:26.687731+0000","last_fullsized":"2023-10-21T16:20:26.687731+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:12.700942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470792+0000","last_change":"2023-10-21T16:20:25.500829+0000","last_active":"2023-10-21T16:20:26.470792+0000","last_peered":"2023-10-21T16:20:26.470792+0000","last_clean":"2023-10-21T16:20:26.470792+0000","last_became_active":"2023-10-21T16:20:25.500337+0000","last_became_peered":"2023-10-21T16:20:25.500337+0000","last_unstale":"2023-10-21T16:20:26.470792+0000","last_undegraded":"2023-10-21T16:20:26.470792+0000","last_fullsized":"2023-10-21T16:20:26.470792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:59:30.783483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217071+0000","last_change":"2023-10-21T16:20:25.507586+0000","last_active":"2023-10-21T16:20:27.217071+0000","last_peered":"2023-10-21T16:20:27.217071+0000","last_clean":"2023-10-21T16:20:27.217071+0000","last_became_active":"2023-10-21T16:20:25.503472+0000","last_became_peered":"2023-10-21T16:20:25.503472+0000","last_unstale":"2023-10-21T16:20:27.217071+0000","last_undegraded":"2023-10-21T16:20:27.217071+0000","last_fullsized":"2023-10-21T16:20:27.217071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:53.863086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217130+0000","last_change":"2023-10-21T16:20:25.507551+0000","last_active":"2023-10-21T16:20:27.217130+0000","last_peered":"2023-10-21T16:20:27.217130+0000","last_clean":"2023-10-21T16:20:27.217130+0000","last_became_active":"2023-10-21T16:20:25.507085+0000","last_became_peered":"2023-10-21T16:20:25.507085+0000","last_unstale":"2023-10-21T16:20:27.217130+0000","last_undegraded":"2023-10-21T16:20:27.217130+0000","last_fullsized":"2023-10-21T16:20:27.217130+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:52:25.809487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216570+0000","last_change":"2023-10-21T16:20:25.499521+0000","last_active":"2023-10-21T16:20:27.216570+0000","last_peered":"2023-10-21T16:20:27.216570+0000","last_clean":"2023-10-21T16:20:27.216570+0000","last_became_active":"2023-10-21T16:20:25.499209+0000","last_became_peered":"2023-10-21T16:20:25.499209+0000","last_unstale":"2023-10-21T16:20:27.216570+0000","last_undegraded":"2023-10-21T16:20:27.216570+0000","last_fullsized":"2023-10-21T16:20:27.216570+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:14.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474857+0000","last_change":"2023-10-21T16:20:25.520411+0000","last_active":"2023-10-21T16:20:26.474857+0000","last_peered":"2023-10-21T16:20:26.474857+0000","last_clean":"2023-10-21T16:20:26.474857+0000","last_became_active":"2023-10-21T16:20:25.520100+0000","last_became_peered":"2023-10-21T16:20:25.520100+0000","last_unstale":"2023-10-21T16:20:26.474857+0000","last_undegraded":"2023-10-21T16:20:26.474857+0000","last_fullsized":"2023-10-21T16:20:26.474857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:11.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218446+0000","last_change":"2023-10-21T16:20:25.506546+0000","last_active":"2023-10-21T16:20:27.218446+0000","last_peered":"2023-10-21T16:20:27.218446+0000","last_clean":"2023-10-21T16:20:27.218446+0000","last_became_active":"2023-10-21T16:20:25.504739+0000","last_became_peered":"2023-10-21T16:20:25.504739+0000","last_unstale":"2023-10-21T16:20:27.218446+0000","last_undegraded":"2023-10-21T16:20:27.218446+0000","last_fullsized":"2023-10-21T16:20:27.218446+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:15.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217242+0000","last_change":"2023-10-21T16:20:25.516538+0000","last_active":"2023-10-21T16:20:27.217242+0000","last_peered":"2023-10-21T16:20:27.217242+0000","last_clean":"2023-10-21T16:20:27.217242+0000","last_became_active":"2023-10-21T16:20:25.516243+0000","last_became_peered":"2023-10-21T16:20:25.516243+0000","last_unstale":"2023-10-21T16:20:27.217242+0000","last_undegraded":"2023-10-21T16:20:27.217242+0000","last_fullsized":"2023-10-21T16:20:27.217242+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:50:18.987282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.899025+0000","last_change":"2023-10-21T16:20:25.498593+0000","last_active":"2023-10-21T16:20:26.899025+0000","last_peered":"2023-10-21T16:20:26.899025+0000","last_clean":"2023-10-21T16:20:26.899025+0000","last_became_active":"2023-10-21T16:20:25.498244+0000","last_became_peered":"2023-10-21T16:20:25.498244+0000","last_unstale":"2023-10-21T16:20:26.899025+0000","last_undegraded":"2023-10-21T16:20:26.899025+0000","last_fullsized":"2023-10-21T16:20:26.899025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:07:33.886401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475780+0000","last_change":"2023-10-21T16:20:25.501223+0000","last_active":"2023-10-21T16:20:26.475780+0000","last_peered":"2023-10-21T16:20:26.475780+0000","last_clean":"2023-10-21T16:20:26.475780+0000","last_became_active":"2023-10-21T16:20:25.500245+0000","last_became_peered":"2023-10-21T16:20:25.500245+0000","last_unstale":"2023-10-21T16:20:26.475780+0000","last_undegraded":"2023-10-21T16:20:26.475780+0000","last_fullsized":"2023-10-21T16:20:26.475780+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:56:37.087017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470400+0000","last_change":"2023-10-21T16:20:25.500938+0000","last_active":"2023-10-21T16:20:26.470400+0000","last_peered":"2023-10-21T16:20:26.470400+0000","last_clean":"2023-10-21T16:20:26.470400+0000","last_became_active":"2023-10-21T16:20:25.499117+0000","last_became_peered":"2023-10-21T16:20:25.499117+0000","last_unstale":"2023-10-21T16:20:26.470400+0000","last_undegraded":"2023-10-21T16:20:26.470400+0000","last_fullsized":"2023-10-21T16:20:26.470400+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:28.356874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217378+0000","last_change":"2023-10-21T16:20:25.499494+0000","last_active":"2023-10-21T16:20:27.217378+0000","last_peered":"2023-10-21T16:20:27.217378+0000","last_clean":"2023-10-21T16:20:27.217378+0000","last_became_active":"2023-10-21T16:20:25.498174+0000","last_became_peered":"2023-10-21T16:20:25.498174+0000","last_unstale":"2023-10-21T16:20:27.217378+0000","last_undegraded":"2023-10-21T16:20:27.217378+0000","last_fullsized":"2023-10-21T16:20:27.217378+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:06.511126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216083+0000","last_change":"2023-10-21T16:20:25.504658+0000","last_active":"2023-10-21T16:20:27.216083+0000","last_peered":"2023-10-21T16:20:27.216083+0000","last_clean":"2023-10-21T16:20:27.216083+0000","last_became_active":"2023-10-21T16:20:25.504396+0000","last_became_peered":"2023-10-21T16:20:25.504396+0000","last_unstale":"2023-10-21T16:20:27.216083+0000","last_undegraded":"2023-10-21T16:20:27.216083+0000","last_fullsized":"2023-10-21T16:20:27.216083+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:41:05.529613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216008+0000","last_change":"2023-10-21T16:20:25.498612+0000","last_active":"2023-10-21T16:20:27.216008+0000","last_peered":"2023-10-21T16:20:27.216008+0000","last_clean":"2023-10-21T16:20:27.216008+0000","last_became_active":"2023-10-21T16:20:25.498337+0000","last_became_peered":"2023-10-21T16:20:25.498337+0000","last_unstale":"2023-10-21T16:20:27.216008+0000","last_undegraded":"2023-10-21T16:20:27.216008+0000","last_fullsized":"2023-10-21T16:20:27.216008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:47:45.030989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688039+0000","last_change":"2023-10-21T16:20:25.511129+0000","last_active":"2023-10-21T16:20:26.688039+0000","last_peered":"2023-10-21T16:20:26.688039+0000","last_clean":"2023-10-21T16:20:26.688039+0000","last_became_active":"2023-10-21T16:20:25.505765+0000","last_became_peered":"2023-10-21T16:20:25.505765+0000","last_unstale":"2023-10-21T16:20:26.688039+0000","last_undegraded":"2023-10-21T16:20:26.688039+0000","last_fullsized":"2023-10-21T16:20:26.688039+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:57:45.358473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218580+0000","last_change":"2023-10-21T16:20:25.517735+0000","last_active":"2023-10-21T16:20:27.218580+0000","last_peered":"2023-10-21T16:20:27.218580+0000","last_clean":"2023-10-21T16:20:27.218580+0000","last_became_active":"2023-10-21T16:20:25.517385+0000","last_became_peered":"2023-10-21T16:20:25.517385+0000","last_unstale":"2023-10-21T16:20:27.218580+0000","last_undegraded":"2023-10-21T16:20:27.218580+0000","last_fullsized":"2023-10-21T16:20:27.218580+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:31.044939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688032+0000","last_change":"2023-10-21T16:20:25.498972+0000","last_active":"2023-10-21T16:20:26.688032+0000","last_peered":"2023-10-21T16:20:26.688032+0000","last_clean":"2023-10-21T16:20:26.688032+0000","last_became_active":"2023-10-21T16:20:25.498670+0000","last_became_peered":"2023-10-21T16:20:25.498670+0000","last_unstale":"2023-10-21T16:20:26.688032+0000","last_undegraded":"2023-10-21T16:20:26.688032+0000","last_fullsized":"2023-10-21T16:20:26.688032+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:59:30.783483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476134+0000","last_change":"2023-10-21T16:20:25.493789+0000","last_active":"2023-10-21T16:20:26.476134+0000","last_peered":"2023-10-21T16:20:26.476134+0000","last_clean":"2023-10-21T16:20:26.476134+0000","last_became_active":"2023-10-21T16:20:25.493529+0000","last_became_peered":"2023-10-21T16:20:25.493529+0000","last_unstale":"2023-10-21T16:20:26.476134+0000","last_undegraded":"2023-10-21T16:20:26.476134+0000","last_fullsized":"2023-10-21T16:20:26.476134+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:47.320111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475650+0000","last_change":"2023-10-21T16:20:25.484414+0000","last_active":"2023-10-21T16:20:26.475650+0000","last_peered":"2023-10-21T16:20:26.475650+0000","last_clean":"2023-10-21T16:20:26.475650+0000","last_became_active":"2023-10-21T16:20:25.483637+0000","last_became_peered":"2023-10-21T16:20:25.483637+0000","last_unstale":"2023-10-21T16:20:26.475650+0000","last_undegraded":"2023-10-21T16:20:26.475650+0000","last_fullsized":"2023-10-21T16:20:26.475650+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:42.388519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216519+0000","last_change":"2023-10-21T16:20:25.509186+0000","last_active":"2023-10-21T16:20:27.216519+0000","last_peered":"2023-10-21T16:20:27.216519+0000","last_clean":"2023-10-21T16:20:27.216519+0000","last_became_active":"2023-10-21T16:20:25.506702+0000","last_became_peered":"2023-10-21T16:20:25.506702+0000","last_unstale":"2023-10-21T16:20:27.216519+0000","last_undegraded":"2023-10-21T16:20:27.216519+0000","last_fullsized":"2023-10-21T16:20:27.216519+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:04.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687451+0000","last_change":"2023-10-21T16:20:25.507235+0000","last_active":"2023-10-21T16:20:26.687451+0000","last_peered":"2023-10-21T16:20:26.687451+0000","last_clean":"2023-10-21T16:20:26.687451+0000","last_became_active":"2023-10-21T16:20:25.506941+0000","last_became_peered":"2023-10-21T16:20:25.506941+0000","last_unstale":"2023-10-21T16:20:26.687451+0000","last_undegraded":"2023-10-21T16:20:26.687451+0000","last_fullsized":"2023-10-21T16:20:26.687451+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:48.852794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216587+0000","last_change":"2023-10-21T16:20:25.510813+0000","last_active":"2023-10-21T16:20:27.216587+0000","last_peered":"2023-10-21T16:20:27.216587+0000","last_clean":"2023-10-21T16:20:27.216587+0000","last_became_active":"2023-10-21T16:20:25.510481+0000","last_became_peered":"2023-10-21T16:20:25.510481+0000","last_unstale":"2023-10-21T16:20:27.216587+0000","last_undegraded":"2023-10-21T16:20:27.216587+0000","last_fullsized":"2023-10-21T16:20:27.216587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:58.161062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476341+0000","last_change":"2023-10-21T16:20:25.496440+0000","last_active":"2023-10-21T16:20:26.476341+0000","last_peered":"2023-10-21T16:20:26.476341+0000","last_clean":"2023-10-21T16:20:26.476341+0000","last_became_active":"2023-10-21T16:20:25.496164+0000","last_became_peered":"2023-10-21T16:20:25.496164+0000","last_unstale":"2023-10-21T16:20:26.476341+0000","last_undegraded":"2023-10-21T16:20:26.476341+0000","last_fullsized":"2023-10-21T16:20:26.476341+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:22.403828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476343+0000","last_change":"2023-10-21T16:20:25.495907+0000","last_active":"2023-10-21T16:20:26.476343+0000","last_peered":"2023-10-21T16:20:26.476343+0000","last_clean":"2023-10-21T16:20:26.476343+0000","last_became_active":"2023-10-21T16:20:25.495590+0000","last_became_peered":"2023-10-21T16:20:25.495590+0000","last_unstale":"2023-10-21T16:20:26.476343+0000","last_undegraded":"2023-10-21T16:20:26.476343+0000","last_fullsized":"2023-10-21T16:20:26.476343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:23:21.854679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898932+0000","last_change":"2023-10-21T16:20:25.496972+0000","last_active":"2023-10-21T16:20:26.898932+0000","last_peered":"2023-10-21T16:20:26.898932+0000","last_clean":"2023-10-21T16:20:26.898932+0000","last_became_active":"2023-10-21T16:20:25.496686+0000","last_became_peered":"2023-10-21T16:20:25.496686+0000","last_unstale":"2023-10-21T16:20:26.898932+0000","last_undegraded":"2023-10-21T16:20:26.898932+0000","last_fullsized":"2023-10-21T16:20:26.898932+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:05:11.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217619+0000","last_change":"2023-10-21T16:20:25.507501+0000","last_active":"2023-10-21T16:20:27.217619+0000","last_peered":"2023-10-21T16:20:27.217619+0000","last_clean":"2023-10-21T16:20:27.217619+0000","last_became_active":"2023-10-21T16:20:25.507075+0000","last_became_peered":"2023-10-21T16:20:25.507075+0000","last_unstale":"2023-10-21T16:20:27.217619+0000","last_undegraded":"2023-10-21T16:20:27.217619+0000","last_fullsized":"2023-10-21T16:20:27.217619+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:21.412057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475705+0000","last_change":"2023-10-21T16:20:25.498301+0000","last_active":"2023-10-21T16:20:26.475705+0000","last_peered":"2023-10-21T16:20:26.475705+0000","last_clean":"2023-10-21T16:20:26.475705+0000","last_became_active":"2023-10-21T16:20:25.497856+0000","last_became_peered":"2023-10-21T16:20:25.497856+0000","last_unstale":"2023-10-21T16:20:26.475705+0000","last_undegraded":"2023-10-21T16:20:26.475705+0000","last_fullsized":"2023-10-21T16:20:26.475705+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:53:02.501762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475736+0000","last_change":"2023-10-21T16:20:25.484075+0000","last_active":"2023-10-21T16:20:26.475736+0000","last_peered":"2023-10-21T16:20:26.475736+0000","last_clean":"2023-10-21T16:20:26.475736+0000","last_became_active":"2023-10-21T16:20:25.483401+0000","last_became_peered":"2023-10-21T16:20:25.483401+0000","last_unstale":"2023-10-21T16:20:26.475736+0000","last_undegraded":"2023-10-21T16:20:26.475736+0000","last_fullsized":"2023-10-21T16:20:26.475736+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:06:09.553455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216701+0000","last_change":"2023-10-21T16:20:25.509626+0000","last_active":"2023-10-21T16:20:27.216701+0000","last_peered":"2023-10-21T16:20:27.216701+0000","last_clean":"2023-10-21T16:20:27.216701+0000","last_became_active":"2023-10-21T16:20:25.509273+0000","last_became_peered":"2023-10-21T16:20:25.509273+0000","last_unstale":"2023-10-21T16:20:27.216701+0000","last_undegraded":"2023-10-21T16:20:27.216701+0000","last_fullsized":"2023-10-21T16:20:27.216701+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:41:06.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475107+0000","last_change":"2023-10-21T16:20:25.488820+0000","last_active":"2023-10-21T16:20:26.475107+0000","last_peered":"2023-10-21T16:20:26.475107+0000","last_clean":"2023-10-21T16:20:26.475107+0000","last_became_active":"2023-10-21T16:20:25.488127+0000","last_became_peered":"2023-10-21T16:20:25.488127+0000","last_unstale":"2023-10-21T16:20:26.475107+0000","last_undegraded":"2023-10-21T16:20:26.475107+0000","last_fullsized":"2023-10-21T16:20:26.475107+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:40.105114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687385+0000","last_change":"2023-10-21T16:20:25.499698+0000","last_active":"2023-10-21T16:20:26.687385+0000","last_peered":"2023-10-21T16:20:26.687385+0000","last_clean":"2023-10-21T16:20:26.687385+0000","last_became_active":"2023-10-21T16:20:25.499370+0000","last_became_peered":"2023-10-21T16:20:25.499370+0000","last_unstale":"2023-10-21T16:20:26.687385+0000","last_undegraded":"2023-10-21T16:20:26.687385+0000","last_fullsized":"2023-10-21T16:20:26.687385+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:07.250301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"132'4","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.471038+0000","last_change":"2023-10-21T16:20:24.479530+0000","last_active":"2023-10-21T16:20:26.471038+0000","last_peered":"2023-10-21T16:20:26.471038+0000","last_clean":"2023-10-21T16:20:26.471038+0000","last_became_active":"2023-10-21T16:20:24.478965+0000","last_became_peered":"2023-10-21T16:20:24.478965+0000","last_unstale":"2023-10-21T16:20:26.471038+0000","last_undegraded":"2023-10-21T16:20:26.471038+0000","last_fullsized":"2023-10-21T16:20:26.471038+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:11.685852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475042+0000","last_change":"2023-10-21T16:20:25.500905+0000","last_active":"2023-10-21T16:20:26.475042+0000","last_peered":"2023-10-21T16:20:26.475042+0000","last_clean":"2023-10-21T16:20:26.475042+0000","last_became_active":"2023-10-21T16:20:25.500172+0000","last_became_peered":"2023-10-21T16:20:25.500172+0000","last_unstale":"2023-10-21T16:20:26.475042+0000","last_undegraded":"2023-10-21T16:20:26.475042+0000","last_fullsized":"2023-10-21T16:20:26.475042+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:38.785025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476149+0000","last_change":"2023-10-21T16:20:24.472971+0000","last_active":"2023-10-21T16:20:26.476149+0000","last_peered":"2023-10-21T16:20:26.476149+0000","last_clean":"2023-10-21T16:20:26.476149+0000","last_became_active":"2023-10-21T16:20:24.472631+0000","last_became_peered":"2023-10-21T16:20:24.472631+0000","last_unstale":"2023-10-21T16:20:26.476149+0000","last_undegraded":"2023-10-21T16:20:26.476149+0000","last_fullsized":"2023-10-21T16:20:26.476149+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:38:13.204439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476016+0000","last_change":"2023-10-21T16:20:25.500448+0000","last_active":"2023-10-21T16:20:26.476016+0000","last_peered":"2023-10-21T16:20:26.476016+0000","last_clean":"2023-10-21T16:20:26.476016+0000","last_became_active":"2023-10-21T16:20:25.500117+0000","last_became_peered":"2023-10-21T16:20:25.500117+0000","last_unstale":"2023-10-21T16:20:26.476016+0000","last_undegraded":"2023-10-21T16:20:26.476016+0000","last_fullsized":"2023-10-21T16:20:26.476016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:25.477807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476150+0000","last_change":"2023-10-21T16:20:24.468190+0000","last_active":"2023-10-21T16:20:26.476150+0000","last_peered":"2023-10-21T16:20:26.476150+0000","last_clean":"2023-10-21T16:20:26.476150+0000","last_became_active":"2023-10-21T16:20:24.467888+0000","last_became_peered":"2023-10-21T16:20:24.467888+0000","last_unstale":"2023-10-21T16:20:26.476150+0000","last_undegraded":"2023-10-21T16:20:26.476150+0000","last_fullsized":"2023-10-21T16:20:26.476150+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:48:09.502229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898547+0000","last_change":"2023-10-21T16:20:25.486183+0000","last_active":"2023-10-21T16:20:26.898547+0000","last_peered":"2023-10-21T16:20:26.898547+0000","last_clean":"2023-10-21T16:20:26.898547+0000","last_became_active":"2023-10-21T16:20:25.485864+0000","last_became_peered":"2023-10-21T16:20:25.485864+0000","last_unstale":"2023-10-21T16:20:26.898547+0000","last_undegraded":"2023-10-21T16:20:26.898547+0000","last_fullsized":"2023-10-21T16:20:26.898547+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:12:29.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217008+0000","last_change":"2023-10-21T16:20:24.465995+0000","last_active":"2023-10-21T16:20:27.217008+0000","last_peered":"2023-10-21T16:20:27.217008+0000","last_clean":"2023-10-21T16:20:27.217008+0000","last_became_active":"2023-10-21T16:20:24.465739+0000","last_became_peered":"2023-10-21T16:20:24.465739+0000","last_unstale":"2023-10-21T16:20:27.217008+0000","last_undegraded":"2023-10-21T16:20:27.217008+0000","last_fullsized":"2023-10-21T16:20:27.217008+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:48:09.502229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470956+0000","last_change":"2023-10-21T16:20:25.517806+0000","last_active":"2023-10-21T16:20:26.470956+0000","last_peered":"2023-10-21T16:20:26.470956+0000","last_clean":"2023-10-21T16:20:26.470956+0000","last_became_active":"2023-10-21T16:20:25.517479+0000","last_became_peered":"2023-10-21T16:20:25.517479+0000","last_unstale":"2023-10-21T16:20:26.470956+0000","last_undegraded":"2023-10-21T16:20:26.470956+0000","last_fullsized":"2023-10-21T16:20:26.470956+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:13.074880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216470+0000","last_change":"2023-10-21T16:20:24.478509+0000","last_active":"2023-10-21T16:20:27.216470+0000","last_peered":"2023-10-21T16:20:27.216470+0000","last_clean":"2023-10-21T16:20:27.216470+0000","last_became_active":"2023-10-21T16:20:24.478216+0000","last_became_peered":"2023-10-21T16:20:24.478216+0000","last_unstale":"2023-10-21T16:20:27.216470+0000","last_undegraded":"2023-10-21T16:20:27.216470+0000","last_fullsized":"2023-10-21T16:20:27.216470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:40.416746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688071+0000","last_change":"2023-10-21T16:20:25.507301+0000","last_active":"2023-10-21T16:20:26.688071+0000","last_peered":"2023-10-21T16:20:26.688071+0000","last_clean":"2023-10-21T16:20:26.688071+0000","last_became_active":"2023-10-21T16:20:25.506762+0000","last_became_peered":"2023-10-21T16:20:25.506762+0000","last_unstale":"2023-10-21T16:20:26.688071+0000","last_undegraded":"2023-10-21T16:20:26.688071+0000","last_fullsized":"2023-10-21T16:20:26.688071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:10.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898247+0000","last_change":"2023-10-21T16:20:24.466729+0000","last_active":"2023-10-21T16:20:26.898247+0000","last_peered":"2023-10-21T16:20:26.898247+0000","last_clean":"2023-10-21T16:20:26.898247+0000","last_became_active":"2023-10-21T16:20:24.466063+0000","last_became_peered":"2023-10-21T16:20:24.466063+0000","last_unstale":"2023-10-21T16:20:26.898247+0000","last_undegraded":"2023-10-21T16:20:26.898247+0000","last_fullsized":"2023-10-21T16:20:26.898247+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:41:04.514523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218742+0000","last_change":"2023-10-21T16:20:25.505369+0000","last_active":"2023-10-21T16:20:27.218742+0000","last_peered":"2023-10-21T16:20:27.218742+0000","last_clean":"2023-10-21T16:20:27.218742+0000","last_became_active":"2023-10-21T16:20:25.503117+0000","last_became_peered":"2023-10-21T16:20:25.503117+0000","last_unstale":"2023-10-21T16:20:27.218742+0000","last_undegraded":"2023-10-21T16:20:27.218742+0000","last_fullsized":"2023-10-21T16:20:27.218742+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:24.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475612+0000","last_change":"2023-10-21T16:20:24.472342+0000","last_active":"2023-10-21T16:20:26.475612+0000","last_peered":"2023-10-21T16:20:26.475612+0000","last_clean":"2023-10-21T16:20:26.475612+0000","last_became_active":"2023-10-21T16:20:24.471859+0000","last_became_peered":"2023-10-21T16:20:24.471859+0000","last_unstale":"2023-10-21T16:20:26.475612+0000","last_undegraded":"2023-10-21T16:20:26.475612+0000","last_fullsized":"2023-10-21T16:20:26.475612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:56:36.656749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470997+0000","last_change":"2023-10-21T16:20:25.502336+0000","last_active":"2023-10-21T16:20:26.470997+0000","last_peered":"2023-10-21T16:20:26.470997+0000","last_clean":"2023-10-21T16:20:26.470997+0000","last_became_active":"2023-10-21T16:20:25.502042+0000","last_became_peered":"2023-10-21T16:20:25.502042+0000","last_unstale":"2023-10-21T16:20:26.470997+0000","last_undegraded":"2023-10-21T16:20:26.470997+0000","last_fullsized":"2023-10-21T16:20:26.470997+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:10.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470251+0000","last_change":"2023-10-21T16:20:24.465690+0000","last_active":"2023-10-21T16:20:26.470251+0000","last_peered":"2023-10-21T16:20:26.470251+0000","last_clean":"2023-10-21T16:20:26.470251+0000","last_became_active":"2023-10-21T16:20:24.465430+0000","last_became_peered":"2023-10-21T16:20:24.465430+0000","last_unstale":"2023-10-21T16:20:26.470251+0000","last_undegraded":"2023-10-21T16:20:26.470251+0000","last_fullsized":"2023-10-21T16:20:26.470251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:45.395322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687224+0000","last_change":"2023-10-21T16:20:25.499573+0000","last_active":"2023-10-21T16:20:26.687224+0000","last_peered":"2023-10-21T16:20:26.687224+0000","last_clean":"2023-10-21T16:20:26.687224+0000","last_became_active":"2023-10-21T16:20:25.497501+0000","last_became_peered":"2023-10-21T16:20:25.497501+0000","last_unstale":"2023-10-21T16:20:26.687224+0000","last_undegraded":"2023-10-21T16:20:26.687224+0000","last_fullsized":"2023-10-21T16:20:26.687224+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:46.410412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476201+0000","last_change":"2023-10-21T16:20:24.467237+0000","last_active":"2023-10-21T16:20:26.476201+0000","last_peered":"2023-10-21T16:20:26.476201+0000","last_clean":"2023-10-21T16:20:26.476201+0000","last_became_active":"2023-10-21T16:20:24.466916+0000","last_became_peered":"2023-10-21T16:20:24.466916+0000","last_unstale":"2023-10-21T16:20:26.476201+0000","last_undegraded":"2023-10-21T16:20:26.476201+0000","last_fullsized":"2023-10-21T16:20:26.476201+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:03.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216554+0000","last_change":"2023-10-21T16:20:25.505147+0000","last_active":"2023-10-21T16:20:27.216554+0000","last_peered":"2023-10-21T16:20:27.216554+0000","last_clean":"2023-10-21T16:20:27.216554+0000","last_became_active":"2023-10-21T16:20:25.503345+0000","last_became_peered":"2023-10-21T16:20:25.503345+0000","last_unstale":"2023-10-21T16:20:27.216554+0000","last_undegraded":"2023-10-21T16:20:27.216554+0000","last_fullsized":"2023-10-21T16:20:27.216554+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:26.459038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688112+0000","last_change":"2023-10-21T16:20:24.475943+0000","last_active":"2023-10-21T16:20:26.688112+0000","last_peered":"2023-10-21T16:20:26.688112+0000","last_clean":"2023-10-21T16:20:26.688112+0000","last_became_active":"2023-10-21T16:20:24.475587+0000","last_became_peered":"2023-10-21T16:20:24.475587+0000","last_unstale":"2023-10-21T16:20:26.688112+0000","last_undegraded":"2023-10-21T16:20:26.688112+0000","last_fullsized":"2023-10-21T16:20:26.688112+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:23.923808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475130+0000","last_change":"2023-10-21T16:20:25.499530+0000","last_active":"2023-10-21T16:20:26.475130+0000","last_peered":"2023-10-21T16:20:26.475130+0000","last_clean":"2023-10-21T16:20:26.475130+0000","last_became_active":"2023-10-21T16:20:25.499255+0000","last_became_peered":"2023-10-21T16:20:25.499255+0000","last_unstale":"2023-10-21T16:20:26.475130+0000","last_undegraded":"2023-10-21T16:20:26.475130+0000","last_fullsized":"2023-10-21T16:20:26.475130+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:50.768357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"132'3","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216675+0000","last_change":"2023-10-21T16:20:24.476536+0000","last_active":"2023-10-21T16:20:27.216675+0000","last_peered":"2023-10-21T16:20:27.216675+0000","last_clean":"2023-10-21T16:20:27.216675+0000","last_became_active":"2023-10-21T16:20:24.475246+0000","last_became_peered":"2023-10-21T16:20:24.475246+0000","last_unstale":"2023-10-21T16:20:27.216675+0000","last_undegraded":"2023-10-21T16:20:27.216675+0000","last_fullsized":"2023-10-21T16:20:27.216675+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:56.266586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217388+0000","last_change":"2023-10-21T16:20:25.506619+0000","last_active":"2023-10-21T16:20:27.217388+0000","last_peered":"2023-10-21T16:20:27.217388+0000","last_clean":"2023-10-21T16:20:27.217388+0000","last_became_active":"2023-10-21T16:20:25.506367+0000","last_became_peered":"2023-10-21T16:20:25.506367+0000","last_unstale":"2023-10-21T16:20:27.217388+0000","last_undegraded":"2023-10-21T16:20:27.217388+0000","last_fullsized":"2023-10-21T16:20:27.217388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:44.775791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"132'6","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217862+0000","last_change":"2023-10-21T16:20:24.478603+0000","last_active":"2023-10-21T16:20:27.217862+0000","last_peered":"2023-10-21T16:20:27.217862+0000","last_clean":"2023-10-21T16:20:27.217862+0000","last_became_active":"2023-10-21T16:20:24.478268+0000","last_became_peered":"2023-10-21T16:20:24.478268+0000","last_unstale":"2023-10-21T16:20:27.217862+0000","last_undegraded":"2023-10-21T16:20:27.217862+0000","last_fullsized":"2023-10-21T16:20:27.217862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:30:01.838259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898318+0000","last_change":"2023-10-21T16:20:25.501373+0000","last_active":"2023-10-21T16:20:26.898318+0000","last_peered":"2023-10-21T16:20:26.898318+0000","last_clean":"2023-10-21T16:20:26.898318+0000","last_became_active":"2023-10-21T16:20:25.501049+0000","last_became_peered":"2023-10-21T16:20:25.501049+0000","last_unstale":"2023-10-21T16:20:26.898318+0000","last_undegraded":"2023-10-21T16:20:26.898318+0000","last_fullsized":"2023-10-21T16:20:26.898318+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:39.322952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217557+0000","last_change":"2023-10-21T16:20:24.470854+0000","last_active":"2023-10-21T16:20:27.217557+0000","last_peered":"2023-10-21T16:20:27.217557+0000","last_clean":"2023-10-21T16:20:27.217557+0000","last_became_active":"2023-10-21T16:20:24.470586+0000","last_became_peered":"2023-10-21T16:20:24.470586+0000","last_unstale":"2023-10-21T16:20:27.217557+0000","last_undegraded":"2023-10-21T16:20:27.217557+0000","last_fullsized":"2023-10-21T16:20:27.217557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:56:36.656749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218402+0000","last_change":"2023-10-21T16:20:25.506635+0000","last_active":"2023-10-21T16:20:27.218402+0000","last_peered":"2023-10-21T16:20:27.218402+0000","last_clean":"2023-10-21T16:20:27.218402+0000","last_became_active":"2023-10-21T16:20:25.506075+0000","last_became_peered":"2023-10-21T16:20:25.506075+0000","last_unstale":"2023-10-21T16:20:27.218402+0000","last_undegraded":"2023-10-21T16:20:27.218402+0000","last_fullsized":"2023-10-21T16:20:27.218402+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:41:28.587131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470472+0000","last_change":"2023-10-21T16:20:24.474488+0000","last_active":"2023-10-21T16:20:26.470472+0000","last_peered":"2023-10-21T16:20:26.470472+0000","last_clean":"2023-10-21T16:20:26.470472+0000","last_became_active":"2023-10-21T16:20:24.474174+0000","last_became_peered":"2023-10-21T16:20:24.474174+0000","last_unstale":"2023-10-21T16:20:26.470472+0000","last_undegraded":"2023-10-21T16:20:26.470472+0000","last_fullsized":"2023-10-21T16:20:26.470472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:06.235211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476100+0000","last_change":"2023-10-21T16:20:25.487714+0000","last_active":"2023-10-21T16:20:26.476100+0000","last_peered":"2023-10-21T16:20:26.476100+0000","last_clean":"2023-10-21T16:20:26.476100+0000","last_became_active":"2023-10-21T16:20:25.486824+0000","last_became_peered":"2023-10-21T16:20:25.486824+0000","last_unstale":"2023-10-21T16:20:26.476100+0000","last_undegraded":"2023-10-21T16:20:26.476100+0000","last_fullsized":"2023-10-21T16:20:26.476100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:53.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474944+0000","last_change":"2023-10-21T16:20:24.464774+0000","last_active":"2023-10-21T16:20:26.474944+0000","last_peered":"2023-10-21T16:20:26.474944+0000","last_clean":"2023-10-21T16:20:26.474944+0000","last_became_active":"2023-10-21T16:20:24.464508+0000","last_became_peered":"2023-10-21T16:20:24.464508+0000","last_unstale":"2023-10-21T16:20:26.474944+0000","last_undegraded":"2023-10-21T16:20:26.474944+0000","last_fullsized":"2023-10-21T16:20:26.474944+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:18:02.012324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688061+0000","last_change":"2023-10-21T16:20:25.512562+0000","last_active":"2023-10-21T16:20:26.688061+0000","last_peered":"2023-10-21T16:20:26.688061+0000","last_clean":"2023-10-21T16:20:26.688061+0000","last_became_active":"2023-10-21T16:20:25.510596+0000","last_became_peered":"2023-10-21T16:20:25.510596+0000","last_unstale":"2023-10-21T16:20:26.688061+0000","last_undegraded":"2023-10-21T16:20:26.688061+0000","last_fullsized":"2023-10-21T16:20:26.688061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:13.074880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"132'8","reported_seq":20,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475009+0000","last_change":"2023-10-21T16:20:24.467144+0000","last_active":"2023-10-21T16:20:26.475009+0000","last_peered":"2023-10-21T16:20:26.475009+0000","last_clean":"2023-10-21T16:20:26.475009+0000","last_became_active":"2023-10-21T16:20:24.466481+0000","last_became_peered":"2023-10-21T16:20:24.466481+0000","last_unstale":"2023-10-21T16:20:26.475009+0000","last_undegraded":"2023-10-21T16:20:26.475009+0000","last_fullsized":"2023-10-21T16:20:26.475009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:24.892214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475137+0000","last_change":"2023-10-21T16:20:25.498590+0000","last_active":"2023-10-21T16:20:26.475137+0000","last_peered":"2023-10-21T16:20:26.475137+0000","last_clean":"2023-10-21T16:20:26.475137+0000","last_became_active":"2023-10-21T16:20:25.498314+0000","last_became_peered":"2023-10-21T16:20:25.498314+0000","last_unstale":"2023-10-21T16:20:26.475137+0000","last_undegraded":"2023-10-21T16:20:26.475137+0000","last_fullsized":"2023-10-21T16:20:26.475137+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:17.536558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218371+0000","last_change":"2023-10-21T16:20:24.478671+0000","last_active":"2023-10-21T16:20:27.218371+0000","last_peered":"2023-10-21T16:20:27.218371+0000","last_clean":"2023-10-21T16:20:27.218371+0000","last_became_active":"2023-10-21T16:20:24.477880+0000","last_became_peered":"2023-10-21T16:20:24.477880+0000","last_unstale":"2023-10-21T16:20:27.218371+0000","last_undegraded":"2023-10-21T16:20:27.218371+0000","last_fullsized":"2023-10-21T16:20:27.218371+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:05.591448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217179+0000","last_change":"2023-10-21T16:20:25.512409+0000","last_active":"2023-10-21T16:20:27.217179+0000","last_peered":"2023-10-21T16:20:27.217179+0000","last_clean":"2023-10-21T16:20:27.217179+0000","last_became_active":"2023-10-21T16:20:25.512151+0000","last_became_peered":"2023-10-21T16:20:25.512151+0000","last_unstale":"2023-10-21T16:20:27.217179+0000","last_undegraded":"2023-10-21T16:20:27.217179+0000","last_fullsized":"2023-10-21T16:20:27.217179+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:10:43.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216667+0000","last_change":"2023-10-21T16:20:24.472620+0000","last_active":"2023-10-21T16:20:27.216667+0000","last_peered":"2023-10-21T16:20:27.216667+0000","last_clean":"2023-10-21T16:20:27.216667+0000","last_became_active":"2023-10-21T16:20:24.472150+0000","last_became_peered":"2023-10-21T16:20:24.472150+0000","last_unstale":"2023-10-21T16:20:27.216667+0000","last_undegraded":"2023-10-21T16:20:27.216667+0000","last_fullsized":"2023-10-21T16:20:27.216667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:57:02.930510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217388+0000","last_change":"2023-10-21T16:20:25.506150+0000","last_active":"2023-10-21T16:20:27.217388+0000","last_peered":"2023-10-21T16:20:27.217388+0000","last_clean":"2023-10-21T16:20:27.217388+0000","last_became_active":"2023-10-21T16:20:25.502809+0000","last_became_peered":"2023-10-21T16:20:25.502809+0000","last_unstale":"2023-10-21T16:20:27.217388+0000","last_undegraded":"2023-10-21T16:20:27.217388+0000","last_fullsized":"2023-10-21T16:20:27.217388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:06.124993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898527+0000","last_change":"2023-10-21T16:20:24.476385+0000","last_active":"2023-10-21T16:20:26.898527+0000","last_peered":"2023-10-21T16:20:26.898527+0000","last_clean":"2023-10-21T16:20:26.898527+0000","last_became_active":"2023-10-21T16:20:24.475886+0000","last_became_peered":"2023-10-21T16:20:24.475886+0000","last_unstale":"2023-10-21T16:20:26.898527+0000","last_undegraded":"2023-10-21T16:20:26.898527+0000","last_fullsized":"2023-10-21T16:20:26.898527+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:08.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687644+0000","last_change":"2023-10-21T16:20:25.508317+0000","last_active":"2023-10-21T16:20:26.687644+0000","last_peered":"2023-10-21T16:20:26.687644+0000","last_clean":"2023-10-21T16:20:26.687644+0000","last_became_active":"2023-10-21T16:20:25.507983+0000","last_became_peered":"2023-10-21T16:20:25.507983+0000","last_unstale":"2023-10-21T16:20:26.687644+0000","last_undegraded":"2023-10-21T16:20:26.687644+0000","last_fullsized":"2023-10-21T16:20:26.687644+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:53.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475944+0000","last_change":"2023-10-21T16:20:24.466299+0000","last_active":"2023-10-21T16:20:26.475944+0000","last_peered":"2023-10-21T16:20:26.475944+0000","last_clean":"2023-10-21T16:20:26.475944+0000","last_became_active":"2023-10-21T16:20:24.466030+0000","last_became_peered":"2023-10-21T16:20:24.466030+0000","last_unstale":"2023-10-21T16:20:26.475944+0000","last_undegraded":"2023-10-21T16:20:26.475944+0000","last_fullsized":"2023-10-21T16:20:26.475944+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:47.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687912+0000","last_change":"2023-10-21T16:20:25.518821+0000","last_active":"2023-10-21T16:20:26.687912+0000","last_peered":"2023-10-21T16:20:26.687912+0000","last_clean":"2023-10-21T16:20:26.687912+0000","last_became_active":"2023-10-21T16:20:25.518515+0000","last_became_peered":"2023-10-21T16:20:25.518515+0000","last_unstale":"2023-10-21T16:20:26.687912+0000","last_undegraded":"2023-10-21T16:20:26.687912+0000","last_fullsized":"2023-10-21T16:20:26.687912+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:28.356874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474919+0000","last_change":"2023-10-21T16:20:24.465762+0000","last_active":"2023-10-21T16:20:26.474919+0000","last_peered":"2023-10-21T16:20:26.474919+0000","last_clean":"2023-10-21T16:20:26.474919+0000","last_became_active":"2023-10-21T16:20:24.463813+0000","last_became_peered":"2023-10-21T16:20:24.463813+0000","last_unstale":"2023-10-21T16:20:26.474919+0000","last_undegraded":"2023-10-21T16:20:26.474919+0000","last_fullsized":"2023-10-21T16:20:26.474919+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:14.453744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470917+0000","last_change":"2023-10-21T16:20:25.498461+0000","last_active":"2023-10-21T16:20:26.470917+0000","last_peered":"2023-10-21T16:20:26.470917+0000","last_clean":"2023-10-21T16:20:26.470917+0000","last_became_active":"2023-10-21T16:20:25.498168+0000","last_became_peered":"2023-10-21T16:20:25.498168+0000","last_unstale":"2023-10-21T16:20:26.470917+0000","last_undegraded":"2023-10-21T16:20:26.470917+0000","last_fullsized":"2023-10-21T16:20:26.470917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:57:45.358473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216796+0000","last_change":"2023-10-21T16:20:24.473657+0000","last_active":"2023-10-21T16:20:27.216796+0000","last_peered":"2023-10-21T16:20:27.216796+0000","last_clean":"2023-10-21T16:20:27.216796+0000","last_became_active":"2023-10-21T16:20:24.473117+0000","last_became_peered":"2023-10-21T16:20:24.473117+0000","last_unstale":"2023-10-21T16:20:27.216796+0000","last_undegraded":"2023-10-21T16:20:27.216796+0000","last_fullsized":"2023-10-21T16:20:27.216796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:27:14.327872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476087+0000","last_change":"2023-10-21T16:20:25.513788+0000","last_active":"2023-10-21T16:20:26.476087+0000","last_peered":"2023-10-21T16:20:26.476087+0000","last_clean":"2023-10-21T16:20:26.476087+0000","last_became_active":"2023-10-21T16:20:25.512766+0000","last_became_peered":"2023-10-21T16:20:25.512766+0000","last_unstale":"2023-10-21T16:20:26.476087+0000","last_undegraded":"2023-10-21T16:20:26.476087+0000","last_fullsized":"2023-10-21T16:20:26.476087+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:43.633206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475257+0000","last_change":"2023-10-21T16:20:24.464242+0000","last_active":"2023-10-21T16:20:26.475257+0000","last_peered":"2023-10-21T16:20:26.475257+0000","last_clean":"2023-10-21T16:20:26.475257+0000","last_became_active":"2023-10-21T16:20:24.463002+0000","last_became_peered":"2023-10-21T16:20:24.463002+0000","last_unstale":"2023-10-21T16:20:26.475257+0000","last_undegraded":"2023-10-21T16:20:26.475257+0000","last_fullsized":"2023-10-21T16:20:26.475257+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:07:59.125342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687448+0000","last_change":"2023-10-21T16:20:25.507311+0000","last_active":"2023-10-21T16:20:26.687448+0000","last_peered":"2023-10-21T16:20:26.687448+0000","last_clean":"2023-10-21T16:20:26.687448+0000","last_became_active":"2023-10-21T16:20:25.506899+0000","last_became_peered":"2023-10-21T16:20:25.506899+0000","last_unstale":"2023-10-21T16:20:26.687448+0000","last_undegraded":"2023-10-21T16:20:26.687448+0000","last_fullsized":"2023-10-21T16:20:26.687448+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:15.940135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217290+0000","last_change":"2023-10-21T16:20:24.464654+0000","last_active":"2023-10-21T16:20:27.217290+0000","last_peered":"2023-10-21T16:20:27.217290+0000","last_clean":"2023-10-21T16:20:27.217290+0000","last_became_active":"2023-10-21T16:20:24.464381+0000","last_became_peered":"2023-10-21T16:20:24.464381+0000","last_unstale":"2023-10-21T16:20:27.217290+0000","last_undegraded":"2023-10-21T16:20:27.217290+0000","last_fullsized":"2023-10-21T16:20:27.217290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:24.892214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475327+0000","last_change":"2023-10-21T16:20:25.493924+0000","last_active":"2023-10-21T16:20:26.475327+0000","last_peered":"2023-10-21T16:20:26.475327+0000","last_clean":"2023-10-21T16:20:26.475327+0000","last_became_active":"2023-10-21T16:20:25.493634+0000","last_became_peered":"2023-10-21T16:20:25.493634+0000","last_unstale":"2023-10-21T16:20:26.475327+0000","last_undegraded":"2023-10-21T16:20:26.475327+0000","last_fullsized":"2023-10-21T16:20:26.475327+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:44.775791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687845+0000","last_change":"2023-10-21T16:20:24.470748+0000","last_active":"2023-10-21T16:20:26.687845+0000","last_peered":"2023-10-21T16:20:26.687845+0000","last_clean":"2023-10-21T16:20:26.687845+0000","last_became_active":"2023-10-21T16:20:24.469923+0000","last_became_peered":"2023-10-21T16:20:24.469923+0000","last_unstale":"2023-10-21T16:20:26.687845+0000","last_undegraded":"2023-10-21T16:20:26.687845+0000","last_fullsized":"2023-10-21T16:20:26.687845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:08.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470602+0000","last_change":"2023-10-21T16:20:25.498789+0000","last_active":"2023-10-21T16:20:26.470602+0000","last_peered":"2023-10-21T16:20:26.470602+0000","last_clean":"2023-10-21T16:20:26.470602+0000","last_became_active":"2023-10-21T16:20:25.498509+0000","last_became_peered":"2023-10-21T16:20:25.498509+0000","last_unstale":"2023-10-21T16:20:26.470602+0000","last_undegraded":"2023-10-21T16:20:26.470602+0000","last_fullsized":"2023-10-21T16:20:26.470602+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:11:40.223441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218316+0000","last_change":"2023-10-21T16:20:24.478437+0000","last_active":"2023-10-21T16:20:27.218316+0000","last_peered":"2023-10-21T16:20:27.218316+0000","last_clean":"2023-10-21T16:20:27.218316+0000","last_became_active":"2023-10-21T16:20:24.478035+0000","last_became_peered":"2023-10-21T16:20:24.478035+0000","last_unstale":"2023-10-21T16:20:27.218316+0000","last_undegraded":"2023-10-21T16:20:27.218316+0000","last_fullsized":"2023-10-21T16:20:27.218316+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:35:37.750301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898529+0000","last_change":"2023-10-21T16:20:25.515312+0000","last_active":"2023-10-21T16:20:26.898529+0000","last_peered":"2023-10-21T16:20:26.898529+0000","last_clean":"2023-10-21T16:20:26.898529+0000","last_became_active":"2023-10-21T16:20:25.515103+0000","last_became_peered":"2023-10-21T16:20:25.515103+0000","last_unstale":"2023-10-21T16:20:26.898529+0000","last_undegraded":"2023-10-21T16:20:26.898529+0000","last_fullsized":"2023-10-21T16:20:26.898529+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:02.382807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217208+0000","last_change":"2023-10-21T16:20:24.478381+0000","last_active":"2023-10-21T16:20:27.217208+0000","last_peered":"2023-10-21T16:20:27.217208+0000","last_clean":"2023-10-21T16:20:27.217208+0000","last_became_active":"2023-10-21T16:20:24.478136+0000","last_became_peered":"2023-10-21T16:20:24.478136+0000","last_unstale":"2023-10-21T16:20:27.217208+0000","last_undegraded":"2023-10-21T16:20:27.217208+0000","last_fullsized":"2023-10-21T16:20:27.217208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:37:23.526235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475714+0000","last_change":"2023-10-21T16:20:25.515386+0000","last_active":"2023-10-21T16:20:26.475714+0000","last_peered":"2023-10-21T16:20:26.475714+0000","last_clean":"2023-10-21T16:20:26.475714+0000","last_became_active":"2023-10-21T16:20:25.515102+0000","last_became_peered":"2023-10-21T16:20:25.515102+0000","last_unstale":"2023-10-21T16:20:26.475714+0000","last_undegraded":"2023-10-21T16:20:26.475714+0000","last_fullsized":"2023-10-21T16:20:26.475714+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:39:36.293976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476484+0000","last_change":"2023-10-21T16:20:24.475751+0000","last_active":"2023-10-21T16:20:26.476484+0000","last_peered":"2023-10-21T16:20:26.476484+0000","last_clean":"2023-10-21T16:20:26.476484+0000","last_became_active":"2023-10-21T16:20:24.475440+0000","last_became_peered":"2023-10-21T16:20:24.475440+0000","last_unstale":"2023-10-21T16:20:26.476484+0000","last_undegraded":"2023-10-21T16:20:26.476484+0000","last_fullsized":"2023-10-21T16:20:26.476484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:57.145972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475845+0000","last_change":"2023-10-21T16:20:25.516301+0000","last_active":"2023-10-21T16:20:26.475845+0000","last_peered":"2023-10-21T16:20:26.475845+0000","last_clean":"2023-10-21T16:20:26.475845+0000","last_became_active":"2023-10-21T16:20:25.516030+0000","last_became_peered":"2023-10-21T16:20:25.516030+0000","last_unstale":"2023-10-21T16:20:26.475845+0000","last_undegraded":"2023-10-21T16:20:26.475845+0000","last_fullsized":"2023-10-21T16:20:26.475845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:52.244581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475391+0000","last_change":"2023-10-21T16:20:24.462666+0000","last_active":"2023-10-21T16:20:26.475391+0000","last_peered":"2023-10-21T16:20:26.475391+0000","last_clean":"2023-10-21T16:20:26.475391+0000","last_became_active":"2023-10-21T16:20:24.462203+0000","last_became_peered":"2023-10-21T16:20:24.462203+0000","last_unstale":"2023-10-21T16:20:26.475391+0000","last_undegraded":"2023-10-21T16:20:26.475391+0000","last_fullsized":"2023-10-21T16:20:26.475391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:11:59.396608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216990+0000","last_change":"2023-10-21T16:20:25.505460+0000","last_active":"2023-10-21T16:20:27.216990+0000","last_peered":"2023-10-21T16:20:27.216990+0000","last_clean":"2023-10-21T16:20:27.216990+0000","last_became_active":"2023-10-21T16:20:25.504928+0000","last_became_peered":"2023-10-21T16:20:25.504928+0000","last_unstale":"2023-10-21T16:20:27.216990+0000","last_undegraded":"2023-10-21T16:20:27.216990+0000","last_fullsized":"2023-10-21T16:20:27.216990+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.938898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216904+0000","last_change":"2023-10-21T16:20:24.479954+0000","last_active":"2023-10-21T16:20:27.216904+0000","last_peered":"2023-10-21T16:20:27.216904+0000","last_clean":"2023-10-21T16:20:27.216904+0000","last_became_active":"2023-10-21T16:20:24.477790+0000","last_became_peered":"2023-10-21T16:20:24.477790+0000","last_unstale":"2023-10-21T16:20:27.216904+0000","last_undegraded":"2023-10-21T16:20:27.216904+0000","last_fullsized":"2023-10-21T16:20:27.216904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:13.981341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.899233+0000","last_change":"2023-10-21T16:20:25.497087+0000","last_active":"2023-10-21T16:20:26.899233+0000","last_peered":"2023-10-21T16:20:26.899233+0000","last_clean":"2023-10-21T16:20:26.899233+0000","last_became_active":"2023-10-21T16:20:25.496786+0000","last_became_peered":"2023-10-21T16:20:25.496786+0000","last_unstale":"2023-10-21T16:20:26.899233+0000","last_undegraded":"2023-10-21T16:20:26.899233+0000","last_fullsized":"2023-10-21T16:20:26.899233+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:46.410412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475403+0000","last_change":"2023-10-21T16:20:24.459583+0000","last_active":"2023-10-21T16:20:26.475403+0000","last_peered":"2023-10-21T16:20:26.475403+0000","last_clean":"2023-10-21T16:20:26.475403+0000","last_became_active":"2023-10-21T16:20:24.459220+0000","last_became_peered":"2023-10-21T16:20:24.459220+0000","last_unstale":"2023-10-21T16:20:26.475403+0000","last_undegraded":"2023-10-21T16:20:26.475403+0000","last_fullsized":"2023-10-21T16:20:26.475403+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:24:27.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898510+0000","last_change":"2023-10-21T16:20:25.497114+0000","last_active":"2023-10-21T16:20:26.898510+0000","last_peered":"2023-10-21T16:20:26.898510+0000","last_clean":"2023-10-21T16:20:26.898510+0000","last_became_active":"2023-10-21T16:20:25.496759+0000","last_became_peered":"2023-10-21T16:20:25.496759+0000","last_unstale":"2023-10-21T16:20:26.898510+0000","last_undegraded":"2023-10-21T16:20:26.898510+0000","last_fullsized":"2023-10-21T16:20:26.898510+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:10.153223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475888+0000","last_change":"2023-10-21T16:20:24.465095+0000","last_active":"2023-10-21T16:20:26.475888+0000","last_peered":"2023-10-21T16:20:26.475888+0000","last_clean":"2023-10-21T16:20:26.475888+0000","last_became_active":"2023-10-21T16:20:24.464805+0000","last_became_peered":"2023-10-21T16:20:24.464805+0000","last_unstale":"2023-10-21T16:20:26.475888+0000","last_undegraded":"2023-10-21T16:20:26.475888+0000","last_fullsized":"2023-10-21T16:20:26.475888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:46.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"122'4730","reported_seq":5169,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-21T16:20:25.489879+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:20:25.489879+0000","last_peered":"2023-10-21T16:20:25.489879+0000","last_clean":"2023-10-21T16:20:25.489879+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:20:25.489879+0000","last_undegraded":"2023-10-21T16:20:25.489879+0000","last_fullsized":"2023-10-21T16:20:25.489879+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":327,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":327,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4730,"ondisk_log_size":4730,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63908,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307932,"statfs":{"total":96636764160,"available":96571322368,"internally_reserved":0,"allocated":2248704,"data_stored":1425414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0620000000000001,"15min":1.137},"min":{"1min":0.70699999999999996,"5min":0.59799999999999998,"15min":0.51300000000000001},"max":{"1min":1.758,"5min":1.758,"15min":1.8400000000000001},"last":1.669},{"interface":"front","average":{"1min":1.238,"5min":1.1539999999999999,"15min":1.1759999999999999},"min":{"1min":0.95899999999999996,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.7270000000000001,"5min":1.883,"15min":2.448},"last":0.97699999999999998}]},{"osd":1,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.143,"15min":1.163},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.665,"5min":1.851,"15min":1.915},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1850000000000001,"15min":1.23},"min":{"1min":0.627,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":1.2370000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.252,"15min":1.296},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.694,"5min":1.782,"15min":2.085},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2709999999999999,"15min":1.294},"min":{"1min":0.80000000000000004,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.7090000000000001,"5min":1.8169999999999999,"15min":2.6070000000000002},"last":1.079}]},{"osd":3,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.194,"15min":1.2490000000000001},"min":{"1min":0.74199999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.431,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3819999999999999,"15min":1.421},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6359999999999999,"5min":1.903,"15min":1.9750000000000001},"last":1.6140000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.327,"15min":1.347},"min":{"1min":0.76200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.839,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.284},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4159999999999999,"15min":1.4390000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":1.915,"15min":2.355},"last":1.3120000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4279999999999999,"15min":1.393},"min":{"1min":0.84699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6879999999999999,"5min":1.911,"15min":1.9299999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4430000000000001,"15min":1.4419999999999999},"min":{"1min":0.99399999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9259999999999999,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.1839999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.5389999999999999,"15min":1.5449999999999999},"min":{"1min":1.159,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.748,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4590000000000001,"15min":1.458},"min":{"1min":1.0209999999999999,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":1.5940000000000001,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36956,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334884,"statfs":{"total":96636764160,"available":96598921216,"internally_reserved":0,"allocated":1654784,"data_stored":830438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.69599999999999995,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.722,"5min":1.869,"15min":1.869},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1739999999999999,"15min":1.21},"min":{"1min":0.82599999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7789999999999999,"5min":1.839,"15min":1.895},"last":1.861}]},{"osd":1,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.143,"15min":1.1899999999999999},"min":{"1min":0.81799999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.508,"5min":1.72,"15min":2.2309999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.2230000000000001,"15min":1.292},"min":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.538,"5min":2.2349999999999999,"15min":2.2669999999999999},"last":1.1970000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.89400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.895,"15min":2.5339999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.232,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.706,"5min":1.821,"15min":1.821},"last":1.4319999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3320000000000001,"15min":1.3480000000000001},"min":{"1min":0.97199999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8919999999999999,"5min":2.0299999999999998,"15min":2.2000000000000002},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.405,"15min":1.423},"min":{"1min":1.2370000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.8620000000000001,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.831}]},{"osd":4,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.389,"15min":1.3500000000000001},"min":{"1min":0.95799999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.681,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4370000000000001,"15min":1.3899999999999999},"min":{"1min":0.89800000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.004,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.6359999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5369999999999999,"15min":1.502},"min":{"1min":1.135,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":2.1400000000000001,"5min":2.246,"15min":2.246},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3580000000000001,"15min":1.347},"min":{"1min":0.871,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.944,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.5900000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.518,"15min":1.502},"min":{"1min":1.0669999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9830000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6399999999999999,"15min":1.6120000000000001},"min":{"1min":1.0349999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.756}]}]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1634304,"data_stored":828222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.609,"5min":1.609,"15min":1.742},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.1359999999999999,"15min":1.1890000000000001},"min":{"1min":0.70099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.45}]},{"osd":1,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.204,"15min":1.2569999999999999},"min":{"1min":0.81799999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2270000000000001,"15min":1.2370000000000001},"min":{"1min":0.745,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":4.2869999999999999,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":1.3899999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2629999999999999,"15min":1.298},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.4359999999999999,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.238},"min":{"1min":0.53100000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.748}]},{"osd":3,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.302,"15min":1.3169999999999999},"min":{"1min":0.82799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.133},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.353,"15min":1.389},"min":{"1min":0.873,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.206}]},{"osd":4,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.256,"15min":1.2170000000000001},"min":{"1min":0.747,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.377,"15min":1.3520000000000001},"min":{"1min":0.93999999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":4.9100000000000001,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.542}]},{"osd":6,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.484,"15min":1.452},"min":{"1min":1.105,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.583},{"interface":"front","average":{"1min":1.573,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.026,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.992}]},{"osd":7,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":1.109,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.706},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.577,"15min":1.554},"min":{"1min":1.095,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":5.1449999999999996,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.643}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1654784,"data_stored":834978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1990000000000001,"15min":1.1619999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.647,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.2130000000000001,"15min":1.214},"min":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5960000000000001,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":1.22}]},{"osd":1,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.248,"15min":1.266},"min":{"1min":0.83399999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.6779999999999999,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.754},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.264},"min":{"1min":0.83699999999999997,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.681,"5min":7.484,"15min":7.484},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.276,"15min":1.2649999999999999},"min":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6850000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.302,"15min":1.337},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.746,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.581}]},{"osd":3,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3420000000000001,"15min":1.327},"min":{"1min":0.84899999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9430000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4359999999999999,"15min":1.393},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.466}]},{"osd":5,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3540000000000001,"15min":1.304},"min":{"1min":0.78300000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9990000000000001,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.411},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":0.86199999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.855,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.0489999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.532,"15min":1.4259999999999999},"min":{"1min":0.84999999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7509999999999999,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.087},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.617,"15min":1.577},"min":{"1min":1.099,"5min":0.878,"15min":0.878},"max":{"1min":2.1240000000000001,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.673}]},{"osd":7,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.653,"15min":1.522},"min":{"1min":1.0109999999999999,"5min":0.81000000000000005,"15min":0.69599999999999995},"max":{"1min":2.0310000000000001,"5min":12.458,"15min":12.458},"last":1.278},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.6679999999999999,"15min":1.5629999999999999},"min":{"1min":1.117,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.9299999999999999,"5min":3.968,"15min":3.968},"last":1.7}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41120,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330720,"statfs":{"total":96636764160,"available":96594657280,"internally_reserved":0,"allocated":1650688,"data_stored":835468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.92500000000000004,"15min":0.94999999999999996},"min":{"1min":0.48099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.236,"5min":1.946,"15min":1.946},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.212,"5min":1.0940000000000001,"15min":1.1779999999999999},"min":{"1min":0.73599999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6220000000000001,"5min":1.8480000000000001,"15min":2.04},"last":1.123}]},{"osd":1,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93100000000000005,"15min":0.98399999999999999},"min":{"1min":0.68000000000000005,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.589,"5min":1.589,"15min":2.173},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.163,"15min":1.252},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.698,"5min":1.752,"15min":1.8100000000000001},"last":1.226}]},{"osd":2,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.997,"15min":1.056},"min":{"1min":0.64100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.669,"5min":1.669,"15min":2.0049999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.163,"15min":1.216},"min":{"1min":0.91000000000000003,"5min":0.61899999999999999,"15min":0.46600000000000003},"max":{"1min":1.6659999999999999,"5min":1.7250000000000001,"15min":2.847},"last":1.069}]},{"osd":4,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.3320000000000001,"15min":1.399},"min":{"1min":0.77800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8700000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.548},{"interface":"front","average":{"1min":1.494,"5min":1.3620000000000001,"15min":1.3979999999999999},"min":{"1min":1.2170000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8080000000000001,"5min":1.863,"15min":2.919},"last":1.1839999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.363,"15min":1.405},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.7629999999999999,"5min":2.02,"15min":2.02},"last":1.577},{"interface":"front","average":{"1min":1.474,"5min":1.4410000000000001,"15min":1.4910000000000001},"min":{"1min":0.91100000000000003,"5min":0.86299999999999999,"15min":0.75700000000000001},"max":{"1min":1.9279999999999999,"5min":2.0390000000000001,"15min":2.423},"last":1.446}]},{"osd":6,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":1.1819999999999999,"5min":0.84399999999999997,"15min":0.59299999999999997},"max":{"1min":1.829,"5min":2.1000000000000001,"15min":2.6179999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4610000000000001,"15min":1.5189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8999999999999999,"5min":2.1459999999999999,"15min":2.2290000000000001},"last":1.4830000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.516,"15min":1.5820000000000001},"min":{"1min":0.94799999999999995,"5min":0.89700000000000002,"15min":0.83599999999999997},"max":{"1min":1.881,"5min":2.302,"15min":2.3759999999999999},"last":1.341},{"interface":"front","average":{"1min":1.595,"5min":1.544,"15min":1.5980000000000001},"min":{"1min":1.171,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9570000000000001,"5min":2.2050000000000001,"15min":2.7989999999999999},"last":1.3049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46684,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325156,"statfs":{"total":96636764160,"available":96588959744,"internally_reserved":0,"allocated":1650688,"data_stored":834920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.85499999999999998},"min":{"1min":0.47299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.2130000000000001,"5min":1.859,"15min":1.859},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.016,"5min":0.90800000000000003,"15min":0.98299999999999998},"min":{"1min":0.66200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.667,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4419999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0109999999999999,"15min":1.111},"min":{"1min":0.52900000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":1.9390000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.0960000000000001,"15min":1.2509999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5640000000000001,"5min":1.881,"15min":1.881},"last":1.536}]},{"osd":3,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.0309999999999999,"15min":1.1339999999999999},"min":{"1min":0.68200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.8839999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2470000000000001,"15min":1.3640000000000001},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8049999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.4990000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.85799999999999998,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.891,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.341},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.341,"15min":1.375},"min":{"1min":0.92800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.2570000000000001,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":2.278}]},{"osd":5,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2909999999999999,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.7769999999999999,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4650000000000001,"15min":1.5189999999999999},"min":{"1min":0.81299999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1280000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.595}]},{"osd":6,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.458,"15min":1.484},"min":{"1min":1.1000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.444,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":2.3639999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.3759999999999999,"15min":1.4399999999999999},"min":{"1min":1.0269999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.2669999999999999,"5min":2.552,"15min":2.552},"last":1.3899999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.4690000000000001,"15min":1.54},"min":{"1min":1.218,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1880000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":2.2130000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4330000000000001,"15min":1.512},"min":{"1min":0.90200000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.569}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1638400,"data_stored":828256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86099999999999999,"15min":0.90300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.905},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.03,"15min":1.075},"min":{"1min":0.55100000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.601,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":1.1950000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94499999999999995,"15min":0.998},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.429,"5min":2.222,"15min":2.2639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1180000000000001,"15min":1.171},"min":{"1min":0.69899999999999995,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":1.5589999999999999,"5min":1.6910000000000001,"15min":1.974},"last":1.1399999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.21,"15min":1.298},"min":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.623,"5min":1.911,"15min":1.9510000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.212},"min":{"1min":0.55100000000000005,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.6879999999999999,"5min":1.7889999999999999,"15min":1.9159999999999999},"last":1.466}]},{"osd":4,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.242,"15min":1.278},"min":{"1min":1.0680000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.6870000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.28},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.421},"min":{"1min":0.83499999999999996,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.907,"5min":2.145,"15min":2.2719999999999998},"last":1.647}]},{"osd":5,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.429,"15min":1.484},"min":{"1min":1.1000000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8500000000000001,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":1.679},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4159999999999999,"15min":1.466},"min":{"1min":0.82199999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.889,"5min":2.077,"15min":2.0880000000000001},"last":1.621}]},{"osd":6,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.4590000000000001,"15min":1.494},"min":{"1min":0.96499999999999997,"5min":0.749,"15min":0.68500000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.4079999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.411,"15min":1.458},"min":{"1min":1.01,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.9830000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4079999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.55,"15min":1.6020000000000001},"min":{"1min":0.88500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9319999999999999,"5min":2.2530000000000001,"15min":2.3929999999999998},"last":1.804},{"interface":"front","average":{"1min":1.629,"5min":1.5209999999999999,"15min":1.5920000000000001},"min":{"1min":1.119,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0489999999999999,"5min":2.1850000000000001,"15min":2.367},"last":1.573}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41712,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330128,"statfs":{"total":96636764160,"available":96594051072,"internally_reserved":0,"allocated":2256896,"data_stored":1426956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":1.302,"5min":1.649,"15min":1.855},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.996,"15min":1.0549999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.589,"5min":1.589,"15min":1.667},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.45100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.679,"5min":1.708,"15min":1.7749999999999999},"last":1.21},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.1339999999999999},"min":{"1min":0.48499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5580000000000001,"5min":1.694,"15min":1.732},"last":1.04}]},{"osd":3,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0069999999999999,"15min":1.0780000000000001},"min":{"1min":0.53800000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.6080000000000001,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.244,"5min":1.1930000000000001,"15min":1.2769999999999999},"min":{"1min":0.58399999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7969999999999999,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":1.089}]},{"osd":4,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3819999999999999,"15min":1.4039999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.9019999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4550000000000001,"15min":1.516},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8029999999999999,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.431}]},{"osd":5,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.329,"15min":1.3660000000000001},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.823,"5min":1.9419999999999999,"15min":2.137},"last":1.284},{"interface":"front","average":{"1min":1.425,"5min":1.3879999999999999,"15min":1.4790000000000001},"min":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.9259999999999999,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":0.998}]},{"osd":6,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.954,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.4870000000000001,"15min":1.5720000000000001},"min":{"1min":0.72099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.911,"5min":3.097,"15min":3.097},"last":1.2649999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.925,"5min":2.085,"15min":2.2959999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.464,"15min":1.5509999999999999},"min":{"1min":0.79000000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0600000000000001,"5min":2.077,"15min":2.3540000000000001},"last":1.3839999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:20:29.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:20:30.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.249+0000 7faf1aff4640 1 Processor -- start 2023-10-21T16:20:30.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.249+0000 7faf1aff4640 1 -- start start 2023-10-21T16:20:30.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.250+0000 7faf1aff4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf140cf720 0x7faf1406b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.250+0000 7faf1aff4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faf1406bd90 0x7faf140cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf1aff4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf140cb590 con 0x7faf140cd8a0 2023-10-21T16:20:30.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf1aff4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1406c220 con 0x7faf1406bd90 2023-10-21T16:20:30.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf1aff4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf140cfc90 con 0x7faf140cf720 2023-10-21T16:20:30.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf19ff2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faf140cd8a0 0x7faf1406b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46088/0 (socket says 172.21.15.53:46088) 2023-10-21T16:20:30.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf19ff2640 1 -- 172.21.15.53:0/2935485149 learned_addr learned my addr 172.21.15.53:0/2935485149 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:30.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf1a7f3640 1 --2- 172.21.15.53:0/2935485149 >> v2:172.21.15.53:3300/0 conn(0x7faf1406bd90 0x7faf140cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672551545 0 0) 0x7faf140cb590 con 0x7faf140cd8a0 2023-10-21T16:20:30.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf00003520 con 0x7faf140cd8a0 2023-10-21T16:20:30.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.251+0000 7faf197f1640 1 --2- 172.21.15.53:0/2935485149 >> v2:172.21.15.124:3300/0 conn(0x7faf140cf720 0x7faf1406b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.252+0000 7faf1a7f3640 1 -- 172.21.15.53:0/2935485149 >> v2:172.21.15.124:3300/0 conn(0x7faf140cf720 msgr2=0x7faf1406b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:30.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.252+0000 7faf1a7f3640 1 --2- 172.21.15.53:0/2935485149 >> v2:172.21.15.124:3300/0 conn(0x7faf140cf720 0x7faf1406b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.252+0000 7faf1a7f3640 1 -- 172.21.15.53:0/2935485149 >> v1:172.21.15.53:6789/0 conn(0x7faf140cd8a0 legacy=0x7faf1406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.252+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987114122 0 0) 0x7faf00003520 con 0x7faf140cd8a0 2023-10-21T16:20:30.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.252+0000 7faf1a7f3640 1 -- 172.21.15.53:0/2935485149 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140cff70 con 0x7faf1406bd90 2023-10-21T16:20:30.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.253+0000 7faf1a7f3640 1 --2- 172.21.15.53:0/2935485149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf1406bd90 0x7faf140cba20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faf140b2310 tx=0x7faf0c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5351f8d40807c558 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.253+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10001670 con 0x7faf1406bd90 2023-10-21T16:20:30.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.253+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c02c8c0 con 0x7faf1406bd90 2023-10-21T16:20:30.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.253+0000 7faf18ff0640 1 -- 172.21.15.53:0/2935485149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c02cc30 con 0x7faf1406bd90 2023-10-21T16:20:30.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 -- 172.21.15.53:0/2935485149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf1406bd90 msgr2=0x7faf140cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 --2- 172.21.15.53:0/2935485149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf1406bd90 0x7faf140cba20 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7faf140b2310 tx=0x7faf0c02bd90 comp rx=0 tx=0).stop 2023-10-21T16:20:30.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 -- 172.21.15.53:0/2935485149 shutdown_connections 2023-10-21T16:20:30.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 --2- 172.21.15.53:0/2935485149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf1406bd90 0x7faf140cba20 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 --2- 172.21.15.53:0/2935485149 >> v2:172.21.15.124:3300/0 conn(0x7faf140cf720 0x7faf1406b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 -- 172.21.15.53:0/2935485149 >> 172.21.15.53:0/2935485149 conn(0x7faf14099b60 msgr2=0x7faf140d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:30.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 -- 172.21.15.53:0/2935485149 shutdown_connections 2023-10-21T16:20:30.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.254+0000 7faf1aff4640 1 -- 172.21.15.53:0/2935485149 wait complete. 2023-10-21T16:20:30.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.256+0000 7faf1aff4640 1 Processor -- start 2023-10-21T16:20:30.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.256+0000 7faf1aff4640 1 -- start start 2023-10-21T16:20:30.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 0x7faf140c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf140cf720 0x7faf1415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140c8f50 con 0x7faf140cd8a0 2023-10-21T16:20:30.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140c9230 con 0x7faf1406bd90 2023-10-21T16:20:30.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf1aff4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140c9510 con 0x7faf140cf720 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf197f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf197f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37886/0 (socket says 172.21.15.53:37886) 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.257+0000 7faf197f1640 1 -- 172.21.15.53:0/3215258865 learned_addr learned my addr 172.21.15.53:0/3215258865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 -- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf140cf720 msgr2=0x7faf1415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf140cf720 0x7faf1415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf19ff2640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 0x7faf140c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 -- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 msgr2=0x7faf140c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 0x7faf140c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 -- 172.21.15.53:0/3215258865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c009a70 con 0x7faf140cd8a0 2023-10-21T16:20:30.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf19ff2640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 0x7faf140c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:30.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf197f1640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7faf0400d800 tx=0x7faf0400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400b6c0 con 0x7faf140cd8a0 2023-10-21T16:20:30.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400b820 con 0x7faf140cd8a0 2023-10-21T16:20:30.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf140c97f0 con 0x7faf140cd8a0 2023-10-21T16:20:30.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.258+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400bbb0 con 0x7faf140cd8a0 2023-10-21T16:20:30.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.259+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf140c7330 con 0x7faf140cd8a0 2023-10-21T16:20:30.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.260+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1404e160 con 0x7faf140cd8a0 2023-10-21T16:20:30.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.261+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faf0400bd10 con 0x7faf140cd8a0 2023-10-21T16:20:30.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.262+0000 7faf0a7fc640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 0x7faef806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.262+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faf04089f40 con 0x7faf140cd8a0 2023-10-21T16:20:30.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.263+0000 7faf19ff2640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 0x7faef806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.264+0000 7faf19ff2640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 0x7faef806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf10001380 tx=0x7faf10009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.266+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf04054cd0 con 0x7faf140cd8a0 2023-10-21T16:20:30.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.457+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 --> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf140c93a0 con 0x7faef806af90 2023-10-21T16:20:30.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.479+0000 7faf0a7fc640 1 -- 172.21.15.53:0/3215258865 <== mgr.6760 v2:172.21.15.124:6832/596536042 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286860 (secure 0 0 0) 0x7faf140c93a0 con 0x7faef806af90 2023-10-21T16:20:30.485 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:30.486 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:20:30.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 msgr2=0x7faef806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 0x7faef806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf10001380 tx=0x7faf10009040 comp rx=0 tx=0).stop 2023-10-21T16:20:30.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 msgr2=0x7faf140c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7faf0400d800 tx=0x7faf0400dcc0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 shutdown_connections 2023-10-21T16:20:30.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7faef806af90 0x7faef806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf140cf720 0x7faf1415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf140cd8a0 0x7faf140c6df0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 --2- 172.21.15.53:0/3215258865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf1406bd90 0x7faf140c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 >> 172.21.15.53:0/3215258865 conn(0x7faf14099b60 msgr2=0x7faf140b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:30.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 shutdown_connections 2023-10-21T16:20:30.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.485+0000 7faf1aff4640 1 -- 172.21.15.53:0/3215258865 wait complete. 2023-10-21T16:20:30.507 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:20:28.819056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":371,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4768,"ondisk_log_size":4768,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":332904,"kb_used_data":14052,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":754641816,"statfs":{"total":773094113280,"available":772753219584,"internally_reserved":0,"allocated":14389248,"data_stored":7844652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":326356537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.384646"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687907+0000","last_change":"2023-10-21T16:20:25.512333+0000","last_active":"2023-10-21T16:20:26.687907+0000","last_peered":"2023-10-21T16:20:26.687907+0000","last_clean":"2023-10-21T16:20:26.687907+0000","last_became_active":"2023-10-21T16:20:25.508973+0000","last_became_peered":"2023-10-21T16:20:25.508973+0000","last_unstale":"2023-10-21T16:20:26.687907+0000","last_undegraded":"2023-10-21T16:20:26.687907+0000","last_fullsized":"2023-10-21T16:20:26.687907+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:11:40.223441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218540+0000","last_change":"2023-10-21T16:20:25.497809+0000","last_active":"2023-10-21T16:20:27.218540+0000","last_peered":"2023-10-21T16:20:27.218540+0000","last_clean":"2023-10-21T16:20:27.218540+0000","last_became_active":"2023-10-21T16:20:25.497305+0000","last_became_peered":"2023-10-21T16:20:25.497305+0000","last_unstale":"2023-10-21T16:20:27.218540+0000","last_undegraded":"2023-10-21T16:20:27.218540+0000","last_fullsized":"2023-10-21T16:20:27.218540+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:36:33.724529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687731+0000","last_change":"2023-10-21T16:20:25.507106+0000","last_active":"2023-10-21T16:20:26.687731+0000","last_peered":"2023-10-21T16:20:26.687731+0000","last_clean":"2023-10-21T16:20:26.687731+0000","last_became_active":"2023-10-21T16:20:25.506334+0000","last_became_peered":"2023-10-21T16:20:25.506334+0000","last_unstale":"2023-10-21T16:20:26.687731+0000","last_undegraded":"2023-10-21T16:20:26.687731+0000","last_fullsized":"2023-10-21T16:20:26.687731+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:12.700942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470792+0000","last_change":"2023-10-21T16:20:25.500829+0000","last_active":"2023-10-21T16:20:26.470792+0000","last_peered":"2023-10-21T16:20:26.470792+0000","last_clean":"2023-10-21T16:20:26.470792+0000","last_became_active":"2023-10-21T16:20:25.500337+0000","last_became_peered":"2023-10-21T16:20:25.500337+0000","last_unstale":"2023-10-21T16:20:26.470792+0000","last_undegraded":"2023-10-21T16:20:26.470792+0000","last_fullsized":"2023-10-21T16:20:26.470792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:59:30.783483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217071+0000","last_change":"2023-10-21T16:20:25.507586+0000","last_active":"2023-10-21T16:20:27.217071+0000","last_peered":"2023-10-21T16:20:27.217071+0000","last_clean":"2023-10-21T16:20:27.217071+0000","last_became_active":"2023-10-21T16:20:25.503472+0000","last_became_peered":"2023-10-21T16:20:25.503472+0000","last_unstale":"2023-10-21T16:20:27.217071+0000","last_undegraded":"2023-10-21T16:20:27.217071+0000","last_fullsized":"2023-10-21T16:20:27.217071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:01:53.863086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217130+0000","last_change":"2023-10-21T16:20:25.507551+0000","last_active":"2023-10-21T16:20:27.217130+0000","last_peered":"2023-10-21T16:20:27.217130+0000","last_clean":"2023-10-21T16:20:27.217130+0000","last_became_active":"2023-10-21T16:20:25.507085+0000","last_became_peered":"2023-10-21T16:20:25.507085+0000","last_unstale":"2023-10-21T16:20:27.217130+0000","last_undegraded":"2023-10-21T16:20:27.217130+0000","last_fullsized":"2023-10-21T16:20:27.217130+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:52:25.809487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216570+0000","last_change":"2023-10-21T16:20:25.499521+0000","last_active":"2023-10-21T16:20:27.216570+0000","last_peered":"2023-10-21T16:20:27.216570+0000","last_clean":"2023-10-21T16:20:27.216570+0000","last_became_active":"2023-10-21T16:20:25.499209+0000","last_became_peered":"2023-10-21T16:20:25.499209+0000","last_unstale":"2023-10-21T16:20:27.216570+0000","last_undegraded":"2023-10-21T16:20:27.216570+0000","last_fullsized":"2023-10-21T16:20:27.216570+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:49:14.717088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474857+0000","last_change":"2023-10-21T16:20:25.520411+0000","last_active":"2023-10-21T16:20:26.474857+0000","last_peered":"2023-10-21T16:20:26.474857+0000","last_clean":"2023-10-21T16:20:26.474857+0000","last_became_active":"2023-10-21T16:20:25.520100+0000","last_became_peered":"2023-10-21T16:20:25.520100+0000","last_unstale":"2023-10-21T16:20:26.474857+0000","last_undegraded":"2023-10-21T16:20:26.474857+0000","last_fullsized":"2023-10-21T16:20:26.474857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:11.630700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218446+0000","last_change":"2023-10-21T16:20:25.506546+0000","last_active":"2023-10-21T16:20:27.218446+0000","last_peered":"2023-10-21T16:20:27.218446+0000","last_clean":"2023-10-21T16:20:27.218446+0000","last_became_active":"2023-10-21T16:20:25.504739+0000","last_became_peered":"2023-10-21T16:20:25.504739+0000","last_unstale":"2023-10-21T16:20:27.218446+0000","last_undegraded":"2023-10-21T16:20:27.218446+0000","last_fullsized":"2023-10-21T16:20:27.218446+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:15.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217242+0000","last_change":"2023-10-21T16:20:25.516538+0000","last_active":"2023-10-21T16:20:27.217242+0000","last_peered":"2023-10-21T16:20:27.217242+0000","last_clean":"2023-10-21T16:20:27.217242+0000","last_became_active":"2023-10-21T16:20:25.516243+0000","last_became_peered":"2023-10-21T16:20:25.516243+0000","last_unstale":"2023-10-21T16:20:27.217242+0000","last_undegraded":"2023-10-21T16:20:27.217242+0000","last_fullsized":"2023-10-21T16:20:27.217242+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:50:18.987282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.899025+0000","last_change":"2023-10-21T16:20:25.498593+0000","last_active":"2023-10-21T16:20:26.899025+0000","last_peered":"2023-10-21T16:20:26.899025+0000","last_clean":"2023-10-21T16:20:26.899025+0000","last_became_active":"2023-10-21T16:20:25.498244+0000","last_became_peered":"2023-10-21T16:20:25.498244+0000","last_unstale":"2023-10-21T16:20:26.899025+0000","last_undegraded":"2023-10-21T16:20:26.899025+0000","last_fullsized":"2023-10-21T16:20:26.899025+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:07:33.886401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475780+0000","last_change":"2023-10-21T16:20:25.501223+0000","last_active":"2023-10-21T16:20:26.475780+0000","last_peered":"2023-10-21T16:20:26.475780+0000","last_clean":"2023-10-21T16:20:26.475780+0000","last_became_active":"2023-10-21T16:20:25.500245+0000","last_became_peered":"2023-10-21T16:20:25.500245+0000","last_unstale":"2023-10-21T16:20:26.475780+0000","last_undegraded":"2023-10-21T16:20:26.475780+0000","last_fullsized":"2023-10-21T16:20:26.475780+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:56:37.087017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470400+0000","last_change":"2023-10-21T16:20:25.500938+0000","last_active":"2023-10-21T16:20:26.470400+0000","last_peered":"2023-10-21T16:20:26.470400+0000","last_clean":"2023-10-21T16:20:26.470400+0000","last_became_active":"2023-10-21T16:20:25.499117+0000","last_became_peered":"2023-10-21T16:20:25.499117+0000","last_unstale":"2023-10-21T16:20:26.470400+0000","last_undegraded":"2023-10-21T16:20:26.470400+0000","last_fullsized":"2023-10-21T16:20:26.470400+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:28.356874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217378+0000","last_change":"2023-10-21T16:20:25.499494+0000","last_active":"2023-10-21T16:20:27.217378+0000","last_peered":"2023-10-21T16:20:27.217378+0000","last_clean":"2023-10-21T16:20:27.217378+0000","last_became_active":"2023-10-21T16:20:25.498174+0000","last_became_peered":"2023-10-21T16:20:25.498174+0000","last_unstale":"2023-10-21T16:20:27.217378+0000","last_undegraded":"2023-10-21T16:20:27.217378+0000","last_fullsized":"2023-10-21T16:20:27.217378+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:06.511126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216083+0000","last_change":"2023-10-21T16:20:25.504658+0000","last_active":"2023-10-21T16:20:27.216083+0000","last_peered":"2023-10-21T16:20:27.216083+0000","last_clean":"2023-10-21T16:20:27.216083+0000","last_became_active":"2023-10-21T16:20:25.504396+0000","last_became_peered":"2023-10-21T16:20:25.504396+0000","last_unstale":"2023-10-21T16:20:27.216083+0000","last_undegraded":"2023-10-21T16:20:27.216083+0000","last_fullsized":"2023-10-21T16:20:27.216083+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:41:05.529613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216008+0000","last_change":"2023-10-21T16:20:25.498612+0000","last_active":"2023-10-21T16:20:27.216008+0000","last_peered":"2023-10-21T16:20:27.216008+0000","last_clean":"2023-10-21T16:20:27.216008+0000","last_became_active":"2023-10-21T16:20:25.498337+0000","last_became_peered":"2023-10-21T16:20:25.498337+0000","last_unstale":"2023-10-21T16:20:27.216008+0000","last_undegraded":"2023-10-21T16:20:27.216008+0000","last_fullsized":"2023-10-21T16:20:27.216008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:47:45.030989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688039+0000","last_change":"2023-10-21T16:20:25.511129+0000","last_active":"2023-10-21T16:20:26.688039+0000","last_peered":"2023-10-21T16:20:26.688039+0000","last_clean":"2023-10-21T16:20:26.688039+0000","last_became_active":"2023-10-21T16:20:25.505765+0000","last_became_peered":"2023-10-21T16:20:25.505765+0000","last_unstale":"2023-10-21T16:20:26.688039+0000","last_undegraded":"2023-10-21T16:20:26.688039+0000","last_fullsized":"2023-10-21T16:20:26.688039+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:57:45.358473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218580+0000","last_change":"2023-10-21T16:20:25.517735+0000","last_active":"2023-10-21T16:20:27.218580+0000","last_peered":"2023-10-21T16:20:27.218580+0000","last_clean":"2023-10-21T16:20:27.218580+0000","last_became_active":"2023-10-21T16:20:25.517385+0000","last_became_peered":"2023-10-21T16:20:25.517385+0000","last_unstale":"2023-10-21T16:20:27.218580+0000","last_undegraded":"2023-10-21T16:20:27.218580+0000","last_fullsized":"2023-10-21T16:20:27.218580+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:31.044939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688032+0000","last_change":"2023-10-21T16:20:25.498972+0000","last_active":"2023-10-21T16:20:26.688032+0000","last_peered":"2023-10-21T16:20:26.688032+0000","last_clean":"2023-10-21T16:20:26.688032+0000","last_became_active":"2023-10-21T16:20:25.498670+0000","last_became_peered":"2023-10-21T16:20:25.498670+0000","last_unstale":"2023-10-21T16:20:26.688032+0000","last_undegraded":"2023-10-21T16:20:26.688032+0000","last_fullsized":"2023-10-21T16:20:26.688032+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:59:30.783483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476134+0000","last_change":"2023-10-21T16:20:25.493789+0000","last_active":"2023-10-21T16:20:26.476134+0000","last_peered":"2023-10-21T16:20:26.476134+0000","last_clean":"2023-10-21T16:20:26.476134+0000","last_became_active":"2023-10-21T16:20:25.493529+0000","last_became_peered":"2023-10-21T16:20:25.493529+0000","last_unstale":"2023-10-21T16:20:26.476134+0000","last_undegraded":"2023-10-21T16:20:26.476134+0000","last_fullsized":"2023-10-21T16:20:26.476134+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:47.320111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475650+0000","last_change":"2023-10-21T16:20:25.484414+0000","last_active":"2023-10-21T16:20:26.475650+0000","last_peered":"2023-10-21T16:20:26.475650+0000","last_clean":"2023-10-21T16:20:26.475650+0000","last_became_active":"2023-10-21T16:20:25.483637+0000","last_became_peered":"2023-10-21T16:20:25.483637+0000","last_unstale":"2023-10-21T16:20:26.475650+0000","last_undegraded":"2023-10-21T16:20:26.475650+0000","last_fullsized":"2023-10-21T16:20:26.475650+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:42.388519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216519+0000","last_change":"2023-10-21T16:20:25.509186+0000","last_active":"2023-10-21T16:20:27.216519+0000","last_peered":"2023-10-21T16:20:27.216519+0000","last_clean":"2023-10-21T16:20:27.216519+0000","last_became_active":"2023-10-21T16:20:25.506702+0000","last_became_peered":"2023-10-21T16:20:25.506702+0000","last_unstale":"2023-10-21T16:20:27.216519+0000","last_undegraded":"2023-10-21T16:20:27.216519+0000","last_fullsized":"2023-10-21T16:20:27.216519+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:04.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687451+0000","last_change":"2023-10-21T16:20:25.507235+0000","last_active":"2023-10-21T16:20:26.687451+0000","last_peered":"2023-10-21T16:20:26.687451+0000","last_clean":"2023-10-21T16:20:26.687451+0000","last_became_active":"2023-10-21T16:20:25.506941+0000","last_became_peered":"2023-10-21T16:20:25.506941+0000","last_unstale":"2023-10-21T16:20:26.687451+0000","last_undegraded":"2023-10-21T16:20:26.687451+0000","last_fullsized":"2023-10-21T16:20:26.687451+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:48.852794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216587+0000","last_change":"2023-10-21T16:20:25.510813+0000","last_active":"2023-10-21T16:20:27.216587+0000","last_peered":"2023-10-21T16:20:27.216587+0000","last_clean":"2023-10-21T16:20:27.216587+0000","last_became_active":"2023-10-21T16:20:25.510481+0000","last_became_peered":"2023-10-21T16:20:25.510481+0000","last_unstale":"2023-10-21T16:20:27.216587+0000","last_undegraded":"2023-10-21T16:20:27.216587+0000","last_fullsized":"2023-10-21T16:20:27.216587+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:58.161062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476341+0000","last_change":"2023-10-21T16:20:25.496440+0000","last_active":"2023-10-21T16:20:26.476341+0000","last_peered":"2023-10-21T16:20:26.476341+0000","last_clean":"2023-10-21T16:20:26.476341+0000","last_became_active":"2023-10-21T16:20:25.496164+0000","last_became_peered":"2023-10-21T16:20:25.496164+0000","last_unstale":"2023-10-21T16:20:26.476341+0000","last_undegraded":"2023-10-21T16:20:26.476341+0000","last_fullsized":"2023-10-21T16:20:26.476341+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:22.403828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476343+0000","last_change":"2023-10-21T16:20:25.495907+0000","last_active":"2023-10-21T16:20:26.476343+0000","last_peered":"2023-10-21T16:20:26.476343+0000","last_clean":"2023-10-21T16:20:26.476343+0000","last_became_active":"2023-10-21T16:20:25.495590+0000","last_became_peered":"2023-10-21T16:20:25.495590+0000","last_unstale":"2023-10-21T16:20:26.476343+0000","last_undegraded":"2023-10-21T16:20:26.476343+0000","last_fullsized":"2023-10-21T16:20:26.476343+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:23:21.854679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898932+0000","last_change":"2023-10-21T16:20:25.496972+0000","last_active":"2023-10-21T16:20:26.898932+0000","last_peered":"2023-10-21T16:20:26.898932+0000","last_clean":"2023-10-21T16:20:26.898932+0000","last_became_active":"2023-10-21T16:20:25.496686+0000","last_became_peered":"2023-10-21T16:20:25.496686+0000","last_unstale":"2023-10-21T16:20:26.898932+0000","last_undegraded":"2023-10-21T16:20:26.898932+0000","last_fullsized":"2023-10-21T16:20:26.898932+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:05:11.747206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217619+0000","last_change":"2023-10-21T16:20:25.507501+0000","last_active":"2023-10-21T16:20:27.217619+0000","last_peered":"2023-10-21T16:20:27.217619+0000","last_clean":"2023-10-21T16:20:27.217619+0000","last_became_active":"2023-10-21T16:20:25.507075+0000","last_became_peered":"2023-10-21T16:20:25.507075+0000","last_unstale":"2023-10-21T16:20:27.217619+0000","last_undegraded":"2023-10-21T16:20:27.217619+0000","last_fullsized":"2023-10-21T16:20:27.217619+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:51:21.412057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475705+0000","last_change":"2023-10-21T16:20:25.498301+0000","last_active":"2023-10-21T16:20:26.475705+0000","last_peered":"2023-10-21T16:20:26.475705+0000","last_clean":"2023-10-21T16:20:26.475705+0000","last_became_active":"2023-10-21T16:20:25.497856+0000","last_became_peered":"2023-10-21T16:20:25.497856+0000","last_unstale":"2023-10-21T16:20:26.475705+0000","last_undegraded":"2023-10-21T16:20:26.475705+0000","last_fullsized":"2023-10-21T16:20:26.475705+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:53:02.501762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475736+0000","last_change":"2023-10-21T16:20:25.484075+0000","last_active":"2023-10-21T16:20:26.475736+0000","last_peered":"2023-10-21T16:20:26.475736+0000","last_clean":"2023-10-21T16:20:26.475736+0000","last_became_active":"2023-10-21T16:20:25.483401+0000","last_became_peered":"2023-10-21T16:20:25.483401+0000","last_unstale":"2023-10-21T16:20:26.475736+0000","last_undegraded":"2023-10-21T16:20:26.475736+0000","last_fullsized":"2023-10-21T16:20:26.475736+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:06:09.553455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216701+0000","last_change":"2023-10-21T16:20:25.509626+0000","last_active":"2023-10-21T16:20:27.216701+0000","last_peered":"2023-10-21T16:20:27.216701+0000","last_clean":"2023-10-21T16:20:27.216701+0000","last_became_active":"2023-10-21T16:20:25.509273+0000","last_became_peered":"2023-10-21T16:20:25.509273+0000","last_unstale":"2023-10-21T16:20:27.216701+0000","last_undegraded":"2023-10-21T16:20:27.216701+0000","last_fullsized":"2023-10-21T16:20:27.216701+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:41:06.579735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475107+0000","last_change":"2023-10-21T16:20:25.488820+0000","last_active":"2023-10-21T16:20:26.475107+0000","last_peered":"2023-10-21T16:20:26.475107+0000","last_clean":"2023-10-21T16:20:26.475107+0000","last_became_active":"2023-10-21T16:20:25.488127+0000","last_became_peered":"2023-10-21T16:20:25.488127+0000","last_unstale":"2023-10-21T16:20:26.475107+0000","last_undegraded":"2023-10-21T16:20:26.475107+0000","last_fullsized":"2023-10-21T16:20:26.475107+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:45:40.105114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687385+0000","last_change":"2023-10-21T16:20:25.499698+0000","last_active":"2023-10-21T16:20:26.687385+0000","last_peered":"2023-10-21T16:20:26.687385+0000","last_clean":"2023-10-21T16:20:26.687385+0000","last_became_active":"2023-10-21T16:20:25.499370+0000","last_became_peered":"2023-10-21T16:20:25.499370+0000","last_unstale":"2023-10-21T16:20:26.687385+0000","last_undegraded":"2023-10-21T16:20:26.687385+0000","last_fullsized":"2023-10-21T16:20:26.687385+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:07.250301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"132'4","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.471038+0000","last_change":"2023-10-21T16:20:24.479530+0000","last_active":"2023-10-21T16:20:26.471038+0000","last_peered":"2023-10-21T16:20:26.471038+0000","last_clean":"2023-10-21T16:20:26.471038+0000","last_became_active":"2023-10-21T16:20:24.478965+0000","last_became_peered":"2023-10-21T16:20:24.478965+0000","last_unstale":"2023-10-21T16:20:26.471038+0000","last_undegraded":"2023-10-21T16:20:26.471038+0000","last_fullsized":"2023-10-21T16:20:26.471038+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:11.685852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475042+0000","last_change":"2023-10-21T16:20:25.500905+0000","last_active":"2023-10-21T16:20:26.475042+0000","last_peered":"2023-10-21T16:20:26.475042+0000","last_clean":"2023-10-21T16:20:26.475042+0000","last_became_active":"2023-10-21T16:20:25.500172+0000","last_became_peered":"2023-10-21T16:20:25.500172+0000","last_unstale":"2023-10-21T16:20:26.475042+0000","last_undegraded":"2023-10-21T16:20:26.475042+0000","last_fullsized":"2023-10-21T16:20:26.475042+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:38.785025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476149+0000","last_change":"2023-10-21T16:20:24.472971+0000","last_active":"2023-10-21T16:20:26.476149+0000","last_peered":"2023-10-21T16:20:26.476149+0000","last_clean":"2023-10-21T16:20:26.476149+0000","last_became_active":"2023-10-21T16:20:24.472631+0000","last_became_peered":"2023-10-21T16:20:24.472631+0000","last_unstale":"2023-10-21T16:20:26.476149+0000","last_undegraded":"2023-10-21T16:20:26.476149+0000","last_fullsized":"2023-10-21T16:20:26.476149+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:38:13.204439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476016+0000","last_change":"2023-10-21T16:20:25.500448+0000","last_active":"2023-10-21T16:20:26.476016+0000","last_peered":"2023-10-21T16:20:26.476016+0000","last_clean":"2023-10-21T16:20:26.476016+0000","last_became_active":"2023-10-21T16:20:25.500117+0000","last_became_peered":"2023-10-21T16:20:25.500117+0000","last_unstale":"2023-10-21T16:20:26.476016+0000","last_undegraded":"2023-10-21T16:20:26.476016+0000","last_fullsized":"2023-10-21T16:20:26.476016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:25.477807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476150+0000","last_change":"2023-10-21T16:20:24.468190+0000","last_active":"2023-10-21T16:20:26.476150+0000","last_peered":"2023-10-21T16:20:26.476150+0000","last_clean":"2023-10-21T16:20:26.476150+0000","last_became_active":"2023-10-21T16:20:24.467888+0000","last_became_peered":"2023-10-21T16:20:24.467888+0000","last_unstale":"2023-10-21T16:20:26.476150+0000","last_undegraded":"2023-10-21T16:20:26.476150+0000","last_fullsized":"2023-10-21T16:20:26.476150+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:48:09.502229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898547+0000","last_change":"2023-10-21T16:20:25.486183+0000","last_active":"2023-10-21T16:20:26.898547+0000","last_peered":"2023-10-21T16:20:26.898547+0000","last_clean":"2023-10-21T16:20:26.898547+0000","last_became_active":"2023-10-21T16:20:25.485864+0000","last_became_peered":"2023-10-21T16:20:25.485864+0000","last_unstale":"2023-10-21T16:20:26.898547+0000","last_undegraded":"2023-10-21T16:20:26.898547+0000","last_fullsized":"2023-10-21T16:20:26.898547+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:12:29.524896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217008+0000","last_change":"2023-10-21T16:20:24.465995+0000","last_active":"2023-10-21T16:20:27.217008+0000","last_peered":"2023-10-21T16:20:27.217008+0000","last_clean":"2023-10-21T16:20:27.217008+0000","last_became_active":"2023-10-21T16:20:24.465739+0000","last_became_peered":"2023-10-21T16:20:24.465739+0000","last_unstale":"2023-10-21T16:20:27.217008+0000","last_undegraded":"2023-10-21T16:20:27.217008+0000","last_fullsized":"2023-10-21T16:20:27.217008+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:48:09.502229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470956+0000","last_change":"2023-10-21T16:20:25.517806+0000","last_active":"2023-10-21T16:20:26.470956+0000","last_peered":"2023-10-21T16:20:26.470956+0000","last_clean":"2023-10-21T16:20:26.470956+0000","last_became_active":"2023-10-21T16:20:25.517479+0000","last_became_peered":"2023-10-21T16:20:25.517479+0000","last_unstale":"2023-10-21T16:20:26.470956+0000","last_undegraded":"2023-10-21T16:20:26.470956+0000","last_fullsized":"2023-10-21T16:20:26.470956+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:13.074880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216470+0000","last_change":"2023-10-21T16:20:24.478509+0000","last_active":"2023-10-21T16:20:27.216470+0000","last_peered":"2023-10-21T16:20:27.216470+0000","last_clean":"2023-10-21T16:20:27.216470+0000","last_became_active":"2023-10-21T16:20:24.478216+0000","last_became_peered":"2023-10-21T16:20:24.478216+0000","last_unstale":"2023-10-21T16:20:27.216470+0000","last_undegraded":"2023-10-21T16:20:27.216470+0000","last_fullsized":"2023-10-21T16:20:27.216470+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:22:40.416746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688071+0000","last_change":"2023-10-21T16:20:25.507301+0000","last_active":"2023-10-21T16:20:26.688071+0000","last_peered":"2023-10-21T16:20:26.688071+0000","last_clean":"2023-10-21T16:20:26.688071+0000","last_became_active":"2023-10-21T16:20:25.506762+0000","last_became_peered":"2023-10-21T16:20:25.506762+0000","last_unstale":"2023-10-21T16:20:26.688071+0000","last_undegraded":"2023-10-21T16:20:26.688071+0000","last_fullsized":"2023-10-21T16:20:26.688071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:10.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898247+0000","last_change":"2023-10-21T16:20:24.466729+0000","last_active":"2023-10-21T16:20:26.898247+0000","last_peered":"2023-10-21T16:20:26.898247+0000","last_clean":"2023-10-21T16:20:26.898247+0000","last_became_active":"2023-10-21T16:20:24.466063+0000","last_became_peered":"2023-10-21T16:20:24.466063+0000","last_unstale":"2023-10-21T16:20:26.898247+0000","last_undegraded":"2023-10-21T16:20:26.898247+0000","last_fullsized":"2023-10-21T16:20:26.898247+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:41:04.514523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218742+0000","last_change":"2023-10-21T16:20:25.505369+0000","last_active":"2023-10-21T16:20:27.218742+0000","last_peered":"2023-10-21T16:20:27.218742+0000","last_clean":"2023-10-21T16:20:27.218742+0000","last_became_active":"2023-10-21T16:20:25.503117+0000","last_became_peered":"2023-10-21T16:20:25.503117+0000","last_unstale":"2023-10-21T16:20:27.218742+0000","last_undegraded":"2023-10-21T16:20:27.218742+0000","last_fullsized":"2023-10-21T16:20:27.218742+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:24.866100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475612+0000","last_change":"2023-10-21T16:20:24.472342+0000","last_active":"2023-10-21T16:20:26.475612+0000","last_peered":"2023-10-21T16:20:26.475612+0000","last_clean":"2023-10-21T16:20:26.475612+0000","last_became_active":"2023-10-21T16:20:24.471859+0000","last_became_peered":"2023-10-21T16:20:24.471859+0000","last_unstale":"2023-10-21T16:20:26.475612+0000","last_undegraded":"2023-10-21T16:20:26.475612+0000","last_fullsized":"2023-10-21T16:20:26.475612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:56:36.656749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470997+0000","last_change":"2023-10-21T16:20:25.502336+0000","last_active":"2023-10-21T16:20:26.470997+0000","last_peered":"2023-10-21T16:20:26.470997+0000","last_clean":"2023-10-21T16:20:26.470997+0000","last_became_active":"2023-10-21T16:20:25.502042+0000","last_became_peered":"2023-10-21T16:20:25.502042+0000","last_unstale":"2023-10-21T16:20:26.470997+0000","last_undegraded":"2023-10-21T16:20:26.470997+0000","last_fullsized":"2023-10-21T16:20:26.470997+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:10.298845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470251+0000","last_change":"2023-10-21T16:20:24.465690+0000","last_active":"2023-10-21T16:20:26.470251+0000","last_peered":"2023-10-21T16:20:26.470251+0000","last_clean":"2023-10-21T16:20:26.470251+0000","last_became_active":"2023-10-21T16:20:24.465430+0000","last_became_peered":"2023-10-21T16:20:24.465430+0000","last_unstale":"2023-10-21T16:20:26.470251+0000","last_undegraded":"2023-10-21T16:20:26.470251+0000","last_fullsized":"2023-10-21T16:20:26.470251+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:45.395322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687224+0000","last_change":"2023-10-21T16:20:25.499573+0000","last_active":"2023-10-21T16:20:26.687224+0000","last_peered":"2023-10-21T16:20:26.687224+0000","last_clean":"2023-10-21T16:20:26.687224+0000","last_became_active":"2023-10-21T16:20:25.497501+0000","last_became_peered":"2023-10-21T16:20:25.497501+0000","last_unstale":"2023-10-21T16:20:26.687224+0000","last_undegraded":"2023-10-21T16:20:26.687224+0000","last_fullsized":"2023-10-21T16:20:26.687224+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:46.410412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476201+0000","last_change":"2023-10-21T16:20:24.467237+0000","last_active":"2023-10-21T16:20:26.476201+0000","last_peered":"2023-10-21T16:20:26.476201+0000","last_clean":"2023-10-21T16:20:26.476201+0000","last_became_active":"2023-10-21T16:20:24.466916+0000","last_became_peered":"2023-10-21T16:20:24.466916+0000","last_unstale":"2023-10-21T16:20:26.476201+0000","last_undegraded":"2023-10-21T16:20:26.476201+0000","last_fullsized":"2023-10-21T16:20:26.476201+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:03.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216554+0000","last_change":"2023-10-21T16:20:25.505147+0000","last_active":"2023-10-21T16:20:27.216554+0000","last_peered":"2023-10-21T16:20:27.216554+0000","last_clean":"2023-10-21T16:20:27.216554+0000","last_became_active":"2023-10-21T16:20:25.503345+0000","last_became_peered":"2023-10-21T16:20:25.503345+0000","last_unstale":"2023-10-21T16:20:27.216554+0000","last_undegraded":"2023-10-21T16:20:27.216554+0000","last_fullsized":"2023-10-21T16:20:27.216554+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:51:26.459038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688112+0000","last_change":"2023-10-21T16:20:24.475943+0000","last_active":"2023-10-21T16:20:26.688112+0000","last_peered":"2023-10-21T16:20:26.688112+0000","last_clean":"2023-10-21T16:20:26.688112+0000","last_became_active":"2023-10-21T16:20:24.475587+0000","last_became_peered":"2023-10-21T16:20:24.475587+0000","last_unstale":"2023-10-21T16:20:26.688112+0000","last_undegraded":"2023-10-21T16:20:26.688112+0000","last_fullsized":"2023-10-21T16:20:26.688112+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:23.923808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475130+0000","last_change":"2023-10-21T16:20:25.499530+0000","last_active":"2023-10-21T16:20:26.475130+0000","last_peered":"2023-10-21T16:20:26.475130+0000","last_clean":"2023-10-21T16:20:26.475130+0000","last_became_active":"2023-10-21T16:20:25.499255+0000","last_became_peered":"2023-10-21T16:20:25.499255+0000","last_unstale":"2023-10-21T16:20:26.475130+0000","last_undegraded":"2023-10-21T16:20:26.475130+0000","last_fullsized":"2023-10-21T16:20:26.475130+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:50.768357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"132'3","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216675+0000","last_change":"2023-10-21T16:20:24.476536+0000","last_active":"2023-10-21T16:20:27.216675+0000","last_peered":"2023-10-21T16:20:27.216675+0000","last_clean":"2023-10-21T16:20:27.216675+0000","last_became_active":"2023-10-21T16:20:24.475246+0000","last_became_peered":"2023-10-21T16:20:24.475246+0000","last_unstale":"2023-10-21T16:20:27.216675+0000","last_undegraded":"2023-10-21T16:20:27.216675+0000","last_fullsized":"2023-10-21T16:20:27.216675+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:16:56.266586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217388+0000","last_change":"2023-10-21T16:20:25.506619+0000","last_active":"2023-10-21T16:20:27.217388+0000","last_peered":"2023-10-21T16:20:27.217388+0000","last_clean":"2023-10-21T16:20:27.217388+0000","last_became_active":"2023-10-21T16:20:25.506367+0000","last_became_peered":"2023-10-21T16:20:25.506367+0000","last_unstale":"2023-10-21T16:20:27.217388+0000","last_undegraded":"2023-10-21T16:20:27.217388+0000","last_fullsized":"2023-10-21T16:20:27.217388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:44.775791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"132'6","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217862+0000","last_change":"2023-10-21T16:20:24.478603+0000","last_active":"2023-10-21T16:20:27.217862+0000","last_peered":"2023-10-21T16:20:27.217862+0000","last_clean":"2023-10-21T16:20:27.217862+0000","last_became_active":"2023-10-21T16:20:24.478268+0000","last_became_peered":"2023-10-21T16:20:24.478268+0000","last_unstale":"2023-10-21T16:20:27.217862+0000","last_undegraded":"2023-10-21T16:20:27.217862+0000","last_fullsized":"2023-10-21T16:20:27.217862+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:30:01.838259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898318+0000","last_change":"2023-10-21T16:20:25.501373+0000","last_active":"2023-10-21T16:20:26.898318+0000","last_peered":"2023-10-21T16:20:26.898318+0000","last_clean":"2023-10-21T16:20:26.898318+0000","last_became_active":"2023-10-21T16:20:25.501049+0000","last_became_peered":"2023-10-21T16:20:25.501049+0000","last_unstale":"2023-10-21T16:20:26.898318+0000","last_undegraded":"2023-10-21T16:20:26.898318+0000","last_fullsized":"2023-10-21T16:20:26.898318+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:53:39.322952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217557+0000","last_change":"2023-10-21T16:20:24.470854+0000","last_active":"2023-10-21T16:20:27.217557+0000","last_peered":"2023-10-21T16:20:27.217557+0000","last_clean":"2023-10-21T16:20:27.217557+0000","last_became_active":"2023-10-21T16:20:24.470586+0000","last_became_peered":"2023-10-21T16:20:24.470586+0000","last_unstale":"2023-10-21T16:20:27.217557+0000","last_undegraded":"2023-10-21T16:20:27.217557+0000","last_fullsized":"2023-10-21T16:20:27.217557+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:56:36.656749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218402+0000","last_change":"2023-10-21T16:20:25.506635+0000","last_active":"2023-10-21T16:20:27.218402+0000","last_peered":"2023-10-21T16:20:27.218402+0000","last_clean":"2023-10-21T16:20:27.218402+0000","last_became_active":"2023-10-21T16:20:25.506075+0000","last_became_peered":"2023-10-21T16:20:25.506075+0000","last_unstale":"2023-10-21T16:20:27.218402+0000","last_undegraded":"2023-10-21T16:20:27.218402+0000","last_fullsized":"2023-10-21T16:20:27.218402+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:41:28.587131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470472+0000","last_change":"2023-10-21T16:20:24.474488+0000","last_active":"2023-10-21T16:20:26.470472+0000","last_peered":"2023-10-21T16:20:26.470472+0000","last_clean":"2023-10-21T16:20:26.470472+0000","last_became_active":"2023-10-21T16:20:24.474174+0000","last_became_peered":"2023-10-21T16:20:24.474174+0000","last_unstale":"2023-10-21T16:20:26.470472+0000","last_undegraded":"2023-10-21T16:20:26.470472+0000","last_fullsized":"2023-10-21T16:20:26.470472+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:06.235211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476100+0000","last_change":"2023-10-21T16:20:25.487714+0000","last_active":"2023-10-21T16:20:26.476100+0000","last_peered":"2023-10-21T16:20:26.476100+0000","last_clean":"2023-10-21T16:20:26.476100+0000","last_became_active":"2023-10-21T16:20:25.486824+0000","last_became_peered":"2023-10-21T16:20:25.486824+0000","last_unstale":"2023-10-21T16:20:26.476100+0000","last_undegraded":"2023-10-21T16:20:26.476100+0000","last_fullsized":"2023-10-21T16:20:26.476100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:53.016721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474944+0000","last_change":"2023-10-21T16:20:24.464774+0000","last_active":"2023-10-21T16:20:26.474944+0000","last_peered":"2023-10-21T16:20:26.474944+0000","last_clean":"2023-10-21T16:20:26.474944+0000","last_became_active":"2023-10-21T16:20:24.464508+0000","last_became_peered":"2023-10-21T16:20:24.464508+0000","last_unstale":"2023-10-21T16:20:26.474944+0000","last_undegraded":"2023-10-21T16:20:26.474944+0000","last_fullsized":"2023-10-21T16:20:26.474944+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:18:02.012324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.688061+0000","last_change":"2023-10-21T16:20:25.512562+0000","last_active":"2023-10-21T16:20:26.688061+0000","last_peered":"2023-10-21T16:20:26.688061+0000","last_clean":"2023-10-21T16:20:26.688061+0000","last_became_active":"2023-10-21T16:20:25.510596+0000","last_became_peered":"2023-10-21T16:20:25.510596+0000","last_unstale":"2023-10-21T16:20:26.688061+0000","last_undegraded":"2023-10-21T16:20:26.688061+0000","last_fullsized":"2023-10-21T16:20:26.688061+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:13.074880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"132'8","reported_seq":20,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475009+0000","last_change":"2023-10-21T16:20:24.467144+0000","last_active":"2023-10-21T16:20:26.475009+0000","last_peered":"2023-10-21T16:20:26.475009+0000","last_clean":"2023-10-21T16:20:26.475009+0000","last_became_active":"2023-10-21T16:20:24.466481+0000","last_became_peered":"2023-10-21T16:20:24.466481+0000","last_unstale":"2023-10-21T16:20:26.475009+0000","last_undegraded":"2023-10-21T16:20:26.475009+0000","last_fullsized":"2023-10-21T16:20:26.475009+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:24.892214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475137+0000","last_change":"2023-10-21T16:20:25.498590+0000","last_active":"2023-10-21T16:20:26.475137+0000","last_peered":"2023-10-21T16:20:26.475137+0000","last_clean":"2023-10-21T16:20:26.475137+0000","last_became_active":"2023-10-21T16:20:25.498314+0000","last_became_peered":"2023-10-21T16:20:25.498314+0000","last_unstale":"2023-10-21T16:20:26.475137+0000","last_undegraded":"2023-10-21T16:20:26.475137+0000","last_fullsized":"2023-10-21T16:20:26.475137+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:17.536558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218371+0000","last_change":"2023-10-21T16:20:24.478671+0000","last_active":"2023-10-21T16:20:27.218371+0000","last_peered":"2023-10-21T16:20:27.218371+0000","last_clean":"2023-10-21T16:20:27.218371+0000","last_became_active":"2023-10-21T16:20:24.477880+0000","last_became_peered":"2023-10-21T16:20:24.477880+0000","last_unstale":"2023-10-21T16:20:27.218371+0000","last_undegraded":"2023-10-21T16:20:27.218371+0000","last_fullsized":"2023-10-21T16:20:27.218371+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:05.591448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217179+0000","last_change":"2023-10-21T16:20:25.512409+0000","last_active":"2023-10-21T16:20:27.217179+0000","last_peered":"2023-10-21T16:20:27.217179+0000","last_clean":"2023-10-21T16:20:27.217179+0000","last_became_active":"2023-10-21T16:20:25.512151+0000","last_became_peered":"2023-10-21T16:20:25.512151+0000","last_unstale":"2023-10-21T16:20:27.217179+0000","last_undegraded":"2023-10-21T16:20:27.217179+0000","last_fullsized":"2023-10-21T16:20:27.217179+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:10:43.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"132'1","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216667+0000","last_change":"2023-10-21T16:20:24.472620+0000","last_active":"2023-10-21T16:20:27.216667+0000","last_peered":"2023-10-21T16:20:27.216667+0000","last_clean":"2023-10-21T16:20:27.216667+0000","last_became_active":"2023-10-21T16:20:24.472150+0000","last_became_peered":"2023-10-21T16:20:24.472150+0000","last_unstale":"2023-10-21T16:20:27.216667+0000","last_undegraded":"2023-10-21T16:20:27.216667+0000","last_fullsized":"2023-10-21T16:20:27.216667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:57:02.930510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217388+0000","last_change":"2023-10-21T16:20:25.506150+0000","last_active":"2023-10-21T16:20:27.217388+0000","last_peered":"2023-10-21T16:20:27.217388+0000","last_clean":"2023-10-21T16:20:27.217388+0000","last_became_active":"2023-10-21T16:20:25.502809+0000","last_became_peered":"2023-10-21T16:20:25.502809+0000","last_unstale":"2023-10-21T16:20:27.217388+0000","last_undegraded":"2023-10-21T16:20:27.217388+0000","last_fullsized":"2023-10-21T16:20:27.217388+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:06.124993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898527+0000","last_change":"2023-10-21T16:20:24.476385+0000","last_active":"2023-10-21T16:20:26.898527+0000","last_peered":"2023-10-21T16:20:26.898527+0000","last_clean":"2023-10-21T16:20:26.898527+0000","last_became_active":"2023-10-21T16:20:24.475886+0000","last_became_peered":"2023-10-21T16:20:24.475886+0000","last_unstale":"2023-10-21T16:20:26.898527+0000","last_undegraded":"2023-10-21T16:20:26.898527+0000","last_fullsized":"2023-10-21T16:20:26.898527+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:08.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687644+0000","last_change":"2023-10-21T16:20:25.508317+0000","last_active":"2023-10-21T16:20:26.687644+0000","last_peered":"2023-10-21T16:20:26.687644+0000","last_clean":"2023-10-21T16:20:26.687644+0000","last_became_active":"2023-10-21T16:20:25.507983+0000","last_became_peered":"2023-10-21T16:20:25.507983+0000","last_unstale":"2023-10-21T16:20:26.687644+0000","last_undegraded":"2023-10-21T16:20:26.687644+0000","last_fullsized":"2023-10-21T16:20:26.687644+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:53.644289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475944+0000","last_change":"2023-10-21T16:20:24.466299+0000","last_active":"2023-10-21T16:20:26.475944+0000","last_peered":"2023-10-21T16:20:26.475944+0000","last_clean":"2023-10-21T16:20:26.475944+0000","last_became_active":"2023-10-21T16:20:24.466030+0000","last_became_peered":"2023-10-21T16:20:24.466030+0000","last_unstale":"2023-10-21T16:20:26.475944+0000","last_undegraded":"2023-10-21T16:20:26.475944+0000","last_fullsized":"2023-10-21T16:20:26.475944+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:47.895505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687912+0000","last_change":"2023-10-21T16:20:25.518821+0000","last_active":"2023-10-21T16:20:26.687912+0000","last_peered":"2023-10-21T16:20:26.687912+0000","last_clean":"2023-10-21T16:20:26.687912+0000","last_became_active":"2023-10-21T16:20:25.518515+0000","last_became_peered":"2023-10-21T16:20:25.518515+0000","last_unstale":"2023-10-21T16:20:26.687912+0000","last_undegraded":"2023-10-21T16:20:26.687912+0000","last_fullsized":"2023-10-21T16:20:26.687912+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:28.356874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.474919+0000","last_change":"2023-10-21T16:20:24.465762+0000","last_active":"2023-10-21T16:20:26.474919+0000","last_peered":"2023-10-21T16:20:26.474919+0000","last_clean":"2023-10-21T16:20:26.474919+0000","last_became_active":"2023-10-21T16:20:24.463813+0000","last_became_peered":"2023-10-21T16:20:24.463813+0000","last_unstale":"2023-10-21T16:20:26.474919+0000","last_undegraded":"2023-10-21T16:20:26.474919+0000","last_fullsized":"2023-10-21T16:20:26.474919+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:14.453744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470917+0000","last_change":"2023-10-21T16:20:25.498461+0000","last_active":"2023-10-21T16:20:26.470917+0000","last_peered":"2023-10-21T16:20:26.470917+0000","last_clean":"2023-10-21T16:20:26.470917+0000","last_became_active":"2023-10-21T16:20:25.498168+0000","last_became_peered":"2023-10-21T16:20:25.498168+0000","last_unstale":"2023-10-21T16:20:26.470917+0000","last_undegraded":"2023-10-21T16:20:26.470917+0000","last_fullsized":"2023-10-21T16:20:26.470917+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:57:45.358473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216796+0000","last_change":"2023-10-21T16:20:24.473657+0000","last_active":"2023-10-21T16:20:27.216796+0000","last_peered":"2023-10-21T16:20:27.216796+0000","last_clean":"2023-10-21T16:20:27.216796+0000","last_became_active":"2023-10-21T16:20:24.473117+0000","last_became_peered":"2023-10-21T16:20:24.473117+0000","last_unstale":"2023-10-21T16:20:27.216796+0000","last_undegraded":"2023-10-21T16:20:27.216796+0000","last_fullsized":"2023-10-21T16:20:27.216796+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:27:14.327872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476087+0000","last_change":"2023-10-21T16:20:25.513788+0000","last_active":"2023-10-21T16:20:26.476087+0000","last_peered":"2023-10-21T16:20:26.476087+0000","last_clean":"2023-10-21T16:20:26.476087+0000","last_became_active":"2023-10-21T16:20:25.512766+0000","last_became_peered":"2023-10-21T16:20:25.512766+0000","last_unstale":"2023-10-21T16:20:26.476087+0000","last_undegraded":"2023-10-21T16:20:26.476087+0000","last_fullsized":"2023-10-21T16:20:26.476087+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:43.633206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475257+0000","last_change":"2023-10-21T16:20:24.464242+0000","last_active":"2023-10-21T16:20:26.475257+0000","last_peered":"2023-10-21T16:20:26.475257+0000","last_clean":"2023-10-21T16:20:26.475257+0000","last_became_active":"2023-10-21T16:20:24.463002+0000","last_became_peered":"2023-10-21T16:20:24.463002+0000","last_unstale":"2023-10-21T16:20:26.475257+0000","last_undegraded":"2023-10-21T16:20:26.475257+0000","last_fullsized":"2023-10-21T16:20:26.475257+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:07:59.125342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687448+0000","last_change":"2023-10-21T16:20:25.507311+0000","last_active":"2023-10-21T16:20:26.687448+0000","last_peered":"2023-10-21T16:20:26.687448+0000","last_clean":"2023-10-21T16:20:26.687448+0000","last_became_active":"2023-10-21T16:20:25.506899+0000","last_became_peered":"2023-10-21T16:20:25.506899+0000","last_unstale":"2023-10-21T16:20:26.687448+0000","last_undegraded":"2023-10-21T16:20:26.687448+0000","last_fullsized":"2023-10-21T16:20:26.687448+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:15.940135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217290+0000","last_change":"2023-10-21T16:20:24.464654+0000","last_active":"2023-10-21T16:20:27.217290+0000","last_peered":"2023-10-21T16:20:27.217290+0000","last_clean":"2023-10-21T16:20:27.217290+0000","last_became_active":"2023-10-21T16:20:24.464381+0000","last_became_peered":"2023-10-21T16:20:24.464381+0000","last_unstale":"2023-10-21T16:20:27.217290+0000","last_undegraded":"2023-10-21T16:20:27.217290+0000","last_fullsized":"2023-10-21T16:20:27.217290+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:24.892214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475327+0000","last_change":"2023-10-21T16:20:25.493924+0000","last_active":"2023-10-21T16:20:26.475327+0000","last_peered":"2023-10-21T16:20:26.475327+0000","last_clean":"2023-10-21T16:20:26.475327+0000","last_became_active":"2023-10-21T16:20:25.493634+0000","last_became_peered":"2023-10-21T16:20:25.493634+0000","last_unstale":"2023-10-21T16:20:26.475327+0000","last_undegraded":"2023-10-21T16:20:26.475327+0000","last_fullsized":"2023-10-21T16:20:26.475327+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:44.775791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.687845+0000","last_change":"2023-10-21T16:20:24.470748+0000","last_active":"2023-10-21T16:20:26.687845+0000","last_peered":"2023-10-21T16:20:26.687845+0000","last_clean":"2023-10-21T16:20:26.687845+0000","last_became_active":"2023-10-21T16:20:24.469923+0000","last_became_peered":"2023-10-21T16:20:24.469923+0000","last_unstale":"2023-10-21T16:20:26.687845+0000","last_undegraded":"2023-10-21T16:20:26.687845+0000","last_fullsized":"2023-10-21T16:20:26.687845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:14:08.670928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.470602+0000","last_change":"2023-10-21T16:20:25.498789+0000","last_active":"2023-10-21T16:20:26.470602+0000","last_peered":"2023-10-21T16:20:26.470602+0000","last_clean":"2023-10-21T16:20:26.470602+0000","last_became_active":"2023-10-21T16:20:25.498509+0000","last_became_peered":"2023-10-21T16:20:25.498509+0000","last_unstale":"2023-10-21T16:20:26.470602+0000","last_undegraded":"2023-10-21T16:20:26.470602+0000","last_fullsized":"2023-10-21T16:20:26.470602+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:11:40.223441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.218316+0000","last_change":"2023-10-21T16:20:24.478437+0000","last_active":"2023-10-21T16:20:27.218316+0000","last_peered":"2023-10-21T16:20:27.218316+0000","last_clean":"2023-10-21T16:20:27.218316+0000","last_became_active":"2023-10-21T16:20:24.478035+0000","last_became_peered":"2023-10-21T16:20:24.478035+0000","last_unstale":"2023-10-21T16:20:27.218316+0000","last_undegraded":"2023-10-21T16:20:27.218316+0000","last_fullsized":"2023-10-21T16:20:27.218316+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:35:37.750301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898529+0000","last_change":"2023-10-21T16:20:25.515312+0000","last_active":"2023-10-21T16:20:26.898529+0000","last_peered":"2023-10-21T16:20:26.898529+0000","last_clean":"2023-10-21T16:20:26.898529+0000","last_became_active":"2023-10-21T16:20:25.515103+0000","last_became_peered":"2023-10-21T16:20:25.515103+0000","last_unstale":"2023-10-21T16:20:26.898529+0000","last_undegraded":"2023-10-21T16:20:26.898529+0000","last_fullsized":"2023-10-21T16:20:26.898529+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:02.382807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.217208+0000","last_change":"2023-10-21T16:20:24.478381+0000","last_active":"2023-10-21T16:20:27.217208+0000","last_peered":"2023-10-21T16:20:27.217208+0000","last_clean":"2023-10-21T16:20:27.217208+0000","last_became_active":"2023-10-21T16:20:24.478136+0000","last_became_peered":"2023-10-21T16:20:24.478136+0000","last_unstale":"2023-10-21T16:20:27.217208+0000","last_undegraded":"2023-10-21T16:20:27.217208+0000","last_fullsized":"2023-10-21T16:20:27.217208+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:37:23.526235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475714+0000","last_change":"2023-10-21T16:20:25.515386+0000","last_active":"2023-10-21T16:20:26.475714+0000","last_peered":"2023-10-21T16:20:26.475714+0000","last_clean":"2023-10-21T16:20:26.475714+0000","last_became_active":"2023-10-21T16:20:25.515102+0000","last_became_peered":"2023-10-21T16:20:25.515102+0000","last_unstale":"2023-10-21T16:20:26.475714+0000","last_undegraded":"2023-10-21T16:20:26.475714+0000","last_fullsized":"2023-10-21T16:20:26.475714+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:39:36.293976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.476484+0000","last_change":"2023-10-21T16:20:24.475751+0000","last_active":"2023-10-21T16:20:26.476484+0000","last_peered":"2023-10-21T16:20:26.476484+0000","last_clean":"2023-10-21T16:20:26.476484+0000","last_became_active":"2023-10-21T16:20:24.475440+0000","last_became_peered":"2023-10-21T16:20:24.475440+0000","last_unstale":"2023-10-21T16:20:26.476484+0000","last_undegraded":"2023-10-21T16:20:26.476484+0000","last_fullsized":"2023-10-21T16:20:26.476484+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:57.145972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475845+0000","last_change":"2023-10-21T16:20:25.516301+0000","last_active":"2023-10-21T16:20:26.475845+0000","last_peered":"2023-10-21T16:20:26.475845+0000","last_clean":"2023-10-21T16:20:26.475845+0000","last_became_active":"2023-10-21T16:20:25.516030+0000","last_became_peered":"2023-10-21T16:20:25.516030+0000","last_unstale":"2023-10-21T16:20:26.475845+0000","last_undegraded":"2023-10-21T16:20:26.475845+0000","last_fullsized":"2023-10-21T16:20:26.475845+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:52.244581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475391+0000","last_change":"2023-10-21T16:20:24.462666+0000","last_active":"2023-10-21T16:20:26.475391+0000","last_peered":"2023-10-21T16:20:26.475391+0000","last_clean":"2023-10-21T16:20:26.475391+0000","last_became_active":"2023-10-21T16:20:24.462203+0000","last_became_peered":"2023-10-21T16:20:24.462203+0000","last_unstale":"2023-10-21T16:20:26.475391+0000","last_undegraded":"2023-10-21T16:20:26.475391+0000","last_fullsized":"2023-10-21T16:20:26.475391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:11:59.396608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216990+0000","last_change":"2023-10-21T16:20:25.505460+0000","last_active":"2023-10-21T16:20:27.216990+0000","last_peered":"2023-10-21T16:20:27.216990+0000","last_clean":"2023-10-21T16:20:27.216990+0000","last_became_active":"2023-10-21T16:20:25.504928+0000","last_became_peered":"2023-10-21T16:20:25.504928+0000","last_unstale":"2023-10-21T16:20:27.216990+0000","last_undegraded":"2023-10-21T16:20:27.216990+0000","last_fullsized":"2023-10-21T16:20:27.216990+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.938898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:27.216904+0000","last_change":"2023-10-21T16:20:24.479954+0000","last_active":"2023-10-21T16:20:27.216904+0000","last_peered":"2023-10-21T16:20:27.216904+0000","last_clean":"2023-10-21T16:20:27.216904+0000","last_became_active":"2023-10-21T16:20:24.477790+0000","last_became_peered":"2023-10-21T16:20:24.477790+0000","last_unstale":"2023-10-21T16:20:27.216904+0000","last_undegraded":"2023-10-21T16:20:27.216904+0000","last_fullsized":"2023-10-21T16:20:27.216904+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:13.981341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.899233+0000","last_change":"2023-10-21T16:20:25.497087+0000","last_active":"2023-10-21T16:20:26.899233+0000","last_peered":"2023-10-21T16:20:26.899233+0000","last_clean":"2023-10-21T16:20:26.899233+0000","last_became_active":"2023-10-21T16:20:25.496786+0000","last_became_peered":"2023-10-21T16:20:25.496786+0000","last_unstale":"2023-10-21T16:20:26.899233+0000","last_undegraded":"2023-10-21T16:20:26.899233+0000","last_fullsized":"2023-10-21T16:20:26.899233+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:22:46.410412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475403+0000","last_change":"2023-10-21T16:20:24.459583+0000","last_active":"2023-10-21T16:20:26.475403+0000","last_peered":"2023-10-21T16:20:26.475403+0000","last_clean":"2023-10-21T16:20:26.475403+0000","last_became_active":"2023-10-21T16:20:24.459220+0000","last_became_peered":"2023-10-21T16:20:24.459220+0000","last_unstale":"2023-10-21T16:20:26.475403+0000","last_undegraded":"2023-10-21T16:20:26.475403+0000","last_fullsized":"2023-10-21T16:20:26.475403+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:24:27.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.898510+0000","last_change":"2023-10-21T16:20:25.497114+0000","last_active":"2023-10-21T16:20:26.898510+0000","last_peered":"2023-10-21T16:20:26.898510+0000","last_clean":"2023-10-21T16:20:26.898510+0000","last_became_active":"2023-10-21T16:20:25.496759+0000","last_became_peered":"2023-10-21T16:20:25.496759+0000","last_unstale":"2023-10-21T16:20:26.898510+0000","last_undegraded":"2023-10-21T16:20:26.898510+0000","last_fullsized":"2023-10-21T16:20:26.898510+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:24.430237+0000","last_clean_scrub_stamp":"2023-10-21T16:20:24.430237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:10.153223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"132'2","reported_seq":14,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-21T16:20:26.475888+0000","last_change":"2023-10-21T16:20:24.465095+0000","last_active":"2023-10-21T16:20:26.475888+0000","last_peered":"2023-10-21T16:20:26.475888+0000","last_clean":"2023-10-21T16:20:26.475888+0000","last_became_active":"2023-10-21T16:20:24.464805+0000","last_became_peered":"2023-10-21T16:20:24.464805+0000","last_unstale":"2023-10-21T16:20:26.475888+0000","last_undegraded":"2023-10-21T16:20:26.475888+0000","last_fullsized":"2023-10-21T16:20:26.475888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:20:23.415147+0000","last_clean_scrub_stamp":"2023-10-21T16:20:23.415147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:09:46.413219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"122'4730","reported_seq":5169,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-21T16:20:25.489879+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:20:25.489879+0000","last_peered":"2023-10-21T16:20:25.489879+0000","last_clean":"2023-10-21T16:20:25.489879+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:20:25.489879+0000","last_undegraded":"2023-10-21T16:20:25.489879+0000","last_fullsized":"2023-10-21T16:20:25.489879+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":327,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":976,"num_read_kb":1961,"num_write":327,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4730,"ondisk_log_size":4730,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63908,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307932,"statfs":{"total":96636764160,"available":96571322368,"internally_reserved":0,"allocated":2248704,"data_stored":1425414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0620000000000001,"15min":1.137},"min":{"1min":0.70699999999999996,"5min":0.59799999999999998,"15min":0.51300000000000001},"max":{"1min":1.758,"5min":1.758,"15min":1.8400000000000001},"last":1.669},{"interface":"front","average":{"1min":1.238,"5min":1.1539999999999999,"15min":1.1759999999999999},"min":{"1min":0.95899999999999996,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.7270000000000001,"5min":1.883,"15min":2.448},"last":0.97699999999999998}]},{"osd":1,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.143,"15min":1.163},"min":{"1min":0.88100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.665,"5min":1.851,"15min":1.915},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1850000000000001,"15min":1.23},"min":{"1min":0.627,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":1.579,"5min":1.923,"15min":1.923},"last":1.2370000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.252,"15min":1.296},"min":{"1min":0.98799999999999999,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.694,"5min":1.782,"15min":2.085},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2709999999999999,"15min":1.294},"min":{"1min":0.80000000000000004,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.7090000000000001,"5min":1.8169999999999999,"15min":2.6070000000000002},"last":1.079}]},{"osd":3,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.194,"15min":1.2490000000000001},"min":{"1min":0.74199999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.431,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3819999999999999,"15min":1.421},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6359999999999999,"5min":1.903,"15min":1.9750000000000001},"last":1.6140000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.327,"15min":1.347},"min":{"1min":0.76200000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.839,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.284},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4159999999999999,"15min":1.4390000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":1.915,"15min":2.355},"last":1.3120000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4279999999999999,"15min":1.393},"min":{"1min":0.84699999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6879999999999999,"5min":1.911,"15min":1.9299999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.4430000000000001,"15min":1.4419999999999999},"min":{"1min":0.99399999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9259999999999999,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.1839999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.5389999999999999,"15min":1.5449999999999999},"min":{"1min":1.159,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.748,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4590000000000001,"15min":1.458},"min":{"1min":1.0209999999999999,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":1.5940000000000001,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36956,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334884,"statfs":{"total":96636764160,"available":96598921216,"internally_reserved":0,"allocated":1654784,"data_stored":830438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0980000000000001,"15min":1.1240000000000001},"min":{"1min":0.69599999999999995,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.722,"5min":1.869,"15min":1.869},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1739999999999999,"15min":1.21},"min":{"1min":0.82599999999999996,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7789999999999999,"5min":1.839,"15min":1.895},"last":1.861}]},{"osd":1,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.143,"15min":1.1899999999999999},"min":{"1min":0.81799999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.508,"5min":1.72,"15min":2.2309999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.2230000000000001,"15min":1.292},"min":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.538,"5min":2.2349999999999999,"15min":2.2669999999999999},"last":1.1970000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.206,"15min":1.2450000000000001},"min":{"1min":0.89400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7350000000000001,"5min":1.895,"15min":2.5339999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.232,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.706,"5min":1.821,"15min":1.821},"last":1.4319999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3320000000000001,"15min":1.3480000000000001},"min":{"1min":0.97199999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8919999999999999,"5min":2.0299999999999998,"15min":2.2000000000000002},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.405,"15min":1.423},"min":{"1min":1.2370000000000001,"5min":0.63,"15min":0.63},"max":{"1min":1.8620000000000001,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.831}]},{"osd":4,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.389,"15min":1.3500000000000001},"min":{"1min":0.95799999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.681,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4370000000000001,"15min":1.3899999999999999},"min":{"1min":0.89800000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.004,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.6359999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.5369999999999999,"15min":1.502},"min":{"1min":1.135,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":2.1400000000000001,"5min":2.246,"15min":2.246},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3580000000000001,"15min":1.347},"min":{"1min":0.871,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.944,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.5900000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:20:19 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.518,"15min":1.502},"min":{"1min":1.0669999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9830000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6399999999999999,"15min":1.6120000000000001},"min":{"1min":1.0349999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":1.756}]}]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1634304,"data_stored":828222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0660000000000001,"15min":1.1220000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.609,"5min":1.609,"15min":1.742},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.236,"5min":1.1359999999999999,"15min":1.1890000000000001},"min":{"1min":0.70099999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.45}]},{"osd":1,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.204,"15min":1.2569999999999999},"min":{"1min":0.81799999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2270000000000001,"15min":1.2370000000000001},"min":{"1min":0.745,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":4.2869999999999999,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":1.3899999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2629999999999999,"15min":1.298},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.4359999999999999,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1919999999999999,"15min":1.238},"min":{"1min":0.53100000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.748}]},{"osd":3,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.302,"15min":1.3169999999999999},"min":{"1min":0.82799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.133},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.353,"15min":1.389},"min":{"1min":0.873,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.206}]},{"osd":4,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.256,"15min":1.2170000000000001},"min":{"1min":0.747,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.359,"5min":4.359,"15min":4.359},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.377,"15min":1.3520000000000001},"min":{"1min":0.93999999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":4.9100000000000001,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.542}]},{"osd":6,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.484,"15min":1.452},"min":{"1min":1.105,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.583},{"interface":"front","average":{"1min":1.573,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.026,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.992}]},{"osd":7,"last update":"Sat Oct 21 16:19:28 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.514,"15min":1.5029999999999999},"min":{"1min":1.109,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.706},{"interface":"front","average":{"1min":1.7430000000000001,"5min":1.577,"15min":1.554},"min":{"1min":1.095,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":5.1449999999999996,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.643}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1654784,"data_stored":834978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1990000000000001,"15min":1.1619999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.647,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.2130000000000001,"15min":1.214},"min":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5960000000000001,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":1.22}]},{"osd":1,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.248,"15min":1.266},"min":{"1min":0.83399999999999996,"5min":0.621,"15min":0.621},"max":{"1min":1.6779999999999999,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.754},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.264},"min":{"1min":0.83699999999999997,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.681,"5min":7.484,"15min":7.484},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.276,"15min":1.2649999999999999},"min":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6850000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.302,"15min":1.337},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.746,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.581}]},{"osd":3,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3420000000000001,"15min":1.327},"min":{"1min":0.84899999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9430000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4359999999999999,"15min":1.393},"min":{"1min":0.98199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.466}]},{"osd":5,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.3540000000000001,"15min":1.304},"min":{"1min":0.78300000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9990000000000001,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.411},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":0.86199999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.855,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.0489999999999999}]},{"osd":6,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.532,"15min":1.4259999999999999},"min":{"1min":0.84999999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7509999999999999,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.087},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.617,"15min":1.577},"min":{"1min":1.099,"5min":0.878,"15min":0.878},"max":{"1min":2.1240000000000001,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.673}]},{"osd":7,"last update":"Sat Oct 21 16:19:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.653,"15min":1.522},"min":{"1min":1.0109999999999999,"5min":0.81000000000000005,"15min":0.69599999999999995},"max":{"1min":2.0310000000000001,"5min":12.458,"15min":12.458},"last":1.278},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.6679999999999999,"15min":1.5629999999999999},"min":{"1min":1.117,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.9299999999999999,"5min":3.968,"15min":3.968},"last":1.7}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41120,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330720,"statfs":{"total":96636764160,"available":96594657280,"internally_reserved":0,"allocated":1650688,"data_stored":835468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.92500000000000004,"15min":0.94999999999999996},"min":{"1min":0.48099999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.236,"5min":1.946,"15min":1.946},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.212,"5min":1.0940000000000001,"15min":1.1779999999999999},"min":{"1min":0.73599999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6220000000000001,"5min":1.8480000000000001,"15min":2.04},"last":1.123}]},{"osd":1,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93100000000000005,"15min":0.98399999999999999},"min":{"1min":0.68000000000000005,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.589,"5min":1.589,"15min":2.173},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.163,"15min":1.252},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.698,"5min":1.752,"15min":1.8100000000000001},"last":1.226}]},{"osd":2,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":0.997,"15min":1.056},"min":{"1min":0.64100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.669,"5min":1.669,"15min":2.0049999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.163,"15min":1.216},"min":{"1min":0.91000000000000003,"5min":0.61899999999999999,"15min":0.46600000000000003},"max":{"1min":1.6659999999999999,"5min":1.7250000000000001,"15min":2.847},"last":1.069}]},{"osd":4,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.3320000000000001,"15min":1.399},"min":{"1min":0.77800000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8700000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.548},{"interface":"front","average":{"1min":1.494,"5min":1.3620000000000001,"15min":1.3979999999999999},"min":{"1min":1.2170000000000001,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.8080000000000001,"5min":1.863,"15min":2.919},"last":1.1839999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.363,"15min":1.405},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.7629999999999999,"5min":2.02,"15min":2.02},"last":1.577},{"interface":"front","average":{"1min":1.474,"5min":1.4410000000000001,"15min":1.4910000000000001},"min":{"1min":0.91100000000000003,"5min":0.86299999999999999,"15min":0.75700000000000001},"max":{"1min":1.9279999999999999,"5min":2.0390000000000001,"15min":2.423},"last":1.446}]},{"osd":6,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":1.1819999999999999,"5min":0.84399999999999997,"15min":0.59299999999999997},"max":{"1min":1.829,"5min":2.1000000000000001,"15min":2.6179999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4610000000000001,"15min":1.5189999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8999999999999999,"5min":2.1459999999999999,"15min":2.2290000000000001},"last":1.4830000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:20:15 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.516,"15min":1.5820000000000001},"min":{"1min":0.94799999999999995,"5min":0.89700000000000002,"15min":0.83599999999999997},"max":{"1min":1.881,"5min":2.302,"15min":2.3759999999999999},"last":1.341},{"interface":"front","average":{"1min":1.595,"5min":1.544,"15min":1.5980000000000001},"min":{"1min":1.171,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.9570000000000001,"5min":2.2050000000000001,"15min":2.7989999999999999},"last":1.3049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46684,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325156,"statfs":{"total":96636764160,"available":96588959744,"internally_reserved":0,"allocated":1650688,"data_stored":834920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.85499999999999998},"min":{"1min":0.47299999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.2130000000000001,"5min":1.859,"15min":1.859},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.016,"5min":0.90800000000000003,"15min":0.98299999999999998},"min":{"1min":0.66200000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.667,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.4419999999999999}]},{"osd":1,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.0109999999999999,"15min":1.111},"min":{"1min":0.52900000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":1.9390000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.0960000000000001,"15min":1.2509999999999999},"min":{"1min":0.68300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.5640000000000001,"5min":1.881,"15min":1.881},"last":1.536}]},{"osd":3,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.0309999999999999,"15min":1.1339999999999999},"min":{"1min":0.68200000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.8839999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2470000000000001,"15min":1.3640000000000001},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8049999999999999,"5min":1.9419999999999999,"15min":2.0390000000000001},"last":1.4990000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3109999999999999,"15min":1.3680000000000001},"min":{"1min":0.85799999999999998,"5min":0.76600000000000001,"15min":0.73199999999999998},"max":{"1min":1.891,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.341},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.341,"15min":1.375},"min":{"1min":0.92800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.2570000000000001,"5min":2.6960000000000002,"15min":2.6960000000000002},"last":2.278}]},{"osd":5,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2909999999999999,"15min":1.3440000000000001},"min":{"1min":0.85399999999999998,"5min":0.747,"15min":0.747},"max":{"1min":1.7769999999999999,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4650000000000001,"15min":1.5189999999999999},"min":{"1min":0.81299999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1280000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.595}]},{"osd":6,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.458,"15min":1.484},"min":{"1min":1.1000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.444,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":2.3639999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.3759999999999999,"15min":1.4399999999999999},"min":{"1min":1.0269999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.2669999999999999,"5min":2.552,"15min":2.552},"last":1.3899999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:19:26 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.4690000000000001,"15min":1.54},"min":{"1min":1.218,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1880000000000002,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":2.2130000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4330000000000001,"15min":1.512},"min":{"1min":0.90200000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.569}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1638400,"data_stored":828256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86099999999999999,"15min":0.90300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.905},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.03,"15min":1.075},"min":{"1min":0.55100000000000005,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.601,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":1.1950000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94499999999999995,"15min":0.998},"min":{"1min":0.52600000000000002,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.429,"5min":2.222,"15min":2.2639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1180000000000001,"15min":1.171},"min":{"1min":0.69899999999999995,"5min":0.47299999999999998,"15min":0.45800000000000002},"max":{"1min":1.5589999999999999,"5min":1.6910000000000001,"15min":1.974},"last":1.1399999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.21,"15min":1.298},"min":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.623,"5min":1.911,"15min":1.9510000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.137,"15min":1.212},"min":{"1min":0.55100000000000005,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.6879999999999999,"5min":1.7889999999999999,"15min":1.9159999999999999},"last":1.466}]},{"osd":4,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.242,"15min":1.278},"min":{"1min":1.0680000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.6870000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.28},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.421},"min":{"1min":0.83499999999999996,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.907,"5min":2.145,"15min":2.2719999999999998},"last":1.647}]},{"osd":5,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.429,"15min":1.484},"min":{"1min":1.1000000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8500000000000001,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":1.679},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4159999999999999,"15min":1.466},"min":{"1min":0.82199999999999995,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.889,"5min":2.077,"15min":2.0880000000000001},"last":1.621}]},{"osd":6,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.4590000000000001,"15min":1.494},"min":{"1min":0.96499999999999997,"5min":0.749,"15min":0.68500000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.4079999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.411,"15min":1.458},"min":{"1min":1.01,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.9830000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4079999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:14 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.55,"15min":1.6020000000000001},"min":{"1min":0.88500000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9319999999999999,"5min":2.2530000000000001,"15min":2.3929999999999998},"last":1.804},{"interface":"front","average":{"1min":1.629,"5min":1.5209999999999999,"15min":1.5920000000000001},"min":{"1min":1.119,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0489999999999999,"5min":2.1850000000000001,"15min":2.367},"last":1.573}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41712,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330128,"statfs":{"total":96636764160,"available":96594051072,"internally_reserved":0,"allocated":2256896,"data_stored":1426956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":1.302,"5min":1.649,"15min":1.855},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.996,"15min":1.0549999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.589,"5min":1.589,"15min":1.667},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.45100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.679,"5min":1.708,"15min":1.7749999999999999},"last":1.21},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.1339999999999999},"min":{"1min":0.48499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5580000000000001,"5min":1.694,"15min":1.732},"last":1.04}]},{"osd":3,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0069999999999999,"15min":1.0780000000000001},"min":{"1min":0.53800000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.6080000000000001,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.244,"5min":1.1930000000000001,"15min":1.2769999999999999},"min":{"1min":0.58399999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7969999999999999,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":1.089}]},{"osd":4,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3819999999999999,"15min":1.4039999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.9019999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4550000000000001,"15min":1.516},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8029999999999999,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.431}]},{"osd":5,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.329,"15min":1.3660000000000001},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.823,"5min":1.9419999999999999,"15min":2.137},"last":1.284},{"interface":"front","average":{"1min":1.425,"5min":1.3879999999999999,"15min":1.4790000000000001},"min":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.9259999999999999,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":0.998}]},{"osd":6,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.954,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.4870000000000001,"15min":1.5720000000000001},"min":{"1min":0.72099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.911,"5min":3.097,"15min":3.097},"last":1.2649999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.925,"5min":2.085,"15min":2.2959999999999998},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.464,"15min":1.5509999999999999},"min":{"1min":0.79000000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0600000000000001,"5min":2.077,"15min":2.3540000000000001},"last":1.3839999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:20:30.513 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:20:30.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:20:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.792+0000 7f0e755a0640 1 Processor -- start 2023-10-21T16:20:30.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.793+0000 7f0e755a0640 1 -- start start 2023-10-21T16:20:30.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.794+0000 7f0e755a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e70069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.794+0000 7f0e755a0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7015aec0 con 0x7f0e7015aae0 2023-10-21T16:20:30.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.794+0000 7f0e755a0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e700cb600 con 0x7f0e7006a290 2023-10-21T16:20:30.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.794+0000 7f0e755a0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7006a720 con 0x7f0e700c9c80 2023-10-21T16:20:30.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.795+0000 7f0e6ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0e7015aae0 0x7f0e700af9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46090/0 (socket says 172.21.15.53:46090) 2023-10-21T16:20:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.795+0000 7f0e6ffff640 1 -- 172.21.15.53:0/2807210988 learned_addr learned my addr 172.21.15.53:0/2807210988 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.795+0000 7f0e6f7fe640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e70069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.795+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350126105 0 0) 0x7f0e7015aec0 con 0x7f0e7015aae0 2023-10-21T16:20:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.795+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e58003520 con 0x7f0e7015aae0 2023-10-21T16:20:30.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1826837816 0 0) 0x7f0e58003520 con 0x7f0e7015aae0 2023-10-21T16:20:30.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 msgr2=0x7f0e70069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:30.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e70069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 >> v1:172.21.15.124:6789/0 conn(0x7f0e7006a290 legacy=0x7f0e700b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700cb8e0 con 0x7f0e7015aae0 2023-10-21T16:20:30.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0e600017f0 con 0x7f0e7015aae0 2023-10-21T16:20:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 >> v1:172.21.15.53:6789/0 conn(0x7f0e7015aae0 legacy=0x7f0e700af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e580039a0 0x7f0e58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e580047c0 0x7f0e58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e58005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7006a720 con 0x7f0e580039a0 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5800bb10 con 0x7f0e580047c0 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5800bcd0 con 0x7f0e700c9c80 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.796+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e580039a0 0x7f0e58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6f7fe640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e580047c0 0x7f0e58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e74d9f640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e58005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6ffff640 1 -- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 msgr2=0x7f0e58005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:30.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e700c9c80 0x7f0e58005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6ffff640 1 -- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e580047c0 msgr2=0x7f0e58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:30.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e580047c0 0x7f0e58004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.797+0000 7f0e6ffff640 1 -- 172.21.15.53:0/2807210988 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58003520 con 0x7f0e580039a0 2023-10-21T16:20:30.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e580039a0 0x7f0e58003dc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007fb0 tx=0x7f0e580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d20848d30ee57 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60007270 con 0x7f0e580039a0 2023-10-21T16:20:30.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e600073f0 con 0x7f0e580039a0 2023-10-21T16:20:30.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e6effd640 1 -- 172.21.15.53:0/2807210988 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6002c7c0 con 0x7f0e580039a0 2023-10-21T16:20:30.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e755a0640 1 -- 172.21.15.53:0/2807210988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e580039a0 msgr2=0x7f0e58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.799+0000 7f0e755a0640 1 --2- 172.21.15.53:0/2807210988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e580039a0 0x7f0e58003dc0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007fb0 tx=0x7f0e580057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.800+0000 7f0e74d9f640 1 -- 172.21.15.53:0/2807210988 reap_dead start 2023-10-21T16:20:30.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.800+0000 7f0e755a0640 1 -- 172.21.15.53:0/2807210988 shutdown_connections 2023-10-21T16:20:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.800+0000 7f0e755a0640 1 -- 172.21.15.53:0/2807210988 >> 172.21.15.53:0/2807210988 conn(0x7f0e70098a50 msgr2=0x7f0e700cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.800+0000 7f0e755a0640 1 -- 172.21.15.53:0/2807210988 shutdown_connections 2023-10-21T16:20:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.800+0000 7f0e755a0640 1 -- 172.21.15.53:0/2807210988 wait complete. 2023-10-21T16:20:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.802+0000 7f0e755a0640 1 Processor -- start 2023-10-21T16:20:30.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.802+0000 7f0e755a0640 1 -- start start 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 0x7f0e701f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e701ee000 0x7f0e701ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e700b1570 con 0x7f0e700b0bb0 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701f15d0 con 0x7f0e70152fb0 2023-10-21T16:20:30.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e755a0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701eeb10 con 0x7f0e701ee000 2023-10-21T16:20:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e74d9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e74d9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37896/0 (socket says 172.21.15.53:37896) 2023-10-21T16:20:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e74d9f640 1 -- 172.21.15.53:0/1697288596 learned_addr learned my addr 172.21.15.53:0/1697288596 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:30.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 0x7f0e701f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e6f7fe640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e701ee000 0x7f0e701ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.803+0000 7f0e74d9f640 1 -- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e701ee000 msgr2=0x7f0e701ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e74d9f640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e701ee000 0x7f0e701ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e74d9f640 1 -- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 msgr2=0x7f0e701f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e74d9f640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 0x7f0e701f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:30.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e74d9f640 1 -- 172.21.15.53:0/1697288596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e701eedf0 con 0x7f0e700b0bb0 2023-10-21T16:20:30.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 0x7f0e701f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:30.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e74d9f640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0e7009d540 tx=0x7f0e6800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68003970 con 0x7f0e700b0bb0 2023-10-21T16:20:30.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68003ad0 con 0x7f0e700b0bb0 2023-10-21T16:20:30.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68003e80 con 0x7f0e700b0bb0 2023-10-21T16:20:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e701f1890 con 0x7f0e700b0bb0 2023-10-21T16:20:30.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.804+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e700c5be0 con 0x7f0e700b0bb0 2023-10-21T16:20:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.806+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e701eef80 con 0x7f0e700b0bb0 2023-10-21T16:20:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.806+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0e680050d0 con 0x7f0e700b0bb0 2023-10-21T16:20:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.807+0000 7f0e6cff9640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 0x7f0e5c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.808+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e68089740 con 0x7f0e700b0bb0 2023-10-21T16:20:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.808+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 0x7f0e5c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.809+0000 7f0e6ffff640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 0x7f0e5c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007fb0 tx=0x7f0e60007000 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:30.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:30.812+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e680544d0 con 0x7f0e700b0bb0 2023-10-21T16:20:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.078+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e7004e160 con 0x7f0e700b0bb0 2023-10-21T16:20:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.082+0000 7f0e6cff9640 1 -- 172.21.15.53:0/1697288596 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0e68057af0 con 0x7f0e700b0bb0 2023-10-21T16:20:31.084 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:20:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:20:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:20:31.084 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:20:31.085 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:20:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:20:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.085+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 msgr2=0x7f0e5c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.085+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 0x7f0e5c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e60007fb0 tx=0x7f0e60007000 comp rx=0 tx=0).stop 2023-10-21T16:20:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.085+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 msgr2=0x7f0e700b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.085+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f0e7009d540 tx=0x7f0e6800ec10 comp rx=0 tx=0).stop 2023-10-21T16:20:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 shutdown_connections 2023-10-21T16:20:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f0e5c06afc0 0x7f0e5c06d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e700b0bb0 0x7f0e700b1030 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e701ee000 0x7f0e701ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 --2- 172.21.15.53:0/1697288596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e70152fb0 0x7f0e701f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 >> 172.21.15.53:0/1697288596 conn(0x7f0e70098a50 msgr2=0x7f0e700aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 shutdown_connections 2023-10-21T16:20:31.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.086+0000 7f0e755a0640 1 -- 172.21.15.53:0/1697288596 wait complete. 2023-10-21T16:20:31.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:20:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.322+0000 7f91994fc640 1 Processor -- start 2023-10-21T16:20:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.322+0000 7f91994fc640 1 -- start start 2023-10-21T16:20:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91994fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91940cc210 0x7f91940b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91994fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9194159db0 0x7f91940b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91994fc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940cacf0 con 0x7f91941599d0 2023-10-21T16:20:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91994fc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91940b10f0 con 0x7f91940cc210 2023-10-21T16:20:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91994fc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940ade40 con 0x7f9194159db0 2023-10-21T16:20:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91937fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91940cc210 0x7f91940b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91937fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91940cc210 0x7f91940b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37912/0 (socket says 172.21.15.53:37912) 2023-10-21T16:20:31.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.324+0000 7f91937fe640 1 -- 172.21.15.53:0/2530446365 learned_addr learned my addr 172.21.15.53:0/2530446365 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:31.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.325+0000 7f9198cfb640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9194159db0 0x7f91940b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.325+0000 7f91937fe640 1 -- 172.21.15.53:0/2530446365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9194159db0 msgr2=0x7f91940b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.325+0000 7f91937fe640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9194159db0 0x7f91940b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.325+0000 7f91937fe640 1 -- 172.21.15.53:0/2530446365 >> v1:172.21.15.124:6789/0 conn(0x7f91941599d0 legacy=0x7f91940ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.325+0000 7f91937fe640 1 -- 172.21.15.53:0/2530446365 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940ca3c0 con 0x7f91940cc210 2023-10-21T16:20:31.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.326+0000 7f91937fe640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91940cc210 0x7f91940b00c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9188009870 tx=0x7f918802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d301ef137be97f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.326+0000 7f9192ffd640 1 -- 172.21.15.53:0/2530446365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031640 con 0x7f91940cc210 2023-10-21T16:20:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.326+0000 7f9192ffd640 1 -- 172.21.15.53:0/2530446365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880317a0 con 0x7f91940cc210 2023-10-21T16:20:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.326+0000 7f9192ffd640 1 -- 172.21.15.53:0/2530446365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031b30 con 0x7f91940cc210 2023-10-21T16:20:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 -- 172.21.15.53:0/2530446365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91940cc210 msgr2=0x7f91940b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91940cc210 0x7f91940b00c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9188009870 tx=0x7f918802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 -- 172.21.15.53:0/2530446365 shutdown_connections 2023-10-21T16:20:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9194159db0 0x7f91940b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 --2- 172.21.15.53:0/2530446365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91940cc210 0x7f91940b00c0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 -- 172.21.15.53:0/2530446365 >> 172.21.15.53:0/2530446365 conn(0x7f91940cb0c0 msgr2=0x7f91940cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:31.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 -- 172.21.15.53:0/2530446365 shutdown_connections 2023-10-21T16:20:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.327+0000 7f91994fc640 1 -- 172.21.15.53:0/2530446365 wait complete. 2023-10-21T16:20:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.329+0000 7f91994fc640 1 Processor -- start 2023-10-21T16:20:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.329+0000 7f91994fc640 1 -- start start 2023-10-21T16:20:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91940cc210 0x7f9194150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91941599d0 0x7f9194153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940b3b00 con 0x7f9194159db0 2023-10-21T16:20:31.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940b2790 con 0x7f91940cc210 2023-10-21T16:20:31.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f91994fc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940b2a70 con 0x7f91941599d0 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37916/0 (socket says 172.21.15.53:37916) 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 -- 172.21.15.53:0/3985325465 learned_addr learned my addr 172.21.15.53:0/3985325465 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 -- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91941599d0 msgr2=0x7f9194153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91941599d0 0x7f9194153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 -- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91940cc210 msgr2=0x7f9194150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91940cc210 0x7f9194150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.330+0000 7f9198cfb640 1 -- 172.21.15.53:0/3985325465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880094d0 con 0x7f9194159db0 2023-10-21T16:20:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f9198cfb640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f918c007d40 tx=0x7f918c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c014420 con 0x7f9194159db0 2023-10-21T16:20:31.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c0145a0 con 0x7f9194159db0 2023-10-21T16:20:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91940b2d50 con 0x7f9194159db0 2023-10-21T16:20:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c014950 con 0x7f9194159db0 2023-10-21T16:20:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.331+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940b3310 con 0x7f9194159db0 2023-10-21T16:20:31.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.332+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9194066700 con 0x7f9194159db0 2023-10-21T16:20:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.333+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f918c014ab0 con 0x7f9194159db0 2023-10-21T16:20:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.334+0000 7f9190ff9640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 0x7f918006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.334+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f918c08a260 con 0x7f9194159db0 2023-10-21T16:20:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.335+0000 7f9193fff640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 0x7f918006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.336+0000 7f9193fff640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 0x7f918006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9184003cc0 tx=0x7f91840079b0 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:31.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.338+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f918c054ff0 con 0x7f9194159db0 2023-10-21T16:20:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.570+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91941552f0 con 0x7f9194159db0 2023-10-21T16:20:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.572+0000 7f9190ff9640 1 -- 172.21.15.53:0/3985325465 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f918c058610 con 0x7f9194159db0 2023-10-21T16:20:31.573 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:31.574 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:20:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:20:31.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 msgr2=0x7f918006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 0x7f918006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9184003cc0 tx=0x7f91840079b0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 msgr2=0x7f91941e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:31.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f918c007d40 tx=0x7f918c00e4b0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 shutdown_connections 2023-10-21T16:20:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f918006af90 0x7f918006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194159db0 0x7f91941e66b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91941599d0 0x7f9194153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 --2- 172.21.15.53:0/3985325465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91940cc210 0x7f9194150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:31.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 >> 172.21.15.53:0/3985325465 conn(0x7f91940cb0c0 msgr2=0x7f91940701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 shutdown_connections 2023-10-21T16:20:31.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:31.575+0000 7f91994fc640 1 -- 172.21.15.53:0/3985325465 wait complete. 2023-10-21T16:20:31.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:20:24.920100+0000', 'modified': '2023-10-21T16:20:28.035596+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8085}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8085': {'gid': 8085, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/206156413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 206156413}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 206156413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:20:31.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:20:31.997 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-21T16:20:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.225+0000 7f8a0f690640 1 Processor -- start 2023-10-21T16:20:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.225+0000 7f8a0f690640 1 -- start start 2023-10-21T16:20:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.226+0000 7f8a0f690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.226+0000 7f8a0f690640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8a080cdda0 0x7f8a080d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.226+0000 7f8a0f690640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0815d160 con 0x7f8a0815cd80 2023-10-21T16:20:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.226+0000 7f8a0f690640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a080ce230 con 0x7f8a080cdda0 2023-10-21T16:20:32.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.226+0000 7f8a0f690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080b7b90 con 0x7f8a080b87d0 2023-10-21T16:20:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.227+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.227+0000 7f8a0dc2d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8a080cdda0 0x7f8a080d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.227+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:45600/0 (socket says 172.21.15.53:45600) 2023-10-21T16:20:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.227+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/28132358 learned_addr learned my addr 172.21.15.53:0/28132358 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.228+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 msgr2=0x7f8a080cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.228+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.228+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/28132358 >> v1:172.21.15.124:6789/0 conn(0x7f8a0815cd80 legacy=0x7f8a080b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.228+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/28132358 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080b8020 con 0x7f8a080cdda0 2023-10-21T16:20:32.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.228+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:20:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.229+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080d0ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a000097a0 tx=0x7f8a0002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea59759d050f18e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.230+0000 7f8a07fff640 1 -- 172.21.15.53:0/28132358 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0002d040 con 0x7f8a080cdda0 2023-10-21T16:20:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.230+0000 7f8a07fff640 1 -- 172.21.15.53:0/28132358 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00031810 con 0x7f8a080cdda0 2023-10-21T16:20:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.230+0000 7f8a07fff640 1 -- 172.21.15.53:0/28132358 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00031ba0 con 0x7f8a080cdda0 2023-10-21T16:20:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 -- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 msgr2=0x7f8a080d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080d0ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a000097a0 tx=0x7f8a0002b8b0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 -- 172.21.15.53:0/28132358 shutdown_connections 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a080b87d0 0x7f8a080cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 --2- 172.21.15.53:0/28132358 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080d0ed0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 -- 172.21.15.53:0/28132358 >> 172.21.15.53:0/28132358 conn(0x7f8a0806b5d0 msgr2=0x7f8a0806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 -- 172.21.15.53:0/28132358 shutdown_connections 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.231+0000 7f8a0f690640 1 -- 172.21.15.53:0/28132358 wait complete. 2023-10-21T16:20:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.233+0000 7f8a0f690640 1 Processor -- start 2023-10-21T16:20:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.233+0000 7f8a0f690640 1 -- start start 2023-10-21T16:20:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 0x7f8a080c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08157ce0 con 0x7f8a080b87d0 2023-10-21T16:20:32.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08157160 con 0x7f8a080cdda0 2023-10-21T16:20:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0f690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08157440 con 0x7f8a0815cd80 2023-10-21T16:20:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0d42c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0d42c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37928/0 (socket says 172.21.15.53:37928) 2023-10-21T16:20:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0d42c640 1 -- 172.21.15.53:0/3549080535 learned_addr learned my addr 172.21.15.53:0/3549080535 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.234+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 0x7f8a080c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 -- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 msgr2=0x7f8a080c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 0x7f8a080c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 -- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 msgr2=0x7f8a080c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 -- 172.21.15.53:0/3549080535 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0094d0 con 0x7f8a080b87d0 2023-10-21T16:20:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 0x7f8a080c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c870 tx=0x7f89f800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a05ffb640 1 -- 172.21.15.53:0/3549080535 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003b70 con 0x7f8a080b87d0 2023-10-21T16:20:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a05ffb640 1 -- 172.21.15.53:0/3549080535 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8003cf0 con 0x7f8a080b87d0 2023-10-21T16:20:32.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.235+0000 7f8a05ffb640 1 -- 172.21.15.53:0/3549080535 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800aa90 con 0x7f8a080b87d0 2023-10-21T16:20:32.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.236+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a00009400 con 0x7f8a080b87d0 2023-10-21T16:20:32.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.236+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08157a00 con 0x7f8a080b87d0 2023-10-21T16:20:32.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.239+0000 7f8a05ffb640 1 -- 172.21.15.53:0/3549080535 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f89f800abf0 con 0x7f8a080b87d0 2023-10-21T16:20:32.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.240+0000 7f8a05ffb640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 0x7f89f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.240+0000 7f8a05ffb640 1 -- 172.21.15.53:0/3549080535 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f89f80897d0 con 0x7f8a080b87d0 2023-10-21T16:20:32.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.240+0000 7f8a12222740 1 Processor -- start 2023-10-21T16:20:32.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.240+0000 7f8a12222740 1 -- start start 2023-10-21T16:20:32.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 0x7f89f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 0x5561a71bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a12222740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 0x5561a71ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a12222740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a70de7a0 con 0x5561a71eb360 2023-10-21T16:20:32.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.241+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7130740 con 0x5561a71eaf80 2023-10-21T16:20:32.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a712de30 con 0x5561a71be3c0 2023-10-21T16:20:32.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37934/0 (socket says 172.21.15.53:37934) 2023-10-21T16:20:32.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/2480785559 learned_addr learned my addr 172.21.15.53:0/2480785559 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 0x7f89f406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a081548f0 tx=0x7f89fc008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 0x5561a71bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 0x5561a71ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 msgr2=0x5561a71ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 0x5561a71ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 msgr2=0x5561a71bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 0x5561a71bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/2480785559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a70ed910 con 0x5561a71eb360 2023-10-21T16:20:32.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 0x5561a71bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:32.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.242+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 0x5561a71ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:32.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.243+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009f10 tx=0x7f89fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bda4e362336dae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.243+0000 7f89eeffd640 1 -- 172.21.15.53:0/2480785559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc015e50 con 0x5561a71eb360 2023-10-21T16:20:32.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.243+0000 7f89eeffd640 1 -- 172.21.15.53:0/2480785559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc021440 con 0x5561a71eb360 2023-10-21T16:20:32.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.243+0000 7f89eeffd640 1 -- 172.21.15.53:0/2480785559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0217b0 con 0x5561a71eb360 2023-10-21T16:20:32.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 -- 172.21.15.53:0/2480785559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 msgr2=0x5561a71edd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009f10 tx=0x7f89fc014000 comp rx=0 tx=0).stop 2023-10-21T16:20:32.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 -- 172.21.15.53:0/2480785559 shutdown_connections 2023-10-21T16:20:32.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71be3c0 0x5561a71ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71edd10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 --2- 172.21.15.53:0/2480785559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71eaf80 0x5561a71bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 -- 172.21.15.53:0/2480785559 >> 172.21.15.53:0/2480785559 conn(0x5561a71c0190 msgr2=0x5561a7208d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 -- 172.21.15.53:0/2480785559 shutdown_connections 2023-10-21T16:20:32.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.244+0000 7f8a12222740 1 -- 172.21.15.53:0/2480785559 wait complete. 2023-10-21T16:20:32.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.245+0000 7f8a12222740 1 -- start start 2023-10-21T16:20:32.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.245+0000 7f8a12222740 1 Processor -- start 2023-10-21T16:20:32.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a71db6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7129720 con 0x5561a71eb360 2023-10-21T16:20:32.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a70c7970 con 0x5561a71d9800 2023-10-21T16:20:32.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a70c7610 con 0x5561a71eaf80 2023-10-21T16:20:32.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a12222740 10 client.0 _ll_register_callbacks cb 0x5561a70e99d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:20:32.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37948/0 (socket says 172.21.15.53:37948) 2023-10-21T16:20:32.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1749983272 learned_addr learned my addr 172.21.15.53:0/1749983272 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.246+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a71db6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 msgr2=0x5561a71db6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a71db6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 msgr2=0x5561a71debf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71debf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1749983272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc009c50 con 0x5561a71eb360 2023-10-21T16:20:32.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009a30 tx=0x7f89fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d811118f30eef01a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f89e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:20:32.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc015e50 con 0x5561a71eb360 2023-10-21T16:20:32.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f89fc021c60 con 0x5561a71eb360 2023-10-21T16:20:32.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.247+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a7150160 con 0x5561a71eb360 2023-10-21T16:20:32.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89fc021e20 con 0x5561a71eb360 2023-10-21T16:20:32.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 10 client.8220 fetch_fsmap learned FSMap version 61 2023-10-21T16:20:32.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5561a7127da0 con 0x5561a71eb360 2023-10-21T16:20:32.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561a7150160 con 0x5561a71eb360 2023-10-21T16:20:32.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89fc035430 con 0x5561a71eb360 2023-10-21T16:20:32.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 10 client.8220 fetch_fsmap finished waiting for FSMap version 61 2023-10-21T16:20:32.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 10 client.8220 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:20:32.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 20 client.8220 populate_metadata read hostname 'smithi053' 2023-10-21T16:20:32.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71ea520 0x5561a71efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 4 client.8220 mds_command: new command op to 8085 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.248+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 --> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a70fbcc0 con 0x5561a71ea520 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.249+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71ea520 0x5561a71efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.249+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71ea520 0x5561a71efd50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9595281690a692d server_cookie=342df8321b4c0d46 in_seq=0 out_seq=0 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.249+0000 7f89e3fff640 10 client.8220 ms_handle_connect on v2:172.21.15.53:6834/206156413 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f89e3fff640 1 -- 172.21.15.53:0/1749983272 <== mds.0 v2:172.21.15.53:6834/206156413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561a70fbcc0 con 0x5561a71ea520 2023-10-21T16:20:32.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f89e3fff640 10 client.8220 handle_command_reply: tid=0 2023-10-21T16:20:32.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f8a12222740 1 client.8220 shutdown 2023-10-21T16:20:32.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 msgr2=0x5561a71d92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009a30 tx=0x7f89fc0046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 shutdown_connections 2023-10-21T16:20:32.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.250+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71ea520 0x5561a71efd50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71debf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eb360 0x5561a71d92c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 --2- 172.21.15.53:0/1749983272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a71db6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 >> 172.21.15.53:0/1749983272 conn(0x5561a71be3c0 msgr2=0x5561a71c1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f89e3fff640 0 client.8220 ms_handle_reset on v2:172.21.15.53:6834/206156413 2023-10-21T16:20:32.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 shutdown_connections 2023-10-21T16:20:32.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 1 -- 172.21.15.53:0/1749983272 wait complete. 2023-10-21T16:20:32.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.251+0000 7f8a12222740 20 client.8220 trim_cache size 0 max 16384 2023-10-21T16:20:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.267+0000 7f8a12222740 1 Processor -- start 2023-10-21T16:20:32.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.267+0000 7f8a12222740 1 -- start start 2023-10-21T16:20:32.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a7203b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a6e8df20 con 0x5561a72040b0 2023-10-21T16:20:32.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a722b880 con 0x5561a71d9800 2023-10-21T16:20:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a71ed8f0 con 0x5561a71eaf80 2023-10-21T16:20:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a0dc2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a0dc2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37958/0 (socket says 172.21.15.53:37958) 2023-10-21T16:20:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.268+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/3787774456 learned_addr learned my addr 172.21.15.53:0/3787774456 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a7203b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 msgr2=0x5561a7203b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a7203b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 msgr2=0x5561a71c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 -- 172.21.15.53:0/3787774456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a70c6b80 con 0x5561a72040b0 2023-10-21T16:20:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8a000098c0 tx=0x7f8a00030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a595f1c5981706b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f89ef7fe640 1 -- 172.21.15.53:0/3787774456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00031c00 con 0x5561a72040b0 2023-10-21T16:20:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.269+0000 7f89ef7fe640 1 -- 172.21.15.53:0/3787774456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00031d60 con 0x5561a72040b0 2023-10-21T16:20:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f89ef7fe640 1 -- 172.21.15.53:0/3787774456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0003a640 con 0x5561a72040b0 2023-10-21T16:20:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 -- 172.21.15.53:0/3787774456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 msgr2=0x5561a71d7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8a000098c0 tx=0x7f8a00030000 comp rx=0 tx=0).stop 2023-10-21T16:20:32.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 -- 172.21.15.53:0/3787774456 shutdown_connections 2023-10-21T16:20:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a72040b0 0x5561a71d7fe0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a71eaf80 0x5561a7203b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 --2- 172.21.15.53:0/3787774456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a71c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 -- 172.21.15.53:0/3787774456 >> 172.21.15.53:0/3787774456 conn(0x5561a71be3c0 msgr2=0x5561a720c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 -- 172.21.15.53:0/3787774456 shutdown_connections 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.270+0000 7f8a12222740 1 -- 172.21.15.53:0/3787774456 wait complete. 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.271+0000 7f8a12222740 1 -- start start 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.271+0000 7f8a12222740 1 Processor -- start 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a726e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a72040b0 0x5561a71c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a7275030 con 0x5561a71eaf80 2023-10-21T16:20:32.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a71bd990 con 0x5561a71d9800 2023-10-21T16:20:32.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a7274af0 con 0x5561a72040b0 2023-10-21T16:20:32.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a12222740 10 client.0 _ll_register_callbacks cb 0x7f89fc009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:20:32.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a0cc2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37962/0 (socket says 172.21.15.53:37962) 2023-10-21T16:20:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.272+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1967821225 learned_addr learned my addr 172.21.15.53:0/1967821225 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0dc2d640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a72040b0 0x5561a71c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a72040b0 msgr2=0x5561a71c2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a72040b0 0x5561a71c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a726e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 msgr2=0x5561a726e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a726e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 -- 172.21.15.53:0/1967821225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a080b8020 con 0x5561a71eaf80 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a0cc2b640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009f10 tx=0x7f89fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed187ef1120dca00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f89e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:20:32.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc015be0 con 0x5561a71eaf80 2023-10-21T16:20:32.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f89fc015d40 con 0x5561a71eaf80 2023-10-21T16:20:32.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89fc021440 con 0x5561a71eaf80 2023-10-21T16:20:32.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89fc0218c0 con 0x5561a71eaf80 2023-10-21T16:20:32.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a12222740 10 client.8226 fetch_fsmap learned FSMap version 61 2023-10-21T16:20:32.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.273+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5561a70ec170 con 0x5561a71eaf80 2023-10-21T16:20:32.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc021440 con 0x5561a71eaf80 2023-10-21T16:20:32.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89fc021a20 con 0x5561a71eaf80 2023-10-21T16:20:32.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 10 client.8226 fetch_fsmap finished waiting for FSMap version 61 2023-10-21T16:20:32.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 10 client.8226 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T16:20:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 20 client.8226 populate_metadata read hostname 'smithi053' 2023-10-21T16:20:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71c2db0 0x5561a71c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 4 client.8226 mds_command: new command op to 8085 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:20:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 --> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5561a7122700 con 0x5561a71c2db0 2023-10-21T16:20:32.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.274+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71c2db0 0x5561a71c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a0d42c640 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71c2db0 0x5561a71c3190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b224b061c4dfe640 server_cookie=45542f269e77a257 in_seq=0 out_seq=0 2023-10-21T16:20:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f89e3fff640 10 client.8226 ms_handle_connect on v2:172.21.15.53:6834/206156413 2023-10-21T16:20:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f89e3fff640 1 -- 172.21.15.53:0/1967821225 <== mds.0 v2:172.21.15.53:6834/206156413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5561a7122700 con 0x5561a71c2db0 2023-10-21T16:20:32.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f89e3fff640 10 client.8226 handle_command_reply: tid=0 2023-10-21T16:20:32.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 client.8226 shutdown 2023-10-21T16:20:32.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 msgr2=0x5561a71c1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009f10 tx=0x7f89fc016000 comp rx=0 tx=0).stop 2023-10-21T16:20:32.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 shutdown_connections 2023-10-21T16:20:32.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:6834/206156413,v1:172.21.15.53:6835/206156413] conn(0x5561a71c2db0 0x5561a71c3190 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5561a72040b0 0x5561a71c2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5561a71eaf80 0x5561a71c1b20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 --2- 172.21.15.53:0/1967821225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5561a71d9800 0x5561a726e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 >> 172.21.15.53:0/1967821225 conn(0x5561a71be3c0 msgr2=0x5561a721b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f89e3fff640 0 client.8226 ms_handle_reset on v2:172.21.15.53:6834/206156413 2023-10-21T16:20:32.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 shutdown_connections 2023-10-21T16:20:32.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 1 -- 172.21.15.53:0/1967821225 wait complete. 2023-10-21T16:20:32.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.275+0000 7f8a12222740 20 client.8226 trim_cache size 0 max 16384 2023-10-21T16:20:32.290 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 8085, 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:20:32.291 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:20:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.7833416990000002, 2023-10-21T16:20:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 61, 2023-10-21T16:20:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 133, 2023-10-21T16:20:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 133, 2023-10-21T16:20:32.292 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.139514358 2023-10-21T16:20:32.293 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:20:32.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 msgr2=0x7f89f406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 0x7f89f406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a081548f0 tx=0x7f89fc008040 comp rx=0 tx=0).stop 2023-10-21T16:20:32.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 msgr2=0x7f8a08150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c870 tx=0x7f89f800cd30 comp rx=0 tx=0).stop 2023-10-21T16:20:32.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 shutdown_connections 2023-10-21T16:20:32.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f89f406af90 0x7f89f406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a0815cd80 0x7f8a080c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a080cdda0 0x7f8a080c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 --2- 172.21.15.53:0/3549080535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a080b87d0 0x7f8a08150aa0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 >> 172.21.15.53:0/3549080535 conn(0x7f8a0806b5d0 msgr2=0x7f8a080d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 shutdown_connections 2023-10-21T16:20:32.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.276+0000 7f8a0f690640 1 -- 172.21.15.53:0/3549080535 wait complete. 2023-10-21T16:20:32.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:20:32.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.499+0000 7fd5bbfff640 1 Processor -- start 2023-10-21T16:20:32.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.499+0000 7fd5bbfff640 1 -- start start 2023-10-21T16:20:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bbfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc06cfd0 con 0x7fd5bc15a970 2023-10-21T16:20:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bbfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc06d870 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc15ad50 con 0x7fd5bc06dd00 2023-10-21T16:20:32.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37978/0 (socket says 172.21.15.53:37978) 2023-10-21T16:20:32.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2774563025 learned_addr learned my addr 172.21.15.53:0/2774563025 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5baffd640 1 --1- 172.21.15.53:0/2774563025 >> v1:172.21.15.53:6789/0 conn(0x7fd5bc15a970 0x7fd5bc0cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46094/0 (socket says 172.21.15.53:46094) 2023-10-21T16:20:32.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2774563025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 msgr2=0x7fd5bc0ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bb7fe640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2774563025 >> v1:172.21.15.53:6789/0 conn(0x7fd5bc15a970 legacy=0x7fd5bc0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5b9ffb640 1 -- 172.21.15.53:0/2774563025 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463943283 0 0) 0x7fd5bc06cfd0 con 0x7fd5bc15a970 2023-10-21T16:20:32.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2774563025 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0cac20 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.501+0000 7fd5bb7fe640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:32.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5ba7fc640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9993540fc89ce5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5b9ffb640 1 -- 172.21.15.53:0/2774563025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00317c0 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5b9ffb640 1 -- 172.21.15.53:0/2774563025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0031920 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5b9ffb640 1 -- 172.21.15.53:0/2774563025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0031cb0 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2774563025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 msgr2=0x7fd5bc0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.503+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002b860 comp rx=0 tx=0).stop 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2774563025 shutdown_connections 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2774563025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c9e60 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2774563025 >> 172.21.15.53:0/2774563025 conn(0x7fd5bc0ce9d0 msgr2=0x7fd5bc0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2774563025 shutdown_connections 2023-10-21T16:20:32.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.504+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2774563025 wait complete. 2023-10-21T16:20:32.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.505+0000 7fd5bbfff640 1 Processor -- start 2023-10-21T16:20:32.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.506+0000 7fd5bbfff640 1 -- start start 2023-10-21T16:20:32.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.506+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.506+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5bc15a970 0x7fd5bc0c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5bbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0ce210 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5bbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0cdd80 con 0x7fd5bc15a970 2023-10-21T16:20:32.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5bbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0c5430 con 0x7fd5bc06dd00 2023-10-21T16:20:32.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37982/0 (socket says 172.21.15.53:37982) 2023-10-21T16:20:32.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2552789122 learned_addr learned my addr 172.21.15.53:0/2552789122 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:32.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 msgr2=0x7fd5bc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5bc15a970 msgr2=0x7fd5bc0c6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5bc15a970 0x7fd5bc0c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.507+0000 7fd5ba7fc640 1 -- 172.21.15.53:0/2552789122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00094d0 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5ba7fc640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0000c00 tx=0x7fd5b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0004230 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc0cd170 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc0cd730 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0004390 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.508+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003810 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.509+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c005250 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.510+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd5b0043070 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.511+0000 7fd5a37fe640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 0x7fd5a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:32.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.511+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd5b00ab8d0 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.512+0000 7fd5baffd640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 0x7fd5a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:32.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.513+0000 7fd5baffd640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 0x7fd5a806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007fd0 tx=0x7fd5ac007960 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:32.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.516+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5b0076700 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.747+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd59c0057d0 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.748+0000 7fd5a37fe640 1 -- 172.21.15.53:0/2552789122 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd5b003ae60 con 0x7fd5bc0cbb60 2023-10-21T16:20:32.749 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:20:32.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 msgr2=0x7fd5a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 0x7fd5a806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007fd0 tx=0x7fd5ac007960 comp rx=0 tx=0).stop 2023-10-21T16:20:32.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 msgr2=0x7fd5bc0c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:32.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0000c00 tx=0x7fd5b00046c0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 shutdown_connections 2023-10-21T16:20:32.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7fd5a806af90 0x7fd5a806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5bc15a970 0x7fd5bc0c6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5bc0cbb60 0x7fd5bc0c4ef0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 --2- 172.21.15.53:0/2552789122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5bc06dd00 0x7fd5bc0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:32.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 >> 172.21.15.53:0/2552789122 conn(0x7fd5bc0ce9d0 msgr2=0x7fd5bc0702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:32.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.750+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 shutdown_connections 2023-10-21T16:20:32.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:32.751+0000 7fd5bbfff640 1 -- 172.21.15.53:0/2552789122 wait complete. 2023-10-21T16:20:32.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:20:32.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:20:32.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:20:32.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:20:32.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:20:32.774 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:20:32.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:20:32.774 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:20:32.775 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:20:32.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:32.776 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:20:32.831 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:20:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:20:32.831 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:20:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:20:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.832 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:20:32.832 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:20:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:20:32.833 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:20:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:20:32.833 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:20:32.833 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:20:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:20:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:20:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:20:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41260sec preferred_lft 41260sec 2023-10-21T16:20:32.834 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:20:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.835 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:20:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:20:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:20:32.835 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.836 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:20:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:20:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:20:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.837 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:20:32.837 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:20:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:20:32.838 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:20:32.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:32.839 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:20:32.931 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:20:32.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:32.936 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:20:32.993 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:20:32.993 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:20:32.993 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:20:32.993 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:20:32.994 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:20:32.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:32.995 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:20:32.995 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:20:32.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:20:32.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:20:32.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:20:32.996 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:20:33.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:33.202 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:20:33.257 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:20:33.258 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:20:33.259 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:20:33.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:33.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:20:33.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:20:33.362 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:20:33.421 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:20:33.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:20:33.422 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:20:33.480 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:20:33.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:33.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:20:33.622 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:20:33.623 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:20:33.657 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:20:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:20:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:20:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:20:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:20:33.659 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:20:33.660 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:20:33.660 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:20:33.660 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:20:33.660 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:20:33.664 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:20:33.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:33.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:20:33.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:33.756 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:20:33.828 INFO:teuthology.orchestra.run.smithi053.stdout:client36272 2023-10-21T16:20:33.829 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:20:33.829 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:20:33.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:20:34.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.108+0000 7f02c3fff640 1 Processor -- start 2023-10-21T16:20:34.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.108+0000 7f02c3fff640 1 -- start start 2023-10-21T16:20:34.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c3fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f02c40b3c50 0x7f02c40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c3fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c4159db0 con 0x7f02c40cb5e0 2023-10-21T16:20:34.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c3fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c4071fe0 con 0x7f02c41599d0 2023-10-21T16:20:34.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c3fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c40b44c0 con 0x7f02c40b3c50 2023-10-21T16:20:34.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c27fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f02c40cb5e0 0x7f02c40cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40642/0 (socket says 172.21.15.53:40642) 2023-10-21T16:20:34.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.110+0000 7f02c27fc640 1 -- 172.21.15.53:0/1810782061 learned_addr learned my addr 172.21.15.53:0/1810782061 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:34.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c37fe640 1 --2- 172.21.15.53:0/1810782061 >> v2:172.21.15.124:3300/0 conn(0x7f02c40b3c50 0x7f02c40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938278695 0 0) 0x7f02b8001110 con 0x7f02c40cb5e0 2023-10-21T16:20:34.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ac003520 con 0x7f02c40cb5e0 2023-10-21T16:20:34.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531415271 0 0) 0x7f02ac003520 con 0x7f02c40cb5e0 2023-10-21T16:20:34.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 >> v2:172.21.15.124:3300/0 conn(0x7f02c40b3c50 msgr2=0x7f02c40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 --2- 172.21.15.53:0/1810782061 >> v2:172.21.15.124:3300/0 conn(0x7f02c40b3c50 0x7f02c40cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 >> v1:172.21.15.124:6789/0 conn(0x7f02c41599d0 legacy=0x7f02c40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40713d0 con 0x7f02c40cb5e0 2023-10-21T16:20:34.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f02b8001800 con 0x7f02c40cb5e0 2023-10-21T16:20:34.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 >> v1:172.21.15.53:6789/0 conn(0x7f02c40cb5e0 legacy=0x7f02c40cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac0039c0 0x7f02ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02ac0047c0 0x7f02ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 0x7f02ac00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40b44c0 con 0x7f02ac005180 2023-10-21T16:20:34.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac00e290 con 0x7f02ac0047c0 2023-10-21T16:20:34.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.111+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac00e450 con 0x7f02ac0039c0 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.112+0000 7f02c37fe640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 0x7f02ac00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.112+0000 7f02c2ffd640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac0039c0 0x7f02ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.112+0000 7f02c27fc640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02ac0047c0 0x7f02ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.112+0000 7f02c37fe640 1 -- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac0039c0 msgr2=0x7f02ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.112+0000 7f02c37fe640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac0039c0 0x7f02ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.113+0000 7f02c37fe640 1 -- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02ac0047c0 msgr2=0x7f02ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.113+0000 7f02c37fe640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02ac0047c0 0x7f02ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.113+0000 7f02c37fe640 1 -- 172.21.15.53:0/1810782061 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac003520 con 0x7f02ac005180 2023-10-21T16:20:34.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.113+0000 7f02c2ffd640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac0039c0 0x7f02ac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:20:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.114+0000 7f02c37fe640 1 -- 172.21.15.53:0/1810782061 reap_dead start 2023-10-21T16:20:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.114+0000 7f02c37fe640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 0x7f02ac00db90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fc0 tx=0x7f02ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1669274449bbcee server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc02cc30 con 0x7f02ac005180 2023-10-21T16:20:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc00f040 con 0x7f02ac005180 2023-10-21T16:20:34.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c1ffb640 1 -- 172.21.15.53:0/1810782061 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc003990 con 0x7f02ac005180 2023-10-21T16:20:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 -- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 msgr2=0x7f02ac00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 0x7f02ac00db90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fc0 tx=0x7f02ac0057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 -- 172.21.15.53:0/1810782061 shutdown_connections 2023-10-21T16:20:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 --2- 172.21.15.53:0/1810782061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02ac005180 0x7f02ac00db90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 -- 172.21.15.53:0/1810782061 >> 172.21.15.53:0/1810782061 conn(0x7f02c406c050 msgr2=0x7f02c406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 -- 172.21.15.53:0/1810782061 shutdown_connections 2023-10-21T16:20:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.115+0000 7f02c3fff640 1 -- 172.21.15.53:0/1810782061 wait complete. 2023-10-21T16:20:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.117+0000 7f02c3fff640 1 Processor -- start 2023-10-21T16:20:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.117+0000 7f02c3fff640 1 -- start start 2023-10-21T16:20:34.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac005180 0x7f02c414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 0x7f02c40aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40aba90 con 0x7f02c40a8350 2023-10-21T16:20:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40ade10 con 0x7f02c40a7ed0 2023-10-21T16:20:34.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.118+0000 7f02c3fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40ae0f0 con 0x7f02ac005180 2023-10-21T16:20:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45002/0 (socket says 172.21.15.53:45002) 2023-10-21T16:20:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 -- 172.21.15.53:0/725106572 learned_addr learned my addr 172.21.15.53:0/725106572 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c2ffd640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac005180 0x7f02c414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c27fc640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 0x7f02c40aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 -- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac005180 msgr2=0x7f02c414f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac005180 0x7f02c414f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 -- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 msgr2=0x7f02c40aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 0x7f02c40aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c37fe640 1 -- 172.21.15.53:0/725106572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40ae3d0 con 0x7f02c40a8350 2023-10-21T16:20:34.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.119+0000 7f02c27fc640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 0x7f02c40aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:34.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02c37fe640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f02bc02c6a0 tx=0x7f02bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc035f10 con 0x7f02c40a8350 2023-10-21T16:20:34.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc003b90 con 0x7f02c40a8350 2023-10-21T16:20:34.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc004030 con 0x7f02c40a8350 2023-10-21T16:20:34.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c40ae6b0 con 0x7f02c40a8350 2023-10-21T16:20:34.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.120+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c414fe50 con 0x7f02c40a8350 2023-10-21T16:20:34.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.121+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c40aebd0 con 0x7f02c40a8350 2023-10-21T16:20:34.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.122+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f02bc004190 con 0x7f02c40a8350 2023-10-21T16:20:34.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.123+0000 7f02ab7fe640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 0x7f02b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.123+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f02bc0ac560 con 0x7f02c40a8350 2023-10-21T16:20:34.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.124+0000 7f02c2ffd640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 0x7f02b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.125+0000 7f02c2ffd640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 0x7f02b006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007fd0 tx=0x7f02b4009040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.128+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02c40aebd0 con 0x7f02c40a8350 2023-10-21T16:20:34.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.373+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 --> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f02c4071990 con 0x7f02b006af90 2023-10-21T16:20:34.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.377+0000 7f02ab7fe640 1 -- 172.21.15.53:0/725106572 <== mgr.6760 v2:172.21.15.124:6832/596536042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f02b40050d0 con 0x7f02b006af90 2023-10-21T16:20:34.379 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:20:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:20:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:20:34.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:20:34.379 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:20:34.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.379+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 msgr2=0x7f02b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.379+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 0x7f02b006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007fd0 tx=0x7f02b4009040 comp rx=0 tx=0).stop 2023-10-21T16:20:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.379+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 msgr2=0x7f02c40a9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.379+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f02bc02c6a0 tx=0x7f02bc009750 comp rx=0 tx=0).stop 2023-10-21T16:20:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 shutdown_connections 2023-10-21T16:20:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f02b006af90 0x7f02b006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02c40a8350 0x7f02c40a9100 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02c40a7ed0 0x7f02c40aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 --2- 172.21.15.53:0/725106572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02ac005180 0x7f02c414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 >> 172.21.15.53:0/725106572 conn(0x7f02c406c050 msgr2=0x7f02c40c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:34.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 shutdown_connections 2023-10-21T16:20:34.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.380+0000 7f02c3fff640 1 -- 172.21.15.53:0/725106572 wait complete. 2023-10-21T16:20:34.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:20:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.620+0000 7f020b59e640 1 Processor -- start 2023-10-21T16:20:34.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.621+0000 7f020b59e640 1 -- start start 2023-10-21T16:20:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.622+0000 7f020b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f020c15a820 0x7f020c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.622+0000 7f020b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f020c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.622+0000 7f020b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0cab60 con 0x7f020c0ca680 2023-10-21T16:20:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.622+0000 7f020b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0cb960 con 0x7f020c15a820 2023-10-21T16:20:34.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.622+0000 7f020b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0cbdf0 con 0x7f020c15ac00 2023-10-21T16:20:34.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f0203fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f020c0ca680 0x7f020c0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40650/0 (socket says 172.21.15.53:40650) 2023-10-21T16:20:34.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f0203fff640 1 -- 172.21.15.53:0/3420265802 learned_addr learned my addr 172.21.15.53:0/3420265802 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:34.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f020ad9d640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f020c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f020a59c640 1 --2- 172.21.15.53:0/3420265802 >> v2:172.21.15.124:3300/0 conn(0x7f020c15a820 0x7f020c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350372984 0 0) 0x7f020c0cab60 con 0x7f020c0ca680 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.623+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f8003520 con 0x7f020c0ca680 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760383627 0 0) 0x7f01f8003520 con 0x7f020c0ca680 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 msgr2=0x7f020c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f020c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 >> v2:172.21.15.124:3300/0 conn(0x7f020c15a820 msgr2=0x7f020c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 --2- 172.21.15.53:0/3420265802 >> v2:172.21.15.124:3300/0 conn(0x7f020c15a820 0x7f020c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0cb030 con 0x7f020c0ca680 2023-10-21T16:20:34.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f01f4001670 con 0x7f020c0ca680 2023-10-21T16:20:34.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 >> v1:172.21.15.53:6789/0 conn(0x7f020c0ca680 legacy=0x7f020c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8003970 0x7f01f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f01f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8004d00 0x7f01f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0cb960 con 0x7f01f8004d00 2023-10-21T16:20:34.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0cbdf0 con 0x7f01f8003970 2023-10-21T16:20:34.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f800b9b0 con 0x7f020c15ac00 2023-10-21T16:20:34.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.624+0000 7f020ad9d640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8004d00 0x7f01f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f0203fff640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f01f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020a59c640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8003970 0x7f01f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020ad9d640 1 -- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 msgr2=0x7f01f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020ad9d640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c15ac00 0x7f01f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020ad9d640 1 -- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8003970 msgr2=0x7f01f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:34.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020ad9d640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8003970 0x7f01f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.625+0000 7f020ad9d640 1 -- 172.21.15.53:0/3420265802 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8003520 con 0x7f01f8004d00 2023-10-21T16:20:34.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.626+0000 7f020ad9d640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8004d00 0x7f01f8005180 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0204009f90 tx=0x7f01f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea69dd926effc66f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.626+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020402cbd0 con 0x7f01f8004d00 2023-10-21T16:20:34.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.626+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400f040 con 0x7f01f8004d00 2023-10-21T16:20:34.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.626+0000 7f0209d9b640 1 -- 172.21.15.53:0/3420265802 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204034440 con 0x7f01f8004d00 2023-10-21T16:20:34.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 -- 172.21.15.53:0/3420265802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8004d00 msgr2=0x7f01f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 --2- 172.21.15.53:0/3420265802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8004d00 0x7f01f8005180 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0204009f90 tx=0x7f01f80057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020ad9d640 1 -- 172.21.15.53:0/3420265802 reap_dead start 2023-10-21T16:20:34.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 -- 172.21.15.53:0/3420265802 shutdown_connections 2023-10-21T16:20:34.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 -- 172.21.15.53:0/3420265802 >> 172.21.15.53:0/3420265802 conn(0x7f020c06c050 msgr2=0x7f020c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:34.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 -- 172.21.15.53:0/3420265802 shutdown_connections 2023-10-21T16:20:34.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.627+0000 7f020b59e640 1 -- 172.21.15.53:0/3420265802 wait complete. 2023-10-21T16:20:34.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.629+0000 7f020b59e640 1 Processor -- start 2023-10-21T16:20:34.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.629+0000 7f020b59e640 1 -- start start 2023-10-21T16:20:34.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8004d00 0x7f020c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c1f2250 0x7f020c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1f2bc0 con 0x7f020c0c6020 2023-10-21T16:20:34.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1f2ea0 con 0x7f01f8004d00 2023-10-21T16:20:34.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f020b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1f3180 con 0x7f020c1f2250 2023-10-21T16:20:34.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.630+0000 7f0203fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45010/0 (socket says 172.21.15.53:45010) 2023-10-21T16:20:34.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 -- 172.21.15.53:0/4184491408 learned_addr learned my addr 172.21.15.53:0/4184491408 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:34.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f020ad9d640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c1f2250 0x7f020c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f020a59c640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8004d00 0x7f020c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 -- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c1f2250 msgr2=0x7f020c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c1f2250 0x7f020c0cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 -- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8004d00 msgr2=0x7f020c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8004d00 0x7f020c155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 -- 172.21.15.53:0/4184491408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0cc160 con 0x7f020c0c6020 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.631+0000 7f0203fff640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f01f400e780 tx=0x7f01f400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.632+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400ee30 con 0x7f020c0c6020 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.632+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f400c8d0 con 0x7f020c0c6020 2023-10-21T16:20:34.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.632+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400cc80 con 0x7f020c0c6020 2023-10-21T16:20:34.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.632+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0cc440 con 0x7f020c0c6020 2023-10-21T16:20:34.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.632+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0cd080 con 0x7f020c0c6020 2023-10-21T16:20:34.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.633+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0cc630 con 0x7f020c0c6020 2023-10-21T16:20:34.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.634+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 54) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f01f401d460 con 0x7f020c0c6020 2023-10-21T16:20:34.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.635+0000 7f0202ffd640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 0x7f01ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.635+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f01f40896f0 con 0x7f020c0c6020 2023-10-21T16:20:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.636+0000 7f020a59c640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 0x7f01ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.637+0000 7f020a59c640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 0x7f01ec06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0099d0 tx=0x7f01fc008040 comp rx=0 tx=0).ready entity=mgr.6760 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.639+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01f4054510 con 0x7f020c0c6020 2023-10-21T16:20:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:34.886+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f020c0cd080 con 0x7f020c0c6020 2023-10-21T16:20:35.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.549+0000 7f0202ffd640 1 -- 172.21.15.53:0/4184491408 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v55) v1 ==== 92+0+0 (secure 0 0 0) 0x7f020c0cd080 con 0x7f020c0c6020 2023-10-21T16:20:35.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.552+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 msgr2=0x7f01ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.552+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 0x7f01ec06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0099d0 tx=0x7f01fc008040 comp rx=0 tx=0).stop 2023-10-21T16:20:35.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.552+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 msgr2=0x7f020c0cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.552+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f01f400e780 tx=0x7f01f400ec40 comp rx=0 tx=0).stop 2023-10-21T16:20:35.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 shutdown_connections 2023-10-21T16:20:35.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f01ec06af90 0x7f01ec06d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f020c1f2250 0x7f020c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f020c0c6020 0x7f020c0cc960 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 --2- 172.21.15.53:0/4184491408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f8004d00 0x7f020c155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.556+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 >> 172.21.15.53:0/4184491408 conn(0x7f020c06c050 msgr2=0x7f020c0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:35.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.557+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 shutdown_connections 2023-10-21T16:20:35.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.557+0000 7f020b59e640 1 -- 172.21.15.53:0/4184491408 wait complete. 2023-10-21T16:20:35.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:20:35.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.783+0000 7f71e7090640 1 Processor -- start 2023-10-21T16:20:35.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.783+0000 7f71e7090640 1 -- start start 2023-10-21T16:20:35.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e7090640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71e0072220 0x7f71e0165800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e7090640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e00b37c0 con 0x7f71e015ac80 2023-10-21T16:20:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e7090640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e0071460 con 0x7f71e006a460 2023-10-21T16:20:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e7090640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e00aefd0 con 0x7f71e0072220 2023-10-21T16:20:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e4e2c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f71e015ac80 0x7f71e0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40680/0 (socket says 172.21.15.53:40680) 2023-10-21T16:20:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e4e2c640 1 -- 172.21.15.53:0/3115067685 learned_addr learned my addr 172.21.15.53:0/3115067685 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:35.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.785+0000 7f71e562d640 1 --2- 172.21.15.53:0/3115067685 >> v2:172.21.15.53:3300/0 conn(0x7f71e0072220 0x7f71e0165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:35.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.786+0000 7f71df7fe640 1 -- 172.21.15.53:0/3115067685 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240048886 0 0) 0x7f71d4001110 con 0x7f71e015ac80 2023-10-21T16:20:35.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.786+0000 7f71df7fe640 1 -- 172.21.15.53:0/3115067685 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d0003520 con 0x7f71e015ac80 2023-10-21T16:20:35.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.786+0000 7f71e562d640 1 -- 172.21.15.53:0/3115067685 >> v1:172.21.15.124:6789/0 conn(0x7f71e006a460 legacy=0x7f71e00ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:35.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.786+0000 7f71e562d640 1 -- 172.21.15.53:0/3115067685 >> v1:172.21.15.53:6789/0 conn(0x7f71e015ac80 legacy=0x7f71e0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.786+0000 7f71e562d640 1 -- 172.21.15.53:0/3115067685 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e0071740 con 0x7f71e0072220 2023-10-21T16:20:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.787+0000 7f71e562d640 1 --2- 172.21.15.53:0/3115067685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e0072220 0x7f71e0165800 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009d90 tx=0x7f71d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91094a03ad918fd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.787+0000 7f71df7fe640 1 -- 172.21.15.53:0/3115067685 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d802cc60 con 0x7f71e0072220 2023-10-21T16:20:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71df7fe640 1 -- 172.21.15.53:0/3115067685 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d802cdc0 con 0x7f71e0072220 2023-10-21T16:20:35.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71df7fe640 1 -- 172.21.15.53:0/3115067685 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8034660 con 0x7f71e0072220 2023-10-21T16:20:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 -- 172.21.15.53:0/3115067685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e0072220 msgr2=0x7f71e0165800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 --2- 172.21.15.53:0/3115067685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e0072220 0x7f71e0165800 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009d90 tx=0x7f71d802beb0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 -- 172.21.15.53:0/3115067685 shutdown_connections 2023-10-21T16:20:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 --2- 172.21.15.53:0/3115067685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e0072220 0x7f71e0165800 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 -- 172.21.15.53:0/3115067685 >> 172.21.15.53:0/3115067685 conn(0x7f71e006d430 msgr2=0x7f71e006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 -- 172.21.15.53:0/3115067685 shutdown_connections 2023-10-21T16:20:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.788+0000 7f71e7090640 1 -- 172.21.15.53:0/3115067685 wait complete. 2023-10-21T16:20:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.790+0000 7f71e7090640 1 Processor -- start 2023-10-21T16:20:35.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.790+0000 7f71e7090640 1 -- start start 2023-10-21T16:20:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71e0072220 0x7f71e0155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:35.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 0x7f71e00a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00c8a70 con 0x7f71e006a460 2023-10-21T16:20:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00c8d50 con 0x7f71e0072220 2023-10-21T16:20:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.791+0000 7f71e7090640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00c9030 con 0x7f71e015ac80 2023-10-21T16:20:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:35.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45016/0 (socket says 172.21.15.53:45016) 2023-10-21T16:20:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 -- 172.21.15.53:0/585298032 learned_addr learned my addr 172.21.15.53:0/585298032 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71dffff640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71e0072220 0x7f71e0155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e562d640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 0x7f71e00a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 msgr2=0x7f71e00a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 0x7f71e00a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71e0072220 msgr2=0x7f71e0155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71e0072220 0x7f71e0155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e4e2c640 1 -- 172.21.15.53:0/585298032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d8009a70 con 0x7f71e006a460 2023-10-21T16:20:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.792+0000 7f71e562d640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 0x7f71e00a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71e4e2c640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c400 tx=0x7f71d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4005ce0 con 0x7f71e006a460 2023-10-21T16:20:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4005e40 con 0x7f71e006a460 2023-10-21T16:20:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e00c9310 con 0x7f71e006a460 2023-10-21T16:20:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40072d0 con 0x7f71e006a460 2023-10-21T16:20:35.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.793+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e01563a0 con 0x7f71e006a460 2023-10-21T16:20:35.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.794+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e00c9830 con 0x7f71e006a460 2023-10-21T16:20:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.795+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 55) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f71d40074f0 con 0x7f71e006a460 2023-10-21T16:20:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.796+0000 7f71dd7fa640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 0x7f71c406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.796+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f71d4088c90 con 0x7f71e006a460 2023-10-21T16:20:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.796+0000 7f71dffff640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 msgr2=0x7f71c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.796+0000 7f71dffff640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 0x7f71c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.800+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71e00c9830 con 0x7f71e006a460 2023-10-21T16:20:35.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.997+0000 7f71dffff640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 msgr2=0x7f71c406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:35.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.997+0000 7f71dffff640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 0x7f71c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:36.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:35.999+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f71e00b37c0 con 0x7f71e006a460 2023-10-21T16:20:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.004+0000 7f71dd7fa640 1 -- 172.21.15.53:0/585298032 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f71e00b37c0 con 0x7f71e006a460 2023-10-21T16:20:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 msgr2=0x7f71c406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 0x7f71c406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 msgr2=0x7f71e0156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c400 tx=0x7f71d400c8c0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 shutdown_connections 2023-10-21T16:20:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f71c406afe0 0x7f71c406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71e015ac80 0x7f71e00a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71e0072220 0x7f71e0155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 --2- 172.21.15.53:0/585298032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71e006a460 0x7f71e0156820 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.007+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 >> 172.21.15.53:0/585298032 conn(0x7f71e006d430 msgr2=0x7f71e00cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.008+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 shutdown_connections 2023-10-21T16:20:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.008+0000 7f71e7090640 1 -- 172.21.15.53:0/585298032 wait complete. 2023-10-21T16:20:36.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:20:36.073 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.072+0000 7ff58cc30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:20:36.101 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.100+0000 7f6077a2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:20:36.126 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.125+0000 7ff58cc30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:20:36.157 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.156+0000 7f6077a2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:20:36.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.240+0000 7f6dc3dfb640 1 Processor -- start 2023-10-21T16:20:36.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.240+0000 7f6dc3dfb640 1 -- start start 2023-10-21T16:20:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc3dfb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6dbc1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc3dfb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0b8c30 con 0x7f6dbc15cd70 2023-10-21T16:20:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc3dfb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0cdd80 con 0x7f6dbc0b4e10 2023-10-21T16:20:36.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc3dfb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0ce060 con 0x7f6dbc0cd4e0 2023-10-21T16:20:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc1b97640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6dbc15cd70 0x7f6dbc0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40690/0 (socket says 172.21.15.53:40690) 2023-10-21T16:20:36.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.242+0000 7f6dc1b97640 1 -- 172.21.15.53:0/4084761216 learned_addr learned my addr 172.21.15.53:0/4084761216 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc2398640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6dbc1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042483495 0 0) 0x7f6dbc0b8c30 con 0x7f6dbc15cd70 2023-10-21T16:20:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da8003520 con 0x7f6dbc15cd70 2023-10-21T16:20:36.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677253297 0 0) 0x7f6da8003520 con 0x7f6dbc15cd70 2023-10-21T16:20:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 msgr2=0x7f6dbc1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6dbc1612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 >> v1:172.21.15.124:6789/0 conn(0x7f6dbc0b4e10 legacy=0x7f6dbc0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0ce340 con 0x7f6dbc15cd70 2023-10-21T16:20:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6db8001690 con 0x7f6dbc15cd70 2023-10-21T16:20:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.243+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 >> v1:172.21.15.53:6789/0 conn(0x7f6dbc15cd70 legacy=0x7f6dbc0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6da80039a0 0x7f6da8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6da80047c0 0x7f6da8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6da8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0ce060 con 0x7f6da80047c0 2023-10-21T16:20:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da800ba60 con 0x7f6da80039a0 2023-10-21T16:20:36.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da800bc20 con 0x7f6dbc0cd4e0 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6da80047c0 0x7f6da8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1b97640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6da80039a0 0x7f6da8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc2398640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6da8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 -- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 msgr2=0x7f6da8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6da8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 -- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6da80039a0 msgr2=0x7f6da8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6da80039a0 0x7f6da8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1396640 1 -- 172.21.15.53:0/4084761216 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8003520 con 0x7f6da80047c0 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc2398640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6dbc0cd4e0 0x7f6da8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.244+0000 7f6dc1b97640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6da80039a0 0x7f6da8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.245+0000 7f6dc1396640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6da80047c0 0x7f6da8004c40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6dac001720 tx=0x7f6da80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd08e552f1ef3635 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:36.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0052d0 con 0x7f6da80047c0 2023-10-21T16:20:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac02e050 con 0x7f6da80047c0 2023-10-21T16:20:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc0b95640 1 -- 172.21.15.53:0/4084761216 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0035f0 con 0x7f6da80047c0 2023-10-21T16:20:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/4084761216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6da80047c0 msgr2=0x7f6da8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/4084761216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6da80047c0 0x7f6da8004c40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6dac001720 tx=0x7f6da80057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc2398640 1 -- 172.21.15.53:0/4084761216 reap_dead start 2023-10-21T16:20:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/4084761216 shutdown_connections 2023-10-21T16:20:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/4084761216 >> 172.21.15.53:0/4084761216 conn(0x7f6dbc0cf700 msgr2=0x7f6dbc0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/4084761216 shutdown_connections 2023-10-21T16:20:36.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.246+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/4084761216 wait complete. 2023-10-21T16:20:36.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.248+0000 7f6dc3dfb640 1 Processor -- start 2023-10-21T16:20:36.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.248+0000 7f6dc3dfb640 1 -- start start 2023-10-21T16:20:36.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6da80047c0 0x7f6dbc15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dbc0c74a0 0x7f6dbc0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc156500 con 0x7f6dbc1549e0 2023-10-21T16:20:36.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc1558f0 con 0x7f6dbc0c74a0 2023-10-21T16:20:36.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.249+0000 7f6dc3dfb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc155bd0 con 0x7f6da80047c0 2023-10-21T16:20:36.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45036/0 (socket says 172.21.15.53:45036) 2023-10-21T16:20:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 -- 172.21.15.53:0/3469840859 learned_addr learned my addr 172.21.15.53:0/3469840859 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc2398640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dbc0c74a0 0x7f6dbc0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1b97640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6da80047c0 0x7f6dbc15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 -- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6da80047c0 msgr2=0x7f6dbc15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6da80047c0 0x7f6dbc15e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 -- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dbc0c74a0 msgr2=0x7f6dbc0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dbc0c74a0 0x7f6dbc0c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 -- 172.21.15.53:0/3469840859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc155eb0 con 0x7f6dbc1549e0 2023-10-21T16:20:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.250+0000 7f6dc1396640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6dac008fd0 tx=0x7f6dac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.251+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac007640 con 0x7f6dbc1549e0 2023-10-21T16:20:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.251+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac0069e0 con 0x7f6dbc1549e0 2023-10-21T16:20:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.251+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac006d50 con 0x7f6dbc1549e0 2023-10-21T16:20:36.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.251+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc156190 con 0x7f6dbc1549e0 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.251+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc1546e0 con 0x7f6dbc1549e0 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.252+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc156670 con 0x7f6dbc1549e0 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.253+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 55) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f6dac0035f0 con 0x7f6dbc1549e0 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.254+0000 7f6db27fc640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 0x7f6da006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.255+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6dac0ab750 con 0x7f6dbc1549e0 2023-10-21T16:20:36.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.255+0000 7f6dc1b97640 1 -- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 msgr2=0x7f6da006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.255+0000 7f6dc1b97640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 0x7f6da006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:36.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.258+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6dac076460 con 0x7f6dbc1549e0 2023-10-21T16:20:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.444+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6dbc04e160 con 0x7f6dbc1549e0 2023-10-21T16:20:36.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.451+0000 7f6db27fc640 1 -- 172.21.15.53:0/3469840859 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6dac079a80 con 0x7f6dbc1549e0 2023-10-21T16:20:36.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 msgr2=0x7f6da006d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:36.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 0x7f6da006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 msgr2=0x7f6dbc1541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6dac008fd0 tx=0x7f6dac001120 comp rx=0 tx=0).stop 2023-10-21T16:20:36.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 shutdown_connections 2023-10-21T16:20:36.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f6da006b010 0x7f6da006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6dbc0c74a0 0x7f6dbc0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dbc1549e0 0x7f6dbc1541a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 --2- 172.21.15.53:0/3469840859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6da80047c0 0x7f6dbc15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.453+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 >> 172.21.15.53:0/3469840859 conn(0x7f6dbc0cf700 msgr2=0x7f6dbc15b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:36.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.454+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 shutdown_connections 2023-10-21T16:20:36.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.454+0000 7f6dc3dfb640 1 -- 172.21.15.53:0/3469840859 wait complete. 2023-10-21T16:20:36.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:36.473 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T16:20:36.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-21T16:20:36.579 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.578+0000 7ff58cc30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:20:36.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.630+0000 7ff58cc30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:20:36.636 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.635+0000 7f6077a2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:20:36.682 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.681+0000 7ff58cc30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:20:36.691 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.690+0000 7f6077a2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:20:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.738+0000 7f9bff439640 1 Processor -- start 2023-10-21T16:20:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.738+0000 7f9bff439640 1 -- start start 2023-10-21T16:20:36.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bff439640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bf806dfc0 0x7f9bf8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bff439640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80b1cc0 con 0x7f9bf80afbe0 2023-10-21T16:20:36.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bff439640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf80b2620 con 0x7f9bf8071070 2023-10-21T16:20:36.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bff439640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf80b2060 con 0x7f9bf806dfc0 2023-10-21T16:20:36.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bfd1d5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9bf80afbe0 0x7f9bf80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40694/0 (socket says 172.21.15.53:40694) 2023-10-21T16:20:36.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bfd1d5640 1 -- 172.21.15.53:0/1208452756 learned_addr learned my addr 172.21.15.53:0/1208452756 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:36.743 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.742+0000 7ff58cc30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:20:36.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/1208452756 >> v2:172.21.15.124:3300/0 conn(0x7f9bf806dfc0 0x7f9bf8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523982505 0 0) 0x7f9bf80b1cc0 con 0x7f9bf80afbe0 2023-10-21T16:20:36.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.740+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be4003520 con 0x7f9bf80afbe0 2023-10-21T16:20:36.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 937976080 0 0) 0x7f9be4003520 con 0x7f9bf80afbe0 2023-10-21T16:20:36.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 >> v2:172.21.15.124:3300/0 conn(0x7f9bf806dfc0 msgr2=0x7f9bf8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 --2- 172.21.15.53:0/1208452756 >> v2:172.21.15.124:3300/0 conn(0x7f9bf806dfc0 0x7f9bf8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 >> v1:172.21.15.124:6789/0 conn(0x7f9bf8071070 legacy=0x7f9bf80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80b2340 con 0x7f9bf80afbe0 2023-10-21T16:20:36.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9bf4001670 con 0x7f9bf80afbe0 2023-10-21T16:20:36.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 >> v1:172.21.15.53:6789/0 conn(0x7f9bf80afbe0 legacy=0x7f9bf80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40039c0 0x7f9be4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 0x7f9be4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be4005180 0x7f9be400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80b2060 con 0x7f9be40047c0 2023-10-21T16:20:36.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be400c0b0 con 0x7f9be40039c0 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be400c270 con 0x7f9be4005180 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 0x7f9be4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9bfc9d4640 1 -- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be4005180 msgr2=0x7f9be400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.741+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9be4005180 0x7f9be400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfc9d4640 1 -- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40039c0 msgr2=0x7f9be4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:36.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40039c0 0x7f9be4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40039c0 0x7f9be4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfc9d4640 1 -- 172.21.15.53:0/1208452756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be4003520 con 0x7f9be40047c0 2023-10-21T16:20:36.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/1208452756 reap_dead start 2023-10-21T16:20:36.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.742+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40039c0 0x7f9be4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:36.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 0x7f9be4004c40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009a90 tx=0x7f9be40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4e6e69272f68da server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:36.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be802c680 con 0x7f9be40047c0 2023-10-21T16:20:36.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be802c7e0 con 0x7f9be40047c0 2023-10-21T16:20:36.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9beffff640 1 -- 172.21.15.53:0/1208452756 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be802cb90 con 0x7f9be40047c0 2023-10-21T16:20:36.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9bff439640 1 -- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 msgr2=0x7f9be4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.743+0000 7f9bff439640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 0x7f9be4004c40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009a90 tx=0x7f9be40057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f9bff439640 1 -- 172.21.15.53:0/1208452756 shutdown_connections 2023-10-21T16:20:36.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f9bff439640 1 --2- 172.21.15.53:0/1208452756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9be40047c0 0x7f9be4004c40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f9bff439640 1 -- 172.21.15.53:0/1208452756 >> 172.21.15.53:0/1208452756 conn(0x7f9bf806a5d0 msgr2=0x7f9bf80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:36.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f9bff439640 1 -- 172.21.15.53:0/1208452756 shutdown_connections 2023-10-21T16:20:36.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f9bff439640 1 -- 172.21.15.53:0/1208452756 wait complete. 2023-10-21T16:20:36.753 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.744+0000 7f6077a2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:20:36.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.745+0000 7f9bff439640 1 Processor -- start 2023-10-21T16:20:36.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.746+0000 7f9bff439640 1 -- start start 2023-10-21T16:20:36.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.746+0000 7f9bff439640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40047c0 0x7f9bf80cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bff439640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80c3d40 0x7f9bf80cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bff439640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 0x7f9bf80c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bff439640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80cf620 con 0x7f9bf81f2100 2023-10-21T16:20:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bff439640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf815a080 con 0x7f9be40047c0 2023-10-21T16:20:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bff439640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf80c3580 con 0x7f9bf80c3d40 2023-10-21T16:20:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bfd9d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 0x7f9bf80c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bfd9d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 0x7f9bf80c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45050/0 (socket says 172.21.15.53:45050) 2023-10-21T16:20:36.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/3695338421 learned_addr learned my addr 172.21.15.53:0/3695338421 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:36.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40047c0 0x7f9bf80cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.747+0000 7f9bfc9d4640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80c3d40 0x7f9bf80cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:36.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80c3d40 msgr2=0x7f9bf80cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9bf80c3d40 0x7f9bf80cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40047c0 msgr2=0x7f9bf80cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:36.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9be40047c0 0x7f9bf80cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:36.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80c2ce0 con 0x7f9bf81f2100 2023-10-21T16:20:36.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bfd9d6640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 0x7f9bf80c4420 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9bf815b4e0 tx=0x7f9bf000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:36.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0005d40 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf000ced0 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0007310 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf80c2fc0 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.748+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf81588f0 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.750+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8070330 con 0x7f9bf81f2100 2023-10-21T16:20:36.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.751+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 55) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9bf0007470 con 0x7f9bf81f2100 2023-10-21T16:20:36.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.752+0000 7f9bedffb640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:36.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.752+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9bf0088780 con 0x7f9bf81f2100 2023-10-21T16:20:36.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.752+0000 7f9bfd1d5640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 msgr2=0x7f9bdc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:36.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.752+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:20:36.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.756+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bf8070330 con 0x7f9bf81f2100 2023-10-21T16:20:36.793 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.792+0000 7ff58cc30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:20:36.812 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.810+0000 7f6077a2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:20:36.863 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.862+0000 7f6077a2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:20:36.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.903+0000 7ff58cc30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:20:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.953+0000 7f9bfd1d5640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 msgr2=0x7f9bdc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:36.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.953+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:20:36.966 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:36.965+0000 7ff58cc30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:36.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:36.969+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf8070630 con 0x7f9bdc06afe0 2023-10-21T16:20:36.980 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:36.979+0000 7f6077a2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:20:37.015 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.014+0000 7ff58cc30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:20:37.047 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.045+0000 7f6077a2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:37.073 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.071+0000 7ff58cc30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:20:37.098 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.096+0000 7f6077a2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:20:37.122 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.121+0000 7ff58cc30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:20:37.156 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.155+0000 7f6077a2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:20:37.209 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.207+0000 7f6077a2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:20:37.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:37.354+0000 7f9bfd1d5640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 msgr2=0x7f9bdc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:37.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:37.354+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:20:37.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.500+0000 7ff58cc30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:20:37.607 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.606+0000 7f6077a2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:20:37.662 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.661+0000 7ff58cc30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:20:37.770 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.768+0000 7ff58cc30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:20:37.774 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.773+0000 7f6077a2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:20:37.819 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.818+0000 7ff58cc30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:20:37.888 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.886+0000 7f6077a2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:20:37.926 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:37.926+0000 7ff58cc30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:37.940 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:37.938+0000 7f6077a2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:20:38.033 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:38.032+0000 7ff58cc30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:20:38.052 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:38.051+0000 7f6077a2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:38.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:38.155+0000 7f9bfd1d5640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 msgr2=0x7f9bdc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596536042 2023-10-21T16:20:38.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:38.155+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:20:38.163 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:38.162+0000 7f6077a2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:20:38.189 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:38.188+0000 7ff58cc30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:38.238 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:38.237+0000 7ff58cc30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:38.327 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:38.325+0000 7f6077a2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:20:38.357 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:38.355+0000 7ff58cc30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:20:38.379 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:38.377+0000 7f6077a2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:20:38.503 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:38.501+0000 7f6077a2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:20:39.194 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.193+0000 7ff58cc30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:20:39.387 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:39.385+0000 7f6077a2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:20:39.488 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.487+0000 7ff58cc30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:20:39.545 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.544+0000 7ff58cc30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:20:39.659 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.658+0000 7ff58cc30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:20:39.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:39.681+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 56) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9bf0086b30 con 0x7f9bf81f2100 2023-10-21T16:20:39.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:39.682+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 msgr2=0x7f9bdc06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:20:39.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:39.682+0000 7f9bedffb640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/596536042,v1:172.21.15.124:6833/596536042] conn(0x7f9bdc06afe0 0x7f9bdc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:39.702 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:39.700+0000 7f6077a2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:20:39.765 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:39.763+0000 7f6077a2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:20:39.840 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.838+0000 7ff50900d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.840 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.838+0000 7ff50900d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.840 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.838+0000 7ff50900d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.838+0000 7ff50900d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.838+0000 7ff50900d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.839+0000 7ff50c053640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.839+0000 7ff50c053640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.839+0000 7ff50c053640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.842 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.839+0000 7ff50c053640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.842 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:20:39.839+0000 7ff50c053640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:20:39.891 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:20:39.890+0000 7f6077a2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:20:40.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.691+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 57) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f9bf0053a30 con 0x7f9bf81f2100 2023-10-21T16:20:40.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.692+0000 7f9bedffb640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f9bdc035880 0x7f9bdc075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:40.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.692+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 --> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bdc035c60 con 0x7f9bdc035880 2023-10-21T16:20:40.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.699+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f9bdc035880 0x7f9bdc075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:40.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.714+0000 7f9bfd1d5640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f9bdc035880 0x7f9bdc075440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4000c00 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.7039 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:40.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.879+0000 7f9bedffb640 1 -- 172.21.15.53:0/3695338421 <== mgr.7039 v2:172.21.15.124:6832/400910459 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9bdc035c60 con 0x7f9bdc035880 2023-10-21T16:20:40.882 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T16:20:40.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.883+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f9bdc035880 msgr2=0x7f9bdc075440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:40.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.883+0000 7f9bff439640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f9bdc035880 0x7f9bdc075440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4000c00 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2023-10-21T16:20:40.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.883+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 msgr2=0x7f9bf80c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:40.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.883+0000 7f9bff439640 1 --2- 172.21.15.53:0/3695338421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9bf81f2100 0x7f9bf80c4420 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f9bf815b4e0 tx=0x7f9bf000cb90 comp rx=0 tx=0).stop 2023-10-21T16:20:40.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.883+0000 7f9bfd9d6640 1 -- 172.21.15.53:0/3695338421 reap_dead start 2023-10-21T16:20:40.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.884+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 shutdown_connections 2023-10-21T16:20:40.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.884+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 >> 172.21.15.53:0/3695338421 conn(0x7f9bf806a5d0 msgr2=0x7f9bf80c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:40.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.884+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 shutdown_connections 2023-10-21T16:20:40.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:40.884+0000 7f9bff439640 1 -- 172.21.15.53:0/3695338421 wait complete. 2023-10-21T16:20:40.906 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T16:20:40.906 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-21-16_21_00 in ~20s... 2023-10-21T16:20:40.906 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-21T16:20:40.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-21T16:20:41.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.116+0000 7f0c41389640 1 Processor -- start 2023-10-21T16:20:41.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.117+0000 7f0c41389640 1 -- start start 2023-10-21T16:20:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c41389640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c3c06db60 0x7f0c3c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c41389640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c15b310 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c41389640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c06d290 con 0x7f0c3c06c3a0 2023-10-21T16:20:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c41389640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c15b110 con 0x7f0c3c06db60 2023-10-21T16:20:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c3a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0c3c0c9cf0 0x7f0c3c15aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40704/0 (socket says 172.21.15.53:40704) 2023-10-21T16:20:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.118+0000 7f0c3a7fc640 1 -- 172.21.15.53:0/1892223693 learned_addr learned my addr 172.21.15.53:0/1892223693 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:20:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/1892223693 >> v2:172.21.15.124:3300/0 conn(0x7f0c3c06db60 0x7f0c3c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612267609 0 0) 0x7f0c30001110 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c24003520 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903155532 0 0) 0x7f0c24003520 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 >> v2:172.21.15.124:3300/0 conn(0x7f0c3c06db60 msgr2=0x7f0c3c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 --2- 172.21.15.53:0/1892223693 >> v2:172.21.15.124:3300/0 conn(0x7f0c3c06db60 0x7f0c3c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 >> v1:172.21.15.124:6789/0 conn(0x7f0c3c06c3a0 legacy=0x7f0c3c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0c9860 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0c30001800 con 0x7f0c3c0c9cf0 2023-10-21T16:20:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.119+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 >> v1:172.21.15.53:6789/0 conn(0x7f0c3c0c9cf0 legacy=0x7f0c3c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24003970 0x7f0c24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c240047c0 0x7f0c24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 0x7f0c2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c15b110 con 0x7f0c24005180 2023-10-21T16:20:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2400c020 con 0x7f0c240047c0 2023-10-21T16:20:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2400c1e0 con 0x7f0c24003970 2023-10-21T16:20:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 0x7f0c2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24003970 msgr2=0x7f0c24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3affd640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24003970 0x7f0c24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3a7fc640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c240047c0 0x7f0c24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24003970 0x7f0c24003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c240047c0 msgr2=0x7f0c24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:20:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c240047c0 0x7f0c24004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/1892223693 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24003520 con 0x7f0c24005180 2023-10-21T16:20:41.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.120+0000 7f0c3affd640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24003970 0x7f0c24003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.121+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/1892223693 reap_dead start 2023-10-21T16:20:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.121+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 0x7f0c2400b980 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009fc0 tx=0x7f0c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697f09ad80ae491c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:41.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3402cc30 con 0x7f0c24005180 2023-10-21T16:20:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3400f040 con 0x7f0c24005180 2023-10-21T16:20:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c39ffb640 1 -- 172.21.15.53:0/1892223693 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34003990 con 0x7f0c24005180 2023-10-21T16:20:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 -- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 msgr2=0x7f0c2400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 0x7f0c2400b980 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009fc0 tx=0x7f0c240057f0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 -- 172.21.15.53:0/1892223693 shutdown_connections 2023-10-21T16:20:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 --2- 172.21.15.53:0/1892223693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c24005180 0x7f0c2400b980 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 -- 172.21.15.53:0/1892223693 >> 172.21.15.53:0/1892223693 conn(0x7f0c3c06a5d0 msgr2=0x7f0c3c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 -- 172.21.15.53:0/1892223693 shutdown_connections 2023-10-21T16:20:41.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.122+0000 7f0c41389640 1 -- 172.21.15.53:0/1892223693 wait complete. 2023-10-21T16:20:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.124+0000 7f0c41389640 1 Processor -- start 2023-10-21T16:20:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.124+0000 7f0c41389640 1 -- start start 2023-10-21T16:20:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24005180 0x7f0c3c0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 0x7f0c3c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0cded0 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0ce190 con 0x7f0c3c0afbc0 2023-10-21T16:20:41.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.125+0000 7f0c41389640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0ce470 con 0x7f0c24005180 2023-10-21T16:20:41.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45072/0 (socket says 172.21.15.53:45072) 2023-10-21T16:20:41.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/4182277390 learned_addr learned my addr 172.21.15.53:0/4182277390 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:20:41.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3affd640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24005180 0x7f0c3c0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3a7fc640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 0x7f0c3c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24005180 msgr2=0x7f0c3c0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24005180 0x7f0c3c0c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 msgr2=0x7f0c3c146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 0x7f0c3c146df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 -- 172.21.15.53:0/4182277390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0ce750 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3a7fc640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 0x7f0c3c146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:20:41.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.126+0000 7f0c3b7fe640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e230 tx=0x7f0c34009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.127+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34003e10 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.127+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c340044e0 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.127+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004870 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.127+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c0c8400 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.127+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c0c5810 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.128+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c0ce8e0 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.129+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 57) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0c3403f470 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.130+0000 7f0c237fe640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 0x7f0c2806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:20:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.130+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f0c340abd50 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.131+0000 7f0c3affd640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 0x7f0c2806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:20:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.132+0000 7f0c3affd640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 0x7f0c2806d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c159ed0 tx=0x7f0c2c009040 comp rx=0 tx=0).ready entity=mgr.7039 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:20:41.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.134+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c34076da0 con 0x7f0c3c0cd4e0 2023-10-21T16:20:41.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.347+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 --> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c3c0ce300 con 0x7f0c2806ad40 2023-10-21T16:20:41.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.365+0000 7f0c237fe640 1 -- 172.21.15.53:0/4182277390 <== mgr.7039 v2:172.21.15.124:6832/400910459 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f0c3c0ce300 con 0x7f0c2806ad40 2023-10-21T16:20:41.367 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:20:41.368 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-21T16:20:41.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 msgr2=0x7f0c2806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 0x7f0c2806d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c159ed0 tx=0x7f0c2c009040 comp rx=0 tx=0).stop 2023-10-21T16:20:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 msgr2=0x7f0c3c0cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:20:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400e230 tx=0x7f0c34009750 comp rx=0 tx=0).stop 2023-10-21T16:20:41.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 shutdown_connections 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f0c2806ad40 0x7f0c2806d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c3c0cd4e0 0x7f0c3c0cd990 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c0afbc0 0x7f0c3c146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 --2- 172.21.15.53:0/4182277390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c24005180 0x7f0c3c0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 >> 172.21.15.53:0/4182277390 conn(0x7f0c3c06a5d0 msgr2=0x7f0c3c15c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:20:41.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 shutdown_connections 2023-10-21T16:20:41.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:20:41.368+0000 7f0c41389640 1 -- 172.21.15.53:0/4182277390 wait complete. 2023-10-21T16:20:41.390 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-21T16:20:41.390 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2023-10-21T16:20:41.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:41.391 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:41.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:41.450 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:41.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:42.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:42.452 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:42.517 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:42.517 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:42.518 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:43.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:43.520 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:43.587 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:43.588 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:43.588 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:44.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:44.590 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:44.658 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:44.658 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:44.658 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:45.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:45.661 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:45.728 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:45.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:45.729 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:46.730 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:46.797 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:46.797 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:46.798 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:47.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:47.800 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:47.867 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:47.867 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:47.868 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:48.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:48.869 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:48.937 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:48.937 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:48.938 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:49.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:49.940 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:50.006 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:50.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:50.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:51.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:51.008 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:51.075 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:51.076 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:51.076 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:52.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:52.077 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:52.143 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:52.144 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:52.144 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:52.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:20:52.263 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:20:53.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:53.145 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:53.211 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:53.211 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:53.212 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:54.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:54.213 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:54.280 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:54.281 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:54.281 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:55.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:55.283 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:55.350 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:55.350 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:55.350 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:56.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:56.352 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:56.421 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:56.421 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:56.422 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:57.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:57.424 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:57.492 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:57.492 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:57.493 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:58.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:58.495 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:58.563 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:58.564 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:58.564 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:20:59.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:20:59.565 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:20:59.633 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:20:59.634 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:20:59.634 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:21:00.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:00.635 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:21:00.703 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T16:21:00.703 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T16:21:00.705 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:21:00.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:00.902+0000 7ff4fb7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:00.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:00.902+0000 7ff4fb7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:00.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:00.902+0000 7ff4fb7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:00.905 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:00.902+0000 7ff4fb7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:00.905 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:00.902+0000 7ff4fb7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:01.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:01.706 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:21:01.769 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_21_00_UTC 2023-10-21T16:21:01.770 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-16_21_00_UTC'} 2023-10-21T16:21:01.771 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-21-16_21_00_UTC'} 2023-10-21T16:21:01.771 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T16:21:01.771 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-21-16_21_00_UTC'] 2023-10-21T16:21:01.772 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697905240.9065225, now = 1697905261.7722487, timo = 20 2023-10-21T16:21:02.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-21T16:21:03.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.000+0000 7f09a9da0640 1 Processor -- start 2023-10-21T16:21:03.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.000+0000 7f09a9da0640 1 -- start start 2023-10-21T16:21:03.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.002+0000 7f09a9da0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.002+0000 7f09a9da0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f09a40c8b40 0x7f09a40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.002+0000 7f09a9da0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40c94e0 con 0x7f09a40aff70 2023-10-21T16:21:03.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.002+0000 7f09a9da0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40ccbc0 con 0x7f09a40c8b40 2023-10-21T16:21:03.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.002+0000 7f09a9da0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40ce1f0 con 0x7f09a415cae0 2023-10-21T16:21:03.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a959f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f09a40aff70 0x7f09a4070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46760/0 (socket says 172.21.15.53:46760) 2023-10-21T16:21:03.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f09a40c8b40 0x7f09a40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a959f640 1 -- 172.21.15.53:0/919281036 learned_addr learned my addr 172.21.15.53:0/919281036 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:03.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a8d9e640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a3fff640 1 -- 172.21.15.53:0/919281036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 msgr2=0x7f09a40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:03.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a3fff640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a37fe640 1 -- 172.21.15.53:0/919281036 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921445367 0 0) 0x7f099c001490 con 0x7f09a40aff70 2023-10-21T16:21:03.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a3fff640 1 -- 172.21.15.53:0/919281036 >> v1:172.21.15.53:6789/0 conn(0x7f09a40aff70 legacy=0x7f09a4070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.003+0000 7f09a3fff640 1 -- 172.21.15.53:0/919281036 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40cd590 con 0x7f09a40c8b40 2023-10-21T16:21:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a3fff640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40c8fa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f09900097a0 tx=0x7f099002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df5161613264663 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a37fe640 1 -- 172.21.15.53:0/919281036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990031830 con 0x7f09a40c8b40 2023-10-21T16:21:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a37fe640 1 -- 172.21.15.53:0/919281036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990031990 con 0x7f09a40c8b40 2023-10-21T16:21:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a37fe640 1 -- 172.21.15.53:0/919281036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990031d20 con 0x7f09a40c8b40 2023-10-21T16:21:03.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a9da0640 1 -- 172.21.15.53:0/919281036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 msgr2=0x7f09a40c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.005+0000 7f09a9da0640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40c8fa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f09900097a0 tx=0x7f099002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 -- 172.21.15.53:0/919281036 shutdown_connections 2023-10-21T16:21:03.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40c8fa0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 --2- 172.21.15.53:0/919281036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 -- 172.21.15.53:0/919281036 >> 172.21.15.53:0/919281036 conn(0x7f09a406c050 msgr2=0x7f09a406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:03.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 -- 172.21.15.53:0/919281036 shutdown_connections 2023-10-21T16:21:03.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.006+0000 7f09a9da0640 1 -- 172.21.15.53:0/919281036 wait complete. 2023-10-21T16:21:03.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.007+0000 7f09a9da0640 1 Processor -- start 2023-10-21T16:21:03.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.008+0000 7f09a9da0640 1 -- start start 2023-10-21T16:21:03.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a40aff70 0x7f09a41f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41f2020 con 0x7f09a40c8b40 2023-10-21T16:21:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b2e20 con 0x7f09a40aff70 2023-10-21T16:21:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a9da0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4146710 con 0x7f09a415cae0 2023-10-21T16:21:03.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58696/0 (socket says 172.21.15.53:58696) 2023-10-21T16:21:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 -- 172.21.15.53:0/3494910475 learned_addr learned my addr 172.21.15.53:0/3494910475 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 -- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 msgr2=0x7f09a40b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 -- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a40aff70 msgr2=0x7f09a41f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a40aff70 0x7f09a41f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.009+0000 7f09a3fff640 1 -- 172.21.15.53:0/3494910475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990009400 con 0x7f09a40c8b40 2023-10-21T16:21:03.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a3fff640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f09900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990004230 con 0x7f09a40c8b40 2023-10-21T16:21:03.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990004390 con 0x7f09a40c8b40 2023-10-21T16:21:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a4148e70 con 0x7f09a40c8b40 2023-10-21T16:21:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099003a880 con 0x7f09a40c8b40 2023-10-21T16:21:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.010+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a4146c90 con 0x7f09a40c8b40 2023-10-21T16:21:03.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.012+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40721a0 con 0x7f09a40c8b40 2023-10-21T16:21:03.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.013+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 60) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f099003a9e0 con 0x7f09a40c8b40 2023-10-21T16:21:03.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.014+0000 7f09a17fa640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 0x7f098806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.014+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f09900ac6d0 con 0x7f09a40c8b40 2023-10-21T16:21:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.014+0000 7f09a8d9e640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 0x7f098806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.015+0000 7f09a8d9e640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 0x7f098806d610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0998006fd0 tx=0x7f0998008040 comp rx=0 tx=0).ready entity=mgr.7039 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.018+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0990077140 con 0x7f09a40c8b40 2023-10-21T16:21:03.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.235+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 --> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a40cc570 con 0x7f098806b150 2023-10-21T16:21:03.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.416+0000 7f09a17fa640 1 -- 172.21.15.53:0/3494910475 <== mgr.7039 v2:172.21.15.124:6832/400910459 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f099800be20 con 0x7f098806b150 2023-10-21T16:21:03.420 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path snap_test_dir1 2023-10-21T16:21:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 msgr2=0x7f098806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 0x7f098806d610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0998006fd0 tx=0x7f0998008040 comp rx=0 tx=0).stop 2023-10-21T16:21:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 msgr2=0x7f09a40b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f09900046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 shutdown_connections 2023-10-21T16:21:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f098806b150 0x7f098806d610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a415cae0 0x7f09a40b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09a40c8b40 0x7f09a40b22b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 --2- 172.21.15.53:0/3494910475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a40aff70 0x7f09a41f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 >> 172.21.15.53:0/3494910475 conn(0x7f09a406c050 msgr2=0x7f09a4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:03.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 shutdown_connections 2023-10-21T16:21:03.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.420+0000 7f09a9da0640 1 -- 172.21.15.53:0/3494910475 wait complete. 2023-10-21T16:21:03.446 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-21T16:21:03.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:03.447 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T16:21:03.507 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_21_00_UTC 2023-10-21T16:21:03.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_21_00_UTC 2023-10-21T16:21:03.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:03.509 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_21_00_UTC) 2023-10-21T16:21:03.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:03.576 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T16:21:03.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:21:03.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.866+0000 7fe7b3b8a640 1 Processor -- start 2023-10-21T16:21:03.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.866+0000 7fe7b3b8a640 1 -- start start 2023-10-21T16:21:03.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.868+0000 7fe7b3b8a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7ac160660 0x7fe7ac070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.868+0000 7fe7b3b8a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7ac0c9910 0x7fe7ac0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.868+0000 7fe7b3b8a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac06e150 con 0x7fe7ac0cb450 2023-10-21T16:21:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.868+0000 7fe7b3b8a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac06de70 con 0x7fe7ac160660 2023-10-21T16:21:03.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.868+0000 7fe7b3b8a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c8e30 con 0x7fe7ac0c9910 2023-10-21T16:21:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b2127640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe7ac0cb450 0x7fe7ac0b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46772/0 (socket says 172.21.15.53:46772) 2023-10-21T16:21:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1926640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7ac160660 0x7fe7ac070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b2127640 1 -- 172.21.15.53:0/4255397046 learned_addr learned my addr 172.21.15.53:0/4255397046 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1125640 1 --2- 172.21.15.53:0/4255397046 >> v2:172.21.15.124:3300/0 conn(0x7fe7ac0c9910 0x7fe7ac0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b0924640 1 -- 172.21.15.53:0/4255397046 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992064821 0 0) 0x7fe7ac06e150 con 0x7fe7ac0cb450 2023-10-21T16:21:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b0924640 1 -- 172.21.15.53:0/4255397046 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe798003520 con 0x7fe7ac0cb450 2023-10-21T16:21:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1926640 1 -- 172.21.15.53:0/4255397046 >> v2:172.21.15.124:3300/0 conn(0x7fe7ac0c9910 msgr2=0x7fe7ac0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1926640 1 --2- 172.21.15.53:0/4255397046 >> v2:172.21.15.124:3300/0 conn(0x7fe7ac0c9910 0x7fe7ac0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1926640 1 -- 172.21.15.53:0/4255397046 >> v1:172.21.15.53:6789/0 conn(0x7fe7ac0cb450 legacy=0x7fe7ac0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.869+0000 7fe7b1926640 1 -- 172.21.15.53:0/4255397046 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0c9110 con 0x7fe7ac160660 2023-10-21T16:21:03.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.870+0000 7fe7b1926640 1 --2- 172.21.15.53:0/4255397046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac070cc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b0d50 tx=0x7fe7a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec82d54d50ffacc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b0924640 1 -- 172.21.15.53:0/4255397046 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8031830 con 0x7fe7ac160660 2023-10-21T16:21:03.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b0924640 1 -- 172.21.15.53:0/4255397046 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8031990 con 0x7fe7ac160660 2023-10-21T16:21:03.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b0924640 1 -- 172.21.15.53:0/4255397046 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8031d20 con 0x7fe7ac160660 2023-10-21T16:21:03.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/4255397046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 msgr2=0x7fe7ac070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/4255397046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac070cc0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0b0d50 tx=0x7fe7a802b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/4255397046 shutdown_connections 2023-10-21T16:21:03.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/4255397046 >> v2:172.21.15.124:3300/0 conn(0x7fe7ac0c9910 0x7fe7ac0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/4255397046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac070cc0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.871+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/4255397046 >> 172.21.15.53:0/4255397046 conn(0x7fe7ac06a5d0 msgr2=0x7fe7ac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:03.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.872+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/4255397046 shutdown_connections 2023-10-21T16:21:03.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.872+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/4255397046 wait complete. 2023-10-21T16:21:03.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.873+0000 7fe7b3b8a640 1 Processor -- start 2023-10-21T16:21:03.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.873+0000 7fe7b3b8a640 1 -- start start 2023-10-21T16:21:03.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ac0c9910 0x7fe7ac0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ac0cb450 0x7fe7ac15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c3140 con 0x7fe7ac160660 2023-10-21T16:21:03.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c2250 con 0x7fe7ac0cb450 2023-10-21T16:21:03.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.874+0000 7fe7b3b8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c2530 con 0x7fe7ac0c9910 2023-10-21T16:21:03.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58716/0 (socket says 172.21.15.53:58716) 2023-10-21T16:21:03.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 -- 172.21.15.53:0/3648497600 learned_addr learned my addr 172.21.15.53:0/3648497600 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:03.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b1125640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ac0cb450 0x7fe7ac15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b1926640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ac0c9910 0x7fe7ac0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 -- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ac0c9910 msgr2=0x7fe7ac0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ac0c9910 0x7fe7ac0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 -- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ac0cb450 msgr2=0x7fe7ac15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:03.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ac0cb450 0x7fe7ac15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.875+0000 7fe7b2127640 1 -- 172.21.15.53:0/3648497600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8009400 con 0x7fe7ac160660 2023-10-21T16:21:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7b2127640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400cb40 tx=0x7fe7a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a400aca0 con 0x7fe7ac160660 2023-10-21T16:21:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a400ae00 con 0x7fe7ac160660 2023-10-21T16:21:03.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac0c2810 con 0x7fe7ac160660 2023-10-21T16:21:03.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4012620 con 0x7fe7ac160660 2023-10-21T16:21:03.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.876+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0c2dd0 con 0x7fe7ac160660 2023-10-21T16:21:03.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.877+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac0ae650 con 0x7fe7ac160660 2023-10-21T16:21:03.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.878+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 60) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe7a4007070 con 0x7fe7ac160660 2023-10-21T16:21:03.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.879+0000 7fe7a27fc640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 0x7fe79006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:03.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.879+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fe7a4089e60 con 0x7fe7ac160660 2023-10-21T16:21:03.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.880+0000 7fe7b1926640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 0x7fe79006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:03.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.881+0000 7fe7b1926640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 0x7fe79006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8036000 tx=0x7fe7a803c040 comp rx=0 tx=0).ready entity=mgr.7039 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:03.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:03.885+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7a4054840 con 0x7fe7ac160660 2023-10-21T16:21:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.135+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe7ac04e160 con 0x7fe7ac160660 2023-10-21T16:21:04.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.499+0000 7fe7a27fc640 1 -- 172.21.15.53:0/3648497600 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v61) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe7a4017020 con 0x7fe7ac160660 2023-10-21T16:21:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.504+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 msgr2=0x7fe79006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.504+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 0x7fe79006d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8036000 tx=0x7fe7a803c040 comp rx=0 tx=0).stop 2023-10-21T16:21:04.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.504+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 msgr2=0x7fe7ac0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.504+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400cb40 tx=0x7fe7a400a060 comp rx=0 tx=0).stop 2023-10-21T16:21:04.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 shutdown_connections 2023-10-21T16:21:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fe79006b100 0x7fe79006d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ac160660 0x7fe7ac0ac840 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ac0cb450 0x7fe7ac15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:04.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 --2- 172.21.15.53:0/3648497600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ac0c9910 0x7fe7ac0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:04.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 >> 172.21.15.53:0/3648497600 conn(0x7fe7ac06a5d0 msgr2=0x7fe7ac15f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:04.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 shutdown_connections 2023-10-21T16:21:04.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:04.505+0000 7fe7b3b8a640 1 -- 172.21.15.53:0/3648497600 wait complete. 2023-10-21T16:21:04.531 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:21:04.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:04.652 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:04.750 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:21:04.750 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:04.751 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:04.805 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:04.806 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:04.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:05.028 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.026+0000 7f1f0e841040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:05.080 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.078+0000 7f1f0e841040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:05.101 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.099+0000 7f33a5852040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:05.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.121+0000 7f95b92af640 1 Processor -- start 2023-10-21T16:21:05.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.122+0000 7f95b92af640 1 -- start start 2023-10-21T16:21:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b415cd20 0x7f95b40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95b40cb220 0x7f95b40cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 0x7f95b40cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b40ce120 con 0x7f95b40cb220 2023-10-21T16:21:05.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b406b490 con 0x7f95b40cd750 2023-10-21T16:21:05.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b92af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b406b770 con 0x7f95b415cd20 2023-10-21T16:21:05.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95abfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95b40cb220 0x7f95b40cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95abfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95b40cb220 0x7f95b40cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42718/0 (socket says 172.21.15.53:42718) 2023-10-21T16:21:05.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95abfff640 1 -- 172.21.15.53:0/559655373 learned_addr learned my addr 172.21.15.53:0/559655373 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:05.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b2ffd640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b415cd20 0x7f95b40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.124+0000 7f95b37fe640 1 --2- 172.21.15.53:0/559655373 >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 0x7f95b40cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95abfff640 1 -- 172.21.15.53:0/559655373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b415cd20 msgr2=0x7f95b40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95abfff640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b415cd20 0x7f95b40cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95abfff640 1 -- 172.21.15.53:0/559655373 >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 msgr2=0x7f95b40cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95abfff640 1 --2- 172.21.15.53:0/559655373 >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 0x7f95b40cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95abfff640 1 -- 172.21.15.53:0/559655373 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b406ba50 con 0x7f95b40cb220 2023-10-21T16:21:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.125+0000 7f95b37fe640 1 --2- 172.21.15.53:0/559655373 >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 0x7f95b40cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:05.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.126+0000 7f95abfff640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b40cb680 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f95b40b2f70 tx=0x7f959c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc938e56047ff57b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.127+0000 7f95b27fc640 1 -- 172.21.15.53:0/559655373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c031860 con 0x7f95b40cb220 2023-10-21T16:21:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.127+0000 7f95b27fc640 1 -- 172.21.15.53:0/559655373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0319c0 con 0x7f95b40cb220 2023-10-21T16:21:05.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.127+0000 7f95b27fc640 1 -- 172.21.15.53:0/559655373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c031d50 con 0x7f95b40cb220 2023-10-21T16:21:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.127+0000 7f95b92af640 1 -- 172.21.15.53:0/559655373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 msgr2=0x7f95b40cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.127+0000 7f95b92af640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b40cb680 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f95b40b2f70 tx=0x7f959c02b900 comp rx=0 tx=0).stop 2023-10-21T16:21:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 -- 172.21.15.53:0/559655373 shutdown_connections 2023-10-21T16:21:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 --2- 172.21.15.53:0/559655373 >> v2:172.21.15.124:3300/0 conn(0x7f95b40cd750 0x7f95b40cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b40cb680 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 --2- 172.21.15.53:0/559655373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b415cd20 0x7f95b40cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 -- 172.21.15.53:0/559655373 >> 172.21.15.53:0/559655373 conn(0x7f95b4069f30 msgr2=0x7f95b406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 -- 172.21.15.53:0/559655373 shutdown_connections 2023-10-21T16:21:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.128+0000 7f95b92af640 1 -- 172.21.15.53:0/559655373 wait complete. 2023-10-21T16:21:05.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.131+0000 7f95b92af640 1 Processor -- start 2023-10-21T16:21:05.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.131+0000 7f95b92af640 1 -- start start 2023-10-21T16:21:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.132+0000 7f95b92af640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.132+0000 7f95b92af640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b40cd750 0x7f95b40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.132+0000 7f95b92af640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95b415cd20 0x7f95b40c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b92af640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b414e410 con 0x7f95b40cb220 2023-10-21T16:21:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b92af640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40c50b0 con 0x7f95b415cd20 2023-10-21T16:21:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b92af640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b40c44a0 con 0x7f95b40cd750 2023-10-21T16:21:05.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42724/0 (socket says 172.21.15.53:42724) 2023-10-21T16:21:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 -- 172.21.15.53:0/1885046712 learned_addr learned my addr 172.21.15.53:0/1885046712 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95abfff640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b40cd750 0x7f95b40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b37fe640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95b415cd20 0x7f95b40c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b40cd750 msgr2=0x7f95b40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b40cd750 0x7f95b40c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95b415cd20 msgr2=0x7f95b40c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95b415cd20 0x7f95b40c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.133+0000 7f95b2ffd640 1 -- 172.21.15.53:0/1885046712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0094d0 con 0x7f95b40cb220 2023-10-21T16:21:05.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95b2ffd640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d5f0 tx=0x7f95a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:05.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40125f0 con 0x7f95b40cb220 2023-10-21T16:21:05.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a4012750 con 0x7f95b40cb220 2023-10-21T16:21:05.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4012b00 con 0x7f95b40cb220 2023-10-21T16:21:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b40c4780 con 0x7f95b40cb220 2023-10-21T16:21:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.134+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b40c4d40 con 0x7f95b40cb220 2023-10-21T16:21:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.135+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b4066d80 con 0x7f95b40cb220 2023-10-21T16:21:05.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.136+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f95a4012c60 con 0x7f95b40cb220 2023-10-21T16:21:05.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.137+0000 7f95ab7fe640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 0x7f959406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.137+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f95a408a170 con 0x7f95b40cb220 2023-10-21T16:21:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.137+0000 7f95abfff640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 msgr2=0x7f959406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.137+0000 7f95abfff640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 0x7f959406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:05.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.142+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95a4054be0 con 0x7f95b40cb220 2023-10-21T16:21:05.165 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.163+0000 7f33a5852040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.338+0000 7f95abfff640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 msgr2=0x7f959406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.338+0000 7f95abfff640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 0x7f959406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.345+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b40b6970 con 0x7f95b40cb220 2023-10-21T16:21:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.347+0000 7f95ab7fe640 1 -- 172.21.15.53:0/1885046712 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17280 (secure 0 0 0) 0x7f95a4058200 con 0x7f95b40cb220 2023-10-21T16:21:05.350 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:05.350 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:39.664268+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6832/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/897501407":"2023-10-22T16:20:39.664146+0000","172.21.15.124:6833/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3242447571":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1097749659":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2545128602":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1535068499":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3443628834":"2023-10-22T16:20:39.664146+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 msgr2=0x7f959406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:05.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 0x7f959406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 msgr2=0x7f95b414ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d5f0 tx=0x7f95a400dab0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 shutdown_connections 2023-10-21T16:21:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f959406b0b0 0x7f959406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95b415cd20 0x7f95b40c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95b40cd750 0x7f95b40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 --2- 172.21.15.53:0/1885046712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95b40cb220 0x7f95b414ded0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.349+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 >> 172.21.15.53:0/1885046712 conn(0x7f95b4069f30 msgr2=0x7f95b415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.350+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 shutdown_connections 2023-10-21T16:21:05.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.350+0000 7f95b92af640 1 -- 172.21.15.53:0/1885046712 wait complete. 2023-10-21T16:21:05.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:20:22.870339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:20:23.949173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:05.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:05.531 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.529+0000 7f1f0e841040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:05.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.577+0000 7f7d7b075640 1 Processor -- start 2023-10-21T16:21:05.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.577+0000 7f7d7b075640 1 -- start start 2023-10-21T16:21:05.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d7b075640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7d740cdb70 0x7f7d741612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d7b075640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d74161810 con 0x7f7d7415cf50 2023-10-21T16:21:05.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d7b075640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7406b4e0 con 0x7f7d740cef60 2023-10-21T16:21:05.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d7b075640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7406c440 con 0x7f7d740cdb70 2023-10-21T16:21:05.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d78e11640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d7415cf50 0x7f7d7406c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37864/0 (socket says 172.21.15.53:37864) 2023-10-21T16:21:05.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d78e11640 1 -- 172.21.15.53:0/1605797024 learned_addr learned my addr 172.21.15.53:0/1605797024 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.579+0000 7f7d79612640 1 --2- 172.21.15.53:0/1605797024 >> v2:172.21.15.124:3300/0 conn(0x7f7d740cdb70 0x7f7d741612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406823924 0 0) 0x7f7d68001110 con 0x7f7d7415cf50 2023-10-21T16:21:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d64003520 con 0x7f7d7415cf50 2023-10-21T16:21:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3017887344 0 0) 0x7f7d64003520 con 0x7f7d7415cf50 2023-10-21T16:21:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 >> v2:172.21.15.124:3300/0 conn(0x7f7d740cdb70 msgr2=0x7f7d741612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 --2- 172.21.15.53:0/1605797024 >> v2:172.21.15.124:3300/0 conn(0x7f7d740cdb70 0x7f7d741612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 >> v1:172.21.15.124:6789/0 conn(0x7f7d740cef60 legacy=0x7f7d740cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7406b7c0 con 0x7f7d7415cf50 2023-10-21T16:21:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7d68001800 con 0x7f7d7415cf50 2023-10-21T16:21:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 >> v1:172.21.15.53:6789/0 conn(0x7f7d7415cf50 legacy=0x7f7d7406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d64003970 0x7f7d64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 0x7f7d64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d64005180 0x7f7d6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406c440 con 0x7f7d640047c0 2023-10-21T16:21:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400c120 con 0x7f7d64005180 2023-10-21T16:21:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400c2e0 con 0x7f7d64003970 2023-10-21T16:21:05.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.580+0000 7f7d73fff640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 0x7f7d64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d79612640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d64005180 0x7f7d6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d78e11640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d64003970 0x7f7d64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d73fff640 1 -- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d64003970 msgr2=0x7f7d64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d73fff640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d64003970 0x7f7d64003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d73fff640 1 -- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d64005180 msgr2=0x7f7d6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:05.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d73fff640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d64005180 0x7f7d6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d73fff640 1 -- 172.21.15.53:0/1605797024 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64003520 con 0x7f7d640047c0 2023-10-21T16:21:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.581+0000 7f7d79612640 1 -- 172.21.15.53:0/1605797024 reap_dead start 2023-10-21T16:21:05.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.582+0000 7f7d73fff640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 0x7f7d64004c40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7d60001720 tx=0x7f7d640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3e0a219d20d21a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:05.590 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.581+0000 7f1f0e841040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:05.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.582+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600052d0 con 0x7f7d640047c0 2023-10-21T16:21:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.582+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6002e050 con 0x7f7d640047c0 2023-10-21T16:21:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d737fe640 1 -- 172.21.15.53:0/1605797024 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60003830 con 0x7f7d640047c0 2023-10-21T16:21:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 -- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 msgr2=0x7f7d64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 0x7f7d64004c40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7d60001720 tx=0x7f7d640057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 -- 172.21.15.53:0/1605797024 shutdown_connections 2023-10-21T16:21:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 --2- 172.21.15.53:0/1605797024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d640047c0 0x7f7d64004c40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 -- 172.21.15.53:0/1605797024 >> 172.21.15.53:0/1605797024 conn(0x7f7d7406e650 msgr2=0x7f7d7406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 -- 172.21.15.53:0/1605797024 shutdown_connections 2023-10-21T16:21:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.583+0000 7f7d7b075640 1 -- 172.21.15.53:0/1605797024 wait complete. 2023-10-21T16:21:05.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.585+0000 7f7d7b075640 1 Processor -- start 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.585+0000 7f7d7b075640 1 -- start start 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640047c0 0x7f7d740c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d741f2c50 0x7f7d740cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740c7890 con 0x7f7d7415bdb0 2023-10-21T16:21:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740c9390 con 0x7f7d640047c0 2023-10-21T16:21:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.586+0000 7f7d7b075640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740c9670 con 0x7f7d741f2c50 2023-10-21T16:21:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42740/0 (socket says 172.21.15.53:42740) 2023-10-21T16:21:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 -- 172.21.15.53:0/2455201482 learned_addr learned my addr 172.21.15.53:0/2455201482 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:05.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d78e11640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640047c0 0x7f7d740c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d79612640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d741f2c50 0x7f7d740cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d741f2c50 msgr2=0x7f7d740cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d741f2c50 0x7f7d740cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640047c0 msgr2=0x7f7d740c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640047c0 0x7f7d740c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 -- 172.21.15.53:0/2455201482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740c9950 con 0x7f7d7415bdb0 2023-10-21T16:21:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.587+0000 7f7d73fff640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7d60005040 tx=0x7f7d60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.588+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600076c0 con 0x7f7d7415bdb0 2023-10-21T16:21:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.588+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60007a30 con 0x7f7d7415bdb0 2023-10-21T16:21:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.588+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60006be0 con 0x7f7d7415bdb0 2023-10-21T16:21:05.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.588+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d740c9c30 con 0x7f7d7415bdb0 2023-10-21T16:21:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.588+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7415c4c0 con 0x7f7d7415bdb0 2023-10-21T16:21:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.589+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7404e160 con 0x7f7d7415bdb0 2023-10-21T16:21:05.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.590+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f7d60006d40 con 0x7f7d7415bdb0 2023-10-21T16:21:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.591+0000 7f7d717fa640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 0x7f7d5806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.591+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f7d600abe80 con 0x7f7d7415bdb0 2023-10-21T16:21:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.591+0000 7f7d78e11640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 msgr2=0x7f7d5806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:05.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.591+0000 7f7d78e11640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 0x7f7d5806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:05.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.599+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d600768b0 con 0x7f7d7415bdb0 2023-10-21T16:21:05.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.632+0000 7f1f0e841040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:05.660 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.658+0000 7f33a5852040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:05.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.692+0000 7f1f0e841040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:05.715 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.713+0000 7f33a5852040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:05.744 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.742+0000 7f1f0e841040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:05.768 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.767+0000 7f33a5852040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.793+0000 7f7d78e11640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 msgr2=0x7f7d5806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:05.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.793+0000 7f7d78e11640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 0x7f7d5806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:05.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.794+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d74073440 con 0x7f7d7415bdb0 2023-10-21T16:21:05.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.795+0000 7f7d717fa640 1 -- 172.21.15.53:0/2455201482 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d6004b020 con 0x7f7d7415bdb0 2023-10-21T16:21:05.797 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:05.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 msgr2=0x7f7d5806d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:05.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 0x7f7d5806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 msgr2=0x7f7d74151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:05.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7d60005040 tx=0x7f7d60001120 comp rx=0 tx=0).stop 2023-10-21T16:21:05.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 shutdown_connections 2023-10-21T16:21:05.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f7d5806b0e0 0x7f7d5806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d741f2c50 0x7f7d740cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d7415bdb0 0x7f7d74151e30 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 --2- 172.21.15.53:0/2455201482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640047c0 0x7f7d740c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:05.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 >> 172.21.15.53:0/2455201482 conn(0x7f7d7406e650 msgr2=0x7f7d740cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:05.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 shutdown_connections 2023-10-21T16:21:05.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:05.798+0000 7f7d7b075640 1 -- 172.21.15.53:0/2455201482 wait complete. 2023-10-21T16:21:05.818 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:21:05.836 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.834+0000 7f33a5852040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:05.855 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.853+0000 7f1f0e841040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:05.889 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:05.888+0000 7f33a5852040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:05.917 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.915+0000 7f1f0e841040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:05.965 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:05.964+0000 7f1f0e841040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:06.007 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.006+0000 7f33a5852040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:06.023 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.021+0000 7f1f0e841040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:06.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.041+0000 7f083ff41640 1 Processor -- start 2023-10-21T16:21:06.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.041+0000 7f083ff41640 1 -- start start 2023-10-21T16:21:06.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083ff41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f08381612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083ff41640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380b8c30 con 0x7f083815cd70 2023-10-21T16:21:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083ff41640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380cdd80 con 0x7f08380b4e10 2023-10-21T16:21:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083ff41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380ce060 con 0x7f08380cd4e0 2023-10-21T16:21:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083dcdd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f083815cd70 0x7f08380b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37868/0 (socket says 172.21.15.53:37868) 2023-10-21T16:21:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.043+0000 7f083dcdd640 1 -- 172.21.15.53:0/3452810140 learned_addr learned my addr 172.21.15.53:0/3452810140 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254436251 0 0) 0x7f08380b8c30 con 0x7f083815cd70 2023-10-21T16:21:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0824003520 con 0x7f083815cd70 2023-10-21T16:21:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083e4de640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f08381612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129976109 0 0) 0x7f0824003520 con 0x7f083815cd70 2023-10-21T16:21:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 msgr2=0x7f08381612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f08381612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 >> v1:172.21.15.124:6789/0 conn(0x7f08380b4e10 legacy=0x7f08380d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380ce340 con 0x7f083815cd70 2023-10-21T16:21:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0834001690 con 0x7f083815cd70 2023-10-21T16:21:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 >> v1:172.21.15.53:6789/0 conn(0x7f083815cd70 legacy=0x7f08380b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08240039a0 0x7f0824003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08240047c0 0x7f0824004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f0824005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380ce060 con 0x7f08240039a0 2023-10-21T16:21:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082400ba30 con 0x7f08240047c0 2023-10-21T16:21:06.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082400bbf0 con 0x7f08380cd4e0 2023-10-21T16:21:06.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.044+0000 7f083dcdd640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08240039a0 0x7f0824003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083e4de640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f0824005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083d4dc640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08240047c0 0x7f0824004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083dcdd640 1 -- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 msgr2=0x7f0824005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083dcdd640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08380cd4e0 0x7f0824005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083dcdd640 1 -- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08240047c0 msgr2=0x7f0824004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083dcdd640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08240047c0 0x7f0824004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.045+0000 7f083dcdd640 1 -- 172.21.15.53:0/3452810140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824003520 con 0x7f08240039a0 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.046+0000 7f083dcdd640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08240039a0 0x7f0824003dc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0834001eb0 tx=0x7f08240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bc26fc885a342c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.046+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834005ad0 con 0x7f08240039a0 2023-10-21T16:21:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.046+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0834005c30 con 0x7f08240039a0 2023-10-21T16:21:06.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.046+0000 7f083ccdb640 1 -- 172.21.15.53:0/3452810140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083402c870 con 0x7f08240039a0 2023-10-21T16:21:06.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 -- 172.21.15.53:0/3452810140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08240039a0 msgr2=0x7f0824003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 --2- 172.21.15.53:0/3452810140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08240039a0 0x7f0824003dc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0834001eb0 tx=0x7f08240057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083e4de640 1 -- 172.21.15.53:0/3452810140 reap_dead start 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 -- 172.21.15.53:0/3452810140 shutdown_connections 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 -- 172.21.15.53:0/3452810140 >> 172.21.15.53:0/3452810140 conn(0x7f08380cf700 msgr2=0x7f08380cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 -- 172.21.15.53:0/3452810140 shutdown_connections 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.047+0000 7f083ff41640 1 -- 172.21.15.53:0/3452810140 wait complete. 2023-10-21T16:21:06.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.049+0000 7f083ff41640 1 Processor -- start 2023-10-21T16:21:06.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.049+0000 7f083ff41640 1 -- start start 2023-10-21T16:21:06.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 0x7f083815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 0x7f08380cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380ca770 con 0x7f083814dfa0 2023-10-21T16:21:06.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380c8190 con 0x7f08380c7c10 2023-10-21T16:21:06.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.050+0000 7f083ff41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08380c9e80 con 0x7f08240039a0 2023-10-21T16:21:06.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42756/0 (socket says 172.21.15.53:42756) 2023-10-21T16:21:06.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 -- 172.21.15.53:0/2723299625 learned_addr learned my addr 172.21.15.53:0/2723299625 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083dcdd640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 0x7f083815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083e4de640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 0x7f08380cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 -- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 msgr2=0x7f083815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 0x7f083815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 -- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 msgr2=0x7f08380cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 0x7f08380cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 -- 172.21.15.53:0/2723299625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380ca120 con 0x7f083814dfa0 2023-10-21T16:21:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083dcdd640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 0x7f083815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083e4de640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 0x7f08380cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.051+0000 7f083d4dc640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f082800e4d0 tx=0x7f082800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.052+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280052e0 con 0x7f083814dfa0 2023-10-21T16:21:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.052+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0828005440 con 0x7f083814dfa0 2023-10-21T16:21:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.052+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828003810 con 0x7f083814dfa0 2023-10-21T16:21:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.052+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08380ca400 con 0x7f083814dfa0 2023-10-21T16:21:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.052+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083814e420 con 0x7f083814dfa0 2023-10-21T16:21:06.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.053+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083814e640 con 0x7f083814dfa0 2023-10-21T16:21:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.054+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f0828003970 con 0x7f083814dfa0 2023-10-21T16:21:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.055+0000 7f082e7fc640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 0x7f081c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.055+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f0828089ab0 con 0x7f083814dfa0 2023-10-21T16:21:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.055+0000 7f083dcdd640 1 -- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 msgr2=0x7f081c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.055+0000 7f083dcdd640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 0x7f081c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:06.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.059+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0828054520 con 0x7f083814dfa0 2023-10-21T16:21:06.074 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.072+0000 7f1f0e841040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:06.076 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.074+0000 7f33a5852040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:06.128 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.126+0000 7f33a5852040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:06.188 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.186+0000 7f33a5852040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:06.241 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.239+0000 7f33a5852040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:06.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.250+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f083804e160 con 0x7f083814dfa0 2023-10-21T16:21:06.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.251+0000 7f082e7fc640 1 -- 172.21.15.53:0/2723299625 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0828057b40 con 0x7f083814dfa0 2023-10-21T16:21:06.253 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:21:06.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 msgr2=0x7f081c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:06.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 0x7f081c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 msgr2=0x7f08380c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f082800e4d0 tx=0x7f082800e990 comp rx=0 tx=0).stop 2023-10-21T16:21:06.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 shutdown_connections 2023-10-21T16:21:06.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f081c06b0b0 0x7f081c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08380c7c10 0x7f08380cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f083814dfa0 0x7f08380c76d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 --2- 172.21.15.53:0/2723299625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240039a0 0x7f083815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.253+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 >> 172.21.15.53:0/2723299625 conn(0x7f08380cf700 msgr2=0x7f08380d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:06.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.254+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 shutdown_connections 2023-10-21T16:21:06.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.254+0000 7f083ff41640 1 -- 172.21.15.53:0/2723299625 wait complete. 2023-10-21T16:21:06.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:21:06.455 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.453+0000 7f1f0e841040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:06.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.534+0000 7f17b1e75640 1 Processor -- start 2023-10-21T16:21:06.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.534+0000 7f17b1e75640 1 -- start start 2023-10-21T16:21:06.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17b1e75640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17ac15ce00 0x7f17ac0cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17b1e75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17b1e75640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac0cde10 con 0x7f17ac160980 2023-10-21T16:21:06.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17b1e75640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac161920 con 0x7f17ac15ce00 2023-10-21T16:21:06.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17b1e75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0cb8b0 con 0x7f17ac0cd470 2023-10-21T16:21:06.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17abfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f17ac160980 0x7f17ac160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37878/0 (socket says 172.21.15.53:37878) 2023-10-21T16:21:06.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17abfff640 1 -- 172.21.15.53:0/1890460336 learned_addr learned my addr 172.21.15.53:0/1890460336 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:06.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17aaffd640 1 --2- 172.21.15.53:0/1890460336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.536+0000 7f17ab7fe640 1 --2- 172.21.15.53:0/1890460336 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15ce00 0x7f17ac0cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aa7fc640 1 -- 172.21.15.53:0/1890460336 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688342171 0 0) 0x7f17ac0cde10 con 0x7f17ac160980 2023-10-21T16:21:06.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aa7fc640 1 -- 172.21.15.53:0/1890460336 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1794003520 con 0x7f17ac160980 2023-10-21T16:21:06.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aaffd640 1 -- 172.21.15.53:0/1890460336 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15ce00 msgr2=0x7f17ac0cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aaffd640 1 --2- 172.21.15.53:0/1890460336 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15ce00 0x7f17ac0cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aaffd640 1 -- 172.21.15.53:0/1890460336 >> v1:172.21.15.53:6789/0 conn(0x7f17ac160980 legacy=0x7f17ac160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.537+0000 7f17aaffd640 1 -- 172.21.15.53:0/1890460336 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac161640 con 0x7f17ac0cd470 2023-10-21T16:21:06.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.538+0000 7f17aaffd640 1 --2- 172.21.15.53:0/1890460336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0cd8d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17a00097a0 tx=0x7f17a002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab5eebb71c2aa67 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.538+0000 7f17aa7fc640 1 -- 172.21.15.53:0/1890460336 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0031780 con 0x7f17ac0cd470 2023-10-21T16:21:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17aa7fc640 1 -- 172.21.15.53:0/1890460336 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a00318e0 con 0x7f17ac0cd470 2023-10-21T16:21:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17aa7fc640 1 -- 172.21.15.53:0/1890460336 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0031c50 con 0x7f17ac0cd470 2023-10-21T16:21:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 -- 172.21.15.53:0/1890460336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 msgr2=0x7f17ac0cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 --2- 172.21.15.53:0/1890460336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0cd8d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17a00097a0 tx=0x7f17a002b820 comp rx=0 tx=0).stop 2023-10-21T16:21:06.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 -- 172.21.15.53:0/1890460336 shutdown_connections 2023-10-21T16:21:06.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 --2- 172.21.15.53:0/1890460336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0cd8d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 --2- 172.21.15.53:0/1890460336 >> v2:172.21.15.124:3300/0 conn(0x7f17ac15ce00 0x7f17ac0cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 -- 172.21.15.53:0/1890460336 >> 172.21.15.53:0/1890460336 conn(0x7f17ac06e650 msgr2=0x7f17ac06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:06.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.539+0000 7f17b1e75640 1 -- 172.21.15.53:0/1890460336 shutdown_connections 2023-10-21T16:21:06.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.540+0000 7f17b1e75640 1 -- 172.21.15.53:0/1890460336 wait complete. 2023-10-21T16:21:06.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.542+0000 7f17b1e75640 1 Processor -- start 2023-10-21T16:21:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.542+0000 7f17b1e75640 1 -- start start 2023-10-21T16:21:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac15ce00 0x7f17ac1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac15f640 con 0x7f17ac160980 2023-10-21T16:21:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac0b3210 con 0x7f17ac15ce00 2023-10-21T16:21:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17b1e75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac1f2390 con 0x7f17ac0cd470 2023-10-21T16:21:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.543+0000 7f17abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42772/0 (socket says 172.21.15.53:42772) 2023-10-21T16:21:06.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 -- 172.21.15.53:0/2176248775 learned_addr learned my addr 172.21.15.53:0/2176248775 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17ab7fe640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17aaffd640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac15ce00 0x7f17ac1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 -- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 msgr2=0x7f17ac0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 -- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac15ce00 msgr2=0x7f17ac1f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac15ce00 0x7f17ac1f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 -- 172.21.15.53:0/2176248775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0009400 con 0x7f17ac160980 2023-10-21T16:21:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.544+0000 7f17abfff640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f17a400cb10 tx=0x7f17a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.545+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a400ac70 con 0x7f17ac160980 2023-10-21T16:21:06.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.545+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a400add0 con 0x7f17ac160980 2023-10-21T16:21:06.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.545+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4012620 con 0x7f17ac160980 2023-10-21T16:21:06.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.545+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac15ed10 con 0x7f17ac160980 2023-10-21T16:21:06.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.545+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac15f2d0 con 0x7f17ac160980 2023-10-21T16:21:06.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.546+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac06cff0 con 0x7f17ac160980 2023-10-21T16:21:06.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.547+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f17a4007070 con 0x7f17ac160980 2023-10-21T16:21:06.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.548+0000 7f1793fff640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 0x7f179806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.548+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f17a4089ce0 con 0x7f17ac160980 2023-10-21T16:21:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.549+0000 7f17ab7fe640 1 -- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 msgr2=0x7f179806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:06.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.549+0000 7f17ab7fe640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 0x7f179806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:06.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.553+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17a4054770 con 0x7f17ac160980 2023-10-21T16:21:06.617 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.615+0000 7f1f0e841040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:06.649 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.647+0000 7f33a5852040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:06.725 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.723+0000 7f1f0e841040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:06.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.743+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17ac04e160 con 0x7f17ac160980 2023-10-21T16:21:06.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.744+0000 7f1793fff640 1 -- 172.21.15.53:0/2176248775 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17a4017070 con 0x7f17ac160980 2023-10-21T16:21:06.745 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:21:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 msgr2=0x7f179806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 0x7f179806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 msgr2=0x7f17ac15fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f17a400cb10 tx=0x7f17a400a060 comp rx=0 tx=0).stop 2023-10-21T16:21:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 shutdown_connections 2023-10-21T16:21:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f179806b0b0 0x7f179806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17ac160980 0x7f17ac15fad0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17ac15ce00 0x7f17ac1f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 --2- 172.21.15.53:0/2176248775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17ac0cd470 0x7f17ac0b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.746+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 >> 172.21.15.53:0/2176248775 conn(0x7f17ac06e650 msgr2=0x7f17ac160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:06.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.747+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 shutdown_connections 2023-10-21T16:21:06.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.747+0000 7f17b1e75640 1 -- 172.21.15.53:0/2176248775 wait complete. 2023-10-21T16:21:06.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:21:06.774 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.773+0000 7f1f0e841040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:06.818 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.817+0000 7f33a5852040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:06.880 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.878+0000 7f1f0e841040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:06.933 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.931+0000 7f33a5852040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:06.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.967+0000 7f56adcbb640 1 Processor -- start 2023-10-21T16:21:06.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.967+0000 7f56adcbb640 1 -- start start 2023-10-21T16:21:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56adcbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f56a80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56adcbb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56a80cb130 0x7f56a80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56adcbb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a80b2620 con 0x7f56a806dfc0 2023-10-21T16:21:06.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56adcbb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a80cbae0 con 0x7f56a80cb130 2023-10-21T16:21:06.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56adcbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80b2060 con 0x7f56a80afbe0 2023-10-21T16:21:06.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f569ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f56a806dfc0 0x7f56a8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37880/0 (socket says 172.21.15.53:37880) 2023-10-21T16:21:06.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f569ffff640 1 -- 172.21.15.53:0/814955268 learned_addr learned my addr 172.21.15.53:0/814955268 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:06.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.969+0000 7f56a77fe640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f56a80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a7fff640 1 --2- 172.21.15.53:0/814955268 >> v2:172.21.15.124:3300/0 conn(0x7f56a80cb130 0x7f56a80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184363992 0 0) 0x7f56a80b2620 con 0x7f56a806dfc0 2023-10-21T16:21:06.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5694003520 con 0x7f56a806dfc0 2023-10-21T16:21:06.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900695892 0 0) 0x7f5694003520 con 0x7f56a806dfc0 2023-10-21T16:21:06.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 msgr2=0x7f56a80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f56a80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 >> v2:172.21.15.124:3300/0 conn(0x7f56a80cb130 msgr2=0x7f56a80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 --2- 172.21.15.53:0/814955268 >> v2:172.21.15.124:3300/0 conn(0x7f56a80cb130 0x7f56a80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80b2340 con 0x7f56a806dfc0 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5690001670 con 0x7f56a806dfc0 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 >> v1:172.21.15.53:6789/0 conn(0x7f56a806dfc0 legacy=0x7f56a8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f5694003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56940047c0 0x7f5694004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f5694005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80cbae0 con 0x7f5694003970 2023-10-21T16:21:06.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a80b2060 con 0x7f56940047c0 2023-10-21T16:21:06.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.970+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569400b980 con 0x7f56a80afbe0 2023-10-21T16:21:06.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f5694003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a7fff640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f5694005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f569ffff640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56940047c0 0x7f5694004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 -- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 msgr2=0x7f5694005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f5694005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 -- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56940047c0 msgr2=0x7f5694004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:06.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56940047c0 0x7f5694004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a77fe640 1 -- 172.21.15.53:0/814955268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5694003520 con 0x7f5694003970 2023-10-21T16:21:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.971+0000 7f56a7fff640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80afbe0 0x7f5694005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.972+0000 7f56a77fe640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f5694003d90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ac0 tx=0x7f56940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb7442e0ae00d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.972+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569802c6b0 con 0x7f5694003970 2023-10-21T16:21:06.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569802c810 con 0x7f5694003970 2023-10-21T16:21:06.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56a6ffd640 1 -- 172.21.15.53:0/814955268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569802cba0 con 0x7f5694003970 2023-10-21T16:21:06.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56adcbb640 1 -- 172.21.15.53:0/814955268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 msgr2=0x7f5694003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56adcbb640 1 --2- 172.21.15.53:0/814955268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f5694003d90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ac0 tx=0x7f56940057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56a7fff640 1 -- 172.21.15.53:0/814955268 reap_dead start 2023-10-21T16:21:06.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56adcbb640 1 -- 172.21.15.53:0/814955268 shutdown_connections 2023-10-21T16:21:06.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.973+0000 7f56adcbb640 1 -- 172.21.15.53:0/814955268 >> 172.21.15.53:0/814955268 conn(0x7f56a806a5d0 msgr2=0x7f56a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:06.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.974+0000 7f56adcbb640 1 -- 172.21.15.53:0/814955268 shutdown_connections 2023-10-21T16:21:06.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.974+0000 7f56adcbb640 1 -- 172.21.15.53:0/814955268 wait complete. 2023-10-21T16:21:06.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.976+0000 7f56adcbb640 1 Processor -- start 2023-10-21T16:21:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.977+0000 7f56adcbb640 1 -- start start 2023-10-21T16:21:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80c5810 0x7f56a80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a8152650 0x7f56a81580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8151890 con 0x7f5694003970 2023-10-21T16:21:06.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8151b70 con 0x7f56a8152650 2023-10-21T16:21:06.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56adcbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8151e50 con 0x7f56a80c5810 2023-10-21T16:21:06.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.978+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42788/0 (socket says 172.21.15.53:42788) 2023-10-21T16:21:06.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 -- 172.21.15.53:0/2305652364 learned_addr learned my addr 172.21.15.53:0/2305652364 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f569ffff640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80c5810 0x7f56a80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a7fff640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a8152650 0x7f56a81580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80c5810 msgr2=0x7f56a80c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80c5810 0x7f56a80c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a8152650 msgr2=0x7f56a81580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:06.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a8152650 0x7f56a81580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:06.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 -- 172.21.15.53:0/2305652364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8152130 con 0x7f5694003970 2023-10-21T16:21:06.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.979+0000 7f56a77fe640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ac0 tx=0x7f5698004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:06.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.980+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698003810 con 0x7f5694003970 2023-10-21T16:21:06.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.980+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5698003970 con 0x7f5694003970 2023-10-21T16:21:06.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.980+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a8150130 con 0x7f5694003970 2023-10-21T16:21:06.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.980+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698003d00 con 0x7f5694003970 2023-10-21T16:21:06.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.980+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a8152ad0 con 0x7f5694003970 2023-10-21T16:21:06.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.981+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a804e160 con 0x7f5694003970 2023-10-21T16:21:06.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.982+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f5698040450 con 0x7f5694003970 2023-10-21T16:21:06.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:06.986+0000 7f1f0e841040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:06.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.984+0000 7f56a4ff9640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 0x7f568806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:06.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.984+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f56980abc80 con 0x7f5694003970 2023-10-21T16:21:06.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.984+0000 7f569ffff640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 msgr2=0x7f568806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:06.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.984+0000 7f569ffff640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 0x7f568806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:06.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:06.987+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5698076720 con 0x7f5694003970 2023-10-21T16:21:06.992 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:06.989+0000 7f33a5852040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:07.105 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:07.103+0000 7f33a5852040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:07.143 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:07.140+0000 7f1f0e841040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:07.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.185+0000 7f569ffff640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 msgr2=0x7f568806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:07.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.185+0000 7f569ffff640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 0x7f568806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:07.191 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:07.189+0000 7f1f0e841040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:07.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.193+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56a81519d0 con 0x7f5694003970 2023-10-21T16:21:07.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.194+0000 7f56a4ff9640 1 -- 172.21.15.53:0/2305652364 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v134) v1 ==== 80+0+475 (secure 0 0 0) 0x7f56a81519d0 con 0x7f5694003970 2023-10-21T16:21:07.195 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/596536042 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.195 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/897501407 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.196 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/596536042 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.196 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3242447571 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.196 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1097749659 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.196 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2545128602 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.196 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1535068499 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.197 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3443628834 2023-10-22T16:20:39.664146+0000 2023-10-21T16:21:07.198 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T16:21:07.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 msgr2=0x7f568806d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:07.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 0x7f568806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 msgr2=0x7f56a814fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:07.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ac0 tx=0x7f5698004730 comp rx=0 tx=0).stop 2023-10-21T16:21:07.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 shutdown_connections 2023-10-21T16:21:07.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f568806b0e0 0x7f568806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a8152650 0x7f56a81580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a80c5810 0x7f56a80c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 --2- 172.21.15.53:0/2305652364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5694003970 0x7f56a814fbf0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 >> 172.21.15.53:0/2305652364 conn(0x7f56a806a5d0 msgr2=0x7f56a80ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:07.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 shutdown_connections 2023-10-21T16:21:07.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.196+0000 7f56adcbb640 1 -- 172.21.15.53:0/2305652364 wait complete. 2023-10-21T16:21:07.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:21:07.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:07.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:07.219 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:07.217+0000 7f33a5852040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:07.310 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:07.308+0000 7f1f0e841040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:07.318 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:21:07.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:07.319 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:21:07.368 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:07.369 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:07.387 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:07.385+0000 7f33a5852040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:07.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:07.424 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:07.424 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:07.438 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:07.437+0000 7f33a5852040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:07.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:21:07.483 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:07.483 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:07.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:07.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:07.566 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:07.564+0000 7f33a5852040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.759+0000 7fa553978640 1 Processor -- start 2023-10-21T16:21:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.759+0000 7fa553978640 1 -- start start 2023-10-21T16:21:07.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa553978640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa54c0b4db0 0x7fa54c0b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa553978640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa54c0b10b0 0x7fa54c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa553978640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c0b14f0 con 0x7fa54c0cd390 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa553978640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c0cdbc0 con 0x7fa54c0b10b0 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa553978640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c159ef0 con 0x7fa54c0b4db0 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa550f13640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa54c0b10b0 0x7fa54c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa550f13640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa54c0b10b0 0x7fa54c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42790/0 (socket says 172.21.15.53:42790) 2023-10-21T16:21:07.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa550f13640 1 -- 172.21.15.53:0/3068458776 learned_addr learned my addr 172.21.15.53:0/3068458776 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:07.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.761+0000 7fa551714640 1 --2- 172.21.15.53:0/3068458776 >> v2:172.21.15.124:3300/0 conn(0x7fa54c0b4db0 0x7fa54c0b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.762+0000 7fa550f13640 1 -- 172.21.15.53:0/3068458776 >> v2:172.21.15.124:3300/0 conn(0x7fa54c0b4db0 msgr2=0x7fa54c0b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.762+0000 7fa550f13640 1 --2- 172.21.15.53:0/3068458776 >> v2:172.21.15.124:3300/0 conn(0x7fa54c0b4db0 0x7fa54c0b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.762+0000 7fa550f13640 1 -- 172.21.15.53:0/3068458776 >> v1:172.21.15.124:6789/0 conn(0x7fa54c0cd390 legacy=0x7fa54c0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.762+0000 7fa550f13640 1 -- 172.21.15.53:0/3068458776 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c06acb0 con 0x7fa54c0b10b0 2023-10-21T16:21:07.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.763+0000 7fa550f13640 1 --2- 172.21.15.53:0/3068458776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b10b0 0x7fa54c0cce50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0b1730 tx=0x7fa53c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aa578bbc0d79d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:07.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.763+0000 7fa543fff640 1 -- 172.21.15.53:0/3068458776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c030860 con 0x7fa54c0b10b0 2023-10-21T16:21:07.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.763+0000 7fa543fff640 1 -- 172.21.15.53:0/3068458776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c0309c0 con 0x7fa54c0b10b0 2023-10-21T16:21:07.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.763+0000 7fa543fff640 1 -- 172.21.15.53:0/3068458776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c030d50 con 0x7fa54c0b10b0 2023-10-21T16:21:07.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 -- 172.21.15.53:0/3068458776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b10b0 msgr2=0x7fa54c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:07.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 --2- 172.21.15.53:0/3068458776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b10b0 0x7fa54c0cce50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0b1730 tx=0x7fa53c02b9a0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 -- 172.21.15.53:0/3068458776 shutdown_connections 2023-10-21T16:21:07.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 --2- 172.21.15.53:0/3068458776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b10b0 0x7fa54c0cce50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 --2- 172.21.15.53:0/3068458776 >> v2:172.21.15.124:3300/0 conn(0x7fa54c0b4db0 0x7fa54c0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 -- 172.21.15.53:0/3068458776 >> 172.21.15.53:0/3068458776 conn(0x7fa54c0cf890 msgr2=0x7fa54c0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:07.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 -- 172.21.15.53:0/3068458776 shutdown_connections 2023-10-21T16:21:07.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.764+0000 7fa553978640 1 -- 172.21.15.53:0/3068458776 wait complete. 2023-10-21T16:21:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.765+0000 7fa553978640 1 Processor -- start 2023-10-21T16:21:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.766+0000 7fa553978640 1 -- start start 2023-10-21T16:21:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.766+0000 7fa553978640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c0b10b0 0x7fa54c146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.766+0000 7fa553978640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa553978640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 0x7fa54c0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa553978640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c145490 con 0x7fa54c0b4db0 2023-10-21T16:21:07.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa553978640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c145770 con 0x7fa54c0cd390 2023-10-21T16:21:07.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa553978640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c145a50 con 0x7fa54c0b10b0 2023-10-21T16:21:07.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:07.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42794/0 (socket says 172.21.15.53:42794) 2023-10-21T16:21:07.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 -- 172.21.15.53:0/1749858430 learned_addr learned my addr 172.21.15.53:0/1749858430 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:07.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa551f15640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 0x7fa54c0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:07.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c0b10b0 msgr2=0x7fa54c146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:07.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c0b10b0 0x7fa54c146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 msgr2=0x7fa54c0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 0x7fa54c0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa550f13640 1 -- 172.21.15.53:0/1749858430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0094d0 con 0x7fa54c0b4db0 2023-10-21T16:21:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.767+0000 7fa551f15640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 0x7fa54c0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa550f13640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0097f0 tx=0x7fa53c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c039c90 con 0x7fa54c0b4db0 2023-10-21T16:21:07.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c039df0 con 0x7fa54c0b4db0 2023-10-21T16:21:07.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c043620 con 0x7fa54c0b4db0 2023-10-21T16:21:07.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c145d30 con 0x7fa54c0b4db0 2023-10-21T16:21:07.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.768+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c0b3280 con 0x7fa54c0b4db0 2023-10-21T16:21:07.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.769+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c066d80 con 0x7fa54c0b4db0 2023-10-21T16:21:07.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.771+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa53c043780 con 0x7fa54c0b4db0 2023-10-21T16:21:07.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.773+0000 7fa541ffb640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:07.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.773+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fa53c03e070 con 0x7fa54c0b4db0 2023-10-21T16:21:07.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.773+0000 7fa551714640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 msgr2=0x7fa53006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:07.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.773+0000 7fa551714640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:07.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.777+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa53c077950 con 0x7fa54c0b4db0 2023-10-21T16:21:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.974+0000 7fa551714640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 msgr2=0x7fa53006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:07.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:07.974+0000 7fa551714640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.009+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa54c073320 con 0x7fa54c0b4db0 2023-10-21T16:21:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.010+0000 7fa541ffb640 1 -- 172.21.15.53:0/1749858430 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa53c034070 con 0x7fa54c0b4db0 2023-10-21T16:21:08.012 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:08.012 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:21:08.013 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:21:08.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 msgr2=0x7fa53006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:08.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 msgr2=0x7fa54c14a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0097f0 tx=0x7fa53c004730 comp rx=0 tx=0).stop 2023-10-21T16:21:08.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 shutdown_connections 2023-10-21T16:21:08.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0cd390 0x7fa54c0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0b4db0 0x7fa54c14a3e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 --2- 172.21.15.53:0/1749858430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c0b10b0 0x7fa54c146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 >> 172.21.15.53:0/1749858430 conn(0x7fa54c0cf890 msgr2=0x7fa54c14ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:08.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 shutdown_connections 2023-10-21T16:21:08.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.013+0000 7fa553978640 1 -- 172.21.15.53:0/1749858430 wait complete. 2023-10-21T16:21:08.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:08.150 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.148+0000 7f1f0e841040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.268+0000 7f71af59e640 1 Processor -- start 2023-10-21T16:21:08.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.269+0000 7f71af59e640 1 -- start start 2023-10-21T16:21:08.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.270+0000 7f71af59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f71b00d1640 0x7f71b00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.270+0000 7f71af59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00ceb00 con 0x7f71b015cdf0 2023-10-21T16:21:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.270+0000 7f71af59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00ce820 con 0x7f71b00d07c0 2023-10-21T16:21:08.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.270+0000 7f71af59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b00ce610 con 0x7f71b00d1640 2023-10-21T16:21:08.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ae59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f71b015cdf0 0x7f71b00d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37886/0 (socket says 172.21.15.53:37886) 2023-10-21T16:21:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ae59c640 1 -- 172.21.15.53:0/2881951307 learned_addr learned my addr 172.21.15.53:0/2881951307 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71add9b640 1 --2- 172.21.15.53:0/2881951307 >> v2:172.21.15.124:3300/0 conn(0x7f71b00d1640 0x7f71b00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130970221 0 0) 0x7f71b00ceb00 con 0x7f71b015cdf0 2023-10-21T16:21:08.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719c003520 con 0x7f71b015cdf0 2023-10-21T16:21:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154392453 0 0) 0x7f719c003520 con 0x7f71b015cdf0 2023-10-21T16:21:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 >> v2:172.21.15.124:3300/0 conn(0x7f71b00d1640 msgr2=0x7f71b00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 --2- 172.21.15.53:0/2881951307 >> v2:172.21.15.124:3300/0 conn(0x7f71b00d1640 0x7f71b00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 >> v1:172.21.15.124:6789/0 conn(0x7f71b00d07c0 legacy=0x7f71b00b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00b5960 con 0x7f71b015cdf0 2023-10-21T16:21:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f71a00019f0 con 0x7f71b015cdf0 2023-10-21T16:21:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.271+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 >> v1:172.21.15.53:6789/0 conn(0x7f71b015cdf0 legacy=0x7f71b00d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 0x7f719c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f719c0047c0 0x7f719c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c0052a0 0x7f719c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00ce610 con 0x7f719c003970 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c00c170 con 0x7f719c0047c0 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c00c330 con 0x7f719c0052a0 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 0x7f719c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71add9b640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f719c0047c0 0x7f719c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71aed9d640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c0052a0 0x7f719c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 -- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c0052a0 msgr2=0x7f719c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c0052a0 0x7f719c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 -- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f719c0047c0 msgr2=0x7f719c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f719c0047c0 0x7f719c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71ae59c640 1 -- 172.21.15.53:0/2881951307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c003520 con 0x7f719c003970 2023-10-21T16:21:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71aed9d640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c0052a0 0x7f719c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.272+0000 7f71aed9d640 1 -- 172.21.15.53:0/2881951307 reap_dead start 2023-10-21T16:21:08.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.273+0000 7f71ae59c640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 0x7f719c003d90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f71a000a800 tx=0x7f719c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcfd1a7f65dfffb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:08.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00072d0 con 0x7f719c003970 2023-10-21T16:21:08.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a002f070 con 0x7f719c003970 2023-10-21T16:21:08.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71ad59a640 1 -- 172.21.15.53:0/2881951307 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00076f0 con 0x7f719c003970 2023-10-21T16:21:08.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 -- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 msgr2=0x7f719c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 0x7f719c003d90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f71a000a800 tx=0x7f719c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 -- 172.21.15.53:0/2881951307 shutdown_connections 2023-10-21T16:21:08.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 --2- 172.21.15.53:0/2881951307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f719c003970 0x7f719c003d90 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 -- 172.21.15.53:0/2881951307 >> 172.21.15.53:0/2881951307 conn(0x7f71b009bcb0 msgr2=0x7f71b00d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 -- 172.21.15.53:0/2881951307 shutdown_connections 2023-10-21T16:21:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.274+0000 7f71af59e640 1 -- 172.21.15.53:0/2881951307 wait complete. 2023-10-21T16:21:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.276+0000 7f71af59e640 1 Processor -- start 2023-10-21T16:21:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.276+0000 7f71af59e640 1 -- start start 2023-10-21T16:21:08.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c003970 0x7f71b00a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71b00c62b0 0x7f71b0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01f6040 con 0x7f71b01f5680 2023-10-21T16:21:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01f64b0 con 0x7f71b00c62b0 2023-10-21T16:21:08.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71af59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00c6820 con 0x7f719c003970 2023-10-21T16:21:08.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.277+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42820/0 (socket says 172.21.15.53:42820) 2023-10-21T16:21:08.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 -- 172.21.15.53:0/735513561 learned_addr learned my addr 172.21.15.53:0/735513561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:08.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71add9b640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71b00c62b0 0x7f71b0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c003970 msgr2=0x7f71b00a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c003970 0x7f71b00a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71b00c62b0 msgr2=0x7f71b0158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71b00c62b0 0x7f71b0158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 -- 172.21.15.53:0/735513561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00c6b00 con 0x7f71b01f5680 2023-10-21T16:21:08.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71aed9d640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c99d0 tx=0x7f71a800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.278+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800b070 con 0x7f71b01f5680 2023-10-21T16:21:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.279+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a80035f0 con 0x7f71b01f5680 2023-10-21T16:21:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.279+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a80039a0 con 0x7f71b01f5680 2023-10-21T16:21:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.279+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00c6de0 con 0x7f71b01f5680 2023-10-21T16:21:08.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.279+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b00c73a0 con 0x7f71b01f5680 2023-10-21T16:21:08.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.280+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b015f7c0 con 0x7f71b01f5680 2023-10-21T16:21:08.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.281+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 61) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f71a8003b00 con 0x7f71b01f5680 2023-10-21T16:21:08.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.282+0000 7f71a6ffd640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 0x7f719006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.282+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f71a80894e0 con 0x7f71b01f5680 2023-10-21T16:21:08.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.282+0000 7f71ae59c640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 msgr2=0x7f719006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:08.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.282+0000 7f71ae59c640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 0x7f719006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:08.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.286+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71a8053db0 con 0x7f71b01f5680 2023-10-21T16:21:08.413 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:08.411+0000 7f33a5852040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:08.444 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.442+0000 7f1f0e841040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.478+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b00ce3f0 con 0x7f71b01f5680 2023-10-21T16:21:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.481+0000 7f71a6ffd640 1 -- 172.21.15.53:0/735513561 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17280 (secure 0 0 0) 0x7f71a80573d0 con 0x7f71b01f5680 2023-10-21T16:21:08.483 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:08.483 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:20:39.664268+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6832/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/897501407":"2023-10-22T16:20:39.664146+0000","172.21.15.124:6833/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3242447571":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1097749659":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2545128602":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1535068499":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3443628834":"2023-10-22T16:20:39.664146+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.483+0000 7f71ae59c640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 msgr2=0x7f719006d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/400910459 2023-10-21T16:21:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.483+0000 7f71ae59c640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 0x7f719006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 msgr2=0x7f719006d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 0x7f719006d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 msgr2=0x7f71b01f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c99d0 tx=0x7f71a800d890 comp rx=0 tx=0).stop 2023-10-21T16:21:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 shutdown_connections 2023-10-21T16:21:08.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:6832/400910459,v1:172.21.15.124:6833/400910459] conn(0x7f719006b0e0 0x7f719006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b01f5680 0x7f71b01f5b00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71b00c62b0 0x7f71b0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 --2- 172.21.15.53:0/735513561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f719c003970 0x7f71b00a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 >> 172.21.15.53:0/735513561 conn(0x7f71b009bcb0 msgr2=0x7f71b00d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 shutdown_connections 2023-10-21T16:21:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.484+0000 7f71af59e640 1 -- 172.21.15.53:0/735513561 wait complete. 2023-10-21T16:21:08.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.498+0000 7f1f0e841040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:08.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:20:22.870339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:20:23.949173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:08.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:08.615 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.613+0000 7f1f0e841040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:08.709 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e93bba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.709 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e93bba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.709 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e93bba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.709 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e93bba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.710 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e93bba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.710 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e903b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.711 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e903b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.711 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e903b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.711 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e903b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.711 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:08.707+0000 7f1e903b3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.723+0000 7f3537de2640 1 Processor -- start 2023-10-21T16:21:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.723+0000 7f3537de2640 1 -- start start 2023-10-21T16:21:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 0x7f35300d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300d26d0 0x7f35300d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35300b85d0 0x7f35300d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300d1630 con 0x7f35300b85d0 2023-10-21T16:21:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300d1910 con 0x7f353015cb80 2023-10-21T16:21:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.724+0000 7f3537de2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300d1bf0 con 0x7f35300d26d0 2023-10-21T16:21:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35300b85d0 0x7f35300d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35300b85d0 0x7f35300d3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42826/0 (socket says 172.21.15.53:42826) 2023-10-21T16:21:08.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 -- 172.21.15.53:0/3809044908 learned_addr learned my addr 172.21.15.53:0/3809044908 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:08.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353537d640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300d26d0 0x7f35300d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f3535b7e640 1 --2- 172.21.15.53:0/3809044908 >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 0x7f35300d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 -- 172.21.15.53:0/3809044908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300d26d0 msgr2=0x7f35300d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:08.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300d26d0 0x7f35300d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 -- 172.21.15.53:0/3809044908 >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 msgr2=0x7f35300d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:08.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 --2- 172.21.15.53:0/3809044908 >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 0x7f35300d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f353637f640 1 -- 172.21.15.53:0/3809044908 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300d1ed0 con 0x7f35300b85d0 2023-10-21T16:21:08.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.725+0000 7f3535b7e640 1 --2- 172.21.15.53:0/3809044908 >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 0x7f35300d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:08.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.726+0000 7f353637f640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35300b85d0 0x7f35300d3110 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f35300b5790 tx=0x7f352802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453df798399da78f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:08.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3534b7c640 1 -- 172.21.15.53:0/3809044908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352802ca80 con 0x7f35300b85d0 2023-10-21T16:21:08.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3534b7c640 1 -- 172.21.15.53:0/3809044908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352802cbe0 con 0x7f35300b85d0 2023-10-21T16:21:08.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3534b7c640 1 -- 172.21.15.53:0/3809044908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35280343f0 con 0x7f35300b85d0 2023-10-21T16:21:08.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 -- 172.21.15.53:0/3809044908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35300b85d0 msgr2=0x7f35300d3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35300b85d0 0x7f35300d3110 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f35300b5790 tx=0x7f352802bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 -- 172.21.15.53:0/3809044908 shutdown_connections 2023-10-21T16:21:08.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300d26d0 0x7f35300d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 --2- 172.21.15.53:0/3809044908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35300b85d0 0x7f35300d3110 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 --2- 172.21.15.53:0/3809044908 >> v2:172.21.15.124:3300/0 conn(0x7f353015cb80 0x7f35300d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.727+0000 7f3537de2640 1 -- 172.21.15.53:0/3809044908 >> 172.21.15.53:0/3809044908 conn(0x7f35300718a0 msgr2=0x7f3530071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.728+0000 7f3537de2640 1 -- 172.21.15.53:0/3809044908 shutdown_connections 2023-10-21T16:21:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.728+0000 7f3537de2640 1 -- 172.21.15.53:0/3809044908 wait complete. 2023-10-21T16:21:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.729+0000 7f3537de2640 1 Processor -- start 2023-10-21T16:21:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.729+0000 7f3537de2640 1 -- start start 2023-10-21T16:21:08.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 0x7f35300cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 0x7f3530151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:08.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530152190 con 0x7f353015cb80 2023-10-21T16:21:08.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530152380 con 0x7f35300d26d0 2023-10-21T16:21:08.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.730+0000 7f3537de2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3530152660 con 0x7f35300b85d0 2023-10-21T16:21:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42838/0 (socket says 172.21.15.53:42838) 2023-10-21T16:21:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 -- 172.21.15.53:0/4170991804 learned_addr learned my addr 172.21.15.53:0/4170991804 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f3535b7e640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 0x7f35300cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353537d640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 0x7f3530151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 -- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 msgr2=0x7f35300cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 0x7f35300cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 -- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 msgr2=0x7f3530151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 0x7f3530151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 -- 172.21.15.53:0/4170991804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528009b30 con 0x7f353015cb80 2023-10-21T16:21:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f3535b7e640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 0x7f35300cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353537d640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 0x7f3530151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.731+0000 7f353637f640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3528036040 tx=0x7f3528004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.732+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528009510 con 0x7f353015cb80 2023-10-21T16:21:08.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.732+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352803e450 con 0x7f353015cb80 2023-10-21T16:21:08.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.732+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530152940 con 0x7f353015cb80 2023-10-21T16:21:08.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.732+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352803e800 con 0x7f353015cb80 2023-10-21T16:21:08.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.732+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3530152f00 con 0x7f353015cb80 2023-10-21T16:21:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.733+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353004e160 con 0x7f353015cb80 2023-10-21T16:21:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.734+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 62) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f352802cd50 con 0x7f353015cb80 2023-10-21T16:21:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.735+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f35280abf60 con 0x7f353015cb80 2023-10-21T16:21:08.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.740+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35280deca0 con 0x7f353015cb80 2023-10-21T16:21:08.745 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:08.744+0000 7f33a5852040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:08.805 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:08.803+0000 7f33a5852040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:08.924 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:08.922+0000 7f33a5852040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:08.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.932+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35300c9ad0 con 0x7f353015cb80 2023-10-21T16:21:08.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.933+0000 7f35267fc640 1 -- 172.21.15.53:0/4170991804 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3528076ff0 con 0x7f353015cb80 2023-10-21T16:21:08.934 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:08.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 msgr2=0x7f353014fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:08.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3528036040 tx=0x7f3528004a40 comp rx=0 tx=0).stop 2023-10-21T16:21:08.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 shutdown_connections 2023-10-21T16:21:08.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f353015cb80 0x7f353014fab0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300d26d0 0x7f3530151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 --2- 172.21.15.53:0/4170991804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300b85d0 0x7f35300cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:08.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 >> 172.21.15.53:0/4170991804 conn(0x7f35300718a0 msgr2=0x7f35300b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:08.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.935+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 shutdown_connections 2023-10-21T16:21:08.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:08.936+0000 7f3537de2640 1 -- 172.21.15.53:0/4170991804 wait complete. 2023-10-21T16:21:08.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:21:09.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.173+0000 7f89a1e1a640 1 Processor -- start 2023-10-21T16:21:09.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.173+0000 7f89a1e1a640 1 -- start start 2023-10-21T16:21:09.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a1e1a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f899c15a820 0x7f899c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a1e1a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f899c15ac00 0x7f899c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a1e1a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c0cab60 con 0x7f899c0ca680 2023-10-21T16:21:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a1e1a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0cb960 con 0x7f899c15a820 2023-10-21T16:21:09.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a1e1a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0cbdf0 con 0x7f899c15ac00 2023-10-21T16:21:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f899c15a820 0x7f899c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f899c15a820 0x7f899c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42852/0 (socket says 172.21.15.53:42852) 2023-10-21T16:21:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 -- 172.21.15.53:0/671394555 learned_addr learned my addr 172.21.15.53:0/671394555 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 -- 172.21.15.53:0/671394555 >> v2:172.21.15.124:3300/0 conn(0x7f899c15ac00 msgr2=0x7f899c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:09.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 --2- 172.21.15.53:0/671394555 >> v2:172.21.15.124:3300/0 conn(0x7f899c15ac00 0x7f899c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 -- 172.21.15.53:0/671394555 >> v1:172.21.15.53:6789/0 conn(0x7f899c0ca680 legacy=0x7f899c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.175+0000 7f89a0e18640 1 -- 172.21.15.53:0/671394555 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c0cb030 con 0x7f899c15a820 2023-10-21T16:21:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.176+0000 7f89a0e18640 1 --2- 172.21.15.53:0/671394555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c0aff70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f899c0b0e00 tx=0x7f899802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960cbf31e5c60d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89937fe640 1 -- 172.21.15.53:0/671394555 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998031870 con 0x7f899c15a820 2023-10-21T16:21:09.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89937fe640 1 -- 172.21.15.53:0/671394555 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89980319d0 con 0x7f899c15a820 2023-10-21T16:21:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89937fe640 1 -- 172.21.15.53:0/671394555 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998031d60 con 0x7f899c15a820 2023-10-21T16:21:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89a1e1a640 1 -- 172.21.15.53:0/671394555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 msgr2=0x7f899c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/671394555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c0aff70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f899c0b0e00 tx=0x7f899802b910 comp rx=0 tx=0).stop 2023-10-21T16:21:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89a1e1a640 1 -- 172.21.15.53:0/671394555 shutdown_connections 2023-10-21T16:21:09.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/671394555 >> v2:172.21.15.124:3300/0 conn(0x7f899c15ac00 0x7f899c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.177+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/671394555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c0aff70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.178+0000 7f89a1e1a640 1 -- 172.21.15.53:0/671394555 >> 172.21.15.53:0/671394555 conn(0x7f899c06c050 msgr2=0x7f899c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.178+0000 7f89a1e1a640 1 -- 172.21.15.53:0/671394555 shutdown_connections 2023-10-21T16:21:09.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.178+0000 7f89a1e1a640 1 -- 172.21.15.53:0/671394555 wait complete. 2023-10-21T16:21:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.179+0000 7f89a1e1a640 1 Processor -- start 2023-10-21T16:21:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.179+0000 7f89a1e1a640 1 -- start start 2023-10-21T16:21:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0ca680 0x7f899c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 0x7f899c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1f6590 con 0x7f899c15a820 2023-10-21T16:21:09.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c15b6d0 con 0x7f899c15ac00 2023-10-21T16:21:09.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.180+0000 7f89a1e1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c15b9b0 con 0x7f899c0ca680 2023-10-21T16:21:09.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42854/0 (socket says 172.21.15.53:42854) 2023-10-21T16:21:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 -- 172.21.15.53:0/537553944 learned_addr learned my addr 172.21.15.53:0/537553944 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f89a0e18640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0ca680 0x7f899c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 -- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0ca680 msgr2=0x7f899c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f89a1619640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 0x7f899c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0ca680 0x7f899c15b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 -- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 msgr2=0x7f899c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 0x7f899c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f8993fff640 1 -- 172.21.15.53:0/537553944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980094d0 con 0x7f899c15a820 2023-10-21T16:21:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.181+0000 7f89a1619640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 0x7f899c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f8993fff640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c400 tx=0x7f898c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c0052d0 con 0x7f899c15a820 2023-10-21T16:21:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c005430 con 0x7f899c15a820 2023-10-21T16:21:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c15bc90 con 0x7f899c15a820 2023-10-21T16:21:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c003830 con 0x7f899c15a820 2023-10-21T16:21:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.182+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c15bf70 con 0x7f899c15a820 2023-10-21T16:21:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.183+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c074150 con 0x7f899c15a820 2023-10-21T16:21:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.184+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 62) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f898c003990 con 0x7f899c15a820 2023-10-21T16:21:09.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.185+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f898c088bd0 con 0x7f899c15a820 2023-10-21T16:21:09.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.190+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f898c053b70 con 0x7f899c15a820 2023-10-21T16:21:09.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.373+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f899c159e20 con 0x7f899c15a820 2023-10-21T16:21:09.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.375+0000 7f89917fa640 1 -- 172.21.15.53:0/537553944 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7f898c017020 con 0x7f899c15a820 2023-10-21T16:21:09.376 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:21:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 msgr2=0x7f899c1f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c400 tx=0x7f898c00c8c0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 shutdown_connections 2023-10-21T16:21:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c15ac00 0x7f899c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f899c15a820 0x7f899c1f6050 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 --2- 172.21.15.53:0/537553944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0ca680 0x7f899c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 >> 172.21.15.53:0/537553944 conn(0x7f899c06c050 msgr2=0x7f899c0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 shutdown_connections 2023-10-21T16:21:09.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.377+0000 7f89a1e1a640 1 -- 172.21.15.53:0/537553944 wait complete. 2023-10-21T16:21:09.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:21:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.614+0000 7efe8d990640 1 Processor -- start 2023-10-21T16:21:09.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.614+0000 7efe8d990640 1 -- start start 2023-10-21T16:21:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.615+0000 7efe8d990640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe880b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.615+0000 7efe8d990640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe880cb130 0x7efe880cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.615+0000 7efe8d990640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe880b2620 con 0x7efe8806dfc0 2023-10-21T16:21:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.615+0000 7efe8d990640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880cbae0 con 0x7efe880cb130 2023-10-21T16:21:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.615+0000 7efe8d990640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe880b2060 con 0x7efe880afbe0 2023-10-21T16:21:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe7ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe8806dfc0 0x7efe88070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37898/0 (socket says 172.21.15.53:37898) 2023-10-21T16:21:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe7ffff640 1 -- 172.21.15.53:0/2462534994 learned_addr learned my addr 172.21.15.53:0/2462534994 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:09.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe86ffd640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe880b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe877fe640 1 --2- 172.21.15.53:0/2462534994 >> v2:172.21.15.124:3300/0 conn(0x7efe880cb130 0x7efe880cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848482488 0 0) 0x7efe880b2620 con 0x7efe8806dfc0 2023-10-21T16:21:09.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.616+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe74003520 con 0x7efe8806dfc0 2023-10-21T16:21:09.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516805510 0 0) 0x7efe74003520 con 0x7efe8806dfc0 2023-10-21T16:21:09.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 msgr2=0x7efe880b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:09.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe880b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 >> v2:172.21.15.124:3300/0 conn(0x7efe880cb130 msgr2=0x7efe880cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:09.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 --2- 172.21.15.53:0/2462534994 >> v2:172.21.15.124:3300/0 conn(0x7efe880cb130 0x7efe880cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880b2340 con 0x7efe8806dfc0 2023-10-21T16:21:09.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7efe70001670 con 0x7efe8806dfc0 2023-10-21T16:21:09.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 >> v1:172.21.15.53:6789/0 conn(0x7efe8806dfc0 legacy=0x7efe88070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe740047c0 0x7efe74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880cbae0 con 0x7efe74003970 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880b2060 con 0x7efe740047c0 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7400b980 con 0x7efe880afbe0 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe86ffd640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.617+0000 7efe877fe640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe7ffff640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe740047c0 0x7efe74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe86ffd640 1 -- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 msgr2=0x7efe74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:09.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe86ffd640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe880afbe0 0x7efe74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe86ffd640 1 -- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe740047c0 msgr2=0x7efe74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:09.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe86ffd640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe740047c0 0x7efe74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe86ffd640 1 -- 172.21.15.53:0/2462534994 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74003520 con 0x7efe74003970 2023-10-21T16:21:09.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.618+0000 7efe7ffff640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe740047c0 0x7efe74004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:09.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.619+0000 7efe86ffd640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe74003d90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efe78009ac0 tx=0x7efe740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91083ff82520c5ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:09.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.619+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7802c6b0 con 0x7efe74003970 2023-10-21T16:21:09.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.619+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7802c810 con 0x7efe74003970 2023-10-21T16:21:09.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.619+0000 7efe867fc640 1 -- 172.21.15.53:0/2462534994 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7802cba0 con 0x7efe74003970 2023-10-21T16:21:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 -- 172.21.15.53:0/2462534994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 msgr2=0x7efe74003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 --2- 172.21.15.53:0/2462534994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe74003d90 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efe78009ac0 tx=0x7efe740057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe877fe640 1 -- 172.21.15.53:0/2462534994 reap_dead start 2023-10-21T16:21:09.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 -- 172.21.15.53:0/2462534994 shutdown_connections 2023-10-21T16:21:09.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 -- 172.21.15.53:0/2462534994 >> 172.21.15.53:0/2462534994 conn(0x7efe8806a5d0 msgr2=0x7efe880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:09.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 -- 172.21.15.53:0/2462534994 shutdown_connections 2023-10-21T16:21:09.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.620+0000 7efe8d990640 1 -- 172.21.15.53:0/2462534994 wait complete. 2023-10-21T16:21:09.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.622+0000 7efe8d990640 1 Processor -- start 2023-10-21T16:21:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.622+0000 7efe8d990640 1 -- start start 2023-10-21T16:21:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 0x7efe88159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 0x7efe88157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880c6860 con 0x7efe74003970 2023-10-21T16:21:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880c7680 con 0x7efe880c7b10 2023-10-21T16:21:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe8d990640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe88144ba0 con 0x7efe8814ff50 2023-10-21T16:21:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe86ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe86ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42882/0 (socket says 172.21.15.53:42882) 2023-10-21T16:21:09.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.623+0000 7efe86ffd640 1 -- 172.21.15.53:0/625128729 learned_addr learned my addr 172.21.15.53:0/625128729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe877fe640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 0x7efe88157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe7ffff640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 0x7efe88159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 -- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 msgr2=0x7efe88159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 0x7efe88159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 -- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 msgr2=0x7efe88157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 0x7efe88157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 -- 172.21.15.53:0/625128729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880c6a70 con 0x7efe74003970 2023-10-21T16:21:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe877fe640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 0x7efe88157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe7ffff640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 0x7efe88159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:09.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.624+0000 7efe86ffd640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efe7802c2b0 tx=0x7efe78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.625+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78003810 con 0x7efe74003970 2023-10-21T16:21:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.626+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78003970 con 0x7efe74003970 2023-10-21T16:21:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.626+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe880c6d50 con 0x7efe74003970 2023-10-21T16:21:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.626+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78003ce0 con 0x7efe74003970 2023-10-21T16:21:09.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.626+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe880c7310 con 0x7efe74003970 2023-10-21T16:21:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.627+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8804e160 con 0x7efe74003970 2023-10-21T16:21:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.628+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 62) v1 ==== 88940+0+0 (secure 0 0 0) 0x7efe78047450 con 0x7efe74003970 2023-10-21T16:21:09.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.629+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6847+0+0 (secure 0 0 0) 0x7efe780ab880 con 0x7efe74003970 2023-10-21T16:21:09.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.639+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe780768d0 con 0x7efe74003970 2023-10-21T16:21:09.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.646+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 63) v1 ==== 89067+0+0 (secure 0 0 0) 0x7efe780add00 con 0x7efe74003970 2023-10-21T16:21:09.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.646+0000 7efe84ff9640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 0x7efe68073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:09.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.648+0000 7efe7ffff640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 0x7efe68073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:09.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.652+0000 7efe7ffff640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 0x7efe68073040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe881f3690 tx=0x7efe70009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:09.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.829+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe880b0ef0 con 0x7efe74003970 2023-10-21T16:21:09.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.830+0000 7efe84ff9640 1 -- 172.21.15.53:0/625128729 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe7807db70 con 0x7efe74003970 2023-10-21T16:21:09.832 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:21:09.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 msgr2=0x7efe68073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 0x7efe68073040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe881f3690 tx=0x7efe70009040 comp rx=0 tx=0).stop 2023-10-21T16:21:09.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 msgr2=0x7efe88144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:09.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efe7802c2b0 tx=0x7efe78004730 comp rx=0 tx=0).stop 2023-10-21T16:21:09.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 shutdown_connections 2023-10-21T16:21:09.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7efe68070c50 0x7efe68073040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe880c7b10 0x7efe88157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe8814ff50 0x7efe88159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 --2- 172.21.15.53:0/625128729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe74003970 0x7efe88144660 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:09.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.833+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 >> 172.21.15.53:0/625128729 conn(0x7efe8806a5d0 msgr2=0x7efe8815d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:09.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.834+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 shutdown_connections 2023-10-21T16:21:09.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:09.834+0000 7efe8d990640 1 -- 172.21.15.53:0/625128729 wait complete. 2023-10-21T16:21:09.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:10.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.074+0000 7f01ffd05640 1 Processor -- start 2023-10-21T16:21:10.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.074+0000 7f01ffd05640 1 -- start start 2023-10-21T16:21:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01ffd05640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01ffd05640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f01f815e330 0x7f01f8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01ffd05640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f806dee0 con 0x7f01f8159ac0 2023-10-21T16:21:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01ffd05640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f806dcf0 con 0x7f01f815e330 2023-10-21T16:21:10.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01ffd05640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815d850 con 0x7f01f80ad790 2023-10-21T16:21:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01fd2a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.076+0000 7f01fd2a0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:43998/0 (socket says 172.21.15.53:43998) 2023-10-21T16:21:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2274504389 learned_addr learned my addr 172.21.15.53:0/2274504389 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:10.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fe2a2640 1 --2- 172.21.15.53:0/2274504389 >> v2:172.21.15.124:3300/0 conn(0x7f01f815e330 0x7f01f8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2274504389 >> v2:172.21.15.124:3300/0 conn(0x7f01f815e330 msgr2=0x7f01f8070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fd2a0640 1 --2- 172.21.15.53:0/2274504389 >> v2:172.21.15.124:3300/0 conn(0x7f01f815e330 0x7f01f8070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2274504389 >> v1:172.21.15.124:6789/0 conn(0x7f01f8159ac0 legacy=0x7f01f806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.077+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2274504389 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f815db30 con 0x7f01f80ad790 2023-10-21T16:21:10.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.078+0000 7f01fd2a0640 1 --2- 172.21.15.53:0/2274504389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009870 tx=0x7f01e802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681b843b2ca01d16 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.079+0000 7f01fca9f640 1 -- 172.21.15.53:0/2274504389 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8030840 con 0x7f01f80ad790 2023-10-21T16:21:10.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.079+0000 7f01fca9f640 1 -- 172.21.15.53:0/2274504389 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e80309a0 con 0x7f01f80ad790 2023-10-21T16:21:10.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.079+0000 7f01fca9f640 1 -- 172.21.15.53:0/2274504389 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8030d30 con 0x7f01f80ad790 2023-10-21T16:21:10.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 -- 172.21.15.53:0/2274504389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 msgr2=0x7f01f80702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2274504389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009870 tx=0x7f01e802b980 comp rx=0 tx=0).stop 2023-10-21T16:21:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 -- 172.21.15.53:0/2274504389 shutdown_connections 2023-10-21T16:21:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2274504389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f80702d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2274504389 >> v2:172.21.15.124:3300/0 conn(0x7f01f815e330 0x7f01f8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 -- 172.21.15.53:0/2274504389 >> 172.21.15.53:0/2274504389 conn(0x7f01f806a380 msgr2=0x7f01f80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 -- 172.21.15.53:0/2274504389 shutdown_connections 2023-10-21T16:21:10.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.080+0000 7f01ffd05640 1 -- 172.21.15.53:0/2274504389 wait complete. 2023-10-21T16:21:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.082+0000 7f01ffd05640 1 Processor -- start 2023-10-21T16:21:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.082+0000 7f01ffd05640 1 -- start start 2023-10-21T16:21:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 0x7f01f814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f814a180 con 0x7f01f8159ac0 2023-10-21T16:21:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f814afe0 con 0x7f01f815e330 2023-10-21T16:21:10.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01ffd05640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f8148ec0 con 0x7f01f80ad790 2023-10-21T16:21:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fd2a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fd2a0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42886/0 (socket says 172.21.15.53:42886) 2023-10-21T16:21:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2600779188 learned_addr learned my addr 172.21.15.53:0/2600779188 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:10.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fdaa1640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fe2a2640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 0x7f01f814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.083+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 msgr2=0x7f01f814e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fd2a0640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f814e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 msgr2=0x7f01f814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fd2a0640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 0x7f01f814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fd2a0640 1 -- 172.21.15.53:0/2600779188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80094d0 con 0x7f01f8159ac0 2023-10-21T16:21:10.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fdaa1640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f814e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:10.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fe2a2640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 0x7f01f814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01fd2a0640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8039c90 con 0x7f01f8159ac0 2023-10-21T16:21:10.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e8039df0 con 0x7f01f8159ac0 2023-10-21T16:21:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f8149160 con 0x7f01f8159ac0 2023-10-21T16:21:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.084+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8043640 con 0x7f01f8159ac0 2023-10-21T16:21:10.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.085+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f8149720 con 0x7f01f8159ac0 2023-10-21T16:21:10.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.086+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f80c5540 con 0x7f01f8159ac0 2023-10-21T16:21:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.087+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 63) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f01e80437a0 con 0x7f01f8159ac0 2023-10-21T16:21:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.087+0000 7f01ee7fc640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 0x7f01dc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.088+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f01e803e070 con 0x7f01f8159ac0 2023-10-21T16:21:10.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.088+0000 7f01fdaa1640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 0x7f01dc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.089+0000 7f01fdaa1640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 0x7f01dc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01f81f60d0 tx=0x7f01f4009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.093+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01e8077640 con 0x7f01f8159ac0 2023-10-21T16:21:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.336+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01f80b0c90 con 0x7f01f8159ac0 2023-10-21T16:21:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.338+0000 7f01ee7fc640 1 -- 172.21.15.53:0/2600779188 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f01e807ac60 con 0x7f01f8159ac0 2023-10-21T16:21:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:10.340 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:21:10.340 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:21:10.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.340+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 msgr2=0x7f01dc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.340+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 0x7f01dc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01f81f60d0 tx=0x7f01f4009040 comp rx=0 tx=0).stop 2023-10-21T16:21:10.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.340+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 msgr2=0x7f01f8149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.340+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e8004730 comp rx=0 tx=0).stop 2023-10-21T16:21:10.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.340+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 shutdown_connections 2023-10-21T16:21:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f01dc06acc0 0x7f01dc06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01f815e330 0x7f01f814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8159ac0 0x7f01f8149c40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 --2- 172.21.15.53:0/2600779188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01f80ad790 0x7f01f814e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 >> 172.21.15.53:0/2600779188 conn(0x7f01f806a380 msgr2=0x7f01f80cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 shutdown_connections 2023-10-21T16:21:10.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.341+0000 7f01ffd05640 1 -- 172.21.15.53:0/2600779188 wait complete. 2023-10-21T16:21:10.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:10.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.584+0000 7fd867fff640 1 Processor -- start 2023-10-21T16:21:10.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.584+0000 7fd867fff640 1 -- start start 2023-10-21T16:21:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 0x7fd868071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8680c97d0 0x7fd86815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 0x7fd868159850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd868071550 con 0x7fd8680c97d0 2023-10-21T16:21:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86815d7d0 con 0x7fd868150440 2023-10-21T16:21:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.586+0000 7fd867fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c9f90 con 0x7fd86815d380 2023-10-21T16:21:10.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8680c97d0 0x7fd86815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8680c97d0 0x7fd86815ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42892/0 (socket says 172.21.15.53:42892) 2023-10-21T16:21:10.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 -- 172.21.15.53:0/170368551 learned_addr learned my addr 172.21.15.53:0/170368551 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:10.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 -- 172.21.15.53:0/170368551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 msgr2=0x7fd868159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:10.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd866ffd640 1 --2- 172.21.15.53:0/170368551 >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 0x7fd868071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd8677fe640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 0x7fd868159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 0x7fd868159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 -- 172.21.15.53:0/170368551 >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 msgr2=0x7fd868071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 --2- 172.21.15.53:0/170368551 >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 0x7fd868071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd85ffff640 1 -- 172.21.15.53:0/170368551 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680ca270 con 0x7fd8680c97d0 2023-10-21T16:21:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd8677fe640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 0x7fd868159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.587+0000 7fd866ffd640 1 --2- 172.21.15.53:0/170368551 >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 0x7fd868071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:10.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.588+0000 7fd85ffff640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8680c97d0 0x7fd86815ce40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd8680cbe60 tx=0x7fd85002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6cd91ab691aab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.589+0000 7fd8667fc640 1 -- 172.21.15.53:0/170368551 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850031880 con 0x7fd8680c97d0 2023-10-21T16:21:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.589+0000 7fd8667fc640 1 -- 172.21.15.53:0/170368551 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500319e0 con 0x7fd8680c97d0 2023-10-21T16:21:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.589+0000 7fd8667fc640 1 -- 172.21.15.53:0/170368551 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850031d70 con 0x7fd8680c97d0 2023-10-21T16:21:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.589+0000 7fd867fff640 1 -- 172.21.15.53:0/170368551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8680c97d0 msgr2=0x7fd86815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.589+0000 7fd867fff640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8680c97d0 0x7fd86815ce40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd8680cbe60 tx=0x7fd85002b920 comp rx=0 tx=0).stop 2023-10-21T16:21:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 -- 172.21.15.53:0/170368551 shutdown_connections 2023-10-21T16:21:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd86815d380 0x7fd868159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 --2- 172.21.15.53:0/170368551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8680c97d0 0x7fd86815ce40 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 --2- 172.21.15.53:0/170368551 >> v2:172.21.15.124:3300/0 conn(0x7fd868150440 0x7fd868071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 -- 172.21.15.53:0/170368551 >> 172.21.15.53:0/170368551 conn(0x7fd8680cb0c0 msgr2=0x7fd8680cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:10.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 -- 172.21.15.53:0/170368551 shutdown_connections 2023-10-21T16:21:10.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.590+0000 7fd867fff640 1 -- 172.21.15.53:0/170368551 wait complete. 2023-10-21T16:21:10.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.592+0000 7fd867fff640 1 Processor -- start 2023-10-21T16:21:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.592+0000 7fd867fff640 1 -- start start 2023-10-21T16:21:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 0x7fd868158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd868150440 0x7fd86806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd868148340 con 0x7fd86815d380 2023-10-21T16:21:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b1fa0 con 0x7fd868150440 2023-10-21T16:21:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd867fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b2280 con 0x7fd8680c97d0 2023-10-21T16:21:10.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42906/0 (socket says 172.21.15.53:42906) 2023-10-21T16:21:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 -- 172.21.15.53:0/2918527580 learned_addr learned my addr 172.21.15.53:0/2918527580 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd866ffd640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 0x7fd868158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd85ffff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd868150440 0x7fd86806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 -- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 msgr2=0x7fd868158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 0x7fd868158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 -- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd868150440 msgr2=0x7fd86806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd868150440 0x7fd86806d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.593+0000 7fd8677fe640 1 -- 172.21.15.53:0/2918527580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500094d0 con 0x7fd86815d380 2023-10-21T16:21:10.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd866ffd640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 0x7fd868158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:10.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd8677fe640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd86000cef0 tx=0x7fd86000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000d840 con 0x7fd86815d380 2023-10-21T16:21:10.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000d9c0 con 0x7fd86815d380 2023-10-21T16:21:10.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000dd70 con 0x7fd86815d380 2023-10-21T16:21:10.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8680b2560 con 0x7fd86815d380 2023-10-21T16:21:10.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.594+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8680b2b20 con 0x7fd86815d380 2023-10-21T16:21:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.595+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680b2d10 con 0x7fd86815d380 2023-10-21T16:21:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.596+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 63) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd86001e3b0 con 0x7fd86815d380 2023-10-21T16:21:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.597+0000 7fd85f7fe640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 0x7fd84806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.597+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7fd86008a290 con 0x7fd86815d380 2023-10-21T16:21:10.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.598+0000 7fd866ffd640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 0x7fd84806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:10.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.599+0000 7fd866ffd640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 0x7fd84806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8580088b0 tx=0x7fd858008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:10.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.602+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8600551c0 con 0x7fd86815d380 2023-10-21T16:21:10.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.660+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 64) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd860055590 con 0x7fd86815d380 2023-10-21T16:21:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.807+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd86804e160 con 0x7fd86815d380 2023-10-21T16:21:10.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.810+0000 7fd85f7fe640 1 -- 172.21.15.53:0/2918527580 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fd860076400 con 0x7fd86815d380 2023-10-21T16:21:10.812 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:10.812 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":136,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:09.625558+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1244050409":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1104588229":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/45913668":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3828835087":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/897501407":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2837232794":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/4124902964":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3242447571":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1097749659":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2545128602":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1424561960":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1535068499":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3746315143":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3443628834":"2023-10-22T16:20:39.664146+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 msgr2=0x7fd84806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 0x7fd84806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8580088b0 tx=0x7fd858008040 comp rx=0 tx=0).stop 2023-10-21T16:21:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 msgr2=0x7fd86814bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd86000cef0 tx=0x7fd86000a710 comp rx=0 tx=0).stop 2023-10-21T16:21:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 shutdown_connections 2023-10-21T16:21:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd84806acc0 0x7fd84806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd86815d380 0x7fd86814bae0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd868150440 0x7fd86806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 --2- 172.21.15.53:0/2918527580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8680c97d0 0x7fd868158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 >> 172.21.15.53:0/2918527580 conn(0x7fd8680cb0c0 msgr2=0x7fd86814fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 shutdown_connections 2023-10-21T16:21:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:10.813+0000 7fd867fff640 1 -- 172.21.15.53:0/2918527580 wait complete. 2023-10-21T16:21:10.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:21:10.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:11.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.058+0000 7f697720e640 1 Processor -- start 2023-10-21T16:21:11.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.058+0000 7f697720e640 1 -- start start 2023-10-21T16:21:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.060+0000 7f697720e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69700c90b0 0x7f69700cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.060+0000 7f697720e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 0x7f697015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.060+0000 7f697720e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697015b2b0 con 0x7f697015ab40 2023-10-21T16:21:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.060+0000 7f697720e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697015bf30 con 0x7f69700c90b0 2023-10-21T16:21:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.060+0000 7f697720e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700cb820 con 0x7f69700cb3b0 2023-10-21T16:21:11.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69700c90b0 0x7f69700cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69700c90b0 0x7f69700cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42912/0 (socket says 172.21.15.53:42912) 2023-10-21T16:21:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 -- 172.21.15.53:0/1791053083 learned_addr learned my addr 172.21.15.53:0/1791053083 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f69757ab640 1 --2- 172.21.15.53:0/1791053083 >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 0x7f697015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 -- 172.21.15.53:0/1791053083 >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 msgr2=0x7f697015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:11.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 --2- 172.21.15.53:0/1791053083 >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 0x7f697015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 -- 172.21.15.53:0/1791053083 >> v1:172.21.15.53:6789/0 conn(0x7f697015ab40 legacy=0x7f69700ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f696ffff640 1 -- 172.21.15.53:0/1791053083 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697015b590 con 0x7f69700c90b0 2023-10-21T16:21:11.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.061+0000 7f69757ab640 1 --2- 172.21.15.53:0/1791053083 >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 0x7f697015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.063+0000 7f696ffff640 1 --2- 172.21.15.53:0/1791053083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700c90b0 0x7f69700cae70 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f69700b0ec0 tx=0x7f695c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ed97789f34325e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.063+0000 7f696f7fe640 1 -- 172.21.15.53:0/1791053083 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c030870 con 0x7f69700c90b0 2023-10-21T16:21:11.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.063+0000 7f696f7fe640 1 -- 172.21.15.53:0/1791053083 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c0309d0 con 0x7f69700c90b0 2023-10-21T16:21:11.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.063+0000 7f696f7fe640 1 -- 172.21.15.53:0/1791053083 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c030d60 con 0x7f69700c90b0 2023-10-21T16:21:11.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 -- 172.21.15.53:0/1791053083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700c90b0 msgr2=0x7f69700cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 --2- 172.21.15.53:0/1791053083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700c90b0 0x7f69700cae70 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f69700b0ec0 tx=0x7f695c02b9b0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 -- 172.21.15.53:0/1791053083 shutdown_connections 2023-10-21T16:21:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 --2- 172.21.15.53:0/1791053083 >> v2:172.21.15.124:3300/0 conn(0x7f69700cb3b0 0x7f697015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 --2- 172.21.15.53:0/1791053083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700c90b0 0x7f69700cae70 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 -- 172.21.15.53:0/1791053083 >> 172.21.15.53:0/1791053083 conn(0x7f697006a380 msgr2=0x7f69700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 -- 172.21.15.53:0/1791053083 shutdown_connections 2023-10-21T16:21:11.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.064+0000 7f697720e640 1 -- 172.21.15.53:0/1791053083 wait complete. 2023-10-21T16:21:11.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.066+0000 7f697720e640 1 Processor -- start 2023-10-21T16:21:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.066+0000 7f697720e640 1 -- start start 2023-10-21T16:21:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 0x7f697014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015ab40 0x7f697015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697014f980 con 0x7f69700cb3b0 2023-10-21T16:21:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697015c940 con 0x7f69700c90b0 2023-10-21T16:21:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f697720e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697015cc20 con 0x7f697015ab40 2023-10-21T16:21:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f696ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f696ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42926/0 (socket says 172.21.15.53:42926) 2023-10-21T16:21:11.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.067+0000 7f696ffff640 1 -- 172.21.15.53:0/724586646 learned_addr learned my addr 172.21.15.53:0/724586646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f6974faa640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 0x7f697014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f69757ab640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015ab40 0x7f697015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 -- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015ab40 msgr2=0x7f697015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015ab40 0x7f697015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 -- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 msgr2=0x7f697014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 0x7f697014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 -- 172.21.15.53:0/724586646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c0094d0 con 0x7f69700cb3b0 2023-10-21T16:21:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f6974faa640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 0x7f697014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696ffff640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f695c02b9f0 tx=0x7f695c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c039d60 con 0x7f69700cb3b0 2023-10-21T16:21:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695c043400 con 0x7f69700cb3b0 2023-10-21T16:21:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c0437b0 con 0x7f69700cb3b0 2023-10-21T16:21:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.068+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697015cf00 con 0x7f69700cb3b0 2023-10-21T16:21:11.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.069+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697015d4c0 con 0x7f69700cb3b0 2023-10-21T16:21:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.070+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6970067490 con 0x7f69700cb3b0 2023-10-21T16:21:11.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.071+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 64) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f695c043910 con 0x7f69700cb3b0 2023-10-21T16:21:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.072+0000 7f696d7fa640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 0x7f695406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.072+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f695c03e070 con 0x7f69700cb3b0 2023-10-21T16:21:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.072+0000 7f6974faa640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 0x7f695406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.074+0000 7f6974faa640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 0x7f695406d410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6964000f60 tx=0x7f6964009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.076+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f695c0778d0 con 0x7f69700cb3b0 2023-10-21T16:21:11.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.307+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69700b0ef0 con 0x7f69700cb3b0 2023-10-21T16:21:11.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.308+0000 7f696d7fa640 1 -- 172.21.15.53:0/724586646 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f695c034070 con 0x7f69700cb3b0 2023-10-21T16:21:11.310 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:11.311 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:21:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:21:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 msgr2=0x7f695406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 0x7f695406d410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6964000f60 tx=0x7f6964009040 comp rx=0 tx=0).stop 2023-10-21T16:21:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 msgr2=0x7f697015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f695c02b9f0 tx=0x7f695c004730 comp rx=0 tx=0).stop 2023-10-21T16:21:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 shutdown_connections 2023-10-21T16:21:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f695406af50 0x7f695406d410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015ab40 0x7f697015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700cb3b0 0x7f697015d9e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 --2- 172.21.15.53:0/724586646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69700c90b0 0x7f697014f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 >> 172.21.15.53:0/724586646 conn(0x7f697006a380 msgr2=0x7f6970156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:11.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 shutdown_connections 2023-10-21T16:21:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.311+0000 7f697720e640 1 -- 172.21.15.53:0/724586646 wait complete. 2023-10-21T16:21:11.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:11.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.555+0000 7f5c97fff640 1 Processor -- start 2023-10-21T16:21:11.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.556+0000 7f5c97fff640 1 -- start start 2023-10-21T16:21:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.557+0000 7f5c97fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c980b07d0 0x7f5c98165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.557+0000 7f5c97fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9815f7d0 con 0x7f5c9815ab30 2023-10-21T16:21:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.557+0000 7f5c97fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9815efb0 con 0x7f5c980adc70 2023-10-21T16:21:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.557+0000 7f5c97fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9815f210 con 0x7f5c980b07d0 2023-10-21T16:21:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c980b07d0 0x7f5c98165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c96ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5c9815ab30 0x7f5c9815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37926/0 (socket says 172.21.15.53:37926) 2023-10-21T16:21:11.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c96ffd640 1 -- 172.21.15.53:0/2189809690 learned_addr learned my addr 172.21.15.53:0/2189809690 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c977fe640 1 -- 172.21.15.53:0/2189809690 >> v1:172.21.15.124:6789/0 conn(0x7f5c980adc70 legacy=0x7f5c980b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c977fe640 1 -- 172.21.15.53:0/2189809690 >> v1:172.21.15.53:6789/0 conn(0x7f5c9815ab30 legacy=0x7f5c9815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c977fe640 1 -- 172.21.15.53:0/2189809690 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9815f4f0 con 0x7f5c980b07d0 2023-10-21T16:21:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.558+0000 7f5c95ffb640 1 -- 172.21.15.53:0/2189809690 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267538270 0 0) 0x7f5c9815f7d0 con 0x7f5c9815ab30 2023-10-21T16:21:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c977fe640 1 --2- 172.21.15.53:0/2189809690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980b07d0 0x7f5c98165bd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097a0 tx=0x7f5c9002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d4a5f72c660673 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c95ffb640 1 -- 172.21.15.53:0/2189809690 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90030610 con 0x7f5c980b07d0 2023-10-21T16:21:11.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c95ffb640 1 -- 172.21.15.53:0/2189809690 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90030770 con 0x7f5c980b07d0 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c95ffb640 1 -- 172.21.15.53:0/2189809690 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90030b00 con 0x7f5c980b07d0 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c97fff640 1 -- 172.21.15.53:0/2189809690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980b07d0 msgr2=0x7f5c98165bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.560+0000 7f5c97fff640 1 --2- 172.21.15.53:0/2189809690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980b07d0 0x7f5c98165bd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097a0 tx=0x7f5c9002b960 comp rx=0 tx=0).stop 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.561+0000 7f5c97fff640 1 -- 172.21.15.53:0/2189809690 shutdown_connections 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.561+0000 7f5c97fff640 1 --2- 172.21.15.53:0/2189809690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980b07d0 0x7f5c98165bd0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.561+0000 7f5c97fff640 1 -- 172.21.15.53:0/2189809690 >> 172.21.15.53:0/2189809690 conn(0x7f5c980cb0c0 msgr2=0x7f5c980cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.561+0000 7f5c97fff640 1 -- 172.21.15.53:0/2189809690 shutdown_connections 2023-10-21T16:21:11.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.561+0000 7f5c97fff640 1 -- 172.21.15.53:0/2189809690 wait complete. 2023-10-21T16:21:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.562+0000 7f5c97fff640 1 Processor -- start 2023-10-21T16:21:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.563+0000 7f5c97fff640 1 -- start start 2023-10-21T16:21:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.563+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.563+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c980b07d0 0x7f5c98151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c97fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c9815ab30 0x7f5c98152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c97fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980c6590 con 0x7f5c980adc70 2023-10-21T16:21:11.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c97fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9815d930 con 0x7f5c980b07d0 2023-10-21T16:21:11.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c97fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9815cb70 con 0x7f5c9815ab30 2023-10-21T16:21:11.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42936/0 (socket says 172.21.15.53:42936) 2023-10-21T16:21:11.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 -- 172.21.15.53:0/1674567640 learned_addr learned my addr 172.21.15.53:0/1674567640 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:11.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 -- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c9815ab30 msgr2=0x7f5c98152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c9815ab30 0x7f5c98152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 -- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c980b07d0 msgr2=0x7f5c98151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c980b07d0 0x7f5c98151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.564+0000 7f5c96ffd640 1 -- 172.21.15.53:0/1674567640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c90009400 con 0x7f5c980adc70 2023-10-21T16:21:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c96ffd640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c780 tx=0x7f5c8800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800ce90 con 0x7f5c980adc70 2023-10-21T16:21:11.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8800a640 con 0x7f5c980adc70 2023-10-21T16:21:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800aa70 con 0x7f5c980adc70 2023-10-21T16:21:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9815ce50 con 0x7f5c980adc70 2023-10-21T16:21:11.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.565+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9815d410 con 0x7f5c980adc70 2023-10-21T16:21:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.566+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9815d600 con 0x7f5c980adc70 2023-10-21T16:21:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.567+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 64) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5c8800abd0 con 0x7f5c980adc70 2023-10-21T16:21:11.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.568+0000 7f5c7ffff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 0x7f5c8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.568+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f5c88089130 con 0x7f5c980adc70 2023-10-21T16:21:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.569+0000 7f5c967fc640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 0x7f5c8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:11.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.570+0000 7f5c967fc640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 0x7f5c8406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5c9806db10 tx=0x7f5c8c007950 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:11.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.572+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c88053cd0 con 0x7f5c980adc70 2023-10-21T16:21:11.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.811+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c9804e160 con 0x7f5c980adc70 2023-10-21T16:21:11.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.812+0000 7f5c7ffff640 1 -- 172.21.15.53:0/1674567640 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f5c88018020 con 0x7f5c980adc70 2023-10-21T16:21:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:11.815 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:20:24.920100+0000","modified":"2023-10-21T16:20:28.035596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8085},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8085":{"gid":8085,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/206156413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":206156413},{"type":"v1","addr":"172.21.15.53:6835","nonce":206156413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-21T16:21:11.815 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 61 2023-10-21T16:21:11.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 msgr2=0x7f5c8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 0x7f5c8406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5c9806db10 tx=0x7f5c8c007950 comp rx=0 tx=0).stop 2023-10-21T16:21:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 msgr2=0x7f5c9806cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c780 tx=0x7f5c8800cc40 comp rx=0 tx=0).stop 2023-10-21T16:21:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 shutdown_connections 2023-10-21T16:21:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5c8406af00 0x7f5c8406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c9815ab30 0x7f5c98152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.815+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c980b07d0 0x7f5c98151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.816+0000 7f5c97fff640 1 --2- 172.21.15.53:0/1674567640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c980adc70 0x7f5c9806cec0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.816+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 >> 172.21.15.53:0/1674567640 conn(0x7f5c980cb0c0 msgr2=0x7f5c980cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:11.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.816+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 shutdown_connections 2023-10-21T16:21:11.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:11.816+0000 7f5c97fff640 1 -- 172.21.15.53:0/1674567640 wait complete. 2023-10-21T16:21:11.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:12.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.065+0000 7f1618c13640 1 Processor -- start 2023-10-21T16:21:12.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.066+0000 7f1618c13640 1 -- start start 2023-10-21T16:21:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f1618c13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f161406b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f1618c13640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f161406bd90 0x7f16140cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f1618c13640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140cb590 con 0x7f16140cd8a0 2023-10-21T16:21:12.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f1618c13640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161406c220 con 0x7f161406bd90 2023-10-21T16:21:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f1618c13640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16140cfc90 con 0x7f16140cf720 2023-10-21T16:21:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f16137fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f16140cd8a0 0x7f161406b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37930/0 (socket says 172.21.15.53:37930) 2023-10-21T16:21:12.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.067+0000 7f16137fe640 1 -- 172.21.15.53:0/2676946444 learned_addr learned my addr 172.21.15.53:0/2676946444 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> v2:172.21.15.124:3300/0 conn(0x7f161406bd90 0x7f16140cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f160bfff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f161406b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263702647 0 0) 0x7f16140cb590 con 0x7f16140cd8a0 2023-10-21T16:21:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1600003520 con 0x7f16140cd8a0 2023-10-21T16:21:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647306005 0 0) 0x7f1600003520 con 0x7f16140cd8a0 2023-10-21T16:21:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 msgr2=0x7f161406b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f161406b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 >> v2:172.21.15.124:3300/0 conn(0x7f161406bd90 msgr2=0x7f16140cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 --2- 172.21.15.53:0/2676946444 >> v2:172.21.15.124:3300/0 conn(0x7f161406bd90 0x7f16140cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.068+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140cff70 con 0x7f16140cd8a0 2023-10-21T16:21:12.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1604001670 con 0x7f16140cd8a0 2023-10-21T16:21:12.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> v2:172.21.15.124:3300/0 conn(0x7f161406bd90 0x7f16140cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:12.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 >> v1:172.21.15.53:6789/0 conn(0x7f16140cd8a0 legacy=0x7f161406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f1600003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600003f40 0x7f16000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1600004d00 0x7f1600005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161406c220 con 0x7f1600004d00 2023-10-21T16:21:12.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16140cfc90 con 0x7f1600003f40 2023-10-21T16:21:12.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160000ba30 con 0x7f16140cf720 2023-10-21T16:21:12.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1600004d00 0x7f1600005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f16137fe640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f1600003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f160bfff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600003f40 0x7f16000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 -- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 msgr2=0x7f1600003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16140cf720 0x7f1600003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 -- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600003f40 msgr2=0x7f16000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600003f40 0x7f16000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.069+0000 7f1613fff640 1 -- 172.21.15.53:0/2676946444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600003520 con 0x7f1600004d00 2023-10-21T16:21:12.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.070+0000 7f160bfff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600003f40 0x7f16000047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:12.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1613fff640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1600004d00 0x7f1600005180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f160c0073e0 tx=0x7f16000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8cc60360065b9b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c02cdc0 con 0x7f1600004d00 2023-10-21T16:21:12.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c034440 con 0x7f1600004d00 2023-10-21T16:21:12.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1612ffd640 1 -- 172.21.15.53:0/2676946444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0347b0 con 0x7f1600004d00 2023-10-21T16:21:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1618c13640 1 -- 172.21.15.53:0/2676946444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1600004d00 msgr2=0x7f1600005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1618c13640 1 --2- 172.21.15.53:0/2676946444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1600004d00 0x7f1600005180 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f160c0073e0 tx=0x7f16000057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.071+0000 7f1613fff640 1 -- 172.21.15.53:0/2676946444 reap_dead start 2023-10-21T16:21:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.072+0000 7f1618c13640 1 -- 172.21.15.53:0/2676946444 shutdown_connections 2023-10-21T16:21:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.072+0000 7f1618c13640 1 -- 172.21.15.53:0/2676946444 >> 172.21.15.53:0/2676946444 conn(0x7f1614099b60 msgr2=0x7f16140d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.072+0000 7f1618c13640 1 -- 172.21.15.53:0/2676946444 shutdown_connections 2023-10-21T16:21:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.072+0000 7f1618c13640 1 -- 172.21.15.53:0/2676946444 wait complete. 2023-10-21T16:21:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.073+0000 7f1618c13640 1 Processor -- start 2023-10-21T16:21:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.074+0000 7f1618c13640 1 -- start start 2023-10-21T16:21:12.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600004d00 0x7f161409cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161415e4a0 0x7f161409b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614070120 con 0x7f161406f760 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161409e5c0 con 0x7f1600004d00 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1618c13640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161409d520 con 0x7f161415e4a0 2023-10-21T16:21:12.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42952/0 (socket says 172.21.15.53:42952) 2023-10-21T16:21:12.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 -- 172.21.15.53:0/3886452656 learned_addr learned my addr 172.21.15.53:0/3886452656 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:12.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f160bfff640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161415e4a0 0x7f161409b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 -- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161415e4a0 msgr2=0x7f161409b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f16137fe640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600004d00 0x7f161409cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161415e4a0 0x7f161409b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 -- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600004d00 msgr2=0x7f161409cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600004d00 0x7f161409cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.075+0000 7f1613fff640 1 -- 172.21.15.53:0/3886452656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161409d800 con 0x7f161406f760 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1613fff640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f160c037040 tx=0x7f160c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c043070 con 0x7f161406f760 2023-10-21T16:21:12.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c034cc0 con 0x7f161406f760 2023-10-21T16:21:12.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161409dae0 con 0x7f161406f760 2023-10-21T16:21:12.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c03e680 con 0x7f161406f760 2023-10-21T16:21:12.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.076+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161409e0a0 con 0x7f161406f760 2023-10-21T16:21:12.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.077+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161404e160 con 0x7f161406f760 2023-10-21T16:21:12.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.078+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 64) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f160c03e7e0 con 0x7f161406f760 2023-10-21T16:21:12.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.080+0000 7f1610ff9640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 0x7f15f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.080+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f160c0ab7f0 con 0x7f161406f760 2023-10-21T16:21:12.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.080+0000 7f16137fe640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 0x7f15f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.082+0000 7f16137fe640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 0x7f15f406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16040078f0 tx=0x7f1604009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.085+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f160c076300 con 0x7f161406f760 2023-10-21T16:21:12.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.278+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16140cded0 con 0x7f161406f760 2023-10-21T16:21:12.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.280+0000 7f1610ff9640 1 -- 172.21.15.53:0/3886452656 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f160c079920 con 0x7f161406f760 2023-10-21T16:21:12.282 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:12.282 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":136,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:09.625558+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:20:22.870339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-21T16:20:23.949173+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":136,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1244050409":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1104588229":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/45913668":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3828835087":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/897501407":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2837232794":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/4124902964":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3242447571":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1097749659":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2545128602":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1424561960":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1535068499":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3746315143":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3443628834":"2023-10-22T16:20:39.664146+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 msgr2=0x7f15f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 0x7f15f406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f16040078f0 tx=0x7f1604009040 comp rx=0 tx=0).stop 2023-10-21T16:21:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 msgr2=0x7f161406fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f160c037040 tx=0x7f160c004ab0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 shutdown_connections 2023-10-21T16:21:12.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f15f406af00 0x7f15f406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f161406f760 0x7f161406fbe0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f161415e4a0 0x7f161409b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 --2- 172.21.15.53:0/3886452656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1600004d00 0x7f161409cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 >> 172.21.15.53:0/3886452656 conn(0x7f1614099b60 msgr2=0x7f16140b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 shutdown_connections 2023-10-21T16:21:12.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.283+0000 7f1618c13640 1 -- 172.21.15.53:0/3886452656 wait complete. 2023-10-21T16:21:12.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:21:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.534+0000 7f27e1dce640 1 Processor -- start 2023-10-21T16:21:12.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.534+0000 7f27e1dce640 1 -- start start 2023-10-21T16:21:12.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.536+0000 7f27e1dce640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f27dc0cb600 0x7f27dc0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.536+0000 7f27e1dce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc06e1e0 con 0x7f27dc0cb150 2023-10-21T16:21:12.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.536+0000 7f27e1dce640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc0cba50 con 0x7f27dc161690 2023-10-21T16:21:12.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.536+0000 7f27e1dce640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc06cc50 con 0x7f27dc0cb600 2023-10-21T16:21:12.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27d3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f27dc0cb150 0x7f27dc0b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37934/0 (socket says 172.21.15.53:37934) 2023-10-21T16:21:12.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27d3fff640 1 -- 172.21.15.53:0/3623646954 learned_addr learned my addr 172.21.15.53:0/3623646954 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:12.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27dbfff640 1 --2- 172.21.15.53:0/3623646954 >> v2:172.21.15.124:3300/0 conn(0x7f27dc0cb600 0x7f27dc0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530635639 0 0) 0x7f27dc06e1e0 con 0x7f27dc0cb150 2023-10-21T16:21:12.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c8003520 con 0x7f27dc0cb150 2023-10-21T16:21:12.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062549128 0 0) 0x7f27c8003520 con 0x7f27dc0cb150 2023-10-21T16:21:12.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 >> v2:172.21.15.124:3300/0 conn(0x7f27dc0cb600 msgr2=0x7f27dc0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 --2- 172.21.15.53:0/3623646954 >> v2:172.21.15.124:3300/0 conn(0x7f27dc0cb600 0x7f27dc0b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 >> v1:172.21.15.124:6789/0 conn(0x7f27dc161690 legacy=0x7f27dc06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc06cf30 con 0x7f27dc0cb150 2023-10-21T16:21:12.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.537+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f27c4001690 con 0x7f27dc0cb150 2023-10-21T16:21:12.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 >> v1:172.21.15.53:6789/0 conn(0x7f27dc0cb150 legacy=0x7f27dc0b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27c80039c0 0x7f27c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27c8005180 0x7f27c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc06cc50 con 0x7f27c80047c0 2023-10-21T16:21:12.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c800c080 con 0x7f27c8005180 2023-10-21T16:21:12.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c800c240 con 0x7f27c80039c0 2023-10-21T16:21:12.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27db7fe640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27c80039c0 0x7f27c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27dbfff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27c8005180 0x7f27c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 -- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27c80039c0 msgr2=0x7f27c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27c80039c0 0x7f27c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 -- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27c8005180 msgr2=0x7f27c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:12.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27c8005180 0x7f27c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.538+0000 7f27d3fff640 1 -- 172.21.15.53:0/3623646954 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8003520 con 0x7f27c80047c0 2023-10-21T16:21:12.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.539+0000 7f27dbfff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27c8005180 0x7f27c800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:12.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.539+0000 7f27dbfff640 1 -- 172.21.15.53:0/3623646954 reap_dead start 2023-10-21T16:21:12.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27d3fff640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27c8004c40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c160 tx=0x7f27c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1296961e7210b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4005ad0 con 0x7f27c80047c0 2023-10-21T16:21:12.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4005c50 con 0x7f27c80047c0 2023-10-21T16:21:12.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27daffd640 1 -- 172.21.15.53:0/3623646954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c402c7e0 con 0x7f27c80047c0 2023-10-21T16:21:12.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27e1dce640 1 -- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 msgr2=0x7f27c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.540+0000 7f27e1dce640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27c8004c40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f27c400c160 tx=0x7f27c80057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.541+0000 7f27e1dce640 1 -- 172.21.15.53:0/3623646954 shutdown_connections 2023-10-21T16:21:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.541+0000 7f27e1dce640 1 --2- 172.21.15.53:0/3623646954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27c8004c40 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.541+0000 7f27e1dce640 1 -- 172.21.15.53:0/3623646954 >> 172.21.15.53:0/3623646954 conn(0x7f27dc06a5d0 msgr2=0x7f27dc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.541+0000 7f27e1dce640 1 -- 172.21.15.53:0/3623646954 shutdown_connections 2023-10-21T16:21:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.541+0000 7f27e1dce640 1 -- 172.21.15.53:0/3623646954 wait complete. 2023-10-21T16:21:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.543+0000 7f27e1dce640 1 Processor -- start 2023-10-21T16:21:12.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.543+0000 7f27e1dce640 1 -- start start 2023-10-21T16:21:12.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27dc0c2290 0x7f27dc0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 0x7f27dc0befd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc1601f0 con 0x7f27c80047c0 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0c2800 con 0x7f27dc0beb50 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27e1dce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0c2ae0 con 0x7f27dc0c2290 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42974/0 (socket says 172.21.15.53:42974) 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.544+0000 7f27db7fe640 1 -- 172.21.15.53:0/1903877835 learned_addr learned my addr 172.21.15.53:0/1903877835 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:12.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27dbfff640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 0x7f27dc0befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 -- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27dc0c2290 msgr2=0x7f27dc0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27dc0c2290 0x7f27dc0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 -- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 msgr2=0x7f27dc0befd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 0x7f27dc0befd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 -- 172.21.15.53:0/1903877835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0c2dc0 con 0x7f27c80047c0 2023-10-21T16:21:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27dbfff640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 0x7f27dc0befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27db7fe640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e410 tx=0x7f27cc00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0052e0 con 0x7f27c80047c0 2023-10-21T16:21:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc005440 con 0x7f27c80047c0 2023-10-21T16:21:12.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc003810 con 0x7f27c80047c0 2023-10-21T16:21:12.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc0c30a0 con 0x7f27c80047c0 2023-10-21T16:21:12.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.545+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc0c3de0 con 0x7f27c80047c0 2023-10-21T16:21:12.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.547+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc04e160 con 0x7f27c80047c0 2023-10-21T16:21:12.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.547+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 64) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f27cc003970 con 0x7f27c80047c0 2023-10-21T16:21:12.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.548+0000 7f27d8ff9640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 0x7f27bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:12.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.548+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f27cc0893a0 con 0x7f27c80047c0 2023-10-21T16:21:12.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.549+0000 7f27d3fff640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 0x7f27bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:12.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.550+0000 7f27d3fff640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 0x7f27bc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27dc06e370 tx=0x7f27c4039040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:12.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.553+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27cc053f60 con 0x7f27c80047c0 2023-10-21T16:21:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.688+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f27cc054330 con 0x7f27c80047c0 2023-10-21T16:21:12.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.794+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f27dc0b4230 con 0x7f27c80047c0 2023-10-21T16:21:12.797 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:21:12.795+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:21:12.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.799+0000 7f27d8ff9640 1 -- 172.21.15.53:0/1903877835 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f27cc019020 con 0x7f27c80047c0 2023-10-21T16:21:12.800 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:21:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 msgr2=0x7f27bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 0x7f27bc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27dc06e370 tx=0x7f27c4039040 comp rx=0 tx=0).stop 2023-10-21T16:21:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 msgr2=0x7f27dc0c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:12.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e410 tx=0x7f27cc00e8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 shutdown_connections 2023-10-21T16:21:12.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f27bc06af00 0x7f27bc06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27dc0beb50 0x7f27dc0befd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27dc0c2290 0x7f27dc0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 --2- 172.21.15.53:0/1903877835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27c80047c0 0x7f27dc0c38a0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:12.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 >> 172.21.15.53:0/1903877835 conn(0x7f27dc06a5d0 msgr2=0x7f27dc0b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:12.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 shutdown_connections 2023-10-21T16:21:12.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:12.802+0000 7f27e1dce640 1 -- 172.21.15.53:0/1903877835 wait complete. 2023-10-21T16:21:12.822 INFO:tasks.ceph.mds.a.smithi053.stderr: -7297> 2023-10-21T16:20:26.425+0000 7f127a69c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:21:12.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:21:12.902 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:21:13.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.020+0000 7f1a6614e640 1 Processor -- start 2023-10-21T16:21:13.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.021+0000 7f1a6614e640 1 -- start start 2023-10-21T16:21:13.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a6006dfc0 0x7f1a60070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 0x7f1a600cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600b2620 con 0x7f1a6006dfc0 2023-10-21T16:21:13.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600b2060 con 0x7f1a60071070 2023-10-21T16:21:13.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.022+0000 7f1a6614e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600b2340 con 0x7f1a600afbe0 2023-10-21T16:21:13.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a6006dfc0 0x7f1a60070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a6006dfc0 0x7f1a60070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42996/0 (socket says 172.21.15.53:42996) 2023-10-21T16:21:13.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 -- 172.21.15.53:0/1622639923 learned_addr learned my addr 172.21.15.53:0/1622639923 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:13.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a5ffff640 1 --2- 172.21.15.53:0/1622639923 >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 0x7f1a600cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a5f7fe640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 -- 172.21.15.53:0/1622639923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 msgr2=0x7f1a600b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:13.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 -- 172.21.15.53:0/1622639923 >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 msgr2=0x7f1a600cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:13.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 --2- 172.21.15.53:0/1622639923 >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 0x7f1a600cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a57fff640 1 -- 172.21.15.53:0/1622639923 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60070350 con 0x7f1a6006dfc0 2023-10-21T16:21:13.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.023+0000 7f1a5ffff640 1 --2- 172.21.15.53:0/1622639923 >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 0x7f1a600cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:13.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a57fff640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a6006dfc0 0x7f1a60070b30 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f1a600b3980 tx=0x7f1a4802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d303870ccd2419b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:13.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a5effd640 1 -- 172.21.15.53:0/1622639923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48031860 con 0x7f1a6006dfc0 2023-10-21T16:21:13.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a5effd640 1 -- 172.21.15.53:0/1622639923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480319c0 con 0x7f1a6006dfc0 2023-10-21T16:21:13.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a5effd640 1 -- 172.21.15.53:0/1622639923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48031d50 con 0x7f1a6006dfc0 2023-10-21T16:21:13.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a6614e640 1 -- 172.21.15.53:0/1622639923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a6006dfc0 msgr2=0x7f1a60070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:13.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.025+0000 7f1a6614e640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a6006dfc0 0x7f1a60070b30 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f1a600b3980 tx=0x7f1a4802b900 comp rx=0 tx=0).stop 2023-10-21T16:21:13.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 -- 172.21.15.53:0/1622639923 shutdown_connections 2023-10-21T16:21:13.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 --2- 172.21.15.53:0/1622639923 >> v2:172.21.15.124:3300/0 conn(0x7f1a60071070 0x7f1a600cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a6006dfc0 0x7f1a60070b30 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 --2- 172.21.15.53:0/1622639923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 -- 172.21.15.53:0/1622639923 >> 172.21.15.53:0/1622639923 conn(0x7f1a6006a5d0 msgr2=0x7f1a600b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:13.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 -- 172.21.15.53:0/1622639923 shutdown_connections 2023-10-21T16:21:13.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.026+0000 7f1a6614e640 1 -- 172.21.15.53:0/1622639923 wait complete. 2023-10-21T16:21:13.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.028+0000 7f1a6614e640 1 Processor -- start 2023-10-21T16:21:13.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.028+0000 7f1a6614e640 1 -- start start 2023-10-21T16:21:13.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6006dfc0 0x7f1a60144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60154fc0 con 0x7f1a60071070 2023-10-21T16:21:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a601543b0 con 0x7f1a6006dfc0 2023-10-21T16:21:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a6614e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60154690 con 0x7f1a600afbe0 2023-10-21T16:21:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43010/0 (socket says 172.21.15.53:43010) 2023-10-21T16:21:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a57fff640 1 -- 172.21.15.53:0/2773060623 learned_addr learned my addr 172.21.15.53:0/2773060623 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a5f7fe640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6006dfc0 0x7f1a60144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a5ffff640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.029+0000 7f1a57fff640 1 -- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 msgr2=0x7f1a600cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:13.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a57fff640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a57fff640 1 -- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6006dfc0 msgr2=0x7f1a60144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:13.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a57fff640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6006dfc0 0x7f1a60144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a57fff640 1 -- 172.21.15.53:0/2773060623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480094d0 con 0x7f1a60071070 2023-10-21T16:21:13.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a57fff640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1a4802b940 tx=0x7f1a480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:13.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4803ad30 con 0x7f1a60071070 2023-10-21T16:21:13.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480443f0 con 0x7f1a60071070 2023-10-21T16:21:13.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480447a0 con 0x7f1a60071070 2023-10-21T16:21:13.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a60154970 con 0x7f1a60071070 2023-10-21T16:21:13.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.030+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a60153ff0 con 0x7f1a60071070 2023-10-21T16:21:13.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.031+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a60154180 con 0x7f1a60071070 2023-10-21T16:21:13.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.032+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1a4804d450 con 0x7f1a60071070 2023-10-21T16:21:13.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.033+0000 7f1a5cff9640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 0x7f1a4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:13.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.033+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f1a480ac350 con 0x7f1a60071070 2023-10-21T16:21:13.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.034+0000 7f1a5f7fe640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 0x7f1a4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:13.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.035+0000 7f1a5f7fe640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 0x7f1a4006d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:13.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.037+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a48076ce0 con 0x7f1a60071070 2023-10-21T16:21:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.278+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1a6004e160 con 0x7f1a60071070 2023-10-21T16:21:13.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.802+0000 7f1a5cff9640 1 -- 172.21.15.53:0/2773060623 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1a4807a300 con 0x7f1a60071070 2023-10-21T16:21:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 msgr2=0x7f1a4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 0x7f1a4006d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2023-10-21T16:21:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 msgr2=0x7f1a60155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1a4802b940 tx=0x7f1a480046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 shutdown_connections 2023-10-21T16:21:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1a4006af90 0x7f1a4006d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a600afbe0 0x7f1a600cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a60071070 0x7f1a60155450 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 --2- 172.21.15.53:0/2773060623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a6006dfc0 0x7f1a60144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 >> 172.21.15.53:0/2773060623 conn(0x7f1a6006a5d0 msgr2=0x7f1a60159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 shutdown_connections 2023-10-21T16:21:13.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:13.807+0000 7f1a6614e640 1 -- 172.21.15.53:0/2773060623 wait complete. 2023-10-21T16:21:13.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:21:14.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.063+0000 7f322359e640 1 Processor -- start 2023-10-21T16:21:14.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.063+0000 7f322359e640 1 -- start start 2023-10-21T16:21:14.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32240b0680 0x7f322415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32240ca680 0x7f322415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32240cab60 con 0x7f322415a820 2023-10-21T16:21:14.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240cb960 con 0x7f32240ca680 2023-10-21T16:21:14.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32240cbdf0 con 0x7f32240b0680 2023-10-21T16:21:14.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f322415a820 0x7f32240aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37380/0 (socket says 172.21.15.53:37380) 2023-10-21T16:21:14.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f322259c640 1 -- 172.21.15.53:0/772971422 learned_addr learned my addr 172.21.15.53:0/772971422 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:14.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.065+0000 7f3222d9d640 1 --2- 172.21.15.53:0/772971422 >> v2:172.21.15.53:3300/0 conn(0x7f32240ca680 0x7f322415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f321bfff640 1 --2- 172.21.15.53:0/772971422 >> v2:172.21.15.124:3300/0 conn(0x7f32240b0680 0x7f322415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f3222d9d640 1 -- 172.21.15.53:0/772971422 >> v2:172.21.15.124:3300/0 conn(0x7f32240b0680 msgr2=0x7f322415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:14.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f3222d9d640 1 --2- 172.21.15.53:0/772971422 >> v2:172.21.15.124:3300/0 conn(0x7f32240b0680 0x7f322415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f3222d9d640 1 -- 172.21.15.53:0/772971422 >> v1:172.21.15.53:6789/0 conn(0x7f322415a820 legacy=0x7f32240aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f3222d9d640 1 -- 172.21.15.53:0/772971422 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240cb030 con 0x7f32240ca680 2023-10-21T16:21:14.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.066+0000 7f3221d9b640 1 -- 172.21.15.53:0/772971422 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283858288 0 0) 0x7f32240cab60 con 0x7f322415a820 2023-10-21T16:21:14.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.067+0000 7f3222d9d640 1 --2- 172.21.15.53:0/772971422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f322415ea80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f32240718e0 tx=0x7f321c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b66a4e6b21f099 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:14.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.067+0000 7f3221d9b640 1 -- 172.21.15.53:0/772971422 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c02cbe0 con 0x7f32240ca680 2023-10-21T16:21:14.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f3221d9b640 1 -- 172.21.15.53:0/772971422 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c02cd40 con 0x7f32240ca680 2023-10-21T16:21:14.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f3221d9b640 1 -- 172.21.15.53:0/772971422 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c034660 con 0x7f32240ca680 2023-10-21T16:21:14.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 -- 172.21.15.53:0/772971422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 msgr2=0x7f322415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 --2- 172.21.15.53:0/772971422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f322415ea80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f32240718e0 tx=0x7f321c02be30 comp rx=0 tx=0).stop 2023-10-21T16:21:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 -- 172.21.15.53:0/772971422 shutdown_connections 2023-10-21T16:21:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 --2- 172.21.15.53:0/772971422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f322415ea80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 --2- 172.21.15.53:0/772971422 >> v2:172.21.15.124:3300/0 conn(0x7f32240b0680 0x7f322415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 -- 172.21.15.53:0/772971422 >> 172.21.15.53:0/772971422 conn(0x7f322406c050 msgr2=0x7f322406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:14.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 -- 172.21.15.53:0/772971422 shutdown_connections 2023-10-21T16:21:14.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.068+0000 7f322359e640 1 -- 172.21.15.53:0/772971422 wait complete. 2023-10-21T16:21:14.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.070+0000 7f322359e640 1 Processor -- start 2023-10-21T16:21:14.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.070+0000 7f322359e640 1 -- start start 2023-10-21T16:21:14.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 0x7f32240b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 0x7f32240b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32240b1930 con 0x7f32240ca680 2023-10-21T16:21:14.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32240b15e0 con 0x7f322415a820 2023-10-21T16:21:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.071+0000 7f322359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32240cda10 con 0x7f32240b0680 2023-10-21T16:21:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43298/0 (socket says 172.21.15.53:43298) 2023-10-21T16:21:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 -- 172.21.15.53:0/790543245 learned_addr learned my addr 172.21.15.53:0/790543245 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:14.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f3222d9d640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 0x7f32240b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f322259c640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 0x7f32240b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 -- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 msgr2=0x7f32240b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 0x7f32240b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 -- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 msgr2=0x7f32240b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 0x7f32240b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f321bfff640 1 -- 172.21.15.53:0/790543245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009a70 con 0x7f32240ca680 2023-10-21T16:21:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f3222d9d640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 0x7f32240b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.072+0000 7f322259c640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 0x7f32240b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f321bfff640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d800 tx=0x7f320c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c00b6c0 con 0x7f32240ca680 2023-10-21T16:21:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c00b820 con 0x7f32240ca680 2023-10-21T16:21:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c00bbb0 con 0x7f32240ca680 2023-10-21T16:21:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32240cdcf0 con 0x7f32240ca680 2023-10-21T16:21:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.073+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32240b3720 con 0x7f32240ca680 2023-10-21T16:21:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.074+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32240cb5d0 con 0x7f32240ca680 2023-10-21T16:21:14.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.076+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f320c00bd10 con 0x7f32240ca680 2023-10-21T16:21:14.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.076+0000 7f321affd640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 0x7f320406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:14.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.077+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f320c08a230 con 0x7f32240ca680 2023-10-21T16:21:14.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.077+0000 7f322259c640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 0x7f320406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.078+0000 7f322259c640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 0x7f320406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.081+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32240cb5d0 con 0x7f32240ca680 2023-10-21T16:21:14.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.273+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32240cb5d0 con 0x7f32240ca680 2023-10-21T16:21:14.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.813+0000 7f321affd640 1 -- 172.21.15.53:0/790543245 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v138) v1 ==== 181+0+0 (secure 0 0 0) 0x7f32240cb5d0 con 0x7f32240ca680 2023-10-21T16:21:14.815 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:21:14.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 msgr2=0x7f320406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 0x7f320406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214009040 comp rx=0 tx=0).stop 2023-10-21T16:21:14.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 msgr2=0x7f32240cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:14.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d800 tx=0x7f320c00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 shutdown_connections 2023-10-21T16:21:14.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f320406af90 0x7f320406d450 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f60 tx=0x7f3214009040 comp rx=0 tx=0).stop 2023-10-21T16:21:14.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f322415a820 0x7f32240b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32240ca680 0x7f32240cd110 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 --2- 172.21.15.53:0/790543245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32240b0680 0x7f32240b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:14.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.818+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 >> 172.21.15.53:0/790543245 conn(0x7f322406c050 msgr2=0x7f32240b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:14.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.819+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 shutdown_connections 2023-10-21T16:21:14.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:14.819+0000 7f322359e640 1 -- 172.21.15.53:0/790543245 wait complete. 2023-10-21T16:21:14.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:21:15.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.057+0000 7f3708ad1640 1 Processor -- start 2023-10-21T16:21:15.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.057+0000 7f3708ad1640 1 -- start start 2023-10-21T16:21:15.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3708ad1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37040cf2b0 0x7f37040d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3708ad1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37040ce510 0x7f370415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3708ad1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37040cd520 con 0x7f37040cf690 2023-10-21T16:21:15.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3708ad1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040d1910 con 0x7f37040ce510 2023-10-21T16:21:15.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3708ad1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040ce080 con 0x7f37040cf2b0 2023-10-21T16:21:15.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3703fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37040ce510 0x7f370415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3702ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f37040cf690 0x7f37040d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37394/0 (socket says 172.21.15.53:37394) 2023-10-21T16:21:15.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3702ffd640 1 -- 172.21.15.53:0/1123640895 learned_addr learned my addr 172.21.15.53:0/1123640895 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:15.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f3703fff640 1 --2- 172.21.15.53:0/1123640895 >> v2:172.21.15.53:3300/0 conn(0x7f37040ce510 0x7f370415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43304/0 (socket says 172.21.15.53:43304) 2023-10-21T16:21:15.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.059+0000 7f37037fe640 1 --2- 172.21.15.53:0/1123640895 >> v2:172.21.15.124:3300/0 conn(0x7f37040cf2b0 0x7f37040d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.060+0000 7f3703fff640 1 -- 172.21.15.53:0/1123640895 >> v2:172.21.15.124:3300/0 conn(0x7f37040cf2b0 msgr2=0x7f37040d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:15.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.060+0000 7f3703fff640 1 --2- 172.21.15.53:0/1123640895 >> v2:172.21.15.124:3300/0 conn(0x7f37040cf2b0 0x7f37040d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.060+0000 7f3703fff640 1 -- 172.21.15.53:0/1123640895 >> v1:172.21.15.53:6789/0 conn(0x7f37040cf690 legacy=0x7f37040d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.060+0000 7f3703fff640 1 -- 172.21.15.53:0/1123640895 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040cd750 con 0x7f37040ce510 2023-10-21T16:21:15.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.061+0000 7f3703fff640 1 --2- 172.21.15.53:0/1123640895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040ce510 0x7f370415c820 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f37040b54b0 tx=0x7f36fc02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9001bea685afa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:15.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.061+0000 7f37027fc640 1 -- 172.21.15.53:0/1123640895 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc02cbe0 con 0x7f37040ce510 2023-10-21T16:21:15.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.061+0000 7f37027fc640 1 -- 172.21.15.53:0/1123640895 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc02cd40 con 0x7f37040ce510 2023-10-21T16:21:15.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.061+0000 7f37027fc640 1 -- 172.21.15.53:0/1123640895 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc034660 con 0x7f37040ce510 2023-10-21T16:21:15.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 -- 172.21.15.53:0/1123640895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040ce510 msgr2=0x7f370415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 --2- 172.21.15.53:0/1123640895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040ce510 0x7f370415c820 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f37040b54b0 tx=0x7f36fc02be30 comp rx=0 tx=0).stop 2023-10-21T16:21:15.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 -- 172.21.15.53:0/1123640895 shutdown_connections 2023-10-21T16:21:15.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 --2- 172.21.15.53:0/1123640895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040ce510 0x7f370415c820 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 --2- 172.21.15.53:0/1123640895 >> v2:172.21.15.124:3300/0 conn(0x7f37040cf2b0 0x7f37040d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 -- 172.21.15.53:0/1123640895 >> 172.21.15.53:0/1123640895 conn(0x7f370409bc40 msgr2=0x7f37040d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:15.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 -- 172.21.15.53:0/1123640895 shutdown_connections 2023-10-21T16:21:15.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.062+0000 7f3708ad1640 1 -- 172.21.15.53:0/1123640895 wait complete. 2023-10-21T16:21:15.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.064+0000 7f3708ad1640 1 Processor -- start 2023-10-21T16:21:15.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.064+0000 7f3708ad1640 1 -- start start 2023-10-21T16:21:15.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37040ce510 0x7f3704159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 0x7f37041fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37041fad10 con 0x7f37040cf2b0 2023-10-21T16:21:15.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040ca800 con 0x7f37040cf690 2023-10-21T16:21:15.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3708ad1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704158580 con 0x7f37040ce510 2023-10-21T16:21:15.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3702ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3702ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43306/0 (socket says 172.21.15.53:43306) 2023-10-21T16:21:15.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3702ffd640 1 -- 172.21.15.53:0/4218427907 learned_addr learned my addr 172.21.15.53:0/4218427907 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:15.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f3703fff640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 0x7f37041fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.065+0000 7f37037fe640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37040ce510 0x7f3704159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 -- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37040ce510 msgr2=0x7f3704159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37040ce510 0x7f3704159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 -- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 msgr2=0x7f37041fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 0x7f37041fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 -- 172.21.15.53:0/4218427907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009a70 con 0x7f37040cf2b0 2023-10-21T16:21:15.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3703fff640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 0x7f37041fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:15.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3702ffd640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c480 tx=0x7f36f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:15.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8005ce0 con 0x7f37040cf2b0 2023-10-21T16:21:15.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8005e40 con 0x7f37040cf2b0 2023-10-21T16:21:15.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80072f0 con 0x7f37040cf2b0 2023-10-21T16:21:15.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.066+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3704158860 con 0x7f37040cf2b0 2023-10-21T16:21:15.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.067+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3704158e20 con 0x7f37040cf2b0 2023-10-21T16:21:15.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.067+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3704074150 con 0x7f37040cf2b0 2023-10-21T16:21:15.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.069+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f36f8007450 con 0x7f37040cf2b0 2023-10-21T16:21:15.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.070+0000 7f36ebfff640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 0x7f36f006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:15.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.070+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6523+0+0 (secure 0 0 0) 0x7f36f8088f60 con 0x7f37040cf2b0 2023-10-21T16:21:15.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.070+0000 7f37037fe640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 0x7f36f006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:15.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.071+0000 7f37037fe640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 0x7f36f006d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36f4006fd0 tx=0x7f36f4008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:15.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.074+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36f8053c70 con 0x7f37040cf2b0 2023-10-21T16:21:15.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.268+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37040d0070 con 0x7f37040cf2b0 2023-10-21T16:21:15.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.825+0000 7f36ebfff640 1 -- 172.21.15.53:0/4218427907 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v139) v1 ==== 169+0+0 (secure 0 0 0) 0x7f36f8017020 con 0x7f37040cf2b0 2023-10-21T16:21:15.827 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:21:15.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 msgr2=0x7f36f006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 0x7f36f006d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f36f4006fd0 tx=0x7f36f4008040 comp rx=0 tx=0).stop 2023-10-21T16:21:15.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 msgr2=0x7f37041534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:15.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c480 tx=0x7f36f800c940 comp rx=0 tx=0).stop 2023-10-21T16:21:15.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 shutdown_connections 2023-10-21T16:21:15.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f36f006afc0 0x7f36f006d480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37040cf690 0x7f37041fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37040cf2b0 0x7f37041534d0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 --2- 172.21.15.53:0/4218427907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37040ce510 0x7f3704159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:15.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 >> 172.21.15.53:0/4218427907 conn(0x7f370409bc40 msgr2=0x7f37040b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:15.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.829+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 shutdown_connections 2023-10-21T16:21:15.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:15.830+0000 7f3708ad1640 1 -- 172.21.15.53:0/4218427907 wait complete. 2023-10-21T16:21:15.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:21:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.068+0000 7f7ee6d6c640 1 Processor -- start 2023-10-21T16:21:16.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.068+0000 7f7ee6d6c640 1 -- start start 2023-10-21T16:21:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee6d6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ee0160660 0x7f7ee0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee6d6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee6d6c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee006e150 con 0x7f7ee00cb450 2023-10-21T16:21:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee6d6c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee00cb8c0 con 0x7f7ee0160660 2023-10-21T16:21:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee6d6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee006de70 con 0x7f7ee00c9910 2023-10-21T16:21:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee4b08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ee0160660 0x7f7ee0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee4b08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7ee0160660 0x7f7ee0070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43318/0 (socket says 172.21.15.53:43318) 2023-10-21T16:21:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.070+0000 7f7ee4b08640 1 -- 172.21.15.53:0/566738020 learned_addr learned my addr 172.21.15.53:0/566738020 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:16.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.071+0000 7f7edffff640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.071+0000 7f7ee4b08640 1 -- 172.21.15.53:0/566738020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 msgr2=0x7f7ee00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:16.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.071+0000 7f7ee4b08640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee00caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.071+0000 7f7ee4b08640 1 -- 172.21.15.53:0/566738020 >> v1:172.21.15.124:6789/0 conn(0x7f7ee00cb450 legacy=0x7f7ee00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:16.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.071+0000 7f7ee4b08640 1 -- 172.21.15.53:0/566738020 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00c8e30 con 0x7f7ee0160660 2023-10-21T16:21:16.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.072+0000 7f7ee4b08640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee0160660 0x7f7ee0070cc0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009870 tx=0x7f7ed402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c2f276846e6647 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.072+0000 7f7edf7fe640 1 -- 172.21.15.53:0/566738020 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4031640 con 0x7f7ee0160660 2023-10-21T16:21:16.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.072+0000 7f7edf7fe640 1 -- 172.21.15.53:0/566738020 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed40317a0 con 0x7f7ee0160660 2023-10-21T16:21:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.072+0000 7f7edf7fe640 1 -- 172.21.15.53:0/566738020 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4031b30 con 0x7f7ee0160660 2023-10-21T16:21:16.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/566738020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee0160660 msgr2=0x7f7ee0070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee0160660 0x7f7ee0070cc0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009870 tx=0x7f7ed402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/566738020 shutdown_connections 2023-10-21T16:21:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/566738020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee0160660 0x7f7ee0070cc0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/566738020 >> 172.21.15.53:0/566738020 conn(0x7f7ee006a5d0 msgr2=0x7f7ee00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/566738020 shutdown_connections 2023-10-21T16:21:16.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.073+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/566738020 wait complete. 2023-10-21T16:21:16.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.076+0000 7f7ee6d6c640 1 Processor -- start 2023-10-21T16:21:16.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.076+0000 7f7ee6d6c640 1 -- start start 2023-10-21T16:21:16.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee0149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ee00cb450 0x7f7ee01f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0155160 con 0x7f7ee01f2d00 2023-10-21T16:21:16.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee014a260 con 0x7f7ee00cb450 2023-10-21T16:21:16.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.077+0000 7f7ee6d6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01f32f0 con 0x7f7ee00c9910 2023-10-21T16:21:16.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43332/0 (socket says 172.21.15.53:43332) 2023-10-21T16:21:16.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 -- 172.21.15.53:0/3058550537 learned_addr learned my addr 172.21.15.53:0/3058550537 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:16.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7edffff640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ee00cb450 0x7f7ee01f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee4b08640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee0149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 -- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 msgr2=0x7f7ee0149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee0149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 -- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ee00cb450 msgr2=0x7f7ee01f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ee00cb450 0x7f7ee01f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 -- 172.21.15.53:0/3058550537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40094d0 con 0x7f7ee01f2d00 2023-10-21T16:21:16.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.078+0000 7f7ee5309640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b970 tx=0x7f7ed800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.079+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800cb40 con 0x7f7ee01f2d00 2023-10-21T16:21:16.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.079+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed800cca0 con 0x7f7ee01f2d00 2023-10-21T16:21:16.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.079+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8014680 con 0x7f7ee01f2d00 2023-10-21T16:21:16.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.079+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee01f35d0 con 0x7f7ee01f2d00 2023-10-21T16:21:16.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.079+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee0155650 con 0x7f7ee01f2d00 2023-10-21T16:21:16.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.080+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee0155870 con 0x7f7ee01f2d00 2023-10-21T16:21:16.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.081+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7ed801d3b0 con 0x7f7ee01f2d00 2023-10-21T16:21:16.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.082+0000 7f7edd7fa640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 0x7f7ec406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.082+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f7ed80897b0 con 0x7f7ee01f2d00 2023-10-21T16:21:16.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.083+0000 7f7ee4b08640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 0x7f7ec406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.084+0000 7f7ee4b08640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 0x7f7ec406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4036000 tx=0x7f7ed403c040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.086+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ed8054520 con 0x7f7ee01f2d00 2023-10-21T16:21:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.278+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7ee004e160 con 0x7f7ee01f2d00 2023-10-21T16:21:16.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.284+0000 7f7edd7fa640 1 -- 172.21.15.53:0/3058550537 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ed8057b40 con 0x7f7ee01f2d00 2023-10-21T16:21:16.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 msgr2=0x7f7ec406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 0x7f7ec406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4036000 tx=0x7f7ed403c040 comp rx=0 tx=0).stop 2023-10-21T16:21:16.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 msgr2=0x7f7ee00cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b970 tx=0x7f7ed800be30 comp rx=0 tx=0).stop 2023-10-21T16:21:16.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 shutdown_connections 2023-10-21T16:21:16.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f7ec406af90 0x7f7ec406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ee01f2d00 0x7f7ee00cd8d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ee00cb450 0x7f7ee01f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 --2- 172.21.15.53:0/3058550537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ee00c9910 0x7f7ee0149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 >> 172.21.15.53:0/3058550537 conn(0x7f7ee006a5d0 msgr2=0x7f7ee015f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:16.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 shutdown_connections 2023-10-21T16:21:16.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.287+0000 7f7ee6d6c640 1 -- 172.21.15.53:0/3058550537 wait complete. 2023-10-21T16:21:16.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:21:16.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.532+0000 7f82c0b6c640 1 Processor -- start 2023-10-21T16:21:16.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.532+0000 7f82c0b6c640 1 -- start start 2023-10-21T16:21:16.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.534+0000 7f82c0b6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.534+0000 7f82c0b6c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82bc0b04a0 0x7f82bc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.534+0000 7f82c0b6c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82bc15ae90 con 0x7f82bc15aab0 2023-10-21T16:21:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.534+0000 7f82c0b6c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82bc0cb600 con 0x7f82bc0b04a0 2023-10-21T16:21:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.534+0000 7f82c0b6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0b0930 con 0x7f82bc070b50 2023-10-21T16:21:16.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.535+0000 7f82b3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.535+0000 7f82bb7fe640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f82bc15aab0 0x7f82bc0caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:57446/0 (socket says 172.21.15.53:57446) 2023-10-21T16:21:16.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.535+0000 7f82b3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:56830/0 (socket says 172.21.15.53:56830) 2023-10-21T16:21:16.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.535+0000 7f82bb7fe640 1 -- 172.21.15.53:0/3989872106 learned_addr learned my addr 172.21.15.53:0/3989872106 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:16.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.535+0000 7f82bbfff640 1 --2- 172.21.15.53:0/3989872106 >> v2:172.21.15.124:3300/0 conn(0x7f82bc0b04a0 0x7f82bc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.536+0000 7f82b3fff640 1 -- 172.21.15.53:0/3989872106 >> v2:172.21.15.124:3300/0 conn(0x7f82bc0b04a0 msgr2=0x7f82bc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:16.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.536+0000 7f82b3fff640 1 --2- 172.21.15.53:0/3989872106 >> v2:172.21.15.124:3300/0 conn(0x7f82bc0b04a0 0x7f82bc0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.536+0000 7f82b3fff640 1 -- 172.21.15.53:0/3989872106 >> v1:172.21.15.124:6789/0 conn(0x7f82bc15aab0 legacy=0x7f82bc0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.536+0000 7f82b3fff640 1 -- 172.21.15.53:0/3989872106 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc0cbdb0 con 0x7f82bc070b50 2023-10-21T16:21:16.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.538+0000 7f82b3fff640 1 --2- 172.21.15.53:0/3989872106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097a0 tx=0x7f82a402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abbb4808fa022af2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.538+0000 7f82baffd640 1 -- 172.21.15.53:0/3989872106 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4031780 con 0x7f82bc070b50 2023-10-21T16:21:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.538+0000 7f82baffd640 1 -- 172.21.15.53:0/3989872106 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40318e0 con 0x7f82bc070b50 2023-10-21T16:21:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.538+0000 7f82baffd640 1 -- 172.21.15.53:0/3989872106 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4031c70 con 0x7f82bc070b50 2023-10-21T16:21:16.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 -- 172.21.15.53:0/3989872106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 msgr2=0x7f82bc0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/3989872106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82a40097a0 tx=0x7f82a402b820 comp rx=0 tx=0).stop 2023-10-21T16:21:16.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 -- 172.21.15.53:0/3989872106 shutdown_connections 2023-10-21T16:21:16.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/3989872106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc070b50 0x7f82bc0aff60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/3989872106 >> v2:172.21.15.124:3300/0 conn(0x7f82bc0b04a0 0x7f82bc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 -- 172.21.15.53:0/3989872106 >> 172.21.15.53:0/3989872106 conn(0x7f82bc06c010 msgr2=0x7f82bc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 -- 172.21.15.53:0/3989872106 shutdown_connections 2023-10-21T16:21:16.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.539+0000 7f82c0b6c640 1 -- 172.21.15.53:0/3989872106 wait complete. 2023-10-21T16:21:16.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.541+0000 7f82c0b6c640 1 Processor -- start 2023-10-21T16:21:16.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.541+0000 7f82c0b6c640 1 -- start start 2023-10-21T16:21:16.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82bc0b04a0 0x7f82bc1f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 0x7f82bc0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0c3700 con 0x7f82bc070b50 2023-10-21T16:21:16.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0c39a0 con 0x7f82bc0b04a0 2023-10-21T16:21:16.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82c0b6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0c3c80 con 0x7f82bc15aab0 2023-10-21T16:21:16.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82bb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82bb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43336/0 (socket says 172.21.15.53:43336) 2023-10-21T16:21:16.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.542+0000 7f82bb7fe640 1 -- 172.21.15.53:0/271148135 learned_addr learned my addr 172.21.15.53:0/271148135 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:16.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82b3fff640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82bc0b04a0 0x7f82bc1f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bbfff640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 0x7f82bc0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 -- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 msgr2=0x7f82bc0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 0x7f82bc0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 -- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82bc0b04a0 msgr2=0x7f82bc1f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82bc0b04a0 0x7f82bc1f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 -- 172.21.15.53:0/271148135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009400 con 0x7f82bc070b50 2023-10-21T16:21:16.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bbfff640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 0x7f82bc0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82bb7fe640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c480 tx=0x7f82ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac005ce0 con 0x7f82bc070b50 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac005e40 con 0x7f82bc070b50 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82bc0c3f60 con 0x7f82bc070b50 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.543+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0072f0 con 0x7f82bc070b50 2023-10-21T16:21:16.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.544+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82bc158da0 con 0x7f82bc070b50 2023-10-21T16:21:16.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.545+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82bc074150 con 0x7f82bc070b50 2023-10-21T16:21:16.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.546+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82ac007510 con 0x7f82bc070b50 2023-10-21T16:21:16.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.547+0000 7f82b8ff9640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 0x7f829c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.547+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f82ac088cb0 con 0x7f82bc070b50 2023-10-21T16:21:16.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.547+0000 7f82b3fff640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 0x7f829c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.549+0000 7f82b3fff640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 0x7f829c06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82a4036000 tx=0x7f82a403d040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:16.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.551+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82ac053b60 con 0x7f82bc070b50 2023-10-21T16:21:16.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.744+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f82bc15ae90 con 0x7f82bc070b50 2023-10-21T16:21:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.750+0000 7f82b8ff9640 1 -- 172.21.15.53:0/271148135 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7f82bc15ae90 con 0x7f82bc070b50 2023-10-21T16:21:16.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.752+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 msgr2=0x7f829c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.752+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 0x7f829c06d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82a4036000 tx=0x7f82a403d040 comp rx=0 tx=0).stop 2023-10-21T16:21:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.752+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 msgr2=0x7f82bc158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.752+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c480 tx=0x7f82ac00c940 comp rx=0 tx=0).stop 2023-10-21T16:21:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 shutdown_connections 2023-10-21T16:21:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f829c06af90 0x7f829c06d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82bc15aab0 0x7f82bc0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82bc0b04a0 0x7f82bc1f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 --2- 172.21.15.53:0/271148135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82bc070b50 0x7f82bc158860 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 >> 172.21.15.53:0/271148135 conn(0x7f82bc06c010 msgr2=0x7f82bc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 shutdown_connections 2023-10-21T16:21:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.753+0000 7f82c0b6c640 1 -- 172.21.15.53:0/271148135 wait complete. 2023-10-21T16:21:16.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:21:16.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.992+0000 7f5b12459640 1 Processor -- start 2023-10-21T16:21:16.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.992+0000 7f5b12459640 1 -- start start 2023-10-21T16:21:16.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b12459640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5b0c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b12459640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5b0c0ce510 0x7f5b0c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b12459640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c0cd520 con 0x7f5b0c0cf2b0 2023-10-21T16:21:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b12459640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0d1910 con 0x7f5b0c0ce510 2023-10-21T16:21:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b12459640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0ce080 con 0x7f5b0c0cf690 2023-10-21T16:21:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b11457640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5b0c0cf2b0 0x7f5b0c0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37406/0 (socket says 172.21.15.53:37406) 2023-10-21T16:21:16.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.993+0000 7f5b11457640 1 -- 172.21.15.53:0/2175415365 learned_addr learned my addr 172.21.15.53:0/2175415365 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b10c56640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5b0c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189043720 0 0) 0x7f5b0c0cd520 con 0x7f5b0c0cf2b0 2023-10-21T16:21:16.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b11c58640 1 --2- 172.21.15.53:0/2175415365 >> v2:172.21.15.124:3300/0 conn(0x7f5b0c0ce510 0x7f5b0c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:16.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af8003520 con 0x7f5b0c0cf2b0 2023-10-21T16:21:16.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102482204 0 0) 0x7f5af8003520 con 0x7f5b0c0cf2b0 2023-10-21T16:21:16.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 msgr2=0x7f5b0c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:16.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5b0c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 >> v2:172.21.15.124:3300/0 conn(0x7f5b0c0ce510 msgr2=0x7f5b0c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 --2- 172.21.15.53:0/2175415365 >> v2:172.21.15.124:3300/0 conn(0x7f5b0c0ce510 0x7f5b0c15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0cd750 con 0x7f5b0c0cf2b0 2023-10-21T16:21:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5b08001670 con 0x7f5b0c0cf2b0 2023-10-21T16:21:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 >> v1:172.21.15.53:6789/0 conn(0x7f5b0c0cf2b0 legacy=0x7f5b0c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5af8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af8003f40 0x7f5af80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af8004d00 0x7f5af8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0d1910 con 0x7f5af8004d00 2023-10-21T16:21:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0ce080 con 0x7f5af8003f40 2023-10-21T16:21:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.994+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af800ba30 con 0x7f5b0c0cf690 2023-10-21T16:21:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af8004d00 0x7f5af8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 -- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 msgr2=0x7f5af8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11457640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5af8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b10c56640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af8003f40 0x7f5af80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5af8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 -- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af8003f40 msgr2=0x7f5af80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:17.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af8003f40 0x7f5af80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11c58640 1 -- 172.21.15.53:0/2175415365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af8003520 con 0x7f5af8004d00 2023-10-21T16:21:17.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.995+0000 7f5b11457640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0c0cf690 0x7f5af8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:17.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.996+0000 7f5b11c58640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af8004d00 0x7f5af8005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009fc0 tx=0x7f5af80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c1869c69d5c243 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0402cc60 con 0x7f5af8004d00 2023-10-21T16:21:17.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0400f040 con 0x7f5af8004d00 2023-10-21T16:21:17.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b03fff640 1 -- 172.21.15.53:0/2175415365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04034440 con 0x7f5af8004d00 2023-10-21T16:21:17.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 -- 172.21.15.53:0/2175415365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af8004d00 msgr2=0x7f5af8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 --2- 172.21.15.53:0/2175415365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af8004d00 0x7f5af8005180 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009fc0 tx=0x7f5af80057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b11c58640 1 -- 172.21.15.53:0/2175415365 reap_dead start 2023-10-21T16:21:17.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 -- 172.21.15.53:0/2175415365 shutdown_connections 2023-10-21T16:21:17.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 -- 172.21.15.53:0/2175415365 >> 172.21.15.53:0/2175415365 conn(0x7f5b0c09bc40 msgr2=0x7f5b0c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:17.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 -- 172.21.15.53:0/2175415365 shutdown_connections 2023-10-21T16:21:17.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.997+0000 7f5b12459640 1 -- 172.21.15.53:0/2175415365 wait complete. 2023-10-21T16:21:17.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.999+0000 7f5b12459640 1 Processor -- start 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:16.999+0000 7f5b12459640 1 -- start start 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.000+0000 7f5b12459640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af8004d00 0x7f5b0c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.000+0000 7f5b12459640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.000+0000 7f5b12459640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0c15ee50 0x7f5b0c15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.000+0000 7f5b12459640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c06c800 con 0x7f5b0c06be40 2023-10-21T16:21:17.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.000+0000 7f5b12459640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c160160 con 0x7f5b0c15ee50 2023-10-21T16:21:17.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b12459640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c15fba0 con 0x7f5af8004d00 2023-10-21T16:21:17.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43344/0 (socket says 172.21.15.53:43344) 2023-10-21T16:21:17.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 -- 172.21.15.53:0/522970692 learned_addr learned my addr 172.21.15.53:0/522970692 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:17.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b11457640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af8004d00 0x7f5b0c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 -- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af8004d00 msgr2=0x7f5b0c14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af8004d00 0x7f5b0c14ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 -- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0c15ee50 msgr2=0x7f5b0c15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0c15ee50 0x7f5b0c15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.001+0000 7f5b10c56640 1 -- 172.21.15.53:0/522970692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c15fe80 con 0x7f5b0c06be40 2023-10-21T16:21:17.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b10c56640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5afc010860 tx=0x7f5afc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc009610 con 0x7f5b0c06be40 2023-10-21T16:21:17.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc00eb60 con 0x7f5b0c06be40 2023-10-21T16:21:17.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc01c400 con 0x7f5b0c06be40 2023-10-21T16:21:17.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c14d8d0 con 0x7f5b0c06be40 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.002+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c14d200 con 0x7f5b0c06be40 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.003+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c074150 con 0x7f5b0c06be40 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.004+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5afc01c560 con 0x7f5b0c06be40 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.005+0000 7f5b01ffb640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 0x7f5af006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.005+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f5afc089e10 con 0x7f5b0c06be40 2023-10-21T16:21:17.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.006+0000 7f5b11457640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 0x7f5af006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.007+0000 7f5b11457640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 0x7f5af006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001200 tx=0x7f5b08009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.009+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5afc08d030 con 0x7f5b0c06be40 2023-10-21T16:21:17.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.203+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5b0c0d1dd0 con 0x7f5b0c06be40 2023-10-21T16:21:17.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.209+0000 7f5b01ffb640 1 -- 172.21.15.53:0/522970692 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5afc054cc0 con 0x7f5b0c06be40 2023-10-21T16:21:17.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 msgr2=0x7f5af006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 0x7f5af006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b08001200 tx=0x7f5b08009040 comp rx=0 tx=0).stop 2023-10-21T16:21:17.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 msgr2=0x7f5b0c06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5afc010860 tx=0x7f5afc010d20 comp rx=0 tx=0).stop 2023-10-21T16:21:17.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 shutdown_connections 2023-10-21T16:21:17.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5af006af90 0x7f5af006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0c15ee50 0x7f5b0c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b0c06be40 0x7f5b0c06c2c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 --2- 172.21.15.53:0/522970692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af8004d00 0x7f5b0c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 >> 172.21.15.53:0/522970692 conn(0x7f5b0c09bc40 msgr2=0x7f5b0c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:17.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 shutdown_connections 2023-10-21T16:21:17.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.213+0000 7f5b12459640 1 -- 172.21.15.53:0/522970692 wait complete. 2023-10-21T16:21:17.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-21T16:21:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.465+0000 7fc3ba197640 1 Processor -- start 2023-10-21T16:21:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.466+0000 7fc3ba197640 1 -- start start 2023-10-21T16:21:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3ba197640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b415f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3ba197640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3b415bad0 0x7fc3b40b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3ba197640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40b1500 con 0x7fc3b40b10b0 2023-10-21T16:21:17.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3ba197640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b415ee70 con 0x7fc3b415bad0 2023-10-21T16:21:17.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3ba197640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4158af0 con 0x7fc3b4158710 2023-10-21T16:21:17.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3b2ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3b415bad0 0x7fc3b40b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3b3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc3b40b10b0 0x7fc3b4069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37422/0 (socket says 172.21.15.53:37422) 2023-10-21T16:21:17.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3b3fff640 1 -- 172.21.15.53:0/3272462625 learned_addr learned my addr 172.21.15.53:0/3272462625 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:17.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.467+0000 7fc3b37fe640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b415f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.468+0000 7fc3b2ffd640 1 -- 172.21.15.53:0/3272462625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 msgr2=0x7fc3b415f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:17.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.468+0000 7fc3b2ffd640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b415f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.468+0000 7fc3b2ffd640 1 -- 172.21.15.53:0/3272462625 >> v1:172.21.15.53:6789/0 conn(0x7fc3b40b10b0 legacy=0x7fc3b4069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.468+0000 7fc3b2ffd640 1 -- 172.21.15.53:0/3272462625 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40ca370 con 0x7fc3b415bad0 2023-10-21T16:21:17.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.468+0000 7fc3b27fc640 1 -- 172.21.15.53:0/3272462625 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801329947 0 0) 0x7fc3ac001110 con 0x7fc3b40b10b0 2023-10-21T16:21:17.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.469+0000 7fc3b2ffd640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b415bad0 0x7fc3b40b0b70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097a0 tx=0x7fc39c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bc03f8f661f53b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.469+0000 7fc3b27fc640 1 -- 172.21.15.53:0/3272462625 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c031780 con 0x7fc3b415bad0 2023-10-21T16:21:17.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.469+0000 7fc3b27fc640 1 -- 172.21.15.53:0/3272462625 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c0318e0 con 0x7fc3b415bad0 2023-10-21T16:21:17.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.469+0000 7fc3b27fc640 1 -- 172.21.15.53:0/3272462625 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c031c70 con 0x7fc3b415bad0 2023-10-21T16:21:17.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 -- 172.21.15.53:0/3272462625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b415bad0 msgr2=0x7fc3b40b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b415bad0 0x7fc3b40b0b70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0097a0 tx=0x7fc39c02b820 comp rx=0 tx=0).stop 2023-10-21T16:21:17.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 -- 172.21.15.53:0/3272462625 shutdown_connections 2023-10-21T16:21:17.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b415bad0 0x7fc3b40b0b70 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3272462625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 -- 172.21.15.53:0/3272462625 >> 172.21.15.53:0/3272462625 conn(0x7fc3b40cf890 msgr2=0x7fc3b40cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:17.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 -- 172.21.15.53:0/3272462625 shutdown_connections 2023-10-21T16:21:17.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.470+0000 7fc3ba197640 1 -- 172.21.15.53:0/3272462625 wait complete. 2023-10-21T16:21:17.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.472+0000 7fc3ba197640 1 Processor -- start 2023-10-21T16:21:17.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.472+0000 7fc3ba197640 1 -- start start 2023-10-21T16:21:17.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b4143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 0x7fc3b41438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c9080 con 0x7fc3b40b10b0 2023-10-21T16:21:17.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40af670 con 0x7fc3b415bad0 2023-10-21T16:21:17.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.473+0000 7fc3ba197640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40af950 con 0x7fc3b4158710 2023-10-21T16:21:17.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43358/0 (socket says 172.21.15.53:43358) 2023-10-21T16:21:17.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 -- 172.21.15.53:0/3341172506 learned_addr learned my addr 172.21.15.53:0/3341172506 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:17.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b2ffd640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b4143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b3fff640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 0x7fc3b41438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 -- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 msgr2=0x7fc3b4143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b4143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 -- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 msgr2=0x7fc3b41438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 0x7fc3b41438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 -- 172.21.15.53:0/3341172506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009400 con 0x7fc3b40b10b0 2023-10-21T16:21:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b3fff640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 0x7fc3b41438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.474+0000 7fc3b37fe640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400efc0 tx=0x7fc3a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.475+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4014480 con 0x7fc3b40b10b0 2023-10-21T16:21:17.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.475+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a400f040 con 0x7fc3b40b10b0 2023-10-21T16:21:17.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.475+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4014800 con 0x7fc3b40b10b0 2023-10-21T16:21:17.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.475+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40afc30 con 0x7fc3b40b10b0 2023-10-21T16:21:17.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.475+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40b0410 con 0x7fc3b40b10b0 2023-10-21T16:21:17.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.476+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b40b1cb0 con 0x7fc3b40b10b0 2023-10-21T16:21:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.477+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc3a4014960 con 0x7fc3b40b10b0 2023-10-21T16:21:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.478+0000 7fc3abfff640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 0x7fc39406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.479+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fc3a408a700 con 0x7fc3b40b10b0 2023-10-21T16:21:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.479+0000 7fc3b2ffd640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 0x7fc39406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:17.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.481+0000 7fc3b2ffd640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 0x7fc39406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc39c036000 tx=0x7fc39c008030 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:17.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.483+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3a4055360 con 0x7fc3b40b10b0 2023-10-21T16:21:17.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.750+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fc3b4066d80 con 0x7fc3b40b10b0 2023-10-21T16:21:17.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.831+0000 7fc3abfff640 1 -- 172.21.15.53:0/3341172506 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v754) v1 ==== 148+0+0 (secure 0 0 0) 0x7fc3a4058980 con 0x7fc3b40b10b0 2023-10-21T16:21:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 msgr2=0x7fc39406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 0x7fc39406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc39c036000 tx=0x7fc39c008030 comp rx=0 tx=0).stop 2023-10-21T16:21:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 msgr2=0x7fc3b4145fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400efc0 tx=0x7fc3a400c490 comp rx=0 tx=0).stop 2023-10-21T16:21:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 shutdown_connections 2023-10-21T16:21:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fc39406af90 0x7fc39406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b415bad0 0x7fc3b41438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b4158710 0x7fc3b4143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 --2- 172.21.15.53:0/3341172506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3b40b10b0 0x7fc3b4145fa0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.835+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 >> 172.21.15.53:0/3341172506 conn(0x7fc3b40cf890 msgr2=0x7fc3b4157260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.836+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 shutdown_connections 2023-10-21T16:21:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:17.836+0000 7fc3ba197640 1 -- 172.21.15.53:0/3341172506 wait complete. 2023-10-21T16:21:17.857 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-21T16:21:17.858 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:21:17.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:18.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.091+0000 7f1994cd1640 1 Processor -- start 2023-10-21T16:21:18.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.092+0000 7f1994cd1640 1 -- start start 2023-10-21T16:21:18.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f1994cd1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 0x7f199015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f1994cd1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19900b07d0 0x7f199006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f1994cd1640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199015f7d0 con 0x7f19900adc70 2023-10-21T16:21:18.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f1994cd1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199015f210 con 0x7f19900b07d0 2023-10-21T16:21:18.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f1994cd1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f199015f4f0 con 0x7f199015ab30 2023-10-21T16:21:18.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f198ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19900b07d0 0x7f199006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f198ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19900b07d0 0x7f199006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43372/0 (socket says 172.21.15.53:43372) 2023-10-21T16:21:18.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.093+0000 7f198ffff640 1 -- 172.21.15.53:0/1830913717 learned_addr learned my addr 172.21.15.53:0/1830913717 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:18.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198f7fe640 1 --2- 172.21.15.53:0/1830913717 >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 0x7f199015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198ffff640 1 -- 172.21.15.53:0/1830913717 >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 msgr2=0x7f199015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:18.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198ffff640 1 --2- 172.21.15.53:0/1830913717 >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 0x7f199015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198ffff640 1 -- 172.21.15.53:0/1830913717 >> v1:172.21.15.124:6789/0 conn(0x7f19900adc70 legacy=0x7f19900b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:18.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198ffff640 1 -- 172.21.15.53:0/1830913717 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199006c550 con 0x7f19900b07d0 2023-10-21T16:21:18.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.094+0000 7f198f7fe640 1 --2- 172.21.15.53:0/1830913717 >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 0x7f199015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:18.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.095+0000 7f198ffff640 1 --2- 172.21.15.53:0/1830913717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19900b07d0 0x7f199006cd50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f199006ada0 tx=0x7f198802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e3eb155f2e5e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.095+0000 7f198e7fc640 1 -- 172.21.15.53:0/1830913717 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198802ca80 con 0x7f19900b07d0 2023-10-21T16:21:18.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.095+0000 7f198e7fc640 1 -- 172.21.15.53:0/1830913717 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198802cbe0 con 0x7f19900b07d0 2023-10-21T16:21:18.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.095+0000 7f198e7fc640 1 -- 172.21.15.53:0/1830913717 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19880343f0 con 0x7f19900b07d0 2023-10-21T16:21:18.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 -- 172.21.15.53:0/1830913717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19900b07d0 msgr2=0x7f199006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1830913717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19900b07d0 0x7f199006cd50 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f199006ada0 tx=0x7f198802bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 -- 172.21.15.53:0/1830913717 shutdown_connections 2023-10-21T16:21:18.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1830913717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19900b07d0 0x7f199006cd50 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1830913717 >> v2:172.21.15.124:3300/0 conn(0x7f199015ab30 0x7f199015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 -- 172.21.15.53:0/1830913717 >> 172.21.15.53:0/1830913717 conn(0x7f19900cb0c0 msgr2=0x7f19900cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:18.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 -- 172.21.15.53:0/1830913717 shutdown_connections 2023-10-21T16:21:18.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.096+0000 7f1994cd1640 1 -- 172.21.15.53:0/1830913717 wait complete. 2023-10-21T16:21:18.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.098+0000 7f1994cd1640 1 Processor -- start 2023-10-21T16:21:18.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.098+0000 7f1994cd1640 1 -- start start 2023-10-21T16:21:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19900adc70 0x7f19900c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19900b07d0 0x7f1990148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19900c83f0 con 0x7f199015ab30 2023-10-21T16:21:18.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19900c5a50 con 0x7f19900adc70 2023-10-21T16:21:18.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f1994cd1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19900c5d30 con 0x7f19900b07d0 2023-10-21T16:21:18.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f198ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.099+0000 7f198ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43382/0 (socket says 172.21.15.53:43382) 2023-10-21T16:21:18.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 -- 172.21.15.53:0/1002051039 learned_addr learned my addr 172.21.15.53:0/1002051039 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198f7fe640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19900adc70 0x7f19900c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198effd640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19900b07d0 0x7f1990148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 -- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19900b07d0 msgr2=0x7f1990148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19900b07d0 0x7f1990148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 -- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19900adc70 msgr2=0x7f19900c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19900adc70 0x7f19900c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 -- 172.21.15.53:0/1002051039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1988009b30 con 0x7f199015ab30 2023-10-21T16:21:18.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.100+0000 7f198ffff640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1988036040 tx=0x7f1988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.101+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198803e450 con 0x7f199015ab30 2023-10-21T16:21:18.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.101+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198803e5d0 con 0x7f199015ab30 2023-10-21T16:21:18.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.101+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198803e980 con 0x7f199015ab30 2023-10-21T16:21:18.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.101+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19900c6010 con 0x7f199015ab30 2023-10-21T16:21:18.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.101+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19900c65d0 con 0x7f199015ab30 2023-10-21T16:21:18.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.102+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19900c7300 con 0x7f199015ab30 2023-10-21T16:21:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.103+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f198802cd50 con 0x7f199015ab30 2023-10-21T16:21:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.104+0000 7f1977fff640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 0x7f197c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.104+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f19880ac560 con 0x7f199015ab30 2023-10-21T16:21:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.105+0000 7f198f7fe640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 0x7f197c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.106+0000 7f198f7fe640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 0x7f197c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f198000a950 tx=0x7f1980008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.108+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19900c7300 con 0x7f199015ab30 2023-10-21T16:21:18.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.307+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19900c7300 con 0x7f199015ab30 2023-10-21T16:21:18.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.309+0000 7f1977fff640 1 -- 172.21.15.53:0/1002051039 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+14441 (secure 0 0 0) 0x7f19900c7300 con 0x7f199015ab30 2023-10-21T16:21:18.311 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:18.311 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":139,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:15.814441+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6834/206156413":"2023-10-22T16:21:12.796225+0000","172.21.15.53:6835/206156413":"2023-10-22T16:21:12.796225+0000","172.21.15.124:0/1244050409":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1104588229":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/400910459":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/45913668":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3828835087":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6832/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/897501407":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2837232794":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/4124902964":"2023-10-22T16:21:08.619243+0000","172.21.15.124:6833/596536042":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3242447571":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1097749659":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/2545128602":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/1424561960":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/1535068499":"2023-10-22T16:20:39.664146+0000","172.21.15.124:0/3746315143":"2023-10-22T16:21:08.619243+0000","172.21.15.124:0/3443628834":"2023-10-22T16:20:39.664146+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:18.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 msgr2=0x7f197c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 0x7f197c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f198000a950 tx=0x7f1980008040 comp rx=0 tx=0).stop 2023-10-21T16:21:18.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 msgr2=0x7f19900af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1988036040 tx=0x7f1988004a40 comp rx=0 tx=0).stop 2023-10-21T16:21:18.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 shutdown_connections 2023-10-21T16:21:18.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f197c06af90 0x7f197c06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f199015ab30 0x7f19900af0c0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19900b07d0 0x7f1990148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 --2- 172.21.15.53:0/1002051039 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19900adc70 0x7f19900c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 >> 172.21.15.53:0/1002051039 conn(0x7f19900cb0c0 msgr2=0x7f199006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:18.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 shutdown_connections 2023-10-21T16:21:18.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.312+0000 7f1994cd1640 1 -- 172.21.15.53:0/1002051039 wait complete. 2023-10-21T16:21:18.334 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:18.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:18.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.566+0000 7fe59759e640 1 Processor -- start 2023-10-21T16:21:18.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.566+0000 7fe59759e640 1 -- start start 2023-10-21T16:21:18.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.567+0000 7fe59759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe5980cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.567+0000 7fe59759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5980cb810 con 0x7fe598159bc0 2023-10-21T16:21:18.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.567+0000 7fe59759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59815ac90 con 0x7fe5980cdc30 2023-10-21T16:21:18.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.567+0000 7fe59759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815a6d0 con 0x7fe5980caef0 2023-10-21T16:21:18.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.568+0000 7fe596d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe598159bc0 0x7fe5980711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37432/0 (socket says 172.21.15.53:37432) 2023-10-21T16:21:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.568+0000 7fe596d9d640 1 -- 172.21.15.53:0/1022388586 learned_addr learned my addr 172.21.15.53:0/1022388586 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.568+0000 7fe59659c640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe5980cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.568+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299695400 0 0) 0x7fe5980cb810 con 0x7fe598159bc0 2023-10-21T16:21:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.568+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe584003520 con 0x7fe598159bc0 2023-10-21T16:21:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564283174 0 0) 0x7fe59815ac90 con 0x7fe5980cdc30 2023-10-21T16:21:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5980cb810 con 0x7fe5980cdc30 2023-10-21T16:21:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 811575390 0 0) 0x7fe584003520 con 0x7fe598159bc0 2023-10-21T16:21:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 msgr2=0x7fe5980cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe5980cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 >> v1:172.21.15.124:6789/0 conn(0x7fe5980cdc30 legacy=0x7fe5980cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59815a9b0 con 0x7fe598159bc0 2023-10-21T16:21:18.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe5900019f0 con 0x7fe598159bc0 2023-10-21T16:21:18.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 >> v1:172.21.15.53:6789/0 conn(0x7fe598159bc0 legacy=0x7fe5980711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe584003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5840047c0 0x7fe584004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815a6d0 con 0x7fe584003b30 2023-10-21T16:21:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58400dc40 con 0x7fe5840047c0 2023-10-21T16:21:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58400de00 con 0x7fe5980caef0 2023-10-21T16:21:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.569+0000 7fe59659c640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe584003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe596d9d640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe59659c640 1 -- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 msgr2=0x7fe584005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe58ffff640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5840047c0 0x7fe584004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe59659c640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe584005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe59659c640 1 -- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5840047c0 msgr2=0x7fe584004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:18.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe59659c640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5840047c0 0x7fe584004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe59659c640 1 -- 172.21.15.53:0/1022388586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584003520 con 0x7fe584003b30 2023-10-21T16:21:18.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe596d9d640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980caef0 0x7fe584005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:18.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.570+0000 7fe58ffff640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5840047c0 0x7fe584004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:18.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.571+0000 7fe59659c640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe584003f30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ac0 tx=0x7fe5840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd0bd59453f0bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.571+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58802c6b0 con 0x7fe584003b30 2023-10-21T16:21:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.571+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58802c810 con 0x7fe584003b30 2023-10-21T16:21:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.571+0000 7fe595d9b640 1 -- 172.21.15.53:0/1022388586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58802cba0 con 0x7fe584003b30 2023-10-21T16:21:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 -- 172.21.15.53:0/1022388586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 msgr2=0x7fe584003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 --2- 172.21.15.53:0/1022388586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe584003f30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ac0 tx=0x7fe5840057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe596d9d640 1 -- 172.21.15.53:0/1022388586 reap_dead start 2023-10-21T16:21:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 -- 172.21.15.53:0/1022388586 shutdown_connections 2023-10-21T16:21:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 -- 172.21.15.53:0/1022388586 >> 172.21.15.53:0/1022388586 conn(0x7fe59806c010 msgr2=0x7fe59806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 -- 172.21.15.53:0/1022388586 shutdown_connections 2023-10-21T16:21:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.572+0000 7fe59759e640 1 -- 172.21.15.53:0/1022388586 wait complete. 2023-10-21T16:21:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.574+0000 7fe59759e640 1 Processor -- start 2023-10-21T16:21:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.574+0000 7fe59759e640 1 -- start start 2023-10-21T16:21:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980ab8d0 0x7fe5980abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe598152700 0x7fe598152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980acff0 con 0x7fe584003b30 2023-10-21T16:21:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5981530c0 con 0x7fe598152700 2023-10-21T16:21:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980ac370 con 0x7fe5980ab8d0 2023-10-21T16:21:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43394/0 (socket says 172.21.15.53:43394) 2023-10-21T16:21:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.575+0000 7fe59659c640 1 -- 172.21.15.53:0/4137454265 learned_addr learned my addr 172.21.15.53:0/4137454265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe596d9d640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe598152700 0x7fe598152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 -- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980ab8d0 msgr2=0x7fe5980abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980ab8d0 0x7fe5980abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 -- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe598152700 msgr2=0x7fe598152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe598152700 0x7fe598152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 -- 172.21.15.53:0/4137454265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980ac650 con 0x7fe584003b30 2023-10-21T16:21:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59659c640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ac0 tx=0x7fe588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588003810 con 0x7fe584003b30 2023-10-21T16:21:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588003970 con 0x7fe584003b30 2023-10-21T16:21:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588003d00 con 0x7fe584003b30 2023-10-21T16:21:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.576+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5980ac930 con 0x7fe584003b30 2023-10-21T16:21:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.577+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981f1dd0 con 0x7fe584003b30 2023-10-21T16:21:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.578+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598074150 con 0x7fe584003b30 2023-10-21T16:21:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.579+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe588040450 con 0x7fe584003b30 2023-10-21T16:21:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.580+0000 7fe58effd640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 0x7fe57806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.580+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fe5880ab8f0 con 0x7fe584003b30 2023-10-21T16:21:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.580+0000 7fe58ffff640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 0x7fe57806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.582+0000 7fe58ffff640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 0x7fe57806d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5800011e0 tx=0x7fe580009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:18.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.584+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5880766f0 con 0x7fe584003b30 2023-10-21T16:21:18.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.781+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe59806c850 con 0x7fe584003b30 2023-10-21T16:21:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.782+0000 7fe58effd640 1 -- 172.21.15.53:0/4137454265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5880044e0 con 0x7fe584003b30 2023-10-21T16:21:18.784 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 msgr2=0x7fe57806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 0x7fe57806d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5800011e0 tx=0x7fe580009040 comp rx=0 tx=0).stop 2023-10-21T16:21:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 msgr2=0x7fe5981f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fe588009ac0 tx=0x7fe588004730 comp rx=0 tx=0).stop 2023-10-21T16:21:18.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 shutdown_connections 2023-10-21T16:21:18.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fe57806afc0 0x7fe57806d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe598152700 0x7fe598152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5980ab8d0 0x7fe5980abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 --2- 172.21.15.53:0/4137454265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe584003b30 0x7fe5981f1890 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:18.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 >> 172.21.15.53:0/4137454265 conn(0x7fe59806c010 msgr2=0x7fe5980c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:18.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 shutdown_connections 2023-10-21T16:21:18.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:18.785+0000 7fe59759e640 1 -- 172.21.15.53:0/4137454265 wait complete. 2023-10-21T16:21:18.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-21T16:21:19.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.031+0000 7f557f42e640 1 Processor -- start 2023-10-21T16:21:19.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.031+0000 7f557f42e640 1 -- start start 2023-10-21T16:21:19.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.032+0000 7f557f42e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55781599d0 0x7f55780ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.032+0000 7f557f42e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55780cc210 0x7f55780b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557f42e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55780cacf0 con 0x7f5578159db0 2023-10-21T16:21:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557f42e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780b10f0 con 0x7f55780cc210 2023-10-21T16:21:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557f42e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780ca3c0 con 0x7f55781599d0 2023-10-21T16:21:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55780cc210 0x7f55780b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f55780cc210 0x7f55780b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43402/0 (socket says 172.21.15.53:43402) 2023-10-21T16:21:19.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 -- 172.21.15.53:0/3858678357 learned_addr learned my addr 172.21.15.53:0/3858678357 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:19.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 -- 172.21.15.53:0/3858678357 >> v2:172.21.15.124:3300/0 conn(0x7f55781599d0 msgr2=0x7f55780ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 --2- 172.21.15.53:0/3858678357 >> v2:172.21.15.124:3300/0 conn(0x7f55781599d0 0x7f55780ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 -- 172.21.15.53:0/3858678357 >> v1:172.21.15.124:6789/0 conn(0x7f5578159db0 legacy=0x7f557815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:19.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.033+0000 7f557dc2b640 1 -- 172.21.15.53:0/3858678357 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780ca6a0 con 0x7f55780cc210 2023-10-21T16:21:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.034+0000 7f557dc2b640 1 --2- 172.21.15.53:0/3858678357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55780cc210 0x7f55780b00c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5568009870 tx=0x7f556802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b844e1e88603d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:19.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557d42a640 1 -- 172.21.15.53:0/3858678357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568031860 con 0x7f55780cc210 2023-10-21T16:21:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557d42a640 1 -- 172.21.15.53:0/3858678357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55680319c0 con 0x7f55780cc210 2023-10-21T16:21:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557d42a640 1 -- 172.21.15.53:0/3858678357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568031d50 con 0x7f55780cc210 2023-10-21T16:21:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 -- 172.21.15.53:0/3858678357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55780cc210 msgr2=0x7f55780b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:19.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 --2- 172.21.15.53:0/3858678357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55780cc210 0x7f55780b00c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5568009870 tx=0x7f556802b900 comp rx=0 tx=0).stop 2023-10-21T16:21:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 -- 172.21.15.53:0/3858678357 shutdown_connections 2023-10-21T16:21:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 --2- 172.21.15.53:0/3858678357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55780cc210 0x7f55780b00c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 --2- 172.21.15.53:0/3858678357 >> v2:172.21.15.124:3300/0 conn(0x7f55781599d0 0x7f55780ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.035+0000 7f557f42e640 1 -- 172.21.15.53:0/3858678357 >> 172.21.15.53:0/3858678357 conn(0x7f55780cb0c0 msgr2=0x7f55780cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.036+0000 7f557f42e640 1 -- 172.21.15.53:0/3858678357 shutdown_connections 2023-10-21T16:21:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.036+0000 7f557f42e640 1 -- 172.21.15.53:0/3858678357 wait complete. 2023-10-21T16:21:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.037+0000 7f557f42e640 1 Processor -- start 2023-10-21T16:21:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 -- start start 2023-10-21T16:21:19.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55780cc210 0x7f5578148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55781599d0 0x7f55780c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c6e10 con 0x7f5578159db0 2023-10-21T16:21:19.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c6190 con 0x7f55781599d0 2023-10-21T16:21:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.038+0000 7f557f42e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c6470 con 0x7f55780cc210 2023-10-21T16:21:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43408/0 (socket says 172.21.15.53:43408) 2023-10-21T16:21:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 -- 172.21.15.53:0/1930049197 learned_addr learned my addr 172.21.15.53:0/1930049197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:19.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 -- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55780cc210 msgr2=0x7f5578148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:19.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55780cc210 0x7f5578148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 -- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55781599d0 msgr2=0x7f55780c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:19.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557dc2b640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55781599d0 0x7f55780c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:19.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55781599d0 0x7f55780c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.039+0000 7f557ec2d640 1 -- 172.21.15.53:0/1930049197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680094d0 con 0x7f5578159db0 2023-10-21T16:21:19.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f557ec2d640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f557000b930 tx=0x7f557000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:19.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557000cb20 con 0x7f5578159db0 2023-10-21T16:21:19.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557000ccc0 con 0x7f5578159db0 2023-10-21T16:21:19.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570014680 con 0x7f5578159db0 2023-10-21T16:21:19.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55780c6750 con 0x7f5578159db0 2023-10-21T16:21:19.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.040+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55780b3210 con 0x7f5578159db0 2023-10-21T16:21:19.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.041+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55780c6f50 con 0x7f5578159db0 2023-10-21T16:21:19.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.042+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55700147e0 con 0x7f5578159db0 2023-10-21T16:21:19.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.043+0000 7f556effd640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 0x7f555c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:19.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.043+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f5570089660 con 0x7f5578159db0 2023-10-21T16:21:19.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.044+0000 7f557e42c640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 0x7f555c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:19.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.045+0000 7f557e42c640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 0x7f555c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5574009710 tx=0x7f5574006c90 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:19.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.048+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55780c6f50 con 0x7f5578159db0 2023-10-21T16:21:19.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.313+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7f55780c6f50 con 0x7f5578159db0 2023-10-21T16:21:19.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.842+0000 7f556effd640 1 -- 172.21.15.53:0/1930049197 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v756) v1 ==== 163+0+0 (secure 0 0 0) 0x7f55780c6f50 con 0x7f5578159db0 2023-10-21T16:21:19.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 msgr2=0x7f555c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:19.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 0x7f555c06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5574009710 tx=0x7f5574006c90 comp rx=0 tx=0).stop 2023-10-21T16:21:19.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 msgr2=0x7f55780c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:19.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f557000b930 tx=0x7f557000bdf0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 shutdown_connections 2023-10-21T16:21:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f555c06af90 0x7f555c06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5578159db0 0x7f55780c7750 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55781599d0 0x7f55780c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 --2- 172.21.15.53:0/1930049197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55780cc210 0x7f5578148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:19.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 >> 172.21.15.53:0/1930049197 conn(0x7f55780cb0c0 msgr2=0x7f55780700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:19.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 shutdown_connections 2023-10-21T16:21:19.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:19.846+0000 7f557f42e640 1 -- 172.21.15.53:0/1930049197 wait complete. 2023-10-21T16:21:19.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:21:20.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.109+0000 7f4ffe639640 1 Processor -- start 2023-10-21T16:21:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.109+0000 7f4ffe639640 1 -- start start 2023-10-21T16:21:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.110+0000 7f4ffe639640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.110+0000 7f4ffe639640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ff80cb130 0x7f4ff80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.110+0000 7f4ffe639640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff80b2620 con 0x7f4ff806dfc0 2023-10-21T16:21:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.110+0000 7f4ffe639640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff80cbae0 con 0x7f4ff80cb130 2023-10-21T16:21:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.110+0000 7f4ffe639640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80b2060 con 0x7f4ff80afbe0 2023-10-21T16:21:20.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ff80cb130 0x7f4ff80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:20.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ff80cb130 0x7f4ff80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43418/0 (socket says 172.21.15.53:43418) 2023-10-21T16:21:20.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 -- 172.21.15.53:0/2626972195 learned_addr learned my addr 172.21.15.53:0/2626972195 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:20.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ff77fe640 1 --1- 172.21.15.53:0/2626972195 >> v1:172.21.15.124:6789/0 conn(0x7f4ff806dfc0 0x7f4ff8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:57498/0 (socket says 172.21.15.53:57498) 2023-10-21T16:21:20.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ff7fff640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:20.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 -- 172.21.15.53:0/2626972195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 msgr2=0x7f4ff80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:20.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 -- 172.21.15.53:0/2626972195 >> v1:172.21.15.124:6789/0 conn(0x7f4ff806dfc0 legacy=0x7f4ff8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.111+0000 7f4ffcbd6640 1 -- 172.21.15.53:0/2626972195 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80b2340 con 0x7f4ff80cb130 2023-10-21T16:21:20.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.112+0000 7f4ffcbd6640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff80cb130 0x7f4ff80cb5a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e50 tx=0x7f4ff002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d19a263e79ac8a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:20.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ff6ffd640 1 -- 172.21.15.53:0/2626972195 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff002cb00 con 0x7f4ff80cb130 2023-10-21T16:21:20.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ff6ffd640 1 -- 172.21.15.53:0/2626972195 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff002cc60 con 0x7f4ff80cb130 2023-10-21T16:21:20.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ff6ffd640 1 -- 172.21.15.53:0/2626972195 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00343f0 con 0x7f4ff80cb130 2023-10-21T16:21:20.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 -- 172.21.15.53:0/2626972195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff80cb130 msgr2=0x7f4ff80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff80cb130 0x7f4ff80cb5a0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009e50 tx=0x7f4ff002bf60 comp rx=0 tx=0).stop 2023-10-21T16:21:20.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 -- 172.21.15.53:0/2626972195 shutdown_connections 2023-10-21T16:21:20.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff80cb130 0x7f4ff80cb5a0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 --2- 172.21.15.53:0/2626972195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 -- 172.21.15.53:0/2626972195 >> 172.21.15.53:0/2626972195 conn(0x7f4ff806a5d0 msgr2=0x7f4ff80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:20.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.113+0000 7f4ffe639640 1 -- 172.21.15.53:0/2626972195 shutdown_connections 2023-10-21T16:21:20.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.114+0000 7f4ffe639640 1 -- 172.21.15.53:0/2626972195 wait complete. 2023-10-21T16:21:20.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.115+0000 7f4ffe639640 1 Processor -- start 2023-10-21T16:21:20.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.115+0000 7f4ffe639640 1 -- start start 2023-10-21T16:21:20.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff80cb130 0x7f4ff8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80cc850 con 0x7f4ff806dfc0 2023-10-21T16:21:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8157fe0 con 0x7f4ff80cb130 2023-10-21T16:21:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.116+0000 7f4ffe639640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8157740 con 0x7f4ff80afbe0 2023-10-21T16:21:20.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:20.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43422/0 (socket says 172.21.15.53:43422) 2023-10-21T16:21:20.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 -- 172.21.15.53:0/1145795683 learned_addr learned my addr 172.21.15.53:0/1145795683 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:20.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 -- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 msgr2=0x7f4ff80cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 -- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff80cb130 msgr2=0x7f4ff8155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff80cb130 0x7f4ff8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 -- 172.21.15.53:0/1145795683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009b30 con 0x7f4ff806dfc0 2023-10-21T16:21:20.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.117+0000 7f4ff7fff640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800e8a0 tx=0x7f4fe800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:20.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.118+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800c940 con 0x7f4ff806dfc0 2023-10-21T16:21:20.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.118+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe800caa0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.118+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe800ce50 con 0x7f4ff806dfc0 2023-10-21T16:21:20.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.118+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff8157a20 con 0x7f4ff806dfc0 2023-10-21T16:21:20.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.118+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff80cf4a0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.119+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff80c74e0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.120+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fe8013470 con 0x7f4ff806dfc0 2023-10-21T16:21:20.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.121+0000 7f4ff4ff9640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 0x7f4fd806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:20.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.121+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f4fe80899f0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.122+0000 7f4ff77fe640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 0x7f4fd806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:20.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.123+0000 7f4ff77fe640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 0x7f4fd806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815db10 tx=0x7f4fe00079c0 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:20.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.125+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fe80547f0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.319+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4ff804e160 con 0x7f4ff806dfc0 2023-10-21T16:21:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.326+0000 7f4ff4ff9640 1 -- 172.21.15.53:0/1145795683 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fe80170c0 con 0x7f4ff806dfc0 2023-10-21T16:21:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 msgr2=0x7f4fd806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 0x7f4fd806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ff815db10 tx=0x7f4fe00079c0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 msgr2=0x7f4ff8157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800e8a0 tx=0x7f4fe800ed60 comp rx=0 tx=0).stop 2023-10-21T16:21:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 shutdown_connections 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f4fd806af90 0x7f4fd806d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff80cb130 0x7f4ff8155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff80afbe0 0x7f4ff80cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 --2- 172.21.15.53:0/1145795683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ff806dfc0 0x7f4ff8157110 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 >> 172.21.15.53:0/1145795683 conn(0x7f4ff806a5d0 msgr2=0x7f4ff80c5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 shutdown_connections 2023-10-21T16:21:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:20.329+0000 7f4ffe639640 1 -- 172.21.15.53:0/1145795683 wait complete. 2023-10-21T16:21:20.354 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:21:20.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:20.476 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:20.609 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:21:20.609 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:20.610 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:20.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:21:20.676 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:20.676 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:20.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:20.737 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:20.737 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:20.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:21:20.802 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:20.802 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:20.862 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:20.863 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:21:20.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:21.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.099+0000 7f6886b16640 1 Processor -- start 2023-10-21T16:21:21.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.099+0000 7f6886b16640 1 -- start start 2023-10-21T16:21:21.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f688015cc60 0x7f68800cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f68800d16f0 0x7f68800cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 0x7f68800ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f688015d040 con 0x7f688015cc60 2023-10-21T16:21:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68800d0650 con 0x7f68800d16f0 2023-10-21T16:21:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6886b16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68800d0930 con 0x7f68800cdfd0 2023-10-21T16:21:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.101+0000 7f6885b14640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f688015cc60 0x7f68800cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f688015cc60 0x7f68800cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43438/0 (socket says 172.21.15.53:43438) 2023-10-21T16:21:21.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 -- 172.21.15.53:0/2907205249 learned_addr learned my addr 172.21.15.53:0/2907205249 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6886315640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 0x7f68800ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885313640 1 --2- 172.21.15.53:0/2907205249 >> v2:172.21.15.124:3300/0 conn(0x7f68800d16f0 0x7f68800cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 -- 172.21.15.53:0/2907205249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 msgr2=0x7f68800ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 0x7f68800ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 -- 172.21.15.53:0/2907205249 >> v2:172.21.15.124:3300/0 conn(0x7f68800d16f0 msgr2=0x7f68800cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:21.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 --2- 172.21.15.53:0/2907205249 >> v2:172.21.15.124:3300/0 conn(0x7f68800d16f0 0x7f68800cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6885b14640 1 -- 172.21.15.53:0/2907205249 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800d0c10 con 0x7f688015cc60 2023-10-21T16:21:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.102+0000 7f6886315640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 0x7f68800ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.103+0000 7f6885b14640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f688015cc60 0x7f68800cf340 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f68800b5560 tx=0x7f687c02b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bb0ee2b4b0e106 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6884b12640 1 -- 172.21.15.53:0/2907205249 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c004030 con 0x7f688015cc60 2023-10-21T16:21:21.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6884b12640 1 -- 172.21.15.53:0/2907205249 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c004190 con 0x7f688015cc60 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6884b12640 1 -- 172.21.15.53:0/2907205249 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c031640 con 0x7f688015cc60 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 -- 172.21.15.53:0/2907205249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f688015cc60 msgr2=0x7f68800cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f688015cc60 0x7f68800cf340 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f68800b5560 tx=0x7f687c02b8e0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 -- 172.21.15.53:0/2907205249 shutdown_connections 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68800cdfd0 0x7f68800ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 --2- 172.21.15.53:0/2907205249 >> v2:172.21.15.124:3300/0 conn(0x7f68800d16f0 0x7f68800cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 --2- 172.21.15.53:0/2907205249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f688015cc60 0x7f68800cf340 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 -- 172.21.15.53:0/2907205249 >> 172.21.15.53:0/2907205249 conn(0x7f688009bcf0 msgr2=0x7f68800d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.104+0000 7f6886b16640 1 -- 172.21.15.53:0/2907205249 shutdown_connections 2023-10-21T16:21:21.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.105+0000 7f6886b16640 1 -- 172.21.15.53:0/2907205249 wait complete. 2023-10-21T16:21:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.106+0000 7f6886b16640 1 Processor -- start 2023-10-21T16:21:21.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.106+0000 7f6886b16640 1 -- start start 2023-10-21T16:21:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68800d16f0 0x7f6880158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 0x7f688006c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800b5560 con 0x7f68800cdfd0 2023-10-21T16:21:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800b57b0 con 0x7f68800d16f0 2023-10-21T16:21:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.107+0000 7f6886b16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68800b5a90 con 0x7f688015cc60 2023-10-21T16:21:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43440/0 (socket says 172.21.15.53:43440) 2023-10-21T16:21:21.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 -- 172.21.15.53:0/647860560 learned_addr learned my addr 172.21.15.53:0/647860560 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6886315640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 0x7f688006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885313640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68800d16f0 0x7f6880158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 -- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 msgr2=0x7f688006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 0x7f688006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 -- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68800d16f0 msgr2=0x7f6880158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68800d16f0 0x7f6880158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6885b14640 1 -- 172.21.15.53:0/647860560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c009460 con 0x7f68800cdfd0 2023-10-21T16:21:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.108+0000 7f6886315640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 0x7f688006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f6885b14640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c003bc0 con 0x7f68800cdfd0 2023-10-21T16:21:21.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687c03f070 con 0x7f68800cdfd0 2023-10-21T16:21:21.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687c031640 con 0x7f68800cdfd0 2023-10-21T16:21:21.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68800b5d70 con 0x7f68800cdfd0 2023-10-21T16:21:21.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.109+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688006c5d0 con 0x7f68800cdfd0 2023-10-21T16:21:21.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.110+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68800b6290 con 0x7f68800cdfd0 2023-10-21T16:21:21.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.111+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f687c041030 con 0x7f68800cdfd0 2023-10-21T16:21:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.113+0000 7f68767fc640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 0x7f686406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.113+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f687c0abe10 con 0x7f68800cdfd0 2023-10-21T16:21:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.114+0000 7f6885313640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 0x7f686406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.115+0000 7f6885313640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 0x7f686406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6870006fd0 tx=0x7f6870008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.117+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68800b6290 con 0x7f68800cdfd0 2023-10-21T16:21:21.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.350+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68800b6290 con 0x7f68800cdfd0 2023-10-21T16:21:21.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.351+0000 7f68767fc640 1 -- 172.21.15.53:0/647860560 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f68800b6290 con 0x7f68800cdfd0 2023-10-21T16:21:21.353 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:21.353 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1910850368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1910850368},{"type":"v1","addr":"172.21.15.124:6835","nonce":1910850368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6943,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2290263099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2290263099},{"type":"v1","addr":"172.21.15.124:6837","nonce":2290263099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8076,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2271482221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2271482221},{"type":"v1","addr":"172.21.15.53:6833","nonce":2271482221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1902966950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1902966950},{"type":"v1","addr":"172.21.15.53:6835","nonce":1902966950}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[]} 2023-10-21T16:21:21.354 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 63 2023-10-21T16:21:21.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 msgr2=0x7f686406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 0x7f686406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6870006fd0 tx=0x7f6870008040 comp rx=0 tx=0).stop 2023-10-21T16:21:21.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 msgr2=0x7f68801f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 shutdown_connections 2023-10-21T16:21:21.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f686406afc0 0x7f686406d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f688015cc60 0x7f688006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68800d16f0 0x7f6880158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 --2- 172.21.15.53:0/647860560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68800cdfd0 0x7f68801f3660 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 >> 172.21.15.53:0/647860560 conn(0x7f688009bcf0 msgr2=0x7f68800b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:21.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.354+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 shutdown_connections 2023-10-21T16:21:21.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.355+0000 7f6886b16640 1 -- 172.21.15.53:0/647860560 wait complete. 2023-10-21T16:21:21.374 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:21:21.374 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:21:21.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:21:21.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:21:21.375 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:21:21.376 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:21:21.376 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:21:21.376 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:21:21.376 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:21:21.377 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:21:21.377 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:21:21.377 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:21:21.378 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:21:21.378 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:21:21.378 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:21:21.378 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:21:21.391 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:21:21.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:21:21.394 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:21:21.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:21:21.402 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:21:21.402 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:21:21.403 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:21:21.404 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:21:21.437 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:21:21.438 DEBUG:teuthology.parallel:result is None 2023-10-21T16:21:21.441 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:21:21.441 DEBUG:teuthology.parallel:result is None 2023-10-21T16:21:21.450 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:21:21.451 DEBUG:teuthology.parallel:result is None 2023-10-21T16:21:21.457 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:21:21.458 DEBUG:teuthology.parallel:result is None 2023-10-21T16:21:21.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:21:21.598 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:21:21.599 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:21:21.618 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:21:21.627 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:21:21.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.651+0000 7fb0b00da640 1 Processor -- start 2023-10-21T16:21:21.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.651+0000 7fb0b00da640 1 -- start start 2023-10-21T16:21:21.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0b00da640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb0a8161690 0x7fb0a806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0b00da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb0a80b2f30 0x7fb0a80cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0b00da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a806cc50 con 0x7fb0a80cb150 2023-10-21T16:21:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0b00da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80cbb40 con 0x7fb0a80b2f30 2023-10-21T16:21:21.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0b00da640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a806cf30 con 0x7fb0a8161690 2023-10-21T16:21:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0ad675640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb0a80cb150 0x7fb0a80702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37448/0 (socket says 172.21.15.53:37448) 2023-10-21T16:21:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0ad675640 1 -- 172.21.15.53:0/2843657854 learned_addr learned my addr 172.21.15.53:0/2843657854 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0ae677640 1 --2- 172.21.15.53:0/2843657854 >> v2:172.21.15.53:3300/0 conn(0x7fb0a80b2f30 0x7fb0a80cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.653+0000 7fb0ade76640 1 --2- 172.21.15.53:0/2843657854 >> v2:172.21.15.124:3300/0 conn(0x7fb0a8161690 0x7fb0a806dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.654+0000 7fb0ae677640 1 -- 172.21.15.53:0/2843657854 >> v2:172.21.15.124:3300/0 conn(0x7fb0a8161690 msgr2=0x7fb0a806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:21.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.654+0000 7fb0ae677640 1 --2- 172.21.15.53:0/2843657854 >> v2:172.21.15.124:3300/0 conn(0x7fb0a8161690 0x7fb0a806dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.654+0000 7fb0ae677640 1 -- 172.21.15.53:0/2843657854 >> v1:172.21.15.53:6789/0 conn(0x7fb0a80cb150 legacy=0x7fb0a80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.654+0000 7fb0ae677640 1 -- 172.21.15.53:0/2843657854 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a806d210 con 0x7fb0a80b2f30 2023-10-21T16:21:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.654+0000 7fb0ace74640 1 -- 172.21.15.53:0/2843657854 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346372658 0 0) 0x7fb0a806cc50 con 0x7fb0a80cb150 2023-10-21T16:21:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.655+0000 7fb0ae677640 1 --2- 172.21.15.53:0/2843657854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80cb600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b0d50 tx=0x7fb0a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d029e1929a8c91dc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.655+0000 7fb0ace74640 1 -- 172.21.15.53:0/2843657854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a002cc80 con 0x7fb0a80b2f30 2023-10-21T16:21:21.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.655+0000 7fb0ace74640 1 -- 172.21.15.53:0/2843657854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a002cde0 con 0x7fb0a80b2f30 2023-10-21T16:21:21.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.655+0000 7fb0ace74640 1 -- 172.21.15.53:0/2843657854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0034660 con 0x7fb0a80b2f30 2023-10-21T16:21:21.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 -- 172.21.15.53:0/2843657854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80b2f30 msgr2=0x7fb0a80cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 --2- 172.21.15.53:0/2843657854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80cb600 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b0d50 tx=0x7fb0a002bed0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 -- 172.21.15.53:0/2843657854 shutdown_connections 2023-10-21T16:21:21.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 --2- 172.21.15.53:0/2843657854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80cb600 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 --2- 172.21.15.53:0/2843657854 >> v2:172.21.15.124:3300/0 conn(0x7fb0a8161690 0x7fb0a806dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 -- 172.21.15.53:0/2843657854 >> 172.21.15.53:0/2843657854 conn(0x7fb0a806a5d0 msgr2=0x7fb0a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 -- 172.21.15.53:0/2843657854 shutdown_connections 2023-10-21T16:21:21.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.656+0000 7fb0b00da640 1 -- 172.21.15.53:0/2843657854 wait complete. 2023-10-21T16:21:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.658+0000 7fb0b00da640 1 Processor -- start 2023-10-21T16:21:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.658+0000 7fb0b00da640 1 -- start start 2023-10-21T16:21:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0a8161690 0x7fb0a81574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8159e70 con 0x7fb0a80cb150 2023-10-21T16:21:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a81627d0 con 0x7fb0a80b2f30 2023-10-21T16:21:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0b00da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a8159260 con 0x7fb0a8161690 2023-10-21T16:21:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0ad675640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0ad675640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43482/0 (socket says 172.21.15.53:43482) 2023-10-21T16:21:21.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0ad675640 1 -- 172.21.15.53:0/3687853898 learned_addr learned my addr 172.21.15.53:0/3687853898 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0ade76640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.659+0000 7fb0ae677640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0a8161690 0x7fb0a81574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 -- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0a8161690 msgr2=0x7fb0a81574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0a8161690 0x7fb0a81574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 -- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 msgr2=0x7fb0a80c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:21.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:21.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 -- 172.21.15.53:0/3687853898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0009a70 con 0x7fb0a80cb150 2023-10-21T16:21:21.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ade76640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:21.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0ad675640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c4d0 tx=0x7fb09800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0980074e0 con 0x7fb0a80cb150 2023-10-21T16:21:21.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098007660 con 0x7fb0a80cb150 2023-10-21T16:21:21.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098004a20 con 0x7fb0a80cb150 2023-10-21T16:21:21.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a8159540 con 0x7fb0a80cb150 2023-10-21T16:21:21.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.660+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8159b00 con 0x7fb0a80cb150 2023-10-21T16:21:21.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.661+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a80b0df0 con 0x7fb0a80cb150 2023-10-21T16:21:21.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.663+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb098004b80 con 0x7fb0a80cb150 2023-10-21T16:21:21.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.663+0000 7fb09e7fc640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 0x7fb08c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:21.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.664+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fb098088b60 con 0x7fb0a80cb150 2023-10-21T16:21:21.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.664+0000 7fb0ade76640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 0x7fb08c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:21.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.665+0000 7fb0ade76640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 0x7fb08c06d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009af0 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:21.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.668+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0980538d0 con 0x7fb0a80cb150 2023-10-21T16:21:21.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:21.871+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb0a804e160 con 0x7fb0a80cb150 2023-10-21T16:21:22.408 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:21:22.415 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:21:23.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.364+0000 7fb09e7fc640 1 -- 172.21.15.53:0/3687853898 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb098018020 con 0x7fb0a80cb150 2023-10-21T16:21:23.366 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:21:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.367+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 msgr2=0x7fb08c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.367+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 0x7fb08c06d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009af0 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2023-10-21T16:21:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.367+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 msgr2=0x7fb0a80c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.367+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c4d0 tx=0x7fb09800c990 comp rx=0 tx=0).stop 2023-10-21T16:21:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 shutdown_connections 2023-10-21T16:21:23.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb08c06af90 0x7fb08c06d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0a8161690 0x7fb0a81574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0a80cb150 0x7fb0a80c7c50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 --2- 172.21.15.53:0/3687853898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0a80b2f30 0x7fb0a80c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 >> 172.21.15.53:0/3687853898 conn(0x7fb0a806a5d0 msgr2=0x7fb0a80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 shutdown_connections 2023-10-21T16:21:23.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.368+0000 7fb0b00da640 1 -- 172.21.15.53:0/3687853898 wait complete. 2023-10-21T16:21:23.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:21:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.614+0000 7f38fe0e6640 1 Processor -- start 2023-10-21T16:21:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.614+0000 7f38fe0e6640 1 -- start start 2023-10-21T16:21:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.616+0000 7f38fe0e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.616+0000 7f38fe0e6640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f38f80cc210 0x7f38f80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.616+0000 7f38fe0e6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80cacf0 con 0x7f38f8159db0 2023-10-21T16:21:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.616+0000 7f38fe0e6640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80b10f0 con 0x7f38f80cc210 2023-10-21T16:21:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.616+0000 7f38fe0e6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80ca3c0 con 0x7f38f81599d0 2023-10-21T16:21:23.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38fd8e5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f38f8159db0 0x7f38f815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37458/0 (socket says 172.21.15.53:37458) 2023-10-21T16:21:23.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38fd8e5640 1 -- 172.21.15.53:0/2588072587 learned_addr learned my addr 172.21.15.53:0/2588072587 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:23.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/2588072587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38fc8e3640 1 --2- 172.21.15.53:0/2588072587 >> v2:172.21.15.124:3300/0 conn(0x7f38f80cc210 0x7f38f80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38effff640 1 -- 172.21.15.53:0/2588072587 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372513260 0 0) 0x7f38f80cacf0 con 0x7f38f8159db0 2023-10-21T16:21:23.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.617+0000 7f38effff640 1 -- 172.21.15.53:0/2588072587 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4005730 con 0x7f38f8159db0 2023-10-21T16:21:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.618+0000 7f38fd0e4640 1 -- 172.21.15.53:0/2588072587 >> v2:172.21.15.124:3300/0 conn(0x7f38f80cc210 msgr2=0x7f38f80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.618+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/2588072587 >> v2:172.21.15.124:3300/0 conn(0x7f38f80cc210 0x7f38f80b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.618+0000 7f38fd0e4640 1 -- 172.21.15.53:0/2588072587 >> v1:172.21.15.53:6789/0 conn(0x7f38f8159db0 legacy=0x7f38f815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.618+0000 7f38fd0e4640 1 -- 172.21.15.53:0/2588072587 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80ca6a0 con 0x7f38f81599d0 2023-10-21T16:21:23.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.619+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/2588072587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f80ad730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38f40097a0 tx=0x7f38f402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c12e7e7aa2d511 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.619+0000 7f38effff640 1 -- 172.21.15.53:0/2588072587 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f402d040 con 0x7f38f81599d0 2023-10-21T16:21:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38effff640 1 -- 172.21.15.53:0/2588072587 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4031810 con 0x7f38f81599d0 2023-10-21T16:21:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38effff640 1 -- 172.21.15.53:0/2588072587 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4031ba0 con 0x7f38f81599d0 2023-10-21T16:21:23.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 -- 172.21.15.53:0/2588072587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 msgr2=0x7f38f80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/2588072587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f80ad730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38f40097a0 tx=0x7f38f402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 -- 172.21.15.53:0/2588072587 shutdown_connections 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/2588072587 >> v2:172.21.15.124:3300/0 conn(0x7f38f80cc210 0x7f38f80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/2588072587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f80ad730 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 -- 172.21.15.53:0/2588072587 >> 172.21.15.53:0/2588072587 conn(0x7f38f80cb0c0 msgr2=0x7f38f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:23.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.620+0000 7f38fe0e6640 1 -- 172.21.15.53:0/2588072587 shutdown_connections 2023-10-21T16:21:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.621+0000 7f38fe0e6640 1 -- 172.21.15.53:0/2588072587 wait complete. 2023-10-21T16:21:23.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.623+0000 7f38fe0e6640 1 Processor -- start 2023-10-21T16:21:23.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.623+0000 7f38fe0e6640 1 -- start start 2023-10-21T16:21:23.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.624+0000 7f38fe0e6640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 0x7f38f80ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fe0e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fe0e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fe0e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80c7c60 con 0x7f38f8159db0 2023-10-21T16:21:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fe0e6640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80c6bc0 con 0x7f38f80cc210 2023-10-21T16:21:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fe0e6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80c6ea0 con 0x7f38f81599d0 2023-10-21T16:21:23.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fd8e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fd8e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43486/0 (socket says 172.21.15.53:43486) 2023-10-21T16:21:23.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fd8e5640 1 -- 172.21.15.53:0/3548952139 learned_addr learned my addr 172.21.15.53:0/3548952139 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:23.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 0x7f38f80ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.625+0000 7f38fc8e3640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 -- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 msgr2=0x7f38f8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f8157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 -- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 msgr2=0x7f38f80ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 0x7f38f80ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 -- 172.21.15.53:0/3548952139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009400 con 0x7f38f8159db0 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 0x7f38f80ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:23.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fc8e3640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:23.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fd8e5640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f38f000caa0 tx=0x7f38f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000ac00 con 0x7f38f8159db0 2023-10-21T16:21:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000ad60 con 0x7f38f8159db0 2023-10-21T16:21:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0012620 con 0x7f38f8159db0 2023-10-21T16:21:23.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.626+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f80c7180 con 0x7f38f8159db0 2023-10-21T16:21:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.627+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80c7740 con 0x7f38f8159db0 2023-10-21T16:21:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.628+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f804e160 con 0x7f38f8159db0 2023-10-21T16:21:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.629+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38f0007070 con 0x7f38f8159db0 2023-10-21T16:21:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.630+0000 7f38edffb640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 0x7f38dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:23.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.630+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f38f0089580 con 0x7f38f8159db0 2023-10-21T16:21:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.631+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 0x7f38dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:23.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.632+0000 7f38fd0e4640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 0x7f38dc06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f38f4036000 tx=0x7f38f403c040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:23.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.634+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38f00546e0 con 0x7f38f8159db0 2023-10-21T16:21:23.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.897+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f38f80c7900 con 0x7f38f8159db0 2023-10-21T16:21:23.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.898+0000 7f38edffb640 1 -- 172.21.15.53:0/3548952139 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f38f0017090 con 0x7f38f8159db0 2023-10-21T16:21:23.900 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:23.900 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:21:23.901 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:21:23.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.901 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:21:23.901 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:21:23.901 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:21:23.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:21:23.902 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:21:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:21:23.903 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.905 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:21:23.905 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:21:23.905 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.905 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.906 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:21:23.906 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:21:23.906 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.907 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:21:23.908 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:21:23.908 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.910 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:21:23.910 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:21:23.911 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.911 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.911 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.911 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.912 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.913 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.913 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.914 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.915 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:21:23.915 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.917 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:21:23.917 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:21:23.917 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.917 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.918 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:21:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:21:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.919 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.920 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.920 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:21:23.920 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.922 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:21:23.922 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:21:23.922 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.922 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.923 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:21:23.923 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:21:23.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:21:23.924 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:21:23.925 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:21:23.925 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.925 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:21:23.926 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:21:23.926 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:21:23.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:21:23.927 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:21:23.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.927 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:21:23.928 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:21:23.929 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:21:23.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:21:23.930 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:21:23.930 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.931 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:21:23.931 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:21:23.932 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:21:23.932 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:21:23.932 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:21:23.933 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:21:23.934 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:21:23.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:21:23.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:23.934 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:21:23.935 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:21:23.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 msgr2=0x7f38dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 0x7f38dc06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f38f4036000 tx=0x7f38f403c040 comp rx=0 tx=0).stop 2023-10-21T16:21:23.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 msgr2=0x7f38f814f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:23.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f38f000caa0 tx=0x7f38f000cf60 comp rx=0 tx=0).stop 2023-10-21T16:21:23.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 shutdown_connections 2023-10-21T16:21:23.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f38dc06af90 0x7f38dc06d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f8159db0 0x7f38f814f980 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f81599d0 0x7f38f8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 --2- 172.21.15.53:0/3548952139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f80cc210 0x7f38f80ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:23.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 >> 172.21.15.53:0/3548952139 conn(0x7f38f80cb0c0 msgr2=0x7f38f80b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:23.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.901+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 shutdown_connections 2023-10-21T16:21:23.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:23.902+0000 7f38fe0e6640 1 -- 172.21.15.53:0/3548952139 wait complete. 2023-10-21T16:21:23.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:24.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.172+0000 7f14fc843640 1 Processor -- start 2023-10-21T16:21:24.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.173+0000 7f14fc843640 1 -- start start 2023-10-21T16:21:24.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14fc843640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14f80adc70 0x7f14f80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14fc843640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14f815af10 0x7f14f80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14fc843640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f815f7d0 con 0x7f14f815ab30 2023-10-21T16:21:24.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14fc843640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f815efb0 con 0x7f14f815af10 2023-10-21T16:21:24.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14fc843640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f815f210 con 0x7f14f80adc70 2023-10-21T16:21:24.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14f759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14f815af10 0x7f14f80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14f759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14f815af10 0x7f14f80b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49530/0 (socket says 172.21.15.53:49530) 2023-10-21T16:21:24.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.174+0000 7f14f759e640 1 -- 172.21.15.53:0/2884182164 learned_addr learned my addr 172.21.15.53:0/2884182164 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:24.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.175+0000 7f14f759e640 1 -- 172.21.15.53:0/2884182164 >> v2:172.21.15.124:3300/0 conn(0x7f14f80adc70 msgr2=0x7f14f80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:24.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.175+0000 7f14f759e640 1 --2- 172.21.15.53:0/2884182164 >> v2:172.21.15.124:3300/0 conn(0x7f14f80adc70 0x7f14f80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.175+0000 7f14f759e640 1 -- 172.21.15.53:0/2884182164 >> v1:172.21.15.124:6789/0 conn(0x7f14f815ab30 legacy=0x7f14f815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:24.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.175+0000 7f14f759e640 1 -- 172.21.15.53:0/2884182164 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f815f4f0 con 0x7f14f815af10 2023-10-21T16:21:24.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.176+0000 7f14f759e640 1 --2- 172.21.15.53:0/2884182164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f815af10 0x7f14f80b0600 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ca780 tx=0x7f14f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccc4aae5d454122 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.176+0000 7f14f659c640 1 -- 172.21.15.53:0/2884182164 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f002ca50 con 0x7f14f815af10 2023-10-21T16:21:24.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.176+0000 7f14f659c640 1 -- 172.21.15.53:0/2884182164 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f002cbb0 con 0x7f14f815af10 2023-10-21T16:21:24.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.176+0000 7f14f659c640 1 -- 172.21.15.53:0/2884182164 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00343f0 con 0x7f14f815af10 2023-10-21T16:21:24.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 -- 172.21.15.53:0/2884182164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f815af10 msgr2=0x7f14f80b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 --2- 172.21.15.53:0/2884182164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f815af10 0x7f14f80b0600 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ca780 tx=0x7f14f002beb0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 -- 172.21.15.53:0/2884182164 shutdown_connections 2023-10-21T16:21:24.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 --2- 172.21.15.53:0/2884182164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f815af10 0x7f14f80b0600 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 --2- 172.21.15.53:0/2884182164 >> v2:172.21.15.124:3300/0 conn(0x7f14f80adc70 0x7f14f80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 -- 172.21.15.53:0/2884182164 >> 172.21.15.53:0/2884182164 conn(0x7f14f80cb0c0 msgr2=0x7f14f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:24.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 -- 172.21.15.53:0/2884182164 shutdown_connections 2023-10-21T16:21:24.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.177+0000 7f14fc843640 1 -- 172.21.15.53:0/2884182164 wait complete. 2023-10-21T16:21:24.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.179+0000 7f14fc843640 1 Processor -- start 2023-10-21T16:21:24.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.179+0000 7f14fc843640 1 -- start start 2023-10-21T16:21:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 0x7f14f814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14f815af10 0x7f14f815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f815cae0 con 0x7f14f80adc70 2023-10-21T16:21:24.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f814f440 con 0x7f14f815ab30 2023-10-21T16:21:24.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14fc843640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f815be60 con 0x7f14f815af10 2023-10-21T16:21:24.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14f6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14f6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49544/0 (socket says 172.21.15.53:49544) 2023-10-21T16:21:24.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.180+0000 7f14f6d9d640 1 -- 172.21.15.53:0/1546002798 learned_addr learned my addr 172.21.15.53:0/1546002798 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:24.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14effff640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 0x7f14f814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 -- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14f815af10 msgr2=0x7f14f815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14f815af10 0x7f14f815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 -- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 msgr2=0x7f14f814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 0x7f14f814ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 -- 172.21.15.53:0/1546002798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0009b30 con 0x7f14f80adc70 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14effff640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 0x7f14f814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14f6d9d640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c430 tx=0x7f14e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011790 con 0x7f14f80adc70 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e80118f0 con 0x7f14f80adc70 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011ce0 con 0x7f14f80adc70 2023-10-21T16:21:24.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.181+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f815c140 con 0x7f14f80adc70 2023-10-21T16:21:24.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.182+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f815c700 con 0x7f14f80adc70 2023-10-21T16:21:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.183+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f804e160 con 0x7f14f80adc70 2023-10-21T16:21:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.184+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f14e801c460 con 0x7f14f80adc70 2023-10-21T16:21:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.185+0000 7f14ef7fe640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 0x7f14d806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.185+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14e80894d0 con 0x7f14f80adc70 2023-10-21T16:21:24.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.185+0000 7f14effff640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 0x7f14d806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.187+0000 7f14effff640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 0x7f14d806d4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e0006fd0 tx=0x7f14e0008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.189+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14e80546a0 con 0x7f14f80adc70 2023-10-21T16:21:24.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.387+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14f80ada50 con 0x7f14f80adc70 2023-10-21T16:21:24.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.388+0000 7f14ef7fe640 1 -- 172.21.15.53:0/1546002798 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+13175 (secure 0 0 0) 0x7f14e8057cc0 con 0x7f14f80adc70 2023-10-21T16:21:24.390 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:24.391 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":144,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:23.361584+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:24.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 msgr2=0x7f14d806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 0x7f14d806d4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e0006fd0 tx=0x7f14e0008040 comp rx=0 tx=0).stop 2023-10-21T16:21:24.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 msgr2=0x7f14f815cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c430 tx=0x7f14e800c8f0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 shutdown_connections 2023-10-21T16:21:24.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f14d806b010 0x7f14d806d4d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14f815af10 0x7f14f815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14f815ab30 0x7f14f814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 --2- 172.21.15.53:0/1546002798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14f80adc70 0x7f14f815cf00 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.391+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 >> 172.21.15.53:0/1546002798 conn(0x7f14f80cb0c0 msgr2=0x7f14f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:24.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.392+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 shutdown_connections 2023-10-21T16:21:24.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.392+0000 7f14fc843640 1 -- 172.21.15.53:0/1546002798 wait complete. 2023-10-21T16:21:24.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:24.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.648+0000 7fd392abb640 1 Processor -- start 2023-10-21T16:21:24.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.649+0000 7fd392abb640 1 -- start start 2023-10-21T16:21:24.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.650+0000 7fd392abb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd38c1432f0 0x7fd38c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.650+0000 7fd392abb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd38c0b4240 0x7fd38c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.650+0000 7fd392abb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0cbb30 con 0x7fd38c0cb6b0 2023-10-21T16:21:24.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.650+0000 7fd392abb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c1436d0 con 0x7fd38c0b4240 2023-10-21T16:21:24.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.650+0000 7fd392abb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c15d440 con 0x7fd38c1432f0 2023-10-21T16:21:24.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd391058640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd38c0b4240 0x7fd38c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd38bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd38c0cb6b0 0x7fd38c0b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37296/0 (socket says 172.21.15.53:37296) 2023-10-21T16:21:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd38bfff640 1 -- 172.21.15.53:0/2338649794 learned_addr learned my addr 172.21.15.53:0/2338649794 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd390857640 1 --2- 172.21.15.53:0/2338649794 >> v2:172.21.15.124:3300/0 conn(0x7fd38c1432f0 0x7fd38c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd391058640 1 -- 172.21.15.53:0/2338649794 >> v2:172.21.15.124:3300/0 conn(0x7fd38c1432f0 msgr2=0x7fd38c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:24.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd391058640 1 --2- 172.21.15.53:0/2338649794 >> v2:172.21.15.124:3300/0 conn(0x7fd38c1432f0 0x7fd38c0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd38b7fe640 1 -- 172.21.15.53:0/2338649794 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178977983 0 0) 0x7fd38c0cbb30 con 0x7fd38c0cb6b0 2023-10-21T16:21:24.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd391058640 1 -- 172.21.15.53:0/2338649794 >> v1:172.21.15.53:6789/0 conn(0x7fd38c0cb6b0 legacy=0x7fd38c0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.651+0000 7fd391058640 1 -- 172.21.15.53:0/2338649794 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c15d690 con 0x7fd38c0b4240 2023-10-21T16:21:24.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.653+0000 7fd391058640 1 --2- 172.21.15.53:0/2338649794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c159e50 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0cc1a0 tx=0x7fd38402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a89559b52836973 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.653+0000 7fd38b7fe640 1 -- 172.21.15.53:0/2338649794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38402e070 con 0x7fd38c0b4240 2023-10-21T16:21:24.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.653+0000 7fd38b7fe640 1 -- 172.21.15.53:0/2338649794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3840315a0 con 0x7fd38c0b4240 2023-10-21T16:21:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.653+0000 7fd38b7fe640 1 -- 172.21.15.53:0/2338649794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384031930 con 0x7fd38c0b4240 2023-10-21T16:21:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 -- 172.21.15.53:0/2338649794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 msgr2=0x7fd38c159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 --2- 172.21.15.53:0/2338649794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c159e50 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0cc1a0 tx=0x7fd38402b840 comp rx=0 tx=0).stop 2023-10-21T16:21:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 -- 172.21.15.53:0/2338649794 shutdown_connections 2023-10-21T16:21:24.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 --2- 172.21.15.53:0/2338649794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c159e50 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 --2- 172.21.15.53:0/2338649794 >> v2:172.21.15.124:3300/0 conn(0x7fd38c1432f0 0x7fd38c0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 -- 172.21.15.53:0/2338649794 >> 172.21.15.53:0/2338649794 conn(0x7fd38c069d50 msgr2=0x7fd38c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:24.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 -- 172.21.15.53:0/2338649794 shutdown_connections 2023-10-21T16:21:24.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.654+0000 7fd392abb640 1 -- 172.21.15.53:0/2338649794 wait complete. 2023-10-21T16:21:24.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.656+0000 7fd392abb640 1 Processor -- start 2023-10-21T16:21:24.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.656+0000 7fd392abb640 1 -- start start 2023-10-21T16:21:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 0x7fd38c1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 0x7fd38c142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cd3c0 con 0x7fd38c0b4240 2023-10-21T16:21:24.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cdf00 con 0x7fd38c1432f0 2023-10-21T16:21:24.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd392abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0cd5d0 con 0x7fd38c0cb6b0 2023-10-21T16:21:24.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd390857640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd390857640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49568/0 (socket says 172.21.15.53:49568) 2023-10-21T16:21:24.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.657+0000 7fd390857640 1 -- 172.21.15.53:0/2920990967 learned_addr learned my addr 172.21.15.53:0/2920990967 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:24.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 -- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 msgr2=0x7fd38c1670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd38bfff640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 0x7fd38c1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd391058640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 0x7fd38c142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 0x7fd38c1670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 -- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 msgr2=0x7fd38c142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 0x7fd38c142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 -- 172.21.15.53:0/2920990967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009400 con 0x7fd38c0b4240 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd38bfff640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 0x7fd38c1670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd391058640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 0x7fd38c142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd390857640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd37c007e40 tx=0x7fd37c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014620 con 0x7fd38c0b4240 2023-10-21T16:21:24.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c00ee90 con 0x7fd38c0b4240 2023-10-21T16:21:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0149c0 con 0x7fd38c0b4240 2023-10-21T16:21:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c0cd8b0 con 0x7fd38c0b4240 2023-10-21T16:21:24.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.658+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c167610 con 0x7fd38c0b4240 2023-10-21T16:21:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.660+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c04e160 con 0x7fd38c0b4240 2023-10-21T16:21:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.661+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd37c014b20 con 0x7fd38c0b4240 2023-10-21T16:21:24.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.662+0000 7fd3897fa640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 0x7fd37806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.662+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd37c089fe0 con 0x7fd38c0b4240 2023-10-21T16:21:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.663+0000 7fd38bfff640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 0x7fd37806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:24.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.664+0000 7fd38bfff640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 0x7fd37806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd38c1f5b50 tx=0x7fd380009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:24.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.667+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd37c055100 con 0x7fd38c0b4240 2023-10-21T16:21:24.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.858+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd38c0b2140 con 0x7fd38c0b4240 2023-10-21T16:21:24.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.859+0000 7fd3897fa640 1 -- 172.21.15.53:0/2920990967 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd37c058720 con 0x7fd38c0b4240 2023-10-21T16:21:24.861 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:24.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 msgr2=0x7fd37806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 0x7fd37806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd38c1f5b50 tx=0x7fd380009040 comp rx=0 tx=0).stop 2023-10-21T16:21:24.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 msgr2=0x7fd38c1657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:24.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fd37c007e40 tx=0x7fd37c00e4b0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 shutdown_connections 2023-10-21T16:21:24.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd37806af90 0x7fd37806d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd38c1432f0 0x7fd38c142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd38c0cb6b0 0x7fd38c1670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 --2- 172.21.15.53:0/2920990967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd38c0b4240 0x7fd38c1657e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:24.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 >> 172.21.15.53:0/2920990967 conn(0x7fd38c069d50 msgr2=0x7fd38c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:24.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 shutdown_connections 2023-10-21T16:21:24.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:24.862+0000 7fd392abb640 1 -- 172.21.15.53:0/2920990967 wait complete. 2023-10-21T16:21:24.883 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:21:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.105+0000 7f2dfbfff640 1 Processor -- start 2023-10-21T16:21:25.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.106+0000 7f2dfbfff640 1 -- start start 2023-10-21T16:21:25.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.107+0000 7f2dfbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2dfc15ab30 0x7f2dfc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.107+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2dfc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfbfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dfc15f7d0 con 0x7f2dfc0adc70 2023-10-21T16:21:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfbfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dfc15f210 con 0x7f2dfc15ab30 2023-10-21T16:21:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15f4f0 con 0x7f2dfc0b07d0 2023-10-21T16:21:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfa7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2dfc0adc70 0x7f2dfc0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37310/0 (socket says 172.21.15.53:37310) 2023-10-21T16:21:25.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfa7fc640 1 -- 172.21.15.53:0/2273214564 learned_addr learned my addr 172.21.15.53:0/2273214564 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2dfc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130234931 0 0) 0x7f2dfc15f7d0 con 0x7f2dfc0adc70 2023-10-21T16:21:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2dfaffd640 1 --2- 172.21.15.53:0/2273214564 >> v2:172.21.15.124:3300/0 conn(0x7f2dfc15ab30 0x7f2dfc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.108+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de4003520 con 0x7f2dfc0adc70 2023-10-21T16:21:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530476457 0 0) 0x7f2de4003520 con 0x7f2dfc0adc70 2023-10-21T16:21:25.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 msgr2=0x7f2dfc06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2dfc06cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 >> v2:172.21.15.124:3300/0 conn(0x7f2dfc15ab30 msgr2=0x7f2dfc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 --2- 172.21.15.53:0/2273214564 >> v2:172.21.15.124:3300/0 conn(0x7f2dfc15ab30 0x7f2dfc15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc06c550 con 0x7f2dfc0adc70 2023-10-21T16:21:25.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2df0001670 con 0x7f2dfc0adc70 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 >> v1:172.21.15.53:6789/0 conn(0x7f2dfc0adc70 legacy=0x7f2dfc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de4003970 0x7f2de4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2de40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4004d00 0x7f2de4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15f210 con 0x7f2de4004d00 2023-10-21T16:21:25.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc15f4f0 con 0x7f2de4003970 2023-10-21T16:21:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de400b980 con 0x7f2dfc0b07d0 2023-10-21T16:21:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.109+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4004d00 0x7f2de4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfa7fc640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2de40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 msgr2=0x7f2de40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2de40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de4003970 msgr2=0x7f2de4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de4003970 0x7f2de4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2273214564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4003520 con 0x7f2de4004d00 2023-10-21T16:21:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.110+0000 7f2dfa7fc640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2dfc0b07d0 0x7f2de40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:25.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4004d00 0x7f2de4005180 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fc0 tx=0x7f2de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e15f611ef6e847 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df402cbd0 con 0x7f2de4004d00 2023-10-21T16:21:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df400f040 con 0x7f2de4004d00 2023-10-21T16:21:25.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2df9ffb640 1 -- 172.21.15.53:0/2273214564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4034440 con 0x7f2de4004d00 2023-10-21T16:21:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2273214564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4004d00 msgr2=0x7f2de4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2273214564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4004d00 0x7f2de4005180 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fc0 tx=0x7f2de40057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.111+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2273214564 reap_dead start 2023-10-21T16:21:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.112+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2273214564 shutdown_connections 2023-10-21T16:21:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.112+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2273214564 >> 172.21.15.53:0/2273214564 conn(0x7f2dfc0cb0c0 msgr2=0x7f2dfc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.112+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2273214564 shutdown_connections 2023-10-21T16:21:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.112+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2273214564 wait complete. 2023-10-21T16:21:25.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.114+0000 7f2dfbfff640 1 Processor -- start 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.114+0000 7f2dfbfff640 1 -- start start 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de4004d00 0x7f2dfc155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2dfc0c23a0 0x7f2dfc0c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0c8470 con 0x7f2dfc0c7530 2023-10-21T16:21:25.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0c5890 con 0x7f2dfc0c23a0 2023-10-21T16:21:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc0c5b70 con 0x7f2de4004d00 2023-10-21T16:21:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49592/0 (socket says 172.21.15.53:49592) 2023-10-21T16:21:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2061534537 learned_addr learned my addr 172.21.15.53:0/2061534537 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:25.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.115+0000 7f2dfaffd640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de4004d00 0x7f2dfc155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfa7fc640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2dfc0c23a0 0x7f2dfc0c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de4004d00 msgr2=0x7f2dfc155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de4004d00 0x7f2dfc155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2dfc0c23a0 msgr2=0x7f2dfc0c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2dfc0c23a0 0x7f2dfc0c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 -- 172.21.15.53:0/2061534537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc0c5e50 con 0x7f2dfc0c7530 2023-10-21T16:21:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfb7fe640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2df4002230 tx=0x7f2df4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4043070 con 0x7f2dfc0c7530 2023-10-21T16:21:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df402de60 con 0x7f2dfc0c7530 2023-10-21T16:21:25.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc0c6130 con 0x7f2dfc0c7530 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4034d10 con 0x7f2dfc0c7530 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.116+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc0ceaf0 con 0x7f2dfc0c7530 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.118+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc0c6650 con 0x7f2dfc0c7530 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.119+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2df403e440 con 0x7f2dfc0c7530 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.119+0000 7f2de37fe640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 0x7f2de806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.120+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2df40ab070 con 0x7f2dfc0c7530 2023-10-21T16:21:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.120+0000 7f2dfaffd640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 0x7f2de806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.121+0000 7f2dfaffd640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 0x7f2de806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2dec004770 tx=0x7f2dec008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.124+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dfc0c6650 con 0x7f2dfc0c7530 2023-10-21T16:21:25.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.318+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2dfc0c6650 con 0x7f2dfc0c7530 2023-10-21T16:21:25.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.319+0000 7f2de37fe640 1 -- 172.21.15.53:0/2061534537 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2dfc0c6650 con 0x7f2dfc0c7530 2023-10-21T16:21:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:25.321 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:21:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:21:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:21:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:25.322 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:21:25.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 msgr2=0x7f2de806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 0x7f2de806d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2dec004770 tx=0x7f2dec008040 comp rx=0 tx=0).stop 2023-10-21T16:21:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 msgr2=0x7f2dfc0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2df4002230 tx=0x7f2df4009750 comp rx=0 tx=0).stop 2023-10-21T16:21:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 shutdown_connections 2023-10-21T16:21:25.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f2de806afc0 0x7f2de806d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2dfc0c7530 0x7f2dfc0ce5b0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2dfc0c23a0 0x7f2dfc0c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 --2- 172.21.15.53:0/2061534537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de4004d00 0x7f2dfc155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 >> 172.21.15.53:0/2061534537 conn(0x7f2dfc0cb0c0 msgr2=0x7f2dfc1594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 shutdown_connections 2023-10-21T16:21:25.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.322+0000 7f2dfbfff640 1 -- 172.21.15.53:0/2061534537 wait complete. 2023-10-21T16:21:25.340 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:21:25.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:21:25.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.571+0000 7effbd10a640 1 Processor -- start 2023-10-21T16:21:25.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.571+0000 7effbd10a640 1 -- start start 2023-10-21T16:21:25.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.572+0000 7effbd10a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effb80c9390 0x7effb8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.572+0000 7effbd10a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb80b0260 con 0x7effb8160c90 2023-10-21T16:21:25.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.572+0000 7effbd10a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb81612d0 con 0x7effb80c8fb0 2023-10-21T16:21:25.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.572+0000 7effbd10a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb8161590 con 0x7effb80c9390 2023-10-21T16:21:25.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.573+0000 7effbc909640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7effb8160c90 0x7effb815c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37324/0 (socket says 172.21.15.53:37324) 2023-10-21T16:21:25.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.573+0000 7effbc909640 1 -- 172.21.15.53:0/1820083082 learned_addr learned my addr 172.21.15.53:0/1820083082 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:25.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.573+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980159189 0 0) 0x7effb80b0260 con 0x7effb8160c90 2023-10-21T16:21:25.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.573+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003520 con 0x7effb8160c90 2023-10-21T16:21:25.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.573+0000 7effaffff640 1 --2- 172.21.15.53:0/1820083082 >> v2:172.21.15.124:3300/0 conn(0x7effb80c9390 0x7effb8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2609439909 0 0) 0x7effa4003520 con 0x7effb8160c90 2023-10-21T16:21:25.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 >> v2:172.21.15.124:3300/0 conn(0x7effb80c9390 msgr2=0x7effb8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 --2- 172.21.15.53:0/1820083082 >> v2:172.21.15.124:3300/0 conn(0x7effb80c9390 0x7effb8160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 >> v1:172.21.15.124:6789/0 conn(0x7effb80c8fb0 legacy=0x7effb80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80704b0 con 0x7effb8160c90 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7effb00019f0 con 0x7effb8160c90 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 >> v1:172.21.15.53:6789/0 conn(0x7effb8160c90 legacy=0x7effb815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa4003970 0x7effa4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 0x7effa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effa40051d0 0x7effa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb8161590 con 0x7effa40047c0 2023-10-21T16:21:25.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa400c0e0 con 0x7effa4003970 2023-10-21T16:21:25.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa400c2a0 con 0x7effa40051d0 2023-10-21T16:21:25.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effaffff640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 0x7effa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effb7fff640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa4003970 0x7effa4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effbc909640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effa40051d0 0x7effa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.574+0000 7effaffff640 1 -- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effa40051d0 msgr2=0x7effa400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.575+0000 7effaffff640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effa40051d0 0x7effa400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.575+0000 7effaffff640 1 -- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa4003970 msgr2=0x7effa4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:25.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.575+0000 7effaffff640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa4003970 0x7effa4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.575+0000 7effaffff640 1 -- 172.21.15.53:0/1820083082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4003520 con 0x7effa40047c0 2023-10-21T16:21:25.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.575+0000 7effbc909640 1 -- 172.21.15.53:0/1820083082 reap_dead start 2023-10-21T16:21:25.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.576+0000 7effaffff640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 0x7effa4004c40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7effa000ba30 tx=0x7effa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8f0461fa99408d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.576+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0032630 con 0x7effa40047c0 2023-10-21T16:21:25.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.576+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0032790 con 0x7effa40047c0 2023-10-21T16:21:25.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.576+0000 7effb77fe640 1 -- 172.21.15.53:0/1820083082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0032b20 con 0x7effa40047c0 2023-10-21T16:21:25.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 -- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 msgr2=0x7effa4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 0x7effa4004c40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7effa000ba30 tx=0x7effa40057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 -- 172.21.15.53:0/1820083082 shutdown_connections 2023-10-21T16:21:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 --2- 172.21.15.53:0/1820083082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effa40047c0 0x7effa4004c40 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 -- 172.21.15.53:0/1820083082 >> 172.21.15.53:0/1820083082 conn(0x7effb806c010 msgr2=0x7effb806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 -- 172.21.15.53:0/1820083082 shutdown_connections 2023-10-21T16:21:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.577+0000 7effbd10a640 1 -- 172.21.15.53:0/1820083082 wait complete. 2023-10-21T16:21:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.579+0000 7effbd10a640 1 Processor -- start 2023-10-21T16:21:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.579+0000 7effbd10a640 1 -- start start 2023-10-21T16:21:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa40047c0 0x7effb80b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 0x7effb8150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81f5ec0 con 0x7effb81553b0 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80b3ec0 con 0x7effa40047c0 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbd10a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb8151a10 con 0x7effb8151450 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49610/0 (socket says 172.21.15.53:49610) 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 -- 172.21.15.53:0/2233996877 learned_addr learned my addr 172.21.15.53:0/2233996877 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effaffff640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 0x7effb8150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 -- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 msgr2=0x7effb8150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 0x7effb8150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 -- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa40047c0 msgr2=0x7effb80b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa40047c0 0x7effb80b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effbc909640 1 -- 172.21.15.53:0/2233996877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8151cf0 con 0x7effb81553b0 2023-10-21T16:21:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.580+0000 7effaffff640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 0x7effb8150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effbc909640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7effb0007c40 tx=0x7effb000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0004d60 con 0x7effb81553b0 2023-10-21T16:21:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb000bb80 con 0x7effb81553b0 2023-10-21T16:21:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb8151fd0 con 0x7effb81553b0 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0007070 con 0x7effb81553b0 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.581+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb8150d90 con 0x7effb81553b0 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.582+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb8074150 con 0x7effb81553b0 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.583+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7effb00071d0 con 0x7effb81553b0 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.584+0000 7effb57fa640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 0x7eff9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.584+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7effb0089a70 con 0x7effb81553b0 2023-10-21T16:21:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.585+0000 7effb7fff640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 0x7eff9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.586+0000 7effb7fff640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 0x7eff9806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effa8007fd0 tx=0x7effa8009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.588+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effb0054ab0 con 0x7effb81553b0 2023-10-21T16:21:25.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:25.780+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7effb80c9a00 con 0x7effb81553b0 2023-10-21T16:21:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.396+0000 7effb57fa640 1 -- 172.21.15.53:0/2233996877 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v145) v1 ==== 138+0+0 (secure 0 0 0) 0x7effb00580d0 con 0x7effb81553b0 2023-10-21T16:21:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:21:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 msgr2=0x7eff9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 0x7eff9806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effa8007fd0 tx=0x7effa8009040 comp rx=0 tx=0).stop 2023-10-21T16:21:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 msgr2=0x7effb8155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7effb0007c40 tx=0x7effb000b4a0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 shutdown_connections 2023-10-21T16:21:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7eff9806af90 0x7eff9806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effb81553b0 0x7effb8155830 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effb8151450 0x7effb8150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 --2- 172.21.15.53:0/2233996877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effa40047c0 0x7effb80b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.400+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 >> 172.21.15.53:0/2233996877 conn(0x7effb806c010 msgr2=0x7effb80afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.401+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 shutdown_connections 2023-10-21T16:21:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.401+0000 7effbd10a640 1 -- 172.21.15.53:0/2233996877 wait complete. 2023-10-21T16:21:26.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:21:26.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.636+0000 7fd621e95640 1 Processor -- start 2023-10-21T16:21:26.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.636+0000 7fd621e95640 1 -- start start 2023-10-21T16:21:26.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621e95640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd61c160090 0x7fd61c0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621e95640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0d2700 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621e95640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0d0690 con 0x7fd61c0cdc20 2023-10-21T16:21:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621e95640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0d1b60 con 0x7fd61c160090 2023-10-21T16:21:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621694640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd61c0cf3a0 0x7fd61c0b5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37328/0 (socket says 172.21.15.53:37328) 2023-10-21T16:21:26.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd621694640 1 -- 172.21.15.53:0/3966746397 learned_addr learned my addr 172.21.15.53:0/3966746397 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.638+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787880008 0 0) 0x7fd61c0d2700 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd60c003520 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd620e93640 1 --2- 172.21.15.53:0/3966746397 >> v2:172.21.15.124:3300/0 conn(0x7fd61c160090 0x7fd61c0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477529247 0 0) 0x7fd60c003520 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 >> v2:172.21.15.124:3300/0 conn(0x7fd61c160090 msgr2=0x7fd61c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 --2- 172.21.15.53:0/3966746397 >> v2:172.21.15.124:3300/0 conn(0x7fd61c160090 0x7fd61c0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 >> v1:172.21.15.124:6789/0 conn(0x7fd61c0cdc20 legacy=0x7fd61c0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0d0970 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd6140019f0 con 0x7fd61c0cf3a0 2023-10-21T16:21:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 >> v1:172.21.15.53:6789/0 conn(0x7fd61c0cf3a0 legacy=0x7fd61c0b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c003970 0x7fd60c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 0x7fd60c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd60c0052a0 0x7fd60c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0d1b60 con 0x7fd60c0047c0 2023-10-21T16:21:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c00c170 con 0x7fd60c003970 2023-10-21T16:21:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c00c330 con 0x7fd60c0052a0 2023-10-21T16:21:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.639+0000 7fd61bfff640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 0x7fd60c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd620e93640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c003970 0x7fd60c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd61bfff640 1 -- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd60c0052a0 msgr2=0x7fd60c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd61bfff640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd60c0052a0 0x7fd60c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd61bfff640 1 -- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c003970 msgr2=0x7fd60c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd61bfff640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c003970 0x7fd60c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd61bfff640 1 -- 172.21.15.53:0/3966746397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c003520 con 0x7fd60c0047c0 2023-10-21T16:21:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.640+0000 7fd621694640 1 -- 172.21.15.53:0/3966746397 reap_dead start 2023-10-21T16:21:26.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.641+0000 7fd61bfff640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 0x7fd60c004c40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fd608001720 tx=0x7fd60c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a352de5b9dbfcdab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.641+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080052d0 con 0x7fd60c0047c0 2023-10-21T16:21:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.641+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60802e050 con 0x7fd60c0047c0 2023-10-21T16:21:26.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.641+0000 7fd61b7fe640 1 -- 172.21.15.53:0/3966746397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608003830 con 0x7fd60c0047c0 2023-10-21T16:21:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 -- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 msgr2=0x7fd60c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 0x7fd60c004c40 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fd608001720 tx=0x7fd60c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 -- 172.21.15.53:0/3966746397 shutdown_connections 2023-10-21T16:21:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 --2- 172.21.15.53:0/3966746397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd60c0047c0 0x7fd60c004c40 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 -- 172.21.15.53:0/3966746397 >> 172.21.15.53:0/3966746397 conn(0x7fd61c06af50 msgr2=0x7fd61c06b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 -- 172.21.15.53:0/3966746397 shutdown_connections 2023-10-21T16:21:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.642+0000 7fd621e95640 1 -- 172.21.15.53:0/3966746397 wait complete. 2023-10-21T16:21:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.644+0000 7fd621e95640 1 Processor -- start 2023-10-21T16:21:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.644+0000 7fd621e95640 1 -- start start 2023-10-21T16:21:26.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 0x7fd61c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd61c0b6f40 0x7fd61c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0b4aa0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c154760 con 0x7fd60c0047c0 2023-10-21T16:21:26.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621e95640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c1f62d0 con 0x7fd61c0b6f40 2023-10-21T16:21:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49628/0 (socket says 172.21.15.53:49628) 2023-10-21T16:21:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 -- 172.21.15.53:0/2848271153 learned_addr learned my addr 172.21.15.53:0/2848271153 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd61bfff640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd61c0b6f40 0x7fd61c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd620e93640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 0x7fd61c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 -- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd61c0b6f40 msgr2=0x7fd61c1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd61c0b6f40 0x7fd61c1f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 -- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 msgr2=0x7fd61c0b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.646+0000 7fd621694640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 0x7fd61c0b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd621694640 1 -- 172.21.15.53:0/2848271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c1f65b0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd620e93640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 0x7fd61c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd621694640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0ccf70 tx=0x7fd61400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400a610 con 0x7fd61c0b73c0 2023-10-21T16:21:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400a790 con 0x7fd61c0b73c0 2023-10-21T16:21:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c0b43e0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400ab40 con 0x7fd61c0b73c0 2023-10-21T16:21:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.647+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c150870 con 0x7fd61c0b73c0 2023-10-21T16:21:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.648+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c0b4be0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.650+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd61400aca0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.651+0000 7fd6197fa640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 0x7fd60006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.651+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd614089240 con 0x7fd61c0b73c0 2023-10-21T16:21:26.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.652+0000 7fd620e93640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 0x7fd60006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:26.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.653+0000 7fd620e93640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 0x7fd60006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd610006fd0 tx=0x7fd610008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:26.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.655+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6140542a0 con 0x7fd61c0b73c0 2023-10-21T16:21:26.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:26.840+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd61c0b4be0 con 0x7fd61c0b73c0 2023-10-21T16:21:27.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.402+0000 7fd6197fa640 1 -- 172.21.15.53:0/2848271153 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v146) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd61c0b4be0 con 0x7fd61c0b73c0 2023-10-21T16:21:27.405 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:21:27.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.406+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 msgr2=0x7fd60006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.406+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 0x7fd60006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd610006fd0 tx=0x7fd610008040 comp rx=0 tx=0).stop 2023-10-21T16:21:27.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.406+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 msgr2=0x7fd61c1f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.406+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0ccf70 tx=0x7fd61400ce80 comp rx=0 tx=0).stop 2023-10-21T16:21:27.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 shutdown_connections 2023-10-21T16:21:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fd60006af90 0x7fd60006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd61c0b73c0 0x7fd61c1f5c00 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd61c0b6f40 0x7fd61c1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 --2- 172.21.15.53:0/2848271153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd60c0047c0 0x7fd61c0b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.408+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 >> 172.21.15.53:0/2848271153 conn(0x7fd61c06af50 msgr2=0x7fd61c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:27.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.409+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 shutdown_connections 2023-10-21T16:21:27.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.410+0000 7fd621e95640 1 -- 172.21.15.53:0/2848271153 wait complete. 2023-10-21T16:21:27.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:21:27.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.636+0000 7f223481e640 1 Processor -- start 2023-10-21T16:21:27.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.637+0000 7f223481e640 1 -- start start 2023-10-21T16:21:27.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.638+0000 7f223481e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22300c8b40 0x7f22300c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.638+0000 7f223481e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.638+0000 7f223481e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300c94f0 con 0x7f223015cae0 2023-10-21T16:21:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.638+0000 7f223481e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300ccbc0 con 0x7f22300c8b40 2023-10-21T16:21:27.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.638+0000 7f223481e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b0940 con 0x7f22300aff70 2023-10-21T16:21:27.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.639+0000 7f222f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.639+0000 7f222f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:34846/0 (socket says 172.21.15.53:34846) 2023-10-21T16:21:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.639+0000 7f222f59e640 1 -- 172.21.15.53:0/1089749840 learned_addr learned my addr 172.21.15.53:0/1089749840 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.639+0000 7f222e59c640 1 --2- 172.21.15.53:0/1089749840 >> v2:172.21.15.124:3300/0 conn(0x7f22300c8b40 0x7f22300c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.640+0000 7f222f59e640 1 -- 172.21.15.53:0/1089749840 >> v2:172.21.15.124:3300/0 conn(0x7f22300c8b40 msgr2=0x7f22300c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:27.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.640+0000 7f222f59e640 1 --2- 172.21.15.53:0/1089749840 >> v2:172.21.15.124:3300/0 conn(0x7f22300c8b40 0x7f22300c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.640+0000 7f222f59e640 1 -- 172.21.15.53:0/1089749840 >> v1:172.21.15.124:6789/0 conn(0x7f223015cae0 legacy=0x7f22300cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.640+0000 7f222f59e640 1 -- 172.21.15.53:0/1089749840 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300ce1f0 con 0x7f22300aff70 2023-10-21T16:21:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.641+0000 7f222f59e640 1 --2- 172.21.15.53:0/1089749840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2228009e50 tx=0x7f222802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50fcf39f59ca6447 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:27.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f222dd9b640 1 -- 172.21.15.53:0/1089749840 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222802cb00 con 0x7f22300aff70 2023-10-21T16:21:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f222dd9b640 1 -- 172.21.15.53:0/1089749840 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222802cc60 con 0x7f22300aff70 2023-10-21T16:21:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f222dd9b640 1 -- 172.21.15.53:0/1089749840 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22280343f0 con 0x7f22300aff70 2023-10-21T16:21:27.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 -- 172.21.15.53:0/1089749840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 msgr2=0x7f22300b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 --2- 172.21.15.53:0/1089749840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2228009e50 tx=0x7f222802bf60 comp rx=0 tx=0).stop 2023-10-21T16:21:27.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 -- 172.21.15.53:0/1089749840 shutdown_connections 2023-10-21T16:21:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 --2- 172.21.15.53:0/1089749840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22300aff70 0x7f22300b0400 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 --2- 172.21.15.53:0/1089749840 >> v2:172.21.15.124:3300/0 conn(0x7f22300c8b40 0x7f22300c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.642+0000 7f223481e640 1 -- 172.21.15.53:0/1089749840 >> 172.21.15.53:0/1089749840 conn(0x7f223006c050 msgr2=0x7f223006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.643+0000 7f223481e640 1 -- 172.21.15.53:0/1089749840 shutdown_connections 2023-10-21T16:21:27.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.643+0000 7f223481e640 1 -- 172.21.15.53:0/1089749840 wait complete. 2023-10-21T16:21:27.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.644+0000 7f223481e640 1 Processor -- start 2023-10-21T16:21:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.645+0000 7f223481e640 1 -- start start 2023-10-21T16:21:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.645+0000 7f223481e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 0x7f22300c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f223481e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f223481e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f223015cae0 0x7f2230158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f223481e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2230159e30 con 0x7f22300c8b40 2023-10-21T16:21:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f223481e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2230159220 con 0x7f22300aff70 2023-10-21T16:21:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f223481e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2230159500 con 0x7f223015cae0 2023-10-21T16:21:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49638/0 (socket says 172.21.15.53:49638) 2023-10-21T16:21:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 -- 172.21.15.53:0/401921536 learned_addr learned my addr 172.21.15.53:0/401921536 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222f59e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f223015cae0 0x7f2230158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222ed9d640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 0x7f22300c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 -- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f223015cae0 msgr2=0x7f2230158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f223015cae0 0x7f2230158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 -- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 msgr2=0x7f22300c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:27.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 0x7f22300c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:27.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222e59c640 1 -- 172.21.15.53:0/401921536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228009b30 con 0x7f22300c8b40 2023-10-21T16:21:27.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.646+0000 7f222ed9d640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 0x7f22300c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:27.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f222e59c640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f222400d800 tx=0x7f222400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:27.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400b6c0 con 0x7f22300c8b40 2023-10-21T16:21:27.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222400b820 con 0x7f22300c8b40 2023-10-21T16:21:27.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400bbb0 con 0x7f22300c8b40 2023-10-21T16:21:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22301597e0 con 0x7f22300c8b40 2023-10-21T16:21:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.647+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22301576c0 con 0x7f22300c8b40 2023-10-21T16:21:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.648+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230074150 con 0x7f22300c8b40 2023-10-21T16:21:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.650+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f222400bd10 con 0x7f22300c8b40 2023-10-21T16:21:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.651+0000 7f22177fe640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 0x7f221c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:27.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.651+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f2224089e20 con 0x7f22300c8b40 2023-10-21T16:21:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.651+0000 7f222ed9d640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 0x7f221c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.653+0000 7f222ed9d640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 0x7f221c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2220000f60 tx=0x7f2220009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:27.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.656+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2224054c90 con 0x7f22300c8b40 2023-10-21T16:21:27.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:27.889+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f223015c210 con 0x7f22300c8b40 2023-10-21T16:21:28.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.448+0000 7f22177fe640 1 -- 172.21.15.53:0/401921536 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v65) v1 ==== 174+0+0 (secure 0 0 0) 0x7f22240582b0 con 0x7f22300c8b40 2023-10-21T16:21:28.453 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 17 and data pool 18 2023-10-21T16:21:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.456+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 msgr2=0x7f221c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.456+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 0x7f221c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2220000f60 tx=0x7f2220009040 comp rx=0 tx=0).stop 2023-10-21T16:21:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.456+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 msgr2=0x7f2230157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.456+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f222400d800 tx=0x7f222400dcc0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 shutdown_connections 2023-10-21T16:21:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f221c06af90 0x7f221c06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f223015cae0 0x7f2230158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22300c8b40 0x7f2230157180 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 --2- 172.21.15.53:0/401921536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22300aff70 0x7f22300c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 >> 172.21.15.53:0/401921536 conn(0x7f223006c050 msgr2=0x7f2230070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:28.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.457+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 shutdown_connections 2023-10-21T16:21:28.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.459+0000 7f223481e640 1 -- 172.21.15.53:0/401921536 wait complete. 2023-10-21T16:21:28.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:28.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.672+0000 7f348ccdf640 1 Processor -- start 2023-10-21T16:21:28.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.673+0000 7f348ccdf640 1 -- start start 2023-10-21T16:21:28.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f348ccdf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34880adc70 0x7f34880b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:28.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f348ccdf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348815f7d0 con 0x7f348815ab30 2023-10-21T16:21:28.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f348ccdf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348815efb0 con 0x7f348815af10 2023-10-21T16:21:28.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f348ccdf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348815f210 con 0x7f34880adc70 2023-10-21T16:21:28.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f3486ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f34880adc70 0x7f34880b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:28.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f34877fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f348815ab30 0x7f348815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37344/0 (socket says 172.21.15.53:37344) 2023-10-21T16:21:28.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.674+0000 7f34877fe640 1 -- 172.21.15.53:0/609120509 learned_addr learned my addr 172.21.15.53:0/609120509 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.675+0000 7f3486ffd640 1 -- 172.21.15.53:0/609120509 >> v1:172.21.15.124:6789/0 conn(0x7f348815af10 legacy=0x7f348806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.675+0000 7f3486ffd640 1 -- 172.21.15.53:0/609120509 >> v1:172.21.15.53:6789/0 conn(0x7f348815ab30 legacy=0x7f348815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.675+0000 7f3486ffd640 1 -- 172.21.15.53:0/609120509 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348815f4f0 con 0x7f34880adc70 2023-10-21T16:21:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.676+0000 7f3486ffd640 1 --2- 172.21.15.53:0/609120509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34880adc70 0x7f34880b00c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f34700097a0 tx=0x7f347002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e11a2a99769d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:28.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.676+0000 7f34867fc640 1 -- 172.21.15.53:0/609120509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470031830 con 0x7f34880adc70 2023-10-21T16:21:28.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.676+0000 7f34867fc640 1 -- 172.21.15.53:0/609120509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470031990 con 0x7f34880adc70 2023-10-21T16:21:28.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.676+0000 7f34867fc640 1 -- 172.21.15.53:0/609120509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470031d20 con 0x7f34880adc70 2023-10-21T16:21:28.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 -- 172.21.15.53:0/609120509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34880adc70 msgr2=0x7f34880b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 --2- 172.21.15.53:0/609120509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34880adc70 0x7f34880b00c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f34700097a0 tx=0x7f347002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 -- 172.21.15.53:0/609120509 shutdown_connections 2023-10-21T16:21:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 --2- 172.21.15.53:0/609120509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34880adc70 0x7f34880b00c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 -- 172.21.15.53:0/609120509 >> 172.21.15.53:0/609120509 conn(0x7f34880cb0c0 msgr2=0x7f34880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:28.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 -- 172.21.15.53:0/609120509 shutdown_connections 2023-10-21T16:21:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.677+0000 7f348ccdf640 1 -- 172.21.15.53:0/609120509 wait complete. 2023-10-21T16:21:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.679+0000 7f348ccdf640 1 Processor -- start 2023-10-21T16:21:28.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.679+0000 7f348ccdf640 1 -- start start 2023-10-21T16:21:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34880adc70 0x7f34880cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f348815ab30 0x7f34880c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:28.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880c64e0 con 0x7f348815af10 2023-10-21T16:21:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880cc440 con 0x7f348815ab30 2023-10-21T16:21:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f348ccdf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880c5c40 con 0x7f34880adc70 2023-10-21T16:21:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f3487fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:28.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f3487fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49650/0 (socket says 172.21.15.53:49650) 2023-10-21T16:21:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.680+0000 7f3487fff640 1 -- 172.21.15.53:0/1365109083 learned_addr learned my addr 172.21.15.53:0/1365109083 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f34877fe640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34880adc70 0x7f34880cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 -- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34880adc70 msgr2=0x7f34880cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3486ffd640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f348815ab30 0x7f34880c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:28.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34880adc70 0x7f34880cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 -- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f348815ab30 msgr2=0x7f34880c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f348815ab30 0x7f34880c4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 -- 172.21.15.53:0/1365109083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009400 con 0x7f348815af10 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f3487fff640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f348000b440 tx=0x7f348000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480011830 con 0x7f348815af10 2023-10-21T16:21:28.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480011990 con 0x7f348815af10 2023-10-21T16:21:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480011d40 con 0x7f348815af10 2023-10-21T16:21:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34880c5f20 con 0x7f348815af10 2023-10-21T16:21:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.681+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34880c4690 con 0x7f348815af10 2023-10-21T16:21:28.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.682+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34880cc210 con 0x7f348815af10 2023-10-21T16:21:28.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.683+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f348001a400 con 0x7f348815af10 2023-10-21T16:21:28.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.684+0000 7f347ffff640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 0x7f346806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:28.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.684+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3480089610 con 0x7f348815af10 2023-10-21T16:21:28.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.685+0000 7f34877fe640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 0x7f346806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:28.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.686+0000 7f34877fe640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 0x7f346806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34780012e0 tx=0x7f3478009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:28.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.689+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34880cc210 con 0x7f348815af10 2023-10-21T16:21:28.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.880+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34880cc210 con 0x7f348815af10 2023-10-21T16:21:28.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.882+0000 7f347ffff640 1 -- 172.21.15.53:0/1365109083 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f34880cc210 con 0x7f348815af10 2023-10-21T16:21:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:28.884 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":147,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:27.891681+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:28.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 msgr2=0x7f346806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 0x7f346806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34780012e0 tx=0x7f3478009040 comp rx=0 tx=0).stop 2023-10-21T16:21:28.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 msgr2=0x7f34881427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:28.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f348000b440 tx=0x7f348000b900 comp rx=0 tx=0).stop 2023-10-21T16:21:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 shutdown_connections 2023-10-21T16:21:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f346806af90 0x7f346806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f348815af10 0x7f34881427a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f348815ab30 0x7f34880c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 --2- 172.21.15.53:0/1365109083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34880adc70 0x7f34880cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 >> 172.21.15.53:0/1365109083 conn(0x7f34880cb0c0 msgr2=0x7f34880701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 shutdown_connections 2023-10-21T16:21:28.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:28.885+0000 7f348ccdf640 1 -- 172.21.15.53:0/1365109083 wait complete. 2023-10-21T16:21:28.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.130+0000 7f3288adf640 1 Processor -- start 2023-10-21T16:21:29.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.130+0000 7f3288adf640 1 -- start start 2023-10-21T16:21:29.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3288adf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f328415ab30 0x7f328415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3288adf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328415f7d0 con 0x7f32840adc70 2023-10-21T16:21:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3288adf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328415f210 con 0x7f32840b07d0 2023-10-21T16:21:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3288adf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328415f4f0 con 0x7f328415ab30 2023-10-21T16:21:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3282ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32840adc70 0x7f32840b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37348/0 (socket says 172.21.15.53:37348) 2023-10-21T16:21:29.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.132+0000 7f3282ffd640 1 -- 172.21.15.53:0/1186731496 learned_addr learned my addr 172.21.15.53:0/1186731496 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:29.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110845262 0 0) 0x7f328415f7d0 con 0x7f32840adc70 2023-10-21T16:21:29.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3270003520 con 0x7f32840adc70 2023-10-21T16:21:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32837fe640 1 --2- 172.21.15.53:0/1186731496 >> v2:172.21.15.124:3300/0 conn(0x7f328415ab30 0x7f328415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842019753 0 0) 0x7f3270003520 con 0x7f32840adc70 2023-10-21T16:21:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 >> v2:172.21.15.124:3300/0 conn(0x7f328415ab30 msgr2=0x7f328415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 --2- 172.21.15.53:0/1186731496 >> v2:172.21.15.124:3300/0 conn(0x7f328415ab30 0x7f328415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 >> v1:172.21.15.124:6789/0 conn(0x7f32840b07d0 legacy=0x7f328406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328406c550 con 0x7f32840adc70 2023-10-21T16:21:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f326c001670 con 0x7f32840adc70 2023-10-21T16:21:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 >> v1:172.21.15.53:6789/0 conn(0x7f32840adc70 legacy=0x7f32840b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700039c0 0x7f3270003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 0x7f3270004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32700051d0 0x7f327000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328415f4f0 con 0x7f32700047c0 2023-10-21T16:21:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327000c120 con 0x7f32700051d0 2023-10-21T16:21:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.133+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327000c2e0 con 0x7f32700039c0 2023-10-21T16:21:29.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 0x7f3270004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f32837fe640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700039c0 0x7f3270003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 -- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700039c0 msgr2=0x7f3270003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700039c0 0x7f3270003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 -- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32700051d0 msgr2=0x7f327000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32700051d0 0x7f327000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3282ffd640 1 -- 172.21.15.53:0/1186731496 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270003520 con 0x7f32700047c0 2023-10-21T16:21:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.134+0000 7f3283fff640 1 -- 172.21.15.53:0/1186731496 reap_dead start 2023-10-21T16:21:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.135+0000 7f3282ffd640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 0x7f3270004c40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c0e0 tx=0x7f32700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55142352e5e70cdc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c005ad0 con 0x7f32700047c0 2023-10-21T16:21:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c005c30 con 0x7f32700047c0 2023-10-21T16:21:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f32827fc640 1 -- 172.21.15.53:0/1186731496 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c02ca50 con 0x7f32700047c0 2023-10-21T16:21:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 -- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 msgr2=0x7f3270004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 0x7f3270004c40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c0e0 tx=0x7f32700057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 -- 172.21.15.53:0/1186731496 shutdown_connections 2023-10-21T16:21:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 --2- 172.21.15.53:0/1186731496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32700047c0 0x7f3270004c40 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 -- 172.21.15.53:0/1186731496 >> 172.21.15.53:0/1186731496 conn(0x7f32840cb0c0 msgr2=0x7f32840cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 -- 172.21.15.53:0/1186731496 shutdown_connections 2023-10-21T16:21:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.136+0000 7f3288adf640 1 -- 172.21.15.53:0/1186731496 wait complete. 2023-10-21T16:21:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.138+0000 7f3288adf640 1 Processor -- start 2023-10-21T16:21:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.138+0000 7f3288adf640 1 -- start start 2023-10-21T16:21:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 0x7f328415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32840acb90 0x7f32840c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b1bc0 con 0x7f32840c6440 2023-10-21T16:21:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b1ea0 con 0x7f32840acb90 2023-10-21T16:21:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.139+0000 7f3288adf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b2180 con 0x7f32700047c0 2023-10-21T16:21:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49670/0 (socket says 172.21.15.53:49670) 2023-10-21T16:21:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 -- 172.21.15.53:0/273284090 learned_addr learned my addr 172.21.15.53:0/273284090 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f32837fe640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 0x7f328415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3282ffd640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32840acb90 0x7f32840c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 -- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 msgr2=0x7f328415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 0x7f328415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 -- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32840acb90 msgr2=0x7f32840c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32840acb90 0x7f32840c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 -- 172.21.15.53:0/273284090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840b2460 con 0x7f32840c6440 2023-10-21T16:21:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f32837fe640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 0x7f328415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.140+0000 7f3283fff640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f32840cf710 tx=0x7f327c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.141+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c007070 con 0x7f32840c6440 2023-10-21T16:21:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.141+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c0071f0 con 0x7f32840c6440 2023-10-21T16:21:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.141+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0075a0 con 0x7f32840c6440 2023-10-21T16:21:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.141+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840b2740 con 0x7f32840c6440 2023-10-21T16:21:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.141+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32840ad010 con 0x7f32840c6440 2023-10-21T16:21:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.142+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32840ad290 con 0x7f32840c6440 2023-10-21T16:21:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.143+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f327c007700 con 0x7f32840c6440 2023-10-21T16:21:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.144+0000 7f327bfff640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 0x7f326406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.144+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f327c088df0 con 0x7f32840c6440 2023-10-21T16:21:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.145+0000 7f32837fe640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 0x7f326406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.146+0000 7f32837fe640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 0x7f326406d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f328404f940 tx=0x7f3274008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.148+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f327c053b60 con 0x7f32840c6440 2023-10-21T16:21:29.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.342+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f328404e160 con 0x7f32840c6440 2023-10-21T16:21:29.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.345+0000 7f327bfff640 1 -- 172.21.15.53:0/273284090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f327c017070 con 0x7f32840c6440 2023-10-21T16:21:29.347 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:29.347 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":147,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:27.891681+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:29.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 msgr2=0x7f326406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 0x7f326406d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f328404f940 tx=0x7f3274008040 comp rx=0 tx=0).stop 2023-10-21T16:21:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 msgr2=0x7f32841570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f32840cf710 tx=0x7f327c00c980 comp rx=0 tx=0).stop 2023-10-21T16:21:29.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 shutdown_connections 2023-10-21T16:21:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f326406afc0 0x7f326406d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32840c6440 0x7f32841570d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32840acb90 0x7f32840c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 --2- 172.21.15.53:0/273284090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32700047c0 0x7f328415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 >> 172.21.15.53:0/273284090 conn(0x7f32840cb0c0 msgr2=0x7f32840aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:29.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 shutdown_connections 2023-10-21T16:21:29.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.348+0000 7f3288adf640 1 -- 172.21.15.53:0/273284090 wait complete. 2023-10-21T16:21:29.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:21:29.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.597+0000 7f5d04226640 1 Processor -- start 2023-10-21T16:21:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.598+0000 7f5d04226640 1 -- start start 2023-10-21T16:21:29.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.599+0000 7f5d04226640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5cfc0c9cf0 0x7f5cfc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.599+0000 7f5d04226640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc15b310 con 0x7f5cfc15aa00 2023-10-21T16:21:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.599+0000 7f5d04226640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc06d290 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.599+0000 7f5d04226640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc06e0a0 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d017c1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5cfc0c9cf0 0x7f5cfc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d027c3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5cfc15aa00 0x7f5cfc0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37356/0 (socket says 172.21.15.53:37356) 2023-10-21T16:21:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d027c3640 1 -- 172.21.15.53:0/394308132 learned_addr learned my addr 172.21.15.53:0/394308132 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:29.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d017c1640 1 -- 172.21.15.53:0/394308132 >> v1:172.21.15.124:6789/0 conn(0x7f5cfc06c3a0 legacy=0x7f5cfc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:29.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d017c1640 1 -- 172.21.15.53:0/394308132 >> v1:172.21.15.53:6789/0 conn(0x7f5cfc15aa00 legacy=0x7f5cfc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.600+0000 7f5d017c1640 1 -- 172.21.15.53:0/394308132 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0c9860 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.601+0000 7f5d017c1640 1 --2- 172.21.15.53:0/394308132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc06db60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00097a0 tx=0x7f5cf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b56590cd6892b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.602+0000 7f5d00fc0640 1 -- 172.21.15.53:0/394308132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0031830 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.602+0000 7f5d00fc0640 1 -- 172.21.15.53:0/394308132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf0031990 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.602+0000 7f5d00fc0640 1 -- 172.21.15.53:0/394308132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0031d20 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.602+0000 7f5d04226640 1 -- 172.21.15.53:0/394308132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc0c9cf0 msgr2=0x7f5cfc06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.602+0000 7f5d04226640 1 --2- 172.21.15.53:0/394308132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc06db60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00097a0 tx=0x7f5cf002b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.603+0000 7f5d04226640 1 -- 172.21.15.53:0/394308132 shutdown_connections 2023-10-21T16:21:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.603+0000 7f5d04226640 1 --2- 172.21.15.53:0/394308132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc06db60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.603+0000 7f5d04226640 1 -- 172.21.15.53:0/394308132 >> 172.21.15.53:0/394308132 conn(0x7f5cfc06a5d0 msgr2=0x7f5cfc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.603+0000 7f5d04226640 1 -- 172.21.15.53:0/394308132 shutdown_connections 2023-10-21T16:21:29.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.603+0000 7f5d04226640 1 -- 172.21.15.53:0/394308132 wait complete. 2023-10-21T16:21:29.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.604+0000 7f5d04226640 1 Processor -- start 2023-10-21T16:21:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.605+0000 7f5d04226640 1 -- start start 2023-10-21T16:21:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 0x7f5cfc1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc15d5f0 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc15d8d0 con 0x7f5cfc0c9cf0 2023-10-21T16:21:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d04226640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc15dbb0 con 0x7f5cfc15aa00 2023-10-21T16:21:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d01fc2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d01fc2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49688/0 (socket says 172.21.15.53:49688) 2023-10-21T16:21:29.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.606+0000 7f5d01fc2640 1 -- 172.21.15.53:0/3685192836 learned_addr learned my addr 172.21.15.53:0/3685192836 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d027c3640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 0x7f5cfc1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 -- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 msgr2=0x7f5cfc1f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d017c1640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 0x7f5cfc1f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 -- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cfc0c9cf0 msgr2=0x7f5cfc15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:29.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:29.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 -- 172.21.15.53:0/3685192836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009400 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d027c3640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 0x7f5cfc1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d01fc2640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c7c0 tx=0x7f5cec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00aa50 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec00abb0 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec012420 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.607+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc15de90 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.608+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc1f2300 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.609+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc153a10 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.610+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5cec012580 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.611+0000 7f5cfa7fc640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 0x7f5ce806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.611+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cec089880 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.612+0000 7f5d017c1640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 0x7f5ce806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:29.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.613+0000 7f5d017c1640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 0x7f5ce806d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0036000 tx=0x7f5cf0008030 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:29.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.615+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cec054680 con 0x7f5cfc06c3a0 2023-10-21T16:21:29.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:29.850+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5cfc04e160 con 0x7f5cfc06c3a0 2023-10-21T16:21:30.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.464+0000 7f5cfa7fc640 1 -- 172.21.15.53:0/3685192836 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5cec057ca0 con 0x7f5cfc06c3a0 2023-10-21T16:21:30.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 msgr2=0x7f5ce806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 0x7f5ce806d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0036000 tx=0x7f5cf0008030 comp rx=0 tx=0).stop 2023-10-21T16:21:30.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 msgr2=0x7f5cfc0c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00c7c0 tx=0x7f5cec00cc80 comp rx=0 tx=0).stop 2023-10-21T16:21:30.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 shutdown_connections 2023-10-21T16:21:30.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f5ce806afc0 0x7f5ce806d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cfc15aa00 0x7f5cfc1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cfc0c9cf0 0x7f5cfc15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 --2- 172.21.15.53:0/3685192836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5cfc06c3a0 0x7f5cfc0c1820 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 >> 172.21.15.53:0/3685192836 conn(0x7f5cfc06a5d0 msgr2=0x7f5cfc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:30.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 shutdown_connections 2023-10-21T16:21:30.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.468+0000 7f5d04226640 1 -- 172.21.15.53:0/3685192836 wait complete. 2023-10-21T16:21:30.490 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:21:30.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:21:30.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.720+0000 7f547330a640 1 Processor -- start 2023-10-21T16:21:30.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.721+0000 7f547330a640 1 -- start start 2023-10-21T16:21:30.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f547330a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f546c0d0b90 0x7f546c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f547330a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0d2870 con 0x7f546c15ccb0 2023-10-21T16:21:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f547330a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0d1150 con 0x7f546c0d00e0 2023-10-21T16:21:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f547330a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546c0d1430 con 0x7f546c0d0b90 2023-10-21T16:21:30.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f5472308640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f546c15ccb0 0x7f546c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37362/0 (socket says 172.21.15.53:37362) 2023-10-21T16:21:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.722+0000 7f5472308640 1 -- 172.21.15.53:0/1171674523 learned_addr learned my addr 172.21.15.53:0/1171674523 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5472b09640 1 --2- 172.21.15.53:0/1171674523 >> v2:172.21.15.124:3300/0 conn(0x7f546c0d0b90 0x7f546c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123846189 0 0) 0x7f546c0d2870 con 0x7f546c15ccb0 2023-10-21T16:21:30.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5458003520 con 0x7f546c15ccb0 2023-10-21T16:21:30.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1550210219 0 0) 0x7f5458003520 con 0x7f546c15ccb0 2023-10-21T16:21:30.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 >> v2:172.21.15.124:3300/0 conn(0x7f546c0d0b90 msgr2=0x7f546c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:30.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 --2- 172.21.15.53:0/1171674523 >> v2:172.21.15.124:3300/0 conn(0x7f546c0d0b90 0x7f546c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 >> v1:172.21.15.124:6789/0 conn(0x7f546c0d00e0 legacy=0x7f546c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0d1710 con 0x7f546c15ccb0 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5468001670 con 0x7f546c15ccb0 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 >> v1:172.21.15.53:6789/0 conn(0x7f546c15ccb0 legacy=0x7f546c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54580039f0 0x7f5458003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54580047c0 0x7f5458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 0x7f545800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0d1430 con 0x7f5458005180 2023-10-21T16:21:30.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545800c0b0 con 0x7f54580039f0 2023-10-21T16:21:30.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.723+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545800c270 con 0x7f54580047c0 2023-10-21T16:21:30.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 0x7f545800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5471b07640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54580047c0 0x7f5458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472308640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54580039f0 0x7f5458003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 -- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54580047c0 msgr2=0x7f5458004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54580047c0 0x7f5458004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 -- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54580039f0 msgr2=0x7f5458003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:30.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54580039f0 0x7f5458003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.724+0000 7f5472b09640 1 -- 172.21.15.53:0/1171674523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458003520 con 0x7f5458005180 2023-10-21T16:21:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.725+0000 7f5472b09640 1 -- 172.21.15.53:0/1171674523 reap_dead start 2023-10-21T16:21:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.725+0000 7f5472b09640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 0x7f545800b980 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fc0 tx=0x7f54580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec2f8d0c9cf1d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546402cc60 con 0x7f5458005180 2023-10-21T16:21:30.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400f040 con 0x7f5458005180 2023-10-21T16:21:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f5471306640 1 -- 172.21.15.53:0/1171674523 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464003990 con 0x7f5458005180 2023-10-21T16:21:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f547330a640 1 -- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 msgr2=0x7f545800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f547330a640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 0x7f545800b980 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fc0 tx=0x7f54580057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f547330a640 1 -- 172.21.15.53:0/1171674523 shutdown_connections 2023-10-21T16:21:30.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f547330a640 1 --2- 172.21.15.53:0/1171674523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5458005180 0x7f545800b980 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.726+0000 7f547330a640 1 -- 172.21.15.53:0/1171674523 >> 172.21.15.53:0/1171674523 conn(0x7f546c071660 msgr2=0x7f546c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.727+0000 7f547330a640 1 -- 172.21.15.53:0/1171674523 shutdown_connections 2023-10-21T16:21:30.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.727+0000 7f547330a640 1 -- 172.21.15.53:0/1171674523 wait complete. 2023-10-21T16:21:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.728+0000 7f547330a640 1 Processor -- start 2023-10-21T16:21:30.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.729+0000 7f547330a640 1 -- start start 2023-10-21T16:21:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.729+0000 7f547330a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5458005180 0x7f546c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.729+0000 7f547330a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f547330a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f546c15f470 0x7f546c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f547330a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c159670 con 0x7f546c1f2130 2023-10-21T16:21:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f547330a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c15f9e0 con 0x7f5458005180 2023-10-21T16:21:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f547330a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c15fcc0 con 0x7f546c15f470 2023-10-21T16:21:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49702/0 (socket says 172.21.15.53:49702) 2023-10-21T16:21:30.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 -- 172.21.15.53:0/1987485251 learned_addr learned my addr 172.21.15.53:0/1987485251 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:30.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5472b09640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f546c15f470 0x7f546c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5472308640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5458005180 0x7f546c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 -- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f546c15f470 msgr2=0x7f546c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f546c15f470 0x7f546c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 -- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5458005180 msgr2=0x7f546c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:30.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5458005180 0x7f546c159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:30.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.730+0000 7f5471b07640 1 -- 172.21.15.53:0/1987485251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c15d190 con 0x7f546c1f2130 2023-10-21T16:21:30.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f5471b07640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f546000e480 tx=0x7f546000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:30.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600052e0 con 0x7f546c1f2130 2023-10-21T16:21:30.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460005440 con 0x7f546c1f2130 2023-10-21T16:21:30.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460003830 con 0x7f546c1f2130 2023-10-21T16:21:30.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c15d470 con 0x7f546c1f2130 2023-10-21T16:21:30.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.731+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c0c8820 con 0x7f546c1f2130 2023-10-21T16:21:30.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.732+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c0d19d0 con 0x7f546c1f2130 2023-10-21T16:21:30.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.733+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5460003990 con 0x7f546c1f2130 2023-10-21T16:21:30.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.734+0000 7f545effd640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 0x7f545006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:30.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.734+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54600896c0 con 0x7f546c1f2130 2023-10-21T16:21:30.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.735+0000 7f5472308640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 0x7f545006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:30.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.736+0000 7f5472308640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 0x7f545006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54680016d0 tx=0x7f5468000f30 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:30.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.739+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f546c0d19d0 con 0x7f546c1f2130 2023-10-21T16:21:30.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:30.969+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f546c0d19d0 con 0x7f546c1f2130 2023-10-21T16:21:31.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.479+0000 7f545effd640 1 -- 172.21.15.53:0/1987485251 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7f546c0d19d0 con 0x7f546c1f2130 2023-10-21T16:21:31.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 msgr2=0x7f545006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 0x7f545006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f54680016d0 tx=0x7f5468000f30 comp rx=0 tx=0).stop 2023-10-21T16:21:31.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 msgr2=0x7f546c0c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f546000e480 tx=0x7f546000e940 comp rx=0 tx=0).stop 2023-10-21T16:21:31.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 shutdown_connections 2023-10-21T16:21:31.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f545006af90 0x7f545006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f546c15f470 0x7f546c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f546c1f2130 0x7f546c0c82e0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 --2- 172.21.15.53:0/1987485251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5458005180 0x7f546c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.482+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 >> 172.21.15.53:0/1987485251 conn(0x7f546c071660 msgr2=0x7f546c0d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:31.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.483+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 shutdown_connections 2023-10-21T16:21:31.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.483+0000 7f547330a640 1 -- 172.21.15.53:0/1987485251 wait complete. 2023-10-21T16:21:31.506 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:21:31.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:31.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.722+0000 7fb331e67640 1 Processor -- start 2023-10-21T16:21:31.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.722+0000 7fb331e67640 1 -- start start 2023-10-21T16:21:31.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.723+0000 7fb331e67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb331e67640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb32c0b07d0 0x7fb32c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb331e67640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c0cacf0 con 0x7fb32c0cc210 2023-10-21T16:21:31.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb331e67640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0b10f0 con 0x7fb32c0b07d0 2023-10-21T16:21:31.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb331e67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0ca3c0 con 0x7fb32c1599d0 2023-10-21T16:21:31.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb330e65640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb330e65640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:34914/0 (socket says 172.21.15.53:34914) 2023-10-21T16:21:31.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.724+0000 7fb330e65640 1 -- 172.21.15.53:0/2601305989 learned_addr learned my addr 172.21.15.53:0/2601305989 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:31.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.725+0000 7fb331666640 1 --2- 172.21.15.53:0/2601305989 >> v2:172.21.15.124:3300/0 conn(0x7fb32c0b07d0 0x7fb32c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.725+0000 7fb330e65640 1 -- 172.21.15.53:0/2601305989 >> v2:172.21.15.124:3300/0 conn(0x7fb32c0b07d0 msgr2=0x7fb32c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:31.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.725+0000 7fb330e65640 1 --2- 172.21.15.53:0/2601305989 >> v2:172.21.15.124:3300/0 conn(0x7fb32c0b07d0 0x7fb32c06c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.725+0000 7fb330e65640 1 -- 172.21.15.53:0/2601305989 >> v1:172.21.15.124:6789/0 conn(0x7fb32c0cc210 legacy=0x7fb32c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.725+0000 7fb330e65640 1 -- 172.21.15.53:0/2601305989 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0ca6a0 con 0x7fb32c1599d0 2023-10-21T16:21:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.727+0000 7fb330e65640 1 --2- 172.21.15.53:0/2601305989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb328009870 tx=0x7fb32802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2847cabe6420e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.727+0000 7fb3237fe640 1 -- 172.21.15.53:0/2601305989 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328030840 con 0x7fb32c1599d0 2023-10-21T16:21:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.727+0000 7fb3237fe640 1 -- 172.21.15.53:0/2601305989 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280309a0 con 0x7fb32c1599d0 2023-10-21T16:21:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.727+0000 7fb3237fe640 1 -- 172.21.15.53:0/2601305989 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328030d30 con 0x7fb32c1599d0 2023-10-21T16:21:31.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.727+0000 7fb331e67640 1 -- 172.21.15.53:0/2601305989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 msgr2=0x7fb32c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 --2- 172.21.15.53:0/2601305989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb328009870 tx=0x7fb32802b980 comp rx=0 tx=0).stop 2023-10-21T16:21:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 -- 172.21.15.53:0/2601305989 shutdown_connections 2023-10-21T16:21:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 --2- 172.21.15.53:0/2601305989 >> v2:172.21.15.124:3300/0 conn(0x7fb32c0b07d0 0x7fb32c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 --2- 172.21.15.53:0/2601305989 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c1599d0 0x7fb32c0ad730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 -- 172.21.15.53:0/2601305989 >> 172.21.15.53:0/2601305989 conn(0x7fb32c0cb0c0 msgr2=0x7fb32c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 -- 172.21.15.53:0/2601305989 shutdown_connections 2023-10-21T16:21:31.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.728+0000 7fb331e67640 1 -- 172.21.15.53:0/2601305989 wait complete. 2023-10-21T16:21:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.730+0000 7fb331e67640 1 Processor -- start 2023-10-21T16:21:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.730+0000 7fb331e67640 1 -- start start 2023-10-21T16:21:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 0x7fb32c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 0x7fb32c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c1503c0 con 0x7fb32c0b07d0 2023-10-21T16:21:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c154e50 con 0x7fb32c1599d0 2023-10-21T16:21:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331e67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0b20f0 con 0x7fb32c0cc210 2023-10-21T16:21:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb330e65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb330e65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49718/0 (socket says 172.21.15.53:49718) 2023-10-21T16:21:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb330e65640 1 -- 172.21.15.53:0/837865106 learned_addr learned my addr 172.21.15.53:0/837865106 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb323fff640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 0x7fb32c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb331666640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 0x7fb32c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.731+0000 7fb330e65640 1 -- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 msgr2=0x7fb32c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb330e65640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 0x7fb32c0b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb330e65640 1 -- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 msgr2=0x7fb32c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb330e65640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 0x7fb32c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb330e65640 1 -- 172.21.15.53:0/837865106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280094d0 con 0x7fb32c0b07d0 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb323fff640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 0x7fb32c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb331666640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 0x7fb32c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:31.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb330e65640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb3280090b0 tx=0x7fb3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:31.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32803e070 con 0x7fb32c0b07d0 2023-10-21T16:21:31.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328039a10 con 0x7fb32c0b07d0 2023-10-21T16:21:31.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328039de0 con 0x7fb32c0b07d0 2023-10-21T16:21:31.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c153b50 con 0x7fb32c0b07d0 2023-10-21T16:21:31.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.732+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c154110 con 0x7fb32c0b07d0 2023-10-21T16:21:31.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.733+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c04e160 con 0x7fb32c0b07d0 2023-10-21T16:21:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.734+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb3280423f0 con 0x7fb32c0b07d0 2023-10-21T16:21:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.735+0000 7fb3217fa640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 0x7fb31806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.735+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3280ab970 con 0x7fb32c0b07d0 2023-10-21T16:21:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.736+0000 7fb323fff640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 0x7fb31806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:31.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.737+0000 7fb323fff640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 0x7fb31806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb32c1584a0 tx=0x7fb31c009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:31.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.740+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb328076660 con 0x7fb32c0b07d0 2023-10-21T16:21:31.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.970+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32c0b3540 con 0x7fb32c0b07d0 2023-10-21T16:21:31.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.971+0000 7fb3217fa640 1 -- 172.21.15.53:0/837865106 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fb328048090 con 0x7fb32c0b07d0 2023-10-21T16:21:31.973 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:31.974 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:31.974 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:31.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 msgr2=0x7fb31806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 0x7fb31806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb32c1584a0 tx=0x7fb31c009040 comp rx=0 tx=0).stop 2023-10-21T16:21:31.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 msgr2=0x7fb32c154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:31.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb3280090b0 tx=0x7fb3280046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 shutdown_connections 2023-10-21T16:21:31.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fb31806af90 0x7fb31806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32c1599d0 0x7fb32c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32c0cc210 0x7fb32c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 --2- 172.21.15.53:0/837865106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32c0b07d0 0x7fb32c154910 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:31.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 >> 172.21.15.53:0/837865106 conn(0x7fb32c0cb0c0 msgr2=0x7fb32c0ad4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:31.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 shutdown_connections 2023-10-21T16:21:31.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:31.974+0000 7fb331e67640 1 -- 172.21.15.53:0/837865106 wait complete. 2023-10-21T16:21:31.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:32.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.217+0000 7ffacb59e640 1 Processor -- start 2023-10-21T16:21:32.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.217+0000 7ffacb59e640 1 -- start start 2023-10-21T16:21:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.218+0000 7ffacb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc0d16f0 0x7ffacc0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffacc0cfa50 0x7ffacc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc0cea50 con 0x7ffacc15cc60 2023-10-21T16:21:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc15d040 con 0x7ffacc0cfa50 2023-10-21T16:21:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0d0650 con 0x7ffacc0d16f0 2023-10-21T16:21:32.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffaca59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ffacc15cc60 0x7ffacc0cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37374/0 (socket says 172.21.15.53:37374) 2023-10-21T16:21:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffaca59c640 1 -- 172.21.15.53:0/3658036141 learned_addr learned my addr 172.21.15.53:0/3658036141 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:32.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacad9d640 1 --2- 172.21.15.53:0/3658036141 >> v2:172.21.15.53:3300/0 conn(0x7ffacc0cfa50 0x7ffacc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffac9d9b640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc0d16f0 0x7ffacc0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffac959a640 1 -- 172.21.15.53:0/3658036141 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372538815 0 0) 0x7ffacc0cea50 con 0x7ffacc15cc60 2023-10-21T16:21:32.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacad9d640 1 -- 172.21.15.53:0/3658036141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc0d16f0 msgr2=0x7ffacc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:32.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacad9d640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc0d16f0 0x7ffacc0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacad9d640 1 -- 172.21.15.53:0/3658036141 >> v1:172.21.15.53:6789/0 conn(0x7ffacc15cc60 legacy=0x7ffacc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.219+0000 7ffacad9d640 1 -- 172.21.15.53:0/3658036141 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0d0930 con 0x7ffacc0cfa50 2023-10-21T16:21:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.221+0000 7ffacad9d640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0ce510 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ffac40097a0 tx=0x7ffac402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24338c9c57db2186 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.221+0000 7ffac959a640 1 -- 172.21.15.53:0/3658036141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac402d040 con 0x7ffacc0cfa50 2023-10-21T16:21:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.221+0000 7ffac959a640 1 -- 172.21.15.53:0/3658036141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4031810 con 0x7ffacc0cfa50 2023-10-21T16:21:32.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.221+0000 7ffac959a640 1 -- 172.21.15.53:0/3658036141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4031ba0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.221+0000 7ffacb59e640 1 -- 172.21.15.53:0/3658036141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 msgr2=0x7ffacc0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0ce510 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ffac40097a0 tx=0x7ffac402b8b0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 -- 172.21.15.53:0/3658036141 shutdown_connections 2023-10-21T16:21:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0ce510 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3658036141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc0d16f0 0x7ffacc0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 -- 172.21.15.53:0/3658036141 >> 172.21.15.53:0/3658036141 conn(0x7ffacc09bcf0 msgr2=0x7ffacc0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 -- 172.21.15.53:0/3658036141 shutdown_connections 2023-10-21T16:21:32.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.222+0000 7ffacb59e640 1 -- 172.21.15.53:0/3658036141 wait complete. 2023-10-21T16:21:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.223+0000 7ffacb59e640 1 Processor -- start 2023-10-21T16:21:32.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.224+0000 7ffacb59e640 1 -- start start 2023-10-21T16:21:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.224+0000 7ffacb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffacb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffacc0d16f0 0x7ffacc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffacb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc15cc60 0x7ffacc0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffacb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0b49d0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffacb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0b4cb0 con 0x7ffacc0d16f0 2023-10-21T16:21:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffacb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0b4f90 con 0x7ffacc15cc60 2023-10-21T16:21:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49728/0 (socket says 172.21.15.53:49728) 2023-10-21T16:21:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 -- 172.21.15.53:0/3981686226 learned_addr learned my addr 172.21.15.53:0/3981686226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:32.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 -- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc15cc60 msgr2=0x7ffacc0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc15cc60 0x7ffacc0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 -- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffacc0d16f0 msgr2=0x7ffacc0b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffacc0d16f0 0x7ffacc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.225+0000 7ffaca59c640 1 -- 172.21.15.53:0/3981686226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4009400 con 0x7ffacc0cfa50 2023-10-21T16:21:32.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffaca59c640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c840 tx=0x7ffabc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00a640 con 0x7ffacc0cfa50 2023-10-21T16:21:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc00a7a0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00ab50 con 0x7ffacc0cfa50 2023-10-21T16:21:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc0b5270 con 0x7ffacc0cfa50 2023-10-21T16:21:32.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.226+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc0b4510 con 0x7ffacc0cfa50 2023-10-21T16:21:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.227+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc0b4790 con 0x7ffacc0cfa50 2023-10-21T16:21:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.228+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffabc00acb0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.229+0000 7ffac2ffd640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 0x7ffaac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.229+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffabc0898e0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.230+0000 7ffac9d9b640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 0x7ffaac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.231+0000 7ffac9d9b640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 0x7ffaac06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006e80 tx=0x7ffab4008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.233+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffacc0b4790 con 0x7ffacc0cfa50 2023-10-21T16:21:32.414 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-21T16:21:32.413+0000 7f2dce730640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:21:32.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.426+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffacc0c77d0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.429+0000 7ffac2ffd640 1 -- 172.21.15.53:0/3981686226 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16887 (secure 0 0 0) 0x7ffacc0c77d0 con 0x7ffacc0cfa50 2023-10-21T16:21:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:32.432 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":148,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:29.422775+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:32.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 msgr2=0x7ffaac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 0x7ffaac06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006e80 tx=0x7ffab4008040 comp rx=0 tx=0).stop 2023-10-21T16:21:32.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 msgr2=0x7ffacc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c840 tx=0x7ffabc00cd00 comp rx=0 tx=0).stop 2023-10-21T16:21:32.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 shutdown_connections 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7ffaac06af90 0x7ffaac06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffacc15cc60 0x7ffacc0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffacc0d16f0 0x7ffacc0b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 --2- 172.21.15.53:0/3981686226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffacc0cfa50 0x7ffacc0b3fd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 >> 172.21.15.53:0/3981686226 conn(0x7ffacc09bcf0 msgr2=0x7ffacc06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:32.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 shutdown_connections 2023-10-21T16:21:32.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.432+0000 7ffacb59e640 1 -- 172.21.15.53:0/3981686226 wait complete. 2023-10-21T16:21:32.454 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:21:32.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:21:32.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.670+0000 7f9a92090640 1 Processor -- start 2023-10-21T16:21:32.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.671+0000 7f9a92090640 1 -- start start 2023-10-21T16:21:32.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a92090640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 0x7f9a8c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a92090640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a92090640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c0cacf0 con 0x7f9a8c159db0 2023-10-21T16:21:32.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a92090640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0b10f0 con 0x7f9a8c0cc210 2023-10-21T16:21:32.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a92090640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0ca3c0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a9088d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a9188f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9a8c159db0 0x7f9a8c15d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37388/0 (socket says 172.21.15.53:37388) 2023-10-21T16:21:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a9088d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49744/0 (socket says 172.21.15.53:49744) 2023-10-21T16:21:32.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.672+0000 7f9a9188f640 1 -- 172.21.15.53:0/2153821947 learned_addr learned my addr 172.21.15.53:0/2153821947 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:32.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9108e640 1 --2- 172.21.15.53:0/2153821947 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 0x7f9a8c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9088d640 1 -- 172.21.15.53:0/2153821947 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 msgr2=0x7f9a8c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:32.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9088d640 1 --2- 172.21.15.53:0/2153821947 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 0x7f9a8c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9088d640 1 -- 172.21.15.53:0/2153821947 >> v1:172.21.15.53:6789/0 conn(0x7f9a8c159db0 legacy=0x7f9a8c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9088d640 1 -- 172.21.15.53:0/2153821947 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0ca6a0 con 0x7f9a8c0cc210 2023-10-21T16:21:32.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a9108e640 1 --2- 172.21.15.53:0/2153821947 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 0x7f9a8c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.673+0000 7f9a83fff640 1 -- 172.21.15.53:0/2153821947 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991736955 0 0) 0x7f9a8c0cacf0 con 0x7f9a8c159db0 2023-10-21T16:21:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.674+0000 7f9a9088d640 1 --2- 172.21.15.53:0/2153821947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097a0 tx=0x7f9a7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f410520de6b8ae88 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.674+0000 7f9a83fff640 1 -- 172.21.15.53:0/2153821947 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c031830 con 0x7f9a8c0cc210 2023-10-21T16:21:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.674+0000 7f9a83fff640 1 -- 172.21.15.53:0/2153821947 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c031990 con 0x7f9a8c0cc210 2023-10-21T16:21:32.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.674+0000 7f9a83fff640 1 -- 172.21.15.53:0/2153821947 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c031d20 con 0x7f9a8c0cc210 2023-10-21T16:21:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 -- 172.21.15.53:0/2153821947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c0cc210 msgr2=0x7f9a8c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 --2- 172.21.15.53:0/2153821947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0097a0 tx=0x7f9a7c02b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 -- 172.21.15.53:0/2153821947 shutdown_connections 2023-10-21T16:21:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 --2- 172.21.15.53:0/2153821947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b00c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 --2- 172.21.15.53:0/2153821947 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c1599d0 0x7f9a8c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 -- 172.21.15.53:0/2153821947 >> 172.21.15.53:0/2153821947 conn(0x7f9a8c0cb0c0 msgr2=0x7f9a8c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 -- 172.21.15.53:0/2153821947 shutdown_connections 2023-10-21T16:21:32.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.675+0000 7f9a92090640 1 -- 172.21.15.53:0/2153821947 wait complete. 2023-10-21T16:21:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.677+0000 7f9a92090640 1 Processor -- start 2023-10-21T16:21:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.677+0000 7f9a92090640 1 -- start start 2023-10-21T16:21:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 0x7f9a8c070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c155f00 con 0x7f9a8c1599d0 2023-10-21T16:21:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c1561e0 con 0x7f9a8c0cc210 2023-10-21T16:21:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a92090640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c1564c0 con 0x7f9a8c159db0 2023-10-21T16:21:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49760/0 (socket says 172.21.15.53:49760) 2023-10-21T16:21:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 -- 172.21.15.53:0/1937052235 learned_addr learned my addr 172.21.15.53:0/1937052235 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9188f640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 0x7f9a8c070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 -- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 msgr2=0x7f9a8c070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9108e640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 0x7f9a8c070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.678+0000 7f9a9088d640 1 -- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 msgr2=0x7f9a8c0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a9088d640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a9088d640 1 -- 172.21.15.53:0/1937052235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c009400 con 0x7f9a8c1599d0 2023-10-21T16:21:32.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a9188f640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 0x7f9a8c070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a9108e640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a9088d640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c004230 con 0x7f9a8c1599d0 2023-10-21T16:21:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c004390 con 0x7f9a8c1599d0 2023-10-21T16:21:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c03a940 con 0x7f9a8c1599d0 2023-10-21T16:21:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c1567a0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.679+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c155af0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.680+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c156cc0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.681+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9a7c03aaa0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.682+0000 7f9a81ffb640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 0x7f9a7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.682+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a7c0ac1e0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.683+0000 7f9a9108e640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 0x7f9a7006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.684+0000 7f9a9108e640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 0x7f9a7006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.687+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a8c156cc0 con 0x7f9a8c1599d0 2023-10-21T16:21:32.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.879+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 --> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a8c156350 con 0x7f9a7006af90 2023-10-21T16:21:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.907+0000 7f9a81ffb640 1 -- 172.21.15.53:0/1937052235 <== mgr.36350 v2:172.21.15.124:6832/227410594 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287079 (secure 0 0 0) 0x7f9a8c156350 con 0x7f9a7006af90 2023-10-21T16:21:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:32.915 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:21:32.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 msgr2=0x7f9a7006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 0x7f9a7006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006fd0 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2023-10-21T16:21:32.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 msgr2=0x7f9a8c1555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c000c00 tx=0x7f9a7c0046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 shutdown_connections 2023-10-21T16:21:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f9a7006af90 0x7f9a7006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c159db0 0x7f9a8c070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a8c1599d0 0x7f9a8c1555b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 --2- 172.21.15.53:0/1937052235 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cc210 0x7f9a8c0b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:32.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.914+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 >> 172.21.15.53:0/1937052235 conn(0x7f9a8c0cb0c0 msgr2=0x7f9a8c157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:32.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.915+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 shutdown_connections 2023-10-21T16:21:32.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:32.915+0000 7f9a92090640 1 -- 172.21.15.53:0/1937052235 wait complete. 2023-10-21T16:21:32.937 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-21T16:21:32.666216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":411,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4790,"ondisk_log_size":4790,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339916,"kb_used_data":15508,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754634804,"statfs":{"total":773094113280,"available":772746039296,"internally_reserved":0,"allocated":15880192,"data_stored":8627616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":332058169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002095"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.435036+0000","last_change":"2023-10-21T16:21:28.473708+0000","last_active":"2023-10-21T16:21:29.435036+0000","last_peered":"2023-10-21T16:21:29.435036+0000","last_clean":"2023-10-21T16:21:29.435036+0000","last_became_active":"2023-10-21T16:21:28.473409+0000","last_became_peered":"2023-10-21T16:21:28.473409+0000","last_unstale":"2023-10-21T16:21:29.435036+0000","last_undegraded":"2023-10-21T16:21:29.435036+0000","last_fullsized":"2023-10-21T16:21:29.435036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:35.137466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502761+0000","last_change":"2023-10-21T16:21:28.476404+0000","last_active":"2023-10-21T16:21:29.502761+0000","last_peered":"2023-10-21T16:21:29.502761+0000","last_clean":"2023-10-21T16:21:29.502761+0000","last_became_active":"2023-10-21T16:21:28.475837+0000","last_became_peered":"2023-10-21T16:21:28.475837+0000","last_unstale":"2023-10-21T16:21:29.502761+0000","last_undegraded":"2023-10-21T16:21:29.502761+0000","last_fullsized":"2023-10-21T16:21:29.502761+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:24.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503584+0000","last_change":"2023-10-21T16:21:28.461838+0000","last_active":"2023-10-21T16:21:29.503584+0000","last_peered":"2023-10-21T16:21:29.503584+0000","last_clean":"2023-10-21T16:21:29.503584+0000","last_became_active":"2023-10-21T16:21:28.461530+0000","last_became_peered":"2023-10-21T16:21:28.461530+0000","last_unstale":"2023-10-21T16:21:29.503584+0000","last_undegraded":"2023-10-21T16:21:29.503584+0000","last_fullsized":"2023-10-21T16:21:29.503584+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:03.275697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500783+0000","last_change":"2023-10-21T16:21:28.481385+0000","last_active":"2023-10-21T16:21:29.500783+0000","last_peered":"2023-10-21T16:21:29.500783+0000","last_clean":"2023-10-21T16:21:29.500783+0000","last_became_active":"2023-10-21T16:21:28.480722+0000","last_became_peered":"2023-10-21T16:21:28.480722+0000","last_unstale":"2023-10-21T16:21:29.500783+0000","last_undegraded":"2023-10-21T16:21:29.500783+0000","last_fullsized":"2023-10-21T16:21:29.500783+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:59.228330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434062+0000","last_change":"2023-10-21T16:21:28.472376+0000","last_active":"2023-10-21T16:21:29.434062+0000","last_peered":"2023-10-21T16:21:29.434062+0000","last_clean":"2023-10-21T16:21:29.434062+0000","last_became_active":"2023-10-21T16:21:28.471990+0000","last_became_peered":"2023-10-21T16:21:28.471990+0000","last_unstale":"2023-10-21T16:21:29.434062+0000","last_undegraded":"2023-10-21T16:21:29.434062+0000","last_fullsized":"2023-10-21T16:21:29.434062+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:45.479308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502036+0000","last_change":"2023-10-21T16:21:28.463546+0000","last_active":"2023-10-21T16:21:29.502036+0000","last_peered":"2023-10-21T16:21:29.502036+0000","last_clean":"2023-10-21T16:21:29.502036+0000","last_became_active":"2023-10-21T16:21:28.463049+0000","last_became_peered":"2023-10-21T16:21:28.463049+0000","last_unstale":"2023-10-21T16:21:29.502036+0000","last_undegraded":"2023-10-21T16:21:29.502036+0000","last_fullsized":"2023-10-21T16:21:29.502036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:07.571601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501297+0000","last_change":"2023-10-21T16:21:28.467443+0000","last_active":"2023-10-21T16:21:29.501297+0000","last_peered":"2023-10-21T16:21:29.501297+0000","last_clean":"2023-10-21T16:21:29.501297+0000","last_became_active":"2023-10-21T16:21:28.466953+0000","last_became_peered":"2023-10-21T16:21:28.466953+0000","last_unstale":"2023-10-21T16:21:29.501297+0000","last_undegraded":"2023-10-21T16:21:29.501297+0000","last_fullsized":"2023-10-21T16:21:29.501297+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:00.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500807+0000","last_change":"2023-10-21T16:21:28.466024+0000","last_active":"2023-10-21T16:21:29.500807+0000","last_peered":"2023-10-21T16:21:29.500807+0000","last_clean":"2023-10-21T16:21:29.500807+0000","last_became_active":"2023-10-21T16:21:28.465738+0000","last_became_peered":"2023-10-21T16:21:28.465738+0000","last_unstale":"2023-10-21T16:21:29.500807+0000","last_undegraded":"2023-10-21T16:21:29.500807+0000","last_fullsized":"2023-10-21T16:21:29.500807+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:36.764242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501474+0000","last_change":"2023-10-21T16:21:28.482957+0000","last_active":"2023-10-21T16:21:29.501474+0000","last_peered":"2023-10-21T16:21:29.501474+0000","last_clean":"2023-10-21T16:21:29.501474+0000","last_became_active":"2023-10-21T16:21:28.482502+0000","last_became_peered":"2023-10-21T16:21:28.482502+0000","last_unstale":"2023-10-21T16:21:29.501474+0000","last_undegraded":"2023-10-21T16:21:29.501474+0000","last_fullsized":"2023-10-21T16:21:29.501474+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:16:11.456386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500216+0000","last_change":"2023-10-21T16:21:28.457336+0000","last_active":"2023-10-21T16:21:29.500216+0000","last_peered":"2023-10-21T16:21:29.500216+0000","last_clean":"2023-10-21T16:21:29.500216+0000","last_became_active":"2023-10-21T16:21:28.456559+0000","last_became_peered":"2023-10-21T16:21:28.456559+0000","last_unstale":"2023-10-21T16:21:29.500216+0000","last_undegraded":"2023-10-21T16:21:29.500216+0000","last_fullsized":"2023-10-21T16:21:29.500216+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:45.479308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434342+0000","last_change":"2023-10-21T16:21:28.466715+0000","last_active":"2023-10-21T16:21:29.434342+0000","last_peered":"2023-10-21T16:21:29.434342+0000","last_clean":"2023-10-21T16:21:29.434342+0000","last_became_active":"2023-10-21T16:21:28.466378+0000","last_became_peered":"2023-10-21T16:21:28.466378+0000","last_unstale":"2023-10-21T16:21:29.434342+0000","last_undegraded":"2023-10-21T16:21:29.434342+0000","last_fullsized":"2023-10-21T16:21:29.434342+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:23:44.398346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502300+0000","last_change":"2023-10-21T16:21:28.471998+0000","last_active":"2023-10-21T16:21:29.502300+0000","last_peered":"2023-10-21T16:21:29.502300+0000","last_clean":"2023-10-21T16:21:29.502300+0000","last_became_active":"2023-10-21T16:21:28.471634+0000","last_became_peered":"2023-10-21T16:21:28.471634+0000","last_unstale":"2023-10-21T16:21:29.502300+0000","last_undegraded":"2023-10-21T16:21:29.502300+0000","last_fullsized":"2023-10-21T16:21:29.502300+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:44.780264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499583+0000","last_change":"2023-10-21T16:21:28.456083+0000","last_active":"2023-10-21T16:21:29.499583+0000","last_peered":"2023-10-21T16:21:29.499583+0000","last_clean":"2023-10-21T16:21:29.499583+0000","last_became_active":"2023-10-21T16:21:28.454959+0000","last_became_peered":"2023-10-21T16:21:28.454959+0000","last_unstale":"2023-10-21T16:21:29.499583+0000","last_undegraded":"2023-10-21T16:21:29.499583+0000","last_fullsized":"2023-10-21T16:21:29.499583+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:58:13.790735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502712+0000","last_change":"2023-10-21T16:21:28.466868+0000","last_active":"2023-10-21T16:21:29.502712+0000","last_peered":"2023-10-21T16:21:29.502712+0000","last_clean":"2023-10-21T16:21:29.502712+0000","last_became_active":"2023-10-21T16:21:28.466153+0000","last_became_peered":"2023-10-21T16:21:28.466153+0000","last_unstale":"2023-10-21T16:21:29.502712+0000","last_undegraded":"2023-10-21T16:21:29.502712+0000","last_fullsized":"2023-10-21T16:21:29.502712+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:00.289685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.433319+0000","last_change":"2023-10-21T16:21:28.466370+0000","last_active":"2023-10-21T16:21:29.433319+0000","last_peered":"2023-10-21T16:21:29.433319+0000","last_clean":"2023-10-21T16:21:29.433319+0000","last_became_active":"2023-10-21T16:21:28.466066+0000","last_became_peered":"2023-10-21T16:21:28.466066+0000","last_unstale":"2023-10-21T16:21:29.433319+0000","last_undegraded":"2023-10-21T16:21:29.433319+0000","last_fullsized":"2023-10-21T16:21:29.433319+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:06.900826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500437+0000","last_change":"2023-10-21T16:21:28.455875+0000","last_active":"2023-10-21T16:21:29.500437+0000","last_peered":"2023-10-21T16:21:29.500437+0000","last_clean":"2023-10-21T16:21:29.500437+0000","last_became_active":"2023-10-21T16:21:28.455594+0000","last_became_peered":"2023-10-21T16:21:28.455594+0000","last_unstale":"2023-10-21T16:21:29.500437+0000","last_undegraded":"2023-10-21T16:21:29.500437+0000","last_fullsized":"2023-10-21T16:21:29.500437+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:52.085326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500195+0000","last_change":"2023-10-21T16:21:28.465330+0000","last_active":"2023-10-21T16:21:29.500195+0000","last_peered":"2023-10-21T16:21:29.500195+0000","last_clean":"2023-10-21T16:21:29.500195+0000","last_became_active":"2023-10-21T16:21:28.465064+0000","last_became_peered":"2023-10-21T16:21:28.465064+0000","last_unstale":"2023-10-21T16:21:29.500195+0000","last_undegraded":"2023-10-21T16:21:29.500195+0000","last_fullsized":"2023-10-21T16:21:29.500195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:24:13.975919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502968+0000","last_change":"2023-10-21T16:21:28.476523+0000","last_active":"2023-10-21T16:21:29.502968+0000","last_peered":"2023-10-21T16:21:29.502968+0000","last_clean":"2023-10-21T16:21:29.502968+0000","last_became_active":"2023-10-21T16:21:28.476252+0000","last_became_peered":"2023-10-21T16:21:28.476252+0000","last_unstale":"2023-10-21T16:21:29.502968+0000","last_undegraded":"2023-10-21T16:21:29.502968+0000","last_fullsized":"2023-10-21T16:21:29.502968+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:39.608332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502602+0000","last_change":"2023-10-21T16:21:28.478102+0000","last_active":"2023-10-21T16:21:29.502602+0000","last_peered":"2023-10-21T16:21:29.502602+0000","last_clean":"2023-10-21T16:21:29.502602+0000","last_became_active":"2023-10-21T16:21:28.477806+0000","last_became_peered":"2023-10-21T16:21:28.477806+0000","last_unstale":"2023-10-21T16:21:29.502602+0000","last_undegraded":"2023-10-21T16:21:29.502602+0000","last_fullsized":"2023-10-21T16:21:29.502602+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:19.806712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430318+0000","last_change":"2023-10-21T16:21:28.455630+0000","last_active":"2023-10-21T16:21:29.430318+0000","last_peered":"2023-10-21T16:21:29.430318+0000","last_clean":"2023-10-21T16:21:29.430318+0000","last_became_active":"2023-10-21T16:21:28.455186+0000","last_became_peered":"2023-10-21T16:21:28.455186+0000","last_unstale":"2023-10-21T16:21:29.430318+0000","last_undegraded":"2023-10-21T16:21:29.430318+0000","last_fullsized":"2023-10-21T16:21:29.430318+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:51:21.953791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500254+0000","last_change":"2023-10-21T16:21:28.470975+0000","last_active":"2023-10-21T16:21:29.500254+0000","last_peered":"2023-10-21T16:21:29.500254+0000","last_clean":"2023-10-21T16:21:29.500254+0000","last_became_active":"2023-10-21T16:21:28.470722+0000","last_became_peered":"2023-10-21T16:21:28.470722+0000","last_unstale":"2023-10-21T16:21:29.500254+0000","last_undegraded":"2023-10-21T16:21:29.500254+0000","last_fullsized":"2023-10-21T16:21:29.500254+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:47.033809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434455+0000","last_change":"2023-10-21T16:21:28.473895+0000","last_active":"2023-10-21T16:21:29.434455+0000","last_peered":"2023-10-21T16:21:29.434455+0000","last_clean":"2023-10-21T16:21:29.434455+0000","last_became_active":"2023-10-21T16:21:28.473440+0000","last_became_peered":"2023-10-21T16:21:28.473440+0000","last_unstale":"2023-10-21T16:21:29.434455+0000","last_undegraded":"2023-10-21T16:21:29.434455+0000","last_fullsized":"2023-10-21T16:21:29.434455+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:00.248185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499977+0000","last_change":"2023-10-21T16:21:28.472416+0000","last_active":"2023-10-21T16:21:29.499977+0000","last_peered":"2023-10-21T16:21:29.499977+0000","last_clean":"2023-10-21T16:21:29.499977+0000","last_became_active":"2023-10-21T16:21:28.472122+0000","last_became_peered":"2023-10-21T16:21:28.472122+0000","last_unstale":"2023-10-21T16:21:29.499977+0000","last_undegraded":"2023-10-21T16:21:29.499977+0000","last_fullsized":"2023-10-21T16:21:29.499977+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:39:07.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429896+0000","last_change":"2023-10-21T16:21:28.450322+0000","last_active":"2023-10-21T16:21:29.429896+0000","last_peered":"2023-10-21T16:21:29.429896+0000","last_clean":"2023-10-21T16:21:29.429896+0000","last_became_active":"2023-10-21T16:21:28.449626+0000","last_became_peered":"2023-10-21T16:21:28.449626+0000","last_unstale":"2023-10-21T16:21:29.429896+0000","last_undegraded":"2023-10-21T16:21:29.429896+0000","last_fullsized":"2023-10-21T16:21:29.429896+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:55.931672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431610+0000","last_change":"2023-10-21T16:21:28.469492+0000","last_active":"2023-10-21T16:21:29.431610+0000","last_peered":"2023-10-21T16:21:29.431610+0000","last_clean":"2023-10-21T16:21:29.431610+0000","last_became_active":"2023-10-21T16:21:28.469013+0000","last_became_peered":"2023-10-21T16:21:28.469013+0000","last_unstale":"2023-10-21T16:21:29.431610+0000","last_undegraded":"2023-10-21T16:21:29.431610+0000","last_fullsized":"2023-10-21T16:21:29.431610+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:06.787182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429884+0000","last_change":"2023-10-21T16:21:28.452740+0000","last_active":"2023-10-21T16:21:29.429884+0000","last_peered":"2023-10-21T16:21:29.429884+0000","last_clean":"2023-10-21T16:21:29.429884+0000","last_became_active":"2023-10-21T16:21:28.452401+0000","last_became_peered":"2023-10-21T16:21:28.452401+0000","last_unstale":"2023-10-21T16:21:29.429884+0000","last_undegraded":"2023-10-21T16:21:29.429884+0000","last_fullsized":"2023-10-21T16:21:29.429884+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:29.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502784+0000","last_change":"2023-10-21T16:21:28.471414+0000","last_active":"2023-10-21T16:21:29.502784+0000","last_peered":"2023-10-21T16:21:29.502784+0000","last_clean":"2023-10-21T16:21:29.502784+0000","last_became_active":"2023-10-21T16:21:28.471049+0000","last_became_peered":"2023-10-21T16:21:28.471049+0000","last_unstale":"2023-10-21T16:21:29.502784+0000","last_undegraded":"2023-10-21T16:21:29.502784+0000","last_fullsized":"2023-10-21T16:21:29.502784+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:22.734823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502402+0000","last_change":"2023-10-21T16:21:28.457435+0000","last_active":"2023-10-21T16:21:29.502402+0000","last_peered":"2023-10-21T16:21:29.502402+0000","last_clean":"2023-10-21T16:21:29.502402+0000","last_became_active":"2023-10-21T16:21:28.457114+0000","last_became_peered":"2023-10-21T16:21:28.457114+0000","last_unstale":"2023-10-21T16:21:29.502402+0000","last_undegraded":"2023-10-21T16:21:29.502402+0000","last_fullsized":"2023-10-21T16:21:29.502402+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:42.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430117+0000","last_change":"2023-10-21T16:21:28.459447+0000","last_active":"2023-10-21T16:21:29.430117+0000","last_peered":"2023-10-21T16:21:29.430117+0000","last_clean":"2023-10-21T16:21:29.430117+0000","last_became_active":"2023-10-21T16:21:28.459165+0000","last_became_peered":"2023-10-21T16:21:28.459165+0000","last_unstale":"2023-10-21T16:21:29.430117+0000","last_undegraded":"2023-10-21T16:21:29.430117+0000","last_fullsized":"2023-10-21T16:21:29.430117+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:02:56.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502881+0000","last_change":"2023-10-21T16:21:28.481949+0000","last_active":"2023-10-21T16:21:29.502881+0000","last_peered":"2023-10-21T16:21:29.502881+0000","last_clean":"2023-10-21T16:21:29.502881+0000","last_became_active":"2023-10-21T16:21:28.479257+0000","last_became_peered":"2023-10-21T16:21:28.479257+0000","last_unstale":"2023-10-21T16:21:29.502881+0000","last_undegraded":"2023-10-21T16:21:29.502881+0000","last_fullsized":"2023-10-21T16:21:29.502881+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:52.085326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430301+0000","last_change":"2023-10-21T16:21:28.452071+0000","last_active":"2023-10-21T16:21:29.430301+0000","last_peered":"2023-10-21T16:21:29.430301+0000","last_clean":"2023-10-21T16:21:29.430301+0000","last_became_active":"2023-10-21T16:21:28.445206+0000","last_became_peered":"2023-10-21T16:21:28.445206+0000","last_unstale":"2023-10-21T16:21:29.430301+0000","last_undegraded":"2023-10-21T16:21:29.430301+0000","last_fullsized":"2023-10-21T16:21:29.430301+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:51.169048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429894+0000","last_change":"2023-10-21T16:21:28.453071+0000","last_active":"2023-10-21T16:21:29.429894+0000","last_peered":"2023-10-21T16:21:29.429894+0000","last_clean":"2023-10-21T16:21:29.429894+0000","last_became_active":"2023-10-21T16:21:28.452138+0000","last_became_peered":"2023-10-21T16:21:28.452138+0000","last_unstale":"2023-10-21T16:21:29.429894+0000","last_undegraded":"2023-10-21T16:21:29.429894+0000","last_fullsized":"2023-10-21T16:21:29.429894+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:45.773032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499923+0000","last_change":"2023-10-21T16:21:28.458223+0000","last_active":"2023-10-21T16:21:29.499923+0000","last_peered":"2023-10-21T16:21:29.499923+0000","last_clean":"2023-10-21T16:21:29.499923+0000","last_became_active":"2023-10-21T16:21:28.457205+0000","last_became_peered":"2023-10-21T16:21:28.457205+0000","last_unstale":"2023-10-21T16:21:29.499923+0000","last_undegraded":"2023-10-21T16:21:29.499923+0000","last_fullsized":"2023-10-21T16:21:29.499923+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:06.900826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"147'4","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502075+0000","last_change":"2023-10-21T16:21:27.435665+0000","last_active":"2023-10-21T16:21:29.502075+0000","last_peered":"2023-10-21T16:21:29.502075+0000","last_clean":"2023-10-21T16:21:29.502075+0000","last_became_active":"2023-10-21T16:21:27.435385+0000","last_became_peered":"2023-10-21T16:21:27.435385+0000","last_unstale":"2023-10-21T16:21:29.502075+0000","last_undegraded":"2023-10-21T16:21:29.502075+0000","last_fullsized":"2023-10-21T16:21:29.502075+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:20:13.704884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502279+0000","last_change":"2023-10-21T16:21:28.466007+0000","last_active":"2023-10-21T16:21:29.502279+0000","last_peered":"2023-10-21T16:21:29.502279+0000","last_clean":"2023-10-21T16:21:29.502279+0000","last_became_active":"2023-10-21T16:21:28.463978+0000","last_became_peered":"2023-10-21T16:21:28.463978+0000","last_unstale":"2023-10-21T16:21:29.502279+0000","last_undegraded":"2023-10-21T16:21:29.502279+0000","last_fullsized":"2023-10-21T16:21:29.502279+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:37:38.583264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500460+0000","last_change":"2023-10-21T16:21:27.440811+0000","last_active":"2023-10-21T16:21:29.500460+0000","last_peered":"2023-10-21T16:21:29.500460+0000","last_clean":"2023-10-21T16:21:29.500460+0000","last_became_active":"2023-10-21T16:21:27.439874+0000","last_became_peered":"2023-10-21T16:21:27.439874+0000","last_unstale":"2023-10-21T16:21:29.500460+0000","last_undegraded":"2023-10-21T16:21:29.500460+0000","last_fullsized":"2023-10-21T16:21:29.500460+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:21.717995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431031+0000","last_change":"2023-10-21T16:21:28.462746+0000","last_active":"2023-10-21T16:21:29.431031+0000","last_peered":"2023-10-21T16:21:29.431031+0000","last_clean":"2023-10-21T16:21:29.431031+0000","last_became_active":"2023-10-21T16:21:28.461427+0000","last_became_peered":"2023-10-21T16:21:28.461427+0000","last_unstale":"2023-10-21T16:21:29.431031+0000","last_undegraded":"2023-10-21T16:21:29.431031+0000","last_fullsized":"2023-10-21T16:21:29.431031+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:28.462096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501130+0000","last_change":"2023-10-21T16:21:27.438295+0000","last_active":"2023-10-21T16:21:29.501130+0000","last_peered":"2023-10-21T16:21:29.501130+0000","last_clean":"2023-10-21T16:21:29.501130+0000","last_became_active":"2023-10-21T16:21:27.438047+0000","last_became_peered":"2023-10-21T16:21:27.438047+0000","last_unstale":"2023-10-21T16:21:29.501130+0000","last_undegraded":"2023-10-21T16:21:29.501130+0000","last_fullsized":"2023-10-21T16:21:29.501130+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:18.942149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501509+0000","last_change":"2023-10-21T16:21:28.476113+0000","last_active":"2023-10-21T16:21:29.501509+0000","last_peered":"2023-10-21T16:21:29.501509+0000","last_clean":"2023-10-21T16:21:29.501509+0000","last_became_active":"2023-10-21T16:21:28.475764+0000","last_became_peered":"2023-10-21T16:21:28.475764+0000","last_unstale":"2023-10-21T16:21:29.501509+0000","last_undegraded":"2023-10-21T16:21:29.501509+0000","last_fullsized":"2023-10-21T16:21:29.501509+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:52:50.922190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499672+0000","last_change":"2023-10-21T16:21:27.433269+0000","last_active":"2023-10-21T16:21:29.499672+0000","last_peered":"2023-10-21T16:21:29.499672+0000","last_clean":"2023-10-21T16:21:29.499672+0000","last_became_active":"2023-10-21T16:21:27.428115+0000","last_became_peered":"2023-10-21T16:21:27.428115+0000","last_unstale":"2023-10-21T16:21:29.499672+0000","last_undegraded":"2023-10-21T16:21:29.499672+0000","last_fullsized":"2023-10-21T16:21:29.499672+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:16.581883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.498921+0000","last_change":"2023-10-21T16:21:28.457672+0000","last_active":"2023-10-21T16:21:29.498921+0000","last_peered":"2023-10-21T16:21:29.498921+0000","last_clean":"2023-10-21T16:21:29.498921+0000","last_became_active":"2023-10-21T16:21:28.456834+0000","last_became_peered":"2023-10-21T16:21:28.456834+0000","last_unstale":"2023-10-21T16:21:29.498921+0000","last_undegraded":"2023-10-21T16:21:29.498921+0000","last_fullsized":"2023-10-21T16:21:29.498921+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:44.282263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434223+0000","last_change":"2023-10-21T16:21:27.425715+0000","last_active":"2023-10-21T16:21:29.434223+0000","last_peered":"2023-10-21T16:21:29.434223+0000","last_clean":"2023-10-21T16:21:29.434223+0000","last_became_active":"2023-10-21T16:21:27.425404+0000","last_became_peered":"2023-10-21T16:21:27.425404+0000","last_unstale":"2023-10-21T16:21:29.434223+0000","last_undegraded":"2023-10-21T16:21:29.434223+0000","last_fullsized":"2023-10-21T16:21:29.434223+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:13:28.822322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.433821+0000","last_change":"2023-10-21T16:21:28.477998+0000","last_active":"2023-10-21T16:21:29.433821+0000","last_peered":"2023-10-21T16:21:29.433821+0000","last_clean":"2023-10-21T16:21:29.433821+0000","last_became_active":"2023-10-21T16:21:28.477638+0000","last_became_peered":"2023-10-21T16:21:28.477638+0000","last_unstale":"2023-10-21T16:21:29.433821+0000","last_undegraded":"2023-10-21T16:21:29.433821+0000","last_fullsized":"2023-10-21T16:21:29.433821+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:24:13.975919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502160+0000","last_change":"2023-10-21T16:21:27.440307+0000","last_active":"2023-10-21T16:21:29.502160+0000","last_peered":"2023-10-21T16:21:29.502160+0000","last_clean":"2023-10-21T16:21:29.502160+0000","last_became_active":"2023-10-21T16:21:27.440005+0000","last_became_peered":"2023-10-21T16:21:27.440005+0000","last_unstale":"2023-10-21T16:21:29.502160+0000","last_undegraded":"2023-10-21T16:21:29.502160+0000","last_fullsized":"2023-10-21T16:21:29.502160+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.223742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503076+0000","last_change":"2023-10-21T16:21:28.462540+0000","last_active":"2023-10-21T16:21:29.503076+0000","last_peered":"2023-10-21T16:21:29.503076+0000","last_clean":"2023-10-21T16:21:29.503076+0000","last_became_active":"2023-10-21T16:21:28.462142+0000","last_became_peered":"2023-10-21T16:21:28.462142+0000","last_unstale":"2023-10-21T16:21:29.503076+0000","last_undegraded":"2023-10-21T16:21:29.503076+0000","last_fullsized":"2023-10-21T16:21:29.503076+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:50.951577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430546+0000","last_change":"2023-10-21T16:21:27.433035+0000","last_active":"2023-10-21T16:21:29.430546+0000","last_peered":"2023-10-21T16:21:29.430546+0000","last_clean":"2023-10-21T16:21:29.430546+0000","last_became_active":"2023-10-21T16:21:27.432711+0000","last_became_peered":"2023-10-21T16:21:27.432711+0000","last_unstale":"2023-10-21T16:21:29.430546+0000","last_undegraded":"2023-10-21T16:21:29.430546+0000","last_fullsized":"2023-10-21T16:21:29.430546+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:05.698566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499283+0000","last_change":"2023-10-21T16:21:28.463539+0000","last_active":"2023-10-21T16:21:29.499283+0000","last_peered":"2023-10-21T16:21:29.499283+0000","last_clean":"2023-10-21T16:21:29.499283+0000","last_became_active":"2023-10-21T16:21:28.463248+0000","last_became_peered":"2023-10-21T16:21:28.463248+0000","last_unstale":"2023-10-21T16:21:29.499283+0000","last_undegraded":"2023-10-21T16:21:29.499283+0000","last_fullsized":"2023-10-21T16:21:29.499283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:53:09.450501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500689+0000","last_change":"2023-10-21T16:21:27.428365+0000","last_active":"2023-10-21T16:21:29.500689+0000","last_peered":"2023-10-21T16:21:29.500689+0000","last_clean":"2023-10-21T16:21:29.500689+0000","last_became_active":"2023-10-21T16:21:27.427866+0000","last_became_peered":"2023-10-21T16:21:27.427866+0000","last_unstale":"2023-10-21T16:21:29.500689+0000","last_undegraded":"2023-10-21T16:21:29.500689+0000","last_fullsized":"2023-10-21T16:21:29.500689+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:59.272857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502222+0000","last_change":"2023-10-21T16:21:28.474740+0000","last_active":"2023-10-21T16:21:29.502222+0000","last_peered":"2023-10-21T16:21:29.502222+0000","last_clean":"2023-10-21T16:21:29.502222+0000","last_became_active":"2023-10-21T16:21:28.474136+0000","last_became_peered":"2023-10-21T16:21:28.474136+0000","last_unstale":"2023-10-21T16:21:29.502222+0000","last_undegraded":"2023-10-21T16:21:29.502222+0000","last_fullsized":"2023-10-21T16:21:29.502222+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:43.228418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502316+0000","last_change":"2023-10-21T16:21:27.437083+0000","last_active":"2023-10-21T16:21:29.502316+0000","last_peered":"2023-10-21T16:21:29.502316+0000","last_clean":"2023-10-21T16:21:29.502316+0000","last_became_active":"2023-10-21T16:21:27.436717+0000","last_became_peered":"2023-10-21T16:21:27.436717+0000","last_unstale":"2023-10-21T16:21:29.502316+0000","last_undegraded":"2023-10-21T16:21:29.502316+0000","last_fullsized":"2023-10-21T16:21:29.502316+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:28:29.554812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431413+0000","last_change":"2023-10-21T16:21:28.465103+0000","last_active":"2023-10-21T16:21:29.431413+0000","last_peered":"2023-10-21T16:21:29.431413+0000","last_clean":"2023-10-21T16:21:29.431413+0000","last_became_active":"2023-10-21T16:21:28.463554+0000","last_became_peered":"2023-10-21T16:21:28.463554+0000","last_unstale":"2023-10-21T16:21:29.431413+0000","last_undegraded":"2023-10-21T16:21:29.431413+0000","last_fullsized":"2023-10-21T16:21:29.431413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:10.183230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431253+0000","last_change":"2023-10-21T16:21:27.428146+0000","last_active":"2023-10-21T16:21:29.431253+0000","last_peered":"2023-10-21T16:21:29.431253+0000","last_clean":"2023-10-21T16:21:29.431253+0000","last_became_active":"2023-10-21T16:21:27.427541+0000","last_became_peered":"2023-10-21T16:21:27.427541+0000","last_unstale":"2023-10-21T16:21:29.431253+0000","last_undegraded":"2023-10-21T16:21:29.431253+0000","last_fullsized":"2023-10-21T16:21:29.431253+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:05.672003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503120+0000","last_change":"2023-10-21T16:21:28.465818+0000","last_active":"2023-10-21T16:21:29.503120+0000","last_peered":"2023-10-21T16:21:29.503120+0000","last_clean":"2023-10-21T16:21:29.503120+0000","last_became_active":"2023-10-21T16:21:28.465470+0000","last_became_peered":"2023-10-21T16:21:28.465470+0000","last_unstale":"2023-10-21T16:21:29.503120+0000","last_undegraded":"2023-10-21T16:21:29.503120+0000","last_fullsized":"2023-10-21T16:21:29.503120+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:00.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"147'3","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431317+0000","last_change":"2023-10-21T16:21:27.433019+0000","last_active":"2023-10-21T16:21:29.431317+0000","last_peered":"2023-10-21T16:21:29.431317+0000","last_clean":"2023-10-21T16:21:29.431317+0000","last_became_active":"2023-10-21T16:21:27.429603+0000","last_became_peered":"2023-10-21T16:21:27.429603+0000","last_unstale":"2023-10-21T16:21:29.431317+0000","last_undegraded":"2023-10-21T16:21:29.431317+0000","last_fullsized":"2023-10-21T16:21:29.431317+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:38.704039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431931+0000","last_change":"2023-10-21T16:21:28.444626+0000","last_active":"2023-10-21T16:21:29.431931+0000","last_peered":"2023-10-21T16:21:29.431931+0000","last_clean":"2023-10-21T16:21:29.431931+0000","last_became_active":"2023-10-21T16:21:28.443817+0000","last_became_peered":"2023-10-21T16:21:28.443817+0000","last_unstale":"2023-10-21T16:21:29.431931+0000","last_undegraded":"2023-10-21T16:21:29.431931+0000","last_fullsized":"2023-10-21T16:21:29.431931+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:21.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"147'6","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503191+0000","last_change":"2023-10-21T16:21:27.435636+0000","last_active":"2023-10-21T16:21:29.503191+0000","last_peered":"2023-10-21T16:21:29.503191+0000","last_clean":"2023-10-21T16:21:29.503191+0000","last_became_active":"2023-10-21T16:21:27.433316+0000","last_became_peered":"2023-10-21T16:21:27.433316+0000","last_unstale":"2023-10-21T16:21:29.503191+0000","last_undegraded":"2023-10-21T16:21:29.503191+0000","last_fullsized":"2023-10-21T16:21:29.503191+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:41.436043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502139+0000","last_change":"2023-10-21T16:21:28.462713+0000","last_active":"2023-10-21T16:21:29.502139+0000","last_peered":"2023-10-21T16:21:29.502139+0000","last_clean":"2023-10-21T16:21:29.502139+0000","last_became_active":"2023-10-21T16:21:28.460837+0000","last_became_peered":"2023-10-21T16:21:28.460837+0000","last_unstale":"2023-10-21T16:21:29.502139+0000","last_undegraded":"2023-10-21T16:21:29.502139+0000","last_fullsized":"2023-10-21T16:21:29.502139+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:59.844950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502690+0000","last_change":"2023-10-21T16:21:27.436315+0000","last_active":"2023-10-21T16:21:29.502690+0000","last_peered":"2023-10-21T16:21:29.502690+0000","last_clean":"2023-10-21T16:21:29.502690+0000","last_became_active":"2023-10-21T16:21:27.436034+0000","last_became_peered":"2023-10-21T16:21:27.436034+0000","last_unstale":"2023-10-21T16:21:29.502690+0000","last_undegraded":"2023-10-21T16:21:29.502690+0000","last_fullsized":"2023-10-21T16:21:29.502690+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:19.022583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499032+0000","last_change":"2023-10-21T16:21:28.467651+0000","last_active":"2023-10-21T16:21:29.499032+0000","last_peered":"2023-10-21T16:21:29.499032+0000","last_clean":"2023-10-21T16:21:29.499032+0000","last_became_active":"2023-10-21T16:21:28.467321+0000","last_became_peered":"2023-10-21T16:21:28.467321+0000","last_unstale":"2023-10-21T16:21:29.499032+0000","last_undegraded":"2023-10-21T16:21:29.499032+0000","last_fullsized":"2023-10-21T16:21:29.499032+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:48:22.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503444+0000","last_change":"2023-10-21T16:21:27.427412+0000","last_active":"2023-10-21T16:21:29.503444+0000","last_peered":"2023-10-21T16:21:29.503444+0000","last_clean":"2023-10-21T16:21:29.503444+0000","last_became_active":"2023-10-21T16:21:27.427155+0000","last_became_peered":"2023-10-21T16:21:27.427155+0000","last_unstale":"2023-10-21T16:21:29.503444+0000","last_undegraded":"2023-10-21T16:21:29.503444+0000","last_fullsized":"2023-10-21T16:21:29.503444+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:59.385541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429696+0000","last_change":"2023-10-21T16:21:28.475400+0000","last_active":"2023-10-21T16:21:29.429696+0000","last_peered":"2023-10-21T16:21:29.429696+0000","last_clean":"2023-10-21T16:21:29.429696+0000","last_became_active":"2023-10-21T16:21:28.474687+0000","last_became_peered":"2023-10-21T16:21:28.474687+0000","last_unstale":"2023-10-21T16:21:29.429696+0000","last_undegraded":"2023-10-21T16:21:29.429696+0000","last_fullsized":"2023-10-21T16:21:29.429696+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:07.547939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501765+0000","last_change":"2023-10-21T16:21:27.438057+0000","last_active":"2023-10-21T16:21:29.501765+0000","last_peered":"2023-10-21T16:21:29.501765+0000","last_clean":"2023-10-21T16:21:29.501765+0000","last_became_active":"2023-10-21T16:21:27.432472+0000","last_became_peered":"2023-10-21T16:21:27.432472+0000","last_unstale":"2023-10-21T16:21:29.501765+0000","last_undegraded":"2023-10-21T16:21:29.501765+0000","last_fullsized":"2023-10-21T16:21:29.501765+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.223742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430906+0000","last_change":"2023-10-21T16:21:28.461803+0000","last_active":"2023-10-21T16:21:29.430906+0000","last_peered":"2023-10-21T16:21:29.430906+0000","last_clean":"2023-10-21T16:21:29.430906+0000","last_became_active":"2023-10-21T16:21:28.461496+0000","last_became_peered":"2023-10-21T16:21:28.461496+0000","last_unstale":"2023-10-21T16:21:29.430906+0000","last_undegraded":"2023-10-21T16:21:29.430906+0000","last_fullsized":"2023-10-21T16:21:29.430906+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:28.335807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500365+0000","last_change":"2023-10-21T16:21:27.438947+0000","last_active":"2023-10-21T16:21:29.500365+0000","last_peered":"2023-10-21T16:21:29.500365+0000","last_clean":"2023-10-21T16:21:29.500365+0000","last_became_active":"2023-10-21T16:21:27.437970+0000","last_became_peered":"2023-10-21T16:21:27.437970+0000","last_unstale":"2023-10-21T16:21:29.500365+0000","last_undegraded":"2023-10-21T16:21:29.500365+0000","last_fullsized":"2023-10-21T16:21:29.500365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:27.973921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434771+0000","last_change":"2023-10-21T16:21:28.466042+0000","last_active":"2023-10-21T16:21:29.434771+0000","last_peered":"2023-10-21T16:21:29.434771+0000","last_clean":"2023-10-21T16:21:29.434771+0000","last_became_active":"2023-10-21T16:21:28.465574+0000","last_became_peered":"2023-10-21T16:21:28.465574+0000","last_unstale":"2023-10-21T16:21:29.434771+0000","last_undegraded":"2023-10-21T16:21:29.434771+0000","last_fullsized":"2023-10-21T16:21:29.434771+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:58:06.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429977+0000","last_change":"2023-10-21T16:21:27.431513+0000","last_active":"2023-10-21T16:21:29.429977+0000","last_peered":"2023-10-21T16:21:29.429977+0000","last_clean":"2023-10-21T16:21:29.429977+0000","last_became_active":"2023-10-21T16:21:27.429289+0000","last_became_peered":"2023-10-21T16:21:27.429289+0000","last_unstale":"2023-10-21T16:21:29.429977+0000","last_undegraded":"2023-10-21T16:21:29.429977+0000","last_fullsized":"2023-10-21T16:21:29.429977+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:27.856986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502075+0000","last_change":"2023-10-21T16:21:28.460162+0000","last_active":"2023-10-21T16:21:29.502075+0000","last_peered":"2023-10-21T16:21:29.502075+0000","last_clean":"2023-10-21T16:21:29.502075+0000","last_became_active":"2023-10-21T16:21:28.459920+0000","last_became_peered":"2023-10-21T16:21:28.459920+0000","last_unstale":"2023-10-21T16:21:29.502075+0000","last_undegraded":"2023-10-21T16:21:29.502075+0000","last_fullsized":"2023-10-21T16:21:29.502075+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:03.275697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501215+0000","last_change":"2023-10-21T16:21:27.438253+0000","last_active":"2023-10-21T16:21:29.501215+0000","last_peered":"2023-10-21T16:21:29.501215+0000","last_clean":"2023-10-21T16:21:29.501215+0000","last_became_active":"2023-10-21T16:21:27.437990+0000","last_became_peered":"2023-10-21T16:21:27.437990+0000","last_unstale":"2023-10-21T16:21:29.501215+0000","last_undegraded":"2023-10-21T16:21:29.501215+0000","last_fullsized":"2023-10-21T16:21:29.501215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:50.554279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.432083+0000","last_change":"2023-10-21T16:21:28.470573+0000","last_active":"2023-10-21T16:21:29.432083+0000","last_peered":"2023-10-21T16:21:29.432083+0000","last_clean":"2023-10-21T16:21:29.432083+0000","last_became_active":"2023-10-21T16:21:28.470136+0000","last_became_peered":"2023-10-21T16:21:28.470136+0000","last_unstale":"2023-10-21T16:21:29.432083+0000","last_undegraded":"2023-10-21T16:21:29.432083+0000","last_fullsized":"2023-10-21T16:21:29.432083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:23:02.745460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500770+0000","last_change":"2023-10-21T16:21:27.443444+0000","last_active":"2023-10-21T16:21:29.500770+0000","last_peered":"2023-10-21T16:21:29.500770+0000","last_clean":"2023-10-21T16:21:29.500770+0000","last_became_active":"2023-10-21T16:21:27.443171+0000","last_became_peered":"2023-10-21T16:21:27.443171+0000","last_unstale":"2023-10-21T16:21:29.500770+0000","last_undegraded":"2023-10-21T16:21:29.500770+0000","last_fullsized":"2023-10-21T16:21:29.500770+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:03.672987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500616+0000","last_change":"2023-10-21T16:21:28.460162+0000","last_active":"2023-10-21T16:21:29.500616+0000","last_peered":"2023-10-21T16:21:29.500616+0000","last_clean":"2023-10-21T16:21:29.500616+0000","last_became_active":"2023-10-21T16:21:28.459859+0000","last_became_peered":"2023-10-21T16:21:28.459859+0000","last_unstale":"2023-10-21T16:21:29.500616+0000","last_undegraded":"2023-10-21T16:21:29.500616+0000","last_fullsized":"2023-10-21T16:21:29.500616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:39.608332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501343+0000","last_change":"2023-10-21T16:21:27.433574+0000","last_active":"2023-10-21T16:21:29.501343+0000","last_peered":"2023-10-21T16:21:29.501343+0000","last_clean":"2023-10-21T16:21:29.501343+0000","last_became_active":"2023-10-21T16:21:27.432651+0000","last_became_peered":"2023-10-21T16:21:27.432651+0000","last_unstale":"2023-10-21T16:21:29.501343+0000","last_undegraded":"2023-10-21T16:21:29.501343+0000","last_fullsized":"2023-10-21T16:21:29.501343+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:09.166403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501352+0000","last_change":"2023-10-21T16:21:28.472638+0000","last_active":"2023-10-21T16:21:29.501352+0000","last_peered":"2023-10-21T16:21:29.501352+0000","last_clean":"2023-10-21T16:21:29.501352+0000","last_became_active":"2023-10-21T16:21:28.472351+0000","last_became_peered":"2023-10-21T16:21:28.472351+0000","last_unstale":"2023-10-21T16:21:29.501352+0000","last_undegraded":"2023-10-21T16:21:29.501352+0000","last_fullsized":"2023-10-21T16:21:29.501352+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:04.393797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499936+0000","last_change":"2023-10-21T16:21:27.440868+0000","last_active":"2023-10-21T16:21:29.499936+0000","last_peered":"2023-10-21T16:21:29.499936+0000","last_clean":"2023-10-21T16:21:29.499936+0000","last_became_active":"2023-10-21T16:21:27.440392+0000","last_became_peered":"2023-10-21T16:21:27.440392+0000","last_unstale":"2023-10-21T16:21:29.499936+0000","last_undegraded":"2023-10-21T16:21:29.499936+0000","last_fullsized":"2023-10-21T16:21:29.499936+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:27:55.557534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503329+0000","last_change":"2023-10-21T16:21:28.466930+0000","last_active":"2023-10-21T16:21:29.503329+0000","last_peered":"2023-10-21T16:21:29.503329+0000","last_clean":"2023-10-21T16:21:29.503329+0000","last_became_active":"2023-10-21T16:21:28.466524+0000","last_became_peered":"2023-10-21T16:21:28.466524+0000","last_unstale":"2023-10-21T16:21:29.503329+0000","last_undegraded":"2023-10-21T16:21:29.503329+0000","last_fullsized":"2023-10-21T16:21:29.503329+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:07.571601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501582+0000","last_change":"2023-10-21T16:21:27.438818+0000","last_active":"2023-10-21T16:21:29.501582+0000","last_peered":"2023-10-21T16:21:29.501582+0000","last_clean":"2023-10-21T16:21:29.501582+0000","last_became_active":"2023-10-21T16:21:27.437260+0000","last_became_peered":"2023-10-21T16:21:27.437260+0000","last_unstale":"2023-10-21T16:21:29.501582+0000","last_undegraded":"2023-10-21T16:21:29.501582+0000","last_fullsized":"2023-10-21T16:21:29.501582+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:02:31.545386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430821+0000","last_change":"2023-10-21T16:21:28.445852+0000","last_active":"2023-10-21T16:21:29.430821+0000","last_peered":"2023-10-21T16:21:29.430821+0000","last_clean":"2023-10-21T16:21:29.430821+0000","last_became_active":"2023-10-21T16:21:28.443973+0000","last_became_peered":"2023-10-21T16:21:28.443973+0000","last_unstale":"2023-10-21T16:21:29.430821+0000","last_undegraded":"2023-10-21T16:21:29.430821+0000","last_fullsized":"2023-10-21T16:21:29.430821+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:32.513715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499467+0000","last_change":"2023-10-21T16:21:27.436384+0000","last_active":"2023-10-21T16:21:29.499467+0000","last_peered":"2023-10-21T16:21:29.499467+0000","last_clean":"2023-10-21T16:21:29.499467+0000","last_became_active":"2023-10-21T16:21:27.436080+0000","last_became_peered":"2023-10-21T16:21:27.436080+0000","last_unstale":"2023-10-21T16:21:29.499467+0000","last_undegraded":"2023-10-21T16:21:29.499467+0000","last_fullsized":"2023-10-21T16:21:29.499467+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:44:05.464101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502472+0000","last_change":"2023-10-21T16:21:28.452536+0000","last_active":"2023-10-21T16:21:29.502472+0000","last_peered":"2023-10-21T16:21:29.502472+0000","last_clean":"2023-10-21T16:21:29.502472+0000","last_became_active":"2023-10-21T16:21:28.452255+0000","last_became_peered":"2023-10-21T16:21:28.452255+0000","last_unstale":"2023-10-21T16:21:29.502472+0000","last_undegraded":"2023-10-21T16:21:29.502472+0000","last_fullsized":"2023-10-21T16:21:29.502472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:45.344305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501874+0000","last_change":"2023-10-21T16:21:27.430030+0000","last_active":"2023-10-21T16:21:29.501874+0000","last_peered":"2023-10-21T16:21:29.501874+0000","last_clean":"2023-10-21T16:21:29.501874+0000","last_became_active":"2023-10-21T16:21:27.429201+0000","last_became_peered":"2023-10-21T16:21:27.429201+0000","last_unstale":"2023-10-21T16:21:29.501874+0000","last_undegraded":"2023-10-21T16:21:29.501874+0000","last_fullsized":"2023-10-21T16:21:29.501874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:05.770354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499609+0000","last_change":"2023-10-21T16:21:28.455273+0000","last_active":"2023-10-21T16:21:29.499609+0000","last_peered":"2023-10-21T16:21:29.499609+0000","last_clean":"2023-10-21T16:21:29.499609+0000","last_became_active":"2023-10-21T16:21:28.454965+0000","last_became_peered":"2023-10-21T16:21:28.454965+0000","last_unstale":"2023-10-21T16:21:29.499609+0000","last_undegraded":"2023-10-21T16:21:29.499609+0000","last_fullsized":"2023-10-21T16:21:29.499609+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:24.157553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502592+0000","last_change":"2023-10-21T16:21:27.435719+0000","last_active":"2023-10-21T16:21:29.502592+0000","last_peered":"2023-10-21T16:21:29.502592+0000","last_clean":"2023-10-21T16:21:29.502592+0000","last_became_active":"2023-10-21T16:21:27.435438+0000","last_became_peered":"2023-10-21T16:21:27.435438+0000","last_unstale":"2023-10-21T16:21:29.502592+0000","last_undegraded":"2023-10-21T16:21:29.502592+0000","last_fullsized":"2023-10-21T16:21:29.502592+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:46:57.396761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501567+0000","last_change":"2023-10-21T16:21:28.477469+0000","last_active":"2023-10-21T16:21:29.501567+0000","last_peered":"2023-10-21T16:21:29.501567+0000","last_clean":"2023-10-21T16:21:29.501567+0000","last_became_active":"2023-10-21T16:21:28.477197+0000","last_became_peered":"2023-10-21T16:21:28.477197+0000","last_unstale":"2023-10-21T16:21:29.501567+0000","last_undegraded":"2023-10-21T16:21:29.501567+0000","last_fullsized":"2023-10-21T16:21:29.501567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:43:47.499014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502705+0000","last_change":"2023-10-21T16:21:27.437161+0000","last_active":"2023-10-21T16:21:29.502705+0000","last_peered":"2023-10-21T16:21:29.502705+0000","last_clean":"2023-10-21T16:21:29.502705+0000","last_became_active":"2023-10-21T16:21:27.436727+0000","last_became_peered":"2023-10-21T16:21:27.436727+0000","last_unstale":"2023-10-21T16:21:29.502705+0000","last_undegraded":"2023-10-21T16:21:29.502705+0000","last_fullsized":"2023-10-21T16:21:29.502705+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:35:17.742315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430090+0000","last_change":"2023-10-21T16:21:28.444824+0000","last_active":"2023-10-21T16:21:29.430090+0000","last_peered":"2023-10-21T16:21:29.430090+0000","last_clean":"2023-10-21T16:21:29.430090+0000","last_became_active":"2023-10-21T16:21:28.443610+0000","last_became_peered":"2023-10-21T16:21:28.443610+0000","last_unstale":"2023-10-21T16:21:29.430090+0000","last_undegraded":"2023-10-21T16:21:29.430090+0000","last_fullsized":"2023-10-21T16:21:29.430090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:01.127571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502509+0000","last_change":"2023-10-21T16:21:27.440652+0000","last_active":"2023-10-21T16:21:29.502509+0000","last_peered":"2023-10-21T16:21:29.502509+0000","last_clean":"2023-10-21T16:21:29.502509+0000","last_became_active":"2023-10-21T16:21:27.440374+0000","last_became_peered":"2023-10-21T16:21:27.440374+0000","last_unstale":"2023-10-21T16:21:29.502509+0000","last_undegraded":"2023-10-21T16:21:29.502509+0000","last_fullsized":"2023-10-21T16:21:29.502509+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:35.747414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500977+0000","last_change":"2023-10-21T16:21:28.466660+0000","last_active":"2023-10-21T16:21:29.500977+0000","last_peered":"2023-10-21T16:21:29.500977+0000","last_clean":"2023-10-21T16:21:29.500977+0000","last_became_active":"2023-10-21T16:21:28.466340+0000","last_became_peered":"2023-10-21T16:21:28.466340+0000","last_unstale":"2023-10-21T16:21:29.500977+0000","last_undegraded":"2023-10-21T16:21:29.500977+0000","last_fullsized":"2023-10-21T16:21:29.500977+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:50.951577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434427+0000","last_change":"2023-10-21T16:21:27.426963+0000","last_active":"2023-10-21T16:21:29.434427+0000","last_peered":"2023-10-21T16:21:29.434427+0000","last_clean":"2023-10-21T16:21:29.434427+0000","last_became_active":"2023-10-21T16:21:27.426636+0000","last_became_peered":"2023-10-21T16:21:27.426636+0000","last_unstale":"2023-10-21T16:21:29.434427+0000","last_undegraded":"2023-10-21T16:21:29.434427+0000","last_fullsized":"2023-10-21T16:21:29.434427+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:58:12.773907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501346+0000","last_change":"2023-10-21T16:21:28.466502+0000","last_active":"2023-10-21T16:21:29.501346+0000","last_peered":"2023-10-21T16:21:29.501346+0000","last_clean":"2023-10-21T16:21:29.501346+0000","last_became_active":"2023-10-21T16:21:28.465403+0000","last_became_peered":"2023-10-21T16:21:28.465403+0000","last_unstale":"2023-10-21T16:21:29.501346+0000","last_undegraded":"2023-10-21T16:21:29.501346+0000","last_fullsized":"2023-10-21T16:21:29.501346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:04.099050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434062+0000","last_change":"2023-10-21T16:21:27.422560+0000","last_active":"2023-10-21T16:21:29.434062+0000","last_peered":"2023-10-21T16:21:29.434062+0000","last_clean":"2023-10-21T16:21:29.434062+0000","last_became_active":"2023-10-21T16:21:27.422120+0000","last_became_peered":"2023-10-21T16:21:27.422120+0000","last_unstale":"2023-10-21T16:21:29.434062+0000","last_undegraded":"2023-10-21T16:21:29.434062+0000","last_fullsized":"2023-10-21T16:21:29.434062+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:23.140726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499751+0000","last_change":"2023-10-21T16:21:28.469905+0000","last_active":"2023-10-21T16:21:29.499751+0000","last_peered":"2023-10-21T16:21:29.499751+0000","last_clean":"2023-10-21T16:21:29.499751+0000","last_became_active":"2023-10-21T16:21:28.469596+0000","last_became_peered":"2023-10-21T16:21:28.469596+0000","last_unstale":"2023-10-21T16:21:29.499751+0000","last_undegraded":"2023-10-21T16:21:29.499751+0000","last_fullsized":"2023-10-21T16:21:29.499751+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:52:38.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501405+0000","last_change":"2023-10-21T16:21:27.427053+0000","last_active":"2023-10-21T16:21:29.501405+0000","last_peered":"2023-10-21T16:21:29.501405+0000","last_clean":"2023-10-21T16:21:29.501405+0000","last_became_active":"2023-10-21T16:21:27.426754+0000","last_became_peered":"2023-10-21T16:21:27.426754+0000","last_unstale":"2023-10-21T16:21:29.501405+0000","last_undegraded":"2023-10-21T16:21:29.501405+0000","last_fullsized":"2023-10-21T16:21:29.501405+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:23.479236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499449+0000","last_change":"2023-10-21T16:21:28.468429+0000","last_active":"2023-10-21T16:21:29.499449+0000","last_peered":"2023-10-21T16:21:29.499449+0000","last_clean":"2023-10-21T16:21:29.499449+0000","last_became_active":"2023-10-21T16:21:28.468155+0000","last_became_peered":"2023-10-21T16:21:28.468155+0000","last_unstale":"2023-10-21T16:21:29.499449+0000","last_undegraded":"2023-10-21T16:21:29.499449+0000","last_fullsized":"2023-10-21T16:21:29.499449+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:13:29.839149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499516+0000","last_change":"2023-10-21T16:21:27.438692+0000","last_active":"2023-10-21T16:21:29.499516+0000","last_peered":"2023-10-21T16:21:29.499516+0000","last_clean":"2023-10-21T16:21:29.499516+0000","last_became_active":"2023-10-21T16:21:27.437690+0000","last_became_peered":"2023-10-21T16:21:27.437690+0000","last_unstale":"2023-10-21T16:21:29.499516+0000","last_undegraded":"2023-10-21T16:21:29.499516+0000","last_fullsized":"2023-10-21T16:21:29.499516+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:43.391159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500272+0000","last_change":"2023-10-21T16:21:28.471846+0000","last_active":"2023-10-21T16:21:29.500272+0000","last_peered":"2023-10-21T16:21:29.500272+0000","last_clean":"2023-10-21T16:21:29.500272+0000","last_became_active":"2023-10-21T16:21:28.471551+0000","last_became_peered":"2023-10-21T16:21:28.471551+0000","last_unstale":"2023-10-21T16:21:29.500272+0000","last_undegraded":"2023-10-21T16:21:29.500272+0000","last_fullsized":"2023-10-21T16:21:29.500272+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:21.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"147'8","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501757+0000","last_change":"2023-10-21T16:21:27.432039+0000","last_active":"2023-10-21T16:21:29.501757+0000","last_peered":"2023-10-21T16:21:29.501757+0000","last_clean":"2023-10-21T16:21:29.501757+0000","last_became_active":"2023-10-21T16:21:27.431767+0000","last_became_peered":"2023-10-21T16:21:27.431767+0000","last_unstale":"2023-10-21T16:21:29.501757+0000","last_undegraded":"2023-10-21T16:21:29.501757+0000","last_fullsized":"2023-10-21T16:21:29.501757+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:36:29.944714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"136'4752","reported_seq":5261,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-21T16:21:28.456279+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:21:28.456279+0000","last_peered":"2023-10-21T16:21:28.456279+0000","last_clean":"2023-10-21T16:21:28.456279+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:21:28.456279+0000","last_undegraded":"2023-10-21T16:21:28.456279+0000","last_fullsized":"2023-10-21T16:21:28.456279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4752,"log_dups_size":0,"ondisk_log_size":4752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":367,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":367,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4752,"ondisk_log_size":4752,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47448,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324392,"statfs":{"total":96636764160,"available":96588177408,"internally_reserved":0,"allocated":2437120,"data_stored":1520565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.075,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.60399999999999998,"15min":0.51300000000000001},"max":{"1min":1.6919999999999999,"5min":1.758,"15min":1.8400000000000001},"last":1.335},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.169},"min":{"1min":0.755,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.4930000000000001,"5min":1.7270000000000001,"15min":2.448},"last":1.377}]},{"osd":1,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.149,"15min":1.161},"min":{"1min":0.76100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.601,"5min":1.851,"15min":1.915},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1759999999999999,"15min":1.226},"min":{"1min":0.70299999999999996,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.4399999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.2689999999999999,"15min":1.2989999999999999},"min":{"1min":0.79900000000000004,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.704,"5min":1.782,"15min":2.085},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.85199999999999998,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.837,"5min":1.837,"15min":2.6070000000000002},"last":0.72699999999999998}]},{"osd":3,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.206,"15min":1.248},"min":{"1min":0.878,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5880000000000001,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.236},{"interface":"front","average":{"1min":1.417,"5min":1.369,"15min":1.421},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.5780000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3149999999999999,"15min":1.3500000000000001},"min":{"1min":1.032,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.794,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.29},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4339999999999999,"15min":1.4379999999999999},"min":{"1min":1.3120000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.355},"last":1.415}]},{"osd":5,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4299999999999999,"15min":1.403},"min":{"1min":1.0249999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.518},{"interface":"front","average":{"1min":1.46,"5min":1.4359999999999999,"15min":1.4399999999999999},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.016,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.194}]},{"osd":6,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.532,"15min":1.5489999999999999},"min":{"1min":1.129,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.9890000000000001,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4419999999999999,"15min":1.4570000000000001},"min":{"1min":0.95599999999999996,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":2.02,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":0.98899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":930003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0860000000000001,"15min":1.1220000000000001},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.704,"5min":1.869,"15min":1.869},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.61299999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.895},"last":0.82199999999999995}]},{"osd":1,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1379999999999999,"15min":1.1899999999999999},"min":{"1min":0.80200000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.835,"5min":1.835,"15min":2.2309999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1870000000000001,"15min":1.2829999999999999},"min":{"1min":0.81499999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5309999999999999,"5min":1.97,"15min":2.2669999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2270000000000001,"15min":1.2509999999999999},"min":{"1min":0.90400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.726,"5min":1.895,"15min":2.5339999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4330000000000001,"5min":1.821,"15min":1.821},"last":0.91200000000000003}]},{"osd":3,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3129999999999999,"15min":1.3480000000000001},"min":{"1min":1.014,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8049999999999999,"5min":1.907,"15min":2.2000000000000002},"last":1.278},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.4159999999999999},"min":{"1min":0.93000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.845,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.323}]},{"osd":4,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3540000000000001,"15min":1.339},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6890000000000001,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.4299999999999999,"15min":1.395},"min":{"1min":0.74199999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.228}]},{"osd":5,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.5,"15min":1.502},"min":{"1min":0.86399999999999999,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":1.8899999999999999,"5min":2.246,"15min":2.246},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3460000000000001,"15min":1.3520000000000001},"min":{"1min":0.77300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.669,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.1970000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.476,"15min":1.5009999999999999},"min":{"1min":0.76300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9159999999999999,"5min":2.2050000000000001,"15min":2.3570000000000002},"last":1.468},{"interface":"front","average":{"1min":1.492,"5min":1.6060000000000001,"15min":1.609},"min":{"1min":0.95399999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.931,"5min":2.319,"15min":2.319},"last":1.667}]}]},{"osd":5,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51024,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320816,"statfs":{"total":96636764160,"available":96584515584,"internally_reserved":0,"allocated":1835008,"data_stored":929581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.101,"15min":1.1180000000000001},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4239999999999999,"5min":1.609,"15min":1.742},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.169,"15min":1.1850000000000001},"min":{"1min":0.72999999999999998,"5min":0.68100000000000005,"15min":0.52400000000000002},"max":{"1min":1.873,"5min":1.99,"15min":1.99},"last":1.4550000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.72699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6559999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.351},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.2130000000000001,"15min":1.2230000000000001},"min":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.415,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":1.157}]},{"osd":2,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.286,"15min":1.288},"min":{"1min":0.81100000000000005,"5min":0.624,"15min":0.57199999999999995},"max":{"1min":1.5660000000000001,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":0.72799999999999998,"5min":0.53100000000000003,"15min":0.46999999999999997},"max":{"1min":1.748,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.3120000000000001,"15min":1.3129999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.57599999999999996},"max":{"1min":1.7509999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.607},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.377,"15min":1.3819999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.66800000000000004},"max":{"1min":1.8120000000000001,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.069}]},{"osd":4,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2629999999999999,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.52600000000000002,"15min":0.48899999999999999},"max":{"1min":1.494,"5min":4.359,"15min":4.359},"last":1.302},{"interface":"front","average":{"1min":1.365,"5min":1.391,"15min":1.357},"min":{"1min":0.626,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.714,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.663}]},{"osd":6,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4810000000000001,"15min":1.4470000000000001},"min":{"1min":0.89700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6950000000000001,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.405},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4990000000000001,"15min":1.4790000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.60099999999999998},"max":{"1min":1.992,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.5029999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.536,"15min":1.506},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.58299999999999996},"max":{"1min":1.946,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.544},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.593,"15min":1.554},"min":{"1min":0.84899999999999998,"5min":0.76700000000000002,"15min":0.60999999999999999},"max":{"1min":2.0939999999999999,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.4870000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1839104,"data_stored":931033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1459999999999999,"15min":1.151},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3899999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.212,"15min":1.2090000000000001},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.726,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.83299999999999996,"5min":0.73999999999999999,"15min":0.621},"max":{"1min":1.754,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.278,"15min":1.2569999999999999},"min":{"1min":0.81499999999999995,"5min":0.70199999999999996,"15min":0.64500000000000002},"max":{"1min":1.4390000000000001,"5min":7.484,"15min":7.484},"last":0.877}]},{"osd":2,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.248,"15min":1.26},"min":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4430000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.337,"15min":1.343},"min":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.6679999999999999,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.5369999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.337,"15min":1.3240000000000001},"min":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.60399999999999998},"max":{"1min":1.712,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.3899999999999999},"min":{"1min":0.85499999999999998,"5min":0.76900000000000002,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.24}]},{"osd":5,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.323,"15min":1.304},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5740000000000001,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.399,"15min":1.3520000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.603,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.173}]},{"osd":6,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.4690000000000001,"15min":1.425},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.5980000000000001,"15min":1.5780000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7729999999999999,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.611}]},{"osd":7,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.6120000000000001,"15min":1.5149999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.752,"5min":12.458,"15min":12.458},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.6120000000000001,"15min":1.5620000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.871,"5min":3.9510000000000001,"15min":3.968},"last":1.506}]}]},{"osd":3,"up_from":13,"seq":55834575080,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41304,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330536,"statfs":{"total":96636764160,"available":96594468864,"internally_reserved":0,"allocated":1843200,"data_stored":931067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.879,"15min":0.93899999999999995},"min":{"1min":0.442,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4159999999999999,"5min":1.946,"15min":1.946},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1659999999999999},"min":{"1min":0.52300000000000002,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.04},"last":0.67500000000000004}]},{"osd":1,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":0.97399999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.589,"15min":2.173},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.228},"min":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8100000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0509999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.643,"5min":1.669,"15min":2.0049999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.077,"5min":1.135,"15min":1.202},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.46600000000000003},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.847},"last":0.85099999999999998}]},{"osd":4,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.319,"15min":1.393},"min":{"1min":0.70399999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5780000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.006},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3899999999999999},"min":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.5800000000000001,"5min":1.863,"15min":2.919},"last":1.1799999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.327,"15min":1.393},"min":{"1min":0.65900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.02},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4179999999999999,"15min":1.4870000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.75700000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.423},"last":1.097}]},{"osd":6,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4139999999999999,"15min":1.478},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.8400000000000001,"15min":2.6179999999999999},"last":1.079},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.419,"15min":1.508},"min":{"1min":0.75700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.704,"5min":1.8999999999999999,"15min":2.2290000000000001},"last":1.323}]},{"osd":7,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4950000000000001,"15min":1.5740000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3759999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.52,"15min":1.587},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.7989999999999999},"last":1.3520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46852,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324988,"statfs":{"total":96636764160,"available":96588787712,"internally_reserved":0,"allocated":1826816,"data_stored":929421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.857,"5min":1.857,"15min":1.859},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.92900000000000005,"15min":0.96999999999999997},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.28299999999999997},"max":{"1min":1.6259999999999999,"5min":1.905,"15min":2.0539999999999998},"last":0.871}]},{"osd":1,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0269999999999999,"15min":1.103},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.39500000000000002},"max":{"1min":1.5820000000000001,"5min":1.6890000000000001,"15min":1.9390000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.159,"15min":1.232},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6919999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.94099999999999995}]},{"osd":3,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0840000000000001,"15min":1.1160000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.897,"5min":1.9279999999999999,"15min":2.0710000000000002},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.276,"15min":1.347},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.895,"5min":2.069,"15min":2.069},"last":0.91100000000000003}]},{"osd":4,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.321,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.76600000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":2.1589999999999998},{"interface":"front","average":{"1min":1.319,"5min":1.3740000000000001,"15min":1.3700000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60199999999999998},"max":{"1min":1.95,"5min":2.278,"15min":2.6960000000000002},"last":1.95}]},{"osd":5,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.571,"5min":2.024,"15min":2.2730000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4630000000000001,"15min":1.506},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.024,"5min":2.1280000000000001,"15min":2.1349999999999998},"last":2.024}]},{"osd":6,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4650000000000001,"15min":1.478},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"max":{"1min":2.0600000000000001,"5min":2.444,"15min":2.6819999999999999},"last":2.0600000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78900000000000003},"max":{"1min":2.121,"5min":2.2669999999999999,"15min":2.552},"last":1.131}]},{"osd":7,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5049999999999999,"15min":1.5289999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.69999999999999996},"max":{"1min":1.8460000000000001,"5min":2.2130000000000001,"15min":2.4969999999999999},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.468,"15min":1.5049999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.3999999999999999},"last":1.3240000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1830912,"data_stored":929443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.90000000000000002},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.1100000000000001,"5min":1.806,"15min":1.905},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.0469999999999999,"15min":1.0680000000000001},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4990000000000001,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.651,"5min":2.222,"15min":2.2639999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.167},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":1.615,"5min":1.6910000000000001,"15min":1.974},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.228,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.54800000000000004,"15min":0.51600000000000001},"max":{"1min":1.5649999999999999,"5min":1.911,"15min":1.9510000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.526,"5min":1.6879999999999999,"15min":1.9159999999999999},"last":0.499}]},{"osd":4,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.238,"15min":1.268},"min":{"1min":0.82999999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.379,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.403,"15min":1.4239999999999999},"min":{"1min":1.024,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.9019999999999999,"5min":2.145,"15min":2.2719999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":1.2410000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.4590000000000001},"min":{"1min":1.2010000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.627,"5min":2.077,"15min":2.0880000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.482,"15min":1.49},"min":{"1min":0.85099999999999998,"5min":0.77500000000000002,"15min":0.68500000000000005},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.4079999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.417,"15min":1.4510000000000001},"min":{"1min":0.94499999999999995,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.746,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.63}]},{"osd":7,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5549999999999999,"15min":1.5940000000000001},"min":{"1min":1.1619999999999999,"5min":0.64600000000000002,"15min":0.59299999999999997},"max":{"1min":1.804,"5min":2.2530000000000001,"15min":2.3730000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.89000000000000001,"5min":0.74399999999999999,"15min":0.68600000000000005},"max":{"1min":2.1110000000000002,"5min":2.1850000000000001,"15min":2.367},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41888,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329952,"statfs":{"total":96636764160,"available":96593870848,"internally_reserved":0,"allocated":2437120,"data_stored":1526503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":1.302,"5min":1.649,"15min":1.855},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.996,"15min":1.0549999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.589,"5min":1.589,"15min":1.667},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.45100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.679,"5min":1.708,"15min":1.7749999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.1339999999999999},"min":{"1min":0.48499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5580000000000001,"5min":1.694,"15min":1.732},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0069999999999999,"15min":1.0780000000000001},"min":{"1min":0.53800000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.6080000000000001,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.1930000000000001,"15min":1.2769999999999999},"min":{"1min":0.58399999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7969999999999999,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":0.995}]},{"osd":4,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3819999999999999,"15min":1.4039999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.9019999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4550000000000001,"15min":1.516},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8029999999999999,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.351}]},{"osd":5,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.329,"15min":1.3660000000000001},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.823,"5min":1.9419999999999999,"15min":2.137},"last":1.3},{"interface":"front","average":{"1min":1.425,"5min":1.3879999999999999,"15min":1.4790000000000001},"min":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.9259999999999999,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":1.427}]},{"osd":6,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.954,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.4870000000000001,"15min":1.5720000000000001},"min":{"1min":0.72099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.911,"5min":3.097,"15min":3.097},"last":1.2509999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.925,"5min":2.085,"15min":2.2959999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.464,"15min":1.5509999999999999},"min":{"1min":0.79000000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0600000000000001,"5min":2.077,"15min":2.3540000000000001},"last":1.387}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:21:32.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:21:33.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.208+0000 7f0dc629f640 1 Processor -- start 2023-10-21T16:21:33.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.208+0000 7f0dc629f640 1 -- start start 2023-10-21T16:21:33.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.209+0000 7f0dc629f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.209+0000 7f0dc629f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0dc00ce2c0 0x7f0dc015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc629f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dc00cde30 con 0x7f0dc0157480 2023-10-21T16:21:33.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc629f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc0157860 con 0x7f0dc00ce2c0 2023-10-21T16:21:33.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc629f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc015e2c0 con 0x7f0dc006b680 2023-10-21T16:21:33.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc4a9c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc4a9c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:34952/0 (socket says 172.21.15.53:34952) 2023-10-21T16:21:33.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.210+0000 7f0dc4a9c640 1 -- 172.21.15.53:0/3427593663 learned_addr learned my addr 172.21.15.53:0/3427593663 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:33.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.211+0000 7f0dc5a9e640 1 --2- 172.21.15.53:0/3427593663 >> v2:172.21.15.124:3300/0 conn(0x7f0dc00ce2c0 0x7f0dc015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.211+0000 7f0dc4a9c640 1 -- 172.21.15.53:0/3427593663 >> v2:172.21.15.124:3300/0 conn(0x7f0dc00ce2c0 msgr2=0x7f0dc015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:33.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.211+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/3427593663 >> v2:172.21.15.124:3300/0 conn(0x7f0dc00ce2c0 0x7f0dc015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.211+0000 7f0dc4a9c640 1 -- 172.21.15.53:0/3427593663 >> v1:172.21.15.124:6789/0 conn(0x7f0dc0157480 legacy=0x7f0dc006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.211+0000 7f0dc4a9c640 1 -- 172.21.15.53:0/3427593663 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc00cd500 con 0x7f0dc006b680 2023-10-21T16:21:33.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.213+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/3427593663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969a7d7096db3a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.213+0000 7f0db7fff640 1 -- 172.21.15.53:0/3427593663 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0031640 con 0x7f0dc006b680 2023-10-21T16:21:33.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0db7fff640 1 -- 172.21.15.53:0/3427593663 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db00317a0 con 0x7f0dc006b680 2023-10-21T16:21:33.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0db7fff640 1 -- 172.21.15.53:0/3427593663 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0031b30 con 0x7f0dc006b680 2023-10-21T16:21:33.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 -- 172.21.15.53:0/3427593663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 msgr2=0x7f0dc015d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 --2- 172.21.15.53:0/3427593663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 -- 172.21.15.53:0/3427593663 shutdown_connections 2023-10-21T16:21:33.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 --2- 172.21.15.53:0/3427593663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc006b680 0x7f0dc015d550 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 --2- 172.21.15.53:0/3427593663 >> v2:172.21.15.124:3300/0 conn(0x7f0dc00ce2c0 0x7f0dc015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 -- 172.21.15.53:0/3427593663 >> 172.21.15.53:0/3427593663 conn(0x7f0dc00d2ae0 msgr2=0x7f0dc00d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 -- 172.21.15.53:0/3427593663 shutdown_connections 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.214+0000 7f0dc629f640 1 -- 172.21.15.53:0/3427593663 wait complete. 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.216+0000 7f0dc629f640 1 Processor -- start 2023-10-21T16:21:33.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.216+0000 7f0dc629f640 1 -- start start 2023-10-21T16:21:33.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 0x7f0dc00cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc0157480 0x7f0dc00cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00cc220 con 0x7f0dc006b680 2023-10-21T16:21:33.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc00cc500 con 0x7f0dc00ce2c0 2023-10-21T16:21:33.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.217+0000 7f0dc629f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc0157d40 con 0x7f0dc0157480 2023-10-21T16:21:33.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49770/0 (socket says 172.21.15.53:49770) 2023-10-21T16:21:33.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 -- 172.21.15.53:0/1708391714 learned_addr learned my addr 172.21.15.53:0/1708391714 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:33.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 -- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc0157480 msgr2=0x7f0dc00cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc0157480 0x7f0dc00cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 0x7f0dc00cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 -- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 msgr2=0x7f0dc00cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 0x7f0dc00cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc529d640 1 -- 172.21.15.53:0/1708391714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00094d0 con 0x7f0dc006b680 2023-10-21T16:21:33.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.218+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 0x7f0dc00cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:33.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0dc529d640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c380 tx=0x7f0dbc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0052d0 con 0x7f0dc006b680 2023-10-21T16:21:33.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc005430 con 0x7f0dc006b680 2023-10-21T16:21:33.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc0158020 con 0x7f0dc006b680 2023-10-21T16:21:33.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc003830 con 0x7f0dc006b680 2023-10-21T16:21:33.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.219+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc015cab0 con 0x7f0dc006b680 2023-10-21T16:21:33.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.220+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc00cc720 con 0x7f0dc006b680 2023-10-21T16:21:33.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.221+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0dbc003990 con 0x7f0dc006b680 2023-10-21T16:21:33.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.222+0000 7f0db5ffb640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 0x7f0da406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.222+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dbc088e30 con 0x7f0dc006b680 2023-10-21T16:21:33.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.223+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 0x7f0da406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.224+0000 7f0dc4a9c640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 0x7f0da406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0db0036000 tx=0x7f0db003c040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.226+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0dc00cc720 con 0x7f0dc006b680 2023-10-21T16:21:33.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.418+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 --> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dc00cdac0 con 0x7f0da406af90 2023-10-21T16:21:33.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.441+0000 7f0db5ffb640 1 -- 172.21.15.53:0/1708391714 <== mgr.36350 v2:172.21.15.124:6832/227410594 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287079 (secure 0 0 0) 0x7f0db003a420 con 0x7f0da406af90 2023-10-21T16:21:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:33.448 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:21:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 msgr2=0x7f0da406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 0x7f0da406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0db0036000 tx=0x7f0db003c040 comp rx=0 tx=0).stop 2023-10-21T16:21:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 msgr2=0x7f0dc0155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00c380 tx=0x7f0dbc00c840 comp rx=0 tx=0).stop 2023-10-21T16:21:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 shutdown_connections 2023-10-21T16:21:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f0da406af90 0x7f0da406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0dc0157480 0x7f0dc00cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0dc00ce2c0 0x7f0dc00cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 --2- 172.21.15.53:0/1708391714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0dc006b680 0x7f0dc0155df0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.446+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 >> 172.21.15.53:0/1708391714 conn(0x7f0dc00d2ae0 msgr2=0x7f0dc00b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.447+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 shutdown_connections 2023-10-21T16:21:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.447+0000 7f0dc629f640 1 -- 172.21.15.53:0/1708391714 wait complete. 2023-10-21T16:21:33.468 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-21T16:21:32.666216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":411,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4790,"ondisk_log_size":4790,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339916,"kb_used_data":15508,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754634804,"statfs":{"total":773094113280,"available":772746039296,"internally_reserved":0,"allocated":15880192,"data_stored":8627616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":332058169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002095"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.435036+0000","last_change":"2023-10-21T16:21:28.473708+0000","last_active":"2023-10-21T16:21:29.435036+0000","last_peered":"2023-10-21T16:21:29.435036+0000","last_clean":"2023-10-21T16:21:29.435036+0000","last_became_active":"2023-10-21T16:21:28.473409+0000","last_became_peered":"2023-10-21T16:21:28.473409+0000","last_unstale":"2023-10-21T16:21:29.435036+0000","last_undegraded":"2023-10-21T16:21:29.435036+0000","last_fullsized":"2023-10-21T16:21:29.435036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:35.137466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502761+0000","last_change":"2023-10-21T16:21:28.476404+0000","last_active":"2023-10-21T16:21:29.502761+0000","last_peered":"2023-10-21T16:21:29.502761+0000","last_clean":"2023-10-21T16:21:29.502761+0000","last_became_active":"2023-10-21T16:21:28.475837+0000","last_became_peered":"2023-10-21T16:21:28.475837+0000","last_unstale":"2023-10-21T16:21:29.502761+0000","last_undegraded":"2023-10-21T16:21:29.502761+0000","last_fullsized":"2023-10-21T16:21:29.502761+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:24.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503584+0000","last_change":"2023-10-21T16:21:28.461838+0000","last_active":"2023-10-21T16:21:29.503584+0000","last_peered":"2023-10-21T16:21:29.503584+0000","last_clean":"2023-10-21T16:21:29.503584+0000","last_became_active":"2023-10-21T16:21:28.461530+0000","last_became_peered":"2023-10-21T16:21:28.461530+0000","last_unstale":"2023-10-21T16:21:29.503584+0000","last_undegraded":"2023-10-21T16:21:29.503584+0000","last_fullsized":"2023-10-21T16:21:29.503584+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:03.275697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500783+0000","last_change":"2023-10-21T16:21:28.481385+0000","last_active":"2023-10-21T16:21:29.500783+0000","last_peered":"2023-10-21T16:21:29.500783+0000","last_clean":"2023-10-21T16:21:29.500783+0000","last_became_active":"2023-10-21T16:21:28.480722+0000","last_became_peered":"2023-10-21T16:21:28.480722+0000","last_unstale":"2023-10-21T16:21:29.500783+0000","last_undegraded":"2023-10-21T16:21:29.500783+0000","last_fullsized":"2023-10-21T16:21:29.500783+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:59.228330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434062+0000","last_change":"2023-10-21T16:21:28.472376+0000","last_active":"2023-10-21T16:21:29.434062+0000","last_peered":"2023-10-21T16:21:29.434062+0000","last_clean":"2023-10-21T16:21:29.434062+0000","last_became_active":"2023-10-21T16:21:28.471990+0000","last_became_peered":"2023-10-21T16:21:28.471990+0000","last_unstale":"2023-10-21T16:21:29.434062+0000","last_undegraded":"2023-10-21T16:21:29.434062+0000","last_fullsized":"2023-10-21T16:21:29.434062+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:45.479308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502036+0000","last_change":"2023-10-21T16:21:28.463546+0000","last_active":"2023-10-21T16:21:29.502036+0000","last_peered":"2023-10-21T16:21:29.502036+0000","last_clean":"2023-10-21T16:21:29.502036+0000","last_became_active":"2023-10-21T16:21:28.463049+0000","last_became_peered":"2023-10-21T16:21:28.463049+0000","last_unstale":"2023-10-21T16:21:29.502036+0000","last_undegraded":"2023-10-21T16:21:29.502036+0000","last_fullsized":"2023-10-21T16:21:29.502036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:07.571601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501297+0000","last_change":"2023-10-21T16:21:28.467443+0000","last_active":"2023-10-21T16:21:29.501297+0000","last_peered":"2023-10-21T16:21:29.501297+0000","last_clean":"2023-10-21T16:21:29.501297+0000","last_became_active":"2023-10-21T16:21:28.466953+0000","last_became_peered":"2023-10-21T16:21:28.466953+0000","last_unstale":"2023-10-21T16:21:29.501297+0000","last_undegraded":"2023-10-21T16:21:29.501297+0000","last_fullsized":"2023-10-21T16:21:29.501297+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:00.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500807+0000","last_change":"2023-10-21T16:21:28.466024+0000","last_active":"2023-10-21T16:21:29.500807+0000","last_peered":"2023-10-21T16:21:29.500807+0000","last_clean":"2023-10-21T16:21:29.500807+0000","last_became_active":"2023-10-21T16:21:28.465738+0000","last_became_peered":"2023-10-21T16:21:28.465738+0000","last_unstale":"2023-10-21T16:21:29.500807+0000","last_undegraded":"2023-10-21T16:21:29.500807+0000","last_fullsized":"2023-10-21T16:21:29.500807+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:36.764242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501474+0000","last_change":"2023-10-21T16:21:28.482957+0000","last_active":"2023-10-21T16:21:29.501474+0000","last_peered":"2023-10-21T16:21:29.501474+0000","last_clean":"2023-10-21T16:21:29.501474+0000","last_became_active":"2023-10-21T16:21:28.482502+0000","last_became_peered":"2023-10-21T16:21:28.482502+0000","last_unstale":"2023-10-21T16:21:29.501474+0000","last_undegraded":"2023-10-21T16:21:29.501474+0000","last_fullsized":"2023-10-21T16:21:29.501474+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:16:11.456386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500216+0000","last_change":"2023-10-21T16:21:28.457336+0000","last_active":"2023-10-21T16:21:29.500216+0000","last_peered":"2023-10-21T16:21:29.500216+0000","last_clean":"2023-10-21T16:21:29.500216+0000","last_became_active":"2023-10-21T16:21:28.456559+0000","last_became_peered":"2023-10-21T16:21:28.456559+0000","last_unstale":"2023-10-21T16:21:29.500216+0000","last_undegraded":"2023-10-21T16:21:29.500216+0000","last_fullsized":"2023-10-21T16:21:29.500216+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:45.479308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434342+0000","last_change":"2023-10-21T16:21:28.466715+0000","last_active":"2023-10-21T16:21:29.434342+0000","last_peered":"2023-10-21T16:21:29.434342+0000","last_clean":"2023-10-21T16:21:29.434342+0000","last_became_active":"2023-10-21T16:21:28.466378+0000","last_became_peered":"2023-10-21T16:21:28.466378+0000","last_unstale":"2023-10-21T16:21:29.434342+0000","last_undegraded":"2023-10-21T16:21:29.434342+0000","last_fullsized":"2023-10-21T16:21:29.434342+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:23:44.398346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502300+0000","last_change":"2023-10-21T16:21:28.471998+0000","last_active":"2023-10-21T16:21:29.502300+0000","last_peered":"2023-10-21T16:21:29.502300+0000","last_clean":"2023-10-21T16:21:29.502300+0000","last_became_active":"2023-10-21T16:21:28.471634+0000","last_became_peered":"2023-10-21T16:21:28.471634+0000","last_unstale":"2023-10-21T16:21:29.502300+0000","last_undegraded":"2023-10-21T16:21:29.502300+0000","last_fullsized":"2023-10-21T16:21:29.502300+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:44.780264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499583+0000","last_change":"2023-10-21T16:21:28.456083+0000","last_active":"2023-10-21T16:21:29.499583+0000","last_peered":"2023-10-21T16:21:29.499583+0000","last_clean":"2023-10-21T16:21:29.499583+0000","last_became_active":"2023-10-21T16:21:28.454959+0000","last_became_peered":"2023-10-21T16:21:28.454959+0000","last_unstale":"2023-10-21T16:21:29.499583+0000","last_undegraded":"2023-10-21T16:21:29.499583+0000","last_fullsized":"2023-10-21T16:21:29.499583+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:58:13.790735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502712+0000","last_change":"2023-10-21T16:21:28.466868+0000","last_active":"2023-10-21T16:21:29.502712+0000","last_peered":"2023-10-21T16:21:29.502712+0000","last_clean":"2023-10-21T16:21:29.502712+0000","last_became_active":"2023-10-21T16:21:28.466153+0000","last_became_peered":"2023-10-21T16:21:28.466153+0000","last_unstale":"2023-10-21T16:21:29.502712+0000","last_undegraded":"2023-10-21T16:21:29.502712+0000","last_fullsized":"2023-10-21T16:21:29.502712+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:00.289685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.433319+0000","last_change":"2023-10-21T16:21:28.466370+0000","last_active":"2023-10-21T16:21:29.433319+0000","last_peered":"2023-10-21T16:21:29.433319+0000","last_clean":"2023-10-21T16:21:29.433319+0000","last_became_active":"2023-10-21T16:21:28.466066+0000","last_became_peered":"2023-10-21T16:21:28.466066+0000","last_unstale":"2023-10-21T16:21:29.433319+0000","last_undegraded":"2023-10-21T16:21:29.433319+0000","last_fullsized":"2023-10-21T16:21:29.433319+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:06.900826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500437+0000","last_change":"2023-10-21T16:21:28.455875+0000","last_active":"2023-10-21T16:21:29.500437+0000","last_peered":"2023-10-21T16:21:29.500437+0000","last_clean":"2023-10-21T16:21:29.500437+0000","last_became_active":"2023-10-21T16:21:28.455594+0000","last_became_peered":"2023-10-21T16:21:28.455594+0000","last_unstale":"2023-10-21T16:21:29.500437+0000","last_undegraded":"2023-10-21T16:21:29.500437+0000","last_fullsized":"2023-10-21T16:21:29.500437+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:52.085326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500195+0000","last_change":"2023-10-21T16:21:28.465330+0000","last_active":"2023-10-21T16:21:29.500195+0000","last_peered":"2023-10-21T16:21:29.500195+0000","last_clean":"2023-10-21T16:21:29.500195+0000","last_became_active":"2023-10-21T16:21:28.465064+0000","last_became_peered":"2023-10-21T16:21:28.465064+0000","last_unstale":"2023-10-21T16:21:29.500195+0000","last_undegraded":"2023-10-21T16:21:29.500195+0000","last_fullsized":"2023-10-21T16:21:29.500195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:24:13.975919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502968+0000","last_change":"2023-10-21T16:21:28.476523+0000","last_active":"2023-10-21T16:21:29.502968+0000","last_peered":"2023-10-21T16:21:29.502968+0000","last_clean":"2023-10-21T16:21:29.502968+0000","last_became_active":"2023-10-21T16:21:28.476252+0000","last_became_peered":"2023-10-21T16:21:28.476252+0000","last_unstale":"2023-10-21T16:21:29.502968+0000","last_undegraded":"2023-10-21T16:21:29.502968+0000","last_fullsized":"2023-10-21T16:21:29.502968+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:39.608332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502602+0000","last_change":"2023-10-21T16:21:28.478102+0000","last_active":"2023-10-21T16:21:29.502602+0000","last_peered":"2023-10-21T16:21:29.502602+0000","last_clean":"2023-10-21T16:21:29.502602+0000","last_became_active":"2023-10-21T16:21:28.477806+0000","last_became_peered":"2023-10-21T16:21:28.477806+0000","last_unstale":"2023-10-21T16:21:29.502602+0000","last_undegraded":"2023-10-21T16:21:29.502602+0000","last_fullsized":"2023-10-21T16:21:29.502602+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:19.806712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430318+0000","last_change":"2023-10-21T16:21:28.455630+0000","last_active":"2023-10-21T16:21:29.430318+0000","last_peered":"2023-10-21T16:21:29.430318+0000","last_clean":"2023-10-21T16:21:29.430318+0000","last_became_active":"2023-10-21T16:21:28.455186+0000","last_became_peered":"2023-10-21T16:21:28.455186+0000","last_unstale":"2023-10-21T16:21:29.430318+0000","last_undegraded":"2023-10-21T16:21:29.430318+0000","last_fullsized":"2023-10-21T16:21:29.430318+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:51:21.953791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500254+0000","last_change":"2023-10-21T16:21:28.470975+0000","last_active":"2023-10-21T16:21:29.500254+0000","last_peered":"2023-10-21T16:21:29.500254+0000","last_clean":"2023-10-21T16:21:29.500254+0000","last_became_active":"2023-10-21T16:21:28.470722+0000","last_became_peered":"2023-10-21T16:21:28.470722+0000","last_unstale":"2023-10-21T16:21:29.500254+0000","last_undegraded":"2023-10-21T16:21:29.500254+0000","last_fullsized":"2023-10-21T16:21:29.500254+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:47.033809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434455+0000","last_change":"2023-10-21T16:21:28.473895+0000","last_active":"2023-10-21T16:21:29.434455+0000","last_peered":"2023-10-21T16:21:29.434455+0000","last_clean":"2023-10-21T16:21:29.434455+0000","last_became_active":"2023-10-21T16:21:28.473440+0000","last_became_peered":"2023-10-21T16:21:28.473440+0000","last_unstale":"2023-10-21T16:21:29.434455+0000","last_undegraded":"2023-10-21T16:21:29.434455+0000","last_fullsized":"2023-10-21T16:21:29.434455+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:18:00.248185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499977+0000","last_change":"2023-10-21T16:21:28.472416+0000","last_active":"2023-10-21T16:21:29.499977+0000","last_peered":"2023-10-21T16:21:29.499977+0000","last_clean":"2023-10-21T16:21:29.499977+0000","last_became_active":"2023-10-21T16:21:28.472122+0000","last_became_peered":"2023-10-21T16:21:28.472122+0000","last_unstale":"2023-10-21T16:21:29.499977+0000","last_undegraded":"2023-10-21T16:21:29.499977+0000","last_fullsized":"2023-10-21T16:21:29.499977+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:39:07.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429896+0000","last_change":"2023-10-21T16:21:28.450322+0000","last_active":"2023-10-21T16:21:29.429896+0000","last_peered":"2023-10-21T16:21:29.429896+0000","last_clean":"2023-10-21T16:21:29.429896+0000","last_became_active":"2023-10-21T16:21:28.449626+0000","last_became_peered":"2023-10-21T16:21:28.449626+0000","last_unstale":"2023-10-21T16:21:29.429896+0000","last_undegraded":"2023-10-21T16:21:29.429896+0000","last_fullsized":"2023-10-21T16:21:29.429896+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:55.931672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431610+0000","last_change":"2023-10-21T16:21:28.469492+0000","last_active":"2023-10-21T16:21:29.431610+0000","last_peered":"2023-10-21T16:21:29.431610+0000","last_clean":"2023-10-21T16:21:29.431610+0000","last_became_active":"2023-10-21T16:21:28.469013+0000","last_became_peered":"2023-10-21T16:21:28.469013+0000","last_unstale":"2023-10-21T16:21:29.431610+0000","last_undegraded":"2023-10-21T16:21:29.431610+0000","last_fullsized":"2023-10-21T16:21:29.431610+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:06.787182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429884+0000","last_change":"2023-10-21T16:21:28.452740+0000","last_active":"2023-10-21T16:21:29.429884+0000","last_peered":"2023-10-21T16:21:29.429884+0000","last_clean":"2023-10-21T16:21:29.429884+0000","last_became_active":"2023-10-21T16:21:28.452401+0000","last_became_peered":"2023-10-21T16:21:28.452401+0000","last_unstale":"2023-10-21T16:21:29.429884+0000","last_undegraded":"2023-10-21T16:21:29.429884+0000","last_fullsized":"2023-10-21T16:21:29.429884+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:29.425547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502784+0000","last_change":"2023-10-21T16:21:28.471414+0000","last_active":"2023-10-21T16:21:29.502784+0000","last_peered":"2023-10-21T16:21:29.502784+0000","last_clean":"2023-10-21T16:21:29.502784+0000","last_became_active":"2023-10-21T16:21:28.471049+0000","last_became_peered":"2023-10-21T16:21:28.471049+0000","last_unstale":"2023-10-21T16:21:29.502784+0000","last_undegraded":"2023-10-21T16:21:29.502784+0000","last_fullsized":"2023-10-21T16:21:29.502784+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:22.734823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502402+0000","last_change":"2023-10-21T16:21:28.457435+0000","last_active":"2023-10-21T16:21:29.502402+0000","last_peered":"2023-10-21T16:21:29.502402+0000","last_clean":"2023-10-21T16:21:29.502402+0000","last_became_active":"2023-10-21T16:21:28.457114+0000","last_became_peered":"2023-10-21T16:21:28.457114+0000","last_unstale":"2023-10-21T16:21:29.502402+0000","last_undegraded":"2023-10-21T16:21:29.502402+0000","last_fullsized":"2023-10-21T16:21:29.502402+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:42.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430117+0000","last_change":"2023-10-21T16:21:28.459447+0000","last_active":"2023-10-21T16:21:29.430117+0000","last_peered":"2023-10-21T16:21:29.430117+0000","last_clean":"2023-10-21T16:21:29.430117+0000","last_became_active":"2023-10-21T16:21:28.459165+0000","last_became_peered":"2023-10-21T16:21:28.459165+0000","last_unstale":"2023-10-21T16:21:29.430117+0000","last_undegraded":"2023-10-21T16:21:29.430117+0000","last_fullsized":"2023-10-21T16:21:29.430117+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:02:56.829596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502881+0000","last_change":"2023-10-21T16:21:28.481949+0000","last_active":"2023-10-21T16:21:29.502881+0000","last_peered":"2023-10-21T16:21:29.502881+0000","last_clean":"2023-10-21T16:21:29.502881+0000","last_became_active":"2023-10-21T16:21:28.479257+0000","last_became_peered":"2023-10-21T16:21:28.479257+0000","last_unstale":"2023-10-21T16:21:29.502881+0000","last_undegraded":"2023-10-21T16:21:29.502881+0000","last_fullsized":"2023-10-21T16:21:29.502881+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:52.085326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430301+0000","last_change":"2023-10-21T16:21:28.452071+0000","last_active":"2023-10-21T16:21:29.430301+0000","last_peered":"2023-10-21T16:21:29.430301+0000","last_clean":"2023-10-21T16:21:29.430301+0000","last_became_active":"2023-10-21T16:21:28.445206+0000","last_became_peered":"2023-10-21T16:21:28.445206+0000","last_unstale":"2023-10-21T16:21:29.430301+0000","last_undegraded":"2023-10-21T16:21:29.430301+0000","last_fullsized":"2023-10-21T16:21:29.430301+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:51.169048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429894+0000","last_change":"2023-10-21T16:21:28.453071+0000","last_active":"2023-10-21T16:21:29.429894+0000","last_peered":"2023-10-21T16:21:29.429894+0000","last_clean":"2023-10-21T16:21:29.429894+0000","last_became_active":"2023-10-21T16:21:28.452138+0000","last_became_peered":"2023-10-21T16:21:28.452138+0000","last_unstale":"2023-10-21T16:21:29.429894+0000","last_undegraded":"2023-10-21T16:21:29.429894+0000","last_fullsized":"2023-10-21T16:21:29.429894+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:45.773032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499923+0000","last_change":"2023-10-21T16:21:28.458223+0000","last_active":"2023-10-21T16:21:29.499923+0000","last_peered":"2023-10-21T16:21:29.499923+0000","last_clean":"2023-10-21T16:21:29.499923+0000","last_became_active":"2023-10-21T16:21:28.457205+0000","last_became_peered":"2023-10-21T16:21:28.457205+0000","last_unstale":"2023-10-21T16:21:29.499923+0000","last_undegraded":"2023-10-21T16:21:29.499923+0000","last_fullsized":"2023-10-21T16:21:29.499923+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:06.900826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"147'4","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502075+0000","last_change":"2023-10-21T16:21:27.435665+0000","last_active":"2023-10-21T16:21:29.502075+0000","last_peered":"2023-10-21T16:21:29.502075+0000","last_clean":"2023-10-21T16:21:29.502075+0000","last_became_active":"2023-10-21T16:21:27.435385+0000","last_became_peered":"2023-10-21T16:21:27.435385+0000","last_unstale":"2023-10-21T16:21:29.502075+0000","last_undegraded":"2023-10-21T16:21:29.502075+0000","last_fullsized":"2023-10-21T16:21:29.502075+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:20:13.704884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502279+0000","last_change":"2023-10-21T16:21:28.466007+0000","last_active":"2023-10-21T16:21:29.502279+0000","last_peered":"2023-10-21T16:21:29.502279+0000","last_clean":"2023-10-21T16:21:29.502279+0000","last_became_active":"2023-10-21T16:21:28.463978+0000","last_became_peered":"2023-10-21T16:21:28.463978+0000","last_unstale":"2023-10-21T16:21:29.502279+0000","last_undegraded":"2023-10-21T16:21:29.502279+0000","last_fullsized":"2023-10-21T16:21:29.502279+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:37:38.583264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500460+0000","last_change":"2023-10-21T16:21:27.440811+0000","last_active":"2023-10-21T16:21:29.500460+0000","last_peered":"2023-10-21T16:21:29.500460+0000","last_clean":"2023-10-21T16:21:29.500460+0000","last_became_active":"2023-10-21T16:21:27.439874+0000","last_became_peered":"2023-10-21T16:21:27.439874+0000","last_unstale":"2023-10-21T16:21:29.500460+0000","last_undegraded":"2023-10-21T16:21:29.500460+0000","last_fullsized":"2023-10-21T16:21:29.500460+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:21.717995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431031+0000","last_change":"2023-10-21T16:21:28.462746+0000","last_active":"2023-10-21T16:21:29.431031+0000","last_peered":"2023-10-21T16:21:29.431031+0000","last_clean":"2023-10-21T16:21:29.431031+0000","last_became_active":"2023-10-21T16:21:28.461427+0000","last_became_peered":"2023-10-21T16:21:28.461427+0000","last_unstale":"2023-10-21T16:21:29.431031+0000","last_undegraded":"2023-10-21T16:21:29.431031+0000","last_fullsized":"2023-10-21T16:21:29.431031+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:28.462096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501130+0000","last_change":"2023-10-21T16:21:27.438295+0000","last_active":"2023-10-21T16:21:29.501130+0000","last_peered":"2023-10-21T16:21:29.501130+0000","last_clean":"2023-10-21T16:21:29.501130+0000","last_became_active":"2023-10-21T16:21:27.438047+0000","last_became_peered":"2023-10-21T16:21:27.438047+0000","last_unstale":"2023-10-21T16:21:29.501130+0000","last_undegraded":"2023-10-21T16:21:29.501130+0000","last_fullsized":"2023-10-21T16:21:29.501130+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:27:18.942149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501509+0000","last_change":"2023-10-21T16:21:28.476113+0000","last_active":"2023-10-21T16:21:29.501509+0000","last_peered":"2023-10-21T16:21:29.501509+0000","last_clean":"2023-10-21T16:21:29.501509+0000","last_became_active":"2023-10-21T16:21:28.475764+0000","last_became_peered":"2023-10-21T16:21:28.475764+0000","last_unstale":"2023-10-21T16:21:29.501509+0000","last_undegraded":"2023-10-21T16:21:29.501509+0000","last_fullsized":"2023-10-21T16:21:29.501509+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:52:50.922190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499672+0000","last_change":"2023-10-21T16:21:27.433269+0000","last_active":"2023-10-21T16:21:29.499672+0000","last_peered":"2023-10-21T16:21:29.499672+0000","last_clean":"2023-10-21T16:21:29.499672+0000","last_became_active":"2023-10-21T16:21:27.428115+0000","last_became_peered":"2023-10-21T16:21:27.428115+0000","last_unstale":"2023-10-21T16:21:29.499672+0000","last_undegraded":"2023-10-21T16:21:29.499672+0000","last_fullsized":"2023-10-21T16:21:29.499672+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:16.581883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.498921+0000","last_change":"2023-10-21T16:21:28.457672+0000","last_active":"2023-10-21T16:21:29.498921+0000","last_peered":"2023-10-21T16:21:29.498921+0000","last_clean":"2023-10-21T16:21:29.498921+0000","last_became_active":"2023-10-21T16:21:28.456834+0000","last_became_peered":"2023-10-21T16:21:28.456834+0000","last_unstale":"2023-10-21T16:21:29.498921+0000","last_undegraded":"2023-10-21T16:21:29.498921+0000","last_fullsized":"2023-10-21T16:21:29.498921+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:44.282263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434223+0000","last_change":"2023-10-21T16:21:27.425715+0000","last_active":"2023-10-21T16:21:29.434223+0000","last_peered":"2023-10-21T16:21:29.434223+0000","last_clean":"2023-10-21T16:21:29.434223+0000","last_became_active":"2023-10-21T16:21:27.425404+0000","last_became_peered":"2023-10-21T16:21:27.425404+0000","last_unstale":"2023-10-21T16:21:29.434223+0000","last_undegraded":"2023-10-21T16:21:29.434223+0000","last_fullsized":"2023-10-21T16:21:29.434223+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:13:28.822322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.433821+0000","last_change":"2023-10-21T16:21:28.477998+0000","last_active":"2023-10-21T16:21:29.433821+0000","last_peered":"2023-10-21T16:21:29.433821+0000","last_clean":"2023-10-21T16:21:29.433821+0000","last_became_active":"2023-10-21T16:21:28.477638+0000","last_became_peered":"2023-10-21T16:21:28.477638+0000","last_unstale":"2023-10-21T16:21:29.433821+0000","last_undegraded":"2023-10-21T16:21:29.433821+0000","last_fullsized":"2023-10-21T16:21:29.433821+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:24:13.975919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502160+0000","last_change":"2023-10-21T16:21:27.440307+0000","last_active":"2023-10-21T16:21:29.502160+0000","last_peered":"2023-10-21T16:21:29.502160+0000","last_clean":"2023-10-21T16:21:29.502160+0000","last_became_active":"2023-10-21T16:21:27.440005+0000","last_became_peered":"2023-10-21T16:21:27.440005+0000","last_unstale":"2023-10-21T16:21:29.502160+0000","last_undegraded":"2023-10-21T16:21:29.502160+0000","last_fullsized":"2023-10-21T16:21:29.502160+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.223742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503076+0000","last_change":"2023-10-21T16:21:28.462540+0000","last_active":"2023-10-21T16:21:29.503076+0000","last_peered":"2023-10-21T16:21:29.503076+0000","last_clean":"2023-10-21T16:21:29.503076+0000","last_became_active":"2023-10-21T16:21:28.462142+0000","last_became_peered":"2023-10-21T16:21:28.462142+0000","last_unstale":"2023-10-21T16:21:29.503076+0000","last_undegraded":"2023-10-21T16:21:29.503076+0000","last_fullsized":"2023-10-21T16:21:29.503076+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:50.951577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430546+0000","last_change":"2023-10-21T16:21:27.433035+0000","last_active":"2023-10-21T16:21:29.430546+0000","last_peered":"2023-10-21T16:21:29.430546+0000","last_clean":"2023-10-21T16:21:29.430546+0000","last_became_active":"2023-10-21T16:21:27.432711+0000","last_became_peered":"2023-10-21T16:21:27.432711+0000","last_unstale":"2023-10-21T16:21:29.430546+0000","last_undegraded":"2023-10-21T16:21:29.430546+0000","last_fullsized":"2023-10-21T16:21:29.430546+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:05.698566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499283+0000","last_change":"2023-10-21T16:21:28.463539+0000","last_active":"2023-10-21T16:21:29.499283+0000","last_peered":"2023-10-21T16:21:29.499283+0000","last_clean":"2023-10-21T16:21:29.499283+0000","last_became_active":"2023-10-21T16:21:28.463248+0000","last_became_peered":"2023-10-21T16:21:28.463248+0000","last_unstale":"2023-10-21T16:21:29.499283+0000","last_undegraded":"2023-10-21T16:21:29.499283+0000","last_fullsized":"2023-10-21T16:21:29.499283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:53:09.450501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500689+0000","last_change":"2023-10-21T16:21:27.428365+0000","last_active":"2023-10-21T16:21:29.500689+0000","last_peered":"2023-10-21T16:21:29.500689+0000","last_clean":"2023-10-21T16:21:29.500689+0000","last_became_active":"2023-10-21T16:21:27.427866+0000","last_became_peered":"2023-10-21T16:21:27.427866+0000","last_unstale":"2023-10-21T16:21:29.500689+0000","last_undegraded":"2023-10-21T16:21:29.500689+0000","last_fullsized":"2023-10-21T16:21:29.500689+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:59.272857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502222+0000","last_change":"2023-10-21T16:21:28.474740+0000","last_active":"2023-10-21T16:21:29.502222+0000","last_peered":"2023-10-21T16:21:29.502222+0000","last_clean":"2023-10-21T16:21:29.502222+0000","last_became_active":"2023-10-21T16:21:28.474136+0000","last_became_peered":"2023-10-21T16:21:28.474136+0000","last_unstale":"2023-10-21T16:21:29.502222+0000","last_undegraded":"2023-10-21T16:21:29.502222+0000","last_fullsized":"2023-10-21T16:21:29.502222+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:43.228418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502316+0000","last_change":"2023-10-21T16:21:27.437083+0000","last_active":"2023-10-21T16:21:29.502316+0000","last_peered":"2023-10-21T16:21:29.502316+0000","last_clean":"2023-10-21T16:21:29.502316+0000","last_became_active":"2023-10-21T16:21:27.436717+0000","last_became_peered":"2023-10-21T16:21:27.436717+0000","last_unstale":"2023-10-21T16:21:29.502316+0000","last_undegraded":"2023-10-21T16:21:29.502316+0000","last_fullsized":"2023-10-21T16:21:29.502316+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:28:29.554812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431413+0000","last_change":"2023-10-21T16:21:28.465103+0000","last_active":"2023-10-21T16:21:29.431413+0000","last_peered":"2023-10-21T16:21:29.431413+0000","last_clean":"2023-10-21T16:21:29.431413+0000","last_became_active":"2023-10-21T16:21:28.463554+0000","last_became_peered":"2023-10-21T16:21:28.463554+0000","last_unstale":"2023-10-21T16:21:29.431413+0000","last_undegraded":"2023-10-21T16:21:29.431413+0000","last_fullsized":"2023-10-21T16:21:29.431413+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:10.183230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431253+0000","last_change":"2023-10-21T16:21:27.428146+0000","last_active":"2023-10-21T16:21:29.431253+0000","last_peered":"2023-10-21T16:21:29.431253+0000","last_clean":"2023-10-21T16:21:29.431253+0000","last_became_active":"2023-10-21T16:21:27.427541+0000","last_became_peered":"2023-10-21T16:21:27.427541+0000","last_unstale":"2023-10-21T16:21:29.431253+0000","last_undegraded":"2023-10-21T16:21:29.431253+0000","last_fullsized":"2023-10-21T16:21:29.431253+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:05.672003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503120+0000","last_change":"2023-10-21T16:21:28.465818+0000","last_active":"2023-10-21T16:21:29.503120+0000","last_peered":"2023-10-21T16:21:29.503120+0000","last_clean":"2023-10-21T16:21:29.503120+0000","last_became_active":"2023-10-21T16:21:28.465470+0000","last_became_peered":"2023-10-21T16:21:28.465470+0000","last_unstale":"2023-10-21T16:21:29.503120+0000","last_undegraded":"2023-10-21T16:21:29.503120+0000","last_fullsized":"2023-10-21T16:21:29.503120+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:00.999817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"147'3","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431317+0000","last_change":"2023-10-21T16:21:27.433019+0000","last_active":"2023-10-21T16:21:29.431317+0000","last_peered":"2023-10-21T16:21:29.431317+0000","last_clean":"2023-10-21T16:21:29.431317+0000","last_became_active":"2023-10-21T16:21:27.429603+0000","last_became_peered":"2023-10-21T16:21:27.429603+0000","last_unstale":"2023-10-21T16:21:29.431317+0000","last_undegraded":"2023-10-21T16:21:29.431317+0000","last_fullsized":"2023-10-21T16:21:29.431317+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:38.704039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.431931+0000","last_change":"2023-10-21T16:21:28.444626+0000","last_active":"2023-10-21T16:21:29.431931+0000","last_peered":"2023-10-21T16:21:29.431931+0000","last_clean":"2023-10-21T16:21:29.431931+0000","last_became_active":"2023-10-21T16:21:28.443817+0000","last_became_peered":"2023-10-21T16:21:28.443817+0000","last_unstale":"2023-10-21T16:21:29.431931+0000","last_undegraded":"2023-10-21T16:21:29.431931+0000","last_fullsized":"2023-10-21T16:21:29.431931+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:21.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"147'6","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503191+0000","last_change":"2023-10-21T16:21:27.435636+0000","last_active":"2023-10-21T16:21:29.503191+0000","last_peered":"2023-10-21T16:21:29.503191+0000","last_clean":"2023-10-21T16:21:29.503191+0000","last_became_active":"2023-10-21T16:21:27.433316+0000","last_became_peered":"2023-10-21T16:21:27.433316+0000","last_unstale":"2023-10-21T16:21:29.503191+0000","last_undegraded":"2023-10-21T16:21:29.503191+0000","last_fullsized":"2023-10-21T16:21:29.503191+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:41.436043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502139+0000","last_change":"2023-10-21T16:21:28.462713+0000","last_active":"2023-10-21T16:21:29.502139+0000","last_peered":"2023-10-21T16:21:29.502139+0000","last_clean":"2023-10-21T16:21:29.502139+0000","last_became_active":"2023-10-21T16:21:28.460837+0000","last_became_peered":"2023-10-21T16:21:28.460837+0000","last_unstale":"2023-10-21T16:21:29.502139+0000","last_undegraded":"2023-10-21T16:21:29.502139+0000","last_fullsized":"2023-10-21T16:21:29.502139+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:59.844950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502690+0000","last_change":"2023-10-21T16:21:27.436315+0000","last_active":"2023-10-21T16:21:29.502690+0000","last_peered":"2023-10-21T16:21:29.502690+0000","last_clean":"2023-10-21T16:21:29.502690+0000","last_became_active":"2023-10-21T16:21:27.436034+0000","last_became_peered":"2023-10-21T16:21:27.436034+0000","last_unstale":"2023-10-21T16:21:29.502690+0000","last_undegraded":"2023-10-21T16:21:29.502690+0000","last_fullsized":"2023-10-21T16:21:29.502690+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:19.022583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499032+0000","last_change":"2023-10-21T16:21:28.467651+0000","last_active":"2023-10-21T16:21:29.499032+0000","last_peered":"2023-10-21T16:21:29.499032+0000","last_clean":"2023-10-21T16:21:29.499032+0000","last_became_active":"2023-10-21T16:21:28.467321+0000","last_became_peered":"2023-10-21T16:21:28.467321+0000","last_unstale":"2023-10-21T16:21:29.499032+0000","last_undegraded":"2023-10-21T16:21:29.499032+0000","last_fullsized":"2023-10-21T16:21:29.499032+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:48:22.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503444+0000","last_change":"2023-10-21T16:21:27.427412+0000","last_active":"2023-10-21T16:21:29.503444+0000","last_peered":"2023-10-21T16:21:29.503444+0000","last_clean":"2023-10-21T16:21:29.503444+0000","last_became_active":"2023-10-21T16:21:27.427155+0000","last_became_peered":"2023-10-21T16:21:27.427155+0000","last_unstale":"2023-10-21T16:21:29.503444+0000","last_undegraded":"2023-10-21T16:21:29.503444+0000","last_fullsized":"2023-10-21T16:21:29.503444+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:59.385541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429696+0000","last_change":"2023-10-21T16:21:28.475400+0000","last_active":"2023-10-21T16:21:29.429696+0000","last_peered":"2023-10-21T16:21:29.429696+0000","last_clean":"2023-10-21T16:21:29.429696+0000","last_became_active":"2023-10-21T16:21:28.474687+0000","last_became_peered":"2023-10-21T16:21:28.474687+0000","last_unstale":"2023-10-21T16:21:29.429696+0000","last_undegraded":"2023-10-21T16:21:29.429696+0000","last_fullsized":"2023-10-21T16:21:29.429696+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:07.547939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501765+0000","last_change":"2023-10-21T16:21:27.438057+0000","last_active":"2023-10-21T16:21:29.501765+0000","last_peered":"2023-10-21T16:21:29.501765+0000","last_clean":"2023-10-21T16:21:29.501765+0000","last_became_active":"2023-10-21T16:21:27.432472+0000","last_became_peered":"2023-10-21T16:21:27.432472+0000","last_unstale":"2023-10-21T16:21:29.501765+0000","last_undegraded":"2023-10-21T16:21:29.501765+0000","last_fullsized":"2023-10-21T16:21:29.501765+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.223742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430906+0000","last_change":"2023-10-21T16:21:28.461803+0000","last_active":"2023-10-21T16:21:29.430906+0000","last_peered":"2023-10-21T16:21:29.430906+0000","last_clean":"2023-10-21T16:21:29.430906+0000","last_became_active":"2023-10-21T16:21:28.461496+0000","last_became_peered":"2023-10-21T16:21:28.461496+0000","last_unstale":"2023-10-21T16:21:29.430906+0000","last_undegraded":"2023-10-21T16:21:29.430906+0000","last_fullsized":"2023-10-21T16:21:29.430906+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:28.335807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500365+0000","last_change":"2023-10-21T16:21:27.438947+0000","last_active":"2023-10-21T16:21:29.500365+0000","last_peered":"2023-10-21T16:21:29.500365+0000","last_clean":"2023-10-21T16:21:29.500365+0000","last_became_active":"2023-10-21T16:21:27.437970+0000","last_became_peered":"2023-10-21T16:21:27.437970+0000","last_unstale":"2023-10-21T16:21:29.500365+0000","last_undegraded":"2023-10-21T16:21:29.500365+0000","last_fullsized":"2023-10-21T16:21:29.500365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:27.973921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434771+0000","last_change":"2023-10-21T16:21:28.466042+0000","last_active":"2023-10-21T16:21:29.434771+0000","last_peered":"2023-10-21T16:21:29.434771+0000","last_clean":"2023-10-21T16:21:29.434771+0000","last_became_active":"2023-10-21T16:21:28.465574+0000","last_became_peered":"2023-10-21T16:21:28.465574+0000","last_unstale":"2023-10-21T16:21:29.434771+0000","last_undegraded":"2023-10-21T16:21:29.434771+0000","last_fullsized":"2023-10-21T16:21:29.434771+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:58:06.912109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.429977+0000","last_change":"2023-10-21T16:21:27.431513+0000","last_active":"2023-10-21T16:21:29.429977+0000","last_peered":"2023-10-21T16:21:29.429977+0000","last_clean":"2023-10-21T16:21:29.429977+0000","last_became_active":"2023-10-21T16:21:27.429289+0000","last_became_peered":"2023-10-21T16:21:27.429289+0000","last_unstale":"2023-10-21T16:21:29.429977+0000","last_undegraded":"2023-10-21T16:21:29.429977+0000","last_fullsized":"2023-10-21T16:21:29.429977+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:27.856986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502075+0000","last_change":"2023-10-21T16:21:28.460162+0000","last_active":"2023-10-21T16:21:29.502075+0000","last_peered":"2023-10-21T16:21:29.502075+0000","last_clean":"2023-10-21T16:21:29.502075+0000","last_became_active":"2023-10-21T16:21:28.459920+0000","last_became_peered":"2023-10-21T16:21:28.459920+0000","last_unstale":"2023-10-21T16:21:29.502075+0000","last_undegraded":"2023-10-21T16:21:29.502075+0000","last_fullsized":"2023-10-21T16:21:29.502075+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:03.275697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"147'1","reported_seq":13,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501215+0000","last_change":"2023-10-21T16:21:27.438253+0000","last_active":"2023-10-21T16:21:29.501215+0000","last_peered":"2023-10-21T16:21:29.501215+0000","last_clean":"2023-10-21T16:21:29.501215+0000","last_became_active":"2023-10-21T16:21:27.437990+0000","last_became_peered":"2023-10-21T16:21:27.437990+0000","last_unstale":"2023-10-21T16:21:29.501215+0000","last_undegraded":"2023-10-21T16:21:29.501215+0000","last_fullsized":"2023-10-21T16:21:29.501215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:50.554279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.432083+0000","last_change":"2023-10-21T16:21:28.470573+0000","last_active":"2023-10-21T16:21:29.432083+0000","last_peered":"2023-10-21T16:21:29.432083+0000","last_clean":"2023-10-21T16:21:29.432083+0000","last_became_active":"2023-10-21T16:21:28.470136+0000","last_became_peered":"2023-10-21T16:21:28.470136+0000","last_unstale":"2023-10-21T16:21:29.432083+0000","last_undegraded":"2023-10-21T16:21:29.432083+0000","last_fullsized":"2023-10-21T16:21:29.432083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:23:02.745460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500770+0000","last_change":"2023-10-21T16:21:27.443444+0000","last_active":"2023-10-21T16:21:29.500770+0000","last_peered":"2023-10-21T16:21:29.500770+0000","last_clean":"2023-10-21T16:21:29.500770+0000","last_became_active":"2023-10-21T16:21:27.443171+0000","last_became_peered":"2023-10-21T16:21:27.443171+0000","last_unstale":"2023-10-21T16:21:29.500770+0000","last_undegraded":"2023-10-21T16:21:29.500770+0000","last_fullsized":"2023-10-21T16:21:29.500770+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:03.672987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500616+0000","last_change":"2023-10-21T16:21:28.460162+0000","last_active":"2023-10-21T16:21:29.500616+0000","last_peered":"2023-10-21T16:21:29.500616+0000","last_clean":"2023-10-21T16:21:29.500616+0000","last_became_active":"2023-10-21T16:21:28.459859+0000","last_became_peered":"2023-10-21T16:21:28.459859+0000","last_unstale":"2023-10-21T16:21:29.500616+0000","last_undegraded":"2023-10-21T16:21:29.500616+0000","last_fullsized":"2023-10-21T16:21:29.500616+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:44:39.608332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501343+0000","last_change":"2023-10-21T16:21:27.433574+0000","last_active":"2023-10-21T16:21:29.501343+0000","last_peered":"2023-10-21T16:21:29.501343+0000","last_clean":"2023-10-21T16:21:29.501343+0000","last_became_active":"2023-10-21T16:21:27.432651+0000","last_became_peered":"2023-10-21T16:21:27.432651+0000","last_unstale":"2023-10-21T16:21:29.501343+0000","last_undegraded":"2023-10-21T16:21:29.501343+0000","last_fullsized":"2023-10-21T16:21:29.501343+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:09.166403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501352+0000","last_change":"2023-10-21T16:21:28.472638+0000","last_active":"2023-10-21T16:21:29.501352+0000","last_peered":"2023-10-21T16:21:29.501352+0000","last_clean":"2023-10-21T16:21:29.501352+0000","last_became_active":"2023-10-21T16:21:28.472351+0000","last_became_peered":"2023-10-21T16:21:28.472351+0000","last_unstale":"2023-10-21T16:21:29.501352+0000","last_undegraded":"2023-10-21T16:21:29.501352+0000","last_fullsized":"2023-10-21T16:21:29.501352+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:04.393797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499936+0000","last_change":"2023-10-21T16:21:27.440868+0000","last_active":"2023-10-21T16:21:29.499936+0000","last_peered":"2023-10-21T16:21:29.499936+0000","last_clean":"2023-10-21T16:21:29.499936+0000","last_became_active":"2023-10-21T16:21:27.440392+0000","last_became_peered":"2023-10-21T16:21:27.440392+0000","last_unstale":"2023-10-21T16:21:29.499936+0000","last_undegraded":"2023-10-21T16:21:29.499936+0000","last_fullsized":"2023-10-21T16:21:29.499936+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:27:55.557534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.503329+0000","last_change":"2023-10-21T16:21:28.466930+0000","last_active":"2023-10-21T16:21:29.503329+0000","last_peered":"2023-10-21T16:21:29.503329+0000","last_clean":"2023-10-21T16:21:29.503329+0000","last_became_active":"2023-10-21T16:21:28.466524+0000","last_became_peered":"2023-10-21T16:21:28.466524+0000","last_unstale":"2023-10-21T16:21:29.503329+0000","last_undegraded":"2023-10-21T16:21:29.503329+0000","last_fullsized":"2023-10-21T16:21:29.503329+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:07.571601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501582+0000","last_change":"2023-10-21T16:21:27.438818+0000","last_active":"2023-10-21T16:21:29.501582+0000","last_peered":"2023-10-21T16:21:29.501582+0000","last_clean":"2023-10-21T16:21:29.501582+0000","last_became_active":"2023-10-21T16:21:27.437260+0000","last_became_peered":"2023-10-21T16:21:27.437260+0000","last_unstale":"2023-10-21T16:21:29.501582+0000","last_undegraded":"2023-10-21T16:21:29.501582+0000","last_fullsized":"2023-10-21T16:21:29.501582+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:02:31.545386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430821+0000","last_change":"2023-10-21T16:21:28.445852+0000","last_active":"2023-10-21T16:21:29.430821+0000","last_peered":"2023-10-21T16:21:29.430821+0000","last_clean":"2023-10-21T16:21:29.430821+0000","last_became_active":"2023-10-21T16:21:28.443973+0000","last_became_peered":"2023-10-21T16:21:28.443973+0000","last_unstale":"2023-10-21T16:21:29.430821+0000","last_undegraded":"2023-10-21T16:21:29.430821+0000","last_fullsized":"2023-10-21T16:21:29.430821+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:32.513715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499467+0000","last_change":"2023-10-21T16:21:27.436384+0000","last_active":"2023-10-21T16:21:29.499467+0000","last_peered":"2023-10-21T16:21:29.499467+0000","last_clean":"2023-10-21T16:21:29.499467+0000","last_became_active":"2023-10-21T16:21:27.436080+0000","last_became_peered":"2023-10-21T16:21:27.436080+0000","last_unstale":"2023-10-21T16:21:29.499467+0000","last_undegraded":"2023-10-21T16:21:29.499467+0000","last_fullsized":"2023-10-21T16:21:29.499467+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:44:05.464101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502472+0000","last_change":"2023-10-21T16:21:28.452536+0000","last_active":"2023-10-21T16:21:29.502472+0000","last_peered":"2023-10-21T16:21:29.502472+0000","last_clean":"2023-10-21T16:21:29.502472+0000","last_became_active":"2023-10-21T16:21:28.452255+0000","last_became_peered":"2023-10-21T16:21:28.452255+0000","last_unstale":"2023-10-21T16:21:29.502472+0000","last_undegraded":"2023-10-21T16:21:29.502472+0000","last_fullsized":"2023-10-21T16:21:29.502472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:45.344305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"147'2","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501874+0000","last_change":"2023-10-21T16:21:27.430030+0000","last_active":"2023-10-21T16:21:29.501874+0000","last_peered":"2023-10-21T16:21:29.501874+0000","last_clean":"2023-10-21T16:21:29.501874+0000","last_became_active":"2023-10-21T16:21:27.429201+0000","last_became_peered":"2023-10-21T16:21:27.429201+0000","last_unstale":"2023-10-21T16:21:29.501874+0000","last_undegraded":"2023-10-21T16:21:29.501874+0000","last_fullsized":"2023-10-21T16:21:29.501874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:05.770354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499609+0000","last_change":"2023-10-21T16:21:28.455273+0000","last_active":"2023-10-21T16:21:29.499609+0000","last_peered":"2023-10-21T16:21:29.499609+0000","last_clean":"2023-10-21T16:21:29.499609+0000","last_became_active":"2023-10-21T16:21:28.454965+0000","last_became_peered":"2023-10-21T16:21:28.454965+0000","last_unstale":"2023-10-21T16:21:29.499609+0000","last_undegraded":"2023-10-21T16:21:29.499609+0000","last_fullsized":"2023-10-21T16:21:29.499609+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:24.157553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502592+0000","last_change":"2023-10-21T16:21:27.435719+0000","last_active":"2023-10-21T16:21:29.502592+0000","last_peered":"2023-10-21T16:21:29.502592+0000","last_clean":"2023-10-21T16:21:29.502592+0000","last_became_active":"2023-10-21T16:21:27.435438+0000","last_became_peered":"2023-10-21T16:21:27.435438+0000","last_unstale":"2023-10-21T16:21:29.502592+0000","last_undegraded":"2023-10-21T16:21:29.502592+0000","last_fullsized":"2023-10-21T16:21:29.502592+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:46:57.396761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501567+0000","last_change":"2023-10-21T16:21:28.477469+0000","last_active":"2023-10-21T16:21:29.501567+0000","last_peered":"2023-10-21T16:21:29.501567+0000","last_clean":"2023-10-21T16:21:29.501567+0000","last_became_active":"2023-10-21T16:21:28.477197+0000","last_became_peered":"2023-10-21T16:21:28.477197+0000","last_unstale":"2023-10-21T16:21:29.501567+0000","last_undegraded":"2023-10-21T16:21:29.501567+0000","last_fullsized":"2023-10-21T16:21:29.501567+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:43:47.499014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502705+0000","last_change":"2023-10-21T16:21:27.437161+0000","last_active":"2023-10-21T16:21:29.502705+0000","last_peered":"2023-10-21T16:21:29.502705+0000","last_clean":"2023-10-21T16:21:29.502705+0000","last_became_active":"2023-10-21T16:21:27.436727+0000","last_became_peered":"2023-10-21T16:21:27.436727+0000","last_unstale":"2023-10-21T16:21:29.502705+0000","last_undegraded":"2023-10-21T16:21:29.502705+0000","last_fullsized":"2023-10-21T16:21:29.502705+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:35:17.742315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.430090+0000","last_change":"2023-10-21T16:21:28.444824+0000","last_active":"2023-10-21T16:21:29.430090+0000","last_peered":"2023-10-21T16:21:29.430090+0000","last_clean":"2023-10-21T16:21:29.430090+0000","last_became_active":"2023-10-21T16:21:28.443610+0000","last_became_peered":"2023-10-21T16:21:28.443610+0000","last_unstale":"2023-10-21T16:21:29.430090+0000","last_undegraded":"2023-10-21T16:21:29.430090+0000","last_fullsized":"2023-10-21T16:21:29.430090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:01.127571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.502509+0000","last_change":"2023-10-21T16:21:27.440652+0000","last_active":"2023-10-21T16:21:29.502509+0000","last_peered":"2023-10-21T16:21:29.502509+0000","last_clean":"2023-10-21T16:21:29.502509+0000","last_became_active":"2023-10-21T16:21:27.440374+0000","last_became_peered":"2023-10-21T16:21:27.440374+0000","last_unstale":"2023-10-21T16:21:29.502509+0000","last_undegraded":"2023-10-21T16:21:29.502509+0000","last_fullsized":"2023-10-21T16:21:29.502509+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:35.747414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500977+0000","last_change":"2023-10-21T16:21:28.466660+0000","last_active":"2023-10-21T16:21:29.500977+0000","last_peered":"2023-10-21T16:21:29.500977+0000","last_clean":"2023-10-21T16:21:29.500977+0000","last_became_active":"2023-10-21T16:21:28.466340+0000","last_became_peered":"2023-10-21T16:21:28.466340+0000","last_unstale":"2023-10-21T16:21:29.500977+0000","last_undegraded":"2023-10-21T16:21:29.500977+0000","last_fullsized":"2023-10-21T16:21:29.500977+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:50.951577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434427+0000","last_change":"2023-10-21T16:21:27.426963+0000","last_active":"2023-10-21T16:21:29.434427+0000","last_peered":"2023-10-21T16:21:29.434427+0000","last_clean":"2023-10-21T16:21:29.434427+0000","last_became_active":"2023-10-21T16:21:27.426636+0000","last_became_peered":"2023-10-21T16:21:27.426636+0000","last_unstale":"2023-10-21T16:21:29.434427+0000","last_undegraded":"2023-10-21T16:21:29.434427+0000","last_fullsized":"2023-10-21T16:21:29.434427+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:58:12.773907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501346+0000","last_change":"2023-10-21T16:21:28.466502+0000","last_active":"2023-10-21T16:21:29.501346+0000","last_peered":"2023-10-21T16:21:29.501346+0000","last_clean":"2023-10-21T16:21:29.501346+0000","last_became_active":"2023-10-21T16:21:28.465403+0000","last_became_peered":"2023-10-21T16:21:28.465403+0000","last_unstale":"2023-10-21T16:21:29.501346+0000","last_undegraded":"2023-10-21T16:21:29.501346+0000","last_fullsized":"2023-10-21T16:21:29.501346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:04.099050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.434062+0000","last_change":"2023-10-21T16:21:27.422560+0000","last_active":"2023-10-21T16:21:29.434062+0000","last_peered":"2023-10-21T16:21:29.434062+0000","last_clean":"2023-10-21T16:21:29.434062+0000","last_became_active":"2023-10-21T16:21:27.422120+0000","last_became_peered":"2023-10-21T16:21:27.422120+0000","last_unstale":"2023-10-21T16:21:29.434062+0000","last_undegraded":"2023-10-21T16:21:29.434062+0000","last_fullsized":"2023-10-21T16:21:29.434062+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:23.140726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499751+0000","last_change":"2023-10-21T16:21:28.469905+0000","last_active":"2023-10-21T16:21:29.499751+0000","last_peered":"2023-10-21T16:21:29.499751+0000","last_clean":"2023-10-21T16:21:29.499751+0000","last_became_active":"2023-10-21T16:21:28.469596+0000","last_became_peered":"2023-10-21T16:21:28.469596+0000","last_unstale":"2023-10-21T16:21:29.499751+0000","last_undegraded":"2023-10-21T16:21:29.499751+0000","last_fullsized":"2023-10-21T16:21:29.499751+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:52:38.888075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501405+0000","last_change":"2023-10-21T16:21:27.427053+0000","last_active":"2023-10-21T16:21:29.501405+0000","last_peered":"2023-10-21T16:21:29.501405+0000","last_clean":"2023-10-21T16:21:29.501405+0000","last_became_active":"2023-10-21T16:21:27.426754+0000","last_became_peered":"2023-10-21T16:21:27.426754+0000","last_unstale":"2023-10-21T16:21:29.501405+0000","last_undegraded":"2023-10-21T16:21:29.501405+0000","last_fullsized":"2023-10-21T16:21:29.501405+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:23.479236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499449+0000","last_change":"2023-10-21T16:21:28.468429+0000","last_active":"2023-10-21T16:21:29.499449+0000","last_peered":"2023-10-21T16:21:29.499449+0000","last_clean":"2023-10-21T16:21:29.499449+0000","last_became_active":"2023-10-21T16:21:28.468155+0000","last_became_peered":"2023-10-21T16:21:28.468155+0000","last_unstale":"2023-10-21T16:21:29.499449+0000","last_undegraded":"2023-10-21T16:21:29.499449+0000","last_fullsized":"2023-10-21T16:21:29.499449+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:13:29.839149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.499516+0000","last_change":"2023-10-21T16:21:27.438692+0000","last_active":"2023-10-21T16:21:29.499516+0000","last_peered":"2023-10-21T16:21:29.499516+0000","last_clean":"2023-10-21T16:21:29.499516+0000","last_became_active":"2023-10-21T16:21:27.437690+0000","last_became_peered":"2023-10-21T16:21:27.437690+0000","last_unstale":"2023-10-21T16:21:29.499516+0000","last_undegraded":"2023-10-21T16:21:29.499516+0000","last_fullsized":"2023-10-21T16:21:29.499516+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:55:43.391159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.500272+0000","last_change":"2023-10-21T16:21:28.471846+0000","last_active":"2023-10-21T16:21:29.500272+0000","last_peered":"2023-10-21T16:21:29.500272+0000","last_clean":"2023-10-21T16:21:29.500272+0000","last_became_active":"2023-10-21T16:21:28.471551+0000","last_became_peered":"2023-10-21T16:21:28.471551+0000","last_unstale":"2023-10-21T16:21:29.500272+0000","last_undegraded":"2023-10-21T16:21:29.500272+0000","last_fullsized":"2023-10-21T16:21:29.500272+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:27.396747+0000","last_clean_scrub_stamp":"2023-10-21T16:21:27.396747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:04:21.708139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"147'8","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-21T16:21:29.501757+0000","last_change":"2023-10-21T16:21:27.432039+0000","last_active":"2023-10-21T16:21:29.501757+0000","last_peered":"2023-10-21T16:21:29.501757+0000","last_clean":"2023-10-21T16:21:29.501757+0000","last_became_active":"2023-10-21T16:21:27.431767+0000","last_became_peered":"2023-10-21T16:21:27.431767+0000","last_unstale":"2023-10-21T16:21:29.501757+0000","last_undegraded":"2023-10-21T16:21:29.501757+0000","last_fullsized":"2023-10-21T16:21:29.501757+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:21:26.379919+0000","last_clean_scrub_stamp":"2023-10-21T16:21:26.379919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:36:29.944714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"136'4752","reported_seq":5261,"reported_epoch":147,"state":"active+clean","last_fresh":"2023-10-21T16:21:28.456279+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:21:28.456279+0000","last_peered":"2023-10-21T16:21:28.456279+0000","last_clean":"2023-10-21T16:21:28.456279+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:21:28.456279+0000","last_undegraded":"2023-10-21T16:21:28.456279+0000","last_fullsized":"2023-10-21T16:21:28.456279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4752,"log_dups_size":0,"ondisk_log_size":4752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":367,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1168,"num_read_kb":2373,"num_write":367,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4752,"ondisk_log_size":4752,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47448,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324392,"statfs":{"total":96636764160,"available":96588177408,"internally_reserved":0,"allocated":2437120,"data_stored":1520565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.075,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.60399999999999998,"15min":0.51300000000000001},"max":{"1min":1.6919999999999999,"5min":1.758,"15min":1.8400000000000001},"last":1.335},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.169},"min":{"1min":0.755,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.4930000000000001,"5min":1.7270000000000001,"15min":2.448},"last":1.377}]},{"osd":1,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.149,"15min":1.161},"min":{"1min":0.76100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.601,"5min":1.851,"15min":1.915},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1759999999999999,"15min":1.226},"min":{"1min":0.70299999999999996,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.4399999999999999}]},{"osd":2,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.2689999999999999,"15min":1.2989999999999999},"min":{"1min":0.79900000000000004,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.704,"5min":1.782,"15min":2.085},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.85199999999999998,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.837,"5min":1.837,"15min":2.6070000000000002},"last":0.72699999999999998}]},{"osd":3,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.206,"15min":1.248},"min":{"1min":0.878,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5880000000000001,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.236},{"interface":"front","average":{"1min":1.417,"5min":1.369,"15min":1.421},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.5780000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3149999999999999,"15min":1.3500000000000001},"min":{"1min":1.032,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.794,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.29},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4339999999999999,"15min":1.4379999999999999},"min":{"1min":1.3120000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.355},"last":1.415}]},{"osd":5,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4299999999999999,"15min":1.403},"min":{"1min":1.0249999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.518},{"interface":"front","average":{"1min":1.46,"5min":1.4359999999999999,"15min":1.4399999999999999},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.016,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.194}]},{"osd":6,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.532,"15min":1.5489999999999999},"min":{"1min":1.129,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.9890000000000001,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.4419999999999999,"15min":1.4570000000000001},"min":{"1min":0.95599999999999996,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":2.02,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":0.98899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575081,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37128,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334712,"statfs":{"total":96636764160,"available":96598745088,"internally_reserved":0,"allocated":1830912,"data_stored":930003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0860000000000001,"15min":1.1220000000000001},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.704,"5min":1.869,"15min":1.869},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.61299999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.895},"last":0.82199999999999995}]},{"osd":1,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1379999999999999,"15min":1.1899999999999999},"min":{"1min":0.80200000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.835,"5min":1.835,"15min":2.2309999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1870000000000001,"15min":1.2829999999999999},"min":{"1min":0.81499999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5309999999999999,"5min":1.97,"15min":2.2669999999999999},"last":1.1060000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2270000000000001,"15min":1.2509999999999999},"min":{"1min":0.90400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.726,"5min":1.895,"15min":2.5339999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4330000000000001,"5min":1.821,"15min":1.821},"last":0.91200000000000003}]},{"osd":3,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3129999999999999,"15min":1.3480000000000001},"min":{"1min":1.014,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8049999999999999,"5min":1.907,"15min":2.2000000000000002},"last":1.278},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.4159999999999999},"min":{"1min":0.93000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.845,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.323}]},{"osd":4,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3540000000000001,"15min":1.339},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6890000000000001,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.4299999999999999,"15min":1.395},"min":{"1min":0.74199999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.228}]},{"osd":5,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.5,"15min":1.502},"min":{"1min":0.86399999999999999,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":1.8899999999999999,"5min":2.246,"15min":2.246},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3460000000000001,"15min":1.3520000000000001},"min":{"1min":0.77300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.669,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.1970000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.476,"15min":1.5009999999999999},"min":{"1min":0.76300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9159999999999999,"5min":2.2050000000000001,"15min":2.3570000000000002},"last":1.468},{"interface":"front","average":{"1min":1.492,"5min":1.6060000000000001,"15min":1.609},"min":{"1min":0.95399999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.931,"5min":2.319,"15min":2.319},"last":1.667}]}]},{"osd":5,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51024,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320816,"statfs":{"total":96636764160,"available":96584515584,"internally_reserved":0,"allocated":1835008,"data_stored":929581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.101,"15min":1.1180000000000001},"min":{"1min":0.70299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4239999999999999,"5min":1.609,"15min":1.742},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.169,"15min":1.1850000000000001},"min":{"1min":0.72999999999999998,"5min":0.68100000000000005,"15min":0.52400000000000002},"max":{"1min":1.873,"5min":1.99,"15min":1.99},"last":1.4550000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.234,"15min":1.252},"min":{"1min":0.72699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6559999999999999,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.351},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.2130000000000001,"15min":1.2230000000000001},"min":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.415,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":1.157}]},{"osd":2,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.286,"15min":1.288},"min":{"1min":0.81100000000000005,"5min":0.624,"15min":0.57199999999999995},"max":{"1min":1.5660000000000001,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2150000000000001,"15min":1.2310000000000001},"min":{"1min":0.72799999999999998,"5min":0.53100000000000003,"15min":0.46999999999999997},"max":{"1min":1.748,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.3120000000000001,"15min":1.3129999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.57599999999999996},"max":{"1min":1.7509999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.607},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.377,"15min":1.3819999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.66800000000000004},"max":{"1min":1.8120000000000001,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.069}]},{"osd":4,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2629999999999999,"15min":1.218},"min":{"1min":0.58899999999999997,"5min":0.52600000000000002,"15min":0.48899999999999999},"max":{"1min":1.494,"5min":4.359,"15min":4.359},"last":1.302},{"interface":"front","average":{"1min":1.365,"5min":1.391,"15min":1.357},"min":{"1min":0.626,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.714,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.663}]},{"osd":6,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.4810000000000001,"15min":1.4470000000000001},"min":{"1min":0.89700000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6950000000000001,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.405},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4990000000000001,"15min":1.4790000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.60099999999999998},"max":{"1min":1.992,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.5029999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:32 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.536,"15min":1.506},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.58299999999999996},"max":{"1min":1.946,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.544},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.593,"15min":1.554},"min":{"1min":0.84899999999999998,"5min":0.76700000000000002,"15min":0.60999999999999999},"max":{"1min":2.0939999999999999,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.4870000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1839104,"data_stored":931033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.1459999999999999,"15min":1.151},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3899999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.212,"15min":1.2090000000000001},"min":{"1min":0.66500000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.726,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.23,"15min":1.2649999999999999},"min":{"1min":0.83299999999999996,"5min":0.73999999999999999,"15min":0.621},"max":{"1min":1.754,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.278,"15min":1.2569999999999999},"min":{"1min":0.81499999999999995,"5min":0.70199999999999996,"15min":0.64500000000000002},"max":{"1min":1.4390000000000001,"5min":7.484,"15min":7.484},"last":0.877}]},{"osd":2,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.248,"15min":1.26},"min":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4430000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.337,"15min":1.343},"min":{"1min":0.77000000000000002,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.6679999999999999,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.5369999999999999}]},{"osd":3,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.337,"15min":1.3240000000000001},"min":{"1min":0.68600000000000005,"5min":0.64800000000000002,"15min":0.60399999999999998},"max":{"1min":1.712,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.3899999999999999},"min":{"1min":0.85499999999999998,"5min":0.76900000000000002,"15min":0.72399999999999998},"max":{"1min":1.76,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.24}]},{"osd":5,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.323,"15min":1.304},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5740000000000001,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.399,"15min":1.3520000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.603,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.173}]},{"osd":6,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.4690000000000001,"15min":1.425},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.778,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.5980000000000001,"15min":1.5780000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7729999999999999,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.611}]},{"osd":7,"last update":"Sat Oct 21 16:20:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.6120000000000001,"15min":1.5149999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.752,"5min":12.458,"15min":12.458},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.6120000000000001,"15min":1.5620000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.871,"5min":3.9510000000000001,"15min":3.968},"last":1.506}]}]},{"osd":3,"up_from":13,"seq":55834575080,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41304,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330536,"statfs":{"total":96636764160,"available":96594468864,"internally_reserved":0,"allocated":1843200,"data_stored":931067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.879,"15min":0.93899999999999995},"min":{"1min":0.442,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4159999999999999,"5min":1.946,"15min":1.946},"last":0.52900000000000003},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1659999999999999},"min":{"1min":0.52300000000000002,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.04},"last":0.67500000000000004}]},{"osd":1,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":0.97399999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.589,"15min":2.173},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.228},"min":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8100000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0509999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.643,"5min":1.669,"15min":2.0049999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.077,"5min":1.135,"15min":1.202},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.46600000000000003},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.847},"last":0.85099999999999998}]},{"osd":4,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.319,"15min":1.393},"min":{"1min":0.70399999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5780000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.006},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3899999999999999},"min":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.5800000000000001,"5min":1.863,"15min":2.919},"last":1.1799999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.327,"15min":1.393},"min":{"1min":0.65900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.02},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4179999999999999,"15min":1.4870000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.75700000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.423},"last":1.097}]},{"osd":6,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4139999999999999,"15min":1.478},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.8400000000000001,"15min":2.6179999999999999},"last":1.079},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.419,"15min":1.508},"min":{"1min":0.75700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.704,"5min":1.8999999999999999,"15min":2.2290000000000001},"last":1.323}]},{"osd":7,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4950000000000001,"15min":1.5740000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3759999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.52,"15min":1.587},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.7989999999999999},"last":1.3520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46852,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324988,"statfs":{"total":96636764160,"available":96588787712,"internally_reserved":0,"allocated":1826816,"data_stored":929421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.857,"5min":1.857,"15min":1.859},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.92900000000000005,"15min":0.96999999999999997},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.28299999999999997},"max":{"1min":1.6259999999999999,"5min":1.905,"15min":2.0539999999999998},"last":0.871}]},{"osd":1,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0269999999999999,"15min":1.103},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.39500000000000002},"max":{"1min":1.5820000000000001,"5min":1.6890000000000001,"15min":1.9390000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.159,"15min":1.232},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6919999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.94099999999999995}]},{"osd":3,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0840000000000001,"15min":1.1160000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.897,"5min":1.9279999999999999,"15min":2.0710000000000002},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.276,"15min":1.347},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.895,"5min":2.069,"15min":2.069},"last":0.91100000000000003}]},{"osd":4,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.321,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.76600000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":2.1589999999999998},{"interface":"front","average":{"1min":1.319,"5min":1.3740000000000001,"15min":1.3700000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60199999999999998},"max":{"1min":1.95,"5min":2.278,"15min":2.6960000000000002},"last":1.95}]},{"osd":5,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.571,"5min":2.024,"15min":2.2730000000000001},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4630000000000001,"15min":1.506},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.024,"5min":2.1280000000000001,"15min":2.1349999999999998},"last":2.024}]},{"osd":6,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4650000000000001,"15min":1.478},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"max":{"1min":2.0600000000000001,"5min":2.444,"15min":2.6819999999999999},"last":2.0600000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78900000000000003},"max":{"1min":2.121,"5min":2.2669999999999999,"15min":2.552},"last":1.131}]},{"osd":7,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5049999999999999,"15min":1.5289999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.69999999999999996},"max":{"1min":1.8460000000000001,"5min":2.2130000000000001,"15min":2.4969999999999999},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.468,"15min":1.5049999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.3999999999999999},"last":1.3240000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37136,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334704,"statfs":{"total":96636764160,"available":96598736896,"internally_reserved":0,"allocated":1830912,"data_stored":929443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.90000000000000002},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.1100000000000001,"5min":1.806,"15min":1.905},"last":0.33700000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.0469999999999999,"15min":1.0680000000000001},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4990000000000001,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.651,"5min":2.222,"15min":2.2639999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.167},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":1.615,"5min":1.6910000000000001,"15min":1.974},"last":0.52100000000000002}]},{"osd":3,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.228,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.54800000000000004,"15min":0.51600000000000001},"max":{"1min":1.5649999999999999,"5min":1.911,"15min":1.9510000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.526,"5min":1.6879999999999999,"15min":1.9159999999999999},"last":0.499}]},{"osd":4,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.238,"15min":1.268},"min":{"1min":0.82999999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.379,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.403,"15min":1.4239999999999999},"min":{"1min":1.024,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.9019999999999999,"5min":2.145,"15min":2.2719999999999998},"last":0.59799999999999998}]},{"osd":5,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":1.2410000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.4590000000000001},"min":{"1min":1.2010000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.627,"5min":2.077,"15min":2.0880000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.482,"15min":1.49},"min":{"1min":0.85099999999999998,"5min":0.77500000000000002,"15min":0.68500000000000005},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.4079999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.417,"15min":1.4510000000000001},"min":{"1min":0.94499999999999995,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.746,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.63}]},{"osd":7,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5549999999999999,"15min":1.5940000000000001},"min":{"1min":1.1619999999999999,"5min":0.64600000000000002,"15min":0.59299999999999997},"max":{"1min":1.804,"5min":2.2530000000000001,"15min":2.3730000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.89000000000000001,"5min":0.74399999999999999,"15min":0.68600000000000005},"max":{"1min":2.1110000000000002,"5min":2.1850000000000001,"15min":2.367},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41888,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329952,"statfs":{"total":96636764160,"available":96593870848,"internally_reserved":0,"allocated":2437120,"data_stored":1526503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86099999999999999,"15min":0.90700000000000003},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":1.302,"5min":1.649,"15min":1.855},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.996,"15min":1.0549999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.589,"5min":1.589,"15min":1.667},"last":0.57699999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0509999999999999,"15min":1.105},"min":{"1min":0.45100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.679,"5min":1.708,"15min":1.7749999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.1339999999999999},"min":{"1min":0.48499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.5580000000000001,"5min":1.694,"15min":1.732},"last":0.71299999999999997}]},{"osd":3,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0069999999999999,"15min":1.0780000000000001},"min":{"1min":0.53800000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.6080000000000001,"5min":1.8160000000000001,"15min":2.1429999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.1930000000000001,"15min":1.2769999999999999},"min":{"1min":0.58399999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7969999999999999,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":0.995}]},{"osd":4,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3819999999999999,"15min":1.4039999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.9019999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4550000000000001,"15min":1.516},"min":{"1min":0.68500000000000005,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8029999999999999,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.351}]},{"osd":5,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.329,"15min":1.3660000000000001},"min":{"1min":0.66400000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.823,"5min":1.9419999999999999,"15min":2.137},"last":1.3},{"interface":"front","average":{"1min":1.425,"5min":1.3879999999999999,"15min":1.4790000000000001},"min":{"1min":0.63300000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.9259999999999999,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":1.427}]},{"osd":6,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.419,"15min":1.4630000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.954,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.4870000000000001,"15min":1.5720000000000001},"min":{"1min":0.72099999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.911,"5min":3.097,"15min":3.097},"last":1.2509999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:20:26 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.925,"5min":2.085,"15min":2.2959999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.464,"15min":1.5509999999999999},"min":{"1min":0.79000000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0600000000000001,"5min":2.077,"15min":2.3540000000000001},"last":1.387}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:21:33.474 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:21:33.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:21:33.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.755+0000 7f163ffff640 1 Processor -- start 2023-10-21T16:21:33.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.755+0000 7f163ffff640 1 -- start start 2023-10-21T16:21:33.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1640160090 0x7f16400d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f16400cdc20 0x7f16400d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400d2700 con 0x7f16400cf3a0 2023-10-21T16:21:33.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163ffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1640160470 con 0x7f16400cdc20 2023-10-21T16:21:33.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400d0690 con 0x7f1640160090 2023-10-21T16:21:33.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f16400cdc20 0x7f16400d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f16400cf3a0 0x7f16400b5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37404/0 (socket says 172.21.15.53:37404) 2023-10-21T16:21:33.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163f7fe640 1 -- 172.21.15.53:0/1382143106 learned_addr learned my addr 172.21.15.53:0/1382143106 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:33.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.757+0000 7f163effd640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1640160090 0x7f16400d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.758+0000 7f163e7fc640 1 -- 172.21.15.53:0/1382143106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1640160090 msgr2=0x7f16400d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:33.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.758+0000 7f163e7fc640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1640160090 0x7f16400d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.758+0000 7f163e7fc640 1 -- 172.21.15.53:0/1382143106 >> v1:172.21.15.53:6789/0 conn(0x7f16400cf3a0 legacy=0x7f16400b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.758+0000 7f163e7fc640 1 -- 172.21.15.53:0/1382143106 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400d0970 con 0x7f16400cdc20 2023-10-21T16:21:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.759+0000 7f163e7fc640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16400cdc20 0x7f16400d1450 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1628009d90 tx=0x7f162802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982885a9b81f08c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.759+0000 7f163dffb640 1 -- 172.21.15.53:0/1382143106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162802cbc0 con 0x7f16400cdc20 2023-10-21T16:21:33.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.759+0000 7f163dffb640 1 -- 172.21.15.53:0/1382143106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162802cd20 con 0x7f16400cdc20 2023-10-21T16:21:33.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.759+0000 7f163dffb640 1 -- 172.21.15.53:0/1382143106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628034660 con 0x7f16400cdc20 2023-10-21T16:21:33.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 -- 172.21.15.53:0/1382143106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16400cdc20 msgr2=0x7f16400d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16400cdc20 0x7f16400d1450 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1628009d90 tx=0x7f162802be10 comp rx=0 tx=0).stop 2023-10-21T16:21:33.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 -- 172.21.15.53:0/1382143106 shutdown_connections 2023-10-21T16:21:33.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16400cdc20 0x7f16400d1450 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 --2- 172.21.15.53:0/1382143106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1640160090 0x7f16400d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 -- 172.21.15.53:0/1382143106 >> 172.21.15.53:0/1382143106 conn(0x7f164006af50 msgr2=0x7f164006b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 -- 172.21.15.53:0/1382143106 shutdown_connections 2023-10-21T16:21:33.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.760+0000 7f163ffff640 1 -- 172.21.15.53:0/1382143106 wait complete. 2023-10-21T16:21:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.762+0000 7f163ffff640 1 Processor -- start 2023-10-21T16:21:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.762+0000 7f163ffff640 1 -- start start 2023-10-21T16:21:33.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16400cdc20 0x7f16401f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16400cf3a0 0x7f16400a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1640153a70 con 0x7f1640160090 2023-10-21T16:21:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16401f20d0 con 0x7f16400cdc20 2023-10-21T16:21:33.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.763+0000 7f163ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400b3850 con 0x7f16400cf3a0 2023-10-21T16:21:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163effd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16400cdc20 0x7f16401f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49786/0 (socket says 172.21.15.53:49786) 2023-10-21T16:21:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 -- 172.21.15.53:0/1421543977 learned_addr learned my addr 172.21.15.53:0/1421543977 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163e7fc640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16400cf3a0 0x7f16400a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 -- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16400cf3a0 msgr2=0x7f16400a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16400cf3a0 0x7f16400a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 -- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16400cdc20 msgr2=0x7f16401f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16400cdc20 0x7f16401f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.764+0000 7f163f7fe640 1 -- 172.21.15.53:0/1421543977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628009a70 con 0x7f1640160090 2023-10-21T16:21:33.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f163f7fe640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f163800c450 tx=0x7f163800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638005ce0 con 0x7f1640160090 2023-10-21T16:21:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638005e40 con 0x7f1640160090 2023-10-21T16:21:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638007270 con 0x7f1640160090 2023-10-21T16:21:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16400b3b30 con 0x7f1640160090 2023-10-21T16:21:33.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.765+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16400b40f0 con 0x7f1640160090 2023-10-21T16:21:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.766+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16400c96a0 con 0x7f1640160090 2023-10-21T16:21:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.768+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16380073d0 con 0x7f1640160090 2023-10-21T16:21:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.768+0000 7f16377fe640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 0x7f162006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:33.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.769+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1638088dc0 con 0x7f1640160090 2023-10-21T16:21:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.769+0000 7f163effd640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 0x7f162006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.770+0000 7f163effd640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 0x7f162006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:33.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:33.773+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1638053be0 con 0x7f1640160090 2023-10-21T16:21:34.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.041+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f164004e160 con 0x7f1640160090 2023-10-21T16:21:34.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.045+0000 7f16377fe640 1 -- 172.21.15.53:0/1421543977 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1638017020 con 0x7f1640160090 2023-10-21T16:21:34.047 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:21:34.048 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:21:34.048 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:21:34.048 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:21:34.048 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:21:34.048 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:21:34.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 msgr2=0x7f162006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 0x7f162006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1630006fd0 tx=0x7f1630008040 comp rx=0 tx=0).stop 2023-10-21T16:21:34.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 msgr2=0x7f1640150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f163800c450 tx=0x7f163800c910 comp rx=0 tx=0).stop 2023-10-21T16:21:34.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 shutdown_connections 2023-10-21T16:21:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f162006af90 0x7f162006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1640160090 0x7f1640150b10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16400cf3a0 0x7f16400a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 --2- 172.21.15.53:0/1421543977 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16400cdc20 0x7f16401f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 >> 172.21.15.53:0/1421543977 conn(0x7f164006af50 msgr2=0x7f16400cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 shutdown_connections 2023-10-21T16:21:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.049+0000 7f163ffff640 1 -- 172.21.15.53:0/1421543977 wait complete. 2023-10-21T16:21:34.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:34.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.299+0000 7f372eabb640 1 Processor -- start 2023-10-21T16:21:34.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.299+0000 7f372eabb640 1 -- start start 2023-10-21T16:21:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372eabb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f372815ab40 0x7f37280ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372eabb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37280cb580 0x7f37280adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372eabb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372815b2b0 con 0x7f37280c90b0 2023-10-21T16:21:34.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372eabb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372815bf30 con 0x7f37280cb580 2023-10-21T16:21:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372eabb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372815f7a0 con 0x7f372815ab40 2023-10-21T16:21:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.301+0000 7f372d058640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37280cb580 0x7f37280adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37280cb580 0x7f37280adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37936/0 (socket says 172.21.15.53:37936) 2023-10-21T16:21:34.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 -- 172.21.15.53:0/3799834089 learned_addr learned my addr 172.21.15.53:0/3799834089 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:34.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372c857640 1 --2- 172.21.15.53:0/3799834089 >> v2:172.21.15.124:3300/0 conn(0x7f372815ab40 0x7f37280ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 -- 172.21.15.53:0/3799834089 >> v2:172.21.15.124:3300/0 conn(0x7f372815ab40 msgr2=0x7f37280ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:34.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 --2- 172.21.15.53:0/3799834089 >> v2:172.21.15.124:3300/0 conn(0x7f372815ab40 0x7f37280ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 -- 172.21.15.53:0/3799834089 >> v1:172.21.15.124:6789/0 conn(0x7f37280c90b0 legacy=0x7f37280cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:34.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.302+0000 7f372d058640 1 -- 172.21.15.53:0/3799834089 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372815b590 con 0x7f37280cb580 2023-10-21T16:21:34.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.303+0000 7f372d058640 1 --2- 172.21.15.53:0/3799834089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280adc60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f372806e440 tx=0x7f372002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b617f360e7aef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:34.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.303+0000 7f37277fe640 1 -- 172.21.15.53:0/3799834089 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372002ca80 con 0x7f37280cb580 2023-10-21T16:21:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f37277fe640 1 -- 172.21.15.53:0/3799834089 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372002cbe0 con 0x7f37280cb580 2023-10-21T16:21:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f37277fe640 1 -- 172.21.15.53:0/3799834089 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37200343f0 con 0x7f37280cb580 2023-10-21T16:21:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 -- 172.21.15.53:0/3799834089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 msgr2=0x7f37280adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 --2- 172.21.15.53:0/3799834089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280adc60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f372806e440 tx=0x7f372002bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 -- 172.21.15.53:0/3799834089 shutdown_connections 2023-10-21T16:21:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 --2- 172.21.15.53:0/3799834089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280adc60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 --2- 172.21.15.53:0/3799834089 >> v2:172.21.15.124:3300/0 conn(0x7f372815ab40 0x7f37280ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 -- 172.21.15.53:0/3799834089 >> 172.21.15.53:0/3799834089 conn(0x7f372806a380 msgr2=0x7f37280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 -- 172.21.15.53:0/3799834089 shutdown_connections 2023-10-21T16:21:34.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.304+0000 7f372eabb640 1 -- 172.21.15.53:0/3799834089 wait complete. 2023-10-21T16:21:34.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.306+0000 7f372eabb640 1 Processor -- start 2023-10-21T16:21:34.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.306+0000 7f372eabb640 1 -- start start 2023-10-21T16:21:34.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37280c90b0 0x7f372814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 0x7f37280c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b2f30 con 0x7f37280cb580 2023-10-21T16:21:34.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728155500 con 0x7f372815ab40 2023-10-21T16:21:34.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.307+0000 7f372eabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b23b0 con 0x7f37280c90b0 2023-10-21T16:21:34.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37944/0 (socket says 172.21.15.53:37944) 2023-10-21T16:21:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 -- 172.21.15.53:0/84292379 learned_addr learned my addr 172.21.15.53:0/84292379 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f372d058640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 0x7f37280c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f372c857640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37280c90b0 0x7f372814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 -- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37280c90b0 msgr2=0x7f372814a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37280c90b0 0x7f372814a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 -- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 msgr2=0x7f37280c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 0x7f37280c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f3727fff640 1 -- 172.21.15.53:0/84292379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720009b30 con 0x7f37280cb580 2023-10-21T16:21:34.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.308+0000 7f372d058640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 0x7f37280c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f3727fff640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f371400b3c0 tx=0x7f371400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714011830 con 0x7f37280cb580 2023-10-21T16:21:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714011990 con 0x7f37280cb580 2023-10-21T16:21:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37280b2690 con 0x7f37280cb580 2023-10-21T16:21:34.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714011d80 con 0x7f37280cb580 2023-10-21T16:21:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.309+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37280b2c50 con 0x7f37280cb580 2023-10-21T16:21:34.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.310+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37281556c0 con 0x7f37280cb580 2023-10-21T16:21:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.311+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f371401b460 con 0x7f37280cb580 2023-10-21T16:21:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.312+0000 7f37257fa640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 0x7f370c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:34.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.312+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3714089550 con 0x7f37280cb580 2023-10-21T16:21:34.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.313+0000 7f372c857640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 0x7f370c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:34.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.314+0000 7f372c857640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 0x7f370c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f371c006fd0 tx=0x7f371c008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:34.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.317+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37140542c0 con 0x7f37280cb580 2023-10-21T16:21:34.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.550+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3728067490 con 0x7f37280cb580 2023-10-21T16:21:34.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.551+0000 7f37257fa640 1 -- 172.21.15.53:0/84292379 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f37140578e0 con 0x7f37280cb580 2023-10-21T16:21:34.553 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:34.553 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:34.553 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:34.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 msgr2=0x7f370c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 0x7f370c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f371c006fd0 tx=0x7f371c008040 comp rx=0 tx=0).stop 2023-10-21T16:21:34.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 msgr2=0x7f37280c7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f371400b3c0 tx=0x7f371400b880 comp rx=0 tx=0).stop 2023-10-21T16:21:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 shutdown_connections 2023-10-21T16:21:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f370c06af90 0x7f370c06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f372815ab40 0x7f37280c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37280cb580 0x7f37280c7170 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 --2- 172.21.15.53:0/84292379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37280c90b0 0x7f372814a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 >> 172.21.15.53:0/84292379 conn(0x7f372806a380 msgr2=0x7f37280cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 shutdown_connections 2023-10-21T16:21:34.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:34.554+0000 7f372eabb640 1 -- 172.21.15.53:0/84292379 wait complete. 2023-10-21T16:21:34.976 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:21:27.892568+0000', 'modified': '2023-10-21T16:21:30.971234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36473': {'gid': 36473, 'name': 'b', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/582349146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 582349146}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 582349146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:21:34.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:21:34.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-21T16:21:35.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.213+0000 7f67b7e09640 1 Processor -- start 2023-10-21T16:21:35.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.214+0000 7f67b7e09640 1 -- start start 2023-10-21T16:21:35.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f67b006dfc0 0x7f67b0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67b0071070 0x7f67b00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00b2620 con 0x7f67b0071070 2023-10-21T16:21:35.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00b2060 con 0x7f67b006dfc0 2023-10-21T16:21:35.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.215+0000 7f67b7e09640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00b2340 con 0x7f67b00afbe0 2023-10-21T16:21:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67b0071070 0x7f67b00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67b0071070 0x7f67b00cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37960/0 (socket says 172.21.15.53:37960) 2023-10-21T16:21:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 -- 172.21.15.53:0/956564634 learned_addr learned my addr 172.21.15.53:0/956564634 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b53a4640 1 --2- 172.21.15.53:0/956564634 >> v2:172.21.15.124:3300/0 conn(0x7f67b006dfc0 0x7f67b0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 -- 172.21.15.53:0/956564634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 msgr2=0x7f67b00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 -- 172.21.15.53:0/956564634 >> v2:172.21.15.124:3300/0 conn(0x7f67b006dfc0 msgr2=0x7f67b0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 --2- 172.21.15.53:0/956564634 >> v2:172.21.15.124:3300/0 conn(0x7f67b006dfc0 0x7f67b0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.216+0000 7f67b63a6640 1 -- 172.21.15.53:0/956564634 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b0070350 con 0x7f67b0071070 2023-10-21T16:21:35.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.218+0000 7f67b63a6640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b0071070 0x7f67b00cb130 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f67b00cb0b0 tx=0x7f67a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d022ed9989b1a0db server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.218+0000 7f67b4ba3640 1 -- 172.21.15.53:0/956564634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a802ca80 con 0x7f67b0071070 2023-10-21T16:21:35.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.218+0000 7f67b4ba3640 1 -- 172.21.15.53:0/956564634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a802cbe0 con 0x7f67b0071070 2023-10-21T16:21:35.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.218+0000 7f67b4ba3640 1 -- 172.21.15.53:0/956564634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80343f0 con 0x7f67b0071070 2023-10-21T16:21:35.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 -- 172.21.15.53:0/956564634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b0071070 msgr2=0x7f67b00cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b0071070 0x7f67b00cb130 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f67b00cb0b0 tx=0x7f67a802bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 -- 172.21.15.53:0/956564634 shutdown_connections 2023-10-21T16:21:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b0071070 0x7f67b00cb130 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 --2- 172.21.15.53:0/956564634 >> v2:172.21.15.124:3300/0 conn(0x7f67b006dfc0 0x7f67b0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 --2- 172.21.15.53:0/956564634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 -- 172.21.15.53:0/956564634 >> 172.21.15.53:0/956564634 conn(0x7f67b006a5d0 msgr2=0x7f67b00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 -- 172.21.15.53:0/956564634 shutdown_connections 2023-10-21T16:21:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.219+0000 7f67b7e09640 1 -- 172.21.15.53:0/956564634 wait complete. 2023-10-21T16:21:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.221+0000 7f67b7e09640 1 Processor -- start 2023-10-21T16:21:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.221+0000 7f67b7e09640 1 -- start start 2023-10-21T16:21:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67b0071070 0x7f67b0155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00cf1f0 con 0x7f67b006dfc0 2023-10-21T16:21:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00ce570 con 0x7f67b0071070 2023-10-21T16:21:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b7e09640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00ce850 con 0x7f67b00afbe0 2023-10-21T16:21:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b5ba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b5ba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37970/0 (socket says 172.21.15.53:37970) 2023-10-21T16:21:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b5ba5640 1 -- 172.21.15.53:0/1433354336 learned_addr learned my addr 172.21.15.53:0/1433354336 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67b0071070 0x7f67b0155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b63a6640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b5ba5640 1 -- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 msgr2=0x7f67b00c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.222+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b5ba5640 1 -- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67b0071070 msgr2=0x7f67b0155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67b0071070 0x7f67b0155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b5ba5640 1 -- 172.21.15.53:0/1433354336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009b30 con 0x7f67b006dfc0 2023-10-21T16:21:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9b0 tx=0x7f67ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67a67fc640 1 -- 172.21.15.53:0/1433354336 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac009670 con 0x7f67b006dfc0 2023-10-21T16:21:35.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67a67fc640 1 -- 172.21.15.53:0/1433354336 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00c9a0 con 0x7f67b006dfc0 2023-10-21T16:21:35.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67a67fc640 1 -- 172.21.15.53:0/1433354336 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd50 con 0x7f67b006dfc0 2023-10-21T16:21:35.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00ceb30 con 0x7f67b006dfc0 2023-10-21T16:21:35.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.223+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00cdb20 con 0x7f67b006dfc0 2023-10-21T16:21:35.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.226+0000 7f67a67fc640 1 -- 172.21.15.53:0/1433354336 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f67ac013470 con 0x7f67b006dfc0 2023-10-21T16:21:35.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.228+0000 7f67a67fc640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 0x7f679406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.228+0000 7f67a67fc640 1 -- 172.21.15.53:0/1433354336 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67ac089b80 con 0x7f67b006dfc0 2023-10-21T16:21:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.228+0000 7f67ba9a0740 1 Processor -- start 2023-10-21T16:21:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.228+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 0x7f679406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.228+0000 7f67ba9a0740 1 -- start start 2023-10-21T16:21:35.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3724be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3724c3c0 0x562d3727c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3716c7a0 con 0x562d37279360 2023-10-21T16:21:35.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d371be740 con 0x562d37278f80 2023-10-21T16:21:35.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.229+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d371bbe30 con 0x562d3724c3c0 2023-10-21T16:21:35.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37976/0 (socket says 172.21.15.53:37976) 2023-10-21T16:21:35.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 -- 172.21.15.53:0/4062508689 learned_addr learned my addr 172.21.15.53:0/4062508689 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 0x7f679406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b63a6640 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3724c3c0 0x562d3727c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3724be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 -- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3724c3c0 msgr2=0x562d3727c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3724c3c0 0x562d3727c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 -- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 msgr2=0x562d3724be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3724be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 -- 172.21.15.53:0/4062508689 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d3717b910 con 0x562d37279360 2023-10-21T16:21:35.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.230+0000 7f67b53a4640 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f67a000f9f0 tx=0x7f67a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20add3b0e5932756 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.231+0000 7f67937fe640 1 -- 172.21.15.53:0/4062508689 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0014be0 con 0x562d37279360 2023-10-21T16:21:35.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.231+0000 7f67937fe640 1 -- 172.21.15.53:0/4062508689 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0019070 con 0x562d37279360 2023-10-21T16:21:35.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.231+0000 7f67937fe640 1 -- 172.21.15.53:0/4062508689 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0020440 con 0x562d37279360 2023-10-21T16:21:35.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 -- 172.21.15.53:0/4062508689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 msgr2=0x562d3727bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f67a000f9f0 tx=0x7f67a0012000 comp rx=0 tx=0).stop 2023-10-21T16:21:35.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 -- 172.21.15.53:0/4062508689 shutdown_connections 2023-10-21T16:21:35.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3724c3c0 0x562d3727c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d3727bd10 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/4062508689 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3724be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 -- 172.21.15.53:0/4062508689 >> 172.21.15.53:0/4062508689 conn(0x562d3724e190 msgr2=0x562d37296d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 -- 172.21.15.53:0/4062508689 shutdown_connections 2023-10-21T16:21:35.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.232+0000 7f67ba9a0740 1 -- 172.21.15.53:0/4062508689 wait complete. 2023-10-21T16:21:35.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.233+0000 7f67ba9a0740 1 -- start start 2023-10-21T16:21:35.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.233+0000 7f67ba9a0740 1 Processor -- start 2023-10-21T16:21:35.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3727ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 0x562d37290480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d371b7720 con 0x562d3727dde0 2023-10-21T16:21:35.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d37155970 con 0x562d37278f80 2023-10-21T16:21:35.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d37155610 con 0x562d37279360 2023-10-21T16:21:35.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67ba9a0740 10 client.0 _ll_register_callbacks cb 0x562d371779d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:21:35.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67b63a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67b63a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37984/0 (socket says 172.21.15.53:37984) 2023-10-21T16:21:35.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67b63a6640 1 -- 172.21.15.53:0/726569115 learned_addr learned my addr 172.21.15.53:0/726569115 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.234+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3727ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b53a4640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 0x562d37290480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 -- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 msgr2=0x562d37290480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 0x562d37290480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 -- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 msgr2=0x562d3727ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3727ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 -- 172.21.15.53:0/726569115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a000e040 con 0x562d3727dde0 2023-10-21T16:21:35.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3727ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:35.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b53a4640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 0x562d37290480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:35.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67b63a6640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009f10 tx=0x7f67a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeec662b88a19249 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f6790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:21:35.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8034b60 con 0x562d3727dde0 2023-10-21T16:21:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67a8034cc0 con 0x562d3727dde0 2023-10-21T16:21:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.235+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d371de160 con 0x562d3727dde0 2023-10-21T16:21:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a8034e80 con 0x562d3727dde0 2023-10-21T16:21:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 10 client.8694 fetch_fsmap learned FSMap version 69 2023-10-21T16:21:35.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x562d371b5da0 con 0x562d3727dde0 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d371de160 con 0x562d3727dde0 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67a8048410 con 0x562d3727dde0 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 10 client.8694 fetch_fsmap finished waiting for FSMap version 69 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 10 client.8694 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 20 client.8694 populate_metadata read hostname 'smithi053' 2023-10-21T16:21:35.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37269d60 0x562d37268020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 4 client.8694 mds_command: new command op to 36473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:21:35.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.236+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 --> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d37189cc0 con 0x562d37269d60 2023-10-21T16:21:35.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.237+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37269d60 0x562d37268020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.238+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37269d60 0x562d37268020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e2facec7f83610 server_cookie=239d90136c460652 in_seq=0 out_seq=0 2023-10-21T16:21:35.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.238+0000 7f6790ff9640 10 client.8694 ms_handle_connect on v2:172.21.15.124:6836/582349146 2023-10-21T16:21:35.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f6790ff9640 1 -- 172.21.15.53:0/726569115 <== mds.0 v2:172.21.15.124:6836/582349146 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562d37189cc0 con 0x562d37269d60 2023-10-21T16:21:35.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f6790ff9640 10 client.8694 handle_command_reply: tid=0 2023-10-21T16:21:35.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 client.8694 shutdown 2023-10-21T16:21:35.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 msgr2=0x562d372909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009f10 tx=0x7f67a8004ab0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 shutdown_connections 2023-10-21T16:21:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d3727dde0 0x562d372909c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37269d60 0x562d37268020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37279360 0x562d37290480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/726569115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37278f80 0x562d3727ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f6790ff9640 0 client.8694 ms_handle_reset on v2:172.21.15.124:6836/582349146 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.240+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 >> 172.21.15.53:0/726569115 conn(0x562d3724c3c0 msgr2=0x562d3729a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.241+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 shutdown_connections 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.241+0000 7f67ba9a0740 1 -- 172.21.15.53:0/726569115 wait complete. 2023-10-21T16:21:35.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.241+0000 7f67ba9a0740 20 client.8694 trim_cache size 0 max 16384 2023-10-21T16:21:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.258+0000 7f67ba9a0740 1 Processor -- start 2023-10-21T16:21:35.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.258+0000 7f67ba9a0740 1 -- start start 2023-10-21T16:21:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 0x562d3729a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d3727dde0 0x562d372b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d36f1bf20 con 0x562d37279360 2023-10-21T16:21:35.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3727c9b0 con 0x562d3727dde0 2023-10-21T16:21:35.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d37176e70 con 0x562d37278f80 2023-10-21T16:21:35.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67b53a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67b53a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37990/0 (socket says 172.21.15.53:37990) 2023-10-21T16:21:35.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.259+0000 7f67b53a4640 1 -- 172.21.15.53:0/1461323817 learned_addr learned my addr 172.21.15.53:0/1461323817 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 0x562d3729a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b63a6640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d3727dde0 0x562d372b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 -- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 msgr2=0x562d3729a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 0x562d3729a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 -- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d3727dde0 msgr2=0x562d372b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d3727dde0 0x562d372b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 -- 172.21.15.53:0/1461323817 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d37154b80 con 0x562d37279360 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 0x562d3729a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.260+0000 7f67b53a4640 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f67a0008010 tx=0x7f67a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75483cdfde09f94c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f6793fff640 1 -- 172.21.15.53:0/1461323817 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a002e070 con 0x562d37279360 2023-10-21T16:21:35.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f6793fff640 1 -- 172.21.15.53:0/1461323817 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a00149e0 con 0x562d37279360 2023-10-21T16:21:35.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f6793fff640 1 -- 172.21.15.53:0/1461323817 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0014d90 con 0x562d37279360 2023-10-21T16:21:35.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 -- 172.21.15.53:0/1461323817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 msgr2=0x562d37290670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f67a0008010 tx=0x7f67a0016000 comp rx=0 tx=0).stop 2023-10-21T16:21:35.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 -- 172.21.15.53:0/1461323817 shutdown_connections 2023-10-21T16:21:35.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d3727dde0 0x562d372b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37279360 0x562d37290670 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/1461323817 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d37278f80 0x562d3729a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 -- 172.21.15.53:0/1461323817 >> 172.21.15.53:0/1461323817 conn(0x562d3724c3c0 msgr2=0x562d37278330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 -- 172.21.15.53:0/1461323817 shutdown_connections 2023-10-21T16:21:35.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.261+0000 7f67ba9a0740 1 -- 172.21.15.53:0/1461323817 wait complete. 2023-10-21T16:21:35.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.262+0000 7f67ba9a0740 1 -- start start 2023-10-21T16:21:35.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.262+0000 7f67ba9a0740 1 Processor -- start 2023-10-21T16:21:35.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37279360 0x562d372ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3727dde0 0x562d3729a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d37277860 con 0x562d37278f80 2023-10-21T16:21:35.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d37268d20 con 0x562d37279360 2023-10-21T16:21:35.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d37268110 con 0x562d3727dde0 2023-10-21T16:21:35.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.263+0000 7f67ba9a0740 10 client.0 _ll_register_callbacks cb 0x7f67ac009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:21:35.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37996/0 (socket says 172.21.15.53:37996) 2023-10-21T16:21:35.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 -- 172.21.15.53:0/2237525158 learned_addr learned my addr 172.21.15.53:0/2237525158 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b53a4640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37279360 0x562d372ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b63a6640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3727dde0 0x562d3729a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 -- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3727dde0 msgr2=0x562d3729a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3727dde0 0x562d3729a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 -- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37279360 msgr2=0x562d372ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37279360 0x562d372ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 -- 172.21.15.53:0/2237525158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d3717b910 con 0x562d37278f80 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f67b5ba5640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0585d0 tx=0x7f67ac00c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea67840eba85678 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.264+0000 7f6790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac088070 con 0x562d37278f80 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67ac0604b0 con 0x562d37278f80 2023-10-21T16:21:35.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67a0019070 con 0x562d37278f80 2023-10-21T16:21:35.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67ac060670 con 0x562d37278f80 2023-10-21T16:21:35.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 10 client.8700 fetch_fsmap learned FSMap version 69 2023-10-21T16:21:35.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x562d3717a170 con 0x562d37278f80 2023-10-21T16:21:35.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0019070 con 0x562d37278f80 2023-10-21T16:21:35.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67ac074070 con 0x562d37278f80 2023-10-21T16:21:35.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 10 client.8700 fetch_fsmap finished waiting for FSMap version 69 2023-10-21T16:21:35.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 10 client.8700 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T16:21:35.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 20 client.8700 populate_metadata read hostname 'smithi053' 2023-10-21T16:21:35.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37276c40 0x562d37277020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 4 client.8700 mds_command: new command op to 36473 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:21:35.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.265+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 --> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d371b0700 con 0x562d37276c40 2023-10-21T16:21:35.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.266+0000 7f67b53a4640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37276c40 0x562d37277020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.267+0000 7f6790ff9640 10 client.8700 ms_handle_connect on v2:172.21.15.124:6836/582349146 2023-10-21T16:21:35.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.267+0000 7f67b53a4640 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37276c40 0x562d37277020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81abc34be09866e0 server_cookie=e691d85847e15014 in_seq=0 out_seq=0 2023-10-21T16:21:35.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.267+0000 7f6790ff9640 1 -- 172.21.15.53:0/2237525158 <== mds.0 v2:172.21.15.124:6836/582349146 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x562d371b0700 con 0x562d37276c40 2023-10-21T16:21:35.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.267+0000 7f6790ff9640 10 client.8700 handle_command_reply: tid=0 2023-10-21T16:21:35.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.267+0000 7f67ba9a0740 1 client.8700 shutdown 2023-10-21T16:21:35.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 msgr2=0x562d372f8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0585d0 tx=0x7f67ac00c650 comp rx=0 tx=0).stop 2023-10-21T16:21:35.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 shutdown_connections 2023-10-21T16:21:35.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:6836/582349146,v1:172.21.15.124:6837/582349146] conn(0x562d37276c40 0x562d37277020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562d3727dde0 0x562d3729a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562d37279360 0x562d372ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 --2- 172.21.15.53:0/2237525158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d37278f80 0x562d372f8df0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 >> 172.21.15.53:0/2237525158 conn(0x562d3724c3c0 msgr2=0x562d37278330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f6790ff9640 0 client.8700 ms_handle_reset on v2:172.21.15.124:6836/582349146 2023-10-21T16:21:35.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 shutdown_connections 2023-10-21T16:21:35.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 1 -- 172.21.15.53:0/2237525158 wait complete. 2023-10-21T16:21:35.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.268+0000 7f67ba9a0740 20 client.8700 trim_cache size 0 max 16384 2023-10-21T16:21:35.282 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:21:35.282 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:21:35.282 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:21:35.282 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 36473, 2023-10-21T16:21:35.283 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:21:35.283 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:21:35.283 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:21:35.283 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8132494799999996, 2023-10-21T16:21:35.283 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 69, 2023-10-21T16:21:35.284 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 148, 2023-10-21T16:21:35.284 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 148, 2023-10-21T16:21:35.284 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.638096432999999 2023-10-21T16:21:35.284 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:21:35.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 msgr2=0x7f679406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 0x7f679406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0008040 comp rx=0 tx=0).stop 2023-10-21T16:21:35.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 msgr2=0x7f67b00cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9b0 tx=0x7f67ac00ee70 comp rx=0 tx=0).stop 2023-10-21T16:21:35.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 shutdown_connections 2023-10-21T16:21:35.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f679406af90 0x7f679406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67b00afbe0 0x7f67b00c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67b0071070 0x7f67b0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 --2- 172.21.15.53:0/1433354336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67b006dfc0 0x7f67b00cd5e0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 >> 172.21.15.53:0/1433354336 conn(0x7f67b006a5d0 msgr2=0x7f67b0159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 shutdown_connections 2023-10-21T16:21:35.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.269+0000 7f67b7e09640 1 -- 172.21.15.53:0/1433354336 wait complete. 2023-10-21T16:21:35.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:21:35.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.517+0000 7f051c35e640 1 Processor -- start 2023-10-21T16:21:35.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.517+0000 7f051c35e640 1 -- start start 2023-10-21T16:21:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f051c35e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05140ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f051c35e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415b2b0 con 0x7f05140c90b0 2023-10-21T16:21:35.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f051c35e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415b590 con 0x7f05140cb580 2023-10-21T16:21:35.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f051c35e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051415bf30 con 0x7f051415ab40 2023-10-21T16:21:35.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f05198f9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f05140c90b0 0x7f05140cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48908/0 (socket says 172.21.15.53:48908) 2023-10-21T16:21:35.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.519+0000 7f05198f9640 1 -- 172.21.15.53:0/4009518596 learned_addr learned my addr 172.21.15.53:0/4009518596 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f051a0fa640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05140ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427237100 0 0) 0x7f051415b2b0 con 0x7f05140c90b0 2023-10-21T16:21:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0500003520 con 0x7f05140c90b0 2023-10-21T16:21:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044405446 0 0) 0x7f0500003520 con 0x7f05140c90b0 2023-10-21T16:21:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 msgr2=0x7f05140ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05140ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 >> v1:172.21.15.124:6789/0 conn(0x7f05140cb580 legacy=0x7f051415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051415b870 con 0x7f05140c90b0 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0504001670 con 0x7f05140c90b0 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 >> v1:172.21.15.53:6789/0 conn(0x7f05140c90b0 legacy=0x7f05140cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000039a0 0x7f0500003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0500004d00 0x7f0500005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051415bf30 con 0x7f0500004d00 2023-10-21T16:21:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050000ba60 con 0x7f05000039a0 2023-10-21T16:21:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.520+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050000bc20 con 0x7f051415ab40 2023-10-21T16:21:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0500004d00 0x7f0500005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f05198f9640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a0fa640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000039a0 0x7f0500003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 -- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 msgr2=0x7f05000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f051415ab40 0x7f05000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 -- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000039a0 msgr2=0x7f0500003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000039a0 0x7f0500003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.521+0000 7f051a8fb640 1 -- 172.21.15.53:0/4009518596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0500003520 con 0x7f0500004d00 2023-10-21T16:21:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.522+0000 7f051a8fb640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0500004d00 0x7f0500005180 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f05140caa60 tx=0x7f05000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148db68ff562cc5c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.522+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c003b70 con 0x7f0500004d00 2023-10-21T16:21:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.522+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c003cd0 con 0x7f0500004d00 2023-10-21T16:21:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.522+0000 7f05190f8640 1 -- 172.21.15.53:0/4009518596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c005310 con 0x7f0500004d00 2023-10-21T16:21:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 -- 172.21.15.53:0/4009518596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0500004d00 msgr2=0x7f0500005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 --2- 172.21.15.53:0/4009518596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0500004d00 0x7f0500005180 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f05140caa60 tx=0x7f05000057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051a8fb640 1 -- 172.21.15.53:0/4009518596 reap_dead start 2023-10-21T16:21:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 -- 172.21.15.53:0/4009518596 shutdown_connections 2023-10-21T16:21:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 -- 172.21.15.53:0/4009518596 >> 172.21.15.53:0/4009518596 conn(0x7f051406a380 msgr2=0x7f05140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 -- 172.21.15.53:0/4009518596 shutdown_connections 2023-10-21T16:21:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.523+0000 7f051c35e640 1 -- 172.21.15.53:0/4009518596 wait complete. 2023-10-21T16:21:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.525+0000 7f051c35e640 1 Processor -- start 2023-10-21T16:21:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.525+0000 7f051c35e640 1 -- start start 2023-10-21T16:21:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0500004d00 0x7f0514144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05140b3400 0x7f0514147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b4520 con 0x7f05140b1480 2023-10-21T16:21:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b1e50 con 0x7f0500004d00 2023-10-21T16:21:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.526+0000 7f051c35e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05141481c0 con 0x7f05140b3400 2023-10-21T16:21:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38014/0 (socket says 172.21.15.53:38014) 2023-10-21T16:21:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 -- 172.21.15.53:0/2814795515 learned_addr learned my addr 172.21.15.53:0/2814795515 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f05198f9640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05140b3400 0x7f0514147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 -- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05140b3400 msgr2=0x7f0514147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05140b3400 0x7f0514147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 -- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0500004d00 msgr2=0x7f0514144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0500004d00 0x7f0514144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 -- 172.21.15.53:0/2814795515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140b38a0 con 0x7f05140b1480 2023-10-21T16:21:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.527+0000 7f051a8fb640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f050c001f10 tx=0x7f050c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.528+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c03e070 con 0x7f05140b1480 2023-10-21T16:21:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.528+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c003e40 con 0x7f05140b1480 2023-10-21T16:21:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.528+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05140b3b80 con 0x7f05140b1480 2023-10-21T16:21:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.528+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c006d50 con 0x7f05140b1480 2023-10-21T16:21:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.528+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140b4140 con 0x7f05140b1480 2023-10-21T16:21:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.529+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0514067490 con 0x7f05140b1480 2023-10-21T16:21:35.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.530+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f050c007030 con 0x7f05140b1480 2023-10-21T16:21:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.532+0000 7f050affd640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 0x7f04f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.532+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f050c0aace0 con 0x7f05140b1480 2023-10-21T16:21:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.532+0000 7f051a0fa640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 0x7f04f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.533+0000 7f051a0fa640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 0x7f04f806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0510009910 tx=0x7f0510008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:35.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.536+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f050c0759d0 con 0x7f05140b1480 2023-10-21T16:21:35.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.772+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f051406dd30 con 0x7f05140b1480 2023-10-21T16:21:35.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.773+0000 7f050affd640 1 -- 172.21.15.53:0/2814795515 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f050c048070 con 0x7f05140b1480 2023-10-21T16:21:35.775 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:21:35.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.775+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 msgr2=0x7f04f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.775+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 0x7f04f806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0510009910 tx=0x7f0510008040 comp rx=0 tx=0).stop 2023-10-21T16:21:35.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 msgr2=0x7f05140b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:35.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f050c001f10 tx=0x7f050c0014a0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 shutdown_connections 2023-10-21T16:21:35.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f04f806af90 0x7f04f806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05140b1480 0x7f05140b1910 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05140b3400 0x7f0514147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 --2- 172.21.15.53:0/2814795515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0500004d00 0x7f0514144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:35.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 >> 172.21.15.53:0/2814795515 conn(0x7f051406a380 msgr2=0x7f05140ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:35.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 shutdown_connections 2023-10-21T16:21:35.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:35.776+0000 7f051c35e640 1 -- 172.21.15.53:0/2814795515 wait complete. 2023-10-21T16:21:35.796 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:21:35.796 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:21:35.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:21:35.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:21:35.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:35.799 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:21:35.799 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:21:35.799 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:21:35.800 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:21:35.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:35.801 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:21:35.855 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:21:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:21:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:21:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:21:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.858 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:21:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:21:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:21:35.859 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:21:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:21:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:21:35.859 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:21:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:21:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:21:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:21:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41197sec preferred_lft 41197sec 2023-10-21T16:21:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:21:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.861 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:21:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:21:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:21:35.861 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:21:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:21:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:21:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:21:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.862 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:21:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:21:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:21:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.863 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:21:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:21:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:21:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:21:35.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:35.865 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:21:35.966 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:21:35.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:35.971 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:21:36.030 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:21:36.031 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:21:36.031 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:21:36.031 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:21:36.031 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:21:36.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.032 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:21:36.032 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:21:36.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:21:36.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:21:36.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:21:36.034 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:21:36.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.224 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:21:36.282 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:21:36.282 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:21:36.283 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:21:36.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:21:36.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:21:36.392 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:36.453 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:21:36.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:21:36.453 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:36.512 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:21:36.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:21:36.653 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:21:36.654 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:21:36.692 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:21:36.692 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:21:36.692 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:21:36.693 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:21:36.693 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:21:36.694 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:21:36.694 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:21:36.694 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:21:36.695 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:21:36.695 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:21:36.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:21:36.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:36.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:36.783 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:21:36.867 INFO:teuthology.orchestra.run.smithi053.stdout:client36536 2023-10-21T16:21:36.868 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:36.868 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:36.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:21:37.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.161+0000 7fa1bcf93640 1 Processor -- start 2023-10-21T16:21:37.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.161+0000 7fa1bcf93640 1 -- start start 2023-10-21T16:21:37.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.162+0000 7fa1bcf93640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1b8164d10 0x7fa1b80ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.162+0000 7fa1bcf93640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80c9250 con 0x7fa1b80c18d0 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1bcf93640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b8161590 con 0x7fa1b80aff70 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1bcf93640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80cc570 con 0x7fa1b8164d10 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa1b80c18d0 0x7fa1b80c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48920/0 (socket says 172.21.15.53:48920) 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b6ffd640 1 -- 172.21.15.53:0/4168598931 learned_addr learned my addr 172.21.15.53:0/4168598931 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b7fff640 1 --2- 172.21.15.53:0/4168598931 >> v2:172.21.15.53:3300/0 conn(0x7fa1b8164d10 0x7fa1b80ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b67fc640 1 -- 172.21.15.53:0/4168598931 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359284363 0 0) 0x7fa1b80c9250 con 0x7fa1b80c18d0 2023-10-21T16:21:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b67fc640 1 -- 172.21.15.53:0/4168598931 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a4003520 con 0x7fa1b80c18d0 2023-10-21T16:21:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.163+0000 7fa1b7fff640 1 -- 172.21.15.53:0/4168598931 >> v1:172.21.15.124:6789/0 conn(0x7fa1b80aff70 legacy=0x7fa1b80b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.164+0000 7fa1b7fff640 1 -- 172.21.15.53:0/4168598931 >> v1:172.21.15.53:6789/0 conn(0x7fa1b80c18d0 legacy=0x7fa1b80c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.164+0000 7fa1b7fff640 1 -- 172.21.15.53:0/4168598931 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80c94b0 con 0x7fa1b8164d10 2023-10-21T16:21:37.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.165+0000 7fa1b7fff640 1 --2- 172.21.15.53:0/4168598931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b8164d10 0x7fa1b80ccd30 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00097a0 tx=0x7fa1b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb23fae161d1515 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.165+0000 7fa1b67fc640 1 -- 172.21.15.53:0/4168598931 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b002e070 con 0x7fa1b8164d10 2023-10-21T16:21:37.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.165+0000 7fa1b67fc640 1 -- 172.21.15.53:0/4168598931 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b00315a0 con 0x7fa1b8164d10 2023-10-21T16:21:37.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.165+0000 7fa1b67fc640 1 -- 172.21.15.53:0/4168598931 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0031930 con 0x7fa1b8164d10 2023-10-21T16:21:37.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 -- 172.21.15.53:0/4168598931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b8164d10 msgr2=0x7fa1b80ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/4168598931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b8164d10 0x7fa1b80ccd30 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00097a0 tx=0x7fa1b002b840 comp rx=0 tx=0).stop 2023-10-21T16:21:37.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 -- 172.21.15.53:0/4168598931 shutdown_connections 2023-10-21T16:21:37.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/4168598931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b8164d10 0x7fa1b80ccd30 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 -- 172.21.15.53:0/4168598931 >> 172.21.15.53:0/4168598931 conn(0x7fa1b806c050 msgr2=0x7fa1b806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:37.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 -- 172.21.15.53:0/4168598931 shutdown_connections 2023-10-21T16:21:37.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.166+0000 7fa1bcf93640 1 -- 172.21.15.53:0/4168598931 wait complete. 2023-10-21T16:21:37.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.168+0000 7fa1bcf93640 1 Processor -- start 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.168+0000 7fa1bcf93640 1 -- start start 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 0x7fa1b80b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 0x7fa1b8156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c7b10 con 0x7fa1b80c18d0 2023-10-21T16:21:37.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b3780 con 0x7fa1b8164d10 2023-10-21T16:21:37.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1bcf93640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81f7880 con 0x7fa1b80aff70 2023-10-21T16:21:37.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38034/0 (socket says 172.21.15.53:38034) 2023-10-21T16:21:37.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1b6ffd640 1 -- 172.21.15.53:0/2203417553 learned_addr learned my addr 172.21.15.53:0/2203417553 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:37.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.169+0000 7fa1b77fe640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 0x7fa1b80b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 -- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 msgr2=0x7fa1b80b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b7fff640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 0x7fa1b8156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 0x7fa1b80b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 -- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 msgr2=0x7fa1b8156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 0x7fa1b8156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 -- 172.21.15.53:0/2203417553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0009400 con 0x7fa1b80c18d0 2023-10-21T16:21:37.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b77fe640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 0x7fa1b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:37.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b7fff640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 0x7fa1b8156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:37.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1b6ffd640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c500 tx=0x7fa1a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005ce0 con 0x7fa1b80c18d0 2023-10-21T16:21:37.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0005e40 con 0x7fa1b80c18d0 2023-10-21T16:21:37.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0007330 con 0x7fa1b80c18d0 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b8162ad0 con 0x7fa1b80c18d0 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.170+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b80b2480 con 0x7fa1b80c18d0 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.171+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b8074150 con 0x7fa1b80c18d0 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.173+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa1a0007490 con 0x7fa1b80c18d0 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.174+0000 7fa1affff640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 0x7fa19806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.174+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1a0088f60 con 0x7fa1b80c18d0 2023-10-21T16:21:37.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.175+0000 7fa1b77fe640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 0x7fa19806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.176+0000 7fa1b77fe640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 0x7fa19806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80b3a80 tx=0x7fa1a8009040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.179+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1a0053e20 con 0x7fa1b80c18d0 2023-10-21T16:21:37.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.400+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 --> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b80cc810 con 0x7fa19806afc0 2023-10-21T16:21:37.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.404+0000 7fa1affff640 1 -- 172.21.15.53:0/2203417553 <== mgr.36350 v2:172.21.15.124:6832/227410594 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa1a8003990 con 0x7fa19806afc0 2023-10-21T16:21:37.405 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:21:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:21:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:21:37.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:21:37.406 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:21:37.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 msgr2=0x7fa19806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 0x7fa19806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80b3a80 tx=0x7fa1a8009040 comp rx=0 tx=0).stop 2023-10-21T16:21:37.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 msgr2=0x7fa1b81f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c500 tx=0x7fa1a000c9c0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 shutdown_connections 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7fa19806afc0 0x7fa19806d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1b8164d10 0x7fa1b8156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b80c18d0 0x7fa1b81f7340 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 --2- 172.21.15.53:0/2203417553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1b80aff70 0x7fa1b80b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.406+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 >> 172.21.15.53:0/2203417553 conn(0x7fa1b806c050 msgr2=0x7fa1b8160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:37.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.407+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 shutdown_connections 2023-10-21T16:21:37.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.407+0000 7fa1bcf93640 1 -- 172.21.15.53:0/2203417553 wait complete. 2023-10-21T16:21:37.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:21:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.654+0000 7f1eaea1a640 1 Processor -- start 2023-10-21T16:21:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.654+0000 7f1eaea1a640 1 -- start start 2023-10-21T16:21:37.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.656+0000 7f1eaea1a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.656+0000 7f1eaea1a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1ea80d0b90 0x7f1ea80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.656+0000 7f1eaea1a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea80d0360 con 0x7f1ea80cff10 2023-10-21T16:21:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.656+0000 7f1eaea1a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea80d2870 con 0x7f1ea815ccb0 2023-10-21T16:21:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.656+0000 7f1eaea1a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea80d1150 con 0x7f1ea80d0b90 2023-10-21T16:21:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38040/0 (socket says 172.21.15.53:38040) 2023-10-21T16:21:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 -- 172.21.15.53:0/716370852 learned_addr learned my addr 172.21.15.53:0/716370852 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1ead217640 1 --2- 172.21.15.53:0/716370852 >> v2:172.21.15.124:3300/0 conn(0x7f1ea80d0b90 0x7f1ea80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 -- 172.21.15.53:0/716370852 >> v2:172.21.15.124:3300/0 conn(0x7f1ea80d0b90 msgr2=0x7f1ea80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 --2- 172.21.15.53:0/716370852 >> v2:172.21.15.124:3300/0 conn(0x7f1ea80d0b90 0x7f1ea80cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 -- 172.21.15.53:0/716370852 >> v1:172.21.15.124:6789/0 conn(0x7f1ea80cff10 legacy=0x7f1ea80d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.657+0000 7f1eada18640 1 -- 172.21.15.53:0/716370852 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80d1430 con 0x7f1ea815ccb0 2023-10-21T16:21:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.659+0000 7f1eada18640 1 --2- 172.21.15.53:0/716370852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b85d0 tx=0x7f1ea402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd35ccb6796ae39 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.659+0000 7f1eaca16640 1 -- 172.21.15.53:0/716370852 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4031870 con 0x7f1ea815ccb0 2023-10-21T16:21:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.659+0000 7f1eaca16640 1 -- 172.21.15.53:0/716370852 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea40319d0 con 0x7f1ea815ccb0 2023-10-21T16:21:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.659+0000 7f1eaca16640 1 -- 172.21.15.53:0/716370852 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4031d60 con 0x7f1ea815ccb0 2023-10-21T16:21:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 -- 172.21.15.53:0/716370852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea815ccb0 msgr2=0x7f1ea80d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/716370852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b85d0 tx=0x7f1ea402b910 comp rx=0 tx=0).stop 2023-10-21T16:21:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 -- 172.21.15.53:0/716370852 shutdown_connections 2023-10-21T16:21:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/716370852 >> v2:172.21.15.124:3300/0 conn(0x7f1ea80d0b90 0x7f1ea80cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/716370852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea815ccb0 0x7f1ea80d1fb0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 -- 172.21.15.53:0/716370852 >> 172.21.15.53:0/716370852 conn(0x7f1ea8071660 msgr2=0x7f1ea8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:37.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 -- 172.21.15.53:0/716370852 shutdown_connections 2023-10-21T16:21:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.660+0000 7f1eaea1a640 1 -- 172.21.15.53:0/716370852 wait complete. 2023-10-21T16:21:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.662+0000 7f1eaea1a640 1 Processor -- start 2023-10-21T16:21:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.662+0000 7f1eaea1a640 1 -- start start 2023-10-21T16:21:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ea80d0b90 0x7f1ea815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ea815ccb0 0x7f1ea80d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea815fdb0 con 0x7f1ea80cff10 2023-10-21T16:21:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea815f1a0 con 0x7f1ea80d0b90 2023-10-21T16:21:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eaea1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea815f480 con 0x7f1ea815ccb0 2023-10-21T16:21:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eada18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eada18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38056/0 (socket says 172.21.15.53:38056) 2023-10-21T16:21:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.663+0000 7f1eada18640 1 -- 172.21.15.53:0/1539976565 learned_addr learned my addr 172.21.15.53:0/1539976565 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1ead217640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ea80d0b90 0x7f1ea815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eae219640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ea815ccb0 0x7f1ea80d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 -- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ea815ccb0 msgr2=0x7f1ea80d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ea815ccb0 0x7f1ea80d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 -- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ea80d0b90 msgr2=0x7f1ea815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ea80d0b90 0x7f1ea815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 -- 172.21.15.53:0/1539976565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40094d0 con 0x7f1ea80cff10 2023-10-21T16:21:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eada18640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1ea402b950 tx=0x7f1ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea403ad30 con 0x7f1ea80cff10 2023-10-21T16:21:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea40443f0 con 0x7f1ea80cff10 2023-10-21T16:21:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.664+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea815f760 con 0x7f1ea80cff10 2023-10-21T16:21:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.665+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40447a0 con 0x7f1ea80cff10 2023-10-21T16:21:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.665+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea814b490 con 0x7f1ea80cff10 2023-10-21T16:21:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.666+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea8074150 con 0x7f1ea80cff10 2023-10-21T16:21:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.667+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 65) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1ea404d450 con 0x7f1ea80cff10 2023-10-21T16:21:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.668+0000 7f1e9e7fc640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 0x7f1e8c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.668+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ea402d040 con 0x7f1ea80cff10 2023-10-21T16:21:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.669+0000 7f1ead217640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 0x7f1e8c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.670+0000 7f1ead217640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 0x7f1e8c06d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1ea81f54e0 tx=0x7f1e98008040 comp rx=0 tx=0).ready entity=mgr.36350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.673+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ea4076c60 con 0x7f1ea80cff10 2023-10-21T16:21:37.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:37.914+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1ea81e7d70 con 0x7f1ea80cff10 2023-10-21T16:21:38.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.522+0000 7f1e9e7fc640 1 -- 172.21.15.53:0/1539976565 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v66) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1ea4035020 con 0x7f1ea80cff10 2023-10-21T16:21:38.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.527+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 msgr2=0x7f1e8c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.527+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 0x7f1e8c06d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1ea81f54e0 tx=0x7f1e98008040 comp rx=0 tx=0).stop 2023-10-21T16:21:38.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.527+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 msgr2=0x7f1ea814af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.527+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1ea402b950 tx=0x7f1ea40046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 shutdown_connections 2023-10-21T16:21:38.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1e8c06afc0 0x7f1e8c06d480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ea815ccb0 0x7f1ea80d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ea80d0b90 0x7f1ea815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 --2- 172.21.15.53:0/1539976565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ea80cff10 0x7f1ea814af50 secure :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1ea402b950 tx=0x7f1ea40046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.528+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 >> 172.21.15.53:0/1539976565 conn(0x7f1ea8071660 msgr2=0x7f1ea815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:38.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.529+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 shutdown_connections 2023-10-21T16:21:38.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.529+0000 7f1eaea1a640 1 -- 172.21.15.53:0/1539976565 wait complete. 2023-10-21T16:21:38.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:21:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.758+0000 7f38aac56640 1 Processor -- start 2023-10-21T16:21:38.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.758+0000 7f38aac56640 1 -- start start 2023-10-21T16:21:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38aac56640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38a415cd80 0x7f38a40b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38aac56640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 0x7f38a40d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38aac56640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415d160 con 0x7f38a40b6730 2023-10-21T16:21:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38aac56640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40b7b90 con 0x7f38a415cd80 2023-10-21T16:21:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38aac56640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40b8020 con 0x7f38a40cdf70 2023-10-21T16:21:38.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38a89f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38a415cd80 0x7f38a40b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:38.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38a89f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38a415cd80 0x7f38a40b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38090/0 (socket says 172.21.15.53:38090) 2023-10-21T16:21:38.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38a3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f38a40b6730 0x7f38a40cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48938/0 (socket says 172.21.15.53:48938) 2023-10-21T16:21:38.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.760+0000 7f38a89f2640 1 -- 172.21.15.53:0/3606075847 learned_addr learned my addr 172.21.15.53:0/3606075847 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:38.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a91f3640 1 --2- 172.21.15.53:0/3606075847 >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 0x7f38a40d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:38.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a89f2640 1 -- 172.21.15.53:0/3606075847 >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 msgr2=0x7f38a40d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:38.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a89f2640 1 --2- 172.21.15.53:0/3606075847 >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 0x7f38a40d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a89f2640 1 -- 172.21.15.53:0/3606075847 >> v1:172.21.15.53:6789/0 conn(0x7f38a40b6730 legacy=0x7f38a40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a89f2640 1 -- 172.21.15.53:0/3606075847 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40b6f80 con 0x7f38a415cd80 2023-10-21T16:21:38.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.761+0000 7f38a91f3640 1 --2- 172.21.15.53:0/3606075847 >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 0x7f38a40d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:38.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.762+0000 7f38a89f2640 1 --2- 172.21.15.53:0/3606075847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a415cd80 0x7f38a40b87d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38a406c1d0 tx=0x7f389802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b9a1626d3f709b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:38.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.762+0000 7f38a37fe640 1 -- 172.21.15.53:0/3606075847 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898030870 con 0x7f38a415cd80 2023-10-21T16:21:38.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.762+0000 7f38a37fe640 1 -- 172.21.15.53:0/3606075847 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38980309d0 con 0x7f38a415cd80 2023-10-21T16:21:38.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.762+0000 7f38a37fe640 1 -- 172.21.15.53:0/3606075847 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898030d60 con 0x7f38a415cd80 2023-10-21T16:21:38.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 -- 172.21.15.53:0/3606075847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a415cd80 msgr2=0x7f38a40b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 --2- 172.21.15.53:0/3606075847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a415cd80 0x7f38a40b87d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38a406c1d0 tx=0x7f389802b9b0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 -- 172.21.15.53:0/3606075847 shutdown_connections 2023-10-21T16:21:38.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 --2- 172.21.15.53:0/3606075847 >> v2:172.21.15.124:3300/0 conn(0x7f38a40cdf70 0x7f38a40d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 --2- 172.21.15.53:0/3606075847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a415cd80 0x7f38a40b87d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 -- 172.21.15.53:0/3606075847 >> 172.21.15.53:0/3606075847 conn(0x7f38a406b5d0 msgr2=0x7f38a406b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:38.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 -- 172.21.15.53:0/3606075847 shutdown_connections 2023-10-21T16:21:38.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.763+0000 7f38aac56640 1 -- 172.21.15.53:0/3606075847 wait complete. 2023-10-21T16:21:38.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.765+0000 7f38aac56640 1 Processor -- start 2023-10-21T16:21:38.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.765+0000 7f38aac56640 1 -- start start 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38a40cdf70 0x7f38a41462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 0x7f38a415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4156730 con 0x7f38a40b6730 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4160210 con 0x7f38a40cdf70 2023-10-21T16:21:38.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38aac56640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415f9d0 con 0x7f38a415cd80 2023-10-21T16:21:38.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38a89f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:38.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38a89f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38104/0 (socket says 172.21.15.53:38104) 2023-10-21T16:21:38.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.766+0000 7f38a89f2640 1 -- 172.21.15.53:0/4172399870 learned_addr learned my addr 172.21.15.53:0/4172399870 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:38.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a91f3640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 0x7f38a415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:38.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a3fff640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38a40cdf70 0x7f38a41462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 msgr2=0x7f38a415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 0x7f38a415f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38a40cdf70 msgr2=0x7f38a41462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38a40cdf70 0x7f38a41462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 -- 172.21.15.53:0/4172399870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980094d0 con 0x7f38a40b6730 2023-10-21T16:21:38.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a91f3640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 0x7f38a415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:38.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a89f2640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:38.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.767+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898030860 con 0x7f38a40b6730 2023-10-21T16:21:38.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.768+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898039d90 con 0x7f38a40b6730 2023-10-21T16:21:38.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.768+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898043650 con 0x7f38a40b6730 2023-10-21T16:21:38.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.768+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a415fc50 con 0x7f38a40b6730 2023-10-21T16:21:38.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.768+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a4153cd0 con 0x7f38a40b6730 2023-10-21T16:21:38.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.769+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4153b30 con 0x7f38a40b6730 2023-10-21T16:21:38.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.770+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 66) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f389802f3c0 con 0x7f38a40b6730 2023-10-21T16:21:38.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.770+0000 7f38a17fa640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 0x7f388806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:38.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.771+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38980ac2b0 con 0x7f38a40b6730 2023-10-21T16:21:38.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.771+0000 7f38a3fff640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 msgr2=0x7f388806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:38.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.771+0000 7f38a3fff640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 0x7f388806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:38.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.775+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38a4153b30 con 0x7f38a40b6730 2023-10-21T16:21:38.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.972+0000 7f38a3fff640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 msgr2=0x7f388806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:38.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.972+0000 7f38a3fff640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 0x7f388806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:38.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.993+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f38a4153b30 con 0x7f38a40b6730 2023-10-21T16:21:39.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:38.998+0000 7f38a17fa640 1 -- 172.21.15.53:0/4172399870 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f38a4153b30 con 0x7f38a40b6730 2023-10-21T16:21:39.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 msgr2=0x7f388806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:39.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 0x7f388806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 msgr2=0x7f38a40c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3898000c00 tx=0x7f3898004730 comp rx=0 tx=0).stop 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 shutdown_connections 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f388806afe0 0x7f388806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415cd80 0x7f38a415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38a40cdf70 0x7f38a41462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 --2- 172.21.15.53:0/4172399870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a40b6730 0x7f38a40c5950 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 >> 172.21.15.53:0/4172399870 conn(0x7f38a406b5d0 msgr2=0x7f38a415ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.001+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 shutdown_connections 2023-10-21T16:21:39.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.002+0000 7f38aac56640 1 -- 172.21.15.53:0/4172399870 wait complete. 2023-10-21T16:21:39.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:21:39.040 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.039+0000 7fbcc1a2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:39.081 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.080+0000 7fcca1632040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:39.093 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.092+0000 7fbcc1a2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:39.137 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.136+0000 7fcca1632040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:39.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.234+0000 7f1ef77c9640 1 Processor -- start 2023-10-21T16:21:39.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.235+0000 7f1ef77c9640 1 -- start start 2023-10-21T16:21:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef77c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef77c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef015b310 con 0x7f1ef015aa00 2023-10-21T16:21:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef77c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006d290 con 0x7f1ef006db60 2023-10-21T16:21:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef77c9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00ca230 con 0x7f1ef006c3a0 2023-10-21T16:21:39.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef5565640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef5565640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38110/0 (socket says 172.21.15.53:38110) 2023-10-21T16:21:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.236+0000 7f1ef5565640 1 -- 172.21.15.53:0/755350071 learned_addr learned my addr 172.21.15.53:0/755350071 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:39.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.237+0000 7f1ef5565640 1 -- 172.21.15.53:0/755350071 >> v1:172.21.15.124:6789/0 conn(0x7f1ef006db60 legacy=0x7f1ef006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.237+0000 7f1ef5565640 1 -- 172.21.15.53:0/755350071 >> v1:172.21.15.53:6789/0 conn(0x7f1ef015aa00 legacy=0x7f1ef00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.237+0000 7f1ef5565640 1 -- 172.21.15.53:0/755350071 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00c9860 con 0x7f1ef006c3a0 2023-10-21T16:21:39.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.238+0000 7f1ef5565640 1 --2- 172.21.15.53:0/755350071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009870 tx=0x7f1eec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681343c9f143a964 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.239+0000 7f1ee7fff640 1 -- 172.21.15.53:0/755350071 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec030860 con 0x7f1ef006c3a0 2023-10-21T16:21:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.239+0000 7f1ee7fff640 1 -- 172.21.15.53:0/755350071 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0309c0 con 0x7f1ef006c3a0 2023-10-21T16:21:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.239+0000 7f1ee7fff640 1 -- 172.21.15.53:0/755350071 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec030d50 con 0x7f1ef006c3a0 2023-10-21T16:21:39.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.239+0000 7f1ef77c9640 1 -- 172.21.15.53:0/755350071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef006c3a0 msgr2=0x7f1ef00c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.239+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/755350071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1eec009870 tx=0x7f1eec02b9a0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.240+0000 7f1ef77c9640 1 -- 172.21.15.53:0/755350071 shutdown_connections 2023-10-21T16:21:39.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.240+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/755350071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef006c3a0 0x7f1ef00c9cf0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.240+0000 7f1ef77c9640 1 -- 172.21.15.53:0/755350071 >> 172.21.15.53:0/755350071 conn(0x7f1ef006a5d0 msgr2=0x7f1ef00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.240+0000 7f1ef77c9640 1 -- 172.21.15.53:0/755350071 shutdown_connections 2023-10-21T16:21:39.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.240+0000 7f1ef77c9640 1 -- 172.21.15.53:0/755350071 wait complete. 2023-10-21T16:21:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.242+0000 7f1ef77c9640 1 Processor -- start 2023-10-21T16:21:39.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.242+0000 7f1ef77c9640 1 -- start start 2023-10-21T16:21:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef006c3a0 0x7f1ef01f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef006db60 0x7f1ef00c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c28d0 con 0x7f1ef015aa00 2023-10-21T16:21:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01f6ae0 con 0x7f1ef006db60 2023-10-21T16:21:39.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.243+0000 7f1ef77c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef015b650 con 0x7f1ef006c3a0 2023-10-21T16:21:39.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.244+0000 7f1ef5d66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.244+0000 7f1ef5565640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef006c3a0 0x7f1ef01f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.244+0000 7f1ef5d66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38118/0 (socket says 172.21.15.53:38118) 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.244+0000 7f1ef5d66640 1 -- 172.21.15.53:0/2602627785 learned_addr learned my addr 172.21.15.53:0/2602627785 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.244+0000 7f1ef4d64640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef006db60 0x7f1ef00c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef006c3a0 msgr2=0x7f1ef01f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef006c3a0 0x7f1ef01f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef006db60 msgr2=0x7f1ef00c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef006db60 0x7f1ef00c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 -- 172.21.15.53:0/2602627785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0094d0 con 0x7f1ef015aa00 2023-10-21T16:21:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef5d66640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c6b0 tx=0x7f1ee800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:39.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8003b70 con 0x7f1ef015aa00 2023-10-21T16:21:39.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee8003cf0 con 0x7f1ef015aa00 2023-10-21T16:21:39.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8003ea0 con 0x7f1ef015aa00 2023-10-21T16:21:39.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00b3ba0 con 0x7f1ef015aa00 2023-10-21T16:21:39.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.245+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00c25f0 con 0x7f1ef015aa00 2023-10-21T16:21:39.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.246+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef015b800 con 0x7f1ef015aa00 2023-10-21T16:21:39.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.248+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 66) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1ee8005310 con 0x7f1ef015aa00 2023-10-21T16:21:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.249+0000 7f1ee5ffb640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 0x7f1ed406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.249+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ee8089450 con 0x7f1ef015aa00 2023-10-21T16:21:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.250+0000 7f1ef5565640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 msgr2=0x7f1ed406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:39.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.250+0000 7f1ef5565640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 0x7f1ed406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:39.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.255+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ef015b800 con 0x7f1ef015aa00 2023-10-21T16:21:39.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.446+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1ef015b310 con 0x7f1ef015aa00 2023-10-21T16:21:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.451+0000 7f1ef5565640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 msgr2=0x7f1ed406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.451+0000 7f1ef5565640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 0x7f1ed406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.452+0000 7f1ee5ffb640 1 -- 172.21.15.53:0/2602627785 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1ef015b310 con 0x7f1ef015aa00 2023-10-21T16:21:39.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 msgr2=0x7f1ed406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 0x7f1ed406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 msgr2=0x7f1ef01f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800c6b0 tx=0x7f1ee800cb70 comp rx=0 tx=0).stop 2023-10-21T16:21:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 shutdown_connections 2023-10-21T16:21:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f1ed406afe0 0x7f1ed406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ef015aa00 0x7f1ef01f1d30 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef006db60 0x7f1ef00c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 --2- 172.21.15.53:0/2602627785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef006c3a0 0x7f1ef01f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 >> 172.21.15.53:0/2602627785 conn(0x7f1ef006a5d0 msgr2=0x7f1ef00cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 shutdown_connections 2023-10-21T16:21:39.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.455+0000 7f1ef77c9640 1 -- 172.21.15.53:0/2602627785 wait complete. 2023-10-21T16:21:39.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:39.474 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2023-10-21T16:21:39.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2023-10-21T16:21:39.541 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.540+0000 7fbcc1a2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:39.592 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.591+0000 7fbcc1a2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:39.613 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.612+0000 7fcca1632040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:39.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.640+0000 7fbcc1a2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:39.669 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.667+0000 7fcca1632040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:39.702 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.701+0000 7fbcc1a2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:39.723 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.721+0000 7fcca1632040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:39.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.739+0000 7f8dd2860640 1 Processor -- start 2023-10-21T16:21:39.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.739+0000 7f8dd2860640 1 -- start start 2023-10-21T16:21:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd2860640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 0x7f8dcc0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd2860640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8dcc0d1500 0x7f8dcc1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd2860640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc0d0a80 con 0x7f8dcc0b5060 2023-10-21T16:21:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd2860640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc161620 con 0x7f8dcc0d1500 2023-10-21T16:21:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd2860640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc0b1fe0 con 0x7f8dcc0d1120 2023-10-21T16:21:39.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd105d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8dcc0d1500 0x7f8dcc1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd105d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8dcc0d1500 0x7f8dcc1610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38134/0 (socket says 172.21.15.53:38134) 2023-10-21T16:21:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd105d640 1 -- 172.21.15.53:0/1559458271 learned_addr learned my addr 172.21.15.53:0/1559458271 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.741+0000 7f8dd185e640 1 --2- 172.21.15.53:0/1559458271 >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 0x7f8dcc0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.742+0000 7f8dd105d640 1 -- 172.21.15.53:0/1559458271 >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 msgr2=0x7f8dcc0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:39.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.742+0000 7f8dd105d640 1 --2- 172.21.15.53:0/1559458271 >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 0x7f8dcc0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.742+0000 7f8dd105d640 1 -- 172.21.15.53:0/1559458271 >> v1:172.21.15.124:6789/0 conn(0x7f8dcc0b5060 legacy=0x7f8dcc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.742+0000 7f8dd105d640 1 -- 172.21.15.53:0/1559458271 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0b4580 con 0x7f8dcc0d1500 2023-10-21T16:21:39.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.742+0000 7f8dd185e640 1 --2- 172.21.15.53:0/1559458271 >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 0x7f8dcc0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:39.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.743+0000 7f8dd105d640 1 --2- 172.21.15.53:0/1559458271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc1610e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc08cc60 tx=0x7f8dbc02b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106d6f8c3fb7769e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:39.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.743+0000 7f8dd085c640 1 -- 172.21.15.53:0/1559458271 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc031760 con 0x7f8dcc0d1500 2023-10-21T16:21:39.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.743+0000 7f8dd085c640 1 -- 172.21.15.53:0/1559458271 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0318c0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.743+0000 7f8dd085c640 1 -- 172.21.15.53:0/1559458271 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc031c50 con 0x7f8dcc0d1500 2023-10-21T16:21:39.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 -- 172.21.15.53:0/1559458271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 msgr2=0x7f8dcc1610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 --2- 172.21.15.53:0/1559458271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc1610e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc08cc60 tx=0x7f8dbc02b800 comp rx=0 tx=0).stop 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 -- 172.21.15.53:0/1559458271 shutdown_connections 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 --2- 172.21.15.53:0/1559458271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc1610e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 --2- 172.21.15.53:0/1559458271 >> v2:172.21.15.124:3300/0 conn(0x7f8dcc0d1120 0x7f8dcc0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 -- 172.21.15.53:0/1559458271 >> 172.21.15.53:0/1559458271 conn(0x7f8dcc0d2960 msgr2=0x7f8dcc0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 -- 172.21.15.53:0/1559458271 shutdown_connections 2023-10-21T16:21:39.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.744+0000 7f8dd2860640 1 -- 172.21.15.53:0/1559458271 wait complete. 2023-10-21T16:21:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.746+0000 7f8dd2860640 1 Processor -- start 2023-10-21T16:21:39.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.746+0000 7f8dd2860640 1 -- start start 2023-10-21T16:21:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dcc0b5060 0x7f8dcc15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dcc0d1120 0x7f8dcc0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c63e0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c6630 con 0x7f8dcc0b5060 2023-10-21T16:21:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd2860640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c6910 con 0x7f8dcc0d1120 2023-10-21T16:21:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc0c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38146/0 (socket says 172.21.15.53:38146) 2023-10-21T16:21:39.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 -- 172.21.15.53:0/3860002646 learned_addr learned my addr 172.21.15.53:0/3860002646 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dcc0b5060 0x7f8dcc15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd105d640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dcc0d1120 0x7f8dcc0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dcc0d1120 msgr2=0x7f8dcc0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8dcc0d1120 0x7f8dcc0c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dcc0b5060 msgr2=0x7f8dcc15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dcc0b5060 0x7f8dcc15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd205f640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0094d0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.747+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8dcc0b5060 0x7f8dcc15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:39.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dd205f640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc0c9c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c780 tx=0x7f8dc400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003d80 con 0x7f8dcc0d1500 2023-10-21T16:21:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc400ce90 con 0x7f8dcc0d1500 2023-10-21T16:21:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40052b0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc0c6bf0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.748+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc0c71b0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.749+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc0c73a0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.750+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 66) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8dc4005410 con 0x7f8dcc0d1500 2023-10-21T16:21:39.758 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.751+0000 7fbcc1a2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.751+0000 7f8dc27fc640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:39.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.751+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8dc4088db0 con 0x7f8dcc0d1500 2023-10-21T16:21:39.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.751+0000 7f8dd185e640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 msgr2=0x7f8db006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:39.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.751+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:39.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.755+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8dc408d070 con 0x7f8dcc0d1500 2023-10-21T16:21:39.789 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.788+0000 7fcca1632040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:39.841 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.840+0000 7fcca1632040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:39.861 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.860+0000 7fbcc1a2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:39.923 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.922+0000 7fbcc1a2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:39.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.952+0000 7f8dd185e640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 msgr2=0x7f8db006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:39.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.952+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:39.958 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:39.957+0000 7fcca1632040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:39.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:39.968+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dcc0c6a80 con 0x7f8db006afe0 2023-10-21T16:21:39.971 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:39.969+0000 7fbcc1a2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:40.025 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.023+0000 7fcca1632040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:40.028 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.026+0000 7fbcc1a2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:40.077 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.075+0000 7fcca1632040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:40.079 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.077+0000 7fbcc1a2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:40.136 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.134+0000 7fcca1632040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:40.187 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.186+0000 7fcca1632040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:40.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:40.353+0000 7f8dd185e640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 msgr2=0x7f8db006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:40.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:40.353+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:21:40.454 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.452+0000 7fbcc1a2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:40.585 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.584+0000 7fcca1632040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:40.612 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.611+0000 7fbcc1a2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:40.721 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.719+0000 7fbcc1a2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:40.752 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.750+0000 7fcca1632040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:40.771 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.769+0000 7fbcc1a2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:40.865 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.864+0000 7fcca1632040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:40.878 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.876+0000 7fbcc1a2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:40.917 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:40.916+0000 7fcca1632040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:40.983 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:40.981+0000 7fbcc1a2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:41.029 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:41.028+0000 7fcca1632040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:41.139 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:41.137+0000 7fbcc1a2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:41.141 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:41.139+0000 7fcca1632040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:41.154+0000 7f8dd185e640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 msgr2=0x7f8db006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/227410594 2023-10-21T16:21:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:41.154+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:21:41.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:41.185+0000 7fbcc1a2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:41.305 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:41.302+0000 7fbcc1a2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:41.305 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:41.303+0000 7fcca1632040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:41.356 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:41.354+0000 7fcca1632040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:41.482 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:41.480+0000 7fcca1632040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:42.133 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.131+0000 7fbcc1a2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:42.355 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:42.353+0000 7fcca1632040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:42.423 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.421+0000 7fbcc1a2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:42.479 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.477+0000 7fbcc1a2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:42.591 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.590+0000 7fbcc1a2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:42.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:42.611+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 67) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f8dc40056c0 con 0x7f8dcc0d1500 2023-10-21T16:21:42.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:42.611+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 msgr2=0x7f8db006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:42.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:42.611+0000 7f8dc27fc640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/227410594,v1:172.21.15.124:6833/227410594] conn(0x7f8db006afe0 0x7f8db006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:42.675 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:42.674+0000 7fcca1632040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:42.737 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:42.735+0000 7fcca1632040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:42.744 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.742+0000 7fbc3dca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.744 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.742+0000 7fbc3dca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.745 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.742+0000 7fbc3dca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.745 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.742+0000 7fbc3dca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.745 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.742+0000 7fbc3dca9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.743+0000 7fbc40cef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.743+0000 7fbc40cef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.743+0000 7fbc40cef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.743+0000 7fbc40cef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:42.743+0000 7fbc40cef640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:42.857 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:42.856+0000 7fcca1632040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:43.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.624+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 68) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8dc4054030 con 0x7f8dcc0d1500 2023-10-21T16:21:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.625+0000 7f8dc27fc640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f8db00358e0 0x7f8db00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:43.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.625+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dcc0c6a80 con 0x7f8db00358e0 2023-10-21T16:21:43.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.635+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f8db00358e0 0x7f8db00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.637+0000 7f8dd185e640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f8db00358e0 0x7f8db00754a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8008010 tx=0x7f8dc8008040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:43.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.782+0000 7f8dc27fc640 1 -- 172.21.15.53:0/3860002646 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f8dcc0c6a80 con 0x7f8db00358e0 2023-10-21T16:21:43.783 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/minutes 2023-10-21T16:21:43.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f8db00358e0 msgr2=0x7f8db00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:43.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f8db00358e0 0x7f8db00754a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8008010 tx=0x7f8dc8008040 comp rx=0 tx=0).stop 2023-10-21T16:21:43.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 msgr2=0x7f8dcc0c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:43.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 --2- 172.21.15.53:0/3860002646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8dcc0d1500 0x7f8dcc0c9c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c780 tx=0x7f8dc400cc40 comp rx=0 tx=0).stop 2023-10-21T16:21:43.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd205f640 1 -- 172.21.15.53:0/3860002646 reap_dead start 2023-10-21T16:21:43.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 shutdown_connections 2023-10-21T16:21:43.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 >> 172.21.15.53:0/3860002646 conn(0x7f8dcc0d2960 msgr2=0x7f8dcc1602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:43.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 shutdown_connections 2023-10-21T16:21:43.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:43.785+0000 7f8dd2860640 1 -- 172.21.15.53:0/3860002646 wait complete. 2023-10-21T16:21:43.808 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2023-10-21T16:21:43.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:43.809 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2023-10-21T16:21:43.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2023-10-21T16:21:44.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.098+0000 7f1f9d4bb640 1 Processor -- start 2023-10-21T16:21:44.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.098+0000 7f1f9d4bb640 1 -- start start 2023-10-21T16:21:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9d4bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f9815a820 0x7f1f980aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9d4bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9815ac00 0x7f1f980b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9d4bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f980cab60 con 0x7f1f980ca680 2023-10-21T16:21:44.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9d4bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cb960 con 0x7f1f9815ac00 2023-10-21T16:21:44.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9d4bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cbdf0 con 0x7f1f9815a820 2023-10-21T16:21:44.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9ccba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9815ac00 0x7f1f980b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9ccba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f9815ac00 0x7f1f980b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60450/0 (socket says 172.21.15.53:60450) 2023-10-21T16:21:44.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.100+0000 7f1f9ccba640 1 -- 172.21.15.53:0/1381714256 learned_addr learned my addr 172.21.15.53:0/1381714256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:44.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.101+0000 7f1f97fff640 1 --2- 172.21.15.53:0/1381714256 >> v2:172.21.15.124:3300/0 conn(0x7f1f9815a820 0x7f1f980aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.101+0000 7f1f9ccba640 1 -- 172.21.15.53:0/1381714256 >> v2:172.21.15.124:3300/0 conn(0x7f1f9815a820 msgr2=0x7f1f980aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:44.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.101+0000 7f1f9ccba640 1 --2- 172.21.15.53:0/1381714256 >> v2:172.21.15.124:3300/0 conn(0x7f1f9815a820 0x7f1f980aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.101+0000 7f1f9ccba640 1 -- 172.21.15.53:0/1381714256 >> v1:172.21.15.124:6789/0 conn(0x7f1f980ca680 legacy=0x7f1f980c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.101+0000 7f1f9ccba640 1 -- 172.21.15.53:0/1381714256 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980cb030 con 0x7f1f9815ac00 2023-10-21T16:21:44.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.102+0000 7f1f9ccba640 1 --2- 172.21.15.53:0/1381714256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815ac00 0x7f1f980b04b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806da70 tx=0x7f1f9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da28eaa6ffeab98a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.102+0000 7f1f96ffd640 1 -- 172.21.15.53:0/1381714256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9002ca80 con 0x7f1f9815ac00 2023-10-21T16:21:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.102+0000 7f1f96ffd640 1 -- 172.21.15.53:0/1381714256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9002cbe0 con 0x7f1f9815ac00 2023-10-21T16:21:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.102+0000 7f1f96ffd640 1 -- 172.21.15.53:0/1381714256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900343f0 con 0x7f1f9815ac00 2023-10-21T16:21:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/1381714256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815ac00 msgr2=0x7f1f980b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/1381714256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815ac00 0x7f1f980b04b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806da70 tx=0x7f1f9002bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/1381714256 shutdown_connections 2023-10-21T16:21:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/1381714256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815ac00 0x7f1f980b04b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/1381714256 >> v2:172.21.15.124:3300/0 conn(0x7f1f9815a820 0x7f1f980aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/1381714256 >> 172.21.15.53:0/1381714256 conn(0x7f1f9806c050 msgr2=0x7f1f9806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/1381714256 shutdown_connections 2023-10-21T16:21:44.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.103+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/1381714256 wait complete. 2023-10-21T16:21:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.105+0000 7f1f9d4bb640 1 Processor -- start 2023-10-21T16:21:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.105+0000 7f1f9d4bb640 1 -- start start 2023-10-21T16:21:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980ca680 0x7f1f981f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9815ac00 0x7f1f980cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f981f35c0 con 0x7f1f9815a820 2023-10-21T16:21:44.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f981583c0 con 0x7f1f980ca680 2023-10-21T16:21:44.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f9d4bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980af0e0 con 0x7f1f9815ac00 2023-10-21T16:21:44.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60452/0 (socket says 172.21.15.53:60452) 2023-10-21T16:21:44.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.106+0000 7f1f977fe640 1 -- 172.21.15.53:0/16913835 learned_addr learned my addr 172.21.15.53:0/16913835 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:44.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f9ccba640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9815ac00 0x7f1f980cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f97fff640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980ca680 0x7f1f981f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 -- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9815ac00 msgr2=0x7f1f980cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9815ac00 0x7f1f980cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 -- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980ca680 msgr2=0x7f1f981f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980ca680 0x7f1f981f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 -- 172.21.15.53:0/16913835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009b30 con 0x7f1f9815a820 2023-10-21T16:21:44.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f977fe640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c400 tx=0x7f1f8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0052d0 con 0x7f1f9815a820 2023-10-21T16:21:44.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c005430 con 0x7f1f9815a820 2023-10-21T16:21:44.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f981f2ab0 con 0x7f1f9815a820 2023-10-21T16:21:44.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.107+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c003830 con 0x7f1f9815a820 2023-10-21T16:21:44.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.108+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f981f3cb0 con 0x7f1f9815a820 2023-10-21T16:21:44.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.109+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98074150 con 0x7f1f9815a820 2023-10-21T16:21:44.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.110+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 68) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f1f8c003990 con 0x7f1f9815a820 2023-10-21T16:21:44.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.110+0000 7f1f94ff9640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 0x7f1f8406d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.111+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f1f8c088b70 con 0x7f1f9815a820 2023-10-21T16:21:44.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.111+0000 7f1f97fff640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 0x7f1f8406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.112+0000 7f1f97fff640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 0x7f1f8406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006fd0 tx=0x7f1f88008040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.115+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f8c053bb0 con 0x7f1f9815a820 2023-10-21T16:21:44.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.329+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f980cb5f0 con 0x7f1f8406ad40 2023-10-21T16:21:44.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.486+0000 7f1f94ff9640 1 -- 172.21.15.53:0/16913835 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f1f8800b600 con 0x7f1f8406ad40 2023-10-21T16:21:44.488 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/hourly 2023-10-21T16:21:44.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.489+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 msgr2=0x7f1f8406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.489+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 0x7f1f8406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f88006fd0 tx=0x7f1f88008040 comp rx=0 tx=0).stop 2023-10-21T16:21:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.489+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 msgr2=0x7f1f981f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.489+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c400 tx=0x7f1f8c00c8c0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 shutdown_connections 2023-10-21T16:21:44.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f1f8406ad40 0x7f1f8406d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9815ac00 0x7f1f980cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f9815a820 0x7f1f981f3770 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 --2- 172.21.15.53:0/16913835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980ca680 0x7f1f981f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 >> 172.21.15.53:0/16913835 conn(0x7f1f9806c050 msgr2=0x7f1f980ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:44.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 shutdown_connections 2023-10-21T16:21:44.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.490+0000 7f1f9d4bb640 1 -- 172.21.15.53:0/16913835 wait complete. 2023-10-21T16:21:44.513 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2023-10-21T16:21:44.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:44.514 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2023-10-21T16:21:44.574 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2023-10-21T16:21:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.811+0000 7f4b8370f640 1 Processor -- start 2023-10-21T16:21:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.812+0000 7f4b8370f640 1 -- start start 2023-10-21T16:21:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.813+0000 7f4b8370f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b7c0cb130 0x7f4b7c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.813+0000 7f4b8370f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7c0b1cc0 con 0x7f4b7c06dfc0 2023-10-21T16:21:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.813+0000 7f4b8370f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7c0cbad0 con 0x7f4b7c0afbe0 2023-10-21T16:21:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.813+0000 7f4b8370f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7c0b2620 con 0x7f4b7c0cb130 2023-10-21T16:21:44.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.813+0000 7f4b81cac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b7c0cb130 0x7f4b7c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.814+0000 7f4b80caa640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4b7c06dfc0 0x7f4b7c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38968/0 (socket says 172.21.15.53:38968) 2023-10-21T16:21:44.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.814+0000 7f4b80caa640 1 -- 172.21.15.53:0/1635190058 learned_addr learned my addr 172.21.15.53:0/1635190058 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:44.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.814+0000 7f4b81cac640 1 -- 172.21.15.53:0/1635190058 >> v1:172.21.15.124:6789/0 conn(0x7f4b7c0afbe0 legacy=0x7f4b7c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:44.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.814+0000 7f4b81cac640 1 -- 172.21.15.53:0/1635190058 >> v1:172.21.15.53:6789/0 conn(0x7f4b7c06dfc0 legacy=0x7f4b7c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.814+0000 7f4b81cac640 1 -- 172.21.15.53:0/1635190058 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0b2060 con 0x7f4b7c0cb130 2023-10-21T16:21:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.815+0000 7f4b81cac640 1 --2- 172.21.15.53:0/1635190058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c0cb590 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097a0 tx=0x7f4b7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880e8013bfbd573d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b7bfff640 1 -- 172.21.15.53:0/1635190058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74031830 con 0x7f4b7c0cb130 2023-10-21T16:21:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b7bfff640 1 -- 172.21.15.53:0/1635190058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b74031990 con 0x7f4b7c0cb130 2023-10-21T16:21:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b7bfff640 1 -- 172.21.15.53:0/1635190058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b74031d20 con 0x7f4b7c0cb130 2023-10-21T16:21:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 -- 172.21.15.53:0/1635190058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 msgr2=0x7f4b7c0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 --2- 172.21.15.53:0/1635190058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c0cb590 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097a0 tx=0x7f4b7402b8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 -- 172.21.15.53:0/1635190058 shutdown_connections 2023-10-21T16:21:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 --2- 172.21.15.53:0/1635190058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c0cb590 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 -- 172.21.15.53:0/1635190058 >> 172.21.15.53:0/1635190058 conn(0x7f4b7c06a5d0 msgr2=0x7f4b7c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.816+0000 7f4b8370f640 1 -- 172.21.15.53:0/1635190058 shutdown_connections 2023-10-21T16:21:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.817+0000 7f4b8370f640 1 -- 172.21.15.53:0/1635190058 wait complete. 2023-10-21T16:21:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.819+0000 7f4b8370f640 1 Processor -- start 2023-10-21T16:21:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.819+0000 7f4b8370f640 1 -- start start 2023-10-21T16:21:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 0x7f4b7c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b7c0afbe0 0x7f4b7c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c159420 con 0x7f4b7c0cb130 2023-10-21T16:21:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c23d0 con 0x7f4b7c0afbe0 2023-10-21T16:21:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b8370f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c26b0 con 0x7f4b7c06dfc0 2023-10-21T16:21:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b81cac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b81cac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60466/0 (socket says 172.21.15.53:60466) 2023-10-21T16:21:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.820+0000 7f4b81cac640 1 -- 172.21.15.53:0/721485398 learned_addr learned my addr 172.21.15.53:0/721485398 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b814ab640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 0x7f4b7c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b80caa640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b7c0afbe0 0x7f4b7c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 -- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 msgr2=0x7f4b7c0ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 0x7f4b7c0ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 -- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b7c0afbe0 msgr2=0x7f4b7c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b7c0afbe0 0x7f4b7c1589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 -- 172.21.15.53:0/721485398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74009400 con 0x7f4b7c0cb130 2023-10-21T16:21:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b814ab640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 0x7f4b7c0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.821+0000 7f4b81cac640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097a0 tx=0x7f4b740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.822+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7403a9e0 con 0x7f4b7c0cb130 2023-10-21T16:21:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.822+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7403ab40 con 0x7f4b7c0cb130 2023-10-21T16:21:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.822+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b740443f0 con 0x7f4b7c0cb130 2023-10-21T16:21:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.822+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c0c2990 con 0x7f4b7c0cb130 2023-10-21T16:21:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.822+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c0c2f50 con 0x7f4b7c0cb130 2023-10-21T16:21:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.823+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c0c3140 con 0x7f4b7c0cb130 2023-10-21T16:21:44.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.824+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 69) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f4b7404d450 con 0x7f4b7c0cb130 2023-10-21T16:21:44.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.826+0000 7f4b79ffb640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 0x7f4b6806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:44.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.826+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f4b740abdd0 con 0x7f4b7c0cb130 2023-10-21T16:21:44.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.826+0000 7f4b814ab640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 0x7f4b6806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:44.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.827+0000 7f4b814ab640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 0x7f4b6806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c004520 tx=0x7f4b6c009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:44.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:44.831+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b74076970 con 0x7f4b7c0cb130 2023-10-21T16:21:45.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.052+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b7c070350 con 0x7f4b6806afe0 2023-10-21T16:21:45.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.201+0000 7f4b79ffb640 1 -- 172.21.15.53:0/721485398 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4b6c006b50 con 0x7f4b6806afe0 2023-10-21T16:21:45.203 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/daily 2023-10-21T16:21:45.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.204+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 msgr2=0x7f4b6806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.204+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 0x7f4b6806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c004520 tx=0x7f4b6c009040 comp rx=0 tx=0).stop 2023-10-21T16:21:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 msgr2=0x7f4b7c158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097a0 tx=0x7f4b740046c0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 shutdown_connections 2023-10-21T16:21:45.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f4b6806afe0 0x7f4b6806d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b7c0cb130 0x7f4b7c158ee0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b7c0afbe0 0x7f4b7c1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 --2- 172.21.15.53:0/721485398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b7c06dfc0 0x7f4b7c0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 >> 172.21.15.53:0/721485398 conn(0x7f4b7c06a5d0 msgr2=0x7f4b7c070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:45.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 shutdown_connections 2023-10-21T16:21:45.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.205+0000 7f4b8370f640 1 -- 172.21.15.53:0/721485398 wait complete. 2023-10-21T16:21:45.229 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2023-10-21T16:21:45.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:45.230 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2023-10-21T16:21:45.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2023-10-21T16:21:45.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.519+0000 7f20b26d7640 1 Processor -- start 2023-10-21T16:21:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.519+0000 7f20b26d7640 1 -- start start 2023-10-21T16:21:45.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b26d7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f20ac15cd80 0x7f20ac0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b26d7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac15d160 con 0x7f20ac0cdf70 2023-10-21T16:21:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b26d7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0b7b90 con 0x7f20ac0b6730 2023-10-21T16:21:45.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b26d7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0b8020 con 0x7f20ac15cd80 2023-10-21T16:21:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b0c74640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f20ac0cdf70 0x7f20ac0d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38978/0 (socket says 172.21.15.53:38978) 2023-10-21T16:21:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.520+0000 7f20b0c74640 1 -- 172.21.15.53:0/2738924076 learned_addr learned my addr 172.21.15.53:0/2738924076 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:45.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20abfff640 1 --2- 172.21.15.53:0/2738924076 >> v2:172.21.15.124:3300/0 conn(0x7f20ac15cd80 0x7f20ac0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248255623 0 0) 0x7f20ac15d160 con 0x7f20ac0cdf70 2023-10-21T16:21:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2098003520 con 0x7f20ac0cdf70 2023-10-21T16:21:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945492666 0 0) 0x7f2098003520 con 0x7f20ac0cdf70 2023-10-21T16:21:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 >> v2:172.21.15.124:3300/0 conn(0x7f20ac15cd80 msgr2=0x7f20ac0b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:45.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 --2- 172.21.15.53:0/2738924076 >> v2:172.21.15.124:3300/0 conn(0x7f20ac15cd80 0x7f20ac0b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 >> v1:172.21.15.124:6789/0 conn(0x7f20ac0b6730 legacy=0x7f20ac0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0b6f80 con 0x7f20ac0cdf70 2023-10-21T16:21:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.521+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f20a4001670 con 0x7f20ac0cdf70 2023-10-21T16:21:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 >> v1:172.21.15.53:6789/0 conn(0x7f20ac0cdf70 legacy=0x7f20ac0d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 0x7f2098003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980047c0 0x7f2098004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2098005180 0x7f209800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b8020 con 0x7f20980039c0 2023-10-21T16:21:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209800c080 con 0x7f2098005180 2023-10-21T16:21:45.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209800c240 con 0x7f20980047c0 2023-10-21T16:21:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 0x7f2098003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 -- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980047c0 msgr2=0x7f2098004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980047c0 0x7f2098004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 -- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2098005180 msgr2=0x7f209800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:45.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2098005180 0x7f209800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20abfff640 1 -- 172.21.15.53:0/2738924076 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098003520 con 0x7f20980039c0 2023-10-21T16:21:45.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.522+0000 7f20b0c74640 1 -- 172.21.15.53:0/2738924076 reap_dead start 2023-10-21T16:21:45.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.523+0000 7f20abfff640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 0x7f2098003de0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fc0 tx=0x7f20980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b81740fb1d01a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:45.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c02cc60 con 0x7f20980039c0 2023-10-21T16:21:45.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c00f040 con 0x7f20980039c0 2023-10-21T16:21:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20ab7fe640 1 -- 172.21.15.53:0/2738924076 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c034440 con 0x7f20980039c0 2023-10-21T16:21:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 -- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 msgr2=0x7f2098003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 0x7f2098003de0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f209c009fc0 tx=0x7f20980057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 -- 172.21.15.53:0/2738924076 shutdown_connections 2023-10-21T16:21:45.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2738924076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980039c0 0x7f2098003de0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 -- 172.21.15.53:0/2738924076 >> 172.21.15.53:0/2738924076 conn(0x7f20ac06b5d0 msgr2=0x7f20ac06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 -- 172.21.15.53:0/2738924076 shutdown_connections 2023-10-21T16:21:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.524+0000 7f20b26d7640 1 -- 172.21.15.53:0/2738924076 wait complete. 2023-10-21T16:21:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.526+0000 7f20b26d7640 1 Processor -- start 2023-10-21T16:21:45.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.526+0000 7f20b26d7640 1 -- start start 2023-10-21T16:21:45.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980039c0 0x7f20ac14e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20ac0d1f20 0x7f20ac0d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0d28e0 con 0x7f20ac0ca840 2023-10-21T16:21:45.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cbef0 con 0x7f20ac0d1f20 2023-10-21T16:21:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.527+0000 7f20b26d7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cb2e0 con 0x7f20980039c0 2023-10-21T16:21:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60490/0 (socket says 172.21.15.53:60490) 2023-10-21T16:21:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 -- 172.21.15.53:0/2867404278 learned_addr learned my addr 172.21.15.53:0/2867404278 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:45.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20abfff640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980039c0 0x7f20ac14e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 -- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980039c0 msgr2=0x7f20ac14e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980039c0 0x7f20ac14e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 -- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20ac0d1f20 msgr2=0x7f20ac0d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20ac0d1f20 0x7f20ac0d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 -- 172.21.15.53:0/2867404278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0cb5c0 con 0x7f20ac0ca840 2023-10-21T16:21:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.528+0000 7f20a37fe640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f209400c500 tx=0x7f209400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.529+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20940052d0 con 0x7f20ac0ca840 2023-10-21T16:21:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.529+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094005430 con 0x7f20ac0ca840 2023-10-21T16:21:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.529+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094012870 con 0x7f20ac0ca840 2023-10-21T16:21:45.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.529+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac0cb8a0 con 0x7f20ac0ca840 2023-10-21T16:21:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.529+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac0cc980 con 0x7f20ac0ca840 2023-10-21T16:21:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.530+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac04e160 con 0x7f20ac0ca840 2023-10-21T16:21:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.531+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 69) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f20940129d0 con 0x7f20ac0ca840 2023-10-21T16:21:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.532+0000 7f20a97fa640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 0x7f208c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:45.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.532+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f20940895e0 con 0x7f20ac0ca840 2023-10-21T16:21:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.533+0000 7f20abfff640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 0x7f208c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:45.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.534+0000 7f20abfff640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 0x7f208c06d4f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f209c036000 tx=0x7f209c02e000 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.537+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2094054410 con 0x7f20ac0ca840 2023-10-21T16:21:45.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.754+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) v1 -- 0x7f20ac0b7540 con 0x7f208c06b030 2023-10-21T16:21:45.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.909+0000 7f20a97fa640 1 -- 172.21.15.53:0/2867404278 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f209c02cdd0 con 0x7f208c06b030 2023-10-21T16:21:45.911 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/weekly 2023-10-21T16:21:45.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 msgr2=0x7f208c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 0x7f208c06d4f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f209c036000 tx=0x7f209c02e000 comp rx=0 tx=0).stop 2023-10-21T16:21:45.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 msgr2=0x7f20ac0cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:45.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f209400c500 tx=0x7f209400c9c0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 shutdown_connections 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f208c06b030 0x7f208c06d4f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20ac0d1f20 0x7f20ac0d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20ac0ca840 0x7f20ac0cacc0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 --2- 172.21.15.53:0/2867404278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20980039c0 0x7f20ac14e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:45.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 >> 172.21.15.53:0/2867404278 conn(0x7f20ac06b5d0 msgr2=0x7f20ac0d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:45.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 shutdown_connections 2023-10-21T16:21:45.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:45.912+0000 7f20b26d7640 1 -- 172.21.15.53:0/2867404278 wait complete. 2023-10-21T16:21:45.936 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2023-10-21T16:21:45.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:45.937 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2023-10-21T16:21:46.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2023-10-21T16:21:46.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.233+0000 7f238eca2640 1 Processor -- start 2023-10-21T16:21:46.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.233+0000 7f238eca2640 1 -- start start 2023-10-21T16:21:46.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238eca2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f238815cb80 0x7f23880d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238eca2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 0x7f23880b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238eca2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880d1630 con 0x7f23880d26d0 2023-10-21T16:21:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238eca2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880d1910 con 0x7f238815cb80 2023-10-21T16:21:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238eca2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880d1bf0 con 0x7f23880d32e0 2023-10-21T16:21:46.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238ca3e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f238815cb80 0x7f23880d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f237ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f23880d26d0 0x7f23880d2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38990/0 (socket says 172.21.15.53:38990) 2023-10-21T16:21:46.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f237ffff640 1 -- 172.21.15.53:0/1628535707 learned_addr learned my addr 172.21.15.53:0/1628535707 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:46.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.235+0000 7f238ca3e640 1 --2- 172.21.15.53:0/1628535707 >> v2:172.21.15.53:3300/0 conn(0x7f238815cb80 0x7f23880d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60492/0 (socket says 172.21.15.53:60492) 2023-10-21T16:21:46.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238d23f640 1 --2- 172.21.15.53:0/1628535707 >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 0x7f23880b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238ca3e640 1 -- 172.21.15.53:0/1628535707 >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 msgr2=0x7f23880b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:46.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238ca3e640 1 --2- 172.21.15.53:0/1628535707 >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 0x7f23880b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238ca3e640 1 -- 172.21.15.53:0/1628535707 >> v1:172.21.15.53:6789/0 conn(0x7f23880d26d0 legacy=0x7f23880d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f237f7fe640 1 -- 172.21.15.53:0/1628535707 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437139694 0 0) 0x7f23880d1630 con 0x7f23880d26d0 2023-10-21T16:21:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238ca3e640 1 -- 172.21.15.53:0/1628535707 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880d1ed0 con 0x7f238815cb80 2023-10-21T16:21:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.236+0000 7f238d23f640 1 --2- 172.21.15.53:0/1628535707 >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 0x7f23880b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.237+0000 7f238ca3e640 1 --2- 172.21.15.53:0/1628535707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f23880d0d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f23840097a0 tx=0x7f238402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ebd9a53a8f1221 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.237+0000 7f237f7fe640 1 -- 172.21.15.53:0/1628535707 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238402e070 con 0x7f238815cb80 2023-10-21T16:21:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.237+0000 7f237f7fe640 1 -- 172.21.15.53:0/1628535707 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840315a0 con 0x7f238815cb80 2023-10-21T16:21:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.237+0000 7f237f7fe640 1 -- 172.21.15.53:0/1628535707 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384031930 con 0x7f238815cb80 2023-10-21T16:21:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 -- 172.21.15.53:0/1628535707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 msgr2=0x7f23880d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 --2- 172.21.15.53:0/1628535707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f23880d0d30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f23840097a0 tx=0x7f238402b840 comp rx=0 tx=0).stop 2023-10-21T16:21:46.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 -- 172.21.15.53:0/1628535707 shutdown_connections 2023-10-21T16:21:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 --2- 172.21.15.53:0/1628535707 >> v2:172.21.15.124:3300/0 conn(0x7f23880d32e0 0x7f23880b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 --2- 172.21.15.53:0/1628535707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f23880d0d30 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 -- 172.21.15.53:0/1628535707 >> 172.21.15.53:0/1628535707 conn(0x7f23880718a0 msgr2=0x7f2388071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 -- 172.21.15.53:0/1628535707 shutdown_connections 2023-10-21T16:21:46.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.238+0000 7f238eca2640 1 -- 172.21.15.53:0/1628535707 wait complete. 2023-10-21T16:21:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.240+0000 7f238eca2640 1 Processor -- start 2023-10-21T16:21:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.240+0000 7f238eca2640 1 -- start start 2023-10-21T16:21:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 0x7f2388152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23880d32e0 0x7f238815f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388153290 con 0x7f238815cb80 2023-10-21T16:21:46.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388153570 con 0x7f23880d26d0 2023-10-21T16:21:46.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238eca2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2388153850 con 0x7f23880d32e0 2023-10-21T16:21:46.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238d23f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238d23f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60506/0 (socket says 172.21.15.53:60506) 2023-10-21T16:21:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.241+0000 7f238d23f640 1 -- 172.21.15.53:0/3803154980 learned_addr learned my addr 172.21.15.53:0/3803154980 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f237ffff640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23880d32e0 0x7f238815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238ca3e640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 0x7f2388152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 -- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23880d32e0 msgr2=0x7f238815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23880d32e0 0x7f238815f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 -- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 msgr2=0x7f2388152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 0x7f2388152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 -- 172.21.15.53:0/3803154980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009400 con 0x7f238815cb80 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238ca3e640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 0x7f2388152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:46.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.242+0000 7f238d23f640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f238000cef0 tx=0x7f238000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.243+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000d840 con 0x7f238815cb80 2023-10-21T16:21:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.243+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238000d9c0 con 0x7f238815cb80 2023-10-21T16:21:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.243+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000dd70 con 0x7f238815cb80 2023-10-21T16:21:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.243+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2388153b30 con 0x7f238815cb80 2023-10-21T16:21:46.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.243+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388152f60 con 0x7f238815cb80 2023-10-21T16:21:46.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.244+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2388154050 con 0x7f238815cb80 2023-10-21T16:21:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.245+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 69) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f238001e3b0 con 0x7f238815cb80 2023-10-21T16:21:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.246+0000 7f237d7fa640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 0x7f237406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.246+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f238008a650 con 0x7f238815cb80 2023-10-21T16:21:46.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.246+0000 7f238ca3e640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 0x7f237406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.248+0000 7f238ca3e640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 0x7f237406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2384036000 tx=0x7f238403d040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.250+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2388154050 con 0x7f238815cb80 2023-10-21T16:21:46.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.469+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f23880d21b0 con 0x7f237406afe0 2023-10-21T16:21:46.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.632+0000 7f237d7fa640 1 -- 172.21.15.53:0/3803154980 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7f2384031bc0 con 0x7f237406afe0 2023-10-21T16:21:46.634 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/monthly 2023-10-21T16:21:46.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 msgr2=0x7f237406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 0x7f237406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2384036000 tx=0x7f238403d040 comp rx=0 tx=0).stop 2023-10-21T16:21:46.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 msgr2=0x7f238815e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f238000cef0 tx=0x7f238000a710 comp rx=0 tx=0).stop 2023-10-21T16:21:46.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 shutdown_connections 2023-10-21T16:21:46.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.636+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f237406afe0 0x7f237406d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f238815cb80 0x7f238815e670 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23880d32e0 0x7f238815f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 --2- 172.21.15.53:0/3803154980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23880d26d0 0x7f2388152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 >> 172.21.15.53:0/3803154980 conn(0x7f23880718a0 msgr2=0x7f23880b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 shutdown_connections 2023-10-21T16:21:46.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.637+0000 7f238eca2640 1 -- 172.21.15.53:0/3803154980 wait complete. 2023-10-21T16:21:46.661 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2023-10-21T16:21:46.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:46.662 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2023-10-21T16:21:46.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1Y 2023-10-21T16:21:46.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.960+0000 7fb731440640 1 Processor -- start 2023-10-21T16:21:46.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.961+0000 7fb731440640 1 -- start start 2023-10-21T16:21:46.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.962+0000 7fb731440640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 0x7fb72c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.962+0000 7fb731440640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb72c0d0b90 0x7fb72c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.962+0000 7fb731440640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c0d2870 con 0x7fb72c15ccb0 2023-10-21T16:21:46.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.962+0000 7fb731440640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c0d1150 con 0x7fb72c0d0b90 2023-10-21T16:21:46.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.962+0000 7fb731440640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0d1430 con 0x7fb72c0d00e0 2023-10-21T16:21:46.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.963+0000 7fb723fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 0x7fb72c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.963+0000 7fb723fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 0x7fb72c0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35386/0 (socket says 172.21.15.53:35386) 2023-10-21T16:21:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.963+0000 7fb723fff640 1 -- 172.21.15.53:0/275134462 learned_addr learned my addr 172.21.15.53:0/275134462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.963+0000 7fb730c3f640 1 --2- 172.21.15.53:0/275134462 >> v2:172.21.15.124:3300/0 conn(0x7fb72c0d0b90 0x7fb72c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.964+0000 7fb730c3f640 1 -- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 msgr2=0x7fb72c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:46.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.964+0000 7fb730c3f640 1 --2- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 0x7fb72c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.964+0000 7fb730c3f640 1 -- 172.21.15.53:0/275134462 >> v1:172.21.15.124:6789/0 conn(0x7fb72c15ccb0 legacy=0x7fb72c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.964+0000 7fb730c3f640 1 -- 172.21.15.53:0/275134462 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c0d1710 con 0x7fb72c0d0b90 2023-10-21T16:21:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.965+0000 7fb730c3f640 1 --2- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d0b90 0x7fb72c0b6060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb724009d90 tx=0x7fb72402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df50c143e0411f36 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.966+0000 7fb72b7fe640 1 -- 172.21.15.53:0/275134462 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72402cc50 con 0x7fb72c0d0b90 2023-10-21T16:21:46.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.966+0000 7fb72b7fe640 1 -- 172.21.15.53:0/275134462 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72402cdb0 con 0x7fb72c0d0b90 2023-10-21T16:21:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.966+0000 7fb72b7fe640 1 -- 172.21.15.53:0/275134462 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724034660 con 0x7fb72c0d0b90 2023-10-21T16:21:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 -- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d0b90 msgr2=0x7fb72c0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 --2- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d0b90 0x7fb72c0b6060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb724009d90 tx=0x7fb72402bea0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 -- 172.21.15.53:0/275134462 shutdown_connections 2023-10-21T16:21:46.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 --2- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d00e0 0x7fb72c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 --2- 172.21.15.53:0/275134462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d0b90 0x7fb72c0b6060 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 -- 172.21.15.53:0/275134462 >> 172.21.15.53:0/275134462 conn(0x7fb72c071660 msgr2=0x7fb72c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 -- 172.21.15.53:0/275134462 shutdown_connections 2023-10-21T16:21:46.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.967+0000 7fb731440640 1 -- 172.21.15.53:0/275134462 wait complete. 2023-10-21T16:21:46.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.969+0000 7fb731440640 1 Processor -- start 2023-10-21T16:21:46.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.969+0000 7fb731440640 1 -- start start 2023-10-21T16:21:46.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 0x7fb72c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d0b90 0x7fb72c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c9780 con 0x7fb72c15ccb0 2023-10-21T16:21:46.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c7430 con 0x7fb72c0d00e0 2023-10-21T16:21:46.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb731440640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c76d0 con 0x7fb72c0d0b90 2023-10-21T16:21:46.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb730c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb730c3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60516/0 (socket says 172.21.15.53:60516) 2023-10-21T16:21:46.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.970+0000 7fb730c3f640 1 -- 172.21.15.53:0/2733392764 learned_addr learned my addr 172.21.15.53:0/2733392764 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:46.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb723fff640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d0b90 0x7fb72c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb72bfff640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 0x7fb72c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 -- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d0b90 msgr2=0x7fb72c0c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d0b90 0x7fb72c0c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 -- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 msgr2=0x7fb72c0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 0x7fb72c0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:46.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 -- 172.21.15.53:0/2733392764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724009a70 con 0x7fb72c15ccb0 2023-10-21T16:21:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb72bfff640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 0x7fb72c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.971+0000 7fb730c3f640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb724009d90 tx=0x7fb724004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.972+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724040650 con 0x7fb72c15ccb0 2023-10-21T16:21:46.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.972+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7240407b0 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.972+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c0c79b0 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.972+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724040b60 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.972+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c0c9470 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.973+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c0c7ed0 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.974+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 69) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fb724040cc0 con 0x7fb72c15ccb0 2023-10-21T16:21:46.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.975+0000 7fb7297fa640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 0x7fb70c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.975+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fb7240ac820 con 0x7fb72c15ccb0 2023-10-21T16:21:46.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.975+0000 7fb72bfff640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 0x7fb70c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.977+0000 7fb72bfff640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 0x7fb70c06d4a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb71c007fd0 tx=0x7fb71c009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:46.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:46.980+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb72c0c7ed0 con 0x7fb72c15ccb0 2023-10-21T16:21:47.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.199+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1Y", "target": ["mon-mgr", ""]}) v1 -- 0x7fb72c0b5b10 con 0x7fb70c06afe0 2023-10-21T16:21:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.352+0000 7fb7297fa640 1 -- 172.21.15.53:0/2733392764 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7fb71c001340 con 0x7fb70c06afe0 2023-10-21T16:21:47.354 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1/yearly 2023-10-21T16:21:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.355+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 msgr2=0x7fb70c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.355+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 0x7fb70c06d4a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb71c007fd0 tx=0x7fb71c009040 comp rx=0 tx=0).stop 2023-10-21T16:21:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.355+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 msgr2=0x7fb72c1f1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.355+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb724009d90 tx=0x7fb724004a40 comp rx=0 tx=0).stop 2023-10-21T16:21:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 shutdown_connections 2023-10-21T16:21:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb70c06afe0 0x7fb70c06d4a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb72c15ccb0 0x7fb72c1f1940 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0d0b90 0x7fb72c0c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 --2- 172.21.15.53:0/2733392764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb72c0d00e0 0x7fb72c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 >> 172.21.15.53:0/2733392764 conn(0x7fb72c071660 msgr2=0x7fb72c073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 shutdown_connections 2023-10-21T16:21:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.356+0000 7fb731440640 1 -- 172.21.15.53:0/2733392764 wait complete. 2023-10-21T16:21:47.380 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2023-10-21T16:21:47.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:47.382 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2023-10-21T16:21:47.447 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2023-10-21T16:21:47.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.670+0000 7f860b997640 1 Processor -- start 2023-10-21T16:21:47.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.670+0000 7f860b997640 1 -- start start 2023-10-21T16:21:47.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f860b997640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 0x7f86040c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f860b997640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f860b997640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860415b310 con 0x7f860406db60 2023-10-21T16:21:47.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f860b997640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040ca230 con 0x7f860406c3a0 2023-10-21T16:21:47.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f860b997640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860406d290 con 0x7f860415aa00 2023-10-21T16:21:47.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f8608f32640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f860406db60 0x7f860406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:36506/0 (socket says 172.21.15.53:36506) 2023-10-21T16:21:47.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f8608f32640 1 -- 172.21.15.53:0/1277946156 learned_addr learned my addr 172.21.15.53:0/1277946156 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:47.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.672+0000 7f8609f34640 1 --2- 172.21.15.53:0/1277946156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609733640 1 --2- 172.21.15.53:0/1277946156 >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 0x7f86040c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609f34640 1 -- 172.21.15.53:0/1277946156 >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 msgr2=0x7f86040c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:47.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609f34640 1 --2- 172.21.15.53:0/1277946156 >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 0x7f86040c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8603fff640 1 -- 172.21.15.53:0/1277946156 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786170674 0 0) 0x7f85f8001110 con 0x7f860406db60 2023-10-21T16:21:47.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609f34640 1 -- 172.21.15.53:0/1277946156 >> v1:172.21.15.124:6789/0 conn(0x7f860406db60 legacy=0x7f860406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609f34640 1 -- 172.21.15.53:0/1277946156 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040c9860 con 0x7f860415aa00 2023-10-21T16:21:47.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.673+0000 7f8609733640 1 --2- 172.21.15.53:0/1277946156 >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 0x7f86040c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T16:21:47.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.675+0000 7f8609f34640 1 --2- 172.21.15.53:0/1277946156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040b2a90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00ba40 tx=0x7f85fc02e080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faedc5a234e11c00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:47.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.675+0000 7f8603fff640 1 -- 172.21.15.53:0/1277946156 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc02ee30 con 0x7f860415aa00 2023-10-21T16:21:47.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.675+0000 7f8603fff640 1 -- 172.21.15.53:0/1277946156 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc036440 con 0x7f860415aa00 2023-10-21T16:21:47.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.675+0000 7f8603fff640 1 -- 172.21.15.53:0/1277946156 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0367d0 con 0x7f860415aa00 2023-10-21T16:21:47.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 -- 172.21.15.53:0/1277946156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 msgr2=0x7f86040b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 --2- 172.21.15.53:0/1277946156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040b2a90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00ba40 tx=0x7f85fc02e080 comp rx=0 tx=0).stop 2023-10-21T16:21:47.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 -- 172.21.15.53:0/1277946156 shutdown_connections 2023-10-21T16:21:47.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 --2- 172.21.15.53:0/1277946156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040b2a90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 --2- 172.21.15.53:0/1277946156 >> v2:172.21.15.124:3300/0 conn(0x7f860406c3a0 0x7f86040c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 -- 172.21.15.53:0/1277946156 >> 172.21.15.53:0/1277946156 conn(0x7f860406a5d0 msgr2=0x7f86040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:47.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 -- 172.21.15.53:0/1277946156 shutdown_connections 2023-10-21T16:21:47.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.676+0000 7f860b997640 1 -- 172.21.15.53:0/1277946156 wait complete. 2023-10-21T16:21:47.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.678+0000 7f860b997640 1 Processor -- start 2023-10-21T16:21:47.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.678+0000 7f860b997640 1 -- start start 2023-10-21T16:21:47.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f860406c3a0 0x7f8604151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f860406db60 0x7f86040c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c4c00 con 0x7f860406c3a0 2023-10-21T16:21:47.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c3ff0 con 0x7f860406db60 2023-10-21T16:21:47.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.679+0000 7f860b997640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c42d0 con 0x7f860415aa00 2023-10-21T16:21:47.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f860406c3a0 0x7f8604151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f860406c3a0 0x7f8604151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60528/0 (socket says 172.21.15.53:60528) 2023-10-21T16:21:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 -- 172.21.15.53:0/2521698966 learned_addr learned my addr 172.21.15.53:0/2521698966 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8608f32640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f860406db60 0x7f86040c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609f34640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 -- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 msgr2=0x7f86040c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 -- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f860406db60 msgr2=0x7f86040c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f860406db60 0x7f86040c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609733640 1 -- 172.21.15.53:0/2521698966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f860406c3a0 2023-10-21T16:21:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.680+0000 7f8609f34640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f860415aa00 0x7f86040c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f8609733640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f860406c3a0 0x7f8604151e30 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f85f400efc0 tx=0x7f85f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:47.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4014680 con 0x7f860406c3a0 2023-10-21T16:21:47.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400f040 con 0x7f860406c3a0 2023-10-21T16:21:47.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4014a00 con 0x7f860406c3a0 2023-10-21T16:21:47.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f860b997640 1 -- 172.21.15.53:0/2521698966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc00b6a0 con 0x7f860406c3a0 2023-10-21T16:21:47.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.681+0000 7f860b997640 1 -- 172.21.15.53:0/2521698966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86040c4890 con 0x7f860406c3a0 2023-10-21T16:21:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.682+0000 7f860b997640 1 -- 172.21.15.53:0/2521698966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860404e160 con 0x7f860406c3a0 2023-10-21T16:21:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.683+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f85f4014b60 con 0x7f860406c3a0 2023-10-21T16:21:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.685+0000 7f8601ffb640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f85f006b070 0x7f85f006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.685+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f85f408ade0 con 0x7f860406c3a0 2023-10-21T16:21:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.685+0000 7f8608f32640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f85f006b070 0x7f85f006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.686+0000 7f8608f32640 1 --2- 172.21.15.53:0/2521698966 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f85f006b070 0x7f85f006d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f860415a220 tx=0x7f85f8007b90 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:47.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.689+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85f4055780 con 0x7f860406c3a0 2023-10-21T16:21:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.906+0000 7f860b997640 1 -- 172.21.15.53:0/2521698966 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) v1 -- 0x7f860415d8f0 con 0x7f85f006b070 2023-10-21T16:21:47.911 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:47.909+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-21T16:21:47.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:47.910+0000 7f8601ffb640 1 -- 172.21.15.53:0/2521698966 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,Y) v1 ==== 74+0+0 (secure 0 0 0) 0x7f860415d8f0 con 0x7f85f006b070 2023-10-21T16:21:47.912 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-21T16:21:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:47.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2023-10-21T16:21:48.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.163+0000 7fbb50fb3640 1 Processor -- start 2023-10-21T16:21:48.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.163+0000 7fbb50fb3640 1 -- start start 2023-10-21T16:21:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb50fb3640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb4c15cae0 0x7fbb4c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb50fb3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb50fb3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c0ccbc0 con 0x7fbb4c0aff70 2023-10-21T16:21:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb50fb3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c0c9510 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb50fb3640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb4c0ce1f0 con 0x7fbb4c15cae0 2023-10-21T16:21:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb4affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb4affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60532/0 (socket says 172.21.15.53:60532) 2023-10-21T16:21:48.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.165+0000 7fbb4affd640 1 -- 172.21.15.53:0/1379122881 learned_addr learned my addr 172.21.15.53:0/1379122881 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:48.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.166+0000 7fbb4b7fe640 1 --2- 172.21.15.53:0/1379122881 >> v2:172.21.15.124:3300/0 conn(0x7fbb4c15cae0 0x7fbb4c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.166+0000 7fbb4affd640 1 -- 172.21.15.53:0/1379122881 >> v2:172.21.15.124:3300/0 conn(0x7fbb4c15cae0 msgr2=0x7fbb4c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:48.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.166+0000 7fbb4affd640 1 --2- 172.21.15.53:0/1379122881 >> v2:172.21.15.124:3300/0 conn(0x7fbb4c15cae0 0x7fbb4c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.166+0000 7fbb4affd640 1 -- 172.21.15.53:0/1379122881 >> v1:172.21.15.53:6789/0 conn(0x7fbb4c0aff70 legacy=0x7fbb4c0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.166+0000 7fbb4affd640 1 -- 172.21.15.53:0/1379122881 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0cd590 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.167+0000 7fbb4affd640 1 --2- 172.21.15.53:0/1379122881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0caed0 tx=0x7fbb4002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c017d112b6e23c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.167+0000 7fbb4a7fc640 1 -- 172.21.15.53:0/1379122881 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40030860 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.167+0000 7fbb4a7fc640 1 -- 172.21.15.53:0/1379122881 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb400309c0 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.167+0000 7fbb4a7fc640 1 -- 172.21.15.53:0/1379122881 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40030d50 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 -- 172.21.15.53:0/1379122881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0c8b40 msgr2=0x7fbb4c0c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 --2- 172.21.15.53:0/1379122881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0caed0 tx=0x7fbb4002b9a0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 -- 172.21.15.53:0/1379122881 shutdown_connections 2023-10-21T16:21:48.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 --2- 172.21.15.53:0/1379122881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0c8b40 0x7fbb4c0c8fd0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 --2- 172.21.15.53:0/1379122881 >> v2:172.21.15.124:3300/0 conn(0x7fbb4c15cae0 0x7fbb4c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 -- 172.21.15.53:0/1379122881 >> 172.21.15.53:0/1379122881 conn(0x7fbb4c06c050 msgr2=0x7fbb4c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:48.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 -- 172.21.15.53:0/1379122881 shutdown_connections 2023-10-21T16:21:48.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.168+0000 7fbb50fb3640 1 -- 172.21.15.53:0/1379122881 wait complete. 2023-10-21T16:21:48.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.171+0000 7fbb50fb3640 1 Processor -- start 2023-10-21T16:21:48.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.171+0000 7fbb50fb3640 1 -- start start 2023-10-21T16:21:48.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0aff70 0x7fbb4c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb4c0c8b40 0x7fbb4c1f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb4c15cae0 0x7fbb4c157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c157f60 con 0x7fbb4c0aff70 2023-10-21T16:21:48.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0c37e0 con 0x7fbb4c0c8b40 2023-10-21T16:21:48.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb50fb3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0c3a30 con 0x7fbb4c15cae0 2023-10-21T16:21:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0aff70 0x7fbb4c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0aff70 0x7fbb4c15f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60538/0 (socket says 172.21.15.53:60538) 2023-10-21T16:21:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.172+0000 7fbb4b7fe640 1 -- 172.21.15.53:0/571387327 learned_addr learned my addr 172.21.15.53:0/571387327 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 -- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb4c15cae0 msgr2=0x7fbb4c157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb4c15cae0 0x7fbb4c157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 -- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb4c0c8b40 msgr2=0x7fbb4c1f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb4c0c8b40 0x7fbb4c1f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 -- 172.21.15.53:0/571387327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400094d0 con 0x7fbb4c0aff70 2023-10-21T16:21:48.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb4b7fe640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbb4c0aff70 0x7fbb4c15f740 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d5f0 tx=0x7fbb3c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0125f0 con 0x7fbb4c0aff70 2023-10-21T16:21:48.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c012750 con 0x7fbb4c0aff70 2023-10-21T16:21:48.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c012b00 con 0x7fbb4c0aff70 2023-10-21T16:21:48.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.173+0000 7fbb50fb3640 1 -- 172.21.15.53:0/571387327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c0c3d10 con 0x7fbb4c0aff70 2023-10-21T16:21:48.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.174+0000 7fbb50fb3640 1 -- 172.21.15.53:0/571387327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c0c42d0 con 0x7fbb4c0aff70 2023-10-21T16:21:48.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.175+0000 7fbb50fb3640 1 -- 172.21.15.53:0/571387327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c074150 con 0x7fbb4c0aff70 2023-10-21T16:21:48.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.176+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fbb3c012c60 con 0x7fbb4c0aff70 2023-10-21T16:21:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.177+0000 7fbb33fff640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbb3806b070 0x7fbb3806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.177+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fbb3c08a200 con 0x7fbb4c0aff70 2023-10-21T16:21:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.178+0000 7fbb4affd640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbb3806b070 0x7fbb3806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.179+0000 7fbb4affd640 1 --2- 172.21.15.53:0/571387327 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbb3806b070 0x7fbb3806d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb40036000 tx=0x7fbb4003b040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.182+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb3c054cb0 con 0x7fbb4c0aff70 2023-10-21T16:21:48.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.397+0000 7fbb50fb3640 1 -- 172.21.15.53:0/571387327 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4c0cc570 con 0x7fbb3806b070 2023-10-21T16:21:48.401 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:48.399+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:48.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.401+0000 7fbb33fff640 1 -- 172.21.15.53:0/571387327 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb40030ca0 con 0x7fbb3806b070 2023-10-21T16:21:48.402 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:48.424 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:48.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2023-10-21T16:21:48.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.658+0000 7fbf7759e640 1 Processor -- start 2023-10-21T16:21:48.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.658+0000 7fbf7759e640 1 -- start start 2023-10-21T16:21:48.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.659+0000 7fbf7759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf7815a820 0x7fbf780aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.659+0000 7fbf7759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 0x7fbf780b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.659+0000 7fbf7759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf780cab60 con 0x7fbf780ca680 2023-10-21T16:21:48.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.659+0000 7fbf7759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf780cb960 con 0x7fbf7815a820 2023-10-21T16:21:48.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.659+0000 7fbf7759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780cbdf0 con 0x7fbf7815ac00 2023-10-21T16:21:48.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf7815a820 0x7fbf780aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf75d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbf780ca680 0x7fbf780c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39010/0 (socket says 172.21.15.53:39010) 2023-10-21T16:21:48.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf75d9b640 1 -- 172.21.15.53:0/2274053983 learned_addr learned my addr 172.21.15.53:0/2274053983 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:48.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7659c640 1 -- 172.21.15.53:0/2274053983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 msgr2=0x7fbf780b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:48.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7659c640 1 --2- 172.21.15.53:0/2274053983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 0x7fbf780b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7659c640 1 -- 172.21.15.53:0/2274053983 >> v1:172.21.15.53:6789/0 conn(0x7fbf780ca680 legacy=0x7fbf780c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7659c640 1 -- 172.21.15.53:0/2274053983 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf780cb030 con 0x7fbf7815a820 2023-10-21T16:21:48.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.660+0000 7fbf7559a640 1 -- 172.21.15.53:0/2274053983 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499961823 0 0) 0x7fbf780cab60 con 0x7fbf780ca680 2023-10-21T16:21:48.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7659c640 1 --2- 172.21.15.53:0/2274053983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7815a820 0x7fbf780aff70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fbf680097a0 tx=0x7fbf6802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3300ad0596bc8bb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7559a640 1 -- 172.21.15.53:0/2274053983 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68031780 con 0x7fbf7815a820 2023-10-21T16:21:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7559a640 1 -- 172.21.15.53:0/2274053983 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf680318e0 con 0x7fbf7815a820 2023-10-21T16:21:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7559a640 1 -- 172.21.15.53:0/2274053983 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68031c70 con 0x7fbf7815a820 2023-10-21T16:21:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7759e640 1 -- 172.21.15.53:0/2274053983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7815a820 msgr2=0x7fbf780aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.662+0000 7fbf7759e640 1 --2- 172.21.15.53:0/2274053983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7815a820 0x7fbf780aff70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fbf680097a0 tx=0x7fbf6802b820 comp rx=0 tx=0).stop 2023-10-21T16:21:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 -- 172.21.15.53:0/2274053983 shutdown_connections 2023-10-21T16:21:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 --2- 172.21.15.53:0/2274053983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 0x7fbf780b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 --2- 172.21.15.53:0/2274053983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf7815a820 0x7fbf780aff70 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 -- 172.21.15.53:0/2274053983 >> 172.21.15.53:0/2274053983 conn(0x7fbf7806c050 msgr2=0x7fbf7806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 -- 172.21.15.53:0/2274053983 shutdown_connections 2023-10-21T16:21:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.663+0000 7fbf7759e640 1 -- 172.21.15.53:0/2274053983 wait complete. 2023-10-21T16:21:48.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.664+0000 7fbf7759e640 1 Processor -- start 2023-10-21T16:21:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.665+0000 7fbf7759e640 1 -- start start 2023-10-21T16:21:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780ca680 0x7fbf781f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf7815a820 0x7fbf780af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 0x7fbf78150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780ab8d0 con 0x7fbf780ca680 2023-10-21T16:21:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780ac450 con 0x7fbf7815a820 2023-10-21T16:21:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf781f1be0 con 0x7fbf7815ac00 2023-10-21T16:21:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780ca680 0x7fbf781f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780ca680 0x7fbf781f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60554/0 (socket says 172.21.15.53:60554) 2023-10-21T16:21:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 -- 172.21.15.53:0/2104494642 learned_addr learned my addr 172.21.15.53:0/2104494642 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 -- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 msgr2=0x7fbf78150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ac00 0x7fbf78150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 -- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf7815a820 msgr2=0x7fbf780af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf7815a820 0x7fbf780af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.666+0000 7fbf7659c640 1 -- 172.21.15.53:0/2104494642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68009400 con 0x7fbf780ca680 2023-10-21T16:21:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf7659c640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780ca680 0x7fbf781f16a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fbf680097a0 tx=0x7fbf680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68004230 con 0x7fbf780ca680 2023-10-21T16:21:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf680043b0 con 0x7fbf780ca680 2023-10-21T16:21:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68004520 con 0x7fbf780ca680 2023-10-21T16:21:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf7759e640 1 -- 172.21.15.53:0/2104494642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf780abb20 con 0x7fbf780ca680 2023-10-21T16:21:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.667+0000 7fbf7759e640 1 -- 172.21.15.53:0/2104494642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf780ac0e0 con 0x7fbf780ca680 2023-10-21T16:21:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.668+0000 7fbf7759e640 1 -- 172.21.15.53:0/2104494642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf780ac270 con 0x7fbf780ca680 2023-10-21T16:21:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.669+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fbf68032860 con 0x7fbf780ca680 2023-10-21T16:21:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.670+0000 7fbf66ffd640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbf5806b070 0x7fbf5806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.670+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fbf680ac1b0 con 0x7fbf780ca680 2023-10-21T16:21:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.671+0000 7fbf75d9b640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbf5806b070 0x7fbf5806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:48.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.672+0000 7fbf75d9b640 1 --2- 172.21.15.53:0/2104494642 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fbf5806b070 0x7fbf5806d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c007fd0 tx=0x7fbf6c007960 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:48.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.674+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf68076c70 con 0x7fbf780ca680 2023-10-21T16:21:48.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.889+0000 7fbf7759e640 1 -- 172.21.15.53:0/2104494642 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf780cb5f0 con 0x7fbf5806b070 2023-10-21T16:21:48.894 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:48.892+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:48.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:48.893+0000 7fbf66ffd640 1 -- 172.21.15.53:0/2104494642 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbf6c0070d0 con 0x7fbf5806b070 2023-10-21T16:21:48.895 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:48.907 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:48.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2023-10-21T16:21:49.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.139+0000 7fa6eef1a640 1 Processor -- start 2023-10-21T16:21:49.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.139+0000 7fa6eef1a640 1 -- start start 2023-10-21T16:21:49.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.140+0000 7fa6eef1a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.140+0000 7fa6eef1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 0x7fa6e80b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.140+0000 7fa6eef1a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e80d2870 con 0x7fa6e80d0b90 2023-10-21T16:21:49.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.140+0000 7fa6eef1a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e80d1150 con 0x7fa6e815ccb0 2023-10-21T16:21:49.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.140+0000 7fa6eef1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80d1430 con 0x7fa6e80d00e0 2023-10-21T16:21:49.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60560/0 (socket says 172.21.15.53:60560) 2023-10-21T16:21:49.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 -- 172.21.15.53:0/1988664581 learned_addr learned my addr 172.21.15.53:0/1988664581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:49.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6ee719640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 0x7fa6e80b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 -- 172.21.15.53:0/1988664581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 msgr2=0x7fa6e80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 0x7fa6e80b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 -- 172.21.15.53:0/1988664581 >> v1:172.21.15.124:6789/0 conn(0x7fa6e80d0b90 legacy=0x7fa6e80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6edf18640 1 -- 172.21.15.53:0/1988664581 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80d1710 con 0x7fa6e815ccb0 2023-10-21T16:21:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.141+0000 7fa6ee719640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 0x7fa6e80b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:49.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.143+0000 7fa6edf18640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009870 tx=0x7fa6e402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969ab1c5b1b21b43 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.143+0000 7fa6ecf16640 1 -- 172.21.15.53:0/1988664581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40317c0 con 0x7fa6e815ccb0 2023-10-21T16:21:49.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.143+0000 7fa6ecf16640 1 -- 172.21.15.53:0/1988664581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4031920 con 0x7fa6e815ccb0 2023-10-21T16:21:49.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.143+0000 7fa6ecf16640 1 -- 172.21.15.53:0/1988664581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4031cb0 con 0x7fa6e815ccb0 2023-10-21T16:21:49.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 -- 172.21.15.53:0/1988664581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e815ccb0 msgr2=0x7fa6e80d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009870 tx=0x7fa6e402b860 comp rx=0 tx=0).stop 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 -- 172.21.15.53:0/1988664581 shutdown_connections 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d00e0 0x7fa6e80b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 --2- 172.21.15.53:0/1988664581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e815ccb0 0x7fa6e80d1fb0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 -- 172.21.15.53:0/1988664581 >> 172.21.15.53:0/1988664581 conn(0x7fa6e8071660 msgr2=0x7fa6e8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:49.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 -- 172.21.15.53:0/1988664581 shutdown_connections 2023-10-21T16:21:49.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.144+0000 7fa6eef1a640 1 -- 172.21.15.53:0/1988664581 wait complete. 2023-10-21T16:21:49.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.146+0000 7fa6eef1a640 1 Processor -- start 2023-10-21T16:21:49.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.146+0000 7fa6eef1a640 1 -- start start 2023-10-21T16:21:49.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e80d00e0 0x7fa6e81f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d0b90 0x7fa6e80c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6e815ccb0 0x7fa6e81f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80c75c0 con 0x7fa6e80d00e0 2023-10-21T16:21:49.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80c78a0 con 0x7fa6e815ccb0 2023-10-21T16:21:49.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6eef1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80c7b80 con 0x7fa6e80d0b90 2023-10-21T16:21:49.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6edf18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e80d00e0 0x7fa6e81f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6edf18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e80d00e0 0x7fa6e81f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60568/0 (socket says 172.21.15.53:60568) 2023-10-21T16:21:49.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.147+0000 7fa6edf18640 1 -- 172.21.15.53:0/2571697123 learned_addr learned my addr 172.21.15.53:0/2571697123 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:49.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6ee719640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6e815ccb0 0x7fa6e81f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6ed717640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d0b90 0x7fa6e80c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 -- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d0b90 msgr2=0x7fa6e80c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6e80d0b90 0x7fa6e80c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 -- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6e815ccb0 msgr2=0x7fa6e81f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6e815ccb0 0x7fa6e81f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 -- 172.21.15.53:0/2571697123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40094d0 con 0x7fa6e80d00e0 2023-10-21T16:21:49.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6ee719640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6e815ccb0 0x7fa6e81f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:49.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6edf18640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa6e80d00e0 0x7fa6e81f4cd0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4000c00 tx=0x7fa6e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40329a0 con 0x7fa6e80d00e0 2023-10-21T16:21:49.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4032b20 con 0x7fa6e80d00e0 2023-10-21T16:21:49.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40453f0 con 0x7fa6e80d00e0 2023-10-21T16:21:49.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.148+0000 7fa6eef1a640 1 -- 172.21.15.53:0/2571697123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e80c7e60 con 0x7fa6e80d00e0 2023-10-21T16:21:49.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.149+0000 7fa6eef1a640 1 -- 172.21.15.53:0/2571697123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80c7200 con 0x7fa6e80d00e0 2023-10-21T16:21:49.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.150+0000 7fa6eef1a640 1 -- 172.21.15.53:0/2571697123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e8074150 con 0x7fa6e80d00e0 2023-10-21T16:21:49.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.151+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fa6e404d3f0 con 0x7fa6e80d00e0 2023-10-21T16:21:49.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.152+0000 7fa6de7fc640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa6cc06b070 0x7fa6cc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.152+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fa6e40abc10 con 0x7fa6e80d00e0 2023-10-21T16:21:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.153+0000 7fa6ed717640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa6cc06b070 0x7fa6cc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.154+0000 7fa6ed717640 1 --2- 172.21.15.53:0/2571697123 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa6cc06b070 0x7fa6cc06d530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6e815b150 tx=0x7fa6d8009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.157+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6e4076760 con 0x7fa6e80d00e0 2023-10-21T16:21:49.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.371+0000 7fa6eef1a640 1 -- 172.21.15.53:0/2571697123 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e80b5b10 con 0x7fa6cc06b070 2023-10-21T16:21:49.375 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:49.374+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.375+0000 7fa6de7fc640 1 -- 172.21.15.53:0/2571697123 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6d8003980 con 0x7fa6cc06b070 2023-10-21T16:21:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:49.387 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:49.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2023-10-21T16:21:49.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.617+0000 7fcaad56a640 1 Processor -- start 2023-10-21T16:21:49.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.617+0000 7fcaad56a640 1 -- start start 2023-10-21T16:21:49.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaad56a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcaa80b0680 0x7fcaa815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaad56a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 0x7fcaa815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaad56a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa80cab60 con 0x7fcaa815a820 2023-10-21T16:21:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaad56a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa80cb960 con 0x7fcaa80b0680 2023-10-21T16:21:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaad56a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80cbdf0 con 0x7fcaa80ca680 2023-10-21T16:21:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.619+0000 7fcaa7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcaa815a820 0x7fcaa80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39026/0 (socket says 172.21.15.53:39026) 2023-10-21T16:21:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa7fff640 1 -- 172.21.15.53:0/2978527890 learned_addr learned my addr 172.21.15.53:0/2978527890 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa77fe640 1 --2- 172.21.15.53:0/2978527890 >> v2:172.21.15.124:3300/0 conn(0x7fcaa80b0680 0x7fcaa815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaacd69640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 0x7fcaa815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903286142 0 0) 0x7fcaa80cab60 con 0x7fcaa815a820 2023-10-21T16:21:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca90003520 con 0x7fcaa815a820 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005895037 0 0) 0x7fca90003520 con 0x7fcaa815a820 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 msgr2=0x7fcaa815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 0x7fcaa815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 >> v2:172.21.15.124:3300/0 conn(0x7fcaa80b0680 msgr2=0x7fcaa815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 --2- 172.21.15.53:0/2978527890 >> v2:172.21.15.124:3300/0 conn(0x7fcaa80b0680 0x7fcaa815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.620+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80cb030 con 0x7fcaa815a820 2023-10-21T16:21:49.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fca98001670 con 0x7fcaa815a820 2023-10-21T16:21:49.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 >> v1:172.21.15.53:6789/0 conn(0x7fcaa815a820 legacy=0x7fcaa80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 0x7fca90003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003f40 0x7fca900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90004d00 0x7fca90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80cb960 con 0x7fca90004d00 2023-10-21T16:21:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80cbdf0 con 0x7fca90003f40 2023-10-21T16:21:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9000ba30 con 0x7fcaa80ca680 2023-10-21T16:21:49.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90004d00 0x7fca90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 -- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 msgr2=0x7fca90003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80ca680 0x7fca90003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 -- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003f40 msgr2=0x7fca900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90003f40 0x7fca900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.621+0000 7fcaacd69640 1 -- 172.21.15.53:0/2978527890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca90003520 con 0x7fca90004d00 2023-10-21T16:21:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaacd69640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90004d00 0x7fca90005180 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009f90 tx=0x7fca900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5845895f9966f53f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa002cc20 con 0x7fca90004d00 2023-10-21T16:21:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa000f040 con 0x7fca90004d00 2023-10-21T16:21:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaa6ffd640 1 -- 172.21.15.53:0/2978527890 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0034440 con 0x7fca90004d00 2023-10-21T16:21:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaad56a640 1 -- 172.21.15.53:0/2978527890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90004d00 msgr2=0x7fca90005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaad56a640 1 --2- 172.21.15.53:0/2978527890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca90004d00 0x7fca90005180 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009f90 tx=0x7fca900057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaacd69640 1 -- 172.21.15.53:0/2978527890 reap_dead start 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaad56a640 1 -- 172.21.15.53:0/2978527890 shutdown_connections 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.623+0000 7fcaad56a640 1 -- 172.21.15.53:0/2978527890 >> 172.21.15.53:0/2978527890 conn(0x7fcaa806c050 msgr2=0x7fcaa806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.624+0000 7fcaad56a640 1 -- 172.21.15.53:0/2978527890 shutdown_connections 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.624+0000 7fcaad56a640 1 -- 172.21.15.53:0/2978527890 wait complete. 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.625+0000 7fcaad56a640 1 Processor -- start 2023-10-21T16:21:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.626+0000 7fcaad56a640 1 -- start start 2023-10-21T16:21:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90004d00 0x7fcaa80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa80cd9a0 0x7fcaa80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80cc0c0 0x7fcaa80cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80c2a30 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80c89f0 con 0x7fca90004d00 2023-10-21T16:21:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaad56a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa80c2c40 con 0x7fcaa80cc0c0 2023-10-21T16:21:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa80cd9a0 0x7fcaa80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa80cd9a0 0x7fcaa80cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60586/0 (socket says 172.21.15.53:60586) 2023-10-21T16:21:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 -- 172.21.15.53:0/1457462670 learned_addr learned my addr 172.21.15.53:0/1457462670 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 -- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80cc0c0 msgr2=0x7fcaa80cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcaa80cc0c0 0x7fcaa80cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa7fff640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90004d00 0x7fcaa80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 -- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90004d00 msgr2=0x7fcaa80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90004d00 0x7fcaa80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa77fe640 1 -- 172.21.15.53:0/1457462670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80c2f20 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.627+0000 7fcaa7fff640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca90004d00 0x7fcaa80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaa77fe640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaa80cd9a0 0x7fcaa80cdde0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00e7c0 tx=0x7fca9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00eed0 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c00c860 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaad56a640 1 -- 172.21.15.53:0/1457462670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa80c3200 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00cc10 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.628+0000 7fcaad56a640 1 -- 172.21.15.53:0/1457462670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa80c37c0 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.629+0000 7fcaad56a640 1 -- 172.21.15.53:0/1457462670 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa8072050 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.630+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fca9c00cd70 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.631+0000 7fcaa4ff9640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fca9406b070 0x7fca9406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.631+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fca9c089dc0 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.632+0000 7fcaa7fff640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fca9406b070 0x7fca9406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.633+0000 7fcaa7fff640 1 --2- 172.21.15.53:0/1457462670 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fca9406b070 0x7fca9406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fca98007b10 tx=0x7fca98009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:49.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.635+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca9c0547e0 con 0x7fcaa80cd9a0 2023-10-21T16:21:49.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.855+0000 7fcaad56a640 1 -- 172.21.15.53:0/1457462670 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaa80cb5f0 con 0x7fca9406b070 2023-10-21T16:21:49.859 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:49.857+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-21T16:21:49.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:49.859+0000 7fcaa4ff9640 1 -- 172.21.15.53:0/1457462670 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) v1 ==== 96+0+0 (secure 0 0 0) 0x7fca9800d8c0 con 0x7fca9406b070 2023-10-21T16:21:49.860 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-21T16:21:49.874 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:49.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2023-10-21T16:21:50.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.103+0000 7fb84b59e640 1 Processor -- start 2023-10-21T16:21:50.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.103+0000 7fb84b59e640 1 -- start start 2023-10-21T16:21:50.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.104+0000 7fb84b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb84c0b0680 0x7fb84c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.104+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb84c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.104+0000 7fb84b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c0cab60 con 0x7fb84c15a820 2023-10-21T16:21:50.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.104+0000 7fb84b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c0cb960 con 0x7fb84c0b0680 2023-10-21T16:21:50.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.104+0000 7fb84b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0cbdf0 con 0x7fb84c0ca680 2023-10-21T16:21:50.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb84a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb84c15a820 0x7fb84c0aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39030/0 (socket says 172.21.15.53:39030) 2023-10-21T16:21:50.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb84a59c640 1 -- 172.21.15.53:0/2893937723 learned_addr learned my addr 172.21.15.53:0/2893937723 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb849d9b640 1 --2- 172.21.15.53:0/2893937723 >> v2:172.21.15.124:3300/0 conn(0x7fb84c0b0680 0x7fb84c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb84c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636125770 0 0) 0x7fb84c0cab60 con 0x7fb84c15a820 2023-10-21T16:21:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.105+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb838003520 con 0x7fb84c15a820 2023-10-21T16:21:50.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 726571797 0 0) 0x7fb838003520 con 0x7fb84c15a820 2023-10-21T16:21:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 msgr2=0x7fb84c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb84c15ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 >> v2:172.21.15.124:3300/0 conn(0x7fb84c0b0680 msgr2=0x7fb84c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:50.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 --2- 172.21.15.53:0/2893937723 >> v2:172.21.15.124:3300/0 conn(0x7fb84c0b0680 0x7fb84c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0cb030 con 0x7fb84c15a820 2023-10-21T16:21:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb83c0019f0 con 0x7fb84c15a820 2023-10-21T16:21:50.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 >> v1:172.21.15.53:6789/0 conn(0x7fb84c15a820 legacy=0x7fb84c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838003970 0x7fb838003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb8380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb838004d00 0x7fb838005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0cb960 con 0x7fb838004d00 2023-10-21T16:21:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0cbdf0 con 0x7fb838003970 2023-10-21T16:21:50.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83800b980 con 0x7fb84c0ca680 2023-10-21T16:21:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.106+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb838004d00 0x7fb838005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84a59c640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838003970 0x7fb838003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb849d9b640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb8380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84ad9d640 1 -- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 msgr2=0x7fb8380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c0ca680 0x7fb8380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84ad9d640 1 -- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838003970 msgr2=0x7fb838003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838003970 0x7fb838003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.107+0000 7fb84ad9d640 1 -- 172.21.15.53:0/2893937723 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838003520 con 0x7fb838004d00 2023-10-21T16:21:50.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.108+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb838004d00 0x7fb838005180 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb844009a90 tx=0x7fb8380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb5a590363c9469 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:50.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.108+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84402c680 con 0x7fb838004d00 2023-10-21T16:21:50.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.108+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84402c7e0 con 0x7fb838004d00 2023-10-21T16:21:50.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.108+0000 7fb84959a640 1 -- 172.21.15.53:0/2893937723 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84402cb70 con 0x7fb838004d00 2023-10-21T16:21:50.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 -- 172.21.15.53:0/2893937723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb838004d00 msgr2=0x7fb838005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:50.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 --2- 172.21.15.53:0/2893937723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb838004d00 0x7fb838005180 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb844009a90 tx=0x7fb8380057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84ad9d640 1 -- 172.21.15.53:0/2893937723 reap_dead start 2023-10-21T16:21:50.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 -- 172.21.15.53:0/2893937723 shutdown_connections 2023-10-21T16:21:50.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 -- 172.21.15.53:0/2893937723 >> 172.21.15.53:0/2893937723 conn(0x7fb84c06c050 msgr2=0x7fb84c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:50.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 -- 172.21.15.53:0/2893937723 shutdown_connections 2023-10-21T16:21:50.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.109+0000 7fb84b59e640 1 -- 172.21.15.53:0/2893937723 wait complete. 2023-10-21T16:21:50.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.111+0000 7fb84b59e640 1 Processor -- start 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.111+0000 7fb84b59e640 1 -- start start 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838004d00 0x7fb84c0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c15ac00 0x7fb84c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0ccb20 0x7fb84c0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0cd4e0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c15bd10 con 0x7fb838004d00 2023-10-21T16:21:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0c42b0 con 0x7fb84c15ac00 2023-10-21T16:21:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0ccb20 0x7fb84c0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0ccb20 0x7fb84c0ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60612/0 (socket says 172.21.15.53:60612) 2023-10-21T16:21:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.112+0000 7fb84ad9d640 1 -- 172.21.15.53:0/715025850 learned_addr learned my addr 172.21.15.53:0/715025850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:50.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84a59c640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838004d00 0x7fb84c0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb849d9b640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c15ac00 0x7fb84c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 -- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c15ac00 msgr2=0x7fb84c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:50.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb84c15ac00 0x7fb84c14da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 -- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838004d00 msgr2=0x7fb84c0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb838004d00 0x7fb84c0c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 -- 172.21.15.53:0/715025850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c14dfc0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb84ad9d640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb84c0ccb20 0x7fb84c0ccfa0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb844037040 tx=0x7fb844004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.113+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84402c680 con 0x7fb84c0ccb20 2023-10-21T16:21:50.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.114+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844003810 con 0x7fb84c0ccb20 2023-10-21T16:21:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.114+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84402dea0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.114+0000 7fb84b59e640 1 -- 172.21.15.53:0/715025850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c15b100 con 0x7fb84c0ccb20 2023-10-21T16:21:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.114+0000 7fb84b59e640 1 -- 172.21.15.53:0/715025850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c15b6c0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.115+0000 7fb84b59e640 1 -- 172.21.15.53:0/715025850 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c074150 con 0x7fb84c0ccb20 2023-10-21T16:21:50.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.116+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fb844003bc0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.117+0000 7fb842ffd640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb82c06b070 0x7fb82c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:50.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.117+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fb8440aac10 con 0x7fb84c0ccb20 2023-10-21T16:21:50.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.117+0000 7fb84a59c640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb82c06b070 0x7fb82c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:50.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.119+0000 7fb84a59c640 1 --2- 172.21.15.53:0/715025850 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fb82c06b070 0x7fb82c06d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000f80 tx=0x7fb83c009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:50.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.122+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8440756c0 con 0x7fb84c0ccb20 2023-10-21T16:21:50.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.335+0000 7fb84b59e640 1 -- 172.21.15.53:0/715025850 --> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fb84c0cb5f0 con 0x7fb82c06b070 2023-10-21T16:21:50.339 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:50.337+0000 7fbc47cfd640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:50.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:50.338+0000 7fb842ffd640 1 -- 172.21.15.53:0/715025850 <== mgr.7396 v2:172.21.15.124:6832/1829772531 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb83c015460 con 0x7fb82c06b070 2023-10-21T16:21:50.340 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-21T16:21:50.353 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T16:21:50.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.354 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2023-10-21T16:21:50.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.416 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2023-10-21T16:21:50.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.479 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2023-10-21T16:21:50.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.546 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2023-10-21T16:21:50.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.610 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2023-10-21T16:21:50.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.673 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2023-10-21T16:21:50.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:50.739 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2023-10-21T16:21:50.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T16:21:51.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.031+0000 7fd1b59c4640 1 Processor -- start 2023-10-21T16:21:51.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.031+0000 7fd1b59c4640 1 -- start start 2023-10-21T16:21:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.033+0000 7fd1b59c4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd1b00c18d0 0x7fd1b00c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.033+0000 7fd1b59c4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0161590 con 0x7fd1b0164d10 2023-10-21T16:21:51.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.033+0000 7fd1b59c4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00cc570 con 0x7fd1b00aff70 2023-10-21T16:21:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.033+0000 7fd1b59c4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b00c94b0 con 0x7fd1b00c18d0 2023-10-21T16:21:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd1b00c18d0 0x7fd1b00c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd1b00c18d0 0x7fd1b00c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60624/0 (socket says 172.21.15.53:60624) 2023-10-21T16:21:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 -- 172.21.15.53:0/699759098 learned_addr learned my addr 172.21.15.53:0/699759098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:51.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1b51c3640 1 --1- 172.21.15.53:0/699759098 >> v1:172.21.15.53:6789/0 conn(0x7fd1b0164d10 0x7fd1b00ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39042/0 (socket says 172.21.15.53:39042) 2023-10-21T16:21:51.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 -- 172.21.15.53:0/699759098 >> v1:172.21.15.124:6789/0 conn(0x7fd1b00aff70 legacy=0x7fd1b00b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 -- 172.21.15.53:0/699759098 >> v1:172.21.15.53:6789/0 conn(0x7fd1b0164d10 legacy=0x7fd1b00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.034+0000 7fd1affff640 1 -- 172.21.15.53:0/699759098 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00c9790 con 0x7fd1b00c18d0 2023-10-21T16:21:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.035+0000 7fd1affff640 1 --2- 172.21.15.53:0/699759098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b00c18d0 0x7fd1b00c8b40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009870 tx=0x7fd19c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7863a5c1cacab9f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.036+0000 7fd1af7fe640 1 -- 172.21.15.53:0/699759098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0317e0 con 0x7fd1b00c18d0 2023-10-21T16:21:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.036+0000 7fd1af7fe640 1 -- 172.21.15.53:0/699759098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c031940 con 0x7fd1b00c18d0 2023-10-21T16:21:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.036+0000 7fd1af7fe640 1 -- 172.21.15.53:0/699759098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c031cd0 con 0x7fd1b00c18d0 2023-10-21T16:21:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.036+0000 7fd1b59c4640 1 -- 172.21.15.53:0/699759098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b00c18d0 msgr2=0x7fd1b00c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.036+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/699759098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b00c18d0 0x7fd1b00c8b40 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009870 tx=0x7fd19c02b880 comp rx=0 tx=0).stop 2023-10-21T16:21:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.037+0000 7fd1b59c4640 1 -- 172.21.15.53:0/699759098 shutdown_connections 2023-10-21T16:21:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.037+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/699759098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b00c18d0 0x7fd1b00c8b40 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.037+0000 7fd1b59c4640 1 -- 172.21.15.53:0/699759098 >> 172.21.15.53:0/699759098 conn(0x7fd1b006c050 msgr2=0x7fd1b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:51.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.037+0000 7fd1b59c4640 1 -- 172.21.15.53:0/699759098 shutdown_connections 2023-10-21T16:21:51.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.037+0000 7fd1b59c4640 1 -- 172.21.15.53:0/699759098 wait complete. 2023-10-21T16:21:51.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.038+0000 7fd1b59c4640 1 Processor -- start 2023-10-21T16:21:51.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.039+0000 7fd1b59c4640 1 -- start start 2023-10-21T16:21:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 0x7fd1b00c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1b00c18d0 0x7fd1b0163310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:51.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0163850 con 0x7fd1b0164d10 2023-10-21T16:21:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00c6ea0 con 0x7fd1b00aff70 2023-10-21T16:21:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b59c4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00c7180 con 0x7fd1b00c18d0 2023-10-21T16:21:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60634/0 (socket says 172.21.15.53:60634) 2023-10-21T16:21:51.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 -- 172.21.15.53:0/4182418776 learned_addr learned my addr 172.21.15.53:0/4182418776 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b49c2640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 0x7fd1b00c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1affff640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1b00c18d0 0x7fd1b0163310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 -- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1b00c18d0 msgr2=0x7fd1b0163310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1b00c18d0 0x7fd1b0163310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 -- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 msgr2=0x7fd1b00c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 0x7fd1b00c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.040+0000 7fd1b51c3640 1 -- 172.21.15.53:0/4182418776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0094d0 con 0x7fd1b0164d10 2023-10-21T16:21:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1b49c2640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 0x7fd1b00c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1b51c3640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c730 tx=0x7fd1a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800ce40 con 0x7fd1b0164d10 2023-10-21T16:21:51.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800a640 con 0x7fd1b0164d10 2023-10-21T16:21:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800a9f0 con 0x7fd1b0164d10 2023-10-21T16:21:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b00c7460 con 0x7fd1b0164d10 2023-10-21T16:21:51.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.041+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b00c7a20 con 0x7fd1b0164d10 2023-10-21T16:21:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.042+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b00cc7f0 con 0x7fd1b0164d10 2023-10-21T16:21:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.044+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 70) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fd1a800ab50 con 0x7fd1b0164d10 2023-10-21T16:21:51.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.044+0000 7fd1ad7fa640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 0x7fd19406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.045+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fd1a80893a0 con 0x7fd1b0164d10 2023-10-21T16:21:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.045+0000 7fd1b49c2640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 0x7fd19406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:51.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.046+0000 7fd1b49c2640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 0x7fd19406d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4003eb0 tx=0x7fd1a4009040 comp rx=0 tx=0).ready entity=mgr.7396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:51.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.049+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1b00cc7f0 con 0x7fd1b0164d10 2023-10-21T16:21:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.294+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd1b00cc7f0 con 0x7fd1b0164d10 2023-10-21T16:21:51.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.447+0000 7fd1ad7fa640 1 -- 172.21.15.53:0/4182418776 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v71) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd1b00cc7f0 con 0x7fd1b0164d10 2023-10-21T16:21:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.450+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 msgr2=0x7fd19406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.450+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 0x7fd19406d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4003eb0 tx=0x7fd1a4009040 comp rx=0 tx=0).stop 2023-10-21T16:21:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.450+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 msgr2=0x7fd1b015fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:51.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.450+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c730 tx=0x7fd1a800cbf0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.450+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 shutdown_connections 2023-10-21T16:21:51.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fd19406b070 0x7fd19406d530 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4003eb0 tx=0x7fd1a4009040 comp rx=0 tx=0).stop 2023-10-21T16:21:51.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1b0164d10 0x7fd1b015fb30 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1b00c18d0 0x7fd1b0163310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 --2- 172.21.15.53:0/4182418776 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1b00aff70 0x7fd1b00c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:51.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 >> 172.21.15.53:0/4182418776 conn(0x7fd1b006c050 msgr2=0x7fd1b00af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:51.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 shutdown_connections 2023-10-21T16:21:51.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:51.451+0000 7fd1b59c4640 1 -- 172.21.15.53:0/4182418776 wait complete. 2023-10-21T16:21:51.478 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:21:51.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:51.600 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:21:51.699 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:21:51.699 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:51.700 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:51.755 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:51.756 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:51.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:51.987 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:51.985+0000 7fca50637040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:52.014 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.012+0000 7f0db7ac9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:21:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.024+0000 7f79b09c9640 1 Processor -- start 2023-10-21T16:21:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.024+0000 7f79b09c9640 1 -- start start 2023-10-21T16:21:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79b09c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79ac158710 0x7f79ac15f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79b09c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79b09c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac158af0 con 0x7f79ac15bad0 2023-10-21T16:21:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79b09c9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac15ee70 con 0x7f79ac158710 2023-10-21T16:21:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79b09c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0ca370 con 0x7f79ac0b1280 2023-10-21T16:21:52.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.025+0000 7f79aa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79ac158710 0x7f79ac15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f79ac158710 0x7f79ac15f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60658/0 (socket says 172.21.15.53:60658) 2023-10-21T16:21:52.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 -- 172.21.15.53:0/124980116 learned_addr learned my addr 172.21.15.53:0/124980116 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aad9d640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 -- 172.21.15.53:0/124980116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 msgr2=0x7f79ac069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 -- 172.21.15.53:0/124980116 >> v1:172.21.15.124:6789/0 conn(0x7f79ac15bad0 legacy=0x7f79ac0b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.026+0000 7f79aa59c640 1 -- 172.21.15.53:0/124980116 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac15e7b0 con 0x7f79ac158710 2023-10-21T16:21:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.027+0000 7f79aa59c640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac15f290 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f79ac071f80 tx=0x7f799c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208427fa06a920eb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.028+0000 7f79a959a640 1 -- 172.21.15.53:0/124980116 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c02e070 con 0x7f79ac158710 2023-10-21T16:21:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.028+0000 7f79a959a640 1 -- 172.21.15.53:0/124980116 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c0315a0 con 0x7f79ac158710 2023-10-21T16:21:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.028+0000 7f79a959a640 1 -- 172.21.15.53:0/124980116 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c031930 con 0x7f79ac158710 2023-10-21T16:21:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 -- 172.21.15.53:0/124980116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 msgr2=0x7f79ac15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac15f290 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f79ac071f80 tx=0x7f799c02b840 comp rx=0 tx=0).stop 2023-10-21T16:21:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 -- 172.21.15.53:0/124980116 shutdown_connections 2023-10-21T16:21:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 --2- 172.21.15.53:0/124980116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac15f290 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 -- 172.21.15.53:0/124980116 >> 172.21.15.53:0/124980116 conn(0x7f79ac0cf890 msgr2=0x7f79ac0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 -- 172.21.15.53:0/124980116 shutdown_connections 2023-10-21T16:21:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.029+0000 7f79b09c9640 1 -- 172.21.15.53:0/124980116 wait complete. 2023-10-21T16:21:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.031+0000 7f79b09c9640 1 Processor -- start 2023-10-21T16:21:52.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.031+0000 7f79b09c9640 1 -- start start 2023-10-21T16:21:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac1f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79ac15bad0 0x7f79ac15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0c5440 con 0x7f79ac158710 2023-10-21T16:21:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0c4830 con 0x7f79ac15bad0 2023-10-21T16:21:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79b09c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0c4b10 con 0x7f79ac0b1280 2023-10-21T16:21:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79a9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79a9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60668/0 (socket says 172.21.15.53:60668) 2023-10-21T16:21:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79a9d9b640 1 -- 172.21.15.53:0/659152895 learned_addr learned my addr 172.21.15.53:0/659152895 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.032+0000 7f79aa59c640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac1f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79aad9d640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79ac15bad0 0x7f79ac15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 msgr2=0x7f79ac1f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac1f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79ac15bad0 msgr2=0x7f79ac15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79ac15bad0 0x7f79ac15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 -- 172.21.15.53:0/659152895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0094d0 con 0x7f79ac158710 2023-10-21T16:21:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a9d9b640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7c0 tx=0x7f799400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400aa50 con 0x7f79ac158710 2023-10-21T16:21:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac0c4df0 con 0x7f79ac158710 2023-10-21T16:21:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799400abb0 con 0x7f79ac158710 2023-10-21T16:21:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.033+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994012420 con 0x7f79ac158710 2023-10-21T16:21:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.034+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac0c3a90 con 0x7f79ac158710 2023-10-21T16:21:52.041 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.038+0000 7fca50637040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.035+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac06b770 con 0x7f79ac158710 2023-10-21T16:21:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.036+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f7994012580 con 0x7f79ac158710 2023-10-21T16:21:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.037+0000 7f79a2ffd640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 0x7f798c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.037+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f7994089f90 con 0x7f79ac158710 2023-10-21T16:21:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.037+0000 7f79aa59c640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 msgr2=0x7f798c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.037+0000 7f79aa59c640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 0x7f798c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.041+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7994054a50 con 0x7f79ac158710 2023-10-21T16:21:52.078 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.075+0000 7f0db7ac9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:21:52.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.237+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79ac06b3e0 con 0x7f79ac158710 2023-10-21T16:21:52.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.238+0000 7f79aa59c640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 msgr2=0x7f798c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:52.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.238+0000 7f79aa59c640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 0x7f798c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:52.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.238+0000 7f79a2ffd640 1 -- 172.21.15.53:0/659152895 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+17392 (secure 0 0 0) 0x7f7994058070 con 0x7f79ac158710 2023-10-21T16:21:52.242 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:52.242 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":149,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:42.595553+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2276161522":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/4259428584":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6833/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6832/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/907346555":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/667593812":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2229194042":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2878176284":"2023-10-22T16:21:42.595479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:52.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 msgr2=0x7f798c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 0x7f798c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 msgr2=0x7f79ac159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7c0 tx=0x7f799400cc80 comp rx=0 tx=0).stop 2023-10-21T16:21:52.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 shutdown_connections 2023-10-21T16:21:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f798c06b020 0x7f798c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79ac15bad0 0x7f79ac15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79ac158710 0x7f79ac159cd0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 --2- 172.21.15.53:0/659152895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79ac0b1280 0x7f79ac1f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 >> 172.21.15.53:0/659152895 conn(0x7f79ac0cf890 msgr2=0x7f79ac15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:52.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 shutdown_connections 2023-10-21T16:21:52.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.241+0000 7f79b09c9640 1 -- 172.21.15.53:0/659152895 wait complete. 2023-10-21T16:21:52.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:21:25.840789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:21:26.894786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:52.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:52.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.474+0000 7f209161a640 1 Processor -- start 2023-10-21T16:21:52.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.475+0000 7f209161a640 1 -- start start 2023-10-21T16:21:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.476+0000 7f209161a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f208c15a9f0 0x7f208c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.476+0000 7f209161a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c0cd710 con 0x7f208c0ae390 2023-10-21T16:21:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.476+0000 7f209161a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c0cda00 con 0x7f208c06ad60 2023-10-21T16:21:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.476+0000 7f209161a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c0661f0 con 0x7f208c15a9f0 2023-10-21T16:21:52.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f208c0ae390 0x7f208c0cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39066/0 (socket says 172.21.15.53:39066) 2023-10-21T16:21:52.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208b7fe640 1 -- 172.21.15.53:0/2420841043 learned_addr learned my addr 172.21.15.53:0/2420841043 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208bfff640 1 --2- 172.21.15.53:0/2420841043 >> v2:172.21.15.124:3300/0 conn(0x7f208c15a9f0 0x7f208c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227481454 0 0) 0x7f208c0cd710 con 0x7f208c0ae390 2023-10-21T16:21:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2074003520 con 0x7f208c0ae390 2023-10-21T16:21:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793827636 0 0) 0x7f2074003520 con 0x7f208c0ae390 2023-10-21T16:21:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 >> v2:172.21.15.124:3300/0 conn(0x7f208c15a9f0 msgr2=0x7f208c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 --2- 172.21.15.53:0/2420841043 >> v2:172.21.15.124:3300/0 conn(0x7f208c15a9f0 0x7f208c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 >> v1:172.21.15.124:6789/0 conn(0x7f208c06ad60 legacy=0x7f208c0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c0cdce0 con 0x7f208c0ae390 2023-10-21T16:21:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2080001670 con 0x7f208c0ae390 2023-10-21T16:21:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.477+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 >> v1:172.21.15.53:6789/0 conn(0x7f208c0ae390 legacy=0x7f208c0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074003970 0x7f2074003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20740047c0 0x7f2074004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 0x7f207400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207400bff0 con 0x7f2074005180 2023-10-21T16:21:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207400c160 con 0x7f20740047c0 2023-10-21T16:21:52.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207400c320 con 0x7f2074003970 2023-10-21T16:21:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 0x7f207400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208b7fe640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20740047c0 0x7f2074004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208bfff640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074003970 0x7f2074003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 -- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074003970 msgr2=0x7f2074003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074003970 0x7f2074003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 -- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20740047c0 msgr2=0x7f2074004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20740047c0 0x7f2074004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f2090e19640 1 -- 172.21.15.53:0/2420841043 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074003520 con 0x7f2074005180 2023-10-21T16:21:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.478+0000 7f208bfff640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074003970 0x7f2074003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.479+0000 7f208b7fe640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20740047c0 0x7f2074004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.479+0000 7f2090e19640 1 -- 172.21.15.53:0/2420841043 reap_dead start 2023-10-21T16:21:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.480+0000 7f2090e19640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 0x7f207400b980 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2084001e20 tx=0x7f20740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ad95fa79ef4e61 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.480+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084003b70 con 0x7f2074005180 2023-10-21T16:21:52.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.480+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2084003cd0 con 0x7f2074005180 2023-10-21T16:21:52.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.480+0000 7f208affd640 1 -- 172.21.15.53:0/2420841043 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084005310 con 0x7f2074005180 2023-10-21T16:21:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 -- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 msgr2=0x7f207400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 0x7f207400b980 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f2084001e20 tx=0x7f20740057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 -- 172.21.15.53:0/2420841043 shutdown_connections 2023-10-21T16:21:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 --2- 172.21.15.53:0/2420841043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2074005180 0x7f207400b980 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 -- 172.21.15.53:0/2420841043 >> 172.21.15.53:0/2420841043 conn(0x7f208c0b08a0 msgr2=0x7f208c0b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 -- 172.21.15.53:0/2420841043 shutdown_connections 2023-10-21T16:21:52.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.481+0000 7f209161a640 1 -- 172.21.15.53:0/2420841043 wait complete. 2023-10-21T16:21:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.483+0000 7f209161a640 1 Processor -- start 2023-10-21T16:21:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.483+0000 7f209161a640 1 -- start start 2023-10-21T16:21:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074005180 0x7f208c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f208c0c8490 0x7f208c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0c8910 con 0x7f208c1f1890 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c158070 con 0x7f208c0c8490 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f209161a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c1f64e0 con 0x7f2074005180 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60688/0 (socket says 172.21.15.53:60688) 2023-10-21T16:21:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 -- 172.21.15.53:0/3054171475 learned_addr learned my addr 172.21.15.53:0/3054171475 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208bfff640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074005180 0x7f208c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f2090e19640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f208c0c8490 0x7f208c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 -- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074005180 msgr2=0x7f208c157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074005180 0x7f208c157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 -- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f208c0c8490 msgr2=0x7f208c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.484+0000 7f208b7fe640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f208c0c8490 0x7f208c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f208b7fe640 1 -- 172.21.15.53:0/3054171475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c156d70 con 0x7f208c1f1890 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f208b7fe640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f208000d7a0 tx=0x7f208000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800074e0 con 0x7f208c1f1890 2023-10-21T16:21:52.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080007640 con 0x7f208c1f1890 2023-10-21T16:21:52.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800049e0 con 0x7f208c1f1890 2023-10-21T16:21:52.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c157050 con 0x7f208c1f1890 2023-10-21T16:21:52.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.485+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c157610 con 0x7f208c1f1890 2023-10-21T16:21:52.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.487+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c074150 con 0x7f208c1f1890 2023-10-21T16:21:52.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.487+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f2080004b40 con 0x7f208c1f1890 2023-10-21T16:21:52.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.488+0000 7f2088ff9640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 0x7f207806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.488+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f2080089530 con 0x7f208c1f1890 2023-10-21T16:21:52.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.489+0000 7f208bfff640 1 -- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 msgr2=0x7f207806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:52.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.489+0000 7f208bfff640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 0x7f207806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:52.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.490+0000 7fca50637040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:52.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.493+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2080053f70 con 0x7f208c1f1890 2023-10-21T16:21:52.546 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.543+0000 7fca50637040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:52.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:21:52.571 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:21:52.572 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.565+0000 7f0db7ac9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:21:52.596 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.595+0000 7fca50637040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:52.623 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.621+0000 7f0db7ac9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:21:52.661 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.659+0000 7fca50637040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:52.677 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.674+0000 7f0db7ac9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:21:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.684+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f208c06b670 con 0x7f208c1f1890 2023-10-21T16:21:52.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.685+0000 7f2088ff9640 1 -- 172.21.15.53:0/3054171475 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2080057590 con 0x7f208c1f1890 2023-10-21T16:21:52.688 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:52.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 msgr2=0x7f207806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 0x7f207806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 msgr2=0x7f208c0c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f208000d7a0 tx=0x7f208000dc60 comp rx=0 tx=0).stop 2023-10-21T16:21:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 shutdown_connections 2023-10-21T16:21:52.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f207806b050 0x7f207806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f208c0c8490 0x7f208c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208c1f1890 0x7f208c0c7f50 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 --2- 172.21.15.53:0/3054171475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2074005180 0x7f208c157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 >> 172.21.15.53:0/3054171475 conn(0x7f208c0b08a0 msgr2=0x7f208c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:52.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 shutdown_connections 2023-10-21T16:21:52.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.687+0000 7f209161a640 1 -- 172.21.15.53:0/3054171475 wait complete. 2023-10-21T16:21:52.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:21:52.712 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.710+0000 7fca50637040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:52.743 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.740+0000 7f0db7ac9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:21:52.795 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.793+0000 7f0db7ac9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:21:52.822 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.820+0000 7fca50637040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:52.885 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.883+0000 7fca50637040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:52.914 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.912+0000 7f0db7ac9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:21:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.916+0000 7f9e17fff640 1 Processor -- start 2023-10-21T16:21:52.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.916+0000 7f9e17fff640 1 -- start start 2023-10-21T16:21:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e180cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9e180b3c50 0x7f9e180caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e180cb410 0x7f9e180c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180cb8a0 con 0x7f9e180cb410 2023-10-21T16:21:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e18159db0 con 0x7f9e180b3c50 2023-10-21T16:21:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18071fe0 con 0x7f9e181599d0 2023-10-21T16:21:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e177fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e180cb410 0x7f9e180c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e177fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e180cb410 0x7f9e180c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60696/0 (socket says 172.21.15.53:60696) 2023-10-21T16:21:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e177fe640 1 -- 172.21.15.53:0/44652527 learned_addr learned my addr 172.21.15.53:0/44652527 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e16ffd640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e180cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.918+0000 7f9e0ffff640 1 --2- 172.21.15.53:0/44652527 >> v2:172.21.15.124:3300/0 conn(0x7f9e180b3c50 0x7f9e180caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.919+0000 7f9e177fe640 1 -- 172.21.15.53:0/44652527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 msgr2=0x7f9e180cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.919+0000 7f9e177fe640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e180cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.919+0000 7f9e177fe640 1 -- 172.21.15.53:0/44652527 >> v2:172.21.15.124:3300/0 conn(0x7f9e180b3c50 msgr2=0x7f9e180caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.919+0000 7f9e177fe640 1 --2- 172.21.15.53:0/44652527 >> v2:172.21.15.124:3300/0 conn(0x7f9e180b3c50 0x7f9e180caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.919+0000 7f9e177fe640 1 -- 172.21.15.53:0/44652527 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180b44c0 con 0x7f9e180cb410 2023-10-21T16:21:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.920+0000 7f9e177fe640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180cb410 0x7f9e180c8b40 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806dad0 tx=0x7f9e1002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd063cd9552157e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.920+0000 7f9e167fc640 1 -- 172.21.15.53:0/44652527 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1002ca80 con 0x7f9e180cb410 2023-10-21T16:21:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.920+0000 7f9e167fc640 1 -- 172.21.15.53:0/44652527 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1002cbe0 con 0x7f9e180cb410 2023-10-21T16:21:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.920+0000 7f9e167fc640 1 -- 172.21.15.53:0/44652527 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100343f0 con 0x7f9e180cb410 2023-10-21T16:21:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 -- 172.21.15.53:0/44652527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180cb410 msgr2=0x7f9e180c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180cb410 0x7f9e180c8b40 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806dad0 tx=0x7f9e1002bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 -- 172.21.15.53:0/44652527 shutdown_connections 2023-10-21T16:21:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180cb410 0x7f9e180c8b40 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 --2- 172.21.15.53:0/44652527 >> v2:172.21.15.124:3300/0 conn(0x7f9e180b3c50 0x7f9e180caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 --2- 172.21.15.53:0/44652527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e180cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 -- 172.21.15.53:0/44652527 >> 172.21.15.53:0/44652527 conn(0x7f9e1806c050 msgr2=0x7f9e1806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:52.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 -- 172.21.15.53:0/44652527 shutdown_connections 2023-10-21T16:21:52.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.921+0000 7f9e17fff640 1 -- 172.21.15.53:0/44652527 wait complete. 2023-10-21T16:21:52.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.923+0000 7f9e17fff640 1 Processor -- start 2023-10-21T16:21:52.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.923+0000 7f9e17fff640 1 -- start start 2023-10-21T16:21:52.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 0x7f9e1815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e181578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c7fa0 con 0x7f9e180b3c50 2023-10-21T16:21:52.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180aa350 con 0x7f9e180cb410 2023-10-21T16:21:52.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e17fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180aa070 con 0x7f9e181599d0 2023-10-21T16:21:52.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e16ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60700/0 (socket says 172.21.15.53:60700) 2023-10-21T16:21:52.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e16ffd640 1 -- 172.21.15.53:0/2684804517 learned_addr learned my addr 172.21.15.53:0/2684804517 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:52.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e0ffff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 0x7f9e1815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.925+0000 7f9e177fe640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e181578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:52.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 msgr2=0x7f9e181578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e181578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 msgr2=0x7f9e1815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:52.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 0x7f9e1815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:52.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 -- 172.21.15.53:0/2684804517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10009b30 con 0x7f9e180b3c50 2023-10-21T16:21:52.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e0ffff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 0x7f9e1815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:52.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e16ffd640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d5f0 tx=0x7f9e0800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:52.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080125f0 con 0x7f9e180b3c50 2023-10-21T16:21:52.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08012750 con 0x7f9e180b3c50 2023-10-21T16:21:52.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08012b00 con 0x7f9e180b3c50 2023-10-21T16:21:52.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.926+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e180c7950 con 0x7f9e180b3c50 2023-10-21T16:21:52.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.927+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e180a9c80 con 0x7f9e180b3c50 2023-10-21T16:21:52.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.928+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e180a9e10 con 0x7f9e180b3c50 2023-10-21T16:21:52.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.929+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f9e08012c60 con 0x7f9e180b3c50 2023-10-21T16:21:52.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.930+0000 7f9e0f7fe640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 0x7f9df806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:52.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.930+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f9e0808a0e0 con 0x7f9e180b3c50 2023-10-21T16:21:52.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.931+0000 7f9e0ffff640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 msgr2=0x7f9df806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:52.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.931+0000 7f9e0ffff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 0x7f9df806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:52.945 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.935+0000 7fca50637040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:52.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:52.937+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e08054c30 con 0x7f9e180b3c50 2023-10-21T16:21:52.984 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:52.982+0000 7f0db7ac9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:52.993 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:52.991+0000 7fca50637040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:53.038 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.035+0000 7f0db7ac9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:21:53.044 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.041+0000 7fca50637040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:53.097 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.095+0000 7f0db7ac9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:21:53.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.129+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e180c7790 con 0x7f9e180b3c50 2023-10-21T16:21:53.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.130+0000 7f9e0f7fe640 1 -- 172.21.15.53:0/2684804517 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e08058250 con 0x7f9e180b3c50 2023-10-21T16:21:53.132 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:21:53.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.132+0000 7f9e0ffff640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 msgr2=0x7f9df806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:53.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.132+0000 7f9e0ffff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 0x7f9df806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:53.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 msgr2=0x7f9df806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 0x7f9df806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 msgr2=0x7f9e180a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d5f0 tx=0x7f9e0800dab0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 shutdown_connections 2023-10-21T16:21:53.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f9df806b020 0x7f9df806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e181599d0 0x7f9e181578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e180cb410 0x7f9e1815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 --2- 172.21.15.53:0/2684804517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e180b3c50 0x7f9e180a9740 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 >> 172.21.15.53:0/2684804517 conn(0x7f9e1806c050 msgr2=0x7f9e180b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:53.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 shutdown_connections 2023-10-21T16:21:53.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.133+0000 7f9e17fff640 1 -- 172.21.15.53:0/2684804517 wait complete. 2023-10-21T16:21:53.153 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.150+0000 7f0db7ac9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:21:53.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:21:53.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.364+0000 7f472ea02640 1 Processor -- start 2023-10-21T16:21:53.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.365+0000 7f472ea02640 1 -- start start 2023-10-21T16:21:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.366+0000 7f472ea02640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f472815cf80 0x7f47280b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.366+0000 7f472ea02640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280b54a0 con 0x7f472815cba0 2023-10-21T16:21:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.366+0000 7f472ea02640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47280b7130 con 0x7f47280b4b40 2023-10-21T16:21:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.366+0000 7f472ea02640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47280b4ff0 con 0x7f472815cf80 2023-10-21T16:21:53.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f4727fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f472815cba0 0x7f47280b84a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39074/0 (socket says 172.21.15.53:39074) 2023-10-21T16:21:53.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f4727fff640 1 -- 172.21.15.53:0/541878635 learned_addr learned my addr 172.21.15.53:0/541878635 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172592013 0 0) 0x7f4718001110 con 0x7f472815cba0 2023-10-21T16:21:53.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f471ffff640 1 --2- 172.21.15.53:0/541878635 >> v2:172.21.15.124:3300/0 conn(0x7f472815cf80 0x7f47280b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4714003520 con 0x7f472815cba0 2023-10-21T16:21:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1245782562 0 0) 0x7f4714003520 con 0x7f472815cba0 2023-10-21T16:21:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 >> v2:172.21.15.124:3300/0 conn(0x7f472815cf80 msgr2=0x7f47280b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 --2- 172.21.15.53:0/541878635 >> v2:172.21.15.124:3300/0 conn(0x7f472815cf80 0x7f47280b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 >> v1:172.21.15.124:6789/0 conn(0x7f47280b4b40 legacy=0x7f47280d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.367+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47280b7410 con 0x7f472815cba0 2023-10-21T16:21:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4718001800 con 0x7f472815cba0 2023-10-21T16:21:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 >> v1:172.21.15.53:6789/0 conn(0x7f472815cba0 legacy=0x7f47280b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47140039a0 0x7f4714003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140047c0 0x7f4714004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 0x7f471400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47280b4ff0 con 0x7f47140052a0 2023-10-21T16:21:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471400c170 con 0x7f47140047c0 2023-10-21T16:21:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471400c330 con 0x7f47140039a0 2023-10-21T16:21:53.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f472cf9f640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 0x7f471400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f4727fff640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47140039a0 0x7f4714003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f471ffff640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140047c0 0x7f4714004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f472cf9f640 1 -- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47140039a0 msgr2=0x7f4714003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f472cf9f640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47140039a0 0x7f4714003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.368+0000 7f472cf9f640 1 -- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140047c0 msgr2=0x7f4714004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:53.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.369+0000 7f472cf9f640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140047c0 0x7f4714004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.369+0000 7f472cf9f640 1 -- 172.21.15.53:0/541878635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714003520 con 0x7f47140052a0 2023-10-21T16:21:53.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.369+0000 7f472cf9f640 1 -- 172.21.15.53:0/541878635 reap_dead start 2023-10-21T16:21:53.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f472cf9f640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 0x7f471400b980 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47200044b0 tx=0x7f47140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824464a3a74712ed server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:53.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720006d50 con 0x7f47140052a0 2023-10-21T16:21:53.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720007020 con 0x7f47140052a0 2023-10-21T16:21:53.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f47277fe640 1 -- 172.21.15.53:0/541878635 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200073d0 con 0x7f47140052a0 2023-10-21T16:21:53.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f472ea02640 1 -- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 msgr2=0x7f471400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.370+0000 7f472ea02640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 0x7f471400b980 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47200044b0 tx=0x7f47140057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.371+0000 7f472ea02640 1 -- 172.21.15.53:0/541878635 shutdown_connections 2023-10-21T16:21:53.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.371+0000 7f472ea02640 1 --2- 172.21.15.53:0/541878635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47140052a0 0x7f471400b980 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.371+0000 7f472ea02640 1 -- 172.21.15.53:0/541878635 >> 172.21.15.53:0/541878635 conn(0x7f47280718a0 msgr2=0x7f4728071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:53.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.371+0000 7f472ea02640 1 -- 172.21.15.53:0/541878635 shutdown_connections 2023-10-21T16:21:53.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.371+0000 7f472ea02640 1 -- 172.21.15.53:0/541878635 wait complete. 2023-10-21T16:21:53.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.372+0000 7f472ea02640 1 Processor -- start 2023-10-21T16:21:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 -- start start 2023-10-21T16:21:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 0x7f47280cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 0x7f4728159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728153c80 con 0x7f47280c9040 2023-10-21T16:21:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4728154a70 con 0x7f47140052a0 2023-10-21T16:21:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.373+0000 7f472ea02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4728153e60 con 0x7f47280c8bc0 2023-10-21T16:21:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60728/0 (socket says 172.21.15.53:60728) 2023-10-21T16:21:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 -- 172.21.15.53:0/168726009 learned_addr learned my addr 172.21.15.53:0/168726009 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f471ffff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 0x7f4728159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f4727fff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 0x7f47280cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 msgr2=0x7f4728159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 0x7f4728159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 msgr2=0x7f47280cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 0x7f47280cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f472cf9f640 1 -- 172.21.15.53:0/168726009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4728154140 con 0x7f47280c9040 2023-10-21T16:21:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f471ffff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 0x7f4728159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:53.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.374+0000 7f4727fff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 0x7f47280cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f472cf9f640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f472002bf80 tx=0x7f4720003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472003e070 con 0x7f47280c9040 2023-10-21T16:21:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720007650 con 0x7f47280c9040 2023-10-21T16:21:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200079e0 con 0x7f47280c9040 2023-10-21T16:21:53.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4728154420 con 0x7f47280c9040 2023-10-21T16:21:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.375+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47280cc130 con 0x7f47280c9040 2023-10-21T16:21:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.376+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4728154c00 con 0x7f47280c9040 2023-10-21T16:21:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.377+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f4720030420 con 0x7f47280c9040 2023-10-21T16:21:53.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.378+0000 7f47257fa640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 0x7f470806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.378+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f47200aad80 con 0x7f47280c9040 2023-10-21T16:21:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.378+0000 7f4727fff640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 msgr2=0x7f470806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:53.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.378+0000 7f4727fff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 0x7f470806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:53.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.386+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4720075850 con 0x7f47280c9040 2023-10-21T16:21:53.423 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.421+0000 7fca50637040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:53.568 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.566+0000 7f0db7ac9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:21:53.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.579+0000 7f4727fff640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 msgr2=0x7f470806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:53.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.579+0000 7f4727fff640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 0x7f470806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:53.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.584+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f472804e160 con 0x7f47280c9040 2023-10-21T16:21:53.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.585+0000 7f47257fa640 1 -- 172.21.15.53:0/168726009 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4720034070 con 0x7f47280c9040 2023-10-21T16:21:53.588 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:21:53.589 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.587+0000 7fca50637040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:53.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.587+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 msgr2=0x7f470806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:53.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.587+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 0x7f470806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 msgr2=0x7f472815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f472002bf80 tx=0x7f4720003a70 comp rx=0 tx=0).stop 2023-10-21T16:21:53.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 shutdown_connections 2023-10-21T16:21:53.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f470806b050 0x7f470806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47280c9040 0x7f472815a280 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47280c8bc0 0x7f4728159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 --2- 172.21.15.53:0/168726009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47140052a0 0x7f47280cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 >> 172.21.15.53:0/168726009 conn(0x7f47280718a0 msgr2=0x7f47280d2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:53.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 shutdown_connections 2023-10-21T16:21:53.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.588+0000 7f472ea02640 1 -- 172.21.15.53:0/168726009 wait complete. 2023-10-21T16:21:53.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T16:21:53.696 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.694+0000 7fca50637040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:53.741 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.739+0000 7f0db7ac9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:21:53.747 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.745+0000 7fca50637040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:53.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.814+0000 7f88bda7c640 1 Processor -- start 2023-10-21T16:21:53.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.814+0000 7f88bda7c640 1 -- start start 2023-10-21T16:21:53.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88bda7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88b80afbe0 0x7f88b80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88bda7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b80cb130 0x7f88b80cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88bda7c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80cbad0 con 0x7f88b806dfc0 2023-10-21T16:21:53.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88bda7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b80b2620 con 0x7f88b80afbe0 2023-10-21T16:21:53.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88bda7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80b2060 con 0x7f88b80cb130 2023-10-21T16:21:53.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88b80afbe0 0x7f88b80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88b77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f88b80afbe0 0x7f88b80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60732/0 (socket says 172.21.15.53:60732) 2023-10-21T16:21:53.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88aeffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f88b806dfc0 0x7f88b8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39082/0 (socket says 172.21.15.53:39082) 2023-10-21T16:21:53.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88b77fe640 1 -- 172.21.15.53:0/3613520069 learned_addr learned my addr 172.21.15.53:0/3613520069 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:53.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.816+0000 7f88b7fff640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b80cb130 0x7f88b80cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.817+0000 7f88b6ffd640 1 -- 172.21.15.53:0/3613520069 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53024380 0 0) 0x7f88b80cbad0 con 0x7f88b806dfc0 2023-10-21T16:21:53.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.817+0000 7f88b77fe640 1 -- 172.21.15.53:0/3613520069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b80cb130 msgr2=0x7f88b80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:53.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.817+0000 7f88b77fe640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b80cb130 0x7f88b80cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.817+0000 7f88b77fe640 1 -- 172.21.15.53:0/3613520069 >> v1:172.21.15.53:6789/0 conn(0x7f88b806dfc0 legacy=0x7f88b8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.817+0000 7f88b77fe640 1 -- 172.21.15.53:0/3613520069 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80b2340 con 0x7f88b80afbe0 2023-10-21T16:21:53.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.818+0000 7f88b77fe640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80b15b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009870 tx=0x7f88a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcddb2ae8a8ad4b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:53.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.818+0000 7f88b6ffd640 1 -- 172.21.15.53:0/3613520069 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8030840 con 0x7f88b80afbe0 2023-10-21T16:21:53.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.818+0000 7f88b6ffd640 1 -- 172.21.15.53:0/3613520069 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a80309a0 con 0x7f88b80afbe0 2023-10-21T16:21:53.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.818+0000 7f88b6ffd640 1 -- 172.21.15.53:0/3613520069 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8030d30 con 0x7f88b80afbe0 2023-10-21T16:21:53.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 -- 172.21.15.53:0/3613520069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 msgr2=0x7f88b80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80b15b0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009870 tx=0x7f88a802b980 comp rx=0 tx=0).stop 2023-10-21T16:21:53.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 -- 172.21.15.53:0/3613520069 shutdown_connections 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b80cb130 0x7f88b80cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 --2- 172.21.15.53:0/3613520069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80b15b0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 -- 172.21.15.53:0/3613520069 >> 172.21.15.53:0/3613520069 conn(0x7f88b806a5d0 msgr2=0x7f88b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 -- 172.21.15.53:0/3613520069 shutdown_connections 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.819+0000 7f88bda7c640 1 -- 172.21.15.53:0/3613520069 wait complete. 2023-10-21T16:21:53.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.821+0000 7f88bda7c640 1 Processor -- start 2023-10-21T16:21:53.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.821+0000 7f88bda7c640 1 -- start start 2023-10-21T16:21:53.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b806dfc0 0x7f88b81f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88b80cb130 0x7f88b80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80c2250 con 0x7f88b80afbe0 2023-10-21T16:21:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80c5060 con 0x7f88b80cb130 2023-10-21T16:21:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88bda7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80ce200 con 0x7f88b806dfc0 2023-10-21T16:21:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88aeffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60746/0 (socket says 172.21.15.53:60746) 2023-10-21T16:21:53.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88aeffd640 1 -- 172.21.15.53:0/217656606 learned_addr learned my addr 172.21.15.53:0/217656606 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88b77fe640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b806dfc0 0x7f88b81f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88b7fff640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88b80cb130 0x7f88b80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88aeffd640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b806dfc0 msgr2=0x7f88b81f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.822+0000 7f88aeffd640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b806dfc0 0x7f88b81f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88aeffd640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88b80cb130 msgr2=0x7f88b80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:53.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88aeffd640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88b80cb130 0x7f88b80cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:53.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88aeffd640 1 -- 172.21.15.53:0/217656606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80094d0 con 0x7f88b80afbe0 2023-10-21T16:21:53.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88aeffd640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c410 tx=0x7f88a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:53.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00074e0 con 0x7f88b80afbe0 2023-10-21T16:21:53.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0007660 con 0x7f88b80afbe0 2023-10-21T16:21:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004a20 con 0x7f88b80afbe0 2023-10-21T16:21:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b80ce4e0 con 0x7f88b80afbe0 2023-10-21T16:21:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.823+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b80c2780 con 0x7f88b80afbe0 2023-10-21T16:21:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.825+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b80c29a0 con 0x7f88b80afbe0 2023-10-21T16:21:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.826+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f88a0004b80 con 0x7f88b80afbe0 2023-10-21T16:21:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.826+0000 7f88b4ff9640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 0x7f889806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:53.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.826+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f88a0089ac0 con 0x7f88b80afbe0 2023-10-21T16:21:53.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.827+0000 7f88b77fe640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 msgr2=0x7f889806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:53.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.827+0000 7f88b77fe640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 0x7f889806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:53.834+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88a0054580 con 0x7f88b80afbe0 2023-10-21T16:21:53.855 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.853+0000 7fca50637040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:53.862 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.859+0000 7f0db7ac9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:21:53.915 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:53.913+0000 7f0db7ac9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:21:53.965 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:53.962+0000 7fca50637040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:54.030 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:54.027+0000 7f0db7ac9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:54.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.028+0000 7f88b77fe640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 msgr2=0x7f889806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:54.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.028+0000 7f88b77fe640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 0x7f889806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:54.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.039+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f88b804e160 con 0x7f88b80afbe0 2023-10-21T16:21:54.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.039+0000 7f88b4ff9640 1 -- 172.21.15.53:0/217656606 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v149) v1 ==== 80+0+474 (secure 0 0 0) 0x7f88a0057ba0 con 0x7f88b80afbe0 2023-10-21T16:21:54.042 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2276161522 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.042 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4259428584 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.043 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/227410594 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.043 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/227410594 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.043 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/907346555 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.043 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/667593812 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.043 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2229194042 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.044 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2878176284 2023-10-22T16:21:42.595479+0000 2023-10-21T16:21:54.045 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T16:21:54.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 msgr2=0x7f889806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:54.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 0x7f889806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 msgr2=0x7f88b80c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c410 tx=0x7f88a000c8d0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 shutdown_connections 2023-10-21T16:21:54.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f889806b020 0x7f889806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88b80cb130 0x7f88b80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88b80afbe0 0x7f88b80c4b20 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 --2- 172.21.15.53:0/217656606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88b806dfc0 0x7f88b81f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 >> 172.21.15.53:0/217656606 conn(0x7f88b806a5d0 msgr2=0x7f88b81595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:54.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 shutdown_connections 2023-10-21T16:21:54.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.042+0000 7f88bda7c640 1 -- 172.21.15.53:0/217656606 wait complete. 2023-10-21T16:21:54.063 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T16:21:54.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:54.063 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T16:21:54.122 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:54.119+0000 7fca50637040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:54.146 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:54.144+0000 7f0db7ac9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:21:54.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T16:21:54.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:21:54.164 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T16:21:54.172 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:54.170+0000 7fca50637040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:54.211 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:54.211 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:54.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:54.266 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:21:54.267 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:54.295 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:54.293+0000 7fca50637040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:54.315 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:54.312+0000 7f0db7ac9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:21:54.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:21:54.324 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:21:54.325 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:21:54.368 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:54.365+0000 7f0db7ac9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:21:54.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:21:54.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:54.495 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:54.493+0000 7f0db7ac9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:21:54.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.597+0000 7fab8a11d640 1 Processor -- start 2023-10-21T16:21:54.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.597+0000 7fab8a11d640 1 -- start start 2023-10-21T16:21:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab8a11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab840cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab8a11d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab840cdd00 0x7fab840b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab8a11d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab840b39f0 con 0x7fab8415e280 2023-10-21T16:21:54.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab8a11d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840cfb70 con 0x7fab840cdd00 2023-10-21T16:21:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab8a11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840ce190 con 0x7fab840ceac0 2023-10-21T16:21:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab837fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fab8415e280 0x7fab840cffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40004/0 (socket says 172.21.15.53:40004) 2023-10-21T16:21:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab837fe640 1 -- 172.21.15.53:0/1493950792 learned_addr learned my addr 172.21.15.53:0/1493950792 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.599+0000 7fab83fff640 1 --2- 172.21.15.53:0/1493950792 >> v2:172.21.15.53:3300/0 conn(0x7fab840cdd00 0x7fab840b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:54.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab7bfff640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab840cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab82ffd640 1 -- 172.21.15.53:0/1493950792 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560320900 0 0) 0x7fab840b39f0 con 0x7fab8415e280 2023-10-21T16:21:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab82ffd640 1 -- 172.21.15.53:0/1493950792 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab70003520 con 0x7fab8415e280 2023-10-21T16:21:54.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab83fff640 1 -- 172.21.15.53:0/1493950792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 msgr2=0x7fab840cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab83fff640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab840cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab83fff640 1 -- 172.21.15.53:0/1493950792 >> v1:172.21.15.53:6789/0 conn(0x7fab8415e280 legacy=0x7fab840cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.600+0000 7fab83fff640 1 -- 172.21.15.53:0/1493950792 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840ceff0 con 0x7fab840cdd00 2023-10-21T16:21:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.601+0000 7fab83fff640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab840cdd00 0x7fab840b3eb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ebfd881af739be server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:54.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.601+0000 7fab82ffd640 1 -- 172.21.15.53:0/1493950792 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c02cce0 con 0x7fab840cdd00 2023-10-21T16:21:54.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab82ffd640 1 -- 172.21.15.53:0/1493950792 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c02ce40 con 0x7fab840cdd00 2023-10-21T16:21:54.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab82ffd640 1 -- 172.21.15.53:0/1493950792 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c034640 con 0x7fab840cdd00 2023-10-21T16:21:54.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 -- 172.21.15.53:0/1493950792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab840cdd00 msgr2=0x7fab840b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab840cdd00 0x7fab840b3eb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c02bf30 comp rx=0 tx=0).stop 2023-10-21T16:21:54.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 -- 172.21.15.53:0/1493950792 shutdown_connections 2023-10-21T16:21:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab840cdd00 0x7fab840b3eb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 --2- 172.21.15.53:0/1493950792 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab840cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 -- 172.21.15.53:0/1493950792 >> 172.21.15.53:0/1493950792 conn(0x7fab84069f30 msgr2=0x7fab8406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 -- 172.21.15.53:0/1493950792 shutdown_connections 2023-10-21T16:21:54.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.602+0000 7fab8a11d640 1 -- 172.21.15.53:0/1493950792 wait complete. 2023-10-21T16:21:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.604+0000 7fab8a11d640 1 Processor -- start 2023-10-21T16:21:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.604+0000 7fab8a11d640 1 -- start start 2023-10-21T16:21:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 0x7fab840b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab841f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f3310 con 0x7fab8415e280 2023-10-21T16:21:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f2940 con 0x7fab840cdd00 2023-10-21T16:21:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab8a11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f2310 con 0x7fab840ceac0 2023-10-21T16:21:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50432/0 (socket says 172.21.15.53:50432) 2023-10-21T16:21:54.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.605+0000 7fab83fff640 1 -- 172.21.15.53:0/2573985144 learned_addr learned my addr 172.21.15.53:0/2573985144 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab837fe640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 0x7fab840b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab7bfff640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab841f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 msgr2=0x7fab841f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab841f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 msgr2=0x7fab840b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 0x7fab840b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 -- 172.21.15.53:0/2573985144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c009a70 con 0x7fab8415e280 2023-10-21T16:21:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab837fe640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 0x7fab840b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.606+0000 7fab83fff640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.607+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c007000 con 0x7fab8415e280 2023-10-21T16:21:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.607+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c03e610 con 0x7fab8415e280 2023-10-21T16:21:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.607+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c03e9c0 con 0x7fab8415e280 2023-10-21T16:21:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.607+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab840c9190 con 0x7fab8415e280 2023-10-21T16:21:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.607+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab841f25d0 con 0x7fab8415e280 2023-10-21T16:21:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.608+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab840d07f0 con 0x7fab8415e280 2023-10-21T16:21:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.610+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fab7c03b050 con 0x7fab8415e280 2023-10-21T16:21:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.611+0000 7fab80ff9640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 0x7fab6406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.611+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fab7c0ac470 con 0x7fab8415e280 2023-10-21T16:21:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.611+0000 7fab837fe640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 msgr2=0x7fab6406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.611+0000 7fab837fe640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 0x7fab6406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.615+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab7c076fe0 con 0x7fab8415e280 2023-10-21T16:21:54.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.812+0000 7fab837fe640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 msgr2=0x7fab6406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:54.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.812+0000 7fab837fe640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 0x7fab6406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:54.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.844+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab840cf860 con 0x7fab8415e280 2023-10-21T16:21:54.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.845+0000 7fab80ff9640 1 -- 172.21.15.53:0/2573985144 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fab7c009450 con 0x7fab8415e280 2023-10-21T16:21:54.848 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:54.849 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:54.849 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.848+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 msgr2=0x7fab6406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:54.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.848+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 0x7fab6406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.848+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 msgr2=0x7fab840c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.848+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c004a90 comp rx=0 tx=0).stop 2023-10-21T16:21:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 shutdown_connections 2023-10-21T16:21:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fab6406b020 0x7fab6406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab8415e280 0x7fab840c86f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab840ceac0 0x7fab841f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 --2- 172.21.15.53:0/2573985144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab840cdd00 0x7fab840b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 >> 172.21.15.53:0/2573985144 conn(0x7fab84069f30 msgr2=0x7fab8415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 shutdown_connections 2023-10-21T16:21:54.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:54.849+0000 7fab8a11d640 1 -- 172.21.15.53:0/2573985144 wait complete. 2023-10-21T16:21:54.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:55.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.071+0000 7f138bfa5640 1 Processor -- start 2023-10-21T16:21:55.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.071+0000 7f138bfa5640 1 -- start start 2023-10-21T16:21:55.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f138415ab40 0x7f13840ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13840c90b0 0x7f13840adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13840cae70 0x7f13840cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138415bf30 con 0x7f138415ab40 2023-10-21T16:21:55.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13840cb840 con 0x7f13840cae70 2023-10-21T16:21:55.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f138bfa5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138415b2b0 con 0x7f13840c90b0 2023-10-21T16:21:55.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f1389d41640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f138415ab40 0x7f13840ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f1389d41640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f138415ab40 0x7f13840ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50440/0 (socket says 172.21.15.53:50440) 2023-10-21T16:21:55.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.072+0000 7f1389d41640 1 -- 172.21.15.53:0/3312482778 learned_addr learned my addr 172.21.15.53:0/3312482778 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f138a542640 1 --2- 172.21.15.53:0/3312482778 >> v2:172.21.15.124:3300/0 conn(0x7f13840cae70 0x7f13840cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389540640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13840c90b0 0x7f13840adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389d41640 1 -- 172.21.15.53:0/3312482778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13840c90b0 msgr2=0x7f13840adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389d41640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13840c90b0 0x7f13840adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389d41640 1 -- 172.21.15.53:0/3312482778 >> v2:172.21.15.124:3300/0 conn(0x7f13840cae70 msgr2=0x7f13840cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389d41640 1 --2- 172.21.15.53:0/3312482778 >> v2:172.21.15.124:3300/0 conn(0x7f13840cae70 0x7f13840cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.073+0000 7f1389d41640 1 -- 172.21.15.53:0/3312482778 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138415b590 con 0x7f138415ab40 2023-10-21T16:21:55.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.074+0000 7f1389d41640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f138415ab40 0x7f13840ad720 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f13840c8d50 tx=0x7f137402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76a185b4f25acdf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.074+0000 7f1388d3f640 1 -- 172.21.15.53:0/3312482778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137402ca70 con 0x7f138415ab40 2023-10-21T16:21:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.074+0000 7f1388d3f640 1 -- 172.21.15.53:0/3312482778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137402cbd0 con 0x7f138415ab40 2023-10-21T16:21:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.074+0000 7f1388d3f640 1 -- 172.21.15.53:0/3312482778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13740343f0 con 0x7f138415ab40 2023-10-21T16:21:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 -- 172.21.15.53:0/3312482778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f138415ab40 msgr2=0x7f13840ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f138415ab40 0x7f13840ad720 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f13840c8d50 tx=0x7f137402bed0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 -- 172.21.15.53:0/3312482778 shutdown_connections 2023-10-21T16:21:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13840c90b0 0x7f13840adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3312482778 >> v2:172.21.15.124:3300/0 conn(0x7f13840cae70 0x7f13840cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3312482778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f138415ab40 0x7f13840ad720 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 -- 172.21.15.53:0/3312482778 >> 172.21.15.53:0/3312482778 conn(0x7f138406a380 msgr2=0x7f13840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:55.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 -- 172.21.15.53:0/3312482778 shutdown_connections 2023-10-21T16:21:55.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.075+0000 7f138bfa5640 1 -- 172.21.15.53:0/3312482778 wait complete. 2023-10-21T16:21:55.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.077+0000 7f138bfa5640 1 Processor -- start 2023-10-21T16:21:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.077+0000 7f138bfa5640 1 -- start start 2023-10-21T16:21:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 0x7f13840c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f138415ab40 0x7f13840c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138415cbb0 con 0x7f13840cae70 2023-10-21T16:21:55.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13840c5f10 con 0x7f13840c90b0 2023-10-21T16:21:55.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f138bfa5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138415cdf0 con 0x7f138415ab40 2023-10-21T16:21:55.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f1389540640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f1389540640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50448/0 (socket says 172.21.15.53:50448) 2023-10-21T16:21:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f1389540640 1 -- 172.21.15.53:0/3680287544 learned_addr learned my addr 172.21.15.53:0/3680287544 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.078+0000 7f1389d41640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 0x7f13840c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f138a542640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f138415ab40 0x7f13840c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f138415ab40 msgr2=0x7f13840c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f138415ab40 0x7f13840c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 msgr2=0x7f13840c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 0x7f13840c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 -- 172.21.15.53:0/3680287544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374009b30 con 0x7f13840cae70 2023-10-21T16:21:55.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389d41640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 0x7f13840c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:21:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f1389540640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f137800d800 tx=0x7f137800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780096a0 con 0x7f13840cae70 2023-10-21T16:21:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378009800 con 0x7f13840cae70 2023-10-21T16:21:55.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378009990 con 0x7f13840cae70 2023-10-21T16:21:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.079+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138415d0d0 con 0x7f13840cae70 2023-10-21T16:21:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.080+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138415d690 con 0x7f13840cae70 2023-10-21T16:21:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.081+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1384067490 con 0x7f13840cae70 2023-10-21T16:21:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.082+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f1378009af0 con 0x7f13840cae70 2023-10-21T16:21:55.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.083+0000 7f13827fc640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 0x7f137006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.083+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f137808a0b0 con 0x7f13840cae70 2023-10-21T16:21:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.083+0000 7f1389d41640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 msgr2=0x7f137006d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:55.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.083+0000 7f1389d41640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 0x7f137006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:55.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.088+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1378054c20 con 0x7f13840cae70 2023-10-21T16:21:55.131 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.129+0000 7fca50637040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:55.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.284+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13840cd3b0 con 0x7f13840cae70 2023-10-21T16:21:55.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.284+0000 7f1389d41640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 msgr2=0x7f137006d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:55.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.284+0000 7f1389d41640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 0x7f137006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:21:55.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.285+0000 7f13827fc640 1 -- 172.21.15.53:0/3680287544 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+17392 (secure 0 0 0) 0x7f1378058240 con 0x7f13840cae70 2023-10-21T16:21:55.288 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:55.288 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":149,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:42.595553+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2276161522":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/4259428584":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6833/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6832/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/907346555":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/667593812":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2229194042":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2878176284":"2023-10-22T16:21:42.595479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:55.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 msgr2=0x7f137006d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 0x7f137006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 msgr2=0x7f13841597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f137800d800 tx=0x7f137800dcc0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 shutdown_connections 2023-10-21T16:21:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7f137006b050 0x7f137006d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f138415ab40 0x7f13840c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f13840cae70 0x7f13841597c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 --2- 172.21.15.53:0/3680287544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13840c90b0 0x7f13840c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 >> 172.21.15.53:0/3680287544 conn(0x7f138406a380 msgr2=0x7f138406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 shutdown_connections 2023-10-21T16:21:55.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.288+0000 7f138bfa5640 1 -- 172.21.15.53:0/3680287544 wait complete. 2023-10-21T16:21:55.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:21:25.840789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:21:26.894786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:21:55.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:21:55.364 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:55.362+0000 7f0db7ac9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:21:55.424 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.422+0000 7fca50637040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:55.482 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.480+0000 7fca50637040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.524+0000 7fa282ae5640 1 Processor -- start 2023-10-21T16:21:55.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.524+0000 7fa282ae5640 1 -- start start 2023-10-21T16:21:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b45e0 0x7fa27c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa27c0b83a0 0x7fa27c0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa27c0b8800 0x7fa27c0b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0b6b40 con 0x7fa27c0b83a0 2023-10-21T16:21:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0b4f00 con 0x7fa27c0b8800 2023-10-21T16:21:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa282ae5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0b5ec0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa27bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa27c0b83a0 0x7fa27c0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa27bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa27c0b83a0 0x7fa27c0b6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50458/0 (socket says 172.21.15.53:50458) 2023-10-21T16:21:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa27bfff640 1 -- 172.21.15.53:0/3086878968 learned_addr learned my addr 172.21.15.53:0/3086878968 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa281082640 1 --2- 172.21.15.53:0/3086878968 >> v2:172.21.15.124:3300/0 conn(0x7fa27c0b8800 0x7fa27c0b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.526+0000 7fa280881640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b45e0 0x7fa27c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.527+0000 7fa27bfff640 1 -- 172.21.15.53:0/3086878968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b45e0 msgr2=0x7fa27c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.527+0000 7fa27bfff640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b45e0 0x7fa27c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.527+0000 7fa27bfff640 1 -- 172.21.15.53:0/3086878968 >> v2:172.21.15.124:3300/0 conn(0x7fa27c0b8800 msgr2=0x7fa27c0b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.527+0000 7fa27bfff640 1 --2- 172.21.15.53:0/3086878968 >> v2:172.21.15.124:3300/0 conn(0x7fa27c0b8800 0x7fa27c0b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.527+0000 7fa27bfff640 1 -- 172.21.15.53:0/3086878968 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0b61a0 con 0x7fa27c0b83a0 2023-10-21T16:21:55.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.528+0000 7fa27bfff640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b83a0 0x7fa27c0b6f60 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa27c08bd10 tx=0x7fa26802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6f68bcf8ce55b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.528+0000 7fa27b7fe640 1 -- 172.21.15.53:0/3086878968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268031870 con 0x7fa27c0b83a0 2023-10-21T16:21:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.528+0000 7fa27b7fe640 1 -- 172.21.15.53:0/3086878968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2680319d0 con 0x7fa27c0b83a0 2023-10-21T16:21:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.528+0000 7fa27b7fe640 1 -- 172.21.15.53:0/3086878968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268031d60 con 0x7fa27c0b83a0 2023-10-21T16:21:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 -- 172.21.15.53:0/3086878968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b83a0 msgr2=0x7fa27c0b6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b83a0 0x7fa27c0b6f60 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa27c08bd10 tx=0x7fa26802b910 comp rx=0 tx=0).stop 2023-10-21T16:21:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 -- 172.21.15.53:0/3086878968 shutdown_connections 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3086878968 >> v2:172.21.15.124:3300/0 conn(0x7fa27c0b8800 0x7fa27c0b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b83a0 0x7fa27c0b6f60 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3086878968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b45e0 0x7fa27c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 -- 172.21.15.53:0/3086878968 >> 172.21.15.53:0/3086878968 conn(0x7fa27c0718a0 msgr2=0x7fa27c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 -- 172.21.15.53:0/3086878968 shutdown_connections 2023-10-21T16:21:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.529+0000 7fa282ae5640 1 -- 172.21.15.53:0/3086878968 wait complete. 2023-10-21T16:21:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.530+0000 7fa282ae5640 1 Processor -- start 2023-10-21T16:21:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.531+0000 7fa282ae5640 1 -- start start 2023-10-21T16:21:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.531+0000 7fa282ae5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.531+0000 7fa282ae5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b83a0 0x7fa27c1e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa282ae5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 0x7fa27c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa282ae5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c15c690 con 0x7fa27c0b45e0 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa282ae5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c15c970 con 0x7fa27c0b8800 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa282ae5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c15cc50 con 0x7fa27c0b83a0 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50460/0 (socket says 172.21.15.53:50460) 2023-10-21T16:21:55.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 -- 172.21.15.53:0/3514697249 learned_addr learned my addr 172.21.15.53:0/3514697249 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa281082640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 0x7fa27c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 -- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b83a0 msgr2=0x7fa27c1e8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b83a0 0x7fa27c1e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 -- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 msgr2=0x7fa27c158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 0x7fa27c158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa280881640 1 -- 172.21.15.53:0/3514697249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680094d0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.532+0000 7fa281082640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 0x7fa27c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:55.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa280881640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d800 tx=0x7fa27000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2700096a0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270009800 con 0x7fa27c0b45e0 2023-10-21T16:21:55.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270009b90 con 0x7fa27c0b45e0 2023-10-21T16:21:55.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c15b1f0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.533+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c15b7b0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.534+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c04e160 con 0x7fa27c0b45e0 2023-10-21T16:21:55.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.536+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 71) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fa270009cf0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.537+0000 7fa2797fa640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 0x7fa26006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.537+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fa27008a130 con 0x7fa27c0b45e0 2023-10-21T16:21:55.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.537+0000 7fa27bfff640 1 -- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 msgr2=0x7fa26006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1829772531 2023-10-21T16:21:55.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.537+0000 7fa27bfff640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 0x7fa26006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:21:55.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.541+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa270054be0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.597 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.594+0000 7fca50637040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:55.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.613+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa270054fb0 con 0x7fa27c0b45e0 2023-10-21T16:21:55.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.614+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 msgr2=0x7fa26006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:21:55.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.614+0000 7fa2797fa640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 0x7fa26006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.692 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d18b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d18b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d18b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d18b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d18b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d50c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d50c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d50c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d50c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:21:55.690+0000 7fc9d50c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:21:55.695 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:55.692+0000 7f0db7ac9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:21:55.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.744+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa27c0b5780 con 0x7fa27c0b45e0 2023-10-21T16:21:55.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.746+0000 7fa2797fa640 1 -- 172.21.15.53:0/3514697249 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa270087370 con 0x7fa27c0b45e0 2023-10-21T16:21:55.748 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:21:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 msgr2=0x7fa27c1e50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d800 tx=0x7fa27000dcc0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 shutdown_connections 2023-10-21T16:21:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:6832/1829772531,v1:172.21.15.124:6833/1829772531] conn(0x7fa26006b020 0x7fa26006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa27c0b8800 0x7fa27c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa27c0b83a0 0x7fa27c1e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 --2- 172.21.15.53:0/3514697249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa27c0b45e0 0x7fa27c1e50e0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.748+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 >> 172.21.15.53:0/3514697249 conn(0x7fa27c0718a0 msgr2=0x7fa27c0d2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.749+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 shutdown_connections 2023-10-21T16:21:55.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.749+0000 7fa282ae5640 1 -- 172.21.15.53:0/3514697249 wait complete. 2023-10-21T16:21:55.756 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:55.754+0000 7f0db7ac9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:21:55.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T16:21:55.880 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:21:55.878+0000 7f0db7ac9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:21:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.981+0000 7fa325d7c640 1 Processor -- start 2023-10-21T16:21:55.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.981+0000 7fa325d7c640 1 -- start start 2023-10-21T16:21:55.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa32015cdb0 0x7fa3200cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32015d190 0x7fa32006af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3200ccb40 0x7fa3200ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3200cd4b0 con 0x7fa3200ccb40 2023-10-21T16:21:55.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32015d5f0 con 0x7fa32015cdb0 2023-10-21T16:21:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.982+0000 7fa325d7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200b19e0 con 0x7fa32015d190 2023-10-21T16:21:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3200ccb40 0x7fa3200ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa3200ccb40 0x7fa3200ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50476/0 (socket says 172.21.15.53:50476) 2023-10-21T16:21:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 -- 172.21.15.53:0/400812125 learned_addr learned my addr 172.21.15.53:0/400812125 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31effd640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32015d190 0x7fa32006af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 -- 172.21.15.53:0/400812125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32015d190 msgr2=0x7fa32006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32015d190 0x7fa32006af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31f7fe640 1 --2- 172.21.15.53:0/400812125 >> v2:172.21.15.124:3300/0 conn(0x7fa32015cdb0 0x7fa3200cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 -- 172.21.15.53:0/400812125 >> v2:172.21.15.124:3300/0 conn(0x7fa32015cdb0 msgr2=0x7fa3200cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:55.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 --2- 172.21.15.53:0/400812125 >> v2:172.21.15.124:3300/0 conn(0x7fa32015cdb0 0x7fa3200cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.983+0000 7fa31ffff640 1 -- 172.21.15.53:0/400812125 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320069ef0 con 0x7fa3200ccb40 2023-10-21T16:21:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.984+0000 7fa31ffff640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3200ccb40 0x7fa3200ccf70 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa32009d060 tx=0x7fa31802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1d1e60f2f7ce12 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa31e7fc640 1 -- 172.21.15.53:0/400812125 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031870 con 0x7fa3200ccb40 2023-10-21T16:21:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa31e7fc640 1 -- 172.21.15.53:0/400812125 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180319d0 con 0x7fa3200ccb40 2023-10-21T16:21:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa31e7fc640 1 -- 172.21.15.53:0/400812125 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031d60 con 0x7fa3200ccb40 2023-10-21T16:21:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 -- 172.21.15.53:0/400812125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3200ccb40 msgr2=0x7fa3200ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3200ccb40 0x7fa3200ccf70 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa32009d060 tx=0x7fa31802b910 comp rx=0 tx=0).stop 2023-10-21T16:21:55.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 -- 172.21.15.53:0/400812125 shutdown_connections 2023-10-21T16:21:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32015d190 0x7fa32006af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 --2- 172.21.15.53:0/400812125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3200ccb40 0x7fa3200ccf70 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 --2- 172.21.15.53:0/400812125 >> v2:172.21.15.124:3300/0 conn(0x7fa32015cdb0 0x7fa3200cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 -- 172.21.15.53:0/400812125 >> 172.21.15.53:0/400812125 conn(0x7fa3200b0af0 msgr2=0x7fa3200b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 -- 172.21.15.53:0/400812125 shutdown_connections 2023-10-21T16:21:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.985+0000 7fa325d7c640 1 -- 172.21.15.53:0/400812125 wait complete. 2023-10-21T16:21:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.987+0000 7fa325d7c640 1 Processor -- start 2023-10-21T16:21:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.987+0000 7fa325d7c640 1 -- start start 2023-10-21T16:21:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3200ccb40 0x7fa3201590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32015d190 0x7fa320157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200c3560 con 0x7fa32015cdb0 2023-10-21T16:21:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201e6710 con 0x7fa32015d190 2023-10-21T16:21:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa325d7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201e6960 con 0x7fa3200ccb40 2023-10-21T16:21:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa31effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa31effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50482/0 (socket says 172.21.15.53:50482) 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.988+0000 7fa31effd640 1 -- 172.21.15.53:0/107058586 learned_addr learned my addr 172.21.15.53:0/107058586 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31f7fe640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3200ccb40 0x7fa3201590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31ffff640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32015d190 0x7fa320157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 -- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3200ccb40 msgr2=0x7fa3201590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3200ccb40 0x7fa3201590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 -- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32015d190 msgr2=0x7fa320157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32015d190 0x7fa320157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 -- 172.21.15.53:0/107058586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180094d0 con 0x7fa32015cdb0 2023-10-21T16:21:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa31effd640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c7f0 tx=0x7fa30800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308004220 con 0x7fa32015cdb0 2023-10-21T16:21:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3080043a0 con 0x7fa32015cdb0 2023-10-21T16:21:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308007860 con 0x7fa32015cdb0 2023-10-21T16:21:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.989+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3201e6c40 con 0x7fa32015cdb0 2023-10-21T16:21:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.990+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3201e7200 con 0x7fa32015cdb0 2023-10-21T16:21:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.991+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3201e73f0 con 0x7fa32015cdb0 2023-10-21T16:21:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.992+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa308007bd0 con 0x7fa32015cdb0 2023-10-21T16:21:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.992+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fa3080895c0 con 0x7fa32015cdb0 2023-10-21T16:21:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:55.997+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa308054590 con 0x7fa32015cdb0 2023-10-21T16:21:56.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.180+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa32004e160 con 0x7fa32015cdb0 2023-10-21T16:21:56.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.180+0000 7fa317fff640 1 -- 172.21.15.53:0/107058586 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa30801e020 con 0x7fa32015cdb0 2023-10-21T16:21:56.183 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T16:21:56.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 msgr2=0x7fa3200c2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c7f0 tx=0x7fa30800ccb0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 shutdown_connections 2023-10-21T16:21:56.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32015d190 0x7fa320157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32015cdb0 0x7fa3200c2aa0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 --2- 172.21.15.53:0/107058586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3200ccb40 0x7fa3201590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 >> 172.21.15.53:0/107058586 conn(0x7fa3200b0af0 msgr2=0x7fa32006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:56.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 shutdown_connections 2023-10-21T16:21:56.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.183+0000 7fa325d7c640 1 -- 172.21.15.53:0/107058586 wait complete. 2023-10-21T16:21:56.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T16:21:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.428+0000 7f27e5490640 1 Processor -- start 2023-10-21T16:21:56.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.428+0000 7f27e5490640 1 -- start start 2023-10-21T16:21:56.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27e5490640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e0071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27e5490640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27e0071b80 0x7f27e0160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27e5490640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e00c9ff0 con 0x7f27e01603a0 2023-10-21T16:21:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27e5490640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e00c3100 con 0x7f27e0071b80 2023-10-21T16:21:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27e5490640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0071ff0 con 0x7f27e00ce1c0 2023-10-21T16:21:56.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27df7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27e0071b80 0x7f27e0160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27df7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f27e0071b80 0x7f27e0160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50496/0 (socket says 172.21.15.53:50496) 2023-10-21T16:21:56.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.429+0000 7f27df7fe640 1 -- 172.21.15.53:0/308278504 learned_addr learned my addr 172.21.15.53:0/308278504 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:56.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.430+0000 7f27d7fff640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e0071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.430+0000 7f27df7fe640 1 -- 172.21.15.53:0/308278504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 msgr2=0x7f27e0071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:56.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.430+0000 7f27df7fe640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e0071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.430+0000 7f27df7fe640 1 -- 172.21.15.53:0/308278504 >> v1:172.21.15.124:6789/0 conn(0x7f27e01603a0 legacy=0x7f27e00cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.430+0000 7f27df7fe640 1 -- 172.21.15.53:0/308278504 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0072290 con 0x7f27e0071b80 2023-10-21T16:21:56.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27df7fe640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0071b80 0x7f27e0160910 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e50 tx=0x7f27d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad548e1a5f376fa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27de7fc640 1 -- 172.21.15.53:0/308278504 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d802ca70 con 0x7f27e0071b80 2023-10-21T16:21:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27de7fc640 1 -- 172.21.15.53:0/308278504 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d802cbd0 con 0x7f27e0071b80 2023-10-21T16:21:56.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27de7fc640 1 -- 172.21.15.53:0/308278504 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80343f0 con 0x7f27e0071b80 2023-10-21T16:21:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 -- 172.21.15.53:0/308278504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0071b80 msgr2=0x7f27e0160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0071b80 0x7f27e0160910 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e50 tx=0x7f27d802bed0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 -- 172.21.15.53:0/308278504 shutdown_connections 2023-10-21T16:21:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e0071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 --2- 172.21.15.53:0/308278504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e0071b80 0x7f27e0160910 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 -- 172.21.15.53:0/308278504 >> 172.21.15.53:0/308278504 conn(0x7f27e00cc2e0 msgr2=0x7f27e00cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 -- 172.21.15.53:0/308278504 shutdown_connections 2023-10-21T16:21:56.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.432+0000 7f27e5490640 1 -- 172.21.15.53:0/308278504 wait complete. 2023-10-21T16:21:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.434+0000 7f27e5490640 1 Processor -- start 2023-10-21T16:21:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.434+0000 7f27e5490640 1 -- start start 2023-10-21T16:21:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27e0071b80 0x7f27e0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e0141aa0 con 0x7f27e01603a0 2023-10-21T16:21:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00add80 con 0x7f27e0071b80 2023-10-21T16:21:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27e5490640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00ae060 con 0x7f27e00ce1c0 2023-10-21T16:21:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50504/0 (socket says 172.21.15.53:50504) 2023-10-21T16:21:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 -- 172.21.15.53:0/1112082253 learned_addr learned my addr 172.21.15.53:0/1112082253 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27deffd640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27e0071b80 0x7f27e0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27d7fff640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 -- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 msgr2=0x7f27e00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e00ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 -- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27e0071b80 msgr2=0x7f27e0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27e0071b80 0x7f27e0156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 -- 172.21.15.53:0/1112082253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009b30 con 0x7f27e01603a0 2023-10-21T16:21:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.436+0000 7f27df7fe640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e50 tx=0x7f27d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.437+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009510 con 0x7f27e01603a0 2023-10-21T16:21:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.437+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d803e450 con 0x7f27e01603a0 2023-10-21T16:21:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.437+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e00ae340 con 0x7f27e01603a0 2023-10-21T16:21:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.437+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d803e800 con 0x7f27e01603a0 2023-10-21T16:21:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.437+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e01566a0 con 0x7f27e01603a0 2023-10-21T16:21:56.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.438+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e0141be0 con 0x7f27e01603a0 2023-10-21T16:21:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.440+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 72) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f27d803e960 con 0x7f27e01603a0 2023-10-21T16:21:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.440+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f27d80ac480 con 0x7f27e01603a0 2023-10-21T16:21:56.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.445+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27e0141be0 con 0x7f27e01603a0 2023-10-21T16:21:56.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.619+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f27d8076f20 con 0x7f27e01603a0 2023-10-21T16:21:56.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.620+0000 7f27d77fe640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 0x7f27c00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.623+0000 7f27deffd640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 0x7f27c00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.633+0000 7f27deffd640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 0x7f27c00730a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27d0003d90 tx=0x7f27d0009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.637+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27e004e160 con 0x7f27e01603a0 2023-10-21T16:21:56.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.638+0000 7f27d77fe640 1 -- 172.21.15.53:0/1112082253 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27d808ee60 con 0x7f27e01603a0 2023-10-21T16:21:56.643 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T16:21:56.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.640+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 msgr2=0x7f27c00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.640+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 0x7f27c00730a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27d0003d90 tx=0x7f27d0009040 comp rx=0 tx=0).stop 2023-10-21T16:21:56.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.640+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 msgr2=0x7f27e00ad770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.640+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009e50 tx=0x7f27d8004a40 comp rx=0 tx=0).stop 2023-10-21T16:21:56.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 shutdown_connections 2023-10-21T16:21:56.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f27c0070cb0 0x7f27c00730a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f27e01603a0 0x7f27e00ad770 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27e00ce1c0 0x7f27e00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 --2- 172.21.15.53:0/1112082253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27e0071b80 0x7f27e0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 >> 172.21.15.53:0/1112082253 conn(0x7f27e00cc2e0 msgr2=0x7f27e0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:56.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 shutdown_connections 2023-10-21T16:21:56.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.641+0000 7f27e5490640 1 -- 172.21.15.53:0/1112082253 wait complete. 2023-10-21T16:21:56.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.880+0000 7f8510abb640 1 Processor -- start 2023-10-21T16:21:56.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.881+0000 7f8510abb640 1 -- start start 2023-10-21T16:21:56.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f850c0d16f0 0x7f850c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f850c0cdfd0 0x7f850c0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c15d040 con 0x7f850c0cdfd0 2023-10-21T16:21:56.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0d0650 con 0x7f850c0d16f0 2023-10-21T16:21:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f8510abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0d0930 con 0x7f850c15cc60 2023-10-21T16:21:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f850bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f850c0cdfd0 0x7f850c0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f850bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f850c0cdfd0 0x7f850c0ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50510/0 (socket says 172.21.15.53:50510) 2023-10-21T16:21:56.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.882+0000 7f850bfff640 1 -- 172.21.15.53:0/3275231968 learned_addr learned my addr 172.21.15.53:0/3275231968 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f8503fff640 1 --2- 172.21.15.53:0/3275231968 >> v2:172.21.15.124:3300/0 conn(0x7f850c0d16f0 0x7f850c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f850bfff640 1 -- 172.21.15.53:0/3275231968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 msgr2=0x7f850c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f850bfff640 1 --2- 172.21.15.53:0/3275231968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f850bfff640 1 -- 172.21.15.53:0/3275231968 >> v2:172.21.15.124:3300/0 conn(0x7f850c0d16f0 msgr2=0x7f850c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f850bfff640 1 --2- 172.21.15.53:0/3275231968 >> v2:172.21.15.124:3300/0 conn(0x7f850c0d16f0 0x7f850c0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.883+0000 7f850bfff640 1 -- 172.21.15.53:0/3275231968 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0d0c10 con 0x7f850c0cdfd0 2023-10-21T16:21:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.884+0000 7f850bfff640 1 --2- 172.21.15.53:0/3275231968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c0ce460 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f850c08ceb0 tx=0x7f850402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2173b38de080d58 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.884+0000 7f850affd640 1 -- 172.21.15.53:0/3275231968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850402ca80 con 0x7f850c0cdfd0 2023-10-21T16:21:56.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.884+0000 7f850affd640 1 -- 172.21.15.53:0/3275231968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850402cbe0 con 0x7f850c0cdfd0 2023-10-21T16:21:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.884+0000 7f850affd640 1 -- 172.21.15.53:0/3275231968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040343f0 con 0x7f850c0cdfd0 2023-10-21T16:21:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 -- 172.21.15.53:0/3275231968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 msgr2=0x7f850c0ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 --2- 172.21.15.53:0/3275231968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c0ce460 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f850c08ceb0 tx=0x7f850402bee0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 -- 172.21.15.53:0/3275231968 shutdown_connections 2023-10-21T16:21:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 --2- 172.21.15.53:0/3275231968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c0ce460 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 --2- 172.21.15.53:0/3275231968 >> v2:172.21.15.124:3300/0 conn(0x7f850c0d16f0 0x7f850c0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 --2- 172.21.15.53:0/3275231968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 -- 172.21.15.53:0/3275231968 >> 172.21.15.53:0/3275231968 conn(0x7f850c09bcf0 msgr2=0x7f850c0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 -- 172.21.15.53:0/3275231968 shutdown_connections 2023-10-21T16:21:56.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.885+0000 7f8510abb640 1 -- 172.21.15.53:0/3275231968 wait complete. 2023-10-21T16:21:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.887+0000 7f8510abb640 1 Processor -- start 2023-10-21T16:21:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.887+0000 7f8510abb640 1 -- start start 2023-10-21T16:21:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 0x7f850c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1f2c50 con 0x7f850c0cdfd0 2023-10-21T16:21:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c15b2a0 con 0x7f850c0d16f0 2023-10-21T16:21:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8510abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c1f6680 con 0x7f850c15cc60 2023-10-21T16:21:56.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f850b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50522/0 (socket says 172.21.15.53:50522) 2023-10-21T16:21:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f850b7fe640 1 -- 172.21.15.53:0/2908611136 learned_addr learned my addr 172.21.15.53:0/2908611136 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.888+0000 7f8503fff640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 0x7f850c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 -- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 msgr2=0x7f850c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 -- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 msgr2=0x7f850c0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 0x7f850c0cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 -- 172.21.15.53:0/2908611136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009b30 con 0x7f850c0cdfd0 2023-10-21T16:21:56.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8503fff640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 0x7f850c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f850b7fe640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00b4a0 tx=0x7f84fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc005a10 con 0x7f850c0cdfd0 2023-10-21T16:21:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc005b70 con 0x7f850c0cdfd0 2023-10-21T16:21:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc010610 con 0x7f850c0cdfd0 2023-10-21T16:21:56.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c1f2590 con 0x7f850c0cdfd0 2023-10-21T16:21:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.889+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1f2280 con 0x7f850c0cdfd0 2023-10-21T16:21:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.890+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c04e160 con 0x7f850c0cdfd0 2023-10-21T16:21:56.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.891+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f84fc010770 con 0x7f850c0cdfd0 2023-10-21T16:21:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.892+0000 7f8508ff9640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 0x7f84ec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.892+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f84fc0890a0 con 0x7f850c0cdfd0 2023-10-21T16:21:56.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.893+0000 7f8503fff640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 0x7f84ec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.894+0000 7f8503fff640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 0x7f84ec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:56.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:56.897+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84fc053f00 con 0x7f850c0cdfd0 2023-10-21T16:21:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.118+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f850c0d3080 con 0x7f850c0cdfd0 2023-10-21T16:21:57.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.119+0000 7f8508ff9640 1 -- 172.21.15.53:0/2908611136 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f84fc057520 con 0x7f850c0cdfd0 2023-10-21T16:21:57.121 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:57.121 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:57.121 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.121+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 msgr2=0x7f84ec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.121+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 0x7f84ec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4008040 comp rx=0 tx=0).stop 2023-10-21T16:21:57.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.121+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 msgr2=0x7f850c15ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.121+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00b4a0 tx=0x7f84fc00b960 comp rx=0 tx=0).stop 2023-10-21T16:21:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 shutdown_connections 2023-10-21T16:21:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f84ec06acc0 0x7f84ec06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f850c15cc60 0x7f850c0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f850c0d16f0 0x7f850c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 --2- 172.21.15.53:0/2908611136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f850c0cdfd0 0x7f850c15ad60 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 >> 172.21.15.53:0/2908611136 conn(0x7f850c09bcf0 msgr2=0x7f850c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 shutdown_connections 2023-10-21T16:21:57.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.122+0000 7f8510abb640 1 -- 172.21.15.53:0/2908611136 wait complete. 2023-10-21T16:21:57.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.365+0000 7fdd9a79e640 1 Processor -- start 2023-10-21T16:21:57.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.365+0000 7fdd9a79e640 1 -- start start 2023-10-21T16:21:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd9a79e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd940add60 0x7fdd94162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd9a79e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdd9406d4c0 0x7fdd940cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd9a79e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940ae140 con 0x7fdd940cb150 2023-10-21T16:21:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd9a79e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd94161f10 con 0x7fdd940add60 2023-10-21T16:21:57.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd9a79e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd94161630 con 0x7fdd9406d4c0 2023-10-21T16:21:57.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd93fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd940add60 0x7fdd94162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd93fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd940add60 0x7fdd94162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50536/0 (socket says 172.21.15.53:50536) 2023-10-21T16:21:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd93fff640 1 -- 172.21.15.53:0/2875392245 learned_addr learned my addr 172.21.15.53:0/2875392245 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.367+0000 7fdd8b7fe640 1 --2- 172.21.15.53:0/2875392245 >> v2:172.21.15.124:3300/0 conn(0x7fdd9406d4c0 0x7fdd940cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.368+0000 7fdd93fff640 1 -- 172.21.15.53:0/2875392245 >> v2:172.21.15.124:3300/0 conn(0x7fdd9406d4c0 msgr2=0x7fdd940cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:57.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.368+0000 7fdd93fff640 1 --2- 172.21.15.53:0/2875392245 >> v2:172.21.15.124:3300/0 conn(0x7fdd9406d4c0 0x7fdd940cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.368+0000 7fdd93fff640 1 -- 172.21.15.53:0/2875392245 >> v1:172.21.15.53:6789/0 conn(0x7fdd940cb150 legacy=0x7fdd94164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.368+0000 7fdd93fff640 1 -- 172.21.15.53:0/2875392245 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940cd1c0 con 0x7fdd940add60 2023-10-21T16:21:57.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.369+0000 7fdd93fff640 1 --2- 172.21.15.53:0/2875392245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940add60 0x7fdd94162330 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdd9406e560 tx=0x7fdd8402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb9055b3957f34 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.369+0000 7fdd937fe640 1 -- 172.21.15.53:0/2875392245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84031860 con 0x7fdd940add60 2023-10-21T16:21:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.369+0000 7fdd937fe640 1 -- 172.21.15.53:0/2875392245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd840319c0 con 0x7fdd940add60 2023-10-21T16:21:57.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.369+0000 7fdd937fe640 1 -- 172.21.15.53:0/2875392245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84031d50 con 0x7fdd940add60 2023-10-21T16:21:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 -- 172.21.15.53:0/2875392245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940add60 msgr2=0x7fdd94162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/2875392245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940add60 0x7fdd94162330 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdd9406e560 tx=0x7fdd8402b900 comp rx=0 tx=0).stop 2023-10-21T16:21:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 -- 172.21.15.53:0/2875392245 shutdown_connections 2023-10-21T16:21:57.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/2875392245 >> v2:172.21.15.124:3300/0 conn(0x7fdd9406d4c0 0x7fdd940cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/2875392245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940add60 0x7fdd94162330 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 -- 172.21.15.53:0/2875392245 >> 172.21.15.53:0/2875392245 conn(0x7fdd9406a380 msgr2=0x7fdd940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:57.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 -- 172.21.15.53:0/2875392245 shutdown_connections 2023-10-21T16:21:57.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.370+0000 7fdd9a79e640 1 -- 172.21.15.53:0/2875392245 wait complete. 2023-10-21T16:21:57.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.372+0000 7fdd9a79e640 1 Processor -- start 2023-10-21T16:21:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.372+0000 7fdd9a79e640 1 -- start start 2023-10-21T16:21:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd9406d4c0 0x7fdd94156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 0x7fdd940af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94156c80 con 0x7fdd940cb150 2023-10-21T16:21:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940af040 con 0x7fdd9406d4c0 2023-10-21T16:21:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd9a79e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940ae430 con 0x7fdd940add60 2023-10-21T16:21:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50540/0 (socket says 172.21.15.53:50540) 2023-10-21T16:21:57.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 -- 172.21.15.53:0/3025088015 learned_addr learned my addr 172.21.15.53:0/3025088015 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd8b7fe640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 0x7fdd940af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd93fff640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd9406d4c0 0x7fdd94156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 -- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 msgr2=0x7fdd940af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 0x7fdd940af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 -- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd9406d4c0 msgr2=0x7fdd94156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.373+0000 7fdd98d3b640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd9406d4c0 0x7fdd94156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd98d3b640 1 -- 172.21.15.53:0/3025088015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840094d0 con 0x7fdd940cb150 2023-10-21T16:21:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd8b7fe640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 0x7fdd940af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd98d3b640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c6b0 tx=0x7fdd8c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c003b70 con 0x7fdd940cb150 2023-10-21T16:21:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c003cf0 con 0x7fdd940cb150 2023-10-21T16:21:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940ae710 con 0x7fdd940cb150 2023-10-21T16:21:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c003ea0 con 0x7fdd940cb150 2023-10-21T16:21:57.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.374+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd940aeaa0 con 0x7fdd940cb150 2023-10-21T16:21:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.376+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94067490 con 0x7fdd940cb150 2023-10-21T16:21:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.377+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 73) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fdd8c005310 con 0x7fdd940cb150 2023-10-21T16:21:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.378+0000 7fdd917fa640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 0x7fdd7406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.378+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fdd8c089430 con 0x7fdd940cb150 2023-10-21T16:21:57.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.378+0000 7fdd93fff640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 0x7fdd7406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.380+0000 7fdd93fff640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 0x7fdd7406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdd84036000 tx=0x7fdd8403c040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.382+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd8c0543c0 con 0x7fdd940cb150 2023-10-21T16:21:57.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.578+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd940b0ef0 con 0x7fdd940cb150 2023-10-21T16:21:57.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.580+0000 7fdd917fa640 1 -- 172.21.15.53:0/3025088015 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17965 (secure 0 0 0) 0x7fdd8c0579e0 con 0x7fdd940cb150 2023-10-21T16:21:57.582 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:57.582 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":150,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:55.600274+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3267096500":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1834008746":"2023-10-22T16:21:55.600213+0000","172.21.15.124:6832/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1781939354":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1295722096":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2527601840":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3356731651":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2276161522":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/4259428584":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6833/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6832/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/907346555":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/3504147713":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/667593812":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2229194042":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2878176284":"2023-10-22T16:21:42.595479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 msgr2=0x7fdd7406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 0x7fdd7406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdd84036000 tx=0x7fdd8403c040 comp rx=0 tx=0).stop 2023-10-21T16:21:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 msgr2=0x7fdd94148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c6b0 tx=0x7fdd8c00cb70 comp rx=0 tx=0).stop 2023-10-21T16:21:57.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 shutdown_connections 2023-10-21T16:21:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd7406acc0 0x7fdd7406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd940cb150 0x7fdd94148860 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd940add60 0x7fdd940af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 --2- 172.21.15.53:0/3025088015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd9406d4c0 0x7fdd94156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 >> 172.21.15.53:0/3025088015 conn(0x7fdd9406a380 msgr2=0x7fdd9406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:57.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 shutdown_connections 2023-10-21T16:21:57.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.583+0000 7fdd9a79e640 1 -- 172.21.15.53:0/3025088015 wait complete. 2023-10-21T16:21:57.606 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T16:21:57.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:57.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.813+0000 7f369d661640 1 Processor -- start 2023-10-21T16:21:57.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.813+0000 7f369d661640 1 -- start start 2023-10-21T16:21:57.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.814+0000 7f369d661640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36981680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f369d661640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36980c48f0 con 0x7f3698160940 2023-10-21T16:21:57.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f369d661640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36980710f0 con 0x7f369815cd90 2023-10-21T16:21:57.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f369d661640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980c4d80 con 0x7f3698165cc0 2023-10-21T16:21:57.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f368ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3698160940 0x7f36980709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40024/0 (socket says 172.21.15.53:40024) 2023-10-21T16:21:57.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f368ffff640 1 -- 172.21.15.53:0/2194854441 learned_addr learned my addr 172.21.15.53:0/2194854441 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:57.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497785557 0 0) 0x7f36980c48f0 con 0x7f3698160940 2023-10-21T16:21:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3684003520 con 0x7f3698160940 2023-10-21T16:21:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.815+0000 7f36977fe640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36981680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338726833 0 0) 0x7f3684003520 con 0x7f3698160940 2023-10-21T16:21:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 msgr2=0x7f36981680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:57.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36981680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 >> v1:172.21.15.124:6789/0 conn(0x7f369815cd90 legacy=0x7f36980702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980c3ce0 con 0x7f3698160940 2023-10-21T16:21:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3680001670 con 0x7f3698160940 2023-10-21T16:21:57.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 >> v1:172.21.15.53:6789/0 conn(0x7f3698160940 legacy=0x7f36980709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36840039a0 0x7f3684003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3684005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368400bb60 con 0x7f3684004d00 2023-10-21T16:21:57.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368400bcd0 con 0x7f36840039a0 2023-10-21T16:21:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368400be90 con 0x7f3698165cc0 2023-10-21T16:21:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.816+0000 7f36977fe640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3684005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f3696ffd640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36840039a0 0x7f3684003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f368ffff640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f36977fe640 1 -- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 msgr2=0x7f36840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f36977fe640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3698165cc0 0x7f36840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f36977fe640 1 -- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36840039a0 msgr2=0x7f3684003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f36977fe640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36840039a0 0x7f3684003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f36977fe640 1 -- 172.21.15.53:0/2194854441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3684003520 con 0x7f3684004d00 2023-10-21T16:21:57.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.817+0000 7f3696ffd640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f36840039a0 0x7f3684003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:57.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f36977fe640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3684005180 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3690007360 tx=0x7f36840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b26e4cf408a489 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369002ccd0 con 0x7f3684004d00 2023-10-21T16:21:57.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369002ce30 con 0x7f3684004d00 2023-10-21T16:21:57.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f36967fc640 1 -- 172.21.15.53:0/2194854441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690034660 con 0x7f3684004d00 2023-10-21T16:21:57.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f369d661640 1 -- 172.21.15.53:0/2194854441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 msgr2=0x7f3684005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f369d661640 1 --2- 172.21.15.53:0/2194854441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3684005180 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f3690007360 tx=0x7f36840057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.818+0000 7f36977fe640 1 -- 172.21.15.53:0/2194854441 reap_dead start 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.819+0000 7f369d661640 1 -- 172.21.15.53:0/2194854441 shutdown_connections 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.819+0000 7f369d661640 1 -- 172.21.15.53:0/2194854441 >> 172.21.15.53:0/2194854441 conn(0x7f369806a380 msgr2=0x7f36980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.819+0000 7f369d661640 1 -- 172.21.15.53:0/2194854441 shutdown_connections 2023-10-21T16:21:57.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.819+0000 7f369d661640 1 -- 172.21.15.53:0/2194854441 wait complete. 2023-10-21T16:21:57.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.820+0000 7f369d661640 1 Processor -- start 2023-10-21T16:21:57.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.821+0000 7f369d661640 1 -- start start 2023-10-21T16:21:57.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.821+0000 7f369d661640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.821+0000 7f369d661640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f369815bdf0 0x7f369815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.821+0000 7f369d661640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36980c5850 0x7f3698156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.821+0000 7f369d661640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980b2a40 con 0x7f3684004d00 2023-10-21T16:21:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f369d661640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980b2ce0 con 0x7f369815bdf0 2023-10-21T16:21:57.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f369d661640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980b2fc0 con 0x7f36980c5850 2023-10-21T16:21:57.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50564/0 (socket says 172.21.15.53:50564) 2023-10-21T16:21:57.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 -- 172.21.15.53:0/3760986908 learned_addr learned my addr 172.21.15.53:0/3760986908 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:57.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 -- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36980c5850 msgr2=0x7f3698156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36980c5850 0x7f3698156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f368ffff640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f369815bdf0 0x7f369815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 -- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f369815bdf0 msgr2=0x7f369815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f369815bdf0 0x7f369815b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.822+0000 7f3696ffd640 1 -- 172.21.15.53:0/3760986908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980b32a0 con 0x7f3684004d00 2023-10-21T16:21:57.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f3696ffd640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f368800d6f0 tx=0x7f368800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3688005270 con 0x7f3684004d00 2023-10-21T16:21:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36880053d0 con 0x7f3684004d00 2023-10-21T16:21:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800b8f0 con 0x7f3684004d00 2023-10-21T16:21:57.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36980b3580 con 0x7f3684004d00 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.823+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36980c5cd0 con 0x7f3684004d00 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.824+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3698067490 con 0x7f3684004d00 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.825+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f368800ba50 con 0x7f3684004d00 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.826+0000 7f368f7fe640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.826+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f3688089c20 con 0x7f3684004d00 2023-10-21T16:21:57.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.827+0000 7f368ffff640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.828+0000 7f368ffff640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 0x7f367806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3680001200 tx=0x7f3680009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:57.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:57.830+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3688054690 con 0x7f3684004d00 2023-10-21T16:21:58.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.058+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36980b0ef0 con 0x7f3684004d00 2023-10-21T16:21:58.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.060+0000 7f368f7fe640 1 -- 172.21.15.53:0/3760986908 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f3688057cb0 con 0x7f3684004d00 2023-10-21T16:21:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:58.062 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:58.062 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:58.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 msgr2=0x7f367806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 0x7f367806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3680001200 tx=0x7f3680009040 comp rx=0 tx=0).stop 2023-10-21T16:21:58.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 msgr2=0x7f3698152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f368800d6f0 tx=0x7f368800dbb0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 shutdown_connections 2023-10-21T16:21:58.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f367806af00 0x7f367806d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36980c5850 0x7f3698156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f369815bdf0 0x7f369815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 --2- 172.21.15.53:0/3760986908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3684004d00 0x7f3698152470 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 >> 172.21.15.53:0/3760986908 conn(0x7f369806a380 msgr2=0x7f369806cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:58.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 shutdown_connections 2023-10-21T16:21:58.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.063+0000 7f369d661640 1 -- 172.21.15.53:0/3760986908 wait complete. 2023-10-21T16:21:58.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:21:58.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.300+0000 7fa56f59e640 1 Processor -- start 2023-10-21T16:21:58.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.301+0000 7fa56f59e640 1 -- start start 2023-10-21T16:21:58.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.302+0000 7fa56f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa5700adc70 0x7fa5700b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.302+0000 7fa56f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57015f7d0 con 0x7fa57015af10 2023-10-21T16:21:58.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.302+0000 7fa56f59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57015efb0 con 0x7fa57015ab30 2023-10-21T16:21:58.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.302+0000 7fa56f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57015f210 con 0x7fa5700adc70 2023-10-21T16:21:58.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.303+0000 7fa56ed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa57015af10 0x7fa57006cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40038/0 (socket says 172.21.15.53:40038) 2023-10-21T16:21:58.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.303+0000 7fa56ed9d640 1 -- 172.21.15.53:0/1854369332 learned_addr learned my addr 172.21.15.53:0/1854369332 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:21:58.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.303+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/1854369332 >> v2:172.21.15.124:3300/0 conn(0x7fa5700adc70 0x7fa5700b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.303+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166375146 0 0) 0x7fa57015f7d0 con 0x7fa57015af10 2023-10-21T16:21:58.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.303+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa558003520 con 0x7fa57015af10 2023-10-21T16:21:58.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690843691 0 0) 0x7fa558003520 con 0x7fa57015af10 2023-10-21T16:21:58.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 >> v2:172.21.15.124:3300/0 conn(0x7fa5700adc70 msgr2=0x7fa5700b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:58.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 --2- 172.21.15.53:0/1854369332 >> v2:172.21.15.124:3300/0 conn(0x7fa5700adc70 0x7fa5700b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 >> v1:172.21.15.124:6789/0 conn(0x7fa57015ab30 legacy=0x7fa57015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57015f4f0 con 0x7fa57015af10 2023-10-21T16:21:58.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa568001690 con 0x7fa57015af10 2023-10-21T16:21:58.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 >> v1:172.21.15.53:6789/0 conn(0x7fa57015af10 legacy=0x7fa57006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa558003970 0x7fa558003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 0x7fa558004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa558005180 0x7fa55800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015f210 con 0x7fa5580047c0 2023-10-21T16:21:58.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800c020 con 0x7fa558005180 2023-10-21T16:21:58.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800c1e0 con 0x7fa558003970 2023-10-21T16:21:58.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.304+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 0x7fa558004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56e59c640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa558003970 0x7fa558003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56ed9d640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa558005180 0x7fa55800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56dd9b640 1 -- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa558003970 msgr2=0x7fa558003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:58.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa558003970 0x7fa558003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56dd9b640 1 -- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa558005180 msgr2=0x7fa55800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa558005180 0x7fa55800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56dd9b640 1 -- 172.21.15.53:0/1854369332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558003520 con 0x7fa5580047c0 2023-10-21T16:21:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56ed9d640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa558005180 0x7fa55800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:58.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.305+0000 7fa56ed9d640 1 -- 172.21.15.53:0/1854369332 reap_dead start 2023-10-21T16:21:58.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.306+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 0x7fa558004c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ac0 tx=0x7fa5580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9093c13b6699ad8b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.306+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56402c6b0 con 0x7fa5580047c0 2023-10-21T16:21:58.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.306+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56402c810 con 0x7fa5580047c0 2023-10-21T16:21:58.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.306+0000 7fa56d59a640 1 -- 172.21.15.53:0/1854369332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56402cba0 con 0x7fa5580047c0 2023-10-21T16:21:58.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 -- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 msgr2=0x7fa558004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 0x7fa558004c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ac0 tx=0x7fa5580057f0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 -- 172.21.15.53:0/1854369332 shutdown_connections 2023-10-21T16:21:58.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 --2- 172.21.15.53:0/1854369332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5580047c0 0x7fa558004c40 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 -- 172.21.15.53:0/1854369332 >> 172.21.15.53:0/1854369332 conn(0x7fa5700cb0c0 msgr2=0x7fa5700cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:58.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 -- 172.21.15.53:0/1854369332 shutdown_connections 2023-10-21T16:21:58.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.307+0000 7fa56f59e640 1 -- 172.21.15.53:0/1854369332 wait complete. 2023-10-21T16:21:58.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.309+0000 7fa56f59e640 1 Processor -- start 2023-10-21T16:21:58.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.309+0000 7fa56f59e640 1 -- start start 2023-10-21T16:21:58.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5580047c0 0x7fa570155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 0x7fa57015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700ad660 con 0x7fa570155660 2023-10-21T16:21:58.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700c7870 con 0x7fa5580047c0 2023-10-21T16:21:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa570155480 con 0x7fa57015d680 2023-10-21T16:21:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50586/0 (socket says 172.21.15.53:50586) 2023-10-21T16:21:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.310+0000 7fa56dd9b640 1 -- 172.21.15.53:0/3053977266 learned_addr learned my addr 172.21.15.53:0/3053977266 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:58.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56e59c640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5580047c0 0x7fa570155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56ed9d640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 0x7fa57015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 -- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 msgr2=0x7fa57015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 0x7fa57015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 -- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5580047c0 msgr2=0x7fa570155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5580047c0 0x7fa570155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 -- 172.21.15.53:0/3053977266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700c72b0 con 0x7fa570155660 2023-10-21T16:21:58.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56ed9d640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 0x7fa57015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:58.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.311+0000 7fa56dd9b640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa56402c250 tx=0x7fa564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.312+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564003810 con 0x7fa570155660 2023-10-21T16:21:58.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.312+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564003970 con 0x7fa570155660 2023-10-21T16:21:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.312+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564003d00 con 0x7fa570155660 2023-10-21T16:21:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.312+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700c7590 con 0x7fa570155660 2023-10-21T16:21:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.312+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5700af4b0 con 0x7fa570155660 2023-10-21T16:21:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.313+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5700af640 con 0x7fa570155660 2023-10-21T16:21:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.314+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa564034440 con 0x7fa570155660 2023-10-21T16:21:58.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.315+0000 7fa55effd640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 0x7fa55006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.315+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fa5640abdf0 con 0x7fa570155660 2023-10-21T16:21:58.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.316+0000 7fa56e59c640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 0x7fa55006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.317+0000 7fa56e59c640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 0x7fa55006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa560007fd0 tx=0x7fa560009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.319+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa564076ab0 con 0x7fa570155660 2023-10-21T16:21:58.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.557+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa57004e160 con 0x7fa570155660 2023-10-21T16:21:58.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.558+0000 7fa55effd640 1 -- 172.21.15.53:0/3053977266 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fa564003e60 con 0x7fa570155660 2023-10-21T16:21:58.560 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:58.560 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:21:27.892568+0000","modified":"2023-10-21T16:21:30.971234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36473":{"gid":36473,"name":"b","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/582349146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":582349146},{"type":"v1","addr":"172.21.15.124:6837","nonce":582349146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-21T16:21:58.561 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 69 2023-10-21T16:21:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.561+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 msgr2=0x7fa55006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.561+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 0x7fa55006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa560007fd0 tx=0x7fa560009040 comp rx=0 tx=0).stop 2023-10-21T16:21:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.561+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 msgr2=0x7fa570154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.561+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa56402c250 tx=0x7fa564004730 comp rx=0 tx=0).stop 2023-10-21T16:21:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 shutdown_connections 2023-10-21T16:21:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa55006af00 0x7fa55006d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa57015d680 0x7fa57015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa570155660 0x7fa570154f40 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 --2- 172.21.15.53:0/3053977266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5580047c0 0x7fa570155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 >> 172.21.15.53:0/3053977266 conn(0x7fa5700cb0c0 msgr2=0x7fa5700cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:58.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 shutdown_connections 2023-10-21T16:21:58.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.562+0000 7fa56f59e640 1 -- 172.21.15.53:0/3053977266 wait complete. 2023-10-21T16:21:58.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:21:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.809+0000 7fdd8759e640 1 Processor -- start 2023-10-21T16:21:58.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.810+0000 7fdd8759e640 1 -- start start 2023-10-21T16:21:58.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.811+0000 7fdd8759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd8806b680 0x7fdd8815d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.811+0000 7fdd8759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd880ce2c0 0x7fdd8815da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.811+0000 7fdd8759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880cde30 con 0x7fdd88157480 2023-10-21T16:21:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.811+0000 7fdd8759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd88157860 con 0x7fdd8806b680 2023-10-21T16:21:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.811+0000 7fdd8759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8815e2c0 con 0x7fdd880ce2c0 2023-10-21T16:21:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd8806b680 0x7fdd8815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd8806b680 0x7fdd8815d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50590/0 (socket says 172.21.15.53:50590) 2023-10-21T16:21:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 -- 172.21.15.53:0/4168688319 learned_addr learned my addr 172.21.15.53:0/4168688319 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd86d9d640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd880ce2c0 0x7fdd8815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 -- 172.21.15.53:0/4168688319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd880ce2c0 msgr2=0x7fdd8815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd880ce2c0 0x7fdd8815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.812+0000 7fdd85d9b640 1 -- 172.21.15.53:0/4168688319 >> v1:172.21.15.124:6789/0 conn(0x7fdd88157480 legacy=0x7fdd8806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.813+0000 7fdd85d9b640 1 -- 172.21.15.53:0/4168688319 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880cd500 con 0x7fdd8806b680 2023-10-21T16:21:58.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.814+0000 7fdd85d9b640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd8815d550 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e67af8f58d0956a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.814+0000 7fdd8559a640 1 -- 172.21.15.53:0/4168688319 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70031640 con 0x7fdd8806b680 2023-10-21T16:21:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.814+0000 7fdd8559a640 1 -- 172.21.15.53:0/4168688319 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd700317a0 con 0x7fdd8806b680 2023-10-21T16:21:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.814+0000 7fdd8559a640 1 -- 172.21.15.53:0/4168688319 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70031b30 con 0x7fdd8806b680 2023-10-21T16:21:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 -- 172.21.15.53:0/4168688319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 msgr2=0x7fdd8815d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd8815d550 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002b8f0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 -- 172.21.15.53:0/4168688319 shutdown_connections 2023-10-21T16:21:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd880ce2c0 0x7fdd8815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 --2- 172.21.15.53:0/4168688319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd8815d550 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 -- 172.21.15.53:0/4168688319 >> 172.21.15.53:0/4168688319 conn(0x7fdd880d2ae0 msgr2=0x7fdd880d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 -- 172.21.15.53:0/4168688319 shutdown_connections 2023-10-21T16:21:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.815+0000 7fdd8759e640 1 -- 172.21.15.53:0/4168688319 wait complete. 2023-10-21T16:21:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.817+0000 7fdd8759e640 1 Processor -- start 2023-10-21T16:21:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.817+0000 7fdd8759e640 1 -- start start 2023-10-21T16:21:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd880ce2c0 0x7fdd881f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd88157480 0x7fdd880b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8815c8f0 con 0x7fdd8806b680 2023-10-21T16:21:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880cb2a0 con 0x7fdd880ce2c0 2023-10-21T16:21:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880cb580 con 0x7fdd88157480 2023-10-21T16:21:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.818+0000 7fdd8659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50600/0 (socket says 172.21.15.53:50600) 2023-10-21T16:21:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 -- 172.21.15.53:0/829455186 learned_addr learned my addr 172.21.15.53:0/829455186 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd85d9b640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd880ce2c0 0x7fdd881f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd86d9d640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd88157480 0x7fdd880b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 -- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd88157480 msgr2=0x7fdd880b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd88157480 0x7fdd880b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 -- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd880ce2c0 msgr2=0x7fdd881f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd880ce2c0 0x7fdd881f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 -- 172.21.15.53:0/829455186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700094d0 con 0x7fdd8806b680 2023-10-21T16:21:58.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.819+0000 7fdd8659c640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c870 tx=0x7fdd7800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.820+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78003b70 con 0x7fdd8806b680 2023-10-21T16:21:58.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.820+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78003cf0 con 0x7fdd8806b680 2023-10-21T16:21:58.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.820+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800aa90 con 0x7fdd8806b680 2023-10-21T16:21:58.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.820+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd880cb860 con 0x7fdd8806b680 2023-10-21T16:21:58.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.820+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd880cbe20 con 0x7fdd8806b680 2023-10-21T16:21:58.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.821+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8804e160 con 0x7fdd8806b680 2023-10-21T16:21:58.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.822+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdd7800abf0 con 0x7fdd8806b680 2023-10-21T16:21:58.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.823+0000 7fdd7effd640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 0x7fdd6806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:58.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.824+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fdd78089a30 con 0x7fdd8806b680 2023-10-21T16:21:58.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.824+0000 7fdd85d9b640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 0x7fdd6806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:58.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.825+0000 7fdd85d9b640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 0x7fdd6806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd70036000 tx=0x7fdd7003c040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:58.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:58.828+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd780545b0 con 0x7fdd8806b680 2023-10-21T16:21:59.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.021+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd880d21e0 con 0x7fdd8806b680 2023-10-21T16:21:59.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.023+0000 7fdd7effd640 1 -- 172.21.15.53:0/829455186 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17965 (secure 0 0 0) 0x7fdd78003e60 con 0x7fdd8806b680 2023-10-21T16:21:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:21:59.026 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":150,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:21:55.600274+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:21:25.840789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-21T16:21:26.894786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3267096500":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1834008746":"2023-10-22T16:21:55.600213+0000","172.21.15.124:6832/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1781939354":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1295722096":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2527601840":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3356731651":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2276161522":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/4259428584":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6833/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6832/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/907346555":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/3504147713":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/667593812":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2229194042":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2878176284":"2023-10-22T16:21:42.595479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:21:59.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 msgr2=0x7fdd6806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 0x7fdd6806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd70036000 tx=0x7fdd7003c040 comp rx=0 tx=0).stop 2023-10-21T16:21:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 msgr2=0x7fdd88152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c870 tx=0x7fdd7800cd30 comp rx=0 tx=0).stop 2023-10-21T16:21:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 shutdown_connections 2023-10-21T16:21:59.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdd6806af00 0x7fdd6806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd88157480 0x7fdd880b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd880ce2c0 0x7fdd881f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 --2- 172.21.15.53:0/829455186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd8806b680 0x7fdd88152f80 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 >> 172.21.15.53:0/829455186 conn(0x7fdd880d2ae0 msgr2=0x7fdd880b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:59.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 shutdown_connections 2023-10-21T16:21:59.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.027+0000 7fdd8759e640 1 -- 172.21.15.53:0/829455186 wait complete. 2023-10-21T16:21:59.049 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T16:21:59.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.273+0000 7fcdfb59e640 1 Processor -- start 2023-10-21T16:21:59.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.273+0000 7fcdfb59e640 1 -- start start 2023-10-21T16:21:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0cf690 0x7fcdfc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfb59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc0cd520 con 0x7fcdfc0d1100 2023-10-21T16:21:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc0d1910 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc0ce080 con 0x7fcdfc0cf690 2023-10-21T16:21:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.274+0000 7fcdfa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50602/0 (socket says 172.21.15.53:50602) 2023-10-21T16:21:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 -- 172.21.15.53:0/1558540194 learned_addr learned my addr 172.21.15.53:0/1558540194 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdf9d9b640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0cf690 0x7fcdfc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 -- 172.21.15.53:0/1558540194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0cf690 msgr2=0x7fcdfc0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0cf690 0x7fcdfc0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 -- 172.21.15.53:0/1558540194 >> v1:172.21.15.124:6789/0 conn(0x7fcdfc0d1100 legacy=0x7fcdfc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.275+0000 7fcdfa59c640 1 -- 172.21.15.53:0/1558540194 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0cd750 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.276+0000 7fcdfa59c640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009e50 tx=0x7fcdec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff93be74fb2ec51d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.276+0000 7fcdf959a640 1 -- 172.21.15.53:0/1558540194 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec02c9e0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdf959a640 1 -- 172.21.15.53:0/1558540194 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec02cb40 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdf959a640 1 -- 172.21.15.53:0/1558540194 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0343f0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 -- 172.21.15.53:0/1558540194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 msgr2=0x7fcdfc0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009e50 tx=0x7fcdec02be40 comp rx=0 tx=0).stop 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 -- 172.21.15.53:0/1558540194 shutdown_connections 2023-10-21T16:21:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0cf690 0x7fcdfc0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/1558540194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc0d34d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 -- 172.21.15.53:0/1558540194 >> 172.21.15.53:0/1558540194 conn(0x7fcdfc09bc40 msgr2=0x7fcdfc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 -- 172.21.15.53:0/1558540194 shutdown_connections 2023-10-21T16:21:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.277+0000 7fcdfb59e640 1 -- 172.21.15.53:0/1558540194 wait complete. 2023-10-21T16:21:59.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.279+0000 7fcdfb59e640 1 Processor -- start 2023-10-21T16:21:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.279+0000 7fcdfb59e640 1 -- start start 2023-10-21T16:21:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 0x7fcdfc09ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 0x7fcdfc09f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1f2190 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc15ef40 con 0x7fcdfc0cf690 2023-10-21T16:21:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.280+0000 7fcdfb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc09c650 con 0x7fcdfc0d1100 2023-10-21T16:21:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50616/0 (socket says 172.21.15.53:50616) 2023-10-21T16:21:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 -- 172.21.15.53:0/2522141440 learned_addr learned my addr 172.21.15.53:0/2522141440 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdf9d9b640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 0x7fcdfc09ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 -- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 msgr2=0x7fcdfc09f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfad9d640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 0x7fcdfc09f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 0x7fcdfc09f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 -- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 msgr2=0x7fcdfc09ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 0x7fcdfc09ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfa59c640 1 -- 172.21.15.53:0/2522141440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009b30 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdf9d9b640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 0x7fcdfc09ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.281+0000 7fcdfad9d640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 0x7fcdfc09f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdfa59c640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000f80 tx=0x7fcdec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec03e450 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec03e5f0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec03e9a0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc09c930 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.282+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc09cef0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.283+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc074150 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.284+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcdec03eb00 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.285+0000 7fcdf2ffd640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 0x7fcddc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.285+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fcdec0aca40 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.286+0000 7fcdf9d9b640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 0x7fcddc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.287+0000 7fcdf9d9b640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 0x7fcddc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0c9040 tx=0x7fcde4008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.290+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdec077790 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.525+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcdfc0d0070 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.528 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T16:21:59.526+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T16:21:59.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.530+0000 7fcdf2ffd640 1 -- 172.21.15.53:0/2522141440 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcdec03edb0 con 0x7fcdfc0cf2b0 2023-10-21T16:21:59.532 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T16:21:59.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.533+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 msgr2=0x7fcddc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.533+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 0x7fcddc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0c9040 tx=0x7fcde4008040 comp rx=0 tx=0).stop 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.533+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 msgr2=0x7fcdfc14fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.533+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000f80 tx=0x7fcdec004a40 comp rx=0 tx=0).stop 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 shutdown_connections 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fcddc06af00 0x7fcddc06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdfc0d1100 0x7fcdfc09f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdfc0cf690 0x7fcdfc09ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 --2- 172.21.15.53:0/2522141440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdfc0cf2b0 0x7fcdfc14fd10 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 >> 172.21.15.53:0/2522141440 conn(0x7fcdfc09bc40 msgr2=0x7fcdfc1e70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:59.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 shutdown_connections 2023-10-21T16:21:59.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.534+0000 7fcdfb59e640 1 -- 172.21.15.53:0/2522141440 wait complete. 2023-10-21T16:21:59.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T16:21:59.573 INFO:tasks.ceph.mds.b.smithi124.stderr: -8824> 2023-10-21T16:21:32.413+0000 7f2dce730640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:21:59.676 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:21:59.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.773+0000 7f50225f5640 1 Processor -- start 2023-10-21T16:21:59.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.773+0000 7f50225f5640 1 -- start start 2023-10-21T16:21:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f50225f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f501c0afbe0 0x7f501c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f50225f5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f501c06dfc0 0x7f501c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f50225f5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f501c0b2620 con 0x7f501c071070 2023-10-21T16:21:59.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f50225f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501c0b2060 con 0x7f501c0afbe0 2023-10-21T16:21:59.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f50225f5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501c0b2340 con 0x7f501c06dfc0 2023-10-21T16:21:59.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.775+0000 7f501bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f501c0afbe0 0x7f501c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f501c0afbe0 0x7f501c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50626/0 (socket says 172.21.15.53:50626) 2023-10-21T16:21:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 -- 172.21.15.53:0/334374320 learned_addr learned my addr 172.21.15.53:0/334374320 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f50137fe640 1 --2- 172.21.15.53:0/334374320 >> v2:172.21.15.124:3300/0 conn(0x7f501c06dfc0 0x7f501c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 -- 172.21.15.53:0/334374320 >> v2:172.21.15.124:3300/0 conn(0x7f501c06dfc0 msgr2=0x7f501c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:21:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 --2- 172.21.15.53:0/334374320 >> v2:172.21.15.124:3300/0 conn(0x7f501c06dfc0 0x7f501c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 -- 172.21.15.53:0/334374320 >> v1:172.21.15.124:6789/0 conn(0x7f501c071070 legacy=0x7f501c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.776+0000 7f501bfff640 1 -- 172.21.15.53:0/334374320 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c070350 con 0x7f501c0afbe0 2023-10-21T16:21:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.778+0000 7f501bfff640 1 --2- 172.21.15.53:0/334374320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c0afbe0 0x7f501c0b15b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b3980 tx=0x7f500c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189e5ad1263fcff7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.778+0000 7f501b7fe640 1 -- 172.21.15.53:0/334374320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c02cb20 con 0x7f501c0afbe0 2023-10-21T16:21:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.778+0000 7f501b7fe640 1 -- 172.21.15.53:0/334374320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c02cc80 con 0x7f501c0afbe0 2023-10-21T16:21:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.778+0000 7f501b7fe640 1 -- 172.21.15.53:0/334374320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c0343f0 con 0x7f501c0afbe0 2023-10-21T16:21:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 -- 172.21.15.53:0/334374320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c0afbe0 msgr2=0x7f501c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 --2- 172.21.15.53:0/334374320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c0afbe0 0x7f501c0b15b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b3980 tx=0x7f500c02bf80 comp rx=0 tx=0).stop 2023-10-21T16:21:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 -- 172.21.15.53:0/334374320 shutdown_connections 2023-10-21T16:21:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 --2- 172.21.15.53:0/334374320 >> v2:172.21.15.124:3300/0 conn(0x7f501c06dfc0 0x7f501c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 --2- 172.21.15.53:0/334374320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c0afbe0 0x7f501c0b15b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 -- 172.21.15.53:0/334374320 >> 172.21.15.53:0/334374320 conn(0x7f501c06a5d0 msgr2=0x7f501c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:21:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 -- 172.21.15.53:0/334374320 shutdown_connections 2023-10-21T16:21:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.779+0000 7f50225f5640 1 -- 172.21.15.53:0/334374320 wait complete. 2023-10-21T16:21:59.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.781+0000 7f50225f5640 1 Processor -- start 2023-10-21T16:21:59.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.781+0000 7f50225f5640 1 -- start start 2023-10-21T16:21:59.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 0x7f501c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 0x7f501c0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c147110 con 0x7f501c06dfc0 2023-10-21T16:21:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c0cd010 con 0x7f501c0afbe0 2023-10-21T16:21:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f50225f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501c155eb0 con 0x7f501c071070 2023-10-21T16:21:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f501bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f501bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50636/0 (socket says 172.21.15.53:50636) 2023-10-21T16:21:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.782+0000 7f501bfff640 1 -- 172.21.15.53:0/2761952811 learned_addr learned my addr 172.21.15.53:0/2761952811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:21:59.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f5020b92640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 0x7f501c0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 -- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 msgr2=0x7f501c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f50137fe640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 0x7f501c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 0x7f501c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 -- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 msgr2=0x7f501c0c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:21:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 0x7f501c0c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:21:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 -- 172.21.15.53:0/2761952811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c009b30 con 0x7f501c06dfc0 2023-10-21T16:21:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f5020b92640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 0x7f501c0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f50137fe640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 0x7f501c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:21:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.783+0000 7f501bfff640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f500c000f80 tx=0x7f500c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.784+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c009510 con 0x7f501c06dfc0 2023-10-21T16:21:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.784+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c007e60 con 0x7f501c06dfc0 2023-10-21T16:21:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.784+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c03e690 con 0x7f501c06dfc0 2023-10-21T16:21:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.784+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c0c35b0 con 0x7f501c06dfc0 2023-10-21T16:21:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.784+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c0cc9c0 con 0x7f501c06dfc0 2023-10-21T16:21:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.785+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c04e160 con 0x7f501c06dfc0 2023-10-21T16:21:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.786+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 74) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f500c03e7f0 con 0x7f501c06dfc0 2023-10-21T16:21:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.787+0000 7f50197fa640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 0x7f4ffc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:21:59.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.787+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f500c0ac6b0 con 0x7f501c06dfc0 2023-10-21T16:21:59.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.787+0000 7f50137fe640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 0x7f4ffc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:21:59.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.789+0000 7f50137fe640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 0x7f4ffc06d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f501c0cf790 tx=0x7f5004008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:21:59.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:21:59.791+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f500c0771b0 con 0x7f501c06dfc0 2023-10-21T16:22:00.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.027+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f501c0b0ef0 con 0x7f501c06dfc0 2023-10-21T16:22:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.545+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f500c077580 con 0x7f501c06dfc0 2023-10-21T16:22:00.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.650+0000 7f50197fa640 1 -- 172.21.15.53:0/2761952811 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f500c0900e0 con 0x7f501c06dfc0 2023-10-21T16:22:00.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 msgr2=0x7f4ffc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 0x7f4ffc06d3f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f501c0cf790 tx=0x7f5004008040 comp rx=0 tx=0).stop 2023-10-21T16:22:00.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 msgr2=0x7f501c146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f500c000f80 tx=0x7f500c004a40 comp rx=0 tx=0).stop 2023-10-21T16:22:00.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 shutdown_connections 2023-10-21T16:22:00.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4ffc06af30 0x7f4ffc06d3f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f501c0afbe0 0x7f501c0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f501c071070 0x7f501c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 --2- 172.21.15.53:0/2761952811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f501c06dfc0 0x7f501c146bd0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.654+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 >> 172.21.15.53:0/2761952811 conn(0x7f501c06a5d0 msgr2=0x7f501c06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.655+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 shutdown_connections 2023-10-21T16:22:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.655+0000 7f50225f5640 1 -- 172.21.15.53:0/2761952811 wait complete. 2023-10-21T16:22:00.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T16:22:00.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.914+0000 7f1d8759e640 1 Processor -- start 2023-10-21T16:22:00.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.914+0000 7f1d8759e640 1 -- start start 2023-10-21T16:22:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.915+0000 7f1d8759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d8815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.915+0000 7f1d8759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880cab60 con 0x7f1d880b0680 2023-10-21T16:22:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.915+0000 7f1d8759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880cb960 con 0x7f1d8815a820 2023-10-21T16:22:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.915+0000 7f1d8759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880cbdf0 con 0x7f1d880ca680 2023-10-21T16:22:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.916+0000 7f1d7ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1d880b0680 0x7f1d8815e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40040/0 (socket says 172.21.15.53:40040) 2023-10-21T16:22:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.916+0000 7f1d7ffff640 1 -- 172.21.15.53:0/3606983179 learned_addr learned my addr 172.21.15.53:0/3606983179 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.916+0000 7f1d86d9d640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d8815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.916+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865458773 0 0) 0x7f1d880cab60 con 0x7f1d880b0680 2023-10-21T16:22:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.916+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d74003520 con 0x7f1d880b0680 2023-10-21T16:22:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434408188 0 0) 0x7f1d74003520 con 0x7f1d880b0680 2023-10-21T16:22:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 msgr2=0x7f1d8815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d8815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 >> v1:172.21.15.124:6789/0 conn(0x7f1d8815a820 legacy=0x7f1d880aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d880cb030 con 0x7f1d880b0680 2023-10-21T16:22:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1d70001670 con 0x7f1d880b0680 2023-10-21T16:22:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 >> v1:172.21.15.53:6789/0 conn(0x7f1d880b0680 legacy=0x7f1d8815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d740039a0 0x7f1d74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740047c0 0x7f1d74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880cbdf0 con 0x7f1d740039a0 2023-10-21T16:22:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7400bab0 con 0x7f1d740047c0 2023-10-21T16:22:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7400bc70 con 0x7f1d880ca680 2023-10-21T16:22:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.917+0000 7f1d8659c640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d740039a0 0x7f1d74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d7ffff640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740047c0 0x7f1d74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d86d9d640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d8659c640 1 -- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 msgr2=0x7f1d740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d8659c640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d880ca680 0x7f1d740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d8659c640 1 -- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740047c0 msgr2=0x7f1d74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d8659c640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740047c0 0x7f1d74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d8659c640 1 -- 172.21.15.53:0/3606983179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74003520 con 0x7f1d740039a0 2023-10-21T16:22:00.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.918+0000 7f1d7ffff640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740047c0 0x7f1d74004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.919+0000 7f1d8659c640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d740039a0 0x7f1d74003dc0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001750 tx=0x7f1d740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0dd81625950f05 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.919+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780052d0 con 0x7f1d740039a0 2023-10-21T16:22:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7802e050 con 0x7f1d740039a0 2023-10-21T16:22:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d85d9b640 1 -- 172.21.15.53:0/3606983179 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780035f0 con 0x7f1d740039a0 2023-10-21T16:22:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d8759e640 1 -- 172.21.15.53:0/3606983179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d740039a0 msgr2=0x7f1d74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d8759e640 1 --2- 172.21.15.53:0/3606983179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d740039a0 0x7f1d74003dc0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001750 tx=0x7f1d740057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d86d9d640 1 -- 172.21.15.53:0/3606983179 reap_dead start 2023-10-21T16:22:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d8759e640 1 -- 172.21.15.53:0/3606983179 shutdown_connections 2023-10-21T16:22:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d8759e640 1 -- 172.21.15.53:0/3606983179 >> 172.21.15.53:0/3606983179 conn(0x7f1d8806c050 msgr2=0x7f1d8806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.920+0000 7f1d8759e640 1 -- 172.21.15.53:0/3606983179 shutdown_connections 2023-10-21T16:22:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.921+0000 7f1d8759e640 1 -- 172.21.15.53:0/3606983179 wait complete. 2023-10-21T16:22:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.922+0000 7f1d8759e640 1 Processor -- start 2023-10-21T16:22:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.922+0000 7f1d8759e640 1 -- start start 2023-10-21T16:22:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740039a0 0x7f1d880b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8815b5e0 0x7f1d88158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8815ccd0 con 0x7f1d8815ba60 2023-10-21T16:22:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c5c10 con 0x7f1d740039a0 2023-10-21T16:22:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.923+0000 7f1d8759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8815d4e0 con 0x7f1d8815b5e0 2023-10-21T16:22:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50654/0 (socket says 172.21.15.53:50654) 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 -- 172.21.15.53:0/1211840035 learned_addr learned my addr 172.21.15.53:0/1211840035 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d8659c640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740039a0 0x7f1d880b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d7ffff640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8815b5e0 0x7f1d88158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 -- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8815b5e0 msgr2=0x7f1d88158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8815b5e0 0x7f1d88158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 -- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740039a0 msgr2=0x7f1d880b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740039a0 0x7f1d880b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.924+0000 7f1d86d9d640 1 -- 172.21.15.53:0/1211840035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8815d7c0 con 0x7f1d8815ba60 2023-10-21T16:22:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d86d9d640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1d74004740 tx=0x7f1d800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000e070 con 0x7f1d8815ba60 2023-10-21T16:22:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80011e10 con 0x7f1d8815ba60 2023-10-21T16:22:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d880c55c0 con 0x7f1d8815ba60 2023-10-21T16:22:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003990 con 0x7f1d8815ba60 2023-10-21T16:22:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.925+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8815d170 con 0x7f1d8815ba60 2023-10-21T16:22:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.926+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d880c5da0 con 0x7f1d8815ba60 2023-10-21T16:22:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.927+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d80003af0 con 0x7f1d8815ba60 2023-10-21T16:22:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.928+0000 7f1d7effd640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.928+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f1d8008a600 con 0x7f1d8815ba60 2023-10-21T16:22:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.929+0000 7f1d8659c640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.930+0000 7f1d8659c640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 0x7f1d6806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d78033000 tx=0x7f1d78009030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:00.932+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d80054fc0 con 0x7f1d8815ba60 2023-10-21T16:22:01.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.126+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d88074150 con 0x7f1d8815ba60 2023-10-21T16:22:01.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.654+0000 7f1d7effd640 1 -- 172.21.15.53:0/1211840035 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v152) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1d800585e0 con 0x7f1d8815ba60 2023-10-21T16:22:01.657 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T16:22:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 msgr2=0x7f1d6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 0x7f1d6806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d78033000 tx=0x7f1d78009030 comp rx=0 tx=0).stop 2023-10-21T16:22:01.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 msgr2=0x7f1d88158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1d74004740 tx=0x7f1d800115a0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 shutdown_connections 2023-10-21T16:22:01.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8815ba60 0x7f1d88158c70 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8815b5e0 0x7f1d88158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 --2- 172.21.15.53:0/1211840035 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d740039a0 0x7f1d880b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 >> 172.21.15.53:0/1211840035 conn(0x7f1d8806c050 msgr2=0x7f1d880707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:01.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.659+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 shutdown_connections 2023-10-21T16:22:01.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.660+0000 7f1d8759e640 1 -- 172.21.15.53:0/1211840035 wait complete. 2023-10-21T16:22:01.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T16:22:01.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.912+0000 7f5e9b5a9640 1 Processor -- start 2023-10-21T16:22:01.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.912+0000 7f5e9b5a9640 1 -- start start 2023-10-21T16:22:01.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e9b5a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e9406dfc0 0x7f5e94070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e9b5a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e94071070 0x7f5e940cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e9b5a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e940b1cc0 con 0x7f5e940afbe0 2023-10-21T16:22:01.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e9b5a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e940b2620 con 0x7f5e94071070 2023-10-21T16:22:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e9b5a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940b2060 con 0x7f5e9406dfc0 2023-10-21T16:22:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e99345640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5e940afbe0 0x7f5e940b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40042/0 (socket says 172.21.15.53:40042) 2023-10-21T16:22:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e99345640 1 -- 172.21.15.53:0/2122405849 learned_addr learned my addr 172.21.15.53:0/2122405849 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.914+0000 7f5e99b46640 1 --2- 172.21.15.53:0/2122405849 >> v2:172.21.15.53:3300/0 conn(0x7f5e94071070 0x7f5e940cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.915+0000 7f5e98b44640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e9406dfc0 0x7f5e94070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.915+0000 7f5e99b46640 1 -- 172.21.15.53:0/2122405849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e9406dfc0 msgr2=0x7f5e94070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:01.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.915+0000 7f5e99b46640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e9406dfc0 0x7f5e94070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.915+0000 7f5e99b46640 1 -- 172.21.15.53:0/2122405849 >> v1:172.21.15.53:6789/0 conn(0x7f5e940afbe0 legacy=0x7f5e940b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.915+0000 7f5e99b46640 1 -- 172.21.15.53:0/2122405849 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940b2340 con 0x7f5e94071070 2023-10-21T16:22:01.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.916+0000 7f5e99b46640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cb130 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009d90 tx=0x7f5e8c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b9fe4492e25161 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:01.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.916+0000 7f5e8bfff640 1 -- 172.21.15.53:0/2122405849 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c02cc60 con 0x7f5e94071070 2023-10-21T16:22:01.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e8bfff640 1 -- 172.21.15.53:0/2122405849 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8c02cdc0 con 0x7f5e94071070 2023-10-21T16:22:01.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e8bfff640 1 -- 172.21.15.53:0/2122405849 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c034660 con 0x7f5e94071070 2023-10-21T16:22:01.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2122405849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 msgr2=0x7f5e940cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cb130 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009d90 tx=0x7f5e8c02beb0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2122405849 shutdown_connections 2023-10-21T16:22:01.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cb130 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2122405849 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e9406dfc0 0x7f5e94070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2122405849 >> 172.21.15.53:0/2122405849 conn(0x7f5e9406a5d0 msgr2=0x7f5e940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:01.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2122405849 shutdown_connections 2023-10-21T16:22:01.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.917+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2122405849 wait complete. 2023-10-21T16:22:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.919+0000 7f5e9b5a9640 1 Processor -- start 2023-10-21T16:22:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.919+0000 7f5e9b5a9640 1 -- start start 2023-10-21T16:22:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 0x7f5e9415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 0x7f5e940ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9415bde0 con 0x7f5e94071070 2023-10-21T16:22:01.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9415b4d0 con 0x7f5e9406dfc0 2023-10-21T16:22:01.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e9b5a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9415e2d0 con 0x7f5e940afbe0 2023-10-21T16:22:01.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.920+0000 7f5e98b44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50672/0 (socket says 172.21.15.53:50672) 2023-10-21T16:22:01.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 -- 172.21.15.53:0/2964425637 learned_addr learned my addr 172.21.15.53:0/2964425637 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:01.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 -- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 msgr2=0x7f5e940ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e99345640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 0x7f5e9415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e99b46640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 0x7f5e940ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 0x7f5e940ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 -- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 msgr2=0x7f5e9415c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:01.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 0x7f5e9415c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:01.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 -- 172.21.15.53:0/2964425637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009a70 con 0x7f5e94071070 2023-10-21T16:22:01.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e99b46640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 0x7f5e940ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:01.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e99345640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 0x7f5e9415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.921+0000 7f5e98b44640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d7d0 tx=0x7f5e8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.922+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84018070 con 0x7f5e94071070 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.922+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84004020 con 0x7f5e94071070 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.922+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e840041b0 con 0x7f5e94071070 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.922+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9415b720 con 0x7f5e94071070 2023-10-21T16:22:01.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.922+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9415df80 con 0x7f5e94071070 2023-10-21T16:22:01.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.923+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9415bf20 con 0x7f5e94071070 2023-10-21T16:22:01.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.924+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e84004310 con 0x7f5e94071070 2023-10-21T16:22:01.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.925+0000 7f5e89ffb640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 0x7f5e7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:01.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.925+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6480+0+0 (secure 0 0 0) 0x7f5e84089910 con 0x7f5e94071070 2023-10-21T16:22:01.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.926+0000 7f5e99345640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 0x7f5e7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:01.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.927+0000 7f5e99345640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 0x7f5e7806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:01.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:01.930+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e9415bf20 con 0x7f5e94071070 2023-10-21T16:22:02.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.122+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e9415bf20 con 0x7f5e94071070 2023-10-21T16:22:02.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.681+0000 7f5e89ffb640 1 -- 172.21.15.53:0/2964425637 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v153) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5e9415bf20 con 0x7f5e94071070 2023-10-21T16:22:02.683 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T16:22:02.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 msgr2=0x7f5e7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 0x7f5e7806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007fd0 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-10-21T16:22:02.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 msgr2=0x7f5e940cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d7d0 tx=0x7f5e8400dc90 comp rx=0 tx=0).stop 2023-10-21T16:22:02.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 shutdown_connections 2023-10-21T16:22:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5e7806af90 0x7f5e7806d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e940afbe0 0x7f5e940ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e94071070 0x7f5e940cdc30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 --2- 172.21.15.53:0/2964425637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e9406dfc0 0x7f5e9415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.687+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 >> 172.21.15.53:0/2964425637 conn(0x7f5e9406a5d0 msgr2=0x7f5e940c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:02.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.689+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 shutdown_connections 2023-10-21T16:22:02.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.689+0000 7f5e9b5a9640 1 -- 172.21.15.53:0/2964425637 wait complete. 2023-10-21T16:22:02.716 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T16:22:02.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.920+0000 7f4c67661640 1 Processor -- start 2023-10-21T16:22:02.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.920+0000 7f4c67661640 1 -- start start 2023-10-21T16:22:02.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c67661640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4c600cb130 0x7f4c600cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:02.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c67661640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600b1cc0 con 0x7f4c600afbe0 2023-10-21T16:22:02.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c67661640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600cbad0 con 0x7f4c6006dfc0 2023-10-21T16:22:02.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c67661640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600b2620 con 0x7f4c600cb130 2023-10-21T16:22:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c653fd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4c600afbe0 0x7f4c600b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40044/0 (socket says 172.21.15.53:40044) 2023-10-21T16:22:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.922+0000 7f4c653fd640 1 -- 172.21.15.53:0/2749215934 learned_addr learned my addr 172.21.15.53:0/2749215934 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c65bfe640 1 --2- 172.21.15.53:0/2749215934 >> v2:172.21.15.124:3300/0 conn(0x7f4c600cb130 0x7f4c600cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:02.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c57fff640 1 -- 172.21.15.53:0/2749215934 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552645268 0 0) 0x7f4c600b1cc0 con 0x7f4c600afbe0 2023-10-21T16:22:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c57fff640 1 -- 172.21.15.53:0/2749215934 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c003520 con 0x7f4c600afbe0 2023-10-21T16:22:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c65bfe640 1 -- 172.21.15.53:0/2749215934 >> v1:172.21.15.124:6789/0 conn(0x7f4c6006dfc0 legacy=0x7f4c60070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c65bfe640 1 -- 172.21.15.53:0/2749215934 >> v1:172.21.15.53:6789/0 conn(0x7f4c600afbe0 legacy=0x7f4c600b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.923+0000 7f4c65bfe640 1 -- 172.21.15.53:0/2749215934 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600b2060 con 0x7f4c600cb130 2023-10-21T16:22:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.925+0000 7f4c65bfe640 1 --2- 172.21.15.53:0/2749215934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c600cb130 0x7f4c600cb590 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009d90 tx=0x7f4c5802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4a7b14a885c202 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:02.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.925+0000 7f4c57fff640 1 -- 172.21.15.53:0/2749215934 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5802ccf0 con 0x7f4c600cb130 2023-10-21T16:22:02.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.925+0000 7f4c57fff640 1 -- 172.21.15.53:0/2749215934 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5802ce50 con 0x7f4c600cb130 2023-10-21T16:22:02.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.925+0000 7f4c57fff640 1 -- 172.21.15.53:0/2749215934 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58034660 con 0x7f4c600cb130 2023-10-21T16:22:02.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 -- 172.21.15.53:0/2749215934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c600cb130 msgr2=0x7f4c600cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 --2- 172.21.15.53:0/2749215934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c600cb130 0x7f4c600cb590 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009d90 tx=0x7f4c5802bf40 comp rx=0 tx=0).stop 2023-10-21T16:22:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 -- 172.21.15.53:0/2749215934 shutdown_connections 2023-10-21T16:22:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 --2- 172.21.15.53:0/2749215934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c600cb130 0x7f4c600cb590 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 -- 172.21.15.53:0/2749215934 >> 172.21.15.53:0/2749215934 conn(0x7f4c6006a5d0 msgr2=0x7f4c600b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 -- 172.21.15.53:0/2749215934 shutdown_connections 2023-10-21T16:22:02.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.926+0000 7f4c67661640 1 -- 172.21.15.53:0/2749215934 wait complete. 2023-10-21T16:22:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.928+0000 7f4c67661640 1 Processor -- start 2023-10-21T16:22:02.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.928+0000 7f4c67661640 1 -- start start 2023-10-21T16:22:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 0x7f4c600cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c600afbe0 0x7f4c601f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:02.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601f6a30 con 0x7f4c600cb130 2023-10-21T16:22:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60149ce0 con 0x7f4c6006dfc0 2023-10-21T16:22:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c67661640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6014fea0 con 0x7f4c600afbe0 2023-10-21T16:22:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c65bfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:02.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c65bfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50682/0 (socket says 172.21.15.53:50682) 2023-10-21T16:22:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.929+0000 7f4c65bfe640 1 -- 172.21.15.53:0/743581116 learned_addr learned my addr 172.21.15.53:0/743581116 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c653fd640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 0x7f4c600cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c64bfc640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c600afbe0 0x7f4c601f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 -- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c600afbe0 msgr2=0x7f4c601f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c600afbe0 0x7f4c601f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 -- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 msgr2=0x7f4c600cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 0x7f4c600cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 -- 172.21.15.53:0/743581116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58009a70 con 0x7f4c600cb130 2023-10-21T16:22:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c653fd640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 0x7f4c600cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:22:02.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.930+0000 7f4c65bfe640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f4c58037040 tx=0x7f4c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.931+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5803d600 con 0x7f4c600cb130 2023-10-21T16:22:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.931+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5803d760 con 0x7f4c600cb130 2023-10-21T16:22:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.931+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5803db10 con 0x7f4c600cb130 2023-10-21T16:22:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.931+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c601f6190 con 0x7f4c600cb130 2023-10-21T16:22:02.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.931+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c601f6750 con 0x7f4c600cb130 2023-10-21T16:22:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.932+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60149e20 con 0x7f4c600cb130 2023-10-21T16:22:02.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.933+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c5803b040 con 0x7f4c600cb130 2023-10-21T16:22:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.934+0000 7f4c55ffb640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 0x7f4c4406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.934+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4c580abe90 con 0x7f4c600cb130 2023-10-21T16:22:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.935+0000 7f4c653fd640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 0x7f4c4406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:02.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.936+0000 7f4c653fd640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 0x7f4c4406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0011e0 tx=0x7f4c5c009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:02.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:02.938+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c60149e20 con 0x7f4c600cb130 2023-10-21T16:22:03.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.131+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4c60149e20 con 0x7f4c600cb130 2023-10-21T16:22:03.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.139+0000 7f4c55ffb640 1 -- 172.21.15.53:0/743581116 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4c60149e20 con 0x7f4c600cb130 2023-10-21T16:22:03.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 msgr2=0x7f4c4406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 0x7f4c4406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0011e0 tx=0x7f4c5c009040 comp rx=0 tx=0).stop 2023-10-21T16:22:03.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 msgr2=0x7f4c601f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f4c58037040 tx=0x7f4c58004a40 comp rx=0 tx=0).stop 2023-10-21T16:22:03.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 shutdown_connections 2023-10-21T16:22:03.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.141+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4c4406af90 0x7f4c4406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c600cb130 0x7f4c601f1ce0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c600afbe0 0x7f4c601f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 --2- 172.21.15.53:0/743581116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c6006dfc0 0x7f4c600cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 >> 172.21.15.53:0/743581116 conn(0x7f4c6006a5d0 msgr2=0x7f4c60070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:03.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 shutdown_connections 2023-10-21T16:22:03.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.142+0000 7f4c67661640 1 -- 172.21.15.53:0/743581116 wait complete. 2023-10-21T16:22:03.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T16:22:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.386+0000 7f3f364b4640 1 Processor -- start 2023-10-21T16:22:03.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.386+0000 7f3f364b4640 1 -- start start 2023-10-21T16:22:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.388+0000 7f3f364b4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3f3015aad0 0x7f3f300b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.388+0000 7f3f364b4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3f3006d410 0x7f3f300c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.388+0000 7f3f364b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3015bf30 con 0x7f3f300c82b0 2023-10-21T16:22:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.388+0000 7f3f364b4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3015b320 con 0x7f3f3006d410 2023-10-21T16:22:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.388+0000 7f3f364b4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3015b600 con 0x7f3f3015aad0 2023-10-21T16:22:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f27fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3f300c82b0 0x7f3f3006cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40060/0 (socket says 172.21.15.53:40060) 2023-10-21T16:22:03.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f27fff640 1 -- 172.21.15.53:0/376565160 learned_addr learned my addr 172.21.15.53:0/376565160 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:03.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f34a51640 1 --2- 172.21.15.53:0/376565160 >> v2:172.21.15.53:3300/0 conn(0x7f3f3006d410 0x7f3f300c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f2ffff640 1 --2- 172.21.15.53:0/376565160 >> v2:172.21.15.124:3300/0 conn(0x7f3f3015aad0 0x7f3f300b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f34a51640 1 -- 172.21.15.53:0/376565160 >> v2:172.21.15.124:3300/0 conn(0x7f3f3015aad0 msgr2=0x7f3f300b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f34a51640 1 --2- 172.21.15.53:0/376565160 >> v2:172.21.15.124:3300/0 conn(0x7f3f3015aad0 0x7f3f300b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f34a51640 1 -- 172.21.15.53:0/376565160 >> v1:172.21.15.53:6789/0 conn(0x7f3f300c82b0 legacy=0x7f3f3006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.389+0000 7f3f34a51640 1 -- 172.21.15.53:0/376565160 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3015b8e0 con 0x7f3f3006d410 2023-10-21T16:22:03.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f34a51640 1 --2- 172.21.15.53:0/376565160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f300c8710 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3f3006e6a0 tx=0x7f3f2802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2261a4a8e77c78c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/376565160 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2802cc70 con 0x7f3f3006d410 2023-10-21T16:22:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/376565160 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2802cdd0 con 0x7f3f3006d410 2023-10-21T16:22:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f2f7fe640 1 -- 172.21.15.53:0/376565160 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28034660 con 0x7f3f3006d410 2023-10-21T16:22:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f364b4640 1 -- 172.21.15.53:0/376565160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 msgr2=0x7f3f300c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.391+0000 7f3f364b4640 1 --2- 172.21.15.53:0/376565160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f300c8710 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3f3006e6a0 tx=0x7f3f2802bec0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 -- 172.21.15.53:0/376565160 shutdown_connections 2023-10-21T16:22:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 --2- 172.21.15.53:0/376565160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f300c8710 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 --2- 172.21.15.53:0/376565160 >> v2:172.21.15.124:3300/0 conn(0x7f3f3015aad0 0x7f3f300b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 -- 172.21.15.53:0/376565160 >> 172.21.15.53:0/376565160 conn(0x7f3f30069d50 msgr2=0x7f3f3006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:03.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 -- 172.21.15.53:0/376565160 shutdown_connections 2023-10-21T16:22:03.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.392+0000 7f3f364b4640 1 -- 172.21.15.53:0/376565160 wait complete. 2023-10-21T16:22:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.394+0000 7f3f364b4640 1 Processor -- start 2023-10-21T16:22:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.394+0000 7f3f364b4640 1 -- start start 2023-10-21T16:22:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 0x7f3f30149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 0x7f3f3014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30156f40 con 0x7f3f3006d410 2023-10-21T16:22:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300c91e0 con 0x7f3f3015aad0 2023-10-21T16:22:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f364b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f30157190 con 0x7f3f300c82b0 2023-10-21T16:22:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50698/0 (socket says 172.21.15.53:50698) 2023-10-21T16:22:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 -- 172.21.15.53:0/1866783619 learned_addr learned my addr 172.21.15.53:0/1866783619 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f34a51640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 0x7f3f3014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 -- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 msgr2=0x7f3f30149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f27fff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 0x7f3f30149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 0x7f3f30149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 -- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 msgr2=0x7f3f3014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.395+0000 7f3f2ffff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 0x7f3f3014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f2ffff640 1 -- 172.21.15.53:0/1866783619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f28009a70 con 0x7f3f3006d410 2023-10-21T16:22:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f34a51640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 0x7f3f3014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:03.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f27fff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 0x7f3f30149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:03.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f2ffff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000d5f0 tx=0x7f3f2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200125f0 con 0x7f3f3006d410 2023-10-21T16:22:03.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20012750 con 0x7f3f3006d410 2023-10-21T16:22:03.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20012b00 con 0x7f3f3006d410 2023-10-21T16:22:03.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f30157470 con 0x7f3f3006d410 2023-10-21T16:22:03.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.396+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f30157750 con 0x7f3f3006d410 2023-10-21T16:22:03.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.397+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3004e160 con 0x7f3f3006d410 2023-10-21T16:22:03.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.399+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f20012c60 con 0x7f3f3006d410 2023-10-21T16:22:03.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.400+0000 7f3f2d7fa640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 0x7f3f1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.400+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3f20089e30 con 0x7f3f3006d410 2023-10-21T16:22:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.400+0000 7f3f27fff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 0x7f3f1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.402+0000 7f3f27fff640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 0x7f3f1006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f300caca0 tx=0x7f3f18009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.404+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f20054bc0 con 0x7f3f3006d410 2023-10-21T16:22:03.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.599+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3f300b33c0 con 0x7f3f3006d410 2023-10-21T16:22:03.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.607+0000 7f3f2d7fa640 1 -- 172.21.15.53:0/1866783619 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3f200581e0 con 0x7f3f3006d410 2023-10-21T16:22:03.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 msgr2=0x7f3f1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 0x7f3f1006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f300caca0 tx=0x7f3f18009040 comp rx=0 tx=0).stop 2023-10-21T16:22:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 msgr2=0x7f3f3014ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000d5f0 tx=0x7f3f2000dab0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 shutdown_connections 2023-10-21T16:22:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3f1006af90 0x7f3f1006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f3015aad0 0x7f3f3014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f300c82b0 0x7f3f30149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 --2- 172.21.15.53:0/1866783619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f3006d410 0x7f3f3014ef60 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 >> 172.21.15.53:0/1866783619 conn(0x7f3f30069d50 msgr2=0x7f3f3015c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:03.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 shutdown_connections 2023-10-21T16:22:03.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.610+0000 7f3f364b4640 1 -- 172.21.15.53:0/1866783619 wait complete. 2023-10-21T16:22:03.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T16:22:03.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.848+0000 7f402fbc9640 1 Processor -- start 2023-10-21T16:22:03.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.849+0000 7f402fbc9640 1 -- start start 2023-10-21T16:22:03.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.850+0000 7f402fbc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f40280c9cd0 0x7f40280cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.850+0000 7f402fbc9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4028069d50 con 0x7f40280c5bf0 2023-10-21T16:22:03.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.850+0000 7f402fbc9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402806b5a0 con 0x7f402815ab10 2023-10-21T16:22:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.850+0000 7f402fbc9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402815aef0 con 0x7f40280c9cd0 2023-10-21T16:22:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f40280c9cd0 0x7f40280cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f40280c9cd0 0x7f40280cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50714/0 (socket says 172.21.15.53:50714) 2023-10-21T16:22:03.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 -- 172.21.15.53:0/2527733535 learned_addr learned my addr 172.21.15.53:0/2527733535 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 -- 172.21.15.53:0/2527733535 >> v1:172.21.15.124:6789/0 conn(0x7f402815ab10 legacy=0x7f40280c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:03.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 -- 172.21.15.53:0/2527733535 >> v1:172.21.15.53:6789/0 conn(0x7f40280c5bf0 legacy=0x7f402806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.851+0000 7f402d164640 1 -- 172.21.15.53:0/2527733535 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028069f80 con 0x7f40280c9cd0 2023-10-21T16:22:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.853+0000 7f402d164640 1 --2- 172.21.15.53:0/2527733535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40280c9cd0 0x7f40280cb520 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f40180098a0 tx=0x7f401802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5c9132cf117057 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.853+0000 7f402c963640 1 -- 172.21.15.53:0/2527733535 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018031640 con 0x7f40280c9cd0 2023-10-21T16:22:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.853+0000 7f402c963640 1 -- 172.21.15.53:0/2527733535 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180317a0 con 0x7f40280c9cd0 2023-10-21T16:22:03.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.853+0000 7f402c963640 1 -- 172.21.15.53:0/2527733535 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018031b30 con 0x7f40280c9cd0 2023-10-21T16:22:03.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 -- 172.21.15.53:0/2527733535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40280c9cd0 msgr2=0x7f40280cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2527733535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40280c9cd0 0x7f40280cb520 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f40180098a0 tx=0x7f401802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 -- 172.21.15.53:0/2527733535 shutdown_connections 2023-10-21T16:22:03.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2527733535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40280c9cd0 0x7f40280cb520 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 -- 172.21.15.53:0/2527733535 >> 172.21.15.53:0/2527733535 conn(0x7f40280b0af0 msgr2=0x7f40280b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:03.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 -- 172.21.15.53:0/2527733535 shutdown_connections 2023-10-21T16:22:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.854+0000 7f402fbc9640 1 -- 172.21.15.53:0/2527733535 wait complete. 2023-10-21T16:22:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.856+0000 7f402fbc9640 1 Processor -- start 2023-10-21T16:22:03.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.856+0000 7f402fbc9640 1 -- start start 2023-10-21T16:22:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40280c5bf0 0x7f40280ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 0x7f40280c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b39b0 con 0x7f402815ab10 2023-10-21T16:22:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b2630 con 0x7f40280c9cd0 2023-10-21T16:22:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402fbc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280b2910 con 0x7f40280c5bf0 2023-10-21T16:22:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402e166640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402e166640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50728/0 (socket says 172.21.15.53:50728) 2023-10-21T16:22:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402e166640 1 -- 172.21.15.53:0/2296951591 learned_addr learned my addr 172.21.15.53:0/2296951591 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.858+0000 7f402d965640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40280c5bf0 0x7f40280ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402d164640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 0x7f40280c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 -- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40280c5bf0 msgr2=0x7f40280ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40280c5bf0 0x7f40280ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 -- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 msgr2=0x7f40280c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 0x7f40280c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 -- 172.21.15.53:0/2296951591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180094d0 con 0x7f402815ab10 2023-10-21T16:22:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402d164640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 0x7f40280c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:22:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.859+0000 7f402e166640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f402000c730 tx=0x7f402000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.860+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020003b70 con 0x7f402815ab10 2023-10-21T16:22:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.860+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020003cd0 con 0x7f402815ab10 2023-10-21T16:22:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.860+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402000aa90 con 0x7f402815ab10 2023-10-21T16:22:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.860+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280b2bf0 con 0x7f402815ab10 2023-10-21T16:22:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.860+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40280b31b0 con 0x7f402815ab10 2023-10-21T16:22:03.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.861+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40280caed0 con 0x7f402815ab10 2023-10-21T16:22:03.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.863+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f402000abf0 con 0x7f402815ab10 2023-10-21T16:22:03.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.863+0000 7f401e7fc640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 0x7f400c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:03.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.864+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4020089610 con 0x7f402815ab10 2023-10-21T16:22:03.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.864+0000 7f402d965640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 0x7f400c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:03.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.865+0000 7f402d965640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 0x7f400c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40280c7940 tx=0x7f4024009060 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:03.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:03.868+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4020054440 con 0x7f402815ab10 2023-10-21T16:22:04.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.062+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f402804e160 con 0x7f402815ab10 2023-10-21T16:22:04.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.070+0000 7f401e7fc640 1 -- 172.21.15.53:0/2296951591 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4020057a60 con 0x7f402815ab10 2023-10-21T16:22:04.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.072+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 msgr2=0x7f400c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.072+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 0x7f400c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40280c7940 tx=0x7f4024009060 comp rx=0 tx=0).stop 2023-10-21T16:22:04.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.072+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 msgr2=0x7f4028155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.072+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f402000c730 tx=0x7f402000cbf0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 shutdown_connections 2023-10-21T16:22:04.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f400c06af90 0x7f400c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f402815ab10 0x7f4028155a80 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40280c9cd0 0x7f40280c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 --2- 172.21.15.53:0/2296951591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40280c5bf0 0x7f40280ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 >> 172.21.15.53:0/2296951591 conn(0x7f40280b0af0 msgr2=0x7f40280b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:04.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 shutdown_connections 2023-10-21T16:22:04.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.073+0000 7f402fbc9640 1 -- 172.21.15.53:0/2296951591 wait complete. 2023-10-21T16:22:04.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-21T16:22:04.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.321+0000 7fa5ea41a640 1 Processor -- start 2023-10-21T16:22:04.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.321+0000 7fa5ea41a640 1 -- start start 2023-10-21T16:22:04.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.322+0000 7fa5ea41a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5e40cd820 0x7fa5e40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.322+0000 7fa5ea41a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cf670 con 0x7fa5e415d1d0 2023-10-21T16:22:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.322+0000 7fa5ea41a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40caf80 con 0x7fa5e415cdf0 2023-10-21T16:22:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.322+0000 7fa5ea41a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40cf8c0 con 0x7fa5e40cd820 2023-10-21T16:22:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5e40cd820 0x7fa5e40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5e40cd820 0x7fa5e40cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48186/0 (socket says 172.21.15.53:48186) 2023-10-21T16:22:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 -- 172.21.15.53:0/4008720406 learned_addr learned my addr 172.21.15.53:0/4008720406 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:04.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 -- 172.21.15.53:0/4008720406 >> v1:172.21.15.124:6789/0 conn(0x7fa5e415cdf0 legacy=0x7fa5e40cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 -- 172.21.15.53:0/4008720406 >> v1:172.21.15.53:6789/0 conn(0x7fa5e415d1d0 legacy=0x7fa5e40cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.323+0000 7fa5e8c17640 1 -- 172.21.15.53:0/4008720406 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40cfba0 con 0x7fa5e40cd820 2023-10-21T16:22:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.324+0000 7fa5e8c17640 1 --2- 172.21.15.53:0/4008720406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e40cd820 0x7fa5e40cb430 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20332b8f7e5d37 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.324+0000 7fa5dbfff640 1 -- 172.21.15.53:0/4008720406 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d402f850 con 0x7fa5e40cd820 2023-10-21T16:22:04.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.324+0000 7fa5dbfff640 1 -- 172.21.15.53:0/4008720406 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d402f9b0 con 0x7fa5e40cd820 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.324+0000 7fa5dbfff640 1 -- 172.21.15.53:0/4008720406 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d402fd40 con 0x7fa5e40cd820 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 -- 172.21.15.53:0/4008720406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e40cd820 msgr2=0x7fa5e40cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/4008720406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e40cd820 0x7fa5e40cb430 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009870 tx=0x7fa5d402ba30 comp rx=0 tx=0).stop 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 -- 172.21.15.53:0/4008720406 shutdown_connections 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/4008720406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e40cd820 0x7fa5e40cb430 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 -- 172.21.15.53:0/4008720406 >> 172.21.15.53:0/4008720406 conn(0x7fa5e40d0890 msgr2=0x7fa5e40d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 -- 172.21.15.53:0/4008720406 shutdown_connections 2023-10-21T16:22:04.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.325+0000 7fa5ea41a640 1 -- 172.21.15.53:0/4008720406 wait complete. 2023-10-21T16:22:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.327+0000 7fa5ea41a640 1 Processor -- start 2023-10-21T16:22:04.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.327+0000 7fa5ea41a640 1 -- start start 2023-10-21T16:22:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5e40cd820 0x7fa5e40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5e415cdf0 0x7fa5e4160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40b6530 con 0x7fa5e415d1d0 2023-10-21T16:22:04.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40c5620 con 0x7fa5e415cdf0 2023-10-21T16:22:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5ea41a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40c5900 con 0x7fa5e40cd820 2023-10-21T16:22:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.328+0000 7fa5e9c19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48194/0 (socket says 172.21.15.53:48194) 2023-10-21T16:22:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 -- 172.21.15.53:0/2025105270 learned_addr learned my addr 172.21.15.53:0/2025105270 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:04.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e8c17640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5e415cdf0 0x7fa5e4160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9418640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5e40cd820 0x7fa5e40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 -- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5e40cd820 msgr2=0x7fa5e40c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5e40cd820 0x7fa5e40c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 -- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5e415cdf0 msgr2=0x7fa5e4160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5e415cdf0 0x7fa5e4160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 -- 172.21.15.53:0/2025105270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40094d0 con 0x7fa5e415d1d0 2023-10-21T16:22:04.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.329+0000 7fa5e9c19640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c3c0 tx=0x7fa5dc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.330+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc0052d0 con 0x7fa5e415d1d0 2023-10-21T16:22:04.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.330+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc005430 con 0x7fa5e415d1d0 2023-10-21T16:22:04.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.330+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc003830 con 0x7fa5e415d1d0 2023-10-21T16:22:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.330+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e40c5be0 con 0x7fa5e415d1d0 2023-10-21T16:22:04.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.330+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e41593f0 con 0x7fa5e415d1d0 2023-10-21T16:22:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.331+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e40c6100 con 0x7fa5e415d1d0 2023-10-21T16:22:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.332+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa5dc003990 con 0x7fa5e415d1d0 2023-10-21T16:22:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.333+0000 7fa5d9ffb640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 0x7fa5c806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.333+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa5dc088ed0 con 0x7fa5e415d1d0 2023-10-21T16:22:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.334+0000 7fa5e9418640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 0x7fa5c806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.335+0000 7fa5e9418640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 0x7fa5c806d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00012e0 tx=0x7fa5e0009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.337+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5e40c6100 con 0x7fa5e415d1d0 2023-10-21T16:22:04.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.600+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7fa5e40c6100 con 0x7fa5e415d1d0 2023-10-21T16:22:04.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.686+0000 7fa5d9ffb640 1 -- 172.21.15.53:0/2025105270 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v795) v1 ==== 160+0+0 (secure 0 0 0) 0x7fa5e40c6100 con 0x7fa5e415d1d0 2023-10-21T16:22:04.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 msgr2=0x7fa5c806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 0x7fa5c806d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00012e0 tx=0x7fa5e0009040 comp rx=0 tx=0).stop 2023-10-21T16:22:04.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 msgr2=0x7fa5e4158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc00c3c0 tx=0x7fa5dc00c7c0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 shutdown_connections 2023-10-21T16:22:04.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fa5c806afc0 0x7fa5c806d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5e415d1d0 0x7fa5e4158eb0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5e415cdf0 0x7fa5e4160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 --2- 172.21.15.53:0/2025105270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5e40cd820 0x7fa5e40c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 >> 172.21.15.53:0/2025105270 conn(0x7fa5e40d0890 msgr2=0x7fa5e40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:04.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 shutdown_connections 2023-10-21T16:22:04.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.690+0000 7fa5ea41a640 1 -- 172.21.15.53:0/2025105270 wait complete. 2023-10-21T16:22:04.712 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-10-21T16:22:04.713 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-21T16:22:04.713 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T16:22:04.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:22:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.932+0000 7f17a9b97640 1 Processor -- start 2023-10-21T16:22:04.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.933+0000 7f17a9b97640 1 -- start start 2023-10-21T16:22:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.934+0000 7f17a9b97640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17a415ab40 0x7f17a40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.934+0000 7f17a9b97640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17a40c90b0 0x7f17a40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a9b97640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a415b2b0 con 0x7f17a40cae70 2023-10-21T16:22:04.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a9b97640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a415bf30 con 0x7f17a415ab40 2023-10-21T16:22:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a9b97640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a415f7a0 con 0x7f17a40c90b0 2023-10-21T16:22:04.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17a415ab40 0x7f17a40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f17a415ab40 0x7f17a40ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48206/0 (socket says 172.21.15.53:48206) 2023-10-21T16:22:04.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 -- 172.21.15.53:0/3095177632 learned_addr learned my addr 172.21.15.53:0/3095177632 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:04.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 -- 172.21.15.53:0/3095177632 >> v2:172.21.15.124:3300/0 conn(0x7f17a40c90b0 msgr2=0x7f17a40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:04.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 --2- 172.21.15.53:0/3095177632 >> v2:172.21.15.124:3300/0 conn(0x7f17a40c90b0 0x7f17a40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 -- 172.21.15.53:0/3095177632 >> v1:172.21.15.124:6789/0 conn(0x7f17a40cae70 legacy=0x7f17a40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:04.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.935+0000 7f17a37fe640 1 -- 172.21.15.53:0/3095177632 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a415b590 con 0x7f17a415ab40 2023-10-21T16:22:04.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.937+0000 7f17a37fe640 1 --2- 172.21.15.53:0/3095177632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40ad720 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f17a406e560 tx=0x7f179402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa46b63367c06c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.937+0000 7f17a27fc640 1 -- 172.21.15.53:0/3095177632 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179402ca80 con 0x7f17a415ab40 2023-10-21T16:22:04.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.937+0000 7f17a27fc640 1 -- 172.21.15.53:0/3095177632 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179402cbe0 con 0x7f17a415ab40 2023-10-21T16:22:04.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.937+0000 7f17a27fc640 1 -- 172.21.15.53:0/3095177632 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940343f0 con 0x7f17a415ab40 2023-10-21T16:22:04.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 -- 172.21.15.53:0/3095177632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 msgr2=0x7f17a40ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 --2- 172.21.15.53:0/3095177632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40ad720 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f17a406e560 tx=0x7f179402bee0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 -- 172.21.15.53:0/3095177632 shutdown_connections 2023-10-21T16:22:04.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 --2- 172.21.15.53:0/3095177632 >> v2:172.21.15.124:3300/0 conn(0x7f17a40c90b0 0x7f17a40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 --2- 172.21.15.53:0/3095177632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40ad720 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 -- 172.21.15.53:0/3095177632 >> 172.21.15.53:0/3095177632 conn(0x7f17a406a380 msgr2=0x7f17a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:04.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 -- 172.21.15.53:0/3095177632 shutdown_connections 2023-10-21T16:22:04.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.938+0000 7f17a9b97640 1 -- 172.21.15.53:0/3095177632 wait complete. 2023-10-21T16:22:04.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.940+0000 7f17a9b97640 1 Processor -- start 2023-10-21T16:22:04.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.940+0000 7f17a9b97640 1 -- start start 2023-10-21T16:22:04.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17a40c90b0 0x7f17a40c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17a40cae70 0x7f17a415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40c5580 con 0x7f17a415ab40 2023-10-21T16:22:04.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a415dc90 con 0x7f17a40c90b0 2023-10-21T16:22:04.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a9b97640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a415c6b0 con 0x7f17a40cae70 2023-10-21T16:22:04.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48218/0 (socket says 172.21.15.53:48218) 2023-10-21T16:22:04.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.941+0000 7f17a3fff640 1 -- 172.21.15.53:0/2887480863 learned_addr learned my addr 172.21.15.53:0/2887480863 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:04.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a2ffd640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17a40cae70 0x7f17a415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a37fe640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17a40c90b0 0x7f17a40c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 -- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17a40cae70 msgr2=0x7f17a415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17a40cae70 0x7f17a415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 -- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17a40c90b0 msgr2=0x7f17a40c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:04.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17a40c90b0 0x7f17a40c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:04.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 -- 172.21.15.53:0/2887480863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009b30 con 0x7f17a415ab40 2023-10-21T16:22:04.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f17a3fff640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c390 tx=0x7f179c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.942+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0035f0 con 0x7f17a415ab40 2023-10-21T16:22:04.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.943+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c003750 con 0x7f17a415ab40 2023-10-21T16:22:04.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.943+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c003b00 con 0x7f17a415ab40 2023-10-21T16:22:04.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.943+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a415c990 con 0x7f17a415ab40 2023-10-21T16:22:04.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.943+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a415cf50 con 0x7f17a415ab40 2023-10-21T16:22:04.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.944+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a4067490 con 0x7f17a415ab40 2023-10-21T16:22:04.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.945+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f179c003c60 con 0x7f17a415ab40 2023-10-21T16:22:04.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.946+0000 7f179bfff640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 0x7f178406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:04.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.946+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f179c088b70 con 0x7f17a415ab40 2023-10-21T16:22:04.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.947+0000 7f17a37fe640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 0x7f178406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:04.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.948+0000 7f17a37fe640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 0x7f178406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1794035000 tx=0x7f1794030000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:04.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:04.951+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f179c053900 con 0x7f17a415ab40 2023-10-21T16:22:05.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.147+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17a40b0ef0 con 0x7f17a415ab40 2023-10-21T16:22:05.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.149+0000 7f179bfff640 1 -- 172.21.15.53:0/2887480863 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+14383 (secure 0 0 0) 0x7f179c056f20 con 0x7f17a415ab40 2023-10-21T16:22:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:05.151 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":153,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:22:02.669156+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/582349146":"2023-10-22T16:21:59.527235+0000","172.21.15.124:6836/582349146":"2023-10-22T16:21:59.527235+0000","172.21.15.124:6833/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3267096500":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1834008746":"2023-10-22T16:21:55.600213+0000","172.21.15.124:6832/1829772531":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1781939354":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/1295722096":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2527601840":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/3356731651":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/2276161522":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/4259428584":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6833/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:6832/227410594":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/907346555":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/3504147713":"2023-10-22T16:21:55.600213+0000","172.21.15.124:0/667593812":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2229194042":"2023-10-22T16:21:42.595479+0000","172.21.15.124:0/2878176284":"2023-10-22T16:21:42.595479+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:22:05.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 msgr2=0x7f178406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 0x7f178406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1794035000 tx=0x7f1794030000 comp rx=0 tx=0).stop 2023-10-21T16:22:05.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 msgr2=0x7f17a40c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c390 tx=0x7f179c00c850 comp rx=0 tx=0).stop 2023-10-21T16:22:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 shutdown_connections 2023-10-21T16:22:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f178406af90 0x7f178406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17a415ab40 0x7f17a40c5850 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17a40cae70 0x7f17a415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 --2- 172.21.15.53:0/2887480863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17a40c90b0 0x7f17a40c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 >> 172.21.15.53:0/2887480863 conn(0x7f17a406a380 msgr2=0x7f17a406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 shutdown_connections 2023-10-21T16:22:05.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.152+0000 7f17a9b97640 1 -- 172.21.15.53:0/2887480863 wait complete. 2023-10-21T16:22:05.172 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:22:05.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:22:05.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.405+0000 7f6555def640 1 Processor -- start 2023-10-21T16:22:05.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.405+0000 7f6555def640 1 -- start start 2023-10-21T16:22:05.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.406+0000 7f6555def640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f655015aab0 0x7f6550070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f6555def640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f65500cb350 0x7f65500b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f6555def640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655015ae90 con 0x7f65500caef0 2023-10-21T16:22:05.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f6555def640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500cbdb0 con 0x7f65500cb350 2023-10-21T16:22:05.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f6555def640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500cb7f0 con 0x7f655015aab0 2023-10-21T16:22:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f6554ded640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f655015aab0 0x7f6550070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f65555ee640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f65500cb350 0x7f65500b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f65555ee640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f65500cb350 0x7f65500b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:60326/0 (socket says 172.21.15.53:60326) 2023-10-21T16:22:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.407+0000 7f65555ee640 1 -- 172.21.15.53:0/2691016284 learned_addr learned my addr 172.21.15.53:0/2691016284 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:05.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.408+0000 7f65555ee640 1 -- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f655015aab0 msgr2=0x7f6550070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:05.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.408+0000 7f65555ee640 1 --2- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f655015aab0 0x7f6550070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.408+0000 7f65555ee640 1 -- 172.21.15.53:0/2691016284 >> v1:172.21.15.124:6789/0 conn(0x7f65500caef0 legacy=0x7f65500aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.408+0000 7f65555ee640 1 -- 172.21.15.53:0/2691016284 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500cbad0 con 0x7f65500cb350 2023-10-21T16:22:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.410+0000 7f65555ee640 1 --2- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65500cb350 0x7f65500b1c60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6548009870 tx=0x7f654802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b153686b708df4a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.410+0000 7f654f7fe640 1 -- 172.21.15.53:0/2691016284 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548031640 con 0x7f65500cb350 2023-10-21T16:22:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.410+0000 7f654f7fe640 1 -- 172.21.15.53:0/2691016284 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65480317a0 con 0x7f65500cb350 2023-10-21T16:22:05.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.410+0000 7f654f7fe640 1 -- 172.21.15.53:0/2691016284 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548031b30 con 0x7f65500cb350 2023-10-21T16:22:05.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 -- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65500cb350 msgr2=0x7f65500b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 --2- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65500cb350 0x7f65500b1c60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6548009870 tx=0x7f654802b8f0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 -- 172.21.15.53:0/2691016284 shutdown_connections 2023-10-21T16:22:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 --2- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65500cb350 0x7f65500b1c60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 --2- 172.21.15.53:0/2691016284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f655015aab0 0x7f6550070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 -- 172.21.15.53:0/2691016284 >> 172.21.15.53:0/2691016284 conn(0x7f655006c010 msgr2=0x7f655006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.411+0000 7f6555def640 1 -- 172.21.15.53:0/2691016284 shutdown_connections 2023-10-21T16:22:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.412+0000 7f6555def640 1 -- 172.21.15.53:0/2691016284 wait complete. 2023-10-21T16:22:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.414+0000 7f6555def640 1 Processor -- start 2023-10-21T16:22:05.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.414+0000 7f6555def640 1 -- start start 2023-10-21T16:22:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65500caef0 0x7f655014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f655015aab0 0x7f65500c2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655014e1f0 con 0x7f65500cb350 2023-10-21T16:22:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655014e4b0 con 0x7f655015aab0 2023-10-21T16:22:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.415+0000 7f6555def640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655014e790 con 0x7f65500caef0 2023-10-21T16:22:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48222/0 (socket says 172.21.15.53:48222) 2023-10-21T16:22:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 -- 172.21.15.53:0/3711862632 learned_addr learned my addr 172.21.15.53:0/3711862632 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f65555ee640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f655015aab0 0x7f65500c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f6554ded640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65500caef0 0x7f655014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 -- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65500caef0 msgr2=0x7f655014f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65500caef0 0x7f655014f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 -- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f655015aab0 msgr2=0x7f65500c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f655015aab0 0x7f65500c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.416+0000 7f654ffff640 1 -- 172.21.15.53:0/3711862632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480094d0 con 0x7f65500cb350 2023-10-21T16:22:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f654ffff640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f654400c400 tx=0x7f654400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440052d0 con 0x7f65500cb350 2023-10-21T16:22:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544005430 con 0x7f65500cb350 2023-10-21T16:22:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544003830 con 0x7f65500cb350 2023-10-21T16:22:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655014ea70 con 0x7f65500cb350 2023-10-21T16:22:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.417+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655014f7b0 con 0x7f65500cb350 2023-10-21T16:22:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.418+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655014ef90 con 0x7f65500cb350 2023-10-21T16:22:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.420+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6544003990 con 0x7f65500cb350 2023-10-21T16:22:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.420+0000 7f654d7fa640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.421+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6544088dd0 con 0x7f65500cb350 2023-10-21T16:22:05.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.421+0000 7f6554ded640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.422+0000 7f6554ded640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 0x7f653c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65400074a0 tx=0x7f654000b040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.425+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f655014ef90 con 0x7f65500cb350 2023-10-21T16:22:05.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.619+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f655014ef90 con 0x7f65500cb350 2023-10-21T16:22:05.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.620+0000 7f654d7fa640 1 -- 172.21.15.53:0/3711862632 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f655014ef90 con 0x7f65500cb350 2023-10-21T16:22:05.622 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:22:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 msgr2=0x7f653c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 0x7f653c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65400074a0 tx=0x7f654000b040 comp rx=0 tx=0).stop 2023-10-21T16:22:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 msgr2=0x7f65500af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f654400c400 tx=0x7f654400c8c0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 shutdown_connections 2023-10-21T16:22:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f655015aab0 0x7f65500c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65500cb350 0x7f65500af510 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 --2- 172.21.15.53:0/3711862632 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65500caef0 0x7f655014f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 >> 172.21.15.53:0/3711862632 conn(0x7f655006c010 msgr2=0x7f6550150220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 shutdown_connections 2023-10-21T16:22:05.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.623+0000 7f6555def640 1 -- 172.21.15.53:0/3711862632 wait complete. 2023-10-21T16:22:05.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-21T16:22:05.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.865+0000 7febb740e640 1 Processor -- start 2023-10-21T16:22:05.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.866+0000 7febb740e640 1 -- start start 2023-10-21T16:22:05.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.867+0000 7febb740e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7febb00c90b0 0x7febb00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.867+0000 7febb740e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7febb015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.867+0000 7febb740e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb015b2b0 con 0x7febb015ab40 2023-10-21T16:22:05.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.867+0000 7febb740e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb015bf30 con 0x7febb00c90b0 2023-10-21T16:22:05.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.867+0000 7febb740e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00cb820 con 0x7febb00cb3b0 2023-10-21T16:22:05.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7febb51aa640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7febb015ab40 0x7febb00ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40586/0 (socket says 172.21.15.53:40586) 2023-10-21T16:22:05.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7febb51aa640 1 -- 172.21.15.53:0/4130330342 learned_addr learned my addr 172.21.15.53:0/4130330342 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7febb59ab640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7febb015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107144847 0 0) 0x7febb015b2b0 con 0x7febb015ab40 2023-10-21T16:22:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb9c003520 con 0x7febb015ab40 2023-10-21T16:22:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.868+0000 7febb49a9640 1 --2- 172.21.15.53:0/4130330342 >> v2:172.21.15.124:3300/0 conn(0x7febb00c90b0 0x7febb00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 227979695 0 0) 0x7feb9c003520 con 0x7febb015ab40 2023-10-21T16:22:05.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 msgr2=0x7febb015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:05.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7febb015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 >> v2:172.21.15.124:3300/0 conn(0x7febb00c90b0 msgr2=0x7febb00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:05.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 --2- 172.21.15.53:0/4130330342 >> v2:172.21.15.124:3300/0 conn(0x7febb00c90b0 0x7febb00cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb015b590 con 0x7febb015ab40 2023-10-21T16:22:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7febac001a10 con 0x7febb015ab40 2023-10-21T16:22:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 >> v1:172.21.15.53:6789/0 conn(0x7febb015ab40 legacy=0x7febb00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb9c0039a0 0x7feb9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9c0047c0 0x7feb9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7feb9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb015bf30 con 0x7feb9c0039a0 2023-10-21T16:22:05.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00cb820 con 0x7feb9c0047c0 2023-10-21T16:22:05.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c00b980 con 0x7febb00cb3b0 2023-10-21T16:22:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.869+0000 7febb51aa640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb9c0039a0 0x7feb9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb49a9640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9c0047c0 0x7feb9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb59ab640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7feb9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb51aa640 1 -- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 msgr2=0x7feb9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb51aa640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7febb00cb3b0 0x7feb9c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb51aa640 1 -- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9c0047c0 msgr2=0x7feb9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:05.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb51aa640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9c0047c0 0x7feb9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.870+0000 7febb51aa640 1 -- 172.21.15.53:0/4130330342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c003520 con 0x7feb9c0039a0 2023-10-21T16:22:05.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.871+0000 7febb51aa640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb9c0039a0 0x7feb9c003dc0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7febac00a810 tx=0x7feb9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd43484a2373d45d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.871+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac034630 con 0x7feb9c0039a0 2023-10-21T16:22:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.871+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac02f070 con 0x7feb9c0039a0 2023-10-21T16:22:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.871+0000 7feba7fff640 1 -- 172.21.15.53:0/4130330342 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0349d0 con 0x7feb9c0039a0 2023-10-21T16:22:05.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 -- 172.21.15.53:0/4130330342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb9c0039a0 msgr2=0x7feb9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 --2- 172.21.15.53:0/4130330342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb9c0039a0 0x7feb9c003dc0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7febac00a810 tx=0x7feb9c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb59ab640 1 -- 172.21.15.53:0/4130330342 reap_dead start 2023-10-21T16:22:05.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 -- 172.21.15.53:0/4130330342 shutdown_connections 2023-10-21T16:22:05.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 -- 172.21.15.53:0/4130330342 >> 172.21.15.53:0/4130330342 conn(0x7febb006a380 msgr2=0x7febb00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 -- 172.21.15.53:0/4130330342 shutdown_connections 2023-10-21T16:22:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.872+0000 7febb740e640 1 -- 172.21.15.53:0/4130330342 wait complete. 2023-10-21T16:22:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.874+0000 7febb740e640 1 Processor -- start 2023-10-21T16:22:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.874+0000 7febb740e640 1 -- start start 2023-10-21T16:22:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9c0039a0 0x7febb006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febb00c6390 0x7febb006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb01f0540 con 0x7febb00c6810 2023-10-21T16:22:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c58f0 con 0x7febb00c6390 2023-10-21T16:22:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb740e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c80c0 con 0x7feb9c0039a0 2023-10-21T16:22:05.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb59ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb59ab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48242/0 (socket says 172.21.15.53:48242) 2023-10-21T16:22:05.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.875+0000 7febb59ab640 1 -- 172.21.15.53:0/381456297 learned_addr learned my addr 172.21.15.53:0/381456297 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:05.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb51aa640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9c0039a0 0x7febb006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 -- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9c0039a0 msgr2=0x7febb006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9c0039a0 0x7febb006d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 -- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febb00c6390 msgr2=0x7febb006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb49a9640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febb00c6390 0x7febb006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febb00c6390 0x7febb006c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:05.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 -- 172.21.15.53:0/381456297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb00c5b90 con 0x7febb00c6810 2023-10-21T16:22:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7febb59ab640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7febb00b0c20 tx=0x7feba8010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800e8a0 con 0x7febb00c6810 2023-10-21T16:22:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800ea20 con 0x7febb00c6810 2023-10-21T16:22:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.876+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800edd0 con 0x7febb00c6810 2023-10-21T16:22:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.877+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb00c5e70 con 0x7febb00c6810 2023-10-21T16:22:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.877+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00b33e0 con 0x7febb00c6810 2023-10-21T16:22:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.878+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb00c6060 con 0x7febb00c6810 2023-10-21T16:22:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.879+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feba8020410 con 0x7febb00c6810 2023-10-21T16:22:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.881+0000 7feba5ffb640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 0x7feb9406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.881+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7feba8089240 con 0x7febb00c6810 2023-10-21T16:22:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.882+0000 7febb51aa640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 0x7feb9406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.883+0000 7febb51aa640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 0x7feb9406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac02f000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:05.889+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba8054080 con 0x7febb00c6810 2023-10-21T16:22:06.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.150+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7febb015bb20 con 0x7febb00c6810 2023-10-21T16:22:06.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.698+0000 7feba5ffb640 1 -- 172.21.15.53:0/381456297 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v797) v1 ==== 181+0+0 (secure 0 0 0) 0x7feba80576a0 con 0x7febb00c6810 2023-10-21T16:22:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 msgr2=0x7feb9406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 0x7feb9406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febac034000 tx=0x7febac02f000 comp rx=0 tx=0).stop 2023-10-21T16:22:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 msgr2=0x7febb006cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7febb00b0c20 tx=0x7feba8010c20 comp rx=0 tx=0).stop 2023-10-21T16:22:06.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 shutdown_connections 2023-10-21T16:22:06.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7feb9406afc0 0x7feb9406d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7febb00c6810 0x7febb006cdb0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7febb00c6390 0x7febb006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 --2- 172.21.15.53:0/381456297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9c0039a0 0x7febb006d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 >> 172.21.15.53:0/381456297 conn(0x7febb006a380 msgr2=0x7febb00ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:06.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 shutdown_connections 2023-10-21T16:22:06.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.702+0000 7febb740e640 1 -- 172.21.15.53:0/381456297 wait complete. 2023-10-21T16:22:06.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T16:22:06.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.936+0000 7f1023535640 1 Processor -- start 2023-10-21T16:22:06.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.936+0000 7f1023535640 1 -- start start 2023-10-21T16:22:06.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1023535640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c15a820 0x7f101c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1023535640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c0ca680 0x7f101c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1023535640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c0cab60 con 0x7f101c15ac00 2023-10-21T16:22:06.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1023535640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c0cb960 con 0x7f101c15a820 2023-10-21T16:22:06.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1023535640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0cbdf0 con 0x7f101c0ca680 2023-10-21T16:22:06.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.938+0000 7f1022533640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c15a820 0x7f101c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f101c15a820 0x7f101c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48258/0 (socket says 172.21.15.53:48258) 2023-10-21T16:22:06.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 -- 172.21.15.53:0/4028073213 learned_addr learned my addr 172.21.15.53:0/4028073213 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:06.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1021d32640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c0ca680 0x7f101c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 -- 172.21.15.53:0/4028073213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c0ca680 msgr2=0x7f101c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:06.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c0ca680 0x7f101c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 -- 172.21.15.53:0/4028073213 >> v1:172.21.15.124:6789/0 conn(0x7f101c15ac00 legacy=0x7f101c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:06.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.939+0000 7f1022533640 1 -- 172.21.15.53:0/4028073213 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0cb030 con 0x7f101c15a820 2023-10-21T16:22:06.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.941+0000 7f1022533640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15a820 0x7f101c0aff70 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1018009870 tx=0x7f101802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b6f3ccefa51bae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.941+0000 7f1021531640 1 -- 172.21.15.53:0/4028073213 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180317c0 con 0x7f101c15a820 2023-10-21T16:22:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.941+0000 7f1021531640 1 -- 172.21.15.53:0/4028073213 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018031920 con 0x7f101c15a820 2023-10-21T16:22:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.941+0000 7f1021531640 1 -- 172.21.15.53:0/4028073213 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018031cb0 con 0x7f101c15a820 2023-10-21T16:22:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 -- 172.21.15.53:0/4028073213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15a820 msgr2=0x7f101c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15a820 0x7f101c0aff70 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1018009870 tx=0x7f101802b860 comp rx=0 tx=0).stop 2023-10-21T16:22:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 -- 172.21.15.53:0/4028073213 shutdown_connections 2023-10-21T16:22:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c0ca680 0x7f101c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 --2- 172.21.15.53:0/4028073213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15a820 0x7f101c0aff70 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 -- 172.21.15.53:0/4028073213 >> 172.21.15.53:0/4028073213 conn(0x7f101c06c050 msgr2=0x7f101c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:06.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 -- 172.21.15.53:0/4028073213 shutdown_connections 2023-10-21T16:22:06.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.942+0000 7f1023535640 1 -- 172.21.15.53:0/4028073213 wait complete. 2023-10-21T16:22:06.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.944+0000 7f1023535640 1 Processor -- start 2023-10-21T16:22:06.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.945+0000 7f1023535640 1 -- start start 2023-10-21T16:22:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 0x7f101c0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c15a820 0x7f101c15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0afcb0 con 0x7f101c15ac00 2023-10-21T16:22:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0aef20 con 0x7f101c0ca680 2023-10-21T16:22:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1023535640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0c2e10 con 0x7f101c15a820 2023-10-21T16:22:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1022d34640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1022d34640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48272/0 (socket says 172.21.15.53:48272) 2023-10-21T16:22:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.946+0000 7f1022d34640 1 -- 172.21.15.53:0/4057037122 learned_addr learned my addr 172.21.15.53:0/4057037122 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022533640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 0x7f101c0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1021d32640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c15a820 0x7f101c15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 -- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c15a820 msgr2=0x7f101c15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c15a820 0x7f101c15d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 -- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 msgr2=0x7f101c0ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 0x7f101c0ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 -- 172.21.15.53:0/4057037122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10180094d0 con 0x7f101c15ac00 2023-10-21T16:22:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022533640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 0x7f101c0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:22:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.947+0000 7f1022d34640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f101400c6d0 tx=0x7f101400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.948+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400a840 con 0x7f101c15ac00 2023-10-21T16:22:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.948+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400a9a0 con 0x7f101c15ac00 2023-10-21T16:22:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.948+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400ad90 con 0x7f101c15ac00 2023-10-21T16:22:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.948+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c0cdbf0 con 0x7f101c15ac00 2023-10-21T16:22:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.948+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c0af410 con 0x7f101c15ac00 2023-10-21T16:22:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.949+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c0c3010 con 0x7f101c15ac00 2023-10-21T16:22:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.950+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1014012630 con 0x7f101c15ac00 2023-10-21T16:22:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.951+0000 7f1012ffd640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 0x7f100006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.951+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1014089f40 con 0x7f101c15ac00 2023-10-21T16:22:06.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.951+0000 7f1022533640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 0x7f100006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:06.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.953+0000 7f1022533640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 0x7f100006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f101803d040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:06.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:06.955+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f101c0c3010 con 0x7f101c15ac00 2023-10-21T16:22:07.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.150+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f101c0b1310 con 0x7f101c15ac00 2023-10-21T16:22:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.157+0000 7f1012ffd640 1 -- 172.21.15.53:0/4057037122 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1014016070 con 0x7f101c15ac00 2023-10-21T16:22:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.159+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 msgr2=0x7f100006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.159+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 0x7f100006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f101803d040 comp rx=0 tx=0).stop 2023-10-21T16:22:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.159+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 msgr2=0x7f101c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.159+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f101400c6d0 tx=0x7f101400cb90 comp rx=0 tx=0).stop 2023-10-21T16:22:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 shutdown_connections 2023-10-21T16:22:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f100006af90 0x7f100006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f101c15ac00 0x7f101c15d950 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f101c15a820 0x7f101c15d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 --2- 172.21.15.53:0/4057037122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f101c0ca680 0x7f101c0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 >> 172.21.15.53:0/4057037122 conn(0x7f101c06c050 msgr2=0x7f101c0ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 shutdown_connections 2023-10-21T16:22:07.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.160+0000 7f1023535640 1 -- 172.21.15.53:0/4057037122 wait complete. 2023-10-21T16:22:07.181 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T16:22:07.181 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:22:07.307 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T16:22:07.431 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T16:22:07.432 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:22:07.432 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:22:07.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:22:07.499 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:22:07.500 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:22:07.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:22:07.561 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:22:07.562 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:22:07.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T16:22:07.628 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T16:22:07.628 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:22:07.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T16:22:07.688 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T16:22:07.689 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:22:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.924+0000 7f02e5fff640 1 Processor -- start 2023-10-21T16:22:07.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.924+0000 7f02e5fff640 1 -- start start 2023-10-21T16:22:07.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02e5fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f02e00b07d0 0x7f02e01648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02e5fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00cacf0 con 0x7f02e00cc210 2023-10-21T16:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02e5fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00ca3c0 con 0x7f02e01599d0 2023-10-21T16:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02e5fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e00b10f0 con 0x7f02e00b07d0 2023-10-21T16:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02dffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f02e00cc210 0x7f02e00b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40600/0 (socket says 172.21.15.53:40600) 2023-10-21T16:22:07.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02dffff640 1 -- 172.21.15.53:0/481045087 learned_addr learned my addr 172.21.15.53:0/481045087 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:07.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02e57fe640 1 --2- 172.21.15.53:0/481045087 >> v2:172.21.15.124:3300/0 conn(0x7f02e00b07d0 0x7f02e01648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594985355 0 0) 0x7f02e00cacf0 con 0x7f02e00cc210 2023-10-21T16:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.926+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d0003520 con 0x7f02e00cc210 2023-10-21T16:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3103536295 0 0) 0x7f02d0003520 con 0x7f02e00cc210 2023-10-21T16:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 >> v2:172.21.15.124:3300/0 conn(0x7f02e00b07d0 msgr2=0x7f02e01648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:07.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 --2- 172.21.15.53:0/481045087 >> v2:172.21.15.124:3300/0 conn(0x7f02e00b07d0 0x7f02e01648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 >> v1:172.21.15.124:6789/0 conn(0x7f02e01599d0 legacy=0x7f02e00ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e00ca6a0 con 0x7f02e00cc210 2023-10-21T16:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f02cc001670 con 0x7f02e00cc210 2023-10-21T16:22:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 >> v1:172.21.15.53:6789/0 conn(0x7f02e00cc210 legacy=0x7f02e00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 0x7f02d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0005180 0x7f02d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b10f0 con 0x7f02d0003970 2023-10-21T16:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c020 con 0x7f02d00047c0 2023-10-21T16:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c1e0 con 0x7f02d0005180 2023-10-21T16:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.927+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 0x7f02d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02dffff640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e57fe640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0005180 0x7f02d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e4ffd640 1 -- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0005180 msgr2=0x7f02d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0005180 0x7f02d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e4ffd640 1 -- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02d00047c0 msgr2=0x7f02d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e4ffd640 1 -- 172.21.15.53:0/481045087 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0003520 con 0x7f02d0003970 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.928+0000 7f02e57fe640 1 -- 172.21.15.53:0/481045087 reap_dead start 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.929+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 0x7f02d0003d90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f02d4001750 tx=0x7f02d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80f52476b8c1f57 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.929+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d40052d0 con 0x7f02d0003970 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.929+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d402e050 con 0x7f02d0003970 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.929+0000 7f02df7fe640 1 -- 172.21.15.53:0/481045087 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4003830 con 0x7f02d0003970 2023-10-21T16:22:07.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 -- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 msgr2=0x7f02d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 0x7f02d0003d90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f02d4001750 tx=0x7f02d00057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 -- 172.21.15.53:0/481045087 shutdown_connections 2023-10-21T16:22:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 --2- 172.21.15.53:0/481045087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02d0003970 0x7f02d0003d90 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 -- 172.21.15.53:0/481045087 >> 172.21.15.53:0/481045087 conn(0x7f02e00cb0c0 msgr2=0x7f02e00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 -- 172.21.15.53:0/481045087 shutdown_connections 2023-10-21T16:22:07.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.930+0000 7f02e5fff640 1 -- 172.21.15.53:0/481045087 wait complete. 2023-10-21T16:22:07.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.932+0000 7f02e5fff640 1 Processor -- start 2023-10-21T16:22:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.932+0000 7f02e5fff640 1 -- start start 2023-10-21T16:22:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0003970 0x7f02e0157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02e01585e0 0x7f02e0158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e015b4b0 con 0x7f02e00c83b0 2023-10-21T16:22:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e015b8d0 con 0x7f02e01585e0 2023-10-21T16:22:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02e5fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0158fa0 con 0x7f02d0003970 2023-10-21T16:22:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48290/0 (socket says 172.21.15.53:48290) 2023-10-21T16:22:07.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.933+0000 7f02dffff640 1 -- 172.21.15.53:0/1016586307 learned_addr learned my addr 172.21.15.53:0/1016586307 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:07.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0003970 0x7f02e0157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02e57fe640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02e01585e0 0x7f02e0158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 -- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0003970 msgr2=0x7f02e0157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0003970 0x7f02e0157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 -- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02e01585e0 msgr2=0x7f02e0158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02e01585e0 0x7f02e0158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 -- 172.21.15.53:0/1016586307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e015aba0 con 0x7f02e00c83b0 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dffff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e780 tx=0x7f02cc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00c660 con 0x7f02e00c83b0 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc00c7c0 con 0x7f02e00c83b0 2023-10-21T16:22:07.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00cb70 con 0x7f02e00c83b0 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.934+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e015a6b0 con 0x7f02e00c83b0 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.935+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e015b0d0 con 0x7f02e00c83b0 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.936+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e004e160 con 0x7f02e00c83b0 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.937+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f02cc00ccd0 con 0x7f02e00c83b0 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.938+0000 7f02dd7fa640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 0x7f02c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:07.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.938+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f02cc0893d0 con 0x7f02e00c83b0 2023-10-21T16:22:07.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.938+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 0x7f02c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:07.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.940+0000 7f02e4ffd640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 0x7f02c406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02d4033000 tx=0x7f02d4009030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:07.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:07.942+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02cc0541f0 con 0x7f02e00c83b0 2023-10-21T16:22:08.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.174+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02e0158100 con 0x7f02e00c83b0 2023-10-21T16:22:08.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.175+0000 7f02dd7fa640 1 -- 172.21.15.53:0/1016586307 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f02cc057810 con 0x7f02e00c83b0 2023-10-21T16:22:08.177 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:08.177 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8550,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1809563676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1809563676},{"type":"v1","addr":"172.21.15.53:6835","nonce":1809563676}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36458,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1842396496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1842396496},{"type":"v1","addr":"172.21.15.53:6834","nonce":1842396496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36470,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/678588991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":678588991},{"type":"v1","addr":"172.21.15.124:6835","nonce":678588991}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":36710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3846994086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3846994086},{"type":"v1","addr":"172.21.15.124:6837","nonce":3846994086}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[]} 2023-10-21T16:22:08.177 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 71 2023-10-21T16:22:08.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.177+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 msgr2=0x7f02c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.177+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 0x7f02c406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02d4033000 tx=0x7f02d4009030 comp rx=0 tx=0).stop 2023-10-21T16:22:08.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.177+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 msgr2=0x7f02e0159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.177+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e780 tx=0x7f02cc00ec40 comp rx=0 tx=0).stop 2023-10-21T16:22:08.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 shutdown_connections 2023-10-21T16:22:08.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f02c406afc0 0x7f02c406d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f02e01585e0 0x7f02e0158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02e00c83b0 0x7f02e0159db0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 --2- 172.21.15.53:0/1016586307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02d0003970 0x7f02e0157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 >> 172.21.15.53:0/1016586307 conn(0x7f02e00cb0c0 msgr2=0x7f02e00af380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:08.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 shutdown_connections 2023-10-21T16:22:08.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.178+0000 7f02e5fff640 1 -- 172.21.15.53:0/1016586307 wait complete. 2023-10-21T16:22:08.199 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T16:22:08.200 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T16:22:08.200 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T16:22:08.200 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:22:08.201 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T16:22:08.201 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T16:22:08.201 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T16:22:08.202 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:22:08.202 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T16:22:08.202 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T16:22:08.203 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T16:22:08.203 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:22:08.203 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T16:22:08.204 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T16:22:08.204 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T16:22:08.204 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T16:22:08.221 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T16:22:08.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T16:22:08.223 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T16:22:08.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T16:22:08.226 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T16:22:08.226 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T16:22:08.228 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T16:22:08.228 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T16:22:08.270 INFO:tasks.ceph.mds.c:Started 2023-10-21T16:22:08.271 DEBUG:teuthology.parallel:result is None 2023-10-21T16:22:08.275 INFO:tasks.ceph.mds.d:Started 2023-10-21T16:22:08.276 DEBUG:teuthology.parallel:result is None 2023-10-21T16:22:08.277 INFO:tasks.ceph.mds.a:Started 2023-10-21T16:22:08.277 DEBUG:teuthology.parallel:result is None 2023-10-21T16:22:08.283 INFO:tasks.ceph.mds.b:Started 2023-10-21T16:22:08.284 DEBUG:teuthology.parallel:result is None 2023-10-21T16:22:08.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T16:22:08.442 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T16:22:08.443 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T16:22:08.443 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T16:22:08.443 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T16:22:08.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.477+0000 7f5d50f3c640 1 Processor -- start 2023-10-21T16:22:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.477+0000 7f5d50f3c640 1 -- start start 2023-10-21T16:22:08.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.478+0000 7f5d50f3c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d4c15e400 0x7f5d4c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.478+0000 7f5d50f3c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d4c15b4b0 0x7f5d4c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.478+0000 7f5d50f3c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0b63e0 con 0x7f5d4c1585a0 2023-10-21T16:22:08.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.478+0000 7f5d50f3c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c15be00 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.478+0000 7f5d50f3c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c15e8b0 con 0x7f5d4c15e400 2023-10-21T16:22:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d4c15b4b0 0x7f5d4c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5d4c1585a0 0x7f5d4c0ce2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40618/0 (socket says 172.21.15.53:40618) 2023-10-21T16:22:08.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4b7fe640 1 -- 172.21.15.53:0/1685589728 learned_addr learned my addr 172.21.15.53:0/1685589728 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:08.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4affd640 1 --2- 172.21.15.53:0/1685589728 >> v2:172.21.15.124:3300/0 conn(0x7f5d4c15e400 0x7f5d4c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4bfff640 1 -- 172.21.15.53:0/1685589728 >> v2:172.21.15.124:3300/0 conn(0x7f5d4c15e400 msgr2=0x7f5d4c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:08.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4bfff640 1 --2- 172.21.15.53:0/1685589728 >> v2:172.21.15.124:3300/0 conn(0x7f5d4c15e400 0x7f5d4c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4bfff640 1 -- 172.21.15.53:0/1685589728 >> v1:172.21.15.53:6789/0 conn(0x7f5d4c1585a0 legacy=0x7f5d4c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4bfff640 1 -- 172.21.15.53:0/1685589728 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c15b020 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.479+0000 7f5d4a7fc640 1 -- 172.21.15.53:0/1685589728 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61207925 0 0) 0x7f5d4c0b63e0 con 0x7f5d4c1585a0 2023-10-21T16:22:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.480+0000 7f5d4bfff640 1 --2- 172.21.15.53:0/1685589728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15b8c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0afde0 tx=0x7f5d4402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f345bc7501c0d04 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d4a7fc640 1 -- 172.21.15.53:0/1685589728 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4402cc70 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d4a7fc640 1 -- 172.21.15.53:0/1685589728 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4402cdd0 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d4a7fc640 1 -- 172.21.15.53:0/1685589728 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44034660 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 -- 172.21.15.53:0/1685589728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c15b4b0 msgr2=0x7f5d4c15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/1685589728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15b8c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0afde0 tx=0x7f5d4402bec0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 -- 172.21.15.53:0/1685589728 shutdown_connections 2023-10-21T16:22:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/1685589728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15b8c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/1685589728 >> v2:172.21.15.124:3300/0 conn(0x7f5d4c15e400 0x7f5d4c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 -- 172.21.15.53:0/1685589728 >> 172.21.15.53:0/1685589728 conn(0x7f5d4c0cd180 msgr2=0x7f5d4c0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:08.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 -- 172.21.15.53:0/1685589728 shutdown_connections 2023-10-21T16:22:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.481+0000 7f5d50f3c640 1 -- 172.21.15.53:0/1685589728 wait complete. 2023-10-21T16:22:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.483+0000 7f5d50f3c640 1 Processor -- start 2023-10-21T16:22:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.483+0000 7f5d50f3c640 1 -- start start 2023-10-21T16:22:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 0x7f5d4c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0b4c40 con 0x7f5d4c1585a0 2023-10-21T16:22:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0b3660 con 0x7f5d4c15b4b0 2023-10-21T16:22:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d50f3c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0b3940 con 0x7f5d4c15e400 2023-10-21T16:22:08.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.484+0000 7f5d4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48328/0 (socket says 172.21.15.53:48328) 2023-10-21T16:22:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 -- 172.21.15.53:0/327595790 learned_addr learned my addr 172.21.15.53:0/327595790 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4bfff640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 0x7f5d4c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4affd640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 -- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 msgr2=0x7f5d4c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 0x7f5d4c150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 -- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 msgr2=0x7f5d4c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 -- 172.21.15.53:0/327595790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44009a70 con 0x7f5d4c1585a0 2023-10-21T16:22:08.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4affd640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4bfff640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 0x7f5d4c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:22:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.485+0000 7f5d4b7fe640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c4d0 tx=0x7f5d3c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.486+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0074e0 con 0x7f5d4c1585a0 2023-10-21T16:22:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.486+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c007660 con 0x7f5d4c1585a0 2023-10-21T16:22:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.486+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004a20 con 0x7f5d4c1585a0 2023-10-21T16:22:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.486+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c0b3c20 con 0x7f5d4c1585a0 2023-10-21T16:22:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.486+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c0b41e0 con 0x7f5d4c1585a0 2023-10-21T16:22:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.487+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c074150 con 0x7f5d4c1585a0 2023-10-21T16:22:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.488+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d3c004b80 con 0x7f5d4c1585a0 2023-10-21T16:22:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.489+0000 7f5d33fff640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 0x7f5d3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.489+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5d3c088b30 con 0x7f5d4c1585a0 2023-10-21T16:22:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.489+0000 7f5d4affd640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 0x7f5d3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.490+0000 7f5d4affd640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 0x7f5d3806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.493+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d3c0538c0 con 0x7f5d4c1585a0 2023-10-21T16:22:08.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:08.700+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5d4c15a6e0 con 0x7f5d4c1585a0 2023-10-21T16:22:09.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.181+0000 7f5d33fff640 1 -- 172.21.15.53:0/327595790 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v155) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d3c018020 con 0x7f5d4c1585a0 2023-10-21T16:22:09.183 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T16:22:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 msgr2=0x7f5d3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 0x7f5d3806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2023-10-21T16:22:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 msgr2=0x7f5d4c0b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c4d0 tx=0x7f5d3c00c990 comp rx=0 tx=0).stop 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 shutdown_connections 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5d3806af90 0x7f5d3806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15e400 0x7f5d4c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c15b4b0 0x7f5d4c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.184+0000 7f5d50f3c640 1 --2- 172.21.15.53:0/327595790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d4c1585a0 0x7f5d4c0b4700 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.185+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 >> 172.21.15.53:0/327595790 conn(0x7f5d4c0cd180 msgr2=0x7f5d4c0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.185+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 shutdown_connections 2023-10-21T16:22:09.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.185+0000 7f5d50f3c640 1 -- 172.21.15.53:0/327595790 wait complete. 2023-10-21T16:22:09.212 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T16:22:09.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.426+0000 7f52e9b27640 1 Processor -- start 2023-10-21T16:22:09.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.426+0000 7f52e9b27640 1 -- start start 2023-10-21T16:22:09.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.428+0000 7f52e9b27640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52e415cca0 0x7f52e41614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.428+0000 7f52e9b27640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e40c9020 con 0x7f52e40cab40 2023-10-21T16:22:09.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.428+0000 7f52e9b27640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e40ccc20 con 0x7f52e40b07d0 2023-10-21T16:22:09.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.428+0000 7f52e9b27640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e40c92e0 con 0x7f52e415cca0 2023-10-21T16:22:09.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e8b25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52e415cca0 0x7f52e41614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f52e40cab40 0x7f52e40b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40634/0 (socket says 172.21.15.53:40634) 2023-10-21T16:22:09.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e8b25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52e415cca0 0x7f52e41614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48344/0 (socket says 172.21.15.53:48344) 2023-10-21T16:22:09.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e3fff640 1 -- 172.21.15.53:0/212298326 learned_addr learned my addr 172.21.15.53:0/212298326 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:09.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e8b25640 1 -- 172.21.15.53:0/212298326 >> v1:172.21.15.124:6789/0 conn(0x7f52e40b07d0 legacy=0x7f52e40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:09.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e8b25640 1 -- 172.21.15.53:0/212298326 >> v1:172.21.15.53:6789/0 conn(0x7f52e40cab40 legacy=0x7f52e40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.429+0000 7f52e8b25640 1 -- 172.21.15.53:0/212298326 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40cc7b0 con 0x7f52e415cca0 2023-10-21T16:22:09.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.430+0000 7f52e8b25640 1 --2- 172.21.15.53:0/212298326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e41614b0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097a0 tx=0x7f52d802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b775785ba48ea7f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:09.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.430+0000 7f52e37fe640 1 -- 172.21.15.53:0/212298326 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d802f050 con 0x7f52e415cca0 2023-10-21T16:22:09.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e37fe640 1 -- 172.21.15.53:0/212298326 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d8031700 con 0x7f52e415cca0 2023-10-21T16:22:09.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e37fe640 1 -- 172.21.15.53:0/212298326 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d8031a90 con 0x7f52e415cca0 2023-10-21T16:22:09.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 -- 172.21.15.53:0/212298326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 msgr2=0x7f52e41614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 --2- 172.21.15.53:0/212298326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e41614b0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097a0 tx=0x7f52d802b7a0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 -- 172.21.15.53:0/212298326 shutdown_connections 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 --2- 172.21.15.53:0/212298326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e41614b0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 -- 172.21.15.53:0/212298326 >> 172.21.15.53:0/212298326 conn(0x7f52e40cb0c0 msgr2=0x7f52e40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 -- 172.21.15.53:0/212298326 shutdown_connections 2023-10-21T16:22:09.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.431+0000 7f52e9b27640 1 -- 172.21.15.53:0/212298326 wait complete. 2023-10-21T16:22:09.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.433+0000 7f52e9b27640 1 Processor -- start 2023-10-21T16:22:09.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.433+0000 7f52e9b27640 1 -- start start 2023-10-21T16:22:09.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52e40b07d0 0x7f52e4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 0x7f52e415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e41f7010 con 0x7f52e415cca0 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e415a020 con 0x7f52e40cab40 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9b27640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e41f6d30 con 0x7f52e40b07d0 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.434+0000 7f52e9326640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48346/0 (socket says 172.21.15.53:48346) 2023-10-21T16:22:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 -- 172.21.15.53:0/3713984812 learned_addr learned my addr 172.21.15.53:0/3713984812 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e8b25640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52e40b07d0 0x7f52e4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e3fff640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 0x7f52e415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 -- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52e40b07d0 msgr2=0x7f52e4159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52e40b07d0 0x7f52e4159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 -- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 msgr2=0x7f52e415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 0x7f52e415e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 -- 172.21.15.53:0/3713984812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d8009400 con 0x7f52e415cca0 2023-10-21T16:22:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e3fff640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 0x7f52e415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:09.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.435+0000 7f52e9326640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b970 tx=0x7f52dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.436+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00cb40 con 0x7f52e415cca0 2023-10-21T16:22:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.436+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc00cca0 con 0x7f52e415cca0 2023-10-21T16:22:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.436+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc014680 con 0x7f52e415cca0 2023-10-21T16:22:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.436+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e40c5820 con 0x7f52e415cca0 2023-10-21T16:22:09.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.436+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e40c5de0 con 0x7f52e415cca0 2023-10-21T16:22:09.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.437+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e40c6000 con 0x7f52e415cca0 2023-10-21T16:22:09.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.438+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f52dc01d3b0 con 0x7f52e415cca0 2023-10-21T16:22:09.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.439+0000 7f52e17fa640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 0x7f52c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.439+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f52dc089460 con 0x7f52e415cca0 2023-10-21T16:22:09.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.440+0000 7f52e8b25640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 0x7f52c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.441+0000 7f52e8b25640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 0x7f52c806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52d8035000 tx=0x7f52d8008030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:09.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.443+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52dc054520 con 0x7f52e415cca0 2023-10-21T16:22:09.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.705+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f52e404e160 con 0x7f52e415cca0 2023-10-21T16:22:09.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.706+0000 7f52e17fa640 1 -- 172.21.15.53:0/3713984812 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f52dc057b40 con 0x7f52e415cca0 2023-10-21T16:22:09.708 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:09.708 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:22:09.708 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T16:22:09.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.708 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T16:22:09.709 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T16:22:09.709 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.709 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.709 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:22:09.709 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:22:09.710 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.710 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:22:09.711 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T16:22:09.712 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.713 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.713 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:22:09.713 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:22:09.713 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.714 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T16:22:09.714 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T16:22:09.714 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.716 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T16:22:09.717 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T16:22:09.718 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.718 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.718 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.718 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.719 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.720 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.722 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T16:22:09.722 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T16:22:09.722 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.723 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.723 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.723 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.725 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.725 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.727 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T16:22:09.727 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T16:22:09.727 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.729 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T16:22:09.729 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T16:22:09.729 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T16:22:09.730 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T16:22:09.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.730 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:22:09.731 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:22:09.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.732 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T16:22:09.732 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:22:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T16:22:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T16:22:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T16:22:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T16:22:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:22:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T16:22:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T16:22:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:22:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:22:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T16:22:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T16:22:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T16:22:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T16:22:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T16:22:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T16:22:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T16:22:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T16:22:09.741 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:22:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.708+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 msgr2=0x7f52c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.708+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 0x7f52c806d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52d8035000 tx=0x7f52d8008030 comp rx=0 tx=0).stop 2023-10-21T16:22:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.708+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 msgr2=0x7f52e40aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.708+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00b970 tx=0x7f52dc00be30 comp rx=0 tx=0).stop 2023-10-21T16:22:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 shutdown_connections 2023-10-21T16:22:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f52c806af90 0x7f52c806d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52e415cca0 0x7f52e40aed20 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52e40cab40 0x7f52e415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 --2- 172.21.15.53:0/3713984812 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52e40b07d0 0x7f52e4159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 >> 172.21.15.53:0/3713984812 conn(0x7f52e40cb0c0 msgr2=0x7f52e40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 shutdown_connections 2023-10-21T16:22:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.709+0000 7f52e9b27640 1 -- 172.21.15.53:0/3713984812 wait complete. 2023-10-21T16:22:09.745 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:22:09.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.977+0000 7f87e0982640 1 Processor -- start 2023-10-21T16:22:09.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.978+0000 7f87e0982640 1 -- start start 2023-10-21T16:22:09.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.979+0000 7f87e0982640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87dc0d16f0 0x7f87dc0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.979+0000 7f87e0982640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc15d040 con 0x7f87dc15cc60 2023-10-21T16:22:09.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.979+0000 7f87e0982640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc0d0650 con 0x7f87dc0cfa50 2023-10-21T16:22:09.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87e0982640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0d0930 con 0x7f87dc0d16f0 2023-10-21T16:22:09.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87dad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f87dc15cc60 0x7f87dc0cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40644/0 (socket says 172.21.15.53:40644) 2023-10-21T16:22:09.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87dad9d640 1 -- 172.21.15.53:0/1724727148 learned_addr learned my addr 172.21.15.53:0/1724727148 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:09.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87da59c640 1 --2- 172.21.15.53:0/1724727148 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0d16f0 0x7f87dc0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591545665 0 0) 0x7f87dc15d040 con 0x7f87dc15cc60 2023-10-21T16:22:09.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.980+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c4003520 con 0x7f87dc15cc60 2023-10-21T16:22:09.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974720297 0 0) 0x7f87c4003520 con 0x7f87dc15cc60 2023-10-21T16:22:09.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0d16f0 msgr2=0x7f87dc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:09.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 --2- 172.21.15.53:0/1724727148 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0d16f0 0x7f87dc0cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 >> v1:172.21.15.124:6789/0 conn(0x7f87dc0cfa50 legacy=0x7f87dc166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0d0c10 con 0x7f87dc15cc60 2023-10-21T16:22:09.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f87cc0017f0 con 0x7f87dc15cc60 2023-10-21T16:22:09.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 >> v1:172.21.15.53:6789/0 conn(0x7f87dc15cc60 legacy=0x7f87dc0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 0x7f87c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0d0930 con 0x7f87c4003970 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400c020 con 0x7f87c40047c0 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400c1e0 con 0x7f87c4005180 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.981+0000 7f87dad9d640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 0x7f87c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87db59e640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87da59c640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87dad9d640 1 -- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4005180 msgr2=0x7f87c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:09.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87dad9d640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87dad9d640 1 -- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c40047c0 msgr2=0x7f87c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:09.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87dad9d640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87dad9d640 1 -- 172.21.15.53:0/1724727148 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4003520 con 0x7f87c4003970 2023-10-21T16:22:09.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.982+0000 7f87db59e640 1 -- 172.21.15.53:0/1724727148 reap_dead start 2023-10-21T16:22:09.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.983+0000 7f87dad9d640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 0x7f87c4003d90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00c190 tx=0x7f87c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a0732d31530178 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:09.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc007270 con 0x7f87c4003970 2023-10-21T16:22:09.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0073f0 con 0x7f87c4003970 2023-10-21T16:22:09.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87d9d9b640 1 -- 172.21.15.53:0/1724727148 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc02c8a0 con 0x7f87c4003970 2023-10-21T16:22:09.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 -- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 msgr2=0x7f87c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 0x7f87c4003d90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00c190 tx=0x7f87c40057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 -- 172.21.15.53:0/1724727148 shutdown_connections 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 --2- 172.21.15.53:0/1724727148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87c4003970 0x7f87c4003d90 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 -- 172.21.15.53:0/1724727148 >> 172.21.15.53:0/1724727148 conn(0x7f87dc09bcf0 msgr2=0x7f87dc0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 -- 172.21.15.53:0/1724727148 shutdown_connections 2023-10-21T16:22:09.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.984+0000 7f87e0982640 1 -- 172.21.15.53:0/1724727148 wait complete. 2023-10-21T16:22:09.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.986+0000 7f87e0982640 1 Processor -- start 2023-10-21T16:22:09.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.986+0000 7f87e0982640 1 -- start start 2023-10-21T16:22:09.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4003970 0x7f87dc0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc0b7fd0 0x7f87dc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:09.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b6060 con 0x7f87dc158580 2023-10-21T16:22:09.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b87b0 con 0x7f87dc0b7fd0 2023-10-21T16:22:09.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.987+0000 7f87e0982640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b56c0 con 0x7f87c4003970 2023-10-21T16:22:09.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48368/0 (socket says 172.21.15.53:48368) 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 -- 172.21.15.53:0/3928668500 learned_addr learned my addr 172.21.15.53:0/3928668500 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87dad9d640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4003970 0x7f87dc0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87da59c640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc0b7fd0 0x7f87dc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 -- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4003970 msgr2=0x7f87dc0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4003970 0x7f87dc0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 -- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc0b7fd0 msgr2=0x7f87dc158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:09.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc0b7fd0 0x7f87dc158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:09.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.988+0000 7f87db59e640 1 -- 172.21.15.53:0/3928668500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0b59a0 con 0x7f87dc158580 2023-10-21T16:22:09.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87db59e640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f87dc08cd90 tx=0x7f87d400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:09.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400eed0 con 0x7f87dc158580 2023-10-21T16:22:09.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d4003970 con 0x7f87dc158580 2023-10-21T16:22:09.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4003d40 con 0x7f87dc158580 2023-10-21T16:22:09.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0b5c80 con 0x7f87dc158580 2023-10-21T16:22:09.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.989+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc1593c0 con 0x7f87dc158580 2023-10-21T16:22:09.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.990+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc0b61a0 con 0x7f87dc158580 2023-10-21T16:22:09.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.991+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f87d40050d0 con 0x7f87dc158580 2023-10-21T16:22:09.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.993+0000 7f87c37fe640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 0x7f87c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.993+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f87d4089260 con 0x7f87dc158580 2023-10-21T16:22:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.993+0000 7f87dad9d640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 0x7f87c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.994+0000 7f87dad9d640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 0x7f87c806d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f87cc034000 tx=0x7f87cc009030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:09.997+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87dc0b61a0 con 0x7f87dc158580 2023-10-21T16:22:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.194+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87dc0b61a0 con 0x7f87dc158580 2023-10-21T16:22:10.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.196+0000 7f87c37fe640 1 -- 172.21.15.53:0/3928668500 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+13175 (secure 0 0 0) 0x7f87dc0b61a0 con 0x7f87dc158580 2023-10-21T16:22:10.197 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":157,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:22:09.194361+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:22:10.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 msgr2=0x7f87c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 0x7f87c806d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f87cc034000 tx=0x7f87cc009030 comp rx=0 tx=0).stop 2023-10-21T16:22:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 msgr2=0x7f87dc0c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f87dc08cd90 tx=0x7f87d400eb90 comp rx=0 tx=0).stop 2023-10-21T16:22:10.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 shutdown_connections 2023-10-21T16:22:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f87c806af90 0x7f87c806d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87dc158580 0x7f87dc0c80f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc0b7fd0 0x7f87dc158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 --2- 172.21.15.53:0/3928668500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c4003970 0x7f87dc0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 >> 172.21.15.53:0/3928668500 conn(0x7f87dc09bcf0 msgr2=0x7f87dc0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 shutdown_connections 2023-10-21T16:22:10.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.199+0000 7f87e0982640 1 -- 172.21.15.53:0/3928668500 wait complete. 2023-10-21T16:22:10.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T16:22:10.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T16:22:10.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.450+0000 7f5ffffff640 1 Processor -- start 2023-10-21T16:22:10.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.450+0000 7f5ffffff640 1 -- start start 2023-10-21T16:22:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f600015cd80 0x7f600015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60000d06c0 0x7f600015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 0x7f60000d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600015d160 con 0x7f60000d06c0 2023-10-21T16:22:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000d3800 con 0x7f600015cd80 2023-10-21T16:22:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ffffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000d1850 con 0x7f60000d0ec0 2023-10-21T16:22:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ff7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60000d06c0 0x7f600015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ff7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60000d06c0 0x7f600015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48376/0 (socket says 172.21.15.53:48376) 2023-10-21T16:22:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.452+0000 7f5ff7fff640 1 -- 172.21.15.53:0/58139164 learned_addr learned my addr 172.21.15.53:0/58139164 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ffeffd640 1 --2- 172.21.15.53:0/58139164 >> v2:172.21.15.124:3300/0 conn(0x7f600015cd80 0x7f600015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5fff7fe640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 0x7f60000d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ff7fff640 1 -- 172.21.15.53:0/58139164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 msgr2=0x7f60000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 0x7f60000d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ff7fff640 1 -- 172.21.15.53:0/58139164 >> v2:172.21.15.124:3300/0 conn(0x7f600015cd80 msgr2=0x7f600015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/58139164 >> v2:172.21.15.124:3300/0 conn(0x7f600015cd80 0x7f600015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5ff7fff640 1 -- 172.21.15.53:0/58139164 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000ce370 con 0x7f60000d06c0 2023-10-21T16:22:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.453+0000 7f5fff7fe640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 0x7f60000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.454+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d06c0 0x7f600015e5e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f600008cff0 tx=0x7f5fe802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d041934dc158e4a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.454+0000 7f5ffe7fc640 1 -- 172.21.15.53:0/58139164 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80317e0 con 0x7f60000d06c0 2023-10-21T16:22:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffe7fc640 1 -- 172.21.15.53:0/58139164 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8031940 con 0x7f60000d06c0 2023-10-21T16:22:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffe7fc640 1 -- 172.21.15.53:0/58139164 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8031cd0 con 0x7f60000d06c0 2023-10-21T16:22:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 -- 172.21.15.53:0/58139164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d06c0 msgr2=0x7f600015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d06c0 0x7f600015e5e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f600008cff0 tx=0x7f5fe802b880 comp rx=0 tx=0).stop 2023-10-21T16:22:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 -- 172.21.15.53:0/58139164 shutdown_connections 2023-10-21T16:22:10.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60000d0ec0 0x7f60000d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 --2- 172.21.15.53:0/58139164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d06c0 0x7f600015e5e0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 --2- 172.21.15.53:0/58139164 >> v2:172.21.15.124:3300/0 conn(0x7f600015cd80 0x7f600015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 -- 172.21.15.53:0/58139164 >> 172.21.15.53:0/58139164 conn(0x7f600009bc40 msgr2=0x7f60000d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 -- 172.21.15.53:0/58139164 shutdown_connections 2023-10-21T16:22:10.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.455+0000 7f5ffffff640 1 -- 172.21.15.53:0/58139164 wait complete. 2023-10-21T16:22:10.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.457+0000 7f5ffffff640 1 Processor -- start 2023-10-21T16:22:10.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.457+0000 7f5ffffff640 1 -- start start 2023-10-21T16:22:10.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 0x7f60000b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f600015cd80 0x7f60000c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000c68a0 con 0x7f60000d0ec0 2023-10-21T16:22:10.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000c6b80 con 0x7f60000d06c0 2023-10-21T16:22:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ffffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000c6e60 con 0x7f600015cd80 2023-10-21T16:22:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.458+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48388/0 (socket says 172.21.15.53:48388) 2023-10-21T16:22:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 -- 172.21.15.53:0/505000611 learned_addr learned my addr 172.21.15.53:0/505000611 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:10.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5fff7fe640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f600015cd80 0x7f60000c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ffeffd640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 0x7f60000b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 -- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f600015cd80 msgr2=0x7f60000c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f600015cd80 0x7f60000c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 -- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 msgr2=0x7f60000b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 0x7f60000b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 -- 172.21.15.53:0/505000611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80094d0 con 0x7f60000d0ec0 2023-10-21T16:22:10.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ffeffd640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 0x7f60000b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:10.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.459+0000 7f5ff7fff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000c00 tx=0x7f5fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.460+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8032900 con 0x7f60000d0ec0 2023-10-21T16:22:10.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.460+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8032a60 con 0x7f60000d0ec0 2023-10-21T16:22:10.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.460+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8032e10 con 0x7f60000d0ec0 2023-10-21T16:22:10.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.460+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60000c7140 con 0x7f60000d0ec0 2023-10-21T16:22:10.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.460+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60000b7d50 con 0x7f60000d0ec0 2023-10-21T16:22:10.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.461+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60000c7660 con 0x7f60000d0ec0 2023-10-21T16:22:10.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.462+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5fe804e450 con 0x7f60000d0ec0 2023-10-21T16:22:10.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.463+0000 7f5ff77fe640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 0x7f5fe006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.464+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5fe80ab950 con 0x7f60000d0ec0 2023-10-21T16:22:10.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.464+0000 7f5ffeffd640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 0x7f5fe006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.465+0000 7f5ffeffd640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 0x7f5fe006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b800 tx=0x7f5ff0008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.468+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60000c7660 con 0x7f60000d0ec0 2023-10-21T16:22:10.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.662+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60000c7660 con 0x7f60000d0ec0 2023-10-21T16:22:10.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.663+0000 7f5ff77fe640 1 -- 172.21.15.53:0/505000611 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f60000c7660 con 0x7f60000d0ec0 2023-10-21T16:22:10.665 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T16:22:10.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 msgr2=0x7f5fe006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 0x7f5fe006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b800 tx=0x7f5ff0008040 comp rx=0 tx=0).stop 2023-10-21T16:22:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 msgr2=0x7f60001521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000c00 tx=0x7f5fe80046c0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 shutdown_connections 2023-10-21T16:22:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5fe006af90 0x7f5fe006d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f600015cd80 0x7f60000c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60000d0ec0 0x7f60001521b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 --2- 172.21.15.53:0/505000611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60000d06c0 0x7f60000b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.665+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 >> 172.21.15.53:0/505000611 conn(0x7f600009bc40 msgr2=0x7f60000d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.666+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 shutdown_connections 2023-10-21T16:22:10.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.666+0000 7f5ffffff640 1 -- 172.21.15.53:0/505000611 wait complete. 2023-10-21T16:22:10.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T16:22:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.905+0000 7f591759e640 1 Processor -- start 2023-10-21T16:22:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.905+0000 7f591759e640 1 -- start start 2023-10-21T16:22:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f591815ca60 0x7f59180af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f59180cfcc0 0x7f59180ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591806ba80 con 0x7f59180ad400 2023-10-21T16:22:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59180ceba0 con 0x7f591815ca60 2023-10-21T16:22:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591806bcd0 con 0x7f59180cfcc0 2023-10-21T16:22:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f591815ca60 0x7f59180af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f590ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f59180ad400 0x7f59180cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40648/0 (socket says 172.21.15.53:40648) 2023-10-21T16:22:10.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f591659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f591815ca60 0x7f59180af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48400/0 (socket says 172.21.15.53:48400) 2023-10-21T16:22:10.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.907+0000 7f590ffff640 1 -- 172.21.15.53:0/58962550 learned_addr learned my addr 172.21.15.53:0/58962550 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:10.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f5916d9d640 1 --2- 172.21.15.53:0/58962550 >> v2:172.21.15.124:3300/0 conn(0x7f59180cfcc0 0x7f59180ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f591659c640 1 -- 172.21.15.53:0/58962550 >> v2:172.21.15.124:3300/0 conn(0x7f59180cfcc0 msgr2=0x7f59180ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:10.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f591659c640 1 --2- 172.21.15.53:0/58962550 >> v2:172.21.15.124:3300/0 conn(0x7f59180cfcc0 0x7f59180ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f591659c640 1 -- 172.21.15.53:0/58962550 >> v1:172.21.15.53:6789/0 conn(0x7f59180ad400 legacy=0x7f59180cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f591659c640 1 -- 172.21.15.53:0/58962550 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591806bfb0 con 0x7f591815ca60 2023-10-21T16:22:10.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.908+0000 7f5915d9b640 1 -- 172.21.15.53:0/58962550 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700895777 0 0) 0x7f591806ba80 con 0x7f59180ad400 2023-10-21T16:22:10.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.909+0000 7f591659c640 1 --2- 172.21.15.53:0/58962550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f591815ca60 0x7f59180af770 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f591808a070 tx=0x7f590802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa290b5430ccd11 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.909+0000 7f5915d9b640 1 -- 172.21.15.53:0/58962550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590802e070 con 0x7f591815ca60 2023-10-21T16:22:10.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.909+0000 7f5915d9b640 1 -- 172.21.15.53:0/58962550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080315a0 con 0x7f591815ca60 2023-10-21T16:22:10.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.909+0000 7f5915d9b640 1 -- 172.21.15.53:0/58962550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908031930 con 0x7f591815ca60 2023-10-21T16:22:10.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 -- 172.21.15.53:0/58962550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f591815ca60 msgr2=0x7f59180af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 --2- 172.21.15.53:0/58962550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f591815ca60 0x7f59180af770 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f591808a070 tx=0x7f590802b840 comp rx=0 tx=0).stop 2023-10-21T16:22:10.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 -- 172.21.15.53:0/58962550 shutdown_connections 2023-10-21T16:22:10.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 --2- 172.21.15.53:0/58962550 >> v2:172.21.15.124:3300/0 conn(0x7f59180cfcc0 0x7f59180ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 --2- 172.21.15.53:0/58962550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f591815ca60 0x7f59180af770 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 -- 172.21.15.53:0/58962550 >> 172.21.15.53:0/58962550 conn(0x7f59180d0830 msgr2=0x7f59180d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:10.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 -- 172.21.15.53:0/58962550 shutdown_connections 2023-10-21T16:22:10.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.910+0000 7f591759e640 1 -- 172.21.15.53:0/58962550 wait complete. 2023-10-21T16:22:10.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.912+0000 7f591759e640 1 Processor -- start 2023-10-21T16:22:10.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.912+0000 7f591759e640 1 -- start start 2023-10-21T16:22:10.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59180ad400 0x7f5918150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f591815ca60 0x7f59180b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b41d0 con 0x7f59180cfcc0 2023-10-21T16:22:10.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b44b0 con 0x7f591815ca60 2023-10-21T16:22:10.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f591759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59180b4790 con 0x7f59180ad400 2023-10-21T16:22:10.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f590ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f590ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48404/0 (socket says 172.21.15.53:48404) 2023-10-21T16:22:10.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.913+0000 7f590ffff640 1 -- 172.21.15.53:0/3891592884 learned_addr learned my addr 172.21.15.53:0/3891592884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:10.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f591659c640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59180ad400 0x7f5918150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 -- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59180ad400 msgr2=0x7f5918150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f5916d9d640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f591815ca60 0x7f59180b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59180ad400 0x7f5918150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 -- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f591815ca60 msgr2=0x7f59180b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f591815ca60 0x7f59180b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:10.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 -- 172.21.15.53:0/3891592884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080093a0 con 0x7f59180cfcc0 2023-10-21T16:22:10.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590ffff640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f590000c4d0 tx=0x7f590000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000074e0 con 0x7f59180cfcc0 2023-10-21T16:22:10.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5900007660 con 0x7f59180cfcc0 2023-10-21T16:22:10.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.914+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900004a20 con 0x7f59180cfcc0 2023-10-21T16:22:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.915+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59180b4a70 con 0x7f59180cfcc0 2023-10-21T16:22:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.915+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5918151050 con 0x7f59180cfcc0 2023-10-21T16:22:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.916+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59180b4f90 con 0x7f59180cfcc0 2023-10-21T16:22:10.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.917+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5900004b80 con 0x7f59180cfcc0 2023-10-21T16:22:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.918+0000 7f590effd640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 0x7f58f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.918+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5900088810 con 0x7f59180cfcc0 2023-10-21T16:22:10.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.918+0000 7f591659c640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 0x7f58f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.920+0000 7f591659c640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 0x7f58f806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5908036000 tx=0x7f590803d040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:10.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:10.922+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59180b4f90 con 0x7f59180cfcc0 2023-10-21T16:22:11.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.107+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59180b4f90 con 0x7f59180cfcc0 2023-10-21T16:22:11.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.108+0000 7f590effd640 1 -- 172.21.15.53:0/3891592884 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v157) v1 ==== 84+0+65 (secure 0 0 0) 0x7f59180b4f90 con 0x7f59180cfcc0 2023-10-21T16:22:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:11.110 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:22:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T16:22:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T16:22:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:11.111 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:22:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 msgr2=0x7f58f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 0x7f58f806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5908036000 tx=0x7f590803d040 comp rx=0 tx=0).stop 2023-10-21T16:22:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 msgr2=0x7f59180c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f590000c4d0 tx=0x7f590000c990 comp rx=0 tx=0).stop 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 shutdown_connections 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f58f806af90 0x7f58f806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f591815ca60 0x7f59180b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59180cfcc0 0x7f59180c55f0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 --2- 172.21.15.53:0/3891592884 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59180ad400 0x7f5918150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 >> 172.21.15.53:0/3891592884 conn(0x7f59180d0830 msgr2=0x7f59180afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 shutdown_connections 2023-10-21T16:22:11.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.110+0000 7f591759e640 1 -- 172.21.15.53:0/3891592884 wait complete. 2023-10-21T16:22:11.130 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T16:22:11.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T16:22:11.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.346+0000 7fd1a1573640 1 Processor -- start 2023-10-21T16:22:11.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.347+0000 7fd1a1573640 1 -- start start 2023-10-21T16:22:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.348+0000 7fd1a1573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.348+0000 7fd1a1573640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd19c160c90 0x7fd19c15c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.348+0000 7fd1a1573640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0b0260 con 0x7fd19c0c8fb0 2023-10-21T16:22:11.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.348+0000 7fd1a1573640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c1610d0 con 0x7fd19c160c90 2023-10-21T16:22:11.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.348+0000 7fd1a1573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c1612d0 con 0x7fd19c0c9390 2023-10-21T16:22:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.349+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.349+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35026/0 (socket says 172.21.15.53:35026) 2023-10-21T16:22:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.349+0000 7fd19b7fe640 1 -- 172.21.15.53:0/26012739 learned_addr learned my addr 172.21.15.53:0/26012739 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.349+0000 7fd1a0d72640 1 --2- 172.21.15.53:0/26012739 >> v2:172.21.15.124:3300/0 conn(0x7fd19c160c90 0x7fd19c15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.350+0000 7fd19b7fe640 1 -- 172.21.15.53:0/26012739 >> v2:172.21.15.124:3300/0 conn(0x7fd19c160c90 msgr2=0x7fd19c15c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.350+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/26012739 >> v2:172.21.15.124:3300/0 conn(0x7fd19c160c90 0x7fd19c15c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.350+0000 7fd19b7fe640 1 -- 172.21.15.53:0/26012739 >> v1:172.21.15.124:6789/0 conn(0x7fd19c0c8fb0 legacy=0x7fd19c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.350+0000 7fd19b7fe640 1 -- 172.21.15.53:0/26012739 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c161590 con 0x7fd19c0c9390 2023-10-21T16:22:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.351+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/26012739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4179d9ce47aea94e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:11.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd19affd640 1 -- 172.21.15.53:0/26012739 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19002f830 con 0x7fd19c0c9390 2023-10-21T16:22:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd19affd640 1 -- 172.21.15.53:0/26012739 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19002f990 con 0x7fd19c0c9390 2023-10-21T16:22:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd19affd640 1 -- 172.21.15.53:0/26012739 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19002fd20 con 0x7fd19c0c9390 2023-10-21T16:22:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 -- 172.21.15.53:0/26012739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 msgr2=0x7fd19c160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 --2- 172.21.15.53:0/26012739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002ba10 comp rx=0 tx=0).stop 2023-10-21T16:22:11.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 -- 172.21.15.53:0/26012739 shutdown_connections 2023-10-21T16:22:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 --2- 172.21.15.53:0/26012739 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c9390 0x7fd19c160750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 --2- 172.21.15.53:0/26012739 >> v2:172.21.15.124:3300/0 conn(0x7fd19c160c90 0x7fd19c15c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.352+0000 7fd1a1573640 1 -- 172.21.15.53:0/26012739 >> 172.21.15.53:0/26012739 conn(0x7fd19c06c010 msgr2=0x7fd19c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.353+0000 7fd1a1573640 1 -- 172.21.15.53:0/26012739 shutdown_connections 2023-10-21T16:22:11.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.353+0000 7fd1a1573640 1 -- 172.21.15.53:0/26012739 wait complete. 2023-10-21T16:22:11.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.354+0000 7fd1a1573640 1 Processor -- start 2023-10-21T16:22:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.355+0000 7fd1a1573640 1 -- start start 2023-10-21T16:22:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.355+0000 7fd1a1573640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c8fb0 0x7fd19c146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.355+0000 7fd1a1573640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd19c0c9390 0x7fd19c0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.355+0000 7fd1a1573640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a1573640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b2680 con 0x7fd19c160c90 2023-10-21T16:22:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a1573640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b2aa0 con 0x7fd19c0c9390 2023-10-21T16:22:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a1573640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b1a00 con 0x7fd19c0c8fb0 2023-10-21T16:22:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48414/0 (socket says 172.21.15.53:48414) 2023-10-21T16:22:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 -- 172.21.15.53:0/1121114598 learned_addr learned my addr 172.21.15.53:0/1121114598 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd19bfff640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c8fb0 0x7fd19c146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd19b7fe640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd19c0c9390 0x7fd19c0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 -- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c8fb0 msgr2=0x7fd19c146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c8fb0 0x7fd19c146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 -- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd19c0c9390 msgr2=0x7fd19c0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd19c0c9390 0x7fd19c0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.356+0000 7fd1a0d72640 1 -- 172.21.15.53:0/1121114598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900094d0 con 0x7fd19c160c90 2023-10-21T16:22:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd1a0d72640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd194007e40 tx=0x7fd19400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:11.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194014620 con 0x7fd19c160c90 2023-10-21T16:22:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400ee90 con 0x7fd19c160c90 2023-10-21T16:22:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1940149e0 con 0x7fd19c160c90 2023-10-21T16:22:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c0b1ce0 con 0x7fd19c160c90 2023-10-21T16:22:11.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.357+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c0b22a0 con 0x7fd19c160c90 2023-10-21T16:22:11.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.358+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c0b27c0 con 0x7fd19c160c90 2023-10-21T16:22:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.359+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd194014b40 con 0x7fd19c160c90 2023-10-21T16:22:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.360+0000 7fd198ff9640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 0x7fd18806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.360+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd194089f60 con 0x7fd19c160c90 2023-10-21T16:22:11.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.361+0000 7fd19bfff640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 0x7fd18806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.362+0000 7fd19bfff640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 0x7fd18806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000f60 tx=0x7fd18c009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:11.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.365+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd19c0b27c0 con 0x7fd19c160c90 2023-10-21T16:22:11.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:11.562+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd19c0b27c0 con 0x7fd19c160c90 2023-10-21T16:22:12.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.434+0000 7fd198ff9640 1 -- 172.21.15.53:0/1121114598 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v158) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd19c0b27c0 con 0x7fd19c160c90 2023-10-21T16:22:12.436 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T16:22:12.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 msgr2=0x7fd18806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 0x7fd18806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd18c000f60 tx=0x7fd18c009040 comp rx=0 tx=0).stop 2023-10-21T16:22:12.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 msgr2=0x7fd19c0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd194007e40 tx=0x7fd19400e4b0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 shutdown_connections 2023-10-21T16:22:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fd18806af90 0x7fd18806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd19c160c90 0x7fd19c0c7a80 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd19c0c9390 0x7fd19c0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 --2- 172.21.15.53:0/1121114598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd19c0c8fb0 0x7fd19c146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.437+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 >> 172.21.15.53:0/1121114598 conn(0x7fd19c06c010 msgr2=0x7fd19c15a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:12.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.438+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 shutdown_connections 2023-10-21T16:22:12.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.438+0000 7fd1a1573640 1 -- 172.21.15.53:0/1121114598 wait complete. 2023-10-21T16:22:12.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T16:22:12.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.694+0000 7f57af59e640 1 Processor -- start 2023-10-21T16:22:12.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.695+0000 7f57af59e640 1 -- start start 2023-10-21T16:22:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.696+0000 7f57af59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f57b01658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.696+0000 7f57af59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b015b1f0 con 0x7f57b015ae10 2023-10-21T16:22:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.696+0000 7f57af59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00cc990 con 0x7f57b00ce800 2023-10-21T16:22:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.696+0000 7f57af59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00cc6b0 con 0x7f57b00cf450 2023-10-21T16:22:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.697+0000 7f57ae59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f57b015ae10 0x7f57b00cbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40664/0 (socket says 172.21.15.53:40664) 2023-10-21T16:22:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.697+0000 7f57ae59c640 1 -- 172.21.15.53:0/3839292065 learned_addr learned my addr 172.21.15.53:0/3839292065 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.697+0000 7f57aed9d640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f57b01658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371156967 0 0) 0x7f57a0001110 con 0x7f57b015ae10 2023-10-21T16:22:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f579c003520 con 0x7f57b015ae10 2023-10-21T16:22:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310735168 0 0) 0x7f579c003520 con 0x7f57b015ae10 2023-10-21T16:22:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 msgr2=0x7f57b01658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:12.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f57b01658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 >> v1:172.21.15.124:6789/0 conn(0x7f57b00ce800 legacy=0x7f57b00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00aff20 con 0x7f57b015ae10 2023-10-21T16:22:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f57a0001800 con 0x7f57b015ae10 2023-10-21T16:22:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 >> v1:172.21.15.53:6789/0 conn(0x7f57b015ae10 legacy=0x7f57b00cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f579c0039a0 0x7f579c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0047c0 0x7f579c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f579c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00cc6b0 con 0x7f579c0039a0 2023-10-21T16:22:12.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c00bae0 con 0x7f579c0047c0 2023-10-21T16:22:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c00bca0 con 0x7f57b00cf450 2023-10-21T16:22:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.698+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f579c0039a0 0x7f579c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57aed9d640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f579c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57a7fff640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0047c0 0x7f579c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57ae59c640 1 -- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 msgr2=0x7f579c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f579c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57ae59c640 1 -- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0047c0 msgr2=0x7f579c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0047c0 0x7f579c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57ae59c640 1 -- 172.21.15.53:0/3839292065 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c003520 con 0x7f579c0039a0 2023-10-21T16:22:12.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.699+0000 7f57aed9d640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b00cf450 0x7f579c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.700+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f579c0039a0 0x7f579c003dc0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c240 tx=0x7f579c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c004a6eee2c7a1e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.700+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0005ad0 con 0x7f579c0039a0 2023-10-21T16:22:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.700+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0005c50 con 0x7f579c0039a0 2023-10-21T16:22:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.700+0000 7f57add9b640 1 -- 172.21.15.53:0/3839292065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a002caf0 con 0x7f579c0039a0 2023-10-21T16:22:12.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 -- 172.21.15.53:0/3839292065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f579c0039a0 msgr2=0x7f579c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 --2- 172.21.15.53:0/3839292065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f579c0039a0 0x7f579c003dc0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f57a000c240 tx=0x7f579c0057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57aed9d640 1 -- 172.21.15.53:0/3839292065 reap_dead start 2023-10-21T16:22:12.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 -- 172.21.15.53:0/3839292065 shutdown_connections 2023-10-21T16:22:12.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 -- 172.21.15.53:0/3839292065 >> 172.21.15.53:0/3839292065 conn(0x7f57b00d0860 msgr2=0x7f57b00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:12.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 -- 172.21.15.53:0/3839292065 shutdown_connections 2023-10-21T16:22:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.701+0000 7f57af59e640 1 -- 172.21.15.53:0/3839292065 wait complete. 2023-10-21T16:22:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.703+0000 7f57af59e640 1 Processor -- start 2023-10-21T16:22:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.704+0000 7f57af59e640 1 -- start start 2023-10-21T16:22:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 0x7f57b015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b01592a0 0x7f57b014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b01576b0 con 0x7f57b006f7d0 2023-10-21T16:22:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0157990 con 0x7f579c0039a0 2023-10-21T16:22:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57af59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0157c70 con 0x7f57b01592a0 2023-10-21T16:22:12.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48434/0 (socket says 172.21.15.53:48434) 2023-10-21T16:22:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 -- 172.21.15.53:0/3190993620 learned_addr learned my addr 172.21.15.53:0/3190993620 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 0x7f57b015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57aed9d640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b01592a0 0x7f57b014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 -- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b01592a0 msgr2=0x7f57b014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b01592a0 0x7f57b014b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 -- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 msgr2=0x7f57b015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:12.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 0x7f57b015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:12.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57a7fff640 1 -- 172.21.15.53:0/3190993620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0157f50 con 0x7f57b006f7d0 2023-10-21T16:22:12.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.705+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 0x7f57b015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:12.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57a7fff640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f579800e410 tx=0x7f579800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:12.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980052e0 con 0x7f57b006f7d0 2023-10-21T16:22:12.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798005440 con 0x7f57b006f7d0 2023-10-21T16:22:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798003810 con 0x7f57b006f7d0 2023-10-21T16:22:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b015d5e0 con 0x7f57b006f7d0 2023-10-21T16:22:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.706+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b006fc50 con 0x7f57b006f7d0 2023-10-21T16:22:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.707+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b004e160 con 0x7f57b006f7d0 2023-10-21T16:22:12.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.709+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5798003970 con 0x7f57b006f7d0 2023-10-21T16:22:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.709+0000 7f57a6ffd640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 0x7f579006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:12.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.710+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f5798088f10 con 0x7f57b006f7d0 2023-10-21T16:22:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.710+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 0x7f579006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.711+0000 7f57ae59c640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 0x7f579006d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57a0034000 tx=0x7f57a0009030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:12.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.715+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5798053fa0 con 0x7f57b006f7d0 2023-10-21T16:22:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:12.910+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f57b01580e0 con 0x7f57b006f7d0 2023-10-21T16:22:13.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.441+0000 7f57a6ffd640 1 -- 172.21.15.53:0/3190993620 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v159) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5798019020 con 0x7f57b006f7d0 2023-10-21T16:22:13.443 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T16:22:13.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.447+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 msgr2=0x7f579006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.447+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 0x7f579006d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57a0034000 tx=0x7f57a0009030 comp rx=0 tx=0).stop 2023-10-21T16:22:13.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.447+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 msgr2=0x7f57b0158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.447+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f579800e410 tx=0x7f579800e8d0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 shutdown_connections 2023-10-21T16:22:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f579006afc0 0x7f579006d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57b01592a0 0x7f57b014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57b006f7d0 0x7f57b0158d60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 --2- 172.21.15.53:0/3190993620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f579c0039a0 0x7f57b015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.448+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 >> 172.21.15.53:0/3190993620 conn(0x7f57b00d0860 msgr2=0x7f57b00af440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.450+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 shutdown_connections 2023-10-21T16:22:13.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.450+0000 7f57af59e640 1 -- 172.21.15.53:0/3190993620 wait complete. 2023-10-21T16:22:13.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T16:22:13.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.678+0000 7fdc1214e640 1 Processor -- start 2023-10-21T16:22:13.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.679+0000 7fdc1214e640 1 -- start start 2023-10-21T16:22:13.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.680+0000 7fdc1214e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdc0c0cd190 0x7fdc0c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.680+0000 7fdc1214e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdc0c0c9d50 0x7fdc0c0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.680+0000 7fdc1214e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0c0ccfa0 con 0x7fdc0c0ccb40 2023-10-21T16:22:13.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.680+0000 7fdc1214e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0ae300 con 0x7fdc0c0cd190 2023-10-21T16:22:13.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.680+0000 7fdc1214e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0cb560 con 0x7fdc0c0c9d50 2023-10-21T16:22:13.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdc0c0ccb40 0x7fdc0c0cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40678/0 (socket says 172.21.15.53:40678) 2023-10-21T16:22:13.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0affd640 1 -- 172.21.15.53:0/2146100227 learned_addr learned my addr 172.21.15.53:0/2146100227 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:13.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2146100227 >> v2:172.21.15.53:3300/0 conn(0x7fdc0c0cd190 0x7fdc0c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0bfff640 1 --2- 172.21.15.53:0/2146100227 >> v2:172.21.15.124:3300/0 conn(0x7fdc0c0c9d50 0x7fdc0c0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0a7fc640 1 -- 172.21.15.53:0/2146100227 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243881479 0 0) 0x7fdc0c0ccfa0 con 0x7fdc0c0ccb40 2023-10-21T16:22:13.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0a7fc640 1 -- 172.21.15.53:0/2146100227 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf8005730 con 0x7fdc0c0ccb40 2023-10-21T16:22:13.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0b7fe640 1 -- 172.21.15.53:0/2146100227 >> v2:172.21.15.124:3300/0 conn(0x7fdc0c0c9d50 msgr2=0x7fdc0c0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:13.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2146100227 >> v2:172.21.15.124:3300/0 conn(0x7fdc0c0c9d50 0x7fdc0c0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0b7fe640 1 -- 172.21.15.53:0/2146100227 >> v1:172.21.15.53:6789/0 conn(0x7fdc0c0ccb40 legacy=0x7fdc0c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.681+0000 7fdc0b7fe640 1 -- 172.21.15.53:0/2146100227 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0cb840 con 0x7fdc0c0cd190 2023-10-21T16:22:13.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2146100227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c0ae490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0b1610 tx=0x7fdbfc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2130c20a0b63da3c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:13.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc0a7fc640 1 -- 172.21.15.53:0/2146100227 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc02cb40 con 0x7fdc0c0cd190 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc0a7fc640 1 -- 172.21.15.53:0/2146100227 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc02cca0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc0a7fc640 1 -- 172.21.15.53:0/2146100227 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc034660 con 0x7fdc0c0cd190 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc1214e640 1 -- 172.21.15.53:0/2146100227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 msgr2=0x7fdc0c0ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.683+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2146100227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c0ae490 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0b1610 tx=0x7fdbfc02bd90 comp rx=0 tx=0).stop 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 -- 172.21.15.53:0/2146100227 shutdown_connections 2023-10-21T16:22:13.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2146100227 >> v2:172.21.15.124:3300/0 conn(0x7fdc0c0c9d50 0x7fdc0c0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2146100227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c0ae490 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 -- 172.21.15.53:0/2146100227 >> 172.21.15.53:0/2146100227 conn(0x7fdc0c0b0af0 msgr2=0x7fdc0c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:13.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 -- 172.21.15.53:0/2146100227 shutdown_connections 2023-10-21T16:22:13.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.684+0000 7fdc1214e640 1 -- 172.21.15.53:0/2146100227 wait complete. 2023-10-21T16:22:13.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.686+0000 7fdc1214e640 1 Processor -- start 2023-10-21T16:22:13.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.686+0000 7fdc1214e640 1 -- start start 2023-10-21T16:22:13.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 0x7fdc0c15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc0c0ccb40 0x7fdc0c0c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c15a900 con 0x7fdc0c0cd190 2023-10-21T16:22:13.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c159f30 con 0x7fdc0c0c9d50 2023-10-21T16:22:13.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc1214e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c1e2800 con 0x7fdc0c0ccb40 2023-10-21T16:22:13.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48444/0 (socket says 172.21.15.53:48444) 2023-10-21T16:22:13.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc0bfff640 1 -- 172.21.15.53:0/2616393106 learned_addr learned my addr 172.21.15.53:0/2616393106 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:13.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc0affd640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc0c0ccb40 0x7fdc0c0c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.687+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 0x7fdc0c15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 -- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc0c0ccb40 msgr2=0x7fdc0c0c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc0c0ccb40 0x7fdc0c0c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 -- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 msgr2=0x7fdc0c15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:13.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 0x7fdc0c15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 -- 172.21.15.53:0/2616393106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc009a70 con 0x7fdc0c0cd190 2023-10-21T16:22:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 0x7fdc0c15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc0bfff640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d5d0 tx=0x7fdc0400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040125f0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04012770 con 0x7fdc0c0cd190 2023-10-21T16:22:13.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04012b20 con 0x7fdc0c0cd190 2023-10-21T16:22:13.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.688+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c159600 con 0x7fdc0c0cd190 2023-10-21T16:22:13.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.689+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c159bc0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.690+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c0662c0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.691+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc04012c80 con 0x7fdc0c0cd190 2023-10-21T16:22:13.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.692+0000 7fdc03fff640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 0x7fdbec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:13.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.692+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fdc04089de0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.693+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 0x7fdbec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:13.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.694+0000 7fdc0b7fe640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 0x7fdbec06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc036000 tx=0x7fdbfc030000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:13.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.697+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc04054bc0 con 0x7fdc0c0cd190 2023-10-21T16:22:13.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:13.930+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdc0c04e160 con 0x7fdc0c0cd190 2023-10-21T16:22:14.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.488+0000 7fdc03fff640 1 -- 172.21.15.53:0/2616393106 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdc040581e0 con 0x7fdc0c0cd190 2023-10-21T16:22:14.498 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 19 and data pool 20 2023-10-21T16:22:14.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.499+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 msgr2=0x7fdbec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.499+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 0x7fdbec06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc036000 tx=0x7fdbfc030000 comp rx=0 tx=0).stop 2023-10-21T16:22:14.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.499+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 msgr2=0x7fdc0c1e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.499+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400d5d0 tx=0x7fdc0400da90 comp rx=0 tx=0).stop 2023-10-21T16:22:14.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 shutdown_connections 2023-10-21T16:22:14.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fdbec06af90 0x7fdbec06d450 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc036000 tx=0x7fdbfc030000 comp rx=0 tx=0).stop 2023-10-21T16:22:14.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc0c0cd190 0x7fdc0c1e1ad0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc0c0ccb40 0x7fdc0c0c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 --2- 172.21.15.53:0/2616393106 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc0c0c9d50 0x7fdc0c15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.500+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 >> 172.21.15.53:0/2616393106 conn(0x7fdc0c0b0af0 msgr2=0x7fdc0c0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:14.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.502+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 shutdown_connections 2023-10-21T16:22:14.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.502+0000 7fdc1214e640 1 -- 172.21.15.53:0/2616393106 wait complete. 2023-10-21T16:22:14.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:22:14.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.721+0000 7f92c2a90640 1 Processor -- start 2023-10-21T16:22:14.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.722+0000 7f92c2a90640 1 -- start start 2023-10-21T16:22:14.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.723+0000 7f92c2a90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92bc15ab40 0x7f92bc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.723+0000 7f92c2a90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f92bc0c90b0 0x7f92bc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.723+0000 7f92c2a90640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc15b2b0 con 0x7f92bc0cae70 2023-10-21T16:22:14.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.723+0000 7f92c2a90640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc15bf30 con 0x7f92bc15ab40 2023-10-21T16:22:14.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.723+0000 7f92c2a90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc15f7a0 con 0x7f92bc0c90b0 2023-10-21T16:22:14.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92bc15ab40 0x7f92bc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:14.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f92bc15ab40 0x7f92bc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50990/0 (socket says 172.21.15.53:50990) 2023-10-21T16:22:14.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 -- 172.21.15.53:0/598010961 learned_addr learned my addr 172.21.15.53:0/598010961 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:14.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 -- 172.21.15.53:0/598010961 >> v2:172.21.15.124:3300/0 conn(0x7f92bc0c90b0 msgr2=0x7f92bc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:14.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 --2- 172.21.15.53:0/598010961 >> v2:172.21.15.124:3300/0 conn(0x7f92bc0c90b0 0x7f92bc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 -- 172.21.15.53:0/598010961 >> v1:172.21.15.124:6789/0 conn(0x7f92bc0cae70 legacy=0x7f92bc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:14.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.724+0000 7f92c082c640 1 -- 172.21.15.53:0/598010961 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc15b590 con 0x7f92bc15ab40 2023-10-21T16:22:14.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.725+0000 7f92c082c640 1 --2- 172.21.15.53:0/598010961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc15ab40 0x7f92bc0ad720 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f92bc06e560 tx=0x7f92ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40301d712d252fc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:14.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92bb7fe640 1 -- 172.21.15.53:0/598010961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac02ca80 con 0x7f92bc15ab40 2023-10-21T16:22:14.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92bb7fe640 1 -- 172.21.15.53:0/598010961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac02cbe0 con 0x7f92bc15ab40 2023-10-21T16:22:14.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92bb7fe640 1 -- 172.21.15.53:0/598010961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0343f0 con 0x7f92bc15ab40 2023-10-21T16:22:14.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 -- 172.21.15.53:0/598010961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc15ab40 msgr2=0x7f92bc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 --2- 172.21.15.53:0/598010961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc15ab40 0x7f92bc0ad720 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f92bc06e560 tx=0x7f92ac02bee0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 -- 172.21.15.53:0/598010961 shutdown_connections 2023-10-21T16:22:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 --2- 172.21.15.53:0/598010961 >> v2:172.21.15.124:3300/0 conn(0x7f92bc0c90b0 0x7f92bc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 --2- 172.21.15.53:0/598010961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc15ab40 0x7f92bc0ad720 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.726+0000 7f92c2a90640 1 -- 172.21.15.53:0/598010961 >> 172.21.15.53:0/598010961 conn(0x7f92bc06a380 msgr2=0x7f92bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.727+0000 7f92c2a90640 1 -- 172.21.15.53:0/598010961 shutdown_connections 2023-10-21T16:22:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.727+0000 7f92c2a90640 1 -- 172.21.15.53:0/598010961 wait complete. 2023-10-21T16:22:14.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.728+0000 7f92c2a90640 1 Processor -- start 2023-10-21T16:22:14.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.728+0000 7f92c2a90640 1 -- start start 2023-10-21T16:22:14.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92bc0cae70 0x7f92bc151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92bc15ab40 0x7f92bc0c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15d450 con 0x7f92bc0c90b0 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc151880 con 0x7f92bc0cae70 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.729+0000 7f92c2a90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15c7d0 con 0x7f92bc15ab40 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51000/0 (socket says 172.21.15.53:51000) 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 -- 172.21.15.53:0/2853865152 learned_addr learned my addr 172.21.15.53:0/2853865152 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:14.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92bbfff640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92bc0cae70 0x7f92bc151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c102d640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92bc15ab40 0x7f92bc0c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 -- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92bc15ab40 msgr2=0x7f92bc0c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92bc15ab40 0x7f92bc0c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 -- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92bc0cae70 msgr2=0x7f92bc151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92bc0cae70 0x7f92bc151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 -- 172.21.15.53:0/2853865152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac009b30 con 0x7f92bc0c90b0 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.730+0000 7f92c082c640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0093e0 tx=0x7f92ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.731+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac03e450 con 0x7f92bc0c90b0 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.731+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac03e5b0 con 0x7f92bc0c90b0 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.731+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac03e960 con 0x7f92bc0c90b0 2023-10-21T16:22:14.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.731+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc15cab0 con 0x7f92bc0c90b0 2023-10-21T16:22:14.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.731+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc15d070 con 0x7f92bc0c90b0 2023-10-21T16:22:14.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.732+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc067490 con 0x7f92bc0c90b0 2023-10-21T16:22:14.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.733+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f92ac02cd50 con 0x7f92bc0c90b0 2023-10-21T16:22:14.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.734+0000 7f92b97fa640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 0x7f929c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:14.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.734+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92ac0ac600 con 0x7f92bc0c90b0 2023-10-21T16:22:14.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.735+0000 7f92bbfff640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 0x7f929c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:14.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.736+0000 7f92bbfff640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 0x7f929c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92bc15a260 tx=0x7f92a4008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:14.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.738+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92ac077400 con 0x7f92bc0c90b0 2023-10-21T16:22:14.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.936+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92bc0b0ef0 con 0x7f92bc0c90b0 2023-10-21T16:22:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.938+0000 7f92b97fa640 1 -- 172.21.15.53:0/2853865152 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f92ac03eac0 con 0x7f92bc0c90b0 2023-10-21T16:22:14.940 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:14.941 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":160,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:22:13.932049+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:22:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 msgr2=0x7f929c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 0x7f929c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92bc15a260 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2023-10-21T16:22:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 msgr2=0x7f92bc15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0093e0 tx=0x7f92ac004a40 comp rx=0 tx=0).stop 2023-10-21T16:22:14.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 shutdown_connections 2023-10-21T16:22:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f929c06af90 0x7f929c06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f92bc15ab40 0x7f92bc0c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92bc0cae70 0x7f92bc151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 --2- 172.21.15.53:0/2853865152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92bc0c90b0 0x7f92bc15d870 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:14.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 >> 172.21.15.53:0/2853865152 conn(0x7f92bc06a380 msgr2=0x7f92bc0af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:14.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 shutdown_connections 2023-10-21T16:22:14.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:14.941+0000 7f92c2a90640 1 -- 172.21.15.53:0/2853865152 wait complete. 2023-10-21T16:22:14.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:22:15.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.186+0000 7efe24abb640 1 Processor -- start 2023-10-21T16:22:15.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.186+0000 7efe24abb640 1 -- start start 2023-10-21T16:22:15.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe24abb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe200cc210 0x7efe200b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe24abb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200cacf0 con 0x7efe20159db0 2023-10-21T16:22:15.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe24abb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200ca3c0 con 0x7efe201599d0 2023-10-21T16:22:15.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe24abb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe200b10f0 con 0x7efe200cc210 2023-10-21T16:22:15.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe1ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe20159db0 0x7efe2006c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45722/0 (socket says 172.21.15.53:45722) 2023-10-21T16:22:15.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.188+0000 7efe1ffff640 1 -- 172.21.15.53:0/531739140 learned_addr learned my addr 172.21.15.53:0/531739140 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:15.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1effd640 1 --2- 172.21.15.53:0/531739140 >> v2:172.21.15.124:3300/0 conn(0x7efe200cc210 0x7efe200b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209030760 0 0) 0x7efe200cacf0 con 0x7efe20159db0 2023-10-21T16:22:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe08003520 con 0x7efe20159db0 2023-10-21T16:22:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529966539 0 0) 0x7efe08003520 con 0x7efe20159db0 2023-10-21T16:22:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 >> v2:172.21.15.124:3300/0 conn(0x7efe200cc210 msgr2=0x7efe200b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 --2- 172.21.15.53:0/531739140 >> v2:172.21.15.124:3300/0 conn(0x7efe200cc210 0x7efe200b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 >> v1:172.21.15.124:6789/0 conn(0x7efe201599d0 legacy=0x7efe200ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200ca6a0 con 0x7efe20159db0 2023-10-21T16:22:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7efe18001670 con 0x7efe20159db0 2023-10-21T16:22:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 >> v1:172.21.15.53:6789/0 conn(0x7efe20159db0 legacy=0x7efe2006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe080039c0 0x7efe08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe08005180 0x7efe0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200b10f0 con 0x7efe080047c0 2023-10-21T16:22:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0800c080 con 0x7efe080039c0 2023-10-21T16:22:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0800c240 con 0x7efe08005180 2023-10-21T16:22:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.189+0000 7efe1effd640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1effd640 1 -- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe08005180 msgr2=0x7efe0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1effd640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe08005180 0x7efe0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1effd640 1 -- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe080039c0 msgr2=0x7efe08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1effd640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe080039c0 0x7efe08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1effd640 1 -- 172.21.15.53:0/531739140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe08003520 con 0x7efe080047c0 2023-10-21T16:22:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.190+0000 7efe1ffff640 1 -- 172.21.15.53:0/531739140 reap_dead start 2023-10-21T16:22:15.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.191+0000 7efe1effd640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe08004c40 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7efe14009ac0 tx=0x7efe080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc138c34fe9b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.191+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1402c6b0 con 0x7efe080047c0 2023-10-21T16:22:15.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.191+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1402c810 con 0x7efe080047c0 2023-10-21T16:22:15.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.191+0000 7efe1e7fc640 1 -- 172.21.15.53:0/531739140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1402cba0 con 0x7efe080047c0 2023-10-21T16:22:15.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 -- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 msgr2=0x7efe08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe08004c40 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7efe14009ac0 tx=0x7efe080057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 -- 172.21.15.53:0/531739140 shutdown_connections 2023-10-21T16:22:15.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 --2- 172.21.15.53:0/531739140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe08004c40 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 -- 172.21.15.53:0/531739140 >> 172.21.15.53:0/531739140 conn(0x7efe200cb0c0 msgr2=0x7efe200cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:15.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 -- 172.21.15.53:0/531739140 shutdown_connections 2023-10-21T16:22:15.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.192+0000 7efe24abb640 1 -- 172.21.15.53:0/531739140 wait complete. 2023-10-21T16:22:15.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.194+0000 7efe24abb640 1 Processor -- start 2023-10-21T16:22:15.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.194+0000 7efe24abb640 1 -- start start 2023-10-21T16:22:15.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 0x7efe200ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe200abe10 0x7efe200c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200ac2c0 con 0x7efe080047c0 2023-10-21T16:22:15.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200c6700 con 0x7efe20149a00 2023-10-21T16:22:15.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe24abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2015e270 con 0x7efe200abe10 2023-10-21T16:22:15.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51024/0 (socket says 172.21.15.53:51024) 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe1f7fe640 1 -- 172.21.15.53:0/817204916 learned_addr learned my addr 172.21.15.53:0/817204916 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.195+0000 7efe1effd640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 0x7efe200ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1ffff640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe200abe10 0x7efe200c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 -- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe200abe10 msgr2=0x7efe200c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe200abe10 0x7efe200c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 -- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 msgr2=0x7efe200ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 0x7efe200ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 -- 172.21.15.53:0/817204916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2015d8a0 con 0x7efe080047c0 2023-10-21T16:22:15.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1effd640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 0x7efe200ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:15.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe1f7fe640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c7a0 tx=0x7efe1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10003b70 con 0x7efe080047c0 2023-10-21T16:22:15.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10003cf0 con 0x7efe080047c0 2023-10-21T16:22:15.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000aa30 con 0x7efe080047c0 2023-10-21T16:22:15.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.196+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2015cf70 con 0x7efe080047c0 2023-10-21T16:22:15.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.197+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2015d530 con 0x7efe080047c0 2023-10-21T16:22:15.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.198+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2015d6f0 con 0x7efe080047c0 2023-10-21T16:22:15.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.199+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe1000ab90 con 0x7efe080047c0 2023-10-21T16:22:15.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.199+0000 7efe07fff640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 0x7efe0c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.200+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe10089570 con 0x7efe080047c0 2023-10-21T16:22:15.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.200+0000 7efe1effd640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 0x7efe0c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.201+0000 7efe1effd640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 0x7efe0c06d4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe14036000 tx=0x7efe14030000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.204+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe100544f0 con 0x7efe080047c0 2023-10-21T16:22:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.400+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe2004e160 con 0x7efe080047c0 2023-10-21T16:22:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.404+0000 7efe07fff640 1 -- 172.21.15.53:0/817204916 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16848 (secure 0 0 0) 0x7efe10003e60 con 0x7efe080047c0 2023-10-21T16:22:15.406 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:15.406 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":160,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:22:13.932049+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:22:15.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 msgr2=0x7efe0c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 0x7efe0c06d4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe14036000 tx=0x7efe14030000 comp rx=0 tx=0).stop 2023-10-21T16:22:15.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 msgr2=0x7efe2015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c7a0 tx=0x7efe1000cc60 comp rx=0 tx=0).stop 2023-10-21T16:22:15.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 shutdown_connections 2023-10-21T16:22:15.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7efe0c06afe0 0x7efe0c06d4a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe200abe10 0x7efe200c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe20149a00 0x7efe200ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 --2- 172.21.15.53:0/817204916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe080047c0 0x7efe2015dd30 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 >> 172.21.15.53:0/817204916 conn(0x7efe200cb0c0 msgr2=0x7efe200b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:15.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 shutdown_connections 2023-10-21T16:22:15.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.407+0000 7efe24abb640 1 -- 172.21.15.53:0/817204916 wait complete. 2023-10-21T16:22:15.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T16:22:15.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.653+0000 7f01a094c640 1 Processor -- start 2023-10-21T16:22:15.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.654+0000 7f01a094c640 1 -- start start 2023-10-21T16:22:15.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.655+0000 7f01a094c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f019c06dfc0 0x7f019c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.655+0000 7f01a094c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f019c071070 0x7f019c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f01a094c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0b1cc0 con 0x7f019c0afbe0 2023-10-21T16:22:15.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f01a094c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019c0b2620 con 0x7f019c06dfc0 2023-10-21T16:22:15.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f01a094c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019c0b2060 con 0x7f019c071070 2023-10-21T16:22:15.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f019a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f019c0afbe0 0x7f019c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45726/0 (socket says 172.21.15.53:45726) 2023-10-21T16:22:15.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f019a59c640 1 -- 172.21.15.53:0/3822340930 learned_addr learned my addr 172.21.15.53:0/3822340930 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:15.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f0199d9b640 1 --2- 172.21.15.53:0/3822340930 >> v2:172.21.15.53:3300/0 conn(0x7f019c06dfc0 0x7f019c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f019ad9d640 1 --2- 172.21.15.53:0/3822340930 >> v2:172.21.15.124:3300/0 conn(0x7f019c071070 0x7f019c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f0199d9b640 1 -- 172.21.15.53:0/3822340930 >> v2:172.21.15.124:3300/0 conn(0x7f019c071070 msgr2=0x7f019c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:15.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f0199d9b640 1 --2- 172.21.15.53:0/3822340930 >> v2:172.21.15.124:3300/0 conn(0x7f019c071070 0x7f019c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f0199d9b640 1 -- 172.21.15.53:0/3822340930 >> v1:172.21.15.53:6789/0 conn(0x7f019c0afbe0 legacy=0x7f019c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.656+0000 7f0199d9b640 1 -- 172.21.15.53:0/3822340930 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0b2340 con 0x7f019c06dfc0 2023-10-21T16:22:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f0199d9b640 1 --2- 172.21.15.53:0/3822340930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c070b30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f019c0c8d10 tx=0x7f019002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb627b41af8d868b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f019959a640 1 -- 172.21.15.53:0/3822340930 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190030710 con 0x7f019c06dfc0 2023-10-21T16:22:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f019959a640 1 -- 172.21.15.53:0/3822340930 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019002e050 con 0x7f019c06dfc0 2023-10-21T16:22:15.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f019959a640 1 -- 172.21.15.53:0/3822340930 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190030a90 con 0x7f019c06dfc0 2023-10-21T16:22:15.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f01a094c640 1 -- 172.21.15.53:0/3822340930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 msgr2=0x7f019c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.658+0000 7f01a094c640 1 --2- 172.21.15.53:0/3822340930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c070b30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f019c0c8d10 tx=0x7f019002b710 comp rx=0 tx=0).stop 2023-10-21T16:22:15.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 -- 172.21.15.53:0/3822340930 shutdown_connections 2023-10-21T16:22:15.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 --2- 172.21.15.53:0/3822340930 >> v2:172.21.15.124:3300/0 conn(0x7f019c071070 0x7f019c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 --2- 172.21.15.53:0/3822340930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c070b30 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 -- 172.21.15.53:0/3822340930 >> 172.21.15.53:0/3822340930 conn(0x7f019c06a5d0 msgr2=0x7f019c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:15.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 -- 172.21.15.53:0/3822340930 shutdown_connections 2023-10-21T16:22:15.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.659+0000 7f01a094c640 1 -- 172.21.15.53:0/3822340930 wait complete. 2023-10-21T16:22:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.660+0000 7f01a094c640 1 Processor -- start 2023-10-21T16:22:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.661+0000 7f01a094c640 1 -- start start 2023-10-21T16:22:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.661+0000 7f01a094c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f01a094c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f019c071070 0x7f019c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f01a094c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 0x7f019c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f01a094c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c14b0e0 con 0x7f019c06dfc0 2023-10-21T16:22:15.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f01a094c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0c9a50 con 0x7f019c071070 2023-10-21T16:22:15.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f01a094c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0c9d30 con 0x7f019c0afbe0 2023-10-21T16:22:15.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51046/0 (socket says 172.21.15.53:51046) 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 -- 172.21.15.53:0/1471234571 learned_addr learned my addr 172.21.15.53:0/1471234571 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019ad9d640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 0x7f019c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f0199d9b640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f019c071070 0x7f019c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 -- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 msgr2=0x7f019c15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 0x7f019c15ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 -- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f019c071070 msgr2=0x7f019c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:15.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f019c071070 0x7f019c15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:15.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019a59c640 1 -- 172.21.15.53:0/1471234571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190009400 con 0x7f019c06dfc0 2023-10-21T16:22:15.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.662+0000 7f019ad9d640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 0x7f019c15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:15.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f019a59c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c5f0 tx=0x7f018c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0074e0 con 0x7f019c06dfc0 2023-10-21T16:22:15.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c007640 con 0x7f019c06dfc0 2023-10-21T16:22:15.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c004a00 con 0x7f019c06dfc0 2023-10-21T16:22:15.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0ca010 con 0x7f019c06dfc0 2023-10-21T16:22:15.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.663+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c0ca5d0 con 0x7f019c06dfc0 2023-10-21T16:22:15.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.664+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c04e160 con 0x7f019c06dfc0 2023-10-21T16:22:15.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.666+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f018c004b60 con 0x7f019c06dfc0 2023-10-21T16:22:15.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.666+0000 7f018affd640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 0x7f017c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:15.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.667+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f018c088c20 con 0x7f019c06dfc0 2023-10-21T16:22:15.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.667+0000 7f0199d9b640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 0x7f017c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:15.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.668+0000 7f0199d9b640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 0x7f017c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0190034000 tx=0x7f0190008030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:15.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.671+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f018c053a30 con 0x7f019c06dfc0 2023-10-21T16:22:15.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:15.902+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f019c0c5280 con 0x7f019c06dfc0 2023-10-21T16:22:16.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.508+0000 7f018affd640 1 -- 172.21.15.53:0/1471234571 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f018c018020 con 0x7f019c06dfc0 2023-10-21T16:22:16.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.512+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 msgr2=0x7f017c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.512+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 0x7f017c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0190034000 tx=0x7f0190008030 comp rx=0 tx=0).stop 2023-10-21T16:22:16.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.512+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 msgr2=0x7f019c14aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.512+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c5f0 tx=0x7f018c00cab0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 shutdown_connections 2023-10-21T16:22:16.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f017c06af90 0x7f017c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f019c0afbe0 0x7f019c15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f019c071070 0x7f019c15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 --2- 172.21.15.53:0/1471234571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f019c06dfc0 0x7f019c14aba0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 >> 172.21.15.53:0/1471234571 conn(0x7f019c06a5d0 msgr2=0x7f019c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:16.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 shutdown_connections 2023-10-21T16:22:16.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.513+0000 7f01a094c640 1 -- 172.21.15.53:0/1471234571 wait complete. 2023-10-21T16:22:16.537 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T16:22:16.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T16:22:16.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.768+0000 7f10c480d640 1 Processor -- start 2023-10-21T16:22:16.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.768+0000 7f10c480d640 1 -- start start 2023-10-21T16:22:16.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10c480d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f10c015a820 0x7f10c00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10c480d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f10c015ac00 0x7f10c00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10c480d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00cab60 con 0x7f10c00ca680 2023-10-21T16:22:16.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10c480d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00cb960 con 0x7f10c015ac00 2023-10-21T16:22:16.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10c480d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00cbdf0 con 0x7f10c015a820 2023-10-21T16:22:16.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.769+0000 7f10bffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f10c015ac00 0x7f10c00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f10c015ac00 0x7f10c00b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51060/0 (socket says 172.21.15.53:51060) 2023-10-21T16:22:16.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10beffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10c00ca680 0x7f10c00c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45738/0 (socket says 172.21.15.53:45738) 2023-10-21T16:22:16.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 -- 172.21.15.53:0/2823936134 learned_addr learned my addr 172.21.15.53:0/2823936134 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:16.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bf7fe640 1 --2- 172.21.15.53:0/2823936134 >> v2:172.21.15.124:3300/0 conn(0x7f10c015a820 0x7f10c00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 -- 172.21.15.53:0/2823936134 >> v2:172.21.15.124:3300/0 conn(0x7f10c015a820 msgr2=0x7f10c00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:16.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 --2- 172.21.15.53:0/2823936134 >> v2:172.21.15.124:3300/0 conn(0x7f10c015a820 0x7f10c00aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 -- 172.21.15.53:0/2823936134 >> v1:172.21.15.53:6789/0 conn(0x7f10c00ca680 legacy=0x7f10c00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10bffff640 1 -- 172.21.15.53:0/2823936134 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00cb030 con 0x7f10c015ac00 2023-10-21T16:22:16.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.770+0000 7f10be7fc640 1 -- 172.21.15.53:0/2823936134 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217129441 0 0) 0x7f10c00cab60 con 0x7f10c00ca680 2023-10-21T16:22:16.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.771+0000 7f10bffff640 1 --2- 172.21.15.53:0/2823936134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00b04b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f10c006da70 tx=0x7f10b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f505adeebe67c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10be7fc640 1 -- 172.21.15.53:0/2823936134 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8031870 con 0x7f10c015ac00 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10be7fc640 1 -- 172.21.15.53:0/2823936134 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b80319d0 con 0x7f10c015ac00 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10be7fc640 1 -- 172.21.15.53:0/2823936134 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8031d60 con 0x7f10c015ac00 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 -- 172.21.15.53:0/2823936134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 msgr2=0x7f10c00b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 --2- 172.21.15.53:0/2823936134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00b04b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f10c006da70 tx=0x7f10b802b910 comp rx=0 tx=0).stop 2023-10-21T16:22:16.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 -- 172.21.15.53:0/2823936134 shutdown_connections 2023-10-21T16:22:16.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 --2- 172.21.15.53:0/2823936134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00b04b0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 --2- 172.21.15.53:0/2823936134 >> v2:172.21.15.124:3300/0 conn(0x7f10c015a820 0x7f10c00aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 -- 172.21.15.53:0/2823936134 >> 172.21.15.53:0/2823936134 conn(0x7f10c006c050 msgr2=0x7f10c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:16.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 -- 172.21.15.53:0/2823936134 shutdown_connections 2023-10-21T16:22:16.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.772+0000 7f10c480d640 1 -- 172.21.15.53:0/2823936134 wait complete. 2023-10-21T16:22:16.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.774+0000 7f10c480d640 1 Processor -- start 2023-10-21T16:22:16.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.774+0000 7f10c480d640 1 -- start start 2023-10-21T16:22:16.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c00ca680 0x7f10c014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c015a820 0x7f10c015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c2f30 con 0x7f10c015ac00 2023-10-21T16:22:16.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c3210 con 0x7f10c00ca680 2023-10-21T16:22:16.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.775+0000 7f10c480d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00c34f0 con 0x7f10c015a820 2023-10-21T16:22:16.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51070/0 (socket says 172.21.15.53:51070) 2023-10-21T16:22:16.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 -- 172.21.15.53:0/118512529 learned_addr learned my addr 172.21.15.53:0/118512529 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:16.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bf7fe640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c00ca680 0x7f10c014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10beffd640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c015a820 0x7f10c015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 -- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c015a820 msgr2=0x7f10c015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c015a820 0x7f10c015d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 -- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c00ca680 msgr2=0x7f10c014b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c00ca680 0x7f10c014b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.776+0000 7f10bffff640 1 -- 172.21.15.53:0/118512529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80094d0 con 0x7f10c015ac00 2023-10-21T16:22:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10bffff640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f10b80090b0 tx=0x7f10b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b803ac60 con 0x7f10c015ac00 2023-10-21T16:22:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b803adc0 con 0x7f10c015ac00 2023-10-21T16:22:16.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c00c37d0 con 0x7f10c015ac00 2023-10-21T16:22:16.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8044630 con 0x7f10c015ac00 2023-10-21T16:22:16.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.777+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c014be30 con 0x7f10c015ac00 2023-10-21T16:22:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.778+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c00c3cf0 con 0x7f10c015ac00 2023-10-21T16:22:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.779+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f10b804d450 con 0x7f10c015ac00 2023-10-21T16:22:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.780+0000 7f10b7fff640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 0x7f10a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.780+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10b802d040 con 0x7f10c015ac00 2023-10-21T16:22:16.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.781+0000 7f10bf7fe640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 0x7f10a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:16.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.782+0000 7f10bf7fe640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 0x7f10a006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009a60 tx=0x7f10b0003990 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:16.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:16.785+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10b8076cc0 con 0x7f10c015ac00 2023-10-21T16:22:17.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.024+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f10c0074150 con 0x7f10c015ac00 2023-10-21T16:22:17.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.512+0000 7f10b7fff640 1 -- 172.21.15.53:0/118512529 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f10b807a2e0 con 0x7f10c015ac00 2023-10-21T16:22:17.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.515+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 msgr2=0x7f10a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.515+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 0x7f10a006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009a60 tx=0x7f10b0003990 comp rx=0 tx=0).stop 2023-10-21T16:22:17.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.515+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 msgr2=0x7f10c00ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.515+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f10b80090b0 tx=0x7f10b80046c0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 shutdown_connections 2023-10-21T16:22:17.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f10a006af90 0x7f10a006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10c015ac00 0x7f10c00ce3f0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10c015a820 0x7f10c015d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 --2- 172.21.15.53:0/118512529 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10c00ca680 0x7f10c014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 >> 172.21.15.53:0/118512529 conn(0x7f10c006c050 msgr2=0x7f10c0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:17.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 shutdown_connections 2023-10-21T16:22:17.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.516+0000 7f10c480d640 1 -- 172.21.15.53:0/118512529 wait complete. 2023-10-21T16:22:17.541 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T16:22:17.542 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:22:17.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.767+0000 7f2645733640 1 Processor -- start 2023-10-21T16:22:17.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.767+0000 7f2645733640 1 -- start start 2023-10-21T16:22:17.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f2645733640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26400ad790 0x7f26400702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:17.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f2645733640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264006dee0 con 0x7f264015e330 2023-10-21T16:22:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f2645733640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264015d850 con 0x7f2640159ac0 2023-10-21T16:22:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f2645733640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264006dcf0 con 0x7f26400ad790 2023-10-21T16:22:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f2637fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f26400ad790 0x7f26400702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f263f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f264015e330 0x7f26400c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45746/0 (socket says 172.21.15.53:45746) 2023-10-21T16:22:17.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.770+0000 7f263f7fe640 1 -- 172.21.15.53:0/98820980 learned_addr learned my addr 172.21.15.53:0/98820980 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.771+0000 7f2637fff640 1 -- 172.21.15.53:0/98820980 >> v1:172.21.15.124:6789/0 conn(0x7f2640159ac0 legacy=0x7f264006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.771+0000 7f2637fff640 1 -- 172.21.15.53:0/98820980 >> v1:172.21.15.53:6789/0 conn(0x7f264015e330 legacy=0x7f26400c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.771+0000 7f2637fff640 1 -- 172.21.15.53:0/98820980 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264015db30 con 0x7f26400ad790 2023-10-21T16:22:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.771+0000 7f263e7fc640 1 -- 172.21.15.53:0/98820980 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601010609 0 0) 0x7f264006dee0 con 0x7f264015e330 2023-10-21T16:22:17.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.772+0000 7f2637fff640 1 --2- 172.21.15.53:0/98820980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26400ad790 0x7f26400702d0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f26280097a0 tx=0x7f262802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709af208d6a64d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:17.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.772+0000 7f263e7fc640 1 -- 172.21.15.53:0/98820980 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628030610 con 0x7f26400ad790 2023-10-21T16:22:17.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.772+0000 7f263e7fc640 1 -- 172.21.15.53:0/98820980 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628030770 con 0x7f26400ad790 2023-10-21T16:22:17.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.772+0000 7f263e7fc640 1 -- 172.21.15.53:0/98820980 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628030b00 con 0x7f26400ad790 2023-10-21T16:22:17.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 -- 172.21.15.53:0/98820980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26400ad790 msgr2=0x7f26400702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 --2- 172.21.15.53:0/98820980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26400ad790 0x7f26400702d0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f26280097a0 tx=0x7f262802b960 comp rx=0 tx=0).stop 2023-10-21T16:22:17.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 -- 172.21.15.53:0/98820980 shutdown_connections 2023-10-21T16:22:17.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 --2- 172.21.15.53:0/98820980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26400ad790 0x7f26400702d0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 -- 172.21.15.53:0/98820980 >> 172.21.15.53:0/98820980 conn(0x7f264006a380 msgr2=0x7f26400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 -- 172.21.15.53:0/98820980 shutdown_connections 2023-10-21T16:22:17.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.773+0000 7f2645733640 1 -- 172.21.15.53:0/98820980 wait complete. 2023-10-21T16:22:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.775+0000 7f2645733640 1 Processor -- start 2023-10-21T16:22:17.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.775+0000 7f2645733640 1 -- start start 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26400ad790 0x7f2640153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2640159ac0 0x7f2640154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264015a020 con 0x7f264015e330 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2640155230 con 0x7f2640159ac0 2023-10-21T16:22:17.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f2645733640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2640154110 con 0x7f26400ad790 2023-10-21T16:22:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f263f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f263f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51084/0 (socket says 172.21.15.53:51084) 2023-10-21T16:22:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.776+0000 7f263f7fe640 1 -- 172.21.15.53:0/4077850269 learned_addr learned my addr 172.21.15.53:0/4077850269 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263effd640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26400ad790 0x7f2640153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:17.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 -- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26400ad790 msgr2=0x7f2640153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26400ad790 0x7f2640153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 -- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2640159ac0 msgr2=0x7f2640154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:17.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2640159ac0 0x7f2640154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:17.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 -- 172.21.15.53:0/4077850269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628009400 con 0x7f264015e330 2023-10-21T16:22:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.777+0000 7f263f7fe640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f263800c700 tx=0x7f263800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.778+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800ce10 con 0x7f264015e330 2023-10-21T16:22:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.778+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263800a640 con 0x7f264015e330 2023-10-21T16:22:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.778+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800aa70 con 0x7f264015e330 2023-10-21T16:22:17.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.778+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26401543f0 con 0x7f264015e330 2023-10-21T16:22:17.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.778+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26401549b0 con 0x7f264015e330 2023-10-21T16:22:17.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.779+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2640070790 con 0x7f264015e330 2023-10-21T16:22:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.780+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f263800abd0 con 0x7f264015e330 2023-10-21T16:22:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.781+0000 7f26377fe640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 0x7f262006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.781+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2638088ee0 con 0x7f264015e330 2023-10-21T16:22:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.782+0000 7f263effd640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 0x7f262006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:17.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.783+0000 7f263effd640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 0x7f262006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2630000f60 tx=0x7f2630009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:17.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:17.785+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2638053c50 con 0x7f264015e330 2023-10-21T16:22:18.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.018+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2640067490 con 0x7f264015e330 2023-10-21T16:22:18.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.019+0000 7f26377fe640 1 -- 172.21.15.53:0/4077850269 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f2638018070 con 0x7f264015e330 2023-10-21T16:22:18.021 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:18.021 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T16:22:18.022 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T16:22:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 msgr2=0x7f262006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 0x7f262006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2630000f60 tx=0x7f2630009040 comp rx=0 tx=0).stop 2023-10-21T16:22:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 msgr2=0x7f26400af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f263800c700 tx=0x7f263800cbc0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 shutdown_connections 2023-10-21T16:22:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f262006af90 0x7f262006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f264015e330 0x7f26400af270 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2640159ac0 0x7f2640154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 --2- 172.21.15.53:0/4077850269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26400ad790 0x7f2640153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 >> 172.21.15.53:0/4077850269 conn(0x7f264006a380 msgr2=0x7f26400cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 shutdown_connections 2023-10-21T16:22:18.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.022+0000 7f2645733640 1 -- 172.21.15.53:0/4077850269 wait complete. 2023-10-21T16:22:18.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T16:22:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.266+0000 7f3cb6511640 1 Processor -- start 2023-10-21T16:22:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.266+0000 7f3cb6511640 1 -- start start 2023-10-21T16:22:18.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.268+0000 7f3cb6511640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0161320 0x7f3cb00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.268+0000 7f3cb6511640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3cb0164f70 0x7f3cb00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.268+0000 7f3cb6511640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb0165470 con 0x7f3cb006c710 2023-10-21T16:22:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.268+0000 7f3cb6511640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb0165750 con 0x7f3cb0164f70 2023-10-21T16:22:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.268+0000 7f3cb6511640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0165a30 con 0x7f3cb0161320 2023-10-21T16:22:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb4d0e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3cb0164f70 0x7f3cb00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb5d10640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3cb006c710 0x7f3cb00c95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45750/0 (socket says 172.21.15.53:45750) 2023-10-21T16:22:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb5d10640 1 -- 172.21.15.53:0/1293964489 learned_addr learned my addr 172.21.15.53:0/1293964489 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:18.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb550f640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0161320 0x7f3cb00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb4d0e640 1 -- 172.21.15.53:0/1293964489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0161320 msgr2=0x7f3cb00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb4d0e640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0161320 0x7f3cb00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb4d0e640 1 -- 172.21.15.53:0/1293964489 >> v1:172.21.15.53:6789/0 conn(0x7f3cb006c710 legacy=0x7f3cb00c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3cb4d0e640 1 -- 172.21.15.53:0/1293964489 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0165d10 con 0x7f3cb0164f70 2023-10-21T16:22:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.269+0000 7f3ca7fff640 1 -- 172.21.15.53:0/1293964489 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257133828 0 0) 0x7f3cb0165470 con 0x7f3cb006c710 2023-10-21T16:22:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.270+0000 7f3cb4d0e640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb0164f70 0x7f3cb00b00c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00097a0 tx=0x7f3ca002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afa004d3a6e296a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3ca7fff640 1 -- 172.21.15.53:0/1293964489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0030800 con 0x7f3cb0164f70 2023-10-21T16:22:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3ca7fff640 1 -- 172.21.15.53:0/1293964489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca0030960 con 0x7f3cb0164f70 2023-10-21T16:22:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3ca7fff640 1 -- 172.21.15.53:0/1293964489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0030cf0 con 0x7f3cb0164f70 2023-10-21T16:22:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 -- 172.21.15.53:0/1293964489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb0164f70 msgr2=0x7f3cb00b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb0164f70 0x7f3cb00b00c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00097a0 tx=0x7f3ca002b940 comp rx=0 tx=0).stop 2023-10-21T16:22:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 -- 172.21.15.53:0/1293964489 shutdown_connections 2023-10-21T16:22:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb0164f70 0x7f3cb00b00c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 --2- 172.21.15.53:0/1293964489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0161320 0x7f3cb00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 -- 172.21.15.53:0/1293964489 >> 172.21.15.53:0/1293964489 conn(0x7f3cb00cb0c0 msgr2=0x7f3cb00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 -- 172.21.15.53:0/1293964489 shutdown_connections 2023-10-21T16:22:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.271+0000 7f3cb6511640 1 -- 172.21.15.53:0/1293964489 wait complete. 2023-10-21T16:22:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.273+0000 7f3cb6511640 1 Processor -- start 2023-10-21T16:22:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.273+0000 7f3cb6511640 1 -- start start 2023-10-21T16:22:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0161320 0x7f3cb00b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0164f70 0x7f3cb0162180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00b3b40 con 0x7f3cb006c710 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01626c0 con 0x7f3cb0161320 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.274+0000 7f3cb6511640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01fa500 con 0x7f3cb0164f70 2023-10-21T16:22:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51100/0 (socket says 172.21.15.53:51100) 2023-10-21T16:22:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 -- 172.21.15.53:0/136844760 learned_addr learned my addr 172.21.15.53:0/136844760 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 -- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0164f70 msgr2=0x7f3cb0162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0164f70 0x7f3cb0162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 -- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0161320 msgr2=0x7f3cb00b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0161320 0x7f3cb00b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.275+0000 7f3cb550f640 1 -- 172.21.15.53:0/136844760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca0009400 con 0x7f3cb006c710 2023-10-21T16:22:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3cb550f640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d800 tx=0x7f3cac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00b6c0 con 0x7f3cb006c710 2023-10-21T16:22:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac00b820 con 0x7f3cb006c710 2023-10-21T16:22:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00bbb0 con 0x7f3cb006c710 2023-10-21T16:22:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb01fa7e0 con 0x7f3cb006c710 2023-10-21T16:22:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.276+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb01fada0 con 0x7f3cb006c710 2023-10-21T16:22:18.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.277+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb004e160 con 0x7f3cb006c710 2023-10-21T16:22:18.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.278+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3cac00bd10 con 0x7f3cb006c710 2023-10-21T16:22:18.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.279+0000 7f3ca5ffb640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 0x7f3c9406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.279+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cac089f60 con 0x7f3cb006c710 2023-10-21T16:22:18.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.280+0000 7f3cb4d0e640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 0x7f3c9406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.281+0000 7f3cb4d0e640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 0x7f3c9406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0036000 tx=0x7f3ca0030000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.283+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cac054cd0 con 0x7f3cb006c710 2023-10-21T16:22:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.476+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cb00c8a90 con 0x7f3cb006c710 2023-10-21T16:22:18.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.479+0000 7f3ca5ffb640 1 -- 172.21.15.53:0/136844760 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16835 (secure 0 0 0) 0x7f3cac0582f0 con 0x7f3cb006c710 2023-10-21T16:22:18.481 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:18.481 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":161,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T16:22:15.488109+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T16:22:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 msgr2=0x7f3c9406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 0x7f3c9406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0036000 tx=0x7f3ca0030000 comp rx=0 tx=0).stop 2023-10-21T16:22:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 msgr2=0x7f3cb015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d800 tx=0x7f3cac00dcc0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 shutdown_connections 2023-10-21T16:22:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f3c9406af90 0x7f3c9406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cb0164f70 0x7f3cb0162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cb0161320 0x7f3cb00b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 --2- 172.21.15.53:0/136844760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cb006c710 0x7f3cb015bed0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 >> 172.21.15.53:0/136844760 conn(0x7f3cb00cb0c0 msgr2=0x7f3cb0160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:18.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 shutdown_connections 2023-10-21T16:22:18.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.481+0000 7f3cb6511640 1 -- 172.21.15.53:0/136844760 wait complete. 2023-10-21T16:22:18.502 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T16:22:18.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:22:18.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.725+0000 7f63c98f8640 1 Processor -- start 2023-10-21T16:22:18.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.726+0000 7f63c98f8640 1 -- start start 2023-10-21T16:22:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.727+0000 7f63c98f8640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f63c40b07d0 0x7f63c4165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.727+0000 7f63c98f8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c415f7d0 con 0x7f63c40adc70 2023-10-21T16:22:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.727+0000 7f63c98f8640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c415efb0 con 0x7f63c415ab30 2023-10-21T16:22:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.727+0000 7f63c98f8640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c415f210 con 0x7f63c40b07d0 2023-10-21T16:22:18.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.728+0000 7f63c3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f63c40adc70 0x7f63c40b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45754/0 (socket says 172.21.15.53:45754) 2023-10-21T16:22:18.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.728+0000 7f63c3fff640 1 -- 172.21.15.53:0/212505502 learned_addr learned my addr 172.21.15.53:0/212505502 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:18.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.728+0000 7f63c90f7640 1 --2- 172.21.15.53:0/212505502 >> v2:172.21.15.124:3300/0 conn(0x7f63c40b07d0 0x7f63c4165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.728+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659962853 0 0) 0x7f63c415f7d0 con 0x7f63c40adc70 2023-10-21T16:22:18.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.728+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac003520 con 0x7f63c40adc70 2023-10-21T16:22:18.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287562392 0 0) 0x7f63ac003520 con 0x7f63c40adc70 2023-10-21T16:22:18.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 >> v2:172.21.15.124:3300/0 conn(0x7f63c40b07d0 msgr2=0x7f63c4165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:18.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 --2- 172.21.15.53:0/212505502 >> v2:172.21.15.124:3300/0 conn(0x7f63c40b07d0 0x7f63c4165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 >> v1:172.21.15.124:6789/0 conn(0x7f63c415ab30 legacy=0x7f63c415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c415f4f0 con 0x7f63c40adc70 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f63b8001670 con 0x7f63c40adc70 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 >> v1:172.21.15.53:6789/0 conn(0x7f63c40adc70 legacy=0x7f63c40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c415f210 con 0x7f63ac005180 2023-10-21T16:22:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac00c020 con 0x7f63ac003970 2023-10-21T16:22:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac00c1e0 con 0x7f63ac0047c0 2023-10-21T16:22:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c90f7640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c90f7640 1 -- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac0047c0 msgr2=0x7f63ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:18.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c90f7640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac0047c0 0x7f63ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c90f7640 1 -- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac003970 msgr2=0x7f63ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:18.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.729+0000 7f63c90f7640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac003970 0x7f63ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.730+0000 7f63c90f7640 1 -- 172.21.15.53:0/212505502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac003520 con 0x7f63ac005180 2023-10-21T16:22:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.730+0000 7f63c90f7640 1 -- 172.21.15.53:0/212505502 reap_dead start 2023-10-21T16:22:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c90f7640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63ac00b980 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bba97591f62fae1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc004020 con 0x7f63ac005180 2023-10-21T16:22:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc004180 con 0x7f63ac005180 2023-10-21T16:22:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c37fe640 1 -- 172.21.15.53:0/212505502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc02c710 con 0x7f63ac005180 2023-10-21T16:22:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c98f8640 1 -- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 msgr2=0x7f63ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.731+0000 7f63c98f8640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63ac00b980 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ac0 tx=0x7f63ac0057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.732+0000 7f63c98f8640 1 -- 172.21.15.53:0/212505502 shutdown_connections 2023-10-21T16:22:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.732+0000 7f63c98f8640 1 --2- 172.21.15.53:0/212505502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63ac00b980 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.732+0000 7f63c98f8640 1 -- 172.21.15.53:0/212505502 >> 172.21.15.53:0/212505502 conn(0x7f63c40cb0c0 msgr2=0x7f63c40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.732+0000 7f63c98f8640 1 -- 172.21.15.53:0/212505502 shutdown_connections 2023-10-21T16:22:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.732+0000 7f63c98f8640 1 -- 172.21.15.53:0/212505502 wait complete. 2023-10-21T16:22:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.733+0000 7f63c98f8640 1 Processor -- start 2023-10-21T16:22:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.734+0000 7f63c98f8640 1 -- start start 2023-10-21T16:22:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.734+0000 7f63c98f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c98f8640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63c4151850 0x7f63c40c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c98f8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63c40cc4f0 0x7f63c40cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c98f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4152540 con 0x7f63ac005180 2023-10-21T16:22:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c98f8640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40c22d0 con 0x7f63c4151850 2023-10-21T16:22:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c98f8640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40c25b0 con 0x7f63c40cc4f0 2023-10-21T16:22:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51122/0 (socket says 172.21.15.53:51122) 2023-10-21T16:22:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 -- 172.21.15.53:0/3759973414 learned_addr learned my addr 172.21.15.53:0/3759973414 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 -- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63c40cc4f0 msgr2=0x7f63c40cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c90f7640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63c40cc4f0 0x7f63c40cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63c40cc4f0 0x7f63c40cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 -- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63c4151850 msgr2=0x7f63c40c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63c4151850 0x7f63c40c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.735+0000 7f63c88f6640 1 -- 172.21.15.53:0/3759973414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40c2890 con 0x7f63ac005180 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c88f6640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f63b400c800 tx=0x7f63b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4003b90 con 0x7f63ac005180 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4003cf0 con 0x7f63ac005180 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b400aa90 con 0x7f63ac005180 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c40c2b70 con 0x7f63ac005180 2023-10-21T16:22:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.736+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c40cceb0 con 0x7f63ac005180 2023-10-21T16:22:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.737+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c404e160 con 0x7f63ac005180 2023-10-21T16:22:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.738+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f63b400abf0 con 0x7f63ac005180 2023-10-21T16:22:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.739+0000 7f63c17fa640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 0x7f63b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.740+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63b4089830 con 0x7f63ac005180 2023-10-21T16:22:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.740+0000 7f63c3fff640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 0x7f63b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:18.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.741+0000 7f63c3fff640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 0x7f63b006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b80011e0 tx=0x7f63b8001550 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:18.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.744+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63b4054640 con 0x7f63ac005180 2023-10-21T16:22:18.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.933+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 --> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63c406c830 con 0x7f63b006af90 2023-10-21T16:22:18.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.962+0000 7f63c17fa640 1 -- 172.21.15.53:0/3759973414 <== mgr.36626 v2:172.21.15.124:6832/870299393 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287087 (secure 0 0 0) 0x7f63b80070d0 con 0x7f63b006af90 2023-10-21T16:22:18.968 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:18.969 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:22:18.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.968+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 msgr2=0x7f63b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.968+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 0x7f63b006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b80011e0 tx=0x7f63b8001550 comp rx=0 tx=0).stop 2023-10-21T16:22:18.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.968+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 msgr2=0x7f63c40c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.968+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f63b400c800 tx=0x7f63b400ccc0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 shutdown_connections 2023-10-21T16:22:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f63b006af90 0x7f63b006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63c40cc4f0 0x7f63c40cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63c4151850 0x7f63c40c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 --2- 172.21.15.53:0/3759973414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ac005180 0x7f63c40c7bd0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:18.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 >> 172.21.15.53:0/3759973414 conn(0x7f63c40cb0c0 msgr2=0x7f63c406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:18.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 shutdown_connections 2023-10-21T16:22:18.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:18.969+0000 7f63c98f8640 1 -- 172.21.15.53:0/3759973414 wait complete. 2023-10-21T16:22:18.992 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:22:17.646871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":451,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4812,"ondisk_log_size":4812,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341272,"kb_used_data":16844,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754633448,"statfs":{"total":773094113280,"available":772744650752,"internally_reserved":0,"allocated":17248256,"data_stored":9324448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":332058169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.208339"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504236+0000","last_change":"2023-10-21T16:22:14.504757+0000","last_active":"2023-10-21T16:22:15.504236+0000","last_peered":"2023-10-21T16:22:15.504236+0000","last_clean":"2023-10-21T16:22:15.504236+0000","last_became_active":"2023-10-21T16:22:14.504451+0000","last_became_peered":"2023-10-21T16:22:14.504451+0000","last_unstale":"2023-10-21T16:22:15.504236+0000","last_undegraded":"2023-10-21T16:22:15.504236+0000","last_fullsized":"2023-10-21T16:22:15.504236+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:03.268724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501479+0000","last_change":"2023-10-21T16:22:14.493483+0000","last_active":"2023-10-21T16:22:15.501479+0000","last_peered":"2023-10-21T16:22:15.501479+0000","last_clean":"2023-10-21T16:22:15.501479+0000","last_became_active":"2023-10-21T16:22:14.492712+0000","last_became_peered":"2023-10-21T16:22:14.492712+0000","last_unstale":"2023-10-21T16:22:15.501479+0000","last_undegraded":"2023-10-21T16:22:15.501479+0000","last_fullsized":"2023-10-21T16:22:15.501479+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:59:50.197807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513240+0000","last_change":"2023-10-21T16:22:14.511503+0000","last_active":"2023-10-21T16:22:14.513240+0000","last_peered":"2023-10-21T16:22:14.513240+0000","last_clean":"2023-10-21T16:22:14.513240+0000","last_became_active":"2023-10-21T16:22:14.510923+0000","last_became_peered":"2023-10-21T16:22:14.510923+0000","last_unstale":"2023-10-21T16:22:14.513240+0000","last_undegraded":"2023-10-21T16:22:14.513240+0000","last_fullsized":"2023-10-21T16:22:14.513240+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:25.550536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503264+0000","last_change":"2023-10-21T16:22:14.503525+0000","last_active":"2023-10-21T16:22:15.503264+0000","last_peered":"2023-10-21T16:22:15.503264+0000","last_clean":"2023-10-21T16:22:15.503264+0000","last_became_active":"2023-10-21T16:22:14.503255+0000","last_became_peered":"2023-10-21T16:22:14.503255+0000","last_unstale":"2023-10-21T16:22:15.503264+0000","last_undegraded":"2023-10-21T16:22:15.503264+0000","last_fullsized":"2023-10-21T16:22:15.503264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:55.610506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508635+0000","last_change":"2023-10-21T16:22:14.500503+0000","last_active":"2023-10-21T16:22:14.508635+0000","last_peered":"2023-10-21T16:22:14.508635+0000","last_clean":"2023-10-21T16:22:14.508635+0000","last_became_active":"2023-10-21T16:22:14.499935+0000","last_became_peered":"2023-10-21T16:22:14.499935+0000","last_unstale":"2023-10-21T16:22:14.508635+0000","last_undegraded":"2023-10-21T16:22:14.508635+0000","last_fullsized":"2023-10-21T16:22:14.508635+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.519772+0000","last_change":"2023-10-21T16:22:14.516762+0000","last_active":"2023-10-21T16:22:14.519772+0000","last_peered":"2023-10-21T16:22:14.519772+0000","last_clean":"2023-10-21T16:22:14.519772+0000","last_became_active":"2023-10-21T16:22:14.516478+0000","last_became_peered":"2023-10-21T16:22:14.516478+0000","last_unstale":"2023-10-21T16:22:14.519772+0000","last_undegraded":"2023-10-21T16:22:14.519772+0000","last_fullsized":"2023-10-21T16:22:14.519772+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:40.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515725+0000","last_change":"2023-10-21T16:22:14.512324+0000","last_active":"2023-10-21T16:22:14.515725+0000","last_peered":"2023-10-21T16:22:14.515725+0000","last_clean":"2023-10-21T16:22:14.515725+0000","last_became_active":"2023-10-21T16:22:14.511444+0000","last_became_peered":"2023-10-21T16:22:14.511444+0000","last_unstale":"2023-10-21T16:22:14.515725+0000","last_undegraded":"2023-10-21T16:22:14.515725+0000","last_fullsized":"2023-10-21T16:22:14.515725+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:46.439939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.505919+0000","last_change":"2023-10-21T16:22:14.505159+0000","last_active":"2023-10-21T16:22:15.505919+0000","last_peered":"2023-10-21T16:22:15.505919+0000","last_clean":"2023-10-21T16:22:15.505919+0000","last_became_active":"2023-10-21T16:22:14.500209+0000","last_became_peered":"2023-10-21T16:22:14.500209+0000","last_unstale":"2023-10-21T16:22:15.505919+0000","last_undegraded":"2023-10-21T16:22:15.505919+0000","last_fullsized":"2023-10-21T16:22:15.505919+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:02.078960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515637+0000","last_change":"2023-10-21T16:22:14.501707+0000","last_active":"2023-10-21T16:22:14.515637+0000","last_peered":"2023-10-21T16:22:14.515637+0000","last_clean":"2023-10-21T16:22:14.515637+0000","last_became_active":"2023-10-21T16:22:14.501420+0000","last_became_peered":"2023-10-21T16:22:14.501420+0000","last_unstale":"2023-10-21T16:22:14.515637+0000","last_undegraded":"2023-10-21T16:22:14.515637+0000","last_fullsized":"2023-10-21T16:22:14.515637+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:19.787563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515853+0000","last_change":"2023-10-21T16:22:14.510464+0000","last_active":"2023-10-21T16:22:14.515853+0000","last_peered":"2023-10-21T16:22:14.515853+0000","last_clean":"2023-10-21T16:22:14.515853+0000","last_became_active":"2023-10-21T16:22:14.510154+0000","last_became_peered":"2023-10-21T16:22:14.510154+0000","last_unstale":"2023-10-21T16:22:14.515853+0000","last_undegraded":"2023-10-21T16:22:14.515853+0000","last_fullsized":"2023-10-21T16:22:14.515853+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:55.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503673+0000","last_change":"2023-10-21T16:22:14.510567+0000","last_active":"2023-10-21T16:22:15.503673+0000","last_peered":"2023-10-21T16:22:15.503673+0000","last_clean":"2023-10-21T16:22:15.503673+0000","last_became_active":"2023-10-21T16:22:14.509992+0000","last_became_peered":"2023-10-21T16:22:14.509992+0000","last_unstale":"2023-10-21T16:22:15.503673+0000","last_undegraded":"2023-10-21T16:22:15.503673+0000","last_fullsized":"2023-10-21T16:22:15.503673+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504114+0000","last_change":"2023-10-21T16:22:14.512977+0000","last_active":"2023-10-21T16:22:15.504114+0000","last_peered":"2023-10-21T16:22:15.504114+0000","last_clean":"2023-10-21T16:22:15.504114+0000","last_became_active":"2023-10-21T16:22:14.512646+0000","last_became_peered":"2023-10-21T16:22:14.512646+0000","last_unstale":"2023-10-21T16:22:15.504114+0000","last_undegraded":"2023-10-21T16:22:15.504114+0000","last_fullsized":"2023-10-21T16:22:15.504114+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.239579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.521275+0000","last_change":"2023-10-21T16:22:14.515210+0000","last_active":"2023-10-21T16:22:14.521275+0000","last_peered":"2023-10-21T16:22:14.521275+0000","last_clean":"2023-10-21T16:22:14.521275+0000","last_became_active":"2023-10-21T16:22:14.514945+0000","last_became_peered":"2023-10-21T16:22:14.514945+0000","last_unstale":"2023-10-21T16:22:14.521275+0000","last_undegraded":"2023-10-21T16:22:14.521275+0000","last_fullsized":"2023-10-21T16:22:14.521275+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:37.856874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503824+0000","last_change":"2023-10-21T16:22:14.508021+0000","last_active":"2023-10-21T16:22:15.503824+0000","last_peered":"2023-10-21T16:22:15.503824+0000","last_clean":"2023-10-21T16:22:15.503824+0000","last_became_active":"2023-10-21T16:22:14.505727+0000","last_became_peered":"2023-10-21T16:22:14.505727+0000","last_unstale":"2023-10-21T16:22:15.503824+0000","last_undegraded":"2023-10-21T16:22:15.503824+0000","last_fullsized":"2023-10-21T16:22:15.503824+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:05.844283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503333+0000","last_change":"2023-10-21T16:22:14.504152+0000","last_active":"2023-10-21T16:22:15.503333+0000","last_peered":"2023-10-21T16:22:15.503333+0000","last_clean":"2023-10-21T16:22:15.503333+0000","last_became_active":"2023-10-21T16:22:14.503854+0000","last_became_peered":"2023-10-21T16:22:14.503854+0000","last_unstale":"2023-10-21T16:22:15.503333+0000","last_undegraded":"2023-10-21T16:22:15.503333+0000","last_fullsized":"2023-10-21T16:22:15.503333+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:29:16.609210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.521644+0000","last_change":"2023-10-21T16:22:14.517533+0000","last_active":"2023-10-21T16:22:14.521644+0000","last_peered":"2023-10-21T16:22:14.521644+0000","last_clean":"2023-10-21T16:22:14.521644+0000","last_became_active":"2023-10-21T16:22:14.517257+0000","last_became_peered":"2023-10-21T16:22:14.517257+0000","last_unstale":"2023-10-21T16:22:14.521644+0000","last_undegraded":"2023-10-21T16:22:14.521644+0000","last_fullsized":"2023-10-21T16:22:14.521644+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:56.254381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525004+0000","last_change":"2023-10-21T16:22:14.521071+0000","last_active":"2023-10-21T16:22:14.525004+0000","last_peered":"2023-10-21T16:22:14.525004+0000","last_clean":"2023-10-21T16:22:14.525004+0000","last_became_active":"2023-10-21T16:22:14.515838+0000","last_became_peered":"2023-10-21T16:22:14.515838+0000","last_unstale":"2023-10-21T16:22:14.525004+0000","last_undegraded":"2023-10-21T16:22:14.525004+0000","last_fullsized":"2023-10-21T16:22:14.525004+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:02.078960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513305+0000","last_change":"2023-10-21T16:22:14.510118+0000","last_active":"2023-10-21T16:22:14.513305+0000","last_peered":"2023-10-21T16:22:14.513305+0000","last_clean":"2023-10-21T16:22:14.513305+0000","last_became_active":"2023-10-21T16:22:14.509450+0000","last_became_peered":"2023-10-21T16:22:14.509450+0000","last_unstale":"2023-10-21T16:22:14.513305+0000","last_undegraded":"2023-10-21T16:22:14.513305+0000","last_fullsized":"2023-10-21T16:22:14.513305+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:27:49.644053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.516485+0000","last_change":"2023-10-21T16:22:14.513847+0000","last_active":"2023-10-21T16:22:14.516485+0000","last_peered":"2023-10-21T16:22:14.516485+0000","last_clean":"2023-10-21T16:22:14.516485+0000","last_became_active":"2023-10-21T16:22:14.513590+0000","last_became_peered":"2023-10-21T16:22:14.513590+0000","last_unstale":"2023-10-21T16:22:14.516485+0000","last_undegraded":"2023-10-21T16:22:14.516485+0000","last_fullsized":"2023-10-21T16:22:14.516485+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:46.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513028+0000","last_change":"2023-10-21T16:22:14.509959+0000","last_active":"2023-10-21T16:22:14.513028+0000","last_peered":"2023-10-21T16:22:14.513028+0000","last_clean":"2023-10-21T16:22:14.513028+0000","last_became_active":"2023-10-21T16:22:14.509655+0000","last_became_peered":"2023-10-21T16:22:14.509655+0000","last_unstale":"2023-10-21T16:22:14.513028+0000","last_undegraded":"2023-10-21T16:22:14.513028+0000","last_fullsized":"2023-10-21T16:22:14.513028+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:57.204127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502858+0000","last_change":"2023-10-21T16:22:14.500133+0000","last_active":"2023-10-21T16:22:15.502858+0000","last_peered":"2023-10-21T16:22:15.502858+0000","last_clean":"2023-10-21T16:22:15.502858+0000","last_became_active":"2023-10-21T16:22:14.499858+0000","last_became_peered":"2023-10-21T16:22:14.499858+0000","last_unstale":"2023-10-21T16:22:15.502858+0000","last_undegraded":"2023-10-21T16:22:15.502858+0000","last_fullsized":"2023-10-21T16:22:15.502858+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:23:15.041609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503554+0000","last_change":"2023-10-21T16:22:14.504922+0000","last_active":"2023-10-21T16:22:15.503554+0000","last_peered":"2023-10-21T16:22:15.503554+0000","last_clean":"2023-10-21T16:22:15.503554+0000","last_became_active":"2023-10-21T16:22:14.504571+0000","last_became_peered":"2023-10-21T16:22:14.504571+0000","last_unstale":"2023-10-21T16:22:15.503554+0000","last_undegraded":"2023-10-21T16:22:15.503554+0000","last_fullsized":"2023-10-21T16:22:15.503554+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:04.796713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501674+0000","last_change":"2023-10-21T16:22:14.505457+0000","last_active":"2023-10-21T16:22:15.501674+0000","last_peered":"2023-10-21T16:22:15.501674+0000","last_clean":"2023-10-21T16:22:15.501674+0000","last_became_active":"2023-10-21T16:22:14.502162+0000","last_became_peered":"2023-10-21T16:22:14.502162+0000","last_unstale":"2023-10-21T16:22:15.501674+0000","last_undegraded":"2023-10-21T16:22:15.501674+0000","last_fullsized":"2023-10-21T16:22:15.501674+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:22.604905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.514171+0000","last_change":"2023-10-21T16:22:14.501052+0000","last_active":"2023-10-21T16:22:14.514171+0000","last_peered":"2023-10-21T16:22:14.514171+0000","last_clean":"2023-10-21T16:22:14.514171+0000","last_became_active":"2023-10-21T16:22:14.500455+0000","last_became_peered":"2023-10-21T16:22:14.500455+0000","last_unstale":"2023-10-21T16:22:14.514171+0000","last_undegraded":"2023-10-21T16:22:14.514171+0000","last_fullsized":"2023-10-21T16:22:14.514171+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:35.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.514477+0000","last_change":"2023-10-21T16:22:14.510939+0000","last_active":"2023-10-21T16:22:14.514477+0000","last_peered":"2023-10-21T16:22:14.514477+0000","last_clean":"2023-10-21T16:22:14.514477+0000","last_became_active":"2023-10-21T16:22:14.510669+0000","last_became_peered":"2023-10-21T16:22:14.510669+0000","last_unstale":"2023-10-21T16:22:14.514477+0000","last_undegraded":"2023-10-21T16:22:14.514477+0000","last_fullsized":"2023-10-21T16:22:14.514477+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:13.942978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520917+0000","last_change":"2023-10-21T16:22:14.518046+0000","last_active":"2023-10-21T16:22:14.520917+0000","last_peered":"2023-10-21T16:22:14.520917+0000","last_clean":"2023-10-21T16:22:14.520917+0000","last_became_active":"2023-10-21T16:22:14.517748+0000","last_became_peered":"2023-10-21T16:22:14.517748+0000","last_unstale":"2023-10-21T16:22:14.520917+0000","last_undegraded":"2023-10-21T16:22:14.520917+0000","last_fullsized":"2023-10-21T16:22:14.520917+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:01.267963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523942+0000","last_change":"2023-10-21T16:22:14.521950+0000","last_active":"2023-10-21T16:22:14.523942+0000","last_peered":"2023-10-21T16:22:14.523942+0000","last_clean":"2023-10-21T16:22:14.523942+0000","last_became_active":"2023-10-21T16:22:14.521689+0000","last_became_peered":"2023-10-21T16:22:14.521689+0000","last_unstale":"2023-10-21T16:22:14.523942+0000","last_undegraded":"2023-10-21T16:22:14.523942+0000","last_fullsized":"2023-10-21T16:22:14.523942+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:47.037388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.502579+0000","last_change":"2023-10-21T16:22:14.496623+0000","last_active":"2023-10-21T16:22:14.502579+0000","last_peered":"2023-10-21T16:22:14.502579+0000","last_clean":"2023-10-21T16:22:14.502579+0000","last_became_active":"2023-10-21T16:22:14.496316+0000","last_became_peered":"2023-10-21T16:22:14.496316+0000","last_unstale":"2023-10-21T16:22:14.502579+0000","last_undegraded":"2023-10-21T16:22:14.502579+0000","last_fullsized":"2023-10-21T16:22:14.502579+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:48.253170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522367+0000","last_change":"2023-10-21T16:22:14.518532+0000","last_active":"2023-10-21T16:22:14.522367+0000","last_peered":"2023-10-21T16:22:14.522367+0000","last_clean":"2023-10-21T16:22:14.522367+0000","last_became_active":"2023-10-21T16:22:14.518257+0000","last_became_peered":"2023-10-21T16:22:14.518257+0000","last_unstale":"2023-10-21T16:22:14.522367+0000","last_undegraded":"2023-10-21T16:22:14.522367+0000","last_fullsized":"2023-10-21T16:22:14.522367+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:21.175037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520912+0000","last_change":"2023-10-21T16:22:14.515236+0000","last_active":"2023-10-21T16:22:14.520912+0000","last_peered":"2023-10-21T16:22:14.520912+0000","last_clean":"2023-10-21T16:22:14.520912+0000","last_became_active":"2023-10-21T16:22:14.514214+0000","last_became_peered":"2023-10-21T16:22:14.514214+0000","last_unstale":"2023-10-21T16:22:14.520912+0000","last_undegraded":"2023-10-21T16:22:14.520912+0000","last_fullsized":"2023-10-21T16:22:14.520912+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:58.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506007+0000","last_change":"2023-10-21T16:22:14.498790+0000","last_active":"2023-10-21T16:22:15.506007+0000","last_peered":"2023-10-21T16:22:15.506007+0000","last_clean":"2023-10-21T16:22:15.506007+0000","last_became_active":"2023-10-21T16:22:14.498443+0000","last_became_peered":"2023-10-21T16:22:14.498443+0000","last_unstale":"2023-10-21T16:22:15.506007+0000","last_undegraded":"2023-10-21T16:22:15.506007+0000","last_fullsized":"2023-10-21T16:22:15.506007+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:59:42.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508628+0000","last_change":"2023-10-21T16:22:14.498870+0000","last_active":"2023-10-21T16:22:14.508628+0000","last_peered":"2023-10-21T16:22:14.508628+0000","last_clean":"2023-10-21T16:22:14.508628+0000","last_became_active":"2023-10-21T16:22:14.498588+0000","last_became_peered":"2023-10-21T16:22:14.498588+0000","last_unstale":"2023-10-21T16:22:14.508628+0000","last_undegraded":"2023-10-21T16:22:14.508628+0000","last_fullsized":"2023-10-21T16:22:14.508628+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:55.610506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.505919+0000","last_change":"2023-10-21T16:22:14.499944+0000","last_active":"2023-10-21T16:22:15.505919+0000","last_peered":"2023-10-21T16:22:15.505919+0000","last_clean":"2023-10-21T16:22:15.505919+0000","last_became_active":"2023-10-21T16:22:14.499595+0000","last_became_peered":"2023-10-21T16:22:14.499595+0000","last_unstale":"2023-10-21T16:22:15.505919+0000","last_undegraded":"2023-10-21T16:22:15.505919+0000","last_fullsized":"2023-10-21T16:22:15.505919+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:59.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"160'4","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529091+0000","last_change":"2023-10-21T16:22:13.481657+0000","last_active":"2023-10-21T16:22:14.529091+0000","last_peered":"2023-10-21T16:22:14.529091+0000","last_clean":"2023-10-21T16:22:14.529091+0000","last_became_active":"2023-10-21T16:22:13.480257+0000","last_became_peered":"2023-10-21T16:22:13.480257+0000","last_unstale":"2023-10-21T16:22:14.529091+0000","last_undegraded":"2023-10-21T16:22:14.529091+0000","last_fullsized":"2023-10-21T16:22:14.529091+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:28.038285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503554+0000","last_change":"2023-10-21T16:22:14.490386+0000","last_active":"2023-10-21T16:22:15.503554+0000","last_peered":"2023-10-21T16:22:15.503554+0000","last_clean":"2023-10-21T16:22:15.503554+0000","last_became_active":"2023-10-21T16:22:14.490035+0000","last_became_peered":"2023-10-21T16:22:14.490035+0000","last_unstale":"2023-10-21T16:22:15.503554+0000","last_undegraded":"2023-10-21T16:22:15.503554+0000","last_fullsized":"2023-10-21T16:22:15.503554+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:45.882520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528796+0000","last_change":"2023-10-21T16:22:13.476830+0000","last_active":"2023-10-21T16:22:14.528796+0000","last_peered":"2023-10-21T16:22:14.528796+0000","last_clean":"2023-10-21T16:22:14.528796+0000","last_became_active":"2023-10-21T16:22:13.471642+0000","last_became_peered":"2023-10-21T16:22:13.471642+0000","last_unstale":"2023-10-21T16:22:14.528796+0000","last_undegraded":"2023-10-21T16:22:14.528796+0000","last_fullsized":"2023-10-21T16:22:14.528796+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:39.388093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502220+0000","last_change":"2023-10-21T16:22:14.503952+0000","last_active":"2023-10-21T16:22:15.502220+0000","last_peered":"2023-10-21T16:22:15.502220+0000","last_clean":"2023-10-21T16:22:15.502220+0000","last_became_active":"2023-10-21T16:22:14.503675+0000","last_became_peered":"2023-10-21T16:22:14.503675+0000","last_unstale":"2023-10-21T16:22:15.502220+0000","last_undegraded":"2023-10-21T16:22:15.502220+0000","last_fullsized":"2023-10-21T16:22:15.502220+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:30.875927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525718+0000","last_change":"2023-10-21T16:22:13.477799+0000","last_active":"2023-10-21T16:22:14.525718+0000","last_peered":"2023-10-21T16:22:14.525718+0000","last_clean":"2023-10-21T16:22:14.525718+0000","last_became_active":"2023-10-21T16:22:13.477487+0000","last_became_peered":"2023-10-21T16:22:13.477487+0000","last_unstale":"2023-10-21T16:22:14.525718+0000","last_undegraded":"2023-10-21T16:22:14.525718+0000","last_fullsized":"2023-10-21T16:22:14.525718+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:52.593235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503715+0000","last_change":"2023-10-21T16:22:14.504774+0000","last_active":"2023-10-21T16:22:15.503715+0000","last_peered":"2023-10-21T16:22:15.503715+0000","last_clean":"2023-10-21T16:22:15.503715+0000","last_became_active":"2023-10-21T16:22:14.504353+0000","last_became_peered":"2023-10-21T16:22:14.504353+0000","last_unstale":"2023-10-21T16:22:15.503715+0000","last_undegraded":"2023-10-21T16:22:15.503715+0000","last_fullsized":"2023-10-21T16:22:15.503715+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:05:29.265989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525062+0000","last_change":"2023-10-21T16:22:13.483247+0000","last_active":"2023-10-21T16:22:14.525062+0000","last_peered":"2023-10-21T16:22:14.525062+0000","last_clean":"2023-10-21T16:22:14.525062+0000","last_became_active":"2023-10-21T16:22:13.482554+0000","last_became_peered":"2023-10-21T16:22:13.482554+0000","last_unstale":"2023-10-21T16:22:14.525062+0000","last_undegraded":"2023-10-21T16:22:14.525062+0000","last_fullsized":"2023-10-21T16:22:14.525062+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:05.061044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506994+0000","last_change":"2023-10-21T16:22:14.498509+0000","last_active":"2023-10-21T16:22:15.506994+0000","last_peered":"2023-10-21T16:22:15.506994+0000","last_clean":"2023-10-21T16:22:15.506994+0000","last_became_active":"2023-10-21T16:22:14.498111+0000","last_became_peered":"2023-10-21T16:22:14.498111+0000","last_unstale":"2023-10-21T16:22:15.506994+0000","last_undegraded":"2023-10-21T16:22:15.506994+0000","last_fullsized":"2023-10-21T16:22:15.506994+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:22.679691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"160'1","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506795+0000","last_change":"2023-10-21T16:22:13.469329+0000","last_active":"2023-10-21T16:22:15.506795+0000","last_peered":"2023-10-21T16:22:15.506795+0000","last_clean":"2023-10-21T16:22:15.506795+0000","last_became_active":"2023-10-21T16:22:13.469036+0000","last_became_peered":"2023-10-21T16:22:13.469036+0000","last_unstale":"2023-10-21T16:22:15.506795+0000","last_undegraded":"2023-10-21T16:22:15.506795+0000","last_fullsized":"2023-10-21T16:22:15.506795+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:33.346616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507515+0000","last_change":"2023-10-21T16:22:14.499996+0000","last_active":"2023-10-21T16:22:15.507515+0000","last_peered":"2023-10-21T16:22:15.507515+0000","last_clean":"2023-10-21T16:22:15.507515+0000","last_became_active":"2023-10-21T16:22:14.499636+0000","last_became_peered":"2023-10-21T16:22:14.499636+0000","last_unstale":"2023-10-21T16:22:15.507515+0000","last_undegraded":"2023-10-21T16:22:15.507515+0000","last_fullsized":"2023-10-21T16:22:15.507515+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:40.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529052+0000","last_change":"2023-10-21T16:22:13.468390+0000","last_active":"2023-10-21T16:22:14.529052+0000","last_peered":"2023-10-21T16:22:14.529052+0000","last_clean":"2023-10-21T16:22:14.529052+0000","last_became_active":"2023-10-21T16:22:13.468118+0000","last_became_peered":"2023-10-21T16:22:13.468118+0000","last_unstale":"2023-10-21T16:22:14.529052+0000","last_undegraded":"2023-10-21T16:22:14.529052+0000","last_fullsized":"2023-10-21T16:22:14.529052+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:53.287507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513359+0000","last_change":"2023-10-21T16:22:14.510289+0000","last_active":"2023-10-21T16:22:14.513359+0000","last_peered":"2023-10-21T16:22:14.513359+0000","last_clean":"2023-10-21T16:22:14.513359+0000","last_became_active":"2023-10-21T16:22:14.509831+0000","last_became_peered":"2023-10-21T16:22:14.509831+0000","last_unstale":"2023-10-21T16:22:14.513359+0000","last_undegraded":"2023-10-21T16:22:14.513359+0000","last_fullsized":"2023-10-21T16:22:14.513359+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:24.955473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496575+0000","last_change":"2023-10-21T16:22:13.478388+0000","last_active":"2023-10-21T16:22:14.496575+0000","last_peered":"2023-10-21T16:22:14.496575+0000","last_clean":"2023-10-21T16:22:14.496575+0000","last_became_active":"2023-10-21T16:22:13.477016+0000","last_became_peered":"2023-10-21T16:22:13.477016+0000","last_unstale":"2023-10-21T16:22:14.496575+0000","last_undegraded":"2023-10-21T16:22:14.496575+0000","last_fullsized":"2023-10-21T16:22:14.496575+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:58.286988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520375+0000","last_change":"2023-10-21T16:22:14.517302+0000","last_active":"2023-10-21T16:22:14.520375+0000","last_peered":"2023-10-21T16:22:14.520375+0000","last_clean":"2023-10-21T16:22:14.520375+0000","last_became_active":"2023-10-21T16:22:14.517024+0000","last_became_peered":"2023-10-21T16:22:14.517024+0000","last_unstale":"2023-10-21T16:22:14.520375+0000","last_undegraded":"2023-10-21T16:22:14.520375+0000","last_fullsized":"2023-10-21T16:22:14.520375+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:45:25.645903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.500798+0000","last_change":"2023-10-21T16:22:13.477906+0000","last_active":"2023-10-21T16:22:14.500798+0000","last_peered":"2023-10-21T16:22:14.500798+0000","last_clean":"2023-10-21T16:22:14.500798+0000","last_became_active":"2023-10-21T16:22:13.477158+0000","last_became_peered":"2023-10-21T16:22:13.477158+0000","last_unstale":"2023-10-21T16:22:14.500798+0000","last_undegraded":"2023-10-21T16:22:14.500798+0000","last_fullsized":"2023-10-21T16:22:14.500798+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:54.499269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503331+0000","last_change":"2023-10-21T16:22:14.518585+0000","last_active":"2023-10-21T16:22:15.503331+0000","last_peered":"2023-10-21T16:22:15.503331+0000","last_clean":"2023-10-21T16:22:15.503331+0000","last_became_active":"2023-10-21T16:22:14.518277+0000","last_became_peered":"2023-10-21T16:22:14.518277+0000","last_unstale":"2023-10-21T16:22:15.503331+0000","last_undegraded":"2023-10-21T16:22:15.503331+0000","last_fullsized":"2023-10-21T16:22:15.503331+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:21:00.759282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501572+0000","last_change":"2023-10-21T16:22:13.466837+0000","last_active":"2023-10-21T16:22:15.501572+0000","last_peered":"2023-10-21T16:22:15.501572+0000","last_clean":"2023-10-21T16:22:15.501572+0000","last_became_active":"2023-10-21T16:22:13.466494+0000","last_became_peered":"2023-10-21T16:22:13.466494+0000","last_unstale":"2023-10-21T16:22:15.501572+0000","last_undegraded":"2023-10-21T16:22:15.501572+0000","last_fullsized":"2023-10-21T16:22:15.501572+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:54:54.138220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525500+0000","last_change":"2023-10-21T16:22:14.521702+0000","last_active":"2023-10-21T16:22:14.525500+0000","last_peered":"2023-10-21T16:22:14.525500+0000","last_clean":"2023-10-21T16:22:14.525500+0000","last_became_active":"2023-10-21T16:22:14.511786+0000","last_became_peered":"2023-10-21T16:22:14.511786+0000","last_unstale":"2023-10-21T16:22:14.525500+0000","last_undegraded":"2023-10-21T16:22:14.525500+0000","last_fullsized":"2023-10-21T16:22:14.525500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:04.944215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504070+0000","last_change":"2023-10-21T16:22:13.456080+0000","last_active":"2023-10-21T16:22:15.504070+0000","last_peered":"2023-10-21T16:22:15.504070+0000","last_clean":"2023-10-21T16:22:15.504070+0000","last_became_active":"2023-10-21T16:22:13.455463+0000","last_became_peered":"2023-10-21T16:22:13.455463+0000","last_unstale":"2023-10-21T16:22:15.504070+0000","last_undegraded":"2023-10-21T16:22:15.504070+0000","last_fullsized":"2023-10-21T16:22:15.504070+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:33.898538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522414+0000","last_change":"2023-10-21T16:22:14.519074+0000","last_active":"2023-10-21T16:22:14.522414+0000","last_peered":"2023-10-21T16:22:14.522414+0000","last_clean":"2023-10-21T16:22:14.522414+0000","last_became_active":"2023-10-21T16:22:14.517849+0000","last_became_peered":"2023-10-21T16:22:14.517849+0000","last_unstale":"2023-10-21T16:22:14.522414+0000","last_undegraded":"2023-10-21T16:22:14.522414+0000","last_fullsized":"2023-10-21T16:22:14.522414+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:16.912931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"160'3","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525498+0000","last_change":"2023-10-21T16:22:13.481087+0000","last_active":"2023-10-21T16:22:14.525498+0000","last_peered":"2023-10-21T16:22:14.525498+0000","last_clean":"2023-10-21T16:22:14.525498+0000","last_became_active":"2023-10-21T16:22:13.480759+0000","last_became_peered":"2023-10-21T16:22:13.480759+0000","last_unstale":"2023-10-21T16:22:14.525498+0000","last_undegraded":"2023-10-21T16:22:14.525498+0000","last_fullsized":"2023-10-21T16:22:14.525498+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:48.297330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.512921+0000","last_change":"2023-10-21T16:22:14.510046+0000","last_active":"2023-10-21T16:22:14.512921+0000","last_peered":"2023-10-21T16:22:14.512921+0000","last_clean":"2023-10-21T16:22:14.512921+0000","last_became_active":"2023-10-21T16:22:14.509677+0000","last_became_peered":"2023-10-21T16:22:14.509677+0000","last_unstale":"2023-10-21T16:22:14.512921+0000","last_undegraded":"2023-10-21T16:22:14.512921+0000","last_fullsized":"2023-10-21T16:22:14.512921+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:58:52.949680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"160'6","reported_seq":17,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.530492+0000","last_change":"2023-10-21T16:22:13.481392+0000","last_active":"2023-10-21T16:22:14.530492+0000","last_peered":"2023-10-21T16:22:14.530492+0000","last_clean":"2023-10-21T16:22:14.530492+0000","last_became_active":"2023-10-21T16:22:13.481108+0000","last_became_peered":"2023-10-21T16:22:13.481108+0000","last_unstale":"2023-10-21T16:22:14.530492+0000","last_undegraded":"2023-10-21T16:22:14.530492+0000","last_fullsized":"2023-10-21T16:22:14.530492+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:21.785876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523695+0000","last_change":"2023-10-21T16:22:14.514009+0000","last_active":"2023-10-21T16:22:14.523695+0000","last_peered":"2023-10-21T16:22:14.523695+0000","last_clean":"2023-10-21T16:22:14.523695+0000","last_became_active":"2023-10-21T16:22:14.513723+0000","last_became_peered":"2023-10-21T16:22:14.513723+0000","last_unstale":"2023-10-21T16:22:14.523695+0000","last_undegraded":"2023-10-21T16:22:14.523695+0000","last_fullsized":"2023-10-21T16:22:14.523695+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503666+0000","last_change":"2023-10-21T16:22:13.453344+0000","last_active":"2023-10-21T16:22:15.503666+0000","last_peered":"2023-10-21T16:22:15.503666+0000","last_clean":"2023-10-21T16:22:15.503666+0000","last_became_active":"2023-10-21T16:22:13.452704+0000","last_became_peered":"2023-10-21T16:22:13.452704+0000","last_unstale":"2023-10-21T16:22:15.503666+0000","last_undegraded":"2023-10-21T16:22:15.503666+0000","last_fullsized":"2023-10-21T16:22:15.503666+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:06:14.002394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.516679+0000","last_change":"2023-10-21T16:22:14.515562+0000","last_active":"2023-10-21T16:22:14.516679+0000","last_peered":"2023-10-21T16:22:14.516679+0000","last_clean":"2023-10-21T16:22:14.516679+0000","last_became_active":"2023-10-21T16:22:14.515050+0000","last_became_peered":"2023-10-21T16:22:14.515050+0000","last_unstale":"2023-10-21T16:22:14.516679+0000","last_undegraded":"2023-10-21T16:22:14.516679+0000","last_fullsized":"2023-10-21T16:22:14.516679+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:03:36.989147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.505101+0000","last_change":"2023-10-21T16:22:13.469495+0000","last_active":"2023-10-21T16:22:14.505101+0000","last_peered":"2023-10-21T16:22:14.505101+0000","last_clean":"2023-10-21T16:22:14.505101+0000","last_became_active":"2023-10-21T16:22:13.469193+0000","last_became_peered":"2023-10-21T16:22:13.469193+0000","last_unstale":"2023-10-21T16:22:14.505101+0000","last_undegraded":"2023-10-21T16:22:14.505101+0000","last_fullsized":"2023-10-21T16:22:14.505101+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:59:13.505853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.509328+0000","last_change":"2023-10-21T16:22:14.502618+0000","last_active":"2023-10-21T16:22:14.509328+0000","last_peered":"2023-10-21T16:22:14.509328+0000","last_clean":"2023-10-21T16:22:14.509328+0000","last_became_active":"2023-10-21T16:22:14.501803+0000","last_became_peered":"2023-10-21T16:22:14.501803+0000","last_unstale":"2023-10-21T16:22:14.509328+0000","last_undegraded":"2023-10-21T16:22:14.509328+0000","last_fullsized":"2023-10-21T16:22:14.509328+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:48.253170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"160'8","reported_seq":19,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529310+0000","last_change":"2023-10-21T16:22:13.474420+0000","last_active":"2023-10-21T16:22:14.529310+0000","last_peered":"2023-10-21T16:22:14.529310+0000","last_clean":"2023-10-21T16:22:14.529310+0000","last_became_active":"2023-10-21T16:22:13.474107+0000","last_became_peered":"2023-10-21T16:22:13.474107+0000","last_unstale":"2023-10-21T16:22:14.529310+0000","last_undegraded":"2023-10-21T16:22:14.529310+0000","last_fullsized":"2023-10-21T16:22:14.529310+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:16.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513236+0000","last_change":"2023-10-21T16:22:14.510554+0000","last_active":"2023-10-21T16:22:14.513236+0000","last_peered":"2023-10-21T16:22:14.513236+0000","last_clean":"2023-10-21T16:22:14.513236+0000","last_became_active":"2023-10-21T16:22:14.510100+0000","last_became_peered":"2023-10-21T16:22:14.510100+0000","last_unstale":"2023-10-21T16:22:14.513236+0000","last_undegraded":"2023-10-21T16:22:14.513236+0000","last_fullsized":"2023-10-21T16:22:14.513236+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:49:34.035069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"160'1","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501683+0000","last_change":"2023-10-21T16:22:13.470430+0000","last_active":"2023-10-21T16:22:15.501683+0000","last_peered":"2023-10-21T16:22:15.501683+0000","last_clean":"2023-10-21T16:22:15.501683+0000","last_became_active":"2023-10-21T16:22:13.469563+0000","last_became_peered":"2023-10-21T16:22:13.469563+0000","last_unstale":"2023-10-21T16:22:15.501683+0000","last_undegraded":"2023-10-21T16:22:15.501683+0000","last_fullsized":"2023-10-21T16:22:15.501683+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:12:03.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515706+0000","last_change":"2023-10-21T16:22:14.512688+0000","last_active":"2023-10-21T16:22:14.515706+0000","last_peered":"2023-10-21T16:22:14.515706+0000","last_clean":"2023-10-21T16:22:14.515706+0000","last_became_active":"2023-10-21T16:22:14.512417+0000","last_became_peered":"2023-10-21T16:22:14.512417+0000","last_unstale":"2023-10-21T16:22:14.515706+0000","last_undegraded":"2023-10-21T16:22:14.515706+0000","last_fullsized":"2023-10-21T16:22:14.515706+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:28.490441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528298+0000","last_change":"2023-10-21T16:22:13.479960+0000","last_active":"2023-10-21T16:22:14.528298+0000","last_peered":"2023-10-21T16:22:14.528298+0000","last_clean":"2023-10-21T16:22:14.528298+0000","last_became_active":"2023-10-21T16:22:13.478731+0000","last_became_peered":"2023-10-21T16:22:13.478731+0000","last_unstale":"2023-10-21T16:22:14.528298+0000","last_undegraded":"2023-10-21T16:22:14.528298+0000","last_fullsized":"2023-10-21T16:22:14.528298+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:49:08.306960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523700+0000","last_change":"2023-10-21T16:22:14.521797+0000","last_active":"2023-10-21T16:22:14.523700+0000","last_peered":"2023-10-21T16:22:14.523700+0000","last_clean":"2023-10-21T16:22:14.523700+0000","last_became_active":"2023-10-21T16:22:14.521468+0000","last_became_peered":"2023-10-21T16:22:14.521468+0000","last_unstale":"2023-10-21T16:22:14.523700+0000","last_undegraded":"2023-10-21T16:22:14.523700+0000","last_fullsized":"2023-10-21T16:22:14.523700+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:10.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528344+0000","last_change":"2023-10-21T16:22:13.480152+0000","last_active":"2023-10-21T16:22:14.528344+0000","last_peered":"2023-10-21T16:22:14.528344+0000","last_clean":"2023-10-21T16:22:14.528344+0000","last_became_active":"2023-10-21T16:22:13.479768+0000","last_became_peered":"2023-10-21T16:22:13.479768+0000","last_unstale":"2023-10-21T16:22:14.528344+0000","last_undegraded":"2023-10-21T16:22:14.528344+0000","last_fullsized":"2023-10-21T16:22:14.528344+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:41.595996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.518069+0000","last_change":"2023-10-21T16:22:14.512268+0000","last_active":"2023-10-21T16:22:14.518069+0000","last_peered":"2023-10-21T16:22:14.518069+0000","last_clean":"2023-10-21T16:22:14.518069+0000","last_became_active":"2023-10-21T16:22:14.511856+0000","last_became_peered":"2023-10-21T16:22:14.511856+0000","last_unstale":"2023-10-21T16:22:14.518069+0000","last_undegraded":"2023-10-21T16:22:14.518069+0000","last_fullsized":"2023-10-21T16:22:14.518069+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:20.207076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.501655+0000","last_change":"2023-10-21T16:22:13.468999+0000","last_active":"2023-10-21T16:22:14.501655+0000","last_peered":"2023-10-21T16:22:14.501655+0000","last_clean":"2023-10-21T16:22:14.501655+0000","last_became_active":"2023-10-21T16:22:13.468202+0000","last_became_peered":"2023-10-21T16:22:13.468202+0000","last_unstale":"2023-10-21T16:22:14.501655+0000","last_undegraded":"2023-10-21T16:22:14.501655+0000","last_fullsized":"2023-10-21T16:22:14.501655+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:22.231166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.526771+0000","last_change":"2023-10-21T16:22:14.521492+0000","last_active":"2023-10-21T16:22:14.526771+0000","last_peered":"2023-10-21T16:22:14.526771+0000","last_clean":"2023-10-21T16:22:14.526771+0000","last_became_active":"2023-10-21T16:22:14.520945+0000","last_became_peered":"2023-10-21T16:22:14.520945+0000","last_unstale":"2023-10-21T16:22:14.526771+0000","last_undegraded":"2023-10-21T16:22:14.526771+0000","last_fullsized":"2023-10-21T16:22:14.526771+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:17.858273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506924+0000","last_change":"2023-10-21T16:22:13.478863+0000","last_active":"2023-10-21T16:22:15.506924+0000","last_peered":"2023-10-21T16:22:15.506924+0000","last_clean":"2023-10-21T16:22:15.506924+0000","last_became_active":"2023-10-21T16:22:13.478551+0000","last_became_peered":"2023-10-21T16:22:13.478551+0000","last_unstale":"2023-10-21T16:22:15.506924+0000","last_undegraded":"2023-10-21T16:22:15.506924+0000","last_fullsized":"2023-10-21T16:22:15.506924+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:18.771626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504112+0000","last_change":"2023-10-21T16:22:14.502994+0000","last_active":"2023-10-21T16:22:15.504112+0000","last_peered":"2023-10-21T16:22:15.504112+0000","last_clean":"2023-10-21T16:22:15.504112+0000","last_became_active":"2023-10-21T16:22:14.502672+0000","last_became_peered":"2023-10-21T16:22:14.502672+0000","last_unstale":"2023-10-21T16:22:15.504112+0000","last_undegraded":"2023-10-21T16:22:15.504112+0000","last_fullsized":"2023-10-21T16:22:15.504112+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:08.772394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503691+0000","last_change":"2023-10-21T16:22:13.466592+0000","last_active":"2023-10-21T16:22:15.503691+0000","last_peered":"2023-10-21T16:22:15.503691+0000","last_clean":"2023-10-21T16:22:15.503691+0000","last_became_active":"2023-10-21T16:22:13.463481+0000","last_became_peered":"2023-10-21T16:22:13.463481+0000","last_unstale":"2023-10-21T16:22:15.503691+0000","last_undegraded":"2023-10-21T16:22:15.503691+0000","last_fullsized":"2023-10-21T16:22:15.503691+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:33.851202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.524462+0000","last_change":"2023-10-21T16:22:14.518717+0000","last_active":"2023-10-21T16:22:14.524462+0000","last_peered":"2023-10-21T16:22:14.524462+0000","last_clean":"2023-10-21T16:22:14.524462+0000","last_became_active":"2023-10-21T16:22:14.518447+0000","last_became_peered":"2023-10-21T16:22:14.518447+0000","last_unstale":"2023-10-21T16:22:14.524462+0000","last_undegraded":"2023-10-21T16:22:14.524462+0000","last_fullsized":"2023-10-21T16:22:14.524462+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:55.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.499021+0000","last_change":"2023-10-21T16:22:13.479657+0000","last_active":"2023-10-21T16:22:14.499021+0000","last_peered":"2023-10-21T16:22:14.499021+0000","last_clean":"2023-10-21T16:22:14.499021+0000","last_became_active":"2023-10-21T16:22:13.479367+0000","last_became_peered":"2023-10-21T16:22:13.479367+0000","last_unstale":"2023-10-21T16:22:14.499021+0000","last_undegraded":"2023-10-21T16:22:14.499021+0000","last_fullsized":"2023-10-21T16:22:14.499021+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:17.583848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501468+0000","last_change":"2023-10-21T16:22:14.518644+0000","last_active":"2023-10-21T16:22:15.501468+0000","last_peered":"2023-10-21T16:22:15.501468+0000","last_clean":"2023-10-21T16:22:15.501468+0000","last_became_active":"2023-10-21T16:22:14.518344+0000","last_became_peered":"2023-10-21T16:22:14.518344+0000","last_unstale":"2023-10-21T16:22:15.501468+0000","last_undegraded":"2023-10-21T16:22:15.501468+0000","last_fullsized":"2023-10-21T16:22:15.501468+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:38:27.311112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.494101+0000","last_change":"2023-10-21T16:22:13.476381+0000","last_active":"2023-10-21T16:22:14.494101+0000","last_peered":"2023-10-21T16:22:14.494101+0000","last_clean":"2023-10-21T16:22:14.494101+0000","last_became_active":"2023-10-21T16:22:13.476012+0000","last_became_peered":"2023-10-21T16:22:13.476012+0000","last_unstale":"2023-10-21T16:22:14.494101+0000","last_undegraded":"2023-10-21T16:22:14.494101+0000","last_fullsized":"2023-10-21T16:22:14.494101+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:33.346616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.509040+0000","last_change":"2023-10-21T16:22:14.504142+0000","last_active":"2023-10-21T16:22:14.509040+0000","last_peered":"2023-10-21T16:22:14.509040+0000","last_clean":"2023-10-21T16:22:14.509040+0000","last_became_active":"2023-10-21T16:22:14.503814+0000","last_became_peered":"2023-10-21T16:22:14.503814+0000","last_unstale":"2023-10-21T16:22:14.509040+0000","last_undegraded":"2023-10-21T16:22:14.509040+0000","last_fullsized":"2023-10-21T16:22:14.509040+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:20.207076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504193+0000","last_change":"2023-10-21T16:22:13.475285+0000","last_active":"2023-10-21T16:22:15.504193+0000","last_peered":"2023-10-21T16:22:15.504193+0000","last_clean":"2023-10-21T16:22:15.504193+0000","last_became_active":"2023-10-21T16:22:13.474841+0000","last_became_peered":"2023-10-21T16:22:13.474841+0000","last_unstale":"2023-10-21T16:22:15.504193+0000","last_undegraded":"2023-10-21T16:22:15.504193+0000","last_fullsized":"2023-10-21T16:22:15.504193+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:47:43.435223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513532+0000","last_change":"2023-10-21T16:22:14.512145+0000","last_active":"2023-10-21T16:22:14.513532+0000","last_peered":"2023-10-21T16:22:14.513532+0000","last_clean":"2023-10-21T16:22:14.513532+0000","last_became_active":"2023-10-21T16:22:14.510260+0000","last_became_peered":"2023-10-21T16:22:14.510260+0000","last_unstale":"2023-10-21T16:22:14.513532+0000","last_undegraded":"2023-10-21T16:22:14.513532+0000","last_fullsized":"2023-10-21T16:22:14.513532+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:42:54.533693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507874+0000","last_change":"2023-10-21T16:22:13.476020+0000","last_active":"2023-10-21T16:22:15.507874+0000","last_peered":"2023-10-21T16:22:15.507874+0000","last_clean":"2023-10-21T16:22:15.507874+0000","last_became_active":"2023-10-21T16:22:13.475739+0000","last_became_peered":"2023-10-21T16:22:13.475739+0000","last_unstale":"2023-10-21T16:22:15.507874+0000","last_undegraded":"2023-10-21T16:22:15.507874+0000","last_fullsized":"2023-10-21T16:22:15.507874+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:28.211584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515174+0000","last_change":"2023-10-21T16:22:14.511494+0000","last_active":"2023-10-21T16:22:14.515174+0000","last_peered":"2023-10-21T16:22:14.515174+0000","last_clean":"2023-10-21T16:22:14.515174+0000","last_became_active":"2023-10-21T16:22:14.511113+0000","last_became_peered":"2023-10-21T16:22:14.511113+0000","last_unstale":"2023-10-21T16:22:14.515174+0000","last_undegraded":"2023-10-21T16:22:14.515174+0000","last_fullsized":"2023-10-21T16:22:14.515174+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:50.431368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507400+0000","last_change":"2023-10-21T16:22:13.462464+0000","last_active":"2023-10-21T16:22:15.507400+0000","last_peered":"2023-10-21T16:22:15.507400+0000","last_clean":"2023-10-21T16:22:15.507400+0000","last_became_active":"2023-10-21T16:22:13.461958+0000","last_became_peered":"2023-10-21T16:22:13.461958+0000","last_unstale":"2023-10-21T16:22:15.507400+0000","last_undegraded":"2023-10-21T16:22:15.507400+0000","last_fullsized":"2023-10-21T16:22:15.507400+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:16.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508073+0000","last_change":"2023-10-21T16:22:14.498837+0000","last_active":"2023-10-21T16:22:14.508073+0000","last_peered":"2023-10-21T16:22:14.508073+0000","last_clean":"2023-10-21T16:22:14.508073+0000","last_became_active":"2023-10-21T16:22:14.498516+0000","last_became_peered":"2023-10-21T16:22:14.498516+0000","last_unstale":"2023-10-21T16:22:14.508073+0000","last_undegraded":"2023-10-21T16:22:14.508073+0000","last_fullsized":"2023-10-21T16:22:14.508073+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:57.204127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496628+0000","last_change":"2023-10-21T16:22:13.478469+0000","last_active":"2023-10-21T16:22:14.496628+0000","last_peered":"2023-10-21T16:22:14.496628+0000","last_clean":"2023-10-21T16:22:14.496628+0000","last_became_active":"2023-10-21T16:22:13.477075+0000","last_became_peered":"2023-10-21T16:22:13.477075+0000","last_unstale":"2023-10-21T16:22:14.496628+0000","last_undegraded":"2023-10-21T16:22:14.496628+0000","last_fullsized":"2023-10-21T16:22:14.496628+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:59:41.087335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520648+0000","last_change":"2023-10-21T16:22:14.514824+0000","last_active":"2023-10-21T16:22:14.520648+0000","last_peered":"2023-10-21T16:22:14.520648+0000","last_clean":"2023-10-21T16:22:14.520648+0000","last_became_active":"2023-10-21T16:22:14.514530+0000","last_became_peered":"2023-10-21T16:22:14.514530+0000","last_unstale":"2023-10-21T16:22:14.520648+0000","last_undegraded":"2023-10-21T16:22:14.520648+0000","last_fullsized":"2023-10-21T16:22:14.520648+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:01.705022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502728+0000","last_change":"2023-10-21T16:22:13.467401+0000","last_active":"2023-10-21T16:22:15.502728+0000","last_peered":"2023-10-21T16:22:15.502728+0000","last_clean":"2023-10-21T16:22:15.502728+0000","last_became_active":"2023-10-21T16:22:13.464603+0000","last_became_peered":"2023-10-21T16:22:13.464603+0000","last_unstale":"2023-10-21T16:22:15.502728+0000","last_undegraded":"2023-10-21T16:22:15.502728+0000","last_fullsized":"2023-10-21T16:22:15.502728+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:30.365938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.519200+0000","last_change":"2023-10-21T16:22:14.516556+0000","last_active":"2023-10-21T16:22:14.519200+0000","last_peered":"2023-10-21T16:22:14.519200+0000","last_clean":"2023-10-21T16:22:14.519200+0000","last_became_active":"2023-10-21T16:22:14.516225+0000","last_became_peered":"2023-10-21T16:22:14.516225+0000","last_unstale":"2023-10-21T16:22:14.519200+0000","last_undegraded":"2023-10-21T16:22:14.519200+0000","last_fullsized":"2023-10-21T16:22:14.519200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:42.648369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.502407+0000","last_change":"2023-10-21T16:22:13.482154+0000","last_active":"2023-10-21T16:22:14.502407+0000","last_peered":"2023-10-21T16:22:14.502407+0000","last_clean":"2023-10-21T16:22:14.502407+0000","last_became_active":"2023-10-21T16:22:13.481776+0000","last_became_peered":"2023-10-21T16:22:13.481776+0000","last_unstale":"2023-10-21T16:22:14.502407+0000","last_undegraded":"2023-10-21T16:22:14.502407+0000","last_fullsized":"2023-10-21T16:22:14.502407+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:46:37.106960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.517250+0000","last_change":"2023-10-21T16:22:14.511673+0000","last_active":"2023-10-21T16:22:14.517250+0000","last_peered":"2023-10-21T16:22:14.517250+0000","last_clean":"2023-10-21T16:22:14.517250+0000","last_became_active":"2023-10-21T16:22:14.511319+0000","last_became_peered":"2023-10-21T16:22:14.511319+0000","last_unstale":"2023-10-21T16:22:14.517250+0000","last_undegraded":"2023-10-21T16:22:14.517250+0000","last_fullsized":"2023-10-21T16:22:14.517250+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:29.227521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496785+0000","last_change":"2023-10-21T16:22:13.479607+0000","last_active":"2023-10-21T16:22:14.496785+0000","last_peered":"2023-10-21T16:22:14.496785+0000","last_clean":"2023-10-21T16:22:14.496785+0000","last_became_active":"2023-10-21T16:22:13.479334+0000","last_became_peered":"2023-10-21T16:22:13.479334+0000","last_unstale":"2023-10-21T16:22:14.496785+0000","last_undegraded":"2023-10-21T16:22:14.496785+0000","last_fullsized":"2023-10-21T16:22:14.496785+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:21.663754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522803+0000","last_change":"2023-10-21T16:22:14.518891+0000","last_active":"2023-10-21T16:22:14.522803+0000","last_peered":"2023-10-21T16:22:14.522803+0000","last_clean":"2023-10-21T16:22:14.522803+0000","last_became_active":"2023-10-21T16:22:14.518567+0000","last_became_peered":"2023-10-21T16:22:14.518567+0000","last_unstale":"2023-10-21T16:22:14.522803+0000","last_undegraded":"2023-10-21T16:22:14.522803+0000","last_fullsized":"2023-10-21T16:22:14.522803+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:24.620834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.499025+0000","last_change":"2023-10-21T16:22:13.477915+0000","last_active":"2023-10-21T16:22:14.499025+0000","last_peered":"2023-10-21T16:22:14.499025+0000","last_clean":"2023-10-21T16:22:14.499025+0000","last_became_active":"2023-10-21T16:22:13.477118+0000","last_became_peered":"2023-10-21T16:22:13.477118+0000","last_unstale":"2023-10-21T16:22:14.499025+0000","last_undegraded":"2023-10-21T16:22:14.499025+0000","last_fullsized":"2023-10-21T16:22:14.499025+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:27.176527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513629+0000","last_change":"2023-10-21T16:22:14.511874+0000","last_active":"2023-10-21T16:22:14.513629+0000","last_peered":"2023-10-21T16:22:14.513629+0000","last_clean":"2023-10-21T16:22:14.513629+0000","last_became_active":"2023-10-21T16:22:14.511610+0000","last_became_peered":"2023-10-21T16:22:14.511610+0000","last_unstale":"2023-10-21T16:22:14.513629+0000","last_undegraded":"2023-10-21T16:22:14.513629+0000","last_fullsized":"2023-10-21T16:22:14.513629+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:34:33.716145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.500929+0000","last_change":"2023-10-21T16:22:13.479695+0000","last_active":"2023-10-21T16:22:14.500929+0000","last_peered":"2023-10-21T16:22:14.500929+0000","last_clean":"2023-10-21T16:22:14.500929+0000","last_became_active":"2023-10-21T16:22:13.479414+0000","last_became_peered":"2023-10-21T16:22:13.479414+0000","last_unstale":"2023-10-21T16:22:14.500929+0000","last_undegraded":"2023-10-21T16:22:14.500929+0000","last_fullsized":"2023-10-21T16:22:14.500929+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:49.120684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"150'4774","reported_seq":5350,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-21T16:22:13.467116+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:22:13.467116+0000","last_peered":"2023-10-21T16:22:13.467116+0000","last_clean":"2023-10-21T16:22:13.467116+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:22:13.467116+0000","last_undegraded":"2023-10-21T16:22:13.467116+0000","last_fullsized":"2023-10-21T16:22:13.467116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":407,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":407,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4774,"ondisk_log_size":4774,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575088,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":2605056,"data_stored":1606922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.075,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.60399999999999998,"15min":0.51300000000000001},"max":{"1min":1.6919999999999999,"5min":1.758,"15min":1.8400000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.169},"min":{"1min":0.755,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.4930000000000001,"5min":1.7270000000000001,"15min":2.448},"last":1.1040000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.149,"15min":1.161},"min":{"1min":0.76100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.601,"5min":1.851,"15min":1.915},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1759999999999999,"15min":1.226},"min":{"1min":0.70299999999999996,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.349}]},{"osd":2,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.2689999999999999,"15min":1.2989999999999999},"min":{"1min":0.79900000000000004,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.704,"5min":1.782,"15min":2.085},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.85199999999999998,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.837,"5min":1.837,"15min":2.6070000000000002},"last":1.393}]},{"osd":3,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.206,"15min":1.248},"min":{"1min":0.878,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5880000000000001,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.369,"15min":1.421},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.4530000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3149999999999999,"15min":1.3500000000000001},"min":{"1min":1.032,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.794,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4339999999999999,"15min":1.4379999999999999},"min":{"1min":1.3120000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.355},"last":1.7929999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4299999999999999,"15min":1.403},"min":{"1min":1.0249999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4359999999999999,"15min":1.4399999999999999},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.016,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.6000000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.532,"15min":1.5489999999999999},"min":{"1min":1.129,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.9890000000000001,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4419999999999999,"15min":1.4570000000000001},"min":{"1min":0.95599999999999996,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":2.02,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":1.8819999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":2002944,"data_stored":1015524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0860000000000001,"15min":1.1220000000000001},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.704,"5min":1.869,"15min":1.869},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.125,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.61299999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.895},"last":0.96099999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1379999999999999,"15min":1.1899999999999999},"min":{"1min":0.80200000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.835,"5min":1.835,"15min":2.2309999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1870000000000001,"15min":1.2829999999999999},"min":{"1min":0.81499999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5309999999999999,"5min":1.97,"15min":2.2669999999999999},"last":1.3280000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2270000000000001,"15min":1.2509999999999999},"min":{"1min":0.90400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.726,"5min":1.895,"15min":2.5339999999999998},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4330000000000001,"5min":1.821,"15min":1.821},"last":1.028}]},{"osd":3,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3129999999999999,"15min":1.3480000000000001},"min":{"1min":1.014,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8049999999999999,"5min":1.907,"15min":2.2000000000000002},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.4159999999999999},"min":{"1min":0.93000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.845,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.49}]},{"osd":4,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3540000000000001,"15min":1.339},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6890000000000001,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4299999999999999,"15min":1.395},"min":{"1min":0.74199999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.0569999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.5,"15min":1.502},"min":{"1min":0.86399999999999999,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":1.8899999999999999,"5min":2.246,"15min":2.246},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3460000000000001,"15min":1.3520000000000001},"min":{"1min":0.77300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.669,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.1619999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.476,"15min":1.5009999999999999},"min":{"1min":0.76300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9159999999999999,"5min":2.2050000000000001,"15min":2.3570000000000002},"last":1.397},{"interface":"front","average":{"1min":1.492,"5min":1.6060000000000001,"15min":1.609},"min":{"1min":0.95399999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.931,"5min":2.319,"15min":2.319},"last":1.425}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2011136,"data_stored":1021566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.60999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.6919999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1759999999999999,"15min":1.1879999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.52400000000000002},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1830000000000001,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.47,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.363},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.222},"min":{"1min":0.81200000000000006,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.579,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.69799999999999995,"5min":0.624,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.21,"15min":1.2290000000000001},"min":{"1min":0.80500000000000005,"5min":0.53100000000000003,"15min":0.46999999999999997},"max":{"1min":1.879,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.292}]},{"osd":3,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.93700000000000006,"5min":0.78900000000000003,"15min":0.57599999999999996},"max":{"1min":1.6659999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.524},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3839999999999999,"15min":1.371},"min":{"1min":0.83999999999999997,"5min":0.77000000000000002,"15min":0.66800000000000004},"max":{"1min":1.6319999999999999,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.6870000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2769999999999999,"15min":1.2150000000000001},"min":{"1min":0.77900000000000003,"5min":0.52600000000000002,"15min":0.48899999999999999},"max":{"1min":1.7829999999999999,"5min":4.359,"15min":4.359},"last":1.472},{"interface":"front","average":{"1min":1.286,"5min":1.399,"15min":1.343},"min":{"1min":0.85499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.766,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.627}]},{"osd":6,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4650000000000001,"15min":1.4319999999999999},"min":{"1min":0.878,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7490000000000001,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.502,"15min":1.472},"min":{"1min":0.94699999999999995,"5min":0.79400000000000004,"15min":0.60099999999999998},"max":{"1min":2.0139999999999998,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.8620000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.5169999999999999,"15min":1.492},"min":{"1min":1.01,"5min":0.68999999999999995,"15min":0.58299999999999996},"max":{"1min":1.6639999999999999,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.718},{"interface":"front","average":{"1min":1.476,"5min":1.599,"15min":1.538},"min":{"1min":0.83099999999999996,"5min":0.76700000000000002,"15min":0.60999999999999999},"max":{"1min":1.8340000000000001,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.7769999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575088,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2007040,"data_stored":1021544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.163},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6679999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.349},{"interface":"front","average":{"1min":1.242,"5min":1.222,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8360000000000001,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":1.1970000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.268},"min":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.621},"max":{"1min":1.6279999999999999,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.2549999999999999,"15min":1.244},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.64500000000000002},"max":{"1min":1.3640000000000001,"5min":7.484,"15min":7.484},"last":1.548}]},{"osd":2,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.794,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":0.92400000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0539999999999998,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.351,"15min":1.323},"min":{"1min":0.79500000000000004,"5min":0.64800000000000002,"15min":0.60399999999999998},"max":{"1min":1.6930000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.429,"15min":1.4019999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0009999999999999,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.407}]},{"osd":5,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3759999999999999,"15min":1.3180000000000001},"min":{"1min":0.97999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2309999999999999,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.085},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4039999999999999,"15min":1.3520000000000001},"min":{"1min":0.88,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9490000000000001,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.486,"15min":1.4430000000000001},"min":{"1min":0.84399999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.9399999999999999,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.669,"5min":1.625,"15min":1.5880000000000001},"min":{"1min":1.4690000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.2919999999999998,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.379}]},{"osd":7,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.611,"15min":1.518},"min":{"1min":1.04,"5min":0.62,"15min":0.62},"max":{"1min":2.1779999999999999,"5min":12.458,"15min":12.458},"last":1.147},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.615,"15min":1.575},"min":{"1min":0.88,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1349999999999998,"5min":3.9510000000000001,"15min":3.968},"last":1.226}]}]},{"osd":3,"up_from":13,"seq":55834575088,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41472,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330368,"statfs":{"total":96636764160,"available":96594296832,"internally_reserved":0,"allocated":2011136,"data_stored":1016534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.879,"15min":0.93899999999999995},"min":{"1min":0.442,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4159999999999999,"5min":1.946,"15min":1.946},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1659999999999999},"min":{"1min":0.52300000000000002,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.04},"last":1.254}]},{"osd":1,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":0.97399999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.589,"15min":2.173},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.228},"min":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8100000000000001},"last":1.3140000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0509999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.643,"5min":1.669,"15min":2.0049999999999999},"last":1.123},{"interface":"front","average":{"1min":1.077,"5min":1.135,"15min":1.202},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.46600000000000003},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.847},"last":1.0489999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.319,"15min":1.393},"min":{"1min":0.70399999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5780000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3899999999999999},"min":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.5800000000000001,"5min":1.863,"15min":2.919},"last":1.3879999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.327,"15min":1.393},"min":{"1min":0.65900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.02},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4179999999999999,"15min":1.4870000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.75700000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.423},"last":1.6930000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4139999999999999,"15min":1.478},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.8400000000000001,"15min":2.6179999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.419,"15min":1.508},"min":{"1min":0.75700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.704,"5min":1.8999999999999999,"15min":2.2290000000000001},"last":1.766}]},{"osd":7,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4950000000000001,"15min":1.5740000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3759999999999999},"last":1.804},{"interface":"front","average":{"1min":1.504,"5min":1.52,"15min":1.587},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.7989999999999999},"last":1.724}]}]},{"osd":2,"up_from":13,"seq":55834575089,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2015232,"data_stored":1022160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.857,"5min":1.857,"15min":1.859},"last":0.372},{"interface":"front","average":{"1min":0.87,"5min":0.92900000000000005,"15min":0.96999999999999997},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.28299999999999997},"max":{"1min":1.6259999999999999,"5min":1.905,"15min":2.0539999999999998},"last":0.499}]},{"osd":1,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0269999999999999,"15min":1.103},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.39500000000000002},"max":{"1min":1.5820000000000001,"5min":1.6890000000000001,"15min":1.9390000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.159,"15min":1.232},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6919999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0840000000000001,"15min":1.1160000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.897,"5min":1.9279999999999999,"15min":2.0710000000000002},"last":0.623},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.276,"15min":1.347},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.895,"5min":2.069,"15min":2.069},"last":0.71299999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.321,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.76600000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.319,"5min":1.3740000000000001,"15min":1.3700000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60199999999999998},"max":{"1min":1.95,"5min":2.278,"15min":2.6960000000000002},"last":1.0389999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.571,"5min":2.024,"15min":2.2730000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.4630000000000001,"15min":1.506},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.024,"5min":2.1280000000000001,"15min":2.1349999999999998},"last":1}]},{"osd":6,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4650000000000001,"15min":1.478},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"max":{"1min":2.0600000000000001,"5min":2.444,"15min":2.6819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78900000000000003},"max":{"1min":2.121,"5min":2.2669999999999999,"15min":2.552},"last":1.05}]},{"osd":7,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5049999999999999,"15min":1.5289999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.69999999999999996},"max":{"1min":1.8460000000000001,"5min":2.2130000000000001,"15min":2.4969999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.468,"15min":1.5049999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.3999999999999999},"last":0.69299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575088,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1014922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.90000000000000002},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.1100000000000001,"5min":1.806,"15min":1.905},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.0469999999999999,"15min":1.0680000000000001},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4990000000000001,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.651,"5min":2.222,"15min":2.2639999999999998},"last":1.304},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.167},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":1.615,"5min":1.6910000000000001,"15min":1.974},"last":1.1910000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.228,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.54800000000000004,"15min":0.51600000000000001},"max":{"1min":1.5649999999999999,"5min":1.911,"15min":1.9510000000000001},"last":1.496},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.526,"5min":1.6879999999999999,"15min":1.9159999999999999},"last":1.238}]},{"osd":4,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.238,"15min":1.268},"min":{"1min":0.82999999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.379,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.395},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.403,"15min":1.4239999999999999},"min":{"1min":1.024,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.9019999999999999,"5min":2.145,"15min":2.2719999999999998},"last":1.357}]},{"osd":5,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":1.2410000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.4590000000000001},"min":{"1min":1.2010000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.627,"5min":2.077,"15min":2.0880000000000001},"last":1.575}]},{"osd":6,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.482,"15min":1.49},"min":{"1min":0.85099999999999998,"5min":0.77500000000000002,"15min":0.68500000000000005},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.4079999999999999},"last":1.429},{"interface":"front","average":{"1min":1.367,"5min":1.417,"15min":1.4510000000000001},"min":{"1min":0.94499999999999995,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.746,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.46}]},{"osd":7,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5549999999999999,"15min":1.5940000000000001},"min":{"1min":1.1619999999999999,"5min":0.64600000000000002,"15min":0.59299999999999997},"max":{"1min":1.804,"5min":2.2530000000000001,"15min":2.3730000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.89000000000000001,"5min":0.74399999999999999,"15min":0.68600000000000005},"max":{"1min":2.1110000000000002,"5min":2.1850000000000001,"15min":2.367},"last":1.7729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575088,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42044,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329796,"statfs":{"total":96636764160,"available":96593711104,"internally_reserved":0,"allocated":2596864,"data_stored":1605276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84799999999999998,"15min":0.90300000000000002},"min":{"1min":0.51200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.194,"5min":1.623,"15min":1.855},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.96799999999999997,"15min":1.046},"min":{"1min":0.45600000000000002,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.667},"last":0.82899999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.018,"15min":1.0840000000000001},"min":{"1min":0.57299999999999995,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4370000000000001,"5min":1.679,"15min":1.7410000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0409999999999999,"15min":1.1220000000000001},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.488,"5min":1.603,"15min":1.732},"last":1.643}]},{"osd":3,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97699999999999998,"15min":1.073},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.375},"max":{"1min":1.3680000000000001,"5min":1.702,"15min":2.1429999999999998},"last":1.169},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1699999999999999,"15min":1.264},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6040000000000001,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":1.0289999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.3959999999999999},"min":{"1min":0.81599999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.724,"5min":1.9019999999999999,"15min":2.0339999999999998},"last":1.393},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.431,"15min":1.5049999999999999},"min":{"1min":1.113,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8120000000000001,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.5800000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2989999999999999,"15min":1.3640000000000001},"min":{"1min":0.89900000000000002,"5min":0.64000000000000001,"15min":0.57699999999999996},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.137},"last":1.083},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.47},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":1.365}]},{"osd":6,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3720000000000001,"15min":1.4550000000000001},"min":{"1min":0.76400000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.829,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.23},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4630000000000001,"15min":1.5569999999999999},"min":{"1min":1.006,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.885,"5min":3.097,"15min":3.097},"last":1.415}]},{"osd":7,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.333,"15min":1.423},"min":{"1min":0.84799999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.9379999999999999,"5min":2.0289999999999999,"15min":2.2959999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.4239999999999999,"15min":1.538},"min":{"1min":0.82099999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"max":{"1min":1.7310000000000001,"5min":2.0600000000000001,"15min":2.3540000000000001},"last":1.4610000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:22:19.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T16:22:19.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.268+0000 7f899fca2640 1 Processor -- start 2023-10-21T16:22:19.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.268+0000 7f899fca2640 1 -- start start 2023-10-21T16:22:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.270+0000 7f899fca2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 0x7f8998162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.270+0000 7f899fca2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980ae140 con 0x7f89980cb150 2023-10-21T16:22:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.270+0000 7f899fca2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980ccf20 con 0x7f89980add60 2023-10-21T16:22:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.270+0000 7f899fca2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8998161f10 con 0x7f899806d4c0 2023-10-21T16:22:19.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.271+0000 7f899e23f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f89980cb150 0x7f8998164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45764/0 (socket says 172.21.15.53:45764) 2023-10-21T16:22:19.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.271+0000 7f899e23f640 1 -- 172.21.15.53:0/2610649579 learned_addr learned my addr 172.21.15.53:0/2610649579 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.271+0000 7f899d23d640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 0x7f8998162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.271+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415537911 0 0) 0x7f89980ae140 con 0x7f89980cb150 2023-10-21T16:22:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.271+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8984003520 con 0x7f89980cb150 2023-10-21T16:22:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614904205 0 0) 0x7f89980ccf20 con 0x7f89980add60 2023-10-21T16:22:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89980ae140 con 0x7f89980add60 2023-10-21T16:22:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171298061 0 0) 0x7f8984003520 con 0x7f89980cb150 2023-10-21T16:22:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 msgr2=0x7f8998162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 0x7f8998162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 >> v1:172.21.15.124:6789/0 conn(0x7f89980add60 legacy=0x7f89980cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980cd1c0 con 0x7f89980cb150 2023-10-21T16:22:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8990001670 con 0x7f89980cb150 2023-10-21T16:22:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 >> v1:172.21.15.53:6789/0 conn(0x7f89980cb150 legacy=0x7f8998164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8984003b30 0x7f8984003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89840047c0 0x7f8984004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 0x7f89840051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998161f10 con 0x7f89840047c0 2023-10-21T16:22:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898400bb10 con 0x7f8984003b30 2023-10-21T16:22:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898400bca0 con 0x7f899806d4c0 2023-10-21T16:22:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.272+0000 7f899d23d640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89840047c0 0x7f8984004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899d23d640 1 -- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 msgr2=0x7f89840051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899d23d640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899806d4c0 0x7f89840051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899d23d640 1 -- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8984003b30 msgr2=0x7f8984003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899da3e640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8984003b30 0x7f8984003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899d23d640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8984003b30 0x7f8984003f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.273+0000 7f899d23d640 1 -- 172.21.15.53:0/2610649579 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8984003520 con 0x7f89840047c0 2023-10-21T16:22:19.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.274+0000 7f899d23d640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89840047c0 0x7f8984004c40 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8988009af0 tx=0x7f89840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee17ae36f13b76e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898802c710 con 0x7f89840047c0 2023-10-21T16:22:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898802c870 con 0x7f89840047c0 2023-10-21T16:22:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899ca3c640 1 -- 172.21.15.53:0/2610649579 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898802cc00 con 0x7f89840047c0 2023-10-21T16:22:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899fca2640 1 -- 172.21.15.53:0/2610649579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89840047c0 msgr2=0x7f8984004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899fca2640 1 --2- 172.21.15.53:0/2610649579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89840047c0 0x7f8984004c40 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8988009af0 tx=0x7f89840057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899e23f640 1 -- 172.21.15.53:0/2610649579 reap_dead start 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899fca2640 1 -- 172.21.15.53:0/2610649579 shutdown_connections 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.275+0000 7f899fca2640 1 -- 172.21.15.53:0/2610649579 >> 172.21.15.53:0/2610649579 conn(0x7f899806a380 msgr2=0x7f89980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.276+0000 7f899fca2640 1 -- 172.21.15.53:0/2610649579 shutdown_connections 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.276+0000 7f899fca2640 1 -- 172.21.15.53:0/2610649579 wait complete. 2023-10-21T16:22:19.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.277+0000 7f899fca2640 1 Processor -- start 2023-10-21T16:22:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.277+0000 7f899fca2640 1 -- start start 2023-10-21T16:22:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89840047c0 0x7f8998155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89980ae2f0 0x7f89980b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998158ab0 con 0x7f89980b28b0 2023-10-21T16:22:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998156270 con 0x7f89840047c0 2023-10-21T16:22:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.278+0000 7f899fca2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8998157e30 con 0x7f89980ae2f0 2023-10-21T16:22:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51146/0 (socket says 172.21.15.53:51146) 2023-10-21T16:22:19.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 -- 172.21.15.53:0/2846923294 learned_addr learned my addr 172.21.15.53:0/2846923294 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 -- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89980ae2f0 msgr2=0x7f89980b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89980ae2f0 0x7f89980b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 -- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89840047c0 msgr2=0x7f8998155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89840047c0 0x7f8998155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 -- 172.21.15.53:0/2846923294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998158110 con 0x7f89980b28b0 2023-10-21T16:22:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.279+0000 7f899e23f640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f89980cb0e0 tx=0x7f899000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.280+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900074e0 con 0x7f89980b28b0 2023-10-21T16:22:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.280+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990007640 con 0x7f89980b28b0 2023-10-21T16:22:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.280+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89981583f0 con 0x7f89980b28b0 2023-10-21T16:22:19.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.280+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990004a20 con 0x7f89980b28b0 2023-10-21T16:22:19.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.280+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89980b1640 con 0x7f89980b28b0 2023-10-21T16:22:19.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.281+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8998067490 con 0x7f89980b28b0 2023-10-21T16:22:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.282+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8990004b80 con 0x7f89980b28b0 2023-10-21T16:22:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.283+0000 7f898e7fc640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 0x7f897c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.283+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89900892f0 con 0x7f89980b28b0 2023-10-21T16:22:19.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.284+0000 7f899da3e640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 0x7f897c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.285+0000 7f899da3e640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 0x7f897c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89940012e0 tx=0x7f8994009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.287+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89900540f0 con 0x7f89980b28b0 2023-10-21T16:22:19.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.476+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 --> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8998161850 con 0x7f897c06af90 2023-10-21T16:22:19.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.498+0000 7f898e7fc640 1 -- 172.21.15.53:0/2846923294 <== mgr.36626 v2:172.21.15.124:6832/870299393 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287087 (secure 0 0 0) 0x7f8994003990 con 0x7f897c06af90 2023-10-21T16:22:19.501 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:19.505 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T16:22:19.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.504+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 msgr2=0x7f897c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.504+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 0x7f897c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f89940012e0 tx=0x7f8994009040 comp rx=0 tx=0).stop 2023-10-21T16:22:19.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.504+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 msgr2=0x7f89980b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.504+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f89980cb0e0 tx=0x7f899000c890 comp rx=0 tx=0).stop 2023-10-21T16:22:19.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 shutdown_connections 2023-10-21T16:22:19.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f897c06af90 0x7f897c06d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89980b28b0 0x7f89980b3620 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89980ae2f0 0x7f89980b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 --2- 172.21.15.53:0/2846923294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89840047c0 0x7f8998155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 >> 172.21.15.53:0/2846923294 conn(0x7f899806a380 msgr2=0x7f89981650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:19.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 shutdown_connections 2023-10-21T16:22:19.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.505+0000 7f899fca2640 1 -- 172.21.15.53:0/2846923294 wait complete. 2023-10-21T16:22:19.530 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T16:22:17.646871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":451,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4812,"ondisk_log_size":4812,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341272,"kb_used_data":16844,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754633448,"statfs":{"total":773094113280,"available":772744650752,"internally_reserved":0,"allocated":17248256,"data_stored":9324448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":332058169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.208339"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504236+0000","last_change":"2023-10-21T16:22:14.504757+0000","last_active":"2023-10-21T16:22:15.504236+0000","last_peered":"2023-10-21T16:22:15.504236+0000","last_clean":"2023-10-21T16:22:15.504236+0000","last_became_active":"2023-10-21T16:22:14.504451+0000","last_became_peered":"2023-10-21T16:22:14.504451+0000","last_unstale":"2023-10-21T16:22:15.504236+0000","last_undegraded":"2023-10-21T16:22:15.504236+0000","last_fullsized":"2023-10-21T16:22:15.504236+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:03.268724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501479+0000","last_change":"2023-10-21T16:22:14.493483+0000","last_active":"2023-10-21T16:22:15.501479+0000","last_peered":"2023-10-21T16:22:15.501479+0000","last_clean":"2023-10-21T16:22:15.501479+0000","last_became_active":"2023-10-21T16:22:14.492712+0000","last_became_peered":"2023-10-21T16:22:14.492712+0000","last_unstale":"2023-10-21T16:22:15.501479+0000","last_undegraded":"2023-10-21T16:22:15.501479+0000","last_fullsized":"2023-10-21T16:22:15.501479+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:59:50.197807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513240+0000","last_change":"2023-10-21T16:22:14.511503+0000","last_active":"2023-10-21T16:22:14.513240+0000","last_peered":"2023-10-21T16:22:14.513240+0000","last_clean":"2023-10-21T16:22:14.513240+0000","last_became_active":"2023-10-21T16:22:14.510923+0000","last_became_peered":"2023-10-21T16:22:14.510923+0000","last_unstale":"2023-10-21T16:22:14.513240+0000","last_undegraded":"2023-10-21T16:22:14.513240+0000","last_fullsized":"2023-10-21T16:22:14.513240+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:25.550536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503264+0000","last_change":"2023-10-21T16:22:14.503525+0000","last_active":"2023-10-21T16:22:15.503264+0000","last_peered":"2023-10-21T16:22:15.503264+0000","last_clean":"2023-10-21T16:22:15.503264+0000","last_became_active":"2023-10-21T16:22:14.503255+0000","last_became_peered":"2023-10-21T16:22:14.503255+0000","last_unstale":"2023-10-21T16:22:15.503264+0000","last_undegraded":"2023-10-21T16:22:15.503264+0000","last_fullsized":"2023-10-21T16:22:15.503264+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:55.610506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508635+0000","last_change":"2023-10-21T16:22:14.500503+0000","last_active":"2023-10-21T16:22:14.508635+0000","last_peered":"2023-10-21T16:22:14.508635+0000","last_clean":"2023-10-21T16:22:14.508635+0000","last_became_active":"2023-10-21T16:22:14.499935+0000","last_became_peered":"2023-10-21T16:22:14.499935+0000","last_unstale":"2023-10-21T16:22:14.508635+0000","last_undegraded":"2023-10-21T16:22:14.508635+0000","last_fullsized":"2023-10-21T16:22:14.508635+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.519772+0000","last_change":"2023-10-21T16:22:14.516762+0000","last_active":"2023-10-21T16:22:14.519772+0000","last_peered":"2023-10-21T16:22:14.519772+0000","last_clean":"2023-10-21T16:22:14.519772+0000","last_became_active":"2023-10-21T16:22:14.516478+0000","last_became_peered":"2023-10-21T16:22:14.516478+0000","last_unstale":"2023-10-21T16:22:14.519772+0000","last_undegraded":"2023-10-21T16:22:14.519772+0000","last_fullsized":"2023-10-21T16:22:14.519772+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:40.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515725+0000","last_change":"2023-10-21T16:22:14.512324+0000","last_active":"2023-10-21T16:22:14.515725+0000","last_peered":"2023-10-21T16:22:14.515725+0000","last_clean":"2023-10-21T16:22:14.515725+0000","last_became_active":"2023-10-21T16:22:14.511444+0000","last_became_peered":"2023-10-21T16:22:14.511444+0000","last_unstale":"2023-10-21T16:22:14.515725+0000","last_undegraded":"2023-10-21T16:22:14.515725+0000","last_fullsized":"2023-10-21T16:22:14.515725+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:46.439939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.505919+0000","last_change":"2023-10-21T16:22:14.505159+0000","last_active":"2023-10-21T16:22:15.505919+0000","last_peered":"2023-10-21T16:22:15.505919+0000","last_clean":"2023-10-21T16:22:15.505919+0000","last_became_active":"2023-10-21T16:22:14.500209+0000","last_became_peered":"2023-10-21T16:22:14.500209+0000","last_unstale":"2023-10-21T16:22:15.505919+0000","last_undegraded":"2023-10-21T16:22:15.505919+0000","last_fullsized":"2023-10-21T16:22:15.505919+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:02.078960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515637+0000","last_change":"2023-10-21T16:22:14.501707+0000","last_active":"2023-10-21T16:22:14.515637+0000","last_peered":"2023-10-21T16:22:14.515637+0000","last_clean":"2023-10-21T16:22:14.515637+0000","last_became_active":"2023-10-21T16:22:14.501420+0000","last_became_peered":"2023-10-21T16:22:14.501420+0000","last_unstale":"2023-10-21T16:22:14.515637+0000","last_undegraded":"2023-10-21T16:22:14.515637+0000","last_fullsized":"2023-10-21T16:22:14.515637+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:19.787563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515853+0000","last_change":"2023-10-21T16:22:14.510464+0000","last_active":"2023-10-21T16:22:14.515853+0000","last_peered":"2023-10-21T16:22:14.515853+0000","last_clean":"2023-10-21T16:22:14.515853+0000","last_became_active":"2023-10-21T16:22:14.510154+0000","last_became_peered":"2023-10-21T16:22:14.510154+0000","last_unstale":"2023-10-21T16:22:14.515853+0000","last_undegraded":"2023-10-21T16:22:14.515853+0000","last_fullsized":"2023-10-21T16:22:14.515853+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:55.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503673+0000","last_change":"2023-10-21T16:22:14.510567+0000","last_active":"2023-10-21T16:22:15.503673+0000","last_peered":"2023-10-21T16:22:15.503673+0000","last_clean":"2023-10-21T16:22:15.503673+0000","last_became_active":"2023-10-21T16:22:14.509992+0000","last_became_peered":"2023-10-21T16:22:14.509992+0000","last_unstale":"2023-10-21T16:22:15.503673+0000","last_undegraded":"2023-10-21T16:22:15.503673+0000","last_fullsized":"2023-10-21T16:22:15.503673+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504114+0000","last_change":"2023-10-21T16:22:14.512977+0000","last_active":"2023-10-21T16:22:15.504114+0000","last_peered":"2023-10-21T16:22:15.504114+0000","last_clean":"2023-10-21T16:22:15.504114+0000","last_became_active":"2023-10-21T16:22:14.512646+0000","last_became_peered":"2023-10-21T16:22:14.512646+0000","last_unstale":"2023-10-21T16:22:15.504114+0000","last_undegraded":"2023-10-21T16:22:15.504114+0000","last_fullsized":"2023-10-21T16:22:15.504114+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.239579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.521275+0000","last_change":"2023-10-21T16:22:14.515210+0000","last_active":"2023-10-21T16:22:14.521275+0000","last_peered":"2023-10-21T16:22:14.521275+0000","last_clean":"2023-10-21T16:22:14.521275+0000","last_became_active":"2023-10-21T16:22:14.514945+0000","last_became_peered":"2023-10-21T16:22:14.514945+0000","last_unstale":"2023-10-21T16:22:14.521275+0000","last_undegraded":"2023-10-21T16:22:14.521275+0000","last_fullsized":"2023-10-21T16:22:14.521275+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:16:37.856874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503824+0000","last_change":"2023-10-21T16:22:14.508021+0000","last_active":"2023-10-21T16:22:15.503824+0000","last_peered":"2023-10-21T16:22:15.503824+0000","last_clean":"2023-10-21T16:22:15.503824+0000","last_became_active":"2023-10-21T16:22:14.505727+0000","last_became_peered":"2023-10-21T16:22:14.505727+0000","last_unstale":"2023-10-21T16:22:15.503824+0000","last_undegraded":"2023-10-21T16:22:15.503824+0000","last_fullsized":"2023-10-21T16:22:15.503824+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:05.844283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503333+0000","last_change":"2023-10-21T16:22:14.504152+0000","last_active":"2023-10-21T16:22:15.503333+0000","last_peered":"2023-10-21T16:22:15.503333+0000","last_clean":"2023-10-21T16:22:15.503333+0000","last_became_active":"2023-10-21T16:22:14.503854+0000","last_became_peered":"2023-10-21T16:22:14.503854+0000","last_unstale":"2023-10-21T16:22:15.503333+0000","last_undegraded":"2023-10-21T16:22:15.503333+0000","last_fullsized":"2023-10-21T16:22:15.503333+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:29:16.609210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.521644+0000","last_change":"2023-10-21T16:22:14.517533+0000","last_active":"2023-10-21T16:22:14.521644+0000","last_peered":"2023-10-21T16:22:14.521644+0000","last_clean":"2023-10-21T16:22:14.521644+0000","last_became_active":"2023-10-21T16:22:14.517257+0000","last_became_peered":"2023-10-21T16:22:14.517257+0000","last_unstale":"2023-10-21T16:22:14.521644+0000","last_undegraded":"2023-10-21T16:22:14.521644+0000","last_fullsized":"2023-10-21T16:22:14.521644+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:56.254381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525004+0000","last_change":"2023-10-21T16:22:14.521071+0000","last_active":"2023-10-21T16:22:14.525004+0000","last_peered":"2023-10-21T16:22:14.525004+0000","last_clean":"2023-10-21T16:22:14.525004+0000","last_became_active":"2023-10-21T16:22:14.515838+0000","last_became_peered":"2023-10-21T16:22:14.515838+0000","last_unstale":"2023-10-21T16:22:14.525004+0000","last_undegraded":"2023-10-21T16:22:14.525004+0000","last_fullsized":"2023-10-21T16:22:14.525004+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:10:02.078960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513305+0000","last_change":"2023-10-21T16:22:14.510118+0000","last_active":"2023-10-21T16:22:14.513305+0000","last_peered":"2023-10-21T16:22:14.513305+0000","last_clean":"2023-10-21T16:22:14.513305+0000","last_became_active":"2023-10-21T16:22:14.509450+0000","last_became_peered":"2023-10-21T16:22:14.509450+0000","last_unstale":"2023-10-21T16:22:14.513305+0000","last_undegraded":"2023-10-21T16:22:14.513305+0000","last_fullsized":"2023-10-21T16:22:14.513305+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:27:49.644053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.516485+0000","last_change":"2023-10-21T16:22:14.513847+0000","last_active":"2023-10-21T16:22:14.516485+0000","last_peered":"2023-10-21T16:22:14.516485+0000","last_clean":"2023-10-21T16:22:14.516485+0000","last_became_active":"2023-10-21T16:22:14.513590+0000","last_became_peered":"2023-10-21T16:22:14.513590+0000","last_unstale":"2023-10-21T16:22:14.516485+0000","last_undegraded":"2023-10-21T16:22:14.516485+0000","last_fullsized":"2023-10-21T16:22:14.516485+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:46.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513028+0000","last_change":"2023-10-21T16:22:14.509959+0000","last_active":"2023-10-21T16:22:14.513028+0000","last_peered":"2023-10-21T16:22:14.513028+0000","last_clean":"2023-10-21T16:22:14.513028+0000","last_became_active":"2023-10-21T16:22:14.509655+0000","last_became_peered":"2023-10-21T16:22:14.509655+0000","last_unstale":"2023-10-21T16:22:14.513028+0000","last_undegraded":"2023-10-21T16:22:14.513028+0000","last_fullsized":"2023-10-21T16:22:14.513028+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:57.204127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502858+0000","last_change":"2023-10-21T16:22:14.500133+0000","last_active":"2023-10-21T16:22:15.502858+0000","last_peered":"2023-10-21T16:22:15.502858+0000","last_clean":"2023-10-21T16:22:15.502858+0000","last_became_active":"2023-10-21T16:22:14.499858+0000","last_became_peered":"2023-10-21T16:22:14.499858+0000","last_unstale":"2023-10-21T16:22:15.502858+0000","last_undegraded":"2023-10-21T16:22:15.502858+0000","last_fullsized":"2023-10-21T16:22:15.502858+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:23:15.041609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503554+0000","last_change":"2023-10-21T16:22:14.504922+0000","last_active":"2023-10-21T16:22:15.503554+0000","last_peered":"2023-10-21T16:22:15.503554+0000","last_clean":"2023-10-21T16:22:15.503554+0000","last_became_active":"2023-10-21T16:22:14.504571+0000","last_became_peered":"2023-10-21T16:22:14.504571+0000","last_unstale":"2023-10-21T16:22:15.503554+0000","last_undegraded":"2023-10-21T16:22:15.503554+0000","last_fullsized":"2023-10-21T16:22:15.503554+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:04.796713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501674+0000","last_change":"2023-10-21T16:22:14.505457+0000","last_active":"2023-10-21T16:22:15.501674+0000","last_peered":"2023-10-21T16:22:15.501674+0000","last_clean":"2023-10-21T16:22:15.501674+0000","last_became_active":"2023-10-21T16:22:14.502162+0000","last_became_peered":"2023-10-21T16:22:14.502162+0000","last_unstale":"2023-10-21T16:22:15.501674+0000","last_undegraded":"2023-10-21T16:22:15.501674+0000","last_fullsized":"2023-10-21T16:22:15.501674+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:22.604905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.514171+0000","last_change":"2023-10-21T16:22:14.501052+0000","last_active":"2023-10-21T16:22:14.514171+0000","last_peered":"2023-10-21T16:22:14.514171+0000","last_clean":"2023-10-21T16:22:14.514171+0000","last_became_active":"2023-10-21T16:22:14.500455+0000","last_became_peered":"2023-10-21T16:22:14.500455+0000","last_unstale":"2023-10-21T16:22:14.514171+0000","last_undegraded":"2023-10-21T16:22:14.514171+0000","last_fullsized":"2023-10-21T16:22:14.514171+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:35.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.514477+0000","last_change":"2023-10-21T16:22:14.510939+0000","last_active":"2023-10-21T16:22:14.514477+0000","last_peered":"2023-10-21T16:22:14.514477+0000","last_clean":"2023-10-21T16:22:14.514477+0000","last_became_active":"2023-10-21T16:22:14.510669+0000","last_became_peered":"2023-10-21T16:22:14.510669+0000","last_unstale":"2023-10-21T16:22:14.514477+0000","last_undegraded":"2023-10-21T16:22:14.514477+0000","last_fullsized":"2023-10-21T16:22:14.514477+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:51:13.942978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520917+0000","last_change":"2023-10-21T16:22:14.518046+0000","last_active":"2023-10-21T16:22:14.520917+0000","last_peered":"2023-10-21T16:22:14.520917+0000","last_clean":"2023-10-21T16:22:14.520917+0000","last_became_active":"2023-10-21T16:22:14.517748+0000","last_became_peered":"2023-10-21T16:22:14.517748+0000","last_unstale":"2023-10-21T16:22:14.520917+0000","last_undegraded":"2023-10-21T16:22:14.520917+0000","last_fullsized":"2023-10-21T16:22:14.520917+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:01.267963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523942+0000","last_change":"2023-10-21T16:22:14.521950+0000","last_active":"2023-10-21T16:22:14.523942+0000","last_peered":"2023-10-21T16:22:14.523942+0000","last_clean":"2023-10-21T16:22:14.523942+0000","last_became_active":"2023-10-21T16:22:14.521689+0000","last_became_peered":"2023-10-21T16:22:14.521689+0000","last_unstale":"2023-10-21T16:22:14.523942+0000","last_undegraded":"2023-10-21T16:22:14.523942+0000","last_fullsized":"2023-10-21T16:22:14.523942+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:47.037388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.502579+0000","last_change":"2023-10-21T16:22:14.496623+0000","last_active":"2023-10-21T16:22:14.502579+0000","last_peered":"2023-10-21T16:22:14.502579+0000","last_clean":"2023-10-21T16:22:14.502579+0000","last_became_active":"2023-10-21T16:22:14.496316+0000","last_became_peered":"2023-10-21T16:22:14.496316+0000","last_unstale":"2023-10-21T16:22:14.502579+0000","last_undegraded":"2023-10-21T16:22:14.502579+0000","last_fullsized":"2023-10-21T16:22:14.502579+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:48.253170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522367+0000","last_change":"2023-10-21T16:22:14.518532+0000","last_active":"2023-10-21T16:22:14.522367+0000","last_peered":"2023-10-21T16:22:14.522367+0000","last_clean":"2023-10-21T16:22:14.522367+0000","last_became_active":"2023-10-21T16:22:14.518257+0000","last_became_peered":"2023-10-21T16:22:14.518257+0000","last_unstale":"2023-10-21T16:22:14.522367+0000","last_undegraded":"2023-10-21T16:22:14.522367+0000","last_fullsized":"2023-10-21T16:22:14.522367+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:21.175037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520912+0000","last_change":"2023-10-21T16:22:14.515236+0000","last_active":"2023-10-21T16:22:14.520912+0000","last_peered":"2023-10-21T16:22:14.520912+0000","last_clean":"2023-10-21T16:22:14.520912+0000","last_became_active":"2023-10-21T16:22:14.514214+0000","last_became_peered":"2023-10-21T16:22:14.514214+0000","last_unstale":"2023-10-21T16:22:14.520912+0000","last_undegraded":"2023-10-21T16:22:14.520912+0000","last_fullsized":"2023-10-21T16:22:14.520912+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:58.690098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506007+0000","last_change":"2023-10-21T16:22:14.498790+0000","last_active":"2023-10-21T16:22:15.506007+0000","last_peered":"2023-10-21T16:22:15.506007+0000","last_clean":"2023-10-21T16:22:15.506007+0000","last_became_active":"2023-10-21T16:22:14.498443+0000","last_became_peered":"2023-10-21T16:22:14.498443+0000","last_unstale":"2023-10-21T16:22:15.506007+0000","last_undegraded":"2023-10-21T16:22:15.506007+0000","last_fullsized":"2023-10-21T16:22:15.506007+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:59:42.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508628+0000","last_change":"2023-10-21T16:22:14.498870+0000","last_active":"2023-10-21T16:22:14.508628+0000","last_peered":"2023-10-21T16:22:14.508628+0000","last_clean":"2023-10-21T16:22:14.508628+0000","last_became_active":"2023-10-21T16:22:14.498588+0000","last_became_peered":"2023-10-21T16:22:14.498588+0000","last_unstale":"2023-10-21T16:22:14.508628+0000","last_undegraded":"2023-10-21T16:22:14.508628+0000","last_fullsized":"2023-10-21T16:22:14.508628+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:08:55.610506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.505919+0000","last_change":"2023-10-21T16:22:14.499944+0000","last_active":"2023-10-21T16:22:15.505919+0000","last_peered":"2023-10-21T16:22:15.505919+0000","last_clean":"2023-10-21T16:22:15.505919+0000","last_became_active":"2023-10-21T16:22:14.499595+0000","last_became_peered":"2023-10-21T16:22:14.499595+0000","last_unstale":"2023-10-21T16:22:15.505919+0000","last_undegraded":"2023-10-21T16:22:15.505919+0000","last_fullsized":"2023-10-21T16:22:15.505919+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:59.302925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"160'4","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529091+0000","last_change":"2023-10-21T16:22:13.481657+0000","last_active":"2023-10-21T16:22:14.529091+0000","last_peered":"2023-10-21T16:22:14.529091+0000","last_clean":"2023-10-21T16:22:14.529091+0000","last_became_active":"2023-10-21T16:22:13.480257+0000","last_became_peered":"2023-10-21T16:22:13.480257+0000","last_unstale":"2023-10-21T16:22:14.529091+0000","last_undegraded":"2023-10-21T16:22:14.529091+0000","last_fullsized":"2023-10-21T16:22:14.529091+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:28.038285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503554+0000","last_change":"2023-10-21T16:22:14.490386+0000","last_active":"2023-10-21T16:22:15.503554+0000","last_peered":"2023-10-21T16:22:15.503554+0000","last_clean":"2023-10-21T16:22:15.503554+0000","last_became_active":"2023-10-21T16:22:14.490035+0000","last_became_peered":"2023-10-21T16:22:14.490035+0000","last_unstale":"2023-10-21T16:22:15.503554+0000","last_undegraded":"2023-10-21T16:22:15.503554+0000","last_fullsized":"2023-10-21T16:22:15.503554+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:06:45.882520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528796+0000","last_change":"2023-10-21T16:22:13.476830+0000","last_active":"2023-10-21T16:22:14.528796+0000","last_peered":"2023-10-21T16:22:14.528796+0000","last_clean":"2023-10-21T16:22:14.528796+0000","last_became_active":"2023-10-21T16:22:13.471642+0000","last_became_peered":"2023-10-21T16:22:13.471642+0000","last_unstale":"2023-10-21T16:22:14.528796+0000","last_undegraded":"2023-10-21T16:22:14.528796+0000","last_fullsized":"2023-10-21T16:22:14.528796+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:32:39.388093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502220+0000","last_change":"2023-10-21T16:22:14.503952+0000","last_active":"2023-10-21T16:22:15.502220+0000","last_peered":"2023-10-21T16:22:15.502220+0000","last_clean":"2023-10-21T16:22:15.502220+0000","last_became_active":"2023-10-21T16:22:14.503675+0000","last_became_peered":"2023-10-21T16:22:14.503675+0000","last_unstale":"2023-10-21T16:22:15.502220+0000","last_undegraded":"2023-10-21T16:22:15.502220+0000","last_fullsized":"2023-10-21T16:22:15.502220+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:53:30.875927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525718+0000","last_change":"2023-10-21T16:22:13.477799+0000","last_active":"2023-10-21T16:22:14.525718+0000","last_peered":"2023-10-21T16:22:14.525718+0000","last_clean":"2023-10-21T16:22:14.525718+0000","last_became_active":"2023-10-21T16:22:13.477487+0000","last_became_peered":"2023-10-21T16:22:13.477487+0000","last_unstale":"2023-10-21T16:22:14.525718+0000","last_undegraded":"2023-10-21T16:22:14.525718+0000","last_fullsized":"2023-10-21T16:22:14.525718+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:52.593235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503715+0000","last_change":"2023-10-21T16:22:14.504774+0000","last_active":"2023-10-21T16:22:15.503715+0000","last_peered":"2023-10-21T16:22:15.503715+0000","last_clean":"2023-10-21T16:22:15.503715+0000","last_became_active":"2023-10-21T16:22:14.504353+0000","last_became_peered":"2023-10-21T16:22:14.504353+0000","last_unstale":"2023-10-21T16:22:15.503715+0000","last_undegraded":"2023-10-21T16:22:15.503715+0000","last_fullsized":"2023-10-21T16:22:15.503715+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:05:29.265989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525062+0000","last_change":"2023-10-21T16:22:13.483247+0000","last_active":"2023-10-21T16:22:14.525062+0000","last_peered":"2023-10-21T16:22:14.525062+0000","last_clean":"2023-10-21T16:22:14.525062+0000","last_became_active":"2023-10-21T16:22:13.482554+0000","last_became_peered":"2023-10-21T16:22:13.482554+0000","last_unstale":"2023-10-21T16:22:14.525062+0000","last_undegraded":"2023-10-21T16:22:14.525062+0000","last_fullsized":"2023-10-21T16:22:14.525062+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:05.061044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506994+0000","last_change":"2023-10-21T16:22:14.498509+0000","last_active":"2023-10-21T16:22:15.506994+0000","last_peered":"2023-10-21T16:22:15.506994+0000","last_clean":"2023-10-21T16:22:15.506994+0000","last_became_active":"2023-10-21T16:22:14.498111+0000","last_became_peered":"2023-10-21T16:22:14.498111+0000","last_unstale":"2023-10-21T16:22:15.506994+0000","last_undegraded":"2023-10-21T16:22:15.506994+0000","last_fullsized":"2023-10-21T16:22:15.506994+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:22.679691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"160'1","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506795+0000","last_change":"2023-10-21T16:22:13.469329+0000","last_active":"2023-10-21T16:22:15.506795+0000","last_peered":"2023-10-21T16:22:15.506795+0000","last_clean":"2023-10-21T16:22:15.506795+0000","last_became_active":"2023-10-21T16:22:13.469036+0000","last_became_peered":"2023-10-21T16:22:13.469036+0000","last_unstale":"2023-10-21T16:22:15.506795+0000","last_undegraded":"2023-10-21T16:22:15.506795+0000","last_fullsized":"2023-10-21T16:22:15.506795+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:33.346616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507515+0000","last_change":"2023-10-21T16:22:14.499996+0000","last_active":"2023-10-21T16:22:15.507515+0000","last_peered":"2023-10-21T16:22:15.507515+0000","last_clean":"2023-10-21T16:22:15.507515+0000","last_became_active":"2023-10-21T16:22:14.499636+0000","last_became_peered":"2023-10-21T16:22:14.499636+0000","last_unstale":"2023-10-21T16:22:15.507515+0000","last_undegraded":"2023-10-21T16:22:15.507515+0000","last_fullsized":"2023-10-21T16:22:15.507515+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:40.402301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529052+0000","last_change":"2023-10-21T16:22:13.468390+0000","last_active":"2023-10-21T16:22:14.529052+0000","last_peered":"2023-10-21T16:22:14.529052+0000","last_clean":"2023-10-21T16:22:14.529052+0000","last_became_active":"2023-10-21T16:22:13.468118+0000","last_became_peered":"2023-10-21T16:22:13.468118+0000","last_unstale":"2023-10-21T16:22:14.529052+0000","last_undegraded":"2023-10-21T16:22:14.529052+0000","last_fullsized":"2023-10-21T16:22:14.529052+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:53.287507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513359+0000","last_change":"2023-10-21T16:22:14.510289+0000","last_active":"2023-10-21T16:22:14.513359+0000","last_peered":"2023-10-21T16:22:14.513359+0000","last_clean":"2023-10-21T16:22:14.513359+0000","last_became_active":"2023-10-21T16:22:14.509831+0000","last_became_peered":"2023-10-21T16:22:14.509831+0000","last_unstale":"2023-10-21T16:22:14.513359+0000","last_undegraded":"2023-10-21T16:22:14.513359+0000","last_fullsized":"2023-10-21T16:22:14.513359+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:24.955473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496575+0000","last_change":"2023-10-21T16:22:13.478388+0000","last_active":"2023-10-21T16:22:14.496575+0000","last_peered":"2023-10-21T16:22:14.496575+0000","last_clean":"2023-10-21T16:22:14.496575+0000","last_became_active":"2023-10-21T16:22:13.477016+0000","last_became_peered":"2023-10-21T16:22:13.477016+0000","last_unstale":"2023-10-21T16:22:14.496575+0000","last_undegraded":"2023-10-21T16:22:14.496575+0000","last_fullsized":"2023-10-21T16:22:14.496575+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:58.286988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520375+0000","last_change":"2023-10-21T16:22:14.517302+0000","last_active":"2023-10-21T16:22:14.520375+0000","last_peered":"2023-10-21T16:22:14.520375+0000","last_clean":"2023-10-21T16:22:14.520375+0000","last_became_active":"2023-10-21T16:22:14.517024+0000","last_became_peered":"2023-10-21T16:22:14.517024+0000","last_unstale":"2023-10-21T16:22:14.520375+0000","last_undegraded":"2023-10-21T16:22:14.520375+0000","last_fullsized":"2023-10-21T16:22:14.520375+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:45:25.645903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.500798+0000","last_change":"2023-10-21T16:22:13.477906+0000","last_active":"2023-10-21T16:22:14.500798+0000","last_peered":"2023-10-21T16:22:14.500798+0000","last_clean":"2023-10-21T16:22:14.500798+0000","last_became_active":"2023-10-21T16:22:13.477158+0000","last_became_peered":"2023-10-21T16:22:13.477158+0000","last_unstale":"2023-10-21T16:22:14.500798+0000","last_undegraded":"2023-10-21T16:22:14.500798+0000","last_fullsized":"2023-10-21T16:22:14.500798+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:08:54.499269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503331+0000","last_change":"2023-10-21T16:22:14.518585+0000","last_active":"2023-10-21T16:22:15.503331+0000","last_peered":"2023-10-21T16:22:15.503331+0000","last_clean":"2023-10-21T16:22:15.503331+0000","last_became_active":"2023-10-21T16:22:14.518277+0000","last_became_peered":"2023-10-21T16:22:14.518277+0000","last_unstale":"2023-10-21T16:22:15.503331+0000","last_undegraded":"2023-10-21T16:22:15.503331+0000","last_fullsized":"2023-10-21T16:22:15.503331+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:21:00.759282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501572+0000","last_change":"2023-10-21T16:22:13.466837+0000","last_active":"2023-10-21T16:22:15.501572+0000","last_peered":"2023-10-21T16:22:15.501572+0000","last_clean":"2023-10-21T16:22:15.501572+0000","last_became_active":"2023-10-21T16:22:13.466494+0000","last_became_peered":"2023-10-21T16:22:13.466494+0000","last_unstale":"2023-10-21T16:22:15.501572+0000","last_undegraded":"2023-10-21T16:22:15.501572+0000","last_fullsized":"2023-10-21T16:22:15.501572+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:54:54.138220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525500+0000","last_change":"2023-10-21T16:22:14.521702+0000","last_active":"2023-10-21T16:22:14.525500+0000","last_peered":"2023-10-21T16:22:14.525500+0000","last_clean":"2023-10-21T16:22:14.525500+0000","last_became_active":"2023-10-21T16:22:14.511786+0000","last_became_peered":"2023-10-21T16:22:14.511786+0000","last_unstale":"2023-10-21T16:22:14.525500+0000","last_undegraded":"2023-10-21T16:22:14.525500+0000","last_fullsized":"2023-10-21T16:22:14.525500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:04.944215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504070+0000","last_change":"2023-10-21T16:22:13.456080+0000","last_active":"2023-10-21T16:22:15.504070+0000","last_peered":"2023-10-21T16:22:15.504070+0000","last_clean":"2023-10-21T16:22:15.504070+0000","last_became_active":"2023-10-21T16:22:13.455463+0000","last_became_peered":"2023-10-21T16:22:13.455463+0000","last_unstale":"2023-10-21T16:22:15.504070+0000","last_undegraded":"2023-10-21T16:22:15.504070+0000","last_fullsized":"2023-10-21T16:22:15.504070+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:41:33.898538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522414+0000","last_change":"2023-10-21T16:22:14.519074+0000","last_active":"2023-10-21T16:22:14.522414+0000","last_peered":"2023-10-21T16:22:14.522414+0000","last_clean":"2023-10-21T16:22:14.522414+0000","last_became_active":"2023-10-21T16:22:14.517849+0000","last_became_peered":"2023-10-21T16:22:14.517849+0000","last_unstale":"2023-10-21T16:22:14.522414+0000","last_undegraded":"2023-10-21T16:22:14.522414+0000","last_fullsized":"2023-10-21T16:22:14.522414+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:22:16.912931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"160'3","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.525498+0000","last_change":"2023-10-21T16:22:13.481087+0000","last_active":"2023-10-21T16:22:14.525498+0000","last_peered":"2023-10-21T16:22:14.525498+0000","last_clean":"2023-10-21T16:22:14.525498+0000","last_became_active":"2023-10-21T16:22:13.480759+0000","last_became_peered":"2023-10-21T16:22:13.480759+0000","last_unstale":"2023-10-21T16:22:14.525498+0000","last_undegraded":"2023-10-21T16:22:14.525498+0000","last_fullsized":"2023-10-21T16:22:14.525498+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:20:48.297330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.512921+0000","last_change":"2023-10-21T16:22:14.510046+0000","last_active":"2023-10-21T16:22:14.512921+0000","last_peered":"2023-10-21T16:22:14.512921+0000","last_clean":"2023-10-21T16:22:14.512921+0000","last_became_active":"2023-10-21T16:22:14.509677+0000","last_became_peered":"2023-10-21T16:22:14.509677+0000","last_unstale":"2023-10-21T16:22:14.512921+0000","last_undegraded":"2023-10-21T16:22:14.512921+0000","last_fullsized":"2023-10-21T16:22:14.512921+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:58:52.949680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"160'6","reported_seq":17,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.530492+0000","last_change":"2023-10-21T16:22:13.481392+0000","last_active":"2023-10-21T16:22:14.530492+0000","last_peered":"2023-10-21T16:22:14.530492+0000","last_clean":"2023-10-21T16:22:14.530492+0000","last_became_active":"2023-10-21T16:22:13.481108+0000","last_became_peered":"2023-10-21T16:22:13.481108+0000","last_unstale":"2023-10-21T16:22:14.530492+0000","last_undegraded":"2023-10-21T16:22:14.530492+0000","last_fullsized":"2023-10-21T16:22:14.530492+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:21.785876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523695+0000","last_change":"2023-10-21T16:22:14.514009+0000","last_active":"2023-10-21T16:22:14.523695+0000","last_peered":"2023-10-21T16:22:14.523695+0000","last_clean":"2023-10-21T16:22:14.523695+0000","last_became_active":"2023-10-21T16:22:14.513723+0000","last_became_peered":"2023-10-21T16:22:14.513723+0000","last_unstale":"2023-10-21T16:22:14.523695+0000","last_undegraded":"2023-10-21T16:22:14.523695+0000","last_fullsized":"2023-10-21T16:22:14.523695+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:39:45.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503666+0000","last_change":"2023-10-21T16:22:13.453344+0000","last_active":"2023-10-21T16:22:15.503666+0000","last_peered":"2023-10-21T16:22:15.503666+0000","last_clean":"2023-10-21T16:22:15.503666+0000","last_became_active":"2023-10-21T16:22:13.452704+0000","last_became_peered":"2023-10-21T16:22:13.452704+0000","last_unstale":"2023-10-21T16:22:15.503666+0000","last_undegraded":"2023-10-21T16:22:15.503666+0000","last_fullsized":"2023-10-21T16:22:15.503666+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:06:14.002394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.516679+0000","last_change":"2023-10-21T16:22:14.515562+0000","last_active":"2023-10-21T16:22:14.516679+0000","last_peered":"2023-10-21T16:22:14.516679+0000","last_clean":"2023-10-21T16:22:14.516679+0000","last_became_active":"2023-10-21T16:22:14.515050+0000","last_became_peered":"2023-10-21T16:22:14.515050+0000","last_unstale":"2023-10-21T16:22:14.516679+0000","last_undegraded":"2023-10-21T16:22:14.516679+0000","last_fullsized":"2023-10-21T16:22:14.516679+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:03:36.989147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.505101+0000","last_change":"2023-10-21T16:22:13.469495+0000","last_active":"2023-10-21T16:22:14.505101+0000","last_peered":"2023-10-21T16:22:14.505101+0000","last_clean":"2023-10-21T16:22:14.505101+0000","last_became_active":"2023-10-21T16:22:13.469193+0000","last_became_peered":"2023-10-21T16:22:13.469193+0000","last_unstale":"2023-10-21T16:22:14.505101+0000","last_undegraded":"2023-10-21T16:22:14.505101+0000","last_fullsized":"2023-10-21T16:22:14.505101+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:59:13.505853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.509328+0000","last_change":"2023-10-21T16:22:14.502618+0000","last_active":"2023-10-21T16:22:14.509328+0000","last_peered":"2023-10-21T16:22:14.509328+0000","last_clean":"2023-10-21T16:22:14.509328+0000","last_became_active":"2023-10-21T16:22:14.501803+0000","last_became_peered":"2023-10-21T16:22:14.501803+0000","last_unstale":"2023-10-21T16:22:14.509328+0000","last_undegraded":"2023-10-21T16:22:14.509328+0000","last_fullsized":"2023-10-21T16:22:14.509328+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:48.253170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"160'8","reported_seq":19,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.529310+0000","last_change":"2023-10-21T16:22:13.474420+0000","last_active":"2023-10-21T16:22:14.529310+0000","last_peered":"2023-10-21T16:22:14.529310+0000","last_clean":"2023-10-21T16:22:14.529310+0000","last_became_active":"2023-10-21T16:22:13.474107+0000","last_became_peered":"2023-10-21T16:22:13.474107+0000","last_unstale":"2023-10-21T16:22:14.529310+0000","last_undegraded":"2023-10-21T16:22:14.529310+0000","last_fullsized":"2023-10-21T16:22:14.529310+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:16.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513236+0000","last_change":"2023-10-21T16:22:14.510554+0000","last_active":"2023-10-21T16:22:14.513236+0000","last_peered":"2023-10-21T16:22:14.513236+0000","last_clean":"2023-10-21T16:22:14.513236+0000","last_became_active":"2023-10-21T16:22:14.510100+0000","last_became_peered":"2023-10-21T16:22:14.510100+0000","last_unstale":"2023-10-21T16:22:14.513236+0000","last_undegraded":"2023-10-21T16:22:14.513236+0000","last_fullsized":"2023-10-21T16:22:14.513236+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:49:34.035069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"160'1","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501683+0000","last_change":"2023-10-21T16:22:13.470430+0000","last_active":"2023-10-21T16:22:15.501683+0000","last_peered":"2023-10-21T16:22:15.501683+0000","last_clean":"2023-10-21T16:22:15.501683+0000","last_became_active":"2023-10-21T16:22:13.469563+0000","last_became_peered":"2023-10-21T16:22:13.469563+0000","last_unstale":"2023-10-21T16:22:15.501683+0000","last_undegraded":"2023-10-21T16:22:15.501683+0000","last_fullsized":"2023-10-21T16:22:15.501683+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:12:03.453771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515706+0000","last_change":"2023-10-21T16:22:14.512688+0000","last_active":"2023-10-21T16:22:14.515706+0000","last_peered":"2023-10-21T16:22:14.515706+0000","last_clean":"2023-10-21T16:22:14.515706+0000","last_became_active":"2023-10-21T16:22:14.512417+0000","last_became_peered":"2023-10-21T16:22:14.512417+0000","last_unstale":"2023-10-21T16:22:14.515706+0000","last_undegraded":"2023-10-21T16:22:14.515706+0000","last_fullsized":"2023-10-21T16:22:14.515706+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:28.490441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"160'2","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528298+0000","last_change":"2023-10-21T16:22:13.479960+0000","last_active":"2023-10-21T16:22:14.528298+0000","last_peered":"2023-10-21T16:22:14.528298+0000","last_clean":"2023-10-21T16:22:14.528298+0000","last_became_active":"2023-10-21T16:22:13.478731+0000","last_became_peered":"2023-10-21T16:22:13.478731+0000","last_unstale":"2023-10-21T16:22:14.528298+0000","last_undegraded":"2023-10-21T16:22:14.528298+0000","last_fullsized":"2023-10-21T16:22:14.528298+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:49:08.306960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.523700+0000","last_change":"2023-10-21T16:22:14.521797+0000","last_active":"2023-10-21T16:22:14.523700+0000","last_peered":"2023-10-21T16:22:14.523700+0000","last_clean":"2023-10-21T16:22:14.523700+0000","last_became_active":"2023-10-21T16:22:14.521468+0000","last_became_peered":"2023-10-21T16:22:14.521468+0000","last_unstale":"2023-10-21T16:22:14.523700+0000","last_undegraded":"2023-10-21T16:22:14.523700+0000","last_fullsized":"2023-10-21T16:22:14.523700+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:10.533635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"160'1","reported_seq":12,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.528344+0000","last_change":"2023-10-21T16:22:13.480152+0000","last_active":"2023-10-21T16:22:14.528344+0000","last_peered":"2023-10-21T16:22:14.528344+0000","last_clean":"2023-10-21T16:22:14.528344+0000","last_became_active":"2023-10-21T16:22:13.479768+0000","last_became_peered":"2023-10-21T16:22:13.479768+0000","last_unstale":"2023-10-21T16:22:14.528344+0000","last_undegraded":"2023-10-21T16:22:14.528344+0000","last_fullsized":"2023-10-21T16:22:14.528344+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:28:41.595996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.518069+0000","last_change":"2023-10-21T16:22:14.512268+0000","last_active":"2023-10-21T16:22:14.518069+0000","last_peered":"2023-10-21T16:22:14.518069+0000","last_clean":"2023-10-21T16:22:14.518069+0000","last_became_active":"2023-10-21T16:22:14.511856+0000","last_became_peered":"2023-10-21T16:22:14.511856+0000","last_unstale":"2023-10-21T16:22:14.518069+0000","last_undegraded":"2023-10-21T16:22:14.518069+0000","last_fullsized":"2023-10-21T16:22:14.518069+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:20.207076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.501655+0000","last_change":"2023-10-21T16:22:13.468999+0000","last_active":"2023-10-21T16:22:14.501655+0000","last_peered":"2023-10-21T16:22:14.501655+0000","last_clean":"2023-10-21T16:22:14.501655+0000","last_became_active":"2023-10-21T16:22:13.468202+0000","last_became_peered":"2023-10-21T16:22:13.468202+0000","last_unstale":"2023-10-21T16:22:14.501655+0000","last_undegraded":"2023-10-21T16:22:14.501655+0000","last_fullsized":"2023-10-21T16:22:14.501655+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:22.231166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.526771+0000","last_change":"2023-10-21T16:22:14.521492+0000","last_active":"2023-10-21T16:22:14.526771+0000","last_peered":"2023-10-21T16:22:14.526771+0000","last_clean":"2023-10-21T16:22:14.526771+0000","last_became_active":"2023-10-21T16:22:14.520945+0000","last_became_peered":"2023-10-21T16:22:14.520945+0000","last_unstale":"2023-10-21T16:22:14.526771+0000","last_undegraded":"2023-10-21T16:22:14.526771+0000","last_fullsized":"2023-10-21T16:22:14.526771+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:17.858273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.506924+0000","last_change":"2023-10-21T16:22:13.478863+0000","last_active":"2023-10-21T16:22:15.506924+0000","last_peered":"2023-10-21T16:22:15.506924+0000","last_clean":"2023-10-21T16:22:15.506924+0000","last_became_active":"2023-10-21T16:22:13.478551+0000","last_became_peered":"2023-10-21T16:22:13.478551+0000","last_unstale":"2023-10-21T16:22:15.506924+0000","last_undegraded":"2023-10-21T16:22:15.506924+0000","last_fullsized":"2023-10-21T16:22:15.506924+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:18.771626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504112+0000","last_change":"2023-10-21T16:22:14.502994+0000","last_active":"2023-10-21T16:22:15.504112+0000","last_peered":"2023-10-21T16:22:15.504112+0000","last_clean":"2023-10-21T16:22:15.504112+0000","last_became_active":"2023-10-21T16:22:14.502672+0000","last_became_peered":"2023-10-21T16:22:14.502672+0000","last_unstale":"2023-10-21T16:22:15.504112+0000","last_undegraded":"2023-10-21T16:22:15.504112+0000","last_fullsized":"2023-10-21T16:22:15.504112+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:08.772394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.503691+0000","last_change":"2023-10-21T16:22:13.466592+0000","last_active":"2023-10-21T16:22:15.503691+0000","last_peered":"2023-10-21T16:22:15.503691+0000","last_clean":"2023-10-21T16:22:15.503691+0000","last_became_active":"2023-10-21T16:22:13.463481+0000","last_became_peered":"2023-10-21T16:22:13.463481+0000","last_unstale":"2023-10-21T16:22:15.503691+0000","last_undegraded":"2023-10-21T16:22:15.503691+0000","last_fullsized":"2023-10-21T16:22:15.503691+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:33.851202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.524462+0000","last_change":"2023-10-21T16:22:14.518717+0000","last_active":"2023-10-21T16:22:14.524462+0000","last_peered":"2023-10-21T16:22:14.524462+0000","last_clean":"2023-10-21T16:22:14.524462+0000","last_became_active":"2023-10-21T16:22:14.518447+0000","last_became_peered":"2023-10-21T16:22:14.518447+0000","last_unstale":"2023-10-21T16:22:14.524462+0000","last_undegraded":"2023-10-21T16:22:14.524462+0000","last_fullsized":"2023-10-21T16:22:14.524462+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:55.299526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.499021+0000","last_change":"2023-10-21T16:22:13.479657+0000","last_active":"2023-10-21T16:22:14.499021+0000","last_peered":"2023-10-21T16:22:14.499021+0000","last_clean":"2023-10-21T16:22:14.499021+0000","last_became_active":"2023-10-21T16:22:13.479367+0000","last_became_peered":"2023-10-21T16:22:13.479367+0000","last_unstale":"2023-10-21T16:22:14.499021+0000","last_undegraded":"2023-10-21T16:22:14.499021+0000","last_fullsized":"2023-10-21T16:22:14.499021+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:03:17.583848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.501468+0000","last_change":"2023-10-21T16:22:14.518644+0000","last_active":"2023-10-21T16:22:15.501468+0000","last_peered":"2023-10-21T16:22:15.501468+0000","last_clean":"2023-10-21T16:22:15.501468+0000","last_became_active":"2023-10-21T16:22:14.518344+0000","last_became_peered":"2023-10-21T16:22:14.518344+0000","last_unstale":"2023-10-21T16:22:15.501468+0000","last_undegraded":"2023-10-21T16:22:15.501468+0000","last_fullsized":"2023-10-21T16:22:15.501468+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:38:27.311112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.494101+0000","last_change":"2023-10-21T16:22:13.476381+0000","last_active":"2023-10-21T16:22:14.494101+0000","last_peered":"2023-10-21T16:22:14.494101+0000","last_clean":"2023-10-21T16:22:14.494101+0000","last_became_active":"2023-10-21T16:22:13.476012+0000","last_became_peered":"2023-10-21T16:22:13.476012+0000","last_unstale":"2023-10-21T16:22:14.494101+0000","last_undegraded":"2023-10-21T16:22:14.494101+0000","last_fullsized":"2023-10-21T16:22:14.494101+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:59:33.346616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.509040+0000","last_change":"2023-10-21T16:22:14.504142+0000","last_active":"2023-10-21T16:22:14.509040+0000","last_peered":"2023-10-21T16:22:14.509040+0000","last_clean":"2023-10-21T16:22:14.509040+0000","last_became_active":"2023-10-21T16:22:14.503814+0000","last_became_peered":"2023-10-21T16:22:14.503814+0000","last_unstale":"2023-10-21T16:22:14.509040+0000","last_undegraded":"2023-10-21T16:22:14.509040+0000","last_fullsized":"2023-10-21T16:22:14.509040+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:20.207076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"160'2","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.504193+0000","last_change":"2023-10-21T16:22:13.475285+0000","last_active":"2023-10-21T16:22:15.504193+0000","last_peered":"2023-10-21T16:22:15.504193+0000","last_clean":"2023-10-21T16:22:15.504193+0000","last_became_active":"2023-10-21T16:22:13.474841+0000","last_became_peered":"2023-10-21T16:22:13.474841+0000","last_unstale":"2023-10-21T16:22:15.504193+0000","last_undegraded":"2023-10-21T16:22:15.504193+0000","last_fullsized":"2023-10-21T16:22:15.504193+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:47:43.435223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513532+0000","last_change":"2023-10-21T16:22:14.512145+0000","last_active":"2023-10-21T16:22:14.513532+0000","last_peered":"2023-10-21T16:22:14.513532+0000","last_clean":"2023-10-21T16:22:14.513532+0000","last_became_active":"2023-10-21T16:22:14.510260+0000","last_became_peered":"2023-10-21T16:22:14.510260+0000","last_unstale":"2023-10-21T16:22:14.513532+0000","last_undegraded":"2023-10-21T16:22:14.513532+0000","last_fullsized":"2023-10-21T16:22:14.513532+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:42:54.533693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507874+0000","last_change":"2023-10-21T16:22:13.476020+0000","last_active":"2023-10-21T16:22:15.507874+0000","last_peered":"2023-10-21T16:22:15.507874+0000","last_clean":"2023-10-21T16:22:15.507874+0000","last_became_active":"2023-10-21T16:22:13.475739+0000","last_became_peered":"2023-10-21T16:22:13.475739+0000","last_unstale":"2023-10-21T16:22:15.507874+0000","last_undegraded":"2023-10-21T16:22:15.507874+0000","last_fullsized":"2023-10-21T16:22:15.507874+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:28.211584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.515174+0000","last_change":"2023-10-21T16:22:14.511494+0000","last_active":"2023-10-21T16:22:14.515174+0000","last_peered":"2023-10-21T16:22:14.515174+0000","last_clean":"2023-10-21T16:22:14.515174+0000","last_became_active":"2023-10-21T16:22:14.511113+0000","last_became_peered":"2023-10-21T16:22:14.511113+0000","last_unstale":"2023-10-21T16:22:14.515174+0000","last_undegraded":"2023-10-21T16:22:14.515174+0000","last_fullsized":"2023-10-21T16:22:14.515174+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:50.431368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.507400+0000","last_change":"2023-10-21T16:22:13.462464+0000","last_active":"2023-10-21T16:22:15.507400+0000","last_peered":"2023-10-21T16:22:15.507400+0000","last_clean":"2023-10-21T16:22:15.507400+0000","last_became_active":"2023-10-21T16:22:13.461958+0000","last_became_peered":"2023-10-21T16:22:13.461958+0000","last_unstale":"2023-10-21T16:22:15.507400+0000","last_undegraded":"2023-10-21T16:22:15.507400+0000","last_fullsized":"2023-10-21T16:22:15.507400+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:57:16.345017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.508073+0000","last_change":"2023-10-21T16:22:14.498837+0000","last_active":"2023-10-21T16:22:14.508073+0000","last_peered":"2023-10-21T16:22:14.508073+0000","last_clean":"2023-10-21T16:22:14.508073+0000","last_became_active":"2023-10-21T16:22:14.498516+0000","last_became_peered":"2023-10-21T16:22:14.498516+0000","last_unstale":"2023-10-21T16:22:14.508073+0000","last_undegraded":"2023-10-21T16:22:14.508073+0000","last_fullsized":"2023-10-21T16:22:14.508073+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:57.204127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496628+0000","last_change":"2023-10-21T16:22:13.478469+0000","last_active":"2023-10-21T16:22:14.496628+0000","last_peered":"2023-10-21T16:22:14.496628+0000","last_clean":"2023-10-21T16:22:14.496628+0000","last_became_active":"2023-10-21T16:22:13.477075+0000","last_became_peered":"2023-10-21T16:22:13.477075+0000","last_unstale":"2023-10-21T16:22:14.496628+0000","last_undegraded":"2023-10-21T16:22:14.496628+0000","last_fullsized":"2023-10-21T16:22:14.496628+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T16:59:41.087335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.520648+0000","last_change":"2023-10-21T16:22:14.514824+0000","last_active":"2023-10-21T16:22:14.520648+0000","last_peered":"2023-10-21T16:22:14.520648+0000","last_clean":"2023-10-21T16:22:14.520648+0000","last_became_active":"2023-10-21T16:22:14.514530+0000","last_became_peered":"2023-10-21T16:22:14.514530+0000","last_unstale":"2023-10-21T16:22:14.520648+0000","last_undegraded":"2023-10-21T16:22:14.520648+0000","last_fullsized":"2023-10-21T16:22:14.520648+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:01.705022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-21T16:22:15.502728+0000","last_change":"2023-10-21T16:22:13.467401+0000","last_active":"2023-10-21T16:22:15.502728+0000","last_peered":"2023-10-21T16:22:15.502728+0000","last_clean":"2023-10-21T16:22:15.502728+0000","last_became_active":"2023-10-21T16:22:13.464603+0000","last_became_peered":"2023-10-21T16:22:13.464603+0000","last_unstale":"2023-10-21T16:22:15.502728+0000","last_undegraded":"2023-10-21T16:22:15.502728+0000","last_fullsized":"2023-10-21T16:22:15.502728+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:30.365938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.519200+0000","last_change":"2023-10-21T16:22:14.516556+0000","last_active":"2023-10-21T16:22:14.519200+0000","last_peered":"2023-10-21T16:22:14.519200+0000","last_clean":"2023-10-21T16:22:14.519200+0000","last_became_active":"2023-10-21T16:22:14.516225+0000","last_became_peered":"2023-10-21T16:22:14.516225+0000","last_unstale":"2023-10-21T16:22:14.519200+0000","last_undegraded":"2023-10-21T16:22:14.519200+0000","last_fullsized":"2023-10-21T16:22:14.519200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:42.648369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.502407+0000","last_change":"2023-10-21T16:22:13.482154+0000","last_active":"2023-10-21T16:22:14.502407+0000","last_peered":"2023-10-21T16:22:14.502407+0000","last_clean":"2023-10-21T16:22:14.502407+0000","last_became_active":"2023-10-21T16:22:13.481776+0000","last_became_peered":"2023-10-21T16:22:13.481776+0000","last_unstale":"2023-10-21T16:22:14.502407+0000","last_undegraded":"2023-10-21T16:22:14.502407+0000","last_fullsized":"2023-10-21T16:22:14.502407+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:46:37.106960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.517250+0000","last_change":"2023-10-21T16:22:14.511673+0000","last_active":"2023-10-21T16:22:14.517250+0000","last_peered":"2023-10-21T16:22:14.517250+0000","last_clean":"2023-10-21T16:22:14.517250+0000","last_became_active":"2023-10-21T16:22:14.511319+0000","last_became_peered":"2023-10-21T16:22:14.511319+0000","last_unstale":"2023-10-21T16:22:14.517250+0000","last_undegraded":"2023-10-21T16:22:14.517250+0000","last_fullsized":"2023-10-21T16:22:14.517250+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:29.227521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.496785+0000","last_change":"2023-10-21T16:22:13.479607+0000","last_active":"2023-10-21T16:22:14.496785+0000","last_peered":"2023-10-21T16:22:14.496785+0000","last_clean":"2023-10-21T16:22:14.496785+0000","last_became_active":"2023-10-21T16:22:13.479334+0000","last_became_peered":"2023-10-21T16:22:13.479334+0000","last_unstale":"2023-10-21T16:22:14.496785+0000","last_undegraded":"2023-10-21T16:22:14.496785+0000","last_fullsized":"2023-10-21T16:22:14.496785+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:21.663754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.522803+0000","last_change":"2023-10-21T16:22:14.518891+0000","last_active":"2023-10-21T16:22:14.522803+0000","last_peered":"2023-10-21T16:22:14.522803+0000","last_clean":"2023-10-21T16:22:14.522803+0000","last_became_active":"2023-10-21T16:22:14.518567+0000","last_became_peered":"2023-10-21T16:22:14.518567+0000","last_unstale":"2023-10-21T16:22:14.522803+0000","last_undegraded":"2023-10-21T16:22:14.522803+0000","last_fullsized":"2023-10-21T16:22:14.522803+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:24.620834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.499025+0000","last_change":"2023-10-21T16:22:13.477915+0000","last_active":"2023-10-21T16:22:14.499025+0000","last_peered":"2023-10-21T16:22:14.499025+0000","last_clean":"2023-10-21T16:22:14.499025+0000","last_became_active":"2023-10-21T16:22:13.477118+0000","last_became_peered":"2023-10-21T16:22:13.477118+0000","last_unstale":"2023-10-21T16:22:14.499025+0000","last_undegraded":"2023-10-21T16:22:14.499025+0000","last_fullsized":"2023-10-21T16:22:14.499025+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:09:27.176527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.513629+0000","last_change":"2023-10-21T16:22:14.511874+0000","last_active":"2023-10-21T16:22:14.513629+0000","last_peered":"2023-10-21T16:22:14.513629+0000","last_clean":"2023-10-21T16:22:14.513629+0000","last_became_active":"2023-10-21T16:22:14.511610+0000","last_became_peered":"2023-10-21T16:22:14.511610+0000","last_unstale":"2023-10-21T16:22:14.513629+0000","last_undegraded":"2023-10-21T16:22:14.513629+0000","last_fullsized":"2023-10-21T16:22:14.513629+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:13.434317+0000","last_clean_scrub_stamp":"2023-10-21T16:22:13.434317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:34:33.716145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-21T16:22:14.500929+0000","last_change":"2023-10-21T16:22:13.479695+0000","last_active":"2023-10-21T16:22:14.500929+0000","last_peered":"2023-10-21T16:22:14.500929+0000","last_clean":"2023-10-21T16:22:14.500929+0000","last_became_active":"2023-10-21T16:22:13.479414+0000","last_became_peered":"2023-10-21T16:22:13.479414+0000","last_unstale":"2023-10-21T16:22:14.500929+0000","last_undegraded":"2023-10-21T16:22:14.500929+0000","last_fullsized":"2023-10-21T16:22:14.500929+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:22:12.418381+0000","last_clean_scrub_stamp":"2023-10-21T16:22:12.418381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:49.120684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"150'4774","reported_seq":5350,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-21T16:22:13.467116+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T16:22:13.467116+0000","last_peered":"2023-10-21T16:22:13.467116+0000","last_clean":"2023-10-21T16:22:13.467116+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T16:22:13.467116+0000","last_undegraded":"2023-10-21T16:22:13.467116+0000","last_fullsized":"2023-10-21T16:22:13.467116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":407,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1360,"num_read_kb":2785,"num_write":407,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4774,"ondisk_log_size":4774,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575088,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47620,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324220,"statfs":{"total":96636764160,"available":96588001280,"internally_reserved":0,"allocated":2605056,"data_stored":1606922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.075,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.60399999999999998,"15min":0.51300000000000001},"max":{"1min":1.6919999999999999,"5min":1.758,"15min":1.8400000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.169},"min":{"1min":0.755,"5min":0.64200000000000002,"15min":0.44},"max":{"1min":1.4930000000000001,"5min":1.7270000000000001,"15min":2.448},"last":1.1040000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.149,"15min":1.161},"min":{"1min":0.76100000000000001,"5min":0.67800000000000005,"15min":0.66900000000000004},"max":{"1min":1.601,"5min":1.851,"15min":1.915},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1759999999999999,"15min":1.226},"min":{"1min":0.70299999999999996,"5min":0.51600000000000001,"15min":0.373},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.349}]},{"osd":2,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.2689999999999999,"15min":1.2989999999999999},"min":{"1min":0.79900000000000004,"5min":0.753,"15min":0.47199999999999998},"max":{"1min":1.704,"5min":1.782,"15min":2.085},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.85199999999999998,"5min":0.73599999999999999,"15min":0.51400000000000001},"max":{"1min":1.837,"5min":1.837,"15min":2.6070000000000002},"last":1.393}]},{"osd":3,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.206,"15min":1.248},"min":{"1min":0.878,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.5880000000000001,"5min":1.7370000000000001,"15min":2.2160000000000002},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.369,"15min":1.421},"min":{"1min":0.93400000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.4530000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3149999999999999,"15min":1.3500000000000001},"min":{"1min":1.032,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.794,"5min":2.0579999999999998,"15min":2.7269999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.4339999999999999,"15min":1.4379999999999999},"min":{"1min":1.3120000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.355},"last":1.7929999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.4299999999999999,"15min":1.403},"min":{"1min":1.0249999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4359999999999999,"15min":1.4399999999999999},"min":{"1min":0.93100000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.016,"5min":2.3380000000000001,"15min":3.2970000000000002},"last":1.6000000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.532,"15min":1.5489999999999999},"min":{"1min":1.129,"5min":1.028,"15min":0.88500000000000001},"max":{"1min":1.9890000000000001,"5min":2.1859999999999999,"15min":2.8119999999999998},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4419999999999999,"15min":1.4570000000000001},"min":{"1min":0.95599999999999996,"5min":0.873,"15min":0.60999999999999999},"max":{"1min":2.02,"5min":2.3050000000000002,"15min":2.8719999999999999},"last":1.8819999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575089,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":2002944,"data_stored":1015524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0860000000000001,"15min":1.1220000000000001},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.33000000000000002},"max":{"1min":1.704,"5min":1.869,"15min":1.869},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.125,"5min":1.1539999999999999,"15min":1.2050000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.61299999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.895},"last":0.96099999999999997}]},{"osd":1,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.1379999999999999,"15min":1.1899999999999999},"min":{"1min":0.80200000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.835,"5min":1.835,"15min":2.2309999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1870000000000001,"15min":1.2829999999999999},"min":{"1min":0.81499999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5309999999999999,"5min":1.97,"15min":2.2669999999999999},"last":1.3280000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2270000000000001,"15min":1.2509999999999999},"min":{"1min":0.90400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.726,"5min":1.895,"15min":2.5339999999999998},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1950000000000001,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.4330000000000001,"5min":1.821,"15min":1.821},"last":1.028}]},{"osd":3,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3129999999999999,"15min":1.3480000000000001},"min":{"1min":1.014,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8049999999999999,"5min":1.907,"15min":2.2000000000000002},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.4159999999999999},"min":{"1min":0.93000000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.845,"5min":1.9970000000000001,"15min":4.6210000000000004},"last":1.49}]},{"osd":4,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3540000000000001,"15min":1.339},"min":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6890000000000001,"5min":1.9379999999999999,"15min":2.1930000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4299999999999999,"15min":1.395},"min":{"1min":0.74199999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1299999999999999,"15min":2.2269999999999999},"last":1.0569999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.5,"15min":1.502},"min":{"1min":0.86399999999999999,"5min":0.73099999999999998,"15min":0.55200000000000005},"max":{"1min":1.8899999999999999,"5min":2.246,"15min":2.246},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3460000000000001,"15min":1.3520000000000001},"min":{"1min":0.77300000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.669,"5min":2.0150000000000001,"15min":2.4390000000000001},"last":1.1619999999999999}]},{"osd":7,"last update":"Sat Oct 21 16:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.476,"15min":1.5009999999999999},"min":{"1min":0.76300000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9159999999999999,"5min":2.2050000000000001,"15min":2.3570000000000002},"last":1.397},{"interface":"front","average":{"1min":1.492,"5min":1.6060000000000001,"15min":1.609},"min":{"1min":0.95399999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.931,"5min":2.319,"15min":2.319},"last":1.425}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51192,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320648,"statfs":{"total":96636764160,"available":96584343552,"internally_reserved":0,"allocated":2011136,"data_stored":1021566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.60999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.661,"5min":1.661,"15min":1.6919999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1759999999999999,"15min":1.1879999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.52400000000000002},"max":{"1min":2.2839999999999998,"5min":2.2839999999999998,"15min":2.2839999999999998},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1830000000000001,"15min":1.2410000000000001},"min":{"1min":0.71599999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.47,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.363},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.222},"min":{"1min":0.81200000000000006,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.579,"5min":4.2869999999999999,"15min":4.2869999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.69799999999999995,"5min":0.624,"15min":0.57199999999999995},"max":{"1min":1.5880000000000001,"5min":4.4359999999999999,"15min":4.4359999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.21,"15min":1.2290000000000001},"min":{"1min":0.80500000000000005,"5min":0.53100000000000003,"15min":0.46999999999999997},"max":{"1min":1.879,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.292}]},{"osd":3,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.29,"15min":1.3080000000000001},"min":{"1min":0.93700000000000006,"5min":0.78900000000000003,"15min":0.57599999999999996},"max":{"1min":1.6659999999999999,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":1.524},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3839999999999999,"15min":1.371},"min":{"1min":0.83999999999999997,"5min":0.77000000000000002,"15min":0.66800000000000004},"max":{"1min":1.6319999999999999,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":1.6870000000000001}]},{"osd":4,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2769999999999999,"15min":1.2150000000000001},"min":{"1min":0.77900000000000003,"5min":0.52600000000000002,"15min":0.48899999999999999},"max":{"1min":1.7829999999999999,"5min":4.359,"15min":4.359},"last":1.472},{"interface":"front","average":{"1min":1.286,"5min":1.399,"15min":1.343},"min":{"1min":0.85499999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.766,"5min":4.9100000000000001,"15min":4.9100000000000001},"last":1.627}]},{"osd":6,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.4650000000000001,"15min":1.4319999999999999},"min":{"1min":0.878,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.7490000000000001,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":1.8300000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.502,"15min":1.472},"min":{"1min":0.94699999999999995,"5min":0.79400000000000004,"15min":0.60099999999999998},"max":{"1min":2.0139999999999998,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.8620000000000001}]},{"osd":7,"last update":"Sat Oct 21 16:21:34 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.5169999999999999,"15min":1.492},"min":{"1min":1.01,"5min":0.68999999999999995,"15min":0.58299999999999996},"max":{"1min":1.6639999999999999,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.718},{"interface":"front","average":{"1min":1.476,"5min":1.599,"15min":1.538},"min":{"1min":0.83099999999999996,"5min":0.76700000000000002,"15min":0.60999999999999999},"max":{"1min":1.8340000000000001,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.7769999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575088,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2007040,"data_stored":1021544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.167,"15min":1.163},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6679999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.349},{"interface":"front","average":{"1min":1.242,"5min":1.222,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8360000000000001,"5min":5.7309999999999999,"15min":5.7309999999999999},"last":1.1970000000000001}]},{"osd":1,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.268},"min":{"1min":0.82399999999999995,"5min":0.73999999999999999,"15min":0.621},"max":{"1min":1.6279999999999999,"5min":5.8179999999999996,"15min":5.8179999999999996},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.2549999999999999,"15min":1.244},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.64500000000000002},"max":{"1min":1.3640000000000001,"5min":7.484,"15min":7.484},"last":1.548}]},{"osd":2,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.794,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3740000000000001,"15min":1.351},"min":{"1min":0.92400000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.0539999999999998,"5min":5.6059999999999999,"15min":5.6059999999999999},"last":1.3160000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.351,"15min":1.323},"min":{"1min":0.79500000000000004,"5min":0.64800000000000002,"15min":0.60399999999999998},"max":{"1min":1.6930000000000001,"5min":7.5880000000000001,"15min":7.5880000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.429,"15min":1.4019999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.0009999999999999,"5min":3.6280000000000001,"15min":3.6280000000000001},"last":1.407}]},{"osd":5,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3759999999999999,"15min":1.3180000000000001},"min":{"1min":0.97999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.2309999999999999,"5min":5.8929999999999998,"15min":5.8929999999999998},"last":1.085},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.4039999999999999,"15min":1.3520000000000001},"min":{"1min":0.88,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9490000000000001,"5min":7.6699999999999999,"15min":7.6699999999999999},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.486,"15min":1.4430000000000001},"min":{"1min":0.84399999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.9399999999999999,"5min":5.5099999999999998,"15min":5.5099999999999998},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.669,"5min":1.625,"15min":1.5880000000000001},"min":{"1min":1.4690000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.2919999999999998,"5min":6.1630000000000003,"15min":6.1630000000000003},"last":1.379}]},{"osd":7,"last update":"Sat Oct 21 16:21:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.611,"15min":1.518},"min":{"1min":1.04,"5min":0.62,"15min":0.62},"max":{"1min":2.1779999999999999,"5min":12.458,"15min":12.458},"last":1.147},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.615,"15min":1.575},"min":{"1min":0.88,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1349999999999998,"5min":3.9510000000000001,"15min":3.968},"last":1.226}]}]},{"osd":3,"up_from":13,"seq":55834575088,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41472,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330368,"statfs":{"total":96636764160,"available":96594296832,"internally_reserved":0,"allocated":2011136,"data_stored":1016534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.879,"15min":0.93899999999999995},"min":{"1min":0.442,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4159999999999999,"5min":1.946,"15min":1.946},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0649999999999999,"15min":1.1659999999999999},"min":{"1min":0.52300000000000002,"5min":0.48199999999999998,"15min":0.47799999999999998},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":2.04},"last":1.254}]},{"osd":1,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.90700000000000003,"15min":0.97399999999999998},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.589,"15min":2.173},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.115,"15min":1.228},"min":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8100000000000001},"last":1.3140000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.97299999999999998,"15min":1.0509999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.643,"5min":1.669,"15min":2.0049999999999999},"last":1.123},{"interface":"front","average":{"1min":1.077,"5min":1.135,"15min":1.202},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.46600000000000003},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":2.847},"last":1.0489999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.319,"15min":1.393},"min":{"1min":0.70399999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5780000000000001,"5min":1.96,"15min":2.2349999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3899999999999999},"min":{"1min":0.89900000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.5800000000000001,"5min":1.863,"15min":2.919},"last":1.3879999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.327,"15min":1.393},"min":{"1min":0.65900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.02},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4179999999999999,"15min":1.4870000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.75700000000000001},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.423},"last":1.6930000000000001}]},{"osd":6,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4139999999999999,"15min":1.478},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59299999999999997},"max":{"1min":1.794,"5min":1.8400000000000001,"15min":2.6179999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.419,"15min":1.508},"min":{"1min":0.75700000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.704,"5min":1.8999999999999999,"15min":2.2290000000000001},"last":1.766}]},{"osd":7,"last update":"Sat Oct 21 16:21:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4950000000000001,"15min":1.5740000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3759999999999999},"last":1.804},{"interface":"front","average":{"1min":1.504,"5min":1.52,"15min":1.587},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.7989999999999999},"last":1.724}]}]},{"osd":2,"up_from":13,"seq":55834575089,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2015232,"data_stored":1022160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.857,"5min":1.857,"15min":1.859},"last":0.372},{"interface":"front","average":{"1min":0.87,"5min":0.92900000000000005,"15min":0.96999999999999997},"min":{"1min":0.51500000000000001,"5min":0.375,"15min":0.28299999999999997},"max":{"1min":1.6259999999999999,"5min":1.905,"15min":2.0539999999999998},"last":0.499}]},{"osd":1,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0269999999999999,"15min":1.103},"min":{"1min":0.58199999999999996,"5min":0.52000000000000002,"15min":0.39500000000000002},"max":{"1min":1.5820000000000001,"5min":1.6890000000000001,"15min":1.9390000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.159,"15min":1.232},"min":{"1min":0.60599999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6919999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0840000000000001,"15min":1.1160000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.897,"5min":1.9279999999999999,"15min":2.0710000000000002},"last":0.623},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.276,"15min":1.347},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.895,"5min":2.069,"15min":2.069},"last":0.71299999999999997}]},{"osd":4,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.321,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.76600000000000001},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.319,"5min":1.3740000000000001,"15min":1.3700000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60199999999999998},"max":{"1min":1.95,"5min":2.278,"15min":2.6960000000000002},"last":1.0389999999999999}]},{"osd":5,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.327},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.571,"5min":2.024,"15min":2.2730000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.4630000000000001,"15min":1.506},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.024,"5min":2.1280000000000001,"15min":2.1349999999999998},"last":1}]},{"osd":6,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4650000000000001,"15min":1.478},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"max":{"1min":2.0600000000000001,"5min":2.444,"15min":2.6819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3979999999999999,"15min":1.423},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.78900000000000003},"max":{"1min":2.121,"5min":2.2669999999999999,"15min":2.552},"last":1.05}]},{"osd":7,"last update":"Sat Oct 21 16:21:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5049999999999999,"15min":1.5289999999999999},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.69999999999999996},"max":{"1min":1.8460000000000001,"5min":2.2130000000000001,"15min":2.4969999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.468,"15min":1.5049999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.3999999999999999},"last":0.69299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575088,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1998848,"data_stored":1014922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86899999999999999,"15min":0.90000000000000002},"min":{"1min":0.47699999999999998,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":1.1100000000000001,"5min":1.806,"15min":1.905},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.0469999999999999,"15min":1.0680000000000001},"min":{"1min":0.53500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4990000000000001,"5min":1.8440000000000001,"15min":1.8759999999999999},"last":1.1120000000000001}]},{"osd":2,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.95899999999999996,"15min":0.998},"min":{"1min":0.51500000000000001,"5min":0.436,"15min":0.28999999999999998},"max":{"1min":1.651,"5min":2.222,"15min":2.2639999999999998},"last":1.304},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.105,"15min":1.167},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":1.615,"5min":1.6910000000000001,"15min":1.974},"last":1.1910000000000001}]},{"osd":3,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.228,"15min":1.292},"min":{"1min":0.80500000000000005,"5min":0.54800000000000004,"15min":0.51600000000000001},"max":{"1min":1.5649999999999999,"5min":1.911,"15min":1.9510000000000001},"last":1.496},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.60399999999999998,"5min":0.45300000000000001,"15min":0.41699999999999998},"max":{"1min":1.526,"5min":1.6879999999999999,"15min":1.9159999999999999},"last":1.238}]},{"osd":4,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.238,"15min":1.268},"min":{"1min":0.82999999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.379,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.395},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.403,"15min":1.4239999999999999},"min":{"1min":1.024,"5min":0.68700000000000006,"15min":0.64100000000000001},"max":{"1min":1.9019999999999999,"5min":2.145,"15min":2.2719999999999998},"last":1.357}]},{"osd":5,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4570000000000001,"15min":1.482},"min":{"1min":1.2410000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.9419999999999999,"5min":1.9830000000000001,"15min":2.2410000000000001},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.4179999999999999,"15min":1.4590000000000001},"min":{"1min":1.2010000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.627,"5min":2.077,"15min":2.0880000000000001},"last":1.575}]},{"osd":6,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.482,"15min":1.49},"min":{"1min":0.85099999999999998,"5min":0.77500000000000002,"15min":0.68500000000000005},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.4079999999999999},"last":1.429},{"interface":"front","average":{"1min":1.367,"5min":1.417,"15min":1.4510000000000001},"min":{"1min":0.94499999999999995,"5min":0.63800000000000001,"15min":0.52600000000000002},"max":{"1min":1.746,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.46}]},{"osd":7,"last update":"Sat Oct 21 16:21:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5549999999999999,"15min":1.5940000000000001},"min":{"1min":1.1619999999999999,"5min":0.64600000000000002,"15min":0.59299999999999997},"max":{"1min":1.804,"5min":2.2530000000000001,"15min":2.3730000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.5649999999999999,"15min":1.593},"min":{"1min":0.89000000000000001,"5min":0.74399999999999999,"15min":0.68600000000000005},"max":{"1min":2.1110000000000002,"5min":2.1850000000000001,"15min":2.367},"last":1.7729999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575088,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42044,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329796,"statfs":{"total":96636764160,"available":96593711104,"internally_reserved":0,"allocated":2596864,"data_stored":1605276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.84799999999999998,"15min":0.90300000000000002},"min":{"1min":0.51200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.194,"5min":1.623,"15min":1.855},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.96799999999999997,"15min":1.046},"min":{"1min":0.45600000000000002,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.667},"last":0.82899999999999996}]},{"osd":2,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.018,"15min":1.0840000000000001},"min":{"1min":0.57299999999999995,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4370000000000001,"5min":1.679,"15min":1.7410000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.0409999999999999,"15min":1.1220000000000001},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.488,"5min":1.603,"15min":1.732},"last":1.643}]},{"osd":3,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97699999999999998,"15min":1.073},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.375},"max":{"1min":1.3680000000000001,"5min":1.702,"15min":2.1429999999999998},"last":1.169},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1699999999999999,"15min":1.264},"min":{"1min":0.51200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6040000000000001,"5min":1.8100000000000001,"15min":2.1619999999999999},"last":1.0289999999999999}]},{"osd":4,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3500000000000001,"15min":1.3959999999999999},"min":{"1min":0.81599999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.724,"5min":1.9019999999999999,"15min":2.0339999999999998},"last":1.393},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.431,"15min":1.5049999999999999},"min":{"1min":1.113,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8120000000000001,"5min":2.0550000000000002,"15min":2.2890000000000001},"last":1.5800000000000001}]},{"osd":5,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2989999999999999,"15min":1.3640000000000001},"min":{"1min":0.89900000000000002,"5min":0.64000000000000001,"15min":0.57699999999999996},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.137},"last":1.083},{"interface":"front","average":{"1min":1.383,"5min":1.3799999999999999,"15min":1.47},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":2.0699999999999998,"15min":2.4670000000000001},"last":1.365}]},{"osd":6,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3720000000000001,"15min":1.4550000000000001},"min":{"1min":0.76400000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.829,"5min":2.0880000000000001,"15min":2.5379999999999998},"last":1.23},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4630000000000001,"15min":1.5569999999999999},"min":{"1min":1.006,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.885,"5min":3.097,"15min":3.097},"last":1.415}]},{"osd":7,"last update":"Sat Oct 21 16:21:30 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.333,"15min":1.423},"min":{"1min":0.84799999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.9379999999999999,"5min":2.0289999999999999,"15min":2.2959999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.4239999999999999,"15min":1.538},"min":{"1min":0.82099999999999995,"5min":0.65100000000000002,"15min":0.64800000000000002},"max":{"1min":1.7310000000000001,"5min":2.0600000000000001,"15min":2.3540000000000001},"last":1.4610000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T16:22:19.536 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T16:22:19.536 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T16:22:19.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.815+0000 7f8793ebb640 1 Processor -- start 2023-10-21T16:22:19.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.816+0000 7f8793ebb640 1 -- start start 2023-10-21T16:22:19.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.817+0000 7f8793ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0cf620 0x7f878c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.817+0000 7f8793ebb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f878c0af840 0x7f878c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.817+0000 7f8793ebb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878c0cc820 con 0x7f878c15af00 2023-10-21T16:22:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.817+0000 7f8793ebb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878c0ceb90 con 0x7f878c0af840 2023-10-21T16:22:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.817+0000 7f8793ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0d18d0 con 0x7f878c0cf620 2023-10-21T16:22:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f878c0af840 0x7f878c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f878c0af840 0x7f878c0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51156/0 (socket says 172.21.15.53:51156) 2023-10-21T16:22:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 -- 172.21.15.53:0/246590685 learned_addr learned my addr 172.21.15.53:0/246590685 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8791456640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0cf620 0x7f878c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 -- 172.21.15.53:0/246590685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0cf620 msgr2=0x7f878c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0cf620 0x7f878c0adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 -- 172.21.15.53:0/246590685 >> v1:172.21.15.124:6789/0 conn(0x7f878c15af00 legacy=0x7f878c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.818+0000 7f8792458640 1 -- 172.21.15.53:0/246590685 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0ce8b0 con 0x7f878c0af840 2023-10-21T16:22:19.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.820+0000 7f8792458640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0af840 0x7f878c0ae100 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8784009e50 tx=0x7f878402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9b0e57312c609f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.820+0000 7f8790c55640 1 -- 172.21.15.53:0/246590685 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878402caf0 con 0x7f878c0af840 2023-10-21T16:22:19.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.820+0000 7f8790c55640 1 -- 172.21.15.53:0/246590685 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878402cc50 con 0x7f878c0af840 2023-10-21T16:22:19.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.820+0000 7f8790c55640 1 -- 172.21.15.53:0/246590685 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840343f0 con 0x7f878c0af840 2023-10-21T16:22:19.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 -- 172.21.15.53:0/246590685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0af840 msgr2=0x7f878c0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0af840 0x7f878c0ae100 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8784009e50 tx=0x7f878402bf50 comp rx=0 tx=0).stop 2023-10-21T16:22:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 -- 172.21.15.53:0/246590685 shutdown_connections 2023-10-21T16:22:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0cf620 0x7f878c0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 --2- 172.21.15.53:0/246590685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0af840 0x7f878c0ae100 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 -- 172.21.15.53:0/246590685 >> 172.21.15.53:0/246590685 conn(0x7f878c0d0930 msgr2=0x7f878c0d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 -- 172.21.15.53:0/246590685 shutdown_connections 2023-10-21T16:22:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.821+0000 7f8793ebb640 1 -- 172.21.15.53:0/246590685 wait complete. 2023-10-21T16:22:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.823+0000 7f8793ebb640 1 Processor -- start 2023-10-21T16:22:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.823+0000 7f8793ebb640 1 -- start start 2023-10-21T16:22:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0af840 0x7f878c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c15af00 0x7f878c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0c8190 con 0x7f878c0cf620 2023-10-21T16:22:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b5010 con 0x7f878c15af00 2023-10-21T16:22:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8793ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c0b52b0 con 0x7f878c0af840 2023-10-21T16:22:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8791456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8791456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51166/0 (socket says 172.21.15.53:51166) 2023-10-21T16:22:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.824+0000 7f8791456640 1 -- 172.21.15.53:0/3030624364 learned_addr learned my addr 172.21.15.53:0/3030624364 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8792458640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c15af00 0x7f878c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791c57640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0af840 0x7f878c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 -- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0af840 msgr2=0x7f878c0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0af840 0x7f878c0b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 -- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c15af00 msgr2=0x7f878c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c15af00 0x7f878c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 -- 172.21.15.53:0/3030624364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009b30 con 0x7f878c0cf620 2023-10-21T16:22:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f8791456640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e930 tx=0x7f877c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c009690 con 0x7f878c0cf620 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c00ca00 con 0x7f878c0cf620 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.825+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00cdb0 con 0x7f878c0cf620 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.826+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c0b5590 con 0x7f878c0cf620 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.826+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c0b5b50 con 0x7f878c0cf620 2023-10-21T16:22:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.827+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c04e160 con 0x7f878c0cf620 2023-10-21T16:22:19.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.828+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f877c013470 con 0x7f878c0cf620 2023-10-21T16:22:19.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.829+0000 7f87827fc640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 0x7f877006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:19.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.829+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f877c089c00 con 0x7f878c0cf620 2023-10-21T16:22:19.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.829+0000 7f8791c57640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 0x7f877006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:19.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.831+0000 7f8791c57640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 0x7f877006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8788000f60 tx=0x7f8788009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:19.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:19.833+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f877c090080 con 0x7f878c0cf620 2023-10-21T16:22:20.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.101+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f878c0cecf0 con 0x7f878c0cf620 2023-10-21T16:22:20.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.106+0000 7f87827fc640 1 -- 172.21.15.53:0/3030624364 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f877c0210b0 con 0x7f878c0cf620 2023-10-21T16:22:20.107 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T16:22:20.108 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T16:22:20.108 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T16:22:20.108 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T16:22:20.108 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T16:22:20.109 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T16:22:20.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.108+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 msgr2=0x7f877006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.108+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 0x7f877006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8788000f60 tx=0x7f8788009040 comp rx=0 tx=0).stop 2023-10-21T16:22:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.108+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 msgr2=0x7f878c1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.108+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e930 tx=0x7f877c00edf0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 shutdown_connections 2023-10-21T16:22:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f877006af90 0x7f877006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f878c15af00 0x7f878c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f878c0cf620 0x7f878c1571c0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 --2- 172.21.15.53:0/3030624364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f878c0af840 0x7f878c0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 >> 172.21.15.53:0/3030624364 conn(0x7f878c0d0930 msgr2=0x7f878c0ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 shutdown_connections 2023-10-21T16:22:20.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.109+0000 7f8793ebb640 1 -- 172.21.15.53:0/3030624364 wait complete. 2023-10-21T16:22:20.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T16:22:20.198 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-10-21T16:22:20.196+0000 7f5e0b155640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T16:22:20.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.351+0000 7fec2ef9a640 1 Processor -- start 2023-10-21T16:22:20.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.351+0000 7fec2ef9a640 1 -- start start 2023-10-21T16:22:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.352+0000 7fec2ef9a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec280d1640 0x7fec280ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.352+0000 7fec2ef9a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec280d07c0 0x7fec280b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.352+0000 7fec2ef9a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec280ceb00 con 0x7fec2815cdf0 2023-10-21T16:22:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.352+0000 7fec2ef9a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280ce610 con 0x7fec280d1640 2023-10-21T16:22:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.352+0000 7fec2ef9a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2815d1d0 con 0x7fec280d07c0 2023-10-21T16:22:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec280d1640 0x7fec280ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:20.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec280d1640 0x7fec280ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51174/0 (socket says 172.21.15.53:51174) 2023-10-21T16:22:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 -- 172.21.15.53:0/955892366 learned_addr learned my addr 172.21.15.53:0/955892366 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 -- 172.21.15.53:0/955892366 >> v2:172.21.15.124:3300/0 conn(0x7fec280d07c0 msgr2=0x7fec280b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 --2- 172.21.15.53:0/955892366 >> v2:172.21.15.124:3300/0 conn(0x7fec280d07c0 0x7fec280b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 -- 172.21.15.53:0/955892366 >> v1:172.21.15.124:6789/0 conn(0x7fec2815cdf0 legacy=0x7fec280d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:20.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.353+0000 7fec2d797640 1 -- 172.21.15.53:0/955892366 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280ce820 con 0x7fec280d1640 2023-10-21T16:22:20.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2d797640 1 --2- 172.21.15.53:0/955892366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d1640 0x7fec280ce0d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fec2808c080 tx=0x7fec1c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fba26db2db980d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2cf96640 1 -- 172.21.15.53:0/955892366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c031750 con 0x7fec280d1640 2023-10-21T16:22:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2cf96640 1 -- 172.21.15.53:0/955892366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c0318b0 con 0x7fec280d1640 2023-10-21T16:22:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2cf96640 1 -- 172.21.15.53:0/955892366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c031c40 con 0x7fec280d1640 2023-10-21T16:22:20.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2ef9a640 1 -- 172.21.15.53:0/955892366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d1640 msgr2=0x7fec280ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.355+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/955892366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d1640 0x7fec280ce0d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fec2808c080 tx=0x7fec1c02b7f0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 -- 172.21.15.53:0/955892366 shutdown_connections 2023-10-21T16:22:20.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/955892366 >> v2:172.21.15.124:3300/0 conn(0x7fec280d07c0 0x7fec280b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/955892366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d1640 0x7fec280ce0d0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 -- 172.21.15.53:0/955892366 >> 172.21.15.53:0/955892366 conn(0x7fec2809bcb0 msgr2=0x7fec280d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:20.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 -- 172.21.15.53:0/955892366 shutdown_connections 2023-10-21T16:22:20.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.356+0000 7fec2ef9a640 1 -- 172.21.15.53:0/955892366 wait complete. 2023-10-21T16:22:20.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.357+0000 7fec2ef9a640 1 Processor -- start 2023-10-21T16:22:20.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.358+0000 7fec2ef9a640 1 -- start start 2023-10-21T16:22:20.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.358+0000 7fec2ef9a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2ef9a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec280d1640 0x7fec2806c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2ef9a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 0x7fec2806c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2ef9a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec281e56e0 con 0x7fec280d07c0 2023-10-21T16:22:20.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2ef9a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec281e5950 con 0x7fec2815cdf0 2023-10-21T16:22:20.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2ef9a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec281e5c30 con 0x7fec280d1640 2023-10-21T16:22:20.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51178/0 (socket says 172.21.15.53:51178) 2023-10-21T16:22:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 -- 172.21.15.53:0/1652552627 learned_addr learned my addr 172.21.15.53:0/1652552627 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2e799640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 0x7fec2806c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 -- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec280d1640 msgr2=0x7fec2806c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2d797640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec280d1640 0x7fec2806c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec280d1640 0x7fec2806c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 -- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 msgr2=0x7fec2806c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 0x7fec2806c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2df98640 1 -- 172.21.15.53:0/1652552627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0094d0 con 0x7fec280d07c0 2023-10-21T16:22:20.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.359+0000 7fec2e799640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 0x7fec2806c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec2df98640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c7c0 tx=0x7fec1800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800aa50 con 0x7fec280d07c0 2023-10-21T16:22:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800abb0 con 0x7fec280d07c0 2023-10-21T16:22:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18012420 con 0x7fec280d07c0 2023-10-21T16:22:20.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec281e5f10 con 0x7fec280d07c0 2023-10-21T16:22:20.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.360+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2815c300 con 0x7fec280d07c0 2023-10-21T16:22:20.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.362+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec281e6430 con 0x7fec280d07c0 2023-10-21T16:22:20.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.362+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fec18012580 con 0x7fec280d07c0 2023-10-21T16:22:20.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.363+0000 7fec267fc640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 0x7fec1406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.363+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec18089890 con 0x7fec280d07c0 2023-10-21T16:22:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.364+0000 7fec2d797640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 0x7fec1406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.365+0000 7fec2d797640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 0x7fec1406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec1c035000 tx=0x7fec1c008030 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.368+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec281e6430 con 0x7fec280d07c0 2023-10-21T16:22:20.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.598+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec281e6430 con 0x7fec280d07c0 2023-10-21T16:22:20.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.599+0000 7fec267fc640 1 -- 172.21.15.53:0/1652552627 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fec281e6430 con 0x7fec280d07c0 2023-10-21T16:22:20.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:20.601 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T16:22:20.601 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T16:22:20.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 msgr2=0x7fec1406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 0x7fec1406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fec1c035000 tx=0x7fec1c008030 comp rx=0 tx=0).stop 2023-10-21T16:22:20.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 msgr2=0x7fec2815bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:20.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c7c0 tx=0x7fec1800cc80 comp rx=0 tx=0).stop 2023-10-21T16:22:20.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 shutdown_connections 2023-10-21T16:22:20.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7fec1406af90 0x7fec1406d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec2815cdf0 0x7fec2806c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec280d1640 0x7fec2806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 --2- 172.21.15.53:0/1652552627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec280d07c0 0x7fec2815bdc0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:20.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 >> 172.21.15.53:0/1652552627 conn(0x7fec2809bcb0 msgr2=0x7fec2815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:20.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.601+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 shutdown_connections 2023-10-21T16:22:20.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:20.602+0000 7fec2ef9a640 1 -- 172.21.15.53:0/1652552627 wait complete. 2023-10-21T16:22:21.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T16:22:13.932823+0000', 'modified': '2023-10-21T16:22:17.025961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9120}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9120': {'gid': 9120, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/3880141158', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 3880141158}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 3880141158}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T16:22:21.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T16:22:21.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-21T16:22:21.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.263+0000 7f819435e640 1 Processor -- start 2023-10-21T16:22:21.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.263+0000 7f819435e640 1 -- start start 2023-10-21T16:22:21.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f819435e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f818c0afbe0 0x7f818c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f819435e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c06dfc0 0x7f818c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f819435e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0b2620 con 0x7f818c071070 2023-10-21T16:22:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f819435e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0b2060 con 0x7f818c0afbe0 2023-10-21T16:22:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f819435e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b2340 con 0x7f818c06dfc0 2023-10-21T16:22:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f81920fa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f818c0afbe0 0x7f818c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f81928fb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f818c071070 0x7f818c0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45772/0 (socket says 172.21.15.53:45772) 2023-10-21T16:22:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.265+0000 7f81928fb640 1 -- 172.21.15.53:0/205354819 learned_addr learned my addr 172.21.15.53:0/205354819 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81918f9640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c06dfc0 0x7f818c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81920fa640 1 -- 172.21.15.53:0/205354819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c06dfc0 msgr2=0x7f818c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81920fa640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c06dfc0 0x7f818c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81920fa640 1 -- 172.21.15.53:0/205354819 >> v1:172.21.15.53:6789/0 conn(0x7f818c071070 legacy=0x7f818c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81920fa640 1 -- 172.21.15.53:0/205354819 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c070350 con 0x7f818c0afbe0 2023-10-21T16:22:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.266+0000 7f81910f8640 1 -- 172.21.15.53:0/205354819 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890833134 0 0) 0x7f818c0b2620 con 0x7f818c071070 2023-10-21T16:22:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.267+0000 7f81920fa640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c0afbe0 0x7f818c0b15b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f818802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265a997f72d7b1d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.267+0000 7f81910f8640 1 -- 172.21.15.53:0/205354819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818802d040 con 0x7f818c0afbe0 2023-10-21T16:22:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.267+0000 7f81910f8640 1 -- 172.21.15.53:0/205354819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188031810 con 0x7f818c0afbe0 2023-10-21T16:22:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.267+0000 7f81910f8640 1 -- 172.21.15.53:0/205354819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188031ba0 con 0x7f818c0afbe0 2023-10-21T16:22:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 -- 172.21.15.53:0/205354819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c0afbe0 msgr2=0x7f818c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c0afbe0 0x7f818c0b15b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f81880097a0 tx=0x7f818802b8b0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 -- 172.21.15.53:0/205354819 shutdown_connections 2023-10-21T16:22:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c06dfc0 0x7f818c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 --2- 172.21.15.53:0/205354819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c0afbe0 0x7f818c0b15b0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 -- 172.21.15.53:0/205354819 >> 172.21.15.53:0/205354819 conn(0x7f818c06a5d0 msgr2=0x7f818c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 -- 172.21.15.53:0/205354819 shutdown_connections 2023-10-21T16:22:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.268+0000 7f819435e640 1 -- 172.21.15.53:0/205354819 wait complete. 2023-10-21T16:22:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.270+0000 7f819435e640 1 Processor -- start 2023-10-21T16:22:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.270+0000 7f819435e640 1 -- start start 2023-10-21T16:22:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c071070 0x7f818c144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f818c0afbe0 0x7f818c15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c1567d0 con 0x7f818c06dfc0 2023-10-21T16:22:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0cd490 con 0x7f818c0afbe0 2023-10-21T16:22:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f819435e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c146d40 con 0x7f818c071070 2023-10-21T16:22:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f81920fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f81920fa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51208/0 (socket says 172.21.15.53:51208) 2023-10-21T16:22:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f81920fa640 1 -- 172.21.15.53:0/3611921366 learned_addr learned my addr 172.21.15.53:0/3611921366 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f81920fa640 1 -- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c071070 msgr2=0x7f818c144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.271+0000 7f81920fa640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c071070 0x7f818c144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f81920fa640 1 -- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f818c0afbe0 msgr2=0x7f818c15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f81920fa640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f818c0afbe0 0x7f818c15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f81920fa640 1 -- 172.21.15.53:0/3611921366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188009400 con 0x7f818c06dfc0 2023-10-21T16:22:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f81920fa640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f818802bec0 tx=0x7f81880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f8182ffd640 1 -- 172.21.15.53:0/3611921366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818803ad60 con 0x7f818c06dfc0 2023-10-21T16:22:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f8182ffd640 1 -- 172.21.15.53:0/3611921366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81880423f0 con 0x7f818c06dfc0 2023-10-21T16:22:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f8182ffd640 1 -- 172.21.15.53:0/3611921366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81880427a0 con 0x7f818c06dfc0 2023-10-21T16:22:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c145080 con 0x7f818c06dfc0 2023-10-21T16:22:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.272+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c1466f0 con 0x7f818c06dfc0 2023-10-21T16:22:21.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.275+0000 7f8182ffd640 1 -- 172.21.15.53:0/3611921366 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8188042900 con 0x7f818c06dfc0 2023-10-21T16:22:21.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.276+0000 7f8182ffd640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 0x7f8170071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.276+0000 7f8182ffd640 1 -- 172.21.15.53:0/3611921366 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81880abaa0 con 0x7f818c06dfc0 2023-10-21T16:22:21.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.276+0000 7f8196fcd740 1 Processor -- start 2023-10-21T16:22:21.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.276+0000 7f81918f9640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 0x7f8170071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.276+0000 7f8196fcd740 1 -- start start 2023-10-21T16:22:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426cfce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d2cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426c1d7a0 con 0x55f426cfd3c0 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426c6f740 con 0x55f426d2a360 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.277+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f426c6ce30 con 0x55f426d29f80 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81918f9640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 0x7f8170071890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c008040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51216/0 (socket says 172.21.15.53:51216) 2023-10-21T16:22:21.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 -- 172.21.15.53:0/3593033820 learned_addr learned my addr 172.21.15.53:0/3593033820 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81920fa640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426cfce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81918f9640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d2cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 -- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 msgr2=0x55f426cfce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426cfce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 -- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 msgr2=0x55f426d2cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d2cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 -- 172.21.15.53:0/3593033820 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f426c2c910 con 0x55f426cfd3c0 2023-10-21T16:22:21.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81920fa640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426cfce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:21.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.278+0000 7f81928fb640 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8184001d40 tx=0x7f818400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bce575fe0764a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f816ffff640 1 -- 172.21.15.53:0/3593033820 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400c8b0 con 0x55f426cfd3c0 2023-10-21T16:22:21.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f816ffff640 1 -- 172.21.15.53:0/3593033820 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400ca10 con 0x55f426cfd3c0 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f816ffff640 1 -- 172.21.15.53:0/3593033820 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400cd80 con 0x55f426cfd3c0 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 -- 172.21.15.53:0/3593033820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 msgr2=0x55f426d2d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8184001d40 tx=0x7f818400c0e0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 -- 172.21.15.53:0/3593033820 shutdown_connections 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426cfd3c0 0x55f426d2d250 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d2cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3593033820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426cfce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 -- 172.21.15.53:0/3593033820 >> 172.21.15.53:0/3593033820 conn(0x55f426cff190 msgr2=0x55f426d47d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 -- 172.21.15.53:0/3593033820 shutdown_connections 2023-10-21T16:22:21.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.279+0000 7f8196fcd740 1 -- 172.21.15.53:0/3593033820 wait complete. 2023-10-21T16:22:21.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.280+0000 7f8196fcd740 1 -- start start 2023-10-21T16:22:21.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.280+0000 7f8196fcd740 1 Processor -- start 2023-10-21T16:22:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426d49d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d2a360 0x55f426d2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426c68720 con 0x55f426d3e830 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426c06970 con 0x55f426d29f80 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f426c06610 con 0x55f426d2a360 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.281+0000 7f8196fcd740 10 client.0 _ll_register_callbacks cb 0x55f426c289d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51232/0 (socket says 172.21.15.53:51232) 2023-10-21T16:22:21.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 -- 172.21.15.53:0/3901304433 learned_addr learned my addr 172.21.15.53:0/3901304433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81918f9640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d2a360 0x55f426d2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81920fa640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426d49d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 -- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d2a360 msgr2=0x55f426d2f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d2a360 0x55f426d2f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 -- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 msgr2=0x55f426d49d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426d49d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 -- 172.21.15.53:0/3901304433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8184001940 con 0x55f426d3e830 2023-10-21T16:22:21.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81920fa640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426d49d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:21.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f81928fb640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8184010000 tx=0x7f8184001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba58ca754d61595 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.282+0000 7f816d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184007560 con 0x55f426d3e830 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81840076c0 con 0x55f426d3e830 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f426c8f160 con 0x55f426d3e830 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8184007880 con 0x55f426d3e830 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 10 client.9264 fetch_fsmap learned FSMap version 78 2023-10-21T16:22:21.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55f426c66da0 con 0x55f426d3e830 2023-10-21T16:22:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f426c8f160 con 0x55f426d3e830 2023-10-21T16:22:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8184020480 con 0x55f426d3e830 2023-10-21T16:22:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 10 client.9264 fetch_fsmap finished waiting for FSMap version 78 2023-10-21T16:22:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 10 client.9264 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T16:22:21.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 20 client.9264 populate_metadata read hostname 'smithi053' 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426d18480 0x55f426d18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 4 client.9264 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.283+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 --> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f426c3acc0 con 0x55f426d18480 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.284+0000 7f81920fa640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426d18480 0x55f426d18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.284+0000 7f81920fa640 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426d18480 0x55f426d18cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670493bd9eee5f31 server_cookie=63dbe15be1e2a59f in_seq=0 out_seq=0 2023-10-21T16:22:21.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.284+0000 7f816d7fa640 10 client.9264 ms_handle_connect on v2:172.21.15.53:6833/3880141158 2023-10-21T16:22:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f816d7fa640 1 -- 172.21.15.53:0/3901304433 <== mds.0 v2:172.21.15.53:6833/3880141158 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f426c3acc0 con 0x55f426d18480 2023-10-21T16:22:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f816d7fa640 10 client.9264 handle_command_reply: tid=0 2023-10-21T16:22:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 client.9264 shutdown 2023-10-21T16:22:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 msgr2=0x55f426d15ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8184010000 tx=0x7f8184001470 comp rx=0 tx=0).stop 2023-10-21T16:22:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 shutdown_connections 2023-10-21T16:22:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426d18480 0x55f426d18cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d15ff0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d2a360 0x55f426d2f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 --2- 172.21.15.53:0/3901304433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426d49d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f816d7fa640 0 client.9264 ms_handle_reset on v2:172.21.15.53:6833/3880141158 2023-10-21T16:22:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.285+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 >> 172.21.15.53:0/3901304433 conn(0x55f426cfd3c0 msgr2=0x55f426d00f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.286+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 shutdown_connections 2023-10-21T16:22:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.286+0000 7f8196fcd740 1 -- 172.21.15.53:0/3901304433 wait complete. 2023-10-21T16:22:21.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.286+0000 7f8196fcd740 20 client.9264 trim_cache size 0 max 16384 2023-10-21T16:22:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.302+0000 7f8196fcd740 1 Processor -- start 2023-10-21T16:22:21.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 -- start start 2023-10-21T16:22:21.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426d4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d5a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4269ccf20 con 0x55f426d3e830 2023-10-21T16:22:21.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426cfd930 con 0x55f426d2a360 2023-10-21T16:22:21.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.303+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f426c0e520 con 0x55f426d29f80 2023-10-21T16:22:21.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51244/0 (socket says 172.21.15.53:51244) 2023-10-21T16:22:21.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 -- 172.21.15.53:0/1645440363 learned_addr learned my addr 172.21.15.53:0/1645440363 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81920fa640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426d4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81918f9640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d5a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 -- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 msgr2=0x55f426d4a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426d4a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 -- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 msgr2=0x55f426d5a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d5a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81928fb640 1 -- 172.21.15.53:0/1645440363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f426c05b80 con 0x55f426d3e830 2023-10-21T16:22:21.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.304+0000 7f81920fa640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426d4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:21.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f81928fb640 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8184001d40 tx=0x7f818401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaca353957c47ef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f8180e01640 1 -- 172.21.15.53:0/1645440363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400c8b0 con 0x55f426d3e830 2023-10-21T16:22:21.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f8180e01640 1 -- 172.21.15.53:0/1645440363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400ce90 con 0x55f426d3e830 2023-10-21T16:22:21.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f8180e01640 1 -- 172.21.15.53:0/1645440363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840075a0 con 0x55f426d3e830 2023-10-21T16:22:21.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f8196fcd740 1 -- 172.21.15.53:0/1645440363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 msgr2=0x55f426d5ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.305+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8184001d40 tx=0x7f818401e000 comp rx=0 tx=0).stop 2023-10-21T16:22:21.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 -- 172.21.15.53:0/1645440363 shutdown_connections 2023-10-21T16:22:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d3e830 0x55f426d5ac10 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d2a360 0x55f426d5a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1645440363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d29f80 0x55f426d4a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 -- 172.21.15.53:0/1645440363 >> 172.21.15.53:0/1645440363 conn(0x55f426cfd3c0 msgr2=0x55f426d6a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 -- 172.21.15.53:0/1645440363 shutdown_connections 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 -- 172.21.15.53:0/1645440363 wait complete. 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.306+0000 7f8196fcd740 1 -- start start 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.307+0000 7f8196fcd740 1 Processor -- start 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.307+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426da6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d3e830 0x55f426d26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426db4810 con 0x55f426d2a360 2023-10-21T16:22:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f426d15bb0 con 0x55f426d29f80 2023-10-21T16:22:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f426d15df0 con 0x55f426d3e830 2023-10-21T16:22:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f8196fcd740 10 client.0 _ll_register_callbacks cb 0x7f8184001b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T16:22:21.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51248/0 (socket says 172.21.15.53:51248) 2023-10-21T16:22:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 -- 172.21.15.53:0/1845334467 learned_addr learned my addr 172.21.15.53:0/1845334467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81928fb640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d3e830 0x55f426d26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 -- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d3e830 msgr2=0x55f426d26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d3e830 0x55f426d26e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 -- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 msgr2=0x55f426da6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426da6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.308+0000 7f81918f9640 1 -- 172.21.15.53:0/1845334467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f426c66da0 con 0x55f426d2a360 2023-10-21T16:22:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f81918f9640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f818c0c20e0 tx=0x7f817c006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5927ddff2c5d0c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f816d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T16:22:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c014b10 con 0x55f426d2a360 2023-10-21T16:22:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f817c014c70 con 0x55f426d2a360 2023-10-21T16:22:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f426d27bb0 con 0x55f426d2a360 2023-10-21T16:22:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f817c014e30 con 0x55f426d2a360 2023-10-21T16:22:21.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f8196fcd740 10 client.9270 fetch_fsmap learned FSMap version 78 2023-10-21T16:22:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.309+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55f426c2b170 con 0x55f426d2a360 2023-10-21T16:22:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f426d27bb0 con 0x55f426d2a360 2023-10-21T16:22:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f817c021400 con 0x55f426d2a360 2023-10-21T16:22:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 10 client.9270 fetch_fsmap finished waiting for FSMap version 78 2023-10-21T16:22:21.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 10 client.9270 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T16:22:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 20 client.9270 populate_metadata read hostname 'smithi053' 2023-10-21T16:22:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426db3c10 0x55f426db3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 4 client.9270 mds_command: new command op to 9120 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T16:22:21.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 --> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f818400ca10 con 0x55f426db3c10 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f81920fa640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426db3c10 0x55f426db3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f81920fa640 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426db3c10 0x55f426db3ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523d8035be640c91 server_cookie=867364e569fef04c in_seq=0 out_seq=0 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.310+0000 7f816d7fa640 10 client.9270 ms_handle_connect on v2:172.21.15.53:6833/3880141158 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f816d7fa640 1 -- 172.21.15.53:0/1845334467 <== mds.0 v2:172.21.15.53:6833/3880141158 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f818400ca10 con 0x55f426db3c10 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f816d7fa640 10 client.9270 handle_command_reply: tid=0 2023-10-21T16:22:21.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 client.9270 shutdown 2023-10-21T16:22:21.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 msgr2=0x55f426d268d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f818c0c20e0 tx=0x7f817c006e10 comp rx=0 tx=0).stop 2023-10-21T16:22:21.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 shutdown_connections 2023-10-21T16:22:21.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:6833/3880141158,v1:172.21.15.53:6835/3880141158] conn(0x55f426db3c10 0x55f426db3ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f426d3e830 0x55f426d26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f426d2a360 0x55f426d268d0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 --2- 172.21.15.53:0/1845334467 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f426d29f80 0x55f426da6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f816d7fa640 0 client.9270 ms_handle_reset on v2:172.21.15.53:6833/3880141158 2023-10-21T16:22:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 >> 172.21.15.53:0/1845334467 conn(0x55f426cfd3c0 msgr2=0x55f426cfbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 shutdown_connections 2023-10-21T16:22:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 1 -- 172.21.15.53:0/1845334467 wait complete. 2023-10-21T16:22:21.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.311+0000 7f8196fcd740 20 client.9270 trim_cache size 0 max 16384 2023-10-21T16:22:21.326 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T16:22:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T16:22:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T16:22:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 9120, 2023-10-21T16:22:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T16:22:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T16:22:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8183690319999997, 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 78, 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 161, 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 161, 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.865684248999999 2023-10-21T16:22:21.328 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T16:22:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 msgr2=0x7f8170071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 0x7f8170071890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f817c006fd0 tx=0x7f817c008040 comp rx=0 tx=0).stop 2023-10-21T16:22:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 msgr2=0x7f818c0ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f818802bec0 tx=0x7f81880046c0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 shutdown_connections 2023-10-21T16:22:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f817006f3d0 0x7f8170071890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f818c0afbe0 0x7f818c15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f818c071070 0x7f818c144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 --2- 172.21.15.53:0/3611921366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f818c06dfc0 0x7f818c0ccf50 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 >> 172.21.15.53:0/3611921366 conn(0x7f818c06a5d0 msgr2=0x7f818c0c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 shutdown_connections 2023-10-21T16:22:21.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.312+0000 7f819435e640 1 -- 172.21.15.53:0/3611921366 wait complete. 2023-10-21T16:22:21.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T16:22:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.549+0000 7f3457452640 1 Processor -- start 2023-10-21T16:22:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.550+0000 7f3457452640 1 -- start start 2023-10-21T16:22:21.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3457452640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34500ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3457452640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345006cfd0 con 0x7f345015a970 2023-10-21T16:22:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3457452640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345006d870 con 0x7f34500cbb60 2023-10-21T16:22:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3457452640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345015ad50 con 0x7f345006dd00 2023-10-21T16:22:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3456450640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f345015a970 0x7f34500cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45784/0 (socket says 172.21.15.53:45784) 2023-10-21T16:22:21.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.551+0000 7f3456450640 1 -- 172.21.15.53:0/3578154793 learned_addr learned my addr 172.21.15.53:0/3578154793 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433953889 0 0) 0x7f345006cfd0 con 0x7f345015a970 2023-10-21T16:22:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f3456c51640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34500ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3438003520 con 0x7f345015a970 2023-10-21T16:22:21.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 184328069 0 0) 0x7f3438003520 con 0x7f345015a970 2023-10-21T16:22:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 msgr2=0x7f34500ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34500ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 >> v1:172.21.15.124:6789/0 conn(0x7f34500cbb60 legacy=0x7f34500c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500cac20 con 0x7f345015a970 2023-10-21T16:22:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f34400019f0 con 0x7f345015a970 2023-10-21T16:22:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 >> v1:172.21.15.53:6789/0 conn(0x7f345015a970 legacy=0x7f34500cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f3438003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3438004d00 0x7f3438005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345015ad50 con 0x7f34380039a0 2023-10-21T16:22:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343800bb90 con 0x7f3438004d00 2023-10-21T16:22:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.552+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343800bd50 con 0x7f345006dd00 2023-10-21T16:22:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f3438003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 -- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 msgr2=0x7f34380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3455c4f640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456c51640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3438004d00 0x7f3438005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 -- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3438004d00 msgr2=0x7f3438005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3438004d00 0x7f3438005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3456450640 1 -- 172.21.15.53:0/3578154793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438003520 con 0x7f34380039a0 2023-10-21T16:22:21.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.553+0000 7f3455c4f640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345006dd00 0x7f34380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:21.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.554+0000 7f3456450640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f3438003dc0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f344000a760 tx=0x7f34380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8088c414a3e17ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440034630 con 0x7f34380039a0 2023-10-21T16:22:21.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344002f070 con 0x7f34380039a0 2023-10-21T16:22:21.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f345544e640 1 -- 172.21.15.53:0/3578154793 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400349f0 con 0x7f34380039a0 2023-10-21T16:22:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 -- 172.21.15.53:0/3578154793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 msgr2=0x7f3438003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 --2- 172.21.15.53:0/3578154793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f3438003dc0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f344000a760 tx=0x7f34380057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3456c51640 1 -- 172.21.15.53:0/3578154793 reap_dead start 2023-10-21T16:22:21.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 -- 172.21.15.53:0/3578154793 shutdown_connections 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 -- 172.21.15.53:0/3578154793 >> 172.21.15.53:0/3578154793 conn(0x7f34500ce9d0 msgr2=0x7f34500cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 -- 172.21.15.53:0/3578154793 shutdown_connections 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.555+0000 7f3457452640 1 -- 172.21.15.53:0/3578154793 wait complete. 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.557+0000 7f3457452640 1 Processor -- start 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.557+0000 7f3457452640 1 -- start start 2023-10-21T16:22:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ae90 0x7f34501f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34501f3e50 0x7f34500ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34501f3480 con 0x7f34380039a0 2023-10-21T16:22:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345015b920 con 0x7f345015ae90 2023-10-21T16:22:21.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.558+0000 7f3457452640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345015bea0 con 0x7f34501f3e50 2023-10-21T16:22:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3455c4f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ae90 0x7f34501f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51258/0 (socket says 172.21.15.53:51258) 2023-10-21T16:22:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 -- 172.21.15.53:0/2065130790 learned_addr learned my addr 172.21.15.53:0/2065130790 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:21.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456c51640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34501f3e50 0x7f34500ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 -- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34501f3e50 msgr2=0x7f34500ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34501f3e50 0x7f34500ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 -- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ae90 msgr2=0x7f34501f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ae90 0x7f34501f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.559+0000 7f3456450640 1 -- 172.21.15.53:0/2065130790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345015bbc0 con 0x7f34380039a0 2023-10-21T16:22:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f3456450640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f344000a760 tx=0x7f3440001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440043070 con 0x7f34380039a0 2023-10-21T16:22:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3440034430 con 0x7f34380039a0 2023-10-21T16:22:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400347a0 con 0x7f34380039a0 2023-10-21T16:22:21.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345015b370 con 0x7f34380039a0 2023-10-21T16:22:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.560+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34500ab700 con 0x7f34380039a0 2023-10-21T16:22:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.561+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34501f3270 con 0x7f34380039a0 2023-10-21T16:22:21.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.562+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3440031030 con 0x7f34380039a0 2023-10-21T16:22:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.563+0000 7f344effd640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 0x7f343c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.563+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34400ab6a0 con 0x7f34380039a0 2023-10-21T16:22:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.564+0000 7f3455c4f640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 0x7f343c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:21.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.565+0000 7f3455c4f640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 0x7f343c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f345015b710 tx=0x7f3444009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.567+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3440076300 con 0x7f34380039a0 2023-10-21T16:22:21.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.802+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34501f3270 con 0x7f34380039a0 2023-10-21T16:22:21.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.803+0000 7f344effd640 1 -- 172.21.15.53:0/2065130790 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3440079920 con 0x7f34380039a0 2023-10-21T16:22:21.805 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T16:22:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 msgr2=0x7f343c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 0x7f343c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f345015b710 tx=0x7f3444009040 comp rx=0 tx=0).stop 2023-10-21T16:22:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 msgr2=0x7f34501598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f344000a760 tx=0x7f3440001cb0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 shutdown_connections 2023-10-21T16:22:21.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f343c06afc0 0x7f343c06d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34501f3e50 0x7f34500ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345015ae90 0x7f34501f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 --2- 172.21.15.53:0/2065130790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f34380039a0 0x7f34501598f0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:21.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 >> 172.21.15.53:0/2065130790 conn(0x7f34500ce9d0 msgr2=0x7f3450069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:21.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 shutdown_connections 2023-10-21T16:22:21.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:21.806+0000 7f3457452640 1 -- 172.21.15.53:0/2065130790 wait complete. 2023-10-21T16:22:21.826 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T16:22:21.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T16:22:21.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T16:22:21.828 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T16:22:21.828 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T16:22:21.828 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T16:22:21.829 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T16:22:21.829 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T16:22:21.829 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:22:21.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:21.830 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T16:22:21.891 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T16:22:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T16:22:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T16:22:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T16:22:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.893 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:22:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T16:22:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T16:22:21.894 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:22:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T16:22:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 41151sec preferred_lft 41151sec 2023-10-21T16:22:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T16:22:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.896 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T16:22:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T16:22:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T16:22:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.898 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:22:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T16:22:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T16:22:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.898 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T16:22:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T16:22:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T16:22:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T16:22:21.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:21.900 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T16:22:21.998 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T16:22:22.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.003 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T16:22:22.061 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T16:22:22.062 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T16:22:22.062 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T16:22:22.062 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T16:22:22.062 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T16:22:22.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.063 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T16:22:22.064 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T16:22:22.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T16:22:22.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T16:22:22.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T16:22:22.066 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T16:22:22.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.260 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T16:22:22.313 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T16:22:22.314 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T16:22:22.315 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:22:22.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T16:22:22.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:22:22.420 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T16:22:22.482 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T16:22:22.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T16:22:22.483 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:22:22.544 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T16:22:22.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:22:22.694 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:22:22.694 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:22:22.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:22:22.723 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:22:22.727 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T16:22:22.727 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T16:22:22.727 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T16:22:22.728 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T16:22:22.728 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T16:22:22.729 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T16:22:22.729 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T16:22:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T16:22:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T16:22:22.730 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T16:22:22.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T16:22:22.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T16:22:22.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:22.855 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T16:22:22.975 INFO:teuthology.orchestra.run.smithi053.stdout:client9279 2023-10-21T16:22:22.975 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T16:22:22.976 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T16:22:23.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T16:22:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.267+0000 7f907d735640 1 Processor -- start 2023-10-21T16:22:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.268+0000 7f907d735640 1 -- start start 2023-10-21T16:22:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f907d735640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 0x7f907806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f907d735640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90780b2180 0x7f90780b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f907d735640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907806c1f0 con 0x7f907815ccd0 2023-10-21T16:22:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f907d735640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90780b2b50 con 0x7f90780b2180 2023-10-21T16:22:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f907d735640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780ce730 con 0x7f90780ce2d0 2023-10-21T16:22:23.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f9077fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f907815ccd0 0x7f9078160da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45792/0 (socket says 172.21.15.53:45792) 2023-10-21T16:22:23.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.269+0000 7f9077fff640 1 -- 172.21.15.53:0/2724786009 learned_addr learned my addr 172.21.15.53:0/2724786009 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:23.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f906ffff640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 0x7f907806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f907cf34640 1 --2- 172.21.15.53:0/2724786009 >> v2:172.21.15.124:3300/0 conn(0x7f90780b2180 0x7f90780b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352409629 0 0) 0x7f907806c1f0 con 0x7f907815ccd0 2023-10-21T16:22:23.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9064003520 con 0x7f907815ccd0 2023-10-21T16:22:23.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896559038 0 0) 0x7f9064003520 con 0x7f907815ccd0 2023-10-21T16:22:23.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 msgr2=0x7f907806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:23.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 0x7f907806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 >> v2:172.21.15.124:3300/0 conn(0x7f90780b2180 msgr2=0x7f90780b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:23.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 --2- 172.21.15.53:0/2724786009 >> v2:172.21.15.124:3300/0 conn(0x7f90780b2180 0x7f90780b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780ce9b0 con 0x7f907815ccd0 2023-10-21T16:22:23.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.270+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9068001670 con 0x7f907815ccd0 2023-10-21T16:22:23.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 >> v1:172.21.15.53:6789/0 conn(0x7f907815ccd0 legacy=0x7f9078160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 0x7f9064003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9064004d00 0x7f9064005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780b2b50 con 0x7f9064003f40 2023-10-21T16:22:23.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780ce730 con 0x7f9064004d00 2023-10-21T16:22:23.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906400b980 con 0x7f90780ce2d0 2023-10-21T16:22:23.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f907cf34640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9064004d00 0x7f9064005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 -- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 msgr2=0x7f9064003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:23.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780ce2d0 0x7f9064003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 -- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9064004d00 msgr2=0x7f9064005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:23.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9064004d00 0x7f9064005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.271+0000 7f906ffff640 1 -- 172.21.15.53:0/2724786009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064003520 con 0x7f9064003f40 2023-10-21T16:22:23.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f906ffff640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90640047c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9060009af0 tx=0x7f90640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3978258717efac45 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906002c710 con 0x7f9064003f40 2023-10-21T16:22:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906002c870 con 0x7f9064003f40 2023-10-21T16:22:23.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f90777fe640 1 -- 172.21.15.53:0/2724786009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906002cc00 con 0x7f9064003f40 2023-10-21T16:22:23.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f907d735640 1 -- 172.21.15.53:0/2724786009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 msgr2=0x7f90640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f907d735640 1 --2- 172.21.15.53:0/2724786009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90640047c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9060009af0 tx=0x7f90640057f0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.273+0000 7f907cf34640 1 -- 172.21.15.53:0/2724786009 reap_dead start 2023-10-21T16:22:23.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.274+0000 7f907d735640 1 -- 172.21.15.53:0/2724786009 shutdown_connections 2023-10-21T16:22:23.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.274+0000 7f907d735640 1 -- 172.21.15.53:0/2724786009 >> 172.21.15.53:0/2724786009 conn(0x7f90780cd180 msgr2=0x7f90780cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:23.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.274+0000 7f907d735640 1 -- 172.21.15.53:0/2724786009 shutdown_connections 2023-10-21T16:22:23.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.274+0000 7f907d735640 1 -- 172.21.15.53:0/2724786009 wait complete. 2023-10-21T16:22:23.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.276+0000 7f907d735640 1 Processor -- start 2023-10-21T16:22:23.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.276+0000 7f907d735640 1 -- start start 2023-10-21T16:22:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 0x7f90780b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780b3320 0x7f90780b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780c7970 con 0x7f9064003f40 2023-10-21T16:22:23.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90780c7c50 con 0x7f907815f990 2023-10-21T16:22:23.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907d735640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90780c7f30 con 0x7f90780b3320 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f9077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f9077fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51272/0 (socket says 172.21.15.53:51272) 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f9077fff640 1 -- 172.21.15.53:0/3670762038 learned_addr learned my addr 172.21.15.53:0/3670762038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f906ffff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 0x7f90780b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f907cf34640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780b3320 0x7f90780b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.277+0000 7f9077fff640 1 -- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780b3320 msgr2=0x7f90780b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f9077fff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780b3320 0x7f90780b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f9077fff640 1 -- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 msgr2=0x7f90780b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f9077fff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 0x7f90780b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f9077fff640 1 -- 172.21.15.53:0/3670762038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780c8210 con 0x7f9064003f40 2023-10-21T16:22:23.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f906ffff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 0x7f90780b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:23.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f9077fff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f906800c680 tx=0x7f906800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680072d0 con 0x7f9064003f40 2023-10-21T16:22:23.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068007450 con 0x7f9064003f40 2023-10-21T16:22:23.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680047e0 con 0x7f9064003f40 2023-10-21T16:22:23.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780b3d10 con 0x7f9064003f40 2023-10-21T16:22:23.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.278+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9078158ed0 con 0x7f9064003f40 2023-10-21T16:22:23.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.280+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9078074150 con 0x7f9064003f40 2023-10-21T16:22:23.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.281+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9068004960 con 0x7f9064003f40 2023-10-21T16:22:23.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.281+0000 7f90757fa640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 0x7f905806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.282+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9068088c70 con 0x7f9064003f40 2023-10-21T16:22:23.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.282+0000 7f906ffff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 0x7f905806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.283+0000 7f906ffff640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 0x7f905806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9060036000 tx=0x7f9060030000 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.286+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9068053a80 con 0x7f9064003f40 2023-10-21T16:22:23.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.511+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 --> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f90780cef70 con 0x7f905806afc0 2023-10-21T16:22:23.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.515+0000 7f90757fa640 1 -- 172.21.15.53:0/3670762038 <== mgr.36626 v2:172.21.15.124:6832/870299393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9060004040 con 0x7f905806afc0 2023-10-21T16:22:23.516 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T16:22:23.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T16:22:23.517 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T16:22:23.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T16:22:23.517 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T16:22:23.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 msgr2=0x7f905806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 0x7f905806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9060036000 tx=0x7f9060030000 comp rx=0 tx=0).stop 2023-10-21T16:22:23.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 msgr2=0x7f90780b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f906800c680 tx=0x7f906800cb40 comp rx=0 tx=0).stop 2023-10-21T16:22:23.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 shutdown_connections 2023-10-21T16:22:23.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f905806afc0 0x7f905806d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90780b3320 0x7f90780b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f907815f990 0x7f90780b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 --2- 172.21.15.53:0/3670762038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9064003f40 0x7f90780b0020 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 >> 172.21.15.53:0/3670762038 conn(0x7f90780cd180 msgr2=0x7f90780d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:23.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 shutdown_connections 2023-10-21T16:22:23.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.518+0000 7f907d735640 1 -- 172.21.15.53:0/3670762038 wait complete. 2023-10-21T16:22:23.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T16:22:23.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.766+0000 7f5c3ada5640 1 Processor -- start 2023-10-21T16:22:23.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.766+0000 7f5c3ada5640 1 -- start start 2023-10-21T16:22:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.767+0000 7f5c3ada5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c340c18d0 0x7f5c340c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.767+0000 7f5c3ada5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c34164d10 0x7f5c340ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.767+0000 7f5c3ada5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c34161590 con 0x7f5c340aff70 2023-10-21T16:22:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.767+0000 7f5c3ada5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340cc570 con 0x7f5c340c18d0 2023-10-21T16:22:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.767+0000 7f5c3ada5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340c94b0 con 0x7f5c34164d10 2023-10-21T16:22:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c39da3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5c340aff70 0x7f5c340b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45804/0 (socket says 172.21.15.53:45804) 2023-10-21T16:22:23.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c39da3640 1 -- 172.21.15.53:0/2674829491 learned_addr learned my addr 172.21.15.53:0/2674829491 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:23.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c395a2640 1 --2- 172.21.15.53:0/2674829491 >> v2:172.21.15.53:3300/0 conn(0x7f5c340c18d0 0x7f5c340c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c3a5a4640 1 --2- 172.21.15.53:0/2674829491 >> v2:172.21.15.124:3300/0 conn(0x7f5c34164d10 0x7f5c340ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c38da1640 1 -- 172.21.15.53:0/2674829491 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210060904 0 0) 0x7f5c34161590 con 0x7f5c340aff70 2023-10-21T16:22:23.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c38da1640 1 -- 172.21.15.53:0/2674829491 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c20003520 con 0x7f5c340aff70 2023-10-21T16:22:23.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c395a2640 1 -- 172.21.15.53:0/2674829491 >> v2:172.21.15.124:3300/0 conn(0x7f5c34164d10 msgr2=0x7f5c340ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:23.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c395a2640 1 --2- 172.21.15.53:0/2674829491 >> v2:172.21.15.124:3300/0 conn(0x7f5c34164d10 0x7f5c340ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c395a2640 1 -- 172.21.15.53:0/2674829491 >> v1:172.21.15.53:6789/0 conn(0x7f5c340aff70 legacy=0x7f5c340b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.768+0000 7f5c395a2640 1 -- 172.21.15.53:0/2674829491 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340c9790 con 0x7f5c340c18d0 2023-10-21T16:22:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c395a2640 1 --2- 172.21.15.53:0/2674829491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c340c18d0 0x7f5c340c8b40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5c340cdf00 tx=0x7f5c2402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68638c601d9d6712 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c38da1640 1 -- 172.21.15.53:0/2674829491 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24031710 con 0x7f5c340c18d0 2023-10-21T16:22:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c38da1640 1 -- 172.21.15.53:0/2674829491 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2402f050 con 0x7f5c340c18d0 2023-10-21T16:22:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c38da1640 1 -- 172.21.15.53:0/2674829491 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24031a70 con 0x7f5c340c18d0 2023-10-21T16:22:23.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2674829491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c340c18d0 msgr2=0x7f5c340c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.770+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2674829491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c340c18d0 0x7f5c340c8b40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5c340cdf00 tx=0x7f5c2402b7b0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2674829491 shutdown_connections 2023-10-21T16:22:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2674829491 >> v2:172.21.15.124:3300/0 conn(0x7f5c34164d10 0x7f5c340ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2674829491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c340c18d0 0x7f5c340c8b40 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2674829491 >> 172.21.15.53:0/2674829491 conn(0x7f5c3406c050 msgr2=0x7f5c3406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2674829491 shutdown_connections 2023-10-21T16:22:23.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.771+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2674829491 wait complete. 2023-10-21T16:22:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.772+0000 7f5c3ada5640 1 Processor -- start 2023-10-21T16:22:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.773+0000 7f5c3ada5640 1 -- start start 2023-10-21T16:22:23.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 0x7f5c340c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c340c18d0 0x7f5c340c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414d450 con 0x7f5c34164d10 2023-10-21T16:22:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341fa710 con 0x7f5c340aff70 2023-10-21T16:22:23.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3ada5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34150570 con 0x7f5c340c18d0 2023-10-21T16:22:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51284/0 (socket says 172.21.15.53:51284) 2023-10-21T16:22:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 -- 172.21.15.53:0/2832507642 learned_addr learned my addr 172.21.15.53:0/2832507642 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c39da3640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 0x7f5c340c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 -- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c340c18d0 msgr2=0x7f5c340c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c340c18d0 0x7f5c340c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 -- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 msgr2=0x7f5c340c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 0x7f5c340c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c3a5a4640 1 -- 172.21.15.53:0/2832507642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009400 con 0x7f5c34164d10 2023-10-21T16:22:23.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.774+0000 7f5c39da3640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 0x7f5c340c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c3a5a4640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007e90 tx=0x7f5c2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c014420 con 0x7f5c34164d10 2023-10-21T16:22:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c014580 con 0x7f5c34164d10 2023-10-21T16:22:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c014930 con 0x7f5c34164d10 2023-10-21T16:22:23.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c341f9de0 con 0x7f5c34164d10 2023-10-21T16:22:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.775+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c341fa3a0 con 0x7f5c34164d10 2023-10-21T16:22:23.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.776+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c341fa8a0 con 0x7f5c34164d10 2023-10-21T16:22:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.777+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 75) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c2c014a90 con 0x7f5c34164d10 2023-10-21T16:22:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.778+0000 7f5c2a7fc640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 0x7f5c1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.778+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c2c08a200 con 0x7f5c34164d10 2023-10-21T16:22:23.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.779+0000 7f5c39da3640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 0x7f5c1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:23.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.780+0000 7f5c39da3640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 0x7f5c1806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005880 tx=0x7f5c30009040 comp rx=0 tx=0).ready entity=mgr.36626 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:23.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:23.782+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c2c0550a0 con 0x7f5c34164d10 2023-10-21T16:22:24.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.026+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f5c34074150 con 0x7f5c34164d10 2023-10-21T16:22:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.558+0000 7f5c2a7fc640 1 -- 172.21.15.53:0/2832507642 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v76) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5c2c0586c0 con 0x7f5c34164d10 2023-10-21T16:22:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 msgr2=0x7f5c1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 0x7f5c1806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005880 tx=0x7f5c30009040 comp rx=0 tx=0).stop 2023-10-21T16:22:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 msgr2=0x7f5c340c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c007e90 tx=0x7f5c2c00e4b0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 shutdown_connections 2023-10-21T16:22:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f5c1806af90 0x7f5c1806d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.563+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c34164d10 0x7f5c340c7490 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.564+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c340c18d0 0x7f5c340c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.564+0000 7f5c3ada5640 1 --2- 172.21.15.53:0/2832507642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c340aff70 0x7f5c340c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.564+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 >> 172.21.15.53:0/2832507642 conn(0x7f5c3406c050 msgr2=0x7f5c34073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.564+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 shutdown_connections 2023-10-21T16:22:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.564+0000 7f5c3ada5640 1 -- 172.21.15.53:0/2832507642 wait complete. 2023-10-21T16:22:24.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T16:22:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.797+0000 7f1fdb7dc640 1 Processor -- start 2023-10-21T16:22:24.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.797+0000 7f1fdb7dc640 1 -- start start 2023-10-21T16:22:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 0x7f1fd41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 0x7f1fd40b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fd40b4c40 0x7f1fd406b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40b5090 con 0x7f1fd40b4c40 2023-10-21T16:22:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd40b8c30 con 0x7f1fd415cd70 2023-10-21T16:22:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fdb7dc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40cdd80 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fd9d79640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fd40b4c40 0x7f1fd406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fd9d79640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fd40b4c40 0x7f1fd406b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54470/0 (socket says 172.21.15.53:54470) 2023-10-21T16:22:24.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.799+0000 7f1fd9d79640 1 -- 172.21.15.53:0/14202107 learned_addr learned my addr 172.21.15.53:0/14202107 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:24.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9578640 1 --2- 172.21.15.53:0/14202107 >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 0x7f1fd41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd8d77640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 0x7f1fd40b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9d79640 1 -- 172.21.15.53:0/14202107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 msgr2=0x7f1fd40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:24.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9d79640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 0x7f1fd40b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9d79640 1 -- 172.21.15.53:0/14202107 >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 msgr2=0x7f1fd41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:24.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9d79640 1 --2- 172.21.15.53:0/14202107 >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 0x7f1fd41612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9d79640 1 -- 172.21.15.53:0/14202107 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40ce060 con 0x7f1fd40b4c40 2023-10-21T16:22:24.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd8d77640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 0x7f1fd40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:24.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.800+0000 7f1fd9578640 1 --2- 172.21.15.53:0/14202107 >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 0x7f1fd41612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:24.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.802+0000 7f1fd9d79640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40b4c40 0x7f1fd406b290 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40d0800 tx=0x7f1fcc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82463b8b98c7e9ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.802+0000 7f1fcbfff640 1 -- 172.21.15.53:0/14202107 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc02ca80 con 0x7f1fd40b4c40 2023-10-21T16:22:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.802+0000 7f1fcbfff640 1 -- 172.21.15.53:0/14202107 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc02cbe0 con 0x7f1fd40b4c40 2023-10-21T16:22:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.802+0000 7f1fcbfff640 1 -- 172.21.15.53:0/14202107 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0343f0 con 0x7f1fd40b4c40 2023-10-21T16:22:24.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/14202107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40b4c40 msgr2=0x7f1fd406b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40b4c40 0x7f1fd406b290 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40d0800 tx=0x7f1fcc02bee0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/14202107 shutdown_connections 2023-10-21T16:22:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd40cd4e0 0x7f1fd40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/14202107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40b4c40 0x7f1fd406b290 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/14202107 >> v2:172.21.15.124:3300/0 conn(0x7f1fd415cd70 0x7f1fd41612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/14202107 >> 172.21.15.53:0/14202107 conn(0x7f1fd40cf700 msgr2=0x7f1fd40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/14202107 shutdown_connections 2023-10-21T16:22:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.803+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/14202107 wait complete. 2023-10-21T16:22:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.805+0000 7f1fdb7dc640 1 Processor -- start 2023-10-21T16:22:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.805+0000 7f1fdb7dc640 1 -- start start 2023-10-21T16:22:24.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fd40b4c40 0x7f1fd40c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd415cd70 0x7f1fd40d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40ca0c0 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40d2040 con 0x7f1fd40b4c40 2023-10-21T16:22:24.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fdb7dc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd40d22f0 con 0x7f1fd415cd70 2023-10-21T16:22:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.806+0000 7f1fd8d77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54486/0 (socket says 172.21.15.53:54486) 2023-10-21T16:22:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 -- 172.21.15.53:0/2270123931 learned_addr learned my addr 172.21.15.53:0/2270123931 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd9578640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fd40b4c40 0x7f1fd40c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd9d79640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd415cd70 0x7f1fd40d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd415cd70 msgr2=0x7f1fd40d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd415cd70 0x7f1fd40d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fd40b4c40 msgr2=0x7f1fd40c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fd40b4c40 0x7f1fd40c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:24.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 -- 172.21.15.53:0/2270123931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc009b30 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.807+0000 7f1fd8d77640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400c7f0 tx=0x7f1fc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:24.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.808+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4007820 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.808+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4007980 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.808+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4007d30 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.808+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd40d25d0 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.808+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd40c9d50 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.809+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd40ca230 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.810+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1fc40113f0 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.811+0000 7f1fc9ffb640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 0x7f1fb806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.811+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fc4089d20 con 0x7f1fd40cd4e0 2023-10-21T16:22:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.811+0000 7f1fd9578640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 msgr2=0x7f1fb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:24.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.811+0000 7f1fd9578640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 0x7f1fb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:22:24.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:24.815+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fc4054a30 con 0x7f1fd40cd4e0 2023-10-21T16:22:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.012+0000 7f1fd9578640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 msgr2=0x7f1fb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.012+0000 7f1fd9578640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 0x7f1fb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:22:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.015+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1fd404e160 con 0x7f1fd40cd4e0 2023-10-21T16:22:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.020+0000 7f1fc9ffb640 1 -- 172.21.15.53:0/2270123931 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1fc4058050 con 0x7f1fd40cd4e0 2023-10-21T16:22:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 msgr2=0x7f1fb806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:22:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 0x7f1fb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 msgr2=0x7f1fd40d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400c7f0 tx=0x7f1fc400ccb0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 shutdown_connections 2023-10-21T16:22:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1fb806afe0 0x7f1fb806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fd415cd70 0x7f1fd40d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fd40cd4e0 0x7f1fd40d15c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 --2- 172.21.15.53:0/2270123931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1fd40b4c40 0x7f1fd40c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 >> 172.21.15.53:0/2270123931 conn(0x7f1fd40cf700 msgr2=0x7f1fd415c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 shutdown_connections 2023-10-21T16:22:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.023+0000 7f1fdb7dc640 1 -- 172.21.15.53:0/2270123931 wait complete. 2023-10-21T16:22:25.042 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T16:22:25.085 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.084+0000 7f41422b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:22:25.130 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.128+0000 7fdb46225040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T16:22:25.137 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.136+0000 7f41422b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:22:25.186 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.185+0000 7fdb46225040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T16:22:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.254+0000 7f1c5b59e640 1 Processor -- start 2023-10-21T16:22:25.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.255+0000 7f1c5b59e640 1 -- start start 2023-10-21T16:22:25.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 0x7f1c5c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1c5c065ab0 0x7f1c5c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c069d50 con 0x7f1c5c15dde0 2023-10-21T16:22:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c06e6c0 con 0x7f1c5c065ab0 2023-10-21T16:22:25.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c06c750 con 0x7f1c5c0ce290 2023-10-21T16:22:25.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5ad9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1c5c15dde0 0x7f1c5c160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36450/0 (socket says 172.21.15.53:36450) 2023-10-21T16:22:25.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5ad9d640 1 -- 172.21.15.53:0/83796782 learned_addr learned my addr 172.21.15.53:0/83796782 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T16:22:25.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.256+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/83796782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 0x7f1c5c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c53fff640 1 --2- 172.21.15.53:0/83796782 >> v2:172.21.15.124:3300/0 conn(0x7f1c5c065ab0 0x7f1c5c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855632824 0 0) 0x7f1c5c069d50 con 0x7f1c5c15dde0 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c48003520 con 0x7f1c5c15dde0 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1486490581 0 0) 0x7f1c48003520 con 0x7f1c5c15dde0 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c5a59c640 1 -- 172.21.15.53:0/83796782 >> v2:172.21.15.124:3300/0 conn(0x7f1c5c065ab0 msgr2=0x7f1c5c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/83796782 >> v2:172.21.15.124:3300/0 conn(0x7f1c5c065ab0 0x7f1c5c1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c5a59c640 1 -- 172.21.15.53:0/83796782 >> v1:172.21.15.53:6789/0 conn(0x7f1c5c15dde0 legacy=0x7f1c5c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.257+0000 7f1c5a59c640 1 -- 172.21.15.53:0/83796782 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0cebb0 con 0x7f1c5c0ce290 2023-10-21T16:22:25.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.258+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/83796782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 0x7f1c5c0ce670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa36fe427805599 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:25.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.259+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540019f0 con 0x7f1c5c0ce290 2023-10-21T16:22:25.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.259+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c02e050 con 0x7f1c5c0ce290 2023-10-21T16:22:25.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.259+0000 7f1c59d9b640 1 -- 172.21.15.53:0/83796782 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c030730 con 0x7f1c5c0ce290 2023-10-21T16:22:25.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.259+0000 7f1c5b59e640 1 -- 172.21.15.53:0/83796782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 msgr2=0x7f1c5c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.259+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/83796782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 0x7f1c5c0ce670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0097a0 tx=0x7f1c4c02b790 comp rx=0 tx=0).stop 2023-10-21T16:22:25.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 -- 172.21.15.53:0/83796782 shutdown_connections 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/83796782 >> v2:172.21.15.124:3300/0 conn(0x7f1c5c065ab0 0x7f1c5c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/83796782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c0ce290 0x7f1c5c0ce670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 -- 172.21.15.53:0/83796782 >> 172.21.15.53:0/83796782 conn(0x7f1c5c0b08d0 msgr2=0x7f1c5c0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 -- 172.21.15.53:0/83796782 shutdown_connections 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.260+0000 7f1c5b59e640 1 -- 172.21.15.53:0/83796782 wait complete. 2023-10-21T16:22:25.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.261+0000 7f1c5b59e640 1 Processor -- start 2023-10-21T16:22:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.262+0000 7f1c5b59e640 1 -- start start 2023-10-21T16:22:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.262+0000 7f1c5b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 0x7f1c5c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c5c0ce290 0x7f1c5c1f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0c2cd0 con 0x7f1c5c15dde0 2023-10-21T16:22:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c156b00 con 0x7f1c5c0ce290 2023-10-21T16:22:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b2f90 con 0x7f1c5c065ab0 2023-10-21T16:22:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54496/0 (socket says 172.21.15.53:54496) 2023-10-21T16:22:25.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 -- 172.21.15.53:0/2836394537 learned_addr learned my addr 172.21.15.53:0/2836394537 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 0x7f1c5c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c53fff640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c5c0ce290 0x7f1c5c1f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 msgr2=0x7f1c5c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 0x7f1c5c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c5c0ce290 msgr2=0x7f1c5c1f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c5c0ce290 0x7f1c5c1f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5ad9d640 1 -- 172.21.15.53:0/2836394537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c009400 con 0x7f1c5c15dde0 2023-10-21T16:22:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.263+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 0x7f1c5c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c5ad9d640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001890 tx=0x7f1c5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54012400 con 0x7f1c5c15dde0 2023-10-21T16:22:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c54012560 con 0x7f1c5c15dde0 2023-10-21T16:22:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54012910 con 0x7f1c5c15dde0 2023-10-21T16:22:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c0b4080 con 0x7f1c5c15dde0 2023-10-21T16:22:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.264+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0b4640 con 0x7f1c5c15dde0 2023-10-21T16:22:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.265+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c074150 con 0x7f1c5c15dde0 2023-10-21T16:22:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.266+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1c54012a70 con 0x7f1c5c15dde0 2023-10-21T16:22:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.267+0000 7f1c52ffd640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 0x7f1c3c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.267+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c5408a1c0 con 0x7f1c5c15dde0 2023-10-21T16:22:25.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.267+0000 7f1c5a59c640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 msgr2=0x7f1c3c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:25.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.267+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 0x7f1c3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:22:25.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.275+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c54054fc0 con 0x7f1c5c15dde0 2023-10-21T16:22:25.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.468+0000 7f1c5a59c640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 msgr2=0x7f1c3c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:25.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.468+0000 7f1c5a59c640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 0x7f1c3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:22:25.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.471+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1c5c0b14f0 con 0x7f1c5c15dde0 2023-10-21T16:22:25.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.476+0000 7f1c52ffd640 1 -- 172.21.15.53:0/2836394537 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1c540585e0 con 0x7f1c5c15dde0 2023-10-21T16:22:25.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.478+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 msgr2=0x7f1c3c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:22:25.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.478+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 0x7f1c3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.478+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 msgr2=0x7f1c5c1f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.478+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1c54001890 tx=0x7f1c5400a430 comp rx=0 tx=0).stop 2023-10-21T16:22:25.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 shutdown_connections 2023-10-21T16:22:25.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f1c3c06afe0 0x7f1c3c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c15dde0 0x7f1c5c1f5980 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c5c0ce290 0x7f1c5c1f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 --2- 172.21.15.53:0/2836394537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c5c065ab0 0x7f1c5c1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 >> 172.21.15.53:0/2836394537 conn(0x7f1c5c0b08d0 msgr2=0x7f1c5c06cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:25.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 shutdown_connections 2023-10-21T16:22:25.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.479+0000 7f1c5b59e640 1 -- 172.21.15.53:0/2836394537 wait complete. 2023-10-21T16:22:25.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T16:22:25.499 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T16:22:25.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2023-10-21T16:22:25.584 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.583+0000 7f41422b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:22:25.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.635+0000 7f41422b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:22:25.678 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.676+0000 7fdb46225040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T16:22:25.686 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.685+0000 7f41422b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:22:25.734 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.732+0000 7fdb46225040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T16:22:25.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.745+0000 7f41422b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:22:25.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.779+0000 7f4b5511d640 1 Processor -- start 2023-10-21T16:22:25.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.779+0000 7f4b5511d640 1 -- start start 2023-10-21T16:22:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b5511d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b500cb0f0 0x7f4b5006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b5511d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4b500ce560 0x7f4b500ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b5511d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b50069d50 con 0x7f4b500cb4d0 2023-10-21T16:22:25.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b5511d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500ca8f0 con 0x7f4b500cb0f0 2023-10-21T16:22:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b5511d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5006fc00 con 0x7f4b500ce560 2023-10-21T16:22:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b4ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b500cb0f0 0x7f4b5006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b4ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4b500cb0f0 0x7f4b5006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54498/0 (socket says 172.21.15.53:54498) 2023-10-21T16:22:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/3902253589 learned_addr learned my addr 172.21.15.53:0/3902253589 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:25.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.781+0000 7f4b4f59e640 1 --2- 172.21.15.53:0/3902253589 >> v2:172.21.15.124:3300/0 conn(0x7f4b500ce560 0x7f4b500ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.782+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/3902253589 >> v2:172.21.15.124:3300/0 conn(0x7f4b500ce560 msgr2=0x7f4b500ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.782+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/3902253589 >> v2:172.21.15.124:3300/0 conn(0x7f4b500ce560 0x7f4b500ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.782+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/3902253589 >> v1:172.21.15.124:6789/0 conn(0x7f4b500cb4d0 legacy=0x7f4b500cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:25.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.782+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/3902253589 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50069ff0 con 0x7f4b500cb0f0 2023-10-21T16:22:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.783+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/3902253589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500cb0f0 0x7f4b5006f5b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4b500cea40 tx=0x7f4b4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c442410ea312cd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.783+0000 7f4b4e59c640 1 -- 172.21.15.53:0/3902253589 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4002ca80 con 0x7f4b500cb0f0 2023-10-21T16:22:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.783+0000 7f4b4e59c640 1 -- 172.21.15.53:0/3902253589 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4002cbe0 con 0x7f4b500cb0f0 2023-10-21T16:22:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.783+0000 7f4b4e59c640 1 -- 172.21.15.53:0/3902253589 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b400343f0 con 0x7f4b500cb0f0 2023-10-21T16:22:25.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 -- 172.21.15.53:0/3902253589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500cb0f0 msgr2=0x7f4b5006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 --2- 172.21.15.53:0/3902253589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500cb0f0 0x7f4b5006f5b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4b500cea40 tx=0x7f4b4002bee0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 -- 172.21.15.53:0/3902253589 shutdown_connections 2023-10-21T16:22:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 --2- 172.21.15.53:0/3902253589 >> v2:172.21.15.124:3300/0 conn(0x7f4b500ce560 0x7f4b500ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 --2- 172.21.15.53:0/3902253589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500cb0f0 0x7f4b5006f5b0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 -- 172.21.15.53:0/3902253589 >> 172.21.15.53:0/3902253589 conn(0x7f4b500cc2d0 msgr2=0x7f4b500cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 -- 172.21.15.53:0/3902253589 shutdown_connections 2023-10-21T16:22:25.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.784+0000 7f4b5511d640 1 -- 172.21.15.53:0/3902253589 wait complete. 2023-10-21T16:22:25.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.786+0000 7f4b5511d640 1 Processor -- start 2023-10-21T16:22:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.786+0000 7f4b5511d640 1 -- start start 2023-10-21T16:22:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500cb0f0 0x7f4b50150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b500cb4d0 0x7f4b500af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 0x7f4b500afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c85a0 con 0x7f4b500ce560 2023-10-21T16:22:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c7f90 con 0x7f4b500cb4d0 2023-10-21T16:22:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b5511d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50151520 con 0x7f4b500cb0f0 2023-10-21T16:22:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b4f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 0x7f4b500afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b4f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 0x7f4b500afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54506/0 (socket says 172.21.15.53:54506) 2023-10-21T16:22:25.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.787+0000 7f4b4f59e640 1 -- 172.21.15.53:0/196304112 learned_addr learned my addr 172.21.15.53:0/196304112 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4659c640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b500cb4d0 0x7f4b500af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500cb0f0 0x7f4b50150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500cb0f0 msgr2=0x7f4b50150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b500cb0f0 0x7f4b50150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b500cb4d0 msgr2=0x7f4b500af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b500cb4d0 0x7f4b500af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b40009b30 con 0x7f4b500ce560 2023-10-21T16:22:25.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4659c640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4b500cb4d0 0x7f4b500af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T16:22:25.794 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.788+0000 7fdb46225040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T16:22:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b4f59e640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 0x7f4b500afd40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d5a0 tx=0x7f4b4800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.788+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480125f0 con 0x7f4b500ce560 2023-10-21T16:22:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.789+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48012750 con 0x7f4b500ce560 2023-10-21T16:22:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.789+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b500c7880 con 0x7f4b500ce560 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.789+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48012ae0 con 0x7f4b500ce560 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.789+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b500c7b60 con 0x7f4b500ce560 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.790+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500c7d80 con 0x7f4b500ce560 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.791+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 76) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4b48012c40 con 0x7f4b500ce560 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.792+0000 7f4b47fff640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.792+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b48089e40 con 0x7f4b500ce560 2023-10-21T16:22:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.792+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 msgr2=0x7f4b3006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:25.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.792+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T16:22:25.798 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.794+0000 7f41422b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:22:25.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.800+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b48054bc0 con 0x7f4b500ce560 2023-10-21T16:22:25.858 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.857+0000 7fdb46225040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T16:22:25.906 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.905+0000 7f41422b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:22:25.910 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:25.909+0000 7fdb46225040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T16:22:25.968 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:25.967+0000 7f41422b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:22:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.993+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 msgr2=0x7f4b3006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:25.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:25.993+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T16:22:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:26.008+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5006a5b0 con 0x7f4b3006afe0 2023-10-21T16:22:26.017 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.017+0000 7f41422b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:22:26.027 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.026+0000 7fdb46225040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T16:22:26.076 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.075+0000 7f41422b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:22:26.094 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.093+0000 7fdb46225040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T16:22:26.127 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.126+0000 7f41422b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:22:26.146 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.145+0000 7fdb46225040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T16:22:26.206 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.204+0000 7fdb46225040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T16:22:26.259 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.258+0000 7fdb46225040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T16:22:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:26.394+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 msgr2=0x7f4b3006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:26.394+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T16:22:26.507 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.506+0000 7f41422b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:22:26.664 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.662+0000 7fdb46225040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T16:22:26.666 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.665+0000 7f41422b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:22:26.773 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.772+0000 7f41422b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:22:26.823 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.822+0000 7f41422b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:22:26.832 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.830+0000 7fdb46225040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T16:22:26.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:26.930+0000 7f41422b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:22:26.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.944+0000 7fdb46225040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T16:22:26.999 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:26.997+0000 7fdb46225040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T16:22:27.037 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:27.036+0000 7f41422b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:22:27.113 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:27.111+0000 7fdb46225040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:22:27.194 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:27.192+0000 7f41422b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:22:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:27.195+0000 7f4b4ed9d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 msgr2=0x7f4b3006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/870299393 2023-10-21T16:22:27.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:27.195+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T16:22:27.226 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:27.224+0000 7fdb46225040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T16:22:27.242 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:27.242+0000 7f41422b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:22:27.360 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:27.359+0000 7f41422b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:22:27.391 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:27.389+0000 7fdb46225040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T16:22:27.443 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:27.441+0000 7fdb46225040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T16:22:27.569 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:27.567+0000 7fdb46225040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T16:22:28.193 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.191+0000 7f41422b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:22:28.432 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:28.430+0000 7fdb46225040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T16:22:28.487 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.486+0000 7f41422b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:22:28.543 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.542+0000 7f41422b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:22:28.655 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.654+0000 7f41422b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:22:28.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:28.675+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 77) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f4b48054f90 con 0x7f4b500ce560 2023-10-21T16:22:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:28.676+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 msgr2=0x7f4b3006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T16:22:28.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:28.676+0000 7f4b47fff640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/870299393,v1:172.21.15.124:6833/870299393] conn(0x7f4b3006afe0 0x7f4b3006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:28.753 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:28.752+0000 7fdb46225040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T16:22:28.820 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:28.819+0000 7fdb46225040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T16:22:28.834 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40c253e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40c253e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40c253e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40c253e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40c253e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40bf4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40bf4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40bf4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40bf4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:22:28.833+0000 7f40bf4f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:22:28.943 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T16:22:28.941+0000 7fdb46225040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T16:22:29.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.687+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 78) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4b48057d70 con 0x7f4b500ce560 2023-10-21T16:22:29.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.688+0000 7f4b47fff640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f4b300358e0 0x7f4b300754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:29.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.688+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 --> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b30035cc0 con 0x7f4b300358e0 2023-10-21T16:22:29.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.692+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f4b300358e0 0x7f4b300754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:29.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.700+0000 7f4b4ed9d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f4b300358e0 0x7f4b300754a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b40035000 tx=0x7f4b40030000 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:29.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.863+0000 7f4b47fff640 1 -- 172.21.15.53:0/196304112 <== mgr.7756 v2:172.21.15.124:6832/2988307648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4b30035cc0 con 0x7f4b300358e0 2023-10-21T16:22:29.865 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1 2023-10-21T16:22:29.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.866+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f4b300358e0 msgr2=0x7f4b300754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:29.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.866+0000 7f4b5511d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f4b300358e0 0x7f4b300754a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b40035000 tx=0x7f4b40030000 comp rx=0 tx=0).stop 2023-10-21T16:22:29.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.866+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 msgr2=0x7f4b500afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:29.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.866+0000 7f4b5511d640 1 --2- 172.21.15.53:0/196304112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4b500ce560 0x7f4b500afd40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d5a0 tx=0x7f4b4800da60 comp rx=0 tx=0).stop 2023-10-21T16:22:29.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.866+0000 7f4b4f59e640 1 -- 172.21.15.53:0/196304112 reap_dead start 2023-10-21T16:22:29.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.867+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 shutdown_connections 2023-10-21T16:22:29.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.867+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 >> 172.21.15.53:0/196304112 conn(0x7f4b500cc2d0 msgr2=0x7f4b500cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:29.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.867+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 shutdown_connections 2023-10-21T16:22:29.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:29.867+0000 7f4b5511d640 1 -- 172.21.15.53:0/196304112 wait complete. 2023-10-21T16:22:29.890 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-21T16:22:29.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-10-21T16:22:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.119+0000 7f91709a9640 1 Processor -- start 2023-10-21T16:22:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.119+0000 7f91709a9640 1 -- start start 2023-10-21T16:22:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f91709a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 0x7f916c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f91709a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f916c071070 0x7f916c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f91709a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0b1cc0 con 0x7f916c0afbe0 2023-10-21T16:22:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f91709a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0b2620 con 0x7f916c071070 2023-10-21T16:22:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f91709a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b2060 con 0x7f916c06dfc0 2023-10-21T16:22:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f916ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f916c071070 0x7f916c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f916ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f916c071070 0x7f916c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54512/0 (socket says 172.21.15.53:54512) 2023-10-21T16:22:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f916ad9d640 1 -- 172.21.15.53:0/235433872 learned_addr learned my addr 172.21.15.53:0/235433872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:30.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f9169d9b640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 0x7f916c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f916ad9d640 1 -- 172.21.15.53:0/235433872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 msgr2=0x7f916c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.121+0000 7f916ad9d640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 0x7f916c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.122+0000 7f916ad9d640 1 -- 172.21.15.53:0/235433872 >> v1:172.21.15.124:6789/0 conn(0x7f916c0afbe0 legacy=0x7f916c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.122+0000 7f916ad9d640 1 -- 172.21.15.53:0/235433872 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0b2340 con 0x7f916c071070 2023-10-21T16:22:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.122+0000 7f9169d9b640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 0x7f916c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f916ad9d640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c071070 0x7f916c0cb130 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9164009870 tx=0x7f916402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53fee947b93ab1f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f916959a640 1 -- 172.21.15.53:0/235433872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164031640 con 0x7f916c071070 2023-10-21T16:22:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f916959a640 1 -- 172.21.15.53:0/235433872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640317a0 con 0x7f916c071070 2023-10-21T16:22:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f916959a640 1 -- 172.21.15.53:0/235433872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164031b30 con 0x7f916c071070 2023-10-21T16:22:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f91709a9640 1 -- 172.21.15.53:0/235433872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c071070 msgr2=0x7f916c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.123+0000 7f91709a9640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c071070 0x7f916c0cb130 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9164009870 tx=0x7f916402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 -- 172.21.15.53:0/235433872 shutdown_connections 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c06dfc0 0x7f916c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 --2- 172.21.15.53:0/235433872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c071070 0x7f916c0cb130 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 -- 172.21.15.53:0/235433872 >> 172.21.15.53:0/235433872 conn(0x7f916c06a5d0 msgr2=0x7f916c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:30.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 -- 172.21.15.53:0/235433872 shutdown_connections 2023-10-21T16:22:30.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.124+0000 7f91709a9640 1 -- 172.21.15.53:0/235433872 wait complete. 2023-10-21T16:22:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.125+0000 7f91709a9640 1 Processor -- start 2023-10-21T16:22:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.126+0000 7f91709a9640 1 -- start start 2023-10-21T16:22:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.126+0000 7f91709a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.126+0000 7f91709a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c071070 0x7f916c1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.126+0000 7f91709a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 0x7f916c1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f91709a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c1f0a60 con 0x7f916c06dfc0 2023-10-21T16:22:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f91709a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cd380 con 0x7f916c0afbe0 2023-10-21T16:22:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f91709a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cc770 con 0x7f916c071070 2023-10-21T16:22:30.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54520/0 (socket says 172.21.15.53:54520) 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 -- 172.21.15.53:0/374848173 learned_addr learned my addr 172.21.15.53:0/374848173 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f9169d9b640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c071070 0x7f916c1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916ad9d640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 0x7f916c1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 -- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c071070 msgr2=0x7f916c1f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c071070 0x7f916c1f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 -- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 msgr2=0x7f916c1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 0x7f916c1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916a59c640 1 -- 172.21.15.53:0/374848173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640094d0 con 0x7f916c06dfc0 2023-10-21T16:22:30.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.127+0000 7f916ad9d640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 0x7f916c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f916a59c640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c7c0 tx=0x7f915c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00aa50 con 0x7f916c06dfc0 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c00abb0 con 0x7f916c06dfc0 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c012420 con 0x7f916c06dfc0 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0cca50 con 0x7f916c06dfc0 2023-10-21T16:22:30.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.128+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0cd010 con 0x7f916c06dfc0 2023-10-21T16:22:30.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.129+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c04e160 con 0x7f916c06dfc0 2023-10-21T16:22:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.130+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 78) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f915c012580 con 0x7f916c06dfc0 2023-10-21T16:22:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.132+0000 7f915affd640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 0x7f914c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.132+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f915c0894e0 con 0x7f916c06dfc0 2023-10-21T16:22:30.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.132+0000 7f9169d9b640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 0x7f914c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.133+0000 7f9169d9b640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 0x7f914c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f916c15b1a0 tx=0x7f9160008040 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.137+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f915c0545f0 con 0x7f916c06dfc0 2023-10-21T16:22:30.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.351+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 --> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f916c070630 con 0x7f914c06ad10 2023-10-21T16:22:30.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.490+0000 7f915affd640 1 -- 172.21.15.53:0/374848173 <== mgr.7756 v2:172.21.15.124:6832/2988307648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f916000c040 con 0x7f914c06ad10 2023-10-21T16:22:30.492 INFO:teuthology.orchestra.run.smithi053.stdout:Retention added to path /snap_test_dir1 2023-10-21T16:22:30.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 msgr2=0x7f914c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 0x7f914c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f916c15b1a0 tx=0x7f9160008040 comp rx=0 tx=0).stop 2023-10-21T16:22:30.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 msgr2=0x7f916c1f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c7c0 tx=0x7f915c00cc80 comp rx=0 tx=0).stop 2023-10-21T16:22:30.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 shutdown_connections 2023-10-21T16:22:30.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f916c0afbe0 0x7f916c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f916c071070 0x7f916c1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f914c06ad10 0x7f914c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 --2- 172.21.15.53:0/374848173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c06dfc0 0x7f916c1f0520 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 >> 172.21.15.53:0/374848173 conn(0x7f916c06a5d0 msgr2=0x7f916c0c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:30.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 shutdown_connections 2023-10-21T16:22:30.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.494+0000 7f91709a9640 1 -- 172.21.15.53:0/374848173 wait complete. 2023-10-21T16:22:30.521 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-21T16:22:30.522 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-21T16:22:30.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-10-21T16:22:30.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.754+0000 7fb32aa59640 1 Processor -- start 2023-10-21T16:22:30.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.754+0000 7fb32aa59640 1 -- start start 2023-10-21T16:22:30.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32aa59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32415a820 0x7fb3240aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32aa59640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3240ca680 0x7fb3240b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32aa59640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3240cab60 con 0x7fb32415ac00 2023-10-21T16:22:30.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32aa59640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3240cb960 con 0x7fb3240ca680 2023-10-21T16:22:30.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32aa59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240cbdf0 con 0x7fb32415a820 2023-10-21T16:22:30.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb329256640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3240ca680 0x7fb3240b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb329256640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3240ca680 0x7fb3240b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54530/0 (socket says 172.21.15.53:54530) 2023-10-21T16:22:30.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb329256640 1 -- 172.21.15.53:0/2369305850 learned_addr learned my addr 172.21.15.53:0/2369305850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:30.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb32a258640 1 --1- 172.21.15.53:0/2369305850 >> v1:172.21.15.53:6789/0 conn(0x7fb32415ac00 0x7fb3240c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36452/0 (socket says 172.21.15.53:36452) 2023-10-21T16:22:30.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.756+0000 7fb329a57640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32415a820 0x7fb3240aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.757+0000 7fb329256640 1 -- 172.21.15.53:0/2369305850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32415a820 msgr2=0x7fb3240aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T16:22:30.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.757+0000 7fb329256640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32415a820 0x7fb3240aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.757+0000 7fb329256640 1 -- 172.21.15.53:0/2369305850 >> v1:172.21.15.53:6789/0 conn(0x7fb32415ac00 legacy=0x7fb3240c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.757+0000 7fb329256640 1 -- 172.21.15.53:0/2369305850 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240cb030 con 0x7fb3240ca680 2023-10-21T16:22:30.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.758+0000 7fb329256640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3240ca680 0x7fb3240b04b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb314009870 tx=0x7fb31402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4d2cfa0c544321 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.759+0000 7fb328a55640 1 -- 172.21.15.53:0/2369305850 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314031640 con 0x7fb3240ca680 2023-10-21T16:22:30.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.759+0000 7fb328a55640 1 -- 172.21.15.53:0/2369305850 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3140317a0 con 0x7fb3240ca680 2023-10-21T16:22:30.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.759+0000 7fb328a55640 1 -- 172.21.15.53:0/2369305850 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314031b30 con 0x7fb3240ca680 2023-10-21T16:22:30.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.759+0000 7fb32aa59640 1 -- 172.21.15.53:0/2369305850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3240ca680 msgr2=0x7fb3240b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.759+0000 7fb32aa59640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3240ca680 0x7fb3240b04b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb314009870 tx=0x7fb31402b8f0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 -- 172.21.15.53:0/2369305850 shutdown_connections 2023-10-21T16:22:30.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3240ca680 0x7fb3240b04b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 --2- 172.21.15.53:0/2369305850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb32415a820 0x7fb3240aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 -- 172.21.15.53:0/2369305850 >> 172.21.15.53:0/2369305850 conn(0x7fb32406c050 msgr2=0x7fb32406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:30.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 -- 172.21.15.53:0/2369305850 shutdown_connections 2023-10-21T16:22:30.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.760+0000 7fb32aa59640 1 -- 172.21.15.53:0/2369305850 wait complete. 2023-10-21T16:22:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.761+0000 7fb32aa59640 1 Processor -- start 2023-10-21T16:22:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.762+0000 7fb32aa59640 1 -- start start 2023-10-21T16:22:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240ca680 0x7fb32414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 0x7fb3240b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240ac770 con 0x7fb32415a820 2023-10-21T16:22:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240aca50 con 0x7fb32415ac00 2023-10-21T16:22:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32aa59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240acd30 con 0x7fb3240ca680 2023-10-21T16:22:30.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb329256640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb329256640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54542/0 (socket says 172.21.15.53:54542) 2023-10-21T16:22:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb329256640 1 -- 172.21.15.53:0/3937942211 learned_addr learned my addr 172.21.15.53:0/3937942211 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T16:22:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb329a57640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240ca680 0x7fb32414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.763+0000 7fb32a258640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 0x7fb3240b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 -- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240ca680 msgr2=0x7fb32414f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240ca680 0x7fb32414f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 -- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 msgr2=0x7fb3240b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 0x7fb3240b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 -- 172.21.15.53:0/3937942211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140094d0 con 0x7fb32415a820 2023-10-21T16:22:30.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb32a258640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 0x7fb3240b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T16:22:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb329256640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb314000c00 tx=0x7fb3140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31403a930 con 0x7fb32415a820 2023-10-21T16:22:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31403aa90 con 0x7fb32415a820 2023-10-21T16:22:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3240ad010 con 0x7fb32415a820 2023-10-21T16:22:30.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31403ae40 con 0x7fb32415a820 2023-10-21T16:22:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.764+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3240ac2e0 con 0x7fb32415a820 2023-10-21T16:22:30.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.766+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3240ac530 con 0x7fb32415a820 2023-10-21T16:22:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.767+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 79) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fb31402f050 con 0x7fb32415a820 2023-10-21T16:22:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.768+0000 7fb31a7fc640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 0x7fb30806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T16:22:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.768+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fb3140ab9f0 con 0x7fb32415a820 2023-10-21T16:22:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.769+0000 7fb329a57640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 0x7fb30806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T16:22:30.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.770+0000 7fb329a57640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 0x7fb30806d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb320009ad0 tx=0x7fb320008040 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T16:22:30.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.774+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3240ac530 con 0x7fb32415a820 2023-10-21T16:22:30.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:30.993+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 --> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3240cb5f0 con 0x7fb30806b010 2023-10-21T16:22:31.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.010+0000 7fb31a7fc640 1 -- 172.21.15.53:0/3937942211 <== mgr.7756 v2:172.21.15.124:6832/2988307648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7fb320009800 con 0x7fb30806b010 2023-10-21T16:22:31.012 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T16:22:31.012 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-21T16:22:31.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 msgr2=0x7fb30806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:31.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 0x7fb30806d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb320009ad0 tx=0x7fb320008040 comp rx=0 tx=0).stop 2023-10-21T16:22:31.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 msgr2=0x7fb3240b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T16:22:31.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb314000c00 tx=0x7fb3140046c0 comp rx=0 tx=0).stop 2023-10-21T16:22:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 shutdown_connections 2023-10-21T16:22:31.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.012+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb32415ac00 0x7fb3240b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fb30806b010 0x7fb30806d4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:31.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb32415a820 0x7fb3240b25d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 --2- 172.21.15.53:0/3937942211 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3240ca680 0x7fb32414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T16:22:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 >> 172.21.15.53:0/3937942211 conn(0x7fb32406c050 msgr2=0x7fb3241598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T16:22:31.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 shutdown_connections 2023-10-21T16:22:31.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T16:22:31.013+0000 7fb32aa59640 1 -- 172.21.15.53:0/3937942211 wait complete. 2023-10-21T16:22:31.036 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-21T16:22:31.037 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2023-10-21T16:22:52.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:22:52.864 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:23:00.514 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:23:00.512+0000 7f40b14dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:23:00.515 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:23:00.512+0000 7f40b14dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:23:00.515 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:23:00.512+0000 7f40b14dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:23:00.515 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:23:00.512+0000 7f40b14dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:23:00.515 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T16:23:00.512+0000 7f40b14dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T16:23:23.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:23:23.041 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:23:53.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:23:53.184 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:24:23.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:24:23.334 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:24:53.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:24:53.481 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:25:23.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:25:23.654 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:25:53.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:25:53.811 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:26:23.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:26:23.967 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:26:54.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:26:54.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:27:24.259 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:27:24.269 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:27:54.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:27:54.425 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:28:24.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:28:24.605 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:28:54.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:28:54.754 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:29:24.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:29:24.895 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:29:55.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:29:55.042 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:30:25.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:30:25.182 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:30:55.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:30:55.336 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:31:25.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:31:25.526 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:31:55.689 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:31:55.698 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:32:25.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:32:25.841 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:32:55.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:32:55.988 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:33:26.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:33:26.141 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:33:56.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:33:56.273 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:34:26.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:34:26.419 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:34:56.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:34:56.562 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:35:26.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:35:26.706 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:35:56.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:35:56.849 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:36:26.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:36:26.997 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:36:57.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:36:57.157 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:37:27.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:37:27.330 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:37:57.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:37:57.478 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:38:27.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:38:27.621 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:38:57.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:38:57.777 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:39:27.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:39:27.926 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:39:58.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:39:58.080 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:40:28.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:40:28.239 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:40:58.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:40:58.414 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:41:28.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:41:28.554 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:41:58.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:41:58.705 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:42:28.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:42:28.849 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:42:58.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:42:58.992 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:43:29.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:43:29.178 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:43:59.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:43:59.351 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:44:29.483 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:44:29.494 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:44:59.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:44:59.667 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:45:29.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:45:29.831 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:45:59.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:45:59.974 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:46:30.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:46:30.119 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:47:00.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:47:00.296 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:47:30.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:47:30.446 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:48:00.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:48:00.598 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:48:30.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:48:30.809 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:49:00.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:49:00.989 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:49:31.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:49:31.170 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:50:01.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:50:01.360 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:50:31.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:50:31.516 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:51:01.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:51:01.731 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:51:31.894 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:51:31.904 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:52:02.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:52:02.153 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:52:32.295 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:52:32.305 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:53:02.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:53:02.489 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:53:32.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:53:32.653 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:54:02.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:54:02.870 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:54:33.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:54:33.084 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:55:03.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:55:03.232 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:55:33.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:55:33.374 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:56:03.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:56:03.581 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:56:33.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:56:34.000 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:57:04.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:57:04.193 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:57:34.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:57:34.355 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:58:04.542 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:58:04.553 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:58:34.716 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:58:34.726 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:59:04.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:59:04.874 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:59:35.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T16:59:35.080 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:00:05.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:00:05.235 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:00:35.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:00:35.433 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:01:05.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:01:05.665 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:01:35.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:01:35.814 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:02:05.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:02:06.003 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:02:36.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:02:36.198 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:03:06.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:03:06.380 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:03:36.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:03:36.544 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:04:06.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:04:06.738 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:04:36.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:04:36.906 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:05:07.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:05:07.122 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:05:37.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:05:37.281 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:06:07.432 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:06:07.442 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:06:37.616 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:06:37.626 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:07:07.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:07:07.770 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:07:37.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:07:37.994 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:08:08.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:08:08.183 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:08:38.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:08:38.326 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:09:08.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:09:08.512 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:09:38.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:09:38.710 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:10:08.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:10:09.370 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:10:39.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:10:39.523 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:11:09.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:11:09.714 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:11:39.848 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:11:39.858 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:12:09.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:12:10.007 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:12:40.169 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:12:40.179 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:13:10.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:13:10.394 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:13:40.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:13:40.568 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:14:10.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:14:10.756 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:14:40.945 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:14:40.956 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:15:11.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:15:11.144 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:15:41.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:15:41.359 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:16:11.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:16:11.531 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:16:41.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:16:41.679 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:17:11.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:17:11.826 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:17:42.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:17:42.041 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:18:12.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:18:12.188 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:18:16.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:16.039 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:18:16.111 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_24_00_UTC 2023-10-21T17:18:16.112 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_25_00_UTC 2023-10-21T17:18:16.112 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_26_01_UTC 2023-10-21T17:18:16.112 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_27_00_UTC 2023-10-21T17:18:16.113 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_28_00_UTC 2023-10-21T17:18:16.113 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_29_00_UTC 2023-10-21T17:18:16.113 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_30_00_UTC 2023-10-21T17:18:16.113 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_31_00_UTC 2023-10-21T17:18:16.113 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_32_00_UTC 2023-10-21T17:18:16.114 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_33_00_UTC 2023-10-21T17:18:16.114 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_34_00_UTC 2023-10-21T17:18:16.114 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_35_00_UTC 2023-10-21T17:18:16.114 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_36_00_UTC 2023-10-21T17:18:16.114 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_37_00_UTC 2023-10-21T17:18:16.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_38_00_UTC 2023-10-21T17:18:16.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_39_00_UTC 2023-10-21T17:18:16.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_40_00_UTC 2023-10-21T17:18:16.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_41_00_UTC 2023-10-21T17:18:16.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_42_00_UTC 2023-10-21T17:18:16.116 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_43_00_UTC 2023-10-21T17:18:16.116 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_44_00_UTC 2023-10-21T17:18:16.116 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_45_00_UTC 2023-10-21T17:18:16.116 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_46_00_UTC 2023-10-21T17:18:16.116 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_47_00_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_48_00_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_49_00_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_50_00_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_51_01_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_52_00_UTC 2023-10-21T17:18:16.117 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_53_00_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_54_00_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_55_00_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_56_00_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_57_01_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_58_00_UTC 2023-10-21T17:18:16.118 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_59_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_00_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_01_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_02_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_03_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_04_00_UTC 2023-10-21T17:18:16.119 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_05_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_06_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_07_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_08_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_09_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_10_00_UTC 2023-10-21T17:18:16.120 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_11_00_UTC 2023-10-21T17:18:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_12_00_UTC 2023-10-21T17:18:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_13_00_UTC 2023-10-21T17:18:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_14_00_UTC 2023-10-21T17:18:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_15_00_UTC 2023-10-21T17:18:16.121 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_16_00_UTC 2023-10-21T17:18:16.122 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_17_00_UTC 2023-10-21T17:18:16.122 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_18_00_UTC 2023-10-21T17:18:16.123 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-16_24_00_UTC', 'scheduled-2023-10-21-16_25_00_UTC', 'scheduled-2023-10-21-16_26_01_UTC', 'scheduled-2023-10-21-16_27_00_UTC', 'scheduled-2023-10-21-16_28_00_UTC', 'scheduled-2023-10-21-16_29_00_UTC', 'scheduled-2023-10-21-16_30_00_UTC', 'scheduled-2023-10-21-16_31_00_UTC', 'scheduled-2023-10-21-16_32_00_UTC', 'scheduled-2023-10-21-16_33_00_UTC', 'scheduled-2023-10-21-16_34_00_UTC', 'scheduled-2023-10-21-16_35_00_UTC', 'scheduled-2023-10-21-16_36_00_UTC', 'scheduled-2023-10-21-16_37_00_UTC', 'scheduled-2023-10-21-16_38_00_UTC', 'scheduled-2023-10-21-16_39_00_UTC', 'scheduled-2023-10-21-16_40_00_UTC', 'scheduled-2023-10-21-16_41_00_UTC', 'scheduled-2023-10-21-16_42_00_UTC', 'scheduled-2023-10-21-16_43_00_UTC', 'scheduled-2023-10-21-16_44_00_UTC', 'scheduled-2023-10-21-16_45_00_UTC', 'scheduled-2023-10-21-16_46_00_UTC', 'scheduled-2023-10-21-16_47_00_UTC', 'scheduled-2023-10-21-16_48_00_UTC', 'scheduled-2023-10-21-16_49_00_UTC', 'scheduled-2023-10-21-16_50_00_UTC', 'scheduled-2023-10-21-16_51_01_UTC', 'scheduled-2023-10-21-16_52_00_UTC', 'scheduled-2023-10-21-16_53_00_UTC', 'scheduled-2023-10-21-16_54_00_UTC', 'scheduled-2023-10-21-16_55_00_UTC', 'scheduled-2023-10-21-16_56_00_UTC', 'scheduled-2023-10-21-16_57_01_UTC', 'scheduled-2023-10-21-16_58_00_UTC', 'scheduled-2023-10-21-16_59_00_UTC', 'scheduled-2023-10-21-17_00_00_UTC', 'scheduled-2023-10-21-17_01_00_UTC', 'scheduled-2023-10-21-17_02_00_UTC', 'scheduled-2023-10-21-17_03_00_UTC', 'scheduled-2023-10-21-17_04_00_UTC', 'scheduled-2023-10-21-17_05_00_UTC', 'scheduled-2023-10-21-17_06_00_UTC', 'scheduled-2023-10-21-17_07_00_UTC', 'scheduled-2023-10-21-17_08_00_UTC', 'scheduled-2023-10-21-17_09_00_UTC', 'scheduled-2023-10-21-17_10_00_UTC', 'scheduled-2023-10-21-17_11_00_UTC', 'scheduled-2023-10-21-17_12_00_UTC', 'scheduled-2023-10-21-17_13_00_UTC', 'scheduled-2023-10-21-17_14_00_UTC', 'scheduled-2023-10-21-17_15_00_UTC', 'scheduled-2023-10-21-17_16_00_UTC', 'scheduled-2023-10-21-17_17_00_UTC', 'scheduled-2023-10-21-17_18_00_UTC'] 2023-10-21T17:18:16.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-10-21T17:18:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.354+0000 7fe5e9df4640 1 Processor -- start 2023-10-21T17:18:16.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.354+0000 7fe5e9df4640 1 -- start start 2023-10-21T17:18:16.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e9df4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5e415a820 0x7fe5e40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e9df4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415ac00 0x7fe5e40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e9df4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40cab60 con 0x7fe5e40ca680 2023-10-21T17:18:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e9df4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40cb960 con 0x7fe5e415a820 2023-10-21T17:18:16.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e9df4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40cbdf0 con 0x7fe5e415ac00 2023-10-21T17:18:16.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e8df2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5e415a820 0x7fe5e40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.356+0000 7fe5e95f3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415ac00 0x7fe5e40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5e415a820 0x7fe5e40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:39792/0 (socket says 172.21.15.53:39792) 2023-10-21T17:18:16.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 -- 172.21.15.53:0/1440973897 learned_addr learned my addr 172.21.15.53:0/1440973897 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:16.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 -- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415ac00 msgr2=0x7fe5e40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:16.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415ac00 0x7fe5e40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 -- 172.21.15.53:0/1440973897 >> v1:172.21.15.124:6789/0 conn(0x7fe5e40ca680 legacy=0x7fe5e40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.357+0000 7fe5e8df2640 1 -- 172.21.15.53:0/1440973897 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40cb030 con 0x7fe5e415a820 2023-10-21T17:18:16.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.359+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e415a820 0x7fe5e40aff70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c816ecbb9561a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.359+0000 7fe5e37fe640 1 -- 172.21.15.53:0/1440973897 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8031640 con 0x7fe5e415a820 2023-10-21T17:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.359+0000 7fe5e37fe640 1 -- 172.21.15.53:0/1440973897 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d80317a0 con 0x7fe5e415a820 2023-10-21T17:18:16.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.359+0000 7fe5e37fe640 1 -- 172.21.15.53:0/1440973897 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8031b30 con 0x7fe5e415a820 2023-10-21T17:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e415a820 msgr2=0x7fe5e40aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e415a820 0x7fe5e40aff70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009870 tx=0x7fe5d802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1440973897 shutdown_connections 2023-10-21T17:18:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415ac00 0x7fe5e40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1440973897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e415a820 0x7fe5e40aff70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1440973897 >> 172.21.15.53:0/1440973897 conn(0x7fe5e406c050 msgr2=0x7fe5e406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1440973897 shutdown_connections 2023-10-21T17:18:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.360+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1440973897 wait complete. 2023-10-21T17:18:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.363+0000 7fe5e9df4640 1 Processor -- start 2023-10-21T17:18:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.363+0000 7fe5e9df4640 1 -- start start 2023-10-21T17:18:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.364+0000 7fe5e9df4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 0x7fe5e40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.364+0000 7fe5e9df4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 0x7fe5e4151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e9df4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e9df4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40c4610 con 0x7fe5e415ac00 2023-10-21T17:18:16.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e9df4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40afec0 con 0x7fe5e40ca680 2023-10-21T17:18:16.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e9df4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40b0820 con 0x7fe5e415a820 2023-10-21T17:18:16.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e95f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e95f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48214/0 (socket says 172.21.15.53:48214) 2023-10-21T17:18:16.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e95f3640 1 -- 172.21.15.53:0/1757336585 learned_addr learned my addr 172.21.15.53:0/1757336585 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e3fff640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 0x7fe5e4151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.365+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 0x7fe5e40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 -- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 msgr2=0x7fe5e4151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 0x7fe5e4151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 -- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 msgr2=0x7fe5e40b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 0x7fe5e40b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 -- 172.21.15.53:0/1757336585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80094d0 con 0x7fe5e415ac00 2023-10-21T17:18:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 0x7fe5e40b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e3fff640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 0x7fe5e4151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e95f3640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009cb0 tx=0x7fe5dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc009b20 con 0x7fe5e415ac00 2023-10-21T17:18:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc00f040 con 0x7fe5e415ac00 2023-10-21T17:18:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc0146c0 con 0x7fe5e415ac00 2023-10-21T17:18:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.366+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e40af240 con 0x7fe5e415ac00 2023-10-21T17:18:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.367+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e40af800 con 0x7fe5e415ac00 2023-10-21T17:18:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.368+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e40b0000 con 0x7fe5e415ac00 2023-10-21T17:18:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.369+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 82) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fe5dc014820 con 0x7fe5e415ac00 2023-10-21T17:18:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.370+0000 7fe5e17fa640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 0x7fe5d006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.370+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fe5dc08aa40 con 0x7fe5e415ac00 2023-10-21T17:18:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.370+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 0x7fe5d006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.372+0000 7fe5e8df2640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 0x7fe5d006d4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8036000 tx=0x7fe5d803c040 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.375+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5e40b0000 con 0x7fe5e415ac00 2023-10-21T17:18:16.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.590+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 --> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5e40cb5f0 con 0x7fe5d006afe0 2023-10-21T17:18:16.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.610+0000 7fe5e17fa640 1 -- 172.21.15.53:0/1757336585 <== mgr.7756 v2:172.21.15.124:6832/2988307648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe5d8031a10 con 0x7fe5d006afe0 2023-10-21T17:18:16.611 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:16.611 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:22:29", "first": "2023-10-21T16:23:00", "last": "2023-10-21T17:18:00", "last_pruned": "2023-10-21T17:18:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-21T17:18:16.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 msgr2=0x7fe5d006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 0x7fe5d006d4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8036000 tx=0x7fe5d803c040 comp rx=0 tx=0).stop 2023-10-21T17:18:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 msgr2=0x7fe5e40c2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009cb0 tx=0x7fe5dc00c490 comp rx=0 tx=0).stop 2023-10-21T17:18:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 shutdown_connections 2023-10-21T17:18:16.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7fe5d006afe0 0x7fe5d006d4a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e415ac00 0x7fe5e40c2c40 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e415a820 0x7fe5e4151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 --2- 172.21.15.53:0/1757336585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ca680 0x7fe5e40b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 >> 172.21.15.53:0/1757336585 conn(0x7fe5e406c050 msgr2=0x7fe5e40ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:16.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 shutdown_connections 2023-10-21T17:18:16.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.612+0000 7fe5e9df4640 1 -- 172.21.15.53:0/1757336585 wait complete. 2023-10-21T17:18:16.633 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T16:22:29", "first": "2023-10-21T16:23:00", "last": "2023-10-21T17:18:00", "last_pruned": "2023-10-21T17:18:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-21T17:18:16.634 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T16:22:29', 'first': '2023-10-21T16:23:00', 'last': '2023-10-21T17:18:00', 'last_pruned': '2023-10-21T17:18:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-21T17:18:16.634 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-21T17:18:16.634 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-21T17:18:16.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-10-21T17:18:16.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.864+0000 7f737a0bb640 1 Processor -- start 2023-10-21T17:18:16.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.865+0000 7f737a0bb640 1 -- start start 2023-10-21T17:18:16.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.866+0000 7f737a0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73740c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.866+0000 7f737a0bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73740ccbc0 con 0x7f737415cae0 2023-10-21T17:18:16.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f737a0bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73740ce1f0 con 0x7f73740aff70 2023-10-21T17:18:16.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f737a0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c94c0 con 0x7f73740c8b40 2023-10-21T17:18:16.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f73790b9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f737415cae0 0x7f73740cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40902/0 (socket says 172.21.15.53:40902) 2023-10-21T17:18:16.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f73790b9640 1 -- 172.21.15.53:0/1674216263 learned_addr learned my addr 172.21.15.53:0/1674216263 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:16.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164761998 0 0) 0x7f73740ccbc0 con 0x7f737415cae0 2023-10-21T17:18:16.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7364003520 con 0x7f737415cae0 2023-10-21T17:18:16.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f73788b8640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73740c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.867+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768449025 0 0) 0x7f7364003520 con 0x7f737415cae0 2023-10-21T17:18:16.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 msgr2=0x7f73740c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:16.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73740c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 >> v1:172.21.15.124:6789/0 conn(0x7f73740aff70 legacy=0x7f7374070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740c9720 con 0x7f737415cae0 2023-10-21T17:18:16.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f73680019f0 con 0x7f737415cae0 2023-10-21T17:18:16.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 >> v1:172.21.15.53:6789/0 conn(0x7f737415cae0 legacy=0x7f73740cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73640039a0 0x7f7364003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364004d00 0x7f7364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c94c0 con 0x7f7364004d00 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736400bb10 con 0x7f73640039a0 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736400bca0 con 0x7f73740c8b40 2023-10-21T17:18:16.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f73798ba640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364004d00 0x7f7364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f73790b9640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73640039a0 0x7f7364003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.868+0000 7f73788b8640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.869+0000 7f73798ba640 1 -- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 msgr2=0x7f73640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.869+0000 7f73798ba640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73740c8b40 0x7f73640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.869+0000 7f73798ba640 1 -- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73640039a0 msgr2=0x7f7364003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.869+0000 7f73798ba640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73640039a0 0x7f7364003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.869+0000 7f73798ba640 1 -- 172.21.15.53:0/1674216263 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364003520 con 0x7f7364004d00 2023-10-21T17:18:16.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.870+0000 7f73798ba640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364004d00 0x7f7364005180 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f736c001780 tx=0x7f73640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196f1b623521adb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.870+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c0052d0 con 0x7f7364004d00 2023-10-21T17:18:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.870+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c02e050 con 0x7f7364004d00 2023-10-21T17:18:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.870+0000 7f7373fff640 1 -- 172.21.15.53:0/1674216263 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c0035f0 con 0x7f7364004d00 2023-10-21T17:18:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 -- 172.21.15.53:0/1674216263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364004d00 msgr2=0x7f7364005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 --2- 172.21.15.53:0/1674216263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7364004d00 0x7f7364005180 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f736c001780 tx=0x7f73640057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f73798ba640 1 -- 172.21.15.53:0/1674216263 reap_dead start 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 -- 172.21.15.53:0/1674216263 shutdown_connections 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 -- 172.21.15.53:0/1674216263 >> 172.21.15.53:0/1674216263 conn(0x7f737406c050 msgr2=0x7f737406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 -- 172.21.15.53:0/1674216263 shutdown_connections 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.871+0000 7f737a0bb640 1 -- 172.21.15.53:0/1674216263 wait complete. 2023-10-21T17:18:16.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.873+0000 7f737a0bb640 1 Processor -- start 2023-10-21T17:18:16.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.873+0000 7f737a0bb640 1 -- start start 2023-10-21T17:18:16.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7364004d00 0x7f737414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73741f3130 0x7f73740c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c5010 con 0x7f73741f2500 2023-10-21T17:18:16.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c6840 con 0x7f7364004d00 2023-10-21T17:18:16.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f737a0bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73741f2ef0 con 0x7f73741f3130 2023-10-21T17:18:16.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48238/0 (socket says 172.21.15.53:48238) 2023-10-21T17:18:16.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 -- 172.21.15.53:0/3306405598 learned_addr learned my addr 172.21.15.53:0/3306405598 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73790b9640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7364004d00 0x7f737414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73788b8640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73741f3130 0x7f73740c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 -- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73741f3130 msgr2=0x7f73740c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73741f3130 0x7f73740c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 -- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7364004d00 msgr2=0x7f737414e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7364004d00 0x7f737414e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.874+0000 7f73798ba640 1 -- 172.21.15.53:0/3306405598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740c4770 con 0x7f73741f2500 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f73798ba640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f736c034040 tx=0x7f736c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c007a30 con 0x7f73741f2500 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c0067d0 con 0x7f73741f2500 2023-10-21T17:18:16.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c006b80 con 0x7f73741f2500 2023-10-21T17:18:16.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73740c4a50 con 0x7f73741f2500 2023-10-21T17:18:16.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.875+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737414ee60 con 0x7f73741f2500 2023-10-21T17:18:16.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.876+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73740c6610 con 0x7f73741f2500 2023-10-21T17:18:16.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.877+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 82) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f736c006ce0 con 0x7f73741f2500 2023-10-21T17:18:16.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.878+0000 7f7371ffb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 0x7f735806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:16.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.878+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f736c0aaaf0 con 0x7f73741f2500 2023-10-21T17:18:16.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.879+0000 7f73790b9640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 0x7f735806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:16.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.880+0000 7f73790b9640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 0x7f735806d4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7368002010 tx=0x7f7368009040 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:16.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:16.883+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73740c6610 con 0x7f73741f2500 2023-10-21T17:18:17.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.101+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 --> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f73740cc850 con 0x7f735806afe0 2023-10-21T17:18:17.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.269+0000 7f7371ffb640 1 -- 172.21.15.53:0/3306405598 <== mgr.7756 v2:172.21.15.124:6832/2988307648 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f7368015460 con 0x7f735806afe0 2023-10-21T17:18:17.270 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1 2023-10-21T17:18:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 msgr2=0x7f735806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 0x7f735806d4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7368002010 tx=0x7f7368009040 comp rx=0 tx=0).stop 2023-10-21T17:18:17.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 msgr2=0x7f73741f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:17.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f736c034040 tx=0x7f736c001120 comp rx=0 tx=0).stop 2023-10-21T17:18:17.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 shutdown_connections 2023-10-21T17:18:17.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f735806afe0 0x7f735806d4a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:17.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73741f2500 0x7f73741f29b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73741f3130 0x7f73740c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 --2- 172.21.15.53:0/3306405598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7364004d00 0x7f737414e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 >> 172.21.15.53:0/3306405598 conn(0x7f737406c050 msgr2=0x7f73740ac750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 shutdown_connections 2023-10-21T17:18:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:17.272+0000 7f737a0bb640 1 -- 172.21.15.53:0/3306405598 wait complete. 2023-10-21T17:18:17.296 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-21T17:18:17.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.297 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:18:17.362 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_24_00_UTC 2023-10-21T17:18:17.362 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_25_00_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_26_01_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_27_00_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_28_00_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_29_00_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_30_00_UTC 2023-10-21T17:18:17.363 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_31_00_UTC 2023-10-21T17:18:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_32_00_UTC 2023-10-21T17:18:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_33_00_UTC 2023-10-21T17:18:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_34_00_UTC 2023-10-21T17:18:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_35_00_UTC 2023-10-21T17:18:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_36_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_37_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_38_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_39_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_40_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_41_00_UTC 2023-10-21T17:18:17.365 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_42_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_43_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_44_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_45_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_46_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_47_00_UTC 2023-10-21T17:18:17.366 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_48_00_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_49_00_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_50_00_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_51_01_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_52_00_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_53_00_UTC 2023-10-21T17:18:17.367 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_54_00_UTC 2023-10-21T17:18:17.368 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_55_00_UTC 2023-10-21T17:18:17.368 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_56_00_UTC 2023-10-21T17:18:17.368 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_57_01_UTC 2023-10-21T17:18:17.368 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_58_00_UTC 2023-10-21T17:18:17.368 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-16_59_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_00_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_01_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_02_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_03_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_04_00_UTC 2023-10-21T17:18:17.369 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_05_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_06_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_07_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_08_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_09_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_10_00_UTC 2023-10-21T17:18:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_11_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_12_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_13_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_14_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_15_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_16_00_UTC 2023-10-21T17:18:17.371 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_17_00_UTC 2023-10-21T17:18:17.372 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_18_00_UTC 2023-10-21T17:18:17.372 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_24_00_UTC 2023-10-21T17:18:17.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.373 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_24_00_UTC) 2023-10-21T17:18:17.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_25_00_UTC 2023-10-21T17:18:17.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.444 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_25_00_UTC) 2023-10-21T17:18:17.521 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_26_01_UTC 2023-10-21T17:18:17.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.522 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_26_01_UTC) 2023-10-21T17:18:17.592 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_27_00_UTC 2023-10-21T17:18:17.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.593 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_27_00_UTC) 2023-10-21T17:18:17.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_28_00_UTC 2023-10-21T17:18:17.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.662 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_28_00_UTC) 2023-10-21T17:18:17.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_29_00_UTC 2023-10-21T17:18:17.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.732 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_29_00_UTC) 2023-10-21T17:18:17.806 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_30_00_UTC 2023-10-21T17:18:17.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.806 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_30_00_UTC) 2023-10-21T17:18:17.881 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_31_00_UTC 2023-10-21T17:18:17.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.882 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_31_00_UTC) 2023-10-21T17:18:17.954 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_32_00_UTC 2023-10-21T17:18:17.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:17.955 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_32_00_UTC) 2023-10-21T17:18:18.028 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_33_00_UTC 2023-10-21T17:18:18.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.029 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_33_00_UTC) 2023-10-21T17:18:18.101 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_34_00_UTC 2023-10-21T17:18:18.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.102 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_34_00_UTC) 2023-10-21T17:18:18.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_35_00_UTC 2023-10-21T17:18:18.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.172 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_35_00_UTC) 2023-10-21T17:18:18.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_36_00_UTC 2023-10-21T17:18:18.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.245 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_36_00_UTC) 2023-10-21T17:18:18.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_37_00_UTC 2023-10-21T17:18:18.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.318 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_37_00_UTC) 2023-10-21T17:18:18.393 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_38_00_UTC 2023-10-21T17:18:18.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.394 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_38_00_UTC) 2023-10-21T17:18:18.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_39_00_UTC 2023-10-21T17:18:18.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.469 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_39_00_UTC) 2023-10-21T17:18:18.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_40_00_UTC 2023-10-21T17:18:18.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.542 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_40_00_UTC) 2023-10-21T17:18:18.610 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_41_00_UTC 2023-10-21T17:18:18.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.611 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_41_00_UTC) 2023-10-21T17:18:18.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_42_00_UTC 2023-10-21T17:18:18.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.684 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_42_00_UTC) 2023-10-21T17:18:18.755 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_43_00_UTC 2023-10-21T17:18:18.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.756 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_43_00_UTC) 2023-10-21T17:18:18.825 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_44_00_UTC 2023-10-21T17:18:18.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.826 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_44_00_UTC) 2023-10-21T17:18:18.899 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_45_00_UTC 2023-10-21T17:18:18.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.900 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_45_00_UTC) 2023-10-21T17:18:18.975 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_46_00_UTC 2023-10-21T17:18:18.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:18.975 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_46_00_UTC) 2023-10-21T17:18:19.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_47_00_UTC 2023-10-21T17:18:19.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.051 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_47_00_UTC) 2023-10-21T17:18:19.153 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_48_00_UTC 2023-10-21T17:18:19.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.154 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_48_00_UTC) 2023-10-21T17:18:19.211 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_49_00_UTC 2023-10-21T17:18:19.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.212 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_49_00_UTC) 2023-10-21T17:18:19.278 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_50_00_UTC 2023-10-21T17:18:19.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.279 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_50_00_UTC) 2023-10-21T17:18:19.335 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_51_01_UTC 2023-10-21T17:18:19.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.336 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_51_01_UTC) 2023-10-21T17:18:19.387 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_52_00_UTC 2023-10-21T17:18:19.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.388 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_52_00_UTC) 2023-10-21T17:18:19.446 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_53_00_UTC 2023-10-21T17:18:19.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.446 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_53_00_UTC) 2023-10-21T17:18:19.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_54_00_UTC 2023-10-21T17:18:19.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.513 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_54_00_UTC) 2023-10-21T17:18:19.566 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_55_00_UTC 2023-10-21T17:18:19.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.567 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_55_00_UTC) 2023-10-21T17:18:19.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_56_00_UTC 2023-10-21T17:18:19.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.628 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_56_00_UTC) 2023-10-21T17:18:19.695 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_57_01_UTC 2023-10-21T17:18:19.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.696 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_57_01_UTC) 2023-10-21T17:18:19.765 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_58_00_UTC 2023-10-21T17:18:19.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.766 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_58_00_UTC) 2023-10-21T17:18:19.841 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-16_59_00_UTC 2023-10-21T17:18:19.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.842 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-16_59_00_UTC) 2023-10-21T17:18:19.892 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_00_00_UTC 2023-10-21T17:18:19.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.893 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_00_00_UTC) 2023-10-21T17:18:19.947 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_01_00_UTC 2023-10-21T17:18:19.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:19.948 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_01_00_UTC) 2023-10-21T17:18:20.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_02_00_UTC 2023-10-21T17:18:20.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.009 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_02_00_UTC) 2023-10-21T17:18:20.111 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_03_00_UTC 2023-10-21T17:18:20.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.112 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_03_00_UTC) 2023-10-21T17:18:20.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_04_00_UTC 2023-10-21T17:18:20.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.179 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_04_00_UTC) 2023-10-21T17:18:20.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_05_00_UTC 2023-10-21T17:18:20.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.230 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_05_00_UTC) 2023-10-21T17:18:20.284 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_06_00_UTC 2023-10-21T17:18:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.284 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_06_00_UTC) 2023-10-21T17:18:20.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_07_00_UTC 2023-10-21T17:18:20.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.348 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_07_00_UTC) 2023-10-21T17:18:20.418 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_08_00_UTC 2023-10-21T17:18:20.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.418 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_08_00_UTC) 2023-10-21T17:18:20.489 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_09_00_UTC 2023-10-21T17:18:20.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.490 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_09_00_UTC) 2023-10-21T17:18:20.556 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_10_00_UTC 2023-10-21T17:18:20.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.557 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_10_00_UTC) 2023-10-21T17:18:20.622 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_11_00_UTC 2023-10-21T17:18:20.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.623 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_11_00_UTC) 2023-10-21T17:18:20.687 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_12_00_UTC 2023-10-21T17:18:20.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.688 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_12_00_UTC) 2023-10-21T17:18:20.751 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_13_00_UTC 2023-10-21T17:18:20.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.752 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_13_00_UTC) 2023-10-21T17:18:20.815 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_14_00_UTC 2023-10-21T17:18:20.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.816 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_14_00_UTC) 2023-10-21T17:18:20.879 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_15_00_UTC 2023-10-21T17:18:20.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.880 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_15_00_UTC) 2023-10-21T17:18:20.943 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_16_00_UTC 2023-10-21T17:18:20.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:20.943 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_16_00_UTC) 2023-10-21T17:18:21.013 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_17_00_UTC 2023-10-21T17:18:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:21.014 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_17_00_UTC) 2023-10-21T17:18:21.146 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_18_00_UTC 2023-10-21T17:18:21.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:21.164 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_18_00_UTC) 2023-10-21T17:18:21.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:21.221 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T17:18:21.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:18:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.535+0000 7f5b0283f640 1 Processor -- start 2023-10-21T17:18:21.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.535+0000 7f5b0283f640 1 -- start start 2023-10-21T17:18:21.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5afc06c3a0 0x7f5afc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06db60 0x7f5afc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5afc15aa00 0x7f5afc0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc0ca230 con 0x7f5afc15aa00 2023-10-21T17:18:21.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc15b310 con 0x7f5afc06c3a0 2023-10-21T17:18:21.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b0283f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc06d290 con 0x7f5afc06db60 2023-10-21T17:18:21.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.538+0000 7f5b00ddc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5afc15aa00 0x7f5afc0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5afc15aa00 0x7f5afc0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48240/0 (socket says 172.21.15.53:48240) 2023-10-21T17:18:21.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 -- 172.21.15.53:0/1316768740 learned_addr learned my addr 172.21.15.53:0/1316768740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:21.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5afbfff640 1 --2- 172.21.15.53:0/1316768740 >> v2:172.21.15.124:3300/0 conn(0x7f5afc06c3a0 0x7f5afc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5afb7fe640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06db60 0x7f5afc06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 -- 172.21.15.53:0/1316768740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06db60 msgr2=0x7f5afc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:21.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06db60 0x7f5afc06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 -- 172.21.15.53:0/1316768740 >> v2:172.21.15.124:3300/0 conn(0x7f5afc06c3a0 msgr2=0x7f5afc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:21.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 --2- 172.21.15.53:0/1316768740 >> v2:172.21.15.124:3300/0 conn(0x7f5afc06c3a0 0x7f5afc0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.539+0000 7f5b00ddc640 1 -- 172.21.15.53:0/1316768740 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0c9860 con 0x7f5afc15aa00 2023-10-21T17:18:21.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.541+0000 7f5b00ddc640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc15aa00 0x7f5afc0b2a90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0716b0 tx=0x7f5af402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9856a34f5e9f4b49 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:21.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.542+0000 7f5afaffd640 1 -- 172.21.15.53:0/1316768740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af402ea80 con 0x7f5afc15aa00 2023-10-21T17:18:21.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.542+0000 7f5afaffd640 1 -- 172.21.15.53:0/1316768740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af402ebe0 con 0x7f5afc15aa00 2023-10-21T17:18:21.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.542+0000 7f5afaffd640 1 -- 172.21.15.53:0/1316768740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4009600 con 0x7f5afc15aa00 2023-10-21T17:18:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 -- 172.21.15.53:0/1316768740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc15aa00 msgr2=0x7f5afc0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc15aa00 0x7f5afc0b2a90 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0716b0 tx=0x7f5af402e0e0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 -- 172.21.15.53:0/1316768740 shutdown_connections 2023-10-21T17:18:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06db60 0x7f5afc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1316768740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc15aa00 0x7f5afc0b2a90 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1316768740 >> v2:172.21.15.124:3300/0 conn(0x7f5afc06c3a0 0x7f5afc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 -- 172.21.15.53:0/1316768740 >> 172.21.15.53:0/1316768740 conn(0x7f5afc06a5d0 msgr2=0x7f5afc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:21.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 -- 172.21.15.53:0/1316768740 shutdown_connections 2023-10-21T17:18:21.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.543+0000 7f5b0283f640 1 -- 172.21.15.53:0/1316768740 wait complete. 2023-10-21T17:18:21.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.546+0000 7f5b0283f640 1 Processor -- start 2023-10-21T17:18:21.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.546+0000 7f5b0283f640 1 -- start start 2023-10-21T17:18:21.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06c3a0 0x7f5afc0c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5afc15aa00 0x7f5afc1f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc14d4a0 con 0x7f5afc06db60 2023-10-21T17:18:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1f0010 con 0x7f5afc15aa00 2023-10-21T17:18:21.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.547+0000 7f5b0283f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1f3590 con 0x7f5afc06c3a0 2023-10-21T17:18:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.548+0000 7f5afb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.548+0000 7f5afb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48248/0 (socket says 172.21.15.53:48248) 2023-10-21T17:18:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.548+0000 7f5afb7fe640 1 -- 172.21.15.53:0/1572794258 learned_addr learned my addr 172.21.15.53:0/1572794258 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.548+0000 7f5b00ddc640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5afc15aa00 0x7f5afc1f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 -- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06c3a0 msgr2=0x7f5afc0c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06c3a0 0x7f5afc0c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 -- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5afc15aa00 msgr2=0x7f5afc1f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:21.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5afc15aa00 0x7f5afc1f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:21.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 -- 172.21.15.53:0/1572794258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af400bd40 con 0x7f5afc06db60 2023-10-21T17:18:21.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.549+0000 7f5afb7fe640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5af000d800 tx=0x7f5af000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.550+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000b6c0 con 0x7f5afc06db60 2023-10-21T17:18:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.550+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af000b820 con 0x7f5afc06db60 2023-10-21T17:18:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.550+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000bb90 con 0x7f5afc06db60 2023-10-21T17:18:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.550+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc1f3850 con 0x7f5afc06db60 2023-10-21T17:18:21.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.551+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc1f3e10 con 0x7f5afc06db60 2023-10-21T17:18:21.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.553+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc04e160 con 0x7f5afc06db60 2023-10-21T17:18:21.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.554+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 82) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f5af000bcf0 con 0x7f5afc06db60 2023-10-21T17:18:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.556+0000 7f5af8ff9640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 0x7f5ae806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:21.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.556+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f5af008a000 con 0x7f5afc06db60 2023-10-21T17:18:21.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.557+0000 7f5afbfff640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 0x7f5ae806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:21.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.561+0000 7f5afbfff640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 0x7f5ae806d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009870 tx=0x7f5aec008040 comp rx=0 tx=0).ready entity=mgr.7756 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:21.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.564+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5af00bcca0 con 0x7f5afc06db60 2023-10-21T17:18:21.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:21.835+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f5afc159f50 con 0x7f5afc06db60 2023-10-21T17:18:22.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.113+0000 7f5af8ff9640 1 -- 172.21.15.53:0/1572794258 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v83) v1 ==== 93+0+0 (secure 0 0 0) 0x7f5af0054cd0 con 0x7f5afc06db60 2023-10-21T17:18:22.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.122+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 msgr2=0x7f5ae806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.122+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 0x7f5ae806d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009870 tx=0x7f5aec008040 comp rx=0 tx=0).stop 2023-10-21T17:18:22.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.122+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 msgr2=0x7f5afc0c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.122+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5af000d800 tx=0x7f5af000dcc0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 shutdown_connections 2023-10-21T17:18:22.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5afc15aa00 0x7f5afc1f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f5ae806b0b0 0x7f5ae806d570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5afc06db60 0x7f5afc0c3b20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 --2- 172.21.15.53:0/1572794258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5afc06c3a0 0x7f5afc0c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.123+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 >> 172.21.15.53:0/1572794258 conn(0x7f5afc06a5d0 msgr2=0x7f5afc0ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.125+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 shutdown_connections 2023-10-21T17:18:22.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.125+0000 7f5b0283f640 1 -- 172.21.15.53:0/1572794258 wait complete. 2023-10-21T17:18:22.162 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:18:22.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:18:22.276 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:18:22.390 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:18:22.391 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:22.391 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:22.434 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:22.434 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:22.530 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:22.755 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:22.755+0000 7fb2aec41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:18:22.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.756+0000 7f4897716640 1 Processor -- start 2023-10-21T17:18:22.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.757+0000 7f4897716640 1 -- start start 2023-10-21T17:18:22.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4897716640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4890069d50 0x7f489006a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4897716640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 0x7f48900b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4897716640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900ce550 con 0x7f48900cbce0 2023-10-21T17:18:22.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4897716640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48901609d0 con 0x7f4890069d50 2023-10-21T17:18:22.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4897716640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48900cdcf0 con 0x7f48900cb880 2023-10-21T17:18:22.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f48954b2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4890069d50 0x7f489006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:22.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4895cb3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f48900cbce0 0x7f4890160e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40918/0 (socket says 172.21.15.53:40918) 2023-10-21T17:18:22.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4895cb3640 1 -- 172.21.15.53:0/441478627 learned_addr learned my addr 172.21.15.53:0/441478627 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:22.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f48954b2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4890069d50 0x7f489006a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48276/0 (socket says 172.21.15.53:48276) 2023-10-21T17:18:22.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.758+0000 7f4894cb1640 1 --2- 172.21.15.53:0/441478627 >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 0x7f48900b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:22.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f4887fff640 1 -- 172.21.15.53:0/441478627 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051965337 0 0) 0x7f48900ce550 con 0x7f48900cbce0 2023-10-21T17:18:22.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f4887fff640 1 -- 172.21.15.53:0/441478627 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c003520 con 0x7f48900cbce0 2023-10-21T17:18:22.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f48954b2640 1 -- 172.21.15.53:0/441478627 >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 msgr2=0x7f48900b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:22.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f48954b2640 1 --2- 172.21.15.53:0/441478627 >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 0x7f48900b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f48954b2640 1 -- 172.21.15.53:0/441478627 >> v1:172.21.15.53:6789/0 conn(0x7f48900cbce0 legacy=0x7f4890160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f48954b2640 1 -- 172.21.15.53:0/441478627 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489006a670 con 0x7f4890069d50 2023-10-21T17:18:22.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.759+0000 7f4894cb1640 1 --2- 172.21.15.53:0/441478627 >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 0x7f48900b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:18:22.764 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:22.762+0000 7f08c5353040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:18:22.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.760+0000 7f48954b2640 1 --2- 172.21.15.53:0/441478627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f489006a130 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f489006e260 tx=0x7f488c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646760a56c580ded server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:22.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.760+0000 7f4887fff640 1 -- 172.21.15.53:0/441478627 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c031830 con 0x7f4890069d50 2023-10-21T17:18:22.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.760+0000 7f4887fff640 1 -- 172.21.15.53:0/441478627 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c031990 con 0x7f4890069d50 2023-10-21T17:18:22.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.760+0000 7f4887fff640 1 -- 172.21.15.53:0/441478627 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c031d00 con 0x7f4890069d50 2023-10-21T17:18:22.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 -- 172.21.15.53:0/441478627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 msgr2=0x7f489006a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 --2- 172.21.15.53:0/441478627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f489006a130 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f489006e260 tx=0x7f488c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 -- 172.21.15.53:0/441478627 shutdown_connections 2023-10-21T17:18:22.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 --2- 172.21.15.53:0/441478627 >> v2:172.21.15.124:3300/0 conn(0x7f48900cb880 0x7f48900b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 --2- 172.21.15.53:0/441478627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f489006a130 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 -- 172.21.15.53:0/441478627 >> 172.21.15.53:0/441478627 conn(0x7f489006d450 msgr2=0x7f489006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:22.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 -- 172.21.15.53:0/441478627 shutdown_connections 2023-10-21T17:18:22.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.761+0000 7f4897716640 1 -- 172.21.15.53:0/441478627 wait complete. 2023-10-21T17:18:22.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.764+0000 7f4897716640 1 Processor -- start 2023-10-21T17:18:22.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.764+0000 7f4897716640 1 -- start start 2023-10-21T17:18:22.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.765+0000 7f4897716640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.765+0000 7f4897716640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48900cb880 0x7f489015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4897716640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48900cbce0 0x7f48900ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4897716640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489015ec90 con 0x7f4890069d50 2023-10-21T17:18:22.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4897716640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489015dbf0 con 0x7f48900cbce0 2023-10-21T17:18:22.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4897716640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489015ded0 con 0x7f48900cb880 2023-10-21T17:18:22.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:22.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48290/0 (socket says 172.21.15.53:48290) 2023-10-21T17:18:22.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 -- 172.21.15.53:0/1423618370 learned_addr learned my addr 172.21.15.53:0/1423618370 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:22.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4894cb1640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48900cb880 0x7f489015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:22.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f4895cb3640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48900cbce0 0x7f48900ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:22.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48900cb880 msgr2=0x7f489015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48900cb880 0x7f489015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48900cbce0 msgr2=0x7f48900ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48900cbce0 0x7f48900ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.766+0000 7f48954b2640 1 -- 172.21.15.53:0/1423618370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880009400 con 0x7f4890069d50 2023-10-21T17:18:22.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f48954b2640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f488c02b910 tx=0x7f488c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:22.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c03a830 con 0x7f4890069d50 2023-10-21T17:18:22.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c03a990 con 0x7f4890069d50 2023-10-21T17:18:22.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c03ad20 con 0x7f4890069d50 2023-10-21T17:18:22.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c009400 con 0x7f4890069d50 2023-10-21T17:18:22.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.767+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489015e490 con 0x7f4890069d50 2023-10-21T17:18:22.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.768+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890067490 con 0x7f4890069d50 2023-10-21T17:18:22.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.770+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f488c04d440 con 0x7f4890069d50 2023-10-21T17:18:22.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.771+0000 7f4885ffb640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 0x7f487406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:22.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.771+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f488c0abc90 con 0x7f4890069d50 2023-10-21T17:18:22.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.771+0000 7f4894cb1640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 msgr2=0x7f487406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:22.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.771+0000 7f4894cb1640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 0x7f487406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:22.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.775+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f488c076740 con 0x7f4890069d50 2023-10-21T17:18:22.823 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:22.823+0000 7f08c5353040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:18:22.824 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:22.824+0000 7fb2aec41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:18:22.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.972+0000 7f4894cb1640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 msgr2=0x7f487406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:22.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.972+0000 7f4894cb1640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 0x7f487406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:22.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.984+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48900b5810 con 0x7f4890069d50 2023-10-21T17:18:22.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.986+0000 7f4885ffb640 1 -- 172.21.15.53:0/1423618370 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17579 (secure 0 0 0) 0x7f488c035090 con 0x7f4890069d50 2023-10-21T17:18:22.988 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:22.988 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":169,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:22.791622+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":169,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3657298397":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3304201904":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3058690420":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3358065251":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/4258504518":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/2638777920":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6833/870299393":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6832/870299393":"2023-10-22T16:22:28.659493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":18,"length":39},{"begin":74,"length":39}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":55,"length":2},{"begin":111,"length":2}]}],"new_purged_snaps":[{"pool":20,"snaps":[{"begin":3,"length":15},{"begin":59,"length":15}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 msgr2=0x7f487406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 0x7f487406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 msgr2=0x7f48901475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f488c02b910 tx=0x7f488c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 shutdown_connections 2023-10-21T17:18:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48900cbce0 0x7f48900ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48900cb880 0x7f489015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f487406af90 0x7f487406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 --2- 172.21.15.53:0/1423618370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4890069d50 0x7f48901475f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:22.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.988+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 >> 172.21.15.53:0/1423618370 conn(0x7f489006d450 msgr2=0x7f48900cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:22.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.989+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 shutdown_connections 2023-10-21T17:18:22.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:22.989+0000 7f4897716640 1 -- 172.21.15.53:0/1423618370 wait complete. 2023-10-21T17:18:23.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:22:11.619571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:22:12.967359+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 169, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:18:23.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:18:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.214+0000 7f7902e51640 1 Processor -- start 2023-10-21T17:18:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.214+0000 7f7902e51640 1 -- start start 2023-10-21T17:18:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.215+0000 7f7902e51640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78fc15aad0 0x7f78fc0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.215+0000 7f7902e51640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.215+0000 7f7902e51640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc15bf30 con 0x7f78fc06cd00 2023-10-21T17:18:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.215+0000 7f7902e51640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc15b320 con 0x7f78fc15aad0 2023-10-21T17:18:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.215+0000 7f7902e51640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc15b600 con 0x7f78fc0c82b0 2023-10-21T17:18:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78fc15aad0 0x7f78fc0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78fc15aad0 0x7f78fc0b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48298/0 (socket says 172.21.15.53:48298) 2023-10-21T17:18:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f79013ee640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f78fc06cd00 0x7f78fc070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40922/0 (socket says 172.21.15.53:40922) 2023-10-21T17:18:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 -- 172.21.15.53:0/1467435432 learned_addr learned my addr 172.21.15.53:0/1467435432 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f78f3fff640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 -- 172.21.15.53:0/1467435432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 msgr2=0x7f78fc0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 -- 172.21.15.53:0/1467435432 >> v1:172.21.15.53:6789/0 conn(0x7f78fc06cd00 legacy=0x7f78fc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f7900bed640 1 -- 172.21.15.53:0/1467435432 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc15b8e0 con 0x7f78fc15aad0 2023-10-21T17:18:23.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.216+0000 7f78f3fff640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:23.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.217+0000 7f7900bed640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc15aad0 0x7f78fc0b1750 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009e50 tx=0x7f78f802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5848f3b4dd746d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:23.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f78f37fe640 1 -- 172.21.15.53:0/1467435432 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f802cb00 con 0x7f78fc15aad0 2023-10-21T17:18:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f78f37fe640 1 -- 172.21.15.53:0/1467435432 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f802cc60 con 0x7f78fc15aad0 2023-10-21T17:18:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f78f37fe640 1 -- 172.21.15.53:0/1467435432 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80343f0 con 0x7f78fc15aad0 2023-10-21T17:18:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 -- 172.21.15.53:0/1467435432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc15aad0 msgr2=0x7f78fc0b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc15aad0 0x7f78fc0b1750 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009e50 tx=0x7f78f802bf60 comp rx=0 tx=0).stop 2023-10-21T17:18:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 -- 172.21.15.53:0/1467435432 shutdown_connections 2023-10-21T17:18:23.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 --2- 172.21.15.53:0/1467435432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc15aad0 0x7f78fc0b1750 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 -- 172.21.15.53:0/1467435432 >> 172.21.15.53:0/1467435432 conn(0x7f78fc069d50 msgr2=0x7f78fc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:23.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 -- 172.21.15.53:0/1467435432 shutdown_connections 2023-10-21T17:18:23.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.218+0000 7f7902e51640 1 -- 172.21.15.53:0/1467435432 wait complete. 2023-10-21T17:18:23.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.220+0000 7f7902e51640 1 Processor -- start 2023-10-21T17:18:23.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.220+0000 7f7902e51640 1 -- start start 2023-10-21T17:18:23.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc1f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78fc15aad0 0x7f78fc1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1f40b0 con 0x7f78fc06cd00 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1f4390 con 0x7f78fc15aad0 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7902e51640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1f02a0 con 0x7f78fc0c82b0 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7900bed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7900bed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48302/0 (socket says 172.21.15.53:48302) 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.221+0000 7f7900bed640 1 -- 172.21.15.53:0/4001053754 learned_addr learned my addr 172.21.15.53:0/4001053754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:23.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f78f3fff640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc1f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f79013ee640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78fc15aad0 0x7f78fc1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 msgr2=0x7f78fc1f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc1f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78fc15aad0 msgr2=0x7f78fc1f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78fc15aad0 0x7f78fc1f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 -- 172.21.15.53:0/4001053754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8009b30 con 0x7f78fc06cd00 2023-10-21T17:18:23.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f78f3fff640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc1f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:23.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.222+0000 7f7900bed640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f80 tx=0x7f78f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:23.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.223+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f803e450 con 0x7f78fc06cd00 2023-10-21T17:18:23.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.223+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f803e5f0 con 0x7f78fc06cd00 2023-10-21T17:18:23.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.223+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f803e9a0 con 0x7f78fc06cd00 2023-10-21T17:18:23.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.223+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc1f0580 con 0x7f78fc06cd00 2023-10-21T17:18:23.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.223+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0cd380 con 0x7f78fc06cd00 2023-10-21T17:18:23.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.224+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc1f07a0 con 0x7f78fc06cd00 2023-10-21T17:18:23.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.225+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f78f803eb00 con 0x7f78fc06cd00 2023-10-21T17:18:23.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.226+0000 7f78f17fa640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 0x7f78e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.226+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f78f80aca70 con 0x7f78fc06cd00 2023-10-21T17:18:23.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.226+0000 7f78f3fff640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 msgr2=0x7f78e806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:23.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.226+0000 7f78f3fff640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 0x7f78e806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:23.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.235+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78f80777d0 con 0x7f78fc06cd00 2023-10-21T17:18:23.278 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.278+0000 7f08c5353040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:18:23.321 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.320+0000 7fb2aec41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:18:23.331 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.331+0000 7f08c5353040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:18:23.375 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.374+0000 7fb2aec41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:18:23.381 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.381+0000 7f08c5353040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:18:23.427 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.426+0000 7fb2aec41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:18:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.427+0000 7f78f3fff640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 msgr2=0x7f78e806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.427+0000 7f78f3fff640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 0x7f78e806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.435+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78fc04e160 con 0x7f78fc06cd00 2023-10-21T17:18:23.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.435+0000 7f78f17fa640 1 -- 172.21.15.53:0/4001053754 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78f803edb0 con 0x7f78fc06cd00 2023-10-21T17:18:23.437 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:18:23.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 msgr2=0x7f78e806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 0x7f78e806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 msgr2=0x7f78fc1f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f80 tx=0x7f78f8004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 shutdown_connections 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78fc15aad0 0x7f78fc1f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78fc0c82b0 0x7f78fc1f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f78e806af90 0x7f78e806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 --2- 172.21.15.53:0/4001053754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78fc06cd00 0x7f78fc1f0aa0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 >> 172.21.15.53:0/4001053754 conn(0x7f78fc069d50 msgr2=0x7f78fc070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:23.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 shutdown_connections 2023-10-21T17:18:23.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.438+0000 7f7902e51640 1 -- 172.21.15.53:0/4001053754 wait complete. 2023-10-21T17:18:23.442 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.441+0000 7f08c5353040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:18:23.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:18:23.497 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.497+0000 7fb2aec41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:18:23.499 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.498+0000 7f08c5353040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:18:23.551 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.550+0000 7fb2aec41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:18:23.623 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.623+0000 7f08c5353040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:18:23.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.663+0000 7f6c6877c640 1 Processor -- start 2023-10-21T17:18:23.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.663+0000 7f6c6877c640 1 -- start start 2023-10-21T17:18:23.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.664+0000 7f6c6877c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6c6015cce0 0x7f6c600cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c6877c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015d0c0 0x7f6c600b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c6877c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c600b30d0 0x7f6c600b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c6877c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6006ce20 con 0x7f6c600b30d0 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c6877c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c600cf850 con 0x7f6c6015cce0 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c6877c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006c060 con 0x7f6c6015d0c0 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c600b30d0 0x7f6c600b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c600b30d0 0x7f6c600b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48318/0 (socket says 172.21.15.53:48318) 2023-10-21T17:18:23.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 -- 172.21.15.53:0/2490845824 learned_addr learned my addr 172.21.15.53:0/2490845824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:23.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66518640 1 --2- 172.21.15.53:0/2490845824 >> v2:172.21.15.124:3300/0 conn(0x7f6c6015cce0 0x7f6c600cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c65d17640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015d0c0 0x7f6c600b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 -- 172.21.15.53:0/2490845824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015d0c0 msgr2=0x7f6c600b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:23.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015d0c0 0x7f6c600b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 -- 172.21.15.53:0/2490845824 >> v2:172.21.15.124:3300/0 conn(0x7f6c6015cce0 msgr2=0x7f6c600cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:23.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 --2- 172.21.15.53:0/2490845824 >> v2:172.21.15.124:3300/0 conn(0x7f6c6015cce0 0x7f6c600cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.665+0000 7f6c66d19640 1 -- 172.21.15.53:0/2490845824 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6006c340 con 0x7f6c600b30d0 2023-10-21T17:18:23.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.666+0000 7f6c66d19640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c600b30d0 0x7f6c600b2a70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f6c600ce110 tx=0x7f6c5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3139eac95de3b08a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:23.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.667+0000 7f6c65516640 1 -- 172.21.15.53:0/2490845824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5802ca70 con 0x7f6c600b30d0 2023-10-21T17:18:23.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.667+0000 7f6c65516640 1 -- 172.21.15.53:0/2490845824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5802cbd0 con 0x7f6c600b30d0 2023-10-21T17:18:23.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.667+0000 7f6c65516640 1 -- 172.21.15.53:0/2490845824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c580343f0 con 0x7f6c600b30d0 2023-10-21T17:18:23.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.667+0000 7f6c6877c640 1 -- 172.21.15.53:0/2490845824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c600b30d0 msgr2=0x7f6c600b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.667+0000 7f6c6877c640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c600b30d0 0x7f6c600b2a70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f6c600ce110 tx=0x7f6c5802bed0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 -- 172.21.15.53:0/2490845824 shutdown_connections 2023-10-21T17:18:23.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015d0c0 0x7f6c600b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 --2- 172.21.15.53:0/2490845824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c600b30d0 0x7f6c600b2a70 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 --2- 172.21.15.53:0/2490845824 >> v2:172.21.15.124:3300/0 conn(0x7f6c6015cce0 0x7f6c600cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 -- 172.21.15.53:0/2490845824 >> 172.21.15.53:0/2490845824 conn(0x7f6c6006e650 msgr2=0x7f6c6006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:23.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 -- 172.21.15.53:0/2490845824 shutdown_connections 2023-10-21T17:18:23.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.668+0000 7f6c6877c640 1 -- 172.21.15.53:0/2490845824 wait complete. 2023-10-21T17:18:23.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.670+0000 7f6c6877c640 1 Processor -- start 2023-10-21T17:18:23.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.670+0000 7f6c6877c640 1 -- start start 2023-10-21T17:18:23.674 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.671+0000 7fb2aec41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:18:23.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 0x7f6c6014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015cce0 0x7f6c600c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600c62d0 con 0x7f6c60150810 2023-10-21T17:18:23.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600c5750 con 0x7f6c600b30d0 2023-10-21T17:18:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.671+0000 7f6c6877c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600c5a30 con 0x7f6c6015cce0 2023-10-21T17:18:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48328/0 (socket says 172.21.15.53:48328) 2023-10-21T17:18:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 -- 172.21.15.53:0/1773005041 learned_addr learned my addr 172.21.15.53:0/1773005041 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:23.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66518640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 0x7f6c6014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c65d17640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015cce0 0x7f6c600c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 -- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015cce0 msgr2=0x7f6c600c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015cce0 0x7f6c600c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 -- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 msgr2=0x7f6c6014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 0x7f6c6014ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 -- 172.21.15.53:0/1773005041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58009b30 con 0x7f6c60150810 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66518640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 0x7f6c6014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.672+0000 7f6c66d19640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f6c58036040 tx=0x7f6c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.673+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58009510 con 0x7f6c60150810 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.673+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5803e450 con 0x7f6c60150810 2023-10-21T17:18:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.673+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5803e800 con 0x7f6c60150810 2023-10-21T17:18:23.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.673+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c600c5d10 con 0x7f6c60150810 2023-10-21T17:18:23.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.673+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c60150510 con 0x7f6c60150810 2023-10-21T17:18:23.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.674+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c60160710 con 0x7f6c60150810 2023-10-21T17:18:23.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.675+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f6c5803e960 con 0x7f6c60150810 2023-10-21T17:18:23.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.677+0000 7f6c56ffd640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 0x7f6c4406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:23.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.677+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f6c580ac940 con 0x7f6c60150810 2023-10-21T17:18:23.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.677+0000 7f6c66518640 1 -- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 msgr2=0x7f6c4406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:23.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.677+0000 7f6c66518640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 0x7f6c4406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.683+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c580774d0 con 0x7f6c60150810 2023-10-21T17:18:23.686 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.686+0000 7f08c5353040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:23.736 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.735+0000 7f08c5353040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:18:23.739 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.739+0000 7fb2aec41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:23.792 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.791+0000 7fb2aec41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:18:23.793 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.792+0000 7f08c5353040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:18:23.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:23.843+0000 7f08c5353040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:18:23.852 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.851+0000 7fb2aec41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:18:23.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.873+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c6004e160 con 0x7f6c60150810 2023-10-21T17:18:23.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.874+0000 7f6c56ffd640 1 -- 172.21.15.53:0/1773005041 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c5807aaf0 con 0x7f6c60150810 2023-10-21T17:18:23.875 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:18:23.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.876+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 msgr2=0x7f6c4406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:23.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.876+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 0x7f6c4406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.876+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 msgr2=0x7f6c6015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:23.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.876+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f6c58036040 tx=0x7f6c58004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:23.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 shutdown_connections 2023-10-21T17:18:23.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f6c4406afc0 0x7f6c4406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c60150810 0x7f6c6015e590 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c6015cce0 0x7f6c600c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 --2- 172.21.15.53:0/1773005041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c600b30d0 0x7f6c6014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:23.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 >> 172.21.15.53:0/1773005041 conn(0x7f6c6006e650 msgr2=0x7f6c600cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:23.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 shutdown_connections 2023-10-21T17:18:23.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:23.877+0000 7f6c6877c640 1 -- 172.21.15.53:0/1773005041 wait complete. 2023-10-21T17:18:23.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:18:23.907 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:23.906+0000 7fb2aec41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:18:24.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.122+0000 7efcd5621640 1 Processor -- start 2023-10-21T17:18:24.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.123+0000 7efcd5621640 1 -- start start 2023-10-21T17:18:24.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efcd5621640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcd015cf30 0x7efcd00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efcd5621640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd00cd520 con 0x7efcd00cf2b0 2023-10-21T17:18:24.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efcd5621640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd00d1910 con 0x7efcd00cf690 2023-10-21T17:18:24.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efcd5621640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd00ce080 con 0x7efcd015cf30 2023-10-21T17:18:24.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efccffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efcd00cf2b0 0x7efcd00d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41578/0 (socket says 172.21.15.53:41578) 2023-10-21T17:18:24.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efccffff640 1 -- 172.21.15.53:0/2107675480 learned_addr learned my addr 172.21.15.53:0/2107675480 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:24.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.124+0000 7efcd4e20640 1 --2- 172.21.15.53:0/2107675480 >> v2:172.21.15.53:3300/0 conn(0x7efcd015cf30 0x7efcd00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.125+0000 7efcceffd640 1 -- 172.21.15.53:0/2107675480 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500344706 0 0) 0x7efcd00cd520 con 0x7efcd00cf2b0 2023-10-21T17:18:24.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.125+0000 7efcceffd640 1 -- 172.21.15.53:0/2107675480 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb8003520 con 0x7efcd00cf2b0 2023-10-21T17:18:24.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.125+0000 7efcd4e20640 1 -- 172.21.15.53:0/2107675480 >> v1:172.21.15.124:6789/0 conn(0x7efcd00cf690 legacy=0x7efcd015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.125+0000 7efcd4e20640 1 -- 172.21.15.53:0/2107675480 >> v1:172.21.15.53:6789/0 conn(0x7efcd00cf2b0 legacy=0x7efcd00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.125+0000 7efcd4e20640 1 -- 172.21.15.53:0/2107675480 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00cd750 con 0x7efcd015cf30 2023-10-21T17:18:24.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.126+0000 7efcd4e20640 1 --2- 172.21.15.53:0/2107675480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd00d34d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009d90 tx=0x7efcc802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a877b88ff8c841 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:24.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.126+0000 7efcceffd640 1 -- 172.21.15.53:0/2107675480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc802cc60 con 0x7efcd015cf30 2023-10-21T17:18:24.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.126+0000 7efcceffd640 1 -- 172.21.15.53:0/2107675480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc802cdc0 con 0x7efcd015cf30 2023-10-21T17:18:24.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.126+0000 7efcceffd640 1 -- 172.21.15.53:0/2107675480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8034640 con 0x7efcd015cf30 2023-10-21T17:18:24.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 -- 172.21.15.53:0/2107675480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 msgr2=0x7efcd00d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 --2- 172.21.15.53:0/2107675480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd00d34d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009d90 tx=0x7efcc802beb0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 -- 172.21.15.53:0/2107675480 shutdown_connections 2023-10-21T17:18:24.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 --2- 172.21.15.53:0/2107675480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd00d34d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 -- 172.21.15.53:0/2107675480 >> 172.21.15.53:0/2107675480 conn(0x7efcd009bc40 msgr2=0x7efcd00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:24.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 -- 172.21.15.53:0/2107675480 shutdown_connections 2023-10-21T17:18:24.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.127+0000 7efcd5621640 1 -- 172.21.15.53:0/2107675480 wait complete. 2023-10-21T17:18:24.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.129+0000 7efcd5621640 1 Processor -- start 2023-10-21T17:18:24.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.129+0000 7efcd5621640 1 -- start start 2023-10-21T17:18:24.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 0x7efcd00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcd00cf690 0x7efcd00b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00b86e0 con 0x7efcd015cf30 2023-10-21T17:18:24.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00b7b60 con 0x7efcd00cf2b0 2023-10-21T17:18:24.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd5621640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00b7e40 con 0x7efcd00cf690 2023-10-21T17:18:24.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd4e20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd4e20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41378/0 (socket says 172.21.15.53:41378) 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efcd4e20640 1 -- 172.21.15.53:0/1367433133 learned_addr learned my addr 172.21.15.53:0/1367433133 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efccffff640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 0x7efcd00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.130+0000 7efccf7fe640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcd00cf690 0x7efcd00b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 -- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcd00cf690 msgr2=0x7efcd00b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcd00cf690 0x7efcd00b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 -- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 msgr2=0x7efcd00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 0x7efcd00b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 -- 172.21.15.53:0/1367433133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009a70 con 0x7efcd015cf30 2023-10-21T17:18:24.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efccffff640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 0x7efcd00b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:24.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd4e20640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009d90 tx=0x7efcc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:24.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8003b90 con 0x7efcd015cf30 2023-10-21T17:18:24.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8003d10 con 0x7efcd015cf30 2023-10-21T17:18:24.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004270 con 0x7efcd015cf30 2023-10-21T17:18:24.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.131+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd00b8120 con 0x7efcd015cf30 2023-10-21T17:18:24.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.132+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0149a60 con 0x7efcd015cf30 2023-10-21T17:18:24.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.133+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd015da20 con 0x7efcd015cf30 2023-10-21T17:18:24.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.134+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7efcc803b050 con 0x7efcd015cf30 2023-10-21T17:18:24.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.135+0000 7efcccff9640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 0x7efcbc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.135+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7efcc80ac5d0 con 0x7efcd015cf30 2023-10-21T17:18:24.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.136+0000 7efccffff640 1 -- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 msgr2=0x7efcbc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:24.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.136+0000 7efccffff640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 0x7efcbc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:24.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.142+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcc8077270 con 0x7efcd015cf30 2023-10-21T17:18:24.238 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.238+0000 7f08c5353040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:18:24.316 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.315+0000 7fb2aec41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:18:24.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.333+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcd015da20 con 0x7efcd015cf30 2023-10-21T17:18:24.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.334+0000 7efcccff9640 1 -- 172.21.15.53:0/1367433133 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcc8039020 con 0x7efcd015cf30 2023-10-21T17:18:24.335 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:18:24.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.336+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 msgr2=0x7efcbc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:24.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.336+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 0x7efcbc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.336+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 msgr2=0x7efcd0145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.336+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009d90 tx=0x7efcc8004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:24.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 shutdown_connections 2023-10-21T17:18:24.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efcbc06af90 0x7efcbc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcd015cf30 0x7efcd0145a90 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcd00cf690 0x7efcd00b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 --2- 172.21.15.53:0/1367433133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcd00cf2b0 0x7efcd00b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 >> 172.21.15.53:0/1367433133 conn(0x7efcd009bc40 msgr2=0x7efcd00af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:24.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 shutdown_connections 2023-10-21T17:18:24.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.337+0000 7efcd5621640 1 -- 172.21.15.53:0/1367433133 wait complete. 2023-10-21T17:18:24.356 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:18:24.400 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.399+0000 7f08c5353040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:18:24.487 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.486+0000 7fb2aec41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:18:24.509 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.509+0000 7f08c5353040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:18:24.559 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.559+0000 7f08c5353040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:18:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.565+0000 7efebca6d640 1 Processor -- start 2023-10-21T17:18:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.565+0000 7efebca6d640 1 -- start start 2023-10-21T17:18:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.567+0000 7efebca6d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 0x7efeb80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.567+0000 7efebca6d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efeb80ce510 0x7efeb815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.567+0000 7efebca6d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb80cd520 con 0x7efeb80cf690 2023-10-21T17:18:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.567+0000 7efebca6d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb80d1910 con 0x7efeb80ce510 2023-10-21T17:18:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.567+0000 7efebca6d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80ce080 con 0x7efeb80cf2b0 2023-10-21T17:18:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efeb80ce510 0x7efeb815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb77fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 0x7efeb80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeaffff640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7efeb80cf690 0x7efeb80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:54258/0 (socket says 172.21.15.53:54258) 2023-10-21T17:18:24.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeaffff640 1 -- 172.21.15.53:0/2149497026 learned_addr learned my addr 172.21.15.53:0/2149497026 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb77fe640 1 -- 172.21.15.53:0/2149497026 >> v2:172.21.15.124:3300/0 conn(0x7efeb80ce510 msgr2=0x7efeb815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb77fe640 1 --2- 172.21.15.53:0/2149497026 >> v2:172.21.15.124:3300/0 conn(0x7efeb80ce510 0x7efeb815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb77fe640 1 -- 172.21.15.53:0/2149497026 >> v1:172.21.15.124:6789/0 conn(0x7efeb80cf690 legacy=0x7efeb80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.568+0000 7efeb77fe640 1 -- 172.21.15.53:0/2149497026 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb80cd750 con 0x7efeb80cf2b0 2023-10-21T17:18:24.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.570+0000 7efeb77fe640 1 --2- 172.21.15.53:0/2149497026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 0x7efeb80d34d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efea80097a0 tx=0x7efea802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc97bf2c2aeaf97 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:24.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.570+0000 7efeb6ffd640 1 -- 172.21.15.53:0/2149497026 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8031780 con 0x7efeb80cf2b0 2023-10-21T17:18:24.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.570+0000 7efeb6ffd640 1 -- 172.21.15.53:0/2149497026 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea80318e0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.570+0000 7efeb6ffd640 1 -- 172.21.15.53:0/2149497026 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8031c70 con 0x7efeb80cf2b0 2023-10-21T17:18:24.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 -- 172.21.15.53:0/2149497026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 msgr2=0x7efeb80d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 --2- 172.21.15.53:0/2149497026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 0x7efeb80d34d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efea80097a0 tx=0x7efea802b820 comp rx=0 tx=0).stop 2023-10-21T17:18:24.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 -- 172.21.15.53:0/2149497026 shutdown_connections 2023-10-21T17:18:24.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 --2- 172.21.15.53:0/2149497026 >> v2:172.21.15.124:3300/0 conn(0x7efeb80ce510 0x7efeb815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 --2- 172.21.15.53:0/2149497026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf2b0 0x7efeb80d34d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 -- 172.21.15.53:0/2149497026 >> 172.21.15.53:0/2149497026 conn(0x7efeb809bc40 msgr2=0x7efeb80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 -- 172.21.15.53:0/2149497026 shutdown_connections 2023-10-21T17:18:24.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.571+0000 7efebca6d640 1 -- 172.21.15.53:0/2149497026 wait complete. 2023-10-21T17:18:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.573+0000 7efebca6d640 1 Processor -- start 2023-10-21T17:18:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.573+0000 7efebca6d640 1 -- start start 2023-10-21T17:18:24.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efeb80ce510 0x7efeb815f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 0x7efeb815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b4520 con 0x7efeb80cf2b0 2023-10-21T17:18:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b4800 con 0x7efeb80ce510 2023-10-21T17:18:24.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efebca6d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b4ae0 con 0x7efeb80cf690 2023-10-21T17:18:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efeaffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efeaffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41388/0 (socket says 172.21.15.53:41388) 2023-10-21T17:18:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efeaffff640 1 -- 172.21.15.53:0/1177379549 learned_addr learned my addr 172.21.15.53:0/1177379549 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efeb77fe640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efeb80ce510 0x7efeb815f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.574+0000 7efeb7fff640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 0x7efeb815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 msgr2=0x7efeb815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 0x7efeb815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efeb80ce510 msgr2=0x7efeb815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efeb80ce510 0x7efeb815f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 -- 172.21.15.53:0/1177379549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009400 con 0x7efeb80cf2b0 2023-10-21T17:18:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeb7fff640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 0x7efeb815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeaffff640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7efea000c480 tx=0x7efea000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0005ce0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0005e40 con 0x7efeb80cf2b0 2023-10-21T17:18:24.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00072d0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb80b4dc0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.575+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80c9ba0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.577+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80b52e0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.577+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7efea00074f0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.578+0000 7efeb4ff9640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 0x7efe9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:24.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.578+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7efea0088c10 con 0x7efeb80cf2b0 2023-10-21T17:18:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.578+0000 7efeb77fe640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 msgr2=0x7efe9806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.579+0000 7efeb77fe640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 0x7efe9806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:24.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.583+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efeb80b52e0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.603 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.603+0000 7fb2aec41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:18:24.657 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.656+0000 7fb2aec41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:18:24.668 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.668+0000 7f08c5353040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:24.771 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.770+0000 7fb2aec41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:24.776 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.776+0000 7f08c5353040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:18:24.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.780+0000 7efeb77fe640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 msgr2=0x7efe9806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:24.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.780+0000 7efeb77fe640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 0x7efe9806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:24.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.786+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efeb80b52e0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.787+0000 7efeb4ff9640 1 -- 172.21.15.53:0/1177379549 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v169) v1 ==== 80+0+476 (secure 0 0 0) 0x7efeb80b52e0 con 0x7efeb80cf2b0 2023-10-21T17:18:24.788 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3657298397 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.788 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3304201904 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.788 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3058690420 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.789 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3358065251 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.789 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4258504518 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.789 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2638777920 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.789 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/870299393 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.789 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/870299393 2023-10-22T16:22:28.659493+0000 2023-10-21T17:18:24.790 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T17:18:24.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 msgr2=0x7efe9806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 0x7efe9806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 msgr2=0x7efeb80cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7efea000c480 tx=0x7efea000c940 comp rx=0 tx=0).stop 2023-10-21T17:18:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 shutdown_connections 2023-10-21T17:18:24.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efeb80cf690 0x7efeb815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7efe9806af90 0x7efe9806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efeb80cf2b0 0x7efeb80cbf00 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 --2- 172.21.15.53:0/1177379549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efeb80ce510 0x7efeb815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:24.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 >> 172.21.15.53:0/1177379549 conn(0x7efeb809bc40 msgr2=0x7efeb815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:24.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.790+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 shutdown_connections 2023-10-21T17:18:24.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:24.791+0000 7efebca6d640 1 -- 172.21.15.53:0/1177379549 wait complete. 2023-10-21T17:18:24.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:18:24.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:24.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:18:24.888 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:24.887+0000 7fb2aec41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:18:24.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:18:24.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:24.904 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:18:24.934 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.933+0000 7f08c5353040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:24.954 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:18:24.955 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:24.984 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:24.984+0000 7f08c5353040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:25.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:18:25.009 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:25.009 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:25.061 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:25.060+0000 7fb2aec41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:25.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:18:25.067 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:18:25.068 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:25.107 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:25.107+0000 7f08c5353040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:18:25.112 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:25.112+0000 7fb2aec41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:25.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:18:25.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:25.241 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:25.240+0000 7fb2aec41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:18:25.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.336+0000 7f37d9990640 1 Processor -- start 2023-10-21T17:18:25.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.336+0000 7f37d9990640 1 -- start start 2023-10-21T17:18:25.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.338+0000 7f37d9990640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d4159ac0 0x7f37d406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.338+0000 7f37d9990640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37d40ad790 0x7f37d40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.338+0000 7f37d9990640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d406dee0 con 0x7f37d415e330 2023-10-21T17:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.338+0000 7f37d9990640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d415d850 con 0x7f37d40ad790 2023-10-21T17:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.338+0000 7f37d9990640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d415db30 con 0x7f37d4159ac0 2023-10-21T17:18:25.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37d40ad790 0x7f37d40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37d40ad790 0x7f37d40702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41396/0 (socket says 172.21.15.53:41396) 2023-10-21T17:18:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 -- 172.21.15.53:0/3110136774 learned_addr learned my addr 172.21.15.53:0/3110136774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:25.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d2ffd640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d4159ac0 0x7f37d406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 -- 172.21.15.53:0/3110136774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d4159ac0 msgr2=0x7f37d406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:25.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d4159ac0 0x7f37d406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 -- 172.21.15.53:0/3110136774 >> v1:172.21.15.124:6789/0 conn(0x7f37d415e330 legacy=0x7f37d406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.339+0000 7f37d27fc640 1 -- 172.21.15.53:0/3110136774 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d415de10 con 0x7f37d40ad790 2023-10-21T17:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.341+0000 7f37d27fc640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d40ad790 0x7f37d40702d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009870 tx=0x7f37c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4688137c05506b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.341+0000 7f37d1ffb640 1 -- 172.21.15.53:0/3110136774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8030840 con 0x7f37d40ad790 2023-10-21T17:18:25.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.341+0000 7f37d1ffb640 1 -- 172.21.15.53:0/3110136774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c80309a0 con 0x7f37d40ad790 2023-10-21T17:18:25.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.341+0000 7f37d1ffb640 1 -- 172.21.15.53:0/3110136774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8030d30 con 0x7f37d40ad790 2023-10-21T17:18:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 -- 172.21.15.53:0/3110136774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d40ad790 msgr2=0x7f37d40702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d40ad790 0x7f37d40702d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009870 tx=0x7f37c802b980 comp rx=0 tx=0).stop 2023-10-21T17:18:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 -- 172.21.15.53:0/3110136774 shutdown_connections 2023-10-21T17:18:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d40ad790 0x7f37d40702d0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 --2- 172.21.15.53:0/3110136774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d4159ac0 0x7f37d406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 -- 172.21.15.53:0/3110136774 >> 172.21.15.53:0/3110136774 conn(0x7f37d406a380 msgr2=0x7f37d40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 -- 172.21.15.53:0/3110136774 shutdown_connections 2023-10-21T17:18:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.342+0000 7f37d9990640 1 -- 172.21.15.53:0/3110136774 wait complete. 2023-10-21T17:18:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.344+0000 7f37d9990640 1 Processor -- start 2023-10-21T17:18:25.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.344+0000 7f37d9990640 1 -- start start 2023-10-21T17:18:25.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37d40ad790 0x7f37d4145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 0x7f37d4144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4143ef0 con 0x7f37d4159ac0 2023-10-21T17:18:25.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4158560 con 0x7f37d40ad790 2023-10-21T17:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d9990640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4146400 con 0x7f37d415e330 2023-10-21T17:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41408/0 (socket says 172.21.15.53:41408) 2023-10-21T17:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d27fc640 1 -- 172.21.15.53:0/4031326532 learned_addr learned my addr 172.21.15.53:0/4031326532 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:25.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.345+0000 7f37d2ffd640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37d40ad790 0x7f37d4145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d37fe640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 0x7f37d4144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 msgr2=0x7f37d4144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 0x7f37d4144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37d40ad790 msgr2=0x7f37d4145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37d40ad790 0x7f37d4145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 -- 172.21.15.53:0/4031326532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80094d0 con 0x7f37d4159ac0 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d37fe640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 0x7f37d4144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d27fc640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f37c802b9c0 tx=0x7f37c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8039c90 con 0x7f37d4159ac0 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8039df0 con 0x7f37d4159ac0 2023-10-21T17:18:25.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c80446c0 con 0x7f37d4159ac0 2023-10-21T17:18:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.346+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4157950 con 0x7f37d4159ac0 2023-10-21T17:18:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.347+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d4157f10 con 0x7f37d4159ac0 2023-10-21T17:18:25.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.347+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4067490 con 0x7f37d4159ac0 2023-10-21T17:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.349+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f37c802f390 con 0x7f37d4159ac0 2023-10-21T17:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.350+0000 7f37bbfff640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 0x7f37c006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.350+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f37c803e070 con 0x7f37d4159ac0 2023-10-21T17:18:25.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.350+0000 7f37d2ffd640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 msgr2=0x7f37c006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:25.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.350+0000 7f37d2ffd640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 0x7f37c006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:25.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.353+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37c8076f00 con 0x7f37d4159ac0 2023-10-21T17:18:25.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.551+0000 7f37d2ffd640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 msgr2=0x7f37c006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:25.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.551+0000 7f37d2ffd640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 0x7f37c006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:25.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.584+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37d40b0ff0 con 0x7f37d4159ac0 2023-10-21T17:18:25.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.585+0000 7f37bbfff640 1 -- 172.21.15.53:0/4031326532 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f37c807a520 con 0x7f37d4159ac0 2023-10-21T17:18:25.587 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:25.587 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T17:18:25.587 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T17:18:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 msgr2=0x7f37c006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:25.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 0x7f37c006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 msgr2=0x7f37d40b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f37c802b9c0 tx=0x7f37c8004730 comp rx=0 tx=0).stop 2023-10-21T17:18:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 shutdown_connections 2023-10-21T17:18:25.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37d415e330 0x7f37d4144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f37c006afc0 0x7f37c006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37d4159ac0 0x7f37d40b1e90 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 --2- 172.21.15.53:0/4031326532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37d40ad790 0x7f37d4145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 >> 172.21.15.53:0/4031326532 conn(0x7f37d406a380 msgr2=0x7f37d40701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:25.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 shutdown_connections 2023-10-21T17:18:25.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.588+0000 7f37d9990640 1 -- 172.21.15.53:0/4031326532 wait complete. 2023-10-21T17:18:25.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:25.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.812+0000 7f42fec90640 1 Processor -- start 2023-10-21T17:18:25.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.812+0000 7f42fec90640 1 -- start start 2023-10-21T17:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fec90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42f815cd70 0x7f42f80d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fec90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fec90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80d2910 con 0x7f42f80b8530 2023-10-21T17:18:25.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fec90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f815d150 con 0x7f42f815cd70 2023-10-21T17:18:25.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fec90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80d0c90 con 0x7f42f80d0290 2023-10-21T17:18:25.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42f7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f42f80b8530 0x7f42f80cfb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41586/0 (socket says 172.21.15.53:41586) 2023-10-21T17:18:25.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42f7fff640 1 -- 172.21.15.53:0/2270386329 learned_addr learned my addr 172.21.15.53:0/2270386329 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:25.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fca2c640 1 --2- 172.21.15.53:0/2270386329 >> v2:172.21.15.124:3300/0 conn(0x7f42f815cd70 0x7f42f80d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.814+0000 7f42fd22d640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307358153 0 0) 0x7f42f80d2910 con 0x7f42f80b8530 2023-10-21T17:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e8003520 con 0x7f42f80b8530 2023-10-21T17:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828141207 0 0) 0x7f42e8003520 con 0x7f42f80b8530 2023-10-21T17:18:25.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42fca2c640 1 -- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 msgr2=0x7f42f80b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42fca2c640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42fca2c640 1 -- 172.21.15.53:0/2270386329 >> v1:172.21.15.53:6789/0 conn(0x7f42f80b8530 legacy=0x7f42f80cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.815+0000 7f42fca2c640 1 -- 172.21.15.53:0/2270386329 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80d0f70 con 0x7f42f815cd70 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.816+0000 7f42fca2c640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f815cd70 0x7f42f80d2010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f42e40097a0 tx=0x7f42e402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e8a4ca6195a759 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.817+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec001670 con 0x7f42f815cd70 2023-10-21T17:18:25.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.817+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e402e050 con 0x7f42f815cd70 2023-10-21T17:18:25.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.817+0000 7f42f77fe640 1 -- 172.21.15.53:0/2270386329 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4030710 con 0x7f42f815cd70 2023-10-21T17:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.817+0000 7f42fec90640 1 -- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f815cd70 msgr2=0x7f42f80d2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.817+0000 7f42fec90640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f815cd70 0x7f42f80d2010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f42e40097a0 tx=0x7f42e402b790 comp rx=0 tx=0).stop 2023-10-21T17:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 -- 172.21.15.53:0/2270386329 shutdown_connections 2023-10-21T17:18:25.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 --2- 172.21.15.53:0/2270386329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f815cd70 0x7f42f80d2010 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 -- 172.21.15.53:0/2270386329 >> 172.21.15.53:0/2270386329 conn(0x7f42f80718a0 msgr2=0x7f42f8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:25.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 -- 172.21.15.53:0/2270386329 shutdown_connections 2023-10-21T17:18:25.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.818+0000 7f42fec90640 1 -- 172.21.15.53:0/2270386329 wait complete. 2023-10-21T17:18:25.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.819+0000 7f42fec90640 1 Processor -- start 2023-10-21T17:18:25.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.820+0000 7f42fec90640 1 -- start start 2023-10-21T17:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 0x7f42f8154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f815d290 con 0x7f42f815cd70 2023-10-21T17:18:25.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80ca530 con 0x7f42f80b8530 2023-10-21T17:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fec90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80ca810 con 0x7f42f80d0290 2023-10-21T17:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41416/0 (socket says 172.21.15.53:41416) 2023-10-21T17:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 -- 172.21.15.53:0/686303921 learned_addr learned my addr 172.21.15.53:0/686303921 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fca2c640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 0x7f42f8154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42f7fff640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 -- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 msgr2=0x7f42f80c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 -- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 msgr2=0x7f42f8154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 0x7f42f8154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.821+0000 7f42fd22d640 1 -- 172.21.15.53:0/686303921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4009400 con 0x7f42f815cd70 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42fca2c640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 0x7f42f8154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42fd22d640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f42f000efc0 tx=0x7f42f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0014480 con 0x7f42f815cd70 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f000f040 con 0x7f42f815cd70 2023-10-21T17:18:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f80caaf0 con 0x7f42f815cd70 2023-10-21T17:18:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0014820 con 0x7f42f815cd70 2023-10-21T17:18:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.822+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f80ca220 con 0x7f42f815cd70 2023-10-21T17:18:25.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.823+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f8067490 con 0x7f42f815cd70 2023-10-21T17:18:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.824+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f42f0014980 con 0x7f42f815cd70 2023-10-21T17:18:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.826+0000 7f42f57fa640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 0x7f42dc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.826+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f42f008a980 con 0x7f42f815cd70 2023-10-21T17:18:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.826+0000 7f42fca2c640 1 -- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 msgr2=0x7f42dc06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.826+0000 7f42fca2c640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 0x7f42dc06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:25.830+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42f0055370 con 0x7f42f815cd70 2023-10-21T17:18:25.956 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:25.955+0000 7f08c5353040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:18:26.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.022+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f80b8060 con 0x7f42f815cd70 2023-10-21T17:18:26.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.024+0000 7f42f57fa640 1 -- 172.21.15.53:0/686303921 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17579 (secure 0 0 0) 0x7f42f0018020 con 0x7f42f815cd70 2023-10-21T17:18:26.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:26.025 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":169,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:22.791622+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":169,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3657298397":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3304201904":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3058690420":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3358065251":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/4258504518":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/2638777920":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6833/870299393":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6832/870299393":"2023-10-22T16:22:28.659493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":18,"length":39},{"begin":74,"length":39}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":55,"length":2},{"begin":111,"length":2}]}],"new_purged_snaps":[{"pool":20,"snaps":[{"begin":3,"length":15},{"begin":59,"length":15}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:26.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 msgr2=0x7f42dc06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:26.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 0x7f42dc06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 msgr2=0x7f42f80c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f42f000efc0 tx=0x7f42f000c490 comp rx=0 tx=0).stop 2023-10-21T17:18:26.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 shutdown_connections 2023-10-21T17:18:26.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f42dc06afc0 0x7f42dc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42f815cd70 0x7f42f80c8be0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42f80d0290 0x7f42f80c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 --2- 172.21.15.53:0/686303921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42f80b8530 0x7f42f8154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 >> 172.21.15.53:0/686303921 conn(0x7f42f80718a0 msgr2=0x7f42f815aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:26.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 shutdown_connections 2023-10-21T17:18:26.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.026+0000 7f42fec90640 1 -- 172.21.15.53:0/686303921 wait complete. 2023-10-21T17:18:26.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T16:22:11.619571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T16:22:12.967359+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 169, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:18:26.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:18:26.084 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:26.083+0000 7fb2aec41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:18:26.252 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.252+0000 7f08c5353040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:18:26.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.258+0000 7f8f55947640 1 Processor -- start 2023-10-21T17:18:26.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.258+0000 7f8f55947640 1 -- start start 2023-10-21T17:18:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.259+0000 7f8f55947640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f500ce140 0x7f8f500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.259+0000 7f8f55947640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f50065ab0 0x7f8f50160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.259+0000 7f8f55947640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f500cabe0 con 0x7f8f500cb360 2023-10-21T17:18:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.259+0000 7f8f55947640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500ccb90 con 0x7f8f500ce140 2023-10-21T17:18:26.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.259+0000 7f8f55947640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f500c9740 con 0x7f8f50065ab0 2023-10-21T17:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f500ce140 0x7f8f500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8f500ce140 0x7f8f500ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41430/0 (socket says 172.21.15.53:41430) 2023-10-21T17:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 -- 172.21.15.53:0/195813929 learned_addr learned my addr 172.21.15.53:0/195813929 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:26.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f4ffff640 1 --2- 172.21.15.53:0/195813929 >> v2:172.21.15.124:3300/0 conn(0x7f8f50065ab0 0x7f8f50160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 -- 172.21.15.53:0/195813929 >> v2:172.21.15.124:3300/0 conn(0x7f8f50065ab0 msgr2=0x7f8f50160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 --2- 172.21.15.53:0/195813929 >> v2:172.21.15.124:3300/0 conn(0x7f8f50065ab0 0x7f8f50160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 -- 172.21.15.53:0/195813929 >> v1:172.21.15.124:6789/0 conn(0x7f8f500cb360 legacy=0x7f8f50160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.260+0000 7f8f54945640 1 -- 172.21.15.53:0/195813929 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500c9410 con 0x7f8f500ce140 2023-10-21T17:18:26.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.262+0000 7f8f54945640 1 --2- 172.21.15.53:0/195813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f500ce140 0x7f8f500ce520 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8f50070510 tx=0x7f8f4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c6432eabd224e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.262+0000 7f8f4f7fe640 1 -- 172.21.15.53:0/195813929 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44031870 con 0x7f8f500ce140 2023-10-21T17:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.262+0000 7f8f4f7fe640 1 -- 172.21.15.53:0/195813929 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f440319d0 con 0x7f8f500ce140 2023-10-21T17:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.262+0000 7f8f4f7fe640 1 -- 172.21.15.53:0/195813929 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44031d60 con 0x7f8f500ce140 2023-10-21T17:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 -- 172.21.15.53:0/195813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f500ce140 msgr2=0x7f8f500ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 --2- 172.21.15.53:0/195813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f500ce140 0x7f8f500ce520 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8f50070510 tx=0x7f8f4402b910 comp rx=0 tx=0).stop 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 -- 172.21.15.53:0/195813929 shutdown_connections 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 --2- 172.21.15.53:0/195813929 >> v2:172.21.15.124:3300/0 conn(0x7f8f50065ab0 0x7f8f50160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 --2- 172.21.15.53:0/195813929 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f500ce140 0x7f8f500ce520 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 -- 172.21.15.53:0/195813929 >> 172.21.15.53:0/195813929 conn(0x7f8f500b08a0 msgr2=0x7f8f500b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 -- 172.21.15.53:0/195813929 shutdown_connections 2023-10-21T17:18:26.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.263+0000 7f8f55947640 1 -- 172.21.15.53:0/195813929 wait complete. 2023-10-21T17:18:26.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.265+0000 7f8f55947640 1 Processor -- start 2023-10-21T17:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.265+0000 7f8f55947640 1 -- start start 2023-10-21T17:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f500cb360 0x7f8f50156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 0x7f8f5015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50157520 con 0x7f8f50065ab0 2023-10-21T17:18:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5009e6b0 con 0x7f8f500ce140 2023-10-21T17:18:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f55947640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5009e990 con 0x7f8f500cb360 2023-10-21T17:18:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f54945640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f54945640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41446/0 (socket says 172.21.15.53:41446) 2023-10-21T17:18:26.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.266+0000 7f8f54945640 1 -- 172.21.15.53:0/1162671966 learned_addr learned my addr 172.21.15.53:0/1162671966 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f4ffff640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f500cb360 0x7f8f50156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f55146640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 0x7f8f5015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 -- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f500cb360 msgr2=0x7f8f50156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f500cb360 0x7f8f50156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 -- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 msgr2=0x7f8f5015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 0x7f8f5015e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 -- 172.21.15.53:0/1162671966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440094d0 con 0x7f8f50065ab0 2023-10-21T17:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f55146640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 0x7f8f5015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f54945640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8f4402b950 tx=0x7f8f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.267+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4403abe0 con 0x7f8f50065ab0 2023-10-21T17:18:26.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.268+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4403ad60 con 0x7f8f50065ab0 2023-10-21T17:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.268+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44044630 con 0x7f8f50065ab0 2023-10-21T17:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.268+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5009ec70 con 0x7f8f50065ab0 2023-10-21T17:18:26.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.268+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f500c6c70 con 0x7f8f50065ab0 2023-10-21T17:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.269+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5009f190 con 0x7f8f50065ab0 2023-10-21T17:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.270+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 83) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f8f4404d450 con 0x7f8f50065ab0 2023-10-21T17:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.271+0000 7f8f4d7fa640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 0x7f8f3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.271+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6577+0+0 (secure 0 0 0) 0x7f8f4402d040 con 0x7f8f50065ab0 2023-10-21T17:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.271+0000 7f8f4ffff640 1 -- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 msgr2=0x7f8f3406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2988307648 2023-10-21T17:18:26.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.271+0000 7f8f4ffff640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 0x7f8f3406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:26.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.275+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f5009f190 con 0x7f8f50065ab0 2023-10-21T17:18:26.310 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.310+0000 7f08c5353040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:18:26.400 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:26.399+0000 7fb2aec41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:18:26.423 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.423+0000 7f08c5353040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:18:26.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.444+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 84) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8f440aa2a0 con 0x7f8f50065ab0 2023-10-21T17:18:26.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.445+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 msgr2=0x7f8f3406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:26.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.445+0000 7f8f4d7fa640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 0x7f8f3406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.469 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:26.469+0000 7fb2aec41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:18:26.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.477+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f5009f190 con 0x7f8f50065ab0 2023-10-21T17:18:26.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.479+0000 7f8f4d7fa640 1 -- 172.21.15.53:0/1162671966 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f5009f190 con 0x7f8f50065ab0 2023-10-21T17:18:26.480 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:18:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.481+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 msgr2=0x7f8f50157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.481+0000 7f8f55947640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8f4402b950 tx=0x7f8f440046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.481+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 shutdown_connections 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.481+0000 7f8f55947640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f500ce140 0x7f8f5015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.481+0000 7f8f55947640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f500cb360 0x7f8f50156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.482+0000 7f8f55947640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.124:6832/2988307648,v1:172.21.15.124:6833/2988307648] conn(0x7f8f3406af90 0x7f8f3406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.482+0000 7f8f55947640 1 --2- 172.21.15.53:0/1162671966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f50065ab0 0x7f8f50157bc0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.482+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 >> 172.21.15.53:0/1162671966 conn(0x7f8f500b08a0 msgr2=0x7f8f50071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:26.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.482+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 shutdown_connections 2023-10-21T17:18:26.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.482+0000 7f8f55947640 1 -- 172.21.15.53:0/1162671966 wait complete. 2023-10-21T17:18:26.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:18:26.518 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.518+0000 7f084ade1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.518 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.518+0000 7f084ade1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.518+0000 7f084ade1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.518+0000 7f084ade1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.518+0000 7f084ade1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.520 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.519+0000 7f0846dd9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.520 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.519+0000 7f0846dd9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.520 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.519+0000 7f0846dd9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.520 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.519+0000 7f0846dd9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.520 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:26.519+0000 7f0846dd9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:18:26.594 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:26.593+0000 7fb2aec41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:18:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.758+0000 7fe213ec2640 1 Processor -- start 2023-10-21T17:18:26.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.758+0000 7fe213ec2640 1 -- start start 2023-10-21T17:18:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.760+0000 7fe213ec2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.760+0000 7fe213ec2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe20c06db60 0x7fe20c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.760+0000 7fe213ec2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c15b310 con 0x7fe20c15aa00 2023-10-21T17:18:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.760+0000 7fe213ec2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c0ca230 con 0x7fe20c06db60 2023-10-21T17:18:26.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.760+0000 7fe213ec2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06d290 con 0x7fe20c06c3a0 2023-10-21T17:18:26.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe211c5e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe21145d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe20c06db60 0x7fe20c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe211c5e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:47208/0 (socket says 172.21.15.53:47208) 2023-10-21T17:18:26.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe211c5e640 1 -- 172.21.15.53:0/4163672952 learned_addr learned my addr 172.21.15.53:0/4163672952 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:26.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe211c5e640 1 -- 172.21.15.53:0/4163672952 >> v2:172.21.15.124:3300/0 conn(0x7fe20c06db60 msgr2=0x7fe20c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:26.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.761+0000 7fe211c5e640 1 --2- 172.21.15.53:0/4163672952 >> v2:172.21.15.124:3300/0 conn(0x7fe20c06db60 0x7fe20c06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.762+0000 7fe211c5e640 1 -- 172.21.15.53:0/4163672952 >> v1:172.21.15.124:6789/0 conn(0x7fe20c15aa00 legacy=0x7fe20c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.762+0000 7fe211c5e640 1 -- 172.21.15.53:0/4163672952 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0c9860 con 0x7fe20c06c3a0 2023-10-21T17:18:26.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.763+0000 7fe211c5e640 1 --2- 172.21.15.53:0/4163672952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe208009870 tx=0x7fe20802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc97263e7c5a2ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.763+0000 7fe210c5c640 1 -- 172.21.15.53:0/4163672952 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208030840 con 0x7fe20c06c3a0 2023-10-21T17:18:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe210c5c640 1 -- 172.21.15.53:0/4163672952 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2080309a0 con 0x7fe20c06c3a0 2023-10-21T17:18:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe210c5c640 1 -- 172.21.15.53:0/4163672952 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208030d30 con 0x7fe20c06c3a0 2023-10-21T17:18:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 -- 172.21.15.53:0/4163672952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 msgr2=0x7fe20c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 --2- 172.21.15.53:0/4163672952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe208009870 tx=0x7fe20802b980 comp rx=0 tx=0).stop 2023-10-21T17:18:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 -- 172.21.15.53:0/4163672952 shutdown_connections 2023-10-21T17:18:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 --2- 172.21.15.53:0/4163672952 >> v2:172.21.15.124:3300/0 conn(0x7fe20c06db60 0x7fe20c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 --2- 172.21.15.53:0/4163672952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c9cf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.764+0000 7fe213ec2640 1 -- 172.21.15.53:0/4163672952 >> 172.21.15.53:0/4163672952 conn(0x7fe20c06a5d0 msgr2=0x7fe20c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:26.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.765+0000 7fe213ec2640 1 -- 172.21.15.53:0/4163672952 shutdown_connections 2023-10-21T17:18:26.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.765+0000 7fe213ec2640 1 -- 172.21.15.53:0/4163672952 wait complete. 2023-10-21T17:18:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.766+0000 7fe213ec2640 1 Processor -- start 2023-10-21T17:18:26.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.766+0000 7fe213ec2640 1 -- start start 2023-10-21T17:18:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe20c15aa00 0x7fe20c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c66e0 con 0x7fe20c06db60 2023-10-21T17:18:26.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c3cb0 con 0x7fe20c15aa00 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.767+0000 7fe213ec2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0c5810 con 0x7fe20c06c3a0 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41448/0 (socket says 172.21.15.53:41448) 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 -- 172.21.15.53:0/2456483247 learned_addr learned my addr 172.21.15.53:0/2456483247 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe211c5e640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 -- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 msgr2=0x7fe20c0c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21245f640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe20c15aa00 0x7fe20c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 -- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe20c15aa00 msgr2=0x7fe20c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe20c15aa00 0x7fe20c0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.768+0000 7fe21145d640 1 -- 172.21.15.53:0/2456483247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2080094d0 con 0x7fe20c06db60 2023-10-21T17:18:26.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe21145d640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00e9b0 tx=0x7fe1fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:26.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc009670 con 0x7fe20c06db60 2023-10-21T17:18:26.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00c9a0 con 0x7fe20c06db60 2023-10-21T17:18:26.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00cd50 con 0x7fe20c06db60 2023-10-21T17:18:26.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c0c5a60 con 0x7fe20c06db60 2023-10-21T17:18:26.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.769+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c0c6020 con 0x7fe20c06db60 2023-10-21T17:18:26.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.770+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c0c91f0 con 0x7fe20c06db60 2023-10-21T17:18:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.771+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 84) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe1fc013470 con 0x7fe20c06db60 2023-10-21T17:18:26.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.772+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fe1fc089870 con 0x7fe20c06db60 2023-10-21T17:18:26.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.777+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe20c0c91f0 con 0x7fe20c06db60 2023-10-21T17:18:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.963+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe20c0c91f0 con 0x7fe20c06db60 2023-10-21T17:18:26.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.964+0000 7fe2027fc640 1 -- 172.21.15.53:0/2456483247 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe20c0c91f0 con 0x7fe20c06db60 2023-10-21T17:18:26.965 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:18:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 msgr2=0x7fe20c0c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00e9b0 tx=0x7fe1fc00ee70 comp rx=0 tx=0).stop 2023-10-21T17:18:26.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 shutdown_connections 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe20c15aa00 0x7fe20c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe20c06db60 0x7fe20c0c2270 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 --2- 172.21.15.53:0/2456483247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe20c06c3a0 0x7fe20c0c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 >> 172.21.15.53:0/2456483247 conn(0x7fe20c06a5d0 msgr2=0x7fe20c159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 shutdown_connections 2023-10-21T17:18:26.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:26.966+0000 7fe213ec2640 1 -- 172.21.15.53:0/2456483247 wait complete. 2023-10-21T17:18:26.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:18:27.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.214+0000 7f97409a9640 1 Processor -- start 2023-10-21T17:18:27.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.214+0000 7f97409a9640 1 -- start start 2023-10-21T17:18:27.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f97409a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973c160660 0x7f973c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f97409a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c06e150 con 0x7f973c0c9910 2023-10-21T17:18:27.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f97409a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c06de70 con 0x7f973c0cb620 2023-10-21T17:18:27.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f97409a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0c8e30 con 0x7f973c160660 2023-10-21T17:18:27.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f973a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973c160660 0x7f973c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f9731d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f973c0c9910 0x7f973c0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41588/0 (socket says 172.21.15.53:41588) 2023-10-21T17:18:27.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f9731d9b640 1 -- 172.21.15.53:0/2043715001 learned_addr learned my addr 172.21.15.53:0/2043715001 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:27.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.216+0000 7f973a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f973c160660 0x7f973c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41452/0 (socket says 172.21.15.53:41452) 2023-10-21T17:18:27.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.217+0000 7f973a59c640 1 -- 172.21.15.53:0/2043715001 >> v1:172.21.15.124:6789/0 conn(0x7f973c0cb620 legacy=0x7f973c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.217+0000 7f973a59c640 1 -- 172.21.15.53:0/2043715001 >> v1:172.21.15.53:6789/0 conn(0x7f973c0c9910 legacy=0x7f973c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.217+0000 7f973a59c640 1 -- 172.21.15.53:0/2043715001 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0c9110 con 0x7f973c160660 2023-10-21T17:18:27.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.218+0000 7f973a59c640 1 --2- 172.21.15.53:0/2043715001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c070cc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f973c06e340 tx=0x7f972c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912d4e5426617ef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:27.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.219+0000 7f9739d9b640 1 -- 172.21.15.53:0/2043715001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c02d040 con 0x7f973c160660 2023-10-21T17:18:27.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.219+0000 7f9739d9b640 1 -- 172.21.15.53:0/2043715001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c031810 con 0x7f973c160660 2023-10-21T17:18:27.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.219+0000 7f9739d9b640 1 -- 172.21.15.53:0/2043715001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c031ba0 con 0x7f973c160660 2023-10-21T17:18:27.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.219+0000 7f97409a9640 1 -- 172.21.15.53:0/2043715001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 msgr2=0x7f973c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.219+0000 7f97409a9640 1 --2- 172.21.15.53:0/2043715001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c070cc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f973c06e340 tx=0x7f972c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.220+0000 7f97409a9640 1 -- 172.21.15.53:0/2043715001 shutdown_connections 2023-10-21T17:18:27.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.220+0000 7f97409a9640 1 --2- 172.21.15.53:0/2043715001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c070cc0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.220+0000 7f97409a9640 1 -- 172.21.15.53:0/2043715001 >> 172.21.15.53:0/2043715001 conn(0x7f973c06a5d0 msgr2=0x7f973c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:27.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.220+0000 7f97409a9640 1 -- 172.21.15.53:0/2043715001 shutdown_connections 2023-10-21T17:18:27.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.220+0000 7f97409a9640 1 -- 172.21.15.53:0/2043715001 wait complete. 2023-10-21T17:18:27.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.222+0000 7f97409a9640 1 Processor -- start 2023-10-21T17:18:27.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.222+0000 7f97409a9640 1 -- start start 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f973c0c9910 0x7f973c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f973c0cb620 0x7f973c15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c159450 con 0x7f973c160660 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c158840 con 0x7f973c0cb620 2023-10-21T17:18:27.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f97409a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c158b20 con 0x7f973c0c9910 2023-10-21T17:18:27.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f973ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f973ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41458/0 (socket says 172.21.15.53:41458) 2023-10-21T17:18:27.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f973ad9d640 1 -- 172.21.15.53:0/161156165 learned_addr learned my addr 172.21.15.53:0/161156165 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:27.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f9731d9b640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f973c0cb620 0x7f973c15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.223+0000 7f973a59c640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f973c0c9910 0x7f973c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 -- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f973c0c9910 msgr2=0x7f973c153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f973c0c9910 0x7f973c153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 -- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f973c0cb620 msgr2=0x7f973c15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f973c0cb620 0x7f973c15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 -- 172.21.15.53:0/161156165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c009400 con 0x7f973c160660 2023-10-21T17:18:27.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f973ad9d640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f973400c7a0 tx=0x7f973400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:27.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734003b70 con 0x7f973c160660 2023-10-21T17:18:27.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734003cf0 con 0x7f973c160660 2023-10-21T17:18:27.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734003ea0 con 0x7f973c160660 2023-10-21T17:18:27.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c158e00 con 0x7f973c160660 2023-10-21T17:18:27.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.224+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c154420 con 0x7f973c160660 2023-10-21T17:18:27.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.225+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c159600 con 0x7f973c160660 2023-10-21T17:18:27.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.227+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 84) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f973400aa90 con 0x7f973c160660 2023-10-21T17:18:27.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.228+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f9734089420 con 0x7f973c160660 2023-10-21T17:18:27.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.233+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f973c159600 con 0x7f973c160660 2023-10-21T17:18:27.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.432+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f973c04e160 con 0x7f973c160660 2023-10-21T17:18:27.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.434+0000 7f97337fe640 1 -- 172.21.15.53:0/161156165 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97340543c0 con 0x7f973c160660 2023-10-21T17:18:27.435 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:18:27.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.436+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 msgr2=0x7f973c147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.436+0000 7f97409a9640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f973400c7a0 tx=0x7f973400cc60 comp rx=0 tx=0).stop 2023-10-21T17:18:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 shutdown_connections 2023-10-21T17:18:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f973c160660 0x7f973c147490 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f973c0cb620 0x7f973c15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 --2- 172.21.15.53:0/161156165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f973c0c9910 0x7f973c153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 >> 172.21.15.53:0/161156165 conn(0x7f973c06a5d0 msgr2=0x7f973c073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 shutdown_connections 2023-10-21T17:18:27.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.437+0000 7f97409a9640 1 -- 172.21.15.53:0/161156165 wait complete. 2023-10-21T17:18:27.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.674+0000 7f0d88800640 1 Processor -- start 2023-10-21T17:18:27.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.674+0000 7f0d88800640 1 -- start start 2023-10-21T17:18:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d88800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0d80165cc0 0x7f0d801680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d88800640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800c48f0 con 0x7f0d80160940 2023-10-21T17:18:27.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d88800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800710f0 con 0x7f0d8015cd90 2023-10-21T17:18:27.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d88800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c4d80 con 0x7f0d80165cc0 2023-10-21T17:18:27.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d85d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0d80160940 0x7f0d800709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41598/0 (socket says 172.21.15.53:41598) 2023-10-21T17:18:27.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.676+0000 7f0d85d9b640 1 -- 172.21.15.53:0/4092134669 learned_addr learned my addr 172.21.15.53:0/4092134669 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:27.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d86d9d640 1 --2- 172.21.15.53:0/4092134669 >> v2:172.21.15.124:3300/0 conn(0x7f0d80165cc0 0x7f0d801680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147091782 0 0) 0x7f0d800c48f0 con 0x7f0d80160940 2023-10-21T17:18:27.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d6c003520 con 0x7f0d80160940 2023-10-21T17:18:27.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128286264 0 0) 0x7f0d6c003520 con 0x7f0d80160940 2023-10-21T17:18:27.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 >> v2:172.21.15.124:3300/0 conn(0x7f0d80165cc0 msgr2=0x7f0d801680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 --2- 172.21.15.53:0/4092134669 >> v2:172.21.15.124:3300/0 conn(0x7f0d80165cc0 0x7f0d801680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 >> v1:172.21.15.124:6789/0 conn(0x7f0d8015cd90 legacy=0x7f0d800702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800c3ce0 con 0x7f0d80160940 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0d78001670 con 0x7f0d80160940 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 >> v1:172.21.15.53:6789/0 conn(0x7f0d80160940 legacy=0x7f0d800709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 0x7f0d6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0047c0 0x7f0d6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.677+0000 7f0d8559a640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d6c0051d0 0x7f0d6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00c0d0 con 0x7f0d6c0039a0 2023-10-21T17:18:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00c240 con 0x7f0d6c0047c0 2023-10-21T17:18:27.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00c400 con 0x7f0d6c0051d0 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 0x7f0d6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d85d9b640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0047c0 0x7f0d6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d86d9d640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d6c0051d0 0x7f0d6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 -- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d6c0051d0 msgr2=0x7f0d6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d6c0051d0 0x7f0d6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 -- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0047c0 msgr2=0x7f0d6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0047c0 0x7f0d6c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d8659c640 1 -- 172.21.15.53:0/4092134669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c003520 con 0x7f0d6c0039a0 2023-10-21T17:18:27.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.678+0000 7f0d86d9d640 1 -- 172.21.15.53:0/4092134669 reap_dead start 2023-10-21T17:18:27.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d8659c640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 0x7f0d6c003dc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001720 tx=0x7f0d6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc719185964e7139 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:27.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740052d0 con 0x7f0d6c0039a0 2023-10-21T17:18:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7402e050 con 0x7f0d6c0039a0 2023-10-21T17:18:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d8559a640 1 -- 172.21.15.53:0/4092134669 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003830 con 0x7f0d6c0039a0 2023-10-21T17:18:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d88800640 1 -- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 msgr2=0x7f0d6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d88800640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 0x7f0d6c003dc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f0d74001720 tx=0x7f0d6c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d88800640 1 -- 172.21.15.53:0/4092134669 shutdown_connections 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d88800640 1 --2- 172.21.15.53:0/4092134669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d6c0039a0 0x7f0d6c003dc0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.680+0000 7f0d88800640 1 -- 172.21.15.53:0/4092134669 >> 172.21.15.53:0/4092134669 conn(0x7f0d8006a380 msgr2=0x7f0d800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.681+0000 7f0d88800640 1 -- 172.21.15.53:0/4092134669 shutdown_connections 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.681+0000 7f0d88800640 1 -- 172.21.15.53:0/4092134669 wait complete. 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.682+0000 7f0d88800640 1 Processor -- start 2023-10-21T17:18:27.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.683+0000 7f0d88800640 1 -- start start 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.683+0000 7f0d88800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0039a0 0x7f0d80152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.683+0000 7f0d88800640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.683+0000 7f0d88800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 0x7f0d80150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.683+0000 7f0d88800640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8015ed80 con 0x7f0d801491b0 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d88800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b1480 con 0x7f0d6c0039a0 2023-10-21T17:18:27.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d88800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8015efd0 con 0x7f0d80150a50 2023-10-21T17:18:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41476/0 (socket says 172.21.15.53:41476) 2023-10-21T17:18:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 -- 172.21.15.53:0/1553816315 learned_addr learned my addr 172.21.15.53:0/1553816315 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d86d9d640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 0x7f0d80150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 -- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 msgr2=0x7f0d80150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 0x7f0d80150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 -- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0039a0 msgr2=0x7f0d80152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0039a0 0x7f0d80152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d85d9b640 1 -- 172.21.15.53:0/1553816315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8015f2b0 con 0x7f0d801491b0 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.684+0000 7f0d86d9d640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 0x7f0d80150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:27.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d85d9b640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e7b0 tx=0x7f0d7800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800c660 con 0x7f0d801491b0 2023-10-21T17:18:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800c7c0 con 0x7f0d801491b0 2023-10-21T17:18:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800cb50 con 0x7f0d801491b0 2023-10-21T17:18:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8015f590 con 0x7f0d801491b0 2023-10-21T17:18:27.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.685+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80155900 con 0x7f0d801491b0 2023-10-21T17:18:27.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.686+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d80067490 con 0x7f0d801491b0 2023-10-21T17:18:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.687+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 85) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0d7800ccb0 con 0x7f0d801491b0 2023-10-21T17:18:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.688+0000 7f0d72ffd640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 0x7f0d6406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.688+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f0d78089600 con 0x7f0d801491b0 2023-10-21T17:18:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.689+0000 7f0d8659c640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 0x7f0d6406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:27.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.690+0000 7f0d8659c640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 0x7f0d6406d1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d74033000 tx=0x7f0d7402f000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:27.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.693+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d78054490 con 0x7f0d801491b0 2023-10-21T17:18:27.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.930+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d800b0ef0 con 0x7f0d801491b0 2023-10-21T17:18:27.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.931+0000 7f0d72ffd640 1 -- 172.21.15.53:0/1553816315 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f0d78057ab0 con 0x7f0d801491b0 2023-10-21T17:18:27.933 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:27.933 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T17:18:27.933 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T17:18:27.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 msgr2=0x7f0d6406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 0x7f0d6406d1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d74033000 tx=0x7f0d7402f000 comp rx=0 tx=0).stop 2023-10-21T17:18:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 msgr2=0x7f0d801553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e7b0 tx=0x7f0d7800ebb0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 shutdown_connections 2023-10-21T17:18:27.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0d6406acf0 0x7f0d6406d1b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d80150a50 0x7f0d80150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d801491b0 0x7f0d801553c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 --2- 172.21.15.53:0/1553816315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d6c0039a0 0x7f0d80152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:27.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.934+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 >> 172.21.15.53:0/1553816315 conn(0x7f0d8006a380 msgr2=0x7f0d800ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:27.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.935+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 shutdown_connections 2023-10-21T17:18:27.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:27.935+0000 7f0d88800640 1 -- 172.21.15.53:0/1553816315 wait complete. 2023-10-21T17:18:27.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:28.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.162+0000 7fa191b2e640 1 Processor -- start 2023-10-21T17:18:28.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.162+0000 7fa191b2e640 1 -- start start 2023-10-21T17:18:28.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.164+0000 7fa191b2e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa18c15ab10 0x7fa18c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa191b2e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa18c06a290 0x7fa18c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa191b2e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c0cf660 con 0x7fa18c0ca440 2023-10-21T17:18:28.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa191b2e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c06a740 con 0x7fa18c06a290 2023-10-21T17:18:28.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa191b2e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c15aef0 con 0x7fa18c15ab10 2023-10-21T17:18:28.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa183fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa18c06a290 0x7fa18c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.165+0000 7fa19132d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa18c0ca440 0x7fa18c071200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41602/0 (socket says 172.21.15.53:41602) 2023-10-21T17:18:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.166+0000 7fa19132d640 1 -- 172.21.15.53:0/1591651634 learned_addr learned my addr 172.21.15.53:0/1591651634 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.166+0000 7fa190b2c640 1 --2- 172.21.15.53:0/1591651634 >> v2:172.21.15.124:3300/0 conn(0x7fa18c15ab10 0x7fa18c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.167+0000 7fa183fff640 1 -- 172.21.15.53:0/1591651634 >> v2:172.21.15.124:3300/0 conn(0x7fa18c15ab10 msgr2=0x7fa18c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.167+0000 7fa183fff640 1 --2- 172.21.15.53:0/1591651634 >> v2:172.21.15.124:3300/0 conn(0x7fa18c15ab10 0x7fa18c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.167+0000 7fa1837fe640 1 -- 172.21.15.53:0/1591651634 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914842384 0 0) 0x7fa18c0cf660 con 0x7fa18c0ca440 2023-10-21T17:18:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.167+0000 7fa183fff640 1 -- 172.21.15.53:0/1591651634 >> v1:172.21.15.53:6789/0 conn(0x7fa18c0ca440 legacy=0x7fa18c071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.167+0000 7fa183fff640 1 -- 172.21.15.53:0/1591651634 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0cf8b0 con 0x7fa18c06a290 2023-10-21T17:18:28.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.168+0000 7fa183fff640 1 --2- 172.21.15.53:0/1591651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9f00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa18c06c960 tx=0x7fa17c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51932845e2580d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.168+0000 7fa1837fe640 1 -- 172.21.15.53:0/1591651634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c031830 con 0x7fa18c06a290 2023-10-21T17:18:28.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa1837fe640 1 -- 172.21.15.53:0/1591651634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c031990 con 0x7fa18c06a290 2023-10-21T17:18:28.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa1837fe640 1 -- 172.21.15.53:0/1591651634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c031d20 con 0x7fa18c06a290 2023-10-21T17:18:28.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 -- 172.21.15.53:0/1591651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c06a290 msgr2=0x7fa18c0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 --2- 172.21.15.53:0/1591651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9f00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa18c06c960 tx=0x7fa17c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 -- 172.21.15.53:0/1591651634 shutdown_connections 2023-10-21T17:18:28.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 --2- 172.21.15.53:0/1591651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9f00 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 --2- 172.21.15.53:0/1591651634 >> v2:172.21.15.124:3300/0 conn(0x7fa18c15ab10 0x7fa18c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 -- 172.21.15.53:0/1591651634 >> 172.21.15.53:0/1591651634 conn(0x7fa18c06ae60 msgr2=0x7fa18c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:28.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.169+0000 7fa191b2e640 1 -- 172.21.15.53:0/1591651634 shutdown_connections 2023-10-21T17:18:28.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.170+0000 7fa191b2e640 1 -- 172.21.15.53:0/1591651634 wait complete. 2023-10-21T17:18:28.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.171+0000 7fa191b2e640 1 Processor -- start 2023-10-21T17:18:28.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.172+0000 7fa191b2e640 1 -- start start 2023-10-21T17:18:28.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.172+0000 7fa191b2e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.172+0000 7fa191b2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa191b2e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa18c15ab10 0x7fa18c0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa191b2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0b3f40 con 0x7fa18c0ca440 2023-10-21T17:18:28.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa191b2e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c0b13f0 con 0x7fa18c06a290 2023-10-21T17:18:28.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa191b2e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c151f20 con 0x7fa18c15ab10 2023-10-21T17:18:28.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa183fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa183fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41496/0 (socket says 172.21.15.53:41496) 2023-10-21T17:18:28.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa183fff640 1 -- 172.21.15.53:0/2619093518 learned_addr learned my addr 172.21.15.53:0/2619093518 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:28.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa19132d640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa18c15ab10 0x7fa18c0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.173+0000 7fa190b2c640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 -- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa18c15ab10 msgr2=0x7fa18c0c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa18c15ab10 0x7fa18c0c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 -- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa18c06a290 msgr2=0x7fa18c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 -- 172.21.15.53:0/2619093518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c009400 con 0x7fa18c0ca440 2023-10-21T17:18:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa183fff640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fa17c000c00 tx=0x7fa17c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c004230 con 0x7fa18c0ca440 2023-10-21T17:18:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c004390 con 0x7fa18c0ca440 2023-10-21T17:18:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c0b36a0 con 0x7fa18c0ca440 2023-10-21T17:18:28.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c03a870 con 0x7fa18c0ca440 2023-10-21T17:18:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.174+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c0b3c60 con 0x7fa18c0ca440 2023-10-21T17:18:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.176+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c04e160 con 0x7fa18c0ca440 2023-10-21T17:18:28.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.177+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 85) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa17c03a9d0 con 0x7fa18c0ca440 2023-10-21T17:18:28.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.178+0000 7fa1817fa640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 0x7fa17806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.178+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fa17c0ab800 con 0x7fa18c0ca440 2023-10-21T17:18:28.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.179+0000 7fa190b2c640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 0x7fa17806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.180+0000 7fa190b2c640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 0x7fa17806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1880074a0 tx=0x7fa18800b040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.182+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa17c076690 con 0x7fa18c0ca440 2023-10-21T17:18:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.379+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa18c0b1530 con 0x7fa18c0ca440 2023-10-21T17:18:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.382+0000 7fa1817fa640 1 -- 172.21.15.53:0/2619093518 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18019 (secure 0 0 0) 0x7fa17c079cb0 con 0x7fa18c0ca440 2023-10-21T17:18:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:28.384 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":171,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:27.432415+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1493657297":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6833/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1967999416":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6832/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2914043524":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1768082194":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3657298397":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3304201904":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/2610966345":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3058690420":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3358065251":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/4258504518":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3974743798":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2638777920":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6833/870299393":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6832/870299393":"2023-10-22T16:22:28.659493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":18,"length":40},{"begin":74,"length":40}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:28.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 msgr2=0x7fa17806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 0x7fa17806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa1880074a0 tx=0x7fa18800b040 comp rx=0 tx=0).stop 2023-10-21T17:18:28.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 msgr2=0x7fa18c159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fa17c000c00 tx=0x7fa17c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 shutdown_connections 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa17806acc0 0x7fa17806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa18c15ab10 0x7fa18c0c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa18c0ca440 0x7fa18c159290 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 --2- 172.21.15.53:0/2619093518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa18c06a290 0x7fa18c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 >> 172.21.15.53:0/2619093518 conn(0x7fa18c06ae60 msgr2=0x7fa18c0cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 shutdown_connections 2023-10-21T17:18:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.385+0000 7fa191b2e640 1 -- 172.21.15.53:0/2619093518 wait complete. 2023-10-21T17:18:28.404 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:18:28.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.638+0000 7f3ff25a7640 1 Processor -- start 2023-10-21T17:18:28.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.638+0000 7f3ff25a7640 1 -- start start 2023-10-21T17:18:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff25a7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3fec0b07d0 0x7f3fec165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff25a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fec15f7d0 con 0x7f3fec0adc70 2023-10-21T17:18:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff25a7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fec15efb0 con 0x7f3fec15ab30 2023-10-21T17:18:28.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff25a7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec15f210 con 0x7f3fec0b07d0 2023-10-21T17:18:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff0da4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3fec0adc70 0x7f3fec0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41618/0 (socket says 172.21.15.53:41618) 2023-10-21T17:18:28.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff0da4640 1 -- 172.21.15.53:0/2380143568 learned_addr learned my addr 172.21.15.53:0/2380143568 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.640+0000 7f3ff1da6640 1 --2- 172.21.15.53:0/2380143568 >> v2:172.21.15.124:3300/0 conn(0x7f3fec0b07d0 0x7f3fec165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.641+0000 7f3fe3fff640 1 -- 172.21.15.53:0/2380143568 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962153362 0 0) 0x7f3fec15f7d0 con 0x7f3fec0adc70 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.641+0000 7f3fe3fff640 1 -- 172.21.15.53:0/2380143568 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd8003520 con 0x7f3fec0adc70 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.641+0000 7f3ff1da6640 1 -- 172.21.15.53:0/2380143568 >> v1:172.21.15.124:6789/0 conn(0x7f3fec15ab30 legacy=0x7f3fec15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.641+0000 7f3ff1da6640 1 -- 172.21.15.53:0/2380143568 >> v1:172.21.15.53:6789/0 conn(0x7f3fec0adc70 legacy=0x7f3fec0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.641+0000 7f3ff1da6640 1 -- 172.21.15.53:0/2380143568 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec15f4f0 con 0x7f3fec0b07d0 2023-10-21T17:18:28.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.642+0000 7f3ff1da6640 1 --2- 172.21.15.53:0/2380143568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec0b07d0 0x7f3fec165bd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009d90 tx=0x7f3fe402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ba0d6a1ea90aa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3fe3fff640 1 -- 172.21.15.53:0/2380143568 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe402ccf0 con 0x7f3fec0b07d0 2023-10-21T17:18:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3fe3fff640 1 -- 172.21.15.53:0/2380143568 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe402ce50 con 0x7f3fec0b07d0 2023-10-21T17:18:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3fe3fff640 1 -- 172.21.15.53:0/2380143568 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4034660 con 0x7f3fec0b07d0 2023-10-21T17:18:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2380143568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec0b07d0 msgr2=0x7f3fec165bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2380143568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec0b07d0 0x7f3fec165bd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009d90 tx=0x7f3fe402bf40 comp rx=0 tx=0).stop 2023-10-21T17:18:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2380143568 shutdown_connections 2023-10-21T17:18:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2380143568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec0b07d0 0x7f3fec165bd0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.643+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2380143568 >> 172.21.15.53:0/2380143568 conn(0x7f3fec0cb0c0 msgr2=0x7f3fec0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.644+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2380143568 shutdown_connections 2023-10-21T17:18:28.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.644+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2380143568 wait complete. 2023-10-21T17:18:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.646+0000 7f3ff25a7640 1 Processor -- start 2023-10-21T17:18:28.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.646+0000 7f3ff25a7640 1 -- start start 2023-10-21T17:18:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fec0b07d0 0x7f3fec158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 0x7f3fec158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0c5710 con 0x7f3fec0adc70 2023-10-21T17:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0c59f0 con 0x7f3fec15ab30 2023-10-21T17:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.647+0000 7f3ff25a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec0c5cd0 con 0x7f3fec0b07d0 2023-10-21T17:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41504/0 (socket says 172.21.15.53:41504) 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 -- 172.21.15.53:0/2175449808 learned_addr learned my addr 172.21.15.53:0/2175449808 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff1da6640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 0x7f3fec158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff0da4640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fec0b07d0 0x7f3fec158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 -- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fec0b07d0 msgr2=0x7f3fec158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fec0b07d0 0x7f3fec158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 -- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 msgr2=0x7f3fec158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 0x7f3fec158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff15a5640 1 -- 172.21.15.53:0/2175449808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4009a70 con 0x7f3fec0adc70 2023-10-21T17:18:28.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.648+0000 7f3ff1da6640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 0x7f3fec158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:28.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3ff15a5640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c390 tx=0x7f3fe800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80052d0 con 0x7f3fec0adc70 2023-10-21T17:18:28.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8005430 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8003830 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec0c5fb0 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.649+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec0c6570 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.650+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec04e160 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.651+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 85) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3fe8003990 con 0x7f3fec0adc70 2023-10-21T17:18:28.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.652+0000 7f3fe1ffb640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 0x7f3fd006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:28.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.652+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f3fe8088d30 con 0x7f3fec0adc70 2023-10-21T17:18:28.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.653+0000 7f3ff0da4640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 0x7f3fd006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:28.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.654+0000 7f3ff0da4640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 0x7f3fd006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fec15e3a0 tx=0x7f3fdc009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:28.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.657+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fe8053bc0 con 0x7f3fec0adc70 2023-10-21T17:18:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.891+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fec06dc60 con 0x7f3fec0adc70 2023-10-21T17:18:28.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.892+0000 7f3fe1ffb640 1 -- 172.21.15.53:0/2175449808 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f3fe8017090 con 0x7f3fec0adc70 2023-10-21T17:18:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:28.893 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T17:18:28.894 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 msgr2=0x7f3fd006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 0x7f3fd006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fec15e3a0 tx=0x7f3fdc009040 comp rx=0 tx=0).stop 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 msgr2=0x7f3fec150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c390 tx=0x7f3fe800c850 comp rx=0 tx=0).stop 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 shutdown_connections 2023-10-21T17:18:28.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fd006acc0 0x7f3fd006d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fec15ab30 0x7f3fec158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.894+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fec0b07d0 0x7f3fec158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.895+0000 7f3ff25a7640 1 --2- 172.21.15.53:0/2175449808 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fec0adc70 0x7f3fec150a90 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:28.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.895+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 >> 172.21.15.53:0/2175449808 conn(0x7f3fec0cb0c0 msgr2=0x7f3fec070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:28.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.895+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 shutdown_connections 2023-10-21T17:18:28.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:28.895+0000 7f3ff25a7640 1 -- 172.21.15.53:0/2175449808 wait complete. 2023-10-21T17:18:28.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:29.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.136+0000 7f2522d16640 1 Processor -- start 2023-10-21T17:18:29.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.137+0000 7f2522d16640 1 -- start start 2023-10-21T17:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c15ab40 0x7f251c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f251c0c90b0 0x7f251c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f251c0cae70 0x7f251c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c15bf30 con 0x7f251c0cae70 2023-10-21T17:18:29.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251c0cb840 con 0x7f251c0c90b0 2023-10-21T17:18:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.138+0000 7f2522d16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c15b2b0 con 0x7f251c15ab40 2023-10-21T17:18:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f251c0cae70 0x7f251c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f251c0cae70 0x7f251c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41512/0 (socket says 172.21.15.53:41512) 2023-10-21T17:18:29.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 -- 172.21.15.53:0/2210717976 learned_addr learned my addr 172.21.15.53:0/2210717976 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 -- 172.21.15.53:0/2210717976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c15ab40 msgr2=0x7f251c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 --2- 172.21.15.53:0/2210717976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c15ab40 0x7f251c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 -- 172.21.15.53:0/2210717976 >> v2:172.21.15.124:3300/0 conn(0x7f251c0c90b0 msgr2=0x7f251c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.139+0000 7f25212b3640 1 --2- 172.21.15.53:0/2210717976 >> v2:172.21.15.124:3300/0 conn(0x7f251c0c90b0 0x7f251c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.140+0000 7f25212b3640 1 -- 172.21.15.53:0/2210717976 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c15b590 con 0x7f251c0cae70 2023-10-21T17:18:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.141+0000 7f25212b3640 1 --2- 172.21.15.53:0/2210717976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0cae70 0x7f251c0cb300 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f251c06e470 tx=0x7f251402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344ce8d979635451 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.142+0000 7f251b7fe640 1 -- 172.21.15.53:0/2210717976 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251402ca70 con 0x7f251c0cae70 2023-10-21T17:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.142+0000 7f251b7fe640 1 -- 172.21.15.53:0/2210717976 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251402cbd0 con 0x7f251c0cae70 2023-10-21T17:18:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.142+0000 7f251b7fe640 1 -- 172.21.15.53:0/2210717976 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140343f0 con 0x7f251c0cae70 2023-10-21T17:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.142+0000 7f2522d16640 1 -- 172.21.15.53:0/2210717976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0cae70 msgr2=0x7f251c0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.142+0000 7f2522d16640 1 --2- 172.21.15.53:0/2210717976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0cae70 0x7f251c0cb300 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f251c06e470 tx=0x7f251402bed0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 -- 172.21.15.53:0/2210717976 shutdown_connections 2023-10-21T17:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 --2- 172.21.15.53:0/2210717976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0cae70 0x7f251c0cb300 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 --2- 172.21.15.53:0/2210717976 >> v2:172.21.15.124:3300/0 conn(0x7f251c0c90b0 0x7f251c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 --2- 172.21.15.53:0/2210717976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c15ab40 0x7f251c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 -- 172.21.15.53:0/2210717976 >> 172.21.15.53:0/2210717976 conn(0x7f251c06a380 msgr2=0x7f251c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 -- 172.21.15.53:0/2210717976 shutdown_connections 2023-10-21T17:18:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.143+0000 7f2522d16640 1 -- 172.21.15.53:0/2210717976 wait complete. 2023-10-21T17:18:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.144+0000 7f2522d16640 1 Processor -- start 2023-10-21T17:18:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.145+0000 7f2522d16640 1 -- start start 2023-10-21T17:18:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.145+0000 7f2522d16640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2522d16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c0cae70 0x7f251c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2522d16640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f251c15ab40 0x7f251c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2522d16640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1570e0 con 0x7f251c0c90b0 2023-10-21T17:18:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2522d16640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c14b8b0 con 0x7f251c15ab40 2023-10-21T17:18:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2522d16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1564d0 con 0x7f251c0cae70 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41528/0 (socket says 172.21.15.53:41528) 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 -- 172.21.15.53:0/4255836961 learned_addr learned my addr 172.21.15.53:0/4255836961 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 -- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c0cae70 msgr2=0x7f251c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c0cae70 0x7f251c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 -- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f251c15ab40 msgr2=0x7f251c155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f251c15ab40 0x7f251c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.146+0000 7f2520ab2640 1 -- 172.21.15.53:0/4255836961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009b30 con 0x7f251c0c90b0 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f2520ab2640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f251000b4a0 tx=0x7f251000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510010610 con 0x7f251c0c90b0 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510010770 con 0x7f251c0c90b0 2023-10-21T17:18:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510010b20 con 0x7f251c0c90b0 2023-10-21T17:18:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c1567b0 con 0x7f251c0c90b0 2023-10-21T17:18:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.147+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c156d70 con 0x7f251c0c90b0 2023-10-21T17:18:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.149+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c15b850 con 0x7f251c0c90b0 2023-10-21T17:18:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.149+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 86) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2510010c80 con 0x7f251c0c90b0 2023-10-21T17:18:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.150+0000 7f25197fa640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 0x7f250006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.150+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f25100898f0 con 0x7f251c0c90b0 2023-10-21T17:18:29.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.152+0000 7f251bfff640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 0x7f250006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.153+0000 7f251bfff640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 0x7f250006d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f251c071110 tx=0x7f2508006bf0 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.156+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f251c15b850 con 0x7f251c0c90b0 2023-10-21T17:18:29.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.386+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f251c15b850 con 0x7f251c0c90b0 2023-10-21T17:18:29.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.388+0000 7f25197fa640 1 -- 172.21.15.53:0/4255836961 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f251c15b850 con 0x7f251c0c90b0 2023-10-21T17:18:29.389 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:29.389 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T16:22:13.932823+0000","modified":"2023-10-21T16:22:17.025961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9120":{"gid":9120,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/3880141158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3880141158},{"type":"v1","addr":"172.21.15.53:6835","nonce":3880141158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-21T17:18:29.389 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 78 2023-10-21T17:18:29.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 msgr2=0x7f250006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 0x7f250006d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f251c071110 tx=0x7f2508006bf0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 msgr2=0x7f251c0acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f251000b4a0 tx=0x7f251000b960 comp rx=0 tx=0).stop 2023-10-21T17:18:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 shutdown_connections 2023-10-21T17:18:29.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f250006af00 0x7f250006d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f251c15ab40 0x7f251c155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f251c0cae70 0x7f251c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 --2- 172.21.15.53:0/4255836961 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f251c0c90b0 0x7f251c0acc60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 >> 172.21.15.53:0/4255836961 conn(0x7f251c06a380 msgr2=0x7f251c0ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:29.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.390+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 shutdown_connections 2023-10-21T17:18:29.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.391+0000 7f2522d16640 1 -- 172.21.15.53:0/4255836961 wait complete. 2023-10-21T17:18:29.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.618+0000 7f5875e59640 1 Processor -- start 2023-10-21T17:18:29.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.618+0000 7f5875e59640 1 -- start start 2023-10-21T17:18:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.620+0000 7f5875e59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700b07d0 0x7f58701614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.620+0000 7f5875e59640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58700cab40 0x7f58700c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.620+0000 7f5875e59640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700c9020 con 0x7f587015cca0 2023-10-21T17:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.620+0000 7f5875e59640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700ccc20 con 0x7f58700cab40 2023-10-21T17:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.620+0000 7f5875e59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c92e0 con 0x7f58700b07d0 2023-10-21T17:18:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5875658640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58700cab40 0x7f58700c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5874e57640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f587015cca0 0x7f58700b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41630/0 (socket says 172.21.15.53:41630) 2023-10-21T17:18:29.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5874e57640 1 -- 172.21.15.53:0/3062528442 learned_addr learned my addr 172.21.15.53:0/3062528442 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f586ffff640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700b07d0 0x7f58701614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5875658640 1 -- 172.21.15.53:0/3062528442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700b07d0 msgr2=0x7f58701614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5875658640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700b07d0 0x7f58701614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5875658640 1 -- 172.21.15.53:0/3062528442 >> v1:172.21.15.53:6789/0 conn(0x7f587015cca0 legacy=0x7f58700b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f586f7fe640 1 -- 172.21.15.53:0/3062528442 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759998610 0 0) 0x7f58700c9020 con 0x7f587015cca0 2023-10-21T17:18:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.621+0000 7f5875658640 1 -- 172.21.15.53:0/3062528442 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700cc7b0 con 0x7f58700cab40 2023-10-21T17:18:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.623+0000 7f5875658640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58700cab40 0x7f58700c9770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5868009d90 tx=0x7f586802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae953e6ad63e038 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.623+0000 7f586f7fe640 1 -- 172.21.15.53:0/3062528442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586802cb30 con 0x7f58700cab40 2023-10-21T17:18:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.623+0000 7f586f7fe640 1 -- 172.21.15.53:0/3062528442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586802cc90 con 0x7f58700cab40 2023-10-21T17:18:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.623+0000 7f586f7fe640 1 -- 172.21.15.53:0/3062528442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868034440 con 0x7f58700cab40 2023-10-21T17:18:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 -- 172.21.15.53:0/3062528442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58700cab40 msgr2=0x7f58700c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58700cab40 0x7f58700c9770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f5868009d90 tx=0x7f586802bd80 comp rx=0 tx=0).stop 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 -- 172.21.15.53:0/3062528442 shutdown_connections 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58700cab40 0x7f58700c9770 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 --2- 172.21.15.53:0/3062528442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700b07d0 0x7f58701614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 -- 172.21.15.53:0/3062528442 >> 172.21.15.53:0/3062528442 conn(0x7f58700cb0c0 msgr2=0x7f58700cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 -- 172.21.15.53:0/3062528442 shutdown_connections 2023-10-21T17:18:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.624+0000 7f5875e59640 1 -- 172.21.15.53:0/3062528442 wait complete. 2023-10-21T17:18:29.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.626+0000 7f5875e59640 1 Processor -- start 2023-10-21T17:18:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.626+0000 7f5875e59640 1 -- start start 2023-10-21T17:18:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58700b07d0 0x7f587014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 0x7f58700c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c7f80 con 0x7f587015cca0 2023-10-21T17:18:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587014ebd0 con 0x7f58700b07d0 2023-10-21T17:18:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875e59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587014f040 con 0x7f58700cab40 2023-10-21T17:18:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.627+0000 7f5875658640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41546/0 (socket says 172.21.15.53:41546) 2023-10-21T17:18:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 -- 172.21.15.53:0/4041665520 learned_addr learned my addr 172.21.15.53:0/4041665520 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f586ffff640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 0x7f58700c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5874e57640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58700b07d0 0x7f587014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 -- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 msgr2=0x7f58700c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 0x7f58700c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 -- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58700b07d0 msgr2=0x7f587014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58700b07d0 0x7f587014e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 -- 172.21.15.53:0/4041665520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868009a70 con 0x7f587015cca0 2023-10-21T17:18:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f586ffff640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 0x7f58700c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.628+0000 7f5875658640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5868009eb0 tx=0x7f5868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.629+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868009450 con 0x7f587015cca0 2023-10-21T17:18:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.629+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868034440 con 0x7f587015cca0 2023-10-21T17:18:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.629+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680408a0 con 0x7f587015cca0 2023-10-21T17:18:29.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.629+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58700c58b0 con 0x7f587015cca0 2023-10-21T17:18:29.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.629+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58700c5e70 con 0x7f587015cca0 2023-10-21T17:18:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.630+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587004e160 con 0x7f587015cca0 2023-10-21T17:18:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.632+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 86) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5868040a00 con 0x7f587015cca0 2023-10-21T17:18:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.633+0000 7f586d7fa640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 0x7f585406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.633+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f58680acc00 con 0x7f587015cca0 2023-10-21T17:18:29.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.634+0000 7f5874e57640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 0x7f585406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.636+0000 7f5874e57640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 0x7f585406d410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5864000f60 tx=0x7f5864009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:29.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.637+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5868077960 con 0x7f587015cca0 2023-10-21T17:18:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.831+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58700b45c0 con 0x7f587015cca0 2023-10-21T17:18:29.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.835+0000 7f586d7fa640 1 -- 172.21.15.53:0/4041665520 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+18019 (secure 0 0 0) 0x7f5868039090 con 0x7f587015cca0 2023-10-21T17:18:29.836 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:29.836 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":171,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:27.432415+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-21T16:22:11.619571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-21T16:22:12.967359+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1493657297":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6833/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1967999416":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6832/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2914043524":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1768082194":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3657298397":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3304201904":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/2610966345":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3058690420":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3358065251":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/4258504518":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3974743798":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2638777920":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6833/870299393":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6832/870299393":"2023-10-22T16:22:28.659493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":18,"length":40},{"begin":74,"length":40}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 msgr2=0x7f585406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 0x7f585406d410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5864000f60 tx=0x7f5864009040 comp rx=0 tx=0).stop 2023-10-21T17:18:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 msgr2=0x7f58700c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5868009eb0 tx=0x7f5868004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 shutdown_connections 2023-10-21T17:18:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f585406af50 0x7f585406d410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587015cca0 0x7f58700c7410 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58700cab40 0x7f58700c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 --2- 172.21.15.53:0/4041665520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58700b07d0 0x7f587014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 >> 172.21.15.53:0/4041665520 conn(0x7f58700cb0c0 msgr2=0x7f58700709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 shutdown_connections 2023-10-21T17:18:29.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:29.837+0000 7f5875e59640 1 -- 172.21.15.53:0/4041665520 wait complete. 2023-10-21T17:18:29.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:18:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.068+0000 7fe033034640 1 Processor -- start 2023-10-21T17:18:30.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.068+0000 7fe033034640 1 -- start start 2023-10-21T17:18:30.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.069+0000 7fe033034640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe02c160660 0x7fe02c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.069+0000 7fe033034640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe02c0cb620 0x7fe02c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.069+0000 7fe033034640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c06e150 con 0x7fe02c0c9910 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.069+0000 7fe033034640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02c06de70 con 0x7fe02c160660 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.069+0000 7fe033034640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02c0c8e30 con 0x7fe02c0cb620 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe02c160660 0x7fe02c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe02c160660 0x7fe02c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41558/0 (socket says 172.21.15.53:41558) 2023-10-21T17:18:30.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 -- 172.21.15.53:0/1803223978 learned_addr learned my addr 172.21.15.53:0/1803223978 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 -- 172.21.15.53:0/1803223978 >> v2:172.21.15.124:3300/0 conn(0x7fe02c0cb620 msgr2=0x7fe02c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 --2- 172.21.15.53:0/1803223978 >> v2:172.21.15.124:3300/0 conn(0x7fe02c0cb620 0x7fe02c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 -- 172.21.15.53:0/1803223978 >> v1:172.21.15.124:6789/0 conn(0x7fe02c0c9910 legacy=0x7fe02c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.070+0000 7fe030dd0640 1 -- 172.21.15.53:0/1803223978 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0c9110 con 0x7fe02c160660 2023-10-21T17:18:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.071+0000 7fe030dd0640 1 --2- 172.21.15.53:0/1803223978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c160660 0x7fe02c070cc0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0b1760 tx=0x7fe02002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920459c26f33e1b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.071+0000 7fe02b7fe640 1 -- 172.21.15.53:0/1803223978 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200317e0 con 0x7fe02c160660 2023-10-21T17:18:30.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.071+0000 7fe02b7fe640 1 -- 172.21.15.53:0/1803223978 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020031940 con 0x7fe02c160660 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.071+0000 7fe02b7fe640 1 -- 172.21.15.53:0/1803223978 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020031cd0 con 0x7fe02c160660 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 -- 172.21.15.53:0/1803223978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c160660 msgr2=0x7fe02c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 --2- 172.21.15.53:0/1803223978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c160660 0x7fe02c070cc0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0b1760 tx=0x7fe02002b880 comp rx=0 tx=0).stop 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 -- 172.21.15.53:0/1803223978 shutdown_connections 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 --2- 172.21.15.53:0/1803223978 >> v2:172.21.15.124:3300/0 conn(0x7fe02c0cb620 0x7fe02c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 --2- 172.21.15.53:0/1803223978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c160660 0x7fe02c070cc0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 -- 172.21.15.53:0/1803223978 >> 172.21.15.53:0/1803223978 conn(0x7fe02c06a5d0 msgr2=0x7fe02c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 -- 172.21.15.53:0/1803223978 shutdown_connections 2023-10-21T17:18:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.072+0000 7fe033034640 1 -- 172.21.15.53:0/1803223978 wait complete. 2023-10-21T17:18:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.074+0000 7fe033034640 1 Processor -- start 2023-10-21T17:18:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.074+0000 7fe033034640 1 -- start start 2023-10-21T17:18:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 0x7fe02c0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe02c160660 0x7fe02c1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1542c0 con 0x7fe02c0cb620 2023-10-21T17:18:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1545a0 con 0x7fe02c160660 2023-10-21T17:18:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe033034640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c154880 con 0x7fe02c0c9910 2023-10-21T17:18:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41574/0 (socket says 172.21.15.53:41574) 2023-10-21T17:18:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe02bfff640 1 -- 172.21.15.53:0/2905261007 learned_addr learned my addr 172.21.15.53:0/2905261007 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.075+0000 7fe030dd0640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 0x7fe02c0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 -- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 msgr2=0x7fe02c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 0x7fe02c0cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 -- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe02c160660 msgr2=0x7fe02c1f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe0315d1640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe02c160660 0x7fe02c1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe02c160660 0x7fe02c1f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 -- 172.21.15.53:0/2905261007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200094d0 con 0x7fe02c0cb620 2023-10-21T17:18:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe030dd0640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 0x7fe02c0cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe02bfff640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c410 tx=0x7fe01c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0035f0 con 0x7fe02c0cb620 2023-10-21T17:18:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c003750 con 0x7fe02c0cb620 2023-10-21T17:18:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c154b60 con 0x7fe02c0cb620 2023-10-21T17:18:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c003b40 con 0x7fe02c0cb620 2023-10-21T17:18:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.076+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c0c7ad0 con 0x7fe02c0cb620 2023-10-21T17:18:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.078+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c155080 con 0x7fe02c0cb620 2023-10-21T17:18:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.079+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 86) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe01c003ca0 con 0x7fe02c0cb620 2023-10-21T17:18:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.079+0000 7fe0297fa640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 0x7fe01806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.080+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fe01c0890c0 con 0x7fe02c0cb620 2023-10-21T17:18:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.080+0000 7fe030dd0640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 0x7fe01806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.081+0000 7fe030dd0640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 0x7fe01806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe020036000 tx=0x7fe02003d040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.084+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe02c155080 con 0x7fe02c0cb620 2023-10-21T17:18:30.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.310+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe02c155080 con 0x7fe02c0cb620 2023-10-21T17:18:30.312 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:18:30.312+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:18:30.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.316+0000 7fe0297fa640 1 -- 172.21.15.53:0/2905261007 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe02c155080 con 0x7fe02c0cb620 2023-10-21T17:18:30.317 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:18:30.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.318+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 msgr2=0x7fe01806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.318+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 0x7fe01806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe020036000 tx=0x7fe02003d040 comp rx=0 tx=0).stop 2023-10-21T17:18:30.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.318+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 msgr2=0x7fe02c0cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.318+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c410 tx=0x7fe01c00c8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 shutdown_connections 2023-10-21T17:18:30.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe01806af30 0x7fe01806d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe02c160660 0x7fe02c1f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe02c0cb620 0x7fe02c0cdec0 secure :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c410 tx=0x7fe01c00c8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 --2- 172.21.15.53:0/2905261007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe02c0c9910 0x7fe02c0cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 >> 172.21.15.53:0/2905261007 conn(0x7fe02c06a5d0 msgr2=0x7fe02c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:30.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.319+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 shutdown_connections 2023-10-21T17:18:30.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.320+0000 7fe033034640 1 -- 172.21.15.53:0/2905261007 wait complete. 2023-10-21T17:18:30.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:18:30.422 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:18:30.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.534+0000 7f55119c0640 1 Processor -- start 2023-10-21T17:18:30.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.534+0000 7f55119c0640 1 -- start start 2023-10-21T17:18:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f55119c0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f550c0afbe0 0x7f550c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f55119c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 0x7f550c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f55119c0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c0cbad0 con 0x7f550c06dfc0 2023-10-21T17:18:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f55119c0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c0b2620 con 0x7f550c0afbe0 2023-10-21T17:18:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f55119c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b2060 con 0x7f550c0cb130 2023-10-21T17:18:30.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f550a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f550c06dfc0 0x7f550c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41632/0 (socket says 172.21.15.53:41632) 2023-10-21T17:18:30.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f550a7fc640 1 -- 172.21.15.53:0/725862806 learned_addr learned my addr 172.21.15.53:0/725862806 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:30.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f550b7fe640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 0x7f550c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.536+0000 7f550affd640 1 --2- 172.21.15.53:0/725862806 >> v2:172.21.15.124:3300/0 conn(0x7f550c0afbe0 0x7f550c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430663499 0 0) 0x7f550c0cbad0 con 0x7f550c06dfc0 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f4003520 con 0x7f550c06dfc0 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123071655 0 0) 0x7f54f4003520 con 0x7f550c06dfc0 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 msgr2=0x7f550c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 0x7f550c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 >> v2:172.21.15.124:3300/0 conn(0x7f550c0afbe0 msgr2=0x7f550c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 --2- 172.21.15.53:0/725862806 >> v2:172.21.15.124:3300/0 conn(0x7f550c0afbe0 0x7f550c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0b2340 con 0x7f550c06dfc0 2023-10-21T17:18:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5500001670 con 0x7f550c06dfc0 2023-10-21T17:18:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.537+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 >> v1:172.21.15.53:6789/0 conn(0x7f550c06dfc0 legacy=0x7f550c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54f4003970 0x7f54f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 0x7f54f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f54f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b2620 con 0x7f54f4004d00 2023-10-21T17:18:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b2060 con 0x7f54f4003970 2023-10-21T17:18:30.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f400b980 con 0x7f550c0cb130 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f54f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 -- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 msgr2=0x7f54f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cb130 0x7f54f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 -- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54f4003970 msgr2=0x7f54f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54f4003970 0x7f54f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.538+0000 7f550b7fe640 1 -- 172.21.15.53:0/725862806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4003520 con 0x7f54f4004d00 2023-10-21T17:18:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f550b7fe640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f54f4005180 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f550400e170 tx=0x7f54f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feec02bc75b694f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550402cc20 con 0x7f54f4004d00 2023-10-21T17:18:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400f040 con 0x7f54f4004d00 2023-10-21T17:18:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f5509ffb640 1 -- 172.21.15.53:0/725862806 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504034440 con 0x7f54f4004d00 2023-10-21T17:18:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f55119c0640 1 -- 172.21.15.53:0/725862806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 msgr2=0x7f54f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.540+0000 7f55119c0640 1 --2- 172.21.15.53:0/725862806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f54f4005180 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f550400e170 tx=0x7f54f40057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.541+0000 7f550b7fe640 1 -- 172.21.15.53:0/725862806 reap_dead start 2023-10-21T17:18:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.541+0000 7f55119c0640 1 -- 172.21.15.53:0/725862806 shutdown_connections 2023-10-21T17:18:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.541+0000 7f55119c0640 1 -- 172.21.15.53:0/725862806 >> 172.21.15.53:0/725862806 conn(0x7f550c06a5d0 msgr2=0x7f550c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.541+0000 7f55119c0640 1 -- 172.21.15.53:0/725862806 shutdown_connections 2023-10-21T17:18:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.541+0000 7f55119c0640 1 -- 172.21.15.53:0/725862806 wait complete. 2023-10-21T17:18:30.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.542+0000 7f55119c0640 1 Processor -- start 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.543+0000 7f55119c0640 1 -- start start 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 0x7f550c15c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0ca2f0 0x7f550c0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0cacc0 con 0x7f54f4004d00 2023-10-21T17:18:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c15dae0 con 0x7f550c0c9c80 2023-10-21T17:18:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f55119c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1f26b0 con 0x7f550c0ca2f0 2023-10-21T17:18:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41606/0 (socket says 172.21.15.53:41606) 2023-10-21T17:18:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 -- 172.21.15.53:0/1757814982 learned_addr learned my addr 172.21.15.53:0/1757814982 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550a7fc640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 0x7f550c15c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 -- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0ca2f0 msgr2=0x7f550c0ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550b7fe640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0ca2f0 0x7f550c0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0ca2f0 0x7f550c0ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 -- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 msgr2=0x7f550c15c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 0x7f550c15c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550affd640 1 -- 172.21.15.53:0/1757814982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c15ced0 con 0x7f54f4004d00 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.544+0000 7f550a7fc640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 0x7f550c15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f550affd640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e7c0 tx=0x7f54fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00eed0 con 0x7f54f4004d00 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc00c940 con 0x7f54f4004d00 2023-10-21T17:18:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00ccf0 con 0x7f54f4004d00 2023-10-21T17:18:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c15d1b0 con 0x7f54f4004d00 2023-10-21T17:18:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.545+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c15d770 con 0x7f54f4004d00 2023-10-21T17:18:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.546+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550c070610 con 0x7f54f4004d00 2023-10-21T17:18:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.547+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 86) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f54fc013470 con 0x7f54f4004d00 2023-10-21T17:18:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.548+0000 7f54f3fff640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 0x7f54f806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.549+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f54fc089c50 con 0x7f54f4004d00 2023-10-21T17:18:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.549+0000 7f550a7fc640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 0x7f54f806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.550+0000 7f550a7fc640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 0x7f54f806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f550c15a640 tx=0x7f5500009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.553+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f550c070610 con 0x7f54f4004d00 2023-10-21T17:18:30.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:30.796+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f550c070610 con 0x7f54f4004d00 2023-10-21T17:18:31.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.319+0000 7f54f3fff640 1 -- 172.21.15.53:0/1757814982 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f550c070610 con 0x7f54f4004d00 2023-10-21T17:18:31.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 msgr2=0x7f54f806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 0x7f54f806d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f550c15a640 tx=0x7f5500009040 comp rx=0 tx=0).stop 2023-10-21T17:18:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 msgr2=0x7f550c1f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e7c0 tx=0x7f54fc00ec80 comp rx=0 tx=0).stop 2023-10-21T17:18:31.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 shutdown_connections 2023-10-21T17:18:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f54f806af30 0x7f54f806d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0ca2f0 0x7f550c0ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.322+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0c9c80 0x7f550c15c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.323+0000 7f55119c0640 1 --2- 172.21.15.53:0/1757814982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54f4004d00 0x7f550c1f2170 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.323+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 >> 172.21.15.53:0/1757814982 conn(0x7f550c06a5d0 msgr2=0x7f550c0cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.323+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 shutdown_connections 2023-10-21T17:18:31.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.323+0000 7f55119c0640 1 -- 172.21.15.53:0/1757814982 wait complete. 2023-10-21T17:18:31.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:18:31.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.557+0000 7f6626185640 1 Processor -- start 2023-10-21T17:18:31.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.557+0000 7f6626185640 1 -- start start 2023-10-21T17:18:31.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6626185640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f66200cf2b0 0x7f66200d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6626185640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf690 0x7f66200ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6626185640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66200cd520 con 0x7f66200d1100 2023-10-21T17:18:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6626185640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66200d1910 con 0x7f66200cf2b0 2023-10-21T17:18:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6626185640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200ce080 con 0x7f66200cf690 2023-10-21T17:18:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6625183640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f66200cf2b0 0x7f66200d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6625183640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f66200cf2b0 0x7f66200d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41614/0 (socket says 172.21.15.53:41614) 2023-10-21T17:18:31.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6625984640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f66200d1100 0x7f662015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:41648/0 (socket says 172.21.15.53:41648) 2023-10-21T17:18:31.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.559+0000 7f6625183640 1 -- 172.21.15.53:0/3536559997 learned_addr learned my addr 172.21.15.53:0/3536559997 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:31.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.560+0000 7f6624982640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf690 0x7f66200ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.560+0000 7f6625183640 1 -- 172.21.15.53:0/3536559997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf690 msgr2=0x7f66200ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:31.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.560+0000 7f6625183640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf690 0x7f66200ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.560+0000 7f6625183640 1 -- 172.21.15.53:0/3536559997 >> v1:172.21.15.53:6789/0 conn(0x7f66200d1100 legacy=0x7f662015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.560+0000 7f6625183640 1 -- 172.21.15.53:0/3536559997 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200cd750 con 0x7f66200cf2b0 2023-10-21T17:18:31.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.561+0000 7f6625183640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200cf2b0 0x7f66200d34d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f661c009870 tx=0x7f661c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e70ed1e55fcf042 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:31.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.561+0000 7f6617fff640 1 -- 172.21.15.53:0/3536559997 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c031640 con 0x7f66200cf2b0 2023-10-21T17:18:31.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.561+0000 7f6617fff640 1 -- 172.21.15.53:0/3536559997 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c0317a0 con 0x7f66200cf2b0 2023-10-21T17:18:31.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.561+0000 7f6617fff640 1 -- 172.21.15.53:0/3536559997 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c031b30 con 0x7f66200cf2b0 2023-10-21T17:18:31.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 -- 172.21.15.53:0/3536559997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200cf2b0 msgr2=0x7f66200d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200cf2b0 0x7f66200d34d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f661c009870 tx=0x7f661c02b8f0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 -- 172.21.15.53:0/3536559997 shutdown_connections 2023-10-21T17:18:31.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf690 0x7f66200ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 --2- 172.21.15.53:0/3536559997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200cf2b0 0x7f66200d34d0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 -- 172.21.15.53:0/3536559997 >> 172.21.15.53:0/3536559997 conn(0x7f662009bc40 msgr2=0x7f66200d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:31.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 -- 172.21.15.53:0/3536559997 shutdown_connections 2023-10-21T17:18:31.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.562+0000 7f6626185640 1 -- 172.21.15.53:0/3536559997 wait complete. 2023-10-21T17:18:31.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.564+0000 7f6626185640 1 Processor -- start 2023-10-21T17:18:31.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.564+0000 7f6626185640 1 -- start start 2023-10-21T17:18:31.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 0x7f662015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 0x7f662009c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662009caf0 con 0x7f66200d1100 2023-10-21T17:18:31.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662009f730 con 0x7f66200cf690 2023-10-21T17:18:31.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6626185640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662009eb20 con 0x7f66200cf2b0 2023-10-21T17:18:31.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.565+0000 7f6625984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41620/0 (socket says 172.21.15.53:41620) 2023-10-21T17:18:31.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 -- 172.21.15.53:0/1186636992 learned_addr learned my addr 172.21.15.53:0/1186636992 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625183640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 0x7f662015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6624982640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 0x7f662009c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 -- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 msgr2=0x7f662015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 0x7f662015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 -- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 msgr2=0x7f662009c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 0x7f662009c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 -- 172.21.15.53:0/1186636992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0094d0 con 0x7f66200d1100 2023-10-21T17:18:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625183640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 0x7f662015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6624982640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 0x7f662009c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:31.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.566+0000 7f6625984640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f661800c7b0 tx=0x7f661800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.567+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800a640 con 0x7f66200d1100 2023-10-21T17:18:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.567+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661800a7a0 con 0x7f66200d1100 2023-10-21T17:18:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.567+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800ab10 con 0x7f66200d1100 2023-10-21T17:18:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.567+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662009ee00 con 0x7f66200d1100 2023-10-21T17:18:31.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.567+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662009f3c0 con 0x7f66200d1100 2023-10-21T17:18:31.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.568+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6620074150 con 0x7f66200d1100 2023-10-21T17:18:31.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.569+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f661800ac70 con 0x7f66200d1100 2023-10-21T17:18:31.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.570+0000 7f6615ffb640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:31.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.570+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f6618089340 con 0x7f66200d1100 2023-10-21T17:18:31.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.571+0000 7f6625183640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:31.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.572+0000 7f6625183640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 0x7f660406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f661c036000 tx=0x7f661c03c040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:31.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.575+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6618053ed0 con 0x7f66200d1100 2023-10-21T17:18:31.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:31.768+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f662009f870 con 0x7f66200d1100 2023-10-21T17:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.333+0000 7f6615ffb640 1 -- 172.21.15.53:0/1186636992 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v173) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66180574f0 con 0x7f66200d1100 2023-10-21T17:18:32.334 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:18:32.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.336+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 msgr2=0x7f660406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:32.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.336+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 0x7f660406d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f661c036000 tx=0x7f661c03c040 comp rx=0 tx=0).stop 2023-10-21T17:18:32.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.336+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 msgr2=0x7f662009ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:32.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.336+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f661800c7b0 tx=0x7f661800cc70 comp rx=0 tx=0).stop 2023-10-21T17:18:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 shutdown_connections 2023-10-21T17:18:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200d1100 0x7f662009ce90 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66200cf690 0x7f662009c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 --2- 172.21.15.53:0/1186636992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cf2b0 0x7f662015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 >> 172.21.15.53:0/1186636992 conn(0x7f662009bc40 msgr2=0x7f66200b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:32.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 shutdown_connections 2023-10-21T17:18:32.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.337+0000 7f6626185640 1 -- 172.21.15.53:0/1186636992 wait complete. 2023-10-21T17:18:32.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.589+0000 7f249cda7640 1 Processor -- start 2023-10-21T17:18:32.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.589+0000 7f249cda7640 1 -- start start 2023-10-21T17:18:32.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.590+0000 7f249cda7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24980b0680 0x7f249815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f249cda7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24980ca680 0x7f249815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f249cda7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24980cab60 con 0x7f249815a820 2023-10-21T17:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f249cda7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980cb960 con 0x7f24980ca680 2023-10-21T17:18:32.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f249cda7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980cbdf0 con 0x7f24980b0680 2023-10-21T17:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f2497fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24980ca680 0x7f249815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f2497fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f24980ca680 0x7f249815ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41630/0 (socket says 172.21.15.53:41630) 2023-10-21T17:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f2497fff640 1 -- 172.21.15.53:0/3300134977 learned_addr learned my addr 172.21.15.53:0/3300134977 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.591+0000 7f2496ffd640 1 --2- 172.21.15.53:0/3300134977 >> v2:172.21.15.124:3300/0 conn(0x7f24980b0680 0x7f249815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.592+0000 7f2497fff640 1 -- 172.21.15.53:0/3300134977 >> v2:172.21.15.124:3300/0 conn(0x7f24980b0680 msgr2=0x7f249815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.592+0000 7f2497fff640 1 --2- 172.21.15.53:0/3300134977 >> v2:172.21.15.124:3300/0 conn(0x7f24980b0680 0x7f249815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.592+0000 7f2497fff640 1 -- 172.21.15.53:0/3300134977 >> v1:172.21.15.124:6789/0 conn(0x7f249815a820 legacy=0x7f24980aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.592+0000 7f2497fff640 1 -- 172.21.15.53:0/3300134977 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980cb030 con 0x7f24980ca680 2023-10-21T17:18:32.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.593+0000 7f2497fff640 1 --2- 172.21.15.53:0/3300134977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980ca680 0x7f249815ea80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f24980718e0 tx=0x7f249002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed381f364a5fbd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.593+0000 7f24967fc640 1 -- 172.21.15.53:0/3300134977 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249002c9f0 con 0x7f24980ca680 2023-10-21T17:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.593+0000 7f24967fc640 1 -- 172.21.15.53:0/3300134977 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249002cb50 con 0x7f24980ca680 2023-10-21T17:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.593+0000 7f24967fc640 1 -- 172.21.15.53:0/3300134977 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24900343f0 con 0x7f24980ca680 2023-10-21T17:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 -- 172.21.15.53:0/3300134977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980ca680 msgr2=0x7f249815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 --2- 172.21.15.53:0/3300134977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980ca680 0x7f249815ea80 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f24980718e0 tx=0x7f249002be50 comp rx=0 tx=0).stop 2023-10-21T17:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 -- 172.21.15.53:0/3300134977 shutdown_connections 2023-10-21T17:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 --2- 172.21.15.53:0/3300134977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980ca680 0x7f249815ea80 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 --2- 172.21.15.53:0/3300134977 >> v2:172.21.15.124:3300/0 conn(0x7f24980b0680 0x7f249815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 -- 172.21.15.53:0/3300134977 >> 172.21.15.53:0/3300134977 conn(0x7f249806c050 msgr2=0x7f249806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:32.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 -- 172.21.15.53:0/3300134977 shutdown_connections 2023-10-21T17:18:32.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.594+0000 7f249cda7640 1 -- 172.21.15.53:0/3300134977 wait complete. 2023-10-21T17:18:32.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.596+0000 7f249cda7640 1 Processor -- start 2023-10-21T17:18:32.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.596+0000 7f249cda7640 1 -- start start 2023-10-21T17:18:32.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980ca680 0x7f249815d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249815a820 0x7f24980ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498158d70 con 0x7f24980b0680 2023-10-21T17:18:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981e6f70 con 0x7f249815a820 2023-10-21T17:18:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f249cda7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2498158160 con 0x7f24980ca680 2023-10-21T17:18:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f24977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f24977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41636/0 (socket says 172.21.15.53:41636) 2023-10-21T17:18:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.597+0000 7f24977fe640 1 -- 172.21.15.53:0/679394266 learned_addr learned my addr 172.21.15.53:0/679394266 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f2496ffd640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980ca680 0x7f249815d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f2497fff640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249815a820 0x7f24980ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f24977fe640 1 -- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980ca680 msgr2=0x7f249815d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f24977fe640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980ca680 0x7f249815d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f24977fe640 1 -- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249815a820 msgr2=0x7f24980ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f24977fe640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249815a820 0x7f24980ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.598+0000 7f24977fe640 1 -- 172.21.15.53:0/679394266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490009b30 con 0x7f24980b0680 2023-10-21T17:18:32.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f24977fe640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f248800b420 tx=0x7f248800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:32.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011830 con 0x7f24980b0680 2023-10-21T17:18:32.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488011990 con 0x7f24980b0680 2023-10-21T17:18:32.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011d40 con 0x7f24980b0680 2023-10-21T17:18:32.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498158440 con 0x7f24980b0680 2023-10-21T17:18:32.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.599+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2498158a00 con 0x7f24980b0680 2023-10-21T17:18:32.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.600+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2498158f20 con 0x7f24980b0680 2023-10-21T17:18:32.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.601+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f248801b460 con 0x7f24980b0680 2023-10-21T17:18:32.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.602+0000 7f247ffff640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 0x7f248406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:32.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.602+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f24880896b0 con 0x7f24980b0680 2023-10-21T17:18:32.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.603+0000 7f2496ffd640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 0x7f248406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:32.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.604+0000 7f2496ffd640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 0x7f248406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24980b0130 tx=0x7f248c008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:32.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.606+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2498158f20 con 0x7f24980b0680 2023-10-21T17:18:32.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:32.799+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24980b1310 con 0x7f24980b0680 2023-10-21T17:18:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.809+0000 7f247ffff640 1 -- 172.21.15.53:0/679394266 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v175) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2488054330 con 0x7f24980b0680 2023-10-21T17:18:33.811 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:18:33.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.816+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 msgr2=0x7f248406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.816+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 0x7f248406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24980b0130 tx=0x7f248c008040 comp rx=0 tx=0).stop 2023-10-21T17:18:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.816+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 msgr2=0x7f24981e5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:33.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.816+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f248800b420 tx=0x7f248800b8e0 comp rx=0 tx=0).stop 2023-10-21T17:18:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 shutdown_connections 2023-10-21T17:18:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f248406af90 0x7f248406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:33.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f249815a820 0x7f24980ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980ca680 0x7f249815d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 --2- 172.21.15.53:0/679394266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24980b0680 0x7f24981e5070 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:33.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.820+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 >> 172.21.15.53:0/679394266 conn(0x7f249806c050 msgr2=0x7f2498159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.821+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 shutdown_connections 2023-10-21T17:18:33.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:33.822+0000 7f249cda7640 1 -- 172.21.15.53:0/679394266 wait complete. 2023-10-21T17:18:33.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:18:34.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.047+0000 7f1cc359e640 1 Processor -- start 2023-10-21T17:18:34.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.047+0000 7f1cc359e640 1 -- start start 2023-10-21T17:18:34.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.048+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 0x7f1cc415e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.048+0000 7f1cc359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1cc40cb6f0 0x7f1cc40ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.048+0000 7f1cc359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc40b63e0 con 0x7f1cc40ce810 2023-10-21T17:18:34.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.048+0000 7f1cc359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc415b020 con 0x7f1cc40cb6f0 2023-10-21T17:18:34.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.048+0000 7f1cc359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40cbb70 con 0x7f1cc41585a0 2023-10-21T17:18:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1cc40ce810 0x7f1cc415b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54202/0 (socket says 172.21.15.53:54202) 2023-10-21T17:18:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc2d9d640 1 -- 172.21.15.53:0/1454291415 learned_addr learned my addr 172.21.15.53:0/1454291415 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:34.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc259c640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 0x7f1cc415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc1d9b640 1 --2- 172.21.15.53:0/1454291415 >> v2:172.21.15.124:3300/0 conn(0x7f1cc40cb6f0 0x7f1cc40ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552660173 0 0) 0x7f1cc40b63e0 con 0x7f1cc40ce810 2023-10-21T17:18:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.049+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb0003520 con 0x7f1cc40ce810 2023-10-21T17:18:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1240628967 0 0) 0x7f1cb0003520 con 0x7f1cc40ce810 2023-10-21T17:18:34.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 msgr2=0x7f1cc415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 0x7f1cc415e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 >> v2:172.21.15.124:3300/0 conn(0x7f1cc40cb6f0 msgr2=0x7f1cc40ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 --2- 172.21.15.53:0/1454291415 >> v2:172.21.15.124:3300/0 conn(0x7f1cc40cb6f0 0x7f1cc40ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc415f360 con 0x7f1cc40ce810 2023-10-21T17:18:34.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1cbc001670 con 0x7f1cc40ce810 2023-10-21T17:18:34.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 >> v1:172.21.15.53:6789/0 conn(0x7f1cc40ce810 legacy=0x7f1cc415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb0003970 0x7f1cb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cb00047c0 0x7f1cb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 0x7f1cb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc415b020 con 0x7f1cb0003970 2023-10-21T17:18:34.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40cbb70 con 0x7f1cb00047c0 2023-10-21T17:18:34.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000b980 con 0x7f1cc41585a0 2023-10-21T17:18:34.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.050+0000 7f1cc259c640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb0003970 0x7f1cb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc1d9b640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cb00047c0 0x7f1cb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc259c640 1 -- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 msgr2=0x7f1cb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc259c640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cc41585a0 0x7f1cb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc259c640 1 -- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cb00047c0 msgr2=0x7f1cb0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc259c640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cb00047c0 0x7f1cb0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.051+0000 7f1cc259c640 1 -- 172.21.15.53:0/1454291415 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0003520 con 0x7f1cb0003970 2023-10-21T17:18:34.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.052+0000 7f1cc259c640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb0003970 0x7f1cb0003d90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fc0 tx=0x7f1cb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c941432a9245f1f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.052+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb402cc40 con 0x7f1cb0003970 2023-10-21T17:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.052+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb400f040 con 0x7f1cb0003970 2023-10-21T17:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.052+0000 7f1cc159a640 1 -- 172.21.15.53:0/1454291415 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4034440 con 0x7f1cb0003970 2023-10-21T17:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 -- 172.21.15.53:0/1454291415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb0003970 msgr2=0x7f1cb0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 --2- 172.21.15.53:0/1454291415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cb0003970 0x7f1cb0003d90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fc0 tx=0x7f1cb00057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc2d9d640 1 -- 172.21.15.53:0/1454291415 reap_dead start 2023-10-21T17:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 -- 172.21.15.53:0/1454291415 shutdown_connections 2023-10-21T17:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 -- 172.21.15.53:0/1454291415 >> 172.21.15.53:0/1454291415 conn(0x7f1cc40cd180 msgr2=0x7f1cc40cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 -- 172.21.15.53:0/1454291415 shutdown_connections 2023-10-21T17:18:34.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.053+0000 7f1cc359e640 1 -- 172.21.15.53:0/1454291415 wait complete. 2023-10-21T17:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.055+0000 7f1cc359e640 1 Processor -- start 2023-10-21T17:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.055+0000 7f1cc359e640 1 -- start start 2023-10-21T17:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cb0003970 0x7f1cc41573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cc4154790 0x7f1cc4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c4bf0 con 0x7f1cc40c4230 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4156430 con 0x7f1cc4154790 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4156960 con 0x7f1cb0003970 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47548/0 (socket says 172.21.15.53:47548) 2023-10-21T17:18:34.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.056+0000 7f1cc1d9b640 1 -- 172.21.15.53:0/946912424 learned_addr learned my addr 172.21.15.53:0/946912424 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc2d9d640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cc4154790 0x7f1cc4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc259c640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cb0003970 0x7f1cc41573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 -- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cb0003970 msgr2=0x7f1cc41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cb0003970 0x7f1cc41573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 -- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cc4154790 msgr2=0x7f1cc4154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cc4154790 0x7f1cc4154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 -- 172.21.15.53:0/946912424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4155160 con 0x7f1cc40c4230 2023-10-21T17:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc1d9b640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0116d0 tx=0x7f1cac011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00f960 con 0x7f1cc40c4230 2023-10-21T17:18:34.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac00fac0 con 0x7f1cc40c4230 2023-10-21T17:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00fe30 con 0x7f1cc40c4230 2023-10-21T17:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.057+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc4156680 con 0x7f1cc40c4230 2023-10-21T17:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.058+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc40b0550 con 0x7f1cc40c4230 2023-10-21T17:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.059+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc4074150 con 0x7f1cc40c4230 2023-10-21T17:18:34.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.060+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1cac015460 con 0x7f1cc40c4230 2023-10-21T17:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.061+0000 7f1cbaffd640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 0x7f1ca406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.061+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1cac089d80 con 0x7f1cc40c4230 2023-10-21T17:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.061+0000 7f1cc259c640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 0x7f1ca406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.063+0000 7f1cc259c640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 0x7f1ca406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4036000 tx=0x7f1cb402e000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.065+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cac054c50 con 0x7f1cc40c4230 2023-10-21T17:18:34.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.263+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1cc40cdf40 con 0x7f1cc40c4230 2023-10-21T17:18:34.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.269+0000 7f1cbaffd640 1 -- 172.21.15.53:0/946912424 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1cac058270 con 0x7f1cc40c4230 2023-10-21T17:18:34.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 msgr2=0x7f1ca406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 0x7f1ca406d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4036000 tx=0x7f1cb402e000 comp rx=0 tx=0).stop 2023-10-21T17:18:34.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 msgr2=0x7f1cc40c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0116d0 tx=0x7f1cac011b90 comp rx=0 tx=0).stop 2023-10-21T17:18:34.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 shutdown_connections 2023-10-21T17:18:34.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1ca406af90 0x7f1ca406d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1cc4154790 0x7f1cc4154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1cc40c4230 0x7f1cc40c46b0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 --2- 172.21.15.53:0/946912424 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1cb0003970 0x7f1cc41573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.272+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 >> 172.21.15.53:0/946912424 conn(0x7f1cc40cd180 msgr2=0x7f1cc4158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:34.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.273+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 shutdown_connections 2023-10-21T17:18:34.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.273+0000 7f1cc359e640 1 -- 172.21.15.53:0/946912424 wait complete. 2023-10-21T17:18:34.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:18:34.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.523+0000 7f71a9793640 1 Processor -- start 2023-10-21T17:18:34.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.523+0000 7f71a9793640 1 -- start start 2023-10-21T17:18:34.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a9793640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f71a40c8fb0 0x7f71a40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a9793640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f71a415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a9793640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a40b0260 con 0x7f71a40c9390 2023-10-21T17:18:34.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a9793640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a415cfc0 con 0x7f71a40c8fb0 2023-10-21T17:18:34.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a9793640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41612d0 con 0x7f71a415c630 2023-10-21T17:18:34.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f71a40c9390 0x7f71a4160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54218/0 (socket says 172.21.15.53:54218) 2023-10-21T17:18:34.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.525+0000 7f71a37fe640 1 -- 172.21.15.53:0/3098934629 learned_addr learned my addr 172.21.15.53:0/3098934629 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:34.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a8f92640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f71a415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182327745 0 0) 0x7f71a40b0260 con 0x7f71a40c9390 2023-10-21T17:18:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a3fff640 1 --2- 172.21.15.53:0/3098934629 >> v2:172.21.15.124:3300/0 conn(0x7f71a40c8fb0 0x7f71a40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c003520 con 0x7f71a40c9390 2023-10-21T17:18:34.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201650042 0 0) 0x7f718c003520 con 0x7f71a40c9390 2023-10-21T17:18:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 msgr2=0x7f71a415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f71a415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 >> v2:172.21.15.124:3300/0 conn(0x7f71a40c8fb0 msgr2=0x7f71a40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 --2- 172.21.15.53:0/3098934629 >> v2:172.21.15.124:3300/0 conn(0x7f71a40c8fb0 0x7f71a40cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4161590 con 0x7f71a40c9390 2023-10-21T17:18:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7198001670 con 0x7f71a40c9390 2023-10-21T17:18:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 >> v1:172.21.15.53:6789/0 conn(0x7f71a40c9390 legacy=0x7f71a4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f718c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a415cfc0 con 0x7f718c0047c0 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41612d0 con 0x7f718c003970 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.526+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00ba60 con 0x7f71a415c630 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a3fff640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a8f92640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f718c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 -- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 msgr2=0x7f718c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a415c630 0x7f718c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 -- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c003970 msgr2=0x7f718c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:34.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a37fe640 1 -- 172.21.15.53:0/3098934629 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c003520 con 0x7f718c0047c0 2023-10-21T17:18:34.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.527+0000 7f71a3fff640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c003970 0x7f718c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:34.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.528+0000 7f71a37fe640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f719800c210 tx=0x7f718c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e75e7462374766 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198005ad0 con 0x7f718c0047c0 2023-10-21T17:18:34.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198005c30 con 0x7f718c0047c0 2023-10-21T17:18:34.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a2ffd640 1 -- 172.21.15.53:0/3098934629 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719802c8d0 con 0x7f718c0047c0 2023-10-21T17:18:34.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a9793640 1 -- 172.21.15.53:0/3098934629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c0047c0 msgr2=0x7f718c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a9793640 1 --2- 172.21.15.53:0/3098934629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f718c0047c0 0x7f718c004c40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f719800c210 tx=0x7f718c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a8f92640 1 -- 172.21.15.53:0/3098934629 reap_dead start 2023-10-21T17:18:34.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a9793640 1 -- 172.21.15.53:0/3098934629 shutdown_connections 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.529+0000 7f71a9793640 1 -- 172.21.15.53:0/3098934629 >> 172.21.15.53:0/3098934629 conn(0x7f71a406c010 msgr2=0x7f71a406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.530+0000 7f71a9793640 1 -- 172.21.15.53:0/3098934629 shutdown_connections 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.530+0000 7f71a9793640 1 -- 172.21.15.53:0/3098934629 wait complete. 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.531+0000 7f71a9793640 1 Processor -- start 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.532+0000 7f71a9793640 1 -- start start 2023-10-21T17:18:34.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.532+0000 7f71a9793640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 0x7f71a41f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a9793640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a4150c50 0x7f71a415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a9793640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a9793640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a415e9d0 con 0x7f71a4070bb0 2023-10-21T17:18:34.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a9793640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4071570 con 0x7f718c0047c0 2023-10-21T17:18:34.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a9793640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a415e660 con 0x7f71a4150c50 2023-10-21T17:18:34.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47578/0 (socket says 172.21.15.53:47578) 2023-10-21T17:18:34.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 -- 172.21.15.53:0/1063503098 learned_addr learned my addr 172.21.15.53:0/1063503098 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:34.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a3fff640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 0x7f71a41f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 -- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a4150c50 msgr2=0x7f71a415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a4150c50 0x7f71a415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 -- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 msgr2=0x7f71a41f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 0x7f71a41f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a8f92640 1 -- 172.21.15.53:0/1063503098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a415d950 con 0x7f71a4070bb0 2023-10-21T17:18:34.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.533+0000 7f71a3fff640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 0x7f71a41f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a8f92640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f71a40cf5e0 tx=0x7f719c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c017400 con 0x7f71a4070bb0 2023-10-21T17:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c011040 con 0x7f71a4070bb0 2023-10-21T17:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c0177c0 con 0x7f71a4070bb0 2023-10-21T17:18:34.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a415dc30 con 0x7f71a4070bb0 2023-10-21T17:18:34.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.534+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a415d640 con 0x7f71a4070bb0 2023-10-21T17:18:34.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.535+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a4074150 con 0x7f71a4070bb0 2023-10-21T17:18:34.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.536+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f719c021440 con 0x7f71a4070bb0 2023-10-21T17:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.538+0000 7f71a0ff9640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 0x7f719006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.538+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f719c089ac0 con 0x7f71a4070bb0 2023-10-21T17:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.539+0000 7f71a3fff640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 0x7f719006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:34.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.540+0000 7f71a3fff640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 0x7f719006d4a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71a40c8a20 tx=0x7f7194008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:34.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.542+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f719c054850 con 0x7f71a4070bb0 2023-10-21T17:18:34.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.737+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f71a40af910 con 0x7f71a4070bb0 2023-10-21T17:18:34.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.744+0000 7f71a0ff9640 1 -- 172.21.15.53:0/1063503098 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7f719c057e70 con 0x7f71a4070bb0 2023-10-21T17:18:34.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 msgr2=0x7f719006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 0x7f719006d4a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f71a40c8a20 tx=0x7f7194008040 comp rx=0 tx=0).stop 2023-10-21T17:18:34.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 msgr2=0x7f71a4071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:34.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f71a40cf5e0 tx=0x7f719c00f4a0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 shutdown_connections 2023-10-21T17:18:34.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f719006afe0 0x7f719006d4a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71a4070bb0 0x7f71a4071030 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71a4150c50 0x7f71a415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 --2- 172.21.15.53:0/1063503098 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f718c0047c0 0x7f71a41f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:34.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 >> 172.21.15.53:0/1063503098 conn(0x7f71a406c010 msgr2=0x7f71a40b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:34.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 shutdown_connections 2023-10-21T17:18:34.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:34.747+0000 7f71a9793640 1 -- 172.21.15.53:0/1063503098 wait complete. 2023-10-21T17:18:34.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:18:35.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.000+0000 7fd892ee0640 1 Processor -- start 2023-10-21T17:18:35.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.001+0000 7fd892ee0640 1 -- start start 2023-10-21T17:18:35.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd892ee0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd88c161690 0x7fd88c06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd892ee0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88c0cb150 0x7fd88c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd892ee0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c06cc50 con 0x7fd88c0702b0 2023-10-21T17:18:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd892ee0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0cbb40 con 0x7fd88c0cb150 2023-10-21T17:18:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd892ee0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c06cf30 con 0x7fd88c161690 2023-10-21T17:18:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.002+0000 7fd88bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88c0cb150 0x7fd88c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd88c0cb150 0x7fd88c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47580/0 (socket says 172.21.15.53:47580) 2023-10-21T17:18:35.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 -- 172.21.15.53:0/1661742604 learned_addr learned my addr 172.21.15.53:0/1661742604 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:35.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd89147d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd88c0702b0 0x7fd88c070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54226/0 (socket says 172.21.15.53:54226) 2023-10-21T17:18:35.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd890c7c640 1 --2- 172.21.15.53:0/1661742604 >> v2:172.21.15.124:3300/0 conn(0x7fd88c161690 0x7fd88c06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 -- 172.21.15.53:0/1661742604 >> v2:172.21.15.124:3300/0 conn(0x7fd88c161690 msgr2=0x7fd88c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:35.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 --2- 172.21.15.53:0/1661742604 >> v2:172.21.15.124:3300/0 conn(0x7fd88c161690 0x7fd88c06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 -- 172.21.15.53:0/1661742604 >> v1:172.21.15.53:6789/0 conn(0x7fd88c0702b0 legacy=0x7fd88c070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88bfff640 1 -- 172.21.15.53:0/1661742604 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c06d210 con 0x7fd88c0cb150 2023-10-21T17:18:35.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.003+0000 7fd88b7fe640 1 -- 172.21.15.53:0/1661742604 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965011020 0 0) 0x7fd88c06cc50 con 0x7fd88c0702b0 2023-10-21T17:18:35.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.004+0000 7fd88bfff640 1 --2- 172.21.15.53:0/1661742604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c0cb150 0x7fd88c0cb600 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0b0d50 tx=0x7fd87802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39809b4b6ea5449b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.005+0000 7fd88b7fe640 1 -- 172.21.15.53:0/1661742604 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878030880 con 0x7fd88c0cb150 2023-10-21T17:18:35.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.005+0000 7fd88b7fe640 1 -- 172.21.15.53:0/1661742604 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780309e0 con 0x7fd88c0cb150 2023-10-21T17:18:35.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.005+0000 7fd88b7fe640 1 -- 172.21.15.53:0/1661742604 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878030d70 con 0x7fd88c0cb150 2023-10-21T17:18:35.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.005+0000 7fd892ee0640 1 -- 172.21.15.53:0/1661742604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c0cb150 msgr2=0x7fd88c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.005+0000 7fd892ee0640 1 --2- 172.21.15.53:0/1661742604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c0cb150 0x7fd88c0cb600 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0b0d50 tx=0x7fd87802b9c0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 -- 172.21.15.53:0/1661742604 shutdown_connections 2023-10-21T17:18:35.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 --2- 172.21.15.53:0/1661742604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c0cb150 0x7fd88c0cb600 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 --2- 172.21.15.53:0/1661742604 >> v2:172.21.15.124:3300/0 conn(0x7fd88c161690 0x7fd88c06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 -- 172.21.15.53:0/1661742604 >> 172.21.15.53:0/1661742604 conn(0x7fd88c06a5d0 msgr2=0x7fd88c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:35.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 -- 172.21.15.53:0/1661742604 shutdown_connections 2023-10-21T17:18:35.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.006+0000 7fd892ee0640 1 -- 172.21.15.53:0/1661742604 wait complete. 2023-10-21T17:18:35.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.007+0000 7fd892ee0640 1 Processor -- start 2023-10-21T17:18:35.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.008+0000 7fd892ee0640 1 -- start start 2023-10-21T17:18:35.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.008+0000 7fd892ee0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88c0702b0 0x7fd88c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd892ee0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88c0cb150 0x7fd88c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd892ee0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd892ee0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1554f0 con 0x7fd88c161690 2023-10-21T17:18:35.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd892ee0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c161a70 con 0x7fd88c0cb150 2023-10-21T17:18:35.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd892ee0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0c77b0 con 0x7fd88c0702b0 2023-10-21T17:18:35.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47588/0 (socket says 172.21.15.53:47588) 2023-10-21T17:18:35.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 -- 172.21.15.53:0/262334457 learned_addr learned my addr 172.21.15.53:0/262334457 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:35.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 -- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88c0702b0 msgr2=0x7fd88c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd890c7c640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88c0702b0 0x7fd88c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd88bfff640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88c0cb150 0x7fd88c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88c0702b0 0x7fd88c0c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 -- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88c0cb150 msgr2=0x7fd88c0c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88c0cb150 0x7fd88c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.009+0000 7fd89147d640 1 -- 172.21.15.53:0/262334457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780094d0 con 0x7fd88c161690 2023-10-21T17:18:35.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd89147d640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c7e0 tx=0x7fd88400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400a5e0 con 0x7fd88c161690 2023-10-21T17:18:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88400a760 con 0x7fd88c161690 2023-10-21T17:18:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400ab10 con 0x7fd88c161690 2023-10-21T17:18:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c15a690 con 0x7fd88c161690 2023-10-21T17:18:35.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.010+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c163f60 con 0x7fd88c161690 2023-10-21T17:18:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.011+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c161c20 con 0x7fd88c161690 2023-10-21T17:18:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.012+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd88400ac70 con 0x7fd88c161690 2023-10-21T17:18:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.013+0000 7fd8897fa640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 0x7fd87006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.013+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fd884088de0 con 0x7fd88c161690 2023-10-21T17:18:35.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.014+0000 7fd890c7c640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 0x7fd87006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.015+0000 7fd890c7c640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 0x7fd87006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.018+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd88c161c20 con 0x7fd88c161690 2023-10-21T17:18:35.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.212+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd88c161c20 con 0x7fd88c161690 2023-10-21T17:18:35.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.220+0000 7fd8897fa640 1 -- 172.21.15.53:0/262334457 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd88c161c20 con 0x7fd88c161690 2023-10-21T17:18:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 msgr2=0x7fd87006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 0x7fd87006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).stop 2023-10-21T17:18:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 msgr2=0x7fd88c0c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c7e0 tx=0x7fd88400cca0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 shutdown_connections 2023-10-21T17:18:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fd87006afc0 0x7fd87006d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd88c161690 0x7fd88c0c9a00 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88c0cb150 0x7fd88c0c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 --2- 172.21.15.53:0/262334457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88c0702b0 0x7fd88c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 >> 172.21.15.53:0/262334457 conn(0x7fd88c06a5d0 msgr2=0x7fd88c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 shutdown_connections 2023-10-21T17:18:35.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.223+0000 7fd892ee0640 1 -- 172.21.15.53:0/262334457 wait complete. 2023-10-21T17:18:35.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-21T17:18:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.465+0000 7fb88992e640 1 Processor -- start 2023-10-21T17:18:35.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.466+0000 7fb88992e640 1 -- start start 2023-10-21T17:18:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.467+0000 7fb88992e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb88415a970 0x7fb8840cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.467+0000 7fb88992e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88406cfd0 con 0x7fb8840cb990 2023-10-21T17:18:35.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.467+0000 7fb88992e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88406d870 con 0x7fb88406dd00 2023-10-21T17:18:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.467+0000 7fb88992e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8840cbe00 con 0x7fb88415a970 2023-10-21T17:18:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.467+0000 7fb88912d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb8840cb990 0x7fb8840b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54234/0 (socket says 172.21.15.53:54234) 2023-10-21T17:18:35.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.468+0000 7fb88912d640 1 -- 172.21.15.53:0/284106532 learned_addr learned my addr 172.21.15.53:0/284106532 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.468+0000 7fb88892c640 1 --2- 172.21.15.53:0/284106532 >> v2:172.21.15.124:3300/0 conn(0x7fb88415a970 0x7fb8840cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.469+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759097600 0 0) 0x7fb88406d870 con 0x7fb88406dd00 2023-10-21T17:18:35.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.469+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86c003520 con 0x7fb88406dd00 2023-10-21T17:18:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.469+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435797213 0 0) 0x7fb88406cfd0 con 0x7fb8840cb990 2023-10-21T17:18:35.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.469+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb88406d870 con 0x7fb8840cb990 2023-10-21T17:18:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434391225 0 0) 0x7fb88406d870 con 0x7fb8840cb990 2023-10-21T17:18:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 >> v2:172.21.15.124:3300/0 conn(0x7fb88415a970 msgr2=0x7fb8840cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 --2- 172.21.15.53:0/284106532 >> v2:172.21.15.124:3300/0 conn(0x7fb88415a970 0x7fb8840cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 >> v1:172.21.15.124:6789/0 conn(0x7fb88406dd00 legacy=0x7fb8840c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88415ad50 con 0x7fb8840cb990 2023-10-21T17:18:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb87c001670 con 0x7fb8840cb990 2023-10-21T17:18:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 >> v1:172.21.15.53:6789/0 conn(0x7fb8840cb990 legacy=0x7fb8840b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c003b50 0x7fb86c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86c005240 0x7fb86c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840cbe00 con 0x7fb86c0047c0 2023-10-21T17:18:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c00c180 con 0x7fb86c003b50 2023-10-21T17:18:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.470+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c00c340 con 0x7fb86c005240 2023-10-21T17:18:35.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.471+0000 7fb87bfff640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.471+0000 7fb87bfff640 1 -- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86c005240 msgr2=0x7fb86c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:35.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.471+0000 7fb87bfff640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb86c005240 0x7fb86c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.471+0000 7fb87bfff640 1 -- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c003b50 msgr2=0x7fb86c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:35.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.471+0000 7fb87bfff640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c003b50 0x7fb86c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.472+0000 7fb87bfff640 1 -- 172.21.15.53:0/284106532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88406d870 con 0x7fb86c0047c0 2023-10-21T17:18:35.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.472+0000 7fb88912d640 1 -- 172.21.15.53:0/284106532 reap_dead start 2023-10-21T17:18:35.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.473+0000 7fb87bfff640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb8740014e0 tx=0x7fb86c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc484ad5eb33f22e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.474+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740052d0 con 0x7fb86c0047c0 2023-10-21T17:18:35.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.474+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874005450 con 0x7fb86c0047c0 2023-10-21T17:18:35.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.474+0000 7fb87b7fe640 1 -- 172.21.15.53:0/284106532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740308c0 con 0x7fb86c0047c0 2023-10-21T17:18:35.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 -- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 msgr2=0x7fb86c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb8740014e0 tx=0x7fb86c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 -- 172.21.15.53:0/284106532 shutdown_connections 2023-10-21T17:18:35.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 --2- 172.21.15.53:0/284106532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb86c0047c0 0x7fb86c004c40 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 -- 172.21.15.53:0/284106532 >> 172.21.15.53:0/284106532 conn(0x7fb8840ce9d0 msgr2=0x7fb8840cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:35.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 -- 172.21.15.53:0/284106532 shutdown_connections 2023-10-21T17:18:35.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.475+0000 7fb88992e640 1 -- 172.21.15.53:0/284106532 wait complete. 2023-10-21T17:18:35.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.477+0000 7fb88992e640 1 Processor -- start 2023-10-21T17:18:35.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.477+0000 7fb88992e640 1 -- start start 2023-10-21T17:18:35.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c0047c0 0x7fb8841535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8840af760 0x7fb8840afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840b0120 con 0x7fb88415cd80 2023-10-21T17:18:35.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c34f0 con 0x7fb86c0047c0 2023-10-21T17:18:35.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb88992e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c37d0 con 0x7fb8840af760 2023-10-21T17:18:35.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47596/0 (socket says 172.21.15.53:47596) 2023-10-21T17:18:35.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb87bfff640 1 -- 172.21.15.53:0/248982667 learned_addr learned my addr 172.21.15.53:0/248982667 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:35.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb87bfff640 1 -- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8840af760 msgr2=0x7fb8840afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.478+0000 7fb87bfff640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8840af760 0x7fb8840afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb87bfff640 1 -- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c0047c0 msgr2=0x7fb8841535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb88892c640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c0047c0 0x7fb8841535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb87bfff640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c0047c0 0x7fb8841535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb87bfff640 1 -- 172.21.15.53:0/248982667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840c3ab0 con 0x7fb88415cd80 2023-10-21T17:18:35.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb87bfff640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb8740019c0 tx=0x7fb8740012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740052e0 con 0x7fb88415cd80 2023-10-21T17:18:35.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874007a30 con 0x7fb88415cd80 2023-10-21T17:18:35.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874007230 con 0x7fb88415cd80 2023-10-21T17:18:35.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8840c3d90 con 0x7fb88415cd80 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.479+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb884153af0 con 0x7fb88415cd80 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.481+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88404e160 con 0x7fb88415cd80 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.482+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb874007390 con 0x7fb88415cd80 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.483+0000 7fb8797fa640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 0x7fb87006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.483+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fb8740af2f0 con 0x7fb88415cd80 2023-10-21T17:18:35.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.484+0000 7fb88892c640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 0x7fb87006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:35.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.486+0000 7fb88892c640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 0x7fb87006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb88404fa60 tx=0x7fb88000b040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:35.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.487+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb87407a1b0 con 0x7fb88415cd80 2023-10-21T17:18:35.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.751+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fb88404e160 con 0x7fb88415cd80 2023-10-21T17:18:35.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.815+0000 7fb8797fa640 1 -- 172.21.15.53:0/248982667 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2510) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb87407d7d0 con 0x7fb88415cd80 2023-10-21T17:18:35.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 msgr2=0x7fb87006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 0x7fb87006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb88404fa60 tx=0x7fb88000b040 comp rx=0 tx=0).stop 2023-10-21T17:18:35.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 msgr2=0x7fb88415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:35.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb8740019c0 tx=0x7fb8740012d0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 shutdown_connections 2023-10-21T17:18:35.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb87006af90 0x7fb87006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8840af760 0x7fb8840afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb88415cd80 0x7fb88415d200 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 --2- 172.21.15.53:0/248982667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb86c0047c0 0x7fb8841535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:35.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 >> 172.21.15.53:0/248982667 conn(0x7fb8840ce9d0 msgr2=0x7fb8840b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:35.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 shutdown_connections 2023-10-21T17:18:35.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:35.818+0000 7fb88992e640 1 -- 172.21.15.53:0/248982667 wait complete. 2023-10-21T17:18:35.840 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-21T17:18:35.841 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-21T17:18:35.841 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T17:18:35.842 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:36.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.069+0000 7fad4efdc640 1 Processor -- start 2023-10-21T17:18:36.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.069+0000 7fad4efdc640 1 -- start start 2023-10-21T17:18:36.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4efdc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad48159ac0 0x7fad4806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4efdc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 0x7fad480c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4efdc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4806dee0 con 0x7fad480ad790 2023-10-21T17:18:36.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4efdc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4815d850 con 0x7fad48159ac0 2023-10-21T17:18:36.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4efdc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4815db30 con 0x7fad480709e0 2023-10-21T17:18:36.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4cd78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad48159ac0 0x7fad4806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4cd78640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad48159ac0 0x7fad4806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47604/0 (socket says 172.21.15.53:47604) 2023-10-21T17:18:36.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.071+0000 7fad4cd78640 1 -- 172.21.15.53:0/3456152316 learned_addr learned my addr 172.21.15.53:0/3456152316 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:36.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4d579640 1 --2- 172.21.15.53:0/3456152316 >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 0x7fad480c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4cd78640 1 -- 172.21.15.53:0/3456152316 >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 msgr2=0x7fad480c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:36.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4cd78640 1 --2- 172.21.15.53:0/3456152316 >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 0x7fad480c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4cd78640 1 -- 172.21.15.53:0/3456152316 >> v1:172.21.15.124:6789/0 conn(0x7fad480ad790 legacy=0x7fad480702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:36.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4cd78640 1 -- 172.21.15.53:0/3456152316 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4815de10 con 0x7fad48159ac0 2023-10-21T17:18:36.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.072+0000 7fad4d579640 1 --2- 172.21.15.53:0/3456152316 >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 0x7fad480c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:36.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.073+0000 7fad4cd78640 1 --2- 172.21.15.53:0/3456152316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4806d5e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fad480caab0 tx=0x7fad3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8487c11a47107300 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.073+0000 7fad477fe640 1 -- 172.21.15.53:0/3456152316 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c031870 con 0x7fad48159ac0 2023-10-21T17:18:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.073+0000 7fad477fe640 1 -- 172.21.15.53:0/3456152316 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c0319d0 con 0x7fad48159ac0 2023-10-21T17:18:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.073+0000 7fad477fe640 1 -- 172.21.15.53:0/3456152316 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c031d60 con 0x7fad48159ac0 2023-10-21T17:18:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 -- 172.21.15.53:0/3456152316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 msgr2=0x7fad4806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 --2- 172.21.15.53:0/3456152316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4806d5e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fad480caab0 tx=0x7fad3c02b910 comp rx=0 tx=0).stop 2023-10-21T17:18:36.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 -- 172.21.15.53:0/3456152316 shutdown_connections 2023-10-21T17:18:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 --2- 172.21.15.53:0/3456152316 >> v2:172.21.15.124:3300/0 conn(0x7fad480709e0 0x7fad480c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 --2- 172.21.15.53:0/3456152316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4806d5e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 -- 172.21.15.53:0/3456152316 >> 172.21.15.53:0/3456152316 conn(0x7fad4806a380 msgr2=0x7fad480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 -- 172.21.15.53:0/3456152316 shutdown_connections 2023-10-21T17:18:36.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.074+0000 7fad4efdc640 1 -- 172.21.15.53:0/3456152316 wait complete. 2023-10-21T17:18:36.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.077+0000 7fad4efdc640 1 Processor -- start 2023-10-21T17:18:36.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.077+0000 7fad4efdc640 1 -- start start 2023-10-21T17:18:36.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad480709e0 0x7fad4815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad480ad790 0x7fad4815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815a850 con 0x7fad48159ac0 2023-10-21T17:18:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815ab30 con 0x7fad480709e0 2023-10-21T17:18:36.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4efdc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815ae10 con 0x7fad480ad790 2023-10-21T17:18:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.078+0000 7fad4d579640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47614/0 (socket says 172.21.15.53:47614) 2023-10-21T17:18:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 -- 172.21.15.53:0/2617839317 learned_addr learned my addr 172.21.15.53:0/2617839317 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad47fff640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad480ad790 0x7fad4815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4cd78640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad480709e0 0x7fad4815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 -- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad480ad790 msgr2=0x7fad4815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad480ad790 0x7fad4815a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 -- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad480709e0 msgr2=0x7fad4815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad480709e0 0x7fad4815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 -- 172.21.15.53:0/2617839317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0094d0 con 0x7fad48159ac0 2023-10-21T17:18:36.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.079+0000 7fad4d579640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fad4000ce80 tx=0x7fad4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.080+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000d840 con 0x7fad48159ac0 2023-10-21T17:18:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.080+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4000d9e0 con 0x7fad48159ac0 2023-10-21T17:18:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.080+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000dd90 con 0x7fad48159ac0 2023-10-21T17:18:36.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.080+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4815b0f0 con 0x7fad48159ac0 2023-10-21T17:18:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.080+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4815a560 con 0x7fad48159ac0 2023-10-21T17:18:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.081+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4815be30 con 0x7fad48159ac0 2023-10-21T17:18:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.082+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad40015440 con 0x7fad48159ac0 2023-10-21T17:18:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.083+0000 7fad457fa640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 0x7fad3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.083+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fad4008a1c0 con 0x7fad48159ac0 2023-10-21T17:18:36.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.084+0000 7fad4cd78640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 0x7fad3406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.085+0000 7fad4cd78640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 0x7fad3406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036000 tx=0x7fad3c03c040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.087+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad40055080 con 0x7fad48159ac0 2023-10-21T17:18:36.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.283+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad4815b610 con 0x7fad48159ac0 2023-10-21T17:18:36.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.284+0000 7fad457fa640 1 -- 172.21.15.53:0/2617839317 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+14393 (secure 0 0 0) 0x7fad4815b610 con 0x7fad48159ac0 2023-10-21T17:18:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:36.286 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":175,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:33.806391+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/3880141158":"2023-10-22T17:18:30.312328+0000","172.21.15.124:0/1493657297":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6833/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1967999416":"2023-10-22T17:18:26.427157+0000","172.21.15.124:6832/2988307648":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2914043524":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/1768082194":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3657298397":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3304201904":"2023-10-22T16:22:28.659493+0000","172.21.15.53:6833/3880141158":"2023-10-22T17:18:30.312328+0000","172.21.15.124:0/2610966345":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/3058690420":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3358065251":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/4258504518":"2023-10-22T16:22:28.659493+0000","172.21.15.124:0/3974743798":"2023-10-22T17:18:26.427157+0000","172.21.15.124:0/2638777920":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6833/870299393":"2023-10-22T16:22:28.659493+0000","172.21.15.124:6832/870299393":"2023-10-22T16:22:28.659493+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:36.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 msgr2=0x7fad3406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 0x7fad3406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad3c036000 tx=0x7fad3c03c040 comp rx=0 tx=0).stop 2023-10-21T17:18:36.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 msgr2=0x7fad4815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fad4000ce80 tx=0x7fad4000a710 comp rx=0 tx=0).stop 2023-10-21T17:18:36.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 shutdown_connections 2023-10-21T17:18:36.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fad3406af90 0x7fad3406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad48159ac0 0x7fad4815c460 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad480ad790 0x7fad4815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 --2- 172.21.15.53:0/2617839317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad480709e0 0x7fad4815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.287+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 >> 172.21.15.53:0/2617839317 conn(0x7fad4806a380 msgr2=0x7fad480c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:36.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.288+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 shutdown_connections 2023-10-21T17:18:36.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.288+0000 7fad4efdc640 1 -- 172.21.15.53:0/2617839317 wait complete. 2023-10-21T17:18:36.308 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:18:36.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:18:36.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.525+0000 7f94e93b0640 1 Processor -- start 2023-10-21T17:18:36.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.526+0000 7f94e93b0640 1 -- start start 2023-10-21T17:18:36.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.527+0000 7f94e93b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94e415aad0 0x7f94e40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.527+0000 7f94e93b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f94e40c82b0 0x7f94e40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.527+0000 7f94e93b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e415bf30 con 0x7f94e406cd00 2023-10-21T17:18:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e93b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e415b320 con 0x7f94e415aad0 2023-10-21T17:18:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e93b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e415b600 con 0x7f94e40c82b0 2023-10-21T17:18:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e2ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94e415aad0 0x7f94e40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f94e406cd00 0x7f94e4070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54242/0 (socket says 172.21.15.53:54242) 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e37fe640 1 -- 172.21.15.53:0/2144901473 learned_addr learned my addr 172.21.15.53:0/2144901473 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e27fc640 1 --2- 172.21.15.53:0/2144901473 >> v2:172.21.15.124:3300/0 conn(0x7f94e40c82b0 0x7f94e40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e1ffb640 1 -- 172.21.15.53:0/2144901473 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807151458 0 0) 0x7f94e415bf30 con 0x7f94e406cd00 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2144901473 >> v2:172.21.15.124:3300/0 conn(0x7f94e40c82b0 msgr2=0x7f94e40c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e2ffd640 1 --2- 172.21.15.53:0/2144901473 >> v2:172.21.15.124:3300/0 conn(0x7f94e40c82b0 0x7f94e40c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2144901473 >> v1:172.21.15.53:6789/0 conn(0x7f94e406cd00 legacy=0x7f94e4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.528+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2144901473 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e415b8e0 con 0x7f94e415aad0 2023-10-21T17:18:36.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e2ffd640 1 --2- 172.21.15.53:0/2144901473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e415aad0 0x7f94e40b1750 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f94e406e330 tx=0x7f94d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960be0e6f3879d3a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e1ffb640 1 -- 172.21.15.53:0/2144901473 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4031830 con 0x7f94e415aad0 2023-10-21T17:18:36.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e1ffb640 1 -- 172.21.15.53:0/2144901473 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4031990 con 0x7f94e415aad0 2023-10-21T17:18:36.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e1ffb640 1 -- 172.21.15.53:0/2144901473 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4031d20 con 0x7f94e415aad0 2023-10-21T17:18:36.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e93b0640 1 -- 172.21.15.53:0/2144901473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e415aad0 msgr2=0x7f94e40b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.530+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2144901473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e415aad0 0x7f94e40b1750 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f94e406e330 tx=0x7f94d402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 -- 172.21.15.53:0/2144901473 shutdown_connections 2023-10-21T17:18:36.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2144901473 >> v2:172.21.15.124:3300/0 conn(0x7f94e40c82b0 0x7f94e40c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2144901473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e415aad0 0x7f94e40b1750 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 -- 172.21.15.53:0/2144901473 >> 172.21.15.53:0/2144901473 conn(0x7f94e4069d50 msgr2=0x7f94e406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:36.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 -- 172.21.15.53:0/2144901473 shutdown_connections 2023-10-21T17:18:36.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.531+0000 7f94e93b0640 1 -- 172.21.15.53:0/2144901473 wait complete. 2023-10-21T17:18:36.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.532+0000 7f94e93b0640 1 Processor -- start 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.532+0000 7f94e93b0640 1 -- start start 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e40c82b0 0x7f94e41f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e415aad0 0x7f94e40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4145da0 con 0x7f94e406cd00 2023-10-21T17:18:36.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4146080 con 0x7f94e415aad0 2023-10-21T17:18:36.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.533+0000 7f94e93b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e4146360 con 0x7f94e40c82b0 2023-10-21T17:18:36.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47636/0 (socket says 172.21.15.53:47636) 2023-10-21T17:18:36.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2301148396 learned_addr learned my addr 172.21.15.53:0/2301148396 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:36.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e40c82b0 msgr2=0x7f94e41f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e40c82b0 0x7f94e41f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e415aad0 msgr2=0x7f94e40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e415aad0 0x7f94e40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 -- 172.21.15.53:0/2301148396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4009400 con 0x7f94e406cd00 2023-10-21T17:18:36.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.534+0000 7f94e2ffd640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f94d4000c00 tx=0x7f94d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.535+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d403aa60 con 0x7f94e406cd00 2023-10-21T17:18:36.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.535+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d403abe0 con 0x7f94e406cd00 2023-10-21T17:18:36.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.535+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d40443f0 con 0x7f94e406cd00 2023-10-21T17:18:36.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.535+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e4146640 con 0x7f94e406cd00 2023-10-21T17:18:36.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.535+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e41459c0 con 0x7f94e406cd00 2023-10-21T17:18:36.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.536+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e4146b60 con 0x7f94e406cd00 2023-10-21T17:18:36.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.537+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94d404d450 con 0x7f94e406cd00 2023-10-21T17:18:36.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.538+0000 7f94db7fe640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.538+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f94d40abb20 con 0x7f94e406cd00 2023-10-21T17:18:36.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.539+0000 7f94e27fc640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.540+0000 7f94e27fc640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 0x7f94c406d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94e40cd590 tx=0x7f94cc008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.542+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94e4146b60 con 0x7f94e406cd00 2023-10-21T17:18:36.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.733+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94e4146b60 con 0x7f94e406cd00 2023-10-21T17:18:36.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.734+0000 7f94db7fe640 1 -- 172.21.15.53:0/2301148396 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94e4146b60 con 0x7f94e406cd00 2023-10-21T17:18:36.735 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:18:36.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 msgr2=0x7f94c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 0x7f94c406d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f94e40cd590 tx=0x7f94cc008040 comp rx=0 tx=0).stop 2023-10-21T17:18:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 msgr2=0x7f94e41f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f94d4000c00 tx=0x7f94d40046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 shutdown_connections 2023-10-21T17:18:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e415aad0 0x7f94e40cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e40c82b0 0x7f94e41f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 --2- 172.21.15.53:0/2301148396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e406cd00 0x7f94e41f3530 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 >> 172.21.15.53:0/2301148396 conn(0x7f94e4069d50 msgr2=0x7f94e40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 shutdown_connections 2023-10-21T17:18:36.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.737+0000 7f94e93b0640 1 -- 172.21.15.53:0/2301148396 wait complete. 2023-10-21T17:18:36.757 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-21T17:18:36.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.986+0000 7f125544d640 1 Processor -- start 2023-10-21T17:18:36.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.986+0000 7f125544d640 1 -- start start 2023-10-21T17:18:36.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.987+0000 7f125544d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f125015ab30 0x7f125015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.987+0000 7f125544d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125015f7d0 con 0x7f12500b07d0 2023-10-21T17:18:36.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.987+0000 7f125544d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125015f210 con 0x7f12500adc70 2023-10-21T17:18:36.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.987+0000 7f125544d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125015f4f0 con 0x7f125015ab30 2023-10-21T17:18:36.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f124ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f125015ab30 0x7f125015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f1254c4c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f12500b07d0 0x7f125006cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54256/0 (socket says 172.21.15.53:54256) 2023-10-21T17:18:36.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f1254c4c640 1 -- 172.21.15.53:0/1779627320 learned_addr learned my addr 172.21.15.53:0/1779627320 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:36.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f124ffff640 1 -- 172.21.15.53:0/1779627320 >> v1:172.21.15.124:6789/0 conn(0x7f12500adc70 legacy=0x7f12500b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:36.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f124ffff640 1 -- 172.21.15.53:0/1779627320 >> v1:172.21.15.53:6789/0 conn(0x7f12500b07d0 legacy=0x7f125006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f124ffff640 1 -- 172.21.15.53:0/1779627320 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125006c550 con 0x7f125015ab30 2023-10-21T17:18:36.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.988+0000 7f124effd640 1 -- 172.21.15.53:0/1779627320 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214906639 0 0) 0x7f125015f7d0 con 0x7f12500b07d0 2023-10-21T17:18:36.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f124ffff640 1 --2- 172.21.15.53:0/1779627320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015e8a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1240009d90 tx=0x7f124002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111e361f686e16d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:36.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f124effd640 1 -- 172.21.15.53:0/1779627320 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124002cc60 con 0x7f125015ab30 2023-10-21T17:18:36.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f124effd640 1 -- 172.21.15.53:0/1779627320 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124002cdc0 con 0x7f125015ab30 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f124effd640 1 -- 172.21.15.53:0/1779627320 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240034660 con 0x7f125015ab30 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f125544d640 1 -- 172.21.15.53:0/1779627320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 msgr2=0x7f125015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.990+0000 7f125544d640 1 --2- 172.21.15.53:0/1779627320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015e8a0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1240009d90 tx=0x7f124002beb0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.991+0000 7f125544d640 1 -- 172.21.15.53:0/1779627320 shutdown_connections 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.991+0000 7f125544d640 1 --2- 172.21.15.53:0/1779627320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015e8a0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.991+0000 7f125544d640 1 -- 172.21.15.53:0/1779627320 >> 172.21.15.53:0/1779627320 conn(0x7f12500cb0c0 msgr2=0x7f12500cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.991+0000 7f125544d640 1 -- 172.21.15.53:0/1779627320 shutdown_connections 2023-10-21T17:18:36.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.991+0000 7f125544d640 1 -- 172.21.15.53:0/1779627320 wait complete. 2023-10-21T17:18:36.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.992+0000 7f125544d640 1 Processor -- start 2023-10-21T17:18:36.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.992+0000 7f125544d640 1 -- start start 2023-10-21T17:18:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12500adc70 0x7f1250157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12500b07d0 0x7f125015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500c5e80 con 0x7f125015ab30 2023-10-21T17:18:36.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12500c6160 con 0x7f12500b07d0 2023-10-21T17:18:36.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.993+0000 7f125544d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12500c6440 con 0x7f12500adc70 2023-10-21T17:18:36.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47658/0 (socket says 172.21.15.53:47658) 2023-10-21T17:18:36.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 -- 172.21.15.53:0/2585623909 learned_addr learned my addr 172.21.15.53:0/2585623909 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:36.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f124ffff640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12500adc70 0x7f1250157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 -- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12500adc70 msgr2=0x7f1250157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12500adc70 0x7f1250157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:36.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 -- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12500b07d0 msgr2=0x7f125015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:36.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f124f7fe640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12500b07d0 0x7f125015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:36.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12500b07d0 0x7f125015af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.994+0000 7f1254c4c640 1 -- 172.21.15.53:0/2585623909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1240009a70 con 0x7f125015ab30 2023-10-21T17:18:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f1254c4c640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f124800c450 tx=0x7f124800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12480074e0 con 0x7f125015ab30 2023-10-21T17:18:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1248007660 con 0x7f125015ab30 2023-10-21T17:18:37.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1248004a20 con 0x7f125015ab30 2023-10-21T17:18:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12500c6720 con 0x7f125015ab30 2023-10-21T17:18:37.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.995+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125015d1b0 con 0x7f125015ab30 2023-10-21T17:18:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.997+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12500c6c40 con 0x7f125015ab30 2023-10-21T17:18:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.997+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1248004b80 con 0x7f125015ab30 2023-10-21T17:18:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.998+0000 7f124cff9640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 0x7f123c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:37.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.998+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1248088980 con 0x7f125015ab30 2023-10-21T17:18:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:36.999+0000 7f124ffff640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 0x7f123c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.000+0000 7f124ffff640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 0x7f123c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1240036000 tx=0x7f1240030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:37.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.003+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12500c6c40 con 0x7f125015ab30 2023-10-21T17:18:37.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.273+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f12500c6c40 con 0x7f125015ab30 2023-10-21T17:18:37.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.827+0000 7f124cff9640 1 -- 172.21.15.53:0/2585623909 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2512) v1 ==== 177+0+0 (secure 0 0 0) 0x7f12500c6c40 con 0x7f125015ab30 2023-10-21T17:18:37.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 msgr2=0x7f123c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:37.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 0x7f123c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1240036000 tx=0x7f1240030000 comp rx=0 tx=0).stop 2023-10-21T17:18:37.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 msgr2=0x7f125015b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:37.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f124800c450 tx=0x7f124800c910 comp rx=0 tx=0).stop 2023-10-21T17:18:37.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 shutdown_connections 2023-10-21T17:18:37.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f123c06af90 0x7f123c06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:37.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125015ab30 0x7f125015b450 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:37.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12500b07d0 0x7f125015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:37.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 --2- 172.21.15.53:0/2585623909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12500adc70 0x7f1250157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:37.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 >> 172.21.15.53:0/2585623909 conn(0x7f12500cb0c0 msgr2=0x7f12500cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:37.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 shutdown_connections 2023-10-21T17:18:37.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:37.832+0000 7f125544d640 1 -- 172.21.15.53:0/2585623909 wait complete. 2023-10-21T17:18:37.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:18:38.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.089+0000 7fdd2bfff640 1 Processor -- start 2023-10-21T17:18:38.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.089+0000 7fdd2bfff640 1 -- start start 2023-10-21T17:18:38.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd2c0b3c50 0x7fdd2c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c159db0 con 0x7fdd2c1599d0 2023-10-21T17:18:38.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c071fe0 con 0x7fdd2c0cb5e0 2023-10-21T17:18:38.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b44c0 con 0x7fdd2c0b3c50 2023-10-21T17:18:38.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdd2c1599d0 0x7fdd2c0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54264/0 (socket says 172.21.15.53:54264) 2023-10-21T17:18:38.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.091+0000 7fdd2affd640 1 -- 172.21.15.53:0/3229568528 learned_addr learned my addr 172.21.15.53:0/3229568528 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:38.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2b7fe640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd2c0b3c50 0x7fdd2c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097564368 0 0) 0x7fdd1c001110 con 0x7fdd2c1599d0 2023-10-21T17:18:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd18003520 con 0x7fdd2c1599d0 2023-10-21T17:18:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145417173 0 0) 0x7fdd18003520 con 0x7fdd2c1599d0 2023-10-21T17:18:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd2c0b3c50 msgr2=0x7fdd2c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:38.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd2c0b3c50 0x7fdd2c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 >> v1:172.21.15.124:6789/0 conn(0x7fdd2c0cb5e0 legacy=0x7fdd2c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c0713d0 con 0x7fdd2c1599d0 2023-10-21T17:18:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fdd1c001800 con 0x7fdd2c1599d0 2023-10-21T17:18:38.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 >> v1:172.21.15.53:6789/0 conn(0x7fdd2c1599d0 legacy=0x7fdd2c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180039a0 0x7fdd18003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd180047c0 0x7fdd18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd18005180 0x7fdd1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b44c0 con 0x7fdd180047c0 2023-10-21T17:18:38.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1800c050 con 0x7fdd18005180 2023-10-21T17:18:38.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1800c210 con 0x7fdd180039a0 2023-10-21T17:18:38.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.092+0000 7fdd2b7fe640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd2c0b3c50 0x7fdd2c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:38.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd180047c0 0x7fdd18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd2affd640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180039a0 0x7fdd18003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 -- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180039a0 msgr2=0x7fdd18003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180039a0 0x7fdd18003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 -- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd18005180 msgr2=0x7fdd1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd18005180 0x7fdd1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd23fff640 1 -- 172.21.15.53:0/3229568528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18003520 con 0x7fdd180047c0 2023-10-21T17:18:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.093+0000 7fdd2affd640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180039a0 0x7fdd18003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.094+0000 7fdd23fff640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd180047c0 0x7fdd18004c40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fdd14001750 tx=0x7fdd180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbfe8d02d9e1911 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.094+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140052d0 con 0x7fdd180047c0 2023-10-21T17:18:38.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.094+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1402e050 con 0x7fdd180047c0 2023-10-21T17:18:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.094+0000 7fdd2a7fc640 1 -- 172.21.15.53:0/3229568528 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140035f0 con 0x7fdd180047c0 2023-10-21T17:18:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 -- 172.21.15.53:0/3229568528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd180047c0 msgr2=0x7fdd18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/3229568528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd180047c0 0x7fdd18004c40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fdd14001750 tx=0x7fdd180057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2b7fe640 1 -- 172.21.15.53:0/3229568528 reap_dead start 2023-10-21T17:18:38.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 -- 172.21.15.53:0/3229568528 shutdown_connections 2023-10-21T17:18:38.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 -- 172.21.15.53:0/3229568528 >> 172.21.15.53:0/3229568528 conn(0x7fdd2c06c050 msgr2=0x7fdd2c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:38.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 -- 172.21.15.53:0/3229568528 shutdown_connections 2023-10-21T17:18:38.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.095+0000 7fdd2bfff640 1 -- 172.21.15.53:0/3229568528 wait complete. 2023-10-21T17:18:38.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.097+0000 7fdd2bfff640 1 Processor -- start 2023-10-21T17:18:38.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.097+0000 7fdd2bfff640 1 -- start start 2023-10-21T17:18:38.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180047c0 0x7fdd2c1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd2c1f30d0 0x7fdd2c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0a9780 con 0x7fdd2c1f3550 2023-10-21T17:18:38.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1e62b0 con 0x7fdd2c1f30d0 2023-10-21T17:18:38.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c1e6590 con 0x7fdd180047c0 2023-10-21T17:18:38.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.098+0000 7fdd2b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47680/0 (socket says 172.21.15.53:47680) 2023-10-21T17:18:38.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 -- 172.21.15.53:0/1801138883 learned_addr learned my addr 172.21.15.53:0/1801138883 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2affd640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180047c0 0x7fdd2c1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 -- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180047c0 msgr2=0x7fdd2c1f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180047c0 0x7fdd2c1f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 -- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd2c1f30d0 msgr2=0x7fdd2c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd2c1f30d0 0x7fdd2c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 -- 172.21.15.53:0/1801138883 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c1e6870 con 0x7fdd2c1f3550 2023-10-21T17:18:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.099+0000 7fdd2b7fe640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c06fff0 tx=0x7fdd2400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.100+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400dc90 con 0x7fdd2c1f3550 2023-10-21T17:18:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.100+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2400f040 con 0x7fdd2c1f3550 2023-10-21T17:18:38.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.100+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd24003990 con 0x7fdd2c1f3550 2023-10-21T17:18:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.100+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c1e6b50 con 0x7fdd2c1f3550 2023-10-21T17:18:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.100+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c1f5f40 con 0x7fdd2c1f3550 2023-10-21T17:18:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.101+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c1e6d40 con 0x7fdd2c1f3550 2023-10-21T17:18:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.102+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd24003af0 con 0x7fdd2c1f3550 2023-10-21T17:18:38.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.103+0000 7fdd237fe640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 0x7fdd0c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.103+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fdd24089770 con 0x7fdd2c1f3550 2023-10-21T17:18:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.104+0000 7fdd2affd640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 0x7fdd0c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:38.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.105+0000 7fdd2affd640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 0x7fdd0c06d3b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:38.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.108+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd24057f10 con 0x7fdd2c1f3550 2023-10-21T17:18:38.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.305+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd2c1f5f40 con 0x7fdd2c1f3550 2023-10-21T17:18:38.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.312+0000 7fdd237fe640 1 -- 172.21.15.53:0/1801138883 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd2c1f5f40 con 0x7fdd2c1f3550 2023-10-21T17:18:38.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.315+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 msgr2=0x7fdd0c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.315+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 0x7fdd0c06d3b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c000c00 tx=0x7fdd1c009040 comp rx=0 tx=0).stop 2023-10-21T17:18:38.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.315+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 msgr2=0x7fdd2c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:38.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.315+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c06fff0 tx=0x7fdd2400d4a0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 shutdown_connections 2023-10-21T17:18:38.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdd0c06aef0 0x7fdd0c06d3b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd2c1f3550 0x7fdd2c157cf0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd2c1f30d0 0x7fdd2c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 --2- 172.21.15.53:0/1801138883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180047c0 0x7fdd2c1f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 >> 172.21.15.53:0/1801138883 conn(0x7fdd2c06c050 msgr2=0x7fdd2c15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 shutdown_connections 2023-10-21T17:18:38.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:38.316+0000 7fdd2bfff640 1 -- 172.21.15.53:0/1801138883 wait complete. 2023-10-21T17:18:38.335 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:18:38.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:18:38.449 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:18:38.570 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:18:38.571 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:38.571 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:38.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:18:38.634 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:18:38.635 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:38.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:18:38.697 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:38.697 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:38.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:18:38.759 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:18:38.759 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:38.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:18:38.819 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:18:38.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:39.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.057+0000 7f187596c640 1 Processor -- start 2023-10-21T17:18:39.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.057+0000 7f187596c640 1 -- start start 2023-10-21T17:18:39.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1870159ac0 0x7f187006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18700ad790 0x7f18700702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f187015e330 0x7f1870070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187006dee0 con 0x7f1870159ac0 2023-10-21T17:18:39.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187015d850 con 0x7f187015e330 2023-10-21T17:18:39.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f187596c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187015db30 con 0x7f18700ad790 2023-10-21T17:18:39.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f186effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1870159ac0 0x7f187006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f186effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1870159ac0 0x7f187006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47686/0 (socket says 172.21.15.53:47686) 2023-10-21T17:18:39.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.059+0000 7f186effd640 1 -- 172.21.15.53:0/3536956058 learned_addr learned my addr 172.21.15.53:0/3536956058 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:39.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186f7fe640 1 --2- 172.21.15.53:0/3536956058 >> v2:172.21.15.124:3300/0 conn(0x7f187015e330 0x7f1870070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186e7fc640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18700ad790 0x7f18700702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186effd640 1 -- 172.21.15.53:0/3536956058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18700ad790 msgr2=0x7f18700702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:39.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186effd640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18700ad790 0x7f18700702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186effd640 1 -- 172.21.15.53:0/3536956058 >> v2:172.21.15.124:3300/0 conn(0x7f187015e330 msgr2=0x7f1870070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:39.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186effd640 1 --2- 172.21.15.53:0/3536956058 >> v2:172.21.15.124:3300/0 conn(0x7f187015e330 0x7f1870070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.060+0000 7f186effd640 1 -- 172.21.15.53:0/3536956058 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187015de10 con 0x7f1870159ac0 2023-10-21T17:18:39.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.061+0000 7f186effd640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f187006d5e0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18700ce730 tx=0x7f186002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd5ad32f77f708 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.061+0000 7f186dffb640 1 -- 172.21.15.53:0/3536956058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186002ca70 con 0x7f1870159ac0 2023-10-21T17:18:39.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f186dffb640 1 -- 172.21.15.53:0/3536956058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186002cbd0 con 0x7f1870159ac0 2023-10-21T17:18:39.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f186dffb640 1 -- 172.21.15.53:0/3536956058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18600343f0 con 0x7f1870159ac0 2023-10-21T17:18:39.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 -- 172.21.15.53:0/3536956058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 msgr2=0x7f187006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f187006d5e0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f18700ce730 tx=0x7f186002bed0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 -- 172.21.15.53:0/3536956058 shutdown_connections 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18700ad790 0x7f18700702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 --2- 172.21.15.53:0/3536956058 >> v2:172.21.15.124:3300/0 conn(0x7f187015e330 0x7f1870070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 --2- 172.21.15.53:0/3536956058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f187006d5e0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.062+0000 7f187596c640 1 -- 172.21.15.53:0/3536956058 >> 172.21.15.53:0/3536956058 conn(0x7f187006a380 msgr2=0x7f18700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:39.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.063+0000 7f187596c640 1 -- 172.21.15.53:0/3536956058 shutdown_connections 2023-10-21T17:18:39.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.063+0000 7f187596c640 1 -- 172.21.15.53:0/3536956058 wait complete. 2023-10-21T17:18:39.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.064+0000 7f187596c640 1 Processor -- start 2023-10-21T17:18:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.065+0000 7f187596c640 1 -- start start 2023-10-21T17:18:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.065+0000 7f187596c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18700ad790 0x7f18700cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.065+0000 7f187596c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f187596c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f187015e330 0x7f18700cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f187596c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870164ce0 con 0x7f1870159ac0 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f187596c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18700cc680 con 0x7f18700ad790 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f187596c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18700cc960 con 0x7f187015e330 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47700/0 (socket says 172.21.15.53:47700) 2023-10-21T17:18:39.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 -- 172.21.15.53:0/3661630301 learned_addr learned my addr 172.21.15.53:0/3661630301 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186f7fe640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f187015e330 0x7f18700cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186effd640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18700ad790 0x7f18700cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 -- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f187015e330 msgr2=0x7f18700cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f187015e330 0x7f18700cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 -- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18700ad790 msgr2=0x7f18700cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18700ad790 0x7f18700cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.066+0000 7f186e7fc640 1 -- 172.21.15.53:0/3661630301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009b30 con 0x7f1870159ac0 2023-10-21T17:18:39.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f186e7fc640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f185800c7f0 tx=0x7f185800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858004220 con 0x7f1870159ac0 2023-10-21T17:18:39.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858004380 con 0x7f1870159ac0 2023-10-21T17:18:39.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858007860 con 0x7f1870159ac0 2023-10-21T17:18:39.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1870164fa0 con 0x7f1870159ac0 2023-10-21T17:18:39.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.067+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187014a250 con 0x7f1870159ac0 2023-10-21T17:18:39.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.068+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18701654c0 con 0x7f1870159ac0 2023-10-21T17:18:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.069+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1858007bd0 con 0x7f1870159ac0 2023-10-21T17:18:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.070+0000 7f1867fff640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 0x7f185006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.070+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f18580899e0 con 0x7f1870159ac0 2023-10-21T17:18:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.071+0000 7f186effd640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 0x7f185006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.072+0000 7f186effd640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 0x7f185006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1860035000 tx=0x7f1860030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.074+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18701654c0 con 0x7f1870159ac0 2023-10-21T17:18:39.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.306+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18701654c0 con 0x7f1870159ac0 2023-10-21T17:18:39.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.307+0000 7f1867fff640 1 -- 172.21.15.53:0/3661630301 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f18701654c0 con 0x7f1870159ac0 2023-10-21T17:18:39.308 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:39.309 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7669,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2009143044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2009143044},{"type":"v1","addr":"172.21.15.124:6836","nonce":2009143044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9114,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/670897628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":670897628},{"type":"v1","addr":"172.21.15.124:6837","nonce":670897628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/297635375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":297635375},{"type":"v1","addr":"172.21.15.53:6834","nonce":297635375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9546,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1707680137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1707680137},{"type":"v1","addr":"172.21.15.53:6835","nonce":1707680137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[]} 2023-10-21T17:18:39.309 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 80 2023-10-21T17:18:39.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.309+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 msgr2=0x7f185006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.309+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 0x7f185006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1860035000 tx=0x7f1860030000 comp rx=0 tx=0).stop 2023-10-21T17:18:39.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.309+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 msgr2=0x7f1870149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.309+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f185800c7f0 tx=0x7f185800ccb0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 shutdown_connections 2023-10-21T17:18:39.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f185006af90 0x7f185006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f187015e330 0x7f18700cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1870159ac0 0x7f1870149780 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 --2- 172.21.15.53:0/3661630301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18700ad790 0x7f18700cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 >> 172.21.15.53:0/3661630301 conn(0x7f187006a380 msgr2=0x7f18700af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:39.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 shutdown_connections 2023-10-21T17:18:39.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.310+0000 7f187596c640 1 -- 172.21.15.53:0/3661630301 wait complete. 2023-10-21T17:18:39.329 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:18:39.330 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:18:39.330 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:18:39.330 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:18:39.331 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:18:39.331 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:18:39.331 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:18:39.332 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:18:39.332 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:18:39.332 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:18:39.332 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:18:39.333 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:18:39.333 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:18:39.333 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:18:39.334 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:18:39.334 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:18:39.352 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:18:39.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:18:39.355 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:18:39.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:18:39.360 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:18:39.360 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:18:39.367 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:18:39.367 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:18:39.401 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:18:39.402 DEBUG:teuthology.parallel:result is None 2023-10-21T17:18:39.408 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:18:39.409 DEBUG:teuthology.parallel:result is None 2023-10-21T17:18:39.415 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:18:39.416 DEBUG:teuthology.parallel:result is None 2023-10-21T17:18:39.423 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:18:39.424 DEBUG:teuthology.parallel:result is None 2023-10-21T17:18:39.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:18:39.568 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:18:39.569 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:18:39.581 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:18:39.584 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:18:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.621+0000 7f5d4184d640 1 Processor -- start 2023-10-21T17:18:39.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.621+0000 7f5d4184d640 1 -- start start 2023-10-21T17:18:39.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.622+0000 7f5d4184d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c15a820 0x7f5d3c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.622+0000 7f5d4184d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.622+0000 7f5d4184d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c0cab60 con 0x7f5d3c0ca680 2023-10-21T17:18:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.622+0000 7f5d4184d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cb960 con 0x7f5d3c15ac00 2023-10-21T17:18:39.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.622+0000 7f5d4184d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0cbdf0 con 0x7f5d3c15a820 2023-10-21T17:18:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47724/0 (socket says 172.21.15.53:47724) 2023-10-21T17:18:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 -- 172.21.15.53:0/315577260 learned_addr learned my addr 172.21.15.53:0/315577260 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d33fff640 1 --1- 172.21.15.53:0/315577260 >> v1:172.21.15.53:6789/0 conn(0x7f5d3c0ca680 0x7f5d3c0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54286/0 (socket says 172.21.15.53:54286) 2023-10-21T17:18:39.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4084b640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c15a820 0x7f5d3c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 -- 172.21.15.53:0/315577260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c15a820 msgr2=0x7f5d3c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c15a820 0x7f5d3c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 -- 172.21.15.53:0/315577260 >> v1:172.21.15.53:6789/0 conn(0x7f5d3c0ca680 legacy=0x7f5d3c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.623+0000 7f5d4104c640 1 -- 172.21.15.53:0/315577260 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0cb030 con 0x7f5d3c15ac00 2023-10-21T17:18:39.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.624+0000 7f5d4104c640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009870 tx=0x7f5d3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6057b6083b826161 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d337fe640 1 -- 172.21.15.53:0/315577260 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34031640 con 0x7f5d3c15ac00 2023-10-21T17:18:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d337fe640 1 -- 172.21.15.53:0/315577260 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340317a0 con 0x7f5d3c15ac00 2023-10-21T17:18:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d337fe640 1 -- 172.21.15.53:0/315577260 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34031b30 con 0x7f5d3c15ac00 2023-10-21T17:18:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 -- 172.21.15.53:0/315577260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 msgr2=0x7f5d3c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009870 tx=0x7f5d3402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 -- 172.21.15.53:0/315577260 shutdown_connections 2023-10-21T17:18:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c0b04b0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 --2- 172.21.15.53:0/315577260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c15a820 0x7f5d3c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.625+0000 7f5d4184d640 1 -- 172.21.15.53:0/315577260 >> 172.21.15.53:0/315577260 conn(0x7f5d3c06c050 msgr2=0x7f5d3c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:39.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.626+0000 7f5d4184d640 1 -- 172.21.15.53:0/315577260 shutdown_connections 2023-10-21T17:18:39.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.626+0000 7f5d4184d640 1 -- 172.21.15.53:0/315577260 wait complete. 2023-10-21T17:18:39.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.627+0000 7f5d4184d640 1 Processor -- start 2023-10-21T17:18:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.627+0000 7f5d4184d640 1 -- start start 2023-10-21T17:18:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c0ca680 0x7f5d3c1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d3c15a820 0x7f5d3c1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f2720 con 0x7f5d3c15ac00 2023-10-21T17:18:39.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c1f66e0 con 0x7f5d3c15a820 2023-10-21T17:18:39.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.628+0000 7f5d4184d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0aefe0 con 0x7f5d3c0ca680 2023-10-21T17:18:39.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47726/0 (socket says 172.21.15.53:47726) 2023-10-21T17:18:39.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 -- 172.21.15.53:0/3669008307 learned_addr learned my addr 172.21.15.53:0/3669008307 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:39.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4084b640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c0ca680 0x7f5d3c1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d33fff640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d3c15a820 0x7f5d3c1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 -- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c0ca680 msgr2=0x7f5d3c1f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c0ca680 0x7f5d3c1f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 -- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d3c15a820 msgr2=0x7f5d3c1f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:39.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d3c15a820 0x7f5d3c1f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:39.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.629+0000 7f5d4104c640 1 -- 172.21.15.53:0/3669008307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340094d0 con 0x7f5d3c15ac00 2023-10-21T17:18:39.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d4104c640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f5d340090b0 tx=0x7f5d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3403b940 con 0x7f5d3c15ac00 2023-10-21T17:18:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3403bae0 con 0x7f5d3c15ac00 2023-10-21T17:18:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3403be90 con 0x7f5d3c15ac00 2023-10-21T17:18:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c0aea20 con 0x7f5d3c15ac00 2023-10-21T17:18:39.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.630+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c1f23b0 con 0x7f5d3c15ac00 2023-10-21T17:18:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.631+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c1f2890 con 0x7f5d3c15ac00 2023-10-21T17:18:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.632+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d3402f050 con 0x7f5d3c15ac00 2023-10-21T17:18:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.633+0000 7f5d317fa640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 0x7f5d2806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.633+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f5d340ab7c0 con 0x7f5d3c15ac00 2023-10-21T17:18:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.634+0000 7f5d4084b640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 0x7f5d2806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.635+0000 7f5d4084b640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 0x7f5d2806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c151850 tx=0x7f5d3800b040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.637+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d34076620 con 0x7f5d3c15ac00 2023-10-21T17:18:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:39.846+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5d3c074150 con 0x7f5d3c15ac00 2023-10-21T17:18:40.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.866+0000 7f5d317fa640 1 -- 172.21.15.53:0/3669008307 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d34079c40 con 0x7f5d3c15ac00 2023-10-21T17:18:40.867 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.870+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 msgr2=0x7f5d2806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.870+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 0x7f5d2806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c151850 tx=0x7f5d3800b040 comp rx=0 tx=0).stop 2023-10-21T17:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.870+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 msgr2=0x7f5d3c1f1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:40.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.870+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f5d340090b0 tx=0x7f5d340046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 shutdown_connections 2023-10-21T17:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f5d2806af90 0x7f5d2806d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d3c15ac00 0x7f5d3c1f1b10 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d3c15a820 0x7f5d3c1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:40.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 --2- 172.21.15.53:0/3669008307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d3c0ca680 0x7f5d3c1f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 >> 172.21.15.53:0/3669008307 conn(0x7f5d3c06c050 msgr2=0x7f5d3c0c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 shutdown_connections 2023-10-21T17:18:40.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:40.871+0000 7f5d4184d640 1 -- 172.21.15.53:0/3669008307 wait complete. 2023-10-21T17:18:40.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:18:41.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.114+0000 7f312c900640 1 Processor -- start 2023-10-21T17:18:41.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.114+0000 7f312c900640 1 -- start start 2023-10-21T17:18:41.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.115+0000 7f312c900640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f312815ab30 0x7f312815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.115+0000 7f312c900640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 0x7f312806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.115+0000 7f312c900640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312815f7d0 con 0x7f31280adc70 2023-10-21T17:18:41.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.115+0000 7f312c900640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312815f210 con 0x7f312815ab30 2023-10-21T17:18:41.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.115+0000 7f312c900640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312815f4f0 con 0x7f31280b07d0 2023-10-21T17:18:41.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.116+0000 7f3126d9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f312815ab30 0x7f312815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.116+0000 7f312759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 0x7f312806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.116+0000 7f311e59c640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f31280adc70 0x7f31280b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:44940/0 (socket says 172.21.15.53:44940) 2023-10-21T17:18:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.116+0000 7f311e59c640 1 -- 172.21.15.53:0/2257080951 learned_addr learned my addr 172.21.15.53:0/2257080951 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f312659c640 1 -- 172.21.15.53:0/2257080951 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046955030 0 0) 0x7f312815f7d0 con 0x7f31280adc70 2023-10-21T17:18:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f312659c640 1 -- 172.21.15.53:0/2257080951 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3114003520 con 0x7f31280adc70 2023-10-21T17:18:41.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f3126d9d640 1 -- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 msgr2=0x7f312806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f3126d9d640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 0x7f312806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f3126d9d640 1 -- 172.21.15.53:0/2257080951 >> v1:172.21.15.124:6789/0 conn(0x7f31280adc70 legacy=0x7f31280b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f3126d9d640 1 -- 172.21.15.53:0/2257080951 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312806c550 con 0x7f312815ab30 2023-10-21T17:18:41.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.117+0000 7f312759e640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 0x7f312806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:18:41.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.119+0000 7f3126d9d640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f312815ab30 0x7f312815e8a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f31180097a0 tx=0x7f311802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6569bde66f246d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.119+0000 7f312659c640 1 -- 172.21.15.53:0/2257080951 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311802d040 con 0x7f312815ab30 2023-10-21T17:18:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.119+0000 7f312659c640 1 -- 172.21.15.53:0/2257080951 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118031810 con 0x7f312815ab30 2023-10-21T17:18:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.119+0000 7f312659c640 1 -- 172.21.15.53:0/2257080951 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118031ba0 con 0x7f312815ab30 2023-10-21T17:18:41.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 -- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f312815ab30 msgr2=0x7f312815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f312815ab30 0x7f312815e8a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f31180097a0 tx=0x7f311802b8b0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 -- 172.21.15.53:0/2257080951 shutdown_connections 2023-10-21T17:18:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31280b07d0 0x7f312806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 --2- 172.21.15.53:0/2257080951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f312815ab30 0x7f312815e8a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 -- 172.21.15.53:0/2257080951 >> 172.21.15.53:0/2257080951 conn(0x7f31280cb0c0 msgr2=0x7f31280cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 -- 172.21.15.53:0/2257080951 shutdown_connections 2023-10-21T17:18:41.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.120+0000 7f312c900640 1 -- 172.21.15.53:0/2257080951 wait complete. 2023-10-21T17:18:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.123+0000 7f312c900640 1 Processor -- start 2023-10-21T17:18:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.123+0000 7f312c900640 1 -- start start 2023-10-21T17:18:41.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31280b07d0 0x7f3128156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f312815ab30 0x7f31280c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c6ce0 con 0x7f31280adc70 2023-10-21T17:18:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c6a00 con 0x7f31280b07d0 2023-10-21T17:18:41.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f312c900640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312815d000 con 0x7f312815ab30 2023-10-21T17:18:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f3126d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f3126d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47736/0 (socket says 172.21.15.53:47736) 2023-10-21T17:18:41.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.124+0000 7f3126d9d640 1 -- 172.21.15.53:0/4055293352 learned_addr learned my addr 172.21.15.53:0/4055293352 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f311e59c640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31280b07d0 0x7f3128156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f312759e640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f312815ab30 0x7f31280c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 -- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f312815ab30 msgr2=0x7f31280c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f312815ab30 0x7f31280c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 -- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31280b07d0 msgr2=0x7f3128156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31280b07d0 0x7f3128156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 -- 172.21.15.53:0/4055293352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009a70 con 0x7f31280adc70 2023-10-21T17:18:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.125+0000 7f3126d9d640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f31180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.126+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311803ad60 con 0x7f31280adc70 2023-10-21T17:18:41.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.126+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31180443f0 con 0x7f31280adc70 2023-10-21T17:18:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.126+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180445a0 con 0x7f31280adc70 2023-10-21T17:18:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.126+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3118009400 con 0x7f31280adc70 2023-10-21T17:18:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.126+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312815d5c0 con 0x7f31280adc70 2023-10-21T17:18:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.127+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312804e160 con 0x7f31280adc70 2023-10-21T17:18:41.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.128+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f311804d450 con 0x7f31280adc70 2023-10-21T17:18:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.129+0000 7f311ffff640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 0x7f310806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.129+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f31180ab790 con 0x7f31280adc70 2023-10-21T17:18:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.130+0000 7f311e59c640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 0x7f310806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.131+0000 7f311e59c640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 0x7f310806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312806e210 tx=0x7f3110009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.133+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3118076830 con 0x7f31280adc70 2023-10-21T17:18:41.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.397+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f312815d7b0 con 0x7f31280adc70 2023-10-21T17:18:41.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.398+0000 7f311ffff640 1 -- 172.21.15.53:0/4055293352 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3118079e50 con 0x7f31280adc70 2023-10-21T17:18:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:41.400 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:18:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:18:41.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:18:41.401 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:18:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:18:41.403 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:18:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.404 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:18:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:18:41.406 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.408 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:18:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:18:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.410 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.411 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:18:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:18:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.412 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:18:41.413 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.415 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.417 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:18:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:18:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:18:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.420 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.421 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:18:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:18:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.422 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:18:41.423 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:18:41.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:18:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:18:41.425 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:18:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:18:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.427 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:18:41.429 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:18:41.430 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.430 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:18:41.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:41.432 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:18:41.432 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:18:41.432 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:18:41.433 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:18:41.433 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:18:41.433 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:18:41.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:41.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:18:41.434 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:18:41.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 msgr2=0x7f310806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 0x7f310806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f312806e210 tx=0x7f3110009040 comp rx=0 tx=0).stop 2023-10-21T17:18:41.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 msgr2=0x7f3128157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f31180046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 shutdown_connections 2023-10-21T17:18:41.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f310806af90 0x7f310806d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f312815ab30 0x7f31280c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31280b07d0 0x7f3128156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 --2- 172.21.15.53:0/4055293352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31280adc70 0x7f3128157bb0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 >> 172.21.15.53:0/4055293352 conn(0x7f31280cb0c0 msgr2=0x7f31280c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:41.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.401+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 shutdown_connections 2023-10-21T17:18:41.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.402+0000 7f312c900640 1 -- 172.21.15.53:0/4055293352 wait complete. 2023-10-21T17:18:41.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:41.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.669+0000 7f06c11b0640 1 Processor -- start 2023-10-21T17:18:41.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.669+0000 7f06c11b0640 1 -- start start 2023-10-21T17:18:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06c11b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f06bc0c90b0 0x7f06bc0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06c11b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06c11b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc15b2b0 con 0x7f06bc15ab40 2023-10-21T17:18:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06c11b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc15bf30 con 0x7f06bc0c90b0 2023-10-21T17:18:41.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06c11b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cb820 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06bb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06ba59c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f06bc0c90b0 0x7f06bc0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06bb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:34916/0 (socket says 172.21.15.53:34916) 2023-10-21T17:18:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.671+0000 7f06bb59e640 1 -- 172.21.15.53:0/1015641191 learned_addr learned my addr 172.21.15.53:0/1015641191 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:41.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.672+0000 7f06bb59e640 1 -- 172.21.15.53:0/1015641191 >> v2:172.21.15.124:3300/0 conn(0x7f06bc0c90b0 msgr2=0x7f06bc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:41.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.672+0000 7f06bb59e640 1 --2- 172.21.15.53:0/1015641191 >> v2:172.21.15.124:3300/0 conn(0x7f06bc0c90b0 0x7f06bc0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.672+0000 7f06bb59e640 1 -- 172.21.15.53:0/1015641191 >> v1:172.21.15.124:6789/0 conn(0x7f06bc15ab40 legacy=0x7f06bc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.672+0000 7f06bb59e640 1 -- 172.21.15.53:0/1015641191 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc15b590 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.673+0000 7f06bb59e640 1 --2- 172.21.15.53:0/1015641191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009e50 tx=0x7f06b402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262dfd981210c066 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.674+0000 7f06b9d9b640 1 -- 172.21.15.53:0/1015641191 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b402cb00 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.674+0000 7f06b9d9b640 1 -- 172.21.15.53:0/1015641191 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b402cc60 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.674+0000 7f06b9d9b640 1 -- 172.21.15.53:0/1015641191 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b40343f0 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 -- 172.21.15.53:0/1015641191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 msgr2=0x7f06bc15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 --2- 172.21.15.53:0/1015641191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009e50 tx=0x7f06b402bf60 comp rx=0 tx=0).stop 2023-10-21T17:18:41.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 -- 172.21.15.53:0/1015641191 shutdown_connections 2023-10-21T17:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 --2- 172.21.15.53:0/1015641191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc15e790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 --2- 172.21.15.53:0/1015641191 >> v2:172.21.15.124:3300/0 conn(0x7f06bc0c90b0 0x7f06bc0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 -- 172.21.15.53:0/1015641191 >> 172.21.15.53:0/1015641191 conn(0x7f06bc06a380 msgr2=0x7f06bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 -- 172.21.15.53:0/1015641191 shutdown_connections 2023-10-21T17:18:41.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.675+0000 7f06c11b0640 1 -- 172.21.15.53:0/1015641191 wait complete. 2023-10-21T17:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.676+0000 7f06c11b0640 1 Processor -- start 2023-10-21T17:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 -- start start 2023-10-21T17:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 0x7f06bc14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc154520 con 0x7f06bc15ab40 2023-10-21T17:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc15a000 con 0x7f06bc0c90b0 2023-10-21T17:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.677+0000 7f06c11b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc155ca0 con 0x7f06bc0cb3b0 2023-10-21T17:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47752/0 (socket says 172.21.15.53:47752) 2023-10-21T17:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 -- 172.21.15.53:0/4009865486 learned_addr learned my addr 172.21.15.53:0/4009865486 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bad9d640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 0x7f06bc14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06ba59c640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 -- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 msgr2=0x7f06bc0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 -- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 msgr2=0x7f06bc14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 0x7f06bc14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.678+0000 7f06bb59e640 1 -- 172.21.15.53:0/4009865486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b4009b30 con 0x7f06bc15ab40 2023-10-21T17:18:41.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06bad9d640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 0x7f06bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:41.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06bb59e640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f06b4036040 tx=0x7f06b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4009510 con 0x7f06bc15ab40 2023-10-21T17:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b4040470 con 0x7f06bc15ab40 2023-10-21T17:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4040800 con 0x7f06bc15ab40 2023-10-21T17:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc155f80 con 0x7f06bc15ab40 2023-10-21T17:18:41.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.679+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc15a2e0 con 0x7f06bc15ab40 2023-10-21T17:18:41.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.680+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc1564a0 con 0x7f06bc15ab40 2023-10-21T17:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.681+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06b4040960 con 0x7f06bc15ab40 2023-10-21T17:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.682+0000 7f06a37fe640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 0x7f06a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.682+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f06b40ac3c0 con 0x7f06bc15ab40 2023-10-21T17:18:41.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.683+0000 7f06bad9d640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 0x7f06a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.684+0000 7f06bad9d640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 0x7f06a806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:41.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.687+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06bc1564a0 con 0x7f06bc15ab40 2023-10-21T17:18:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.881+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06bc1564a0 con 0x7f06bc15ab40 2023-10-21T17:18:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.882+0000 7f06a37fe640 1 -- 172.21.15.53:0/4009865486 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f06bc1564a0 con 0x7f06bc15ab40 2023-10-21T17:18:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:41.883 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":180,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:40.863217+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 msgr2=0x7f06a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 0x7f06a806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac009040 comp rx=0 tx=0).stop 2023-10-21T17:18:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 msgr2=0x7f06bc0af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f06b4036040 tx=0x7f06b4004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 shutdown_connections 2023-10-21T17:18:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f06a806af90 0x7f06a806d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06bc15ab40 0x7f06bc0af8c0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06bc0cb3b0 0x7f06bc0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 --2- 172.21.15.53:0/4009865486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06bc0c90b0 0x7f06bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:41.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 >> 172.21.15.53:0/4009865486 conn(0x7f06bc06a380 msgr2=0x7f06bc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:41.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 shutdown_connections 2023-10-21T17:18:41.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:41.885+0000 7f06c11b0640 1 -- 172.21.15.53:0/4009865486 wait complete. 2023-10-21T17:18:41.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:18:41.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:18:42.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.140+0000 7f1064be2640 1 Processor -- start 2023-10-21T17:18:42.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.140+0000 7f1064be2640 1 -- start start 2023-10-21T17:18:42.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f1064be2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f10600cb7b0 0x7f10600cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f1064be2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600721b0 con 0x7f1060071d40 2023-10-21T17:18:42.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f1064be2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106015d320 con 0x7f10600ce4f0 2023-10-21T17:18:42.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f1064be2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106015d600 con 0x7f10600cb7b0 2023-10-21T17:18:42.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f105effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1060071d40 0x7f10600cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:54292/0 (socket says 172.21.15.53:54292) 2023-10-21T17:18:42.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f105effd640 1 -- 172.21.15.53:0/2230338255 learned_addr learned my addr 172.21.15.53:0/2230338255 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:42.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f105ffff640 1 --2- 172.21.15.53:0/2230338255 >> v2:172.21.15.124:3300/0 conn(0x7f10600cb7b0 0x7f10600cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.142+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115143139 0 0) 0x7f10600721b0 con 0x7f1060071d40 2023-10-21T17:18:42.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1048005730 con 0x7f1060071d40 2023-10-21T17:18:42.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361292692 0 0) 0x7f1048005730 con 0x7f1060071d40 2023-10-21T17:18:42.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 >> v2:172.21.15.124:3300/0 conn(0x7f10600cb7b0 msgr2=0x7f10600cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 --2- 172.21.15.53:0/2230338255 >> v2:172.21.15.124:3300/0 conn(0x7f10600cb7b0 0x7f10600cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 >> v1:172.21.15.124:6789/0 conn(0x7f10600ce4f0 legacy=0x7f10600ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106015d8e0 con 0x7f1060071d40 2023-10-21T17:18:42.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1054001670 con 0x7f1060071d40 2023-10-21T17:18:42.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 >> v1:172.21.15.53:6789/0 conn(0x7f1060071d40 legacy=0x7f10600cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048005bd0 0x7f1048005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480069d0 0x7f1048006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 0x7f104800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106015d600 con 0x7f1048007390 2023-10-21T17:18:42.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104800e260 con 0x7f1048005bd0 2023-10-21T17:18:42.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.143+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104800e420 con 0x7f10480069d0 2023-10-21T17:18:42.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 0x7f104800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105f7fe640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048005bd0 0x7f1048005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105effd640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480069d0 0x7f1048006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 -- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480069d0 msgr2=0x7f1048006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10480069d0 0x7f1048006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 -- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048005bd0 msgr2=0x7f1048005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048005bd0 0x7f1048005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.144+0000 7f105ffff640 1 -- 172.21.15.53:0/2230338255 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048005730 con 0x7f1048007390 2023-10-21T17:18:42.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.146+0000 7f105ffff640 1 -- 172.21.15.53:0/2230338255 reap_dead start 2023-10-21T17:18:42.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.146+0000 7f105ffff640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 0x7f104800db90 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1058009ac0 tx=0x7f1048007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2560b76f4bd37cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.146+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058004020 con 0x7f1048007390 2023-10-21T17:18:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.146+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1058004180 con 0x7f1048007390 2023-10-21T17:18:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.146+0000 7f105e7fc640 1 -- 172.21.15.53:0/2230338255 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105802c710 con 0x7f1048007390 2023-10-21T17:18:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 -- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 msgr2=0x7f104800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 0x7f104800db90 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1058009ac0 tx=0x7f1048007a00 comp rx=0 tx=0).stop 2023-10-21T17:18:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 -- 172.21.15.53:0/2230338255 shutdown_connections 2023-10-21T17:18:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 --2- 172.21.15.53:0/2230338255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1048007390 0x7f104800db90 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 -- 172.21.15.53:0/2230338255 >> 172.21.15.53:0/2230338255 conn(0x7f106006b250 msgr2=0x7f106006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 -- 172.21.15.53:0/2230338255 shutdown_connections 2023-10-21T17:18:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.147+0000 7f1064be2640 1 -- 172.21.15.53:0/2230338255 wait complete. 2023-10-21T17:18:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.149+0000 7f1064be2640 1 Processor -- start 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.149+0000 7f1064be2640 1 -- start start 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048007390 0x7f10600ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 0x7f10600b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600b0c20 con 0x7f10601f1950 2023-10-21T17:18:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600adbd0 con 0x7f1048007390 2023-10-21T17:18:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f1064be2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600b0010 con 0x7f10601f1dd0 2023-10-21T17:18:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f105effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f105effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47782/0 (socket says 172.21.15.53:47782) 2023-10-21T17:18:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f105effd640 1 -- 172.21.15.53:0/2565617327 learned_addr learned my addr 172.21.15.53:0/2565617327 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.150+0000 7f105f7fe640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048007390 0x7f10600ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105ffff640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 0x7f10600b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 -- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 msgr2=0x7f10600b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 0x7f10600b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 -- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048007390 msgr2=0x7f10600ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048007390 0x7f10600ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 -- 172.21.15.53:0/2565617327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600b02f0 con 0x7f10601f1950 2023-10-21T17:18:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105ffff640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 0x7f10600b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f105effd640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f1054007fd0 tx=0x7f1054007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054001500 con 0x7f10601f1950 2023-10-21T17:18:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105400f9c0 con 0x7f10601f1950 2023-10-21T17:18:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400fd70 con 0x7f10601f1950 2023-10-21T17:18:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10600b05d0 con 0x7f10601f1950 2023-10-21T17:18:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.151+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10600afc30 con 0x7f10601f1950 2023-10-21T17:18:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.153+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1060071380 con 0x7f10601f1950 2023-10-21T17:18:42.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.154+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f105401d460 con 0x7f10601f1950 2023-10-21T17:18:42.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.155+0000 7f1047fff640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 0x7f104c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.155+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f1054089770 con 0x7f10601f1950 2023-10-21T17:18:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.155+0000 7f105f7fe640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 0x7f104c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.156+0000 7f105f7fe640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 0x7f104c06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1050005600 tx=0x7f1050009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.159+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10540548b0 con 0x7f10601f1950 2023-10-21T17:18:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.359+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1060074150 con 0x7f10601f1950 2023-10-21T17:18:42.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.360+0000 7f1047fff640 1 -- 172.21.15.53:0/2565617327 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1054057ed0 con 0x7f10601f1950 2023-10-21T17:18:42.362 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:18:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 msgr2=0x7f104c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 0x7f104c06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1050005600 tx=0x7f1050009040 comp rx=0 tx=0).stop 2023-10-21T17:18:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 msgr2=0x7f10600b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f1054007fd0 tx=0x7f1054007720 comp rx=0 tx=0).stop 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 shutdown_connections 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f104c06afc0 0x7f104c06d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10601f1dd0 0x7f10600b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10601f1950 0x7f10600b14b0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 --2- 172.21.15.53:0/2565617327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1048007390 0x7f10600ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 >> 172.21.15.53:0/2565617327 conn(0x7f106006b250 msgr2=0x7f106006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 shutdown_connections 2023-10-21T17:18:42.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.363+0000 7f1064be2640 1 -- 172.21.15.53:0/2565617327 wait complete. 2023-10-21T17:18:42.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:18:42.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:18:42.438 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:18:42.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.595+0000 7f0bdfb33640 1 Processor -- start 2023-10-21T17:18:42.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.595+0000 7f0bdfb33640 1 -- start start 2023-10-21T17:18:42.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdfb33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd8159bc0 0x7f0bd80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdfb33640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0bd8070810 0x7f0bd80c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdfb33640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd815e000 con 0x7f0bd80c4f80 2023-10-21T17:18:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdfb33640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd80c9de0 con 0x7f0bd8070810 2023-10-21T17:18:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdfb33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8159fa0 con 0x7f0bd8159bc0 2023-10-21T17:18:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdd0ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0bd8070810 0x7f0bd80c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdd0ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0bd8070810 0x7f0bd80c4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47796/0 (socket says 172.21.15.53:47796) 2023-10-21T17:18:42.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.597+0000 7f0bdd0ce640 1 -- 172.21.15.53:0/2405446276 learned_addr learned my addr 172.21.15.53:0/2405446276 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.598+0000 7f0bdd0ce640 1 -- 172.21.15.53:0/2405446276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd8159bc0 msgr2=0x7f0bd80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.598+0000 7f0bdd0ce640 1 --2- 172.21.15.53:0/2405446276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd8159bc0 0x7f0bd80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.598+0000 7f0bdd0ce640 1 -- 172.21.15.53:0/2405446276 >> v1:172.21.15.124:6789/0 conn(0x7f0bd80c4f80 legacy=0x7f0bd806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.598+0000 7f0bdd0ce640 1 -- 172.21.15.53:0/2405446276 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8070d20 con 0x7f0bd8070810 2023-10-21T17:18:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.599+0000 7f0bdd0ce640 1 --2- 172.21.15.53:0/2405446276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80c4a40 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800bb10 tx=0x7f0bc802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78712d60bb1ace4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.599+0000 7f0bdc8cd640 1 -- 172.21.15.53:0/2405446276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc802ee70 con 0x7f0bd8070810 2023-10-21T17:18:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.599+0000 7f0bdc8cd640 1 -- 172.21.15.53:0/2405446276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc8036440 con 0x7f0bd8070810 2023-10-21T17:18:42.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.599+0000 7f0bdc8cd640 1 -- 172.21.15.53:0/2405446276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80367d0 con 0x7f0bd8070810 2023-10-21T17:18:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2405446276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 msgr2=0x7f0bd80c4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2405446276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80c4a40 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800bb10 tx=0x7f0bc802e0c0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2405446276 shutdown_connections 2023-10-21T17:18:42.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2405446276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80c4a40 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2405446276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd8159bc0 0x7f0bd80702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2405446276 >> 172.21.15.53:0/2405446276 conn(0x7f0bd806a380 msgr2=0x7f0bd80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2405446276 shutdown_connections 2023-10-21T17:18:42.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.600+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2405446276 wait complete. 2023-10-21T17:18:42.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.602+0000 7f0bdfb33640 1 Processor -- start 2023-10-21T17:18:42.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.602+0000 7f0bdfb33640 1 -- start start 2023-10-21T17:18:42.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd80c4f80 0x7f0bd814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bd8159bc0 0x7f0bd815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80acec0 con 0x7f0bd8070810 2023-10-21T17:18:42.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd815d180 con 0x7f0bd8159bc0 2023-10-21T17:18:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdfb33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd80b22c0 con 0x7f0bd80c4f80 2023-10-21T17:18:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdd8cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdd8cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47798/0 (socket says 172.21.15.53:47798) 2023-10-21T17:18:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.603+0000 7f0bdd8cf640 1 -- 172.21.15.53:0/2288665644 learned_addr learned my addr 172.21.15.53:0/2288665644 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:42.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd0ce640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd80c4f80 0x7f0bd814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bde0d0640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bd8159bc0 0x7f0bd815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 -- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd80c4f80 msgr2=0x7f0bd814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd80c4f80 0x7f0bd814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 -- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bd8159bc0 msgr2=0x7f0bd815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bd8159bc0 0x7f0bd815ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 -- 172.21.15.53:0/2288665644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc800b770 con 0x7f0bd8070810 2023-10-21T17:18:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdd8cf640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400b410 tx=0x7f0bd400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4011830 con 0x7f0bd8070810 2023-10-21T17:18:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd40119b0 con 0x7f0bd8070810 2023-10-21T17:18:42.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4011d60 con 0x7f0bd8070810 2023-10-21T17:18:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.604+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd80ad0e0 con 0x7f0bd8070810 2023-10-21T17:18:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.605+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd80ad6a0 con 0x7f0bd8070810 2023-10-21T17:18:42.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.606+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd80b2430 con 0x7f0bd8070810 2023-10-21T17:18:42.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.607+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0bd401a400 con 0x7f0bd8070810 2023-10-21T17:18:42.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.608+0000 7f0bce7fc640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 0x7f0bbc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:42.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.608+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f0bd40897c0 con 0x7f0bd8070810 2023-10-21T17:18:42.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.609+0000 7f0bdd0ce640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 0x7f0bbc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:42.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.610+0000 7f0bdd0ce640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 0x7f0bbc06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8038000 tx=0x7f0bc8008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:42.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.612+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bd4054860 con 0x7f0bd8070810 2023-10-21T17:18:42.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.807+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bd8067490 con 0x7f0bd8070810 2023-10-21T17:18:42.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.808+0000 7f0bce7fc640 1 -- 172.21.15.53:0/2288665644 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0bd4015090 con 0x7f0bd8070810 2023-10-21T17:18:42.810 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:42.810 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:18:42.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:42.810 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:18:42.811 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:18:42.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:42.811 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:18:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 msgr2=0x7f0bbc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 0x7f0bbc06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8038000 tx=0x7f0bc8008040 comp rx=0 tx=0).stop 2023-10-21T17:18:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 msgr2=0x7f0bd80ac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400b410 tx=0x7f0bd400b8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 shutdown_connections 2023-10-21T17:18:42.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f0bbc06af90 0x7f0bbc06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bd8159bc0 0x7f0bd815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bd80c4f80 0x7f0bd814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 --2- 172.21.15.53:0/2288665644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bd8070810 0x7f0bd80ac980 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:42.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.811+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 >> 172.21.15.53:0/2288665644 conn(0x7f0bd806a380 msgr2=0x7f0bd806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:42.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.812+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 shutdown_connections 2023-10-21T17:18:42.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:42.812+0000 7f0bdfb33640 1 -- 172.21.15.53:0/2288665644 wait complete. 2023-10-21T17:18:42.832 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:18:42.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:18:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.062+0000 7f7d01ba5640 1 Processor -- start 2023-10-21T17:18:43.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.062+0000 7f7d01ba5640 1 -- start start 2023-10-21T17:18:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.063+0000 7f7d01ba5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.063+0000 7f7d01ba5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7cfc06bf70 0x7f7cfc0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.063+0000 7f7d01ba5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0cf930 con 0x7f7cfc0b2ac0 2023-10-21T17:18:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.063+0000 7f7d01ba5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc15d280 con 0x7f7cfc15cea0 2023-10-21T17:18:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.063+0000 7f7d01ba5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b2ed0 con 0x7f7cfc06bf70 2023-10-21T17:18:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.064+0000 7f7cfb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.064+0000 7f7cfb7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47802/0 (socket says 172.21.15.53:47802) 2023-10-21T17:18:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.064+0000 7f7cfb7fe640 1 -- 172.21.15.53:0/3435929432 learned_addr learned my addr 172.21.15.53:0/3435929432 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.064+0000 7f7cfaffd640 1 --2- 172.21.15.53:0/3435929432 >> v2:172.21.15.124:3300/0 conn(0x7f7cfc06bf70 0x7f7cfc0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.065+0000 7f7cfb7fe640 1 -- 172.21.15.53:0/3435929432 >> v2:172.21.15.124:3300/0 conn(0x7f7cfc06bf70 msgr2=0x7f7cfc0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:43.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.065+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/3435929432 >> v2:172.21.15.124:3300/0 conn(0x7f7cfc06bf70 0x7f7cfc0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.065+0000 7f7cfb7fe640 1 -- 172.21.15.53:0/3435929432 >> v1:172.21.15.124:6789/0 conn(0x7f7cfc0b2ac0 legacy=0x7f7cfc160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.065+0000 7f7cfb7fe640 1 -- 172.21.15.53:0/3435929432 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc06c710 con 0x7f7cfc15cea0 2023-10-21T17:18:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.066+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/3435929432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc071120 tx=0x7f7ce402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abd12e79b549c8e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.067+0000 7f7cfa7fc640 1 -- 172.21.15.53:0/3435929432 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4030860 con 0x7f7cfc15cea0 2023-10-21T17:18:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.067+0000 7f7cfa7fc640 1 -- 172.21.15.53:0/3435929432 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce40309c0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.067+0000 7f7cfa7fc640 1 -- 172.21.15.53:0/3435929432 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4030d50 con 0x7f7cfc15cea0 2023-10-21T17:18:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.067+0000 7f7d01ba5640 1 -- 172.21.15.53:0/3435929432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 msgr2=0x7f7cfc0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.067+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/3435929432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc071120 tx=0x7f7ce402b9a0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 -- 172.21.15.53:0/3435929432 shutdown_connections 2023-10-21T17:18:43.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/3435929432 >> v2:172.21.15.124:3300/0 conn(0x7f7cfc06bf70 0x7f7cfc0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/3435929432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc0cf3f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 -- 172.21.15.53:0/3435929432 >> 172.21.15.53:0/3435929432 conn(0x7f7cfc06e670 msgr2=0x7f7cfc06ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 -- 172.21.15.53:0/3435929432 shutdown_connections 2023-10-21T17:18:43.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.068+0000 7f7d01ba5640 1 -- 172.21.15.53:0/3435929432 wait complete. 2023-10-21T17:18:43.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.069+0000 7f7d01ba5640 1 Processor -- start 2023-10-21T17:18:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.070+0000 7f7d01ba5640 1 -- start start 2023-10-21T17:18:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 0x7f7cfc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7cfc0b2ac0 0x7f7cfc14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c9730 con 0x7f7cfc15cea0 2023-10-21T17:18:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c9a10 con 0x7f7cfc0b2ac0 2023-10-21T17:18:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7d01ba5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c9cf0 con 0x7f7cfc06bf70 2023-10-21T17:18:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47810/0 (socket says 172.21.15.53:47810) 2023-10-21T17:18:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 -- 172.21.15.53:0/416824364 learned_addr learned my addr 172.21.15.53:0/416824364 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 0x7f7cfc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfaffd640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7cfc0b2ac0 0x7f7cfc14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 -- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 msgr2=0x7f7cfc0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 0x7f7cfc0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 -- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7cfc0b2ac0 msgr2=0x7f7cfc14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7cfc0b2ac0 0x7f7cfc14cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfbfff640 1 -- 172.21.15.53:0/416824364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40094d0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.071+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 0x7f7cfc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:43.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7cfbfff640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4001eb0 tx=0x7f7cf400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40052d0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4005430 con 0x7f7cfc15cea0 2023-10-21T17:18:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4003810 con 0x7f7cfc15cea0 2023-10-21T17:18:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0c9fd0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.072+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc0c76a0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.073+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc0ca4f0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.074+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7cf4003970 con 0x7f7cfc15cea0 2023-10-21T17:18:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.076+0000 7f7cf3fff640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 0x7f7cdc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.076+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f7cf4088a60 con 0x7f7cfc15cea0 2023-10-21T17:18:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.077+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 0x7f7cdc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:43.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.078+0000 7f7cfb7fe640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 0x7f7cdc06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036000 tx=0x7f7ce403b040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:43.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.081+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cfc0ca4f0 con 0x7f7cfc15cea0 2023-10-21T17:18:43.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.273+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7cfc0cf930 con 0x7f7cfc15cea0 2023-10-21T17:18:43.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.884+0000 7f7cf3fff640 1 -- 172.21.15.53:0/416824364 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7cfc0cf930 con 0x7f7cfc15cea0 2023-10-21T17:18:43.885 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:18:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 msgr2=0x7f7cdc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 0x7f7cdc06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4036000 tx=0x7f7ce403b040 comp rx=0 tx=0).stop 2023-10-21T17:18:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 msgr2=0x7f7cfc14d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4001eb0 tx=0x7f7cf400c890 comp rx=0 tx=0).stop 2023-10-21T17:18:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 shutdown_connections 2023-10-21T17:18:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f7cdc06af90 0x7f7cdc06d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7cfc15cea0 0x7f7cfc14d0a0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7cfc0b2ac0 0x7f7cfc14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 --2- 172.21.15.53:0/416824364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7cfc06bf70 0x7f7cfc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:43.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.888+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 >> 172.21.15.53:0/416824364 conn(0x7f7cfc06e670 msgr2=0x7f7cfc06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:43.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.889+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 shutdown_connections 2023-10-21T17:18:43.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:43.889+0000 7f7d01ba5640 1 -- 172.21.15.53:0/416824364 wait complete. 2023-10-21T17:18:43.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:18:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.138+0000 7f8b5e3d7640 1 Processor -- start 2023-10-21T17:18:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.138+0000 7f8b5e3d7640 1 -- start start 2023-10-21T17:18:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.139+0000 7f8b5e3d7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b580ca9a0 0x7f8b5815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.139+0000 7f8b5e3d7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b580c9c90 con 0x7f8b5815de70 2023-10-21T17:18:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.139+0000 7f8b5e3d7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5815e250 con 0x7f8b580cb970 2023-10-21T17:18:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.139+0000 7f8b5e3d7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5815f480 con 0x7f8b580ca9a0 2023-10-21T17:18:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5d3d5640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8b5815de70 0x7f8b580cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49962/0 (socket says 172.21.15.53:49962) 2023-10-21T17:18:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5d3d5640 1 -- 172.21.15.53:0/4026699271 learned_addr learned my addr 172.21.15.53:0/4026699271 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5dbd6640 1 --2- 172.21.15.53:0/4026699271 >> v2:172.21.15.53:3300/0 conn(0x7f8b580ca9a0 0x7f8b5815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b4ffff640 1 -- 172.21.15.53:0/4026699271 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527701303 0 0) 0x7f8b580c9c90 con 0x7f8b5815de70 2023-10-21T17:18:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b4ffff640 1 -- 172.21.15.53:0/4026699271 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b44003520 con 0x7f8b5815de70 2023-10-21T17:18:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5dbd6640 1 -- 172.21.15.53:0/4026699271 >> v1:172.21.15.124:6789/0 conn(0x7f8b580cb970 legacy=0x7f8b5815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5dbd6640 1 -- 172.21.15.53:0/4026699271 >> v1:172.21.15.53:6789/0 conn(0x7f8b5815de70 legacy=0x7f8b580cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.140+0000 7f8b5dbd6640 1 -- 172.21.15.53:0/4026699271 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5815f1a0 con 0x7f8b580ca9a0 2023-10-21T17:18:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b5dbd6640 1 --2- 172.21.15.53:0/4026699271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580ca9a0 0x7f8b5815eba0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097a0 tx=0x7f8b5002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc85d61b7b79a5d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b4ffff640 1 -- 172.21.15.53:0/4026699271 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50030610 con 0x7f8b580ca9a0 2023-10-21T17:18:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b4ffff640 1 -- 172.21.15.53:0/4026699271 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b50030770 con 0x7f8b580ca9a0 2023-10-21T17:18:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b4ffff640 1 -- 172.21.15.53:0/4026699271 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50030ae0 con 0x7f8b580ca9a0 2023-10-21T17:18:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/4026699271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580ca9a0 msgr2=0x7f8b5815eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.142+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/4026699271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580ca9a0 0x7f8b5815eba0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097a0 tx=0x7f8b5002b960 comp rx=0 tx=0).stop 2023-10-21T17:18:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.143+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/4026699271 shutdown_connections 2023-10-21T17:18:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.143+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/4026699271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580ca9a0 0x7f8b5815eba0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.143+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/4026699271 >> 172.21.15.53:0/4026699271 conn(0x7f8b580ce9f0 msgr2=0x7f8b580cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.143+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/4026699271 shutdown_connections 2023-10-21T17:18:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.143+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/4026699271 wait complete. 2023-10-21T17:18:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.144+0000 7f8b5e3d7640 1 Processor -- start 2023-10-21T17:18:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.145+0000 7f8b5e3d7640 1 -- start start 2023-10-21T17:18:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.145+0000 7f8b5e3d7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b580ca9a0 0x7f8b58152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.145+0000 7f8b5e3d7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5e3d7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b5815de70 0x7f8b581f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5e3d7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580cb580 con 0x7f8b580cb970 2023-10-21T17:18:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5e3d7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580c5d60 con 0x7f8b580ca9a0 2023-10-21T17:18:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5e3d7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58159480 con 0x7f8b5815de70 2023-10-21T17:18:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5cbd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5cbd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42234/0 (socket says 172.21.15.53:42234) 2023-10-21T17:18:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5cbd4640 1 -- 172.21.15.53:0/51623641 learned_addr learned my addr 172.21.15.53:0/51623641 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5d3d5640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b580ca9a0 0x7f8b58152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.146+0000 7f8b5dbd6640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b5815de70 0x7f8b581f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 -- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b5815de70 msgr2=0x7f8b581f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b5815de70 0x7f8b581f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 -- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b580ca9a0 msgr2=0x7f8b58152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b580ca9a0 0x7f8b58152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 -- 172.21.15.53:0/51623641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50009400 con 0x7f8b580cb970 2023-10-21T17:18:44.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5cbd4640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800c400 tx=0x7f8b4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:44.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480052d0 con 0x7f8b580cb970 2023-10-21T17:18:44.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48005430 con 0x7f8b580cb970 2023-10-21T17:18:44.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b580c5fb0 con 0x7f8b580cb970 2023-10-21T17:18:44.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48003830 con 0x7f8b580cb970 2023-10-21T17:18:44.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.147+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b58159110 con 0x7f8b580cb970 2023-10-21T17:18:44.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.149+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b581592d0 con 0x7f8b580cb970 2023-10-21T17:18:44.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.151+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8b48003990 con 0x7f8b580cb970 2023-10-21T17:18:44.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.152+0000 7f8b4dffb640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 0x7f8b3c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:44.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.152+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f8b480891a0 con 0x7f8b580cb970 2023-10-21T17:18:44.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.153+0000 7f8b5d3d5640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 0x7f8b3c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:44.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.154+0000 7f8b5d3d5640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 0x7f8b3c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b54001560 tx=0x7f8b54009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:44.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.156+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b48054150 con 0x7f8b580cb970 2023-10-21T17:18:44.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.347+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8b58159630 con 0x7f8b580cb970 2023-10-21T17:18:44.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.890+0000 7f8b4dffb640 1 -- 172.21.15.53:0/51623641 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8b58159630 con 0x7f8b580cb970 2023-10-21T17:18:44.893 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:18:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 msgr2=0x7f8b3c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 0x7f8b3c06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b54001560 tx=0x7f8b54009040 comp rx=0 tx=0).stop 2023-10-21T17:18:44.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 msgr2=0x7f8b581f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800c400 tx=0x7f8b4800c8c0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 shutdown_connections 2023-10-21T17:18:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8b3c06af90 0x7f8b3c06d450 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b54001560 tx=0x7f8b54009040 comp rx=0 tx=0).stop 2023-10-21T17:18:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b5815de70 0x7f8b581f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b580cb970 0x7f8b581f3420 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 --2- 172.21.15.53:0/51623641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b580ca9a0 0x7f8b58152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.896+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 >> 172.21.15.53:0/51623641 conn(0x7f8b580ce9f0 msgr2=0x7f8b580ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.898+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 shutdown_connections 2023-10-21T17:18:44.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:44.898+0000 7f8b5e3d7640 1 -- 172.21.15.53:0/51623641 wait complete. 2023-10-21T17:18:44.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:18:45.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.125+0000 7f1d8b94e640 1 Processor -- start 2023-10-21T17:18:45.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.126+0000 7f1d8b94e640 1 -- start start 2023-10-21T17:18:45.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d8406c3a0 0x7f1d840c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d8406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 0x7f1d840b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d840ca230 con 0x7f1d8406c3a0 2023-10-21T17:18:45.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8415b310 con 0x7f1d8415aa00 2023-10-21T17:18:45.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d8b94e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8406d290 con 0x7f1d8406db60 2023-10-21T17:18:45.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.127+0000 7f1d896ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d8406c3a0 0x7f1d840c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1d8406c3a0 0x7f1d840c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42238/0 (socket says 172.21.15.53:42238) 2023-10-21T17:18:45.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 -- 172.21.15.53:0/4120937119 learned_addr learned my addr 172.21.15.53:0/4120937119 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:45.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 -- 172.21.15.53:0/4120937119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 msgr2=0x7f1d8406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:45.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 --2- 172.21.15.53:0/4120937119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d8406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 -- 172.21.15.53:0/4120937119 >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 msgr2=0x7f1d840b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:45.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d89eeb640 1 --2- 172.21.15.53:0/4120937119 >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 0x7f1d840b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 --2- 172.21.15.53:0/4120937119 >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 0x7f1d840b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d896ea640 1 -- 172.21.15.53:0/4120937119 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840c9860 con 0x7f1d8406c3a0 2023-10-21T17:18:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.128+0000 7f1d89eeb640 1 --2- 172.21.15.53:0/4120937119 >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 0x7f1d840b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.129+0000 7f1d896ea640 1 --2- 172.21.15.53:0/4120937119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c9cf0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1d84070240 tx=0x7f1d8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339f11aa52061309 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.130+0000 7f1d7bfff640 1 -- 172.21.15.53:0/4120937119 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80031870 con 0x7f1d8406c3a0 2023-10-21T17:18:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.130+0000 7f1d7bfff640 1 -- 172.21.15.53:0/4120937119 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800319d0 con 0x7f1d8406c3a0 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.130+0000 7f1d7bfff640 1 -- 172.21.15.53:0/4120937119 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80031d60 con 0x7f1d8406c3a0 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 -- 172.21.15.53:0/4120937119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8406c3a0 msgr2=0x7f1d840c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/4120937119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c9cf0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1d84070240 tx=0x7f1d8002b910 comp rx=0 tx=0).stop 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 -- 172.21.15.53:0/4120937119 shutdown_connections 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/4120937119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d8406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/4120937119 >> v2:172.21.15.124:3300/0 conn(0x7f1d8415aa00 0x7f1d840b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/4120937119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c9cf0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 -- 172.21.15.53:0/4120937119 >> 172.21.15.53:0/4120937119 conn(0x7f1d8406a5d0 msgr2=0x7f1d840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:45.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 -- 172.21.15.53:0/4120937119 shutdown_connections 2023-10-21T17:18:45.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.131+0000 7f1d8b94e640 1 -- 172.21.15.53:0/4120937119 wait complete. 2023-10-21T17:18:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.133+0000 7f1d8b94e640 1 Processor -- start 2023-10-21T17:18:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.133+0000 7f1d8b94e640 1 -- start start 2023-10-21T17:18:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d84152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840c4670 con 0x7f1d8415aa00 2023-10-21T17:18:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840c3a60 con 0x7f1d8406c3a0 2023-10-21T17:18:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d8b94e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840c3d40 con 0x7f1d8406db60 2023-10-21T17:18:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d89eeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d89eeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42250/0 (socket says 172.21.15.53:42250) 2023-10-21T17:18:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d89eeb640 1 -- 172.21.15.53:0/1079377698 learned_addr learned my addr 172.21.15.53:0/1079377698 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d896ea640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.135+0000 7f1d88ee9640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d84152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 -- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 msgr2=0x7f1d84152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d84152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 -- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d8406c3a0 msgr2=0x7f1d840c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 -- 172.21.15.53:0/1079377698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800094d0 con 0x7f1d8415aa00 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d88ee9640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d84152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d89eeb640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00cd50 tx=0x7f1d7c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00f680 con 0x7f1d8415aa00 2023-10-21T17:18:45.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00f800 con 0x7f1d8415aa00 2023-10-21T17:18:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00fbb0 con 0x7f1d8415aa00 2023-10-21T17:18:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.136+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d840c4020 con 0x7f1d8415aa00 2023-10-21T17:18:45.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.137+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d840c4b00 con 0x7f1d8415aa00 2023-10-21T17:18:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.138+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d840c4d20 con 0x7f1d8415aa00 2023-10-21T17:18:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.139+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d7c00b040 con 0x7f1d8415aa00 2023-10-21T17:18:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.140+0000 7f1d79ffb640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.140+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f1d7c08a430 con 0x7f1d8415aa00 2023-10-21T17:18:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.140+0000 7f1d896ea640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.142+0000 7f1d896ea640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 0x7f1d6806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d80036000 tx=0x7f1d8003c040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.145+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d7c0550d0 con 0x7f1d8415aa00 2023-10-21T17:18:45.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.379+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1d8404e160 con 0x7f1d8415aa00 2023-10-21T17:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.940+0000 7f1d79ffb640 1 -- 172.21.15.53:0/1079377698 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1d7c0586f0 con 0x7f1d8415aa00 2023-10-21T17:18:45.943 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 21 and data pool 22 2023-10-21T17:18:45.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.948+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 msgr2=0x7f1d6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:45.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.948+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 0x7f1d6806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d80036000 tx=0x7f1d8003c040 comp rx=0 tx=0).stop 2023-10-21T17:18:45.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.948+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 msgr2=0x7f1d84155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:45.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.948+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00cd50 tx=0x7f1d7c00a650 comp rx=0 tx=0).stop 2023-10-21T17:18:45.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.951+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 shutdown_connections 2023-10-21T17:18:45.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.951+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f1d6806af90 0x7f1d6806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.952+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d8415aa00 0x7f1d84155e40 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.952+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d8406db60 0x7f1d84152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.952+0000 7f1d8b94e640 1 --2- 172.21.15.53:0/1079377698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d8406c3a0 0x7f1d840c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:45.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.952+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 >> 172.21.15.53:0/1079377698 conn(0x7f1d8406a5d0 msgr2=0x7f1d840ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.955+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 shutdown_connections 2023-10-21T17:18:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:45.955+0000 7f1d8b94e640 1 -- 172.21.15.53:0/1079377698 wait complete. 2023-10-21T17:18:45.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:46.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.173+0000 7f317d232640 1 Processor -- start 2023-10-21T17:18:46.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.174+0000 7f317d232640 1 -- start start 2023-10-21T17:18:46.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.175+0000 7f317d232640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f31780adc70 0x7f31780b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.175+0000 7f317d232640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317815f7d0 con 0x7f317815ab30 2023-10-21T17:18:46.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.175+0000 7f317d232640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317815efb0 con 0x7f317815af10 2023-10-21T17:18:46.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.175+0000 7f317d232640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317815f210 con 0x7f31780adc70 2023-10-21T17:18:46.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.176+0000 7f3177fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f317815ab30 0x7f317815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49976/0 (socket says 172.21.15.53:49976) 2023-10-21T17:18:46.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.176+0000 7f3177fff640 1 -- 172.21.15.53:0/3897822420 learned_addr learned my addr 172.21.15.53:0/3897822420 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:46.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.176+0000 7f316ffff640 1 --2- 172.21.15.53:0/3897822420 >> v2:172.21.15.124:3300/0 conn(0x7f31780adc70 0x7f31780b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.176+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379875085 0 0) 0x7f317815f7d0 con 0x7f317815ab30 2023-10-21T17:18:46.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.176+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3164003520 con 0x7f317815ab30 2023-10-21T17:18:46.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439101160 0 0) 0x7f3164003520 con 0x7f317815ab30 2023-10-21T17:18:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 >> v2:172.21.15.124:3300/0 conn(0x7f31780adc70 msgr2=0x7f31780b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 --2- 172.21.15.53:0/3897822420 >> v2:172.21.15.124:3300/0 conn(0x7f31780adc70 0x7f31780b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 >> v1:172.21.15.124:6789/0 conn(0x7f317815af10 legacy=0x7f317806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317815f4f0 con 0x7f317815ab30 2023-10-21T17:18:46.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3168001690 con 0x7f317815ab30 2023-10-21T17:18:46.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 >> v1:172.21.15.53:6789/0 conn(0x7f317815ab30 legacy=0x7f317815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3164003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640047c0 0x7f3164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164005180 0x7f316400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317815f210 con 0x7f3164003970 2023-10-21T17:18:46.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316400c020 con 0x7f31640047c0 2023-10-21T17:18:46.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316400c1e0 con 0x7f3164005180 2023-10-21T17:18:46.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f3177fff640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3164003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f316ffff640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640047c0 0x7f3164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.177+0000 7f317ca31640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164005180 0x7f316400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f3177fff640 1 -- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164005180 msgr2=0x7f316400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:46.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f3177fff640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164005180 0x7f316400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f3177fff640 1 -- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640047c0 msgr2=0x7f3164004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:46.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f3177fff640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31640047c0 0x7f3164004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f3177fff640 1 -- 172.21.15.53:0/3897822420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164003520 con 0x7f3164003970 2023-10-21T17:18:46.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.178+0000 7f317ca31640 1 -- 172.21.15.53:0/3897822420 reap_dead start 2023-10-21T17:18:46.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.179+0000 7f3177fff640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3164003d90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f317815f960 tx=0x7f31640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffe1e5d28ffb688 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.179+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168005ad0 con 0x7f3164003970 2023-10-21T17:18:46.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.179+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3168005c30 con 0x7f3164003970 2023-10-21T17:18:46.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.179+0000 7f31777fe640 1 -- 172.21.15.53:0/3897822420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316802c890 con 0x7f3164003970 2023-10-21T17:18:46.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 -- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 msgr2=0x7f3164003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3164003d90 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f317815f960 tx=0x7f31640057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 -- 172.21.15.53:0/3897822420 shutdown_connections 2023-10-21T17:18:46.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 --2- 172.21.15.53:0/3897822420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3164003d90 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 -- 172.21.15.53:0/3897822420 >> 172.21.15.53:0/3897822420 conn(0x7f31780cb0c0 msgr2=0x7f31780cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:46.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 -- 172.21.15.53:0/3897822420 shutdown_connections 2023-10-21T17:18:46.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.180+0000 7f317d232640 1 -- 172.21.15.53:0/3897822420 wait complete. 2023-10-21T17:18:46.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.182+0000 7f317d232640 1 Processor -- start 2023-10-21T17:18:46.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.182+0000 7f317d232640 1 -- start start 2023-10-21T17:18:46.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31781594a0 0x7f317815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31780c4c50 0x7f317815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31781592a0 con 0x7f3164003970 2023-10-21T17:18:46.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780cc5c0 con 0x7f31781594a0 2023-10-21T17:18:46.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f317d232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317815c290 con 0x7f31780c4c50 2023-10-21T17:18:46.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f3177fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f3177fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42280/0 (socket says 172.21.15.53:42280) 2023-10-21T17:18:46.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.183+0000 7f3177fff640 1 -- 172.21.15.53:0/782710708 learned_addr learned my addr 172.21.15.53:0/782710708 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:46.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f317ca31640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31780c4c50 0x7f317815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f316ffff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31781594a0 0x7f317815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 -- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31780c4c50 msgr2=0x7f317815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31780c4c50 0x7f317815b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 -- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31781594a0 msgr2=0x7f317815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31781594a0 0x7f317815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 -- 172.21.15.53:0/782710708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317815c570 con 0x7f3164003970 2023-10-21T17:18:46.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.184+0000 7f3177fff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f316802c430 tx=0x7f316802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.185+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316803c040 con 0x7f3164003970 2023-10-21T17:18:46.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.185+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316802cca0 con 0x7f3164003970 2023-10-21T17:18:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.185+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317815c850 con 0x7f3164003970 2023-10-21T17:18:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.185+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31680072b0 con 0x7f3164003970 2023-10-21T17:18:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.185+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317815be20 con 0x7f3164003970 2023-10-21T17:18:46.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.186+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317804e160 con 0x7f3164003970 2023-10-21T17:18:46.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.187+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3168032070 con 0x7f3164003970 2023-10-21T17:18:46.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.188+0000 7f31757fa640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 0x7f315806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.188+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f31680aac00 con 0x7f3164003970 2023-10-21T17:18:46.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.189+0000 7f316ffff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 0x7f315806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.190+0000 7f316ffff640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 0x7f315806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31781f5e80 tx=0x7f3160008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.193+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31680b1080 con 0x7f3164003970 2023-10-21T17:18:46.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.386+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31780ada50 con 0x7f3164003970 2023-10-21T17:18:46.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.388+0000 7f31757fa640 1 -- 172.21.15.53:0/782710708 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16911 (secure 0 0 0) 0x7f3168075a10 con 0x7f3164003970 2023-10-21T17:18:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:46.391 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":183,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:45.382630+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:46.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 msgr2=0x7f315806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 0x7f315806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31781f5e80 tx=0x7f3160008040 comp rx=0 tx=0).stop 2023-10-21T17:18:46.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 msgr2=0x7f3178158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f316802c430 tx=0x7f316802e070 comp rx=0 tx=0).stop 2023-10-21T17:18:46.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 shutdown_connections 2023-10-21T17:18:46.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f315806af90 0x7f315806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31780c4c50 0x7f317815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31781594a0 0x7f317815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 --2- 172.21.15.53:0/782710708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3164003970 0x7f3178158d60 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 >> 172.21.15.53:0/782710708 conn(0x7f31780cb0c0 msgr2=0x7f31780b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:46.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 shutdown_connections 2023-10-21T17:18:46.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.391+0000 7f317d232640 1 -- 172.21.15.53:0/782710708 wait complete. 2023-10-21T17:18:46.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:46.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.634+0000 7fa578e32640 1 Processor -- start 2023-10-21T17:18:46.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.635+0000 7fa578e32640 1 -- start start 2023-10-21T17:18:46.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.636+0000 7fa578e32640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa574157480 0x7fa57406af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.636+0000 7fa578e32640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa57415dc60 0x7fa57406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.636+0000 7fa578e32640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740cde30 con 0x7fa5740ce2c0 2023-10-21T17:18:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.636+0000 7fa578e32640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57415e070 con 0x7fa57415dc60 2023-10-21T17:18:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.636+0000 7fa578e32640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa574157860 con 0x7fa574157480 2023-10-21T17:18:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa57415dc60 0x7fa57406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa57415dc60 0x7fa57406b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42294/0 (socket says 172.21.15.53:42294) 2023-10-21T17:18:46.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 -- 172.21.15.53:0/1631398998 learned_addr learned my addr 172.21.15.53:0/1631398998 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:46.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa572ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa5740ce2c0 0x7fa57415d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49992/0 (socket says 172.21.15.53:49992) 2023-10-21T17:18:46.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa5737fe640 1 --2- 172.21.15.53:0/1631398998 >> v2:172.21.15.124:3300/0 conn(0x7fa574157480 0x7fa57406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 -- 172.21.15.53:0/1631398998 >> v2:172.21.15.124:3300/0 conn(0x7fa574157480 msgr2=0x7fa57406af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:46.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 --2- 172.21.15.53:0/1631398998 >> v2:172.21.15.124:3300/0 conn(0x7fa574157480 0x7fa57406af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 -- 172.21.15.53:0/1631398998 >> v1:172.21.15.53:6789/0 conn(0x7fa5740ce2c0 legacy=0x7fa57415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa573fff640 1 -- 172.21.15.53:0/1631398998 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57415e2c0 con 0x7fa57415dc60 2023-10-21T17:18:46.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.637+0000 7fa5727fc640 1 -- 172.21.15.53:0/1631398998 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604289713 0 0) 0x7fa5740cde30 con 0x7fa5740ce2c0 2023-10-21T17:18:46.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.638+0000 7fa573fff640 1 --2- 172.21.15.53:0/1631398998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa57406b4b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa574066ff0 tx=0x7fa56c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421936f4026a08ee server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa5727fc640 1 -- 172.21.15.53:0/1631398998 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c031870 con 0x7fa57415dc60 2023-10-21T17:18:46.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa5727fc640 1 -- 172.21.15.53:0/1631398998 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c0319d0 con 0x7fa57415dc60 2023-10-21T17:18:46.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa5727fc640 1 -- 172.21.15.53:0/1631398998 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c031d60 con 0x7fa57415dc60 2023-10-21T17:18:46.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa578e32640 1 -- 172.21.15.53:0/1631398998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 msgr2=0x7fa57406b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa578e32640 1 --2- 172.21.15.53:0/1631398998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa57406b4b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fa574066ff0 tx=0x7fa56c02b910 comp rx=0 tx=0).stop 2023-10-21T17:18:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa578e32640 1 -- 172.21.15.53:0/1631398998 shutdown_connections 2023-10-21T17:18:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa578e32640 1 --2- 172.21.15.53:0/1631398998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa57406b4b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.639+0000 7fa578e32640 1 --2- 172.21.15.53:0/1631398998 >> v2:172.21.15.124:3300/0 conn(0x7fa574157480 0x7fa57406af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.640+0000 7fa578e32640 1 -- 172.21.15.53:0/1631398998 >> 172.21.15.53:0/1631398998 conn(0x7fa5740d2ae0 msgr2=0x7fa5740d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:46.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.640+0000 7fa578e32640 1 -- 172.21.15.53:0/1631398998 shutdown_connections 2023-10-21T17:18:46.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.640+0000 7fa578e32640 1 -- 172.21.15.53:0/1631398998 wait complete. 2023-10-21T17:18:46.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.641+0000 7fa578e32640 1 Processor -- start 2023-10-21T17:18:46.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.642+0000 7fa578e32640 1 -- start start 2023-10-21T17:18:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5740ce2c0 0x7fa574159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 0x7fa57415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740cce10 con 0x7fa57415dc60 2023-10-21T17:18:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415b340 con 0x7fa574157480 2023-10-21T17:18:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa578e32640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740ccb30 con 0x7fa5740ce2c0 2023-10-21T17:18:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42310/0 (socket says 172.21.15.53:42310) 2023-10-21T17:18:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 -- 172.21.15.53:0/2101983703 learned_addr learned my addr 172.21.15.53:0/2101983703 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa5737fe640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5740ce2c0 0x7fa574159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa572ffd640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 0x7fa57415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 -- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5740ce2c0 msgr2=0x7fa574159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5740ce2c0 0x7fa574159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.643+0000 7fa573fff640 1 -- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 msgr2=0x7fa57415c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa573fff640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 0x7fa57415c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa573fff640 1 -- 172.21.15.53:0/2101983703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c0094d0 con 0x7fa57415dc60 2023-10-21T17:18:46.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa572ffd640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 0x7fa57415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa573fff640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0090b0 tx=0x7fa56c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c03ac60 con 0x7fa57415dc60 2023-10-21T17:18:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c03adc0 con 0x7fa57415dc60 2023-10-21T17:18:46.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c044630 con 0x7fa57415dc60 2023-10-21T17:18:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57415b5e0 con 0x7fa57415dc60 2023-10-21T17:18:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.644+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57415bba0 con 0x7fa57415dc60 2023-10-21T17:18:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.646+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57415bd90 con 0x7fa57415dc60 2023-10-21T17:18:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.647+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa56c04d450 con 0x7fa57415dc60 2023-10-21T17:18:46.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.647+0000 7fa56bfff640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 0x7fa55406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.648+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa56c02d040 con 0x7fa57415dc60 2023-10-21T17:18:46.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.648+0000 7fa5737fe640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 0x7fa55406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.649+0000 7fa5737fe640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 0x7fa55406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa564009f30 tx=0x7fa56400b040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:46.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.652+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa56c076d50 con 0x7fa57415dc60 2023-10-21T17:18:46.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.843+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa57404e160 con 0x7fa57415dc60 2023-10-21T17:18:46.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.847+0000 7fa56bfff640 1 -- 172.21.15.53:0/2101983703 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16911 (secure 0 0 0) 0x7fa56c035070 con 0x7fa57415dc60 2023-10-21T17:18:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:46.849 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":183,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:45.382630+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:46.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 msgr2=0x7fa55406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 0x7fa55406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa564009f30 tx=0x7fa56400b040 comp rx=0 tx=0).stop 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 msgr2=0x7fa574153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0090b0 tx=0x7fa56c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 shutdown_connections 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa55406af90 0x7fa55406d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa57415dc60 0x7fa574153700 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa574157480 0x7fa57415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 --2- 172.21.15.53:0/2101983703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5740ce2c0 0x7fa574159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:46.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 >> 172.21.15.53:0/2101983703 conn(0x7fa5740d2ae0 msgr2=0x7fa5741517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:46.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 shutdown_connections 2023-10-21T17:18:46.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:46.849+0000 7fa578e32640 1 -- 172.21.15.53:0/2101983703 wait complete. 2023-10-21T17:18:46.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:18:47.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.096+0000 7f2b0254e640 1 Processor -- start 2023-10-21T17:18:47.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.096+0000 7f2b0254e640 1 -- start start 2023-10-21T17:18:47.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.097+0000 7f2b0254e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2afc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.097+0000 7f2b0254e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc15d160 con 0x7f2afc0cdda0 2023-10-21T17:18:47.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.097+0000 7f2b0254e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc0b7b90 con 0x7f2afc15cd80 2023-10-21T17:18:47.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.097+0000 7f2b0254e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ce230 con 0x7f2afc0b87d0 2023-10-21T17:18:47.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2b00aeb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2afc0cdda0 0x7f2afc0d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50002/0 (socket says 172.21.15.53:50002) 2023-10-21T17:18:47.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2b00aeb640 1 -- 172.21.15.53:0/866847485 learned_addr learned my addr 172.21.15.53:0/866847485 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:47.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2af37fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2afc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041950431 0 0) 0x7f2afc15d160 con 0x7f2afc0cdda0 2023-10-21T17:18:47.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae8003520 con 0x7f2afc0cdda0 2023-10-21T17:18:47.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619877462 0 0) 0x7f2ae8003520 con 0x7f2afc0cdda0 2023-10-21T17:18:47.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 msgr2=0x7f2afc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:47.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2afc0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.098+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 >> v1:172.21.15.124:6789/0 conn(0x7f2afc15cd80 legacy=0x7f2afc0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0b8020 con 0x7f2afc0cdda0 2023-10-21T17:18:47.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2af40019f0 con 0x7f2afc0cdda0 2023-10-21T17:18:47.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 >> v1:172.21.15.53:6789/0 conn(0x7f2afc0cdda0 legacy=0x7f2afc0d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2ae80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8003f70 0x7f2ae80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8004d00 0x7f2ae8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ce230 con 0x7f2ae8004d00 2023-10-21T17:18:47.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae800bac0 con 0x7f2ae8003f70 2023-10-21T17:18:47.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae800bc80 con 0x7f2afc0b87d0 2023-10-21T17:18:47.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8004d00 0x7f2ae8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2afbfff640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2ae80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2af37fe640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8003f70 0x7f2ae80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2b00aeb640 1 -- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 msgr2=0x7f2ae80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:47.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2afc0b87d0 0x7f2ae80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.099+0000 7f2b00aeb640 1 -- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8003f70 msgr2=0x7f2ae80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:47.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.100+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae8003f70 0x7f2ae80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.100+0000 7f2b00aeb640 1 -- 172.21.15.53:0/866847485 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8003520 con 0x7f2ae8004d00 2023-10-21T17:18:47.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8004d00 0x7f2ae8005180 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2af400a750 tx=0x7f2ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e45b2776a7b2aef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:47.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4034630 con 0x7f2ae8004d00 2023-10-21T17:18:47.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af402f070 con 0x7f2ae8004d00 2023-10-21T17:18:47.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2afb7fe640 1 -- 172.21.15.53:0/866847485 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40349f0 con 0x7f2ae8004d00 2023-10-21T17:18:47.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2b0254e640 1 -- 172.21.15.53:0/866847485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8004d00 msgr2=0x7f2ae8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2b0254e640 1 --2- 172.21.15.53:0/866847485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8004d00 0x7f2ae8005180 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2af400a750 tx=0x7f2ae80057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.101+0000 7f2b00aeb640 1 -- 172.21.15.53:0/866847485 reap_dead start 2023-10-21T17:18:47.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.102+0000 7f2b0254e640 1 -- 172.21.15.53:0/866847485 shutdown_connections 2023-10-21T17:18:47.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.102+0000 7f2b0254e640 1 -- 172.21.15.53:0/866847485 >> 172.21.15.53:0/866847485 conn(0x7f2afc06b5d0 msgr2=0x7f2afc06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:47.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.102+0000 7f2b0254e640 1 -- 172.21.15.53:0/866847485 shutdown_connections 2023-10-21T17:18:47.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.102+0000 7f2b0254e640 1 -- 172.21.15.53:0/866847485 wait complete. 2023-10-21T17:18:47.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.103+0000 7f2b0254e640 1 Processor -- start 2023-10-21T17:18:47.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.104+0000 7f2b0254e640 1 -- start start 2023-10-21T17:18:47.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.104+0000 7f2b0254e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae8004d00 0x7f2afc1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.104+0000 7f2b0254e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 0x7f2afc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b0254e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b0254e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1563b0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b0254e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15dc10 con 0x7f2afc1559f0 2023-10-21T17:18:47.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b0254e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15def0 con 0x7f2ae8004d00 2023-10-21T17:18:47.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42328/0 (socket says 172.21.15.53:42328) 2023-10-21T17:18:47.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 -- 172.21.15.53:0/4145853054 learned_addr learned my addr 172.21.15.53:0/4145853054 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:47.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2af37fe640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 0x7f2afc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2afbfff640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae8004d00 0x7f2afc1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 -- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae8004d00 msgr2=0x7f2afc1f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae8004d00 0x7f2afc1f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 -- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 msgr2=0x7f2afc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 0x7f2afc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2b00aeb640 1 -- 172.21.15.53:0/4145853054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc15e1d0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.105+0000 7f2af37fe640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 0x7f2afc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:47.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2b00aeb640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2af402caa0 tx=0x7f2afc0b7b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4034840 con 0x7f2afc15f4c0 2023-10-21T17:18:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af40349a0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4034d10 con 0x7f2afc15f4c0 2023-10-21T17:18:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc15e4b0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.106+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc15fe80 con 0x7f2afc15f4c0 2023-10-21T17:18:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.107+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04e160 con 0x7f2afc15f4c0 2023-10-21T17:18:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.108+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2af4031030 con 0x7f2afc15f4c0 2023-10-21T17:18:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.109+0000 7f2af97fa640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 0x7f2adc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:47.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.109+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2af40abe80 con 0x7f2afc15f4c0 2023-10-21T17:18:47.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.110+0000 7f2afbfff640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 0x7f2adc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:47.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.111+0000 7f2afbfff640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 0x7f2adc06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007fd0 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:47.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.113+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2af4076bd0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.347+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2afc04e160 con 0x7f2afc15f4c0 2023-10-21T17:18:47.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.952+0000 7f2af97fa640 1 -- 172.21.15.53:0/4145853054 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2af407a1f0 con 0x7f2afc15f4c0 2023-10-21T17:18:47.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 msgr2=0x7f2adc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 0x7f2adc06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007fd0 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2023-10-21T17:18:47.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 msgr2=0x7f2afc15f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:47.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2af402caa0 tx=0x7f2afc0b7b00 comp rx=0 tx=0).stop 2023-10-21T17:18:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 shutdown_connections 2023-10-21T17:18:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f2adc06afc0 0x7f2adc06d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2afc15f4c0 0x7f2afc15f940 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2afc1559f0 0x7f2afc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 --2- 172.21.15.53:0/4145853054 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae8004d00 0x7f2afc1f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:47.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 >> 172.21.15.53:0/4145853054 conn(0x7f2afc06b5d0 msgr2=0x7f2afc0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:47.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.956+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 shutdown_connections 2023-10-21T17:18:47.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:47.957+0000 7f2b0254e640 1 -- 172.21.15.53:0/4145853054 wait complete. 2023-10-21T17:18:47.980 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:18:47.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:18:48.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.213+0000 7fa78af9e640 1 Processor -- start 2023-10-21T17:18:48.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.213+0000 7fa78af9e640 1 -- start start 2023-10-21T17:18:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78af9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa7840d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78af9e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa7840d0b90 0x7fa7840cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78af9e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7840d2870 con 0x7fa7840cff10 2023-10-21T17:18:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78af9e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7840d0380 con 0x7fa7840d0b90 2023-10-21T17:18:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78af9e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840d1150 con 0x7fa78415ccb0 2023-10-21T17:18:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78a79d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa7840cff10 0x7fa7840d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50012/0 (socket says 172.21.15.53:50012) 2023-10-21T17:18:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78a79d640 1 -- 172.21.15.53:0/1694039635 learned_addr learned my addr 172.21.15.53:0/1694039635 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa78979b640 1 --2- 172.21.15.53:0/1694039635 >> v2:172.21.15.124:3300/0 conn(0x7fa7840d0b90 0x7fa7840cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.215+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa7840d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119309505 0 0) 0x7fa7840d2870 con 0x7fa7840cff10 2023-10-21T17:18:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa770003520 con 0x7fa7840cff10 2023-10-21T17:18:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900158813 0 0) 0x7fa770003520 con 0x7fa7840cff10 2023-10-21T17:18:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 msgr2=0x7fa7840d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa7840d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 >> v2:172.21.15.124:3300/0 conn(0x7fa7840d0b90 msgr2=0x7fa7840cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 --2- 172.21.15.53:0/1694039635 >> v2:172.21.15.124:3300/0 conn(0x7fa7840d0b90 0x7fa7840cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840d1430 con 0x7fa7840cff10 2023-10-21T17:18:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa77c001690 con 0x7fa7840cff10 2023-10-21T17:18:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 >> v1:172.21.15.53:6789/0 conn(0x7fa7840cff10 legacy=0x7fa7840d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa770003970 0x7fa770003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840d0380 con 0x7fa7700047c0 2023-10-21T17:18:48.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840d1150 con 0x7fa770003970 2023-10-21T17:18:48.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.216+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77000b980 con 0x7fa78415ccb0 2023-10-21T17:18:48.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa770003970 0x7fa770003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78a79d640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 -- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 msgr2=0x7fa770005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:48.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa770005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 -- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa770003970 msgr2=0x7fa770003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:48.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa770003970 0x7fa770003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78979b640 1 -- 172.21.15.53:0/1694039635 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770003520 con 0x7fa7700047c0 2023-10-21T17:18:48.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa770003970 0x7fa770003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:48.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.217+0000 7fa78a79d640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa78415ccb0 0x7fa770005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:48.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.218+0000 7fa78979b640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa770004c40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa774009ac0 tx=0x7fa7700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774cd61dcdc9014a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:48.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77402c6d0 con 0x7fa7700047c0 2023-10-21T17:18:48.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77402c830 con 0x7fa7700047c0 2023-10-21T17:18:48.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa788f9a640 1 -- 172.21.15.53:0/1694039635 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77402cbc0 con 0x7fa7700047c0 2023-10-21T17:18:48.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 -- 172.21.15.53:0/1694039635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 msgr2=0x7fa770004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1694039635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa770004c40 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa774009ac0 tx=0x7fa7700057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78a79d640 1 -- 172.21.15.53:0/1694039635 reap_dead start 2023-10-21T17:18:48.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 -- 172.21.15.53:0/1694039635 shutdown_connections 2023-10-21T17:18:48.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 -- 172.21.15.53:0/1694039635 >> 172.21.15.53:0/1694039635 conn(0x7fa784071660 msgr2=0x7fa784071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:48.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 -- 172.21.15.53:0/1694039635 shutdown_connections 2023-10-21T17:18:48.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.219+0000 7fa78af9e640 1 -- 172.21.15.53:0/1694039635 wait complete. 2023-10-21T17:18:48.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.221+0000 7fa78af9e640 1 Processor -- start 2023-10-21T17:18:48.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.221+0000 7fa78af9e640 1 -- start start 2023-10-21T17:18:48.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7841f5af0 0x7fa7841ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7840c8650 0x7fa7840b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78415ed00 con 0x7fa7700047c0 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841f3450 con 0x7fa7841f5af0 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.222+0000 7fa78af9e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841f6020 con 0x7fa7840c8650 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42344/0 (socket says 172.21.15.53:42344) 2023-10-21T17:18:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 -- 172.21.15.53:0/1463938655 learned_addr learned my addr 172.21.15.53:0/1463938655 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 -- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7840c8650 msgr2=0x7fa7840b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7840c8650 0x7fa7840b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 -- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7841f5af0 msgr2=0x7fa7841ef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa78979b640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7841f5af0 0x7fa7841ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7841f5af0 0x7fa7841ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 -- 172.21.15.53:0/1463938655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7841f6300 con 0x7fa7700047c0 2023-10-21T17:18:48.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.223+0000 7fa789f9c640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa78000f350 tx=0x7fa78000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:48.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.224+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780003b90 con 0x7fa7700047c0 2023-10-21T17:18:48.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.224+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780011040 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.224+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780004020 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.224+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7841f65e0 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.224+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78415d120 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.225+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78415d2b0 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.226+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa780004180 con 0x7fa7700047c0 2023-10-21T17:18:48.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.227+0000 7fa77a7fc640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 0x7fa76806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:48.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.228+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa78008a210 con 0x7fa7700047c0 2023-10-21T17:18:48.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.228+0000 7fa78979b640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 0x7fa76806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:48.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.229+0000 7fa78979b640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 0x7fa76806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa774036000 tx=0x7fa774030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:48.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.233+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa780054e40 con 0x7fa7700047c0 2023-10-21T17:18:48.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.465+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa7840d19c0 con 0x7fa7700047c0 2023-10-21T17:18:48.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.964+0000 7fa77a7fc640 1 -- 172.21.15.53:0/1463938655 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa780058460 con 0x7fa7700047c0 2023-10-21T17:18:48.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.968+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 msgr2=0x7fa76806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.968+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 0x7fa76806d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa774036000 tx=0x7fa774030000 comp rx=0 tx=0).stop 2023-10-21T17:18:48.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.968+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 msgr2=0x7fa7841f2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:48.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.968+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa78000f350 tx=0x7fa78000f810 comp rx=0 tx=0).stop 2023-10-21T17:18:48.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 shutdown_connections 2023-10-21T17:18:48.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fa76806af90 0x7fa76806d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7840c8650 0x7fa7840b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7841f5af0 0x7fa7841ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 --2- 172.21.15.53:0/1463938655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7700047c0 0x7fa7841f2f10 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 >> 172.21.15.53:0/1463938655 conn(0x7fa784071660 msgr2=0x7fa7840cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 shutdown_connections 2023-10-21T17:18:48.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:48.969+0000 7fa78af9e640 1 -- 172.21.15.53:0/1463938655 wait complete. 2023-10-21T17:18:48.995 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:18:48.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:49.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.227+0000 7fb91cfdc640 1 Processor -- start 2023-10-21T17:18:49.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.227+0000 7fb91cfdc640 1 -- start start 2023-10-21T17:18:49.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb91cfdc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9180709e0 0x7fb918164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb91cfdc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91806dee0 con 0x7fb9180ad790 2023-10-21T17:18:49.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb91cfdc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91815d850 con 0x7fb918159ac0 2023-10-21T17:18:49.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb91cfdc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91806dcf0 con 0x7fb9180709e0 2023-10-21T17:18:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb916d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9180709e0 0x7fb918164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb916d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb9180709e0 0x7fb918164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42350/0 (socket says 172.21.15.53:42350) 2023-10-21T17:18:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb915d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb9180ad790 0x7fb9180702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50020/0 (socket says 172.21.15.53:50020) 2023-10-21T17:18:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb916d9d640 1 -- 172.21.15.53:0/3721815532 learned_addr learned my addr 172.21.15.53:0/3721815532 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb916d9d640 1 -- 172.21.15.53:0/3721815532 >> v1:172.21.15.124:6789/0 conn(0x7fb918159ac0 legacy=0x7fb91806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.229+0000 7fb916d9d640 1 -- 172.21.15.53:0/3721815532 >> v1:172.21.15.53:6789/0 conn(0x7fb9180ad790 legacy=0x7fb9180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.230+0000 7fb916d9d640 1 -- 172.21.15.53:0/3721815532 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91815db30 con 0x7fb9180709e0 2023-10-21T17:18:49.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb916d9d640 1 --2- 172.21.15.53:0/3721815532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180709e0 0x7fb918164a60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb910009870 tx=0x7fb91002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479aa040d2b4a430 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb91559a640 1 -- 172.21.15.53:0/3721815532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910030860 con 0x7fb9180709e0 2023-10-21T17:18:49.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb91559a640 1 -- 172.21.15.53:0/3721815532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9100309c0 con 0x7fb9180709e0 2023-10-21T17:18:49.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb91559a640 1 -- 172.21.15.53:0/3721815532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910030d50 con 0x7fb9180709e0 2023-10-21T17:18:49.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb91cfdc640 1 -- 172.21.15.53:0/3721815532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180709e0 msgr2=0x7fb918164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.231+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/3721815532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180709e0 0x7fb918164a60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb910009870 tx=0x7fb91002b9a0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.232+0000 7fb91cfdc640 1 -- 172.21.15.53:0/3721815532 shutdown_connections 2023-10-21T17:18:49.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.232+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/3721815532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180709e0 0x7fb918164a60 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.232+0000 7fb91cfdc640 1 -- 172.21.15.53:0/3721815532 >> 172.21.15.53:0/3721815532 conn(0x7fb91806a380 msgr2=0x7fb9180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:49.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.232+0000 7fb91cfdc640 1 -- 172.21.15.53:0/3721815532 shutdown_connections 2023-10-21T17:18:49.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.232+0000 7fb91cfdc640 1 -- 172.21.15.53:0/3721815532 wait complete. 2023-10-21T17:18:49.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.233+0000 7fb91cfdc640 1 Processor -- start 2023-10-21T17:18:49.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.234+0000 7fb91cfdc640 1 -- start start 2023-10-21T17:18:49.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.234+0000 7fb91cfdc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 0x7fb91815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.234+0000 7fb91cfdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91cfdc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb918159ac0 0x7fb9180cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91cfdc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918147b40 con 0x7fb9180ad790 2023-10-21T17:18:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91cfdc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180cac80 con 0x7fb9180709e0 2023-10-21T17:18:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91cfdc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815a020 con 0x7fb918159ac0 2023-10-21T17:18:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42364/0 (socket says 172.21.15.53:42364) 2023-10-21T17:18:49.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 -- 172.21.15.53:0/4022793439 learned_addr learned my addr 172.21.15.53:0/4022793439 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 -- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb918159ac0 msgr2=0x7fb9180cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb916d9d640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb918159ac0 0x7fb9180cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91659c640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 0x7fb91815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb918159ac0 0x7fb9180cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 -- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 msgr2=0x7fb91815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 0x7fb91815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb915d9b640 1 -- 172.21.15.53:0/4022793439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9100094d0 con 0x7fb9180ad790 2023-10-21T17:18:49.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.235+0000 7fb91659c640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 0x7fb91815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:49.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb915d9b640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb90000c4d0 tx=0x7fb90000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900007480 con 0x7fb9180ad790 2023-10-21T17:18:49.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9000075e0 con 0x7fb9180ad790 2023-10-21T17:18:49.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9000049e0 con 0x7fb9180ad790 2023-10-21T17:18:49.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91815a300 con 0x7fb9180ad790 2023-10-21T17:18:49.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.236+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91815a8c0 con 0x7fb9180ad790 2023-10-21T17:18:49.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.237+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb918067490 con 0x7fb9180ad790 2023-10-21T17:18:49.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.238+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb900004b40 con 0x7fb9180ad790 2023-10-21T17:18:49.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.241+0000 7fb90effd640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 0x7fb8f806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.241+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb900088ea0 con 0x7fb9180ad790 2023-10-21T17:18:49.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.241+0000 7fb91659c640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 0x7fb8f806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.243+0000 7fb91659c640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 0x7fb8f806d4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb908007eb0 tx=0x7fb9080073c0 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.244+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb900053c80 con 0x7fb9180ad790 2023-10-21T17:18:49.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.479+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb91815aab0 con 0x7fb9180ad790 2023-10-21T17:18:49.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.480+0000 7fb90effd640 1 -- 172.21.15.53:0/4022793439 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fb900018090 con 0x7fb9180ad790 2023-10-21T17:18:49.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:49.483 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:18:49.483 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:18:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 msgr2=0x7fb8f806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 0x7fb8f806d4a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb908007eb0 tx=0x7fb9080073c0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 msgr2=0x7fb91815ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb90000c4d0 tx=0x7fb90000c990 comp rx=0 tx=0).stop 2023-10-21T17:18:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 shutdown_connections 2023-10-21T17:18:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fb8f806afe0 0x7fb8f806d4a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb918159ac0 0x7fb9180cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9180ad790 0x7fb91815ade0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 --2- 172.21.15.53:0/4022793439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9180709e0 0x7fb91815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 >> 172.21.15.53:0/4022793439 conn(0x7fb91806a380 msgr2=0x7fb918158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 shutdown_connections 2023-10-21T17:18:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.484+0000 7fb91cfdc640 1 -- 172.21.15.53:0/4022793439 wait complete. 2023-10-21T17:18:49.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:18:49.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.727+0000 7fe7eb59e640 1 Processor -- start 2023-10-21T17:18:49.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.727+0000 7fe7eb59e640 1 -- start start 2023-10-21T17:18:49.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7eb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7ec0cab40 0x7fe7ec1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7eb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0c9020 con 0x7fe7ec15cca0 2023-10-21T17:18:49.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7eb59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0ccc20 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7eb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0c92e0 con 0x7fe7ec0cab40 2023-10-21T17:18:49.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ead9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe7ec0cab40 0x7fe7ec1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ea59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe7ec15cca0 0x7fe7ec0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50032/0 (socket says 172.21.15.53:50032) 2023-10-21T17:18:49.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ea59c640 1 -- 172.21.15.53:0/3749250023 learned_addr learned my addr 172.21.15.53:0/3749250023 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:49.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ead9d640 1 -- 172.21.15.53:0/3749250023 >> v1:172.21.15.124:6789/0 conn(0x7fe7ec0b07d0 legacy=0x7fe7ec0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:49.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ead9d640 1 -- 172.21.15.53:0/3749250023 >> v1:172.21.15.53:6789/0 conn(0x7fe7ec15cca0 legacy=0x7fe7ec0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.729+0000 7fe7ead9d640 1 -- 172.21.15.53:0/3749250023 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0cc7b0 con 0x7fe7ec0cab40 2023-10-21T17:18:49.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.730+0000 7fe7ead9d640 1 --2- 172.21.15.53:0/3749250023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0cab40 0x7fe7ec1614b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009d90 tx=0x7fe7e402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd238b909cb4fe3d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.730+0000 7fe7e9d9b640 1 -- 172.21.15.53:0/3749250023 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e402cb40 con 0x7fe7ec0cab40 2023-10-21T17:18:49.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.730+0000 7fe7e9d9b640 1 -- 172.21.15.53:0/3749250023 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e402cca0 con 0x7fe7ec0cab40 2023-10-21T17:18:49.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.730+0000 7fe7e9d9b640 1 -- 172.21.15.53:0/3749250023 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4034660 con 0x7fe7ec0cab40 2023-10-21T17:18:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 -- 172.21.15.53:0/3749250023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0cab40 msgr2=0x7fe7ec1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/3749250023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0cab40 0x7fe7ec1614b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009d90 tx=0x7fe7e402bd90 comp rx=0 tx=0).stop 2023-10-21T17:18:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 -- 172.21.15.53:0/3749250023 shutdown_connections 2023-10-21T17:18:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/3749250023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0cab40 0x7fe7ec1614b0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 -- 172.21.15.53:0/3749250023 >> 172.21.15.53:0/3749250023 conn(0x7fe7ec0cb0c0 msgr2=0x7fe7ec0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:49.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 -- 172.21.15.53:0/3749250023 shutdown_connections 2023-10-21T17:18:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.731+0000 7fe7eb59e640 1 -- 172.21.15.53:0/3749250023 wait complete. 2023-10-21T17:18:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.733+0000 7fe7eb59e640 1 Processor -- start 2023-10-21T17:18:49.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.733+0000 7fe7eb59e640 1 -- start start 2023-10-21T17:18:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 0x7fe7ec1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 0x7fe7ec159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b1300 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0c7d50 con 0x7fe7ec15cca0 2023-10-21T17:18:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7eb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0c8030 con 0x7fe7ec0cab40 2023-10-21T17:18:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7ea59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7ea59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42380/0 (socket says 172.21.15.53:42380) 2023-10-21T17:18:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7ea59c640 1 -- 172.21.15.53:0/4133086729 learned_addr learned my addr 172.21.15.53:0/4133086729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7ead9d640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 0x7fe7ec159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7e3fff640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 0x7fe7ec1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.734+0000 7fe7ea59c640 1 -- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 msgr2=0x7fe7ec1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ea59c640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 0x7fe7ec1595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ea59c640 1 -- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 msgr2=0x7fe7ec159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ea59c640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 0x7fe7ec159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ea59c640 1 -- 172.21.15.53:0/4133086729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009a70 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ead9d640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 0x7fe7ec159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:49.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7e3fff640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 0x7fe7ec1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:49.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7ea59c640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c4a0 tx=0x7fe7dc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc005ce0 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc005e40 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc007270 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec0c8310 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.735+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec0c7020 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.738+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec0b1060 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.739+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe7dc007490 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.740+0000 7fe7e2ffd640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 0x7fe7cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:49.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.740+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe7dc088d80 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.741+0000 7fe7e3fff640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 0x7fe7cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:49.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.742+0000 7fe7e3fff640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 0x7fe7cc06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec14e4e0 tx=0x7fe7d4008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:49.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.744+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7ec0b1060 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.941+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7ec0b1060 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.945+0000 7fe7e2ffd640 1 -- 172.21.15.53:0/4133086729 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16898 (secure 0 0 0) 0x7fe7ec0b1060 con 0x7fe7ec0b07d0 2023-10-21T17:18:49.949 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:49.949 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":184,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:18:46.940944+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:18:49.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 msgr2=0x7fe7cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 0x7fe7cc06d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec14e4e0 tx=0x7fe7d4008040 comp rx=0 tx=0).stop 2023-10-21T17:18:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 msgr2=0x7fe7ec147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c4a0 tx=0x7fe7dc00c960 comp rx=0 tx=0).stop 2023-10-21T17:18:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 shutdown_connections 2023-10-21T17:18:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe7cc06af90 0x7fe7cc06d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7ec15cca0 0x7fe7ec159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7ec0cab40 0x7fe7ec1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 --2- 172.21.15.53:0/4133086729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec0b07d0 0x7fe7ec147500 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:49.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.947+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 >> 172.21.15.53:0/4133086729 conn(0x7fe7ec0cb0c0 msgr2=0x7fe7ec06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:49.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.948+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 shutdown_connections 2023-10-21T17:18:49.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:49.948+0000 7fe7eb59e640 1 -- 172.21.15.53:0/4133086729 wait complete. 2023-10-21T17:18:49.969 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:18:49.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:18:50.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.191+0000 7fe22e50c640 1 Processor -- start 2023-10-21T17:18:50.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.191+0000 7fe22e50c640 1 -- start start 2023-10-21T17:18:50.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe22e50c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe228165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe22e50c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22815b2b0 con 0x7fe2280c90b0 2023-10-21T17:18:50.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe22e50c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22815b590 con 0x7fe22815ab40 2023-10-21T17:18:50.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe22e50c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22815bf30 con 0x7fe2280cb580 2023-10-21T17:18:50.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe2277fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe2280c90b0 0x7fe2280cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50036/0 (socket says 172.21.15.53:50036) 2023-10-21T17:18:50.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.193+0000 7fe2277fe640 1 -- 172.21.15.53:0/3074574795 learned_addr learned my addr 172.21.15.53:0/3074574795 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:50.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe22caa9640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe228165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121081442 0 0) 0x7fe22815b2b0 con 0x7fe2280c90b0 2023-10-21T17:18:50.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe210003520 con 0x7fe2280c90b0 2023-10-21T17:18:50.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755016550 0 0) 0x7fe210003520 con 0x7fe2280c90b0 2023-10-21T17:18:50.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 msgr2=0x7fe228165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe228165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 >> v1:172.21.15.124:6789/0 conn(0x7fe22815ab40 legacy=0x7fe2280ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22815b870 con 0x7fe2280c90b0 2023-10-21T17:18:50.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe21c001690 con 0x7fe2280c90b0 2023-10-21T17:18:50.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 >> v1:172.21.15.53:6789/0 conn(0x7fe2280c90b0 legacy=0x7fe2280cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe2100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210003f70 0x7fe2100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe210004d00 0x7fe210005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22815bf30 con 0x7fe210004d00 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21000bac0 con 0x7fe210003f70 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.194+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21000bc80 con 0x7fe2280cb580 2023-10-21T17:18:50.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe210004d00 0x7fe210005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe2277fe640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210003f70 0x7fe2100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe227fff640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe2100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 -- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 msgr2=0x7fe2100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe2100039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 -- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210003f70 msgr2=0x7fe2100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210003f70 0x7fe2100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe22caa9640 1 -- 172.21.15.53:0/3074574795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210003520 con 0x7fe210004d00 2023-10-21T17:18:50.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.195+0000 7fe227fff640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2280cb580 0x7fe2100039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:50.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.196+0000 7fe22caa9640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe210004d00 0x7fe210005180 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe220009fc0 tx=0x7fe2100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93f79938792edb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22002cbf0 con 0x7fe210004d00 2023-10-21T17:18:50.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000f040 con 0x7fe210004d00 2023-10-21T17:18:50.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe226ffd640 1 -- 172.21.15.53:0/3074574795 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220034440 con 0x7fe210004d00 2023-10-21T17:18:50.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22e50c640 1 -- 172.21.15.53:0/3074574795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe210004d00 msgr2=0x7fe210005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22e50c640 1 --2- 172.21.15.53:0/3074574795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe210004d00 0x7fe210005180 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe220009fc0 tx=0x7fe2100057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22caa9640 1 -- 172.21.15.53:0/3074574795 reap_dead start 2023-10-21T17:18:50.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22e50c640 1 -- 172.21.15.53:0/3074574795 shutdown_connections 2023-10-21T17:18:50.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22e50c640 1 -- 172.21.15.53:0/3074574795 >> 172.21.15.53:0/3074574795 conn(0x7fe22806a380 msgr2=0x7fe2280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:50.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.197+0000 7fe22e50c640 1 -- 172.21.15.53:0/3074574795 shutdown_connections 2023-10-21T17:18:50.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.198+0000 7fe22e50c640 1 -- 172.21.15.53:0/3074574795 wait complete. 2023-10-21T17:18:50.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.199+0000 7fe22e50c640 1 Processor -- start 2023-10-21T17:18:50.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.200+0000 7fe22e50c640 1 -- start start 2023-10-21T17:18:50.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.200+0000 7fe22e50c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210004d00 0x7fe2281f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22e50c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 0x7fe2280b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22e50c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22e50c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22815f7a0 con 0x7fe2280aea90 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22e50c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280af450 con 0x7fe210004d00 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22e50c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280b3bf0 con 0x7fe228152500 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42400/0 (socket says 172.21.15.53:42400) 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 -- 172.21.15.53:0/1119782523 learned_addr learned my addr 172.21.15.53:0/1119782523 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe2277fe640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 0x7fe2280b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe227fff640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210004d00 0x7fe2281f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 -- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 msgr2=0x7fe2280b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 0x7fe2280b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 -- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210004d00 msgr2=0x7fe2281f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210004d00 0x7fe2281f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe22caa9640 1 -- 172.21.15.53:0/1119782523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228152980 con 0x7fe2280aea90 2023-10-21T17:18:50.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.201+0000 7fe2277fe640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 0x7fe2280b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:50.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe22caa9640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e230 tx=0x7fe220009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220043070 con 0x7fe2280aea90 2023-10-21T17:18:50.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22002ce90 con 0x7fe2280aea90 2023-10-21T17:18:50.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220034cd0 con 0x7fe2280aea90 2023-10-21T17:18:50.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2280b2860 con 0x7fe2280aea90 2023-10-21T17:18:50.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.202+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280b2de0 con 0x7fe2280aea90 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.203+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe228067490 con 0x7fe2280aea90 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.204+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe22003e440 con 0x7fe2280aea90 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.206+0000 7fe224ff9640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 0x7fe21406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.206+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe2200ab210 con 0x7fe2280aea90 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.207+0000 7fe227fff640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 0x7fe21406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.208+0000 7fe227fff640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 0x7fe21406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe218000f60 tx=0x7fe218009040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.210+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe220076000 con 0x7fe2280aea90 2023-10-21T17:18:50.332 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-21T17:18:50.331+0000 7f339b851640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:18:50.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.402+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 --> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2280cd3e0 con 0x7fe21406afc0 2023-10-21T17:18:50.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.431+0000 7fe224ff9640 1 -- 172.21.15.53:0/1119782523 <== mgr.36914 v2:172.21.15.124:6832/234236878 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286696 (secure 0 0 0) 0x7fe21800d040 con 0x7fe21406afc0 2023-10-21T17:18:50.437 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:50.439 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:18:50.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 msgr2=0x7fe21406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 0x7fe21406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe218000f60 tx=0x7fe218009040 comp rx=0 tx=0).stop 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 msgr2=0x7fe2280aef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e230 tx=0x7fe220009750 comp rx=0 tx=0).stop 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 shutdown_connections 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fe21406afc0 0x7fe21406d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2280aea90 0x7fe2280aef10 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe228152500 0x7fe2280b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 --2- 172.21.15.53:0/1119782523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe210004d00 0x7fe2281f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.438+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 >> 172.21.15.53:0/1119782523 conn(0x7fe22806a380 msgr2=0x7fe22806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.439+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 shutdown_connections 2023-10-21T17:18:50.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.439+0000 7fe22e50c640 1 -- 172.21.15.53:0/1119782523 wait complete. 2023-10-21T17:18:50.460 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-21T17:18:48.471439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":491,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4834,"ondisk_log_size":4834,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361548,"kb_used_data":19092,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754613172,"statfs":{"total":773094113280,"available":772723888128,"internally_reserved":0,"allocated":19550208,"data_stored":10571548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":350539321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584876"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956064+0000","last_change":"2023-10-21T17:18:45.946472+0000","last_active":"2023-10-21T17:18:46.956064+0000","last_peered":"2023-10-21T17:18:46.956064+0000","last_clean":"2023-10-21T17:18:46.956064+0000","last_became_active":"2023-10-21T17:18:45.945911+0000","last_became_peered":"2023-10-21T17:18:45.945911+0000","last_unstale":"2023-10-21T17:18:46.956064+0000","last_undegraded":"2023-10-21T17:18:46.956064+0000","last_fullsized":"2023-10-21T17:18:46.956064+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:16.311690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.968511+0000","last_change":"2023-10-21T17:18:45.966495+0000","last_active":"2023-10-21T17:18:45.968511+0000","last_peered":"2023-10-21T17:18:45.968511+0000","last_clean":"2023-10-21T17:18:45.968511+0000","last_became_active":"2023-10-21T17:18:45.966200+0000","last_became_peered":"2023-10-21T17:18:45.966200+0000","last_unstale":"2023-10-21T17:18:45.968511+0000","last_undegraded":"2023-10-21T17:18:45.968511+0000","last_fullsized":"2023-10-21T17:18:45.968511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:49:17.231532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961003+0000","last_change":"2023-10-21T17:18:45.958769+0000","last_active":"2023-10-21T17:18:45.961003+0000","last_peered":"2023-10-21T17:18:45.961003+0000","last_clean":"2023-10-21T17:18:45.961003+0000","last_became_active":"2023-10-21T17:18:45.958284+0000","last_became_peered":"2023-10-21T17:18:45.958284+0000","last_unstale":"2023-10-21T17:18:45.961003+0000","last_undegraded":"2023-10-21T17:18:45.961003+0000","last_fullsized":"2023-10-21T17:18:45.961003+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:18.826366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272106+0000","last_change":"2023-10-21T17:18:45.957072+0000","last_active":"2023-10-21T17:18:47.272106+0000","last_peered":"2023-10-21T17:18:47.272106+0000","last_clean":"2023-10-21T17:18:47.272106+0000","last_became_active":"2023-10-21T17:18:45.955774+0000","last_became_peered":"2023-10-21T17:18:45.955774+0000","last_unstale":"2023-10-21T17:18:47.272106+0000","last_undegraded":"2023-10-21T17:18:47.272106+0000","last_fullsized":"2023-10-21T17:18:47.272106+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:10:29.091615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961057+0000","last_change":"2023-10-21T17:18:45.956112+0000","last_active":"2023-10-21T17:18:45.961057+0000","last_peered":"2023-10-21T17:18:45.961057+0000","last_clean":"2023-10-21T17:18:45.961057+0000","last_became_active":"2023-10-21T17:18:45.955836+0000","last_became_peered":"2023-10-21T17:18:45.955836+0000","last_unstale":"2023-10-21T17:18:45.961057+0000","last_undegraded":"2023-10-21T17:18:45.961057+0000","last_fullsized":"2023-10-21T17:18:45.961057+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:54:46.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960636+0000","last_change":"2023-10-21T17:18:45.955850+0000","last_active":"2023-10-21T17:18:45.960636+0000","last_peered":"2023-10-21T17:18:45.960636+0000","last_clean":"2023-10-21T17:18:45.960636+0000","last_became_active":"2023-10-21T17:18:45.955242+0000","last_became_peered":"2023-10-21T17:18:45.955242+0000","last_unstale":"2023-10-21T17:18:45.960636+0000","last_undegraded":"2023-10-21T17:18:45.960636+0000","last_fullsized":"2023-10-21T17:18:45.960636+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:40.885115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954495+0000","last_change":"2023-10-21T17:18:45.953767+0000","last_active":"2023-10-21T17:18:46.954495+0000","last_peered":"2023-10-21T17:18:46.954495+0000","last_clean":"2023-10-21T17:18:46.954495+0000","last_became_active":"2023-10-21T17:18:45.953494+0000","last_became_peered":"2023-10-21T17:18:45.953494+0000","last_unstale":"2023-10-21T17:18:46.954495+0000","last_undegraded":"2023-10-21T17:18:46.954495+0000","last_fullsized":"2023-10-21T17:18:46.954495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:33.869515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958554+0000","last_change":"2023-10-21T17:18:45.952114+0000","last_active":"2023-10-21T17:18:46.958554+0000","last_peered":"2023-10-21T17:18:46.958554+0000","last_clean":"2023-10-21T17:18:46.958554+0000","last_became_active":"2023-10-21T17:18:45.951623+0000","last_became_peered":"2023-10-21T17:18:45.951623+0000","last_unstale":"2023-10-21T17:18:46.958554+0000","last_undegraded":"2023-10-21T17:18:46.958554+0000","last_fullsized":"2023-10-21T17:18:46.958554+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:35.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966326+0000","last_change":"2023-10-21T17:18:45.962523+0000","last_active":"2023-10-21T17:18:45.966326+0000","last_peered":"2023-10-21T17:18:45.966326+0000","last_clean":"2023-10-21T17:18:45.966326+0000","last_became_active":"2023-10-21T17:18:45.962185+0000","last_became_peered":"2023-10-21T17:18:45.962185+0000","last_unstale":"2023-10-21T17:18:45.966326+0000","last_undegraded":"2023-10-21T17:18:45.966326+0000","last_fullsized":"2023-10-21T17:18:45.966326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:28.037660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.955261+0000","last_change":"2023-10-21T17:18:45.946981+0000","last_active":"2023-10-21T17:18:45.955261+0000","last_peered":"2023-10-21T17:18:45.955261+0000","last_clean":"2023-10-21T17:18:45.955261+0000","last_became_active":"2023-10-21T17:18:45.946698+0000","last_became_peered":"2023-10-21T17:18:45.946698+0000","last_unstale":"2023-10-21T17:18:45.955261+0000","last_undegraded":"2023-10-21T17:18:45.955261+0000","last_fullsized":"2023-10-21T17:18:45.955261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:24:54.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956876+0000","last_change":"2023-10-21T17:18:45.952692+0000","last_active":"2023-10-21T17:18:46.956876+0000","last_peered":"2023-10-21T17:18:46.956876+0000","last_clean":"2023-10-21T17:18:46.956876+0000","last_became_active":"2023-10-21T17:18:45.951479+0000","last_became_peered":"2023-10-21T17:18:45.951479+0000","last_unstale":"2023-10-21T17:18:46.956876+0000","last_undegraded":"2023-10-21T17:18:46.956876+0000","last_fullsized":"2023-10-21T17:18:46.956876+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:09:58.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272560+0000","last_change":"2023-10-21T17:18:45.958253+0000","last_active":"2023-10-21T17:18:47.272560+0000","last_peered":"2023-10-21T17:18:47.272560+0000","last_clean":"2023-10-21T17:18:47.272560+0000","last_became_active":"2023-10-21T17:18:45.956090+0000","last_became_peered":"2023-10-21T17:18:45.956090+0000","last_unstale":"2023-10-21T17:18:47.272560+0000","last_undegraded":"2023-10-21T17:18:47.272560+0000","last_fullsized":"2023-10-21T17:18:47.272560+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:32.751413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966913+0000","last_change":"2023-10-21T17:18:45.963839+0000","last_active":"2023-10-21T17:18:45.966913+0000","last_peered":"2023-10-21T17:18:45.966913+0000","last_clean":"2023-10-21T17:18:45.966913+0000","last_became_active":"2023-10-21T17:18:45.962390+0000","last_became_peered":"2023-10-21T17:18:45.962390+0000","last_unstale":"2023-10-21T17:18:45.966913+0000","last_undegraded":"2023-10-21T17:18:45.966913+0000","last_fullsized":"2023-10-21T17:18:45.966913+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:04.692255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272052+0000","last_change":"2023-10-21T17:18:45.960201+0000","last_active":"2023-10-21T17:18:47.272052+0000","last_peered":"2023-10-21T17:18:47.272052+0000","last_clean":"2023-10-21T17:18:47.272052+0000","last_became_active":"2023-10-21T17:18:45.955991+0000","last_became_peered":"2023-10-21T17:18:45.955991+0000","last_unstale":"2023-10-21T17:18:47.272052+0000","last_undegraded":"2023-10-21T17:18:47.272052+0000","last_fullsized":"2023-10-21T17:18:47.272052+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:31.904410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960858+0000","last_change":"2023-10-21T17:18:45.958326+0000","last_active":"2023-10-21T17:18:45.960858+0000","last_peered":"2023-10-21T17:18:45.960858+0000","last_clean":"2023-10-21T17:18:45.960858+0000","last_became_active":"2023-10-21T17:18:45.957964+0000","last_became_peered":"2023-10-21T17:18:45.957964+0000","last_unstale":"2023-10-21T17:18:45.960858+0000","last_undegraded":"2023-10-21T17:18:45.960858+0000","last_fullsized":"2023-10-21T17:18:45.960858+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:19:25.038155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960020+0000","last_change":"2023-10-21T17:18:45.955632+0000","last_active":"2023-10-21T17:18:45.960020+0000","last_peered":"2023-10-21T17:18:45.960020+0000","last_clean":"2023-10-21T17:18:45.960020+0000","last_became_active":"2023-10-21T17:18:45.955316+0000","last_became_peered":"2023-10-21T17:18:45.955316+0000","last_unstale":"2023-10-21T17:18:45.960020+0000","last_undegraded":"2023-10-21T17:18:45.960020+0000","last_fullsized":"2023-10-21T17:18:45.960020+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:27:37.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272900+0000","last_change":"2023-10-21T17:18:45.957861+0000","last_active":"2023-10-21T17:18:47.272900+0000","last_peered":"2023-10-21T17:18:47.272900+0000","last_clean":"2023-10-21T17:18:47.272900+0000","last_became_active":"2023-10-21T17:18:45.954908+0000","last_became_peered":"2023-10-21T17:18:45.954908+0000","last_unstale":"2023-10-21T17:18:47.272900+0000","last_undegraded":"2023-10-21T17:18:47.272900+0000","last_fullsized":"2023-10-21T17:18:47.272900+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:19.423240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957272+0000","last_change":"2023-10-21T17:18:45.951014+0000","last_active":"2023-10-21T17:18:46.957272+0000","last_peered":"2023-10-21T17:18:46.957272+0000","last_clean":"2023-10-21T17:18:46.957272+0000","last_became_active":"2023-10-21T17:18:45.950697+0000","last_became_peered":"2023-10-21T17:18:45.950697+0000","last_unstale":"2023-10-21T17:18:46.957272+0000","last_undegraded":"2023-10-21T17:18:46.957272+0000","last_fullsized":"2023-10-21T17:18:46.957272+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:05:52.180676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.964203+0000","last_change":"2023-10-21T17:18:45.961773+0000","last_active":"2023-10-21T17:18:45.964203+0000","last_peered":"2023-10-21T17:18:45.964203+0000","last_clean":"2023-10-21T17:18:45.964203+0000","last_became_active":"2023-10-21T17:18:45.961514+0000","last_became_peered":"2023-10-21T17:18:45.961514+0000","last_unstale":"2023-10-21T17:18:45.964203+0000","last_undegraded":"2023-10-21T17:18:45.964203+0000","last_fullsized":"2023-10-21T17:18:45.964203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:24:54.830195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446540+0000","last_change":"2023-10-21T17:18:45.949351+0000","last_active":"2023-10-21T17:18:47.446540+0000","last_peered":"2023-10-21T17:18:47.446540+0000","last_clean":"2023-10-21T17:18:47.446540+0000","last_became_active":"2023-10-21T17:18:45.948962+0000","last_became_peered":"2023-10-21T17:18:45.948962+0000","last_unstale":"2023-10-21T17:18:47.446540+0000","last_undegraded":"2023-10-21T17:18:47.446540+0000","last_fullsized":"2023-10-21T17:18:47.446540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:21.538295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966732+0000","last_change":"2023-10-21T17:18:45.963931+0000","last_active":"2023-10-21T17:18:45.966732+0000","last_peered":"2023-10-21T17:18:45.966732+0000","last_clean":"2023-10-21T17:18:45.966732+0000","last_became_active":"2023-10-21T17:18:45.963627+0000","last_became_peered":"2023-10-21T17:18:45.963627+0000","last_unstale":"2023-10-21T17:18:45.966732+0000","last_undegraded":"2023-10-21T17:18:45.966732+0000","last_fullsized":"2023-10-21T17:18:45.966732+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:25.875229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954837+0000","last_change":"2023-10-21T17:18:45.954257+0000","last_active":"2023-10-21T17:18:46.954837+0000","last_peered":"2023-10-21T17:18:46.954837+0000","last_clean":"2023-10-21T17:18:46.954837+0000","last_became_active":"2023-10-21T17:18:45.953890+0000","last_became_peered":"2023-10-21T17:18:45.953890+0000","last_unstale":"2023-10-21T17:18:46.954837+0000","last_undegraded":"2023-10-21T17:18:46.954837+0000","last_fullsized":"2023-10-21T17:18:46.954837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:26:25.002205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958642+0000","last_change":"2023-10-21T17:18:45.941632+0000","last_active":"2023-10-21T17:18:46.958642+0000","last_peered":"2023-10-21T17:18:46.958642+0000","last_clean":"2023-10-21T17:18:46.958642+0000","last_became_active":"2023-10-21T17:18:45.941221+0000","last_became_peered":"2023-10-21T17:18:45.941221+0000","last_unstale":"2023-10-21T17:18:46.958642+0000","last_undegraded":"2023-10-21T17:18:46.958642+0000","last_fullsized":"2023-10-21T17:18:46.958642+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:51.561437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956702+0000","last_change":"2023-10-21T17:18:45.954318+0000","last_active":"2023-10-21T17:18:46.956702+0000","last_peered":"2023-10-21T17:18:46.956702+0000","last_clean":"2023-10-21T17:18:46.956702+0000","last_became_active":"2023-10-21T17:18:45.953922+0000","last_became_peered":"2023-10-21T17:18:45.953922+0000","last_unstale":"2023-10-21T17:18:46.956702+0000","last_undegraded":"2023-10-21T17:18:46.956702+0000","last_fullsized":"2023-10-21T17:18:46.956702+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:26:59.882134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956120+0000","last_change":"2023-10-21T17:18:45.945455+0000","last_active":"2023-10-21T17:18:46.956120+0000","last_peered":"2023-10-21T17:18:46.956120+0000","last_clean":"2023-10-21T17:18:46.956120+0000","last_became_active":"2023-10-21T17:18:45.945008+0000","last_became_peered":"2023-10-21T17:18:45.945008+0000","last_unstale":"2023-10-21T17:18:46.956120+0000","last_undegraded":"2023-10-21T17:18:46.956120+0000","last_fullsized":"2023-10-21T17:18:46.956120+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:24:54.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955623+0000","last_change":"2023-10-21T17:18:45.960509+0000","last_active":"2023-10-21T17:18:46.955623+0000","last_peered":"2023-10-21T17:18:46.955623+0000","last_clean":"2023-10-21T17:18:46.955623+0000","last_became_active":"2023-10-21T17:18:45.957659+0000","last_became_peered":"2023-10-21T17:18:45.957659+0000","last_unstale":"2023-10-21T17:18:46.955623+0000","last_undegraded":"2023-10-21T17:18:46.955623+0000","last_fullsized":"2023-10-21T17:18:46.955623+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:00:07.625445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966630+0000","last_change":"2023-10-21T17:18:45.963416+0000","last_active":"2023-10-21T17:18:45.966630+0000","last_peered":"2023-10-21T17:18:45.966630+0000","last_clean":"2023-10-21T17:18:45.966630+0000","last_became_active":"2023-10-21T17:18:45.963011+0000","last_became_peered":"2023-10-21T17:18:45.963011+0000","last_unstale":"2023-10-21T17:18:45.966630+0000","last_undegraded":"2023-10-21T17:18:45.966630+0000","last_fullsized":"2023-10-21T17:18:45.966630+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:04:23.140791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956704+0000","last_change":"2023-10-21T17:18:45.946331+0000","last_active":"2023-10-21T17:18:46.956704+0000","last_peered":"2023-10-21T17:18:46.956704+0000","last_clean":"2023-10-21T17:18:46.956704+0000","last_became_active":"2023-10-21T17:18:45.945573+0000","last_became_peered":"2023-10-21T17:18:45.945573+0000","last_unstale":"2023-10-21T17:18:46.956704+0000","last_undegraded":"2023-10-21T17:18:46.956704+0000","last_fullsized":"2023-10-21T17:18:46.956704+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:52:12.511776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958988+0000","last_change":"2023-10-21T17:18:45.959220+0000","last_active":"2023-10-21T17:18:46.958988+0000","last_peered":"2023-10-21T17:18:46.958988+0000","last_clean":"2023-10-21T17:18:46.958988+0000","last_became_active":"2023-10-21T17:18:45.958868+0000","last_became_peered":"2023-10-21T17:18:45.958868+0000","last_unstale":"2023-10-21T17:18:46.958988+0000","last_undegraded":"2023-10-21T17:18:46.958988+0000","last_fullsized":"2023-10-21T17:18:46.958988+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:00.576689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956766+0000","last_change":"2023-10-21T17:18:45.951341+0000","last_active":"2023-10-21T17:18:46.956766+0000","last_peered":"2023-10-21T17:18:46.956766+0000","last_clean":"2023-10-21T17:18:46.956766+0000","last_became_active":"2023-10-21T17:18:45.951073+0000","last_became_peered":"2023-10-21T17:18:45.951073+0000","last_unstale":"2023-10-21T17:18:46.956766+0000","last_undegraded":"2023-10-21T17:18:46.956766+0000","last_fullsized":"2023-10-21T17:18:46.956766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:06.433984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.959078+0000","last_change":"2023-10-21T17:18:45.954735+0000","last_active":"2023-10-21T17:18:45.959078+0000","last_peered":"2023-10-21T17:18:45.959078+0000","last_clean":"2023-10-21T17:18:45.959078+0000","last_became_active":"2023-10-21T17:18:45.954342+0000","last_became_peered":"2023-10-21T17:18:45.954342+0000","last_unstale":"2023-10-21T17:18:45.959078+0000","last_undegraded":"2023-10-21T17:18:45.959078+0000","last_fullsized":"2023-10-21T17:18:45.959078+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:31:56.073758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959635+0000","last_change":"2023-10-21T17:18:45.953330+0000","last_active":"2023-10-21T17:18:46.959635+0000","last_peered":"2023-10-21T17:18:46.959635+0000","last_clean":"2023-10-21T17:18:46.959635+0000","last_became_active":"2023-10-21T17:18:45.953003+0000","last_became_peered":"2023-10-21T17:18:45.953003+0000","last_unstale":"2023-10-21T17:18:46.959635+0000","last_undegraded":"2023-10-21T17:18:46.959635+0000","last_fullsized":"2023-10-21T17:18:46.959635+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:38.574434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963363+0000","last_change":"2023-10-21T17:18:45.959076+0000","last_active":"2023-10-21T17:18:45.963363+0000","last_peered":"2023-10-21T17:18:45.963363+0000","last_clean":"2023-10-21T17:18:45.963363+0000","last_became_active":"2023-10-21T17:18:45.957531+0000","last_became_peered":"2023-10-21T17:18:45.957531+0000","last_unstale":"2023-10-21T17:18:45.963363+0000","last_undegraded":"2023-10-21T17:18:45.963363+0000","last_fullsized":"2023-10-21T17:18:45.963363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:25.135652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"183'4","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.976054+0000","last_change":"2023-10-21T17:18:44.921449+0000","last_active":"2023-10-21T17:18:45.976054+0000","last_peered":"2023-10-21T17:18:45.976054+0000","last_clean":"2023-10-21T17:18:45.976054+0000","last_became_active":"2023-10-21T17:18:44.921204+0000","last_became_peered":"2023-10-21T17:18:44.921204+0000","last_unstale":"2023-10-21T17:18:45.976054+0000","last_undegraded":"2023-10-21T17:18:45.976054+0000","last_fullsized":"2023-10-21T17:18:45.976054+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:01.569523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960984+0000","last_change":"2023-10-21T17:18:45.958560+0000","last_active":"2023-10-21T17:18:45.960984+0000","last_peered":"2023-10-21T17:18:45.960984+0000","last_clean":"2023-10-21T17:18:45.960984+0000","last_became_active":"2023-10-21T17:18:45.957866+0000","last_became_peered":"2023-10-21T17:18:45.957866+0000","last_unstale":"2023-10-21T17:18:45.960984+0000","last_undegraded":"2023-10-21T17:18:45.960984+0000","last_fullsized":"2023-10-21T17:18:45.960984+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:38.443203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975986+0000","last_change":"2023-10-21T17:18:44.930566+0000","last_active":"2023-10-21T17:18:45.975986+0000","last_peered":"2023-10-21T17:18:45.975986+0000","last_clean":"2023-10-21T17:18:45.975986+0000","last_became_active":"2023-10-21T17:18:44.930272+0000","last_became_peered":"2023-10-21T17:18:44.930272+0000","last_unstale":"2023-10-21T17:18:45.975986+0000","last_undegraded":"2023-10-21T17:18:45.975986+0000","last_fullsized":"2023-10-21T17:18:45.975986+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:08.700057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959431+0000","last_change":"2023-10-21T17:18:45.951750+0000","last_active":"2023-10-21T17:18:46.959431+0000","last_peered":"2023-10-21T17:18:46.959431+0000","last_clean":"2023-10-21T17:18:46.959431+0000","last_became_active":"2023-10-21T17:18:45.951466+0000","last_became_peered":"2023-10-21T17:18:45.951466+0000","last_unstale":"2023-10-21T17:18:46.959431+0000","last_undegraded":"2023-10-21T17:18:46.959431+0000","last_fullsized":"2023-10-21T17:18:46.959431+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:25.239123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975530+0000","last_change":"2023-10-21T17:18:44.935832+0000","last_active":"2023-10-21T17:18:45.975530+0000","last_peered":"2023-10-21T17:18:45.975530+0000","last_clean":"2023-10-21T17:18:45.975530+0000","last_became_active":"2023-10-21T17:18:44.935539+0000","last_became_peered":"2023-10-21T17:18:44.935539+0000","last_unstale":"2023-10-21T17:18:45.975530+0000","last_undegraded":"2023-10-21T17:18:45.975530+0000","last_fullsized":"2023-10-21T17:18:45.975530+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:56.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966720+0000","last_change":"2023-10-21T17:18:45.962418+0000","last_active":"2023-10-21T17:18:45.966720+0000","last_peered":"2023-10-21T17:18:45.966720+0000","last_clean":"2023-10-21T17:18:45.966720+0000","last_became_active":"2023-10-21T17:18:45.960824+0000","last_became_peered":"2023-10-21T17:18:45.960824+0000","last_unstale":"2023-10-21T17:18:45.966720+0000","last_undegraded":"2023-10-21T17:18:45.966720+0000","last_fullsized":"2023-10-21T17:18:45.966720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:01.336383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.974133+0000","last_change":"2023-10-21T17:18:44.928291+0000","last_active":"2023-10-21T17:18:45.974133+0000","last_peered":"2023-10-21T17:18:45.974133+0000","last_clean":"2023-10-21T17:18:45.974133+0000","last_became_active":"2023-10-21T17:18:44.927669+0000","last_became_peered":"2023-10-21T17:18:44.927669+0000","last_unstale":"2023-10-21T17:18:45.974133+0000","last_undegraded":"2023-10-21T17:18:45.974133+0000","last_fullsized":"2023-10-21T17:18:45.974133+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:57:18.809182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446326+0000","last_change":"2023-10-21T17:18:45.946656+0000","last_active":"2023-10-21T17:18:47.446326+0000","last_peered":"2023-10-21T17:18:47.446326+0000","last_clean":"2023-10-21T17:18:47.446326+0000","last_became_active":"2023-10-21T17:18:45.946321+0000","last_became_peered":"2023-10-21T17:18:45.946321+0000","last_unstale":"2023-10-21T17:18:47.446326+0000","last_undegraded":"2023-10-21T17:18:47.446326+0000","last_fullsized":"2023-10-21T17:18:47.446326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:09.704931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"183'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959033+0000","last_change":"2023-10-21T17:18:44.928731+0000","last_active":"2023-10-21T17:18:46.959033+0000","last_peered":"2023-10-21T17:18:46.959033+0000","last_clean":"2023-10-21T17:18:46.959033+0000","last_became_active":"2023-10-21T17:18:44.928449+0000","last_became_peered":"2023-10-21T17:18:44.928449+0000","last_unstale":"2023-10-21T17:18:46.959033+0000","last_undegraded":"2023-10-21T17:18:46.959033+0000","last_fullsized":"2023-10-21T17:18:46.959033+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:27.690328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.271896+0000","last_change":"2023-10-21T17:18:45.961875+0000","last_active":"2023-10-21T17:18:47.271896+0000","last_peered":"2023-10-21T17:18:47.271896+0000","last_clean":"2023-10-21T17:18:47.271896+0000","last_became_active":"2023-10-21T17:18:45.961274+0000","last_became_peered":"2023-10-21T17:18:45.961274+0000","last_unstale":"2023-10-21T17:18:47.271896+0000","last_undegraded":"2023-10-21T17:18:47.271896+0000","last_fullsized":"2023-10-21T17:18:47.271896+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:37:14.180774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.973719+0000","last_change":"2023-10-21T17:18:44.929379+0000","last_active":"2023-10-21T17:18:45.973719+0000","last_peered":"2023-10-21T17:18:45.973719+0000","last_clean":"2023-10-21T17:18:45.973719+0000","last_became_active":"2023-10-21T17:18:44.929123+0000","last_became_peered":"2023-10-21T17:18:44.929123+0000","last_unstale":"2023-10-21T17:18:45.973719+0000","last_undegraded":"2023-10-21T17:18:45.973719+0000","last_fullsized":"2023-10-21T17:18:45.973719+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:20.560809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963075+0000","last_change":"2023-10-21T17:18:45.958845+0000","last_active":"2023-10-21T17:18:45.963075+0000","last_peered":"2023-10-21T17:18:45.963075+0000","last_clean":"2023-10-21T17:18:45.963075+0000","last_became_active":"2023-10-21T17:18:45.958520+0000","last_became_peered":"2023-10-21T17:18:45.958520+0000","last_unstale":"2023-10-21T17:18:45.963075+0000","last_undegraded":"2023-10-21T17:18:45.963075+0000","last_fullsized":"2023-10-21T17:18:45.963075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:44:55.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955191+0000","last_change":"2023-10-21T17:18:44.928515+0000","last_active":"2023-10-21T17:18:46.955191+0000","last_peered":"2023-10-21T17:18:46.955191+0000","last_clean":"2023-10-21T17:18:46.955191+0000","last_became_active":"2023-10-21T17:18:44.928173+0000","last_became_peered":"2023-10-21T17:18:44.928173+0000","last_unstale":"2023-10-21T17:18:46.955191+0000","last_undegraded":"2023-10-21T17:18:46.955191+0000","last_fullsized":"2023-10-21T17:18:46.955191+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:13:13.490105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955247+0000","last_change":"2023-10-21T17:18:45.939278+0000","last_active":"2023-10-21T17:18:46.955247+0000","last_peered":"2023-10-21T17:18:46.955247+0000","last_clean":"2023-10-21T17:18:46.955247+0000","last_became_active":"2023-10-21T17:18:45.938531+0000","last_became_peered":"2023-10-21T17:18:45.938531+0000","last_unstale":"2023-10-21T17:18:46.955247+0000","last_undegraded":"2023-10-21T17:18:46.955247+0000","last_fullsized":"2023-10-21T17:18:46.955247+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:15.146123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.936560+0000","last_change":"2023-10-21T17:18:44.927882+0000","last_active":"2023-10-21T17:18:45.936560+0000","last_peered":"2023-10-21T17:18:45.936560+0000","last_clean":"2023-10-21T17:18:45.936560+0000","last_became_active":"2023-10-21T17:18:44.927603+0000","last_became_peered":"2023-10-21T17:18:44.927603+0000","last_unstale":"2023-10-21T17:18:45.936560+0000","last_undegraded":"2023-10-21T17:18:45.936560+0000","last_fullsized":"2023-10-21T17:18:45.936560+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:12.844392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959540+0000","last_change":"2023-10-21T17:18:45.951306+0000","last_active":"2023-10-21T17:18:46.959540+0000","last_peered":"2023-10-21T17:18:46.959540+0000","last_clean":"2023-10-21T17:18:46.959540+0000","last_became_active":"2023-10-21T17:18:45.951025+0000","last_became_peered":"2023-10-21T17:18:45.951025+0000","last_unstale":"2023-10-21T17:18:46.959540+0000","last_undegraded":"2023-10-21T17:18:46.959540+0000","last_fullsized":"2023-10-21T17:18:46.959540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:39:28.204765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.943705+0000","last_change":"2023-10-21T17:18:44.928130+0000","last_active":"2023-10-21T17:18:45.943705+0000","last_peered":"2023-10-21T17:18:45.943705+0000","last_clean":"2023-10-21T17:18:45.943705+0000","last_became_active":"2023-10-21T17:18:44.927860+0000","last_became_peered":"2023-10-21T17:18:44.927860+0000","last_unstale":"2023-10-21T17:18:45.943705+0000","last_undegraded":"2023-10-21T17:18:45.943705+0000","last_fullsized":"2023-10-21T17:18:45.943705+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:41.144389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957077+0000","last_change":"2023-10-21T17:18:45.951752+0000","last_active":"2023-10-21T17:18:46.957077+0000","last_peered":"2023-10-21T17:18:46.957077+0000","last_clean":"2023-10-21T17:18:46.957077+0000","last_became_active":"2023-10-21T17:18:45.951083+0000","last_became_peered":"2023-10-21T17:18:45.951083+0000","last_unstale":"2023-10-21T17:18:46.957077+0000","last_undegraded":"2023-10-21T17:18:46.957077+0000","last_fullsized":"2023-10-21T17:18:46.957077+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:21.975896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956092+0000","last_change":"2023-10-21T17:18:44.927565+0000","last_active":"2023-10-21T17:18:46.956092+0000","last_peered":"2023-10-21T17:18:46.956092+0000","last_clean":"2023-10-21T17:18:46.956092+0000","last_became_active":"2023-10-21T17:18:44.926608+0000","last_became_peered":"2023-10-21T17:18:44.926608+0000","last_unstale":"2023-10-21T17:18:46.956092+0000","last_undegraded":"2023-10-21T17:18:46.956092+0000","last_fullsized":"2023-10-21T17:18:46.956092+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:06.981238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957167+0000","last_change":"2023-10-21T17:18:45.951701+0000","last_active":"2023-10-21T17:18:46.957167+0000","last_peered":"2023-10-21T17:18:46.957167+0000","last_clean":"2023-10-21T17:18:46.957167+0000","last_became_active":"2023-10-21T17:18:45.951382+0000","last_became_peered":"2023-10-21T17:18:45.951382+0000","last_unstale":"2023-10-21T17:18:46.957167+0000","last_undegraded":"2023-10-21T17:18:46.957167+0000","last_fullsized":"2023-10-21T17:18:46.957167+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:42.986072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"183'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957027+0000","last_change":"2023-10-21T17:18:44.905516+0000","last_active":"2023-10-21T17:18:46.957027+0000","last_peered":"2023-10-21T17:18:46.957027+0000","last_clean":"2023-10-21T17:18:46.957027+0000","last_became_active":"2023-10-21T17:18:44.904602+0000","last_became_peered":"2023-10-21T17:18:44.904602+0000","last_unstale":"2023-10-21T17:18:46.957027+0000","last_undegraded":"2023-10-21T17:18:46.957027+0000","last_fullsized":"2023-10-21T17:18:46.957027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:23:18.904211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958940+0000","last_change":"2023-10-21T17:18:45.954223+0000","last_active":"2023-10-21T17:18:46.958940+0000","last_peered":"2023-10-21T17:18:46.958940+0000","last_clean":"2023-10-21T17:18:46.958940+0000","last_became_active":"2023-10-21T17:18:45.953964+0000","last_became_peered":"2023-10-21T17:18:45.953964+0000","last_unstale":"2023-10-21T17:18:46.958940+0000","last_undegraded":"2023-10-21T17:18:46.958940+0000","last_fullsized":"2023-10-21T17:18:46.958940+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:42:10.240710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975998+0000","last_change":"2023-10-21T17:18:44.921410+0000","last_active":"2023-10-21T17:18:45.975998+0000","last_peered":"2023-10-21T17:18:45.975998+0000","last_clean":"2023-10-21T17:18:45.975998+0000","last_became_active":"2023-10-21T17:18:44.921130+0000","last_became_peered":"2023-10-21T17:18:44.921130+0000","last_unstale":"2023-10-21T17:18:45.975998+0000","last_undegraded":"2023-10-21T17:18:45.975998+0000","last_fullsized":"2023-10-21T17:18:45.975998+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:11.516018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960950+0000","last_change":"2023-10-21T17:18:45.958827+0000","last_active":"2023-10-21T17:18:45.960950+0000","last_peered":"2023-10-21T17:18:45.960950+0000","last_clean":"2023-10-21T17:18:45.960950+0000","last_became_active":"2023-10-21T17:18:45.958503+0000","last_became_peered":"2023-10-21T17:18:45.958503+0000","last_unstale":"2023-10-21T17:18:45.960950+0000","last_undegraded":"2023-10-21T17:18:45.960950+0000","last_fullsized":"2023-10-21T17:18:45.960950+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:19.423240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272027+0000","last_change":"2023-10-21T17:18:44.933084+0000","last_active":"2023-10-21T17:18:47.272027+0000","last_peered":"2023-10-21T17:18:47.272027+0000","last_clean":"2023-10-21T17:18:47.272027+0000","last_became_active":"2023-10-21T17:18:44.932808+0000","last_became_peered":"2023-10-21T17:18:44.932808+0000","last_unstale":"2023-10-21T17:18:47.272027+0000","last_undegraded":"2023-10-21T17:18:47.272027+0000","last_fullsized":"2023-10-21T17:18:47.272027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:45.814539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961734+0000","last_change":"2023-10-21T17:18:45.960132+0000","last_active":"2023-10-21T17:18:45.961734+0000","last_peered":"2023-10-21T17:18:45.961734+0000","last_clean":"2023-10-21T17:18:45.961734+0000","last_became_active":"2023-10-21T17:18:45.956738+0000","last_became_peered":"2023-10-21T17:18:45.956738+0000","last_unstale":"2023-10-21T17:18:45.961734+0000","last_undegraded":"2023-10-21T17:18:45.961734+0000","last_fullsized":"2023-10-21T17:18:45.961734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:16.311690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272103+0000","last_change":"2023-10-21T17:18:44.924137+0000","last_active":"2023-10-21T17:18:47.272103+0000","last_peered":"2023-10-21T17:18:47.272103+0000","last_clean":"2023-10-21T17:18:47.272103+0000","last_became_active":"2023-10-21T17:18:44.923842+0000","last_became_peered":"2023-10-21T17:18:44.923842+0000","last_unstale":"2023-10-21T17:18:47.272103+0000","last_undegraded":"2023-10-21T17:18:47.272103+0000","last_fullsized":"2023-10-21T17:18:47.272103+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:52.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960614+0000","last_change":"2023-10-21T17:18:45.956532+0000","last_active":"2023-10-21T17:18:45.960614+0000","last_peered":"2023-10-21T17:18:45.960614+0000","last_clean":"2023-10-21T17:18:45.960614+0000","last_became_active":"2023-10-21T17:18:45.956193+0000","last_became_peered":"2023-10-21T17:18:45.956193+0000","last_unstale":"2023-10-21T17:18:45.960614+0000","last_undegraded":"2023-10-21T17:18:45.960614+0000","last_fullsized":"2023-10-21T17:18:45.960614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:42.736079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"183'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954468+0000","last_change":"2023-10-21T17:18:44.932179+0000","last_active":"2023-10-21T17:18:46.954468+0000","last_peered":"2023-10-21T17:18:46.954468+0000","last_clean":"2023-10-21T17:18:46.954468+0000","last_became_active":"2023-10-21T17:18:44.930361+0000","last_became_peered":"2023-10-21T17:18:44.930361+0000","last_unstale":"2023-10-21T17:18:46.954468+0000","last_undegraded":"2023-10-21T17:18:46.954468+0000","last_fullsized":"2023-10-21T17:18:46.954468+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966971+0000","last_change":"2023-10-21T17:18:45.963617+0000","last_active":"2023-10-21T17:18:45.966971+0000","last_peered":"2023-10-21T17:18:45.966971+0000","last_clean":"2023-10-21T17:18:45.966971+0000","last_became_active":"2023-10-21T17:18:45.963319+0000","last_became_peered":"2023-10-21T17:18:45.963319+0000","last_unstale":"2023-10-21T17:18:45.966971+0000","last_undegraded":"2023-10-21T17:18:45.966971+0000","last_fullsized":"2023-10-21T17:18:45.966971+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:25:58.232488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"183'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956842+0000","last_change":"2023-10-21T17:18:44.922866+0000","last_active":"2023-10-21T17:18:46.956842+0000","last_peered":"2023-10-21T17:18:46.956842+0000","last_clean":"2023-10-21T17:18:46.956842+0000","last_became_active":"2023-10-21T17:18:44.922597+0000","last_became_peered":"2023-10-21T17:18:44.922597+0000","last_unstale":"2023-10-21T17:18:46.956842+0000","last_undegraded":"2023-10-21T17:18:46.956842+0000","last_fullsized":"2023-10-21T17:18:46.956842+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:17.621264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967299+0000","last_change":"2023-10-21T17:18:45.965443+0000","last_active":"2023-10-21T17:18:45.967299+0000","last_peered":"2023-10-21T17:18:45.967299+0000","last_clean":"2023-10-21T17:18:45.967299+0000","last_became_active":"2023-10-21T17:18:45.964963+0000","last_became_peered":"2023-10-21T17:18:45.964963+0000","last_unstale":"2023-10-21T17:18:45.967299+0000","last_undegraded":"2023-10-21T17:18:45.967299+0000","last_fullsized":"2023-10-21T17:18:45.967299+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:07.986111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975964+0000","last_change":"2023-10-21T17:18:44.927358+0000","last_active":"2023-10-21T17:18:45.975964+0000","last_peered":"2023-10-21T17:18:45.975964+0000","last_clean":"2023-10-21T17:18:45.975964+0000","last_became_active":"2023-10-21T17:18:44.926468+0000","last_became_peered":"2023-10-21T17:18:44.926468+0000","last_unstale":"2023-10-21T17:18:45.975964+0000","last_undegraded":"2023-10-21T17:18:45.975964+0000","last_fullsized":"2023-10-21T17:18:45.975964+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:20.533421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955816+0000","last_change":"2023-10-21T17:18:45.963974+0000","last_active":"2023-10-21T17:18:46.955816+0000","last_peered":"2023-10-21T17:18:46.955816+0000","last_clean":"2023-10-21T17:18:46.955816+0000","last_became_active":"2023-10-21T17:18:45.963721+0000","last_became_peered":"2023-10-21T17:18:45.963721+0000","last_unstale":"2023-10-21T17:18:46.955816+0000","last_undegraded":"2023-10-21T17:18:46.955816+0000","last_fullsized":"2023-10-21T17:18:46.955816+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:23.766133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.973254+0000","last_change":"2023-10-21T17:18:44.926416+0000","last_active":"2023-10-21T17:18:45.973254+0000","last_peered":"2023-10-21T17:18:45.973254+0000","last_clean":"2023-10-21T17:18:45.973254+0000","last_became_active":"2023-10-21T17:18:44.926135+0000","last_became_peered":"2023-10-21T17:18:44.926135+0000","last_unstale":"2023-10-21T17:18:45.973254+0000","last_undegraded":"2023-10-21T17:18:45.973254+0000","last_fullsized":"2023-10-21T17:18:45.973254+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:57.851590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956229+0000","last_change":"2023-10-21T17:18:45.958438+0000","last_active":"2023-10-21T17:18:46.956229+0000","last_peered":"2023-10-21T17:18:46.956229+0000","last_clean":"2023-10-21T17:18:46.956229+0000","last_became_active":"2023-10-21T17:18:45.958120+0000","last_became_peered":"2023-10-21T17:18:45.958120+0000","last_unstale":"2023-10-21T17:18:46.956229+0000","last_undegraded":"2023-10-21T17:18:46.956229+0000","last_fullsized":"2023-10-21T17:18:46.956229+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:40.885115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272433+0000","last_change":"2023-10-21T17:18:44.933895+0000","last_active":"2023-10-21T17:18:47.272433+0000","last_peered":"2023-10-21T17:18:47.272433+0000","last_clean":"2023-10-21T17:18:47.272433+0000","last_became_active":"2023-10-21T17:18:44.933635+0000","last_became_peered":"2023-10-21T17:18:44.933635+0000","last_unstale":"2023-10-21T17:18:47.272433+0000","last_undegraded":"2023-10-21T17:18:47.272433+0000","last_fullsized":"2023-10-21T17:18:47.272433+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:07.107043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963016+0000","last_change":"2023-10-21T17:18:45.958300+0000","last_active":"2023-10-21T17:18:45.963016+0000","last_peered":"2023-10-21T17:18:45.963016+0000","last_clean":"2023-10-21T17:18:45.963016+0000","last_became_active":"2023-10-21T17:18:45.958019+0000","last_became_peered":"2023-10-21T17:18:45.958019+0000","last_unstale":"2023-10-21T17:18:45.963016+0000","last_undegraded":"2023-10-21T17:18:45.963016+0000","last_fullsized":"2023-10-21T17:18:45.963016+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:57.831529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"183'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446283+0000","last_change":"2023-10-21T17:18:44.918987+0000","last_active":"2023-10-21T17:18:47.446283+0000","last_peered":"2023-10-21T17:18:47.446283+0000","last_clean":"2023-10-21T17:18:47.446283+0000","last_became_active":"2023-10-21T17:18:44.918617+0000","last_became_peered":"2023-10-21T17:18:44.918617+0000","last_unstale":"2023-10-21T17:18:47.446283+0000","last_undegraded":"2023-10-21T17:18:47.446283+0000","last_fullsized":"2023-10-21T17:18:47.446283+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.958174+0000","last_change":"2023-10-21T17:18:45.952576+0000","last_active":"2023-10-21T17:18:45.958174+0000","last_peered":"2023-10-21T17:18:45.958174+0000","last_clean":"2023-10-21T17:18:45.958174+0000","last_became_active":"2023-10-21T17:18:45.952233+0000","last_became_peered":"2023-10-21T17:18:45.952233+0000","last_unstale":"2023-10-21T17:18:45.958174+0000","last_undegraded":"2023-10-21T17:18:45.958174+0000","last_fullsized":"2023-10-21T17:18:45.958174+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:39.429149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958905+0000","last_change":"2023-10-21T17:18:44.930761+0000","last_active":"2023-10-21T17:18:46.958905+0000","last_peered":"2023-10-21T17:18:46.958905+0000","last_clean":"2023-10-21T17:18:46.958905+0000","last_became_active":"2023-10-21T17:18:44.929671+0000","last_became_peered":"2023-10-21T17:18:44.929671+0000","last_unstale":"2023-10-21T17:18:46.958905+0000","last_undegraded":"2023-10-21T17:18:46.958905+0000","last_fullsized":"2023-10-21T17:18:46.958905+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:52.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963052+0000","last_change":"2023-10-21T17:18:45.955841+0000","last_active":"2023-10-21T17:18:45.963052+0000","last_peered":"2023-10-21T17:18:45.963052+0000","last_clean":"2023-10-21T17:18:45.963052+0000","last_became_active":"2023-10-21T17:18:45.955505+0000","last_became_peered":"2023-10-21T17:18:45.955505+0000","last_unstale":"2023-10-21T17:18:45.963052+0000","last_undegraded":"2023-10-21T17:18:45.963052+0000","last_fullsized":"2023-10-21T17:18:45.963052+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:22.958231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.445931+0000","last_change":"2023-10-21T17:18:44.917343+0000","last_active":"2023-10-21T17:18:47.445931+0000","last_peered":"2023-10-21T17:18:47.445931+0000","last_clean":"2023-10-21T17:18:47.445931+0000","last_became_active":"2023-10-21T17:18:44.916935+0000","last_became_peered":"2023-10-21T17:18:44.916935+0000","last_unstale":"2023-10-21T17:18:47.445931+0000","last_undegraded":"2023-10-21T17:18:47.445931+0000","last_fullsized":"2023-10-21T17:18:47.445931+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:36.162430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963837+0000","last_change":"2023-10-21T17:18:45.959935+0000","last_active":"2023-10-21T17:18:45.963837+0000","last_peered":"2023-10-21T17:18:45.963837+0000","last_clean":"2023-10-21T17:18:45.963837+0000","last_became_active":"2023-10-21T17:18:45.959648+0000","last_became_peered":"2023-10-21T17:18:45.959648+0000","last_unstale":"2023-10-21T17:18:45.963837+0000","last_undegraded":"2023-10-21T17:18:45.963837+0000","last_fullsized":"2023-10-21T17:18:45.963837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:21.984308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959522+0000","last_change":"2023-10-21T17:18:44.926990+0000","last_active":"2023-10-21T17:18:46.959522+0000","last_peered":"2023-10-21T17:18:46.959522+0000","last_clean":"2023-10-21T17:18:46.959522+0000","last_became_active":"2023-10-21T17:18:44.926705+0000","last_became_peered":"2023-10-21T17:18:44.926705+0000","last_unstale":"2023-10-21T17:18:46.959522+0000","last_undegraded":"2023-10-21T17:18:46.959522+0000","last_fullsized":"2023-10-21T17:18:46.959522+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:58.789688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967489+0000","last_change":"2023-10-21T17:18:45.963937+0000","last_active":"2023-10-21T17:18:45.967489+0000","last_peered":"2023-10-21T17:18:45.967489+0000","last_clean":"2023-10-21T17:18:45.967489+0000","last_became_active":"2023-10-21T17:18:45.963539+0000","last_became_peered":"2023-10-21T17:18:45.963539+0000","last_unstale":"2023-10-21T17:18:45.967489+0000","last_undegraded":"2023-10-21T17:18:45.967489+0000","last_fullsized":"2023-10-21T17:18:45.967489+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:37:49.963150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"183'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272968+0000","last_change":"2023-10-21T17:18:44.932226+0000","last_active":"2023-10-21T17:18:47.272968+0000","last_peered":"2023-10-21T17:18:47.272968+0000","last_clean":"2023-10-21T17:18:47.272968+0000","last_became_active":"2023-10-21T17:18:44.931948+0000","last_became_peered":"2023-10-21T17:18:44.931948+0000","last_unstale":"2023-10-21T17:18:47.272968+0000","last_undegraded":"2023-10-21T17:18:47.272968+0000","last_fullsized":"2023-10-21T17:18:47.272968+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:45:10.929847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272844+0000","last_change":"2023-10-21T17:18:45.963319+0000","last_active":"2023-10-21T17:18:47.272844+0000","last_peered":"2023-10-21T17:18:47.272844+0000","last_clean":"2023-10-21T17:18:47.272844+0000","last_became_active":"2023-10-21T17:18:45.963051+0000","last_became_peered":"2023-10-21T17:18:45.963051+0000","last_unstale":"2023-10-21T17:18:47.272844+0000","last_undegraded":"2023-10-21T17:18:47.272844+0000","last_fullsized":"2023-10-21T17:18:47.272844+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:15.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955971+0000","last_change":"2023-10-21T17:18:44.927320+0000","last_active":"2023-10-21T17:18:46.955971+0000","last_peered":"2023-10-21T17:18:46.955971+0000","last_clean":"2023-10-21T17:18:46.955971+0000","last_became_active":"2023-10-21T17:18:44.925951+0000","last_became_peered":"2023-10-21T17:18:44.925951+0000","last_unstale":"2023-10-21T17:18:46.955971+0000","last_undegraded":"2023-10-21T17:18:46.955971+0000","last_fullsized":"2023-10-21T17:18:46.955971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:56.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955835+0000","last_change":"2023-10-21T17:18:45.941961+0000","last_active":"2023-10-21T17:18:46.955835+0000","last_peered":"2023-10-21T17:18:46.955835+0000","last_clean":"2023-10-21T17:18:46.955835+0000","last_became_active":"2023-10-21T17:18:45.941388+0000","last_became_peered":"2023-10-21T17:18:45.941388+0000","last_unstale":"2023-10-21T17:18:46.955835+0000","last_undegraded":"2023-10-21T17:18:46.955835+0000","last_fullsized":"2023-10-21T17:18:46.955835+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:34:59.636779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.941178+0000","last_change":"2023-10-21T17:18:44.931655+0000","last_active":"2023-10-21T17:18:45.941178+0000","last_peered":"2023-10-21T17:18:45.941178+0000","last_clean":"2023-10-21T17:18:45.941178+0000","last_became_active":"2023-10-21T17:18:44.931368+0000","last_became_peered":"2023-10-21T17:18:44.931368+0000","last_unstale":"2023-10-21T17:18:45.941178+0000","last_undegraded":"2023-10-21T17:18:45.941178+0000","last_fullsized":"2023-10-21T17:18:45.941178+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:10:58.330034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967832+0000","last_change":"2023-10-21T17:18:45.965446+0000","last_active":"2023-10-21T17:18:45.967832+0000","last_peered":"2023-10-21T17:18:45.967832+0000","last_clean":"2023-10-21T17:18:45.967832+0000","last_became_active":"2023-10-21T17:18:45.964965+0000","last_became_peered":"2023-10-21T17:18:45.964965+0000","last_unstale":"2023-10-21T17:18:45.967832+0000","last_undegraded":"2023-10-21T17:18:45.967832+0000","last_fullsized":"2023-10-21T17:18:45.967832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:37.753186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.938548+0000","last_change":"2023-10-21T17:18:44.932809+0000","last_active":"2023-10-21T17:18:45.938548+0000","last_peered":"2023-10-21T17:18:45.938548+0000","last_clean":"2023-10-21T17:18:45.938548+0000","last_became_active":"2023-10-21T17:18:44.932520+0000","last_became_peered":"2023-10-21T17:18:44.932520+0000","last_unstale":"2023-10-21T17:18:45.938548+0000","last_undegraded":"2023-10-21T17:18:45.938548+0000","last_fullsized":"2023-10-21T17:18:45.938548+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:11.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959322+0000","last_change":"2023-10-21T17:18:45.951919+0000","last_active":"2023-10-21T17:18:46.959322+0000","last_peered":"2023-10-21T17:18:46.959322+0000","last_clean":"2023-10-21T17:18:46.959322+0000","last_became_active":"2023-10-21T17:18:45.949844+0000","last_became_peered":"2023-10-21T17:18:45.949844+0000","last_unstale":"2023-10-21T17:18:46.959322+0000","last_undegraded":"2023-10-21T17:18:46.959322+0000","last_fullsized":"2023-10-21T17:18:46.959322+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:02.581645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955387+0000","last_change":"2023-10-21T17:18:44.930158+0000","last_active":"2023-10-21T17:18:46.955387+0000","last_peered":"2023-10-21T17:18:46.955387+0000","last_clean":"2023-10-21T17:18:46.955387+0000","last_became_active":"2023-10-21T17:18:44.929871+0000","last_became_peered":"2023-10-21T17:18:44.929871+0000","last_unstale":"2023-10-21T17:18:46.955387+0000","last_undegraded":"2023-10-21T17:18:46.955387+0000","last_fullsized":"2023-10-21T17:18:46.955387+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:20.533421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.271953+0000","last_change":"2023-10-21T17:18:45.957045+0000","last_active":"2023-10-21T17:18:47.271953+0000","last_peered":"2023-10-21T17:18:47.271953+0000","last_clean":"2023-10-21T17:18:47.271953+0000","last_became_active":"2023-10-21T17:18:45.956757+0000","last_became_peered":"2023-10-21T17:18:45.956757+0000","last_unstale":"2023-10-21T17:18:47.271953+0000","last_undegraded":"2023-10-21T17:18:47.271953+0000","last_fullsized":"2023-10-21T17:18:47.271953+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:18.337103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956880+0000","last_change":"2023-10-21T17:18:44.910769+0000","last_active":"2023-10-21T17:18:46.956880+0000","last_peered":"2023-10-21T17:18:46.956880+0000","last_clean":"2023-10-21T17:18:46.956880+0000","last_became_active":"2023-10-21T17:18:44.909085+0000","last_became_peered":"2023-10-21T17:18:44.909085+0000","last_unstale":"2023-10-21T17:18:46.956880+0000","last_undegraded":"2023-10-21T17:18:46.956880+0000","last_fullsized":"2023-10-21T17:18:46.956880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:19.299591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.958711+0000","last_change":"2023-10-21T17:18:45.953642+0000","last_active":"2023-10-21T17:18:45.958711+0000","last_peered":"2023-10-21T17:18:45.958711+0000","last_clean":"2023-10-21T17:18:45.958711+0000","last_became_active":"2023-10-21T17:18:45.953268+0000","last_became_peered":"2023-10-21T17:18:45.953268+0000","last_unstale":"2023-10-21T17:18:45.958711+0000","last_undegraded":"2023-10-21T17:18:45.958711+0000","last_fullsized":"2023-10-21T17:18:45.958711+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:52:44.780229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954617+0000","last_change":"2023-10-21T17:18:44.925780+0000","last_active":"2023-10-21T17:18:46.954617+0000","last_peered":"2023-10-21T17:18:46.954617+0000","last_clean":"2023-10-21T17:18:46.954617+0000","last_became_active":"2023-10-21T17:18:44.925412+0000","last_became_peered":"2023-10-21T17:18:44.925412+0000","last_unstale":"2023-10-21T17:18:46.954617+0000","last_undegraded":"2023-10-21T17:18:46.954617+0000","last_fullsized":"2023-10-21T17:18:46.954617+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:47.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958892+0000","last_change":"2023-10-21T17:18:45.955163+0000","last_active":"2023-10-21T17:18:46.958892+0000","last_peered":"2023-10-21T17:18:46.958892+0000","last_clean":"2023-10-21T17:18:46.958892+0000","last_became_active":"2023-10-21T17:18:45.954902+0000","last_became_peered":"2023-10-21T17:18:45.954902+0000","last_unstale":"2023-10-21T17:18:46.958892+0000","last_undegraded":"2023-10-21T17:18:46.958892+0000","last_fullsized":"2023-10-21T17:18:46.958892+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:54.022901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.941954+0000","last_change":"2023-10-21T17:18:44.928885+0000","last_active":"2023-10-21T17:18:45.941954+0000","last_peered":"2023-10-21T17:18:45.941954+0000","last_clean":"2023-10-21T17:18:45.941954+0000","last_became_active":"2023-10-21T17:18:44.926700+0000","last_became_peered":"2023-10-21T17:18:44.926700+0000","last_unstale":"2023-10-21T17:18:45.941954+0000","last_undegraded":"2023-10-21T17:18:45.941954+0000","last_fullsized":"2023-10-21T17:18:45.941954+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959561+0000","last_change":"2023-10-21T17:18:45.951064+0000","last_active":"2023-10-21T17:18:46.959561+0000","last_peered":"2023-10-21T17:18:46.959561+0000","last_clean":"2023-10-21T17:18:46.959561+0000","last_became_active":"2023-10-21T17:18:45.950732+0000","last_became_peered":"2023-10-21T17:18:45.950732+0000","last_unstale":"2023-10-21T17:18:46.959561+0000","last_undegraded":"2023-10-21T17:18:46.959561+0000","last_fullsized":"2023-10-21T17:18:46.959561+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:44:24.131908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"183'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446802+0000","last_change":"2023-10-21T17:18:44.920467+0000","last_active":"2023-10-21T17:18:47.446802+0000","last_peered":"2023-10-21T17:18:47.446802+0000","last_clean":"2023-10-21T17:18:47.446802+0000","last_became_active":"2023-10-21T17:18:44.920041+0000","last_became_peered":"2023-10-21T17:18:44.920041+0000","last_unstale":"2023-10-21T17:18:47.446802+0000","last_undegraded":"2023-10-21T17:18:47.446802+0000","last_fullsized":"2023-10-21T17:18:47.446802+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:21.953357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"171'4796","reported_seq":5476,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-21T17:18:44.920980+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:18:44.920980+0000","last_peered":"2023-10-21T17:18:44.920980+0000","last_clean":"2023-10-21T17:18:44.920980+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:18:44.920980+0000","last_undegraded":"2023-10-21T17:18:44.920980+0000","last_fullsized":"2023-10-21T17:18:44.920980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":447,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":447,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4796,"ondisk_log_size":4796,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52048,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319792,"statfs":{"total":96636764160,"available":96583467008,"internally_reserved":0,"allocated":2883584,"data_stored":1762797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.84099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5920000000000001,"5min":1.841,"15min":1.9430000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.65900000000000003},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2130000000000001,"15min":1.2370000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.68300000000000005},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.448},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.272,"15min":1.276},"min":{"1min":0.93700000000000006,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.839,"5min":1.839,"15min":2.081},"last":0.97699999999999998}]},{"osd":2,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3440000000000001,"15min":1.349},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.589,"5min":1.8140000000000001,"15min":2.1440000000000001},"last":1.304},{"interface":"front","average":{"1min":1.335,"5min":1.357,"15min":1.365},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88300000000000001},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.1579999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.304,"15min":1.306},"min":{"1min":1.0840000000000001,"5min":0.94999999999999996,"15min":0.81699999999999995},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.216},{"interface":"front","average":{"1min":1.466,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":0.95199999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.585}]},{"osd":4,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.755},"max":{"1min":1.6639999999999999,"5min":1.865,"15min":2.1269999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4650000000000001,"15min":1.484},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85899999999999999},"max":{"1min":1.7769999999999999,"5min":1.8540000000000001,"15min":2.0830000000000002},"last":1.5469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.80100000000000005},"max":{"1min":2.089,"5min":2.089,"15min":2.1560000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.52,"15min":1.5049999999999999},"min":{"1min":1.1439999999999999,"5min":0.93999999999999995,"15min":0.91900000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.369}]},{"osd":6,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5940000000000001,"15min":1.5920000000000001},"min":{"1min":1.2090000000000001,"5min":0.996,"15min":0.92400000000000004},"max":{"1min":1.9610000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.506},{"interface":"front","average":{"1min":1.446,"5min":1.4890000000000001,"15min":1.522},"min":{"1min":1.0680000000000001,"5min":1,"15min":0.92900000000000005},"max":{"1min":1.895,"5min":2.0150000000000001,"15min":2.1970000000000001},"last":1.631}]}]},{"osd":6,"up_from":13,"seq":55834575766,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41736,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330104,"statfs":{"total":96636764160,"available":96594026496,"internally_reserved":0,"allocated":2289664,"data_stored":1172477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.254,"15min":1.21},"min":{"1min":0.89600000000000002,"5min":0.82199999999999995,"15min":0.78400000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.827},"last":1.169},{"interface":"front","average":{"1min":1.177,"5min":1.2350000000000001,"15min":1.262},"min":{"1min":0.82099999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.8680000000000001},"last":1.073}]},{"osd":1,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.274,"15min":1.2549999999999999},"min":{"1min":0.98999999999999999,"5min":0.85299999999999998,"15min":0.80200000000000005},"max":{"1min":1.677,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.748},{"interface":"front","average":{"1min":1.302,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.90900000000000003,"15min":0.86299999999999999},"max":{"1min":1.671,"5min":1.9359999999999999,"15min":1.984},"last":0.97099999999999997}]},{"osd":2,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3149999999999999,"15min":1.3180000000000001},"min":{"1min":0.90700000000000003,"5min":0.878,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":1.9970000000000001,"15min":2.0659999999999998},"last":1.8819999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3320000000000001,"15min":1.3140000000000001},"min":{"1min":1.006,"5min":0.92100000000000004,"15min":0.70999999999999996},"max":{"1min":1.6040000000000001,"5min":1.7969999999999999,"15min":2.415},"last":1.595}]},{"osd":3,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.421,"15min":1.407},"min":{"1min":1.0509999999999999,"5min":1.034,"15min":0.81799999999999995},"max":{"1min":1.7709999999999999,"5min":1.9039999999999999,"15min":2.0150000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.4730000000000001,"15min":1.474},"min":{"1min":1.133,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":2.6509999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.369},"min":{"1min":0.96399999999999997,"5min":0.95099999999999996,"15min":0.63500000000000001},"max":{"1min":1.7,"5min":1.839,"15min":1.956},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4339999999999999,"15min":1.4139999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89100000000000001},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.161},"last":1.458}]},{"osd":5,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5880000000000001,"15min":1.5509999999999999},"min":{"1min":1.26,"5min":1.173,"15min":0.89600000000000002},"max":{"1min":1.998,"5min":2.1019999999999999,"15min":2.5419999999999998},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4259999999999999,"15min":1.387},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90400000000000003},"max":{"1min":1.8360000000000001,"5min":1.948,"15min":2.016},"last":1.9830000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.5309999999999999,"15min":1.532},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.71099999999999997},"max":{"1min":1.8999999999999999,"5min":2.1059999999999999,"15min":2.4049999999999998},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.746,"5min":1.681,"15min":1.6519999999999999},"min":{"1min":1.3819999999999999,"5min":1.181,"15min":1.0760000000000001},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":3.4180000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2285568,"data_stored":1172407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.157},"min":{"1min":0.73499999999999999,"5min":0.69199999999999995,"15min":0.625},"max":{"1min":1.718,"5min":1.718,"15min":1.7649999999999999},"last":1.27},{"interface":"front","average":{"1min":1.28,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.86299999999999999,"5min":0.79200000000000004,"15min":0.73999999999999999},"max":{"1min":1.661,"5min":1.726,"15min":1.9079999999999999},"last":1.2909999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2849999999999999,"15min":1.304},"min":{"1min":0.94999999999999996,"5min":0.876,"15min":0.85299999999999998},"max":{"1min":1.6679999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.268,"15min":1.2549999999999999},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.357}]},{"osd":2,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.351,"15min":1.3400000000000001},"min":{"1min":0.98899999999999999,"5min":0.93999999999999995,"15min":0.90300000000000002},"max":{"1min":1.6970000000000001,"5min":1.885,"15min":1.927},"last":1.645},{"interface":"front","average":{"1min":1.292,"5min":1.278,"15min":1.282},"min":{"1min":1.02,"5min":0.81499999999999995,"15min":0.76000000000000001},"max":{"1min":1.6100000000000001,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.95899999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.367,"15min":1.3819999999999999},"min":{"1min":1.0920000000000001,"5min":1.008,"15min":0.84999999999999998},"max":{"1min":1.7390000000000001,"5min":1.7529999999999999,"15min":1.948},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.387,"15min":1.4239999999999999},"min":{"1min":1.0569999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.635,"5min":1.903,"15min":1.903},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2130000000000001,"15min":1.2390000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.718,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.379},"min":{"1min":0.93300000000000005,"5min":0.77300000000000002,"15min":0.70199999999999996},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.548}]},{"osd":6,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.454,"15min":1.478},"min":{"1min":0.92300000000000004,"5min":0.90000000000000002,"15min":0.89600000000000002},"max":{"1min":1.75,"5min":2.234,"15min":2.234},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.504},"min":{"1min":1.097,"5min":0.74199999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.8100000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.526,"15min":1.548},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":0.79000000000000004},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1549999999999998},"last":1.615},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.548,"15min":1.5609999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.012,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.869}]}]},{"osd":4,"up_from":13,"seq":55834575765,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45912,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325928,"statfs":{"total":96636764160,"available":96589750272,"internally_reserved":0,"allocated":2297856,"data_stored":1174019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2230000000000001,"15min":1.2},"min":{"1min":0.85299999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.212,"15min":1.2310000000000001},"min":{"1min":0.88800000000000001,"5min":0.85699999999999998,"15min":0.78600000000000003},"max":{"1min":1.633,"5min":1.907,"15min":1.907},"last":0.94699999999999995}]},{"osd":1,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3049999999999999,"15min":1.282},"min":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.81399999999999995},"max":{"1min":1.5880000000000001,"5min":1.6859999999999999,"15min":1.855},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2709999999999999,"15min":1.268},"min":{"1min":0.89700000000000002,"5min":0.88100000000000001,"15min":0.81499999999999995},"max":{"1min":1.587,"5min":1.794,"15min":1.9139999999999999},"last":1.264}]},{"osd":2,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.252,"15min":1.2589999999999999},"min":{"1min":0.93100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.821},"last":1.153},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.403,"15min":1.3879999999999999},"min":{"1min":1.181,"5min":0.96499999999999997,"15min":0.95299999999999996},"max":{"1min":1.6319999999999999,"5min":1.923,"15min":1.923},"last":1.0660000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.353},"min":{"1min":0.999,"5min":0.96699999999999997,"15min":0.82799999999999996},"max":{"1min":1.7350000000000001,"5min":1.877,"15min":1.877},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.425,"15min":1.4119999999999999},"min":{"1min":1.0580000000000001,"5min":0.94499999999999995,"15min":0.89100000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.9299999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3160000000000001,"15min":1.3100000000000001},"min":{"1min":0.98099999999999998,"5min":0.88400000000000001,"15min":0.83199999999999996},"max":{"1min":1.611,"5min":1.802,"15min":1.895},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.56699999999999995},"max":{"1min":1.895,"5min":1.944,"15min":1.9870000000000001},"last":1.5649999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.429,"15min":1.4139999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9399999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.327},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6100000000000001,"15min":1.5900000000000001},"min":{"1min":1.4119999999999999,"5min":1.294,"15min":0.85599999999999998},"max":{"1min":1.871,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.367}]},{"osd":7,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5589999999999999,"15min":1.522},"min":{"1min":1.23,"5min":1.008,"15min":0.96599999999999997},"max":{"1min":2.0990000000000002,"5min":2.1110000000000002,"15min":2.2429999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.609,"15min":1.575},"min":{"1min":1.069,"5min":1.069,"15min":1.0109999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.417}]}]},{"osd":3,"up_from":13,"seq":55834575765,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2293760,"data_stored":1173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.72099999999999997,"5min":0.61799999999999999,"15min":0.47299999999999998},"max":{"1min":1.3300000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2949999999999999,"15min":1.288},"min":{"1min":1.0129999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.766,"5min":1.77,"15min":2.121},"last":1.071}]},{"osd":1,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.141,"15min":1.0960000000000001},"min":{"1min":0.78400000000000003,"5min":0.755,"15min":0.51600000000000001},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":1.032,"5min":0.79800000000000004,"15min":0.70399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.5129999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.145,"15min":1.125},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.64500000000000002},"max":{"1min":1.726,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3009999999999999,"15min":1.2949999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.73499999999999999},"max":{"1min":1.6990000000000001,"5min":2.173,"15min":2.173},"last":1.3720000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":1.081,"5min":1.0509999999999999,"15min":0.73499999999999999},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.331},{"interface":"front","average":{"1min":1.542,"5min":1.508,"15min":1.4850000000000001},"min":{"1min":1.242,"5min":1.1870000000000001,"15min":0.98299999999999998},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":2.0009999999999999},"last":1.552}]},{"osd":5,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":1.131,"5min":1.0409999999999999,"15min":0.79400000000000004},"max":{"1min":2.0150000000000001,"5min":2.2210000000000001,"15min":2.3239999999999998},"last":1.599},{"interface":"front","average":{"1min":1.575,"5min":1.5569999999999999,"15min":1.5609999999999999},"min":{"1min":1.125,"5min":1.0780000000000001,"15min":1.054},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.2450000000000001},"last":1.462}]},{"osd":6,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.6200000000000001,"15min":1.6080000000000001},"min":{"1min":1.2230000000000001,"5min":1.0940000000000001,"15min":0.876},"max":{"1min":2.2210000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.663},{"interface":"front","average":{"1min":1.625,"5min":1.5589999999999999,"15min":1.587},"min":{"1min":1.157,"5min":1.1060000000000001,"15min":0.88500000000000001},"max":{"1min":2.0219999999999998,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.833}]},{"osd":7,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.6699999999999999,"15min":1.6719999999999999},"min":{"1min":1.323,"5min":1.21,"15min":0.97399999999999998},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.4239999999999999},"last":1.627},{"interface":"front","average":{"1min":1.7969999999999999,"5min":1.7430000000000001,"15min":1.722},"min":{"1min":1.272,"5min":1.2629999999999999,"15min":0.93200000000000005},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.4729999999999999},"last":1.8009999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575766,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51480,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320360,"statfs":{"total":96636764160,"available":96584048640,"internally_reserved":0,"allocated":2306048,"data_stored":1179255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88,"15min":0.89700000000000002},"min":{"1min":0.69499999999999995,"5min":0.66000000000000003,"15min":0.65700000000000003},"max":{"1min":1.226,"5min":1.8859999999999999,"15min":2.3199999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.077,"15min":1.0600000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.72299999999999998},"max":{"1min":1.8180000000000001,"5min":1.835,"15min":1.875},"last":1.45}]},{"osd":1,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.157,"15min":1.1830000000000001},"min":{"1min":0.89300000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.9650000000000001},"last":1.377},{"interface":"front","average":{"1min":1.353,"5min":1.3939999999999999,"15min":1.4039999999999999},"min":{"1min":1.085,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1070000000000002},"last":1.143}]},{"osd":3,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2230000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"max":{"1min":1.825,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.997},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.492,"15min":1.492},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.891,"5min":1.9450000000000001,"15min":2.1789999999999998},"last":1.52}]},{"osd":4,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.419,"15min":1.407},"min":{"1min":1.167,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.768,"5min":1.992,"15min":3.0510000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":1.0620000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.7230000000000001,"5min":2.0489999999999999,"15min":2.0579999999999998},"last":1.6379999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.446,"15min":1.417},"min":{"1min":1.0549999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8069999999999999,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.196},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.599,"15min":1.6140000000000001},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":1.843,"5min":1.9590000000000001,"15min":3.2050000000000001},"last":1.5960000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.0190000000000001,"5min":2.21,"15min":2.25},"last":1.76},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":1.1559999999999999,"5min":1.02,"15min":1.008},"max":{"1min":1.9930000000000001,"5min":2.4470000000000001,"15min":2.7389999999999999},"last":1.5620000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.6459999999999999,"15min":1.6479999999999999},"min":{"1min":1.2569999999999999,"5min":1.107,"15min":1.107},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.8959999999999999},"last":1.8440000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.605,"15min":1.603},"min":{"1min":1.264,"5min":1.085,"15min":1.085},"max":{"1min":1.9259999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.6930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575765,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29260,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342580,"statfs":{"total":96636764160,"available":96606801920,"internally_reserved":0,"allocated":2289664,"data_stored":1172477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95999999999999996,"15min":0.95899999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.427,"5min":1.593,"15min":1.7330000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.3999999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.042,"15min":1.0569999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.724,"5min":1.724,"15min":2.79},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.202,"5min":1.298,"15min":1.2490000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.806,"5min":1.891,"15min":2.2549999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.387,"15min":1.395},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6870000000000001,"5min":1.9339999999999999,"15min":3.0920000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.252,"15min":1.282},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.627,"5min":1.75,"15min":1.976},"last":1.5649999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.365,"15min":1.3169999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.849,"5min":2.105,"15min":2.105},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5209999999999999,"15min":1.5089999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.966,"5min":2.2410000000000001,"15min":2.512},"last":1.591}]},{"osd":5,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.536,"15min":1.542},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.867},{"interface":"front","average":{"1min":1.532,"5min":1.5760000000000001,"15min":1.5489999999999999},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.958,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.532}]},{"osd":6,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5840000000000001,"15min":1.5580000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.9690000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.766},{"interface":"front","average":{"1min":1.544,"5min":1.597,"15min":1.5489999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0489999999999999,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.732}]},{"osd":7,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.6950000000000001,"15min":1.681},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.0979999999999999,"5min":2.3220000000000001,"15min":2.391},"last":1.821},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.7130000000000001,"15min":1.6910000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.129,"5min":2.4769999999999999,"15min":2.4969999999999999},"last":1.954}]}]},{"osd":0,"up_from":13,"seq":55834575765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29856,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341984,"statfs":{"total":96636764160,"available":96606191616,"internally_reserved":0,"allocated":2904064,"data_stored":1765093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96799999999999997,"15min":0.95299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5840000000000001,"5min":1.631,"15min":1.726},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.7989999999999999,"15min":2.6200000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1679999999999999,"15min":1.171},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.7330000000000001,"15min":2.8140000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.204,"15min":1.2070000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.46,"5min":1.7170000000000001,"15min":1.784},"last":1.3089999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.1060000000000001,"15min":1.147},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.718,"5min":1.883,"15min":1.8919999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6539999999999999,"5min":1.9239999999999999,"15min":2.6949999999999998},"last":1.4730000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.395},{"interface":"front","average":{"1min":1.637,"5min":1.6299999999999999,"15min":1.619},"min":{"1min":1.278,"5min":1.278,"15min":1.1799999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.766},"last":1.754}]},{"osd":5,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.96199999999999997},"max":{"1min":1.7729999999999999,"5min":2.125,"15min":2.6549999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5660000000000001,"15min":1.5600000000000001},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.952,"5min":2.3300000000000001,"15min":2.9209999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5229999999999999,"15min":1.5129999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2759999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.6990000000000001,"15min":1.665},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0230000000000001,"5min":2.4510000000000001,"15min":3.069},"last":2.145}]},{"osd":7,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.556,"15min":1.556},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.794,"5min":2.0910000000000002,"15min":2.4790000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.639,"15min":1.631},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.0529999999999999,"5min":2.1930000000000001,"15min":2.613},"last":1.607}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:18:50.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:18:50.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.734+0000 7f761f4ad640 1 Processor -- start 2023-10-21T17:18:50.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.734+0000 7f761f4ad640 1 -- start start 2023-10-21T17:18:50.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.735+0000 7f761f4ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f761815ab30 0x7f761815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.735+0000 7f761f4ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76180b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761f4ad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761815f7d0 con 0x7f761815af10 2023-10-21T17:18:50.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761f4ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761815f210 con 0x7f761815ab30 2023-10-21T17:18:50.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761f4ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761815f4f0 con 0x7f76180adc70 2023-10-21T17:18:50.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761ecac640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f761815af10 0x7f76180701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50044/0 (socket says 172.21.15.53:50044) 2023-10-21T17:18:50.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761ecac640 1 -- 172.21.15.53:0/2448003703 learned_addr learned my addr 172.21.15.53:0/2448003703 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:50.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761dcaa640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76180b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.736+0000 7f761e4ab640 1 --2- 172.21.15.53:0/2448003703 >> v2:172.21.15.124:3300/0 conn(0x7f761815ab30 0x7f761815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087145629 0 0) 0x7f761815f7d0 con 0x7f761815af10 2023-10-21T17:18:50.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7604003520 con 0x7f761815af10 2023-10-21T17:18:50.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448847902 0 0) 0x7f7604003520 con 0x7f761815af10 2023-10-21T17:18:50.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 msgr2=0x7f76180b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76180b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 >> v2:172.21.15.124:3300/0 conn(0x7f761815ab30 msgr2=0x7f761815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 --2- 172.21.15.53:0/2448003703 >> v2:172.21.15.124:3300/0 conn(0x7f761815ab30 0x7f761815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761806c550 con 0x7f761815af10 2023-10-21T17:18:50.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7610001a10 con 0x7f761815af10 2023-10-21T17:18:50.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 >> v1:172.21.15.53:6789/0 conn(0x7f761815af10 legacy=0x7f76180701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7604003970 0x7f7604003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7604004d00 0x7f7604005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815f210 con 0x7f7604003970 2023-10-21T17:18:50.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815f4f0 con 0x7f7604004d00 2023-10-21T17:18:50.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760400b980 con 0x7f76180adc70 2023-10-21T17:18:50.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.737+0000 7f761e4ab640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7604003970 0x7f7604003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761ecac640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7604004d00 0x7f7604005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761dcaa640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761e4ab640 1 -- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 msgr2=0x7f76040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761e4ab640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76180adc70 0x7f76040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761e4ab640 1 -- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7604004d00 msgr2=0x7f7604005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:50.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761e4ab640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7604004d00 0x7f7604005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.738+0000 7f761e4ab640 1 -- 172.21.15.53:0/2448003703 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604003520 con 0x7f7604003970 2023-10-21T17:18:50.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.739+0000 7f761e4ab640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7604003970 0x7f7604003d90 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7614009af0 tx=0x7f76040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae353bff993cf82 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.739+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761402c710 con 0x7f7604003970 2023-10-21T17:18:50.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.739+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761402c870 con 0x7f7604003970 2023-10-21T17:18:50.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.739+0000 7f761d4a9640 1 -- 172.21.15.53:0/2448003703 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761402cc00 con 0x7f7604003970 2023-10-21T17:18:50.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 -- 172.21.15.53:0/2448003703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7604003970 msgr2=0x7f7604003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 --2- 172.21.15.53:0/2448003703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7604003970 0x7f7604003d90 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f7614009af0 tx=0x7f76040057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761ecac640 1 -- 172.21.15.53:0/2448003703 reap_dead start 2023-10-21T17:18:50.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 -- 172.21.15.53:0/2448003703 shutdown_connections 2023-10-21T17:18:50.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 -- 172.21.15.53:0/2448003703 >> 172.21.15.53:0/2448003703 conn(0x7f76180cb0c0 msgr2=0x7f76180cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:50.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 -- 172.21.15.53:0/2448003703 shutdown_connections 2023-10-21T17:18:50.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.740+0000 7f761f4ad640 1 -- 172.21.15.53:0/2448003703 wait complete. 2023-10-21T17:18:50.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.742+0000 7f761f4ad640 1 Processor -- start 2023-10-21T17:18:50.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.742+0000 7f761f4ad640 1 -- start start 2023-10-21T17:18:50.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 0x7f7618155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761815b380 0x7f76180c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815c570 con 0x7f76180c27a0 2023-10-21T17:18:50.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761815b8f0 con 0x7f761815b380 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.743+0000 7f761f4ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761815bbd0 con 0x7f7604003970 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42426/0 (socket says 172.21.15.53:42426) 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 -- 172.21.15.53:0/65631028 learned_addr learned my addr 172.21.15.53:0/65631028 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761e4ab640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 0x7f7618155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761dcaa640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761815b380 0x7f76180c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 -- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 msgr2=0x7f7618155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 0x7f7618155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 -- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761815b380 msgr2=0x7f76180c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761815b380 0x7f76180c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 -- 172.21.15.53:0/65631028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761815beb0 con 0x7f76180c27a0 2023-10-21T17:18:50.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761e4ab640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 0x7f7618155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:50.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.744+0000 7f761ecac640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7610000f80 tx=0x7f761000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.745+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761000de70 con 0x7f76180c27a0 2023-10-21T17:18:50.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.745+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761000bb80 con 0x7f76180c27a0 2023-10-21T17:18:50.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.745+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761815c190 con 0x7f76180c27a0 2023-10-21T17:18:50.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.745+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610007070 con 0x7f76180c27a0 2023-10-21T17:18:50.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.745+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76180c66f0 con 0x7f76180c27a0 2023-10-21T17:18:50.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.746+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761815c6b0 con 0x7f76180c27a0 2023-10-21T17:18:50.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.747+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f76100071d0 con 0x7f76180c27a0 2023-10-21T17:18:50.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.748+0000 7f760affd640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 0x7f75fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:50.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.748+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7610089e70 con 0x7f76180c27a0 2023-10-21T17:18:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.749+0000 7f761e4ab640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 0x7f75fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.750+0000 7f761e4ab640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 0x7f75fc06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7614036000 tx=0x7f7614030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:50.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.752+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f761815c6b0 con 0x7f76180c27a0 2023-10-21T17:18:50.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.944+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 --> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f761806c830 con 0x7f75fc06af90 2023-10-21T17:18:50.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.968+0000 7f760affd640 1 -- 172.21.15.53:0/65631028 <== mgr.36914 v2:172.21.15.124:6832/234236878 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286696 (secure 0 0 0) 0x7f761402cae0 con 0x7f75fc06af90 2023-10-21T17:18:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:50.975 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:18:50.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 msgr2=0x7f75fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 0x7f75fc06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7614036000 tx=0x7f7614030000 comp rx=0 tx=0).stop 2023-10-21T17:18:50.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 msgr2=0x7f76180afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:50.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7610000f80 tx=0x7f761000b4a0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 shutdown_connections 2023-10-21T17:18:50.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f75fc06af90 0x7f75fc06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76180c27a0 0x7f76180afdd0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761815b380 0x7f76180c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 --2- 172.21.15.53:0/65631028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7604003970 0x7f7618155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:50.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.973+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 >> 172.21.15.53:0/65631028 conn(0x7f76180cb0c0 msgr2=0x7f76181589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:50.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.974+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 shutdown_connections 2023-10-21T17:18:50.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:50.974+0000 7f761f4ad640 1 -- 172.21.15.53:0/65631028 wait complete. 2023-10-21T17:18:50.993 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-21T17:18:50.472298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":491,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4834,"ondisk_log_size":4834,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361548,"kb_used_data":19092,"kb_used_omap":12,"kb_used_meta":342323,"kb_avail":754613172,"statfs":{"total":773094113280,"available":772723888128,"internally_reserved":0,"allocated":19550208,"data_stored":10571548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":350539321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002048"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956064+0000","last_change":"2023-10-21T17:18:45.946472+0000","last_active":"2023-10-21T17:18:46.956064+0000","last_peered":"2023-10-21T17:18:46.956064+0000","last_clean":"2023-10-21T17:18:46.956064+0000","last_became_active":"2023-10-21T17:18:45.945911+0000","last_became_peered":"2023-10-21T17:18:45.945911+0000","last_unstale":"2023-10-21T17:18:46.956064+0000","last_undegraded":"2023-10-21T17:18:46.956064+0000","last_fullsized":"2023-10-21T17:18:46.956064+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:16.311690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.968511+0000","last_change":"2023-10-21T17:18:45.966495+0000","last_active":"2023-10-21T17:18:45.968511+0000","last_peered":"2023-10-21T17:18:45.968511+0000","last_clean":"2023-10-21T17:18:45.968511+0000","last_became_active":"2023-10-21T17:18:45.966200+0000","last_became_peered":"2023-10-21T17:18:45.966200+0000","last_unstale":"2023-10-21T17:18:45.968511+0000","last_undegraded":"2023-10-21T17:18:45.968511+0000","last_fullsized":"2023-10-21T17:18:45.968511+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:49:17.231532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961003+0000","last_change":"2023-10-21T17:18:45.958769+0000","last_active":"2023-10-21T17:18:45.961003+0000","last_peered":"2023-10-21T17:18:45.961003+0000","last_clean":"2023-10-21T17:18:45.961003+0000","last_became_active":"2023-10-21T17:18:45.958284+0000","last_became_peered":"2023-10-21T17:18:45.958284+0000","last_unstale":"2023-10-21T17:18:45.961003+0000","last_undegraded":"2023-10-21T17:18:45.961003+0000","last_fullsized":"2023-10-21T17:18:45.961003+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:18.826366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272106+0000","last_change":"2023-10-21T17:18:45.957072+0000","last_active":"2023-10-21T17:18:47.272106+0000","last_peered":"2023-10-21T17:18:47.272106+0000","last_clean":"2023-10-21T17:18:47.272106+0000","last_became_active":"2023-10-21T17:18:45.955774+0000","last_became_peered":"2023-10-21T17:18:45.955774+0000","last_unstale":"2023-10-21T17:18:47.272106+0000","last_undegraded":"2023-10-21T17:18:47.272106+0000","last_fullsized":"2023-10-21T17:18:47.272106+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:10:29.091615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961057+0000","last_change":"2023-10-21T17:18:45.956112+0000","last_active":"2023-10-21T17:18:45.961057+0000","last_peered":"2023-10-21T17:18:45.961057+0000","last_clean":"2023-10-21T17:18:45.961057+0000","last_became_active":"2023-10-21T17:18:45.955836+0000","last_became_peered":"2023-10-21T17:18:45.955836+0000","last_unstale":"2023-10-21T17:18:45.961057+0000","last_undegraded":"2023-10-21T17:18:45.961057+0000","last_fullsized":"2023-10-21T17:18:45.961057+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:54:46.352514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960636+0000","last_change":"2023-10-21T17:18:45.955850+0000","last_active":"2023-10-21T17:18:45.960636+0000","last_peered":"2023-10-21T17:18:45.960636+0000","last_clean":"2023-10-21T17:18:45.960636+0000","last_became_active":"2023-10-21T17:18:45.955242+0000","last_became_peered":"2023-10-21T17:18:45.955242+0000","last_unstale":"2023-10-21T17:18:45.960636+0000","last_undegraded":"2023-10-21T17:18:45.960636+0000","last_fullsized":"2023-10-21T17:18:45.960636+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:40.885115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954495+0000","last_change":"2023-10-21T17:18:45.953767+0000","last_active":"2023-10-21T17:18:46.954495+0000","last_peered":"2023-10-21T17:18:46.954495+0000","last_clean":"2023-10-21T17:18:46.954495+0000","last_became_active":"2023-10-21T17:18:45.953494+0000","last_became_peered":"2023-10-21T17:18:45.953494+0000","last_unstale":"2023-10-21T17:18:46.954495+0000","last_undegraded":"2023-10-21T17:18:46.954495+0000","last_fullsized":"2023-10-21T17:18:46.954495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:33.869515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958554+0000","last_change":"2023-10-21T17:18:45.952114+0000","last_active":"2023-10-21T17:18:46.958554+0000","last_peered":"2023-10-21T17:18:46.958554+0000","last_clean":"2023-10-21T17:18:46.958554+0000","last_became_active":"2023-10-21T17:18:45.951623+0000","last_became_peered":"2023-10-21T17:18:45.951623+0000","last_unstale":"2023-10-21T17:18:46.958554+0000","last_undegraded":"2023-10-21T17:18:46.958554+0000","last_fullsized":"2023-10-21T17:18:46.958554+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:37:35.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966326+0000","last_change":"2023-10-21T17:18:45.962523+0000","last_active":"2023-10-21T17:18:45.966326+0000","last_peered":"2023-10-21T17:18:45.966326+0000","last_clean":"2023-10-21T17:18:45.966326+0000","last_became_active":"2023-10-21T17:18:45.962185+0000","last_became_peered":"2023-10-21T17:18:45.962185+0000","last_unstale":"2023-10-21T17:18:45.966326+0000","last_undegraded":"2023-10-21T17:18:45.966326+0000","last_fullsized":"2023-10-21T17:18:45.966326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:28.037660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.955261+0000","last_change":"2023-10-21T17:18:45.946981+0000","last_active":"2023-10-21T17:18:45.955261+0000","last_peered":"2023-10-21T17:18:45.955261+0000","last_clean":"2023-10-21T17:18:45.955261+0000","last_became_active":"2023-10-21T17:18:45.946698+0000","last_became_peered":"2023-10-21T17:18:45.946698+0000","last_unstale":"2023-10-21T17:18:45.955261+0000","last_undegraded":"2023-10-21T17:18:45.955261+0000","last_fullsized":"2023-10-21T17:18:45.955261+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:24:54.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956876+0000","last_change":"2023-10-21T17:18:45.952692+0000","last_active":"2023-10-21T17:18:46.956876+0000","last_peered":"2023-10-21T17:18:46.956876+0000","last_clean":"2023-10-21T17:18:46.956876+0000","last_became_active":"2023-10-21T17:18:45.951479+0000","last_became_peered":"2023-10-21T17:18:45.951479+0000","last_unstale":"2023-10-21T17:18:46.956876+0000","last_undegraded":"2023-10-21T17:18:46.956876+0000","last_fullsized":"2023-10-21T17:18:46.956876+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:09:58.219012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272560+0000","last_change":"2023-10-21T17:18:45.958253+0000","last_active":"2023-10-21T17:18:47.272560+0000","last_peered":"2023-10-21T17:18:47.272560+0000","last_clean":"2023-10-21T17:18:47.272560+0000","last_became_active":"2023-10-21T17:18:45.956090+0000","last_became_peered":"2023-10-21T17:18:45.956090+0000","last_unstale":"2023-10-21T17:18:47.272560+0000","last_undegraded":"2023-10-21T17:18:47.272560+0000","last_fullsized":"2023-10-21T17:18:47.272560+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:32.751413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966913+0000","last_change":"2023-10-21T17:18:45.963839+0000","last_active":"2023-10-21T17:18:45.966913+0000","last_peered":"2023-10-21T17:18:45.966913+0000","last_clean":"2023-10-21T17:18:45.966913+0000","last_became_active":"2023-10-21T17:18:45.962390+0000","last_became_peered":"2023-10-21T17:18:45.962390+0000","last_unstale":"2023-10-21T17:18:45.966913+0000","last_undegraded":"2023-10-21T17:18:45.966913+0000","last_fullsized":"2023-10-21T17:18:45.966913+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:04.692255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272052+0000","last_change":"2023-10-21T17:18:45.960201+0000","last_active":"2023-10-21T17:18:47.272052+0000","last_peered":"2023-10-21T17:18:47.272052+0000","last_clean":"2023-10-21T17:18:47.272052+0000","last_became_active":"2023-10-21T17:18:45.955991+0000","last_became_peered":"2023-10-21T17:18:45.955991+0000","last_unstale":"2023-10-21T17:18:47.272052+0000","last_undegraded":"2023-10-21T17:18:47.272052+0000","last_fullsized":"2023-10-21T17:18:47.272052+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:29:31.904410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960858+0000","last_change":"2023-10-21T17:18:45.958326+0000","last_active":"2023-10-21T17:18:45.960858+0000","last_peered":"2023-10-21T17:18:45.960858+0000","last_clean":"2023-10-21T17:18:45.960858+0000","last_became_active":"2023-10-21T17:18:45.957964+0000","last_became_peered":"2023-10-21T17:18:45.957964+0000","last_unstale":"2023-10-21T17:18:45.960858+0000","last_undegraded":"2023-10-21T17:18:45.960858+0000","last_fullsized":"2023-10-21T17:18:45.960858+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:19:25.038155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960020+0000","last_change":"2023-10-21T17:18:45.955632+0000","last_active":"2023-10-21T17:18:45.960020+0000","last_peered":"2023-10-21T17:18:45.960020+0000","last_clean":"2023-10-21T17:18:45.960020+0000","last_became_active":"2023-10-21T17:18:45.955316+0000","last_became_peered":"2023-10-21T17:18:45.955316+0000","last_unstale":"2023-10-21T17:18:45.960020+0000","last_undegraded":"2023-10-21T17:18:45.960020+0000","last_fullsized":"2023-10-21T17:18:45.960020+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:27:37.015321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272900+0000","last_change":"2023-10-21T17:18:45.957861+0000","last_active":"2023-10-21T17:18:47.272900+0000","last_peered":"2023-10-21T17:18:47.272900+0000","last_clean":"2023-10-21T17:18:47.272900+0000","last_became_active":"2023-10-21T17:18:45.954908+0000","last_became_peered":"2023-10-21T17:18:45.954908+0000","last_unstale":"2023-10-21T17:18:47.272900+0000","last_undegraded":"2023-10-21T17:18:47.272900+0000","last_fullsized":"2023-10-21T17:18:47.272900+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:19.423240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957272+0000","last_change":"2023-10-21T17:18:45.951014+0000","last_active":"2023-10-21T17:18:46.957272+0000","last_peered":"2023-10-21T17:18:46.957272+0000","last_clean":"2023-10-21T17:18:46.957272+0000","last_became_active":"2023-10-21T17:18:45.950697+0000","last_became_peered":"2023-10-21T17:18:45.950697+0000","last_unstale":"2023-10-21T17:18:46.957272+0000","last_undegraded":"2023-10-21T17:18:46.957272+0000","last_fullsized":"2023-10-21T17:18:46.957272+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:05:52.180676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.964203+0000","last_change":"2023-10-21T17:18:45.961773+0000","last_active":"2023-10-21T17:18:45.964203+0000","last_peered":"2023-10-21T17:18:45.964203+0000","last_clean":"2023-10-21T17:18:45.964203+0000","last_became_active":"2023-10-21T17:18:45.961514+0000","last_became_peered":"2023-10-21T17:18:45.961514+0000","last_unstale":"2023-10-21T17:18:45.964203+0000","last_undegraded":"2023-10-21T17:18:45.964203+0000","last_fullsized":"2023-10-21T17:18:45.964203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:24:54.830195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446540+0000","last_change":"2023-10-21T17:18:45.949351+0000","last_active":"2023-10-21T17:18:47.446540+0000","last_peered":"2023-10-21T17:18:47.446540+0000","last_clean":"2023-10-21T17:18:47.446540+0000","last_became_active":"2023-10-21T17:18:45.948962+0000","last_became_peered":"2023-10-21T17:18:45.948962+0000","last_unstale":"2023-10-21T17:18:47.446540+0000","last_undegraded":"2023-10-21T17:18:47.446540+0000","last_fullsized":"2023-10-21T17:18:47.446540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:21.538295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966732+0000","last_change":"2023-10-21T17:18:45.963931+0000","last_active":"2023-10-21T17:18:45.966732+0000","last_peered":"2023-10-21T17:18:45.966732+0000","last_clean":"2023-10-21T17:18:45.966732+0000","last_became_active":"2023-10-21T17:18:45.963627+0000","last_became_peered":"2023-10-21T17:18:45.963627+0000","last_unstale":"2023-10-21T17:18:45.966732+0000","last_undegraded":"2023-10-21T17:18:45.966732+0000","last_fullsized":"2023-10-21T17:18:45.966732+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:25.875229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954837+0000","last_change":"2023-10-21T17:18:45.954257+0000","last_active":"2023-10-21T17:18:46.954837+0000","last_peered":"2023-10-21T17:18:46.954837+0000","last_clean":"2023-10-21T17:18:46.954837+0000","last_became_active":"2023-10-21T17:18:45.953890+0000","last_became_peered":"2023-10-21T17:18:45.953890+0000","last_unstale":"2023-10-21T17:18:46.954837+0000","last_undegraded":"2023-10-21T17:18:46.954837+0000","last_fullsized":"2023-10-21T17:18:46.954837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:26:25.002205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958642+0000","last_change":"2023-10-21T17:18:45.941632+0000","last_active":"2023-10-21T17:18:46.958642+0000","last_peered":"2023-10-21T17:18:46.958642+0000","last_clean":"2023-10-21T17:18:46.958642+0000","last_became_active":"2023-10-21T17:18:45.941221+0000","last_became_peered":"2023-10-21T17:18:45.941221+0000","last_unstale":"2023-10-21T17:18:46.958642+0000","last_undegraded":"2023-10-21T17:18:46.958642+0000","last_fullsized":"2023-10-21T17:18:46.958642+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:51.561437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956702+0000","last_change":"2023-10-21T17:18:45.954318+0000","last_active":"2023-10-21T17:18:46.956702+0000","last_peered":"2023-10-21T17:18:46.956702+0000","last_clean":"2023-10-21T17:18:46.956702+0000","last_became_active":"2023-10-21T17:18:45.953922+0000","last_became_peered":"2023-10-21T17:18:45.953922+0000","last_unstale":"2023-10-21T17:18:46.956702+0000","last_undegraded":"2023-10-21T17:18:46.956702+0000","last_fullsized":"2023-10-21T17:18:46.956702+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:26:59.882134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956120+0000","last_change":"2023-10-21T17:18:45.945455+0000","last_active":"2023-10-21T17:18:46.956120+0000","last_peered":"2023-10-21T17:18:46.956120+0000","last_clean":"2023-10-21T17:18:46.956120+0000","last_became_active":"2023-10-21T17:18:45.945008+0000","last_became_peered":"2023-10-21T17:18:45.945008+0000","last_unstale":"2023-10-21T17:18:46.956120+0000","last_undegraded":"2023-10-21T17:18:46.956120+0000","last_fullsized":"2023-10-21T17:18:46.956120+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:24:54.877893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955623+0000","last_change":"2023-10-21T17:18:45.960509+0000","last_active":"2023-10-21T17:18:46.955623+0000","last_peered":"2023-10-21T17:18:46.955623+0000","last_clean":"2023-10-21T17:18:46.955623+0000","last_became_active":"2023-10-21T17:18:45.957659+0000","last_became_peered":"2023-10-21T17:18:45.957659+0000","last_unstale":"2023-10-21T17:18:46.955623+0000","last_undegraded":"2023-10-21T17:18:46.955623+0000","last_fullsized":"2023-10-21T17:18:46.955623+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:00:07.625445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966630+0000","last_change":"2023-10-21T17:18:45.963416+0000","last_active":"2023-10-21T17:18:45.966630+0000","last_peered":"2023-10-21T17:18:45.966630+0000","last_clean":"2023-10-21T17:18:45.966630+0000","last_became_active":"2023-10-21T17:18:45.963011+0000","last_became_peered":"2023-10-21T17:18:45.963011+0000","last_unstale":"2023-10-21T17:18:45.966630+0000","last_undegraded":"2023-10-21T17:18:45.966630+0000","last_fullsized":"2023-10-21T17:18:45.966630+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:04:23.140791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956704+0000","last_change":"2023-10-21T17:18:45.946331+0000","last_active":"2023-10-21T17:18:46.956704+0000","last_peered":"2023-10-21T17:18:46.956704+0000","last_clean":"2023-10-21T17:18:46.956704+0000","last_became_active":"2023-10-21T17:18:45.945573+0000","last_became_peered":"2023-10-21T17:18:45.945573+0000","last_unstale":"2023-10-21T17:18:46.956704+0000","last_undegraded":"2023-10-21T17:18:46.956704+0000","last_fullsized":"2023-10-21T17:18:46.956704+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:52:12.511776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958988+0000","last_change":"2023-10-21T17:18:45.959220+0000","last_active":"2023-10-21T17:18:46.958988+0000","last_peered":"2023-10-21T17:18:46.958988+0000","last_clean":"2023-10-21T17:18:46.958988+0000","last_became_active":"2023-10-21T17:18:45.958868+0000","last_became_peered":"2023-10-21T17:18:45.958868+0000","last_unstale":"2023-10-21T17:18:46.958988+0000","last_undegraded":"2023-10-21T17:18:46.958988+0000","last_fullsized":"2023-10-21T17:18:46.958988+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:00.576689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956766+0000","last_change":"2023-10-21T17:18:45.951341+0000","last_active":"2023-10-21T17:18:46.956766+0000","last_peered":"2023-10-21T17:18:46.956766+0000","last_clean":"2023-10-21T17:18:46.956766+0000","last_became_active":"2023-10-21T17:18:45.951073+0000","last_became_peered":"2023-10-21T17:18:45.951073+0000","last_unstale":"2023-10-21T17:18:46.956766+0000","last_undegraded":"2023-10-21T17:18:46.956766+0000","last_fullsized":"2023-10-21T17:18:46.956766+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:06.433984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.959078+0000","last_change":"2023-10-21T17:18:45.954735+0000","last_active":"2023-10-21T17:18:45.959078+0000","last_peered":"2023-10-21T17:18:45.959078+0000","last_clean":"2023-10-21T17:18:45.959078+0000","last_became_active":"2023-10-21T17:18:45.954342+0000","last_became_peered":"2023-10-21T17:18:45.954342+0000","last_unstale":"2023-10-21T17:18:45.959078+0000","last_undegraded":"2023-10-21T17:18:45.959078+0000","last_fullsized":"2023-10-21T17:18:45.959078+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:31:56.073758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959635+0000","last_change":"2023-10-21T17:18:45.953330+0000","last_active":"2023-10-21T17:18:46.959635+0000","last_peered":"2023-10-21T17:18:46.959635+0000","last_clean":"2023-10-21T17:18:46.959635+0000","last_became_active":"2023-10-21T17:18:45.953003+0000","last_became_peered":"2023-10-21T17:18:45.953003+0000","last_unstale":"2023-10-21T17:18:46.959635+0000","last_undegraded":"2023-10-21T17:18:46.959635+0000","last_fullsized":"2023-10-21T17:18:46.959635+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:38.574434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963363+0000","last_change":"2023-10-21T17:18:45.959076+0000","last_active":"2023-10-21T17:18:45.963363+0000","last_peered":"2023-10-21T17:18:45.963363+0000","last_clean":"2023-10-21T17:18:45.963363+0000","last_became_active":"2023-10-21T17:18:45.957531+0000","last_became_peered":"2023-10-21T17:18:45.957531+0000","last_unstale":"2023-10-21T17:18:45.963363+0000","last_undegraded":"2023-10-21T17:18:45.963363+0000","last_fullsized":"2023-10-21T17:18:45.963363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:25.135652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"183'4","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.976054+0000","last_change":"2023-10-21T17:18:44.921449+0000","last_active":"2023-10-21T17:18:45.976054+0000","last_peered":"2023-10-21T17:18:45.976054+0000","last_clean":"2023-10-21T17:18:45.976054+0000","last_became_active":"2023-10-21T17:18:44.921204+0000","last_became_peered":"2023-10-21T17:18:44.921204+0000","last_unstale":"2023-10-21T17:18:45.976054+0000","last_undegraded":"2023-10-21T17:18:45.976054+0000","last_fullsized":"2023-10-21T17:18:45.976054+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:01.569523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960984+0000","last_change":"2023-10-21T17:18:45.958560+0000","last_active":"2023-10-21T17:18:45.960984+0000","last_peered":"2023-10-21T17:18:45.960984+0000","last_clean":"2023-10-21T17:18:45.960984+0000","last_became_active":"2023-10-21T17:18:45.957866+0000","last_became_peered":"2023-10-21T17:18:45.957866+0000","last_unstale":"2023-10-21T17:18:45.960984+0000","last_undegraded":"2023-10-21T17:18:45.960984+0000","last_fullsized":"2023-10-21T17:18:45.960984+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:38.443203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975986+0000","last_change":"2023-10-21T17:18:44.930566+0000","last_active":"2023-10-21T17:18:45.975986+0000","last_peered":"2023-10-21T17:18:45.975986+0000","last_clean":"2023-10-21T17:18:45.975986+0000","last_became_active":"2023-10-21T17:18:44.930272+0000","last_became_peered":"2023-10-21T17:18:44.930272+0000","last_unstale":"2023-10-21T17:18:45.975986+0000","last_undegraded":"2023-10-21T17:18:45.975986+0000","last_fullsized":"2023-10-21T17:18:45.975986+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:08.700057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959431+0000","last_change":"2023-10-21T17:18:45.951750+0000","last_active":"2023-10-21T17:18:46.959431+0000","last_peered":"2023-10-21T17:18:46.959431+0000","last_clean":"2023-10-21T17:18:46.959431+0000","last_became_active":"2023-10-21T17:18:45.951466+0000","last_became_peered":"2023-10-21T17:18:45.951466+0000","last_unstale":"2023-10-21T17:18:46.959431+0000","last_undegraded":"2023-10-21T17:18:46.959431+0000","last_fullsized":"2023-10-21T17:18:46.959431+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:25.239123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975530+0000","last_change":"2023-10-21T17:18:44.935832+0000","last_active":"2023-10-21T17:18:45.975530+0000","last_peered":"2023-10-21T17:18:45.975530+0000","last_clean":"2023-10-21T17:18:45.975530+0000","last_became_active":"2023-10-21T17:18:44.935539+0000","last_became_peered":"2023-10-21T17:18:44.935539+0000","last_unstale":"2023-10-21T17:18:45.975530+0000","last_undegraded":"2023-10-21T17:18:45.975530+0000","last_fullsized":"2023-10-21T17:18:45.975530+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:56.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966720+0000","last_change":"2023-10-21T17:18:45.962418+0000","last_active":"2023-10-21T17:18:45.966720+0000","last_peered":"2023-10-21T17:18:45.966720+0000","last_clean":"2023-10-21T17:18:45.966720+0000","last_became_active":"2023-10-21T17:18:45.960824+0000","last_became_peered":"2023-10-21T17:18:45.960824+0000","last_unstale":"2023-10-21T17:18:45.966720+0000","last_undegraded":"2023-10-21T17:18:45.966720+0000","last_fullsized":"2023-10-21T17:18:45.966720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:01.336383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.974133+0000","last_change":"2023-10-21T17:18:44.928291+0000","last_active":"2023-10-21T17:18:45.974133+0000","last_peered":"2023-10-21T17:18:45.974133+0000","last_clean":"2023-10-21T17:18:45.974133+0000","last_became_active":"2023-10-21T17:18:44.927669+0000","last_became_peered":"2023-10-21T17:18:44.927669+0000","last_unstale":"2023-10-21T17:18:45.974133+0000","last_undegraded":"2023-10-21T17:18:45.974133+0000","last_fullsized":"2023-10-21T17:18:45.974133+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:57:18.809182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446326+0000","last_change":"2023-10-21T17:18:45.946656+0000","last_active":"2023-10-21T17:18:47.446326+0000","last_peered":"2023-10-21T17:18:47.446326+0000","last_clean":"2023-10-21T17:18:47.446326+0000","last_became_active":"2023-10-21T17:18:45.946321+0000","last_became_peered":"2023-10-21T17:18:45.946321+0000","last_unstale":"2023-10-21T17:18:47.446326+0000","last_undegraded":"2023-10-21T17:18:47.446326+0000","last_fullsized":"2023-10-21T17:18:47.446326+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:09.704931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"183'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959033+0000","last_change":"2023-10-21T17:18:44.928731+0000","last_active":"2023-10-21T17:18:46.959033+0000","last_peered":"2023-10-21T17:18:46.959033+0000","last_clean":"2023-10-21T17:18:46.959033+0000","last_became_active":"2023-10-21T17:18:44.928449+0000","last_became_peered":"2023-10-21T17:18:44.928449+0000","last_unstale":"2023-10-21T17:18:46.959033+0000","last_undegraded":"2023-10-21T17:18:46.959033+0000","last_fullsized":"2023-10-21T17:18:46.959033+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:27.690328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.271896+0000","last_change":"2023-10-21T17:18:45.961875+0000","last_active":"2023-10-21T17:18:47.271896+0000","last_peered":"2023-10-21T17:18:47.271896+0000","last_clean":"2023-10-21T17:18:47.271896+0000","last_became_active":"2023-10-21T17:18:45.961274+0000","last_became_peered":"2023-10-21T17:18:45.961274+0000","last_unstale":"2023-10-21T17:18:47.271896+0000","last_undegraded":"2023-10-21T17:18:47.271896+0000","last_fullsized":"2023-10-21T17:18:47.271896+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:37:14.180774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.973719+0000","last_change":"2023-10-21T17:18:44.929379+0000","last_active":"2023-10-21T17:18:45.973719+0000","last_peered":"2023-10-21T17:18:45.973719+0000","last_clean":"2023-10-21T17:18:45.973719+0000","last_became_active":"2023-10-21T17:18:44.929123+0000","last_became_peered":"2023-10-21T17:18:44.929123+0000","last_unstale":"2023-10-21T17:18:45.973719+0000","last_undegraded":"2023-10-21T17:18:45.973719+0000","last_fullsized":"2023-10-21T17:18:45.973719+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:20.560809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963075+0000","last_change":"2023-10-21T17:18:45.958845+0000","last_active":"2023-10-21T17:18:45.963075+0000","last_peered":"2023-10-21T17:18:45.963075+0000","last_clean":"2023-10-21T17:18:45.963075+0000","last_became_active":"2023-10-21T17:18:45.958520+0000","last_became_peered":"2023-10-21T17:18:45.958520+0000","last_unstale":"2023-10-21T17:18:45.963075+0000","last_undegraded":"2023-10-21T17:18:45.963075+0000","last_fullsized":"2023-10-21T17:18:45.963075+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:44:55.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955191+0000","last_change":"2023-10-21T17:18:44.928515+0000","last_active":"2023-10-21T17:18:46.955191+0000","last_peered":"2023-10-21T17:18:46.955191+0000","last_clean":"2023-10-21T17:18:46.955191+0000","last_became_active":"2023-10-21T17:18:44.928173+0000","last_became_peered":"2023-10-21T17:18:44.928173+0000","last_unstale":"2023-10-21T17:18:46.955191+0000","last_undegraded":"2023-10-21T17:18:46.955191+0000","last_fullsized":"2023-10-21T17:18:46.955191+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:13:13.490105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955247+0000","last_change":"2023-10-21T17:18:45.939278+0000","last_active":"2023-10-21T17:18:46.955247+0000","last_peered":"2023-10-21T17:18:46.955247+0000","last_clean":"2023-10-21T17:18:46.955247+0000","last_became_active":"2023-10-21T17:18:45.938531+0000","last_became_peered":"2023-10-21T17:18:45.938531+0000","last_unstale":"2023-10-21T17:18:46.955247+0000","last_undegraded":"2023-10-21T17:18:46.955247+0000","last_fullsized":"2023-10-21T17:18:46.955247+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:15.146123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.936560+0000","last_change":"2023-10-21T17:18:44.927882+0000","last_active":"2023-10-21T17:18:45.936560+0000","last_peered":"2023-10-21T17:18:45.936560+0000","last_clean":"2023-10-21T17:18:45.936560+0000","last_became_active":"2023-10-21T17:18:44.927603+0000","last_became_peered":"2023-10-21T17:18:44.927603+0000","last_unstale":"2023-10-21T17:18:45.936560+0000","last_undegraded":"2023-10-21T17:18:45.936560+0000","last_fullsized":"2023-10-21T17:18:45.936560+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:12.844392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959540+0000","last_change":"2023-10-21T17:18:45.951306+0000","last_active":"2023-10-21T17:18:46.959540+0000","last_peered":"2023-10-21T17:18:46.959540+0000","last_clean":"2023-10-21T17:18:46.959540+0000","last_became_active":"2023-10-21T17:18:45.951025+0000","last_became_peered":"2023-10-21T17:18:45.951025+0000","last_unstale":"2023-10-21T17:18:46.959540+0000","last_undegraded":"2023-10-21T17:18:46.959540+0000","last_fullsized":"2023-10-21T17:18:46.959540+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:39:28.204765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.943705+0000","last_change":"2023-10-21T17:18:44.928130+0000","last_active":"2023-10-21T17:18:45.943705+0000","last_peered":"2023-10-21T17:18:45.943705+0000","last_clean":"2023-10-21T17:18:45.943705+0000","last_became_active":"2023-10-21T17:18:44.927860+0000","last_became_peered":"2023-10-21T17:18:44.927860+0000","last_unstale":"2023-10-21T17:18:45.943705+0000","last_undegraded":"2023-10-21T17:18:45.943705+0000","last_fullsized":"2023-10-21T17:18:45.943705+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:22:41.144389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957077+0000","last_change":"2023-10-21T17:18:45.951752+0000","last_active":"2023-10-21T17:18:46.957077+0000","last_peered":"2023-10-21T17:18:46.957077+0000","last_clean":"2023-10-21T17:18:46.957077+0000","last_became_active":"2023-10-21T17:18:45.951083+0000","last_became_peered":"2023-10-21T17:18:45.951083+0000","last_unstale":"2023-10-21T17:18:46.957077+0000","last_undegraded":"2023-10-21T17:18:46.957077+0000","last_fullsized":"2023-10-21T17:18:46.957077+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:48:21.975896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956092+0000","last_change":"2023-10-21T17:18:44.927565+0000","last_active":"2023-10-21T17:18:46.956092+0000","last_peered":"2023-10-21T17:18:46.956092+0000","last_clean":"2023-10-21T17:18:46.956092+0000","last_became_active":"2023-10-21T17:18:44.926608+0000","last_became_peered":"2023-10-21T17:18:44.926608+0000","last_unstale":"2023-10-21T17:18:46.956092+0000","last_undegraded":"2023-10-21T17:18:46.956092+0000","last_fullsized":"2023-10-21T17:18:46.956092+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:06.981238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957167+0000","last_change":"2023-10-21T17:18:45.951701+0000","last_active":"2023-10-21T17:18:46.957167+0000","last_peered":"2023-10-21T17:18:46.957167+0000","last_clean":"2023-10-21T17:18:46.957167+0000","last_became_active":"2023-10-21T17:18:45.951382+0000","last_became_peered":"2023-10-21T17:18:45.951382+0000","last_unstale":"2023-10-21T17:18:46.957167+0000","last_undegraded":"2023-10-21T17:18:46.957167+0000","last_fullsized":"2023-10-21T17:18:46.957167+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:42.986072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"183'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.957027+0000","last_change":"2023-10-21T17:18:44.905516+0000","last_active":"2023-10-21T17:18:46.957027+0000","last_peered":"2023-10-21T17:18:46.957027+0000","last_clean":"2023-10-21T17:18:46.957027+0000","last_became_active":"2023-10-21T17:18:44.904602+0000","last_became_peered":"2023-10-21T17:18:44.904602+0000","last_unstale":"2023-10-21T17:18:46.957027+0000","last_undegraded":"2023-10-21T17:18:46.957027+0000","last_fullsized":"2023-10-21T17:18:46.957027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:23:18.904211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958940+0000","last_change":"2023-10-21T17:18:45.954223+0000","last_active":"2023-10-21T17:18:46.958940+0000","last_peered":"2023-10-21T17:18:46.958940+0000","last_clean":"2023-10-21T17:18:46.958940+0000","last_became_active":"2023-10-21T17:18:45.953964+0000","last_became_peered":"2023-10-21T17:18:45.953964+0000","last_unstale":"2023-10-21T17:18:46.958940+0000","last_undegraded":"2023-10-21T17:18:46.958940+0000","last_fullsized":"2023-10-21T17:18:46.958940+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:42:10.240710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975998+0000","last_change":"2023-10-21T17:18:44.921410+0000","last_active":"2023-10-21T17:18:45.975998+0000","last_peered":"2023-10-21T17:18:45.975998+0000","last_clean":"2023-10-21T17:18:45.975998+0000","last_became_active":"2023-10-21T17:18:44.921130+0000","last_became_peered":"2023-10-21T17:18:44.921130+0000","last_unstale":"2023-10-21T17:18:45.975998+0000","last_undegraded":"2023-10-21T17:18:45.975998+0000","last_fullsized":"2023-10-21T17:18:45.975998+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:11.516018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960950+0000","last_change":"2023-10-21T17:18:45.958827+0000","last_active":"2023-10-21T17:18:45.960950+0000","last_peered":"2023-10-21T17:18:45.960950+0000","last_clean":"2023-10-21T17:18:45.960950+0000","last_became_active":"2023-10-21T17:18:45.958503+0000","last_became_peered":"2023-10-21T17:18:45.958503+0000","last_unstale":"2023-10-21T17:18:45.960950+0000","last_undegraded":"2023-10-21T17:18:45.960950+0000","last_fullsized":"2023-10-21T17:18:45.960950+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:19.423240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272027+0000","last_change":"2023-10-21T17:18:44.933084+0000","last_active":"2023-10-21T17:18:47.272027+0000","last_peered":"2023-10-21T17:18:47.272027+0000","last_clean":"2023-10-21T17:18:47.272027+0000","last_became_active":"2023-10-21T17:18:44.932808+0000","last_became_peered":"2023-10-21T17:18:44.932808+0000","last_unstale":"2023-10-21T17:18:47.272027+0000","last_undegraded":"2023-10-21T17:18:47.272027+0000","last_fullsized":"2023-10-21T17:18:47.272027+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:33:45.814539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.961734+0000","last_change":"2023-10-21T17:18:45.960132+0000","last_active":"2023-10-21T17:18:45.961734+0000","last_peered":"2023-10-21T17:18:45.961734+0000","last_clean":"2023-10-21T17:18:45.961734+0000","last_became_active":"2023-10-21T17:18:45.956738+0000","last_became_peered":"2023-10-21T17:18:45.956738+0000","last_unstale":"2023-10-21T17:18:45.961734+0000","last_undegraded":"2023-10-21T17:18:45.961734+0000","last_fullsized":"2023-10-21T17:18:45.961734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:35:16.311690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272103+0000","last_change":"2023-10-21T17:18:44.924137+0000","last_active":"2023-10-21T17:18:47.272103+0000","last_peered":"2023-10-21T17:18:47.272103+0000","last_clean":"2023-10-21T17:18:47.272103+0000","last_became_active":"2023-10-21T17:18:44.923842+0000","last_became_peered":"2023-10-21T17:18:44.923842+0000","last_unstale":"2023-10-21T17:18:47.272103+0000","last_undegraded":"2023-10-21T17:18:47.272103+0000","last_fullsized":"2023-10-21T17:18:47.272103+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:52.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.960614+0000","last_change":"2023-10-21T17:18:45.956532+0000","last_active":"2023-10-21T17:18:45.960614+0000","last_peered":"2023-10-21T17:18:45.960614+0000","last_clean":"2023-10-21T17:18:45.960614+0000","last_became_active":"2023-10-21T17:18:45.956193+0000","last_became_peered":"2023-10-21T17:18:45.956193+0000","last_unstale":"2023-10-21T17:18:45.960614+0000","last_undegraded":"2023-10-21T17:18:45.960614+0000","last_fullsized":"2023-10-21T17:18:45.960614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:42.736079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"183'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954468+0000","last_change":"2023-10-21T17:18:44.932179+0000","last_active":"2023-10-21T17:18:46.954468+0000","last_peered":"2023-10-21T17:18:46.954468+0000","last_clean":"2023-10-21T17:18:46.954468+0000","last_became_active":"2023-10-21T17:18:44.930361+0000","last_became_peered":"2023-10-21T17:18:44.930361+0000","last_unstale":"2023-10-21T17:18:46.954468+0000","last_undegraded":"2023-10-21T17:18:46.954468+0000","last_fullsized":"2023-10-21T17:18:46.954468+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.966971+0000","last_change":"2023-10-21T17:18:45.963617+0000","last_active":"2023-10-21T17:18:45.966971+0000","last_peered":"2023-10-21T17:18:45.966971+0000","last_clean":"2023-10-21T17:18:45.966971+0000","last_became_active":"2023-10-21T17:18:45.963319+0000","last_became_peered":"2023-10-21T17:18:45.963319+0000","last_unstale":"2023-10-21T17:18:45.966971+0000","last_undegraded":"2023-10-21T17:18:45.966971+0000","last_fullsized":"2023-10-21T17:18:45.966971+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:25:58.232488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"183'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956842+0000","last_change":"2023-10-21T17:18:44.922866+0000","last_active":"2023-10-21T17:18:46.956842+0000","last_peered":"2023-10-21T17:18:46.956842+0000","last_clean":"2023-10-21T17:18:46.956842+0000","last_became_active":"2023-10-21T17:18:44.922597+0000","last_became_peered":"2023-10-21T17:18:44.922597+0000","last_unstale":"2023-10-21T17:18:46.956842+0000","last_undegraded":"2023-10-21T17:18:46.956842+0000","last_fullsized":"2023-10-21T17:18:46.956842+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:17.621264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967299+0000","last_change":"2023-10-21T17:18:45.965443+0000","last_active":"2023-10-21T17:18:45.967299+0000","last_peered":"2023-10-21T17:18:45.967299+0000","last_clean":"2023-10-21T17:18:45.967299+0000","last_became_active":"2023-10-21T17:18:45.964963+0000","last_became_peered":"2023-10-21T17:18:45.964963+0000","last_unstale":"2023-10-21T17:18:45.967299+0000","last_undegraded":"2023-10-21T17:18:45.967299+0000","last_fullsized":"2023-10-21T17:18:45.967299+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:07.986111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"183'2","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.975964+0000","last_change":"2023-10-21T17:18:44.927358+0000","last_active":"2023-10-21T17:18:45.975964+0000","last_peered":"2023-10-21T17:18:45.975964+0000","last_clean":"2023-10-21T17:18:45.975964+0000","last_became_active":"2023-10-21T17:18:44.926468+0000","last_became_peered":"2023-10-21T17:18:44.926468+0000","last_unstale":"2023-10-21T17:18:45.975964+0000","last_undegraded":"2023-10-21T17:18:45.975964+0000","last_fullsized":"2023-10-21T17:18:45.975964+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:20.533421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955816+0000","last_change":"2023-10-21T17:18:45.963974+0000","last_active":"2023-10-21T17:18:46.955816+0000","last_peered":"2023-10-21T17:18:46.955816+0000","last_clean":"2023-10-21T17:18:46.955816+0000","last_became_active":"2023-10-21T17:18:45.963721+0000","last_became_peered":"2023-10-21T17:18:45.963721+0000","last_unstale":"2023-10-21T17:18:46.955816+0000","last_undegraded":"2023-10-21T17:18:46.955816+0000","last_fullsized":"2023-10-21T17:18:46.955816+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:23.766133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"183'1","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.973254+0000","last_change":"2023-10-21T17:18:44.926416+0000","last_active":"2023-10-21T17:18:45.973254+0000","last_peered":"2023-10-21T17:18:45.973254+0000","last_clean":"2023-10-21T17:18:45.973254+0000","last_became_active":"2023-10-21T17:18:44.926135+0000","last_became_peered":"2023-10-21T17:18:44.926135+0000","last_unstale":"2023-10-21T17:18:45.973254+0000","last_undegraded":"2023-10-21T17:18:45.973254+0000","last_fullsized":"2023-10-21T17:18:45.973254+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:57.851590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956229+0000","last_change":"2023-10-21T17:18:45.958438+0000","last_active":"2023-10-21T17:18:46.956229+0000","last_peered":"2023-10-21T17:18:46.956229+0000","last_clean":"2023-10-21T17:18:46.956229+0000","last_became_active":"2023-10-21T17:18:45.958120+0000","last_became_peered":"2023-10-21T17:18:45.958120+0000","last_unstale":"2023-10-21T17:18:46.956229+0000","last_undegraded":"2023-10-21T17:18:46.956229+0000","last_fullsized":"2023-10-21T17:18:46.956229+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:40.885115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272433+0000","last_change":"2023-10-21T17:18:44.933895+0000","last_active":"2023-10-21T17:18:47.272433+0000","last_peered":"2023-10-21T17:18:47.272433+0000","last_clean":"2023-10-21T17:18:47.272433+0000","last_became_active":"2023-10-21T17:18:44.933635+0000","last_became_peered":"2023-10-21T17:18:44.933635+0000","last_unstale":"2023-10-21T17:18:47.272433+0000","last_undegraded":"2023-10-21T17:18:47.272433+0000","last_fullsized":"2023-10-21T17:18:47.272433+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:07.107043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963016+0000","last_change":"2023-10-21T17:18:45.958300+0000","last_active":"2023-10-21T17:18:45.963016+0000","last_peered":"2023-10-21T17:18:45.963016+0000","last_clean":"2023-10-21T17:18:45.963016+0000","last_became_active":"2023-10-21T17:18:45.958019+0000","last_became_peered":"2023-10-21T17:18:45.958019+0000","last_unstale":"2023-10-21T17:18:45.963016+0000","last_undegraded":"2023-10-21T17:18:45.963016+0000","last_fullsized":"2023-10-21T17:18:45.963016+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:57.831529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"183'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446283+0000","last_change":"2023-10-21T17:18:44.918987+0000","last_active":"2023-10-21T17:18:47.446283+0000","last_peered":"2023-10-21T17:18:47.446283+0000","last_clean":"2023-10-21T17:18:47.446283+0000","last_became_active":"2023-10-21T17:18:44.918617+0000","last_became_peered":"2023-10-21T17:18:44.918617+0000","last_unstale":"2023-10-21T17:18:47.446283+0000","last_undegraded":"2023-10-21T17:18:47.446283+0000","last_fullsized":"2023-10-21T17:18:47.446283+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.958174+0000","last_change":"2023-10-21T17:18:45.952576+0000","last_active":"2023-10-21T17:18:45.958174+0000","last_peered":"2023-10-21T17:18:45.958174+0000","last_clean":"2023-10-21T17:18:45.958174+0000","last_became_active":"2023-10-21T17:18:45.952233+0000","last_became_peered":"2023-10-21T17:18:45.952233+0000","last_unstale":"2023-10-21T17:18:45.958174+0000","last_undegraded":"2023-10-21T17:18:45.958174+0000","last_fullsized":"2023-10-21T17:18:45.958174+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:39.429149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958905+0000","last_change":"2023-10-21T17:18:44.930761+0000","last_active":"2023-10-21T17:18:46.958905+0000","last_peered":"2023-10-21T17:18:46.958905+0000","last_clean":"2023-10-21T17:18:46.958905+0000","last_became_active":"2023-10-21T17:18:44.929671+0000","last_became_peered":"2023-10-21T17:18:44.929671+0000","last_unstale":"2023-10-21T17:18:46.958905+0000","last_undegraded":"2023-10-21T17:18:46.958905+0000","last_fullsized":"2023-10-21T17:18:46.958905+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:52.603947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963052+0000","last_change":"2023-10-21T17:18:45.955841+0000","last_active":"2023-10-21T17:18:45.963052+0000","last_peered":"2023-10-21T17:18:45.963052+0000","last_clean":"2023-10-21T17:18:45.963052+0000","last_became_active":"2023-10-21T17:18:45.955505+0000","last_became_peered":"2023-10-21T17:18:45.955505+0000","last_unstale":"2023-10-21T17:18:45.963052+0000","last_undegraded":"2023-10-21T17:18:45.963052+0000","last_fullsized":"2023-10-21T17:18:45.963052+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:22.958231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.445931+0000","last_change":"2023-10-21T17:18:44.917343+0000","last_active":"2023-10-21T17:18:47.445931+0000","last_peered":"2023-10-21T17:18:47.445931+0000","last_clean":"2023-10-21T17:18:47.445931+0000","last_became_active":"2023-10-21T17:18:44.916935+0000","last_became_peered":"2023-10-21T17:18:44.916935+0000","last_unstale":"2023-10-21T17:18:47.445931+0000","last_undegraded":"2023-10-21T17:18:47.445931+0000","last_fullsized":"2023-10-21T17:18:47.445931+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:33:36.162430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.963837+0000","last_change":"2023-10-21T17:18:45.959935+0000","last_active":"2023-10-21T17:18:45.963837+0000","last_peered":"2023-10-21T17:18:45.963837+0000","last_clean":"2023-10-21T17:18:45.963837+0000","last_became_active":"2023-10-21T17:18:45.959648+0000","last_became_peered":"2023-10-21T17:18:45.959648+0000","last_unstale":"2023-10-21T17:18:45.963837+0000","last_undegraded":"2023-10-21T17:18:45.963837+0000","last_fullsized":"2023-10-21T17:18:45.963837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:21.984308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959522+0000","last_change":"2023-10-21T17:18:44.926990+0000","last_active":"2023-10-21T17:18:46.959522+0000","last_peered":"2023-10-21T17:18:46.959522+0000","last_clean":"2023-10-21T17:18:46.959522+0000","last_became_active":"2023-10-21T17:18:44.926705+0000","last_became_peered":"2023-10-21T17:18:44.926705+0000","last_unstale":"2023-10-21T17:18:46.959522+0000","last_undegraded":"2023-10-21T17:18:46.959522+0000","last_fullsized":"2023-10-21T17:18:46.959522+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:51:58.789688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967489+0000","last_change":"2023-10-21T17:18:45.963937+0000","last_active":"2023-10-21T17:18:45.967489+0000","last_peered":"2023-10-21T17:18:45.967489+0000","last_clean":"2023-10-21T17:18:45.967489+0000","last_became_active":"2023-10-21T17:18:45.963539+0000","last_became_peered":"2023-10-21T17:18:45.963539+0000","last_unstale":"2023-10-21T17:18:45.967489+0000","last_undegraded":"2023-10-21T17:18:45.967489+0000","last_fullsized":"2023-10-21T17:18:45.967489+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:37:49.963150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"183'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272968+0000","last_change":"2023-10-21T17:18:44.932226+0000","last_active":"2023-10-21T17:18:47.272968+0000","last_peered":"2023-10-21T17:18:47.272968+0000","last_clean":"2023-10-21T17:18:47.272968+0000","last_became_active":"2023-10-21T17:18:44.931948+0000","last_became_peered":"2023-10-21T17:18:44.931948+0000","last_unstale":"2023-10-21T17:18:47.272968+0000","last_undegraded":"2023-10-21T17:18:47.272968+0000","last_fullsized":"2023-10-21T17:18:47.272968+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:45:10.929847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.272844+0000","last_change":"2023-10-21T17:18:45.963319+0000","last_active":"2023-10-21T17:18:47.272844+0000","last_peered":"2023-10-21T17:18:47.272844+0000","last_clean":"2023-10-21T17:18:47.272844+0000","last_became_active":"2023-10-21T17:18:45.963051+0000","last_became_peered":"2023-10-21T17:18:45.963051+0000","last_unstale":"2023-10-21T17:18:47.272844+0000","last_undegraded":"2023-10-21T17:18:47.272844+0000","last_fullsized":"2023-10-21T17:18:47.272844+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:15.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955971+0000","last_change":"2023-10-21T17:18:44.927320+0000","last_active":"2023-10-21T17:18:46.955971+0000","last_peered":"2023-10-21T17:18:46.955971+0000","last_clean":"2023-10-21T17:18:46.955971+0000","last_became_active":"2023-10-21T17:18:44.925951+0000","last_became_peered":"2023-10-21T17:18:44.925951+0000","last_unstale":"2023-10-21T17:18:46.955971+0000","last_undegraded":"2023-10-21T17:18:46.955971+0000","last_fullsized":"2023-10-21T17:18:46.955971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:56.076437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955835+0000","last_change":"2023-10-21T17:18:45.941961+0000","last_active":"2023-10-21T17:18:46.955835+0000","last_peered":"2023-10-21T17:18:46.955835+0000","last_clean":"2023-10-21T17:18:46.955835+0000","last_became_active":"2023-10-21T17:18:45.941388+0000","last_became_peered":"2023-10-21T17:18:45.941388+0000","last_unstale":"2023-10-21T17:18:46.955835+0000","last_undegraded":"2023-10-21T17:18:46.955835+0000","last_fullsized":"2023-10-21T17:18:46.955835+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:34:59.636779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.941178+0000","last_change":"2023-10-21T17:18:44.931655+0000","last_active":"2023-10-21T17:18:45.941178+0000","last_peered":"2023-10-21T17:18:45.941178+0000","last_clean":"2023-10-21T17:18:45.941178+0000","last_became_active":"2023-10-21T17:18:44.931368+0000","last_became_peered":"2023-10-21T17:18:44.931368+0000","last_unstale":"2023-10-21T17:18:45.941178+0000","last_undegraded":"2023-10-21T17:18:45.941178+0000","last_fullsized":"2023-10-21T17:18:45.941178+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:10:58.330034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.967832+0000","last_change":"2023-10-21T17:18:45.965446+0000","last_active":"2023-10-21T17:18:45.967832+0000","last_peered":"2023-10-21T17:18:45.967832+0000","last_clean":"2023-10-21T17:18:45.967832+0000","last_became_active":"2023-10-21T17:18:45.964965+0000","last_became_peered":"2023-10-21T17:18:45.964965+0000","last_unstale":"2023-10-21T17:18:45.967832+0000","last_undegraded":"2023-10-21T17:18:45.967832+0000","last_fullsized":"2023-10-21T17:18:45.967832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:37.753186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.938548+0000","last_change":"2023-10-21T17:18:44.932809+0000","last_active":"2023-10-21T17:18:45.938548+0000","last_peered":"2023-10-21T17:18:45.938548+0000","last_clean":"2023-10-21T17:18:45.938548+0000","last_became_active":"2023-10-21T17:18:44.932520+0000","last_became_peered":"2023-10-21T17:18:44.932520+0000","last_unstale":"2023-10-21T17:18:45.938548+0000","last_undegraded":"2023-10-21T17:18:45.938548+0000","last_fullsized":"2023-10-21T17:18:45.938548+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:11.521567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959322+0000","last_change":"2023-10-21T17:18:45.951919+0000","last_active":"2023-10-21T17:18:46.959322+0000","last_peered":"2023-10-21T17:18:46.959322+0000","last_clean":"2023-10-21T17:18:46.959322+0000","last_became_active":"2023-10-21T17:18:45.949844+0000","last_became_peered":"2023-10-21T17:18:45.949844+0000","last_unstale":"2023-10-21T17:18:46.959322+0000","last_undegraded":"2023-10-21T17:18:46.959322+0000","last_fullsized":"2023-10-21T17:18:46.959322+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:02.581645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.955387+0000","last_change":"2023-10-21T17:18:44.930158+0000","last_active":"2023-10-21T17:18:46.955387+0000","last_peered":"2023-10-21T17:18:46.955387+0000","last_clean":"2023-10-21T17:18:46.955387+0000","last_became_active":"2023-10-21T17:18:44.929871+0000","last_became_peered":"2023-10-21T17:18:44.929871+0000","last_unstale":"2023-10-21T17:18:46.955387+0000","last_undegraded":"2023-10-21T17:18:46.955387+0000","last_fullsized":"2023-10-21T17:18:46.955387+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:20.533421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.271953+0000","last_change":"2023-10-21T17:18:45.957045+0000","last_active":"2023-10-21T17:18:47.271953+0000","last_peered":"2023-10-21T17:18:47.271953+0000","last_clean":"2023-10-21T17:18:47.271953+0000","last_became_active":"2023-10-21T17:18:45.956757+0000","last_became_peered":"2023-10-21T17:18:45.956757+0000","last_unstale":"2023-10-21T17:18:47.271953+0000","last_undegraded":"2023-10-21T17:18:47.271953+0000","last_fullsized":"2023-10-21T17:18:47.271953+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:18.337103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.956880+0000","last_change":"2023-10-21T17:18:44.910769+0000","last_active":"2023-10-21T17:18:46.956880+0000","last_peered":"2023-10-21T17:18:46.956880+0000","last_clean":"2023-10-21T17:18:46.956880+0000","last_became_active":"2023-10-21T17:18:44.909085+0000","last_became_peered":"2023-10-21T17:18:44.909085+0000","last_unstale":"2023-10-21T17:18:46.956880+0000","last_undegraded":"2023-10-21T17:18:46.956880+0000","last_fullsized":"2023-10-21T17:18:46.956880+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:04:19.299591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.958711+0000","last_change":"2023-10-21T17:18:45.953642+0000","last_active":"2023-10-21T17:18:45.958711+0000","last_peered":"2023-10-21T17:18:45.958711+0000","last_clean":"2023-10-21T17:18:45.958711+0000","last_became_active":"2023-10-21T17:18:45.953268+0000","last_became_peered":"2023-10-21T17:18:45.953268+0000","last_unstale":"2023-10-21T17:18:45.958711+0000","last_undegraded":"2023-10-21T17:18:45.958711+0000","last_fullsized":"2023-10-21T17:18:45.958711+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:52:44.780229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.954617+0000","last_change":"2023-10-21T17:18:44.925780+0000","last_active":"2023-10-21T17:18:46.954617+0000","last_peered":"2023-10-21T17:18:46.954617+0000","last_clean":"2023-10-21T17:18:46.954617+0000","last_became_active":"2023-10-21T17:18:44.925412+0000","last_became_peered":"2023-10-21T17:18:44.925412+0000","last_unstale":"2023-10-21T17:18:46.954617+0000","last_undegraded":"2023-10-21T17:18:46.954617+0000","last_fullsized":"2023-10-21T17:18:46.954617+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:47.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.958892+0000","last_change":"2023-10-21T17:18:45.955163+0000","last_active":"2023-10-21T17:18:46.958892+0000","last_peered":"2023-10-21T17:18:46.958892+0000","last_clean":"2023-10-21T17:18:46.958892+0000","last_became_active":"2023-10-21T17:18:45.954902+0000","last_became_peered":"2023-10-21T17:18:45.954902+0000","last_unstale":"2023-10-21T17:18:46.958892+0000","last_undegraded":"2023-10-21T17:18:46.958892+0000","last_fullsized":"2023-10-21T17:18:46.958892+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:54.022901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-21T17:18:45.941954+0000","last_change":"2023-10-21T17:18:44.928885+0000","last_active":"2023-10-21T17:18:45.941954+0000","last_peered":"2023-10-21T17:18:45.941954+0000","last_clean":"2023-10-21T17:18:45.941954+0000","last_became_active":"2023-10-21T17:18:44.926700+0000","last_became_peered":"2023-10-21T17:18:44.926700+0000","last_unstale":"2023-10-21T17:18:45.941954+0000","last_undegraded":"2023-10-21T17:18:45.941954+0000","last_fullsized":"2023-10-21T17:18:45.941954+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:07.782727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:46.959561+0000","last_change":"2023-10-21T17:18:45.951064+0000","last_active":"2023-10-21T17:18:46.959561+0000","last_peered":"2023-10-21T17:18:46.959561+0000","last_clean":"2023-10-21T17:18:46.959561+0000","last_became_active":"2023-10-21T17:18:45.950732+0000","last_became_peered":"2023-10-21T17:18:45.950732+0000","last_unstale":"2023-10-21T17:18:46.959561+0000","last_undegraded":"2023-10-21T17:18:46.959561+0000","last_fullsized":"2023-10-21T17:18:46.959561+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:44.883700+0000","last_clean_scrub_stamp":"2023-10-21T17:18:44.883700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:44:24.131908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"183'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-21T17:18:47.446802+0000","last_change":"2023-10-21T17:18:44.920467+0000","last_active":"2023-10-21T17:18:47.446802+0000","last_peered":"2023-10-21T17:18:47.446802+0000","last_clean":"2023-10-21T17:18:47.446802+0000","last_became_active":"2023-10-21T17:18:44.920041+0000","last_became_peered":"2023-10-21T17:18:44.920041+0000","last_unstale":"2023-10-21T17:18:47.446802+0000","last_undegraded":"2023-10-21T17:18:47.446802+0000","last_fullsized":"2023-10-21T17:18:47.446802+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:18:43.878827+0000","last_clean_scrub_stamp":"2023-10-21T17:18:43.878827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:21.953357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"171'4796","reported_seq":5476,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-21T17:18:44.920980+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:18:44.920980+0000","last_peered":"2023-10-21T17:18:44.920980+0000","last_clean":"2023-10-21T17:18:44.920980+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:18:44.920980+0000","last_undegraded":"2023-10-21T17:18:44.920980+0000","last_fullsized":"2023-10-21T17:18:44.920980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":447,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1552,"num_read_kb":3197,"num_write":447,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4796,"ondisk_log_size":4796,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52048,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319792,"statfs":{"total":96636764160,"available":96583467008,"internally_reserved":0,"allocated":2883584,"data_stored":1762797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.84099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5920000000000001,"5min":1.841,"15min":1.9430000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.65900000000000003},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2130000000000001,"15min":1.2370000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.68300000000000005},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.448},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.272,"15min":1.276},"min":{"1min":0.93700000000000006,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.839,"5min":1.839,"15min":2.081},"last":0.97699999999999998}]},{"osd":2,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3440000000000001,"15min":1.349},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.589,"5min":1.8140000000000001,"15min":2.1440000000000001},"last":1.304},{"interface":"front","average":{"1min":1.335,"5min":1.357,"15min":1.365},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88300000000000001},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.1579999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.304,"15min":1.306},"min":{"1min":1.0840000000000001,"5min":0.94999999999999996,"15min":0.81699999999999995},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.216},{"interface":"front","average":{"1min":1.466,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":0.95199999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.585}]},{"osd":4,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.755},"max":{"1min":1.6639999999999999,"5min":1.865,"15min":2.1269999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4650000000000001,"15min":1.484},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85899999999999999},"max":{"1min":1.7769999999999999,"5min":1.8540000000000001,"15min":2.0830000000000002},"last":1.5469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.80100000000000005},"max":{"1min":2.089,"5min":2.089,"15min":2.1560000000000001},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.52,"15min":1.5049999999999999},"min":{"1min":1.1439999999999999,"5min":0.93999999999999995,"15min":0.91900000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.369}]},{"osd":6,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5940000000000001,"15min":1.5920000000000001},"min":{"1min":1.2090000000000001,"5min":0.996,"15min":0.92400000000000004},"max":{"1min":1.9610000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.506},{"interface":"front","average":{"1min":1.446,"5min":1.4890000000000001,"15min":1.522},"min":{"1min":1.0680000000000001,"5min":1,"15min":0.92900000000000005},"max":{"1min":1.895,"5min":2.0150000000000001,"15min":2.1970000000000001},"last":1.631}]}]},{"osd":6,"up_from":13,"seq":55834575766,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41736,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330104,"statfs":{"total":96636764160,"available":96594026496,"internally_reserved":0,"allocated":2289664,"data_stored":1172477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.254,"15min":1.21},"min":{"1min":0.89600000000000002,"5min":0.82199999999999995,"15min":0.78400000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.827},"last":1.169},{"interface":"front","average":{"1min":1.177,"5min":1.2350000000000001,"15min":1.262},"min":{"1min":0.82099999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.8680000000000001},"last":1.073}]},{"osd":1,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.274,"15min":1.2549999999999999},"min":{"1min":0.98999999999999999,"5min":0.85299999999999998,"15min":0.80200000000000005},"max":{"1min":1.677,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.748},{"interface":"front","average":{"1min":1.302,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.95199999999999996,"5min":0.90900000000000003,"15min":0.86299999999999999},"max":{"1min":1.671,"5min":1.9359999999999999,"15min":1.984},"last":0.97099999999999997}]},{"osd":2,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3149999999999999,"15min":1.3180000000000001},"min":{"1min":0.90700000000000003,"5min":0.878,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":1.9970000000000001,"15min":2.0659999999999998},"last":1.8819999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.3320000000000001,"15min":1.3140000000000001},"min":{"1min":1.006,"5min":0.92100000000000004,"15min":0.70999999999999996},"max":{"1min":1.6040000000000001,"5min":1.7969999999999999,"15min":2.415},"last":1.595}]},{"osd":3,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.421,"15min":1.407},"min":{"1min":1.0509999999999999,"5min":1.034,"15min":0.81799999999999995},"max":{"1min":1.7709999999999999,"5min":1.9039999999999999,"15min":2.0150000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.4730000000000001,"15min":1.474},"min":{"1min":1.133,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":2.6509999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.369},"min":{"1min":0.96399999999999997,"5min":0.95099999999999996,"15min":0.63500000000000001},"max":{"1min":1.7,"5min":1.839,"15min":1.956},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4339999999999999,"15min":1.4139999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89100000000000001},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.161},"last":1.458}]},{"osd":5,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5880000000000001,"15min":1.5509999999999999},"min":{"1min":1.26,"5min":1.173,"15min":0.89600000000000002},"max":{"1min":1.998,"5min":2.1019999999999999,"15min":2.5419999999999998},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4259999999999999,"15min":1.387},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90400000000000003},"max":{"1min":1.8360000000000001,"5min":1.948,"15min":2.016},"last":1.9830000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.5309999999999999,"15min":1.532},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.71099999999999997},"max":{"1min":1.8999999999999999,"5min":2.1059999999999999,"15min":2.4049999999999998},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.746,"5min":1.681,"15min":1.6519999999999999},"min":{"1min":1.3819999999999999,"5min":1.181,"15min":1.0760000000000001},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":3.4180000000000001},"last":1.794}]}]},{"osd":5,"up_from":13,"seq":55834575767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2285568,"data_stored":1172407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1639999999999999,"15min":1.157},"min":{"1min":0.73499999999999999,"5min":0.69199999999999995,"15min":0.625},"max":{"1min":1.718,"5min":1.718,"15min":1.7649999999999999},"last":1.27},{"interface":"front","average":{"1min":1.28,"5min":1.236,"15min":1.2549999999999999},"min":{"1min":0.86299999999999999,"5min":0.79200000000000004,"15min":0.73999999999999999},"max":{"1min":1.661,"5min":1.726,"15min":1.9079999999999999},"last":1.2909999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2849999999999999,"15min":1.304},"min":{"1min":0.94999999999999996,"5min":0.876,"15min":0.85299999999999998},"max":{"1min":1.6679999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.268,"15min":1.2549999999999999},"min":{"1min":0.86499999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.357}]},{"osd":2,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.351,"15min":1.3400000000000001},"min":{"1min":0.98899999999999999,"5min":0.93999999999999995,"15min":0.90300000000000002},"max":{"1min":1.6970000000000001,"5min":1.885,"15min":1.927},"last":1.645},{"interface":"front","average":{"1min":1.292,"5min":1.278,"15min":1.282},"min":{"1min":1.02,"5min":0.81499999999999995,"15min":0.76000000000000001},"max":{"1min":1.6100000000000001,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.95899999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.367,"15min":1.3819999999999999},"min":{"1min":1.0920000000000001,"5min":1.008,"15min":0.84999999999999998},"max":{"1min":1.7390000000000001,"5min":1.7529999999999999,"15min":1.948},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.387,"15min":1.4239999999999999},"min":{"1min":1.0569999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.635,"5min":1.903,"15min":1.903},"last":1.2210000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2130000000000001,"15min":1.2390000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.718,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3520000000000001,"15min":1.379},"min":{"1min":0.93300000000000005,"5min":0.77300000000000002,"15min":0.70199999999999996},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.548}]},{"osd":6,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.454,"15min":1.478},"min":{"1min":0.92300000000000004,"5min":0.90000000000000002,"15min":0.89600000000000002},"max":{"1min":1.75,"5min":2.234,"15min":2.234},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.504},"min":{"1min":1.097,"5min":0.74199999999999999,"15min":0.67000000000000004},"max":{"1min":1.8380000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.8100000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.526,"15min":1.548},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":0.79000000000000004},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1549999999999998},"last":1.615},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.548,"15min":1.5609999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.012,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.869}]}]},{"osd":4,"up_from":13,"seq":55834575765,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45912,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325928,"statfs":{"total":96636764160,"available":96589750272,"internally_reserved":0,"allocated":2297856,"data_stored":1174019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2230000000000001,"15min":1.2},"min":{"1min":0.85299999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.212,"15min":1.2310000000000001},"min":{"1min":0.88800000000000001,"5min":0.85699999999999998,"15min":0.78600000000000003},"max":{"1min":1.633,"5min":1.907,"15min":1.907},"last":0.94699999999999995}]},{"osd":1,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3049999999999999,"15min":1.282},"min":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.81399999999999995},"max":{"1min":1.5880000000000001,"5min":1.6859999999999999,"15min":1.855},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2709999999999999,"15min":1.268},"min":{"1min":0.89700000000000002,"5min":0.88100000000000001,"15min":0.81499999999999995},"max":{"1min":1.587,"5min":1.794,"15min":1.9139999999999999},"last":1.264}]},{"osd":2,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.252,"15min":1.2589999999999999},"min":{"1min":0.93100000000000005,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.821},"last":1.153},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.403,"15min":1.3879999999999999},"min":{"1min":1.181,"5min":0.96499999999999997,"15min":0.95299999999999996},"max":{"1min":1.6319999999999999,"5min":1.923,"15min":1.923},"last":1.0660000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3560000000000001,"15min":1.353},"min":{"1min":0.999,"5min":0.96699999999999997,"15min":0.82799999999999996},"max":{"1min":1.7350000000000001,"5min":1.877,"15min":1.877},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.425,"15min":1.4119999999999999},"min":{"1min":1.0580000000000001,"5min":0.94499999999999995,"15min":0.89100000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.9299999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3160000000000001,"15min":1.3100000000000001},"min":{"1min":0.98099999999999998,"5min":0.88400000000000001,"15min":0.83199999999999996},"max":{"1min":1.611,"5min":1.802,"15min":1.895},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.56699999999999995},"max":{"1min":1.895,"5min":1.944,"15min":1.9870000000000001},"last":1.5649999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.429,"15min":1.4139999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9399999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.327},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6100000000000001,"15min":1.5900000000000001},"min":{"1min":1.4119999999999999,"5min":1.294,"15min":0.85599999999999998},"max":{"1min":1.871,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.367}]},{"osd":7,"last update":"Sat Oct 21 17:18:18 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5589999999999999,"15min":1.522},"min":{"1min":1.23,"5min":1.008,"15min":0.96599999999999997},"max":{"1min":2.0990000000000002,"5min":2.1110000000000002,"15min":2.2429999999999999},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.589,"5min":1.609,"15min":1.575},"min":{"1min":1.069,"5min":1.069,"15min":1.0109999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.417}]}]},{"osd":3,"up_from":13,"seq":55834575765,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51472,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320368,"statfs":{"total":96636764160,"available":96584056832,"internally_reserved":0,"allocated":2293760,"data_stored":1173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.95099999999999996,"15min":0.96399999999999997},"min":{"1min":0.72099999999999997,"5min":0.61799999999999999,"15min":0.47299999999999998},"max":{"1min":1.3300000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2949999999999999,"15min":1.288},"min":{"1min":1.0129999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.766,"5min":1.77,"15min":2.121},"last":1.071}]},{"osd":1,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.141,"15min":1.0960000000000001},"min":{"1min":0.78400000000000003,"5min":0.755,"15min":0.51600000000000001},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3029999999999999,"15min":1.3120000000000001},"min":{"1min":1.032,"5min":0.79800000000000004,"15min":0.70399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.5129999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.145,"15min":1.125},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.64500000000000002},"max":{"1min":1.726,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3009999999999999,"15min":1.2949999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.73499999999999999},"max":{"1min":1.6990000000000001,"5min":2.173,"15min":2.173},"last":1.3720000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":1.081,"5min":1.0509999999999999,"15min":0.73499999999999999},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.331},{"interface":"front","average":{"1min":1.542,"5min":1.508,"15min":1.4850000000000001},"min":{"1min":1.242,"5min":1.1870000000000001,"15min":0.98299999999999998},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":2.0009999999999999},"last":1.552}]},{"osd":5,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4630000000000001,"15min":1.492},"min":{"1min":1.131,"5min":1.0409999999999999,"15min":0.79400000000000004},"max":{"1min":2.0150000000000001,"5min":2.2210000000000001,"15min":2.3239999999999998},"last":1.599},{"interface":"front","average":{"1min":1.575,"5min":1.5569999999999999,"15min":1.5609999999999999},"min":{"1min":1.125,"5min":1.0780000000000001,"15min":1.054},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.2450000000000001},"last":1.462}]},{"osd":6,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.681,"5min":1.6200000000000001,"15min":1.6080000000000001},"min":{"1min":1.2230000000000001,"5min":1.0940000000000001,"15min":0.876},"max":{"1min":2.2210000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.663},{"interface":"front","average":{"1min":1.625,"5min":1.5589999999999999,"15min":1.587},"min":{"1min":1.157,"5min":1.1060000000000001,"15min":0.88500000000000001},"max":{"1min":2.0219999999999998,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.833}]},{"osd":7,"last update":"Sat Oct 21 17:18:16 2023","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.6699999999999999,"15min":1.6719999999999999},"min":{"1min":1.323,"5min":1.21,"15min":0.97399999999999998},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.4239999999999999},"last":1.627},{"interface":"front","average":{"1min":1.7969999999999999,"5min":1.7430000000000001,"15min":1.722},"min":{"1min":1.272,"5min":1.2629999999999999,"15min":0.93200000000000005},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.4729999999999999},"last":1.8009999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575766,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51480,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320360,"statfs":{"total":96636764160,"available":96584048640,"internally_reserved":0,"allocated":2306048,"data_stored":1179255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88,"15min":0.89700000000000002},"min":{"1min":0.69499999999999995,"5min":0.66000000000000003,"15min":0.65700000000000003},"max":{"1min":1.226,"5min":1.8859999999999999,"15min":2.3199999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.091,"5min":1.077,"15min":1.0600000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.72299999999999998},"max":{"1min":1.8180000000000001,"5min":1.835,"15min":1.875},"last":1.45}]},{"osd":1,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.157,"15min":1.1830000000000001},"min":{"1min":0.89300000000000002,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.9650000000000001},"last":1.377},{"interface":"front","average":{"1min":1.353,"5min":1.3939999999999999,"15min":1.4039999999999999},"min":{"1min":1.085,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1070000000000002},"last":1.143}]},{"osd":3,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2230000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.78000000000000003,"15min":0.76100000000000001},"max":{"1min":1.825,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.997},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.492,"15min":1.492},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.891,"5min":1.9450000000000001,"15min":2.1789999999999998},"last":1.52}]},{"osd":4,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.419,"15min":1.407},"min":{"1min":1.167,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.768,"5min":1.992,"15min":3.0510000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":1.0620000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":1.7230000000000001,"5min":2.0489999999999999,"15min":2.0579999999999998},"last":1.6379999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.446,"15min":1.417},"min":{"1min":1.0549999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.8069999999999999,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.196},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.599,"15min":1.6140000000000001},"min":{"1min":1.127,"5min":1.127,"15min":1.127},"max":{"1min":1.843,"5min":1.9590000000000001,"15min":3.2050000000000001},"last":1.5960000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.577,"15min":1.5629999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.0190000000000001,"5min":2.21,"15min":2.25},"last":1.76},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":1.1559999999999999,"5min":1.02,"15min":1.008},"max":{"1min":1.9930000000000001,"5min":2.4470000000000001,"15min":2.7389999999999999},"last":1.5620000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:17 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.6459999999999999,"15min":1.6479999999999999},"min":{"1min":1.2569999999999999,"5min":1.107,"15min":1.107},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.8959999999999999},"last":1.8440000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.605,"15min":1.603},"min":{"1min":1.264,"5min":1.085,"15min":1.085},"max":{"1min":1.9259999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.6930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575765,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29260,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342580,"statfs":{"total":96636764160,"available":96606801920,"internally_reserved":0,"allocated":2289664,"data_stored":1172477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.95999999999999996,"15min":0.95899999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.427,"5min":1.593,"15min":1.7330000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.107,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.3999999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.042,"15min":1.0569999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.724,"5min":1.724,"15min":2.79},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.202,"5min":1.298,"15min":1.2490000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.806,"5min":1.891,"15min":2.2549999999999999},"last":1.1559999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.387,"15min":1.395},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6870000000000001,"5min":1.9339999999999999,"15min":3.0920000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.252,"15min":1.282},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.627,"5min":1.75,"15min":1.976},"last":1.5649999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.365,"15min":1.3169999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.849,"5min":2.105,"15min":2.105},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5209999999999999,"15min":1.5089999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.966,"5min":2.2410000000000001,"15min":2.512},"last":1.591}]},{"osd":5,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.536,"15min":1.542},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.867},{"interface":"front","average":{"1min":1.532,"5min":1.5760000000000001,"15min":1.5489999999999999},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.958,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.532}]},{"osd":6,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5840000000000001,"15min":1.5580000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.9690000000000001,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.766},{"interface":"front","average":{"1min":1.544,"5min":1.597,"15min":1.5489999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.0489999999999999,"5min":2.2570000000000001,"15min":2.2570000000000001},"last":1.732}]},{"osd":7,"last update":"Sat Oct 21 17:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.6950000000000001,"15min":1.681},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.0979999999999999,"5min":2.3220000000000001,"15min":2.391},"last":1.821},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.7130000000000001,"15min":1.6910000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.129,"5min":2.4769999999999999,"15min":2.4969999999999999},"last":1.954}]}]},{"osd":0,"up_from":13,"seq":55834575765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29856,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341984,"statfs":{"total":96636764160,"available":96606191616,"internally_reserved":0,"allocated":2904064,"data_stored":1765093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96799999999999997,"15min":0.95299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5840000000000001,"5min":1.631,"15min":1.726},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.7989999999999999,"15min":2.6200000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1679999999999999,"15min":1.171},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.7330000000000001,"15min":2.8140000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.204,"15min":1.2070000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.46,"5min":1.7170000000000001,"15min":1.784},"last":1.3089999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.1060000000000001,"15min":1.147},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.718,"5min":1.883,"15min":1.8919999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6539999999999999,"5min":1.9239999999999999,"15min":2.6949999999999998},"last":1.4730000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.395},{"interface":"front","average":{"1min":1.637,"5min":1.6299999999999999,"15min":1.619},"min":{"1min":1.278,"5min":1.278,"15min":1.1799999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.766},"last":1.754}]},{"osd":5,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.96199999999999997},"max":{"1min":1.7729999999999999,"5min":2.125,"15min":2.6549999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5660000000000001,"15min":1.5600000000000001},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.952,"5min":2.3300000000000001,"15min":2.9209999999999998},"last":2.0310000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5229999999999999,"15min":1.5129999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2759999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.6990000000000001,"15min":1.665},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0230000000000001,"5min":2.4510000000000001,"15min":3.069},"last":2.145}]},{"osd":7,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.556,"15min":1.556},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.794,"5min":2.0910000000000002,"15min":2.4790000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.639,"15min":1.631},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.0529999999999999,"5min":2.1930000000000001,"15min":2.613},"last":1.607}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:18:50.999 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:18:51.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:18:51.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.278+0000 7f0482adc640 1 Processor -- start 2023-10-21T17:18:51.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.278+0000 7f0482adc640 1 -- start start 2023-10-21T17:18:51.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.279+0000 7f0482adc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f047c0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.279+0000 7f0482adc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0b0930 con 0x7f047c06a460 2023-10-21T17:18:51.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.279+0000 7f0482adc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047c0cf490 con 0x7f047c15cdc0 2023-10-21T17:18:51.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.279+0000 7f0482adc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0ceed0 con 0x7f047c0cd200 2023-10-21T17:18:51.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.280+0000 7f04812d9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f047c06a460 0x7f047c0b0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50048/0 (socket says 172.21.15.53:50048) 2023-10-21T17:18:51.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.280+0000 7f04812d9640 1 -- 172.21.15.53:0/3743072995 learned_addr learned my addr 172.21.15.53:0/3743072995 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:51.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.280+0000 7f04822db640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f047c0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.280+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382573612 0 0) 0x7f047c0b0930 con 0x7f047c06a460 2023-10-21T17:18:51.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.280+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0464003520 con 0x7f047c06a460 2023-10-21T17:18:51.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406607758 0 0) 0x7f0464003520 con 0x7f047c06a460 2023-10-21T17:18:51.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 msgr2=0x7f047c0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:51.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f047c0cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 >> v1:172.21.15.124:6789/0 conn(0x7f047c15cdc0 legacy=0x7f047c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c0cf1b0 con 0x7f047c06a460 2023-10-21T17:18:51.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0470001670 con 0x7f047c06a460 2023-10-21T17:18:51.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 >> v1:172.21.15.53:6789/0 conn(0x7f047c06a460 legacy=0x7f047c0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04640039a0 0x7f0464003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f04640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0464004d00 0x7f0464005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c0ceed0 con 0x7f04640039a0 2023-10-21T17:18:51.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046400bb20 con 0x7f0464004d00 2023-10-21T17:18:51.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046400bce0 con 0x7f047c0cd200 2023-10-21T17:18:51.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.281+0000 7f0481ada640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04640039a0 0x7f0464003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f04812d9640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f04640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f0481ada640 1 -- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 msgr2=0x7f04640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:51.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f0481ada640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047c0cd200 0x7f04640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f0481ada640 1 -- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0464004d00 msgr2=0x7f0464005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:51.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f0481ada640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0464004d00 0x7f0464005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.282+0000 7f0481ada640 1 -- 172.21.15.53:0/3743072995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0464003520 con 0x7f04640039a0 2023-10-21T17:18:51.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.283+0000 7f0481ada640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04640039a0 0x7f0464003dc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f046c007fd0 tx=0x7f04640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357ad3e84ff1f4dd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.283+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c003bc0 con 0x7f04640039a0 2023-10-21T17:18:51.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.283+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c003d20 con 0x7f04640039a0 2023-10-21T17:18:51.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.283+0000 7f0480ad8640 1 -- 172.21.15.53:0/3743072995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c005310 con 0x7f04640039a0 2023-10-21T17:18:51.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 -- 172.21.15.53:0/3743072995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04640039a0 msgr2=0x7f0464003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 --2- 172.21.15.53:0/3743072995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04640039a0 0x7f0464003dc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f046c007fd0 tx=0x7f04640057f0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f04822db640 1 -- 172.21.15.53:0/3743072995 reap_dead start 2023-10-21T17:18:51.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 -- 172.21.15.53:0/3743072995 shutdown_connections 2023-10-21T17:18:51.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 -- 172.21.15.53:0/3743072995 >> 172.21.15.53:0/3743072995 conn(0x7f047c098a50 msgr2=0x7f047c0cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:51.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 -- 172.21.15.53:0/3743072995 shutdown_connections 2023-10-21T17:18:51.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.284+0000 7f0482adc640 1 -- 172.21.15.53:0/3743072995 wait complete. 2023-10-21T17:18:51.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.286+0000 7f0482adc640 1 Processor -- start 2023-10-21T17:18:51.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.286+0000 7f0482adc640 1 -- start start 2023-10-21T17:18:51.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04640039a0 0x7f047c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 0x7f047c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1f3980 con 0x7f047c1f1fa0 2023-10-21T17:18:51.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1f3c60 con 0x7f047c1f4a20 2023-10-21T17:18:51.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f0482adc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1f3f40 con 0x7f04640039a0 2023-10-21T17:18:51.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f04812d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f04812d9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42448/0 (socket says 172.21.15.53:42448) 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.287+0000 7f04812d9640 1 -- 172.21.15.53:0/781859526 learned_addr learned my addr 172.21.15.53:0/781859526 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04822db640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 0x7f047c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f0481ada640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04640039a0 0x7f047c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 -- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04640039a0 msgr2=0x7f047c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04640039a0 0x7f047c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 -- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 msgr2=0x7f047c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 0x7f047c0c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 -- 172.21.15.53:0/781859526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c1f4220 con 0x7f047c1f1fa0 2023-10-21T17:18:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04822db640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 0x7f047c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f04812d9640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0470001050 tx=0x7f047000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000c870 con 0x7f047c1f1fa0 2023-10-21T17:18:51.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047000c9d0 con 0x7f047c1f1fa0 2023-10-21T17:18:51.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000cd80 con 0x7f047c1f1fa0 2023-10-21T17:18:51.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.288+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c1f4500 con 0x7f047c1f1fa0 2023-10-21T17:18:51.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.289+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c1f3730 con 0x7f047c1f1fa0 2023-10-21T17:18:51.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.290+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c04e160 con 0x7f047c1f1fa0 2023-10-21T17:18:51.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.291+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f047001d460 con 0x7f047c1f1fa0 2023-10-21T17:18:51.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.291+0000 7f047a7fc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 0x7f046806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.292+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f04700899a0 con 0x7f047c1f1fa0 2023-10-21T17:18:51.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.292+0000 7f0481ada640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 0x7f046806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.293+0000 7f0481ada640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 0x7f046806d4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f046c034000 tx=0x7f046c009030 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.296+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0470054760 con 0x7f047c1f1fa0 2023-10-21T17:18:51.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.559+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f047c0ccbc0 con 0x7f047c1f1fa0 2023-10-21T17:18:51.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.563+0000 7f047a7fc640 1 -- 172.21.15.53:0/781859526 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v43) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0470057d80 con 0x7f047c1f1fa0 2023-10-21T17:18:51.564 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:18:51.564 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:18:51.564 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:18:51.565 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:18:51.565 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:18:51.565 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:18:51.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.565+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 msgr2=0x7f046806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.565+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 0x7f046806d4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f046c034000 tx=0x7f046c009030 comp rx=0 tx=0).stop 2023-10-21T17:18:51.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.565+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 msgr2=0x7f047c1f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.565+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0470001050 tx=0x7f047000ec80 comp rx=0 tx=0).stop 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 shutdown_connections 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f046806afe0 0x7f046806d4a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f047c1f4a20 0x7f047c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c1f1fa0 0x7f047c1f3010 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 --2- 172.21.15.53:0/781859526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04640039a0 0x7f047c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 >> 172.21.15.53:0/781859526 conn(0x7f047c098a50 msgr2=0x7f047c0ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:51.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 shutdown_connections 2023-10-21T17:18:51.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.566+0000 7f0482adc640 1 -- 172.21.15.53:0/781859526 wait complete. 2023-10-21T17:18:51.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:18:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.797+0000 7f8bfd547640 1 Processor -- start 2023-10-21T17:18:51.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.798+0000 7f8bfd547640 1 -- start start 2023-10-21T17:18:51.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf815cc60 0x7f8bf80cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 0x7f8bf80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8bf80cdfd0 0x7f8bf80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815d040 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80d0650 con 0x7f8bf80d16f0 2023-10-21T17:18:51.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.799+0000 7f8bfd547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d0930 con 0x7f8bf815cc60 2023-10-21T17:18:51.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8bf80cdfd0 0x7f8bf80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8bf80cdfd0 0x7f8bf80ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42456/0 (socket says 172.21.15.53:42456) 2023-10-21T17:18:51.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 -- 172.21.15.53:0/2859600892 learned_addr learned my addr 172.21.15.53:0/2859600892 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:51.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bf7fff640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf815cc60 0x7f8bf80cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8beffff640 1 --2- 172.21.15.53:0/2859600892 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 0x7f8bf80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 -- 172.21.15.53:0/2859600892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf815cc60 msgr2=0x7f8bf80cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:51.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf815cc60 0x7f8bf80cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 -- 172.21.15.53:0/2859600892 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 msgr2=0x7f8bf80cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:51.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 --2- 172.21.15.53:0/2859600892 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 0x7f8bf80cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8bfcd46640 1 -- 172.21.15.53:0/2859600892 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80d0c10 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.800+0000 7f8beffff640 1 --2- 172.21.15.53:0/2859600892 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 0x7f8bf80cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:51.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.801+0000 7f8bfcd46640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80cdfd0 0x7f8bf80ce460 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f8bf808ceb0 tx=0x7f8bf002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be69c3ab29ccf55 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bf77fe640 1 -- 172.21.15.53:0/2859600892 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf002ca70 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bf77fe640 1 -- 172.21.15.53:0/2859600892 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf002cbd0 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bf77fe640 1 -- 172.21.15.53:0/2859600892 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00343f0 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 -- 172.21.15.53:0/2859600892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80cdfd0 msgr2=0x7f8bf80ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80cdfd0 0x7f8bf80ce460 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f8bf808ceb0 tx=0x7f8bf002bed0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 -- 172.21.15.53:0/2859600892 shutdown_connections 2023-10-21T17:18:51.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80cdfd0 0x7f8bf80ce460 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2859600892 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80d16f0 0x7f8bf80cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2859600892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf815cc60 0x7f8bf80cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 -- 172.21.15.53:0/2859600892 >> 172.21.15.53:0/2859600892 conn(0x7f8bf809bcf0 msgr2=0x7f8bf80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:51.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.802+0000 7f8bfd547640 1 -- 172.21.15.53:0/2859600892 shutdown_connections 2023-10-21T17:18:51.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.803+0000 7f8bfd547640 1 -- 172.21.15.53:0/2859600892 wait complete. 2023-10-21T17:18:51.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.804+0000 7f8bfd547640 1 Processor -- start 2023-10-21T17:18:51.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.804+0000 7f8bfd547640 1 -- start start 2023-10-21T17:18:51.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cdfd0 0x7f8bf815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf815cc60 0x7f8bf80afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf815e1d0 con 0x7f8bf80d16f0 2023-10-21T17:18:51.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf815b340 con 0x7f8bf815cc60 2023-10-21T17:18:51.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.805+0000 7f8bfd547640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf815e420 con 0x7f8bf80cdfd0 2023-10-21T17:18:51.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42462/0 (socket says 172.21.15.53:42462) 2023-10-21T17:18:51.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 -- 172.21.15.53:0/2275532011 learned_addr learned my addr 172.21.15.53:0/2275532011 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 -- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cdfd0 msgr2=0x7f8bf815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cdfd0 0x7f8bf815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 -- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf815cc60 msgr2=0x7f8bf80afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf815cc60 0x7f8bf80afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:51.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 -- 172.21.15.53:0/2275532011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0009b30 con 0x7f8bf80d16f0 2023-10-21T17:18:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.806+0000 7f8beffff640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c7f0 tx=0x7f8be000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.807+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0007820 con 0x7f8bf80d16f0 2023-10-21T17:18:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.807+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0007980 con 0x7f8bf80d16f0 2023-10-21T17:18:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.807+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf815e700 con 0x7f8bf80d16f0 2023-10-21T17:18:51.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.807+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0007d30 con 0x7f8bf80d16f0 2023-10-21T17:18:51.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.807+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf815ecc0 con 0x7f8bf80d16f0 2023-10-21T17:18:51.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.808+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf804e160 con 0x7f8bf80d16f0 2023-10-21T17:18:51.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.809+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8be00113f0 con 0x7f8bf80d16f0 2023-10-21T17:18:51.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.810+0000 7f8bf57fa640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 0x7f8bd806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:51.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.810+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8be0089e90 con 0x7f8bf80d16f0 2023-10-21T17:18:51.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.811+0000 7f8bf7fff640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 0x7f8bd806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:51.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.812+0000 7f8bf7fff640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 0x7f8bd806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006e20 tx=0x7f8be8008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:51.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:51.814+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8be0054a40 con 0x7f8bf80d16f0 2023-10-21T17:18:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.037+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bf80d3080 con 0x7f8bf80d16f0 2023-10-21T17:18:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.038+0000 7f8bf57fa640 1 -- 172.21.15.53:0/2275532011 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f8be0058060 con 0x7f8bf80d16f0 2023-10-21T17:18:52.039 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:18:52.039 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:18:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:18:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.040+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 msgr2=0x7f8bd806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.040+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 0x7f8bd806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006e20 tx=0x7f8be8008040 comp rx=0 tx=0).stop 2023-10-21T17:18:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.040+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 msgr2=0x7f8bf80af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.040+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c7f0 tx=0x7f8be000ccb0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 shutdown_connections 2023-10-21T17:18:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f8bd806af90 0x7f8bd806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf815cc60 0x7f8bf80afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bf80d16f0 0x7f8bf80af4e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 --2- 172.21.15.53:0/2275532011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cdfd0 0x7f8bf815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 >> 172.21.15.53:0/2275532011 conn(0x7f8bf809bcf0 msgr2=0x7f8bf80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 shutdown_connections 2023-10-21T17:18:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.041+0000 7f8bfd547640 1 -- 172.21.15.53:0/2275532011 wait complete. 2023-10-21T17:18:52.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:18:45.383517+0000', 'modified': '2023-10-21T17:18:48.467714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 37037}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_37037': {'gid': 37037, 'name': 'd', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/84861935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6834', 'nonce': 84861935}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 84861935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:18:52.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:18:52.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-21T17:18:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.700+0000 7f41cf59e640 1 Processor -- start 2023-10-21T17:18:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.700+0000 7f41cf59e640 1 -- start start 2023-10-21T17:18:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.701+0000 7f41cf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41cf59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d00b3c50 0x7f41d00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41cf59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d0159db0 con 0x7f41d01599d0 2023-10-21T17:18:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41cf59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d0071fe0 con 0x7f41d00b3c50 2023-10-21T17:18:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41cf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b44c0 con 0x7f41d00cb5e0 2023-10-21T17:18:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41ced9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d00b3c50 0x7f41d00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.702+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41d00b3c50 0x7f41d00cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:41668/0 (socket says 172.21.15.53:41668) 2023-10-21T17:18:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 -- 172.21.15.53:0/1275930454 learned_addr learned my addr 172.21.15.53:0/1275930454 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 -- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 msgr2=0x7f41d00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 --2- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 -- 172.21.15.53:0/1275930454 >> v1:172.21.15.124:6789/0 conn(0x7f41d01599d0 legacy=0x7f41d00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.703+0000 7f41ced9d640 1 -- 172.21.15.53:0/1275930454 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00713d0 con 0x7f41d00b3c50 2023-10-21T17:18:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.705+0000 7f41ced9d640 1 --2- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d00b3c50 0x7f41d00cc0c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009e50 tx=0x7f41c802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b3351c31fe0545 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.705+0000 7f41cdd9b640 1 -- 172.21.15.53:0/1275930454 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c802cb00 con 0x7f41d00b3c50 2023-10-21T17:18:52.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.705+0000 7f41cdd9b640 1 -- 172.21.15.53:0/1275930454 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c802cc60 con 0x7f41d00b3c50 2023-10-21T17:18:52.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.705+0000 7f41cdd9b640 1 -- 172.21.15.53:0/1275930454 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80343f0 con 0x7f41d00b3c50 2023-10-21T17:18:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 -- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d00b3c50 msgr2=0x7f41d00cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 --2- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d00b3c50 0x7f41d00cc0c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009e50 tx=0x7f41c802bf60 comp rx=0 tx=0).stop 2023-10-21T17:18:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 -- 172.21.15.53:0/1275930454 shutdown_connections 2023-10-21T17:18:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 --2- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 --2- 172.21.15.53:0/1275930454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d00b3c50 0x7f41d00cc0c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 -- 172.21.15.53:0/1275930454 >> 172.21.15.53:0/1275930454 conn(0x7f41d006c050 msgr2=0x7f41d006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 -- 172.21.15.53:0/1275930454 shutdown_connections 2023-10-21T17:18:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.706+0000 7f41cf59e640 1 -- 172.21.15.53:0/1275930454 wait complete. 2023-10-21T17:18:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.709+0000 7f41cf59e640 1 Processor -- start 2023-10-21T17:18:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.709+0000 7f41cf59e640 1 -- start start 2023-10-21T17:18:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d014c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d01599d0 0x7f41d014c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00ad1f0 con 0x7f41d00b3c50 2023-10-21T17:18:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00ad4d0 con 0x7f41d01599d0 2023-10-21T17:18:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.710+0000 7f41cf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b1670 con 0x7f41d00cb5e0 2023-10-21T17:18:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42476/0 (socket says 172.21.15.53:42476) 2023-10-21T17:18:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 -- 172.21.15.53:0/2704740819 learned_addr learned my addr 172.21.15.53:0/2704740819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ced9d640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d01599d0 0x7f41d014c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d014c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 -- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 msgr2=0x7f41d014c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d014c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 -- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d01599d0 msgr2=0x7f41d014c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d01599d0 0x7f41d014c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.711+0000 7f41ce59c640 1 -- 172.21.15.53:0/2704740819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c8009b30 con 0x7f41d00b3c50 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41ce59c640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f41c000c450 tx=0x7f41c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41c6ffd640 1 -- 172.21.15.53:0/2704740819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00052d0 con 0x7f41d00b3c50 2023-10-21T17:18:52.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41c6ffd640 1 -- 172.21.15.53:0/2704740819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c0005430 con 0x7f41d00b3c50 2023-10-21T17:18:52.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41c6ffd640 1 -- 172.21.15.53:0/2704740819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0012870 con 0x7f41d00b3c50 2023-10-21T17:18:52.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d00b1950 con 0x7f41d00b3c50 2023-10-21T17:18:52.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.712+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d0149f80 con 0x7f41d00b3c50 2023-10-21T17:18:52.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.714+0000 7f41c6ffd640 1 -- 172.21.15.53:0/2704740819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f41c00129d0 con 0x7f41d00b3c50 2023-10-21T17:18:52.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.715+0000 7f41c6ffd640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 0x7f41b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.715+0000 7f41c6ffd640 1 -- 172.21.15.53:0/2704740819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f41c00893b0 con 0x7f41d00b3c50 2023-10-21T17:18:52.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.716+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 0x7f41b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.716+0000 7f41d760d740 1 Processor -- start 2023-10-21T17:18:52.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.716+0000 7f41d760d740 1 -- start start 2023-10-21T17:18:52.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41c7fff640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 0x7f41b006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41d0157270 tx=0x7f41b8008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f2054fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f2054fb5b0 0x55f20552b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f20541b7a0 con 0x55f205528550 2023-10-21T17:18:52.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f20546d740 con 0x55f205528170 2023-10-21T17:18:52.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f20546ae30 con 0x55f2054fb5b0 2023-10-21T17:18:52.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.717+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42484/0 (socket says 172.21.15.53:42484) 2023-10-21T17:18:52.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 -- 172.21.15.53:0/4122265640 learned_addr learned my addr 172.21.15.53:0/4122265640 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41ce59c640 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f2054fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 -- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f2054fb5b0 msgr2=0x55f20552b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f2054fb5b0 0x55f20552b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 -- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 msgr2=0x55f2054fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f2054fb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 -- 172.21.15.53:0/4122265640 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f20542a910 con 0x55f205528550 2023-10-21T17:18:52.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41c7fff640 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f41b8011d00 tx=0x7f41b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e84c54dc36f264 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.718+0000 7f41affff640 1 -- 172.21.15.53:0/4122265640 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8011d30 con 0x55f205528550 2023-10-21T17:18:52.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41affff640 1 -- 172.21.15.53:0/4122265640 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8011e90 con 0x55f205528550 2023-10-21T17:18:52.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41affff640 1 -- 172.21.15.53:0/4122265640 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8021640 con 0x55f205528550 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 -- 172.21.15.53:0/4122265640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 msgr2=0x55f20552af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f41b8011d00 tx=0x7f41b8014000 comp rx=0 tx=0).stop 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 -- 172.21.15.53:0/4122265640 shutdown_connections 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f2054fb5b0 0x55f20552b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f20552af00 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 --2- 172.21.15.53:0/4122265640 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f2054fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 -- 172.21.15.53:0/4122265640 >> 172.21.15.53:0/4122265640 conn(0x55f2054fd380 msgr2=0x55f205545f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 -- 172.21.15.53:0/4122265640 shutdown_connections 2023-10-21T17:18:52.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.719+0000 7f41d760d740 1 -- 172.21.15.53:0/4122265640 wait complete. 2023-10-21T17:18:52.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.720+0000 7f41d760d740 1 -- start start 2023-10-21T17:18:52.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.720+0000 7f41d760d740 1 Processor -- start 2023-10-21T17:18:52.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.721+0000 7f41d760d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.721+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f205515e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.721+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f20553d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.721+0000 7f41d760d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f205466720 con 0x55f205528170 2023-10-21T17:18:52.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.721+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f205404970 con 0x55f205516540 2023-10-21T17:18:52.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f205404610 con 0x55f205528550 2023-10-21T17:18:52.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41d760d740 10 client.0 _ll_register_callbacks cb 0x55f2054269d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:18:52.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42498/0 (socket says 172.21.15.53:42498) 2023-10-21T17:18:52.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 -- 172.21.15.53:0/3114865437 learned_addr learned my addr 172.21.15.53:0/3114865437 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 -- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 msgr2=0x55f205515e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f205515e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 -- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 msgr2=0x55f20553d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ced9d640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f20553d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f20553d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.722+0000 7f41ce59c640 1 -- 172.21.15.53:0/3114865437 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b80119b0 con 0x55f205528170 2023-10-21T17:18:52.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ce59c640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f41c0087560 tx=0x7f41c00585f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9bb9b29f0c61e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:18:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00600b0 con 0x55f205528170 2023-10-21T17:18:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f41c0087670 con 0x55f205528170 2023-10-21T17:18:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f20548d160 con 0x55f205528170 2023-10-21T17:18:52.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c0087170 con 0x55f205528170 2023-10-21T17:18:52.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 10 client.9777 fetch_fsmap learned FSMap version 86 2023-10-21T17:18:52.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55f205464da0 con 0x55f205528170 2023-10-21T17:18:52.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f20548d160 con 0x55f205528170 2023-10-21T17:18:52.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f41c0019020 con 0x55f205528170 2023-10-21T17:18:52.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 10 client.9777 fetch_fsmap finished waiting for FSMap version 86 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 10 client.9777 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 20 client.9777 populate_metadata read hostname 'smithi053' 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205518ba0 0x55f20552d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 4 client.9777 mds_command: new command op to 37037 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.723+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 --> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f205438cc0 con 0x55f205518ba0 2023-10-21T17:18:52.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.724+0000 7f41c7fff640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205518ba0 0x55f20552d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.725+0000 7f41c7fff640 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205518ba0 0x55f20552d1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f536682fb9275b7 server_cookie=8a3a9d5173748764 in_seq=0 out_seq=0 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.725+0000 7f41ad7fa640 10 client.9777 ms_handle_connect on v2:172.21.15.124:6834/84861935 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41ad7fa640 1 -- 172.21.15.53:0/3114865437 <== mds.0 v2:172.21.15.124:6834/84861935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f205438cc0 con 0x55f205518ba0 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41ad7fa640 10 client.9777 handle_command_reply: tid=0 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 client.9777 shutdown 2023-10-21T17:18:52.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 msgr2=0x55f205517f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f41c0087560 tx=0x7f41c00585f0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 shutdown_connections 2023-10-21T17:18:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205518ba0 0x55f20552d1c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f20553d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f205515e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 --2- 172.21.15.53:0/3114865437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f205517f50 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41ad7fa640 0 client.9777 ms_handle_reset on v2:172.21.15.124:6834/84861935 2023-10-21T17:18:52.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 >> 172.21.15.53:0/3114865437 conn(0x55f2054fb5b0 msgr2=0x55f2054fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 shutdown_connections 2023-10-21T17:18:52.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 1 -- 172.21.15.53:0/3114865437 wait complete. 2023-10-21T17:18:52.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.727+0000 7f41d760d740 20 client.9777 trim_cache size 0 max 16384 2023-10-21T17:18:52.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.745+0000 7f41d760d740 1 Processor -- start 2023-10-21T17:18:52.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.745+0000 7f41d760d740 1 -- start start 2023-10-21T17:18:52.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205516540 0x55f20552e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f205525670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2051caf20 con 0x55f205528550 2023-10-21T17:18:52.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2054fea30 con 0x55f205528170 2023-10-21T17:18:52.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f205528d20 con 0x55f205516540 2023-10-21T17:18:52.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.746+0000 7f41ced9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42504/0 (socket says 172.21.15.53:42504) 2023-10-21T17:18:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 -- 172.21.15.53:0/3451752116 learned_addr learned my addr 172.21.15.53:0/3451752116 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ce59c640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205516540 0x55f20552e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41c7fff640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f205525670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 -- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205516540 msgr2=0x55f20552e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205516540 0x55f20552e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 -- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 msgr2=0x55f205525670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f205525670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 -- 172.21.15.53:0/3451752116 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f205403b80 con 0x55f205528550 2023-10-21T17:18:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41c7fff640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f205525670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:52.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.747+0000 7f41ced9d640 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f41c802c570 tx=0x7f41c802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746f2ede8d21277e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41c4ff9640 1 -- 172.21.15.53:0/3451752116 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c802cb00 con 0x55f205528550 2023-10-21T17:18:52.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41c4ff9640 1 -- 172.21.15.53:0/3451752116 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8047440 con 0x55f205528550 2023-10-21T17:18:52.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41c4ff9640 1 -- 172.21.15.53:0/3451752116 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80477f0 con 0x55f205528550 2023-10-21T17:18:52.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 -- 172.21.15.53:0/3451752116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 msgr2=0x55f205525bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f41c802c570 tx=0x7f41c802e070 comp rx=0 tx=0).stop 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 -- 172.21.15.53:0/3451752116 shutdown_connections 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528550 0x55f205525bb0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205528170 0x55f205525670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 --2- 172.21.15.53:0/3451752116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205516540 0x55f20552e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 -- 172.21.15.53:0/3451752116 >> 172.21.15.53:0/3451752116 conn(0x55f2054fb5b0 msgr2=0x55f205502130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 -- 172.21.15.53:0/3451752116 shutdown_connections 2023-10-21T17:18:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.748+0000 7f41d760d740 1 -- 172.21.15.53:0/3451752116 wait complete. 2023-10-21T17:18:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.749+0000 7f41d760d740 1 -- start start 2023-10-21T17:18:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.749+0000 7f41d760d740 1 Processor -- start 2023-10-21T17:18:52.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f2055006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f2055b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2055b7f60 con 0x55f205528170 2023-10-21T17:18:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f20555fce0 con 0x55f205516540 2023-10-21T17:18:52.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2054feb70 con 0x55f205528550 2023-10-21T17:18:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41d760d740 10 client.0 _ll_register_callbacks cb 0x7f41c00011e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:18:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.750+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42516/0 (socket says 172.21.15.53:42516) 2023-10-21T17:18:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 -- 172.21.15.53:0/363124875 learned_addr learned my addr 172.21.15.53:0/363124875 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:52.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 -- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 msgr2=0x55f2055b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f2055b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 -- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 msgr2=0x55f2055006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f2055006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 -- 172.21.15.53:0/363124875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d00713d0 con 0x55f205528170 2023-10-21T17:18:52.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41c7fff640 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f41b8023040 tx=0x7f41b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b77d73e43cb9e95 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:18:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.751+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8031050 con 0x55f205528170 2023-10-21T17:18:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f41b80342c0 con 0x55f205528170 2023-10-21T17:18:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41b8011e90 con 0x55f205528170 2023-10-21T17:18:52.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b8034480 con 0x55f205528170 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 10 client.9783 fetch_fsmap learned FSMap version 86 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55f205429170 con 0x55f205528170 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b8011e90 con 0x55f205528170 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f41b8034820 con 0x55f205528170 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 10 client.9783 fetch_fsmap finished waiting for FSMap version 86 2023-10-21T17:18:52.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 10 client.9783 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:18:52.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 20 client.9783 populate_metadata read hostname 'smithi053' 2023-10-21T17:18:52.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205545da0 0x55f205546180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 4 client.9783 mds_command: new command op to 37037 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:18:52.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.752+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 --> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f20545f700 con 0x55f205545da0 2023-10-21T17:18:52.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.753+0000 7f41ce59c640 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205545da0 0x55f205546180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.754+0000 7f41ce59c640 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205545da0 0x55f205546180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e0b26b4e40edab server_cookie=4d99c9812092987c in_seq=0 out_seq=0 2023-10-21T17:18:52.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.754+0000 7f41ad7fa640 10 client.9783 ms_handle_connect on v2:172.21.15.124:6834/84861935 2023-10-21T17:18:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.754+0000 7f41ad7fa640 1 -- 172.21.15.53:0/363124875 <== mds.0 v2:172.21.15.124:6834/84861935 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f20545f700 con 0x55f205545da0 2023-10-21T17:18:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.754+0000 7f41ad7fa640 10 client.9783 handle_command_reply: tid=0 2023-10-21T17:18:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.754+0000 7f41d760d740 1 client.9783 shutdown 2023-10-21T17:18:52.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 msgr2=0x55f2055b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f41b8023040 tx=0x7f41b80046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 shutdown_connections 2023-10-21T17:18:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f205528550 0x55f2055b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f205528170 0x55f2055b7160 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:6834/84861935,v1:172.21.15.124:6835/84861935] conn(0x55f205545da0 0x55f205546180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 --2- 172.21.15.53:0/363124875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f205516540 0x55f2055006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 >> 172.21.15.53:0/363124875 conn(0x55f2054fb5b0 msgr2=0x55f205502130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41ad7fa640 0 client.9783 ms_handle_reset on v2:172.21.15.124:6834/84861935 2023-10-21T17:18:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 shutdown_connections 2023-10-21T17:18:52.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 1 -- 172.21.15.53:0/363124875 wait complete. 2023-10-21T17:18:52.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.755+0000 7f41d760d740 20 client.9783 trim_cache size 0 max 16384 2023-10-21T17:18:52.768 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:18:52.768 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:18:52.768 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 37037, 2023-10-21T17:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:18:52.769 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8074150119999999, 2023-10-21T17:18:52.770 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 86, 2023-10-21T17:18:52.770 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 184, 2023-10-21T17:18:52.770 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 184, 2023-10-21T17:18:52.770 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.169632481000001 2023-10-21T17:18:52.770 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:18:52.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 msgr2=0x7f41b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 0x7f41b006d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41d0157270 tx=0x7f41b8008040 comp rx=0 tx=0).stop 2023-10-21T17:18:52.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 msgr2=0x7f41d0149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:52.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f41c000c450 tx=0x7f41c000c910 comp rx=0 tx=0).stop 2023-10-21T17:18:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 shutdown_connections 2023-10-21T17:18:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f41b006af90 0x7f41b006d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41d01599d0 0x7f41d014c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41d00cb5e0 0x7f41d014c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 --2- 172.21.15.53:0/2704740819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41d00b3c50 0x7f41d0149a40 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:52.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 >> 172.21.15.53:0/2704740819 conn(0x7f41d006c050 msgr2=0x7f41d015a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:52.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 shutdown_connections 2023-10-21T17:18:52.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.756+0000 7f41cf59e640 1 -- 172.21.15.53:0/2704740819 wait complete. 2023-10-21T17:18:52.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:18:52.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.994+0000 7f3fd6aec640 1 Processor -- start 2023-10-21T17:18:52.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.994+0000 7f3fd6aec640 1 -- start start 2023-10-21T17:18:52.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd6aec640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3fd00cb140 0x7f3fd0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd6aec640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3fd006a290 0x7f3fd00cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:52.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd6aec640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd006a6f0 con 0x7f3fd015cb20 2023-10-21T17:18:52.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd6aec640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00cb5a0 con 0x7f3fd00cb140 2023-10-21T17:18:52.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd6aec640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00cacb0 con 0x7f3fd006a290 2023-10-21T17:18:52.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd5aea640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3fd015cb20 0x7f3fd00cb950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50064/0 (socket says 172.21.15.53:50064) 2023-10-21T17:18:52.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd5aea640 1 -- 172.21.15.53:0/3704628403 learned_addr learned my addr 172.21.15.53:0/3704628403 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:52.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.996+0000 7f3fd52e9640 1 --2- 172.21.15.53:0/3704628403 >> v2:172.21.15.53:3300/0 conn(0x7f3fd00cb140 0x7f3fd0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:53.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd4ae8640 1 -- 172.21.15.53:0/3704628403 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785068484 0 0) 0x7f3fd006a6f0 con 0x7f3fd015cb20 2023-10-21T17:18:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd4ae8640 1 -- 172.21.15.53:0/3704628403 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fbc003520 con 0x7f3fd015cb20 2023-10-21T17:18:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd52e9640 1 -- 172.21.15.53:0/3704628403 >> v2:172.21.15.124:3300/0 conn(0x7f3fd006a290 msgr2=0x7f3fd00cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd62eb640 1 --2- 172.21.15.53:0/3704628403 >> v2:172.21.15.124:3300/0 conn(0x7f3fd006a290 0x7f3fd00cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd52e9640 1 --2- 172.21.15.53:0/3704628403 >> v2:172.21.15.124:3300/0 conn(0x7f3fd006a290 0x7f3fd00cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd52e9640 1 -- 172.21.15.53:0/3704628403 >> v1:172.21.15.53:6789/0 conn(0x7f3fd015cb20 legacy=0x7f3fd00cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.997+0000 7f3fd52e9640 1 -- 172.21.15.53:0/3704628403 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00c9dc0 con 0x7f3fd00cb140 2023-10-21T17:18:53.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.998+0000 7f3fd52e9640 1 --2- 172.21.15.53:0/3704628403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd00cb140 0x7f3fd0069d50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0072020 tx=0x7f3fc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd9ffa4c2d8cacd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:53.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.998+0000 7f3fd4ae8640 1 -- 172.21.15.53:0/3704628403 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0031830 con 0x7f3fd00cb140 2023-10-21T17:18:53.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.998+0000 7f3fd4ae8640 1 -- 172.21.15.53:0/3704628403 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0031990 con 0x7f3fd00cb140 2023-10-21T17:18:53.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.998+0000 7f3fd4ae8640 1 -- 172.21.15.53:0/3704628403 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0031d20 con 0x7f3fd00cb140 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 -- 172.21.15.53:0/3704628403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd00cb140 msgr2=0x7f3fd0069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/3704628403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd00cb140 0x7f3fd0069d50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0072020 tx=0x7f3fc002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 -- 172.21.15.53:0/3704628403 shutdown_connections 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/3704628403 >> v2:172.21.15.124:3300/0 conn(0x7f3fd006a290 0x7f3fd00cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/3704628403 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd00cb140 0x7f3fd0069d50 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 -- 172.21.15.53:0/3704628403 >> 172.21.15.53:0/3704628403 conn(0x7f3fd0071790 msgr2=0x7f3fd0098a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:53.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 -- 172.21.15.53:0/3704628403 shutdown_connections 2023-10-21T17:18:53.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:52.999+0000 7f3fd6aec640 1 -- 172.21.15.53:0/3704628403 wait complete. 2023-10-21T17:18:53.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.001+0000 7f3fd6aec640 1 Processor -- start 2023-10-21T17:18:53.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.001+0000 7f3fd6aec640 1 -- start start 2023-10-21T17:18:53.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:53.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fd00cb140 0x7f3fd00c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:53.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fd015cb20 0x7f3fd00c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:53.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00b4510 con 0x7f3fd006a290 2023-10-21T17:18:53.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00c7610 con 0x7f3fd00cb140 2023-10-21T17:18:53.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd6aec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00b4310 con 0x7f3fd015cb20 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd5aea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd5aea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:42532/0 (socket says 172.21.15.53:42532) 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.002+0000 7f3fd5aea640 1 -- 172.21.15.53:0/1706971268 learned_addr learned my addr 172.21.15.53:0/1706971268 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 -- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fd015cb20 msgr2=0x7f3fd00c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fd015cb20 0x7f3fd00c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 -- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fd00cb140 msgr2=0x7f3fd00c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fd00cb140 0x7f3fd00c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 -- 172.21.15.53:0/1706971268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009400 con 0x7f3fd006a290 2023-10-21T17:18:53.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd5aea640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c410 tx=0x7f3fcc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:53.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc005ce0 con 0x7f3fd006a290 2023-10-21T17:18:53.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc005e40 con 0x7f3fd006a290 2023-10-21T17:18:53.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0072f0 con 0x7f3fd006a290 2023-10-21T17:18:53.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd0148cc0 con 0x7f3fd006a290 2023-10-21T17:18:53.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.003+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00c4db0 con 0x7f3fd006a290 2023-10-21T17:18:53.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.005+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd00c9c10 con 0x7f3fd006a290 2023-10-21T17:18:53.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.005+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3fcc007450 con 0x7f3fd006a290 2023-10-21T17:18:53.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.006+0000 7f3fc67fc640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 0x7f3fb406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:53.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.007+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3fcc088fa0 con 0x7f3fd006a290 2023-10-21T17:18:53.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.007+0000 7f3fd52e9640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 0x7f3fb406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:53.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.008+0000 7f3fd52e9640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 0x7f3fb406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0036000 tx=0x7f3fc0030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:53.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.011+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fcc0578c0 con 0x7f3fd006a290 2023-10-21T17:18:53.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.246+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3fd00b4650 con 0x7f3fd006a290 2023-10-21T17:18:53.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.247+0000 7f3fc67fc640 1 -- 172.21.15.53:0/1706971268 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3fd00b4650 con 0x7f3fd006a290 2023-10-21T17:18:53.248 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:18:53.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 msgr2=0x7f3fb406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 0x7f3fb406d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0036000 tx=0x7f3fc0030000 comp rx=0 tx=0).stop 2023-10-21T17:18:53.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 msgr2=0x7f3fd0142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:53.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c410 tx=0x7f3fcc00c8d0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 shutdown_connections 2023-10-21T17:18:53.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f3fb406afc0 0x7f3fb406d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fd015cb20 0x7f3fd00c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fd00cb140 0x7f3fd00c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 --2- 172.21.15.53:0/1706971268 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fd006a290 0x7f3fd0142e10 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:53.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 >> 172.21.15.53:0/1706971268 conn(0x7f3fd0071790 msgr2=0x7f3fd00c5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:53.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 shutdown_connections 2023-10-21T17:18:53.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:53.250+0000 7f3fd6aec640 1 -- 172.21.15.53:0/1706971268 wait complete. 2023-10-21T17:18:53.271 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:18:53.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:18:53.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:18:53.272 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:18:53.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:18:53.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:18:53.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:18:53.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:18:53.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:18:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.275 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:18:53.329 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:18:53.329 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:18:53.329 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:18:53.329 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.330 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:18:53.330 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.330 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:18:53.330 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:18:53.330 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:18:53.331 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:18:53.332 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:18:53.332 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 37760sec preferred_lft 37760sec 2023-10-21T17:18:53.332 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:18:53.332 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.332 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:18:53.333 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:18:53.333 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:18:53.333 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:18:53.333 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:18:53.334 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:18:53.334 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.334 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:18:53.334 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.334 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:18:53.335 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:18:53.335 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:18:53.335 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.335 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:18:53.335 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:18:53.336 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:18:53.336 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:18:53.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.337 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:18:53.433 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:18:53.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.438 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:18:53.496 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:18:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:18:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:18:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:18:53.497 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:18:53.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.498 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:18:53.499 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:18:53.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:18:53.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:18:53.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:18:53.500 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:18:53.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.703 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:18:53.758 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:18:53.759 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:18:53.759 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:18:53.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.760 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:18:53.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:18:53.855 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:18:53.915 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:18:53.915 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:18:53.916 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:18:53.973 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:18:53.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:53.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:18:54.124 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:18:54.124 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:18:54.164 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:18:54.164 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:18:54.164 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:18:54.165 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:18:54.165 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:18:54.168 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:18:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:18:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:18:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:18:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:18:54.172 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:18:54.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:54.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:18:54.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:54.266 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:18:54.347 INFO:teuthology.orchestra.run.smithi053.stdout:client37094 2023-10-21T17:18:54.347 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:18:54.348 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:18:54.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:18:54.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.645+0000 7ff22504d640 1 Processor -- start 2023-10-21T17:18:54.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.645+0000 7ff22504d640 1 -- start start 2023-10-21T17:18:54.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.646+0000 7ff22504d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff2200c8b40 0x7ff2200c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.646+0000 7ff22504d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200aff70 0x7ff2200b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.646+0000 7ff22504d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2200c94f0 con 0x7ff22015cae0 2023-10-21T17:18:54.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.646+0000 7ff22504d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200ccbc0 con 0x7ff2200c8b40 2023-10-21T17:18:54.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.646+0000 7ff22504d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b0940 con 0x7ff2200aff70 2023-10-21T17:18:54.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff22015cae0 0x7ff2200cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49348/0 (socket says 172.21.15.53:49348) 2023-10-21T17:18:54.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21ffff640 1 -- 172.21.15.53:0/3316652943 learned_addr learned my addr 172.21.15.53:0/3316652943 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:18:54.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21f7fe640 1 --2- 172.21.15.53:0/3316652943 >> v2:172.21.15.53:3300/0 conn(0x7ff2200c8b40 0x7ff2200c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff22484c640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200aff70 0x7ff2200b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21effd640 1 -- 172.21.15.53:0/3316652943 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383568910 0 0) 0x7ff2200c94f0 con 0x7ff22015cae0 2023-10-21T17:18:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21effd640 1 -- 172.21.15.53:0/3316652943 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff208003520 con 0x7ff22015cae0 2023-10-21T17:18:54.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21f7fe640 1 -- 172.21.15.53:0/3316652943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200aff70 msgr2=0x7ff2200b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21f7fe640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200aff70 0x7ff2200b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21f7fe640 1 -- 172.21.15.53:0/3316652943 >> v1:172.21.15.53:6789/0 conn(0x7ff22015cae0 legacy=0x7ff2200cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.647+0000 7ff21f7fe640 1 -- 172.21.15.53:0/3316652943 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200ce1f0 con 0x7ff2200c8b40 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.649+0000 7ff21f7fe640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200c8b40 0x7ff2200c8fb0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff2140097a0 tx=0x7ff21402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5178ae205202e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.649+0000 7ff21effd640 1 -- 172.21.15.53:0/3316652943 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21402d040 con 0x7ff2200c8b40 2023-10-21T17:18:54.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.649+0000 7ff21effd640 1 -- 172.21.15.53:0/3316652943 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214031810 con 0x7ff2200c8b40 2023-10-21T17:18:54.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.649+0000 7ff21effd640 1 -- 172.21.15.53:0/3316652943 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214031ba0 con 0x7ff2200c8b40 2023-10-21T17:18:54.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 -- 172.21.15.53:0/3316652943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200c8b40 msgr2=0x7ff2200c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200c8b40 0x7ff2200c8fb0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff2140097a0 tx=0x7ff21402b8b0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 -- 172.21.15.53:0/3316652943 shutdown_connections 2023-10-21T17:18:54.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200aff70 0x7ff2200b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 --2- 172.21.15.53:0/3316652943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200c8b40 0x7ff2200c8fb0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 -- 172.21.15.53:0/3316652943 >> 172.21.15.53:0/3316652943 conn(0x7ff22006c050 msgr2=0x7ff22006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:54.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 -- 172.21.15.53:0/3316652943 shutdown_connections 2023-10-21T17:18:54.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.650+0000 7ff22504d640 1 -- 172.21.15.53:0/3316652943 wait complete. 2023-10-21T17:18:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.652+0000 7ff22504d640 1 Processor -- start 2023-10-21T17:18:54.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.652+0000 7ff22504d640 1 -- start start 2023-10-21T17:18:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200c8b40 0x7ff22015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff22015cae0 0x7ff22014d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c7380 con 0x7ff2200aff70 2023-10-21T17:18:54.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201572a0 con 0x7ff22015cae0 2023-10-21T17:18:54.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff22504d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22015b6b0 con 0x7ff2200c8b40 2023-10-21T17:18:54.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:54.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37712/0 (socket says 172.21.15.53:37712) 2023-10-21T17:18:54.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 -- 172.21.15.53:0/677279659 learned_addr learned my addr 172.21.15.53:0/677279659 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:54.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 -- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200c8b40 msgr2=0x7ff22015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200c8b40 0x7ff22015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 -- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff22015cae0 msgr2=0x7ff22014d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff22015cae0 0x7ff22014d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.653+0000 7ff21ffff640 1 -- 172.21.15.53:0/677279659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214009400 con 0x7ff2200aff70 2023-10-21T17:18:54.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff21ffff640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cbc0 tx=0x7ff21000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000ad40 con 0x7ff2200aff70 2023-10-21T17:18:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210012400 con 0x7ff2200aff70 2023-10-21T17:18:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000aea0 con 0x7ff2200aff70 2023-10-21T17:18:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2200c6700 con 0x7ff2200aff70 2023-10-21T17:18:54.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.654+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2200c6cc0 con 0x7ff2200aff70 2023-10-21T17:18:54.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.655+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff220074150 con 0x7ff2200aff70 2023-10-21T17:18:54.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.656+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff210007070 con 0x7ff2200aff70 2023-10-21T17:18:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.657+0000 7ff21cff9640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 0x7ff20c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.657+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff210089910 con 0x7ff2200aff70 2023-10-21T17:18:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.658+0000 7ff21f7fe640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 0x7ff20c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:54.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.659+0000 7ff21f7fe640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 0x7ff20c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff214030000 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:54.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.662+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff210058220 con 0x7ff2200aff70 2023-10-21T17:18:54.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.882+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 --> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2200cc570 con 0x7ff20c06af90 2023-10-21T17:18:54.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.886+0000 7ff21cff9640 1 -- 172.21.15.53:0/677279659 <== mgr.36914 v2:172.21.15.124:6832/234236878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff214031a80 con 0x7ff20c06af90 2023-10-21T17:18:54.887 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:18:54.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:18:54.887 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:18:54.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:18:54.888 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:18:54.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.888+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 msgr2=0x7ff20c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.888+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 0x7ff20c06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff214030000 comp rx=0 tx=0).stop 2023-10-21T17:18:54.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.888+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 msgr2=0x7ff2200b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:54.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.888+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff21000cbc0 tx=0x7ff21000a430 comp rx=0 tx=0).stop 2023-10-21T17:18:54.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 shutdown_connections 2023-10-21T17:18:54.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7ff20c06af90 0x7ff20c06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff22015cae0 0x7ff22014d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2200c8b40 0x7ff22015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 --2- 172.21.15.53:0/677279659 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2200aff70 0x7ff2200b0d50 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:54.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 >> 172.21.15.53:0/677279659 conn(0x7ff22006c050 msgr2=0x7ff2200caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:54.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 shutdown_connections 2023-10-21T17:18:54.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:54.889+0000 7ff22504d640 1 -- 172.21.15.53:0/677279659 wait complete. 2023-10-21T17:18:54.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:18:55.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.136+0000 7fdd1daa2640 1 Processor -- start 2023-10-21T17:18:55.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.136+0000 7fdd1daa2640 1 -- start start 2023-10-21T17:18:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.137+0000 7fdd1daa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd180b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd1daa2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1806dfc0 0x7fdd18070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd1daa2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd180b2620 con 0x7fdd18071070 2023-10-21T17:18:55.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd1daa2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd180b2060 con 0x7fdd1806dfc0 2023-10-21T17:18:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd1daa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180b2340 con 0x7fdd180afbe0 2023-10-21T17:18:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd0ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1806dfc0 0x7fdd18070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd0ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1806dfc0 0x7fdd18070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37724/0 (socket says 172.21.15.53:37724) 2023-10-21T17:18:55.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd0ffff640 1 -- 172.21.15.53:0/1700182244 learned_addr learned my addr 172.21.15.53:0/1700182244 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd177fe640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd180b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.138+0000 7fdd0ffff640 1 -- 172.21.15.53:0/1700182244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 msgr2=0x7fdd180b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:55.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.139+0000 7fdd0ffff640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd180b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.139+0000 7fdd0ffff640 1 -- 172.21.15.53:0/1700182244 >> v1:172.21.15.124:6789/0 conn(0x7fdd18071070 legacy=0x7fdd180cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:55.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.139+0000 7fdd0ffff640 1 -- 172.21.15.53:0/1700182244 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd18070350 con 0x7fdd1806dfc0 2023-10-21T17:18:55.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.140+0000 7fdd0ffff640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1806dfc0 0x7fdd18070b30 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc1c0c455a9a8d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:55.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.140+0000 7fdd16ffd640 1 -- 172.21.15.53:0/1700182244 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031640 con 0x7fdd1806dfc0 2023-10-21T17:18:55.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.140+0000 7fdd16ffd640 1 -- 172.21.15.53:0/1700182244 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd000317a0 con 0x7fdd1806dfc0 2023-10-21T17:18:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.140+0000 7fdd16ffd640 1 -- 172.21.15.53:0/1700182244 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031b30 con 0x7fdd1806dfc0 2023-10-21T17:18:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 -- 172.21.15.53:0/1700182244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1806dfc0 msgr2=0x7fdd18070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1806dfc0 0x7fdd18070b30 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd0002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 -- 172.21.15.53:0/1700182244 shutdown_connections 2023-10-21T17:18:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1806dfc0 0x7fdd18070b30 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/1700182244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd180b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 -- 172.21.15.53:0/1700182244 >> 172.21.15.53:0/1700182244 conn(0x7fdd1806a5d0 msgr2=0x7fdd180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 -- 172.21.15.53:0/1700182244 shutdown_connections 2023-10-21T17:18:55.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.141+0000 7fdd1daa2640 1 -- 172.21.15.53:0/1700182244 wait complete. 2023-10-21T17:18:55.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.143+0000 7fdd1daa2640 1 Processor -- start 2023-10-21T17:18:55.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.143+0000 7fdd1daa2640 1 -- start start 2023-10-21T17:18:55.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1806dfc0 0x7fdd180cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd181f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180cd090 con 0x7fdd18071070 2023-10-21T17:18:55.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180cd370 con 0x7fdd1806dfc0 2023-10-21T17:18:55.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd1daa2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180cd650 con 0x7fdd180afbe0 2023-10-21T17:18:55.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:55.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37728/0 (socket says 172.21.15.53:37728) 2023-10-21T17:18:55.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 -- 172.21.15.53:0/2005351944 learned_addr learned my addr 172.21.15.53:0/2005351944 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd177fe640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1806dfc0 0x7fdd180cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 -- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 msgr2=0x7fdd181f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd181f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 -- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1806dfc0 msgr2=0x7fdd180cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1806dfc0 0x7fdd180cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:55.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.144+0000 7fdd0ffff640 1 -- 172.21.15.53:0/2005351944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000094d0 con 0x7fdd18071070 2023-10-21T17:18:55.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd0ffff640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:55.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0003ac60 con 0x7fdd18071070 2023-10-21T17:18:55.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0003adc0 con 0x7fdd18071070 2023-10-21T17:18:55.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00044630 con 0x7fdd18071070 2023-10-21T17:18:55.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd180cdbc0 con 0x7fdd18071070 2023-10-21T17:18:55.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.145+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd180c2e00 con 0x7fdd18071070 2023-10-21T17:18:55.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.147+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd180ce0e0 con 0x7fdd18071070 2023-10-21T17:18:55.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.148+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 87) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd0002f050 con 0x7fdd18071070 2023-10-21T17:18:55.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.149+0000 7fdd14ff9640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 0x7fdcf806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:55.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.149+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdd000aba40 con 0x7fdd18071070 2023-10-21T17:18:55.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.149+0000 7fdd177fe640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 0x7fdcf806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:55.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.151+0000 7fdd177fe640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 0x7fdcf806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd08006fd0 tx=0x7fdd08008040 comp rx=0 tx=0).ready entity=mgr.36914 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:55.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.152+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd00076880 con 0x7fdd18071070 2023-10-21T17:18:55.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.401+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdd180ce0e0 con 0x7fdd18071070 2023-10-21T17:18:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.996+0000 7fdd14ff9640 1 -- 172.21.15.53:0/2005351944 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v88) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdd180ce0e0 con 0x7fdd18071070 2023-10-21T17:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.999+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 msgr2=0x7fdcf806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:55.999+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 0x7fdcf806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd08006fd0 tx=0x7fdd08008040 comp rx=0 tx=0).stop 2023-10-21T17:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 msgr2=0x7fdd181efd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009870 tx=0x7fdd000046c0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 shutdown_connections 2023-10-21T17:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fdcf806af90 0x7fdcf806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd180afbe0 0x7fdd181f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd18071070 0x7fdd181efd60 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 --2- 172.21.15.53:0/2005351944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1806dfc0 0x7fdd180cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.000+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 >> 172.21.15.53:0/2005351944 conn(0x7fdd1806a5d0 msgr2=0x7fdd18159790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.001+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 shutdown_connections 2023-10-21T17:18:56.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.001+0000 7fdd1daa2640 1 -- 172.21.15.53:0/2005351944 wait complete. 2023-10-21T17:18:56.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:18:56.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.254+0000 7fac149d5640 1 Processor -- start 2023-10-21T17:18:56.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.254+0000 7fac149d5640 1 -- start start 2023-10-21T17:18:56.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fac10160660 0x7fac10070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac100c9910 0x7fac100caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 0x7fac100b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1006e150 con 0x7fac100c9910 2023-10-21T17:18:56.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac100cb8c0 con 0x7fac10160660 2023-10-21T17:18:56.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.256+0000 7fac149d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1006de70 con 0x7fac100cb450 2023-10-21T17:18:56.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac100c9910 0x7fac100caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fac100c9910 0x7fac100caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37732/0 (socket says 172.21.15.53:37732) 2023-10-21T17:18:56.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 -- 172.21.15.53:0/3302063262 learned_addr learned my addr 172.21.15.53:0/3302063262 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:56.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac0ed9d640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 0x7fac100b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac0e59c640 1 --2- 172.21.15.53:0/3302063262 >> v2:172.21.15.124:3300/0 conn(0x7fac10160660 0x7fac10070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 -- 172.21.15.53:0/3302063262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 msgr2=0x7fac100b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:56.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 0x7fac100b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 -- 172.21.15.53:0/3302063262 >> v2:172.21.15.124:3300/0 conn(0x7fac10160660 msgr2=0x7fac10070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:56.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 --2- 172.21.15.53:0/3302063262 >> v2:172.21.15.124:3300/0 conn(0x7fac10160660 0x7fac10070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac07fff640 1 -- 172.21.15.53:0/3302063262 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac100c8e30 con 0x7fac100c9910 2023-10-21T17:18:56.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.257+0000 7fac0ed9d640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 0x7fac100b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:18:56.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.258+0000 7fac07fff640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac100c9910 0x7fac100caf10 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fac100b0d80 tx=0x7fabf802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f61e07488efdd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:56.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac0dd9b640 1 -- 172.21.15.53:0/3302063262 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8031860 con 0x7fac100c9910 2023-10-21T17:18:56.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac0dd9b640 1 -- 172.21.15.53:0/3302063262 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf80319c0 con 0x7fac100c9910 2023-10-21T17:18:56.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac0dd9b640 1 -- 172.21.15.53:0/3302063262 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8031d50 con 0x7fac100c9910 2023-10-21T17:18:56.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac149d5640 1 -- 172.21.15.53:0/3302063262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac100c9910 msgr2=0x7fac100caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac149d5640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac100c9910 0x7fac100caf10 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fac100b0d80 tx=0x7fabf802b900 comp rx=0 tx=0).stop 2023-10-21T17:18:56.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.259+0000 7fac149d5640 1 -- 172.21.15.53:0/3302063262 shutdown_connections 2023-10-21T17:18:56.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100cb450 0x7fac100b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 --2- 172.21.15.53:0/3302063262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac100c9910 0x7fac100caf10 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 --2- 172.21.15.53:0/3302063262 >> v2:172.21.15.124:3300/0 conn(0x7fac10160660 0x7fac10070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 -- 172.21.15.53:0/3302063262 >> 172.21.15.53:0/3302063262 conn(0x7fac1006a5d0 msgr2=0x7fac100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:56.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 -- 172.21.15.53:0/3302063262 shutdown_connections 2023-10-21T17:18:56.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.260+0000 7fac149d5640 1 -- 172.21.15.53:0/3302063262 wait complete. 2023-10-21T17:18:56.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.262+0000 7fac149d5640 1 Processor -- start 2023-10-21T17:18:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.263+0000 7fac149d5640 1 -- start start 2023-10-21T17:18:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100c9910 0x7fac100c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac100cb450 0x7fac100c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c8670 con 0x7fac10160660 2023-10-21T17:18:56.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c6330 con 0x7fac100cb450 2023-10-21T17:18:56.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.264+0000 7fac149d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac100c6610 con 0x7fac100c9910 2023-10-21T17:18:56.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37748/0 (socket says 172.21.15.53:37748) 2023-10-21T17:18:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 -- 172.21.15.53:0/3608274784 learned_addr learned my addr 172.21.15.53:0/3608274784 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0e59c640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100c9910 0x7fac100c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac07fff640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac100cb450 0x7fac100c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100c9910 msgr2=0x7fac100c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100c9910 0x7fac100c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac100cb450 msgr2=0x7fac100c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac100cb450 0x7fac100c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.265+0000 7fac0ed9d640 1 -- 172.21.15.53:0/3608274784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80094d0 con 0x7fac10160660 2023-10-21T17:18:56.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac0ed9d640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fac08007e10 tx=0x7fac0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:56.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08003b90 con 0x7fac10160660 2023-10-21T17:18:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800f040 con 0x7fac10160660 2023-10-21T17:18:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800de80 con 0x7fac10160660 2023-10-21T17:18:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac100c68f0 con 0x7fac10160660 2023-10-21T17:18:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.266+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac10148040 con 0x7fac10160660 2023-10-21T17:18:56.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.267+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac100c6e10 con 0x7fac10160660 2023-10-21T17:18:56.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.269+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 88) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fac08004030 con 0x7fac10160660 2023-10-21T17:18:56.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.271+0000 7fac06ffd640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 0x7fabf006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.271+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fac08089eb0 con 0x7fac10160660 2023-10-21T17:18:56.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.271+0000 7fac0e59c640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 msgr2=0x7fabf006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:56.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.271+0000 7fac0e59c640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 0x7fabf006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:56.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.276+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac100c6e10 con 0x7fac10160660 2023-10-21T17:18:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.472+0000 7fac0e59c640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 msgr2=0x7fabf006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:56.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.472+0000 7fac0e59c640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 0x7fabf006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:56.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.509+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fac100c6e10 con 0x7fac10160660 2023-10-21T17:18:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.514+0000 7fac06ffd640 1 -- 172.21.15.53:0/3608274784 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fac100c6e10 con 0x7fac10160660 2023-10-21T17:18:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 msgr2=0x7fabf006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 0x7fabf006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 msgr2=0x7fac10147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fac08007e10 tx=0x7fac0800d4a0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 shutdown_connections 2023-10-21T17:18:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7fabf006afe0 0x7fabf006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac10160660 0x7fac10147b00 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac100cb450 0x7fac100c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 --2- 172.21.15.53:0/3608274784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac100c9910 0x7fac100c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 >> 172.21.15.53:0/3608274784 conn(0x7fac1006a5d0 msgr2=0x7fac100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 shutdown_connections 2023-10-21T17:18:56.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.517+0000 7fac149d5640 1 -- 172.21.15.53:0/3608274784 wait complete. 2023-10-21T17:18:56.527 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:56.526+0000 7f38bd216040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:18:56.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:18:56.579 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:56.578+0000 7f38bd216040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:18:56.607 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:56.606+0000 7fd1cd0da040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:18:56.664 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:56.663+0000 7fd1cd0da040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:18:56.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.750+0000 7f6bdb167640 1 Processor -- start 2023-10-21T17:18:56.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.751+0000 7f6bdb167640 1 -- start start 2023-10-21T17:18:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.752+0000 7f6bdb167640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 0x7f6bd40cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bdb167640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6bd406ae60 0x7f6bd40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bdb167640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd40cc9b0 con 0x7f6bd415ce70 2023-10-21T17:18:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bdb167640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd40cdf10 con 0x7f6bd406ae60 2023-10-21T17:18:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bdb167640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd40cd1c0 con 0x7f6bd40b1600 2023-10-21T17:18:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6bd406ae60 0x7f6bd40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6bd406ae60 0x7f6bd40cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37756/0 (socket says 172.21.15.53:37756) 2023-10-21T17:18:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 -- 172.21.15.53:0/3081257954 learned_addr learned my addr 172.21.15.53:0/3081257954 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bd9964640 1 --2- 172.21.15.53:0/3081257954 >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 0x7f6bd40cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 -- 172.21.15.53:0/3081257954 >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 msgr2=0x7f6bd40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:56.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 --2- 172.21.15.53:0/3081257954 >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 0x7f6bd40cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 -- 172.21.15.53:0/3081257954 >> v1:172.21.15.124:6789/0 conn(0x7f6bd415ce70 legacy=0x7f6bd406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:56.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.753+0000 7f6bda966640 1 -- 172.21.15.53:0/3081257954 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40ca6c0 con 0x7f6bd406ae60 2023-10-21T17:18:56.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.754+0000 7f6bd9964640 1 --2- 172.21.15.53:0/3081257954 >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 0x7f6bd40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:56.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.755+0000 7f6bda966640 1 --2- 172.21.15.53:0/3081257954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd406ae60 0x7f6bd40cd870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4072240 tx=0x7f6bcc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589875069ff5ca80 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:56.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.755+0000 7f6bd9163640 1 -- 172.21.15.53:0/3081257954 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc02c9f0 con 0x7f6bd406ae60 2023-10-21T17:18:56.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.755+0000 7f6bd9163640 1 -- 172.21.15.53:0/3081257954 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc02cb50 con 0x7f6bd406ae60 2023-10-21T17:18:56.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.755+0000 7f6bd9163640 1 -- 172.21.15.53:0/3081257954 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0343f0 con 0x7f6bd406ae60 2023-10-21T17:18:56.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 -- 172.21.15.53:0/3081257954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd406ae60 msgr2=0x7f6bd40cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 --2- 172.21.15.53:0/3081257954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd406ae60 0x7f6bd40cd870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4072240 tx=0x7f6bcc02be50 comp rx=0 tx=0).stop 2023-10-21T17:18:56.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 -- 172.21.15.53:0/3081257954 shutdown_connections 2023-10-21T17:18:56.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 --2- 172.21.15.53:0/3081257954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd406ae60 0x7f6bd40cd870 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 --2- 172.21.15.53:0/3081257954 >> v2:172.21.15.124:3300/0 conn(0x7f6bd40b1600 0x7f6bd40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 -- 172.21.15.53:0/3081257954 >> 172.21.15.53:0/3081257954 conn(0x7f6bd40b08d0 msgr2=0x7f6bd40b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:56.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 -- 172.21.15.53:0/3081257954 shutdown_connections 2023-10-21T17:18:56.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.756+0000 7f6bdb167640 1 -- 172.21.15.53:0/3081257954 wait complete. 2023-10-21T17:18:56.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.758+0000 7f6bdb167640 1 Processor -- start 2023-10-21T17:18:56.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.758+0000 7f6bdb167640 1 -- start start 2023-10-21T17:18:56.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd406ae60 0x7f6bd4151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd40b1600 0x7f6bd414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd409fb80 con 0x7f6bd415ce70 2023-10-21T17:18:56.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd414e180 con 0x7f6bd40b1600 2023-10-21T17:18:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bdb167640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd414e460 con 0x7f6bd406ae60 2023-10-21T17:18:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bda966640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bda966640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37758/0 (socket says 172.21.15.53:37758) 2023-10-21T17:18:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bda165640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd406ae60 0x7f6bd4151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bda966640 1 -- 172.21.15.53:0/1737188892 learned_addr learned my addr 172.21.15.53:0/1737188892 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:56.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.759+0000 7f6bd9964640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd40b1600 0x7f6bd414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:56.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd406ae60 msgr2=0x7f6bd4151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd406ae60 0x7f6bd4151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd40b1600 msgr2=0x7f6bd414b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:56.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd40b1600 0x7f6bd414b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 -- 172.21.15.53:0/1737188892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc009b30 con 0x7f6bd415ce70 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bda966640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc036040 tx=0x7f6bcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc03e450 con 0x7f6bd415ce70 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc03e5d0 con 0x7f6bd415ce70 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.760+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc03e980 con 0x7f6bd415ce70 2023-10-21T17:18:56.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.761+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd40c8070 con 0x7f6bd415ce70 2023-10-21T17:18:56.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.761+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd4151ad0 con 0x7f6bd415ce70 2023-10-21T17:18:56.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.762+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd40c8530 con 0x7f6bd415ce70 2023-10-21T17:18:56.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.763+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 88) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f6bcc03eae0 con 0x7f6bd415ce70 2023-10-21T17:18:56.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.764+0000 7f6bcaffd640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 0x7f6bb806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:56.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.764+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6bcc0ac7e0 con 0x7f6bd415ce70 2023-10-21T17:18:56.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.764+0000 7f6bda165640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 msgr2=0x7f6bb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:56.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.764+0000 7f6bda165640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 0x7f6bb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:56.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.769+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bcc0775b0 con 0x7f6bd415ce70 2023-10-21T17:18:56.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.959+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6bd4074150 con 0x7f6bd415ce70 2023-10-21T17:18:57.031 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.026+0000 7f38bd216040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:18:57.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.964+0000 7f6bcaffd640 1 -- 172.21.15.53:0/1737188892 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6bcc07abd0 con 0x7f6bd415ce70 2023-10-21T17:18:57.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.965+0000 7f6bda165640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 msgr2=0x7f6bb806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:57.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.965+0000 7f6bda165640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 0x7f6bb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:57.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 msgr2=0x7f6bb806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:18:57.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 0x7f6bb806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 msgr2=0x7f6bd414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:57.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc036040 tx=0x7f6bcc004a40 comp rx=0 tx=0).stop 2023-10-21T17:18:57.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 shutdown_connections 2023-10-21T17:18:57.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f6bb806afe0 0x7f6bb806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6bd415ce70 0x7f6bd414d650 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd40b1600 0x7f6bd414b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 --2- 172.21.15.53:0/1737188892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd406ae60 0x7f6bd4151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 >> 172.21.15.53:0/1737188892 conn(0x7f6bd40b08d0 msgr2=0x7f6bd406c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:57.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 shutdown_connections 2023-10-21T17:18:57.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:56.967+0000 7f6bdb167640 1 -- 172.21.15.53:0/1737188892 wait complete. 2023-10-21T17:18:57.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:18:57.036 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T17:18:57.080 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.079+0000 7f38bd216040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:18:57.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-10-21T17:18:57.131 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.129+0000 7f38bd216040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:18:57.147 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.147+0000 7fd1cd0da040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:18:57.191 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.190+0000 7f38bd216040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:18:57.204 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.203+0000 7fd1cd0da040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:18:57.239 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.238+0000 7f38bd216040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:18:57.258 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.257+0000 7fd1cd0da040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:18:57.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.311+0000 7f89db8c2640 1 Processor -- start 2023-10-21T17:18:57.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.312+0000 7f89db8c2640 1 -- start start 2023-10-21T17:18:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.313+0000 7f89db8c2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89d415cc90 0x7f89d4161290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.313+0000 7f89db8c2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 0x7f89d406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.313+0000 7f89db8c2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d406aa70 con 0x7f89d406b440 2023-10-21T17:18:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.313+0000 7f89db8c2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d41617d0 con 0x7f89d415cc90 2023-10-21T17:18:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.313+0000 7f89db8c2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4069e60 con 0x7f89d40cd500 2023-10-21T17:18:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89d415cc90 0x7f89d4161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:57.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89d415cc90 0x7f89d4161290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37766/0 (socket says 172.21.15.53:37766) 2023-10-21T17:18:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 -- 172.21.15.53:0/1090307002 learned_addr learned my addr 172.21.15.53:0/1090307002 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 0x7f89d406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 -- 172.21.15.53:0/1090307002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 msgr2=0x7f89d406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 0x7f89d406af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 -- 172.21.15.53:0/1090307002 >> v1:172.21.15.124:6789/0 conn(0x7f89d406b440 legacy=0x7f89d40cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:18:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d965e640 1 -- 172.21.15.53:0/1090307002 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d406a140 con 0x7f89d415cc90 2023-10-21T17:18:57.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.314+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 0x7f89d406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:18:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.315+0000 7f89d965e640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d415cc90 0x7f89d4161290 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009870 tx=0x7f89d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a346d26370242e49 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.316+0000 7f89cbfff640 1 -- 172.21.15.53:0/1090307002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030840 con 0x7f89d415cc90 2023-10-21T17:18:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.316+0000 7f89cbfff640 1 -- 172.21.15.53:0/1090307002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00309a0 con 0x7f89d415cc90 2023-10-21T17:18:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.316+0000 7f89cbfff640 1 -- 172.21.15.53:0/1090307002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030d30 con 0x7f89d415cc90 2023-10-21T17:18:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.316+0000 7f89db8c2640 1 -- 172.21.15.53:0/1090307002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d415cc90 msgr2=0x7f89d4161290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.316+0000 7f89db8c2640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d415cc90 0x7f89d4161290 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009870 tx=0x7f89d002b980 comp rx=0 tx=0).stop 2023-10-21T17:18:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 -- 172.21.15.53:0/1090307002 shutdown_connections 2023-10-21T17:18:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d40cd500 0x7f89d406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 --2- 172.21.15.53:0/1090307002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d415cc90 0x7f89d4161290 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 -- 172.21.15.53:0/1090307002 >> 172.21.15.53:0/1090307002 conn(0x7f89d406cb10 msgr2=0x7f89d406cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:18:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 -- 172.21.15.53:0/1090307002 shutdown_connections 2023-10-21T17:18:57.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.317+0000 7f89db8c2640 1 -- 172.21.15.53:0/1090307002 wait complete. 2023-10-21T17:18:57.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.318+0000 7f89db8c2640 1 Processor -- start 2023-10-21T17:18:57.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.318+0000 7f89db8c2640 1 -- start start 2023-10-21T17:18:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 0x7f89d4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89d40cd500 0x7f89d40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d415cc90 0x7f89d41492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4146d80 con 0x7f89d406b440 2023-10-21T17:18:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d415a000 con 0x7f89d40cd500 2023-10-21T17:18:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.319+0000 7f89db8c2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d415a2e0 con 0x7f89d415cc90 2023-10-21T17:18:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 0x7f89d4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:57.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 0x7f89d4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37774/0 (socket says 172.21.15.53:37774) 2023-10-21T17:18:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 -- 172.21.15.53:0/1679558591 learned_addr learned my addr 172.21.15.53:0/1679558591 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:18:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d9e5f640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d415cc90 0x7f89d41492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89d40cd500 0x7f89d40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:18:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d415cc90 msgr2=0x7f89d41492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:57.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89d415cc90 0x7f89d41492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89d40cd500 msgr2=0x7f89d40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89d40cd500 0x7f89d40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.320+0000 7f89d965e640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00094d0 con 0x7f89d406b440 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89d965e640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 0x7f89d4158940 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f89d002b9c0 tx=0x7f89d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0030810 con 0x7f89d406b440 2023-10-21T17:18:57.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d0039d90 con 0x7f89d406b440 2023-10-21T17:18:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0043670 con 0x7f89d406b440 2023-10-21T17:18:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d4159200 con 0x7f89d406b440 2023-10-21T17:18:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.321+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d41597c0 con 0x7f89d406b440 2023-10-21T17:18:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.322+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d404e160 con 0x7f89d406b440 2023-10-21T17:18:57.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.323+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 88) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f89d002f390 con 0x7f89d406b440 2023-10-21T17:18:57.329 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.324+0000 7fd1cd0da040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:18:57.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.324+0000 7f89c9ffb640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:18:57.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.325+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f89d00abd90 con 0x7f89d406b440 2023-10-21T17:18:57.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.325+0000 7f89d8e5d640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 msgr2=0x7f89b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:57.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.325+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:18:57.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.328+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89d0076af0 con 0x7f89d406b440 2023-10-21T17:18:57.351 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.350+0000 7f38bd216040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:18:57.378 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.377+0000 7fd1cd0da040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:18:57.413 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.411+0000 7f38bd216040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:57.462 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.460+0000 7f38bd216040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:18:57.496 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.495+0000 7fd1cd0da040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:18:57.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.518+0000 7f38bd216040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:18:57.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.526+0000 7f89d8e5d640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 msgr2=0x7f89b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:57.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.526+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:18:57.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.540+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f89d406a700 con 0x7f89b806afe0 2023-10-21T17:18:57.562 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.561+0000 7fd1cd0da040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:57.569 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.568+0000 7f38bd216040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:18:57.615 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.613+0000 7fd1cd0da040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:18:57.674 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.673+0000 7fd1cd0da040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:18:57.727 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:57.726+0000 7fd1cd0da040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:18:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.927+0000 7f89d8e5d640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 msgr2=0x7f89b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:57.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:57.927+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:18:57.948 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:57.947+0000 7f38bd216040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:18:58.107 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.106+0000 7f38bd216040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:18:58.132 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.131+0000 7fd1cd0da040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:18:58.213 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.212+0000 7f38bd216040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:18:58.262 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.261+0000 7f38bd216040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:18:58.301 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.300+0000 7fd1cd0da040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:18:58.369 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.368+0000 7f38bd216040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:58.416 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.415+0000 7fd1cd0da040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:18:58.469 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.468+0000 7fd1cd0da040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:18:58.475 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.474+0000 7f38bd216040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:18:58.583 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.581+0000 7fd1cd0da040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:58.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.631+0000 7f38bd216040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:58.681 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.680+0000 7f38bd216040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:58.696 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.695+0000 7fd1cd0da040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:18:58.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:58.728+0000 7f89d8e5d640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 msgr2=0x7f89b806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/234236878 2023-10-21T17:18:58.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:18:58.728+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:18:58.800 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:58.798+0000 7f38bd216040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:18:58.863 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.862+0000 7fd1cd0da040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:18:58.915 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:58.914+0000 7fd1cd0da040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:18:59.041 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:59.039+0000 7fd1cd0da040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:18:59.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:59.635+0000 7f38bd216040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:18:59.933 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:59.932+0000 7f38bd216040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:18:59.934 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:18:59.933+0000 7fd1cd0da040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:18:59.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:18:59.989+0000 7f38bd216040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:00.104 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.103+0000 7f38bd216040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:00.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:00.122+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 89) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f89d00aa0f0 con 0x7f89d406b440 2023-10-21T17:19:00.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:00.122+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 msgr2=0x7f89b806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:00.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:00.122+0000 7f89c9ffb640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/234236878,v1:172.21.15.124:6833/234236878] conn(0x7f89b806afe0 0x7f89b806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:00.262 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:00.261+0000 7fd1cd0da040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:19:00.275 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.274+0000 7f383c87c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.276 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.274+0000 7f383c87c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.276 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.274+0000 7f383c87c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.276 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.274+0000 7f383c87c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.276 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.274+0000 7f383c87c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.277 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.275+0000 7f3839836640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.277 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.275+0000 7f3839836640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.277 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.275+0000 7f3839836640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.277 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.275+0000 7f3839836640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.278 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:00.275+0000 7f3839836640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:00.323 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:00.322+0000 7fd1cd0da040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:00.443 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:00.442+0000 7fd1cd0da040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.137+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 90) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f89d008e500 con 0x7f89d406b440 2023-10-21T17:19:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.138+0000 7f89c9ffb640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f89b8035880 0x7f89b8075530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.139+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 --> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f89b8035c60 con 0x7f89b8035880 2023-10-21T17:19:01.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.140+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f89b8035880 0x7f89b8075530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.146+0000 7f89d8e5d640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f89b8035880 0x7f89b8075530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89c4006fd0 tx=0x7f89c4008040 comp rx=0 tx=0).ready entity=mgr.37106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:01.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.311+0000 7f89c9ffb640 1 -- 172.21.15.53:0/1679558591 <== mgr.37106 v2:172.21.15.124:6832/97801354 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f89b8035c60 con 0x7f89b8035880 2023-10-21T17:19:01.313 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T17:19:01.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f89b8035880 msgr2=0x7f89b8075530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f89b8035880 0x7f89b8075530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89c4006fd0 tx=0x7f89c4008040 comp rx=0 tx=0).stop 2023-10-21T17:19:01.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 msgr2=0x7f89d4158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 --2- 172.21.15.53:0/1679558591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89d406b440 0x7f89d4158940 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f89d002b9c0 tx=0x7f89d0004730 comp rx=0 tx=0).stop 2023-10-21T17:19:01.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89d9e5f640 1 -- 172.21.15.53:0/1679558591 reap_dead start 2023-10-21T17:19:01.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 shutdown_connections 2023-10-21T17:19:01.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 >> 172.21.15.53:0/1679558591 conn(0x7f89d406cb10 msgr2=0x7f89d415d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:01.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 shutdown_connections 2023-10-21T17:19:01.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.315+0000 7f89db8c2640 1 -- 172.21.15.53:0/1679558591 wait complete. 2023-10-21T17:19:01.338 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T17:19:01.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-21T17:19:01.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.615+0000 7fc67f59e640 1 Processor -- start 2023-10-21T17:19:01.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.616+0000 7fc67f59e640 1 -- start start 2023-10-21T17:19:01.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.617+0000 7fc67f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc68015a820 0x7fc6800aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.617+0000 7fc67f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6800cab60 con 0x7fc6800ca680 2023-10-21T17:19:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.617+0000 7fc67f59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6800cb960 con 0x7fc68015ac00 2023-10-21T17:19:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.617+0000 7fc67f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6800cbdf0 con 0x7fc68015a820 2023-10-21T17:19:01.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.618+0000 7fc677fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc6800ca680 0x7fc6800c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49360/0 (socket says 172.21.15.53:49360) 2023-10-21T17:19:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.618+0000 7fc677fff640 1 -- 172.21.15.53:0/3460527252 learned_addr learned my addr 172.21.15.53:0/3460527252 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.618+0000 7fc67e59c640 1 --2- 172.21.15.53:0/3460527252 >> v2:172.21.15.124:3300/0 conn(0x7fc68015a820 0x7fc6800aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.618+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110560987 0 0) 0x7fc6800cab60 con 0x7fc6800ca680 2023-10-21T17:19:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.618+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc66c003520 con 0x7fc6800ca680 2023-10-21T17:19:01.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1908197301 0 0) 0x7fc66c003520 con 0x7fc6800ca680 2023-10-21T17:19:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 >> v2:172.21.15.124:3300/0 conn(0x7fc68015a820 msgr2=0x7fc6800aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 --2- 172.21.15.53:0/3460527252 >> v2:172.21.15.124:3300/0 conn(0x7fc68015a820 0x7fc6800aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 >> v1:172.21.15.124:6789/0 conn(0x7fc68015ac00 legacy=0x7fc6800c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800c9f20 con 0x7fc6800ca680 2023-10-21T17:19:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc668001670 con 0x7fc6800ca680 2023-10-21T17:19:01.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 >> v1:172.21.15.53:6789/0 conn(0x7fc6800ca680 legacy=0x7fc6800c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc66c0039c0 0x7fc66c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0047c0 0x7fc66c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800cbdf0 con 0x7fc66c0051d0 2023-10-21T17:19:01.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c00c120 con 0x7fc66c0039c0 2023-10-21T17:19:01.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c00c2e0 con 0x7fc66c0047c0 2023-10-21T17:19:01.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.619+0000 7fc67e59c640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc66c0039c0 0x7fc66c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc677fff640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0047c0 0x7fc66c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc67ed9d640 1 -- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0047c0 msgr2=0x7fc66c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:01.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0047c0 0x7fc66c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc67ed9d640 1 -- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc66c0039c0 msgr2=0x7fc66c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:01.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc66c0039c0 0x7fc66c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.620+0000 7fc67ed9d640 1 -- 172.21.15.53:0/3460527252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c003520 con 0x7fc66c0051d0 2023-10-21T17:19:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67ed9d640 1 -- 172.21.15.53:0/3460527252 reap_dead start 2023-10-21T17:19:01.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b980 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc68015b010 tx=0x7fc66c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73782d67dffde8f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678003b70 con 0x7fc66c0051d0 2023-10-21T17:19:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678003cd0 con 0x7fc66c0051d0 2023-10-21T17:19:01.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67dd9b640 1 -- 172.21.15.53:0/3460527252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780052f0 con 0x7fc66c0051d0 2023-10-21T17:19:01.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67f59e640 1 -- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 msgr2=0x7fc66c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.621+0000 7fc67f59e640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b980 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc68015b010 tx=0x7fc66c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.622+0000 7fc67f59e640 1 -- 172.21.15.53:0/3460527252 shutdown_connections 2023-10-21T17:19:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.622+0000 7fc67f59e640 1 --2- 172.21.15.53:0/3460527252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc66c0051d0 0x7fc66c00b980 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.622+0000 7fc67f59e640 1 -- 172.21.15.53:0/3460527252 >> 172.21.15.53:0/3460527252 conn(0x7fc68006c050 msgr2=0x7fc68006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:01.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.622+0000 7fc67f59e640 1 -- 172.21.15.53:0/3460527252 shutdown_connections 2023-10-21T17:19:01.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.622+0000 7fc67f59e640 1 -- 172.21.15.53:0/3460527252 wait complete. 2023-10-21T17:19:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.623+0000 7fc67f59e640 1 Processor -- start 2023-10-21T17:19:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.624+0000 7fc67f59e640 1 -- start start 2023-10-21T17:19:01.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.624+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0051d0 0x7fc6800ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.624+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc68014f550 0x7fc6800c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.624+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.624+0000 7fc67f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800c7ff0 con 0x7fc6800c7b40 2023-10-21T17:19:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800adcc0 con 0x7fc68014f550 2023-10-21T17:19:01.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800cc0c0 con 0x7fc66c0051d0 2023-10-21T17:19:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37792/0 (socket says 172.21.15.53:37792) 2023-10-21T17:19:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 -- 172.21.15.53:0/2506779934 learned_addr learned my addr 172.21.15.53:0/2506779934 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67e59c640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0051d0 0x7fc6800ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc677fff640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc68014f550 0x7fc6800c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 -- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0051d0 msgr2=0x7fc6800ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0051d0 0x7fc6800ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 -- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc68014f550 msgr2=0x7fc6800c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc68014f550 0x7fc6800c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:01.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.625+0000 7fc67ed9d640 1 -- 172.21.15.53:0/2506779934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800ac850 con 0x7fc6800c7b40 2023-10-21T17:19:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc67ed9d640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc678001ee0 tx=0x7fc6780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678039450 con 0x7fc6800c7b40 2023-10-21T17:19:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6780395b0 con 0x7fc6800c7b40 2023-10-21T17:19:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678039920 con 0x7fc6800c7b40 2023-10-21T17:19:01.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6800acb30 con 0x7fc6800c7b40 2023-10-21T17:19:01.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.626+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6800ad0f0 con 0x7fc6800c7b40 2023-10-21T17:19:01.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.627+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc680074150 con 0x7fc6800c7b40 2023-10-21T17:19:01.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.628+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 90) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fc67803f020 con 0x7fc6800c7b40 2023-10-21T17:19:01.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.629+0000 7fc676ffd640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 0x7fc66006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:01.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.629+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fc678003e40 con 0x7fc6800c7b40 2023-10-21T17:19:01.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.630+0000 7fc67e59c640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 0x7fc66006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:01.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.631+0000 7fc67e59c640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 0x7fc66006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670008040 comp rx=0 tx=0).ready entity=mgr.37106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:01.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.634+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc678076140 con 0x7fc6800c7b40 2023-10-21T17:19:01.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:01.839+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 --> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6800cb310 con 0x7fc66006ad40 2023-10-21T17:19:02.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.002+0000 7fc676ffd640 1 -- 172.21.15.53:0/2506779934 <== mgr.37106 v2:172.21.15.124:6832/97801354 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc670013960 con 0x7fc66006ad40 2023-10-21T17:19:02.003 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path snap_test_dir1 2023-10-21T17:19:02.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 msgr2=0x7fc66006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 0x7fc66006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc670009990 tx=0x7fc670008040 comp rx=0 tx=0).stop 2023-10-21T17:19:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 msgr2=0x7fc6800cc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc678001ee0 tx=0x7fc6780014a0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 shutdown_connections 2023-10-21T17:19:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fc66006ad40 0x7fc66006d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6800c7b40 0x7fc6800cc580 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc68014f550 0x7fc6800c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 --2- 172.21.15.53:0/2506779934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc66c0051d0 0x7fc6800ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 >> 172.21.15.53:0/2506779934 conn(0x7fc68006c050 msgr2=0x7fc68015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:02.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 shutdown_connections 2023-10-21T17:19:02.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.005+0000 7fc67f59e640 1 -- 172.21.15.53:0/2506779934 wait complete. 2023-10-21T17:19:02.028 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-21T17:19:02.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-21T17:19:02.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.266+0000 7f792bfff640 1 Processor -- start 2023-10-21T17:19:02.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.266+0000 7f792bfff640 1 -- start start 2023-10-21T17:19:02.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f792c15a820 0x7f792c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c15ac00 0x7f792c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792c0cab60 con 0x7f792c0ca680 2023-10-21T17:19:02.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792c0cb960 con 0x7f792c15a820 2023-10-21T17:19:02.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0cbdf0 con 0x7f792c15ac00 2023-10-21T17:19:02.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792affd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f792c15a820 0x7f792c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792a7fc640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f792c0ca680 0x7f792c0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:43250/0 (socket says 172.21.15.53:43250) 2023-10-21T17:19:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792a7fc640 1 -- 172.21.15.53:0/2298507662 learned_addr learned my addr 172.21.15.53:0/2298507662 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:02.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.268+0000 7f792b7fe640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c15ac00 0x7f792c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f7929ffb640 1 -- 172.21.15.53:0/2298507662 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194961156 0 0) 0x7f792c0cab60 con 0x7f792c0ca680 2023-10-21T17:19:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f7929ffb640 1 -- 172.21.15.53:0/2298507662 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7914003520 con 0x7f792c0ca680 2023-10-21T17:19:02.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f792affd640 1 -- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c15ac00 msgr2=0x7f792c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f792affd640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c15ac00 0x7f792c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f792affd640 1 -- 172.21.15.53:0/2298507662 >> v1:172.21.15.124:6789/0 conn(0x7f792c0ca680 legacy=0x7f792c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.269+0000 7f792affd640 1 -- 172.21.15.53:0/2298507662 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0cb030 con 0x7f792c15a820 2023-10-21T17:19:02.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.270+0000 7f792affd640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15a820 0x7f792c0aff70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f791c0097a0 tx=0x7f791c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1395302700c43603 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.270+0000 7f7929ffb640 1 -- 172.21.15.53:0/2298507662 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c02e070 con 0x7f792c15a820 2023-10-21T17:19:02.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f7929ffb640 1 -- 172.21.15.53:0/2298507662 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c0315a0 con 0x7f792c15a820 2023-10-21T17:19:02.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f7929ffb640 1 -- 172.21.15.53:0/2298507662 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c031910 con 0x7f792c15a820 2023-10-21T17:19:02.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 -- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15a820 msgr2=0x7f792c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15a820 0x7f792c0aff70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f791c0097a0 tx=0x7f791c02b840 comp rx=0 tx=0).stop 2023-10-21T17:19:02.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 -- 172.21.15.53:0/2298507662 shutdown_connections 2023-10-21T17:19:02.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c15ac00 0x7f792c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 --2- 172.21.15.53:0/2298507662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15a820 0x7f792c0aff70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 -- 172.21.15.53:0/2298507662 >> 172.21.15.53:0/2298507662 conn(0x7f792c06c050 msgr2=0x7f792c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:02.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 -- 172.21.15.53:0/2298507662 shutdown_connections 2023-10-21T17:19:02.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.271+0000 7f792bfff640 1 -- 172.21.15.53:0/2298507662 wait complete. 2023-10-21T17:19:02.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.273+0000 7f792bfff640 1 Processor -- start 2023-10-21T17:19:02.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.273+0000 7f792bfff640 1 -- start start 2023-10-21T17:19:02.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.274+0000 7f792bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c0ca680 0x7f792c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.274+0000 7f792bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f792c15a820 0x7f792c0c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.274+0000 7f792bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15ac00 0x7f792c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.274+0000 7f792bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c4f00 con 0x7f792c15a820 2023-10-21T17:19:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.274+0000 7f792bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c3e60 con 0x7f792c15ac00 2023-10-21T17:19:02.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c4140 con 0x7f792c0ca680 2023-10-21T17:19:02.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f792c15a820 0x7f792c0c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f792c15a820 0x7f792c0c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37798/0 (socket says 172.21.15.53:37798) 2023-10-21T17:19:02.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 -- 172.21.15.53:0/4251224339 learned_addr learned my addr 172.21.15.53:0/4251224339 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:02.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792affd640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c0ca680 0x7f792c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792b7fe640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15ac00 0x7f792c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 -- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c0ca680 msgr2=0x7f792c0afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f792c0ca680 0x7f792c0afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 -- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15ac00 msgr2=0x7f792c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15ac00 0x7f792c14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792a7fc640 1 -- 172.21.15.53:0/4251224339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c009400 con 0x7f792c15a820 2023-10-21T17:19:02.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.275+0000 7f792b7fe640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f792c15ac00 0x7f792c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:02.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f792a7fc640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f792c15a820 0x7f792c0c3610 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f7920001eb0 tx=0x7f792000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920005ce0 con 0x7f792c15a820 2023-10-21T17:19:02.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920005e60 con 0x7f792c15a820 2023-10-21T17:19:02.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79200072f0 con 0x7f792c15a820 2023-10-21T17:19:02.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f792bfff640 1 -- 172.21.15.53:0/4251224339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792c0c4420 con 0x7f792c15a820 2023-10-21T17:19:02.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.276+0000 7f792bfff640 1 -- 172.21.15.53:0/4251224339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c0c49e0 con 0x7f792c15a820 2023-10-21T17:19:02.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.277+0000 7f792bfff640 1 -- 172.21.15.53:0/4251224339 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792c074150 con 0x7f792c15a820 2023-10-21T17:19:02.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.278+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 91) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f7920007450 con 0x7f792c15a820 2023-10-21T17:19:02.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.279+0000 7f79137fe640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f791806afe0 0x7f791806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.279+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f7920089140 con 0x7f792c15a820 2023-10-21T17:19:02.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.280+0000 7f792affd640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f791806afe0 0x7f791806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.281+0000 7f792affd640 1 --2- 172.21.15.53:0/4251224339 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f791806afe0 0x7f791806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f791c036000 tx=0x7f791c008030 comp rx=0 tx=0).ready entity=mgr.37106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.284+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7920053d50 con 0x7f792c15a820 2023-10-21T17:19:02.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.497+0000 7f792bfff640 1 -- 172.21.15.53:0/4251224339 --> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f792c0cb310 con 0x7f791806afe0 2023-10-21T17:19:02.516 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:02.515+0000 7f384388a640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-21T17:19:02.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.516+0000 7f79137fe640 1 -- 172.21.15.53:0/4251224339 <== mgr.37106 v2:172.21.15.124:6832/97801354 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f791c004320 con 0x7f791806afe0 2023-10-21T17:19:02.517 INFO:teuthology.orchestra.run.smithi053.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-21T17:19:02.536 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-21T17:19:02.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:02.538 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T17:19:02.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:19:02.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.832+0000 7f2153d1d640 1 Processor -- start 2023-10-21T17:19:02.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.832+0000 7f2153d1d640 1 -- start start 2023-10-21T17:19:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2153d1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f214c06dfc0 0x7f214c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2153d1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 0x7f214c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2153d1d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214c0b1cc0 con 0x7f214c0afbe0 2023-10-21T17:19:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2153d1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f214c0b2620 con 0x7f214c06dfc0 2023-10-21T17:19:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2153d1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0b2060 con 0x7f214c071070 2023-10-21T17:19:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f21512b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f214c06dfc0 0x7f214c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2151ab9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f214c0afbe0 0x7f214c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:49362/0 (socket says 172.21.15.53:49362) 2023-10-21T17:19:02.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.834+0000 7f2151ab9640 1 -- 172.21.15.53:0/2171910907 learned_addr learned my addr 172.21.15.53:0/2171910907 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:02.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21522ba640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 0x7f214c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21512b8640 1 -- 172.21.15.53:0/2171910907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 msgr2=0x7f214c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21512b8640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 0x7f214c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21512b8640 1 -- 172.21.15.53:0/2171910907 >> v1:172.21.15.53:6789/0 conn(0x7f214c0afbe0 legacy=0x7f214c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21512b8640 1 -- 172.21.15.53:0/2171910907 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0b2340 con 0x7f214c06dfc0 2023-10-21T17:19:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f21522ba640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 0x7f214c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.835+0000 7f2150ab7640 1 -- 172.21.15.53:0/2171910907 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254202800 0 0) 0x7f214c0b1cc0 con 0x7f214c0afbe0 2023-10-21T17:19:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.836+0000 7f21512b8640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c06dfc0 0x7f214c070b30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f213c0097a0 tx=0x7f213c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47fcda3f32f1355 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.836+0000 7f2150ab7640 1 -- 172.21.15.53:0/2171910907 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c02d040 con 0x7f214c06dfc0 2023-10-21T17:19:02.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.836+0000 7f2150ab7640 1 -- 172.21.15.53:0/2171910907 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c031810 con 0x7f214c06dfc0 2023-10-21T17:19:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.836+0000 7f2150ab7640 1 -- 172.21.15.53:0/2171910907 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c031ba0 con 0x7f214c06dfc0 2023-10-21T17:19:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 -- 172.21.15.53:0/2171910907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c06dfc0 msgr2=0x7f214c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c06dfc0 0x7f214c070b30 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f213c0097a0 tx=0x7f213c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 -- 172.21.15.53:0/2171910907 shutdown_connections 2023-10-21T17:19:02.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c071070 0x7f214c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2171910907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c06dfc0 0x7f214c070b30 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 -- 172.21.15.53:0/2171910907 >> 172.21.15.53:0/2171910907 conn(0x7f214c06a5d0 msgr2=0x7f214c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 -- 172.21.15.53:0/2171910907 shutdown_connections 2023-10-21T17:19:02.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.837+0000 7f2153d1d640 1 -- 172.21.15.53:0/2171910907 wait complete. 2023-10-21T17:19:02.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.839+0000 7f2153d1d640 1 Processor -- start 2023-10-21T17:19:02.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.839+0000 7f2153d1d640 1 -- start start 2023-10-21T17:19:02.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 0x7f214c1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f214c071070 0x7f214c0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c0cce20 con 0x7f214c0afbe0 2023-10-21T17:19:02.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15d530 con 0x7f214c071070 2023-10-21T17:19:02.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f2153d1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214c15d810 con 0x7f214c06dfc0 2023-10-21T17:19:02.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f21522ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f21522ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:37816/0 (socket says 172.21.15.53:37816) 2023-10-21T17:19:02.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.840+0000 7f21522ba640 1 -- 172.21.15.53:0/2997513261 learned_addr learned my addr 172.21.15.53:0/2997513261 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:02.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f2151ab9640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 0x7f214c1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21512b8640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f214c071070 0x7f214c0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 -- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 msgr2=0x7f214c1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 0x7f214c1f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 -- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f214c071070 msgr2=0x7f214c0c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f214c071070 0x7f214c0c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 -- 172.21.15.53:0/2997513261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c009400 con 0x7f214c0afbe0 2023-10-21T17:19:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f2151ab9640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 0x7f214c1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:02.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.841+0000 7f21522ba640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f214400ce80 tx=0x7f214400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.842+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214400d840 con 0x7f214c0afbe0 2023-10-21T17:19:02.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.842+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214400d9e0 con 0x7f214c0afbe0 2023-10-21T17:19:02.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.842+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214400dd90 con 0x7f214c0afbe0 2023-10-21T17:19:02.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.842+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c15daf0 con 0x7f214c0afbe0 2023-10-21T17:19:02.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.842+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c1f2270 con 0x7f214c0afbe0 2023-10-21T17:19:02.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.843+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c0ccf60 con 0x7f214c0afbe0 2023-10-21T17:19:02.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.844+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 91) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f2144015440 con 0x7f214c0afbe0 2023-10-21T17:19:02.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.845+0000 7f21427fc640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 0x7f213006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:02.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.846+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f214408a370 con 0x7f214c0afbe0 2023-10-21T17:19:02.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.846+0000 7f2151ab9640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 0x7f213006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:02.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.847+0000 7f2151ab9640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 0x7f213006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2148000f60 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.37106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:02.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:02.850+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2144055140 con 0x7f214c0afbe0 2023-10-21T17:19:03.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.096+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f214c04e160 con 0x7f214c0afbe0 2023-10-21T17:19:03.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.828+0000 7f21427fc640 1 -- 172.21.15.53:0/2997513261 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v92) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2144058760 con 0x7f214c0afbe0 2023-10-21T17:19:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.832+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 msgr2=0x7f213006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.832+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 0x7f213006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2148000f60 tx=0x7f2148009040 comp rx=0 tx=0).stop 2023-10-21T17:19:03.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.832+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 msgr2=0x7f214c0c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:03.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.832+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f214400ce80 tx=0x7f214400a710 comp rx=0 tx=0).stop 2023-10-21T17:19:03.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 shutdown_connections 2023-10-21T17:19:03.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f213006b010 0x7f213006d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:03.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f214c0afbe0 0x7f214c0c2790 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:03.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f214c071070 0x7f214c0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:03.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 --2- 172.21.15.53:0/2997513261 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f214c06dfc0 0x7f214c1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:03.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.833+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 >> 172.21.15.53:0/2997513261 conn(0x7f214c06a5d0 msgr2=0x7f214c159410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:03.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.834+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 shutdown_connections 2023-10-21T17:19:03.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:03.835+0000 7f2153d1d640 1 -- 172.21.15.53:0/2997513261 wait complete. 2023-10-21T17:19:03.857 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:19:03.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:19:03.963 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:19:04.058 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:19:04.059 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:04.059 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:04.117 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:04.118 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:04.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:04.357 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:04.355+0000 7f52a4c12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:19:04.362 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:04.360+0000 7fb3f8812040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:19:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.371+0000 7fe5e4843640 1 Processor -- start 2023-10-21T17:19:04.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.371+0000 7fe5e4843640 1 -- start start 2023-10-21T17:19:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.372+0000 7fe5e4843640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5e015cca0 0x7fe5e01614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.372+0000 7fe5e4843640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.372+0000 7fe5e4843640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e00c9020 con 0x7fe5e00cab40 2023-10-21T17:19:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.372+0000 7fe5e4843640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00ccc20 con 0x7fe5e015cca0 2023-10-21T17:19:04.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.372+0000 7fe5e4843640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00c92e0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.373+0000 7fe5df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.373+0000 7fe5df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:51934/0 (socket says 172.21.15.53:51934) 2023-10-21T17:19:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.373+0000 7fe5df59e640 1 -- 172.21.15.53:0/408791824 learned_addr learned my addr 172.21.15.53:0/408791824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.373+0000 7fe5ded9d640 1 --2- 172.21.15.53:0/408791824 >> v2:172.21.15.124:3300/0 conn(0x7fe5e015cca0 0x7fe5e01614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.374+0000 7fe5df59e640 1 -- 172.21.15.53:0/408791824 >> v2:172.21.15.124:3300/0 conn(0x7fe5e015cca0 msgr2=0x7fe5e01614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:04.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.374+0000 7fe5df59e640 1 --2- 172.21.15.53:0/408791824 >> v2:172.21.15.124:3300/0 conn(0x7fe5e015cca0 0x7fe5e01614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.374+0000 7fe5df59e640 1 -- 172.21.15.53:0/408791824 >> v1:172.21.15.124:6789/0 conn(0x7fe5e00cab40 legacy=0x7fe5e00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.374+0000 7fe5df59e640 1 -- 172.21.15.53:0/408791824 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00cc7b0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.375+0000 7fe5df59e640 1 --2- 172.21.15.53:0/408791824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009e50 tx=0x7fe5d802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6974f2bc540612c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:04.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.376+0000 7fe5de59c640 1 -- 172.21.15.53:0/408791824 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d802c9e0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.376+0000 7fe5de59c640 1 -- 172.21.15.53:0/408791824 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d802cb40 con 0x7fe5e00b07d0 2023-10-21T17:19:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.376+0000 7fe5de59c640 1 -- 172.21.15.53:0/408791824 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80343f0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 -- 172.21.15.53:0/408791824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 msgr2=0x7fe5e00c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 --2- 172.21.15.53:0/408791824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009e50 tx=0x7fe5d802be40 comp rx=0 tx=0).stop 2023-10-21T17:19:04.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 -- 172.21.15.53:0/408791824 shutdown_connections 2023-10-21T17:19:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 --2- 172.21.15.53:0/408791824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00b07d0 0x7fe5e00c9770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 --2- 172.21.15.53:0/408791824 >> v2:172.21.15.124:3300/0 conn(0x7fe5e015cca0 0x7fe5e01614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 -- 172.21.15.53:0/408791824 >> 172.21.15.53:0/408791824 conn(0x7fe5e00cb0c0 msgr2=0x7fe5e00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 -- 172.21.15.53:0/408791824 shutdown_connections 2023-10-21T17:19:04.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.377+0000 7fe5e4843640 1 -- 172.21.15.53:0/408791824 wait complete. 2023-10-21T17:19:04.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.378+0000 7fe5e4843640 1 Processor -- start 2023-10-21T17:19:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.379+0000 7fe5e4843640 1 -- start start 2023-10-21T17:19:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00cab40 0x7fe5e015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e015cca0 0x7fe5e0159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015e430 con 0x7fe5e00b07d0 2023-10-21T17:19:04.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015d5c0 con 0x7fe5e015cca0 2023-10-21T17:19:04.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5e4843640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015a040 con 0x7fe5e00cab40 2023-10-21T17:19:04.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60666/0 (socket says 172.21.15.53:60666) 2023-10-21T17:19:04.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 -- 172.21.15.53:0/656573390 learned_addr learned my addr 172.21.15.53:0/656573390 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:04.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5d7fff640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00cab40 0x7fe5e015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00cab40 msgr2=0x7fe5e015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00cab40 0x7fe5e015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e015cca0 msgr2=0x7fe5e0159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5df59e640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e015cca0 0x7fe5e0159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e015cca0 0x7fe5e0159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.380+0000 7fe5ded9d640 1 -- 172.21.15.53:0/656573390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8009b30 con 0x7fe5e00b07d0 2023-10-21T17:19:04.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5ded9d640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9b0 tx=0x7fe5d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:04.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0009670 con 0x7fe5e00b07d0 2023-10-21T17:19:04.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d000c9a0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e015d820 con 0x7fe5e00b07d0 2023-10-21T17:19:04.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000cd50 con 0x7fe5e00b07d0 2023-10-21T17:19:04.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.381+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e015dde0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.382+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e00cd130 con 0x7fe5e00b07d0 2023-10-21T17:19:04.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.383+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fe5d0013470 con 0x7fe5e00b07d0 2023-10-21T17:19:04.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.384+0000 7fe5d77fe640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 0x7fe5c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.384+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fe5d0089de0 con 0x7fe5e00b07d0 2023-10-21T17:19:04.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.384+0000 7fe5d7fff640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 msgr2=0x7fe5c006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:04.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.384+0000 7fe5d7fff640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 0x7fe5c006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:04.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.388+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5e00cd130 con 0x7fe5e00b07d0 2023-10-21T17:19:04.411 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:04.409+0000 7f52a4c12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:19:04.420 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:04.419+0000 7fb3f8812040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:19:04.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.581+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e00cd130 con 0x7fe5e00b07d0 2023-10-21T17:19:04.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.584+0000 7fe5d77fe640 1 -- 172.21.15.53:0/656573390 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17403 (secure 0 0 0) 0x7fe5e00cd130 con 0x7fe5e00b07d0 2023-10-21T17:19:04.586 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:04.586 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":185,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:00.108345+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:6832/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1818943929":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1207144287":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/207583581":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1931157828":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/880607286":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/3805066261":"2023-10-22T17:19:00.108230+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.585+0000 7fe5d7fff640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 msgr2=0x7fe5c006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:04.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.585+0000 7fe5d7fff640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 0x7fe5c006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 msgr2=0x7fe5c006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:04.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 0x7fe5c006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 msgr2=0x7fe5e01f3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000e9b0 tx=0x7fe5d000ee70 comp rx=0 tx=0).stop 2023-10-21T17:19:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 shutdown_connections 2023-10-21T17:19:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fe5c006af90 0x7fe5c006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e015cca0 0x7fe5e0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e00cab40 0x7fe5e015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 --2- 172.21.15.53:0/656573390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5e00b07d0 0x7fe5e01f3690 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 >> 172.21.15.53:0/656573390 conn(0x7fe5e00cb0c0 msgr2=0x7fe5e00ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:04.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 shutdown_connections 2023-10-21T17:19:04.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.587+0000 7fe5e4843640 1 -- 172.21.15.53:0/656573390 wait complete. 2023-10-21T17:19:04.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:18:43.336266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:18:44.404625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:19:04.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:19:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.818+0000 7f1697fff640 1 Processor -- start 2023-10-21T17:19:04.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.818+0000 7f1697fff640 1 -- start start 2023-10-21T17:19:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1697fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815d060 0x7f16980caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1697fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f169815dc50 0x7f16980cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1697fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16980caa90 con 0x7f169806b180 2023-10-21T17:19:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1697fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169815e3f0 con 0x7f169815dc50 2023-10-21T17:19:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1697fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169815e110 con 0x7f169815d060 2023-10-21T17:19:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f16967fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f169806b180 0x7f169815d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43598/0 (socket says 172.21.15.53:43598) 2023-10-21T17:19:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f16967fc640 1 -- 172.21.15.53:0/2162526105 learned_addr learned my addr 172.21.15.53:0/2162526105 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:04.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f16977fe640 1 --2- 172.21.15.53:0/2162526105 >> v2:172.21.15.53:3300/0 conn(0x7f169815dc50 0x7f16980cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.820+0000 7f1696ffd640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815d060 0x7f16980caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.821+0000 7f16977fe640 1 -- 172.21.15.53:0/2162526105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815d060 msgr2=0x7f16980caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.821+0000 7f16977fe640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815d060 0x7f16980caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.821+0000 7f16977fe640 1 -- 172.21.15.53:0/2162526105 >> v1:172.21.15.53:6789/0 conn(0x7f169806b180 legacy=0x7f169815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.821+0000 7f16977fe640 1 -- 172.21.15.53:0/2162526105 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16980ca440 con 0x7f169815dc50 2023-10-21T17:19:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.822+0000 7f16977fe640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815dc50 0x7f16980cd900 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1690009d90 tx=0x7f169002e020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35fc179d9b3270b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.822+0000 7f1695ffb640 1 -- 172.21.15.53:0/2162526105 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169002edd0 con 0x7f169815dc50 2023-10-21T17:19:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.822+0000 7f1695ffb640 1 -- 172.21.15.53:0/2162526105 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690036440 con 0x7f169815dc50 2023-10-21T17:19:04.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.822+0000 7f1695ffb640 1 -- 172.21.15.53:0/2162526105 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900367d0 con 0x7f169815dc50 2023-10-21T17:19:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 -- 172.21.15.53:0/2162526105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815dc50 msgr2=0x7f16980cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815dc50 0x7f16980cd900 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1690009d90 tx=0x7f169002e020 comp rx=0 tx=0).stop 2023-10-21T17:19:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 -- 172.21.15.53:0/2162526105 shutdown_connections 2023-10-21T17:19:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815dc50 0x7f16980cd900 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 --2- 172.21.15.53:0/2162526105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815d060 0x7f16980caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 -- 172.21.15.53:0/2162526105 >> 172.21.15.53:0/2162526105 conn(0x7f16980b08a0 msgr2=0x7f16980b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 -- 172.21.15.53:0/2162526105 shutdown_connections 2023-10-21T17:19:04.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.823+0000 7f1697fff640 1 -- 172.21.15.53:0/2162526105 wait complete. 2023-10-21T17:19:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.825+0000 7f1697fff640 1 Processor -- start 2023-10-21T17:19:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.825+0000 7f1697fff640 1 -- start start 2023-10-21T17:19:04.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169806b180 0x7f16980af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815dc50 0x7f16981f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169809ce60 con 0x7f169815d060 2023-10-21T17:19:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169809d700 con 0x7f169806b180 2023-10-21T17:19:04.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1697fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169809ee50 con 0x7f169815dc50 2023-10-21T17:19:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f16967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f16967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60694/0 (socket says 172.21.15.53:60694) 2023-10-21T17:19:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f16967fc640 1 -- 172.21.15.53:0/3370353811 learned_addr learned my addr 172.21.15.53:0/3370353811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f16977fe640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815dc50 0x7f16981f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.826+0000 7f1696ffd640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169806b180 0x7f16980af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:04.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815dc50 msgr2=0x7f16981f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815dc50 0x7f16981f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169806b180 msgr2=0x7f16980af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:04.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169806b180 0x7f16980af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:04.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 -- 172.21.15.53:0/3370353811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009a70 con 0x7f169815d060 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f16967fc640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f168c00c530 tx=0x7f168c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c005ce0 con 0x7f169815d060 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c005e40 con 0x7f169815d060 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.827+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c007270 con 0x7f169815d060 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.828+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169809d0b0 con 0x7f169815d060 2023-10-21T17:19:04.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.828+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16980afd60 con 0x7f169815d060 2023-10-21T17:19:04.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.829+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16980aff50 con 0x7f169815d060 2023-10-21T17:19:04.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.830+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f168c0073d0 con 0x7f169815d060 2023-10-21T17:19:04.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.831+0000 7f167f7fe640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 0x7f168406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:04.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.831+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f168c088eb0 con 0x7f169815d060 2023-10-21T17:19:04.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.831+0000 7f1696ffd640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 msgr2=0x7f168406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:04.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.831+0000 7f1696ffd640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 0x7f168406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:04.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:04.837+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f168c053bf0 con 0x7f169815d060 2023-10-21T17:19:04.857 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:04.856+0000 7f52a4c12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:19:04.902 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:04.900+0000 7fb3f8812040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:19:04.909 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:04.908+0000 7f52a4c12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:19:04.957 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:04.956+0000 7fb3f8812040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:19:04.959 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:04.957+0000 7f52a4c12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:19:05.010 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.009+0000 7fb3f8812040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:19:05.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.018+0000 7f52a4c12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:19:05.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.029+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16980720a0 con 0x7f169815d060 2023-10-21T17:19:05.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.030+0000 7f167f7fe640 1 -- 172.21.15.53:0/3370353811 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f168c017020 con 0x7f169815d060 2023-10-21T17:19:05.031 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:19:05.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1696ffd640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 msgr2=0x7f168406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:05.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1696ffd640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 0x7f168406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 msgr2=0x7f168406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 0x7f168406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 msgr2=0x7f169809e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.032+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f168c00c530 tx=0x7f168c00c9f0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 shutdown_connections 2023-10-21T17:19:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f168406afc0 0x7f168406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f169815dc50 0x7f16981f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f169815d060 0x7f169809e4d0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 --2- 172.21.15.53:0/3370353811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169806b180 0x7f16980af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 >> 172.21.15.53:0/3370353811 conn(0x7f16980b08a0 msgr2=0x7f169806db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 shutdown_connections 2023-10-21T17:19:05.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.033+0000 7f1697fff640 1 -- 172.21.15.53:0/3370353811 wait complete. 2023-10-21T17:19:05.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:19:05.068 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.067+0000 7f52a4c12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:19:05.077 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.076+0000 7fb3f8812040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:19:05.132 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.129+0000 7fb3f8812040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:19:05.177 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.176+0000 7f52a4c12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:19:05.239 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.237+0000 7f52a4c12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:05.247 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.246+0000 7fb3f8812040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:19:05.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.266+0000 7f981b97c640 1 Processor -- start 2023-10-21T17:19:05.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.267+0000 7f981b97c640 1 -- start start 2023-10-21T17:19:05.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f981b97c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f98140cb130 0x7f98140cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f981b97c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98140b1cc0 con 0x7f981406dfc0 2023-10-21T17:19:05.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f981b97c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98140cbad0 con 0x7f98140afbe0 2023-10-21T17:19:05.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f981b97c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98140b2620 con 0x7f98140cb130 2023-10-21T17:19:05.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f9818f17640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f981406dfc0 0x7f9814070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43610/0 (socket says 172.21.15.53:43610) 2023-10-21T17:19:05.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.268+0000 7f9818f17640 1 -- 172.21.15.53:0/2895054897 learned_addr learned my addr 172.21.15.53:0/2895054897 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:05.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f9819f19640 1 --2- 172.21.15.53:0/2895054897 >> v2:172.21.15.124:3300/0 conn(0x7f98140cb130 0x7f98140cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88282729 0 0) 0x7f98140b1cc0 con 0x7f981406dfc0 2023-10-21T17:19:05.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9800003520 con 0x7f981406dfc0 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506282782 0 0) 0x7f9800003520 con 0x7f981406dfc0 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 >> v2:172.21.15.124:3300/0 conn(0x7f98140cb130 msgr2=0x7f98140cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 --2- 172.21.15.53:0/2895054897 >> v2:172.21.15.124:3300/0 conn(0x7f98140cb130 0x7f98140cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 >> v1:172.21.15.124:6789/0 conn(0x7f98140afbe0 legacy=0x7f98140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140b2060 con 0x7f981406dfc0 2023-10-21T17:19:05.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9804001670 con 0x7f981406dfc0 2023-10-21T17:19:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 >> v1:172.21.15.53:6789/0 conn(0x7f981406dfc0 legacy=0x7f9814070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98000047c0 0x7f9800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f980000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98140b2620 con 0x7f9800005180 2023-10-21T17:19:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980000c080 con 0x7f98000047c0 2023-10-21T17:19:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.269+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980000c240 con 0x7f98000039c0 2023-10-21T17:19:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f980000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819718640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9818f17640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98000047c0 0x7f9800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 -- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98000039c0 msgr2=0x7f9800003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:05.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98000039c0 0x7f9800003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 -- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98000047c0 msgr2=0x7f9800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:05.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98000047c0 0x7f9800004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.270+0000 7f9819f19640 1 -- 172.21.15.53:0/2895054897 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800003520 con 0x7f9800005180 2023-10-21T17:19:05.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.271+0000 7f9819f19640 1 -- 172.21.15.53:0/2895054897 reap_dead start 2023-10-21T17:19:05.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.271+0000 7f9819f19640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f980000b980 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f980c009ac0 tx=0x7f98000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be785a6ca52de9d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.272+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c004020 con 0x7f9800005180 2023-10-21T17:19:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.272+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c004180 con 0x7f9800005180 2023-10-21T17:19:05.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.272+0000 7f980bfff640 1 -- 172.21.15.53:0/2895054897 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0044f0 con 0x7f9800005180 2023-10-21T17:19:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.272+0000 7f981b97c640 1 -- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 msgr2=0x7f980000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.272+0000 7f981b97c640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f980000b980 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f980c009ac0 tx=0x7f98000057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.273+0000 7f981b97c640 1 -- 172.21.15.53:0/2895054897 shutdown_connections 2023-10-21T17:19:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.273+0000 7f981b97c640 1 --2- 172.21.15.53:0/2895054897 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f980000b980 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.273+0000 7f981b97c640 1 -- 172.21.15.53:0/2895054897 >> 172.21.15.53:0/2895054897 conn(0x7f981406a5d0 msgr2=0x7f98140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:05.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.273+0000 7f981b97c640 1 -- 172.21.15.53:0/2895054897 shutdown_connections 2023-10-21T17:19:05.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.273+0000 7f981b97c640 1 -- 172.21.15.53:0/2895054897 wait complete. 2023-10-21T17:19:05.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.274+0000 7f981b97c640 1 Processor -- start 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 -- start start 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 0x7f98140709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9814070ee0 0x7f98141f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814071360 con 0x7f9800005180 2023-10-21T17:19:05.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814159810 con 0x7f9814070ee0 2023-10-21T17:19:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.275+0000 7f981b97c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98140adaf0 con 0x7f98140c4c40 2023-10-21T17:19:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60714/0 (socket says 172.21.15.53:60714) 2023-10-21T17:19:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 -- 172.21.15.53:0/3521343936 learned_addr learned my addr 172.21.15.53:0/3521343936 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:05.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9818f17640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 0x7f98140709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819f19640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9814070ee0 0x7f98141f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 msgr2=0x7f98140709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 0x7f98140709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9814070ee0 msgr2=0x7f98141f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9814070ee0 0x7f98141f2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9819718640 1 -- 172.21.15.53:0/3521343936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140acf70 con 0x7f9800005180 2023-10-21T17:19:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.276+0000 7f9818f17640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 0x7f98140709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f9819718640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f981000f910 tx=0x7f981000fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810003b80 con 0x7f9800005180 2023-10-21T17:19:05.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810003ce0 con 0x7f9800005180 2023-10-21T17:19:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000da10 con 0x7f9800005180 2023-10-21T17:19:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98140ad250 con 0x7f9800005180 2023-10-21T17:19:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.277+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98140ad810 con 0x7f9800005180 2023-10-21T17:19:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.278+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981404e160 con 0x7f9800005180 2023-10-21T17:19:05.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.279+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f981000db70 con 0x7f9800005180 2023-10-21T17:19:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.281+0000 7f9809ffb640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 0x7f97f806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.281+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f9810089c70 con 0x7f9800005180 2023-10-21T17:19:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.281+0000 7f9818f17640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 msgr2=0x7f97f806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.281+0000 7f9818f17640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 0x7f97f806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:05.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.285+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9810054900 con 0x7f9800005180 2023-10-21T17:19:05.289 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.287+0000 7f52a4c12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:19:05.315 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.315+0000 7fb3f8812040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:05.346 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.344+0000 7f52a4c12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:19:05.370 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.369+0000 7fb3f8812040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:19:05.395 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.394+0000 7f52a4c12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:19:05.430 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.429+0000 7fb3f8812040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:19:05.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.478+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98141f2430 con 0x7f9800005180 2023-10-21T17:19:05.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.479+0000 7f9809ffb640 1 -- 172.21.15.53:0/3521343936 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9810057f20 con 0x7f9800005180 2023-10-21T17:19:05.480 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:19:05.482 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.481+0000 7fb3f8812040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:19:05.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f9818f17640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 msgr2=0x7f97f806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:05.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 msgr2=0x7f97f806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-21T17:19:05.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f9818f17640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 0x7f97f806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:05.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 0x7f97f806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 msgr2=0x7f98141592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f981000f910 tx=0x7f981000fdd0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 shutdown_connections 2023-10-21T17:19:05.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f97f806afc0 0x7f97f806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9814070ee0 0x7f98141f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98140c4c40 0x7f98140709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 --2- 172.21.15.53:0/3521343936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9800005180 0x7f98141592d0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 >> 172.21.15.53:0/3521343936 conn(0x7f981406a5d0 msgr2=0x7f981415acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 shutdown_connections 2023-10-21T17:19:05.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.482+0000 7f981b97c640 1 -- 172.21.15.53:0/3521343936 wait complete. 2023-10-21T17:19:05.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:19:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.759+0000 7efcfdf89640 1 Processor -- start 2023-10-21T17:19:05.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.759+0000 7efcfdf89640 1 -- start start 2023-10-21T17:19:05.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcf80afbe0 0x7efcf80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf806dfc0 0x7efcf8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 0x7efcf80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b2620 con 0x7efcf80afbe0 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf80b2060 con 0x7efcf8071070 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcfdf89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80b2340 con 0x7efcf806dfc0 2023-10-21T17:19:05.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcf80afbe0 0x7efcf80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efcf80afbe0 0x7efcf80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60718/0 (socket says 172.21.15.53:60718) 2023-10-21T17:19:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 -- 172.21.15.53:0/2197021559 learned_addr learned my addr 172.21.15.53:0/2197021559 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf6ffd640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf806dfc0 0x7efcf8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf7fff640 1 --2- 172.21.15.53:0/2197021559 >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 0x7efcf80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 -- 172.21.15.53:0/2197021559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf806dfc0 msgr2=0x7efcf8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf806dfc0 0x7efcf8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 -- 172.21.15.53:0/2197021559 >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 msgr2=0x7efcf80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 --2- 172.21.15.53:0/2197021559 >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 0x7efcf80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf77fe640 1 -- 172.21.15.53:0/2197021559 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8070350 con 0x7efcf80afbe0 2023-10-21T17:19:05.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.761+0000 7efcf7fff640 1 --2- 172.21.15.53:0/2197021559 >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 0x7efcf80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.763+0000 7efcf77fe640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf80afbe0 0x7efcf80b15b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7efcf815e970 tx=0x7efce802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba050c745ad8032d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.763+0000 7efcf67fc640 1 -- 172.21.15.53:0/2197021559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8031860 con 0x7efcf80afbe0 2023-10-21T17:19:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.763+0000 7efcf67fc640 1 -- 172.21.15.53:0/2197021559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce80319c0 con 0x7efcf80afbe0 2023-10-21T17:19:05.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.763+0000 7efcf67fc640 1 -- 172.21.15.53:0/2197021559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8031d50 con 0x7efcf80afbe0 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 -- 172.21.15.53:0/2197021559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf80afbe0 msgr2=0x7efcf80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf80afbe0 0x7efcf80b15b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7efcf815e970 tx=0x7efce802b900 comp rx=0 tx=0).stop 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 -- 172.21.15.53:0/2197021559 shutdown_connections 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf806dfc0 0x7efcf8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 --2- 172.21.15.53:0/2197021559 >> v2:172.21.15.124:3300/0 conn(0x7efcf8071070 0x7efcf80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 --2- 172.21.15.53:0/2197021559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf80afbe0 0x7efcf80b15b0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 -- 172.21.15.53:0/2197021559 >> 172.21.15.53:0/2197021559 conn(0x7efcf806a5d0 msgr2=0x7efcf80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 -- 172.21.15.53:0/2197021559 shutdown_connections 2023-10-21T17:19:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.764+0000 7efcfdf89640 1 -- 172.21.15.53:0/2197021559 wait complete. 2023-10-21T17:19:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.766+0000 7efcfdf89640 1 Processor -- start 2023-10-21T17:19:05.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.766+0000 7efcfdf89640 1 -- start start 2023-10-21T17:19:05.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf8071070 0x7efcf815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcf80afbe0 0x7efcf80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cc9b0 con 0x7efcf806dfc0 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80ccc90 con 0x7efcf80afbe0 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcfdf89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80ccf70 con 0x7efcf8071070 2023-10-21T17:19:05.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60720/0 (socket says 172.21.15.53:60720) 2023-10-21T17:19:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcf77fe640 1 -- 172.21.15.53:0/4134132443 learned_addr learned my addr 172.21.15.53:0/4134132443 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcf6ffd640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf8071070 0x7efcf815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.767+0000 7efcf7fff640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcf80afbe0 0x7efcf80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:05.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 -- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf8071070 msgr2=0x7efcf815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf8071070 0x7efcf815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 -- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcf80afbe0 msgr2=0x7efcf80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcf80afbe0 0x7efcf80cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 -- 172.21.15.53:0/4134132443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce80094d0 con 0x7efcf806dfc0 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcf77fe640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7efce802b940 tx=0x7efce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce803ad30 con 0x7efcf806dfc0 2023-10-21T17:19:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce80443f0 con 0x7efcf806dfc0 2023-10-21T17:19:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce80447a0 con 0x7efcf806dfc0 2023-10-21T17:19:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf80cd250 con 0x7efcf806dfc0 2023-10-21T17:19:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.768+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf80cc640 con 0x7efcf806dfc0 2023-10-21T17:19:05.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.770+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf804e160 con 0x7efcf806dfc0 2023-10-21T17:19:05.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.771+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7efce804d450 con 0x7efcf806dfc0 2023-10-21T17:19:05.776 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.769+0000 7f52a4c12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:19:05.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.771+0000 7efcdffff640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 0x7efce406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:05.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.772+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7efce80abff0 con 0x7efcf806dfc0 2023-10-21T17:19:05.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.772+0000 7efcf6ffd640 1 -- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 msgr2=0x7efce406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:05.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.772+0000 7efcf6ffd640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 0x7efce406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:05.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.776+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efce8076c00 con 0x7efcf806dfc0 2023-10-21T17:19:05.895 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:05.894+0000 7fb3f8812040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:19:05.932 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:05.930+0000 7f52a4c12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:19:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.968+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcf80b0ef0 con 0x7efcf806dfc0 2023-10-21T17:19:05.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.969+0000 7efcdffff640 1 -- 172.21.15.53:0/4134132443 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7efce8035020 con 0x7efcf806dfc0 2023-10-21T17:19:05.971 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:19:05.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 msgr2=0x7efce406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:05.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 0x7efce406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 msgr2=0x7efcf80c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:05.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7efce802b940 tx=0x7efce80046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 shutdown_connections 2023-10-21T17:19:05.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7efce406af90 0x7efce406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efcf80afbe0 0x7efcf80cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efcf8071070 0x7efcf815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 --2- 172.21.15.53:0/4134132443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efcf806dfc0 0x7efcf80c9190 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:05.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 >> 172.21.15.53:0/4134132443 conn(0x7efcf806a5d0 msgr2=0x7efcf80c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:05.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 shutdown_connections 2023-10-21T17:19:05.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:05.972+0000 7efcfdf89640 1 -- 172.21.15.53:0/4134132443 wait complete. 2023-10-21T17:19:05.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:19:06.042 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.040+0000 7f52a4c12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:19:06.071 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.070+0000 7fb3f8812040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:19:06.091 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.090+0000 7f52a4c12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:19:06.187 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.186+0000 7fb3f8812040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:19:06.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.196+0000 7f2e9e2bb640 1 Processor -- start 2023-10-21T17:19:06.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.196+0000 7f2e9e2bb640 1 -- start start 2023-10-21T17:19:06.198 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.197+0000 7f52a4c12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.198+0000 7f2e9e2bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e9815ce00 0x7f2e980cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9e2bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e98161090 0x7f2e980cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9e2bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980cde10 con 0x7f2e980cd470 2023-10-21T17:19:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9e2bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e98161920 con 0x7f2e98161090 2023-10-21T17:19:06.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9e2bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e980cb8b0 con 0x7f2e9815ce00 2023-10-21T17:19:06.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e98161090 0x7f2e980cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:06.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e98161090 0x7f2e980cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60736/0 (socket says 172.21.15.53:60736) 2023-10-21T17:19:06.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 -- 172.21.15.53:0/1605650836 learned_addr learned my addr 172.21.15.53:0/1605650836 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:06.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e97fff640 1 --2- 172.21.15.53:0/1605650836 >> v2:172.21.15.124:3300/0 conn(0x7f2e9815ce00 0x7f2e980cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:06.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 -- 172.21.15.53:0/1605650836 >> v2:172.21.15.124:3300/0 conn(0x7f2e9815ce00 msgr2=0x7f2e980cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:06.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 --2- 172.21.15.53:0/1605650836 >> v2:172.21.15.124:3300/0 conn(0x7f2e9815ce00 0x7f2e980cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 -- 172.21.15.53:0/1605650836 >> v1:172.21.15.124:6789/0 conn(0x7f2e980cd470 legacy=0x7f2e98160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:06.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.199+0000 7f2e9c858640 1 -- 172.21.15.53:0/1605650836 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98161640 con 0x7f2e98161090 2023-10-21T17:19:06.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.200+0000 7f2e9c858640 1 --2- 172.21.15.53:0/1605650836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e98161090 0x7f2e980cd8d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2e98071120 tx=0x7f2e9002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeee1e5cea7b885 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.201+0000 7f2e977fe640 1 -- 172.21.15.53:0/1605650836 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9002c9f0 con 0x7f2e98161090 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.201+0000 7f2e977fe640 1 -- 172.21.15.53:0/1605650836 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9002cb50 con 0x7f2e98161090 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.201+0000 7f2e977fe640 1 -- 172.21.15.53:0/1605650836 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900343f0 con 0x7f2e98161090 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.201+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/1605650836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e98161090 msgr2=0x7f2e980cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.201+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/1605650836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e98161090 0x7f2e980cd8d0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2e98071120 tx=0x7f2e9002be50 comp rx=0 tx=0).stop 2023-10-21T17:19:06.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/1605650836 shutdown_connections 2023-10-21T17:19:06.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/1605650836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e98161090 0x7f2e980cd8d0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/1605650836 >> v2:172.21.15.124:3300/0 conn(0x7f2e9815ce00 0x7f2e980cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/1605650836 >> 172.21.15.53:0/1605650836 conn(0x7f2e9806e650 msgr2=0x7f2e9806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:06.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/1605650836 shutdown_connections 2023-10-21T17:19:06.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.202+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/1605650836 wait complete. 2023-10-21T17:19:06.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.203+0000 7f2e9e2bb640 1 Processor -- start 2023-10-21T17:19:06.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.203+0000 7f2e9e2bb640 1 -- start start 2023-10-21T17:19:06.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e980cd470 0x7f2e980b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e98161090 0x7f2e981f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b6690 con 0x7f2e9815ce00 2023-10-21T17:19:06.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9815a360 con 0x7f2e98161090 2023-10-21T17:19:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.204+0000 7f2e9e2bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b5720 con 0x7f2e980cd470 2023-10-21T17:19:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60738/0 (socket says 172.21.15.53:60738) 2023-10-21T17:19:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 -- 172.21.15.53:0/3248926158 learned_addr learned my addr 172.21.15.53:0/3248926158 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:06.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e9c858640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e98161090 0x7f2e981f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:06.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e980cd470 msgr2=0x7f2e980b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:06.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e980cd470 0x7f2e980b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e98161090 msgr2=0x7f2e981f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:06.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e98161090 0x7f2e981f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 -- 172.21.15.53:0/3248926158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e90009b30 con 0x7f2e9815ce00 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.205+0000 7f2e8ffff640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000b3e0 tx=0x7f2e8000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.206+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011830 con 0x7f2e9815ce00 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.206+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e800119b0 con 0x7f2e9815ce00 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.206+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011d60 con 0x7f2e9815ce00 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.206+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e981f6470 con 0x7f2e9815ce00 2023-10-21T17:19:06.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.206+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e980b5970 con 0x7f2e9815ce00 2023-10-21T17:19:06.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.207+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9806cff0 con 0x7f2e9815ce00 2023-10-21T17:19:06.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.208+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f2e8001a400 con 0x7f2e9815ce00 2023-10-21T17:19:06.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.209+0000 7f2e957fa640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 0x7f2e7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:06.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.209+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f2e80089bf0 con 0x7f2e9815ce00 2023-10-21T17:19:06.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.209+0000 7f2e97fff640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 msgr2=0x7f2e7806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:06.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.209+0000 7f2e97fff640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 0x7f2e7806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:06.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.213+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e80054870 con 0x7f2e9815ce00 2023-10-21T17:19:06.243 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.242+0000 7fb3f8812040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:19:06.305 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.304+0000 7f52a4c12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:19:06.358 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.357+0000 7fb3f8812040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:06.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.410+0000 7f2e97fff640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 msgr2=0x7f2e7806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:06.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.410+0000 7f2e97fff640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 0x7f2e7806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:06.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.417+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e9804e160 con 0x7f2e9815ce00 2023-10-21T17:19:06.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.418+0000 7f2e957fa640 1 -- 172.21.15.53:0/3248926158 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v185) v1 ==== 80+0+474 (secure 0 0 0) 0x7f2e80015020 con 0x7f2e9815ce00 2023-10-21T17:19:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/234236878 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/234236878 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1818943929 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1207144287 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/207583581 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1931157828 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/880607286 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3805066261 2023-10-22T17:19:00.108230+0000 2023-10-21T17:19:06.421 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T17:19:06.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 msgr2=0x7f2e7806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:06.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 0x7f2e7806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 msgr2=0x7f2e98159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:06.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000b3e0 tx=0x7f2e8000b8a0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 shutdown_connections 2023-10-21T17:19:06.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f2e7806af90 0x7f2e7806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e98161090 0x7f2e981f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e9815ce00 0x7f2e98159e20 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 --2- 172.21.15.53:0/3248926158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e980cd470 0x7f2e980b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:06.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.420+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 >> 172.21.15.53:0/3248926158 conn(0x7f2e9806e650 msgr2=0x7f2e9815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:06.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.421+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 shutdown_connections 2023-10-21T17:19:06.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:06.421+0000 7f2e9e2bb640 1 -- 172.21.15.53:0/3248926158 wait complete. 2023-10-21T17:19:06.438 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:19:06.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:06.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:19:06.461 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.459+0000 7f52a4c12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:06.476 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.475+0000 7fb3f8812040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:19:06.510 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.509+0000 7f52a4c12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:06.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:19:06.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:06.540 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:19:06.628 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:06.627+0000 7f52a4c12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:19:06.635 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:19:06.635 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:06.647 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.646+0000 7fb3f8812040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:06.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:19:06.694 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:06.695 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:06.704 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.699+0000 7fb3f8812040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:06.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:19:06.752 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:19:06.752 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:06.808 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:19:06.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:06.829 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:06.828+0000 7fb3f8812040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:19:07.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.029+0000 7fca3bfff640 1 Processor -- start 2023-10-21T17:19:07.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.029+0000 7fca3bfff640 1 -- start start 2023-10-21T17:19:07.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3bfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fca3c15ab30 0x7fca3c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3bfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c15f7d0 con 0x7fca3c0adc70 2023-10-21T17:19:07.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3bfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c15f210 con 0x7fca3c0b07d0 2023-10-21T17:19:07.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3bfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c15f4f0 con 0x7fca3c15ab30 2023-10-21T17:19:07.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fca3c0adc70 0x7fca3c0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43618/0 (socket says 172.21.15.53:43618) 2023-10-21T17:19:07.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3a7fc640 1 -- 172.21.15.53:0/3684223690 learned_addr learned my addr 172.21.15.53:0/3684223690 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:07.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca3affd640 1 --2- 172.21.15.53:0/3684223690 >> v2:172.21.15.124:3300/0 conn(0x7fca3c15ab30 0x7fca3c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197724700 0 0) 0x7fca3c15f7d0 con 0x7fca3c0adc70 2023-10-21T17:19:07.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.031+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca24003520 con 0x7fca3c0adc70 2023-10-21T17:19:07.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448217649 0 0) 0x7fca24003520 con 0x7fca3c0adc70 2023-10-21T17:19:07.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 >> v2:172.21.15.124:3300/0 conn(0x7fca3c15ab30 msgr2=0x7fca3c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:07.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 --2- 172.21.15.53:0/3684223690 >> v2:172.21.15.124:3300/0 conn(0x7fca3c15ab30 0x7fca3c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 >> v1:172.21.15.124:6789/0 conn(0x7fca3c0b07d0 legacy=0x7fca3c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c06c550 con 0x7fca3c0adc70 2023-10-21T17:19:07.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fca30001670 con 0x7fca3c0adc70 2023-10-21T17:19:07.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 >> v1:172.21.15.53:6789/0 conn(0x7fca3c0adc70 legacy=0x7fca3c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24003970 0x7fca24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca240047c0 0x7fca24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 0x7fca2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15f4f0 con 0x7fca24005180 2023-10-21T17:19:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2400c020 con 0x7fca24003970 2023-10-21T17:19:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2400c1e0 con 0x7fca240047c0 2023-10-21T17:19:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.032+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 0x7fca2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3a7fc640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca240047c0 0x7fca24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3affd640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24003970 0x7fca24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3b7fe640 1 -- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca240047c0 msgr2=0x7fca24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca240047c0 0x7fca24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3b7fe640 1 -- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24003970 msgr2=0x7fca24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:07.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24003970 0x7fca24003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3b7fe640 1 -- 172.21.15.53:0/3684223690 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24003520 con 0x7fca24005180 2023-10-21T17:19:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3a7fc640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca240047c0 0x7fca24004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.033+0000 7fca3affd640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24003970 0x7fca24003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.034+0000 7fca3b7fe640 1 -- 172.21.15.53:0/3684223690 reap_dead start 2023-10-21T17:19:07.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.034+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 0x7fca2400b980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fca34001e50 tx=0x7fca240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e028d48fb0d6b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.034+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34003b70 con 0x7fca24005180 2023-10-21T17:19:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.034+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34003d10 con 0x7fca24005180 2023-10-21T17:19:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.034+0000 7fca39ffb640 1 -- 172.21.15.53:0/3684223690 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34005310 con 0x7fca24005180 2023-10-21T17:19:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 -- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 msgr2=0x7fca2400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 0x7fca2400b980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fca34001e50 tx=0x7fca240057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 -- 172.21.15.53:0/3684223690 shutdown_connections 2023-10-21T17:19:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 --2- 172.21.15.53:0/3684223690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca24005180 0x7fca2400b980 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 -- 172.21.15.53:0/3684223690 >> 172.21.15.53:0/3684223690 conn(0x7fca3c0cb0c0 msgr2=0x7fca3c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 -- 172.21.15.53:0/3684223690 shutdown_connections 2023-10-21T17:19:07.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.035+0000 7fca3bfff640 1 -- 172.21.15.53:0/3684223690 wait complete. 2023-10-21T17:19:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.037+0000 7fca3bfff640 1 Processor -- start 2023-10-21T17:19:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.037+0000 7fca3bfff640 1 -- start start 2023-10-21T17:19:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24005180 0x7fca3c14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca3c0c5710 0x7fca3c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0ad880 con 0x7fca3c157e40 2023-10-21T17:19:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c14f440 con 0x7fca24005180 2023-10-21T17:19:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3bfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1582c0 con 0x7fca3c0c5710 2023-10-21T17:19:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.038+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60750/0 (socket says 172.21.15.53:60750) 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 -- 172.21.15.53:0/4266930581 learned_addr learned my addr 172.21.15.53:0/4266930581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3affd640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24005180 0x7fca3c14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3a7fc640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca3c0c5710 0x7fca3c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca3c0c5710 msgr2=0x7fca3c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca3c0c5710 0x7fca3c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24005180 msgr2=0x7fca3c14ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24005180 0x7fca3c14ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 -- 172.21.15.53:0/4266930581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0c5e80 con 0x7fca3c157e40 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.039+0000 7fca3b7fe640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fca34001f70 tx=0x7fca340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.040+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34006d50 con 0x7fca3c157e40 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.040+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34003b80 con 0x7fca3c157e40 2023-10-21T17:19:07.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.040+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c0c6160 con 0x7fca3c157e40 2023-10-21T17:19:07.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.040+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34039680 con 0x7fca3c157e40 2023-10-21T17:19:07.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.040+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c0cc590 con 0x7fca3c157e40 2023-10-21T17:19:07.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.041+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0cc7b0 con 0x7fca3c157e40 2023-10-21T17:19:07.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.042+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fca340397e0 con 0x7fca3c157e40 2023-10-21T17:19:07.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.044+0000 7fca23fff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 0x7fca2806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.044+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fca340aba20 con 0x7fca3c157e40 2023-10-21T17:19:07.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.044+0000 7fca3affd640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 msgr2=0x7fca2806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:07.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.044+0000 7fca3affd640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 0x7fca2806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:07.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.047+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca340766e0 con 0x7fca3c157e40 2023-10-21T17:19:07.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.245+0000 7fca3affd640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 msgr2=0x7fca2806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:07.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.245+0000 7fca3affd640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 0x7fca2806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:07.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.277+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca3c04e160 con 0x7fca3c157e40 2023-10-21T17:19:07.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.278+0000 7fca23fff640 1 -- 172.21.15.53:0/4266930581 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fca34047020 con 0x7fca3c157e40 2023-10-21T17:19:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:07.280 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:19:07.280 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:19:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 msgr2=0x7fca2806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:07.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 0x7fca2806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 msgr2=0x7fca3c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fca34001f70 tx=0x7fca340014a0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 shutdown_connections 2023-10-21T17:19:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7fca2806afc0 0x7fca2806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca3c157e40 0x7fca3c0ab8d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fca3c0c5710 0x7fca3c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 --2- 172.21.15.53:0/4266930581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fca24005180 0x7fca3c14ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 >> 172.21.15.53:0/4266930581 conn(0x7fca3c0cb0c0 msgr2=0x7fca3c070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 shutdown_connections 2023-10-21T17:19:07.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.281+0000 7fca3bfff640 1 -- 172.21.15.53:0/4266930581 wait complete. 2023-10-21T17:19:07.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:07.455 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:07.454+0000 7f52a4c12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:19:07.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.535+0000 7f6e13fff640 1 Processor -- start 2023-10-21T17:19:07.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.535+0000 7f6e13fff640 1 -- start start 2023-10-21T17:19:07.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.537+0000 7f6e13fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e1415ab30 0x7f6e1415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.537+0000 7f6e13fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1415f7d0 con 0x7f6e140adc70 2023-10-21T17:19:07.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.537+0000 7f6e13fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1415f210 con 0x7f6e140b07d0 2023-10-21T17:19:07.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.537+0000 7f6e13fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1415f4f0 con 0x7f6e1415ab30 2023-10-21T17:19:07.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.538+0000 7f6e0a7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6e140adc70 0x7f6e140b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43632/0 (socket says 172.21.15.53:43632) 2023-10-21T17:19:07.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.538+0000 7f6e0a7fc640 1 -- 172.21.15.53:0/3813687296 learned_addr learned my addr 172.21.15.53:0/3813687296 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:07.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.538+0000 7f6e12ffd640 1 --2- 172.21.15.53:0/3813687296 >> v2:172.21.15.124:3300/0 conn(0x7f6e1415ab30 0x7f6e1415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.538+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100235071 0 0) 0x7f6e1415f7d0 con 0x7f6e140adc70 2023-10-21T17:19:07.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.538+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e00003520 con 0x7f6e140adc70 2023-10-21T17:19:07.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.539+0000 7f6e12ffd640 1 -- 172.21.15.53:0/3813687296 >> v1:172.21.15.124:6789/0 conn(0x7f6e140b07d0 legacy=0x7f6e1406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.539+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105330990 0 0) 0x7f6e00003520 con 0x7f6e140adc70 2023-10-21T17:19:07.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.539+0000 7f6e12ffd640 1 -- 172.21.15.53:0/3813687296 >> v1:172.21.15.53:6789/0 conn(0x7f6e140adc70 legacy=0x7f6e140b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.539+0000 7f6e12ffd640 1 -- 172.21.15.53:0/3813687296 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1406c550 con 0x7f6e1415ab30 2023-10-21T17:19:07.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.540+0000 7f6e12ffd640 1 --2- 172.21.15.53:0/3813687296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e1415ab30 0x7f6e1415e8a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb1dbfe94fd1c9b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:07.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.540+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc001670 con 0x7f6e1415ab30 2023-10-21T17:19:07.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.540+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0402c940 con 0x7f6e1415ab30 2023-10-21T17:19:07.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.540+0000 7f6e127fc640 1 -- 172.21.15.53:0/3813687296 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402ccd0 con 0x7f6e1415ab30 2023-10-21T17:19:07.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 -- 172.21.15.53:0/3813687296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e1415ab30 msgr2=0x7f6e1415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 --2- 172.21.15.53:0/3813687296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e1415ab30 0x7f6e1415e8a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0402be10 comp rx=0 tx=0).stop 2023-10-21T17:19:07.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 -- 172.21.15.53:0/3813687296 shutdown_connections 2023-10-21T17:19:07.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 --2- 172.21.15.53:0/3813687296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e1415ab30 0x7f6e1415e8a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 -- 172.21.15.53:0/3813687296 >> 172.21.15.53:0/3813687296 conn(0x7f6e140cb0c0 msgr2=0x7f6e140cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:07.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.541+0000 7f6e13fff640 1 -- 172.21.15.53:0/3813687296 shutdown_connections 2023-10-21T17:19:07.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.542+0000 7f6e13fff640 1 -- 172.21.15.53:0/3813687296 wait complete. 2023-10-21T17:19:07.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.544+0000 7f6e13fff640 1 Processor -- start 2023-10-21T17:19:07.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.544+0000 7f6e13fff640 1 -- start start 2023-10-21T17:19:07.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 0x7f6e14150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 0x7f6e140b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c7420 con 0x7f6e140adc70 2023-10-21T17:19:07.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c6390 con 0x7f6e140b07d0 2023-10-21T17:19:07.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.545+0000 7f6e13fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140c6670 con 0x7f6e1415ab30 2023-10-21T17:19:07.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60764/0 (socket says 172.21.15.53:60764) 2023-10-21T17:19:07.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 -- 172.21.15.53:0/1575236189 learned_addr learned my addr 172.21.15.53:0/1575236189 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:07.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e137fe640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 0x7f6e140b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e0a7fc640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 0x7f6e14150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 msgr2=0x7f6e140b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 0x7f6e140b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 msgr2=0x7f6e14150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 0x7f6e14150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 -- 172.21.15.53:0/1575236189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04009a70 con 0x7f6e140adc70 2023-10-21T17:19:07.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e0a7fc640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 0x7f6e14150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:07.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e137fe640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 0x7f6e140b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:07.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.546+0000 7f6e12ffd640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6e04037040 tx=0x7f6e04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:07.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.547+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04009450 con 0x7f6e140adc70 2023-10-21T17:19:07.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.547+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04003990 con 0x7f6e140adc70 2023-10-21T17:19:07.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.547+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140c6dd0 con 0x7f6e140adc70 2023-10-21T17:19:07.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.547+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003d40 con 0x7f6e140adc70 2023-10-21T17:19:07.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.547+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e140c7df0 con 0x7f6e140adc70 2023-10-21T17:19:07.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.548+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1404e160 con 0x7f6e140adc70 2023-10-21T17:19:07.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.549+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 92) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6e0403b050 con 0x7f6e140adc70 2023-10-21T17:19:07.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.550+0000 7f6e0bfff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 0x7f6df406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.550+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f6e040ac0e0 con 0x7f6e140adc70 2023-10-21T17:19:07.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.550+0000 7f6e0a7fc640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 msgr2=0x7f6df406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:07.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.550+0000 7f6e0a7fc640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 0x7f6df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:07.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.558+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e04076cf0 con 0x7f6e140adc70 2023-10-21T17:19:07.685 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:07.684+0000 7fb3f8812040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:19:07.751 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:07.750+0000 7f52a4c12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:19:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.751+0000 7f6e0a7fc640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 msgr2=0x7f6df406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/97801354 2023-10-21T17:19:07.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.751+0000 7f6e0a7fc640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 0x7f6df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:07.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.752+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e140ada50 con 0x7f6e140adc70 2023-10-21T17:19:07.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.755+0000 7f6e0bfff640 1 -- 172.21.15.53:0/1575236189 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17403 (secure 0 0 0) 0x7f6e0407a310 con 0x7f6e140adc70 2023-10-21T17:19:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:07.757 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":185,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:00.108345+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:6832/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1818943929":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1207144287":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/207583581":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1931157828":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/880607286":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/3805066261":"2023-10-22T17:19:00.108230+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:07.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 msgr2=0x7f6df406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:07.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 0x7f6df406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 msgr2=0x7f6e140c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6e04037040 tx=0x7f6e04004a40 comp rx=0 tx=0).stop 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 shutdown_connections 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:6832/97801354,v1:172.21.15.124:6833/97801354] conn(0x7f6df406af90 0x7f6df406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e1415ab30 0x7f6e140b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e140b07d0 0x7f6e14150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 --2- 172.21.15.53:0/1575236189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e140adc70 0x7f6e140c78b0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 >> 172.21.15.53:0/1575236189 conn(0x7f6e140cb0c0 msgr2=0x7f6e140ac4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 shutdown_connections 2023-10-21T17:19:07.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.758+0000 7f6e13fff640 1 -- 172.21.15.53:0/1575236189 wait complete. 2023-10-21T17:19:07.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:18:43.336266+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:18:44.404625+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:19:07.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:19:07.808 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:07.806+0000 7f52a4c12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:07.923 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:07.922+0000 7f52a4c12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:07.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.992+0000 7f61eea1f640 1 Processor -- start 2023-10-21T17:19:07.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.992+0000 7f61eea1f640 1 -- start start 2023-10-21T17:19:07.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.994+0000 7f61eea1f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61e815a820 0x7f61e8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.994+0000 7f61eea1f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:07.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.994+0000 7f61eea1f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80c7ad0 con 0x7f61e8070b50 2023-10-21T17:19:07.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.994+0000 7f61eea1f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e80c9a40 con 0x7f61e815a820 2023-10-21T17:19:07.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.994+0000 7f61eea1f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e815ac00 con 0x7f61e80c9080 2023-10-21T17:19:07.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61dffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61e815a820 0x7f61e8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:07.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61dffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:52022/0 (socket says 172.21.15.53:52022) 2023-10-21T17:19:07.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61ecfbc640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f61e8070b50 0x7f61e80c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:56086/0 (socket says 172.21.15.53:56086) 2023-10-21T17:19:07.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61e815a820 0x7f61e8160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:46796/0 (socket says 172.21.15.53:46796) 2023-10-21T17:19:07.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61dffff640 1 -- 172.21.15.53:0/3784152855 learned_addr learned my addr 172.21.15.53:0/3784152855 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:08.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e77fe640 1 -- 172.21.15.53:0/3784152855 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448345094 0 0) 0x7f61e80c7ad0 con 0x7f61e8070b50 2023-10-21T17:19:08.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e77fe640 1 -- 172.21.15.53:0/3784152855 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003520 con 0x7f61e8070b50 2023-10-21T17:19:08.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 -- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 msgr2=0x7f61e80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:08.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 -- 172.21.15.53:0/3784152855 >> v1:172.21.15.124:6789/0 conn(0x7f61e8070b50 legacy=0x7f61e80c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61e7fff640 1 -- 172.21.15.53:0/3784152855 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80cb210 con 0x7f61e815a820 2023-10-21T17:19:08.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.995+0000 7f61dffff640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:19:08.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.996+0000 7f61e7fff640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e8160750 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f61d80097a0 tx=0x7f61d802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e633f37da0a1590b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61e77fe640 1 -- 172.21.15.53:0/3784152855 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d802d040 con 0x7f61e815a820 2023-10-21T17:19:08.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61e77fe640 1 -- 172.21.15.53:0/3784152855 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d8031810 con 0x7f61e815a820 2023-10-21T17:19:08.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61e77fe640 1 -- 172.21.15.53:0/3784152855 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8031ba0 con 0x7f61e815a820 2023-10-21T17:19:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 -- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 msgr2=0x7f61e8160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e8160750 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f61d80097a0 tx=0x7f61d802b8b0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 -- 172.21.15.53:0/3784152855 shutdown_connections 2023-10-21T17:19:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e80c9080 0x7f61e80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 --2- 172.21.15.53:0/3784152855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e8160750 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.997+0000 7f61eea1f640 1 -- 172.21.15.53:0/3784152855 >> 172.21.15.53:0/3784152855 conn(0x7f61e8069d50 msgr2=0x7f61e806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:08.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.998+0000 7f61eea1f640 1 -- 172.21.15.53:0/3784152855 shutdown_connections 2023-10-21T17:19:08.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:07.998+0000 7f61eea1f640 1 -- 172.21.15.53:0/3784152855 wait complete. 2023-10-21T17:19:08.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.000+0000 7f61eea1f640 1 Processor -- start 2023-10-21T17:19:08.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.000+0000 7f61eea1f640 1 -- start start 2023-10-21T17:19:08.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 0x7f61e8144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e81418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8158a70 con 0x7f61e80c9080 2023-10-21T17:19:08.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e8144fc0 con 0x7f61e815a820 2023-10-21T17:19:08.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.001+0000 7f61eea1f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e81452a0 con 0x7f61e8070b50 2023-10-21T17:19:08.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61e7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 0x7f61e8144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61ecfbc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e81418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60778/0 (socket says 172.21.15.53:60778) 2023-10-21T17:19:08.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 -- 172.21.15.53:0/1940144160 learned_addr learned my addr 172.21.15.53:0/1940144160 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:08.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 -- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 msgr2=0x7f61e8144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 0x7f61e8144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 -- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 msgr2=0x7f61e81418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e81418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 -- 172.21.15.53:0/1940144160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00094d0 con 0x7f61e80c9080 2023-10-21T17:19:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61e7fff640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 0x7f61e8144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:19:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.002+0000 7f61dffff640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e9d0 tx=0x7f61d000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.003+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000ca60 con 0x7f61e80c9080 2023-10-21T17:19:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.003+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d000cbc0 con 0x7f61e80c9080 2023-10-21T17:19:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.003+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d0013470 con 0x7f61e80c9080 2023-10-21T17:19:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.003+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d8009400 con 0x7f61e80c9080 2023-10-21T17:19:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.003+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e8145580 con 0x7f61e80c9080 2023-10-21T17:19:08.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.004+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e81588d0 con 0x7f61e80c9080 2023-10-21T17:19:08.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.005+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 93) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f61d00135d0 con 0x7f61e80c9080 2023-10-21T17:19:08.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.006+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f61d0089860 con 0x7f61e80c9080 2023-10-21T17:19:08.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.011+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61e81588d0 con 0x7f61e80c9080 2023-10-21T17:19:08.021 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:08.020+0000 7fb3f8812040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:19:08.023 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5226acb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5226acb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5226acb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5226acb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5226acb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.025 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5229ad1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.025 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5229ad1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.025 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5229ad1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.026 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5229ad1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.026 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:08.022+0000 7f5229ad1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:08.082 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:08.081+0000 7fb3f8812040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:08.202 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:08.201+0000 7fb3f8812040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:08.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.202+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61e81588d0 con 0x7f61e80c9080 2023-10-21T17:19:08.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.203+0000 7f61e57fa640 1 -- 172.21.15.53:0/1940144160 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61e81588d0 con 0x7f61e80c9080 2023-10-21T17:19:08.204 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:19:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 msgr2=0x7f61e80cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e9d0 tx=0x7f61d000ee90 comp rx=0 tx=0).stop 2023-10-21T17:19:08.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 shutdown_connections 2023-10-21T17:19:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61e815a820 0x7f61e81418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61e80c9080 0x7f61e80cce10 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 --2- 172.21.15.53:0/1940144160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61e8070b50 0x7f61e8144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 >> 172.21.15.53:0/1940144160 conn(0x7f61e8069d50 msgr2=0x7f61e80c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:08.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 shutdown_connections 2023-10-21T17:19:08.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.206+0000 7f61eea1f640 1 -- 172.21.15.53:0/1940144160 wait complete. 2023-10-21T17:19:08.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:19:08.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.433+0000 7f0ff9467640 1 Processor -- start 2023-10-21T17:19:08.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.433+0000 7f0ff9467640 1 -- start start 2023-10-21T17:19:08.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.434+0000 7f0ff9467640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0ff40cf2b0 0x7f0ff40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff9467640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40cd520 con 0x7f0ff40cf690 2023-10-21T17:19:08.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff9467640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40d1910 con 0x7f0ff40ce510 2023-10-21T17:19:08.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff9467640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40ce080 con 0x7f0ff40cf2b0 2023-10-21T17:19:08.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff37fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0ff40cf690 0x7f0ff40d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43636/0 (socket says 172.21.15.53:43636) 2023-10-21T17:19:08.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff37fe640 1 -- 172.21.15.53:0/900978404 learned_addr learned my addr 172.21.15.53:0/900978404 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:08.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.435+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/900978404 >> v2:172.21.15.124:3300/0 conn(0x7f0ff40cf2b0 0x7f0ff40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768593740 0 0) 0x7f0ff40cd520 con 0x7f0ff40cf690 2023-10-21T17:19:08.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc003520 con 0x7f0ff40cf690 2023-10-21T17:19:08.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185963330 0 0) 0x7f0fdc003520 con 0x7f0ff40cf690 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 >> v2:172.21.15.124:3300/0 conn(0x7f0ff40cf2b0 msgr2=0x7f0ff40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 --2- 172.21.15.53:0/900978404 >> v2:172.21.15.124:3300/0 conn(0x7f0ff40cf2b0 0x7f0ff40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 >> v1:172.21.15.124:6789/0 conn(0x7f0ff40ce510 legacy=0x7f0ff415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40cd750 con 0x7f0ff40cf690 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0fe8001670 con 0x7f0ff40cf690 2023-10-21T17:19:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 >> v1:172.21.15.53:6789/0 conn(0x7f0ff40cf690 legacy=0x7f0ff40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0fdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fdc0051d0 0x7f0fdc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40ce080 con 0x7f0fdc0039c0 2023-10-21T17:19:08.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc00c120 con 0x7f0fdc0047c0 2023-10-21T17:19:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc00c2e0 con 0x7f0fdc0051d0 2023-10-21T17:19:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.436+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0fdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff37fe640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff3fff640 1 -- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fdc0051d0 msgr2=0x7f0fdc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:08.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0fdc0051d0 0x7f0fdc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff3fff640 1 -- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fdc0047c0 msgr2=0x7f0fdc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff3fff640 1 -- 172.21.15.53:0/900978404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc003520 con 0x7f0fdc0039c0 2023-10-21T17:19:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff37fe640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.437+0000 7f0ff8c66640 1 -- 172.21.15.53:0/900978404 reap_dead start 2023-10-21T17:19:08.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.438+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0fdc003de0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009f90 tx=0x7f0fdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc47efe1fc8f6fc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.438+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe402cbf0 con 0x7f0fdc0039c0 2023-10-21T17:19:08.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.438+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400f040 con 0x7f0fdc0039c0 2023-10-21T17:19:08.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.438+0000 7f0ff2ffd640 1 -- 172.21.15.53:0/900978404 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4034440 con 0x7f0fdc0039c0 2023-10-21T17:19:08.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 -- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 msgr2=0x7f0fdc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0fdc003de0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009f90 tx=0x7f0fdc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 -- 172.21.15.53:0/900978404 shutdown_connections 2023-10-21T17:19:08.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 --2- 172.21.15.53:0/900978404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0fdc003de0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 -- 172.21.15.53:0/900978404 >> 172.21.15.53:0/900978404 conn(0x7f0ff409bc40 msgr2=0x7f0ff40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:08.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 -- 172.21.15.53:0/900978404 shutdown_connections 2023-10-21T17:19:08.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.439+0000 7f0ff9467640 1 -- 172.21.15.53:0/900978404 wait complete. 2023-10-21T17:19:08.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.441+0000 7f0ff9467640 1 Processor -- start 2023-10-21T17:19:08.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.441+0000 7f0ff9467640 1 -- start start 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ff41f32c0 0x7f0ff4151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ff40cba30 0x7f0ff40cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cc3f0 con 0x7f0fdc0039c0 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cac80 con 0x7f0ff40cba30 2023-10-21T17:19:08.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff9467640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41f3740 con 0x7f0ff41f32c0 2023-10-21T17:19:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60788/0 (socket says 172.21.15.53:60788) 2023-10-21T17:19:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.442+0000 7f0ff3fff640 1 -- 172.21.15.53:0/2739323182 learned_addr learned my addr 172.21.15.53:0/2739323182 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff37fe640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ff41f32c0 0x7f0ff4151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff8c66640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ff40cba30 0x7f0ff40cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 -- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ff41f32c0 msgr2=0x7f0ff4151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ff41f32c0 0x7f0ff4151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 -- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ff40cba30 msgr2=0x7f0ff40cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ff40cba30 0x7f0ff40cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 -- 172.21.15.53:0/2739323182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40ca070 con 0x7f0fdc0039c0 2023-10-21T17:19:08.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff3fff640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c590 tx=0x7f0fe4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4034e50 con 0x7f0fdc0039c0 2023-10-21T17:19:08.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe4040450 con 0x7f0fdc0039c0 2023-10-21T17:19:08.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.443+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4040800 con 0x7f0fdc0039c0 2023-10-21T17:19:08.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.444+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40ca350 con 0x7f0fdc0039c0 2023-10-21T17:19:08.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.444+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40ca910 con 0x7f0fdc0039c0 2023-10-21T17:19:08.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.445+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff40caaa0 con 0x7f0fdc0039c0 2023-10-21T17:19:08.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.446+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 93) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0fe4040960 con 0x7f0fdc0039c0 2023-10-21T17:19:08.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.448+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f0fe40abe90 con 0x7f0fdc0039c0 2023-10-21T17:19:08.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.452+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fe4076e30 con 0x7f0fdc0039c0 2023-10-21T17:19:08.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.646+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ff4074150 con 0x7f0fdc0039c0 2023-10-21T17:19:08.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.647+0000 7f0ff0ff9640 1 -- 172.21.15.53:0/2739323182 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0fe4044020 con 0x7f0fdc0039c0 2023-10-21T17:19:08.649 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:19:08.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 msgr2=0x7f0ff40c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402c590 tx=0x7f0fe4009750 comp rx=0 tx=0).stop 2023-10-21T17:19:08.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 shutdown_connections 2023-10-21T17:19:08.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ff40cba30 0x7f0ff40cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ff41f32c0 0x7f0ff4151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 --2- 172.21.15.53:0/2739323182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fdc0039c0 0x7f0ff40c99e0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 >> 172.21.15.53:0/2739323182 conn(0x7f0ff409bc40 msgr2=0x7f0ff40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:08.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 shutdown_connections 2023-10-21T17:19:08.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.650+0000 7f0ff9467640 1 -- 172.21.15.53:0/2739323182 wait complete. 2023-10-21T17:19:08.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:19:08.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.889+0000 7fad442a9640 1 Processor -- start 2023-10-21T17:19:08.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.890+0000 7fad442a9640 1 -- start start 2023-10-21T17:19:08.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.891+0000 7fad442a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad3c15ab10 0x7fad3c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.891+0000 7fad442a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c069d50 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.891+0000 7fad442a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c06b5a0 con 0x7fad3c0c9cd0 2023-10-21T17:19:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.891+0000 7fad442a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c15aef0 con 0x7fad3c15ab10 2023-10-21T17:19:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad3c15ab10 0x7fad3c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fad3c15ab10 0x7fad3c0cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60800/0 (socket says 172.21.15.53:60800) 2023-10-21T17:19:08.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 -- 172.21.15.53:0/288340065 learned_addr learned my addr 172.21.15.53:0/288340065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:08.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 -- 172.21.15.53:0/288340065 >> v1:172.21.15.124:6789/0 conn(0x7fad3c0c9cd0 legacy=0x7fad3c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:08.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 -- 172.21.15.53:0/288340065 >> v1:172.21.15.53:6789/0 conn(0x7fad3c0c5bf0 legacy=0x7fad3c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.892+0000 7fad42045640 1 -- 172.21.15.53:0/288340065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c069f80 con 0x7fad3c15ab10 2023-10-21T17:19:08.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.893+0000 7fad42045640 1 --2- 172.21.15.53:0/288340065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c15ab10 0x7fad3c0cb520 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fad38009870 tx=0x7fad3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b05bf646319be server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad41043640 1 -- 172.21.15.53:0/288340065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38031870 con 0x7fad3c15ab10 2023-10-21T17:19:08.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad41043640 1 -- 172.21.15.53:0/288340065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad380319d0 con 0x7fad3c15ab10 2023-10-21T17:19:08.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad41043640 1 -- 172.21.15.53:0/288340065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38031d60 con 0x7fad3c15ab10 2023-10-21T17:19:08.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad442a9640 1 -- 172.21.15.53:0/288340065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c15ab10 msgr2=0x7fad3c0cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad442a9640 1 --2- 172.21.15.53:0/288340065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c15ab10 0x7fad3c0cb520 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fad38009870 tx=0x7fad3802b910 comp rx=0 tx=0).stop 2023-10-21T17:19:08.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad442a9640 1 -- 172.21.15.53:0/288340065 shutdown_connections 2023-10-21T17:19:08.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad442a9640 1 --2- 172.21.15.53:0/288340065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c15ab10 0x7fad3c0cb520 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.894+0000 7fad442a9640 1 -- 172.21.15.53:0/288340065 >> 172.21.15.53:0/288340065 conn(0x7fad3c0b0af0 msgr2=0x7fad3c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:08.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.895+0000 7fad442a9640 1 -- 172.21.15.53:0/288340065 shutdown_connections 2023-10-21T17:19:08.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.895+0000 7fad442a9640 1 -- 172.21.15.53:0/288340065 wait complete. 2023-10-21T17:19:08.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.896+0000 7fad442a9640 1 Processor -- start 2023-10-21T17:19:08.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.896+0000 7fad442a9640 1 -- start start 2023-10-21T17:19:08.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad3c0c9cd0 0x7fad3c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad3c15ab10 0x7fad3c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0cd450 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0b3b90 con 0x7fad3c0c9cd0 2023-10-21T17:19:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.897+0000 7fad442a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0cd6a0 con 0x7fad3c15ab10 2023-10-21T17:19:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60806/0 (socket says 172.21.15.53:60806) 2023-10-21T17:19:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 -- 172.21.15.53:0/1717473310 learned_addr learned my addr 172.21.15.53:0/1717473310 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 -- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad3c15ab10 msgr2=0x7fad3c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad3c15ab10 0x7fad3c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 -- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad3c0c9cd0 msgr2=0x7fad3c151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:08.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad3c0c9cd0 0x7fad3c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 -- 172.21.15.53:0/1717473310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380094d0 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.898+0000 7fad42045640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fad38000c00 tx=0x7fad380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.899+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38031820 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.899+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad380443f0 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.899+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad380447a0 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.899+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c0cd980 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.899+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c0b40c0 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.900+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c0caed0 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.901+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 93) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fad3804d450 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.902+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fad3802d040 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.907+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad38076d60 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.949+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 94) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fad38076810 con 0x7fad3c0c5bf0 2023-10-21T17:19:08.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.950+0000 7fad32ffd640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 0x7fad20073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:08.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.951+0000 7fad41844640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 0x7fad20073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:08.970+0000 7fad41844640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 0x7fad20073090 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000f60 tx=0x7fad2c0079b0 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.106+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad3c0b42b0 con 0x7fad3c0c5bf0 2023-10-21T17:19:09.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.106+0000 7fad32ffd640 1 -- 172.21.15.53:0/1717473310 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad38091b70 con 0x7fad3c0c5bf0 2023-10-21T17:19:09.108 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:19:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 msgr2=0x7fad20073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 0x7fad20073090 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad2c000f60 tx=0x7fad2c0079b0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 msgr2=0x7fad3c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fad38000c00 tx=0x7fad380046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 shutdown_connections 2023-10-21T17:19:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad3c15ab10 0x7fad3c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad3c0c9cd0 0x7fad3c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fad20070ca0 0x7fad20073090 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 --2- 172.21.15.53:0/1717473310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad3c0c5bf0 0x7fad3c0b45e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 >> 172.21.15.53:0/1717473310 conn(0x7fad3c0b0af0 msgr2=0x7fad3c0ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:09.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 shutdown_connections 2023-10-21T17:19:09.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.109+0000 7fad442a9640 1 -- 172.21.15.53:0/1717473310 wait complete. 2023-10-21T17:19:09.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:09.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.349+0000 7f7545067640 1 Processor -- start 2023-10-21T17:19:09.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.349+0000 7f7545067640 1 -- start start 2023-10-21T17:19:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7545067640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f75400ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7545067640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f75400b07d0 0x7f754006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7545067640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75400cacf0 con 0x7f75400cc210 2023-10-21T17:19:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7545067640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75400b10f0 con 0x7f75400b07d0 2023-10-21T17:19:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7545067640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400ca3c0 con 0x7f75401599d0 2023-10-21T17:19:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7537fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f75400cc210 0x7f75400b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43658/0 (socket says 172.21.15.53:43658) 2023-10-21T17:19:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.351+0000 7f7537fff640 1 -- 172.21.15.53:0/1160120196 learned_addr learned my addr 172.21.15.53:0/1160120196 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:09.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753ffff640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f75400ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f7544866640 1 --2- 172.21.15.53:0/1160120196 >> v2:172.21.15.124:3300/0 conn(0x7f75400b07d0 0x7f754006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685846155 0 0) 0x7f75400cacf0 con 0x7f75400cc210 2023-10-21T17:19:09.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c003520 con 0x7f75400cc210 2023-10-21T17:19:09.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085267592 0 0) 0x7f752c003520 con 0x7f75400cc210 2023-10-21T17:19:09.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 msgr2=0x7f75400ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f75400ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 >> v2:172.21.15.124:3300/0 conn(0x7f75400b07d0 msgr2=0x7f754006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 --2- 172.21.15.53:0/1160120196 >> v2:172.21.15.124:3300/0 conn(0x7f75400b07d0 0x7f754006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400ca6a0 con 0x7f75400cc210 2023-10-21T17:19:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7528001670 con 0x7f75400cc210 2023-10-21T17:19:09.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 >> v1:172.21.15.53:6789/0 conn(0x7f75400cc210 legacy=0x7f75400b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f752c003970 0x7f752c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f752c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b10f0 con 0x7f752c0047c0 2023-10-21T17:19:09.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400ca3c0 con 0x7f752c003970 2023-10-21T17:19:09.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.352+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00b980 con 0x7f75401599d0 2023-10-21T17:19:09.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7544866640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f752c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 -- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 msgr2=0x7f752c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:09.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f752c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 -- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f752c003970 msgr2=0x7f752c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:09.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f752c003970 0x7f752c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7537fff640 1 -- 172.21.15.53:0/1160120196 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c003520 con 0x7f752c0047c0 2023-10-21T17:19:09.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.353+0000 7f7544866640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75401599d0 0x7f752c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:09.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.354+0000 7f7537fff640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f752800c0e0 tx=0x7f752c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1945f5641af95d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528005ad0 con 0x7f752c0047c0 2023-10-21T17:19:09.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7528005c50 con 0x7f752c0047c0 2023-10-21T17:19:09.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f753f7fe640 1 -- 172.21.15.53:0/1160120196 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752802c7f0 con 0x7f752c0047c0 2023-10-21T17:19:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 -- 172.21.15.53:0/1160120196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f752c0047c0 msgr2=0x7f752c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 --2- 172.21.15.53:0/1160120196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f752c0047c0 0x7f752c004c40 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f752800c0e0 tx=0x7f752c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7544866640 1 -- 172.21.15.53:0/1160120196 reap_dead start 2023-10-21T17:19:09.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 -- 172.21.15.53:0/1160120196 shutdown_connections 2023-10-21T17:19:09.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 -- 172.21.15.53:0/1160120196 >> 172.21.15.53:0/1160120196 conn(0x7f75400cb0c0 msgr2=0x7f75400cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:09.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 -- 172.21.15.53:0/1160120196 shutdown_connections 2023-10-21T17:19:09.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.355+0000 7f7545067640 1 -- 172.21.15.53:0/1160120196 wait complete. 2023-10-21T17:19:09.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.357+0000 7f7545067640 1 Processor -- start 2023-10-21T17:19:09.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.357+0000 7f7545067640 1 -- start start 2023-10-21T17:19:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f752c0047c0 0x7f75400c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7540158ef0 0x7f75400c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400c7bc0 con 0x7f75400c60d0 2023-10-21T17:19:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7540158ad0 con 0x7f7540158ef0 2023-10-21T17:19:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.358+0000 7f7545067640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400c6560 con 0x7f752c0047c0 2023-10-21T17:19:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60824/0 (socket says 172.21.15.53:60824) 2023-10-21T17:19:09.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 -- 172.21.15.53:0/1927314451 learned_addr learned my addr 172.21.15.53:0/1927314451 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f753ffff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f752c0047c0 0x7f75400c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 -- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f752c0047c0 msgr2=0x7f75400c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f752c0047c0 0x7f75400c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 -- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7540158ef0 msgr2=0x7f75400c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7540158ef0 0x7f75400c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 -- 172.21.15.53:0/1927314451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75400c6840 con 0x7f75400c60d0 2023-10-21T17:19:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.359+0000 7f7537fff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f752802c3f0 tx=0x7f7528001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.360+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752803c040 con 0x7f75400c60d0 2023-10-21T17:19:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.360+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752802cc00 con 0x7f75400c60d0 2023-10-21T17:19:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.360+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528007070 con 0x7f75400c60d0 2023-10-21T17:19:09.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.360+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75400c6b20 con 0x7f75400c60d0 2023-10-21T17:19:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.360+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75400c4600 con 0x7f75400c60d0 2023-10-21T17:19:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.361+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7540158c10 con 0x7f75400c60d0 2023-10-21T17:19:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.362+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 94) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f7528032070 con 0x7f75400c60d0 2023-10-21T17:19:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.363+0000 7f753d7fa640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 0x7f752006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.363+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f75280aac40 con 0x7f75400c60d0 2023-10-21T17:19:09.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.364+0000 7f753ffff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 0x7f752006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.365+0000 7f753ffff640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 0x7f752006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7530009970 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.367+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7540158c10 con 0x7f75400c60d0 2023-10-21T17:19:09.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.600+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7540158c10 con 0x7f75400c60d0 2023-10-21T17:19:09.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.601+0000 7f753d7fa640 1 -- 172.21.15.53:0/1927314451 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f7540158c10 con 0x7f75400c60d0 2023-10-21T17:19:09.605 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:09.606 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:19:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:19:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 msgr2=0x7f752006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 0x7f752006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7530009970 tx=0x7f7530008040 comp rx=0 tx=0).stop 2023-10-21T17:19:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 msgr2=0x7f7540151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f752802c3f0 tx=0x7f7528001930 comp rx=0 tx=0).stop 2023-10-21T17:19:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 shutdown_connections 2023-10-21T17:19:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7540158ef0 0x7f75400c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f752006acc0 0x7f752006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75400c60d0 0x7f7540151740 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 --2- 172.21.15.53:0/1927314451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f752c0047c0 0x7f75400c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 >> 172.21.15.53:0/1927314451 conn(0x7f75400cb0c0 msgr2=0x7f75400af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 shutdown_connections 2023-10-21T17:19:09.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.604+0000 7f7545067640 1 -- 172.21.15.53:0/1927314451 wait complete. 2023-10-21T17:19:09.623 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:09.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.839+0000 7f9d05232640 1 Processor -- start 2023-10-21T17:19:09.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.839+0000 7f9d05232640 1 -- start start 2023-10-21T17:19:09.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9d05232640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d0015ab30 0x7f9d0015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9d05232640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d000b07d0 0x7f9d0006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9d05232640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0015f7d0 con 0x7f9d000adc70 2023-10-21T17:19:09.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9d05232640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0015f210 con 0x7f9d0015ab30 2023-10-21T17:19:09.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9d05232640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0015f4f0 con 0x7f9d000b07d0 2023-10-21T17:19:09.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9cfffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d0015ab30 0x7f9d0015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9cff7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9d000adc70 0x7f9d000b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43662/0 (socket says 172.21.15.53:43662) 2023-10-21T17:19:09.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.841+0000 7f9cff7fe640 1 -- 172.21.15.53:0/402407189 learned_addr learned my addr 172.21.15.53:0/402407189 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:09.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9d04a31640 1 --2- 172.21.15.53:0/402407189 >> v2:172.21.15.124:3300/0 conn(0x7f9d000b07d0 0x7f9d0006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9cfffff640 1 -- 172.21.15.53:0/402407189 >> v2:172.21.15.124:3300/0 conn(0x7f9d000b07d0 msgr2=0x7f9d0006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:09.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9cfffff640 1 --2- 172.21.15.53:0/402407189 >> v2:172.21.15.124:3300/0 conn(0x7f9d000b07d0 0x7f9d0006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9cfffff640 1 -- 172.21.15.53:0/402407189 >> v1:172.21.15.53:6789/0 conn(0x7f9d000adc70 legacy=0x7f9d000b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9cfffff640 1 -- 172.21.15.53:0/402407189 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0006c550 con 0x7f9d0015ab30 2023-10-21T17:19:09.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.842+0000 7f9cfeffd640 1 -- 172.21.15.53:0/402407189 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398311164 0 0) 0x7f9d0015f7d0 con 0x7f9d000adc70 2023-10-21T17:19:09.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.843+0000 7f9cfffff640 1 --2- 172.21.15.53:0/402407189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d0015e8a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9d000ca4a0 tx=0x7f9cf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27ec261d61ec88d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.843+0000 7f9cfeffd640 1 -- 172.21.15.53:0/402407189 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031830 con 0x7f9d0015ab30 2023-10-21T17:19:09.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.843+0000 7f9cfeffd640 1 -- 172.21.15.53:0/402407189 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf0031990 con 0x7f9d0015ab30 2023-10-21T17:19:09.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.843+0000 7f9cfeffd640 1 -- 172.21.15.53:0/402407189 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0031d20 con 0x7f9d0015ab30 2023-10-21T17:19:09.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 -- 172.21.15.53:0/402407189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 msgr2=0x7f9d0015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 --2- 172.21.15.53:0/402407189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d0015e8a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9d000ca4a0 tx=0x7f9cf002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 -- 172.21.15.53:0/402407189 shutdown_connections 2023-10-21T17:19:09.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 --2- 172.21.15.53:0/402407189 >> v2:172.21.15.124:3300/0 conn(0x7f9d000b07d0 0x7f9d0006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 --2- 172.21.15.53:0/402407189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d0015e8a0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 -- 172.21.15.53:0/402407189 >> 172.21.15.53:0/402407189 conn(0x7f9d000cb0c0 msgr2=0x7f9d000cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:09.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 -- 172.21.15.53:0/402407189 shutdown_connections 2023-10-21T17:19:09.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.844+0000 7f9d05232640 1 -- 172.21.15.53:0/402407189 wait complete. 2023-10-21T17:19:09.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.846+0000 7f9d05232640 1 Processor -- start 2023-10-21T17:19:09.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.846+0000 7f9d05232640 1 -- start start 2023-10-21T17:19:09.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d000adc70 0x7f9d0014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d000b07d0 0x7f9d001573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d001f0400 con 0x7f9d0015ab30 2023-10-21T17:19:09.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000c7750 con 0x7f9d000b07d0 2023-10-21T17:19:09.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d05232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0014b970 con 0x7f9d000adc70 2023-10-21T17:19:09.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d04a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d04a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60842/0 (socket says 172.21.15.53:60842) 2023-10-21T17:19:09.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.847+0000 7f9d04a31640 1 -- 172.21.15.53:0/433197332 learned_addr learned my addr 172.21.15.53:0/433197332 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:09.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9cfffff640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d000adc70 0x7f9d0014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9cff7fe640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d000b07d0 0x7f9d001573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 -- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d000adc70 msgr2=0x7f9d0014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d000adc70 0x7f9d0014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 -- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d000b07d0 msgr2=0x7f9d001573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:09.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d000b07d0 0x7f9d001573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:09.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 -- 172.21.15.53:0/433197332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009400 con 0x7f9d0015ab30 2023-10-21T17:19:09.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9d04a31640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007e40 tx=0x7f9cf800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8014620 con 0x7f9d0015ab30 2023-10-21T17:19:09.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.848+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf800ee90 con 0x7f9d0015ab30 2023-10-21T17:19:09.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.849+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80149e0 con 0x7f9d0015ab30 2023-10-21T17:19:09.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.849+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d000c79f0 con 0x7f9d0015ab30 2023-10-21T17:19:09.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.849+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d000c7fb0 con 0x7f9d0015ab30 2023-10-21T17:19:09.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.850+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0004e160 con 0x7f9d0015ab30 2023-10-21T17:19:09.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.851+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 94) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9cf8014b40 con 0x7f9d0015ab30 2023-10-21T17:19:09.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.852+0000 7f9cfcff9640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 0x7f9cec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:09.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.852+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f9cf808a220 con 0x7f9d0015ab30 2023-10-21T17:19:09.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.852+0000 7f9cfffff640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 0x7f9cec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:09.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.853+0000 7f9cfffff640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 0x7f9cec06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0036000 tx=0x7f9cf003c040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:09.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.855+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9cf8055040 con 0x7f9d0015ab30 2023-10-21T17:19:09.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:09.967+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 95) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9cf8055410 con 0x7f9d0015ab30 2023-10-21T17:19:10.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.063+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d000ada50 con 0x7f9d0015ab30 2023-10-21T17:19:10.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.066+0000 7f9cfcff9640 1 -- 172.21.15.53:0/433197332 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f9cf80872c0 con 0x7f9d0015ab30 2023-10-21T17:19:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:10.068 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":186,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:07.927726+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/930439389":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6833/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6832/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/57805179":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3870614335":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/1655250226":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/4079618600":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3580514343":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6833/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:6832/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1818943929":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1207144287":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/207583581":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1931157828":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/880607286":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/3805066261":"2023-10-22T17:19:00.108230+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.068+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 msgr2=0x7f9cec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.068+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 0x7f9cec06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0036000 tx=0x7f9cf003c040 comp rx=0 tx=0).stop 2023-10-21T17:19:10.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 msgr2=0x7f9d00157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8007e40 tx=0x7f9cf800e4b0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 shutdown_connections 2023-10-21T17:19:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f9cec06acc0 0x7f9cec06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d0015ab30 0x7f9d00157900 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d000b07d0 0x7f9d001573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 --2- 172.21.15.53:0/433197332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d000adc70 0x7f9d0014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 >> 172.21.15.53:0/433197332 conn(0x7f9d000cb0c0 msgr2=0x7f9d000c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 shutdown_connections 2023-10-21T17:19:10.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.069+0000 7f9d05232640 1 -- 172.21.15.53:0/433197332 wait complete. 2023-10-21T17:19:10.091 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:19:10.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.309+0000 7f7d8359e640 1 Processor -- start 2023-10-21T17:19:10.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.309+0000 7f7d8359e640 1 -- start start 2023-10-21T17:19:10.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.310+0000 7f7d8359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d8415ab30 0x7f7d8415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.310+0000 7f7d8359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8415f7d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.310+0000 7f7d8359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8415f210 con 0x7f7d840adc70 2023-10-21T17:19:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.310+0000 7f7d8359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8415f4f0 con 0x7f7d8415ab30 2023-10-21T17:19:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d8259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d8415ab30 0x7f7d8415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d82d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d840b07d0 0x7f7d8406cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43676/0 (socket says 172.21.15.53:43676) 2023-10-21T17:19:10.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d82d9d640 1 -- 172.21.15.53:0/610809483 learned_addr learned my addr 172.21.15.53:0/610809483 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:10.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d8259c640 1 -- 172.21.15.53:0/610809483 >> v1:172.21.15.124:6789/0 conn(0x7f7d840adc70 legacy=0x7f7d840b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:10.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d8259c640 1 -- 172.21.15.53:0/610809483 >> v1:172.21.15.53:6789/0 conn(0x7f7d840b07d0 legacy=0x7f7d8406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d8259c640 1 -- 172.21.15.53:0/610809483 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8406c550 con 0x7f7d8415ab30 2023-10-21T17:19:10.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.311+0000 7f7d81d9b640 1 -- 172.21.15.53:0/610809483 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109300546 0 0) 0x7f7d8415f7d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.312+0000 7f7d8259c640 1 --2- 172.21.15.53:0/610809483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d8415ab30 0x7f7d8415e8a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7d740097a0 tx=0x7f7d7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832a5feb99578ea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.312+0000 7f7d81d9b640 1 -- 172.21.15.53:0/610809483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74031830 con 0x7f7d8415ab30 2023-10-21T17:19:10.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.312+0000 7f7d81d9b640 1 -- 172.21.15.53:0/610809483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74031990 con 0x7f7d8415ab30 2023-10-21T17:19:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.312+0000 7f7d81d9b640 1 -- 172.21.15.53:0/610809483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74031d20 con 0x7f7d8415ab30 2023-10-21T17:19:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 -- 172.21.15.53:0/610809483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d8415ab30 msgr2=0x7f7d8415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 --2- 172.21.15.53:0/610809483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d8415ab30 0x7f7d8415e8a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f7d740097a0 tx=0x7f7d7402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 -- 172.21.15.53:0/610809483 shutdown_connections 2023-10-21T17:19:10.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 --2- 172.21.15.53:0/610809483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d8415ab30 0x7f7d8415e8a0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 -- 172.21.15.53:0/610809483 >> 172.21.15.53:0/610809483 conn(0x7f7d840cb0c0 msgr2=0x7f7d840cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 -- 172.21.15.53:0/610809483 shutdown_connections 2023-10-21T17:19:10.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.313+0000 7f7d8359e640 1 -- 172.21.15.53:0/610809483 wait complete. 2023-10-21T17:19:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.315+0000 7f7d8359e640 1 Processor -- start 2023-10-21T17:19:10.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.315+0000 7f7d8359e640 1 -- start start 2023-10-21T17:19:10.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d840adc70 0x7f7d8415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d8415ab30 0x7f7d840c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8415bd60 con 0x7f7d840b07d0 2023-10-21T17:19:10.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840ac0c0 con 0x7f7d840adc70 2023-10-21T17:19:10.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d8359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d840c7bc0 con 0x7f7d8415ab30 2023-10-21T17:19:10.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60862/0 (socket says 172.21.15.53:60862) 2023-10-21T17:19:10.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.316+0000 7f7d7bfff640 1 -- 172.21.15.53:0/3057952419 learned_addr learned my addr 172.21.15.53:0/3057952419 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:10.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d8259c640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d840adc70 0x7f7d8415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 -- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d8415ab30 msgr2=0x7f7d840c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d8415ab30 0x7f7d840c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 -- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d840adc70 msgr2=0x7f7d8415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d840adc70 0x7f7d8415b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 -- 172.21.15.53:0/3057952419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74009400 con 0x7f7d840b07d0 2023-10-21T17:19:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7bfff640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c430 tx=0x7f7d6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c011790 con 0x7f7d840b07d0 2023-10-21T17:19:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c0118f0 con 0x7f7d840b07d0 2023-10-21T17:19:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d840aba70 con 0x7f7d840b07d0 2023-10-21T17:19:10.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c011ca0 con 0x7f7d840b07d0 2023-10-21T17:19:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.317+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d84157080 con 0x7f7d840b07d0 2023-10-21T17:19:10.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.318+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d840ab8d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.319+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 95) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7d6c01c460 con 0x7f7d840b07d0 2023-10-21T17:19:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.320+0000 7f7d7affd640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 0x7f7d6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.320+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f7d6c089ac0 con 0x7f7d840b07d0 2023-10-21T17:19:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.321+0000 7f7d8259c640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 0x7f7d6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.323+0000 7f7d8259c640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 0x7f7d6406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d74036000 tx=0x7f7d7403c040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.325+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d840ab8d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.554+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d840ab8d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.555+0000 7f7d7affd640 1 -- 172.21.15.53:0/3057952419 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f7d840ab8d0 con 0x7f7d840b07d0 2023-10-21T17:19:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:10.556 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:19:10.557 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:19:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.557+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 msgr2=0x7f7d6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.557+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 0x7f7d6406d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7d74036000 tx=0x7f7d7403c040 comp rx=0 tx=0).stop 2023-10-21T17:19:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.557+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 msgr2=0x7f7d84156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.557+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c430 tx=0x7f7d6c00c8f0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 shutdown_connections 2023-10-21T17:19:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d8415ab30 0x7f7d840c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6406af00 0x7f7d6406d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d840b07d0 0x7f7d84156b40 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 --2- 172.21.15.53:0/3057952419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d840adc70 0x7f7d8415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 >> 172.21.15.53:0/3057952419 conn(0x7f7d840cb0c0 msgr2=0x7f7d840701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 shutdown_connections 2023-10-21T17:19:10.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.558+0000 7f7d8359e640 1 -- 172.21.15.53:0/3057952419 wait complete. 2023-10-21T17:19:10.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:10.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.805+0000 7f823eeca640 1 Processor -- start 2023-10-21T17:19:10.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.805+0000 7f823eeca640 1 -- start start 2023-10-21T17:19:10.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823eeca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82380b07d0 0x7f82381648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823eeca640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380cacf0 con 0x7f82381599d0 2023-10-21T17:19:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823eeca640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380ca3c0 con 0x7f82380cc210 2023-10-21T17:19:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823eeca640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82380b10f0 con 0x7f82380b07d0 2023-10-21T17:19:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823dec8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f82381599d0 0x7f82380ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43688/0 (socket says 172.21.15.53:43688) 2023-10-21T17:19:10.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823dec8640 1 -- 172.21.15.53:0/2624324290 learned_addr learned my addr 172.21.15.53:0/2624324290 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:10.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.807+0000 7f823e6c9640 1 --2- 172.21.15.53:0/2624324290 >> v2:172.21.15.53:3300/0 conn(0x7f82380b07d0 0x7f82381648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.808+0000 7f823e6c9640 1 -- 172.21.15.53:0/2624324290 >> v1:172.21.15.124:6789/0 conn(0x7f82380cc210 legacy=0x7f82380b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.808+0000 7f823e6c9640 1 -- 172.21.15.53:0/2624324290 >> v1:172.21.15.53:6789/0 conn(0x7f82381599d0 legacy=0x7f82380ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.808+0000 7f823e6c9640 1 -- 172.21.15.53:0/2624324290 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380ca6a0 con 0x7f82380b07d0 2023-10-21T17:19:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.808+0000 7f823cec6640 1 -- 172.21.15.53:0/2624324290 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000967902 0 0) 0x7f82380cacf0 con 0x7f82381599d0 2023-10-21T17:19:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.810+0000 7f823e6c9640 1 --2- 172.21.15.53:0/2624324290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380b07d0 0x7f82381648d0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8230009d90 tx=0x7f823002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f174dc6f9898908a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.810+0000 7f823cec6640 1 -- 172.21.15.53:0/2624324290 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823002ccf0 con 0x7f82380b07d0 2023-10-21T17:19:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.810+0000 7f823cec6640 1 -- 172.21.15.53:0/2624324290 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823002ce50 con 0x7f82380b07d0 2023-10-21T17:19:10.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.810+0000 7f823cec6640 1 -- 172.21.15.53:0/2624324290 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230034660 con 0x7f82380b07d0 2023-10-21T17:19:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 -- 172.21.15.53:0/2624324290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380b07d0 msgr2=0x7f82381648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 --2- 172.21.15.53:0/2624324290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380b07d0 0x7f82381648d0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8230009d90 tx=0x7f823002bf40 comp rx=0 tx=0).stop 2023-10-21T17:19:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 -- 172.21.15.53:0/2624324290 shutdown_connections 2023-10-21T17:19:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 --2- 172.21.15.53:0/2624324290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380b07d0 0x7f82381648d0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 -- 172.21.15.53:0/2624324290 >> 172.21.15.53:0/2624324290 conn(0x7f82380cb0c0 msgr2=0x7f82380cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:10.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 -- 172.21.15.53:0/2624324290 shutdown_connections 2023-10-21T17:19:10.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.811+0000 7f823eeca640 1 -- 172.21.15.53:0/2624324290 wait complete. 2023-10-21T17:19:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.813+0000 7f823eeca640 1 Processor -- start 2023-10-21T17:19:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.813+0000 7f823eeca640 1 -- start start 2023-10-21T17:19:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.814+0000 7f823eeca640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82380b07d0 0x7f82380c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.814+0000 7f823eeca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823eeca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82381599d0 0x7f82380acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823eeca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82380c77b0 con 0x7f82380cc210 2023-10-21T17:19:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823eeca640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381f5720 con 0x7f82380b07d0 2023-10-21T17:19:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823eeca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381f5a00 con 0x7f82381599d0 2023-10-21T17:19:10.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60870/0 (socket says 172.21.15.53:60870) 2023-10-21T17:19:10.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 -- 172.21.15.53:0/811841749 learned_addr learned my addr 172.21.15.53:0/811841749 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:10.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823dec8640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82380b07d0 0x7f82380c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 -- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82381599d0 msgr2=0x7f82380acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82381599d0 0x7f82380acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 -- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82380b07d0 msgr2=0x7f82380c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82380b07d0 0x7f82380c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:10.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.815+0000 7f823d6c7640 1 -- 172.21.15.53:0/811841749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230009a70 con 0x7f82380cc210 2023-10-21T17:19:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f823d6c7640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f822800c450 tx=0x7f822800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280052d0 con 0x7f82380cc210 2023-10-21T17:19:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228005430 con 0x7f82380cc210 2023-10-21T17:19:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228012870 con 0x7f82380cc210 2023-10-21T17:19:10.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82381f5ce0 con 0x7f82380cc210 2023-10-21T17:19:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.816+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82381f62a0 con 0x7f82380cc210 2023-10-21T17:19:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.817+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82381f21a0 con 0x7f82380cc210 2023-10-21T17:19:10.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.818+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 95) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f82280129d0 con 0x7f82380cc210 2023-10-21T17:19:10.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.819+0000 7f822e7fc640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 0x7f821c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:10.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.820+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f82280896e0 con 0x7f82380cc210 2023-10-21T17:19:10.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.820+0000 7f823dec8640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 0x7f821c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:10.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.821+0000 7f823dec8640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 0x7f821c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8234000f60 tx=0x7f8234009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:10.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:10.824+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82381f21a0 con 0x7f82380cc210 2023-10-21T17:19:11.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.056+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82381f21a0 con 0x7f82380cc210 2023-10-21T17:19:11.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.056+0000 7f822e7fc640 1 -- 172.21.15.53:0/811841749 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f82381f21a0 con 0x7f82380cc210 2023-10-21T17:19:11.058 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:11.058 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":86,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:18:45.383517+0000","modified":"2023-10-21T17:18:48.467714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37037},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37037":{"gid":37037,"name":"d","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/84861935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":84861935},{"type":"v1","addr":"172.21.15.124:6835","nonce":84861935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-21T17:19:11.058 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 86 2023-10-21T17:19:11.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 msgr2=0x7f821c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 0x7f821c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8234000f60 tx=0x7f8234009040 comp rx=0 tx=0).stop 2023-10-21T17:19:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 msgr2=0x7f82380c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f822800c450 tx=0x7f822800c910 comp rx=0 tx=0).stop 2023-10-21T17:19:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 shutdown_connections 2023-10-21T17:19:11.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82381599d0 0x7f82380acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f821c06af00 0x7f821c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82380cc210 0x7f82380c7270 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 --2- 172.21.15.53:0/811841749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82380b07d0 0x7f82380c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 >> 172.21.15.53:0/811841749 conn(0x7f82380cb0c0 msgr2=0x7f82380af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:11.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.059+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 shutdown_connections 2023-10-21T17:19:11.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.060+0000 7f823eeca640 1 -- 172.21.15.53:0/811841749 wait complete. 2023-10-21T17:19:11.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:11.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.309+0000 7f7a948bb640 1 Processor -- start 2023-10-21T17:19:11.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.309+0000 7f7a948bb640 1 -- start start 2023-10-21T17:19:11.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 0x7f7a900cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a901597d0 0x7f7a9006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a900cb190 0x7f7a900cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a900cbb80 con 0x7f7a901597d0 2023-10-21T17:19:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a900adfc0 con 0x7f7a90157000 2023-10-21T17:19:11.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.310+0000 7f7a948bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900ccce0 con 0x7f7a900cb190 2023-10-21T17:19:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a901597d0 0x7f7a9006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a901597d0 0x7f7a9006ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60880/0 (socket says 172.21.15.53:60880) 2023-10-21T17:19:11.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 -- 172.21.15.53:0/3793790130 learned_addr learned my addr 172.21.15.53:0/3793790130 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8ed9d640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a900cb190 0x7f7a900cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/3793790130 >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 0x7f7a900cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 -- 172.21.15.53:0/3793790130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a900cb190 msgr2=0x7f7a900cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a900cb190 0x7f7a900cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 -- 172.21.15.53:0/3793790130 >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 msgr2=0x7f7a900cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 --2- 172.21.15.53:0/3793790130 >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 0x7f7a900cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8dd9b640 1 -- 172.21.15.53:0/3793790130 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9006d380 con 0x7f7a901597d0 2023-10-21T17:19:11.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.311+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/3793790130 >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 0x7f7a900cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:11.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.312+0000 7f7a8dd9b640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a901597d0 0x7f7a9006ce40 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7a90071700 tx=0x7f7a7802deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791076c06b889992 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.313+0000 7f7a8d59a640 1 -- 172.21.15.53:0/3793790130 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7802ec60 con 0x7f7a901597d0 2023-10-21T17:19:11.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.313+0000 7f7a8d59a640 1 -- 172.21.15.53:0/3793790130 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78032070 con 0x7f7a901597d0 2023-10-21T17:19:11.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.313+0000 7f7a8d59a640 1 -- 172.21.15.53:0/3793790130 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78038400 con 0x7f7a901597d0 2023-10-21T17:19:11.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.313+0000 7f7a948bb640 1 -- 172.21.15.53:0/3793790130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a901597d0 msgr2=0x7f7a9006ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.313+0000 7f7a948bb640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a901597d0 0x7f7a9006ce40 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7a90071700 tx=0x7f7a7802deb0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 -- 172.21.15.53:0/3793790130 shutdown_connections 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a900cb190 0x7f7a900cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 --2- 172.21.15.53:0/3793790130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a901597d0 0x7f7a9006ce40 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 --2- 172.21.15.53:0/3793790130 >> v2:172.21.15.124:3300/0 conn(0x7f7a90157000 0x7f7a900cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 -- 172.21.15.53:0/3793790130 >> 172.21.15.53:0/3793790130 conn(0x7f7a9006a380 msgr2=0x7f7a900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:11.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 -- 172.21.15.53:0/3793790130 shutdown_connections 2023-10-21T17:19:11.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.314+0000 7f7a948bb640 1 -- 172.21.15.53:0/3793790130 wait complete. 2023-10-21T17:19:11.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.315+0000 7f7a948bb640 1 Processor -- start 2023-10-21T17:19:11.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.316+0000 7f7a948bb640 1 -- start start 2023-10-21T17:19:11.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.316+0000 7f7a948bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 0x7f7a90149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.316+0000 7f7a948bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 0x7f7a9014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a948bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a948bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90142d50 con 0x7f7a90150070 2023-10-21T17:19:11.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a948bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90151e00 con 0x7f7a900cb190 2023-10-21T17:19:11.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a948bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90151970 con 0x7f7a901597d0 2023-10-21T17:19:11.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60882/0 (socket says 172.21.15.53:60882) 2023-10-21T17:19:11.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 -- 172.21.15.53:0/2544568754 learned_addr learned my addr 172.21.15.53:0/2544568754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:11.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8dd9b640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 0x7f7a9014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 0x7f7a90149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 -- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 msgr2=0x7f7a9014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 0x7f7a9014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 -- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 msgr2=0x7f7a90149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 0x7f7a90149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8ed9d640 1 -- 172.21.15.53:0/2544568754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7800b760 con 0x7f7a90150070 2023-10-21T17:19:11.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8dd9b640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 0x7f7a9014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:11.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.317+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 0x7f7a90149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:11.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a8ed9d640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c4f0 tx=0x7f7a8800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88004220 con 0x7f7a90150070 2023-10-21T17:19:11.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88004380 con 0x7f7a90150070 2023-10-21T17:19:11.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003810 con 0x7f7a90150070 2023-10-21T17:19:11.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a90150d60 con 0x7f7a90150070 2023-10-21T17:19:11.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.318+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a90151320 con 0x7f7a90150070 2023-10-21T17:19:11.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.319+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90067490 con 0x7f7a90150070 2023-10-21T17:19:11.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.320+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 95) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7a88003970 con 0x7f7a90150070 2023-10-21T17:19:11.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.321+0000 7f7a86ffd640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 0x7f7a7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.322+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f7a88089930 con 0x7f7a90150070 2023-10-21T17:19:11.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.322+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 0x7f7a7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.323+0000 7f7a8e59c640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 0x7f7a7006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007f40 tx=0x7f7a80007490 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.326+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a88054370 con 0x7f7a90150070 2023-10-21T17:19:11.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.523+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a90151ab0 con 0x7f7a90150070 2023-10-21T17:19:11.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.526+0000 7f7a86ffd640 1 -- 172.21.15.53:0/2544568754 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17906 (secure 0 0 0) 0x7f7a88057990 con 0x7f7a90150070 2023-10-21T17:19:11.528 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:11.528 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":186,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:07.927726+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:18:43.336266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-21T17:18:44.404625+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/930439389":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6833/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6832/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/57805179":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3870614335":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/1655250226":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/4079618600":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3580514343":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6833/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:6832/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1818943929":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1207144287":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/207583581":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1931157828":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/880607286":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/3805066261":"2023-10-22T17:19:00.108230+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:11.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 msgr2=0x7f7a7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 0x7f7a7006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7a80007f40 tx=0x7f7a80007490 comp rx=0 tx=0).stop 2023-10-21T17:19:11.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 msgr2=0x7f7a901521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c4f0 tx=0x7f7a8800c9b0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 shutdown_connections 2023-10-21T17:19:11.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a7006af00 0x7f7a7006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a90150070 0x7f7a901521c0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a901597d0 0x7f7a9014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 --2- 172.21.15.53:0/2544568754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a900cb190 0x7f7a90149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 >> 172.21.15.53:0/2544568754 conn(0x7f7a9006a380 msgr2=0x7f7a900ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 shutdown_connections 2023-10-21T17:19:11.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.530+0000 7f7a948bb640 1 -- 172.21.15.53:0/2544568754 wait complete. 2023-10-21T17:19:11.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:19:11.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.770+0000 7f839d227640 1 Processor -- start 2023-10-21T17:19:11.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.771+0000 7f839d227640 1 -- start start 2023-10-21T17:19:11.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.772+0000 7f839d227640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 0x7f83980d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.772+0000 7f839d227640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83980d00e0 0x7f83980b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.772+0000 7f839d227640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83980d2870 con 0x7f83980d0b90 2023-10-21T17:19:11.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.772+0000 7f839d227640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980d1150 con 0x7f83980d00e0 2023-10-21T17:19:11.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.772+0000 7f839d227640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980d1430 con 0x7f839815ccb0 2023-10-21T17:19:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f839ca26640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83980d00e0 0x7f83980b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f839ca26640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83980d00e0 0x7f83980b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60886/0 (socket says 172.21.15.53:60886) 2023-10-21T17:19:11.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f839ca26640 1 -- 172.21.15.53:0/4026508382 learned_addr learned my addr 172.21.15.53:0/4026508382 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:11.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f8397fff640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 0x7f83980d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f839ca26640 1 -- 172.21.15.53:0/4026508382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 msgr2=0x7f83980d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:11.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.773+0000 7f839ca26640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 0x7f83980d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.774+0000 7f839ca26640 1 -- 172.21.15.53:0/4026508382 >> v1:172.21.15.124:6789/0 conn(0x7f83980d0b90 legacy=0x7f83980cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.774+0000 7f839ca26640 1 -- 172.21.15.53:0/4026508382 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980d1710 con 0x7f83980d00e0 2023-10-21T17:19:11.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.774+0000 7f8397fff640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 0x7f83980d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:11.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f839ca26640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f83980b6060 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dceb2a9e36881d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f83977fe640 1 -- 172.21.15.53:0/4026508382 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839002cb00 con 0x7f83980d00e0 2023-10-21T17:19:11.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f83977fe640 1 -- 172.21.15.53:0/4026508382 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839002cc60 con 0x7f83980d00e0 2023-10-21T17:19:11.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f83977fe640 1 -- 172.21.15.53:0/4026508382 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83900343f0 con 0x7f83980d00e0 2023-10-21T17:19:11.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f839d227640 1 -- 172.21.15.53:0/4026508382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 msgr2=0x7f83980b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.775+0000 7f839d227640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f83980b6060 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8390009e50 tx=0x7f839002bf60 comp rx=0 tx=0).stop 2023-10-21T17:19:11.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 -- 172.21.15.53:0/4026508382 shutdown_connections 2023-10-21T17:19:11.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f83980b6060 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 --2- 172.21.15.53:0/4026508382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f839815ccb0 0x7f83980d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 -- 172.21.15.53:0/4026508382 >> 172.21.15.53:0/4026508382 conn(0x7f8398071660 msgr2=0x7f8398071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:11.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 -- 172.21.15.53:0/4026508382 shutdown_connections 2023-10-21T17:19:11.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.776+0000 7f839d227640 1 -- 172.21.15.53:0/4026508382 wait complete. 2023-10-21T17:19:11.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.778+0000 7f839d227640 1 Processor -- start 2023-10-21T17:19:11.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.778+0000 7f839d227640 1 -- start start 2023-10-21T17:19:11.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83980d0b90 0x7f83980cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f839815ccb0 0x7f839814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398160020 con 0x7f83980d00e0 2023-10-21T17:19:11.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839815f780 con 0x7f839815ccb0 2023-10-21T17:19:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839d227640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839815fa60 con 0x7f83980d0b90 2023-10-21T17:19:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f8397fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f8397fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60898/0 (socket says 172.21.15.53:60898) 2023-10-21T17:19:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f8397fff640 1 -- 172.21.15.53:0/552481733 learned_addr learned my addr 172.21.15.53:0/552481733 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:11.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f838ffff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83980d0b90 0x7f83980cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.779+0000 7f839ca26640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f839815ccb0 0x7f839814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 -- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83980d0b90 msgr2=0x7f83980cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83980d0b90 0x7f83980cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 -- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f839815ccb0 msgr2=0x7f839814ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:11.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f839815ccb0 0x7f839814ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:11.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 -- 172.21.15.53:0/552481733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390009b30 con 0x7f83980d00e0 2023-10-21T17:19:11.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f8397fff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f838800d7d0 tx=0x7f838800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388004030 con 0x7f83980d00e0 2023-10-21T17:19:11.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388004190 con 0x7f83980d00e0 2023-10-21T17:19:11.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388004520 con 0x7f83980d00e0 2023-10-21T17:19:11.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839815fd40 con 0x7f83980d00e0 2023-10-21T17:19:11.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.780+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839814b280 con 0x7f83980d00e0 2023-10-21T17:19:11.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.782+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8398074150 con 0x7f83980d00e0 2023-10-21T17:19:11.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.783+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 95) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f83880096a0 con 0x7f83980d00e0 2023-10-21T17:19:11.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.784+0000 7f83957fa640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 0x7f837806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:11.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.784+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f8388089d50 con 0x7f83980d00e0 2023-10-21T17:19:11.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.784+0000 7f838ffff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 0x7f837806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:11.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.785+0000 7f838ffff640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 0x7f837806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83980c6010 tx=0x7f8380009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:11.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.788+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83880548a0 con 0x7f83980d00e0 2023-10-21T17:19:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:11.989+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8388054c70 con 0x7f83980d00e0 2023-10-21T17:19:12.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.024+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83980b5ae0 con 0x7f83980d00e0 2023-10-21T17:19:12.026 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:19:12.025+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:19:12.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.029+0000 7f83957fa640 1 -- 172.21.15.53:0/552481733 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7f83880707b0 con 0x7f83980d00e0 2023-10-21T17:19:12.030 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:19:12.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 msgr2=0x7f837806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 0x7f837806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83980c6010 tx=0x7f8380009040 comp rx=0 tx=0).stop 2023-10-21T17:19:12.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 msgr2=0x7f839815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f838800d7d0 tx=0x7f838800dc90 comp rx=0 tx=0).stop 2023-10-21T17:19:12.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 shutdown_connections 2023-10-21T17:19:12.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f839815ccb0 0x7f839814ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83980d0b90 0x7f83980cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f837806af00 0x7f837806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 --2- 172.21.15.53:0/552481733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83980d00e0 0x7f839815d090 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.032+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 >> 172.21.15.53:0/552481733 conn(0x7f8398071660 msgr2=0x7f839815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:12.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.033+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 shutdown_connections 2023-10-21T17:19:12.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.033+0000 7f839d227640 1 -- 172.21.15.53:0/552481733 wait complete. 2023-10-21T17:19:12.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:19:12.063 INFO:tasks.ceph.mds.d.smithi124.stderr: -3869> 2023-10-21T17:18:50.331+0000 7f339b851640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:19:12.148 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:19:12.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.284+0000 7f8bd9f63640 1 Processor -- start 2023-10-21T17:19:12.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.284+0000 7f8bd9f63640 1 -- start start 2023-10-21T17:19:12.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.285+0000 7f8bd9f63640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8bd40cf2b0 0x7f8bd40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.285+0000 7f8bd9f63640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8bd40ce510 0x7f8bd415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.285+0000 7f8bd9f63640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd40cd520 con 0x7f8bd40cf690 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.285+0000 7f8bd9f63640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40d1910 con 0x7f8bd40ce510 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.285+0000 7f8bd9f63640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40ce080 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8bd40cf690 0x7f8bd40d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43694/0 (socket says 172.21.15.53:43694) 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd9762640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8bd40ce510 0x7f8bd415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd3fff640 1 -- 172.21.15.53:0/3764355081 learned_addr learned my addr 172.21.15.53:0/3764355081 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd8f61640 1 --2- 172.21.15.53:0/3764355081 >> v2:172.21.15.124:3300/0 conn(0x7f8bd40cf2b0 0x7f8bd40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd37fe640 1 -- 172.21.15.53:0/3764355081 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685742660 0 0) 0x7f8bd40cd520 con 0x7f8bd40cf690 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd37fe640 1 -- 172.21.15.53:0/3764355081 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bbc003520 con 0x7f8bd40cf690 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd9762640 1 -- 172.21.15.53:0/3764355081 >> v2:172.21.15.124:3300/0 conn(0x7f8bd40cf2b0 msgr2=0x7f8bd40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd9762640 1 --2- 172.21.15.53:0/3764355081 >> v2:172.21.15.124:3300/0 conn(0x7f8bd40cf2b0 0x7f8bd40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd9762640 1 -- 172.21.15.53:0/3764355081 >> v1:172.21.15.53:6789/0 conn(0x7f8bd40cf690 legacy=0x7f8bd40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.286+0000 7f8bd9762640 1 -- 172.21.15.53:0/3764355081 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40cd750 con 0x7f8bd40ce510 2023-10-21T17:19:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.288+0000 7f8bd9762640 1 --2- 172.21.15.53:0/3764355081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40ce510 0x7f8bd415c820 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40b54b0 tx=0x7f8bcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2d340d84b75967 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.288+0000 7f8bd37fe640 1 -- 172.21.15.53:0/3764355081 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02e070 con 0x7f8bd40ce510 2023-10-21T17:19:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.288+0000 7f8bd37fe640 1 -- 172.21.15.53:0/3764355081 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0315a0 con 0x7f8bd40ce510 2023-10-21T17:19:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.288+0000 7f8bd37fe640 1 -- 172.21.15.53:0/3764355081 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc031930 con 0x7f8bd40ce510 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3764355081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40ce510 msgr2=0x7f8bd415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3764355081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40ce510 0x7f8bd415c820 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40b54b0 tx=0x7f8bcc02b840 comp rx=0 tx=0).stop 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3764355081 shutdown_connections 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3764355081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40ce510 0x7f8bd415c820 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3764355081 >> v2:172.21.15.124:3300/0 conn(0x7f8bd40cf2b0 0x7f8bd40d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3764355081 >> 172.21.15.53:0/3764355081 conn(0x7f8bd409bc40 msgr2=0x7f8bd40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:12.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3764355081 shutdown_connections 2023-10-21T17:19:12.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.289+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3764355081 wait complete. 2023-10-21T17:19:12.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.290+0000 7f8bd9f63640 1 Processor -- start 2023-10-21T17:19:12.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.291+0000 7f8bd9f63640 1 -- start start 2023-10-21T17:19:12.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bd40ce510 0x7f8bd40b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bd40cf690 0x7f8bd4151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd409caf0 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd409cd00 con 0x7f8bd40cf690 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd9f63640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd409cfe0 con 0x7f8bd40ce510 2023-10-21T17:19:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60926/0 (socket says 172.21.15.53:60926) 2023-10-21T17:19:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 -- 172.21.15.53:0/3845440617 learned_addr learned my addr 172.21.15.53:0/3845440617 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 -- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bd40ce510 msgr2=0x7f8bd40b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bd40ce510 0x7f8bd40b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 -- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bd40cf690 msgr2=0x7f8bd4151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bd40cf690 0x7f8bd4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.292+0000 7f8bd3fff640 1 -- 172.21.15.53:0/3845440617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009400 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd3fff640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c410 tx=0x7f8bc800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8005ce0 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc8005e40 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80072f0 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd409d2c0 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.293+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd409d880 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.294+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4074150 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.296+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8bc8007510 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.297+0000 7f8bd17fa640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 0x7f8bc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.297+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f8bc8089370 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.297+0000 7f8bd8f61640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 0x7f8bc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.299+0000 7f8bd8f61640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 0x7f8bc006d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40070b0 tx=0x7f8bc4007040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.301+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bc8053e70 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:12.530+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8bd40d0070 con 0x7f8bd40cf2b0 2023-10-21T17:19:12.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:19:12.642 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:19:13.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.032+0000 7f8bd17fa640 1 -- 172.21.15.53:0/3845440617 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8bc8057490 con 0x7f8bd40cf2b0 2023-10-21T17:19:13.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 msgr2=0x7f8bc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 0x7f8bc006d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40070b0 tx=0x7f8bc4007040 comp rx=0 tx=0).stop 2023-10-21T17:19:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 msgr2=0x7f8bd409c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c410 tx=0x7f8bc800c8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 shutdown_connections 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bd40cf690 0x7f8bd4151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f8bc006af90 0x7f8bc006d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bd40cf2b0 0x7f8bd409c5b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 --2- 172.21.15.53:0/3845440617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bd40ce510 0x7f8bd40b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.036+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 >> 172.21.15.53:0/3845440617 conn(0x7f8bd409bc40 msgr2=0x7f8bd415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:13.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.037+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 shutdown_connections 2023-10-21T17:19:13.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.037+0000 7f8bd9f63640 1 -- 172.21.15.53:0/3845440617 wait complete. 2023-10-21T17:19:13.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:19:13.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.283+0000 7f94e6de9640 1 Processor -- start 2023-10-21T17:19:13.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.284+0000 7f94e6de9640 1 -- start start 2023-10-21T17:19:13.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.285+0000 7f94e6de9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94e00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.285+0000 7f94e6de9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e0159db0 con 0x7f94e01599d0 2023-10-21T17:19:13.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.285+0000 7f94e6de9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e0071fe0 con 0x7f94e00cb5e0 2023-10-21T17:19:13.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.285+0000 7f94e6de9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00b44c0 con 0x7f94e00b3c50 2023-10-21T17:19:13.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.286+0000 7f94e5de7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f94e01599d0 0x7f94e00caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43706/0 (socket says 172.21.15.53:43706) 2023-10-21T17:19:13.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.286+0000 7f94e5de7640 1 -- 172.21.15.53:0/2523866231 learned_addr learned my addr 172.21.15.53:0/2523866231 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:13.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.286+0000 7f94e65e8640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94e00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.286+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826324543 0 0) 0x7f94dc001110 con 0x7f94e01599d0 2023-10-21T17:19:13.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.286+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94cc003520 con 0x7f94e01599d0 2023-10-21T17:19:13.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789676949 0 0) 0x7f94cc003520 con 0x7f94e01599d0 2023-10-21T17:19:13.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 msgr2=0x7f94e00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:13.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94e00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 >> v1:172.21.15.124:6789/0 conn(0x7f94e00cb5e0 legacy=0x7f94e00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00713d0 con 0x7f94e01599d0 2023-10-21T17:19:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f94dc001800 con 0x7f94e01599d0 2023-10-21T17:19:13.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 >> v1:172.21.15.53:6789/0 conn(0x7f94e01599d0 legacy=0x7f94e00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94cc0039a0 0x7f94cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94cc004d00 0x7f94cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00b44c0 con 0x7f94cc004d00 2023-10-21T17:19:13.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc00ba30 con 0x7f94cc0039a0 2023-10-21T17:19:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc00de00 con 0x7f94e00b3c50 2023-10-21T17:19:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e65e8640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94cc004d00 0x7f94cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e5de7640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94cc0039a0 0x7f94cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.287+0000 7f94e55e6640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e65e8640 1 -- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 msgr2=0x7f94cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e65e8640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e65e8640 1 -- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94cc0039a0 msgr2=0x7f94cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e65e8640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94cc0039a0 0x7f94cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e65e8640 1 -- 172.21.15.53:0/2523866231 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc003520 con 0x7f94cc004d00 2023-10-21T17:19:13.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.288+0000 7f94e55e6640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94e00b3c50 0x7f94cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.289+0000 7f94e65e8640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94cc004d00 0x7f94cc005180 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009f90 tx=0x7f94cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b90e0f77b0ac0d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.289+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d802cbd0 con 0x7f94cc004d00 2023-10-21T17:19:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.289+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d800f040 con 0x7f94cc004d00 2023-10-21T17:19:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.289+0000 7f94e4de5640 1 -- 172.21.15.53:0/2523866231 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8034440 con 0x7f94cc004d00 2023-10-21T17:19:13.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 -- 172.21.15.53:0/2523866231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94cc004d00 msgr2=0x7f94cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 --2- 172.21.15.53:0/2523866231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94cc004d00 0x7f94cc005180 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009f90 tx=0x7f94cc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e65e8640 1 -- 172.21.15.53:0/2523866231 reap_dead start 2023-10-21T17:19:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 -- 172.21.15.53:0/2523866231 shutdown_connections 2023-10-21T17:19:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 -- 172.21.15.53:0/2523866231 >> 172.21.15.53:0/2523866231 conn(0x7f94e006c050 msgr2=0x7f94e006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:13.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 -- 172.21.15.53:0/2523866231 shutdown_connections 2023-10-21T17:19:13.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.290+0000 7f94e6de9640 1 -- 172.21.15.53:0/2523866231 wait complete. 2023-10-21T17:19:13.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.292+0000 7f94e6de9640 1 Processor -- start 2023-10-21T17:19:13.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.292+0000 7f94e6de9640 1 -- start start 2023-10-21T17:19:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 0x7f94e01f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 0x7f94e014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01f5c40 con 0x7f94e00c70c0 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00c7910 con 0x7f94e00c4ff0 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e6de9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e00c7bf0 con 0x7f94cc004d00 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e65e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e65e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:60934/0 (socket says 172.21.15.53:60934) 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e65e8640 1 -- 172.21.15.53:0/4057524359 learned_addr learned my addr 172.21.15.53:0/4057524359 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:13.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e55e6640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 0x7f94e014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.293+0000 7f94e5de7640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 0x7f94e01f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 -- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 msgr2=0x7f94e01f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 0x7f94e01f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 -- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 msgr2=0x7f94e014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:13.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 0x7f94e014db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 -- 172.21.15.53:0/4057524359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e01f5e90 con 0x7f94e00c70c0 2023-10-21T17:19:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e55e6640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 0x7f94e014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e5de7640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 0x7f94e01f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e65e8640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f94d8037040 tx=0x7f94d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:13.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8003b90 con 0x7f94e00c70c0 2023-10-21T17:19:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8003d10 con 0x7f94e00c70c0 2023-10-21T17:19:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8034ac0 con 0x7f94e00c70c0 2023-10-21T17:19:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.294+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e01f6170 con 0x7f94e00c70c0 2023-10-21T17:19:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.295+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e00c7540 con 0x7f94e00c70c0 2023-10-21T17:19:13.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.296+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e01f6360 con 0x7f94e00c70c0 2023-10-21T17:19:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.297+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94d8034c20 con 0x7f94e00c70c0 2023-10-21T17:19:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.298+0000 7f94d67fc640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.298+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f94d80ab140 con 0x7f94e00c70c0 2023-10-21T17:19:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.298+0000 7f94e5de7640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:13.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.300+0000 7f94e5de7640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 0x7f94c406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc000ed0 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:13.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.303+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94d8075cd0 con 0x7f94e00c70c0 2023-10-21T17:19:13.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:13.500+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94e0074150 con 0x7f94e00c70c0 2023-10-21T17:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.043+0000 7f94d67fc640 1 -- 172.21.15.53:0/4057524359 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v188) v1 ==== 181+0+0 (secure 0 0 0) 0x7f94d80792f0 con 0x7f94e00c70c0 2023-10-21T17:19:14.044 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:19:14.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.047+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 msgr2=0x7f94c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.047+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 0x7f94c406d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f94dc000c00 tx=0x7f94dc000ed0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.047+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 msgr2=0x7f94e01509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.047+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f94d8037040 tx=0x7f94d8009750 comp rx=0 tx=0).stop 2023-10-21T17:19:14.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 shutdown_connections 2023-10-21T17:19:14.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f94c406afc0 0x7f94c406d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94e00c70c0 0x7f94e01509e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94e00c4ff0 0x7f94e014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 --2- 172.21.15.53:0/4057524359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94cc004d00 0x7f94e01f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 >> 172.21.15.53:0/4057524359 conn(0x7f94e006c050 msgr2=0x7f94e00c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 shutdown_connections 2023-10-21T17:19:14.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.048+0000 7f94e6de9640 1 -- 172.21.15.53:0/4057524359 wait complete. 2023-10-21T17:19:14.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:19:14.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.301+0000 7f4a6d421640 1 Processor -- start 2023-10-21T17:19:14.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.302+0000 7f4a6d421640 1 -- start start 2023-10-21T17:19:14.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.303+0000 7f4a6d421640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a68160660 0x7f4a68070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.303+0000 7f4a6d421640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a680c9910 0x7f4a680caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.303+0000 7f4a6d421640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6806e150 con 0x7f4a680cb450 2023-10-21T17:19:14.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.303+0000 7f4a6d421640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a680cb8c0 con 0x7f4a680c9910 2023-10-21T17:19:14.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.303+0000 7f4a6d421640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6806de70 con 0x7f4a68160660 2023-10-21T17:19:14.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.304+0000 7f4a5ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a680c9910 0x7f4a680caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.304+0000 7f4a66ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a68160660 0x7f4a68070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.304+0000 7f4a5ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4a680c9910 0x7f4a680caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:39504/0 (socket says 172.21.15.53:39504) 2023-10-21T17:19:14.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.304+0000 7f4a5ffff640 1 -- 172.21.15.53:0/979544326 learned_addr learned my addr 172.21.15.53:0/979544326 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:14.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.305+0000 7f4a5ffff640 1 -- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a68160660 msgr2=0x7f4a68070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:14.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.305+0000 7f4a5ffff640 1 --2- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a68160660 0x7f4a68070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.305+0000 7f4a5ffff640 1 -- 172.21.15.53:0/979544326 >> v1:172.21.15.124:6789/0 conn(0x7f4a680cb450 legacy=0x7f4a680b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.305+0000 7f4a5ffff640 1 -- 172.21.15.53:0/979544326 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a680c8e30 con 0x7f4a680c9910 2023-10-21T17:19:14.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.306+0000 7f4a5ffff640 1 --2- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680c9910 0x7f4a680caf10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009870 tx=0x7f4a5002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0a2b5f3b23cb10 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:14.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.307+0000 7f4a667fc640 1 -- 172.21.15.53:0/979544326 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50030840 con 0x7f4a680c9910 2023-10-21T17:19:14.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.307+0000 7f4a667fc640 1 -- 172.21.15.53:0/979544326 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500309a0 con 0x7f4a680c9910 2023-10-21T17:19:14.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.307+0000 7f4a667fc640 1 -- 172.21.15.53:0/979544326 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50030d30 con 0x7f4a680c9910 2023-10-21T17:19:14.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 -- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680c9910 msgr2=0x7f4a680caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 --2- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680c9910 0x7f4a680caf10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009870 tx=0x7f4a5002b980 comp rx=0 tx=0).stop 2023-10-21T17:19:14.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 -- 172.21.15.53:0/979544326 shutdown_connections 2023-10-21T17:19:14.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 --2- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680c9910 0x7f4a680caf10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 --2- 172.21.15.53:0/979544326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a68160660 0x7f4a68070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 -- 172.21.15.53:0/979544326 >> 172.21.15.53:0/979544326 conn(0x7f4a6806a5d0 msgr2=0x7f4a680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:14.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 -- 172.21.15.53:0/979544326 shutdown_connections 2023-10-21T17:19:14.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.308+0000 7f4a6d421640 1 -- 172.21.15.53:0/979544326 wait complete. 2023-10-21T17:19:14.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.310+0000 7f4a6d421640 1 Processor -- start 2023-10-21T17:19:14.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.310+0000 7f4a6d421640 1 -- start start 2023-10-21T17:19:14.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a680c9910 0x7f4a68153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 0x7f4a68156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6815e450 con 0x7f4a68160660 2023-10-21T17:19:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6815e170 con 0x7f4a680cb450 2023-10-21T17:19:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a6d421640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680cc770 con 0x7f4a680c9910 2023-10-21T17:19:14.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.311+0000 7f4a677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59416/0 (socket says 172.21.15.53:59416) 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 -- 172.21.15.53:0/3259697067 learned_addr learned my addr 172.21.15.53:0/3259697067 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a66ffd640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a680c9910 0x7f4a68153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a5ffff640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 0x7f4a68156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 -- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a680c9910 msgr2=0x7f4a68153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a680c9910 0x7f4a68153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 -- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 msgr2=0x7f4a68156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:14.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 0x7f4a68156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:14.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a677fe640 1 -- 172.21.15.53:0/3259697067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500094d0 con 0x7f4a68160660 2023-10-21T17:19:14.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.312+0000 7f4a5ffff640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 0x7f4a68156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:14.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a677fe640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c7a0 tx=0x7f4a6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:14.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60003b70 con 0x7f4a68160660 2023-10-21T17:19:14.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60003cf0 con 0x7f4a68160660 2023-10-21T17:19:14.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60003ea0 con 0x7f4a68160660 2023-10-21T17:19:14.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a680cca50 con 0x7f4a68160660 2023-10-21T17:19:14.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.313+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a68154400 con 0x7f4a68160660 2023-10-21T17:19:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.314+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a680ccf70 con 0x7f4a68160660 2023-10-21T17:19:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.315+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a6000aa90 con 0x7f4a68160660 2023-10-21T17:19:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.316+0000 7f4a5f7fe640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 0x7f4a4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.316+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f4a60089830 con 0x7f4a68160660 2023-10-21T17:19:14.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.317+0000 7f4a66ffd640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 0x7f4a4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.318+0000 7f4a66ffd640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 0x7f4a4806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009aa0 tx=0x7f4a58009190 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.321+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a680ccf70 con 0x7f4a68160660 2023-10-21T17:19:14.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:14.515+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a680ccf70 con 0x7f4a68160660 2023-10-21T17:19:15.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.046+0000 7f4a5f7fe640 1 -- 172.21.15.53:0/3259697067 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v189) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4a680ccf70 con 0x7f4a68160660 2023-10-21T17:19:15.048 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:19:15.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.050+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 msgr2=0x7f4a4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.050+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 0x7f4a4806d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009aa0 tx=0x7f4a58009190 comp rx=0 tx=0).stop 2023-10-21T17:19:15.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.050+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 msgr2=0x7f4a68157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.050+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c7a0 tx=0x7f4a6000cc60 comp rx=0 tx=0).stop 2023-10-21T17:19:15.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 shutdown_connections 2023-10-21T17:19:15.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4a4806af90 0x7f4a4806d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a68160660 0x7f4a68157530 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a680cb450 0x7f4a68156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 --2- 172.21.15.53:0/3259697067 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a680c9910 0x7f4a68153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 >> 172.21.15.53:0/3259697067 conn(0x7f4a6806a5d0 msgr2=0x7f4a6815ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:15.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 shutdown_connections 2023-10-21T17:19:15.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.052+0000 7f4a6d421640 1 -- 172.21.15.53:0/3259697067 wait complete. 2023-10-21T17:19:15.080 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:19:15.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.287+0000 7f7746906640 1 Processor -- start 2023-10-21T17:19:15.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.287+0000 7f7746906640 1 -- start start 2023-10-21T17:19:15.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.289+0000 7f7746906640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f7740071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.289+0000 7f7746906640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7740160d10 con 0x7f774006a3a0 2023-10-21T17:19:15.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.289+0000 7f7746906640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7740072070 con 0x7f774006ec00 2023-10-21T17:19:15.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.289+0000 7f7746906640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7740071580 con 0x7f7740069fc0 2023-10-21T17:19:15.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7746105640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f774006a3a0 0x7f7740160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35346/0 (socket says 172.21.15.53:35346) 2023-10-21T17:19:15.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7746105640 1 -- 172.21.15.53:0/3804106295 learned_addr learned my addr 172.21.15.53:0/3804106295 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:15.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7745904640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f7740071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043652254 0 0) 0x7f7740160d10 con 0x7f774006a3a0 2023-10-21T17:19:15.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c003520 con 0x7f774006a3a0 2023-10-21T17:19:15.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979093721 0 0) 0x7f772c003520 con 0x7f774006a3a0 2023-10-21T17:19:15.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 msgr2=0x7f7740071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:15.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f7740071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 >> v1:172.21.15.124:6789/0 conn(0x7f774006ec00 legacy=0x7f77400c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740072330 con 0x7f774006a3a0 2023-10-21T17:19:15.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.290+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f77380019f0 con 0x7f774006a3a0 2023-10-21T17:19:15.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 >> v1:172.21.15.53:6789/0 conn(0x7f774006a3a0 legacy=0x7f7740160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f772c0039a0 0x7f772c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772c0047c0 0x7f772c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f772c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7740071580 con 0x7f772c0047c0 2023-10-21T17:19:15.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c00ba30 con 0x7f772c0039a0 2023-10-21T17:19:15.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c00bbf0 con 0x7f7740069fc0 2023-10-21T17:19:15.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745103640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772c0047c0 0x7f772c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745904640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f772c0039a0 0x7f772c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7746105640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f772c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745103640 1 -- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 msgr2=0x7f772c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:15.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745103640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7740069fc0 0x7f772c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745103640 1 -- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f772c0039a0 msgr2=0x7f772c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:15.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.291+0000 7f7745103640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f772c0039a0 0x7f772c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.292+0000 7f7745103640 1 -- 172.21.15.53:0/3804106295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c003520 con 0x7f772c0047c0 2023-10-21T17:19:15.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7745103640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772c0047c0 0x7f772c004c40 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f774006f040 tx=0x7f772c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fc03bf46c6dab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300052d0 con 0x7f772c0047c0 2023-10-21T17:19:15.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773002e050 con 0x7f772c0047c0 2023-10-21T17:19:15.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7744902640 1 -- 172.21.15.53:0/3804106295 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300035f0 con 0x7f772c0047c0 2023-10-21T17:19:15.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7746906640 1 -- 172.21.15.53:0/3804106295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772c0047c0 msgr2=0x7f772c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7746906640 1 --2- 172.21.15.53:0/3804106295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772c0047c0 0x7f772c004c40 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f774006f040 tx=0x7f772c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.293+0000 7f7746105640 1 -- 172.21.15.53:0/3804106295 reap_dead start 2023-10-21T17:19:15.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.294+0000 7f7746906640 1 -- 172.21.15.53:0/3804106295 shutdown_connections 2023-10-21T17:19:15.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.294+0000 7f7746906640 1 -- 172.21.15.53:0/3804106295 >> 172.21.15.53:0/3804106295 conn(0x7f77400cf5f0 msgr2=0x7f77400cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:15.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.294+0000 7f7746906640 1 -- 172.21.15.53:0/3804106295 shutdown_connections 2023-10-21T17:19:15.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.294+0000 7f7746906640 1 -- 172.21.15.53:0/3804106295 wait complete. 2023-10-21T17:19:15.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.295+0000 7f7746906640 1 Processor -- start 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.296+0000 7f7746906640 1 -- start start 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772c0047c0 0x7f7740153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 0x7f77400ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400ace80 con 0x7f77400b00f0 2023-10-21T17:19:15.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400c6e20 con 0x7f77400ac4c0 2023-10-21T17:19:15.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746906640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400b0e20 con 0x7f772c0047c0 2023-10-21T17:19:15.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59440/0 (socket says 172.21.15.53:59440) 2023-10-21T17:19:15.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 -- 172.21.15.53:0/3549332078 learned_addr learned my addr 172.21.15.53:0/3549332078 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:15.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7745103640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 0x7f77400ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7745904640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772c0047c0 0x7f7740153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 -- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772c0047c0 msgr2=0x7f7740153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772c0047c0 0x7f7740153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 -- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 msgr2=0x7f77400ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 0x7f77400ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.297+0000 7f7746105640 1 -- 172.21.15.53:0/3549332078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400c67d0 con 0x7f77400b00f0 2023-10-21T17:19:15.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f7745103640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 0x7f77400ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:15.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f7746105640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7738007c40 tx=0x7f773800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800bb80 con 0x7f77400b00f0 2023-10-21T17:19:15.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773800bce0 con 0x7f77400b00f0 2023-10-21T17:19:15.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738007290 con 0x7f77400b00f0 2023-10-21T17:19:15.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77400c6ab0 con 0x7f77400b00f0 2023-10-21T17:19:15.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.298+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400b0ab0 con 0x7f77400b00f0 2023-10-21T17:19:15.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.300+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77400c6630 con 0x7f77400b00f0 2023-10-21T17:19:15.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.301+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77380073f0 con 0x7f77400b00f0 2023-10-21T17:19:15.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.302+0000 7f77367fc640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 0x7f772406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.302+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f7738089d40 con 0x7f77400b00f0 2023-10-21T17:19:15.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.302+0000 7f7745904640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 0x7f772406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.303+0000 7f7745904640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 0x7f772406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f773c009a10 tx=0x7f773c008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.306+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77400c6630 con 0x7f77400b00f0 2023-10-21T17:19:15.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.501+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f77400c6630 con 0x7f77400b00f0 2023-10-21T17:19:15.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.506+0000 7f77367fc640 1 -- 172.21.15.53:0/3549332078 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77400c6630 con 0x7f77400b00f0 2023-10-21T17:19:15.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.508+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 msgr2=0x7f772406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 0x7f772406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f773c009a10 tx=0x7f773c008040 comp rx=0 tx=0).stop 2023-10-21T17:19:15.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 msgr2=0x7f77400b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7738007c40 tx=0x7f773800b4a0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 shutdown_connections 2023-10-21T17:19:15.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f772406af90 0x7f772406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77400b00f0 0x7f77400b0570 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77400ac4c0 0x7f77400ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 --2- 172.21.15.53:0/3549332078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772c0047c0 0x7f7740153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 >> 172.21.15.53:0/3549332078 conn(0x7f77400cf5f0 msgr2=0x7f77400cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:15.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 shutdown_connections 2023-10-21T17:19:15.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.509+0000 7f7746906640 1 -- 172.21.15.53:0/3549332078 wait complete. 2023-10-21T17:19:15.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:19:15.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.756+0000 7f97b66b4640 1 Processor -- start 2023-10-21T17:19:15.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.756+0000 7f97b66b4640 1 -- start start 2023-10-21T17:19:15.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.757+0000 7f97b66b4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97b00d1500 0x7f97b00d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.757+0000 7f97b66b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00ce290 con 0x7f97b015dcd0 2023-10-21T17:19:15.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b66b4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00d0a80 con 0x7f97b00d1120 2023-10-21T17:19:15.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b66b4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00b1fe0 con 0x7f97b00d1500 2023-10-21T17:19:15.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b4eb1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97b00d1500 0x7f97b00d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b5eb3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f97b015dcd0 0x7f97b00cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35358/0 (socket says 172.21.15.53:35358) 2023-10-21T17:19:15.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b5eb3640 1 -- 172.21.15.53:0/2245411873 learned_addr learned my addr 172.21.15.53:0/2245411873 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:15.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b4eb1640 1 -- 172.21.15.53:0/2245411873 >> v1:172.21.15.124:6789/0 conn(0x7f97b00d1120 legacy=0x7f97b00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:15.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b4eb1640 1 -- 172.21.15.53:0/2245411873 >> v1:172.21.15.53:6789/0 conn(0x7f97b015dcd0 legacy=0x7f97b00cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97b4eb1640 1 -- 172.21.15.53:0/2245411873 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00b4580 con 0x7f97b00d1500 2023-10-21T17:19:15.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.758+0000 7f97a7fff640 1 -- 172.21.15.53:0/2245411873 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528428806 0 0) 0x7f97b00ce290 con 0x7f97b015dcd0 2023-10-21T17:19:15.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97b4eb1640 1 --2- 172.21.15.53:0/2245411873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b00d1500 0x7f97b00d04a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097a0 tx=0x7f97a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79166fa9fba20997 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97a7fff640 1 -- 172.21.15.53:0/2245411873 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a002e070 con 0x7f97b00d1500 2023-10-21T17:19:15.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97a7fff640 1 -- 172.21.15.53:0/2245411873 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00315a0 con 0x7f97b00d1500 2023-10-21T17:19:15.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97a7fff640 1 -- 172.21.15.53:0/2245411873 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0031930 con 0x7f97b00d1500 2023-10-21T17:19:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97b66b4640 1 -- 172.21.15.53:0/2245411873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b00d1500 msgr2=0x7f97b00d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.760+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2245411873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b00d1500 0x7f97b00d04a0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097a0 tx=0x7f97a002b840 comp rx=0 tx=0).stop 2023-10-21T17:19:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.761+0000 7f97b66b4640 1 -- 172.21.15.53:0/2245411873 shutdown_connections 2023-10-21T17:19:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.761+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2245411873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b00d1500 0x7f97b00d04a0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.761+0000 7f97b66b4640 1 -- 172.21.15.53:0/2245411873 >> 172.21.15.53:0/2245411873 conn(0x7f97b00d2960 msgr2=0x7f97b00d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.761+0000 7f97b66b4640 1 -- 172.21.15.53:0/2245411873 shutdown_connections 2023-10-21T17:19:15.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.761+0000 7f97b66b4640 1 -- 172.21.15.53:0/2245411873 wait complete. 2023-10-21T17:19:15.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.762+0000 7f97b66b4640 1 Processor -- start 2023-10-21T17:19:15.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.763+0000 7f97b66b4640 1 -- start start 2023-10-21T17:19:15.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97b00d1120 0x7f97b00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97b00d1500 0x7f97b0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f2cf0 con 0x7f97b015dcd0 2023-10-21T17:19:15.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f2fd0 con 0x7f97b00d1120 2023-10-21T17:19:15.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b66b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b01f32b0 con 0x7f97b00d1500 2023-10-21T17:19:15.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b5eb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b5eb3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59460/0 (socket says 172.21.15.53:59460) 2023-10-21T17:19:15.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b5eb3640 1 -- 172.21.15.53:0/2679001712 learned_addr learned my addr 172.21.15.53:0/2679001712 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:15.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b4eb1640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97b00d1500 0x7f97b0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.764+0000 7f97b56b2640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97b00d1120 0x7f97b00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 -- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97b00d1500 msgr2=0x7f97b0150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97b00d1500 0x7f97b0150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 -- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97b00d1120 msgr2=0x7f97b00c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97b00d1120 0x7f97b00c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 -- 172.21.15.53:0/2679001712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009400 con 0x7f97b015dcd0 2023-10-21T17:19:15.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b5eb3640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c780 tx=0x7f97a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800ce90 con 0x7f97b015dcd0 2023-10-21T17:19:15.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a800a640 con 0x7f97b015dcd0 2023-10-21T17:19:15.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800a9d0 con 0x7f97b015dcd0 2023-10-21T17:19:15.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.765+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b01f3610 con 0x7f97b015dcd0 2023-10-21T17:19:15.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.766+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b0151290 con 0x7f97b015dcd0 2023-10-21T17:19:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.767+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b01f3b30 con 0x7f97b015dcd0 2023-10-21T17:19:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.768+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f97a800ab30 con 0x7f97b015dcd0 2023-10-21T17:19:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.768+0000 7f97a5ffb640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 0x7f979406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.769+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f97a8089170 con 0x7f97b015dcd0 2023-10-21T17:19:15.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.769+0000 7f97b56b2640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 0x7f979406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:15.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.770+0000 7f97b56b2640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 0x7f979406d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f97b01f59a0 tx=0x7f97ac009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:15.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.773+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97b01f3b30 con 0x7f97b015dcd0 2023-10-21T17:19:15.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.966+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f97b01f3b30 con 0x7f97b015dcd0 2023-10-21T17:19:15.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.972+0000 7f97a5ffb640 1 -- 172.21.15.53:0/2679001712 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7f97b01f3b30 con 0x7f97b015dcd0 2023-10-21T17:19:15.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 msgr2=0x7f979406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 0x7f979406d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f97b01f59a0 tx=0x7f97ac009040 comp rx=0 tx=0).stop 2023-10-21T17:19:15.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 msgr2=0x7f97b01f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:15.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c780 tx=0x7f97a800cc40 comp rx=0 tx=0).stop 2023-10-21T17:19:15.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 shutdown_connections 2023-10-21T17:19:15.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f979406afe0 0x7f979406d4a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97b015dcd0 0x7f97b01f2420 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97b00d1500 0x7f97b0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 --2- 172.21.15.53:0/2679001712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97b00d1120 0x7f97b00c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:15.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 >> 172.21.15.53:0/2679001712 conn(0x7f97b00d2960 msgr2=0x7f97b00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:15.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 shutdown_connections 2023-10-21T17:19:15.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:15.975+0000 7f97b66b4640 1 -- 172.21.15.53:0/2679001712 wait complete. 2023-10-21T17:19:15.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:19:16.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.225+0000 7f662740e640 1 Processor -- start 2023-10-21T17:19:16.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.225+0000 7f662740e640 1 -- start start 2023-10-21T17:19:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.226+0000 7f662740e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.226+0000 7f662740e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 0x7f66200cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.226+0000 7f662740e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662015d160 con 0x7f66200cdda0 2023-10-21T17:19:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.226+0000 7f662740e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66200ce230 con 0x7f66200b6730 2023-10-21T17:19:16.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.226+0000 7f662740e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200b7b90 con 0x7f662015cd80 2023-10-21T17:19:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.227+0000 7f66251aa640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.227+0000 7f66249a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 0x7f66200cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.227+0000 7f66251aa640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:38830/0 (socket says 172.21.15.53:38830) 2023-10-21T17:19:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.227+0000 7f66251aa640 1 -- 172.21.15.53:0/1826529767 learned_addr learned my addr 172.21.15.53:0/1826529767 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.228+0000 7f66251aa640 1 -- 172.21.15.53:0/1826529767 >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 msgr2=0x7f66200cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.228+0000 7f66251aa640 1 --2- 172.21.15.53:0/1826529767 >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 0x7f66200cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.228+0000 7f66251aa640 1 -- 172.21.15.53:0/1826529767 >> v1:172.21.15.124:6789/0 conn(0x7f66200cdda0 legacy=0x7f66200b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.228+0000 7f66251aa640 1 -- 172.21.15.53:0/1826529767 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200b8020 con 0x7f662015cd80 2023-10-21T17:19:16.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.228+0000 7f66249a9640 1 --2- 172.21.15.53:0/1826529767 >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 0x7f66200cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:19:16.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.230+0000 7f66251aa640 1 --2- 172.21.15.53:0/1826529767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f661c009e50 tx=0x7f661c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0967dfaae16e3a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.230+0000 7f6617fff640 1 -- 172.21.15.53:0/1826529767 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c02cb00 con 0x7f662015cd80 2023-10-21T17:19:16.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.230+0000 7f6617fff640 1 -- 172.21.15.53:0/1826529767 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c02cc60 con 0x7f662015cd80 2023-10-21T17:19:16.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.230+0000 7f6617fff640 1 -- 172.21.15.53:0/1826529767 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0343f0 con 0x7f662015cd80 2023-10-21T17:19:16.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 -- 172.21.15.53:0/1826529767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 msgr2=0x7f66200b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 --2- 172.21.15.53:0/1826529767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f661c009e50 tx=0x7f661c02bf60 comp rx=0 tx=0).stop 2023-10-21T17:19:16.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 -- 172.21.15.53:0/1826529767 shutdown_connections 2023-10-21T17:19:16.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 --2- 172.21.15.53:0/1826529767 >> v2:172.21.15.124:3300/0 conn(0x7f66200b6730 0x7f66200cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 --2- 172.21.15.53:0/1826529767 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f662015cd80 0x7f66200b87d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 -- 172.21.15.53:0/1826529767 >> 172.21.15.53:0/1826529767 conn(0x7f662006b5d0 msgr2=0x7f662006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:16.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 -- 172.21.15.53:0/1826529767 shutdown_connections 2023-10-21T17:19:16.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.231+0000 7f662740e640 1 -- 172.21.15.53:0/1826529767 wait complete. 2023-10-21T17:19:16.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.233+0000 7f662740e640 1 Processor -- start 2023-10-21T17:19:16.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.233+0000 7f662740e640 1 -- start start 2023-10-21T17:19:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 0x7f66200c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 0x7f66200c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620153af0 con 0x7f66200b6730 2023-10-21T17:19:16.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66200c5eb0 con 0x7f662015cd80 2023-10-21T17:19:16.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f662740e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200caae0 con 0x7f66200cdda0 2023-10-21T17:19:16.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f66251aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f66251aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59476/0 (socket says 172.21.15.53:59476) 2023-10-21T17:19:16.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f66251aa640 1 -- 172.21.15.53:0/1104621486 learned_addr learned my addr 172.21.15.53:0/1104621486 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:16.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.234+0000 7f66259ab640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 0x7f66200c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66249a9640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 0x7f66200c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 -- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 msgr2=0x7f66200c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 0x7f66200c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 -- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 msgr2=0x7f66200c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 0x7f66200c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 -- 172.21.15.53:0/1104621486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009400 con 0x7f66200b6730 2023-10-21T17:19:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66259ab640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 0x7f66200c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66249a9640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 0x7f66200c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:16.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f66251aa640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f661c000f80 tx=0x7f661c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c009510 con 0x7f66200b6730 2023-10-21T17:19:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c03e450 con 0x7f66200b6730 2023-10-21T17:19:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c009b30 con 0x7f66200b6730 2023-10-21T17:19:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.235+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c03e7e0 con 0x7f66200b6730 2023-10-21T17:19:16.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.236+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66200cb350 con 0x7f66200b6730 2023-10-21T17:19:16.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.237+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662004e160 con 0x7f66200b6730 2023-10-21T17:19:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.238+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f661c03e940 con 0x7f66200b6730 2023-10-21T17:19:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.238+0000 7f6615ffb640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.239+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f661c0ac9a0 con 0x7f66200b6730 2023-10-21T17:19:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.239+0000 7f66249a9640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.240+0000 7f66249a9640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 0x7f660406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6620160330 tx=0x7f6610008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.243+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f661c077750 con 0x7f66200b6730 2023-10-21T17:19:16.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.446+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f66200b88d0 con 0x7f66200b6730 2023-10-21T17:19:16.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.453+0000 7f6615ffb640 1 -- 172.21.15.53:0/1104621486 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7f661c033e80 con 0x7f66200b6730 2023-10-21T17:19:16.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 msgr2=0x7f660406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 0x7f660406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6620160330 tx=0x7f6610008040 comp rx=0 tx=0).stop 2023-10-21T17:19:16.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 msgr2=0x7f66200c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f661c000f80 tx=0x7f661c004a40 comp rx=0 tx=0).stop 2023-10-21T17:19:16.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 shutdown_connections 2023-10-21T17:19:16.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f662015cd80 0x7f66200c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66200cdda0 0x7f66200c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f660406af90 0x7f660406d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 --2- 172.21.15.53:0/1104621486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66200b6730 0x7f66200c8db0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 >> 172.21.15.53:0/1104621486 conn(0x7f662006b5d0 msgr2=0x7f662015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:16.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 shutdown_connections 2023-10-21T17:19:16.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.456+0000 7f662740e640 1 -- 172.21.15.53:0/1104621486 wait complete. 2023-10-21T17:19:16.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-21T17:19:16.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.696+0000 7f5e05ed1640 1 Processor -- start 2023-10-21T17:19:16.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.696+0000 7f5e05ed1640 1 -- start start 2023-10-21T17:19:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.697+0000 7f5e05ed1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5e0015ab30 0x7f5e0015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.697+0000 7f5e05ed1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e000adc70 0x7f5e000b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.697+0000 7f5e05ed1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0015f7d0 con 0x7f5e0015af10 2023-10-21T17:19:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.697+0000 7f5e05ed1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0015f210 con 0x7f5e000adc70 2023-10-21T17:19:16.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.697+0000 7f5e05ed1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0015f4f0 con 0x7f5e0015ab30 2023-10-21T17:19:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5e056d0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5e0015af10 0x7f5e000701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35374/0 (socket says 172.21.15.53:35374) 2023-10-21T17:19:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5e056d0640 1 -- 172.21.15.53:0/2884489623 learned_addr learned my addr 172.21.15.53:0/2884489623 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:16.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dfffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e000adc70 0x7f5e000b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5e04ecf640 1 --2- 172.21.15.53:0/2884489623 >> v2:172.21.15.124:3300/0 conn(0x7f5e0015ab30 0x7f5e0015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dfffff640 1 -- 172.21.15.53:0/2884489623 >> v2:172.21.15.124:3300/0 conn(0x7f5e0015ab30 msgr2=0x7f5e0015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:16.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dfffff640 1 --2- 172.21.15.53:0/2884489623 >> v2:172.21.15.124:3300/0 conn(0x7f5e0015ab30 0x7f5e0015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dfffff640 1 -- 172.21.15.53:0/2884489623 >> v1:172.21.15.53:6789/0 conn(0x7f5e0015af10 legacy=0x7f5e000701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dff7fe640 1 -- 172.21.15.53:0/2884489623 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233761447 0 0) 0x7f5e0015f7d0 con 0x7f5e0015af10 2023-10-21T17:19:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.698+0000 7f5dfffff640 1 -- 172.21.15.53:0/2884489623 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0006c550 con 0x7f5e000adc70 2023-10-21T17:19:16.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.700+0000 7f5dfffff640 1 --2- 172.21.15.53:0/2884489623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e000adc70 0x7f5e000b00c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006ada0 tx=0x7f5dec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd993e1c3cceff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.700+0000 7f5dff7fe640 1 -- 172.21.15.53:0/2884489623 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031830 con 0x7f5e000adc70 2023-10-21T17:19:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.700+0000 7f5dff7fe640 1 -- 172.21.15.53:0/2884489623 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec031990 con 0x7f5e000adc70 2023-10-21T17:19:16.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.700+0000 7f5dff7fe640 1 -- 172.21.15.53:0/2884489623 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031d20 con 0x7f5e000adc70 2023-10-21T17:19:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 -- 172.21.15.53:0/2884489623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e000adc70 msgr2=0x7f5e000b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/2884489623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e000adc70 0x7f5e000b00c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006ada0 tx=0x7f5dec02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 -- 172.21.15.53:0/2884489623 shutdown_connections 2023-10-21T17:19:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/2884489623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e000adc70 0x7f5e000b00c0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/2884489623 >> v2:172.21.15.124:3300/0 conn(0x7f5e0015ab30 0x7f5e0015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 -- 172.21.15.53:0/2884489623 >> 172.21.15.53:0/2884489623 conn(0x7f5e000cb0c0 msgr2=0x7f5e000cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:16.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 -- 172.21.15.53:0/2884489623 shutdown_connections 2023-10-21T17:19:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.701+0000 7f5e05ed1640 1 -- 172.21.15.53:0/2884489623 wait complete. 2023-10-21T17:19:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.703+0000 7f5e05ed1640 1 Processor -- start 2023-10-21T17:19:16.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.703+0000 7f5e05ed1640 1 -- start start 2023-10-21T17:19:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 0x7f5e00148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0015ab30 0x7f5e00147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00148090 con 0x7f5e0015af10 2023-10-21T17:19:16.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0015b9d0 con 0x7f5e0015ab30 2023-10-21T17:19:16.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e05ed1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000b0a00 con 0x7f5e000adc70 2023-10-21T17:19:16.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.704+0000 7f5e056d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59508/0 (socket says 172.21.15.53:59508) 2023-10-21T17:19:16.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 -- 172.21.15.53:0/129726974 learned_addr learned my addr 172.21.15.53:0/129726974 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:16.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5dfffff640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0015ab30 0x7f5e00147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e04ecf640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 0x7f5e00148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 -- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 msgr2=0x7f5e00148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 0x7f5e00148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 -- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0015ab30 msgr2=0x7f5e00147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0015ab30 0x7f5e00147b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 -- 172.21.15.53:0/129726974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec009400 con 0x7f5e0015af10 2023-10-21T17:19:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e04ecf640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 0x7f5e00148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:16.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.705+0000 7f5e056d0640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5df800cac0 tx=0x7f5df800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.706+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800ac30 con 0x7f5e0015af10 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.706+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df800ad90 con 0x7f5e0015af10 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.706+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8012620 con 0x7f5e0015af10 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.706+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0015b6f0 con 0x7f5e0015af10 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.706+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e000b0690 con 0x7f5e0015af10 2023-10-21T17:19:16.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.707+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0004e160 con 0x7f5e0015af10 2023-10-21T17:19:16.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.708+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5df8007070 con 0x7f5e0015af10 2023-10-21T17:19:16.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.709+0000 7f5dfd7fa640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 0x7f5de406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:16.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.709+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f5df8089900 con 0x7f5e0015af10 2023-10-21T17:19:16.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.710+0000 7f5e04ecf640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 0x7f5de406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:16.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.711+0000 7f5e04ecf640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 0x7f5de406d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5df4006fd0 tx=0x7f5df4008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:16.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.714+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5df8054690 con 0x7f5e0015af10 2023-10-21T17:19:16.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:16.981+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f5e000b0b40 con 0x7f5e0015af10 2023-10-21T17:19:17.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.066+0000 7f5dfd7fa640 1 -- 172.21.15.53:0/129726974 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2546) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5df8017020 con 0x7f5e0015af10 2023-10-21T17:19:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 msgr2=0x7f5de406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 0x7f5de406d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5df4006fd0 tx=0x7f5df4008040 comp rx=0 tx=0).stop 2023-10-21T17:19:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 msgr2=0x7f5e0015db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5df800cac0 tx=0x7f5df800cf80 comp rx=0 tx=0).stop 2023-10-21T17:19:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 shutdown_connections 2023-10-21T17:19:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5de406afc0 0x7f5de406d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e0015af10 0x7f5e0015db40 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e0015ab30 0x7f5e00147b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 --2- 172.21.15.53:0/129726974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000adc70 0x7f5e00148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 >> 172.21.15.53:0/129726974 conn(0x7f5e000cb0c0 msgr2=0x7f5e000b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:17.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 shutdown_connections 2023-10-21T17:19:17.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.070+0000 7f5e05ed1640 1 -- 172.21.15.53:0/129726974 wait complete. 2023-10-21T17:19:17.093 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-21T17:19:17.094 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-21T17:19:17.094 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T17:19:17.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:17.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.326+0000 7ff93a3ac640 1 Processor -- start 2023-10-21T17:19:17.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.327+0000 7ff93a3ac640 1 -- start start 2023-10-21T17:19:17.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.328+0000 7ff93a3ac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff93415ab30 0x7ff93415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.328+0000 7ff93a3ac640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff9340b07d0 0x7ff93406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.328+0000 7ff93a3ac640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93415f7d0 con 0x7ff9340adc70 2023-10-21T17:19:17.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.328+0000 7ff93a3ac640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93415f210 con 0x7ff93415ab30 2023-10-21T17:19:17.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.328+0000 7ff93a3ac640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93415f4f0 con 0x7ff9340b07d0 2023-10-21T17:19:17.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff93415ab30 0x7ff93415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff93415ab30 0x7ff93415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59520/0 (socket says 172.21.15.53:59520) 2023-10-21T17:19:17.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 -- 172.21.15.53:0/3062596864 learned_addr learned my addr 172.21.15.53:0/3062596864 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:17.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff939bab640 1 --2- 172.21.15.53:0/3062596864 >> v2:172.21.15.124:3300/0 conn(0x7ff9340b07d0 0x7ff93406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 -- 172.21.15.53:0/3062596864 >> v2:172.21.15.124:3300/0 conn(0x7ff9340b07d0 msgr2=0x7ff93406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 --2- 172.21.15.53:0/3062596864 >> v2:172.21.15.124:3300/0 conn(0x7ff9340b07d0 0x7ff93406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 -- 172.21.15.53:0/3062596864 >> v1:172.21.15.124:6789/0 conn(0x7ff9340adc70 legacy=0x7ff9340b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:17.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.329+0000 7ff9393aa640 1 -- 172.21.15.53:0/3062596864 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93406c550 con 0x7ff93415ab30 2023-10-21T17:19:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.331+0000 7ff9393aa640 1 --2- 172.21.15.53:0/3062596864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff93415ab30 0x7ff93415e8a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ff9340ca4a0 tx=0x7ff93002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea44d6a0f1f6590 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff92bfff640 1 -- 172.21.15.53:0/3062596864 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930031870 con 0x7ff93415ab30 2023-10-21T17:19:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff92bfff640 1 -- 172.21.15.53:0/3062596864 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9300319d0 con 0x7ff93415ab30 2023-10-21T17:19:17.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff92bfff640 1 -- 172.21.15.53:0/3062596864 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930031d60 con 0x7ff93415ab30 2023-10-21T17:19:17.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 -- 172.21.15.53:0/3062596864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff93415ab30 msgr2=0x7ff93415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/3062596864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff93415ab30 0x7ff93415e8a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7ff9340ca4a0 tx=0x7ff93002b910 comp rx=0 tx=0).stop 2023-10-21T17:19:17.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 -- 172.21.15.53:0/3062596864 shutdown_connections 2023-10-21T17:19:17.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/3062596864 >> v2:172.21.15.124:3300/0 conn(0x7ff9340b07d0 0x7ff93406cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/3062596864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff93415ab30 0x7ff93415e8a0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 -- 172.21.15.53:0/3062596864 >> 172.21.15.53:0/3062596864 conn(0x7ff9340cb0c0 msgr2=0x7ff9340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:17.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 -- 172.21.15.53:0/3062596864 shutdown_connections 2023-10-21T17:19:17.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.332+0000 7ff93a3ac640 1 -- 172.21.15.53:0/3062596864 wait complete. 2023-10-21T17:19:17.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.334+0000 7ff93a3ac640 1 Processor -- start 2023-10-21T17:19:17.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.334+0000 7ff93a3ac640 1 -- start start 2023-10-21T17:19:17.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 0x7ff9340c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 0x7ff9341f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340aba90 con 0x7ff9340adc70 2023-10-21T17:19:17.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff934149480 con 0x7ff9340b07d0 2023-10-21T17:19:17.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff93a3ac640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c8530 con 0x7ff93415ab30 2023-10-21T17:19:17.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff9393aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff9393aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59534/0 (socket says 172.21.15.53:59534) 2023-10-21T17:19:17.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff9393aa640 1 -- 172.21.15.53:0/2381461642 learned_addr learned my addr 172.21.15.53:0/2381461642 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff938ba9640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 0x7ff9340c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff939bab640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 0x7ff9341f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.336+0000 7ff9393aa640 1 -- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 msgr2=0x7ff9341f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff9393aa640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 0x7ff9341f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff9393aa640 1 -- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 msgr2=0x7ff9340c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff9393aa640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 0x7ff9340c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff9393aa640 1 -- 172.21.15.53:0/2381461642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300094d0 con 0x7ff9340adc70 2023-10-21T17:19:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff938ba9640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 0x7ff9340c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff939bab640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 0x7ff9341f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:17.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff9393aa640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff9300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93003abe0 con 0x7ff9340adc70 2023-10-21T17:19:17.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93003ad40 con 0x7ff9340adc70 2023-10-21T17:19:17.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930044610 con 0x7ff9340adc70 2023-10-21T17:19:17.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.337+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340c7c90 con 0x7ff9340adc70 2023-10-21T17:19:17.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.338+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9340c8250 con 0x7ff9340adc70 2023-10-21T17:19:17.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.339+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93406dc60 con 0x7ff9340adc70 2023-10-21T17:19:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.340+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff93004d450 con 0x7ff9340adc70 2023-10-21T17:19:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.340+0000 7ff929ffb640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.341+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7ff93002d040 con 0x7ff9340adc70 2023-10-21T17:19:17.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.341+0000 7ff938ba9640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.343+0000 7ff938ba9640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 0x7ff91806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9340702c0 tx=0x7ff924009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.345+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff930076d10 con 0x7ff9340adc70 2023-10-21T17:19:17.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.544+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff93404e160 con 0x7ff9340adc70 2023-10-21T17:19:17.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.546+0000 7ff929ffb640 1 -- 172.21.15.53:0/2381461642 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+14382 (secure 0 0 0) 0x7ff930035020 con 0x7ff9340adc70 2023-10-21T17:19:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:17.547 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":189,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:15.043296+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6835/84861935":"2023-10-22T17:19:12.025588+0000","172.21.15.124:0/930439389":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6834/84861935":"2023-10-22T17:19:12.025588+0000","172.21.15.124:6833/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6832/97801354":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/57805179":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3870614335":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/1655250226":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/4079618600":"2023-10-22T17:19:07.927655+0000","172.21.15.124:0/3580514343":"2023-10-22T17:19:07.927655+0000","172.21.15.124:6833/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:6832/234236878":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1818943929":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1207144287":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/207583581":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/1931157828":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/880607286":"2023-10-22T17:19:00.108230+0000","172.21.15.124:0/3805066261":"2023-10-22T17:19:00.108230+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 msgr2=0x7ff91806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 0x7ff91806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9340702c0 tx=0x7ff924009040 comp rx=0 tx=0).stop 2023-10-21T17:19:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 msgr2=0x7ff934148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff9300046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 shutdown_connections 2023-10-21T17:19:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff93415ab30 0x7ff9341f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9340b07d0 0x7ff9340c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 --2- 172.21.15.53:0/2381461642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9340adc70 0x7ff934148f40 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 >> 172.21.15.53:0/2381461642 conn(0x7ff9340cb0c0 msgr2=0x7ff934159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 shutdown_connections 2023-10-21T17:19:17.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.548+0000 7ff93a3ac640 1 -- 172.21.15.53:0/2381461642 wait complete. 2023-10-21T17:19:17.568 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:19:17.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:19:17.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.794+0000 7efdec82d640 1 Processor -- start 2023-10-21T17:19:17.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.795+0000 7efdec82d640 1 -- start start 2023-10-21T17:19:17.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.796+0000 7efdec82d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efde80b1750 0x7efde806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.796+0000 7efdec82d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.796+0000 7efdec82d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde815bf30 con 0x7efde815aad0 2023-10-21T17:19:17.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.796+0000 7efdec82d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde806d6b0 con 0x7efde80b1750 2023-10-21T17:19:17.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.796+0000 7efdec82d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde80b1bb0 con 0x7efde806d240 2023-10-21T17:19:17.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.797+0000 7efddffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efde80b1750 0x7efde806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.797+0000 7efddffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efde80b1750 0x7efde806cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:39608/0 (socket says 172.21.15.53:39608) 2023-10-21T17:19:17.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.797+0000 7efddffff640 1 -- 172.21.15.53:0/2412242178 learned_addr learned my addr 172.21.15.53:0/2412242178 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:17.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.797+0000 7efde6d9d640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.798+0000 7efddffff640 1 -- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 msgr2=0x7efde8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:17.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.798+0000 7efddffff640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.798+0000 7efddffff640 1 -- 172.21.15.53:0/2412242178 >> v1:172.21.15.124:6789/0 conn(0x7efde815aad0 legacy=0x7efde80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.798+0000 7efddffff640 1 -- 172.21.15.53:0/2412242178 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde815b320 con 0x7efde80b1750 2023-10-21T17:19:17.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.799+0000 7efddffff640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b1750 0x7efde806cd00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009870 tx=0x7efdd002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbf25343f4d1b6a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efde5d9b640 1 -- 172.21.15.53:0/2412242178 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0030840 con 0x7efde80b1750 2023-10-21T17:19:17.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efde5d9b640 1 -- 172.21.15.53:0/2412242178 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00309a0 con 0x7efde80b1750 2023-10-21T17:19:17.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efde5d9b640 1 -- 172.21.15.53:0/2412242178 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0030d30 con 0x7efde80b1750 2023-10-21T17:19:17.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 -- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b1750 msgr2=0x7efde806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b1750 0x7efde806cd00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009870 tx=0x7efdd002b980 comp rx=0 tx=0).stop 2023-10-21T17:19:17.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 -- 172.21.15.53:0/2412242178 shutdown_connections 2023-10-21T17:19:17.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 --2- 172.21.15.53:0/2412242178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b1750 0x7efde806cd00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.800+0000 7efdec82d640 1 -- 172.21.15.53:0/2412242178 >> 172.21.15.53:0/2412242178 conn(0x7efde8069d50 msgr2=0x7efde806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:17.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.801+0000 7efdec82d640 1 -- 172.21.15.53:0/2412242178 shutdown_connections 2023-10-21T17:19:17.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.801+0000 7efdec82d640 1 -- 172.21.15.53:0/2412242178 wait complete. 2023-10-21T17:19:17.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.802+0000 7efdec82d640 1 Processor -- start 2023-10-21T17:19:17.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.803+0000 7efdec82d640 1 -- start start 2023-10-21T17:19:17.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde815aad0 0x7efde8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde8159470 con 0x7efde80b1750 2023-10-21T17:19:17.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde8152180 con 0x7efde815aad0 2023-10-21T17:19:17.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efdec82d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde8156a90 con 0x7efde806d240 2023-10-21T17:19:17.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efddffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efddffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59550/0 (socket says 172.21.15.53:59550) 2023-10-21T17:19:17.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efddffff640 1 -- 172.21.15.53:0/3835532209 learned_addr learned my addr 172.21.15.53:0/3835532209 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:17.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efde6d9d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde815aad0 0x7efde8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.804+0000 7efde659c640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 -- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 msgr2=0x7efde8151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 -- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde815aad0 msgr2=0x7efde8158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:17.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde815aad0 0x7efde8158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:17.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 -- 172.21.15.53:0/3835532209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00094d0 con 0x7efde80b1750 2023-10-21T17:19:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddffff640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000c00 tx=0x7efdd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.805+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0039c90 con 0x7efde80b1750 2023-10-21T17:19:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.806+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0039df0 con 0x7efde80b1750 2023-10-21T17:19:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.806+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0043640 con 0x7efde80b1750 2023-10-21T17:19:17.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.806+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde80c8f50 con 0x7efde80b1750 2023-10-21T17:19:17.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.806+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde80c9510 con 0x7efde80b1750 2023-10-21T17:19:17.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.807+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde804e160 con 0x7efde80b1750 2023-10-21T17:19:17.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.808+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdd00437a0 con 0x7efde80b1750 2023-10-21T17:19:17.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.809+0000 7efddeffd640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 0x7efdc806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:17.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.809+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7efdd003e070 con 0x7efde80b1750 2023-10-21T17:19:17.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.809+0000 7efde659c640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 0x7efdc806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:17.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.810+0000 7efde659c640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 0x7efdc806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdd8007fd0 tx=0x7efdd8009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:17.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:17.813+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdd0077760 con 0x7efde80b1750 2023-10-21T17:19:18.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.006+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efde80008d0 con 0x7efde80b1750 2023-10-21T17:19:18.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.007+0000 7efddeffd640 1 -- 172.21.15.53:0/3835532209 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdd0034020 con 0x7efde80b1750 2023-10-21T17:19:18.010 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:19:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 msgr2=0x7efdc806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 0x7efdc806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efdd8007fd0 tx=0x7efdd8009040 comp rx=0 tx=0).stop 2023-10-21T17:19:18.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 msgr2=0x7efde81589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000c00 tx=0x7efdd0004730 comp rx=0 tx=0).stop 2023-10-21T17:19:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 shutdown_connections 2023-10-21T17:19:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde815aad0 0x7efde8158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efdc806af90 0x7efdc806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efde80b1750 0x7efde81589f0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 --2- 172.21.15.53:0/3835532209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde806d240 0x7efde8151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 >> 172.21.15.53:0/3835532209 conn(0x7efde8069d50 msgr2=0x7efde80c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 shutdown_connections 2023-10-21T17:19:18.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.010+0000 7efdec82d640 1 -- 172.21.15.53:0/3835532209 wait complete. 2023-10-21T17:19:18.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-21T17:19:18.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.244+0000 7f9605d87640 1 Processor -- start 2023-10-21T17:19:18.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.245+0000 7f9605d87640 1 -- start start 2023-10-21T17:19:18.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.246+0000 7f9605d87640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9600161320 0x7f96000cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.246+0000 7f9605d87640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f96000b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.246+0000 7f9605d87640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9600165470 con 0x7f960006c710 2023-10-21T17:19:18.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.246+0000 7f9605d87640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9600165750 con 0x7f9600161320 2023-10-21T17:19:18.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.246+0000 7f9605d87640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9600165a30 con 0x7f9600164f70 2023-10-21T17:19:18.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f9605586640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f960006c710 0x7f96000c95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35376/0 (socket says 172.21.15.53:35376) 2023-10-21T17:19:18.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f9605586640 1 -- 172.21.15.53:0/1318813107 learned_addr learned my addr 172.21.15.53:0/1318813107 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:18.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95fffff640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f96000b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f9604d85640 1 --2- 172.21.15.53:0/1318813107 >> v2:172.21.15.124:3300/0 conn(0x7f9600161320 0x7f96000cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049181369 0 0) 0x7f9600165470 con 0x7f960006c710 2023-10-21T17:19:18.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f0003520 con 0x7f960006c710 2023-10-21T17:19:18.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175507706 0 0) 0x7f95f0003520 con 0x7f960006c710 2023-10-21T17:19:18.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 msgr2=0x7f96000b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:18.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f96000b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 >> v2:172.21.15.124:3300/0 conn(0x7f9600161320 msgr2=0x7f96000cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.247+0000 7f95ff7fe640 1 --2- 172.21.15.53:0/1318813107 >> v2:172.21.15.124:3300/0 conn(0x7f9600161320 0x7f96000cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600165d10 con 0x7f960006c710 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f95f80019f0 con 0x7f960006c710 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 >> v1:172.21.15.53:6789/0 conn(0x7f960006c710 legacy=0x7f96000c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f95f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f0003f40 0x7f95f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95f0004d00 0x7f95f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600165750 con 0x7f95f0004d00 2023-10-21T17:19:18.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600165a30 con 0x7f95f0003f40 2023-10-21T17:19:18.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f000ba00 con 0x7f9600164f70 2023-10-21T17:19:18.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f9605586640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95f0004d00 0x7f95f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f9604d85640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f95f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.248+0000 7f95fffff640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f0003f40 0x7f95f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.249+0000 7f9605586640 1 -- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 msgr2=0x7f95f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:18.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.249+0000 7f9605586640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9600164f70 0x7f95f0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.249+0000 7f9605586640 1 -- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f0003f40 msgr2=0x7f95f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:18.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.249+0000 7f9605586640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f0003f40 0x7f95f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.249+0000 7f9605586640 1 -- 172.21.15.53:0/1318813107 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0003520 con 0x7f95f0004d00 2023-10-21T17:19:18.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f9605586640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95f0004d00 0x7f95f0005180 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f960006cbc0 tx=0x7f95f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc83ef194186828 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:18.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8034630 con 0x7f95f0004d00 2023-10-21T17:19:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f802f070 con 0x7f95f0004d00 2023-10-21T17:19:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f95ff7fe640 1 -- 172.21.15.53:0/1318813107 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8034a10 con 0x7f95f0004d00 2023-10-21T17:19:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f9605d87640 1 -- 172.21.15.53:0/1318813107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95f0004d00 msgr2=0x7f95f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f9605d87640 1 --2- 172.21.15.53:0/1318813107 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95f0004d00 0x7f95f0005180 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f960006cbc0 tx=0x7f95f00057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.250+0000 7f9605586640 1 -- 172.21.15.53:0/1318813107 reap_dead start 2023-10-21T17:19:18.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.251+0000 7f9605d87640 1 -- 172.21.15.53:0/1318813107 shutdown_connections 2023-10-21T17:19:18.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.251+0000 7f9605d87640 1 -- 172.21.15.53:0/1318813107 >> 172.21.15.53:0/1318813107 conn(0x7f96000cb0c0 msgr2=0x7f96000cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:18.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.251+0000 7f9605d87640 1 -- 172.21.15.53:0/1318813107 shutdown_connections 2023-10-21T17:19:18.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.251+0000 7f9605d87640 1 -- 172.21.15.53:0/1318813107 wait complete. 2023-10-21T17:19:18.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.252+0000 7f9605d87640 1 Processor -- start 2023-10-21T17:19:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 -- start start 2023-10-21T17:19:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f0004d00 0x7f960015f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9600163530 0x7f96001639b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96000c5710 con 0x7f9600162560 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600163ef0 con 0x7f9600163530 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.253+0000 7f9605d87640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96001f78d0 con 0x7f95f0004d00 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59568/0 (socket says 172.21.15.53:59568) 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 -- 172.21.15.53:0/2257635214 learned_addr learned my addr 172.21.15.53:0/2257635214 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9604d85640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f0004d00 0x7f960015f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f95fffff640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9600163530 0x7f96001639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 -- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f0004d00 msgr2=0x7f960015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f0004d00 0x7f960015f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 -- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9600163530 msgr2=0x7f96001639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:18.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9600163530 0x7f96001639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:18.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.254+0000 7f9605586640 1 -- 172.21.15.53:0/2257635214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96001f7b70 con 0x7f9600162560 2023-10-21T17:19:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f9605586640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f95f8000f80 tx=0x7f95f8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8034840 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f80349f0 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8034d60 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96001f7e50 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.255+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96001f8410 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.256+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96001f8600 con 0x7f9600162560 2023-10-21T17:19:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.257+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f95f8031030 con 0x7f9600162560 2023-10-21T17:19:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.258+0000 7f95fd7fa640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 0x7f95e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.258+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f95f80abf20 con 0x7f9600162560 2023-10-21T17:19:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.259+0000 7f9604d85640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 0x7f95e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.260+0000 7f9604d85640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 0x7f95e406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009a90 tx=0x7f95f4008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.263+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95f8076cc0 con 0x7f9600162560 2023-10-21T17:19:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:18.527+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f96001f8600 con 0x7f9600162560 2023-10-21T17:19:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.072+0000 7f95fd7fa640 1 -- 172.21.15.53:0/2257635214 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2548) v1 ==== 168+0+0 (secure 0 0 0) 0x7f95f807a2e0 con 0x7f9600162560 2023-10-21T17:19:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 msgr2=0x7f95e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 0x7f95e406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009a90 tx=0x7f95f4008040 comp rx=0 tx=0).stop 2023-10-21T17:19:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 msgr2=0x7f96001629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f95f8000f80 tx=0x7f95f8001360 comp rx=0 tx=0).stop 2023-10-21T17:19:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 shutdown_connections 2023-10-21T17:19:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f95e406af90 0x7f95e406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9600162560 0x7f96001629e0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9600163530 0x7f96001639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 --2- 172.21.15.53:0/2257635214 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f0004d00 0x7f960015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.077+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 >> 172.21.15.53:0/2257635214 conn(0x7f96000cb0c0 msgr2=0x7f9600160860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.078+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 shutdown_connections 2023-10-21T17:19:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.078+0000 7f9605d87640 1 -- 172.21.15.53:0/2257635214 wait complete. 2023-10-21T17:19:19.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:19:19.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.332+0000 7fe2156a3640 1 Processor -- start 2023-10-21T17:19:19.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.332+0000 7fe2156a3640 1 -- start start 2023-10-21T17:19:19.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.333+0000 7fe2156a3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe21015a820 0x7fe2100aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:19.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.333+0000 7fe2156a3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100cab60 con 0x7fe21015ac00 2023-10-21T17:19:19.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.333+0000 7fe2156a3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100cb960 con 0x7fe2100ca680 2023-10-21T17:19:19.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.333+0000 7fe2156a3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2100cbdf0 con 0x7fe21015a820 2023-10-21T17:19:19.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe21015a820 0x7fe2100aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:19.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe21015a820 0x7fe2100aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59578/0 (socket says 172.21.15.53:59578) 2023-10-21T17:19:19.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 -- 172.21.15.53:0/1705320441 learned_addr learned my addr 172.21.15.53:0/1705320441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:19.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 -- 172.21.15.53:0/1705320441 >> v1:172.21.15.124:6789/0 conn(0x7fe2100ca680 legacy=0x7fe2100c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:19.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 -- 172.21.15.53:0/1705320441 >> v1:172.21.15.53:6789/0 conn(0x7fe21015ac00 legacy=0x7fe2100c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.334+0000 7fe20ffff640 1 -- 172.21.15.53:0/1705320441 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100c9f20 con 0x7fe21015a820 2023-10-21T17:19:19.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe20ffff640 1 --2- 172.21.15.53:0/1705320441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe21015a820 0x7fe2100aff70 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe200009e50 tx=0x7fe20002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7475bef1eac0c7a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:19.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe20effd640 1 -- 172.21.15.53:0/1705320441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20002ca80 con 0x7fe21015a820 2023-10-21T17:19:19.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe20effd640 1 -- 172.21.15.53:0/1705320441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20002cbe0 con 0x7fe21015a820 2023-10-21T17:19:19.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe20effd640 1 -- 172.21.15.53:0/1705320441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000343f0 con 0x7fe21015a820 2023-10-21T17:19:19.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe2156a3640 1 -- 172.21.15.53:0/1705320441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe21015a820 msgr2=0x7fe2100aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.336+0000 7fe2156a3640 1 --2- 172.21.15.53:0/1705320441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe21015a820 0x7fe2100aff70 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe200009e50 tx=0x7fe20002bee0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.337+0000 7fe2156a3640 1 -- 172.21.15.53:0/1705320441 shutdown_connections 2023-10-21T17:19:19.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.337+0000 7fe2156a3640 1 --2- 172.21.15.53:0/1705320441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe21015a820 0x7fe2100aff70 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.337+0000 7fe2156a3640 1 -- 172.21.15.53:0/1705320441 >> 172.21.15.53:0/1705320441 conn(0x7fe21006c050 msgr2=0x7fe21006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:19.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.337+0000 7fe2156a3640 1 -- 172.21.15.53:0/1705320441 shutdown_connections 2023-10-21T17:19:19.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.337+0000 7fe2156a3640 1 -- 172.21.15.53:0/1705320441 wait complete. 2023-10-21T17:19:19.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.338+0000 7fe2156a3640 1 Processor -- start 2023-10-21T17:19:19.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.339+0000 7fe2156a3640 1 -- start start 2023-10-21T17:19:19.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:19.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe21015a820 0x7fe2100c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:19.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe21015ac00 0x7fe2100addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:19.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21014be10 con 0x7fe2100ca680 2023-10-21T17:19:19.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100c43d0 con 0x7fe21015ac00 2023-10-21T17:19:19.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe2156a3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100c6370 con 0x7fe21015a820 2023-10-21T17:19:19.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:19.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59590/0 (socket says 172.21.15.53:59590) 2023-10-21T17:19:19.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 -- 172.21.15.53:0/4248461179 learned_addr learned my addr 172.21.15.53:0/4248461179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 -- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe21015a820 msgr2=0x7fe2100c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe21015a820 0x7fe2100c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 -- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe21015ac00 msgr2=0x7fe2100addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe21015ac00 0x7fe2100addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.340+0000 7fe20ffff640 1 -- 172.21.15.53:0/4248461179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200009b30 con 0x7fe2100ca680 2023-10-21T17:19:19.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe20ffff640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe200009e50 tx=0x7fe200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:19.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20003e450 con 0x7fe2100ca680 2023-10-21T17:19:19.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20003e5f0 con 0x7fe2100ca680 2023-10-21T17:19:19.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20003e9a0 con 0x7fe2100ca680 2023-10-21T17:19:19.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2100c6650 con 0x7fe2100ca680 2023-10-21T17:19:19.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.341+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2100c6c10 con 0x7fe2100ca680 2023-10-21T17:19:19.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.343+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210074150 con 0x7fe2100ca680 2023-10-21T17:19:19.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.343+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe20002cd50 con 0x7fe2100ca680 2023-10-21T17:19:19.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.344+0000 7fe20cff9640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 0x7fe1fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:19.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.344+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fe2000ac510 con 0x7fe2100ca680 2023-10-21T17:19:19.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.345+0000 7fe20f7fe640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 0x7fe1fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:19.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.346+0000 7fe20f7fe640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 0x7fe1fc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2100c87f0 tx=0x7fe2040079b0 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:19.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.349+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2000773d0 con 0x7fe2100ca680 2023-10-21T17:19:19.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.541+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe210073110 con 0x7fe2100ca680 2023-10-21T17:19:19.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.546+0000 7fe20cff9640 1 -- 172.21.15.53:0/4248461179 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe20007a9f0 con 0x7fe2100ca680 2023-10-21T17:19:19.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 msgr2=0x7fe1fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 0x7fe1fc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe2100c87f0 tx=0x7fe2040079b0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 msgr2=0x7fe21014b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:19.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe200009e50 tx=0x7fe200004a40 comp rx=0 tx=0).stop 2023-10-21T17:19:19.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 shutdown_connections 2023-10-21T17:19:19.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe21015ac00 0x7fe2100addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe21015a820 0x7fe2100c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe1fc06af90 0x7fe1fc06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 --2- 172.21.15.53:0/4248461179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2100ca680 0x7fe21014b8d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:19.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.549+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 >> 172.21.15.53:0/4248461179 conn(0x7fe21006c050 msgr2=0x7fe2100707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:19.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.550+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 shutdown_connections 2023-10-21T17:19:19.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:19.550+0000 7fe2156a3640 1 -- 172.21.15.53:0/4248461179 wait complete. 2023-10-21T17:19:19.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:19:19.571 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:19:19.694 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:19:19.818 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:19:19.819 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:19.819 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:19.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:19:19.887 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:19:19.887 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:19.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:19:19.949 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:19.949 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:20.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:19:20.013 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:19:20.013 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:20.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:19:20.075 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:19:20.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:20.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.311+0000 7fcdd7fff640 1 Processor -- start 2023-10-21T17:19:20.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.312+0000 7fcdd7fff640 1 -- start start 2023-10-21T17:19:20.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.313+0000 7fcdd7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcdd80cd8a0 0x7fcdd806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.313+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 0x7fcdd80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.313+0000 7fcdd7fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd80cb590 con 0x7fcdd80cf720 2023-10-21T17:19:20.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.313+0000 7fcdd7fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd80cfc90 con 0x7fcdd80cd8a0 2023-10-21T17:19:20.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.313+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80cff70 con 0x7fcdd806bf60 2023-10-21T17:19:20.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd67fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcdd80cf720 0x7fcdd806b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35398/0 (socket says 172.21.15.53:35398) 2023-10-21T17:19:20.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd67fc640 1 -- 172.21.15.53:0/3565208242 learned_addr learned my addr 172.21.15.53:0/3565208242 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:20.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/3565208242 >> v2:172.21.15.124:3300/0 conn(0x7fcdd80cd8a0 0x7fcdd806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506231678 0 0) 0x7fcdd80cb590 con 0x7fcdd80cf720 2023-10-21T17:19:20.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd77fe640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 0x7fcdd80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.314+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc4003520 con 0x7fcdd80cf720 2023-10-21T17:19:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354219899 0 0) 0x7fcdc4003520 con 0x7fcdd80cf720 2023-10-21T17:19:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 msgr2=0x7fcdd80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 0x7fcdd80cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 >> v2:172.21.15.124:3300/0 conn(0x7fcdd80cd8a0 msgr2=0x7fcdd806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 --2- 172.21.15.53:0/3565208242 >> v2:172.21.15.124:3300/0 conn(0x7fcdd80cd8a0 0x7fcdd806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80caf40 con 0x7fcdd80cf720 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcdc0001670 con 0x7fcdd80cf720 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 >> v1:172.21.15.53:6789/0 conn(0x7fcdd80cf720 legacy=0x7fcdd806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc4003970 0x7fcdc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc40047c0 0x7fcdc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 0x7fcdc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80cfc90 con 0x7fcdc40047c0 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80cff70 con 0x7fcdc4003970 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc400b980 con 0x7fcdd806bf60 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.315+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc40047c0 0x7fcdc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc4003970 0x7fcdc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd67fc640 1 -- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 msgr2=0x7fcdc4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:20.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd806bf60 0x7fcdc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd67fc640 1 -- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc4003970 msgr2=0x7fcdc4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:20.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc4003970 0x7fcdc4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd67fc640 1 -- 172.21.15.53:0/3565208242 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4003520 con 0x7fcdc40047c0 2023-10-21T17:19:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.316+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc4003970 0x7fcdc4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:20.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.317+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc40047c0 0x7fcdc4004c40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c110 tx=0x7fcdc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd1005c079bfd76 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.317+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0005ad0 con 0x7fcdc40047c0 2023-10-21T17:19:20.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.317+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0005c50 con 0x7fcdc40047c0 2023-10-21T17:19:20.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.317+0000 7fcdd5ffb640 1 -- 172.21.15.53:0/3565208242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002c830 con 0x7fcdc40047c0 2023-10-21T17:19:20.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 -- 172.21.15.53:0/3565208242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc40047c0 msgr2=0x7fcdc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/3565208242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdc40047c0 0x7fcdc4004c40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c110 tx=0x7fcdc40057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd77fe640 1 -- 172.21.15.53:0/3565208242 reap_dead start 2023-10-21T17:19:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 -- 172.21.15.53:0/3565208242 shutdown_connections 2023-10-21T17:19:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 -- 172.21.15.53:0/3565208242 >> 172.21.15.53:0/3565208242 conn(0x7fcdd8099b60 msgr2=0x7fcdd80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 -- 172.21.15.53:0/3565208242 shutdown_connections 2023-10-21T17:19:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.318+0000 7fcdd7fff640 1 -- 172.21.15.53:0/3565208242 wait complete. 2023-10-21T17:19:20.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.320+0000 7fcdd7fff640 1 Processor -- start 2023-10-21T17:19:20.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.320+0000 7fcdd7fff640 1 -- start start 2023-10-21T17:19:20.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc40047c0 0x7fcdd809aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd809b620 0x7fcdd809baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80cc2e0 con 0x7fcdd80cb920 2023-10-21T17:19:20.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd809bfe0 con 0x7fcdc40047c0 2023-10-21T17:19:20.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80c4380 con 0x7fcdd809b620 2023-10-21T17:19:20.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59596/0 (socket says 172.21.15.53:59596) 2023-10-21T17:19:20.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.321+0000 7fcdd67fc640 1 -- 172.21.15.53:0/1740156442 learned_addr learned my addr 172.21.15.53:0/1740156442 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc40047c0 0x7fcdd809aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 -- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd809b620 msgr2=0x7fcdd809baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd809b620 0x7fcdd809baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 -- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc40047c0 msgr2=0x7fcdd809aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc40047c0 0x7fcdd809aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 -- 172.21.15.53:0/1740156442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd809afd0 con 0x7fcdd80cb920 2023-10-21T17:19:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd67fc640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c230 tx=0x7fcdc0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002cc40 con 0x7fcdd80cb920 2023-10-21T17:19:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc002cda0 con 0x7fcdd80cb920 2023-10-21T17:19:20.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd80c4620 con 0x7fcdd80cb920 2023-10-21T17:19:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0007270 con 0x7fcdd80cb920 2023-10-21T17:19:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.322+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd80c4be0 con 0x7fcdd80cb920 2023-10-21T17:19:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.324+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd804e160 con 0x7fcdd80cb920 2023-10-21T17:19:20.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.325+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcdc0032070 con 0x7fcdd80cb920 2023-10-21T17:19:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.326+0000 7fcdcf7fe640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 0x7fcdb806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.326+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fcdc003c040 con 0x7fcdd80cb920 2023-10-21T17:19:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.326+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 0x7fcdb806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.327+0000 7fcdd6ffd640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 0x7fcdb806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8003b60 tx=0x7fcdc800b040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.330+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdc00757f0 con 0x7fcdd80cb920 2023-10-21T17:19:20.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.562+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdd80cded0 con 0x7fcdd80cb920 2023-10-21T17:19:20.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.562+0000 7fcdcf7fe640 1 -- 172.21.15.53:0/1740156442 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fcdc0045020 con 0x7fcdd80cb920 2023-10-21T17:19:20.564 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:20.564 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2103860266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2103860266},{"type":"v1","addr":"172.21.15.53:6835","nonce":2103860266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3143408071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3143408071},{"type":"v1","addr":"172.21.15.124:6837","nonce":3143408071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1417007243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1417007243},{"type":"v1","addr":"172.21.15.124:6835","nonce":1417007243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88},{"gid":9636,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1521573551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1521573551},{"type":"v1","addr":"172.21.15.53:6834","nonce":1521573551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[]} 2023-10-21T17:19:20.565 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 88 2023-10-21T17:19:20.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.565+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 msgr2=0x7fcdb806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.565+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 0x7fcdb806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8003b60 tx=0x7fcdc800b040 comp rx=0 tx=0).stop 2023-10-21T17:19:20.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.565+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 msgr2=0x7fcdd80cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.565+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c230 tx=0x7fcdc0001930 comp rx=0 tx=0).stop 2023-10-21T17:19:20.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 shutdown_connections 2023-10-21T17:19:20.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcdd809b620 0x7fcdd809baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fcdb806afc0 0x7fcdb806d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcdd80cb920 0x7fcdd80cbda0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 --2- 172.21.15.53:0/1740156442 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcdc40047c0 0x7fcdd809aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 >> 172.21.15.53:0/1740156442 conn(0x7fcdd8099b60 msgr2=0x7fcdd80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:20.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 shutdown_connections 2023-10-21T17:19:20.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.566+0000 7fcdd7fff640 1 -- 172.21.15.53:0/1740156442 wait complete. 2023-10-21T17:19:20.587 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:19:20.588 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:19:20.588 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:19:20.588 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:19:20.589 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:19:20.589 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:19:20.589 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:19:20.589 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:19:20.590 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:19:20.590 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:19:20.590 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:19:20.591 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:19:20.591 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:19:20.591 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:19:20.591 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:19:20.592 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:19:20.604 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:19:20.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:19:20.606 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:19:20.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:19:20.610 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:19:20.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:19:20.614 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:19:20.614 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:19:20.654 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:19:20.654 DEBUG:teuthology.parallel:result is None 2023-10-21T17:19:20.660 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:19:20.661 DEBUG:teuthology.parallel:result is None 2023-10-21T17:19:20.662 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:19:20.663 DEBUG:teuthology.parallel:result is None 2023-10-21T17:19:20.670 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:19:20.670 DEBUG:teuthology.parallel:result is None 2023-10-21T17:19:20.671 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:19:20.819 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:19:20.820 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:19:20.825 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:19:20.829 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:19:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.863+0000 7f4bfc200640 1 Processor -- start 2023-10-21T17:19:20.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.863+0000 7f4bfc200640 1 -- start start 2023-10-21T17:19:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.864+0000 7f4bfc200640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4bf40cb130 0x7f4bf40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.864+0000 7f4bfc200640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40b1cc0 con 0x7f4bf40afbe0 2023-10-21T17:19:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.864+0000 7f4bfc200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40cbad0 con 0x7f4bf406dfc0 2023-10-21T17:19:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.864+0000 7f4bfc200640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40b2620 con 0x7f4bf40cb130 2023-10-21T17:19:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.864+0000 7f4bfa79d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4bf40cb130 0x7f4bf40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.865+0000 7f4bfa79d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4bf40cb130 0x7f4bf40cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59626/0 (socket says 172.21.15.53:59626) 2023-10-21T17:19:20.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.865+0000 7f4bfa79d640 1 -- 172.21.15.53:0/702892065 learned_addr learned my addr 172.21.15.53:0/702892065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:20.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.865+0000 7f4bfa79d640 1 -- 172.21.15.53:0/702892065 >> v1:172.21.15.124:6789/0 conn(0x7f4bf406dfc0 legacy=0x7f4bf4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.865+0000 7f4bfa79d640 1 -- 172.21.15.53:0/702892065 >> v1:172.21.15.53:6789/0 conn(0x7f4bf40afbe0 legacy=0x7f4bf40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.865+0000 7f4bfa79d640 1 -- 172.21.15.53:0/702892065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40b2060 con 0x7f4bf40cb130 2023-10-21T17:19:20.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.866+0000 7f4bfa79d640 1 --2- 172.21.15.53:0/702892065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf40cb130 0x7f4bf40cb590 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009e50 tx=0x7f4bec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4f8ec9b9147b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.866+0000 7f4bf8f9a640 1 -- 172.21.15.53:0/702892065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec02cba0 con 0x7f4bf40cb130 2023-10-21T17:19:20.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.866+0000 7f4bf8f9a640 1 -- 172.21.15.53:0/702892065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec02cd00 con 0x7f4bf40cb130 2023-10-21T17:19:20.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.866+0000 7f4bf8f9a640 1 -- 172.21.15.53:0/702892065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec034610 con 0x7f4bf40cb130 2023-10-21T17:19:20.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 -- 172.21.15.53:0/702892065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf40cb130 msgr2=0x7f4bf40cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 --2- 172.21.15.53:0/702892065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf40cb130 0x7f4bf40cb590 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009e50 tx=0x7f4bec02c000 comp rx=0 tx=0).stop 2023-10-21T17:19:20.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 -- 172.21.15.53:0/702892065 shutdown_connections 2023-10-21T17:19:20.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 --2- 172.21.15.53:0/702892065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf40cb130 0x7f4bf40cb590 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 -- 172.21.15.53:0/702892065 >> 172.21.15.53:0/702892065 conn(0x7f4bf406a5d0 msgr2=0x7f4bf40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:20.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 -- 172.21.15.53:0/702892065 shutdown_connections 2023-10-21T17:19:20.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.867+0000 7f4bfc200640 1 -- 172.21.15.53:0/702892065 wait complete. 2023-10-21T17:19:20.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.869+0000 7f4bfc200640 1 Processor -- start 2023-10-21T17:19:20.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.869+0000 7f4bfc200640 1 -- start start 2023-10-21T17:19:20.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bf40afbe0 0x7f4bf40c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bf40cb130 0x7f4bf40c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40ce620 con 0x7f4bf406dfc0 2023-10-21T17:19:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40ce900 con 0x7f4bf40cb130 2023-10-21T17:19:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfc200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cebe0 con 0x7f4bf40afbe0 2023-10-21T17:19:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bf9f9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bf9f9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59638/0 (socket says 172.21.15.53:59638) 2023-10-21T17:19:20.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bf9f9c640 1 -- 172.21.15.53:0/3718253253 learned_addr learned my addr 172.21.15.53:0/3718253253 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:20.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bf979b640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bf40afbe0 0x7f4bf40c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.870+0000 7f4bfa79d640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bf40cb130 0x7f4bf40c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 -- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bf40afbe0 msgr2=0x7f4bf40c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bf40afbe0 0x7f4bf40c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 -- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bf40cb130 msgr2=0x7f4bf40c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bf40cb130 0x7f4bf40c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 -- 172.21.15.53:0/3718253253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec009b30 con 0x7f4bf406dfc0 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bf9f9c640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000c450 tx=0x7f4bf000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00052d0 con 0x7f4bf406dfc0 2023-10-21T17:19:20.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0005430 con 0x7f4bf406dfc0 2023-10-21T17:19:20.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0012870 con 0x7f4bf406dfc0 2023-10-21T17:19:20.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.871+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf40ceec0 con 0x7f4bf406dfc0 2023-10-21T17:19:20.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.872+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf41521c0 con 0x7f4bf406dfc0 2023-10-21T17:19:20.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.873+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf404e160 con 0x7f4bf406dfc0 2023-10-21T17:19:20.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.874+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4bf00129d0 con 0x7f4bf406dfc0 2023-10-21T17:19:20.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.875+0000 7f4bea7fc640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 0x7f4bd806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:20.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.875+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f4bf0089330 con 0x7f4bf406dfc0 2023-10-21T17:19:20.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.875+0000 7f4bf979b640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 0x7f4bd806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:20.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.876+0000 7f4bf979b640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 0x7f4bd806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4bf415e060 tx=0x7f4be4009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:20.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:20.879+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bf0054210 con 0x7f4bf406dfc0 2023-10-21T17:19:21.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.087+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4bf40c7fa0 con 0x7f4bf406dfc0 2023-10-21T17:19:21.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.570+0000 7f4bea7fc640 1 -- 172.21.15.53:0/3718253253 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v191) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4bf0057830 con 0x7f4bf406dfc0 2023-10-21T17:19:21.572 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:19:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 msgr2=0x7f4bd806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 0x7f4bd806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4bf415e060 tx=0x7f4be4009040 comp rx=0 tx=0).stop 2023-10-21T17:19:21.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 msgr2=0x7f4bf4151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000c450 tx=0x7f4bf000c910 comp rx=0 tx=0).stop 2023-10-21T17:19:21.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 shutdown_connections 2023-10-21T17:19:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bf40cb130 0x7f4bf40c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bf40afbe0 0x7f4bf40c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f4bd806af90 0x7f4bd806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 --2- 172.21.15.53:0/3718253253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4bf406dfc0 0x7f4bf4151c80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.574+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 >> 172.21.15.53:0/3718253253 conn(0x7f4bf406a5d0 msgr2=0x7f4bf40c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.575+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 shutdown_connections 2023-10-21T17:19:21.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.575+0000 7f4bfc200640 1 -- 172.21.15.53:0/3718253253 wait complete. 2023-10-21T17:19:21.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:19:21.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.819+0000 7fa22ea27640 1 Processor -- start 2023-10-21T17:19:21.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.819+0000 7fa22ea27640 1 -- start start 2023-10-21T17:19:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22ea27640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa22815ab30 0x7fa22815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22ea27640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280b07d0 0x7fa22806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22ea27640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22815f7d0 con 0x7fa2280adc70 2023-10-21T17:19:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22ea27640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22815f210 con 0x7fa22815ab30 2023-10-21T17:19:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22ea27640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815f4f0 con 0x7fa2280b07d0 2023-10-21T17:19:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22da25640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa22815ab30 0x7fa22815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22d224640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa2280adc70 0x7fa2280b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35422/0 (socket says 172.21.15.53:35422) 2023-10-21T17:19:21.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.821+0000 7fa22d224640 1 -- 172.21.15.53:0/226051397 learned_addr learned my addr 172.21.15.53:0/226051397 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22e226640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280b07d0 0x7fa22806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:21.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22da25640 1 -- 172.21.15.53:0/226051397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280b07d0 msgr2=0x7fa22806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22ca23640 1 -- 172.21.15.53:0/226051397 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536471403 0 0) 0x7fa22815f7d0 con 0x7fa2280adc70 2023-10-21T17:19:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22da25640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280b07d0 0x7fa22806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22da25640 1 -- 172.21.15.53:0/226051397 >> v1:172.21.15.53:6789/0 conn(0x7fa2280adc70 legacy=0x7fa2280b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.822+0000 7fa22da25640 1 -- 172.21.15.53:0/226051397 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22806c550 con 0x7fa22815ab30 2023-10-21T17:19:21.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.823+0000 7fa22da25640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa22815ab30 0x7fa22815e8a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa22402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d2242cb74495c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ca23640 1 -- 172.21.15.53:0/226051397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224031830 con 0x7fa22815ab30 2023-10-21T17:19:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ca23640 1 -- 172.21.15.53:0/226051397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224031990 con 0x7fa22815ab30 2023-10-21T17:19:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ca23640 1 -- 172.21.15.53:0/226051397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224031d20 con 0x7fa22815ab30 2023-10-21T17:19:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 -- 172.21.15.53:0/226051397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa22815ab30 msgr2=0x7fa22815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa22815ab30 0x7fa22815e8a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097a0 tx=0x7fa22402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 -- 172.21.15.53:0/226051397 shutdown_connections 2023-10-21T17:19:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280b07d0 0x7fa22806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 --2- 172.21.15.53:0/226051397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa22815ab30 0x7fa22815e8a0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 -- 172.21.15.53:0/226051397 >> 172.21.15.53:0/226051397 conn(0x7fa2280cb0c0 msgr2=0x7fa2280cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:21.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 -- 172.21.15.53:0/226051397 shutdown_connections 2023-10-21T17:19:21.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.824+0000 7fa22ea27640 1 -- 172.21.15.53:0/226051397 wait complete. 2023-10-21T17:19:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.826+0000 7fa22ea27640 1 Processor -- start 2023-10-21T17:19:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.826+0000 7fa22ea27640 1 -- start start 2023-10-21T17:19:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 0x7fa22815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa22815ab30 0x7fa22815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815be60 con 0x7fa2280b07d0 2023-10-21T17:19:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281f0480 con 0x7fa22815ab30 2023-10-21T17:19:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.827+0000 7fa22ea27640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2281500c0 con 0x7fa2280adc70 2023-10-21T17:19:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:21.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59658/0 (socket says 172.21.15.53:59658) 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 -- 172.21.15.53:0/1948233780 learned_addr learned my addr 172.21.15.53:0/1948233780 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22da25640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 0x7fa22815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 -- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 msgr2=0x7fa22815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 0x7fa22815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 -- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa22815ab30 msgr2=0x7fa22815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:21.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa22815ab30 0x7fa22815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 -- 172.21.15.53:0/1948233780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224009400 con 0x7fa2280b07d0 2023-10-21T17:19:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22da25640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 0x7fa22815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.828+0000 7fa22d224640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c4d0 tx=0x7fa21800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.829+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180074e0 con 0x7fa2280b07d0 2023-10-21T17:19:21.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.829+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218007660 con 0x7fa2280b07d0 2023-10-21T17:19:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.829+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218004a20 con 0x7fa2280b07d0 2023-10-21T17:19:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.829+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2281503a0 con 0x7fa2280b07d0 2023-10-21T17:19:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.829+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa228150960 con 0x7fa2280b07d0 2023-10-21T17:19:21.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.830+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22804e160 con 0x7fa2280b07d0 2023-10-21T17:19:21.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.831+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa218004b80 con 0x7fa2280b07d0 2023-10-21T17:19:21.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.832+0000 7fa21e7fc640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 0x7fa20c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.832+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fa218088830 con 0x7fa2280b07d0 2023-10-21T17:19:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.832+0000 7fa22da25640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 0x7fa20c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.834+0000 7fa22da25640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 0x7fa20c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403c040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:21.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:21.836+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2180538d0 con 0x7fa2280b07d0 2023-10-21T17:19:22.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.090+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa2280ada50 con 0x7fa2280b07d0 2023-10-21T17:19:22.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.092+0000 7fa21e7fc640 1 -- 172.21.15.53:0/1948233780 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa218018090 con 0x7fa2280b07d0 2023-10-21T17:19:22.093 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:22.093 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:19:22.093 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:19:22.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.094 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:19:22.094 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:19:22.094 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:19:22.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:19:22.095 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.096 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:19:22.096 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:19:22.096 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:19:22.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:19:22.097 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.098 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:19:22.098 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:19:22.098 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.100 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:19:22.100 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:19:22.100 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:19:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:19:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:19:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:19:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:19:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:19:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:19:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:19:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:19:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:19:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:19:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:19:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:19:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:19:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:19:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:19:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:19:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:19:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:19:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:19:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:19:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:19:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:19:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:19:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:19:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:19:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:19:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:19:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:19:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:19:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:19:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:19:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:19:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:19:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:19:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:19:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:19:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:19:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:19:22.128 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:19:22.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 msgr2=0x7fa20c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 0x7fa20c06d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403c040 comp rx=0 tx=0).stop 2023-10-21T17:19:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 msgr2=0x7fa2281eff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c4d0 tx=0x7fa21800c990 comp rx=0 tx=0).stop 2023-10-21T17:19:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 shutdown_connections 2023-10-21T17:19:22.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa22815ab30 0x7fa22815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fa20c06af90 0x7fa20c06d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2280b07d0 0x7fa2281eff40 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 --2- 172.21.15.53:0/1948233780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2280adc70 0x7fa22815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 >> 172.21.15.53:0/1948233780 conn(0x7fa2280cb0c0 msgr2=0x7fa2280700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:22.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.094+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 shutdown_connections 2023-10-21T17:19:22.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.095+0000 7fa22ea27640 1 -- 172.21.15.53:0/1948233780 wait complete. 2023-10-21T17:19:22.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:22.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.353+0000 7f34623a5640 1 Processor -- start 2023-10-21T17:19:22.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.353+0000 7f34623a5640 1 -- start start 2023-10-21T17:19:22.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.354+0000 7f34623a5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f345c1599d0 0x7f345c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.354+0000 7f34623a5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f345c0b07d0 0x7f345c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.354+0000 7f34623a5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c0cacf0 con 0x7f345c0cc210 2023-10-21T17:19:22.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f34623a5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0b10f0 con 0x7f345c0b07d0 2023-10-21T17:19:22.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f34623a5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0ca3c0 con 0x7f345c1599d0 2023-10-21T17:19:22.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f345c0b07d0 0x7f345c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f345c0b07d0 0x7f345c06c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59672/0 (socket says 172.21.15.53:59672) 2023-10-21T17:19:22.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 -- 172.21.15.53:0/2373585250 learned_addr learned my addr 172.21.15.53:0/2373585250 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:22.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3460ba2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f345c0cc210 0x7f345c0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35434/0 (socket says 172.21.15.53:35434) 2023-10-21T17:19:22.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f34613a3640 1 --2- 172.21.15.53:0/2373585250 >> v2:172.21.15.124:3300/0 conn(0x7f345c1599d0 0x7f345c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 -- 172.21.15.53:0/2373585250 >> v2:172.21.15.124:3300/0 conn(0x7f345c1599d0 msgr2=0x7f345c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:22.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 --2- 172.21.15.53:0/2373585250 >> v2:172.21.15.124:3300/0 conn(0x7f345c1599d0 0x7f345c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 -- 172.21.15.53:0/2373585250 >> v1:172.21.15.53:6789/0 conn(0x7f345c0cc210 legacy=0x7f345c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.355+0000 7f3461ba4640 1 -- 172.21.15.53:0/2373585250 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0ca6a0 con 0x7f345c0b07d0 2023-10-21T17:19:22.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.356+0000 7f3461ba4640 1 --2- 172.21.15.53:0/2373585250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c0b07d0 0x7f345c06c310 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f345c071350 tx=0x7f345402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d4b47c49db4a3d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f3453fff640 1 -- 172.21.15.53:0/2373585250 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345402cb10 con 0x7f345c0b07d0 2023-10-21T17:19:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f3453fff640 1 -- 172.21.15.53:0/2373585250 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345402cc70 con 0x7f345c0b07d0 2023-10-21T17:19:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f3453fff640 1 -- 172.21.15.53:0/2373585250 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540343f0 con 0x7f345c0b07d0 2023-10-21T17:19:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f34623a5640 1 -- 172.21.15.53:0/2373585250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c0b07d0 msgr2=0x7f345c06c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f34623a5640 1 --2- 172.21.15.53:0/2373585250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c0b07d0 0x7f345c06c310 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f345c071350 tx=0x7f345402bf70 comp rx=0 tx=0).stop 2023-10-21T17:19:22.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f34623a5640 1 -- 172.21.15.53:0/2373585250 shutdown_connections 2023-10-21T17:19:22.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.357+0000 7f34623a5640 1 --2- 172.21.15.53:0/2373585250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c0b07d0 0x7f345c06c310 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.358+0000 7f34623a5640 1 --2- 172.21.15.53:0/2373585250 >> v2:172.21.15.124:3300/0 conn(0x7f345c1599d0 0x7f345c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.358+0000 7f34623a5640 1 -- 172.21.15.53:0/2373585250 >> 172.21.15.53:0/2373585250 conn(0x7f345c0cb0c0 msgr2=0x7f345c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:22.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.358+0000 7f34623a5640 1 -- 172.21.15.53:0/2373585250 shutdown_connections 2023-10-21T17:19:22.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.358+0000 7f34623a5640 1 -- 172.21.15.53:0/2373585250 wait complete. 2023-10-21T17:19:22.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.359+0000 7f34623a5640 1 Processor -- start 2023-10-21T17:19:22.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 -- start start 2023-10-21T17:19:22.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0b07d0 0x7f345c1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345c0cc210 0x7f345c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f6080 con 0x7f345c1599d0 2023-10-21T17:19:22.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c8320 con 0x7f345c0b07d0 2023-10-21T17:19:22.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.360+0000 7f34623a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f5a30 con 0x7f345c0cc210 2023-10-21T17:19:22.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59684/0 (socket says 172.21.15.53:59684) 2023-10-21T17:19:22.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 -- 172.21.15.53:0/680349725 learned_addr learned my addr 172.21.15.53:0/680349725 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:22.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 -- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345c0cc210 msgr2=0x7f345c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345c0cc210 0x7f345c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 -- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0b07d0 msgr2=0x7f345c1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0b07d0 0x7f345c1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 -- 172.21.15.53:0/680349725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454009b30 con 0x7f345c1599d0 2023-10-21T17:19:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.361+0000 7f3461ba4640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f3454036040 tx=0x7f3454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.362+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454009510 con 0x7f345c1599d0 2023-10-21T17:19:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.362+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345403e450 con 0x7f345c1599d0 2023-10-21T17:19:22.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.362+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c1f5d10 con 0x7f345c1599d0 2023-10-21T17:19:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.362+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345403e800 con 0x7f345c1599d0 2023-10-21T17:19:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.362+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c0c7f50 con 0x7f345c1599d0 2023-10-21T17:19:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.363+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c04e160 con 0x7f345c1599d0 2023-10-21T17:19:22.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.364+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f345403e960 con 0x7f345c1599d0 2023-10-21T17:19:22.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.365+0000 7f3451ffb640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 0x7f344006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.365+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f34540ac410 con 0x7f345c1599d0 2023-10-21T17:19:22.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.366+0000 7f34613a3640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 0x7f344006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.367+0000 7f34613a3640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 0x7f344006d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.369+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34540774b0 con 0x7f345c1599d0 2023-10-21T17:19:22.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.567+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f345c1f61c0 con 0x7f345c1599d0 2023-10-21T17:19:22.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.570+0000 7f3451ffb640 1 -- 172.21.15.53:0/680349725 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f34540aae60 con 0x7f345c1599d0 2023-10-21T17:19:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:22.572 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":193,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:21.585056+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:22.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 msgr2=0x7f344006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 0x7f344006d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3458006fd0 tx=0x7f3458008040 comp rx=0 tx=0).stop 2023-10-21T17:19:22.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 msgr2=0x7f345c0c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f3454036040 tx=0x7f3454004a40 comp rx=0 tx=0).stop 2023-10-21T17:19:22.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 shutdown_connections 2023-10-21T17:19:22.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f344006afc0 0x7f344006d480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f345c1599d0 0x7f345c0c5ec0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345c0cc210 0x7f345c0c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 --2- 172.21.15.53:0/680349725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0b07d0 0x7f345c1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 >> 172.21.15.53:0/680349725 conn(0x7f345c0cb0c0 msgr2=0x7f345c158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 shutdown_connections 2023-10-21T17:19:22.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.573+0000 7f34623a5640 1 -- 172.21.15.53:0/680349725 wait complete. 2023-10-21T17:19:22.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:19:22.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:19:22.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.808+0000 7fbbf83d5640 1 Processor -- start 2023-10-21T17:19:22.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.808+0000 7fbbf83d5640 1 -- start start 2023-10-21T17:19:22.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf83d5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbf015aad0 0x7fbbf00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf83d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf83d5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf015bf30 con 0x7fbbf006cd00 2023-10-21T17:19:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf83d5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf015b320 con 0x7fbbf015aad0 2023-10-21T17:19:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf83d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf015b600 con 0x7fbbf00c82b0 2023-10-21T17:19:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.809+0000 7fbbf6171640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbf015aad0 0x7fbbf00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbf015aad0 0x7fbbf00b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59692/0 (socket says 172.21.15.53:59692) 2023-10-21T17:19:22.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 -- 172.21.15.53:0/1876498160 learned_addr learned my addr 172.21.15.53:0/1876498160 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:22.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf5970640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 -- 172.21.15.53:0/1876498160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 msgr2=0x7fbbf00c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf00c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 -- 172.21.15.53:0/1876498160 >> v1:172.21.15.124:6789/0 conn(0x7fbbf006cd00 legacy=0x7fbbf0070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf6171640 1 -- 172.21.15.53:0/1876498160 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf015b8e0 con 0x7fbbf015aad0 2023-10-21T17:19:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.810+0000 7fbbf5970640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:22.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.811+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1750 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210c881ef7479983 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.811+0000 7fbbf516f640 1 -- 172.21.15.53:0/1876498160 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec02ca80 con 0x7fbbf015aad0 2023-10-21T17:19:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf516f640 1 -- 172.21.15.53:0/1876498160 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec02cbe0 con 0x7fbbf015aad0 2023-10-21T17:19:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf516f640 1 -- 172.21.15.53:0/1876498160 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0343f0 con 0x7fbbf015aad0 2023-10-21T17:19:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1876498160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 msgr2=0x7fbbf00b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1750 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec02bee0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1876498160 shutdown_connections 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1876498160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1750 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1876498160 >> 172.21.15.53:0/1876498160 conn(0x7fbbf0069d50 msgr2=0x7fbbf006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1876498160 shutdown_connections 2023-10-21T17:19:22.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.812+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1876498160 wait complete. 2023-10-21T17:19:22.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.814+0000 7fbbf83d5640 1 Processor -- start 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.814+0000 7fbbf83d5640 1 -- start start 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 0x7fbbf00b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0147c20 con 0x7fbbf015aad0 2023-10-21T17:19:22.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0159d20 con 0x7fbbf006cd00 2023-10-21T17:19:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.815+0000 7fbbf83d5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0158e40 con 0x7fbbf00c82b0 2023-10-21T17:19:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59700/0 (socket says 172.21.15.53:59700) 2023-10-21T17:19:22.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 -- 172.21.15.53:0/1113193352 learned_addr learned my addr 172.21.15.53:0/1113193352 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf5970640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 0x7fbbf00b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 -- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 msgr2=0x7fbbf0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf0158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 -- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 msgr2=0x7fbbf00b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:22.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 0x7fbbf00b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:22.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 -- 172.21.15.53:0/1113193352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec009b30 con 0x7fbbf015aad0 2023-10-21T17:19:22.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 0x7fbbf00b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:22.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.816+0000 7fbbf6972640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c410 tx=0x7fbbe800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.817+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8011790 con 0x7fbbf015aad0 2023-10-21T17:19:22.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.817+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe80118f0 con 0x7fbbf015aad0 2023-10-21T17:19:22.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.817+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8011ce0 con 0x7fbbf015aad0 2023-10-21T17:19:22.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.817+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf0159110 con 0x7fbbf015aad0 2023-10-21T17:19:22.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.817+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf01596d0 con 0x7fbbf015aad0 2023-10-21T17:19:22.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.818+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf004e160 con 0x7fbbf015aad0 2023-10-21T17:19:22.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.819+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbe801c460 con 0x7fbbf015aad0 2023-10-21T17:19:22.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.820+0000 7fbbe6ffd640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 0x7fbbd406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:22.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.820+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fbbe8089390 con 0x7fbbf015aad0 2023-10-21T17:19:22.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.821+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 0x7fbbd406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:22.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.822+0000 7fbbf6171640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 0x7fbbd406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbec035000 tx=0x7fbbec030000 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:22.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:22.825+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbe8054430 con 0x7fbbf015aad0 2023-10-21T17:19:23.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.017+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbf0159e60 con 0x7fbbf015aad0 2023-10-21T17:19:23.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.018+0000 7fbbe6ffd640 1 -- 172.21.15.53:0/1113193352 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbe8057a50 con 0x7fbbf015aad0 2023-10-21T17:19:23.019 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:19:23.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 msgr2=0x7fbbd406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 0x7fbbd406d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbbec035000 tx=0x7fbbec030000 comp rx=0 tx=0).stop 2023-10-21T17:19:23.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 msgr2=0x7fbbf00b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c410 tx=0x7fbbe800c8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 shutdown_connections 2023-10-21T17:19:23.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbd406af90 0x7fbbd406d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbf015aad0 0x7fbbf00b1540 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbf00c82b0 0x7fbbf0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 --2- 172.21.15.53:0/1113193352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbf006cd00 0x7fbbf00b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 >> 172.21.15.53:0/1113193352 conn(0x7fbbf0069d50 msgr2=0x7fbbf0154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 shutdown_connections 2023-10-21T17:19:23.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.021+0000 7fbbf83d5640 1 -- 172.21.15.53:0/1113193352 wait complete. 2023-10-21T17:19:23.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:19:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.255+0000 7fbbce347640 1 Processor -- start 2023-10-21T17:19:23.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.255+0000 7fbbce347640 1 -- start start 2023-10-21T17:19:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.256+0000 7fbbce347640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbc81599d0 0x7fbbc80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.256+0000 7fbbce347640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80b07d0 0x7fbbc806c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbce347640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80cacf0 con 0x7fbbc80cc210 2023-10-21T17:19:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbce347640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80b10f0 con 0x7fbbc81599d0 2023-10-21T17:19:23.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbce347640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80ca3c0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbcd345640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbc81599d0 0x7fbbc80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbccb44640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbbc80cc210 0x7fbbc80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:35436/0 (socket says 172.21.15.53:35436) 2023-10-21T17:19:23.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbccb44640 1 -- 172.21.15.53:0/4149143440 learned_addr learned my addr 172.21.15.53:0/4149143440 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:23.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbcdb46640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80b07d0 0x7fbbc806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbcd345640 1 -- 172.21.15.53:0/4149143440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80b07d0 msgr2=0x7fbbc806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:23.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbcd345640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80b07d0 0x7fbbc806c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.257+0000 7fbbcd345640 1 -- 172.21.15.53:0/4149143440 >> v1:172.21.15.53:6789/0 conn(0x7fbbc80cc210 legacy=0x7fbbc80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.258+0000 7fbbcd345640 1 -- 172.21.15.53:0/4149143440 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80ca6a0 con 0x7fbbc81599d0 2023-10-21T17:19:23.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.258+0000 7fbbbffff640 1 -- 172.21.15.53:0/4149143440 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733491904 0 0) 0x7fbbc80cacf0 con 0x7fbbc80cc210 2023-10-21T17:19:23.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.259+0000 7fbbcd345640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc81599d0 0x7fbbc80ad730 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40097a0 tx=0x7fbbc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ab841e26319057 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.259+0000 7fbbbffff640 1 -- 172.21.15.53:0/4149143440 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4031830 con 0x7fbbc81599d0 2023-10-21T17:19:23.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.259+0000 7fbbbffff640 1 -- 172.21.15.53:0/4149143440 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc4031990 con 0x7fbbc81599d0 2023-10-21T17:19:23.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.259+0000 7fbbbffff640 1 -- 172.21.15.53:0/4149143440 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4031d20 con 0x7fbbc81599d0 2023-10-21T17:19:23.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 -- 172.21.15.53:0/4149143440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc81599d0 msgr2=0x7fbbc80ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc81599d0 0x7fbbc80ad730 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40097a0 tx=0x7fbbc402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 -- 172.21.15.53:0/4149143440 shutdown_connections 2023-10-21T17:19:23.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80b07d0 0x7fbbc806c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 --2- 172.21.15.53:0/4149143440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc81599d0 0x7fbbc80ad730 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 -- 172.21.15.53:0/4149143440 >> 172.21.15.53:0/4149143440 conn(0x7fbbc80cb0c0 msgr2=0x7fbbc80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:23.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 -- 172.21.15.53:0/4149143440 shutdown_connections 2023-10-21T17:19:23.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.260+0000 7fbbce347640 1 -- 172.21.15.53:0/4149143440 wait complete. 2023-10-21T17:19:23.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.262+0000 7fbbce347640 1 Processor -- start 2023-10-21T17:19:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.262+0000 7fbbce347640 1 -- start start 2023-10-21T17:19:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80cc210 0x7fbbc8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbc81599d0 0x7fbbc815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815b1f0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815a5e0 con 0x7fbbc81599d0 2023-10-21T17:19:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbce347640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc815a8c0 con 0x7fbbc80cc210 2023-10-21T17:19:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbcd345640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbcd345640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59724/0 (socket says 172.21.15.53:59724) 2023-10-21T17:19:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.263+0000 7fbbcd345640 1 -- 172.21.15.53:0/352436304 learned_addr learned my addr 172.21.15.53:0/352436304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 -- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80cc210 msgr2=0x7fbbc8159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80cc210 0x7fbbc8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 -- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbc81599d0 msgr2=0x7fbbc815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbc81599d0 0x7fbbc815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 -- 172.21.15.53:0/352436304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009400 con 0x7fbbc80b07d0 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.264+0000 7fbbcd345640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4000c00 tx=0x7fbbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.265+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc403aa60 con 0x7fbbc80b07d0 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.265+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc403abc0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.265+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40443f0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.265+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc815aba0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.265+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc815a2f0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.266+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc804e160 con 0x7fbbc80b07d0 2023-10-21T17:19:23.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.267+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbc404d450 con 0x7fbbc80b07d0 2023-10-21T17:19:23.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.268+0000 7fbbbdffb640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 0x7fbbac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.268+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fbbc40ab8d0 con 0x7fbbc80b07d0 2023-10-21T17:19:23.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.268+0000 7fbbccb44640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 0x7fbbac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.269+0000 7fbbccb44640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 0x7fbbac06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8007fd0 tx=0x7fbbb8007920 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.272+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbc4076860 con 0x7fbbc80b07d0 2023-10-21T17:19:23.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.465+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbbc80ce850 con 0x7fbbc80b07d0 2023-10-21T17:19:23.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.466+0000 7fbbbdffb640 1 -- 172.21.15.53:0/352436304 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v193) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbbc4035020 con 0x7fbbc80b07d0 2023-10-21T17:19:23.467 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:23.467 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:19:23.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:23.469 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:19:23.469 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:19:23.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:23.470 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:19:23.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 msgr2=0x7fbbac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 0x7fbbac06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8007fd0 tx=0x7fbbb8007920 comp rx=0 tx=0).stop 2023-10-21T17:19:23.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 msgr2=0x7fbbc814ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4000c00 tx=0x7fbbc40046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 shutdown_connections 2023-10-21T17:19:23.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbc81599d0 0x7fbbc815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbc80cc210 0x7fbbc8159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fbbac06af90 0x7fbbac06d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 --2- 172.21.15.53:0/352436304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbc80b07d0 0x7fbbc814ab10 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 >> 172.21.15.53:0/352436304 conn(0x7fbbc80cb0c0 msgr2=0x7fbbc8158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:23.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 shutdown_connections 2023-10-21T17:19:23.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.468+0000 7fbbce347640 1 -- 172.21.15.53:0/352436304 wait complete. 2023-10-21T17:19:23.488 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:19:23.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:19:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.719+0000 7f0c8711a640 1 Processor -- start 2023-10-21T17:19:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.719+0000 7f0c8711a640 1 -- start start 2023-10-21T17:19:23.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.720+0000 7f0c8711a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c800c1710 0x7f0c800c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.720+0000 7f0c8711a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 0x7f0c800c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.720+0000 7f0c8711a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c801622f0 con 0x7f0c801657e0 2023-10-21T17:19:23.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.720+0000 7f0c8711a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c801625d0 con 0x7f0c800c1710 2023-10-21T17:19:23.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.720+0000 7f0c8711a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c80164d00 con 0x7f0c800c1af0 2023-10-21T17:19:23.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c800c1710 0x7f0c800c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c800c1710 0x7f0c800c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59730/0 (socket says 172.21.15.53:59730) 2023-10-21T17:19:23.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 -- 172.21.15.53:0/3549759256 learned_addr learned my addr 172.21.15.53:0/3549759256 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:23.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c85917640 1 --2- 172.21.15.53:0/3549759256 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 0x7f0c800c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 -- 172.21.15.53:0/3549759256 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 msgr2=0x7f0c800c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:23.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 --2- 172.21.15.53:0/3549759256 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 0x7f0c800c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 -- 172.21.15.53:0/3549759256 >> v1:172.21.15.124:6789/0 conn(0x7f0c801657e0 legacy=0x7f0c80161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:23.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c86118640 1 -- 172.21.15.53:0/3549759256 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80164fe0 con 0x7f0c800c1710 2023-10-21T17:19:23.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.721+0000 7f0c85917640 1 --2- 172.21.15.53:0/3549759256 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 0x7f0c800c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:23.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.722+0000 7f0c86118640 1 --2- 172.21.15.53:0/3549759256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1710 0x7f0c800c8d40 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0c80070020 tx=0x7f0c7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60af93b44b577973 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.723+0000 7f0c85116640 1 -- 172.21.15.53:0/3549759256 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c030860 con 0x7f0c800c1710 2023-10-21T17:19:23.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.723+0000 7f0c85116640 1 -- 172.21.15.53:0/3549759256 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c0309c0 con 0x7f0c800c1710 2023-10-21T17:19:23.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.723+0000 7f0c85116640 1 -- 172.21.15.53:0/3549759256 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c030d50 con 0x7f0c800c1710 2023-10-21T17:19:23.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.723+0000 7f0c8711a640 1 -- 172.21.15.53:0/3549759256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1710 msgr2=0x7f0c800c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.723+0000 7f0c8711a640 1 --2- 172.21.15.53:0/3549759256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1710 0x7f0c800c8d40 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0c80070020 tx=0x7f0c7c02b9a0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 -- 172.21.15.53:0/3549759256 shutdown_connections 2023-10-21T17:19:23.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 --2- 172.21.15.53:0/3549759256 >> v2:172.21.15.124:3300/0 conn(0x7f0c800c1af0 0x7f0c800c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 --2- 172.21.15.53:0/3549759256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1710 0x7f0c800c8d40 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 -- 172.21.15.53:0/3549759256 >> 172.21.15.53:0/3549759256 conn(0x7f0c8006c010 msgr2=0x7f0c8006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:23.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 -- 172.21.15.53:0/3549759256 shutdown_connections 2023-10-21T17:19:23.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.724+0000 7f0c8711a640 1 -- 172.21.15.53:0/3549759256 wait complete. 2023-10-21T17:19:23.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.725+0000 7f0c8711a640 1 Processor -- start 2023-10-21T17:19:23.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.726+0000 7f0c8711a640 1 -- start start 2023-10-21T17:19:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c800c1710 0x7f0c8015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c801657e0 0x7f0c800b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8015d150 con 0x7f0c800c1af0 2023-10-21T17:19:23.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c4f00 con 0x7f0c801657e0 2023-10-21T17:19:23.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c8711a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8015d8b0 con 0x7f0c800c1710 2023-10-21T17:19:23.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59740/0 (socket says 172.21.15.53:59740) 2023-10-21T17:19:23.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 -- 172.21.15.53:0/167214831 learned_addr learned my addr 172.21.15.53:0/167214831 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:23.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c86118640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c800c1710 0x7f0c8015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c86919640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c801657e0 0x7f0c800b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 -- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c800c1710 msgr2=0x7f0c8015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c800c1710 0x7f0c8015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 -- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c801657e0 msgr2=0x7f0c800b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:23.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c801657e0 0x7f0c800b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.727+0000 7f0c85917640 1 -- 172.21.15.53:0/167214831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0094d0 con 0x7f0c800c1af0 2023-10-21T17:19:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c85917640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c820 tx=0x7f0c7000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70004230 con 0x7f0c800c1af0 2023-10-21T17:19:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70004390 con 0x7f0c800c1af0 2023-10-21T17:19:23.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70007a70 con 0x7f0c800c1af0 2023-10-21T17:19:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c800c4940 con 0x7f0c800c1af0 2023-10-21T17:19:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.728+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8015d540 con 0x7f0c800c1af0 2023-10-21T17:19:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.729+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8015d3a0 con 0x7f0c800c1af0 2023-10-21T17:19:23.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.730+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0c70007bd0 con 0x7f0c800c1af0 2023-10-21T17:19:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.732+0000 7f0c76ffd640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 0x7f0c6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:23.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.732+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f0c70089680 con 0x7f0c800c1af0 2023-10-21T17:19:23.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.732+0000 7f0c86118640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 0x7f0c6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:23.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.733+0000 7f0c86118640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 0x7f0c6406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c036000 tx=0x7f0c7c008030 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:23.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.736+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c8015d3a0 con 0x7f0c800c1af0 2023-10-21T17:19:23.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:23.931+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0c8015d3a0 con 0x7f0c800c1af0 2023-10-21T17:19:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.583+0000 7f0c76ffd640 1 -- 172.21.15.53:0/167214831 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v194) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0c8015d3a0 con 0x7f0c800c1af0 2023-10-21T17:19:24.585 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:19:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 msgr2=0x7f0c6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 0x7f0c6406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c036000 tx=0x7f0c7c008030 comp rx=0 tx=0).stop 2023-10-21T17:19:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 msgr2=0x7f0c800b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c820 tx=0x7f0c7000cce0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 shutdown_connections 2023-10-21T17:19:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c801657e0 0x7f0c800b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f0c6406af90 0x7f0c6406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c800c1af0 0x7f0c800b2830 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 --2- 172.21.15.53:0/167214831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c800c1710 0x7f0c8015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 >> 172.21.15.53:0/167214831 conn(0x7f0c8006c010 msgr2=0x7f0c8006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:24.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 shutdown_connections 2023-10-21T17:19:24.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.587+0000 7f0c8711a640 1 -- 172.21.15.53:0/167214831 wait complete. 2023-10-21T17:19:24.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:19:24.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.822+0000 7f33c6901640 1 Processor -- start 2023-10-21T17:19:24.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.823+0000 7f33c6901640 1 -- start start 2023-10-21T17:19:24.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.824+0000 7f33c6901640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f33c0159d50 0x7f33c015e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:24.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.824+0000 7f33c6901640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c00b2890 con 0x7f33c00b16e0 2023-10-21T17:19:24.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.824+0000 7f33c6901640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c015d8b0 con 0x7f33c00cbc10 2023-10-21T17:19:24.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.824+0000 7f33c6901640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c00b1ff0 con 0x7f33c0159d50 2023-10-21T17:19:24.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f33c0159d50 0x7f33c015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:24.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bf7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f33c00b16e0 0x7f33c00cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40722/0 (socket says 172.21.15.53:40722) 2023-10-21T17:19:24.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bf7fe640 1 -- 172.21.15.53:0/4154524251 learned_addr learned my addr 172.21.15.53:0/4154524251 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:24.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33beffd640 1 -- 172.21.15.53:0/4154524251 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378609590 0 0) 0x7f33c00b2890 con 0x7f33c00b16e0 2023-10-21T17:19:24.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33beffd640 1 -- 172.21.15.53:0/4154524251 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a8003520 con 0x7f33c00b16e0 2023-10-21T17:19:24.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bffff640 1 -- 172.21.15.53:0/4154524251 >> v1:172.21.15.124:6789/0 conn(0x7f33c00cbc10 legacy=0x7f33c00c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:24.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bffff640 1 -- 172.21.15.53:0/4154524251 >> v1:172.21.15.53:6789/0 conn(0x7f33c00b16e0 legacy=0x7f33c00cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.825+0000 7f33bffff640 1 -- 172.21.15.53:0/4154524251 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00b22d0 con 0x7f33c0159d50 2023-10-21T17:19:24.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.827+0000 7f33bffff640 1 --2- 172.21.15.53:0/4154524251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c0159d50 0x7f33c015e300 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f33b00097a0 tx=0x7f33b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ead0ba72b00f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:24.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.827+0000 7f33beffd640 1 -- 172.21.15.53:0/4154524251 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0031830 con 0x7f33c0159d50 2023-10-21T17:19:24.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.827+0000 7f33beffd640 1 -- 172.21.15.53:0/4154524251 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0031990 con 0x7f33c0159d50 2023-10-21T17:19:24.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.827+0000 7f33beffd640 1 -- 172.21.15.53:0/4154524251 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0031d20 con 0x7f33c0159d50 2023-10-21T17:19:24.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 -- 172.21.15.53:0/4154524251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c0159d50 msgr2=0x7f33c015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 --2- 172.21.15.53:0/4154524251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c0159d50 0x7f33c015e300 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f33b00097a0 tx=0x7f33b002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 -- 172.21.15.53:0/4154524251 shutdown_connections 2023-10-21T17:19:24.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 --2- 172.21.15.53:0/4154524251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c0159d50 0x7f33c015e300 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 -- 172.21.15.53:0/4154524251 >> 172.21.15.53:0/4154524251 conn(0x7f33c006a140 msgr2=0x7f33c006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:24.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 -- 172.21.15.53:0/4154524251 shutdown_connections 2023-10-21T17:19:24.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.828+0000 7f33c6901640 1 -- 172.21.15.53:0/4154524251 wait complete. 2023-10-21T17:19:24.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.830+0000 7f33c6901640 1 Processor -- start 2023-10-21T17:19:24.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.830+0000 7f33c6901640 1 -- start start 2023-10-21T17:19:24.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:24.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 0x7f33c014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:24.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 0x7f33c0153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:24.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00c2b60 con 0x7f33c00b16e0 2023-10-21T17:19:24.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c014a930 con 0x7f33c0159d50 2023-10-21T17:19:24.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c6901640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0149ab0 con 0x7f33c00cbc10 2023-10-21T17:19:24.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:24.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34274/0 (socket says 172.21.15.53:34274) 2023-10-21T17:19:24.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 -- 172.21.15.53:0/475695141 learned_addr learned my addr 172.21.15.53:0/475695141 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:24.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bf7fe640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 0x7f33c014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:24.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 -- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 msgr2=0x7f33c014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 0x7f33c014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33bffff640 1 -- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 msgr2=0x7f33c0153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:24.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.831+0000 7f33c4e9e640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 0x7f33c0153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:24.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bffff640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 0x7f33c0153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:24.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bffff640 1 -- 172.21.15.53:0/475695141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b0009400 con 0x7f33c00b16e0 2023-10-21T17:19:24.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bf7fe640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 0x7f33c014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:24.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33c4e9e640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 0x7f33c0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:24.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bffff640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f33b002bee0 tx=0x7f33b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:24.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b003a8f0 con 0x7f33c00b16e0 2023-10-21T17:19:24.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b003aa50 con 0x7f33c00b16e0 2023-10-21T17:19:24.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c0152170 con 0x7f33c00b16e0 2023-10-21T17:19:24.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b003ae00 con 0x7f33c00b16e0 2023-10-21T17:19:24.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.832+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c014a000 con 0x7f33c00b16e0 2023-10-21T17:19:24.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.834+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c0071cb0 con 0x7f33c00b16e0 2023-10-21T17:19:24.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.835+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33b004d450 con 0x7f33c00b16e0 2023-10-21T17:19:24.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.836+0000 7f33bcff9640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 0x7f33ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:24.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.836+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f33b00ab810 con 0x7f33c00b16e0 2023-10-21T17:19:24.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.836+0000 7f33bf7fe640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 0x7f33ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:24.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.838+0000 7f33bf7fe640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 0x7f33ac06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33c0153820 tx=0x7f33b4009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:24.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:24.841+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33b00766c0 con 0x7f33c00b16e0 2023-10-21T17:19:25.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.034+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f33c004e240 con 0x7f33c00b16e0 2023-10-21T17:19:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.591+0000 7f33bcff9640 1 -- 172.21.15.53:0/475695141 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v195) v1 ==== 170+0+0 (secure 0 0 0) 0x7f33b0079ce0 con 0x7f33c00b16e0 2023-10-21T17:19:25.593 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:19:25.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 msgr2=0x7f33ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 0x7f33ac06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33c0153820 tx=0x7f33b4009040 comp rx=0 tx=0).stop 2023-10-21T17:19:25.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 msgr2=0x7f33c0151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f33b002bee0 tx=0x7f33b00046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 shutdown_connections 2023-10-21T17:19:25.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33c0159d50 0x7f33c0153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33c00cbc10 0x7f33c014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f33ac06af90 0x7f33ac06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 --2- 172.21.15.53:0/475695141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f33c00b16e0 0x7f33c0151c30 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.597+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 >> 172.21.15.53:0/475695141 conn(0x7f33c006a140 msgr2=0x7f33c00cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:25.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.599+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 shutdown_connections 2023-10-21T17:19:25.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.599+0000 7f33c6901640 1 -- 172.21.15.53:0/475695141 wait complete. 2023-10-21T17:19:25.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:19:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.824+0000 7f7d8dcb4640 1 Processor -- start 2023-10-21T17:19:25.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.825+0000 7f7d8dcb4640 1 -- start start 2023-10-21T17:19:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d8dcb4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7d880cb130 0x7f7d880cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d8dcb4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880b1cc0 con 0x7f7d880afbe0 2023-10-21T17:19:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d8dcb4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880cbad0 con 0x7f7d8806dfc0 2023-10-21T17:19:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d8dcb4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880b2620 con 0x7f7d880cb130 2023-10-21T17:19:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d877fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d880afbe0 0x7f7d880b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40728/0 (socket says 172.21.15.53:40728) 2023-10-21T17:19:25.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.826+0000 7f7d877fe640 1 -- 172.21.15.53:0/2732805058 learned_addr learned my addr 172.21.15.53:0/2732805058 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.827+0000 7f7d87fff640 1 --2- 172.21.15.53:0/2732805058 >> v2:172.21.15.124:3300/0 conn(0x7f7d880cb130 0x7f7d880cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.827+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206692587 0 0) 0x7f7d880b1cc0 con 0x7f7d880afbe0 2023-10-21T17:19:25.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.827+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d74003520 con 0x7f7d880afbe0 2023-10-21T17:19:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.827+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3429176106 0 0) 0x7f7d74003520 con 0x7f7d880afbe0 2023-10-21T17:19:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.827+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 >> v2:172.21.15.124:3300/0 conn(0x7f7d880cb130 msgr2=0x7f7d880cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 --2- 172.21.15.53:0/2732805058 >> v2:172.21.15.124:3300/0 conn(0x7f7d880cb130 0x7f7d880cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 >> v1:172.21.15.124:6789/0 conn(0x7f7d8806dfc0 legacy=0x7f7d88070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880b2060 con 0x7f7d880afbe0 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7d780019f0 con 0x7f7d880afbe0 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 >> v1:172.21.15.53:6789/0 conn(0x7f7d880afbe0 legacy=0x7f7d880b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 0x7f7d74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d740047c0 0x7f7d74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74005180 0x7f7d7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b2620 con 0x7f7d74003970 2023-10-21T17:19:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400c020 con 0x7f7d740047c0 2023-10-21T17:19:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7400c1e0 con 0x7f7d74005180 2023-10-21T17:19:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d877fe640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 0x7f7d74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d86ffd640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d740047c0 0x7f7d74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.828+0000 7f7d87fff640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74005180 0x7f7d7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d877fe640 1 -- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74005180 msgr2=0x7f7d7400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d877fe640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74005180 0x7f7d7400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d877fe640 1 -- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d740047c0 msgr2=0x7f7d74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d877fe640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d740047c0 0x7f7d74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d877fe640 1 -- 172.21.15.53:0/2732805058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74003520 con 0x7f7d74003970 2023-10-21T17:19:25.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.829+0000 7f7d87fff640 1 -- 172.21.15.53:0/2732805058 reap_dead start 2023-10-21T17:19:25.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.830+0000 7f7d877fe640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 0x7f7d74003d90 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800a830 tx=0x7f7d740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd56461480fb792 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:25.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.830+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780072d0 con 0x7f7d74003970 2023-10-21T17:19:25.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.830+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7802f070 con 0x7f7d74003970 2023-10-21T17:19:25.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.830+0000 7f7d867fc640 1 -- 172.21.15.53:0/2732805058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780076f0 con 0x7f7d74003970 2023-10-21T17:19:25.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 msgr2=0x7f7d74003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 0x7f7d74003d90 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800a830 tx=0x7f7d740057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/2732805058 shutdown_connections 2023-10-21T17:19:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/2732805058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d74003970 0x7f7d74003d90 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/2732805058 >> 172.21.15.53:0/2732805058 conn(0x7f7d8806a5d0 msgr2=0x7f7d880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/2732805058 shutdown_connections 2023-10-21T17:19:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.831+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/2732805058 wait complete. 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.833+0000 7f7d8dcb4640 1 Processor -- start 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.834+0000 7f7d8dcb4640 1 -- start start 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74003970 0x7f7d88151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d8815c6d0 0x7f7d880c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881f33a0 con 0x7f7d880c8790 2023-10-21T17:19:25.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881521c0 con 0x7f7d8815c6d0 2023-10-21T17:19:25.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d8dcb4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881f2a70 con 0x7f7d74003970 2023-10-21T17:19:25.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d87fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34282/0 (socket says 172.21.15.53:34282) 2023-10-21T17:19:25.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d87fff640 1 -- 172.21.15.53:0/1156121811 learned_addr learned my addr 172.21.15.53:0/1156121811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:25.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.835+0000 7f7d86ffd640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d8815c6d0 0x7f7d880c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d877fe640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74003970 0x7f7d88151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 -- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74003970 msgr2=0x7f7d88151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74003970 0x7f7d88151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 -- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d8815c6d0 msgr2=0x7f7d880c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d8815c6d0 0x7f7d880c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 -- 172.21.15.53:0/1156121811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d881f2d50 con 0x7f7d880c8790 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d87fff640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f7d880cb080 tx=0x7f7d80011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80004500 con 0x7f7d880c8790 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80003610 con 0x7f7d880c8790 2023-10-21T17:19:25.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800039e0 con 0x7f7d880c8790 2023-10-21T17:19:25.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.836+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d881f3030 con 0x7f7d880c8790 2023-10-21T17:19:25.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.837+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8815e090 con 0x7f7d880c8790 2023-10-21T17:19:25.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.838+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88070330 con 0x7f7d880c8790 2023-10-21T17:19:25.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.839+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7d80003b40 con 0x7f7d880c8790 2023-10-21T17:19:25.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.840+0000 7f7d7ffff640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 0x7f7d6806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:25.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.840+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f7d80089ef0 con 0x7f7d880c8790 2023-10-21T17:19:25.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.841+0000 7f7d877fe640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 0x7f7d6806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:25.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.842+0000 7f7d877fe640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 0x7f7d6806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d78034000 tx=0x7f7d78007000 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:25.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:25.843+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d80054dd0 con 0x7f7d880c8790 2023-10-21T17:19:26.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.076+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7d88070330 con 0x7f7d880c8790 2023-10-21T17:19:26.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.641+0000 7f7d7ffff640 1 -- 172.21.15.53:0/1156121811 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v91) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7d88070330 con 0x7f7d880c8790 2023-10-21T17:19:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 23 and data pool 24 2023-10-21T17:19:26.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.653+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 msgr2=0x7f7d6806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.653+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 0x7f7d6806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d78034000 tx=0x7f7d78007000 comp rx=0 tx=0).stop 2023-10-21T17:19:26.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.653+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 msgr2=0x7f7d880c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.653+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f7d880cb080 tx=0x7f7d80011920 comp rx=0 tx=0).stop 2023-10-21T17:19:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 shutdown_connections 2023-10-21T17:19:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7d6806afc0 0x7f7d6806d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d880c8790 0x7f7d880c4260 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d8815c6d0 0x7f7d880c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 --2- 172.21.15.53:0/1156121811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d74003970 0x7f7d88151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.654+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 >> 172.21.15.53:0/1156121811 conn(0x7f7d8806a5d0 msgr2=0x7f7d880cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:26.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.655+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 shutdown_connections 2023-10-21T17:19:26.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.660+0000 7f7d8dcb4640 1 -- 172.21.15.53:0/1156121811 wait complete. 2023-10-21T17:19:26.689 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:26.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.870+0000 7f7bbb59e640 1 Processor -- start 2023-10-21T17:19:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.871+0000 7f7bbb59e640 1 -- start start 2023-10-21T17:19:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bbb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7bbc0adc70 0x7f7bbc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bbb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc15f7d0 con 0x7f7bbc15ab30 2023-10-21T17:19:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bbb59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc15efb0 con 0x7f7bbc15af10 2023-10-21T17:19:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bbb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc15f210 con 0x7f7bbc0adc70 2023-10-21T17:19:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bba59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7bbc15ab30 0x7f7bbc15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40730/0 (socket says 172.21.15.53:40730) 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bba59c640 1 -- 172.21.15.53:0/3149737041 learned_addr learned my addr 172.21.15.53:0/3149737041 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.872+0000 7f7bb3fff640 1 --2- 172.21.15.53:0/3149737041 >> v2:172.21.15.53:3300/0 conn(0x7f7bbc0adc70 0x7f7bbc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.873+0000 7f7bb9d9b640 1 -- 172.21.15.53:0/3149737041 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712886915 0 0) 0x7f7bbc15f7d0 con 0x7f7bbc15ab30 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.873+0000 7f7bb9d9b640 1 -- 172.21.15.53:0/3149737041 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba8003520 con 0x7f7bbc15ab30 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.873+0000 7f7bb3fff640 1 -- 172.21.15.53:0/3149737041 >> v1:172.21.15.124:6789/0 conn(0x7f7bbc15af10 legacy=0x7f7bbc06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.873+0000 7f7bb3fff640 1 -- 172.21.15.53:0/3149737041 >> v1:172.21.15.53:6789/0 conn(0x7f7bbc15ab30 legacy=0x7f7bbc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.873+0000 7f7bb3fff640 1 -- 172.21.15.53:0/3149737041 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc15f4f0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.874+0000 7f7bb3fff640 1 --2- 172.21.15.53:0/3149737041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc0b00c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40097a0 tx=0x7f7ba402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5764c089d35885 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.874+0000 7f7bb9d9b640 1 -- 172.21.15.53:0/3149737041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4031830 con 0x7f7bbc0adc70 2023-10-21T17:19:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.874+0000 7f7bb9d9b640 1 -- 172.21.15.53:0/3149737041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4031990 con 0x7f7bbc0adc70 2023-10-21T17:19:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.874+0000 7f7bb9d9b640 1 -- 172.21.15.53:0/3149737041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4031d00 con 0x7f7bbc0adc70 2023-10-21T17:19:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 -- 172.21.15.53:0/3149737041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 msgr2=0x7f7bbc0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/3149737041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc0b00c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40097a0 tx=0x7f7ba402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 -- 172.21.15.53:0/3149737041 shutdown_connections 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/3149737041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc0b00c0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 -- 172.21.15.53:0/3149737041 >> 172.21.15.53:0/3149737041 conn(0x7f7bbc0cb0c0 msgr2=0x7f7bbc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 -- 172.21.15.53:0/3149737041 shutdown_connections 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.875+0000 7f7bbb59e640 1 -- 172.21.15.53:0/3149737041 wait complete. 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.876+0000 7f7bbb59e640 1 Processor -- start 2023-10-21T17:19:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 -- start start 2023-10-21T17:19:26.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:26.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bbc15ab30 0x7f7bbc156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bbc15af10 0x7f7bbc0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0ae9d0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0ae050 con 0x7f7bbc15af10 2023-10-21T17:19:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.877+0000 7f7bbb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1562b0 con 0x7f7bbc15ab30 2023-10-21T17:19:26.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34308/0 (socket says 172.21.15.53:34308) 2023-10-21T17:19:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 -- 172.21.15.53:0/2444657508 learned_addr learned my addr 172.21.15.53:0/2444657508 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 -- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bbc15ab30 msgr2=0x7f7bbc156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bbc15ab30 0x7f7bbc156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 -- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bbc15af10 msgr2=0x7f7bbc0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bbc15af10 0x7f7bbc0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 -- 172.21.15.53:0/2444657508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba4009400 con 0x7f7bbc0adc70 2023-10-21T17:19:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.878+0000 7f7bba59c640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0010e0 tx=0x7f7bac00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.879+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac005ce0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.879+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac005e60 con 0x7f7bbc0adc70 2023-10-21T17:19:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.879+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0072b0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.879+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc1f33c0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.879+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc0ae5f0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.880+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc0aeb10 con 0x7f7bbc0adc70 2023-10-21T17:19:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.881+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7bac007410 con 0x7f7bbc0adc70 2023-10-21T17:19:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.882+0000 7f7bb2ffd640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 0x7f7b9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.882+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7bac088fe0 con 0x7f7bbc0adc70 2023-10-21T17:19:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.882+0000 7f7bb3fff640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 0x7f7b9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.883+0000 7f7bb3fff640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 0x7f7b9c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4036000 tx=0x7f7ba4030000 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:26.886+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bbc0aeb10 con 0x7f7bbc0adc70 2023-10-21T17:19:27.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.072+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bbc0aeb10 con 0x7f7bbc0adc70 2023-10-21T17:19:27.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.075+0000 7f7bb2ffd640 1 -- 172.21.15.53:0/2444657508 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16903 (secure 0 0 0) 0x7f7bbc0aeb10 con 0x7f7bbc0adc70 2023-10-21T17:19:27.077 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:27.077 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":196,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:26.078648+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:27.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 msgr2=0x7f7b9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 0x7f7b9c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4036000 tx=0x7f7ba4030000 comp rx=0 tx=0).stop 2023-10-21T17:19:27.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 msgr2=0x7f7bbc155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0010e0 tx=0x7f7bac00c960 comp rx=0 tx=0).stop 2023-10-21T17:19:27.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 shutdown_connections 2023-10-21T17:19:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bbc15af10 0x7f7bbc0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bbc15ab30 0x7f7bbc156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7b9c06af90 0x7f7b9c06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 --2- 172.21.15.53:0/2444657508 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7bbc0adc70 0x7f7bbc155d70 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 >> 172.21.15.53:0/2444657508 conn(0x7f7bbc0cb0c0 msgr2=0x7f7bbc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:27.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 shutdown_connections 2023-10-21T17:19:27.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.078+0000 7f7bbb59e640 1 -- 172.21.15.53:0/2444657508 wait complete. 2023-10-21T17:19:27.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.320+0000 7f1aab3ff640 1 Processor -- start 2023-10-21T17:19:27.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.320+0000 7f1aab3ff640 1 -- start start 2023-10-21T17:19:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.321+0000 7f1aab3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aab3ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1aa40b07d0 0x7f1aa406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aab3ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa40cacf0 con 0x7f1aa40cc210 2023-10-21T17:19:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aab3ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa40b10f0 con 0x7f1aa40b07d0 2023-10-21T17:19:27.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aab3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40ca3c0 con 0x7f1aa41599d0 2023-10-21T17:19:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aaabfe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1aa40b07d0 0x7f1aa406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aaabfe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1aa40b07d0 0x7f1aa406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34318/0 (socket says 172.21.15.53:34318) 2023-10-21T17:19:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aaa3fd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.322+0000 7f1aaabfe640 1 -- 172.21.15.53:0/3725734814 learned_addr learned my addr 172.21.15.53:0/3725734814 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:27.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.323+0000 7f1aaabfe640 1 -- 172.21.15.53:0/3725734814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 msgr2=0x7f1aa40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:27.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.323+0000 7f1aaabfe640 1 --2- 172.21.15.53:0/3725734814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.323+0000 7f1aaabfe640 1 -- 172.21.15.53:0/3725734814 >> v1:172.21.15.124:6789/0 conn(0x7f1aa40cc210 legacy=0x7f1aa40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.323+0000 7f1aaabfe640 1 -- 172.21.15.53:0/3725734814 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40ca6a0 con 0x7f1aa40b07d0 2023-10-21T17:19:27.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aaabfe640 1 --2- 172.21.15.53:0/3725734814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40b07d0 0x7f1aa406c310 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009e50 tx=0x7f1a9c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb51140782b3642e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aa93fb640 1 -- 172.21.15.53:0/3725734814 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02cb00 con 0x7f1aa40b07d0 2023-10-21T17:19:27.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aa93fb640 1 -- 172.21.15.53:0/3725734814 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c02cc60 con 0x7f1aa40b07d0 2023-10-21T17:19:27.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aa93fb640 1 -- 172.21.15.53:0/3725734814 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c0343f0 con 0x7f1aa40b07d0 2023-10-21T17:19:27.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aab3ff640 1 -- 172.21.15.53:0/3725734814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40b07d0 msgr2=0x7f1aa406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.324+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/3725734814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40b07d0 0x7f1aa406c310 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009e50 tx=0x7f1a9c02bf60 comp rx=0 tx=0).stop 2023-10-21T17:19:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 -- 172.21.15.53:0/3725734814 shutdown_connections 2023-10-21T17:19:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/3725734814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40b07d0 0x7f1aa406c310 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/3725734814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 -- 172.21.15.53:0/3725734814 >> 172.21.15.53:0/3725734814 conn(0x7f1aa40cb0c0 msgr2=0x7f1aa40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:27.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 -- 172.21.15.53:0/3725734814 shutdown_connections 2023-10-21T17:19:27.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.325+0000 7f1aab3ff640 1 -- 172.21.15.53:0/3725734814 wait complete. 2023-10-21T17:19:27.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.326+0000 7f1aab3ff640 1 Processor -- start 2023-10-21T17:19:27.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 -- start start 2023-10-21T17:19:27.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1aa40b07d0 0x7f1aa4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa41561e0 con 0x7f1aa40cc210 2023-10-21T17:19:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415adb0 con 0x7f1aa40b07d0 2023-10-21T17:19:27.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.327+0000 7f1aab3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa415b090 con 0x7f1aa41599d0 2023-10-21T17:19:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34322/0 (socket says 172.21.15.53:34322) 2023-10-21T17:19:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 -- 172.21.15.53:0/1580356001 learned_addr learned my addr 172.21.15.53:0/1580356001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aaa3fd640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1aa40b07d0 0x7f1aa4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aaabfe640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 -- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 msgr2=0x7f1aa40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 -- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1aa40b07d0 msgr2=0x7f1aa4155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1aa40b07d0 0x7f1aa4155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.328+0000 7f1aa9bfc640 1 -- 172.21.15.53:0/1580356001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009b30 con 0x7f1aa40cc210 2023-10-21T17:19:27.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1aa9bfc640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400c400 tx=0x7f1a9400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a940052d0 con 0x7f1aa40cc210 2023-10-21T17:19:27.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a94005430 con 0x7f1aa40cc210 2023-10-21T17:19:27.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94003830 con 0x7f1aa40cc210 2023-10-21T17:19:27.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa415b370 con 0x7f1aa40cc210 2023-10-21T17:19:27.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.329+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa40a85e0 con 0x7f1aa40cc210 2023-10-21T17:19:27.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.330+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa404e160 con 0x7f1aa40cc210 2023-10-21T17:19:27.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.331+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1a94003990 con 0x7f1aa40cc210 2023-10-21T17:19:27.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.332+0000 7f1a9affd640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 0x7f1a8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.332+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1a94088f90 con 0x7f1aa40cc210 2023-10-21T17:19:27.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.333+0000 7f1aaa3fd640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 0x7f1a8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.334+0000 7f1aaa3fd640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 0x7f1a8806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40cd9a0 tx=0x7f1aa0008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.338+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a94053d60 con 0x7f1aa40cc210 2023-10-21T17:19:27.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.538+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa40a87d0 con 0x7f1aa40cc210 2023-10-21T17:19:27.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.540+0000 7f1a9affd640 1 -- 172.21.15.53:0/1580356001 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16903 (secure 0 0 0) 0x7f1a94017090 con 0x7f1aa40cc210 2023-10-21T17:19:27.542 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:27.542 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":196,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:26.078648+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:27.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 msgr2=0x7f1a8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 0x7f1a8806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40cd9a0 tx=0x7f1aa0008040 comp rx=0 tx=0).stop 2023-10-21T17:19:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 msgr2=0x7f1aa415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400c400 tx=0x7f1a9400c8c0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 shutdown_connections 2023-10-21T17:19:27.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1aa41599d0 0x7f1aa40cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f1a8806af90 0x7f1a8806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1aa40cc210 0x7f1aa415a310 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 --2- 172.21.15.53:0/1580356001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1aa40b07d0 0x7f1aa4155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.543+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 >> 172.21.15.53:0/1580356001 conn(0x7f1aa40cb0c0 msgr2=0x7f1aa40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:27.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.544+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 shutdown_connections 2023-10-21T17:19:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.544+0000 7f1aab3ff640 1 -- 172.21.15.53:0/1580356001 wait complete. 2023-10-21T17:19:27.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:19:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.779+0000 7f78decbb640 1 Processor -- start 2023-10-21T17:19:27.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.780+0000 7f78decbb640 1 -- start start 2023-10-21T17:19:27.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.781+0000 7f78decbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d815ccb0 0x7f78d80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.781+0000 7f78decbb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78d80d00e0 0x7f78d80b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.781+0000 7f78decbb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80d2870 con 0x7f78d80d0b90 2023-10-21T17:19:27.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.781+0000 7f78decbb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80d1150 con 0x7f78d80d00e0 2023-10-21T17:19:27.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.781+0000 7f78decbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80d1430 con 0x7f78d815ccb0 2023-10-21T17:19:27.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.782+0000 7f78de4ba640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78d80d00e0 0x7f78d80b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.782+0000 7f78ddcb9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d815ccb0 0x7f78d80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.782+0000 7f78dd4b8640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f78d80d0b90 0x7f78d80cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:37888/0 (socket says 172.21.15.53:37888) 2023-10-21T17:19:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.782+0000 7f78dd4b8640 1 -- 172.21.15.53:0/3132793290 learned_addr learned my addr 172.21.15.53:0/3132793290 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.783+0000 7f78de4ba640 1 -- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d815ccb0 msgr2=0x7f78d80d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.783+0000 7f78de4ba640 1 --2- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d815ccb0 0x7f78d80d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.783+0000 7f78de4ba640 1 -- 172.21.15.53:0/3132793290 >> v1:172.21.15.124:6789/0 conn(0x7f78d80d0b90 legacy=0x7f78d80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.783+0000 7f78de4ba640 1 -- 172.21.15.53:0/3132793290 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80d1710 con 0x7f78d80d00e0 2023-10-21T17:19:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.784+0000 7f78de4ba640 1 --2- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d80d00e0 0x7f78d80b6060 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22733640ea3cad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78dccb7640 1 -- 172.21.15.53:0/3132793290 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d002cc50 con 0x7f78d80d00e0 2023-10-21T17:19:27.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78dccb7640 1 -- 172.21.15.53:0/3132793290 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d002cdb0 con 0x7f78d80d00e0 2023-10-21T17:19:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78dccb7640 1 -- 172.21.15.53:0/3132793290 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0034660 con 0x7f78d80d00e0 2023-10-21T17:19:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 -- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d80d00e0 msgr2=0x7f78d80b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 --2- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d80d00e0 0x7f78d80b6060 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d90 tx=0x7f78d002bea0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 -- 172.21.15.53:0/3132793290 shutdown_connections 2023-10-21T17:19:27.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 --2- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d80d00e0 0x7f78d80b6060 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 --2- 172.21.15.53:0/3132793290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d815ccb0 0x7f78d80d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 -- 172.21.15.53:0/3132793290 >> 172.21.15.53:0/3132793290 conn(0x7f78d8071660 msgr2=0x7f78d8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 -- 172.21.15.53:0/3132793290 shutdown_connections 2023-10-21T17:19:27.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.785+0000 7f78decbb640 1 -- 172.21.15.53:0/3132793290 wait complete. 2023-10-21T17:19:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.787+0000 7f78decbb640 1 Processor -- start 2023-10-21T17:19:27.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.787+0000 7f78decbb640 1 -- start start 2023-10-21T17:19:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 0x7f78d80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d815ccb0 0x7f78d80d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8159550 con 0x7f78d80d0b90 2023-10-21T17:19:27.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d815f1d0 con 0x7f78d815ccb0 2023-10-21T17:19:27.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.788+0000 7f78decbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8158940 con 0x7f78d80d00e0 2023-10-21T17:19:27.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34338/0 (socket says 172.21.15.53:34338) 2023-10-21T17:19:27.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 -- 172.21.15.53:0/1561770876 learned_addr learned my addr 172.21.15.53:0/1561770876 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:27.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78de4ba640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d815ccb0 0x7f78d80d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78ddcb9640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 0x7f78d80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 -- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 msgr2=0x7f78d80c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 0x7f78d80c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 -- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d815ccb0 msgr2=0x7f78d80d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d815ccb0 0x7f78d80d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 -- 172.21.15.53:0/1561770876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009a70 con 0x7f78d80d0b90 2023-10-21T17:19:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78ddcb9640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 0x7f78d80c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.789+0000 7f78dd4b8640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c4a0 tx=0x7f78c800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.790+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8005ce0 con 0x7f78d80d0b90 2023-10-21T17:19:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.790+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c8005e40 con 0x7f78d80d0b90 2023-10-21T17:19:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.790+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8007270 con 0x7f78d80d0b90 2023-10-21T17:19:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.790+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d8158c20 con 0x7f78d80d0b90 2023-10-21T17:19:27.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.790+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d81591e0 con 0x7f78d80d0b90 2023-10-21T17:19:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.791+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8074150 con 0x7f78d80d0b90 2023-10-21T17:19:27.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.792+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f78c8007490 con 0x7f78d80d0b90 2023-10-21T17:19:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.793+0000 7f78ce7fc640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 0x7f78bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.793+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f78c8088e30 con 0x7f78d80d0b90 2023-10-21T17:19:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.793+0000 7f78ddcb9640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 0x7f78bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:27.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.795+0000 7f78ddcb9640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 0x7f78bc06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009a20 tx=0x7f78d4008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:27.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:27.797+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78c8053c20 con 0x7f78d80d0b90 2023-10-21T17:19:28.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.036+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f78d80b5ae0 con 0x7f78d80d0b90 2023-10-21T17:19:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.841+0000 7f78ce7fc640 1 -- 172.21.15.53:0/1561770876 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) v1 ==== 118+0+0 (secure 0 0 0) 0x7f78c8017020 con 0x7f78d80d0b90 2023-10-21T17:19:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 msgr2=0x7f78bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 0x7f78bc06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009a20 tx=0x7f78d4008040 comp rx=0 tx=0).stop 2023-10-21T17:19:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 msgr2=0x7f78d815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:28.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c4a0 tx=0x7f78c800c960 comp rx=0 tx=0).stop 2023-10-21T17:19:28.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 shutdown_connections 2023-10-21T17:19:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d815ccb0 0x7f78d80d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f78bc06af90 0x7f78bc06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78d80d0b90 0x7f78d815ec90 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 --2- 172.21.15.53:0/1561770876 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d80d00e0 0x7f78d80c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 >> 172.21.15.53:0/1561770876 conn(0x7f78d8071660 msgr2=0x7f78d815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 shutdown_connections 2023-10-21T17:19:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:28.845+0000 7f78decbb640 1 -- 172.21.15.53:0/1561770876 wait complete. 2023-10-21T17:19:28.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:19:28.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:19:29.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.104+0000 7ff36bda9640 1 Processor -- start 2023-10-21T17:19:29.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.104+0000 7ff36bda9640 1 -- start start 2023-10-21T17:19:29.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.105+0000 7ff36bda9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3640afbe0 0x7ff3640b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.105+0000 7ff36bda9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3640cb130 0x7ff3640cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.105+0000 7ff36bda9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3640cbad0 con 0x7ff36406dfc0 2023-10-21T17:19:29.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.105+0000 7ff36bda9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3640b2620 con 0x7ff3640afbe0 2023-10-21T17:19:29.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.105+0000 7ff36bda9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640b2060 con 0x7ff3640cb130 2023-10-21T17:19:29.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3640afbe0 0x7ff3640b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:29.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff3640afbe0 0x7ff3640b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34354/0 (socket says 172.21.15.53:34354) 2023-10-21T17:19:29.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 -- 172.21.15.53:0/1979625933 learned_addr learned my addr 172.21.15.53:0/1979625933 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:29.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 -- 172.21.15.53:0/1979625933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3640cb130 msgr2=0x7ff3640cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:29.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 --2- 172.21.15.53:0/1979625933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3640cb130 0x7ff3640cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 -- 172.21.15.53:0/1979625933 >> v1:172.21.15.124:6789/0 conn(0x7ff36406dfc0 legacy=0x7ff364070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:29.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.106+0000 7ff369b45640 1 -- 172.21.15.53:0/1979625933 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3640b2340 con 0x7ff3640afbe0 2023-10-21T17:19:29.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.107+0000 7ff369b45640 1 --2- 172.21.15.53:0/1979625933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff3640b15b0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7ff360009870 tx=0x7ff36002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dce51fe31994f4b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:29.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.108+0000 7ff368b43640 1 -- 172.21.15.53:0/1979625933 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360031640 con 0x7ff3640afbe0 2023-10-21T17:19:29.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.108+0000 7ff368b43640 1 -- 172.21.15.53:0/1979625933 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3600317a0 con 0x7ff3640afbe0 2023-10-21T17:19:29.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.108+0000 7ff368b43640 1 -- 172.21.15.53:0/1979625933 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360031b30 con 0x7ff3640afbe0 2023-10-21T17:19:29.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.108+0000 7ff36bda9640 1 -- 172.21.15.53:0/1979625933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 msgr2=0x7ff3640b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:29.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.108+0000 7ff36bda9640 1 --2- 172.21.15.53:0/1979625933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff3640b15b0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7ff360009870 tx=0x7ff36002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 -- 172.21.15.53:0/1979625933 shutdown_connections 2023-10-21T17:19:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 --2- 172.21.15.53:0/1979625933 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3640cb130 0x7ff3640cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 --2- 172.21.15.53:0/1979625933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff3640b15b0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 -- 172.21.15.53:0/1979625933 >> 172.21.15.53:0/1979625933 conn(0x7ff36406a5d0 msgr2=0x7ff3640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:29.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 -- 172.21.15.53:0/1979625933 shutdown_connections 2023-10-21T17:19:29.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.109+0000 7ff36bda9640 1 -- 172.21.15.53:0/1979625933 wait complete. 2023-10-21T17:19:29.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.110+0000 7ff36bda9640 1 Processor -- start 2023-10-21T17:19:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.111+0000 7ff36bda9640 1 -- start start 2023-10-21T17:19:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff36406dfc0 0x7ff364159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3640cb130 0x7ff3641574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3641572a0 con 0x7ff3640afbe0 2023-10-21T17:19:29.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364158830 con 0x7ff3640cb130 2023-10-21T17:19:29.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36bda9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640cda80 con 0x7ff36406dfc0 2023-10-21T17:19:29.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:29.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34364/0 (socket says 172.21.15.53:34364) 2023-10-21T17:19:29.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 -- 172.21.15.53:0/899575794 learned_addr learned my addr 172.21.15.53:0/899575794 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:29.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff36a346640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3640cb130 0x7ff3641574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:29.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369b45640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff36406dfc0 0x7ff364159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:29.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 -- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff36406dfc0 msgr2=0x7ff364159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:29.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff36406dfc0 0x7ff364159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 -- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3640cb130 msgr2=0x7ff3641574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3640cb130 0x7ff3641574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.112+0000 7ff369344640 1 -- 172.21.15.53:0/899575794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600094d0 con 0x7ff3640afbe0 2023-10-21T17:19:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff369344640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b3c0 tx=0x7ff35400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:29.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354011830 con 0x7ff3640afbe0 2023-10-21T17:19:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff354011990 con 0x7ff3640afbe0 2023-10-21T17:19:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff354011d60 con 0x7ff3640afbe0 2023-10-21T17:19:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3641579e0 con 0x7ff3640afbe0 2023-10-21T17:19:29.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.113+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff364157f90 con 0x7ff3640afbe0 2023-10-21T17:19:29.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.114+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36404e160 con 0x7ff3640afbe0 2023-10-21T17:19:29.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.115+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff35401b460 con 0x7ff3640afbe0 2023-10-21T17:19:29.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.116+0000 7ff35a7fc640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 0x7ff34806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:29.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.116+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff354089790 con 0x7ff3640afbe0 2023-10-21T17:19:29.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.117+0000 7ff369b45640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 0x7ff34806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:29.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.118+0000 7ff369b45640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 0x7ff34806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff360036000 tx=0x7ff36003c040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:29.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.121+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3540544d0 con 0x7ff3640afbe0 2023-10-21T17:19:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.354+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff3640c5280 con 0x7ff3640afbe0 2023-10-21T17:19:29.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.844+0000 7ff35a7fc640 1 -- 172.21.15.53:0/899575794 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff354057af0 con 0x7ff3640afbe0 2023-10-21T17:19:29.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.847+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 msgr2=0x7ff34806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:29.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.847+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 0x7ff34806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff360036000 tx=0x7ff36003c040 comp rx=0 tx=0).stop 2023-10-21T17:19:29.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 msgr2=0x7ff364156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:29.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b3c0 tx=0x7ff35400b880 comp rx=0 tx=0).stop 2023-10-21T17:19:29.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 shutdown_connections 2023-10-21T17:19:29.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3640cb130 0x7ff3641574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7ff34806af90 0x7ff34806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3640afbe0 0x7ff364156d60 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 --2- 172.21.15.53:0/899575794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff36406dfc0 0x7ff364159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 >> 172.21.15.53:0/899575794 conn(0x7ff36406a5d0 msgr2=0x7ff3640ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 shutdown_connections 2023-10-21T17:19:29.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:29.848+0000 7ff36bda9640 1 -- 172.21.15.53:0/899575794 wait complete. 2023-10-21T17:19:29.870 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:19:29.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:30.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.098+0000 7f234759e640 1 Processor -- start 2023-10-21T17:19:30.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.098+0000 7f234759e640 1 -- start start 2023-10-21T17:19:30.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f234759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2348150440 0x7f2348071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f234759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480c9f90 con 0x7f23480c97d0 2023-10-21T17:19:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f234759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480ca270 con 0x7f234815d550 2023-10-21T17:19:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f234759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2348071550 con 0x7f2348150440 2023-10-21T17:19:30.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f2345d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f23480c97d0 0x7f234815ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40738/0 (socket says 172.21.15.53:40738) 2023-10-21T17:19:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.100+0000 7f2345d9b640 1 -- 172.21.15.53:0/2717195372 learned_addr learned my addr 172.21.15.53:0/2717195372 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234659c640 1 --2- 172.21.15.53:0/2717195372 >> v2:172.21.15.124:3300/0 conn(0x7f2348150440 0x7f2348071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391298208 0 0) 0x7f23480c9f90 con 0x7f23480c97d0 2023-10-21T17:19:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2330003440 con 0x7f23480c97d0 2023-10-21T17:19:30.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261685345 0 0) 0x7f2330003440 con 0x7f23480c97d0 2023-10-21T17:19:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 >> v2:172.21.15.124:3300/0 conn(0x7f2348150440 msgr2=0x7f2348071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 --2- 172.21.15.53:0/2717195372 >> v2:172.21.15.124:3300/0 conn(0x7f2348150440 0x7f2348071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 >> v1:172.21.15.124:6789/0 conn(0x7f234815d550 legacy=0x7f23480cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480ca550 con 0x7f23480c97d0 2023-10-21T17:19:30.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f233c001670 con 0x7f23480c97d0 2023-10-21T17:19:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 >> v1:172.21.15.53:6789/0 conn(0x7f23480c97d0 legacy=0x7f234815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2330003a60 0x7f2330003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 0x7f233000ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f233000b480 0x7f233000d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233000e1f0 con 0x7f2330008850 2023-10-21T17:19:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233000e360 con 0x7f2330003a60 2023-10-21T17:19:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.101+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233000e520 con 0x7f233000b480 2023-10-21T17:19:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 0x7f233000ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f234659c640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2330003a60 0x7f2330003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 -- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f233000b480 msgr2=0x7f233000d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2346d9d640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f233000b480 0x7f233000d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f233000b480 0x7f233000d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 -- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2330003a60 msgr2=0x7f2330003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2330003a60 0x7f2330003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2345d9b640 1 -- 172.21.15.53:0/2717195372 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2330003440 con 0x7f2330008850 2023-10-21T17:19:30.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f2346d9d640 1 -- 172.21.15.53:0/2717195372 reap_dead start 2023-10-21T17:19:30.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.102+0000 7f234659c640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2330003a60 0x7f2330003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:30.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.103+0000 7f2345d9b640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 0x7f233000ac80 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f233c00c240 tx=0x7f23300036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9febdd9a6c968dbe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c005ad0 con 0x7f2330008850 2023-10-21T17:19:30.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c005c30 con 0x7f2330008850 2023-10-21T17:19:30.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234559a640 1 -- 172.21.15.53:0/2717195372 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c02caf0 con 0x7f2330008850 2023-10-21T17:19:30.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 -- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 msgr2=0x7f233000ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 0x7f233000ac80 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f233c00c240 tx=0x7f23300036f0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 -- 172.21.15.53:0/2717195372 shutdown_connections 2023-10-21T17:19:30.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 --2- 172.21.15.53:0/2717195372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2330008850 0x7f233000ac80 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 -- 172.21.15.53:0/2717195372 >> 172.21.15.53:0/2717195372 conn(0x7f23480cb0c0 msgr2=0x7f23480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 -- 172.21.15.53:0/2717195372 shutdown_connections 2023-10-21T17:19:30.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.104+0000 7f234759e640 1 -- 172.21.15.53:0/2717195372 wait complete. 2023-10-21T17:19:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.106+0000 7f234759e640 1 Processor -- start 2023-10-21T17:19:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.106+0000 7f234759e640 1 -- start start 2023-10-21T17:19:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2330008850 0x7f23480c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23480b2580 0x7f23480b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234814f3b0 con 0x7f234814f5e0 2023-10-21T17:19:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480b3490 con 0x7f23480b2580 2023-10-21T17:19:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.107+0000 7f234759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480b3770 con 0x7f2330008850 2023-10-21T17:19:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34384/0 (socket says 172.21.15.53:34384) 2023-10-21T17:19:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 -- 172.21.15.53:0/4280538863 learned_addr learned my addr 172.21.15.53:0/4280538863 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f234659c640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2330008850 0x7f23480c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2346d9d640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23480b2580 0x7f23480b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 -- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2330008850 msgr2=0x7f23480c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2330008850 0x7f23480c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 -- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23480b2580 msgr2=0x7f23480b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23480b2580 0x7f23480b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.108+0000 7f2345d9b640 1 -- 172.21.15.53:0/4280538863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23481e2e90 con 0x7f234814f5e0 2023-10-21T17:19:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f2345d9b640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f233c001930 tx=0x7f233c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c03c040 con 0x7f234814f5e0 2023-10-21T17:19:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c02ccc0 con 0x7f234814f5e0 2023-10-21T17:19:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c007290 con 0x7f234814f5e0 2023-10-21T17:19:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23481e3170 con 0x7f234814f5e0 2023-10-21T17:19:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.109+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480b3120 con 0x7f234814f5e0 2023-10-21T17:19:30.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.111+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23481e2cf0 con 0x7f234814f5e0 2023-10-21T17:19:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.112+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f233c032070 con 0x7f234814f5e0 2023-10-21T17:19:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.113+0000 7f2336ffd640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 0x7f232806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.113+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f233c0aada0 con 0x7f234814f5e0 2023-10-21T17:19:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.113+0000 7f234659c640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 0x7f232806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.114+0000 7f234659c640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 0x7f232806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f234804f940 tx=0x7f2338008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.117+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23481e2cf0 con 0x7f234814f5e0 2023-10-21T17:19:30.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.351+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23481e2cf0 con 0x7f234814f5e0 2023-10-21T17:19:30.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.352+0000 7f2336ffd640 1 -- 172.21.15.53:0/4280538863 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f23481e2cf0 con 0x7f234814f5e0 2023-10-21T17:19:30.354 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:30.354 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:19:30.354 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:19:30.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 msgr2=0x7f232806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 0x7f232806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f234804f940 tx=0x7f2338008040 comp rx=0 tx=0).stop 2023-10-21T17:19:30.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 msgr2=0x7f234814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f233c001930 tx=0x7f233c02e070 comp rx=0 tx=0).stop 2023-10-21T17:19:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 shutdown_connections 2023-10-21T17:19:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23480b2580 0x7f23480b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f232806afc0 0x7f232806d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f234814f5e0 0x7f234814ee70 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 --2- 172.21.15.53:0/4280538863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2330008850 0x7f23480c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 >> 172.21.15.53:0/4280538863 conn(0x7f23480cb0c0 msgr2=0x7f234815aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:30.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 shutdown_connections 2023-10-21T17:19:30.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.355+0000 7f234759e640 1 -- 172.21.15.53:0/4280538863 wait complete. 2023-10-21T17:19:30.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:19:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.602+0000 7fb999e21640 1 Processor -- start 2023-10-21T17:19:30.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.602+0000 7fb999e21640 1 -- start start 2023-10-21T17:19:30.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.603+0000 7fb999e21640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb99406b520 0x7fb99406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.603+0000 7fb999e21640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9940b0340 con 0x7fb9940af8b0 2023-10-21T17:19:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.603+0000 7fb999e21640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9940cf8f0 con 0x7fb9940b0ab0 2023-10-21T17:19:30.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.603+0000 7fb999e21640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99406b3a0 con 0x7fb99406b520 2023-10-21T17:19:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.604+0000 7fb992ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb9940af8b0 0x7fb99406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40746/0 (socket says 172.21.15.53:40746) 2023-10-21T17:19:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.604+0000 7fb992ffd640 1 -- 172.21.15.53:0/3855066806 learned_addr learned my addr 172.21.15.53:0/3855066806 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:30.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.604+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3855066806 >> v2:172.21.15.124:3300/0 conn(0x7fb99406b520 0x7fb99406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.605+0000 7fb9927fc640 1 -- 172.21.15.53:0/3855066806 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827511284 0 0) 0x7fb9940b0340 con 0x7fb9940af8b0 2023-10-21T17:19:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.605+0000 7fb9927fc640 1 -- 172.21.15.53:0/3855066806 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb980003520 con 0x7fb9940af8b0 2023-10-21T17:19:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.605+0000 7fb9937fe640 1 -- 172.21.15.53:0/3855066806 >> v1:172.21.15.124:6789/0 conn(0x7fb9940b0ab0 legacy=0x7fb9940b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.605+0000 7fb9937fe640 1 -- 172.21.15.53:0/3855066806 >> v1:172.21.15.53:6789/0 conn(0x7fb9940af8b0 legacy=0x7fb99406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.605+0000 7fb9937fe640 1 -- 172.21.15.53:0/3855066806 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940cfbd0 con 0x7fb99406b520 2023-10-21T17:19:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.606+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3855066806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb99406ae60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb984009de0 tx=0x7fb98402bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5999b02d0b955683 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.607+0000 7fb9927fc640 1 -- 172.21.15.53:0/3855066806 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98402ccb0 con 0x7fb99406b520 2023-10-21T17:19:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.607+0000 7fb9927fc640 1 -- 172.21.15.53:0/3855066806 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98402ce10 con 0x7fb99406b520 2023-10-21T17:19:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.607+0000 7fb9927fc640 1 -- 172.21.15.53:0/3855066806 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984034640 con 0x7fb99406b520 2023-10-21T17:19:30.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 -- 172.21.15.53:0/3855066806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 msgr2=0x7fb99406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 --2- 172.21.15.53:0/3855066806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb99406ae60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb984009de0 tx=0x7fb98402bf00 comp rx=0 tx=0).stop 2023-10-21T17:19:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 -- 172.21.15.53:0/3855066806 shutdown_connections 2023-10-21T17:19:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 --2- 172.21.15.53:0/3855066806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb99406ae60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 -- 172.21.15.53:0/3855066806 >> 172.21.15.53:0/3855066806 conn(0x7fb9940d09e0 msgr2=0x7fb9940d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:30.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 -- 172.21.15.53:0/3855066806 shutdown_connections 2023-10-21T17:19:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.608+0000 7fb999e21640 1 -- 172.21.15.53:0/3855066806 wait complete. 2023-10-21T17:19:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.610+0000 7fb999e21640 1 Processor -- start 2023-10-21T17:19:30.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.610+0000 7fb999e21640 1 -- start start 2023-10-21T17:19:30.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.611+0000 7fb999e21640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb9941efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb999e21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9940af8b0 0x7fb9941f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb999e21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb999e21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9941f0080 con 0x7fb9940b0ab0 2023-10-21T17:19:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb999e21640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9940c5560 con 0x7fb99406b520 2023-10-21T17:19:30.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb999e21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99414b3d0 con 0x7fb9940af8b0 2023-10-21T17:19:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb993fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb993fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34394/0 (socket says 172.21.15.53:34394) 2023-10-21T17:19:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb993fff640 1 -- 172.21.15.53:0/3788572811 learned_addr learned my addr 172.21.15.53:0/3788572811 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb9941efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.612+0000 7fb992ffd640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9940af8b0 0x7fb9941f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 -- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9940af8b0 msgr2=0x7fb9941f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9940af8b0 0x7fb9941f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 -- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 msgr2=0x7fb9941efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb9941efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 -- 172.21.15.53:0/3788572811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984009a70 con 0x7fb9940b0ab0 2023-10-21T17:19:30.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb9941efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:30.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb993fff640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb98c001eb0 tx=0x7fb98c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0052d0 con 0x7fb9940b0ab0 2023-10-21T17:19:30.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c005430 con 0x7fb9940b0ab0 2023-10-21T17:19:30.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9940c5820 con 0x7fb9940b0ab0 2023-10-21T17:19:30.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.613+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c003810 con 0x7fb9940b0ab0 2023-10-21T17:19:30.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.614+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9940c5de0 con 0x7fb9940b0ab0 2023-10-21T17:19:30.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.615+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb994067490 con 0x7fb9940b0ab0 2023-10-21T17:19:30.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.616+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb98c003970 con 0x7fb9940b0ab0 2023-10-21T17:19:30.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.617+0000 7fb98bfff640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 0x7fb97406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:30.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.617+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb98c088f00 con 0x7fb9940b0ab0 2023-10-21T17:19:30.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.617+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 0x7fb97406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:30.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.618+0000 7fb9937fe640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 0x7fb97406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb984036000 tx=0x7fb984030000 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:30.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.621+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb98c053ce0 con 0x7fb9940b0ab0 2023-10-21T17:19:30.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.813+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb99408aae0 con 0x7fb9940b0ab0 2023-10-21T17:19:30.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.816+0000 7fb98bfff640 1 -- 172.21.15.53:0/3788572811 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16890 (secure 0 0 0) 0x7fb98c017090 con 0x7fb9940b0ab0 2023-10-21T17:19:30.818 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:30.818 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":197,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:19:27.616131+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:19:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.818+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 msgr2=0x7fb97406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.818+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 0x7fb97406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb984036000 tx=0x7fb984030000 comp rx=0 tx=0).stop 2023-10-21T17:19:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.818+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 msgr2=0x7fb99409dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:30.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.818+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb98c001eb0 tx=0x7fb98c00c890 comp rx=0 tx=0).stop 2023-10-21T17:19:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 shutdown_connections 2023-10-21T17:19:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fb97406af90 0x7fb97406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9940b0ab0 0x7fb99409dd20 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9940af8b0 0x7fb9941f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 --2- 172.21.15.53:0/3788572811 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb99406b520 0x7fb9941efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 >> 172.21.15.53:0/3788572811 conn(0x7fb9940d09e0 msgr2=0x7fb9940cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 shutdown_connections 2023-10-21T17:19:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:30.819+0000 7fb999e21640 1 -- 172.21.15.53:0/3788572811 wait complete. 2023-10-21T17:19:30.840 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:19:30.841 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:19:31.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.062+0000 7f82eb5c9640 1 Processor -- start 2023-10-21T17:19:31.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.063+0000 7f82eb5c9640 1 -- start start 2023-10-21T17:19:31.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.064+0000 7f82eb5c9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82e40cb580 0x7f82e4165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.064+0000 7f82eb5c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e415b2b0 con 0x7f82e40c90b0 2023-10-21T17:19:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.064+0000 7f82eb5c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e415b590 con 0x7f82e415ab40 2023-10-21T17:19:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.064+0000 7f82eb5c9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e415bf30 con 0x7f82e40cb580 2023-10-21T17:19:31.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.065+0000 7f82e8b64640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f82e40c90b0 0x7f82e40cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40760/0 (socket says 172.21.15.53:40760) 2023-10-21T17:19:31.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.065+0000 7f82e8b64640 1 -- 172.21.15.53:0/3340969238 learned_addr learned my addr 172.21.15.53:0/3340969238 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.065+0000 7f82e9b66640 1 --2- 172.21.15.53:0/3340969238 >> v2:172.21.15.124:3300/0 conn(0x7f82e40cb580 0x7f82e4165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.065+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590364542 0 0) 0x7f82e415b2b0 con 0x7f82e40c90b0 2023-10-21T17:19:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.065+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d0003520 con 0x7f82e40c90b0 2023-10-21T17:19:31.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971572169 0 0) 0x7f82d0003520 con 0x7f82e40c90b0 2023-10-21T17:19:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 >> v2:172.21.15.124:3300/0 conn(0x7f82e40cb580 msgr2=0x7f82e4165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 --2- 172.21.15.53:0/3340969238 >> v2:172.21.15.124:3300/0 conn(0x7f82e40cb580 0x7f82e4165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 >> v1:172.21.15.124:6789/0 conn(0x7f82e415ab40 legacy=0x7f82e40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e415b870 con 0x7f82e40c90b0 2023-10-21T17:19:31.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f82d4001670 con 0x7f82e40c90b0 2023-10-21T17:19:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 >> v1:172.21.15.53:6789/0 conn(0x7f82e40c90b0 legacy=0x7f82e40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 0x7f82d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00047c0 0x7f82d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d0005180 0x7f82d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e415bf30 con 0x7f82d00039c0 2023-10-21T17:19:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000c080 con 0x7f82d00047c0 2023-10-21T17:19:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d000c240 con 0x7f82d0005180 2023-10-21T17:19:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.066+0000 7f82e9365640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 0x7f82d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e8b64640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00047c0 0x7f82d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9365640 1 -- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d0005180 msgr2=0x7f82d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9365640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82d0005180 0x7f82d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9365640 1 -- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00047c0 msgr2=0x7f82d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9365640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00047c0 0x7f82d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9365640 1 -- 172.21.15.53:0/3340969238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0003520 con 0x7f82d00039c0 2023-10-21T17:19:31.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e8b64640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00047c0 0x7f82d0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.067+0000 7f82e9b66640 1 -- 172.21.15.53:0/3340969238 reap_dead start 2023-10-21T17:19:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.068+0000 7f82e9365640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 0x7f82d0003de0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f82e0001750 tx=0x7f82d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9781868c134cef6b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.068+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00052d0 con 0x7f82d00039c0 2023-10-21T17:19:31.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.068+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e002e050 con 0x7f82d00039c0 2023-10-21T17:19:31.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.068+0000 7f82dbfff640 1 -- 172.21.15.53:0/3340969238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0003830 con 0x7f82d00039c0 2023-10-21T17:19:31.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 -- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 msgr2=0x7f82d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 0x7f82d0003de0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f82e0001750 tx=0x7f82d00057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 -- 172.21.15.53:0/3340969238 shutdown_connections 2023-10-21T17:19:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/3340969238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82d00039c0 0x7f82d0003de0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 -- 172.21.15.53:0/3340969238 >> 172.21.15.53:0/3340969238 conn(0x7f82e406a380 msgr2=0x7f82e40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 -- 172.21.15.53:0/3340969238 shutdown_connections 2023-10-21T17:19:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.069+0000 7f82eb5c9640 1 -- 172.21.15.53:0/3340969238 wait complete. 2023-10-21T17:19:31.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.071+0000 7f82eb5c9640 1 Processor -- start 2023-10-21T17:19:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.071+0000 7f82eb5c9640 1 -- start start 2023-10-21T17:19:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 0x7f82e41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40ac610 0x7f82e4158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e415f7a0 con 0x7f82e406d080 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e406da40 con 0x7f82d00039c0 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82eb5c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c36c0 con 0x7f82e40ac610 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82e9b66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82e9b66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34420/0 (socket says 172.21.15.53:34420) 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.072+0000 7f82e9b66640 1 -- 172.21.15.53:0/2552509418 learned_addr learned my addr 172.21.15.53:0/2552509418 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:31.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9365640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 0x7f82e41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e8b64640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40ac610 0x7f82e4158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 -- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40ac610 msgr2=0x7f82e4158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40ac610 0x7f82e4158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 -- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 msgr2=0x7f82e41533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 0x7f82e41533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 -- 172.21.15.53:0/2552509418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4158d50 con 0x7f82e406d080 2023-10-21T17:19:31.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9365640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 0x7f82e41533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:31.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.073+0000 7f82e9b66640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f82e40b0c20 tx=0x7f82dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.074+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc010040 con 0x7f82e406d080 2023-10-21T17:19:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.074+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00bc80 con 0x7f82e406d080 2023-10-21T17:19:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.074+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e4159030 con 0x7f82e406d080 2023-10-21T17:19:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.074+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc014400 con 0x7f82e406d080 2023-10-21T17:19:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.074+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e41595f0 con 0x7f82e406d080 2023-10-21T17:19:31.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.075+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e4067490 con 0x7f82e406d080 2023-10-21T17:19:31.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.076+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f82dc014560 con 0x7f82e406d080 2023-10-21T17:19:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.077+0000 7f82d9ffb640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 0x7f82c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.077+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f82dc08a250 con 0x7f82e406d080 2023-10-21T17:19:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.078+0000 7f82e9365640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 0x7f82c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.079+0000 7f82e9365640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 0x7f82c806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82e0033000 tx=0x7f82e0009030 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.081+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82dc054fc0 con 0x7f82e406d080 2023-10-21T17:19:31.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.269+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 --> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82e40cd3e0 con 0x7f82c806af90 2023-10-21T17:19:31.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.299+0000 7f82d9ffb640 1 -- 172.21.15.53:0/2552509418 <== mgr.37184 v2:172.21.15.124:6832/1955466907 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287261 (secure 0 0 0) 0x7f82e0007040 con 0x7f82c806af90 2023-10-21T17:19:31.303 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:31.306 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:19:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 msgr2=0x7f82c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 0x7f82c806d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82e0033000 tx=0x7f82e0009030 comp rx=0 tx=0).stop 2023-10-21T17:19:31.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 msgr2=0x7f82e406d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f82e40b0c20 tx=0x7f82dc00b480 comp rx=0 tx=0).stop 2023-10-21T17:19:31.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 shutdown_connections 2023-10-21T17:19:31.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f82c806af90 0x7f82c806d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82e406d080 0x7f82e406d500 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e40ac610 0x7f82e4158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 --2- 172.21.15.53:0/2552509418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82d00039c0 0x7f82e41533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 >> 172.21.15.53:0/2552509418 conn(0x7f82e406a380 msgr2=0x7f82e406c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:31.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 shutdown_connections 2023-10-21T17:19:31.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.307+0000 7f82eb5c9640 1 -- 172.21.15.53:0/2552509418 wait complete. 2023-10-21T17:19:31.330 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T17:19:29.986891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":531,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4856,"ondisk_log_size":4856,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346312,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754628408,"statfs":{"total":773094113280,"available":772739489792,"internally_reserved":0,"allocated":20979712,"data_stored":11304380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":333499961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400629"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900066+0000","last_change":"2023-10-21T17:19:26.658109+0000","last_active":"2023-10-21T17:19:27.900066+0000","last_peered":"2023-10-21T17:19:27.900066+0000","last_clean":"2023-10-21T17:19:27.900066+0000","last_became_active":"2023-10-21T17:19:26.656933+0000","last_became_peered":"2023-10-21T17:19:26.656933+0000","last_unstale":"2023-10-21T17:19:27.900066+0000","last_undegraded":"2023-10-21T17:19:27.900066+0000","last_fullsized":"2023-10-21T17:19:27.900066+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:59.526154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627478+0000","last_change":"2023-10-21T17:19:26.659020+0000","last_active":"2023-10-21T17:19:27.627478+0000","last_peered":"2023-10-21T17:19:27.627478+0000","last_clean":"2023-10-21T17:19:27.627478+0000","last_became_active":"2023-10-21T17:19:26.658750+0000","last_became_peered":"2023-10-21T17:19:26.658750+0000","last_unstale":"2023-10-21T17:19:27.627478+0000","last_undegraded":"2023-10-21T17:19:27.627478+0000","last_fullsized":"2023-10-21T17:19:27.627478+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:08.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899613+0000","last_change":"2023-10-21T17:19:26.644243+0000","last_active":"2023-10-21T17:19:27.899613+0000","last_peered":"2023-10-21T17:19:27.899613+0000","last_clean":"2023-10-21T17:19:27.899613+0000","last_became_active":"2023-10-21T17:19:26.643819+0000","last_became_peered":"2023-10-21T17:19:26.643819+0000","last_unstale":"2023-10-21T17:19:27.899613+0000","last_undegraded":"2023-10-21T17:19:27.899613+0000","last_fullsized":"2023-10-21T17:19:27.899613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:05.835440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899729+0000","last_change":"2023-10-21T17:19:26.645519+0000","last_active":"2023-10-21T17:19:27.899729+0000","last_peered":"2023-10-21T17:19:27.899729+0000","last_clean":"2023-10-21T17:19:27.899729+0000","last_became_active":"2023-10-21T17:19:26.645220+0000","last_became_peered":"2023-10-21T17:19:26.645220+0000","last_unstale":"2023-10-21T17:19:27.899729+0000","last_undegraded":"2023-10-21T17:19:27.899729+0000","last_fullsized":"2023-10-21T17:19:27.899729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:19.142990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899116+0000","last_change":"2023-10-21T17:19:26.660273+0000","last_active":"2023-10-21T17:19:27.899116+0000","last_peered":"2023-10-21T17:19:27.899116+0000","last_clean":"2023-10-21T17:19:27.899116+0000","last_became_active":"2023-10-21T17:19:26.660015+0000","last_became_peered":"2023-10-21T17:19:26.660015+0000","last_unstale":"2023-10-21T17:19:27.899116+0000","last_undegraded":"2023-10-21T17:19:27.899116+0000","last_fullsized":"2023-10-21T17:19:27.899116+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:22.831978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626524+0000","last_change":"2023-10-21T17:19:26.667811+0000","last_active":"2023-10-21T17:19:27.626524+0000","last_peered":"2023-10-21T17:19:27.626524+0000","last_clean":"2023-10-21T17:19:27.626524+0000","last_became_active":"2023-10-21T17:19:26.667501+0000","last_became_peered":"2023-10-21T17:19:26.667501+0000","last_unstale":"2023-10-21T17:19:27.626524+0000","last_undegraded":"2023-10-21T17:19:27.626524+0000","last_fullsized":"2023-10-21T17:19:27.626524+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:48.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896945+0000","last_change":"2023-10-21T17:19:26.658879+0000","last_active":"2023-10-21T17:19:27.896945+0000","last_peered":"2023-10-21T17:19:27.896945+0000","last_clean":"2023-10-21T17:19:27.896945+0000","last_became_active":"2023-10-21T17:19:26.653573+0000","last_became_peered":"2023-10-21T17:19:26.653573+0000","last_unstale":"2023-10-21T17:19:27.896945+0000","last_undegraded":"2023-10-21T17:19:27.896945+0000","last_fullsized":"2023-10-21T17:19:27.896945+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896390+0000","last_change":"2023-10-21T17:19:26.661762+0000","last_active":"2023-10-21T17:19:27.896390+0000","last_peered":"2023-10-21T17:19:27.896390+0000","last_clean":"2023-10-21T17:19:27.896390+0000","last_became_active":"2023-10-21T17:19:26.658627+0000","last_became_peered":"2023-10-21T17:19:26.658627+0000","last_unstale":"2023-10-21T17:19:27.896390+0000","last_undegraded":"2023-10-21T17:19:27.896390+0000","last_fullsized":"2023-10-21T17:19:27.896390+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:23:22.849050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901280+0000","last_change":"2023-10-21T17:19:26.660406+0000","last_active":"2023-10-21T17:19:27.901280+0000","last_peered":"2023-10-21T17:19:27.901280+0000","last_clean":"2023-10-21T17:19:27.901280+0000","last_became_active":"2023-10-21T17:19:26.660097+0000","last_became_peered":"2023-10-21T17:19:26.660097+0000","last_unstale":"2023-10-21T17:19:27.901280+0000","last_undegraded":"2023-10-21T17:19:27.901280+0000","last_fullsized":"2023-10-21T17:19:27.901280+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:36:14.756079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901775+0000","last_change":"2023-10-21T17:19:26.660982+0000","last_active":"2023-10-21T17:19:27.901775+0000","last_peered":"2023-10-21T17:19:27.901775+0000","last_clean":"2023-10-21T17:19:27.901775+0000","last_became_active":"2023-10-21T17:19:26.660678+0000","last_became_peered":"2023-10-21T17:19:26.660678+0000","last_unstale":"2023-10-21T17:19:27.901775+0000","last_undegraded":"2023-10-21T17:19:27.901775+0000","last_fullsized":"2023-10-21T17:19:27.901775+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:52:25.806442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624270+0000","last_change":"2023-10-21T17:19:26.659014+0000","last_active":"2023-10-21T17:19:27.624270+0000","last_peered":"2023-10-21T17:19:27.624270+0000","last_clean":"2023-10-21T17:19:27.624270+0000","last_became_active":"2023-10-21T17:19:26.658234+0000","last_became_peered":"2023-10-21T17:19:26.658234+0000","last_unstale":"2023-10-21T17:19:27.624270+0000","last_undegraded":"2023-10-21T17:19:27.624270+0000","last_fullsized":"2023-10-21T17:19:27.624270+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:04.562111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627688+0000","last_change":"2023-10-21T17:19:26.667654+0000","last_active":"2023-10-21T17:19:27.627688+0000","last_peered":"2023-10-21T17:19:27.627688+0000","last_clean":"2023-10-21T17:19:27.627688+0000","last_became_active":"2023-10-21T17:19:26.667357+0000","last_became_peered":"2023-10-21T17:19:26.667357+0000","last_unstale":"2023-10-21T17:19:27.627688+0000","last_undegraded":"2023-10-21T17:19:27.627688+0000","last_fullsized":"2023-10-21T17:19:27.627688+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:36.912519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899883+0000","last_change":"2023-10-21T17:19:26.654520+0000","last_active":"2023-10-21T17:19:27.899883+0000","last_peered":"2023-10-21T17:19:27.899883+0000","last_clean":"2023-10-21T17:19:27.899883+0000","last_became_active":"2023-10-21T17:19:26.652767+0000","last_became_peered":"2023-10-21T17:19:26.652767+0000","last_unstale":"2023-10-21T17:19:27.899883+0000","last_undegraded":"2023-10-21T17:19:27.899883+0000","last_fullsized":"2023-10-21T17:19:27.899883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:38.345273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898682+0000","last_change":"2023-10-21T17:19:26.659469+0000","last_active":"2023-10-21T17:19:27.898682+0000","last_peered":"2023-10-21T17:19:27.898682+0000","last_clean":"2023-10-21T17:19:27.898682+0000","last_became_active":"2023-10-21T17:19:26.659128+0000","last_became_peered":"2023-10-21T17:19:26.659128+0000","last_unstale":"2023-10-21T17:19:27.898682+0000","last_undegraded":"2023-10-21T17:19:27.898682+0000","last_fullsized":"2023-10-21T17:19:27.898682+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:03.840579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624029+0000","last_change":"2023-10-21T17:19:26.661526+0000","last_active":"2023-10-21T17:19:27.624029+0000","last_peered":"2023-10-21T17:19:27.624029+0000","last_clean":"2023-10-21T17:19:27.624029+0000","last_became_active":"2023-10-21T17:19:26.661164+0000","last_became_peered":"2023-10-21T17:19:26.661164+0000","last_unstale":"2023-10-21T17:19:27.624029+0000","last_undegraded":"2023-10-21T17:19:27.624029+0000","last_fullsized":"2023-10-21T17:19:27.624029+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:55.160079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898780+0000","last_change":"2023-10-21T17:19:26.665823+0000","last_active":"2023-10-21T17:19:27.898780+0000","last_peered":"2023-10-21T17:19:27.898780+0000","last_clean":"2023-10-21T17:19:27.898780+0000","last_became_active":"2023-10-21T17:19:26.665511+0000","last_became_peered":"2023-10-21T17:19:26.665511+0000","last_unstale":"2023-10-21T17:19:27.898780+0000","last_undegraded":"2023-10-21T17:19:27.898780+0000","last_fullsized":"2023-10-21T17:19:27.898780+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:45:35.821926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901318+0000","last_change":"2023-10-21T17:19:26.658932+0000","last_active":"2023-10-21T17:19:27.901318+0000","last_peered":"2023-10-21T17:19:27.901318+0000","last_clean":"2023-10-21T17:19:27.901318+0000","last_became_active":"2023-10-21T17:19:26.653673+0000","last_became_peered":"2023-10-21T17:19:26.653673+0000","last_unstale":"2023-10-21T17:19:27.901318+0000","last_undegraded":"2023-10-21T17:19:27.901318+0000","last_fullsized":"2023-10-21T17:19:27.901318+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:50:20.910807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898946+0000","last_change":"2023-10-21T17:19:26.662010+0000","last_active":"2023-10-21T17:19:27.898946+0000","last_peered":"2023-10-21T17:19:27.898946+0000","last_clean":"2023-10-21T17:19:27.898946+0000","last_became_active":"2023-10-21T17:19:26.661721+0000","last_became_peered":"2023-10-21T17:19:26.661721+0000","last_unstale":"2023-10-21T17:19:27.898946+0000","last_undegraded":"2023-10-21T17:19:27.898946+0000","last_fullsized":"2023-10-21T17:19:27.898946+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:46:57.203785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.625171+0000","last_change":"2023-10-21T17:19:26.655662+0000","last_active":"2023-10-21T17:19:27.625171+0000","last_peered":"2023-10-21T17:19:27.625171+0000","last_clean":"2023-10-21T17:19:27.625171+0000","last_became_active":"2023-10-21T17:19:26.653571+0000","last_became_peered":"2023-10-21T17:19:26.653571+0000","last_unstale":"2023-10-21T17:19:27.625171+0000","last_undegraded":"2023-10-21T17:19:27.625171+0000","last_fullsized":"2023-10-21T17:19:27.625171+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:30:07.818930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623786+0000","last_change":"2023-10-21T17:19:26.660538+0000","last_active":"2023-10-21T17:19:27.623786+0000","last_peered":"2023-10-21T17:19:27.623786+0000","last_clean":"2023-10-21T17:19:27.623786+0000","last_became_active":"2023-10-21T17:19:26.660154+0000","last_became_peered":"2023-10-21T17:19:26.660154+0000","last_unstale":"2023-10-21T17:19:27.623786+0000","last_undegraded":"2023-10-21T17:19:27.623786+0000","last_fullsized":"2023-10-21T17:19:27.623786+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:02.934088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900286+0000","last_change":"2023-10-21T17:19:26.646645+0000","last_active":"2023-10-21T17:19:27.900286+0000","last_peered":"2023-10-21T17:19:27.900286+0000","last_clean":"2023-10-21T17:19:27.900286+0000","last_became_active":"2023-10-21T17:19:26.646383+0000","last_became_peered":"2023-10-21T17:19:26.646383+0000","last_unstale":"2023-10-21T17:19:27.900286+0000","last_undegraded":"2023-10-21T17:19:27.900286+0000","last_fullsized":"2023-10-21T17:19:27.900286+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:30:30.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626843+0000","last_change":"2023-10-21T17:19:26.660178+0000","last_active":"2023-10-21T17:19:27.626843+0000","last_peered":"2023-10-21T17:19:27.626843+0000","last_clean":"2023-10-21T17:19:27.626843+0000","last_became_active":"2023-10-21T17:19:26.659891+0000","last_became_peered":"2023-10-21T17:19:26.659891+0000","last_unstale":"2023-10-21T17:19:27.626843+0000","last_undegraded":"2023-10-21T17:19:27.626843+0000","last_fullsized":"2023-10-21T17:19:27.626843+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:53.807618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897384+0000","last_change":"2023-10-21T17:19:26.662366+0000","last_active":"2023-10-21T17:19:27.897384+0000","last_peered":"2023-10-21T17:19:27.897384+0000","last_clean":"2023-10-21T17:19:27.897384+0000","last_became_active":"2023-10-21T17:19:26.662114+0000","last_became_peered":"2023-10-21T17:19:26.662114+0000","last_unstale":"2023-10-21T17:19:27.897384+0000","last_undegraded":"2023-10-21T17:19:27.897384+0000","last_fullsized":"2023-10-21T17:19:27.897384+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:45.392042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898764+0000","last_change":"2023-10-21T17:19:26.648214+0000","last_active":"2023-10-21T17:19:27.898764+0000","last_peered":"2023-10-21T17:19:27.898764+0000","last_clean":"2023-10-21T17:19:27.898764+0000","last_became_active":"2023-10-21T17:19:26.647825+0000","last_became_peered":"2023-10-21T17:19:26.647825+0000","last_unstale":"2023-10-21T17:19:27.898764+0000","last_undegraded":"2023-10-21T17:19:27.898764+0000","last_fullsized":"2023-10-21T17:19:27.898764+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:23.435867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900929+0000","last_change":"2023-10-21T17:19:26.651987+0000","last_active":"2023-10-21T17:19:27.900929+0000","last_peered":"2023-10-21T17:19:27.900929+0000","last_clean":"2023-10-21T17:19:27.900929+0000","last_became_active":"2023-10-21T17:19:26.651673+0000","last_became_peered":"2023-10-21T17:19:26.651673+0000","last_unstale":"2023-10-21T17:19:27.900929+0000","last_undegraded":"2023-10-21T17:19:27.900929+0000","last_fullsized":"2023-10-21T17:19:27.900929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:57:57.688867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623963+0000","last_change":"2023-10-21T17:19:26.639823+0000","last_active":"2023-10-21T17:19:27.623963+0000","last_peered":"2023-10-21T17:19:27.623963+0000","last_clean":"2023-10-21T17:19:27.623963+0000","last_became_active":"2023-10-21T17:19:26.639414+0000","last_became_peered":"2023-10-21T17:19:26.639414+0000","last_unstale":"2023-10-21T17:19:27.623963+0000","last_undegraded":"2023-10-21T17:19:27.623963+0000","last_fullsized":"2023-10-21T17:19:27.623963+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:46:07.889981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627784+0000","last_change":"2023-10-21T17:19:26.660271+0000","last_active":"2023-10-21T17:19:27.627784+0000","last_peered":"2023-10-21T17:19:27.627784+0000","last_clean":"2023-10-21T17:19:27.627784+0000","last_became_active":"2023-10-21T17:19:26.659818+0000","last_became_peered":"2023-10-21T17:19:26.659818+0000","last_unstale":"2023-10-21T17:19:27.627784+0000","last_undegraded":"2023-10-21T17:19:27.627784+0000","last_fullsized":"2023-10-21T17:19:27.627784+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:04.562111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897775+0000","last_change":"2023-10-21T17:19:26.648562+0000","last_active":"2023-10-21T17:19:27.897775+0000","last_peered":"2023-10-21T17:19:27.897775+0000","last_clean":"2023-10-21T17:19:27.897775+0000","last_became_active":"2023-10-21T17:19:26.648077+0000","last_became_peered":"2023-10-21T17:19:26.648077+0000","last_unstale":"2023-10-21T17:19:27.897775+0000","last_undegraded":"2023-10-21T17:19:27.897775+0000","last_fullsized":"2023-10-21T17:19:27.897775+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:07:02.155620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627026+0000","last_change":"2023-10-21T17:19:26.652544+0000","last_active":"2023-10-21T17:19:27.627026+0000","last_peered":"2023-10-21T17:19:27.627026+0000","last_clean":"2023-10-21T17:19:27.627026+0000","last_became_active":"2023-10-21T17:19:26.649430+0000","last_became_peered":"2023-10-21T17:19:26.649430+0000","last_unstale":"2023-10-21T17:19:27.627026+0000","last_undegraded":"2023-10-21T17:19:27.627026+0000","last_fullsized":"2023-10-21T17:19:27.627026+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:23.732089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898050+0000","last_change":"2023-10-21T17:19:26.657540+0000","last_active":"2023-10-21T17:19:27.898050+0000","last_peered":"2023-10-21T17:19:27.898050+0000","last_clean":"2023-10-21T17:19:27.898050+0000","last_became_active":"2023-10-21T17:19:26.657240+0000","last_became_peered":"2023-10-21T17:19:26.657240+0000","last_unstale":"2023-10-21T17:19:27.898050+0000","last_undegraded":"2023-10-21T17:19:27.898050+0000","last_fullsized":"2023-10-21T17:19:27.898050+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:51.179302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624335+0000","last_change":"2023-10-21T17:19:26.657388+0000","last_active":"2023-10-21T17:19:27.624335+0000","last_peered":"2023-10-21T17:19:27.624335+0000","last_clean":"2023-10-21T17:19:27.624335+0000","last_became_active":"2023-10-21T17:19:26.657041+0000","last_became_peered":"2023-10-21T17:19:26.657041+0000","last_unstale":"2023-10-21T17:19:27.624335+0000","last_undegraded":"2023-10-21T17:19:27.624335+0000","last_fullsized":"2023-10-21T17:19:27.624335+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:56.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897855+0000","last_change":"2023-10-21T17:19:26.660563+0000","last_active":"2023-10-21T17:19:27.897855+0000","last_peered":"2023-10-21T17:19:27.897855+0000","last_clean":"2023-10-21T17:19:27.897855+0000","last_became_active":"2023-10-21T17:19:26.660258+0000","last_became_peered":"2023-10-21T17:19:26.660258+0000","last_unstale":"2023-10-21T17:19:27.897855+0000","last_undegraded":"2023-10-21T17:19:27.897855+0000","last_fullsized":"2023-10-21T17:19:27.897855+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:37:05.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626174+0000","last_change":"2023-10-21T17:19:26.657006+0000","last_active":"2023-10-21T17:19:27.626174+0000","last_peered":"2023-10-21T17:19:27.626174+0000","last_clean":"2023-10-21T17:19:27.626174+0000","last_became_active":"2023-10-21T17:19:26.653146+0000","last_became_peered":"2023-10-21T17:19:26.653146+0000","last_unstale":"2023-10-21T17:19:27.626174+0000","last_undegraded":"2023-10-21T17:19:27.626174+0000","last_fullsized":"2023-10-21T17:19:27.626174+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:08:23.545216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"196'4","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900303+0000","last_change":"2023-10-21T17:19:25.619174+0000","last_active":"2023-10-21T17:19:27.900303+0000","last_peered":"2023-10-21T17:19:27.900303+0000","last_clean":"2023-10-21T17:19:27.900303+0000","last_became_active":"2023-10-21T17:19:25.618607+0000","last_became_peered":"2023-10-21T17:19:25.618607+0000","last_unstale":"2023-10-21T17:19:27.900303+0000","last_undegraded":"2023-10-21T17:19:27.900303+0000","last_fullsized":"2023-10-21T17:19:27.900303+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:53:24.466647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897560+0000","last_change":"2023-10-21T17:19:26.666188+0000","last_active":"2023-10-21T17:19:27.897560+0000","last_peered":"2023-10-21T17:19:27.897560+0000","last_clean":"2023-10-21T17:19:27.897560+0000","last_became_active":"2023-10-21T17:19:26.664453+0000","last_became_peered":"2023-10-21T17:19:26.664453+0000","last_unstale":"2023-10-21T17:19:27.897560+0000","last_undegraded":"2023-10-21T17:19:27.897560+0000","last_fullsized":"2023-10-21T17:19:27.897560+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:40.114373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899017+0000","last_change":"2023-10-21T17:19:25.635787+0000","last_active":"2023-10-21T17:19:27.899017+0000","last_peered":"2023-10-21T17:19:27.899017+0000","last_clean":"2023-10-21T17:19:27.899017+0000","last_became_active":"2023-10-21T17:19:25.635322+0000","last_became_peered":"2023-10-21T17:19:25.635322+0000","last_unstale":"2023-10-21T17:19:27.899017+0000","last_undegraded":"2023-10-21T17:19:27.899017+0000","last_fullsized":"2023-10-21T17:19:27.899017+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:07.724078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899090+0000","last_change":"2023-10-21T17:19:26.657176+0000","last_active":"2023-10-21T17:19:27.899090+0000","last_peered":"2023-10-21T17:19:27.899090+0000","last_clean":"2023-10-21T17:19:27.899090+0000","last_became_active":"2023-10-21T17:19:26.656875+0000","last_became_peered":"2023-10-21T17:19:26.656875+0000","last_unstale":"2023-10-21T17:19:27.899090+0000","last_undegraded":"2023-10-21T17:19:27.899090+0000","last_fullsized":"2023-10-21T17:19:27.899090+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:16.038189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624776+0000","last_change":"2023-10-21T17:19:25.612917+0000","last_active":"2023-10-21T17:19:27.624776+0000","last_peered":"2023-10-21T17:19:27.624776+0000","last_clean":"2023-10-21T17:19:27.624776+0000","last_became_active":"2023-10-21T17:19:25.606671+0000","last_became_peered":"2023-10-21T17:19:25.606671+0000","last_unstale":"2023-10-21T17:19:27.624776+0000","last_undegraded":"2023-10-21T17:19:27.624776+0000","last_fullsized":"2023-10-21T17:19:27.624776+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:25.904957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627306+0000","last_change":"2023-10-21T17:19:26.653519+0000","last_active":"2023-10-21T17:19:27.627306+0000","last_peered":"2023-10-21T17:19:27.627306+0000","last_clean":"2023-10-21T17:19:27.627306+0000","last_became_active":"2023-10-21T17:19:26.653189+0000","last_became_peered":"2023-10-21T17:19:26.653189+0000","last_unstale":"2023-10-21T17:19:27.627306+0000","last_undegraded":"2023-10-21T17:19:27.627306+0000","last_fullsized":"2023-10-21T17:19:27.627306+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:26.918326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623959+0000","last_change":"2023-10-21T17:19:25.629774+0000","last_active":"2023-10-21T17:19:27.623959+0000","last_peered":"2023-10-21T17:19:27.623959+0000","last_clean":"2023-10-21T17:19:27.623959+0000","last_became_active":"2023-10-21T17:19:25.629378+0000","last_became_peered":"2023-10-21T17:19:25.629378+0000","last_unstale":"2023-10-21T17:19:27.623959+0000","last_undegraded":"2023-10-21T17:19:27.623959+0000","last_fullsized":"2023-10-21T17:19:27.623959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:40.263107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900044+0000","last_change":"2023-10-21T17:19:26.659624+0000","last_active":"2023-10-21T17:19:27.900044+0000","last_peered":"2023-10-21T17:19:27.900044+0000","last_clean":"2023-10-21T17:19:27.900044+0000","last_became_active":"2023-10-21T17:19:26.659365+0000","last_became_peered":"2023-10-21T17:19:26.659365+0000","last_unstale":"2023-10-21T17:19:27.900044+0000","last_undegraded":"2023-10-21T17:19:27.900044+0000","last_fullsized":"2023-10-21T17:19:27.900044+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:29.679124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899814+0000","last_change":"2023-10-21T17:19:25.629460+0000","last_active":"2023-10-21T17:19:27.899814+0000","last_peered":"2023-10-21T17:19:27.899814+0000","last_clean":"2023-10-21T17:19:27.899814+0000","last_became_active":"2023-10-21T17:19:25.629152+0000","last_became_peered":"2023-10-21T17:19:25.629152+0000","last_unstale":"2023-10-21T17:19:27.899814+0000","last_undegraded":"2023-10-21T17:19:27.899814+0000","last_fullsized":"2023-10-21T17:19:27.899814+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:54.610923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623907+0000","last_change":"2023-10-21T17:19:26.648808+0000","last_active":"2023-10-21T17:19:27.623907+0000","last_peered":"2023-10-21T17:19:27.623907+0000","last_clean":"2023-10-21T17:19:27.623907+0000","last_became_active":"2023-10-21T17:19:26.648448+0000","last_became_peered":"2023-10-21T17:19:26.648448+0000","last_unstale":"2023-10-21T17:19:27.623907+0000","last_undegraded":"2023-10-21T17:19:27.623907+0000","last_fullsized":"2023-10-21T17:19:27.623907+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:12:43.958557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899356+0000","last_change":"2023-10-21T17:19:25.626670+0000","last_active":"2023-10-21T17:19:27.899356+0000","last_peered":"2023-10-21T17:19:27.899356+0000","last_clean":"2023-10-21T17:19:27.899356+0000","last_became_active":"2023-10-21T17:19:25.626375+0000","last_became_peered":"2023-10-21T17:19:25.626375+0000","last_unstale":"2023-10-21T17:19:27.899356+0000","last_undegraded":"2023-10-21T17:19:27.899356+0000","last_fullsized":"2023-10-21T17:19:27.899356+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:52.212859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624254+0000","last_change":"2023-10-21T17:19:26.654396+0000","last_active":"2023-10-21T17:19:27.624254+0000","last_peered":"2023-10-21T17:19:27.624254+0000","last_clean":"2023-10-21T17:19:27.624254+0000","last_became_active":"2023-10-21T17:19:26.654119+0000","last_became_peered":"2023-10-21T17:19:26.654119+0000","last_unstale":"2023-10-21T17:19:27.624254+0000","last_undegraded":"2023-10-21T17:19:27.624254+0000","last_fullsized":"2023-10-21T17:19:27.624254+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:48.811705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623471+0000","last_change":"2023-10-21T17:19:25.624312+0000","last_active":"2023-10-21T17:19:27.623471+0000","last_peered":"2023-10-21T17:19:27.623471+0000","last_clean":"2023-10-21T17:19:27.623471+0000","last_became_active":"2023-10-21T17:19:25.623553+0000","last_became_peered":"2023-10-21T17:19:25.623553+0000","last_unstale":"2023-10-21T17:19:27.623471+0000","last_undegraded":"2023-10-21T17:19:27.623471+0000","last_fullsized":"2023-10-21T17:19:27.623471+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:42.268063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900105+0000","last_change":"2023-10-21T17:19:26.646397+0000","last_active":"2023-10-21T17:19:27.900105+0000","last_peered":"2023-10-21T17:19:27.900105+0000","last_clean":"2023-10-21T17:19:27.900105+0000","last_became_active":"2023-10-21T17:19:26.643812+0000","last_became_peered":"2023-10-21T17:19:26.643812+0000","last_unstale":"2023-10-21T17:19:27.900105+0000","last_undegraded":"2023-10-21T17:19:27.900105+0000","last_fullsized":"2023-10-21T17:19:27.900105+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:56:39.556251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896490+0000","last_change":"2023-10-21T17:19:25.630004+0000","last_active":"2023-10-21T17:19:27.896490+0000","last_peered":"2023-10-21T17:19:27.896490+0000","last_clean":"2023-10-21T17:19:27.896490+0000","last_became_active":"2023-10-21T17:19:25.629660+0000","last_became_peered":"2023-10-21T17:19:25.629660+0000","last_unstale":"2023-10-21T17:19:27.896490+0000","last_undegraded":"2023-10-21T17:19:27.896490+0000","last_fullsized":"2023-10-21T17:19:27.896490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:49.391349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626165+0000","last_change":"2023-10-21T17:19:26.656542+0000","last_active":"2023-10-21T17:19:27.626165+0000","last_peered":"2023-10-21T17:19:27.626165+0000","last_clean":"2023-10-21T17:19:27.626165+0000","last_became_active":"2023-10-21T17:19:26.653530+0000","last_became_peered":"2023-10-21T17:19:26.653530+0000","last_unstale":"2023-10-21T17:19:27.626165+0000","last_undegraded":"2023-10-21T17:19:27.626165+0000","last_fullsized":"2023-10-21T17:19:27.626165+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:07.917409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900941+0000","last_change":"2023-10-21T17:19:25.634629+0000","last_active":"2023-10-21T17:19:27.900941+0000","last_peered":"2023-10-21T17:19:27.900941+0000","last_clean":"2023-10-21T17:19:27.900941+0000","last_became_active":"2023-10-21T17:19:25.634306+0000","last_became_peered":"2023-10-21T17:19:25.634306+0000","last_unstale":"2023-10-21T17:19:27.900941+0000","last_undegraded":"2023-10-21T17:19:27.900941+0000","last_fullsized":"2023-10-21T17:19:27.900941+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:32:39.760979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897879+0000","last_change":"2023-10-21T17:19:26.648113+0000","last_active":"2023-10-21T17:19:27.897879+0000","last_peered":"2023-10-21T17:19:27.897879+0000","last_clean":"2023-10-21T17:19:27.897879+0000","last_became_active":"2023-10-21T17:19:26.647815+0000","last_became_peered":"2023-10-21T17:19:26.647815+0000","last_unstale":"2023-10-21T17:19:27.897879+0000","last_undegraded":"2023-10-21T17:19:27.897879+0000","last_fullsized":"2023-10-21T17:19:27.897879+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:20.839847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"196'3","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899667+0000","last_change":"2023-10-21T17:19:25.622210+0000","last_active":"2023-10-21T17:19:27.899667+0000","last_peered":"2023-10-21T17:19:27.899667+0000","last_clean":"2023-10-21T17:19:27.899667+0000","last_became_active":"2023-10-21T17:19:25.621949+0000","last_became_peered":"2023-10-21T17:19:25.621949+0000","last_unstale":"2023-10-21T17:19:27.899667+0000","last_undegraded":"2023-10-21T17:19:27.899667+0000","last_fullsized":"2023-10-21T17:19:27.899667+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:19.115567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623801+0000","last_change":"2023-10-21T17:19:26.660957+0000","last_active":"2023-10-21T17:19:27.623801+0000","last_peered":"2023-10-21T17:19:27.623801+0000","last_clean":"2023-10-21T17:19:27.623801+0000","last_became_active":"2023-10-21T17:19:26.659366+0000","last_became_peered":"2023-10-21T17:19:26.659366+0000","last_unstale":"2023-10-21T17:19:27.623801+0000","last_undegraded":"2023-10-21T17:19:27.623801+0000","last_fullsized":"2023-10-21T17:19:27.623801+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:29:26.571178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"196'6","reported_seq":18,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898027+0000","last_change":"2023-10-21T17:19:25.633039+0000","last_active":"2023-10-21T17:19:27.898027+0000","last_peered":"2023-10-21T17:19:27.898027+0000","last_clean":"2023-10-21T17:19:27.898027+0000","last_became_active":"2023-10-21T17:19:25.632746+0000","last_became_peered":"2023-10-21T17:19:25.632746+0000","last_unstale":"2023-10-21T17:19:27.898027+0000","last_undegraded":"2023-10-21T17:19:27.898027+0000","last_fullsized":"2023-10-21T17:19:27.898027+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:11:51.090011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898104+0000","last_change":"2023-10-21T17:19:26.662996+0000","last_active":"2023-10-21T17:19:27.898104+0000","last_peered":"2023-10-21T17:19:27.898104+0000","last_clean":"2023-10-21T17:19:27.898104+0000","last_became_active":"2023-10-21T17:19:26.662423+0000","last_became_peered":"2023-10-21T17:19:26.662423+0000","last_unstale":"2023-10-21T17:19:27.898104+0000","last_undegraded":"2023-10-21T17:19:27.898104+0000","last_fullsized":"2023-10-21T17:19:27.898104+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:42.401111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900741+0000","last_change":"2023-10-21T17:19:25.626771+0000","last_active":"2023-10-21T17:19:27.900741+0000","last_peered":"2023-10-21T17:19:27.900741+0000","last_clean":"2023-10-21T17:19:27.900741+0000","last_became_active":"2023-10-21T17:19:25.626501+0000","last_became_peered":"2023-10-21T17:19:25.626501+0000","last_unstale":"2023-10-21T17:19:27.900741+0000","last_undegraded":"2023-10-21T17:19:27.900741+0000","last_fullsized":"2023-10-21T17:19:27.900741+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:53.535684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900588+0000","last_change":"2023-10-21T17:19:26.663734+0000","last_active":"2023-10-21T17:19:27.900588+0000","last_peered":"2023-10-21T17:19:27.900588+0000","last_clean":"2023-10-21T17:19:27.900588+0000","last_became_active":"2023-10-21T17:19:26.663145+0000","last_became_peered":"2023-10-21T17:19:26.663145+0000","last_unstale":"2023-10-21T17:19:27.900588+0000","last_undegraded":"2023-10-21T17:19:27.900588+0000","last_fullsized":"2023-10-21T17:19:27.900588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:32.867580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898300+0000","last_change":"2023-10-21T17:19:25.628686+0000","last_active":"2023-10-21T17:19:27.898300+0000","last_peered":"2023-10-21T17:19:27.898300+0000","last_clean":"2023-10-21T17:19:27.898300+0000","last_became_active":"2023-10-21T17:19:25.628369+0000","last_became_peered":"2023-10-21T17:19:25.628369+0000","last_unstale":"2023-10-21T17:19:27.898300+0000","last_undegraded":"2023-10-21T17:19:27.898300+0000","last_fullsized":"2023-10-21T17:19:27.898300+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:08.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.625106+0000","last_change":"2023-10-21T17:19:26.653646+0000","last_active":"2023-10-21T17:19:27.625106+0000","last_peered":"2023-10-21T17:19:27.625106+0000","last_clean":"2023-10-21T17:19:27.625106+0000","last_became_active":"2023-10-21T17:19:26.653338+0000","last_became_peered":"2023-10-21T17:19:26.653338+0000","last_unstale":"2023-10-21T17:19:27.625106+0000","last_undegraded":"2023-10-21T17:19:27.625106+0000","last_fullsized":"2023-10-21T17:19:27.625106+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:56.693712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624586+0000","last_change":"2023-10-21T17:19:25.624949+0000","last_active":"2023-10-21T17:19:27.624586+0000","last_peered":"2023-10-21T17:19:27.624586+0000","last_clean":"2023-10-21T17:19:27.624586+0000","last_became_active":"2023-10-21T17:19:25.624679+0000","last_became_peered":"2023-10-21T17:19:25.624679+0000","last_unstale":"2023-10-21T17:19:27.624586+0000","last_undegraded":"2023-10-21T17:19:27.624586+0000","last_fullsized":"2023-10-21T17:19:27.624586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:58:10.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624165+0000","last_change":"2023-10-21T17:19:26.646318+0000","last_active":"2023-10-21T17:19:27.624165+0000","last_peered":"2023-10-21T17:19:27.624165+0000","last_clean":"2023-10-21T17:19:27.624165+0000","last_became_active":"2023-10-21T17:19:26.645811+0000","last_became_peered":"2023-10-21T17:19:26.645811+0000","last_unstale":"2023-10-21T17:19:27.624165+0000","last_undegraded":"2023-10-21T17:19:27.624165+0000","last_fullsized":"2023-10-21T17:19:27.624165+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:25:12.007712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"196'8","reported_seq":20,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896592+0000","last_change":"2023-10-21T17:19:25.626771+0000","last_active":"2023-10-21T17:19:27.896592+0000","last_peered":"2023-10-21T17:19:27.896592+0000","last_clean":"2023-10-21T17:19:27.896592+0000","last_became_active":"2023-10-21T17:19:25.626434+0000","last_became_peered":"2023-10-21T17:19:25.626434+0000","last_unstale":"2023-10-21T17:19:27.896592+0000","last_undegraded":"2023-10-21T17:19:27.896592+0000","last_fullsized":"2023-10-21T17:19:27.896592+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:48.474019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896572+0000","last_change":"2023-10-21T17:19:26.652979+0000","last_active":"2023-10-21T17:19:27.896572+0000","last_peered":"2023-10-21T17:19:27.896572+0000","last_clean":"2023-10-21T17:19:27.896572+0000","last_became_active":"2023-10-21T17:19:26.650878+0000","last_became_peered":"2023-10-21T17:19:26.650878+0000","last_unstale":"2023-10-21T17:19:27.896572+0000","last_undegraded":"2023-10-21T17:19:27.896572+0000","last_fullsized":"2023-10-21T17:19:27.896572+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:35.529983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901029+0000","last_change":"2023-10-21T17:19:25.618828+0000","last_active":"2023-10-21T17:19:27.901029+0000","last_peered":"2023-10-21T17:19:27.901029+0000","last_clean":"2023-10-21T17:19:27.901029+0000","last_became_active":"2023-10-21T17:19:25.618514+0000","last_became_peered":"2023-10-21T17:19:25.618514+0000","last_unstale":"2023-10-21T17:19:27.901029+0000","last_undegraded":"2023-10-21T17:19:27.901029+0000","last_fullsized":"2023-10-21T17:19:27.901029+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:37:06.545630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898258+0000","last_change":"2023-10-21T17:19:26.664664+0000","last_active":"2023-10-21T17:19:27.898258+0000","last_peered":"2023-10-21T17:19:27.898258+0000","last_clean":"2023-10-21T17:19:27.898258+0000","last_became_active":"2023-10-21T17:19:26.664299+0000","last_became_peered":"2023-10-21T17:19:26.664299+0000","last_unstale":"2023-10-21T17:19:27.898258+0000","last_undegraded":"2023-10-21T17:19:27.898258+0000","last_fullsized":"2023-10-21T17:19:27.898258+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:32.314927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901623+0000","last_change":"2023-10-21T17:19:25.634016+0000","last_active":"2023-10-21T17:19:27.901623+0000","last_peered":"2023-10-21T17:19:27.901623+0000","last_clean":"2023-10-21T17:19:27.901623+0000","last_became_active":"2023-10-21T17:19:25.633299+0000","last_became_peered":"2023-10-21T17:19:25.633299+0000","last_unstale":"2023-10-21T17:19:27.901623+0000","last_undegraded":"2023-10-21T17:19:27.901623+0000","last_fullsized":"2023-10-21T17:19:27.901623+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:26:37.918906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624669+0000","last_change":"2023-10-21T17:19:26.639334+0000","last_active":"2023-10-21T17:19:27.624669+0000","last_peered":"2023-10-21T17:19:27.624669+0000","last_clean":"2023-10-21T17:19:27.624669+0000","last_became_active":"2023-10-21T17:19:26.638702+0000","last_became_peered":"2023-10-21T17:19:26.638702+0000","last_unstale":"2023-10-21T17:19:27.624669+0000","last_undegraded":"2023-10-21T17:19:27.624669+0000","last_fullsized":"2023-10-21T17:19:27.624669+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:54:13.594481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899380+0000","last_change":"2023-10-21T17:19:25.621517+0000","last_active":"2023-10-21T17:19:27.899380+0000","last_peered":"2023-10-21T17:19:27.899380+0000","last_clean":"2023-10-21T17:19:27.899380+0000","last_became_active":"2023-10-21T17:19:25.620567+0000","last_became_peered":"2023-10-21T17:19:25.620567+0000","last_unstale":"2023-10-21T17:19:27.899380+0000","last_undegraded":"2023-10-21T17:19:27.899380+0000","last_fullsized":"2023-10-21T17:19:27.899380+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:01:05.561647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624233+0000","last_change":"2023-10-21T17:19:26.654232+0000","last_active":"2023-10-21T17:19:27.624233+0000","last_peered":"2023-10-21T17:19:27.624233+0000","last_clean":"2023-10-21T17:19:27.624233+0000","last_became_active":"2023-10-21T17:19:26.653935+0000","last_became_peered":"2023-10-21T17:19:26.653935+0000","last_unstale":"2023-10-21T17:19:27.624233+0000","last_undegraded":"2023-10-21T17:19:27.624233+0000","last_fullsized":"2023-10-21T17:19:27.624233+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:36.912519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900325+0000","last_change":"2023-10-21T17:19:25.629607+0000","last_active":"2023-10-21T17:19:27.900325+0000","last_peered":"2023-10-21T17:19:27.900325+0000","last_clean":"2023-10-21T17:19:27.900325+0000","last_became_active":"2023-10-21T17:19:25.629301+0000","last_became_peered":"2023-10-21T17:19:25.629301+0000","last_unstale":"2023-10-21T17:19:27.900325+0000","last_undegraded":"2023-10-21T17:19:27.900325+0000","last_fullsized":"2023-10-21T17:19:27.900325+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:34.291382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897929+0000","last_change":"2023-10-21T17:19:26.648390+0000","last_active":"2023-10-21T17:19:27.897929+0000","last_peered":"2023-10-21T17:19:27.897929+0000","last_clean":"2023-10-21T17:19:27.897929+0000","last_became_active":"2023-10-21T17:19:26.647681+0000","last_became_peered":"2023-10-21T17:19:26.647681+0000","last_unstale":"2023-10-21T17:19:27.897929+0000","last_undegraded":"2023-10-21T17:19:27.897929+0000","last_fullsized":"2023-10-21T17:19:27.897929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:57.090952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899827+0000","last_change":"2023-10-21T17:19:25.629030+0000","last_active":"2023-10-21T17:19:27.899827+0000","last_peered":"2023-10-21T17:19:27.899827+0000","last_clean":"2023-10-21T17:19:27.899827+0000","last_became_active":"2023-10-21T17:19:25.627969+0000","last_became_peered":"2023-10-21T17:19:25.627969+0000","last_unstale":"2023-10-21T17:19:27.899827+0000","last_undegraded":"2023-10-21T17:19:27.899827+0000","last_fullsized":"2023-10-21T17:19:27.899827+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:29.649568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898705+0000","last_change":"2023-10-21T17:19:26.655821+0000","last_active":"2023-10-21T17:19:27.898705+0000","last_peered":"2023-10-21T17:19:27.898705+0000","last_clean":"2023-10-21T17:19:27.898705+0000","last_became_active":"2023-10-21T17:19:26.655472+0000","last_became_peered":"2023-10-21T17:19:26.655472+0000","last_unstale":"2023-10-21T17:19:27.898705+0000","last_undegraded":"2023-10-21T17:19:27.898705+0000","last_fullsized":"2023-10-21T17:19:27.898705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:53.220679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624847+0000","last_change":"2023-10-21T17:19:25.623410+0000","last_active":"2023-10-21T17:19:27.624847+0000","last_peered":"2023-10-21T17:19:27.624847+0000","last_clean":"2023-10-21T17:19:27.624847+0000","last_became_active":"2023-10-21T17:19:25.623107+0000","last_became_peered":"2023-10-21T17:19:25.623107+0000","last_unstale":"2023-10-21T17:19:27.624847+0000","last_undegraded":"2023-10-21T17:19:27.624847+0000","last_fullsized":"2023-10-21T17:19:27.624847+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:47.022544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898723+0000","last_change":"2023-10-21T17:19:26.667084+0000","last_active":"2023-10-21T17:19:27.898723+0000","last_peered":"2023-10-21T17:19:27.898723+0000","last_clean":"2023-10-21T17:19:27.898723+0000","last_became_active":"2023-10-21T17:19:26.666671+0000","last_became_peered":"2023-10-21T17:19:26.666671+0000","last_unstale":"2023-10-21T17:19:27.898723+0000","last_undegraded":"2023-10-21T17:19:27.898723+0000","last_fullsized":"2023-10-21T17:19:27.898723+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:11:08.494433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623597+0000","last_change":"2023-10-21T17:19:25.625391+0000","last_active":"2023-10-21T17:19:27.623597+0000","last_peered":"2023-10-21T17:19:27.623597+0000","last_clean":"2023-10-21T17:19:27.623597+0000","last_became_active":"2023-10-21T17:19:25.625121+0000","last_became_peered":"2023-10-21T17:19:25.625121+0000","last_unstale":"2023-10-21T17:19:27.623597+0000","last_undegraded":"2023-10-21T17:19:27.623597+0000","last_fullsized":"2023-10-21T17:19:27.623597+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:04.925541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627673+0000","last_change":"2023-10-21T17:19:26.655274+0000","last_active":"2023-10-21T17:19:27.627673+0000","last_peered":"2023-10-21T17:19:27.627673+0000","last_clean":"2023-10-21T17:19:27.627673+0000","last_became_active":"2023-10-21T17:19:26.654958+0000","last_became_peered":"2023-10-21T17:19:26.654958+0000","last_unstale":"2023-10-21T17:19:27.627673+0000","last_undegraded":"2023-10-21T17:19:27.627673+0000","last_fullsized":"2023-10-21T17:19:27.627673+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:55.160079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626606+0000","last_change":"2023-10-21T17:19:25.631332+0000","last_active":"2023-10-21T17:19:27.626606+0000","last_peered":"2023-10-21T17:19:27.626606+0000","last_clean":"2023-10-21T17:19:27.626606+0000","last_became_active":"2023-10-21T17:19:25.630926+0000","last_became_peered":"2023-10-21T17:19:25.630926+0000","last_unstale":"2023-10-21T17:19:27.626606+0000","last_undegraded":"2023-10-21T17:19:27.626606+0000","last_fullsized":"2023-10-21T17:19:27.626606+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:42.260815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623664+0000","last_change":"2023-10-21T17:19:26.663712+0000","last_active":"2023-10-21T17:19:27.623664+0000","last_peered":"2023-10-21T17:19:27.623664+0000","last_clean":"2023-10-21T17:19:27.623664+0000","last_became_active":"2023-10-21T17:19:26.661440+0000","last_became_peered":"2023-10-21T17:19:26.661440+0000","last_unstale":"2023-10-21T17:19:27.623664+0000","last_undegraded":"2023-10-21T17:19:27.623664+0000","last_fullsized":"2023-10-21T17:19:27.623664+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:08.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899221+0000","last_change":"2023-10-21T17:19:25.628326+0000","last_active":"2023-10-21T17:19:27.899221+0000","last_peered":"2023-10-21T17:19:27.899221+0000","last_clean":"2023-10-21T17:19:27.899221+0000","last_became_active":"2023-10-21T17:19:25.628062+0000","last_became_peered":"2023-10-21T17:19:25.628062+0000","last_unstale":"2023-10-21T17:19:27.899221+0000","last_undegraded":"2023-10-21T17:19:27.899221+0000","last_fullsized":"2023-10-21T17:19:27.899221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:17.439605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897288+0000","last_change":"2023-10-21T17:19:26.651773+0000","last_active":"2023-10-21T17:19:27.897288+0000","last_peered":"2023-10-21T17:19:27.897288+0000","last_clean":"2023-10-21T17:19:27.897288+0000","last_became_active":"2023-10-21T17:19:26.651479+0000","last_became_peered":"2023-10-21T17:19:26.651479+0000","last_unstale":"2023-10-21T17:19:27.897288+0000","last_undegraded":"2023-10-21T17:19:27.897288+0000","last_fullsized":"2023-10-21T17:19:27.897288+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:37.781098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900711+0000","last_change":"2023-10-21T17:19:25.636689+0000","last_active":"2023-10-21T17:19:27.900711+0000","last_peered":"2023-10-21T17:19:27.900711+0000","last_clean":"2023-10-21T17:19:27.900711+0000","last_became_active":"2023-10-21T17:19:25.636402+0000","last_became_peered":"2023-10-21T17:19:25.636402+0000","last_unstale":"2023-10-21T17:19:27.900711+0000","last_undegraded":"2023-10-21T17:19:27.900711+0000","last_fullsized":"2023-10-21T17:19:27.900711+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:37.591035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623654+0000","last_change":"2023-10-21T17:19:26.654287+0000","last_active":"2023-10-21T17:19:27.623654+0000","last_peered":"2023-10-21T17:19:27.623654+0000","last_clean":"2023-10-21T17:19:27.623654+0000","last_became_active":"2023-10-21T17:19:26.654004+0000","last_became_peered":"2023-10-21T17:19:26.654004+0000","last_unstale":"2023-10-21T17:19:27.623654+0000","last_undegraded":"2023-10-21T17:19:27.623654+0000","last_fullsized":"2023-10-21T17:19:27.623654+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:45.996748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900218+0000","last_change":"2023-10-21T17:19:25.628646+0000","last_active":"2023-10-21T17:19:27.900218+0000","last_peered":"2023-10-21T17:19:27.900218+0000","last_clean":"2023-10-21T17:19:27.900218+0000","last_became_active":"2023-10-21T17:19:25.628371+0000","last_became_peered":"2023-10-21T17:19:25.628371+0000","last_unstale":"2023-10-21T17:19:27.900218+0000","last_undegraded":"2023-10-21T17:19:27.900218+0000","last_fullsized":"2023-10-21T17:19:27.900218+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:04:51.053443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897847+0000","last_change":"2023-10-21T17:19:26.657534+0000","last_active":"2023-10-21T17:19:27.897847+0000","last_peered":"2023-10-21T17:19:27.897847+0000","last_clean":"2023-10-21T17:19:27.897847+0000","last_became_active":"2023-10-21T17:19:26.655656+0000","last_became_peered":"2023-10-21T17:19:26.655656+0000","last_unstale":"2023-10-21T17:19:27.897847+0000","last_undegraded":"2023-10-21T17:19:27.897847+0000","last_fullsized":"2023-10-21T17:19:27.897847+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:13:55.194766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623971+0000","last_change":"2023-10-21T17:19:25.630873+0000","last_active":"2023-10-21T17:19:27.623971+0000","last_peered":"2023-10-21T17:19:27.623971+0000","last_clean":"2023-10-21T17:19:27.623971+0000","last_became_active":"2023-10-21T17:19:25.630560+0000","last_became_peered":"2023-10-21T17:19:25.630560+0000","last_unstale":"2023-10-21T17:19:27.623971+0000","last_undegraded":"2023-10-21T17:19:27.623971+0000","last_fullsized":"2023-10-21T17:19:27.623971+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:45:03.818326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900067+0000","last_change":"2023-10-21T17:19:26.659594+0000","last_active":"2023-10-21T17:19:27.900067+0000","last_peered":"2023-10-21T17:19:27.900067+0000","last_clean":"2023-10-21T17:19:27.900067+0000","last_became_active":"2023-10-21T17:19:26.658634+0000","last_became_peered":"2023-10-21T17:19:26.658634+0000","last_unstale":"2023-10-21T17:19:27.900067+0000","last_undegraded":"2023-10-21T17:19:27.900067+0000","last_fullsized":"2023-10-21T17:19:27.900067+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:02.684095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899269+0000","last_change":"2023-10-21T17:19:25.618289+0000","last_active":"2023-10-21T17:19:27.899269+0000","last_peered":"2023-10-21T17:19:27.899269+0000","last_clean":"2023-10-21T17:19:27.899269+0000","last_became_active":"2023-10-21T17:19:25.617765+0000","last_became_peered":"2023-10-21T17:19:25.617765+0000","last_unstale":"2023-10-21T17:19:27.899269+0000","last_undegraded":"2023-10-21T17:19:27.899269+0000","last_fullsized":"2023-10-21T17:19:27.899269+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627873+0000","last_change":"2023-10-21T17:19:26.655938+0000","last_active":"2023-10-21T17:19:27.627873+0000","last_peered":"2023-10-21T17:19:27.627873+0000","last_clean":"2023-10-21T17:19:27.627873+0000","last_became_active":"2023-10-21T17:19:26.655227+0000","last_became_peered":"2023-10-21T17:19:26.655227+0000","last_unstale":"2023-10-21T17:19:27.627873+0000","last_undegraded":"2023-10-21T17:19:27.627873+0000","last_fullsized":"2023-10-21T17:19:27.627873+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:56.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898518+0000","last_change":"2023-10-21T17:19:25.622295+0000","last_active":"2023-10-21T17:19:27.898518+0000","last_peered":"2023-10-21T17:19:27.898518+0000","last_clean":"2023-10-21T17:19:27.898518+0000","last_became_active":"2023-10-21T17:19:25.622016+0000","last_became_peered":"2023-10-21T17:19:25.622016+0000","last_unstale":"2023-10-21T17:19:27.898518+0000","last_undegraded":"2023-10-21T17:19:27.898518+0000","last_fullsized":"2023-10-21T17:19:27.898518+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:36.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900467+0000","last_change":"2023-10-21T17:19:26.642886+0000","last_active":"2023-10-21T17:19:27.900467+0000","last_peered":"2023-10-21T17:19:27.900467+0000","last_clean":"2023-10-21T17:19:27.900467+0000","last_became_active":"2023-10-21T17:19:26.642510+0000","last_became_peered":"2023-10-21T17:19:26.642510+0000","last_unstale":"2023-10-21T17:19:27.900467+0000","last_undegraded":"2023-10-21T17:19:27.900467+0000","last_fullsized":"2023-10-21T17:19:27.900467+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:10:06.936997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624783+0000","last_change":"2023-10-21T17:19:25.608862+0000","last_active":"2023-10-21T17:19:27.624783+0000","last_peered":"2023-10-21T17:19:27.624783+0000","last_clean":"2023-10-21T17:19:27.624783+0000","last_became_active":"2023-10-21T17:19:25.608001+0000","last_became_peered":"2023-10-21T17:19:25.608001+0000","last_unstale":"2023-10-21T17:19:27.624783+0000","last_undegraded":"2023-10-21T17:19:27.624783+0000","last_fullsized":"2023-10-21T17:19:27.624783+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:06.904040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624363+0000","last_change":"2023-10-21T17:19:26.660439+0000","last_active":"2023-10-21T17:19:27.624363+0000","last_peered":"2023-10-21T17:19:27.624363+0000","last_clean":"2023-10-21T17:19:27.624363+0000","last_became_active":"2023-10-21T17:19:26.660143+0000","last_became_peered":"2023-10-21T17:19:26.660143+0000","last_unstale":"2023-10-21T17:19:27.624363+0000","last_undegraded":"2023-10-21T17:19:27.624363+0000","last_fullsized":"2023-10-21T17:19:27.624363+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:43.707718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899569+0000","last_change":"2023-10-21T17:19:25.628158+0000","last_active":"2023-10-21T17:19:27.899569+0000","last_peered":"2023-10-21T17:19:27.899569+0000","last_clean":"2023-10-21T17:19:27.899569+0000","last_became_active":"2023-10-21T17:19:25.627779+0000","last_became_peered":"2023-10-21T17:19:25.627779+0000","last_unstale":"2023-10-21T17:19:27.899569+0000","last_undegraded":"2023-10-21T17:19:27.899569+0000","last_fullsized":"2023-10-21T17:19:27.899569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:32:35.760176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626336+0000","last_change":"2023-10-21T17:19:26.666341+0000","last_active":"2023-10-21T17:19:27.626336+0000","last_peered":"2023-10-21T17:19:27.626336+0000","last_clean":"2023-10-21T17:19:27.626336+0000","last_became_active":"2023-10-21T17:19:26.666044+0000","last_became_peered":"2023-10-21T17:19:26.666044+0000","last_unstale":"2023-10-21T17:19:27.626336+0000","last_undegraded":"2023-10-21T17:19:27.626336+0000","last_fullsized":"2023-10-21T17:19:27.626336+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:43:00.990344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900709+0000","last_change":"2023-10-21T17:19:25.635439+0000","last_active":"2023-10-21T17:19:27.900709+0000","last_peered":"2023-10-21T17:19:27.900709+0000","last_clean":"2023-10-21T17:19:27.900709+0000","last_became_active":"2023-10-21T17:19:25.635177+0000","last_became_peered":"2023-10-21T17:19:25.635177+0000","last_unstale":"2023-10-21T17:19:27.900709+0000","last_undegraded":"2023-10-21T17:19:27.900709+0000","last_fullsized":"2023-10-21T17:19:27.900709+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:30.032047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"186'4818","reported_seq":5565,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-21T17:19:26.642247+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:19:26.642247+0000","last_peered":"2023-10-21T17:19:26.642247+0000","last_clean":"2023-10-21T17:19:26.642247+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:19:26.642247+0000","last_undegraded":"2023-10-21T17:19:26.642247+0000","last_fullsized":"2023-10-21T17:19:26.642247+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4818,"log_dups_size":0,"ondisk_log_size":4818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":487,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":487,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4818,"ondisk_log_size":4818,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575772,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3059712,"data_stored":1855221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.84099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5920000000000001,"5min":1.841,"15min":1.9430000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.65900000000000003},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2130000000000001,"15min":1.2370000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.68300000000000005},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.751},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.272,"15min":1.276},"min":{"1min":0.93700000000000006,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.839,"5min":1.839,"15min":2.081},"last":0.68999999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3440000000000001,"15min":1.349},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.589,"5min":1.8140000000000001,"15min":2.1440000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.335,"5min":1.357,"15min":1.365},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88300000000000001},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.83899999999999997}]},{"osd":3,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.304,"15min":1.306},"min":{"1min":1.0840000000000001,"5min":0.94999999999999996,"15min":0.81699999999999995},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.466,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":0.95199999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.755},"max":{"1min":1.6639999999999999,"5min":1.865,"15min":2.1269999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4650000000000001,"15min":1.484},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85899999999999999},"max":{"1min":1.7769999999999999,"5min":1.8540000000000001,"15min":2.0830000000000002},"last":1.01}]},{"osd":5,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.80100000000000005},"max":{"1min":2.089,"5min":2.089,"15min":2.1560000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.52,"15min":1.5049999999999999},"min":{"1min":1.1439999999999999,"5min":0.93999999999999995,"15min":0.91900000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.0349999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5940000000000001,"15min":1.5920000000000001},"min":{"1min":1.2090000000000001,"5min":0.996,"15min":0.92400000000000004},"max":{"1min":1.9610000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.083},{"interface":"front","average":{"1min":1.446,"5min":1.4890000000000001,"15min":1.522},"min":{"1min":1.0680000000000001,"5min":1,"15min":0.92900000000000005},"max":{"1min":1.895,"5min":2.0150000000000001,"15min":2.1970000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575773,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41908,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329932,"statfs":{"total":96636764160,"available":96593850368,"internally_reserved":0,"allocated":2465792,"data_stored":1264853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.206},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.212,"5min":1.2210000000000001,"15min":1.262},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.8680000000000001},"last":1.3999999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.268,"15min":1.244},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2849999999999999,"15min":1.3200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.016}]},{"osd":2,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3069999999999999,"15min":1.3129999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8819999999999999,"5min":1.9970000000000001,"15min":2.0659999999999998},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6970000000000001,"5min":1.7969999999999999,"15min":2.415},"last":1.254}]},{"osd":3,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.383,"15min":1.391},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.468,"15min":1.466},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.6509999999999998},"last":1.4370000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3660000000000001,"15min":1.363},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.63500000000000001},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.367,"5min":1.429,"15min":1.403},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7769999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.575,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.9370000000000001,"5min":2.1019999999999999,"15min":2.5419999999999998},"last":1.9350000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.429,"15min":1.3839999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.016},"last":1.546}]},{"osd":7,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5129999999999999,"15min":1.524},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8200000000000001,"5min":2.02,"15min":2.4049999999999998},"last":1.732},{"interface":"front","average":{"1min":1.621,"5min":1.6830000000000001,"15min":1.651},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.129,"5min":2.3210000000000002,"15min":3.4180000000000001},"last":1.8779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1264977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.143,"15min":1.143},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"max":{"1min":1.6140000000000001,"5min":1.718,"15min":1.7649999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.9079999999999999},"last":1.6659999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.264,"15min":1.2909999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6259999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.5},{"interface":"front","average":{"1min":1.145,"5min":1.234,"15min":1.25},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8120000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.91600000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.349,"15min":1.3400000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.742,"5min":1.7789999999999999,"15min":1.927},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.252,"15min":1.2729999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.76000000000000001},"max":{"1min":1.909,"5min":1.909,"15min":2.2829999999999999},"last":1.448}]},{"osd":3,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.377,"15min":1.385},"min":{"1min":0.996,"5min":0.996,"15min":0.84999999999999998},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.367,"15min":1.4179999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.5449999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.2509999999999999,"15min":1.248},"min":{"1min":0.92400000000000004,"5min":0.752,"15min":0.752},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1800000000000002},"last":1.304},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3799999999999999,"15min":1.3859999999999999},"min":{"1min":0.88700000000000001,"5min":0.77300000000000002,"15min":0.70199999999999996},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.379}]},{"osd":6,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.454,"15min":1.484},"min":{"1min":1.125,"5min":0.90000000000000002,"15min":0.89600000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.335},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.518,"15min":1.5129999999999999},"min":{"1min":1.0289999999999999,"5min":0.74199999999999999,"15min":0.67000000000000004},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3479999999999999},"last":1.569}]},{"osd":7,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.548,"15min":1.5569999999999999},"min":{"1min":1.004,"5min":1.004,"15min":0.79000000000000004},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.724},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5649999999999999,"15min":1.569},"min":{"1min":1.25,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.3069999999999999},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834575772,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29452,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342388,"statfs":{"total":96636764160,"available":96606605312,"internally_reserved":0,"allocated":2482176,"data_stored":1265617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1879999999999999,"15min":1.1870000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6200000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.024},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.23},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.78600000000000003},"max":{"1min":1.627,"5min":1.907,"15min":1.907},"last":1.375}]},{"osd":1,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2989999999999999,"15min":1.2809999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.621,"5min":1.6479999999999999,"15min":1.855},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5780000000000001,"5min":1.794,"15min":1.9139999999999999},"last":1.1830000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.232,"15min":1.2509999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.643,"5min":1.6719999999999999,"15min":1.821},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.391,"15min":1.3839999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.653,"5min":1.923,"15min":1.923},"last":1.4399999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.3440000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.82799999999999996},"max":{"1min":1.6479999999999999,"5min":1.877,"15min":1.877},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.399,"15min":1.407},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5740000000000001,"5min":1.8740000000000001,"15min":1.9299999999999999},"last":1.3120000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3360000000000001,"15min":1.3149999999999999},"min":{"1min":1.052,"5min":0.88400000000000001,"15min":0.83199999999999996},"max":{"1min":1.6819999999999999,"5min":1.802,"15min":1.895},"last":1.246},{"interface":"front","average":{"1min":1.329,"5min":1.3500000000000001,"15min":1.347},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7649999999999999,"5min":1.944,"15min":1.9870000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4159999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.8420000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.6080000000000001,"15min":1.5920000000000001},"min":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":0.85599999999999998},"max":{"1min":1.863,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.498}]},{"osd":7,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5600000000000001,"15min":1.5249999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8420000000000001,"5min":2.1110000000000002,"15min":2.2429999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.6080000000000001,"15min":1.5800000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.8320000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.4670000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51660,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320180,"statfs":{"total":96636764160,"available":96583864320,"internally_reserved":0,"allocated":2486272,"data_stored":1267069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.92600000000000005,"15min":0.95499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.2330000000000001,"15min":1.2669999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4259999999999999,"5min":1.766,"15min":2.121},"last":1.292}]},{"osd":1,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.073,"15min":1.083},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.331,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.2589999999999999,"15min":1.2909999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.7130000000000001,"5min":1.766,"15min":1.766},"last":1.242}]},{"osd":2,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.093,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6080000000000001,"5min":1.784,"15min":1.9790000000000001},"last":1.423},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.26,"15min":1.284},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.056}]},{"osd":4,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3959999999999999,"15min":1.4530000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.73499999999999999},"max":{"1min":1.702,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.647},{"interface":"front","average":{"1min":1.304,"5min":1.4670000000000001,"15min":1.472},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.5469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4339999999999999,"15min":1.476},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8100000000000001,"5min":2.2210000000000001,"15min":2.3239999999999998},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.2450000000000001},"last":1.7609999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.5680000000000001,"15min":1.593},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1970000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.923},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.55,"15min":1.579},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.88500000000000001},"max":{"1min":2.1659999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.696}]},{"osd":7,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.645,"15min":1.667},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.97399999999999998},"max":{"1min":2.3119999999999998,"5min":2.3319999999999999,"15min":2.4239999999999999},"last":1.958},{"interface":"front","average":{"1min":1.484,"5min":1.694,"15min":1.71},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.996}]}]},{"osd":2,"up_from":13,"seq":55834575773,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51644,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320196,"statfs":{"total":96636764160,"available":96583880704,"internally_reserved":0,"allocated":2465792,"data_stored":1264853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.89700000000000002,"15min":0.90200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.891,"5min":1.891,"15min":2.3199999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.036,"15min":1.0569999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6639999999999999,"5min":1.835,"15min":1.875},"last":1.1140000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.179,"15min":1.1830000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.087,"5min":1.3300000000000001,"15min":1.3859999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.1070000000000002},"last":1.3560000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.179,"15min":1.2110000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":1.075},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4510000000000001,"15min":1.478},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.6480000000000001,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.425,"15min":1.4079999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":3.0510000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4099999999999999,"15min":1.4530000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8899999999999999,"5min":2.0489999999999999,"15min":2.0579999999999998},"last":1.853}]},{"osd":5,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.421,"15min":1.415},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.599,"15min":1.6120000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":3.2050000000000001},"last":1.571}]},{"osd":6,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5489999999999999,"15min":1.556},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0259999999999998,"5min":2.21,"15min":2.25},"last":2.089},{"interface":"front","average":{"1min":1.395,"5min":1.4950000000000001,"15min":1.5289999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.429}]},{"osd":7,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.6000000000000001,"15min":1.6419999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.351,"5min":2.351,"15min":2.8959999999999999},"last":1.978},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5740000000000001,"15min":1.597},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.331}]}]},{"osd":1,"up_from":13,"seq":55834575772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29444,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342396,"statfs":{"total":96636764160,"available":96606613504,"internally_reserved":0,"allocated":2478080,"data_stored":1266465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.94399999999999995,"15min":0.94899999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.7330000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3999999999999999,"5min":1.966,"15min":1.966},"last":1.274}]},{"osd":2,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0469999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.302,"5min":1.724,"15min":2.79},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.25,"15min":1.24},"min":{"1min":0.54300000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.748,"5min":1.891,"15min":2.2549999999999999},"last":1.222}]},{"osd":3,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.3300000000000001,"15min":1.375},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7749999999999999,"5min":1.9339999999999999,"15min":3.0920000000000001},"last":1.085},{"interface":"front","average":{"1min":1.139,"5min":1.228,"15min":1.274},"min":{"1min":0.628,"5min":0.44,"15min":0.44},"max":{"1min":1.968,"5min":1.968,"15min":1.976},"last":1.0189999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.347,"15min":1.319},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.806,"5min":2.105,"15min":2.105},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4950000000000001,"15min":1.51},"min":{"1min":0.86099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1190000000000002,"5min":2.2410000000000001,"15min":2.512},"last":1.385}]},{"osd":5,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5429999999999999},"min":{"1min":0.90000000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.0329999999999999,"5min":2.246,"15min":2.246},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.5620000000000001,"15min":1.5489999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.0059999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.357}]},{"osd":6,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.569,"15min":1.5629999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0739999999999998,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.575,"15min":1.552},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.911,"5min":2.214,"15min":2.2570000000000001},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.6599999999999999,"15min":1.6739999999999999},"min":{"1min":0.97199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.254,"5min":2.3220000000000001,"15min":2.391},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.645,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.99099999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.1429999999999998,"5min":2.4769999999999999,"15min":2.4969999999999999},"last":1.907}]}]},{"osd":0,"up_from":13,"seq":55834575772,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30024,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341816,"statfs":{"total":96636764160,"available":96606019584,"internally_reserved":0,"allocated":3067904,"data_stored":1855325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96799999999999997,"15min":0.95299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5840000000000001,"5min":1.631,"15min":1.726},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.7989999999999999,"15min":2.6200000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1679999999999999,"15min":1.171},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.7330000000000001,"15min":2.8140000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.204,"15min":1.2070000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.46,"5min":1.7170000000000001,"15min":1.784},"last":1.143}]},{"osd":3,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.1060000000000001,"15min":1.147},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.718,"5min":1.883,"15min":1.8919999999999999},"last":0.752},{"interface":"front","average":{"1min":1.258,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6539999999999999,"5min":1.9239999999999999,"15min":2.6949999999999998},"last":1.2250000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.6299999999999999,"15min":1.619},"min":{"1min":1.278,"5min":1.278,"15min":1.1799999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.766},"last":1.4570000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.96199999999999997},"max":{"1min":1.7729999999999999,"5min":2.125,"15min":2.6549999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5660000000000001,"15min":1.5600000000000001},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.952,"5min":2.3300000000000001,"15min":2.9209999999999998},"last":1.55}]},{"osd":6,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5229999999999999,"15min":1.5129999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2759999999999998},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.6990000000000001,"15min":1.665},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0230000000000001,"5min":2.4510000000000001,"15min":3.069},"last":1.6100000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.556,"15min":1.556},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.794,"5min":2.0910000000000002,"15min":2.4790000000000001},"last":1.752},{"interface":"front","average":{"1min":1.504,"5min":1.639,"15min":1.631},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.0529999999999999,"5min":2.1930000000000001,"15min":2.613},"last":1.4019999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:19:31.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:19:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.609+0000 7f904b190640 1 Processor -- start 2023-10-21T17:19:31.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.610+0000 7f904b190640 1 -- start start 2023-10-21T17:19:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.611+0000 7f904b190640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 0x7f90440ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.611+0000 7f904b190640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.611+0000 7f904b190640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904415b2b0 con 0x7f90440cae70 2023-10-21T17:19:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.611+0000 7f904b190640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904415bf30 con 0x7f904415ab40 2023-10-21T17:19:31.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.611+0000 7f904b190640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904415f7a0 con 0x7f90440c90b0 2023-10-21T17:19:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.612+0000 7f9043fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.612+0000 7f9048f2c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 0x7f90440ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.612+0000 7f9043fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:43158/0 (socket says 172.21.15.53:43158) 2023-10-21T17:19:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.612+0000 7f9043fff640 1 -- 172.21.15.53:0/1250505679 learned_addr learned my addr 172.21.15.53:0/1250505679 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:31.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.612+0000 7f9048f2c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 0x7f90440ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:45342/0 (socket says 172.21.15.53:45342) 2023-10-21T17:19:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.613+0000 7f9043fff640 1 -- 172.21.15.53:0/1250505679 >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 msgr2=0x7f90440ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.613+0000 7f9043fff640 1 --2- 172.21.15.53:0/1250505679 >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 0x7f90440ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.613+0000 7f9043fff640 1 -- 172.21.15.53:0/1250505679 >> v1:172.21.15.124:6789/0 conn(0x7f90440cae70 legacy=0x7f90440702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.613+0000 7f9043fff640 1 -- 172.21.15.53:0/1250505679 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904415b590 con 0x7f90440c90b0 2023-10-21T17:19:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.614+0000 7f9043fff640 1 --2- 172.21.15.53:0/1250505679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9030009870 tx=0x7f903002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cbbd7480d76c3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.615+0000 7f90437fe640 1 -- 172.21.15.53:0/1250505679 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030030840 con 0x7f90440c90b0 2023-10-21T17:19:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.615+0000 7f90437fe640 1 -- 172.21.15.53:0/1250505679 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90300309a0 con 0x7f90440c90b0 2023-10-21T17:19:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.615+0000 7f90437fe640 1 -- 172.21.15.53:0/1250505679 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030030d30 con 0x7f90440c90b0 2023-10-21T17:19:31.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 -- 172.21.15.53:0/1250505679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 msgr2=0x7f90440adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 --2- 172.21.15.53:0/1250505679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9030009870 tx=0x7f903002b980 comp rx=0 tx=0).stop 2023-10-21T17:19:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 -- 172.21.15.53:0/1250505679 shutdown_connections 2023-10-21T17:19:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 --2- 172.21.15.53:0/1250505679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f90440adc60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 --2- 172.21.15.53:0/1250505679 >> v2:172.21.15.124:3300/0 conn(0x7f904415ab40 0x7f90440ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 -- 172.21.15.53:0/1250505679 >> 172.21.15.53:0/1250505679 conn(0x7f904406a380 msgr2=0x7f90440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 -- 172.21.15.53:0/1250505679 shutdown_connections 2023-10-21T17:19:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.616+0000 7f904b190640 1 -- 172.21.15.53:0/1250505679 wait complete. 2023-10-21T17:19:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.618+0000 7f904b190640 1 Processor -- start 2023-10-21T17:19:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.618+0000 7f904b190640 1 -- start start 2023-10-21T17:19:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f904415d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 0x7f9044159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044159a40 con 0x7f90440cae70 2023-10-21T17:19:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044156c60 con 0x7f904415ab40 2023-10-21T17:19:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f904b190640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90441566a0 con 0x7f90440c90b0 2023-10-21T17:19:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f9043fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f9043fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34424/0 (socket says 172.21.15.53:34424) 2023-10-21T17:19:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.619+0000 7f9043fff640 1 -- 172.21.15.53:0/2000296362 learned_addr learned my addr 172.21.15.53:0/2000296362 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9048f2c640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f904415d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f904972d640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 0x7f9044159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 -- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 msgr2=0x7f904415d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f904415d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 -- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 msgr2=0x7f9044159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 0x7f9044159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 -- 172.21.15.53:0/2000296362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300094d0 con 0x7f90440cae70 2023-10-21T17:19:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f904972d640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 0x7f9044159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.620+0000 7f9043fff640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.621+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030039a90 con 0x7f90440cae70 2023-10-21T17:19:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.621+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9030039bf0 con 0x7f90440cae70 2023-10-21T17:19:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.621+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9044156980 con 0x7f90440cae70 2023-10-21T17:19:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.621+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030043410 con 0x7f90440cae70 2023-10-21T17:19:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.621+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9044158fe0 con 0x7f90440cae70 2023-10-21T17:19:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.622+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90441591d0 con 0x7f90440cae70 2023-10-21T17:19:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.623+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f903004c3b0 con 0x7f90440cae70 2023-10-21T17:19:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.624+0000 7f90417fa640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 0x7f902806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.624+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f90300abf10 con 0x7f90440cae70 2023-10-21T17:19:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.625+0000 7f9048f2c640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 0x7f902806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.626+0000 7f9048f2c640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 0x7f902806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ad0 tx=0x7f9038008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.629+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9030076b40 con 0x7f90440cae70 2023-10-21T17:19:31.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.818+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 --> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f904415bb50 con 0x7f902806af90 2023-10-21T17:19:31.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.841+0000 7f90417fa640 1 -- 172.21.15.53:0/2000296362 <== mgr.37184 v2:172.21.15.124:6832/1955466907 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287261 (secure 0 0 0) 0x7f90380119a0 con 0x7f902806af90 2023-10-21T17:19:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:31.848 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:19:31.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 msgr2=0x7f902806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 0x7f902806d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9038009ad0 tx=0x7f9038008040 comp rx=0 tx=0).stop 2023-10-21T17:19:31.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 msgr2=0x7f904414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f9030000c00 tx=0x7f9030004730 comp rx=0 tx=0).stop 2023-10-21T17:19:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 shutdown_connections 2023-10-21T17:19:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f904415ab40 0x7f9044159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f902806af90 0x7f902806d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90440cae70 0x7f904414ba70 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 --2- 172.21.15.53:0/2000296362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90440c90b0 0x7f904415d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 >> 172.21.15.53:0/2000296362 conn(0x7f904406a380 msgr2=0x7f90440cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 shutdown_connections 2023-10-21T17:19:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:31.849+0000 7f904b190640 1 -- 172.21.15.53:0/2000296362 wait complete. 2023-10-21T17:19:31.874 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-21T17:19:29.986891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":531,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4856,"ondisk_log_size":4856,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":346312,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":325683,"kb_avail":754628408,"statfs":{"total":773094113280,"available":772739489792,"internally_reserved":0,"allocated":20979712,"data_stored":11304380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":333499961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400629"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900066+0000","last_change":"2023-10-21T17:19:26.658109+0000","last_active":"2023-10-21T17:19:27.900066+0000","last_peered":"2023-10-21T17:19:27.900066+0000","last_clean":"2023-10-21T17:19:27.900066+0000","last_became_active":"2023-10-21T17:19:26.656933+0000","last_became_peered":"2023-10-21T17:19:26.656933+0000","last_unstale":"2023-10-21T17:19:27.900066+0000","last_undegraded":"2023-10-21T17:19:27.900066+0000","last_fullsized":"2023-10-21T17:19:27.900066+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:59.526154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627478+0000","last_change":"2023-10-21T17:19:26.659020+0000","last_active":"2023-10-21T17:19:27.627478+0000","last_peered":"2023-10-21T17:19:27.627478+0000","last_clean":"2023-10-21T17:19:27.627478+0000","last_became_active":"2023-10-21T17:19:26.658750+0000","last_became_peered":"2023-10-21T17:19:26.658750+0000","last_unstale":"2023-10-21T17:19:27.627478+0000","last_undegraded":"2023-10-21T17:19:27.627478+0000","last_fullsized":"2023-10-21T17:19:27.627478+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:08.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899613+0000","last_change":"2023-10-21T17:19:26.644243+0000","last_active":"2023-10-21T17:19:27.899613+0000","last_peered":"2023-10-21T17:19:27.899613+0000","last_clean":"2023-10-21T17:19:27.899613+0000","last_became_active":"2023-10-21T17:19:26.643819+0000","last_became_peered":"2023-10-21T17:19:26.643819+0000","last_unstale":"2023-10-21T17:19:27.899613+0000","last_undegraded":"2023-10-21T17:19:27.899613+0000","last_fullsized":"2023-10-21T17:19:27.899613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:05.835440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899729+0000","last_change":"2023-10-21T17:19:26.645519+0000","last_active":"2023-10-21T17:19:27.899729+0000","last_peered":"2023-10-21T17:19:27.899729+0000","last_clean":"2023-10-21T17:19:27.899729+0000","last_became_active":"2023-10-21T17:19:26.645220+0000","last_became_peered":"2023-10-21T17:19:26.645220+0000","last_unstale":"2023-10-21T17:19:27.899729+0000","last_undegraded":"2023-10-21T17:19:27.899729+0000","last_fullsized":"2023-10-21T17:19:27.899729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:19.142990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899116+0000","last_change":"2023-10-21T17:19:26.660273+0000","last_active":"2023-10-21T17:19:27.899116+0000","last_peered":"2023-10-21T17:19:27.899116+0000","last_clean":"2023-10-21T17:19:27.899116+0000","last_became_active":"2023-10-21T17:19:26.660015+0000","last_became_peered":"2023-10-21T17:19:26.660015+0000","last_unstale":"2023-10-21T17:19:27.899116+0000","last_undegraded":"2023-10-21T17:19:27.899116+0000","last_fullsized":"2023-10-21T17:19:27.899116+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:22.831978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626524+0000","last_change":"2023-10-21T17:19:26.667811+0000","last_active":"2023-10-21T17:19:27.626524+0000","last_peered":"2023-10-21T17:19:27.626524+0000","last_clean":"2023-10-21T17:19:27.626524+0000","last_became_active":"2023-10-21T17:19:26.667501+0000","last_became_peered":"2023-10-21T17:19:26.667501+0000","last_unstale":"2023-10-21T17:19:27.626524+0000","last_undegraded":"2023-10-21T17:19:27.626524+0000","last_fullsized":"2023-10-21T17:19:27.626524+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:48.035914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896945+0000","last_change":"2023-10-21T17:19:26.658879+0000","last_active":"2023-10-21T17:19:27.896945+0000","last_peered":"2023-10-21T17:19:27.896945+0000","last_clean":"2023-10-21T17:19:27.896945+0000","last_became_active":"2023-10-21T17:19:26.653573+0000","last_became_peered":"2023-10-21T17:19:26.653573+0000","last_unstale":"2023-10-21T17:19:27.896945+0000","last_undegraded":"2023-10-21T17:19:27.896945+0000","last_fullsized":"2023-10-21T17:19:27.896945+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896390+0000","last_change":"2023-10-21T17:19:26.661762+0000","last_active":"2023-10-21T17:19:27.896390+0000","last_peered":"2023-10-21T17:19:27.896390+0000","last_clean":"2023-10-21T17:19:27.896390+0000","last_became_active":"2023-10-21T17:19:26.658627+0000","last_became_peered":"2023-10-21T17:19:26.658627+0000","last_unstale":"2023-10-21T17:19:27.896390+0000","last_undegraded":"2023-10-21T17:19:27.896390+0000","last_fullsized":"2023-10-21T17:19:27.896390+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:23:22.849050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901280+0000","last_change":"2023-10-21T17:19:26.660406+0000","last_active":"2023-10-21T17:19:27.901280+0000","last_peered":"2023-10-21T17:19:27.901280+0000","last_clean":"2023-10-21T17:19:27.901280+0000","last_became_active":"2023-10-21T17:19:26.660097+0000","last_became_peered":"2023-10-21T17:19:26.660097+0000","last_unstale":"2023-10-21T17:19:27.901280+0000","last_undegraded":"2023-10-21T17:19:27.901280+0000","last_fullsized":"2023-10-21T17:19:27.901280+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:36:14.756079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901775+0000","last_change":"2023-10-21T17:19:26.660982+0000","last_active":"2023-10-21T17:19:27.901775+0000","last_peered":"2023-10-21T17:19:27.901775+0000","last_clean":"2023-10-21T17:19:27.901775+0000","last_became_active":"2023-10-21T17:19:26.660678+0000","last_became_peered":"2023-10-21T17:19:26.660678+0000","last_unstale":"2023-10-21T17:19:27.901775+0000","last_undegraded":"2023-10-21T17:19:27.901775+0000","last_fullsized":"2023-10-21T17:19:27.901775+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:52:25.806442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624270+0000","last_change":"2023-10-21T17:19:26.659014+0000","last_active":"2023-10-21T17:19:27.624270+0000","last_peered":"2023-10-21T17:19:27.624270+0000","last_clean":"2023-10-21T17:19:27.624270+0000","last_became_active":"2023-10-21T17:19:26.658234+0000","last_became_peered":"2023-10-21T17:19:26.658234+0000","last_unstale":"2023-10-21T17:19:27.624270+0000","last_undegraded":"2023-10-21T17:19:27.624270+0000","last_fullsized":"2023-10-21T17:19:27.624270+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:04.562111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627688+0000","last_change":"2023-10-21T17:19:26.667654+0000","last_active":"2023-10-21T17:19:27.627688+0000","last_peered":"2023-10-21T17:19:27.627688+0000","last_clean":"2023-10-21T17:19:27.627688+0000","last_became_active":"2023-10-21T17:19:26.667357+0000","last_became_peered":"2023-10-21T17:19:26.667357+0000","last_unstale":"2023-10-21T17:19:27.627688+0000","last_undegraded":"2023-10-21T17:19:27.627688+0000","last_fullsized":"2023-10-21T17:19:27.627688+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:36.912519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899883+0000","last_change":"2023-10-21T17:19:26.654520+0000","last_active":"2023-10-21T17:19:27.899883+0000","last_peered":"2023-10-21T17:19:27.899883+0000","last_clean":"2023-10-21T17:19:27.899883+0000","last_became_active":"2023-10-21T17:19:26.652767+0000","last_became_peered":"2023-10-21T17:19:26.652767+0000","last_unstale":"2023-10-21T17:19:27.899883+0000","last_undegraded":"2023-10-21T17:19:27.899883+0000","last_fullsized":"2023-10-21T17:19:27.899883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:38.345273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898682+0000","last_change":"2023-10-21T17:19:26.659469+0000","last_active":"2023-10-21T17:19:27.898682+0000","last_peered":"2023-10-21T17:19:27.898682+0000","last_clean":"2023-10-21T17:19:27.898682+0000","last_became_active":"2023-10-21T17:19:26.659128+0000","last_became_peered":"2023-10-21T17:19:26.659128+0000","last_unstale":"2023-10-21T17:19:27.898682+0000","last_undegraded":"2023-10-21T17:19:27.898682+0000","last_fullsized":"2023-10-21T17:19:27.898682+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:05:03.840579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624029+0000","last_change":"2023-10-21T17:19:26.661526+0000","last_active":"2023-10-21T17:19:27.624029+0000","last_peered":"2023-10-21T17:19:27.624029+0000","last_clean":"2023-10-21T17:19:27.624029+0000","last_became_active":"2023-10-21T17:19:26.661164+0000","last_became_peered":"2023-10-21T17:19:26.661164+0000","last_unstale":"2023-10-21T17:19:27.624029+0000","last_undegraded":"2023-10-21T17:19:27.624029+0000","last_fullsized":"2023-10-21T17:19:27.624029+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:55.160079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898780+0000","last_change":"2023-10-21T17:19:26.665823+0000","last_active":"2023-10-21T17:19:27.898780+0000","last_peered":"2023-10-21T17:19:27.898780+0000","last_clean":"2023-10-21T17:19:27.898780+0000","last_became_active":"2023-10-21T17:19:26.665511+0000","last_became_peered":"2023-10-21T17:19:26.665511+0000","last_unstale":"2023-10-21T17:19:27.898780+0000","last_undegraded":"2023-10-21T17:19:27.898780+0000","last_fullsized":"2023-10-21T17:19:27.898780+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:45:35.821926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901318+0000","last_change":"2023-10-21T17:19:26.658932+0000","last_active":"2023-10-21T17:19:27.901318+0000","last_peered":"2023-10-21T17:19:27.901318+0000","last_clean":"2023-10-21T17:19:27.901318+0000","last_became_active":"2023-10-21T17:19:26.653673+0000","last_became_peered":"2023-10-21T17:19:26.653673+0000","last_unstale":"2023-10-21T17:19:27.901318+0000","last_undegraded":"2023-10-21T17:19:27.901318+0000","last_fullsized":"2023-10-21T17:19:27.901318+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:50:20.910807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898946+0000","last_change":"2023-10-21T17:19:26.662010+0000","last_active":"2023-10-21T17:19:27.898946+0000","last_peered":"2023-10-21T17:19:27.898946+0000","last_clean":"2023-10-21T17:19:27.898946+0000","last_became_active":"2023-10-21T17:19:26.661721+0000","last_became_peered":"2023-10-21T17:19:26.661721+0000","last_unstale":"2023-10-21T17:19:27.898946+0000","last_undegraded":"2023-10-21T17:19:27.898946+0000","last_fullsized":"2023-10-21T17:19:27.898946+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:46:57.203785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.625171+0000","last_change":"2023-10-21T17:19:26.655662+0000","last_active":"2023-10-21T17:19:27.625171+0000","last_peered":"2023-10-21T17:19:27.625171+0000","last_clean":"2023-10-21T17:19:27.625171+0000","last_became_active":"2023-10-21T17:19:26.653571+0000","last_became_peered":"2023-10-21T17:19:26.653571+0000","last_unstale":"2023-10-21T17:19:27.625171+0000","last_undegraded":"2023-10-21T17:19:27.625171+0000","last_fullsized":"2023-10-21T17:19:27.625171+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:30:07.818930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623786+0000","last_change":"2023-10-21T17:19:26.660538+0000","last_active":"2023-10-21T17:19:27.623786+0000","last_peered":"2023-10-21T17:19:27.623786+0000","last_clean":"2023-10-21T17:19:27.623786+0000","last_became_active":"2023-10-21T17:19:26.660154+0000","last_became_peered":"2023-10-21T17:19:26.660154+0000","last_unstale":"2023-10-21T17:19:27.623786+0000","last_undegraded":"2023-10-21T17:19:27.623786+0000","last_fullsized":"2023-10-21T17:19:27.623786+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:02.934088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900286+0000","last_change":"2023-10-21T17:19:26.646645+0000","last_active":"2023-10-21T17:19:27.900286+0000","last_peered":"2023-10-21T17:19:27.900286+0000","last_clean":"2023-10-21T17:19:27.900286+0000","last_became_active":"2023-10-21T17:19:26.646383+0000","last_became_peered":"2023-10-21T17:19:26.646383+0000","last_unstale":"2023-10-21T17:19:27.900286+0000","last_undegraded":"2023-10-21T17:19:27.900286+0000","last_fullsized":"2023-10-21T17:19:27.900286+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:30:30.718010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626843+0000","last_change":"2023-10-21T17:19:26.660178+0000","last_active":"2023-10-21T17:19:27.626843+0000","last_peered":"2023-10-21T17:19:27.626843+0000","last_clean":"2023-10-21T17:19:27.626843+0000","last_became_active":"2023-10-21T17:19:26.659891+0000","last_became_peered":"2023-10-21T17:19:26.659891+0000","last_unstale":"2023-10-21T17:19:27.626843+0000","last_undegraded":"2023-10-21T17:19:27.626843+0000","last_fullsized":"2023-10-21T17:19:27.626843+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:06:53.807618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897384+0000","last_change":"2023-10-21T17:19:26.662366+0000","last_active":"2023-10-21T17:19:27.897384+0000","last_peered":"2023-10-21T17:19:27.897384+0000","last_clean":"2023-10-21T17:19:27.897384+0000","last_became_active":"2023-10-21T17:19:26.662114+0000","last_became_peered":"2023-10-21T17:19:26.662114+0000","last_unstale":"2023-10-21T17:19:27.897384+0000","last_undegraded":"2023-10-21T17:19:27.897384+0000","last_fullsized":"2023-10-21T17:19:27.897384+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:45.392042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898764+0000","last_change":"2023-10-21T17:19:26.648214+0000","last_active":"2023-10-21T17:19:27.898764+0000","last_peered":"2023-10-21T17:19:27.898764+0000","last_clean":"2023-10-21T17:19:27.898764+0000","last_became_active":"2023-10-21T17:19:26.647825+0000","last_became_peered":"2023-10-21T17:19:26.647825+0000","last_unstale":"2023-10-21T17:19:27.898764+0000","last_undegraded":"2023-10-21T17:19:27.898764+0000","last_fullsized":"2023-10-21T17:19:27.898764+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:23.435867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900929+0000","last_change":"2023-10-21T17:19:26.651987+0000","last_active":"2023-10-21T17:19:27.900929+0000","last_peered":"2023-10-21T17:19:27.900929+0000","last_clean":"2023-10-21T17:19:27.900929+0000","last_became_active":"2023-10-21T17:19:26.651673+0000","last_became_peered":"2023-10-21T17:19:26.651673+0000","last_unstale":"2023-10-21T17:19:27.900929+0000","last_undegraded":"2023-10-21T17:19:27.900929+0000","last_fullsized":"2023-10-21T17:19:27.900929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:57:57.688867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623963+0000","last_change":"2023-10-21T17:19:26.639823+0000","last_active":"2023-10-21T17:19:27.623963+0000","last_peered":"2023-10-21T17:19:27.623963+0000","last_clean":"2023-10-21T17:19:27.623963+0000","last_became_active":"2023-10-21T17:19:26.639414+0000","last_became_peered":"2023-10-21T17:19:26.639414+0000","last_unstale":"2023-10-21T17:19:27.623963+0000","last_undegraded":"2023-10-21T17:19:27.623963+0000","last_fullsized":"2023-10-21T17:19:27.623963+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:46:07.889981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627784+0000","last_change":"2023-10-21T17:19:26.660271+0000","last_active":"2023-10-21T17:19:27.627784+0000","last_peered":"2023-10-21T17:19:27.627784+0000","last_clean":"2023-10-21T17:19:27.627784+0000","last_became_active":"2023-10-21T17:19:26.659818+0000","last_became_peered":"2023-10-21T17:19:26.659818+0000","last_unstale":"2023-10-21T17:19:27.627784+0000","last_undegraded":"2023-10-21T17:19:27.627784+0000","last_fullsized":"2023-10-21T17:19:27.627784+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:04.562111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897775+0000","last_change":"2023-10-21T17:19:26.648562+0000","last_active":"2023-10-21T17:19:27.897775+0000","last_peered":"2023-10-21T17:19:27.897775+0000","last_clean":"2023-10-21T17:19:27.897775+0000","last_became_active":"2023-10-21T17:19:26.648077+0000","last_became_peered":"2023-10-21T17:19:26.648077+0000","last_unstale":"2023-10-21T17:19:27.897775+0000","last_undegraded":"2023-10-21T17:19:27.897775+0000","last_fullsized":"2023-10-21T17:19:27.897775+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:07:02.155620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627026+0000","last_change":"2023-10-21T17:19:26.652544+0000","last_active":"2023-10-21T17:19:27.627026+0000","last_peered":"2023-10-21T17:19:27.627026+0000","last_clean":"2023-10-21T17:19:27.627026+0000","last_became_active":"2023-10-21T17:19:26.649430+0000","last_became_peered":"2023-10-21T17:19:26.649430+0000","last_unstale":"2023-10-21T17:19:27.627026+0000","last_undegraded":"2023-10-21T17:19:27.627026+0000","last_fullsized":"2023-10-21T17:19:27.627026+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:24:23.732089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898050+0000","last_change":"2023-10-21T17:19:26.657540+0000","last_active":"2023-10-21T17:19:27.898050+0000","last_peered":"2023-10-21T17:19:27.898050+0000","last_clean":"2023-10-21T17:19:27.898050+0000","last_became_active":"2023-10-21T17:19:26.657240+0000","last_became_peered":"2023-10-21T17:19:26.657240+0000","last_unstale":"2023-10-21T17:19:27.898050+0000","last_undegraded":"2023-10-21T17:19:27.898050+0000","last_fullsized":"2023-10-21T17:19:27.898050+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:51.179302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624335+0000","last_change":"2023-10-21T17:19:26.657388+0000","last_active":"2023-10-21T17:19:27.624335+0000","last_peered":"2023-10-21T17:19:27.624335+0000","last_clean":"2023-10-21T17:19:27.624335+0000","last_became_active":"2023-10-21T17:19:26.657041+0000","last_became_peered":"2023-10-21T17:19:26.657041+0000","last_unstale":"2023-10-21T17:19:27.624335+0000","last_undegraded":"2023-10-21T17:19:27.624335+0000","last_fullsized":"2023-10-21T17:19:27.624335+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:56.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897855+0000","last_change":"2023-10-21T17:19:26.660563+0000","last_active":"2023-10-21T17:19:27.897855+0000","last_peered":"2023-10-21T17:19:27.897855+0000","last_clean":"2023-10-21T17:19:27.897855+0000","last_became_active":"2023-10-21T17:19:26.660258+0000","last_became_peered":"2023-10-21T17:19:26.660258+0000","last_unstale":"2023-10-21T17:19:27.897855+0000","last_undegraded":"2023-10-21T17:19:27.897855+0000","last_fullsized":"2023-10-21T17:19:27.897855+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:37:05.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626174+0000","last_change":"2023-10-21T17:19:26.657006+0000","last_active":"2023-10-21T17:19:27.626174+0000","last_peered":"2023-10-21T17:19:27.626174+0000","last_clean":"2023-10-21T17:19:27.626174+0000","last_became_active":"2023-10-21T17:19:26.653146+0000","last_became_peered":"2023-10-21T17:19:26.653146+0000","last_unstale":"2023-10-21T17:19:27.626174+0000","last_undegraded":"2023-10-21T17:19:27.626174+0000","last_fullsized":"2023-10-21T17:19:27.626174+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:08:23.545216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"196'4","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900303+0000","last_change":"2023-10-21T17:19:25.619174+0000","last_active":"2023-10-21T17:19:27.900303+0000","last_peered":"2023-10-21T17:19:27.900303+0000","last_clean":"2023-10-21T17:19:27.900303+0000","last_became_active":"2023-10-21T17:19:25.618607+0000","last_became_peered":"2023-10-21T17:19:25.618607+0000","last_unstale":"2023-10-21T17:19:27.900303+0000","last_undegraded":"2023-10-21T17:19:27.900303+0000","last_fullsized":"2023-10-21T17:19:27.900303+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:53:24.466647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897560+0000","last_change":"2023-10-21T17:19:26.666188+0000","last_active":"2023-10-21T17:19:27.897560+0000","last_peered":"2023-10-21T17:19:27.897560+0000","last_clean":"2023-10-21T17:19:27.897560+0000","last_became_active":"2023-10-21T17:19:26.664453+0000","last_became_peered":"2023-10-21T17:19:26.664453+0000","last_unstale":"2023-10-21T17:19:27.897560+0000","last_undegraded":"2023-10-21T17:19:27.897560+0000","last_fullsized":"2023-10-21T17:19:27.897560+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:40.114373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899017+0000","last_change":"2023-10-21T17:19:25.635787+0000","last_active":"2023-10-21T17:19:27.899017+0000","last_peered":"2023-10-21T17:19:27.899017+0000","last_clean":"2023-10-21T17:19:27.899017+0000","last_became_active":"2023-10-21T17:19:25.635322+0000","last_became_peered":"2023-10-21T17:19:25.635322+0000","last_unstale":"2023-10-21T17:19:27.899017+0000","last_undegraded":"2023-10-21T17:19:27.899017+0000","last_fullsized":"2023-10-21T17:19:27.899017+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:07.724078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899090+0000","last_change":"2023-10-21T17:19:26.657176+0000","last_active":"2023-10-21T17:19:27.899090+0000","last_peered":"2023-10-21T17:19:27.899090+0000","last_clean":"2023-10-21T17:19:27.899090+0000","last_became_active":"2023-10-21T17:19:26.656875+0000","last_became_peered":"2023-10-21T17:19:26.656875+0000","last_unstale":"2023-10-21T17:19:27.899090+0000","last_undegraded":"2023-10-21T17:19:27.899090+0000","last_fullsized":"2023-10-21T17:19:27.899090+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:16.038189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624776+0000","last_change":"2023-10-21T17:19:25.612917+0000","last_active":"2023-10-21T17:19:27.624776+0000","last_peered":"2023-10-21T17:19:27.624776+0000","last_clean":"2023-10-21T17:19:27.624776+0000","last_became_active":"2023-10-21T17:19:25.606671+0000","last_became_peered":"2023-10-21T17:19:25.606671+0000","last_unstale":"2023-10-21T17:19:27.624776+0000","last_undegraded":"2023-10-21T17:19:27.624776+0000","last_fullsized":"2023-10-21T17:19:27.624776+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:25.904957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627306+0000","last_change":"2023-10-21T17:19:26.653519+0000","last_active":"2023-10-21T17:19:27.627306+0000","last_peered":"2023-10-21T17:19:27.627306+0000","last_clean":"2023-10-21T17:19:27.627306+0000","last_became_active":"2023-10-21T17:19:26.653189+0000","last_became_peered":"2023-10-21T17:19:26.653189+0000","last_unstale":"2023-10-21T17:19:27.627306+0000","last_undegraded":"2023-10-21T17:19:27.627306+0000","last_fullsized":"2023-10-21T17:19:27.627306+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:26.918326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623959+0000","last_change":"2023-10-21T17:19:25.629774+0000","last_active":"2023-10-21T17:19:27.623959+0000","last_peered":"2023-10-21T17:19:27.623959+0000","last_clean":"2023-10-21T17:19:27.623959+0000","last_became_active":"2023-10-21T17:19:25.629378+0000","last_became_peered":"2023-10-21T17:19:25.629378+0000","last_unstale":"2023-10-21T17:19:27.623959+0000","last_undegraded":"2023-10-21T17:19:27.623959+0000","last_fullsized":"2023-10-21T17:19:27.623959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:06:40.263107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900044+0000","last_change":"2023-10-21T17:19:26.659624+0000","last_active":"2023-10-21T17:19:27.900044+0000","last_peered":"2023-10-21T17:19:27.900044+0000","last_clean":"2023-10-21T17:19:27.900044+0000","last_became_active":"2023-10-21T17:19:26.659365+0000","last_became_peered":"2023-10-21T17:19:26.659365+0000","last_unstale":"2023-10-21T17:19:27.900044+0000","last_undegraded":"2023-10-21T17:19:27.900044+0000","last_fullsized":"2023-10-21T17:19:27.900044+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:29.679124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899814+0000","last_change":"2023-10-21T17:19:25.629460+0000","last_active":"2023-10-21T17:19:27.899814+0000","last_peered":"2023-10-21T17:19:27.899814+0000","last_clean":"2023-10-21T17:19:27.899814+0000","last_became_active":"2023-10-21T17:19:25.629152+0000","last_became_peered":"2023-10-21T17:19:25.629152+0000","last_unstale":"2023-10-21T17:19:27.899814+0000","last_undegraded":"2023-10-21T17:19:27.899814+0000","last_fullsized":"2023-10-21T17:19:27.899814+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:54.610923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623907+0000","last_change":"2023-10-21T17:19:26.648808+0000","last_active":"2023-10-21T17:19:27.623907+0000","last_peered":"2023-10-21T17:19:27.623907+0000","last_clean":"2023-10-21T17:19:27.623907+0000","last_became_active":"2023-10-21T17:19:26.648448+0000","last_became_peered":"2023-10-21T17:19:26.648448+0000","last_unstale":"2023-10-21T17:19:27.623907+0000","last_undegraded":"2023-10-21T17:19:27.623907+0000","last_fullsized":"2023-10-21T17:19:27.623907+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:12:43.958557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899356+0000","last_change":"2023-10-21T17:19:25.626670+0000","last_active":"2023-10-21T17:19:27.899356+0000","last_peered":"2023-10-21T17:19:27.899356+0000","last_clean":"2023-10-21T17:19:27.899356+0000","last_became_active":"2023-10-21T17:19:25.626375+0000","last_became_peered":"2023-10-21T17:19:25.626375+0000","last_unstale":"2023-10-21T17:19:27.899356+0000","last_undegraded":"2023-10-21T17:19:27.899356+0000","last_fullsized":"2023-10-21T17:19:27.899356+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:52.212859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624254+0000","last_change":"2023-10-21T17:19:26.654396+0000","last_active":"2023-10-21T17:19:27.624254+0000","last_peered":"2023-10-21T17:19:27.624254+0000","last_clean":"2023-10-21T17:19:27.624254+0000","last_became_active":"2023-10-21T17:19:26.654119+0000","last_became_peered":"2023-10-21T17:19:26.654119+0000","last_unstale":"2023-10-21T17:19:27.624254+0000","last_undegraded":"2023-10-21T17:19:27.624254+0000","last_fullsized":"2023-10-21T17:19:27.624254+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:53:48.811705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623471+0000","last_change":"2023-10-21T17:19:25.624312+0000","last_active":"2023-10-21T17:19:27.623471+0000","last_peered":"2023-10-21T17:19:27.623471+0000","last_clean":"2023-10-21T17:19:27.623471+0000","last_became_active":"2023-10-21T17:19:25.623553+0000","last_became_peered":"2023-10-21T17:19:25.623553+0000","last_unstale":"2023-10-21T17:19:27.623471+0000","last_undegraded":"2023-10-21T17:19:27.623471+0000","last_fullsized":"2023-10-21T17:19:27.623471+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:42.268063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900105+0000","last_change":"2023-10-21T17:19:26.646397+0000","last_active":"2023-10-21T17:19:27.900105+0000","last_peered":"2023-10-21T17:19:27.900105+0000","last_clean":"2023-10-21T17:19:27.900105+0000","last_became_active":"2023-10-21T17:19:26.643812+0000","last_became_peered":"2023-10-21T17:19:26.643812+0000","last_unstale":"2023-10-21T17:19:27.900105+0000","last_undegraded":"2023-10-21T17:19:27.900105+0000","last_fullsized":"2023-10-21T17:19:27.900105+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:56:39.556251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896490+0000","last_change":"2023-10-21T17:19:25.630004+0000","last_active":"2023-10-21T17:19:27.896490+0000","last_peered":"2023-10-21T17:19:27.896490+0000","last_clean":"2023-10-21T17:19:27.896490+0000","last_became_active":"2023-10-21T17:19:25.629660+0000","last_became_peered":"2023-10-21T17:19:25.629660+0000","last_unstale":"2023-10-21T17:19:27.896490+0000","last_undegraded":"2023-10-21T17:19:27.896490+0000","last_fullsized":"2023-10-21T17:19:27.896490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:49.391349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626165+0000","last_change":"2023-10-21T17:19:26.656542+0000","last_active":"2023-10-21T17:19:27.626165+0000","last_peered":"2023-10-21T17:19:27.626165+0000","last_clean":"2023-10-21T17:19:27.626165+0000","last_became_active":"2023-10-21T17:19:26.653530+0000","last_became_peered":"2023-10-21T17:19:26.653530+0000","last_unstale":"2023-10-21T17:19:27.626165+0000","last_undegraded":"2023-10-21T17:19:27.626165+0000","last_fullsized":"2023-10-21T17:19:27.626165+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:07.917409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900941+0000","last_change":"2023-10-21T17:19:25.634629+0000","last_active":"2023-10-21T17:19:27.900941+0000","last_peered":"2023-10-21T17:19:27.900941+0000","last_clean":"2023-10-21T17:19:27.900941+0000","last_became_active":"2023-10-21T17:19:25.634306+0000","last_became_peered":"2023-10-21T17:19:25.634306+0000","last_unstale":"2023-10-21T17:19:27.900941+0000","last_undegraded":"2023-10-21T17:19:27.900941+0000","last_fullsized":"2023-10-21T17:19:27.900941+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:32:39.760979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897879+0000","last_change":"2023-10-21T17:19:26.648113+0000","last_active":"2023-10-21T17:19:27.897879+0000","last_peered":"2023-10-21T17:19:27.897879+0000","last_clean":"2023-10-21T17:19:27.897879+0000","last_became_active":"2023-10-21T17:19:26.647815+0000","last_became_peered":"2023-10-21T17:19:26.647815+0000","last_unstale":"2023-10-21T17:19:27.897879+0000","last_undegraded":"2023-10-21T17:19:27.897879+0000","last_fullsized":"2023-10-21T17:19:27.897879+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:20.839847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"196'3","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899667+0000","last_change":"2023-10-21T17:19:25.622210+0000","last_active":"2023-10-21T17:19:27.899667+0000","last_peered":"2023-10-21T17:19:27.899667+0000","last_clean":"2023-10-21T17:19:27.899667+0000","last_became_active":"2023-10-21T17:19:25.621949+0000","last_became_peered":"2023-10-21T17:19:25.621949+0000","last_unstale":"2023-10-21T17:19:27.899667+0000","last_undegraded":"2023-10-21T17:19:27.899667+0000","last_fullsized":"2023-10-21T17:19:27.899667+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:58:19.115567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623801+0000","last_change":"2023-10-21T17:19:26.660957+0000","last_active":"2023-10-21T17:19:27.623801+0000","last_peered":"2023-10-21T17:19:27.623801+0000","last_clean":"2023-10-21T17:19:27.623801+0000","last_became_active":"2023-10-21T17:19:26.659366+0000","last_became_peered":"2023-10-21T17:19:26.659366+0000","last_unstale":"2023-10-21T17:19:27.623801+0000","last_undegraded":"2023-10-21T17:19:27.623801+0000","last_fullsized":"2023-10-21T17:19:27.623801+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:29:26.571178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"196'6","reported_seq":18,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898027+0000","last_change":"2023-10-21T17:19:25.633039+0000","last_active":"2023-10-21T17:19:27.898027+0000","last_peered":"2023-10-21T17:19:27.898027+0000","last_clean":"2023-10-21T17:19:27.898027+0000","last_became_active":"2023-10-21T17:19:25.632746+0000","last_became_peered":"2023-10-21T17:19:25.632746+0000","last_unstale":"2023-10-21T17:19:27.898027+0000","last_undegraded":"2023-10-21T17:19:27.898027+0000","last_fullsized":"2023-10-21T17:19:27.898027+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:11:51.090011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898104+0000","last_change":"2023-10-21T17:19:26.662996+0000","last_active":"2023-10-21T17:19:27.898104+0000","last_peered":"2023-10-21T17:19:27.898104+0000","last_clean":"2023-10-21T17:19:27.898104+0000","last_became_active":"2023-10-21T17:19:26.662423+0000","last_became_peered":"2023-10-21T17:19:26.662423+0000","last_unstale":"2023-10-21T17:19:27.898104+0000","last_undegraded":"2023-10-21T17:19:27.898104+0000","last_fullsized":"2023-10-21T17:19:27.898104+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:42.401111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900741+0000","last_change":"2023-10-21T17:19:25.626771+0000","last_active":"2023-10-21T17:19:27.900741+0000","last_peered":"2023-10-21T17:19:27.900741+0000","last_clean":"2023-10-21T17:19:27.900741+0000","last_became_active":"2023-10-21T17:19:25.626501+0000","last_became_peered":"2023-10-21T17:19:25.626501+0000","last_unstale":"2023-10-21T17:19:27.900741+0000","last_undegraded":"2023-10-21T17:19:27.900741+0000","last_fullsized":"2023-10-21T17:19:27.900741+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:53.535684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900588+0000","last_change":"2023-10-21T17:19:26.663734+0000","last_active":"2023-10-21T17:19:27.900588+0000","last_peered":"2023-10-21T17:19:27.900588+0000","last_clean":"2023-10-21T17:19:27.900588+0000","last_became_active":"2023-10-21T17:19:26.663145+0000","last_became_peered":"2023-10-21T17:19:26.663145+0000","last_unstale":"2023-10-21T17:19:27.900588+0000","last_undegraded":"2023-10-21T17:19:27.900588+0000","last_fullsized":"2023-10-21T17:19:27.900588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:32.867580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898300+0000","last_change":"2023-10-21T17:19:25.628686+0000","last_active":"2023-10-21T17:19:27.898300+0000","last_peered":"2023-10-21T17:19:27.898300+0000","last_clean":"2023-10-21T17:19:27.898300+0000","last_became_active":"2023-10-21T17:19:25.628369+0000","last_became_peered":"2023-10-21T17:19:25.628369+0000","last_unstale":"2023-10-21T17:19:27.898300+0000","last_undegraded":"2023-10-21T17:19:27.898300+0000","last_fullsized":"2023-10-21T17:19:27.898300+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:08.076214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.625106+0000","last_change":"2023-10-21T17:19:26.653646+0000","last_active":"2023-10-21T17:19:27.625106+0000","last_peered":"2023-10-21T17:19:27.625106+0000","last_clean":"2023-10-21T17:19:27.625106+0000","last_became_active":"2023-10-21T17:19:26.653338+0000","last_became_peered":"2023-10-21T17:19:26.653338+0000","last_unstale":"2023-10-21T17:19:27.625106+0000","last_undegraded":"2023-10-21T17:19:27.625106+0000","last_fullsized":"2023-10-21T17:19:27.625106+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:56.693712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624586+0000","last_change":"2023-10-21T17:19:25.624949+0000","last_active":"2023-10-21T17:19:27.624586+0000","last_peered":"2023-10-21T17:19:27.624586+0000","last_clean":"2023-10-21T17:19:27.624586+0000","last_became_active":"2023-10-21T17:19:25.624679+0000","last_became_peered":"2023-10-21T17:19:25.624679+0000","last_unstale":"2023-10-21T17:19:27.624586+0000","last_undegraded":"2023-10-21T17:19:27.624586+0000","last_fullsized":"2023-10-21T17:19:27.624586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:58:10.991179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624165+0000","last_change":"2023-10-21T17:19:26.646318+0000","last_active":"2023-10-21T17:19:27.624165+0000","last_peered":"2023-10-21T17:19:27.624165+0000","last_clean":"2023-10-21T17:19:27.624165+0000","last_became_active":"2023-10-21T17:19:26.645811+0000","last_became_peered":"2023-10-21T17:19:26.645811+0000","last_unstale":"2023-10-21T17:19:27.624165+0000","last_undegraded":"2023-10-21T17:19:27.624165+0000","last_fullsized":"2023-10-21T17:19:27.624165+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:25:12.007712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"196'8","reported_seq":20,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896592+0000","last_change":"2023-10-21T17:19:25.626771+0000","last_active":"2023-10-21T17:19:27.896592+0000","last_peered":"2023-10-21T17:19:27.896592+0000","last_clean":"2023-10-21T17:19:27.896592+0000","last_became_active":"2023-10-21T17:19:25.626434+0000","last_became_peered":"2023-10-21T17:19:25.626434+0000","last_unstale":"2023-10-21T17:19:27.896592+0000","last_undegraded":"2023-10-21T17:19:27.896592+0000","last_fullsized":"2023-10-21T17:19:27.896592+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:48.474019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.896572+0000","last_change":"2023-10-21T17:19:26.652979+0000","last_active":"2023-10-21T17:19:27.896572+0000","last_peered":"2023-10-21T17:19:27.896572+0000","last_clean":"2023-10-21T17:19:27.896572+0000","last_became_active":"2023-10-21T17:19:26.650878+0000","last_became_peered":"2023-10-21T17:19:26.650878+0000","last_unstale":"2023-10-21T17:19:27.896572+0000","last_undegraded":"2023-10-21T17:19:27.896572+0000","last_fullsized":"2023-10-21T17:19:27.896572+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:25:35.529983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901029+0000","last_change":"2023-10-21T17:19:25.618828+0000","last_active":"2023-10-21T17:19:27.901029+0000","last_peered":"2023-10-21T17:19:27.901029+0000","last_clean":"2023-10-21T17:19:27.901029+0000","last_became_active":"2023-10-21T17:19:25.618514+0000","last_became_peered":"2023-10-21T17:19:25.618514+0000","last_unstale":"2023-10-21T17:19:27.901029+0000","last_undegraded":"2023-10-21T17:19:27.901029+0000","last_fullsized":"2023-10-21T17:19:27.901029+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:37:06.545630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898258+0000","last_change":"2023-10-21T17:19:26.664664+0000","last_active":"2023-10-21T17:19:27.898258+0000","last_peered":"2023-10-21T17:19:27.898258+0000","last_clean":"2023-10-21T17:19:27.898258+0000","last_became_active":"2023-10-21T17:19:26.664299+0000","last_became_peered":"2023-10-21T17:19:26.664299+0000","last_unstale":"2023-10-21T17:19:27.898258+0000","last_undegraded":"2023-10-21T17:19:27.898258+0000","last_fullsized":"2023-10-21T17:19:27.898258+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:03:32.314927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.901623+0000","last_change":"2023-10-21T17:19:25.634016+0000","last_active":"2023-10-21T17:19:27.901623+0000","last_peered":"2023-10-21T17:19:27.901623+0000","last_clean":"2023-10-21T17:19:27.901623+0000","last_became_active":"2023-10-21T17:19:25.633299+0000","last_became_peered":"2023-10-21T17:19:25.633299+0000","last_unstale":"2023-10-21T17:19:27.901623+0000","last_undegraded":"2023-10-21T17:19:27.901623+0000","last_fullsized":"2023-10-21T17:19:27.901623+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:26:37.918906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624669+0000","last_change":"2023-10-21T17:19:26.639334+0000","last_active":"2023-10-21T17:19:27.624669+0000","last_peered":"2023-10-21T17:19:27.624669+0000","last_clean":"2023-10-21T17:19:27.624669+0000","last_became_active":"2023-10-21T17:19:26.638702+0000","last_became_peered":"2023-10-21T17:19:26.638702+0000","last_unstale":"2023-10-21T17:19:27.624669+0000","last_undegraded":"2023-10-21T17:19:27.624669+0000","last_fullsized":"2023-10-21T17:19:27.624669+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:54:13.594481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"196'1","reported_seq":13,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899380+0000","last_change":"2023-10-21T17:19:25.621517+0000","last_active":"2023-10-21T17:19:27.899380+0000","last_peered":"2023-10-21T17:19:27.899380+0000","last_clean":"2023-10-21T17:19:27.899380+0000","last_became_active":"2023-10-21T17:19:25.620567+0000","last_became_peered":"2023-10-21T17:19:25.620567+0000","last_unstale":"2023-10-21T17:19:27.899380+0000","last_undegraded":"2023-10-21T17:19:27.899380+0000","last_fullsized":"2023-10-21T17:19:27.899380+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:01:05.561647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624233+0000","last_change":"2023-10-21T17:19:26.654232+0000","last_active":"2023-10-21T17:19:27.624233+0000","last_peered":"2023-10-21T17:19:27.624233+0000","last_clean":"2023-10-21T17:19:27.624233+0000","last_became_active":"2023-10-21T17:19:26.653935+0000","last_became_peered":"2023-10-21T17:19:26.653935+0000","last_unstale":"2023-10-21T17:19:27.624233+0000","last_undegraded":"2023-10-21T17:19:27.624233+0000","last_fullsized":"2023-10-21T17:19:27.624233+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:58:36.912519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900325+0000","last_change":"2023-10-21T17:19:25.629607+0000","last_active":"2023-10-21T17:19:27.900325+0000","last_peered":"2023-10-21T17:19:27.900325+0000","last_clean":"2023-10-21T17:19:27.900325+0000","last_became_active":"2023-10-21T17:19:25.629301+0000","last_became_peered":"2023-10-21T17:19:25.629301+0000","last_unstale":"2023-10-21T17:19:27.900325+0000","last_undegraded":"2023-10-21T17:19:27.900325+0000","last_fullsized":"2023-10-21T17:19:27.900325+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:56:34.291382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897929+0000","last_change":"2023-10-21T17:19:26.648390+0000","last_active":"2023-10-21T17:19:27.897929+0000","last_peered":"2023-10-21T17:19:27.897929+0000","last_clean":"2023-10-21T17:19:27.897929+0000","last_became_active":"2023-10-21T17:19:26.647681+0000","last_became_peered":"2023-10-21T17:19:26.647681+0000","last_unstale":"2023-10-21T17:19:27.897929+0000","last_undegraded":"2023-10-21T17:19:27.897929+0000","last_fullsized":"2023-10-21T17:19:27.897929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:57.090952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899827+0000","last_change":"2023-10-21T17:19:25.629030+0000","last_active":"2023-10-21T17:19:27.899827+0000","last_peered":"2023-10-21T17:19:27.899827+0000","last_clean":"2023-10-21T17:19:27.899827+0000","last_became_active":"2023-10-21T17:19:25.627969+0000","last_became_peered":"2023-10-21T17:19:25.627969+0000","last_unstale":"2023-10-21T17:19:27.899827+0000","last_undegraded":"2023-10-21T17:19:27.899827+0000","last_fullsized":"2023-10-21T17:19:27.899827+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:38:29.649568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898705+0000","last_change":"2023-10-21T17:19:26.655821+0000","last_active":"2023-10-21T17:19:27.898705+0000","last_peered":"2023-10-21T17:19:27.898705+0000","last_clean":"2023-10-21T17:19:27.898705+0000","last_became_active":"2023-10-21T17:19:26.655472+0000","last_became_peered":"2023-10-21T17:19:26.655472+0000","last_unstale":"2023-10-21T17:19:27.898705+0000","last_undegraded":"2023-10-21T17:19:27.898705+0000","last_fullsized":"2023-10-21T17:19:27.898705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:53.220679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624847+0000","last_change":"2023-10-21T17:19:25.623410+0000","last_active":"2023-10-21T17:19:27.624847+0000","last_peered":"2023-10-21T17:19:27.624847+0000","last_clean":"2023-10-21T17:19:27.624847+0000","last_became_active":"2023-10-21T17:19:25.623107+0000","last_became_peered":"2023-10-21T17:19:25.623107+0000","last_unstale":"2023-10-21T17:19:27.624847+0000","last_undegraded":"2023-10-21T17:19:27.624847+0000","last_fullsized":"2023-10-21T17:19:27.624847+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:47.022544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898723+0000","last_change":"2023-10-21T17:19:26.667084+0000","last_active":"2023-10-21T17:19:27.898723+0000","last_peered":"2023-10-21T17:19:27.898723+0000","last_clean":"2023-10-21T17:19:27.898723+0000","last_became_active":"2023-10-21T17:19:26.666671+0000","last_became_peered":"2023-10-21T17:19:26.666671+0000","last_unstale":"2023-10-21T17:19:27.898723+0000","last_undegraded":"2023-10-21T17:19:27.898723+0000","last_fullsized":"2023-10-21T17:19:27.898723+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:11:08.494433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623597+0000","last_change":"2023-10-21T17:19:25.625391+0000","last_active":"2023-10-21T17:19:27.623597+0000","last_peered":"2023-10-21T17:19:27.623597+0000","last_clean":"2023-10-21T17:19:27.623597+0000","last_became_active":"2023-10-21T17:19:25.625121+0000","last_became_peered":"2023-10-21T17:19:25.625121+0000","last_unstale":"2023-10-21T17:19:27.623597+0000","last_undegraded":"2023-10-21T17:19:27.623597+0000","last_fullsized":"2023-10-21T17:19:27.623597+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:51:04.925541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627673+0000","last_change":"2023-10-21T17:19:26.655274+0000","last_active":"2023-10-21T17:19:27.627673+0000","last_peered":"2023-10-21T17:19:27.627673+0000","last_clean":"2023-10-21T17:19:27.627673+0000","last_became_active":"2023-10-21T17:19:26.654958+0000","last_became_peered":"2023-10-21T17:19:26.654958+0000","last_unstale":"2023-10-21T17:19:27.627673+0000","last_undegraded":"2023-10-21T17:19:27.627673+0000","last_fullsized":"2023-10-21T17:19:27.627673+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:55.160079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626606+0000","last_change":"2023-10-21T17:19:25.631332+0000","last_active":"2023-10-21T17:19:27.626606+0000","last_peered":"2023-10-21T17:19:27.626606+0000","last_clean":"2023-10-21T17:19:27.626606+0000","last_became_active":"2023-10-21T17:19:25.630926+0000","last_became_peered":"2023-10-21T17:19:25.630926+0000","last_unstale":"2023-10-21T17:19:27.626606+0000","last_undegraded":"2023-10-21T17:19:27.626606+0000","last_fullsized":"2023-10-21T17:19:27.626606+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:40:42.260815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623664+0000","last_change":"2023-10-21T17:19:26.663712+0000","last_active":"2023-10-21T17:19:27.623664+0000","last_peered":"2023-10-21T17:19:27.623664+0000","last_clean":"2023-10-21T17:19:27.623664+0000","last_became_active":"2023-10-21T17:19:26.661440+0000","last_became_peered":"2023-10-21T17:19:26.661440+0000","last_unstale":"2023-10-21T17:19:27.623664+0000","last_undegraded":"2023-10-21T17:19:27.623664+0000","last_fullsized":"2023-10-21T17:19:27.623664+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:08.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"196'2","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899221+0000","last_change":"2023-10-21T17:19:25.628326+0000","last_active":"2023-10-21T17:19:27.899221+0000","last_peered":"2023-10-21T17:19:27.899221+0000","last_clean":"2023-10-21T17:19:27.899221+0000","last_became_active":"2023-10-21T17:19:25.628062+0000","last_became_peered":"2023-10-21T17:19:25.628062+0000","last_unstale":"2023-10-21T17:19:27.899221+0000","last_undegraded":"2023-10-21T17:19:27.899221+0000","last_fullsized":"2023-10-21T17:19:27.899221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:17.439605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897288+0000","last_change":"2023-10-21T17:19:26.651773+0000","last_active":"2023-10-21T17:19:27.897288+0000","last_peered":"2023-10-21T17:19:27.897288+0000","last_clean":"2023-10-21T17:19:27.897288+0000","last_became_active":"2023-10-21T17:19:26.651479+0000","last_became_peered":"2023-10-21T17:19:26.651479+0000","last_unstale":"2023-10-21T17:19:27.897288+0000","last_undegraded":"2023-10-21T17:19:27.897288+0000","last_fullsized":"2023-10-21T17:19:27.897288+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:10:37.781098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900711+0000","last_change":"2023-10-21T17:19:25.636689+0000","last_active":"2023-10-21T17:19:27.900711+0000","last_peered":"2023-10-21T17:19:27.900711+0000","last_clean":"2023-10-21T17:19:27.900711+0000","last_became_active":"2023-10-21T17:19:25.636402+0000","last_became_peered":"2023-10-21T17:19:25.636402+0000","last_unstale":"2023-10-21T17:19:27.900711+0000","last_undegraded":"2023-10-21T17:19:27.900711+0000","last_fullsized":"2023-10-21T17:19:27.900711+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:37.591035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623654+0000","last_change":"2023-10-21T17:19:26.654287+0000","last_active":"2023-10-21T17:19:27.623654+0000","last_peered":"2023-10-21T17:19:27.623654+0000","last_clean":"2023-10-21T17:19:27.623654+0000","last_became_active":"2023-10-21T17:19:26.654004+0000","last_became_peered":"2023-10-21T17:19:26.654004+0000","last_unstale":"2023-10-21T17:19:27.623654+0000","last_undegraded":"2023-10-21T17:19:27.623654+0000","last_fullsized":"2023-10-21T17:19:27.623654+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:09:45.996748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900218+0000","last_change":"2023-10-21T17:19:25.628646+0000","last_active":"2023-10-21T17:19:27.900218+0000","last_peered":"2023-10-21T17:19:27.900218+0000","last_clean":"2023-10-21T17:19:27.900218+0000","last_became_active":"2023-10-21T17:19:25.628371+0000","last_became_peered":"2023-10-21T17:19:25.628371+0000","last_unstale":"2023-10-21T17:19:27.900218+0000","last_undegraded":"2023-10-21T17:19:27.900218+0000","last_fullsized":"2023-10-21T17:19:27.900218+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:04:51.053443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.897847+0000","last_change":"2023-10-21T17:19:26.657534+0000","last_active":"2023-10-21T17:19:27.897847+0000","last_peered":"2023-10-21T17:19:27.897847+0000","last_clean":"2023-10-21T17:19:27.897847+0000","last_became_active":"2023-10-21T17:19:26.655656+0000","last_became_peered":"2023-10-21T17:19:26.655656+0000","last_unstale":"2023-10-21T17:19:27.897847+0000","last_undegraded":"2023-10-21T17:19:27.897847+0000","last_fullsized":"2023-10-21T17:19:27.897847+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:13:55.194766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.623971+0000","last_change":"2023-10-21T17:19:25.630873+0000","last_active":"2023-10-21T17:19:27.623971+0000","last_peered":"2023-10-21T17:19:27.623971+0000","last_clean":"2023-10-21T17:19:27.623971+0000","last_became_active":"2023-10-21T17:19:25.630560+0000","last_became_peered":"2023-10-21T17:19:25.630560+0000","last_unstale":"2023-10-21T17:19:27.623971+0000","last_undegraded":"2023-10-21T17:19:27.623971+0000","last_fullsized":"2023-10-21T17:19:27.623971+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:45:03.818326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900067+0000","last_change":"2023-10-21T17:19:26.659594+0000","last_active":"2023-10-21T17:19:27.900067+0000","last_peered":"2023-10-21T17:19:27.900067+0000","last_clean":"2023-10-21T17:19:27.900067+0000","last_became_active":"2023-10-21T17:19:26.658634+0000","last_became_peered":"2023-10-21T17:19:26.658634+0000","last_unstale":"2023-10-21T17:19:27.900067+0000","last_undegraded":"2023-10-21T17:19:27.900067+0000","last_fullsized":"2023-10-21T17:19:27.900067+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:02.684095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899269+0000","last_change":"2023-10-21T17:19:25.618289+0000","last_active":"2023-10-21T17:19:27.899269+0000","last_peered":"2023-10-21T17:19:27.899269+0000","last_clean":"2023-10-21T17:19:27.899269+0000","last_became_active":"2023-10-21T17:19:25.617765+0000","last_became_peered":"2023-10-21T17:19:25.617765+0000","last_unstale":"2023-10-21T17:19:27.899269+0000","last_undegraded":"2023-10-21T17:19:27.899269+0000","last_fullsized":"2023-10-21T17:19:27.899269+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:11.590828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.627873+0000","last_change":"2023-10-21T17:19:26.655938+0000","last_active":"2023-10-21T17:19:27.627873+0000","last_peered":"2023-10-21T17:19:27.627873+0000","last_clean":"2023-10-21T17:19:27.627873+0000","last_became_active":"2023-10-21T17:19:26.655227+0000","last_became_peered":"2023-10-21T17:19:26.655227+0000","last_unstale":"2023-10-21T17:19:27.627873+0000","last_undegraded":"2023-10-21T17:19:27.627873+0000","last_fullsized":"2023-10-21T17:19:27.627873+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:56.628893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.898518+0000","last_change":"2023-10-21T17:19:25.622295+0000","last_active":"2023-10-21T17:19:27.898518+0000","last_peered":"2023-10-21T17:19:27.898518+0000","last_clean":"2023-10-21T17:19:27.898518+0000","last_became_active":"2023-10-21T17:19:25.622016+0000","last_became_peered":"2023-10-21T17:19:25.622016+0000","last_unstale":"2023-10-21T17:19:27.898518+0000","last_undegraded":"2023-10-21T17:19:27.898518+0000","last_fullsized":"2023-10-21T17:19:27.898518+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:36.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900467+0000","last_change":"2023-10-21T17:19:26.642886+0000","last_active":"2023-10-21T17:19:27.900467+0000","last_peered":"2023-10-21T17:19:27.900467+0000","last_clean":"2023-10-21T17:19:27.900467+0000","last_became_active":"2023-10-21T17:19:26.642510+0000","last_became_peered":"2023-10-21T17:19:26.642510+0000","last_unstale":"2023-10-21T17:19:27.900467+0000","last_undegraded":"2023-10-21T17:19:27.900467+0000","last_fullsized":"2023-10-21T17:19:27.900467+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:10:06.936997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624783+0000","last_change":"2023-10-21T17:19:25.608862+0000","last_active":"2023-10-21T17:19:27.624783+0000","last_peered":"2023-10-21T17:19:27.624783+0000","last_clean":"2023-10-21T17:19:27.624783+0000","last_became_active":"2023-10-21T17:19:25.608001+0000","last_became_peered":"2023-10-21T17:19:25.608001+0000","last_unstale":"2023-10-21T17:19:27.624783+0000","last_undegraded":"2023-10-21T17:19:27.624783+0000","last_fullsized":"2023-10-21T17:19:27.624783+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:23:06.904040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.624363+0000","last_change":"2023-10-21T17:19:26.660439+0000","last_active":"2023-10-21T17:19:27.624363+0000","last_peered":"2023-10-21T17:19:27.624363+0000","last_clean":"2023-10-21T17:19:27.624363+0000","last_became_active":"2023-10-21T17:19:26.660143+0000","last_became_peered":"2023-10-21T17:19:26.660143+0000","last_unstale":"2023-10-21T17:19:27.624363+0000","last_undegraded":"2023-10-21T17:19:27.624363+0000","last_fullsized":"2023-10-21T17:19:27.624363+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:39:43.707718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.899569+0000","last_change":"2023-10-21T17:19:25.628158+0000","last_active":"2023-10-21T17:19:27.899569+0000","last_peered":"2023-10-21T17:19:27.899569+0000","last_clean":"2023-10-21T17:19:27.899569+0000","last_became_active":"2023-10-21T17:19:25.627779+0000","last_became_peered":"2023-10-21T17:19:25.627779+0000","last_unstale":"2023-10-21T17:19:27.899569+0000","last_undegraded":"2023-10-21T17:19:27.899569+0000","last_fullsized":"2023-10-21T17:19:27.899569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:32:35.760176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.626336+0000","last_change":"2023-10-21T17:19:26.666341+0000","last_active":"2023-10-21T17:19:27.626336+0000","last_peered":"2023-10-21T17:19:27.626336+0000","last_clean":"2023-10-21T17:19:27.626336+0000","last_became_active":"2023-10-21T17:19:26.666044+0000","last_became_peered":"2023-10-21T17:19:26.666044+0000","last_unstale":"2023-10-21T17:19:27.626336+0000","last_undegraded":"2023-10-21T17:19:27.626336+0000","last_fullsized":"2023-10-21T17:19:27.626336+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:25.583488+0000","last_clean_scrub_stamp":"2023-10-21T17:19:25.583488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:43:00.990344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-21T17:19:27.900709+0000","last_change":"2023-10-21T17:19:25.635439+0000","last_active":"2023-10-21T17:19:27.900709+0000","last_peered":"2023-10-21T17:19:27.900709+0000","last_clean":"2023-10-21T17:19:27.900709+0000","last_became_active":"2023-10-21T17:19:25.635177+0000","last_became_peered":"2023-10-21T17:19:25.635177+0000","last_unstale":"2023-10-21T17:19:27.900709+0000","last_undegraded":"2023-10-21T17:19:27.900709+0000","last_fullsized":"2023-10-21T17:19:27.900709+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:19:24.570118+0000","last_clean_scrub_stamp":"2023-10-21T17:19:24.570118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:30.032047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"186'4818","reported_seq":5565,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-21T17:19:26.642247+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:19:26.642247+0000","last_peered":"2023-10-21T17:19:26.642247+0000","last_clean":"2023-10-21T17:19:26.642247+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:19:26.642247+0000","last_undegraded":"2023-10-21T17:19:26.642247+0000","last_fullsized":"2023-10-21T17:19:26.642247+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4818,"log_dups_size":0,"ondisk_log_size":4818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":487,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1744,"num_read_kb":3609,"num_write":487,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4818,"ondisk_log_size":4818,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575772,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3059712,"data_stored":1855221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1519999999999999,"15min":1.1859999999999999},"min":{"1min":0.84099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5920000000000001,"5min":1.841,"15min":1.9430000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.65900000000000003},"max":{"1min":2.6240000000000001,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2130000000000001,"15min":1.2370000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.68300000000000005},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.751},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.272,"15min":1.276},"min":{"1min":0.93700000000000006,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.839,"5min":1.839,"15min":2.081},"last":0.68999999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3440000000000001,"15min":1.349},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.589,"5min":1.8140000000000001,"15min":2.1440000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.335,"5min":1.357,"15min":1.365},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88300000000000001},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.83899999999999997}]},{"osd":3,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.304,"15min":1.306},"min":{"1min":1.0840000000000001,"5min":0.94999999999999996,"15min":0.81699999999999995},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.466,"5min":1.4890000000000001,"15min":1.496},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":0.95199999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.05}]},{"osd":4,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.755},"max":{"1min":1.6639999999999999,"5min":1.865,"15min":2.1269999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4650000000000001,"15min":1.484},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.85899999999999999},"max":{"1min":1.7769999999999999,"5min":1.8540000000000001,"15min":2.0830000000000002},"last":1.01}]},{"osd":5,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4550000000000001,"15min":1.46},"min":{"1min":0.92100000000000004,"5min":0.91300000000000003,"15min":0.80100000000000005},"max":{"1min":2.089,"5min":2.089,"15min":2.1560000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.52,"15min":1.5049999999999999},"min":{"1min":1.1439999999999999,"5min":0.93999999999999995,"15min":0.91900000000000004},"max":{"1min":2.7429999999999999,"5min":2.7429999999999999,"15min":2.7429999999999999},"last":1.0349999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:18:28 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5940000000000001,"15min":1.5920000000000001},"min":{"1min":1.2090000000000001,"5min":0.996,"15min":0.92400000000000004},"max":{"1min":1.9610000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":1.083},{"interface":"front","average":{"1min":1.446,"5min":1.4890000000000001,"15min":1.522},"min":{"1min":1.0680000000000001,"5min":1,"15min":0.92900000000000005},"max":{"1min":1.895,"5min":2.0150000000000001,"15min":2.1970000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575773,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41908,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329932,"statfs":{"total":96636764160,"available":96593850368,"internally_reserved":0,"allocated":2465792,"data_stored":1264853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.206},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.212,"5min":1.2210000000000001,"15min":1.262},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.8680000000000001},"last":1.3999999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.268,"15min":1.244},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2849999999999999,"15min":1.3200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.016}]},{"osd":2,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.3069999999999999,"15min":1.3129999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8819999999999999,"5min":1.9970000000000001,"15min":2.0659999999999998},"last":1.7789999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6970000000000001,"5min":1.7969999999999999,"15min":2.415},"last":1.254}]},{"osd":3,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.383,"15min":1.391},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.468,"15min":1.466},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.6509999999999998},"last":1.4370000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3660000000000001,"15min":1.363},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.63500000000000001},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.615},{"interface":"front","average":{"1min":1.367,"5min":1.429,"15min":1.403},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7769999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.575,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.9370000000000001,"5min":2.1019999999999999,"15min":2.5419999999999998},"last":1.9350000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.429,"15min":1.3839999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.016},"last":1.546}]},{"osd":7,"last update":"Sat Oct 21 17:19:16 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5129999999999999,"15min":1.524},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8200000000000001,"5min":2.02,"15min":2.4049999999999998},"last":1.732},{"interface":"front","average":{"1min":1.621,"5min":1.6830000000000001,"15min":1.651},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.129,"5min":2.3210000000000002,"15min":3.4180000000000001},"last":1.8779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1264977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.143,"15min":1.143},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.625},"max":{"1min":1.6140000000000001,"5min":1.718,"15min":1.7649999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.9079999999999999},"last":1.6659999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.264,"15min":1.2909999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.6259999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.5},{"interface":"front","average":{"1min":1.145,"5min":1.234,"15min":1.25},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8120000000000001,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.91600000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.349,"15min":1.3400000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.742,"5min":1.7789999999999999,"15min":1.927},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.252,"15min":1.2729999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.76000000000000001},"max":{"1min":1.909,"5min":1.909,"15min":2.2829999999999999},"last":1.448}]},{"osd":3,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.377,"15min":1.385},"min":{"1min":0.996,"5min":0.996,"15min":0.84999999999999998},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.367,"15min":1.4179999999999999},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.5449999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.2509999999999999,"15min":1.248},"min":{"1min":0.92400000000000004,"5min":0.752,"15min":0.752},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1800000000000002},"last":1.304},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.3799999999999999,"15min":1.3859999999999999},"min":{"1min":0.88700000000000001,"5min":0.77300000000000002,"15min":0.70199999999999996},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":1.379}]},{"osd":6,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.454,"15min":1.484},"min":{"1min":1.125,"5min":0.90000000000000002,"15min":0.89600000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.335},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.518,"15min":1.5129999999999999},"min":{"1min":1.0289999999999999,"5min":0.74199999999999999,"15min":0.67000000000000004},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3479999999999999},"last":1.569}]},{"osd":7,"last update":"Sat Oct 21 17:19:21 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.548,"15min":1.5569999999999999},"min":{"1min":1.004,"5min":1.004,"15min":0.79000000000000004},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.724},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.5649999999999999,"15min":1.569},"min":{"1min":1.25,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.3069999999999999},"last":1.625}]}]},{"osd":4,"up_from":13,"seq":55834575772,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29452,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342388,"statfs":{"total":96636764160,"available":96606605312,"internally_reserved":0,"allocated":2482176,"data_stored":1265617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1879999999999999,"15min":1.1870000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.6200000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.024},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2210000000000001,"15min":1.23},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.78600000000000003},"max":{"1min":1.627,"5min":1.907,"15min":1.907},"last":1.375}]},{"osd":1,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2989999999999999,"15min":1.2809999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.621,"5min":1.6479999999999999,"15min":1.855},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2410000000000001,"15min":1.258},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5780000000000001,"5min":1.794,"15min":1.9139999999999999},"last":1.1830000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.232,"15min":1.2509999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.643,"5min":1.6719999999999999,"15min":1.821},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.391,"15min":1.3839999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":1.653,"5min":1.923,"15min":1.923},"last":1.4399999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.351,"15min":1.3440000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.82799999999999996},"max":{"1min":1.6479999999999999,"5min":1.877,"15min":1.877},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.399,"15min":1.407},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.5740000000000001,"5min":1.8740000000000001,"15min":1.9299999999999999},"last":1.3120000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3360000000000001,"15min":1.3149999999999999},"min":{"1min":1.052,"5min":0.88400000000000001,"15min":0.83199999999999996},"max":{"1min":1.6819999999999999,"5min":1.802,"15min":1.895},"last":1.246},{"interface":"front","average":{"1min":1.329,"5min":1.3500000000000001,"15min":1.347},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.7649999999999999,"5min":1.944,"15min":1.9870000000000001},"last":0.94099999999999995}]},{"osd":6,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4179999999999999,"15min":1.4159999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.8420000000000001,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.6080000000000001,"15min":1.5920000000000001},"min":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":0.85599999999999998},"max":{"1min":1.863,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.498}]},{"osd":7,"last update":"Sat Oct 21 17:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5600000000000001,"15min":1.5249999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.8420000000000001,"5min":2.1110000000000002,"15min":2.2429999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.6080000000000001,"15min":1.5800000000000001},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":1.8320000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.4670000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51660,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320180,"statfs":{"total":96636764160,"available":96583864320,"internally_reserved":0,"allocated":2486272,"data_stored":1267069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.92600000000000005,"15min":0.95499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.47299999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.2330000000000001,"15min":1.2669999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4259999999999999,"5min":1.766,"15min":2.121},"last":1.292}]},{"osd":1,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.073,"15min":1.083},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.331,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.8080000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.2589999999999999,"15min":1.2909999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.7130000000000001,"5min":1.766,"15min":1.766},"last":1.242}]},{"osd":2,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.093,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6080000000000001,"5min":1.784,"15min":1.9790000000000001},"last":1.423},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.26,"15min":1.284},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.7610000000000001,"5min":2.7610000000000001,"15min":2.7610000000000001},"last":1.056}]},{"osd":4,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3959999999999999,"15min":1.4530000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.73499999999999999},"max":{"1min":1.702,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.647},{"interface":"front","average":{"1min":1.304,"5min":1.4670000000000001,"15min":1.472},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.5469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4339999999999999,"15min":1.476},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.8100000000000001,"5min":2.2210000000000001,"15min":2.3239999999999998},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.5189999999999999,"15min":1.55},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.2450000000000001},"last":1.7609999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.5680000000000001,"15min":1.593},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1970000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.923},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.55,"15min":1.579},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.88500000000000001},"max":{"1min":2.1659999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.696}]},{"osd":7,"last update":"Sat Oct 21 17:19:19 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.645,"15min":1.667},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.97399999999999998},"max":{"1min":2.3119999999999998,"5min":2.3319999999999999,"15min":2.4239999999999999},"last":1.958},{"interface":"front","average":{"1min":1.484,"5min":1.694,"15min":1.71},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":1.996}]}]},{"osd":2,"up_from":13,"seq":55834575773,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51644,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320196,"statfs":{"total":96636764160,"available":96583880704,"internally_reserved":0,"allocated":2465792,"data_stored":1264853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.89700000000000002,"15min":0.90200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.891,"5min":1.891,"15min":2.3199999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.036,"15min":1.0569999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.6639999999999999,"5min":1.835,"15min":1.875},"last":1.1140000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.179,"15min":1.1830000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.087,"5min":1.3300000000000001,"15min":1.3859999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.1070000000000002},"last":1.3560000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.179,"15min":1.2110000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":1.075},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.4510000000000001,"15min":1.478},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.6480000000000001,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.425,"15min":1.4079999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":3.0510000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4099999999999999,"15min":1.4530000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8899999999999999,"5min":2.0489999999999999,"15min":2.0579999999999998},"last":1.853}]},{"osd":5,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.421,"15min":1.415},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.599,"15min":1.6120000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":3.2050000000000001},"last":1.571}]},{"osd":6,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5489999999999999,"15min":1.556},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.0259999999999998,"5min":2.21,"15min":2.25},"last":2.089},{"interface":"front","average":{"1min":1.395,"5min":1.4950000000000001,"15min":1.5289999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.429}]},{"osd":7,"last update":"Sat Oct 21 17:19:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.6000000000000001,"15min":1.6419999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.351,"5min":2.351,"15min":2.8959999999999999},"last":1.978},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5740000000000001,"15min":1.597},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.331}]}]},{"osd":1,"up_from":13,"seq":55834575772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29444,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342396,"statfs":{"total":96636764160,"available":96606613504,"internally_reserved":0,"allocated":2478080,"data_stored":1266465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.94399999999999995,"15min":0.94899999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.7330000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.081,"15min":1.1140000000000001},"min":{"1min":0.46000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3999999999999999,"5min":1.966,"15min":1.966},"last":1.274}]},{"osd":2,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.0249999999999999,"15min":1.0469999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.302,"5min":1.724,"15min":2.79},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.25,"15min":1.24},"min":{"1min":0.54300000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.748,"5min":1.891,"15min":2.2549999999999999},"last":1.222}]},{"osd":3,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.3300000000000001,"15min":1.375},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7749999999999999,"5min":1.9339999999999999,"15min":3.0920000000000001},"last":1.085},{"interface":"front","average":{"1min":1.139,"5min":1.228,"15min":1.274},"min":{"1min":0.628,"5min":0.44,"15min":0.44},"max":{"1min":1.968,"5min":1.968,"15min":1.976},"last":1.0189999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.347,"15min":1.319},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.806,"5min":2.105,"15min":2.105},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4950000000000001,"15min":1.51},"min":{"1min":0.86099999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1190000000000002,"5min":2.2410000000000001,"15min":2.512},"last":1.385}]},{"osd":5,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5429999999999999},"min":{"1min":0.90000000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.0329999999999999,"5min":2.246,"15min":2.246},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.5620000000000001,"15min":1.5489999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.0059999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.357}]},{"osd":6,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.569,"15min":1.5629999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0739999999999998,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.575,"15min":1.552},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.911,"5min":2.214,"15min":2.2570000000000001},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.6599999999999999,"15min":1.6739999999999999},"min":{"1min":0.97199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":2.254,"5min":2.3220000000000001,"15min":2.391},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.645,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.99099999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.1429999999999998,"5min":2.4769999999999999,"15min":2.4969999999999999},"last":1.907}]}]},{"osd":0,"up_from":13,"seq":55834575772,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30024,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341816,"statfs":{"total":96636764160,"available":96606019584,"internally_reserved":0,"allocated":3067904,"data_stored":1855325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96799999999999997,"15min":0.95299999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5840000000000001,"5min":1.631,"15min":1.726},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.107,"15min":1.1120000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.4890000000000001,"5min":1.7989999999999999,"15min":2.6200000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1679999999999999,"15min":1.171},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.7330000000000001,"15min":2.8140000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.204,"15min":1.2070000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.46,"5min":1.7170000000000001,"15min":1.784},"last":1.143}]},{"osd":3,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.1060000000000001,"15min":1.147},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.718,"5min":1.883,"15min":1.8919999999999999},"last":0.752},{"interface":"front","average":{"1min":1.258,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6539999999999999,"5min":1.9239999999999999,"15min":2.6949999999999998},"last":1.2250000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.6299999999999999,"15min":1.619},"min":{"1min":1.278,"5min":1.278,"15min":1.1799999999999999},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.766},"last":1.4570000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.96199999999999997},"max":{"1min":1.7729999999999999,"5min":2.125,"15min":2.6549999999999998},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5660000000000001,"15min":1.5600000000000001},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":1.952,"5min":2.3300000000000001,"15min":2.9209999999999998},"last":1.55}]},{"osd":6,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.5229999999999999,"15min":1.5129999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.2759999999999998},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.589,"5min":1.6990000000000001,"15min":1.665},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.0230000000000001,"5min":2.4510000000000001,"15min":3.069},"last":1.6100000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:18:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.556,"15min":1.556},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.794,"5min":2.0910000000000002,"15min":2.4790000000000001},"last":1.752},{"interface":"front","average":{"1min":1.504,"5min":1.639,"15min":1.631},"min":{"1min":1.034,"5min":1.034,"15min":1.034},"max":{"1min":2.0529999999999999,"5min":2.1930000000000001,"15min":2.613},"last":1.4019999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:19:31.880 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:19:31.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:19:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.157+0000 7fde3cfe9640 1 Processor -- start 2023-10-21T17:19:32.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.157+0000 7fde3cfe9640 1 -- start start 2023-10-21T17:19:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.158+0000 7fde3cfe9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde380cdc20 0x7fde380d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.158+0000 7fde3cfe9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fde380cf3a0 0x7fde380cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.158+0000 7fde3cfe9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde380d2700 con 0x7fde38160090 2023-10-21T17:19:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.158+0000 7fde3cfe9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde380cfd70 con 0x7fde380cf3a0 2023-10-21T17:19:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.158+0000 7fde3cfe9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde38160470 con 0x7fde380cdc20 2023-10-21T17:19:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde37fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fde380cf3a0 0x7fde380cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde377fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fde38160090 0x7fde380d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40772/0 (socket says 172.21.15.53:40772) 2023-10-21T17:19:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde377fe640 1 -- 172.21.15.53:0/2810988717 learned_addr learned my addr 172.21.15.53:0/2810988717 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde2effd640 1 --2- 172.21.15.53:0/2810988717 >> v2:172.21.15.124:3300/0 conn(0x7fde380cdc20 0x7fde380d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde37fff640 1 -- 172.21.15.53:0/2810988717 >> v2:172.21.15.124:3300/0 conn(0x7fde380cdc20 msgr2=0x7fde380d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde37fff640 1 --2- 172.21.15.53:0/2810988717 >> v2:172.21.15.124:3300/0 conn(0x7fde380cdc20 0x7fde380d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde37fff640 1 -- 172.21.15.53:0/2810988717 >> v1:172.21.15.53:6789/0 conn(0x7fde38160090 legacy=0x7fde380d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.159+0000 7fde37fff640 1 -- 172.21.15.53:0/2810988717 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380d0690 con 0x7fde380cf3a0 2023-10-21T17:19:32.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde37fff640 1 --2- 172.21.15.53:0/2810988717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cf3a0 0x7fde380cf830 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fde380b5580 tx=0x7fde3002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c317df0d9c20f6b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde36ffd640 1 -- 172.21.15.53:0/2810988717 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3002cbe0 con 0x7fde380cf3a0 2023-10-21T17:19:32.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde36ffd640 1 -- 172.21.15.53:0/2810988717 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3002cd40 con 0x7fde380cf3a0 2023-10-21T17:19:32.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde36ffd640 1 -- 172.21.15.53:0/2810988717 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30034660 con 0x7fde380cf3a0 2023-10-21T17:19:32.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde3cfe9640 1 -- 172.21.15.53:0/2810988717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cf3a0 msgr2=0x7fde380cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.161+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/2810988717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cf3a0 0x7fde380cf830 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fde380b5580 tx=0x7fde3002be30 comp rx=0 tx=0).stop 2023-10-21T17:19:32.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 -- 172.21.15.53:0/2810988717 shutdown_connections 2023-10-21T17:19:32.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/2810988717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cf3a0 0x7fde380cf830 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/2810988717 >> v2:172.21.15.124:3300/0 conn(0x7fde380cdc20 0x7fde380d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 -- 172.21.15.53:0/2810988717 >> 172.21.15.53:0/2810988717 conn(0x7fde3806af50 msgr2=0x7fde3806b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:32.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 -- 172.21.15.53:0/2810988717 shutdown_connections 2023-10-21T17:19:32.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.162+0000 7fde3cfe9640 1 -- 172.21.15.53:0/2810988717 wait complete. 2023-10-21T17:19:32.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.163+0000 7fde3cfe9640 1 Processor -- start 2023-10-21T17:19:32.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.164+0000 7fde3cfe9640 1 -- start start 2023-10-21T17:19:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.164+0000 7fde3cfe9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.164+0000 7fde3cfe9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde380cf3a0 0x7fde38155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.164+0000 7fde3cfe9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde38160090 0x7fde38147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde3cfe9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380c9550 con 0x7fde380cdc20 2023-10-21T17:19:32.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde3cfe9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde38156190 con 0x7fde380cf3a0 2023-10-21T17:19:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde3cfe9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde38158500 con 0x7fde38160090 2023-10-21T17:19:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34444/0 (socket says 172.21.15.53:34444) 2023-10-21T17:19:32.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 -- 172.21.15.53:0/3616743494 learned_addr learned my addr 172.21.15.53:0/3616743494 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:32.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde37fff640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde38160090 0x7fde38147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde2effd640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde380cf3a0 0x7fde38155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 -- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde38160090 msgr2=0x7fde38147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde38160090 0x7fde38147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 -- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde380cf3a0 msgr2=0x7fde38155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde380cf3a0 0x7fde38155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.165+0000 7fde377fe640 1 -- 172.21.15.53:0/3616743494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30009a70 con 0x7fde380cdc20 2023-10-21T17:19:32.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde377fe640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c4a0 tx=0x7fde2800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005ce0 con 0x7fde380cdc20 2023-10-21T17:19:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28005e40 con 0x7fde380cdc20 2023-10-21T17:19:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28007270 con 0x7fde380cdc20 2023-10-21T17:19:32.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde381587e0 con 0x7fde380cdc20 2023-10-21T17:19:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.166+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde38158da0 con 0x7fde380cdc20 2023-10-21T17:19:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.167+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde38073110 con 0x7fde380cdc20 2023-10-21T17:19:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.168+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde28007490 con 0x7fde380cdc20 2023-10-21T17:19:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.169+0000 7fde34ff9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 0x7fde1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.169+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fde28088d70 con 0x7fde380cdc20 2023-10-21T17:19:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.170+0000 7fde2effd640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 0x7fde1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.171+0000 7fde2effd640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 0x7fde1806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.174+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde28053bf0 con 0x7fde380cdc20 2023-10-21T17:19:32.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.438+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde380b5560 con 0x7fde380cdc20 2023-10-21T17:19:32.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.443+0000 7fde34ff9640 1 -- 172.21.15.53:0/3616743494 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+147 (secure 0 0 0) 0x7fde28017020 con 0x7fde380cdc20 2023-10-21T17:19:32.445 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:19:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:19:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:19:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:19:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:19:32.446 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:19:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 msgr2=0x7fde1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 0x7fde1806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).stop 2023-10-21T17:19:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 msgr2=0x7fde381f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c4a0 tx=0x7fde2800c960 comp rx=0 tx=0).stop 2023-10-21T17:19:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 shutdown_connections 2023-10-21T17:19:32.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde38160090 0x7fde38147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde380cf3a0 0x7fde38155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fde1806af90 0x7fde1806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 --2- 172.21.15.53:0/3616743494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde380cdc20 0x7fde381f5670 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 >> 172.21.15.53:0/3616743494 conn(0x7fde3806af50 msgr2=0x7fde380cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:32.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 shutdown_connections 2023-10-21T17:19:32.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.446+0000 7fde3cfe9640 1 -- 172.21.15.53:0/3616743494 wait complete. 2023-10-21T17:19:32.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:19:32.579 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-21T17:19:32.578+0000 7f1ae304d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:19:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.682+0000 7efc77fff640 1 Processor -- start 2023-10-21T17:19:32.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.682+0000 7efc77fff640 1 -- start start 2023-10-21T17:19:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 0x7efc78071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 0x7efc7815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc7815d380 0x7efc78159850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc78071550 con 0x7efc7815d380 2023-10-21T17:19:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc7815d7f0 con 0x7efc78150440 2023-10-21T17:19:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.684+0000 7efc77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc780c9f90 con 0x7efc780c97d0 2023-10-21T17:19:32.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc7815d380 0x7efc78159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efc7815d380 0x7efc78159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34452/0 (socket says 172.21.15.53:34452) 2023-10-21T17:19:32.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 -- 172.21.15.53:0/863210123 learned_addr learned my addr 172.21.15.53:0/863210123 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc76ffd640 1 --2- 172.21.15.53:0/863210123 >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 0x7efc78071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc767fc640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 0x7efc7815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 -- 172.21.15.53:0/863210123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 msgr2=0x7efc7815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:32.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 0x7efc7815ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 -- 172.21.15.53:0/863210123 >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 msgr2=0x7efc78071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 --2- 172.21.15.53:0/863210123 >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 0x7efc78071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc777fe640 1 -- 172.21.15.53:0/863210123 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc780ca270 con 0x7efc7815d380 2023-10-21T17:19:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc767fc640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 0x7efc7815ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:32.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.685+0000 7efc76ffd640 1 --2- 172.21.15.53:0/863210123 >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 0x7efc78071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.686+0000 7efc777fe640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7815d380 0x7efc78159850 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7efc780cbe60 tx=0x7efc7002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f085ed650f35170c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc75ffb640 1 -- 172.21.15.53:0/863210123 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7002ca80 con 0x7efc7815d380 2023-10-21T17:19:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc75ffb640 1 -- 172.21.15.53:0/863210123 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7002cbe0 con 0x7efc7815d380 2023-10-21T17:19:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc75ffb640 1 -- 172.21.15.53:0/863210123 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc700343f0 con 0x7efc7815d380 2023-10-21T17:19:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc77fff640 1 -- 172.21.15.53:0/863210123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7815d380 msgr2=0x7efc78159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc77fff640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7815d380 0x7efc78159850 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7efc780cbe60 tx=0x7efc7002bee0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.687+0000 7efc77fff640 1 -- 172.21.15.53:0/863210123 shutdown_connections 2023-10-21T17:19:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc780c97d0 0x7efc7815ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 --2- 172.21.15.53:0/863210123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc7815d380 0x7efc78159850 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 --2- 172.21.15.53:0/863210123 >> v2:172.21.15.124:3300/0 conn(0x7efc78150440 0x7efc78071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 -- 172.21.15.53:0/863210123 >> 172.21.15.53:0/863210123 conn(0x7efc780cb0c0 msgr2=0x7efc780cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 -- 172.21.15.53:0/863210123 shutdown_connections 2023-10-21T17:19:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.688+0000 7efc77fff640 1 -- 172.21.15.53:0/863210123 wait complete. 2023-10-21T17:19:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.690+0000 7efc77fff640 1 Processor -- start 2023-10-21T17:19:32.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.690+0000 7efc77fff640 1 -- start start 2023-10-21T17:19:32.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc78150440 0x7efc781e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc7815d380 0x7efc781e5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780b31a0 con 0x7efc780c97d0 2023-10-21T17:19:32.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780b3480 con 0x7efc78150440 2023-10-21T17:19:32.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc780b3760 con 0x7efc7815d380 2023-10-21T17:19:32.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34464/0 (socket says 172.21.15.53:34464) 2023-10-21T17:19:32.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc76ffd640 1 -- 172.21.15.53:0/619678041 learned_addr learned my addr 172.21.15.53:0/619678041 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:32.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.691+0000 7efc767fc640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc78150440 0x7efc781e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc777fe640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc7815d380 0x7efc781e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 -- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc7815d380 msgr2=0x7efc781e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc7815d380 0x7efc781e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 -- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc78150440 msgr2=0x7efc781e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc78150440 0x7efc781e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 -- 172.21.15.53:0/619678041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70009b30 con 0x7efc780c97d0 2023-10-21T17:19:32.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc76ffd640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c7f0 tx=0x7efc6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68007a30 con 0x7efc780c97d0 2023-10-21T17:19:32.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc68007b90 con 0x7efc780c97d0 2023-10-21T17:19:32.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6800ce40 con 0x7efc780c97d0 2023-10-21T17:19:32.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.692+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc780b3a40 con 0x7efc780c97d0 2023-10-21T17:19:32.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.693+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc780b2cc0 con 0x7efc780c97d0 2023-10-21T17:19:32.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.694+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc780ca530 con 0x7efc780c97d0 2023-10-21T17:19:32.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.695+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc680113f0 con 0x7efc780c97d0 2023-10-21T17:19:32.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.696+0000 7efc6f7fe640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 0x7efc5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:32.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.696+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7efc68089bc0 con 0x7efc780c97d0 2023-10-21T17:19:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.696+0000 7efc767fc640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 0x7efc5806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.698+0000 7efc767fc640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 0x7efc5806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc7815b900 tx=0x7efc60008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:32.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.701+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc780ca530 con 0x7efc780c97d0 2023-10-21T17:19:32.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.933+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc780ca530 con 0x7efc780c97d0 2023-10-21T17:19:32.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.934+0000 7efc6f7fe640 1 -- 172.21.15.53:0/619678041 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7efc780ca530 con 0x7efc780c97d0 2023-10-21T17:19:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:19:32.936 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:19:32.936 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:19:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 msgr2=0x7efc5806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 0x7efc5806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc7815b900 tx=0x7efc60008040 comp rx=0 tx=0).stop 2023-10-21T17:19:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 msgr2=0x7efc781f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:32.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c7f0 tx=0x7efc6800cbf0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 shutdown_connections 2023-10-21T17:19:32.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc7815d380 0x7efc781e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc78150440 0x7efc781e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7efc5806af90 0x7efc5806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 --2- 172.21.15.53:0/619678041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc780c97d0 0x7efc781f20a0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:32.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 >> 172.21.15.53:0/619678041 conn(0x7efc780cb0c0 msgr2=0x7efc780cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:32.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 shutdown_connections 2023-10-21T17:19:32.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:32.937+0000 7efc77fff640 1 -- 172.21.15.53:0/619678041 wait complete. 2023-10-21T17:19:33.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:19:26.079520+0000', 'modified': '2023-10-21T17:19:29.356039+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 37331}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_37331': {'gid': 37331, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/2015564828', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 2015564828}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 2015564828}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:19:33.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:19:33.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-21T17:19:33.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.595+0000 7feb2f59e640 1 Processor -- start 2023-10-21T17:19:33.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.595+0000 7feb2f59e640 1 -- start start 2023-10-21T17:19:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2f59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb300cf2b0 0x7feb300d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feb300cf690 0x7feb300ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300cd520 con 0x7feb300d1100 2023-10-21T17:19:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2f59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300d1910 con 0x7feb300cf2b0 2023-10-21T17:19:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300ce080 con 0x7feb300cf690 2023-10-21T17:19:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feb300cf2b0 0x7feb300d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2ed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb300d1100 0x7feb3015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40780/0 (socket says 172.21.15.53:40780) 2023-10-21T17:19:33.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2440804634 learned_addr learned my addr 172.21.15.53:0/2440804634 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:33.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.597+0000 7feb27fff640 1 --2- 172.21.15.53:0/2440804634 >> v2:172.21.15.124:3300/0 conn(0x7feb300cf690 0x7feb300ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.598+0000 7feb2dd9b640 1 -- 172.21.15.53:0/2440804634 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082881742 0 0) 0x7feb300cd520 con 0x7feb300d1100 2023-10-21T17:19:33.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.598+0000 7feb2e59c640 1 -- 172.21.15.53:0/2440804634 >> v2:172.21.15.124:3300/0 conn(0x7feb300cf690 msgr2=0x7feb300ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.598+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2440804634 >> v2:172.21.15.124:3300/0 conn(0x7feb300cf690 0x7feb300ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.598+0000 7feb2e59c640 1 -- 172.21.15.53:0/2440804634 >> v1:172.21.15.53:6789/0 conn(0x7feb300d1100 legacy=0x7feb3015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.598+0000 7feb2e59c640 1 -- 172.21.15.53:0/2440804634 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300cd750 con 0x7feb300cf2b0 2023-10-21T17:19:33.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.599+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2440804634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb300d34d0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7feb3008baa0 tx=0x7feb2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0a15b9220ba965 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.600+0000 7feb2dd9b640 1 -- 172.21.15.53:0/2440804634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2002e070 con 0x7feb300cf2b0 2023-10-21T17:19:33.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.600+0000 7feb2dd9b640 1 -- 172.21.15.53:0/2440804634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200315a0 con 0x7feb300cf2b0 2023-10-21T17:19:33.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.600+0000 7feb2dd9b640 1 -- 172.21.15.53:0/2440804634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20031930 con 0x7feb300cf2b0 2023-10-21T17:19:33.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.600+0000 7feb2f59e640 1 -- 172.21.15.53:0/2440804634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 msgr2=0x7feb300d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.600+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2440804634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb300d34d0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7feb3008baa0 tx=0x7feb2002b840 comp rx=0 tx=0).stop 2023-10-21T17:19:33.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 -- 172.21.15.53:0/2440804634 shutdown_connections 2023-10-21T17:19:33.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2440804634 >> v2:172.21.15.124:3300/0 conn(0x7feb300cf690 0x7feb300ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2440804634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb300d34d0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 -- 172.21.15.53:0/2440804634 >> 172.21.15.53:0/2440804634 conn(0x7feb3009bc40 msgr2=0x7feb300d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 -- 172.21.15.53:0/2440804634 shutdown_connections 2023-10-21T17:19:33.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.601+0000 7feb2f59e640 1 -- 172.21.15.53:0/2440804634 wait complete. 2023-10-21T17:19:33.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.602+0000 7feb2f59e640 1 Processor -- start 2023-10-21T17:19:33.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 -- start start 2023-10-21T17:19:33.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb300cf690 0x7feb300c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb300d1100 0x7feb300c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300b5660 con 0x7feb300cf2b0 2023-10-21T17:19:33.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300b5940 con 0x7feb300d1100 2023-10-21T17:19:33.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.603+0000 7feb2f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300b5c20 con 0x7feb300cf690 2023-10-21T17:19:33.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34478/0 (socket says 172.21.15.53:34478) 2023-10-21T17:19:33.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 -- 172.21.15.53:0/2861837507 learned_addr learned my addr 172.21.15.53:0/2861837507 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb27fff640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb300cf690 0x7feb300c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 -- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb300cf690 msgr2=0x7feb300c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb300cf690 0x7feb300c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 -- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb300d1100 msgr2=0x7feb300c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb300d1100 0x7feb300c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.604+0000 7feb2e59c640 1 -- 172.21.15.53:0/2861837507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20009400 con 0x7feb300cf2b0 2023-10-21T17:19:33.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7feb20009890 tx=0x7feb200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb26ffd640 1 -- 172.21.15.53:0/2861837507 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20032840 con 0x7feb300cf2b0 2023-10-21T17:19:33.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb26ffd640 1 -- 172.21.15.53:0/2861837507 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb20031f00 con 0x7feb300cf2b0 2023-10-21T17:19:33.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb26ffd640 1 -- 172.21.15.53:0/2861837507 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20032be0 con 0x7feb300cf2b0 2023-10-21T17:19:33.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb300b5f00 con 0x7feb300cf2b0 2023-10-21T17:19:33.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.605+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb300b6700 con 0x7feb300cf2b0 2023-10-21T17:19:33.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.608+0000 7feb26ffd640 1 -- 172.21.15.53:0/2861837507 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb20032d40 con 0x7feb300cf2b0 2023-10-21T17:19:33.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.609+0000 7feb26ffd640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 0x7feb1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.609+0000 7feb26ffd640 1 -- 172.21.15.53:0/2861837507 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feb200ab9d0 con 0x7feb300cf2b0 2023-10-21T17:19:33.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.609+0000 7feb37436740 1 Processor -- start 2023-10-21T17:19:33.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.609+0000 7feb37436740 1 -- start start 2023-10-21T17:19:33.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.609+0000 7feb27fff640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 0x7feb1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c6c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591b8c7a0 con 0x55c591c6c5b0 2023-10-21T17:19:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591bde740 con 0x55c591c99550 2023-10-21T17:19:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c591bdbe30 con 0x55c591c99170 2023-10-21T17:19:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34482/0 (socket says 172.21.15.53:34482) 2023-10-21T17:19:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.610+0000 7feb2ed9d640 1 -- 172.21.15.53:0/280600888 learned_addr learned my addr 172.21.15.53:0/280600888 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb27fff640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 0x7feb1006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb301e7090 tx=0x7feb18008040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2e59c640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c6c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb27fff640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 -- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 msgr2=0x55c591c6c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c6c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 -- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 msgr2=0x55c591c9bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c9bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 -- 172.21.15.53:0/280600888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c591b9b910 con 0x55c591c6c5b0 2023-10-21T17:19:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb27fff640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7feb28001db0 tx=0x7feb2800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c5bda93d3a0899 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb0ffff640 1 -- 172.21.15.53:0/280600888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800cdc0 con 0x55c591c6c5b0 2023-10-21T17:19:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb0ffff640 1 -- 172.21.15.53:0/280600888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28004b60 con 0x55c591c6c5b0 2023-10-21T17:19:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.611+0000 7feb0ffff640 1 -- 172.21.15.53:0/280600888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280070d0 con 0x55c591c6c5b0 2023-10-21T17:19:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 -- 172.21.15.53:0/280600888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 msgr2=0x55c591c9c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7feb28001db0 tx=0x7feb2800c4b0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 -- 172.21.15.53:0/280600888 shutdown_connections 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c6c5b0 0x55c591c9c440 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 --2- 172.21.15.53:0/280600888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c6c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 -- 172.21.15.53:0/280600888 >> 172.21.15.53:0/280600888 conn(0x55c591c6e380 msgr2=0x55c591cb6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 -- 172.21.15.53:0/280600888 shutdown_connections 2023-10-21T17:19:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.612+0000 7feb37436740 1 -- 172.21.15.53:0/280600888 wait complete. 2023-10-21T17:19:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.613+0000 7feb37436740 1 -- start start 2023-10-21T17:19:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.613+0000 7feb37436740 1 Processor -- start 2023-10-21T17:19:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c9fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591bd7720 con 0x55c591c97080 2023-10-21T17:19:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591b75970 con 0x55c591c99550 2023-10-21T17:19:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c591b75610 con 0x55c591c99170 2023-10-21T17:19:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.614+0000 7feb37436740 10 client.0 _ll_register_callbacks cb 0x55c591b979d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:19:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34488/0 (socket says 172.21.15.53:34488) 2023-10-21T17:19:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 -- 172.21.15.53:0/258248591 learned_addr learned my addr 172.21.15.53:0/258248591 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 -- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 msgr2=0x55c591c9fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c9fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 -- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 msgr2=0x55c591c963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 -- 172.21.15.53:0/258248591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb280019f0 con 0x55c591c97080 2023-10-21T17:19:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7feb280012e0 tx=0x7feb28001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c12fae19402917 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.615+0000 7feb0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:19:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280074e0 con 0x55c591c97080 2023-10-21T17:19:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c591bfe160 con 0x55c591c97080 2023-10-21T17:19:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feb28007640 con 0x55c591c97080 2023-10-21T17:19:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb28007800 con 0x55c591c97080 2023-10-21T17:19:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 10 client.10275 fetch_fsmap learned FSMap version 95 2023-10-21T17:19:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55c591bd5da0 con 0x55c591c97080 2023-10-21T17:19:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c591bfe160 con 0x55c591c97080 2023-10-21T17:19:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 95) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb28020480 con 0x55c591c97080 2023-10-21T17:19:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 10 client.10275 fetch_fsmap finished waiting for FSMap version 95 2023-10-21T17:19:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 10 client.10275 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:19:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 20 client.10275 populate_metadata read hostname 'smithi053' 2023-10-21T17:19:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591ccaff0 0x55c591cd4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 4 client.10275 mds_command: new command op to 37331 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:19:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.616+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 --> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c591ba9cc0 con 0x55c591ccaff0 2023-10-21T17:19:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.617+0000 7feb2e59c640 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591ccaff0 0x55c591cd4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.618+0000 7feb2e59c640 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591ccaff0 0x55c591cd4820 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44918c5b4ce6703c server_cookie=7032583831b3e384 in_seq=0 out_seq=0 2023-10-21T17:19:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.618+0000 7feb0d7fa640 10 client.10275 ms_handle_connect on v2:172.21.15.124:6836/2015564828 2023-10-21T17:19:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.620+0000 7feb0d7fa640 1 -- 172.21.15.53:0/258248591 <== mds.0 v2:172.21.15.124:6836/2015564828 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c591ba9cc0 con 0x55c591ccaff0 2023-10-21T17:19:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.620+0000 7feb0d7fa640 10 client.10275 handle_command_reply: tid=0 2023-10-21T17:19:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.620+0000 7feb37436740 1 client.10275 shutdown 2023-10-21T17:19:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.620+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 msgr2=0x55c591cafcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.620+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7feb280012e0 tx=0x7feb28001470 comp rx=0 tx=0).stop 2023-10-21T17:19:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 shutdown_connections 2023-10-21T17:19:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591ccaff0 0x55c591cd4820 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c97080 0x55c591cafcc0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99550 0x55c591c963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 --2- 172.21.15.53:0/258248591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591c9fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb0d7fa640 0 client.10275 ms_handle_reset on v2:172.21.15.124:6836/2015564828 2023-10-21T17:19:33.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 >> 172.21.15.53:0/258248591 conn(0x55c591c6c5b0 msgr2=0x55c591c6ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 shutdown_connections 2023-10-21T17:19:33.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 1 -- 172.21.15.53:0/258248591 wait complete. 2023-10-21T17:19:33.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.621+0000 7feb37436740 20 client.10275 trim_cache size 0 max 16384 2023-10-21T17:19:33.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.641+0000 7feb37436740 1 Processor -- start 2023-10-21T17:19:33.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.641+0000 7feb37436740 1 -- start start 2023-10-21T17:19:33.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 0x55c591cd8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591cd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c59193bf20 con 0x55c591c99550 2023-10-21T17:19:33.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591cb0f70 con 0x55c591c97080 2023-10-21T17:19:33.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c591c9e920 con 0x55c591c99170 2023-10-21T17:19:33.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34504/0 (socket says 172.21.15.53:34504) 2023-10-21T17:19:33.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2738764121 learned_addr learned my addr 172.21.15.53:0/2738764121 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 0x55c591cd8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 msgr2=0x55c591cd8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.642+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591cd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 msgr2=0x55c591cd8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 0x55c591cd8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2738764121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c591b74b80 con 0x55c591c99550 2023-10-21T17:19:33.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 0x55c591cd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:33.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c4f0 tx=0x7feb2801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776cf12ec4ea8755 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb24ff9640 1 -- 172.21.15.53:0/2738764121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800ce80 con 0x55c591c99550 2023-10-21T17:19:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb24ff9640 1 -- 172.21.15.53:0/2738764121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280072e0 con 0x55c591c99550 2023-10-21T17:19:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.643+0000 7feb24ff9640 1 -- 172.21.15.53:0/2738764121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28007690 con 0x55c591c99550 2023-10-21T17:19:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 -- 172.21.15.53:0/2738764121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 msgr2=0x55c591cc98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c4f0 tx=0x7feb2801e000 comp rx=0 tx=0).stop 2023-10-21T17:19:33.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 -- 172.21.15.53:0/2738764121 shutdown_connections 2023-10-21T17:19:33.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591cc98b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c99170 0x55c591cd8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 --2- 172.21.15.53:0/2738764121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c97080 0x55c591cd8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 -- 172.21.15.53:0/2738764121 >> 172.21.15.53:0/2738764121 conn(0x55c591c6c5b0 msgr2=0x55c591cb1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 -- 172.21.15.53:0/2738764121 shutdown_connections 2023-10-21T17:19:33.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.644+0000 7feb37436740 1 -- 172.21.15.53:0/2738764121 wait complete. 2023-10-21T17:19:33.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.645+0000 7feb37436740 1 -- start start 2023-10-21T17:19:33.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.645+0000 7feb37436740 1 Processor -- start 2023-10-21T17:19:33.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c97080 0x55c591cb80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99170 0x55c591cb8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591d1f990 con 0x55c591c99550 2023-10-21T17:19:33.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c591c9c240 con 0x55c591c99170 2023-10-21T17:19:33.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c591cafd50 con 0x55c591c97080 2023-10-21T17:19:33.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb37436740 10 client.0 _ll_register_callbacks cb 0x7feb28001c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:19:33.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.646+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34512/0 (socket says 172.21.15.53:34512) 2023-10-21T17:19:33.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2237636481 learned_addr learned my addr 172.21.15.53:0/2237636481 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb27fff640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99170 0x55c591cb8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c97080 0x55c591cb80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c97080 msgr2=0x55c591cb80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c97080 0x55c591cb80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99170 msgr2=0x55c591cb8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99170 0x55c591cb8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 -- 172.21.15.53:0/2237636481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c591bd5da0 con 0x55c591c99550 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb2ed9d640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7feb28020940 tx=0x7feb28009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcfcfa6b5564492 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.647+0000 7feb0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28041070 con 0x55c591c99550 2023-10-21T17:19:33.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feb280318a0 con 0x55c591c99550 2023-10-21T17:19:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb28004b60 con 0x55c591c99550 2023-10-21T17:19:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb28031a60 con 0x55c591c99550 2023-10-21T17:19:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 10 client.10281 fetch_fsmap learned FSMap version 95 2023-10-21T17:19:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x55c591b9a170 con 0x55c591c99550 2023-10-21T17:19:33.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb28004b60 con 0x55c591c99550 2023-10-21T17:19:33.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 95) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb28044030 con 0x55c591c99550 2023-10-21T17:19:33.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 10 client.10281 fetch_fsmap finished waiting for FSMap version 95 2023-10-21T17:19:33.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 10 client.10281 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:19:33.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 20 client.10281 populate_metadata read hostname 'smithi053' 2023-10-21T17:19:33.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591cd8f50 0x55c591caf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 4 client.10281 mds_command: new command op to 37331 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:19:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.648+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 --> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c591bd0700 con 0x55c591cd8f50 2023-10-21T17:19:33.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.649+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591cd8f50 0x55c591caf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.650+0000 7feb2e59c640 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591cd8f50 0x55c591caf390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1394b6d04289df82 server_cookie=9f9b722f2e5fbf7e in_seq=0 out_seq=0 2023-10-21T17:19:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.650+0000 7feb0d7fa640 10 client.10281 ms_handle_connect on v2:172.21.15.124:6836/2015564828 2023-10-21T17:19:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb0d7fa640 1 -- 172.21.15.53:0/2237636481 <== mds.0 v2:172.21.15.124:6836/2015564828 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55c591bd0700 con 0x55c591cd8f50 2023-10-21T17:19:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb0d7fa640 10 client.10281 handle_command_reply: tid=0 2023-10-21T17:19:33.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 client.10281 shutdown 2023-10-21T17:19:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 msgr2=0x55c591caee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7feb28020940 tx=0x7feb28009000 comp rx=0 tx=0).stop 2023-10-21T17:19:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 shutdown_connections 2023-10-21T17:19:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:6836/2015564828,v1:172.21.15.124:6837/2015564828] conn(0x55c591cd8f50 0x55c591caf390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c591c99550 0x55c591caee50 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c591c99170 0x55c591cb8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb0d7fa640 0 client.10281 ms_handle_reset on v2:172.21.15.124:6836/2015564828 2023-10-21T17:19:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 --2- 172.21.15.53:0/2237636481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c591c97080 0x55c591cb80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 >> 172.21.15.53:0/2237636481 conn(0x55c591c6c5b0 msgr2=0x55c591cd8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 shutdown_connections 2023-10-21T17:19:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 1 -- 172.21.15.53:0/2237636481 wait complete. 2023-10-21T17:19:33.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.651+0000 7feb37436740 20 client.10281 trim_cache size 0 max 16384 2023-10-21T17:19:33.665 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:19:33.665 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:19:33.666 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:19:33.666 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 37331, 2023-10-21T17:19:33.666 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:19:33.666 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:19:33.666 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:19:33.667 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 7.0047984999999997, 2023-10-21T17:19:33.667 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 95, 2023-10-21T17:19:33.667 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 197, 2023-10-21T17:19:33.667 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 197, 2023-10-21T17:19:33.667 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 12.818080546999999 2023-10-21T17:19:33.668 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:19:33.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 msgr2=0x7feb1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 0x7feb1006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feb301e7090 tx=0x7feb18008040 comp rx=0 tx=0).stop 2023-10-21T17:19:33.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 msgr2=0x7feb3015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7feb20009890 tx=0x7feb200046c0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 shutdown_connections 2023-10-21T17:19:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb300d1100 0x7feb300c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb300cf690 0x7feb300c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7feb1006af90 0x7feb1006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 --2- 172.21.15.53:0/2861837507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb300cf2b0 0x7feb3015e1c0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 >> 172.21.15.53:0/2861837507 conn(0x7feb3009bc40 msgr2=0x7feb301e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 shutdown_connections 2023-10-21T17:19:33.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.652+0000 7feb2f59e640 1 -- 172.21.15.53:0/2861837507 wait complete. 2023-10-21T17:19:33.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:19:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.884+0000 7f169aa09640 1 Processor -- start 2023-10-21T17:19:33.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.884+0000 7f169aa09640 1 -- start start 2023-10-21T17:19:33.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f169aa09640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f169415ab70 0x7f16940cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f169aa09640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940d04e0 0x7f16940cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f169aa09640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169415af50 con 0x7f16940cdca0 2023-10-21T17:19:33.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f169aa09640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16940ce150 con 0x7f169415ab70 2023-10-21T17:19:33.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f169aa09640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16940ced20 con 0x7f16940d04e0 2023-10-21T17:19:33.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f1698fa6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f16940cdca0 0x7f16940d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40790/0 (socket says 172.21.15.53:40790) 2023-10-21T17:19:33.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f1698fa6640 1 -- 172.21.15.53:0/3430617382 learned_addr learned my addr 172.21.15.53:0/3430617382 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:33.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f1693fff640 1 --2- 172.21.15.53:0/3430617382 >> v2:172.21.15.124:3300/0 conn(0x7f169415ab70 0x7f16940cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.886+0000 7f168b7fe640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940d04e0 0x7f16940cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f16937fe640 1 -- 172.21.15.53:0/3430617382 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111873296 0 0) 0x7f169415af50 con 0x7f16940cdca0 2023-10-21T17:19:33.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f16937fe640 1 -- 172.21.15.53:0/3430617382 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1680003520 con 0x7f16940cdca0 2023-10-21T17:19:33.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f1693fff640 1 -- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940d04e0 msgr2=0x7f16940cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:33.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f1693fff640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940d04e0 0x7f16940cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f1693fff640 1 -- 172.21.15.53:0/3430617382 >> v1:172.21.15.53:6789/0 conn(0x7f16940cdca0 legacy=0x7f16940d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.887+0000 7f1693fff640 1 -- 172.21.15.53:0/3430617382 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16940cf000 con 0x7f169415ab70 2023-10-21T17:19:33.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.888+0000 7f1693fff640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940cffa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1684009d90 tx=0x7f168402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a462fa1a2ab1e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.889+0000 7f16937fe640 1 -- 172.21.15.53:0/3430617382 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168402cc50 con 0x7f169415ab70 2023-10-21T17:19:33.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.889+0000 7f16937fe640 1 -- 172.21.15.53:0/3430617382 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168402cdb0 con 0x7f169415ab70 2023-10-21T17:19:33.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.889+0000 7f16937fe640 1 -- 172.21.15.53:0/3430617382 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684034660 con 0x7f169415ab70 2023-10-21T17:19:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 -- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 msgr2=0x7f16940cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940cffa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1684009d90 tx=0x7f168402bea0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 -- 172.21.15.53:0/3430617382 shutdown_connections 2023-10-21T17:19:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940d04e0 0x7f16940cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 --2- 172.21.15.53:0/3430617382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940cffa0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 -- 172.21.15.53:0/3430617382 >> 172.21.15.53:0/3430617382 conn(0x7f1694069f30 msgr2=0x7f169406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:33.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 -- 172.21.15.53:0/3430617382 shutdown_connections 2023-10-21T17:19:33.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.890+0000 7f169aa09640 1 -- 172.21.15.53:0/3430617382 wait complete. 2023-10-21T17:19:33.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.892+0000 7f169aa09640 1 Processor -- start 2023-10-21T17:19:33.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.893+0000 7f169aa09640 1 -- start start 2023-10-21T17:19:33.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940cdca0 0x7f16940c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694151610 con 0x7f16940d04e0 2023-10-21T17:19:33.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16940c4b00 con 0x7f169415ab70 2023-10-21T17:19:33.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f169aa09640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169415aff0 con 0x7f16940cdca0 2023-10-21T17:19:33.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f168b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f168b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34524/0 (socket says 172.21.15.53:34524) 2023-10-21T17:19:33.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.894+0000 7f168b7fe640 1 -- 172.21.15.53:0/497293001 learned_addr learned my addr 172.21.15.53:0/497293001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:33.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f1698fa6640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f1693fff640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940cdca0 0x7f16940c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 -- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940cdca0 msgr2=0x7f16940c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940cdca0 0x7f16940c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 -- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 msgr2=0x7f16940c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:33.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:33.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 -- 172.21.15.53:0/497293001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684009a70 con 0x7f16940d04e0 2023-10-21T17:19:33.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f1698fa6640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:33.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.895+0000 7f168b7fe640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9b0 tx=0x7f167c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.896+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c009670 con 0x7f16940d04e0 2023-10-21T17:19:33.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.896+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c00c9a0 con 0x7f16940d04e0 2023-10-21T17:19:33.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.896+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c00cd50 con 0x7f16940d04e0 2023-10-21T17:19:33.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.896+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169415b2d0 con 0x7f16940d04e0 2023-10-21T17:19:33.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.896+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169415b890 con 0x7f16940d04e0 2023-10-21T17:19:33.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.898+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16940cf2c0 con 0x7f16940d04e0 2023-10-21T17:19:33.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.899+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f167c013470 con 0x7f16940d04e0 2023-10-21T17:19:33.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.900+0000 7f16917fa640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 0x7f167406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:33.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.900+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f167c089c00 con 0x7f16940d04e0 2023-10-21T17:19:33.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.901+0000 7f1693fff640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 0x7f167406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:33.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.902+0000 7f1693fff640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 0x7f167406d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1684036000 tx=0x7f1684030000 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:33.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:33.905+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16940cf2c0 con 0x7f16940d04e0 2023-10-21T17:19:34.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.141+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f16940cf2c0 con 0x7f16940d04e0 2023-10-21T17:19:34.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.142+0000 7f16917fa640 1 -- 172.21.15.53:0/497293001 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16940cf2c0 con 0x7f16940d04e0 2023-10-21T17:19:34.144 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:19:34.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.144+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 msgr2=0x7f167406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:34.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.144+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 0x7f167406d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1684036000 tx=0x7f1684030000 comp rx=0 tx=0).stop 2023-10-21T17:19:34.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.144+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 msgr2=0x7f1694150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:34.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.144+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9b0 tx=0x7f167c00ee70 comp rx=0 tx=0).stop 2023-10-21T17:19:34.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 shutdown_connections 2023-10-21T17:19:34.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f169415ab70 0x7f16940c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:34.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f167406afc0 0x7f167406d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:34.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16940d04e0 0x7f1694150c50 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:34.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 --2- 172.21.15.53:0/497293001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f16940cdca0 0x7f16940c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:34.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 >> 172.21.15.53:0/497293001 conn(0x7f1694069f30 msgr2=0x7f169406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:34.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 shutdown_connections 2023-10-21T17:19:34.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:34.145+0000 7f169aa09640 1 -- 172.21.15.53:0/497293001 wait complete. 2023-10-21T17:19:34.166 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:19:34.167 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:19:34.167 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:19:34.167 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:19:34.168 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:19:34.168 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:19:34.168 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:19:34.168 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:19:34.169 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:19:34.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.170 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:19:34.224 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:19:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:19:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:19:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:19:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.225 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:19:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:19:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:19:34.226 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:19:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:19:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:19:34.226 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:19:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:19:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:19:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:19:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 37719sec preferred_lft 37719sec 2023-10-21T17:19:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:19:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.228 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:19:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:19:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:19:34.228 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:19:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:19:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:19:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:19:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.230 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:19:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:19:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:19:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.230 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:19:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:19:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:19:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:19:34.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.232 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:19:34.328 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:19:34.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.333 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:19:34.392 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:19:34.392 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:19:34.392 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:19:34.392 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:19:34.392 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:19:34.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.394 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:19:34.394 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:19:34.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:19:34.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:19:34.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:19:34.397 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:19:34.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.594 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:19:34.653 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:19:34.653 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:19:34.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:19:34.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:19:34.758 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:19:34.758 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:19:34.813 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:19:34.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:19:34.814 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:19:34.871 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:19:34.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:34.872 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:19:35.012 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:19:35.012 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:19:35.046 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:19:35.046 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:19:35.047 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:19:35.047 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:19:35.047 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:19:35.048 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:19:35.048 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:19:35.049 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:19:35.049 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:19:35.049 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:19:35.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:19:35.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:35.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:19:35.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:35.143 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:19:35.224 INFO:teuthology.orchestra.run.smithi053.stdout:client8473 2023-10-21T17:19:35.225 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:19:35.225 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:19:35.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:19:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.518+0000 7f41b8c40640 1 Processor -- start 2023-10-21T17:19:35.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.518+0000 7f41b8c40640 1 -- start start 2023-10-21T17:19:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b8c40640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41b41594e0 0x7f41b4070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b8c40640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b4071d90 con 0x7f41b40b0680 2023-10-21T17:19:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b8c40640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b4071180 con 0x7f41b4156a10 2023-10-21T17:19:35.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b8c40640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b4071460 con 0x7f41b41594e0 2023-10-21T17:19:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41b41594e0 0x7f41b4070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b2ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41b40b0680 0x7f41b40c8cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52460/0 (socket says 172.21.15.53:52460) 2023-10-21T17:19:35.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.520+0000 7f41b2ffd640 1 -- 172.21.15.53:0/1630541956 learned_addr learned my addr 172.21.15.53:0/1630541956 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.521+0000 7f41b3fff640 1 -- 172.21.15.53:0/1630541956 >> v1:172.21.15.124:6789/0 conn(0x7f41b4156a10 legacy=0x7f41b40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:35.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.521+0000 7f41b3fff640 1 -- 172.21.15.53:0/1630541956 >> v1:172.21.15.53:6789/0 conn(0x7f41b40b0680 legacy=0x7f41b40c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.521+0000 7f41b27fc640 1 -- 172.21.15.53:0/1630541956 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076137500 0 0) 0x7f41b4071d90 con 0x7f41b40b0680 2023-10-21T17:19:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.521+0000 7f41b3fff640 1 -- 172.21.15.53:0/1630541956 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4071740 con 0x7f41b41594e0 2023-10-21T17:19:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.522+0000 7f41b3fff640 1 --2- 172.21.15.53:0/1630541956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b41594e0 0x7f41b4070ac0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0097a0 tx=0x7f41ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c299fd618d2f7ac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:35.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.522+0000 7f41b27fc640 1 -- 172.21.15.53:0/1630541956 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac030610 con 0x7f41b41594e0 2023-10-21T17:19:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.522+0000 7f41b27fc640 1 -- 172.21.15.53:0/1630541956 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac030770 con 0x7f41b41594e0 2023-10-21T17:19:35.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.522+0000 7f41b27fc640 1 -- 172.21.15.53:0/1630541956 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac030b00 con 0x7f41b41594e0 2023-10-21T17:19:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 -- 172.21.15.53:0/1630541956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b41594e0 msgr2=0x7f41b4070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 --2- 172.21.15.53:0/1630541956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b41594e0 0x7f41b4070ac0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0097a0 tx=0x7f41ac02b960 comp rx=0 tx=0).stop 2023-10-21T17:19:35.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 -- 172.21.15.53:0/1630541956 shutdown_connections 2023-10-21T17:19:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 --2- 172.21.15.53:0/1630541956 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b41594e0 0x7f41b4070ac0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 -- 172.21.15.53:0/1630541956 >> 172.21.15.53:0/1630541956 conn(0x7f41b406c050 msgr2=0x7f41b406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 -- 172.21.15.53:0/1630541956 shutdown_connections 2023-10-21T17:19:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.523+0000 7f41b8c40640 1 -- 172.21.15.53:0/1630541956 wait complete. 2023-10-21T17:19:35.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.525+0000 7f41b8c40640 1 Processor -- start 2023-10-21T17:19:35.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.525+0000 7f41b8c40640 1 -- start start 2023-10-21T17:19:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 0x7f41b414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41b41594e0 0x7f41b4154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41583f0 con 0x7f41b4156a10 2023-10-21T17:19:35.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40af500 con 0x7f41b40b0680 2023-10-21T17:19:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b8c40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b40af7e0 con 0x7f41b41594e0 2023-10-21T17:19:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58600/0 (socket says 172.21.15.53:58600) 2023-10-21T17:19:35.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.526+0000 7f41b2ffd640 1 -- 172.21.15.53:0/3691349577 learned_addr learned my addr 172.21.15.53:0/3691349577 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b37fe640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 0x7f41b414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 -- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41b41594e0 msgr2=0x7f41b4154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41b41594e0 0x7f41b4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 -- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 msgr2=0x7f41b414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 0x7f41b414b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 -- 172.21.15.53:0/3691349577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac009400 con 0x7f41b4156a10 2023-10-21T17:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b37fe640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 0x7f41b414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b2ffd640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c3f0 tx=0x7f41a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:35.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80074e0 con 0x7f41b4156a10 2023-10-21T17:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8007640 con 0x7f41b4156a10 2023-10-21T17:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8004a00 con 0x7f41b4156a10 2023-10-21T17:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.527+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b40afac0 con 0x7f41b4156a10 2023-10-21T17:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.528+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b414b820 con 0x7f41b4156a10 2023-10-21T17:19:35.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.529+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b4074150 con 0x7f41b4156a10 2023-10-21T17:19:35.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.530+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f41a8004b60 con 0x7f41b4156a10 2023-10-21T17:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.531+0000 7f419bfff640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 0x7f41a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.531+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f41a8089870 con 0x7f41b4156a10 2023-10-21T17:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.531+0000 7f41b37fe640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 0x7f41a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:35.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.533+0000 7f41b37fe640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 0x7f41a006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a40012e0 tx=0x7f41a4009040 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:35.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.535+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41a80545b0 con 0x7f41b4156a10 2023-10-21T17:19:35.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.751+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 --> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f41b40cbd80 con 0x7f41a006af90 2023-10-21T17:19:35.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.756+0000 7f419bfff640 1 -- 172.21.15.53:0/3691349577 <== mgr.37184 v2:172.21.15.124:6832/1955466907 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f41a4006b50 con 0x7f41a006af90 2023-10-21T17:19:35.757 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:19:35.758 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:19:35.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 msgr2=0x7f41a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 0x7f41a006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a40012e0 tx=0x7f41a4009040 comp rx=0 tx=0).stop 2023-10-21T17:19:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 msgr2=0x7f41b4154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c3f0 tx=0x7f41a800c8b0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 shutdown_connections 2023-10-21T17:19:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41b41594e0 0x7f41b4154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f41a006af90 0x7f41a006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41b4156a10 0x7f41b4154410 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 --2- 172.21.15.53:0/3691349577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41b40b0680 0x7f41b414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:35.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 >> 172.21.15.53:0/3691349577 conn(0x7f41b406c050 msgr2=0x7f41b40ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:35.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 shutdown_connections 2023-10-21T17:19:35.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:35.758+0000 7f41b8c40640 1 -- 172.21.15.53:0/3691349577 wait complete. 2023-10-21T17:19:35.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:19:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.001+0000 7fe3e2bdc640 1 Processor -- start 2023-10-21T17:19:36.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.001+0000 7fe3e2bdc640 1 -- start start 2023-10-21T17:19:36.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.002+0000 7fe3e2bdc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe3dc0cdf50 0x7fe3dc0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.002+0000 7fe3e2bdc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc15d0b0 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.002+0000 7fe3e2bdc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc0b60b0 con 0x7fe3dc0cff70 2023-10-21T17:19:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.002+0000 7fe3e2bdc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc0d0400 con 0x7fe3dc0cdf50 2023-10-21T17:19:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3e0978640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe3dc15ccd0 0x7fe3dc0d1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52462/0 (socket says 172.21.15.53:52462) 2023-10-21T17:19:36.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3e0978640 1 -- 172.21.15.53:0/2599136082 learned_addr learned my addr 172.21.15.53:0/2599136082 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055156806 0 0) 0x7fe3dc15d0b0 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c4003520 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3dbfff640 1 --2- 172.21.15.53:0/2599136082 >> v2:172.21.15.124:3300/0 conn(0x7fe3dc0cdf50 0x7fe3dc0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1073180122 0 0) 0x7fe3c4003520 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 >> v2:172.21.15.124:3300/0 conn(0x7fe3dc0cdf50 msgr2=0x7fe3dc0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 --2- 172.21.15.53:0/2599136082 >> v2:172.21.15.124:3300/0 conn(0x7fe3dc0cdf50 0x7fe3dc0cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.003+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 >> v1:172.21.15.124:6789/0 conn(0x7fe3dc0cff70 legacy=0x7fe3dc0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0d2160 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe3cc001670 con 0x7fe3dc15ccd0 2023-10-21T17:19:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 >> v1:172.21.15.53:6789/0 conn(0x7fe3dc15ccd0 legacy=0x7fe3dc0d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 0x7fe3c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40047c0 0x7fe3c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3c40051d0 0x7fe3c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0d0400 con 0x7fe3c40039c0 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c400c120 con 0x7fe3c40047c0 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c400c2e0 con 0x7fe3c40051d0 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 0x7fe3c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3dbfff640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40047c0 0x7fe3c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e1179640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3c40051d0 0x7fe3c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e0978640 1 -- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3c40051d0 msgr2=0x7fe3c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3c40051d0 0x7fe3c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e0978640 1 -- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40047c0 msgr2=0x7fe3c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.004+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40047c0 0x7fe3c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.005+0000 7fe3e0978640 1 -- 172.21.15.53:0/2599136082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4003520 con 0x7fe3c40039c0 2023-10-21T17:19:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.005+0000 7fe3e1179640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3c40051d0 0x7fe3c400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.005+0000 7fe3e1179640 1 -- 172.21.15.53:0/2599136082 reap_dead start 2023-10-21T17:19:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.006+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 0x7fe3c4003de0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00c0e0 tx=0x7fe3c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9f465185076054 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:36.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.006+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc005ad0 con 0x7fe3c40039c0 2023-10-21T17:19:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.006+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc005c50 con 0x7fe3c40039c0 2023-10-21T17:19:36.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.006+0000 7fe3db7fe640 1 -- 172.21.15.53:0/2599136082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc02c7f0 con 0x7fe3c40039c0 2023-10-21T17:19:36.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 msgr2=0x7fe3c4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 0x7fe3c4003de0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00c0e0 tx=0x7fe3c40057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2599136082 shutdown_connections 2023-10-21T17:19:36.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2599136082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3c40039c0 0x7fe3c4003de0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2599136082 >> 172.21.15.53:0/2599136082 conn(0x7fe3dc0718d0 msgr2=0x7fe3dc071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:36.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2599136082 shutdown_connections 2023-10-21T17:19:36.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.007+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2599136082 wait complete. 2023-10-21T17:19:36.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.009+0000 7fe3e2bdc640 1 Processor -- start 2023-10-21T17:19:36.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.009+0000 7fe3e2bdc640 1 -- start start 2023-10-21T17:19:36.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 0x7fe3dc14be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3dc0cb2b0 0x7fe3dc0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc1600a0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0cc0a0 con 0x7fe3c40039c0 2023-10-21T17:19:36.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e2bdc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0cc380 con 0x7fe3dc0cb2b0 2023-10-21T17:19:36.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58616/0 (socket says 172.21.15.53:58616) 2023-10-21T17:19:36.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3dbfff640 1 -- 172.21.15.53:0/2865315438 learned_addr learned my addr 172.21.15.53:0/2865315438 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:36.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.010+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 0x7fe3dc14be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3e1179640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3dc0cb2b0 0x7fe3dc0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 -- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3dc0cb2b0 msgr2=0x7fe3dc0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3dc0cb2b0 0x7fe3dc0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 -- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 msgr2=0x7fe3dc14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 0x7fe3dc14be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 -- 172.21.15.53:0/2865315438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0cc660 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 0x7fe3dc14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:36.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3dbfff640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000f650 tx=0x7fe3d000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:36.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.011+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000d8e0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.012+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d000da40 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.012+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000ddb0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.012+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc0cc940 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.012+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc14c3c0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.013+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc04e160 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.014+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 96) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe3d0014460 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.015+0000 7fe3d97fa640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 0x7fe3c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:36.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.015+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe3d008a2e0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.016+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 0x7fe3c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:36.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.017+0000 7fe3e0978640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 0x7fe3c806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc034000 tx=0x7fe3cc009030 comp rx=0 tx=0).ready entity=mgr.37184 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:36.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.019+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3d00550b0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.262+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe3dc0720c0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.894+0000 7fe3d97fa640 1 -- 172.21.15.53:0/2865315438 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v97) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe3d00586d0 con 0x7fe3dc15f6e0 2023-10-21T17:19:36.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.900+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 msgr2=0x7fe3c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.900+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 0x7fe3c806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc034000 tx=0x7fe3cc009030 comp rx=0 tx=0).stop 2023-10-21T17:19:36.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.900+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 msgr2=0x7fe3dc15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:36.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.900+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000f650 tx=0x7fe3d000fb10 comp rx=0 tx=0).stop 2023-10-21T17:19:36.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 shutdown_connections 2023-10-21T17:19:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3dc0cb2b0 0x7fe3dc0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fe3c806af90 0x7fe3c806d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3dc15f6e0 0x7fe3dc15fb60 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 --2- 172.21.15.53:0/2865315438 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3c40039c0 0x7fe3dc14be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:36.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.902+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 >> 172.21.15.53:0/2865315438 conn(0x7fe3dc0718d0 msgr2=0x7fe3dc0b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.903+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 shutdown_connections 2023-10-21T17:19:36.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:36.903+0000 7fe3e2bdc640 1 -- 172.21.15.53:0/2865315438 wait complete. 2023-10-21T17:19:36.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:19:37.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.127+0000 7f7a5f11d640 1 Processor -- start 2023-10-21T17:19:37.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.127+0000 7f7a5f11d640 1 -- start start 2023-10-21T17:19:37.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.128+0000 7f7a5f11d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7a580b4e10 0x7f7a581612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.128+0000 7f7a5f11d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a580cd4e0 0x7f7a580d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.128+0000 7f7a5f11d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a580b8c30 con 0x7f7a5815cd70 2023-10-21T17:19:37.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.128+0000 7f7a5f11d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580cdd80 con 0x7f7a580cd4e0 2023-10-21T17:19:37.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.128+0000 7f7a5f11d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580ce060 con 0x7f7a580b4e10 2023-10-21T17:19:37.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a580cd4e0 0x7f7a580d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7a580cd4e0 0x7f7a580d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58646/0 (socket says 172.21.15.53:58646) 2023-10-21T17:19:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 -- 172.21.15.53:0/3028720340 learned_addr learned my addr 172.21.15.53:0/3028720340 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a57fff640 1 --2- 172.21.15.53:0/3028720340 >> v2:172.21.15.124:3300/0 conn(0x7f7a580b4e10 0x7f7a581612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 -- 172.21.15.53:0/3028720340 >> v2:172.21.15.124:3300/0 conn(0x7f7a580b4e10 msgr2=0x7f7a581612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 --2- 172.21.15.53:0/3028720340 >> v2:172.21.15.124:3300/0 conn(0x7f7a580b4e10 0x7f7a581612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 -- 172.21.15.53:0/3028720340 >> v1:172.21.15.124:6789/0 conn(0x7f7a5815cd70 legacy=0x7f7a580b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.129+0000 7f7a5d6ba640 1 -- 172.21.15.53:0/3028720340 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580ce340 con 0x7f7a580cd4e0 2023-10-21T17:19:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.130+0000 7f7a5d6ba640 1 --2- 172.21.15.53:0/3028720340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580d1650 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f7a580d07d0 tx=0x7f7a5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468c4bd2af5bf14f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.131+0000 7f7a577fe640 1 -- 172.21.15.53:0/3028720340 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5002ca80 con 0x7f7a580cd4e0 2023-10-21T17:19:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.131+0000 7f7a577fe640 1 -- 172.21.15.53:0/3028720340 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5002cbe0 con 0x7f7a580cd4e0 2023-10-21T17:19:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.131+0000 7f7a577fe640 1 -- 172.21.15.53:0/3028720340 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500343f0 con 0x7f7a580cd4e0 2023-10-21T17:19:37.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.131+0000 7f7a5f11d640 1 -- 172.21.15.53:0/3028720340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 msgr2=0x7f7a580d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.131+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/3028720340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580d1650 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f7a580d07d0 tx=0x7f7a5002bee0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 -- 172.21.15.53:0/3028720340 shutdown_connections 2023-10-21T17:19:37.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/3028720340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580d1650 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/3028720340 >> v2:172.21.15.124:3300/0 conn(0x7f7a580b4e10 0x7f7a581612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 -- 172.21.15.53:0/3028720340 >> 172.21.15.53:0/3028720340 conn(0x7f7a580cf700 msgr2=0x7f7a580cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:37.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 -- 172.21.15.53:0/3028720340 shutdown_connections 2023-10-21T17:19:37.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.132+0000 7f7a5f11d640 1 -- 172.21.15.53:0/3028720340 wait complete. 2023-10-21T17:19:37.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.133+0000 7f7a5f11d640 1 Processor -- start 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.134+0000 7f7a5f11d640 1 -- start start 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.134+0000 7f7a5f11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 0x7f7a5815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.134+0000 7f7a5f11d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5f11d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a5815cd70 0x7f7a5815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5f11d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580c9d50 con 0x7f7a580cd4e0 2023-10-21T17:19:37.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5f11d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58160390 con 0x7f7a5815cd70 2023-10-21T17:19:37.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5f11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580c9fa0 con 0x7f7a580b4e10 2023-10-21T17:19:37.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58656/0 (socket says 172.21.15.53:58656) 2023-10-21T17:19:37.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 -- 172.21.15.53:0/2838821552 learned_addr learned my addr 172.21.15.53:0/2838821552 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:37.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5d6ba640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a5815cd70 0x7f7a5815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5ceb9640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 0x7f7a5815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 msgr2=0x7f7a5815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 0x7f7a5815fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a5815cd70 msgr2=0x7f7a5815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a5815cd70 0x7f7a5815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a57fff640 1 -- 172.21.15.53:0/2838821552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009b30 con 0x7f7a580cd4e0 2023-10-21T17:19:37.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.135+0000 7f7a5ceb9640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 0x7f7a5815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:37.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a57fff640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d620 tx=0x7f7a4400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:37.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400b620 con 0x7f7a580cd4e0 2023-10-21T17:19:37.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400b780 con 0x7f7a580cd4e0 2023-10-21T17:19:37.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a580ca280 con 0x7f7a580cd4e0 2023-10-21T17:19:37.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400bb30 con 0x7f7a580cd4e0 2023-10-21T17:19:37.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.136+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a580ca840 con 0x7f7a580cd4e0 2023-10-21T17:19:37.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.137+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a580ca9d0 con 0x7f7a580cd4e0 2023-10-21T17:19:37.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.138+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 97) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7a4400bc90 con 0x7f7a580cd4e0 2023-10-21T17:19:37.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.139+0000 7f7a557fa640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 0x7f7a3c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.139+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7a44089f10 con 0x7f7a580cd4e0 2023-10-21T17:19:37.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.140+0000 7f7a5ceb9640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 msgr2=0x7f7a3c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:37.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.140+0000 7f7a5ceb9640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 0x7f7a3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:37.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.144+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a44054c50 con 0x7f7a580cd4e0 2023-10-21T17:19:37.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.336+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7a5804e160 con 0x7f7a580cd4e0 2023-10-21T17:19:37.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.341+0000 7f7a5ceb9640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 msgr2=0x7f7a3c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:37.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.341+0000 7f7a5ceb9640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 0x7f7a3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:37.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.341+0000 7f7a557fa640 1 -- 172.21.15.53:0/2838821552 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a44058270 con 0x7f7a580cd4e0 2023-10-21T17:19:37.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.343+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 msgr2=0x7f7a3c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:37.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.343+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 0x7f7a3c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.343+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 msgr2=0x7f7a580c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.343+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400d620 tx=0x7f7a4400dae0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 shutdown_connections 2023-10-21T17:19:37.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a5815cd70 0x7f7a5815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f7a3c06afe0 0x7f7a3c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a580cd4e0 0x7f7a580c71a0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 --2- 172.21.15.53:0/2838821552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a580b4e10 0x7f7a5815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 >> 172.21.15.53:0/2838821552 conn(0x7f7a580cf700 msgr2=0x7f7a580d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:37.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 shutdown_connections 2023-10-21T17:19:37.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.344+0000 7f7a5f11d640 1 -- 172.21.15.53:0/2838821552 wait complete. 2023-10-21T17:19:37.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:19:37.409 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:37.408+0000 7f1b48432040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:19:37.434 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:37.432+0000 7f5b5842b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:19:37.461 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:37.460+0000 7f1b48432040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:19:37.491 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:37.489+0000 7f5b5842b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:19:37.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.565+0000 7fc2df59e640 1 Processor -- start 2023-10-21T17:19:37.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.565+0000 7fc2df59e640 1 -- start start 2023-10-21T17:19:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 0x7fc2e00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2df59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc2e00b0680 0x7fc2e00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2df59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00c94e0 con 0x7fc2e00c8b40 2023-10-21T17:19:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2df59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00ccbc0 con 0x7fc2e00b0680 2023-10-21T17:19:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2df59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00ce1f0 con 0x7fc2e015cae0 2023-10-21T17:19:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2ddd9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2e00c8b40 0x7fc2e00aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52466/0 (socket says 172.21.15.53:52466) 2023-10-21T17:19:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.567+0000 7fc2ddd9b640 1 -- 172.21.15.53:0/1794470550 learned_addr learned my addr 172.21.15.53:0/1794470550 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2ded9d640 1 --2- 172.21.15.53:0/1794470550 >> v2:172.21.15.124:3300/0 conn(0x7fc2e00b0680 0x7fc2e00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2de59c640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 0x7fc2e00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819099842 0 0) 0x7fc2d4001110 con 0x7fc2e00c8b40 2023-10-21T17:19:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c8003520 con 0x7fc2e00c8b40 2023-10-21T17:19:37.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939207684 0 0) 0x7fc2c8003520 con 0x7fc2e00c8b40 2023-10-21T17:19:37.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 msgr2=0x7fc2e00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 0x7fc2e00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 >> v2:172.21.15.124:3300/0 conn(0x7fc2e00b0680 msgr2=0x7fc2e00c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 --2- 172.21.15.53:0/1794470550 >> v2:172.21.15.124:3300/0 conn(0x7fc2e00b0680 0x7fc2e00c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00cd590 con 0x7fc2e00c8b40 2023-10-21T17:19:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc2d4001800 con 0x7fc2e00c8b40 2023-10-21T17:19:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 >> v1:172.21.15.53:6789/0 conn(0x7fc2e00c8b40 legacy=0x7fc2e00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.568+0000 7fc2dd59a640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c8003970 0x7fc2c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2dd59a640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c80047c0 0x7fc2c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2dd59a640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 0x7fc2c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00ccbc0 con 0x7fc2c8003970 2023-10-21T17:19:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00ce1f0 con 0x7fc2c80047c0 2023-10-21T17:19:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c800b980 con 0x7fc2e015cae0 2023-10-21T17:19:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c8003970 0x7fc2c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2ddd9b640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c80047c0 0x7fc2c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 -- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 msgr2=0x7fc2c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e015cae0 0x7fc2c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 -- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c80047c0 msgr2=0x7fc2c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c80047c0 0x7fc2c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2de59c640 1 -- 172.21.15.53:0/1794470550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8003520 con 0x7fc2c8003970 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.569+0000 7fc2ddd9b640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c80047c0 0x7fc2c8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.570+0000 7fc2de59c640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c8003970 0x7fc2c8003d90 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009ac0 tx=0x7fc2c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888cc444812220e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.571+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d002c710 con 0x7fc2c8003970 2023-10-21T17:19:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.571+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d002c870 con 0x7fc2c8003970 2023-10-21T17:19:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.571+0000 7fc2dd59a640 1 -- 172.21.15.53:0/1794470550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d002cc00 con 0x7fc2c8003970 2023-10-21T17:19:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 -- 172.21.15.53:0/1794470550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c8003970 msgr2=0x7fc2c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 --2- 172.21.15.53:0/1794470550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2c8003970 0x7fc2c8003d90 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009ac0 tx=0x7fc2c80057f0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2ded9d640 1 -- 172.21.15.53:0/1794470550 reap_dead start 2023-10-21T17:19:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 -- 172.21.15.53:0/1794470550 shutdown_connections 2023-10-21T17:19:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 -- 172.21.15.53:0/1794470550 >> 172.21.15.53:0/1794470550 conn(0x7fc2e006c050 msgr2=0x7fc2e006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 -- 172.21.15.53:0/1794470550 shutdown_connections 2023-10-21T17:19:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.572+0000 7fc2df59e640 1 -- 172.21.15.53:0/1794470550 wait complete. 2023-10-21T17:19:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.574+0000 7fc2df59e640 1 Processor -- start 2023-10-21T17:19:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.574+0000 7fc2df59e640 1 -- start start 2023-10-21T17:19:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c8003970 0x7fc2e01f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 0x7fc2e0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015de20 con 0x7fc2e015cec0 2023-10-21T17:19:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01f31f0 con 0x7fc2c8003970 2023-10-21T17:19:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2df59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015d880 con 0x7fc2e00c48b0 2023-10-21T17:19:37.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2ded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2ded9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58680/0 (socket says 172.21.15.53:58680) 2023-10-21T17:19:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2ded9d640 1 -- 172.21.15.53:0/3368578396 learned_addr learned my addr 172.21.15.53:0/3368578396 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2de59c640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c8003970 0x7fc2e01f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.575+0000 7fc2ddd9b640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 0x7fc2e0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:37.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 msgr2=0x7fc2e0157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 0x7fc2e0157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c8003970 msgr2=0x7fc2e01f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c8003970 0x7fc2e01f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 -- 172.21.15.53:0/3368578396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e015e860 con 0x7fc2e015cec0 2023-10-21T17:19:37.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ddd9b640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 0x7fc2e0157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ded9d640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00724d0 tx=0x7fc2d800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8010040 con 0x7fc2e015cec0 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800ba80 con 0x7fc2e015cec0 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e015eb40 con 0x7fc2e015cec0 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.576+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800be10 con 0x7fc2e015cec0 2023-10-21T17:19:37.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.577+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e015f100 con 0x7fc2e015cec0 2023-10-21T17:19:37.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.578+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e0074150 con 0x7fc2e015cec0 2023-10-21T17:19:37.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.579+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 97) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc2d801e410 con 0x7fc2e015cec0 2023-10-21T17:19:37.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.579+0000 7fc2ceffd640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 0x7fc2c006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:37.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.580+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc2d8089c60 con 0x7fc2e015cec0 2023-10-21T17:19:37.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.580+0000 7fc2de59c640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 msgr2=0x7fc2c006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:37.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.580+0000 7fc2de59c640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 0x7fc2c006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:37.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.584+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2d8054890 con 0x7fc2e015cec0 2023-10-21T17:19:37.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.773+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc2e00b4470 con 0x7fc2e015cec0 2023-10-21T17:19:37.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.780+0000 7fc2ceffd640 1 -- 172.21.15.53:0/3368578396 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc2d8057eb0 con 0x7fc2e015cec0 2023-10-21T17:19:37.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.781+0000 7fc2de59c640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 msgr2=0x7fc2c006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:37.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.781+0000 7fc2de59c640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 0x7fc2c006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:37.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 msgr2=0x7fc2c006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:37.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 0x7fc2c006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 msgr2=0x7fc2e015d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:37.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00724d0 tx=0x7fc2d800b480 comp rx=0 tx=0).stop 2023-10-21T17:19:37.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 shutdown_connections 2023-10-21T17:19:37.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7fc2c006afe0 0x7fc2c006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2e015cec0 0x7fc2e015d340 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e00c48b0 0x7fc2e0157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 --2- 172.21.15.53:0/3368578396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2c8003970 0x7fc2e01f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:37.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 >> 172.21.15.53:0/3368578396 conn(0x7fc2e006c050 msgr2=0x7fc2e00ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:37.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 shutdown_connections 2023-10-21T17:19:37.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:37.783+0000 7fc2df59e640 1 -- 172.21.15.53:0/3368578396 wait complete. 2023-10-21T17:19:37.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:37.805 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T17:19:37.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:19:37.868 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-21T17:19:37.913 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:37.911+0000 7f1b48432040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:19:37.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2023-10-21T17:19:37.965 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:37.964+0000 7f1b48432040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:19:37.978 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:37.976+0000 7f5b5842b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:19:38.014 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.013+0000 7f1b48432040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:19:38.034 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.032+0000 7f5b5842b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:19:38.074 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.073+0000 7f1b48432040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:19:38.089 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.085+0000 7f5b5842b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:19:38.123 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.121+0000 7f1b48432040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:19:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.133+0000 7f5dcb59e640 1 Processor -- start 2023-10-21T17:19:38.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.133+0000 7f5dcb59e640 1 -- start start 2023-10-21T17:19:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcb59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5dcc0cf2b0 0x7f5dcc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5dcc0ce510 0x7f5dcc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc0cd520 con 0x7f5dcc0cf690 2023-10-21T17:19:38.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc0d1910 con 0x7f5dcc0ce510 2023-10-21T17:19:38.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcb59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc0ce080 con 0x7f5dcc0cf2b0 2023-10-21T17:19:38.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dc3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5dcc0cf690 0x7f5dcc0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52468/0 (socket says 172.21.15.53:52468) 2023-10-21T17:19:38.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dc3fff640 1 -- 172.21.15.53:0/2658478122 learned_addr learned my addr 172.21.15.53:0/2658478122 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:19:38.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.135+0000 7f5dcad9d640 1 --2- 172.21.15.53:0/2658478122 >> v2:172.21.15.53:3300/0 conn(0x7f5dcc0ce510 0x7f5dcc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:38.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dca59c640 1 --2- 172.21.15.53:0/2658478122 >> v2:172.21.15.124:3300/0 conn(0x7f5dcc0cf2b0 0x7f5dcc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dc9d9b640 1 -- 172.21.15.53:0/2658478122 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199957658 0 0) 0x7f5dcc0cd520 con 0x7f5dcc0cf690 2023-10-21T17:19:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dcad9d640 1 -- 172.21.15.53:0/2658478122 >> v2:172.21.15.124:3300/0 conn(0x7f5dcc0cf2b0 msgr2=0x7f5dcc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dcad9d640 1 --2- 172.21.15.53:0/2658478122 >> v2:172.21.15.124:3300/0 conn(0x7f5dcc0cf2b0 0x7f5dcc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dcad9d640 1 -- 172.21.15.53:0/2658478122 >> v1:172.21.15.53:6789/0 conn(0x7f5dcc0cf690 legacy=0x7f5dcc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:38.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.136+0000 7f5dcad9d640 1 -- 172.21.15.53:0/2658478122 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc0cd750 con 0x7f5dcc0ce510 2023-10-21T17:19:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.137+0000 7f5dcad9d640 1 --2- 172.21.15.53:0/2658478122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc15c820 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b54b0 tx=0x7f5dc402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1235f718610b4aea server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dc9d9b640 1 -- 172.21.15.53:0/2658478122 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc402cbf0 con 0x7f5dcc0ce510 2023-10-21T17:19:38.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dc9d9b640 1 -- 172.21.15.53:0/2658478122 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc402cd50 con 0x7f5dcc0ce510 2023-10-21T17:19:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dc9d9b640 1 -- 172.21.15.53:0/2658478122 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4034660 con 0x7f5dcc0ce510 2023-10-21T17:19:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 -- 172.21.15.53:0/2658478122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 msgr2=0x7f5dcc15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 --2- 172.21.15.53:0/2658478122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc15c820 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0b54b0 tx=0x7f5dc402be40 comp rx=0 tx=0).stop 2023-10-21T17:19:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 -- 172.21.15.53:0/2658478122 shutdown_connections 2023-10-21T17:19:38.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 --2- 172.21.15.53:0/2658478122 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc15c820 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 --2- 172.21.15.53:0/2658478122 >> v2:172.21.15.124:3300/0 conn(0x7f5dcc0cf2b0 0x7f5dcc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 -- 172.21.15.53:0/2658478122 >> 172.21.15.53:0/2658478122 conn(0x7f5dcc09bc40 msgr2=0x7f5dcc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.138+0000 7f5dcb59e640 1 -- 172.21.15.53:0/2658478122 shutdown_connections 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.139+0000 7f5dcb59e640 1 -- 172.21.15.53:0/2658478122 wait complete. 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.140+0000 7f5dcb59e640 1 Processor -- start 2023-10-21T17:19:38.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.140+0000 7f5dcb59e640 1 -- start start 2023-10-21T17:19:38.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc1e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dcc0cf2b0 0x7f5dcc0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dcc0cf690 0x7f5dcc158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc1e6f00 con 0x7f5dcc0ce510 2023-10-21T17:19:38.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc160150 con 0x7f5dcc0cf2b0 2023-10-21T17:19:38.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.141+0000 7f5dcb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc1e7840 con 0x7f5dcc0cf690 2023-10-21T17:19:38.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc1e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:38.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc1e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58702/0 (socket says 172.21.15.53:58702) 2023-10-21T17:19:38.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 -- 172.21.15.53:0/738266964 learned_addr learned my addr 172.21.15.53:0/738266964 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:38.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dcc0cf690 msgr2=0x7f5dcc158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:38.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5dcc0cf690 0x7f5dcc158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:38.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dcc0cf2b0 msgr2=0x7f5dcc0cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:38.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5dcc0cf2b0 0x7f5dcc0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:38.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009a70 con 0x7f5dcc0ce510 2023-10-21T17:19:38.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.142+0000 7f5dca59c640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc1e7300 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00e9b0 tx=0x7f5dbc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:38.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.143+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc009670 con 0x7f5dcc0ce510 2023-10-21T17:19:38.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.143+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00c9a0 con 0x7f5dcc0ce510 2023-10-21T17:19:38.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.143+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00cd50 con 0x7f5dcc0ce510 2023-10-21T17:19:38.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.143+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc1e62e0 con 0x7f5dcc0ce510 2023-10-21T17:19:38.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.143+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc1e6b20 con 0x7f5dcc0ce510 2023-10-21T17:19:38.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.144+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc1e7170 con 0x7f5dcc0ce510 2023-10-21T17:19:38.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.145+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 97) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f5dbc013470 con 0x7f5dcc0ce510 2023-10-21T17:19:38.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.146+0000 7f5dc2ffd640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:38.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.146+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5dbc089c10 con 0x7f5dcc0ce510 2023-10-21T17:19:38.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.146+0000 7f5dc3fff640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 msgr2=0x7f5dac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:38.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.146+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:19:38.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.150+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dbc0549e0 con 0x7f5dcc0ce510 2023-10-21T17:19:38.156 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.152+0000 7f5b5842b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:19:38.206 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.204+0000 7f5b5842b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:19:38.233 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.232+0000 7f1b48432040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:19:38.295 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.293+0000 7f1b48432040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:38.322 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.321+0000 7f5b5842b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:19:38.343 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.342+0000 7f1b48432040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:19:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.347+0000 7f5dc3fff640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 msgr2=0x7f5dac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:38.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.347+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:19:38.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.366+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dcc0cda30 con 0x7f5dac06afe0 2023-10-21T17:19:38.390 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.388+0000 7f5b5842b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:38.401 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.400+0000 7f1b48432040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:19:38.441 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.439+0000 7f5b5842b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:19:38.451 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.449+0000 7f1b48432040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:19:38.501 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.499+0000 7f5b5842b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:19:38.553 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.551+0000 7f5b5842b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:19:38.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.748+0000 7f5dc3fff640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 msgr2=0x7f5dac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:38.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:38.748+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:19:38.829 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.828+0000 7f1b48432040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:19:38.952 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:38.950+0000 7f5b5842b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:19:38.987 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:38.986+0000 7f1b48432040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:19:39.098 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.097+0000 7f1b48432040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:19:39.120 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.118+0000 7f5b5842b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:19:39.148 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.147+0000 7f1b48432040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:19:39.233 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.231+0000 7f5b5842b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:19:39.254 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.253+0000 7f1b48432040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:39.285 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.283+0000 7f5b5842b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:19:39.361 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.359+0000 7f1b48432040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:19:39.397 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.395+0000 7f5b5842b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:39.509 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.507+0000 7f5b5842b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:19:39.516 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.514+0000 7f1b48432040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:39.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:39.549+0000 7f5dc3fff640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 msgr2=0x7f5dac06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1955466907 2023-10-21T17:19:39.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:39.549+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:19:39.564 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.563+0000 7f1b48432040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:39.673 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.671+0000 7f5b5842b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:19:39.683 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:39.681+0000 7f1b48432040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:19:39.724 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.723+0000 7f5b5842b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:19:39.848 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:39.847+0000 7f5b5842b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:19:40.504 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:40.502+0000 7f1b48432040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:19:40.725 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:40.724+0000 7f5b5842b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:19:40.796 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:40.795+0000 7f1b48432040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:19:40.854 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:40.853+0000 7f1b48432040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:40.967 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:40.966+0000 7f1b48432040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:40.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:40.985+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 98) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f5dbc054db0 con 0x7f5dcc0ce510 2023-10-21T17:19:40.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:40.986+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 msgr2=0x7f5dac06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:19:40.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:40.986+0000 7f5dc2ffd640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/1955466907,v1:172.21.15.124:6833/1955466907] conn(0x7f5dac06afe0 0x7f5dac06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:41.050 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:41.049+0000 7f5b5842b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:19:41.119 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:41.118+0000 7f5b5842b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:19:41.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac481e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac481e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac481e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.188 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac481e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.188 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac481e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.188 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac7864640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.189 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac7864640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.189 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac7864640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.189 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac7864640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.189 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:19:41.186+0000 7f1ac7864640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:19:41.245 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:19:41.243+0000 7f5b5842b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:19:42.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:41.999+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 99) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f5dbc0564c0 con 0x7f5dcc0ce510 2023-10-21T17:19:42.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.000+0000 7f5dc2ffd640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f5dac0355a0 0x7f5dac035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.000+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 --> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dac035ec0 con 0x7f5dac0355a0 2023-10-21T17:19:42.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.004+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f5dac0355a0 0x7f5dac035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.017+0000 7f5dc3fff640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f5dac0355a0 0x7f5dac035980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc15ef70 tx=0x7f5db4007a20 comp rx=0 tx=0).ready entity=mgr.37400 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:42.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.189+0000 7f5dc2ffd640 1 -- 172.21.15.53:0/738266964 <== mgr.37400 v2:172.21.15.124:6832/3632125819 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5dac035ec0 con 0x7f5dac0355a0 2023-10-21T17:19:42.191 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-21T17:19:42.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.192+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f5dac0355a0 msgr2=0x7f5dac035980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.192+0000 7f5dcb59e640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f5dac0355a0 0x7f5dac035980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc15ef70 tx=0x7f5db4007a20 comp rx=0 tx=0).stop 2023-10-21T17:19:42.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 msgr2=0x7f5dcc1e7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 --2- 172.21.15.53:0/738266964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dcc0ce510 0x7f5dcc1e7300 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00e9b0 tx=0x7f5dbc00ee70 comp rx=0 tx=0).stop 2023-10-21T17:19:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcad9d640 1 -- 172.21.15.53:0/738266964 reap_dead start 2023-10-21T17:19:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 shutdown_connections 2023-10-21T17:19:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 >> 172.21.15.53:0/738266964 conn(0x7f5dcc09bc40 msgr2=0x7f5dcc0d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 shutdown_connections 2023-10-21T17:19:42.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.193+0000 7f5dcb59e640 1 -- 172.21.15.53:0/738266964 wait complete. 2023-10-21T17:19:42.215 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-21T17:19:42.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2023-10-21T17:19:42.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.444+0000 7effcb59e640 1 Processor -- start 2023-10-21T17:19:42.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.445+0000 7effcb59e640 1 -- start start 2023-10-21T17:19:42.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effcc15a820 0x7effcc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effcc15ac00 0x7effcc0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0cab60 con 0x7effcc15a820 2023-10-21T17:19:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0cb960 con 0x7effcc15ac00 2023-10-21T17:19:42.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effcb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0cbdf0 con 0x7effcc0ca680 2023-10-21T17:19:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.446+0000 7effca59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effcc15a820 0x7effcc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effcc15a820 0x7effcc0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58710/0 (socket says 172.21.15.53:58710) 2023-10-21T17:19:42.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 -- 172.21.15.53:0/518524651 learned_addr learned my addr 172.21.15.53:0/518524651 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:42.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 -- 172.21.15.53:0/518524651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 msgr2=0x7effcc0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 --2- 172.21.15.53:0/518524651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 -- 172.21.15.53:0/518524651 >> v2:172.21.15.124:3300/0 conn(0x7effcc15ac00 msgr2=0x7effcc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:19:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 --2- 172.21.15.53:0/518524651 >> v2:172.21.15.124:3300/0 conn(0x7effcc15ac00 0x7effcc0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.447+0000 7effca59c640 1 -- 172.21.15.53:0/518524651 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc0cb030 con 0x7effcc15a820 2023-10-21T17:19:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.448+0000 7effca59c640 1 --2- 172.21.15.53:0/518524651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15a820 0x7effcc0aff70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7effcc0ce340 tx=0x7effbc02be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baaa7d513388b50 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.448+0000 7effc959a640 1 -- 172.21.15.53:0/518524651 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc02ca10 con 0x7effcc15a820 2023-10-21T17:19:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.448+0000 7effc959a640 1 -- 172.21.15.53:0/518524651 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc02cb70 con 0x7effcc15a820 2023-10-21T17:19:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.448+0000 7effc959a640 1 -- 172.21.15.53:0/518524651 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0343f0 con 0x7effcc15a820 2023-10-21T17:19:42.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 -- 172.21.15.53:0/518524651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15a820 msgr2=0x7effcc0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 --2- 172.21.15.53:0/518524651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15a820 0x7effcc0aff70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7effcc0ce340 tx=0x7effbc02be70 comp rx=0 tx=0).stop 2023-10-21T17:19:42.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 -- 172.21.15.53:0/518524651 shutdown_connections 2023-10-21T17:19:42.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 --2- 172.21.15.53:0/518524651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 --2- 172.21.15.53:0/518524651 >> v2:172.21.15.124:3300/0 conn(0x7effcc15ac00 0x7effcc0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 --2- 172.21.15.53:0/518524651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15a820 0x7effcc0aff70 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 -- 172.21.15.53:0/518524651 >> 172.21.15.53:0/518524651 conn(0x7effcc06c050 msgr2=0x7effcc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:42.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 -- 172.21.15.53:0/518524651 shutdown_connections 2023-10-21T17:19:42.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.449+0000 7effcb59e640 1 -- 172.21.15.53:0/518524651 wait complete. 2023-10-21T17:19:42.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.451+0000 7effcb59e640 1 Processor -- start 2023-10-21T17:19:42.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.451+0000 7effcb59e640 1 -- start start 2023-10-21T17:19:42.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc15a820 0x7effcc1444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0c3080 con 0x7effcc15ac00 2023-10-21T17:19:42.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc147ca0 con 0x7effcc15a820 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc1449e0 con 0x7effcc0ca680 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58714/0 (socket says 172.21.15.53:58714) 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.452+0000 7effcad9d640 1 -- 172.21.15.53:0/3817892456 learned_addr learned my addr 172.21.15.53:0/3817892456 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effc9d9b640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc15a820 0x7effcc1444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effca59c640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 -- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 msgr2=0x7effcc147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 -- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc15a820 msgr2=0x7effcc1444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc15a820 0x7effcc1444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 -- 172.21.15.53:0/3817892456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc009b30 con 0x7effcc15ac00 2023-10-21T17:19:42.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effcad9d640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7effc400b3c0 tx=0x7effc400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:42.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.453+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4011830 con 0x7effcc15ac00 2023-10-21T17:19:42.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.454+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4011990 con 0x7effcc15ac00 2023-10-21T17:19:42.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.454+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4011d40 con 0x7effcc15ac00 2023-10-21T17:19:42.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.454+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc0c34c0 con 0x7effcc15ac00 2023-10-21T17:19:42.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.454+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc0c2750 con 0x7effcc15ac00 2023-10-21T17:19:42.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.455+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc0c6fe0 con 0x7effcc15ac00 2023-10-21T17:19:42.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.456+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 99) v1 ==== 89084+0+0 (secure 0 0 0) 0x7effc401b460 con 0x7effcc15ac00 2023-10-21T17:19:42.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.457+0000 7effc2ffd640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 0x7effac06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:19:42.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.457+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6489+0+0 (secure 0 0 0) 0x7effc40893b0 con 0x7effcc15ac00 2023-10-21T17:19:42.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.457+0000 7effca59c640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 0x7effac06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:19:42.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.459+0000 7effca59c640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 0x7effac06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7effbc035000 tx=0x7effbc030000 comp rx=0 tx=0).ready entity=mgr.37400 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:19:42.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.461+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effcc0c6fe0 con 0x7effcc15ac00 2023-10-21T17:19:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.686+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 --> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7effcc0cb5f0 con 0x7effac06ad10 2023-10-21T17:19:42.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:19:42.801 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:19:42.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.806+0000 7effc2ffd640 1 -- 172.21.15.53:0/3817892456 <== mgr.37400 v2:172.21.15.124:6832/3632125819 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7effbc0440d0 con 0x7effac06ad10 2023-10-21T17:19:42.807 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-21T17:19:42.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.809+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 msgr2=0x7effac06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.809+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 0x7effac06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7effbc035000 tx=0x7effbc030000 comp rx=0 tx=0).stop 2023-10-21T17:19:42.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 msgr2=0x7effcc0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:19:42.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7effc400b3c0 tx=0x7effc400b880 comp rx=0 tx=0).stop 2023-10-21T17:19:42.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 shutdown_connections 2023-10-21T17:19:42.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7effac06ad10 0x7effac06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effcc15ac00 0x7effcc0b2ae0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effcc15a820 0x7effcc1444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 --2- 172.21.15.53:0/3817892456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effcc0ca680 0x7effcc147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:19:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 >> 172.21.15.53:0/3817892456 conn(0x7effcc06c050 msgr2=0x7effcc0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:19:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 shutdown_connections 2023-10-21T17:19:42.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:19:42.810+0000 7effcb59e640 1 -- 172.21.15.53:0/3817892456 wait complete. 2023-10-21T17:19:42.833 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-21T17:20:00.835 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:20:00.833+0000 7f1ab6802640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:20:00.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:20:00.833+0000 7f1ab6802640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:20:00.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:20:00.833+0000 7f1ab6802640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:20:00.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:20:00.833+0000 7f1ab6802640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:20:00.837 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:20:00.833+0000 7f1ab6802640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:20:12.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:20:13.008 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:20:43.141 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:20:43.151 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:21:13.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:21:13.328 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:21:43.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:21:43.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:22:13.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:22:13.799 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:22:43.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:22:43.997 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:23:14.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:23:14.178 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:23:44.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:23:44.360 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:24:14.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:24:14.574 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:24:44.770 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:24:44.780 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:25:15.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:25:15.012 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:25:45.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:25:45.158 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:26:15.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:26:15.303 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:26:45.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:26:45.450 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:27:15.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:27:15.627 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:27:45.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:27:45.808 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:28:16.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:28:16.025 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:28:46.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:28:46.178 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:29:16.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:29:16.351 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:29:46.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:29:46.574 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:30:15.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:30:15.835 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-21T17:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_20_00_UTC 2023-10-21T17:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_21_00_UTC 2023-10-21T17:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_22_00_UTC 2023-10-21T17:30:15.902 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_23_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_24_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_25_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_26_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_27_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_28_00_UTC 2023-10-21T17:30:15.903 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_29_00_UTC 2023-10-21T17:30:15.904 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_30_00_UTC 2023-10-21T17:30:15.905 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-17_20_00_UTC', 'scheduled-2023-10-21-17_21_00_UTC', 'scheduled-2023-10-21-17_22_00_UTC', 'scheduled-2023-10-21-17_23_00_UTC', 'scheduled-2023-10-21-17_24_00_UTC', 'scheduled-2023-10-21-17_25_00_UTC', 'scheduled-2023-10-21-17_26_00_UTC', 'scheduled-2023-10-21-17_27_00_UTC', 'scheduled-2023-10-21-17_28_00_UTC', 'scheduled-2023-10-21-17_29_00_UTC', 'scheduled-2023-10-21-17_30_00_UTC'] 2023-10-21T17:30:15.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-21T17:30:16.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.135+0000 7fa14e9b3640 1 Processor -- start 2023-10-21T17:30:16.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.135+0000 7fa14e9b3640 1 -- start start 2023-10-21T17:30:16.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14e9b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1481599d0 0x7fa1480cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14e9b3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa1480caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14e9b3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1480cb8a0 con 0x7fa1480cb410 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14e9b3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa148159db0 con 0x7fa1481599d0 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14e9b3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa148071fe0 con 0x7fa1480b3c50 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.137+0000 7fa14d9b1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1481599d0 0x7fa1480cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1481599d0 0x7fa1480cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43642/0 (socket says 172.21.15.53:43642) 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 -- 172.21.15.53:0/1087367844 learned_addr learned my addr 172.21.15.53:0/1087367844 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:30:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 -- 172.21.15.53:0/1087367844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 msgr2=0x7fa1480caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:30:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 --2- 172.21.15.53:0/1087367844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa1480caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 -- 172.21.15.53:0/1087367844 >> v1:172.21.15.124:6789/0 conn(0x7fa1480cb410 legacy=0x7fa1480c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:30:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.138+0000 7fa14d9b1640 1 -- 172.21.15.53:0/1087367844 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480b44c0 con 0x7fa1481599d0 2023-10-21T17:30:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.139+0000 7fa14d9b1640 1 --2- 172.21.15.53:0/1087367844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1481599d0 0x7fa1480cdcd0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa14400bb10 tx=0x7fa14402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1355024416f3f23 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14c9af640 1 -- 172.21.15.53:0/1087367844 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14402ee70 con 0x7fa1481599d0 2023-10-21T17:30:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14c9af640 1 -- 172.21.15.53:0/1087367844 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144036440 con 0x7fa1481599d0 2023-10-21T17:30:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14c9af640 1 -- 172.21.15.53:0/1087367844 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1440367d0 con 0x7fa1481599d0 2023-10-21T17:30:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 -- 172.21.15.53:0/1087367844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1481599d0 msgr2=0x7fa1480cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/1087367844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1481599d0 0x7fa1480cdcd0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa14400bb10 tx=0x7fa14402e0c0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 -- 172.21.15.53:0/1087367844 shutdown_connections 2023-10-21T17:30:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/1087367844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa1480caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/1087367844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1481599d0 0x7fa1480cdcd0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 -- 172.21.15.53:0/1087367844 >> 172.21.15.53:0/1087367844 conn(0x7fa14806c050 msgr2=0x7fa14806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:30:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.140+0000 7fa14e9b3640 1 -- 172.21.15.53:0/1087367844 shutdown_connections 2023-10-21T17:30:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.141+0000 7fa14e9b3640 1 -- 172.21.15.53:0/1087367844 wait complete. 2023-10-21T17:30:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.142+0000 7fa14e9b3640 1 Processor -- start 2023-10-21T17:30:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.142+0000 7fa14e9b3640 1 -- start start 2023-10-21T17:30:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa148154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1481599d0 0x7fa1480a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480a7b60 con 0x7fa1480cb410 2023-10-21T17:30:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148154ea0 con 0x7fa1481599d0 2023-10-21T17:30:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.143+0000 7fa14e9b3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480a7e00 con 0x7fa1480b3c50 2023-10-21T17:30:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43658/0 (socket says 172.21.15.53:43658) 2023-10-21T17:30:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 -- 172.21.15.53:0/3337373007 learned_addr learned my addr 172.21.15.53:0/3337373007 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:30:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d9b1640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa148154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 -- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 msgr2=0x7fa148154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa148154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14e1b2640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1481599d0 0x7fa1480a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 -- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1481599d0 msgr2=0x7fa1480a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1481599d0 0x7fa1480a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 -- 172.21.15.53:0/3337373007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14400b770 con 0x7fa1480cb410 2023-10-21T17:30:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.144+0000 7fa14d1b0640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b4a0 tx=0x7fa13800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.145+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138010610 con 0x7fa1480cb410 2023-10-21T17:30:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.145+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa138010770 con 0x7fa1480cb410 2023-10-21T17:30:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.145+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138010b20 con 0x7fa1480cb410 2023-10-21T17:30:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.145+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1480a80e0 con 0x7fa1480cb410 2023-10-21T17:30:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.145+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1480a86a0 con 0x7fa1480cb410 2023-10-21T17:30:16.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.146+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa148074150 con 0x7fa1480cb410 2023-10-21T17:30:16.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.147+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 103) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fa138010c80 con 0x7fa1480cb410 2023-10-21T17:30:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.148+0000 7fa13e7fc640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 0x7fa12c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.148+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fa1380897c0 con 0x7fa1480cb410 2023-10-21T17:30:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.149+0000 7fa14d9b1640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 0x7fa12c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.150+0000 7fa14d9b1640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 0x7fa12c06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa144038000 tx=0x7fa144008040 comp rx=0 tx=0).ready entity=mgr.37400 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.153+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1380543b0 con 0x7fa1480cb410 2023-10-21T17:30:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.369+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 --> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1480716b0 con 0x7fa12c06afe0 2023-10-21T17:30:16.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.388+0000 7fa13e7fc640 1 -- 172.21.15.53:0/3337373007 <== mgr.37400 v2:172.21.15.124:6832/3632125819 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fa144036960 con 0x7fa12c06afe0 2023-10-21T17:30:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:30:16.390 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:19:42", "first": "2023-10-21T17:20:00", "last": "2023-10-21T17:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-21T17:30:16.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.390+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 msgr2=0x7fa12c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.390+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 0x7fa12c06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa144038000 tx=0x7fa144008040 comp rx=0 tx=0).stop 2023-10-21T17:30:16.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 msgr2=0x7fa1480a8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b4a0 tx=0x7fa13800b960 comp rx=0 tx=0).stop 2023-10-21T17:30:16.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 shutdown_connections 2023-10-21T17:30:16.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7fa12c06afe0 0x7fa12c06d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1481599d0 0x7fa1480a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1480cb410 0x7fa1480a8bc0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 --2- 172.21.15.53:0/3337373007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1480b3c50 0x7fa148154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 >> 172.21.15.53:0/3337373007 conn(0x7fa14806c050 msgr2=0x7fa1480affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:30:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 shutdown_connections 2023-10-21T17:30:16.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.391+0000 7fa14e9b3640 1 -- 172.21.15.53:0/3337373007 wait complete. 2023-10-21T17:30:16.412 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:19:42", "first": "2023-10-21T17:20:00", "last": "2023-10-21T17:30:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-21T17:30:16.413 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T17:19:42', 'first': '2023-10-21T17:20:00', 'last': '2023-10-21T17:30:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-21T17:30:16.413 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-21T17:30:16.413 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-21T17:30:16.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-21T17:30:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.639+0000 7f6a151a7640 1 Processor -- start 2023-10-21T17:30:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.639+0000 7f6a151a7640 1 -- start start 2023-10-21T17:30:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.640+0000 7f6a151a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a1015ab30 0x7f6a1015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a151a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a151a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1015f7d0 con 0x7f6a1015af10 2023-10-21T17:30:16.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a151a7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1015f210 con 0x7f6a1015ab30 2023-10-21T17:30:16.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a151a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1015f4f0 con 0x7f6a100adc70 2023-10-21T17:30:16.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a0ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6a1015ab30 0x7f6a1015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a149a6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6a1015af10 0x7f6a100701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:51078/0 (socket says 172.21.15.53:51078) 2023-10-21T17:30:16.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a149a6640 1 -- 172.21.15.53:0/3440434266 learned_addr learned my addr 172.21.15.53:0/3440434266 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:30:16.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.641+0000 7f6a07fff640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.642+0000 7f6a0ffff640 1 -- 172.21.15.53:0/3440434266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 msgr2=0x7f6a100b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:30:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.642+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.642+0000 7f6a0ffff640 1 -- 172.21.15.53:0/3440434266 >> v1:172.21.15.53:6789/0 conn(0x7f6a1015af10 legacy=0x7f6a100701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.642+0000 7f6a0ffff640 1 -- 172.21.15.53:0/3440434266 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1006c550 con 0x7f6a1015ab30 2023-10-21T17:30:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.642+0000 7f6a0f7fe640 1 -- 172.21.15.53:0/3440434266 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126801324 0 0) 0x7f6a1015f7d0 con 0x7f6a1015af10 2023-10-21T17:30:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015ab30 0x7f6a1015e8a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7686e12f48653d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a0f7fe640 1 -- 172.21.15.53:0/3440434266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00031830 con 0x7f6a1015ab30 2023-10-21T17:30:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a0f7fe640 1 -- 172.21.15.53:0/3440434266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00031990 con 0x7f6a1015ab30 2023-10-21T17:30:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a0f7fe640 1 -- 172.21.15.53:0/3440434266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00031d20 con 0x7f6a1015ab30 2023-10-21T17:30:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a151a7640 1 -- 172.21.15.53:0/3440434266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015ab30 msgr2=0x7f6a1015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.643+0000 7f6a151a7640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015ab30 0x7f6a1015e8a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a0002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 -- 172.21.15.53:0/3440434266 shutdown_connections 2023-10-21T17:30:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 --2- 172.21.15.53:0/3440434266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015ab30 0x7f6a1015e8a0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 -- 172.21.15.53:0/3440434266 >> 172.21.15.53:0/3440434266 conn(0x7f6a100cb0c0 msgr2=0x7f6a100cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:30:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 -- 172.21.15.53:0/3440434266 shutdown_connections 2023-10-21T17:30:16.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.644+0000 7f6a151a7640 1 -- 172.21.15.53:0/3440434266 wait complete. 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.645+0000 7f6a151a7640 1 Processor -- start 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.646+0000 7f6a151a7640 1 -- start start 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.646+0000 7f6a151a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a151a7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 0x7f6a100cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a151a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a151a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100ce2a0 con 0x7f6a1015af10 2023-10-21T17:30:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a151a7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100cc880 con 0x7f6a1015ab30 2023-10-21T17:30:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a151a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100ccb60 con 0x7f6a100adc70 2023-10-21T17:30:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:43662/0 (socket says 172.21.15.53:43662) 2023-10-21T17:30:16.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 -- 172.21.15.53:0/1589361478 learned_addr learned my addr 172.21.15.53:0/1589361478 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:30:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a07fff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 0x7f6a100cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 -- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 msgr2=0x7f6a100cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 -- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 msgr2=0x7f6a100cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 0x7f6a100cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a149a6640 1 -- 172.21.15.53:0/1589361478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a00009400 con 0x7f6a1015af10 2023-10-21T17:30:16.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a07fff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 0x7f6a100cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:30:16.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.647+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:30:16.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a149a6640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6a08002230 tx=0x7f6a0800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800ac30 con 0x7f6a1015af10 2023-10-21T17:30:16.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0800ad90 con 0x7f6a1015af10 2023-10-21T17:30:16.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08012620 con 0x7f6a1015af10 2023-10-21T17:30:16.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a100cce40 con 0x7f6a1015af10 2023-10-21T17:30:16.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.648+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1014f0b0 con 0x7f6a1015af10 2023-10-21T17:30:16.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.649+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1004e160 con 0x7f6a1015af10 2023-10-21T17:30:16.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.650+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 103) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f6a08007070 con 0x7f6a1015af10 2023-10-21T17:30:16.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.651+0000 7f6a0d7fa640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 0x7f69f006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:30:16.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.651+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f6a08089a60 con 0x7f6a1015af10 2023-10-21T17:30:16.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.652+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 0x7f69f006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:30:16.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.653+0000 7f6a0ffff640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 0x7f69f006d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6a00036000 tx=0x7f6a00008030 comp rx=0 tx=0).ready entity=mgr.37400 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:30:16.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.656+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a080546e0 con 0x7f6a1015af10 2023-10-21T17:30:16.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:30:16.764 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:30:16.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.901+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a100ada50 con 0x7f6a1015af10 2023-10-21T17:30:16.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.909+0000 7f6a0d7fa640 1 -- 172.21.15.53:0/1589361478 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v103) v1 ==== 74+0+171248 (secure 0 0 0) 0x7f6a08017090 con 0x7f6a1015af10 2023-10-21T17:30:16.913 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:30:16.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.913+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 msgr2=0x7f69f006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.913+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 0x7f69f006d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6a00036000 tx=0x7f6a00008030 comp rx=0 tx=0).stop 2023-10-21T17:30:16.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.913+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 msgr2=0x7f6a1014eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:30:16.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.913+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6a08002230 tx=0x7f6a0800cf50 comp rx=0 tx=0).stop 2023-10-21T17:30:16.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 shutdown_connections 2023-10-21T17:30:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:6832/3632125819,v1:172.21.15.124:6833/3632125819] conn(0x7f69f006afe0 0x7f69f006d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6a1015af10 0x7f6a1014eb70 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1015ab30 0x7f6a100cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 --2- 172.21.15.53:0/1589361478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100adc70 0x7f6a100cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:30:16.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 >> 172.21.15.53:0/1589361478 conn(0x7f6a100cb0c0 msgr2=0x7f6a100701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:30:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 shutdown_connections 2023-10-21T17:30:16.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:30:16.914+0000 7f6a151a7640 1 -- 172.21.15.53:0/1589361478 wait complete. 2023-10-21T17:30:16.936 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"active_gid":37400,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":3632125819},{"type":"v1","addr":"172.21.15.124:6833","nonce":3632125819}]},"active_addr":"172.21.15.124:6833/3632125819","active_change":"2023-10-21T17:19:40.971783+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":10308,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":198,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":3194556637}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":2744557552}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":1897991612}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":4199687649}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":766800588}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":3755084077}]}]} 2023-10-21T17:30:16.941 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-10-21T17:30:16.941 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-21T17:30:16.941 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-10-21T17:30:16.942 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-21T17:30:16.942 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:30:17.018 INFO:tasks.ceph.mgr.x:Stopped 2023-10-21T17:30:17.018 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-21T17:30:17.028 INFO:tasks.ceph.mgr.x:Started 2023-10-21T17:30:17.622 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:17.621+0000 7f2139f7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:30:17.675 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:17.674+0000 7f2139f7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:30:18.118 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.116+0000 7f2139f7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:30:18.169 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.167+0000 7f2139f7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:30:18.218 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.217+0000 7f2139f7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:30:18.278 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.276+0000 7f2139f7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:30:18.326 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.325+0000 7f2139f7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:30:18.436 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.435+0000 7f2139f7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:30:18.498 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.497+0000 7f2139f7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:30:18.547 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.545+0000 7f2139f7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:30:18.602 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.601+0000 7f2139f7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:30:18.651 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:18.650+0000 7f2139f7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:30:19.030 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.029+0000 7f2139f7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:30:19.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.186+0000 7f2139f7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:30:19.293 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.292+0000 7f2139f7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:30:19.343 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.341+0000 7f2139f7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:30:19.451 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.450+0000 7f2139f7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:30:19.556 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.554+0000 7f2139f7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:30:19.712 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.710+0000 7f2139f7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:30:19.760 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.759+0000 7f2139f7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:30:19.879 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:19.877+0000 7f2139f7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:30:20.699 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:20.697+0000 7f2139f7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:30:20.993 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:20.991+0000 7f2139f7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:30:21.049 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.048+0000 7f2139f7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:30:21.161 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.159+0000 7f2139f7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:30:21.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b6f32640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b6f32640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b6f32640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b6f32640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b6f32640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b4f2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b4f2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b4f2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.502 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b4f2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:21.502 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:30:21.498+0000 7f20b4f2e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:30:46.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:30:46.929 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:31:00.496 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:31:00.494+0000 7f20bff84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:31:00.496 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:31:00.494+0000 7f20bff84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:31:00.496 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:31:00.494+0000 7f20bff84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:31:00.497 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:31:00.494+0000 7f20bff84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:31:00.497 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:31:00.494+0000 7f20bff84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:31:17.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:31:17.086 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:31:47.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:31:47.276 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:32:17.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:32:17.465 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:32:47.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:32:47.647 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:33:17.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:33:17.870 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:33:48.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:33:48.084 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:34:18.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:34:18.291 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:34:48.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:34:48.489 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:35:17.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2023-10-21T17:35:17.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.269+0000 7f0c3c600640 1 Processor -- start 2023-10-21T17:35:17.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.269+0000 7f0c3c600640 1 -- start start 2023-10-21T17:35:17.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3c600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c3406dfc0 0x7f0c34070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3c600640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c340b1cc0 con 0x7f0c340afbe0 2023-10-21T17:35:17.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3c600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c340b2620 con 0x7f0c34071070 2023-10-21T17:35:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3c600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c340b2060 con 0x7f0c3406dfc0 2023-10-21T17:35:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3a39c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0c340afbe0 0x7f0c340b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52832/0 (socket says 172.21.15.53:52832) 2023-10-21T17:35:17.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3a39c640 1 -- 172.21.15.53:0/1123735248 learned_addr learned my addr 172.21.15.53:0/1123735248 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:17.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c39b9b640 1 --2- 172.21.15.53:0/1123735248 >> v2:172.21.15.124:3300/0 conn(0x7f0c3406dfc0 0x7f0c34070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.271+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492082135 0 0) 0x7f0c340b1cc0 con 0x7f0c340afbe0 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c20003520 con 0x7f0c340afbe0 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2399517381 0 0) 0x7f0c20003520 con 0x7f0c340afbe0 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 >> v2:172.21.15.124:3300/0 conn(0x7f0c3406dfc0 msgr2=0x7f0c34070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 --2- 172.21.15.53:0/1123735248 >> v2:172.21.15.124:3300/0 conn(0x7f0c3406dfc0 0x7f0c34070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 >> v1:172.21.15.124:6789/0 conn(0x7f0c34071070 legacy=0x7f0c340cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c340b2340 con 0x7f0c340afbe0 2023-10-21T17:35:17.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f0c30001670 con 0x7f0c340afbe0 2023-10-21T17:35:17.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 >> v1:172.21.15.53:6789/0 conn(0x7f0c340afbe0 legacy=0x7f0c340b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c200047c0 0x7f0c20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c20005180 0x7f0c2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c340b2060 con 0x7f0c200039c0 2023-10-21T17:35:17.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2000c0b0 con 0x7f0c20005180 2023-10-21T17:35:17.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2000c270 con 0x7f0c200047c0 2023-10-21T17:35:17.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.272+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3a39c640 1 -- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c200047c0 msgr2=0x7f0c20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c200047c0 0x7f0c20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3a39c640 1 -- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c20005180 msgr2=0x7f0c2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c20005180 0x7f0c2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3a39c640 1 -- 172.21.15.53:0/1123735248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20003520 con 0x7f0c200039c0 2023-10-21T17:35:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.273+0000 7f0c3ab9d640 1 -- 172.21.15.53:0/1123735248 reap_dead start 2023-10-21T17:35:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.274+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c20003e00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c0e0 tx=0x7f0c200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b0c11f57eb59eb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.274+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30005ad0 con 0x7f0c200039c0 2023-10-21T17:35:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.274+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30005c30 con 0x7f0c200039c0 2023-10-21T17:35:17.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.274+0000 7f0c3939a640 1 -- 172.21.15.53:0/1123735248 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3002ca50 con 0x7f0c200039c0 2023-10-21T17:35:17.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 -- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 msgr2=0x7f0c20003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c20003e00 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c0e0 tx=0x7f0c200057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 -- 172.21.15.53:0/1123735248 shutdown_connections 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 --2- 172.21.15.53:0/1123735248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c20003e00 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 -- 172.21.15.53:0/1123735248 >> 172.21.15.53:0/1123735248 conn(0x7f0c3406a5d0 msgr2=0x7f0c340b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 -- 172.21.15.53:0/1123735248 shutdown_connections 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.275+0000 7f0c3c600640 1 -- 172.21.15.53:0/1123735248 wait complete. 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.277+0000 7f0c3c600640 1 Processor -- start 2023-10-21T17:35:17.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.277+0000 7f0c3c600640 1 -- start start 2023-10-21T17:35:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c341f16e0 0x7f0c341593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c34159930 0x7f0c341f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34159db0 con 0x7f0c200039c0 2023-10-21T17:35:17.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341f1250 con 0x7f0c34159930 2023-10-21T17:35:17.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c341f09a0 con 0x7f0c341f16e0 2023-10-21T17:35:17.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3a39c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3a39c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56868/0 (socket says 172.21.15.53:56868) 2023-10-21T17:35:17.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.278+0000 7f0c3a39c640 1 -- 172.21.15.53:0/3723016757 learned_addr learned my addr 172.21.15.53:0/3723016757 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:17.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 -- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c341f16e0 msgr2=0x7f0c341593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c341f16e0 0x7f0c341593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 -- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c34159930 msgr2=0x7f0c341f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c34159930 0x7f0c341f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 -- 172.21.15.53:0/3723016757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c341f4c80 con 0x7f0c200039c0 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.279+0000 7f0c3a39c640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c0e0 tx=0x7f0c3002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.280+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003c040 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.280+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3002cd60 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.280+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c300072b0 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.280+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c341f0c00 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.280+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c341500c0 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.281+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3404e160 con 0x7f0c200039c0 2023-10-21T17:35:17.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.282+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 109) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f0c30032070 con 0x7f0c200039c0 2023-10-21T17:35:17.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.283+0000 7f0c2affd640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 0x7f0c1806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.283+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f0c300ab140 con 0x7f0c200039c0 2023-10-21T17:35:17.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.284+0000 7f0c39b9b640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 0x7f0c1806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.285+0000 7f0c39b9b640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 0x7f0c1806d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c24006e80 tx=0x7f0c24006dd0 comp rx=0 tx=0).ready entity=mgr.37466 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.288+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c30075d00 con 0x7f0c200039c0 2023-10-21T17:35:17.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.508+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 --> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c34070630 con 0x7f0c1806afe0 2023-10-21T17:35:17.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.661+0000 7f0c2affd640 1 -- 172.21.15.53:0/3723016757 <== mgr.37466 v2:172.21.15.124:6832/2750944240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f0c240095c0 con 0x7f0c1806afe0 2023-10-21T17:35:17.663 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-21T17:35:17.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 msgr2=0x7f0c1806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 0x7f0c1806d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c24006e80 tx=0x7f0c24006dd0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 msgr2=0x7f0c3414fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c0e0 tx=0x7f0c3002e070 comp rx=0 tx=0).stop 2023-10-21T17:35:17.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 shutdown_connections 2023-10-21T17:35:17.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0c1806afe0 0x7f0c1806d4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c34159930 0x7f0c341f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c341f16e0 0x7f0c341593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 --2- 172.21.15.53:0/3723016757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c200039c0 0x7f0c3414fb80 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.665+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 >> 172.21.15.53:0/3723016757 conn(0x7f0c3406a5d0 msgr2=0x7f0c3415aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:17.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.666+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 shutdown_connections 2023-10-21T17:35:17.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.666+0000 7f0c3c600640 1 -- 172.21.15.53:0/3723016757 wait complete. 2023-10-21T17:35:17.690 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-21T17:35:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:17.691 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-21T17:35:17.745 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_20_00_UTC 2023-10-21T17:35:17.745 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_21_00_UTC 2023-10-21T17:35:17.745 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_22_00_UTC 2023-10-21T17:35:17.745 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_23_00_UTC 2023-10-21T17:35:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_24_00_UTC 2023-10-21T17:35:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_25_00_UTC 2023-10-21T17:35:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_26_00_UTC 2023-10-21T17:35:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_27_00_UTC 2023-10-21T17:35:17.746 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_28_00_UTC 2023-10-21T17:35:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_29_00_UTC 2023-10-21T17:35:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_30_00_UTC 2023-10-21T17:35:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_31_00_UTC 2023-10-21T17:35:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_32_00_UTC 2023-10-21T17:35:17.747 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_33_00_UTC 2023-10-21T17:35:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_34_00_UTC 2023-10-21T17:35:17.748 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_35_00_UTC 2023-10-21T17:35:17.749 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-21-17_20_00_UTC', 'scheduled-2023-10-21-17_21_00_UTC', 'scheduled-2023-10-21-17_22_00_UTC', 'scheduled-2023-10-21-17_23_00_UTC', 'scheduled-2023-10-21-17_24_00_UTC', 'scheduled-2023-10-21-17_25_00_UTC', 'scheduled-2023-10-21-17_26_00_UTC', 'scheduled-2023-10-21-17_27_00_UTC', 'scheduled-2023-10-21-17_28_00_UTC', 'scheduled-2023-10-21-17_29_00_UTC', 'scheduled-2023-10-21-17_30_00_UTC', 'scheduled-2023-10-21-17_31_00_UTC', 'scheduled-2023-10-21-17_32_00_UTC', 'scheduled-2023-10-21-17_33_00_UTC', 'scheduled-2023-10-21-17_34_00_UTC', 'scheduled-2023-10-21-17_35_00_UTC'] 2023-10-21T17:35:17.749 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-21T17:35:17.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.968+0000 7f39dc65d640 1 Processor -- start 2023-10-21T17:35:17.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.968+0000 7f39dc65d640 1 -- start start 2023-10-21T17:35:17.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39dc65d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f39d406c3a0 0x7f39d40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39dc65d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d415b310 con 0x7f39d406db60 2023-10-21T17:35:17.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39dc65d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d406d290 con 0x7f39d415aa00 2023-10-21T17:35:17.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39dc65d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d40ca230 con 0x7f39d406c3a0 2023-10-21T17:35:17.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39d9bf8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39d406db60 0x7f39d406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52848/0 (socket says 172.21.15.53:52848) 2023-10-21T17:35:17.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.970+0000 7f39d9bf8640 1 -- 172.21.15.53:0/1967001245 learned_addr learned my addr 172.21.15.53:0/1967001245 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:17.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1967001245 >> v2:172.21.15.124:3300/0 conn(0x7f39d406c3a0 0x7f39d40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846059186 0 0) 0x7f39c4001110 con 0x7f39d406db60 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c0003520 con 0x7f39d406db60 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3433832147 0 0) 0x7f39c0003520 con 0x7f39d406db60 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 >> v2:172.21.15.124:3300/0 conn(0x7f39d406c3a0 msgr2=0x7f39d40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 --2- 172.21.15.53:0/1967001245 >> v2:172.21.15.124:3300/0 conn(0x7f39d406c3a0 0x7f39d40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 >> v1:172.21.15.124:6789/0 conn(0x7f39d415aa00 legacy=0x7f39d40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40c9860 con 0x7f39d406db60 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.971+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f39c4001800 con 0x7f39d406db60 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 >> v1:172.21.15.53:6789/0 conn(0x7f39d406db60 legacy=0x7f39d406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 0x7f39c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c00047c0 0x7f39c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39c0005180 0x7f39c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c000bff0 con 0x7f39c0003970 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c000c160 con 0x7f39c00047c0 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c000c320 con 0x7f39c0005180 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 0x7f39c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d9bf8640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c00047c0 0x7f39c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 -- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39c0005180 msgr2=0x7f39c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39c0005180 0x7f39c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 -- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c00047c0 msgr2=0x7f39c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:17.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c00047c0 0x7f39c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39da3f9640 1 -- 172.21.15.53:0/1967001245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0003520 con 0x7f39c0003970 2023-10-21T17:35:17.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39d9bf8640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c00047c0 0x7f39c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:17.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.972+0000 7f39dabfa640 1 -- 172.21.15.53:0/1967001245 reap_dead start 2023-10-21T17:35:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 0x7f39c0003d90 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009ac0 tx=0x7f39c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282a15b0a5444304 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d002c6b0 con 0x7f39c0003970 2023-10-21T17:35:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d002c810 con 0x7f39c0003970 2023-10-21T17:35:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39d93f7640 1 -- 172.21.15.53:0/1967001245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d002cba0 con 0x7f39c0003970 2023-10-21T17:35:17.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39dc65d640 1 -- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 msgr2=0x7f39c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 0x7f39c0003d90 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009ac0 tx=0x7f39c00057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39dc65d640 1 -- 172.21.15.53:0/1967001245 shutdown_connections 2023-10-21T17:35:17.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1967001245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39c0003970 0x7f39c0003d90 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.974+0000 7f39dc65d640 1 -- 172.21.15.53:0/1967001245 >> 172.21.15.53:0/1967001245 conn(0x7f39d406a5d0 msgr2=0x7f39d40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.975+0000 7f39dc65d640 1 -- 172.21.15.53:0/1967001245 shutdown_connections 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.975+0000 7f39dc65d640 1 -- 172.21.15.53:0/1967001245 wait complete. 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.976+0000 7f39dc65d640 1 Processor -- start 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.977+0000 7f39dc65d640 1 -- start start 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.977+0000 7f39dc65d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c0003970 0x7f39d41f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39dc65d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39dc65d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d40c3f70 0x7f39d40c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39dc65d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40c51f0 con 0x7f39d40cc640 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39dc65d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40cd000 con 0x7f39c0003970 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39dc65d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41f26c0 con 0x7f39d40c3f70 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56882/0 (socket says 172.21.15.53:56882) 2023-10-21T17:35:17.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 -- 172.21.15.53:0/1472762740 learned_addr learned my addr 172.21.15.53:0/1472762740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:17.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c0003970 0x7f39d41f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 -- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d40c3f70 msgr2=0x7f39d40c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d40c3f70 0x7f39d40c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 -- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c0003970 msgr2=0x7f39d41f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c0003970 0x7f39d41f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.978+0000 7f39d9bf8640 1 -- 172.21.15.53:0/1472762740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40c2580 con 0x7f39d40cc640 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39d9bf8640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001270 tx=0x7f39c400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40074e0 con 0x7f39d40cc640 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4007640 con 0x7f39d40cc640 2023-10-21T17:35:17.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c40049e0 con 0x7f39d40cc640 2023-10-21T17:35:17.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40c4960 con 0x7f39d40cc640 2023-10-21T17:35:17.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.979+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40c4f10 con 0x7f39d40cc640 2023-10-21T17:35:17.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.980+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d404e160 con 0x7f39d40cc640 2023-10-21T17:35:17.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.981+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 109) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f39c4004b40 con 0x7f39d40cc640 2023-10-21T17:35:17.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.982+0000 7f39caffd640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 0x7f39b806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:17.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.982+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f39c4089630 con 0x7f39d40cc640 2023-10-21T17:35:17.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.983+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 0x7f39b806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:17.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.984+0000 7f39da3f9640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 0x7f39b806d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39d0036000 tx=0x7f39d0030000 comp rx=0 tx=0).ready entity=mgr.37466 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:17.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:17.987+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39c4053fb0 con 0x7f39d40cc640 2023-10-21T17:35:18.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.203+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 --> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39d40c9210 con 0x7f39b806b010 2023-10-21T17:35:18.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.219+0000 7f39caffd640 1 -- 172.21.15.53:0/1472762740 <== mgr.37466 v2:172.21.15.124:6832/2750944240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f39d00040c0 con 0x7f39b806b010 2023-10-21T17:35:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:18.221 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:19:42", "first": "2023-10-21T17:20:00", "last": "2023-10-21T17:35:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-21T17:35:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 msgr2=0x7f39b806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 0x7f39b806d4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39d0036000 tx=0x7f39d0030000 comp rx=0 tx=0).stop 2023-10-21T17:35:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 msgr2=0x7f39d40ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001270 tx=0x7f39c400dd40 comp rx=0 tx=0).stop 2023-10-21T17:35:18.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 shutdown_connections 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f39b806b010 0x7f39b806d4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39d40c3f70 0x7f39d40c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40cc640 0x7f39d40ccac0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 --2- 172.21.15.53:0/1472762740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39c0003970 0x7f39d41f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 >> 172.21.15.53:0/1472762740 conn(0x7f39d406a5d0 msgr2=0x7f39d40cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:18.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 shutdown_connections 2023-10-21T17:35:18.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.222+0000 7f39dc65d640 1 -- 172.21.15.53:0/1472762740 wait complete. 2023-10-21T17:35:18.243 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:19:42", "first": "2023-10-21T17:20:00", "last": "2023-10-21T17:35:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-21T17:35:18.243 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-21T00:00:00', 'created': '2023-10-21T17:19:42', 'first': '2023-10-21T17:20:00', 'last': '2023-10-21T17:35:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-21T17:35:18.244 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-21T17:35:18.244 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-21T17:35:18.244 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2023-10-21T17:35:18.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.459+0000 7f0b7dd6c640 1 Processor -- start 2023-10-21T17:35:18.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.459+0000 7f0b7dd6c640 1 -- start start 2023-10-21T17:35:18.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b7dd6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 0x7f0b780b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b7dd6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b780cb130 0x7f0b780cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b7dd6c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780b2620 con 0x7f0b7806dfc0 2023-10-21T17:35:18.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b7dd6c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780cbae0 con 0x7f0b780cb130 2023-10-21T17:35:18.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b7dd6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b2060 con 0x7f0b780afbe0 2023-10-21T17:35:18.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b76ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0b7806dfc0 0x7f0b78070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52850/0 (socket says 172.21.15.53:52850) 2023-10-21T17:35:18.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b76ffd640 1 -- 172.21.15.53:0/2795046376 learned_addr learned my addr 172.21.15.53:0/2795046376 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:18.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.461+0000 7f0b77fff640 1 --2- 172.21.15.53:0/2795046376 >> v2:172.21.15.53:3300/0 conn(0x7f0b780cb130 0x7f0b780cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b777fe640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 0x7f0b780b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b767fc640 1 -- 172.21.15.53:0/2795046376 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261460773 0 0) 0x7f0b780b2620 con 0x7f0b7806dfc0 2023-10-21T17:35:18.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b767fc640 1 -- 172.21.15.53:0/2795046376 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b64003520 con 0x7f0b7806dfc0 2023-10-21T17:35:18.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b77fff640 1 -- 172.21.15.53:0/2795046376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 msgr2=0x7f0b780b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:18.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b77fff640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 0x7f0b780b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b77fff640 1 -- 172.21.15.53:0/2795046376 >> v1:172.21.15.53:6789/0 conn(0x7f0b7806dfc0 legacy=0x7f0b78070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b77fff640 1 -- 172.21.15.53:0/2795046376 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780b2340 con 0x7f0b780cb130 2023-10-21T17:35:18.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.462+0000 7f0b777fe640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 0x7f0b780b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:35:18.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.463+0000 7f0b77fff640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cb5a0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f0b700097a0 tx=0x7f0b7002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0f90eb1fb18b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:18.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b767fc640 1 -- 172.21.15.53:0/2795046376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7002d040 con 0x7f0b780cb130 2023-10-21T17:35:18.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b767fc640 1 -- 172.21.15.53:0/2795046376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b70031810 con 0x7f0b780cb130 2023-10-21T17:35:18.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b767fc640 1 -- 172.21.15.53:0/2795046376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70031ba0 con 0x7f0b780cb130 2023-10-21T17:35:18.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/2795046376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 msgr2=0x7f0b780cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cb5a0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f0b700097a0 tx=0x7f0b7002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/2795046376 shutdown_connections 2023-10-21T17:35:18.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cb5a0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/2795046376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b780afbe0 0x7f0b780b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/2795046376 >> 172.21.15.53:0/2795046376 conn(0x7f0b7806a5d0 msgr2=0x7f0b780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:18.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.464+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/2795046376 shutdown_connections 2023-10-21T17:35:18.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.465+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/2795046376 wait complete. 2023-10-21T17:35:18.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.466+0000 7f0b7dd6c640 1 Processor -- start 2023-10-21T17:35:18.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.466+0000 7f0b7dd6c640 1 -- start start 2023-10-21T17:35:18.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 0x7f0b78147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b780afbe0 0x7f0b780cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780c26e0 con 0x7f0b780cb130 2023-10-21T17:35:18.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780c29c0 con 0x7f0b780afbe0 2023-10-21T17:35:18.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.467+0000 7f0b7dd6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780c2ca0 con 0x7f0b7806dfc0 2023-10-21T17:35:18.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56894/0 (socket says 172.21.15.53:56894) 2023-10-21T17:35:18.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 -- 172.21.15.53:0/854386754 learned_addr learned my addr 172.21.15.53:0/854386754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:18.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b777fe640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 0x7f0b78147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b76ffd640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b780afbe0 0x7f0b780cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 -- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 msgr2=0x7f0b78147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 0x7f0b78147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 -- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b780afbe0 msgr2=0x7f0b780cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b780afbe0 0x7f0b780cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b77fff640 1 -- 172.21.15.53:0/854386754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70009400 con 0x7f0b780cb130 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.468+0000 7f0b777fe640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 0x7f0b78147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b77fff640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002b8f0 tx=0x7f0b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:18.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70006e10 con 0x7f0b780cb130 2023-10-21T17:35:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7003ace0 con 0x7f0b780cb130 2023-10-21T17:35:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70044610 con 0x7f0b780cb130 2023-10-21T17:35:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b780c2f80 con 0x7f0b780cb130 2023-10-21T17:35:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.469+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b780c2250 con 0x7f0b780cb130 2023-10-21T17:35:18.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.470+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b780c24a0 con 0x7f0b780cb130 2023-10-21T17:35:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.471+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 109) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f0b70044770 con 0x7f0b780cb130 2023-10-21T17:35:18.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.472+0000 7f0b6ffff640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 0x7f0b5806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:18.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.472+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f0b700abfa0 con 0x7f0b780cb130 2023-10-21T17:35:18.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.473+0000 7f0b777fe640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 0x7f0b5806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:18.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.474+0000 7f0b777fe640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 0x7f0b5806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0b68006fd0 tx=0x7f0b68008040 comp rx=0 tx=0).ready entity=mgr.37466 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:18.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.477+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b700deca0 con 0x7f0b780cb130 2023-10-21T17:35:18.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:35:18.685 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:35:18.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.688+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 --> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b780c2b30 con 0x7f0b5806afe0 2023-10-21T17:35:18.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.779+0000 7f0b6ffff640 1 -- 172.21.15.53:0/854386754 <== mgr.37466 v2:172.21.15.124:6832/2750944240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f0b780c2b30 con 0x7f0b5806afe0 2023-10-21T17:35:18.781 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-21T17:35:18.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 msgr2=0x7f0b5806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 0x7f0b5806d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0b68006fd0 tx=0x7f0b68008040 comp rx=0 tx=0).stop 2023-10-21T17:35:18.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 msgr2=0x7f0b780cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f0b7002b8f0 tx=0x7f0b700046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 shutdown_connections 2023-10-21T17:35:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f0b5806afe0 0x7f0b5806d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b780cb130 0x7f0b780cdc60 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b780afbe0 0x7f0b780cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 --2- 172.21.15.53:0/854386754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b7806dfc0 0x7f0b78147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:18.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.781+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 >> 172.21.15.53:0/854386754 conn(0x7f0b7806a5d0 msgr2=0x7f0b780c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:18.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.782+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 shutdown_connections 2023-10-21T17:35:18.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:18.782+0000 7f0b7dd6c640 1 -- 172.21.15.53:0/854386754 wait complete. 2023-10-21T17:35:18.799 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-21T17:35:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:18.799 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-21T17:35:18.912 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_20_00_UTC 2023-10-21T17:35:18.912 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_21_00_UTC 2023-10-21T17:35:18.913 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_22_00_UTC 2023-10-21T17:35:18.913 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_23_00_UTC 2023-10-21T17:35:18.913 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_24_00_UTC 2023-10-21T17:35:18.913 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_25_00_UTC 2023-10-21T17:35:18.913 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_26_00_UTC 2023-10-21T17:35:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_27_00_UTC 2023-10-21T17:35:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_28_00_UTC 2023-10-21T17:35:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_29_00_UTC 2023-10-21T17:35:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_30_00_UTC 2023-10-21T17:35:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_31_00_UTC 2023-10-21T17:35:18.915 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_32_00_UTC 2023-10-21T17:35:18.915 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_33_00_UTC 2023-10-21T17:35:18.915 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_34_00_UTC 2023-10-21T17:35:18.915 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_35_00_UTC 2023-10-21T17:35:18.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_20_00_UTC 2023-10-21T17:35:18.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:18.917 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_20_00_UTC) 2023-10-21T17:35:18.985 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_21_00_UTC 2023-10-21T17:35:18.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:18.986 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_21_00_UTC) 2023-10-21T17:35:19.057 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_22_00_UTC 2023-10-21T17:35:19.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.058 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_22_00_UTC) 2023-10-21T17:35:19.131 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_23_00_UTC 2023-10-21T17:35:19.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.132 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_23_00_UTC) 2023-10-21T17:35:19.204 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_24_00_UTC 2023-10-21T17:35:19.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.205 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_24_00_UTC) 2023-10-21T17:35:19.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_25_00_UTC 2023-10-21T17:35:19.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.273 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_25_00_UTC) 2023-10-21T17:35:19.346 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_26_00_UTC 2023-10-21T17:35:19.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.346 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_26_00_UTC) 2023-10-21T17:35:19.418 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_27_00_UTC 2023-10-21T17:35:19.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.419 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_27_00_UTC) 2023-10-21T17:35:19.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_28_00_UTC 2023-10-21T17:35:19.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.487 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_28_00_UTC) 2023-10-21T17:35:19.548 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_29_00_UTC 2023-10-21T17:35:19.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.549 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_29_00_UTC) 2023-10-21T17:35:19.614 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_30_00_UTC 2023-10-21T17:35:19.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.615 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_30_00_UTC) 2023-10-21T17:35:19.691 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_31_00_UTC 2023-10-21T17:35:19.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.692 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_31_00_UTC) 2023-10-21T17:35:19.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_32_00_UTC 2023-10-21T17:35:19.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.770 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_32_00_UTC) 2023-10-21T17:35:19.843 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_33_00_UTC 2023-10-21T17:35:19.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.844 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_33_00_UTC) 2023-10-21T17:35:19.919 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_34_00_UTC 2023-10-21T17:35:19.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.920 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_34_00_UTC) 2023-10-21T17:35:19.993 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_35_00_UTC 2023-10-21T17:35:19.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:19.994 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-21-17_35_00_UTC) 2023-10-21T17:35:20.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:20.069 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-21T17:35:20.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:35:20.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.370+0000 7fa205333640 1 Processor -- start 2023-10-21T17:35:20.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.370+0000 7fa205333640 1 -- start start 2023-10-21T17:35:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.371+0000 7fa205333640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2000afbe0 0x7fa2000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.371+0000 7fa205333640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 0x7fa2000cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa205333640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000cbad0 con 0x7fa20006dfc0 2023-10-21T17:35:20.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa205333640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000b2620 con 0x7fa2000afbe0 2023-10-21T17:35:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa205333640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2000b2060 con 0x7fa2000cb130 2023-10-21T17:35:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa1feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2000afbe0 0x7fa2000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa1feffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa2000afbe0 0x7fa2000b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56900/0 (socket says 172.21.15.53:56900) 2023-10-21T17:35:20.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa1feffd640 1 -- 172.21.15.53:0/4145872688 learned_addr learned my addr 172.21.15.53:0/4145872688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:20.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.372+0000 7fa1ff7fe640 1 --2- 172.21.15.53:0/4145872688 >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 0x7fa2000cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.373+0000 7fa1feffd640 1 -- 172.21.15.53:0/4145872688 >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 msgr2=0x7fa2000cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.373+0000 7fa1feffd640 1 --2- 172.21.15.53:0/4145872688 >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 0x7fa2000cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.373+0000 7fa1feffd640 1 -- 172.21.15.53:0/4145872688 >> v1:172.21.15.53:6789/0 conn(0x7fa20006dfc0 legacy=0x7fa200070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.373+0000 7fa1feffd640 1 -- 172.21.15.53:0/4145872688 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000b2340 con 0x7fa2000afbe0 2023-10-21T17:35:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.373+0000 7fa1ff7fe640 1 --2- 172.21.15.53:0/4145872688 >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 0x7fa2000cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:20.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.374+0000 7fa1feffd640 1 --2- 172.21.15.53:0/4145872688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000afbe0 0x7fa2000b15b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fa2000c8d10 tx=0x7fa1f002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee9dc8efb7845e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.374+0000 7fa1fdffb640 1 -- 172.21.15.53:0/4145872688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002cb10 con 0x7fa2000afbe0 2023-10-21T17:35:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.374+0000 7fa1fdffb640 1 -- 172.21.15.53:0/4145872688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f002cc70 con 0x7fa2000afbe0 2023-10-21T17:35:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.374+0000 7fa1fdffb640 1 -- 172.21.15.53:0/4145872688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00343f0 con 0x7fa2000afbe0 2023-10-21T17:35:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 -- 172.21.15.53:0/4145872688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000afbe0 msgr2=0x7fa2000b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:20.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 --2- 172.21.15.53:0/4145872688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000afbe0 0x7fa2000b15b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fa2000c8d10 tx=0x7fa1f002bf70 comp rx=0 tx=0).stop 2023-10-21T17:35:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 -- 172.21.15.53:0/4145872688 shutdown_connections 2023-10-21T17:35:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 --2- 172.21.15.53:0/4145872688 >> v2:172.21.15.124:3300/0 conn(0x7fa2000cb130 0x7fa2000cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 --2- 172.21.15.53:0/4145872688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000afbe0 0x7fa2000b15b0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 -- 172.21.15.53:0/4145872688 >> 172.21.15.53:0/4145872688 conn(0x7fa20006a5d0 msgr2=0x7fa2000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:20.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 -- 172.21.15.53:0/4145872688 shutdown_connections 2023-10-21T17:35:20.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.375+0000 7fa205333640 1 -- 172.21.15.53:0/4145872688 wait complete. 2023-10-21T17:35:20.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.377+0000 7fa205333640 1 Processor -- start 2023-10-21T17:35:20.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.377+0000 7fa205333640 1 -- start start 2023-10-21T17:35:20.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa20006dfc0 0x7fa200152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000afbe0 0x7fa2000c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000c2250 con 0x7fa2000cb130 2023-10-21T17:35:20.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000c3d60 con 0x7fa2000afbe0 2023-10-21T17:35:20.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa205333640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000c24a0 con 0x7fa20006dfc0 2023-10-21T17:35:20.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56906/0 (socket says 172.21.15.53:56906) 2023-10-21T17:35:20.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.378+0000 7fa1ff7fe640 1 -- 172.21.15.53:0/1898740462 learned_addr learned my addr 172.21.15.53:0/1898740462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:20.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1feffd640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa20006dfc0 0x7fa200152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1fe7fc640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000afbe0 0x7fa2000c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 -- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa20006dfc0 msgr2=0x7fa200152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:20.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa20006dfc0 0x7fa200152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:20.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 -- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000afbe0 msgr2=0x7fa2000c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:20.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000afbe0 0x7fa2000c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:20.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 -- 172.21.15.53:0/1898740462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009b30 con 0x7fa2000cb130 2023-10-21T17:35:20.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1ff7fe640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c7f0 tx=0x7fa1f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:20.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8007820 con 0x7fa2000cb130 2023-10-21T17:35:20.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8007980 con 0x7fa2000cb130 2023-10-21T17:35:20.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8007d70 con 0x7fa2000cb130 2023-10-21T17:35:20.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.379+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2000c2780 con 0x7fa2000cb130 2023-10-21T17:35:20.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.380+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2001531f0 con 0x7fa2000cb130 2023-10-21T17:35:20.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.381+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000c2ca0 con 0x7fa2000cb130 2023-10-21T17:35:20.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.382+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 109) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fa1f80113f0 con 0x7fa2000cb130 2023-10-21T17:35:20.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.383+0000 7fa1f77fe640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 0x7fa1e006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:20.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.383+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fa1f808a060 con 0x7fa2000cb130 2023-10-21T17:35:20.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.383+0000 7fa1feffd640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 0x7fa1e006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:20.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.385+0000 7fa1feffd640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 0x7fa1e006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0035000 tx=0x7fa1f0030000 comp rx=0 tx=0).ready entity=mgr.37466 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:20.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.388+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2000c2ca0 con 0x7fa2000cb130 2023-10-21T17:35:20.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:20.632+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa2000c2ca0 con 0x7fa2000cb130 2023-10-21T17:35:21.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.261+0000 7fa1f77fe640 1 -- 172.21.15.53:0/1898740462 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v110) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa2000c2ca0 con 0x7fa2000cb130 2023-10-21T17:35:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.267+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 msgr2=0x7fa1e006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.267+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 0x7fa1e006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0035000 tx=0x7fa1f0030000 comp rx=0 tx=0).stop 2023-10-21T17:35:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.267+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 msgr2=0x7fa20015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.267+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c7f0 tx=0x7fa1f800ccb0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 shutdown_connections 2023-10-21T17:35:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa1e006b010 0x7fa1e006d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa2000cb130 0x7fa20015c840 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2000afbe0 0x7fa2000c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 --2- 172.21.15.53:0/1898740462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa20006dfc0 0x7fa200152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.270+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 >> 172.21.15.53:0/1898740462 conn(0x7fa20006a5d0 msgr2=0x7fa200159640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.273+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 shutdown_connections 2023-10-21T17:35:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.274+0000 7fa205333640 1 -- 172.21.15.53:0/1898740462 wait complete. 2023-10-21T17:35:21.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:35:21.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:35:21.435 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:35:21.533 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:35:21.534 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:21.534 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:21.604 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:21.604 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:21.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:21.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.864+0000 7fb34a40e640 1 Processor -- start 2023-10-21T17:35:21.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.864+0000 7fb34a40e640 1 -- start start 2023-10-21T17:35:21.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.865+0000 7fb34a40e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb34406b520 0x7fb34406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.865+0000 7fb34a40e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3440b0ab0 0x7fb3440b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.865+0000 7fb34a40e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440b0340 con 0x7fb3440af8b0 2023-10-21T17:35:21.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.865+0000 7fb34a40e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34406b3a0 con 0x7fb3440b0ab0 2023-10-21T17:35:21.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.865+0000 7fb34a40e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440b0150 con 0x7fb34406b520 2023-10-21T17:35:21.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3440b0ab0 0x7fb3440b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:21.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3440b0ab0 0x7fb3440b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56926/0 (socket says 172.21.15.53:56926) 2023-10-21T17:35:21.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 -- 172.21.15.53:0/3274983702 learned_addr learned my addr 172.21.15.53:0/3274983702 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:21.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb343fff640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb34406b520 0x7fb34406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:21.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 -- 172.21.15.53:0/3274983702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb34406b520 msgr2=0x7fb34406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:21.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb34406b520 0x7fb34406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.866+0000 7fb3489ab640 1 -- 172.21.15.53:0/3274983702 >> v1:172.21.15.124:6789/0 conn(0x7fb3440af8b0 legacy=0x7fb34406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.867+0000 7fb3489ab640 1 -- 172.21.15.53:0/3274983702 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440cf8f0 con 0x7fb3440b0ab0 2023-10-21T17:35:21.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb3489ab640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3440b0ab0 0x7fb3440b0f40 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35695835d897dbb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:21.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb342ffd640 1 -- 172.21.15.53:0/3274983702 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c02ca70 con 0x7fb3440b0ab0 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb342ffd640 1 -- 172.21.15.53:0/3274983702 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c02cbd0 con 0x7fb3440b0ab0 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb342ffd640 1 -- 172.21.15.53:0/3274983702 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0343f0 con 0x7fb3440b0ab0 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb34a40e640 1 -- 172.21.15.53:0/3274983702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3440b0ab0 msgr2=0x7fb3440b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.868+0000 7fb34a40e640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3440b0ab0 0x7fb3440b0f40 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02bed0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 -- 172.21.15.53:0/3274983702 shutdown_connections 2023-10-21T17:35:21.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3440b0ab0 0x7fb3440b0f40 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 --2- 172.21.15.53:0/3274983702 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb34406b520 0x7fb34406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 -- 172.21.15.53:0/3274983702 >> 172.21.15.53:0/3274983702 conn(0x7fb3440d09e0 msgr2=0x7fb3440d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:21.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 -- 172.21.15.53:0/3274983702 shutdown_connections 2023-10-21T17:35:21.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.869+0000 7fb34a40e640 1 -- 172.21.15.53:0/3274983702 wait complete. 2023-10-21T17:35:21.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.871+0000 7fb34a40e640 1 Processor -- start 2023-10-21T17:35:21.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.871+0000 7fb34a40e640 1 -- start start 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3440af8b0 0x7fb3440c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 0x7fb344158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c4840 con 0x7fb34406b520 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c4b20 con 0x7fb3440af8b0 2023-10-21T17:35:21.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb34a40e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c4e00 con 0x7fb3440b0ab0 2023-10-21T17:35:21.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:21.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56934/0 (socket says 172.21.15.53:56934) 2023-10-21T17:35:21.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 -- 172.21.15.53:0/2241877227 learned_addr learned my addr 172.21.15.53:0/2241877227 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:21.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb3489ab640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 0x7fb344158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:21.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb3437fe640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3440af8b0 0x7fb3440c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:21.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 -- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 msgr2=0x7fb344158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 0x7fb344158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 -- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3440af8b0 msgr2=0x7fb3440c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:21.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3440af8b0 0x7fb3440c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:21.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb343fff640 1 -- 172.21.15.53:0/2241877227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009b30 con 0x7fb34406b520 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.872+0000 7fb3489ab640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 0x7fb344158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb343fff640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb33400d7d0 tx=0x7fb33400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340096c0 con 0x7fb34406b520 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334009820 con 0x7fb34406b520 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334009bd0 con 0x7fb34406b520 2023-10-21T17:35:21.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440c50e0 con 0x7fb34406b520 2023-10-21T17:35:21.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.873+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3440c56a0 con 0x7fb34406b520 2023-10-21T17:35:21.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.875+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb344067490 con 0x7fb34406b520 2023-10-21T17:35:21.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.876+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fb334009d30 con 0x7fb34406b520 2023-10-21T17:35:21.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.876+0000 7fb340ff9640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 0x7fb33006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:21.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.877+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fb33408a4e0 con 0x7fb34406b520 2023-10-21T17:35:21.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.877+0000 7fb3437fe640 1 -- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 msgr2=0x7fb33006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:21.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.877+0000 7fb3437fe640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 0x7fb33006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:21.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:21.880+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb334054fc0 con 0x7fb34406b520 2023-10-21T17:35:21.886 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:21.885+0000 7fd5f1e41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:35:21.904 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:21.903+0000 7fcf23b7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:35:21.939 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:21.938+0000 7fd5f1e41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:35:21.961 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:21.959+0000 7fcf23b7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:35:22.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.072+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3440b0560 con 0x7fb34406b520 2023-10-21T17:35:22.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.074+0000 7fb340ff9640 1 -- 172.21.15.53:0/2241877227 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+18073 (secure 0 0 0) 0x7fb3340585e0 con 0x7fb34406b520 2023-10-21T17:35:22.075 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:22.076 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":200,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:21.248385+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":200,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2744557552":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3194556637":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/1897991612":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/4199687649":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3755084077":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3409964486":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/766800588":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:6832/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3041665938":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3287676299":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3295254396":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3702020697":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/1180026628":"2023-10-22T17:19:40.971451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:22.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.076+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 msgr2=0x7fb33006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:22.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.076+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 0x7fb33006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 msgr2=0x7fb344153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fb33400d7d0 tx=0x7fb33400dc90 comp rx=0 tx=0).stop 2023-10-21T17:35:22.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 shutdown_connections 2023-10-21T17:35:22.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fb33006af90 0x7fb33006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3440b0ab0 0x7fb344158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3440af8b0 0x7fb3440c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 --2- 172.21.15.53:0/2241877227 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34406b520 0x7fb344153cf0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 >> 172.21.15.53:0/2241877227 conn(0x7fb3440d09e0 msgr2=0x7fb3440aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:22.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 shutdown_connections 2023-10-21T17:35:22.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.077+0000 7fb34a40e640 1 -- 172.21.15.53:0/2241877227 wait complete. 2023-10-21T17:35:22.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:19:23.993283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:19:25.097499+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 200, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:35:22.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:35:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.295+0000 7f6963bf0640 1 Processor -- start 2023-10-21T17:35:22.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.296+0000 7f6963bf0640 1 -- start start 2023-10-21T17:35:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c15a820 0x7f695c160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f695c0c9080 0x7f695c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f695c070b50 0x7f695c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c0c9a40 con 0x7f695c070b50 2023-10-21T17:35:22.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c071520 con 0x7f695c0c9080 2023-10-21T17:35:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f6963bf0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15ac00 con 0x7f695c15a820 2023-10-21T17:35:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f696218d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f695c070b50 0x7f695c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f696218d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f695c070b50 0x7f695c070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56940/0 (socket says 172.21.15.53:56940) 2023-10-21T17:35:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.297+0000 7f696218d640 1 -- 172.21.15.53:0/4113511855 learned_addr learned my addr 172.21.15.53:0/4113511855 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696118b640 1 --2- 172.21.15.53:0/4113511855 >> v2:172.21.15.124:3300/0 conn(0x7f695c0c9080 0x7f695c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696198c640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c15a820 0x7f695c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696218d640 1 -- 172.21.15.53:0/4113511855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c15a820 msgr2=0x7f695c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:22.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696218d640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c15a820 0x7f695c160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696218d640 1 -- 172.21.15.53:0/4113511855 >> v2:172.21.15.124:3300/0 conn(0x7f695c0c9080 msgr2=0x7f695c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696218d640 1 --2- 172.21.15.53:0/4113511855 >> v2:172.21.15.124:3300/0 conn(0x7f695c0c9080 0x7f695c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.298+0000 7f696218d640 1 -- 172.21.15.53:0/4113511855 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c0c7ad0 con 0x7f695c070b50 2023-10-21T17:35:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.299+0000 7f696218d640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c070b50 0x7f695c070fe0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f695c06e7e0 tx=0x7f695402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eeacde3173b4fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.299+0000 7f696098a640 1 -- 172.21.15.53:0/4113511855 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695402ca80 con 0x7f695c070b50 2023-10-21T17:35:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.299+0000 7f696098a640 1 -- 172.21.15.53:0/4113511855 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695402cbe0 con 0x7f695c070b50 2023-10-21T17:35:22.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.299+0000 7f696098a640 1 -- 172.21.15.53:0/4113511855 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540343f0 con 0x7f695c070b50 2023-10-21T17:35:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 -- 172.21.15.53:0/4113511855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c070b50 msgr2=0x7f695c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c070b50 0x7f695c070fe0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f695c06e7e0 tx=0x7f695402bee0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 -- 172.21.15.53:0/4113511855 shutdown_connections 2023-10-21T17:35:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c070b50 0x7f695c070fe0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 --2- 172.21.15.53:0/4113511855 >> v2:172.21.15.124:3300/0 conn(0x7f695c0c9080 0x7f695c0c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 --2- 172.21.15.53:0/4113511855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c15a820 0x7f695c160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 -- 172.21.15.53:0/4113511855 >> 172.21.15.53:0/4113511855 conn(0x7f695c069d50 msgr2=0x7f695c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 -- 172.21.15.53:0/4113511855 shutdown_connections 2023-10-21T17:35:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.300+0000 7f6963bf0640 1 -- 172.21.15.53:0/4113511855 wait complete. 2023-10-21T17:35:22.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.302+0000 7f6963bf0640 1 Processor -- start 2023-10-21T17:35:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.302+0000 7f6963bf0640 1 -- start start 2023-10-21T17:35:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c070b50 0x7f695c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 0x7f695c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c146af0 con 0x7f695c0c9080 2023-10-21T17:35:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c1f2330 con 0x7f695c15a820 2023-10-21T17:35:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f6963bf0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c1568d0 con 0x7f695c070b50 2023-10-21T17:35:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f696118b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f696118b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56956/0 (socket says 172.21.15.53:56956) 2023-10-21T17:35:22.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f696118b640 1 -- 172.21.15.53:0/560764995 learned_addr learned my addr 172.21.15.53:0/560764995 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.303+0000 7f696198c640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c070b50 0x7f695c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696218d640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 0x7f695c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c070b50 msgr2=0x7f695c1f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c070b50 0x7f695c1f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 msgr2=0x7f695c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 0x7f695c15f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 -- 172.21.15.53:0/560764995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009b30 con 0x7f695c0c9080 2023-10-21T17:35:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696218d640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 0x7f695c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:35:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f696118b640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d800 tx=0x7f694c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:22.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00b6c0 con 0x7f695c0c9080 2023-10-21T17:35:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c00b820 con 0x7f695c0c9080 2023-10-21T17:35:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c00bbb0 con 0x7f695c0c9080 2023-10-21T17:35:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.304+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c156bb0 con 0x7f695c0c9080 2023-10-21T17:35:22.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.305+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c157170 con 0x7f695c0c9080 2023-10-21T17:35:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.306+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c04e160 con 0x7f695c0c9080 2023-10-21T17:35:22.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.307+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f694c00bd10 con 0x7f695c0c9080 2023-10-21T17:35:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.308+0000 7f69527fc640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 0x7f694006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.308+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f694c08a160 con 0x7f695c0c9080 2023-10-21T17:35:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.308+0000 7f696198c640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 msgr2=0x7f694006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:22.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.308+0000 7f696198c640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 0x7f694006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:22.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.312+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f694c054bb0 con 0x7f695c0c9080 2023-10-21T17:35:22.395 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.394+0000 7fd5f1e41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:35:22.446 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.445+0000 7fd5f1e41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:35:22.447 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.446+0000 7fcf23b7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:35:22.501 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.499+0000 7fd5f1e41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:35:22.502 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.501+0000 7fcf23b7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:35:22.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.508+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f695c0c8940 con 0x7f695c0c9080 2023-10-21T17:35:22.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.509+0000 7f69527fc640 1 -- 172.21.15.53:0/560764995 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v201) v1 ==== 93+0+10 (secure 0 0 0) 0x7f694c0581d0 con 0x7f695c0c9080 2023-10-21T17:35:22.510 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:35:22.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.509+0000 7f696198c640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 msgr2=0x7f694006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:22.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.509+0000 7f696198c640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 0x7f694006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:22.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.511+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 msgr2=0x7f694006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:22.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.511+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 0x7f694006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.511+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 msgr2=0x7f695c1497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.511+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f694c00d800 tx=0x7f694c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 shutdown_connections 2023-10-21T17:35:22.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7f694006af90 0x7f694006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f695c15a820 0x7f695c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f695c0c9080 0x7f695c1497d0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 --2- 172.21.15.53:0/560764995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f695c070b50 0x7f695c1f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 >> 172.21.15.53:0/560764995 conn(0x7f695c069d50 msgr2=0x7f695c155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:22.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 shutdown_connections 2023-10-21T17:35:22.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.512+0000 7f6963bf0640 1 -- 172.21.15.53:0/560764995 wait complete. 2023-10-21T17:35:22.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:35:22.557 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.556+0000 7fcf23b7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:35:22.564 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.563+0000 7fd5f1e41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:35:22.614 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.613+0000 7fd5f1e41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:35:22.627 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.626+0000 7fcf23b7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:35:22.679 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.678+0000 7fcf23b7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:35:22.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.729+0000 7fc57f121640 1 Processor -- start 2023-10-21T17:35:22.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.730+0000 7fc57f121640 1 -- start start 2023-10-21T17:35:22.731 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.731+0000 7fd5f1e41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:35:22.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.731+0000 7fc57f121640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780cf2b0 0x7fc5780d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.731+0000 7fc57f121640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5780ce510 0x7fc57815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.731+0000 7fc57f121640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5780cd520 con 0x7fc5780cf690 2023-10-21T17:35:22.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.731+0000 7fc57f121640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5780d1910 con 0x7fc5780ce510 2023-10-21T17:35:22.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.731+0000 7fc57f121640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ce080 con 0x7fc5780cf2b0 2023-10-21T17:35:22.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57d91e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc5780cf690 0x7fc5780d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52880/0 (socket says 172.21.15.53:52880) 2023-10-21T17:35:22.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57e920640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5780ce510 0x7fc57815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57d91e640 1 -- 172.21.15.53:0/680794652 learned_addr learned my addr 172.21.15.53:0/680794652 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:22.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57e11f640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780cf2b0 0x7fc5780d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57d11d640 1 -- 172.21.15.53:0/680794652 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343813122 0 0) 0x7fc5780cd520 con 0x7fc5780cf690 2023-10-21T17:35:22.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57d11d640 1 -- 172.21.15.53:0/680794652 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc564003520 con 0x7fc5780cf690 2023-10-21T17:35:22.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57e920640 1 -- 172.21.15.53:0/680794652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780cf2b0 msgr2=0x7fc5780d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:22.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.732+0000 7fc57e920640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780cf2b0 0x7fc5780d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.733+0000 7fc57e920640 1 -- 172.21.15.53:0/680794652 >> v1:172.21.15.53:6789/0 conn(0x7fc5780cf690 legacy=0x7fc5780d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.733+0000 7fc57e920640 1 -- 172.21.15.53:0/680794652 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780cd750 con 0x7fc5780ce510 2023-10-21T17:35:22.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.734+0000 7fc57e920640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780ce510 0x7fc57815c820 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fc570009d90 tx=0x7fc57002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db15b6ae70c58b28 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:22.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.735+0000 7fc57d11d640 1 -- 172.21.15.53:0/680794652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57002cbd0 con 0x7fc5780ce510 2023-10-21T17:35:22.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.735+0000 7fc57d11d640 1 -- 172.21.15.53:0/680794652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57002cd30 con 0x7fc5780ce510 2023-10-21T17:35:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.735+0000 7fc57d11d640 1 -- 172.21.15.53:0/680794652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57002cea0 con 0x7fc5780ce510 2023-10-21T17:35:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.735+0000 7fc57f121640 1 -- 172.21.15.53:0/680794652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780ce510 msgr2=0x7fc57815c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.735+0000 7fc57f121640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780ce510 0x7fc57815c820 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fc570009d90 tx=0x7fc57002be20 comp rx=0 tx=0).stop 2023-10-21T17:35:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 -- 172.21.15.53:0/680794652 shutdown_connections 2023-10-21T17:35:22.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780ce510 0x7fc57815c820 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 --2- 172.21.15.53:0/680794652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780cf2b0 0x7fc5780d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 -- 172.21.15.53:0/680794652 >> 172.21.15.53:0/680794652 conn(0x7fc57809bc40 msgr2=0x7fc5780d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:22.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 -- 172.21.15.53:0/680794652 shutdown_connections 2023-10-21T17:35:22.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.736+0000 7fc57f121640 1 -- 172.21.15.53:0/680794652 wait complete. 2023-10-21T17:35:22.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.738+0000 7fc57f121640 1 Processor -- start 2023-10-21T17:35:22.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.738+0000 7fc57f121640 1 -- start start 2023-10-21T17:35:22.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780ce510 0x7fc5781f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5780cf690 0x7fc5780cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780cacf0 con 0x7fc5780cf2b0 2023-10-21T17:35:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ca070 con 0x7fc5780cf690 2023-10-21T17:35:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57f121640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ca350 con 0x7fc5780ce510 2023-10-21T17:35:22.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57d91e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57d91e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56974/0 (socket says 172.21.15.53:56974) 2023-10-21T17:35:22.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.740+0000 7fc57d91e640 1 -- 172.21.15.53:0/3902111591 learned_addr learned my addr 172.21.15.53:0/3902111591 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:22.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57e920640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5780cf690 0x7fc5780cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57e11f640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780ce510 0x7fc5781f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 -- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780ce510 msgr2=0x7fc5781f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780ce510 0x7fc5781f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 -- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5780cf690 msgr2=0x7fc5780cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5780cf690 0x7fc5780cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 -- 172.21.15.53:0/3902111591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009a70 con 0x7fc5780cf2b0 2023-10-21T17:35:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57d91e640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c390 tx=0x7fc56800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568005ce0 con 0x7fc5780cf2b0 2023-10-21T17:35:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc568005e40 con 0x7fc5780cf2b0 2023-10-21T17:35:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680072f0 con 0x7fc5780cf2b0 2023-10-21T17:35:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.741+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5780ca630 con 0x7fc5780cf2b0 2023-10-21T17:35:22.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.742+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5780cc840 con 0x7fc5780cf2b0 2023-10-21T17:35:22.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.743+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5780cae30 con 0x7fc5780cf2b0 2023-10-21T17:35:22.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.743+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fc568007450 con 0x7fc5780cf2b0 2023-10-21T17:35:22.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.744+0000 7fc56effd640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 0x7fc55c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:22.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.744+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fc568089270 con 0x7fc5780cf2b0 2023-10-21T17:35:22.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.745+0000 7fc57e11f640 1 -- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 msgr2=0x7fc55c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:22.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.745+0000 7fc57e11f640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 0x7fc55c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:22.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.749+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5780cae30 con 0x7fc5780cf2b0 2023-10-21T17:35:22.796 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.796+0000 7fd5f1e41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:22.804 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.803+0000 7fcf23b7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:35:22.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.845+0000 7fd5f1e41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:35:22.870 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.869+0000 7fcf23b7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:22.902 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.901+0000 7fd5f1e41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:35:22.922 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.920+0000 7fcf23b7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:35:22.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.938+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5780cae30 con 0x7fc5780cf2b0 2023-10-21T17:35:22.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.938+0000 7fc56effd640 1 -- 172.21.15.53:0/3902111591 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v201) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5780cae30 con 0x7fc5780cf2b0 2023-10-21T17:35:22.941 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:35:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 msgr2=0x7fc55c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:22.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 0x7fc55c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 msgr2=0x7fc5780c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c390 tx=0x7fc56800c850 comp rx=0 tx=0).stop 2023-10-21T17:35:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 shutdown_connections 2023-10-21T17:35:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fc55c06af90 0x7fc55c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5780cf690 0x7fc5780cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5780cf2b0 0x7fc5780c99e0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 --2- 172.21.15.53:0/3902111591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5780ce510 0x7fc5781f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 >> 172.21.15.53:0/3902111591 conn(0x7fc57809bc40 msgr2=0x7fc5780b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 shutdown_connections 2023-10-21T17:35:22.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:22.941+0000 7fc57f121640 1 -- 172.21.15.53:0/3902111591 wait complete. 2023-10-21T17:35:22.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:22.951+0000 7fd5f1e41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:35:22.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:35:22.985 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:22.983+0000 7fcf23b7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:35:23.038 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.036+0000 7fcf23b7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:35:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.198+0000 7feaed4de640 1 Processor -- start 2023-10-21T17:35:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.198+0000 7feaed4de640 1 -- start start 2023-10-21T17:35:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feaed4de640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feae815af60 0x7feae806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feaed4de640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae80c9840 con 0x7feae815ab80 2023-10-21T17:35:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feaed4de640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae80ca430 con 0x7feae806e960 2023-10-21T17:35:23.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feaed4de640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80c9a90 con 0x7feae815af60 2023-10-21T17:35:23.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feae6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feae815ab80 0x7feae80b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52888/0 (socket says 172.21.15.53:52888) 2023-10-21T17:35:23.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feae6ffd640 1 -- 172.21.15.53:0/2359454882 learned_addr learned my addr 172.21.15.53:0/2359454882 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feae67fc640 1 --2- 172.21.15.53:0/2359454882 >> v2:172.21.15.124:3300/0 conn(0x7feae815af60 0x7feae806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.200+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122648614 0 0) 0x7feae80c9840 con 0x7feae815ab80 2023-10-21T17:35:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead0003520 con 0x7feae815ab80 2023-10-21T17:35:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267625737 0 0) 0x7fead0003520 con 0x7feae815ab80 2023-10-21T17:35:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 >> v2:172.21.15.124:3300/0 conn(0x7feae815af60 msgr2=0x7feae806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 --2- 172.21.15.53:0/2359454882 >> v2:172.21.15.124:3300/0 conn(0x7feae815af60 0x7feae806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 >> v1:172.21.15.124:6789/0 conn(0x7feae806e960 legacy=0x7feae806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80c9d70 con 0x7feae815ab80 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fead8001670 con 0x7feae815ab80 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 >> v1:172.21.15.53:6789/0 conn(0x7feae815ab80 legacy=0x7feae80b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fead00039c0 0x7fead0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7fead0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fead00051d0 0x7fead000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c9a90 con 0x7fead00047c0 2023-10-21T17:35:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead000c120 con 0x7fead00039c0 2023-10-21T17:35:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead000c2e0 con 0x7fead00051d0 2023-10-21T17:35:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.201+0000 7feae67fc640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7fead0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae6ffd640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fead00039c0 0x7fead0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae77fe640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fead00051d0 0x7fead000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae67fc640 1 -- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fead00051d0 msgr2=0x7fead000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae67fc640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fead00051d0 0x7fead000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae67fc640 1 -- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fead00039c0 msgr2=0x7fead0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae67fc640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fead00039c0 0x7fead0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae67fc640 1 -- 172.21.15.53:0/2359454882 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0003520 con 0x7fead00047c0 2023-10-21T17:35:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.202+0000 7feae77fe640 1 -- 172.21.15.53:0/2359454882 reap_dead start 2023-10-21T17:35:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.203+0000 7feae67fc640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7fead0004c40 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7feadc009a90 tx=0x7fead00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd50e9bfc11c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:23.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.203+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02c6b0 con 0x7fead00047c0 2023-10-21T17:35:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.203+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc02c810 con 0x7fead00047c0 2023-10-21T17:35:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.203+0000 7feae5ffb640 1 -- 172.21.15.53:0/2359454882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02cba0 con 0x7fead00047c0 2023-10-21T17:35:23.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 -- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 msgr2=0x7fead0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7fead0004c40 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7feadc009a90 tx=0x7fead00057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 -- 172.21.15.53:0/2359454882 shutdown_connections 2023-10-21T17:35:23.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 --2- 172.21.15.53:0/2359454882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7fead0004c40 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 -- 172.21.15.53:0/2359454882 >> 172.21.15.53:0/2359454882 conn(0x7feae80cba60 msgr2=0x7feae80cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:23.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 -- 172.21.15.53:0/2359454882 shutdown_connections 2023-10-21T17:35:23.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.204+0000 7feaed4de640 1 -- 172.21.15.53:0/2359454882 wait complete. 2023-10-21T17:35:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.206+0000 7feaed4de640 1 Processor -- start 2023-10-21T17:35:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.206+0000 7feaed4de640 1 -- start start 2023-10-21T17:35:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815b360 0x7feae81efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 0x7feae80b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c4ee0 con 0x7fead00047c0 2023-10-21T17:35:23.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae815bb00 con 0x7feae815b360 2023-10-21T17:35:23.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feaed4de640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae81f2a20 con 0x7feae81f3720 2023-10-21T17:35:23.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.207+0000 7feae6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56990/0 (socket says 172.21.15.53:56990) 2023-10-21T17:35:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 -- 172.21.15.53:0/3615275854 learned_addr learned my addr 172.21.15.53:0/3615275854 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae67fc640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815b360 0x7feae81efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae77fe640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 0x7feae80b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 msgr2=0x7feae80b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 0x7feae80b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815b360 msgr2=0x7feae81efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815b360 0x7feae81efde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 -- 172.21.15.53:0/3615275854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae81f2c80 con 0x7fead00047c0 2023-10-21T17:35:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae77fe640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 0x7feae80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:23.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.208+0000 7feae6ffd640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fead80072f0 tx=0x7fead80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.209+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800f940 con 0x7fead00047c0 2023-10-21T17:35:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.209+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead800fac0 con 0x7fead00047c0 2023-10-21T17:35:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.209+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead800fe70 con 0x7fead00047c0 2023-10-21T17:35:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.209+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae815b820 con 0x7fead00047c0 2023-10-21T17:35:23.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.209+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae815c230 con 0x7fead00047c0 2023-10-21T17:35:23.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.210+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae804e160 con 0x7fead00047c0 2023-10-21T17:35:23.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.211+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fead801d460 con 0x7fead00047c0 2023-10-21T17:35:23.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.212+0000 7feacf7fe640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 0x7fead406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.212+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fead8089d90 con 0x7fead00047c0 2023-10-21T17:35:23.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.212+0000 7feae67fc640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 msgr2=0x7fead406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:23.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.213+0000 7feae67fc640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 0x7fead406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:23.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.220+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fead80547e0 con 0x7fead00047c0 2023-10-21T17:35:23.336 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.336+0000 7fd5f1e41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:35:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.414+0000 7feae67fc640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 msgr2=0x7fead406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.414+0000 7feae67fc640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 0x7fead406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.417+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feae80cc7f0 con 0x7fead00047c0 2023-10-21T17:35:23.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.418+0000 7feacf7fe640 1 -- 172.21.15.53:0/3615275854 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v201) v1 ==== 100+0+11 (secure 0 0 0) 0x7fead8057e00 con 0x7fead00047c0 2023-10-21T17:35:23.420 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:35:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 msgr2=0x7fead406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 0x7fead406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 msgr2=0x7feae815bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fead80072f0 tx=0x7fead80077b0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 shutdown_connections 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fead406af90 0x7fead406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae81f3720 0x7feae80b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815b360 0x7feae81efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 --2- 172.21.15.53:0/3615275854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fead00047c0 0x7feae815bcf0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 >> 172.21.15.53:0/3615275854 conn(0x7feae80cba60 msgr2=0x7feae80ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 shutdown_connections 2023-10-21T17:35:23.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.420+0000 7feaed4de640 1 -- 172.21.15.53:0/3615275854 wait complete. 2023-10-21T17:35:23.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:35:23.472 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.470+0000 7fcf23b7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:35:23.497 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.496+0000 7fd5f1e41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:35:23.605 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.604+0000 7fd5f1e41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:35:23.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.637+0000 7fa5bd701640 1 Processor -- start 2023-10-21T17:35:23.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.637+0000 7fa5bd701640 1 -- start start 2023-10-21T17:35:23.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5bd701640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa5b806b430 0x7fa5b80cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5bd701640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa5b8065ab0 0x7fa5b80cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5bd701640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b81554d0 con 0x7fa5b81550f0 2023-10-21T17:35:23.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5bd701640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b80cb2c0 con 0x7fa5b806b430 2023-10-21T17:35:23.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5bd701640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b80ca5f0 con 0x7fa5b8065ab0 2023-10-21T17:35:23.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5b6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa5b81550f0 0x7fa5b815e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:52898/0 (socket says 172.21.15.53:52898) 2023-10-21T17:35:23.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5b6ffd640 1 -- 172.21.15.53:0/2831981964 learned_addr learned my addr 172.21.15.53:0/2831981964 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:23.643 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fcf23b7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:35:23.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2831981964 >> v2:172.21.15.53:3300/0 conn(0x7fa5b806b430 0x7fa5b80cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.639+0000 7fa5b77fe640 1 --2- 172.21.15.53:0/2831981964 >> v2:172.21.15.124:3300/0 conn(0x7fa5b8065ab0 0x7fa5b80cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b5ffb640 1 -- 172.21.15.53:0/2831981964 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795507396 0 0) 0x7fa5b81554d0 con 0x7fa5b81550f0 2023-10-21T17:35:23.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b5ffb640 1 -- 172.21.15.53:0/2831981964 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a0003520 con 0x7fa5b81550f0 2023-10-21T17:35:23.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2831981964 >> v2:172.21.15.124:3300/0 conn(0x7fa5b8065ab0 msgr2=0x7fa5b80cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2831981964 >> v2:172.21.15.124:3300/0 conn(0x7fa5b8065ab0 0x7fa5b80cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2831981964 >> v1:172.21.15.53:6789/0 conn(0x7fa5b81550f0 legacy=0x7fa5b815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.640+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2831981964 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8070e40 con 0x7fa5b806b430 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.641+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2831981964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80cb9d0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80c9860 tx=0x7fa5ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87e53a5f957fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.641+0000 7fa5b5ffb640 1 -- 172.21.15.53:0/2831981964 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac030610 con 0x7fa5b806b430 2023-10-21T17:35:23.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.641+0000 7fa5b5ffb640 1 -- 172.21.15.53:0/2831981964 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac030770 con 0x7fa5b806b430 2023-10-21T17:35:23.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.641+0000 7fa5b5ffb640 1 -- 172.21.15.53:0/2831981964 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac030b00 con 0x7fa5b806b430 2023-10-21T17:35:23.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 -- 172.21.15.53:0/2831981964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 msgr2=0x7fa5b80cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2831981964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80cb9d0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80c9860 tx=0x7fa5ac02b960 comp rx=0 tx=0).stop 2023-10-21T17:35:23.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 -- 172.21.15.53:0/2831981964 shutdown_connections 2023-10-21T17:35:23.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2831981964 >> v2:172.21.15.124:3300/0 conn(0x7fa5b8065ab0 0x7fa5b80cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2831981964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80cb9d0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 -- 172.21.15.53:0/2831981964 >> 172.21.15.53:0/2831981964 conn(0x7fa5b806cb10 msgr2=0x7fa5b806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:23.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 -- 172.21.15.53:0/2831981964 shutdown_connections 2023-10-21T17:35:23.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.642+0000 7fa5bd701640 1 -- 172.21.15.53:0/2831981964 wait complete. 2023-10-21T17:35:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.644+0000 7fa5bd701640 1 Processor -- start 2023-10-21T17:35:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.644+0000 7fa5bd701640 1 -- start start 2023-10-21T17:35:23.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5b8065ab0 0x7fa5b81f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5b81550f0 0x7fa5b80c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b81f5820 con 0x7fa5b806b430 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8159ba0 con 0x7fa5b8065ab0 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5bd701640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b8159e40 con 0x7fa5b81550f0 2023-10-21T17:35:23.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5b67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5b67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:56996/0 (socket says 172.21.15.53:56996) 2023-10-21T17:35:23.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.645+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2344895434 learned_addr learned my addr 172.21.15.53:0/2344895434 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:23.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b6ffd640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5b8065ab0 0x7fa5b81f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b77fe640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5b81550f0 0x7fa5b80c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:23.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5b81550f0 msgr2=0x7fa5b80c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5b81550f0 0x7fa5b80c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5b8065ab0 msgr2=0x7fa5b81f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5b8065ab0 0x7fa5b81f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 -- 172.21.15.53:0/2344895434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac009400 con 0x7fa5b806b430 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa5b67fc640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009890 tx=0x7fa5ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:23.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.646+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac03e040 con 0x7fa5b806b430 2023-10-21T17:35:23.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.647+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac039a40 con 0x7fa5b806b430 2023-10-21T17:35:23.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.647+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac039df0 con 0x7fa5b806b430 2023-10-21T17:35:23.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.647+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b815a120 con 0x7fa5b806b430 2023-10-21T17:35:23.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.647+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b815a6e0 con 0x7fa5b806b430 2023-10-21T17:35:23.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.648+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b804e160 con 0x7fa5b806b430 2023-10-21T17:35:23.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.649+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fa5ac0443f0 con 0x7fa5b806b430 2023-10-21T17:35:23.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.651+0000 7fa59f7fe640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 0x7fa5a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:23.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.651+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fa5ac0ac080 con 0x7fa5b806b430 2023-10-21T17:35:23.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.651+0000 7fa5b6ffd640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 msgr2=0x7fa5a406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:23.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.651+0000 7fa5b6ffd640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 0x7fa5a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:23.656 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.655+0000 7fd5f1e41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:35:23.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.655+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5ac0deca0 con 0x7fa5b806b430 2023-10-21T17:35:23.758 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.756+0000 7fcf23b7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:35:23.764 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.763+0000 7fd5f1e41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:23.811 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.808+0000 7fcf23b7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:35:23.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.852+0000 7fa5b6ffd640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 msgr2=0x7fa5a406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:23.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.852+0000 7fa5b6ffd640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 0x7fa5a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:23.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.855+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5b80b1d10 con 0x7fa5b806b430 2023-10-21T17:35:23.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.856+0000 7fa59f7fe640 1 -- 172.21.15.53:0/2344895434 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v201) v1 ==== 81+0+955 (secure 0 0 0) 0x7fa5ac034080 con 0x7fa5b806b430 2023-10-21T17:35:23.858 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2744557552 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.858 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3194556637 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.858 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1897991612 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4199687649 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3632125819 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3632125819 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3755084077 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3409964486 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.859 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/766800588 2023-10-22T17:30:21.164866+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/1955466907 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/1955466907 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3041665938 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3287676299 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3295254396 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.860 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3702020697 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.861 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1180026628 2023-10-22T17:19:40.971451+0000 2023-10-21T17:35:23.861 INFO:teuthology.orchestra.run.smithi053.stderr:listed 16 entries 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 msgr2=0x7fa5a406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 0x7fa5a406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 msgr2=0x7fa5b80c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009890 tx=0x7fa5ac004730 comp rx=0 tx=0).stop 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 shutdown_connections 2023-10-21T17:35:23.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fa5a406af90 0x7fa5a406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5b81550f0 0x7fa5b80c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5b806b430 0x7fa5b80c61c0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 --2- 172.21.15.53:0/2344895434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5b8065ab0 0x7fa5b81f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 >> 172.21.15.53:0/2344895434 conn(0x7fa5b806cb10 msgr2=0x7fa5b80cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 shutdown_connections 2023-10-21T17:35:23.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:23.858+0000 7fa5bd701640 1 -- 172.21.15.53:0/2344895434 wait complete. 2023-10-21T17:35:23.869 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:23.868+0000 7fd5f1e41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:35:23.879 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:35:23.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:23.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:35:23.925 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:23.923+0000 7fcf23b7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:24.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:24.024+0000 7fd5f1e41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:24.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:35:24.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:24.027 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:35:24.040 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:24.038+0000 7fcf23b7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:35:24.072 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:35:24.072 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:24.073 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:24.073+0000 7fd5f1e41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:24.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:35:24.131 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:24.131 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:24.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:35:24.186 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:35:24.186 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:24.197 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:24.197+0000 7fd5f1e41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:35:24.206 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:24.204+0000 7fcf23b7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:24.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:35:24.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:24.258 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:24.256+0000 7fcf23b7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:24.386 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:24.383+0000 7fcf23b7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:35:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.457+0000 7fdb8b461640 1 Processor -- start 2023-10-21T17:35:24.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.457+0000 7fdb8b461640 1 -- start start 2023-10-21T17:35:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb8b461640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb84157000 0x7fdb840cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb8b461640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdb840cb190 0x7fdb840cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb8b461640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb840adfc0 con 0x7fdb841597d0 2023-10-21T17:35:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb8b461640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840cbb40 con 0x7fdb84157000 2023-10-21T17:35:24.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb8b461640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840ccce0 con 0x7fdb840cb190 2023-10-21T17:35:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb891fd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb84157000 0x7fdb840cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb891fd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb84157000 0x7fdb840cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59214/0 (socket says 172.21.15.53:59214) 2023-10-21T17:35:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb891fd640 1 -- 172.21.15.53:0/492088441 learned_addr learned my addr 172.21.15.53:0/492088441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.459+0000 7fdb899fe640 1 --2- 172.21.15.53:0/492088441 >> v2:172.21.15.124:3300/0 conn(0x7fdb840cb190 0x7fdb840cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.460+0000 7fdb891fd640 1 -- 172.21.15.53:0/492088441 >> v2:172.21.15.124:3300/0 conn(0x7fdb840cb190 msgr2=0x7fdb840cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.460+0000 7fdb891fd640 1 --2- 172.21.15.53:0/492088441 >> v2:172.21.15.124:3300/0 conn(0x7fdb840cb190 0x7fdb840cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.460+0000 7fdb891fd640 1 -- 172.21.15.53:0/492088441 >> v1:172.21.15.124:6789/0 conn(0x7fdb841597d0 legacy=0x7fdb8406ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:24.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.460+0000 7fdb891fd640 1 -- 172.21.15.53:0/492088441 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84159cb0 con 0x7fdb84157000 2023-10-21T17:35:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.461+0000 7fdb891fd640 1 --2- 172.21.15.53:0/492088441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb84157000 0x7fdb840cc7a0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fdb840b1450 tx=0x7fdb8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa7b5465840df5f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.461+0000 7fdb7bfff640 1 -- 172.21.15.53:0/492088441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80031870 con 0x7fdb84157000 2023-10-21T17:35:24.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.461+0000 7fdb7bfff640 1 -- 172.21.15.53:0/492088441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb800319d0 con 0x7fdb84157000 2023-10-21T17:35:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.461+0000 7fdb7bfff640 1 -- 172.21.15.53:0/492088441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80031d60 con 0x7fdb84157000 2023-10-21T17:35:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 -- 172.21.15.53:0/492088441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb84157000 msgr2=0x7fdb840cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 --2- 172.21.15.53:0/492088441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb84157000 0x7fdb840cc7a0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fdb840b1450 tx=0x7fdb8002b910 comp rx=0 tx=0).stop 2023-10-21T17:35:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 -- 172.21.15.53:0/492088441 shutdown_connections 2023-10-21T17:35:24.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 --2- 172.21.15.53:0/492088441 >> v2:172.21.15.124:3300/0 conn(0x7fdb840cb190 0x7fdb840cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 --2- 172.21.15.53:0/492088441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb84157000 0x7fdb840cc7a0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 -- 172.21.15.53:0/492088441 >> 172.21.15.53:0/492088441 conn(0x7fdb8406a380 msgr2=0x7fdb840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:24.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 -- 172.21.15.53:0/492088441 shutdown_connections 2023-10-21T17:35:24.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.462+0000 7fdb8b461640 1 -- 172.21.15.53:0/492088441 wait complete. 2023-10-21T17:35:24.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.463+0000 7fdb8b461640 1 Processor -- start 2023-10-21T17:35:24.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 -- start start 2023-10-21T17:35:24.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 0x7fdb8414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 0x7fdb8414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84135200 con 0x7fdb840cb190 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8414e8a0 con 0x7fdb8414ed30 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.464+0000 7fdb8b461640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8414da10 con 0x7fdb841597d0 2023-10-21T17:35:24.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59220/0 (socket says 172.21.15.53:59220) 2023-10-21T17:35:24.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 -- 172.21.15.53:0/3219176688 learned_addr learned my addr 172.21.15.53:0/3219176688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:24.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb889fc640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 0x7fdb8414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 msgr2=0x7fdb8414d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 0x7fdb8414d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 msgr2=0x7fdb8414f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb899fe640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 0x7fdb8414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 0x7fdb8414f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 -- 172.21.15.53:0/3219176688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800094d0 con 0x7fdb840cb190 2023-10-21T17:35:24.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb889fc640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 0x7fdb8414d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:24.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb899fe640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 0x7fdb8414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:24.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.465+0000 7fdb891fd640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002b950 tx=0x7fdb800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:24.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.466+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8003abe0 con 0x7fdb840cb190 2023-10-21T17:35:24.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.466+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8003ad40 con 0x7fdb840cb190 2023-10-21T17:35:24.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.466+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb80044610 con 0x7fdb840cb190 2023-10-21T17:35:24.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.466+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8414dc90 con 0x7fdb840cb190 2023-10-21T17:35:24.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.466+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8414e250 con 0x7fdb840cb190 2023-10-21T17:35:24.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.467+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb840b1220 con 0x7fdb840cb190 2023-10-21T17:35:24.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.468+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fdb80044770 con 0x7fdb840cb190 2023-10-21T17:35:24.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.469+0000 7fdb79ffb640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 0x7fdb6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.469+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fdb8002d040 con 0x7fdb840cb190 2023-10-21T17:35:24.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.469+0000 7fdb889fc640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 msgr2=0x7fdb6806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:24.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.469+0000 7fdb889fc640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 0x7fdb6806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:24.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.473+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb80076ec0 con 0x7fdb840cb190 2023-10-21T17:35:24.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.670+0000 7fdb889fc640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 msgr2=0x7fdb6806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:24.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.670+0000 7fdb889fc640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 0x7fdb6806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:24.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.701+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb84067490 con 0x7fdb840cb190 2023-10-21T17:35:24.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.702+0000 7fdb79ffb640 1 -- 172.21.15.53:0/3219176688 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fdb80035090 con 0x7fdb840cb190 2023-10-21T17:35:24.705 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:24.705 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:35:24.705 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:35:24.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 msgr2=0x7fdb6806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:24.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 0x7fdb6806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 msgr2=0x7fdb84134cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fdb8002b950 tx=0x7fdb800046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 shutdown_connections 2023-10-21T17:35:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fdb6806af90 0x7fdb6806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb8414ed30 0x7fdb8414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb841597d0 0x7fdb8414d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 --2- 172.21.15.53:0/3219176688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb840cb190 0x7fdb84134cc0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.704+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 >> 172.21.15.53:0/3219176688 conn(0x7fdb8406a380 msgr2=0x7fdb8415bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:24.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.705+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 shutdown_connections 2023-10-21T17:35:24.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.705+0000 7fdb8b461640 1 -- 172.21.15.53:0/3219176688 wait complete. 2023-10-21T17:35:24.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.967+0000 7fe7a9d7c640 1 Processor -- start 2023-10-21T17:35:24.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.968+0000 7fe7a9d7c640 1 -- start start 2023-10-21T17:35:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe7a9d7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7a40c90b0 0x7fe7a40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe7a9d7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40cb3b0 0x7fe7a415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe7a9d7c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a415b2b0 con 0x7fe7a415ab40 2023-10-21T17:35:24.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe7a9d7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a415bf30 con 0x7fe7a40c90b0 2023-10-21T17:35:24.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe7a9d7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40cb820 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe79affd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7a40c90b0 0x7fe7a40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe79affd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7a40c90b0 0x7fe7a40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:44824/0 (socket says 172.21.15.53:44824) 2023-10-21T17:35:24.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.969+0000 7fe79affd640 1 -- 172.21.15.53:0/3874051607 learned_addr learned my addr 172.21.15.53:0/3874051607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:24.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.970+0000 7fe7a3fff640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40cb3b0 0x7fe7a415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.970+0000 7fe79affd640 1 -- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40cb3b0 msgr2=0x7fe7a415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:24.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.970+0000 7fe79affd640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40cb3b0 0x7fe7a415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.970+0000 7fe79affd640 1 -- 172.21.15.53:0/3874051607 >> v1:172.21.15.124:6789/0 conn(0x7fe7a415ab40 legacy=0x7fe7a40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:24.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.970+0000 7fe79affd640 1 -- 172.21.15.53:0/3874051607 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a415b590 con 0x7fe7a40c90b0 2023-10-21T17:35:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.971+0000 7fe79affd640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a40c90b0 0x7fe7a40cae70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe7a406e560 tx=0x7fe78c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3c9e45cc89b91 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a2ffd640 1 -- 172.21.15.53:0/3874051607 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c02d040 con 0x7fe7a40c90b0 2023-10-21T17:35:24.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a2ffd640 1 -- 172.21.15.53:0/3874051607 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c031810 con 0x7fe7a40c90b0 2023-10-21T17:35:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a2ffd640 1 -- 172.21.15.53:0/3874051607 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c031ba0 con 0x7fe7a40c90b0 2023-10-21T17:35:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a40c90b0 msgr2=0x7fe7a40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a40c90b0 0x7fe7a40cae70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe7a406e560 tx=0x7fe78c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3874051607 shutdown_connections 2023-10-21T17:35:24.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40cb3b0 0x7fe7a415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3874051607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a40c90b0 0x7fe7a40cae70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3874051607 >> 172.21.15.53:0/3874051607 conn(0x7fe7a406a380 msgr2=0x7fe7a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3874051607 shutdown_connections 2023-10-21T17:35:24.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.972+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3874051607 wait complete. 2023-10-21T17:35:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.974+0000 7fe7a9d7c640 1 Processor -- start 2023-10-21T17:35:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 -- start start 2023-10-21T17:35:24.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40c90b0 0x7fe7a40ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a415ab40 0x7fe7a415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c42a0 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.975+0000 7fe7a9d7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c3620 con 0x7fe7a415ab40 2023-10-21T17:35:24.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe7a9d7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c3900 con 0x7fe7a40c90b0 2023-10-21T17:35:24.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59234/0 (socket says 172.21.15.53:59234) 2023-10-21T17:35:24.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 -- 172.21.15.53:0/3618509858 learned_addr learned my addr 172.21.15.53:0/3618509858 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:24.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe7a37fe640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40c90b0 0x7fe7a40ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe7a3fff640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a415ab40 0x7fe7a415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:24.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40c90b0 msgr2=0x7fe7a40ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40c90b0 0x7fe7a40ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a415ab40 msgr2=0x7fe7a415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:24.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a415ab40 0x7fe7a415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.976+0000 7fe79affd640 1 -- 172.21.15.53:0/3618509858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c0094d0 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe79affd640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fe78c02b8f0 tx=0x7fe78c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c031e40 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c03ad60 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c044610 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a40c3be0 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.977+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a40ad640 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.978+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a40c43e0 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.980+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe78c04d450 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.981+0000 7fe7a0ff9640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 0x7fe78406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:24.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.981+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe78c0abec0 con 0x7fe7a40cb3b0 2023-10-21T17:35:24.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.981+0000 7fe7a37fe640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 msgr2=0x7fe78406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:24.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.981+0000 7fe7a37fe640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 0x7fe78406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:24.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:24.985+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7a40c43e0 con 0x7fe7a40cb3b0 2023-10-21T17:35:25.041 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.040+0000 7fd5f1e41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:35:25.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.179+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7a40c43e0 con 0x7fe7a40cb3b0 2023-10-21T17:35:25.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.181+0000 7fe7a0ff9640 1 -- 172.21.15.53:0/3618509858 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+18047 (secure 0 0 0) 0x7fe7a40c43e0 con 0x7fe7a40cb3b0 2023-10-21T17:35:25.184 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:25.184 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":201,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:22.260735+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":201,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2744557552":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3194556637":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/1897991612":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/4199687649":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3755084077":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3409964486":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/766800588":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:6832/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3041665938":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3287676299":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3295254396":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3702020697":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/1180026628":"2023-10-22T17:19:40.971451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:25.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.182+0000 7fe7a37fe640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 msgr2=0x7fe78406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:25.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.182+0000 7fe7a37fe640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 0x7fe78406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:25.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 msgr2=0x7fe78406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:25.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 0x7fe78406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 msgr2=0x7fe7a40c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fe78c02b8f0 tx=0x7fe78c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 shutdown_connections 2023-10-21T17:35:25.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe78406af90 0x7fe78406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7a415ab40 0x7fe7a415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7a40cb3b0 0x7fe7a40c46c0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 --2- 172.21.15.53:0/3618509858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7a40c90b0 0x7fe7a40ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 >> 172.21.15.53:0/3618509858 conn(0x7fe7a406a380 msgr2=0x7fe7a406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:25.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 shutdown_connections 2023-10-21T17:35:25.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.184+0000 7fe7a9d7c640 1 -- 172.21.15.53:0/3618509858 wait complete. 2023-10-21T17:35:25.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:19:23.993283+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:19:25.097499+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 201, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:35:25.205 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:35:25.243 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:25.241+0000 7fcf23b7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:35:25.336 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.336+0000 7fd5f1e41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:35:25.392 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.392+0000 7fd5f1e41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:35:25.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.420+0000 7fe8032e0640 1 Processor -- start 2023-10-21T17:35:25.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.420+0000 7fe8032e0640 1 -- start start 2023-10-21T17:35:25.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe8032e0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7fc0702d0 0x7fe7fc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe8032e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7fc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe8032e0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc0b30a0 con 0x7fe7fc0706b0 2023-10-21T17:35:25.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe8032e0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc15dcf0 con 0x7fe7fc0702d0 2023-10-21T17:35:25.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe8032e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b3320 con 0x7fe7fc0b1750 2023-10-21T17:35:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe80187d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe7fc0706b0 0x7fe7fc0c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38472/0 (socket says 172.21.15.53:38472) 2023-10-21T17:35:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe80187d640 1 -- 172.21.15.53:0/691993121 learned_addr learned my addr 172.21.15.53:0/691993121 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe80087b640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7fc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.422+0000 7fe80107c640 1 --2- 172.21.15.53:0/691993121 >> v2:172.21.15.124:3300/0 conn(0x7fe7fc0702d0 0x7fe7fc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433460196 0 0) 0x7fe7fc0b30a0 con 0x7fe7fc0706b0 2023-10-21T17:35:25.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e8003520 con 0x7fe7fc0706b0 2023-10-21T17:35:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2395036108 0 0) 0x7fe7e8003520 con 0x7fe7fc0706b0 2023-10-21T17:35:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 msgr2=0x7fe7fc0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7fc0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 >> v2:172.21.15.124:3300/0 conn(0x7fe7fc0702d0 msgr2=0x7fe7fc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 --2- 172.21.15.53:0/691993121 >> v2:172.21.15.124:3300/0 conn(0x7fe7fc0702d0 0x7fe7fc15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0b3600 con 0x7fe7fc0706b0 2023-10-21T17:35:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe7f40019f0 con 0x7fe7fc0706b0 2023-10-21T17:35:25.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 >> v1:172.21.15.53:6789/0 conn(0x7fe7fc0706b0 legacy=0x7fe7fc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7e80047c0 0x7fe7e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15dcf0 con 0x7fe7e8003970 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b3320 con 0x7fe7e80047c0 2023-10-21T17:35:25.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e800b9d0 con 0x7fe7fc0b1750 2023-10-21T17:35:25.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.423+0000 7fe80107c640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80087b640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7e80047c0 0x7fe7e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80187d640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80107c640 1 -- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 msgr2=0x7fe7e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80107c640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80107c640 1 -- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7e80047c0 msgr2=0x7fe7e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80107c640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7e80047c0 0x7fe7e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80107c640 1 -- 172.21.15.53:0/691993121 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8003520 con 0x7fe7e8003970 2023-10-21T17:35:25.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.424+0000 7fe80187d640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7fc0b1750 0x7fe7e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:25.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.425+0000 7fe80107c640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009af0 tx=0x7fe7e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993646748fc063e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.425+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f802c710 con 0x7fe7e8003970 2023-10-21T17:35:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.425+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f802c870 con 0x7fe7e8003970 2023-10-21T17:35:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.425+0000 7fe7f3fff640 1 -- 172.21.15.53:0/691993121 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f802cc00 con 0x7fe7e8003970 2023-10-21T17:35:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 -- 172.21.15.53:0/691993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7e8003970 msgr2=0x7fe7e8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 --2- 172.21.15.53:0/691993121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7e8003970 0x7fe7e8003d90 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009af0 tx=0x7fe7e80057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe80187d640 1 -- 172.21.15.53:0/691993121 reap_dead start 2023-10-21T17:35:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 -- 172.21.15.53:0/691993121 shutdown_connections 2023-10-21T17:35:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 -- 172.21.15.53:0/691993121 >> 172.21.15.53:0/691993121 conn(0x7fe7fc069d50 msgr2=0x7fe7fc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 -- 172.21.15.53:0/691993121 shutdown_connections 2023-10-21T17:35:25.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.426+0000 7fe8032e0640 1 -- 172.21.15.53:0/691993121 wait complete. 2023-10-21T17:35:25.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.428+0000 7fe8032e0640 1 Processor -- start 2023-10-21T17:35:25.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.428+0000 7fe8032e0640 1 -- start start 2023-10-21T17:35:25.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7e8003970 0x7fe7fc13e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7fc15ac80 0x7fe7fc1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc13e930 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15b100 con 0x7fe7fc15ac80 2023-10-21T17:35:25.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe8032e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15b3e0 con 0x7fe7e8003970 2023-10-21T17:35:25.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80187d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80187d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59254/0 (socket says 172.21.15.53:59254) 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80187d640 1 -- 172.21.15.53:0/1275387090 learned_addr learned my addr 172.21.15.53:0/1275387090 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80107c640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7e8003970 0x7fe7fc13e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80187d640 1 -- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7e8003970 msgr2=0x7fe7fc13e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.429+0000 7fe80187d640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7e8003970 0x7fe7fc13e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe80187d640 1 -- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7fc15ac80 msgr2=0x7fe7fc1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe80187d640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7fc15ac80 0x7fe7fc1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe80187d640 1 -- 172.21.15.53:0/1275387090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc15b6c0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe80187d640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc070280 tx=0x7fe7f400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004d60 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400bb80 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc15b9a0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4007070 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.430+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc159630 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.431+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc15bec0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.432+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 110) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe7f40073e0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.433+0000 7fe7f1ffb640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 0x7fe7e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.433+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6905+0+0 (secure 0 0 0) 0x7fe7f408a410 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.434+0000 7fe80107c640 1 -- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 msgr2=0x7fe7e006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2750944240 2023-10-21T17:35:25.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.434+0000 7fe80107c640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 0x7fe7e006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:25.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.437+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7fc15bec0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.507 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.507+0000 7fd5f1e41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:35:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.528+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 111) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe7f4055120 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.528+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 msgr2=0x7fe7e006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:25.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.528+0000 7fe7f1ffb640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 0x7fe7e006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.569 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:25.567+0000 7fcf23b7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:35:25.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5771ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5771ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5771ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5771ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5771ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5741b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5741b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5741b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5741b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:25.592+0000 7fd5741b4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:25.631 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:25.629+0000 7fcf23b7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:35:25.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.638+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7fc15bec0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.639+0000 7fe7f1ffb640 1 -- 172.21.15.53:0/1275387090 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7fc15bec0 con 0x7fe7fc15c1a0 2023-10-21T17:35:25.641 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:35:25.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.641+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 msgr2=0x7fe7fc0cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.641+0000 7fe8032e0640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc070280 tx=0x7fe7f400b4a0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 shutdown_connections 2023-10-21T17:35:25.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:6832/2750944240,v1:172.21.15.124:6833/2750944240] conn(0x7fe7e006af90 0x7fe7e006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7fc15c1a0 0x7fe7fc0cd130 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe7fc15ac80 0x7fe7fc1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 --2- 172.21.15.53:0/1275387090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7e8003970 0x7fe7fc13e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 >> 172.21.15.53:0/1275387090 conn(0x7fe7fc069d50 msgr2=0x7fe7fc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:25.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 shutdown_connections 2023-10-21T17:35:25.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.642+0000 7fe8032e0640 1 -- 172.21.15.53:0/1275387090 wait complete. 2023-10-21T17:35:25.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:35:25.755 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:25.753+0000 7fcf23b7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:35:25.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.926+0000 7fd5439e9640 1 Processor -- start 2023-10-21T17:35:25.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.927+0000 7fd5439e9640 1 -- start start 2023-10-21T17:35:25.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.928+0000 7fd5439e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd53c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.928+0000 7fd5439e9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53c15b310 con 0x7fd53c06c3a0 2023-10-21T17:35:25.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.928+0000 7fd5439e9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53c06d290 con 0x7fd53c15aa00 2023-10-21T17:35:25.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.928+0000 7fd5439e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c06e0a0 con 0x7fd53c0c9cf0 2023-10-21T17:35:25.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.929+0000 7fd541785640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd53c06c3a0 0x7fd53c06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38482/0 (socket says 172.21.15.53:38482) 2023-10-21T17:35:25.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.929+0000 7fd541785640 1 -- 172.21.15.53:0/2993777228 learned_addr learned my addr 172.21.15.53:0/2993777228 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:25.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.929+0000 7fd540f84640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd53c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.929+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082098479 0 0) 0x7fd538001110 con 0x7fd53c06c3a0 2023-10-21T17:35:25.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.929+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd528005730 con 0x7fd53c06c3a0 2023-10-21T17:35:25.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3062120897 0 0) 0x7fd528005730 con 0x7fd53c06c3a0 2023-10-21T17:35:25.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 msgr2=0x7fd53c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd53c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 >> v1:172.21.15.124:6789/0 conn(0x7fd53c15aa00 legacy=0x7fd53c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c0c9860 con 0x7fd53c06c3a0 2023-10-21T17:35:25.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd538001800 con 0x7fd53c06c3a0 2023-10-21T17:35:25.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 >> v1:172.21.15.53:6789/0 conn(0x7fd53c06c3a0 legacy=0x7fd53c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528005bb0 0x7fd528005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd5280069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd528006f10 0x7fd528007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c06e0a0 con 0x7fd528006f10 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52800dc70 con 0x7fd528005bb0 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52800de30 con 0x7fd53c0c9cf0 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd541f86640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd528006f10 0x7fd528007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd540f84640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd5280069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.930+0000 7fd541785640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528005bb0 0x7fd528005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.931+0000 7fd541f86640 1 -- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 msgr2=0x7fd5280069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.931+0000 7fd541f86640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c0c9cf0 0x7fd5280069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.931+0000 7fd541f86640 1 -- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528005bb0 msgr2=0x7fd528005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.931+0000 7fd541f86640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528005bb0 0x7fd528005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.931+0000 7fd541f86640 1 -- 172.21.15.53:0/2993777228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd528005730 con 0x7fd528006f10 2023-10-21T17:35:25.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.932+0000 7fd541f86640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd528006f10 0x7fd528007390 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fd534001e20 tx=0x7fd528007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68287d06d3b02a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.932+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534003b70 con 0x7fd528006f10 2023-10-21T17:35:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.932+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534003cd0 con 0x7fd528006f10 2023-10-21T17:35:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.932+0000 7fd533fff640 1 -- 172.21.15.53:0/2993777228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534005310 con 0x7fd528006f10 2023-10-21T17:35:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 -- 172.21.15.53:0/2993777228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd528006f10 msgr2=0x7fd528007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 --2- 172.21.15.53:0/2993777228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd528006f10 0x7fd528007390 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fd534001e20 tx=0x7fd528007a00 comp rx=0 tx=0).stop 2023-10-21T17:35:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd541f86640 1 -- 172.21.15.53:0/2993777228 reap_dead start 2023-10-21T17:35:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 -- 172.21.15.53:0/2993777228 shutdown_connections 2023-10-21T17:35:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 -- 172.21.15.53:0/2993777228 >> 172.21.15.53:0/2993777228 conn(0x7fd53c06a5d0 msgr2=0x7fd53c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 -- 172.21.15.53:0/2993777228 shutdown_connections 2023-10-21T17:35:25.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.933+0000 7fd5439e9640 1 -- 172.21.15.53:0/2993777228 wait complete. 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.935+0000 7fd5439e9640 1 Processor -- start 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.935+0000 7fd5439e9640 1 -- start start 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528006f10 0x7fd53c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c1f5d00 0x7fd53c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:25.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c1530c0 con 0x7fd53c14e380 2023-10-21T17:35:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c14fae0 con 0x7fd528006f10 2023-10-21T17:35:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd5439e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c14ed40 con 0x7fd53c1f5d00 2023-10-21T17:35:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd541f86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd541f86640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59274/0 (socket says 172.21.15.53:59274) 2023-10-21T17:35:25.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd541f86640 1 -- 172.21.15.53:0/3416045513 learned_addr learned my addr 172.21.15.53:0/3416045513 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd541785640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528006f10 0x7fd53c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.936+0000 7fd540f84640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c1f5d00 0x7fd53c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 -- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c1f5d00 msgr2=0x7fd53c1534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c1f5d00 0x7fd53c1534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 -- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528006f10 msgr2=0x7fd53c150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:25.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528006f10 0x7fd53c150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:25.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 -- 172.21.15.53:0/3416045513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c153a20 con 0x7fd53c14e380 2023-10-21T17:35:25.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd541f86640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd534001f40 tx=0x7fd5340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:25.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53402f640 con 0x7fd53c14e380 2023-10-21T17:35:25.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53402f7a0 con 0x7fd53c14e380 2023-10-21T17:35:25.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53402fb30 con 0x7fd53c14e380 2023-10-21T17:35:25.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.937+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c1f6180 con 0x7fd53c14e380 2023-10-21T17:35:25.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.938+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c152720 con 0x7fd53c14e380 2023-10-21T17:35:25.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.939+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c04e160 con 0x7fd53c14e380 2023-10-21T17:35:25.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.940+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 111) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd53402fc90 con 0x7fd53c14e380 2023-10-21T17:35:25.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.941+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7205+0+0 (secure 0 0 0) 0x7fd5340aae50 con 0x7fd53c14e380 2023-10-21T17:35:25.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:25.946+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd534075c80 con 0x7fd53c14e380 2023-10-21T17:35:26.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.141+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd53c06d660 con 0x7fd53c14e380 2023-10-21T17:35:26.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.142+0000 7fd531ffb640 1 -- 172.21.15.53:0/3416045513 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v202) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5340470a0 con 0x7fd53c14e380 2023-10-21T17:35:26.144 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:35:26.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.144+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 msgr2=0x7fd53c14e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.144+0000 7fd5439e9640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd534001f40 tx=0x7fd5340014a0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 shutdown_connections 2023-10-21T17:35:26.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd53c14e380 0x7fd53c14e800 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd53c1f5d00 0x7fd53c1534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 --2- 172.21.15.53:0/3416045513 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd528006f10 0x7fd53c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 >> 172.21.15.53:0/3416045513 conn(0x7fd53c06a5d0 msgr2=0x7fd53c0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:26.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 shutdown_connections 2023-10-21T17:35:26.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.145+0000 7fd5439e9640 1 -- 172.21.15.53:0/3416045513 wait complete. 2023-10-21T17:35:26.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:35:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.390+0000 7f913f440640 1 Processor -- start 2023-10-21T17:35:26.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.390+0000 7f913f440640 1 -- start start 2023-10-21T17:35:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 0x7f91380d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 0x7f91380ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91380d1100 0x7f913815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91380cd520 con 0x7f91380d1100 2023-10-21T17:35:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91380d1910 con 0x7f91380cf690 2023-10-21T17:35:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913f440640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91380ce080 con 0x7f91380cf2b0 2023-10-21T17:35:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913ec3f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91380d1100 0x7f913815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913ec3f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91380d1100 0x7f913815c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59288/0 (socket says 172.21.15.53:59288) 2023-10-21T17:35:26.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913ec3f640 1 -- 172.21.15.53:0/2388860887 learned_addr learned my addr 172.21.15.53:0/2388860887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:26.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913dc3d640 1 --2- 172.21.15.53:0/2388860887 >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 0x7f91380ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.392+0000 7f913e43e640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 0x7f91380d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913ec3f640 1 -- 172.21.15.53:0/2388860887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 msgr2=0x7f91380d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:26.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913ec3f640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 0x7f91380d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913ec3f640 1 -- 172.21.15.53:0/2388860887 >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 msgr2=0x7f91380ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913ec3f640 1 --2- 172.21.15.53:0/2388860887 >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 0x7f91380ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913ec3f640 1 -- 172.21.15.53:0/2388860887 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91380cd750 con 0x7f91380d1100 2023-10-21T17:35:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913e43e640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 0x7f91380d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:26.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.393+0000 7f913dc3d640 1 --2- 172.21.15.53:0/2388860887 >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 0x7f91380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.394+0000 7f913ec3f640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380d1100 0x7f913815c820 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f913808c2b0 tx=0x7f913002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2842c5cdce86c78 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.394+0000 7f913d43c640 1 -- 172.21.15.53:0/2388860887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913002c9f0 con 0x7f91380d1100 2023-10-21T17:35:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.394+0000 7f913d43c640 1 -- 172.21.15.53:0/2388860887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913002cb50 con 0x7f91380d1100 2023-10-21T17:35:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.394+0000 7f913d43c640 1 -- 172.21.15.53:0/2388860887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300343f0 con 0x7f91380d1100 2023-10-21T17:35:26.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 -- 172.21.15.53:0/2388860887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380d1100 msgr2=0x7f913815c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380d1100 0x7f913815c820 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f913808c2b0 tx=0x7f913002be50 comp rx=0 tx=0).stop 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 -- 172.21.15.53:0/2388860887 shutdown_connections 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380d1100 0x7f913815c820 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 --2- 172.21.15.53:0/2388860887 >> v2:172.21.15.124:3300/0 conn(0x7f91380cf690 0x7f91380ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 --2- 172.21.15.53:0/2388860887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380cf2b0 0x7f91380d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 -- 172.21.15.53:0/2388860887 >> 172.21.15.53:0/2388860887 conn(0x7f913809bc40 msgr2=0x7f91380d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 -- 172.21.15.53:0/2388860887 shutdown_connections 2023-10-21T17:35:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.395+0000 7f913f440640 1 -- 172.21.15.53:0/2388860887 wait complete. 2023-10-21T17:35:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.397+0000 7f913f440640 1 Processor -- start 2023-10-21T17:35:26.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.397+0000 7f913f440640 1 -- start start 2023-10-21T17:35:26.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 0x7f913815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380d1100 0x7f91380d2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913809d6a0 con 0x7f91380cf690 2023-10-21T17:35:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913809d980 con 0x7f91380cf2b0 2023-10-21T17:35:26.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.398+0000 7f913f440640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913809dc60 con 0x7f91380d1100 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59296/0 (socket says 172.21.15.53:59296) 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 -- 172.21.15.53:0/4056788433 learned_addr learned my addr 172.21.15.53:0/4056788433 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913e43e640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 0x7f913815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913ec3f640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380d1100 0x7f91380d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 -- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380d1100 msgr2=0x7f91380d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380d1100 0x7f91380d2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 -- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 msgr2=0x7f913815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 0x7f913815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 -- 172.21.15.53:0/4056788433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130009b30 con 0x7f91380cf690 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913e43e640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 0x7f913815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:26.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.399+0000 7f913dc3d640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f912800c7f0 tx=0x7f912800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.400+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007820 con 0x7f91380cf690 2023-10-21T17:35:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.400+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128007980 con 0x7f91380cf690 2023-10-21T17:35:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.400+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913809df40 con 0x7f91380cf690 2023-10-21T17:35:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.400+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128007cf0 con 0x7f91380cf690 2023-10-21T17:35:26.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.400+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913809f430 con 0x7f91380cf690 2023-10-21T17:35:26.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.401+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9138074150 con 0x7f91380cf690 2023-10-21T17:35:26.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.402+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 111) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f91280113f0 con 0x7f91380cf690 2023-10-21T17:35:26.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.403+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f9128089a10 con 0x7f91380cf690 2023-10-21T17:35:26.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.407+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9128054840 con 0x7f91380cf690 2023-10-21T17:35:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.533+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 112) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f91280542f0 con 0x7f91380cf690 2023-10-21T17:35:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.534+0000 7f912effd640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 0x7f911c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.543+0000 7f913e43e640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 0x7f911c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.546+0000 7f913e43e640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 0x7f911c073390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9134006fd0 tx=0x7f9134008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.609+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91380d0070 con 0x7f91380cf690 2023-10-21T17:35:26.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.609+0000 7f912effd640 1 -- 172.21.15.53:0/4056788433 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v202) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91280716d0 con 0x7f91380cf690 2023-10-21T17:35:26.612 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:35:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 msgr2=0x7f911c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 0x7f911c073390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9134006fd0 tx=0x7f9134008040 comp rx=0 tx=0).stop 2023-10-21T17:35:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 msgr2=0x7f91380d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f912800c7f0 tx=0x7f912800ccb0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 shutdown_connections 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f911c070fa0 0x7f911c073390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91380d1100 0x7f91380d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91380cf690 0x7f91380d2430 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 --2- 172.21.15.53:0/4056788433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91380cf2b0 0x7f913815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 >> 172.21.15.53:0/4056788433 conn(0x7f913809bc40 msgr2=0x7f91380c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:26.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.612+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 shutdown_connections 2023-10-21T17:35:26.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.613+0000 7f913f440640 1 -- 172.21.15.53:0/4056788433 wait complete. 2023-10-21T17:35:26.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.859+0000 7f8aea361640 1 Processor -- start 2023-10-21T17:35:26.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.860+0000 7f8aea361640 1 -- start start 2023-10-21T17:35:26.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.861+0000 7f8aea361640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 0x7f8ae40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.861+0000 7f8aea361640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ae40c90b0 0x7f8ae40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.861+0000 7f8aea361640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae415b2b0 con 0x7f8ae40cae70 2023-10-21T17:35:26.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.861+0000 7f8aea361640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae415bf30 con 0x7f8ae40c90b0 2023-10-21T17:35:26.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.861+0000 7f8aea361640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae415f7a0 con 0x7f8ae415ab40 2023-10-21T17:35:26.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ae40c90b0 0x7f8ae40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ae40c90b0 0x7f8ae40adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59308/0 (socket says 172.21.15.53:59308) 2023-10-21T17:35:26.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 -- 172.21.15.53:0/595250021 learned_addr learned my addr 172.21.15.53:0/595250021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae3fff640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 0x7f8ae40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 -- 172.21.15.53:0/595250021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 msgr2=0x7f8ae40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 0x7f8ae40ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 -- 172.21.15.53:0/595250021 >> v1:172.21.15.53:6789/0 conn(0x7f8ae40cae70 legacy=0x7f8ae40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae37fe640 1 -- 172.21.15.53:0/595250021 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae415b590 con 0x7f8ae40c90b0 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.862+0000 7f8ae3fff640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 0x7f8ae40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.863+0000 7f8ae37fe640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae40c90b0 0x7f8ae40adc60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009870 tx=0x7f8ad802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9874e59be43ffe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/595250021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8030840 con 0x7f8ae40c90b0 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/595250021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80309a0 con 0x7f8ae40c90b0 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8ae2ffd640 1 -- 172.21.15.53:0/595250021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8030d30 con 0x7f8ae40c90b0 2023-10-21T17:35:26.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 -- 172.21.15.53:0/595250021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae40c90b0 msgr2=0x7f8ae40adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae40c90b0 0x7f8ae40adc60 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009870 tx=0x7f8ad802b980 comp rx=0 tx=0).stop 2023-10-21T17:35:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 -- 172.21.15.53:0/595250021 shutdown_connections 2023-10-21T17:35:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae40c90b0 0x7f8ae40adc60 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 --2- 172.21.15.53:0/595250021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae415ab40 0x7f8ae40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 -- 172.21.15.53:0/595250021 >> 172.21.15.53:0/595250021 conn(0x7f8ae406a380 msgr2=0x7f8ae40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 -- 172.21.15.53:0/595250021 shutdown_connections 2023-10-21T17:35:26.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.864+0000 7f8aea361640 1 -- 172.21.15.53:0/595250021 wait complete. 2023-10-21T17:35:26.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.866+0000 7f8aea361640 1 Processor -- start 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.866+0000 7f8aea361640 1 -- start start 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ae40c90b0 0x7f8ae4158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cae70 0x7f8ae41f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c86b0 con 0x7f8ae415ab40 2023-10-21T17:35:26.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c5ef0 con 0x7f8ae40c90b0 2023-10-21T17:35:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.867+0000 7f8aea361640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4150c30 con 0x7f8ae40cae70 2023-10-21T17:35:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59310/0 (socket says 172.21.15.53:59310) 2023-10-21T17:35:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 -- 172.21.15.53:0/3000606112 learned_addr learned my addr 172.21.15.53:0/3000606112 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:26.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae37fe640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cae70 0x7f8ae41f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 -- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cae70 msgr2=0x7f8ae41f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cae70 0x7f8ae41f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 -- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ae40c90b0 msgr2=0x7f8ae4158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ae40c90b0 0x7f8ae4158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 -- 172.21.15.53:0/3000606112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80094d0 con 0x7f8ae415ab40 2023-10-21T17:35:26.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.868+0000 7f8ae88fe640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001eb0 tx=0x7f8adc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.869+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0052d0 con 0x7f8ae415ab40 2023-10-21T17:35:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.869+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc005430 con 0x7f8ae415ab40 2023-10-21T17:35:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.869+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc003810 con 0x7f8ae415ab40 2023-10-21T17:35:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.869+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae4150f10 con 0x7f8ae415ab40 2023-10-21T17:35:26.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.869+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae41f0ff0 con 0x7f8ae415ab40 2023-10-21T17:35:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.870+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4151430 con 0x7f8ae415ab40 2023-10-21T17:35:26.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.871+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 112) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8adc003970 con 0x7f8ae415ab40 2023-10-21T17:35:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.872+0000 7f8ae0ff9640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 0x7f8ad006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.872+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f8adc088e80 con 0x7f8ae415ab40 2023-10-21T17:35:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.873+0000 7f8ae3fff640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 0x7f8ad006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:26.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.874+0000 7f8ae3fff640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 0x7f8ad006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ae415a550 tx=0x7f8ad40083d0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:26.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:26.877+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae4151430 con 0x7f8ae415ab40 2023-10-21T17:35:27.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.109+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ae4151430 con 0x7f8ae415ab40 2023-10-21T17:35:27.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.110+0000 7f8ae0ff9640 1 -- 172.21.15.53:0/3000606112 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f8ae4151430 con 0x7f8ae415ab40 2023-10-21T17:35:27.112 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:27.112 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:35:27.113 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:35:27.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 msgr2=0x7f8ad006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 0x7f8ad006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ae415a550 tx=0x7f8ad40083d0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 msgr2=0x7f8ae414fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001eb0 tx=0x7f8adc00c890 comp rx=0 tx=0).stop 2023-10-21T17:35:27.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 shutdown_connections 2023-10-21T17:35:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.112+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8ad006acc0 0x7f8ad006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ae415ab40 0x7f8ae414fe70 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cae70 0x7f8ae41f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 --2- 172.21.15.53:0/3000606112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ae40c90b0 0x7f8ae4158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 >> 172.21.15.53:0/3000606112 conn(0x7f8ae406a380 msgr2=0x7f8ae40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 shutdown_connections 2023-10-21T17:35:27.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.113+0000 7f8aea361640 1 -- 172.21.15.53:0/3000606112 wait complete. 2023-10-21T17:35:27.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:27.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.360+0000 7f9fe93e9640 1 Processor -- start 2023-10-21T17:35:27.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.360+0000 7f9fe93e9640 1 -- start start 2023-10-21T17:35:27.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe40add60 0x7f9fe4162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fe406d4c0 0x7f9fe40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fe40cb150 0x7f9fe40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe4161f10 con 0x7f9fe40cb150 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40cbb20 con 0x7f9fe406d4c0 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.362+0000 7f9fe93e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4161630 con 0x7f9fe40add60 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fe40cb150 0x7f9fe40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9fe40cb150 0x7f9fe40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59316/0 (socket says 172.21.15.53:59316) 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 -- 172.21.15.53:0/2384714197 learned_addr learned my addr 172.21.15.53:0/2384714197 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:27.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe2ffd640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe40add60 0x7f9fe4162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe27fc640 1 --2- 172.21.15.53:0/2384714197 >> v2:172.21.15.124:3300/0 conn(0x7f9fe406d4c0 0x7f9fe40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 -- 172.21.15.53:0/2384714197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe40add60 msgr2=0x7f9fe4162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe40add60 0x7f9fe4162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 -- 172.21.15.53:0/2384714197 >> v2:172.21.15.124:3300/0 conn(0x7f9fe406d4c0 msgr2=0x7f9fe40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 --2- 172.21.15.53:0/2384714197 >> v2:172.21.15.124:3300/0 conn(0x7f9fe406d4c0 0x7f9fe40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.363+0000 7f9fe37fe640 1 -- 172.21.15.53:0/2384714197 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40ae140 con 0x7f9fe40cb150 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.365+0000 7f9fe37fe640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40cb150 0x7f9fe40cb5e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406e470 tx=0x7f9fdc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47b8d9f8532e8a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.365+0000 7f9fe1ffb640 1 -- 172.21.15.53:0/2384714197 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc02c9e0 con 0x7f9fe40cb150 2023-10-21T17:35:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.365+0000 7f9fe1ffb640 1 -- 172.21.15.53:0/2384714197 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc02cb40 con 0x7f9fe40cb150 2023-10-21T17:35:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.365+0000 7f9fe1ffb640 1 -- 172.21.15.53:0/2384714197 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0343f0 con 0x7f9fe40cb150 2023-10-21T17:35:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 -- 172.21.15.53:0/2384714197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40cb150 msgr2=0x7f9fe40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40cb150 0x7f9fe40cb5e0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406e470 tx=0x7f9fdc02be40 comp rx=0 tx=0).stop 2023-10-21T17:35:27.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 -- 172.21.15.53:0/2384714197 shutdown_connections 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40cb150 0x7f9fe40cb5e0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/2384714197 >> v2:172.21.15.124:3300/0 conn(0x7f9fe406d4c0 0x7f9fe40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/2384714197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe40add60 0x7f9fe4162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 -- 172.21.15.53:0/2384714197 >> 172.21.15.53:0/2384714197 conn(0x7f9fe406a380 msgr2=0x7f9fe40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 -- 172.21.15.53:0/2384714197 shutdown_connections 2023-10-21T17:35:27.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.366+0000 7f9fe93e9640 1 -- 172.21.15.53:0/2384714197 wait complete. 2023-10-21T17:35:27.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.368+0000 7f9fe93e9640 1 Processor -- start 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.368+0000 7f9fe93e9640 1 -- start start 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 0x7f9fe41fab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fe40cb150 0x7f9fe40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40b43a0 con 0x7f9fe40add60 2023-10-21T17:35:27.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41fb070 con 0x7f9fe40cb150 2023-10-21T17:35:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe93e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4157df0 con 0x7f9fe406d4c0 2023-10-21T17:35:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59320/0 (socket says 172.21.15.53:59320) 2023-10-21T17:35:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.369+0000 7f9fe27fc640 1 -- 172.21.15.53:0/1312545699 learned_addr learned my addr 172.21.15.53:0/1312545699 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:27.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe2ffd640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 0x7f9fe41fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe37fe640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fe40cb150 0x7f9fe40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 -- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 msgr2=0x7f9fe41fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 0x7f9fe41fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 -- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fe40cb150 msgr2=0x7f9fe40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fe40cb150 0x7f9fe40b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 -- 172.21.15.53:0/1312545699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc009b30 con 0x7f9fe40add60 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe2ffd640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 0x7f9fe41fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe27fc640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800d830 tx=0x7f9fd800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80096a0 con 0x7f9fe40add60 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8009800 con 0x7f9fe40add60 2023-10-21T17:35:27.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8009b90 con 0x7f9fe40add60 2023-10-21T17:35:27.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.370+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe40b40c0 con 0x7f9fe40add60 2023-10-21T17:35:27.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.371+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4158370 con 0x7f9fe40add60 2023-10-21T17:35:27.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.372+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4067490 con 0x7f9fe40add60 2023-10-21T17:35:27.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.373+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 112) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9fd8014410 con 0x7f9fe40add60 2023-10-21T17:35:27.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.374+0000 7f9fcb7fe640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 0x7f9fd006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.374+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f9fd8089f70 con 0x7f9fe40add60 2023-10-21T17:35:27.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.375+0000 7f9fe2ffd640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 0x7f9fd006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.376+0000 7f9fe2ffd640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 0x7f9fd006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4006fd0 tx=0x7f9fd4008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.381+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fd8054cb0 con 0x7f9fe40add60 2023-10-21T17:35:27.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.545+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 113) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9fd8055080 con 0x7f9fe40add60 2023-10-21T17:35:27.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.578+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fe40b4990 con 0x7f9fe40add60 2023-10-21T17:35:27.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.580+0000 7f9fcb7fe640 1 -- 172.21.15.53:0/1312545699 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+18485 (secure 0 0 0) 0x7f9fd80872e0 con 0x7f9fe40add60 2023-10-21T17:35:27.583 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:27.583 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":202,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:25.511480+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":201,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/93601398":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3884449134":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/2744557552":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3194556637":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/1897991612":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/4199687649":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/2953831507":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3755084077":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3409964486":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/380566591":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/766800588":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:6832/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3041665938":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3287676299":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3768127330":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3295254396":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3702020697":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3770250885":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/1180026628":"2023-10-22T17:19:40.971451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:27.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 msgr2=0x7f9fd006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 0x7f9fd006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4006fd0 tx=0x7f9fd4008040 comp rx=0 tx=0).stop 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 msgr2=0x7f9fe41f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800d830 tx=0x7f9fd800dcf0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 shutdown_connections 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9fd006acc0 0x7f9fd006d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fe40cb150 0x7f9fe40b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fe40add60 0x7f9fe41f63e0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 --2- 172.21.15.53:0/1312545699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fe406d4c0 0x7f9fe41fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 >> 172.21.15.53:0/1312545699 conn(0x7f9fe406a380 msgr2=0x7f9fe4160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:27.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 shutdown_connections 2023-10-21T17:35:27.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.584+0000 7f9fe93e9640 1 -- 172.21.15.53:0/1312545699 wait complete. 2023-10-21T17:35:27.604 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:35:27.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:27.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.815+0000 7efda7fff640 1 Processor -- start 2023-10-21T17:35:27.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.816+0000 7efda7fff640 1 -- start start 2023-10-21T17:35:27.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efda815cc60 0x7efda80cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda80d16f0 0x7efda80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda7fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda815d040 con 0x7efda80cdfd0 2023-10-21T17:35:27.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda7fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80d0650 con 0x7efda815cc60 2023-10-21T17:35:27.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80d0930 con 0x7efda80d16f0 2023-10-21T17:35:27.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efda815cc60 0x7efda80cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efda815cc60 0x7efda80cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59336/0 (socket says 172.21.15.53:59336) 2023-10-21T17:35:27.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 -- 172.21.15.53:0/259990500 learned_addr learned my addr 172.21.15.53:0/259990500 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:27.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 -- 172.21.15.53:0/259990500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda80d16f0 msgr2=0x7efda80cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:27.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 --2- 172.21.15.53:0/259990500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda80d16f0 0x7efda80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 -- 172.21.15.53:0/259990500 >> v1:172.21.15.124:6789/0 conn(0x7efda80cdfd0 legacy=0x7efda80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:27.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.817+0000 7efda6ffd640 1 -- 172.21.15.53:0/259990500 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80d0c10 con 0x7efda815cc60 2023-10-21T17:35:27.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.818+0000 7efda6ffd640 1 --2- 172.21.15.53:0/259990500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda815cc60 0x7efda80cf340 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7efd98009860 tx=0x7efd9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bb4cfafe8a132b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.819+0000 7efda67fc640 1 -- 172.21.15.53:0/259990500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98031640 con 0x7efda815cc60 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.819+0000 7efda67fc640 1 -- 172.21.15.53:0/259990500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980317a0 con 0x7efda815cc60 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.819+0000 7efda67fc640 1 -- 172.21.15.53:0/259990500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98031b30 con 0x7efda815cc60 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.819+0000 7efda7fff640 1 -- 172.21.15.53:0/259990500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda815cc60 msgr2=0x7efda80cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.819+0000 7efda7fff640 1 --2- 172.21.15.53:0/259990500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda815cc60 0x7efda80cf340 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7efd98009860 tx=0x7efd9802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 -- 172.21.15.53:0/259990500 shutdown_connections 2023-10-21T17:35:27.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 --2- 172.21.15.53:0/259990500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda80d16f0 0x7efda80cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 --2- 172.21.15.53:0/259990500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda815cc60 0x7efda80cf340 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 -- 172.21.15.53:0/259990500 >> 172.21.15.53:0/259990500 conn(0x7efda809bcf0 msgr2=0x7efda80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:27.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 -- 172.21.15.53:0/259990500 shutdown_connections 2023-10-21T17:35:27.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.820+0000 7efda7fff640 1 -- 172.21.15.53:0/259990500 wait complete. 2023-10-21T17:35:27.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.822+0000 7efda7fff640 1 Processor -- start 2023-10-21T17:35:27.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.822+0000 7efda7fff640 1 -- start start 2023-10-21T17:35:27.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efda80cdfd0 0x7efda8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda815cc60 0x7efda80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda815a070 con 0x7efda80d16f0 2023-10-21T17:35:27.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda815a350 con 0x7efda80cdfd0 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efda7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda815a630 con 0x7efda815cc60 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efd9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efd9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59350/0 (socket says 172.21.15.53:59350) 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.823+0000 7efd9ffff640 1 -- 172.21.15.53:0/4217860973 learned_addr learned my addr 172.21.15.53:0/4217860973 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efda6ffd640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efda80cdfd0 0x7efda8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efda77fe640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda815cc60 0x7efda80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 -- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda815cc60 msgr2=0x7efda80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda815cc60 0x7efda80b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 -- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efda80cdfd0 msgr2=0x7efda8154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efda80cdfd0 0x7efda8154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 -- 172.21.15.53:0/4217860973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009470 con 0x7efda80d16f0 2023-10-21T17:35:27.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.824+0000 7efd9ffff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b410 tx=0x7efd9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.825+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011830 con 0x7efda80d16f0 2023-10-21T17:35:27.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.825+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd900119b0 con 0x7efda80d16f0 2023-10-21T17:35:27.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.825+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011d60 con 0x7efda80d16f0 2023-10-21T17:35:27.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.825+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda815a910 con 0x7efda80d16f0 2023-10-21T17:35:27.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.825+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8155350 con 0x7efda80d16f0 2023-10-21T17:35:27.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.826+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda815ae30 con 0x7efda80d16f0 2023-10-21T17:35:27.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.827+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 113) v1 ==== 89275+0+0 (secure 0 0 0) 0x7efd9001a400 con 0x7efda80d16f0 2023-10-21T17:35:27.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.828+0000 7efda4ff9640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 0x7efd8806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:27.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.828+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 7205+0+0 (secure 0 0 0) 0x7efd90089e50 con 0x7efda80d16f0 2023-10-21T17:35:27.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.828+0000 7efda6ffd640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 0x7efd8806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:27.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.831+0000 7efda6ffd640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 0x7efd8806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd98036000 tx=0x7efd9803c040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:27.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:27.833+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efda815ae30 con 0x7efda80d16f0 2023-10-21T17:35:28.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.065+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda815ae30 con 0x7efda80d16f0 2023-10-21T17:35:28.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.066+0000 7efda4ff9640 1 -- 172.21.15.53:0/4217860973 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7efda815ae30 con 0x7efda80d16f0 2023-10-21T17:35:28.069 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:28.069 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:35:28.070 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:35:28.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 msgr2=0x7efd8806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 0x7efd8806d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd98036000 tx=0x7efd9803c040 comp rx=0 tx=0).stop 2023-10-21T17:35:28.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 msgr2=0x7efda815b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b410 tx=0x7efd9000b8d0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 shutdown_connections 2023-10-21T17:35:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7efd8806af00 0x7efd8806d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efda815cc60 0x7efda80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efda80d16f0 0x7efda815b840 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 --2- 172.21.15.53:0/4217860973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efda80cdfd0 0x7efda8154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 >> 172.21.15.53:0/4217860973 conn(0x7efda809bcf0 msgr2=0x7efda815d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:28.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 shutdown_connections 2023-10-21T17:35:28.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.069+0000 7efda7fff640 1 -- 172.21.15.53:0/4217860973 wait complete. 2023-10-21T17:35:28.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.318+0000 7f1c3afc9640 1 Processor -- start 2023-10-21T17:35:28.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.318+0000 7f1c3afc9640 1 -- start start 2023-10-21T17:35:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c3afc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 0x7f1c340702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c3afc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c3415e330 0x7f1c34070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c3afc9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3406dee0 con 0x7f1c34159ac0 2023-10-21T17:35:28.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c3afc9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3406dcf0 con 0x7f1c3415e330 2023-10-21T17:35:28.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c3afc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3415d850 con 0x7f1c340ad790 2023-10-21T17:35:28.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c3415e330 0x7f1c34070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1c3415e330 0x7f1c34070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59358/0 (socket says 172.21.15.53:59358) 2023-10-21T17:35:28.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 -- 172.21.15.53:0/4111013374 learned_addr learned my addr 172.21.15.53:0/4111013374 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:28.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 -- 172.21.15.53:0/4111013374 >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 msgr2=0x7f1c340702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:28.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c33fff640 1 --2- 172.21.15.53:0/4111013374 >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 0x7f1c340702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 --2- 172.21.15.53:0/4111013374 >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 0x7f1c340702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.320+0000 7f1c39566640 1 -- 172.21.15.53:0/4111013374 >> v1:172.21.15.124:6789/0 conn(0x7f1c34159ac0 legacy=0x7f1c3406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.321+0000 7f1c39566640 1 -- 172.21.15.53:0/4111013374 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3415db30 con 0x7f1c3415e330 2023-10-21T17:35:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.321+0000 7f1c33fff640 1 --2- 172.21.15.53:0/4111013374 >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 0x7f1c340702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:28.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.322+0000 7f1c39566640 1 --2- 172.21.15.53:0/4111013374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c3415e330 0x7f1c34070810 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b4970 tx=0x7f1c2c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbafb5552aadc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.322+0000 7f1c337fe640 1 -- 172.21.15.53:0/4111013374 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c02ca80 con 0x7f1c3415e330 2023-10-21T17:35:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.322+0000 7f1c337fe640 1 -- 172.21.15.53:0/4111013374 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2c02cbe0 con 0x7f1c3415e330 2023-10-21T17:35:28.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.322+0000 7f1c337fe640 1 -- 172.21.15.53:0/4111013374 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0343f0 con 0x7f1c3415e330 2023-10-21T17:35:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 -- 172.21.15.53:0/4111013374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c3415e330 msgr2=0x7f1c34070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/4111013374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c3415e330 0x7f1c34070810 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f1c340b4970 tx=0x7f1c2c02bee0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 -- 172.21.15.53:0/4111013374 shutdown_connections 2023-10-21T17:35:28.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/4111013374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c3415e330 0x7f1c34070810 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/4111013374 >> v2:172.21.15.124:3300/0 conn(0x7f1c340ad790 0x7f1c340702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 -- 172.21.15.53:0/4111013374 >> 172.21.15.53:0/4111013374 conn(0x7f1c3406a380 msgr2=0x7f1c340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 -- 172.21.15.53:0/4111013374 shutdown_connections 2023-10-21T17:35:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.323+0000 7f1c3afc9640 1 -- 172.21.15.53:0/4111013374 wait complete. 2023-10-21T17:35:28.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.325+0000 7f1c3afc9640 1 Processor -- start 2023-10-21T17:35:28.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.325+0000 7f1c3afc9640 1 -- start start 2023-10-21T17:35:28.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c34159ac0 0x7f1c341497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c3415e330 0x7f1c340701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34152930 con 0x7f1c340ad790 2023-10-21T17:35:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34151cb0 con 0x7f1c34159ac0 2023-10-21T17:35:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.326+0000 7f1c3afc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34151f90 con 0x7f1c3415e330 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59364/0 (socket says 172.21.15.53:59364) 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 -- 172.21.15.53:0/1498303833 learned_addr learned my addr 172.21.15.53:0/1498303833 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c39566640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c3415e330 0x7f1c340701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c33fff640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c34159ac0 0x7f1c341497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 -- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c3415e330 msgr2=0x7f1c340701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c3415e330 0x7f1c340701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 -- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c34159ac0 msgr2=0x7f1c341497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c34159ac0 0x7f1c341497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.327+0000 7f1c38d65640 1 -- 172.21.15.53:0/1498303833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c009b30 con 0x7f1c340ad790 2023-10-21T17:35:28.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c38d65640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c430 tx=0x7f1c2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28011790 con 0x7f1c340ad790 2023-10-21T17:35:28.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c280118f0 con 0x7f1c340ad790 2023-10-21T17:35:28.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28011ca0 con 0x7f1c340ad790 2023-10-21T17:35:28.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c34152270 con 0x7f1c340ad790 2023-10-21T17:35:28.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.328+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c34149ce0 con 0x7f1c340ad790 2023-10-21T17:35:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.329+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3406db10 con 0x7f1c340ad790 2023-10-21T17:35:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.330+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 113) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1c2801c460 con 0x7f1c340ad790 2023-10-21T17:35:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.331+0000 7f1c317fa640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 0x7f1c1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.331+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f1c28089ac0 con 0x7f1c340ad790 2023-10-21T17:35:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.332+0000 7f1c33fff640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 0x7f1c1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.333+0000 7f1c33fff640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 0x7f1c1806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c340ca700 tx=0x7f1c20008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.335+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c28054630 con 0x7f1c340ad790 2023-10-21T17:35:28.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.573+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c34067490 con 0x7f1c340ad790 2023-10-21T17:35:28.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.574+0000 7f1c317fa640 1 -- 172.21.15.53:0/1498303833 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f1c28057c50 con 0x7f1c340ad790 2023-10-21T17:35:28.576 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:28.576 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":95,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:19:26.079520+0000","modified":"2023-10-21T17:19:29.356039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37331},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37331":{"gid":37331,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2015564828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":2015564828},{"type":"v1","addr":"172.21.15.124:6837","nonce":2015564828}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-21T17:35:28.576 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 95 2023-10-21T17:35:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 msgr2=0x7f1c1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 0x7f1c1806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c340ca700 tx=0x7f1c20008040 comp rx=0 tx=0).stop 2023-10-21T17:35:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 msgr2=0x7f1c3415a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c430 tx=0x7f1c2800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 shutdown_connections 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1c1806af00 0x7f1c1806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c3415e330 0x7f1c340701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c34159ac0 0x7f1c341497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 --2- 172.21.15.53:0/1498303833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c340ad790 0x7f1c3415a570 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 >> 172.21.15.53:0/1498303833 conn(0x7f1c3406a380 msgr2=0x7f1c34156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:28.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 shutdown_connections 2023-10-21T17:35:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.577+0000 7f1c3afc9640 1 -- 172.21.15.53:0/1498303833 wait complete. 2023-10-21T17:35:28.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:28.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.825+0000 7fdce5097640 1 Processor -- start 2023-10-21T17:35:28.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.825+0000 7fdce5097640 1 -- start start 2023-10-21T17:35:28.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdce5097640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdce0157000 0x7fdce00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdce5097640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdce00cb190 0x7fdce00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdce5097640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00adfc0 con 0x7fdce01597d0 2023-10-21T17:35:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdce5097640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00cbb40 con 0x7fdce00cb190 2023-10-21T17:35:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdce5097640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00ccce0 con 0x7fdce0157000 2023-10-21T17:35:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdcd659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdce01597d0 0x7fdce006ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38504/0 (socket says 172.21.15.53:38504) 2023-10-21T17:35:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdcd659c640 1 -- 172.21.15.53:0/3326963904 learned_addr learned my addr 172.21.15.53:0/3326963904 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:28.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.827+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/3326963904 >> v2:172.21.15.53:3300/0 conn(0x7fdce00cb190 0x7fdce00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcded9d640 1 --2- 172.21.15.53:0/3326963904 >> v2:172.21.15.124:3300/0 conn(0x7fdce0157000 0x7fdce00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcde59c640 1 -- 172.21.15.53:0/3326963904 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617768224 0 0) 0x7fdce00adfc0 con 0x7fdce01597d0 2023-10-21T17:35:28.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcde59c640 1 -- 172.21.15.53:0/3326963904 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdccc003520 con 0x7fdce01597d0 2023-10-21T17:35:28.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcdf59e640 1 -- 172.21.15.53:0/3326963904 >> v2:172.21.15.124:3300/0 conn(0x7fdce0157000 msgr2=0x7fdce00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:28.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/3326963904 >> v2:172.21.15.124:3300/0 conn(0x7fdce0157000 0x7fdce00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcdf59e640 1 -- 172.21.15.53:0/3326963904 >> v1:172.21.15.53:6789/0 conn(0x7fdce01597d0 legacy=0x7fdce006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.828+0000 7fdcdf59e640 1 -- 172.21.15.53:0/3326963904 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0159cb0 con 0x7fdce00cb190 2023-10-21T17:35:28.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.829+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/3326963904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce00cb190 0x7fdce00cb600 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdce0071700 tx=0x7fdcd802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9463f3420a93ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.830+0000 7fdcde59c640 1 -- 172.21.15.53:0/3326963904 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd802cc70 con 0x7fdce00cb190 2023-10-21T17:35:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.830+0000 7fdcde59c640 1 -- 172.21.15.53:0/3326963904 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd802cdd0 con 0x7fdce00cb190 2023-10-21T17:35:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.830+0000 7fdcde59c640 1 -- 172.21.15.53:0/3326963904 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8034640 con 0x7fdce00cb190 2023-10-21T17:35:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.830+0000 7fdce5097640 1 -- 172.21.15.53:0/3326963904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce00cb190 msgr2=0x7fdce00cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.830+0000 7fdce5097640 1 --2- 172.21.15.53:0/3326963904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce00cb190 0x7fdce00cb600 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fdce0071700 tx=0x7fdcd802bec0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 -- 172.21.15.53:0/3326963904 shutdown_connections 2023-10-21T17:35:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 --2- 172.21.15.53:0/3326963904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce00cb190 0x7fdce00cb600 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 --2- 172.21.15.53:0/3326963904 >> v2:172.21.15.124:3300/0 conn(0x7fdce0157000 0x7fdce00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 -- 172.21.15.53:0/3326963904 >> 172.21.15.53:0/3326963904 conn(0x7fdce006a380 msgr2=0x7fdce00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:28.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 -- 172.21.15.53:0/3326963904 shutdown_connections 2023-10-21T17:35:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.831+0000 7fdce5097640 1 -- 172.21.15.53:0/3326963904 wait complete. 2023-10-21T17:35:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.832+0000 7fdce5097640 1 Processor -- start 2023-10-21T17:35:28.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.833+0000 7fdce5097640 1 -- start start 2023-10-21T17:35:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdce00cb190 0x7fdce014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdce01597d0 0x7fdce0155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce0156720 con 0x7fdce0156290 2023-10-21T17:35:28.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00b3900 con 0x7fdce01597d0 2023-10-21T17:35:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdce5097640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00b3340 con 0x7fdce00cb190 2023-10-21T17:35:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59384/0 (socket says 172.21.15.53:59384) 2023-10-21T17:35:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 -- 172.21.15.53:0/695630819 learned_addr learned my addr 172.21.15.53:0/695630819 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:28.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcd659c640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdce01597d0 0x7fdce0155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcded9d640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdce00cb190 0x7fdce014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 -- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdce00cb190 msgr2=0x7fdce014a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdce00cb190 0x7fdce014a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 -- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdce01597d0 msgr2=0x7fdce0155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.834+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdce01597d0 0x7fdce0155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:28.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdcdf59e640 1 -- 172.21.15.53:0/695630819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8009a70 con 0x7fdce0156290 2023-10-21T17:35:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdcdf59e640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802c4d0 tx=0x7fdcd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd803e610 con 0x7fdce0156290 2023-10-21T17:35:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd803e770 con 0x7fdce0156290 2023-10-21T17:35:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd803eb20 con 0x7fdce0156290 2023-10-21T17:35:28.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce00b3620 con 0x7fdce0156290 2023-10-21T17:35:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.835+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce014b250 con 0x7fdce0156290 2023-10-21T17:35:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.836+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce0067490 con 0x7fdce0156290 2023-10-21T17:35:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.838+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 113) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdcd803b050 con 0x7fdce0156290 2023-10-21T17:35:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.838+0000 7fdcd7fff640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 0x7fdcc006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:28.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.839+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7fdcd80abcd0 con 0x7fdce0156290 2023-10-21T17:35:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.839+0000 7fdcded9d640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 0x7fdcc006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.840+0000 7fdcded9d640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 0x7fdcc006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006fd0 tx=0x7fdcd0008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:28.843+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcd8076840 con 0x7fdce0156290 2023-10-21T17:35:29.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.038+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdce014b410 con 0x7fdce0156290 2023-10-21T17:35:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.041+0000 7fdcd7fff640 1 -- 172.21.15.53:0/695630819 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+18484 (secure 0 0 0) 0x7fdcd8039090 con 0x7fdce0156290 2023-10-21T17:35:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:29.043 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":203,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:28.234016+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:19:23.993283+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-21T17:19:25.097499+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":201,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/93601398":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3884449134":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/2744557552":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3194556637":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/1897991612":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/4199687649":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/2953831507":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3755084077":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6832/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3409964486":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/380566591":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/766800588":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:6832/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3041665938":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3287676299":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3768127330":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3295254396":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3702020697":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3770250885":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/1180026628":"2023-10-22T17:19:40.971451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":24,"snaps":[{"begin":2,"length":32}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 msgr2=0x7fdcc006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 0x7fdcc006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006fd0 tx=0x7fdcd0008040 comp rx=0 tx=0).stop 2023-10-21T17:35:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 msgr2=0x7fdce00b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802c4d0 tx=0x7fdcd8004a40 comp rx=0 tx=0).stop 2023-10-21T17:35:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 shutdown_connections 2023-10-21T17:35:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fdcc006af00 0x7fdcc006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdce0156290 0x7fdce00b25a0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdce01597d0 0x7fdce0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 --2- 172.21.15.53:0/695630819 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdce00cb190 0x7fdce014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 >> 172.21.15.53:0/695630819 conn(0x7fdce006a380 msgr2=0x7fdce006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 shutdown_connections 2023-10-21T17:35:29.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.044+0000 7fdce5097640 1 -- 172.21.15.53:0/695630819 wait complete. 2023-10-21T17:35:29.065 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:35:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.288+0000 7feaefa4b640 1 Processor -- start 2023-10-21T17:35:29.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.288+0000 7feaefa4b640 1 -- start start 2023-10-21T17:35:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaefa4b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feae80c9cf0 0x7feae806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaefa4b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 0x7feae80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaefa4b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae815b310 con 0x7feae806c3a0 2023-10-21T17:35:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaefa4b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae806e0a0 con 0x7feae80c9cf0 2023-10-21T17:35:29.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaefa4b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae806d290 con 0x7feae815aa00 2023-10-21T17:35:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaecfe6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feae80c9cf0 0x7feae806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaecfe6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feae80c9cf0 0x7feae806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59386/0 (socket says 172.21.15.53:59386) 2023-10-21T17:35:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaecfe6640 1 -- 172.21.15.53:0/2687914164 learned_addr learned my addr 172.21.15.53:0/2687914164 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaedfe8640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 0x7feae80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.290+0000 7feaecfe6640 1 -- 172.21.15.53:0/2687914164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 msgr2=0x7feae80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.291+0000 7feaecfe6640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 0x7feae80b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.291+0000 7feaecfe6640 1 -- 172.21.15.53:0/2687914164 >> v1:172.21.15.53:6789/0 conn(0x7feae806c3a0 legacy=0x7feae806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.291+0000 7feaecfe6640 1 -- 172.21.15.53:0/2687914164 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80c9860 con 0x7feae80c9cf0 2023-10-21T17:35:29.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.291+0000 7feaedfe8640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 0x7feae80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.292+0000 7feaecfe6640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae80c9cf0 0x7feae806db60 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feadc009870 tx=0x7feadc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4358bbd7e13e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.292+0000 7feadbfff640 1 -- 172.21.15.53:0/2687914164 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc030840 con 0x7feae80c9cf0 2023-10-21T17:35:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.292+0000 7feadbfff640 1 -- 172.21.15.53:0/2687914164 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc0309a0 con 0x7feae80c9cf0 2023-10-21T17:35:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.292+0000 7feadbfff640 1 -- 172.21.15.53:0/2687914164 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc030d30 con 0x7feae80c9cf0 2023-10-21T17:35:29.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 -- 172.21.15.53:0/2687914164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae80c9cf0 msgr2=0x7feae806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae80c9cf0 0x7feae806db60 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feadc009870 tx=0x7feadc02b980 comp rx=0 tx=0).stop 2023-10-21T17:35:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 -- 172.21.15.53:0/2687914164 shutdown_connections 2023-10-21T17:35:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae815aa00 0x7feae80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 --2- 172.21.15.53:0/2687914164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae80c9cf0 0x7feae806db60 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 -- 172.21.15.53:0/2687914164 >> 172.21.15.53:0/2687914164 conn(0x7feae806a5d0 msgr2=0x7feae80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 -- 172.21.15.53:0/2687914164 shutdown_connections 2023-10-21T17:35:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.293+0000 7feaefa4b640 1 -- 172.21.15.53:0/2687914164 wait complete. 2023-10-21T17:35:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.295+0000 7feaefa4b640 1 Processor -- start 2023-10-21T17:35:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.295+0000 7feaefa4b640 1 -- start start 2023-10-21T17:35:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae80c9cf0 0x7feae81f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815aa00 0x7feae815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c2e50 con 0x7feae806c3a0 2023-10-21T17:35:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81f25c0 con 0x7feae815aa00 2023-10-21T17:35:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaefa4b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae80c3080 con 0x7feae80c9cf0 2023-10-21T17:35:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaed7e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaed7e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59400/0 (socket says 172.21.15.53:59400) 2023-10-21T17:35:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.296+0000 7feaed7e7640 1 -- 172.21.15.53:0/737314617 learned_addr learned my addr 172.21.15.53:0/737314617 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaecfe6640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae80c9cf0 0x7feae81f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaedfe8640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815aa00 0x7feae815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 -- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae80c9cf0 msgr2=0x7feae81f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae80c9cf0 0x7feae81f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 -- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815aa00 msgr2=0x7feae815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815aa00 0x7feae815ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 -- 172.21.15.53:0/737314617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc0094d0 con 0x7feae806c3a0 2023-10-21T17:35:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7feaed7e7640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7feae400c450 tx=0x7feae400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40052d0 con 0x7feae806c3a0 2023-10-21T17:35:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.297+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4005430 con 0x7feae806c3a0 2023-10-21T17:35:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.298+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4012870 con 0x7feae806c3a0 2023-10-21T17:35:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.298+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae80c3360 con 0x7feae806c3a0 2023-10-21T17:35:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.298+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae80c3920 con 0x7feae806c3a0 2023-10-21T17:35:29.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.299+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae804e160 con 0x7feae806c3a0 2023-10-21T17:35:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.300+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 113) v1 ==== 89275+0+0 (secure 0 0 0) 0x7feae40129d0 con 0x7feae806c3a0 2023-10-21T17:35:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.301+0000 7fead9ffb640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 0x7feacc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.301+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7feae40896d0 con 0x7feae806c3a0 2023-10-21T17:35:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.301+0000 7feaecfe6640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 0x7feacc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.303+0000 7feaecfe6640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 0x7feacc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc008030 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.305+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feae40540b0 con 0x7feae806c3a0 2023-10-21T17:35:29.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.536+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feae815b310 con 0x7feae806c3a0 2023-10-21T17:35:29.538 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:35:29.537+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:35:29.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.541+0000 7fead9ffb640 1 -- 172.21.15.53:0/737314617 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7feae815b310 con 0x7feae806c3a0 2023-10-21T17:35:29.542 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:35:29.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.543+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 msgr2=0x7feacc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.543+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 0x7feacc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc008030 comp rx=0 tx=0).stop 2023-10-21T17:35:29.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.543+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 msgr2=0x7feae80c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.543+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7feae400c450 tx=0x7feae400c910 comp rx=0 tx=0).stop 2023-10-21T17:35:29.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 shutdown_connections 2023-10-21T17:35:29.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7feacc06af00 0x7feacc06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feae815aa00 0x7feae815ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feae80c9cf0 0x7feae81f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 --2- 172.21.15.53:0/737314617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feae806c3a0 0x7feae80c2910 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.544+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 >> 172.21.15.53:0/737314617 conn(0x7feae806a5d0 msgr2=0x7feae8159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:29.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.545+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 shutdown_connections 2023-10-21T17:35:29.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.545+0000 7feaefa4b640 1 -- 172.21.15.53:0/737314617 wait complete. 2023-10-21T17:35:29.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:35:29.677 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:35:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.792+0000 7f9c8d690640 1 Processor -- start 2023-10-21T17:35:29.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.792+0000 7f9c8d690640 1 -- start start 2023-10-21T17:35:29.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.794+0000 7f9c8d690640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c881601b0 0x7f9c880cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.794+0000 7f9c8d690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8806a460 0x7f9c880ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.794+0000 7f9c8d690640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880ce210 con 0x7f9c88071c30 2023-10-21T17:35:29.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.794+0000 7f9c8d690640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880cdee0 con 0x7f9c881601b0 2023-10-21T17:35:29.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.794+0000 7f9c8d690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88161530 con 0x7f9c8806a460 2023-10-21T17:35:29.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c881601b0 0x7f9c880cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c877fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8806a460 0x7f9c880ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c881601b0 0x7f9c880cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:44986/0 (socket says 172.21.15.53:44986) 2023-10-21T17:35:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 -- 172.21.15.53:0/1680624970 learned_addr learned my addr 172.21.15.53:0/1680624970 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 -- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8806a460 msgr2=0x7f9c880ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:29.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 --2- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8806a460 0x7f9c880ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 -- 172.21.15.53:0/1680624970 >> v1:172.21.15.124:6789/0 conn(0x7f9c88071c30 legacy=0x7f9c88069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.795+0000 7f9c86ffd640 1 -- 172.21.15.53:0/1680624970 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88161810 con 0x7f9c881601b0 2023-10-21T17:35:29.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.796+0000 7f9c86ffd640 1 --2- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c881601b0 0x7f9c880cd430 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009e50 tx=0x7f9c7802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5835b6d6fc18403a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.797+0000 7f9c85ffb640 1 -- 172.21.15.53:0/1680624970 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802cb00 con 0x7f9c881601b0 2023-10-21T17:35:29.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.797+0000 7f9c85ffb640 1 -- 172.21.15.53:0/1680624970 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7802cc60 con 0x7f9c881601b0 2023-10-21T17:35:29.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.797+0000 7f9c85ffb640 1 -- 172.21.15.53:0/1680624970 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c780343f0 con 0x7f9c881601b0 2023-10-21T17:35:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 -- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c881601b0 msgr2=0x7f9c880cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 --2- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c881601b0 0x7f9c880cd430 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009e50 tx=0x7f9c7802bf60 comp rx=0 tx=0).stop 2023-10-21T17:35:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 -- 172.21.15.53:0/1680624970 shutdown_connections 2023-10-21T17:35:29.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 --2- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c8806a460 0x7f9c880ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 --2- 172.21.15.53:0/1680624970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c881601b0 0x7f9c880cd430 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 -- 172.21.15.53:0/1680624970 >> 172.21.15.53:0/1680624970 conn(0x7f9c8806d430 msgr2=0x7f9c8806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 -- 172.21.15.53:0/1680624970 shutdown_connections 2023-10-21T17:35:29.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.798+0000 7f9c8d690640 1 -- 172.21.15.53:0/1680624970 wait complete. 2023-10-21T17:35:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.801+0000 7f9c8d690640 1 Processor -- start 2023-10-21T17:35:29.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.801+0000 7f9c8d690640 1 -- start start 2023-10-21T17:35:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.802+0000 7f9c8d690640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8806a460 0x7f9c880ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.802+0000 7f9c8d690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c88071c30 0x7f9c880c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c8d690640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c8d690640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880ad870 con 0x7f9c881601b0 2023-10-21T17:35:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c8d690640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c3c50 con 0x7f9c8806a460 2023-10-21T17:35:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c8d690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c6fc0 con 0x7f9c88071c30 2023-10-21T17:35:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c877fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c877fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59408/0 (socket says 172.21.15.53:59408) 2023-10-21T17:35:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c877fe640 1 -- 172.21.15.53:0/2850302007 learned_addr learned my addr 172.21.15.53:0/2850302007 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c86ffd640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8806a460 0x7f9c880ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.803+0000 7f9c867fc640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c88071c30 0x7f9c880c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 -- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c88071c30 msgr2=0x7f9c880c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c88071c30 0x7f9c880c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 -- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8806a460 msgr2=0x7f9c880ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8806a460 0x7f9c880ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 -- 172.21.15.53:0/2850302007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009b30 con 0x7f9c881601b0 2023-10-21T17:35:29.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c877fe640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000e7f0 tx=0x7f9c8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c800094b0 con 0x7f9c881601b0 2023-10-21T17:35:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000c730 con 0x7f9c881601b0 2023-10-21T17:35:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000cae0 con 0x7f9c881601b0 2023-10-21T17:35:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.804+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880adaa0 con 0x7f9c881601b0 2023-10-21T17:35:29.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.805+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880ae060 con 0x7f9c881601b0 2023-10-21T17:35:29.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.806+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8804e160 con 0x7f9c881601b0 2023-10-21T17:35:29.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.807+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9c8000cc40 con 0x7f9c881601b0 2023-10-21T17:35:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.808+0000 7f9c6f7fe640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 0x7f9c7406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.808+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7263+0+0 (secure 0 0 0) 0x7f9c80089d80 con 0x7f9c881601b0 2023-10-21T17:35:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.808+0000 7f9c86ffd640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 0x7f9c7406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:29.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.810+0000 7f9c86ffd640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 0x7f9c7406d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c78035000 tx=0x7f9c78030000 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:29.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:29.813+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c800547c0 con 0x7f9c881601b0 2023-10-21T17:35:30.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.042+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9c880c7100 con 0x7f9c881601b0 2023-10-21T17:35:30.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.564+0000 7f9c6f7fe640 1 -- 172.21.15.53:0/2850302007 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c80057de0 con 0x7f9c881601b0 2023-10-21T17:35:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.567+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 msgr2=0x7f9c7406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.567+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 0x7f9c7406d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9c78035000 tx=0x7f9c78030000 comp rx=0 tx=0).stop 2023-10-21T17:35:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.567+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 msgr2=0x7f9c881f9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.567+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000e7f0 tx=0x7f9c8000ecb0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 shutdown_connections 2023-10-21T17:35:30.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9c7406af90 0x7f9c7406d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c881601b0 0x7f9c881f9d50 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c88071c30 0x7f9c880c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 --2- 172.21.15.53:0/2850302007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8806a460 0x7f9c880ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 >> 172.21.15.53:0/2850302007 conn(0x7f9c8806d430 msgr2=0x7f9c880b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 shutdown_connections 2023-10-21T17:35:30.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.568+0000 7f9c8d690640 1 -- 172.21.15.53:0/2850302007 wait complete. 2023-10-21T17:35:30.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:35:30.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.817+0000 7fd87f59e640 1 Processor -- start 2023-10-21T17:35:30.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.818+0000 7fd87f59e640 1 -- start start 2023-10-21T17:35:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.819+0000 7fd87f59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd88015a820 0x7fd8800aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.819+0000 7fd87f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 0x7fd8800b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.819+0000 7fd87f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8800cab60 con 0x7fd88015ac00 2023-10-21T17:35:30.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.819+0000 7fd87f59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800cb960 con 0x7fd88015a820 2023-10-21T17:35:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.819+0000 7fd87f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8800cbdf0 con 0x7fd8800ca680 2023-10-21T17:35:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.820+0000 7fd87ed9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd88015ac00 0x7fd8800c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38522/0 (socket says 172.21.15.53:38522) 2023-10-21T17:35:30.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.820+0000 7fd87ed9d640 1 -- 172.21.15.53:0/2204042368 learned_addr learned my addr 172.21.15.53:0/2204042368 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.820+0000 7fd877fff640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 0x7fd8800b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.820+0000 7fd87e59c640 1 --2- 172.21.15.53:0/2204042368 >> v2:172.21.15.124:3300/0 conn(0x7fd88015a820 0x7fd8800aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87dd9b640 1 -- 172.21.15.53:0/2204042368 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798875741 0 0) 0x7fd8800cab60 con 0x7fd88015ac00 2023-10-21T17:35:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87dd9b640 1 -- 172.21.15.53:0/2204042368 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd86c003520 con 0x7fd88015ac00 2023-10-21T17:35:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87e59c640 1 -- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 msgr2=0x7fd8800b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87e59c640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 0x7fd8800b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87e59c640 1 -- 172.21.15.53:0/2204042368 >> v1:172.21.15.53:6789/0 conn(0x7fd88015ac00 legacy=0x7fd8800c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd87e59c640 1 -- 172.21.15.53:0/2204042368 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800cb030 con 0x7fd88015a820 2023-10-21T17:35:30.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.821+0000 7fd877fff640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 0x7fd8800b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:35:30.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.822+0000 7fd87e59c640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd8800aff70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd8700097a0 tx=0x7fd87002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56511e2175df3952 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:30.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.823+0000 7fd87dd9b640 1 -- 172.21.15.53:0/2204042368 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870031780 con 0x7fd88015a820 2023-10-21T17:35:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.823+0000 7fd87dd9b640 1 -- 172.21.15.53:0/2204042368 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8700318e0 con 0x7fd88015a820 2023-10-21T17:35:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.823+0000 7fd87dd9b640 1 -- 172.21.15.53:0/2204042368 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870031c50 con 0x7fd88015a820 2023-10-21T17:35:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.823+0000 7fd87f59e640 1 -- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 msgr2=0x7fd8800aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.823+0000 7fd87f59e640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd8800aff70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd8700097a0 tx=0x7fd87002b820 comp rx=0 tx=0).stop 2023-10-21T17:35:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 -- 172.21.15.53:0/2204042368 shutdown_connections 2023-10-21T17:35:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8800ca680 0x7fd8800b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 --2- 172.21.15.53:0/2204042368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd8800aff70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 -- 172.21.15.53:0/2204042368 >> 172.21.15.53:0/2204042368 conn(0x7fd88006c050 msgr2=0x7fd88006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:30.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 -- 172.21.15.53:0/2204042368 shutdown_connections 2023-10-21T17:35:30.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.824+0000 7fd87f59e640 1 -- 172.21.15.53:0/2204042368 wait complete. 2023-10-21T17:35:30.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.825+0000 7fd87f59e640 1 Processor -- start 2023-10-21T17:35:30.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.826+0000 7fd87f59e640 1 -- start start 2023-10-21T17:35:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd880154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 0x7fd8800b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801f3b50 con 0x7fd8800ca680 2023-10-21T17:35:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801f2ed0 con 0x7fd88015a820 2023-10-21T17:35:30.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801f31b0 con 0x7fd88015ac00 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59412/0 (socket says 172.21.15.53:59412) 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 -- 172.21.15.53:0/55124800 learned_addr learned my addr 172.21.15.53:0/55124800 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87ed9d640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 0x7fd8800b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd877fff640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd880154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 -- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 msgr2=0x7fd8800b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 0x7fd8800b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 -- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 msgr2=0x7fd880154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:30.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd880154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:30.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87e59c640 1 -- 172.21.15.53:0/55124800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009400 con 0x7fd8800ca680 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd877fff640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd880154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.827+0000 7fd87ed9d640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 0x7fd8800b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd87e59c640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fd870034040 tx=0x7fd8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003810 con 0x7fd8800ca680 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd870040070 con 0x7fd8800ca680 2023-10-21T17:35:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870003b70 con 0x7fd8800ca680 2023-10-21T17:35:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd870009400 con 0x7fd8800ca680 2023-10-21T17:35:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.828+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8801f3770 con 0x7fd8800ca680 2023-10-21T17:35:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.829+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880074150 con 0x7fd8800ca680 2023-10-21T17:35:30.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.830+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd870003cd0 con 0x7fd8800ca680 2023-10-21T17:35:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.831+0000 7fd876ffd640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 0x7fd86006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.831+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 7263+0+0 (secure 0 0 0) 0x7fd870042070 con 0x7fd8800ca680 2023-10-21T17:35:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.832+0000 7fd877fff640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 0x7fd86006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.833+0000 7fd877fff640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 0x7fd86006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd880159a60 tx=0x7fd868008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:30.835+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd870076ac0 con 0x7fd8800ca680 2023-10-21T17:35:31.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.028+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8800cb2e0 con 0x7fd8800ca680 2023-10-21T17:35:31.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.582+0000 7fd876ffd640 1 -- 172.21.15.53:0/55124800 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v205) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd87004b020 con 0x7fd8800ca680 2023-10-21T17:35:31.584 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:35:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.590+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 msgr2=0x7fd86006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.590+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 0x7fd86006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd880159a60 tx=0x7fd868008040 comp rx=0 tx=0).stop 2023-10-21T17:35:31.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.590+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 msgr2=0x7fd880151690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.590+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fd870034040 tx=0x7fd8700046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 shutdown_connections 2023-10-21T17:35:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd86006af90 0x7fd86006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd88015ac00 0x7fd8800b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd88015a820 0x7fd880154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 --2- 172.21.15.53:0/55124800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8800ca680 0x7fd880151690 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 >> 172.21.15.53:0/55124800 conn(0x7fd88006c050 msgr2=0x7fd8800acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 shutdown_connections 2023-10-21T17:35:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.591+0000 7fd87f59e640 1 -- 172.21.15.53:0/55124800 wait complete. 2023-10-21T17:35:31.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:35:31.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.837+0000 7f795e3ff640 1 Processor -- start 2023-10-21T17:35:31.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.837+0000 7f795e3ff640 1 -- start start 2023-10-21T17:35:31.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795e3ff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79580ca680 0x7f795815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795e3ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580cab60 con 0x7f795815a820 2023-10-21T17:35:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795e3ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580cb960 con 0x7f79580b0680 2023-10-21T17:35:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795e3ff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79580cbdf0 con 0x7f79580ca680 2023-10-21T17:35:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795d3fd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f795815a820 0x7f79580aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38528/0 (socket says 172.21.15.53:38528) 2023-10-21T17:35:31.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.839+0000 7f795d3fd640 1 -- 172.21.15.53:0/1499133901 learned_addr learned my addr 172.21.15.53:0/1499133901 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:31.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f795dbfe640 1 --2- 172.21.15.53:0/1499133901 >> v2:172.21.15.124:3300/0 conn(0x7f79580ca680 0x7f795815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461777181 0 0) 0x7f79580cab60 con 0x7f795815a820 2023-10-21T17:35:31.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7944003520 con 0x7f795815a820 2023-10-21T17:35:31.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 349043659 0 0) 0x7f7944003520 con 0x7f795815a820 2023-10-21T17:35:31.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 >> v2:172.21.15.124:3300/0 conn(0x7f79580ca680 msgr2=0x7f795815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:31.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 --2- 172.21.15.53:0/1499133901 >> v2:172.21.15.124:3300/0 conn(0x7f79580ca680 0x7f795815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 >> v1:172.21.15.124:6789/0 conn(0x7f79580b0680 legacy=0x7f795815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:31.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580cb030 con 0x7f795815a820 2023-10-21T17:35:31.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7954001670 con 0x7f795815a820 2023-10-21T17:35:31.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 >> v1:172.21.15.53:6789/0 conn(0x7f795815a820 legacy=0x7f79580aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 0x7f7944003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79440047c0 0x7f7944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944005180 0x7f794400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580cbdf0 con 0x7f7944003970 2023-10-21T17:35:31.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794400c0b0 con 0x7f79440047c0 2023-10-21T17:35:31.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.840+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794400c270 con 0x7f7944005180 2023-10-21T17:35:31.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 0x7f7944003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795cbfc640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79440047c0 0x7f7944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 -- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944005180 msgr2=0x7f794400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:31.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944005180 0x7f794400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 -- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79440047c0 msgr2=0x7f7944004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:31.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79440047c0 0x7f7944004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795d3fd640 1 -- 172.21.15.53:0/1499133901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7944003520 con 0x7f7944003970 2023-10-21T17:35:31.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.841+0000 7f795dbfe640 1 -- 172.21.15.53:0/1499133901 reap_dead start 2023-10-21T17:35:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.842+0000 7f795d3fd640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 0x7f7944003d70 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f795400c210 tx=0x7f79440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a67043c78f3e460 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954005ad0 con 0x7f7944003970 2023-10-21T17:35:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954005c50 con 0x7f7944003970 2023-10-21T17:35:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f794ffff640 1 -- 172.21.15.53:0/1499133901 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795402c8a0 con 0x7f7944003970 2023-10-21T17:35:31.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 -- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 msgr2=0x7f7944003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 0x7f7944003d70 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f795400c210 tx=0x7f79440057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 -- 172.21.15.53:0/1499133901 shutdown_connections 2023-10-21T17:35:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 --2- 172.21.15.53:0/1499133901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7944003970 0x7f7944003d70 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 -- 172.21.15.53:0/1499133901 >> 172.21.15.53:0/1499133901 conn(0x7f795806c050 msgr2=0x7f795806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:31.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.843+0000 7f795e3ff640 1 -- 172.21.15.53:0/1499133901 shutdown_connections 2023-10-21T17:35:31.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.844+0000 7f795e3ff640 1 -- 172.21.15.53:0/1499133901 wait complete. 2023-10-21T17:35:31.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.845+0000 7f795e3ff640 1 Processor -- start 2023-10-21T17:35:31.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.845+0000 7f795e3ff640 1 -- start start 2023-10-21T17:35:31.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944003970 0x7f795815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79581f2050 0x7f79581f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79581f6420 con 0x7f795815bad0 2023-10-21T17:35:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79581f5e60 con 0x7f79581f2050 2023-10-21T17:35:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.846+0000 7f795e3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79581f6140 con 0x7f7944003970 2023-10-21T17:35:31.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59428/0 (socket says 172.21.15.53:59428) 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 -- 172.21.15.53:0/3222461212 learned_addr learned my addr 172.21.15.53:0/3222461212 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795cbfc640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79581f2050 0x7f79581f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795d3fd640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944003970 0x7f795815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 -- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944003970 msgr2=0x7f795815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944003970 0x7f795815cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 -- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79581f2050 msgr2=0x7f79581f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79581f2050 0x7f79581f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.847+0000 7f795dbfe640 1 -- 172.21.15.53:0/3222461212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580c7140 con 0x7f795815bad0 2023-10-21T17:35:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f795dbfe640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f79580caa90 tx=0x7f795000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:31.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950010040 con 0x7f795815bad0 2023-10-21T17:35:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795000bce0 con 0x7f795815bad0 2023-10-21T17:35:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79580c7420 con 0x7f795815bad0 2023-10-21T17:35:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950014640 con 0x7f795815bad0 2023-10-21T17:35:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.848+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79581f5aa0 con 0x7f795815bad0 2023-10-21T17:35:31.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.849+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79580c7660 con 0x7f795815bad0 2023-10-21T17:35:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.850+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f79500147a0 con 0x7f795815bad0 2023-10-21T17:35:31.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.851+0000 7f794dffb640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 0x7f793c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.851+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f795008a460 con 0x7f795815bad0 2023-10-21T17:35:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.852+0000 7f795d3fd640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 0x7f793c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.853+0000 7f795d3fd640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 0x7f793c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7954034000 tx=0x7f7954009030 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:31.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:31.855+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79580c7660 con 0x7f795815bad0 2023-10-21T17:35:32.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.048+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79580c7660 con 0x7f795815bad0 2023-10-21T17:35:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.594+0000 7f794dffb640 1 -- 172.21.15.53:0/3222461212 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v206) v1 ==== 169+0+0 (secure 0 0 0) 0x7f79580c7660 con 0x7f795815bad0 2023-10-21T17:35:32.596 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:35:32.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 msgr2=0x7f793c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:32.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 0x7f793c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7954034000 tx=0x7f7954009030 comp rx=0 tx=0).stop 2023-10-21T17:35:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 msgr2=0x7f795815bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f79580caa90 tx=0x7f795000b480 comp rx=0 tx=0).stop 2023-10-21T17:35:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 shutdown_connections 2023-10-21T17:35:32.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f793c06af90 0x7f793c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f795815bad0 0x7f795815bf80 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f79581f2050 0x7f79581f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 --2- 172.21.15.53:0/3222461212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7944003970 0x7f795815cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.598+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 >> 172.21.15.53:0/3222461212 conn(0x7f795806c050 msgr2=0x7f79580af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:32.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.599+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 shutdown_connections 2023-10-21T17:35:32.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.599+0000 7f795e3ff640 1 -- 172.21.15.53:0/3222461212 wait complete. 2023-10-21T17:35:32.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:35:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.835+0000 7fe67eea9640 1 Processor -- start 2023-10-21T17:35:32.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.835+0000 7fe67eea9640 1 -- start start 2023-10-21T17:35:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe67806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 0x7fe6780cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe6780702b0 0x7fe678070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6780cbb10 con 0x7fe6780702b0 2023-10-21T17:35:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67806cc50 con 0x7fe6780cb150 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67eea9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67806cf30 con 0x7fe678161690 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67d446640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe6780702b0 0x7fe678070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67d446640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe6780702b0 0x7fe678070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59436/0 (socket says 172.21.15.53:59436) 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.836+0000 7fe67d446640 1 -- 172.21.15.53:0/1853706285 learned_addr learned my addr 172.21.15.53:0/1853706285 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67cc45640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe67806dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe66ffff640 1 --2- 172.21.15.53:0/1853706285 >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 0x7fe6780cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67d446640 1 -- 172.21.15.53:0/1853706285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 msgr2=0x7fe67806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67d446640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe67806dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67d446640 1 -- 172.21.15.53:0/1853706285 >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 msgr2=0x7fe6780cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67d446640 1 --2- 172.21.15.53:0/1853706285 >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 0x7fe6780cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe67d446640 1 -- 172.21.15.53:0/1853706285 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67806d210 con 0x7fe6780702b0 2023-10-21T17:35:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.837+0000 7fe66ffff640 1 --2- 172.21.15.53:0/1853706285 >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 0x7fe6780cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:35:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.838+0000 7fe67d446640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780702b0 0x7fe678070700 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fe67806e4b0 tx=0x7fe67002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4c99e034e31f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:32.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.838+0000 7fe66f7fe640 1 -- 172.21.15.53:0/1853706285 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67002ca70 con 0x7fe6780702b0 2023-10-21T17:35:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.838+0000 7fe66f7fe640 1 -- 172.21.15.53:0/1853706285 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67002cbd0 con 0x7fe6780702b0 2023-10-21T17:35:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.838+0000 7fe66f7fe640 1 -- 172.21.15.53:0/1853706285 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700343f0 con 0x7fe6780702b0 2023-10-21T17:35:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 -- 172.21.15.53:0/1853706285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780702b0 msgr2=0x7fe678070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780702b0 0x7fe678070700 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fe67806e4b0 tx=0x7fe67002bed0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 -- 172.21.15.53:0/1853706285 shutdown_connections 2023-10-21T17:35:32.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780702b0 0x7fe678070700 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 --2- 172.21.15.53:0/1853706285 >> v2:172.21.15.124:3300/0 conn(0x7fe6780cb150 0x7fe6780cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 --2- 172.21.15.53:0/1853706285 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe67806dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 -- 172.21.15.53:0/1853706285 >> 172.21.15.53:0/1853706285 conn(0x7fe67806a5d0 msgr2=0x7fe6780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:32.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 -- 172.21.15.53:0/1853706285 shutdown_connections 2023-10-21T17:35:32.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.839+0000 7fe67eea9640 1 -- 172.21.15.53:0/1853706285 wait complete. 2023-10-21T17:35:32.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.841+0000 7fe67eea9640 1 Processor -- start 2023-10-21T17:35:32.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.841+0000 7fe67eea9640 1 -- start start 2023-10-21T17:35:32.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 0x7fe6780cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe6780c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780cf5e0 con 0x7fe6780cb150 2023-10-21T17:35:32.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780cf300 con 0x7fe6780702b0 2023-10-21T17:35:32.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.842+0000 7fe67eea9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780c2280 con 0x7fe678161690 2023-10-21T17:35:32.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59438/0 (socket says 172.21.15.53:59438) 2023-10-21T17:35:32.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 -- 172.21.15.53:0/330516158 learned_addr learned my addr 172.21.15.53:0/330516158 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:32.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe67cc45640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 0x7fe6780cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 -- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 msgr2=0x7fe6780c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:32.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe6780c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 -- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 msgr2=0x7fe6780cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:32.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 0x7fe6780cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:32.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 -- 172.21.15.53:0/330516158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670009b30 con 0x7fe6780cb150 2023-10-21T17:35:32.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe67cc45640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 0x7fe6780cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:32.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.843+0000 7fe66ffff640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e810 tx=0x7fe66800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:32.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.844+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004030 con 0x7fe6780cb150 2023-10-21T17:35:32.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.844+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668004190 con 0x7fe6780cb150 2023-10-21T17:35:32.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.844+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668004520 con 0x7fe6780cb150 2023-10-21T17:35:32.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.844+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6780c2560 con 0x7fe6780cb150 2023-10-21T17:35:32.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.844+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6780cefa0 con 0x7fe6780cb150 2023-10-21T17:35:32.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.845+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6780c20e0 con 0x7fe6780cb150 2023-10-21T17:35:32.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.846+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe66800c6d0 con 0x7fe6780cb150 2023-10-21T17:35:32.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.847+0000 7fe66d7fa640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 0x7fe66406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:32.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.847+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fe6680896a0 con 0x7fe6780cb150 2023-10-21T17:35:32.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.848+0000 7fe67cc45640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 0x7fe66406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:32.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.848+0000 7fe67cc45640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 0x7fe66406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe674006fd0 tx=0x7fe674008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:32.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:32.851+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6780c20e0 con 0x7fe6780cb150 2023-10-21T17:35:33.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.046+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe6780c20e0 con 0x7fe6780cb150 2023-10-21T17:35:33.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.053+0000 7fe66d7fa640 1 -- 172.21.15.53:0/330516158 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6780c20e0 con 0x7fe6780cb150 2023-10-21T17:35:33.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 msgr2=0x7fe66406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 0x7fe66406d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe674006fd0 tx=0x7fe674008040 comp rx=0 tx=0).stop 2023-10-21T17:35:33.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 msgr2=0x7fe6780cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e810 tx=0x7fe66800ecd0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 shutdown_connections 2023-10-21T17:35:33.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fe66406af90 0x7fe66406d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe678161690 0x7fe6780c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6780cb150 0x7fe6780cea60 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 --2- 172.21.15.53:0/330516158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6780702b0 0x7fe6780cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 >> 172.21.15.53:0/330516158 conn(0x7fe67806a5d0 msgr2=0x7fe67815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:33.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 shutdown_connections 2023-10-21T17:35:33.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.056+0000 7fe67eea9640 1 -- 172.21.15.53:0/330516158 wait complete. 2023-10-21T17:35:33.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:35:33.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.303+0000 7f380f59e640 1 Processor -- start 2023-10-21T17:35:33.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.303+0000 7f380f59e640 1 -- start start 2023-10-21T17:35:33.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100cfa10 0x7f3810071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380f59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3810160c90 0x7f3810161120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380f59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3810071750 con 0x7f38100b1a90 2023-10-21T17:35:33.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380f59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3810071ff0 con 0x7f3810160c90 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3810071a30 con 0x7f38100cfa10 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3810160c90 0x7f3810161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3810160c90 0x7f3810161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59454/0 (socket says 172.21.15.53:59454) 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380ed9d640 1 -- 172.21.15.53:0/1101066573 learned_addr learned my addr 172.21.15.53:0/1101066573 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.305+0000 7f380e59c640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100cfa10 0x7f3810071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.306+0000 7f380ed9d640 1 -- 172.21.15.53:0/1101066573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100cfa10 msgr2=0x7f3810071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:33.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.306+0000 7f380ed9d640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100cfa10 0x7f3810071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.306+0000 7f380ed9d640 1 -- 172.21.15.53:0/1101066573 >> v1:172.21.15.53:6789/0 conn(0x7f38100b1a90 legacy=0x7f38100cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.306+0000 7f380ed9d640 1 -- 172.21.15.53:0/1101066573 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810071d10 con 0x7f3810160c90 2023-10-21T17:35:33.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.307+0000 7f380ed9d640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3810160c90 0x7f3810161120 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3808009870 tx=0x7f380802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d774f904a340f7f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.307+0000 7f380d59a640 1 -- 172.21.15.53:0/1101066573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808030840 con 0x7f3810160c90 2023-10-21T17:35:33.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.307+0000 7f380d59a640 1 -- 172.21.15.53:0/1101066573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38080309a0 con 0x7f3810160c90 2023-10-21T17:35:33.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.307+0000 7f380d59a640 1 -- 172.21.15.53:0/1101066573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808030d30 con 0x7f3810160c90 2023-10-21T17:35:33.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 -- 172.21.15.53:0/1101066573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3810160c90 msgr2=0x7f3810161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3810160c90 0x7f3810161120 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f3808009870 tx=0x7f380802b980 comp rx=0 tx=0).stop 2023-10-21T17:35:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 -- 172.21.15.53:0/1101066573 shutdown_connections 2023-10-21T17:35:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3810160c90 0x7f3810161120 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 --2- 172.21.15.53:0/1101066573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100cfa10 0x7f3810071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 -- 172.21.15.53:0/1101066573 >> 172.21.15.53:0/1101066573 conn(0x7f381006b250 msgr2=0x7f381006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:33.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 -- 172.21.15.53:0/1101066573 shutdown_connections 2023-10-21T17:35:33.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.308+0000 7f380f59e640 1 -- 172.21.15.53:0/1101066573 wait complete. 2023-10-21T17:35:33.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.310+0000 7f380f59e640 1 Processor -- start 2023-10-21T17:35:33.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.310+0000 7f380f59e640 1 -- start start 2023-10-21T17:35:33.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 0x7f3810158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3810160c90 0x7f381015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38100ac550 con 0x7f38100cfa10 2023-10-21T17:35:33.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38101591c0 con 0x7f3810160c90 2023-10-21T17:35:33.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380f59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38100ab8d0 con 0x7f38100b1a90 2023-10-21T17:35:33.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59468/0 (socket says 172.21.15.53:59468) 2023-10-21T17:35:33.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.311+0000 7f380dd9b640 1 -- 172.21.15.53:0/2921885200 learned_addr learned my addr 172.21.15.53:0/2921885200 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:33.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380e59c640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 0x7f3810158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 -- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 msgr2=0x7f3810158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380ed9d640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3810160c90 0x7f381015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 0x7f3810158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 -- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3810160c90 msgr2=0x7f381015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3810160c90 0x7f381015d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 -- 172.21.15.53:0/2921885200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080094d0 con 0x7f38100cfa10 2023-10-21T17:35:33.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380e59c640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 0x7f3810158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:33.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380dd9b640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c400 tx=0x7f37f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80052d0 con 0x7f38100cfa10 2023-10-21T17:35:33.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8005430 con 0x7f38100cfa10 2023-10-21T17:35:33.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8003830 con 0x7f38100cfa10 2023-10-21T17:35:33.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.312+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38100abbb0 con 0x7f38100cfa10 2023-10-21T17:35:33.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.313+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38100ac170 con 0x7f38100cfa10 2023-10-21T17:35:33.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.314+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38100ac690 con 0x7f38100cfa10 2023-10-21T17:35:33.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.315+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f37f8003990 con 0x7f38100cfa10 2023-10-21T17:35:33.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.316+0000 7f3806ffd640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 0x7f37f006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.316+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f37f8088f70 con 0x7f38100cfa10 2023-10-21T17:35:33.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.316+0000 7f380e59c640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 0x7f37f006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.318+0000 7f380e59c640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 0x7f37f006d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f381015c190 tx=0x7f380000b040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.320+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37f80bbca0 con 0x7f38100cfa10 2023-10-21T17:35:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.518+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3810074150 con 0x7f38100cfa10 2023-10-21T17:35:33.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.525+0000 7f3806ffd640 1 -- 172.21.15.53:0/2921885200 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7f37f80170b0 con 0x7f38100cfa10 2023-10-21T17:35:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 msgr2=0x7f37f006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 0x7f37f006d4a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f381015c190 tx=0x7f380000b040 comp rx=0 tx=0).stop 2023-10-21T17:35:33.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 msgr2=0x7f38101f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c400 tx=0x7f37f800c8c0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 shutdown_connections 2023-10-21T17:35:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f37f006afe0 0x7f37f006d4a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3810160c90 0x7f381015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38100cfa10 0x7f38101f20d0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 --2- 172.21.15.53:0/2921885200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38100b1a90 0x7f3810158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.527+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 >> 172.21.15.53:0/2921885200 conn(0x7f381006b250 msgr2=0x7f381006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.528+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 shutdown_connections 2023-10-21T17:35:33.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.528+0000 7f380f59e640 1 -- 172.21.15.53:0/2921885200 wait complete. 2023-10-21T17:35:33.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:35:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.778+0000 7f95cc026640 1 Processor -- start 2023-10-21T17:35:33.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.778+0000 7f95cc026640 1 -- start start 2023-10-21T17:35:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.779+0000 7f95cc026640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f95c40afbe0 0x7f95c40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95cc026640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c40b2620 con 0x7f95c40cb130 2023-10-21T17:35:33.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95cc026640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c40b2060 con 0x7f95c406dfc0 2023-10-21T17:35:33.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95cc026640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c40b2340 con 0x7f95c40afbe0 2023-10-21T17:35:33.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95ca5c3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f95c40cb130 0x7f95c406c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:38556/0 (socket says 172.21.15.53:38556) 2023-10-21T17:35:33.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95ca5c3640 1 -- 172.21.15.53:0/1583554820 learned_addr learned my addr 172.21.15.53:0/1583554820 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:33.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.780+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/1583554820 >> v2:172.21.15.124:3300/0 conn(0x7f95c40afbe0 0x7f95c40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749454243 0 0) 0x7f95c40b2620 con 0x7f95c40cb130 2023-10-21T17:35:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b0003520 con 0x7f95c40cb130 2023-10-21T17:35:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945349990 0 0) 0x7f95b0003520 con 0x7f95c40cb130 2023-10-21T17:35:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c9dc2640 1 -- 172.21.15.53:0/1583554820 >> v1:172.21.15.124:6789/0 conn(0x7f95c406dfc0 legacy=0x7f95c4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c9dc2640 1 -- 172.21.15.53:0/1583554820 >> v1:172.21.15.53:6789/0 conn(0x7f95c40cb130 legacy=0x7f95c406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.781+0000 7f95c9dc2640 1 -- 172.21.15.53:0/1583554820 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4070350 con 0x7f95c40afbe0 2023-10-21T17:35:33.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.782+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/1583554820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40afbe0 0x7f95c40b15b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097a0 tx=0x7f95c002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b4df557691a40e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c002f050 con 0x7f95c40afbe0 2023-10-21T17:35:33.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0031500 con 0x7f95c40afbe0 2023-10-21T17:35:33.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95c8dc0640 1 -- 172.21.15.53:0/1583554820 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0019f0 con 0x7f95c40afbe0 2023-10-21T17:35:33.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 -- 172.21.15.53:0/1583554820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40afbe0 msgr2=0x7f95c40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 --2- 172.21.15.53:0/1583554820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40afbe0 0x7f95c40b15b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097a0 tx=0x7f95c002b820 comp rx=0 tx=0).stop 2023-10-21T17:35:33.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 -- 172.21.15.53:0/1583554820 shutdown_connections 2023-10-21T17:35:33.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 --2- 172.21.15.53:0/1583554820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40afbe0 0x7f95c40b15b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 -- 172.21.15.53:0/1583554820 >> 172.21.15.53:0/1583554820 conn(0x7f95c406a5d0 msgr2=0x7f95c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:33.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.783+0000 7f95cc026640 1 -- 172.21.15.53:0/1583554820 shutdown_connections 2023-10-21T17:35:33.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.784+0000 7f95cc026640 1 -- 172.21.15.53:0/1583554820 wait complete. 2023-10-21T17:35:33.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.785+0000 7f95cc026640 1 Processor -- start 2023-10-21T17:35:33.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.785+0000 7f95cc026640 1 -- start start 2023-10-21T17:35:33.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 0x7f95c40af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 0x7f95c41527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c40ca540 con 0x7f95c40afbe0 2023-10-21T17:35:33.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4152d10 con 0x7f95c40cb130 2023-10-21T17:35:33.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.786+0000 7f95cc026640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c40ca770 con 0x7f95c406dfc0 2023-10-21T17:35:33.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:59478/0 (socket says 172.21.15.53:59478) 2023-10-21T17:35:33.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 -- 172.21.15.53:0/4040888385 learned_addr learned my addr 172.21.15.53:0/4040888385 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:33.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 0x7f95c40af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95ca5c3640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 0x7f95c41527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 -- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 msgr2=0x7f95c40af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 0x7f95c40af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 -- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 msgr2=0x7f95c41527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:33.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 0x7f95c41527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 -- 172.21.15.53:0/4040888385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0009400 con 0x7f95c40afbe0 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95ca5c3640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 0x7f95c41527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 0x7f95c40af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.787+0000 7f95c95c1640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c3c0 tx=0x7f95b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.788+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b40052d0 con 0x7f95c40afbe0 2023-10-21T17:35:33.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.788+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4005450 con 0x7f95c40afbe0 2023-10-21T17:35:33.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.788+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c40caa50 con 0x7f95c40afbe0 2023-10-21T17:35:33.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.788+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4003830 con 0x7f95c40afbe0 2023-10-21T17:35:33.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.788+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41f3690 con 0x7f95c40afbe0 2023-10-21T17:35:33.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.789+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c41f38b0 con 0x7f95c40afbe0 2023-10-21T17:35:33.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.791+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f95b4003990 con 0x7f95c40afbe0 2023-10-21T17:35:33.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.791+0000 7f95ba7fc640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 0x7f95a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:33.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.792+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f95b4088f60 con 0x7f95c40afbe0 2023-10-21T17:35:33.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.792+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 0x7f95a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:33.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.793+0000 7f95c9dc2640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 0x7f95a806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f95c0035000 tx=0x7f95c0008030 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:33.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.797+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95b4053c30 con 0x7f95c40afbe0 2023-10-21T17:35:33.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.991+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f95c40c8380 con 0x7f95c40afbe0 2023-10-21T17:35:33.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:33.998+0000 7f95ba7fc640 1 -- 172.21.15.53:0/4040888385 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7f95b4057250 con 0x7f95c40afbe0 2023-10-21T17:35:34.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.000+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 msgr2=0x7f95a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.000+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 0x7f95a806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f95c0035000 tx=0x7f95c0008030 comp rx=0 tx=0).stop 2023-10-21T17:35:34.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.000+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 msgr2=0x7f95c40ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.000+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c3c0 tx=0x7f95b400c880 comp rx=0 tx=0).stop 2023-10-21T17:35:34.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 shutdown_connections 2023-10-21T17:35:34.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f95a806af90 0x7f95a806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95c40cb130 0x7f95c41527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95c40afbe0 0x7f95c40ca000 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 --2- 172.21.15.53:0/4040888385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95c406dfc0 0x7f95c40af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 >> 172.21.15.53:0/4040888385 conn(0x7f95c406a5d0 msgr2=0x7f95c41595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:34.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 shutdown_connections 2023-10-21T17:35:34.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.001+0000 7f95cc026640 1 -- 172.21.15.53:0/4040888385 wait complete. 2023-10-21T17:35:34.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-21T17:35:34.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.240+0000 7fcfe3ccf640 1 Processor -- start 2023-10-21T17:35:34.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.241+0000 7fcfe3ccf640 1 -- start start 2023-10-21T17:35:34.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.242+0000 7fcfe3ccf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfdc15ab40 0x7fcfdc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.242+0000 7fcfe3ccf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.242+0000 7fcfe3ccf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc15b2b0 con 0x7fcfdc0c90b0 2023-10-21T17:35:34.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.242+0000 7fcfe3ccf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc15bf30 con 0x7fcfdc15ab40 2023-10-21T17:35:34.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.242+0000 7fcfe3ccf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc15f7a0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.243+0000 7fcfe226c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.243+0000 7fcfe1a6b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfdc15ab40 0x7fcfdc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.243+0000 7fcfe226c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35788/0 (socket says 172.21.15.53:35788) 2023-10-21T17:35:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.243+0000 7fcfe226c640 1 -- 172.21.15.53:0/2649199226 learned_addr learned my addr 172.21.15.53:0/2649199226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.244+0000 7fcfe226c640 1 -- 172.21.15.53:0/2649199226 >> v2:172.21.15.124:3300/0 conn(0x7fcfdc15ab40 msgr2=0x7fcfdc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:34.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.244+0000 7fcfe226c640 1 --2- 172.21.15.53:0/2649199226 >> v2:172.21.15.124:3300/0 conn(0x7fcfdc15ab40 0x7fcfdc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.244+0000 7fcfe226c640 1 -- 172.21.15.53:0/2649199226 >> v1:172.21.15.124:6789/0 conn(0x7fcfdc0c90b0 legacy=0x7fcfdc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.244+0000 7fcfe226c640 1 -- 172.21.15.53:0/2649199226 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc15b590 con 0x7fcfdc0cb580 2023-10-21T17:35:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.245+0000 7fcfe226c640 1 --2- 172.21.15.53:0/2649199226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009870 tx=0x7fcfd402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4b7fc7523624381 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe0a69640 1 -- 172.21.15.53:0/2649199226 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4030840 con 0x7fcfdc0cb580 2023-10-21T17:35:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe0a69640 1 -- 172.21.15.53:0/2649199226 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40309a0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe0a69640 1 -- 172.21.15.53:0/2649199226 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4030d30 con 0x7fcfdc0cb580 2023-10-21T17:35:34.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/2649199226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 msgr2=0x7fcfdc0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/2649199226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009870 tx=0x7fcfd402b980 comp rx=0 tx=0).stop 2023-10-21T17:35:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/2649199226 shutdown_connections 2023-10-21T17:35:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/2649199226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc0cb580 0x7fcfdc0adc60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/2649199226 >> v2:172.21.15.124:3300/0 conn(0x7fcfdc15ab40 0x7fcfdc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/2649199226 >> 172.21.15.53:0/2649199226 conn(0x7fcfdc06a380 msgr2=0x7fcfdc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/2649199226 shutdown_connections 2023-10-21T17:35:34.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.246+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/2649199226 wait complete. 2023-10-21T17:35:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.248+0000 7fcfe3ccf640 1 Processor -- start 2023-10-21T17:35:34.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.248+0000 7fcfe3ccf640 1 -- start start 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfdc0c90b0 0x7fcfdc1462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc15ab40 0x7fcfdc1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc146bc0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c8720 con 0x7fcfdc0c90b0 2023-10-21T17:35:34.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.249+0000 7fcfe3ccf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1f5e70 con 0x7fcfdc15ab40 2023-10-21T17:35:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40232/0 (socket says 172.21.15.53:40232) 2023-10-21T17:35:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 -- 172.21.15.53:0/449580010 learned_addr learned my addr 172.21.15.53:0/449580010 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe1a6b640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfdc0c90b0 0x7fcfdc1462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe226c640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc15ab40 0x7fcfdc1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 -- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc15ab40 msgr2=0x7fcfdc1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc15ab40 0x7fcfdc1f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 -- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfdc0c90b0 msgr2=0x7fcfdc1462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfdc0c90b0 0x7fcfdc1462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 -- 172.21.15.53:0/449580010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd40094d0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.250+0000 7fcfe126a640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c400 tx=0x7fcfcc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.251+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0052d0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.251+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc005430 con 0x7fcfdc0cb580 2023-10-21T17:35:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.251+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc0c7ea0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.251+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc003830 con 0x7fcfdc0cb580 2023-10-21T17:35:34.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.251+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc0c8440 con 0x7fcfdc0cb580 2023-10-21T17:35:34.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.252+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc067490 con 0x7fcfdc0cb580 2023-10-21T17:35:34.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.253+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcfcc003990 con 0x7fcfdc0cb580 2023-10-21T17:35:34.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.254+0000 7fcfd27fc640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 0x7fcfc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.255+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fcfcc088ff0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.255+0000 7fcfe1a6b640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 0x7fcfc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.256+0000 7fcfe1a6b640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 0x7fcfc006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc1f5ca0 tx=0x7fcfd800b040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.259+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfcc053d60 con 0x7fcfdc0cb580 2023-10-21T17:35:34.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.524+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fcfdc0cd3b0 con 0x7fcfdc0cb580 2023-10-21T17:35:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.600+0000 7fcfd27fc640 1 -- 172.21.15.53:0/449580010 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3053) v1 ==== 165+0+0 (secure 0 0 0) 0x7fcfcc057380 con 0x7fcfdc0cb580 2023-10-21T17:35:34.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 msgr2=0x7fcfc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 0x7fcfc006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc1f5ca0 tx=0x7fcfd800b040 comp rx=0 tx=0).stop 2023-10-21T17:35:34.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 msgr2=0x7fcfdc0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c400 tx=0x7fcfcc00c8c0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 shutdown_connections 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fcfc006af90 0x7fcfc006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfdc15ab40 0x7fcfdc1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfdc0cb580 0x7fcfdc0c7960 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 --2- 172.21.15.53:0/449580010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfdc0c90b0 0x7fcfdc1462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.604+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 >> 172.21.15.53:0/449580010 conn(0x7fcfdc06a380 msgr2=0x7fcfdc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:34.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.605+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 shutdown_connections 2023-10-21T17:35:34.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.605+0000 7fcfe3ccf640 1 -- 172.21.15.53:0/449580010 wait complete. 2023-10-21T17:35:34.626 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-21T17:35:34.626 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-21T17:35:34.626 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-21T17:35:34.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:34.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.870+0000 7f508d3ff640 1 Processor -- start 2023-10-21T17:35:34.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.870+0000 7f508d3ff640 1 -- start start 2023-10-21T17:35:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f508d3ff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f508815cca0 0x7f50881614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f508d3ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f50880cab40 0x7f50880b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f508d3ff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50880c9020 con 0x7f508815d080 2023-10-21T17:35:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f508d3ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880ccc20 con 0x7f50880cab40 2023-10-21T17:35:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f508d3ff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880c92e0 con 0x7f508815cca0 2023-10-21T17:35:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f50877fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f50880cab40 0x7f50880b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f50877fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f50880cab40 0x7f50880b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40238/0 (socket says 172.21.15.53:40238) 2023-10-21T17:35:34.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.872+0000 7f50877fe640 1 -- 172.21.15.53:0/3153169660 learned_addr learned my addr 172.21.15.53:0/3153169660 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.873+0000 7f50877fe640 1 -- 172.21.15.53:0/3153169660 >> v2:172.21.15.124:3300/0 conn(0x7f508815cca0 msgr2=0x7f50881614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.873+0000 7f5087fff640 1 --2- 172.21.15.53:0/3153169660 >> v2:172.21.15.124:3300/0 conn(0x7f508815cca0 0x7f50881614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.873+0000 7f50877fe640 1 --2- 172.21.15.53:0/3153169660 >> v2:172.21.15.124:3300/0 conn(0x7f508815cca0 0x7f50881614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.873+0000 7f50877fe640 1 -- 172.21.15.53:0/3153169660 >> v1:172.21.15.124:6789/0 conn(0x7f508815d080 legacy=0x7f50880701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.873+0000 7f50877fe640 1 -- 172.21.15.53:0/3153169660 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880cc7b0 con 0x7f50880cab40 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.874+0000 7f50877fe640 1 --2- 172.21.15.53:0/3153169660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50880cab40 0x7f50880b00c0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f508806e650 tx=0x7f507c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a543ee738c6c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.874+0000 7f5086ffd640 1 -- 172.21.15.53:0/3153169660 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c031750 con 0x7f50880cab40 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.874+0000 7f5086ffd640 1 -- 172.21.15.53:0/3153169660 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c0318b0 con 0x7f50880cab40 2023-10-21T17:35:34.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.874+0000 7f5086ffd640 1 -- 172.21.15.53:0/3153169660 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c031c40 con 0x7f50880cab40 2023-10-21T17:35:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 -- 172.21.15.53:0/3153169660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50880cab40 msgr2=0x7f50880b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 --2- 172.21.15.53:0/3153169660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50880cab40 0x7f50880b00c0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f508806e650 tx=0x7f507c02b7f0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 -- 172.21.15.53:0/3153169660 shutdown_connections 2023-10-21T17:35:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 --2- 172.21.15.53:0/3153169660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50880cab40 0x7f50880b00c0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 --2- 172.21.15.53:0/3153169660 >> v2:172.21.15.124:3300/0 conn(0x7f508815cca0 0x7f50881614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 -- 172.21.15.53:0/3153169660 >> 172.21.15.53:0/3153169660 conn(0x7f50880cb0c0 msgr2=0x7f50880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:34.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 -- 172.21.15.53:0/3153169660 shutdown_connections 2023-10-21T17:35:34.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.875+0000 7f508d3ff640 1 -- 172.21.15.53:0/3153169660 wait complete. 2023-10-21T17:35:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.877+0000 7f508d3ff640 1 Processor -- start 2023-10-21T17:35:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.877+0000 7f508d3ff640 1 -- start start 2023-10-21T17:35:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50880cab40 0x7f5088148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f508815d080 0x7f50880706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880c8880 con 0x7f508815cca0 2023-10-21T17:35:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508814a190 con 0x7f50880cab40 2023-10-21T17:35:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f508d3ff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5088149d00 con 0x7f508815d080 2023-10-21T17:35:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f50877fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f50877fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40246/0 (socket says 172.21.15.53:40246) 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.878+0000 7f50877fe640 1 -- 172.21.15.53:0/1793279416 learned_addr learned my addr 172.21.15.53:0/1793279416 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 -- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f508815d080 msgr2=0x7f50880706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f508815d080 0x7f50880706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 -- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50880cab40 msgr2=0x7f5088148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50880cab40 0x7f5088148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:34.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 -- 172.21.15.53:0/1793279416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0094d0 con 0x7f508815cca0 2023-10-21T17:35:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f50877fe640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f507c02b830 tx=0x7f507c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c033c80 con 0x7f508815cca0 2023-10-21T17:35:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c033de0 con 0x7f508815cca0 2023-10-21T17:35:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c03c640 con 0x7f508815cca0 2023-10-21T17:35:34.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.879+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50881471a0 con 0x7f508815cca0 2023-10-21T17:35:34.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.880+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50881496b0 con 0x7f508815cca0 2023-10-21T17:35:34.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.881+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508804e160 con 0x7f508815cca0 2023-10-21T17:35:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.882+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f507c03a040 con 0x7f508815cca0 2023-10-21T17:35:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.883+0000 7f5084ff9640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 0x7f507406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.883+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f507c0ab790 con 0x7f508815cca0 2023-10-21T17:35:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.883+0000 7f5087fff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 0x7f507406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:34.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.884+0000 7f5087fff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 0x7f507406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f507800a810 tx=0x7f5078008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:34.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:34.887+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f507c076470 con 0x7f508815cca0 2023-10-21T17:35:35.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.082+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50880cd400 con 0x7f508815cca0 2023-10-21T17:35:35.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.084+0000 7f5084ff9640 1 -- 172.21.15.53:0/1793279416 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+14903 (secure 0 0 0) 0x7f507c079a90 con 0x7f508815cca0 2023-10-21T17:35:35.086 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:35.086 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":206,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:32.583837+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/93601398":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3884449134":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/2744557552":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/3194556637":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/1897991612":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/4199687649":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6837/2015564828":"2023-10-22T17:35:29.537927+0000","172.21.15.124:6832/3632125819":"2023-10-22T17:30:21.164866+0000","172.21.15.124:0/2953831507":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3755084077":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6836/2015564828":"2023-10-22T17:35:29.537927+0000","172.21.15.124:6832/2750944240":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3409964486":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/380566591":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/766800588":"2023-10-22T17:30:21.164866+0000","172.21.15.124:6833/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:6832/1955466907":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3041665938":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3287676299":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3768127330":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/3295254396":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3702020697":"2023-10-22T17:19:40.971451+0000","172.21.15.124:0/3770250885":"2023-10-22T17:35:25.511427+0000","172.21.15.124:0/1180026628":"2023-10-22T17:19:40.971451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 msgr2=0x7f507406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 0x7f507406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f507800a810 tx=0x7f5078008040 comp rx=0 tx=0).stop 2023-10-21T17:35:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 msgr2=0x7f50880701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f507c02b830 tx=0x7f507c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 shutdown_connections 2023-10-21T17:35:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f507406af90 0x7f507406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f508815d080 0x7f50880706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f508815cca0 0x7f50880701b0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 --2- 172.21.15.53:0/1793279416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50880cab40 0x7f5088148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.087+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 >> 172.21.15.53:0/1793279416 conn(0x7f50880cb0c0 msgr2=0x7f50880c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.088+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 shutdown_connections 2023-10-21T17:35:35.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.088+0000 7f508d3ff640 1 -- 172.21.15.53:0/1793279416 wait complete. 2023-10-21T17:35:35.107 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:35:35.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:35:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.333+0000 7f31b9116640 1 Processor -- start 2023-10-21T17:35:35.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.333+0000 7f31b9116640 1 -- start start 2023-10-21T17:35:35.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b9116640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f31b40cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b9116640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b415ac90 con 0x7f31b40cce90 2023-10-21T17:35:35.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b9116640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b415a6d0 con 0x7f31b40caef0 2023-10-21T17:35:35.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b9116640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415a9b0 con 0x7f31b40cdc30 2023-10-21T17:35:35.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b8915640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f31b40cce90 0x7f31b4159bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50810/0 (socket says 172.21.15.53:50810) 2023-10-21T17:35:35.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.335+0000 7f31b8915640 1 -- 172.21.15.53:0/1512589017 learned_addr learned my addr 172.21.15.53:0/1512589017 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b37fe640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f31b40cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147787347 0 0) 0x7f31b415ac90 con 0x7f31b40cce90 2023-10-21T17:35:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f319c003520 con 0x7f31b40cce90 2023-10-21T17:35:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1353771973 0 0) 0x7f319c003520 con 0x7f31b40cce90 2023-10-21T17:35:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 msgr2=0x7f31b40cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f31b40cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 >> v1:172.21.15.124:6789/0 conn(0x7f31b40caef0 legacy=0x7f31b40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40cc150 con 0x7f31b40cce90 2023-10-21T17:35:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f31ac001670 con 0x7f31b40cce90 2023-10-21T17:35:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 >> v1:172.21.15.53:6789/0 conn(0x7f31b40cce90 legacy=0x7f31b4159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f319c0039a0 0x7f319c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c0047c0 0x7f319c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f319c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b415a9b0 con 0x7f319c0047c0 2023-10-21T17:35:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.336+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c00ba30 con 0x7f319c0039a0 2023-10-21T17:35:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c00ddd0 con 0x7f31b40cdc30 2023-10-21T17:35:35.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c0047c0 0x7f319c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b3fff640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f319c0039a0 0x7f319c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b8915640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f319c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 -- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 msgr2=0x7f319c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31b40cdc30 0x7f319c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 -- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f319c0039a0 msgr2=0x7f319c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:35.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f319c0039a0 0x7f319c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.337+0000 7f31b37fe640 1 -- 172.21.15.53:0/1512589017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c003520 con 0x7f319c0047c0 2023-10-21T17:35:35.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.338+0000 7f31b37fe640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c0047c0 0x7f319c004c40 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ce070 tx=0x7f319c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c2169e09aeacb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a802c630 con 0x7f319c0047c0 2023-10-21T17:35:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a802c790 con 0x7f319c0047c0 2023-10-21T17:35:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b2ffd640 1 -- 172.21.15.53:0/1512589017 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a802cb20 con 0x7f319c0047c0 2023-10-21T17:35:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b9116640 1 -- 172.21.15.53:0/1512589017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c0047c0 msgr2=0x7f319c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b9116640 1 --2- 172.21.15.53:0/1512589017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f319c0047c0 0x7f319c004c40 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ce070 tx=0x7f319c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b8915640 1 -- 172.21.15.53:0/1512589017 reap_dead start 2023-10-21T17:35:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.339+0000 7f31b9116640 1 -- 172.21.15.53:0/1512589017 shutdown_connections 2023-10-21T17:35:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.340+0000 7f31b9116640 1 -- 172.21.15.53:0/1512589017 >> 172.21.15.53:0/1512589017 conn(0x7f31b406c010 msgr2=0x7f31b406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.340+0000 7f31b9116640 1 -- 172.21.15.53:0/1512589017 shutdown_connections 2023-10-21T17:35:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.340+0000 7f31b9116640 1 -- 172.21.15.53:0/1512589017 wait complete. 2023-10-21T17:35:35.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.341+0000 7f31b9116640 1 Processor -- start 2023-10-21T17:35:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.342+0000 7f31b9116640 1 -- start start 2023-10-21T17:35:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f319c0047c0 0x7f31b414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31b4146940 0x7f31b4146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b414a670 con 0x7f31b40b2330 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b28b0 con 0x7f31b4146940 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b9116640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b2b90 con 0x7f319c0047c0 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40260/0 (socket says 172.21.15.53:40260) 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 -- 172.21.15.53:0/2158911823 learned_addr learned my addr 172.21.15.53:0/2158911823 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:35.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b8915640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31b4146940 0x7f31b4146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b3fff640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f319c0047c0 0x7f31b414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 -- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f319c0047c0 msgr2=0x7f31b414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f319c0047c0 0x7f31b414a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 -- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31b4146940 msgr2=0x7f31b4146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31b4146940 0x7f31b4146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.343+0000 7f31b37fe640 1 -- 172.21.15.53:0/2158911823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40b2e70 con 0x7f31b40b2330 2023-10-21T17:35:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b37fe640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f31a8037040 tx=0x7f31a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8003810 con 0x7f31b40b2330 2023-10-21T17:35:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a8003970 con 0x7f31b40b2330 2023-10-21T17:35:35.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8003d00 con 0x7f31b40b2330 2023-10-21T17:35:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b40b3150 con 0x7f31b40b2330 2023-10-21T17:35:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.344+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b4147300 con 0x7f31b40b2330 2023-10-21T17:35:35.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.346+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b4074150 con 0x7f31b40b2330 2023-10-21T17:35:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.347+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f31a8034440 con 0x7f31b40b2330 2023-10-21T17:35:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.347+0000 7f31b0ff9640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 0x7f31a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.348+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f31a80aba10 con 0x7f31b40b2330 2023-10-21T17:35:35.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.348+0000 7f31b3fff640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 0x7f31a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.349+0000 7f31b3fff640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 0x7f31a006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31a4001e20 tx=0x7f31a4007ef0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.352+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31a8076820 con 0x7f31b40b2330 2023-10-21T17:35:35.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.545+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31b40cd9b0 con 0x7f31b40b2330 2023-10-21T17:35:35.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.546+0000 7f31b0ff9640 1 -- 172.21.15.53:0/2158911823 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31a8003e60 con 0x7f31b40b2330 2023-10-21T17:35:35.547 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:35:35.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 msgr2=0x7f31a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 0x7f31a006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31a4001e20 tx=0x7f31a4007ef0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 msgr2=0x7f31b40c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f31a8037040 tx=0x7f31a8004730 comp rx=0 tx=0).stop 2023-10-21T17:35:35.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 shutdown_connections 2023-10-21T17:35:35.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f31a006af90 0x7f31a006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f31b4146940 0x7f31b4146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31b40b2330 0x7f31b40c5820 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 --2- 172.21.15.53:0/2158911823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f319c0047c0 0x7f31b414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.548+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 >> 172.21.15.53:0/2158911823 conn(0x7f31b406c010 msgr2=0x7f31b4073210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:35.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.549+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 shutdown_connections 2023-10-21T17:35:35.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.549+0000 7f31b9116640 1 -- 172.21.15.53:0/2158911823 wait complete. 2023-10-21T17:35:35.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-21T17:35:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.797+0000 7f5e3c56c640 1 Processor -- start 2023-10-21T17:35:35.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.797+0000 7f5e3c56c640 1 -- start start 2023-10-21T17:35:35.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e3415ab40 0x7f5e340ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e340c90b0 0x7f5e340adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 0x7f5e340cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3415bf30 con 0x7f5e340c90b0 2023-10-21T17:35:35.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e340cb840 con 0x7f5e340cae70 2023-10-21T17:35:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.798+0000 7f5e3c56c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3415b2b0 con 0x7f5e3415ab40 2023-10-21T17:35:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e340c90b0 0x7f5e340adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5e340c90b0 0x7f5e340adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40264/0 (socket says 172.21.15.53:40264) 2023-10-21T17:35:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 -- 172.21.15.53:0/3555687958 learned_addr learned my addr 172.21.15.53:0/3555687958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:35.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 -- 172.21.15.53:0/3555687958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e3415ab40 msgr2=0x7f5e340ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 --2- 172.21.15.53:0/3555687958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e3415ab40 0x7f5e340ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 -- 172.21.15.53:0/3555687958 >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 msgr2=0x7f5e340cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e3ab09640 1 --2- 172.21.15.53:0/3555687958 >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 0x7f5e340cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 --2- 172.21.15.53:0/3555687958 >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 0x7f5e340cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e39b07640 1 -- 172.21.15.53:0/3555687958 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3415b590 con 0x7f5e340c90b0 2023-10-21T17:35:35.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.799+0000 7f5e3ab09640 1 --2- 172.21.15.53:0/3555687958 >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 0x7f5e340cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.800+0000 7f5e39b07640 1 --2- 172.21.15.53:0/3555687958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e340c90b0 0x7f5e340adc60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f5e340c8d50 tx=0x7f5e2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a26b9bde1d688 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e39306640 1 -- 172.21.15.53:0/3555687958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24031870 con 0x7f5e340c90b0 2023-10-21T17:35:35.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e39306640 1 -- 172.21.15.53:0/3555687958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e240319d0 con 0x7f5e340c90b0 2023-10-21T17:35:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e39306640 1 -- 172.21.15.53:0/3555687958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24031d60 con 0x7f5e340c90b0 2023-10-21T17:35:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 -- 172.21.15.53:0/3555687958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e340c90b0 msgr2=0x7f5e340adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/3555687958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e340c90b0 0x7f5e340adc60 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f5e340c8d50 tx=0x7f5e2402b910 comp rx=0 tx=0).stop 2023-10-21T17:35:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 -- 172.21.15.53:0/3555687958 shutdown_connections 2023-10-21T17:35:35.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/3555687958 >> v2:172.21.15.124:3300/0 conn(0x7f5e340cae70 0x7f5e340cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/3555687958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e340c90b0 0x7f5e340adc60 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/3555687958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e3415ab40 0x7f5e340ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.801+0000 7f5e3c56c640 1 -- 172.21.15.53:0/3555687958 >> 172.21.15.53:0/3555687958 conn(0x7f5e3406a380 msgr2=0x7f5e340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.802+0000 7f5e3c56c640 1 -- 172.21.15.53:0/3555687958 shutdown_connections 2023-10-21T17:35:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.802+0000 7f5e3c56c640 1 -- 172.21.15.53:0/3555687958 wait complete. 2023-10-21T17:35:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.803+0000 7f5e3c56c640 1 Processor -- start 2023-10-21T17:35:35.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.804+0000 7f5e3c56c640 1 -- start start 2023-10-21T17:35:35.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.804+0000 7f5e3c56c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 0x7f5e34147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3c56c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e340cae70 0x7f5e340ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3c56c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3c56c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34155d60 con 0x7f5e3415ab40 2023-10-21T17:35:35.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3c56c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34156040 con 0x7f5e340c90b0 2023-10-21T17:35:35.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3c56c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34156320 con 0x7f5e340cae70 2023-10-21T17:35:35.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40272/0 (socket says 172.21.15.53:40272) 2023-10-21T17:35:35.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 -- 172.21.15.53:0/1804643300 learned_addr learned my addr 172.21.15.53:0/1804643300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:35.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3a308640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 0x7f5e34147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 -- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e340cae70 msgr2=0x7f5e340ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e39b07640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e340cae70 0x7f5e340ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e340cae70 0x7f5e340ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 -- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 msgr2=0x7f5e34147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:35.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 0x7f5e34147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:35.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3ab09640 1 -- 172.21.15.53:0/1804643300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240094d0 con 0x7f5e3415ab40 2023-10-21T17:35:35.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.805+0000 7f5e3a308640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 0x7f5e34147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:35.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e3ab09640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00cef0 tx=0x7f5e2c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00d840 con 0x7f5e3415ab40 2023-10-21T17:35:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00d9c0 con 0x7f5e3415ab40 2023-10-21T17:35:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00dd70 con 0x7f5e3415ab40 2023-10-21T17:35:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e34156600 con 0x7f5e3415ab40 2023-10-21T17:35:35.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.806+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341475b0 con 0x7f5e3415ab40 2023-10-21T17:35:35.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.807+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34156b20 con 0x7f5e3415ab40 2023-10-21T17:35:35.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.808+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5e2c01e3b0 con 0x7f5e3415ab40 2023-10-21T17:35:35.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.809+0000 7f5e2affd640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 0x7f5e1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:35.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.809+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f5e2c08a5d0 con 0x7f5e3415ab40 2023-10-21T17:35:35.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.810+0000 7f5e3a308640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 0x7f5e1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:35.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.811+0000 7f5e3a308640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 0x7f5e1806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006fd0 tx=0x7f5e30008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:35.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:35.813+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e34156b20 con 0x7f5e3415ab40 2023-10-21T17:35:36.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.079+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f5e34156b20 con 0x7f5e3415ab40 2023-10-21T17:35:36.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.613+0000 7f5e2affd640 1 -- 172.21.15.53:0/1804643300 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3055) v1 ==== 166+0+0 (secure 0 0 0) 0x7f5e34156b20 con 0x7f5e3415ab40 2023-10-21T17:35:36.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.616+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 msgr2=0x7f5e1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:36.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 0x7f5e1806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e30006fd0 tx=0x7f5e30008040 comp rx=0 tx=0).stop 2023-10-21T17:35:36.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 msgr2=0x7f5e340aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:36.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00cef0 tx=0x7f5e2c00a710 comp rx=0 tx=0).stop 2023-10-21T17:35:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 shutdown_connections 2023-10-21T17:35:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f5e1806af90 0x7f5e1806d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e3415ab40 0x7f5e340aca20 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e340cae70 0x7f5e340ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 --2- 172.21.15.53:0/1804643300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e340c90b0 0x7f5e34147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 >> 172.21.15.53:0/1804643300 conn(0x7f5e3406a380 msgr2=0x7f5e340cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 shutdown_connections 2023-10-21T17:35:36.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.617+0000 7f5e3c56c640 1 -- 172.21.15.53:0/1804643300 wait complete. 2023-10-21T17:35:36.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:35:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.875+0000 7faef44cf640 1 Processor -- start 2023-10-21T17:35:36.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.876+0000 7faef44cf640 1 -- start start 2023-10-21T17:35:36.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef44cf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec06dfc0 0x7faeec070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef44cf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faeec071070 0x7faeec0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef44cf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0b1cc0 con 0x7faeec0afbe0 2023-10-21T17:35:36.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef44cf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeec0b2620 con 0x7faeec071070 2023-10-21T17:35:36.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef44cf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0b2060 con 0x7faeec06dfc0 2023-10-21T17:35:36.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef2a6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faeec071070 0x7faeec0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef2a6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faeec071070 0x7faeec0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40280/0 (socket says 172.21.15.53:40280) 2023-10-21T17:35:36.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.877+0000 7faef2a6c640 1 -- 172.21.15.53:0/4137251714 learned_addr learned my addr 172.21.15.53:0/4137251714 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:36.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.878+0000 7faef1a6a640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec06dfc0 0x7faeec070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.878+0000 7faef2a6c640 1 -- 172.21.15.53:0/4137251714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec06dfc0 msgr2=0x7faeec070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:36.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.878+0000 7faef2a6c640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec06dfc0 0x7faeec070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.878+0000 7faef2a6c640 1 -- 172.21.15.53:0/4137251714 >> v1:172.21.15.124:6789/0 conn(0x7faeec0afbe0 legacy=0x7faeec0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:36.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.878+0000 7faef2a6c640 1 -- 172.21.15.53:0/4137251714 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0b2340 con 0x7faeec071070 2023-10-21T17:35:36.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.879+0000 7faef2a6c640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec071070 0x7faeec0cb130 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7faee4009870 tx=0x7faee402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f6ac4ef17daae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:36.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.879+0000 7faef1269640 1 -- 172.21.15.53:0/4137251714 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4031640 con 0x7faeec071070 2023-10-21T17:35:36.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.879+0000 7faef1269640 1 -- 172.21.15.53:0/4137251714 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee40317a0 con 0x7faeec071070 2023-10-21T17:35:36.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.879+0000 7faef1269640 1 -- 172.21.15.53:0/4137251714 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4031b30 con 0x7faeec071070 2023-10-21T17:35:36.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 -- 172.21.15.53:0/4137251714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec071070 msgr2=0x7faeec0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:36.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec071070 0x7faeec0cb130 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7faee4009870 tx=0x7faee402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 -- 172.21.15.53:0/4137251714 shutdown_connections 2023-10-21T17:35:36.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec06dfc0 0x7faeec070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 --2- 172.21.15.53:0/4137251714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec071070 0x7faeec0cb130 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 -- 172.21.15.53:0/4137251714 >> 172.21.15.53:0/4137251714 conn(0x7faeec06a5d0 msgr2=0x7faeec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:36.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 -- 172.21.15.53:0/4137251714 shutdown_connections 2023-10-21T17:35:36.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.880+0000 7faef44cf640 1 -- 172.21.15.53:0/4137251714 wait complete. 2023-10-21T17:35:36.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.882+0000 7faef44cf640 1 Processor -- start 2023-10-21T17:35:36.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.882+0000 7faef44cf640 1 -- start start 2023-10-21T17:35:36.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec071070 0x7faeec0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 0x7faeec14f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0c2680 con 0x7faeec06dfc0 2023-10-21T17:35:36.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0c2960 con 0x7faeec0afbe0 2023-10-21T17:35:36.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef44cf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0c2c40 con 0x7faeec071070 2023-10-21T17:35:36.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef226b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef226b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40290/0 (socket says 172.21.15.53:40290) 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.883+0000 7faef226b640 1 -- 172.21.15.53:0/53351288 learned_addr learned my addr 172.21.15.53:0/53351288 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef1a6a640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec071070 0x7faeec0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef2a6c640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 0x7faeec14f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 -- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec071070 msgr2=0x7faeec0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:36.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec071070 0x7faeec0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 -- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 msgr2=0x7faeec14f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:36.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 0x7faeec14f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:36.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 -- 172.21.15.53:0/53351288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40094d0 con 0x7faeec06dfc0 2023-10-21T17:35:36.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef2a6c640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 0x7faeec14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:36.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef226b640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7faee800c7c0 tx=0x7faee800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800aa50 con 0x7faeec06dfc0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee800abb0 con 0x7faeec06dfc0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8012420 con 0x7faeec06dfc0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.884+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec0c2f20 con 0x7faeec06dfc0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.885+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec0c2230 con 0x7faeec06dfc0 2023-10-21T17:35:36.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.886+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec0c2450 con 0x7faeec06dfc0 2023-10-21T17:35:36.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.887+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faee8012580 con 0x7faeec06dfc0 2023-10-21T17:35:36.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.888+0000 7faee2ffd640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 0x7faed006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:36.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.888+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7faee8089a00 con 0x7faeec06dfc0 2023-10-21T17:35:36.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.889+0000 7faef1a6a640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 0x7faed006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:36.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.890+0000 7faef1a6a640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 0x7faed006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faedc006fd0 tx=0x7faedc008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:36.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:36.892+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faee80546d0 con 0x7faeec06dfc0 2023-10-21T17:35:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.091+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faeec0c3440 con 0x7faeec06dfc0 2023-10-21T17:35:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.098+0000 7faee2ffd640 1 -- 172.21.15.53:0/53351288 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7faeec0c3440 con 0x7faeec06dfc0 2023-10-21T17:35:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 msgr2=0x7faed006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 0x7faed006d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faedc006fd0 tx=0x7faedc008040 comp rx=0 tx=0).stop 2023-10-21T17:35:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 msgr2=0x7faeec0ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7faee800c7c0 tx=0x7faee800cc80 comp rx=0 tx=0).stop 2023-10-21T17:35:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 shutdown_connections 2023-10-21T17:35:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7faed006af90 0x7faed006d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faeec0afbe0 0x7faeec14f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faeec071070 0x7faeec0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 --2- 172.21.15.53:0/53351288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faeec06dfc0 0x7faeec0ceea0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 >> 172.21.15.53:0/53351288 conn(0x7faeec06a5d0 msgr2=0x7faeec0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 shutdown_connections 2023-10-21T17:35:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.101+0000 7faef44cf640 1 -- 172.21.15.53:0/53351288 wait complete. 2023-10-21T17:35:37.122 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:35:37.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:35:37.247 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:35:37.374 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:35:37.375 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:37.375 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:37.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:35:37.440 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:35:37.440 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:37.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:35:37.501 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:37.502 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:37.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:35:37.566 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:35:37.566 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:37.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:35:37.629 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:35:37.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:37.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.866+0000 7f2b2a483640 1 Processor -- start 2023-10-21T17:35:37.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.866+0000 7f2b2a483640 1 -- start start 2023-10-21T17:35:37.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b2a483640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b240ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b2a483640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b240cb580 0x7f2b240adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b2a483640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2415b2b0 con 0x7f2b240c90b0 2023-10-21T17:35:37.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b2a483640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2415bf30 con 0x7f2b240cb580 2023-10-21T17:35:37.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b2a483640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415f7a0 con 0x7f2b2415ab40 2023-10-21T17:35:37.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.868+0000 7f2b28a20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b240cb580 0x7f2b240adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:37.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b240cb580 0x7f2b240adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40294/0 (socket says 172.21.15.53:40294) 2023-10-21T17:35:37.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 -- 172.21.15.53:0/3822252455 learned_addr learned my addr 172.21.15.53:0/3822252455 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:37.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b23fff640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b240ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:37.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 -- 172.21.15.53:0/3822252455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 msgr2=0x7f2b240ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:37.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b240ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 -- 172.21.15.53:0/3822252455 >> v1:172.21.15.53:6789/0 conn(0x7f2b240c90b0 legacy=0x7f2b240cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.869+0000 7f2b28a20640 1 -- 172.21.15.53:0/3822252455 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2415b590 con 0x7f2b240cb580 2023-10-21T17:35:37.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.870+0000 7f2b28a20640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240cb580 0x7f2b240adc60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204a98bf81cf0c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:37.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b22ffd640 1 -- 172.21.15.53:0/3822252455 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c02cb00 con 0x7f2b240cb580 2023-10-21T17:35:37.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b22ffd640 1 -- 172.21.15.53:0/3822252455 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02cc60 con 0x7f2b240cb580 2023-10-21T17:35:37.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b22ffd640 1 -- 172.21.15.53:0/3822252455 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c0343f0 con 0x7f2b240cb580 2023-10-21T17:35:37.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 -- 172.21.15.53:0/3822252455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240cb580 msgr2=0x7f2b240adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240cb580 0x7f2b240adc60 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c02bf60 comp rx=0 tx=0).stop 2023-10-21T17:35:37.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 -- 172.21.15.53:0/3822252455 shutdown_connections 2023-10-21T17:35:37.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240cb580 0x7f2b240adc60 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3822252455 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b240ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 -- 172.21.15.53:0/3822252455 >> 172.21.15.53:0/3822252455 conn(0x7f2b2406a380 msgr2=0x7f2b240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:37.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 -- 172.21.15.53:0/3822252455 shutdown_connections 2023-10-21T17:35:37.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.871+0000 7f2b2a483640 1 -- 172.21.15.53:0/3822252455 wait complete. 2023-10-21T17:35:37.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.873+0000 7f2b2a483640 1 Processor -- start 2023-10-21T17:35:37.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.873+0000 7f2b2a483640 1 -- start start 2023-10-21T17:35:37.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 0x7f2b24151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b24151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415cd50 con 0x7f2b240c90b0 2023-10-21T17:35:37.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415d5b0 con 0x7f2b240cb580 2023-10-21T17:35:37.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.874+0000 7f2b2a483640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240c5090 con 0x7f2b2415ab40 2023-10-21T17:35:37.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:37.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40296/0 (socket says 172.21.15.53:40296) 2023-10-21T17:35:37.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 -- 172.21.15.53:0/3133980091 learned_addr learned my addr 172.21.15.53:0/3133980091 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:37.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 -- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 msgr2=0x7f2b24151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b24151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 -- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 msgr2=0x7f2b24151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:37.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b237fe640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 0x7f2b24151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:37.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 0x7f2b24151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:37.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b23fff640 1 -- 172.21.15.53:0/3133980091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009b30 con 0x7f2b240c90b0 2023-10-21T17:35:37.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.875+0000 7f2b237fe640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 0x7f2b24151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:37.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b23fff640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d800 tx=0x7f2b1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:37.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400b6c0 con 0x7f2b240c90b0 2023-10-21T17:35:37.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1400b820 con 0x7f2b240c90b0 2023-10-21T17:35:37.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400bbb0 con 0x7f2b240c90b0 2023-10-21T17:35:37.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2415a1f0 con 0x7f2b240c90b0 2023-10-21T17:35:37.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.876+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2415d240 con 0x7f2b240c90b0 2023-10-21T17:35:37.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.877+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2415af20 con 0x7f2b240c90b0 2023-10-21T17:35:37.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.878+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2b1400bd10 con 0x7f2b240c90b0 2023-10-21T17:35:37.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.879+0000 7f2b20ff9640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 0x7f2b1006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:37.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.879+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f2b14089ef0 con 0x7f2b240c90b0 2023-10-21T17:35:37.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.880+0000 7f2b237fe640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 0x7f2b1006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:37.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.882+0000 7f2b237fe640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 0x7f2b1006d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b240b4790 tx=0x7f2b18009040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:37.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:37.884+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b14054c60 con 0x7f2b240c90b0 2023-10-21T17:35:38.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.113+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b24067490 con 0x7f2b240c90b0 2023-10-21T17:35:38.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.114+0000 7f2b20ff9640 1 -- 172.21.15.53:0/3133980091 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+3707 (secure 0 0 0) 0x7f2b14058280 con 0x7f2b240c90b0 2023-10-21T17:35:38.116 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:38.116 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":97,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8386,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1254440720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1254440720},{"type":"v1","addr":"172.21.15.124:6835","nonce":1254440720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":8731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3166041823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3166041823},{"type":"v1","addr":"172.21.15.124:6837","nonce":3166041823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10128,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/2884252254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2884252254},{"type":"v1","addr":"172.21.15.53:6834","nonce":2884252254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2414240429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":2414240429},{"type":"v1","addr":"172.21.15.53:6835","nonce":2414240429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[]} 2023-10-21T17:35:38.116 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 97 2023-10-21T17:35:38.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 msgr2=0x7f2b1006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 0x7f2b1006d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b240b4790 tx=0x7f2b18009040 comp rx=0 tx=0).stop 2023-10-21T17:35:38.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 msgr2=0x7f2b240c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400d800 tx=0x7f2b1400dcc0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 shutdown_connections 2023-10-21T17:35:38.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f2b1006afc0 0x7f2b1006d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b2415ab40 0x7f2b24151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b240cb580 0x7f2b24151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 --2- 172.21.15.53:0/3133980091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b240c90b0 0x7f2b240c4540 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 >> 172.21.15.53:0/3133980091 conn(0x7f2b2406a380 msgr2=0x7f2b240cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:38.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 shutdown_connections 2023-10-21T17:35:38.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.117+0000 7f2b2a483640 1 -- 172.21.15.53:0/3133980091 wait complete. 2023-10-21T17:35:38.137 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:35:38.137 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:35:38.138 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:35:38.138 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:35:38.138 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:35:38.139 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:35:38.139 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:35:38.139 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:35:38.139 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:35:38.140 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:35:38.140 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:35:38.140 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:35:38.141 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:35:38.141 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:35:38.141 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:35:38.141 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:35:38.159 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:35:38.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:35:38.163 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:35:38.164 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:35:38.165 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:35:38.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:35:38.166 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:35:38.167 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:35:38.213 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:35:38.213 DEBUG:teuthology.parallel:result is None 2023-10-21T17:35:38.214 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:35:38.215 DEBUG:teuthology.parallel:result is None 2023-10-21T17:35:38.221 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:35:38.221 DEBUG:teuthology.parallel:result is None 2023-10-21T17:35:38.222 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:35:38.222 DEBUG:teuthology.parallel:result is None 2023-10-21T17:35:38.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:35:38.376 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:35:38.377 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:35:38.378 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:35:38.381 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:35:38.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.416+0000 7f529b301640 1 Processor -- start 2023-10-21T17:35:38.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.416+0000 7f529b301640 1 -- start start 2023-10-21T17:35:38.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529b301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 0x7f52940b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:38.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529b301640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529415b310 con 0x7f52940c9cf0 2023-10-21T17:35:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529b301640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529406d290 con 0x7f529406c3a0 2023-10-21T17:35:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529b301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529415b110 con 0x7f529406db60 2023-10-21T17:35:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529889c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f52940c9cf0 0x7f529415aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50848/0 (socket says 172.21.15.53:50848) 2023-10-21T17:35:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529889c640 1 -- 172.21.15.53:0/2734030653 learned_addr learned my addr 172.21.15.53:0/2734030653 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:38.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.417+0000 7f529989e640 1 --2- 172.21.15.53:0/2734030653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 0x7f52940b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:38.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161142719 0 0) 0x7f5284001110 con 0x7f52940c9cf0 2023-10-21T17:35:38.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5280003520 con 0x7f52940c9cf0 2023-10-21T17:35:38.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588110123 0 0) 0x7f529406d290 con 0x7f529406c3a0 2023-10-21T17:35:38.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5284001110 con 0x7f529406c3a0 2023-10-21T17:35:38.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f529989e640 1 -- 172.21.15.53:0/2734030653 >> v1:172.21.15.124:6789/0 conn(0x7f529406c3a0 legacy=0x7f529406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f529989e640 1 -- 172.21.15.53:0/2734030653 >> v1:172.21.15.53:6789/0 conn(0x7f52940c9cf0 legacy=0x7f529415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.418+0000 7f529989e640 1 -- 172.21.15.53:0/2734030653 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940c9860 con 0x7f529406db60 2023-10-21T17:35:38.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.419+0000 7f529989e640 1 --2- 172.21.15.53:0/2734030653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 0x7f52940b2a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f528c009dc0 tx=0x7f528c02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63699bd7c7480102 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:38.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.419+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c02ccc0 con 0x7f529406db60 2023-10-21T17:35:38.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.419+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c02ce20 con 0x7f529406db60 2023-10-21T17:35:38.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.419+0000 7f528bfff640 1 -- 172.21.15.53:0/2734030653 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c034660 con 0x7f529406db60 2023-10-21T17:35:38.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 -- 172.21.15.53:0/2734030653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 msgr2=0x7f52940b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 --2- 172.21.15.53:0/2734030653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 0x7f52940b2a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f528c009dc0 tx=0x7f528c02bf10 comp rx=0 tx=0).stop 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 -- 172.21.15.53:0/2734030653 shutdown_connections 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 --2- 172.21.15.53:0/2734030653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f529406db60 0x7f52940b2a90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 -- 172.21.15.53:0/2734030653 >> 172.21.15.53:0/2734030653 conn(0x7f529406a5d0 msgr2=0x7f52940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 -- 172.21.15.53:0/2734030653 shutdown_connections 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.420+0000 7f529b301640 1 -- 172.21.15.53:0/2734030653 wait complete. 2023-10-21T17:35:38.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.422+0000 7f529b301640 1 Processor -- start 2023-10-21T17:35:38.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.422+0000 7f529b301640 1 -- start start 2023-10-21T17:35:38.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:38.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f529406db60 0x7f529414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:38.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52940c9cf0 0x7f52940c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:38.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c4e90 con 0x7f529406c3a0 2023-10-21T17:35:38.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c2270 con 0x7f529406db60 2023-10-21T17:35:38.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529b301640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940c8470 con 0x7f52940c9cf0 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529909d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529909d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40318/0 (socket says 172.21.15.53:40318) 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529909d640 1 -- 172.21.15.53:0/4021565958 learned_addr learned my addr 172.21.15.53:0/4021565958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529889c640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f529406db60 0x7f529414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.423+0000 7f529989e640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52940c9cf0 0x7f52940c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:38.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 -- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52940c9cf0 msgr2=0x7f52940c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52940c9cf0 0x7f52940c2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 -- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f529406db60 msgr2=0x7f529414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:38.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f529406db60 0x7f529414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:38.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 -- 172.21.15.53:0/4021565958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009a70 con 0x7f529406c3a0 2023-10-21T17:35:38.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f529909d640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f529000c490 tx=0x7f529000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:38.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.424+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900052d0 con 0x7f529406c3a0 2023-10-21T17:35:38.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.425+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290005430 con 0x7f529406c3a0 2023-10-21T17:35:38.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.425+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52940c24c0 con 0x7f529406c3a0 2023-10-21T17:35:38.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.425+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290012870 con 0x7f529406c3a0 2023-10-21T17:35:38.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.425+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52940c2a80 con 0x7f529406c3a0 2023-10-21T17:35:38.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.426+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529404e160 con 0x7f529406c3a0 2023-10-21T17:35:38.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.427+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f52900129d0 con 0x7f529406c3a0 2023-10-21T17:35:38.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.428+0000 7f5289ffb640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 0x7f527806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:38.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.428+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f52900894f0 con 0x7f529406c3a0 2023-10-21T17:35:38.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.428+0000 7f529889c640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 0x7f527806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:38.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.429+0000 7f529889c640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 0x7f527806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52940c20f0 tx=0x7f5284007d30 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:38.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.432+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5290054270 con 0x7f529406c3a0 2023-10-21T17:35:38.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:38.639+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f529406cd60 con 0x7f529406c3a0 2023-10-21T17:35:40.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.135+0000 7f5289ffb640 1 -- 172.21.15.53:0/4021565958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v211) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5290057890 con 0x7f529406c3a0 2023-10-21T17:35:40.137 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:35:40.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 msgr2=0x7f527806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 0x7f527806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52940c20f0 tx=0x7f5284007d30 comp rx=0 tx=0).stop 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 msgr2=0x7f52940ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f529000c490 tx=0x7f529000c950 comp rx=0 tx=0).stop 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 shutdown_connections 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f527806af90 0x7f527806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52940c9cf0 0x7f52940c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f529406db60 0x7f529414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 --2- 172.21.15.53:0/4021565958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f529406c3a0 0x7f52940ca8b0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 >> 172.21.15.53:0/4021565958 conn(0x7f529406a5d0 msgr2=0x7f5294159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:40.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 shutdown_connections 2023-10-21T17:35:40.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.140+0000 7f529b301640 1 -- 172.21.15.53:0/4021565958 wait complete. 2023-10-21T17:35:40.166 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:35:40.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.390+0000 7f9baebff640 1 Processor -- start 2023-10-21T17:35:40.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.390+0000 7f9baebff640 1 -- start start 2023-10-21T17:35:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9baebff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ba815cca0 0x7f9ba81614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9baebff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba80b07d0 0x7f9ba80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9baebff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba80c9020 con 0x7f9ba80cab40 2023-10-21T17:35:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9baebff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba80ccc20 con 0x7f9ba80b07d0 2023-10-21T17:35:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9baebff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba80c92e0 con 0x7f9ba815cca0 2023-10-21T17:35:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9bae3fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba80b07d0 0x7f9ba80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9bae3fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9ba80b07d0 0x7f9ba80c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40324/0 (socket says 172.21.15.53:40324) 2023-10-21T17:35:40.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.392+0000 7f9bae3fe640 1 -- 172.21.15.53:0/2285088705 learned_addr learned my addr 172.21.15.53:0/2285088705 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:40.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.393+0000 7f9bae3fe640 1 -- 172.21.15.53:0/2285088705 >> v2:172.21.15.124:3300/0 conn(0x7f9ba815cca0 msgr2=0x7f9ba81614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:40.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.393+0000 7f9bae3fe640 1 --2- 172.21.15.53:0/2285088705 >> v2:172.21.15.124:3300/0 conn(0x7f9ba815cca0 0x7f9ba81614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.393+0000 7f9bae3fe640 1 -- 172.21.15.53:0/2285088705 >> v1:172.21.15.124:6789/0 conn(0x7f9ba80cab40 legacy=0x7f9ba80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.393+0000 7f9bae3fe640 1 -- 172.21.15.53:0/2285088705 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80cc7b0 con 0x7f9ba80b07d0 2023-10-21T17:35:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.394+0000 7f9bae3fe640 1 --2- 172.21.15.53:0/2285088705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c9770 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9ba806e650 tx=0x7f9ba002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf05badd2654b14 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.394+0000 7f9bacbfb640 1 -- 172.21.15.53:0/2285088705 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba002c960 con 0x7f9ba80b07d0 2023-10-21T17:35:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.394+0000 7f9bacbfb640 1 -- 172.21.15.53:0/2285088705 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba002cac0 con 0x7f9ba80b07d0 2023-10-21T17:35:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.394+0000 7f9bacbfb640 1 -- 172.21.15.53:0/2285088705 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba002ce50 con 0x7f9ba80b07d0 2023-10-21T17:35:40.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 -- 172.21.15.53:0/2285088705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba80b07d0 msgr2=0x7f9ba80c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 --2- 172.21.15.53:0/2285088705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c9770 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9ba806e650 tx=0x7f9ba002bdc0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 -- 172.21.15.53:0/2285088705 shutdown_connections 2023-10-21T17:35:40.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 --2- 172.21.15.53:0/2285088705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c9770 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 --2- 172.21.15.53:0/2285088705 >> v2:172.21.15.124:3300/0 conn(0x7f9ba815cca0 0x7f9ba81614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 -- 172.21.15.53:0/2285088705 >> 172.21.15.53:0/2285088705 conn(0x7f9ba80cb0c0 msgr2=0x7f9ba80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:40.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 -- 172.21.15.53:0/2285088705 shutdown_connections 2023-10-21T17:35:40.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.395+0000 7f9baebff640 1 -- 172.21.15.53:0/2285088705 wait complete. 2023-10-21T17:35:40.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.397+0000 7f9baebff640 1 Processor -- start 2023-10-21T17:35:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.397+0000 7f9baebff640 1 -- start start 2023-10-21T17:35:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba8157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c6fe0 con 0x7f9ba815cca0 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c72c0 con 0x7f9ba80b07d0 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9baebff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80c75a0 con 0x7f9ba80cab40 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.398+0000 7f9bae3fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9badbfd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bad3fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba8157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40334/0 (socket says 172.21.15.53:40334) 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 -- 172.21.15.53:0/4120892861 learned_addr learned my addr 172.21.15.53:0/4120892861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 -- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 msgr2=0x7f9ba8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba8157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 -- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba80b07d0 msgr2=0x7f9ba80c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.399+0000 7f9bae3fe640 1 -- 172.21.15.53:0/4120892861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0009b30 con 0x7f9ba815cca0 2023-10-21T17:35:40.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9bae3fe640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0036040 tx=0x7f9ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0009510 con 0x7f9ba815cca0 2023-10-21T17:35:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba003e450 con 0x7f9ba815cca0 2023-10-21T17:35:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba003e7e0 con 0x7f9ba815cca0 2023-10-21T17:35:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba80c7880 con 0x7f9ba815cca0 2023-10-21T17:35:40.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.400+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba80c85c0 con 0x7f9ba815cca0 2023-10-21T17:35:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.401+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba804e160 con 0x7f9ba815cca0 2023-10-21T17:35:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.402+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ba003e940 con 0x7f9ba815cca0 2023-10-21T17:35:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.403+0000 7f9b9e7fc640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 0x7f9b8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.403+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f9ba00ac3b0 con 0x7f9ba815cca0 2023-10-21T17:35:40.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.404+0000 7f9badbfd640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 0x7f9b8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.405+0000 7f9badbfd640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 0x7f9b8c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009990 tx=0x7f9ba4009110 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.408+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ba0077600 con 0x7f9ba815cca0 2023-10-21T17:35:40.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.669+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9ba80ae4b0 con 0x7f9ba815cca0 2023-10-21T17:35:40.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.670+0000 7f9b9e7fc640 1 -- 172.21.15.53:0/4120892861 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9ba007ac20 con 0x7f9ba815cca0 2023-10-21T17:35:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:40.672 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:35:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:35:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:35:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:35:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:35:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:35:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:35:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:35:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:35:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:35:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:35:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:35:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:35:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:35:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:35:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:35:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:35:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:35:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:35:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:35:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:35:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:35:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:35:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:35:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:35:40.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:40.702 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:35:40.702 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:35:40.702 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:35:40.702 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:35:40.703 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 msgr2=0x7f9b8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 0x7f9b8c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009990 tx=0x7f9ba4009110 comp rx=0 tx=0).stop 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 msgr2=0x7f9ba80b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0036040 tx=0x7f9ba0004a40 comp rx=0 tx=0).stop 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 shutdown_connections 2023-10-21T17:35:40.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9b8c06af90 0x7f9b8c06d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ba815cca0 0x7f9ba80b3980 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ba80cab40 0x7f9ba8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 --2- 172.21.15.53:0/4120892861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ba80b07d0 0x7f9ba80c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 >> 172.21.15.53:0/4120892861 conn(0x7f9ba80cb0c0 msgr2=0x7f9ba8070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:40.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.673+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 shutdown_connections 2023-10-21T17:35:40.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.674+0000 7f9baebff640 1 -- 172.21.15.53:0/4120892861 wait complete. 2023-10-21T17:35:40.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:40.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.936+0000 7fa32cb27640 1 Processor -- start 2023-10-21T17:35:40.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.937+0000 7fa32cb27640 1 -- start start 2023-10-21T17:35:40.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.938+0000 7fa32cb27640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa32815ab30 0x7fa32815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.938+0000 7fa32cb27640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 0x7fa3280b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.938+0000 7fa32cb27640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32815f7d0 con 0x7fa32815af10 2023-10-21T17:35:40.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.938+0000 7fa32cb27640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32815f210 con 0x7fa32815ab30 2023-10-21T17:35:40.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.938+0000 7fa32cb27640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32815f4f0 con 0x7fa3280adc70 2023-10-21T17:35:40.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa32815ab30 0x7fa32815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa32815ab30 0x7fa32815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40342/0 (socket says 172.21.15.53:40342) 2023-10-21T17:35:40.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 -- 172.21.15.53:0/980551324 learned_addr learned my addr 172.21.15.53:0/980551324 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa31effd640 1 --2- 172.21.15.53:0/980551324 >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 0x7fa3280b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 -- 172.21.15.53:0/980551324 >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 msgr2=0x7fa3280b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 --2- 172.21.15.53:0/980551324 >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 0x7fa3280b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.939+0000 7fa3277fe640 1 -- 172.21.15.53:0/980551324 >> v1:172.21.15.124:6789/0 conn(0x7fa32815af10 legacy=0x7fa3280701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:40.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.940+0000 7fa3277fe640 1 -- 172.21.15.53:0/980551324 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32806c550 con 0x7fa32815ab30 2023-10-21T17:35:40.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.940+0000 7fa31effd640 1 --2- 172.21.15.53:0/980551324 >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 0x7fa3280b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:40.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.941+0000 7fa3277fe640 1 --2- 172.21.15.53:0/980551324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32815ab30 0x7fa32815e8a0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fa32806ada0 tx=0x7fa31802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c41f2048db3a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa326ffd640 1 -- 172.21.15.53:0/980551324 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031640 con 0x7fa32815ab30 2023-10-21T17:35:40.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa326ffd640 1 -- 172.21.15.53:0/980551324 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180317a0 con 0x7fa32815ab30 2023-10-21T17:35:40.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa326ffd640 1 -- 172.21.15.53:0/980551324 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031b30 con 0x7fa32815ab30 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 -- 172.21.15.53:0/980551324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32815ab30 msgr2=0x7fa32815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 --2- 172.21.15.53:0/980551324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32815ab30 0x7fa32815e8a0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fa32806ada0 tx=0x7fa31802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 -- 172.21.15.53:0/980551324 shutdown_connections 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 --2- 172.21.15.53:0/980551324 >> v2:172.21.15.124:3300/0 conn(0x7fa3280adc70 0x7fa3280b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 --2- 172.21.15.53:0/980551324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa32815ab30 0x7fa32815e8a0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 -- 172.21.15.53:0/980551324 >> 172.21.15.53:0/980551324 conn(0x7fa3280cb0c0 msgr2=0x7fa3280cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:40.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 -- 172.21.15.53:0/980551324 shutdown_connections 2023-10-21T17:35:40.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.942+0000 7fa32cb27640 1 -- 172.21.15.53:0/980551324 wait complete. 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.944+0000 7fa32cb27640 1 Processor -- start 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.944+0000 7fa32cb27640 1 -- start start 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 0x7fa32814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 0x7fa32806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280c6d20 con 0x7fa3280adc70 2023-10-21T17:35:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32814aeb0 con 0x7fa32815ab30 2023-10-21T17:35:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.945+0000 7fa32cb27640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3280c5c80 con 0x7fa32815af10 2023-10-21T17:35:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40354/0 (socket says 172.21.15.53:40354) 2023-10-21T17:35:40.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 -- 172.21.15.53:0/3411440505 learned_addr learned my addr 172.21.15.53:0/3411440505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa327fff640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 0x7fa32806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 -- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 msgr2=0x7fa32806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa31effd640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 0x7fa32814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 0x7fa32806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 -- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 msgr2=0x7fa32814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:40.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 0x7fa32814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:40.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa3277fe640 1 -- 172.21.15.53:0/3411440505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180094d0 con 0x7fa3280adc70 2023-10-21T17:35:40.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa327fff640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 0x7fa32806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:40.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.946+0000 7fa31effd640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 0x7fa32814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:40.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa3277fe640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31803ac60 con 0x7fa3280adc70 2023-10-21T17:35:40.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31803adc0 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318044630 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3280c5f60 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.947+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3280c6520 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.948+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32804e160 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.949+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa31802f050 con 0x7fa3280adc70 2023-10-21T17:35:40.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.950+0000 7fa324ff9640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 0x7fa30806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:40.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.950+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fa3180ab830 con 0x7fa3280adc70 2023-10-21T17:35:40.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.951+0000 7fa31effd640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 0x7fa30806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:40.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.952+0000 7fa31effd640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 0x7fa30806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa310006fd0 tx=0x7fa310008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:40.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:40.955+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3180768f0 con 0x7fa3280adc70 2023-10-21T17:35:41.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.150+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3280ada50 con 0x7fa3280adc70 2023-10-21T17:35:41.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.152+0000 7fa324ff9640 1 -- 172.21.15.53:0/3411440505 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fa318032040 con 0x7fa3280adc70 2023-10-21T17:35:41.154 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:41.154 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":211,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:40.132556+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 msgr2=0x7fa30806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 0x7fa30806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa310006fd0 tx=0x7fa310008040 comp rx=0 tx=0).stop 2023-10-21T17:35:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 msgr2=0x7fa328157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fa318000c00 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 shutdown_connections 2023-10-21T17:35:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa30806afc0 0x7fa30806d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa32815af10 0x7fa32806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa32815ab30 0x7fa32814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 --2- 172.21.15.53:0/3411440505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa3280adc70 0x7fa328157ea0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 >> 172.21.15.53:0/3411440505 conn(0x7fa3280cb0c0 msgr2=0x7fa3281587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 shutdown_connections 2023-10-21T17:35:41.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.155+0000 7fa32cb27640 1 -- 172.21.15.53:0/3411440505 wait complete. 2023-10-21T17:35:41.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:35:41.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:35:41.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.399+0000 7f307759e640 1 Processor -- start 2023-10-21T17:35:41.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.399+0000 7f307759e640 1 -- start start 2023-10-21T17:35:41.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f30780d1120 0x7f30780d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780d1500 0x7f30781610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30780b5060 0x7f30780cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3078161620 con 0x7f30780b5060 2023-10-21T17:35:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30780b1fe0 con 0x7f30780d1120 2023-10-21T17:35:41.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f307759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780d0a80 con 0x7f30780d1500 2023-10-21T17:35:41.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f3076d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30780b5060 0x7f30780cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f3076d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30780b5060 0x7f30780cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40368/0 (socket says 172.21.15.53:40368) 2023-10-21T17:35:41.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f3076d9d640 1 -- 172.21.15.53:0/3341717607 learned_addr learned my addr 172.21.15.53:0/3341717607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:41.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.401+0000 7f3075d9b640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780d1500 0x7f30781610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f307659c640 1 --2- 172.21.15.53:0/3341717607 >> v2:172.21.15.124:3300/0 conn(0x7f30780d1120 0x7f30780d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f3076d9d640 1 -- 172.21.15.53:0/3341717607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780d1500 msgr2=0x7f30781610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f3076d9d640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780d1500 0x7f30781610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f3076d9d640 1 -- 172.21.15.53:0/3341717607 >> v2:172.21.15.124:3300/0 conn(0x7f30780d1120 msgr2=0x7f30780d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f3076d9d640 1 --2- 172.21.15.53:0/3341717607 >> v2:172.21.15.124:3300/0 conn(0x7f30780d1120 0x7f30780d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.402+0000 7f3076d9d640 1 -- 172.21.15.53:0/3341717607 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780b4580 con 0x7f30780b5060 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.403+0000 7f3076d9d640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780b5060 0x7f30780cd470 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f307808cc60 tx=0x7f307002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83fa9dfd65784ef server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.403+0000 7f307559a640 1 -- 172.21.15.53:0/3341717607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031740 con 0x7f30780b5060 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307559a640 1 -- 172.21.15.53:0/3341717607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700318a0 con 0x7f30780b5060 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307559a640 1 -- 172.21.15.53:0/3341717607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031c30 con 0x7f30780b5060 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 -- 172.21.15.53:0/3341717607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780b5060 msgr2=0x7f30780cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780b5060 0x7f30780cd470 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f307808cc60 tx=0x7f307002b7e0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 -- 172.21.15.53:0/3341717607 shutdown_connections 2023-10-21T17:35:41.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780d1500 0x7f30781610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 --2- 172.21.15.53:0/3341717607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780b5060 0x7f30780cd470 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 --2- 172.21.15.53:0/3341717607 >> v2:172.21.15.124:3300/0 conn(0x7f30780d1120 0x7f30780d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 -- 172.21.15.53:0/3341717607 >> 172.21.15.53:0/3341717607 conn(0x7f30780d2960 msgr2=0x7f30780d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:41.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.404+0000 7f307759e640 1 -- 172.21.15.53:0/3341717607 shutdown_connections 2023-10-21T17:35:41.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.405+0000 7f307759e640 1 -- 172.21.15.53:0/3341717607 wait complete. 2023-10-21T17:35:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.406+0000 7f307759e640 1 Processor -- start 2023-10-21T17:35:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.406+0000 7f307759e640 1 -- start start 2023-10-21T17:35:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780b5060 0x7f3078157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30780d1120 0x7f307815fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30780c61b0 con 0x7f30780d1500 2023-10-21T17:35:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30780c6490 con 0x7f30780d1120 2023-10-21T17:35:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.407+0000 7f307759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30780c6770 con 0x7f30780b5060 2023-10-21T17:35:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40382/0 (socket says 172.21.15.53:40382) 2023-10-21T17:35:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 -- 172.21.15.53:0/117653244 learned_addr learned my addr 172.21.15.53:0/117653244 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3075d9b640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30780d1120 0x7f307815fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f307659c640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780b5060 0x7f3078157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 -- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780b5060 msgr2=0x7f3078157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780b5060 0x7f3078157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 -- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30780d1120 msgr2=0x7f307815fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30780d1120 0x7f307815fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.408+0000 7f3076d9d640 1 -- 172.21.15.53:0/117653244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700094d0 con 0x7f30780d1500 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f3076d9d640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f307002b820 tx=0x7f30700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070033a40 con 0x7f30780d1500 2023-10-21T17:35:41.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070033ba0 con 0x7f30780d1500 2023-10-21T17:35:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307003d470 con 0x7f30780d1500 2023-10-21T17:35:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30780c6a50 con 0x7f30780d1500 2023-10-21T17:35:41.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.409+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3078157f60 con 0x7f30780d1500 2023-10-21T17:35:41.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.410+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30781f20b0 con 0x7f30780d1500 2023-10-21T17:35:41.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.412+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f307003a040 con 0x7f30780d1500 2023-10-21T17:35:41.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.413+0000 7f3066ffd640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 0x7f305806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.413+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f30700ab560 con 0x7f30780d1500 2023-10-21T17:35:41.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.413+0000 7f307659c640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 0x7f305806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.414+0000 7f307659c640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 0x7f305806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3068009a30 tx=0x7f30680039f0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.417+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3070076630 con 0x7f30780d1500 2023-10-21T17:35:41.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.615+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3078074150 con 0x7f30780d1500 2023-10-21T17:35:41.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.616+0000 7f3066ffd640 1 -- 172.21.15.53:0/117653244 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v211) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3070079c50 con 0x7f30780d1500 2023-10-21T17:35:41.617 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:35:41.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 msgr2=0x7f305806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 0x7f305806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3068009a30 tx=0x7f30680039f0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 msgr2=0x7f307806c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f307002b820 tx=0x7f30700046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 shutdown_connections 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f305806af90 0x7f305806d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30780d1500 0x7f307806c120 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30780d1120 0x7f307815fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 --2- 172.21.15.53:0/117653244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30780b5060 0x7f3078157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 >> 172.21.15.53:0/117653244 conn(0x7f30780d2960 msgr2=0x7f30780b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:41.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 shutdown_connections 2023-10-21T17:35:41.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.619+0000 7f307759e640 1 -- 172.21.15.53:0/117653244 wait complete. 2023-10-21T17:35:41.638 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:35:41.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.867+0000 7f9eb1832640 1 Processor -- start 2023-10-21T17:35:41.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.868+0000 7f9eb1832640 1 -- start start 2023-10-21T17:35:41.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.869+0000 7f9eb1832640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9eac0b07d0 0x7f9eac165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.869+0000 7f9eb1832640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac15f7d0 con 0x7f9eac15ab30 2023-10-21T17:35:41.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.869+0000 7f9eb1832640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac15efb0 con 0x7f9eac0adc70 2023-10-21T17:35:41.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.869+0000 7f9eb1832640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac15f210 con 0x7f9eac0b07d0 2023-10-21T17:35:41.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.870+0000 7f9eb0830640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9eac15ab30 0x7f9eac15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50854/0 (socket says 172.21.15.53:50854) 2023-10-21T17:35:41.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.870+0000 7f9eb0830640 1 -- 172.21.15.53:0/3097553025 learned_addr learned my addr 172.21.15.53:0/3097553025 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:41.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.870+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411437843 0 0) 0x7f9eac15f7d0 con 0x7f9eac15ab30 2023-10-21T17:35:41.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.870+0000 7f9eb1031640 1 --2- 172.21.15.53:0/3097553025 >> v2:172.21.15.124:3300/0 conn(0x7f9eac0b07d0 0x7f9eac165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.870+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c003520 con 0x7f9eac15ab30 2023-10-21T17:35:41.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 425103983 0 0) 0x7f9e9c003520 con 0x7f9eac15ab30 2023-10-21T17:35:41.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 >> v2:172.21.15.124:3300/0 conn(0x7f9eac0b07d0 msgr2=0x7f9eac165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:41.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 --2- 172.21.15.53:0/3097553025 >> v2:172.21.15.124:3300/0 conn(0x7f9eac0b07d0 0x7f9eac165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 >> v1:172.21.15.124:6789/0 conn(0x7f9eac0adc70 legacy=0x7f9eac0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac15f4f0 con 0x7f9eac15ab30 2023-10-21T17:35:41.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9ea0001670 con 0x7f9eac15ab30 2023-10-21T17:35:41.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 >> v1:172.21.15.53:6789/0 conn(0x7f9eac15ab30 legacy=0x7f9eac15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9e9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9c0047c0 0x7f9e9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac15f210 con 0x7f9e9c0039c0 2023-10-21T17:35:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c00c080 con 0x7f9e9c005180 2023-10-21T17:35:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c00c240 con 0x7f9e9c0047c0 2023-10-21T17:35:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eb0830640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9e9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eb1031640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.871+0000 7f9eabfff640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9c0047c0 0x7f9e9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb0830640 1 -- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9c0047c0 msgr2=0x7f9e9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:41.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb0830640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e9c0047c0 0x7f9e9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb0830640 1 -- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9c005180 msgr2=0x7f9e9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:41.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb0830640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb0830640 1 -- 172.21.15.53:0/3097553025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c003520 con 0x7f9e9c0039c0 2023-10-21T17:35:41.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.872+0000 7f9eb1031640 1 -- 172.21.15.53:0/3097553025 reap_dead start 2023-10-21T17:35:41.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.873+0000 7f9eb0830640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9e9c003de0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c0e0 tx=0x7f9e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109af85519df9a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.873+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0005ad0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.873+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0005c30 con 0x7f9e9c0039c0 2023-10-21T17:35:41.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.873+0000 7f9eab7fe640 1 -- 172.21.15.53:0/3097553025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea002ca00 con 0x7f9e9c0039c0 2023-10-21T17:35:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 -- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 msgr2=0x7f9e9c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9e9c003de0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c0e0 tx=0x7f9e9c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 -- 172.21.15.53:0/3097553025 shutdown_connections 2023-10-21T17:35:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 --2- 172.21.15.53:0/3097553025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9e9c003de0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 -- 172.21.15.53:0/3097553025 >> 172.21.15.53:0/3097553025 conn(0x7f9eac0cb0c0 msgr2=0x7f9eac0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 -- 172.21.15.53:0/3097553025 shutdown_connections 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.874+0000 7f9eb1832640 1 -- 172.21.15.53:0/3097553025 wait complete. 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.876+0000 7f9eb1832640 1 Processor -- start 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.876+0000 7f9eb1832640 1 -- start start 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9eac14fe70 0x7f9eac143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 0x7f9eac158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac143fc0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1442a0 con 0x7f9eac143a80 2023-10-21T17:35:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.877+0000 7f9eb1832640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac144580 con 0x7f9eac14fe70 2023-10-21T17:35:41.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40396/0 (socket says 172.21.15.53:40396) 2023-10-21T17:35:41.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 -- 172.21.15.53:0/2538513710 learned_addr learned my addr 172.21.15.53:0/2538513710 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:41.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eabfff640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9eac14fe70 0x7f9eac143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb1031640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 0x7f9eac158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 -- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9eac14fe70 msgr2=0x7f9eac143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9eac14fe70 0x7f9eac143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 -- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 msgr2=0x7f9eac158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 0x7f9eac158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb0830640 1 -- 172.21.15.53:0/2538513710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac144860 con 0x7f9e9c0039c0 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.878+0000 7f9eb1031640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 0x7f9eac158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:41.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9eb0830640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea003c040 con 0x7f9e9c0039c0 2023-10-21T17:35:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea002cd30 con 0x7f9e9c0039c0 2023-10-21T17:35:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0ad5a0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea002cea0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.879+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac1592a0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.881+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac1449f0 con 0x7f9e9c0039c0 2023-10-21T17:35:41.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.881+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ea0032070 con 0x7f9e9c0039c0 2023-10-21T17:35:41.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.883+0000 7f9ea97fa640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 0x7f9e9006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:41.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.883+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f9ea00aaa90 con 0x7f9e9c0039c0 2023-10-21T17:35:41.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.883+0000 7f9eabfff640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 0x7f9e9006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:41.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.885+0000 7f9eabfff640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 0x7f9e9006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e98007fd0 tx=0x7f9e98009040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:41.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:41.887+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ea0075bc0 con 0x7f9e9c0039c0 2023-10-21T17:35:42.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.083+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9eac144100 con 0x7f9e9c0039c0 2023-10-21T17:35:42.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.084+0000 7f9ea97fa640 1 -- 172.21.15.53:0/2538513710 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v211) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9eac144100 con 0x7f9e9c0039c0 2023-10-21T17:35:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:42.085 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:35:42.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:35:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:35:42.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:42.086 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 msgr2=0x7f9e9006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 0x7f9e9006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e98007fd0 tx=0x7f9e98009040 comp rx=0 tx=0).stop 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 msgr2=0x7f9eac0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000c00 tx=0x7f9ea002e070 comp rx=0 tx=0).stop 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 shutdown_connections 2023-10-21T17:35:42.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f9e9006af90 0x7f9e9006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eac143a80 0x7f9eac158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9eac14fe70 0x7f9eac143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 --2- 172.21.15.53:0/2538513710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e9c0039c0 0x7f9eac0ad060 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 >> 172.21.15.53:0/2538513710 conn(0x7f9eac0cb0c0 msgr2=0x7f9eac0af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:42.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 shutdown_connections 2023-10-21T17:35:42.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.086+0000 7f9eb1832640 1 -- 172.21.15.53:0/2538513710 wait complete. 2023-10-21T17:35:42.108 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:35:42.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:35:42.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.340+0000 7fc957fff640 1 Processor -- start 2023-10-21T17:35:42.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.340+0000 7fc957fff640 1 -- start start 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc957fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc95815aab0 0x7fc958070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc957fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95815ae90 con 0x7fc9580caef0 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc957fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9580cbdb0 con 0x7fc9580b0670 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc957fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9580cb7f0 con 0x7fc95815aab0 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc956ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc95815aab0 0x7fc958070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:42.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc956ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc95815aab0 0x7fc958070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40402/0 (socket says 172.21.15.53:40402) 2023-10-21T17:35:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc956ffd640 1 -- 172.21.15.53:0/2013660065 learned_addr learned my addr 172.21.15.53:0/2013660065 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:42.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.342+0000 7fc9567fc640 1 --1- 172.21.15.53:0/2013660065 >> v1:172.21.15.53:6789/0 conn(0x7fc9580caef0 0x7fc9580aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:50858/0 (socket says 172.21.15.53:50858) 2023-10-21T17:35:42.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.343+0000 7fc956ffd640 1 -- 172.21.15.53:0/2013660065 >> v1:172.21.15.124:6789/0 conn(0x7fc9580b0670 legacy=0x7fc9580b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:42.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.343+0000 7fc956ffd640 1 -- 172.21.15.53:0/2013660065 >> v1:172.21.15.53:6789/0 conn(0x7fc9580caef0 legacy=0x7fc9580aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.343+0000 7fc956ffd640 1 -- 172.21.15.53:0/2013660065 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9580cbad0 con 0x7fc95815aab0 2023-10-21T17:35:42.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.343+0000 7fc955ffb640 1 -- 172.21.15.53:0/2013660065 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153662687 0 0) 0x7fc940001110 con 0x7fc9580caef0 2023-10-21T17:35:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.344+0000 7fc956ffd640 1 --2- 172.21.15.53:0/2013660065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc95815aab0 0x7fc958070b50 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e80 tx=0x7fc94802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdf1331df4f77e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.344+0000 7fc955ffb640 1 -- 172.21.15.53:0/2013660065 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94802ca70 con 0x7fc95815aab0 2023-10-21T17:35:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc955ffb640 1 -- 172.21.15.53:0/2013660065 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94802cbd0 con 0x7fc95815aab0 2023-10-21T17:35:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc955ffb640 1 -- 172.21.15.53:0/2013660065 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480343f0 con 0x7fc95815aab0 2023-10-21T17:35:42.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 -- 172.21.15.53:0/2013660065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc95815aab0 msgr2=0x7fc958070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 --2- 172.21.15.53:0/2013660065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc95815aab0 0x7fc958070b50 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e80 tx=0x7fc94802bed0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 -- 172.21.15.53:0/2013660065 shutdown_connections 2023-10-21T17:35:42.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 --2- 172.21.15.53:0/2013660065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc95815aab0 0x7fc958070b50 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 -- 172.21.15.53:0/2013660065 >> 172.21.15.53:0/2013660065 conn(0x7fc95806c010 msgr2=0x7fc95806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:42.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 -- 172.21.15.53:0/2013660065 shutdown_connections 2023-10-21T17:35:42.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.345+0000 7fc957fff640 1 -- 172.21.15.53:0/2013660065 wait complete. 2023-10-21T17:35:42.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.348+0000 7fc957fff640 1 Processor -- start 2023-10-21T17:35:42.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.348+0000 7fc957fff640 1 -- start start 2023-10-21T17:35:42.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:42.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9580caef0 0x7fc9580adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95815aab0 0x7fc9580ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580c5900 con 0x7fc9580b0670 2023-10-21T17:35:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95815dee0 con 0x7fc9580caef0 2023-10-21T17:35:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.349+0000 7fc957fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580c55b0 con 0x7fc95815aab0 2023-10-21T17:35:42.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40418/0 (socket says 172.21.15.53:40418) 2023-10-21T17:35:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 -- 172.21.15.53:0/4019051550 learned_addr learned my addr 172.21.15.53:0/4019051550 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 -- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95815aab0 msgr2=0x7fc9580ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95815aab0 0x7fc9580ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 -- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9580caef0 msgr2=0x7fc9580adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9580caef0 0x7fc9580adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 -- 172.21.15.53:0/4019051550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009b30 con 0x7fc9580b0670 2023-10-21T17:35:42.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.350+0000 7fc956ffd640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e80 tx=0x7fc948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.351+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948009510 con 0x7fc9580b0670 2023-10-21T17:35:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.351+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94803e450 con 0x7fc9580b0670 2023-10-21T17:35:42.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.351+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9580c46d0 con 0x7fc9580b0670 2023-10-21T17:35:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.351+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94803e7e0 con 0x7fc9580b0670 2023-10-21T17:35:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.351+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9580c4c90 con 0x7fc9580b0670 2023-10-21T17:35:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.352+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9580c4e80 con 0x7fc9580b0670 2023-10-21T17:35:42.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.354+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc94803e940 con 0x7fc9580b0670 2023-10-21T17:35:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.355+0000 7fc94f7fe640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 0x7fc93806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.355+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fc9480ac4f0 con 0x7fc9580b0670 2023-10-21T17:35:42.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.355+0000 7fc9567fc640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 0x7fc93806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.357+0000 7fc9567fc640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 0x7fc93806d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc940007fd0 tx=0x7fc940001310 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:42.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.359+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc948077620 con 0x7fc9580b0670 2023-10-21T17:35:42.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:42.552+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc958074150 con 0x7fc9580b0670 2023-10-21T17:35:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.168+0000 7fc94f7fe640 1 -- 172.21.15.53:0/4019051550 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v212) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc9480810b0 con 0x7fc9580b0670 2023-10-21T17:35:43.169 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:35:43.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.172+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 msgr2=0x7fc93806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:43.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.172+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 0x7fc93806d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc940007fd0 tx=0x7fc940001310 comp rx=0 tx=0).stop 2023-10-21T17:35:43.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.172+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 msgr2=0x7fc95814acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:43.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.172+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc948009e80 tx=0x7fc948004a40 comp rx=0 tx=0).stop 2023-10-21T17:35:43.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 shutdown_connections 2023-10-21T17:35:43.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc93806afc0 0x7fc93806d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc95815aab0 0x7fc9580ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9580caef0 0x7fc9580adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 --2- 172.21.15.53:0/4019051550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9580b0670 0x7fc95814acd0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.173+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 >> 172.21.15.53:0/4019051550 conn(0x7fc95806c010 msgr2=0x7fc95806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:43.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.174+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 shutdown_connections 2023-10-21T17:35:43.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.174+0000 7fc957fff640 1 -- 172.21.15.53:0/4019051550 wait complete. 2023-10-21T17:35:43.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:35:43.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.412+0000 7f54992bb640 1 Processor -- start 2023-10-21T17:35:43.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.413+0000 7f54992bb640 1 -- start start 2023-10-21T17:35:43.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f54992bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f54992bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54940cdd00 0x7f54940cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f54992bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54940cfd50 con 0x7f54940cf3b0 2023-10-21T17:35:43.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f54992bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5494160800 con 0x7f54940cdd00 2023-10-21T17:35:43.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f54992bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940d09c0 con 0x7f54940b30f0 2023-10-21T17:35:43.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f5498aba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54940cdd00 0x7f54940cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f5498aba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f54940cdd00 0x7f54940cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40420/0 (socket says 172.21.15.53:40420) 2023-10-21T17:35:43.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.414+0000 7f5498aba640 1 -- 172.21.15.53:0/2713930924 learned_addr learned my addr 172.21.15.53:0/2713930924 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:43.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.415+0000 7f5493fff640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.415+0000 7f5498aba640 1 -- 172.21.15.53:0/2713930924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 msgr2=0x7f54940b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:43.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.415+0000 7f5498aba640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.415+0000 7f5498aba640 1 -- 172.21.15.53:0/2713930924 >> v1:172.21.15.124:6789/0 conn(0x7f54940cf3b0 legacy=0x7f54940cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:43.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.415+0000 7f5498aba640 1 -- 172.21.15.53:0/2713930924 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940ce190 con 0x7f54940cdd00 2023-10-21T17:35:43.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.416+0000 7f5498aba640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f54940cf810 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f548c009870 tx=0x7f548c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f31fe669c0be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:43.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.416+0000 7f54937fe640 1 -- 172.21.15.53:0/2713930924 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c031640 con 0x7f54940cdd00 2023-10-21T17:35:43.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.416+0000 7f54937fe640 1 -- 172.21.15.53:0/2713930924 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c0317a0 con 0x7f54940cdd00 2023-10-21T17:35:43.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.416+0000 7f54937fe640 1 -- 172.21.15.53:0/2713930924 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c031b30 con 0x7f54940cdd00 2023-10-21T17:35:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 -- 172.21.15.53:0/2713930924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 msgr2=0x7f54940cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f54940cf810 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f548c009870 tx=0x7f548c02b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 -- 172.21.15.53:0/2713930924 shutdown_connections 2023-10-21T17:35:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f54940cf810 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 --2- 172.21.15.53:0/2713930924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 -- 172.21.15.53:0/2713930924 >> 172.21.15.53:0/2713930924 conn(0x7f549409bcf0 msgr2=0x7f54940d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 -- 172.21.15.53:0/2713930924 shutdown_connections 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.417+0000 7f54992bb640 1 -- 172.21.15.53:0/2713930924 wait complete. 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.419+0000 7f54992bb640 1 Processor -- start 2023-10-21T17:35:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.419+0000 7f54992bb640 1 -- start start 2023-10-21T17:35:43.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54940cf3b0 0x7f54940c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940b77f0 con 0x7f54940cdd00 2023-10-21T17:35:43.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940b57e0 con 0x7f54940cf3b0 2023-10-21T17:35:43.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f54992bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54940cb060 con 0x7f54940b30f0 2023-10-21T17:35:43.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f548b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f548b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40436/0 (socket says 172.21.15.53:40436) 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.420+0000 7f548b7fe640 1 -- 172.21.15.53:0/379718856 learned_addr learned my addr 172.21.15.53:0/379718856 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f5493fff640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f5498aba640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54940cf3b0 0x7f54940c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 -- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 msgr2=0x7f54940ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 -- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54940cf3b0 msgr2=0x7f54940c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:43.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54940cf3b0 0x7f54940c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:43.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 -- 172.21.15.53:0/379718856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0094d0 con 0x7f54940cdd00 2023-10-21T17:35:43.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f5493fff640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:43.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f548b7fe640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b3f0 tx=0x7f547c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:43.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c011830 con 0x7f54940cdd00 2023-10-21T17:35:43.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c011990 con 0x7f54940cdd00 2023-10-21T17:35:43.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c011d40 con 0x7f54940cdd00 2023-10-21T17:35:43.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.421+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54940b7130 con 0x7f54940cdd00 2023-10-21T17:35:43.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.422+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494156230 con 0x7f54940cdd00 2023-10-21T17:35:43.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.422+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54940d25f0 con 0x7f54940cdd00 2023-10-21T17:35:43.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.424+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f547c004020 con 0x7f54940cdd00 2023-10-21T17:35:43.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.425+0000 7f54917fa640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 0x7f547406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:43.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.425+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f547c089970 con 0x7f54940cdd00 2023-10-21T17:35:43.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.426+0000 7f5493fff640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 0x7f547406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:43.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.427+0000 7f5493fff640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 0x7f547406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5484007fa0 tx=0x7f54840074f0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:43.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.429+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f547c054ab0 con 0x7f54940cdd00 2023-10-21T17:35:43.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:43.623+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f54940b7930 con 0x7f54940cdd00 2023-10-21T17:35:44.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.178+0000 7f54917fa640 1 -- 172.21.15.53:0/379718856 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v213) v1 ==== 170+0+0 (secure 0 0 0) 0x7f54940b7930 con 0x7f54940cdd00 2023-10-21T17:35:44.180 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:35:44.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 msgr2=0x7f547406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 0x7f547406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5484007fa0 tx=0x7f54840074f0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 msgr2=0x7f5494155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f547c00b3f0 tx=0x7f547c00b8b0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 shutdown_connections 2023-10-21T17:35:44.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f547406af90 0x7f547406d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54940cf3b0 0x7f54940c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54940cdd00 0x7f5494155cf0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 --2- 172.21.15.53:0/379718856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54940b30f0 0x7f54940ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.183+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 >> 172.21.15.53:0/379718856 conn(0x7f549409bcf0 msgr2=0x7f54940af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:44.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.184+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 shutdown_connections 2023-10-21T17:35:44.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.184+0000 7f54992bb640 1 -- 172.21.15.53:0/379718856 wait complete. 2023-10-21T17:35:44.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:35:44.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.415+0000 7f1e5359e640 1 Processor -- start 2023-10-21T17:35:44.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.415+0000 7f1e5359e640 1 -- start start 2023-10-21T17:35:44.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e540cf2b0 0x7f1e540d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1e540ce510 0x7f1e5415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540cd520 con 0x7f1e540cf690 2023-10-21T17:35:44.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540d1910 con 0x7f1e540cf2b0 2023-10-21T17:35:44.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540ce080 con 0x7f1e540ce510 2023-10-21T17:35:44.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e540cf2b0 0x7f1e540d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1e540cf2b0 0x7f1e540d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51702/0 (socket says 172.21.15.53:51702) 2023-10-21T17:35:44.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e5259c640 1 -- 172.21.15.53:0/1557212096 learned_addr learned my addr 172.21.15.53:0/1557212096 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:44.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e51d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1e540cf690 0x7f1e540d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43488/0 (socket says 172.21.15.53:43488) 2023-10-21T17:35:44.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.417+0000 7f1e52d9d640 1 --2- 172.21.15.53:0/1557212096 >> v2:172.21.15.124:3300/0 conn(0x7f1e540ce510 0x7f1e5415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.418+0000 7f1e5259c640 1 -- 172.21.15.53:0/1557212096 >> v2:172.21.15.124:3300/0 conn(0x7f1e540ce510 msgr2=0x7f1e5415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:44.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.418+0000 7f1e5259c640 1 --2- 172.21.15.53:0/1557212096 >> v2:172.21.15.124:3300/0 conn(0x7f1e540ce510 0x7f1e5415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.418+0000 7f1e5259c640 1 -- 172.21.15.53:0/1557212096 >> v1:172.21.15.53:6789/0 conn(0x7f1e540cf690 legacy=0x7f1e540d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.418+0000 7f1e5259c640 1 -- 172.21.15.53:0/1557212096 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540cd750 con 0x7f1e540cf2b0 2023-10-21T17:35:44.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.419+0000 7f1e5259c640 1 --2- 172.21.15.53:0/1557212096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf2b0 0x7f1e540d34d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cfb20 tx=0x7f1e4402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925d559311c7572b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:44.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.419+0000 7f1e5159a640 1 -- 172.21.15.53:0/1557212096 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44031880 con 0x7f1e540cf2b0 2023-10-21T17:35:44.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.419+0000 7f1e5159a640 1 -- 172.21.15.53:0/1557212096 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e440319e0 con 0x7f1e540cf2b0 2023-10-21T17:35:44.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.419+0000 7f1e5159a640 1 -- 172.21.15.53:0/1557212096 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44031d70 con 0x7f1e540cf2b0 2023-10-21T17:35:44.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 -- 172.21.15.53:0/1557212096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf2b0 msgr2=0x7f1e540d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 --2- 172.21.15.53:0/1557212096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf2b0 0x7f1e540d34d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cfb20 tx=0x7f1e4402b920 comp rx=0 tx=0).stop 2023-10-21T17:35:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 -- 172.21.15.53:0/1557212096 shutdown_connections 2023-10-21T17:35:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 --2- 172.21.15.53:0/1557212096 >> v2:172.21.15.124:3300/0 conn(0x7f1e540ce510 0x7f1e5415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 --2- 172.21.15.53:0/1557212096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf2b0 0x7f1e540d34d0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 -- 172.21.15.53:0/1557212096 >> 172.21.15.53:0/1557212096 conn(0x7f1e5409bc40 msgr2=0x7f1e540d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:44.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 -- 172.21.15.53:0/1557212096 shutdown_connections 2023-10-21T17:35:44.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.420+0000 7f1e5359e640 1 -- 172.21.15.53:0/1557212096 wait complete. 2023-10-21T17:35:44.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.422+0000 7f1e5359e640 1 Processor -- start 2023-10-21T17:35:44.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.422+0000 7f1e5359e640 1 -- start start 2023-10-21T17:35:44.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e540ce510 0x7f1e5414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e540cf2b0 0x7f1e5409cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c9b10 con 0x7f1e540cf690 2023-10-21T17:35:44.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c9df0 con 0x7f1e540ce510 2023-10-21T17:35:44.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ca0d0 con 0x7f1e540cf2b0 2023-10-21T17:35:44.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51708/0 (socket says 172.21.15.53:51708) 2023-10-21T17:35:44.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e52d9d640 1 -- 172.21.15.53:0/2555512380 learned_addr learned my addr 172.21.15.53:0/2555512380 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:44.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.423+0000 7f1e5259c640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e540ce510 0x7f1e5414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e51d9b640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e540cf2b0 0x7f1e5409cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 -- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e540cf2b0 msgr2=0x7f1e5409cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e540cf2b0 0x7f1e5409cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 -- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e540ce510 msgr2=0x7f1e5414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:44.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e540ce510 0x7f1e5414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:44.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 -- 172.21.15.53:0/2555512380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440094d0 con 0x7f1e540cf690 2023-10-21T17:35:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e52d9d640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d800 tx=0x7f1e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00b6c0 con 0x7f1e540cf690 2023-10-21T17:35:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00b840 con 0x7f1e540cf690 2023-10-21T17:35:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00bbf0 con 0x7f1e540cf690 2023-10-21T17:35:44.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.424+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540ca3b0 con 0x7f1e540cf690 2023-10-21T17:35:44.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.425+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5414d160 con 0x7f1e540cf690 2023-10-21T17:35:44.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.426+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e540ca8d0 con 0x7f1e540cf690 2023-10-21T17:35:44.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.428+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1e4c00bd50 con 0x7f1e540cf690 2023-10-21T17:35:44.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.429+0000 7f1e4affd640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 0x7f1e3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:44.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.429+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f1e4c08a720 con 0x7f1e540cf690 2023-10-21T17:35:44.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.430+0000 7f1e5259c640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 0x7f1e3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:44.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.431+0000 7f1e5259c640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 0x7f1e3806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e44036000 tx=0x7f1e4403c040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:44.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.433+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e4c0555e0 con 0x7f1e540cf690 2023-10-21T17:35:44.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:44.663+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1e540ca8d0 con 0x7f1e540cf690 2023-10-21T17:35:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.246+0000 7f1e4affd640 1 -- 172.21.15.53:0/2555512380 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1e540ca8d0 con 0x7f1e540cf690 2023-10-21T17:35:45.248 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 25 and data pool 26 2023-10-21T17:35:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 msgr2=0x7f1e3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 0x7f1e3806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1e44036000 tx=0x7f1e4403c040 comp rx=0 tx=0).stop 2023-10-21T17:35:45.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 msgr2=0x7f1e540cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d800 tx=0x7f1e4c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 shutdown_connections 2023-10-21T17:35:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f1e3806af90 0x7f1e3806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e540cf690 0x7f1e540cabb0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1e540cf2b0 0x7f1e5409cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 --2- 172.21.15.53:0/2555512380 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1e540ce510 0x7f1e5414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 >> 172.21.15.53:0/2555512380 conn(0x7f1e5409bc40 msgr2=0x7f1e540b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:45.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.251+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 shutdown_connections 2023-10-21T17:35:45.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.252+0000 7f1e5359e640 1 -- 172.21.15.53:0/2555512380 wait complete. 2023-10-21T17:35:45.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:45.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.458+0000 7f15d87a9640 1 Processor -- start 2023-10-21T17:35:45.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.458+0000 7f15d87a9640 1 -- start start 2023-10-21T17:35:45.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.459+0000 7f15d87a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d015ab40 0x7f15d00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.459+0000 7f15d87a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d015b2b0 con 0x7f15d00cb580 2023-10-21T17:35:45.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.459+0000 7f15d87a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d015b590 con 0x7f15d00c90b0 2023-10-21T17:35:45.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.459+0000 7f15d87a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d015bf30 con 0x7f15d015ab40 2023-10-21T17:35:45.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d015ab40 0x7f15d00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f15d015ab40 0x7f15d00ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51716/0 (socket says 172.21.15.53:51716) 2023-10-21T17:35:45.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 -- 172.21.15.53:0/3448892994 learned_addr learned my addr 172.21.15.53:0/3448892994 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:45.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 -- 172.21.15.53:0/3448892994 >> v1:172.21.15.124:6789/0 conn(0x7f15d00c90b0 legacy=0x7f15d00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:45.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 -- 172.21.15.53:0/3448892994 >> v1:172.21.15.53:6789/0 conn(0x7f15d00cb580 legacy=0x7f15d015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.460+0000 7f15d6545640 1 -- 172.21.15.53:0/3448892994 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d015b870 con 0x7f15d015ab40 2023-10-21T17:35:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.461+0000 7f15d6545640 1 --2- 172.21.15.53:0/3448892994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d00ad720 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e50 tx=0x7f15cc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d48dc247fb818ce server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.461+0000 7f15d5543640 1 -- 172.21.15.53:0/3448892994 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc02cb10 con 0x7f15d015ab40 2023-10-21T17:35:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.461+0000 7f15d5543640 1 -- 172.21.15.53:0/3448892994 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc02cc70 con 0x7f15d015ab40 2023-10-21T17:35:45.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d5543640 1 -- 172.21.15.53:0/3448892994 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0343f0 con 0x7f15d015ab40 2023-10-21T17:35:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 -- 172.21.15.53:0/3448892994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 msgr2=0x7f15d00ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 --2- 172.21.15.53:0/3448892994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d00ad720 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009e50 tx=0x7f15cc02bf70 comp rx=0 tx=0).stop 2023-10-21T17:35:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 -- 172.21.15.53:0/3448892994 shutdown_connections 2023-10-21T17:35:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 --2- 172.21.15.53:0/3448892994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d00ad720 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 -- 172.21.15.53:0/3448892994 >> 172.21.15.53:0/3448892994 conn(0x7f15d006a380 msgr2=0x7f15d00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 -- 172.21.15.53:0/3448892994 shutdown_connections 2023-10-21T17:35:45.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.462+0000 7f15d87a9640 1 -- 172.21.15.53:0/3448892994 wait complete. 2023-10-21T17:35:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.464+0000 7f15d87a9640 1 Processor -- start 2023-10-21T17:35:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.464+0000 7f15d87a9640 1 -- start start 2023-10-21T17:35:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d00c90b0 0x7f15d00c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d00cb580 0x7f15d0148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0149740 con 0x7f15d015ab40 2023-10-21T17:35:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b1470 con 0x7f15d00cb580 2023-10-21T17:35:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d87a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b1750 con 0x7f15d00c90b0 2023-10-21T17:35:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51724/0 (socket says 172.21.15.53:51724) 2023-10-21T17:35:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 -- 172.21.15.53:0/4086060563 learned_addr learned my addr 172.21.15.53:0/4086060563 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 -- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d00c90b0 msgr2=0x7f15d00c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d00c90b0 0x7f15d00c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 -- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d00cb580 msgr2=0x7f15d0148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.465+0000 7f15d6d46640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d00cb580 0x7f15d0148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15d6d46640 1 -- 172.21.15.53:0/4086060563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc009b30 con 0x7f15d015ab40 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15d6d46640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f15c8001eb0 tx=0x7f15c800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80052d0 con 0x7f15d015ab40 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8005430 con 0x7f15d015ab40 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8003810 con 0x7f15d015ab40 2023-10-21T17:35:45.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d00b1a30 con 0x7f15d015ab40 2023-10-21T17:35:45.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.466+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d00b1ff0 con 0x7f15d015ab40 2023-10-21T17:35:45.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.467+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d00b2200 con 0x7f15d015ab40 2023-10-21T17:35:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.468+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f15c8003970 con 0x7f15d015ab40 2023-10-21T17:35:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.469+0000 7f15c6ffd640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 0x7f15b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.469+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f15c8088f40 con 0x7f15d015ab40 2023-10-21T17:35:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.470+0000 7f15d6545640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 0x7f15b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.471+0000 7f15d6545640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 0x7f15b406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15cc035000 tx=0x7f15cc02e000 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.473+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15c8053d00 con 0x7f15d015ab40 2023-10-21T17:35:45.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.667+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d0067490 con 0x7f15d015ab40 2023-10-21T17:35:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.670+0000 7f15c6ffd640 1 -- 172.21.15.53:0/4086060563 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f15c8017090 con 0x7f15d015ab40 2023-10-21T17:35:45.672 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:45.673 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":214,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:44.664460+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 msgr2=0x7f15b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 0x7f15b406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15cc035000 tx=0x7f15cc02e000 comp rx=0 tx=0).stop 2023-10-21T17:35:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 msgr2=0x7f15d0149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f15c8001eb0 tx=0x7f15c800c890 comp rx=0 tx=0).stop 2023-10-21T17:35:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 shutdown_connections 2023-10-21T17:35:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f15b406af90 0x7f15b406d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15d015ab40 0x7f15d0149200 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15d00cb580 0x7f15d0148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 --2- 172.21.15.53:0/4086060563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15d00c90b0 0x7f15d00c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 >> 172.21.15.53:0/4086060563 conn(0x7f15d006a380 msgr2=0x7f15d00ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 shutdown_connections 2023-10-21T17:35:45.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.673+0000 7f15d87a9640 1 -- 172.21.15.53:0/4086060563 wait complete. 2023-10-21T17:35:45.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:45.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.893+0000 7f70d189f640 1 Processor -- start 2023-10-21T17:35:45.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.893+0000 7f70d189f640 1 -- start start 2023-10-21T17:35:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d189f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70cc15ab30 0x7f70cc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d189f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70cc0adc70 0x7f70cc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d189f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70cc15f7d0 con 0x7f70cc15af10 2023-10-21T17:35:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d189f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc15f210 con 0x7f70cc15ab30 2023-10-21T17:35:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d189f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc15f4f0 con 0x7f70cc0adc70 2023-10-21T17:35:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d089d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70cc15ab30 0x7f70cc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d089d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70cc15ab30 0x7f70cc15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51732/0 (socket says 172.21.15.53:51732) 2023-10-21T17:35:45.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.895+0000 7f70d089d640 1 -- 172.21.15.53:0/2074032163 learned_addr learned my addr 172.21.15.53:0/2074032163 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.896+0000 7f70d089d640 1 -- 172.21.15.53:0/2074032163 >> v2:172.21.15.124:3300/0 conn(0x7f70cc0adc70 msgr2=0x7f70cc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.896+0000 7f70d089d640 1 --2- 172.21.15.53:0/2074032163 >> v2:172.21.15.124:3300/0 conn(0x7f70cc0adc70 0x7f70cc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.896+0000 7f70d089d640 1 -- 172.21.15.53:0/2074032163 >> v1:172.21.15.124:6789/0 conn(0x7f70cc15af10 legacy=0x7f70cc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:45.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.896+0000 7f70d089d640 1 -- 172.21.15.53:0/2074032163 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc06c550 con 0x7f70cc15ab30 2023-10-21T17:35:45.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.897+0000 7f70d089d640 1 --2- 172.21.15.53:0/2074032163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15ab30 0x7f70cc15e8a0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f70cc06ada0 tx=0x7f70c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9860e8fba6a3c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.897+0000 7f70cb7fe640 1 -- 172.21.15.53:0/2074032163 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0031870 con 0x7f70cc15ab30 2023-10-21T17:35:45.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.897+0000 7f70cb7fe640 1 -- 172.21.15.53:0/2074032163 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00319d0 con 0x7f70cc15ab30 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.897+0000 7f70cb7fe640 1 -- 172.21.15.53:0/2074032163 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0031d60 con 0x7f70cc15ab30 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 -- 172.21.15.53:0/2074032163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15ab30 msgr2=0x7f70cc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 --2- 172.21.15.53:0/2074032163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15ab30 0x7f70cc15e8a0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f70cc06ada0 tx=0x7f70c002b910 comp rx=0 tx=0).stop 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 -- 172.21.15.53:0/2074032163 shutdown_connections 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 --2- 172.21.15.53:0/2074032163 >> v2:172.21.15.124:3300/0 conn(0x7f70cc0adc70 0x7f70cc0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 --2- 172.21.15.53:0/2074032163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15ab30 0x7f70cc15e8a0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 -- 172.21.15.53:0/2074032163 >> 172.21.15.53:0/2074032163 conn(0x7f70cc0cb0c0 msgr2=0x7f70cc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 -- 172.21.15.53:0/2074032163 shutdown_connections 2023-10-21T17:35:45.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.898+0000 7f70d189f640 1 -- 172.21.15.53:0/2074032163 wait complete. 2023-10-21T17:35:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.900+0000 7f70d189f640 1 Processor -- start 2023-10-21T17:35:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.900+0000 7f70d189f640 1 -- start start 2023-10-21T17:35:45.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70cc0adc70 0x7f70cc0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70cc15ab30 0x7f70cc14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc14ecd0 con 0x7f70cc15af10 2023-10-21T17:35:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0ae9b0 con 0x7f70cc0adc70 2023-10-21T17:35:45.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d189f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc14f190 con 0x7f70cc15ab30 2023-10-21T17:35:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d109e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d109e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51742/0 (socket says 172.21.15.53:51742) 2023-10-21T17:35:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d109e640 1 -- 172.21.15.53:0/3172347505 learned_addr learned my addr 172.21.15.53:0/3172347505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70cbfff640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70cc15ab30 0x7f70cc14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.901+0000 7f70d089d640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70cc0adc70 0x7f70cc0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 -- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70cc15ab30 msgr2=0x7f70cc14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70cc15ab30 0x7f70cc14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 -- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70cc0adc70 msgr2=0x7f70cc0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70cc0adc70 0x7f70cc0c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 -- 172.21.15.53:0/3172347505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c00094d0 con 0x7f70cc15af10 2023-10-21T17:35:45.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d109e640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b990 tx=0x7f70c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c400cba0 con 0x7f70cc15af10 2023-10-21T17:35:45.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c400cd00 con 0x7f70cc15af10 2023-10-21T17:35:45.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4014680 con 0x7f70cc15af10 2023-10-21T17:35:45.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc0c6f90 con 0x7f70cc15af10 2023-10-21T17:35:45.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.902+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc0c7550 con 0x7f70cc15af10 2023-10-21T17:35:45.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.903+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc04e160 con 0x7f70cc15af10 2023-10-21T17:35:45.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.904+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f70c40147e0 con 0x7f70cc15af10 2023-10-21T17:35:45.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.905+0000 7f70c97fa640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 0x7f70b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:45.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.905+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f70c4089800 con 0x7f70cc15af10 2023-10-21T17:35:45.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.906+0000 7f70d089d640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 0x7f70b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:45.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.907+0000 7f70d089d640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 0x7f70b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70c0036000 tx=0x7f70c003c040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:45.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:45.910+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70c4054670 con 0x7f70cc15af10 2023-10-21T17:35:46.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.106+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70cc0ada50 con 0x7f70cc15af10 2023-10-21T17:35:46.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.109+0000 7f70c97fa640 1 -- 172.21.15.53:0/3172347505 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f70c4019090 con 0x7f70cc15af10 2023-10-21T17:35:46.111 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:46.111 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":214,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:44.664460+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 msgr2=0x7f70b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 0x7f70b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f70c0036000 tx=0x7f70c003c040 comp rx=0 tx=0).stop 2023-10-21T17:35:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 msgr2=0x7f70cc15d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b990 tx=0x7f70c400be50 comp rx=0 tx=0).stop 2023-10-21T17:35:46.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 shutdown_connections 2023-10-21T17:35:46.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f70b006af90 0x7f70b006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70cc15af10 0x7f70cc15d710 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70cc15ab30 0x7f70cc14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 --2- 172.21.15.53:0/3172347505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70cc0adc70 0x7f70cc0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 >> 172.21.15.53:0/3172347505 conn(0x7f70cc0cb0c0 msgr2=0x7f70cc06cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:46.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 shutdown_connections 2023-10-21T17:35:46.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.112+0000 7f70d189f640 1 -- 172.21.15.53:0/3172347505 wait complete. 2023-10-21T17:35:46.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:35:46.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.350+0000 7fd36584d640 1 Processor -- start 2023-10-21T17:35:46.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.351+0000 7fd36584d640 1 -- start start 2023-10-21T17:35:46.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.352+0000 7fd36584d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd3600ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.352+0000 7fd36584d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd360065ab0 0x7fd3601608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.352+0000 7fd36584d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd360069d50 con 0x7fd36015dde0 2023-10-21T17:35:46.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.352+0000 7fd36584d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36006e6c0 con 0x7fd360065ab0 2023-10-21T17:35:46.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.352+0000 7fd36584d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006c750 con 0x7fd3600ce290 2023-10-21T17:35:46.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd36504c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd36015dde0 0x7fd360160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43508/0 (socket says 172.21.15.53:43508) 2023-10-21T17:35:46.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd36504c640 1 -- 172.21.15.53:0/3424410115 learned_addr learned my addr 172.21.15.53:0/3424410115 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:46.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd357fff640 1 --2- 172.21.15.53:0/3424410115 >> v2:172.21.15.124:3300/0 conn(0x7fd360065ab0 0x7fd3601608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd36484b640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd3600ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774915543 0 0) 0x7fd360069d50 con 0x7fd36015dde0 2023-10-21T17:35:46.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.353+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd348003520 con 0x7fd36015dde0 2023-10-21T17:35:46.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489760177 0 0) 0x7fd348003520 con 0x7fd36015dde0 2023-10-21T17:35:46.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 msgr2=0x7fd3600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:46.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd3600ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 >> v2:172.21.15.124:3300/0 conn(0x7fd360065ab0 msgr2=0x7fd3601608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:46.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 --2- 172.21.15.53:0/3424410115 >> v2:172.21.15.124:3300/0 conn(0x7fd360065ab0 0x7fd3601608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600cebb0 con 0x7fd36015dde0 2023-10-21T17:35:46.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd3580019f0 con 0x7fd36015dde0 2023-10-21T17:35:46.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 >> v1:172.21.15.53:6789/0 conn(0x7fd36015dde0 legacy=0x7fd360160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd348003970 0x7fd348003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480047c0 0x7fd348004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006c750 con 0x7fd3480047c0 2023-10-21T17:35:46.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34800b980 con 0x7fd348003970 2023-10-21T17:35:46.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34800bac0 con 0x7fd3600ce290 2023-10-21T17:35:46.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd357fff640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480047c0 0x7fd348004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.354+0000 7fd36484b640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd348003970 0x7fd348003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd36504c640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd357fff640 1 -- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 msgr2=0x7fd348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd357fff640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd357fff640 1 -- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd348003970 msgr2=0x7fd348003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:46.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd357fff640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd348003970 0x7fd348003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd357fff640 1 -- 172.21.15.53:0/3424410115 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348003520 con 0x7fd3480047c0 2023-10-21T17:35:46.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.355+0000 7fd36504c640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600ce290 0x7fd348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:46.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.356+0000 7fd357fff640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480047c0 0x7fd348004c40 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ac0 tx=0x7fd3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14fb1fe892278ba server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:46.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.356+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35002c6f0 con 0x7fd3480047c0 2023-10-21T17:35:46.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.356+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35002c850 con 0x7fd3480047c0 2023-10-21T17:35:46.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.356+0000 7fd3577fe640 1 -- 172.21.15.53:0/3424410115 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35002cbe0 con 0x7fd3480047c0 2023-10-21T17:35:46.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 -- 172.21.15.53:0/3424410115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480047c0 msgr2=0x7fd348004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 --2- 172.21.15.53:0/3424410115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480047c0 0x7fd348004c40 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ac0 tx=0x7fd3480057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36504c640 1 -- 172.21.15.53:0/3424410115 reap_dead start 2023-10-21T17:35:46.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 -- 172.21.15.53:0/3424410115 shutdown_connections 2023-10-21T17:35:46.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 -- 172.21.15.53:0/3424410115 >> 172.21.15.53:0/3424410115 conn(0x7fd3600b08d0 msgr2=0x7fd3600b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:46.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 -- 172.21.15.53:0/3424410115 shutdown_connections 2023-10-21T17:35:46.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.357+0000 7fd36584d640 1 -- 172.21.15.53:0/3424410115 wait complete. 2023-10-21T17:35:46.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.359+0000 7fd36584d640 1 Processor -- start 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.359+0000 7fd36584d640 1 -- start start 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 0x7fd3600b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 0x7fd3601f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c4440 con 0x7fd3600c2d90 2023-10-21T17:35:46.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c3360 con 0x7fd3480047c0 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36584d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600c3640 con 0x7fd3600c2910 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51758/0 (socket says 172.21.15.53:51758) 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 -- 172.21.15.53:0/1596289326 learned_addr learned my addr 172.21.15.53:0/1596289326 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36484b640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 0x7fd3600b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd357fff640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 0x7fd3601f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 -- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 msgr2=0x7fd3601f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 0x7fd3601f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 -- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 msgr2=0x7fd3600b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:46.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 0x7fd3600b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:46.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36504c640 1 -- 172.21.15.53:0/1596289326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600c4690 con 0x7fd3600c2d90 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd36484b640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 0x7fd3600b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.360+0000 7fd357fff640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 0x7fd3601f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd36504c640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ae2d0 tx=0x7fd35800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800bb80 con 0x7fd3600c2d90 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800bce0 con 0x7fd3600c2d90 2023-10-21T17:35:46.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3600c4970 con 0x7fd3600c2d90 2023-10-21T17:35:46.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358007350 con 0x7fd3600c2d90 2023-10-21T17:35:46.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.361+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3600b42a0 con 0x7fd3600c2d90 2023-10-21T17:35:46.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.363+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3600b14f0 con 0x7fd3600c2d90 2023-10-21T17:35:46.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.363+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd3580074b0 con 0x7fd3600c2d90 2023-10-21T17:35:46.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.364+0000 7fd3557fa640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 0x7fd34c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.364+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd358089e60 con 0x7fd3600c2d90 2023-10-21T17:35:46.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.365+0000 7fd36484b640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 0x7fd34c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.366+0000 7fd36484b640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 0x7fd34c06d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd35c006ce0 tx=0x7fd35c008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:46.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.369+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd358054bc0 con 0x7fd3600c2d90 2023-10-21T17:35:46.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:46.602+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd3600b4d60 con 0x7fd3600c2d90 2023-10-21T17:35:47.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.263+0000 7fd3557fa640 1 -- 172.21.15.53:0/1596289326 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd3580581e0 con 0x7fd3600c2d90 2023-10-21T17:35:47.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 msgr2=0x7fd34c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 0x7fd34c06d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd35c006ce0 tx=0x7fd35c008040 comp rx=0 tx=0).stop 2023-10-21T17:35:47.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 msgr2=0x7fd3601f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fd3600ae2d0 tx=0x7fd35800b4a0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 shutdown_connections 2023-10-21T17:35:47.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fd34c06afc0 0x7fd34c06d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3600c2d90 0x7fd3601f26d0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3600c2910 0x7fd3601f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 --2- 172.21.15.53:0/1596289326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3480047c0 0x7fd3600b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 >> 172.21.15.53:0/1596289326 conn(0x7fd3600b08d0 msgr2=0x7fd36006cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:47.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 shutdown_connections 2023-10-21T17:35:47.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.267+0000 7fd36584d640 1 -- 172.21.15.53:0/1596289326 wait complete. 2023-10-21T17:35:47.290 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:35:47.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:35:47.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.521+0000 7fc3c0ebb640 1 Processor -- start 2023-10-21T17:35:47.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.521+0000 7fc3c0ebb640 1 -- start start 2023-10-21T17:35:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3c0ebb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3c0ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0b3c50 0x7fc3bc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3c0ebb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3bc159db0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3c0ebb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc071fe0 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3c0ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0b44c0 con 0x7fc3bc0b3c50 2023-10-21T17:35:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3baffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51762/0 (socket says 172.21.15.53:51762) 2023-10-21T17:35:47.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.523+0000 7fc3baffd640 1 -- 172.21.15.53:0/3373869570 learned_addr learned my addr 172.21.15.53:0/3373869570 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.524+0000 7fc3bbfff640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0b3c50 0x7fc3bc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.524+0000 7fc3baffd640 1 -- 172.21.15.53:0/3373869570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0b3c50 msgr2=0x7fc3bc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.524+0000 7fc3baffd640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0b3c50 0x7fc3bc0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.524+0000 7fc3baffd640 1 -- 172.21.15.53:0/3373869570 >> v1:172.21.15.124:6789/0 conn(0x7fc3bc1599d0 legacy=0x7fc3bc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.524+0000 7fc3baffd640 1 -- 172.21.15.53:0/3373869570 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0713d0 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.525+0000 7fc3baffd640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000bb10 tx=0x7fc3b002e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debf09d4254409e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.525+0000 7fc3ba7fc640 1 -- 172.21.15.53:0/3373869570 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0036440 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3ba7fc640 1 -- 172.21.15.53:0/3373869570 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b00365a0 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3ba7fc640 1 -- 172.21.15.53:0/3373869570 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0036930 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/3373869570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc0cb5e0 msgr2=0x7fc3bc0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000bb10 tx=0x7fc3b002e150 comp rx=0 tx=0).stop 2023-10-21T17:35:47.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/3373869570 shutdown_connections 2023-10-21T17:35:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0b3c50 0x7fc3bc0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/3373869570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc0cb5e0 0x7fc3bc0cdcd0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.526+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/3373869570 >> 172.21.15.53:0/3373869570 conn(0x7fc3bc06c050 msgr2=0x7fc3bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.527+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/3373869570 shutdown_connections 2023-10-21T17:35:47.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.527+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/3373869570 wait complete. 2023-10-21T17:35:47.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.529+0000 7fc3c0ebb640 1 Processor -- start 2023-10-21T17:35:47.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.529+0000 7fc3c0ebb640 1 -- start start 2023-10-21T17:35:47.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3bc0b3c50 0x7fc3bc14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0cb5e0 0x7fc3bc14a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc14b0a0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc14d590 con 0x7fc3bc0b3c50 2023-10-21T17:35:47.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.530+0000 7fc3c0ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc14d870 con 0x7fc3bc0cb5e0 2023-10-21T17:35:47.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51778/0 (socket says 172.21.15.53:51778) 2023-10-21T17:35:47.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 -- 172.21.15.53:0/942800370 learned_addr learned my addr 172.21.15.53:0/942800370 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:47.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bb7fe640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3bc0b3c50 0x7fc3bc14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3baffd640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0cb5e0 0x7fc3bc14a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 -- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0cb5e0 msgr2=0x7fc3bc14a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0cb5e0 0x7fc3bc14a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 -- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3bc0b3c50 msgr2=0x7fc3bc14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:47.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3bc0b3c50 0x7fc3bc14ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.531+0000 7fc3bbfff640 1 -- 172.21.15.53:0/942800370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b000b770 con 0x7fc3bc1599d0 2023-10-21T17:35:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3bbfff640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007e40 tx=0x7fc3b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4014620 con 0x7fc3bc1599d0 2023-10-21T17:35:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b400ee90 con 0x7fc3bc1599d0 2023-10-21T17:35:47.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40149e0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc14db50 con 0x7fc3bc1599d0 2023-10-21T17:35:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.532+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc14d1f0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.533+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3bc14b1e0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.534+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc3b4014b40 con 0x7fc3bc1599d0 2023-10-21T17:35:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.535+0000 7fc3a3fff640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 0x7fc3a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.536+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc3b408a3b0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.536+0000 7fc3bb7fe640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 0x7fc3a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:47.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.537+0000 7fc3bb7fe640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 0x7fc3a806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac000f60 tx=0x7fc3ac009040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:47.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.540+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3b40550f0 con 0x7fc3bc1599d0 2023-10-21T17:35:47.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:47.775+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc3bc074150 con 0x7fc3bc1599d0 2023-10-21T17:35:48.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.272+0000 7fc3a3fff640 1 -- 172.21.15.53:0/942800370 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc3b4058710 con 0x7fc3bc1599d0 2023-10-21T17:35:48.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.275+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 msgr2=0x7fc3a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.275+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 0x7fc3a806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac000f60 tx=0x7fc3ac009040 comp rx=0 tx=0).stop 2023-10-21T17:35:48.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.275+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 msgr2=0x7fc3bc14a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.275+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007e40 tx=0x7fc3b400e4b0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 shutdown_connections 2023-10-21T17:35:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc3a806af90 0x7fc3a806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3bc1599d0 0x7fc3bc14a5a0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3bc0cb5e0 0x7fc3bc14a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 --2- 172.21.15.53:0/942800370 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3bc0b3c50 0x7fc3bc14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 >> 172.21.15.53:0/942800370 conn(0x7fc3bc06c050 msgr2=0x7fc3bc0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 shutdown_connections 2023-10-21T17:35:48.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.276+0000 7fc3c0ebb640 1 -- 172.21.15.53:0/942800370 wait complete. 2023-10-21T17:35:48.300 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:35:48.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:48.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.525+0000 7f7773fff640 1 Processor -- start 2023-10-21T17:35:48.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.525+0000 7f7773fff640 1 -- start start 2023-10-21T17:35:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7773fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77740cc210 0x7f77740b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7773fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7774159db0 0x7f77740b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7773fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77740cacf0 con 0x7f77741599d0 2023-10-21T17:35:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7773fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77740b10f0 con 0x7f7774159db0 2023-10-21T17:35:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7773fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740ade40 con 0x7f77740cc210 2023-10-21T17:35:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7772ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f77741599d0 0x7f77740ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43520/0 (socket says 172.21.15.53:43520) 2023-10-21T17:35:48.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f7772ffd640 1 -- 172.21.15.53:0/3279774816 learned_addr learned my addr 172.21.15.53:0/3279774816 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:48.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.527+0000 7f77737fe640 1 --2- 172.21.15.53:0/3279774816 >> v2:172.21.15.53:3300/0 conn(0x7f7774159db0 0x7f77740b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f77727fc640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77740cc210 0x7f77740b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:48.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f7771ffb640 1 -- 172.21.15.53:0/3279774816 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954840811 0 0) 0x7f77740cacf0 con 0x7f77741599d0 2023-10-21T17:35:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f7771ffb640 1 -- 172.21.15.53:0/3279774816 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c005730 con 0x7f77741599d0 2023-10-21T17:35:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f77737fe640 1 -- 172.21.15.53:0/3279774816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77740cc210 msgr2=0x7f77740b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f77737fe640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77740cc210 0x7f77740b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f77737fe640 1 -- 172.21.15.53:0/3279774816 >> v1:172.21.15.53:6789/0 conn(0x7f77741599d0 legacy=0x7f77740ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.528+0000 7f77737fe640 1 -- 172.21.15.53:0/3279774816 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740ca3c0 con 0x7f7774159db0 2023-10-21T17:35:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.529+0000 7f77737fe640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7774159db0 0x7f77740b0600 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f776c009d90 tx=0x7f776c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78237e9ce440321 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.529+0000 7f7771ffb640 1 -- 172.21.15.53:0/3279774816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c02cc60 con 0x7f7774159db0 2023-10-21T17:35:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7771ffb640 1 -- 172.21.15.53:0/3279774816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c02cdc0 con 0x7f7774159db0 2023-10-21T17:35:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7771ffb640 1 -- 172.21.15.53:0/3279774816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c034660 con 0x7f7774159db0 2023-10-21T17:35:48.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 -- 172.21.15.53:0/3279774816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7774159db0 msgr2=0x7f77740b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7774159db0 0x7f77740b0600 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f776c009d90 tx=0x7f776c02beb0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 -- 172.21.15.53:0/3279774816 shutdown_connections 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7774159db0 0x7f77740b0600 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 --2- 172.21.15.53:0/3279774816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77740cc210 0x7f77740b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 -- 172.21.15.53:0/3279774816 >> 172.21.15.53:0/3279774816 conn(0x7f77740cb0c0 msgr2=0x7f77740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:48.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 -- 172.21.15.53:0/3279774816 shutdown_connections 2023-10-21T17:35:48.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.530+0000 7f7773fff640 1 -- 172.21.15.53:0/3279774816 wait complete. 2023-10-21T17:35:48.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.532+0000 7f7773fff640 1 Processor -- start 2023-10-21T17:35:48.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.532+0000 7f7773fff640 1 -- start start 2023-10-21T17:35:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77741599d0 0x7f77741f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7774159db0 0x7f777415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77740c6fd0 con 0x7f77740cc210 2023-10-21T17:35:48.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741f63e0 con 0x7f77741599d0 2023-10-21T17:35:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7773fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740ae860 con 0x7f7774159db0 2023-10-21T17:35:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.533+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51786/0 (socket says 172.21.15.53:51786) 2023-10-21T17:35:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 -- 172.21.15.53:0/2131079688 learned_addr learned my addr 172.21.15.53:0/2131079688 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:48.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 -- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7774159db0 msgr2=0x7f777415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7774159db0 0x7f777415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 -- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77741599d0 msgr2=0x7f77741f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77741599d0 0x7f77741f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 -- 172.21.15.53:0/2131079688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c009a70 con 0x7f77740cc210 2023-10-21T17:35:48.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f7772ffd640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f776400c500 tx=0x7f776400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.534+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764005ce0 con 0x7f77740cc210 2023-10-21T17:35:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.535+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764005e40 con 0x7f77740cc210 2023-10-21T17:35:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.535+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764007330 con 0x7f77740cc210 2023-10-21T17:35:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.535+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741f6640 con 0x7f77740cc210 2023-10-21T17:35:48.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.535+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741f6c00 con 0x7f77740cc210 2023-10-21T17:35:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.536+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77741f6e20 con 0x7f77740cc210 2023-10-21T17:35:48.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.537+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7764007550 con 0x7f77740cc210 2023-10-21T17:35:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.538+0000 7f775b7fe640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 0x7f776006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.538+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7764088fc0 con 0x7f77740cc210 2023-10-21T17:35:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.538+0000 7f77727fc640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 0x7f776006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:48.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.539+0000 7f77727fc640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 0x7f776006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7768006fd0 tx=0x7f7768008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:48.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.543+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7764053e50 con 0x7f77740cc210 2023-10-21T17:35:48.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.777+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f777404e160 con 0x7f77740cc210 2023-10-21T17:35:48.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.778+0000 7f775b7fe640 1 -- 172.21.15.53:0/2131079688 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f7764017020 con 0x7f77740cc210 2023-10-21T17:35:48.779 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:48.780 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:35:48.780 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:35:48.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.780+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 msgr2=0x7f776006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.780+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 0x7f776006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7768006fd0 tx=0x7f7768008040 comp rx=0 tx=0).stop 2023-10-21T17:35:48.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.780+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 msgr2=0x7f777415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.780+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f776400c500 tx=0x7f776400c9c0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 shutdown_connections 2023-10-21T17:35:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f776006af90 0x7f776006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7774159db0 0x7f777415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77741599d0 0x7f77741f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 --2- 172.21.15.53:0/2131079688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77740cc210 0x7f777415a450 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:48.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 >> 172.21.15.53:0/2131079688 conn(0x7f77740cb0c0 msgr2=0x7f77740abb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:48.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 shutdown_connections 2023-10-21T17:35:48.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:48.781+0000 7f7773fff640 1 -- 172.21.15.53:0/2131079688 wait complete. 2023-10-21T17:35:48.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:35:48.873 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:35:48.882 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:35:49.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.015+0000 7f8d52773640 1 Processor -- start 2023-10-21T17:35:49.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.015+0000 7f8d52773640 1 -- start start 2023-10-21T17:35:49.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d52773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d4c1680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d52773640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c0c48f0 con 0x7f8d4c160940 2023-10-21T17:35:49.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d52773640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c0710f0 con 0x7f8d4c15cd90 2023-10-21T17:35:49.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d52773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0c4d80 con 0x7f8d4c165cc0 2023-10-21T17:35:49.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d4b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8d4c160940 0x7f8d4c0709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43526/0 (socket says 172.21.15.53:43526) 2023-10-21T17:35:49.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.017+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/2067335586 learned_addr learned my addr 172.21.15.53:0/2067335586 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:49.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d50d10640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d4c1680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898862464 0 0) 0x7f8d4c0c48f0 con 0x7f8d4c160940 2023-10-21T17:35:49.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d34003520 con 0x7f8d4c160940 2023-10-21T17:35:49.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4069620676 0 0) 0x7f8d34003520 con 0x7f8d4c160940 2023-10-21T17:35:49.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 msgr2=0x7f8d4c1680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:49.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d4c1680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 >> v1:172.21.15.124:6789/0 conn(0x7f8d4c15cd90 legacy=0x7f8d4c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0c3ce0 con 0x7f8d4c160940 2023-10-21T17:35:49.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8d40001670 con 0x7f8d4c160940 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 >> v1:172.21.15.53:6789/0 conn(0x7f8d4c160940 legacy=0x7f8d4c0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d340039a0 0x7f8d34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.018+0000 7f8d4affd640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d340047c0 0x7f8d34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4affd640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3400bab0 con 0x7f8d340047c0 2023-10-21T17:35:49.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3400bc20 con 0x7f8d340039a0 2023-10-21T17:35:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3400bde0 con 0x7f8d4c165cc0 2023-10-21T17:35:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d340047c0 0x7f8d34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d50d10640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4bfff640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d340039a0 0x7f8d34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 msgr2=0x7f8d340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4c165cc0 0x7f8d340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d340039a0 msgr2=0x7f8d34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d340039a0 0x7f8d34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/2067335586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34003520 con 0x7f8d340047c0 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.019+0000 7f8d4bfff640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d340039a0 0x7f8d34003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:49.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.020+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d340047c0 0x7f8d34004c40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c240 tx=0x7f8d340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8ee6ae6d66b66 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40005ad0 con 0x7f8d340047c0 2023-10-21T17:35:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40005c30 con 0x7f8d340047c0 2023-10-21T17:35:49.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d4affd640 1 -- 172.21.15.53:0/2067335586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4002c8b0 con 0x7f8d340047c0 2023-10-21T17:35:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 -- 172.21.15.53:0/2067335586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d340047c0 msgr2=0x7f8d34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 --2- 172.21.15.53:0/2067335586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d340047c0 0x7f8d34004c40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c240 tx=0x7f8d340057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d50d10640 1 -- 172.21.15.53:0/2067335586 reap_dead start 2023-10-21T17:35:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 -- 172.21.15.53:0/2067335586 shutdown_connections 2023-10-21T17:35:49.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 -- 172.21.15.53:0/2067335586 >> 172.21.15.53:0/2067335586 conn(0x7f8d4c06a380 msgr2=0x7f8d4c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 -- 172.21.15.53:0/2067335586 shutdown_connections 2023-10-21T17:35:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.021+0000 7f8d52773640 1 -- 172.21.15.53:0/2067335586 wait complete. 2023-10-21T17:35:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.023+0000 7f8d52773640 1 Processor -- start 2023-10-21T17:35:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.023+0000 7f8d52773640 1 -- start start 2023-10-21T17:35:49.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d340047c0 0x7f8d4c14e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d4c0afba0 0x7f8d4c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b3b80 con 0x7f8d4c0af720 2023-10-21T17:35:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b32e0 con 0x7f8d4c0afba0 2023-10-21T17:35:49.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.024+0000 7f8d52773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b35c0 con 0x7f8d340047c0 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51814/0 (socket says 172.21.15.53:51814) 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/4062080222 learned_addr learned my addr 172.21.15.53:0/4062080222 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4bfff640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d340047c0 0x7f8d4c14e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d50d10640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d4c0afba0 0x7f8d4c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d340047c0 msgr2=0x7f8d4c14e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d340047c0 0x7f8d4c14e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d4c0afba0 msgr2=0x7f8d4c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d4c0afba0 0x7f8d4c14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.025+0000 7f8d4b7fe640 1 -- 172.21.15.53:0/4062080222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0b38a0 con 0x7f8d4c0af720 2023-10-21T17:35:49.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d4b7fe640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c240 tx=0x7f8d40001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4003c040 con 0x7f8d4c0af720 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4002ccc0 con 0x7f8d4c0af720 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40007290 con 0x7f8d4c0af720 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c150180 con 0x7f8d4c0af720 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.026+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c0b04f0 con 0x7f8d4c0af720 2023-10-21T17:35:49.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.028+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c0b3700 con 0x7f8d4c0af720 2023-10-21T17:35:49.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.028+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8d40032070 con 0x7f8d4c0af720 2023-10-21T17:35:49.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.029+0000 7f8d48ff9640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 0x7f8d3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.029+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8d400aaa70 con 0x7f8d4c0af720 2023-10-21T17:35:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.030+0000 7f8d4bfff640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 0x7f8d3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.031+0000 7f8d4bfff640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 0x7f8d3806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c001750 tx=0x7f8d3c009040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.033+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d4c0b3700 con 0x7f8d4c0af720 2023-10-21T17:35:49.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.225+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d4c0b3700 con 0x7f8d4c0af720 2023-10-21T17:35:49.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.227+0000 7f8d48ff9640 1 -- 172.21.15.53:0/4062080222 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f8d4c0b3700 con 0x7f8d4c0af720 2023-10-21T17:35:49.229 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:49.229 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":215,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:35:46.198959+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:35:49.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 msgr2=0x7f8d3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 0x7f8d3806d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c001750 tx=0x7f8d3c009040 comp rx=0 tx=0).stop 2023-10-21T17:35:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 msgr2=0x7f8d4c14eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c240 tx=0x7f8d40001a00 comp rx=0 tx=0).stop 2023-10-21T17:35:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 shutdown_connections 2023-10-21T17:35:49.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f8d3806af90 0x7f8d3806d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d4c0afba0 0x7f8d4c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d4c0af720 0x7f8d4c14eda0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 --2- 172.21.15.53:0/4062080222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d340047c0 0x7f8d4c14e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 >> 172.21.15.53:0/4062080222 conn(0x7f8d4c06a380 msgr2=0x7f8d4c152320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:49.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 shutdown_connections 2023-10-21T17:35:49.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.230+0000 7f8d52773640 1 -- 172.21.15.53:0/4062080222 wait complete. 2023-10-21T17:35:49.252 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:35:49.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:35:49.262 INFO:tasks.ceph.mds.a.smithi053.stderr:2023-10-21T17:35:49.261+0000 7fabbe2bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:35:49.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.472+0000 7f0caf59e640 1 Processor -- start 2023-10-21T17:35:49.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.472+0000 7f0caf59e640 1 -- start start 2023-10-21T17:35:49.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cb015cc60 0x7f0cb00cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb00cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0cb00cdfd0 0x7f0cb00ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb015d040 con 0x7f0cb015cc60 2023-10-21T17:35:49.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb00d0650 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.473+0000 7f0caf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb00d0930 con 0x7f0cb00d16f0 2023-10-21T17:35:49.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cb015cc60 0x7f0cb00cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0cb015cc60 0x7f0cb00cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51830/0 (socket says 172.21.15.53:51830) 2023-10-21T17:35:49.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 -- 172.21.15.53:0/1910936723 learned_addr learned my addr 172.21.15.53:0/1910936723 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 -- 172.21.15.53:0/1910936723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 msgr2=0x7f0cb00cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1910936723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb00cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 -- 172.21.15.53:0/1910936723 >> v2:172.21.15.124:3300/0 conn(0x7f0cb00cdfd0 msgr2=0x7f0cb00ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1910936723 >> v2:172.21.15.124:3300/0 conn(0x7f0cb00cdfd0 0x7f0cb00ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.474+0000 7f0cae59c640 1 -- 172.21.15.53:0/1910936723 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00d0c10 con 0x7f0cb015cc60 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.475+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1910936723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb015cc60 0x7f0cb00cf340 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00cf0e0 tx=0x7f0ca002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2c5587949718ee server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0cadd9b640 1 -- 172.21.15.53:0/1910936723 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0004230 con 0x7f0cb015cc60 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0cadd9b640 1 -- 172.21.15.53:0/1910936723 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca0004390 con 0x7f0cb015cc60 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0cadd9b640 1 -- 172.21.15.53:0/1910936723 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0003810 con 0x7f0cb015cc60 2023-10-21T17:35:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 -- 172.21.15.53:0/1910936723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb015cc60 msgr2=0x7f0cb00cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1910936723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb015cc60 0x7f0cb00cf340 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00cf0e0 tx=0x7f0ca002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 -- 172.21.15.53:0/1910936723 shutdown_connections 2023-10-21T17:35:49.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1910936723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb00cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1910936723 >> v2:172.21.15.124:3300/0 conn(0x7f0cb00cdfd0 0x7f0cb00ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1910936723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb015cc60 0x7f0cb00cf340 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.476+0000 7f0caf59e640 1 -- 172.21.15.53:0/1910936723 >> 172.21.15.53:0/1910936723 conn(0x7f0cb009bcf0 msgr2=0x7f0cb00d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:49.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.477+0000 7f0caf59e640 1 -- 172.21.15.53:0/1910936723 shutdown_connections 2023-10-21T17:35:49.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.477+0000 7f0caf59e640 1 -- 172.21.15.53:0/1910936723 wait complete. 2023-10-21T17:35:49.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.478+0000 7f0caf59e640 1 Processor -- start 2023-10-21T17:35:49.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.478+0000 7f0caf59e640 1 -- start start 2023-10-21T17:35:49.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cb015cc60 0x7f0cb01f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb015d520 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb015f720 con 0x7f0cb015cc60 2023-10-21T17:35:49.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.479+0000 7f0caf59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb015fa00 con 0x7f0cb00d16f0 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51846/0 (socket says 172.21.15.53:51846) 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 -- 172.21.15.53:0/1119119540 learned_addr learned my addr 172.21.15.53:0/1119119540 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 -- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 msgr2=0x7f0cb01f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 -- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cb015cc60 msgr2=0x7f0cb01f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cb015cc60 0x7f0cb01f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 -- 172.21.15.53:0/1119119540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca0009460 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.480+0000 7f0cae59c640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f0ca002b930 tx=0x7f0ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.481+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0003c40 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.481+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca003f070 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.481+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0004500 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.481+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb015fce0 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.481+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb01f1f20 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.482+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb01f2110 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.483+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0ca0041030 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.484+0000 7f0ca77fe640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 0x7f0c9006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:49.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.484+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f0ca00abc40 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.485+0000 7f0ca5d9b640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 0x7f0c9006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:49.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.486+0000 7f0ca5d9b640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 0x7f0c9006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00cdb50 tx=0x7f0c98006bf0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:49.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.489+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cb01f2110 con 0x7f0cb00cdfd0 2023-10-21T17:35:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.680+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 --> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cb00d0ef0 con 0x7f0c9006af90 2023-10-21T17:35:49.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.709+0000 7f0ca77fe640 1 -- 172.21.15.53:0/1119119540 <== mgr.37520 v2:172.21.15.124:6832/248442090 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286153 (secure 0 0 0) 0x7f0c98012390 con 0x7f0c9006af90 2023-10-21T17:35:49.714 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:49.716 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:35:49.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.714+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 msgr2=0x7f0c9006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 0x7f0c9006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00cdb50 tx=0x7f0c98006bf0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 msgr2=0x7f0cb01f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f0ca002b930 tx=0x7f0ca00046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 shutdown_connections 2023-10-21T17:35:49.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f0c9006af90 0x7f0c9006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cb015cc60 0x7f0cb01f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cb00d16f0 0x7f0cb01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 --2- 172.21.15.53:0/1119119540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0cb00cdfd0 0x7f0cb01f19e0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 >> 172.21.15.53:0/1119119540 conn(0x7f0cb009bcf0 msgr2=0x7f0cb00b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:49.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 shutdown_connections 2023-10-21T17:35:49.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:49.715+0000 7f0caf59e640 1 -- 172.21.15.53:0/1119119540 wait complete. 2023-10-21T17:35:49.735 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-21T17:35:49.555432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":591,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4889,"ondisk_log_size":4889,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410484,"kb_used_data":22196,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754564236,"statfs":{"total":773094113280,"available":772673777664,"internally_reserved":0,"allocated":22728704,"data_stored":12269220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":397463097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002283"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206414+0000","last_change":"2023-10-21T17:35:45.234115+0000","last_active":"2023-10-21T17:35:46.206414+0000","last_peered":"2023-10-21T17:35:46.206414+0000","last_clean":"2023-10-21T17:35:46.206414+0000","last_became_active":"2023-10-21T17:35:45.233817+0000","last_became_peered":"2023-10-21T17:35:45.233817+0000","last_unstale":"2023-10-21T17:35:46.206414+0000","last_undegraded":"2023-10-21T17:35:46.206414+0000","last_fullsized":"2023-10-21T17:35:46.206414+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:48.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205979+0000","last_change":"2023-10-21T17:35:45.243702+0000","last_active":"2023-10-21T17:35:46.205979+0000","last_peered":"2023-10-21T17:35:46.205979+0000","last_clean":"2023-10-21T17:35:46.205979+0000","last_became_active":"2023-10-21T17:35:45.243449+0000","last_became_peered":"2023-10-21T17:35:45.243449+0000","last_unstale":"2023-10-21T17:35:46.205979+0000","last_undegraded":"2023-10-21T17:35:46.205979+0000","last_fullsized":"2023-10-21T17:35:46.205979+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:25.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.238897+0000","last_change":"2023-10-21T17:35:45.236275+0000","last_active":"2023-10-21T17:35:45.238897+0000","last_peered":"2023-10-21T17:35:45.238897+0000","last_clean":"2023-10-21T17:35:45.238897+0000","last_became_active":"2023-10-21T17:35:45.235963+0000","last_became_peered":"2023-10-21T17:35:45.235963+0000","last_unstale":"2023-10-21T17:35:45.238897+0000","last_undegraded":"2023-10-21T17:35:45.238897+0000","last_fullsized":"2023-10-21T17:35:45.238897+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:49:06.104591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.233829+0000","last_change":"2023-10-21T17:35:45.232026+0000","last_active":"2023-10-21T17:35:45.233829+0000","last_peered":"2023-10-21T17:35:45.233829+0000","last_clean":"2023-10-21T17:35:45.233829+0000","last_became_active":"2023-10-21T17:35:45.231712+0000","last_became_peered":"2023-10-21T17:35:45.231712+0000","last_unstale":"2023-10-21T17:35:45.233829+0000","last_undegraded":"2023-10-21T17:35:45.233829+0000","last_fullsized":"2023-10-21T17:35:45.233829+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:48.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245996+0000","last_change":"2023-10-21T17:35:45.244455+0000","last_active":"2023-10-21T17:35:45.245996+0000","last_peered":"2023-10-21T17:35:45.245996+0000","last_clean":"2023-10-21T17:35:45.245996+0000","last_became_active":"2023-10-21T17:35:45.244144+0000","last_became_peered":"2023-10-21T17:35:45.244144+0000","last_unstale":"2023-10-21T17:35:45.245996+0000","last_undegraded":"2023-10-21T17:35:45.245996+0000","last_fullsized":"2023-10-21T17:35:45.245996+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:00.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.240763+0000","last_change":"2023-10-21T17:35:45.230916+0000","last_active":"2023-10-21T17:35:45.240763+0000","last_peered":"2023-10-21T17:35:45.240763+0000","last_clean":"2023-10-21T17:35:45.240763+0000","last_became_active":"2023-10-21T17:35:45.230618+0000","last_became_peered":"2023-10-21T17:35:45.230618+0000","last_unstale":"2023-10-21T17:35:45.240763+0000","last_undegraded":"2023-10-21T17:35:45.240763+0000","last_fullsized":"2023-10-21T17:35:45.240763+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:27.144841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209588+0000","last_change":"2023-10-21T17:35:45.233455+0000","last_active":"2023-10-21T17:35:46.209588+0000","last_peered":"2023-10-21T17:35:46.209588+0000","last_clean":"2023-10-21T17:35:46.209588+0000","last_became_active":"2023-10-21T17:35:45.231067+0000","last_became_peered":"2023-10-21T17:35:45.231067+0000","last_unstale":"2023-10-21T17:35:46.209588+0000","last_undegraded":"2023-10-21T17:35:46.209588+0000","last_fullsized":"2023-10-21T17:35:46.209588+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:56.440925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245160+0000","last_change":"2023-10-21T17:35:45.241868+0000","last_active":"2023-10-21T17:35:45.245160+0000","last_peered":"2023-10-21T17:35:45.245160+0000","last_clean":"2023-10-21T17:35:45.245160+0000","last_became_active":"2023-10-21T17:35:45.241344+0000","last_became_peered":"2023-10-21T17:35:45.241344+0000","last_unstale":"2023-10-21T17:35:45.245160+0000","last_undegraded":"2023-10-21T17:35:45.245160+0000","last_fullsized":"2023-10-21T17:35:45.245160+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:15.800521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206551+0000","last_change":"2023-10-21T17:35:45.238404+0000","last_active":"2023-10-21T17:35:46.206551+0000","last_peered":"2023-10-21T17:35:46.206551+0000","last_clean":"2023-10-21T17:35:46.206551+0000","last_became_active":"2023-10-21T17:35:45.237467+0000","last_became_peered":"2023-10-21T17:35:45.237467+0000","last_unstale":"2023-10-21T17:35:46.206551+0000","last_undegraded":"2023-10-21T17:35:46.206551+0000","last_fullsized":"2023-10-21T17:35:46.206551+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:57:17.400080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242122+0000","last_change":"2023-10-21T17:35:45.238905+0000","last_active":"2023-10-21T17:35:45.242122+0000","last_peered":"2023-10-21T17:35:45.242122+0000","last_clean":"2023-10-21T17:35:45.242122+0000","last_became_active":"2023-10-21T17:35:45.238621+0000","last_became_peered":"2023-10-21T17:35:45.238621+0000","last_unstale":"2023-10-21T17:35:45.242122+0000","last_undegraded":"2023-10-21T17:35:45.242122+0000","last_fullsized":"2023-10-21T17:35:45.242122+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:02.923295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241795+0000","last_change":"2023-10-21T17:35:45.233757+0000","last_active":"2023-10-21T17:35:45.241795+0000","last_peered":"2023-10-21T17:35:45.241795+0000","last_clean":"2023-10-21T17:35:45.241795+0000","last_became_active":"2023-10-21T17:35:45.233187+0000","last_became_peered":"2023-10-21T17:35:45.233187+0000","last_unstale":"2023-10-21T17:35:45.241795+0000","last_undegraded":"2023-10-21T17:35:45.241795+0000","last_fullsized":"2023-10-21T17:35:45.241795+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:50.852724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242612+0000","last_change":"2023-10-21T17:35:45.239715+0000","last_active":"2023-10-21T17:35:45.242612+0000","last_peered":"2023-10-21T17:35:45.242612+0000","last_clean":"2023-10-21T17:35:45.242612+0000","last_became_active":"2023-10-21T17:35:45.239435+0000","last_became_peered":"2023-10-21T17:35:45.239435+0000","last_unstale":"2023-10-21T17:35:45.242612+0000","last_undegraded":"2023-10-21T17:35:45.242612+0000","last_fullsized":"2023-10-21T17:35:45.242612+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.274330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.208983+0000","last_change":"2023-10-21T17:35:45.238625+0000","last_active":"2023-10-21T17:35:46.208983+0000","last_peered":"2023-10-21T17:35:46.208983+0000","last_clean":"2023-10-21T17:35:46.208983+0000","last_became_active":"2023-10-21T17:35:45.238226+0000","last_became_peered":"2023-10-21T17:35:45.238226+0000","last_unstale":"2023-10-21T17:35:46.208983+0000","last_undegraded":"2023-10-21T17:35:46.208983+0000","last_fullsized":"2023-10-21T17:35:46.208983+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:42.947024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209672+0000","last_change":"2023-10-21T17:35:45.221930+0000","last_active":"2023-10-21T17:35:46.209672+0000","last_peered":"2023-10-21T17:35:46.209672+0000","last_clean":"2023-10-21T17:35:46.209672+0000","last_became_active":"2023-10-21T17:35:45.221549+0000","last_became_peered":"2023-10-21T17:35:45.221549+0000","last_unstale":"2023-10-21T17:35:46.209672+0000","last_undegraded":"2023-10-21T17:35:46.209672+0000","last_fullsized":"2023-10-21T17:35:46.209672+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:38.535179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205677+0000","last_change":"2023-10-21T17:35:45.227671+0000","last_active":"2023-10-21T17:35:46.205677+0000","last_peered":"2023-10-21T17:35:46.205677+0000","last_clean":"2023-10-21T17:35:46.205677+0000","last_became_active":"2023-10-21T17:35:45.223658+0000","last_became_peered":"2023-10-21T17:35:45.223658+0000","last_unstale":"2023-10-21T17:35:46.205677+0000","last_undegraded":"2023-10-21T17:35:46.205677+0000","last_fullsized":"2023-10-21T17:35:46.205677+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:49:41.942050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.246550+0000","last_change":"2023-10-21T17:35:45.245067+0000","last_active":"2023-10-21T17:35:45.246550+0000","last_peered":"2023-10-21T17:35:45.246550+0000","last_clean":"2023-10-21T17:35:45.246550+0000","last_became_active":"2023-10-21T17:35:45.244422+0000","last_became_peered":"2023-10-21T17:35:45.244422+0000","last_unstale":"2023-10-21T17:35:45.246550+0000","last_undegraded":"2023-10-21T17:35:45.246550+0000","last_fullsized":"2023-10-21T17:35:45.246550+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:33.447916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.246259+0000","last_change":"2023-10-21T17:35:45.245199+0000","last_active":"2023-10-21T17:35:45.246259+0000","last_peered":"2023-10-21T17:35:45.246259+0000","last_clean":"2023-10-21T17:35:45.246259+0000","last_became_active":"2023-10-21T17:35:45.243823+0000","last_became_peered":"2023-10-21T17:35:45.243823+0000","last_unstale":"2023-10-21T17:35:45.246259+0000","last_undegraded":"2023-10-21T17:35:45.246259+0000","last_fullsized":"2023-10-21T17:35:45.246259+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:57.462655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209942+0000","last_change":"2023-10-21T17:35:45.232181+0000","last_active":"2023-10-21T17:35:46.209942+0000","last_peered":"2023-10-21T17:35:46.209942+0000","last_clean":"2023-10-21T17:35:46.209942+0000","last_became_active":"2023-10-21T17:35:45.230703+0000","last_became_peered":"2023-10-21T17:35:45.230703+0000","last_unstale":"2023-10-21T17:35:46.209942+0000","last_undegraded":"2023-10-21T17:35:46.209942+0000","last_fullsized":"2023-10-21T17:35:46.209942+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:08.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209129+0000","last_change":"2023-10-21T17:35:45.231754+0000","last_active":"2023-10-21T17:35:46.209129+0000","last_peered":"2023-10-21T17:35:46.209129+0000","last_clean":"2023-10-21T17:35:46.209129+0000","last_became_active":"2023-10-21T17:35:45.230978+0000","last_became_peered":"2023-10-21T17:35:45.230978+0000","last_unstale":"2023-10-21T17:35:46.209129+0000","last_undegraded":"2023-10-21T17:35:46.209129+0000","last_fullsized":"2023-10-21T17:35:46.209129+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:08.607825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245537+0000","last_change":"2023-10-21T17:35:45.243528+0000","last_active":"2023-10-21T17:35:45.245537+0000","last_peered":"2023-10-21T17:35:45.245537+0000","last_clean":"2023-10-21T17:35:45.245537+0000","last_became_active":"2023-10-21T17:35:45.243269+0000","last_became_peered":"2023-10-21T17:35:45.243269+0000","last_unstale":"2023-10-21T17:35:45.245537+0000","last_undegraded":"2023-10-21T17:35:45.245537+0000","last_fullsized":"2023-10-21T17:35:45.245537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:33:12.456009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205997+0000","last_change":"2023-10-21T17:35:45.239010+0000","last_active":"2023-10-21T17:35:46.205997+0000","last_peered":"2023-10-21T17:35:46.205997+0000","last_clean":"2023-10-21T17:35:46.205997+0000","last_became_active":"2023-10-21T17:35:45.237970+0000","last_became_peered":"2023-10-21T17:35:45.237970+0000","last_unstale":"2023-10-21T17:35:46.205997+0000","last_undegraded":"2023-10-21T17:35:46.205997+0000","last_fullsized":"2023-10-21T17:35:46.205997+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.274330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205429+0000","last_change":"2023-10-21T17:35:45.230336+0000","last_active":"2023-10-21T17:35:46.205429+0000","last_peered":"2023-10-21T17:35:46.205429+0000","last_clean":"2023-10-21T17:35:46.205429+0000","last_became_active":"2023-10-21T17:35:45.230040+0000","last_became_peered":"2023-10-21T17:35:45.230040+0000","last_unstale":"2023-10-21T17:35:46.205429+0000","last_undegraded":"2023-10-21T17:35:46.205429+0000","last_fullsized":"2023-10-21T17:35:46.205429+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:47:11.413681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244168+0000","last_change":"2023-10-21T17:35:45.240746+0000","last_active":"2023-10-21T17:35:45.244168+0000","last_peered":"2023-10-21T17:35:45.244168+0000","last_clean":"2023-10-21T17:35:45.244168+0000","last_became_active":"2023-10-21T17:35:45.240472+0000","last_became_peered":"2023-10-21T17:35:45.240472+0000","last_unstale":"2023-10-21T17:35:45.244168+0000","last_undegraded":"2023-10-21T17:35:45.244168+0000","last_fullsized":"2023-10-21T17:35:45.244168+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:30:26.460976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205090+0000","last_change":"2023-10-21T17:35:45.235725+0000","last_active":"2023-10-21T17:35:46.205090+0000","last_peered":"2023-10-21T17:35:46.205090+0000","last_clean":"2023-10-21T17:35:46.205090+0000","last_became_active":"2023-10-21T17:35:45.235351+0000","last_became_peered":"2023-10-21T17:35:45.235351+0000","last_unstale":"2023-10-21T17:35:46.205090+0000","last_undegraded":"2023-10-21T17:35:46.205090+0000","last_fullsized":"2023-10-21T17:35:46.205090+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:19:10.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209634+0000","last_change":"2023-10-21T17:35:45.244543+0000","last_active":"2023-10-21T17:35:46.209634+0000","last_peered":"2023-10-21T17:35:46.209634+0000","last_clean":"2023-10-21T17:35:46.209634+0000","last_became_active":"2023-10-21T17:35:45.244149+0000","last_became_peered":"2023-10-21T17:35:45.244149+0000","last_unstale":"2023-10-21T17:35:46.209634+0000","last_undegraded":"2023-10-21T17:35:46.209634+0000","last_fullsized":"2023-10-21T17:35:46.209634+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:41.440754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241928+0000","last_change":"2023-10-21T17:35:45.231867+0000","last_active":"2023-10-21T17:35:45.241928+0000","last_peered":"2023-10-21T17:35:45.241928+0000","last_clean":"2023-10-21T17:35:45.241928+0000","last_became_active":"2023-10-21T17:35:45.231607+0000","last_became_peered":"2023-10-21T17:35:45.231607+0000","last_unstale":"2023-10-21T17:35:45.241928+0000","last_undegraded":"2023-10-21T17:35:45.241928+0000","last_fullsized":"2023-10-21T17:35:45.241928+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:24.285732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241543+0000","last_change":"2023-10-21T17:35:45.234489+0000","last_active":"2023-10-21T17:35:45.241543+0000","last_peered":"2023-10-21T17:35:45.241543+0000","last_clean":"2023-10-21T17:35:45.241543+0000","last_became_active":"2023-10-21T17:35:45.234203+0000","last_became_peered":"2023-10-21T17:35:45.234203+0000","last_unstale":"2023-10-21T17:35:45.241543+0000","last_undegraded":"2023-10-21T17:35:45.241543+0000","last_fullsized":"2023-10-21T17:35:45.241543+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:51.322249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245421+0000","last_change":"2023-10-21T17:35:45.242108+0000","last_active":"2023-10-21T17:35:45.245421+0000","last_peered":"2023-10-21T17:35:45.245421+0000","last_clean":"2023-10-21T17:35:45.245421+0000","last_became_active":"2023-10-21T17:35:45.241824+0000","last_became_peered":"2023-10-21T17:35:45.241824+0000","last_unstale":"2023-10-21T17:35:45.245421+0000","last_undegraded":"2023-10-21T17:35:45.245421+0000","last_fullsized":"2023-10-21T17:35:45.245421+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:16.591606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241840+0000","last_change":"2023-10-21T17:35:45.239001+0000","last_active":"2023-10-21T17:35:45.241840+0000","last_peered":"2023-10-21T17:35:45.241840+0000","last_clean":"2023-10-21T17:35:45.241840+0000","last_became_active":"2023-10-21T17:35:45.238729+0000","last_became_peered":"2023-10-21T17:35:45.238729+0000","last_unstale":"2023-10-21T17:35:45.241840+0000","last_undegraded":"2023-10-21T17:35:45.241840+0000","last_fullsized":"2023-10-21T17:35:45.241840+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:30:26.460976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.240307+0000","last_change":"2023-10-21T17:35:45.230074+0000","last_active":"2023-10-21T17:35:45.240307+0000","last_peered":"2023-10-21T17:35:45.240307+0000","last_clean":"2023-10-21T17:35:45.240307+0000","last_became_active":"2023-10-21T17:35:45.229623+0000","last_became_peered":"2023-10-21T17:35:45.229623+0000","last_unstale":"2023-10-21T17:35:45.240307+0000","last_undegraded":"2023-10-21T17:35:45.240307+0000","last_fullsized":"2023-10-21T17:35:45.240307+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:57.030355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209885+0000","last_change":"2023-10-21T17:35:45.229224+0000","last_active":"2023-10-21T17:35:46.209885+0000","last_peered":"2023-10-21T17:35:46.209885+0000","last_clean":"2023-10-21T17:35:46.209885+0000","last_became_active":"2023-10-21T17:35:45.226585+0000","last_became_peered":"2023-10-21T17:35:45.226585+0000","last_unstale":"2023-10-21T17:35:46.209885+0000","last_undegraded":"2023-10-21T17:35:46.209885+0000","last_fullsized":"2023-10-21T17:35:46.209885+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:28.509289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244955+0000","last_change":"2023-10-21T17:35:45.240863+0000","last_active":"2023-10-21T17:35:45.244955+0000","last_peered":"2023-10-21T17:35:45.244955+0000","last_clean":"2023-10-21T17:35:45.244955+0000","last_became_active":"2023-10-21T17:35:45.240597+0000","last_became_peered":"2023-10-21T17:35:45.240597+0000","last_unstale":"2023-10-21T17:35:45.244955+0000","last_undegraded":"2023-10-21T17:35:45.244955+0000","last_fullsized":"2023-10-21T17:35:45.244955+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:29:28.861719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242566+0000","last_change":"2023-10-21T17:35:45.234278+0000","last_active":"2023-10-21T17:35:45.242566+0000","last_peered":"2023-10-21T17:35:45.242566+0000","last_clean":"2023-10-21T17:35:45.242566+0000","last_became_active":"2023-10-21T17:35:45.233439+0000","last_became_peered":"2023-10-21T17:35:45.233439+0000","last_unstale":"2023-10-21T17:35:45.242566+0000","last_undegraded":"2023-10-21T17:35:45.242566+0000","last_fullsized":"2023-10-21T17:35:45.242566+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:12:37.578306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"214'4","reported_seq":15,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261273+0000","last_change":"2023-10-21T17:35:44.220480+0000","last_active":"2023-10-21T17:35:45.261273+0000","last_peered":"2023-10-21T17:35:45.261273+0000","last_clean":"2023-10-21T17:35:45.261273+0000","last_became_active":"2023-10-21T17:35:44.220222+0000","last_became_peered":"2023-10-21T17:35:44.220222+0000","last_unstale":"2023-10-21T17:35:45.261273+0000","last_undegraded":"2023-10-21T17:35:45.261273+0000","last_fullsized":"2023-10-21T17:35:45.261273+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:39:20.044047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209884+0000","last_change":"2023-10-21T17:35:45.234533+0000","last_active":"2023-10-21T17:35:46.209884+0000","last_peered":"2023-10-21T17:35:46.209884+0000","last_clean":"2023-10-21T17:35:46.209884+0000","last_became_active":"2023-10-21T17:35:45.234225+0000","last_became_peered":"2023-10-21T17:35:45.234225+0000","last_unstale":"2023-10-21T17:35:46.209884+0000","last_undegraded":"2023-10-21T17:35:46.209884+0000","last_fullsized":"2023-10-21T17:35:46.209884+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:48.532550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261315+0000","last_change":"2023-10-21T17:35:44.223103+0000","last_active":"2023-10-21T17:35:45.261315+0000","last_peered":"2023-10-21T17:35:45.261315+0000","last_clean":"2023-10-21T17:35:45.261315+0000","last_became_active":"2023-10-21T17:35:44.216646+0000","last_became_peered":"2023-10-21T17:35:44.216646+0000","last_unstale":"2023-10-21T17:35:45.261315+0000","last_undegraded":"2023-10-21T17:35:45.261315+0000","last_fullsized":"2023-10-21T17:35:45.261315+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:46:48.139508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241968+0000","last_change":"2023-10-21T17:35:45.237596+0000","last_active":"2023-10-21T17:35:45.241968+0000","last_peered":"2023-10-21T17:35:45.241968+0000","last_clean":"2023-10-21T17:35:45.241968+0000","last_became_active":"2023-10-21T17:35:45.235744+0000","last_became_peered":"2023-10-21T17:35:45.235744+0000","last_unstale":"2023-10-21T17:35:45.241968+0000","last_undegraded":"2023-10-21T17:35:45.241968+0000","last_fullsized":"2023-10-21T17:35:45.241968+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:43:40.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.259484+0000","last_change":"2023-10-21T17:35:44.204037+0000","last_active":"2023-10-21T17:35:45.259484+0000","last_peered":"2023-10-21T17:35:45.259484+0000","last_clean":"2023-10-21T17:35:45.259484+0000","last_became_active":"2023-10-21T17:35:44.203742+0000","last_became_peered":"2023-10-21T17:35:44.203742+0000","last_unstale":"2023-10-21T17:35:45.259484+0000","last_undegraded":"2023-10-21T17:35:45.259484+0000","last_fullsized":"2023-10-21T17:35:45.259484+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:09.315964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.243919+0000","last_change":"2023-10-21T17:35:45.242387+0000","last_active":"2023-10-21T17:35:45.243919+0000","last_peered":"2023-10-21T17:35:45.243919+0000","last_clean":"2023-10-21T17:35:45.243919+0000","last_became_active":"2023-10-21T17:35:45.242061+0000","last_became_peered":"2023-10-21T17:35:45.242061+0000","last_unstale":"2023-10-21T17:35:45.243919+0000","last_undegraded":"2023-10-21T17:35:45.243919+0000","last_fullsized":"2023-10-21T17:35:45.243919+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:07.662607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209465+0000","last_change":"2023-10-21T17:35:44.203338+0000","last_active":"2023-10-21T17:35:46.209465+0000","last_peered":"2023-10-21T17:35:46.209465+0000","last_clean":"2023-10-21T17:35:46.209465+0000","last_became_active":"2023-10-21T17:35:44.202951+0000","last_became_peered":"2023-10-21T17:35:44.202951+0000","last_unstale":"2023-10-21T17:35:46.209465+0000","last_undegraded":"2023-10-21T17:35:46.209465+0000","last_fullsized":"2023-10-21T17:35:46.209465+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:34.813742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.208441+0000","last_change":"2023-10-21T17:35:45.239493+0000","last_active":"2023-10-21T17:35:46.208441+0000","last_peered":"2023-10-21T17:35:46.208441+0000","last_clean":"2023-10-21T17:35:46.208441+0000","last_became_active":"2023-10-21T17:35:45.239016+0000","last_became_peered":"2023-10-21T17:35:45.239016+0000","last_unstale":"2023-10-21T17:35:46.208441+0000","last_undegraded":"2023-10-21T17:35:46.208441+0000","last_fullsized":"2023-10-21T17:35:46.208441+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:16.531282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261013+0000","last_change":"2023-10-21T17:35:44.213731+0000","last_active":"2023-10-21T17:35:45.261013+0000","last_peered":"2023-10-21T17:35:45.261013+0000","last_clean":"2023-10-21T17:35:45.261013+0000","last_became_active":"2023-10-21T17:35:44.213257+0000","last_became_peered":"2023-10-21T17:35:44.213257+0000","last_unstale":"2023-10-21T17:35:45.261013+0000","last_undegraded":"2023-10-21T17:35:45.261013+0000","last_fullsized":"2023-10-21T17:35:45.261013+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:23.614892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205851+0000","last_change":"2023-10-21T17:35:45.225488+0000","last_active":"2023-10-21T17:35:46.205851+0000","last_peered":"2023-10-21T17:35:46.205851+0000","last_clean":"2023-10-21T17:35:46.205851+0000","last_became_active":"2023-10-21T17:35:45.225186+0000","last_became_peered":"2023-10-21T17:35:45.225186+0000","last_unstale":"2023-10-21T17:35:46.205851+0000","last_undegraded":"2023-10-21T17:35:46.205851+0000","last_fullsized":"2023-10-21T17:35:46.205851+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:29.398804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.257627+0000","last_change":"2023-10-21T17:35:44.221936+0000","last_active":"2023-10-21T17:35:45.257627+0000","last_peered":"2023-10-21T17:35:45.257627+0000","last_clean":"2023-10-21T17:35:45.257627+0000","last_became_active":"2023-10-21T17:35:44.221652+0000","last_became_peered":"2023-10-21T17:35:44.221652+0000","last_unstale":"2023-10-21T17:35:45.257627+0000","last_undegraded":"2023-10-21T17:35:45.257627+0000","last_fullsized":"2023-10-21T17:35:45.257627+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:24.373341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205742+0000","last_change":"2023-10-21T17:35:45.226625+0000","last_active":"2023-10-21T17:35:46.205742+0000","last_peered":"2023-10-21T17:35:46.205742+0000","last_clean":"2023-10-21T17:35:46.205742+0000","last_became_active":"2023-10-21T17:35:45.226339+0000","last_became_peered":"2023-10-21T17:35:45.226339+0000","last_unstale":"2023-10-21T17:35:46.205742+0000","last_undegraded":"2023-10-21T17:35:46.205742+0000","last_fullsized":"2023-10-21T17:35:46.205742+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:34.903077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209475+0000","last_change":"2023-10-21T17:35:44.213771+0000","last_active":"2023-10-21T17:35:46.209475+0000","last_peered":"2023-10-21T17:35:46.209475+0000","last_clean":"2023-10-21T17:35:46.209475+0000","last_became_active":"2023-10-21T17:35:44.213506+0000","last_became_peered":"2023-10-21T17:35:44.213506+0000","last_unstale":"2023-10-21T17:35:46.209475+0000","last_undegraded":"2023-10-21T17:35:46.209475+0000","last_fullsized":"2023-10-21T17:35:46.209475+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:11:04.186955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242586+0000","last_change":"2023-10-21T17:35:45.235426+0000","last_active":"2023-10-21T17:35:45.242586+0000","last_peered":"2023-10-21T17:35:45.242586+0000","last_clean":"2023-10-21T17:35:45.242586+0000","last_became_active":"2023-10-21T17:35:45.233025+0000","last_became_peered":"2023-10-21T17:35:45.233025+0000","last_unstale":"2023-10-21T17:35:45.242586+0000","last_undegraded":"2023-10-21T17:35:45.242586+0000","last_fullsized":"2023-10-21T17:35:45.242586+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:26:35.352137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.236231+0000","last_change":"2023-10-21T17:35:44.224883+0000","last_active":"2023-10-21T17:35:45.236231+0000","last_peered":"2023-10-21T17:35:45.236231+0000","last_clean":"2023-10-21T17:35:45.236231+0000","last_became_active":"2023-10-21T17:35:44.224609+0000","last_became_peered":"2023-10-21T17:35:44.224609+0000","last_unstale":"2023-10-21T17:35:45.236231+0000","last_undegraded":"2023-10-21T17:35:45.236231+0000","last_fullsized":"2023-10-21T17:35:45.236231+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:55:01.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.239980+0000","last_change":"2023-10-21T17:35:45.231159+0000","last_active":"2023-10-21T17:35:45.239980+0000","last_peered":"2023-10-21T17:35:45.239980+0000","last_clean":"2023-10-21T17:35:45.239980+0000","last_became_active":"2023-10-21T17:35:45.230017+0000","last_became_peered":"2023-10-21T17:35:45.230017+0000","last_unstale":"2023-10-21T17:35:45.239980+0000","last_undegraded":"2023-10-21T17:35:45.239980+0000","last_fullsized":"2023-10-21T17:35:45.239980+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:13.873224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206194+0000","last_change":"2023-10-21T17:35:44.221513+0000","last_active":"2023-10-21T17:35:46.206194+0000","last_peered":"2023-10-21T17:35:46.206194+0000","last_clean":"2023-10-21T17:35:46.206194+0000","last_became_active":"2023-10-21T17:35:44.221214+0000","last_became_peered":"2023-10-21T17:35:44.221214+0000","last_unstale":"2023-10-21T17:35:46.206194+0000","last_undegraded":"2023-10-21T17:35:46.206194+0000","last_fullsized":"2023-10-21T17:35:46.206194+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:28.155784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.210034+0000","last_change":"2023-10-21T17:35:45.243652+0000","last_active":"2023-10-21T17:35:46.210034+0000","last_peered":"2023-10-21T17:35:46.210034+0000","last_clean":"2023-10-21T17:35:46.210034+0000","last_became_active":"2023-10-21T17:35:45.243330+0000","last_became_peered":"2023-10-21T17:35:45.243330+0000","last_unstale":"2023-10-21T17:35:46.210034+0000","last_undegraded":"2023-10-21T17:35:46.210034+0000","last_fullsized":"2023-10-21T17:35:46.210034+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:41:12.344281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"214'3","reported_seq":14,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260410+0000","last_change":"2023-10-21T17:35:44.227159+0000","last_active":"2023-10-21T17:35:45.260410+0000","last_peered":"2023-10-21T17:35:45.260410+0000","last_clean":"2023-10-21T17:35:45.260410+0000","last_became_active":"2023-10-21T17:35:44.226068+0000","last_became_peered":"2023-10-21T17:35:44.226068+0000","last_unstale":"2023-10-21T17:35:45.260410+0000","last_undegraded":"2023-10-21T17:35:45.260410+0000","last_fullsized":"2023-10-21T17:35:45.260410+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:33.886176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206827+0000","last_change":"2023-10-21T17:35:45.242375+0000","last_active":"2023-10-21T17:35:46.206827+0000","last_peered":"2023-10-21T17:35:46.206827+0000","last_clean":"2023-10-21T17:35:46.206827+0000","last_became_active":"2023-10-21T17:35:45.241990+0000","last_became_peered":"2023-10-21T17:35:45.241990+0000","last_unstale":"2023-10-21T17:35:46.206827+0000","last_undegraded":"2023-10-21T17:35:46.206827+0000","last_fullsized":"2023-10-21T17:35:46.206827+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:25.390242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"214'6","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209228+0000","last_change":"2023-10-21T17:35:44.207980+0000","last_active":"2023-10-21T17:35:46.209228+0000","last_peered":"2023-10-21T17:35:46.209228+0000","last_clean":"2023-10-21T17:35:46.209228+0000","last_became_active":"2023-10-21T17:35:44.207647+0000","last_became_peered":"2023-10-21T17:35:44.207647+0000","last_unstale":"2023-10-21T17:35:46.209228+0000","last_undegraded":"2023-10-21T17:35:46.209228+0000","last_fullsized":"2023-10-21T17:35:46.209228+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:23.231657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209335+0000","last_change":"2023-10-21T17:35:45.235686+0000","last_active":"2023-10-21T17:35:46.209335+0000","last_peered":"2023-10-21T17:35:46.209335+0000","last_clean":"2023-10-21T17:35:46.209335+0000","last_became_active":"2023-10-21T17:35:45.233499+0000","last_became_peered":"2023-10-21T17:35:45.233499+0000","last_unstale":"2023-10-21T17:35:46.209335+0000","last_undegraded":"2023-10-21T17:35:46.209335+0000","last_fullsized":"2023-10-21T17:35:46.209335+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:07:08.182881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209396+0000","last_change":"2023-10-21T17:35:44.223289+0000","last_active":"2023-10-21T17:35:46.209396+0000","last_peered":"2023-10-21T17:35:46.209396+0000","last_clean":"2023-10-21T17:35:46.209396+0000","last_became_active":"2023-10-21T17:35:44.222980+0000","last_became_peered":"2023-10-21T17:35:44.222980+0000","last_unstale":"2023-10-21T17:35:46.209396+0000","last_undegraded":"2023-10-21T17:35:46.209396+0000","last_fullsized":"2023-10-21T17:35:46.209396+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:22:21.139748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241495+0000","last_change":"2023-10-21T17:35:45.237444+0000","last_active":"2023-10-21T17:35:45.241495+0000","last_peered":"2023-10-21T17:35:45.241495+0000","last_clean":"2023-10-21T17:35:45.241495+0000","last_became_active":"2023-10-21T17:35:45.237059+0000","last_became_peered":"2023-10-21T17:35:45.237059+0000","last_unstale":"2023-10-21T17:35:45.241495+0000","last_undegraded":"2023-10-21T17:35:45.241495+0000","last_fullsized":"2023-10-21T17:35:45.241495+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:31.291034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205638+0000","last_change":"2023-10-21T17:35:44.225472+0000","last_active":"2023-10-21T17:35:46.205638+0000","last_peered":"2023-10-21T17:35:46.205638+0000","last_clean":"2023-10-21T17:35:46.205638+0000","last_became_active":"2023-10-21T17:35:44.225180+0000","last_became_peered":"2023-10-21T17:35:44.225180+0000","last_unstale":"2023-10-21T17:35:46.205638+0000","last_undegraded":"2023-10-21T17:35:46.205638+0000","last_fullsized":"2023-10-21T17:35:46.205638+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:23:26.446533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209018+0000","last_change":"2023-10-21T17:35:45.225834+0000","last_active":"2023-10-21T17:35:46.209018+0000","last_peered":"2023-10-21T17:35:46.209018+0000","last_clean":"2023-10-21T17:35:46.209018+0000","last_became_active":"2023-10-21T17:35:45.224902+0000","last_became_peered":"2023-10-21T17:35:45.224902+0000","last_unstale":"2023-10-21T17:35:46.209018+0000","last_undegraded":"2023-10-21T17:35:46.209018+0000","last_fullsized":"2023-10-21T17:35:46.209018+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:57:12.752341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223250+0000","last_change":"2023-10-21T17:35:44.216564+0000","last_active":"2023-10-21T17:35:45.223250+0000","last_peered":"2023-10-21T17:35:45.223250+0000","last_clean":"2023-10-21T17:35:45.223250+0000","last_became_active":"2023-10-21T17:35:44.216305+0000","last_became_peered":"2023-10-21T17:35:44.216305+0000","last_unstale":"2023-10-21T17:35:45.223250+0000","last_undegraded":"2023-10-21T17:35:45.223250+0000","last_fullsized":"2023-10-21T17:35:45.223250+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:08.559913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209375+0000","last_change":"2023-10-21T17:35:45.234640+0000","last_active":"2023-10-21T17:35:46.209375+0000","last_peered":"2023-10-21T17:35:46.209375+0000","last_clean":"2023-10-21T17:35:46.209375+0000","last_became_active":"2023-10-21T17:35:45.234371+0000","last_became_peered":"2023-10-21T17:35:45.234371+0000","last_unstale":"2023-10-21T17:35:46.209375+0000","last_undegraded":"2023-10-21T17:35:46.209375+0000","last_fullsized":"2023-10-21T17:35:46.209375+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:43:33.498382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"214'8","reported_seq":19,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.262302+0000","last_change":"2023-10-21T17:35:44.210922+0000","last_active":"2023-10-21T17:35:45.262302+0000","last_peered":"2023-10-21T17:35:45.262302+0000","last_clean":"2023-10-21T17:35:45.262302+0000","last_became_active":"2023-10-21T17:35:44.210637+0000","last_became_peered":"2023-10-21T17:35:44.210637+0000","last_unstale":"2023-10-21T17:35:45.262302+0000","last_undegraded":"2023-10-21T17:35:45.262302+0000","last_fullsized":"2023-10-21T17:35:45.262302+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:14.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.210359+0000","last_change":"2023-10-21T17:35:45.227113+0000","last_active":"2023-10-21T17:35:46.210359+0000","last_peered":"2023-10-21T17:35:46.210359+0000","last_clean":"2023-10-21T17:35:46.210359+0000","last_became_active":"2023-10-21T17:35:45.226673+0000","last_became_peered":"2023-10-21T17:35:45.226673+0000","last_unstale":"2023-10-21T17:35:46.210359+0000","last_undegraded":"2023-10-21T17:35:46.210359+0000","last_fullsized":"2023-10-21T17:35:46.210359+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:10.087693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260897+0000","last_change":"2023-10-21T17:35:44.213336+0000","last_active":"2023-10-21T17:35:45.260897+0000","last_peered":"2023-10-21T17:35:45.260897+0000","last_clean":"2023-10-21T17:35:45.260897+0000","last_became_active":"2023-10-21T17:35:44.213037+0000","last_became_peered":"2023-10-21T17:35:44.213037+0000","last_unstale":"2023-10-21T17:35:45.260897+0000","last_undegraded":"2023-10-21T17:35:45.260897+0000","last_fullsized":"2023-10-21T17:35:45.260897+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:40.699779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244974+0000","last_change":"2023-10-21T17:35:45.241185+0000","last_active":"2023-10-21T17:35:45.244974+0000","last_peered":"2023-10-21T17:35:45.244974+0000","last_clean":"2023-10-21T17:35:45.244974+0000","last_became_active":"2023-10-21T17:35:45.237862+0000","last_became_peered":"2023-10-21T17:35:45.237862+0000","last_unstale":"2023-10-21T17:35:45.244974+0000","last_undegraded":"2023-10-21T17:35:45.244974+0000","last_fullsized":"2023-10-21T17:35:45.244974+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:09.437597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.259209+0000","last_change":"2023-10-21T17:35:44.213459+0000","last_active":"2023-10-21T17:35:45.259209+0000","last_peered":"2023-10-21T17:35:45.259209+0000","last_clean":"2023-10-21T17:35:45.259209+0000","last_became_active":"2023-10-21T17:35:44.213164+0000","last_became_peered":"2023-10-21T17:35:44.213164+0000","last_unstale":"2023-10-21T17:35:45.259209+0000","last_undegraded":"2023-10-21T17:35:45.259209+0000","last_fullsized":"2023-10-21T17:35:45.259209+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:28.381903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241951+0000","last_change":"2023-10-21T17:35:45.236322+0000","last_active":"2023-10-21T17:35:45.241951+0000","last_peered":"2023-10-21T17:35:45.241951+0000","last_clean":"2023-10-21T17:35:45.241951+0000","last_became_active":"2023-10-21T17:35:45.235217+0000","last_became_peered":"2023-10-21T17:35:45.235217+0000","last_unstale":"2023-10-21T17:35:45.241951+0000","last_undegraded":"2023-10-21T17:35:45.241951+0000","last_fullsized":"2023-10-21T17:35:45.241951+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:11:37.709638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.258877+0000","last_change":"2023-10-21T17:35:44.223638+0000","last_active":"2023-10-21T17:35:45.258877+0000","last_peered":"2023-10-21T17:35:45.258877+0000","last_clean":"2023-10-21T17:35:45.258877+0000","last_became_active":"2023-10-21T17:35:44.223334+0000","last_became_peered":"2023-10-21T17:35:44.223334+0000","last_unstale":"2023-10-21T17:35:45.258877+0000","last_undegraded":"2023-10-21T17:35:45.258877+0000","last_fullsized":"2023-10-21T17:35:45.258877+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:22:21.139748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206884+0000","last_change":"2023-10-21T17:35:45.244190+0000","last_active":"2023-10-21T17:35:46.206884+0000","last_peered":"2023-10-21T17:35:46.206884+0000","last_clean":"2023-10-21T17:35:46.206884+0000","last_became_active":"2023-10-21T17:35:45.243815+0000","last_became_peered":"2023-10-21T17:35:45.243815+0000","last_unstale":"2023-10-21T17:35:46.206884+0000","last_undegraded":"2023-10-21T17:35:46.206884+0000","last_fullsized":"2023-10-21T17:35:46.206884+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:17.575909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.233389+0000","last_change":"2023-10-21T17:35:44.212510+0000","last_active":"2023-10-21T17:35:45.233389+0000","last_peered":"2023-10-21T17:35:45.233389+0000","last_clean":"2023-10-21T17:35:45.233389+0000","last_became_active":"2023-10-21T17:35:44.212211+0000","last_became_peered":"2023-10-21T17:35:44.212211+0000","last_unstale":"2023-10-21T17:35:45.233389+0000","last_undegraded":"2023-10-21T17:35:45.233389+0000","last_fullsized":"2023-10-21T17:35:45.233389+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:50.305348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205072+0000","last_change":"2023-10-21T17:35:45.243355+0000","last_active":"2023-10-21T17:35:46.205072+0000","last_peered":"2023-10-21T17:35:46.205072+0000","last_clean":"2023-10-21T17:35:46.205072+0000","last_became_active":"2023-10-21T17:35:45.243037+0000","last_became_peered":"2023-10-21T17:35:45.243037+0000","last_unstale":"2023-10-21T17:35:46.205072+0000","last_undegraded":"2023-10-21T17:35:46.205072+0000","last_fullsized":"2023-10-21T17:35:46.205072+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:36:09.029533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209634+0000","last_change":"2023-10-21T17:35:44.216966+0000","last_active":"2023-10-21T17:35:46.209634+0000","last_peered":"2023-10-21T17:35:46.209634+0000","last_clean":"2023-10-21T17:35:46.209634+0000","last_became_active":"2023-10-21T17:35:44.216657+0000","last_became_peered":"2023-10-21T17:35:44.216657+0000","last_unstale":"2023-10-21T17:35:46.209634+0000","last_undegraded":"2023-10-21T17:35:46.209634+0000","last_fullsized":"2023-10-21T17:35:46.209634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:19.728787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.235239+0000","last_change":"2023-10-21T17:35:45.233934+0000","last_active":"2023-10-21T17:35:45.235239+0000","last_peered":"2023-10-21T17:35:45.235239+0000","last_clean":"2023-10-21T17:35:45.235239+0000","last_became_active":"2023-10-21T17:35:45.232750+0000","last_became_peered":"2023-10-21T17:35:45.232750+0000","last_unstale":"2023-10-21T17:35:45.235239+0000","last_undegraded":"2023-10-21T17:35:45.235239+0000","last_fullsized":"2023-10-21T17:35:45.235239+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:22.268282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.221248+0000","last_change":"2023-10-21T17:35:44.218400+0000","last_active":"2023-10-21T17:35:45.221248+0000","last_peered":"2023-10-21T17:35:45.221248+0000","last_clean":"2023-10-21T17:35:45.221248+0000","last_became_active":"2023-10-21T17:35:44.218134+0000","last_became_peered":"2023-10-21T17:35:44.218134+0000","last_unstale":"2023-10-21T17:35:45.221248+0000","last_undegraded":"2023-10-21T17:35:45.221248+0000","last_fullsized":"2023-10-21T17:35:45.221248+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:16.559008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242461+0000","last_change":"2023-10-21T17:35:45.241294+0000","last_active":"2023-10-21T17:35:45.242461+0000","last_peered":"2023-10-21T17:35:45.242461+0000","last_clean":"2023-10-21T17:35:45.242461+0000","last_became_active":"2023-10-21T17:35:45.240992+0000","last_became_peered":"2023-10-21T17:35:45.240992+0000","last_unstale":"2023-10-21T17:35:45.242461+0000","last_undegraded":"2023-10-21T17:35:45.242461+0000","last_fullsized":"2023-10-21T17:35:45.242461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:25.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209828+0000","last_change":"2023-10-21T17:35:44.211526+0000","last_active":"2023-10-21T17:35:46.209828+0000","last_peered":"2023-10-21T17:35:46.209828+0000","last_clean":"2023-10-21T17:35:46.209828+0000","last_became_active":"2023-10-21T17:35:44.211223+0000","last_became_peered":"2023-10-21T17:35:44.211223+0000","last_unstale":"2023-10-21T17:35:46.209828+0000","last_undegraded":"2023-10-21T17:35:46.209828+0000","last_fullsized":"2023-10-21T17:35:46.209828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:03.897360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242167+0000","last_change":"2023-10-21T17:35:45.239714+0000","last_active":"2023-10-21T17:35:45.242167+0000","last_peered":"2023-10-21T17:35:45.242167+0000","last_clean":"2023-10-21T17:35:45.242167+0000","last_became_active":"2023-10-21T17:35:45.239344+0000","last_became_peered":"2023-10-21T17:35:45.239344+0000","last_unstale":"2023-10-21T17:35:45.242167+0000","last_undegraded":"2023-10-21T17:35:45.242167+0000","last_fullsized":"2023-10-21T17:35:45.242167+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:40.037869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.224760+0000","last_change":"2023-10-21T17:35:44.215205+0000","last_active":"2023-10-21T17:35:45.224760+0000","last_peered":"2023-10-21T17:35:45.224760+0000","last_clean":"2023-10-21T17:35:45.224760+0000","last_became_active":"2023-10-21T17:35:44.214654+0000","last_became_peered":"2023-10-21T17:35:44.214654+0000","last_unstale":"2023-10-21T17:35:45.224760+0000","last_undegraded":"2023-10-21T17:35:45.224760+0000","last_fullsized":"2023-10-21T17:35:45.224760+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:19.728787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209985+0000","last_change":"2023-10-21T17:35:45.228595+0000","last_active":"2023-10-21T17:35:46.209985+0000","last_peered":"2023-10-21T17:35:46.209985+0000","last_clean":"2023-10-21T17:35:46.209985+0000","last_became_active":"2023-10-21T17:35:45.228245+0000","last_became_peered":"2023-10-21T17:35:45.228245+0000","last_unstale":"2023-10-21T17:35:46.209985+0000","last_undegraded":"2023-10-21T17:35:46.209985+0000","last_fullsized":"2023-10-21T17:35:46.209985+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:12:09.144034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260314+0000","last_change":"2023-10-21T17:35:44.215713+0000","last_active":"2023-10-21T17:35:45.260314+0000","last_peered":"2023-10-21T17:35:45.260314+0000","last_clean":"2023-10-21T17:35:45.260314+0000","last_became_active":"2023-10-21T17:35:44.215177+0000","last_became_peered":"2023-10-21T17:35:44.215177+0000","last_unstale":"2023-10-21T17:35:45.260314+0000","last_undegraded":"2023-10-21T17:35:45.260314+0000","last_fullsized":"2023-10-21T17:35:45.260314+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:23.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205534+0000","last_change":"2023-10-21T17:35:45.225390+0000","last_active":"2023-10-21T17:35:46.205534+0000","last_peered":"2023-10-21T17:35:46.205534+0000","last_clean":"2023-10-21T17:35:46.205534+0000","last_became_active":"2023-10-21T17:35:45.224824+0000","last_became_peered":"2023-10-21T17:35:45.224824+0000","last_unstale":"2023-10-21T17:35:46.205534+0000","last_undegraded":"2023-10-21T17:35:46.205534+0000","last_fullsized":"2023-10-21T17:35:46.205534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:09.579033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209499+0000","last_change":"2023-10-21T17:35:44.218762+0000","last_active":"2023-10-21T17:35:46.209499+0000","last_peered":"2023-10-21T17:35:46.209499+0000","last_clean":"2023-10-21T17:35:46.209499+0000","last_became_active":"2023-10-21T17:35:44.218461+0000","last_became_peered":"2023-10-21T17:35:44.218461+0000","last_unstale":"2023-10-21T17:35:46.209499+0000","last_undegraded":"2023-10-21T17:35:46.209499+0000","last_fullsized":"2023-10-21T17:35:46.209499+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:20:36.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241985+0000","last_change":"2023-10-21T17:35:45.235709+0000","last_active":"2023-10-21T17:35:45.241985+0000","last_peered":"2023-10-21T17:35:45.241985+0000","last_clean":"2023-10-21T17:35:45.241985+0000","last_became_active":"2023-10-21T17:35:45.235404+0000","last_became_peered":"2023-10-21T17:35:45.235404+0000","last_unstale":"2023-10-21T17:35:45.241985+0000","last_undegraded":"2023-10-21T17:35:45.241985+0000","last_fullsized":"2023-10-21T17:35:45.241985+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:10.332865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.230662+0000","last_change":"2023-10-21T17:35:44.222528+0000","last_active":"2023-10-21T17:35:45.230662+0000","last_peered":"2023-10-21T17:35:45.230662+0000","last_clean":"2023-10-21T17:35:45.230662+0000","last_became_active":"2023-10-21T17:35:44.219207+0000","last_became_peered":"2023-10-21T17:35:44.219207+0000","last_unstale":"2023-10-21T17:35:45.230662+0000","last_undegraded":"2023-10-21T17:35:45.230662+0000","last_fullsized":"2023-10-21T17:35:45.230662+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.488604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209769+0000","last_change":"2023-10-21T17:35:45.239560+0000","last_active":"2023-10-21T17:35:46.209769+0000","last_peered":"2023-10-21T17:35:46.209769+0000","last_clean":"2023-10-21T17:35:46.209769+0000","last_became_active":"2023-10-21T17:35:45.239261+0000","last_became_peered":"2023-10-21T17:35:45.239261+0000","last_unstale":"2023-10-21T17:35:46.209769+0000","last_undegraded":"2023-10-21T17:35:46.209769+0000","last_fullsized":"2023-10-21T17:35:46.209769+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:52.865690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.234731+0000","last_change":"2023-10-21T17:35:44.222790+0000","last_active":"2023-10-21T17:35:45.234731+0000","last_peered":"2023-10-21T17:35:45.234731+0000","last_clean":"2023-10-21T17:35:45.234731+0000","last_became_active":"2023-10-21T17:35:44.220804+0000","last_became_peered":"2023-10-21T17:35:44.220804+0000","last_unstale":"2023-10-21T17:35:45.234731+0000","last_undegraded":"2023-10-21T17:35:45.234731+0000","last_fullsized":"2023-10-21T17:35:45.234731+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:45:22.752940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.237191+0000","last_change":"2023-10-21T17:35:45.232810+0000","last_active":"2023-10-21T17:35:45.237191+0000","last_peered":"2023-10-21T17:35:45.237191+0000","last_clean":"2023-10-21T17:35:45.237191+0000","last_became_active":"2023-10-21T17:35:45.232472+0000","last_became_peered":"2023-10-21T17:35:45.232472+0000","last_unstale":"2023-10-21T17:35:45.237191+0000","last_undegraded":"2023-10-21T17:35:45.237191+0000","last_fullsized":"2023-10-21T17:35:45.237191+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:11:05.203856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223307+0000","last_change":"2023-10-21T17:35:44.218100+0000","last_active":"2023-10-21T17:35:45.223307+0000","last_peered":"2023-10-21T17:35:45.223307+0000","last_clean":"2023-10-21T17:35:45.223307+0000","last_became_active":"2023-10-21T17:35:44.217086+0000","last_became_peered":"2023-10-21T17:35:44.217086+0000","last_unstale":"2023-10-21T17:35:45.223307+0000","last_undegraded":"2023-10-21T17:35:45.223307+0000","last_fullsized":"2023-10-21T17:35:45.223307+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:15.716141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209229+0000","last_change":"2023-10-21T17:35:45.230326+0000","last_active":"2023-10-21T17:35:46.209229+0000","last_peered":"2023-10-21T17:35:46.209229+0000","last_clean":"2023-10-21T17:35:46.209229+0000","last_became_active":"2023-10-21T17:35:45.229981+0000","last_became_peered":"2023-10-21T17:35:45.229981+0000","last_unstale":"2023-10-21T17:35:46.209229+0000","last_undegraded":"2023-10-21T17:35:46.209229+0000","last_fullsized":"2023-10-21T17:35:46.209229+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:56:19.123780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223043+0000","last_change":"2023-10-21T17:35:44.226369+0000","last_active":"2023-10-21T17:35:45.223043+0000","last_peered":"2023-10-21T17:35:45.223043+0000","last_clean":"2023-10-21T17:35:45.223043+0000","last_became_active":"2023-10-21T17:35:44.226051+0000","last_became_peered":"2023-10-21T17:35:44.226051+0000","last_unstale":"2023-10-21T17:35:45.223043+0000","last_undegraded":"2023-10-21T17:35:45.223043+0000","last_fullsized":"2023-10-21T17:35:45.223043+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:40.699779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205551+0000","last_change":"2023-10-21T17:35:45.238108+0000","last_active":"2023-10-21T17:35:46.205551+0000","last_peered":"2023-10-21T17:35:46.205551+0000","last_clean":"2023-10-21T17:35:46.205551+0000","last_became_active":"2023-10-21T17:35:45.237763+0000","last_became_peered":"2023-10-21T17:35:45.237763+0000","last_unstale":"2023-10-21T17:35:46.205551+0000","last_undegraded":"2023-10-21T17:35:46.205551+0000","last_fullsized":"2023-10-21T17:35:46.205551+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:35.653591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209936+0000","last_change":"2023-10-21T17:35:44.211518+0000","last_active":"2023-10-21T17:35:46.209936+0000","last_peered":"2023-10-21T17:35:46.209936+0000","last_clean":"2023-10-21T17:35:46.209936+0000","last_became_active":"2023-10-21T17:35:44.210737+0000","last_became_peered":"2023-10-21T17:35:44.210737+0000","last_unstale":"2023-10-21T17:35:46.209936+0000","last_undegraded":"2023-10-21T17:35:46.209936+0000","last_fullsized":"2023-10-21T17:35:46.209936+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:15:43.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242142+0000","last_change":"2023-10-21T17:35:45.237068+0000","last_active":"2023-10-21T17:35:45.242142+0000","last_peered":"2023-10-21T17:35:45.242142+0000","last_clean":"2023-10-21T17:35:45.242142+0000","last_became_active":"2023-10-21T17:35:45.236785+0000","last_became_peered":"2023-10-21T17:35:45.236785+0000","last_unstale":"2023-10-21T17:35:45.242142+0000","last_undegraded":"2023-10-21T17:35:45.242142+0000","last_fullsized":"2023-10-21T17:35:45.242142+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:33.202011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.224980+0000","last_change":"2023-10-21T17:35:44.218386+0000","last_active":"2023-10-21T17:35:45.224980+0000","last_peered":"2023-10-21T17:35:45.224980+0000","last_clean":"2023-10-21T17:35:45.224980+0000","last_became_active":"2023-10-21T17:35:44.217978+0000","last_became_peered":"2023-10-21T17:35:44.217978+0000","last_unstale":"2023-10-21T17:35:45.224980+0000","last_undegraded":"2023-10-21T17:35:45.224980+0000","last_fullsized":"2023-10-21T17:35:45.224980+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:34.813742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244747+0000","last_change":"2023-10-21T17:35:45.243623+0000","last_active":"2023-10-21T17:35:45.244747+0000","last_peered":"2023-10-21T17:35:45.244747+0000","last_clean":"2023-10-21T17:35:45.244747+0000","last_became_active":"2023-10-21T17:35:45.243221+0000","last_became_peered":"2023-10-21T17:35:45.243221+0000","last_unstale":"2023-10-21T17:35:45.244747+0000","last_undegraded":"2023-10-21T17:35:45.244747+0000","last_fullsized":"2023-10-21T17:35:45.244747+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:51.733192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.220809+0000","last_change":"2023-10-21T17:35:44.213390+0000","last_active":"2023-10-21T17:35:45.220809+0000","last_peered":"2023-10-21T17:35:45.220809+0000","last_clean":"2023-10-21T17:35:45.220809+0000","last_became_active":"2023-10-21T17:35:44.213075+0000","last_became_peered":"2023-10-21T17:35:44.213075+0000","last_unstale":"2023-10-21T17:35:45.220809+0000","last_undegraded":"2023-10-21T17:35:45.220809+0000","last_fullsized":"2023-10-21T17:35:45.220809+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:57:16.383179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"202'4851","reported_seq":5705,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-10-21T17:35:44.212614+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:35:44.212614+0000","last_peered":"2023-10-21T17:35:44.212614+0000","last_clean":"2023-10-21T17:35:44.212614+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:35:44.212614+0000","last_undegraded":"2023-10-21T17:35:44.212614+0000","last_fullsized":"2023-10-21T17:35:44.212614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4851,"log_dups_size":0,"ondisk_log_size":4851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":547,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":547,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4851,"ondisk_log_size":4851,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52436,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319404,"statfs":{"total":96636764160,"available":96583069696,"internally_reserved":0,"allocated":3276800,"data_stored":1974984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.214,"15min":1.202},"min":{"1min":0.76700000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.841,"5min":1.841,"15min":2.2549999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.159,"15min":1.181},"min":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.65600000000000003},"max":{"1min":1.619,"5min":1.6839999999999999,"15min":1.7929999999999999},"last":1.2669999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.85799999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.649,"5min":1.6779999999999999,"15min":1.73},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.3180000000000001,"15min":1.319},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.78700000000000003},"max":{"1min":1.734,"5min":1.885,"15min":1.885},"last":1.7290000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.335},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.7609999999999999,"5min":1.9339999999999999,"15min":1.946},"last":1.599},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.327,"15min":1.3180000000000001},"min":{"1min":1.026,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.962},"last":0.95099999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":1.028,"5min":0.96799999999999997,"15min":0.876},"max":{"1min":1.6479999999999999,"5min":1.6899999999999999,"15min":2.5110000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4730000000000001,"15min":1.4550000000000001},"min":{"1min":1.2110000000000001,"5min":1.0409999999999999,"15min":0.91000000000000003},"max":{"1min":1.73,"5min":1.827,"15min":1.8999999999999999},"last":1.206}]},{"osd":4,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.359,"15min":1.353},"min":{"1min":0.90300000000000002,"5min":0.77400000000000002,"15min":0.76000000000000001},"max":{"1min":1.7110000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.452},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.47,"15min":1.496},"min":{"1min":1.377,"5min":0.91600000000000004,"15min":0.79400000000000004},"max":{"1min":1.827,"5min":1.827,"15min":2.1549999999999998},"last":1.7}]},{"osd":5,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3720000000000001,"15min":1.3979999999999999},"min":{"1min":1.022,"5min":0.872,"15min":0.73099999999999998},"max":{"1min":1.8700000000000001,"5min":2.1240000000000001,"15min":2.2450000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4410000000000001,"15min":1.4470000000000001},"min":{"1min":1.298,"5min":0.96199999999999997,"15min":0.751},"max":{"1min":1.8100000000000001,"5min":2.3540000000000001,"15min":2.3639999999999999},"last":1.5529999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5269999999999999,"15min":1.5309999999999999},"min":{"1min":1.1299999999999999,"5min":1.0640000000000001,"15min":0.96799999999999997},"max":{"1min":1.9079999999999999,"5min":2.2040000000000002,"15min":2.2269999999999999},"last":1.667},{"interface":"front","average":{"1min":1.518,"5min":1.468,"15min":1.4730000000000001},"min":{"1min":1.2470000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.946,"5min":1.946,"15min":2.3050000000000002},"last":1.645}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42132,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329708,"statfs":{"total":96636764160,"available":96593620992,"internally_reserved":0,"allocated":2686976,"data_stored":1384686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2130000000000001,"15min":1.2010000000000001},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.141,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.81599999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6839999999999999,"5min":1.762,"15min":1.762},"last":1.0840000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2030000000000001,"15min":1.2010000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.579,"15min":1.7050000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.319,"15min":1.3220000000000001},"min":{"1min":0.93200000000000005,"5min":0.89800000000000002,"15min":0.82299999999999995},"max":{"1min":1.637,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.21}]},{"osd":2,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"max":{"1min":1.579,"5min":1.825,"15min":1.8300000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3129999999999999,"15min":1.3120000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.79500000000000004},"max":{"1min":1.679,"5min":1.6819999999999999,"15min":1.929},"last":1.1719999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4330000000000001,"15min":1.421},"min":{"1min":1.1659999999999999,"5min":1.044,"15min":0.70599999999999996},"max":{"1min":1.6870000000000001,"5min":1.9419999999999999,"15min":1.9570000000000001},"last":1.373},{"interface":"front","average":{"1min":1.375,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":1.0509999999999999,"5min":0.90500000000000003,"15min":0.83299999999999996},"max":{"1min":1.655,"5min":2.0209999999999999,"15min":2.048},"last":1.423}]},{"osd":4,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3129999999999999,"15min":1.365},"min":{"1min":0.88700000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.762,"5min":1.9930000000000001,"15min":2.028},"last":1.488},{"interface":"front","average":{"1min":1.363,"5min":1.401,"15min":1.4470000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6950000000000001,"5min":1.927,"15min":2.2149999999999999},"last":1.573}]},{"osd":5,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5249999999999999,"15min":1.5669999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":0.997},"max":{"1min":1.794,"5min":2.1739999999999999,"15min":2.339},"last":1.849},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3460000000000001,"15min":1.397},"min":{"1min":0.90200000000000002,"5min":0.875,"15min":0.78800000000000003},"max":{"1min":1.581,"5min":1.825,"15min":2.2890000000000001},"last":1.643}]},{"osd":7,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5289999999999999,"15min":1.5620000000000001},"min":{"1min":1.1160000000000001,"5min":1.093,"15min":1.0589999999999999},"max":{"1min":1.768,"5min":2.0449999999999999,"15min":2.145},"last":1.617},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.6319999999999999,"15min":1.6739999999999999},"min":{"1min":1.266,"5min":1.2,"15min":1.0449999999999999},"max":{"1min":1.8560000000000001,"5min":2.2610000000000001,"15min":2.5409999999999999},"last":1.681}]}]},{"osd":5,"up_from":13,"seq":55834575967,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60192,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311648,"statfs":{"total":96636764160,"available":96575127552,"internally_reserved":0,"allocated":2699264,"data_stored":1386250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1399999999999999,"15min":1.1599999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.512,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.498},{"interface":"front","average":{"1min":1.345,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.59999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.256,"15min":1.2809999999999999},"min":{"1min":0.876,"5min":0.85099999999999998,"15min":0.73799999999999999},"max":{"1min":1.623,"5min":1.806,"15min":1.8680000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.258,"15min":1.248},"min":{"1min":0.96999999999999997,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":1.073}]},{"osd":2,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.3149999999999999},"min":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.91400000000000003},"max":{"1min":1.4910000000000001,"5min":1.6699999999999999,"15min":2.032},"last":1.2},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.262},"min":{"1min":0.96799999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.891},"last":1.125}]},{"osd":3,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.367,"15min":1.369},"min":{"1min":1.004,"5min":0.89800000000000002,"15min":0.84699999999999998},"max":{"1min":1.758,"5min":1.829,"15min":2.4180000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.421,"15min":1.4299999999999999},"min":{"1min":1.083,"5min":1.028,"15min":0.90400000000000003},"max":{"1min":1.8009999999999999,"5min":1.881,"15min":2.1539999999999999},"last":1.3700000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.214,"15min":1.2609999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.526,"5min":1.8520000000000001,"15min":1.9910000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":1.0920000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7290000000000001,"5min":1.877,"15min":2.306},"last":1.5760000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4630000000000001,"15min":1.49},"min":{"1min":1.1970000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8939999999999999,"5min":1.9319999999999999,"15min":2.6520000000000001},"last":1.306},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5249999999999999,"15min":1.538},"min":{"1min":1.3380000000000001,"5min":0.89800000000000002,"15min":0.875},"max":{"1min":1.7350000000000001,"5min":2.2269999999999999,"15min":2.52},"last":1.651}]},{"osd":7,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5569999999999999,"15min":1.5580000000000001},"min":{"1min":1.1579999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.9319999999999999,"5min":2.2530000000000001,"15min":2.5209999999999999},"last":1.782},{"interface":"front","average":{"1min":1.635,"5min":1.5760000000000001,"15min":1.589},"min":{"1min":1.268,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9510000000000001,"5min":2.1299999999999999,"15min":2.7759999999999998},"last":1.468}]}]},{"osd":4,"up_from":13,"seq":55834575965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47716,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324124,"statfs":{"total":96636764160,"available":96587902976,"internally_reserved":0,"allocated":2711552,"data_stored":1392998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.131,"15min":1.179},"min":{"1min":0.79600000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.569,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2490000000000001,"15min":1.2210000000000001},"min":{"1min":0.90400000000000003,"5min":0.76600000000000001,"15min":0.745},"max":{"1min":1.613,"5min":1.869,"15min":1.869},"last":0.98399999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":0.997,"5min":0.86299999999999999,"15min":0.58099999999999996},"max":{"1min":1.7090000000000001,"5min":1.806,"15min":1.806},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2589999999999999,"15min":1.2629999999999999},"min":{"1min":0.98499999999999999,"5min":0.85099999999999998,"15min":0.78700000000000003},"max":{"1min":1.5429999999999999,"5min":1.7090000000000001,"15min":1.76},"last":1.2629999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.262,"15min":1.26},"min":{"1min":0.98999999999999999,"5min":0.88400000000000001,"15min":0.76300000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.7609999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.371,"15min":1.389},"min":{"1min":1.0780000000000001,"5min":1.026,"15min":0.95099999999999996},"max":{"1min":1.6240000000000001,"5min":1.762,"15min":1.762},"last":1.1180000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.304,"15min":1.3340000000000001},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.841},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":1.089,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.825,"5min":1.8260000000000001,"15min":1.9239999999999999},"last":1.323}]},{"osd":5,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":1.022,"5min":0.82699999999999996,"15min":0.80800000000000005},"max":{"1min":1.623,"5min":1.867,"15min":1.9410000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.325,"15min":1.357},"min":{"1min":0.879,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.9810000000000001},"last":1.6739999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.448},"min":{"1min":0.95399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8979999999999999,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.581,"15min":1.5880000000000001},"min":{"1min":1.379,"5min":1.145,"15min":1.133},"max":{"1min":1.806,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.71}]},{"osd":7,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.476,"15min":1.5049999999999999},"min":{"1min":1.0529999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.964,"5min":2.234,"15min":2.234},"last":1.929},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5549999999999999,"15min":1.5800000000000001},"min":{"1min":1.242,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.9470000000000001,"5min":1.952,"15min":2.081},"last":1.637}]}]},{"osd":3,"up_from":13,"seq":55834575965,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56016,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315824,"statfs":{"total":96636764160,"available":96579403776,"internally_reserved":0,"allocated":2691072,"data_stored":1384776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97299999999999998,"15min":0.98599999999999999},"min":{"1min":0.68899999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.4379999999999999,"5min":1.601,"15min":1.71},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.266,"15min":1.266},"min":{"1min":1.048,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0960000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.079,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.72699999999999998,"15min":0.45600000000000002},"max":{"1min":1.583,"5min":1.7769999999999999,"15min":1.8080000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3460000000000001,"15min":1.3480000000000001},"min":{"1min":0.93600000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.637,"5min":1.8120000000000001,"15min":1.9419999999999999},"last":1.26}]},{"osd":2,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.92300000000000004,"5min":0.60999999999999999,"15min":0.60899999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.381,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.92900000000000005,"5min":0.91600000000000004,"15min":0.55300000000000005},"max":{"1min":1.853,"5min":1.853,"15min":2},"last":1.125}]},{"osd":4,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4259999999999999,"15min":1.444},"min":{"1min":1.1479999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6419999999999999,"5min":1.8280000000000001,"15min":1.9690000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.484,"15min":1.4870000000000001},"min":{"1min":1.044,"5min":1.044,"15min":0.88700000000000001},"max":{"1min":1.9430000000000001,"5min":2.169,"15min":2.169},"last":1.419}]},{"osd":5,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5089999999999999,"15min":1.516},"min":{"1min":1.1419999999999999,"5min":0.93500000000000005,"15min":0.90900000000000003},"max":{"1min":1.9630000000000001,"5min":2.0089999999999999,"15min":2.5329999999999999},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5509999999999999,"15min":1.579},"min":{"1min":1.292,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.2069999999999999},"last":1.1850000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.589,"15min":1.6100000000000001},"min":{"1min":1.179,"5min":1.179,"15min":0.93899999999999995},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.603},{"interface":"front","average":{"1min":1.641,"5min":1.621,"15min":1.633},"min":{"1min":1.1719999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.0699999999999998,"5min":2.089,"15min":2.6840000000000002},"last":1.464}]},{"osd":7,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.6819999999999999,"15min":1.6739999999999999},"min":{"1min":1.5029999999999999,"5min":1.026,"15min":1.026},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3460000000000001},"last":1.373},{"interface":"front","average":{"1min":1.786,"5min":1.712,"15min":1.7210000000000001},"min":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.141},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.51}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51856,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319984,"statfs":{"total":96636764160,"available":96583663616,"internally_reserved":0,"allocated":2686976,"data_stored":1385198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92700000000000005,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.8029999999999999,"5min":1.841,"15min":1.841},"last":1.109},{"interface":"front","average":{"1min":1.087,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.75,"5min":0.70599999999999996,"15min":0.66800000000000004},"max":{"1min":1.907,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.90000000000000002}]},{"osd":1,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.194,"15min":1.198},"min":{"1min":0.92600000000000005,"5min":0.79100000000000004,"15min":0.76400000000000001},"max":{"1min":1.794,"5min":1.806,"15min":1.819},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.395,"15min":1.3979999999999999},"min":{"1min":1.103,"5min":1.036,"15min":0.79500000000000004},"max":{"1min":1.958,"5min":1.958,"15min":2.4060000000000001},"last":1.419}]},{"osd":3,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.216,"15min":1.1859999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":1.8149999999999999,"15min":2.1760000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.492,"5min":1.506,"15min":1.4910000000000001},"min":{"1min":1.2310000000000001,"5min":1.155,"15min":1.1200000000000001},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1}]},{"osd":4,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4239999999999999,"15min":1.413},"min":{"1min":1.1140000000000001,"5min":1.044,"15min":1.024},"max":{"1min":1.8660000000000001,"5min":2.1680000000000001,"15min":2.218},"last":1.181},{"interface":"front","average":{"1min":1.522,"5min":1.4670000000000001,"15min":1.454},"min":{"1min":1.133,"5min":1.069,"15min":1.0329999999999999},"max":{"1min":1.9610000000000001,"5min":2.1469999999999998,"15min":2.5939999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.407},"min":{"1min":1.125,"5min":1.0469999999999999,"15min":0.94699999999999995},"max":{"1min":1.819,"5min":1.9239999999999999,"15min":2.1560000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.631,"15min":1.605},"min":{"1min":1.355,"5min":1.2470000000000001,"15min":1.2150000000000001},"max":{"1min":1.972,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.663}]},{"osd":6,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5920000000000001,"15min":1.573},"min":{"1min":1.1970000000000001,"5min":1.129,"15min":1.042},"max":{"1min":2.0830000000000002,"5min":2.286,"15min":2.4460000000000002},"last":1.746},{"interface":"front","average":{"1min":1.571,"5min":1.5309999999999999,"15min":1.514},"min":{"1min":1.179,"5min":1.1579999999999999,"15min":0.80600000000000005},"max":{"1min":1.927,"5min":2.0470000000000002,"15min":2.1030000000000002},"last":1.786}]},{"osd":7,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.643,"15min":1.6399999999999999},"min":{"1min":1.232,"5min":1.1819999999999999,"15min":1.0780000000000001},"max":{"1min":2.1520000000000001,"5min":2.351,"15min":2.4630000000000001},"last":1.841},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.605,"15min":1.595},"min":{"1min":1.3740000000000001,"5min":1.147,"15min":0.77500000000000002},"max":{"1min":2.1259999999999999,"5min":2.1669999999999998,"15min":2.4630000000000001},"last":1.7050000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51848,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319992,"statfs":{"total":96636764160,"available":96583671808,"internally_reserved":0,"allocated":2682880,"data_stored":1384616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.92000000000000004,"15min":0.95899999999999996},"min":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.65900000000000003},"max":{"1min":1.3480000000000001,"5min":1.637,"15min":1.976},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.1950000000000001,"15min":1.1639999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.46700000000000003},"max":{"1min":1.8089999999999999,"5min":1.8400000000000001,"15min":2.0510000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0509999999999999,"15min":1.054},"min":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.56799999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.79},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.262,"5min":1.244,"15min":1.2629999999999999},"min":{"1min":0.89300000000000002,"5min":0.88400000000000001,"15min":0.66700000000000004},"max":{"1min":1.679,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3740000000000001,"15min":1.409},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.83299999999999996},"max":{"1min":1.704,"5min":1.875,"15min":2.129},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.23,"5min":1.2430000000000001,"15min":1.3049999999999999},"min":{"1min":0.88700000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.748,"5min":1.798,"15min":2.456},"last":0.92500000000000004}]},{"osd":4,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.393,"15min":1.373},"min":{"1min":1.02,"5min":1.0029999999999999,"15min":0.91500000000000004},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0950000000000002},"last":1.04},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.51,"15min":1.5109999999999999},"min":{"1min":1.179,"5min":1.1779999999999999,"15min":1.0629999999999999},"max":{"1min":1.8380000000000001,"5min":2.0209999999999999,"15min":2.383},"last":1.226}]},{"osd":5,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.589,"15min":1.581},"min":{"1min":1.1459999999999999,"5min":1.018,"15min":1.018},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.5169999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.5289999999999999,"15min":1.5589999999999999},"min":{"1min":1.008,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.335},"last":1.167}]},{"osd":6,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5980000000000001,"15min":1.6080000000000001},"min":{"1min":1.2410000000000001,"5min":1.1930000000000001,"15min":0.85699999999999998},"max":{"1min":2.27,"5min":2.27,"15min":2.3940000000000001},"last":1.016},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5589999999999999,"15min":1.571},"min":{"1min":1.127,"5min":1.127,"15min":0.98499999999999999},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.3519999999999999},"last":0.97199999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.6559999999999999,"15min":1.7010000000000001},"min":{"1min":1.341,"5min":1.294,"15min":1.2},"max":{"1min":1.758,"5min":2.113,"15min":2.548},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.7070000000000001,"15min":1.7270000000000001},"min":{"1min":1.4970000000000001,"5min":1.149,"15min":0.90200000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.548},"last":1.399}]}]},{"osd":0,"up_from":13,"seq":55834575965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48288,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323552,"statfs":{"total":96636764160,"available":96587317248,"internally_reserved":0,"allocated":3293184,"data_stored":1975712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.78200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.409,"5min":1.774,"15min":1.774},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.133,"5min":1.1459999999999999,"15min":1.1259999999999999},"min":{"1min":0.86799999999999999,"5min":0.71899999999999997,"15min":0.68200000000000005},"max":{"1min":1.407,"5min":1.748,"15min":1.774},"last":0.80500000000000005}]},{"osd":2,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1870000000000001,"15min":1.2},"min":{"1min":0.84099999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.484,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2390000000000001,"15min":1.2130000000000001},"min":{"1min":0.876,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.6459999999999999,"5min":1.851,"15min":1.851},"last":1.137}]},{"osd":3,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.78200000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5960000000000001,"5min":1.643,"15min":1.9590000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.403,"15min":1.4119999999999999},"min":{"1min":1.0389999999999999,"5min":1.01,"15min":0.95999999999999996},"max":{"1min":1.915,"5min":2.0270000000000001,"15min":2.0329999999999999},"last":1.4610000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.4570000000000001},"min":{"1min":1.028,"5min":0.95799999999999996,"15min":0.88600000000000001},"max":{"1min":1.8220000000000001,"5min":2.032,"15min":2.0910000000000002},"last":1.177},{"interface":"front","average":{"1min":1.603,"5min":1.605,"15min":1.6220000000000001},"min":{"1min":1.284,"5min":1.163,"15min":1.123},"max":{"1min":1.9610000000000001,"5min":2.1360000000000001,"15min":2.2770000000000001},"last":1.629}]},{"osd":5,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.407,"15min":1.4259999999999999},"min":{"1min":1.157,"5min":1.0089999999999999,"15min":0.95799999999999996},"max":{"1min":1.6970000000000001,"5min":1.8080000000000001,"15min":2.101},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.5700000000000001,"15min":1.6000000000000001},"min":{"1min":1.3029999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.9039999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.7030000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.494,"15min":1.5109999999999999},"min":{"1min":1.2749999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.9330000000000001,"5min":2.0139999999999998,"15min":2.0750000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.6799999999999999,"15min":1.7050000000000001},"min":{"1min":1.258,"5min":1.258,"15min":1.1299999999999999},"max":{"1min":2.0760000000000001,"5min":2.222,"15min":2.4940000000000002},"last":1.758}]},{"osd":7,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.573,"15min":1.5680000000000001},"min":{"1min":1.206,"5min":1.0649999999999999,"15min":1.034},"max":{"1min":2.1619999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.665,"5min":1.6240000000000001,"15min":1.647},"min":{"1min":1.367,"5min":1.145,"15min":1.145},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.262},"last":1.341}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:35:49.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:35:50.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.005+0000 7f98b5e40640 1 Processor -- start 2023-10-21T17:35:50.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.006+0000 7f98b5e40640 1 -- start start 2023-10-21T17:35:50.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.007+0000 7f98b5e40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b01599d0 0x7f98b00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.007+0000 7f98b5e40640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98b00cc210 0x7f98b00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.007+0000 7f98b5e40640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00cacf0 con 0x7f98b0159db0 2023-10-21T17:35:50.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.007+0000 7f98b5e40640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b00b10f0 con 0x7f98b00cc210 2023-10-21T17:35:50.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.007+0000 7f98b5e40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00ca3c0 con 0x7f98b01599d0 2023-10-21T17:35:50.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.008+0000 7f98affff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98b00cc210 0x7f98b00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.008+0000 7f98affff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98b00cc210 0x7f98b00b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51862/0 (socket says 172.21.15.53:51862) 2023-10-21T17:35:50.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.008+0000 7f98affff640 1 -- 172.21.15.53:0/1204032272 learned_addr learned my addr 172.21.15.53:0/1204032272 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:50.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.008+0000 7f98b4e3e640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b01599d0 0x7f98b00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.008+0000 7f98affff640 1 -- 172.21.15.53:0/1204032272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b01599d0 msgr2=0x7f98b00ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:50.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.009+0000 7f98affff640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b01599d0 0x7f98b00ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.009+0000 7f98affff640 1 -- 172.21.15.53:0/1204032272 >> v1:172.21.15.124:6789/0 conn(0x7f98b0159db0 legacy=0x7f98b015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.009+0000 7f98affff640 1 -- 172.21.15.53:0/1204032272 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00ca6a0 con 0x7f98b00cc210 2023-10-21T17:35:50.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.010+0000 7f98affff640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00cc210 0x7f98b00b00c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c2f88b318c54ab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.010+0000 7f98af7fe640 1 -- 172.21.15.53:0/1204032272 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c030840 con 0x7f98b00cc210 2023-10-21T17:35:50.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.010+0000 7f98af7fe640 1 -- 172.21.15.53:0/1204032272 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0309a0 con 0x7f98b00cc210 2023-10-21T17:35:50.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.010+0000 7f98af7fe640 1 -- 172.21.15.53:0/1204032272 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c030d30 con 0x7f98b00cc210 2023-10-21T17:35:50.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 -- 172.21.15.53:0/1204032272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00cc210 msgr2=0x7f98b00b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00cc210 0x7f98b00b00c0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02b980 comp rx=0 tx=0).stop 2023-10-21T17:35:50.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 -- 172.21.15.53:0/1204032272 shutdown_connections 2023-10-21T17:35:50.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b00cc210 0x7f98b00b00c0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 --2- 172.21.15.53:0/1204032272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b01599d0 0x7f98b00ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 -- 172.21.15.53:0/1204032272 >> 172.21.15.53:0/1204032272 conn(0x7f98b00cb0c0 msgr2=0x7f98b00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:50.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 -- 172.21.15.53:0/1204032272 shutdown_connections 2023-10-21T17:35:50.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.011+0000 7f98b5e40640 1 -- 172.21.15.53:0/1204032272 wait complete. 2023-10-21T17:35:50.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.013+0000 7f98b5e40640 1 Processor -- start 2023-10-21T17:35:50.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.013+0000 7f98b5e40640 1 -- start start 2023-10-21T17:35:50.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b00cc210 0x7f98b01e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 0x7f98b00c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00c6760 con 0x7f98b0159db0 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b01510b0 con 0x7f98b01599d0 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.014+0000 7f98b5e40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00ae990 con 0x7f98b00cc210 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51868/0 (socket says 172.21.15.53:51868) 2023-10-21T17:35:50.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 -- 172.21.15.53:0/3598430137 learned_addr learned my addr 172.21.15.53:0/3598430137 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:50.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b4e3e640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b00cc210 0x7f98b01e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98affff640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 0x7f98b00c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 -- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b00cc210 msgr2=0x7f98b01e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b00cc210 0x7f98b01e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 -- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 msgr2=0x7f98b00c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 0x7f98b00c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98b563f640 1 -- 172.21.15.53:0/3598430137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0094d0 con 0x7f98b0159db0 2023-10-21T17:35:50.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.015+0000 7f98affff640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 0x7f98b00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:50.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98b563f640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c6e0 tx=0x7f98a800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8003b70 con 0x7f98b0159db0 2023-10-21T17:35:50.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8003cf0 con 0x7f98b0159db0 2023-10-21T17:35:50.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00ae0a0 con 0x7f98b0159db0 2023-10-21T17:35:50.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005310 con 0x7f98b0159db0 2023-10-21T17:35:50.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.016+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00ae340 con 0x7f98b0159db0 2023-10-21T17:35:50.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.017+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b004e160 con 0x7f98b0159db0 2023-10-21T17:35:50.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.018+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f98a8005470 con 0x7f98b0159db0 2023-10-21T17:35:50.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.019+0000 7f98ad7fa640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 0x7f989406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.019+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f98a8089670 con 0x7f98b0159db0 2023-10-21T17:35:50.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.020+0000 7f98b4e3e640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 0x7f989406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.021+0000 7f98b4e3e640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 0x7f989406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98a4006fd0 tx=0x7f98a4008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.023+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98a8054440 con 0x7f98b0159db0 2023-10-21T17:35:50.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.213+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 --> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98b00ca070 con 0x7f989406af90 2023-10-21T17:35:50.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.235+0000 7f98ad7fa640 1 -- 172.21.15.53:0/3598430137 <== mgr.37520 v2:172.21.15.124:6832/248442090 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286153 (secure 0 0 0) 0x7f98a400b200 con 0x7f989406af90 2023-10-21T17:35:50.240 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:50.242 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:35:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 msgr2=0x7f989406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 0x7f989406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98a4006fd0 tx=0x7f98a4008040 comp rx=0 tx=0).stop 2023-10-21T17:35:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 msgr2=0x7f98b0150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f98a800c6e0 tx=0x7f98a800cba0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 shutdown_connections 2023-10-21T17:35:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f989406af90 0x7f989406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98b0159db0 0x7f98b0150b70 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98b01599d0 0x7f98b00c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 --2- 172.21.15.53:0/3598430137 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98b00cc210 0x7f98b01e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 >> 172.21.15.53:0/3598430137 conn(0x7f98b00cb0c0 msgr2=0x7f98b00b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 shutdown_connections 2023-10-21T17:35:50.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.241+0000 7f98b5e40640 1 -- 172.21.15.53:0/3598430137 wait complete. 2023-10-21T17:35:50.264 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-21T17:35:49.555432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":591,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4889,"ondisk_log_size":4889,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410484,"kb_used_data":22196,"kb_used_omap":12,"kb_used_meta":388147,"kb_avail":754564236,"statfs":{"total":773094113280,"available":772673777664,"internally_reserved":0,"allocated":22728704,"data_stored":12269220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":397463097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002283"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206414+0000","last_change":"2023-10-21T17:35:45.234115+0000","last_active":"2023-10-21T17:35:46.206414+0000","last_peered":"2023-10-21T17:35:46.206414+0000","last_clean":"2023-10-21T17:35:46.206414+0000","last_became_active":"2023-10-21T17:35:45.233817+0000","last_became_peered":"2023-10-21T17:35:45.233817+0000","last_unstale":"2023-10-21T17:35:46.206414+0000","last_undegraded":"2023-10-21T17:35:46.206414+0000","last_fullsized":"2023-10-21T17:35:46.206414+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:48.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205979+0000","last_change":"2023-10-21T17:35:45.243702+0000","last_active":"2023-10-21T17:35:46.205979+0000","last_peered":"2023-10-21T17:35:46.205979+0000","last_clean":"2023-10-21T17:35:46.205979+0000","last_became_active":"2023-10-21T17:35:45.243449+0000","last_became_peered":"2023-10-21T17:35:45.243449+0000","last_unstale":"2023-10-21T17:35:46.205979+0000","last_undegraded":"2023-10-21T17:35:46.205979+0000","last_fullsized":"2023-10-21T17:35:46.205979+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:25.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.238897+0000","last_change":"2023-10-21T17:35:45.236275+0000","last_active":"2023-10-21T17:35:45.238897+0000","last_peered":"2023-10-21T17:35:45.238897+0000","last_clean":"2023-10-21T17:35:45.238897+0000","last_became_active":"2023-10-21T17:35:45.235963+0000","last_became_peered":"2023-10-21T17:35:45.235963+0000","last_unstale":"2023-10-21T17:35:45.238897+0000","last_undegraded":"2023-10-21T17:35:45.238897+0000","last_fullsized":"2023-10-21T17:35:45.238897+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:49:06.104591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.233829+0000","last_change":"2023-10-21T17:35:45.232026+0000","last_active":"2023-10-21T17:35:45.233829+0000","last_peered":"2023-10-21T17:35:45.233829+0000","last_clean":"2023-10-21T17:35:45.233829+0000","last_became_active":"2023-10-21T17:35:45.231712+0000","last_became_peered":"2023-10-21T17:35:45.231712+0000","last_unstale":"2023-10-21T17:35:45.233829+0000","last_undegraded":"2023-10-21T17:35:45.233829+0000","last_fullsized":"2023-10-21T17:35:45.233829+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:48.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245996+0000","last_change":"2023-10-21T17:35:45.244455+0000","last_active":"2023-10-21T17:35:45.245996+0000","last_peered":"2023-10-21T17:35:45.245996+0000","last_clean":"2023-10-21T17:35:45.245996+0000","last_became_active":"2023-10-21T17:35:45.244144+0000","last_became_peered":"2023-10-21T17:35:45.244144+0000","last_unstale":"2023-10-21T17:35:45.245996+0000","last_undegraded":"2023-10-21T17:35:45.245996+0000","last_fullsized":"2023-10-21T17:35:45.245996+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:00.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.240763+0000","last_change":"2023-10-21T17:35:45.230916+0000","last_active":"2023-10-21T17:35:45.240763+0000","last_peered":"2023-10-21T17:35:45.240763+0000","last_clean":"2023-10-21T17:35:45.240763+0000","last_became_active":"2023-10-21T17:35:45.230618+0000","last_became_peered":"2023-10-21T17:35:45.230618+0000","last_unstale":"2023-10-21T17:35:45.240763+0000","last_undegraded":"2023-10-21T17:35:45.240763+0000","last_fullsized":"2023-10-21T17:35:45.240763+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:27.144841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209588+0000","last_change":"2023-10-21T17:35:45.233455+0000","last_active":"2023-10-21T17:35:46.209588+0000","last_peered":"2023-10-21T17:35:46.209588+0000","last_clean":"2023-10-21T17:35:46.209588+0000","last_became_active":"2023-10-21T17:35:45.231067+0000","last_became_peered":"2023-10-21T17:35:45.231067+0000","last_unstale":"2023-10-21T17:35:46.209588+0000","last_undegraded":"2023-10-21T17:35:46.209588+0000","last_fullsized":"2023-10-21T17:35:46.209588+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:56.440925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245160+0000","last_change":"2023-10-21T17:35:45.241868+0000","last_active":"2023-10-21T17:35:45.245160+0000","last_peered":"2023-10-21T17:35:45.245160+0000","last_clean":"2023-10-21T17:35:45.245160+0000","last_became_active":"2023-10-21T17:35:45.241344+0000","last_became_peered":"2023-10-21T17:35:45.241344+0000","last_unstale":"2023-10-21T17:35:45.245160+0000","last_undegraded":"2023-10-21T17:35:45.245160+0000","last_fullsized":"2023-10-21T17:35:45.245160+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:15.800521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206551+0000","last_change":"2023-10-21T17:35:45.238404+0000","last_active":"2023-10-21T17:35:46.206551+0000","last_peered":"2023-10-21T17:35:46.206551+0000","last_clean":"2023-10-21T17:35:46.206551+0000","last_became_active":"2023-10-21T17:35:45.237467+0000","last_became_peered":"2023-10-21T17:35:45.237467+0000","last_unstale":"2023-10-21T17:35:46.206551+0000","last_undegraded":"2023-10-21T17:35:46.206551+0000","last_fullsized":"2023-10-21T17:35:46.206551+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:57:17.400080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242122+0000","last_change":"2023-10-21T17:35:45.238905+0000","last_active":"2023-10-21T17:35:45.242122+0000","last_peered":"2023-10-21T17:35:45.242122+0000","last_clean":"2023-10-21T17:35:45.242122+0000","last_became_active":"2023-10-21T17:35:45.238621+0000","last_became_peered":"2023-10-21T17:35:45.238621+0000","last_unstale":"2023-10-21T17:35:45.242122+0000","last_undegraded":"2023-10-21T17:35:45.242122+0000","last_fullsized":"2023-10-21T17:35:45.242122+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:10:02.923295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241795+0000","last_change":"2023-10-21T17:35:45.233757+0000","last_active":"2023-10-21T17:35:45.241795+0000","last_peered":"2023-10-21T17:35:45.241795+0000","last_clean":"2023-10-21T17:35:45.241795+0000","last_became_active":"2023-10-21T17:35:45.233187+0000","last_became_peered":"2023-10-21T17:35:45.233187+0000","last_unstale":"2023-10-21T17:35:45.241795+0000","last_undegraded":"2023-10-21T17:35:45.241795+0000","last_fullsized":"2023-10-21T17:35:45.241795+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:00:50.852724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242612+0000","last_change":"2023-10-21T17:35:45.239715+0000","last_active":"2023-10-21T17:35:45.242612+0000","last_peered":"2023-10-21T17:35:45.242612+0000","last_clean":"2023-10-21T17:35:45.242612+0000","last_became_active":"2023-10-21T17:35:45.239435+0000","last_became_peered":"2023-10-21T17:35:45.239435+0000","last_unstale":"2023-10-21T17:35:45.242612+0000","last_undegraded":"2023-10-21T17:35:45.242612+0000","last_fullsized":"2023-10-21T17:35:45.242612+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.274330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.208983+0000","last_change":"2023-10-21T17:35:45.238625+0000","last_active":"2023-10-21T17:35:46.208983+0000","last_peered":"2023-10-21T17:35:46.208983+0000","last_clean":"2023-10-21T17:35:46.208983+0000","last_became_active":"2023-10-21T17:35:45.238226+0000","last_became_peered":"2023-10-21T17:35:45.238226+0000","last_unstale":"2023-10-21T17:35:46.208983+0000","last_undegraded":"2023-10-21T17:35:46.208983+0000","last_fullsized":"2023-10-21T17:35:46.208983+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:42.947024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209672+0000","last_change":"2023-10-21T17:35:45.221930+0000","last_active":"2023-10-21T17:35:46.209672+0000","last_peered":"2023-10-21T17:35:46.209672+0000","last_clean":"2023-10-21T17:35:46.209672+0000","last_became_active":"2023-10-21T17:35:45.221549+0000","last_became_peered":"2023-10-21T17:35:45.221549+0000","last_unstale":"2023-10-21T17:35:46.209672+0000","last_undegraded":"2023-10-21T17:35:46.209672+0000","last_fullsized":"2023-10-21T17:35:46.209672+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:40:38.535179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205677+0000","last_change":"2023-10-21T17:35:45.227671+0000","last_active":"2023-10-21T17:35:46.205677+0000","last_peered":"2023-10-21T17:35:46.205677+0000","last_clean":"2023-10-21T17:35:46.205677+0000","last_became_active":"2023-10-21T17:35:45.223658+0000","last_became_peered":"2023-10-21T17:35:45.223658+0000","last_unstale":"2023-10-21T17:35:46.205677+0000","last_undegraded":"2023-10-21T17:35:46.205677+0000","last_fullsized":"2023-10-21T17:35:46.205677+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:49:41.942050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.246550+0000","last_change":"2023-10-21T17:35:45.245067+0000","last_active":"2023-10-21T17:35:45.246550+0000","last_peered":"2023-10-21T17:35:45.246550+0000","last_clean":"2023-10-21T17:35:45.246550+0000","last_became_active":"2023-10-21T17:35:45.244422+0000","last_became_peered":"2023-10-21T17:35:45.244422+0000","last_unstale":"2023-10-21T17:35:45.246550+0000","last_undegraded":"2023-10-21T17:35:45.246550+0000","last_fullsized":"2023-10-21T17:35:45.246550+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:33.447916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.246259+0000","last_change":"2023-10-21T17:35:45.245199+0000","last_active":"2023-10-21T17:35:45.246259+0000","last_peered":"2023-10-21T17:35:45.246259+0000","last_clean":"2023-10-21T17:35:45.246259+0000","last_became_active":"2023-10-21T17:35:45.243823+0000","last_became_peered":"2023-10-21T17:35:45.243823+0000","last_unstale":"2023-10-21T17:35:45.246259+0000","last_undegraded":"2023-10-21T17:35:45.246259+0000","last_fullsized":"2023-10-21T17:35:45.246259+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:57.462655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209942+0000","last_change":"2023-10-21T17:35:45.232181+0000","last_active":"2023-10-21T17:35:46.209942+0000","last_peered":"2023-10-21T17:35:46.209942+0000","last_clean":"2023-10-21T17:35:46.209942+0000","last_became_active":"2023-10-21T17:35:45.230703+0000","last_became_peered":"2023-10-21T17:35:45.230703+0000","last_unstale":"2023-10-21T17:35:46.209942+0000","last_undegraded":"2023-10-21T17:35:46.209942+0000","last_fullsized":"2023-10-21T17:35:46.209942+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:08.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209129+0000","last_change":"2023-10-21T17:35:45.231754+0000","last_active":"2023-10-21T17:35:46.209129+0000","last_peered":"2023-10-21T17:35:46.209129+0000","last_clean":"2023-10-21T17:35:46.209129+0000","last_became_active":"2023-10-21T17:35:45.230978+0000","last_became_peered":"2023-10-21T17:35:45.230978+0000","last_unstale":"2023-10-21T17:35:46.209129+0000","last_undegraded":"2023-10-21T17:35:46.209129+0000","last_fullsized":"2023-10-21T17:35:46.209129+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:22:08.607825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245537+0000","last_change":"2023-10-21T17:35:45.243528+0000","last_active":"2023-10-21T17:35:45.245537+0000","last_peered":"2023-10-21T17:35:45.245537+0000","last_clean":"2023-10-21T17:35:45.245537+0000","last_became_active":"2023-10-21T17:35:45.243269+0000","last_became_peered":"2023-10-21T17:35:45.243269+0000","last_unstale":"2023-10-21T17:35:45.245537+0000","last_undegraded":"2023-10-21T17:35:45.245537+0000","last_fullsized":"2023-10-21T17:35:45.245537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:33:12.456009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205997+0000","last_change":"2023-10-21T17:35:45.239010+0000","last_active":"2023-10-21T17:35:46.205997+0000","last_peered":"2023-10-21T17:35:46.205997+0000","last_clean":"2023-10-21T17:35:46.205997+0000","last_became_active":"2023-10-21T17:35:45.237970+0000","last_became_peered":"2023-10-21T17:35:45.237970+0000","last_unstale":"2023-10-21T17:35:46.205997+0000","last_undegraded":"2023-10-21T17:35:46.205997+0000","last_fullsized":"2023-10-21T17:35:46.205997+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:37.274330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205429+0000","last_change":"2023-10-21T17:35:45.230336+0000","last_active":"2023-10-21T17:35:46.205429+0000","last_peered":"2023-10-21T17:35:46.205429+0000","last_clean":"2023-10-21T17:35:46.205429+0000","last_became_active":"2023-10-21T17:35:45.230040+0000","last_became_peered":"2023-10-21T17:35:45.230040+0000","last_unstale":"2023-10-21T17:35:46.205429+0000","last_undegraded":"2023-10-21T17:35:46.205429+0000","last_fullsized":"2023-10-21T17:35:46.205429+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:47:11.413681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244168+0000","last_change":"2023-10-21T17:35:45.240746+0000","last_active":"2023-10-21T17:35:45.244168+0000","last_peered":"2023-10-21T17:35:45.244168+0000","last_clean":"2023-10-21T17:35:45.244168+0000","last_became_active":"2023-10-21T17:35:45.240472+0000","last_became_peered":"2023-10-21T17:35:45.240472+0000","last_unstale":"2023-10-21T17:35:45.244168+0000","last_undegraded":"2023-10-21T17:35:45.244168+0000","last_fullsized":"2023-10-21T17:35:45.244168+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:30:26.460976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205090+0000","last_change":"2023-10-21T17:35:45.235725+0000","last_active":"2023-10-21T17:35:46.205090+0000","last_peered":"2023-10-21T17:35:46.205090+0000","last_clean":"2023-10-21T17:35:46.205090+0000","last_became_active":"2023-10-21T17:35:45.235351+0000","last_became_peered":"2023-10-21T17:35:45.235351+0000","last_unstale":"2023-10-21T17:35:46.205090+0000","last_undegraded":"2023-10-21T17:35:46.205090+0000","last_fullsized":"2023-10-21T17:35:46.205090+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:19:10.084399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209634+0000","last_change":"2023-10-21T17:35:45.244543+0000","last_active":"2023-10-21T17:35:46.209634+0000","last_peered":"2023-10-21T17:35:46.209634+0000","last_clean":"2023-10-21T17:35:46.209634+0000","last_became_active":"2023-10-21T17:35:45.244149+0000","last_became_peered":"2023-10-21T17:35:45.244149+0000","last_unstale":"2023-10-21T17:35:46.209634+0000","last_undegraded":"2023-10-21T17:35:46.209634+0000","last_fullsized":"2023-10-21T17:35:46.209634+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:30:41.440754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241928+0000","last_change":"2023-10-21T17:35:45.231867+0000","last_active":"2023-10-21T17:35:45.241928+0000","last_peered":"2023-10-21T17:35:45.241928+0000","last_clean":"2023-10-21T17:35:45.241928+0000","last_became_active":"2023-10-21T17:35:45.231607+0000","last_became_peered":"2023-10-21T17:35:45.231607+0000","last_unstale":"2023-10-21T17:35:45.241928+0000","last_undegraded":"2023-10-21T17:35:45.241928+0000","last_fullsized":"2023-10-21T17:35:45.241928+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:24.285732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241543+0000","last_change":"2023-10-21T17:35:45.234489+0000","last_active":"2023-10-21T17:35:45.241543+0000","last_peered":"2023-10-21T17:35:45.241543+0000","last_clean":"2023-10-21T17:35:45.241543+0000","last_became_active":"2023-10-21T17:35:45.234203+0000","last_became_peered":"2023-10-21T17:35:45.234203+0000","last_unstale":"2023-10-21T17:35:45.241543+0000","last_undegraded":"2023-10-21T17:35:45.241543+0000","last_fullsized":"2023-10-21T17:35:45.241543+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:51.322249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.245421+0000","last_change":"2023-10-21T17:35:45.242108+0000","last_active":"2023-10-21T17:35:45.245421+0000","last_peered":"2023-10-21T17:35:45.245421+0000","last_clean":"2023-10-21T17:35:45.245421+0000","last_became_active":"2023-10-21T17:35:45.241824+0000","last_became_peered":"2023-10-21T17:35:45.241824+0000","last_unstale":"2023-10-21T17:35:45.245421+0000","last_undegraded":"2023-10-21T17:35:45.245421+0000","last_fullsized":"2023-10-21T17:35:45.245421+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:16.591606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241840+0000","last_change":"2023-10-21T17:35:45.239001+0000","last_active":"2023-10-21T17:35:45.241840+0000","last_peered":"2023-10-21T17:35:45.241840+0000","last_clean":"2023-10-21T17:35:45.241840+0000","last_became_active":"2023-10-21T17:35:45.238729+0000","last_became_peered":"2023-10-21T17:35:45.238729+0000","last_unstale":"2023-10-21T17:35:45.241840+0000","last_undegraded":"2023-10-21T17:35:45.241840+0000","last_fullsized":"2023-10-21T17:35:45.241840+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:30:26.460976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.240307+0000","last_change":"2023-10-21T17:35:45.230074+0000","last_active":"2023-10-21T17:35:45.240307+0000","last_peered":"2023-10-21T17:35:45.240307+0000","last_clean":"2023-10-21T17:35:45.240307+0000","last_became_active":"2023-10-21T17:35:45.229623+0000","last_became_peered":"2023-10-21T17:35:45.229623+0000","last_unstale":"2023-10-21T17:35:45.240307+0000","last_undegraded":"2023-10-21T17:35:45.240307+0000","last_fullsized":"2023-10-21T17:35:45.240307+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:12:57.030355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209885+0000","last_change":"2023-10-21T17:35:45.229224+0000","last_active":"2023-10-21T17:35:46.209885+0000","last_peered":"2023-10-21T17:35:46.209885+0000","last_clean":"2023-10-21T17:35:46.209885+0000","last_became_active":"2023-10-21T17:35:45.226585+0000","last_became_peered":"2023-10-21T17:35:45.226585+0000","last_unstale":"2023-10-21T17:35:46.209885+0000","last_undegraded":"2023-10-21T17:35:46.209885+0000","last_fullsized":"2023-10-21T17:35:46.209885+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:07:28.509289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244955+0000","last_change":"2023-10-21T17:35:45.240863+0000","last_active":"2023-10-21T17:35:45.244955+0000","last_peered":"2023-10-21T17:35:45.244955+0000","last_clean":"2023-10-21T17:35:45.244955+0000","last_became_active":"2023-10-21T17:35:45.240597+0000","last_became_peered":"2023-10-21T17:35:45.240597+0000","last_unstale":"2023-10-21T17:35:45.244955+0000","last_undegraded":"2023-10-21T17:35:45.244955+0000","last_fullsized":"2023-10-21T17:35:45.244955+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:29:28.861719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242566+0000","last_change":"2023-10-21T17:35:45.234278+0000","last_active":"2023-10-21T17:35:45.242566+0000","last_peered":"2023-10-21T17:35:45.242566+0000","last_clean":"2023-10-21T17:35:45.242566+0000","last_became_active":"2023-10-21T17:35:45.233439+0000","last_became_peered":"2023-10-21T17:35:45.233439+0000","last_unstale":"2023-10-21T17:35:45.242566+0000","last_undegraded":"2023-10-21T17:35:45.242566+0000","last_fullsized":"2023-10-21T17:35:45.242566+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:12:37.578306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"214'4","reported_seq":15,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261273+0000","last_change":"2023-10-21T17:35:44.220480+0000","last_active":"2023-10-21T17:35:45.261273+0000","last_peered":"2023-10-21T17:35:45.261273+0000","last_clean":"2023-10-21T17:35:45.261273+0000","last_became_active":"2023-10-21T17:35:44.220222+0000","last_became_peered":"2023-10-21T17:35:44.220222+0000","last_unstale":"2023-10-21T17:35:45.261273+0000","last_undegraded":"2023-10-21T17:35:45.261273+0000","last_fullsized":"2023-10-21T17:35:45.261273+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:39:20.044047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209884+0000","last_change":"2023-10-21T17:35:45.234533+0000","last_active":"2023-10-21T17:35:46.209884+0000","last_peered":"2023-10-21T17:35:46.209884+0000","last_clean":"2023-10-21T17:35:46.209884+0000","last_became_active":"2023-10-21T17:35:45.234225+0000","last_became_peered":"2023-10-21T17:35:45.234225+0000","last_unstale":"2023-10-21T17:35:46.209884+0000","last_undegraded":"2023-10-21T17:35:46.209884+0000","last_fullsized":"2023-10-21T17:35:46.209884+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:48.532550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261315+0000","last_change":"2023-10-21T17:35:44.223103+0000","last_active":"2023-10-21T17:35:45.261315+0000","last_peered":"2023-10-21T17:35:45.261315+0000","last_clean":"2023-10-21T17:35:45.261315+0000","last_became_active":"2023-10-21T17:35:44.216646+0000","last_became_peered":"2023-10-21T17:35:44.216646+0000","last_unstale":"2023-10-21T17:35:45.261315+0000","last_undegraded":"2023-10-21T17:35:45.261315+0000","last_fullsized":"2023-10-21T17:35:45.261315+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:46:48.139508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241968+0000","last_change":"2023-10-21T17:35:45.237596+0000","last_active":"2023-10-21T17:35:45.241968+0000","last_peered":"2023-10-21T17:35:45.241968+0000","last_clean":"2023-10-21T17:35:45.241968+0000","last_became_active":"2023-10-21T17:35:45.235744+0000","last_became_peered":"2023-10-21T17:35:45.235744+0000","last_unstale":"2023-10-21T17:35:45.241968+0000","last_undegraded":"2023-10-21T17:35:45.241968+0000","last_fullsized":"2023-10-21T17:35:45.241968+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:43:40.498014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.259484+0000","last_change":"2023-10-21T17:35:44.204037+0000","last_active":"2023-10-21T17:35:45.259484+0000","last_peered":"2023-10-21T17:35:45.259484+0000","last_clean":"2023-10-21T17:35:45.259484+0000","last_became_active":"2023-10-21T17:35:44.203742+0000","last_became_peered":"2023-10-21T17:35:44.203742+0000","last_unstale":"2023-10-21T17:35:45.259484+0000","last_undegraded":"2023-10-21T17:35:45.259484+0000","last_fullsized":"2023-10-21T17:35:45.259484+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:09.315964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.243919+0000","last_change":"2023-10-21T17:35:45.242387+0000","last_active":"2023-10-21T17:35:45.243919+0000","last_peered":"2023-10-21T17:35:45.243919+0000","last_clean":"2023-10-21T17:35:45.243919+0000","last_became_active":"2023-10-21T17:35:45.242061+0000","last_became_peered":"2023-10-21T17:35:45.242061+0000","last_unstale":"2023-10-21T17:35:45.243919+0000","last_undegraded":"2023-10-21T17:35:45.243919+0000","last_fullsized":"2023-10-21T17:35:45.243919+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:07.662607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209465+0000","last_change":"2023-10-21T17:35:44.203338+0000","last_active":"2023-10-21T17:35:46.209465+0000","last_peered":"2023-10-21T17:35:46.209465+0000","last_clean":"2023-10-21T17:35:46.209465+0000","last_became_active":"2023-10-21T17:35:44.202951+0000","last_became_peered":"2023-10-21T17:35:44.202951+0000","last_unstale":"2023-10-21T17:35:46.209465+0000","last_undegraded":"2023-10-21T17:35:46.209465+0000","last_fullsized":"2023-10-21T17:35:46.209465+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:34.813742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.208441+0000","last_change":"2023-10-21T17:35:45.239493+0000","last_active":"2023-10-21T17:35:46.208441+0000","last_peered":"2023-10-21T17:35:46.208441+0000","last_clean":"2023-10-21T17:35:46.208441+0000","last_became_active":"2023-10-21T17:35:45.239016+0000","last_became_peered":"2023-10-21T17:35:45.239016+0000","last_unstale":"2023-10-21T17:35:46.208441+0000","last_undegraded":"2023-10-21T17:35:46.208441+0000","last_fullsized":"2023-10-21T17:35:46.208441+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:16.531282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.261013+0000","last_change":"2023-10-21T17:35:44.213731+0000","last_active":"2023-10-21T17:35:45.261013+0000","last_peered":"2023-10-21T17:35:45.261013+0000","last_clean":"2023-10-21T17:35:45.261013+0000","last_became_active":"2023-10-21T17:35:44.213257+0000","last_became_peered":"2023-10-21T17:35:44.213257+0000","last_unstale":"2023-10-21T17:35:45.261013+0000","last_undegraded":"2023-10-21T17:35:45.261013+0000","last_fullsized":"2023-10-21T17:35:45.261013+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:23.614892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205851+0000","last_change":"2023-10-21T17:35:45.225488+0000","last_active":"2023-10-21T17:35:46.205851+0000","last_peered":"2023-10-21T17:35:46.205851+0000","last_clean":"2023-10-21T17:35:46.205851+0000","last_became_active":"2023-10-21T17:35:45.225186+0000","last_became_peered":"2023-10-21T17:35:45.225186+0000","last_unstale":"2023-10-21T17:35:46.205851+0000","last_undegraded":"2023-10-21T17:35:46.205851+0000","last_fullsized":"2023-10-21T17:35:46.205851+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:29.398804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.257627+0000","last_change":"2023-10-21T17:35:44.221936+0000","last_active":"2023-10-21T17:35:45.257627+0000","last_peered":"2023-10-21T17:35:45.257627+0000","last_clean":"2023-10-21T17:35:45.257627+0000","last_became_active":"2023-10-21T17:35:44.221652+0000","last_became_peered":"2023-10-21T17:35:44.221652+0000","last_unstale":"2023-10-21T17:35:45.257627+0000","last_undegraded":"2023-10-21T17:35:45.257627+0000","last_fullsized":"2023-10-21T17:35:45.257627+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:24.373341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205742+0000","last_change":"2023-10-21T17:35:45.226625+0000","last_active":"2023-10-21T17:35:46.205742+0000","last_peered":"2023-10-21T17:35:46.205742+0000","last_clean":"2023-10-21T17:35:46.205742+0000","last_became_active":"2023-10-21T17:35:45.226339+0000","last_became_peered":"2023-10-21T17:35:45.226339+0000","last_unstale":"2023-10-21T17:35:46.205742+0000","last_undegraded":"2023-10-21T17:35:46.205742+0000","last_fullsized":"2023-10-21T17:35:46.205742+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:34.903077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209475+0000","last_change":"2023-10-21T17:35:44.213771+0000","last_active":"2023-10-21T17:35:46.209475+0000","last_peered":"2023-10-21T17:35:46.209475+0000","last_clean":"2023-10-21T17:35:46.209475+0000","last_became_active":"2023-10-21T17:35:44.213506+0000","last_became_peered":"2023-10-21T17:35:44.213506+0000","last_unstale":"2023-10-21T17:35:46.209475+0000","last_undegraded":"2023-10-21T17:35:46.209475+0000","last_fullsized":"2023-10-21T17:35:46.209475+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:11:04.186955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242586+0000","last_change":"2023-10-21T17:35:45.235426+0000","last_active":"2023-10-21T17:35:45.242586+0000","last_peered":"2023-10-21T17:35:45.242586+0000","last_clean":"2023-10-21T17:35:45.242586+0000","last_became_active":"2023-10-21T17:35:45.233025+0000","last_became_peered":"2023-10-21T17:35:45.233025+0000","last_unstale":"2023-10-21T17:35:45.242586+0000","last_undegraded":"2023-10-21T17:35:45.242586+0000","last_fullsized":"2023-10-21T17:35:45.242586+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:26:35.352137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.236231+0000","last_change":"2023-10-21T17:35:44.224883+0000","last_active":"2023-10-21T17:35:45.236231+0000","last_peered":"2023-10-21T17:35:45.236231+0000","last_clean":"2023-10-21T17:35:45.236231+0000","last_became_active":"2023-10-21T17:35:44.224609+0000","last_became_peered":"2023-10-21T17:35:44.224609+0000","last_unstale":"2023-10-21T17:35:45.236231+0000","last_undegraded":"2023-10-21T17:35:45.236231+0000","last_fullsized":"2023-10-21T17:35:45.236231+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:55:01.441588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.239980+0000","last_change":"2023-10-21T17:35:45.231159+0000","last_active":"2023-10-21T17:35:45.239980+0000","last_peered":"2023-10-21T17:35:45.239980+0000","last_clean":"2023-10-21T17:35:45.239980+0000","last_became_active":"2023-10-21T17:35:45.230017+0000","last_became_peered":"2023-10-21T17:35:45.230017+0000","last_unstale":"2023-10-21T17:35:45.239980+0000","last_undegraded":"2023-10-21T17:35:45.239980+0000","last_fullsized":"2023-10-21T17:35:45.239980+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:53:13.873224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206194+0000","last_change":"2023-10-21T17:35:44.221513+0000","last_active":"2023-10-21T17:35:46.206194+0000","last_peered":"2023-10-21T17:35:46.206194+0000","last_clean":"2023-10-21T17:35:46.206194+0000","last_became_active":"2023-10-21T17:35:44.221214+0000","last_became_peered":"2023-10-21T17:35:44.221214+0000","last_unstale":"2023-10-21T17:35:46.206194+0000","last_undegraded":"2023-10-21T17:35:46.206194+0000","last_fullsized":"2023-10-21T17:35:46.206194+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:15:28.155784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.210034+0000","last_change":"2023-10-21T17:35:45.243652+0000","last_active":"2023-10-21T17:35:46.210034+0000","last_peered":"2023-10-21T17:35:46.210034+0000","last_clean":"2023-10-21T17:35:46.210034+0000","last_became_active":"2023-10-21T17:35:45.243330+0000","last_became_peered":"2023-10-21T17:35:45.243330+0000","last_unstale":"2023-10-21T17:35:46.210034+0000","last_undegraded":"2023-10-21T17:35:46.210034+0000","last_fullsized":"2023-10-21T17:35:46.210034+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:41:12.344281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"214'3","reported_seq":14,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260410+0000","last_change":"2023-10-21T17:35:44.227159+0000","last_active":"2023-10-21T17:35:45.260410+0000","last_peered":"2023-10-21T17:35:45.260410+0000","last_clean":"2023-10-21T17:35:45.260410+0000","last_became_active":"2023-10-21T17:35:44.226068+0000","last_became_peered":"2023-10-21T17:35:44.226068+0000","last_unstale":"2023-10-21T17:35:45.260410+0000","last_undegraded":"2023-10-21T17:35:45.260410+0000","last_fullsized":"2023-10-21T17:35:45.260410+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:33.886176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206827+0000","last_change":"2023-10-21T17:35:45.242375+0000","last_active":"2023-10-21T17:35:46.206827+0000","last_peered":"2023-10-21T17:35:46.206827+0000","last_clean":"2023-10-21T17:35:46.206827+0000","last_became_active":"2023-10-21T17:35:45.241990+0000","last_became_peered":"2023-10-21T17:35:45.241990+0000","last_unstale":"2023-10-21T17:35:46.206827+0000","last_undegraded":"2023-10-21T17:35:46.206827+0000","last_fullsized":"2023-10-21T17:35:46.206827+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:25.390242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"214'6","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209228+0000","last_change":"2023-10-21T17:35:44.207980+0000","last_active":"2023-10-21T17:35:46.209228+0000","last_peered":"2023-10-21T17:35:46.209228+0000","last_clean":"2023-10-21T17:35:46.209228+0000","last_became_active":"2023-10-21T17:35:44.207647+0000","last_became_peered":"2023-10-21T17:35:44.207647+0000","last_unstale":"2023-10-21T17:35:46.209228+0000","last_undegraded":"2023-10-21T17:35:46.209228+0000","last_fullsized":"2023-10-21T17:35:46.209228+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:23.231657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209335+0000","last_change":"2023-10-21T17:35:45.235686+0000","last_active":"2023-10-21T17:35:46.209335+0000","last_peered":"2023-10-21T17:35:46.209335+0000","last_clean":"2023-10-21T17:35:46.209335+0000","last_became_active":"2023-10-21T17:35:45.233499+0000","last_became_peered":"2023-10-21T17:35:45.233499+0000","last_unstale":"2023-10-21T17:35:46.209335+0000","last_undegraded":"2023-10-21T17:35:46.209335+0000","last_fullsized":"2023-10-21T17:35:46.209335+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:07:08.182881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209396+0000","last_change":"2023-10-21T17:35:44.223289+0000","last_active":"2023-10-21T17:35:46.209396+0000","last_peered":"2023-10-21T17:35:46.209396+0000","last_clean":"2023-10-21T17:35:46.209396+0000","last_became_active":"2023-10-21T17:35:44.222980+0000","last_became_peered":"2023-10-21T17:35:44.222980+0000","last_unstale":"2023-10-21T17:35:46.209396+0000","last_undegraded":"2023-10-21T17:35:46.209396+0000","last_fullsized":"2023-10-21T17:35:46.209396+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:22:21.139748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241495+0000","last_change":"2023-10-21T17:35:45.237444+0000","last_active":"2023-10-21T17:35:45.241495+0000","last_peered":"2023-10-21T17:35:45.241495+0000","last_clean":"2023-10-21T17:35:45.241495+0000","last_became_active":"2023-10-21T17:35:45.237059+0000","last_became_peered":"2023-10-21T17:35:45.237059+0000","last_unstale":"2023-10-21T17:35:45.241495+0000","last_undegraded":"2023-10-21T17:35:45.241495+0000","last_fullsized":"2023-10-21T17:35:45.241495+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:31.291034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205638+0000","last_change":"2023-10-21T17:35:44.225472+0000","last_active":"2023-10-21T17:35:46.205638+0000","last_peered":"2023-10-21T17:35:46.205638+0000","last_clean":"2023-10-21T17:35:46.205638+0000","last_became_active":"2023-10-21T17:35:44.225180+0000","last_became_peered":"2023-10-21T17:35:44.225180+0000","last_unstale":"2023-10-21T17:35:46.205638+0000","last_undegraded":"2023-10-21T17:35:46.205638+0000","last_fullsized":"2023-10-21T17:35:46.205638+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:23:26.446533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209018+0000","last_change":"2023-10-21T17:35:45.225834+0000","last_active":"2023-10-21T17:35:46.209018+0000","last_peered":"2023-10-21T17:35:46.209018+0000","last_clean":"2023-10-21T17:35:46.209018+0000","last_became_active":"2023-10-21T17:35:45.224902+0000","last_became_peered":"2023-10-21T17:35:45.224902+0000","last_unstale":"2023-10-21T17:35:46.209018+0000","last_undegraded":"2023-10-21T17:35:46.209018+0000","last_fullsized":"2023-10-21T17:35:46.209018+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:57:12.752341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223250+0000","last_change":"2023-10-21T17:35:44.216564+0000","last_active":"2023-10-21T17:35:45.223250+0000","last_peered":"2023-10-21T17:35:45.223250+0000","last_clean":"2023-10-21T17:35:45.223250+0000","last_became_active":"2023-10-21T17:35:44.216305+0000","last_became_peered":"2023-10-21T17:35:44.216305+0000","last_unstale":"2023-10-21T17:35:45.223250+0000","last_undegraded":"2023-10-21T17:35:45.223250+0000","last_fullsized":"2023-10-21T17:35:45.223250+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:08.559913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209375+0000","last_change":"2023-10-21T17:35:45.234640+0000","last_active":"2023-10-21T17:35:46.209375+0000","last_peered":"2023-10-21T17:35:46.209375+0000","last_clean":"2023-10-21T17:35:46.209375+0000","last_became_active":"2023-10-21T17:35:45.234371+0000","last_became_peered":"2023-10-21T17:35:45.234371+0000","last_unstale":"2023-10-21T17:35:46.209375+0000","last_undegraded":"2023-10-21T17:35:46.209375+0000","last_fullsized":"2023-10-21T17:35:46.209375+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:43:33.498382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"214'8","reported_seq":19,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.262302+0000","last_change":"2023-10-21T17:35:44.210922+0000","last_active":"2023-10-21T17:35:45.262302+0000","last_peered":"2023-10-21T17:35:45.262302+0000","last_clean":"2023-10-21T17:35:45.262302+0000","last_became_active":"2023-10-21T17:35:44.210637+0000","last_became_peered":"2023-10-21T17:35:44.210637+0000","last_unstale":"2023-10-21T17:35:45.262302+0000","last_undegraded":"2023-10-21T17:35:45.262302+0000","last_fullsized":"2023-10-21T17:35:45.262302+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:14.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.210359+0000","last_change":"2023-10-21T17:35:45.227113+0000","last_active":"2023-10-21T17:35:46.210359+0000","last_peered":"2023-10-21T17:35:46.210359+0000","last_clean":"2023-10-21T17:35:46.210359+0000","last_became_active":"2023-10-21T17:35:45.226673+0000","last_became_peered":"2023-10-21T17:35:45.226673+0000","last_unstale":"2023-10-21T17:35:46.210359+0000","last_undegraded":"2023-10-21T17:35:46.210359+0000","last_fullsized":"2023-10-21T17:35:46.210359+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:10.087693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260897+0000","last_change":"2023-10-21T17:35:44.213336+0000","last_active":"2023-10-21T17:35:45.260897+0000","last_peered":"2023-10-21T17:35:45.260897+0000","last_clean":"2023-10-21T17:35:45.260897+0000","last_became_active":"2023-10-21T17:35:44.213037+0000","last_became_peered":"2023-10-21T17:35:44.213037+0000","last_unstale":"2023-10-21T17:35:45.260897+0000","last_undegraded":"2023-10-21T17:35:45.260897+0000","last_fullsized":"2023-10-21T17:35:45.260897+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:40.699779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244974+0000","last_change":"2023-10-21T17:35:45.241185+0000","last_active":"2023-10-21T17:35:45.244974+0000","last_peered":"2023-10-21T17:35:45.244974+0000","last_clean":"2023-10-21T17:35:45.244974+0000","last_became_active":"2023-10-21T17:35:45.237862+0000","last_became_peered":"2023-10-21T17:35:45.237862+0000","last_unstale":"2023-10-21T17:35:45.244974+0000","last_undegraded":"2023-10-21T17:35:45.244974+0000","last_fullsized":"2023-10-21T17:35:45.244974+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:09.437597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.259209+0000","last_change":"2023-10-21T17:35:44.213459+0000","last_active":"2023-10-21T17:35:45.259209+0000","last_peered":"2023-10-21T17:35:45.259209+0000","last_clean":"2023-10-21T17:35:45.259209+0000","last_became_active":"2023-10-21T17:35:44.213164+0000","last_became_peered":"2023-10-21T17:35:44.213164+0000","last_unstale":"2023-10-21T17:35:45.259209+0000","last_undegraded":"2023-10-21T17:35:45.259209+0000","last_fullsized":"2023-10-21T17:35:45.259209+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:31:28.381903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241951+0000","last_change":"2023-10-21T17:35:45.236322+0000","last_active":"2023-10-21T17:35:45.241951+0000","last_peered":"2023-10-21T17:35:45.241951+0000","last_clean":"2023-10-21T17:35:45.241951+0000","last_became_active":"2023-10-21T17:35:45.235217+0000","last_became_peered":"2023-10-21T17:35:45.235217+0000","last_unstale":"2023-10-21T17:35:45.241951+0000","last_undegraded":"2023-10-21T17:35:45.241951+0000","last_fullsized":"2023-10-21T17:35:45.241951+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:11:37.709638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"214'1","reported_seq":12,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.258877+0000","last_change":"2023-10-21T17:35:44.223638+0000","last_active":"2023-10-21T17:35:45.258877+0000","last_peered":"2023-10-21T17:35:45.258877+0000","last_clean":"2023-10-21T17:35:45.258877+0000","last_became_active":"2023-10-21T17:35:44.223334+0000","last_became_peered":"2023-10-21T17:35:44.223334+0000","last_unstale":"2023-10-21T17:35:45.258877+0000","last_undegraded":"2023-10-21T17:35:45.258877+0000","last_fullsized":"2023-10-21T17:35:45.258877+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:22:21.139748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.206884+0000","last_change":"2023-10-21T17:35:45.244190+0000","last_active":"2023-10-21T17:35:46.206884+0000","last_peered":"2023-10-21T17:35:46.206884+0000","last_clean":"2023-10-21T17:35:46.206884+0000","last_became_active":"2023-10-21T17:35:45.243815+0000","last_became_peered":"2023-10-21T17:35:45.243815+0000","last_unstale":"2023-10-21T17:35:46.206884+0000","last_undegraded":"2023-10-21T17:35:46.206884+0000","last_fullsized":"2023-10-21T17:35:46.206884+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:17.575909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.233389+0000","last_change":"2023-10-21T17:35:44.212510+0000","last_active":"2023-10-21T17:35:45.233389+0000","last_peered":"2023-10-21T17:35:45.233389+0000","last_clean":"2023-10-21T17:35:45.233389+0000","last_became_active":"2023-10-21T17:35:44.212211+0000","last_became_peered":"2023-10-21T17:35:44.212211+0000","last_unstale":"2023-10-21T17:35:45.233389+0000","last_undegraded":"2023-10-21T17:35:45.233389+0000","last_fullsized":"2023-10-21T17:35:45.233389+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:50.305348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205072+0000","last_change":"2023-10-21T17:35:45.243355+0000","last_active":"2023-10-21T17:35:46.205072+0000","last_peered":"2023-10-21T17:35:46.205072+0000","last_clean":"2023-10-21T17:35:46.205072+0000","last_became_active":"2023-10-21T17:35:45.243037+0000","last_became_peered":"2023-10-21T17:35:45.243037+0000","last_unstale":"2023-10-21T17:35:46.205072+0000","last_undegraded":"2023-10-21T17:35:46.205072+0000","last_fullsized":"2023-10-21T17:35:46.205072+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:36:09.029533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"214'2","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209634+0000","last_change":"2023-10-21T17:35:44.216966+0000","last_active":"2023-10-21T17:35:46.209634+0000","last_peered":"2023-10-21T17:35:46.209634+0000","last_clean":"2023-10-21T17:35:46.209634+0000","last_became_active":"2023-10-21T17:35:44.216657+0000","last_became_peered":"2023-10-21T17:35:44.216657+0000","last_unstale":"2023-10-21T17:35:46.209634+0000","last_undegraded":"2023-10-21T17:35:46.209634+0000","last_fullsized":"2023-10-21T17:35:46.209634+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:19.728787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.235239+0000","last_change":"2023-10-21T17:35:45.233934+0000","last_active":"2023-10-21T17:35:45.235239+0000","last_peered":"2023-10-21T17:35:45.235239+0000","last_clean":"2023-10-21T17:35:45.235239+0000","last_became_active":"2023-10-21T17:35:45.232750+0000","last_became_peered":"2023-10-21T17:35:45.232750+0000","last_unstale":"2023-10-21T17:35:45.235239+0000","last_undegraded":"2023-10-21T17:35:45.235239+0000","last_fullsized":"2023-10-21T17:35:45.235239+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:22.268282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.221248+0000","last_change":"2023-10-21T17:35:44.218400+0000","last_active":"2023-10-21T17:35:45.221248+0000","last_peered":"2023-10-21T17:35:45.221248+0000","last_clean":"2023-10-21T17:35:45.221248+0000","last_became_active":"2023-10-21T17:35:44.218134+0000","last_became_peered":"2023-10-21T17:35:44.218134+0000","last_unstale":"2023-10-21T17:35:45.221248+0000","last_undegraded":"2023-10-21T17:35:45.221248+0000","last_fullsized":"2023-10-21T17:35:45.221248+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:16.559008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242461+0000","last_change":"2023-10-21T17:35:45.241294+0000","last_active":"2023-10-21T17:35:45.242461+0000","last_peered":"2023-10-21T17:35:45.242461+0000","last_clean":"2023-10-21T17:35:45.242461+0000","last_became_active":"2023-10-21T17:35:45.240992+0000","last_became_peered":"2023-10-21T17:35:45.240992+0000","last_unstale":"2023-10-21T17:35:45.242461+0000","last_undegraded":"2023-10-21T17:35:45.242461+0000","last_fullsized":"2023-10-21T17:35:45.242461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:31:25.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209828+0000","last_change":"2023-10-21T17:35:44.211526+0000","last_active":"2023-10-21T17:35:46.209828+0000","last_peered":"2023-10-21T17:35:46.209828+0000","last_clean":"2023-10-21T17:35:46.209828+0000","last_became_active":"2023-10-21T17:35:44.211223+0000","last_became_peered":"2023-10-21T17:35:44.211223+0000","last_unstale":"2023-10-21T17:35:46.209828+0000","last_undegraded":"2023-10-21T17:35:46.209828+0000","last_fullsized":"2023-10-21T17:35:46.209828+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:03.897360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242167+0000","last_change":"2023-10-21T17:35:45.239714+0000","last_active":"2023-10-21T17:35:45.242167+0000","last_peered":"2023-10-21T17:35:45.242167+0000","last_clean":"2023-10-21T17:35:45.242167+0000","last_became_active":"2023-10-21T17:35:45.239344+0000","last_became_peered":"2023-10-21T17:35:45.239344+0000","last_unstale":"2023-10-21T17:35:45.242167+0000","last_undegraded":"2023-10-21T17:35:45.242167+0000","last_fullsized":"2023-10-21T17:35:45.242167+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:40.037869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.224760+0000","last_change":"2023-10-21T17:35:44.215205+0000","last_active":"2023-10-21T17:35:45.224760+0000","last_peered":"2023-10-21T17:35:45.224760+0000","last_clean":"2023-10-21T17:35:45.224760+0000","last_became_active":"2023-10-21T17:35:44.214654+0000","last_became_peered":"2023-10-21T17:35:44.214654+0000","last_unstale":"2023-10-21T17:35:45.224760+0000","last_undegraded":"2023-10-21T17:35:45.224760+0000","last_fullsized":"2023-10-21T17:35:45.224760+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:19.728787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209985+0000","last_change":"2023-10-21T17:35:45.228595+0000","last_active":"2023-10-21T17:35:46.209985+0000","last_peered":"2023-10-21T17:35:46.209985+0000","last_clean":"2023-10-21T17:35:46.209985+0000","last_became_active":"2023-10-21T17:35:45.228245+0000","last_became_peered":"2023-10-21T17:35:45.228245+0000","last_unstale":"2023-10-21T17:35:46.209985+0000","last_undegraded":"2023-10-21T17:35:46.209985+0000","last_fullsized":"2023-10-21T17:35:46.209985+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:12:09.144034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"214'2","reported_seq":13,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.260314+0000","last_change":"2023-10-21T17:35:44.215713+0000","last_active":"2023-10-21T17:35:45.260314+0000","last_peered":"2023-10-21T17:35:45.260314+0000","last_clean":"2023-10-21T17:35:45.260314+0000","last_became_active":"2023-10-21T17:35:44.215177+0000","last_became_peered":"2023-10-21T17:35:44.215177+0000","last_unstale":"2023-10-21T17:35:45.260314+0000","last_undegraded":"2023-10-21T17:35:45.260314+0000","last_fullsized":"2023-10-21T17:35:45.260314+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:23.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205534+0000","last_change":"2023-10-21T17:35:45.225390+0000","last_active":"2023-10-21T17:35:46.205534+0000","last_peered":"2023-10-21T17:35:46.205534+0000","last_clean":"2023-10-21T17:35:46.205534+0000","last_became_active":"2023-10-21T17:35:45.224824+0000","last_became_peered":"2023-10-21T17:35:45.224824+0000","last_unstale":"2023-10-21T17:35:46.205534+0000","last_undegraded":"2023-10-21T17:35:46.205534+0000","last_fullsized":"2023-10-21T17:35:46.205534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:09.579033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209499+0000","last_change":"2023-10-21T17:35:44.218762+0000","last_active":"2023-10-21T17:35:46.209499+0000","last_peered":"2023-10-21T17:35:46.209499+0000","last_clean":"2023-10-21T17:35:46.209499+0000","last_became_active":"2023-10-21T17:35:44.218461+0000","last_became_peered":"2023-10-21T17:35:44.218461+0000","last_unstale":"2023-10-21T17:35:46.209499+0000","last_undegraded":"2023-10-21T17:35:46.209499+0000","last_fullsized":"2023-10-21T17:35:46.209499+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:20:36.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.241985+0000","last_change":"2023-10-21T17:35:45.235709+0000","last_active":"2023-10-21T17:35:45.241985+0000","last_peered":"2023-10-21T17:35:45.241985+0000","last_clean":"2023-10-21T17:35:45.241985+0000","last_became_active":"2023-10-21T17:35:45.235404+0000","last_became_peered":"2023-10-21T17:35:45.235404+0000","last_unstale":"2023-10-21T17:35:45.241985+0000","last_undegraded":"2023-10-21T17:35:45.241985+0000","last_fullsized":"2023-10-21T17:35:45.241985+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:01:10.332865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.230662+0000","last_change":"2023-10-21T17:35:44.222528+0000","last_active":"2023-10-21T17:35:45.230662+0000","last_peered":"2023-10-21T17:35:45.230662+0000","last_clean":"2023-10-21T17:35:45.230662+0000","last_became_active":"2023-10-21T17:35:44.219207+0000","last_became_peered":"2023-10-21T17:35:44.219207+0000","last_unstale":"2023-10-21T17:35:45.230662+0000","last_undegraded":"2023-10-21T17:35:45.230662+0000","last_fullsized":"2023-10-21T17:35:45.230662+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.488604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209769+0000","last_change":"2023-10-21T17:35:45.239560+0000","last_active":"2023-10-21T17:35:46.209769+0000","last_peered":"2023-10-21T17:35:46.209769+0000","last_clean":"2023-10-21T17:35:46.209769+0000","last_became_active":"2023-10-21T17:35:45.239261+0000","last_became_peered":"2023-10-21T17:35:45.239261+0000","last_unstale":"2023-10-21T17:35:46.209769+0000","last_undegraded":"2023-10-21T17:35:46.209769+0000","last_fullsized":"2023-10-21T17:35:46.209769+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:52.865690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.234731+0000","last_change":"2023-10-21T17:35:44.222790+0000","last_active":"2023-10-21T17:35:45.234731+0000","last_peered":"2023-10-21T17:35:45.234731+0000","last_clean":"2023-10-21T17:35:45.234731+0000","last_became_active":"2023-10-21T17:35:44.220804+0000","last_became_peered":"2023-10-21T17:35:44.220804+0000","last_unstale":"2023-10-21T17:35:45.234731+0000","last_undegraded":"2023-10-21T17:35:45.234731+0000","last_fullsized":"2023-10-21T17:35:45.234731+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:45:22.752940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.237191+0000","last_change":"2023-10-21T17:35:45.232810+0000","last_active":"2023-10-21T17:35:45.237191+0000","last_peered":"2023-10-21T17:35:45.237191+0000","last_clean":"2023-10-21T17:35:45.237191+0000","last_became_active":"2023-10-21T17:35:45.232472+0000","last_became_peered":"2023-10-21T17:35:45.232472+0000","last_unstale":"2023-10-21T17:35:45.237191+0000","last_undegraded":"2023-10-21T17:35:45.237191+0000","last_fullsized":"2023-10-21T17:35:45.237191+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:11:05.203856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223307+0000","last_change":"2023-10-21T17:35:44.218100+0000","last_active":"2023-10-21T17:35:45.223307+0000","last_peered":"2023-10-21T17:35:45.223307+0000","last_clean":"2023-10-21T17:35:45.223307+0000","last_became_active":"2023-10-21T17:35:44.217086+0000","last_became_peered":"2023-10-21T17:35:44.217086+0000","last_unstale":"2023-10-21T17:35:45.223307+0000","last_undegraded":"2023-10-21T17:35:45.223307+0000","last_fullsized":"2023-10-21T17:35:45.223307+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:15.716141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209229+0000","last_change":"2023-10-21T17:35:45.230326+0000","last_active":"2023-10-21T17:35:46.209229+0000","last_peered":"2023-10-21T17:35:46.209229+0000","last_clean":"2023-10-21T17:35:46.209229+0000","last_became_active":"2023-10-21T17:35:45.229981+0000","last_became_peered":"2023-10-21T17:35:45.229981+0000","last_unstale":"2023-10-21T17:35:46.209229+0000","last_undegraded":"2023-10-21T17:35:46.209229+0000","last_fullsized":"2023-10-21T17:35:46.209229+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:56:19.123780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.223043+0000","last_change":"2023-10-21T17:35:44.226369+0000","last_active":"2023-10-21T17:35:45.223043+0000","last_peered":"2023-10-21T17:35:45.223043+0000","last_clean":"2023-10-21T17:35:45.223043+0000","last_became_active":"2023-10-21T17:35:44.226051+0000","last_became_peered":"2023-10-21T17:35:44.226051+0000","last_unstale":"2023-10-21T17:35:45.223043+0000","last_undegraded":"2023-10-21T17:35:45.223043+0000","last_fullsized":"2023-10-21T17:35:45.223043+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:57:40.699779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.205551+0000","last_change":"2023-10-21T17:35:45.238108+0000","last_active":"2023-10-21T17:35:46.205551+0000","last_peered":"2023-10-21T17:35:46.205551+0000","last_clean":"2023-10-21T17:35:46.205551+0000","last_became_active":"2023-10-21T17:35:45.237763+0000","last_became_peered":"2023-10-21T17:35:45.237763+0000","last_unstale":"2023-10-21T17:35:46.205551+0000","last_undegraded":"2023-10-21T17:35:46.205551+0000","last_fullsized":"2023-10-21T17:35:46.205551+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:52:35.653591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-21T17:35:46.209936+0000","last_change":"2023-10-21T17:35:44.211518+0000","last_active":"2023-10-21T17:35:46.209936+0000","last_peered":"2023-10-21T17:35:46.209936+0000","last_clean":"2023-10-21T17:35:46.209936+0000","last_became_active":"2023-10-21T17:35:44.210737+0000","last_became_peered":"2023-10-21T17:35:44.210737+0000","last_unstale":"2023-10-21T17:35:46.209936+0000","last_undegraded":"2023-10-21T17:35:46.209936+0000","last_fullsized":"2023-10-21T17:35:46.209936+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:15:43.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.242142+0000","last_change":"2023-10-21T17:35:45.237068+0000","last_active":"2023-10-21T17:35:45.242142+0000","last_peered":"2023-10-21T17:35:45.242142+0000","last_clean":"2023-10-21T17:35:45.242142+0000","last_became_active":"2023-10-21T17:35:45.236785+0000","last_became_peered":"2023-10-21T17:35:45.236785+0000","last_unstale":"2023-10-21T17:35:45.242142+0000","last_undegraded":"2023-10-21T17:35:45.242142+0000","last_fullsized":"2023-10-21T17:35:45.242142+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:14:33.202011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.224980+0000","last_change":"2023-10-21T17:35:44.218386+0000","last_active":"2023-10-21T17:35:45.224980+0000","last_peered":"2023-10-21T17:35:45.224980+0000","last_clean":"2023-10-21T17:35:45.224980+0000","last_became_active":"2023-10-21T17:35:44.217978+0000","last_became_peered":"2023-10-21T17:35:44.217978+0000","last_unstale":"2023-10-21T17:35:45.224980+0000","last_undegraded":"2023-10-21T17:35:45.224980+0000","last_fullsized":"2023-10-21T17:35:45.224980+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:52:34.813742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.244747+0000","last_change":"2023-10-21T17:35:45.243623+0000","last_active":"2023-10-21T17:35:45.244747+0000","last_peered":"2023-10-21T17:35:45.244747+0000","last_clean":"2023-10-21T17:35:45.244747+0000","last_became_active":"2023-10-21T17:35:45.243221+0000","last_became_peered":"2023-10-21T17:35:45.243221+0000","last_unstale":"2023-10-21T17:35:45.244747+0000","last_undegraded":"2023-10-21T17:35:45.244747+0000","last_fullsized":"2023-10-21T17:35:45.244747+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:44.168240+0000","last_clean_scrub_stamp":"2023-10-21T17:35:44.168240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:51.733192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-10-21T17:35:45.220809+0000","last_change":"2023-10-21T17:35:44.213390+0000","last_active":"2023-10-21T17:35:45.220809+0000","last_peered":"2023-10-21T17:35:45.220809+0000","last_clean":"2023-10-21T17:35:45.220809+0000","last_became_active":"2023-10-21T17:35:44.213075+0000","last_became_peered":"2023-10-21T17:35:44.213075+0000","last_unstale":"2023-10-21T17:35:45.220809+0000","last_undegraded":"2023-10-21T17:35:45.220809+0000","last_fullsized":"2023-10-21T17:35:45.220809+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":212,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:35:43.151339+0000","last_clean_scrub_stamp":"2023-10-21T17:35:43.151339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:57:16.383179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"202'4851","reported_seq":5705,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-10-21T17:35:44.212614+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:35:44.212614+0000","last_peered":"2023-10-21T17:35:44.212614+0000","last_clean":"2023-10-21T17:35:44.212614+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:35:44.212614+0000","last_undegraded":"2023-10-21T17:35:44.212614+0000","last_fullsized":"2023-10-21T17:35:44.212614+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4851,"log_dups_size":0,"ondisk_log_size":4851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":547,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2032,"num_read_kb":4227,"num_write":547,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4851,"ondisk_log_size":4851,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52436,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319404,"statfs":{"total":96636764160,"available":96583069696,"internally_reserved":0,"allocated":3276800,"data_stored":1974984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.214,"15min":1.202},"min":{"1min":0.76700000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.841,"5min":1.841,"15min":2.2549999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.159,"15min":1.181},"min":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.65600000000000003},"max":{"1min":1.619,"5min":1.6839999999999999,"15min":1.7929999999999999},"last":1.2669999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.21,"15min":1.2230000000000001},"min":{"1min":0.85799999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.649,"5min":1.6779999999999999,"15min":1.73},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.3180000000000001,"15min":1.319},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.78700000000000003},"max":{"1min":1.734,"5min":1.885,"15min":1.885},"last":1.7290000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.335},"min":{"1min":0.89500000000000002,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.7609999999999999,"5min":1.9339999999999999,"15min":1.946},"last":1.599},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.327,"15min":1.3180000000000001},"min":{"1min":1.026,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.962},"last":0.95099999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3120000000000001,"15min":1.3220000000000001},"min":{"1min":1.028,"5min":0.96799999999999997,"15min":0.876},"max":{"1min":1.6479999999999999,"5min":1.6899999999999999,"15min":2.5110000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4730000000000001,"15min":1.4550000000000001},"min":{"1min":1.2110000000000001,"5min":1.0409999999999999,"15min":0.91000000000000003},"max":{"1min":1.73,"5min":1.827,"15min":1.8999999999999999},"last":1.206}]},{"osd":4,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.359,"15min":1.353},"min":{"1min":0.90300000000000002,"5min":0.77400000000000002,"15min":0.76000000000000001},"max":{"1min":1.7110000000000001,"5min":2.2850000000000001,"15min":2.2850000000000001},"last":1.452},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.47,"15min":1.496},"min":{"1min":1.377,"5min":0.91600000000000004,"15min":0.79400000000000004},"max":{"1min":1.827,"5min":1.827,"15min":2.1549999999999998},"last":1.7}]},{"osd":5,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3720000000000001,"15min":1.3979999999999999},"min":{"1min":1.022,"5min":0.872,"15min":0.73099999999999998},"max":{"1min":1.8700000000000001,"5min":2.1240000000000001,"15min":2.2450000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4410000000000001,"15min":1.4470000000000001},"min":{"1min":1.298,"5min":0.96199999999999997,"15min":0.751},"max":{"1min":1.8100000000000001,"5min":2.3540000000000001,"15min":2.3639999999999999},"last":1.5529999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:34:55 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5269999999999999,"15min":1.5309999999999999},"min":{"1min":1.1299999999999999,"5min":1.0640000000000001,"15min":0.96799999999999997},"max":{"1min":1.9079999999999999,"5min":2.2040000000000002,"15min":2.2269999999999999},"last":1.667},{"interface":"front","average":{"1min":1.518,"5min":1.468,"15min":1.4730000000000001},"min":{"1min":1.2470000000000001,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":1.946,"5min":1.946,"15min":2.3050000000000002},"last":1.645}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42132,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329708,"statfs":{"total":96636764160,"available":96593620992,"internally_reserved":0,"allocated":2686976,"data_stored":1384686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.2130000000000001,"15min":1.2010000000000001},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.73899999999999999},"max":{"1min":1.548,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.141,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.81599999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6839999999999999,"5min":1.762,"15min":1.762},"last":1.0840000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2030000000000001,"15min":1.2010000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.73799999999999999},"max":{"1min":1.5600000000000001,"5min":1.579,"15min":1.7050000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.319,"15min":1.3220000000000001},"min":{"1min":0.93200000000000005,"5min":0.89800000000000002,"15min":0.82299999999999995},"max":{"1min":1.637,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.21}]},{"osd":2,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.321},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.76300000000000001},"max":{"1min":1.579,"5min":1.825,"15min":1.8300000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3129999999999999,"15min":1.3120000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.79500000000000004},"max":{"1min":1.679,"5min":1.6819999999999999,"15min":1.929},"last":1.1719999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4330000000000001,"15min":1.421},"min":{"1min":1.1659999999999999,"5min":1.044,"15min":0.70599999999999996},"max":{"1min":1.6870000000000001,"5min":1.9419999999999999,"15min":1.9570000000000001},"last":1.373},{"interface":"front","average":{"1min":1.375,"5min":1.4379999999999999,"15min":1.4610000000000001},"min":{"1min":1.0509999999999999,"5min":0.90500000000000003,"15min":0.83299999999999996},"max":{"1min":1.655,"5min":2.0209999999999999,"15min":2.048},"last":1.423}]},{"osd":4,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3129999999999999,"15min":1.365},"min":{"1min":0.88700000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.762,"5min":1.9930000000000001,"15min":2.028},"last":1.488},{"interface":"front","average":{"1min":1.363,"5min":1.401,"15min":1.4470000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6950000000000001,"5min":1.927,"15min":2.2149999999999999},"last":1.573}]},{"osd":5,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5249999999999999,"15min":1.5669999999999999},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":0.997},"max":{"1min":1.794,"5min":2.1739999999999999,"15min":2.339},"last":1.849},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3460000000000001,"15min":1.397},"min":{"1min":0.90200000000000002,"5min":0.875,"15min":0.78800000000000003},"max":{"1min":1.581,"5min":1.825,"15min":2.2890000000000001},"last":1.643}]},{"osd":7,"last update":"Sat Oct 21 17:34:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.5289999999999999,"15min":1.5620000000000001},"min":{"1min":1.1160000000000001,"5min":1.093,"15min":1.0589999999999999},"max":{"1min":1.768,"5min":2.0449999999999999,"15min":2.145},"last":1.617},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.6319999999999999,"15min":1.6739999999999999},"min":{"1min":1.266,"5min":1.2,"15min":1.0449999999999999},"max":{"1min":1.8560000000000001,"5min":2.2610000000000001,"15min":2.5409999999999999},"last":1.681}]}]},{"osd":5,"up_from":13,"seq":55834575967,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60192,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311648,"statfs":{"total":96636764160,"available":96575127552,"internally_reserved":0,"allocated":2699264,"data_stored":1386250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1399999999999999,"15min":1.1599999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.512,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.498},{"interface":"front","average":{"1min":1.345,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.59999999999999998},"max":{"1min":1.6950000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.256,"15min":1.2809999999999999},"min":{"1min":0.876,"5min":0.85099999999999998,"15min":0.73799999999999999},"max":{"1min":1.623,"5min":1.806,"15min":1.8680000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.258,"15min":1.248},"min":{"1min":0.96999999999999997,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":1.073}]},{"osd":2,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2729999999999999,"15min":1.3149999999999999},"min":{"1min":0.96599999999999997,"5min":0.93000000000000005,"15min":0.91400000000000003},"max":{"1min":1.4910000000000001,"5min":1.6699999999999999,"15min":2.032},"last":1.2},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2270000000000001,"15min":1.262},"min":{"1min":0.96799999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.891},"last":1.125}]},{"osd":3,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.367,"15min":1.369},"min":{"1min":1.004,"5min":0.89800000000000002,"15min":0.84699999999999998},"max":{"1min":1.758,"5min":1.829,"15min":2.4180000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.421,"15min":1.4299999999999999},"min":{"1min":1.083,"5min":1.028,"15min":0.90400000000000003},"max":{"1min":1.8009999999999999,"5min":1.881,"15min":2.1539999999999999},"last":1.3700000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.214,"15min":1.2609999999999999},"min":{"1min":0.88800000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.526,"5min":1.8520000000000001,"15min":1.9910000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":1.0920000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7290000000000001,"5min":1.877,"15min":2.306},"last":1.5760000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4630000000000001,"15min":1.49},"min":{"1min":1.1970000000000001,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8939999999999999,"5min":1.9319999999999999,"15min":2.6520000000000001},"last":1.306},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5249999999999999,"15min":1.538},"min":{"1min":1.3380000000000001,"5min":0.89800000000000002,"15min":0.875},"max":{"1min":1.7350000000000001,"5min":2.2269999999999999,"15min":2.52},"last":1.651}]},{"osd":7,"last update":"Sat Oct 21 17:34:53 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5569999999999999,"15min":1.5580000000000001},"min":{"1min":1.1579999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.9319999999999999,"5min":2.2530000000000001,"15min":2.5209999999999999},"last":1.782},{"interface":"front","average":{"1min":1.635,"5min":1.5760000000000001,"15min":1.589},"min":{"1min":1.268,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9510000000000001,"5min":2.1299999999999999,"15min":2.7759999999999998},"last":1.468}]}]},{"osd":4,"up_from":13,"seq":55834575965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47716,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324124,"statfs":{"total":96636764160,"available":96587902976,"internally_reserved":0,"allocated":2711552,"data_stored":1392998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.131,"15min":1.179},"min":{"1min":0.79600000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.569,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2490000000000001,"15min":1.2210000000000001},"min":{"1min":0.90400000000000003,"5min":0.76600000000000001,"15min":0.745},"max":{"1min":1.613,"5min":1.869,"15min":1.869},"last":0.98399999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.252,"15min":1.2529999999999999},"min":{"1min":0.997,"5min":0.86299999999999999,"15min":0.58099999999999996},"max":{"1min":1.7090000000000001,"5min":1.806,"15min":1.806},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2589999999999999,"15min":1.2629999999999999},"min":{"1min":0.98499999999999999,"5min":0.85099999999999998,"15min":0.78700000000000003},"max":{"1min":1.5429999999999999,"5min":1.7090000000000001,"15min":1.76},"last":1.2629999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.262,"15min":1.26},"min":{"1min":0.98999999999999999,"5min":0.88400000000000001,"15min":0.76300000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.7609999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.371,"15min":1.389},"min":{"1min":1.0780000000000001,"5min":1.026,"15min":0.95099999999999996},"max":{"1min":1.6240000000000001,"5min":1.762,"15min":1.762},"last":1.1180000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.304,"15min":1.3340000000000001},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.841},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"min":{"1min":1.089,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":1.825,"5min":1.8260000000000001,"15min":1.9239999999999999},"last":1.323}]},{"osd":5,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.294,"15min":1.3300000000000001},"min":{"1min":1.022,"5min":0.82699999999999996,"15min":0.80800000000000005},"max":{"1min":1.623,"5min":1.867,"15min":1.9410000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.325,"15min":1.357},"min":{"1min":0.879,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.9810000000000001},"last":1.6739999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.448},"min":{"1min":0.95399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8979999999999999,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.581,"15min":1.5880000000000001},"min":{"1min":1.379,"5min":1.145,"15min":1.133},"max":{"1min":1.806,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.71}]},{"osd":7,"last update":"Sat Oct 21 17:34:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.476,"15min":1.5049999999999999},"min":{"1min":1.0529999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.964,"5min":2.234,"15min":2.234},"last":1.929},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5549999999999999,"15min":1.5800000000000001},"min":{"1min":1.242,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":1.9470000000000001,"5min":1.952,"15min":2.081},"last":1.637}]}]},{"osd":3,"up_from":13,"seq":55834575965,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56016,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315824,"statfs":{"total":96636764160,"available":96579403776,"internally_reserved":0,"allocated":2691072,"data_stored":1384776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.97299999999999998,"15min":0.98599999999999999},"min":{"1min":0.68899999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.4379999999999999,"5min":1.601,"15min":1.71},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.266,"15min":1.266},"min":{"1min":1.048,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0960000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.079,"15min":1.0980000000000001},"min":{"1min":0.749,"5min":0.72699999999999998,"15min":0.45600000000000002},"max":{"1min":1.583,"5min":1.7769999999999999,"15min":1.8080000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3460000000000001,"15min":1.3480000000000001},"min":{"1min":0.93600000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.637,"5min":1.8120000000000001,"15min":1.9419999999999999},"last":1.26}]},{"osd":2,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.92300000000000004,"5min":0.60999999999999999,"15min":0.60899999999999999},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.381,"5min":1.3069999999999999,"15min":1.3029999999999999},"min":{"1min":0.92900000000000005,"5min":0.91600000000000004,"15min":0.55300000000000005},"max":{"1min":1.853,"5min":1.853,"15min":2},"last":1.125}]},{"osd":4,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4259999999999999,"15min":1.444},"min":{"1min":1.1479999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6419999999999999,"5min":1.8280000000000001,"15min":1.9690000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.484,"15min":1.4870000000000001},"min":{"1min":1.044,"5min":1.044,"15min":0.88700000000000001},"max":{"1min":1.9430000000000001,"5min":2.169,"15min":2.169},"last":1.419}]},{"osd":5,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5089999999999999,"15min":1.516},"min":{"1min":1.1419999999999999,"5min":0.93500000000000005,"15min":0.90900000000000003},"max":{"1min":1.9630000000000001,"5min":2.0089999999999999,"15min":2.5329999999999999},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5509999999999999,"15min":1.579},"min":{"1min":1.292,"5min":0.95799999999999996,"15min":0.95799999999999996},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.2069999999999999},"last":1.1850000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.589,"15min":1.6100000000000001},"min":{"1min":1.179,"5min":1.179,"15min":0.93899999999999995},"max":{"1min":2.4990000000000001,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.603},{"interface":"front","average":{"1min":1.641,"5min":1.621,"15min":1.633},"min":{"1min":1.1719999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":2.0699999999999998,"5min":2.089,"15min":2.6840000000000002},"last":1.464}]},{"osd":7,"last update":"Sat Oct 21 17:34:47 2023","interfaces":[{"interface":"back","average":{"1min":1.722,"5min":1.6819999999999999,"15min":1.6739999999999999},"min":{"1min":1.5029999999999999,"5min":1.026,"15min":1.026},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3460000000000001},"last":1.373},{"interface":"front","average":{"1min":1.786,"5min":1.712,"15min":1.7210000000000001},"min":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.141},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.51}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51856,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319984,"statfs":{"total":96636764160,"available":96583663616,"internally_reserved":0,"allocated":2686976,"data_stored":1385198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92700000000000005,"15min":0.89100000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.8029999999999999,"5min":1.841,"15min":1.841},"last":1.109},{"interface":"front","average":{"1min":1.087,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.75,"5min":0.70599999999999996,"15min":0.66800000000000004},"max":{"1min":1.907,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.90000000000000002}]},{"osd":1,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.194,"15min":1.198},"min":{"1min":0.92600000000000005,"5min":0.79100000000000004,"15min":0.76400000000000001},"max":{"1min":1.794,"5min":1.806,"15min":1.819},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.395,"15min":1.3979999999999999},"min":{"1min":1.103,"5min":1.036,"15min":0.79500000000000004},"max":{"1min":1.958,"5min":1.958,"15min":2.4060000000000001},"last":1.419}]},{"osd":3,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.216,"15min":1.1859999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.60199999999999998},"max":{"1min":1.575,"5min":1.8149999999999999,"15min":2.1760000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.492,"5min":1.506,"15min":1.4910000000000001},"min":{"1min":1.2310000000000001,"5min":1.155,"15min":1.1200000000000001},"max":{"1min":1.7290000000000001,"5min":2.077,"15min":2.077},"last":1}]},{"osd":4,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4239999999999999,"15min":1.413},"min":{"1min":1.1140000000000001,"5min":1.044,"15min":1.024},"max":{"1min":1.8660000000000001,"5min":2.1680000000000001,"15min":2.218},"last":1.181},{"interface":"front","average":{"1min":1.522,"5min":1.4670000000000001,"15min":1.454},"min":{"1min":1.133,"5min":1.069,"15min":1.0329999999999999},"max":{"1min":1.9610000000000001,"5min":2.1469999999999998,"15min":2.5939999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.45,"15min":1.407},"min":{"1min":1.125,"5min":1.0469999999999999,"15min":0.94699999999999995},"max":{"1min":1.819,"5min":1.9239999999999999,"15min":2.1560000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.631,"15min":1.605},"min":{"1min":1.355,"5min":1.2470000000000001,"15min":1.2150000000000001},"max":{"1min":1.972,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.663}]},{"osd":6,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5920000000000001,"15min":1.573},"min":{"1min":1.1970000000000001,"5min":1.129,"15min":1.042},"max":{"1min":2.0830000000000002,"5min":2.286,"15min":2.4460000000000002},"last":1.746},{"interface":"front","average":{"1min":1.571,"5min":1.5309999999999999,"15min":1.514},"min":{"1min":1.179,"5min":1.1579999999999999,"15min":0.80600000000000005},"max":{"1min":1.927,"5min":2.0470000000000002,"15min":2.1030000000000002},"last":1.786}]},{"osd":7,"last update":"Sat Oct 21 17:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.643,"15min":1.6399999999999999},"min":{"1min":1.232,"5min":1.1819999999999999,"15min":1.0780000000000001},"max":{"1min":2.1520000000000001,"5min":2.351,"15min":2.4630000000000001},"last":1.841},{"interface":"front","average":{"1min":1.6759999999999999,"5min":1.605,"15min":1.595},"min":{"1min":1.3740000000000001,"5min":1.147,"15min":0.77500000000000002},"max":{"1min":2.1259999999999999,"5min":2.1669999999999998,"15min":2.4630000000000001},"last":1.7050000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51848,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319992,"statfs":{"total":96636764160,"available":96583671808,"internally_reserved":0,"allocated":2682880,"data_stored":1384616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.92000000000000004,"15min":0.95899999999999996},"min":{"1min":0.69199999999999995,"5min":0.68799999999999994,"15min":0.65900000000000003},"max":{"1min":1.3480000000000001,"5min":1.637,"15min":1.976},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.1950000000000001,"15min":1.1639999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.46700000000000003},"max":{"1min":1.8089999999999999,"5min":1.8400000000000001,"15min":2.0510000000000002},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0509999999999999,"15min":1.054},"min":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.56799999999999995},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.79},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.262,"5min":1.244,"15min":1.2629999999999999},"min":{"1min":0.89300000000000002,"5min":0.88400000000000001,"15min":0.66700000000000004},"max":{"1min":1.679,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3740000000000001,"15min":1.409},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.83299999999999996},"max":{"1min":1.704,"5min":1.875,"15min":2.129},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.23,"5min":1.2430000000000001,"15min":1.3049999999999999},"min":{"1min":0.88700000000000001,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.748,"5min":1.798,"15min":2.456},"last":0.92500000000000004}]},{"osd":4,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.393,"15min":1.373},"min":{"1min":1.02,"5min":1.0029999999999999,"15min":0.91500000000000004},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0950000000000002},"last":1.04},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.51,"15min":1.5109999999999999},"min":{"1min":1.179,"5min":1.1779999999999999,"15min":1.0629999999999999},"max":{"1min":1.8380000000000001,"5min":2.0209999999999999,"15min":2.383},"last":1.226}]},{"osd":5,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.589,"15min":1.581},"min":{"1min":1.1459999999999999,"5min":1.018,"15min":1.018},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.5169999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.5289999999999999,"15min":1.5589999999999999},"min":{"1min":1.008,"5min":1.008,"15min":0.96999999999999997},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.335},"last":1.167}]},{"osd":6,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5980000000000001,"15min":1.6080000000000001},"min":{"1min":1.2410000000000001,"5min":1.1930000000000001,"15min":0.85699999999999998},"max":{"1min":2.27,"5min":2.27,"15min":2.3940000000000001},"last":1.016},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5589999999999999,"15min":1.571},"min":{"1min":1.127,"5min":1.127,"15min":0.98499999999999999},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.3519999999999999},"last":0.97199999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.6559999999999999,"15min":1.7010000000000001},"min":{"1min":1.341,"5min":1.294,"15min":1.2},"max":{"1min":1.758,"5min":2.113,"15min":2.548},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.7070000000000001,"15min":1.7270000000000001},"min":{"1min":1.4970000000000001,"5min":1.149,"15min":0.90200000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.548},"last":1.399}]}]},{"osd":0,"up_from":13,"seq":55834575965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48288,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323552,"statfs":{"total":96636764160,"available":96587317248,"internally_reserved":0,"allocated":3293184,"data_stored":1975712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.78200000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.409,"5min":1.774,"15min":1.774},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.133,"5min":1.1459999999999999,"15min":1.1259999999999999},"min":{"1min":0.86799999999999999,"5min":0.71899999999999997,"15min":0.68200000000000005},"max":{"1min":1.407,"5min":1.748,"15min":1.774},"last":0.80500000000000005}]},{"osd":2,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1870000000000001,"15min":1.2},"min":{"1min":0.84099999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.484,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2390000000000001,"15min":1.2130000000000001},"min":{"1min":0.876,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.6459999999999999,"5min":1.851,"15min":1.851},"last":1.137}]},{"osd":3,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.085,"15min":1.1599999999999999},"min":{"1min":0.78200000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5960000000000001,"5min":1.643,"15min":1.9590000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.403,"15min":1.4119999999999999},"min":{"1min":1.0389999999999999,"5min":1.01,"15min":0.95999999999999996},"max":{"1min":1.915,"5min":2.0270000000000001,"15min":2.0329999999999999},"last":1.4610000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4219999999999999,"15min":1.4570000000000001},"min":{"1min":1.028,"5min":0.95799999999999996,"15min":0.88600000000000001},"max":{"1min":1.8220000000000001,"5min":2.032,"15min":2.0910000000000002},"last":1.177},{"interface":"front","average":{"1min":1.603,"5min":1.605,"15min":1.6220000000000001},"min":{"1min":1.284,"5min":1.163,"15min":1.123},"max":{"1min":1.9610000000000001,"5min":2.1360000000000001,"15min":2.2770000000000001},"last":1.629}]},{"osd":5,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.407,"15min":1.4259999999999999},"min":{"1min":1.157,"5min":1.0089999999999999,"15min":0.95799999999999996},"max":{"1min":1.6970000000000001,"5min":1.8080000000000001,"15min":2.101},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.5700000000000001,"15min":1.6000000000000001},"min":{"1min":1.3029999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.9039999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.7030000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.494,"15min":1.5109999999999999},"min":{"1min":1.2749999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"max":{"1min":1.9330000000000001,"5min":2.0139999999999998,"15min":2.0750000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.6799999999999999,"15min":1.7050000000000001},"min":{"1min":1.258,"5min":1.258,"15min":1.1299999999999999},"max":{"1min":2.0760000000000001,"5min":2.222,"15min":2.4940000000000002},"last":1.758}]},{"osd":7,"last update":"Sat Oct 21 17:35:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.573,"15min":1.5680000000000001},"min":{"1min":1.206,"5min":1.0649999999999999,"15min":1.034},"max":{"1min":2.1619999999999999,"5min":2.3100000000000001,"15min":2.3100000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.665,"5min":1.6240000000000001,"15min":1.647},"min":{"1min":1.367,"5min":1.145,"15min":1.145},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.262},"last":1.341}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:35:50.270 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:35:50.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:35:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.543+0000 7fceaabbf640 1 Processor -- start 2023-10-21T17:35:50.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.544+0000 7fceaabbf640 1 -- start start 2023-10-21T17:35:50.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.545+0000 7fceaabbf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.545+0000 7fceaabbf640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcea40cb260 0x7fcea40cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.545+0000 7fceaabbf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40c9c90 con 0x7fcea415e490 2023-10-21T17:35:50.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.545+0000 7fceaabbf640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea415f480 con 0x7fcea40cb260 2023-10-21T17:35:50.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.545+0000 7fceaabbf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40cbc30 con 0x7fcea415de70 2023-10-21T17:35:50.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.546+0000 7fcea9bbd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.546+0000 7fcea9bbd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:60582/0 (socket says 172.21.15.53:60582) 2023-10-21T17:35:50.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.546+0000 7fceaa3be640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fcea415e490 0x7fcea415e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:34926/0 (socket says 172.21.15.53:34926) 2023-10-21T17:35:50.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.546+0000 7fcea9bbd640 1 -- 172.21.15.53:0/3097248004 learned_addr learned my addr 172.21.15.53:0/3097248004 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:50.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.546+0000 7fcea93bc640 1 --2- 172.21.15.53:0/3097248004 >> v2:172.21.15.124:3300/0 conn(0x7fcea40cb260 0x7fcea40cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.547+0000 7fcea9bbd640 1 -- 172.21.15.53:0/3097248004 >> v2:172.21.15.124:3300/0 conn(0x7fcea40cb260 msgr2=0x7fcea40cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:50.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.547+0000 7fcea9bbd640 1 --2- 172.21.15.53:0/3097248004 >> v2:172.21.15.124:3300/0 conn(0x7fcea40cb260 0x7fcea40cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.547+0000 7fcea9bbd640 1 -- 172.21.15.53:0/3097248004 >> v1:172.21.15.124:6789/0 conn(0x7fcea415e490 legacy=0x7fcea415e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.547+0000 7fcea9bbd640 1 -- 172.21.15.53:0/3097248004 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea415e250 con 0x7fcea415de70 2023-10-21T17:35:50.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.548+0000 7fcea9bbd640 1 --2- 172.21.15.53:0/3097248004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009870 tx=0x7fcea002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b86c9456df62312 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.549+0000 7fcea8bbb640 1 -- 172.21.15.53:0/3097248004 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0030840 con 0x7fcea415de70 2023-10-21T17:35:50.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.549+0000 7fcea8bbb640 1 -- 172.21.15.53:0/3097248004 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea00309a0 con 0x7fcea415de70 2023-10-21T17:35:50.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.549+0000 7fcea8bbb640 1 -- 172.21.15.53:0/3097248004 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea0030d30 con 0x7fcea415de70 2023-10-21T17:35:50.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 -- 172.21.15.53:0/3097248004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 msgr2=0x7fcea40ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 --2- 172.21.15.53:0/3097248004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009870 tx=0x7fcea002b980 comp rx=0 tx=0).stop 2023-10-21T17:35:50.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 -- 172.21.15.53:0/3097248004 shutdown_connections 2023-10-21T17:35:50.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 --2- 172.21.15.53:0/3097248004 >> v2:172.21.15.124:3300/0 conn(0x7fcea40cb260 0x7fcea40cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 --2- 172.21.15.53:0/3097248004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea415de70 0x7fcea40ca9a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 -- 172.21.15.53:0/3097248004 >> 172.21.15.53:0/3097248004 conn(0x7fcea40ce9f0 msgr2=0x7fcea40cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:50.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 -- 172.21.15.53:0/3097248004 shutdown_connections 2023-10-21T17:35:50.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.550+0000 7fceaabbf640 1 -- 172.21.15.53:0/3097248004 wait complete. 2023-10-21T17:35:50.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.552+0000 7fceaabbf640 1 Processor -- start 2023-10-21T17:35:50.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.553+0000 7fceaabbf640 1 -- start start 2023-10-21T17:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea40cb260 0x7fcea415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 0x7fcea40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41f2220 con 0x7fcea415de70 2023-10-21T17:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41583a0 con 0x7fcea415e490 2023-10-21T17:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fceaabbf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4158680 con 0x7fcea40cb260 2023-10-21T17:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fcea93bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fcea93bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51874/0 (socket says 172.21.15.53:51874) 2023-10-21T17:35:50.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.554+0000 7fcea93bc640 1 -- 172.21.15.53:0/1960823729 learned_addr learned my addr 172.21.15.53:0/1960823729 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:50.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea9bbd640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea40cb260 0x7fcea415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fceaa3be640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 0x7fcea40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 -- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea40cb260 msgr2=0x7fcea415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea40cb260 0x7fcea415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 -- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 msgr2=0x7fcea40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 0x7fcea40cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 -- 172.21.15.53:0/1960823729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00094d0 con 0x7fcea415de70 2023-10-21T17:35:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fceaa3be640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 0x7fcea40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:50.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.555+0000 7fcea93bc640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9b0 tx=0x7fce9400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.556+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94009670 con 0x7fcea415de70 2023-10-21T17:35:50.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.556+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400c9a0 con 0x7fcea415de70 2023-10-21T17:35:50.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.556+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400cd50 con 0x7fcea415de70 2023-10-21T17:35:50.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.556+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea4158960 con 0x7fcea415de70 2023-10-21T17:35:50.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.556+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea4158f20 con 0x7fcea415de70 2023-10-21T17:35:50.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.557+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea404e160 con 0x7fcea415de70 2023-10-21T17:35:50.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.559+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce94013470 con 0x7fcea415de70 2023-10-21T17:35:50.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.560+0000 7fce9a7fc640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 0x7fce8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:50.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.560+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fce94089c00 con 0x7fcea415de70 2023-10-21T17:35:50.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.560+0000 7fcea9bbd640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 0x7fce8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:50.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.562+0000 7fcea9bbd640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 0x7fce8806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcea0036000 tx=0x7fcea0008030 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:50.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.564+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce94054940 con 0x7fcea415de70 2023-10-21T17:35:50.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.826+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcea4157bf0 con 0x7fcea415de70 2023-10-21T17:35:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.831+0000 7fce9a7fc640 1 -- 172.21.15.53:0/1960823729 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+147 (secure 0 0 0) 0x7fce940210b0 con 0x7fcea415de70 2023-10-21T17:35:50.832 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:35:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:35:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:35:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:35:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:35:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:35:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 msgr2=0x7fce8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 0x7fce8806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcea0036000 tx=0x7fcea0008030 comp rx=0 tx=0).stop 2023-10-21T17:35:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 msgr2=0x7fcea4099ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fce9400e9b0 tx=0x7fce9400ee70 comp rx=0 tx=0).stop 2023-10-21T17:35:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 shutdown_connections 2023-10-21T17:35:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fce8806af90 0x7fce8806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcea415e490 0x7fcea40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcea415de70 0x7fcea4099ca0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 --2- 172.21.15.53:0/1960823729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcea40cb260 0x7fcea415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.833+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 >> 172.21.15.53:0/1960823729 conn(0x7fcea40ce9f0 msgr2=0x7fcea40cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:50.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.834+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 shutdown_connections 2023-10-21T17:35:50.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:50.834+0000 7fceaabbf640 1 -- 172.21.15.53:0/1960823729 wait complete. 2023-10-21T17:35:50.852 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:35:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.071+0000 7fc2ed8df640 1 Processor -- start 2023-10-21T17:35:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.071+0000 7fc2ed8df640 1 -- start start 2023-10-21T17:35:51.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ed8df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2e8165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ed8df640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e815f7d0 con 0x7fc2e815ab30 2023-10-21T17:35:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ed8df640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e815efb0 con 0x7fc2e80adc70 2023-10-21T17:35:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ed8df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e815f210 con 0x7fc2e80b07d0 2023-10-21T17:35:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ec8dd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2e815ab30 0x7fc2e815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43528/0 (socket says 172.21.15.53:43528) 2023-10-21T17:35:51.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.073+0000 7fc2ec8dd640 1 -- 172.21.15.53:0/480384326 learned_addr learned my addr 172.21.15.53:0/480384326 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:51.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2e8165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744422650 0 0) 0x7fc2e815f7d0 con 0x7fc2e815ab30 2023-10-21T17:35:51.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d8003520 con 0x7fc2e815ab30 2023-10-21T17:35:51.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3698983596 0 0) 0x7fc2d8003520 con 0x7fc2e815ab30 2023-10-21T17:35:51.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 msgr2=0x7fc2e8165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:51.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2e8165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 >> v1:172.21.15.124:6789/0 conn(0x7fc2e80adc70 legacy=0x7fc2e80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e815f4f0 con 0x7fc2e815ab30 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc2dc001670 con 0x7fc2e815ab30 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 >> v1:172.21.15.53:6789/0 conn(0x7fc2e815ab30 legacy=0x7fc2e815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.074+0000 7fc2e77fe640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2d8003ee0 0x7fc2d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e77fe640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e815f210 con 0x7fc2d8004d00 2023-10-21T17:35:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d800b9e0 con 0x7fc2d8003ee0 2023-10-21T17:35:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d800bba0 con 0x7fc2e80b07d0 2023-10-21T17:35:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ec8dd640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e7fff640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2d8003ee0 0x7fc2d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 -- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 msgr2=0x7fc2d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80b07d0 0x7fc2d80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 -- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2d8003ee0 msgr2=0x7fc2d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:51.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2d8003ee0 0x7fc2d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2ed0de640 1 -- 172.21.15.53:0/480384326 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d8003520 con 0x7fc2d8004d00 2023-10-21T17:35:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.075+0000 7fc2e7fff640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2d8003ee0 0x7fc2d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:51.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.076+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2d8005180 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009f90 tx=0x7fc2d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71629f8a3e60a81 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:51.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e002cbd0 con 0x7fc2d8004d00 2023-10-21T17:35:51.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e000f040 con 0x7fc2d8004d00 2023-10-21T17:35:51.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2e77fe640 1 -- 172.21.15.53:0/480384326 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0034440 con 0x7fc2d8004d00 2023-10-21T17:35:51.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 -- 172.21.15.53:0/480384326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 msgr2=0x7fc2d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/480384326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2d8005180 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009f90 tx=0x7fc2d80057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed0de640 1 -- 172.21.15.53:0/480384326 reap_dead start 2023-10-21T17:35:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 -- 172.21.15.53:0/480384326 shutdown_connections 2023-10-21T17:35:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 -- 172.21.15.53:0/480384326 >> 172.21.15.53:0/480384326 conn(0x7fc2e80cb0c0 msgr2=0x7fc2e80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 -- 172.21.15.53:0/480384326 shutdown_connections 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.077+0000 7fc2ed8df640 1 -- 172.21.15.53:0/480384326 wait complete. 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.079+0000 7fc2ed8df640 1 Processor -- start 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.079+0000 7fc2ed8df640 1 -- start start 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2e80ae4d0 0x7fc2e80cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80ae950 0x7fc2e80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80cd2e0 con 0x7fc2d8004d00 2023-10-21T17:35:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81f3550 con 0x7fc2e80ae4d0 2023-10-21T17:35:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ed8df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80ac2f0 con 0x7fc2e80ae950 2023-10-21T17:35:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.080+0000 7fc2ec8dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51884/0 (socket says 172.21.15.53:51884) 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 -- 172.21.15.53:0/4012832311 learned_addr learned my addr 172.21.15.53:0/4012832311 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ed0de640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80ae950 0x7fc2e80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 -- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80ae950 msgr2=0x7fc2e80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80ae950 0x7fc2e80cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 -- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2e80ae4d0 msgr2=0x7fc2e80cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2e80ae4d0 0x7fc2e80cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 -- 172.21.15.53:0/4012832311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80cc660 con 0x7fc2d8004d00 2023-10-21T17:35:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2ec8dd640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e8c0 tx=0x7fc2dc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.081+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00c8d0 con 0x7fc2d8004d00 2023-10-21T17:35:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.082+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc00ca30 con 0x7fc2d8004d00 2023-10-21T17:35:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.082+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e80cc940 con 0x7fc2d8004d00 2023-10-21T17:35:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.082+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00cdc0 con 0x7fc2d8004d00 2023-10-21T17:35:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.082+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e80ccf00 con 0x7fc2d8004d00 2023-10-21T17:35:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.083+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e804e160 con 0x7fc2d8004d00 2023-10-21T17:35:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.084+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc2dc01d460 con 0x7fc2d8004d00 2023-10-21T17:35:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.085+0000 7fc2e57fa640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 0x7fc2cc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.085+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc2dc0899e0 con 0x7fc2d8004d00 2023-10-21T17:35:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.085+0000 7fc2e7fff640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 0x7fc2cc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.086+0000 7fc2e7fff640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 0x7fc2cc06d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000f60 tx=0x7fc2d4009040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:51.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.089+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2dc0547b0 con 0x7fc2d8004d00 2023-10-21T17:35:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.320+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e80ada50 con 0x7fc2d8004d00 2023-10-21T17:35:51.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.321+0000 7fc2e57fa640 1 -- 172.21.15.53:0/4012832311 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7fc2dc057dd0 con 0x7fc2d8004d00 2023-10-21T17:35:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:35:51.323 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:35:51.323 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:35:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 msgr2=0x7fc2cc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 0x7fc2cc06d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4000f60 tx=0x7fc2d4009040 comp rx=0 tx=0).stop 2023-10-21T17:35:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 msgr2=0x7fc2e81f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e8c0 tx=0x7fc2dc00ed80 comp rx=0 tx=0).stop 2023-10-21T17:35:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 shutdown_connections 2023-10-21T17:35:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fc2cc06afc0 0x7fc2cc06d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2e80ae950 0x7fc2e80cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2e80ae4d0 0x7fc2e80cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 --2- 172.21.15.53:0/4012832311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2d8004d00 0x7fc2e81f3010 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 >> 172.21.15.53:0/4012832311 conn(0x7fc2e80cb0c0 msgr2=0x7fc2e80af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 shutdown_connections 2023-10-21T17:35:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.324+0000 7fc2ed8df640 1 -- 172.21.15.53:0/4012832311 wait complete. 2023-10-21T17:35:51.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:35:44.664802+0000', 'modified': '2023-10-21T17:35:47.776951+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10677}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10677': {'gid': 10677, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.53:6835/542756474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 542756474}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 542756474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:35:51.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:35:51.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-21T17:35:51.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.983+0000 7faba3834640 1 Processor -- start 2023-10-21T17:35:51.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.983+0000 7faba3834640 1 -- start start 2023-10-21T17:35:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba3834640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab9c070cc0 0x7fab9c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba3834640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 0x7fab9c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba3834640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c071170 con 0x7fab9c160660 2023-10-21T17:35:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba3834640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c06e150 con 0x7fab9c070cc0 2023-10-21T17:35:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba3834640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c06de70 con 0x7fab9c0cb450 2023-10-21T17:35:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba0dcf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab9c070cc0 0x7fab9c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba0dcf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fab9c070cc0 0x7fab9c0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51892/0 (socket says 172.21.15.53:51892) 2023-10-21T17:35:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.985+0000 7faba0dcf640 1 -- 172.21.15.53:0/818019539 learned_addr learned my addr 172.21.15.53:0/818019539 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:51.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba0dcf640 1 -- 172.21.15.53:0/818019539 >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 msgr2=0x7fab9c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba1dd1640 1 --2- 172.21.15.53:0/818019539 >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 0x7fab9c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba0dcf640 1 --2- 172.21.15.53:0/818019539 >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 0x7fab9c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba0dcf640 1 -- 172.21.15.53:0/818019539 >> v1:172.21.15.124:6789/0 conn(0x7fab9c160660 legacy=0x7fab9c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba0dcf640 1 -- 172.21.15.53:0/818019539 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0c8e30 con 0x7fab9c070cc0 2023-10-21T17:35:51.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.986+0000 7faba1dd1640 1 --2- 172.21.15.53:0/818019539 >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 0x7fab9c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:51.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.987+0000 7faba0dcf640 1 --2- 172.21.15.53:0/818019539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c070cc0 0x7fab9c0caf10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0b1760 tx=0x7fab8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e9653c7847f26a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:51.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.987+0000 7fab93fff640 1 -- 172.21.15.53:0/818019539 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c031870 con 0x7fab9c070cc0 2023-10-21T17:35:51.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.987+0000 7fab93fff640 1 -- 172.21.15.53:0/818019539 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c0319d0 con 0x7fab9c070cc0 2023-10-21T17:35:51.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.987+0000 7fab93fff640 1 -- 172.21.15.53:0/818019539 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c031d60 con 0x7fab9c070cc0 2023-10-21T17:35:51.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 -- 172.21.15.53:0/818019539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c070cc0 msgr2=0x7fab9c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 --2- 172.21.15.53:0/818019539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c070cc0 0x7fab9c0caf10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0b1760 tx=0x7fab8c02b910 comp rx=0 tx=0).stop 2023-10-21T17:35:51.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 -- 172.21.15.53:0/818019539 shutdown_connections 2023-10-21T17:35:51.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 --2- 172.21.15.53:0/818019539 >> v2:172.21.15.124:3300/0 conn(0x7fab9c0cb450 0x7fab9c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 --2- 172.21.15.53:0/818019539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c070cc0 0x7fab9c0caf10 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 -- 172.21.15.53:0/818019539 >> 172.21.15.53:0/818019539 conn(0x7fab9c06a5d0 msgr2=0x7fab9c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:51.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 -- 172.21.15.53:0/818019539 shutdown_connections 2023-10-21T17:35:51.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.988+0000 7faba3834640 1 -- 172.21.15.53:0/818019539 wait complete. 2023-10-21T17:35:51.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.990+0000 7faba3834640 1 Processor -- start 2023-10-21T17:35:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.990+0000 7faba3834640 1 -- start start 2023-10-21T17:35:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab9c070cc0 0x7fab9c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab9c160660 0x7fab9c153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:51.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c7690 con 0x7fab9c0cb450 2023-10-21T17:35:51.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c15c790 con 0x7fab9c160660 2023-10-21T17:35:51.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.991+0000 7faba3834640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0c67b0 con 0x7fab9c070cc0 2023-10-21T17:35:51.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:51.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51894/0 (socket says 172.21.15.53:51894) 2023-10-21T17:35:51.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 -- 172.21.15.53:0/3853220499 learned_addr learned my addr 172.21.15.53:0/3853220499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 -- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab9c070cc0 msgr2=0x7fab9c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab9c070cc0 0x7fab9c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 -- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab9c160660 msgr2=0x7fab9c153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab9c160660 0x7fab9c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:51.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 -- 172.21.15.53:0/3853220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c0094d0 con 0x7fab9c0cb450 2023-10-21T17:35:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.992+0000 7faba0dcf640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.993+0000 7fab91ffb640 1 -- 172.21.15.53:0/3853220499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c031800 con 0x7fab9c0cb450 2023-10-21T17:35:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.993+0000 7fab91ffb640 1 -- 172.21.15.53:0/3853220499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c0443f0 con 0x7fab9c0cb450 2023-10-21T17:35:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.993+0000 7fab91ffb640 1 -- 172.21.15.53:0/3853220499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c044780 con 0x7fab9c0cb450 2023-10-21T17:35:51.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.993+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0c6a10 con 0x7fab9c0cb450 2023-10-21T17:35:51.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.993+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0c6fd0 con 0x7fab9c0cb450 2023-10-21T17:35:52.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.996+0000 7fab91ffb640 1 -- 172.21.15.53:0/3853220499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fab8c04d450 con 0x7fab9c0cb450 2023-10-21T17:35:52.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.997+0000 7fab91ffb640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 0x7fab8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.997+0000 7faba63c9740 1 Processor -- start 2023-10-21T17:35:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.998+0000 7fab91ffb640 1 -- 172.21.15.53:0/3853220499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fab8c02d040 con 0x7fab9c0cb450 2023-10-21T17:35:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.998+0000 7faba63c9740 1 -- start start 2023-10-21T17:35:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.998+0000 7faba15d0640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 0x7fab8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c2f80 0x55fab2995e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29963c0 0x55fab29c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab28b67a0 con 0x55fab29c3360 2023-10-21T17:35:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2908740 con 0x55fab29963c0 2023-10-21T17:35:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab2905e30 con 0x55fab29c2f80 2023-10-21T17:35:52.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51910/0 (socket says 172.21.15.53:51910) 2023-10-21T17:35:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 -- 172.21.15.53:0/314427205 learned_addr learned my addr 172.21.15.53:0/314427205 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba15d0640 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c2f80 0x55fab2995e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 -- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c2f80 msgr2=0x55fab2995e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c2f80 0x55fab2995e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 -- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29963c0 msgr2=0x55fab29c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29963c0 0x55fab29c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba0dcf640 1 -- 172.21.15.53:0/314427205 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab28c5910 con 0x55fab29c3360 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:51.999+0000 7faba15d0640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 0x7fab8006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab98006fd0 tx=0x7fab98009ad0 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.000+0000 7faba0dcf640 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fab8c03afd0 tx=0x7fab8c07a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8530c7e176647fd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.000+0000 7fab7effd640 1 -- 172.21.15.53:0/314427205 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c07eb20 con 0x55fab29c3360 2023-10-21T17:35:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.000+0000 7fab7effd640 1 -- 172.21.15.53:0/314427205 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c082cd0 con 0x55fab29c3360 2023-10-21T17:35:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.000+0000 7fab7effd640 1 -- 172.21.15.53:0/314427205 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0aa310 con 0x55fab29c3360 2023-10-21T17:35:52.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 -- 172.21.15.53:0/314427205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 msgr2=0x55fab29c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fab8c03afd0 tx=0x7fab8c07a460 comp rx=0 tx=0).stop 2023-10-21T17:35:52.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 -- 172.21.15.53:0/314427205 shutdown_connections 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29963c0 0x55fab29c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c5d10 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 --2- 172.21.15.53:0/314427205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c2f80 0x55fab2995e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 -- 172.21.15.53:0/314427205 >> 172.21.15.53:0/314427205 conn(0x55fab2998190 msgr2=0x55fab29e0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 -- 172.21.15.53:0/314427205 shutdown_connections 2023-10-21T17:35:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.001+0000 7faba63c9740 1 -- 172.21.15.53:0/314427205 wait complete. 2023-10-21T17:35:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.002+0000 7faba63c9740 1 -- start start 2023-10-21T17:35:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.002+0000 7faba63c9740 1 Processor -- start 2023-10-21T17:35:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29c2f80 0x55fab29c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab29c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2901720 con 0x55fab29f08f0 2023-10-21T17:35:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab289f970 con 0x55fab29c2f80 2023-10-21T17:35:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab289f610 con 0x55fab29c3360 2023-10-21T17:35:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba63c9740 10 client.0 _ll_register_callbacks cb 0x55fab28c19d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:35:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51916/0 (socket says 172.21.15.53:51916) 2023-10-21T17:35:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 -- 172.21.15.53:0/15626886 learned_addr learned my addr 172.21.15.53:0/15626886 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 -- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 msgr2=0x55fab29c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab29c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 -- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29c2f80 msgr2=0x55fab29c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.003+0000 7faba1dd1640 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29c2f80 0x55fab29c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7faba1dd1640 1 -- 172.21.15.53:0/15626886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0c6fd0 con 0x55fab29f08f0 2023-10-21T17:35:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7faba1dd1640 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fab9400daa0 tx=0x7fab9400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8bc177e46a1976 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7fab77fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:35:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400bdd0 con 0x55fab29f08f0 2023-10-21T17:35:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fab94016440 con 0x55fab29f08f0 2023-10-21T17:35:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fab2928160 con 0x55fab29f08f0 2023-10-21T17:35:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab94016600 con 0x55fab29f08f0 2023-10-21T17:35:52.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7faba63c9740 10 client.10806 fetch_fsmap learned FSMap version 103 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.004+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55fab28ffda0 con 0x55fab29f08f0 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fab2928160 con 0x55fab29f08f0 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab94016760 con 0x55fab29f08f0 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 10 client.10806 fetch_fsmap finished waiting for FSMap version 103 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 10 client.10806 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T17:35:52.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 20 client.10806 populate_metadata read hostname 'smithi053' 2023-10-21T17:35:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d9e00 0x55fab29c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 4 client.10806 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:35:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 --> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fab28d3cc0 con 0x55fab29d9e00 2023-10-21T17:35:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.005+0000 7faba15d0640 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d9e00 0x55fab29c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.006+0000 7faba15d0640 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d9e00 0x55fab29c7d50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696d401ee1661a59 server_cookie=ea4b4b1dc137917d in_seq=0 out_seq=0 2023-10-21T17:35:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.006+0000 7fab77fff640 10 client.10806 ms_handle_connect on v2:172.21.15.53:6834/542756474 2023-10-21T17:35:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7fab77fff640 1 -- 172.21.15.53:0/15626886 <== mds.0 v2:172.21.15.53:6834/542756474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fab28d3cc0 con 0x55fab29d9e00 2023-10-21T17:35:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7fab77fff640 10 client.10806 handle_command_reply: tid=0 2023-10-21T17:35:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 client.10806 shutdown 2023-10-21T17:35:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 msgr2=0x55fab29dabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fab9400daa0 tx=0x7fab9400df60 comp rx=0 tx=0).stop 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 shutdown_connections 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d9e00 0x55fab29c7d50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29f08f0 0x55fab29dabd0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab29c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 --2- 172.21.15.53:0/15626886 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29c2f80 0x55fab29c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 >> 172.21.15.53:0/15626886 conn(0x55fab29963c0 msgr2=0x55fab2999e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7fab77fff640 0 client.10806 ms_handle_reset on v2:172.21.15.53:6834/542756474 2023-10-21T17:35:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 shutdown_connections 2023-10-21T17:35:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.007+0000 7faba63c9740 1 -- 172.21.15.53:0/15626886 wait complete. 2023-10-21T17:35:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.008+0000 7faba63c9740 20 client.10806 trim_cache size 0 max 16384 2023-10-21T17:35:52.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.024+0000 7faba63c9740 1 Processor -- start 2023-10-21T17:35:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.024+0000 7faba63c9740 1 -- start start 2023-10-21T17:35:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab2994e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2995380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab2665f20 con 0x55fab29c2f80 2023-10-21T17:35:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab29e4c50 con 0x55fab29f08f0 2023-10-21T17:35:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.025+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab2997850 con 0x55fab29c3360 2023-10-21T17:35:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51922/0 (socket says 172.21.15.53:51922) 2023-10-21T17:35:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 -- 172.21.15.53:0/3548632009 learned_addr learned my addr 172.21.15.53:0/3548632009 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba1dd1640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2995380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 -- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 msgr2=0x55fab2994e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba0dcf640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab2994e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab2994e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 -- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 msgr2=0x55fab2995380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2995380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba15d0640 1 -- 172.21.15.53:0/3548632009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fab289eb80 con 0x55fab29c2f80 2023-10-21T17:35:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.026+0000 7faba1dd1640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2995380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.027+0000 7faba15d0640 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0c21e0 tx=0x7fab98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d52c8419671650 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.027+0000 7fab7f7fe640 1 -- 172.21.15.53:0/3548632009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9801a040 con 0x55fab29c2f80 2023-10-21T17:35:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.027+0000 7fab7f7fe640 1 -- 172.21.15.53:0/3548632009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98024070 con 0x55fab29c2f80 2023-10-21T17:35:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.027+0000 7fab7f7fe640 1 -- 172.21.15.53:0/3548632009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98015e30 con 0x55fab29c2f80 2023-10-21T17:35:52.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- 172.21.15.53:0/3548632009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 msgr2=0x55fab29e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0c21e0 tx=0x7fab98014000 comp rx=0 tx=0).stop 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- 172.21.15.53:0/3548632009 shutdown_connections 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2995380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29c3360 0x55fab2994e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 --2- 172.21.15.53:0/3548632009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c2f80 0x55fab29e45e0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- 172.21.15.53:0/3548632009 >> 172.21.15.53:0/3548632009 conn(0x55fab29963c0 msgr2=0x55fab29c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- 172.21.15.53:0/3548632009 shutdown_connections 2023-10-21T17:35:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- 172.21.15.53:0/3548632009 wait complete. 2023-10-21T17:35:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.028+0000 7faba63c9740 1 -- start start 2023-10-21T17:35:52.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.029+0000 7faba63c9740 1 Processor -- start 2023-10-21T17:35:52.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2a48b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29e1920 0x55fab29def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab29df9d0 con 0x55fab29c3360 2023-10-21T17:35:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fab29da940 con 0x55fab29f08f0 2023-10-21T17:35:52.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fab29dab90 con 0x55fab29e1920 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba63c9740 10 client.0 _ll_register_callbacks cb 0x7fab94008640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51928/0 (socket says 172.21.15.53:51928) 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 -- 172.21.15.53:0/76917421 learned_addr learned my addr 172.21.15.53:0/76917421 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba0dcf640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2a48b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 -- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29e1920 msgr2=0x55fab29def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29e1920 0x55fab29def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 -- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 msgr2=0x55fab2a48b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2a48b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.030+0000 7faba15d0640 1 -- 172.21.15.53:0/76917421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab98015450 con 0x55fab29c3360 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7faba15d0640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab98004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64acdb818e4e74b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7fab77fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9801a050 con 0x55fab29c3360 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fab9800b480 con 0x55fab29c3360 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab8c082cd0 con 0x55fab29c3360 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab9800b640 con 0x55fab29c3360 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7faba63c9740 10 client.10812 fetch_fsmap learned FSMap version 103 2023-10-21T17:35:52.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.031+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55fab28c4170 con 0x55fab29c3360 2023-10-21T17:35:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c082cd0 con 0x55fab29c3360 2023-10-21T17:35:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab9800b7a0 con 0x55fab29c3360 2023-10-21T17:35:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 10 client.10812 fetch_fsmap finished waiting for FSMap version 103 2023-10-21T17:35:52.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 10 client.10812 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 20 client.10812 populate_metadata read hostname 'smithi053' 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d7d10 0x55fab29f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 4 client.10812 mds_command: new command op to 10677 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 --> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fab28fa700 con 0x55fab29d7d10 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.032+0000 7faba0dcf640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d7d10 0x55fab29f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba0dcf640 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d7d10 0x55fab29f90f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5fe029bd6c60116 server_cookie=f7782d6287fa55f3 in_seq=0 out_seq=0 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7fab77fff640 10 client.10812 ms_handle_connect on v2:172.21.15.53:6834/542756474 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7fab77fff640 1 -- 172.21.15.53:0/76917421 <== mds.0 v2:172.21.15.53:6834/542756474 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55fab28fa700 con 0x55fab29d7d10 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7fab77fff640 10 client.10812 handle_command_reply: tid=0 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 client.10812 shutdown 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 msgr2=0x55fab29c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fab98000c00 tx=0x7fab98004760 comp rx=0 tx=0).stop 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 shutdown_connections 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:6834/542756474,v1:172.21.15.53:6835/542756474] conn(0x55fab29d7d10 0x55fab29f90f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55fab29e1920 0x55fab29def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55fab29f08f0 0x55fab2a48b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 --2- 172.21.15.53:0/76917421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fab29c3360 0x55fab29c04f0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7fab77fff640 0 client.10812 ms_handle_reset on v2:172.21.15.53:6834/542756474 2023-10-21T17:35:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 >> 172.21.15.53:0/76917421 conn(0x55fab29963c0 msgr2=0x55fab29c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 shutdown_connections 2023-10-21T17:35:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.033+0000 7faba63c9740 1 -- 172.21.15.53:0/76917421 wait complete. 2023-10-21T17:35:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba63c9740 20 client.10812 trim_cache size 0 max 16384 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 10677, 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:35:52.047 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.7855743679999998, 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 103, 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 215, 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 215, 2023-10-21T17:35:52.048 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.645602705 2023-10-21T17:35:52.049 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:35:52.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 msgr2=0x7fab8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 0x7fab8006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab98006fd0 tx=0x7fab98009ad0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 msgr2=0x7fab9c0c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fab8c000c00 tx=0x7fab8c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 shutdown_connections 2023-10-21T17:35:52.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fab8006af90 0x7fab8006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab9c160660 0x7fab9c153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab9c0cb450 0x7fab9c0c6270 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 --2- 172.21.15.53:0/3853220499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab9c070cc0 0x7fab9c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 >> 172.21.15.53:0/3853220499 conn(0x7fab9c06a5d0 msgr2=0x7fab9c0ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 shutdown_connections 2023-10-21T17:35:52.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.034+0000 7faba3834640 1 -- 172.21.15.53:0/3853220499 wait complete. 2023-10-21T17:35:52.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:35:52.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.274+0000 7f288b941640 1 Processor -- start 2023-10-21T17:35:52.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.274+0000 7f288b941640 1 -- start start 2023-10-21T17:35:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.276+0000 7f288b941640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f288406d4c0 0x7f2884162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.276+0000 7f288b941640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28840cb150 0x7f28840cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.276+0000 7f288b941640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840ae140 con 0x7f28840add60 2023-10-21T17:35:52.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.276+0000 7f288b941640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2884161f10 con 0x7f28840cb150 2023-10-21T17:35:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.276+0000 7f288b941640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840cbb20 con 0x7f288406d4c0 2023-10-21T17:35:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2889ede640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f28840cb150 0x7f28840cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f28896dd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f28840add60 0x7f28840cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:43538/0 (socket says 172.21.15.53:43538) 2023-10-21T17:35:52.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f28896dd640 1 -- 172.21.15.53:0/2964495938 learned_addr learned my addr 172.21.15.53:0/2964495938 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2888edc640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f288406d4c0 0x7f2884162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2889ede640 1 -- 172.21.15.53:0/2964495938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f288406d4c0 msgr2=0x7f2884162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2889ede640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f288406d4c0 0x7f2884162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2889ede640 1 -- 172.21.15.53:0/2964495938 >> v1:172.21.15.53:6789/0 conn(0x7f28840add60 legacy=0x7f28840cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.277+0000 7f2889ede640 1 -- 172.21.15.53:0/2964495938 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840ccf20 con 0x7f28840cb150 2023-10-21T17:35:52.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.278+0000 7f2889ede640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28840cb150 0x7f28840cb5e0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f287c0097a0 tx=0x7f287c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b62612e6267948 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f2883fff640 1 -- 172.21.15.53:0/2964495938 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c031780 con 0x7f28840cb150 2023-10-21T17:35:52.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f2883fff640 1 -- 172.21.15.53:0/2964495938 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c0318e0 con 0x7f28840cb150 2023-10-21T17:35:52.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f2883fff640 1 -- 172.21.15.53:0/2964495938 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c031c70 con 0x7f28840cb150 2023-10-21T17:35:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 -- 172.21.15.53:0/2964495938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28840cb150 msgr2=0x7f28840cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28840cb150 0x7f28840cb5e0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f287c0097a0 tx=0x7f287c02b820 comp rx=0 tx=0).stop 2023-10-21T17:35:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 -- 172.21.15.53:0/2964495938 shutdown_connections 2023-10-21T17:35:52.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28840cb150 0x7f28840cb5e0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 --2- 172.21.15.53:0/2964495938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f288406d4c0 0x7f2884162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 -- 172.21.15.53:0/2964495938 >> 172.21.15.53:0/2964495938 conn(0x7f288406a380 msgr2=0x7f28840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.279+0000 7f288b941640 1 -- 172.21.15.53:0/2964495938 shutdown_connections 2023-10-21T17:35:52.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.280+0000 7f288b941640 1 -- 172.21.15.53:0/2964495938 wait complete. 2023-10-21T17:35:52.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.281+0000 7f288b941640 1 Processor -- start 2023-10-21T17:35:52.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.281+0000 7f288b941640 1 -- start start 2023-10-21T17:35:52.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 0x7f2884159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 0x7f2884163b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288415dc00 con 0x7f288406d4c0 2023-10-21T17:35:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288415dec0 con 0x7f28840add60 2023-10-21T17:35:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.282+0000 7f288b941640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288415e1a0 con 0x7f28840cb150 2023-10-21T17:35:52.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51946/0 (socket says 172.21.15.53:51946) 2023-10-21T17:35:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 -- 172.21.15.53:0/2767402252 learned_addr learned my addr 172.21.15.53:0/2767402252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f2888edc640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 0x7f2884159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f2889ede640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 0x7f2884163b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 -- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 msgr2=0x7f2884163b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 0x7f2884163b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 -- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 msgr2=0x7f2884159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 0x7f2884159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f28896dd640 1 -- 172.21.15.53:0/2767402252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c009400 con 0x7f288406d4c0 2023-10-21T17:35:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f2888edc640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 0x7f2884159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.283+0000 7f2889ede640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 0x7f2884163b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f28896dd640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f287400cba0 tx=0x7f287400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400ad40 con 0x7f288406d4c0 2023-10-21T17:35:52.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874012400 con 0x7f288406d4c0 2023-10-21T17:35:52.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400aea0 con 0x7f288406d4c0 2023-10-21T17:35:52.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288415e480 con 0x7f288406d4c0 2023-10-21T17:35:52.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.284+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2884159fa0 con 0x7f288406d4c0 2023-10-21T17:35:52.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.285+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288415a240 con 0x7f288406d4c0 2023-10-21T17:35:52.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.287+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2874007070 con 0x7f288406d4c0 2023-10-21T17:35:52.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.288+0000 7f2881ffb640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 0x7f287006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.288+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2874089890 con 0x7f288406d4c0 2023-10-21T17:35:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.289+0000 7f2888edc640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 0x7f287006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:52.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.290+0000 7f2888edc640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 0x7f287006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28841646b0 tx=0x7f2878008040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:52.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.293+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2874054710 con 0x7f288406d4c0 2023-10-21T17:35:52.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.524+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2884067490 con 0x7f288406d4c0 2023-10-21T17:35:52.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.525+0000 7f2881ffb640 1 -- 172.21.15.53:0/2767402252 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2874017070 con 0x7f288406d4c0 2023-10-21T17:35:52.526 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 msgr2=0x7f287006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 0x7f287006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28841646b0 tx=0x7f2878008040 comp rx=0 tx=0).stop 2023-10-21T17:35:52.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 msgr2=0x7f2884153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:52.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f287400cba0 tx=0x7f287400a430 comp rx=0 tx=0).stop 2023-10-21T17:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 shutdown_connections 2023-10-21T17:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f287006af90 0x7f287006d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28840cb150 0x7f2884163b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28840add60 0x7f2884159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 --2- 172.21.15.53:0/2767402252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f288406d4c0 0x7f2884153030 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.527+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 >> 172.21.15.53:0/2767402252 conn(0x7f288406a380 msgr2=0x7f2884160ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.528+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 shutdown_connections 2023-10-21T17:35:52.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:52.528+0000 7f288b941640 1 -- 172.21.15.53:0/2767402252 wait complete. 2023-10-21T17:35:52.548 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:35:52.548 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:35:52.549 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:35:52.549 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:35:52.550 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:35:52.550 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:35:52.550 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:35:52.551 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:35:52.551 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:35:52.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:52.552 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:35:52.609 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:35:52.609 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:35:52.610 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:35:52.610 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.610 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:35:52.610 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.610 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:35:52.611 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:35:52.611 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:35:52.611 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:35:52.611 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:35:52.611 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:35:52.612 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:35:52.612 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:35:52.612 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:35:52.612 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:35:52.612 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 36741sec preferred_lft 36741sec 2023-10-21T17:35:52.613 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:35:52.613 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.613 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:35:52.613 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:35:52.613 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:35:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.615 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:35:52.615 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:35:52.615 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:35:52.615 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.615 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:35:52.616 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:35:52.616 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:35:52.616 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:35:52.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:52.617 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:35:52.715 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:35:52.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:52.720 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:35:52.775 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:35:52.776 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:35:52.776 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:35:52.776 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:35:52.776 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:35:52.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:52.777 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:35:52.778 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:35:52.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:35:52.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:35:52.779 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:35:52.779 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:35:52.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:52.978 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:35:53.032 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:35:53.032 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:35:53.033 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:35:53.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:53.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:35:53.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:35:53.131 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:35:53.191 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:35:53.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:35:53.191 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:35:53.249 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:35:53.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:53.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:35:53.386 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:35:53.386 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:35:53.421 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:35:53.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:35:53.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:35:53.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:35:53.422 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:35:53.426 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:35:53.426 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:35:53.426 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:35:53.427 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:35:53.427 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:35:53.430 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:35:53.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:53.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:35:53.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:53.520 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:35:53.595 INFO:teuthology.orchestra.run.smithi053.stdout:client10824 2023-10-21T17:35:53.595 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:35:53.596 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:35:53.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:35:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.881+0000 7fa9bd3e4640 1 Processor -- start 2023-10-21T17:35:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.881+0000 7fa9bd3e4640 1 -- start start 2023-10-21T17:35:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bd3e4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa9b815ccd0 0x7fa9b80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bd3e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9b80b2180 0x7fa9b80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bd3e4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b80ce9b0 con 0x7fa9b806bcb0 2023-10-21T17:35:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bd3e4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80b2b30 con 0x7fa9b80b2180 2023-10-21T17:35:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bd3e4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80cec90 con 0x7fa9b815ccd0 2023-10-21T17:35:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bcbe3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9b80b2180 0x7fa9b80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bcbe3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa9b80b2180 0x7fa9b80b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51954/0 (socket says 172.21.15.53:51954) 2023-10-21T17:35:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.882+0000 7fa9bcbe3640 1 -- 172.21.15.53:0/362186210 learned_addr learned my addr 172.21.15.53:0/362186210 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.883+0000 7fa9b7fff640 1 --2- 172.21.15.53:0/362186210 >> v2:172.21.15.124:3300/0 conn(0x7fa9b815ccd0 0x7fa9b80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.883+0000 7fa9bcbe3640 1 -- 172.21.15.53:0/362186210 >> v2:172.21.15.124:3300/0 conn(0x7fa9b815ccd0 msgr2=0x7fa9b80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.883+0000 7fa9bcbe3640 1 --2- 172.21.15.53:0/362186210 >> v2:172.21.15.124:3300/0 conn(0x7fa9b815ccd0 0x7fa9b80ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.883+0000 7fa9bcbe3640 1 -- 172.21.15.53:0/362186210 >> v1:172.21.15.124:6789/0 conn(0x7fa9b806bcb0 legacy=0x7fa9b8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.883+0000 7fa9bcbe3640 1 -- 172.21.15.53:0/362186210 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80cef70 con 0x7fa9b80b2180 2023-10-21T17:35:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.884+0000 7fa9bcbe3640 1 --2- 172.21.15.53:0/362186210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b80b25f0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ce280 tx=0x7fa9b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610f6419eeada7ac server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9b6ffd640 1 -- 172.21.15.53:0/362186210 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0030860 con 0x7fa9b80b2180 2023-10-21T17:35:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9b6ffd640 1 -- 172.21.15.53:0/362186210 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b00309c0 con 0x7fa9b80b2180 2023-10-21T17:35:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9b6ffd640 1 -- 172.21.15.53:0/362186210 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0030d50 con 0x7fa9b80b2180 2023-10-21T17:35:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/362186210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 msgr2=0x7fa9b80b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/362186210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b80b25f0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa9b80ce280 tx=0x7fa9b002b9a0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/362186210 shutdown_connections 2023-10-21T17:35:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/362186210 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b80b25f0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/362186210 >> v2:172.21.15.124:3300/0 conn(0x7fa9b815ccd0 0x7fa9b80ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.885+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/362186210 >> 172.21.15.53:0/362186210 conn(0x7fa9b80cd180 msgr2=0x7fa9b80cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.886+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/362186210 shutdown_connections 2023-10-21T17:35:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.886+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/362186210 wait complete. 2023-10-21T17:35:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.887+0000 7fa9bd3e4640 1 Processor -- start 2023-10-21T17:35:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.887+0000 7fa9bd3e4640 1 -- start start 2023-10-21T17:35:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9b806bcb0 0x7fa9b814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 0x7fa9b815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b814c0e0 con 0x7fa9b80b2180 2023-10-21T17:35:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b815f610 con 0x7fa9b815ccd0 2023-10-21T17:35:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.888+0000 7fa9bd3e4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b815ffe0 con 0x7fa9b806bcb0 2023-10-21T17:35:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.889+0000 7fa9b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.889+0000 7fa9b77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51956/0 (socket says 172.21.15.53:51956) 2023-10-21T17:35:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.889+0000 7fa9b77fe640 1 -- 172.21.15.53:0/2841493363 learned_addr learned my addr 172.21.15.53:0/2841493363 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.889+0000 7fa9bcbe3640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 0x7fa9b815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.889+0000 7fa9b7fff640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9b806bcb0 0x7fa9b814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 -- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9b806bcb0 msgr2=0x7fa9b814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:53.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9b806bcb0 0x7fa9b814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 -- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 msgr2=0x7fa9b815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:53.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 0x7fa9b815aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 -- 172.21.15.53:0/2841493363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00094d0 con 0x7fa9b80b2180 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9bcbe3640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 0x7fa9b815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b77fe640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c450 tx=0x7fa9a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00052d0 con 0x7fa9b80b2180 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a0005430 con 0x7fa9b80b2180 2023-10-21T17:35:53.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0012870 con 0x7fa9b80b2180 2023-10-21T17:35:53.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b815ea00 con 0x7fa9b80b2180 2023-10-21T17:35:53.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.890+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b815efc0 con 0x7fa9b80b2180 2023-10-21T17:35:53.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.892+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b815f7a0 con 0x7fa9b80b2180 2023-10-21T17:35:53.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.893+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9a00129d0 con 0x7fa9b80b2180 2023-10-21T17:35:53.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.894+0000 7fa9b4ff9640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 0x7fa99806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.894+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa9a00894c0 con 0x7fa9b80b2180 2023-10-21T17:35:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.895+0000 7fa9b7fff640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 0x7fa99806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.896+0000 7fa9b7fff640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 0x7fa99806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009f60 tx=0x7fa9a800b040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:53.899+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9a0054200 con 0x7fa9b80b2180 2023-10-21T17:35:54.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.116+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 --> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9b806c110 con 0x7fa99806afc0 2023-10-21T17:35:54.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.120+0000 7fa9b4ff9640 1 -- 172.21.15.53:0/2841493363 <== mgr.37520 v2:172.21.15.124:6832/248442090 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa9a8009b50 con 0x7fa99806afc0 2023-10-21T17:35:54.121 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:35:54.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:35:54.122 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:35:54.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:35:54.122 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:35:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.122+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 msgr2=0x7fa99806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.122+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 0x7fa99806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009f60 tx=0x7fa9a800b040 comp rx=0 tx=0).stop 2023-10-21T17:35:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.122+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 msgr2=0x7fa9b815faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.122+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c450 tx=0x7fa9a000c910 comp rx=0 tx=0).stop 2023-10-21T17:35:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 shutdown_connections 2023-10-21T17:35:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7fa99806afc0 0x7fa99806d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa9b815ccd0 0x7fa9b815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa9b80b2180 0x7fa9b815faa0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 --2- 172.21.15.53:0/2841493363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa9b806bcb0 0x7fa9b814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 >> 172.21.15.53:0/2841493363 conn(0x7fa9b80cd180 msgr2=0x7fa9b80cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 shutdown_connections 2023-10-21T17:35:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.123+0000 7fa9bd3e4640 1 -- 172.21.15.53:0/2841493363 wait complete. 2023-10-21T17:35:54.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:35:54.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.378+0000 7f5977f1d640 1 Processor -- start 2023-10-21T17:35:54.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.379+0000 7f5977f1d640 1 -- start start 2023-10-21T17:35:54.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 0x7f5970070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59700c9910 0x7f59700caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59700cb450 0x7f59700b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597006e150 con 0x7f59700c9910 2023-10-21T17:35:54.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59700cb8c0 con 0x7f5970160660 2023-10-21T17:35:54.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.380+0000 7f5977f1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597006de70 con 0x7f59700cb450 2023-10-21T17:35:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59700c9910 0x7f59700caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f59700c9910 0x7f59700caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45872/0 (socket says 172.21.15.53:45872) 2023-10-21T17:35:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 -- 172.21.15.53:0/2141429807 learned_addr learned my addr 172.21.15.53:0/2141429807 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:54.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59764ba640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59700cb450 0x7f59700b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f5975cb9640 1 --2- 172.21.15.53:0/2141429807 >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 0x7f5970070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 -- 172.21.15.53:0/2141429807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59700cb450 msgr2=0x7f59700b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59700cb450 0x7f59700b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 -- 172.21.15.53:0/2141429807 >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 msgr2=0x7f5970070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 --2- 172.21.15.53:0/2141429807 >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 0x7f5970070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f59754b8640 1 -- 172.21.15.53:0/2141429807 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700c8e30 con 0x7f59700c9910 2023-10-21T17:35:54.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.381+0000 7f5975cb9640 1 --2- 172.21.15.53:0/2141429807 >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 0x7f5970070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:35:54.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.383+0000 7f59754b8640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59700caf10 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f59700b0d80 tx=0x7f596402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70366ae648a66fb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:54.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.383+0000 7f5974cb7640 1 -- 172.21.15.53:0/2141429807 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964031870 con 0x7f59700c9910 2023-10-21T17:35:54.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.383+0000 7f5974cb7640 1 -- 172.21.15.53:0/2141429807 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59640319d0 con 0x7f59700c9910 2023-10-21T17:35:54.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.383+0000 7f5974cb7640 1 -- 172.21.15.53:0/2141429807 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964031d60 con 0x7f59700c9910 2023-10-21T17:35:54.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 -- 172.21.15.53:0/2141429807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 msgr2=0x7f59700caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59700caf10 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f59700b0d80 tx=0x7f596402b910 comp rx=0 tx=0).stop 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 -- 172.21.15.53:0/2141429807 shutdown_connections 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59700cb450 0x7f59700b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 --2- 172.21.15.53:0/2141429807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59700caf10 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 --2- 172.21.15.53:0/2141429807 >> v2:172.21.15.124:3300/0 conn(0x7f5970160660 0x7f5970070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 -- 172.21.15.53:0/2141429807 >> 172.21.15.53:0/2141429807 conn(0x7f597006a5d0 msgr2=0x7f59700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:54.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 -- 172.21.15.53:0/2141429807 shutdown_connections 2023-10-21T17:35:54.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.384+0000 7f5977f1d640 1 -- 172.21.15.53:0/2141429807 wait complete. 2023-10-21T17:35:54.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.386+0000 7f5977f1d640 1 Processor -- start 2023-10-21T17:35:54.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.387+0000 7f5977f1d640 1 -- start start 2023-10-21T17:35:54.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59700cb450 0x7f5970156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5970160660 0x7f59700c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c46d0 con 0x7f59700c9910 2023-10-21T17:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c49a0 con 0x7f59700cb450 2023-10-21T17:35:54.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5977f1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597015b880 con 0x7f5970160660 2023-10-21T17:35:54.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5975cb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5975cb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45880/0 (socket says 172.21.15.53:45880) 2023-10-21T17:35:54.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.388+0000 7f5975cb9640 1 -- 172.21.15.53:0/4267147872 learned_addr learned my addr 172.21.15.53:0/4267147872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:54.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f59764ba640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5970160660 0x7f59700c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f59754b8640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59700cb450 0x7f5970156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 -- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5970160660 msgr2=0x7f59700c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:54.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5970160660 0x7f59700c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 -- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59700cb450 msgr2=0x7f5970156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:54.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59700cb450 0x7f5970156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:54.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 -- 172.21.15.53:0/4267147872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640094d0 con 0x7f59700c9910 2023-10-21T17:35:54.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5975cb9640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f596000ce00 tx=0x7f596000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:54.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000f890 con 0x7f59700c9910 2023-10-21T17:35:54.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000f9f0 con 0x7f59700c9910 2023-10-21T17:35:54.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596000fdc0 con 0x7f59700c9910 2023-10-21T17:35:54.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.389+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597015bb60 con 0x7f59700c9910 2023-10-21T17:35:54.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.390+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700c32a0 con 0x7f59700c9910 2023-10-21T17:35:54.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.391+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597015b6e0 con 0x7f59700c9910 2023-10-21T17:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.392+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 114) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f596000b040 con 0x7f59700c9910 2023-10-21T17:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.393+0000 7f596e7fc640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 0x7f595c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.393+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f596008a400 con 0x7f59700c9910 2023-10-21T17:35:54.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.393+0000 7f59754b8640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 0x7f595c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:54.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.395+0000 7f59754b8640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 0x7f595c06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5964036000 tx=0x7f596403c040 comp rx=0 tx=0).ready entity=mgr.37520 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:54.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.398+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f597015b6e0 con 0x7f59700c9910 2023-10-21T17:35:54.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:54.644+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f597015b6e0 con 0x7f59700c9910 2023-10-21T17:35:55.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.316+0000 7f596e7fc640 1 -- 172.21.15.53:0/4267147872 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v115) v1 ==== 92+0+0 (secure 0 0 0) 0x7f597015b6e0 con 0x7f59700c9910 2023-10-21T17:35:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 msgr2=0x7f595c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 0x7f595c06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5964036000 tx=0x7f596403c040 comp rx=0 tx=0).stop 2023-10-21T17:35:55.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 msgr2=0x7f59701527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f596000ce00 tx=0x7f596000a650 comp rx=0 tx=0).stop 2023-10-21T17:35:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 shutdown_connections 2023-10-21T17:35:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f595c06af90 0x7f595c06d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5970160660 0x7f59700c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59700cb450 0x7f5970156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 --2- 172.21.15.53:0/4267147872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59700c9910 0x7f59701527d0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.321+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 >> 172.21.15.53:0/4267147872 conn(0x7f597006a5d0 msgr2=0x7f59700c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.322+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 shutdown_connections 2023-10-21T17:35:55.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.323+0000 7f5977f1d640 1 -- 172.21.15.53:0/4267147872 wait complete. 2023-10-21T17:35:55.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:35:55.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.545+0000 7f7e341fb640 1 Processor -- start 2023-10-21T17:35:55.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.545+0000 7f7e341fb640 1 -- start start 2023-10-21T17:35:55.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.546+0000 7f7e341fb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.546+0000 7f7e341fb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e341fb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0cbad0 con 0x7f7e2c06dfc0 2023-10-21T17:35:55.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e341fb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b2620 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e341fb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b2060 con 0x7f7e2c0cb130 2023-10-21T17:35:55.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45904/0 (socket says 172.21.15.53:45904) 2023-10-21T17:35:55.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 -- 172.21.15.53:0/3763078095 learned_addr learned my addr 172.21.15.53:0/3763078095 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:55.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e32798640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 -- 172.21.15.53:0/3763078095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 msgr2=0x7f7e2c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:55.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.547+0000 7f7e31f97640 1 -- 172.21.15.53:0/3763078095 >> v1:172.21.15.124:6789/0 conn(0x7f7e2c06dfc0 legacy=0x7f7e2c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:55.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.548+0000 7f7e31f97640 1 -- 172.21.15.53:0/3763078095 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0b2340 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.549+0000 7f7e31f97640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009870 tx=0x7f7e2802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cf3836d7abc384 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:55.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.549+0000 7f7e30f95640 1 -- 172.21.15.53:0/3763078095 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28031640 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.549+0000 7f7e30f95640 1 -- 172.21.15.53:0/3763078095 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e280317a0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.549+0000 7f7e30f95640 1 -- 172.21.15.53:0/3763078095 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28031b30 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 -- 172.21.15.53:0/3763078095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 msgr2=0x7f7e2c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009870 tx=0x7f7e2802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 -- 172.21.15.53:0/3763078095 shutdown_connections 2023-10-21T17:35:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 --2- 172.21.15.53:0/3763078095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0b15b0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 -- 172.21.15.53:0/3763078095 >> 172.21.15.53:0/3763078095 conn(0x7f7e2c06a5d0 msgr2=0x7f7e2c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 -- 172.21.15.53:0/3763078095 shutdown_connections 2023-10-21T17:35:55.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.550+0000 7f7e341fb640 1 -- 172.21.15.53:0/3763078095 wait complete. 2023-10-21T17:35:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.552+0000 7f7e341fb640 1 Processor -- start 2023-10-21T17:35:55.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.552+0000 7f7e341fb640 1 -- start start 2023-10-21T17:35:55.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e2c06dfc0 0x7f7e2c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0cc950 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1ee930 con 0x7f7e2c06dfc0 2023-10-21T17:35:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e341fb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1eec10 con 0x7f7e2c0cb130 2023-10-21T17:35:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e31796640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e31796640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45908/0 (socket says 172.21.15.53:45908) 2023-10-21T17:35:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.553+0000 7f7e31796640 1 -- 172.21.15.53:0/2326727715 learned_addr learned my addr 172.21.15.53:0/2326727715 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:55.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31f97640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e2c06dfc0 0x7f7e2c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e32798640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 msgr2=0x7f7e2c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e2c06dfc0 msgr2=0x7f7e2c15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e2c06dfc0 0x7f7e2c15dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 -- 172.21.15.53:0/2326727715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280094d0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e31796640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b3c0 tx=0x7f7e1c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.554+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c011830 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.555+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c011990 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.555+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c011d80 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.555+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c1eeef0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.555+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c1ef4b0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.556+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c1ef6a0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.557+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 115) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7e1c01b460 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.559+0000 7f7e227fc640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 0x7f7e1006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.559+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7e1c089650 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.559+0000 7f7e31f97640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 msgr2=0x7f7e1006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:55.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.559+0000 7f7e31f97640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 0x7f7e1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:55.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.562+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e1c054390 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.758+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7e2c04e160 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.760+0000 7f7e31f97640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 msgr2=0x7f7e1006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:55.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.760+0000 7f7e31f97640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 0x7f7e1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:55.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.763+0000 7f7e227fc640 1 -- 172.21.15.53:0/2326727715 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7e1c0579b0 con 0x7f7e2c0afbe0 2023-10-21T17:35:55.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 msgr2=0x7f7e1006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 0x7f7e1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 msgr2=0x7f7e2c0cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b3c0 tx=0x7f7e1c00b880 comp rx=0 tx=0).stop 2023-10-21T17:35:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 shutdown_connections 2023-10-21T17:35:55.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f7e1006afe0 0x7f7e1006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e2c0cb130 0x7f7e2c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e2c0afbe0 0x7f7e2c0cef20 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 --2- 172.21.15.53:0/2326727715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e2c06dfc0 0x7f7e2c15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 >> 172.21.15.53:0/2326727715 conn(0x7f7e2c06a5d0 msgr2=0x7f7e2c0c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:55.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 shutdown_connections 2023-10-21T17:35:55.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.766+0000 7f7e341fb640 1 -- 172.21.15.53:0/2326727715 wait complete. 2023-10-21T17:35:55.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:35:55.842 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:55.840+0000 7f43abc52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:35:55.866 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:55.865+0000 7ff6ae82b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:35:55.893 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:55.892+0000 7f43abc52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:35:55.922 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:55.921+0000 7ff6ae82b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:35:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.992+0000 7f19e0133640 1 Processor -- start 2023-10-21T17:35:55.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.992+0000 7f19e0133640 1 -- start start 2023-10-21T17:35:55.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.993+0000 7f19e0133640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19d815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.993+0000 7f19e0133640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d80b12b0 con 0x7f19d80b1910 2023-10-21T17:35:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.993+0000 7f19e0133640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d80b1500 con 0x7f19d815f6d0 2023-10-21T17:35:55.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.993+0000 7f19e0133640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80cf1d0 con 0x7f19d815e3c0 2023-10-21T17:35:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.994+0000 7f19dd6ce640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19d80b1910 0x7f19d815efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48014/0 (socket says 172.21.15.53:48014) 2023-10-21T17:35:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.994+0000 7f19dd6ce640 1 -- 172.21.15.53:0/1601090213 learned_addr learned my addr 172.21.15.53:0/1601090213 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:55.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.994+0000 7f19ddecf640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19d815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:55.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.994+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320877372 0 0) 0x7f19d80b12b0 con 0x7f19d80b1910 2023-10-21T17:35:55.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.994+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c4003520 con 0x7f19d80b1910 2023-10-21T17:35:55.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090497306 0 0) 0x7f19d80b1500 con 0x7f19d815f6d0 2023-10-21T17:35:55.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d80b12b0 con 0x7f19d815f6d0 2023-10-21T17:35:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259102523 0 0) 0x7f19c4003520 con 0x7f19d80b1910 2023-10-21T17:35:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 msgr2=0x7f19d815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19d815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 >> v1:172.21.15.124:6789/0 conn(0x7f19d815f6d0 legacy=0x7f19d815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:55.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80cf4b0 con 0x7f19d80b1910 2023-10-21T17:35:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f19c8001670 con 0x7f19d80b1910 2023-10-21T17:35:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 >> v1:172.21.15.53:6789/0 conn(0x7f19d80b1910 legacy=0x7f19d815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19c4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c4004d00 0x7f19c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80cf1d0 con 0x7f19c4003b30 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c400ba30 con 0x7f19c4004d00 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c400bbf0 con 0x7f19d815e3c0 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19ddecf640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19c4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19de6d0640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c4004d00 0x7f19c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.995+0000 7f19dd6ce640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.996+0000 7f19ddecf640 1 -- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 msgr2=0x7f19c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.996+0000 7f19ddecf640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d815e3c0 0x7f19c40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.996+0000 7f19ddecf640 1 -- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c4004d00 msgr2=0x7f19c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:35:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.996+0000 7f19ddecf640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19c4004d00 0x7f19c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.996+0000 7f19ddecf640 1 -- 172.21.15.53:0/1601090213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4003520 con 0x7f19c4003b30 2023-10-21T17:35:56.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.997+0000 7f19ddecf640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19c4003f30 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009ac0 tx=0x7f19c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e43be1ac9de1baf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.997+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d402c6b0 con 0x7f19c4003b30 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.997+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d402c810 con 0x7f19c4003b30 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.997+0000 7f19dcecd640 1 -- 172.21.15.53:0/1601090213 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d402cba0 con 0x7f19c4003b30 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 -- 172.21.15.53:0/1601090213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 msgr2=0x7f19c4003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 --2- 172.21.15.53:0/1601090213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19c4003f30 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009ac0 tx=0x7f19c40057f0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19de6d0640 1 -- 172.21.15.53:0/1601090213 reap_dead start 2023-10-21T17:35:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 -- 172.21.15.53:0/1601090213 shutdown_connections 2023-10-21T17:35:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 -- 172.21.15.53:0/1601090213 >> 172.21.15.53:0/1601090213 conn(0x7f19d8099d40 msgr2=0x7f19d80d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 -- 172.21.15.53:0/1601090213 shutdown_connections 2023-10-21T17:35:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:55.998+0000 7f19e0133640 1 -- 172.21.15.53:0/1601090213 wait complete. 2023-10-21T17:35:56.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.000+0000 7f19e0133640 1 Processor -- start 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.000+0000 7f19e0133640 1 -- start start 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d80ae3b0 0x7f19d80ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19d80b37e0 0x7f19d80b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80aed50 con 0x7f19c4003b30 2023-10-21T17:35:56.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81f27b0 con 0x7f19d80b37e0 2023-10-21T17:35:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19e0133640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81f29f0 con 0x7f19d80ae3b0 2023-10-21T17:35:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45932/0 (socket says 172.21.15.53:45932) 2023-10-21T17:35:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 -- 172.21.15.53:0/3604991253 learned_addr learned my addr 172.21.15.53:0/3604991253 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:56.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d80ae3b0 msgr2=0x7f19d80ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d80ae3b0 0x7f19d80ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19d80b37e0 msgr2=0x7f19d80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19d80b37e0 0x7f19d80b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.001+0000 7f19ddecf640 1 -- 172.21.15.53:0/3604991253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d81f2cd0 con 0x7f19c4003b30 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19ddecf640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f19d402c250 tx=0x7f19d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:56.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003810 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4003970 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4003ce0 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d81f2fb0 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.002+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81f3570 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.003+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d81f3730 con 0x7f19c4003b30 2023-10-21T17:35:56.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.004+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 115) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f19d4034440 con 0x7f19c4003b30 2023-10-21T17:35:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.005+0000 7f19ce7fc640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 0x7f19bc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.005+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f19d40abc60 con 0x7f19c4003b30 2023-10-21T17:35:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.005+0000 7f19dd6ce640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 msgr2=0x7f19bc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:56.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.006+0000 7f19dd6ce640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 0x7f19bc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:56.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.013+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19d4076af0 con 0x7f19c4003b30 2023-10-21T17:35:56.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.204+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f19d80aed50 con 0x7f19c4003b30 2023-10-21T17:35:56.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.207+0000 7f19dd6ce640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 msgr2=0x7f19bc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.207+0000 7f19dd6ce640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 0x7f19bc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:56.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.209+0000 7f19ce7fc640 1 -- 172.21.15.53:0/3604991253 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f19d40044e0 con 0x7f19c4003b30 2023-10-21T17:35:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 msgr2=0x7f19bc06d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:56.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 0x7f19bc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 msgr2=0x7f19d815b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f19d402c250 tx=0x7f19d4004730 comp rx=0 tx=0).stop 2023-10-21T17:35:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 shutdown_connections 2023-10-21T17:35:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f19bc06b010 0x7f19bc06d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19d80b37e0 0x7f19d80b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19d80ae3b0 0x7f19d80ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 --2- 172.21.15.53:0/3604991253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19c4003b30 0x7f19d815b3d0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 >> 172.21.15.53:0/3604991253 conn(0x7f19d8099d40 msgr2=0x7f19d80ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 shutdown_connections 2023-10-21T17:35:56.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.212+0000 7f19e0133640 1 -- 172.21.15.53:0/3604991253 wait complete. 2023-10-21T17:35:56.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:35:56.232 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T17:35:56.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-21T17:35:56.339 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.338+0000 7f43abc52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:35:56.391 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.389+0000 7f43abc52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:35:56.405 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.404+0000 7ff6ae82b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:35:56.440 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.438+0000 7f43abc52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:35:56.460 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.459+0000 7ff6ae82b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:35:56.499 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.498+0000 7f43abc52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:35:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.500+0000 7f43523c0640 1 Processor -- start 2023-10-21T17:35:56.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.500+0000 7f43523c0640 1 -- start start 2023-10-21T17:35:56.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.502+0000 7f43523c0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f434c071c30 0x7f434c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.502+0000 7f43523c0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c0ce210 con 0x7f434c0720c0 2023-10-21T17:35:56.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f43523c0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c0cdb40 con 0x7f434c1601b0 2023-10-21T17:35:56.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f43523c0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434c0cdee0 con 0x7f434c071c30 2023-10-21T17:35:56.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f434b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f434c071c30 0x7f434c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f435095d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f434c0720c0 0x7f434c0ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48022/0 (socket says 172.21.15.53:48022) 2023-10-21T17:35:56.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f435095d640 1 -- 172.21.15.53:0/2220136711 learned_addr learned my addr 172.21.15.53:0/2220136711 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:35:56.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f434b7fe640 1 -- 172.21.15.53:0/2220136711 >> v1:172.21.15.124:6789/0 conn(0x7f434c1601b0 legacy=0x7f434c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f434b7fe640 1 -- 172.21.15.53:0/2220136711 >> v1:172.21.15.53:6789/0 conn(0x7f434c0720c0 legacy=0x7f434c0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.503+0000 7f434b7fe640 1 -- 172.21.15.53:0/2220136711 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c161530 con 0x7f434c071c30 2023-10-21T17:35:56.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.505+0000 7f434b7fe640 1 --2- 172.21.15.53:0/2220136711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c071c30 0x7f434c069d50 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f43400097a0 tx=0x7f434002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca8780a3e20d40e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:56.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.505+0000 7f434affd640 1 -- 172.21.15.53:0/2220136711 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340031830 con 0x7f434c071c30 2023-10-21T17:35:56.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.505+0000 7f434affd640 1 -- 172.21.15.53:0/2220136711 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4340031990 con 0x7f434c071c30 2023-10-21T17:35:56.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.505+0000 7f434affd640 1 -- 172.21.15.53:0/2220136711 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340031d20 con 0x7f434c071c30 2023-10-21T17:35:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.506+0000 7f43523c0640 1 -- 172.21.15.53:0/2220136711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c071c30 msgr2=0x7f434c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.506+0000 7f43523c0640 1 --2- 172.21.15.53:0/2220136711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c071c30 0x7f434c069d50 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f43400097a0 tx=0x7f434002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.506+0000 7f43523c0640 1 -- 172.21.15.53:0/2220136711 shutdown_connections 2023-10-21T17:35:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.506+0000 7f43523c0640 1 --2- 172.21.15.53:0/2220136711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c071c30 0x7f434c069d50 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.506+0000 7f43523c0640 1 -- 172.21.15.53:0/2220136711 >> 172.21.15.53:0/2220136711 conn(0x7f434c06d430 msgr2=0x7f434c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:35:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.507+0000 7f43523c0640 1 -- 172.21.15.53:0/2220136711 shutdown_connections 2023-10-21T17:35:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.507+0000 7f43523c0640 1 -- 172.21.15.53:0/2220136711 wait complete. 2023-10-21T17:35:56.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.509+0000 7f43523c0640 1 Processor -- start 2023-10-21T17:35:56.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.509+0000 7f43523c0640 1 -- start start 2023-10-21T17:35:56.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f434c071c30 0x7f434c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f434c0720c0 0x7f434c1f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 0x7f434c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0c8470 con 0x7f434c1601b0 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c1f6ae0 con 0x7f434c071c30 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f43523c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0c69d0 con 0x7f434c0720c0 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f435095d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 0x7f434c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f435095d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 0x7f434c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45948/0 (socket says 172.21.15.53:45948) 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.511+0000 7f435095d640 1 -- 172.21.15.53:0/689268657 learned_addr learned my addr 172.21.15.53:0/689268657 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:35:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f434b7fe640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f434c0720c0 0x7f434c1f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f434c071c30 0x7f434c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:35:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f434c0720c0 msgr2=0x7f434c1f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f434c0720c0 0x7f434c1f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f434c071c30 msgr2=0x7f434c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:35:56.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f434c071c30 0x7f434c0b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:56.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009400 con 0x7f434c1601b0 2023-10-21T17:35:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f435095d640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 0x7f434c1585c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f434400ca80 tx=0x7f434400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:35:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.512+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400ac30 con 0x7f434c1601b0 2023-10-21T17:35:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.513+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400ad90 con 0x7f434c1601b0 2023-10-21T17:35:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.513+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c0c6c70 con 0x7f434c1601b0 2023-10-21T17:35:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.513+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344012600 con 0x7f434c1601b0 2023-10-21T17:35:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.513+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c0c8980 con 0x7f434c1601b0 2023-10-21T17:35:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.514+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c0b2cd0 con 0x7f434c1601b0 2023-10-21T17:35:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.515+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 115) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f4344007070 con 0x7f434c1601b0 2023-10-21T17:35:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.516+0000 7f4348ff9640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:35:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.516+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4344089820 con 0x7f434c1601b0 2023-10-21T17:35:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.516+0000 7f434bfff640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 msgr2=0x7f433806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.516+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:35:56.519 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.517+0000 7ff6ae82b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:35:56.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.520+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4344054610 con 0x7f434c1601b0 2023-10-21T17:35:56.548 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.547+0000 7f43abc52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:35:56.582 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.581+0000 7ff6ae82b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:35:56.633 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.632+0000 7ff6ae82b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:35:56.658 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.657+0000 7f43abc52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:35:56.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.717+0000 7f434bfff640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 msgr2=0x7f433806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:56.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.717+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:35:56.721 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.719+0000 7f43abc52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:56.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:56.733+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f434c161810 con 0x7f433806b010 2023-10-21T17:35:56.749 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.748+0000 7ff6ae82b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:35:56.769 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.767+0000 7f43abc52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:35:56.814 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.813+0000 7ff6ae82b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:56.825 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.824+0000 7f43abc52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:35:56.866 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.864+0000 7ff6ae82b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:35:56.876 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:56.874+0000 7f43abc52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:35:56.925 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.923+0000 7ff6ae82b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:35:56.977 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:56.976+0000 7ff6ae82b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:35:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:57.118+0000 7f434bfff640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 msgr2=0x7f433806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:57.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:57.118+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:35:57.256 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.254+0000 7f43abc52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:35:57.375 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.374+0000 7ff6ae82b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:35:57.412 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.410+0000 7f43abc52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:35:57.518 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.517+0000 7f43abc52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:35:57.542 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.541+0000 7ff6ae82b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:35:57.568 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.566+0000 7f43abc52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:35:57.655 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.653+0000 7ff6ae82b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:35:57.674 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.672+0000 7f43abc52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:57.707 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.705+0000 7ff6ae82b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:35:57.779 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.777+0000 7f43abc52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:35:57.818 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.817+0000 7ff6ae82b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:57.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:57.919+0000 7f434bfff640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 msgr2=0x7f433806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/248442090 2023-10-21T17:35:57.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:57.919+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:35:57.930 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:57.928+0000 7ff6ae82b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:35:57.933 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.931+0000 7f43abc52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:57.982 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:57.980+0000 7f43abc52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:58.093 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:58.092+0000 7ff6ae82b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:35:58.100 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:58.098+0000 7f43abc52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:35:58.144 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:58.143+0000 7ff6ae82b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:35:58.270 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:58.269+0000 7ff6ae82b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:35:58.926 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:58.924+0000 7f43abc52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:35:59.152 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:59.151+0000 7ff6ae82b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:35:59.222 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.221+0000 7f43abc52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:35:59.283 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.281+0000 7f43abc52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:35:59.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.395+0000 7f43abc52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:35:59.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:59.416+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 116) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f43440549e0 con 0x7f434c1601b0 2023-10-21T17:35:59.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:59.417+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 msgr2=0x7f433806d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:35:59.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:35:59.417+0000 7f4348ff9640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/248442090,v1:172.21.15.124:6833/248442090] conn(0x7f433806b010 0x7f433806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:35:59.471 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:59.470+0000 7ff6ae82b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:35:59.537 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:59.536+0000 7ff6ae82b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:35:59.610 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f432b79d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.610 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f432b79d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.611 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f432b79d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.611 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f432b79d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.611 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f432b79d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.611 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f4327f56640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.611 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f4327f56640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.612 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f4327f56640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.612 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f4327f56640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.612 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:35:59.609+0000 7f4327f56640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:35:59.665 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:35:59.664+0000 7ff6ae82b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:36:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.427+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 117) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4344087ae0 con 0x7f434c1601b0 2023-10-21T17:36:00.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.428+0000 7f4348ff9640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f4338035910 0x7f4338075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.428+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 --> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4338035cf0 con 0x7f4338035910 2023-10-21T17:36:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.435+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f4338035910 0x7f4338075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.441+0000 7f434bfff640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f4338035910 0x7f4338075560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f433c000f60 tx=0x7f433c001120 comp rx=0 tx=0).ready entity=mgr.37739 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.608+0000 7f4348ff9640 1 -- 172.21.15.53:0/689268657 <== mgr.37739 v2:172.21.15.124:6832/1168372736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4338035cf0 con 0x7f4338035910 2023-10-21T17:36:00.609 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T17:36:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f4338035910 msgr2=0x7f4338075560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f4338035910 0x7f4338075560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f433c000f60 tx=0x7f433c001120 comp rx=0 tx=0).stop 2023-10-21T17:36:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 msgr2=0x7f434c1585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 --2- 172.21.15.53:0/689268657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f434c1601b0 0x7f434c1585c0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f434400ca80 tx=0x7f434400cf40 comp rx=0 tx=0).stop 2023-10-21T17:36:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f435095d640 1 -- 172.21.15.53:0/689268657 reap_dead start 2023-10-21T17:36:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 shutdown_connections 2023-10-21T17:36:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.611+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 >> 172.21.15.53:0/689268657 conn(0x7f434c06d430 msgr2=0x7f434c0cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:36:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.612+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 shutdown_connections 2023-10-21T17:36:00.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.612+0000 7f43523c0640 1 -- 172.21.15.53:0/689268657 wait complete. 2023-10-21T17:36:00.636 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T17:36:00.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-21T17:36:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.861+0000 7f04e5476640 1 Processor -- start 2023-10-21T17:36:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.861+0000 7f04e5476640 1 -- start start 2023-10-21T17:36:00.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04e5476640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f04e006dfc0 0x7f04e0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04e5476640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00b1cc0 con 0x7f04e0071070 2023-10-21T17:36:00.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04e5476640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e00b2620 con 0x7f04e00afbe0 2023-10-21T17:36:00.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04e5476640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e00b2060 con 0x7f04e006dfc0 2023-10-21T17:36:00.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04df7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f04e0071070 0x7f04e00cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48030/0 (socket says 172.21.15.53:48030) 2023-10-21T17:36:00.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.863+0000 7f04df7fe640 1 -- 172.21.15.53:0/141030901 learned_addr learned my addr 172.21.15.53:0/141030901 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:36:00.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04d67fc640 1 --2- 172.21.15.53:0/141030901 >> v2:172.21.15.124:3300/0 conn(0x7f04e006dfc0 0x7f04e0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559030080 0 0) 0x7f04e00b1cc0 con 0x7f04e0071070 2023-10-21T17:36:00.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04cc003520 con 0x7f04e0071070 2023-10-21T17:36:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1751305157 0 0) 0x7f04cc003520 con 0x7f04e0071070 2023-10-21T17:36:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 >> v2:172.21.15.124:3300/0 conn(0x7f04e006dfc0 msgr2=0x7f04e0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:36:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 --2- 172.21.15.53:0/141030901 >> v2:172.21.15.124:3300/0 conn(0x7f04e006dfc0 0x7f04e0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 >> v1:172.21.15.124:6789/0 conn(0x7f04e00afbe0 legacy=0x7f04e00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00b2340 con 0x7f04e0071070 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f04d8001670 con 0x7f04e0071070 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.864+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 >> v1:172.21.15.53:6789/0 conn(0x7f04e0071070 legacy=0x7f04e00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 0x7f04cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04cc0047c0 0x7f04cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc005180 0x7f04cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00b2060 con 0x7f04cc0039c0 2023-10-21T17:36:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc00c0b0 con 0x7f04cc005180 2023-10-21T17:36:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc00c270 con 0x7f04cc0047c0 2023-10-21T17:36:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 0x7f04cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04d67fc640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04cc0047c0 0x7f04cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04df7fe640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc005180 0x7f04cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 -- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04cc0047c0 msgr2=0x7f04cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04cc0047c0 0x7f04cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 -- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc005180 msgr2=0x7f04cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc005180 0x7f04cc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04deffd640 1 -- 172.21.15.53:0/141030901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc003520 con 0x7f04cc0039c0 2023-10-21T17:36:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04df7fe640 1 -- 172.21.15.53:0/141030901 reap_dead start 2023-10-21T17:36:00.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.865+0000 7f04d67fc640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04cc0047c0 0x7f04cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:36:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.867+0000 7f04deffd640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 0x7f04cc003e00 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001750 tx=0x7f04cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738f867bd950d7a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.867+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d00052d0 con 0x7f04cc0039c0 2023-10-21T17:36:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.867+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d002e050 con 0x7f04cc0039c0 2023-10-21T17:36:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.867+0000 7f04de7fc640 1 -- 172.21.15.53:0/141030901 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0003830 con 0x7f04cc0039c0 2023-10-21T17:36:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 -- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 msgr2=0x7f04cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 0x7f04cc003e00 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001750 tx=0x7f04cc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 -- 172.21.15.53:0/141030901 shutdown_connections 2023-10-21T17:36:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 --2- 172.21.15.53:0/141030901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04cc0039c0 0x7f04cc003e00 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 -- 172.21.15.53:0/141030901 >> 172.21.15.53:0/141030901 conn(0x7f04e006a5d0 msgr2=0x7f04e00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:36:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 -- 172.21.15.53:0/141030901 shutdown_connections 2023-10-21T17:36:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.868+0000 7f04e5476640 1 -- 172.21.15.53:0/141030901 wait complete. 2023-10-21T17:36:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.870+0000 7f04e5476640 1 Processor -- start 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.870+0000 7f04e5476640 1 -- start start 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc0039c0 0x7f04e00c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04e00c2770 0x7f04e00c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00c2cf0 con 0x7f04e0155de0 2023-10-21T17:36:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00c2fd0 con 0x7f04cc0039c0 2023-10-21T17:36:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04e5476640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e00c32b0 con 0x7f04e00c2770 2023-10-21T17:36:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45966/0 (socket says 172.21.15.53:45966) 2023-10-21T17:36:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 -- 172.21.15.53:0/3985576967 learned_addr learned my addr 172.21.15.53:0/3985576967 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:36:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04deffd640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc0039c0 0x7f04e00c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 -- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04e00c2770 msgr2=0x7f04e00c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04e00c2770 0x7f04e00c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 -- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc0039c0 msgr2=0x7f04e00c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.871+0000 7f04d67fc640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc0039c0 0x7f04e00c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04d67fc640 1 -- 172.21.15.53:0/3985576967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e00c3590 con 0x7f04e0155de0 2023-10-21T17:36:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04d67fc640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009f90 tx=0x7f04c800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800fee0 con 0x7f04e0155de0 2023-10-21T17:36:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8016620 con 0x7f04e0155de0 2023-10-21T17:36:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80169b0 con 0x7f04e0155de0 2023-10-21T17:36:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e00c3870 con 0x7f04e0155de0 2023-10-21T17:36:00.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.872+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e0156260 con 0x7f04e0155de0 2023-10-21T17:36:00.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.873+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e004e160 con 0x7f04e0155de0 2023-10-21T17:36:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.874+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 117) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f04c8016b10 con 0x7f04e0155de0 2023-10-21T17:36:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.875+0000 7f04d7fff640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 0x7f04c006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.875+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f04c808a520 con 0x7f04e0155de0 2023-10-21T17:36:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.876+0000 7f04deffd640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 0x7f04c006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.877+0000 7f04deffd640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 0x7f04c006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f04d0033000 tx=0x7f04d0009030 comp rx=0 tx=0).ready entity=mgr.37739 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:00.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:00.880+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04c8055530 con 0x7f04e0155de0 2023-10-21T17:36:01.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.096+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 --> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f04e0070630 con 0x7f04c006ad40 2023-10-21T17:36:01.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.244+0000 7f04d7fff640 1 -- 172.21.15.53:0/3985576967 <== mgr.37739 v2:172.21.15.124:6832/1168372736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f04d0005480 con 0x7f04c006ad40 2023-10-21T17:36:01.246 INFO:teuthology.orchestra.run.smithi053.stdout:Retention added to path snap_test_dir1 2023-10-21T17:36:01.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 msgr2=0x7f04c006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 0x7f04c006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f04d0033000 tx=0x7f04d0009030 comp rx=0 tx=0).stop 2023-10-21T17:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 msgr2=0x7f04e00c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009f90 tx=0x7f04c800f520 comp rx=0 tx=0).stop 2023-10-21T17:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 shutdown_connections 2023-10-21T17:36:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f04c006ad40 0x7f04c006d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04e00c2770 0x7f04e00c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e0155de0 0x7f04e00c2230 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 --2- 172.21.15.53:0/3985576967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04cc0039c0 0x7f04e00c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 >> 172.21.15.53:0/3985576967 conn(0x7f04e006a5d0 msgr2=0x7f04e00cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:36:01.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 shutdown_connections 2023-10-21T17:36:01.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.249+0000 7f04e5476640 1 -- 172.21.15.53:0/3985576967 wait complete. 2023-10-21T17:36:01.275 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-21T17:36:01.276 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-21-17_37_00 in ~59s... 2023-10-21T17:36:01.276 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-21T17:36:01.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-21T17:36:01.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.508+0000 7fe322244640 1 Processor -- start 2023-10-21T17:36:01.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.509+0000 7fe322244640 1 -- start start 2023-10-21T17:36:01.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.510+0000 7fe322244640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe31c15ab10 0x7fe31c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.510+0000 7fe322244640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c0c9cd0 0x7fe31c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.510+0000 7fe322244640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31c069d50 con 0x7fe31c0c5bf0 2023-10-21T17:36:01.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.510+0000 7fe322244640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31c06b5a0 con 0x7fe31c15ab10 2023-10-21T17:36:01.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.510+0000 7fe322244640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c15aef0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe31c15ab10 0x7fe31c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c0c9cd0 0x7fe31c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe31c15ab10 0x7fe31c0cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:59042/0 (socket says 172.21.15.53:59042) 2023-10-21T17:36:01.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 -- 172.21.15.53:0/3492893122 learned_addr learned my addr 172.21.15.53:0/3492893122 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:36:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 -- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c0c9cd0 msgr2=0x7fe31c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:36:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 --2- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c0c9cd0 0x7fe31c06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 -- 172.21.15.53:0/3492893122 >> v1:172.21.15.124:6789/0 conn(0x7fe31c0c5bf0 legacy=0x7fe31c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:36:01.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.511+0000 7fe31b7fe640 1 -- 172.21.15.53:0/3492893122 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c069f80 con 0x7fe31c15ab10 2023-10-21T17:36:01.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.513+0000 7fe31b7fe640 1 --2- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c15ab10 0x7fe31c0cb520 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009870 tx=0x7fe30c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e685abfa89a8a481 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:01.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.513+0000 7fe31affd640 1 -- 172.21.15.53:0/3492893122 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0317c0 con 0x7fe31c15ab10 2023-10-21T17:36:01.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.513+0000 7fe31affd640 1 -- 172.21.15.53:0/3492893122 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c031920 con 0x7fe31c15ab10 2023-10-21T17:36:01.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.513+0000 7fe31affd640 1 -- 172.21.15.53:0/3492893122 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c031cb0 con 0x7fe31c15ab10 2023-10-21T17:36:01.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 -- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c15ab10 msgr2=0x7fe31c0cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 --2- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c15ab10 0x7fe31c0cb520 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009870 tx=0x7fe30c02b860 comp rx=0 tx=0).stop 2023-10-21T17:36:01.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 -- 172.21.15.53:0/3492893122 shutdown_connections 2023-10-21T17:36:01.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 --2- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c0c9cd0 0x7fe31c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 --2- 172.21.15.53:0/3492893122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c15ab10 0x7fe31c0cb520 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 -- 172.21.15.53:0/3492893122 >> 172.21.15.53:0/3492893122 conn(0x7fe31c0b0af0 msgr2=0x7fe31c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:36:01.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 -- 172.21.15.53:0/3492893122 shutdown_connections 2023-10-21T17:36:01.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.514+0000 7fe322244640 1 -- 172.21.15.53:0/3492893122 wait complete. 2023-10-21T17:36:01.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.516+0000 7fe322244640 1 Processor -- start 2023-10-21T17:36:01.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.517+0000 7fe322244640 1 -- start start 2023-10-21T17:36:01.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c0c5bf0 0x7fe31c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c15ab10 0x7fe31c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c158de0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ce130 con 0x7fe31c0c5bf0 2023-10-21T17:36:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe322244640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c158510 con 0x7fe31c15ab10 2023-10-21T17:36:01.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe313fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe313fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45982/0 (socket says 172.21.15.53:45982) 2023-10-21T17:36:01.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe313fff640 1 -- 172.21.15.53:0/1759227662 learned_addr learned my addr 172.21.15.53:0/1759227662 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:36:01.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe31b7fe640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c0c5bf0 0x7fe31c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.518+0000 7fe31bfff640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c15ab10 0x7fe31c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 -- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c15ab10 msgr2=0x7fe31c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c15ab10 0x7fe31c156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 -- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c0c5bf0 msgr2=0x7fe31c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c0c5bf0 0x7fe31c0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 -- 172.21.15.53:0/1759227662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0094d0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe313fff640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c430 tx=0x7fe30400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304011790 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3040118f0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.519+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c157470 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.520+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304011ce0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.520+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c157d10 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.521+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c0caed0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.522+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 118) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fe30401c460 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.523+0000 7fe318ff9640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 0x7fe2fc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:36:01.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.523+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fe3040899b0 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.523+0000 7fe31b7fe640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 0x7fe2fc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:36:01.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.525+0000 7fe31b7fe640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 0x7fe2fc06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe30c036000 tx=0x7fe30c03d040 comp rx=0 tx=0).ready entity=mgr.37739 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:36:01.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.527+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe304054630 con 0x7fe31c0c9cd0 2023-10-21T17:36:01.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.748+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 --> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe31c06ac70 con 0x7fe2fc06afe0 2023-10-21T17:36:01.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.764+0000 7fe318ff9640 1 -- 172.21.15.53:0/1759227662 <== mgr.37739 v2:172.21.15.124:6832/1168372736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fe30c031a90 con 0x7fe2fc06afe0 2023-10-21T17:36:01.766 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:36:01.766 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-21T17:36:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 msgr2=0x7fe2fc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 0x7fe2fc06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe30c036000 tx=0x7fe30c03d040 comp rx=0 tx=0).stop 2023-10-21T17:36:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 msgr2=0x7fe31c0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:36:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c430 tx=0x7fe30400c8f0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 shutdown_connections 2023-10-21T17:36:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe2fc06afe0 0x7fe2fc06d4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe31c15ab10 0x7fe31c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe31c0c9cd0 0x7fe31c0ad980 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 --2- 172.21.15.53:0/1759227662 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe31c0c5bf0 0x7fe31c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:36:01.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 >> 172.21.15.53:0/1759227662 conn(0x7fe31c0b0af0 msgr2=0x7fe31c0ac770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:36:01.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.767+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 shutdown_connections 2023-10-21T17:36:01.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:36:01.768+0000 7fe322244640 1 -- 172.21.15.53:0/1759227662 wait complete. 2023-10-21T17:36:01.789 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-21T17:36:01.789 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-21T17:36:01.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:01.790 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:01.848 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:01.849 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:01.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:02.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:02.851 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:02.917 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:02.917 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:02.918 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:03.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:03.919 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:03.988 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:03.988 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:03.989 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:04.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:04.991 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:05.056 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:05.056 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:05.056 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:06.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:06.058 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:06.126 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:06.126 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:06.126 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:07.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:07.129 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:07.196 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:07.196 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:07.196 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:08.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:08.198 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:08.266 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:08.266 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:08.267 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:09.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:09.269 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:09.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:09.334 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:09.334 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:10.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:10.336 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:10.403 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:10.404 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:10.404 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:11.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:11.405 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:11.472 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:11.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:11.472 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:12.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:12.474 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:12.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:12.542 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:12.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:13.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:13.543 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:13.610 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:13.611 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:13.611 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:14.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:14.612 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:14.679 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:14.679 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:14.680 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:15.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:15.681 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:15.748 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:15.748 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:15.749 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:16.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:16.750 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:16.816 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:16.816 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:16.816 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:17.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:17.819 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:17.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:17.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:17.886 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:18.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:18.887 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:18.956 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:18.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:18.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:19.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:36:19.124 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:36:19.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:19.958 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:20.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:20.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:20.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:21.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:21.029 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:21.095 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:21.096 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:21.096 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:22.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:22.098 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:22.165 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:22.166 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:22.166 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:23.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:23.168 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:23.237 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:23.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:23.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:24.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:24.239 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:24.306 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:24.306 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:24.307 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:25.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:25.309 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:25.373 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:25.373 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:25.374 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:26.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:26.375 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:26.442 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:26.442 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:26.443 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:27.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:27.445 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:27.512 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:27.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:27.513 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:28.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:28.515 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:28.580 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:28.580 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:28.581 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:29.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:29.582 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:29.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:29.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:29.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:30.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:30.651 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:30.719 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:30.720 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:30.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:31.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:31.722 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:31.783 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:31.783 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:31.784 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:32.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:32.785 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:32.851 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:32.851 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:32.851 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:33.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:33.853 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:33.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:33.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:33.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:34.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:34.923 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:34.988 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:34.988 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:34.988 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:35.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:35.990 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:36.057 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:36.057 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:36.058 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:37.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:37.059 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:37.126 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:37.126 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:37.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:38.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:38.128 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:38.195 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:38.196 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:38.196 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:39.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:39.197 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:39.264 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:39.265 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:39.265 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:40.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:40.266 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:40.329 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:40.330 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:40.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:41.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:41.332 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:41.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:41.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:41.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:42.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:42.403 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:42.469 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:42.470 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:42.470 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:43.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:43.472 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:43.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:43.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:43.541 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:44.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:44.543 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:44.608 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:44.608 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:44.609 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:45.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:45.611 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:45.678 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:45.678 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:45.679 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:46.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:46.680 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:46.749 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:46.749 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:46.750 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:47.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:47.751 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:47.817 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:47.818 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:47.818 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:48.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:48.820 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:48.888 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:48.888 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:48.889 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:49.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:36:49.296 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:36:49.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:49.891 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:49.958 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:49.958 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:49.959 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:50.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:50.960 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:51.027 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:51.028 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:51.028 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:52.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:52.030 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:52.097 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:52.168 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:52.168 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:53.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:53.170 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:53.236 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:53.236 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:53.237 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:54.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:54.238 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:54.306 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:54.307 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:54.307 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:55.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:55.308 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:55.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:55.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:55.379 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:56.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:56.380 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:56.447 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:56.447 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:56.448 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:57.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:57.449 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:57.517 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:57.517 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:57.517 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:58.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:58.519 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:58.585 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:58.585 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:58.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:36:59.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:36:59.587 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:36:59.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-21T17:36:59.655 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:36:59.656 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:00.269 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:37:00.268+0000 7f431a73b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:37:00.270 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:37:00.268+0000 7f431a73b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:37:00.270 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:37:00.268+0000 7f431a73b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:37:00.270 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:37:00.268+0000 7f431a73b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:37:00.270 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:37:00.268+0000 7f431a73b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:37:00.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:00.657 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:00.723 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:00.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:00.724 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:00.725 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:00.725 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-21-17_37_00_UTC'] 2023-10-21T17:37:00.725 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697909761.27619, now = 1697909820.7256231, timo = 59 2023-10-21T17:37:01.727 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-21-17_37_00 in ~119s... 2023-10-21T17:37:01.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:01.728 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:01.793 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:01.794 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:01.795 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:01.795 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:02.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:02.797 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:02.863 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:02.865 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:02.865 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:02.865 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:03.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:03.867 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:03.934 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:03.935 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:03.936 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:03.936 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:04.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:04.937 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:05.004 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:05.005 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:05.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:05.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:06.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:06.008 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:06.069 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:06.070 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:06.070 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:06.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:07.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:07.072 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:07.137 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:07.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:07.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:07.139 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:08.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:08.141 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:08.208 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:08.209 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:08.209 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:08.210 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:09.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:09.211 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:09.277 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:09.278 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:09.279 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:09.279 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:10.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:10.281 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:10.344 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:10.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:10.346 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:10.346 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:11.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:11.348 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:11.414 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:11.415 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:11.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:11.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:12.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:12.419 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:12.485 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:12.486 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:12.486 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:12.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:13.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:13.489 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:13.556 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:13.557 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:13.557 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:13.558 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:14.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:14.559 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:14.625 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:14.626 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:14.626 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:14.627 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:15.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:15.629 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:15.695 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:15.696 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:15.697 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:15.697 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:16.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:16.699 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:16.763 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:16.764 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:16.764 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:16.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:17.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:17.767 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:17.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:17.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:17.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:18.837 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:18.904 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:18.905 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:18.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:18.906 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:19.502 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:37:19.512 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:37:19.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:19.908 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:19.974 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:19.975 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:19.976 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:19.976 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:20.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:20.978 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:21.045 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:21.046 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:21.046 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:21.047 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:22.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:22.049 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:22.115 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:22.117 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:22.117 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:22.118 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:23.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:23.120 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:23.183 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:23.184 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:23.185 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:23.185 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:24.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:24.186 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:24.252 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:24.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:24.254 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:24.255 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:25.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:25.256 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:25.322 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:25.323 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:25.324 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:25.324 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:26.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:26.326 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:26.392 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:26.393 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:26.393 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:26.394 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:27.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:27.395 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:27.461 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:27.463 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:27.463 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:27.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:28.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:28.465 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:28.530 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:28.531 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:28.531 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:28.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:29.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:29.533 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:29.596 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:29.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:29.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:29.598 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:30.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:30.599 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:30.664 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:30.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:30.666 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:30.666 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:31.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:31.668 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:31.734 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:31.735 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:31.736 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:31.736 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:32.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:32.738 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:32.802 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:32.803 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:32.803 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:32.803 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:33.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:33.805 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:33.871 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:33.873 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:33.873 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:33.874 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:34.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:34.875 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:34.942 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:34.943 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:34.943 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:34.944 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:35.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:35.946 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:36.011 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:36.013 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:36.013 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:36.013 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:37.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:37.016 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:37.082 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:37.083 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:37.083 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:37.084 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:38.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:38.086 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:38.149 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:38.151 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:38.151 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:38.151 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:39.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:39.153 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:39.220 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:39.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:39.222 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:39.222 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:40.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:40.224 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:40.289 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:40.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:40.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:40.291 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:41.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:41.293 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:41.358 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:41.360 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:41.360 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:41.360 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:42.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:42.362 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:42.428 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:42.429 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:42.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:42.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:43.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:43.432 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:43.497 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:43.498 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:43.498 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:43.499 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:44.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:44.500 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:44.568 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:44.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:44.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:44.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:45.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:45.571 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:45.638 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:45.639 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:45.639 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:45.640 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:46.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:46.642 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:46.707 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:46.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:46.709 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:46.709 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:47.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:47.711 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:47.776 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:47.777 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:47.777 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:47.778 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:48.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:48.780 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:48.846 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:48.847 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:48.847 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:48.847 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:49.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:37:49.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:37:49.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:49.849 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:49.958 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:49.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:49.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:49.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:50.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:50.961 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:51.027 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:51.028 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:51.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:51.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:52.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:52.031 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:52.097 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:52.099 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:52.099 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:52.099 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:53.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:53.101 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:53.167 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:53.168 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:53.169 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:53.169 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:54.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:54.171 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:54.237 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:54.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:54.239 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:54.239 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:55.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:55.241 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:55.307 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:55.308 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:55.309 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:55.309 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:56.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:56.311 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:56.376 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:56.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:56.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:56.379 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:57.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:57.380 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:57.446 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:57.447 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:57.447 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:57.447 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:58.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:58.449 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:58.515 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:58.516 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:58.516 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:58.517 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:37:59.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:37:59.518 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:37:59.586 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_37_00_UTC 2023-10-21T17:37:59.587 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:37:59.587 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-21T17:37:59.588 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-21T17:38:00.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:00.589 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:38:00.653 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_38_00_UTC 2023-10-21T17:38:00.654 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-21-17_38_00_UTC'} 2023-10-21T17:38:00.654 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-21-17_38_00_UTC'} 2023-10-21T17:38:00.654 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-21-17_37_00_UTC'} 2023-10-21T17:38:00.655 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-21-17_37_00_UTC'] 2023-10-21T17:38:00.655 DEBUG:tasks.cephfs.test_snap_schedules:exec=1697909761.27619, now = 1697909880.655598, timo = 119 2023-10-21T17:38:01.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-21T17:38:01.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.900+0000 7f0813548640 1 Processor -- start 2023-10-21T17:38:01.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.900+0000 7f0813548640 1 -- start start 2023-10-21T17:38:01.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.901+0000 7f0813548640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f080c15cd80 0x7f080c0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.901+0000 7f0813548640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f080c0b6730 0x7f080c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f0813548640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c15d160 con 0x7f080c0cdda0 2023-10-21T17:38:01.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f0813548640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c0b7b90 con 0x7f080c15cd80 2023-10-21T17:38:01.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f0813548640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c0ce1c0 con 0x7f080c0b6730 2023-10-21T17:38:01.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f080c15cd80 0x7f080c0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:01.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f080c15cd80 0x7f080c0b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47990/0 (socket says 172.21.15.53:47990) 2023-10-21T17:38:01.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 -- 172.21.15.53:0/3041322600 learned_addr learned my addr 172.21.15.53:0/3041322600 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:01.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 -- 172.21.15.53:0/3041322600 >> v2:172.21.15.124:3300/0 conn(0x7f080c0b6730 msgr2=0x7f080c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:01.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 --2- 172.21.15.53:0/3041322600 >> v2:172.21.15.124:3300/0 conn(0x7f080c0b6730 0x7f080c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 -- 172.21.15.53:0/3041322600 >> v1:172.21.15.124:6789/0 conn(0x7f080c0cdda0 legacy=0x7f080c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:01.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.902+0000 7f08112e4640 1 -- 172.21.15.53:0/3041322600 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c0b8020 con 0x7f080c15cd80 2023-10-21T17:38:01.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.903+0000 7f08112e4640 1 --2- 172.21.15.53:0/3041322600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c15cd80 0x7f080c0b87d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f080c160750 tx=0x7f07fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335c4ca193e5832a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:01.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f080bfff640 1 -- 172.21.15.53:0/3041322600 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc02ca80 con 0x7f080c15cd80 2023-10-21T17:38:01.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f080bfff640 1 -- 172.21.15.53:0/3041322600 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc02cbe0 con 0x7f080c15cd80 2023-10-21T17:38:01.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f080bfff640 1 -- 172.21.15.53:0/3041322600 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0343f0 con 0x7f080c15cd80 2023-10-21T17:38:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f0813548640 1 -- 172.21.15.53:0/3041322600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c15cd80 msgr2=0x7f080c0b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f0813548640 1 --2- 172.21.15.53:0/3041322600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c15cd80 0x7f080c0b87d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f080c160750 tx=0x7f07fc02bee0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f0813548640 1 -- 172.21.15.53:0/3041322600 shutdown_connections 2023-10-21T17:38:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.904+0000 7f0813548640 1 --2- 172.21.15.53:0/3041322600 >> v2:172.21.15.124:3300/0 conn(0x7f080c0b6730 0x7f080c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.905+0000 7f0813548640 1 --2- 172.21.15.53:0/3041322600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c15cd80 0x7f080c0b87d0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.905+0000 7f0813548640 1 -- 172.21.15.53:0/3041322600 >> 172.21.15.53:0/3041322600 conn(0x7f080c06b5d0 msgr2=0x7f080c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:01.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.905+0000 7f0813548640 1 -- 172.21.15.53:0/3041322600 shutdown_connections 2023-10-21T17:38:01.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.905+0000 7f0813548640 1 -- 172.21.15.53:0/3041322600 wait complete. 2023-10-21T17:38:01.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.906+0000 7f0813548640 1 Processor -- start 2023-10-21T17:38:01.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.907+0000 7f0813548640 1 -- start start 2023-10-21T17:38:01.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f080c0cdda0 0x7f080c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f080c15cd80 0x7f080c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c15dce0 con 0x7f080c0b6730 2023-10-21T17:38:01.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c15dfc0 con 0x7f080c0cdda0 2023-10-21T17:38:01.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f0813548640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c15e2a0 con 0x7f080c15cd80 2023-10-21T17:38:01.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:01.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47998/0 (socket says 172.21.15.53:47998) 2023-10-21T17:38:01.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 -- 172.21.15.53:0/3876680852 learned_addr learned my addr 172.21.15.53:0/3876680852 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:01.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 -- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f080c15cd80 msgr2=0x7f080c15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:01.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f080c15cd80 0x7f080c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 -- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f080c0cdda0 msgr2=0x7f080c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:01.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f080c0cdda0 0x7f080c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:01.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.908+0000 7f08112e4640 1 -- 172.21.15.53:0/3876680852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc009b30 con 0x7f080c0b6730 2023-10-21T17:38:01.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f08112e4640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0093e0 tx=0x7f07fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:01.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc03e450 con 0x7f080c0b6730 2023-10-21T17:38:01.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc03e5b0 con 0x7f080c0b6730 2023-10-21T17:38:01.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc03e960 con 0x7f080c0b6730 2023-10-21T17:38:01.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c15e580 con 0x7f080c0b6730 2023-10-21T17:38:01.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.909+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c14fa80 con 0x7f080c0b6730 2023-10-21T17:38:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.911+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c15eaa0 con 0x7f080c0b6730 2023-10-21T17:38:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.911+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 121) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f07fc02cd50 con 0x7f080c0b6730 2023-10-21T17:38:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.912+0000 7f0809ffb640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 0x7f07f806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:01.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.913+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6634+0+0 (secure 0 0 0) 0x7f07fc0ac860 con 0x7f080c0b6730 2023-10-21T17:38:01.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.913+0000 7f0810ae3640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 0x7f07f806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:01.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.914+0000 7f0810ae3640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 0x7f07f806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f080c153a50 tx=0x7f0800006c90 comp rx=0 tx=0).ready entity=mgr.37739 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:01.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:01.917+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f080c15eaa0 con 0x7f080c0b6730 2023-10-21T17:38:02.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.131+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 --> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f080c0b7260 con 0x7f07f806afe0 2023-10-21T17:38:02.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.293+0000 7f0809ffb640 1 -- 172.21.15.53:0/3876680852 <== mgr.37739 v2:172.21.15.124:6832/1168372736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0800013400 con 0x7f07f806afe0 2023-10-21T17:38:02.295 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path snap_test_dir1 2023-10-21T17:38:02.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 msgr2=0x7f07f806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 0x7f07f806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f080c153a50 tx=0x7f0800006c90 comp rx=0 tx=0).stop 2023-10-21T17:38:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 msgr2=0x7f080c14f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0093e0 tx=0x7f07fc004a40 comp rx=0 tx=0).stop 2023-10-21T17:38:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 shutdown_connections 2023-10-21T17:38:02.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f07f806afe0 0x7f07f806d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f080c15cd80 0x7f080c15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f080c0cdda0 0x7f080c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 --2- 172.21.15.53:0/3876680852 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f080c0b6730 0x7f080c14f540 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 >> 172.21.15.53:0/3876680852 conn(0x7f080c06b5d0 msgr2=0x7f080c0b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:02.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 shutdown_connections 2023-10-21T17:38:02.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.297+0000 7f0813548640 1 -- 172.21.15.53:0/3876680852 wait complete. 2023-10-21T17:38:02.319 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-21T17:38:02.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:02.320 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:38:02.376 INFO:teuthology.orchestra.run.smithi053.stdout:scheduled-2023-10-21-17_38_00_UTC 2023-10-21T17:38:02.378 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-21-17_38_00_UTC 2023-10-21T17:38:02.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:02.378 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-21-17_38_00_UTC) 2023-10-21T17:38:02.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:02.449 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T17:38:02.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:38:02.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.739+0000 7f629da2b640 1 Processor -- start 2023-10-21T17:38:02.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.739+0000 7f629da2b640 1 -- start start 2023-10-21T17:38:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f629da2b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f62981601b0 0x7f62980cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f629da2b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f629806a460 0x7f62980ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f629da2b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62980ce210 con 0x7f6298071c30 2023-10-21T17:38:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f629da2b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980cdee0 con 0x7f629806a460 2023-10-21T17:38:02.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f629da2b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6298161530 con 0x7f62981601b0 2023-10-21T17:38:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f62977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f629806a460 0x7f62980ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f62977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f629806a460 0x7f62980ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48014/0 (socket says 172.21.15.53:48014) 2023-10-21T17:38:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.741+0000 7f62977fe640 1 -- 172.21.15.53:0/102325419 learned_addr learned my addr 172.21.15.53:0/102325419 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.742+0000 7f62977fe640 1 -- 172.21.15.53:0/102325419 >> v2:172.21.15.124:3300/0 conn(0x7f62981601b0 msgr2=0x7f62980cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:02.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.742+0000 7f62977fe640 1 --2- 172.21.15.53:0/102325419 >> v2:172.21.15.124:3300/0 conn(0x7f62981601b0 0x7f62980cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.742+0000 7f62977fe640 1 -- 172.21.15.53:0/102325419 >> v1:172.21.15.124:6789/0 conn(0x7f6298071c30 legacy=0x7f6298069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.742+0000 7f62977fe640 1 -- 172.21.15.53:0/102325419 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298161810 con 0x7f629806a460 2023-10-21T17:38:02.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.743+0000 7f62977fe640 1 --2- 172.21.15.53:0/102325419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f629806a460 0x7f62980ca090 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f6298067110 tx=0x7f629002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f349c27491d79edd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:02.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.743+0000 7f6295ffb640 1 -- 172.21.15.53:0/102325419 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290031870 con 0x7f629806a460 2023-10-21T17:38:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.743+0000 7f6295ffb640 1 -- 172.21.15.53:0/102325419 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62900319d0 con 0x7f629806a460 2023-10-21T17:38:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.743+0000 7f6295ffb640 1 -- 172.21.15.53:0/102325419 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290031d60 con 0x7f629806a460 2023-10-21T17:38:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 -- 172.21.15.53:0/102325419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f629806a460 msgr2=0x7f62980ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 --2- 172.21.15.53:0/102325419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f629806a460 0x7f62980ca090 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f6298067110 tx=0x7f629002b910 comp rx=0 tx=0).stop 2023-10-21T17:38:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 -- 172.21.15.53:0/102325419 shutdown_connections 2023-10-21T17:38:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 --2- 172.21.15.53:0/102325419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f629806a460 0x7f62980ca090 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 --2- 172.21.15.53:0/102325419 >> v2:172.21.15.124:3300/0 conn(0x7f62981601b0 0x7f62980cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 -- 172.21.15.53:0/102325419 >> 172.21.15.53:0/102325419 conn(0x7f629806d430 msgr2=0x7f629806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 -- 172.21.15.53:0/102325419 shutdown_connections 2023-10-21T17:38:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.744+0000 7f629da2b640 1 -- 172.21.15.53:0/102325419 wait complete. 2023-10-21T17:38:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.746+0000 7f629da2b640 1 Processor -- start 2023-10-21T17:38:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.746+0000 7f629da2b640 1 -- start start 2023-10-21T17:38:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f629806a460 0x7f6298159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62981601b0 0x7f62980cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c5740 con 0x7f6298071c30 2023-10-21T17:38:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c4870 con 0x7f629806a460 2023-10-21T17:38:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f629da2b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6298159790 con 0x7f62981601b0 2023-10-21T17:38:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f62967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f62967fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48030/0 (socket says 172.21.15.53:48030) 2023-10-21T17:38:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.747+0000 7f62967fc640 1 -- 172.21.15.53:0/889509568 learned_addr learned my addr 172.21.15.53:0/889509568 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f6296ffd640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f629806a460 0x7f6298159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62977fe640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62981601b0 0x7f62980cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 -- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62981601b0 msgr2=0x7f62980cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62981601b0 0x7f62980cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 -- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f629806a460 msgr2=0x7f6298159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f629806a460 0x7f6298159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 -- 172.21.15.53:0/889509568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900094d0 con 0x7f6298071c30 2023-10-21T17:38:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.748+0000 7f62967fc640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f628800c7a0 tx=0x7f628800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.749+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288003d80 con 0x7f6298071c30 2023-10-21T17:38:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.749+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288005070 con 0x7f6298071c30 2023-10-21T17:38:02.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.749+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62980c4ac0 con 0x7f6298071c30 2023-10-21T17:38:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.749+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005420 con 0x7f6298071c30 2023-10-21T17:38:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.749+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62980c5080 con 0x7f6298071c30 2023-10-21T17:38:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.750+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62980c5880 con 0x7f6298071c30 2023-10-21T17:38:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.751+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 121) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f6288005580 con 0x7f6298071c30 2023-10-21T17:38:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.753+0000 7f628ffff640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 0x7f627806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.753+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6634+0+0 (secure 0 0 0) 0x7f62880890c0 con 0x7f6298071c30 2023-10-21T17:38:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.754+0000 7f6296ffd640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 0x7f627806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.755+0000 7f6296ffd640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 0x7f627806d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f628000b810 tx=0x7f6280008040 comp rx=0 tx=0).ready entity=mgr.37739 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:02.757+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62980c5880 con 0x7f6298071c30 2023-10-21T17:38:03.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.005+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f62980c5880 con 0x7f6298071c30 2023-10-21T17:38:03.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.463+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v122) v1 ==== 93+0+0 (secure 0 0 0) 0x7f62980c5880 con 0x7f6298071c30 2023-10-21T17:38:03.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.468+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 msgr2=0x7f627806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:03.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.468+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 0x7f627806d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f628000b810 tx=0x7f6280008040 comp rx=0 tx=0).stop 2023-10-21T17:38:03.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.468+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 msgr2=0x7f62980c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:03.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f628ffff640 1 -- 172.21.15.53:0/889509568 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f6288053ff0 con 0x7f6298071c30 2023-10-21T17:38:03.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f628800c7a0 tx=0x7f628800cc60 comp rx=0 tx=0).stop 2023-10-21T17:38:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 shutdown_connections 2023-10-21T17:38:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f627806afe0 0x7f627806d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62981601b0 0x7f62980cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6298071c30 0x7f62980c5b60 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 --2- 172.21.15.53:0/889509568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f629806a460 0x7f6298159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.469+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 >> 172.21.15.53:0/889509568 conn(0x7f629806d430 msgr2=0x7f6298069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.470+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 shutdown_connections 2023-10-21T17:38:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:03.470+0000 7f629da2b640 1 -- 172.21.15.53:0/889509568 wait complete. 2023-10-21T17:38:03.493 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:38:03.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:38:03.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:38:03.706 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:38:03.707 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:03.707 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:03.764 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:03.764 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:03.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:04.015 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.013+0000 7f547a353040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:38:04.016 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.015+0000 7ff678a52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:38:04.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.025+0000 7fe99e07c640 1 Processor -- start 2023-10-21T17:38:04.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.025+0000 7fe99e07c640 1 -- start start 2023-10-21T17:38:04.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe99e07c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe99e07c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99815b2b0 con 0x7fe9980cb580 2023-10-21T17:38:04.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe99e07c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99815b590 con 0x7fe9980c90b0 2023-10-21T17:38:04.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe99e07c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99815bf30 con 0x7fe99815ab40 2023-10-21T17:38:04.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe997fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe9980cb580 0x7fe99815e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48634/0 (socket says 172.21.15.53:48634) 2023-10-21T17:38:04.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe997fff640 1 -- 172.21.15.53:0/587877223 learned_addr learned my addr 172.21.15.53:0/587877223 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:04.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.027+0000 7fe9977fe640 1 --2- 172.21.15.53:0/587877223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743928034 0 0) 0x7fe99815b2b0 con 0x7fe9980cb580 2023-10-21T17:38:04.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe984003520 con 0x7fe9980cb580 2023-10-21T17:38:04.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186092122 0 0) 0x7fe984003520 con 0x7fe9980cb580 2023-10-21T17:38:04.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe9977fe640 1 -- 172.21.15.53:0/587877223 >> v1:172.21.15.124:6789/0 conn(0x7fe9980c90b0 legacy=0x7fe9980cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe9977fe640 1 -- 172.21.15.53:0/587877223 >> v1:172.21.15.53:6789/0 conn(0x7fe9980cb580 legacy=0x7fe99815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.028+0000 7fe9977fe640 1 -- 172.21.15.53:0/587877223 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99815b870 con 0x7fe99815ab40 2023-10-21T17:38:04.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.029+0000 7fe9977fe640 1 --2- 172.21.15.53:0/587877223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980ad720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9880097d0 tx=0x7fe98802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6996565c747cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.030+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900019f0 con 0x7fe99815ab40 2023-10-21T17:38:04.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.030+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98802e050 con 0x7fe99815ab40 2023-10-21T17:38:04.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.030+0000 7fe996ffd640 1 -- 172.21.15.53:0/587877223 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988030710 con 0x7fe99815ab40 2023-10-21T17:38:04.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.030+0000 7fe99e07c640 1 -- 172.21.15.53:0/587877223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 msgr2=0x7fe9980ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.030+0000 7fe99e07c640 1 --2- 172.21.15.53:0/587877223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980ad720 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe9880097d0 tx=0x7fe98802b800 comp rx=0 tx=0).stop 2023-10-21T17:38:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.031+0000 7fe99e07c640 1 -- 172.21.15.53:0/587877223 shutdown_connections 2023-10-21T17:38:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.031+0000 7fe99e07c640 1 --2- 172.21.15.53:0/587877223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980ad720 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.031+0000 7fe99e07c640 1 -- 172.21.15.53:0/587877223 >> 172.21.15.53:0/587877223 conn(0x7fe99806a380 msgr2=0x7fe9980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.031+0000 7fe99e07c640 1 -- 172.21.15.53:0/587877223 shutdown_connections 2023-10-21T17:38:04.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.031+0000 7fe99e07c640 1 -- 172.21.15.53:0/587877223 wait complete. 2023-10-21T17:38:04.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.032+0000 7fe99e07c640 1 Processor -- start 2023-10-21T17:38:04.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.032+0000 7fe99e07c640 1 -- start start 2023-10-21T17:38:04.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9980cb580 0x7fe9980c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9980c87b0 con 0x7fe9980c90b0 2023-10-21T17:38:04.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe998141660 con 0x7fe9980cb580 2023-10-21T17:38:04.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.033+0000 7fe99e07c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe998141940 con 0x7fe99815ab40 2023-10-21T17:38:04.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45656/0 (socket says 172.21.15.53:45656) 2023-10-21T17:38:04.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 -- 172.21.15.53:0/2066747856 learned_addr learned my addr 172.21.15.53:0/2066747856 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:04.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe98ffff640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9980cb580 0x7fe9980c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe997fff640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 msgr2=0x7fe9980c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9980cb580 msgr2=0x7fe9980c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9980cb580 0x7fe9980c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.034+0000 7fe9977fe640 1 -- 172.21.15.53:0/2066747856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009400 con 0x7fe9980c90b0 2023-10-21T17:38:04.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe9977fe640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe98802b840 tx=0x7fe9880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988030710 con 0x7fe9980c90b0 2023-10-21T17:38:04.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe988030870 con 0x7fe9980c90b0 2023-10-21T17:38:04.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988030be0 con 0x7fe9980c90b0 2023-10-21T17:38:04.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998141c20 con 0x7fe9980c90b0 2023-10-21T17:38:04.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.035+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9980c5af0 con 0x7fe9980c90b0 2023-10-21T17:38:04.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.036+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe998067490 con 0x7fe9980c90b0 2023-10-21T17:38:04.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.037+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe988030d40 con 0x7fe9980c90b0 2023-10-21T17:38:04.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.038+0000 7fe994ff9640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 0x7fe97806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.038+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6634+0+0 (secure 0 0 0) 0x7fe9880ac0a0 con 0x7fe9980c90b0 2023-10-21T17:38:04.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.038+0000 7fe98ffff640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 msgr2=0x7fe97806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:04.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.038+0000 7fe98ffff640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 0x7fe97806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:04.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.042+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe988076ca0 con 0x7fe9980c90b0 2023-10-21T17:38:04.071 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.069+0000 7f547a353040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:38:04.093 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.091+0000 7ff678a52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:38:04.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.239+0000 7fe98ffff640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 msgr2=0x7fe97806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:04.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.239+0000 7fe98ffff640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 0x7fe97806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:04.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.252+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9980add30 con 0x7fe9980c90b0 2023-10-21T17:38:04.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.254+0000 7fe994ff9640 1 -- 172.21.15.53:0/2066747856 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+17634 (secure 0 0 0) 0x7fe98807a2c0 con 0x7fe9980c90b0 2023-10-21T17:38:04.256 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:04.256 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":218,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:37:01.449055+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.18","mappings":[{"from":3,"to":6}]},{"pgid":"26.23","mappings":[{"from":1,"to":6}]},{"pgid":"26.28","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:6832/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/2759452384":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/863767263":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1944369314":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1803654246":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/820409426":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/102947359":"2023-10-22T17:35:59.400288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:04.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 msgr2=0x7fe97806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 0x7fe97806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 msgr2=0x7fe9980c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe98802b840 tx=0x7fe9880046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 shutdown_connections 2023-10-21T17:38:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe97806af90 0x7fe97806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe99815ab40 0x7fe9980c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9980cb580 0x7fe9980c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.260 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 --2- 172.21.15.53:0/2066747856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9980c90b0 0x7fe9980c8270 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 >> 172.21.15.53:0/2066747856 conn(0x7fe99806a380 msgr2=0x7fe9980af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 shutdown_connections 2023-10-21T17:38:04.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.257+0000 7fe99e07c640 1 -- 172.21.15.53:0/2066747856 wait complete. 2023-10-21T17:38:04.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:35:42.612998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '214', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:35:43.682480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:38:04.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:38:04.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.480+0000 7ff1fc400640 1 Processor -- start 2023-10-21T17:38:04.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.480+0000 7ff1fc400640 1 -- start start 2023-10-21T17:38:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.482+0000 7ff1fc400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 0x7ff1f415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.482+0000 7ff1fc400640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1f40b51b0 0x7ff1f40b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.482+0000 7ff1fc400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f415b610 con 0x7ff1f40b1850 2023-10-21T17:38:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.482+0000 7ff1fc400640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f415b8f0 con 0x7ff1f40b51b0 2023-10-21T17:38:04.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.482+0000 7ff1fc400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f415bbd0 con 0x7ff1f40ca7c0 2023-10-21T17:38:04.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1f40b51b0 0x7ff1f40b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff1f40b51b0 0x7ff1f40b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45664/0 (socket says 172.21.15.53:45664) 2023-10-21T17:38:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 -- 172.21.15.53:0/2808904362 learned_addr learned my addr 172.21.15.53:0/2808904362 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1fa19c640 1 --2- 172.21.15.53:0/2808904362 >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 0x7ff1f415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 -- 172.21.15.53:0/2808904362 >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 msgr2=0x7ff1f415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:04.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 --2- 172.21.15.53:0/2808904362 >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 0x7ff1f415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 -- 172.21.15.53:0/2808904362 >> v1:172.21.15.124:6789/0 conn(0x7ff1f40b1850 legacy=0x7ff1f40cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:04.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1f999b640 1 -- 172.21.15.53:0/2808904362 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40cc300 con 0x7ff1f40b51b0 2023-10-21T17:38:04.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.483+0000 7ff1fa19c640 1 --2- 172.21.15.53:0/2808904362 >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 0x7ff1f415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:04.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.484+0000 7ff1f999b640 1 --2- 172.21.15.53:0/2808904362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b51b0 0x7ff1f40b1310 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40d0700 tx=0x7ff1e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c5baf481d687a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.484+0000 7ff1f919a640 1 -- 172.21.15.53:0/2808904362 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4031870 con 0x7ff1f40b51b0 2023-10-21T17:38:04.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1f919a640 1 -- 172.21.15.53:0/2808904362 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40319d0 con 0x7ff1f40b51b0 2023-10-21T17:38:04.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1f919a640 1 -- 172.21.15.53:0/2808904362 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4031d60 con 0x7ff1f40b51b0 2023-10-21T17:38:04.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 -- 172.21.15.53:0/2808904362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b51b0 msgr2=0x7ff1f40b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2808904362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b51b0 0x7ff1f40b1310 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40d0700 tx=0x7ff1e402b910 comp rx=0 tx=0).stop 2023-10-21T17:38:04.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 -- 172.21.15.53:0/2808904362 shutdown_connections 2023-10-21T17:38:04.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2808904362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b51b0 0x7ff1f40b1310 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2808904362 >> v2:172.21.15.124:3300/0 conn(0x7ff1f40ca7c0 0x7ff1f415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 -- 172.21.15.53:0/2808904362 >> 172.21.15.53:0/2808904362 conn(0x7ff1f406b070 msgr2=0x7ff1f406b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:04.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 -- 172.21.15.53:0/2808904362 shutdown_connections 2023-10-21T17:38:04.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.485+0000 7ff1fc400640 1 -- 172.21.15.53:0/2808904362 wait complete. 2023-10-21T17:38:04.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.487+0000 7ff1fc400640 1 Processor -- start 2023-10-21T17:38:04.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.487+0000 7ff1fc400640 1 -- start start 2023-10-21T17:38:04.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 0x7ff1f4153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1f40ca7c0 0x7ff1f4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f415dfb0 con 0x7ff1f40b1850 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f40b0480 con 0x7ff1f40ca7c0 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fc400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f415d9f0 con 0x7ff1f40b51b0 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fa19c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.488+0000 7ff1fa19c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45670/0 (socket says 172.21.15.53:45670) 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 -- 172.21.15.53:0/2571028156 learned_addr learned my addr 172.21.15.53:0/2571028156 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:04.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1f999b640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 0x7ff1f4153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 -- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 msgr2=0x7ff1f4153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 0x7ff1f4153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 -- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1f40ca7c0 msgr2=0x7ff1f4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1f40ca7c0 0x7ff1f4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 -- 172.21.15.53:0/2571028156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40094d0 con 0x7ff1f40b1850 2023-10-21T17:38:04.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1f999b640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 0x7ff1f4153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:04.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1fa19c640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000d7d0 tx=0x7ff1f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004030 con 0x7ff1f40b1850 2023-10-21T17:38:04.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.489+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f0004190 con 0x7ff1f40b1850 2023-10-21T17:38:04.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.490+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f415dcd0 con 0x7ff1f40b1850 2023-10-21T17:38:04.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.490+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004520 con 0x7ff1f40b1850 2023-10-21T17:38:04.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.490+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f40b3cf0 con 0x7ff1f40b1850 2023-10-21T17:38:04.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.491+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f404e160 con 0x7ff1f40b1850 2023-10-21T17:38:04.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.492+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff1f00096a0 con 0x7ff1f40b1850 2023-10-21T17:38:04.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.493+0000 7ff1eaffd640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 0x7ff1d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.493+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6722+0+0 (secure 0 0 0) 0x7ff1f0089c50 con 0x7ff1f40b1850 2023-10-21T17:38:04.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.493+0000 7ff1f999b640 1 -- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 msgr2=0x7ff1d806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:04.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.493+0000 7ff1f999b640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 0x7ff1d806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:04.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.497+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1f00547e0 con 0x7ff1f40b1850 2023-10-21T17:38:04.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.536+0000 7f547a353040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:38:04.589 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.587+0000 7f547a353040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:38:04.592 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.591+0000 7ff678a52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:38:04.638 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.636+0000 7f547a353040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:38:04.647 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.646+0000 7ff678a52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:38:04.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.689+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1f40b3cf0 con 0x7ff1f40b1850 2023-10-21T17:38:04.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.690+0000 7ff1eaffd640 1 -- 172.21.15.53:0/2571028156 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v219) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1f40b3cf0 con 0x7ff1f40b1850 2023-10-21T17:38:04.692 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:38:04.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 msgr2=0x7ff1d806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:04.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 0x7ff1d806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 msgr2=0x7ff1f40b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000d7d0 tx=0x7ff1f000dc90 comp rx=0 tx=0).stop 2023-10-21T17:38:04.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 shutdown_connections 2023-10-21T17:38:04.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff1d806af90 0x7ff1d806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1f40ca7c0 0x7ff1f4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1f40b51b0 0x7ff1f4153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 --2- 172.21.15.53:0/2571028156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f40b1850 0x7ff1f40b35d0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.693+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 >> 172.21.15.53:0/2571028156 conn(0x7ff1f406b070 msgr2=0x7ff1f406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:04.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.694+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 shutdown_connections 2023-10-21T17:38:04.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.694+0000 7ff1fc400640 1 -- 172.21.15.53:0/2571028156 wait complete. 2023-10-21T17:38:04.698 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.697+0000 7f547a353040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:38:04.701 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.700+0000 7ff678a52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:38:04.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:38:04.747 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.745+0000 7f547a353040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:38:04.766 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.765+0000 7ff678a52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:38:04.819 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.818+0000 7ff678a52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:38:04.857 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.856+0000 7f547a353040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:38:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.901+0000 7f2cf782d640 1 Processor -- start 2023-10-21T17:38:04.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.901+0000 7f2cf782d640 1 -- start start 2023-10-21T17:38:04.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf782d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cf015abe0 0x7f2cf00d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf782d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf00b69d0 con 0x7f2cf00cd430 2023-10-21T17:38:04.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf782d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf00cf770 con 0x7f2cf00b62a0 2023-10-21T17:38:04.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf782d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf00b6700 con 0x7f2cf015abe0 2023-10-21T17:38:04.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf55c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cf015abe0 0x7f2cf00d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf5dca640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2cf00cd430 0x7f2cf00b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48636/0 (socket says 172.21.15.53:48636) 2023-10-21T17:38:04.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.903+0000 7f2cf5dca640 1 -- 172.21.15.53:0/2093743746 learned_addr learned my addr 172.21.15.53:0/2093743746 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.904+0000 7f2cf55c9640 1 -- 172.21.15.53:0/2093743746 >> v1:172.21.15.124:6789/0 conn(0x7f2cf00b62a0 legacy=0x7f2cf00cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.904+0000 7f2cf55c9640 1 -- 172.21.15.53:0/2093743746 >> v1:172.21.15.53:6789/0 conn(0x7f2cf00cd430 legacy=0x7f2cf00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.904+0000 7f2cf55c9640 1 -- 172.21.15.53:0/2093743746 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00cee40 con 0x7f2cf015abe0 2023-10-21T17:38:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.904+0000 7f2ce7fff640 1 -- 172.21.15.53:0/2093743746 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455322302 0 0) 0x7f2cf00b69d0 con 0x7f2cf00cd430 2023-10-21T17:38:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.905+0000 7f2cf55c9640 1 --2- 172.21.15.53:0/2093743746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf00d0420 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009790 tx=0x7f2cec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dae38fe9b8911f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.905+0000 7f2ce7fff640 1 -- 172.21.15.53:0/2093743746 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec031830 con 0x7f2cf015abe0 2023-10-21T17:38:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.905+0000 7f2ce7fff640 1 -- 172.21.15.53:0/2093743746 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec031990 con 0x7f2cf015abe0 2023-10-21T17:38:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.905+0000 7f2ce7fff640 1 -- 172.21.15.53:0/2093743746 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec031d20 con 0x7f2cf015abe0 2023-10-21T17:38:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 -- 172.21.15.53:0/2093743746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 msgr2=0x7f2cf00d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 --2- 172.21.15.53:0/2093743746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf00d0420 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009790 tx=0x7f2cec02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 -- 172.21.15.53:0/2093743746 shutdown_connections 2023-10-21T17:38:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 --2- 172.21.15.53:0/2093743746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf00d0420 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 -- 172.21.15.53:0/2093743746 >> 172.21.15.53:0/2093743746 conn(0x7f2cf006e650 msgr2=0x7f2cf006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 -- 172.21.15.53:0/2093743746 shutdown_connections 2023-10-21T17:38:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.906+0000 7f2cf782d640 1 -- 172.21.15.53:0/2093743746 wait complete. 2023-10-21T17:38:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.908+0000 7f2cf782d640 1 Processor -- start 2023-10-21T17:38:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.908+0000 7f2cf782d640 1 -- start start 2023-10-21T17:38:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 0x7f2cf0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf00cd430 0x7f2cf00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0152090 con 0x7f2cf015abe0 2023-10-21T17:38:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf00b5980 con 0x7f2cf00b62a0 2023-10-21T17:38:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf782d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf01522e0 con 0x7f2cf00cd430 2023-10-21T17:38:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf5dca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf5dca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45688/0 (socket says 172.21.15.53:45688) 2023-10-21T17:38:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf5dca640 1 -- 172.21.15.53:0/3126920843 learned_addr learned my addr 172.21.15.53:0/3126920843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:04.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf4dc8640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf00cd430 0x7f2cf00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.909+0000 7f2cf55c9640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 0x7f2cf0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:04.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 -- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf00cd430 msgr2=0x7f2cf00b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf00cd430 0x7f2cf00b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 -- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 msgr2=0x7f2cf0151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 0x7f2cf0151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 -- 172.21.15.53:0/3126920843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0093a0 con 0x7f2cf015abe0 2023-10-21T17:38:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf55c9640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 0x7f2cf0151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf5dca640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c810 tx=0x7f2ce800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800a640 con 0x7f2cf015abe0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce800a7c0 con 0x7f2cf015abe0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce800ab70 con 0x7f2cf015abe0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf01525c0 con 0x7f2cf015abe0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.910+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf0152b80 con 0x7f2cf015abe0 2023-10-21T17:38:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.911+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf00b0230 con 0x7f2cf015abe0 2023-10-21T17:38:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.912+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f2ce800acd0 con 0x7f2cf015abe0 2023-10-21T17:38:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.913+0000 7f2ce5ffb640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.913+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f2ce8088fe0 con 0x7f2cf015abe0 2023-10-21T17:38:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.914+0000 7f2cf55c9640 1 -- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 msgr2=0x7f2cd406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.914+0000 7f2cf55c9640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:04.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:04.918+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ce8053b90 con 0x7f2cf015abe0 2023-10-21T17:38:04.919 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.918+0000 7f547a353040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:04.939 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:04.937+0000 7ff678a52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:38:04.970 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:04.968+0000 7f547a353040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:38:05.005 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.004+0000 7ff678a52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:05.026 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.024+0000 7f547a353040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:38:05.057 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.056+0000 7ff678a52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:38:05.076 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.075+0000 7f547a353040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:38:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.105+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cf004e160 con 0x7f2cf015abe0 2023-10-21T17:38:05.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.106+0000 7f2ce5ffb640 1 -- 172.21.15.53:0/3126920843 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v219) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ce80180a0 con 0x7f2cf015abe0 2023-10-21T17:38:05.108 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:38:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.108+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 msgr2=0x7f2cd406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.108+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.108+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 msgr2=0x7f2cf01582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.108+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c810 tx=0x7f2ce800ccd0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 shutdown_connections 2023-10-21T17:38:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf015abe0 0x7f2cf01582b0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf00cd430 0x7f2cf00b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 --2- 172.21.15.53:0/3126920843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf00b62a0 0x7f2cf0151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 >> 172.21.15.53:0/3126920843 conn(0x7f2cf006e650 msgr2=0x7f2cf015b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:05.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 shutdown_connections 2023-10-21T17:38:05.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.109+0000 7f2cf782d640 1 -- 172.21.15.53:0/3126920843 wait complete. 2023-10-21T17:38:05.117 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.116+0000 7ff678a52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:38:05.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:38:05.172 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.171+0000 7ff678a52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:38:05.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.336+0000 7f3ae781f640 1 Processor -- start 2023-10-21T17:38:05.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.336+0000 7f3ae781f640 1 -- start start 2023-10-21T17:38:05.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.337+0000 7f3ae781f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3ae00afbe0 0x7f3ae00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.337+0000 7f3ae781f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ae00cb130 0x7f3ae00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.337+0000 7f3ae781f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00cbad0 con 0x7f3ae006dfc0 2023-10-21T17:38:05.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.337+0000 7f3ae781f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00b2620 con 0x7f3ae00cb130 2023-10-21T17:38:05.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.337+0000 7f3ae781f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00b2060 con 0x7f3ae00afbe0 2023-10-21T17:38:05.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ae00cb130 0x7f3ae00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ae00cb130 0x7f3ae00cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45700/0 (socket says 172.21.15.53:45700) 2023-10-21T17:38:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 -- 172.21.15.53:0/277408073 learned_addr learned my addr 172.21.15.53:0/277408073 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae55bb640 1 --2- 172.21.15.53:0/277408073 >> v2:172.21.15.124:3300/0 conn(0x7f3ae00afbe0 0x7f3ae00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 -- 172.21.15.53:0/277408073 >> v2:172.21.15.124:3300/0 conn(0x7f3ae00afbe0 msgr2=0x7f3ae00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:05.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 --2- 172.21.15.53:0/277408073 >> v2:172.21.15.124:3300/0 conn(0x7f3ae00afbe0 0x7f3ae00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 -- 172.21.15.53:0/277408073 >> v1:172.21.15.124:6789/0 conn(0x7f3ae006dfc0 legacy=0x7f3ae0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:05.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.338+0000 7f3ae5dbc640 1 -- 172.21.15.53:0/277408073 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00b2340 con 0x7f3ae00cb130 2023-10-21T17:38:05.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.340+0000 7f3ae5dbc640 1 --2- 172.21.15.53:0/277408073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00cb130 0x7f3ae00cb590 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f3ae006e640 tx=0x7f3ad802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31ceb3b72bde48e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:05.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.340+0000 7f3ad7fff640 1 -- 172.21.15.53:0/277408073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad802ca80 con 0x7f3ae00cb130 2023-10-21T17:38:05.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ad7fff640 1 -- 172.21.15.53:0/277408073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad802cbe0 con 0x7f3ae00cb130 2023-10-21T17:38:05.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ad7fff640 1 -- 172.21.15.53:0/277408073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80343f0 con 0x7f3ae00cb130 2023-10-21T17:38:05.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 -- 172.21.15.53:0/277408073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00cb130 msgr2=0x7f3ae00cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 --2- 172.21.15.53:0/277408073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00cb130 0x7f3ae00cb590 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f3ae006e640 tx=0x7f3ad802bee0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 -- 172.21.15.53:0/277408073 shutdown_connections 2023-10-21T17:38:05.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 --2- 172.21.15.53:0/277408073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00cb130 0x7f3ae00cb590 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 --2- 172.21.15.53:0/277408073 >> v2:172.21.15.124:3300/0 conn(0x7f3ae00afbe0 0x7f3ae00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 -- 172.21.15.53:0/277408073 >> 172.21.15.53:0/277408073 conn(0x7f3ae006a5d0 msgr2=0x7f3ae00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:05.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 -- 172.21.15.53:0/277408073 shutdown_connections 2023-10-21T17:38:05.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.341+0000 7f3ae781f640 1 -- 172.21.15.53:0/277408073 wait complete. 2023-10-21T17:38:05.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.343+0000 7f3ae781f640 1 Processor -- start 2023-10-21T17:38:05.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.343+0000 7f3ae781f640 1 -- start start 2023-10-21T17:38:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ae006dfc0 0x7f3ae00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ae00cb130 0x7f3ae00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00cdbd0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae014e060 con 0x7f3ae00cb130 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.344+0000 7f3ae781f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00cde00 con 0x7f3ae006dfc0 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45712/0 (socket says 172.21.15.53:45712) 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 -- 172.21.15.53:0/2480936610 learned_addr learned my addr 172.21.15.53:0/2480936610 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae55bb640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ae006dfc0 0x7f3ae00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae5dbc640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ae00cb130 0x7f3ae00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ae006dfc0 msgr2=0x7f3ae00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ae006dfc0 0x7f3ae00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ae00cb130 msgr2=0x7f3ae00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ae00cb130 0x7f3ae00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 -- 172.21.15.53:0/2480936610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad8009b30 con 0x7f3ae00afbe0 2023-10-21T17:38:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.345+0000 7f3ae4dba640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000b3c0 tx=0x7f3ad000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.346+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011830 con 0x7f3ae00afbe0 2023-10-21T17:38:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.346+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0011990 con 0x7f3ae00afbe0 2023-10-21T17:38:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.346+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00ce0e0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.346+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0011d80 con 0x7f3ae00afbe0 2023-10-21T17:38:05.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.346+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae00ce6a0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.347+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae004e160 con 0x7f3ae00afbe0 2023-10-21T17:38:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.348+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3ad001b460 con 0x7f3ae00afbe0 2023-10-21T17:38:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.349+0000 7f3ad5ffb640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 0x7f3ac406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.349+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6706+0+0 (secure 0 0 0) 0x7f3ad0089730 con 0x7f3ae00afbe0 2023-10-21T17:38:05.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.349+0000 7f3ae55bb640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 msgr2=0x7f3ac406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.349+0000 7f3ae55bb640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 0x7f3ac406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:05.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.353+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ad00541b0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.462 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.461+0000 7f547a353040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:38:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.550+0000 7f3ae55bb640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 msgr2=0x7f3ac406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:05.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.550+0000 7f3ae55bb640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 0x7f3ac406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:05.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.567+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ae00b0ef0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.568+0000 7f3ad5ffb640 1 -- 172.21.15.53:0/2480936610 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v220) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ad00577d0 con 0x7f3ae00afbe0 2023-10-21T17:38:05.569 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:38:05.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.570+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 msgr2=0x7f3ac406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:05.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.570+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 0x7f3ac406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 msgr2=0x7f3ae014db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000b3c0 tx=0x7f3ad000b880 comp rx=0 tx=0).stop 2023-10-21T17:38:05.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 shutdown_connections 2023-10-21T17:38:05.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f3ac406af90 0x7f3ac406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3ae00cb130 0x7f3ae00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ae00afbe0 0x7f3ae014db20 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 --2- 172.21.15.53:0/2480936610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3ae006dfc0 0x7f3ae00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 >> 172.21.15.53:0/2480936610 conn(0x7f3ae006a5d0 msgr2=0x7f3ae00c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:05.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 shutdown_connections 2023-10-21T17:38:05.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.571+0000 7f3ae781f640 1 -- 172.21.15.53:0/2480936610 wait complete. 2023-10-21T17:38:05.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:38:05.612 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.611+0000 7ff678a52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:38:05.622 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.621+0000 7f547a353040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:38:05.728 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.726+0000 7f547a353040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:38:05.777 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.775+0000 7f547a353040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:38:05.785 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.784+0000 7ff678a52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:38:05.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.817+0000 7ff65ca7c640 1 Processor -- start 2023-10-21T17:38:05.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.818+0000 7ff65ca7c640 1 -- start start 2023-10-21T17:38:05.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.819+0000 7ff65ca7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff658161690 0x7ff65806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.819+0000 7ff65ca7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6580cb150 0x7ff6580cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.819+0000 7ff65ca7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 0x7ff658070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff65ca7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6580cbb10 con 0x7ff6580cb150 2023-10-21T17:38:05.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff65ca7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65806cc50 con 0x7ff6580702b0 2023-10-21T17:38:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff65ca7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806cf30 con 0x7ff658161690 2023-10-21T17:38:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6580cb150 0x7ff6580cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6580cb150 0x7ff6580cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45718/0 (socket says 172.21.15.53:45718) 2023-10-21T17:38:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 -- 172.21.15.53:0/3149586561 learned_addr learned my addr 172.21.15.53:0/3149586561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:05.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff65659c640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff658161690 0x7ff65806dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff656d9d640 1 --2- 172.21.15.53:0/3149586561 >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 0x7ff658070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 -- 172.21.15.53:0/3149586561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff658161690 msgr2=0x7ff65806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff658161690 0x7ff65806dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 -- 172.21.15.53:0/3149586561 >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 msgr2=0x7ff658070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:05.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 --2- 172.21.15.53:0/3149586561 >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 0x7ff658070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff655d9b640 1 -- 172.21.15.53:0/3149586561 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65806d210 con 0x7ff6580cb150 2023-10-21T17:38:05.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.820+0000 7ff656d9d640 1 --2- 172.21.15.53:0/3149586561 >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 0x7ff658070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.821+0000 7ff655d9b640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580cb5d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7ff6580b1760 tx=0x7ff64802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a84b46d52d2c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.822+0000 7ff65559a640 1 -- 172.21.15.53:0/3149586561 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648031880 con 0x7ff6580cb150 2023-10-21T17:38:05.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.822+0000 7ff65559a640 1 -- 172.21.15.53:0/3149586561 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6480319e0 con 0x7ff6580cb150 2023-10-21T17:38:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.822+0000 7ff65559a640 1 -- 172.21.15.53:0/3149586561 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648031d70 con 0x7ff6580cb150 2023-10-21T17:38:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.822+0000 7ff65ca7c640 1 -- 172.21.15.53:0/3149586561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 msgr2=0x7ff6580cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.822+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580cb5d0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7ff6580b1760 tx=0x7ff64802b920 comp rx=0 tx=0).stop 2023-10-21T17:38:05.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 -- 172.21.15.53:0/3149586561 shutdown_connections 2023-10-21T17:38:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/3149586561 >> v2:172.21.15.124:3300/0 conn(0x7ff6580702b0 0x7ff658070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580cb5d0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/3149586561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff658161690 0x7ff65806dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 -- 172.21.15.53:0/3149586561 >> 172.21.15.53:0/3149586561 conn(0x7ff65806a5d0 msgr2=0x7ff6580b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:05.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 -- 172.21.15.53:0/3149586561 shutdown_connections 2023-10-21T17:38:05.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.823+0000 7ff65ca7c640 1 -- 172.21.15.53:0/3149586561 wait complete. 2023-10-21T17:38:05.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.825+0000 7ff65ca7c640 1 Processor -- start 2023-10-21T17:38:05.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.825+0000 7ff65ca7c640 1 -- start start 2023-10-21T17:38:05.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6580702b0 0x7ff658160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff658161690 0x7ff6580c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658162b60 con 0x7ff6580cb150 2023-10-21T17:38:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580c3030 con 0x7ff658161690 2023-10-21T17:38:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65ca7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580c32d0 con 0x7ff6580702b0 2023-10-21T17:38:05.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45734/0 (socket says 172.21.15.53:45734) 2023-10-21T17:38:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 -- 172.21.15.53:0/4018955185 learned_addr learned my addr 172.21.15.53:0/4018955185 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff65659c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6580702b0 0x7ff658160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff656d9d640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff658161690 0x7ff6580c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6580702b0 msgr2=0x7ff658160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6580702b0 0x7ff658160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff658161690 msgr2=0x7ff6580c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:05.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.826+0000 7ff655d9b640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff658161690 0x7ff6580c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff655d9b640 1 -- 172.21.15.53:0/4018955185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480094d0 con 0x7ff6580cb150 2023-10-21T17:38:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff655d9b640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7ff64802b960 tx=0x7ff6480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64803ad30 con 0x7ff6580cb150 2023-10-21T17:38:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6480443f0 con 0x7ff6580cb150 2023-10-21T17:38:05.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff658162e00 con 0x7ff6580cb150 2023-10-21T17:38:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6480447a0 con 0x7ff6580cb150 2023-10-21T17:38:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.827+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6580c13e0 con 0x7ff6580cb150 2023-10-21T17:38:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.828+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65804e160 con 0x7ff6580cb150 2023-10-21T17:38:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.829+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff64804d450 con 0x7ff6580cb150 2023-10-21T17:38:05.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.831+0000 7ff64effd640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 0x7ff63c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:05.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.831+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6706+0+0 (secure 0 0 0) 0x7ff6480ac180 con 0x7ff6580cb150 2023-10-21T17:38:05.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.831+0000 7ff65659c640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 msgr2=0x7ff63c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:05.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.831+0000 7ff65659c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 0x7ff63c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:05.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:05.835+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff648076cb0 con 0x7ff6580cb150 2023-10-21T17:38:05.885 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.884+0000 7f547a353040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:05.905 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.904+0000 7ff678a52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:38:05.959 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:05.957+0000 7ff678a52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:38:05.991 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:05.989+0000 7f547a353040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:38:06.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.032+0000 7ff65659c640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 msgr2=0x7ff63c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:06.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.032+0000 7ff65659c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 0x7ff63c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:06.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.043+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff6580ca270 con 0x7ff6580cb150 2023-10-21T17:38:06.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.044+0000 7ff64effd640 1 -- 172.21.15.53:0/4018955185 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v220) v1 ==== 80+0+473 (secure 0 0 0) 0x7ff648035080 con 0x7ff6580cb150 2023-10-21T17:38:06.046 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/248442090 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.046 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/248442090 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.046 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2759452384 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.046 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/863767263 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.046 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1944369314 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.047 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1803654246 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.047 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/820409426 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.047 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/102947359 2023-10-22T17:35:59.400288+0000 2023-10-21T17:38:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T17:38:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 msgr2=0x7ff63c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:06.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 0x7ff63c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 msgr2=0x7ff6580c2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7ff64802b960 tx=0x7ff6480046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 shutdown_connections 2023-10-21T17:38:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7ff63c06afc0 0x7ff63c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff658161690 0x7ff6580c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6580cb150 0x7ff6580c2af0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 --2- 172.21.15.53:0/4018955185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6580702b0 0x7ff658160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 >> 172.21.15.53:0/4018955185 conn(0x7ff65806a5d0 msgr2=0x7ff6580c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 shutdown_connections 2023-10-21T17:38:06.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.047+0000 7ff65ca7c640 1 -- 172.21.15.53:0/4018955185 wait complete. 2023-10-21T17:38:06.066 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:38:06.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:06.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:38:06.075 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:06.074+0000 7ff678a52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:06.146 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:06.145+0000 7f547a353040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:06.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:38:06.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:06.174 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:38:06.194 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:06.193+0000 7ff678a52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:38:06.203 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:06.201+0000 7f547a353040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:06.247 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:38:06.247 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:06.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:38:06.300 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:06.300 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:06.328 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:06.326+0000 7f547a353040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:38:06.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:38:06.355 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:38:06.356 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:06.363 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:06.362+0000 7ff678a52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:06.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:38:06.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:06.421 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:06.414+0000 7ff678a52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:06.544 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:06.543+0000 7ff678a52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:38:06.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.629+0000 7fe864439640 1 Processor -- start 2023-10-21T17:38:06.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.630+0000 7fe864439640 1 -- start start 2023-10-21T17:38:06.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.631+0000 7fe864439640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c0cb220 0x7fe85c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:06.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.631+0000 7fe864439640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0cf550 con 0x7fe85c0cd750 2023-10-21T17:38:06.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.631+0000 7fe864439640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c06b490 con 0x7fe85c15cd20 2023-10-21T17:38:06.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.631+0000 7fe864439640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c06b770 con 0x7fe85c0cb220 2023-10-21T17:38:06.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c0cb220 0x7fe85c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:06.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe85c0cb220 0x7fe85c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45744/0 (socket says 172.21.15.53:45744) 2023-10-21T17:38:06.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 -- 172.21.15.53:0/50154413 learned_addr learned my addr 172.21.15.53:0/50154413 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:06.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 -- 172.21.15.53:0/50154413 >> v1:172.21.15.124:6789/0 conn(0x7fe85c15cd20 legacy=0x7fe85c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:06.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 -- 172.21.15.53:0/50154413 >> v1:172.21.15.53:6789/0 conn(0x7fe85c0cd750 legacy=0x7fe85c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.632+0000 7fe8619d4640 1 -- 172.21.15.53:0/50154413 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c06ba50 con 0x7fe85c0cb220 2023-10-21T17:38:06.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.633+0000 7fe8619d4640 1 --2- 172.21.15.53:0/50154413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0cb680 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009870 tx=0x7fe84c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35ccbf155324d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:06.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe8611d3640 1 -- 172.21.15.53:0/50154413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c030860 con 0x7fe85c0cb220 2023-10-21T17:38:06.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe8611d3640 1 -- 172.21.15.53:0/50154413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c0309c0 con 0x7fe85c0cb220 2023-10-21T17:38:06.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe8611d3640 1 -- 172.21.15.53:0/50154413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c030d50 con 0x7fe85c0cb220 2023-10-21T17:38:06.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 -- 172.21.15.53:0/50154413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 msgr2=0x7fe85c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 --2- 172.21.15.53:0/50154413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0cb680 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009870 tx=0x7fe84c02b9a0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 -- 172.21.15.53:0/50154413 shutdown_connections 2023-10-21T17:38:06.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 --2- 172.21.15.53:0/50154413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0cb680 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 -- 172.21.15.53:0/50154413 >> 172.21.15.53:0/50154413 conn(0x7fe85c069f30 msgr2=0x7fe85c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:06.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 -- 172.21.15.53:0/50154413 shutdown_connections 2023-10-21T17:38:06.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.634+0000 7fe864439640 1 -- 172.21.15.53:0/50154413 wait complete. 2023-10-21T17:38:06.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.636+0000 7fe864439640 1 Processor -- start 2023-10-21T17:38:06.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.636+0000 7fe864439640 1 -- start start 2023-10-21T17:38:06.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:06.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe85c0cd750 0x7fe85c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:06.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 0x7fe85c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:06.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0b4c00 con 0x7fe85c0cb220 2023-10-21T17:38:06.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0b4ee0 con 0x7fe85c0cd750 2023-10-21T17:38:06.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe864439640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0b51c0 con 0x7fe85c15cd20 2023-10-21T17:38:06.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe8621d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:06.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe8621d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45746/0 (socket says 172.21.15.53:45746) 2023-10-21T17:38:06.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.637+0000 7fe8621d5640 1 -- 172.21.15.53:0/3422916857 learned_addr learned my addr 172.21.15.53:0/3422916857 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:06.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8629d6640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 0x7fe85c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:06.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8619d4640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe85c0cd750 0x7fe85c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:06.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 msgr2=0x7fe85c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 0x7fe85c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe85c0cd750 msgr2=0x7fe85c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe85c0cd750 0x7fe85c15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 -- 172.21.15.53:0/3422916857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0094d0 con 0x7fe85c0cb220 2023-10-21T17:38:06.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8629d6640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 0x7fe85c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:06.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe8621d5640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c430 tx=0x7fe85800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:06.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011790 con 0x7fe85c0cb220 2023-10-21T17:38:06.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8580118f0 con 0x7fe85c0cb220 2023-10-21T17:38:06.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011ce0 con 0x7fe85c0cb220 2023-10-21T17:38:06.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.638+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c0b54a0 con 0x7fe85c0cb220 2023-10-21T17:38:06.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.639+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c0b4540 con 0x7fe85c0cb220 2023-10-21T17:38:06.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.640+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c066d80 con 0x7fe85c0cb220 2023-10-21T17:38:06.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.642+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fe85801c460 con 0x7fe85c0cb220 2023-10-21T17:38:06.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.643+0000 7fe852ffd640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 0x7fe84006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:06.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.643+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6706+0+0 (secure 0 0 0) 0x7fe858089b50 con 0x7fe85c0cb220 2023-10-21T17:38:06.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.643+0000 7fe8619d4640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 msgr2=0x7fe84006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:06.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.643+0000 7fe8619d4640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 0x7fe84006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:06.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.647+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8580546f0 con 0x7fe85c0cb220 2023-10-21T17:38:06.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.844+0000 7fe8619d4640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 msgr2=0x7fe84006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:06.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.844+0000 7fe8619d4640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 0x7fe84006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:06.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.878+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe85c0b6970 con 0x7fe85c0cb220 2023-10-21T17:38:06.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.879+0000 7fe852ffd640 1 -- 172.21.15.53:0/3422916857 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7fe858057d10 con 0x7fe85c0cb220 2023-10-21T17:38:06.880 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:06.881 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:38:06.881 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:38:06.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 msgr2=0x7fe84006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:06.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 0x7fe84006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 msgr2=0x7fe85c0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:06.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c430 tx=0x7fe85800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 shutdown_connections 2023-10-21T17:38:06.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7fe84006af90 0x7fe84006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe85c15cd20 0x7fe85c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe85c0cd750 0x7fe85c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 --2- 172.21.15.53:0/3422916857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe85c0cb220 0x7fe85c0b4000 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:06.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 >> 172.21.15.53:0/3422916857 conn(0x7fe85c069f30 msgr2=0x7fe85c15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:06.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 shutdown_connections 2023-10-21T17:38:06.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:06.882+0000 7fe864439640 1 -- 172.21.15.53:0/3422916857 wait complete. 2023-10-21T17:38:06.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.113+0000 7f77130bb640 1 Processor -- start 2023-10-21T17:38:07.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.113+0000 7f77130bb640 1 -- start start 2023-10-21T17:38:07.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.114+0000 7f77130bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f770c15ab40 0x7f770c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.114+0000 7f77130bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.114+0000 7f77130bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c15b2b0 con 0x7f770c0c90b0 2023-10-21T17:38:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.114+0000 7f77130bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c15bf30 con 0x7f770c15ab40 2023-10-21T17:38:07.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.114+0000 7f77130bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c15f7a0 con 0x7f770c0cb580 2023-10-21T17:38:07.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35492/0 (socket says 172.21.15.53:35492) 2023-10-21T17:38:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 -- 172.21.15.53:0/542682915 learned_addr learned my addr 172.21.15.53:0/542682915 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7710e57640 1 --2- 172.21.15.53:0/542682915 >> v2:172.21.15.124:3300/0 conn(0x7f770c15ab40 0x7f770c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 -- 172.21.15.53:0/542682915 >> v2:172.21.15.124:3300/0 conn(0x7f770c15ab40 msgr2=0x7f770c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 --2- 172.21.15.53:0/542682915 >> v2:172.21.15.124:3300/0 conn(0x7f770c15ab40 0x7f770c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.115+0000 7f7711658640 1 -- 172.21.15.53:0/542682915 >> v1:172.21.15.124:6789/0 conn(0x7f770c0c90b0 legacy=0x7f770c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:07.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.116+0000 7f7711658640 1 -- 172.21.15.53:0/542682915 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c15b590 con 0x7f770c0cb580 2023-10-21T17:38:07.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.117+0000 7f7711658640 1 --2- 172.21.15.53:0/542682915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7704009870 tx=0x7f770402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cfbd7d700402b00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:07.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.117+0000 7f770b7fe640 1 -- 172.21.15.53:0/542682915 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704031640 con 0x7f770c0cb580 2023-10-21T17:38:07.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.117+0000 7f770b7fe640 1 -- 172.21.15.53:0/542682915 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77040317a0 con 0x7f770c0cb580 2023-10-21T17:38:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.117+0000 7f770b7fe640 1 -- 172.21.15.53:0/542682915 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704031b30 con 0x7f770c0cb580 2023-10-21T17:38:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 -- 172.21.15.53:0/542682915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 msgr2=0x7f770c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 --2- 172.21.15.53:0/542682915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7704009870 tx=0x7f770402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 -- 172.21.15.53:0/542682915 shutdown_connections 2023-10-21T17:38:07.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 --2- 172.21.15.53:0/542682915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c0adc60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 --2- 172.21.15.53:0/542682915 >> v2:172.21.15.124:3300/0 conn(0x7f770c15ab40 0x7f770c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 -- 172.21.15.53:0/542682915 >> 172.21.15.53:0/542682915 conn(0x7f770c06a380 msgr2=0x7f770c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:07.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 -- 172.21.15.53:0/542682915 shutdown_connections 2023-10-21T17:38:07.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.118+0000 7f77130bb640 1 -- 172.21.15.53:0/542682915 wait complete. 2023-10-21T17:38:07.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.120+0000 7f77130bb640 1 Processor -- start 2023-10-21T17:38:07.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.120+0000 7f77130bb640 1 -- start start 2023-10-21T17:38:07.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 0x7f770c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c1506d0 con 0x7f770c15ab40 2023-10-21T17:38:07.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0c7f00 con 0x7f770c0c90b0 2023-10-21T17:38:07.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f77130bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0c7630 con 0x7f770c0cb580 2023-10-21T17:38:07.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f7711658640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f7711658640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45754/0 (socket says 172.21.15.53:45754) 2023-10-21T17:38:07.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f7711658640 1 -- 172.21.15.53:0/3278469416 learned_addr learned my addr 172.21.15.53:0/3278469416 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:07.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f770bfff640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.121+0000 7f7710e57640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 0x7f770c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 msgr2=0x7f770c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 msgr2=0x7f770c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 0x7f770c0c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 -- 172.21.15.53:0/3278469416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040094d0 con 0x7f770c15ab40 2023-10-21T17:38:07.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7710e57640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 0x7f770c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:07.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f7711658640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f77040090b0 tx=0x7f77040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:07.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770403a9f0 con 0x7f770c15ab40 2023-10-21T17:38:07.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770403ab70 con 0x7f770c15ab40 2023-10-21T17:38:07.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.122+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c0c5260 con 0x7f770c15ab40 2023-10-21T17:38:07.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.123+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040443f0 con 0x7f770c15ab40 2023-10-21T17:38:07.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.123+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c0c7b90 con 0x7f770c15ab40 2023-10-21T17:38:07.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.124+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c0c8070 con 0x7f770c15ab40 2023-10-21T17:38:07.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.125+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f770402f050 con 0x7f770c15ab40 2023-10-21T17:38:07.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.126+0000 7f77097fa640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 0x7f76ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.126+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6706+0+0 (secure 0 0 0) 0x7f77040aba80 con 0x7f770c15ab40 2023-10-21T17:38:07.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.126+0000 7f7710e57640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 msgr2=0x7f76ec06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:07.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.126+0000 7f7710e57640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 0x7f76ec06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:07.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.133+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7704076510 con 0x7f770c15ab40 2023-10-21T17:38:07.168 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.166+0000 7f547a353040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:38:07.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.327+0000 7f7710e57640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 msgr2=0x7f76ec06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:07.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.327+0000 7f7710e57640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 0x7f76ec06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:07.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.329+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f770c067490 con 0x7f770c15ab40 2023-10-21T17:38:07.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.331+0000 7f77097fa640 1 -- 172.21.15.53:0/3278469416 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+17769 (secure 0 0 0) 0x7f7704079b30 con 0x7f770c15ab40 2023-10-21T17:38:07.333 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:07.333 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":220,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:05.307131+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":220,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.18","mappings":[{"from":3,"to":6}]},{"pgid":"26.23","mappings":[{"from":1,"to":6}]},{"pgid":"26.28","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6833/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:6832/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/2759452384":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/863767263":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1944369314":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1803654246":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/820409426":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/102947359":"2023-10-22T17:35:59.400288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 msgr2=0x7f76ec06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:07.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 0x7f76ec06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 msgr2=0x7f770c0c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f77040090b0 tx=0x7f77040046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 shutdown_connections 2023-10-21T17:38:07.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f76ec06af90 0x7f76ec06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f770c15ab40 0x7f770c0c47a0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f770c0cb580 0x7f770c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 --2- 172.21.15.53:0/3278469416 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f770c0c90b0 0x7f770c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 >> 172.21.15.53:0/3278469416 conn(0x7f770c06a380 msgr2=0x7f770c0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 shutdown_connections 2023-10-21T17:38:07.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.334+0000 7f77130bb640 1 -- 172.21.15.53:0/3278469416 wait complete. 2023-10-21T17:38:07.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:35:42.612998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '214', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:35:43.682480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 220, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:38:07.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:38:07.410 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:07.409+0000 7ff678a52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:38:07.460 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.459+0000 7f547a353040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:38:07.519 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.517+0000 7f547a353040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:38:07.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.570+0000 7f90fe021640 1 Processor -- start 2023-10-21T17:38:07.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.571+0000 7f90fe021640 1 -- start start 2023-10-21T17:38:07.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f815cb80 0x7f90f80d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90f80d26d0 0x7f90f80d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90f80b85d0 0x7f90f80d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80d1630 con 0x7f90f80b85d0 2023-10-21T17:38:07.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80d1910 con 0x7f90f80d26d0 2023-10-21T17:38:07.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90fe021640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80d1bf0 con 0x7f90f815cb80 2023-10-21T17:38:07.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.572+0000 7f90f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90f80b85d0 0x7f90f80d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f90f80b85d0 0x7f90f80d3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45758/0 (socket says 172.21.15.53:45758) 2023-10-21T17:38:07.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 -- 172.21.15.53:0/3904561160 learned_addr learned my addr 172.21.15.53:0/3904561160 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:07.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f77fe640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f815cb80 0x7f90f80d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90effff640 1 --2- 172.21.15.53:0/3904561160 >> v2:172.21.15.124:3300/0 conn(0x7f90f80d26d0 0x7f90f80d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 -- 172.21.15.53:0/3904561160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f815cb80 msgr2=0x7f90f80d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:07.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f815cb80 0x7f90f80d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 -- 172.21.15.53:0/3904561160 >> v2:172.21.15.124:3300/0 conn(0x7f90f80d26d0 msgr2=0x7f90f80d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:07.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 --2- 172.21.15.53:0/3904561160 >> v2:172.21.15.124:3300/0 conn(0x7f90f80d26d0 0x7f90f80d2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.573+0000 7f90f7fff640 1 -- 172.21.15.53:0/3904561160 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80d1ed0 con 0x7f90f80b85d0 2023-10-21T17:38:07.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.575+0000 7f90f7fff640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80b85d0 0x7f90f80d3110 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f90f80b57c0 tx=0x7f90f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb7a7a4dde23f8e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:07.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.575+0000 7f90f6ffd640 1 -- 172.21.15.53:0/3904561160 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f002ca70 con 0x7f90f80b85d0 2023-10-21T17:38:07.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.575+0000 7f90f6ffd640 1 -- 172.21.15.53:0/3904561160 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f002cbd0 con 0x7f90f80b85d0 2023-10-21T17:38:07.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.575+0000 7f90f6ffd640 1 -- 172.21.15.53:0/3904561160 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f00343f0 con 0x7f90f80b85d0 2023-10-21T17:38:07.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 -- 172.21.15.53:0/3904561160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80b85d0 msgr2=0x7f90f80d3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80b85d0 0x7f90f80d3110 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f90f80b57c0 tx=0x7f90f002bed0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 -- 172.21.15.53:0/3904561160 shutdown_connections 2023-10-21T17:38:07.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80b85d0 0x7f90f80d3110 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 --2- 172.21.15.53:0/3904561160 >> v2:172.21.15.124:3300/0 conn(0x7f90f80d26d0 0x7f90f80d2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 --2- 172.21.15.53:0/3904561160 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f815cb80 0x7f90f80d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 -- 172.21.15.53:0/3904561160 >> 172.21.15.53:0/3904561160 conn(0x7f90f80718a0 msgr2=0x7f90f8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:07.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 -- 172.21.15.53:0/3904561160 shutdown_connections 2023-10-21T17:38:07.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.576+0000 7f90fe021640 1 -- 172.21.15.53:0/3904561160 wait complete. 2023-10-21T17:38:07.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.578+0000 7f90fe021640 1 Processor -- start 2023-10-21T17:38:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.578+0000 7f90fe021640 1 -- start start 2023-10-21T17:38:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f80b85d0 0x7f90f814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 0x7f90f8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8159840 con 0x7f90f80d26d0 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8158c50 con 0x7f90f815cb80 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90fe021640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8158ea0 con 0x7f90f80b85d0 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45770/0 (socket says 172.21.15.53:45770) 2023-10-21T17:38:07.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90effff640 1 -- 172.21.15.53:0/2143044480 learned_addr learned my addr 172.21.15.53:0/2143044480 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:07.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.579+0000 7f90f77fe640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f80b85d0 0x7f90f814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90f7fff640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 0x7f90f8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:07.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 -- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f80b85d0 msgr2=0x7f90f814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f80b85d0 0x7f90f814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 -- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 msgr2=0x7f90f8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 0x7f90f8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 -- 172.21.15.53:0/2143044480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0009b30 con 0x7f90f80d26d0 2023-10-21T17:38:07.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90f7fff640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 0x7f90f8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:07.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90effff640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f90e000d800 tx=0x7f90e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:07.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00096a0 con 0x7f90f80d26d0 2023-10-21T17:38:07.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0009800 con 0x7f90f80d26d0 2023-10-21T17:38:07.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.580+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0009b90 con 0x7f90f80d26d0 2023-10-21T17:38:07.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.581+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f8159180 con 0x7f90f80d26d0 2023-10-21T17:38:07.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.581+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f814a950 con 0x7f90f80d26d0 2023-10-21T17:38:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.582+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f804e160 con 0x7f90f80d26d0 2023-10-21T17:38:07.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.583+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 122) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f90e0009cf0 con 0x7f90f80d26d0 2023-10-21T17:38:07.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.583+0000 7f90f4ff9640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 0x7f90d806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:07.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.584+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6706+0+0 (secure 0 0 0) 0x7f90e008a140 con 0x7f90f80d26d0 2023-10-21T17:38:07.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.584+0000 7f90f77fe640 1 -- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 msgr2=0x7f90d806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1168372736 2023-10-21T17:38:07.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.584+0000 7f90f77fe640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 0x7f90d806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:07.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.588+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90e0054dd0 con 0x7f90f80d26d0 2023-10-21T17:38:07.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.631+0000 7f547a353040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:38:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.653+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 123) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f90e00551a0 con 0x7f90f80d26d0 2023-10-21T17:38:07.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.653+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 msgr2=0x7f90d806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:07.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.653+0000 7f90f4ff9640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 0x7f90d806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.728 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.727+0000 7f53ffde1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.728 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.727+0000 7f53ffde1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.729 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.727+0000 7f53ffde1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.729 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.727+0000 7f53ffde1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.729 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.727+0000 7f53ffde1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.730 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.728+0000 7f53fc5da640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.730 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.728+0000 7f53fc5da640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.730 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.728+0000 7f53fc5da640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.731 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.728+0000 7f53fc5da640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.731 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:07.728+0000 7f53fc5da640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:38:07.743 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:07.742+0000 7ff678a52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:38:07.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.792+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90f80ccb00 con 0x7f90f80d26d0 2023-10-21T17:38:07.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.793+0000 7f90f4ff9640 1 -- 172.21.15.53:0/2143044480 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90e0085b40 con 0x7f90f80d26d0 2023-10-21T17:38:07.795 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:38:07.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 msgr2=0x7f90f80c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:07.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f90e000d800 tx=0x7f90e000dcc0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 shutdown_connections 2023-10-21T17:38:07.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:6832/1168372736,v1:172.21.15.124:6833/1168372736] conn(0x7f90d806afe0 0x7f90d806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90f815cb80 0x7f90f8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f80d26d0 0x7f90f80c69a0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 --2- 172.21.15.53:0/2143044480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90f80b85d0 0x7f90f814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:07.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.795+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 >> 172.21.15.53:0/2143044480 conn(0x7f90f80718a0 msgr2=0x7f90f80b7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:07.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.796+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 shutdown_connections 2023-10-21T17:38:07.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:07.796+0000 7f90fe021640 1 -- 172.21.15.53:0/2143044480 wait complete. 2023-10-21T17:38:07.803 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:07.802+0000 7ff678a52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:38:07.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:38:07.923 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:07.922+0000 7ff678a52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:38:08.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.024+0000 7f95a1e6f640 1 Processor -- start 2023-10-21T17:38:08.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.024+0000 7f95a1e6f640 1 -- start start 2023-10-21T17:38:08.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.025+0000 7f95a1e6f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 0x7f959c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.025+0000 7f95a1e6f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f959c065d20 0x7f959c06a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f95a1e6f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0661b0 con 0x7f959c15cee0 2023-10-21T17:38:08.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f95a1e6f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959c15d2c0 con 0x7f959c065d20 2023-10-21T17:38:08.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f95a1e6f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cd670 con 0x7f959c06e5d0 2023-10-21T17:38:08.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f959b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f959c15cee0 0x7f959c06dec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48646/0 (socket says 172.21.15.53:48646) 2023-10-21T17:38:08.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f959b7fe640 1 -- 172.21.15.53:0/555032386 learned_addr learned my addr 172.21.15.53:0/555032386 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:08.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f9592ffd640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 0x7f959c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f959bfff640 1 --2- 172.21.15.53:0/555032386 >> v2:172.21.15.124:3300/0 conn(0x7f959c065d20 0x7f959c06a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388792125 0 0) 0x7f959c0661b0 con 0x7f959c15cee0 2023-10-21T17:38:08.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.026+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9588003520 con 0x7f959c15cee0 2023-10-21T17:38:08.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156646627 0 0) 0x7f9588003520 con 0x7f959c15cee0 2023-10-21T17:38:08.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 msgr2=0x7f959c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 0x7f959c0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 >> v2:172.21.15.124:3300/0 conn(0x7f959c065d20 msgr2=0x7f959c06a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 --2- 172.21.15.53:0/555032386 >> v2:172.21.15.124:3300/0 conn(0x7f959c065d20 0x7f959c06a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0cd950 con 0x7f959c15cee0 2023-10-21T17:38:08.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f958c0019f0 con 0x7f959c15cee0 2023-10-21T17:38:08.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 >> v1:172.21.15.53:6789/0 conn(0x7f959c15cee0 legacy=0x7f959c06dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f9588003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95880047c0 0x7f9588004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 0x7f9588005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c15d2c0 con 0x7f9588003970 2023-10-21T17:38:08.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cd670 con 0x7f95880047c0 2023-10-21T17:38:08.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958800b980 con 0x7f959c06e5d0 2023-10-21T17:38:08.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.027+0000 7f959b7fe640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f9588003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.028+0000 7f959b7fe640 1 -- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 msgr2=0x7f9588005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.028+0000 7f959b7fe640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c06e5d0 0x7f9588005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.028+0000 7f959b7fe640 1 -- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95880047c0 msgr2=0x7f9588004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.028+0000 7f959b7fe640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95880047c0 0x7f9588004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.028+0000 7f959b7fe640 1 -- 172.21.15.53:0/555032386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588003520 con 0x7f9588003970 2023-10-21T17:38:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f959b7fe640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f9588003d90 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a860 tx=0x7f95880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3229e9f372d5f1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c034630 con 0x7f9588003970 2023-10-21T17:38:08.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c02f070 con 0x7f9588003970 2023-10-21T17:38:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f959affd640 1 -- 172.21.15.53:0/555032386 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c0349b0 con 0x7f9588003970 2023-10-21T17:38:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f95a1e6f640 1 -- 172.21.15.53:0/555032386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 msgr2=0x7f9588003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f95a1e6f640 1 --2- 172.21.15.53:0/555032386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f9588003d90 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a860 tx=0x7f95880057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.029+0000 7f959bfff640 1 -- 172.21.15.53:0/555032386 reap_dead start 2023-10-21T17:38:08.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.030+0000 7f95a1e6f640 1 -- 172.21.15.53:0/555032386 shutdown_connections 2023-10-21T17:38:08.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.030+0000 7f95a1e6f640 1 -- 172.21.15.53:0/555032386 >> 172.21.15.53:0/555032386 conn(0x7f959c0cba60 msgr2=0x7f959c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:08.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.030+0000 7f95a1e6f640 1 -- 172.21.15.53:0/555032386 shutdown_connections 2023-10-21T17:38:08.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.030+0000 7f95a1e6f640 1 -- 172.21.15.53:0/555032386 wait complete. 2023-10-21T17:38:08.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.031+0000 7f95a1e6f640 1 Processor -- start 2023-10-21T17:38:08.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.032+0000 7f95a1e6f640 1 -- start start 2023-10-21T17:38:08.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c1f2c80 0x7f959c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 0x7f959c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c15ee30 con 0x7f9588003970 2023-10-21T17:38:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c15eb50 con 0x7f959c1f3210 2023-10-21T17:38:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.033+0000 7f95a1e6f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0c7770 con 0x7f959c1f2c80 2023-10-21T17:38:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45790/0 (socket says 172.21.15.53:45790) 2023-10-21T17:38:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 -- 172.21.15.53:0/4138965274 learned_addr learned my addr 172.21.15.53:0/4138965274 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:08.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959bfff640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 0x7f959c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 -- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c1f2c80 msgr2=0x7f959c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c1f2c80 0x7f959c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 -- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 msgr2=0x7f959c0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 0x7f959c0af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959b7fe640 1 -- 172.21.15.53:0/4138965274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0c7a50 con 0x7f9588003970 2023-10-21T17:38:08.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.034+0000 7f959bfff640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 0x7f959c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:08.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f959b7fe640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a980 tx=0x7f958c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c005030 con 0x7f9588003970 2023-10-21T17:38:08.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c005190 con 0x7f9588003970 2023-10-21T17:38:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c005500 con 0x7f9588003970 2023-10-21T17:38:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c0c7d30 con 0x7f9588003970 2023-10-21T17:38:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.035+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c1f36c0 con 0x7f9588003970 2023-10-21T17:38:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.036+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c0c7ef0 con 0x7f9588003970 2023-10-21T17:38:08.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.037+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 123) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f958c031030 con 0x7f9588003970 2023-10-21T17:38:08.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.039+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f958c0abb90 con 0x7f9588003970 2023-10-21T17:38:08.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.043+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f958c0768c0 con 0x7f9588003970 2023-10-21T17:38:08.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.238+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f959c0c7ef0 con 0x7f9588003970 2023-10-21T17:38:08.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.239+0000 7f9598ff9640 1 -- 172.21.15.53:0/4138965274 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 104+0+11 (secure 0 0 0) 0x7f958c07a0b0 con 0x7f9588003970 2023-10-21T17:38:08.241 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:38:08.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 msgr2=0x7f959c1f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a980 tx=0x7f958c001d80 comp rx=0 tx=0).stop 2023-10-21T17:38:08.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 shutdown_connections 2023-10-21T17:38:08.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f959c1f3210 0x7f959c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f959c1f2c80 0x7f959c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 --2- 172.21.15.53:0/4138965274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9588003970 0x7f959c1f1a90 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 >> 172.21.15.53:0/4138965274 conn(0x7f959c0cba60 msgr2=0x7f959c071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:08.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 shutdown_connections 2023-10-21T17:38:08.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.242+0000 7f95a1e6f640 1 -- 172.21.15.53:0/4138965274 wait complete. 2023-10-21T17:38:08.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:38:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.490+0000 7f80c4a15640 1 Processor -- start 2023-10-21T17:38:08.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.490+0000 7f80c4a15640 1 -- start start 2023-10-21T17:38:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.491+0000 7f80c4a15640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f80c00c9cf0 0x7f80c006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.491+0000 7f80c4a15640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80c015aa00 0x7f80c00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.491+0000 7f80c4a15640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c015b310 con 0x7f80c006c3a0 2023-10-21T17:38:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.491+0000 7f80c4a15640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c006e0a0 con 0x7f80c015aa00 2023-10-21T17:38:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.491+0000 7f80c4a15640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c006d290 con 0x7f80c00c9cf0 2023-10-21T17:38:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80c015aa00 0x7f80c00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f80c015aa00 0x7f80c00b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45794/0 (socket says 172.21.15.53:45794) 2023-10-21T17:38:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 -- 172.21.15.53:0/1949504143 learned_addr learned my addr 172.21.15.53:0/1949504143 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:08.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80b5d9b640 1 --2- 172.21.15.53:0/1949504143 >> v2:172.21.15.124:3300/0 conn(0x7f80c00c9cf0 0x7f80c006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 -- 172.21.15.53:0/1949504143 >> v2:172.21.15.124:3300/0 conn(0x7f80c00c9cf0 msgr2=0x7f80c006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 --2- 172.21.15.53:0/1949504143 >> v2:172.21.15.124:3300/0 conn(0x7f80c00c9cf0 0x7f80c006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 -- 172.21.15.53:0/1949504143 >> v1:172.21.15.124:6789/0 conn(0x7f80c006c3a0 legacy=0x7f80c006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.492+0000 7f80bed9d640 1 -- 172.21.15.53:0/1949504143 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c00c9860 con 0x7f80c015aa00 2023-10-21T17:38:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.494+0000 7f80bed9d640 1 --2- 172.21.15.53:0/1949504143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c015aa00 0x7f80c00b2a90 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f80c00715c0 tx=0x7f80b802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682e1c33b5c4347e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.494+0000 7f80bdd9b640 1 -- 172.21.15.53:0/1949504143 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b802ea90 con 0x7f80c015aa00 2023-10-21T17:38:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.494+0000 7f80bdd9b640 1 -- 172.21.15.53:0/1949504143 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b802ebf0 con 0x7f80c015aa00 2023-10-21T17:38:08.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.494+0000 7f80bdd9b640 1 -- 172.21.15.53:0/1949504143 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8009600 con 0x7f80c015aa00 2023-10-21T17:38:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 -- 172.21.15.53:0/1949504143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c015aa00 msgr2=0x7f80c00b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 --2- 172.21.15.53:0/1949504143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c015aa00 0x7f80c00b2a90 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f80c00715c0 tx=0x7f80b802e0f0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 -- 172.21.15.53:0/1949504143 shutdown_connections 2023-10-21T17:38:08.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 --2- 172.21.15.53:0/1949504143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c015aa00 0x7f80c00b2a90 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 --2- 172.21.15.53:0/1949504143 >> v2:172.21.15.124:3300/0 conn(0x7f80c00c9cf0 0x7f80c006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 -- 172.21.15.53:0/1949504143 >> 172.21.15.53:0/1949504143 conn(0x7f80c006a5d0 msgr2=0x7f80c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:08.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 -- 172.21.15.53:0/1949504143 shutdown_connections 2023-10-21T17:38:08.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.495+0000 7f80c4a15640 1 -- 172.21.15.53:0/1949504143 wait complete. 2023-10-21T17:38:08.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.497+0000 7f80c4a15640 1 Processor -- start 2023-10-21T17:38:08.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.498+0000 7f80c4a15640 1 -- start start 2023-10-21T17:38:08.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80c006c3a0 0x7f80c015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80c015aa00 0x7f80c01f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00c6590 con 0x7f80c00c9cf0 2023-10-21T17:38:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00c6870 con 0x7f80c015aa00 2023-10-21T17:38:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80c4a15640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00c6b50 con 0x7f80c006c3a0 2023-10-21T17:38:08.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80b5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80b5d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45802/0 (socket says 172.21.15.53:45802) 2023-10-21T17:38:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.499+0000 7f80b5d9b640 1 -- 172.21.15.53:0/3823398340 learned_addr learned my addr 172.21.15.53:0/3823398340 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80bed9d640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80c015aa00 0x7f80c01f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 -- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80c006c3a0 msgr2=0x7f80c015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80be59c640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80c006c3a0 0x7f80c015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80c006c3a0 0x7f80c015c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 -- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80c015aa00 msgr2=0x7f80c01f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80c015aa00 0x7f80c01f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 -- 172.21.15.53:0/3823398340 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b800bd40 con 0x7f80c00c9cf0 2023-10-21T17:38:08.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b5d9b640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d800 tx=0x7f80a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.500+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800b6c0 con 0x7f80c00c9cf0 2023-10-21T17:38:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.501+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a800b820 con 0x7f80c00c9cf0 2023-10-21T17:38:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.501+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800bbb0 con 0x7f80c00c9cf0 2023-10-21T17:38:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.501+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c00c6e30 con 0x7f80c00c9cf0 2023-10-21T17:38:08.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.501+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c0156630 con 0x7f80c00c9cf0 2023-10-21T17:38:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.502+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c004e160 con 0x7f80c00c9cf0 2023-10-21T17:38:08.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.503+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 123) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f80a800bd10 con 0x7f80c00c9cf0 2023-10-21T17:38:08.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.504+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f80a8089c00 con 0x7f80c00c9cf0 2023-10-21T17:38:08.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.510+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80a8054b00 con 0x7f80c00c9cf0 2023-10-21T17:38:08.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.665+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 124) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f80a80545b0 con 0x7f80c00c9cf0 2023-10-21T17:38:08.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.665+0000 7f80b77fe640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 0x7f80a0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.673+0000 7f80be59c640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 0x7f80a0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.684+0000 7f80be59c640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 0x7f80a0073390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f80b0003d90 tx=0x7f80b0009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.716+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80c00c66d0 con 0x7f80c00c9cf0 2023-10-21T17:38:08.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.717+0000 7f80b77fe640 1 -- 172.21.15.53:0/3823398340 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v221) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80c00c66d0 con 0x7f80c00c9cf0 2023-10-21T17:38:08.719 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:38:08.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.720+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 msgr2=0x7f80a0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.720+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 0x7f80a0073390 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f80b0003d90 tx=0x7f80b0009040 comp rx=0 tx=0).stop 2023-10-21T17:38:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.720+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 msgr2=0x7f80c0150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.720+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f80a800d800 tx=0x7f80a800dcc0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 shutdown_connections 2023-10-21T17:38:08.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80c015aa00 0x7f80c01f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80c00c9cf0 0x7f80c0150fb0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f80a0070fa0 0x7f80a0073390 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 --2- 172.21.15.53:0/3823398340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80c006c3a0 0x7f80c015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 >> 172.21.15.53:0/3823398340 conn(0x7f80c006a5d0 msgr2=0x7f80c00c1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 shutdown_connections 2023-10-21T17:38:08.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.721+0000 7f80c4a15640 1 -- 172.21.15.53:0/3823398340 wait complete. 2023-10-21T17:38:08.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:08.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.966+0000 7f8420bcb640 1 Processor -- start 2023-10-21T17:38:08.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.966+0000 7f8420bcb640 1 -- start start 2023-10-21T17:38:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.967+0000 7f8420bcb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f841c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.967+0000 7f8420bcb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c0c9f90 con 0x7f841c0c97d0 2023-10-21T17:38:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.967+0000 7f8420bcb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c0ca270 con 0x7f841c15d550 2023-10-21T17:38:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.967+0000 7f8420bcb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c071550 con 0x7f841c150440 2023-10-21T17:38:08.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.968+0000 7f8413fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f841c0c97d0 0x7f841c15ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48650/0 (socket says 172.21.15.53:48650) 2023-10-21T17:38:08.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.968+0000 7f8413fff640 1 -- 172.21.15.53:0/2394160607 learned_addr learned my addr 172.21.15.53:0/2394160607 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.968+0000 7f841b7fe640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f841c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.968+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263181909 0 0) 0x7f841c0c9f90 con 0x7f841c0c97d0 2023-10-21T17:38:08.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.968+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8408003520 con 0x7f841c0c97d0 2023-10-21T17:38:08.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877089606 0 0) 0x7f8408003520 con 0x7f841c0c97d0 2023-10-21T17:38:08.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 msgr2=0x7f841c071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f841c071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 >> v1:172.21.15.124:6789/0 conn(0x7f841c15d550 legacy=0x7f841c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0ca550 con 0x7f841c0c97d0 2023-10-21T17:38:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8404001690 con 0x7f841c0c97d0 2023-10-21T17:38:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 >> v1:172.21.15.53:6789/0 conn(0x7f841c0c97d0 legacy=0x7f841c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f84080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8408003f70 0x7f84080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f8408005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840800ba90 con 0x7f8408004d00 2023-10-21T17:38:08.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840800bc00 con 0x7f8408003f70 2023-10-21T17:38:08.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840800bdc0 con 0x7f841c150440 2023-10-21T17:38:08.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.969+0000 7f841bfff640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f8408005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841b7fe640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f84080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f8413fff640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8408003f70 0x7f84080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841bfff640 1 -- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 msgr2=0x7f84080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841bfff640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c150440 0x7f84080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841bfff640 1 -- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8408003f70 msgr2=0x7f84080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:08.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841bfff640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8408003f70 0x7f84080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.970+0000 7f841bfff640 1 -- 172.21.15.53:0/2394160607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408003520 con 0x7f8408004d00 2023-10-21T17:38:08.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.971+0000 7f841bfff640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f8408005180 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f8414001e50 tx=0x7f84080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3ba3620bdc1772 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.971+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414003b70 con 0x7f8408004d00 2023-10-21T17:38:08.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.971+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8414003cd0 con 0x7f8408004d00 2023-10-21T17:38:08.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.971+0000 7f841affd640 1 -- 172.21.15.53:0/2394160607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414005310 con 0x7f8408004d00 2023-10-21T17:38:08.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 -- 172.21.15.53:0/2394160607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 msgr2=0x7f8408005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 --2- 172.21.15.53:0/2394160607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f8408005180 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f8414001e50 tx=0x7f84080057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f841bfff640 1 -- 172.21.15.53:0/2394160607 reap_dead start 2023-10-21T17:38:08.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 -- 172.21.15.53:0/2394160607 shutdown_connections 2023-10-21T17:38:08.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 -- 172.21.15.53:0/2394160607 >> 172.21.15.53:0/2394160607 conn(0x7f841c0cb0c0 msgr2=0x7f841c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:08.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 -- 172.21.15.53:0/2394160607 shutdown_connections 2023-10-21T17:38:08.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.972+0000 7f8420bcb640 1 -- 172.21.15.53:0/2394160607 wait complete. 2023-10-21T17:38:08.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.974+0000 7f8420bcb640 1 Processor -- start 2023-10-21T17:38:08.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.974+0000 7f8420bcb640 1 -- start start 2023-10-21T17:38:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 0x7f841c1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c0be620 0x7f841c0beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0befe0 con 0x7f8408004d00 2023-10-21T17:38:08.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1438f0 con 0x7f841c144120 2023-10-21T17:38:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.975+0000 7f8420bcb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c146760 con 0x7f841c0be620 2023-10-21T17:38:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45832/0 (socket says 172.21.15.53:45832) 2023-10-21T17:38:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 -- 172.21.15.53:0/3231558082 learned_addr learned my addr 172.21.15.53:0/3231558082 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f8413fff640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 0x7f841c1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:08.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 -- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c0be620 msgr2=0x7f841c0beaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c0be620 0x7f841c0beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 -- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 msgr2=0x7f841c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:08.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 0x7f841c1433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:08.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 -- 172.21.15.53:0/3231558082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c1446e0 con 0x7f8408004d00 2023-10-21T17:38:08.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f8413fff640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 0x7f841c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:08.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.976+0000 7f841b7fe640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f840c011580 tx=0x7f840c011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:08.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.977+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c015870 con 0x7f8408004d00 2023-10-21T17:38:08.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.977+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c0159d0 con 0x7f8408004d00 2023-10-21T17:38:08.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.977+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c015d40 con 0x7f8408004d00 2023-10-21T17:38:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.977+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c1425f0 con 0x7f8408004d00 2023-10-21T17:38:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.977+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c142bb0 con 0x7f8408004d00 2023-10-21T17:38:09.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.978+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c04e160 con 0x7f8408004d00 2023-10-21T17:38:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.979+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 124) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f840c00f460 con 0x7f8408004d00 2023-10-21T17:38:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.980+0000 7f8418ff9640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 0x7f83fc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.980+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f840c089cb0 con 0x7f8408004d00 2023-10-21T17:38:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.981+0000 7f8413fff640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 0x7f83fc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.982+0000 7f8413fff640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 0x7f83fc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f841c15b5e0 tx=0x7f8404000ff0 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:08.985+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f840c054a40 con 0x7f8408004d00 2023-10-21T17:38:09.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.216+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f841c0b45c0 con 0x7f8408004d00 2023-10-21T17:38:09.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.217+0000 7f8418ff9640 1 -- 172.21.15.53:0/3231558082 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f840c058060 con 0x7f8408004d00 2023-10-21T17:38:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:09.219 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:38:09.219 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:38:09.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 msgr2=0x7f83fc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 0x7f83fc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f841c15b5e0 tx=0x7f8404000ff0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 msgr2=0x7f841c0bda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f840c011580 tx=0x7f840c011a40 comp rx=0 tx=0).stop 2023-10-21T17:38:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 shutdown_connections 2023-10-21T17:38:09.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f83fc06acc0 0x7f83fc06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f841c0be620 0x7f841c0beaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f841c144120 0x7f841c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 --2- 172.21.15.53:0/3231558082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8408004d00 0x7f841c0bda30 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 >> 172.21.15.53:0/3231558082 conn(0x7f841c0cb0c0 msgr2=0x7f841c1582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:09.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 shutdown_connections 2023-10-21T17:38:09.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.220+0000 7f8420bcb640 1 -- 172.21.15.53:0/3231558082 wait complete. 2023-10-21T17:38:09.242 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.463+0000 7faccb267640 1 Processor -- start 2023-10-21T17:38:09.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.463+0000 7faccb267640 1 -- start start 2023-10-21T17:38:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.465+0000 7faccb267640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 0x7facc415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.465+0000 7faccb267640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.465+0000 7faccb267640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc415f7d0 con 0x7facc40adc70 2023-10-21T17:38:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.465+0000 7faccb267640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc415f210 con 0x7facc415ab30 2023-10-21T17:38:09.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.465+0000 7faccb267640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc415f4f0 con 0x7facc40b07d0 2023-10-21T17:38:09.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7facca265640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 0x7facc415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35580/0 (socket says 172.21.15.53:35580) 2023-10-21T17:38:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 -- 172.21.15.53:0/941509675 learned_addr learned my addr 172.21.15.53:0/941509675 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7facca265640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 0x7facc415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:55816/0 (socket says 172.21.15.53:55816) 2023-10-21T17:38:09.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 -- 172.21.15.53:0/941509675 >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 msgr2=0x7facc415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:09.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 --2- 172.21.15.53:0/941509675 >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 0x7facc415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 -- 172.21.15.53:0/941509675 >> v1:172.21.15.124:6789/0 conn(0x7facc40adc70 legacy=0x7facc40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.466+0000 7faccaa66640 1 -- 172.21.15.53:0/941509675 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc406c550 con 0x7facc40b07d0 2023-10-21T17:38:09.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.468+0000 7faccaa66640 1 --2- 172.21.15.53:0/941509675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7facbc009e50 tx=0x7facbc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7afd6e2d3cdb459 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.468+0000 7facc9263640 1 -- 172.21.15.53:0/941509675 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc02cb00 con 0x7facc40b07d0 2023-10-21T17:38:09.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.468+0000 7facc9263640 1 -- 172.21.15.53:0/941509675 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc02cc60 con 0x7facc40b07d0 2023-10-21T17:38:09.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.468+0000 7facc9263640 1 -- 172.21.15.53:0/941509675 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0343f0 con 0x7facc40b07d0 2023-10-21T17:38:09.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 -- 172.21.15.53:0/941509675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 msgr2=0x7facc406cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 --2- 172.21.15.53:0/941509675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7facbc009e50 tx=0x7facbc02bf60 comp rx=0 tx=0).stop 2023-10-21T17:38:09.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 -- 172.21.15.53:0/941509675 shutdown_connections 2023-10-21T17:38:09.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 --2- 172.21.15.53:0/941509675 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40b07d0 0x7facc406cd50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 --2- 172.21.15.53:0/941509675 >> v2:172.21.15.124:3300/0 conn(0x7facc415ab30 0x7facc415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 -- 172.21.15.53:0/941509675 >> 172.21.15.53:0/941509675 conn(0x7facc40cb0c0 msgr2=0x7facc40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:09.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 -- 172.21.15.53:0/941509675 shutdown_connections 2023-10-21T17:38:09.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.469+0000 7faccb267640 1 -- 172.21.15.53:0/941509675 wait complete. 2023-10-21T17:38:09.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.472+0000 7faccb267640 1 Processor -- start 2023-10-21T17:38:09.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.472+0000 7faccb267640 1 -- start start 2023-10-21T17:38:09.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40adc70 0x7facc414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7facc415ab30 0x7facc40c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c6e90 con 0x7facc40b07d0 2023-10-21T17:38:09.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c7170 con 0x7facc415ab30 2023-10-21T17:38:09.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.473+0000 7faccb267640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c7450 con 0x7facc40adc70 2023-10-21T17:38:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45838/0 (socket says 172.21.15.53:45838) 2023-10-21T17:38:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 -- 172.21.15.53:0/1133158661 learned_addr learned my addr 172.21.15.53:0/1133158661 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7faccaa66640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7facc415ab30 0x7facc40c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facca265640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40adc70 0x7facc414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 -- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40adc70 msgr2=0x7facc414a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40adc70 0x7facc414a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 -- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7facc415ab30 msgr2=0x7facc40c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7facc415ab30 0x7facc40c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 -- 172.21.15.53:0/1133158661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc009b30 con 0x7facc40b07d0 2023-10-21T17:38:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.474+0000 7facc9a64640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7facb400c450 tx=0x7facb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.475+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb40052d0 con 0x7facc40b07d0 2023-10-21T17:38:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.475+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb4005430 con 0x7facc40b07d0 2023-10-21T17:38:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.475+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4012870 con 0x7facc40b07d0 2023-10-21T17:38:09.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.475+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c7730 con 0x7facc40b07d0 2023-10-21T17:38:09.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.475+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc40c8470 con 0x7facc40b07d0 2023-10-21T17:38:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.476+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc404e160 con 0x7facc40b07d0 2023-10-21T17:38:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.477+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 124) v1 ==== 89067+0+0 (secure 0 0 0) 0x7facb40129d0 con 0x7facc40b07d0 2023-10-21T17:38:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.478+0000 7facbaffd640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 0x7faca806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.478+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7facb4089340 con 0x7facc40b07d0 2023-10-21T17:38:09.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.479+0000 7facca265640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 0x7faca806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.480+0000 7facca265640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 0x7faca806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7facc0009970 tx=0x7facc0008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.485+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facb4054150 con 0x7facc40b07d0 2023-10-21T17:38:09.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.677+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 125) v1 ==== 89275+0+0 (secure 0 0 0) 0x7facb4054520 con 0x7facc40b07d0 2023-10-21T17:38:09.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.684+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facc40ada50 con 0x7facc40b07d0 2023-10-21T17:38:09.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.687+0000 7facbaffd640 1 -- 172.21.15.53:0/1133158661 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+18212 (secure 0 0 0) 0x7facb4074660 con 0x7facc40b07d0 2023-10-21T17:38:09.689 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:09.689 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":221,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:07.636340+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":220,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.18","mappings":[{"from":3,"to":6}]},{"pgid":"26.23","mappings":[{"from":1,"to":6}]},{"pgid":"26.28","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1098482368":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2555657847":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2763307861":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/1724708130":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2265843370":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2852594978":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6832/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:6832/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/2759452384":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/863767263":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1944369314":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1803654246":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/820409426":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/102947359":"2023-10-22T17:35:59.400288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:09.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 msgr2=0x7faca806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 0x7faca806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7facc0009970 tx=0x7facc0008040 comp rx=0 tx=0).stop 2023-10-21T17:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 msgr2=0x7facc415da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7facb400c450 tx=0x7facb400c910 comp rx=0 tx=0).stop 2023-10-21T17:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 shutdown_connections 2023-10-21T17:38:09.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7facc415ab30 0x7facc40c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc40b07d0 0x7facc415da50 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faca806acc0 0x7faca806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 --2- 172.21.15.53:0/1133158661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7facc40adc70 0x7facc414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 >> 172.21.15.53:0/1133158661 conn(0x7facc40cb0c0 msgr2=0x7facc40cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:09.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 shutdown_connections 2023-10-21T17:38:09.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.690+0000 7faccb267640 1 -- 172.21.15.53:0/1133158661 wait complete. 2023-10-21T17:38:09.711 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:38:09.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.935+0000 7f1f2f3f5640 1 Processor -- start 2023-10-21T17:38:09.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.935+0000 7f1f2f3f5640 1 -- start start 2023-10-21T17:38:09.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2f3f5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 0x7f1f280cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2f3f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f280cb3b0 0x7f1f2815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2f3f5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2815b2b0 con 0x7f1f2815ab40 2023-10-21T17:38:09.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2f3f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2815bf30 con 0x7f1f280cb3b0 2023-10-21T17:38:09.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2f3f5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f280cb820 con 0x7f1f280c90b0 2023-10-21T17:38:09.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2d992640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f280cb3b0 0x7f1f2815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2d992640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1f280cb3b0 0x7f1f2815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45840/0 (socket says 172.21.15.53:45840) 2023-10-21T17:38:09.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2d992640 1 -- 172.21.15.53:0/3829649396 learned_addr learned my addr 172.21.15.53:0/3829649396 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:09.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.938+0000 7f1f2d992640 1 -- 172.21.15.53:0/3829649396 >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 msgr2=0x7f1f280cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:09.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.937+0000 7f1f2c990640 1 --2- 172.21.15.53:0/3829649396 >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 0x7f1f280cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.938+0000 7f1f2d992640 1 --2- 172.21.15.53:0/3829649396 >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 0x7f1f280cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.938+0000 7f1f2d992640 1 -- 172.21.15.53:0/3829649396 >> v1:172.21.15.124:6789/0 conn(0x7f1f2815ab40 legacy=0x7f1f280ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:09.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.938+0000 7f1f2d992640 1 -- 172.21.15.53:0/3829649396 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2815b590 con 0x7f1f280cb3b0 2023-10-21T17:38:09.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.938+0000 7f1f2c990640 1 --2- 172.21.15.53:0/3829649396 >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 0x7f1f280cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:09.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.939+0000 7f1f2d992640 1 --2- 172.21.15.53:0/3829649396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280cb3b0 0x7f1f2815e790 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f1f2806e560 tx=0x7f1f2002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fbc943a6c80f0a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.939+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3829649396 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2002cb10 con 0x7f1f280cb3b0 2023-10-21T17:38:09.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.939+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3829649396 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2002cc70 con 0x7f1f280cb3b0 2023-10-21T17:38:09.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.939+0000 7f1f1ffff640 1 -- 172.21.15.53:0/3829649396 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200343f0 con 0x7f1f280cb3b0 2023-10-21T17:38:09.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/3829649396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280cb3b0 msgr2=0x7f1f2815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/3829649396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280cb3b0 0x7f1f2815e790 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f1f2806e560 tx=0x7f1f2002bf70 comp rx=0 tx=0).stop 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/3829649396 shutdown_connections 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/3829649396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280cb3b0 0x7f1f2815e790 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/3829649396 >> v2:172.21.15.124:3300/0 conn(0x7f1f280c90b0 0x7f1f280cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/3829649396 >> 172.21.15.53:0/3829649396 conn(0x7f1f2806a380 msgr2=0x7f1f280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:09.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/3829649396 shutdown_connections 2023-10-21T17:38:09.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.940+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/3829649396 wait complete. 2023-10-21T17:38:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.942+0000 7f1f2f3f5640 1 Processor -- start 2023-10-21T17:38:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.942+0000 7f1f2f3f5640 1 -- start start 2023-10-21T17:38:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cb3b0 0x7f1f280c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 0x7f1f28148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c8760 con 0x7f1f280c90b0 2023-10-21T17:38:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2815dcd0 con 0x7f1f2815ab40 2023-10-21T17:38:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2f3f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c5ee0 con 0x7f1f280cb3b0 2023-10-21T17:38:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2d191640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2d191640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45844/0 (socket says 172.21.15.53:45844) 2023-10-21T17:38:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.943+0000 7f1f2d191640 1 -- 172.21.15.53:0/1580919717 learned_addr learned my addr 172.21.15.53:0/1580919717 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d992640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 0x7f1f28148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 -- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cb3b0 msgr2=0x7f1f280c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2c990640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cb3b0 0x7f1f280c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cb3b0 0x7f1f280c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 -- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 msgr2=0x7f1f28148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 0x7f1f28148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 -- 172.21.15.53:0/1580919717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20009b30 con 0x7f1f280c90b0 2023-10-21T17:38:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d992640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 0x7f1f28148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:09.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f2d191640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c450 tx=0x7f1f2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.944+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240052d0 con 0x7f1f280c90b0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.945+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24005430 con 0x7f1f280c90b0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.945+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2815d430 con 0x7f1f280c90b0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.945+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24012870 con 0x7f1f280c90b0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.945+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2815d9f0 con 0x7f1f280c90b0 2023-10-21T17:38:09.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.946+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f280cc140 con 0x7f1f280c90b0 2023-10-21T17:38:09.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.947+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 125) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1f240129d0 con 0x7f1f280c90b0 2023-10-21T17:38:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.948+0000 7f1f1dffb640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.948+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f1f240896d0 con 0x7f1f280c90b0 2023-10-21T17:38:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.949+0000 7f1f2c990640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:09.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.950+0000 7f1f2c990640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f280cf2a0 tx=0x7f1f18008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:09.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:09.952+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f24054210 con 0x7f1f280c90b0 2023-10-21T17:38:10.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.187+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f28067490 con 0x7f1f280c90b0 2023-10-21T17:38:10.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.189+0000 7f1f1dffb640 1 -- 172.21.15.53:0/1580919717 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f1f24057830 con 0x7f1f280c90b0 2023-10-21T17:38:10.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:38:10.191 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:38:10.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 msgr2=0x7f1f0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f280cf2a0 tx=0x7f1f18008040 comp rx=0 tx=0).stop 2023-10-21T17:38:10.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 msgr2=0x7f1f280c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c450 tx=0x7f1f2400c910 comp rx=0 tx=0).stop 2023-10-21T17:38:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 shutdown_connections 2023-10-21T17:38:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2815ab40 0x7f1f28148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1f0c06af00 0x7f1f0c06d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cb3b0 0x7f1f280c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.192+0000 7f1f2f3f5640 1 --2- 172.21.15.53:0/1580919717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f280c90b0 0x7f1f280c5850 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.193+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 >> 172.21.15.53:0/1580919717 conn(0x7f1f2806a380 msgr2=0x7f1f280c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.193+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 shutdown_connections 2023-10-21T17:38:10.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.193+0000 7f1f2f3f5640 1 -- 172.21.15.53:0/1580919717 wait complete. 2023-10-21T17:38:10.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:10.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.442+0000 7f71191a9640 1 Processor -- start 2023-10-21T17:38:10.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.442+0000 7f71191a9640 1 -- start start 2023-10-21T17:38:10.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f71191a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f71140ad790 0x7f71140702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f71191a9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711406dee0 con 0x7f7114159ac0 2023-10-21T17:38:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f71191a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711415d850 con 0x7f711415e330 2023-10-21T17:38:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f71191a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711406dcf0 con 0x7f71140ad790 2023-10-21T17:38:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f7112d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7114159ac0 0x7f711406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48658/0 (socket says 172.21.15.53:48658) 2023-10-21T17:38:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f7112d9d640 1 -- 172.21.15.53:0/150530622 learned_addr learned my addr 172.21.15.53:0/150530622 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.444+0000 7f710a59c640 1 --2- 172.21.15.53:0/150530622 >> v2:172.21.15.53:3300/0 conn(0x7f71140ad790 0x7f71140702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.445+0000 7f710a59c640 1 -- 172.21.15.53:0/150530622 >> v1:172.21.15.124:6789/0 conn(0x7f711415e330 legacy=0x7f71140c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.445+0000 7f710a59c640 1 -- 172.21.15.53:0/150530622 >> v1:172.21.15.53:6789/0 conn(0x7f7114159ac0 legacy=0x7f711406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.445+0000 7f710a59c640 1 -- 172.21.15.53:0/150530622 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711415db30 con 0x7f71140ad790 2023-10-21T17:38:10.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.446+0000 7f710a59c640 1 --2- 172.21.15.53:0/150530622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140702d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009d90 tx=0x7f70fc02e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024bfb0f8ba6228 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f711259c640 1 -- 172.21.15.53:0/150530622 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc036440 con 0x7f71140ad790 2023-10-21T17:38:10.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f711259c640 1 -- 172.21.15.53:0/150530622 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc0365a0 con 0x7f71140ad790 2023-10-21T17:38:10.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f711259c640 1 -- 172.21.15.53:0/150530622 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc036930 con 0x7f71140ad790 2023-10-21T17:38:10.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f71191a9640 1 -- 172.21.15.53:0/150530622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 msgr2=0x7f71140702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f71191a9640 1 --2- 172.21.15.53:0/150530622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140702d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009d90 tx=0x7f70fc02e150 comp rx=0 tx=0).stop 2023-10-21T17:38:10.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f71191a9640 1 -- 172.21.15.53:0/150530622 shutdown_connections 2023-10-21T17:38:10.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f71191a9640 1 --2- 172.21.15.53:0/150530622 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140702d0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.447+0000 7f71191a9640 1 -- 172.21.15.53:0/150530622 >> 172.21.15.53:0/150530622 conn(0x7f711406a380 msgr2=0x7f71140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:10.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.448+0000 7f71191a9640 1 -- 172.21.15.53:0/150530622 shutdown_connections 2023-10-21T17:38:10.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.448+0000 7f71191a9640 1 -- 172.21.15.53:0/150530622 wait complete. 2023-10-21T17:38:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.449+0000 7f71191a9640 1 Processor -- start 2023-10-21T17:38:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.449+0000 7f71191a9640 1 -- start start 2023-10-21T17:38:10.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.450+0000 7f71191a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.450+0000 7f71191a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 0x7f7114151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.450+0000 7f71191a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711415e330 0x7f71140b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f71191a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140b19e0 con 0x7f71140ad790 2023-10-21T17:38:10.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f71191a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711415ca00 con 0x7f711415e330 2023-10-21T17:38:10.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f71191a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7114151e60 con 0x7f7114159ac0 2023-10-21T17:38:10.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45876/0 (socket says 172.21.15.53:45876) 2023-10-21T17:38:10.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 -- 172.21.15.53:0/1867693998 learned_addr learned my addr 172.21.15.53:0/1867693998 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:10.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f711359e640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711415e330 0x7f71140b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f710a59c640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 0x7f7114151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 -- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 msgr2=0x7f7114151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 0x7f7114151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 -- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711415e330 msgr2=0x7f71140b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711415e330 0x7f71140b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f7112d9d640 1 -- 172.21.15.53:0/1867693998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009a70 con 0x7f71140ad790 2023-10-21T17:38:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.451+0000 7f710a59c640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 0x7f7114151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f7112d9d640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f710400c4d0 tx=0x7f710400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104005ce0 con 0x7f71140ad790 2023-10-21T17:38:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104005e40 con 0x7f71140ad790 2023-10-21T17:38:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104007270 con 0x7f71140ad790 2023-10-21T17:38:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71140b2800 con 0x7f71140ad790 2023-10-21T17:38:10.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.452+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71140b2dc0 con 0x7f71140ad790 2023-10-21T17:38:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.453+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71140b2fb0 con 0x7f71140ad790 2023-10-21T17:38:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.454+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 125) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f71040073d0 con 0x7f71140ad790 2023-10-21T17:38:10.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.455+0000 7f710bfff640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 0x7f70f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.455+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f7104089030 con 0x7f71140ad790 2023-10-21T17:38:10.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.456+0000 7f710a59c640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 0x7f70f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.457+0000 7f710a59c640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 0x7f70f406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70fc038000 tx=0x7f70fc032000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.460+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7104053c00 con 0x7f71140ad790 2023-10-21T17:38:10.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.693+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71140b45d0 con 0x7f71140ad790 2023-10-21T17:38:10.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.695+0000 7f710bfff640 1 -- 172.21.15.53:0/1867693998 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f7104017020 con 0x7f71140ad790 2023-10-21T17:38:10.697 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:10.697 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":103,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:35:44.664802+0000","modified":"2023-10-21T17:35:47.776951+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10677":{"gid":10677,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.53:6835/542756474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":542756474},{"type":"v1","addr":"172.21.15.53:6835","nonce":542756474}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-21T17:38:10.698 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 103 2023-10-21T17:38:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.697+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 msgr2=0x7f70f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.697+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 0x7f70f406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70fc038000 tx=0x7f70fc032000 comp rx=0 tx=0).stop 2023-10-21T17:38:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.697+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 msgr2=0x7f71140b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.697+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f710400c4d0 tx=0x7f710400c990 comp rx=0 tx=0).stop 2023-10-21T17:38:10.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 shutdown_connections 2023-10-21T17:38:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711415e330 0x7f71140b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f70f406af00 0x7f70f406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7114159ac0 0x7f7114151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 --2- 172.21.15.53:0/1867693998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71140ad790 0x7f71140b14a0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 >> 172.21.15.53:0/1867693998 conn(0x7f711406a380 msgr2=0x7f71140732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:10.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 shutdown_connections 2023-10-21T17:38:10.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.698+0000 7f71191a9640 1 -- 172.21.15.53:0/1867693998 wait complete. 2023-10-21T17:38:10.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.944+0000 7f1d49c02640 1 Processor -- start 2023-10-21T17:38:10.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.944+0000 7f1d49c02640 1 -- start start 2023-10-21T17:38:10.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.945+0000 7f1d49c02640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1d4406b520 0x7f1d4406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.945+0000 7f1d49c02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 0x7f1d4406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d49c02640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440b0340 con 0x7f1d441608e0 2023-10-21T17:38:10.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d49c02640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4406b3a0 con 0x7f1d4406b520 2023-10-21T17:38:10.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d49c02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b0150 con 0x7f1d440af8b0 2023-10-21T17:38:10.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d43fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1d441608e0 0x7f1d44160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48660/0 (socket says 172.21.15.53:48660) 2023-10-21T17:38:10.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d43fff640 1 -- 172.21.15.53:0/131017413 learned_addr learned my addr 172.21.15.53:0/131017413 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507056216 0 0) 0x7f1d440b0340 con 0x7f1d441608e0 2023-10-21T17:38:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d437fe640 1 --2- 172.21.15.53:0/131017413 >> v2:172.21.15.124:3300/0 conn(0x7f1d4406b520 0x7f1d4406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d30003520 con 0x7f1d441608e0 2023-10-21T17:38:10.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.946+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 0x7f1d4406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786633011 0 0) 0x7f1d30003520 con 0x7f1d441608e0 2023-10-21T17:38:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 msgr2=0x7f1d4406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 0x7f1d4406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 >> v2:172.21.15.124:3300/0 conn(0x7f1d4406b520 msgr2=0x7f1d4406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:10.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 --2- 172.21.15.53:0/131017413 >> v2:172.21.15.124:3300/0 conn(0x7f1d4406b520 0x7f1d4406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4406be40 con 0x7f1d441608e0 2023-10-21T17:38:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1d3c0019f0 con 0x7f1d441608e0 2023-10-21T17:38:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 >> v1:172.21.15.53:6789/0 conn(0x7f1d441608e0 legacy=0x7f1d44160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d30003970 0x7f1d30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d300047c0 0x7f1d30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 0x7f1d30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4406b3a0 con 0x7f1d30003970 2023-10-21T17:38:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b0150 con 0x7f1d300047c0 2023-10-21T17:38:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3000ba30 con 0x7f1d440af8b0 2023-10-21T17:38:10.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.947+0000 7f1d437fe640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d30003970 0x7f1d30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d300047c0 0x7f1d30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d437fe640 1 -- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 msgr2=0x7f1d30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:10.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d437fe640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d440af8b0 0x7f1d30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d437fe640 1 -- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d300047c0 msgr2=0x7f1d30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:10.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d437fe640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d300047c0 0x7f1d30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d437fe640 1 -- 172.21.15.53:0/131017413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d30003520 con 0x7f1d30003970 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.948+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d300047c0 0x7f1d30004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.949+0000 7f1d437fe640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d30003970 0x7f1d30003d90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009a80 tx=0x7f1d300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d316ae75a23c17 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.949+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34005990 con 0x7f1d30003970 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.949+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34005af0 con 0x7f1d30003970 2023-10-21T17:38:10.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.949+0000 7f1d42ffd640 1 -- 172.21.15.53:0/131017413 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34005ea0 con 0x7f1d30003970 2023-10-21T17:38:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 -- 172.21.15.53:0/131017413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d30003970 msgr2=0x7f1d30003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 --2- 172.21.15.53:0/131017413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d30003970 0x7f1d30003d90 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009a80 tx=0x7f1d300057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d43fff640 1 -- 172.21.15.53:0/131017413 reap_dead start 2023-10-21T17:38:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 -- 172.21.15.53:0/131017413 shutdown_connections 2023-10-21T17:38:10.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 -- 172.21.15.53:0/131017413 >> 172.21.15.53:0/131017413 conn(0x7f1d440d09e0 msgr2=0x7f1d440d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 -- 172.21.15.53:0/131017413 shutdown_connections 2023-10-21T17:38:10.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.950+0000 7f1d49c02640 1 -- 172.21.15.53:0/131017413 wait complete. 2023-10-21T17:38:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.952+0000 7f1d49c02640 1 Processor -- start 2023-10-21T17:38:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.952+0000 7f1d49c02640 1 -- start start 2023-10-21T17:38:10.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d30003970 0x7f1d440b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 0x7f1d441480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b4d50 con 0x7f1d440c4780 2023-10-21T17:38:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b5030 con 0x7f1d440c4c00 2023-10-21T17:38:10.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d49c02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b5310 con 0x7f1d30003970 2023-10-21T17:38:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45896/0 (socket says 172.21.15.53:45896) 2023-10-21T17:38:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 -- 172.21.15.53:0/2210604222 learned_addr learned my addr 172.21.15.53:0/2210604222 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d43fff640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 0x7f1d441480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d437fe640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d30003970 0x7f1d440b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 -- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d30003970 msgr2=0x7f1d440b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d30003970 0x7f1d440b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 -- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 msgr2=0x7f1d441480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.953+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 0x7f1d441480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:10.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d3bfff640 1 -- 172.21.15.53:0/2210604222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440b55f0 con 0x7f1d440c4780 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d43fff640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 0x7f1d441480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d3bfff640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e7e0 tx=0x7f1d2c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00c6d0 con 0x7f1d440c4780 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c00c830 con 0x7f1d440c4780 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d440b58d0 con 0x7f1d440c4780 2023-10-21T17:38:10.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00cba0 con 0x7f1d440c4780 2023-10-21T17:38:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.954+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d441485e0 con 0x7f1d440c4780 2023-10-21T17:38:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.955+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d44067490 con 0x7f1d440c4780 2023-10-21T17:38:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.956+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 125) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1d2c00cd00 con 0x7f1d440c4780 2023-10-21T17:38:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.957+0000 7f1d40ff9640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 0x7f1d2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:10.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.957+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7f1d2c089b00 con 0x7f1d440c4780 2023-10-21T17:38:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.958+0000 7f1d437fe640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 0x7f1d2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.959+0000 7f1d437fe640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 0x7f1d2406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d34035000 tx=0x7f1d34030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:10.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:10.961+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d2c08be60 con 0x7f1d440c4780 2023-10-21T17:38:11.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.155+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d4408aae0 con 0x7f1d440c4780 2023-10-21T17:38:11.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.159+0000 7f1d40ff9640 1 -- 172.21.15.53:0/2210604222 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+18212 (secure 0 0 0) 0x7f1d2c021090 con 0x7f1d440c4780 2023-10-21T17:38:11.160 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:11.161 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":221,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:07.636340+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:35:42.612998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"214","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-21T17:35:43.682480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":220,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.18","mappings":[{"from":3,"to":6}]},{"pgid":"26.23","mappings":[{"from":1,"to":6}]},{"pgid":"26.28","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/1098482368":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2555657847":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2763307861":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/1724708130":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2265843370":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2852594978":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6832/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:6832/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/2759452384":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/863767263":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1944369314":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1803654246":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/820409426":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/102947359":"2023-10-22T17:35:59.400288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:11.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 msgr2=0x7f1d2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 0x7f1d2406d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d34035000 tx=0x7f1d34030000 comp rx=0 tx=0).stop 2023-10-21T17:38:11.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 msgr2=0x7f1d44147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e7e0 tx=0x7f1d2c00eca0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 shutdown_connections 2023-10-21T17:38:11.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d440c4c00 0x7f1d441480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d440c4780 0x7f1d44147b60 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1d2406af30 0x7f1d2406d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 --2- 172.21.15.53:0/2210604222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d30003970 0x7f1d440b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 >> 172.21.15.53:0/2210604222 conn(0x7f1d440d09e0 msgr2=0x7f1d440add70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:11.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 shutdown_connections 2023-10-21T17:38:11.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.162+0000 7f1d49c02640 1 -- 172.21.15.53:0/2210604222 wait complete. 2023-10-21T17:38:11.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:38:11.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.413+0000 7fa611494640 1 Processor -- start 2023-10-21T17:38:11.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.413+0000 7fa611494640 1 -- start start 2023-10-21T17:38:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa60c0afbe0 0x7fa60c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa60c06dfc0 0x7fa60c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c071070 0x7fa60c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c0b2620 con 0x7fa60c06dfc0 2023-10-21T17:38:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60c0b2060 con 0x7fa60c0afbe0 2023-10-21T17:38:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa611494640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0b2340 con 0x7fa60c071070 2023-10-21T17:38:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa603fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa60c06dfc0 0x7fa60c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa603fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa60c06dfc0 0x7fa60c070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45902/0 (socket says 172.21.15.53:45902) 2023-10-21T17:38:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.415+0000 7fa603fff640 1 -- 172.21.15.53:0/3013177633 learned_addr learned my addr 172.21.15.53:0/3013177633 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa60affd640 1 --2- 172.21.15.53:0/3013177633 >> v2:172.21.15.124:3300/0 conn(0x7fa60c0afbe0 0x7fa60c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa60b7fe640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c071070 0x7fa60c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa603fff640 1 -- 172.21.15.53:0/3013177633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c071070 msgr2=0x7fa60c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa603fff640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c071070 0x7fa60c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa603fff640 1 -- 172.21.15.53:0/3013177633 >> v2:172.21.15.124:3300/0 conn(0x7fa60c0afbe0 msgr2=0x7fa60c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa603fff640 1 --2- 172.21.15.53:0/3013177633 >> v2:172.21.15.124:3300/0 conn(0x7fa60c0afbe0 0x7fa60c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.416+0000 7fa603fff640 1 -- 172.21.15.53:0/3013177633 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c070350 con 0x7fa60c06dfc0 2023-10-21T17:38:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.417+0000 7fa603fff640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c070b30 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06e640 tx=0x7fa5f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02477df13e58bec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.417+0000 7fa60a7fc640 1 -- 172.21.15.53:0/3013177633 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4031870 con 0x7fa60c06dfc0 2023-10-21T17:38:11.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa60a7fc640 1 -- 172.21.15.53:0/3013177633 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f40319d0 con 0x7fa60c06dfc0 2023-10-21T17:38:11.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa60a7fc640 1 -- 172.21.15.53:0/3013177633 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4031d60 con 0x7fa60c06dfc0 2023-10-21T17:38:11.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 -- 172.21.15.53:0/3013177633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 msgr2=0x7fa60c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c070b30 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06e640 tx=0x7fa5f402b910 comp rx=0 tx=0).stop 2023-10-21T17:38:11.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 -- 172.21.15.53:0/3013177633 shutdown_connections 2023-10-21T17:38:11.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c071070 0x7fa60c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 --2- 172.21.15.53:0/3013177633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c070b30 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 --2- 172.21.15.53:0/3013177633 >> v2:172.21.15.124:3300/0 conn(0x7fa60c0afbe0 0x7fa60c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 -- 172.21.15.53:0/3013177633 >> 172.21.15.53:0/3013177633 conn(0x7fa60c06a5d0 msgr2=0x7fa60c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:11.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.418+0000 7fa611494640 1 -- 172.21.15.53:0/3013177633 shutdown_connections 2023-10-21T17:38:11.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.419+0000 7fa611494640 1 -- 172.21.15.53:0/3013177633 wait complete. 2023-10-21T17:38:11.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.420+0000 7fa611494640 1 Processor -- start 2023-10-21T17:38:11.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.421+0000 7fa611494640 1 -- start start 2023-10-21T17:38:11.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 0x7fa60c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c0afbe0 0x7fa60c0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0c3570 con 0x7fa60c06dfc0 2023-10-21T17:38:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0cdd70 con 0x7fa60c071070 2023-10-21T17:38:11.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa611494640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c1f30a0 con 0x7fa60c0afbe0 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45916/0 (socket says 172.21.15.53:45916) 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 -- 172.21.15.53:0/1364421074 learned_addr learned my addr 172.21.15.53:0/1364421074 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa603fff640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 0x7fa60c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 -- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c0afbe0 msgr2=0x7fa60c0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c0afbe0 0x7fa60c0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 -- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 msgr2=0x7fa60c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 0x7fa60c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa60affd640 1 -- 172.21.15.53:0/1364421074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40094d0 con 0x7fa60c06dfc0 2023-10-21T17:38:11.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.422+0000 7fa603fff640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 0x7fa60c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:11.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa60affd640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e870 tx=0x7fa5fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0094f0 con 0x7fa60c06dfc0 2023-10-21T17:38:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00c8e0 con 0x7fa60c06dfc0 2023-10-21T17:38:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00cc90 con 0x7fa60c06dfc0 2023-10-21T17:38:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c1f3530 con 0x7fa60c06dfc0 2023-10-21T17:38:11.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.423+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c1f2a50 con 0x7fa60c06dfc0 2023-10-21T17:38:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.424+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c04e160 con 0x7fa60c06dfc0 2023-10-21T17:38:11.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.425+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 125) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa5fc013470 con 0x7fa60c06dfc0 2023-10-21T17:38:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.426+0000 7fa6037fe640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 0x7fa5ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.427+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7006+0+0 (secure 0 0 0) 0x7fa5fc089c30 con 0x7fa60c06dfc0 2023-10-21T17:38:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.427+0000 7fa603fff640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 0x7fa5ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.428+0000 7fa603fff640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 0x7fa5ec06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403c040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.431+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5fc090080 con 0x7fa60c06dfc0 2023-10-21T17:38:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.663+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa60c0b0ef0 con 0x7fa60c06dfc0 2023-10-21T17:38:11.665 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:38:11.664+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:38:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.668+0000 7fa6037fe640 1 -- 172.21.15.53:0/1364421074 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa5fc054760 con 0x7fa60c06dfc0 2023-10-21T17:38:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:38:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 msgr2=0x7fa5ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 0x7fa5ec06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4036000 tx=0x7fa5f403c040 comp rx=0 tx=0).stop 2023-10-21T17:38:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 msgr2=0x7fa60c1589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e870 tx=0x7fa5fc00ed30 comp rx=0 tx=0).stop 2023-10-21T17:38:11.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 shutdown_connections 2023-10-21T17:38:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa5ec06af00 0x7fa5ec06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa60c0afbe0 0x7fa60c0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa60c071070 0x7fa60c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 --2- 172.21.15.53:0/1364421074 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa60c06dfc0 0x7fa60c1589e0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.672+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 >> 172.21.15.53:0/1364421074 conn(0x7fa60c06a5d0 msgr2=0x7fa60c0c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:11.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.673+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 shutdown_connections 2023-10-21T17:38:11.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.673+0000 7fa611494640 1 -- 172.21.15.53:0/1364421074 wait complete. 2023-10-21T17:38:11.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:38:11.747 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:38:11.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.894+0000 7fbbed00e640 1 Processor -- start 2023-10-21T17:38:11.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.894+0000 7fbbed00e640 1 -- start start 2023-10-21T17:38:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.895+0000 7fbbed00e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbbe806dfc0 0x7fbbe8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.895+0000 7fbbed00e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbbe8071070 0x7fbbe80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.895+0000 7fbbed00e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe80b1cc0 con 0x7fbbe80afbe0 2023-10-21T17:38:11.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.895+0000 7fbbed00e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80b2620 con 0x7fbbe8071070 2023-10-21T17:38:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.895+0000 7fbbed00e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe80b2060 con 0x7fbbe806dfc0 2023-10-21T17:38:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbbe80afbe0 0x7fbbe80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48678/0 (socket says 172.21.15.53:48678) 2023-10-21T17:38:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe6d9d640 1 -- 172.21.15.53:0/465134691 learned_addr learned my addr 172.21.15.53:0/465134691 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:11.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe759e640 1 --2- 172.21.15.53:0/465134691 >> v2:172.21.15.53:3300/0 conn(0x7fbbe8071070 0x7fbbe80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbdffff640 1 --2- 172.21.15.53:0/465134691 >> v2:172.21.15.124:3300/0 conn(0x7fbbe806dfc0 0x7fbbe8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe659c640 1 -- 172.21.15.53:0/465134691 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761160592 0 0) 0x7fbbe80b1cc0 con 0x7fbbe80afbe0 2023-10-21T17:38:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe759e640 1 -- 172.21.15.53:0/465134691 >> v2:172.21.15.124:3300/0 conn(0x7fbbe806dfc0 msgr2=0x7fbbe8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe759e640 1 --2- 172.21.15.53:0/465134691 >> v2:172.21.15.124:3300/0 conn(0x7fbbe806dfc0 0x7fbbe8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe759e640 1 -- 172.21.15.53:0/465134691 >> v1:172.21.15.53:6789/0 conn(0x7fbbe80afbe0 legacy=0x7fbbe80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.896+0000 7fbbe759e640 1 -- 172.21.15.53:0/465134691 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80b2340 con 0x7fbbe8071070 2023-10-21T17:38:11.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.897+0000 7fbbe759e640 1 --2- 172.21.15.53:0/465134691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80cb130 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b3980 tx=0x7fbbe002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba2d8b071f8b85c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbe659c640 1 -- 172.21.15.53:0/465134691 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe002cc70 con 0x7fbbe8071070 2023-10-21T17:38:11.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbe659c640 1 -- 172.21.15.53:0/465134691 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe002cdd0 con 0x7fbbe8071070 2023-10-21T17:38:11.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbe659c640 1 -- 172.21.15.53:0/465134691 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0034660 con 0x7fbbe8071070 2023-10-21T17:38:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 -- 172.21.15.53:0/465134691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 msgr2=0x7fbbe80cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 --2- 172.21.15.53:0/465134691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80cb130 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80b3980 tx=0x7fbbe002bec0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 -- 172.21.15.53:0/465134691 shutdown_connections 2023-10-21T17:38:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 --2- 172.21.15.53:0/465134691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80cb130 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 --2- 172.21.15.53:0/465134691 >> v2:172.21.15.124:3300/0 conn(0x7fbbe806dfc0 0x7fbbe8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 -- 172.21.15.53:0/465134691 >> 172.21.15.53:0/465134691 conn(0x7fbbe806a5d0 msgr2=0x7fbbe80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.898+0000 7fbbed00e640 1 -- 172.21.15.53:0/465134691 shutdown_connections 2023-10-21T17:38:11.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.899+0000 7fbbed00e640 1 -- 172.21.15.53:0/465134691 wait complete. 2023-10-21T17:38:11.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.900+0000 7fbbed00e640 1 Processor -- start 2023-10-21T17:38:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.900+0000 7fbbed00e640 1 -- start start 2023-10-21T17:38:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 0x7fbbe81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbe80afbe0 0x7fbbe8150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe80c5230 con 0x7fbbe8071070 2023-10-21T17:38:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8151220 con 0x7fbbe806dfc0 2023-10-21T17:38:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.901+0000 7fbbed00e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8153c30 con 0x7fbbe80afbe0 2023-10-21T17:38:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45938/0 (socket says 172.21.15.53:45938) 2023-10-21T17:38:11.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 -- 172.21.15.53:0/3175414735 learned_addr learned my addr 172.21.15.53:0/3175414735 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbe6d9d640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 0x7fbbe81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 -- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbe80afbe0 msgr2=0x7fbbe8150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbe80afbe0 0x7fbbe8150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 -- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 msgr2=0x7fbbe81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 0x7fbbe81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:11.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 -- 172.21.15.53:0/3175414735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0009a70 con 0x7fbbe8071070 2023-10-21T17:38:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbe6d9d640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 0x7fbbe81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.902+0000 7fbbdffff640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000d7d0 tx=0x7fbbd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.903+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0018070 con 0x7fbbe8071070 2023-10-21T17:38:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.903+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0004020 con 0x7fbbe8071070 2023-10-21T17:38:11.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.903+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe8153f10 con 0x7fbbe8071070 2023-10-21T17:38:11.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.903+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00041b0 con 0x7fbbe8071070 2023-10-21T17:38:11.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.903+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81544d0 con 0x7fbbe8071070 2023-10-21T17:38:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.904+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe804e160 con 0x7fbbe8071070 2023-10-21T17:38:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.905+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbd0004310 con 0x7fbbe8071070 2023-10-21T17:38:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.907+0000 7fbbdf7fe640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 0x7fbbc806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.907+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fbbd0089b90 con 0x7fbbe8071070 2023-10-21T17:38:11.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.907+0000 7fbbe6d9d640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 0x7fbbc806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:11.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.908+0000 7fbbe6d9d640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 0x7fbbc806d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000f60 tx=0x7fbbd8009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:11.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:11.911+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbd00545c0 con 0x7fbbe8071070 2023-10-21T17:38:12.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.146+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbbe80b0ef0 con 0x7fbbe8071070 2023-10-21T17:38:12.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.691+0000 7fbbdf7fe640 1 -- 172.21.15.53:0/3175414735 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbbd0057be0 con 0x7fbbe8071070 2023-10-21T17:38:12.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 msgr2=0x7fbbc806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 0x7fbbc806d450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000f60 tx=0x7fbbd8009040 comp rx=0 tx=0).stop 2023-10-21T17:38:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 msgr2=0x7fbbe80c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000d7d0 tx=0x7fbbd000dc90 comp rx=0 tx=0).stop 2023-10-21T17:38:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 shutdown_connections 2023-10-21T17:38:12.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbbc806af90 0x7fbbc806d450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbbe80afbe0 0x7fbbe8150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbbe8071070 0x7fbbe80c4cf0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 --2- 172.21.15.53:0/3175414735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbbe806dfc0 0x7fbbe81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 >> 172.21.15.53:0/3175414735 conn(0x7fbbe806a5d0 msgr2=0x7fbbe80c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:12.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 shutdown_connections 2023-10-21T17:38:12.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:12.696+0000 7fbbed00e640 1 -- 172.21.15.53:0/3175414735 wait complete. 2023-10-21T17:38:12.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:38:13.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.000+0000 7f185ffff640 1 Processor -- start 2023-10-21T17:38:13.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.000+0000 7f185ffff640 1 -- start start 2023-10-21T17:38:13.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600b0680 0x7f186015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f18600ca680 0x7f186015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600cab60 con 0x7f186015a820 2023-10-21T17:38:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185ffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600cb960 con 0x7f18600ca680 2023-10-21T17:38:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600cbdf0 con 0x7f18600b0680 2023-10-21T17:38:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185effd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f186015a820 0x7f18600aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48684/0 (socket says 172.21.15.53:48684) 2023-10-21T17:38:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185effd640 1 -- 172.21.15.53:0/3997504720 learned_addr learned my addr 172.21.15.53:0/3997504720 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:13.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.002+0000 7f185f7fe640 1 --2- 172.21.15.53:0/3997504720 >> v2:172.21.15.53:3300/0 conn(0x7f18600ca680 0x7f186015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f18567fc640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600b0680 0x7f186015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185e7fc640 1 -- 172.21.15.53:0/3997504720 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234012889 0 0) 0x7f18600cab60 con 0x7f186015a820 2023-10-21T17:38:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185e7fc640 1 -- 172.21.15.53:0/3997504720 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f184c003520 con 0x7f186015a820 2023-10-21T17:38:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185f7fe640 1 -- 172.21.15.53:0/3997504720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600b0680 msgr2=0x7f186015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185f7fe640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600b0680 0x7f186015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185f7fe640 1 -- 172.21.15.53:0/3997504720 >> v1:172.21.15.53:6789/0 conn(0x7f186015a820 legacy=0x7f18600aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.003+0000 7f185f7fe640 1 -- 172.21.15.53:0/3997504720 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600cb030 con 0x7f18600ca680 2023-10-21T17:38:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.004+0000 7f185f7fe640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18600ca680 0x7f186015ea80 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f18580097a0 tx=0x7f185802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7760ca4f09ae012 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.004+0000 7f185e7fc640 1 -- 172.21.15.53:0/3997504720 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185802e070 con 0x7f18600ca680 2023-10-21T17:38:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.004+0000 7f185e7fc640 1 -- 172.21.15.53:0/3997504720 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18580315a0 con 0x7f18600ca680 2023-10-21T17:38:13.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.004+0000 7f185e7fc640 1 -- 172.21.15.53:0/3997504720 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858031910 con 0x7f18600ca680 2023-10-21T17:38:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 -- 172.21.15.53:0/3997504720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18600ca680 msgr2=0x7f186015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18600ca680 0x7f186015ea80 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f18580097a0 tx=0x7f185802b840 comp rx=0 tx=0).stop 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 -- 172.21.15.53:0/3997504720 shutdown_connections 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18600ca680 0x7f186015ea80 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 --2- 172.21.15.53:0/3997504720 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600b0680 0x7f186015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 -- 172.21.15.53:0/3997504720 >> 172.21.15.53:0/3997504720 conn(0x7f186006c050 msgr2=0x7f186006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 -- 172.21.15.53:0/3997504720 shutdown_connections 2023-10-21T17:38:13.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.005+0000 7f185ffff640 1 -- 172.21.15.53:0/3997504720 wait complete. 2023-10-21T17:38:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.007+0000 7f185ffff640 1 Processor -- start 2023-10-21T17:38:13.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.007+0000 7f185ffff640 1 -- start start 2023-10-21T17:38:13.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18600b0680 0x7f18600c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600ca680 0x7f18601e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600c2020 con 0x7f186015a820 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600c6540 con 0x7f18600b0680 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600c4b90 con 0x7f18600ca680 2023-10-21T17:38:13.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45956/0 (socket says 172.21.15.53:45956) 2023-10-21T17:38:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.008+0000 7f185f7fe640 1 -- 172.21.15.53:0/1952803097 learned_addr learned my addr 172.21.15.53:0/1952803097 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f18567fc640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600ca680 0x7f18601e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185effd640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18600b0680 0x7f18600c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 -- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600ca680 msgr2=0x7f18601e3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600ca680 0x7f18601e3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 -- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18600b0680 msgr2=0x7f18600c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18600b0680 0x7f18600c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 -- 172.21.15.53:0/1952803097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858009400 con 0x7f186015a820 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.009+0000 7f185f7fe640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f1858034040 tx=0x7f18580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.010+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858003810 con 0x7f186015a820 2023-10-21T17:38:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.010+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858040070 con 0x7f186015a820 2023-10-21T17:38:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.010+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858003b70 con 0x7f186015a820 2023-10-21T17:38:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.010+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18600c5930 con 0x7f186015a820 2023-10-21T17:38:13.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.010+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600c5ef0 con 0x7f186015a820 2023-10-21T17:38:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.011+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860074150 con 0x7f186015a820 2023-10-21T17:38:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.012+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1858003cd0 con 0x7f186015a820 2023-10-21T17:38:13.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.014+0000 7f1857fff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 0x7f184006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.014+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1858042070 con 0x7f186015a820 2023-10-21T17:38:13.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.014+0000 7f185effd640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 0x7f184006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.015+0000 7f185effd640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 0x7f184006d4a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1850005880 tx=0x7f1850009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.018+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1858076e10 con 0x7f186015a820 2023-10-21T17:38:13.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.212+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18600c6680 con 0x7f186015a820 2023-10-21T17:38:13.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.308+0000 7f1857fff640 1 -- 172.21.15.53:0/1952803097 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v223) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1858006e10 con 0x7f186015a820 2023-10-21T17:38:13.310 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:38:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.314+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 msgr2=0x7f184006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.314+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 0x7f184006d4a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1850005880 tx=0x7f1850009040 comp rx=0 tx=0).stop 2023-10-21T17:38:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.314+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 msgr2=0x7f18600c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.314+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f1858034040 tx=0x7f18580046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 shutdown_connections 2023-10-21T17:38:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f186015a820 0x7f18600c69d0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f184006afe0 0x7f184006d4a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18600ca680 0x7f18601e3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 --2- 172.21.15.53:0/1952803097 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18600b0680 0x7f18600c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.315+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 >> 172.21.15.53:0/1952803097 conn(0x7f186006c050 msgr2=0x7f18600c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.316+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 shutdown_connections 2023-10-21T17:38:13.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.317+0000 7f185ffff640 1 -- 172.21.15.53:0/1952803097 wait complete. 2023-10-21T17:38:13.342 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:38:13.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.545+0000 7f4f01d90640 1 Processor -- start 2023-10-21T17:38:13.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.545+0000 7f4f01d90640 1 -- start start 2023-10-21T17:38:13.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4f01d90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 0x7f4efc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4f01d90640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4efc071070 0x7f4efc0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4f01d90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc0b1cc0 con 0x7f4efc0afbe0 2023-10-21T17:38:13.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4f01d90640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4efc0b2620 con 0x7f4efc071070 2023-10-21T17:38:13.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4f01d90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b2060 con 0x7f4efc06dfc0 2023-10-21T17:38:13.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4efb7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4efc0afbe0 0x7f4efc0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:48700/0 (socket says 172.21.15.53:48700) 2023-10-21T17:38:13.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.547+0000 7f4efb7fe640 1 -- 172.21.15.53:0/3967066672 learned_addr learned my addr 172.21.15.53:0/3967066672 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:13.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 0x7f4efc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145592188 0 0) 0x7f4efc0b1cc0 con 0x7f4efc0afbe0 2023-10-21T17:38:13.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efbfff640 1 --2- 172.21.15.53:0/3967066672 >> v2:172.21.15.124:3300/0 conn(0x7f4efc071070 0x7f4efc0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee4003520 con 0x7f4efc0afbe0 2023-10-21T17:38:13.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1531363507 0 0) 0x7f4ee4003520 con 0x7f4efc0afbe0 2023-10-21T17:38:13.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 msgr2=0x7f4efc070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:13.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 0x7f4efc070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 >> v2:172.21.15.124:3300/0 conn(0x7f4efc071070 msgr2=0x7f4efc0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:13.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 --2- 172.21.15.53:0/3967066672 >> v2:172.21.15.124:3300/0 conn(0x7f4efc071070 0x7f4efc0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0b2340 con 0x7f4efc0afbe0 2023-10-21T17:38:13.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4eec001670 con 0x7f4efc0afbe0 2023-10-21T17:38:13.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 >> v1:172.21.15.53:6789/0 conn(0x7f4efc0afbe0 legacy=0x7f4efc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 0x7f4ee4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee4003f40 0x7f4ee40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4004d00 0x7f4ee4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b2620 con 0x7f4ee4003f40 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0b2060 con 0x7f4ee4004d00 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.548+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee400b980 con 0x7f4efc06dfc0 2023-10-21T17:38:13.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee4003f40 0x7f4ee40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 -- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 msgr2=0x7f4ee4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:13.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc06dfc0 0x7f4ee4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 -- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4004d00 msgr2=0x7f4ee4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:13.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4004d00 0x7f4ee4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.549+0000 7f4efaffd640 1 -- 172.21.15.53:0/3967066672 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4003520 con 0x7f4ee4003f40 2023-10-21T17:38:13.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.550+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee4003f40 0x7f4ee40047c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009ac0 tx=0x7f4ee40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1ecce1a1bff570 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.550+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002c6b0 con 0x7f4ee4003f40 2023-10-21T17:38:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.550+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef002c810 con 0x7f4ee4003f40 2023-10-21T17:38:13.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.550+0000 7f4efa7fc640 1 -- 172.21.15.53:0/3967066672 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002cbc0 con 0x7f4ee4003f40 2023-10-21T17:38:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 -- 172.21.15.53:0/3967066672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee4003f40 msgr2=0x7f4ee40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3967066672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee4003f40 0x7f4ee40047c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009ac0 tx=0x7f4ee40057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4efbfff640 1 -- 172.21.15.53:0/3967066672 reap_dead start 2023-10-21T17:38:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 -- 172.21.15.53:0/3967066672 shutdown_connections 2023-10-21T17:38:13.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 -- 172.21.15.53:0/3967066672 >> 172.21.15.53:0/3967066672 conn(0x7f4efc06a5d0 msgr2=0x7f4efc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 -- 172.21.15.53:0/3967066672 shutdown_connections 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.551+0000 7f4f01d90640 1 -- 172.21.15.53:0/3967066672 wait complete. 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.553+0000 7f4f01d90640 1 Processor -- start 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.553+0000 7f4f01d90640 1 -- start start 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4003f40 0x7f4efc0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc0c3180 0x7f4efc1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0c3600 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0ca100 con 0x7f4ee4003f40 2023-10-21T17:38:13.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.554+0000 7f4f01d90640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc0ca3e0 con 0x7f4efc0c3180 2023-10-21T17:38:13.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45962/0 (socket says 172.21.15.53:45962) 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 -- 172.21.15.53:0/3289211303 learned_addr learned my addr 172.21.15.53:0/3289211303 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 -- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc0c3180 msgr2=0x7f4efc1f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc0c3180 0x7f4efc1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efb7fe640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4003f40 0x7f4efc0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 -- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4003f40 msgr2=0x7f4efc0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4003f40 0x7f4efc0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 -- 172.21.15.53:0/3289211303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0ca6c0 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.555+0000 7f4efaffd640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0037040 tx=0x7f4ef0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.556+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0034b80 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.556+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef0034ce0 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.556+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc15c540 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.556+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef003e660 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.556+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc151d10 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.557+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc04e160 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.558+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4ef003e7c0 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.559+0000 7f4ee3fff640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 0x7f4ee806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:13.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.559+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f4ef00aba40 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.559+0000 7f4efb7fe640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 0x7f4ee806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:13.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.561+0000 7f4efb7fe640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 0x7f4ee806d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4eec001050 tx=0x7f4eec009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:13.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.564+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ef0076620 con 0x7f4efc0c4ce0 2023-10-21T17:38:13.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:13.762+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4efc0b0ef0 con 0x7f4efc0c4ce0 2023-10-21T17:38:14.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.719+0000 7f4ee3fff640 1 -- 172.21.15.53:0/3289211303 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v224) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4ef0080030 con 0x7f4efc0c4ce0 2023-10-21T17:38:14.721 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:38:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.725+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 msgr2=0x7f4ee806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.725+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 0x7f4ee806d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4eec001050 tx=0x7f4eec009040 comp rx=0 tx=0).stop 2023-10-21T17:38:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.725+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 msgr2=0x7f4efc0c2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.725+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0037040 tx=0x7f4ef0004730 comp rx=0 tx=0).stop 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 shutdown_connections 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4ee806afc0 0x7f4ee806d480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4efc0c3180 0x7f4efc1f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4efc0c4ce0 0x7f4efc0c2c40 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 --2- 172.21.15.53:0/3289211303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ee4003f40 0x7f4efc0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.726+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 >> 172.21.15.53:0/3289211303 conn(0x7f4efc06a5d0 msgr2=0x7f4efc159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:14.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.729+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 shutdown_connections 2023-10-21T17:38:14.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.729+0000 7f4f01d90640 1 -- 172.21.15.53:0/3289211303 wait complete. 2023-10-21T17:38:14.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:38:14.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.972+0000 7ff7bf516640 1 Processor -- start 2023-10-21T17:38:14.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.972+0000 7ff7bf516640 1 -- start start 2023-10-21T17:38:14.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.973+0000 7ff7bf516640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 0x7ff7b80aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.973+0000 7ff7bf516640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.973+0000 7ff7bf516640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80cab60 con 0x7ff7b815ac00 2023-10-21T17:38:14.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.973+0000 7ff7bf516640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b80cb960 con 0x7ff7b815a820 2023-10-21T17:38:14.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.973+0000 7ff7bf516640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80cbdf0 con 0x7ff7b80ca680 2023-10-21T17:38:14.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.974+0000 7ff7bdd13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.974+0000 7ff7bed15640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7ff7b815ac00 0x7ff7b80c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:33344/0 (socket says 172.21.15.53:33344) 2023-10-21T17:38:14.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.974+0000 7ff7bdd13640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:32792/0 (socket says 172.21.15.53:32792) 2023-10-21T17:38:14.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.974+0000 7ff7bed15640 1 -- 172.21.15.53:0/278191728 learned_addr learned my addr 172.21.15.53:0/278191728 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:14.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.974+0000 7ff7be514640 1 --2- 172.21.15.53:0/278191728 >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 0x7ff7b80aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7bdd13640 1 -- 172.21.15.53:0/278191728 >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 msgr2=0x7ff7b80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:14.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7bdd13640 1 --2- 172.21.15.53:0/278191728 >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 0x7ff7b80aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7bd512640 1 -- 172.21.15.53:0/278191728 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230236323 0 0) 0x7ff7b80cab60 con 0x7ff7b815ac00 2023-10-21T17:38:14.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7bdd13640 1 -- 172.21.15.53:0/278191728 >> v1:172.21.15.124:6789/0 conn(0x7ff7b815ac00 legacy=0x7ff7b80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7bdd13640 1 -- 172.21.15.53:0/278191728 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80cb030 con 0x7ff7b80ca680 2023-10-21T17:38:14.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.975+0000 7ff7be514640 1 --2- 172.21.15.53:0/278191728 >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 0x7ff7b80aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:38:14.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.977+0000 7ff7bdd13640 1 --2- 172.21.15.53:0/278191728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d964b3b6df160761 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:14.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.977+0000 7ff7bd512640 1 -- 172.21.15.53:0/278191728 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8031780 con 0x7ff7b80ca680 2023-10-21T17:38:14.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bd512640 1 -- 172.21.15.53:0/278191728 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80318e0 con 0x7ff7b80ca680 2023-10-21T17:38:14.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bd512640 1 -- 172.21.15.53:0/278191728 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8031c70 con 0x7ff7b80ca680 2023-10-21T17:38:14.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 -- 172.21.15.53:0/278191728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 msgr2=0x7ff7b80b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 --2- 172.21.15.53:0/278191728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097a0 tx=0x7ff7a802b820 comp rx=0 tx=0).stop 2023-10-21T17:38:14.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 -- 172.21.15.53:0/278191728 shutdown_connections 2023-10-21T17:38:14.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 --2- 172.21.15.53:0/278191728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b80b04b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 --2- 172.21.15.53:0/278191728 >> v2:172.21.15.124:3300/0 conn(0x7ff7b815a820 0x7ff7b80aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 -- 172.21.15.53:0/278191728 >> 172.21.15.53:0/278191728 conn(0x7ff7b806c050 msgr2=0x7ff7b806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:14.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 -- 172.21.15.53:0/278191728 shutdown_connections 2023-10-21T17:38:14.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.978+0000 7ff7bf516640 1 -- 172.21.15.53:0/278191728 wait complete. 2023-10-21T17:38:14.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.980+0000 7ff7bf516640 1 Processor -- start 2023-10-21T17:38:14.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.980+0000 7ff7bf516640 1 -- start start 2023-10-21T17:38:14.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7b815a820 0x7ff7b81e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81e5ee0 con 0x7ff7b815ac00 2023-10-21T17:38:14.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81e61c0 con 0x7ff7b815a820 2023-10-21T17:38:14.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.981+0000 7ff7bf516640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81e64a0 con 0x7ff7b80ca680 2023-10-21T17:38:14.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55456/0 (socket says 172.21.15.53:55456) 2023-10-21T17:38:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 -- 172.21.15.53:0/1294014524 learned_addr learned my addr 172.21.15.53:0/1294014524 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7be514640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bdd13640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7b815a820 0x7ff7b81e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 -- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 msgr2=0x7ff7b814ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b814ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 -- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7b815a820 msgr2=0x7ff7b81e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:14.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7b815a820 0x7ff7b81e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:14.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.982+0000 7ff7bed15640 1 -- 172.21.15.53:0/1294014524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4009400 con 0x7ff7b815ac00 2023-10-21T17:38:14.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7bed15640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c770 tx=0x7ff7b000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:14.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000ce80 con 0x7ff7b815ac00 2023-10-21T17:38:14.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000a640 con 0x7ff7b815ac00 2023-10-21T17:38:14.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000a9f0 con 0x7ff7b815ac00 2023-10-21T17:38:14.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a8009400 con 0x7ff7b815ac00 2023-10-21T17:38:14.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.983+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b81e6a60 con 0x7ff7b815ac00 2023-10-21T17:38:14.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.984+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b8074150 con 0x7ff7b815ac00 2023-10-21T17:38:14.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.985+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff7b000ab50 con 0x7ff7b815ac00 2023-10-21T17:38:14.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.986+0000 7ff7aeffd640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 0x7ff79c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:14.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.986+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7ff7b0088da0 con 0x7ff7b815ac00 2023-10-21T17:38:14.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.987+0000 7ff7be514640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 0x7ff79c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:14.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.988+0000 7ff7be514640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 0x7ff79c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009c60 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:14.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:14.991+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7b0053c60 con 0x7ff7b815ac00 2023-10-21T17:38:15.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.184+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff7b80b1310 con 0x7ff7b815ac00 2023-10-21T17:38:15.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.190+0000 7ff7aeffd640 1 -- 172.21.15.53:0/1294014524 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7b0018020 con 0x7ff7b815ac00 2023-10-21T17:38:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 msgr2=0x7ff79c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 0x7ff79c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009c60 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-10-21T17:38:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 msgr2=0x7ff7b8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c770 tx=0x7ff7b000cc30 comp rx=0 tx=0).stop 2023-10-21T17:38:15.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 shutdown_connections 2023-10-21T17:38:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7b815ac00 0x7ff7b8070810 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7b815a820 0x7ff7b81e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff79c06af90 0x7ff79c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 --2- 172.21.15.53:0/1294014524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7b80ca680 0x7ff7b814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 >> 172.21.15.53:0/1294014524 conn(0x7ff7b806c050 msgr2=0x7ff7b80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.193+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 shutdown_connections 2023-10-21T17:38:15.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.194+0000 7ff7bf516640 1 -- 172.21.15.53:0/1294014524 wait complete. 2023-10-21T17:38:15.215 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:38:15.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.440+0000 7fa862b28640 1 Processor -- start 2023-10-21T17:38:15.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.440+0000 7fa862b28640 1 -- start start 2023-10-21T17:38:15.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862b28640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa85c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862b28640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c15ae90 con 0x7fa85c0b04a0 2023-10-21T17:38:15.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862b28640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c0cbdb0 con 0x7fa85c15aab0 2023-10-21T17:38:15.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862b28640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0cb600 con 0x7fa85c070b50 2023-10-21T17:38:15.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862327640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa85c0b04a0 0x7fa85c0b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60698/0 (socket says 172.21.15.53:60698) 2023-10-21T17:38:15.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.442+0000 7fa862327640 1 -- 172.21.15.53:0/3161749734 learned_addr learned my addr 172.21.15.53:0/3161749734 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:15.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa861325640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa85c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586920453 0 0) 0x7fa854001110 con 0x7fa85c0b04a0 2023-10-21T17:38:15.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa848003520 con 0x7fa85c0b04a0 2023-10-21T17:38:15.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361003 0 0) 0x7fa848003520 con 0x7fa85c0b04a0 2023-10-21T17:38:15.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 msgr2=0x7fa85c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:15.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa85c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 >> v1:172.21.15.124:6789/0 conn(0x7fa85c15aab0 legacy=0x7fa85c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0cb7f0 con 0x7fa85c0b04a0 2023-10-21T17:38:15.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa854001800 con 0x7fa85c0b04a0 2023-10-21T17:38:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 >> v1:172.21.15.53:6789/0 conn(0x7fa85c0b04a0 legacy=0x7fa85c0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8480039a0 0x7fa848003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa8480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa848004d00 0x7fa848005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800bb10 con 0x7fa848004d00 2023-10-21T17:38:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800bc80 con 0x7fa8480039a0 2023-10-21T17:38:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.443+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800be40 con 0x7fa85c070b50 2023-10-21T17:38:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa848004d00 0x7fa848005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa861325640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa8480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa861b26640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8480039a0 0x7fa848003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 -- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 msgr2=0x7fa8480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa85c070b50 0x7fa8480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 -- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8480039a0 msgr2=0x7fa848003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8480039a0 0x7fa848003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.444+0000 7fa862327640 1 -- 172.21.15.53:0/3161749734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848003520 con 0x7fa848004d00 2023-10-21T17:38:15.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.445+0000 7fa862327640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa848004d00 0x7fa848005180 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fa854007fc0 tx=0x7fa8480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9da0f7b0ef7da81 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854005ad0 con 0x7fa848004d00 2023-10-21T17:38:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854005c50 con 0x7fa848004d00 2023-10-21T17:38:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa860b24640 1 -- 172.21.15.53:0/3161749734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85402c7e0 con 0x7fa848004d00 2023-10-21T17:38:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 -- 172.21.15.53:0/3161749734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa848004d00 msgr2=0x7fa848005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 --2- 172.21.15.53:0/3161749734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa848004d00 0x7fa848005180 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fa854007fc0 tx=0x7fa8480057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862327640 1 -- 172.21.15.53:0/3161749734 reap_dead start 2023-10-21T17:38:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 -- 172.21.15.53:0/3161749734 shutdown_connections 2023-10-21T17:38:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 -- 172.21.15.53:0/3161749734 >> 172.21.15.53:0/3161749734 conn(0x7fa85c06c010 msgr2=0x7fa85c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 -- 172.21.15.53:0/3161749734 shutdown_connections 2023-10-21T17:38:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.446+0000 7fa862b28640 1 -- 172.21.15.53:0/3161749734 wait complete. 2023-10-21T17:38:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.448+0000 7fa862b28640 1 Processor -- start 2023-10-21T17:38:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.448+0000 7fa862b28640 1 -- start start 2023-10-21T17:38:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa848004d00 0x7fa85c0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa85c1570e0 0x7fa85c0c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0af5e0 con 0x7fa85c0ada90 2023-10-21T17:38:15.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0ae000 con 0x7fa85c1570e0 2023-10-21T17:38:15.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa862b28640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0ae2e0 con 0x7fa848004d00 2023-10-21T17:38:15.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.449+0000 7fa861325640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55464/0 (socket says 172.21.15.53:55464) 2023-10-21T17:38:15.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 -- 172.21.15.53:0/1891194569 learned_addr learned my addr 172.21.15.53:0/1891194569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:15.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 -- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa848004d00 msgr2=0x7fa85c0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa848004d00 0x7fa85c0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 -- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa85c1570e0 msgr2=0x7fa85c0c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa862327640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa85c1570e0 0x7fa85c0c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa85c1570e0 0x7fa85c0c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 -- 172.21.15.53:0/1891194569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0ae5c0 con 0x7fa85c0ada90 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa861325640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011600 tx=0x7fa84c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.450+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00f920 con 0x7fa85c0ada90 2023-10-21T17:38:15.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.451+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00fa80 con 0x7fa85c0ada90 2023-10-21T17:38:15.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.451+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c0ae8a0 con 0x7fa85c0ada90 2023-10-21T17:38:15.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.451+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00fdf0 con 0x7fa85c0ada90 2023-10-21T17:38:15.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.451+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c151f40 con 0x7fa85c0ada90 2023-10-21T17:38:15.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.452+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c0b1720 con 0x7fa85c0ada90 2023-10-21T17:38:15.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.453+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa84c015460 con 0x7fa85c0ada90 2023-10-21T17:38:15.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.455+0000 7fa8527fc640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 0x7fa84006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.455+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fa84c08a160 con 0x7fa85c0ada90 2023-10-21T17:38:15.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.456+0000 7fa861b26640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 0x7fa84006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.457+0000 7fa861b26640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 0x7fa84006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0afc60 tx=0x7fa858009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.460+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa85c0b1720 con 0x7fa85c0ada90 2023-10-21T17:38:15.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.654+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fa85c0b1720 con 0x7fa85c0ada90 2023-10-21T17:38:15.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.660+0000 7fa8527fc640 1 -- 172.21.15.53:0/1891194569 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa85c0b1720 con 0x7fa85c0ada90 2023-10-21T17:38:15.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 msgr2=0x7fa84006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 0x7fa84006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0afc60 tx=0x7fa858009040 comp rx=0 tx=0).stop 2023-10-21T17:38:15.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 msgr2=0x7fa85c151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011600 tx=0x7fa84c011ac0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 shutdown_connections 2023-10-21T17:38:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa85c1570e0 0x7fa85c0c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa85c0ada90 0x7fa85c151a00 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fa84006afc0 0x7fa84006d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 --2- 172.21.15.53:0/1891194569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa848004d00 0x7fa85c0af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 >> 172.21.15.53:0/1891194569 conn(0x7fa85c06c010 msgr2=0x7fa85c0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 shutdown_connections 2023-10-21T17:38:15.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.663+0000 7fa862b28640 1 -- 172.21.15.53:0/1891194569 wait complete. 2023-10-21T17:38:15.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:38:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.902+0000 7f1ba4852640 1 Processor -- start 2023-10-21T17:38:15.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.902+0000 7f1ba4852640 1 -- start start 2023-10-21T17:38:15.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1ba4852640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ba006ec00 0x7f1ba0071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1ba4852640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba0160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1ba4852640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba0160d10 con 0x7f1ba0069fc0 2023-10-21T17:38:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1ba4852640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba0072070 con 0x7f1ba006ec00 2023-10-21T17:38:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1ba4852640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba0071580 con 0x7f1ba006a3a0 2023-10-21T17:38:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1b9effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ba006ec00 0x7f1ba0071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1b9effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ba006ec00 0x7f1ba0071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55478/0 (socket says 172.21.15.53:55478) 2023-10-21T17:38:15.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.904+0000 7f1b9effd640 1 -- 172.21.15.53:0/131404140 learned_addr learned my addr 172.21.15.53:0/131404140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:15.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.905+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba0160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.905+0000 7f1b9effd640 1 -- 172.21.15.53:0/131404140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 msgr2=0x7f1ba0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.905+0000 7f1b9effd640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba0160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.905+0000 7f1b9effd640 1 -- 172.21.15.53:0/131404140 >> v1:172.21.15.124:6789/0 conn(0x7f1ba0069fc0 legacy=0x7f1ba00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.905+0000 7f1b9effd640 1 -- 172.21.15.53:0/131404140 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0072330 con 0x7f1ba006ec00 2023-10-21T17:38:15.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.906+0000 7f1b9effd640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba0071860 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009e50 tx=0x7f1b8802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1cdadd705a5ece server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.906+0000 7f1b9e7fc640 1 -- 172.21.15.53:0/131404140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8802cb00 con 0x7f1ba006ec00 2023-10-21T17:38:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.906+0000 7f1b9e7fc640 1 -- 172.21.15.53:0/131404140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8802cc60 con 0x7f1ba006ec00 2023-10-21T17:38:15.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.906+0000 7f1b9e7fc640 1 -- 172.21.15.53:0/131404140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880343f0 con 0x7f1ba006ec00 2023-10-21T17:38:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 -- 172.21.15.53:0/131404140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 msgr2=0x7f1ba0071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba0071860 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009e50 tx=0x7f1b8802bf60 comp rx=0 tx=0).stop 2023-10-21T17:38:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 -- 172.21.15.53:0/131404140 shutdown_connections 2023-10-21T17:38:15.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba0160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 --2- 172.21.15.53:0/131404140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba0071860 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.907+0000 7f1ba4852640 1 -- 172.21.15.53:0/131404140 >> 172.21.15.53:0/131404140 conn(0x7f1ba00cf5f0 msgr2=0x7f1ba00cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.908+0000 7f1ba4852640 1 -- 172.21.15.53:0/131404140 shutdown_connections 2023-10-21T17:38:15.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.908+0000 7f1ba4852640 1 -- 172.21.15.53:0/131404140 wait complete. 2023-10-21T17:38:15.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.909+0000 7f1ba4852640 1 Processor -- start 2023-10-21T17:38:15.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 -- start start 2023-10-21T17:38:15.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 0x7f1ba014bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014c230 con 0x7f1ba006ec00 2023-10-21T17:38:15.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00b1160 con 0x7f1ba0069fc0 2023-10-21T17:38:15.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.910+0000 7f1ba4852640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00b2e40 con 0x7f1ba006a3a0 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55494/0 (socket says 172.21.15.53:55494) 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 -- 172.21.15.53:0/3789094556 learned_addr learned my addr 172.21.15.53:0/3789094556 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 -- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 msgr2=0x7f1ba00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 -- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 msgr2=0x7f1ba014bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:15.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9f7fe640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 0x7f1ba014bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 0x7f1ba014bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:15.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9ffff640 1 -- 172.21.15.53:0/3789094556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88009b30 con 0x7f1ba006ec00 2023-10-21T17:38:15.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.911+0000 7f1b9f7fe640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 0x7f1ba014bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:15.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1b9ffff640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800d5d0 tx=0x7f1b9800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b980125f0 con 0x7f1ba006ec00 2023-10-21T17:38:15.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b98012770 con 0x7f1ba006ec00 2023-10-21T17:38:15.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98012b20 con 0x7f1ba006ec00 2023-10-21T17:38:15.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba00b3120 con 0x7f1ba006ec00 2023-10-21T17:38:15.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.912+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba00b36e0 con 0x7f1ba006ec00 2023-10-21T17:38:15.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.913+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba0074150 con 0x7f1ba006ec00 2023-10-21T17:38:15.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.914+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b98012c80 con 0x7f1ba006ec00 2023-10-21T17:38:15.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.915+0000 7f1b97fff640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 0x7f1b8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:15.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.915+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1b98089e40 con 0x7f1ba006ec00 2023-10-21T17:38:15.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.916+0000 7f1b9f7fe640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 0x7f1b8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:15.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.917+0000 7f1b9f7fe640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 0x7f1b8006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000f60 tx=0x7f1b90009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:15.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:15.920+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b98054be0 con 0x7f1ba006ec00 2023-10-21T17:38:16.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.115+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f1ba00ca3b0 con 0x7f1ba006ec00 2023-10-21T17:38:16.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.122+0000 7f1b97fff640 1 -- 172.21.15.53:0/3789094556 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7f1b98058200 con 0x7f1ba006ec00 2023-10-21T17:38:16.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 msgr2=0x7f1b8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 0x7f1b8006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000f60 tx=0x7f1b90009040 comp rx=0 tx=0).stop 2023-10-21T17:38:16.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 msgr2=0x7f1ba014c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800d5d0 tx=0x7f1b9800da90 comp rx=0 tx=0).stop 2023-10-21T17:38:16.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 shutdown_connections 2023-10-21T17:38:16.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ba006ec00 0x7f1ba014c6b0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b8006af90 0x7f1b8006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ba006a3a0 0x7f1ba00b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 --2- 172.21.15.53:0/3789094556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ba0069fc0 0x7f1ba014bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 >> 172.21.15.53:0/3789094556 conn(0x7f1ba00cf5f0 msgr2=0x7f1ba015f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 shutdown_connections 2023-10-21T17:38:16.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.125+0000 7f1ba4852640 1 -- 172.21.15.53:0/3789094556 wait complete. 2023-10-21T17:38:16.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-21T17:38:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.365+0000 7faa852ec640 1 Processor -- start 2023-10-21T17:38:16.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.365+0000 7faa852ec640 1 -- start start 2023-10-21T17:38:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.366+0000 7faa852ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa800b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa852ec640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8015f7d0 con 0x7faa8015af10 2023-10-21T17:38:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa852ec640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8015efb0 con 0x7faa8015ab30 2023-10-21T17:38:16.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa852ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8015f210 con 0x7faa800adc70 2023-10-21T17:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa84aeb640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faa8015af10 0x7faa8006cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60708/0 (socket says 172.21.15.53:60708) 2023-10-21T17:38:16.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa84aeb640 1 -- 172.21.15.53:0/2057562681 learned_addr learned my addr 172.21.15.53:0/2057562681 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.367+0000 7faa77fff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa800b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573321427 0 0) 0x7faa8015f7d0 con 0x7faa8015af10 2023-10-21T17:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa6c003520 con 0x7faa8015af10 2023-10-21T17:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1269235233 0 0) 0x7faa6c003520 con 0x7faa8015af10 2023-10-21T17:38:16.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 msgr2=0x7faa800b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa800b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 >> v1:172.21.15.124:6789/0 conn(0x7faa8015ab30 legacy=0x7faa8015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8015f4f0 con 0x7faa8015af10 2023-10-21T17:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7faa78001a10 con 0x7faa8015af10 2023-10-21T17:38:16.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 >> v1:172.21.15.53:6789/0 conn(0x7faa8015af10 legacy=0x7faa8006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa6c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c003ee0 0x7faa6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c004d00 0x7faa6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8015f210 con 0x7faa6c003ee0 2023-10-21T17:38:16.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c00ba30 con 0x7faa6c004d00 2023-10-21T17:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.368+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c00bbf0 con 0x7faa800adc70 2023-10-21T17:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c003ee0 0x7faa6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa7ffff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa6c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa84aeb640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c004d00 0x7faa6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 -- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 msgr2=0x7faa6c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa800adc70 0x7faa6c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 -- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c004d00 msgr2=0x7faa6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c004d00 0x7faa6c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa77fff640 1 -- 172.21.15.53:0/2057562681 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c003520 con 0x7faa6c003ee0 2023-10-21T17:38:16.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.369+0000 7faa84aeb640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c004d00 0x7faa6c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:16.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.371+0000 7faa77fff640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c003ee0 0x7faa6c0047c0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ac0 tx=0x7faa6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e204f45de04a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.371+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6802c6b0 con 0x7faa6c003ee0 2023-10-21T17:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.371+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6802c810 con 0x7faa6c003ee0 2023-10-21T17:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.371+0000 7faa7f7fe640 1 -- 172.21.15.53:0/2057562681 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6802cba0 con 0x7faa6c003ee0 2023-10-21T17:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 -- 172.21.15.53:0/2057562681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c003ee0 msgr2=0x7faa6c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 --2- 172.21.15.53:0/2057562681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa6c003ee0 0x7faa6c0047c0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ac0 tx=0x7faa6c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa84aeb640 1 -- 172.21.15.53:0/2057562681 reap_dead start 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 -- 172.21.15.53:0/2057562681 shutdown_connections 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 -- 172.21.15.53:0/2057562681 >> 172.21.15.53:0/2057562681 conn(0x7faa800cb0c0 msgr2=0x7faa800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 -- 172.21.15.53:0/2057562681 shutdown_connections 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.372+0000 7faa852ec640 1 -- 172.21.15.53:0/2057562681 wait complete. 2023-10-21T17:38:16.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.374+0000 7faa852ec640 1 Processor -- start 2023-10-21T17:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.374+0000 7faa852ec640 1 -- start start 2023-10-21T17:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c003ee0 0x7faa800ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 0x7faa80155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa801560e0 con 0x7faa800c7c10 2023-10-21T17:38:16.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c8140 con 0x7faa6c003ee0 2023-10-21T17:38:16.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa852ec640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800c8420 con 0x7faa80155720 2023-10-21T17:38:16.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55516/0 (socket says 172.21.15.53:55516) 2023-10-21T17:38:16.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa77fff640 1 -- 172.21.15.53:0/4876352 learned_addr learned my addr 172.21.15.53:0/4876352 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:16.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa84aeb640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 0x7faa80155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.375+0000 7faa7ffff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c003ee0 0x7faa800ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 -- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 msgr2=0x7faa80155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 0x7faa80155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 -- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c003ee0 msgr2=0x7faa800ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c003ee0 0x7faa800ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 -- 172.21.15.53:0/4876352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800cc2b0 con 0x7faa800c7c10 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa84aeb640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 0x7faa80155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa77fff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7faa68009be0 tx=0x7faa68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68003810 con 0x7faa800c7c10 2023-10-21T17:38:16.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa68003990 con 0x7faa800c7c10 2023-10-21T17:38:16.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68003d40 con 0x7faa800c7c10 2023-10-21T17:38:16.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.376+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa800cc590 con 0x7faa800c7c10 2023-10-21T17:38:16.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.377+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa800ac820 con 0x7faa800c7c10 2023-10-21T17:38:16.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.378+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8004e160 con 0x7faa800c7c10 2023-10-21T17:38:16.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.378+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa6803b070 con 0x7faa800c7c10 2023-10-21T17:38:16.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.380+0000 7faa7d7fa640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 0x7faa6006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.380+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7faa680ab7f0 con 0x7faa800c7c10 2023-10-21T17:38:16.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.381+0000 7faa7ffff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 0x7faa6006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.382+0000 7faa7ffff640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 0x7faa6006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faa70005540 tx=0x7faa70007c50 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.384+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa68076750 con 0x7faa800c7c10 2023-10-21T17:38:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.646+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7faa801560e0 con 0x7faa800c7c10 2023-10-21T17:38:16.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.725+0000 7faa7d7fa640 1 -- 172.21.15.53:0/4876352 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3151) v1 ==== 163+0+0 (secure 0 0 0) 0x7faa68079d70 con 0x7faa800c7c10 2023-10-21T17:38:16.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 msgr2=0x7faa6006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 0x7faa6006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faa70005540 tx=0x7faa70007c50 comp rx=0 tx=0).stop 2023-10-21T17:38:16.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 msgr2=0x7faa8006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7faa68009be0 tx=0x7faa68004730 comp rx=0 tx=0).stop 2023-10-21T17:38:16.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 shutdown_connections 2023-10-21T17:38:16.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7faa6006af90 0x7faa6006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa80155720 0x7faa80155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa800c7c10 0x7faa8006da60 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 --2- 172.21.15.53:0/4876352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa6c003ee0 0x7faa800ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 >> 172.21.15.53:0/4876352 conn(0x7faa800cb0c0 msgr2=0x7faa801589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:16.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 shutdown_connections 2023-10-21T17:38:16.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.728+0000 7faa852ec640 1 -- 172.21.15.53:0/4876352 wait complete. 2023-10-21T17:38:16.751 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-21T17:38:16.751 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-10-21T17:38:16.752 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-21T17:38:16.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:16.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.974+0000 7f3b29edf640 1 Processor -- start 2023-10-21T17:38:16.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.974+0000 7f3b29edf640 1 -- start start 2023-10-21T17:38:16.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.975+0000 7f3b29edf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b241614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.975+0000 7f3b29edf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b240cab40 0x7f3b240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b29edf640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b240c9020 con 0x7f3b2415d080 2023-10-21T17:38:16.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b29edf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240ccc20 con 0x7f3b240cab40 2023-10-21T17:38:16.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b29edf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240c92e0 con 0x7f3b2415cca0 2023-10-21T17:38:16.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b23fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b240cab40 0x7f3b240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b23fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3b240cab40 0x7f3b240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55532/0 (socket says 172.21.15.53:55532) 2023-10-21T17:38:16.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.976+0000 7f3b23fff640 1 -- 172.21.15.53:0/765823378 learned_addr learned my addr 172.21.15.53:0/765823378 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:16.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b23fff640 1 -- 172.21.15.53:0/765823378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 msgr2=0x7f3b241614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:16.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b28edd640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b241614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b23fff640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b241614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b23fff640 1 -- 172.21.15.53:0/765823378 >> v1:172.21.15.124:6789/0 conn(0x7f3b2415d080 legacy=0x7f3b240701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b23fff640 1 -- 172.21.15.53:0/765823378 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b240cc7b0 con 0x7f3b240cab40 2023-10-21T17:38:16.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.977+0000 7f3b28edd640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b241614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:16.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.978+0000 7f3b23fff640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b240b00c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009870 tx=0x7f3b0c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa0c37736bba9e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.978+0000 7f3b237fe640 1 -- 172.21.15.53:0/765823378 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0317c0 con 0x7f3b240cab40 2023-10-21T17:38:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b237fe640 1 -- 172.21.15.53:0/765823378 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c031920 con 0x7f3b240cab40 2023-10-21T17:38:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b237fe640 1 -- 172.21.15.53:0/765823378 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c031cb0 con 0x7f3b240cab40 2023-10-21T17:38:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 -- 172.21.15.53:0/765823378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 msgr2=0x7f3b240b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b240b00c0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009870 tx=0x7f3b0c02b860 comp rx=0 tx=0).stop 2023-10-21T17:38:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 -- 172.21.15.53:0/765823378 shutdown_connections 2023-10-21T17:38:16.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b240b00c0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 --2- 172.21.15.53:0/765823378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b241614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 -- 172.21.15.53:0/765823378 >> 172.21.15.53:0/765823378 conn(0x7f3b240cb0c0 msgr2=0x7f3b240cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:16.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.979+0000 7f3b29edf640 1 -- 172.21.15.53:0/765823378 shutdown_connections 2023-10-21T17:38:16.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.980+0000 7f3b29edf640 1 -- 172.21.15.53:0/765823378 wait complete. 2023-10-21T17:38:16.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.981+0000 7f3b29edf640 1 Processor -- start 2023-10-21T17:38:16.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.981+0000 7f3b29edf640 1 -- start start 2023-10-21T17:38:16.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b240c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b2415d080 0x7f3b240c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b241f23e0 con 0x7f3b240cab40 2023-10-21T17:38:16.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b240af950 con 0x7f3b2415d080 2023-10-21T17:38:16.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.982+0000 7f3b29edf640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2415df00 con 0x7f3b2415cca0 2023-10-21T17:38:16.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55548/0 (socket says 172.21.15.53:55548) 2023-10-21T17:38:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 -- 172.21.15.53:0/3125253426 learned_addr learned my addr 172.21.15.53:0/3125253426 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b23fff640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b240c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 -- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 msgr2=0x7f3b240c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b240c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 -- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b2415d080 msgr2=0x7f3b240c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:16.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b296de640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b2415d080 0x7f3b240c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b2415d080 0x7f3b240c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:16.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.983+0000 7f3b28edd640 1 -- 172.21.15.53:0/3125253426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0094d0 con 0x7f3b240cab40 2023-10-21T17:38:16.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b28edd640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400ce00 tx=0x7f3b1400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400f890 con 0x7f3b240cab40 2023-10-21T17:38:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1400f9f0 con 0x7f3b240cab40 2023-10-21T17:38:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400fdc0 con 0x7f3b240cab40 2023-10-21T17:38:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2415e1e0 con 0x7f3b240cab40 2023-10-21T17:38:16.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.984+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2415e7a0 con 0x7f3b240cab40 2023-10-21T17:38:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.985+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2404e160 con 0x7f3b240cab40 2023-10-21T17:38:16.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.986+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3b1400b040 con 0x7f3b240cab40 2023-10-21T17:38:16.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.987+0000 7f3b217fa640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 0x7f3b0406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:16.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.987+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f3b1408a390 con 0x7f3b240cab40 2023-10-21T17:38:16.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.988+0000 7f3b23fff640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 0x7f3b0406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:16.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.989+0000 7f3b23fff640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 0x7f3b0406d4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c036000 tx=0x7f3b0c03d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:16.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:16.991+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b140551d0 con 0x7f3b240cab40 2023-10-21T17:38:17.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.187+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b240b45c0 con 0x7f3b240cab40 2023-10-21T17:38:17.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.189+0000 7f3b217fa640 1 -- 172.21.15.53:0/3125253426 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v224) v1 ==== 74+0+14388 (secure 0 0 0) 0x7f3b140587f0 con 0x7f3b240cab40 2023-10-21T17:38:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:17.191 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":224,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:14.706119+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/542756474":"2023-10-22T17:38:11.665152+0000","172.21.15.124:0/1098482368":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2555657847":"2023-10-22T17:38:07.636185+0000","172.21.15.53:6834/542756474":"2023-10-22T17:38:11.665152+0000","172.21.15.124:0/2763307861":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/1724708130":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2265843370":"2023-10-22T17:38:07.636185+0000","172.21.15.124:0/2852594978":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6832/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/1168372736":"2023-10-22T17:38:07.636185+0000","172.21.15.124:6833/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:6832/248442090":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/2759452384":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/863767263":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1944369314":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/1803654246":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/820409426":"2023-10-22T17:35:59.400288+0000","172.21.15.124:0/102947359":"2023-10-22T17:35:59.400288+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:17.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 msgr2=0x7f3b0406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 0x7f3b0406d4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c036000 tx=0x7f3b0c03d040 comp rx=0 tx=0).stop 2023-10-21T17:38:17.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 msgr2=0x7f3b241f1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400ce00 tx=0x7f3b1400a650 comp rx=0 tx=0).stop 2023-10-21T17:38:17.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 shutdown_connections 2023-10-21T17:38:17.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b2415d080 0x7f3b240c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3b0406afe0 0x7f3b0406d4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b2415cca0 0x7f3b240c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 --2- 172.21.15.53:0/3125253426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b240cab40 0x7f3b241f1ea0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 >> 172.21.15.53:0/3125253426 conn(0x7f3b240cb0c0 msgr2=0x7f3b24160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:17.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 shutdown_connections 2023-10-21T17:38:17.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.192+0000 7f3b29edf640 1 -- 172.21.15.53:0/3125253426 wait complete. 2023-10-21T17:38:17.214 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:38:17.215 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:38:17.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.449+0000 7f1bbd6ad640 1 Processor -- start 2023-10-21T17:38:17.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.449+0000 7f1bbd6ad640 1 -- start start 2023-10-21T17:38:17.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbd6ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 0x7f1bb80cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbd6ad640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bb815c630 0x7f1bb815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbd6ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb80b0260 con 0x7f1bb80c9390 2023-10-21T17:38:17.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbd6ad640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb815cfc0 con 0x7f1bb815c630 2023-10-21T17:38:17.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbd6ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb81612d0 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbceac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bb815c630 0x7f1bb815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbceac640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bb815c630 0x7f1bb815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55564/0 (socket says 172.21.15.53:55564) 2023-10-21T17:38:17.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.451+0000 7f1bbceac640 1 -- 172.21.15.53:0/2309352577 learned_addr learned my addr 172.21.15.53:0/2309352577 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:17.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bb7fff640 1 --2- 172.21.15.53:0/2309352577 >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 0x7f1bb80cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bbceac640 1 -- 172.21.15.53:0/2309352577 >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 msgr2=0x7f1bb80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:17.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bbceac640 1 --2- 172.21.15.53:0/2309352577 >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 0x7f1bb80cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bbceac640 1 -- 172.21.15.53:0/2309352577 >> v1:172.21.15.124:6789/0 conn(0x7f1bb80c9390 legacy=0x7f1bb8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:17.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bbceac640 1 -- 172.21.15.53:0/2309352577 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8161590 con 0x7f1bb815c630 2023-10-21T17:38:17.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.452+0000 7f1bb7fff640 1 --2- 172.21.15.53:0/2309352577 >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 0x7f1bb80cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:17.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.453+0000 7f1bbceac640 1 --2- 172.21.15.53:0/2309352577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb815c630 0x7f1bb815ca80 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f1bb806e180 tx=0x7f1bb002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c2827c14ffd2cc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.453+0000 7f1bb77fe640 1 -- 172.21.15.53:0/2309352577 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb002cb20 con 0x7f1bb815c630 2023-10-21T17:38:17.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bb77fe640 1 -- 172.21.15.53:0/2309352577 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cc80 con 0x7f1bb815c630 2023-10-21T17:38:17.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bb77fe640 1 -- 172.21.15.53:0/2309352577 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00343f0 con 0x7f1bb815c630 2023-10-21T17:38:17.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/2309352577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb815c630 msgr2=0x7f1bb815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/2309352577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb815c630 0x7f1bb815ca80 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f1bb806e180 tx=0x7f1bb002bf80 comp rx=0 tx=0).stop 2023-10-21T17:38:17.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/2309352577 shutdown_connections 2023-10-21T17:38:17.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/2309352577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb815c630 0x7f1bb815ca80 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/2309352577 >> v2:172.21.15.124:3300/0 conn(0x7f1bb80c8fb0 0x7f1bb80cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/2309352577 >> 172.21.15.53:0/2309352577 conn(0x7f1bb806c010 msgr2=0x7f1bb806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:17.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/2309352577 shutdown_connections 2023-10-21T17:38:17.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.454+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/2309352577 wait complete. 2023-10-21T17:38:17.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.456+0000 7f1bbd6ad640 1 Processor -- start 2023-10-21T17:38:17.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.456+0000 7f1bbd6ad640 1 -- start start 2023-10-21T17:38:17.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.457+0000 7f1bbd6ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.457+0000 7f1bbd6ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bb80c9390 0x7f1bb80c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.457+0000 7f1bbd6ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bb815c630 0x7f1bb80c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bbd6ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb815f120 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bbd6ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb815f400 con 0x7f1bb80c9390 2023-10-21T17:38:17.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bbd6ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb815f6e0 con 0x7f1bb815c630 2023-10-21T17:38:17.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55570/0 (socket says 172.21.15.53:55570) 2023-10-21T17:38:17.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 -- 172.21.15.53:0/918136824 learned_addr learned my addr 172.21.15.53:0/918136824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 -- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bb815c630 msgr2=0x7f1bb80c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bbceac640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bb815c630 0x7f1bb80c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1baf7fe640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bb80c9390 0x7f1bb80c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bb815c630 0x7f1bb80c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 -- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bb80c9390 msgr2=0x7f1bb80c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bb80c9390 0x7f1bb80c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.458+0000 7f1bb7fff640 1 -- 172.21.15.53:0/918136824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0009b30 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bb7fff640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e730 tx=0x7f1ba800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8004230 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba8004390 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800c8f0 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb815f9c0 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.459+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81f2340 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.461+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8074150 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.462+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1ba800ca50 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.462+0000 7f1bb57fa640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 0x7f1b9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.463+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1ba8089580 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.463+0000 7f1baf7fe640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 0x7f1b9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.464+0000 7f1baf7fe640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 0x7f1b9806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80c1f30 tx=0x7f1ba0009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.467+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ba8054330 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.670+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bb8160460 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.671+0000 7f1bb57fa640 1 -- 172.21.15.53:0/918136824 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v224) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ba8057950 con 0x7f1bb80c8fb0 2023-10-21T17:38:17.672 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:38:17.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 msgr2=0x7f1b9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 0x7f1b9806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bb80c1f30 tx=0x7f1ba0009040 comp rx=0 tx=0).stop 2023-10-21T17:38:17.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 msgr2=0x7f1bb81f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800e730 tx=0x7f1ba800ebf0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 shutdown_connections 2023-10-21T17:38:17.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f1b9806af90 0x7f1b9806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bb815c630 0x7f1bb80c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bb80c9390 0x7f1bb80c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 --2- 172.21.15.53:0/918136824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb80c8fb0 0x7f1bb81f1e00 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 >> 172.21.15.53:0/918136824 conn(0x7f1bb806c010 msgr2=0x7f1bb8070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:17.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 shutdown_connections 2023-10-21T17:38:17.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.674+0000 7f1bbd6ad640 1 -- 172.21.15.53:0/918136824 wait complete. 2023-10-21T17:38:17.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-21T17:38:17.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.927+0000 7f2cf8978640 1 Processor -- start 2023-10-21T17:38:17.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.927+0000 7f2cf8978640 1 -- start start 2023-10-21T17:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.928+0000 7f2cf8978640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf4157000 0x7f2cf40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cf8978640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cf41597d0 0x7f2cf406ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cf8978640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40adfc0 con 0x7f2cf40b0ac0 2023-10-21T17:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cf8978640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf40ccce0 con 0x7f2cf41597d0 2023-10-21T17:38:17.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cf8978640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf406d380 con 0x7f2cf4157000 2023-10-21T17:38:17.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cf41597d0 0x7f2cf406ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2cf41597d0 0x7f2cf406ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55586/0 (socket says 172.21.15.53:55586) 2023-10-21T17:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cebfff640 1 -- 172.21.15.53:0/1736245025 learned_addr learned my addr 172.21.15.53:0/1736245025 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.929+0000 7f2cf259c640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf4157000 0x7f2cf40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.930+0000 7f2cebfff640 1 -- 172.21.15.53:0/1736245025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf4157000 msgr2=0x7f2cf40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:17.932 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.930+0000 7f2cebfff640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf4157000 0x7f2cf40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.930+0000 7f2cebfff640 1 -- 172.21.15.53:0/1736245025 >> v1:172.21.15.124:6789/0 conn(0x7f2cf40b0ac0 legacy=0x7f2cf40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:17.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.930+0000 7f2cebfff640 1 -- 172.21.15.53:0/1736245025 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4159cb0 con 0x7f2cf41597d0 2023-10-21T17:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.931+0000 7f2cebfff640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf41597d0 0x7f2cf406ce40 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009870 tx=0x7f2cdc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1384ad2808f5db3c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.931+0000 7f2cf1d9b640 1 -- 172.21.15.53:0/1736245025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc030840 con 0x7f2cf41597d0 2023-10-21T17:38:17.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf1d9b640 1 -- 172.21.15.53:0/1736245025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc0309a0 con 0x7f2cf41597d0 2023-10-21T17:38:17.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf1d9b640 1 -- 172.21.15.53:0/1736245025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc030d30 con 0x7f2cf41597d0 2023-10-21T17:38:17.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 -- 172.21.15.53:0/1736245025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf41597d0 msgr2=0x7f2cf406ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf41597d0 0x7f2cf406ce40 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009870 tx=0x7f2cdc02b980 comp rx=0 tx=0).stop 2023-10-21T17:38:17.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 -- 172.21.15.53:0/1736245025 shutdown_connections 2023-10-21T17:38:17.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf41597d0 0x7f2cf406ce40 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 --2- 172.21.15.53:0/1736245025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf4157000 0x7f2cf40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 -- 172.21.15.53:0/1736245025 >> 172.21.15.53:0/1736245025 conn(0x7f2cf406a380 msgr2=0x7f2cf40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:17.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 -- 172.21.15.53:0/1736245025 shutdown_connections 2023-10-21T17:38:17.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.932+0000 7f2cf8978640 1 -- 172.21.15.53:0/1736245025 wait complete. 2023-10-21T17:38:17.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.934+0000 7f2cf8978640 1 Processor -- start 2023-10-21T17:38:17.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.934+0000 7f2cf8978640 1 -- start start 2023-10-21T17:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b0ac0 0x7f2cf40c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf41597d0 0x7f2cf40ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf415be80 con 0x7f2cf40cac20 2023-10-21T17:38:17.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40c8db0 con 0x7f2cf41597d0 2023-10-21T17:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf8978640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40c9090 con 0x7f2cf40b0ac0 2023-10-21T17:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.935+0000 7f2cf2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55600/0 (socket says 172.21.15.53:55600) 2023-10-21T17:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 -- 172.21.15.53:0/3702692647 learned_addr learned my addr 172.21.15.53:0/3702692647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:17.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf259c640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b0ac0 0x7f2cf40c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 -- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b0ac0 msgr2=0x7f2cf40c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b0ac0 0x7f2cf40c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 -- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf41597d0 msgr2=0x7f2cf40ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:17.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cebfff640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf41597d0 0x7f2cf40ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf41597d0 0x7f2cf40ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:17.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 -- 172.21.15.53:0/3702692647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0094d0 con 0x7f2cf40cac20 2023-10-21T17:38:17.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.936+0000 7f2cf2d9d640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c7a0 tx=0x7f2cec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.937+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec003b70 con 0x7f2cf40cac20 2023-10-21T17:38:17.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.937+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec003cf0 con 0x7f2cf40cac20 2023-10-21T17:38:17.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.937+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec003ea0 con 0x7f2cf40cac20 2023-10-21T17:38:17.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.937+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf40c9370 con 0x7f2cf40cac20 2023-10-21T17:38:17.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.937+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf415baa0 con 0x7f2cf40cac20 2023-10-21T17:38:17.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.938+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf415bc30 con 0x7f2cf40cac20 2023-10-21T17:38:17.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.939+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2cec00aa90 con 0x7f2cf40cac20 2023-10-21T17:38:17.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.940+0000 7f2ceaffd640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:17.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.940+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f2cec0897c0 con 0x7f2cf40cac20 2023-10-21T17:38:17.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.941+0000 7f2cf259c640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:17.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.942+0000 7f2cf259c640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 0x7f2cd406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4006fd0 tx=0x7f2ce4008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:17.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:17.944+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cec054550 con 0x7f2cf40cac20 2023-10-21T17:38:18.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.208+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f2cf40c9890 con 0x7f2cf40cac20 2023-10-21T17:38:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.737+0000 7f2ceaffd640 1 -- 172.21.15.53:0/3702692647 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3153) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2cec057b70 con 0x7f2cf40cac20 2023-10-21T17:38:18.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 msgr2=0x7f2cd406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 0x7f2cd406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4006fd0 tx=0x7f2ce4008040 comp rx=0 tx=0).stop 2023-10-21T17:38:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 msgr2=0x7f2cf40aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c7a0 tx=0x7f2cec00cc60 comp rx=0 tx=0).stop 2023-10-21T17:38:18.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 shutdown_connections 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cf40cac20 0x7f2cf40aea90 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf41597d0 0x7f2cf40ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2cd406afc0 0x7f2cd406d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 --2- 172.21.15.53:0/3702692647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b0ac0 0x7f2cf40c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 >> 172.21.15.53:0/3702692647 conn(0x7f2cf406a380 msgr2=0x7f2cf40afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:18.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.741+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 shutdown_connections 2023-10-21T17:38:18.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.742+0000 7f2cf8978640 1 -- 172.21.15.53:0/3702692647 wait complete. 2023-10-21T17:38:18.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:38:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.993+0000 7fc5fd232640 1 Processor -- start 2023-10-21T17:38:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.993+0000 7fc5fd232640 1 -- start start 2023-10-21T17:38:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5f815a820 0x7fc5f80aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 0x7fc5f80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80cab60 con 0x7fc5f815a820 2023-10-21T17:38:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80cb960 con 0x7fc5f80ca680 2023-10-21T17:38:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.994+0000 7fc5fd232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80cbdf0 con 0x7fc5f815ac00 2023-10-21T17:38:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5f815a820 0x7fc5f80aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc5f815a820 0x7fc5f80aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55606/0 (socket says 172.21.15.53:55606) 2023-10-21T17:38:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 -- 172.21.15.53:0/2517924516 learned_addr learned my addr 172.21.15.53:0/2517924516 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:18.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5effff640 1 --2- 172.21.15.53:0/2517924516 >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 0x7fc5f80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:18.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5fca31640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 -- 172.21.15.53:0/2517924516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 msgr2=0x7fc5f80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:19.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 -- 172.21.15.53:0/2517924516 >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 msgr2=0x7fc5f80b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:19.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/2517924516 >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 0x7fc5f80b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5f7fff640 1 -- 172.21.15.53:0/2517924516 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80cb030 con 0x7fc5f815a820 2023-10-21T17:38:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5fca31640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.995+0000 7fc5effff640 1 --2- 172.21.15.53:0/2517924516 >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 0x7fc5f80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80aff70 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80718e0 tx=0x7fc5e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c401b06b87919d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5f77fe640 1 -- 172.21.15.53:0/2517924516 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80317e0 con 0x7fc5f815a820 2023-10-21T17:38:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5f77fe640 1 -- 172.21.15.53:0/2517924516 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8031940 con 0x7fc5f815a820 2023-10-21T17:38:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5f77fe640 1 -- 172.21.15.53:0/2517924516 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8031cd0 con 0x7fc5f815a820 2023-10-21T17:38:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5fd232640 1 -- 172.21.15.53:0/2517924516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 msgr2=0x7fc5f80aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.997+0000 7fc5fd232640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80aff70 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80718e0 tx=0x7fc5e802b880 comp rx=0 tx=0).stop 2023-10-21T17:38:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 -- 172.21.15.53:0/2517924516 shutdown_connections 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 --2- 172.21.15.53:0/2517924516 >> v2:172.21.15.124:3300/0 conn(0x7fc5f80ca680 0x7fc5f80b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 --2- 172.21.15.53:0/2517924516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80aff70 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 -- 172.21.15.53:0/2517924516 >> 172.21.15.53:0/2517924516 conn(0x7fc5f806c050 msgr2=0x7fc5f806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 -- 172.21.15.53:0/2517924516 shutdown_connections 2023-10-21T17:38:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:18.998+0000 7fc5fd232640 1 -- 172.21.15.53:0/2517924516 wait complete. 2023-10-21T17:38:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.000+0000 7fc5fd232640 1 Processor -- start 2023-10-21T17:38:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.000+0000 7fc5fd232640 1 -- start start 2023-10-21T17:38:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f80ca680 0x7fc5f80aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b0830 con 0x7fc5f815a820 2023-10-21T17:38:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b27d0 con 0x7fc5f80ca680 2023-10-21T17:38:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fd232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b0550 con 0x7fc5f815ac00 2023-10-21T17:38:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55622/0 (socket says 172.21.15.53:55622) 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 -- 172.21.15.53:0/4097693297 learned_addr learned my addr 172.21.15.53:0/4097693297 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f80ca680 0x7fc5f80aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5fca31640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 -- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 msgr2=0x7fc5f80adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 -- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f80ca680 msgr2=0x7fc5f80aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f80ca680 0x7fc5f80aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.001+0000 7fc5effff640 1 -- 172.21.15.53:0/4097693297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80094d0 con 0x7fc5f815a820 2023-10-21T17:38:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5effff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c7c0 tx=0x7fc5e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0007820 con 0x7fc5f815a820 2023-10-21T17:38:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e0007980 con 0x7fc5f815a820 2023-10-21T17:38:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0007d10 con 0x7fc5f815a820 2023-10-21T17:38:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80b2180 con 0x7fc5f815a820 2023-10-21T17:38:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.002+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80c82b0 con 0x7fc5f815a820 2023-10-21T17:38:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.003+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80b1fe0 con 0x7fc5f815a820 2023-10-21T17:38:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.004+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc5e00113f0 con 0x7fc5f815a820 2023-10-21T17:38:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.005+0000 7fc5f57fa640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 0x7fc5d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.005+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fc5e0089a60 con 0x7fc5f815a820 2023-10-21T17:38:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.006+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 0x7fc5d806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.007+0000 7fc5f7fff640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 0x7fc5d806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8036000 tx=0x7fc5e803d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.010+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5f80b1fe0 con 0x7fc5f815a820 2023-10-21T17:38:19.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.205+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc5f80addc0 con 0x7fc5f815a820 2023-10-21T17:38:19.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.215+0000 7fc5f57fa640 1 -- 172.21.15.53:0/4097693297 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5e0054770 con 0x7fc5f815a820 2023-10-21T17:38:19.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 msgr2=0x7fc5d806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 0x7fc5d806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8036000 tx=0x7fc5e803d040 comp rx=0 tx=0).stop 2023-10-21T17:38:19.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 msgr2=0x7fc5f80c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c7c0 tx=0x7fc5e000cc80 comp rx=0 tx=0).stop 2023-10-21T17:38:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 shutdown_connections 2023-10-21T17:38:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc5d806afc0 0x7fc5d806d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f815ac00 0x7fc5f80adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5f815a820 0x7fc5f80c7d70 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 --2- 172.21.15.53:0/4097693297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f80ca680 0x7fc5f80aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.217+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 >> 172.21.15.53:0/4097693297 conn(0x7fc5f806c050 msgr2=0x7fc5f80ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:19.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.218+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 shutdown_connections 2023-10-21T17:38:19.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.218+0000 7fc5fd232640 1 -- 172.21.15.53:0/4097693297 wait complete. 2023-10-21T17:38:19.239 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:38:19.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:38:19.354 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:38:19.483 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:38:19.484 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:19.484 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:19.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:38:19.549 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:38:19.549 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:19.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:38:19.611 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:19.611 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:19.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:38:19.673 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:38:19.673 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:19.734 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:38:19.734 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:38:19.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:19.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:38:19.947 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:38:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.973+0000 7f2dea916640 1 Processor -- start 2023-10-21T17:38:19.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.973+0000 7f2dea916640 1 -- start start 2023-10-21T17:38:19.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea916640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2de41599d0 0x7f2de40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea916640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea916640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40cacf0 con 0x7f2de4159db0 2023-10-21T17:38:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea916640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de40b10f0 con 0x7f2de41599d0 2023-10-21T17:38:19.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea916640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40ca3c0 con 0x7f2de40cc210 2023-10-21T17:38:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2de9914640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2de41599d0 0x7f2de40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea115640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2de4159db0 0x7f2de415d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:60718/0 (socket says 172.21.15.53:60718) 2023-10-21T17:38:19.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.974+0000 7f2dea115640 1 -- 172.21.15.53:0/1999504945 learned_addr learned my addr 172.21.15.53:0/1999504945 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.975+0000 7f2de9113640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.975+0000 7f2de9914640 1 -- 172.21.15.53:0/1999504945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 msgr2=0x7f2de40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.975+0000 7f2de9914640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.975+0000 7f2de9914640 1 -- 172.21.15.53:0/1999504945 >> v1:172.21.15.53:6789/0 conn(0x7f2de4159db0 legacy=0x7f2de415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.975+0000 7f2de9914640 1 -- 172.21.15.53:0/1999504945 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40ca6a0 con 0x7f2de41599d0 2023-10-21T17:38:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.976+0000 7f2de9914640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de41599d0 0x7f2de40ad730 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097a0 tx=0x7f2de002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38a69f53a3e06d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2de8912640 1 -- 172.21.15.53:0/1999504945 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de002d040 con 0x7f2de41599d0 2023-10-21T17:38:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2de8912640 1 -- 172.21.15.53:0/1999504945 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de0031810 con 0x7f2de41599d0 2023-10-21T17:38:19.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2de8912640 1 -- 172.21.15.53:0/1999504945 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0031ba0 con 0x7f2de41599d0 2023-10-21T17:38:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2dea916640 1 -- 172.21.15.53:0/1999504945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de41599d0 msgr2=0x7f2de40ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2dea916640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de41599d0 0x7f2de40ad730 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2de00097a0 tx=0x7f2de002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2dea916640 1 -- 172.21.15.53:0/1999504945 shutdown_connections 2023-10-21T17:38:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2dea916640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.977+0000 7f2dea916640 1 --2- 172.21.15.53:0/1999504945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de41599d0 0x7f2de40ad730 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.978+0000 7f2dea916640 1 -- 172.21.15.53:0/1999504945 >> 172.21.15.53:0/1999504945 conn(0x7f2de40cb0c0 msgr2=0x7f2de40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.978+0000 7f2dea916640 1 -- 172.21.15.53:0/1999504945 shutdown_connections 2023-10-21T17:38:19.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.978+0000 7f2dea916640 1 -- 172.21.15.53:0/1999504945 wait complete. 2023-10-21T17:38:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.979+0000 7f2dea916640 1 Processor -- start 2023-10-21T17:38:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.980+0000 7f2dea916640 1 -- start start 2023-10-21T17:38:19.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de41599d0 0x7f2de413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40c7f00 con 0x7f2de4159db0 2023-10-21T17:38:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40b08d0 con 0x7f2de41599d0 2023-10-21T17:38:19.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea916640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40b0bb0 con 0x7f2de40cc210 2023-10-21T17:38:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55640/0 (socket says 172.21.15.53:55640) 2023-10-21T17:38:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 -- 172.21.15.53:0/1950022397 learned_addr learned my addr 172.21.15.53:0/1950022397 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2de9113640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de41599d0 0x7f2de413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2de9914640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 -- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 msgr2=0x7f2de40c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 -- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de41599d0 msgr2=0x7f2de413c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.981+0000 7f2dea115640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de41599d0 0x7f2de413c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:19.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dea115640 1 -- 172.21.15.53:0/1950022397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0009400 con 0x7f2de4159db0 2023-10-21T17:38:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dea115640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00ca80 tx=0x7f2ddc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00ac30 con 0x7f2de4159db0 2023-10-21T17:38:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00ad90 con 0x7f2de4159db0 2023-10-21T17:38:19.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc012600 con 0x7f2de4159db0 2023-10-21T17:38:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de40b0e90 con 0x7f2de4159db0 2023-10-21T17:38:19.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.982+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40c4600 con 0x7f2de4159db0 2023-10-21T17:38:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.984+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de40c8070 con 0x7f2de4159db0 2023-10-21T17:38:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.985+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ddc007070 con 0x7f2de4159db0 2023-10-21T17:38:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.986+0000 7f2dda7fc640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 0x7f2dc806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:19.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.986+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f2ddc0896e0 con 0x7f2de4159db0 2023-10-21T17:38:19.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.986+0000 7f2de9914640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 0x7f2dc806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:19.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.987+0000 7f2de9914640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 0x7f2dc806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2de0036000 tx=0x7f2de003c040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:19.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:19.990+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2de40c8070 con 0x7f2de4159db0 2023-10-21T17:38:20.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.225+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de40c8040 con 0x7f2de4159db0 2023-10-21T17:38:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.226+0000 7f2dda7fc640 1 -- 172.21.15.53:0/1950022397 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+3710 (secure 0 0 0) 0x7f2ddc0545b0 con 0x7f2de4159db0 2023-10-21T17:38:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:20.228 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":105,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/2796206285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2796206285},{"type":"v1","addr":"172.21.15.124:6837","nonce":2796206285}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10668,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2769090778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2769090778},{"type":"v1","addr":"172.21.15.124:6836","nonce":2769090778}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":10674,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1581539859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1581539859},{"type":"v1","addr":"172.21.15.53:6833","nonce":1581539859}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":11052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1533544120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1533544120},{"type":"v1","addr":"172.21.15.53:6835","nonce":1533544120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":105}],"filesystems":[]} 2023-10-21T17:38:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 105 2023-10-21T17:38:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.228+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 msgr2=0x7f2dc806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 0x7f2dc806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2de0036000 tx=0x7f2de003c040 comp rx=0 tx=0).stop 2023-10-21T17:38:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 msgr2=0x7f2de413ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00ca80 tx=0x7f2ddc00cf40 comp rx=0 tx=0).stop 2023-10-21T17:38:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 shutdown_connections 2023-10-21T17:38:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2de4159db0 0x7f2de413ca20 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2de41599d0 0x7f2de413c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2dc806afc0 0x7f2dc806d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 --2- 172.21.15.53:0/1950022397 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2de40cc210 0x7f2de40c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 >> 172.21.15.53:0/1950022397 conn(0x7f2de40cb0c0 msgr2=0x7f2de40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 shutdown_connections 2023-10-21T17:38:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:20.229+0000 7f2dea916640 1 -- 172.21.15.53:0/1950022397 wait complete. 2023-10-21T17:38:20.249 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:38:20.249 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:38:20.249 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:38:20.249 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:38:20.250 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:38:20.250 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:38:20.250 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:38:20.251 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:38:20.251 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:38:20.251 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:38:20.252 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:38:20.252 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:38:20.252 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:38:20.252 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:38:20.253 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:38:20.253 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:38:20.269 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:38:20.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:38:20.271 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:38:20.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:38:20.279 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:38:20.279 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:38:20.318 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:38:20.319 DEBUG:teuthology.parallel:result is None 2023-10-21T17:38:20.325 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:38:20.325 DEBUG:teuthology.parallel:result is None 2023-10-21T17:38:20.328 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:38:20.328 DEBUG:teuthology.parallel:result is None 2023-10-21T17:38:20.483 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:38:20.485 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:38:20.513 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:38:25.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-21T17:38:26.279 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:38:26.279 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:38:26.289 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:38:26.289 DEBUG:teuthology.parallel:result is None 2023-10-21T17:38:26.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:38:26.471 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:38:26.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.527+0000 7f97b2232640 1 Processor -- start 2023-10-21T17:38:26.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.528+0000 7f97b2232640 1 -- start start 2023-10-21T17:38:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97ac15a820 0x7f97ac0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f97ac0ca680 0x7f97ac0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15ac00 0x7f97ac0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0cab60 con 0x7f97ac15a820 2023-10-21T17:38:26.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0cb960 con 0x7f97ac0ca680 2023-10-21T17:38:26.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b2232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0cbdf0 con 0x7f97ac15ac00 2023-10-21T17:38:26.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.529+0000 7f97b1230640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97ac15a820 0x7f97ac0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97ac15a820 0x7f97ac0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49908/0 (socket says 172.21.15.53:49908) 2023-10-21T17:38:26.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 -- 172.21.15.53:0/2283191810 learned_addr learned my addr 172.21.15.53:0/2283191810 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:26.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1a31640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15ac00 0x7f97ac0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 -- 172.21.15.53:0/2283191810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15ac00 msgr2=0x7f97ac0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:26.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15ac00 0x7f97ac0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b0a2f640 1 --2- 172.21.15.53:0/2283191810 >> v2:172.21.15.124:3300/0 conn(0x7f97ac0ca680 0x7f97ac0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 -- 172.21.15.53:0/2283191810 >> v2:172.21.15.124:3300/0 conn(0x7f97ac0ca680 msgr2=0x7f97ac0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:26.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 --2- 172.21.15.53:0/2283191810 >> v2:172.21.15.124:3300/0 conn(0x7f97ac0ca680 0x7f97ac0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.530+0000 7f97b1230640 1 -- 172.21.15.53:0/2283191810 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0cb030 con 0x7f97ac15a820 2023-10-21T17:38:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.531+0000 7f97b1230640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15a820 0x7f97ac0aff70 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0718e0 tx=0x7f97a802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee04a3712d1f6ff server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97a3fff640 1 -- 172.21.15.53:0/2283191810 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80317d0 con 0x7f97ac15a820 2023-10-21T17:38:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97a3fff640 1 -- 172.21.15.53:0/2283191810 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a8031930 con 0x7f97ac15a820 2023-10-21T17:38:26.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97a3fff640 1 -- 172.21.15.53:0/2283191810 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8031cc0 con 0x7f97ac15a820 2023-10-21T17:38:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 -- 172.21.15.53:0/2283191810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15a820 msgr2=0x7f97ac0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15a820 0x7f97ac0aff70 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0718e0 tx=0x7f97a802b870 comp rx=0 tx=0).stop 2023-10-21T17:38:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 -- 172.21.15.53:0/2283191810 shutdown_connections 2023-10-21T17:38:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15ac00 0x7f97ac0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 --2- 172.21.15.53:0/2283191810 >> v2:172.21.15.124:3300/0 conn(0x7f97ac0ca680 0x7f97ac0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 --2- 172.21.15.53:0/2283191810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15a820 0x7f97ac0aff70 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.532+0000 7f97b2232640 1 -- 172.21.15.53:0/2283191810 >> 172.21.15.53:0/2283191810 conn(0x7f97ac06c050 msgr2=0x7f97ac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:26.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.533+0000 7f97b2232640 1 -- 172.21.15.53:0/2283191810 shutdown_connections 2023-10-21T17:38:26.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.533+0000 7f97b2232640 1 -- 172.21.15.53:0/2283191810 wait complete. 2023-10-21T17:38:26.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.534+0000 7f97b2232640 1 Processor -- start 2023-10-21T17:38:26.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.535+0000 7f97b2232640 1 -- start start 2023-10-21T17:38:26.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97ac0ca680 0x7f97ac0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15a820 0x7f97ac1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ad230 con 0x7f97ac15ac00 2023-10-21T17:38:26.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ac190 con 0x7f97ac0ca680 2023-10-21T17:38:26.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b2232640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0ac470 con 0x7f97ac15a820 2023-10-21T17:38:26.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49920/0 (socket says 172.21.15.53:49920) 2023-10-21T17:38:26.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 -- 172.21.15.53:0/4163699861 learned_addr learned my addr 172.21.15.53:0/4163699861 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:26.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1230640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97ac0ca680 0x7f97ac0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b0a2f640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15a820 0x7f97ac1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 -- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15a820 msgr2=0x7f97ac1f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:26.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15a820 0x7f97ac1f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 -- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97ac0ca680 msgr2=0x7f97ac0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:26.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97ac0ca680 0x7f97ac0ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:26.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.536+0000 7f97b1a31640 1 -- 172.21.15.53:0/4163699861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80094d0 con 0x7f97ac15ac00 2023-10-21T17:38:26.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97b1a31640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007e30 tx=0x7f97a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:26.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014620 con 0x7f97ac15ac00 2023-10-21T17:38:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400ee90 con 0x7f97ac15ac00 2023-10-21T17:38:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40149c0 con 0x7f97ac15ac00 2023-10-21T17:38:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac0ac750 con 0x7f97ac15ac00 2023-10-21T17:38:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.537+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac0acd10 con 0x7f97ac15ac00 2023-10-21T17:38:26.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.538+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac074150 con 0x7f97ac15ac00 2023-10-21T17:38:26.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.539+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f97a4014b20 con 0x7f97ac15ac00 2023-10-21T17:38:26.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.540+0000 7f97a1ffb640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 0x7f979006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:26.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.540+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f97a408a230 con 0x7f97ac15ac00 2023-10-21T17:38:26.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.541+0000 7f97b1230640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 0x7f979006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:26.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.542+0000 7f97b1230640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 0x7f979006d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97a8036000 tx=0x7f97a803d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:26.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.546+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97a4055050 con 0x7f97ac15ac00 2023-10-21T17:38:26.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:26.753+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97ac0ca440 con 0x7f97ac15ac00 2023-10-21T17:38:28.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.328+0000 7f97a1ffb640 1 -- 172.21.15.53:0/4163699861 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v229) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97a4058670 con 0x7f97ac15ac00 2023-10-21T17:38:28.330 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:38:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.332+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 msgr2=0x7f979006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.332+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 0x7f979006d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97a8036000 tx=0x7f97a803d040 comp rx=0 tx=0).stop 2023-10-21T17:38:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.332+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 msgr2=0x7f97ac158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.332+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007e30 tx=0x7f97a400e4b0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 shutdown_connections 2023-10-21T17:38:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97ac15ac00 0x7f97ac158690 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f979006af90 0x7f979006d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97ac15a820 0x7f97ac1f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 --2- 172.21.15.53:0/4163699861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97ac0ca680 0x7f97ac0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 >> 172.21.15.53:0/4163699861 conn(0x7f97ac06c050 msgr2=0x7f97ac15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:28.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 shutdown_connections 2023-10-21T17:38:28.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.333+0000 7f97b2232640 1 -- 172.21.15.53:0/4163699861 wait complete. 2023-10-21T17:38:28.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:38:28.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.586+0000 7f2c494bb640 1 Processor -- start 2023-10-21T17:38:28.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.587+0000 7f2c494bb640 1 -- start start 2023-10-21T17:38:28.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c494bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c494bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2c4406ad00 0x7f2c4406b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c494bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c440c9850 con 0x7f2c44071190 2023-10-21T17:38:28.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c494bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4406a870 con 0x7f2c4406ad00 2023-10-21T17:38:28.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c494bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4415e280 con 0x7f2c44070db0 2023-10-21T17:38:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.589+0000 7f2c42ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:37086/0 (socket says 172.21.15.53:37086) 2023-10-21T17:38:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 -- 172.21.15.53:0/2980383781 learned_addr learned my addr 172.21.15.53:0/2980383781 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c437fe640 1 --2- 172.21.15.53:0/2980383781 >> v2:172.21.15.124:3300/0 conn(0x7f2c4406ad00 0x7f2c4406b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 -- 172.21.15.53:0/2980383781 >> v2:172.21.15.124:3300/0 conn(0x7f2c4406ad00 msgr2=0x7f2c4406b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 --2- 172.21.15.53:0/2980383781 >> v2:172.21.15.124:3300/0 conn(0x7f2c4406ad00 0x7f2c4406b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 -- 172.21.15.53:0/2980383781 >> v1:172.21.15.124:6789/0 conn(0x7f2c44071190 legacy=0x7f2c4415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c42ffd640 1 -- 172.21.15.53:0/2980383781 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4415eb90 con 0x7f2c44070db0 2023-10-21T17:38:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.590+0000 7f2c427fc640 1 -- 172.21.15.53:0/2980383781 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393352951 0 0) 0x7f2c440c9850 con 0x7f2c44071190 2023-10-21T17:38:28.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.592+0000 7f2c42ffd640 1 --2- 172.21.15.53:0/2980383781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009870 tx=0x7f2c3402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd045e4edf887ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:28.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.592+0000 7f2c427fc640 1 -- 172.21.15.53:0/2980383781 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3402f840 con 0x7f2c44070db0 2023-10-21T17:38:28.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.592+0000 7f2c427fc640 1 -- 172.21.15.53:0/2980383781 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3402f9a0 con 0x7f2c44070db0 2023-10-21T17:38:28.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.592+0000 7f2c427fc640 1 -- 172.21.15.53:0/2980383781 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3402fd30 con 0x7f2c44070db0 2023-10-21T17:38:28.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 -- 172.21.15.53:0/2980383781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 msgr2=0x7f2c4415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 --2- 172.21.15.53:0/2980383781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009870 tx=0x7f2c3402ba20 comp rx=0 tx=0).stop 2023-10-21T17:38:28.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 -- 172.21.15.53:0/2980383781 shutdown_connections 2023-10-21T17:38:28.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 --2- 172.21.15.53:0/2980383781 >> v2:172.21.15.124:3300/0 conn(0x7f2c4406ad00 0x7f2c4406b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 --2- 172.21.15.53:0/2980383781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c4415dd40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 -- 172.21.15.53:0/2980383781 >> 172.21.15.53:0/2980383781 conn(0x7f2c440b1480 msgr2=0x7f2c4406cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:28.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 -- 172.21.15.53:0/2980383781 shutdown_connections 2023-10-21T17:38:28.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.593+0000 7f2c494bb640 1 -- 172.21.15.53:0/2980383781 wait complete. 2023-10-21T17:38:28.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.595+0000 7f2c494bb640 1 Processor -- start 2023-10-21T17:38:28.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.595+0000 7f2c494bb640 1 -- start start 2023-10-21T17:38:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c440b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c44071190 0x7f2c440c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440b2a90 con 0x7f2c4406ad00 2023-10-21T17:38:28.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440b2d70 con 0x7f2c44071190 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c494bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c440b3050 con 0x7f2c44070db0 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c42ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c42ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49936/0 (socket says 172.21.15.53:49936) 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.596+0000 7f2c42ffd640 1 -- 172.21.15.53:0/3099580896 learned_addr learned my addr 172.21.15.53:0/3099580896 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c3a7fc640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c440b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 -- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 msgr2=0x7f2c440b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c437fe640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c44071190 0x7f2c440c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c440b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 -- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c44071190 msgr2=0x7f2c440c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c44071190 0x7f2c440c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 -- 172.21.15.53:0/3099580896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c340094d0 con 0x7f2c4406ad00 2023-10-21T17:38:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c3a7fc640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c440b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.597+0000 7f2c42ffd640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.598+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34037c40 con 0x7f2c4406ad00 2023-10-21T17:38:28.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.598+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c34037da0 con 0x7f2c4406ad00 2023-10-21T17:38:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.598+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34041640 con 0x7f2c4406ad00 2023-10-21T17:38:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.598+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c440b3330 con 0x7f2c4406ad00 2023-10-21T17:38:28.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.598+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c44153410 con 0x7f2c4406ad00 2023-10-21T17:38:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.599+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4415fbb0 con 0x7f2c4406ad00 2023-10-21T17:38:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.600+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2c340417a0 con 0x7f2c4406ad00 2023-10-21T17:38:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.601+0000 7f2c3bfff640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 0x7f2c2406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:28.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.601+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f2c3403c070 con 0x7f2c4406ad00 2023-10-21T17:38:28.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.602+0000 7f2c3a7fc640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 0x7f2c2406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:28.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.604+0000 7f2c3a7fc640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 0x7f2c2406d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c440c7870 tx=0x7f2c2c009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:28.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.607+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c34077580 con 0x7f2c4406ad00 2023-10-21T17:38:28.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.869+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2c440cd5c0 con 0x7f2c4406ad00 2023-10-21T17:38:28.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.871+0000 7f2c3bfff640 1 -- 172.21.15.53:0/3099580896 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2c3407aba0 con 0x7f2c4406ad00 2023-10-21T17:38:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:28.873 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:38:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:38:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:38:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:38:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:38:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:38:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:38:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:38:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:38:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:38:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:38:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:38:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:38:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:38:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:38:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:38:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:38:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:38:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:38:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:38:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:38:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:38:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:38:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:38:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:38:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:38:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:38:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:38:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:38:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:38:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:38:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:38:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:38:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:38:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:38:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:38:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:38:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:38:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:38:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:38:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:38:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:38:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:38:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:38:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:38:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:38:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:38:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:38:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:38:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:38:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:38:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:38:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:38:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:38:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:38:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:38:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:38:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:38:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:38:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:38:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:38:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:38:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:38:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:38:28.909 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:38:28.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 msgr2=0x7f2c2406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 0x7f2c2406d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c440c7870 tx=0x7f2c2c009040 comp rx=0 tx=0).stop 2023-10-21T17:38:28.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 msgr2=0x7f2c44152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:28.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f2c34000c00 tx=0x7f2c34004730 comp rx=0 tx=0).stop 2023-10-21T17:38:28.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 shutdown_connections 2023-10-21T17:38:28.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c44071190 0x7f2c440c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2c2406afc0 0x7f2c2406d480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c44070db0 0x7f2c440b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 --2- 172.21.15.53:0/3099580896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c4406ad00 0x7f2c44152ed0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:28.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 >> 172.21.15.53:0/3099580896 conn(0x7f2c440b1480 msgr2=0x7f2c4415c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:28.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 shutdown_connections 2023-10-21T17:38:28.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:28.874+0000 7f2c494bb640 1 -- 172.21.15.53:0/3099580896 wait complete. 2023-10-21T17:38:28.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:29.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.142+0000 7f39e57e2640 1 Processor -- start 2023-10-21T17:38:29.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.142+0000 7f39e57e2640 1 -- start start 2023-10-21T17:38:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e57e2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e006b680 0x7f39e015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e57e2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f39e00ce2c0 0x7f39e015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e57e2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00cde30 con 0x7f39e0157480 2023-10-21T17:38:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e57e2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e0157860 con 0x7f39e00ce2c0 2023-10-21T17:38:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e57e2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e015e2c0 con 0x7f39e006b680 2023-10-21T17:38:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39dffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f39e0157480 0x7f39e006af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58542/0 (socket says 172.21.15.53:58542) 2023-10-21T17:38:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39dffff640 1 -- 172.21.15.53:0/818157730 learned_addr learned my addr 172.21.15.53:0/818157730 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:29.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.144+0000 7f39e4fe1640 1 --2- 172.21.15.53:0/818157730 >> v2:172.21.15.53:3300/0 conn(0x7f39e00ce2c0 0x7f39e015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39df7fe640 1 -- 172.21.15.53:0/818157730 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390036313 0 0) 0x7f39e00cde30 con 0x7f39e0157480 2023-10-21T17:38:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39df7fe640 1 -- 172.21.15.53:0/818157730 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc003520 con 0x7f39e0157480 2023-10-21T17:38:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39d7fff640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e006b680 0x7f39e015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39e4fe1640 1 -- 172.21.15.53:0/818157730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e006b680 msgr2=0x7f39e015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:29.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39e4fe1640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e006b680 0x7f39e015d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39e4fe1640 1 -- 172.21.15.53:0/818157730 >> v1:172.21.15.53:6789/0 conn(0x7f39e0157480 legacy=0x7f39e006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.145+0000 7f39e4fe1640 1 -- 172.21.15.53:0/818157730 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00cd500 con 0x7f39e00ce2c0 2023-10-21T17:38:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.146+0000 7f39e4fe1640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e00ce2c0 0x7f39e015da90 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009d90 tx=0x7f39d802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddf59af0d783fb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.146+0000 7f39df7fe640 1 -- 172.21.15.53:0/818157730 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d802cbd0 con 0x7f39e00ce2c0 2023-10-21T17:38:29.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39df7fe640 1 -- 172.21.15.53:0/818157730 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d802cd30 con 0x7f39e00ce2c0 2023-10-21T17:38:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39df7fe640 1 -- 172.21.15.53:0/818157730 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d802cea0 con 0x7f39e00ce2c0 2023-10-21T17:38:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 -- 172.21.15.53:0/818157730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e00ce2c0 msgr2=0x7f39e015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e00ce2c0 0x7f39e015da90 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009d90 tx=0x7f39d802be20 comp rx=0 tx=0).stop 2023-10-21T17:38:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 -- 172.21.15.53:0/818157730 shutdown_connections 2023-10-21T17:38:29.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e00ce2c0 0x7f39e015da90 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 --2- 172.21.15.53:0/818157730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e006b680 0x7f39e015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 -- 172.21.15.53:0/818157730 >> 172.21.15.53:0/818157730 conn(0x7f39e00d2ae0 msgr2=0x7f39e00d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 -- 172.21.15.53:0/818157730 shutdown_connections 2023-10-21T17:38:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.147+0000 7f39e57e2640 1 -- 172.21.15.53:0/818157730 wait complete. 2023-10-21T17:38:29.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.149+0000 7f39e57e2640 1 Processor -- start 2023-10-21T17:38:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.149+0000 7f39e57e2640 1 -- start start 2023-10-21T17:38:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e00ce2c0 0x7f39e00c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e0157480 0x7f39e0150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01f3950 con 0x7f39e006b680 2023-10-21T17:38:29.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01f2ec0 con 0x7f39e00ce2c0 2023-10-21T17:38:29.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.150+0000 7f39e57e2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e01f61d0 con 0x7f39e0157480 2023-10-21T17:38:29.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49958/0 (socket says 172.21.15.53:49958) 2023-10-21T17:38:29.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 -- 172.21.15.53:0/869688773 learned_addr learned my addr 172.21.15.53:0/869688773 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 -- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e0157480 msgr2=0x7f39e0150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e0157480 0x7f39e0150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 -- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e00ce2c0 msgr2=0x7f39e00c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e00ce2c0 0x7f39e00c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 -- 172.21.15.53:0/869688773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d8009a70 con 0x7f39e006b680 2023-10-21T17:38:29.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.151+0000 7f39dffff640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c500 tx=0x7f39d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.152+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0005ce0 con 0x7f39e006b680 2023-10-21T17:38:29.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.152+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0005e40 con 0x7f39e006b680 2023-10-21T17:38:29.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.152+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0007330 con 0x7f39e006b680 2023-10-21T17:38:29.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.152+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e00c8b30 con 0x7f39e006b680 2023-10-21T17:38:29.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.152+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e01f3390 con 0x7f39e006b680 2023-10-21T17:38:29.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.153+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e01f5fa0 con 0x7f39e006b680 2023-10-21T17:38:29.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.155+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f39d0007550 con 0x7f39e006b680 2023-10-21T17:38:29.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.155+0000 7f39dd7fa640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 0x7f39c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.156+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f39d0088bc0 con 0x7f39e006b680 2023-10-21T17:38:29.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.156+0000 7f39d7fff640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 0x7f39c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.157+0000 7f39d7fff640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 0x7f39c006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39e015eb40 tx=0x7f39c8008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.160+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39e01f5fa0 con 0x7f39e006b680 2023-10-21T17:38:29.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.354+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39e01f5fa0 con 0x7f39e006b680 2023-10-21T17:38:29.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.356+0000 7f39dd7fa640 1 -- 172.21.15.53:0/869688773 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f39e01f5fa0 con 0x7f39e006b680 2023-10-21T17:38:29.357 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:29.358 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":229,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:28.326197+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.358+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 msgr2=0x7f39c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.358+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 0x7f39c006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39e015eb40 tx=0x7f39c8008040 comp rx=0 tx=0).stop 2023-10-21T17:38:29.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.358+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 msgr2=0x7f39e01f2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.358+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c500 tx=0x7f39d000c9c0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 shutdown_connections 2023-10-21T17:38:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f39c006af90 0x7f39c006d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f39e0157480 0x7f39e0150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f39e00ce2c0 0x7f39e00c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 --2- 172.21.15.53:0/869688773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39e006b680 0x7f39e01f2980 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 >> 172.21.15.53:0/869688773 conn(0x7f39e00d2ae0 msgr2=0x7f39e00d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:29.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 shutdown_connections 2023-10-21T17:38:29.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.359+0000 7f39e57e2640 1 -- 172.21.15.53:0/869688773 wait complete. 2023-10-21T17:38:29.382 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:38:29.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:38:29.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.616+0000 7f5076485640 1 Processor -- start 2023-10-21T17:38:29.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.616+0000 7f5076485640 1 -- start start 2023-10-21T17:38:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.618+0000 7f5076485640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50700cdc30 0x7f50700cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.618+0000 7f5076485640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.618+0000 7f5076485640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507015ac90 con 0x7f50700caef0 2023-10-21T17:38:29.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.618+0000 7f5076485640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507015a6d0 con 0x7f50700cdc30 2023-10-21T17:38:29.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.618+0000 7f5076485640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507015a9b0 con 0x7f50700cce90 2023-10-21T17:38:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:37114/0 (socket says 172.21.15.53:37114) 2023-10-21T17:38:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 -- 172.21.15.53:0/1948830777 learned_addr learned my addr 172.21.15.53:0/1948830777 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5074c82640 1 --2- 172.21.15.53:0/1948830777 >> v2:172.21.15.124:3300/0 conn(0x7f50700cdc30 0x7f50700cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 -- 172.21.15.53:0/1948830777 >> v2:172.21.15.124:3300/0 conn(0x7f50700cdc30 msgr2=0x7f50700cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 --2- 172.21.15.53:0/1948830777 >> v2:172.21.15.124:3300/0 conn(0x7f50700cdc30 0x7f50700cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.619+0000 7f5075c84640 1 -- 172.21.15.53:0/1948830777 >> v1:172.21.15.124:6789/0 conn(0x7f50700caef0 legacy=0x7f50700cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.620+0000 7f5075c84640 1 -- 172.21.15.53:0/1948830777 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700cc150 con 0x7f50700cce90 2023-10-21T17:38:29.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.621+0000 7f5075c84640 1 --2- 172.21.15.53:0/1948830777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5068009e50 tx=0x7f506802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f0d2226b26c6b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.621+0000 7f5067fff640 1 -- 172.21.15.53:0/1948830777 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506802cb00 con 0x7f50700cce90 2023-10-21T17:38:29.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5067fff640 1 -- 172.21.15.53:0/1948830777 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506802cc60 con 0x7f50700cce90 2023-10-21T17:38:29.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5067fff640 1 -- 172.21.15.53:0/1948830777 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680343f0 con 0x7f50700cce90 2023-10-21T17:38:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 -- 172.21.15.53:0/1948830777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 msgr2=0x7f5070159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 --2- 172.21.15.53:0/1948830777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5068009e50 tx=0x7f506802bf60 comp rx=0 tx=0).stop 2023-10-21T17:38:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 -- 172.21.15.53:0/1948830777 shutdown_connections 2023-10-21T17:38:29.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 --2- 172.21.15.53:0/1948830777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070159bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 --2- 172.21.15.53:0/1948830777 >> v2:172.21.15.124:3300/0 conn(0x7f50700cdc30 0x7f50700cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.622+0000 7f5076485640 1 -- 172.21.15.53:0/1948830777 >> 172.21.15.53:0/1948830777 conn(0x7f507006c010 msgr2=0x7f507006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.623+0000 7f5076485640 1 -- 172.21.15.53:0/1948830777 shutdown_connections 2023-10-21T17:38:29.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.623+0000 7f5076485640 1 -- 172.21.15.53:0/1948830777 wait complete. 2023-10-21T17:38:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.624+0000 7f5076485640 1 Processor -- start 2023-10-21T17:38:29.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.624+0000 7f5076485640 1 -- start start 2023-10-21T17:38:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.625+0000 7f5076485640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 0x7f5070143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.625+0000 7f5076485640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.625+0000 7f5076485640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5076485640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070146b40 con 0x7f50700cdc30 2023-10-21T17:38:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5076485640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070143db0 con 0x7f50700caef0 2023-10-21T17:38:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5076485640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50701460a0 con 0x7f50700cce90 2023-10-21T17:38:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5075c84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5075c84640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49966/0 (socket says 172.21.15.53:49966) 2023-10-21T17:38:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5075c84640 1 -- 172.21.15.53:0/1342284695 learned_addr learned my addr 172.21.15.53:0/1342284695 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5074c82640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.626+0000 7f5075483640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 0x7f5070143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 -- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 msgr2=0x7f5070145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 -- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 msgr2=0x7f5070143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 0x7f5070143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 -- 172.21.15.53:0/1342284695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5068009b30 con 0x7f50700cdc30 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075483640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 0x7f5070143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5074c82640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5075c84640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5068036040 tx=0x7f5068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068009510 con 0x7f50700cdc30 2023-10-21T17:38:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506803e450 con 0x7f50700cdc30 2023-10-21T17:38:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50701462a0 con 0x7f50700cdc30 2023-10-21T17:38:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.627+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506803e800 con 0x7f50700cdc30 2023-10-21T17:38:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.628+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070146860 con 0x7f50700cdc30 2023-10-21T17:38:29.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.629+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5070143b10 con 0x7f50700cdc30 2023-10-21T17:38:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.630+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f506803e960 con 0x7f50700cdc30 2023-10-21T17:38:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.630+0000 7f5065ffb640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 0x7f505406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:29.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.631+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f50680ac2f0 con 0x7f50700cdc30 2023-10-21T17:38:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.631+0000 7f5075483640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 0x7f505406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:29.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.632+0000 7f5075483640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 0x7f505406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f60 tx=0x7f506c009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:29.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.635+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5070143b10 con 0x7f50700cdc30 2023-10-21T17:38:29.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.826+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5070143b10 con 0x7f50700cdc30 2023-10-21T17:38:29.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.827+0000 7f5065ffb640 1 -- 172.21.15.53:0/1342284695 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v229) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5070143b10 con 0x7f50700cdc30 2023-10-21T17:38:29.828 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:38:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 msgr2=0x7f505406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 0x7f505406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f506c000f60 tx=0x7f506c009040 comp rx=0 tx=0).stop 2023-10-21T17:38:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 msgr2=0x7f50700c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:29.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f5068036040 tx=0x7f5068004a40 comp rx=0 tx=0).stop 2023-10-21T17:38:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 shutdown_connections 2023-10-21T17:38:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f50700cdc30 0x7f50700c5ec0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f505406afc0 0x7f505406d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50700cce90 0x7f5070145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 --2- 172.21.15.53:0/1342284695 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50700caef0 0x7f5070143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.829+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 >> 172.21.15.53:0/1342284695 conn(0x7f507006c010 msgr2=0x7f5070073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.830+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 shutdown_connections 2023-10-21T17:38:29.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:29.830+0000 7f5076485640 1 -- 172.21.15.53:0/1342284695 wait complete. 2023-10-21T17:38:29.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:38:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.075+0000 7f64dcdd5640 1 Processor -- start 2023-10-21T17:38:30.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.075+0000 7f64dcdd5640 1 -- start start 2023-10-21T17:38:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64dcdd5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64d815aad0 0x7f64d80cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64dcdd5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80cb580 con 0x7f64d815aeb0 2023-10-21T17:38:30.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64dcdd5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80cc0e0 con 0x7f64d80cc570 2023-10-21T17:38:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64dcdd5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80b2390 con 0x7f64d815aad0 2023-10-21T17:38:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64d659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64d815aad0 0x7f64d80cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64d5d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f64d815aeb0 0x7f64d80ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58558/0 (socket says 172.21.15.53:58558) 2023-10-21T17:38:30.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64d5d9b640 1 -- 172.21.15.53:0/856361736 learned_addr learned my addr 172.21.15.53:0/856361736 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.077+0000 7f64d659c640 1 -- 172.21.15.53:0/856361736 >> v1:172.21.15.124:6789/0 conn(0x7f64d80cc570 legacy=0x7f64d80cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:30.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.078+0000 7f64d659c640 1 -- 172.21.15.53:0/856361736 >> v1:172.21.15.53:6789/0 conn(0x7f64d815aeb0 legacy=0x7f64d80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.078+0000 7f64d659c640 1 -- 172.21.15.53:0/856361736 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80cb7b0 con 0x7f64d815aad0 2023-10-21T17:38:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.079+0000 7f64d659c640 1 --2- 172.21.15.53:0/856361736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aad0 0x7f64d80cdf60 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009790 tx=0x7f64c802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fbeae3e2075cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.079+0000 7f64d559a640 1 -- 172.21.15.53:0/856361736 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8030800 con 0x7f64d815aad0 2023-10-21T17:38:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.079+0000 7f64d559a640 1 -- 172.21.15.53:0/856361736 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8030960 con 0x7f64d815aad0 2023-10-21T17:38:30.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.079+0000 7f64d559a640 1 -- 172.21.15.53:0/856361736 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8030cf0 con 0x7f64d815aad0 2023-10-21T17:38:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 -- 172.21.15.53:0/856361736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aad0 msgr2=0x7f64d80cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/856361736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aad0 0x7f64d80cdf60 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009790 tx=0x7f64c802b940 comp rx=0 tx=0).stop 2023-10-21T17:38:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 -- 172.21.15.53:0/856361736 shutdown_connections 2023-10-21T17:38:30.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/856361736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aad0 0x7f64d80cdf60 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 -- 172.21.15.53:0/856361736 >> 172.21.15.53:0/856361736 conn(0x7f64d80d09e0 msgr2=0x7f64d80d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 -- 172.21.15.53:0/856361736 shutdown_connections 2023-10-21T17:38:30.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.080+0000 7f64dcdd5640 1 -- 172.21.15.53:0/856361736 wait complete. 2023-10-21T17:38:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.082+0000 7f64dcdd5640 1 Processor -- start 2023-10-21T17:38:30.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.082+0000 7f64dcdd5640 1 -- start start 2023-10-21T17:38:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d80cc570 0x7f64d80b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d815aad0 0x7f64d81f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80cedc0 con 0x7f64d815aeb0 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81f35b0 con 0x7f64d80cc570 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.083+0000 7f64dcdd5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d814e700 con 0x7f64d815aad0 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49982/0 (socket says 172.21.15.53:49982) 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 -- 172.21.15.53:0/3581468806 learned_addr learned my addr 172.21.15.53:0/3581468806 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:30.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 -- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d815aad0 msgr2=0x7f64d81f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d815aad0 0x7f64d81f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 -- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d80cc570 msgr2=0x7f64d80b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d80cc570 0x7f64d80b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 -- 172.21.15.53:0/3581468806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80093a0 con 0x7f64d815aeb0 2023-10-21T17:38:30.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.084+0000 7f64d6d9d640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f64d000c410 tx=0x7f64d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.085+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0011790 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.085+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d00118f0 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.085+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0011ce0 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.085+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d81f32e0 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.085+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80c8f80 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.086+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d8067490 con 0x7f64d815aeb0 2023-10-21T17:38:30.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.087+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f64d001c460 con 0x7f64d815aeb0 2023-10-21T17:38:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.088+0000 7f64ceffd640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 0x7f64b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.088+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f64d00893c0 con 0x7f64d815aeb0 2023-10-21T17:38:30.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.089+0000 7f64d659c640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 0x7f64b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.090+0000 7f64d659c640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 0x7f64b806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64c8036000 tx=0x7f64c803b040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.093+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64d0054460 con 0x7f64d815aeb0 2023-10-21T17:38:30.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.292+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f64d808aae0 con 0x7f64d815aeb0 2023-10-21T17:38:30.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.293+0000 7f64ceffd640 1 -- 172.21.15.53:0/3581468806 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v229) v1 ==== 84+0+65 (secure 0 0 0) 0x7f64d0057a80 con 0x7f64d815aeb0 2023-10-21T17:38:30.294 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:30.295 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:38:30.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 msgr2=0x7f64b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 0x7f64b806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64c8036000 tx=0x7f64c803b040 comp rx=0 tx=0).stop 2023-10-21T17:38:30.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 msgr2=0x7f64d81f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f64d000c410 tx=0x7f64d000c8d0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 shutdown_connections 2023-10-21T17:38:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64d815aeb0 0x7f64d81f3770 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f64b806af90 0x7f64b806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64d815aad0 0x7f64d81f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 --2- 172.21.15.53:0/3581468806 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64d80cc570 0x7f64d80b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 >> 172.21.15.53:0/3581468806 conn(0x7f64d80d09e0 msgr2=0x7f64d80ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:30.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 shutdown_connections 2023-10-21T17:38:30.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.296+0000 7f64dcdd5640 1 -- 172.21.15.53:0/3581468806 wait complete. 2023-10-21T17:38:30.315 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:38:30.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:38:30.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.536+0000 7f8e0c183640 1 Processor -- start 2023-10-21T17:38:30.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.537+0000 7f8e0c183640 1 -- start start 2023-10-21T17:38:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e0415ce00 0x7f8e040b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 0x7f8e040cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e040d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040d0660 con 0x7f8e0415ce00 2023-10-21T17:38:30.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040cdbb0 con 0x7f8e0415d1e0 2023-10-21T17:38:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.538+0000 7f8e0c183640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04072430 con 0x7f8e040cfcb0 2023-10-21T17:38:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e0415ce00 0x7f8e040b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e0415ce00 0x7f8e040b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49992/0 (socket says 172.21.15.53:49992) 2023-10-21T17:38:30.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 -- 172.21.15.53:0/939272679 learned_addr learned my addr 172.21.15.53:0/939272679 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e0a720640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e040d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 -- 172.21.15.53:0/939272679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 msgr2=0x7f8e040d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:30.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e0971e640 1 --2- 172.21.15.53:0/939272679 >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 0x7f8e040cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e040d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 -- 172.21.15.53:0/939272679 >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 msgr2=0x7f8e040cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/939272679 >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 0x7f8e040cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e09f1f640 1 -- 172.21.15.53:0/939272679 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04072150 con 0x7f8e0415ce00 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e0a720640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e040d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:30.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.539+0000 7f8e0971e640 1 --2- 172.21.15.53:0/939272679 >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 0x7f8e040cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.541+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415ce00 0x7f8e040b1ac0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f8e04160a60 tx=0x7f8e0002b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2ebfc551a84dcf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.541+0000 7f8e08f1d640 1 -- 172.21.15.53:0/939272679 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000317f0 con 0x7f8e0415ce00 2023-10-21T17:38:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.541+0000 7f8e08f1d640 1 -- 172.21.15.53:0/939272679 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00031950 con 0x7f8e0415ce00 2023-10-21T17:38:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.541+0000 7f8e08f1d640 1 -- 172.21.15.53:0/939272679 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00031ce0 con 0x7f8e0415ce00 2023-10-21T17:38:30.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 -- 172.21.15.53:0/939272679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415ce00 msgr2=0x7f8e040b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415ce00 0x7f8e040b1ac0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f8e04160a60 tx=0x7f8e0002b890 comp rx=0 tx=0).stop 2023-10-21T17:38:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 -- 172.21.15.53:0/939272679 shutdown_connections 2023-10-21T17:38:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e040d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 --2- 172.21.15.53:0/939272679 >> v2:172.21.15.124:3300/0 conn(0x7f8e0415d1e0 0x7f8e040cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 --2- 172.21.15.53:0/939272679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415ce00 0x7f8e040b1ac0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 -- 172.21.15.53:0/939272679 >> 172.21.15.53:0/939272679 conn(0x7f8e0406b250 msgr2=0x7f8e0406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 -- 172.21.15.53:0/939272679 shutdown_connections 2023-10-21T17:38:30.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.542+0000 7f8e0c183640 1 -- 172.21.15.53:0/939272679 wait complete. 2023-10-21T17:38:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.544+0000 7f8e0c183640 1 Processor -- start 2023-10-21T17:38:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.544+0000 7f8e0c183640 1 -- start start 2023-10-21T17:38:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e0414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 0x7f8e040b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0414e2e0 con 0x7f8e0415d1e0 2023-10-21T17:38:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040c7680 con 0x7f8e0415ce00 2023-10-21T17:38:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0c183640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040c6a70 con 0x7f8e040cfcb0 2023-10-21T17:38:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0a720640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0a720640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50004/0 (socket says 172.21.15.53:50004) 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.545+0000 7f8e0a720640 1 -- 172.21.15.53:0/3626528226 learned_addr learned my addr 172.21.15.53:0/3626528226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e0414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0971e640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 0x7f8e040b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 -- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 msgr2=0x7f8e0414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e0414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 -- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 msgr2=0x7f8e040b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 0x7f8e040b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 -- 172.21.15.53:0/3626528226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000094d0 con 0x7f8e0415d1e0 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0971e640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 0x7f8e040b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0a720640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00cd50 tx=0x7f8dfc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00f680 con 0x7f8e0415d1e0 2023-10-21T17:38:30.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00f800 con 0x7f8e0415d1e0 2023-10-21T17:38:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e040c6d50 con 0x7f8e0415d1e0 2023-10-21T17:38:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.546+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00fbb0 con 0x7f8e0415d1e0 2023-10-21T17:38:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.547+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e040c7310 con 0x7f8e0415d1e0 2023-10-21T17:38:30.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.547+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e040c7830 con 0x7f8e0415d1e0 2023-10-21T17:38:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.549+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8dfc00b040 con 0x7f8e0415d1e0 2023-10-21T17:38:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.550+0000 7f8dfa7fc640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 0x7f8de806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:30.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.550+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f8dfc08a080 con 0x7f8e0415d1e0 2023-10-21T17:38:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.550+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 0x7f8de806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.552+0000 7f8e09f1f640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 0x7f8de806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e0003d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:30.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.554+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e040c7830 con 0x7f8e0415d1e0 2023-10-21T17:38:30.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:30.745+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e04071d80 con 0x7f8e0415d1e0 2023-10-21T17:38:31.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.370+0000 7f8dfa7fc640 1 -- 172.21.15.53:0/3626528226 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v230) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8dfc055140 con 0x7f8e0415d1e0 2023-10-21T17:38:31.372 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:38:31.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 msgr2=0x7f8de806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 0x7f8de806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e0003d040 comp rx=0 tx=0).stop 2023-10-21T17:38:31.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 msgr2=0x7f8e040b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00cd50 tx=0x7f8dfc00a650 comp rx=0 tx=0).stop 2023-10-21T17:38:31.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 shutdown_connections 2023-10-21T17:38:31.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e0415d1e0 0x7f8e040b4b20 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e0415ce00 0x7f8e040b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f8de806af90 0x7f8de806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 --2- 172.21.15.53:0/3626528226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e040cfcb0 0x7f8e0414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.374+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 >> 172.21.15.53:0/3626528226 conn(0x7f8e0406b250 msgr2=0x7f8e040af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.375+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 shutdown_connections 2023-10-21T17:38:31.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.376+0000 7f8e0c183640 1 -- 172.21.15.53:0/3626528226 wait complete. 2023-10-21T17:38:31.403 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:38:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.617+0000 7f6714a40640 1 Processor -- start 2023-10-21T17:38:31.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.618+0000 7f6714a40640 1 -- start start 2023-10-21T17:38:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.619+0000 7f6714a40640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67100cb010 0x7f6710065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:31.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.619+0000 7f6714a40640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100a00a0 con 0x7f671015eec0 2023-10-21T17:38:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.619+0000 7f6714a40640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671006e430 con 0x7f67100cac30 2023-10-21T17:38:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.619+0000 7f6714a40640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671006c730 con 0x7f67100cb010 2023-10-21T17:38:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f6707fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f67100cb010 0x7f6710065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f670ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f671015eec0 0x7f671006ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58574/0 (socket says 172.21.15.53:58574) 2023-10-21T17:38:31.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f670ffff640 1 -- 172.21.15.53:0/2886244748 learned_addr learned my addr 172.21.15.53:0/2886244748 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f6707fff640 1 -- 172.21.15.53:0/2886244748 >> v1:172.21.15.124:6789/0 conn(0x7f67100cac30 legacy=0x7f671006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f6707fff640 1 -- 172.21.15.53:0/2886244748 >> v1:172.21.15.53:6789/0 conn(0x7f671015eec0 legacy=0x7f671006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f6707fff640 1 -- 172.21.15.53:0/2886244748 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671006ca10 con 0x7f67100cb010 2023-10-21T17:38:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.620+0000 7f670effd640 1 -- 172.21.15.53:0/2886244748 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808235896 0 0) 0x7f67100a00a0 con 0x7f671015eec0 2023-10-21T17:38:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f6707fff640 1 --2- 172.21.15.53:0/2886244748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cb010 0x7f6710065d90 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097a0 tx=0x7f66f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff560beed716f4c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f670effd640 1 -- 172.21.15.53:0/2886244748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8030610 con 0x7f67100cb010 2023-10-21T17:38:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f670effd640 1 -- 172.21.15.53:0/2886244748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8030770 con 0x7f67100cb010 2023-10-21T17:38:31.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f670effd640 1 -- 172.21.15.53:0/2886244748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8030b00 con 0x7f67100cb010 2023-10-21T17:38:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f6714a40640 1 -- 172.21.15.53:0/2886244748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cb010 msgr2=0x7f6710065d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.622+0000 7f6714a40640 1 --2- 172.21.15.53:0/2886244748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cb010 0x7f6710065d90 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097a0 tx=0x7f66f802b960 comp rx=0 tx=0).stop 2023-10-21T17:38:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.623+0000 7f6714a40640 1 -- 172.21.15.53:0/2886244748 shutdown_connections 2023-10-21T17:38:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.623+0000 7f6714a40640 1 --2- 172.21.15.53:0/2886244748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cb010 0x7f6710065d90 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.623+0000 7f6714a40640 1 -- 172.21.15.53:0/2886244748 >> 172.21.15.53:0/2886244748 conn(0x7f67100b0990 msgr2=0x7f67100b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.623+0000 7f6714a40640 1 -- 172.21.15.53:0/2886244748 shutdown_connections 2023-10-21T17:38:31.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.623+0000 7f6714a40640 1 -- 172.21.15.53:0/2886244748 wait complete. 2023-10-21T17:38:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.624+0000 7f6714a40640 1 Processor -- start 2023-10-21T17:38:31.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.625+0000 7f6714a40640 1 -- start start 2023-10-21T17:38:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67100cb010 0x7f6710099460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 0x7f67101f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671009c9b0 con 0x7f67100cac30 2023-10-21T17:38:31.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671009bd30 con 0x7f67100cb010 2023-10-21T17:38:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6714a40640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671009c010 con 0x7f671015eec0 2023-10-21T17:38:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50026/0 (socket says 172.21.15.53:50026) 2023-10-21T17:38:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 -- 172.21.15.53:0/504769990 learned_addr learned my addr 172.21.15.53:0/504769990 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:31.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f6707fff640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67100cb010 0x7f6710099460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670ffff640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 0x7f67101f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 -- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 msgr2=0x7f67101f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 0x7f67101f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 -- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67100cb010 msgr2=0x7f6710099460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:31.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.626+0000 7f670f7fe640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67100cb010 0x7f6710099460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670f7fe640 1 -- 172.21.15.53:0/504769990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8009400 con 0x7f67100cac30 2023-10-21T17:38:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670ffff640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 0x7f67101f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670f7fe640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f670000c430 tx=0x7f670000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011790 con 0x7f67100cac30 2023-10-21T17:38:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67000118f0 con 0x7f67100cac30 2023-10-21T17:38:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671009c2f0 con 0x7f67100cac30 2023-10-21T17:38:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011ca0 con 0x7f67100cac30 2023-10-21T17:38:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.627+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671015af70 con 0x7f67100cac30 2023-10-21T17:38:31.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.628+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671015b100 con 0x7f67100cac30 2023-10-21T17:38:31.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.631+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f670001c460 con 0x7f67100cac30 2023-10-21T17:38:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.631+0000 7f670cff9640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 0x7f66f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.632+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f6700089f40 con 0x7f67100cac30 2023-10-21T17:38:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.632+0000 7f6707fff640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 0x7f66f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:31.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.633+0000 7f6707fff640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 0x7f66f006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66f8036000 tx=0x7f66f8008030 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:31.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.636+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6700054ff0 con 0x7f67100cac30 2023-10-21T17:38:31.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:31.836+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f671009caf0 con 0x7f67100cac30 2023-10-21T17:38:32.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.377+0000 7f670cff9640 1 -- 172.21.15.53:0/504769990 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v231) v1 ==== 170+0+0 (secure 0 0 0) 0x7f671009caf0 con 0x7f67100cac30 2023-10-21T17:38:32.379 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:38:32.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 msgr2=0x7f66f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 0x7f66f006d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66f8036000 tx=0x7f66f8008030 comp rx=0 tx=0).stop 2023-10-21T17:38:32.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 msgr2=0x7f67100c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f670000c430 tx=0x7f670000c8f0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 shutdown_connections 2023-10-21T17:38:32.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f66f006af90 0x7f66f006d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f671015eec0 0x7f67101f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67100cb010 0x7f6710099460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 --2- 172.21.15.53:0/504769990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67100cac30 0x7f67100c5400 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 >> 172.21.15.53:0/504769990 conn(0x7f67100b0990 msgr2=0x7f671015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:32.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.381+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 shutdown_connections 2023-10-21T17:38:32.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.382+0000 7f6714a40640 1 -- 172.21.15.53:0/504769990 wait complete. 2023-10-21T17:38:32.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-21T17:38:32.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:38:32.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.613+0000 7f115d21f640 1 Processor -- start 2023-10-21T17:38:32.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.613+0000 7f115d21f640 1 -- start start 2023-10-21T17:38:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f115d21f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f115815adf0 0x7f11580b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f115d21f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f11580cd410 0x7f11580cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f115d21f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580cde00 con 0x7f11580d05b0 2023-10-21T17:38:32.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f115d21f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115815b1d0 con 0x7f11580cd410 2023-10-21T17:38:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f115d21f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580cf510 con 0x7f115815adf0 2023-10-21T17:38:32.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.615+0000 7f1156d9d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f115815adf0 0x7f11580b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f114e59c640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f11580d05b0 0x7f11580ced30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:36696/0 (socket says 172.21.15.53:36696) 2023-10-21T17:38:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f114e59c640 1 -- 172.21.15.53:0/3066955950 learned_addr learned my addr 172.21.15.53:0/3066955950 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f115759e640 1 --2- 172.21.15.53:0/3066955950 >> v2:172.21.15.124:3300/0 conn(0x7f11580cd410 0x7f11580cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f115759e640 1 -- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f115815adf0 msgr2=0x7f11580b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f115759e640 1 --2- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f115815adf0 0x7f11580b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f115759e640 1 -- 172.21.15.53:0/3066955950 >> v1:172.21.15.124:6789/0 conn(0x7f11580d05b0 legacy=0x7f11580ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.616+0000 7f115759e640 1 -- 172.21.15.53:0/3066955950 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580cf7f0 con 0x7f11580cd410 2023-10-21T17:38:32.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.618+0000 7f115759e640 1 --2- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580cd410 0x7f11580cd8c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5767c03d5b5e6a88 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:32.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.618+0000 7f115659c640 1 -- 172.21.15.53:0/3066955950 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115002d040 con 0x7f11580cd410 2023-10-21T17:38:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.618+0000 7f115659c640 1 -- 172.21.15.53:0/3066955950 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150031810 con 0x7f11580cd410 2023-10-21T17:38:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.618+0000 7f115659c640 1 -- 172.21.15.53:0/3066955950 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150031ba0 con 0x7f11580cd410 2023-10-21T17:38:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 -- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580cd410 msgr2=0x7f11580cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 --2- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580cd410 0x7f11580cd8c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 -- 172.21.15.53:0/3066955950 shutdown_connections 2023-10-21T17:38:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 --2- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580cd410 0x7f11580cd8c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 --2- 172.21.15.53:0/3066955950 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f115815adf0 0x7f11580b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 -- 172.21.15.53:0/3066955950 >> 172.21.15.53:0/3066955950 conn(0x7f115806e630 msgr2=0x7f115806ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 -- 172.21.15.53:0/3066955950 shutdown_connections 2023-10-21T17:38:32.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.619+0000 7f115d21f640 1 -- 172.21.15.53:0/3066955950 wait complete. 2023-10-21T17:38:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.621+0000 7f115d21f640 1 Processor -- start 2023-10-21T17:38:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.621+0000 7f115d21f640 1 -- start start 2023-10-21T17:38:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11580cd410 0x7f11580b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580d05b0 0x7f1158156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580c9380 con 0x7f115815adf0 2023-10-21T17:38:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580c9660 con 0x7f11580d05b0 2023-10-21T17:38:32.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115d21f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580c9940 con 0x7f11580cd410 2023-10-21T17:38:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.622+0000 7f115759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50030/0 (socket says 172.21.15.53:50030) 2023-10-21T17:38:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 -- 172.21.15.53:0/433085165 learned_addr learned my addr 172.21.15.53:0/433085165 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f114e59c640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580d05b0 0x7f1158156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f1156d9d640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11580cd410 0x7f11580b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 -- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11580cd410 msgr2=0x7f11580b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11580cd410 0x7f11580b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 -- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580d05b0 msgr2=0x7f1158156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580d05b0 0x7f1158156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 -- 172.21.15.53:0/433085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150009400 con 0x7f115815adf0 2023-10-21T17:38:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.623+0000 7f115759e640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f115002b8f0 tx=0x7f11500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.624+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115003ac60 con 0x7f115815adf0 2023-10-21T17:38:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.624+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115003ade0 con 0x7f115815adf0 2023-10-21T17:38:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.624+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150044630 con 0x7f115815adf0 2023-10-21T17:38:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.624+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580c9c20 con 0x7f115815adf0 2023-10-21T17:38:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.624+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11580ca420 con 0x7f115815adf0 2023-10-21T17:38:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.625+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11580ca140 con 0x7f115815adf0 2023-10-21T17:38:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.627+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1150044790 con 0x7f115815adf0 2023-10-21T17:38:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.627+0000 7f114ffff640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 0x7f113806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.627+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f11500ac280 con 0x7f115815adf0 2023-10-21T17:38:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.628+0000 7f1156d9d640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 0x7f113806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.629+0000 7f1156d9d640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 0x7f113806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11480073f0 tx=0x7f114800b040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:32.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.631+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1150077150 con 0x7f115815adf0 2023-10-21T17:38:32.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:32.858+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f11580ca140 con 0x7f115815adf0 2023-10-21T17:38:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.398+0000 7f114ffff640 1 -- 172.21.15.53:0/433085165 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7f11580ca140 con 0x7f115815adf0 2023-10-21T17:38:33.400 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 27 and data pool 28 2023-10-21T17:38:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 msgr2=0x7f113806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 0x7f113806d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11480073f0 tx=0x7f114800b040 comp rx=0 tx=0).stop 2023-10-21T17:38:33.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 msgr2=0x7f11580b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f115002b8f0 tx=0x7f11500046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 shutdown_connections 2023-10-21T17:38:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f115815adf0 0x7f11580b4f60 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11580d05b0 0x7f1158156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f113806afc0 0x7f113806d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 --2- 172.21.15.53:0/433085165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11580cd410 0x7f11580b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 >> 172.21.15.53:0/433085165 conn(0x7f115806e630 msgr2=0x7f115806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 shutdown_connections 2023-10-21T17:38:33.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.402+0000 7f115d21f640 1 -- 172.21.15.53:0/433085165 wait complete. 2023-10-21T17:38:33.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:33.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.611+0000 7f7b35397640 1 Processor -- start 2023-10-21T17:38:33.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.611+0000 7f7b35397640 1 -- start start 2023-10-21T17:38:33.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b35397640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b3015ab40 0x7f7b300ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b35397640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b300adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b35397640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3015b2b0 con 0x7f7b300cae70 2023-10-21T17:38:33.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b35397640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3015bf30 con 0x7f7b3015ab40 2023-10-21T17:38:33.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b35397640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015f7a0 con 0x7f7b300c90b0 2023-10-21T17:38:33.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b2f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7b300cae70 0x7f7b300702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58578/0 (socket says 172.21.15.53:58578) 2023-10-21T17:38:33.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.613+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/3981881386 learned_addr learned my addr 172.21.15.53:0/3981881386 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2e7fc640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b300adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2effd640 1 --2- 172.21.15.53:0/3981881386 >> v2:172.21.15.124:3300/0 conn(0x7f7b3015ab40 0x7f7b300ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429883634 0 0) 0x7f7b3015b2b0 con 0x7f7b300cae70 2023-10-21T17:38:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b18003520 con 0x7f7b300cae70 2023-10-21T17:38:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2749897925 0 0) 0x7f7b18003520 con 0x7f7b300cae70 2023-10-21T17:38:33.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 msgr2=0x7f7b300adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b300adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 >> v2:172.21.15.124:3300/0 conn(0x7f7b3015ab40 msgr2=0x7f7b300ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 --2- 172.21.15.53:0/3981881386 >> v2:172.21.15.124:3300/0 conn(0x7f7b3015ab40 0x7f7b300ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3015b590 con 0x7f7b300cae70 2023-10-21T17:38:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7b280019f0 con 0x7f7b300cae70 2023-10-21T17:38:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 >> v1:172.21.15.53:6789/0 conn(0x7f7b300cae70 legacy=0x7f7b300702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18003970 0x7f7b18003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b18004d00 0x7f7b18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015bf30 con 0x7f7b18003970 2023-10-21T17:38:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015f7a0 con 0x7f7b18004d00 2023-10-21T17:38:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1800ba30 con 0x7f7b300c90b0 2023-10-21T17:38:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.614+0000 7f7b2effd640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18003970 0x7f7b18003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2e7fc640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2effd640 1 -- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 msgr2=0x7f7b180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2f7fe640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b18004d00 0x7f7b18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2effd640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2effd640 1 -- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b18004d00 msgr2=0x7f7b18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:33.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2effd640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b18004d00 0x7f7b18005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2effd640 1 -- 172.21.15.53:0/3981881386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18003520 con 0x7f7b18003970 2023-10-21T17:38:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.615+0000 7f7b2e7fc640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b300c90b0 0x7f7b180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.616+0000 7f7b2effd640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18003970 0x7f7b18003d90 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000ba60 tx=0x7f7b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24ef1397044eb36 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.616+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20032680 con 0x7f7b18003970 2023-10-21T17:38:33.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.616+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200327e0 con 0x7f7b18003970 2023-10-21T17:38:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.616+0000 7f7b2dffb640 1 -- 172.21.15.53:0/3981881386 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20032b70 con 0x7f7b18003970 2023-10-21T17:38:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 -- 172.21.15.53:0/3981881386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18003970 msgr2=0x7f7b18003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 --2- 172.21.15.53:0/3981881386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b18003970 0x7f7b18003d90 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000ba60 tx=0x7f7b180057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/3981881386 reap_dead start 2023-10-21T17:38:33.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 -- 172.21.15.53:0/3981881386 shutdown_connections 2023-10-21T17:38:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 -- 172.21.15.53:0/3981881386 >> 172.21.15.53:0/3981881386 conn(0x7f7b3006a380 msgr2=0x7f7b300b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 -- 172.21.15.53:0/3981881386 shutdown_connections 2023-10-21T17:38:33.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.617+0000 7f7b35397640 1 -- 172.21.15.53:0/3981881386 wait complete. 2023-10-21T17:38:33.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.619+0000 7f7b35397640 1 Processor -- start 2023-10-21T17:38:33.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.619+0000 7f7b35397640 1 -- start start 2023-10-21T17:38:33.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b18003970 0x7f7b300b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 0x7f7b301f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b46a0 con 0x7f7b301ef9f0 2023-10-21T17:38:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b3b20 con 0x7f7b300c83c0 2023-10-21T17:38:33.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b35397640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300b3e00 con 0x7f7b18003970 2023-10-21T17:38:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50052/0 (socket says 172.21.15.53:50052) 2023-10-21T17:38:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.620+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/4279481305 learned_addr learned my addr 172.21.15.53:0/4279481305 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2e7fc640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 0x7f7b301f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2effd640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b18003970 0x7f7b300b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b18003970 msgr2=0x7f7b300b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b18003970 0x7f7b300b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 msgr2=0x7f7b301f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 0x7f7b301f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 -- 172.21.15.53:0/4279481305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300b40e0 con 0x7f7b301ef9f0 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2e7fc640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 0x7f7b301f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b2f7fe640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006e3f0 tx=0x7f7b2800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004d60 con 0x7f7b301ef9f0 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2800bb80 con 0x7f7b301ef9f0 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28007110 con 0x7f7b301ef9f0 2023-10-21T17:38:33.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b300b43c0 con 0x7f7b301ef9f0 2023-10-21T17:38:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.621+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b300b19c0 con 0x7f7b301ef9f0 2023-10-21T17:38:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.622+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b30067490 con 0x7f7b301ef9f0 2023-10-21T17:38:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.624+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7b28007270 con 0x7f7b301ef9f0 2023-10-21T17:38:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.624+0000 7f7b177fe640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 0x7f7b1c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:33.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.625+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7b28089d00 con 0x7f7b301ef9f0 2023-10-21T17:38:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.625+0000 7f7b2effd640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 0x7f7b1c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.626+0000 7f7b2effd640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 0x7f7b1c06d4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000ba30 tx=0x7f7b20030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:33.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.629+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b2808d030 con 0x7f7b301ef9f0 2023-10-21T17:38:33.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.826+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b30150c20 con 0x7f7b301ef9f0 2023-10-21T17:38:33.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.829+0000 7f7b177fe640 1 -- 172.21.15.53:0/4279481305 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f7b28054b40 con 0x7f7b301ef9f0 2023-10-21T17:38:33.831 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:33.831 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":232,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:32.859935+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:33.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 msgr2=0x7f7b1c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 0x7f7b1c06d4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000ba30 tx=0x7f7b20030000 comp rx=0 tx=0).stop 2023-10-21T17:38:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 msgr2=0x7f7b301efe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006e3f0 tx=0x7f7b2800b4a0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 shutdown_connections 2023-10-21T17:38:33.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b301ef9f0 0x7f7b301efe70 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b300c83c0 0x7f7b301f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7b1c06afe0 0x7f7b1c06d4a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 --2- 172.21.15.53:0/4279481305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b18003970 0x7f7b300b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 >> 172.21.15.53:0/4279481305 conn(0x7f7b3006a380 msgr2=0x7f7b300af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:33.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 shutdown_connections 2023-10-21T17:38:33.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:33.832+0000 7f7b35397640 1 -- 172.21.15.53:0/4279481305 wait complete. 2023-10-21T17:38:33.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:34.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.073+0000 7fbf1e12e640 1 Processor -- start 2023-10-21T17:38:34.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.074+0000 7fbf1e12e640 1 -- start start 2023-10-21T17:38:34.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.075+0000 7fbf1e12e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf1815cc60 0x7fbf180cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1e12e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1815d040 con 0x7fbf180ce6e0 2023-10-21T17:38:34.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1e12e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180d0650 con 0x7fbf180d16f0 2023-10-21T17:38:34.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1e12e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180d0930 con 0x7fbf1815cc60 2023-10-21T17:38:34.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf1815cc60 0x7fbf180cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf1815cc60 0x7fbf180cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49640/0 (socket says 172.21.15.53:49640) 2023-10-21T17:38:34.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 -- 172.21.15.53:0/2898644464 learned_addr learned my addr 172.21.15.53:0/2898644464 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:34.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 -- 172.21.15.53:0/2898644464 >> v1:172.21.15.124:6789/0 conn(0x7fbf180d16f0 legacy=0x7fbf180cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:34.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 -- 172.21.15.53:0/2898644464 >> v1:172.21.15.53:6789/0 conn(0x7fbf180ce6e0 legacy=0x7fbf18166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.076+0000 7fbf1d12c640 1 -- 172.21.15.53:0/2898644464 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180d0c10 con 0x7fbf1815cc60 2023-10-21T17:38:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.077+0000 7fbf1d12c640 1 --2- 172.21.15.53:0/2898644464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf180cf340 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009850 tx=0x7fbf1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fedcc630a5ce88 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.078+0000 7fbf0ffff640 1 -- 172.21.15.53:0/2898644464 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14004230 con 0x7fbf1815cc60 2023-10-21T17:38:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.078+0000 7fbf0ffff640 1 -- 172.21.15.53:0/2898644464 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14004390 con 0x7fbf1815cc60 2023-10-21T17:38:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.078+0000 7fbf0ffff640 1 -- 172.21.15.53:0/2898644464 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14030840 con 0x7fbf1815cc60 2023-10-21T17:38:34.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.078+0000 7fbf1e12e640 1 -- 172.21.15.53:0/2898644464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 msgr2=0x7fbf180cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.078+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/2898644464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf180cf340 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009850 tx=0x7fbf1402b980 comp rx=0 tx=0).stop 2023-10-21T17:38:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.079+0000 7fbf1e12e640 1 -- 172.21.15.53:0/2898644464 shutdown_connections 2023-10-21T17:38:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.079+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/2898644464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf180cf340 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.079+0000 7fbf1e12e640 1 -- 172.21.15.53:0/2898644464 >> 172.21.15.53:0/2898644464 conn(0x7fbf1809bcf0 msgr2=0x7fbf180d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:34.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.079+0000 7fbf1e12e640 1 -- 172.21.15.53:0/2898644464 shutdown_connections 2023-10-21T17:38:34.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.079+0000 7fbf1e12e640 1 -- 172.21.15.53:0/2898644464 wait complete. 2023-10-21T17:38:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.080+0000 7fbf1e12e640 1 Processor -- start 2023-10-21T17:38:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.081+0000 7fbf1e12e640 1 -- start start 2023-10-21T17:38:34.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf180ce6e0 0x7fbf1815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf180d16f0 0x7fbf181f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181467e0 con 0x7fbf1815cc60 2023-10-21T17:38:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1815aad0 con 0x7fbf180d16f0 2023-10-21T17:38:34.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1e12e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181458e0 con 0x7fbf180ce6e0 2023-10-21T17:38:34.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1d92d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1d92d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49646/0 (socket says 172.21.15.53:49646) 2023-10-21T17:38:34.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1d92d640 1 -- 172.21.15.53:0/3422049932 learned_addr learned my addr 172.21.15.53:0/3422049932 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:34.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1d12c640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf180ce6e0 0x7fbf1815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.082+0000 7fbf1c92b640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf180d16f0 0x7fbf181f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 -- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf180ce6e0 msgr2=0x7fbf1815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf180ce6e0 0x7fbf1815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 -- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf180d16f0 msgr2=0x7fbf181f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf180d16f0 0x7fbf181f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 -- 172.21.15.53:0/3422049932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009460 con 0x7fbf1815cc60 2023-10-21T17:38:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1d92d640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c7a0 tx=0x7fbf1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003b70 con 0x7fbf1815cc60 2023-10-21T17:38:34.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf10003cf0 con 0x7fbf1815cc60 2023-10-21T17:38:34.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003ea0 con 0x7fbf1815cc60 2023-10-21T17:38:34.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf18145b60 con 0x7fbf1815cc60 2023-10-21T17:38:34.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.083+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf18146120 con 0x7fbf1815cc60 2023-10-21T17:38:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.085+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1804e160 con 0x7fbf1815cc60 2023-10-21T17:38:34.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.085+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf1000aa90 con 0x7fbf1815cc60 2023-10-21T17:38:34.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.086+0000 7fbf0dffb640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 0x7fbefc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.086+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fbf100897c0 con 0x7fbf1815cc60 2023-10-21T17:38:34.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.087+0000 7fbf1d12c640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 0x7fbefc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.088+0000 7fbf1d12c640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 0x7fbefc06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf14035000 tx=0x7fbf14008030 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.091+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf10054590 con 0x7fbf1815cc60 2023-10-21T17:38:34.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.278+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf180cfc70 con 0x7fbf1815cc60 2023-10-21T17:38:34.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.280+0000 7fbf0dffb640 1 -- 172.21.15.53:0/3422049932 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+16971 (secure 0 0 0) 0x7fbf10057bb0 con 0x7fbf1815cc60 2023-10-21T17:38:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:34.282 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":232,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:32.859935+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:34.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 msgr2=0x7fbefc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 0x7fbefc06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf14035000 tx=0x7fbf14008030 comp rx=0 tx=0).stop 2023-10-21T17:38:34.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 msgr2=0x7fbf1815f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c7a0 tx=0x7fbf1000cc60 comp rx=0 tx=0).stop 2023-10-21T17:38:34.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 shutdown_connections 2023-10-21T17:38:34.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf1815cc60 0x7fbf1815f750 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf180d16f0 0x7fbf181f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbefc06af90 0x7fbefc06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 --2- 172.21.15.53:0/3422049932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf180ce6e0 0x7fbf1815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 >> 172.21.15.53:0/3422049932 conn(0x7fbf1809bcf0 msgr2=0x7fbf180b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:34.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 shutdown_connections 2023-10-21T17:38:34.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.283+0000 7fbf1e12e640 1 -- 172.21.15.53:0/3422049932 wait complete. 2023-10-21T17:38:34.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:38:34.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.517+0000 7f7669490640 1 Processor -- start 2023-10-21T17:38:34.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.517+0000 7f7669490640 1 -- start start 2023-10-21T17:38:34.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.518+0000 7f7669490640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76640cb600 0x7f76640b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.518+0000 7f7669490640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766406e1e0 con 0x7f76640cb150 2023-10-21T17:38:34.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.518+0000 7f7669490640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76640cba50 con 0x7f7664161690 2023-10-21T17:38:34.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.518+0000 7f7669490640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766406cc50 con 0x7f76640cb600 2023-10-21T17:38:34.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.519+0000 7f76627fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f76640cb150 0x7f76640b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46628/0 (socket says 172.21.15.53:46628) 2023-10-21T17:38:34.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.519+0000 7f76627fc640 1 -- 172.21.15.53:0/11604913 learned_addr learned my addr 172.21.15.53:0/11604913 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:34.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.519+0000 7f76637fe640 1 --2- 172.21.15.53:0/11604913 >> v2:172.21.15.124:3300/0 conn(0x7f76640cb600 0x7f76640b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.519+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884564076 0 0) 0x7f766406e1e0 con 0x7f76640cb150 2023-10-21T17:38:34.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7650003520 con 0x7f76640cb150 2023-10-21T17:38:34.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648204780 0 0) 0x7f7650003520 con 0x7f76640cb150 2023-10-21T17:38:34.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 >> v2:172.21.15.124:3300/0 conn(0x7f76640cb600 msgr2=0x7f76640b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:34.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 --2- 172.21.15.53:0/11604913 >> v2:172.21.15.124:3300/0 conn(0x7f76640cb600 0x7f76640b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 >> v1:172.21.15.124:6789/0 conn(0x7f7664161690 legacy=0x7f766406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766406cf30 con 0x7f76640cb150 2023-10-21T17:38:34.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f764c001690 con 0x7f76640cb150 2023-10-21T17:38:34.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 >> v1:172.21.15.53:6789/0 conn(0x7f76640cb150 legacy=0x7f76640b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7650003970 0x7f7650003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f7650004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7650005180 0x7f765000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766406cc50 con 0x7f76500047c0 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765000c020 con 0x7f7650005180 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765000c1e0 con 0x7f7650003970 2023-10-21T17:38:34.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.520+0000 7f76627fc640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f7650004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76637fe640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7650005180 0x7f765000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76627fc640 1 -- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7650003970 msgr2=0x7f7650003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:34.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76627fc640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7650003970 0x7f7650003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76627fc640 1 -- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7650005180 msgr2=0x7f765000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:34.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76627fc640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7650005180 0x7f765000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76627fc640 1 -- 172.21.15.53:0/11604913 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650003520 con 0x7f76500047c0 2023-10-21T17:38:34.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.521+0000 7f76637fe640 1 -- 172.21.15.53:0/11604913 reap_dead start 2023-10-21T17:38:34.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.522+0000 7f76627fc640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f7650004c40 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f766406e370 tx=0x7f76500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3c33dbe3f2fb58 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.522+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c005ad0 con 0x7f76500047c0 2023-10-21T17:38:34.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.522+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c005c30 con 0x7f76500047c0 2023-10-21T17:38:34.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.522+0000 7f7661ffb640 1 -- 172.21.15.53:0/11604913 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c02cab0 con 0x7f76500047c0 2023-10-21T17:38:34.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 -- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 msgr2=0x7f7650004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f7650004c40 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f766406e370 tx=0x7f76500057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 -- 172.21.15.53:0/11604913 shutdown_connections 2023-10-21T17:38:34.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 --2- 172.21.15.53:0/11604913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f7650004c40 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 -- 172.21.15.53:0/11604913 >> 172.21.15.53:0/11604913 conn(0x7f766406a5d0 msgr2=0x7f76640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:34.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 -- 172.21.15.53:0/11604913 shutdown_connections 2023-10-21T17:38:34.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.523+0000 7f7669490640 1 -- 172.21.15.53:0/11604913 wait complete. 2023-10-21T17:38:34.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.525+0000 7f7669490640 1 Processor -- start 2023-10-21T17:38:34.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.525+0000 7f7669490640 1 -- start start 2023-10-21T17:38:34.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 0x7f76641f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7664161ef0 0x7f76641fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640c7f90 con 0x7f76500047c0 2023-10-21T17:38:34.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76640c70d0 con 0x7f7664161a70 2023-10-21T17:38:34.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.526+0000 7f7669490640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76641f66f0 con 0x7f7664161ef0 2023-10-21T17:38:34.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49666/0 (socket says 172.21.15.53:49666) 2023-10-21T17:38:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 -- 172.21.15.53:0/1010084199 learned_addr learned my addr 172.21.15.53:0/1010084199 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f76637fe640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7664161ef0 0x7f76641fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f76627fc640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 0x7f76641f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 -- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7664161ef0 msgr2=0x7f76641fa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7664161ef0 0x7f76641fa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 -- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 msgr2=0x7f76641f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 0x7f76641f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f7662ffd640 1 -- 172.21.15.53:0/1010084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640c7310 con 0x7f76500047c0 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.527+0000 7f76627fc640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 0x7f76641f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f7662ffd640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f765400c710 tx=0x7f765400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654003b70 con 0x7f76500047c0 2023-10-21T17:38:34.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7654003cd0 con 0x7f76500047c0 2023-10-21T17:38:34.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7654005310 con 0x7f76500047c0 2023-10-21T17:38:34.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76640c75f0 con 0x7f76500047c0 2023-10-21T17:38:34.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.528+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76640c7bb0 con 0x7f76500047c0 2023-10-21T17:38:34.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.529+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766404e160 con 0x7f76500047c0 2023-10-21T17:38:34.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.530+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7654005470 con 0x7f76500047c0 2023-10-21T17:38:34.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.532+0000 7f765b7fe640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 0x7f764406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:34.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.532+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7654089670 con 0x7f76500047c0 2023-10-21T17:38:34.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.532+0000 7f76627fc640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 0x7f764406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.534+0000 7f76627fc640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 0x7f764406d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f764c034000 tx=0x7f764c009030 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:34.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.536+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7654054450 con 0x7f76500047c0 2023-10-21T17:38:34.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:34.775+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f76640c8a50 con 0x7f76500047c0 2023-10-21T17:38:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.429+0000 7f765b7fe640 1 -- 172.21.15.53:0/1010084199 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v112) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7654057a70 con 0x7f76500047c0 2023-10-21T17:38:35.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 msgr2=0x7f764406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 0x7f764406d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f764c034000 tx=0x7f764c009030 comp rx=0 tx=0).stop 2023-10-21T17:38:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 msgr2=0x7f76641f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f765400c710 tx=0x7f765400cbd0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 shutdown_connections 2023-10-21T17:38:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f764406afc0 0x7f764406d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7664161ef0 0x7f76641fa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7664161a70 0x7f76641f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 --2- 172.21.15.53:0/1010084199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76500047c0 0x7f76641f61b0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 >> 172.21.15.53:0/1010084199 conn(0x7f766406a5d0 msgr2=0x7f76640b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 shutdown_connections 2023-10-21T17:38:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.434+0000 7f7669490640 1 -- 172.21.15.53:0/1010084199 wait complete. 2023-10-21T17:38:35.458 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:38:35.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:38:35.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.694+0000 7f62abfff640 1 Processor -- start 2023-10-21T17:38:35.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.695+0000 7f62abfff640 1 -- start start 2023-10-21T17:38:35.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.696+0000 7f62abfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f62ac06ec00 0x7f62ac071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.696+0000 7f62abfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62ac06a3a0 0x7f62ac160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.696+0000 7f62abfff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac160d10 con 0x7f62ac069fc0 2023-10-21T17:38:35.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.696+0000 7f62abfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac072070 con 0x7f62ac06a3a0 2023-10-21T17:38:35.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.696+0000 7f62abfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac071580 con 0x7f62ac06ec00 2023-10-21T17:38:35.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62ac06a3a0 0x7f62ac160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:35.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62ac06a3a0 0x7f62ac160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49682/0 (socket says 172.21.15.53:49682) 2023-10-21T17:38:35.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 -- 172.21.15.53:0/2293625983 learned_addr learned my addr 172.21.15.53:0/2293625983 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:35.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 -- 172.21.15.53:0/2293625983 >> v2:172.21.15.124:3300/0 conn(0x7f62ac06ec00 msgr2=0x7f62ac071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:35.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 --2- 172.21.15.53:0/2293625983 >> v2:172.21.15.124:3300/0 conn(0x7f62ac06ec00 0x7f62ac071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 -- 172.21.15.53:0/2293625983 >> v1:172.21.15.124:6789/0 conn(0x7f62ac069fc0 legacy=0x7f62ac0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:35.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.697+0000 7f62ab7fe640 1 -- 172.21.15.53:0/2293625983 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac072330 con 0x7f62ac06a3a0 2023-10-21T17:38:35.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.698+0000 7f62ab7fe640 1 --2- 172.21.15.53:0/2293625983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac06a3a0 0x7f62ac160080 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0cd250 tx=0x7f62a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319e37454e0540c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:35.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62a9ffb640 1 -- 172.21.15.53:0/2293625983 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4031870 con 0x7f62ac06a3a0 2023-10-21T17:38:35.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62a9ffb640 1 -- 172.21.15.53:0/2293625983 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40319d0 con 0x7f62ac06a3a0 2023-10-21T17:38:35.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62a9ffb640 1 -- 172.21.15.53:0/2293625983 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4031d60 con 0x7f62ac06a3a0 2023-10-21T17:38:35.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 -- 172.21.15.53:0/2293625983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac06a3a0 msgr2=0x7f62ac160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:35.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 --2- 172.21.15.53:0/2293625983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac06a3a0 0x7f62ac160080 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0cd250 tx=0x7f62a402b910 comp rx=0 tx=0).stop 2023-10-21T17:38:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 -- 172.21.15.53:0/2293625983 shutdown_connections 2023-10-21T17:38:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 --2- 172.21.15.53:0/2293625983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac06a3a0 0x7f62ac160080 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 --2- 172.21.15.53:0/2293625983 >> v2:172.21.15.124:3300/0 conn(0x7f62ac06ec00 0x7f62ac071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 -- 172.21.15.53:0/2293625983 >> 172.21.15.53:0/2293625983 conn(0x7f62ac0cf5f0 msgr2=0x7f62ac0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.699+0000 7f62abfff640 1 -- 172.21.15.53:0/2293625983 shutdown_connections 2023-10-21T17:38:35.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.700+0000 7f62abfff640 1 -- 172.21.15.53:0/2293625983 wait complete. 2023-10-21T17:38:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.701+0000 7f62abfff640 1 Processor -- start 2023-10-21T17:38:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.701+0000 7f62abfff640 1 -- start start 2023-10-21T17:38:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62ac06a3a0 0x7f62ac0c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62ac06ec00 0x7f62ac15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0c7560 con 0x7f62ac069fc0 2023-10-21T17:38:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0b0340 con 0x7f62ac06ec00 2023-10-21T17:38:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.702+0000 7f62abfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0b0620 con 0x7f62ac06a3a0 2023-10-21T17:38:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:35.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49696/0 (socket says 172.21.15.53:49696) 2023-10-21T17:38:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 -- 172.21.15.53:0/2938625561 learned_addr learned my addr 172.21.15.53:0/2938625561 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 -- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62ac06a3a0 msgr2=0x7f62ac0c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62ac06a3a0 0x7f62ac0c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 -- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62ac06ec00 msgr2=0x7f62ac15a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62ab7fe640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62ac06ec00 0x7f62ac15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62ac06ec00 0x7f62ac15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 -- 172.21.15.53:0/2938625561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a40094d0 con 0x7f62ac069fc0 2023-10-21T17:38:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.703+0000 7f62aaffd640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ba50 tx=0x7f629c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:35.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.704+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00cc10 con 0x7f62ac069fc0 2023-10-21T17:38:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.704+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00cd70 con 0x7f62ac069fc0 2023-10-21T17:38:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.704+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c014680 con 0x7f62ac069fc0 2023-10-21T17:38:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.704+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac0b0900 con 0x7f62ac069fc0 2023-10-21T17:38:35.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.704+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac15adb0 con 0x7f62ac069fc0 2023-10-21T17:38:35.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.705+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac0b0480 con 0x7f62ac069fc0 2023-10-21T17:38:35.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.706+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f629c01d3b0 con 0x7f62ac069fc0 2023-10-21T17:38:35.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.707+0000 7f62a3fff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 0x7f628c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:35.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.707+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f629c089990 con 0x7f62ac069fc0 2023-10-21T17:38:35.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.708+0000 7f62aa7fc640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 0x7f628c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:35.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.709+0000 7f62aa7fc640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 0x7f628c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0af3b0 tx=0x7f6294006c90 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:35.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.711+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62ac0b0480 con 0x7f62ac069fc0 2023-10-21T17:38:35.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:35.942+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f62ac0b0e20 con 0x7f62ac069fc0 2023-10-21T17:38:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.440+0000 7f62a3fff640 1 -- 172.21.15.53:0/2938625561 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v113) v1 ==== 122+0+0 (secure 0 0 0) 0x7f62ac0b0e20 con 0x7f62ac069fc0 2023-10-21T17:38:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 msgr2=0x7f628c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 0x7f628c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0af3b0 tx=0x7f6294006c90 comp rx=0 tx=0).stop 2023-10-21T17:38:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 msgr2=0x7f62ac0c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ba50 tx=0x7f629c00bf10 comp rx=0 tx=0).stop 2023-10-21T17:38:36.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 shutdown_connections 2023-10-21T17:38:36.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62ac06ec00 0x7f62ac15a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628c06af90 0x7f628c06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62ac06a3a0 0x7f62ac0c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 --2- 172.21.15.53:0/2938625561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62ac069fc0 0x7f62ac0c9010 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 >> 172.21.15.53:0/2938625561 conn(0x7f62ac0cf5f0 msgr2=0x7f62ac1605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:36.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.444+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 shutdown_connections 2023-10-21T17:38:36.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.445+0000 7f62abfff640 1 -- 172.21.15.53:0/2938625561 wait complete. 2023-10-21T17:38:36.471 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:38:36.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:36.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.702+0000 7feae3a39640 1 Processor -- start 2023-10-21T17:38:36.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.702+0000 7feae3a39640 1 -- start start 2023-10-21T17:38:36.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.703+0000 7feae3a39640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.703+0000 7feae3a39640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feadc0cbce0 0x7feadc160e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae3a39640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0ce550 con 0x7feadc0cb880 2023-10-21T17:38:36.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae3a39640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc1609d0 con 0x7feadc0cbce0 2023-10-21T17:38:36.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae3a39640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0cdcf0 con 0x7feadc069d50 2023-10-21T17:38:36.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae17d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae17d5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:46098/0 (socket says 172.21.15.53:46098) 2023-10-21T17:38:36.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.704+0000 7feae17d5640 1 -- 172.21.15.53:0/1061894879 learned_addr learned my addr 172.21.15.53:0/1061894879 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:36.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.705+0000 7feae1fd6640 1 --2- 172.21.15.53:0/1061894879 >> v2:172.21.15.124:3300/0 conn(0x7feadc0cbce0 0x7feadc160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.706+0000 7feae1fd6640 1 -- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 msgr2=0x7feadc06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:36.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.706+0000 7feae1fd6640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.706+0000 7feae1fd6640 1 -- 172.21.15.53:0/1061894879 >> v1:172.21.15.124:6789/0 conn(0x7feadc0cb880 legacy=0x7feadc0b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.706+0000 7feae1fd6640 1 -- 172.21.15.53:0/1061894879 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc06a670 con 0x7feadc0cbce0 2023-10-21T17:38:36.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.706+0000 7feae17d5640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:38:36.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.707+0000 7feae1fd6640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc160e60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fead40097a0 tx=0x7fead402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9532699becd13de server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.707+0000 7feadbfff640 1 -- 172.21.15.53:0/1061894879 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead402d040 con 0x7feadc0cbce0 2023-10-21T17:38:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feadbfff640 1 -- 172.21.15.53:0/1061894879 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4031810 con 0x7feadc0cbce0 2023-10-21T17:38:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feadbfff640 1 -- 172.21.15.53:0/1061894879 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4031ba0 con 0x7feadc0cbce0 2023-10-21T17:38:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 -- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 msgr2=0x7feadc160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc160e60 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fead40097a0 tx=0x7fead402b8b0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 -- 172.21.15.53:0/1061894879 shutdown_connections 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc160e60 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 --2- 172.21.15.53:0/1061894879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 -- 172.21.15.53:0/1061894879 >> 172.21.15.53:0/1061894879 conn(0x7feadc06d450 msgr2=0x7feadc06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 -- 172.21.15.53:0/1061894879 shutdown_connections 2023-10-21T17:38:36.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.708+0000 7feae3a39640 1 -- 172.21.15.53:0/1061894879 wait complete. 2023-10-21T17:38:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.710+0000 7feae3a39640 1 Processor -- start 2023-10-21T17:38:36.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.710+0000 7feae3a39640 1 -- start start 2023-10-21T17:38:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc155800 con 0x7feadc0cb880 2023-10-21T17:38:36.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc155520 con 0x7feadc0cbce0 2023-10-21T17:38:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.711+0000 7feae3a39640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15e5c0 con 0x7feadc069d50 2023-10-21T17:38:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49702/0 (socket says 172.21.15.53:49702) 2023-10-21T17:38:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 -- 172.21.15.53:0/4102458893 learned_addr learned my addr 172.21.15.53:0/4102458893 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:36.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae17d5640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae1fd6640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 -- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 msgr2=0x7feadc151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 -- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 msgr2=0x7feadc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 -- 172.21.15.53:0/4102458893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0009b30 con 0x7feadc0cb880 2023-10-21T17:38:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.712+0000 7feae0fd4640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7feac800c400 tx=0x7feac800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.713+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80052d0 con 0x7feadc0cb880 2023-10-21T17:38:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.713+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac8005430 con 0x7feadc0cb880 2023-10-21T17:38:36.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.713+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac8003830 con 0x7feadc0cb880 2023-10-21T17:38:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.713+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead4009400 con 0x7feadc0cb880 2023-10-21T17:38:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.713+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc152400 con 0x7feadc0cb880 2023-10-21T17:38:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.714+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc15edc0 con 0x7feadc0cb880 2023-10-21T17:38:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.715+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feac8003990 con 0x7feadc0cb880 2023-10-21T17:38:36.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.716+0000 7fead9ffb640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 0x7feac006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:36.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.716+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feac8088e40 con 0x7feadc0cb880 2023-10-21T17:38:36.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.717+0000 7feae17d5640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 0x7feac006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:36.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.718+0000 7feae17d5640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 0x7feac006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feadc15af10 tx=0x7fead000b040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:36.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.721+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feadc15edc0 con 0x7feadc0cb880 2023-10-21T17:38:36.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.953+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feadc15edc0 con 0x7feadc0cb880 2023-10-21T17:38:36.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.954+0000 7fead9ffb640 1 -- 172.21.15.53:0/4102458893 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4928 (secure 0 0 0) 0x7feadc15edc0 con 0x7feadc0cb880 2023-10-21T17:38:36.957 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:36.957 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-21T17:38:36.957 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 113 2023-10-21T17:38:36.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 msgr2=0x7feac006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 0x7feac006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feadc15af10 tx=0x7fead000b040 comp rx=0 tx=0).stop 2023-10-21T17:38:36.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 msgr2=0x7feadc15d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:36.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7feac800c400 tx=0x7feac800c8c0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 shutdown_connections 2023-10-21T17:38:36.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feadc0cbce0 0x7feadc15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feadc0cb880 0x7feadc15d410 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7feac006af90 0x7feac006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 --2- 172.21.15.53:0/4102458893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feadc069d50 0x7feadc151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:36.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 >> 172.21.15.53:0/4102458893 conn(0x7feadc06d450 msgr2=0x7feadc0ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:36.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 shutdown_connections 2023-10-21T17:38:36.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:36.957+0000 7feae3a39640 1 -- 172.21.15.53:0/4102458893 wait complete. 2023-10-21T17:38:36.976 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.201+0000 7f5fbd651640 1 Processor -- start 2023-10-21T17:38:37.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.201+0000 7f5fbd651640 1 -- start start 2023-10-21T17:38:37.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.203+0000 7f5fbd651640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fb815ab40 0x7f5fb80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.203+0000 7f5fbd651640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.203+0000 7f5fbd651640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb815b2b0 con 0x7f5fb80cae70 2023-10-21T17:38:37.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.203+0000 7f5fbd651640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb815bf30 con 0x7f5fb815ab40 2023-10-21T17:38:37.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.203+0000 7f5fbd651640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb815f7a0 con 0x7f5fb80c90b0 2023-10-21T17:38:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.204+0000 7f5fb67fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.204+0000 7f5fb6ffd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fb815ab40 0x7f5fb80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.204+0000 7f5fb67fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:46112/0 (socket says 172.21.15.53:46112) 2023-10-21T17:38:37.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.204+0000 7f5fb67fc640 1 -- 172.21.15.53:0/506524740 learned_addr learned my addr 172.21.15.53:0/506524740 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.204+0000 7f5fb77fe640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5fb80cae70 0x7f5fb80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:52536/0 (socket says 172.21.15.53:52536) 2023-10-21T17:38:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.205+0000 7f5fb67fc640 1 -- 172.21.15.53:0/506524740 >> v2:172.21.15.124:3300/0 conn(0x7f5fb815ab40 msgr2=0x7f5fb80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.205+0000 7f5fb67fc640 1 --2- 172.21.15.53:0/506524740 >> v2:172.21.15.124:3300/0 conn(0x7f5fb815ab40 0x7f5fb80ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.205+0000 7f5fb67fc640 1 -- 172.21.15.53:0/506524740 >> v1:172.21.15.124:6789/0 conn(0x7f5fb80cae70 legacy=0x7f5fb80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.205+0000 7f5fb67fc640 1 -- 172.21.15.53:0/506524740 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb815b590 con 0x7f5fb80c90b0 2023-10-21T17:38:37.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.206+0000 7f5fb67fc640 1 --2- 172.21.15.53:0/506524740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009870 tx=0x7f5fac02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa0e4aa6788b91f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.206+0000 7f5fb5ffb640 1 -- 172.21.15.53:0/506524740 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac030840 con 0x7f5fb80c90b0 2023-10-21T17:38:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.206+0000 7f5fb5ffb640 1 -- 172.21.15.53:0/506524740 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac0309a0 con 0x7f5fb80c90b0 2023-10-21T17:38:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.206+0000 7f5fb5ffb640 1 -- 172.21.15.53:0/506524740 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac030d30 con 0x7f5fb80c90b0 2023-10-21T17:38:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 -- 172.21.15.53:0/506524740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 msgr2=0x7f5fb80adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 --2- 172.21.15.53:0/506524740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009870 tx=0x7f5fac02b980 comp rx=0 tx=0).stop 2023-10-21T17:38:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 -- 172.21.15.53:0/506524740 shutdown_connections 2023-10-21T17:38:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 --2- 172.21.15.53:0/506524740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80c90b0 0x7f5fb80adc60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 --2- 172.21.15.53:0/506524740 >> v2:172.21.15.124:3300/0 conn(0x7f5fb815ab40 0x7f5fb80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 -- 172.21.15.53:0/506524740 >> 172.21.15.53:0/506524740 conn(0x7f5fb806a380 msgr2=0x7f5fb80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:37.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 -- 172.21.15.53:0/506524740 shutdown_connections 2023-10-21T17:38:37.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.207+0000 7f5fbd651640 1 -- 172.21.15.53:0/506524740 wait complete. 2023-10-21T17:38:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.209+0000 7f5fbd651640 1 Processor -- start 2023-10-21T17:38:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.209+0000 7f5fbd651640 1 -- start start 2023-10-21T17:38:37.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80cae70 0x7f5fb80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fb815ab40 0x7f5fb80c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb814a930 con 0x7f5fb80c90b0 2023-10-21T17:38:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb80c5680 con 0x7f5fb815ab40 2023-10-21T17:38:37.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fbd651640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb80c4a70 con 0x7f5fb80cae70 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fb6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fb6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49708/0 (socket says 172.21.15.53:49708) 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.210+0000 7f5fb6ffd640 1 -- 172.21.15.53:0/2363605270 learned_addr learned my addr 172.21.15.53:0/2363605270 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 -- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80cae70 msgr2=0x7f5fb80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80cae70 0x7f5fb80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 -- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fb815ab40 msgr2=0x7f5fb80c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fb815ab40 0x7f5fb80c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 -- 172.21.15.53:0/2363605270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0094d0 con 0x7f5fb80c90b0 2023-10-21T17:38:37.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5fb6ffd640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e9e0 tx=0x7f5fa800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8009670 con 0x7f5fb80c90b0 2023-10-21T17:38:37.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa800c9d0 con 0x7f5fb80c90b0 2023-10-21T17:38:37.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.211+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800cd80 con 0x7f5fb80c90b0 2023-10-21T17:38:37.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.212+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb80c4d50 con 0x7f5fb80c90b0 2023-10-21T17:38:37.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.212+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb80c5310 con 0x7f5fb80c90b0 2023-10-21T17:38:37.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.213+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb8067490 con 0x7f5fb80c90b0 2023-10-21T17:38:37.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.214+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5fa8013470 con 0x7f5fb80c90b0 2023-10-21T17:38:37.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.215+0000 7f5f9f7fe640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 0x7f5fa406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.215+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5fa8089af0 con 0x7f5fb80c90b0 2023-10-21T17:38:37.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.215+0000 7f5fb67fc640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 0x7f5fa406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.216+0000 7f5fb67fc640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 0x7f5fa406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fac036000 tx=0x7f5fac03b040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.219+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fa8054960 con 0x7f5fb80c90b0 2023-10-21T17:38:37.352 INFO:tasks.ceph.mds.d.smithi124.stderr:2023-10-21T17:38:37.351+0000 7efef8cb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:38:37.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.407+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fb80c6290 con 0x7f5fb80c90b0 2023-10-21T17:38:37.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.410+0000 7f5f9f7fe640 1 -- 172.21.15.53:0/2363605270 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f5fa8017090 con 0x7f5fb80c90b0 2023-10-21T17:38:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:37.412 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":233,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:34.320176+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.412+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 msgr2=0x7f5fa406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.412+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 0x7f5fa406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5fac036000 tx=0x7f5fac03b040 comp rx=0 tx=0).stop 2023-10-21T17:38:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.412+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 msgr2=0x7f5fb814a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.412+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800e9e0 tx=0x7f5fa800eea0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 shutdown_connections 2023-10-21T17:38:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fb815ab40 0x7f5fb80c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5fa406af90 0x7f5fa406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fb80cae70 0x7f5fb80c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 --2- 172.21.15.53:0/2363605270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fb80c90b0 0x7f5fb814a3f0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 >> 172.21.15.53:0/2363605270 conn(0x7f5fb806a380 msgr2=0x7f5fb80ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 shutdown_connections 2023-10-21T17:38:37.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.413+0000 7f5fbd651640 1 -- 172.21.15.53:0/2363605270 wait complete. 2023-10-21T17:38:37.433 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:38:37.434 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:38:37.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.651+0000 7f2e47501640 1 Processor -- start 2023-10-21T17:38:37.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.652+0000 7f2e47501640 1 -- start start 2023-10-21T17:38:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.653+0000 7f2e47501640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e400cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.653+0000 7f2e47501640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e400cb3b0 0x7f2e4015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.653+0000 7f2e47501640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4015b2b0 con 0x7f2e4015ab40 2023-10-21T17:38:37.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e47501640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4015bf30 con 0x7f2e400cb3b0 2023-10-21T17:38:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e47501640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400cb840 con 0x7f2e400c90b0 2023-10-21T17:38:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e4529d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2e4015ab40 0x7f2e400ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46630/0 (socket says 172.21.15.53:46630) 2023-10-21T17:38:37.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e4529d640 1 -- 172.21.15.53:0/1318049025 learned_addr learned my addr 172.21.15.53:0/1318049025 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e400cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.654+0000 7f2e45a9e640 1 --2- 172.21.15.53:0/1318049025 >> v2:172.21.15.124:3300/0 conn(0x7f2e400cb3b0 0x7f2e4015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130023925 0 0) 0x7f2e4015b2b0 con 0x7f2e4015ab40 2023-10-21T17:38:37.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e2c003520 con 0x7f2e4015ab40 2023-10-21T17:38:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077744813 0 0) 0x7f2e2c003520 con 0x7f2e4015ab40 2023-10-21T17:38:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 msgr2=0x7f2e400cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e400cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 >> v2:172.21.15.124:3300/0 conn(0x7f2e400cb3b0 msgr2=0x7f2e4015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:37.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 --2- 172.21.15.53:0/1318049025 >> v2:172.21.15.124:3300/0 conn(0x7f2e400cb3b0 0x7f2e4015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4015b590 con 0x7f2e4015ab40 2023-10-21T17:38:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2e3c001670 con 0x7f2e4015ab40 2023-10-21T17:38:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 >> v1:172.21.15.53:6789/0 conn(0x7f2e4015ab40 legacy=0x7f2e400ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e2c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e2c003f40 0x7f2e2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e2c004d00 0x7f2e2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4015bf30 con 0x7f2e2c004d00 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400cb840 con 0x7f2e2c003f40 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.655+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c00b980 con 0x7f2e400c90b0 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e2c004d00 0x7f2e2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e2c003f40 0x7f2e2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e4529d640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e2c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 -- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 msgr2=0x7f2e2c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e400c90b0 0x7f2e2c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 -- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e2c003f40 msgr2=0x7f2e2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:37.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e2c003f40 0x7f2e2c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e45a9e640 1 -- 172.21.15.53:0/1318049025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c003520 con 0x7f2e2c004d00 2023-10-21T17:38:37.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.656+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e2c003f40 0x7f2e2c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.657+0000 7f2e45a9e640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e2c004d00 0x7f2e2c005180 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e170 tx=0x7f2e2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e99b46cdfc345bb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3802cc20 con 0x7f2e2c004d00 2023-10-21T17:38:37.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3800f040 con 0x7f2e2c004d00 2023-10-21T17:38:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e37fff640 1 -- 172.21.15.53:0/1318049025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034440 con 0x7f2e2c004d00 2023-10-21T17:38:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e47501640 1 -- 172.21.15.53:0/1318049025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e2c004d00 msgr2=0x7f2e2c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e47501640 1 --2- 172.21.15.53:0/1318049025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e2c004d00 0x7f2e2c005180 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e170 tx=0x7f2e2c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e45a9e640 1 -- 172.21.15.53:0/1318049025 reap_dead start 2023-10-21T17:38:37.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e47501640 1 -- 172.21.15.53:0/1318049025 shutdown_connections 2023-10-21T17:38:37.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e47501640 1 -- 172.21.15.53:0/1318049025 >> 172.21.15.53:0/1318049025 conn(0x7f2e4006a380 msgr2=0x7f2e400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:37.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.658+0000 7f2e47501640 1 -- 172.21.15.53:0/1318049025 shutdown_connections 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.659+0000 7f2e47501640 1 -- 172.21.15.53:0/1318049025 wait complete. 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.660+0000 7f2e47501640 1 Processor -- start 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 -- start start 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e2c004d00 0x7f2e4014f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e40157b50 0x7f2e400ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401510b0 con 0x7f2e400ad480 2023-10-21T17:38:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40150540 con 0x7f2e40157b50 2023-10-21T17:38:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.661+0000 7f2e47501640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c3620 con 0x7f2e2c004d00 2023-10-21T17:38:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49732/0 (socket says 172.21.15.53:49732) 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 -- 172.21.15.53:0/2348665291 learned_addr learned my addr 172.21.15.53:0/2348665291 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 -- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e2c004d00 msgr2=0x7f2e4014f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e2c004d00 0x7f2e4014f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 -- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e40157b50 msgr2=0x7f2e400ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e40157b50 0x7f2e400ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 -- 172.21.15.53:0/2348665291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e40151290 con 0x7f2e400ad480 2023-10-21T17:38:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.662+0000 7f2e44a9c640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f2e300115e0 tx=0x7f2e30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.663+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30015870 con 0x7f2e400ad480 2023-10-21T17:38:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.663+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e300159d0 con 0x7f2e400ad480 2023-10-21T17:38:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.663+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e40151570 con 0x7f2e400ad480 2023-10-21T17:38:37.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.663+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e30015d40 con 0x7f2e400ad480 2023-10-21T17:38:37.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.663+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4014f5c0 con 0x7f2e400ad480 2023-10-21T17:38:37.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.664+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e40067490 con 0x7f2e400ad480 2023-10-21T17:38:37.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.665+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2e3000f460 con 0x7f2e400ad480 2023-10-21T17:38:37.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.666+0000 7f2e35ffb640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 0x7f2e2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:37.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.666+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2e3008a2d0 con 0x7f2e400ad480 2023-10-21T17:38:37.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.667+0000 7f2e4529d640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 0x7f2e2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:37.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.668+0000 7f2e4529d640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 0x7f2e2406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005990 tx=0x7f2e3c000ed0 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:37.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.670+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e30054f80 con 0x7f2e400ad480 2023-10-21T17:38:37.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.864+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e400b4830 con 0x7f2e2406af90 2023-10-21T17:38:37.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.893+0000 7f2e35ffb640 1 -- 172.21.15.53:0/2348665291 <== mgr.37814 v2:172.21.15.124:6832/3930748558 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286467 (secure 0 0 0) 0x7f2e400b4830 con 0x7f2e2406af90 2023-10-21T17:38:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:37.899 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:38:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 msgr2=0x7f2e2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 0x7f2e2406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005990 tx=0x7f2e3c000ed0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 msgr2=0x7f2e400ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f2e300115e0 tx=0x7f2e30011aa0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 shutdown_connections 2023-10-21T17:38:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e40157b50 0x7f2e400ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e400ad480 0x7f2e400ac910 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f2e2406af90 0x7f2e2406d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 --2- 172.21.15.53:0/2348665291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e2c004d00 0x7f2e4014f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 >> 172.21.15.53:0/2348665291 conn(0x7f2e4006a380 msgr2=0x7f2e4006d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:37.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.900+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 shutdown_connections 2023-10-21T17:38:37.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:37.901+0000 7f2e47501640 1 -- 172.21.15.53:0/2348665291 wait complete. 2023-10-21T17:38:37.925 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-21T17:38:37.679960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":631,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4911,"ondisk_log_size":4911,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":434352,"kb_used_data":23856,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754540368,"statfs":{"total":773094113280,"available":772649336832,"internally_reserved":0,"allocated":24428544,"data_stored":13210259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":420204089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001950"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369295+0000","last_change":"2023-10-21T17:38:33.368541+0000","last_active":"2023-10-21T17:38:33.369295+0000","last_peered":"2023-10-21T17:38:33.369295+0000","last_clean":"2023-10-21T17:38:33.369295+0000","last_became_active":"2023-10-21T17:38:33.368186+0000","last_became_peered":"2023-10-21T17:38:33.368186+0000","last_unstale":"2023-10-21T17:38:33.369295+0000","last_undegraded":"2023-10-21T17:38:33.369295+0000","last_fullsized":"2023-10-21T17:38:33.369295+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:29.632092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352509+0000","last_change":"2023-10-21T17:38:33.348939+0000","last_active":"2023-10-21T17:38:33.352509+0000","last_peered":"2023-10-21T17:38:33.352509+0000","last_clean":"2023-10-21T17:38:33.352509+0000","last_became_active":"2023-10-21T17:38:33.348561+0000","last_became_peered":"2023-10-21T17:38:33.348561+0000","last_unstale":"2023-10-21T17:38:33.352509+0000","last_undegraded":"2023-10-21T17:38:33.352509+0000","last_fullsized":"2023-10-21T17:38:33.352509+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:43:26.003956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.360419+0000","last_change":"2023-10-21T17:38:33.354797+0000","last_active":"2023-10-21T17:38:33.360419+0000","last_peered":"2023-10-21T17:38:33.360419+0000","last_clean":"2023-10-21T17:38:33.360419+0000","last_became_active":"2023-10-21T17:38:33.354503+0000","last_became_peered":"2023-10-21T17:38:33.354503+0000","last_unstale":"2023-10-21T17:38:33.360419+0000","last_undegraded":"2023-10-21T17:38:33.360419+0000","last_fullsized":"2023-10-21T17:38:33.360419+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:16.840393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363954+0000","last_change":"2023-10-21T17:38:33.359316+0000","last_active":"2023-10-21T17:38:33.363954+0000","last_peered":"2023-10-21T17:38:33.363954+0000","last_clean":"2023-10-21T17:38:33.363954+0000","last_became_active":"2023-10-21T17:38:33.356311+0000","last_became_peered":"2023-10-21T17:38:33.356311+0000","last_unstale":"2023-10-21T17:38:33.363954+0000","last_undegraded":"2023-10-21T17:38:33.363954+0000","last_fullsized":"2023-10-21T17:38:33.363954+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:31.022494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356957+0000","last_change":"2023-10-21T17:38:33.354002+0000","last_active":"2023-10-21T17:38:33.356957+0000","last_peered":"2023-10-21T17:38:33.356957+0000","last_clean":"2023-10-21T17:38:33.356957+0000","last_became_active":"2023-10-21T17:38:33.353709+0000","last_became_peered":"2023-10-21T17:38:33.353709+0000","last_unstale":"2023-10-21T17:38:33.356957+0000","last_undegraded":"2023-10-21T17:38:33.356957+0000","last_fullsized":"2023-10-21T17:38:33.356957+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:14.017632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.355581+0000","last_change":"2023-10-21T17:38:33.351557+0000","last_active":"2023-10-21T17:38:33.355581+0000","last_peered":"2023-10-21T17:38:33.355581+0000","last_clean":"2023-10-21T17:38:33.355581+0000","last_became_active":"2023-10-21T17:38:33.351230+0000","last_became_peered":"2023-10-21T17:38:33.351230+0000","last_unstale":"2023-10-21T17:38:33.355581+0000","last_undegraded":"2023-10-21T17:38:33.355581+0000","last_fullsized":"2023-10-21T17:38:33.355581+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356671+0000","last_change":"2023-10-21T17:38:33.352800+0000","last_active":"2023-10-21T17:38:33.356671+0000","last_peered":"2023-10-21T17:38:33.356671+0000","last_clean":"2023-10-21T17:38:33.356671+0000","last_became_active":"2023-10-21T17:38:33.351990+0000","last_became_peered":"2023-10-21T17:38:33.351990+0000","last_unstale":"2023-10-21T17:38:33.356671+0000","last_undegraded":"2023-10-21T17:38:33.356671+0000","last_fullsized":"2023-10-21T17:38:33.356671+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:55:23.855999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.367523+0000","last_change":"2023-10-21T17:38:33.363315+0000","last_active":"2023-10-21T17:38:33.367523+0000","last_peered":"2023-10-21T17:38:33.367523+0000","last_clean":"2023-10-21T17:38:33.367523+0000","last_became_active":"2023-10-21T17:38:33.363007+0000","last_became_peered":"2023-10-21T17:38:33.363007+0000","last_unstale":"2023-10-21T17:38:33.367523+0000","last_undegraded":"2023-10-21T17:38:33.367523+0000","last_fullsized":"2023-10-21T17:38:33.367523+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:13.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364051+0000","last_change":"2023-10-21T17:38:33.363542+0000","last_active":"2023-10-21T17:38:33.364051+0000","last_peered":"2023-10-21T17:38:33.364051+0000","last_clean":"2023-10-21T17:38:33.364051+0000","last_became_active":"2023-10-21T17:38:33.363143+0000","last_became_peered":"2023-10-21T17:38:33.363143+0000","last_unstale":"2023-10-21T17:38:33.364051+0000","last_undegraded":"2023-10-21T17:38:33.364051+0000","last_fullsized":"2023-10-21T17:38:33.364051+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:29.955478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369678+0000","last_change":"2023-10-21T17:38:33.368484+0000","last_active":"2023-10-21T17:38:33.369678+0000","last_peered":"2023-10-21T17:38:33.369678+0000","last_clean":"2023-10-21T17:38:33.369678+0000","last_became_active":"2023-10-21T17:38:33.366828+0000","last_became_peered":"2023-10-21T17:38:33.366828+0000","last_unstale":"2023-10-21T17:38:33.369678+0000","last_undegraded":"2023-10-21T17:38:33.369678+0000","last_fullsized":"2023-10-21T17:38:33.369678+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:47:57.779222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370053+0000","last_change":"2023-10-21T17:38:33.369555+0000","last_active":"2023-10-21T17:38:33.370053+0000","last_peered":"2023-10-21T17:38:33.370053+0000","last_clean":"2023-10-21T17:38:33.370053+0000","last_became_active":"2023-10-21T17:38:33.369212+0000","last_became_peered":"2023-10-21T17:38:33.369212+0000","last_unstale":"2023-10-21T17:38:33.370053+0000","last_undegraded":"2023-10-21T17:38:33.370053+0000","last_fullsized":"2023-10-21T17:38:33.370053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:33.952465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357216+0000","last_change":"2023-10-21T17:38:33.349361+0000","last_active":"2023-10-21T17:38:33.357216+0000","last_peered":"2023-10-21T17:38:33.357216+0000","last_clean":"2023-10-21T17:38:33.357216+0000","last_became_active":"2023-10-21T17:38:33.348672+0000","last_became_peered":"2023-10-21T17:38:33.348672+0000","last_unstale":"2023-10-21T17:38:33.357216+0000","last_undegraded":"2023-10-21T17:38:33.357216+0000","last_fullsized":"2023-10-21T17:38:33.357216+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:38:57.231941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363382+0000","last_change":"2023-10-21T17:38:33.358688+0000","last_active":"2023-10-21T17:38:33.363382+0000","last_peered":"2023-10-21T17:38:33.363382+0000","last_clean":"2023-10-21T17:38:33.363382+0000","last_became_active":"2023-10-21T17:38:33.354936+0000","last_became_peered":"2023-10-21T17:38:33.354936+0000","last_unstale":"2023-10-21T17:38:33.363382+0000","last_undegraded":"2023-10-21T17:38:33.363382+0000","last_fullsized":"2023-10-21T17:38:33.363382+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:01.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369293+0000","last_change":"2023-10-21T17:38:33.366456+0000","last_active":"2023-10-21T17:38:33.369293+0000","last_peered":"2023-10-21T17:38:33.369293+0000","last_clean":"2023-10-21T17:38:33.369293+0000","last_became_active":"2023-10-21T17:38:33.366015+0000","last_became_peered":"2023-10-21T17:38:33.366015+0000","last_unstale":"2023-10-21T17:38:33.369293+0000","last_undegraded":"2023-10-21T17:38:33.369293+0000","last_fullsized":"2023-10-21T17:38:33.369293+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:05.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362586+0000","last_change":"2023-10-21T17:38:33.360718+0000","last_active":"2023-10-21T17:38:33.362586+0000","last_peered":"2023-10-21T17:38:33.362586+0000","last_clean":"2023-10-21T17:38:33.362586+0000","last_became_active":"2023-10-21T17:38:33.359530+0000","last_became_peered":"2023-10-21T17:38:33.359530+0000","last_unstale":"2023-10-21T17:38:33.362586+0000","last_undegraded":"2023-10-21T17:38:33.362586+0000","last_fullsized":"2023-10-21T17:38:33.362586+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:28:01.937569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.349500+0000","last_change":"2023-10-21T17:38:33.344462+0000","last_active":"2023-10-21T17:38:33.349500+0000","last_peered":"2023-10-21T17:38:33.349500+0000","last_clean":"2023-10-21T17:38:33.349500+0000","last_became_active":"2023-10-21T17:38:33.344064+0000","last_became_peered":"2023-10-21T17:38:33.344064+0000","last_unstale":"2023-10-21T17:38:33.349500+0000","last_undegraded":"2023-10-21T17:38:33.349500+0000","last_fullsized":"2023-10-21T17:38:33.349500+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:58.893895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361953+0000","last_change":"2023-10-21T17:38:33.360922+0000","last_active":"2023-10-21T17:38:33.361953+0000","last_peered":"2023-10-21T17:38:33.361953+0000","last_clean":"2023-10-21T17:38:33.361953+0000","last_became_active":"2023-10-21T17:38:33.360628+0000","last_became_peered":"2023-10-21T17:38:33.360628+0000","last_unstale":"2023-10-21T17:38:33.361953+0000","last_undegraded":"2023-10-21T17:38:33.361953+0000","last_fullsized":"2023-10-21T17:38:33.361953+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:39.289731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.354841+0000","last_change":"2023-10-21T17:38:33.351005+0000","last_active":"2023-10-21T17:38:33.354841+0000","last_peered":"2023-10-21T17:38:33.354841+0000","last_clean":"2023-10-21T17:38:33.354841+0000","last_became_active":"2023-10-21T17:38:33.350305+0000","last_became_peered":"2023-10-21T17:38:33.350305+0000","last_unstale":"2023-10-21T17:38:33.354841+0000","last_undegraded":"2023-10-21T17:38:33.354841+0000","last_fullsized":"2023-10-21T17:38:33.354841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:33.952465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369428+0000","last_change":"2023-10-21T17:38:33.366914+0000","last_active":"2023-10-21T17:38:33.369428+0000","last_peered":"2023-10-21T17:38:33.369428+0000","last_clean":"2023-10-21T17:38:33.369428+0000","last_became_active":"2023-10-21T17:38:33.366337+0000","last_became_peered":"2023-10-21T17:38:33.366337+0000","last_unstale":"2023-10-21T17:38:33.369428+0000","last_undegraded":"2023-10-21T17:38:33.369428+0000","last_fullsized":"2023-10-21T17:38:33.369428+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:09.263356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.368954+0000","last_change":"2023-10-21T17:38:33.363852+0000","last_active":"2023-10-21T17:38:33.368954+0000","last_peered":"2023-10-21T17:38:33.368954+0000","last_clean":"2023-10-21T17:38:33.368954+0000","last_became_active":"2023-10-21T17:38:33.363427+0000","last_became_peered":"2023-10-21T17:38:33.363427+0000","last_unstale":"2023-10-21T17:38:33.368954+0000","last_undegraded":"2023-10-21T17:38:33.368954+0000","last_fullsized":"2023-10-21T17:38:33.368954+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:28:01.937569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369681+0000","last_change":"2023-10-21T17:38:33.367227+0000","last_active":"2023-10-21T17:38:33.369681+0000","last_peered":"2023-10-21T17:38:33.369681+0000","last_clean":"2023-10-21T17:38:33.369681+0000","last_became_active":"2023-10-21T17:38:33.366922+0000","last_became_peered":"2023-10-21T17:38:33.366922+0000","last_unstale":"2023-10-21T17:38:33.369681+0000","last_undegraded":"2023-10-21T17:38:33.369681+0000","last_fullsized":"2023-10-21T17:38:33.369681+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:37:12.429036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369215+0000","last_change":"2023-10-21T17:38:33.366191+0000","last_active":"2023-10-21T17:38:33.369215+0000","last_peered":"2023-10-21T17:38:33.369215+0000","last_clean":"2023-10-21T17:38:33.369215+0000","last_became_active":"2023-10-21T17:38:33.363694+0000","last_became_peered":"2023-10-21T17:38:33.363694+0000","last_unstale":"2023-10-21T17:38:33.369215+0000","last_undegraded":"2023-10-21T17:38:33.369215+0000","last_fullsized":"2023-10-21T17:38:33.369215+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:40.907600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357130+0000","last_change":"2023-10-21T17:38:33.355761+0000","last_active":"2023-10-21T17:38:33.357130+0000","last_peered":"2023-10-21T17:38:33.357130+0000","last_clean":"2023-10-21T17:38:33.357130+0000","last_became_active":"2023-10-21T17:38:33.355480+0000","last_became_peered":"2023-10-21T17:38:33.355480+0000","last_unstale":"2023-10-21T17:38:33.357130+0000","last_undegraded":"2023-10-21T17:38:33.357130+0000","last_fullsized":"2023-10-21T17:38:33.357130+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:23:08.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352523+0000","last_change":"2023-10-21T17:38:33.346563+0000","last_active":"2023-10-21T17:38:33.352523+0000","last_peered":"2023-10-21T17:38:33.352523+0000","last_clean":"2023-10-21T17:38:33.352523+0000","last_became_active":"2023-10-21T17:38:33.343191+0000","last_became_peered":"2023-10-21T17:38:33.343191+0000","last_unstale":"2023-10-21T17:38:33.352523+0000","last_undegraded":"2023-10-21T17:38:33.352523+0000","last_fullsized":"2023-10-21T17:38:33.352523+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:07.673155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356407+0000","last_change":"2023-10-21T17:38:33.352394+0000","last_active":"2023-10-21T17:38:33.356407+0000","last_peered":"2023-10-21T17:38:33.356407+0000","last_clean":"2023-10-21T17:38:33.356407+0000","last_became_active":"2023-10-21T17:38:33.352048+0000","last_became_peered":"2023-10-21T17:38:33.352048+0000","last_unstale":"2023-10-21T17:38:33.356407+0000","last_undegraded":"2023-10-21T17:38:33.356407+0000","last_fullsized":"2023-10-21T17:38:33.356407+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:37.232213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364739+0000","last_change":"2023-10-21T17:38:33.362478+0000","last_active":"2023-10-21T17:38:33.364739+0000","last_peered":"2023-10-21T17:38:33.364739+0000","last_clean":"2023-10-21T17:38:33.364739+0000","last_became_active":"2023-10-21T17:38:33.361294+0000","last_became_peered":"2023-10-21T17:38:33.361294+0000","last_unstale":"2023-10-21T17:38:33.364739+0000","last_undegraded":"2023-10-21T17:38:33.364739+0000","last_fullsized":"2023-10-21T17:38:33.364739+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370068+0000","last_change":"2023-10-21T17:38:33.369586+0000","last_active":"2023-10-21T17:38:33.370068+0000","last_peered":"2023-10-21T17:38:33.370068+0000","last_clean":"2023-10-21T17:38:33.370068+0000","last_became_active":"2023-10-21T17:38:33.369208+0000","last_became_peered":"2023-10-21T17:38:33.369208+0000","last_unstale":"2023-10-21T17:38:33.370068+0000","last_undegraded":"2023-10-21T17:38:33.370068+0000","last_fullsized":"2023-10-21T17:38:33.370068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:03.956275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357238+0000","last_change":"2023-10-21T17:38:33.349528+0000","last_active":"2023-10-21T17:38:33.357238+0000","last_peered":"2023-10-21T17:38:33.357238+0000","last_clean":"2023-10-21T17:38:33.357238+0000","last_became_active":"2023-10-21T17:38:33.349070+0000","last_became_peered":"2023-10-21T17:38:33.349070+0000","last_unstale":"2023-10-21T17:38:33.357238+0000","last_undegraded":"2023-10-21T17:38:33.357238+0000","last_fullsized":"2023-10-21T17:38:33.357238+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:49:59.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359991+0000","last_change":"2023-10-21T17:38:33.354704+0000","last_active":"2023-10-21T17:38:33.359991+0000","last_peered":"2023-10-21T17:38:33.359991+0000","last_clean":"2023-10-21T17:38:33.359991+0000","last_became_active":"2023-10-21T17:38:33.354428+0000","last_became_peered":"2023-10-21T17:38:33.354428+0000","last_unstale":"2023-10-21T17:38:33.359991+0000","last_undegraded":"2023-10-21T17:38:33.359991+0000","last_fullsized":"2023-10-21T17:38:33.359991+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:17.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.346497+0000","last_change":"2023-10-21T17:38:33.340905+0000","last_active":"2023-10-21T17:38:33.346497+0000","last_peered":"2023-10-21T17:38:33.346497+0000","last_clean":"2023-10-21T17:38:33.346497+0000","last_became_active":"2023-10-21T17:38:33.340134+0000","last_became_peered":"2023-10-21T17:38:33.340134+0000","last_unstale":"2023-10-21T17:38:33.346497+0000","last_undegraded":"2023-10-21T17:38:33.346497+0000","last_fullsized":"2023-10-21T17:38:33.346497+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:40.907600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.355791+0000","last_change":"2023-10-21T17:38:33.353725+0000","last_active":"2023-10-21T17:38:33.355791+0000","last_peered":"2023-10-21T17:38:33.355791+0000","last_clean":"2023-10-21T17:38:33.355791+0000","last_became_active":"2023-10-21T17:38:33.353441+0000","last_became_peered":"2023-10-21T17:38:33.353441+0000","last_unstale":"2023-10-21T17:38:33.355791+0000","last_undegraded":"2023-10-21T17:38:33.355791+0000","last_fullsized":"2023-10-21T17:38:33.355791+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:30.342934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357356+0000","last_change":"2023-10-21T17:38:33.354963+0000","last_active":"2023-10-21T17:38:33.357356+0000","last_peered":"2023-10-21T17:38:33.357356+0000","last_clean":"2023-10-21T17:38:33.357356+0000","last_became_active":"2023-10-21T17:38:33.354183+0000","last_became_peered":"2023-10-21T17:38:33.354183+0000","last_unstale":"2023-10-21T17:38:33.357356+0000","last_undegraded":"2023-10-21T17:38:33.357356+0000","last_fullsized":"2023-10-21T17:38:33.357356+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:54.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.353860+0000","last_change":"2023-10-21T17:38:33.350258+0000","last_active":"2023-10-21T17:38:33.353860+0000","last_peered":"2023-10-21T17:38:33.353860+0000","last_clean":"2023-10-21T17:38:33.353860+0000","last_became_active":"2023-10-21T17:38:33.349849+0000","last_became_peered":"2023-10-21T17:38:33.349849+0000","last_unstale":"2023-10-21T17:38:33.353860+0000","last_undegraded":"2023-10-21T17:38:33.353860+0000","last_fullsized":"2023-10-21T17:38:33.353860+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:07.548870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"232'4","reported_seq":15,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.416255+0000","last_change":"2023-10-21T17:38:32.419554+0000","last_active":"2023-10-21T17:38:33.416255+0000","last_peered":"2023-10-21T17:38:33.416255+0000","last_clean":"2023-10-21T17:38:33.416255+0000","last_became_active":"2023-10-21T17:38:32.419276+0000","last_became_peered":"2023-10-21T17:38:32.419276+0000","last_unstale":"2023-10-21T17:38:33.416255+0000","last_undegraded":"2023-10-21T17:38:33.416255+0000","last_fullsized":"2023-10-21T17:38:33.416255+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:44.216819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359685+0000","last_change":"2023-10-21T17:38:33.352486+0000","last_active":"2023-10-21T17:38:33.359685+0000","last_peered":"2023-10-21T17:38:33.359685+0000","last_clean":"2023-10-21T17:38:33.359685+0000","last_became_active":"2023-10-21T17:38:33.352150+0000","last_became_peered":"2023-10-21T17:38:33.352150+0000","last_unstale":"2023-10-21T17:38:33.359685+0000","last_undegraded":"2023-10-21T17:38:33.359685+0000","last_fullsized":"2023-10-21T17:38:33.359685+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:31:32.268023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412028+0000","last_change":"2023-10-21T17:38:32.401420+0000","last_active":"2023-10-21T17:38:33.412028+0000","last_peered":"2023-10-21T17:38:33.412028+0000","last_clean":"2023-10-21T17:38:33.412028+0000","last_became_active":"2023-10-21T17:38:32.394692+0000","last_became_peered":"2023-10-21T17:38:32.394692+0000","last_unstale":"2023-10-21T17:38:33.412028+0000","last_undegraded":"2023-10-21T17:38:33.412028+0000","last_fullsized":"2023-10-21T17:38:33.412028+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363087+0000","last_change":"2023-10-21T17:38:33.360687+0000","last_active":"2023-10-21T17:38:33.363087+0000","last_peered":"2023-10-21T17:38:33.363087+0000","last_clean":"2023-10-21T17:38:33.363087+0000","last_became_active":"2023-10-21T17:38:33.360369+0000","last_became_peered":"2023-10-21T17:38:33.360369+0000","last_unstale":"2023-10-21T17:38:33.363087+0000","last_undegraded":"2023-10-21T17:38:33.363087+0000","last_fullsized":"2023-10-21T17:38:33.363087+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:36.794962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414885+0000","last_change":"2023-10-21T17:38:32.405671+0000","last_active":"2023-10-21T17:38:33.414885+0000","last_peered":"2023-10-21T17:38:33.414885+0000","last_clean":"2023-10-21T17:38:33.414885+0000","last_became_active":"2023-10-21T17:38:32.404175+0000","last_became_peered":"2023-10-21T17:38:32.404175+0000","last_unstale":"2023-10-21T17:38:33.414885+0000","last_undegraded":"2023-10-21T17:38:33.414885+0000","last_fullsized":"2023-10-21T17:38:33.414885+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:24.818705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356938+0000","last_change":"2023-10-21T17:38:33.350688+0000","last_active":"2023-10-21T17:38:33.356938+0000","last_peered":"2023-10-21T17:38:33.356938+0000","last_clean":"2023-10-21T17:38:33.356938+0000","last_became_active":"2023-10-21T17:38:33.350383+0000","last_became_peered":"2023-10-21T17:38:33.350383+0000","last_unstale":"2023-10-21T17:38:33.356938+0000","last_undegraded":"2023-10-21T17:38:33.356938+0000","last_fullsized":"2023-10-21T17:38:33.356938+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:30.144459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412423+0000","last_change":"2023-10-21T17:38:32.407056+0000","last_active":"2023-10-21T17:38:33.412423+0000","last_peered":"2023-10-21T17:38:33.412423+0000","last_clean":"2023-10-21T17:38:33.412423+0000","last_became_active":"2023-10-21T17:38:32.405548+0000","last_became_peered":"2023-10-21T17:38:32.405548+0000","last_unstale":"2023-10-21T17:38:33.412423+0000","last_undegraded":"2023-10-21T17:38:33.412423+0000","last_fullsized":"2023-10-21T17:38:33.412423+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:02.940331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.346830+0000","last_change":"2023-10-21T17:38:33.341123+0000","last_active":"2023-10-21T17:38:33.346830+0000","last_peered":"2023-10-21T17:38:33.346830+0000","last_clean":"2023-10-21T17:38:33.346830+0000","last_became_active":"2023-10-21T17:38:33.340397+0000","last_became_peered":"2023-10-21T17:38:33.340397+0000","last_unstale":"2023-10-21T17:38:33.346830+0000","last_undegraded":"2023-10-21T17:38:33.346830+0000","last_fullsized":"2023-10-21T17:38:33.346830+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:28.700422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412231+0000","last_change":"2023-10-21T17:38:32.411640+0000","last_active":"2023-10-21T17:38:33.412231+0000","last_peered":"2023-10-21T17:38:33.412231+0000","last_clean":"2023-10-21T17:38:33.412231+0000","last_became_active":"2023-10-21T17:38:32.411258+0000","last_became_peered":"2023-10-21T17:38:32.411258+0000","last_unstale":"2023-10-21T17:38:33.412231+0000","last_undegraded":"2023-10-21T17:38:33.412231+0000","last_fullsized":"2023-10-21T17:38:33.412231+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.349513+0000","last_change":"2023-10-21T17:38:33.346021+0000","last_active":"2023-10-21T17:38:33.349513+0000","last_peered":"2023-10-21T17:38:33.349513+0000","last_clean":"2023-10-21T17:38:33.349513+0000","last_became_active":"2023-10-21T17:38:33.345731+0000","last_became_peered":"2023-10-21T17:38:33.345731+0000","last_unstale":"2023-10-21T17:38:33.349513+0000","last_undegraded":"2023-10-21T17:38:33.349513+0000","last_fullsized":"2023-10-21T17:38:33.349513+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:12.488140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342731+0000","last_change":"2023-10-21T17:38:32.411637+0000","last_active":"2023-10-21T17:38:33.342731+0000","last_peered":"2023-10-21T17:38:33.342731+0000","last_clean":"2023-10-21T17:38:33.342731+0000","last_became_active":"2023-10-21T17:38:32.411322+0000","last_became_peered":"2023-10-21T17:38:32.411322+0000","last_unstale":"2023-10-21T17:38:33.342731+0000","last_undegraded":"2023-10-21T17:38:33.342731+0000","last_fullsized":"2023-10-21T17:38:33.342731+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.616148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361932+0000","last_change":"2023-10-21T17:38:33.357688+0000","last_active":"2023-10-21T17:38:33.361932+0000","last_peered":"2023-10-21T17:38:33.361932+0000","last_clean":"2023-10-21T17:38:33.361932+0000","last_became_active":"2023-10-21T17:38:33.355135+0000","last_became_peered":"2023-10-21T17:38:33.355135+0000","last_unstale":"2023-10-21T17:38:33.361932+0000","last_undegraded":"2023-10-21T17:38:33.361932+0000","last_fullsized":"2023-10-21T17:38:33.361932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:10.598342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.337259+0000","last_change":"2023-10-21T17:38:32.410852+0000","last_active":"2023-10-21T17:38:33.337259+0000","last_peered":"2023-10-21T17:38:33.337259+0000","last_clean":"2023-10-21T17:38:33.337259+0000","last_became_active":"2023-10-21T17:38:32.410413+0000","last_became_peered":"2023-10-21T17:38:32.410413+0000","last_unstale":"2023-10-21T17:38:33.337259+0000","last_undegraded":"2023-10-21T17:38:33.337259+0000","last_fullsized":"2023-10-21T17:38:33.337259+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:26.544773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352012+0000","last_change":"2023-10-21T17:38:33.349513+0000","last_active":"2023-10-21T17:38:33.352012+0000","last_peered":"2023-10-21T17:38:33.352012+0000","last_clean":"2023-10-21T17:38:33.352012+0000","last_became_active":"2023-10-21T17:38:33.349202+0000","last_became_peered":"2023-10-21T17:38:33.349202+0000","last_unstale":"2023-10-21T17:38:33.352012+0000","last_undegraded":"2023-10-21T17:38:33.352012+0000","last_fullsized":"2023-10-21T17:38:33.352012+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:39.055132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339591+0000","last_change":"2023-10-21T17:38:32.417638+0000","last_active":"2023-10-21T17:38:33.339591+0000","last_peered":"2023-10-21T17:38:33.339591+0000","last_clean":"2023-10-21T17:38:33.339591+0000","last_became_active":"2023-10-21T17:38:32.417302+0000","last_became_peered":"2023-10-21T17:38:32.417302+0000","last_unstale":"2023-10-21T17:38:33.339591+0000","last_undegraded":"2023-10-21T17:38:33.339591+0000","last_fullsized":"2023-10-21T17:38:33.339591+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:22.089541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362346+0000","last_change":"2023-10-21T17:38:33.359536+0000","last_active":"2023-10-21T17:38:33.362346+0000","last_peered":"2023-10-21T17:38:33.362346+0000","last_clean":"2023-10-21T17:38:33.362346+0000","last_became_active":"2023-10-21T17:38:33.358634+0000","last_became_peered":"2023-10-21T17:38:33.358634+0000","last_unstale":"2023-10-21T17:38:33.362346+0000","last_undegraded":"2023-10-21T17:38:33.362346+0000","last_fullsized":"2023-10-21T17:38:33.362346+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:31.532929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.341764+0000","last_change":"2023-10-21T17:38:32.414100+0000","last_active":"2023-10-21T17:38:33.341764+0000","last_peered":"2023-10-21T17:38:33.341764+0000","last_clean":"2023-10-21T17:38:33.341764+0000","last_became_active":"2023-10-21T17:38:32.413503+0000","last_became_peered":"2023-10-21T17:38:32.413503+0000","last_unstale":"2023-10-21T17:38:33.341764+0000","last_undegraded":"2023-10-21T17:38:33.341764+0000","last_fullsized":"2023-10-21T17:38:33.341764+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:41.928628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362310+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:33.362310+0000","last_peered":"2023-10-21T17:38:33.362310+0000","last_clean":"2023-10-21T17:38:33.362310+0000","last_became_active":"2023-10-21T17:38:33.361078+0000","last_became_peered":"2023-10-21T17:38:33.361078+0000","last_unstale":"2023-10-21T17:38:33.362310+0000","last_undegraded":"2023-10-21T17:38:33.362310+0000","last_fullsized":"2023-10-21T17:38:33.362310+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:07.568107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"232'3","reported_seq":14,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.413926+0000","last_change":"2023-10-21T17:38:32.404797+0000","last_active":"2023-10-21T17:38:33.413926+0000","last_peered":"2023-10-21T17:38:33.413926+0000","last_clean":"2023-10-21T17:38:33.413926+0000","last_became_active":"2023-10-21T17:38:32.404524+0000","last_became_peered":"2023-10-21T17:38:32.404524+0000","last_unstale":"2023-10-21T17:38:33.413926+0000","last_undegraded":"2023-10-21T17:38:33.413926+0000","last_fullsized":"2023-10-21T17:38:33.413926+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:07.928480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363441+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:33.363441+0000","last_peered":"2023-10-21T17:38:33.363441+0000","last_clean":"2023-10-21T17:38:33.363441+0000","last_became_active":"2023-10-21T17:38:33.361077+0000","last_became_peered":"2023-10-21T17:38:33.361077+0000","last_unstale":"2023-10-21T17:38:33.363441+0000","last_undegraded":"2023-10-21T17:38:33.363441+0000","last_fullsized":"2023-10-21T17:38:33.363441+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:34:17.601212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"232'6","reported_seq":17,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.417661+0000","last_change":"2023-10-21T17:38:32.416726+0000","last_active":"2023-10-21T17:38:33.417661+0000","last_peered":"2023-10-21T17:38:33.417661+0000","last_clean":"2023-10-21T17:38:33.417661+0000","last_became_active":"2023-10-21T17:38:32.416448+0000","last_became_peered":"2023-10-21T17:38:32.416448+0000","last_unstale":"2023-10-21T17:38:33.417661+0000","last_undegraded":"2023-10-21T17:38:33.417661+0000","last_fullsized":"2023-10-21T17:38:33.417661+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352903+0000","last_change":"2023-10-21T17:38:33.347261+0000","last_active":"2023-10-21T17:38:33.352903+0000","last_peered":"2023-10-21T17:38:33.352903+0000","last_clean":"2023-10-21T17:38:33.352903+0000","last_became_active":"2023-10-21T17:38:33.346927+0000","last_became_peered":"2023-10-21T17:38:33.346927+0000","last_unstale":"2023-10-21T17:38:33.352903+0000","last_undegraded":"2023-10-21T17:38:33.352903+0000","last_fullsized":"2023-10-21T17:38:33.352903+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:32.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415763+0000","last_change":"2023-10-21T17:38:32.405640+0000","last_active":"2023-10-21T17:38:33.415763+0000","last_peered":"2023-10-21T17:38:33.415763+0000","last_clean":"2023-10-21T17:38:33.415763+0000","last_became_active":"2023-10-21T17:38:32.404457+0000","last_became_peered":"2023-10-21T17:38:32.404457+0000","last_unstale":"2023-10-21T17:38:33.415763+0000","last_undegraded":"2023-10-21T17:38:33.415763+0000","last_fullsized":"2023-10-21T17:38:33.415763+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:26:53.247310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.353654+0000","last_change":"2023-10-21T17:38:33.351625+0000","last_active":"2023-10-21T17:38:33.353654+0000","last_peered":"2023-10-21T17:38:33.353654+0000","last_clean":"2023-10-21T17:38:33.353654+0000","last_became_active":"2023-10-21T17:38:33.351311+0000","last_became_peered":"2023-10-21T17:38:33.351311+0000","last_unstale":"2023-10-21T17:38:33.353654+0000","last_undegraded":"2023-10-21T17:38:33.353654+0000","last_fullsized":"2023-10-21T17:38:33.353654+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:17.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342820+0000","last_change":"2023-10-21T17:38:32.403441+0000","last_active":"2023-10-21T17:38:33.342820+0000","last_peered":"2023-10-21T17:38:33.342820+0000","last_clean":"2023-10-21T17:38:33.342820+0000","last_became_active":"2023-10-21T17:38:32.403179+0000","last_became_peered":"2023-10-21T17:38:32.403179+0000","last_unstale":"2023-10-21T17:38:33.342820+0000","last_undegraded":"2023-10-21T17:38:33.342820+0000","last_fullsized":"2023-10-21T17:38:33.342820+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:38.273786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361610+0000","last_change":"2023-10-21T17:38:33.358210+0000","last_active":"2023-10-21T17:38:33.361610+0000","last_peered":"2023-10-21T17:38:33.361610+0000","last_clean":"2023-10-21T17:38:33.361610+0000","last_became_active":"2023-10-21T17:38:33.357911+0000","last_became_peered":"2023-10-21T17:38:33.357911+0000","last_unstale":"2023-10-21T17:38:33.361610+0000","last_undegraded":"2023-10-21T17:38:33.361610+0000","last_fullsized":"2023-10-21T17:38:33.361610+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:04.246844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342740+0000","last_change":"2023-10-21T17:38:32.405540+0000","last_active":"2023-10-21T17:38:33.342740+0000","last_peered":"2023-10-21T17:38:33.342740+0000","last_clean":"2023-10-21T17:38:33.342740+0000","last_became_active":"2023-10-21T17:38:32.405267+0000","last_became_peered":"2023-10-21T17:38:32.405267+0000","last_unstale":"2023-10-21T17:38:33.342740+0000","last_undegraded":"2023-10-21T17:38:33.342740+0000","last_fullsized":"2023-10-21T17:38:33.342740+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:15.824449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364775+0000","last_change":"2023-10-21T17:38:33.362626+0000","last_active":"2023-10-21T17:38:33.364775+0000","last_peered":"2023-10-21T17:38:33.364775+0000","last_clean":"2023-10-21T17:38:33.364775+0000","last_became_active":"2023-10-21T17:38:33.361673+0000","last_became_peered":"2023-10-21T17:38:33.361673+0000","last_unstale":"2023-10-21T17:38:33.364775+0000","last_undegraded":"2023-10-21T17:38:33.364775+0000","last_fullsized":"2023-10-21T17:38:33.364775+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:20.548414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"232'8","reported_seq":19,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415469+0000","last_change":"2023-10-21T17:38:32.411814+0000","last_active":"2023-10-21T17:38:33.415469+0000","last_peered":"2023-10-21T17:38:33.415469+0000","last_clean":"2023-10-21T17:38:33.415469+0000","last_became_active":"2023-10-21T17:38:32.411496+0000","last_became_peered":"2023-10-21T17:38:32.411496+0000","last_unstale":"2023-10-21T17:38:33.415469+0000","last_undegraded":"2023-10-21T17:38:33.415469+0000","last_fullsized":"2023-10-21T17:38:33.415469+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:17.401857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359495+0000","last_change":"2023-10-21T17:38:33.357202+0000","last_active":"2023-10-21T17:38:33.359495+0000","last_peered":"2023-10-21T17:38:33.359495+0000","last_clean":"2023-10-21T17:38:33.359495+0000","last_became_active":"2023-10-21T17:38:33.356898+0000","last_became_peered":"2023-10-21T17:38:33.356898+0000","last_unstale":"2023-10-21T17:38:33.359495+0000","last_undegraded":"2023-10-21T17:38:33.359495+0000","last_fullsized":"2023-10-21T17:38:33.359495+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:14.017632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412095+0000","last_change":"2023-10-21T17:38:32.404935+0000","last_active":"2023-10-21T17:38:33.412095+0000","last_peered":"2023-10-21T17:38:33.412095+0000","last_clean":"2023-10-21T17:38:33.412095+0000","last_became_active":"2023-10-21T17:38:32.404139+0000","last_became_peered":"2023-10-21T17:38:32.404139+0000","last_unstale":"2023-10-21T17:38:33.412095+0000","last_undegraded":"2023-10-21T17:38:33.412095+0000","last_fullsized":"2023-10-21T17:38:33.412095+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:53.425393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361440+0000","last_change":"2023-10-21T17:38:33.358480+0000","last_active":"2023-10-21T17:38:33.361440+0000","last_peered":"2023-10-21T17:38:33.361440+0000","last_clean":"2023-10-21T17:38:33.361440+0000","last_became_active":"2023-10-21T17:38:33.358208+0000","last_became_peered":"2023-10-21T17:38:33.358208+0000","last_unstale":"2023-10-21T17:38:33.361440+0000","last_undegraded":"2023-10-21T17:38:33.361440+0000","last_fullsized":"2023-10-21T17:38:33.361440+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:24.254117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414671+0000","last_change":"2023-10-21T17:38:32.413129+0000","last_active":"2023-10-21T17:38:33.414671+0000","last_peered":"2023-10-21T17:38:33.414671+0000","last_clean":"2023-10-21T17:38:33.414671+0000","last_became_active":"2023-10-21T17:38:32.410406+0000","last_became_peered":"2023-10-21T17:38:32.410406+0000","last_unstale":"2023-10-21T17:38:33.414671+0000","last_undegraded":"2023-10-21T17:38:33.414671+0000","last_fullsized":"2023-10-21T17:38:33.414671+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:30.006550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352283+0000","last_change":"2023-10-21T17:38:33.343136+0000","last_active":"2023-10-21T17:38:33.352283+0000","last_peered":"2023-10-21T17:38:33.352283+0000","last_clean":"2023-10-21T17:38:33.352283+0000","last_became_active":"2023-10-21T17:38:33.342344+0000","last_became_peered":"2023-10-21T17:38:33.342344+0000","last_unstale":"2023-10-21T17:38:33.352283+0000","last_undegraded":"2023-10-21T17:38:33.352283+0000","last_fullsized":"2023-10-21T17:38:33.352283+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:28.150954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415181+0000","last_change":"2023-10-21T17:38:32.406815+0000","last_active":"2023-10-21T17:38:33.415181+0000","last_peered":"2023-10-21T17:38:33.415181+0000","last_clean":"2023-10-21T17:38:33.415181+0000","last_became_active":"2023-10-21T17:38:32.406525+0000","last_became_peered":"2023-10-21T17:38:32.406525+0000","last_unstale":"2023-10-21T17:38:33.415181+0000","last_undegraded":"2023-10-21T17:38:33.415181+0000","last_fullsized":"2023-10-21T17:38:33.415181+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:01.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369442+0000","last_change":"2023-10-21T17:38:33.366719+0000","last_active":"2023-10-21T17:38:33.369442+0000","last_peered":"2023-10-21T17:38:33.369442+0000","last_clean":"2023-10-21T17:38:33.369442+0000","last_became_active":"2023-10-21T17:38:33.366173+0000","last_became_peered":"2023-10-21T17:38:33.366173+0000","last_unstale":"2023-10-21T17:38:33.369442+0000","last_undegraded":"2023-10-21T17:38:33.369442+0000","last_fullsized":"2023-10-21T17:38:33.369442+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:05.602488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.335348+0000","last_change":"2023-10-21T17:38:32.403394+0000","last_active":"2023-10-21T17:38:33.335348+0000","last_peered":"2023-10-21T17:38:33.335348+0000","last_clean":"2023-10-21T17:38:33.335348+0000","last_became_active":"2023-10-21T17:38:32.403104+0000","last_became_peered":"2023-10-21T17:38:32.403104+0000","last_unstale":"2023-10-21T17:38:33.335348+0000","last_undegraded":"2023-10-21T17:38:33.335348+0000","last_fullsized":"2023-10-21T17:38:33.335348+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:06:56.765497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352926+0000","last_change":"2023-10-21T17:38:33.348917+0000","last_active":"2023-10-21T17:38:33.352926+0000","last_peered":"2023-10-21T17:38:33.352926+0000","last_clean":"2023-10-21T17:38:33.352926+0000","last_became_active":"2023-10-21T17:38:33.348522+0000","last_became_peered":"2023-10-21T17:38:33.348522+0000","last_unstale":"2023-10-21T17:38:33.352926+0000","last_undegraded":"2023-10-21T17:38:33.352926+0000","last_fullsized":"2023-10-21T17:38:33.352926+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:11:42.293794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.411796+0000","last_change":"2023-10-21T17:38:32.401853+0000","last_active":"2023-10-21T17:38:33.411796+0000","last_peered":"2023-10-21T17:38:33.411796+0000","last_clean":"2023-10-21T17:38:33.411796+0000","last_became_active":"2023-10-21T17:38:32.401567+0000","last_became_peered":"2023-10-21T17:38:32.401567+0000","last_unstale":"2023-10-21T17:38:33.411796+0000","last_undegraded":"2023-10-21T17:38:33.411796+0000","last_fullsized":"2023-10-21T17:38:33.411796+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:24.460794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357777+0000","last_change":"2023-10-21T17:38:33.354105+0000","last_active":"2023-10-21T17:38:33.357777+0000","last_peered":"2023-10-21T17:38:33.357777+0000","last_clean":"2023-10-21T17:38:33.357777+0000","last_became_active":"2023-10-21T17:38:33.353821+0000","last_became_peered":"2023-10-21T17:38:33.353821+0000","last_unstale":"2023-10-21T17:38:33.357777+0000","last_undegraded":"2023-10-21T17:38:33.357777+0000","last_fullsized":"2023-10-21T17:38:33.357777+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:53.603618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.324780+0000","last_change":"2023-10-21T17:38:32.399236+0000","last_active":"2023-10-21T17:38:33.324780+0000","last_peered":"2023-10-21T17:38:33.324780+0000","last_clean":"2023-10-21T17:38:33.324780+0000","last_became_active":"2023-10-21T17:38:32.398909+0000","last_became_peered":"2023-10-21T17:38:32.398909+0000","last_unstale":"2023-10-21T17:38:33.324780+0000","last_undegraded":"2023-10-21T17:38:33.324780+0000","last_fullsized":"2023-10-21T17:38:33.324780+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:04.398168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356416+0000","last_change":"2023-10-21T17:38:33.353848+0000","last_active":"2023-10-21T17:38:33.356416+0000","last_peered":"2023-10-21T17:38:33.356416+0000","last_clean":"2023-10-21T17:38:33.356416+0000","last_became_active":"2023-10-21T17:38:33.351959+0000","last_became_peered":"2023-10-21T17:38:33.351959+0000","last_unstale":"2023-10-21T17:38:33.356416+0000","last_undegraded":"2023-10-21T17:38:33.356416+0000","last_fullsized":"2023-10-21T17:38:33.356416+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:27.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.329328+0000","last_change":"2023-10-21T17:38:32.416515+0000","last_active":"2023-10-21T17:38:33.329328+0000","last_peered":"2023-10-21T17:38:33.329328+0000","last_clean":"2023-10-21T17:38:33.329328+0000","last_became_active":"2023-10-21T17:38:32.415411+0000","last_became_peered":"2023-10-21T17:38:32.415411+0000","last_unstale":"2023-10-21T17:38:33.329328+0000","last_undegraded":"2023-10-21T17:38:33.329328+0000","last_fullsized":"2023-10-21T17:38:33.329328+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:24.896102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.368689+0000","last_change":"2023-10-21T17:38:33.363577+0000","last_active":"2023-10-21T17:38:33.368689+0000","last_peered":"2023-10-21T17:38:33.368689+0000","last_clean":"2023-10-21T17:38:33.368689+0000","last_became_active":"2023-10-21T17:38:33.360945+0000","last_became_peered":"2023-10-21T17:38:33.360945+0000","last_unstale":"2023-10-21T17:38:33.368689+0000","last_undegraded":"2023-10-21T17:38:33.368689+0000","last_fullsized":"2023-10-21T17:38:33.368689+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:10.470691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342027+0000","last_change":"2023-10-21T17:38:32.410021+0000","last_active":"2023-10-21T17:38:33.342027+0000","last_peered":"2023-10-21T17:38:33.342027+0000","last_clean":"2023-10-21T17:38:33.342027+0000","last_became_active":"2023-10-21T17:38:32.409751+0000","last_became_peered":"2023-10-21T17:38:32.409751+0000","last_unstale":"2023-10-21T17:38:33.342027+0000","last_undegraded":"2023-10-21T17:38:33.342027+0000","last_fullsized":"2023-10-21T17:38:33.342027+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.367191+0000","last_change":"2023-10-21T17:38:33.361024+0000","last_active":"2023-10-21T17:38:33.367191+0000","last_peered":"2023-10-21T17:38:33.367191+0000","last_clean":"2023-10-21T17:38:33.367191+0000","last_became_active":"2023-10-21T17:38:33.360523+0000","last_became_peered":"2023-10-21T17:38:33.360523+0000","last_unstale":"2023-10-21T17:38:33.367191+0000","last_undegraded":"2023-10-21T17:38:33.367191+0000","last_fullsized":"2023-10-21T17:38:33.367191+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:25.476738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414330+0000","last_change":"2023-10-21T17:38:32.418683+0000","last_active":"2023-10-21T17:38:33.414330+0000","last_peered":"2023-10-21T17:38:33.414330+0000","last_clean":"2023-10-21T17:38:33.414330+0000","last_became_active":"2023-10-21T17:38:32.418325+0000","last_became_peered":"2023-10-21T17:38:32.418325+0000","last_unstale":"2023-10-21T17:38:33.414330+0000","last_undegraded":"2023-10-21T17:38:33.414330+0000","last_fullsized":"2023-10-21T17:38:33.414330+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:36.165616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361867+0000","last_change":"2023-10-21T17:38:33.358506+0000","last_active":"2023-10-21T17:38:33.361867+0000","last_peered":"2023-10-21T17:38:33.361867+0000","last_clean":"2023-10-21T17:38:33.361867+0000","last_became_active":"2023-10-21T17:38:33.357627+0000","last_became_peered":"2023-10-21T17:38:33.357627+0000","last_unstale":"2023-10-21T17:38:33.361867+0000","last_undegraded":"2023-10-21T17:38:33.361867+0000","last_fullsized":"2023-10-21T17:38:33.361867+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:31.691088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.358242+0000","last_change":"2023-10-21T17:38:32.416259+0000","last_active":"2023-10-21T17:38:33.358242+0000","last_peered":"2023-10-21T17:38:33.358242+0000","last_clean":"2023-10-21T17:38:33.358242+0000","last_became_active":"2023-10-21T17:38:32.415430+0000","last_became_peered":"2023-10-21T17:38:32.415430+0000","last_unstale":"2023-10-21T17:38:33.358242+0000","last_undegraded":"2023-10-21T17:38:33.358242+0000","last_fullsized":"2023-10-21T17:38:33.358242+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361426+0000","last_change":"2023-10-21T17:38:33.358411+0000","last_active":"2023-10-21T17:38:33.361426+0000","last_peered":"2023-10-21T17:38:33.361426+0000","last_clean":"2023-10-21T17:38:33.361426+0000","last_became_active":"2023-10-21T17:38:33.358101+0000","last_became_peered":"2023-10-21T17:38:33.358101+0000","last_unstale":"2023-10-21T17:38:33.361426+0000","last_undegraded":"2023-10-21T17:38:33.361426+0000","last_fullsized":"2023-10-21T17:38:33.361426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:57.303927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.328786+0000","last_change":"2023-10-21T17:38:32.405560+0000","last_active":"2023-10-21T17:38:33.328786+0000","last_peered":"2023-10-21T17:38:33.328786+0000","last_clean":"2023-10-21T17:38:33.328786+0000","last_became_active":"2023-10-21T17:38:32.405233+0000","last_became_peered":"2023-10-21T17:38:32.405233+0000","last_unstale":"2023-10-21T17:38:33.328786+0000","last_undegraded":"2023-10-21T17:38:33.328786+0000","last_fullsized":"2023-10-21T17:38:33.328786+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359581+0000","last_change":"2023-10-21T17:38:33.351368+0000","last_active":"2023-10-21T17:38:33.359581+0000","last_peered":"2023-10-21T17:38:33.359581+0000","last_clean":"2023-10-21T17:38:33.359581+0000","last_became_active":"2023-10-21T17:38:33.350963+0000","last_became_peered":"2023-10-21T17:38:33.350963+0000","last_unstale":"2023-10-21T17:38:33.359581+0000","last_undegraded":"2023-10-21T17:38:33.359581+0000","last_fullsized":"2023-10-21T17:38:33.359581+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339970+0000","last_change":"2023-10-21T17:38:32.405103+0000","last_active":"2023-10-21T17:38:33.339970+0000","last_peered":"2023-10-21T17:38:33.339970+0000","last_clean":"2023-10-21T17:38:33.339970+0000","last_became_active":"2023-10-21T17:38:32.404811+0000","last_became_peered":"2023-10-21T17:38:32.404811+0000","last_unstale":"2023-10-21T17:38:33.339970+0000","last_undegraded":"2023-10-21T17:38:33.339970+0000","last_fullsized":"2023-10-21T17:38:33.339970+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:19.532470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369189+0000","last_change":"2023-10-21T17:38:33.364849+0000","last_active":"2023-10-21T17:38:33.369189+0000","last_peered":"2023-10-21T17:38:33.369189+0000","last_clean":"2023-10-21T17:38:33.369189+0000","last_became_active":"2023-10-21T17:38:33.363791+0000","last_became_peered":"2023-10-21T17:38:33.363791+0000","last_unstale":"2023-10-21T17:38:33.369189+0000","last_undegraded":"2023-10-21T17:38:33.369189+0000","last_fullsized":"2023-10-21T17:38:33.369189+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:39:31.802251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357148+0000","last_change":"2023-10-21T17:38:32.419011+0000","last_active":"2023-10-21T17:38:33.357148+0000","last_peered":"2023-10-21T17:38:33.357148+0000","last_clean":"2023-10-21T17:38:33.357148+0000","last_became_active":"2023-10-21T17:38:32.418410+0000","last_became_peered":"2023-10-21T17:38:32.418410+0000","last_unstale":"2023-10-21T17:38:33.357148+0000","last_undegraded":"2023-10-21T17:38:33.357148+0000","last_fullsized":"2023-10-21T17:38:33.357148+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:22.056635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364223+0000","last_change":"2023-10-21T17:38:33.362147+0000","last_active":"2023-10-21T17:38:33.364223+0000","last_peered":"2023-10-21T17:38:33.364223+0000","last_clean":"2023-10-21T17:38:33.364223+0000","last_became_active":"2023-10-21T17:38:33.360339+0000","last_became_peered":"2023-10-21T17:38:33.360339+0000","last_unstale":"2023-10-21T17:38:33.364223+0000","last_undegraded":"2023-10-21T17:38:33.364223+0000","last_fullsized":"2023-10-21T17:38:33.364223+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:22.754068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342621+0000","last_change":"2023-10-21T17:38:32.408205+0000","last_active":"2023-10-21T17:38:33.342621+0000","last_peered":"2023-10-21T17:38:33.342621+0000","last_clean":"2023-10-21T17:38:33.342621+0000","last_became_active":"2023-10-21T17:38:32.407638+0000","last_became_peered":"2023-10-21T17:38:32.407638+0000","last_unstale":"2023-10-21T17:38:33.342621+0000","last_undegraded":"2023-10-21T17:38:33.342621+0000","last_fullsized":"2023-10-21T17:38:33.342621+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:20.909728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356919+0000","last_change":"2023-10-21T17:38:33.354071+0000","last_active":"2023-10-21T17:38:33.356919+0000","last_peered":"2023-10-21T17:38:33.356919+0000","last_clean":"2023-10-21T17:38:33.356919+0000","last_became_active":"2023-10-21T17:38:33.353746+0000","last_became_peered":"2023-10-21T17:38:33.353746+0000","last_unstale":"2023-10-21T17:38:33.356919+0000","last_undegraded":"2023-10-21T17:38:33.356919+0000","last_fullsized":"2023-10-21T17:38:33.356919+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339062+0000","last_change":"2023-10-21T17:38:32.411663+0000","last_active":"2023-10-21T17:38:33.339062+0000","last_peered":"2023-10-21T17:38:33.339062+0000","last_clean":"2023-10-21T17:38:33.339062+0000","last_became_active":"2023-10-21T17:38:32.411409+0000","last_became_peered":"2023-10-21T17:38:32.411409+0000","last_unstale":"2023-10-21T17:38:33.339062+0000","last_undegraded":"2023-10-21T17:38:33.339062+0000","last_fullsized":"2023-10-21T17:38:33.339062+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:00.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362507+0000","last_change":"2023-10-21T17:38:33.360420+0000","last_active":"2023-10-21T17:38:33.362507+0000","last_peered":"2023-10-21T17:38:33.362507+0000","last_clean":"2023-10-21T17:38:33.362507+0000","last_became_active":"2023-10-21T17:38:33.360143+0000","last_became_peered":"2023-10-21T17:38:33.360143+0000","last_unstale":"2023-10-21T17:38:33.362507+0000","last_undegraded":"2023-10-21T17:38:33.362507+0000","last_fullsized":"2023-10-21T17:38:33.362507+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:08.944424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.333508+0000","last_change":"2023-10-21T17:38:32.411805+0000","last_active":"2023-10-21T17:38:33.333508+0000","last_peered":"2023-10-21T17:38:33.333508+0000","last_clean":"2023-10-21T17:38:33.333508+0000","last_became_active":"2023-10-21T17:38:32.411439+0000","last_became_peered":"2023-10-21T17:38:32.411439+0000","last_unstale":"2023-10-21T17:38:33.333508+0000","last_undegraded":"2023-10-21T17:38:33.333508+0000","last_fullsized":"2023-10-21T17:38:33.333508+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:57.270863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370689+0000","last_change":"2023-10-21T17:38:33.370135+0000","last_active":"2023-10-21T17:38:33.370689+0000","last_peered":"2023-10-21T17:38:33.370689+0000","last_clean":"2023-10-21T17:38:33.370689+0000","last_became_active":"2023-10-21T17:38:33.369798+0000","last_became_peered":"2023-10-21T17:38:33.369798+0000","last_unstale":"2023-10-21T17:38:33.370689+0000","last_undegraded":"2023-10-21T17:38:33.370689+0000","last_fullsized":"2023-10-21T17:38:33.370689+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:42.944572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.411940+0000","last_change":"2023-10-21T17:38:32.397140+0000","last_active":"2023-10-21T17:38:33.411940+0000","last_peered":"2023-10-21T17:38:33.411940+0000","last_clean":"2023-10-21T17:38:33.411940+0000","last_became_active":"2023-10-21T17:38:32.396837+0000","last_became_peered":"2023-10-21T17:38:32.396837+0000","last_unstale":"2023-10-21T17:38:33.411940+0000","last_undegraded":"2023-10-21T17:38:33.411940+0000","last_fullsized":"2023-10-21T17:38:33.411940+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:17.875367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"221'4873","reported_seq":5802,"reported_epoch":231,"state":"active+clean","last_fresh":"2023-10-21T17:38:32.400433+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:38:32.400433+0000","last_peered":"2023-10-21T17:38:32.400433+0000","last_clean":"2023-10-21T17:38:32.400433+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:38:32.400433+0000","last_undegraded":"2023-10-21T17:38:32.400433+0000","last_fullsized":"2023-10-21T17:38:32.400433+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4873,"log_dups_size":0,"ondisk_log_size":4873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4873,"ondisk_log_size":4873,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575997,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70708,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301132,"statfs":{"total":96636764160,"available":96564359168,"internally_reserved":0,"allocated":3506176,"data_stored":2098554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.97399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"max":{"1min":1.6419999999999999,"5min":1.841,"15min":2.2549999999999999},"last":0.998},{"interface":"front","average":{"1min":1.117,"5min":1.101,"15min":1.1599999999999999},"min":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.724,"15min":1.7669999999999999},"last":1.268}]},{"osd":1,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1739999999999999,"15min":1.218},"min":{"1min":0.96299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.589,"5min":1.7170000000000001,"15min":1.73},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.262,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.91200000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.506,"5min":1.792,"15min":1.885},"last":1.1799999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.466},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.86899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2909999999999999,"15min":1.3180000000000001},"min":{"1min":0.998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.472,"5min":2.472,"15min":2.5110000000000001},"last":1.139},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.486,"15min":1.4670000000000001},"min":{"1min":1.23,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.3120000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.383,"15min":1.357},"min":{"1min":0.84599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.5129999999999999,"15min":1.496},"min":{"1min":1.1619999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.173,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.54},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.474,"15min":1.45},"min":{"1min":1.135,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":1.5169999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.587,"15min":1.5469999999999999},"min":{"1min":0.85499999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.3100000000000001,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.488,"15min":1.476},"min":{"1min":0.88800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.3600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42336,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329504,"statfs":{"total":96636764160,"available":96593412096,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1659999999999999,"15min":1.198},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.728,"5min":1.944,"15min":1.944},"last":1.27},{"interface":"front","average":{"1min":1.256,"5min":1.2050000000000001,"15min":1.222},"min":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.643,"5min":1.7090000000000001,"15min":1.762},"last":0.97399999999999998}]},{"osd":1,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.171,"15min":1.1919999999999999},"min":{"1min":0.84099999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6379999999999999},"last":1.123},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2989999999999999,"15min":1.319},"min":{"1min":0.98599999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.782,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.2070000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.272,"15min":1.3169999999999999},"min":{"1min":1.006,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5649999999999999,"5min":1.712,"15min":1.8300000000000001},"last":1.028},{"interface":"front","average":{"1min":1.393,"5min":1.3089999999999999,"15min":1.3069999999999999},"min":{"1min":1.1799999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.611,"5min":1.679,"15min":1.929},"last":0.93600000000000005}]},{"osd":3,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3919999999999999,"15min":1.417},"min":{"1min":0.90700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.593,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.427,"15min":1.4650000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.90500000000000003},"max":{"1min":1.825,"5min":1.9039999999999999,"15min":2.048},"last":1.1659999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":1.704,"5min":1.77,"15min":1.9950000000000001},"last":1.321},{"interface":"front","average":{"1min":1.444,"5min":1.4079999999999999,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.3859999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5349999999999999,"15min":1.571},"min":{"1min":1.274,"5min":1.1000000000000001,"15min":1.0029999999999999},"max":{"1min":1.891,"5min":1.956,"15min":2.339},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.325,"15min":1.3759999999999999},"min":{"1min":0.98199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6870000000000001,"5min":1.796,"15min":2.2890000000000001},"last":1.796}]},{"osd":7,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4970000000000001,"15min":1.536},"min":{"1min":1.1080000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.762,"5min":1.9339999999999999,"15min":2.1200000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.6259999999999999,"15min":1.6679999999999999},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.0449999999999999},"max":{"1min":1.9570000000000001,"5min":2.1499999999999999,"15min":2.5409999999999999},"last":1.4830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575999,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.81599999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6319999999999999,"5min":1.6799999999999999,"15min":1.9930000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.64200000000000002,"15min":0.627},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.206}]},{"osd":1,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":1.012,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.738,"5min":1.806,"15min":1.8620000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.202,"15min":1.2330000000000001},"min":{"1min":0.88500000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.75,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.88600000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.2829999999999999,"15min":1.302},"min":{"1min":1.087,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.802},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.8600000000000001,"15min":1.891},"last":1.27}]},{"osd":3,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.325,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.732,"5min":1.829,"15min":1.9650000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.4079999999999999,"15min":1.4199999999999999},"min":{"1min":1.038,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.772,"5min":1.881,"15min":1.9139999999999999},"last":1.012}]},{"osd":4,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2230000000000001,"15min":1.25},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.462,"5min":1.7050000000000001,"15min":1.9910000000000001},"last":1.381},{"interface":"front","average":{"1min":1.46,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":1.1259999999999999,"5min":0.85399999999999998,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":1.877,"15min":1.9399999999999999},"last":1.589}]},{"osd":6,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.482},"min":{"1min":1.081,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8540000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.508,"15min":1.524},"min":{"1min":0.84999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7509999999999999,"5min":1.9730000000000001,"15min":2.52},"last":1.6659999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5309999999999999,"15min":1.5329999999999999},"min":{"1min":1.1319999999999999,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":1.96,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.544,"15min":1.5640000000000001},"min":{"1min":1.018,"5min":0.68400000000000005,"15min":0.56399999999999995},"max":{"1min":1.833,"5min":2.0750000000000002,"15min":2.1349999999999998},"last":1.631}]}]},{"osd":4,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47920,"kb_used_data":2844,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323920,"statfs":{"total":96636764160,"available":96587694080,"internally_reserved":0,"allocated":2912256,"data_stored":1508186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.117,"15min":1.1599999999999999},"min":{"1min":0.72299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.8480000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.659,"5min":1.855,"15min":1.869},"last":1.3819999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.58099999999999996},"max":{"1min":1.4890000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.321},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.248,"15min":1.268},"min":{"1min":0.93300000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.665,"5min":1.762,"15min":1.762},"last":1.4139999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2649999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":1.71,"15min":1.7609999999999999},"last":1.131},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3620000000000001,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5740000000000001,"5min":1.762,"15min":1.762},"last":1.4870000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2729999999999999,"15min":1.3200000000000001},"min":{"1min":0.92100000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4870000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":1.284},{"interface":"front","average":{"1min":1.387,"5min":1.419,"15min":1.425},"min":{"1min":1.002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7350000000000001,"5min":1.825,"15min":1.9239999999999999},"last":1.2450000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.82699999999999996},"max":{"1min":1.639,"5min":1.8340000000000001,"15min":1.867},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.98799999999999999,"5min":0.879,"15min":0.81000000000000005},"max":{"1min":1.6990000000000001,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":1.0760000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4139999999999999,"15min":1.4430000000000001},"min":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.825,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.595,"15min":1.595},"min":{"1min":1.147,"5min":1.147,"15min":1.133},"max":{"1min":1.7210000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.5469999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.534,"15min":1.5189999999999999},"min":{"1min":1.2090000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.833,"5min":2.234,"15min":2.234},"last":1.171},{"interface":"front","average":{"1min":1.55,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.0880000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9279999999999999,"5min":1.984,"15min":2.081},"last":1.4379999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56224,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315616,"statfs":{"total":96636764160,"available":96579190784,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98399999999999999},"min":{"1min":0.54400000000000004,"5min":0.48899999999999999,"15min":0.432},"max":{"1min":1.4550000000000001,"5min":1.6439999999999999,"15min":1.71},"last":1.169},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.73499999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.639}]},{"osd":1,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0409999999999999,"15min":1.091},"min":{"1min":0.626,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3340000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.875,"5min":1.879,"15min":1.879},"last":1.6870000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1100000000000001,"15min":1.1160000000000001},"min":{"1min":0.59699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.921,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.075},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.823,"5min":1.853,"15min":2},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4350000000000001,"15min":1.4410000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.218},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4710000000000001},"min":{"1min":1.048,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.802,"5min":1.9430000000000001,"15min":2.169},"last":1.4590000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.89200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9330000000000001,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.401},{"interface":"front","average":{"1min":1.534,"5min":1.5389999999999999,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.1800000000000002,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.8600000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.546,"15min":1.601},"min":{"1min":1.0189999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.966,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.593,"15min":1.625},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0579999999999998,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.573}]},{"osd":7,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.135,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2789999999999999,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.659,"15min":1.714},"min":{"1min":1.0529999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1760000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.3480000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52076,"kb_used_data":2848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319764,"statfs":{"total":96636764160,"available":96583438336,"internally_reserved":0,"allocated":2916352,"data_stored":1508276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.90400000000000003},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.149,"5min":1.841,"15min":1.841},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.79500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.7529999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.032}]},{"osd":1,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.78900000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6539999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.538,"5min":1.417,"15min":1.395},"min":{"1min":1.151,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.8300000000000001,"5min":1.958,"15min":2.4060000000000001},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.1970000000000001,"15min":1.198},"min":{"1min":0.78400000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.1760000000000002},"last":1.095},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.92900000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":2.077},"last":1.369}]},{"osd":4,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":1.1870000000000001,"5min":1.0529999999999999,"15min":1.044},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1680000000000001},"last":1.536},{"interface":"front","average":{"1min":1.522,"5min":1.468,"15min":1.46},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0329999999999999},"max":{"1min":1.9079999999999999,"5min":1.978,"15min":2.5939999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4279999999999999,"15min":1.4199999999999999},"min":{"1min":1.0800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.613},{"interface":"front","average":{"1min":1.615,"5min":1.6000000000000001,"15min":1.6120000000000001},"min":{"1min":1.286,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":1.9710000000000001,"5min":1.972,"15min":2.3759999999999999},"last":1.762}]},{"osd":6,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.571,"15min":1.575},"min":{"1min":1.145,"5min":1.1060000000000001,"15min":1.095},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.694},{"interface":"front","average":{"1min":1.595,"5min":1.5349999999999999,"15min":1.5289999999999999},"min":{"1min":1.115,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5049999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.651,"15min":1.647},"min":{"1min":1.337,"5min":1.1080000000000001,"15min":1.0780000000000001},"max":{"1min":2.1240000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5980000000000001,"15min":1.6040000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.732}]}]},{"osd":1,"up_from":13,"seq":55834575997,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52064,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319776,"statfs":{"total":96636764160,"available":96583450624,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3120000000000001,"5min":1.367,"15min":1.976},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.147,"15min":1.1619999999999999},"min":{"1min":0.81200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.679,"5min":1.833,"15min":1.8400000000000001},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.046,"15min":1.048},"min":{"1min":0.83399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5369999999999999,"5min":1.6990000000000001,"15min":1.79},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.238,"15min":1.252},"min":{"1min":1.121,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6100000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.397},"min":{"1min":1.0780000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8620000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.498,"5min":1.28,"15min":1.304},"min":{"1min":0.98899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.9339999999999999},"last":0.73899999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3600000000000001,"15min":1.3580000000000001},"min":{"1min":1.0820000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.643,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.575,"5min":1.498,"15min":1.504},"min":{"1min":1.157,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.867,"5min":2.0750000000000002,"15min":2.383},"last":1.0269999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.202,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.732,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.524,"15min":1.5429999999999999},"min":{"1min":1.1639999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.8240000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.147}]},{"osd":6,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.6080000000000001,"15min":1.6060000000000001},"min":{"1min":1.2729999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.085,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.173},{"interface":"front","average":{"1min":1.556,"5min":1.552,"15min":1.5629999999999999},"min":{"1min":1.2390000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.752,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.073}]},{"osd":7,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.653,"15min":1.6799999999999999},"min":{"1min":1.298,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.994,"5min":2.1829999999999998,"15min":2.548},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.738,"15min":1.7270000000000001},"min":{"1min":1.401,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1739999999999999,"5min":2.492,"15min":2.548},"last":1.087}]}]},{"osd":0,"up_from":13,"seq":55834575997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52640,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319200,"statfs":{"total":96636764160,"available":96582860800,"internally_reserved":0,"allocated":3493888,"data_stored":2091343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93400000000000005,"15min":0.95199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2989999999999999,"5min":1.514,"15min":1.774},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.036,"15min":1.101},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.506,"5min":1.556,"15min":1.774},"last":1.1950000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.121,"15min":1.171},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1140000000000001,"15min":1.181},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.708,"5min":1.708,"15min":1.851},"last":1.014}]},{"osd":3,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0960000000000001,"15min":1.133},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6259999999999999,"5min":1.897,"15min":1.9590000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.266,"5min":1.327,"15min":1.391},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.97499999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.353,"15min":1.4299999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8169999999999999,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.137},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5580000000000001,"15min":1.599},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.0620000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.401,"15min":1.4059999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.772,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.448,"5min":1.51,"15min":1.573},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.1150000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.847,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.611,"15min":1.6759999999999999},"min":{"1min":0.999,"5min":0.996,"15min":0.996},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":1.1559999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4890000000000001,"15min":1.5449999999999999},"min":{"1min":0.82799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8380000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5900000000000001,"15min":1.623},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.952,"5min":2.2519999999999998,"15min":2.262},"last":1.0840000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:38:37.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:38:38.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.198+0000 7f5c3f511640 1 Processor -- start 2023-10-21T17:38:38.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.198+0000 7f5c3f511640 1 -- start start 2023-10-21T17:38:38.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3f511640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c380b07d0 0x7f5c381614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3f511640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 0x7f5c380c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3f511640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c380c9020 con 0x7f5c3815cca0 2023-10-21T17:38:38.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3f511640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380ccc20 con 0x7f5c380b07d0 2023-10-21T17:38:38.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3f511640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380c92e0 con 0x7f5c380cab40 2023-10-21T17:38:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3dd0e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c380b07d0 0x7f5c381614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.200+0000 7f5c3dd0e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5c380b07d0 0x7f5c381614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49734/0 (socket says 172.21.15.53:49734) 2023-10-21T17:38:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/1656878927 learned_addr learned my addr 172.21.15.53:0/1656878927 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/1656878927 >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 msgr2=0x7f5c380c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3ed10640 1 --2- 172.21.15.53:0/1656878927 >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 0x7f5c380c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3dd0e640 1 --2- 172.21.15.53:0/1656878927 >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 0x7f5c380c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/1656878927 >> v1:172.21.15.124:6789/0 conn(0x7f5c3815cca0 legacy=0x7f5c380b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/1656878927 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380cc7b0 con 0x7f5c380b07d0 2023-10-21T17:38:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.201+0000 7f5c3ed10640 1 --2- 172.21.15.53:0/1656878927 >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 0x7f5c380c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:38.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.202+0000 7f5c3dd0e640 1 --2- 172.21.15.53:0/1656878927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380b07d0 0x7f5c381614b0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5c3806c430 tx=0x7f5c2802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0ece76e5f0fdfe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3d50d640 1 -- 172.21.15.53:0/1656878927 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28031750 con 0x7f5c380b07d0 2023-10-21T17:38:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3d50d640 1 -- 172.21.15.53:0/1656878927 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c280318b0 con 0x7f5c380b07d0 2023-10-21T17:38:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3d50d640 1 -- 172.21.15.53:0/1656878927 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28031c40 con 0x7f5c380b07d0 2023-10-21T17:38:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 -- 172.21.15.53:0/1656878927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380b07d0 msgr2=0x7f5c381614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 --2- 172.21.15.53:0/1656878927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380b07d0 0x7f5c381614b0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5c3806c430 tx=0x7f5c2802b7f0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 -- 172.21.15.53:0/1656878927 shutdown_connections 2023-10-21T17:38:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 --2- 172.21.15.53:0/1656878927 >> v2:172.21.15.124:3300/0 conn(0x7f5c380cab40 0x7f5c380c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 --2- 172.21.15.53:0/1656878927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380b07d0 0x7f5c381614b0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 -- 172.21.15.53:0/1656878927 >> 172.21.15.53:0/1656878927 conn(0x7f5c380cb0c0 msgr2=0x7f5c380cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:38.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.203+0000 7f5c3f511640 1 -- 172.21.15.53:0/1656878927 shutdown_connections 2023-10-21T17:38:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.204+0000 7f5c3f511640 1 -- 172.21.15.53:0/1656878927 wait complete. 2023-10-21T17:38:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.205+0000 7f5c3f511640 1 Processor -- start 2023-10-21T17:38:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.205+0000 7f5c3f511640 1 -- start start 2023-10-21T17:38:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c380b07d0 0x7f5c381f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c3815cca0 0x7f5c3815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b1a70 con 0x7f5c380cab40 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b1cc0 con 0x7f5c3815cca0 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.206+0000 7f5c3f511640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c380b1fa0 con 0x7f5c380b07d0 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49742/0 (socket says 172.21.15.53:49742) 2023-10-21T17:38:38.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/4070601276 learned_addr learned my addr 172.21.15.53:0/4070601276 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:38.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c380b07d0 msgr2=0x7f5c381f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c380b07d0 0x7f5c381f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c3815cca0 msgr2=0x7f5c3815fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c3815cca0 0x7f5c3815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 -- 172.21.15.53:0/4070601276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280094d0 con 0x7f5c380cab40 2023-10-21T17:38:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.207+0000 7f5c3dd0e640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.208+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28033c80 con 0x7f5c380cab40 2023-10-21T17:38:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.208+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28033de0 con 0x7f5c380cab40 2023-10-21T17:38:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.208+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2803c640 con 0x7f5c380cab40 2023-10-21T17:38:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.208+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c381f5ba0 con 0x7f5c380cab40 2023-10-21T17:38:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.208+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c380b2280 con 0x7f5c380cab40 2023-10-21T17:38:38.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.209+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c381f60c0 con 0x7f5c380cab40 2023-10-21T17:38:38.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.210+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5c2803a040 con 0x7f5c380cab40 2023-10-21T17:38:38.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.211+0000 7f5c2effd640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 0x7f5c1c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.211+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5c280ab6a0 con 0x7f5c380cab40 2023-10-21T17:38:38.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.212+0000 7f5c3e50f640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 0x7f5c1c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.213+0000 7f5c3e50f640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 0x7f5c1c06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c340043c0 tx=0x7f5c34007d30 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.215+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c381f60c0 con 0x7f5c380cab40 2023-10-21T17:38:38.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.409+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c380cd150 con 0x7f5c1c06afc0 2023-10-21T17:38:38.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.431+0000 7f5c2effd640 1 -- 172.21.15.53:0/4070601276 <== mgr.37814 v2:172.21.15.124:6832/3930748558 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286467 (secure 0 0 0) 0x7f5c34004060 con 0x7f5c1c06afc0 2023-10-21T17:38:38.433 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:38.437 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:38:38.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 msgr2=0x7f5c1c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 0x7f5c1c06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c340043c0 tx=0x7f5c34007d30 comp rx=0 tx=0).stop 2023-10-21T17:38:38.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 msgr2=0x7f5c38157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000c00 tx=0x7f5c280046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 shutdown_connections 2023-10-21T17:38:38.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c3815cca0 0x7f5c3815fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c380cab40 0x7f5c38157820 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5c1c06afc0 0x7f5c1c06d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 --2- 172.21.15.53:0/4070601276 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c380b07d0 0x7f5c381f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 >> 172.21.15.53:0/4070601276 conn(0x7f5c380cb0c0 msgr2=0x7f5c380cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:38.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 shutdown_connections 2023-10-21T17:38:38.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.438+0000 7f5c3f511640 1 -- 172.21.15.53:0/4070601276 wait complete. 2023-10-21T17:38:38.458 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-21T17:38:37.679960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":631,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4911,"ondisk_log_size":4911,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":434352,"kb_used_data":23856,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754540368,"statfs":{"total":773094113280,"available":772649336832,"internally_reserved":0,"allocated":24428544,"data_stored":13210259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":420204089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001950"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369295+0000","last_change":"2023-10-21T17:38:33.368541+0000","last_active":"2023-10-21T17:38:33.369295+0000","last_peered":"2023-10-21T17:38:33.369295+0000","last_clean":"2023-10-21T17:38:33.369295+0000","last_became_active":"2023-10-21T17:38:33.368186+0000","last_became_peered":"2023-10-21T17:38:33.368186+0000","last_unstale":"2023-10-21T17:38:33.369295+0000","last_undegraded":"2023-10-21T17:38:33.369295+0000","last_fullsized":"2023-10-21T17:38:33.369295+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:29.632092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352509+0000","last_change":"2023-10-21T17:38:33.348939+0000","last_active":"2023-10-21T17:38:33.352509+0000","last_peered":"2023-10-21T17:38:33.352509+0000","last_clean":"2023-10-21T17:38:33.352509+0000","last_became_active":"2023-10-21T17:38:33.348561+0000","last_became_peered":"2023-10-21T17:38:33.348561+0000","last_unstale":"2023-10-21T17:38:33.352509+0000","last_undegraded":"2023-10-21T17:38:33.352509+0000","last_fullsized":"2023-10-21T17:38:33.352509+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:43:26.003956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.360419+0000","last_change":"2023-10-21T17:38:33.354797+0000","last_active":"2023-10-21T17:38:33.360419+0000","last_peered":"2023-10-21T17:38:33.360419+0000","last_clean":"2023-10-21T17:38:33.360419+0000","last_became_active":"2023-10-21T17:38:33.354503+0000","last_became_peered":"2023-10-21T17:38:33.354503+0000","last_unstale":"2023-10-21T17:38:33.360419+0000","last_undegraded":"2023-10-21T17:38:33.360419+0000","last_fullsized":"2023-10-21T17:38:33.360419+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:16.840393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363954+0000","last_change":"2023-10-21T17:38:33.359316+0000","last_active":"2023-10-21T17:38:33.363954+0000","last_peered":"2023-10-21T17:38:33.363954+0000","last_clean":"2023-10-21T17:38:33.363954+0000","last_became_active":"2023-10-21T17:38:33.356311+0000","last_became_peered":"2023-10-21T17:38:33.356311+0000","last_unstale":"2023-10-21T17:38:33.363954+0000","last_undegraded":"2023-10-21T17:38:33.363954+0000","last_fullsized":"2023-10-21T17:38:33.363954+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:31.022494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356957+0000","last_change":"2023-10-21T17:38:33.354002+0000","last_active":"2023-10-21T17:38:33.356957+0000","last_peered":"2023-10-21T17:38:33.356957+0000","last_clean":"2023-10-21T17:38:33.356957+0000","last_became_active":"2023-10-21T17:38:33.353709+0000","last_became_peered":"2023-10-21T17:38:33.353709+0000","last_unstale":"2023-10-21T17:38:33.356957+0000","last_undegraded":"2023-10-21T17:38:33.356957+0000","last_fullsized":"2023-10-21T17:38:33.356957+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:14.017632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.355581+0000","last_change":"2023-10-21T17:38:33.351557+0000","last_active":"2023-10-21T17:38:33.355581+0000","last_peered":"2023-10-21T17:38:33.355581+0000","last_clean":"2023-10-21T17:38:33.355581+0000","last_became_active":"2023-10-21T17:38:33.351230+0000","last_became_peered":"2023-10-21T17:38:33.351230+0000","last_unstale":"2023-10-21T17:38:33.355581+0000","last_undegraded":"2023-10-21T17:38:33.355581+0000","last_fullsized":"2023-10-21T17:38:33.355581+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356671+0000","last_change":"2023-10-21T17:38:33.352800+0000","last_active":"2023-10-21T17:38:33.356671+0000","last_peered":"2023-10-21T17:38:33.356671+0000","last_clean":"2023-10-21T17:38:33.356671+0000","last_became_active":"2023-10-21T17:38:33.351990+0000","last_became_peered":"2023-10-21T17:38:33.351990+0000","last_unstale":"2023-10-21T17:38:33.356671+0000","last_undegraded":"2023-10-21T17:38:33.356671+0000","last_fullsized":"2023-10-21T17:38:33.356671+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:55:23.855999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.367523+0000","last_change":"2023-10-21T17:38:33.363315+0000","last_active":"2023-10-21T17:38:33.367523+0000","last_peered":"2023-10-21T17:38:33.367523+0000","last_clean":"2023-10-21T17:38:33.367523+0000","last_became_active":"2023-10-21T17:38:33.363007+0000","last_became_peered":"2023-10-21T17:38:33.363007+0000","last_unstale":"2023-10-21T17:38:33.367523+0000","last_undegraded":"2023-10-21T17:38:33.367523+0000","last_fullsized":"2023-10-21T17:38:33.367523+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:13.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364051+0000","last_change":"2023-10-21T17:38:33.363542+0000","last_active":"2023-10-21T17:38:33.364051+0000","last_peered":"2023-10-21T17:38:33.364051+0000","last_clean":"2023-10-21T17:38:33.364051+0000","last_became_active":"2023-10-21T17:38:33.363143+0000","last_became_peered":"2023-10-21T17:38:33.363143+0000","last_unstale":"2023-10-21T17:38:33.364051+0000","last_undegraded":"2023-10-21T17:38:33.364051+0000","last_fullsized":"2023-10-21T17:38:33.364051+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:29.955478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369678+0000","last_change":"2023-10-21T17:38:33.368484+0000","last_active":"2023-10-21T17:38:33.369678+0000","last_peered":"2023-10-21T17:38:33.369678+0000","last_clean":"2023-10-21T17:38:33.369678+0000","last_became_active":"2023-10-21T17:38:33.366828+0000","last_became_peered":"2023-10-21T17:38:33.366828+0000","last_unstale":"2023-10-21T17:38:33.369678+0000","last_undegraded":"2023-10-21T17:38:33.369678+0000","last_fullsized":"2023-10-21T17:38:33.369678+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:47:57.779222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370053+0000","last_change":"2023-10-21T17:38:33.369555+0000","last_active":"2023-10-21T17:38:33.370053+0000","last_peered":"2023-10-21T17:38:33.370053+0000","last_clean":"2023-10-21T17:38:33.370053+0000","last_became_active":"2023-10-21T17:38:33.369212+0000","last_became_peered":"2023-10-21T17:38:33.369212+0000","last_unstale":"2023-10-21T17:38:33.370053+0000","last_undegraded":"2023-10-21T17:38:33.370053+0000","last_fullsized":"2023-10-21T17:38:33.370053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:33.952465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357216+0000","last_change":"2023-10-21T17:38:33.349361+0000","last_active":"2023-10-21T17:38:33.357216+0000","last_peered":"2023-10-21T17:38:33.357216+0000","last_clean":"2023-10-21T17:38:33.357216+0000","last_became_active":"2023-10-21T17:38:33.348672+0000","last_became_peered":"2023-10-21T17:38:33.348672+0000","last_unstale":"2023-10-21T17:38:33.357216+0000","last_undegraded":"2023-10-21T17:38:33.357216+0000","last_fullsized":"2023-10-21T17:38:33.357216+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:38:57.231941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363382+0000","last_change":"2023-10-21T17:38:33.358688+0000","last_active":"2023-10-21T17:38:33.363382+0000","last_peered":"2023-10-21T17:38:33.363382+0000","last_clean":"2023-10-21T17:38:33.363382+0000","last_became_active":"2023-10-21T17:38:33.354936+0000","last_became_peered":"2023-10-21T17:38:33.354936+0000","last_unstale":"2023-10-21T17:38:33.363382+0000","last_undegraded":"2023-10-21T17:38:33.363382+0000","last_fullsized":"2023-10-21T17:38:33.363382+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:01.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369293+0000","last_change":"2023-10-21T17:38:33.366456+0000","last_active":"2023-10-21T17:38:33.369293+0000","last_peered":"2023-10-21T17:38:33.369293+0000","last_clean":"2023-10-21T17:38:33.369293+0000","last_became_active":"2023-10-21T17:38:33.366015+0000","last_became_peered":"2023-10-21T17:38:33.366015+0000","last_unstale":"2023-10-21T17:38:33.369293+0000","last_undegraded":"2023-10-21T17:38:33.369293+0000","last_fullsized":"2023-10-21T17:38:33.369293+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:05.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362586+0000","last_change":"2023-10-21T17:38:33.360718+0000","last_active":"2023-10-21T17:38:33.362586+0000","last_peered":"2023-10-21T17:38:33.362586+0000","last_clean":"2023-10-21T17:38:33.362586+0000","last_became_active":"2023-10-21T17:38:33.359530+0000","last_became_peered":"2023-10-21T17:38:33.359530+0000","last_unstale":"2023-10-21T17:38:33.362586+0000","last_undegraded":"2023-10-21T17:38:33.362586+0000","last_fullsized":"2023-10-21T17:38:33.362586+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:28:01.937569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.349500+0000","last_change":"2023-10-21T17:38:33.344462+0000","last_active":"2023-10-21T17:38:33.349500+0000","last_peered":"2023-10-21T17:38:33.349500+0000","last_clean":"2023-10-21T17:38:33.349500+0000","last_became_active":"2023-10-21T17:38:33.344064+0000","last_became_peered":"2023-10-21T17:38:33.344064+0000","last_unstale":"2023-10-21T17:38:33.349500+0000","last_undegraded":"2023-10-21T17:38:33.349500+0000","last_fullsized":"2023-10-21T17:38:33.349500+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:10:58.893895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361953+0000","last_change":"2023-10-21T17:38:33.360922+0000","last_active":"2023-10-21T17:38:33.361953+0000","last_peered":"2023-10-21T17:38:33.361953+0000","last_clean":"2023-10-21T17:38:33.361953+0000","last_became_active":"2023-10-21T17:38:33.360628+0000","last_became_peered":"2023-10-21T17:38:33.360628+0000","last_unstale":"2023-10-21T17:38:33.361953+0000","last_undegraded":"2023-10-21T17:38:33.361953+0000","last_fullsized":"2023-10-21T17:38:33.361953+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:39.289731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.354841+0000","last_change":"2023-10-21T17:38:33.351005+0000","last_active":"2023-10-21T17:38:33.354841+0000","last_peered":"2023-10-21T17:38:33.354841+0000","last_clean":"2023-10-21T17:38:33.354841+0000","last_became_active":"2023-10-21T17:38:33.350305+0000","last_became_peered":"2023-10-21T17:38:33.350305+0000","last_unstale":"2023-10-21T17:38:33.354841+0000","last_undegraded":"2023-10-21T17:38:33.354841+0000","last_fullsized":"2023-10-21T17:38:33.354841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:41:33.952465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369428+0000","last_change":"2023-10-21T17:38:33.366914+0000","last_active":"2023-10-21T17:38:33.369428+0000","last_peered":"2023-10-21T17:38:33.369428+0000","last_clean":"2023-10-21T17:38:33.369428+0000","last_became_active":"2023-10-21T17:38:33.366337+0000","last_became_peered":"2023-10-21T17:38:33.366337+0000","last_unstale":"2023-10-21T17:38:33.369428+0000","last_undegraded":"2023-10-21T17:38:33.369428+0000","last_fullsized":"2023-10-21T17:38:33.369428+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:09.263356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.368954+0000","last_change":"2023-10-21T17:38:33.363852+0000","last_active":"2023-10-21T17:38:33.368954+0000","last_peered":"2023-10-21T17:38:33.368954+0000","last_clean":"2023-10-21T17:38:33.368954+0000","last_became_active":"2023-10-21T17:38:33.363427+0000","last_became_peered":"2023-10-21T17:38:33.363427+0000","last_unstale":"2023-10-21T17:38:33.368954+0000","last_undegraded":"2023-10-21T17:38:33.368954+0000","last_fullsized":"2023-10-21T17:38:33.368954+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:28:01.937569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369681+0000","last_change":"2023-10-21T17:38:33.367227+0000","last_active":"2023-10-21T17:38:33.369681+0000","last_peered":"2023-10-21T17:38:33.369681+0000","last_clean":"2023-10-21T17:38:33.369681+0000","last_became_active":"2023-10-21T17:38:33.366922+0000","last_became_peered":"2023-10-21T17:38:33.366922+0000","last_unstale":"2023-10-21T17:38:33.369681+0000","last_undegraded":"2023-10-21T17:38:33.369681+0000","last_fullsized":"2023-10-21T17:38:33.369681+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:37:12.429036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369215+0000","last_change":"2023-10-21T17:38:33.366191+0000","last_active":"2023-10-21T17:38:33.369215+0000","last_peered":"2023-10-21T17:38:33.369215+0000","last_clean":"2023-10-21T17:38:33.369215+0000","last_became_active":"2023-10-21T17:38:33.363694+0000","last_became_peered":"2023-10-21T17:38:33.363694+0000","last_unstale":"2023-10-21T17:38:33.369215+0000","last_undegraded":"2023-10-21T17:38:33.369215+0000","last_fullsized":"2023-10-21T17:38:33.369215+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:40.907600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357130+0000","last_change":"2023-10-21T17:38:33.355761+0000","last_active":"2023-10-21T17:38:33.357130+0000","last_peered":"2023-10-21T17:38:33.357130+0000","last_clean":"2023-10-21T17:38:33.357130+0000","last_became_active":"2023-10-21T17:38:33.355480+0000","last_became_peered":"2023-10-21T17:38:33.355480+0000","last_unstale":"2023-10-21T17:38:33.357130+0000","last_undegraded":"2023-10-21T17:38:33.357130+0000","last_fullsized":"2023-10-21T17:38:33.357130+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:23:08.024962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352523+0000","last_change":"2023-10-21T17:38:33.346563+0000","last_active":"2023-10-21T17:38:33.352523+0000","last_peered":"2023-10-21T17:38:33.352523+0000","last_clean":"2023-10-21T17:38:33.352523+0000","last_became_active":"2023-10-21T17:38:33.343191+0000","last_became_peered":"2023-10-21T17:38:33.343191+0000","last_unstale":"2023-10-21T17:38:33.352523+0000","last_undegraded":"2023-10-21T17:38:33.352523+0000","last_fullsized":"2023-10-21T17:38:33.352523+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:07.673155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356407+0000","last_change":"2023-10-21T17:38:33.352394+0000","last_active":"2023-10-21T17:38:33.356407+0000","last_peered":"2023-10-21T17:38:33.356407+0000","last_clean":"2023-10-21T17:38:33.356407+0000","last_became_active":"2023-10-21T17:38:33.352048+0000","last_became_peered":"2023-10-21T17:38:33.352048+0000","last_unstale":"2023-10-21T17:38:33.356407+0000","last_undegraded":"2023-10-21T17:38:33.356407+0000","last_fullsized":"2023-10-21T17:38:33.356407+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:37.232213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364739+0000","last_change":"2023-10-21T17:38:33.362478+0000","last_active":"2023-10-21T17:38:33.364739+0000","last_peered":"2023-10-21T17:38:33.364739+0000","last_clean":"2023-10-21T17:38:33.364739+0000","last_became_active":"2023-10-21T17:38:33.361294+0000","last_became_peered":"2023-10-21T17:38:33.361294+0000","last_unstale":"2023-10-21T17:38:33.364739+0000","last_undegraded":"2023-10-21T17:38:33.364739+0000","last_fullsized":"2023-10-21T17:38:33.364739+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370068+0000","last_change":"2023-10-21T17:38:33.369586+0000","last_active":"2023-10-21T17:38:33.370068+0000","last_peered":"2023-10-21T17:38:33.370068+0000","last_clean":"2023-10-21T17:38:33.370068+0000","last_became_active":"2023-10-21T17:38:33.369208+0000","last_became_peered":"2023-10-21T17:38:33.369208+0000","last_unstale":"2023-10-21T17:38:33.370068+0000","last_undegraded":"2023-10-21T17:38:33.370068+0000","last_fullsized":"2023-10-21T17:38:33.370068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:03.956275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357238+0000","last_change":"2023-10-21T17:38:33.349528+0000","last_active":"2023-10-21T17:38:33.357238+0000","last_peered":"2023-10-21T17:38:33.357238+0000","last_clean":"2023-10-21T17:38:33.357238+0000","last_became_active":"2023-10-21T17:38:33.349070+0000","last_became_peered":"2023-10-21T17:38:33.349070+0000","last_unstale":"2023-10-21T17:38:33.357238+0000","last_undegraded":"2023-10-21T17:38:33.357238+0000","last_fullsized":"2023-10-21T17:38:33.357238+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:49:59.616089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359991+0000","last_change":"2023-10-21T17:38:33.354704+0000","last_active":"2023-10-21T17:38:33.359991+0000","last_peered":"2023-10-21T17:38:33.359991+0000","last_clean":"2023-10-21T17:38:33.359991+0000","last_became_active":"2023-10-21T17:38:33.354428+0000","last_became_peered":"2023-10-21T17:38:33.354428+0000","last_unstale":"2023-10-21T17:38:33.359991+0000","last_undegraded":"2023-10-21T17:38:33.359991+0000","last_fullsized":"2023-10-21T17:38:33.359991+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:17.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.346497+0000","last_change":"2023-10-21T17:38:33.340905+0000","last_active":"2023-10-21T17:38:33.346497+0000","last_peered":"2023-10-21T17:38:33.346497+0000","last_clean":"2023-10-21T17:38:33.346497+0000","last_became_active":"2023-10-21T17:38:33.340134+0000","last_became_peered":"2023-10-21T17:38:33.340134+0000","last_unstale":"2023-10-21T17:38:33.346497+0000","last_undegraded":"2023-10-21T17:38:33.346497+0000","last_fullsized":"2023-10-21T17:38:33.346497+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:40.907600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.355791+0000","last_change":"2023-10-21T17:38:33.353725+0000","last_active":"2023-10-21T17:38:33.355791+0000","last_peered":"2023-10-21T17:38:33.355791+0000","last_clean":"2023-10-21T17:38:33.355791+0000","last_became_active":"2023-10-21T17:38:33.353441+0000","last_became_peered":"2023-10-21T17:38:33.353441+0000","last_unstale":"2023-10-21T17:38:33.355791+0000","last_undegraded":"2023-10-21T17:38:33.355791+0000","last_fullsized":"2023-10-21T17:38:33.355791+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:30.342934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357356+0000","last_change":"2023-10-21T17:38:33.354963+0000","last_active":"2023-10-21T17:38:33.357356+0000","last_peered":"2023-10-21T17:38:33.357356+0000","last_clean":"2023-10-21T17:38:33.357356+0000","last_became_active":"2023-10-21T17:38:33.354183+0000","last_became_peered":"2023-10-21T17:38:33.354183+0000","last_unstale":"2023-10-21T17:38:33.357356+0000","last_undegraded":"2023-10-21T17:38:33.357356+0000","last_fullsized":"2023-10-21T17:38:33.357356+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:54.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.353860+0000","last_change":"2023-10-21T17:38:33.350258+0000","last_active":"2023-10-21T17:38:33.353860+0000","last_peered":"2023-10-21T17:38:33.353860+0000","last_clean":"2023-10-21T17:38:33.353860+0000","last_became_active":"2023-10-21T17:38:33.349849+0000","last_became_peered":"2023-10-21T17:38:33.349849+0000","last_unstale":"2023-10-21T17:38:33.353860+0000","last_undegraded":"2023-10-21T17:38:33.353860+0000","last_fullsized":"2023-10-21T17:38:33.353860+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:06:07.548870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"232'4","reported_seq":15,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.416255+0000","last_change":"2023-10-21T17:38:32.419554+0000","last_active":"2023-10-21T17:38:33.416255+0000","last_peered":"2023-10-21T17:38:33.416255+0000","last_clean":"2023-10-21T17:38:33.416255+0000","last_became_active":"2023-10-21T17:38:32.419276+0000","last_became_peered":"2023-10-21T17:38:32.419276+0000","last_unstale":"2023-10-21T17:38:33.416255+0000","last_undegraded":"2023-10-21T17:38:33.416255+0000","last_fullsized":"2023-10-21T17:38:33.416255+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:44.216819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359685+0000","last_change":"2023-10-21T17:38:33.352486+0000","last_active":"2023-10-21T17:38:33.359685+0000","last_peered":"2023-10-21T17:38:33.359685+0000","last_clean":"2023-10-21T17:38:33.359685+0000","last_became_active":"2023-10-21T17:38:33.352150+0000","last_became_peered":"2023-10-21T17:38:33.352150+0000","last_unstale":"2023-10-21T17:38:33.359685+0000","last_undegraded":"2023-10-21T17:38:33.359685+0000","last_fullsized":"2023-10-21T17:38:33.359685+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:31:32.268023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412028+0000","last_change":"2023-10-21T17:38:32.401420+0000","last_active":"2023-10-21T17:38:33.412028+0000","last_peered":"2023-10-21T17:38:33.412028+0000","last_clean":"2023-10-21T17:38:33.412028+0000","last_became_active":"2023-10-21T17:38:32.394692+0000","last_became_peered":"2023-10-21T17:38:32.394692+0000","last_unstale":"2023-10-21T17:38:33.412028+0000","last_undegraded":"2023-10-21T17:38:33.412028+0000","last_fullsized":"2023-10-21T17:38:33.412028+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363087+0000","last_change":"2023-10-21T17:38:33.360687+0000","last_active":"2023-10-21T17:38:33.363087+0000","last_peered":"2023-10-21T17:38:33.363087+0000","last_clean":"2023-10-21T17:38:33.363087+0000","last_became_active":"2023-10-21T17:38:33.360369+0000","last_became_peered":"2023-10-21T17:38:33.360369+0000","last_unstale":"2023-10-21T17:38:33.363087+0000","last_undegraded":"2023-10-21T17:38:33.363087+0000","last_fullsized":"2023-10-21T17:38:33.363087+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:36.794962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414885+0000","last_change":"2023-10-21T17:38:32.405671+0000","last_active":"2023-10-21T17:38:33.414885+0000","last_peered":"2023-10-21T17:38:33.414885+0000","last_clean":"2023-10-21T17:38:33.414885+0000","last_became_active":"2023-10-21T17:38:32.404175+0000","last_became_peered":"2023-10-21T17:38:32.404175+0000","last_unstale":"2023-10-21T17:38:33.414885+0000","last_undegraded":"2023-10-21T17:38:33.414885+0000","last_fullsized":"2023-10-21T17:38:33.414885+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:24.818705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356938+0000","last_change":"2023-10-21T17:38:33.350688+0000","last_active":"2023-10-21T17:38:33.356938+0000","last_peered":"2023-10-21T17:38:33.356938+0000","last_clean":"2023-10-21T17:38:33.356938+0000","last_became_active":"2023-10-21T17:38:33.350383+0000","last_became_peered":"2023-10-21T17:38:33.350383+0000","last_unstale":"2023-10-21T17:38:33.356938+0000","last_undegraded":"2023-10-21T17:38:33.356938+0000","last_fullsized":"2023-10-21T17:38:33.356938+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:52:30.144459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412423+0000","last_change":"2023-10-21T17:38:32.407056+0000","last_active":"2023-10-21T17:38:33.412423+0000","last_peered":"2023-10-21T17:38:33.412423+0000","last_clean":"2023-10-21T17:38:33.412423+0000","last_became_active":"2023-10-21T17:38:32.405548+0000","last_became_peered":"2023-10-21T17:38:32.405548+0000","last_unstale":"2023-10-21T17:38:33.412423+0000","last_undegraded":"2023-10-21T17:38:33.412423+0000","last_fullsized":"2023-10-21T17:38:33.412423+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:02.940331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.346830+0000","last_change":"2023-10-21T17:38:33.341123+0000","last_active":"2023-10-21T17:38:33.346830+0000","last_peered":"2023-10-21T17:38:33.346830+0000","last_clean":"2023-10-21T17:38:33.346830+0000","last_became_active":"2023-10-21T17:38:33.340397+0000","last_became_peered":"2023-10-21T17:38:33.340397+0000","last_unstale":"2023-10-21T17:38:33.346830+0000","last_undegraded":"2023-10-21T17:38:33.346830+0000","last_fullsized":"2023-10-21T17:38:33.346830+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:28.700422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412231+0000","last_change":"2023-10-21T17:38:32.411640+0000","last_active":"2023-10-21T17:38:33.412231+0000","last_peered":"2023-10-21T17:38:33.412231+0000","last_clean":"2023-10-21T17:38:33.412231+0000","last_became_active":"2023-10-21T17:38:32.411258+0000","last_became_peered":"2023-10-21T17:38:32.411258+0000","last_unstale":"2023-10-21T17:38:33.412231+0000","last_undegraded":"2023-10-21T17:38:33.412231+0000","last_fullsized":"2023-10-21T17:38:33.412231+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.349513+0000","last_change":"2023-10-21T17:38:33.346021+0000","last_active":"2023-10-21T17:38:33.349513+0000","last_peered":"2023-10-21T17:38:33.349513+0000","last_clean":"2023-10-21T17:38:33.349513+0000","last_became_active":"2023-10-21T17:38:33.345731+0000","last_became_peered":"2023-10-21T17:38:33.345731+0000","last_unstale":"2023-10-21T17:38:33.349513+0000","last_undegraded":"2023-10-21T17:38:33.349513+0000","last_fullsized":"2023-10-21T17:38:33.349513+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:12.488140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342731+0000","last_change":"2023-10-21T17:38:32.411637+0000","last_active":"2023-10-21T17:38:33.342731+0000","last_peered":"2023-10-21T17:38:33.342731+0000","last_clean":"2023-10-21T17:38:33.342731+0000","last_became_active":"2023-10-21T17:38:32.411322+0000","last_became_peered":"2023-10-21T17:38:32.411322+0000","last_unstale":"2023-10-21T17:38:33.342731+0000","last_undegraded":"2023-10-21T17:38:33.342731+0000","last_fullsized":"2023-10-21T17:38:33.342731+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.616148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361932+0000","last_change":"2023-10-21T17:38:33.357688+0000","last_active":"2023-10-21T17:38:33.361932+0000","last_peered":"2023-10-21T17:38:33.361932+0000","last_clean":"2023-10-21T17:38:33.361932+0000","last_became_active":"2023-10-21T17:38:33.355135+0000","last_became_peered":"2023-10-21T17:38:33.355135+0000","last_unstale":"2023-10-21T17:38:33.361932+0000","last_undegraded":"2023-10-21T17:38:33.361932+0000","last_fullsized":"2023-10-21T17:38:33.361932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:10.598342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.337259+0000","last_change":"2023-10-21T17:38:32.410852+0000","last_active":"2023-10-21T17:38:33.337259+0000","last_peered":"2023-10-21T17:38:33.337259+0000","last_clean":"2023-10-21T17:38:33.337259+0000","last_became_active":"2023-10-21T17:38:32.410413+0000","last_became_peered":"2023-10-21T17:38:32.410413+0000","last_unstale":"2023-10-21T17:38:33.337259+0000","last_undegraded":"2023-10-21T17:38:33.337259+0000","last_fullsized":"2023-10-21T17:38:33.337259+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:26.544773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352012+0000","last_change":"2023-10-21T17:38:33.349513+0000","last_active":"2023-10-21T17:38:33.352012+0000","last_peered":"2023-10-21T17:38:33.352012+0000","last_clean":"2023-10-21T17:38:33.352012+0000","last_became_active":"2023-10-21T17:38:33.349202+0000","last_became_peered":"2023-10-21T17:38:33.349202+0000","last_unstale":"2023-10-21T17:38:33.352012+0000","last_undegraded":"2023-10-21T17:38:33.352012+0000","last_fullsized":"2023-10-21T17:38:33.352012+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:39.055132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339591+0000","last_change":"2023-10-21T17:38:32.417638+0000","last_active":"2023-10-21T17:38:33.339591+0000","last_peered":"2023-10-21T17:38:33.339591+0000","last_clean":"2023-10-21T17:38:33.339591+0000","last_became_active":"2023-10-21T17:38:32.417302+0000","last_became_peered":"2023-10-21T17:38:32.417302+0000","last_unstale":"2023-10-21T17:38:33.339591+0000","last_undegraded":"2023-10-21T17:38:33.339591+0000","last_fullsized":"2023-10-21T17:38:33.339591+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:22.089541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362346+0000","last_change":"2023-10-21T17:38:33.359536+0000","last_active":"2023-10-21T17:38:33.362346+0000","last_peered":"2023-10-21T17:38:33.362346+0000","last_clean":"2023-10-21T17:38:33.362346+0000","last_became_active":"2023-10-21T17:38:33.358634+0000","last_became_peered":"2023-10-21T17:38:33.358634+0000","last_unstale":"2023-10-21T17:38:33.362346+0000","last_undegraded":"2023-10-21T17:38:33.362346+0000","last_fullsized":"2023-10-21T17:38:33.362346+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:13:31.532929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.341764+0000","last_change":"2023-10-21T17:38:32.414100+0000","last_active":"2023-10-21T17:38:33.341764+0000","last_peered":"2023-10-21T17:38:33.341764+0000","last_clean":"2023-10-21T17:38:33.341764+0000","last_became_active":"2023-10-21T17:38:32.413503+0000","last_became_peered":"2023-10-21T17:38:32.413503+0000","last_unstale":"2023-10-21T17:38:33.341764+0000","last_undegraded":"2023-10-21T17:38:33.341764+0000","last_fullsized":"2023-10-21T17:38:33.341764+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:41.928628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362310+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:33.362310+0000","last_peered":"2023-10-21T17:38:33.362310+0000","last_clean":"2023-10-21T17:38:33.362310+0000","last_became_active":"2023-10-21T17:38:33.361078+0000","last_became_peered":"2023-10-21T17:38:33.361078+0000","last_unstale":"2023-10-21T17:38:33.362310+0000","last_undegraded":"2023-10-21T17:38:33.362310+0000","last_fullsized":"2023-10-21T17:38:33.362310+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:07.568107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"232'3","reported_seq":14,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.413926+0000","last_change":"2023-10-21T17:38:32.404797+0000","last_active":"2023-10-21T17:38:33.413926+0000","last_peered":"2023-10-21T17:38:33.413926+0000","last_clean":"2023-10-21T17:38:33.413926+0000","last_became_active":"2023-10-21T17:38:32.404524+0000","last_became_peered":"2023-10-21T17:38:32.404524+0000","last_unstale":"2023-10-21T17:38:33.413926+0000","last_undegraded":"2023-10-21T17:38:33.413926+0000","last_fullsized":"2023-10-21T17:38:33.413926+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:07.928480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.363441+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:33.363441+0000","last_peered":"2023-10-21T17:38:33.363441+0000","last_clean":"2023-10-21T17:38:33.363441+0000","last_became_active":"2023-10-21T17:38:33.361077+0000","last_became_peered":"2023-10-21T17:38:33.361077+0000","last_unstale":"2023-10-21T17:38:33.363441+0000","last_undegraded":"2023-10-21T17:38:33.363441+0000","last_fullsized":"2023-10-21T17:38:33.363441+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:34:17.601212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"232'6","reported_seq":17,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.417661+0000","last_change":"2023-10-21T17:38:32.416726+0000","last_active":"2023-10-21T17:38:33.417661+0000","last_peered":"2023-10-21T17:38:33.417661+0000","last_clean":"2023-10-21T17:38:33.417661+0000","last_became_active":"2023-10-21T17:38:32.416448+0000","last_became_peered":"2023-10-21T17:38:32.416448+0000","last_unstale":"2023-10-21T17:38:33.417661+0000","last_undegraded":"2023-10-21T17:38:33.417661+0000","last_fullsized":"2023-10-21T17:38:33.417661+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352903+0000","last_change":"2023-10-21T17:38:33.347261+0000","last_active":"2023-10-21T17:38:33.352903+0000","last_peered":"2023-10-21T17:38:33.352903+0000","last_clean":"2023-10-21T17:38:33.352903+0000","last_became_active":"2023-10-21T17:38:33.346927+0000","last_became_peered":"2023-10-21T17:38:33.346927+0000","last_unstale":"2023-10-21T17:38:33.352903+0000","last_undegraded":"2023-10-21T17:38:33.352903+0000","last_fullsized":"2023-10-21T17:38:33.352903+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:32.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415763+0000","last_change":"2023-10-21T17:38:32.405640+0000","last_active":"2023-10-21T17:38:33.415763+0000","last_peered":"2023-10-21T17:38:33.415763+0000","last_clean":"2023-10-21T17:38:33.415763+0000","last_became_active":"2023-10-21T17:38:32.404457+0000","last_became_peered":"2023-10-21T17:38:32.404457+0000","last_unstale":"2023-10-21T17:38:33.415763+0000","last_undegraded":"2023-10-21T17:38:33.415763+0000","last_fullsized":"2023-10-21T17:38:33.415763+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:26:53.247310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.353654+0000","last_change":"2023-10-21T17:38:33.351625+0000","last_active":"2023-10-21T17:38:33.353654+0000","last_peered":"2023-10-21T17:38:33.353654+0000","last_clean":"2023-10-21T17:38:33.353654+0000","last_became_active":"2023-10-21T17:38:33.351311+0000","last_became_peered":"2023-10-21T17:38:33.351311+0000","last_unstale":"2023-10-21T17:38:33.353654+0000","last_undegraded":"2023-10-21T17:38:33.353654+0000","last_fullsized":"2023-10-21T17:38:33.353654+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:17.648970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342820+0000","last_change":"2023-10-21T17:38:32.403441+0000","last_active":"2023-10-21T17:38:33.342820+0000","last_peered":"2023-10-21T17:38:33.342820+0000","last_clean":"2023-10-21T17:38:33.342820+0000","last_became_active":"2023-10-21T17:38:32.403179+0000","last_became_peered":"2023-10-21T17:38:32.403179+0000","last_unstale":"2023-10-21T17:38:33.342820+0000","last_undegraded":"2023-10-21T17:38:33.342820+0000","last_fullsized":"2023-10-21T17:38:33.342820+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:38.273786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361610+0000","last_change":"2023-10-21T17:38:33.358210+0000","last_active":"2023-10-21T17:38:33.361610+0000","last_peered":"2023-10-21T17:38:33.361610+0000","last_clean":"2023-10-21T17:38:33.361610+0000","last_became_active":"2023-10-21T17:38:33.357911+0000","last_became_peered":"2023-10-21T17:38:33.357911+0000","last_unstale":"2023-10-21T17:38:33.361610+0000","last_undegraded":"2023-10-21T17:38:33.361610+0000","last_fullsized":"2023-10-21T17:38:33.361610+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:04.246844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342740+0000","last_change":"2023-10-21T17:38:32.405540+0000","last_active":"2023-10-21T17:38:33.342740+0000","last_peered":"2023-10-21T17:38:33.342740+0000","last_clean":"2023-10-21T17:38:33.342740+0000","last_became_active":"2023-10-21T17:38:32.405267+0000","last_became_peered":"2023-10-21T17:38:32.405267+0000","last_unstale":"2023-10-21T17:38:33.342740+0000","last_undegraded":"2023-10-21T17:38:33.342740+0000","last_fullsized":"2023-10-21T17:38:33.342740+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:15.824449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364775+0000","last_change":"2023-10-21T17:38:33.362626+0000","last_active":"2023-10-21T17:38:33.364775+0000","last_peered":"2023-10-21T17:38:33.364775+0000","last_clean":"2023-10-21T17:38:33.364775+0000","last_became_active":"2023-10-21T17:38:33.361673+0000","last_became_peered":"2023-10-21T17:38:33.361673+0000","last_unstale":"2023-10-21T17:38:33.364775+0000","last_undegraded":"2023-10-21T17:38:33.364775+0000","last_fullsized":"2023-10-21T17:38:33.364775+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:20.548414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"232'8","reported_seq":19,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415469+0000","last_change":"2023-10-21T17:38:32.411814+0000","last_active":"2023-10-21T17:38:33.415469+0000","last_peered":"2023-10-21T17:38:33.415469+0000","last_clean":"2023-10-21T17:38:33.415469+0000","last_became_active":"2023-10-21T17:38:32.411496+0000","last_became_peered":"2023-10-21T17:38:32.411496+0000","last_unstale":"2023-10-21T17:38:33.415469+0000","last_undegraded":"2023-10-21T17:38:33.415469+0000","last_fullsized":"2023-10-21T17:38:33.415469+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:17.401857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359495+0000","last_change":"2023-10-21T17:38:33.357202+0000","last_active":"2023-10-21T17:38:33.359495+0000","last_peered":"2023-10-21T17:38:33.359495+0000","last_clean":"2023-10-21T17:38:33.359495+0000","last_became_active":"2023-10-21T17:38:33.356898+0000","last_became_peered":"2023-10-21T17:38:33.356898+0000","last_unstale":"2023-10-21T17:38:33.359495+0000","last_undegraded":"2023-10-21T17:38:33.359495+0000","last_fullsized":"2023-10-21T17:38:33.359495+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:14.017632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.412095+0000","last_change":"2023-10-21T17:38:32.404935+0000","last_active":"2023-10-21T17:38:33.412095+0000","last_peered":"2023-10-21T17:38:33.412095+0000","last_clean":"2023-10-21T17:38:33.412095+0000","last_became_active":"2023-10-21T17:38:32.404139+0000","last_became_peered":"2023-10-21T17:38:32.404139+0000","last_unstale":"2023-10-21T17:38:33.412095+0000","last_undegraded":"2023-10-21T17:38:33.412095+0000","last_fullsized":"2023-10-21T17:38:33.412095+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:53.425393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361440+0000","last_change":"2023-10-21T17:38:33.358480+0000","last_active":"2023-10-21T17:38:33.361440+0000","last_peered":"2023-10-21T17:38:33.361440+0000","last_clean":"2023-10-21T17:38:33.361440+0000","last_became_active":"2023-10-21T17:38:33.358208+0000","last_became_peered":"2023-10-21T17:38:33.358208+0000","last_unstale":"2023-10-21T17:38:33.361440+0000","last_undegraded":"2023-10-21T17:38:33.361440+0000","last_fullsized":"2023-10-21T17:38:33.361440+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:24.254117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414671+0000","last_change":"2023-10-21T17:38:32.413129+0000","last_active":"2023-10-21T17:38:33.414671+0000","last_peered":"2023-10-21T17:38:33.414671+0000","last_clean":"2023-10-21T17:38:33.414671+0000","last_became_active":"2023-10-21T17:38:32.410406+0000","last_became_peered":"2023-10-21T17:38:32.410406+0000","last_unstale":"2023-10-21T17:38:33.414671+0000","last_undegraded":"2023-10-21T17:38:33.414671+0000","last_fullsized":"2023-10-21T17:38:33.414671+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:30.006550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352283+0000","last_change":"2023-10-21T17:38:33.343136+0000","last_active":"2023-10-21T17:38:33.352283+0000","last_peered":"2023-10-21T17:38:33.352283+0000","last_clean":"2023-10-21T17:38:33.352283+0000","last_became_active":"2023-10-21T17:38:33.342344+0000","last_became_peered":"2023-10-21T17:38:33.342344+0000","last_unstale":"2023-10-21T17:38:33.352283+0000","last_undegraded":"2023-10-21T17:38:33.352283+0000","last_fullsized":"2023-10-21T17:38:33.352283+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:28.150954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.415181+0000","last_change":"2023-10-21T17:38:32.406815+0000","last_active":"2023-10-21T17:38:33.415181+0000","last_peered":"2023-10-21T17:38:33.415181+0000","last_clean":"2023-10-21T17:38:33.415181+0000","last_became_active":"2023-10-21T17:38:32.406525+0000","last_became_peered":"2023-10-21T17:38:32.406525+0000","last_unstale":"2023-10-21T17:38:33.415181+0000","last_undegraded":"2023-10-21T17:38:33.415181+0000","last_fullsized":"2023-10-21T17:38:33.415181+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:01.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369442+0000","last_change":"2023-10-21T17:38:33.366719+0000","last_active":"2023-10-21T17:38:33.369442+0000","last_peered":"2023-10-21T17:38:33.369442+0000","last_clean":"2023-10-21T17:38:33.369442+0000","last_became_active":"2023-10-21T17:38:33.366173+0000","last_became_peered":"2023-10-21T17:38:33.366173+0000","last_unstale":"2023-10-21T17:38:33.369442+0000","last_undegraded":"2023-10-21T17:38:33.369442+0000","last_fullsized":"2023-10-21T17:38:33.369442+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:05.602488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.335348+0000","last_change":"2023-10-21T17:38:32.403394+0000","last_active":"2023-10-21T17:38:33.335348+0000","last_peered":"2023-10-21T17:38:33.335348+0000","last_clean":"2023-10-21T17:38:33.335348+0000","last_became_active":"2023-10-21T17:38:32.403104+0000","last_became_peered":"2023-10-21T17:38:32.403104+0000","last_unstale":"2023-10-21T17:38:33.335348+0000","last_undegraded":"2023-10-21T17:38:33.335348+0000","last_fullsized":"2023-10-21T17:38:33.335348+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:06:56.765497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.352926+0000","last_change":"2023-10-21T17:38:33.348917+0000","last_active":"2023-10-21T17:38:33.352926+0000","last_peered":"2023-10-21T17:38:33.352926+0000","last_clean":"2023-10-21T17:38:33.352926+0000","last_became_active":"2023-10-21T17:38:33.348522+0000","last_became_peered":"2023-10-21T17:38:33.348522+0000","last_unstale":"2023-10-21T17:38:33.352926+0000","last_undegraded":"2023-10-21T17:38:33.352926+0000","last_fullsized":"2023-10-21T17:38:33.352926+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:11:42.293794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.411796+0000","last_change":"2023-10-21T17:38:32.401853+0000","last_active":"2023-10-21T17:38:33.411796+0000","last_peered":"2023-10-21T17:38:33.411796+0000","last_clean":"2023-10-21T17:38:33.411796+0000","last_became_active":"2023-10-21T17:38:32.401567+0000","last_became_peered":"2023-10-21T17:38:32.401567+0000","last_unstale":"2023-10-21T17:38:33.411796+0000","last_undegraded":"2023-10-21T17:38:33.411796+0000","last_fullsized":"2023-10-21T17:38:33.411796+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:24.460794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357777+0000","last_change":"2023-10-21T17:38:33.354105+0000","last_active":"2023-10-21T17:38:33.357777+0000","last_peered":"2023-10-21T17:38:33.357777+0000","last_clean":"2023-10-21T17:38:33.357777+0000","last_became_active":"2023-10-21T17:38:33.353821+0000","last_became_peered":"2023-10-21T17:38:33.353821+0000","last_unstale":"2023-10-21T17:38:33.357777+0000","last_undegraded":"2023-10-21T17:38:33.357777+0000","last_fullsized":"2023-10-21T17:38:33.357777+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:53.603618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.324780+0000","last_change":"2023-10-21T17:38:32.399236+0000","last_active":"2023-10-21T17:38:33.324780+0000","last_peered":"2023-10-21T17:38:33.324780+0000","last_clean":"2023-10-21T17:38:33.324780+0000","last_became_active":"2023-10-21T17:38:32.398909+0000","last_became_peered":"2023-10-21T17:38:32.398909+0000","last_unstale":"2023-10-21T17:38:33.324780+0000","last_undegraded":"2023-10-21T17:38:33.324780+0000","last_fullsized":"2023-10-21T17:38:33.324780+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:04.398168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356416+0000","last_change":"2023-10-21T17:38:33.353848+0000","last_active":"2023-10-21T17:38:33.356416+0000","last_peered":"2023-10-21T17:38:33.356416+0000","last_clean":"2023-10-21T17:38:33.356416+0000","last_became_active":"2023-10-21T17:38:33.351959+0000","last_became_peered":"2023-10-21T17:38:33.351959+0000","last_unstale":"2023-10-21T17:38:33.356416+0000","last_undegraded":"2023-10-21T17:38:33.356416+0000","last_fullsized":"2023-10-21T17:38:33.356416+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:27.560717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.329328+0000","last_change":"2023-10-21T17:38:32.416515+0000","last_active":"2023-10-21T17:38:33.329328+0000","last_peered":"2023-10-21T17:38:33.329328+0000","last_clean":"2023-10-21T17:38:33.329328+0000","last_became_active":"2023-10-21T17:38:32.415411+0000","last_became_peered":"2023-10-21T17:38:32.415411+0000","last_unstale":"2023-10-21T17:38:33.329328+0000","last_undegraded":"2023-10-21T17:38:33.329328+0000","last_fullsized":"2023-10-21T17:38:33.329328+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:24.896102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.368689+0000","last_change":"2023-10-21T17:38:33.363577+0000","last_active":"2023-10-21T17:38:33.368689+0000","last_peered":"2023-10-21T17:38:33.368689+0000","last_clean":"2023-10-21T17:38:33.368689+0000","last_became_active":"2023-10-21T17:38:33.360945+0000","last_became_peered":"2023-10-21T17:38:33.360945+0000","last_unstale":"2023-10-21T17:38:33.368689+0000","last_undegraded":"2023-10-21T17:38:33.368689+0000","last_fullsized":"2023-10-21T17:38:33.368689+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:10.470691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342027+0000","last_change":"2023-10-21T17:38:32.410021+0000","last_active":"2023-10-21T17:38:33.342027+0000","last_peered":"2023-10-21T17:38:33.342027+0000","last_clean":"2023-10-21T17:38:33.342027+0000","last_became_active":"2023-10-21T17:38:32.409751+0000","last_became_peered":"2023-10-21T17:38:32.409751+0000","last_unstale":"2023-10-21T17:38:33.342027+0000","last_undegraded":"2023-10-21T17:38:33.342027+0000","last_fullsized":"2023-10-21T17:38:33.342027+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.367191+0000","last_change":"2023-10-21T17:38:33.361024+0000","last_active":"2023-10-21T17:38:33.367191+0000","last_peered":"2023-10-21T17:38:33.367191+0000","last_clean":"2023-10-21T17:38:33.367191+0000","last_became_active":"2023-10-21T17:38:33.360523+0000","last_became_peered":"2023-10-21T17:38:33.360523+0000","last_unstale":"2023-10-21T17:38:33.367191+0000","last_undegraded":"2023-10-21T17:38:33.367191+0000","last_fullsized":"2023-10-21T17:38:33.367191+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:25.476738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.414330+0000","last_change":"2023-10-21T17:38:32.418683+0000","last_active":"2023-10-21T17:38:33.414330+0000","last_peered":"2023-10-21T17:38:33.414330+0000","last_clean":"2023-10-21T17:38:33.414330+0000","last_became_active":"2023-10-21T17:38:32.418325+0000","last_became_peered":"2023-10-21T17:38:32.418325+0000","last_unstale":"2023-10-21T17:38:33.414330+0000","last_undegraded":"2023-10-21T17:38:33.414330+0000","last_fullsized":"2023-10-21T17:38:33.414330+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:36.165616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361867+0000","last_change":"2023-10-21T17:38:33.358506+0000","last_active":"2023-10-21T17:38:33.361867+0000","last_peered":"2023-10-21T17:38:33.361867+0000","last_clean":"2023-10-21T17:38:33.361867+0000","last_became_active":"2023-10-21T17:38:33.357627+0000","last_became_peered":"2023-10-21T17:38:33.357627+0000","last_unstale":"2023-10-21T17:38:33.361867+0000","last_undegraded":"2023-10-21T17:38:33.361867+0000","last_fullsized":"2023-10-21T17:38:33.361867+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:31.691088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.358242+0000","last_change":"2023-10-21T17:38:32.416259+0000","last_active":"2023-10-21T17:38:33.358242+0000","last_peered":"2023-10-21T17:38:33.358242+0000","last_clean":"2023-10-21T17:38:33.358242+0000","last_became_active":"2023-10-21T17:38:32.415430+0000","last_became_peered":"2023-10-21T17:38:32.415430+0000","last_unstale":"2023-10-21T17:38:33.358242+0000","last_undegraded":"2023-10-21T17:38:33.358242+0000","last_fullsized":"2023-10-21T17:38:33.358242+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.361426+0000","last_change":"2023-10-21T17:38:33.358411+0000","last_active":"2023-10-21T17:38:33.361426+0000","last_peered":"2023-10-21T17:38:33.361426+0000","last_clean":"2023-10-21T17:38:33.361426+0000","last_became_active":"2023-10-21T17:38:33.358101+0000","last_became_peered":"2023-10-21T17:38:33.358101+0000","last_unstale":"2023-10-21T17:38:33.361426+0000","last_undegraded":"2023-10-21T17:38:33.361426+0000","last_fullsized":"2023-10-21T17:38:33.361426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:11:57.303927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.328786+0000","last_change":"2023-10-21T17:38:32.405560+0000","last_active":"2023-10-21T17:38:33.328786+0000","last_peered":"2023-10-21T17:38:33.328786+0000","last_clean":"2023-10-21T17:38:33.328786+0000","last_became_active":"2023-10-21T17:38:32.405233+0000","last_became_peered":"2023-10-21T17:38:32.405233+0000","last_unstale":"2023-10-21T17:38:33.328786+0000","last_undegraded":"2023-10-21T17:38:33.328786+0000","last_fullsized":"2023-10-21T17:38:33.328786+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.359581+0000","last_change":"2023-10-21T17:38:33.351368+0000","last_active":"2023-10-21T17:38:33.359581+0000","last_peered":"2023-10-21T17:38:33.359581+0000","last_clean":"2023-10-21T17:38:33.359581+0000","last_became_active":"2023-10-21T17:38:33.350963+0000","last_became_peered":"2023-10-21T17:38:33.350963+0000","last_unstale":"2023-10-21T17:38:33.359581+0000","last_undegraded":"2023-10-21T17:38:33.359581+0000","last_fullsized":"2023-10-21T17:38:33.359581+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339970+0000","last_change":"2023-10-21T17:38:32.405103+0000","last_active":"2023-10-21T17:38:33.339970+0000","last_peered":"2023-10-21T17:38:33.339970+0000","last_clean":"2023-10-21T17:38:33.339970+0000","last_became_active":"2023-10-21T17:38:32.404811+0000","last_became_peered":"2023-10-21T17:38:32.404811+0000","last_unstale":"2023-10-21T17:38:33.339970+0000","last_undegraded":"2023-10-21T17:38:33.339970+0000","last_fullsized":"2023-10-21T17:38:33.339970+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:19.532470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.369189+0000","last_change":"2023-10-21T17:38:33.364849+0000","last_active":"2023-10-21T17:38:33.369189+0000","last_peered":"2023-10-21T17:38:33.369189+0000","last_clean":"2023-10-21T17:38:33.369189+0000","last_became_active":"2023-10-21T17:38:33.363791+0000","last_became_peered":"2023-10-21T17:38:33.363791+0000","last_unstale":"2023-10-21T17:38:33.369189+0000","last_undegraded":"2023-10-21T17:38:33.369189+0000","last_fullsized":"2023-10-21T17:38:33.369189+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:39:31.802251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.357148+0000","last_change":"2023-10-21T17:38:32.419011+0000","last_active":"2023-10-21T17:38:33.357148+0000","last_peered":"2023-10-21T17:38:33.357148+0000","last_clean":"2023-10-21T17:38:33.357148+0000","last_became_active":"2023-10-21T17:38:32.418410+0000","last_became_peered":"2023-10-21T17:38:32.418410+0000","last_unstale":"2023-10-21T17:38:33.357148+0000","last_undegraded":"2023-10-21T17:38:33.357148+0000","last_fullsized":"2023-10-21T17:38:33.357148+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:22.056635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.364223+0000","last_change":"2023-10-21T17:38:33.362147+0000","last_active":"2023-10-21T17:38:33.364223+0000","last_peered":"2023-10-21T17:38:33.364223+0000","last_clean":"2023-10-21T17:38:33.364223+0000","last_became_active":"2023-10-21T17:38:33.360339+0000","last_became_peered":"2023-10-21T17:38:33.360339+0000","last_unstale":"2023-10-21T17:38:33.364223+0000","last_undegraded":"2023-10-21T17:38:33.364223+0000","last_fullsized":"2023-10-21T17:38:33.364223+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:22.754068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.342621+0000","last_change":"2023-10-21T17:38:32.408205+0000","last_active":"2023-10-21T17:38:33.342621+0000","last_peered":"2023-10-21T17:38:33.342621+0000","last_clean":"2023-10-21T17:38:33.342621+0000","last_became_active":"2023-10-21T17:38:32.407638+0000","last_became_peered":"2023-10-21T17:38:32.407638+0000","last_unstale":"2023-10-21T17:38:33.342621+0000","last_undegraded":"2023-10-21T17:38:33.342621+0000","last_fullsized":"2023-10-21T17:38:33.342621+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:20.909728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.356919+0000","last_change":"2023-10-21T17:38:33.354071+0000","last_active":"2023-10-21T17:38:33.356919+0000","last_peered":"2023-10-21T17:38:33.356919+0000","last_clean":"2023-10-21T17:38:33.356919+0000","last_became_active":"2023-10-21T17:38:33.353746+0000","last_became_peered":"2023-10-21T17:38:33.353746+0000","last_unstale":"2023-10-21T17:38:33.356919+0000","last_undegraded":"2023-10-21T17:38:33.356919+0000","last_fullsized":"2023-10-21T17:38:33.356919+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.339062+0000","last_change":"2023-10-21T17:38:32.411663+0000","last_active":"2023-10-21T17:38:33.339062+0000","last_peered":"2023-10-21T17:38:33.339062+0000","last_clean":"2023-10-21T17:38:33.339062+0000","last_became_active":"2023-10-21T17:38:32.411409+0000","last_became_peered":"2023-10-21T17:38:32.411409+0000","last_unstale":"2023-10-21T17:38:33.339062+0000","last_undegraded":"2023-10-21T17:38:33.339062+0000","last_fullsized":"2023-10-21T17:38:33.339062+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:00.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.362507+0000","last_change":"2023-10-21T17:38:33.360420+0000","last_active":"2023-10-21T17:38:33.362507+0000","last_peered":"2023-10-21T17:38:33.362507+0000","last_clean":"2023-10-21T17:38:33.362507+0000","last_became_active":"2023-10-21T17:38:33.360143+0000","last_became_peered":"2023-10-21T17:38:33.360143+0000","last_unstale":"2023-10-21T17:38:33.362507+0000","last_undegraded":"2023-10-21T17:38:33.362507+0000","last_fullsized":"2023-10-21T17:38:33.362507+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:08.944424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.333508+0000","last_change":"2023-10-21T17:38:32.411805+0000","last_active":"2023-10-21T17:38:33.333508+0000","last_peered":"2023-10-21T17:38:33.333508+0000","last_clean":"2023-10-21T17:38:33.333508+0000","last_became_active":"2023-10-21T17:38:32.411439+0000","last_became_peered":"2023-10-21T17:38:32.411439+0000","last_unstale":"2023-10-21T17:38:33.333508+0000","last_undegraded":"2023-10-21T17:38:33.333508+0000","last_fullsized":"2023-10-21T17:38:33.333508+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:57.270863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.370689+0000","last_change":"2023-10-21T17:38:33.370135+0000","last_active":"2023-10-21T17:38:33.370689+0000","last_peered":"2023-10-21T17:38:33.370689+0000","last_clean":"2023-10-21T17:38:33.370689+0000","last_became_active":"2023-10-21T17:38:33.369798+0000","last_became_peered":"2023-10-21T17:38:33.369798+0000","last_unstale":"2023-10-21T17:38:33.370689+0000","last_undegraded":"2023-10-21T17:38:33.370689+0000","last_fullsized":"2023-10-21T17:38:33.370689+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:42.944572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-10-21T17:38:33.411940+0000","last_change":"2023-10-21T17:38:32.397140+0000","last_active":"2023-10-21T17:38:33.411940+0000","last_peered":"2023-10-21T17:38:33.411940+0000","last_clean":"2023-10-21T17:38:33.411940+0000","last_became_active":"2023-10-21T17:38:32.396837+0000","last_became_peered":"2023-10-21T17:38:32.396837+0000","last_unstale":"2023-10-21T17:38:33.411940+0000","last_undegraded":"2023-10-21T17:38:33.411940+0000","last_fullsized":"2023-10-21T17:38:33.411940+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:17.875367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"221'4873","reported_seq":5802,"reported_epoch":231,"state":"active+clean","last_fresh":"2023-10-21T17:38:32.400433+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:38:32.400433+0000","last_peered":"2023-10-21T17:38:32.400433+0000","last_clean":"2023-10-21T17:38:32.400433+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:38:32.400433+0000","last_undegraded":"2023-10-21T17:38:32.400433+0000","last_fullsized":"2023-10-21T17:38:32.400433+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4873,"log_dups_size":0,"ondisk_log_size":4873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4873,"ondisk_log_size":4873,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575997,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70708,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301132,"statfs":{"total":96636764160,"available":96564359168,"internally_reserved":0,"allocated":3506176,"data_stored":2098554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":68876747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.97399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"max":{"1min":1.6419999999999999,"5min":1.841,"15min":2.2549999999999999},"last":0.998},{"interface":"front","average":{"1min":1.117,"5min":1.101,"15min":1.1599999999999999},"min":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.724,"15min":1.7669999999999999},"last":1.268}]},{"osd":1,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1739999999999999,"15min":1.218},"min":{"1min":0.96299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.589,"5min":1.7170000000000001,"15min":1.73},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.262,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.91200000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.506,"5min":1.792,"15min":1.885},"last":1.1799999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.466},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.86899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2909999999999999,"15min":1.3180000000000001},"min":{"1min":0.998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.472,"5min":2.472,"15min":2.5110000000000001},"last":1.139},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.486,"15min":1.4670000000000001},"min":{"1min":1.23,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.3120000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.383,"15min":1.357},"min":{"1min":0.84599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.5129999999999999,"15min":1.496},"min":{"1min":1.1619999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.173,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.54},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.474,"15min":1.45},"min":{"1min":1.135,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":1.5169999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.587,"15min":1.5469999999999999},"min":{"1min":0.85499999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.3100000000000001,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.488,"15min":1.476},"min":{"1min":0.88800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.3600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42336,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329504,"statfs":{"total":96636764160,"available":96593412096,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1659999999999999,"15min":1.198},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.728,"5min":1.944,"15min":1.944},"last":1.27},{"interface":"front","average":{"1min":1.256,"5min":1.2050000000000001,"15min":1.222},"min":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.643,"5min":1.7090000000000001,"15min":1.762},"last":0.97399999999999998}]},{"osd":1,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.171,"15min":1.1919999999999999},"min":{"1min":0.84099999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6379999999999999},"last":1.123},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2989999999999999,"15min":1.319},"min":{"1min":0.98599999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.782,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.2070000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.272,"15min":1.3169999999999999},"min":{"1min":1.006,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5649999999999999,"5min":1.712,"15min":1.8300000000000001},"last":1.028},{"interface":"front","average":{"1min":1.393,"5min":1.3089999999999999,"15min":1.3069999999999999},"min":{"1min":1.1799999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.611,"5min":1.679,"15min":1.929},"last":0.93600000000000005}]},{"osd":3,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3919999999999999,"15min":1.417},"min":{"1min":0.90700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.593,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.427,"15min":1.4650000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.90500000000000003},"max":{"1min":1.825,"5min":1.9039999999999999,"15min":2.048},"last":1.1659999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":1.704,"5min":1.77,"15min":1.9950000000000001},"last":1.321},{"interface":"front","average":{"1min":1.444,"5min":1.4079999999999999,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.3859999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5349999999999999,"15min":1.571},"min":{"1min":1.274,"5min":1.1000000000000001,"15min":1.0029999999999999},"max":{"1min":1.891,"5min":1.956,"15min":2.339},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.325,"15min":1.3759999999999999},"min":{"1min":0.98199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6870000000000001,"5min":1.796,"15min":2.2890000000000001},"last":1.796}]},{"osd":7,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4970000000000001,"15min":1.536},"min":{"1min":1.1080000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.762,"5min":1.9339999999999999,"15min":2.1200000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.641,"5min":1.6259999999999999,"15min":1.6679999999999999},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.0449999999999999},"max":{"1min":1.9570000000000001,"5min":2.1499999999999999,"15min":2.5409999999999999},"last":1.4830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575999,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60384,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311456,"statfs":{"total":96636764160,"available":96574930944,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.81599999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6319999999999999,"5min":1.6799999999999999,"15min":1.9930000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.64200000000000002,"15min":0.627},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.206}]},{"osd":1,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":1.012,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.738,"5min":1.806,"15min":1.8620000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.202,"15min":1.2330000000000001},"min":{"1min":0.88500000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.75,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.88600000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.2829999999999999,"15min":1.302},"min":{"1min":1.087,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.802},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.8600000000000001,"15min":1.891},"last":1.27}]},{"osd":3,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.325,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.732,"5min":1.829,"15min":1.9650000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.4079999999999999,"15min":1.4199999999999999},"min":{"1min":1.038,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.772,"5min":1.881,"15min":1.9139999999999999},"last":1.012}]},{"osd":4,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2230000000000001,"15min":1.25},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.462,"5min":1.7050000000000001,"15min":1.9910000000000001},"last":1.381},{"interface":"front","average":{"1min":1.46,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":1.1259999999999999,"5min":0.85399999999999998,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":1.877,"15min":1.9399999999999999},"last":1.589}]},{"osd":6,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.482},"min":{"1min":1.081,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8540000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.508,"15min":1.524},"min":{"1min":0.84999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7509999999999999,"5min":1.9730000000000001,"15min":2.52},"last":1.6659999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5309999999999999,"15min":1.5329999999999999},"min":{"1min":1.1319999999999999,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":1.96,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.544,"15min":1.5640000000000001},"min":{"1min":1.018,"5min":0.68400000000000005,"15min":0.56399999999999995},"max":{"1min":1.833,"5min":2.0750000000000002,"15min":2.1349999999999998},"last":1.631}]}]},{"osd":4,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47920,"kb_used_data":2844,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323920,"statfs":{"total":96636764160,"available":96587694080,"internally_reserved":0,"allocated":2912256,"data_stored":1508186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.117,"15min":1.1599999999999999},"min":{"1min":0.72299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.8480000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.659,"5min":1.855,"15min":1.869},"last":1.3819999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.58099999999999996},"max":{"1min":1.4890000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.321},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.248,"15min":1.268},"min":{"1min":0.93300000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.665,"5min":1.762,"15min":1.762},"last":1.4139999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2649999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":1.71,"15min":1.7609999999999999},"last":1.131},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3620000000000001,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5740000000000001,"5min":1.762,"15min":1.762},"last":1.4870000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2729999999999999,"15min":1.3200000000000001},"min":{"1min":0.92100000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4870000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":1.284},{"interface":"front","average":{"1min":1.387,"5min":1.419,"15min":1.425},"min":{"1min":1.002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7350000000000001,"5min":1.825,"15min":1.9239999999999999},"last":1.2450000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.82699999999999996},"max":{"1min":1.639,"5min":1.8340000000000001,"15min":1.867},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.98799999999999999,"5min":0.879,"15min":0.81000000000000005},"max":{"1min":1.6990000000000001,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":1.0760000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4139999999999999,"15min":1.4430000000000001},"min":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.825,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.595,"15min":1.595},"min":{"1min":1.147,"5min":1.147,"15min":1.133},"max":{"1min":1.7210000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.5469999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.534,"15min":1.5189999999999999},"min":{"1min":1.2090000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.833,"5min":2.234,"15min":2.234},"last":1.171},{"interface":"front","average":{"1min":1.55,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.0880000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9279999999999999,"5min":1.984,"15min":2.081},"last":1.4379999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56224,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315616,"statfs":{"total":96636764160,"available":96579190784,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98399999999999999},"min":{"1min":0.54400000000000004,"5min":0.48899999999999999,"15min":0.432},"max":{"1min":1.4550000000000001,"5min":1.6439999999999999,"15min":1.71},"last":1.169},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.73499999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.639}]},{"osd":1,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0409999999999999,"15min":1.091},"min":{"1min":0.626,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3340000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.875,"5min":1.879,"15min":1.879},"last":1.6870000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1100000000000001,"15min":1.1160000000000001},"min":{"1min":0.59699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.921,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.075},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.823,"5min":1.853,"15min":2},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4350000000000001,"15min":1.4410000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.218},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4710000000000001},"min":{"1min":1.048,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.802,"5min":1.9430000000000001,"15min":2.169},"last":1.4590000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.89200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9330000000000001,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.401},{"interface":"front","average":{"1min":1.534,"5min":1.5389999999999999,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.1800000000000002,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.8600000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.546,"15min":1.601},"min":{"1min":1.0189999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.966,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.593,"15min":1.625},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0579999999999998,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.573}]},{"osd":7,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.135,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2789999999999999,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.659,"15min":1.714},"min":{"1min":1.0529999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1760000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.3480000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52076,"kb_used_data":2848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319764,"statfs":{"total":96636764160,"available":96583438336,"internally_reserved":0,"allocated":2916352,"data_stored":1508276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.90400000000000003},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.149,"5min":1.841,"15min":1.841},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.79500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.7529999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.032}]},{"osd":1,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.78900000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6539999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.538,"5min":1.417,"15min":1.395},"min":{"1min":1.151,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.8300000000000001,"5min":1.958,"15min":2.4060000000000001},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.1970000000000001,"15min":1.198},"min":{"1min":0.78400000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.1760000000000002},"last":1.095},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.92900000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":2.077},"last":1.369}]},{"osd":4,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":1.1870000000000001,"5min":1.0529999999999999,"15min":1.044},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1680000000000001},"last":1.536},{"interface":"front","average":{"1min":1.522,"5min":1.468,"15min":1.46},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0329999999999999},"max":{"1min":1.9079999999999999,"5min":1.978,"15min":2.5939999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4279999999999999,"15min":1.4199999999999999},"min":{"1min":1.0800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.613},{"interface":"front","average":{"1min":1.615,"5min":1.6000000000000001,"15min":1.6120000000000001},"min":{"1min":1.286,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":1.9710000000000001,"5min":1.972,"15min":2.3759999999999999},"last":1.762}]},{"osd":6,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.571,"15min":1.575},"min":{"1min":1.145,"5min":1.1060000000000001,"15min":1.095},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.694},{"interface":"front","average":{"1min":1.595,"5min":1.5349999999999999,"15min":1.5289999999999999},"min":{"1min":1.115,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5049999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.651,"15min":1.647},"min":{"1min":1.337,"5min":1.1080000000000001,"15min":1.0780000000000001},"max":{"1min":2.1240000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5980000000000001,"15min":1.6040000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.732}]}]},{"osd":1,"up_from":13,"seq":55834575997,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52064,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319776,"statfs":{"total":96636764160,"available":96583450624,"internally_reserved":0,"allocated":2899968,"data_stored":1500975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3120000000000001,"5min":1.367,"15min":1.976},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.147,"15min":1.1619999999999999},"min":{"1min":0.81200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.679,"5min":1.833,"15min":1.8400000000000001},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.046,"15min":1.048},"min":{"1min":0.83399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5369999999999999,"5min":1.6990000000000001,"15min":1.79},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.238,"15min":1.252},"min":{"1min":1.121,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6100000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.397},"min":{"1min":1.0780000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8620000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.498,"5min":1.28,"15min":1.304},"min":{"1min":0.98899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.9339999999999999},"last":0.73899999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3600000000000001,"15min":1.3580000000000001},"min":{"1min":1.0820000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.643,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.575,"5min":1.498,"15min":1.504},"min":{"1min":1.157,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.867,"5min":2.0750000000000002,"15min":2.383},"last":1.0269999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.202,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.732,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.524,"15min":1.5429999999999999},"min":{"1min":1.1639999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.8240000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.147}]},{"osd":6,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.6080000000000001,"15min":1.6060000000000001},"min":{"1min":1.2729999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.085,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.173},{"interface":"front","average":{"1min":1.556,"5min":1.552,"15min":1.5629999999999999},"min":{"1min":1.2390000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.752,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.073}]},{"osd":7,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.653,"15min":1.6799999999999999},"min":{"1min":1.298,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.994,"5min":2.1829999999999998,"15min":2.548},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.738,"15min":1.7270000000000001},"min":{"1min":1.401,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1739999999999999,"5min":2.492,"15min":2.548},"last":1.087}]}]},{"osd":0,"up_from":13,"seq":55834575997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52640,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319200,"statfs":{"total":96636764160,"available":96582860800,"internally_reserved":0,"allocated":3493888,"data_stored":2091343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93400000000000005,"15min":0.95199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2989999999999999,"5min":1.514,"15min":1.774},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.036,"15min":1.101},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.506,"5min":1.556,"15min":1.774},"last":1.1950000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.121,"15min":1.171},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1140000000000001,"15min":1.181},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.708,"5min":1.708,"15min":1.851},"last":1.014}]},{"osd":3,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0960000000000001,"15min":1.133},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6259999999999999,"5min":1.897,"15min":1.9590000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.266,"5min":1.327,"15min":1.391},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.97499999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.353,"15min":1.4299999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8169999999999999,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.137},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5580000000000001,"15min":1.599},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.0620000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.401,"15min":1.4059999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.772,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.448,"5min":1.51,"15min":1.573},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.1150000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.847,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.611,"15min":1.6759999999999999},"min":{"1min":0.999,"5min":0.996,"15min":0.996},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":1.1559999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4890000000000001,"15min":1.5449999999999999},"min":{"1min":0.82799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8380000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.5900000000000001,"15min":1.623},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.952,"5min":2.2519999999999998,"15min":2.262},"last":1.0840000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:38:38.464 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:38:38.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:38:38.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.739+0000 7ff3fe3b3640 1 Processor -- start 2023-10-21T17:38:38.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.740+0000 7ff3fe3b3640 1 -- start start 2023-10-21T17:38:38.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.741+0000 7ff3fe3b3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3f80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.741+0000 7ff3fe3b3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f80cd580 con 0x7ff3f80d3380 2023-10-21T17:38:38.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.741+0000 7ff3fe3b3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f80cd820 con 0x7ff3f80d1540 2023-10-21T17:38:38.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.741+0000 7ff3fe3b3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80ce1c0 con 0x7ff3f815cd80 2023-10-21T17:38:38.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.742+0000 7ff3fcbb0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff3f80d3380 0x7ff3f80d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46642/0 (socket says 172.21.15.53:46642) 2023-10-21T17:38:38.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.742+0000 7ff3fcbb0640 1 -- 172.21.15.53:0/1298645025 learned_addr learned my addr 172.21.15.53:0/1298645025 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:38.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.742+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274994399 0 0) 0x7ff3f80cd580 con 0x7ff3f80d3380 2023-10-21T17:38:38.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.742+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e4003520 con 0x7ff3f80d3380 2023-10-21T17:38:38.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.742+0000 7ff3fd3b1640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3f80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993407560 0 0) 0x7ff3e4003520 con 0x7ff3f80d3380 2023-10-21T17:38:38.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 msgr2=0x7ff3f80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:38.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3f80ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 >> v1:172.21.15.124:6789/0 conn(0x7ff3f80d1540 legacy=0x7ff3f806afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80cdb00 con 0x7ff3f80d3380 2023-10-21T17:38:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff3e8001670 con 0x7ff3f80d3380 2023-10-21T17:38:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 >> v1:172.21.15.53:6789/0 conn(0x7ff3f80d3380 legacy=0x7ff3f80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3e40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e4004020 0x7ff3e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e4004d00 0x7ff3e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80ce1c0 con 0x7ff3e4004d00 2023-10-21T17:38:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e400bbc0 con 0x7ff3e4004020 2023-10-21T17:38:38.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e400bd80 con 0x7ff3f815cd80 2023-10-21T17:38:38.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e4004d00 0x7ff3e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.743+0000 7ff3fd3b1640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3e40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fcbb0640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e4004020 0x7ff3e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 msgr2=0x7ff3e40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:38.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f815cd80 0x7ff3e40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e4004020 msgr2=0x7ff3e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:38.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e4004020 0x7ff3e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/1298645025 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4003520 con 0x7ff3e4004d00 2023-10-21T17:38:38.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.744+0000 7ff3fcbb0640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3e4004020 0x7ff3e40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:38.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.745+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e4004d00 0x7ff3e4005180 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001e50 tx=0x7ff3e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c74c7c27bfcb1c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.745+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003b70 con 0x7ff3e4004d00 2023-10-21T17:38:38.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.745+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0003cd0 con 0x7ff3e4004d00 2023-10-21T17:38:38.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.745+0000 7ff3effff640 1 -- 172.21.15.53:0/1298645025 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0005310 con 0x7ff3e4004d00 2023-10-21T17:38:38.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/1298645025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e4004d00 msgr2=0x7ff3e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/1298645025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3e4004d00 0x7ff3e4005180 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001e50 tx=0x7ff3e40057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/1298645025 reap_dead start 2023-10-21T17:38:38.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/1298645025 shutdown_connections 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/1298645025 >> 172.21.15.53:0/1298645025 conn(0x7ff3f809bcf0 msgr2=0x7ff3f80d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/1298645025 shutdown_connections 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.746+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/1298645025 wait complete. 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.748+0000 7ff3fe3b3640 1 Processor -- start 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.748+0000 7ff3fe3b3640 1 -- start start 2023-10-21T17:38:38.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e4004d00 0x7ff3f81f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 0x7ff3f815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f815fb10 con 0x7ff3f81519d0 2023-10-21T17:38:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f80c7620 con 0x7ff3f815ffa0 2023-10-21T17:38:38.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.749+0000 7ff3fe3b3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f81f55a0 con 0x7ff3e4004d00 2023-10-21T17:38:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49748/0 (socket says 172.21.15.53:49748) 2023-10-21T17:38:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/425249329 learned_addr learned my addr 172.21.15.53:0/425249329 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fd3b1640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e4004d00 0x7ff3f81f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fcbb0640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 0x7ff3f815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e4004d00 msgr2=0x7ff3f81f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e4004d00 0x7ff3f81f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 msgr2=0x7ff3f815ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:38.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 0x7ff3f815ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:38.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 -- 172.21.15.53:0/425249329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f815f830 con 0x7ff3f81519d0 2023-10-21T17:38:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fcbb0640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 0x7ff3f815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.750+0000 7ff3fdbb2640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001f70 tx=0x7ff3f00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.751+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f003e070 con 0x7ff3f81519d0 2023-10-21T17:38:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.751+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0003e40 con 0x7ff3f81519d0 2023-10-21T17:38:38.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.751+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0006d70 con 0x7ff3f81519d0 2023-10-21T17:38:38.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.751+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f80c78c0 con 0x7ff3f81519d0 2023-10-21T17:38:38.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.751+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f815f470 con 0x7ff3f81519d0 2023-10-21T17:38:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.752+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f80c7de0 con 0x7ff3f81519d0 2023-10-21T17:38:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.753+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff3f0007030 con 0x7ff3f81519d0 2023-10-21T17:38:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.754+0000 7ff3edffb640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 0x7ff3dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:38.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.754+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff3f00aacb0 con 0x7ff3f81519d0 2023-10-21T17:38:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.755+0000 7ff3fd3b1640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 0x7ff3dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.756+0000 7ff3fd3b1640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 0x7ff3dc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3f815d680 tx=0x7ff3f4008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:38.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:38.759+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3f80c7de0 con 0x7ff3f81519d0 2023-10-21T17:38:39.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.023+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff3f80c7de0 con 0x7ff3f81519d0 2023-10-21T17:38:39.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.027+0000 7ff3edffb640 1 -- 172.21.15.53:0/425249329 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff3f80c7de0 con 0x7ff3f81519d0 2023-10-21T17:38:39.029 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:38:39.029 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:38:39.029 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:38:39.029 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:38:39.029 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:38:39.030 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:38:39.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 msgr2=0x7ff3dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 0x7ff3dc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3f815d680 tx=0x7ff3f4008040 comp rx=0 tx=0).stop 2023-10-21T17:38:39.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 msgr2=0x7ff3f80b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0001f70 tx=0x7ff3f00014a0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 shutdown_connections 2023-10-21T17:38:39.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3f81519d0 0x7ff3f80b4830 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f815ffa0 0x7ff3f815ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7ff3dc06af90 0x7ff3dc06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 --2- 172.21.15.53:0/425249329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3e4004d00 0x7ff3f81f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 >> 172.21.15.53:0/425249329 conn(0x7ff3f809bcf0 msgr2=0x7ff3f80b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:39.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 shutdown_connections 2023-10-21T17:38:39.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.031+0000 7ff3fe3b3640 1 -- 172.21.15.53:0/425249329 wait complete. 2023-10-21T17:38:39.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:39.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.277+0000 7f30c48ba640 1 Processor -- start 2023-10-21T17:38:39.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.277+0000 7f30c48ba640 1 -- start start 2023-10-21T17:38:39.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.278+0000 7f30c48ba640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30c0150440 0x7f30c0071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.278+0000 7f30c48ba640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f30c00c97d0 0x7f30c015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.278+0000 7f30c48ba640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c015d7d0 con 0x7f30c015d380 2023-10-21T17:38:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.278+0000 7f30c48ba640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c0071550 con 0x7f30c0150440 2023-10-21T17:38:39.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.278+0000 7f30c48ba640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c00c9f90 con 0x7f30c00c97d0 2023-10-21T17:38:39.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30c0150440 0x7f30c0071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f30c0150440 0x7f30c0071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49750/0 (socket says 172.21.15.53:49750) 2023-10-21T17:38:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 -- 172.21.15.53:0/300978377 learned_addr learned my addr 172.21.15.53:0/300978377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 -- 172.21.15.53:0/300978377 >> v2:172.21.15.124:3300/0 conn(0x7f30c00c97d0 msgr2=0x7f30c015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30b659c640 1 --2- 172.21.15.53:0/300978377 >> v2:172.21.15.124:3300/0 conn(0x7f30c00c97d0 0x7f30c015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 --2- 172.21.15.53:0/300978377 >> v2:172.21.15.124:3300/0 conn(0x7f30c00c97d0 0x7f30c015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 -- 172.21.15.53:0/300978377 >> v1:172.21.15.124:6789/0 conn(0x7f30c015d380 legacy=0x7f30c0159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:39.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.279+0000 7f30bed9d640 1 -- 172.21.15.53:0/300978377 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c00ca270 con 0x7f30c0150440 2023-10-21T17:38:39.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.281+0000 7f30bed9d640 1 --2- 172.21.15.53:0/300978377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c0150440 0x7f30c0071010 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b0fc0 tx=0x7f30b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac486a9cdb94ed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:39.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.281+0000 7f30be59c640 1 -- 172.21.15.53:0/300978377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0031870 con 0x7f30c0150440 2023-10-21T17:38:39.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.281+0000 7f30be59c640 1 -- 172.21.15.53:0/300978377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b00319d0 con 0x7f30c0150440 2023-10-21T17:38:39.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.281+0000 7f30be59c640 1 -- 172.21.15.53:0/300978377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0031d60 con 0x7f30c0150440 2023-10-21T17:38:39.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 -- 172.21.15.53:0/300978377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c0150440 msgr2=0x7f30c0071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 --2- 172.21.15.53:0/300978377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c0150440 0x7f30c0071010 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b0fc0 tx=0x7f30b002b910 comp rx=0 tx=0).stop 2023-10-21T17:38:39.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 -- 172.21.15.53:0/300978377 shutdown_connections 2023-10-21T17:38:39.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 --2- 172.21.15.53:0/300978377 >> v2:172.21.15.124:3300/0 conn(0x7f30c00c97d0 0x7f30c015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 --2- 172.21.15.53:0/300978377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c0150440 0x7f30c0071010 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 -- 172.21.15.53:0/300978377 >> 172.21.15.53:0/300978377 conn(0x7f30c00cb0c0 msgr2=0x7f30c00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 -- 172.21.15.53:0/300978377 shutdown_connections 2023-10-21T17:38:39.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.282+0000 7f30c48ba640 1 -- 172.21.15.53:0/300978377 wait complete. 2023-10-21T17:38:39.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.284+0000 7f30c48ba640 1 Processor -- start 2023-10-21T17:38:39.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.284+0000 7f30c48ba640 1 -- start start 2023-10-21T17:38:39.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c00c97d0 0x7f30c01f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c0150440 0x7f30c014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01533d0 con 0x7f30c015d380 2023-10-21T17:38:39.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01f19a0 con 0x7f30c0150440 2023-10-21T17:38:39.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30c48ba640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c01f47a0 con 0x7f30c00c97d0 2023-10-21T17:38:39.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30bf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30bf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49760/0 (socket says 172.21.15.53:49760) 2023-10-21T17:38:39.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.285+0000 7f30bf59e640 1 -- 172.21.15.53:0/2132861959 learned_addr learned my addr 172.21.15.53:0/2132861959 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:39.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bed9d640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c00c97d0 0x7f30c01f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30b659c640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c0150440 0x7f30c014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 -- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c00c97d0 msgr2=0x7f30c01f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c00c97d0 0x7f30c01f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 -- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c0150440 msgr2=0x7f30c014eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c0150440 0x7f30c014eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 -- 172.21.15.53:0/2132861959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b00094d0 con 0x7f30c015d380 2023-10-21T17:38:39.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30bf59e640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c780 tx=0x7f30b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:39.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.286+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8004b60 con 0x7f30c015d380 2023-10-21T17:38:39.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.287+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b800ce90 con 0x7f30c015d380 2023-10-21T17:38:39.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.287+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b800a850 con 0x7f30c015d380 2023-10-21T17:38:39.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.287+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c0153880 con 0x7f30c015d380 2023-10-21T17:38:39.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.287+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c01f4150 con 0x7f30c015d380 2023-10-21T17:38:39.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.288+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c01f4950 con 0x7f30c015d380 2023-10-21T17:38:39.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.289+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30b800a9b0 con 0x7f30c015d380 2023-10-21T17:38:39.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.290+0000 7f30b7fff640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 0x7f30a006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:39.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.290+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f30b8089b20 con 0x7f30c015d380 2023-10-21T17:38:39.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.290+0000 7f30bed9d640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 0x7f30a006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:39.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.292+0000 7f30bed9d640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 0x7f30a006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30b0036000 tx=0x7f30b003c040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:39.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.295+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30c01f4950 con 0x7f30c015d380 2023-10-21T17:38:39.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.527+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30c00ca800 con 0x7f30c015d380 2023-10-21T17:38:39.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.529+0000 7f30b7fff640 1 -- 172.21.15.53:0/2132861959 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4928 (secure 0 0 0) 0x7f30b8054860 con 0x7f30c015d380 2023-10-21T17:38:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:39.530 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":113,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-21T17:38:39.531 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 113 2023-10-21T17:38:39.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 msgr2=0x7f30a006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 0x7f30a006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f30b0036000 tx=0x7f30b003c040 comp rx=0 tx=0).stop 2023-10-21T17:38:39.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 msgr2=0x7f30c00c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:39.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f30b800c780 tx=0x7f30b800cc40 comp rx=0 tx=0).stop 2023-10-21T17:38:39.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 shutdown_connections 2023-10-21T17:38:39.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30c015d380 0x7f30c00c3c20 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c0150440 0x7f30c014eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f30a006afc0 0x7f30a006d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 --2- 172.21.15.53:0/2132861959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c00c97d0 0x7f30c01f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:39.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.531+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 >> 172.21.15.53:0/2132861959 conn(0x7f30c00cb0c0 msgr2=0x7f30c014fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:39.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.532+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 shutdown_connections 2023-10-21T17:38:39.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:39.532+0000 7f30c48ba640 1 -- 172.21.15.53:0/2132861959 wait complete. 2023-10-21T17:38:39.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:38:32.860415+0000', 'modified': '2023-10-21T17:38:35.943962+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 37943}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_37943': {'gid': 37943, 'name': 'd', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/748765178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 748765178}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 748765178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:38:39.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:38:39.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-21T17:38:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.193+0000 7f62a6185640 1 Processor -- start 2023-10-21T17:38:40.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.194+0000 7f62a6185640 1 -- start start 2023-10-21T17:38:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.195+0000 7f62a6185640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f62a015a820 0x7f62a00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.195+0000 7f62a6185640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a00cab60 con 0x7f62a00ca680 2023-10-21T17:38:40.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.195+0000 7f62a6185640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a00cb960 con 0x7f62a015ac00 2023-10-21T17:38:40.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.195+0000 7f62a6185640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a00cbdf0 con 0x7f62a015a820 2023-10-21T17:38:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a4982640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f62a00ca680 0x7f62a00c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46658/0 (socket says 172.21.15.53:46658) 2023-10-21T17:38:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a4982640 1 -- 172.21.15.53:0/426985862 learned_addr learned my addr 172.21.15.53:0/426985862 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:40.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a5183640 1 --2- 172.21.15.53:0/426985862 >> v2:172.21.15.53:3300/0 conn(0x7f62a015a820 0x7f62a00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f6297fff640 1 -- 172.21.15.53:0/426985862 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763357878 0 0) 0x7f62a00cab60 con 0x7f62a00ca680 2023-10-21T17:38:40.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f6297fff640 1 -- 172.21.15.53:0/426985862 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f628c003520 con 0x7f62a00ca680 2023-10-21T17:38:40.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a5183640 1 -- 172.21.15.53:0/426985862 >> v1:172.21.15.124:6789/0 conn(0x7f62a015ac00 legacy=0x7f62a00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a5183640 1 -- 172.21.15.53:0/426985862 >> v1:172.21.15.53:6789/0 conn(0x7f62a00ca680 legacy=0x7f62a00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.196+0000 7f62a5183640 1 -- 172.21.15.53:0/426985862 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00c9f20 con 0x7f62a015a820 2023-10-21T17:38:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f62a5183640 1 --2- 172.21.15.53:0/426985862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a00aff70 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f629c0097a0 tx=0x7f629c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482206e3435d7734 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f6297fff640 1 -- 172.21.15.53:0/426985862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c02e070 con 0x7f62a015a820 2023-10-21T17:38:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f6297fff640 1 -- 172.21.15.53:0/426985862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c0315a0 con 0x7f62a015a820 2023-10-21T17:38:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f6297fff640 1 -- 172.21.15.53:0/426985862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c031930 con 0x7f62a015a820 2023-10-21T17:38:40.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f62a6185640 1 -- 172.21.15.53:0/426985862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 msgr2=0x7f62a00aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.198+0000 7f62a6185640 1 --2- 172.21.15.53:0/426985862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a00aff70 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f629c0097a0 tx=0x7f629c02b840 comp rx=0 tx=0).stop 2023-10-21T17:38:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.199+0000 7f62a6185640 1 -- 172.21.15.53:0/426985862 shutdown_connections 2023-10-21T17:38:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.199+0000 7f62a6185640 1 --2- 172.21.15.53:0/426985862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a00aff70 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.199+0000 7f62a6185640 1 -- 172.21.15.53:0/426985862 >> 172.21.15.53:0/426985862 conn(0x7f62a006c050 msgr2=0x7f62a006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.199+0000 7f62a6185640 1 -- 172.21.15.53:0/426985862 shutdown_connections 2023-10-21T17:38:40.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.199+0000 7f62a6185640 1 -- 172.21.15.53:0/426985862 wait complete. 2023-10-21T17:38:40.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.201+0000 7f62a6185640 1 Processor -- start 2023-10-21T17:38:40.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.201+0000 7f62a6185640 1 -- start start 2023-10-21T17:38:40.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62a00ca680 0x7f62a014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62a015ac00 0x7f62a00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0150070 con 0x7f62a015a820 2023-10-21T17:38:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a01f0960 con 0x7f62a015ac00 2023-10-21T17:38:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a6185640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00c8530 con 0x7f62a00ca680 2023-10-21T17:38:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a4982640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a4982640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49784/0 (socket says 172.21.15.53:49784) 2023-10-21T17:38:40.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a4982640 1 -- 172.21.15.53:0/2685985540 learned_addr learned my addr 172.21.15.53:0/2685985540 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a5984640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62a015ac00 0x7f62a00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.202+0000 7f62a5183640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62a00ca680 0x7f62a014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 -- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62a00ca680 msgr2=0x7f62a014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62a00ca680 0x7f62a014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 -- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62a015ac00 msgr2=0x7f62a00ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62a015ac00 0x7f62a00ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 -- 172.21.15.53:0/2685985540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009400 con 0x7f62a015a820 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a4982640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f629000c4d0 tx=0x7f629000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f6295ffb640 1 -- 172.21.15.53:0/2685985540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290005ce0 con 0x7f62a015a820 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f6295ffb640 1 -- 172.21.15.53:0/2685985540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6290005e40 con 0x7f62a015a820 2023-10-21T17:38:40.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f6295ffb640 1 -- 172.21.15.53:0/2685985540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290007270 con 0x7f62a015a820 2023-10-21T17:38:40.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a00c55c0 con 0x7f62a015a820 2023-10-21T17:38:40.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.203+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a00c5b80 con 0x7f62a015a820 2023-10-21T17:38:40.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.207+0000 7f6295ffb640 1 -- 172.21.15.53:0/2685985540 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6290007490 con 0x7f62a015a820 2023-10-21T17:38:40.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.208+0000 7f6295ffb640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 0x7f628406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.208+0000 7f6295ffb640 1 -- 172.21.15.53:0/2685985540 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6290088f60 con 0x7f62a015a820 2023-10-21T17:38:40.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.208+0000 7f62a9f78740 1 Processor -- start 2023-10-21T17:38:40.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.208+0000 7f62a9f78740 1 -- start start 2023-10-21T17:38:40.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.208+0000 7f62a5183640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 0x7f628406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e925d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e928d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e91797a0 con 0x55b5e925d9c0 2023-10-21T17:38:40.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e91cb740 con 0x55b5e928a960 2023-10-21T17:38:40.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.209+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5e91c8e30 con 0x55b5e928a580 2023-10-21T17:38:40.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49792/0 (socket says 172.21.15.53:49792) 2023-10-21T17:38:40.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 -- 172.21.15.53:0/3049321042 learned_addr learned my addr 172.21.15.53:0/3049321042 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5183640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 0x7f628406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f629c036000 tx=0x7f629c03d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a4982640 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e928d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 -- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 msgr2=0x55b5e925d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e925d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 -- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 msgr2=0x55b5e928d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e928d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 -- 172.21.15.53:0/3049321042 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5e9188910 con 0x55b5e925d9c0 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a4982640 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e928d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.210+0000 7f62a5984640 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f6298009f30 tx=0x7f629800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ebc9edf67e93d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f6282ffd640 1 -- 172.21.15.53:0/3049321042 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298011070 con 0x55b5e925d9c0 2023-10-21T17:38:40.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f6282ffd640 1 -- 172.21.15.53:0/3049321042 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800a5f0 con 0x55b5e925d9c0 2023-10-21T17:38:40.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f6282ffd640 1 -- 172.21.15.53:0/3049321042 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800a980 con 0x55b5e925d9c0 2023-10-21T17:38:40.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 -- 172.21.15.53:0/3049321042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 msgr2=0x55b5e928d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f6298009f30 tx=0x7f629800c890 comp rx=0 tx=0).stop 2023-10-21T17:38:40.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 -- 172.21.15.53:0/3049321042 shutdown_connections 2023-10-21T17:38:40.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e925d9c0 0x55b5e928d850 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e928d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3049321042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e925d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.211+0000 7f62a9f78740 1 -- 172.21.15.53:0/3049321042 >> 172.21.15.53:0/3049321042 conn(0x55b5e925f790 msgr2=0x55b5e92a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.212+0000 7f62a9f78740 1 -- 172.21.15.53:0/3049321042 shutdown_connections 2023-10-21T17:38:40.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.212+0000 7f62a9f78740 1 -- 172.21.15.53:0/3049321042 wait complete. 2023-10-21T17:38:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.212+0000 7f62a9f78740 1 -- start start 2023-10-21T17:38:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.213+0000 7f62a9f78740 1 Processor -- start 2023-10-21T17:38:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9288770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9287590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e91c4720 con 0x55b5e9287cd0 2023-10-21T17:38:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e9162970 con 0x55b5e928a580 2023-10-21T17:38:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5e9162610 con 0x55b5e928a960 2023-10-21T17:38:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a9f78740 10 client.0 _ll_register_callbacks cb 0x55b5e91849d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:38:40.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49798/0 (socket says 172.21.15.53:49798) 2023-10-21T17:38:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 -- 172.21.15.53:0/714110926 learned_addr learned my addr 172.21.15.53:0/714110926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5183640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9288770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 -- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 msgr2=0x55b5e9287590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9287590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 -- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 msgr2=0x55b5e9288770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9288770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5984640 1 -- 172.21.15.53:0/714110926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6298009bb0 con 0x55b5e9287cd0 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.214+0000 7f62a5183640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9288770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f62a5984640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6298000f80 tx=0x7f6298004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69baeb60367bde72 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f627bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:38:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800aaf0 con 0x55b5e9287cd0 2023-10-21T17:38:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5e91eb160 con 0x55b5e9287cd0 2023-10-21T17:38:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f629800ac50 con 0x55b5e9287cd0 2023-10-21T17:38:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629800ae10 con 0x55b5e9287cd0 2023-10-21T17:38:40.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f62a9f78740 10 client.11283 fetch_fsmap learned FSMap version 113 2023-10-21T17:38:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.215+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55b5e91c2da0 con 0x55b5e9287cd0 2023-10-21T17:38:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b5e91eb160 con 0x55b5e9287cd0 2023-10-21T17:38:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6298004040 con 0x55b5e9287cd0 2023-10-21T17:38:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 10 client.11283 fetch_fsmap finished waiting for FSMap version 113 2023-10-21T17:38:40.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 10 client.11283 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:38:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 20 client.11283 populate_metadata read hostname 'smithi053' 2023-10-21T17:38:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e92bbdc0 0x55b5e928e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 4 client.11283 mds_command: new command op to 37943 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:38:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.216+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 --> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5e9196cc0 con 0x55b5e92bbdc0 2023-10-21T17:38:40.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.217+0000 7f62a5183640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e92bbdc0 0x55b5e928e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.217+0000 7f62a5183640 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e92bbdc0 0x55b5e928e400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40784b62315a390e server_cookie=46d9ada2dec7a7be in_seq=0 out_seq=0 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.217+0000 7f627bfff640 10 client.11283 ms_handle_connect on v2:172.21.15.124:6836/748765178 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f627bfff640 1 -- 172.21.15.53:0/714110926 <== mds.0 v2:172.21.15.124:6836/748765178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b5e9196cc0 con 0x55b5e92bbdc0 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f627bfff640 10 client.11283 handle_command_reply: tid=0 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 client.11283 shutdown 2023-10-21T17:38:40.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 msgr2=0x55b5e929ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6298000f80 tx=0x7f6298004a00 comp rx=0 tx=0).stop 2023-10-21T17:38:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 shutdown_connections 2023-10-21T17:38:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e92bbdc0 0x55b5e928e400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e929ee30 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9287590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 --2- 172.21.15.53:0/714110926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9288770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f627bfff640 0 client.11283 ms_handle_reset on v2:172.21.15.124:6836/748765178 2023-10-21T17:38:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 >> 172.21.15.53:0/714110926 conn(0x55b5e925d9c0 msgr2=0x55b5e928d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 shutdown_connections 2023-10-21T17:38:40.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.220+0000 7f62a9f78740 1 -- 172.21.15.53:0/714110926 wait complete. 2023-10-21T17:38:40.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.221+0000 7f62a9f78740 20 client.11283 trim_cache size 0 max 16384 2023-10-21T17:38:40.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.237+0000 7f62a9f78740 1 Processor -- start 2023-10-21T17:38:40.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.237+0000 7f62a9f78740 1 -- start start 2023-10-21T17:38:40.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e92873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e92aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e8f28f20 con 0x55b5e9287cd0 2023-10-21T17:38:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e92a19e0 con 0x55b5e928a960 2023-10-21T17:38:40.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5e9261b30 con 0x55b5e928a580 2023-10-21T17:38:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.238+0000 7f62a5183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49808/0 (socket says 172.21.15.53:49808) 2023-10-21T17:38:40.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 -- 172.21.15.53:0/3746820003 learned_addr learned my addr 172.21.15.53:0/3746820003 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 -- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 msgr2=0x55b5e92873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e92873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 -- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 msgr2=0x55b5e92aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e92aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 -- 172.21.15.53:0/3746820003 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5e9161b80 con 0x55b5e9287cd0 2023-10-21T17:38:40.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62a5183640 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f629c031d50 tx=0x7f629c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a3d26ea40bbf59 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62837fe640 1 -- 172.21.15.53:0/3746820003 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c04a040 con 0x55b5e9287cd0 2023-10-21T17:38:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62837fe640 1 -- 172.21.15.53:0/3746820003 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c04b040 con 0x55b5e9287cd0 2023-10-21T17:38:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.239+0000 7f62837fe640 1 -- 172.21.15.53:0/3746820003 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c053300 con 0x55b5e9287cd0 2023-10-21T17:38:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 -- 172.21.15.53:0/3746820003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 msgr2=0x55b5e9264540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f629c031d50 tx=0x7f629c030000 comp rx=0 tx=0).stop 2023-10-21T17:38:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 -- 172.21.15.53:0/3746820003 shutdown_connections 2023-10-21T17:38:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a960 0x55b5e92aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a580 0x55b5e92873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 --2- 172.21.15.53:0/3746820003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9264540 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 -- 172.21.15.53:0/3746820003 >> 172.21.15.53:0/3746820003 conn(0x55b5e925d9c0 msgr2=0x55b5e92a0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 -- 172.21.15.53:0/3746820003 shutdown_connections 2023-10-21T17:38:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.240+0000 7f62a9f78740 1 -- 172.21.15.53:0/3746820003 wait complete. 2023-10-21T17:38:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.241+0000 7f62a9f78740 1 -- start start 2023-10-21T17:38:40.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.241+0000 7f62a9f78740 1 Processor -- start 2023-10-21T17:38:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9262310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9288650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e92891f0 con 0x55b5e9287cd0 2023-10-21T17:38:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5e92a0590 con 0x55b5e928a580 2023-10-21T17:38:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5e9261e80 con 0x55b5e928a960 2023-10-21T17:38:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a9f78740 10 client.0 _ll_register_callbacks cb 0x7f6298018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:38:40.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a5183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a5183640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49810/0 (socket says 172.21.15.53:49810) 2023-10-21T17:38:40.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.242+0000 7f62a5183640 1 -- 172.21.15.53:0/514387909 learned_addr learned my addr 172.21.15.53:0/514387909 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a4982640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9262310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 -- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 msgr2=0x55b5e9288650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5984640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9288650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9288650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 -- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 msgr2=0x55b5e9262310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9262310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 -- 172.21.15.53:0/514387909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a00c9f20 con 0x55b5e9287cd0 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a4982640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9262310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:40.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a5183640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098c0 tx=0x7f629c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78911f9d1d53bc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f627bfff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:38:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c04a040 con 0x55b5e9287cd0 2023-10-21T17:38:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f629c040070 con 0x55b5e9287cd0 2023-10-21T17:38:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.243+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f629800a5f0 con 0x55b5e9287cd0 2023-10-21T17:38:40.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f629c0534f0 con 0x55b5e9287cd0 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 10 client.11289 fetch_fsmap learned FSMap version 113 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55b5e9187170 con 0x55b5e9287cd0 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800a5f0 con 0x55b5e9287cd0 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f629c032880 con 0x55b5e9287cd0 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 10 client.11289 fetch_fsmap finished waiting for FSMap version 113 2023-10-21T17:38:40.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 10 client.11289 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-21T17:38:40.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 20 client.11289 populate_metadata read hostname 'smithi053' 2023-10-21T17:38:40.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e9309a90 0x55b5e9309e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 4 client.11289 mds_command: new command op to 37943 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:38:40.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.244+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 --> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5e91bd700 con 0x55b5e9309a90 2023-10-21T17:38:40.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.245+0000 7f62a4982640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e9309a90 0x55b5e9309e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a4982640 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e9309a90 0x55b5e9309e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad64c66fb0af29c7 server_cookie=4887897f528b0cb4 in_seq=0 out_seq=0 2023-10-21T17:38:40.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f627bfff640 10 client.11289 ms_handle_connect on v2:172.21.15.124:6836/748765178 2023-10-21T17:38:40.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f627bfff640 1 -- 172.21.15.53:0/514387909 <== mds.0 v2:172.21.15.124:6836/748765178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55b5e91bd700 con 0x55b5e9309a90 2023-10-21T17:38:40.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f627bfff640 10 client.11289 handle_command_reply: tid=0 2023-10-21T17:38:40.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 client.11289 shutdown 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 msgr2=0x55b5e9305fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098c0 tx=0x7f629c004730 comp rx=0 tx=0).stop 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 shutdown_connections 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55b5e928a960 0x55b5e9288650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55b5e928a580 0x55b5e9262310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.124:6836/748765178,v1:172.21.15.124:6837/748765178] conn(0x55b5e9309a90 0x55b5e9309e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 --2- 172.21.15.53:0/514387909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b5e9287cd0 0x55b5e9305fd0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.246+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 >> 172.21.15.53:0/514387909 conn(0x55b5e925d9c0 msgr2=0x55b5e92613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f627bfff640 0 client.11289 ms_handle_reset on v2:172.21.15.124:6836/748765178 2023-10-21T17:38:40.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 shutdown_connections 2023-10-21T17:38:40.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a9f78740 1 -- 172.21.15.53:0/514387909 wait complete. 2023-10-21T17:38:40.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a9f78740 20 client.11289 trim_cache size 0 max 16384 2023-10-21T17:38:40.259 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:38:40.259 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:38:40.259 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 37943, 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8461699879999998, 2023-10-21T17:38:40.260 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 113, 2023-10-21T17:38:40.261 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 233, 2023-10-21T17:38:40.261 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 233, 2023-10-21T17:38:40.261 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.772055378999999 2023-10-21T17:38:40.261 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:38:40.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 msgr2=0x7f628406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 0x7f628406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f629c036000 tx=0x7f629c03d040 comp rx=0 tx=0).stop 2023-10-21T17:38:40.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 msgr2=0x7f62a01f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.247+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f629000c4d0 tx=0x7f629000c990 comp rx=0 tx=0).stop 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 shutdown_connections 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62a015ac00 0x7f62a00ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a015a820 0x7f62a01f0420 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f628406af90 0x7f628406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 --2- 172.21.15.53:0/2685985540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62a00ca680 0x7f62a014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 >> 172.21.15.53:0/2685985540 conn(0x7f62a006c050 msgr2=0x7f62a0070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 shutdown_connections 2023-10-21T17:38:40.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.248+0000 7f62a6185640 1 -- 172.21.15.53:0/2685985540 wait complete. 2023-10-21T17:38:40.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:38:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.494+0000 7f5ac48b3640 1 Processor -- start 2023-10-21T17:38:40.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.495+0000 7f5ac48b3640 1 -- start start 2023-10-21T17:38:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5ac48b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ac015a970 0x7f5ac00cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5ac48b3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac006cfd0 con 0x7f5ac00cb990 2023-10-21T17:38:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5ac48b3640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac006d870 con 0x7f5ac006dd00 2023-10-21T17:38:40.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5ac48b3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac00cbe00 con 0x7f5ac015a970 2023-10-21T17:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5abf59e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5ac00cb990 0x7f5ac00b1400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46660/0 (socket says 172.21.15.53:46660) 2023-10-21T17:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5abed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5ac015a970 0x7f5ac00cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.496+0000 7f5abf59e640 1 -- 172.21.15.53:0/458497035 learned_addr learned my addr 172.21.15.53:0/458497035 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.497+0000 7f5abe59c640 1 -- 172.21.15.53:0/458497035 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576320768 0 0) 0x7f5ac006cfd0 con 0x7f5ac00cb990 2023-10-21T17:38:40.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.497+0000 7f5abe59c640 1 -- 172.21.15.53:0/458497035 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aac003520 con 0x7f5ac00cb990 2023-10-21T17:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.497+0000 7f5abed9d640 1 -- 172.21.15.53:0/458497035 >> v1:172.21.15.124:6789/0 conn(0x7f5ac006dd00 legacy=0x7f5ac00c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.497+0000 7f5abed9d640 1 -- 172.21.15.53:0/458497035 >> v1:172.21.15.53:6789/0 conn(0x7f5ac00cb990 legacy=0x7f5ac00b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.497+0000 7f5abed9d640 1 -- 172.21.15.53:0/458497035 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac015ad50 con 0x7f5ac015a970 2023-10-21T17:38:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.499+0000 7f5abed9d640 1 --2- 172.21.15.53:0/458497035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00cb450 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00097a0 tx=0x7f5ab002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d63e1497b2a1852 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.499+0000 7f5abe59c640 1 -- 172.21.15.53:0/458497035 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab002e070 con 0x7f5ac015a970 2023-10-21T17:38:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.499+0000 7f5abe59c640 1 -- 172.21.15.53:0/458497035 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00315a0 con 0x7f5ac015a970 2023-10-21T17:38:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.499+0000 7f5abe59c640 1 -- 172.21.15.53:0/458497035 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0031930 con 0x7f5ac015a970 2023-10-21T17:38:40.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 -- 172.21.15.53:0/458497035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 msgr2=0x7f5ac00cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/458497035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00cb450 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00097a0 tx=0x7f5ab002b840 comp rx=0 tx=0).stop 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 -- 172.21.15.53:0/458497035 shutdown_connections 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/458497035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00cb450 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 -- 172.21.15.53:0/458497035 >> 172.21.15.53:0/458497035 conn(0x7f5ac00ce9d0 msgr2=0x7f5ac00cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 -- 172.21.15.53:0/458497035 shutdown_connections 2023-10-21T17:38:40.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.500+0000 7f5ac48b3640 1 -- 172.21.15.53:0/458497035 wait complete. 2023-10-21T17:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.502+0000 7f5ac48b3640 1 Processor -- start 2023-10-21T17:38:40.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.502+0000 7f5ac48b3640 1 -- start start 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ac006dd00 0x7f5ac0070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ac00cb990 0x7f5ac0070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f34c0 con 0x7f5ac015a970 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f0c30 con 0x7f5ac006dd00 2023-10-21T17:38:40.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5ac48b3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01f36d0 con 0x7f5ac00cb990 2023-10-21T17:38:40.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5abf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5abf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49824/0 (socket says 172.21.15.53:49824) 2023-10-21T17:38:40.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.503+0000 7f5abf59e640 1 -- 172.21.15.53:0/120676832 learned_addr learned my addr 172.21.15.53:0/120676832 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:40.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abed9d640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ac006dd00 0x7f5ac0070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5ab7fff640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ac00cb990 0x7f5ac0070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 -- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ac00cb990 msgr2=0x7f5ac0070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ac00cb990 0x7f5ac0070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 -- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ac006dd00 msgr2=0x7f5ac0070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ac006dd00 0x7f5ac0070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 -- 172.21.15.53:0/120676832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0009400 con 0x7f5ac015a970 2023-10-21T17:38:40.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5abf59e640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800cb40 tx=0x7f5ab800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.504+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab800aca0 con 0x7f5ac015a970 2023-10-21T17:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.505+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab800ae00 con 0x7f5ac015a970 2023-10-21T17:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.505+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8012620 con 0x7f5ac015a970 2023-10-21T17:38:40.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.505+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac01f39b0 con 0x7f5ac015a970 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.505+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac01f41b0 con 0x7f5ac015a970 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.506+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac01f3ed0 con 0x7f5ac015a970 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.507+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5ab8007070 con 0x7f5ac015a970 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.508+0000 7f5ab77fe640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 0x7f5aa006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.508+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5ab8089a40 con 0x7f5ac015a970 2023-10-21T17:38:40.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.508+0000 7f5abed9d640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 0x7f5aa006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:40.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.510+0000 7f5abed9d640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 0x7f5aa006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0036000 tx=0x7f5ab003d040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:40.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.512+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ac01f3ed0 con 0x7f5ac015a970 2023-10-21T17:38:40.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.748+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5ac01f3ed0 con 0x7f5ac015a970 2023-10-21T17:38:40.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.749+0000 7f5ab77fe640 1 -- 172.21.15.53:0/120676832 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5ac01f3ed0 con 0x7f5ac015a970 2023-10-21T17:38:40.750 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:38:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.751+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 msgr2=0x7f5aa006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.751+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 0x7f5aa006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0036000 tx=0x7f5ab003d040 comp rx=0 tx=0).stop 2023-10-21T17:38:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.751+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 msgr2=0x7f5ac00c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:40.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.751+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800cb40 tx=0x7f5ab800a060 comp rx=0 tx=0).stop 2023-10-21T17:38:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 shutdown_connections 2023-10-21T17:38:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ac015a970 0x7f5ac00c5680 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5aa006af90 0x7f5aa006d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ac00cb990 0x7f5ac0070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 --2- 172.21.15.53:0/120676832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ac006dd00 0x7f5ac0070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:40.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 >> 172.21.15.53:0/120676832 conn(0x7f5ac00ce9d0 msgr2=0x7f5ac00b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:40.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 shutdown_connections 2023-10-21T17:38:40.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:40.752+0000 7f5ac48b3640 1 -- 172.21.15.53:0/120676832 wait complete. 2023-10-21T17:38:40.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:38:40.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:38:40.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:38:40.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:38:40.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:38:40.774 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:38:40.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:38:40.774 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:38:40.775 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:38:40.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:40.775 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:38:40.834 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:38:40.834 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:38:40.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:38:40.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.835 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:38:40.835 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.835 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:38:40.836 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:38:40.837 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:38:40.837 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:38:40.837 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:38:40.837 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 36572sec preferred_lft 36572sec 2023-10-21T17:38:40.837 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:38:40.838 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.838 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:38:40.838 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:38:40.838 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:38:40.838 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:38:40.839 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:38:40.839 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:38:40.839 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.839 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:38:40.839 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:38:40.840 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:38:40.841 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:38:40.841 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:38:40.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:40.842 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:38:40.936 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:38:40.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:40.941 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:38:40.998 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:38:40.999 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:38:40.999 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:38:40.999 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:38:40.999 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:38:41.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.000 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:38:41.000 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:38:41.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:38:41.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:38:41.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:38:41.002 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:38:41.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.197 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:38:41.251 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:38:41.251 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:38:41.252 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:38:41.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:38:41.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:38:41.360 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:38:41.422 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:38:41.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:38:41.423 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:38:41.479 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:38:41.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.479 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:38:41.632 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:38:41.632 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:38:41.670 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:38:41.670 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:38:41.670 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:38:41.670 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:38:41.671 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:38:41.672 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:38:41.673 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:38:41.673 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:38:41.673 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:38:41.673 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:38:41.677 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:38:41.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:38:41.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:41.771 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:38:41.858 INFO:teuthology.orchestra.run.smithi053.stdout:client38003 2023-10-21T17:38:41.858 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:38:41.859 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:38:41.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-21T17:38:42.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.154+0000 7fe415ba7640 1 Processor -- start 2023-10-21T17:38:42.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.154+0000 7fe415ba7640 1 -- start start 2023-10-21T17:38:42.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.156+0000 7fe415ba7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.156+0000 7fe415ba7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4100cfab0 0x7fe4100cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.156+0000 7fe415ba7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100d0450 con 0x7fe41015cc70 2023-10-21T17:38:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.156+0000 7fe415ba7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe410160590 con 0x7fe4100cfab0 2023-10-21T17:38:42.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.156+0000 7fe415ba7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ccf90 con 0x7fe41015c890 2023-10-21T17:38:42.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4100cfab0 0x7fe4100cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe4100cfab0 0x7fe4100cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49836/0 (socket says 172.21.15.53:49836) 2023-10-21T17:38:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 -- 172.21.15.53:0/2278484409 learned_addr learned my addr 172.21.15.53:0/2278484409 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe414ba5640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 -- 172.21.15.53:0/2278484409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 msgr2=0x7fe4100cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 -- 172.21.15.53:0/2278484409 >> v1:172.21.15.124:6789/0 conn(0x7fe41015cc70 legacy=0x7fe41015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe40ffff640 1 -- 172.21.15.53:0/2278484409 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100714a0 con 0x7fe4100cfab0 2023-10-21T17:38:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.157+0000 7fe414ba5640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.159+0000 7fe40ffff640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100cfab0 0x7fe4100cff10 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009870 tx=0x7fe3fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef821f8ad1f0e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.159+0000 7fe40f7fe640 1 -- 172.21.15.53:0/2278484409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc031640 con 0x7fe4100cfab0 2023-10-21T17:38:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.159+0000 7fe40f7fe640 1 -- 172.21.15.53:0/2278484409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0317a0 con 0x7fe4100cfab0 2023-10-21T17:38:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.159+0000 7fe40f7fe640 1 -- 172.21.15.53:0/2278484409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc031b30 con 0x7fe4100cfab0 2023-10-21T17:38:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 -- 172.21.15.53:0/2278484409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100cfab0 msgr2=0x7fe4100cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100cfab0 0x7fe4100cff10 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009870 tx=0x7fe3fc02b8f0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 -- 172.21.15.53:0/2278484409 shutdown_connections 2023-10-21T17:38:42.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100cfab0 0x7fe4100cff10 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 --2- 172.21.15.53:0/2278484409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 -- 172.21.15.53:0/2278484409 >> 172.21.15.53:0/2278484409 conn(0x7fe41006b250 msgr2=0x7fe41006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 -- 172.21.15.53:0/2278484409 shutdown_connections 2023-10-21T17:38:42.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.160+0000 7fe415ba7640 1 -- 172.21.15.53:0/2278484409 wait complete. 2023-10-21T17:38:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.162+0000 7fe415ba7640 1 Processor -- start 2023-10-21T17:38:42.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.162+0000 7fe415ba7640 1 -- start start 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4100cfab0 0x7fe4100c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ad6d0 con 0x7fe41015cc70 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ac820 con 0x7fe4100cfab0 2023-10-21T17:38:42.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe415ba7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101590c0 con 0x7fe41015c890 2023-10-21T17:38:42.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.163+0000 7fe4153a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49838/0 (socket says 172.21.15.53:49838) 2023-10-21T17:38:42.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 -- 172.21.15.53:0/1004150945 learned_addr learned my addr 172.21.15.53:0/1004150945 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:42.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 -- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 msgr2=0x7fe4100ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 -- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4100cfab0 msgr2=0x7fe4100c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4100cfab0 0x7fe4100c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 -- 172.21.15.53:0/1004150945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0094d0 con 0x7fe41015cc70 2023-10-21T17:38:42.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe4153a6640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c730 tx=0x7fe40800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.164+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408003b70 con 0x7fe41015cc70 2023-10-21T17:38:42.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.165+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408003cd0 con 0x7fe41015cc70 2023-10-21T17:38:42.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.165+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40800aa90 con 0x7fe41015cc70 2023-10-21T17:38:42.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.165+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100acac0 con 0x7fe41015cc70 2023-10-21T17:38:42.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.165+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100ad080 con 0x7fe41015cc70 2023-10-21T17:38:42.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.166+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410074150 con 0x7fe41015cc70 2023-10-21T17:38:42.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.167+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe40800abf0 con 0x7fe41015cc70 2023-10-21T17:38:42.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.168+0000 7fe40d7fa640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 0x7fe3f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.168+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe408089740 con 0x7fe41015cc70 2023-10-21T17:38:42.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.169+0000 7fe414ba5640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 0x7fe3f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.170+0000 7fe414ba5640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 0x7fe3f406d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe404006fd0 tx=0x7fe404008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.172+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe408054510 con 0x7fe41015cc70 2023-10-21T17:38:42.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.410+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe4100ad810 con 0x7fe41015cc70 2023-10-21T17:38:42.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.473+0000 7fe40d7fa640 1 -- 172.21.15.53:0/1004150945 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe4100ad810 con 0x7fe41015cc70 2023-10-21T17:38:42.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.476+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 msgr2=0x7fe3f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.476+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 0x7fe3f406d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe404006fd0 tx=0x7fe404008040 comp rx=0 tx=0).stop 2023-10-21T17:38:42.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.476+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 msgr2=0x7fe410158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.476+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c730 tx=0x7fe40800cbf0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 shutdown_connections 2023-10-21T17:38:42.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe41015cc70 0x7fe410158b80 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fe3f406afc0 0x7fe3f406d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe41015c890 0x7fe4100ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 --2- 172.21.15.53:0/1004150945 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4100cfab0 0x7fe4100c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 >> 172.21.15.53:0/1004150945 conn(0x7fe41006b250 msgr2=0x7fe41015d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:42.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 shutdown_connections 2023-10-21T17:38:42.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.477+0000 7fe415ba7640 1 -- 172.21.15.53:0/1004150945 wait complete. 2023-10-21T17:38:42.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:42.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.722+0000 7f6c8e3e4640 1 Processor -- start 2023-10-21T17:38:42.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.722+0000 7f6c8e3e4640 1 -- start start 2023-10-21T17:38:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c8815ab30 0x7f6c8815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c880adc70 0x7f6c880b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6c8815af10 0x7f6c880b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8815f7d0 con 0x7f6c880adc70 2023-10-21T17:38:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8815f210 con 0x7f6c8815af10 2023-10-21T17:38:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8e3e4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815f4f0 con 0x7f6c8815ab30 2023-10-21T17:38:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8cbe1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c880adc70 0x7f6c880b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8cbe1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c880adc70 0x7f6c880b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49842/0 (socket says 172.21.15.53:49842) 2023-10-21T17:38:42.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.724+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/782040514 learned_addr learned my addr 172.21.15.53:0/782040514 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8dbe3640 1 --2- 172.21.15.53:0/782040514 >> v2:172.21.15.124:3300/0 conn(0x7f6c8815af10 0x7f6c880b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8d3e2640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c8815ab30 0x7f6c8815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/782040514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c8815ab30 msgr2=0x7f6c8815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c8815ab30 0x7f6c8815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/782040514 >> v2:172.21.15.124:3300/0 conn(0x7f6c8815af10 msgr2=0x7f6c880b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:42.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/782040514 >> v2:172.21.15.124:3300/0 conn(0x7f6c8815af10 0x7f6c880b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.725+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/782040514 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8806c550 con 0x7f6c880adc70 2023-10-21T17:38:42.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.726+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c880adc70 0x7f6c880b00c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f6c880cb000 tx=0x7f6c7802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196a5800c3c9a48 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c7ffff640 1 -- 172.21.15.53:0/782040514 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78031860 con 0x7f6c880adc70 2023-10-21T17:38:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c7ffff640 1 -- 172.21.15.53:0/782040514 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c780319c0 con 0x7f6c880adc70 2023-10-21T17:38:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c7ffff640 1 -- 172.21.15.53:0/782040514 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78031d50 con 0x7f6c880adc70 2023-10-21T17:38:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/782040514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c880adc70 msgr2=0x7f6c880b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c880adc70 0x7f6c880b00c0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f6c880cb000 tx=0x7f6c7802b900 comp rx=0 tx=0).stop 2023-10-21T17:38:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/782040514 shutdown_connections 2023-10-21T17:38:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/782040514 >> v2:172.21.15.124:3300/0 conn(0x7f6c8815af10 0x7f6c880b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c880adc70 0x7f6c880b00c0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/782040514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c8815ab30 0x7f6c8815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.727+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/782040514 >> 172.21.15.53:0/782040514 conn(0x7f6c880cb0c0 msgr2=0x7f6c880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.728+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/782040514 shutdown_connections 2023-10-21T17:38:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.728+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/782040514 wait complete. 2023-10-21T17:38:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.729+0000 7f6c8e3e4640 1 Processor -- start 2023-10-21T17:38:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.729+0000 7f6c8e3e4640 1 -- start start 2023-10-21T17:38:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c880adc70 0x7f6c8814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c8815af10 0x7f6c880b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815b8a0 con 0x7f6c8815ab30 2023-10-21T17:38:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815c960 con 0x7f6c8815af10 2023-10-21T17:38:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.730+0000 7f6c8e3e4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880c6b90 con 0x7f6c880adc70 2023-10-21T17:38:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49848/0 (socket says 172.21.15.53:49848) 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/2183792156 learned_addr learned my addr 172.21.15.53:0/2183792156 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8d3e2640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c880adc70 0x7f6c8814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8dbe3640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c8815af10 0x7f6c880b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c880adc70 msgr2=0x7f6c8814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c880adc70 0x7f6c8814c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c8815af10 msgr2=0x7f6c880b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c8815af10 0x7f6c880b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.731+0000 7f6c8cbe1640 1 -- 172.21.15.53:0/2183792156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780094d0 con 0x7f6c8815ab30 2023-10-21T17:38:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c8cbe1640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f6c7802b940 tx=0x7f6c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7803ac60 con 0x7f6c8815ab30 2023-10-21T17:38:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7803adc0 con 0x7f6c8815ab30 2023-10-21T17:38:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780445f0 con 0x7f6c8815ab30 2023-10-21T17:38:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c880c6e70 con 0x7f6c8815ab30 2023-10-21T17:38:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.732+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c880c7430 con 0x7f6c8815ab30 2023-10-21T17:38:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.734+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8804e160 con 0x7f6c8815ab30 2023-10-21T17:38:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.735+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6c7804d450 con 0x7f6c8815ab30 2023-10-21T17:38:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.736+0000 7f6c7dffb640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 0x7f6c6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.736+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6c780ac0f0 con 0x7f6c8815ab30 2023-10-21T17:38:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.736+0000 7f6c8d3e2640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 0x7f6c6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.737+0000 7f6c8d3e2640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 0x7f6c6c06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.740+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c78076d20 con 0x7f6c8815ab30 2023-10-21T17:38:42.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.943+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c880ada50 con 0x7f6c8815ab30 2023-10-21T17:38:42.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.945+0000 7f6c7dffb640 1 -- 172.21.15.53:0/2183792156 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f6c78035090 con 0x7f6c8815ab30 2023-10-21T17:38:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:42.948 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":233,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:34.320176+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:42.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 msgr2=0x7f6c6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 0x7f6c6c06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84008040 comp rx=0 tx=0).stop 2023-10-21T17:38:42.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 msgr2=0x7f6c8815b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:42.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f6c7802b940 tx=0x7f6c780046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 shutdown_connections 2023-10-21T17:38:42.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c8815af10 0x7f6c880b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c8815ab30 0x7f6c8815b360 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f6c6c06af90 0x7f6c6c06d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 --2- 172.21.15.53:0/2183792156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c880adc70 0x7f6c8814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:42.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 >> 172.21.15.53:0/2183792156 conn(0x7f6c880cb0c0 msgr2=0x7f6c880cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:42.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.948+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 shutdown_connections 2023-10-21T17:38:42.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:42.949+0000 7f6c8e3e4640 1 -- 172.21.15.53:0/2183792156 wait complete. 2023-10-21T17:38:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:38:30.810147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:38:31.897170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:38:42.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:38:43.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.199+0000 7f0946916640 1 Processor -- start 2023-10-21T17:38:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.200+0000 7f0946916640 1 -- start start 2023-10-21T17:38:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0946916640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c9390 0x7f0940160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0946916640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0940160c90 0x7f094015c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0946916640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09400b0260 con 0x7f09400c8fb0 2023-10-21T17:38:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0946916640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09401610d0 con 0x7f0940160c90 2023-10-21T17:38:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0946916640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09401612d0 con 0x7f09400c9390 2023-10-21T17:38:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0945914640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f09400c8fb0 0x7f09400cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46674/0 (socket says 172.21.15.53:46674) 2023-10-21T17:38:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.201+0000 7f0945914640 1 -- 172.21.15.53:0/427062785 learned_addr learned my addr 172.21.15.53:0/427062785 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0946115640 1 --2- 172.21.15.53:0/427062785 >> v2:172.21.15.53:3300/0 conn(0x7f0940160c90 0x7f094015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0945113640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c9390 0x7f0940160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0944912640 1 -- 172.21.15.53:0/427062785 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811690172 0 0) 0x7f09400b0260 con 0x7f09400c8fb0 2023-10-21T17:38:43.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0944912640 1 -- 172.21.15.53:0/427062785 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f092c003520 con 0x7f09400c8fb0 2023-10-21T17:38:43.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0946115640 1 -- 172.21.15.53:0/427062785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c9390 msgr2=0x7f0940160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:43.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0946115640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c9390 0x7f0940160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0946115640 1 -- 172.21.15.53:0/427062785 >> v1:172.21.15.53:6789/0 conn(0x7f09400c8fb0 legacy=0x7f09400cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.202+0000 7f0946115640 1 -- 172.21.15.53:0/427062785 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940161590 con 0x7f0940160c90 2023-10-21T17:38:43.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.203+0000 7f0946115640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0940160c90 0x7f094015c630 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f09380097a0 tx=0x7f093802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1b70f6e8be6424 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0944912640 1 -- 172.21.15.53:0/427062785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938030800 con 0x7f0940160c90 2023-10-21T17:38:43.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0944912640 1 -- 172.21.15.53:0/427062785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938030960 con 0x7f0940160c90 2023-10-21T17:38:43.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0944912640 1 -- 172.21.15.53:0/427062785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938030cd0 con 0x7f0940160c90 2023-10-21T17:38:43.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 -- 172.21.15.53:0/427062785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0940160c90 msgr2=0x7f094015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0940160c90 0x7f094015c630 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f09380097a0 tx=0x7f093802b940 comp rx=0 tx=0).stop 2023-10-21T17:38:43.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 -- 172.21.15.53:0/427062785 shutdown_connections 2023-10-21T17:38:43.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0940160c90 0x7f094015c630 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 --2- 172.21.15.53:0/427062785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c9390 0x7f0940160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.204+0000 7f0946916640 1 -- 172.21.15.53:0/427062785 >> 172.21.15.53:0/427062785 conn(0x7f094006c010 msgr2=0x7f094006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.205+0000 7f0946916640 1 -- 172.21.15.53:0/427062785 shutdown_connections 2023-10-21T17:38:43.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.205+0000 7f0946916640 1 -- 172.21.15.53:0/427062785 wait complete. 2023-10-21T17:38:43.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.206+0000 7f0946916640 1 Processor -- start 2023-10-21T17:38:43.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.206+0000 7f0946916640 1 -- start start 2023-10-21T17:38:43.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c8fb0 0x7f094014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0940160c90 0x7f0940149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400b2f00 con 0x7f09400c9390 2023-10-21T17:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401497d0 con 0x7f0940160c90 2023-10-21T17:38:43.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.207+0000 7f0946916640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0940149fd0 con 0x7f09400c8fb0 2023-10-21T17:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49880/0 (socket says 172.21.15.53:49880) 2023-10-21T17:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 -- 172.21.15.53:0/4274817975 learned_addr learned my addr 172.21.15.53:0/4274817975 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945914640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c8fb0 0x7f094014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0946115640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0940160c90 0x7f0940149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 -- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c8fb0 msgr2=0x7f094014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c8fb0 0x7f094014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 -- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0940160c90 msgr2=0x7f0940149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0940160c90 0x7f0940149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.208+0000 7f0945113640 1 -- 172.21.15.53:0/4274817975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938009400 con 0x7f09400c9390 2023-10-21T17:38:43.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f0945113640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9e0 tx=0x7f093000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930009670 con 0x7f09400c9390 2023-10-21T17:38:43.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000c9d0 con 0x7f09400c9390 2023-10-21T17:38:43.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000cd80 con 0x7f09400c9390 2023-10-21T17:38:43.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09401f2b40 con 0x7f09400c9390 2023-10-21T17:38:43.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.209+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09401f3100 con 0x7f09400c9390 2023-10-21T17:38:43.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.210+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09401f3320 con 0x7f09400c9390 2023-10-21T17:38:43.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.211+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0930013470 con 0x7f09400c9390 2023-10-21T17:38:43.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.212+0000 7f09367fc640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 0x7f092406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.212+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f0930089b40 con 0x7f09400c9390 2023-10-21T17:38:43.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.213+0000 7f0945914640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 0x7f092406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.214+0000 7f0945914640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 0x7f092406d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f093c005c00 tx=0x7f093c009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.216+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0930054910 con 0x7f09400c9390 2023-10-21T17:38:43.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.414+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0940074150 con 0x7f09400c9390 2023-10-21T17:38:43.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.415+0000 7f09367fc640 1 -- 172.21.15.53:0/4274817975 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v233) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09300170a0 con 0x7f09400c9390 2023-10-21T17:38:43.418 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:38:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 msgr2=0x7f092406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 0x7f092406d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f093c005c00 tx=0x7f093c009040 comp rx=0 tx=0).stop 2023-10-21T17:38:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 msgr2=0x7f0940149290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f093000e9e0 tx=0x7f093000eea0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 shutdown_connections 2023-10-21T17:38:43.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0940160c90 0x7f0940149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09400c9390 0x7f0940149290 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f092406af90 0x7f092406d450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 --2- 172.21.15.53:0/4274817975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09400c8fb0 0x7f094014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 >> 172.21.15.53:0/4274817975 conn(0x7f094006c010 msgr2=0x7f09400c2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:43.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 shutdown_connections 2023-10-21T17:38:43.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.418+0000 7f0946916640 1 -- 172.21.15.53:0/4274817975 wait complete. 2023-10-21T17:38:43.439 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:38:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.665+0000 7fbc7df0a640 1 Processor -- start 2023-10-21T17:38:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.665+0000 7fbc7df0a640 1 -- start start 2023-10-21T17:38:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7df0a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc7815cae0 0x7fbc780cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7df0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 0x7fbc780c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7df0a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc780ccbc0 con 0x7fbc780c8b40 2023-10-21T17:38:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7df0a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc780c9510 con 0x7fbc7815cae0 2023-10-21T17:38:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7df0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780ce1f0 con 0x7fbc780b0680 2023-10-21T17:38:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7cf08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc7815cae0 0x7fbc780cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7cf08640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc7815cae0 0x7fbc780cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49892/0 (socket says 172.21.15.53:49892) 2023-10-21T17:38:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.667+0000 7fbc7cf08640 1 -- 172.21.15.53:0/3820764926 learned_addr learned my addr 172.21.15.53:0/3820764926 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7cf08640 1 -- 172.21.15.53:0/3820764926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 msgr2=0x7fbc780c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7d709640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 0x7fbc780c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7cf08640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 0x7fbc780c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7cf08640 1 -- 172.21.15.53:0/3820764926 >> v1:172.21.15.124:6789/0 conn(0x7fbc780c8b40 legacy=0x7fbc780aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7cf08640 1 -- 172.21.15.53:0/3820764926 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc780cd590 con 0x7fbc7815cae0 2023-10-21T17:38:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.668+0000 7fbc7d709640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 0x7fbc780c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.669+0000 7fbc7cf08640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7815cae0 0x7fbc780cd050 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009e50 tx=0x7fbc7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b6cb6abaf1047b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.669+0000 7fbc6f7fe640 1 -- 172.21.15.53:0/3820764926 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7402ca80 con 0x7fbc7815cae0 2023-10-21T17:38:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.669+0000 7fbc6f7fe640 1 -- 172.21.15.53:0/3820764926 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7402cbe0 con 0x7fbc7815cae0 2023-10-21T17:38:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.669+0000 7fbc6f7fe640 1 -- 172.21.15.53:0/3820764926 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740343f0 con 0x7fbc7815cae0 2023-10-21T17:38:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3820764926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7815cae0 msgr2=0x7fbc780cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7815cae0 0x7fbc780cd050 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009e50 tx=0x7fbc7402bee0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3820764926 shutdown_connections 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc780b0680 0x7fbc780c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3820764926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7815cae0 0x7fbc780cd050 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3820764926 >> 172.21.15.53:0/3820764926 conn(0x7fbc7806c050 msgr2=0x7fbc7806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:43.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3820764926 shutdown_connections 2023-10-21T17:38:43.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.670+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3820764926 wait complete. 2023-10-21T17:38:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.672+0000 7fbc7df0a640 1 Processor -- start 2023-10-21T17:38:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.672+0000 7fbc7df0a640 1 -- start start 2023-10-21T17:38:43.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc780b0680 0x7fbc781573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc7815cae0 0x7fbc780c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7815f8b0 con 0x7fbc780c8b40 2023-10-21T17:38:43.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7815cff0 con 0x7fbc780b0680 2023-10-21T17:38:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc7df0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7815d2d0 con 0x7fbc7815cae0 2023-10-21T17:38:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.673+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49904/0 (socket says 172.21.15.53:49904) 2023-10-21T17:38:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 -- 172.21.15.53:0/3023829542 learned_addr learned my addr 172.21.15.53:0/3023829542 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 -- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc7815cae0 msgr2=0x7fbc780c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc7815cae0 0x7fbc780c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc7cf08640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc780b0680 0x7fbc781573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 -- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc780b0680 msgr2=0x7fbc781573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc780b0680 0x7fbc781573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 -- 172.21.15.53:0/3023829542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74009b30 con 0x7fbc780c8b40 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.674+0000 7fbc6ffff640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c430 tx=0x7fbc6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.675+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68011790 con 0x7fbc780c8b40 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.675+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc680118f0 con 0x7fbc780c8b40 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.675+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68011ce0 con 0x7fbc780c8b40 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.675+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7815d5b0 con 0x7fbc780c8b40 2023-10-21T17:38:43.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.675+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc781578e0 con 0x7fbc780c8b40 2023-10-21T17:38:43.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.676+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7815dad0 con 0x7fbc780c8b40 2023-10-21T17:38:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.677+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbc6801c460 con 0x7fbc780c8b40 2023-10-21T17:38:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.678+0000 7fbc6d7fa640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 0x7fbc6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.678+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fbc680898b0 con 0x7fbc780c8b40 2023-10-21T17:38:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.679+0000 7fbc7cf08640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 0x7fbc6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:43.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.680+0000 7fbc7cf08640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 0x7fbc6406d450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc74035000 tx=0x7fbc74030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:43.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.683+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc7815dad0 con 0x7fbc780c8b40 2023-10-21T17:38:43.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.875+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc7815dad0 con 0x7fbc780c8b40 2023-10-21T17:38:43.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.876+0000 7fbc6d7fa640 1 -- 172.21.15.53:0/3023829542 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v233) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc7815dad0 con 0x7fbc780c8b40 2023-10-21T17:38:43.878 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:38:43.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 msgr2=0x7fbc6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 0x7fbc6406d450 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc74035000 tx=0x7fbc74030000 comp rx=0 tx=0).stop 2023-10-21T17:38:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 msgr2=0x7fbc7815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fbc6800c430 tx=0x7fbc6800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 shutdown_connections 2023-10-21T17:38:43.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbc6406af90 0x7fbc6406d450 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc7815cae0 0x7fbc780c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc780c8b40 0x7fbc7815ddb0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 --2- 172.21.15.53:0/3023829542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc780b0680 0x7fbc781573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 >> 172.21.15.53:0/3023829542 conn(0x7fbc7806c050 msgr2=0x7fbc780ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:43.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 shutdown_connections 2023-10-21T17:38:43.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:43.878+0000 7fbc7df0a640 1 -- 172.21.15.53:0/3023829542 wait complete. 2023-10-21T17:38:43.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:38:44.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.125+0000 7fcb5aa39640 1 Processor -- start 2023-10-21T17:38:44.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.125+0000 7fcb5aa39640 1 -- start start 2023-10-21T17:38:44.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.126+0000 7fcb5aa39640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcb5415a820 0x7fcb540aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb5aa39640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540cab60 con 0x7fcb5415ac00 2023-10-21T17:38:44.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb5aa39640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540cb960 con 0x7fcb540ca680 2023-10-21T17:38:44.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb5aa39640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540cbdf0 con 0x7fcb5415a820 2023-10-21T17:38:44.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb5a238640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcb5415ac00 0x7fcb540c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59316/0 (socket says 172.21.15.53:59316) 2023-10-21T17:38:44.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb5a238640 1 -- 172.21.15.53:0/896781132 learned_addr learned my addr 172.21.15.53:0/896781132 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:44.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb59a37640 1 --2- 172.21.15.53:0/896781132 >> v2:172.21.15.124:3300/0 conn(0x7fcb5415a820 0x7fcb540aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309785425 0 0) 0x7fcb540cab60 con 0x7fcb5415ac00 2023-10-21T17:38:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.127+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb40003520 con 0x7fcb5415ac00 2023-10-21T17:38:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946964864 0 0) 0x7fcb40003520 con 0x7fcb5415ac00 2023-10-21T17:38:44.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 >> v2:172.21.15.124:3300/0 conn(0x7fcb5415a820 msgr2=0x7fcb540aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 --2- 172.21.15.53:0/896781132 >> v2:172.21.15.124:3300/0 conn(0x7fcb5415a820 0x7fcb540aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 >> v1:172.21.15.124:6789/0 conn(0x7fcb540ca680 legacy=0x7fcb540c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540c9f20 con 0x7fcb5415ac00 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcb4c001670 con 0x7fcb5415ac00 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 >> v1:172.21.15.53:6789/0 conn(0x7fcb5415ac00 legacy=0x7fcb540c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 0x7fcb40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400047c0 0x7fcb40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb400051d0 0x7fcb4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540cbdf0 con 0x7fcb400039c0 2023-10-21T17:38:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4000c120 con 0x7fcb400051d0 2023-10-21T17:38:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4000c2e0 con 0x7fcb400047c0 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.128+0000 7fcb59a37640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 0x7fcb40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb59a37640 1 -- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400047c0 msgr2=0x7fcb40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb59a37640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400047c0 0x7fcb40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb59a37640 1 -- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb400051d0 msgr2=0x7fcb4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb59a37640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb400051d0 0x7fcb4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb5a238640 1 -- 172.21.15.53:0/896781132 reap_dead start 2023-10-21T17:38:44.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.129+0000 7fcb59a37640 1 -- 172.21.15.53:0/896781132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40003520 con 0x7fcb400039c0 2023-10-21T17:38:44.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.130+0000 7fcb59a37640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 0x7fcb40003de0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009a90 tx=0x7fcb400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2530e5a2c5d8bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.130+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4802c680 con 0x7fcb400039c0 2023-10-21T17:38:44.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.130+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4802c7e0 con 0x7fcb400039c0 2023-10-21T17:38:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.130+0000 7fcb58a35640 1 -- 172.21.15.53:0/896781132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4802cb70 con 0x7fcb400039c0 2023-10-21T17:38:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 -- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 msgr2=0x7fcb40003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 0x7fcb40003de0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009a90 tx=0x7fcb400057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 -- 172.21.15.53:0/896781132 shutdown_connections 2023-10-21T17:38:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/896781132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb400039c0 0x7fcb40003de0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 -- 172.21.15.53:0/896781132 >> 172.21.15.53:0/896781132 conn(0x7fcb5406c050 msgr2=0x7fcb5406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 -- 172.21.15.53:0/896781132 shutdown_connections 2023-10-21T17:38:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.131+0000 7fcb5aa39640 1 -- 172.21.15.53:0/896781132 wait complete. 2023-10-21T17:38:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.133+0000 7fcb5aa39640 1 Processor -- start 2023-10-21T17:38:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.133+0000 7fcb5aa39640 1 -- start start 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400039c0 0x7fcb54155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb540c2f40 0x7fcb540c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb541561d0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5415cbc0 con 0x7fcb540c2f40 2023-10-21T17:38:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5aa39640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540c21c0 con 0x7fcb400039c0 2023-10-21T17:38:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48848/0 (socket says 172.21.15.53:48848) 2023-10-21T17:38:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 -- 172.21.15.53:0/3017772908 learned_addr learned my addr 172.21.15.53:0/3017772908 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59a37640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400039c0 0x7fcb54155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb5a238640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb540c2f40 0x7fcb540c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 -- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400039c0 msgr2=0x7fcb54155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400039c0 0x7fcb54155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 -- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb540c2f40 msgr2=0x7fcb540c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb540c2f40 0x7fcb540c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.134+0000 7fcb59236640 1 -- 172.21.15.53:0/3017772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540c2460 con 0x7fcb540ae7f0 2023-10-21T17:38:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb59236640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcb50002230 tx=0x7fcb5000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003b70 con 0x7fcb540ae7f0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50003cd0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000aa90 con 0x7fcb540ae7f0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb540c2740 con 0x7fcb540ae7f0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.135+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb540af1b0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.136+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54074150 con 0x7fcb540ae7f0 2023-10-21T17:38:44.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.137+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb5000abf0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.139+0000 7fcb467fc640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 0x7fcb3806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.139+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fcb50089730 con 0x7fcb540ae7f0 2023-10-21T17:38:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.140+0000 7fcb59a37640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 0x7fcb3806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.141+0000 7fcb59a37640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 0x7fcb3806d480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb48036000 tx=0x7fcb48030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.143+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb500545b0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.340+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb540c9550 con 0x7fcb540ae7f0 2023-10-21T17:38:44.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.340+0000 7fcb467fc640 1 -- 172.21.15.53:0/3017772908 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v233) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb50057bd0 con 0x7fcb540ae7f0 2023-10-21T17:38:44.343 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:38:44.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 msgr2=0x7fcb3806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 0x7fcb3806d480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcb48036000 tx=0x7fcb48030000 comp rx=0 tx=0).stop 2023-10-21T17:38:44.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 msgr2=0x7fcb540aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcb50002230 tx=0x7fcb5000cc00 comp rx=0 tx=0).stop 2023-10-21T17:38:44.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 shutdown_connections 2023-10-21T17:38:44.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb540c2f40 0x7fcb540c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb540ae7f0 0x7fcb540aec70 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcb3806afc0 0x7fcb3806d480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 --2- 172.21.15.53:0/3017772908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb400039c0 0x7fcb54155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 >> 172.21.15.53:0/3017772908 conn(0x7fcb5406c050 msgr2=0x7fcb540b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 shutdown_connections 2023-10-21T17:38:44.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.343+0000 7fcb5aa39640 1 -- 172.21.15.53:0/3017772908 wait complete. 2023-10-21T17:38:44.366 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:38:44.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.587+0000 7fb719f6c640 1 Processor -- start 2023-10-21T17:38:44.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.587+0000 7fb719f6c640 1 -- start start 2023-10-21T17:38:44.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb719f6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb714162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb719f6c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140ae140 con 0x7fb7140add60 2023-10-21T17:38:44.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb719f6c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140ccf20 con 0x7fb7140cb150 2023-10-21T17:38:44.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb719f6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb714161f10 con 0x7fb71406d4c0 2023-10-21T17:38:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb7137fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb7140add60 0x7fb7140cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59322/0 (socket says 172.21.15.53:59322) 2023-10-21T17:38:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.589+0000 7fb7137fe640 1 -- 172.21.15.53:0/2084809564 learned_addr learned my addr 172.21.15.53:0/2084809564 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb712ffd640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb714162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763624903 0 0) 0x7fb7140ae140 con 0x7fb7140add60 2023-10-21T17:38:44.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6fc003520 con 0x7fb7140add60 2023-10-21T17:38:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580317954 0 0) 0x7fb6fc003520 con 0x7fb7140add60 2023-10-21T17:38:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 msgr2=0x7fb714162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb714162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 >> v1:172.21.15.124:6789/0 conn(0x7fb7140cb150 legacy=0x7fb714164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140cd1c0 con 0x7fb7140add60 2023-10-21T17:38:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fb704001670 con 0x7fb7140add60 2023-10-21T17:38:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 >> v1:172.21.15.53:6789/0 conn(0x7fb7140add60 legacy=0x7fb7140cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6fc0039a0 0x7fb6fc003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6fc0047c0 0x7fb6fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb6fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb714161f10 con 0x7fb6fc0039a0 2023-10-21T17:38:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc00ba80 con 0x7fb6fc0047c0 2023-10-21T17:38:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.590+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc00bc40 con 0x7fb71406d4c0 2023-10-21T17:38:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6fc0039a0 0x7fb6fc003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb712ffd640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6fc0047c0 0x7fb6fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb713fff640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb6fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 -- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 msgr2=0x7fb6fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb6fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 -- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6fc0047c0 msgr2=0x7fb6fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6fc0047c0 0x7fb6fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb7137fe640 1 -- 172.21.15.53:0/2084809564 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc003520 con 0x7fb6fc0039a0 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.591+0000 7fb713fff640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb71406d4c0 0x7fb6fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.592+0000 7fb7137fe640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6fc0039a0 0x7fb6fc003d80 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c0e0 tx=0x7fb6fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18350a8623b838dd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704005ad0 con 0x7fb6fc0039a0 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb704005c30 con 0x7fb6fc0039a0 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb7127fc640 1 -- 172.21.15.53:0/2084809564 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70402c810 con 0x7fb6fc0039a0 2023-10-21T17:38:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb719f6c640 1 -- 172.21.15.53:0/2084809564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6fc0039a0 msgr2=0x7fb6fc003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb719f6c640 1 --2- 172.21.15.53:0/2084809564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6fc0039a0 0x7fb6fc003d80 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c0e0 tx=0x7fb6fc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb713fff640 1 -- 172.21.15.53:0/2084809564 reap_dead start 2023-10-21T17:38:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb719f6c640 1 -- 172.21.15.53:0/2084809564 shutdown_connections 2023-10-21T17:38:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.593+0000 7fb719f6c640 1 -- 172.21.15.53:0/2084809564 >> 172.21.15.53:0/2084809564 conn(0x7fb71406a380 msgr2=0x7fb7140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.594+0000 7fb719f6c640 1 -- 172.21.15.53:0/2084809564 shutdown_connections 2023-10-21T17:38:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.594+0000 7fb719f6c640 1 -- 172.21.15.53:0/2084809564 wait complete. 2023-10-21T17:38:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.595+0000 7fb719f6c640 1 Processor -- start 2023-10-21T17:38:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.596+0000 7fb719f6c640 1 -- start start 2023-10-21T17:38:44.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 0x7fb714162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 0x7fb7141fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb714159700 con 0x7fb714157fc0 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141fa440 con 0x7fb7140b1d30 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb719f6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141fa110 con 0x7fb6fc0039a0 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb713fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb713fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48866/0 (socket says 172.21.15.53:48866) 2023-10-21T17:38:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb713fff640 1 -- 172.21.15.53:0/1643798927 learned_addr learned my addr 172.21.15.53:0/1643798927 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb712ffd640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 0x7fb7141fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.597+0000 7fb7137fe640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 0x7fb714162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 -- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 msgr2=0x7fb714162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 0x7fb714162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 -- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 msgr2=0x7fb7141fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 0x7fb7141fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 -- 172.21.15.53:0/1643798927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714156460 con 0x7fb714157fc0 2023-10-21T17:38:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb7137fe640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 0x7fb714162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb712ffd640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 0x7fb7141fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb713fff640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fb7140cb0e0 tx=0x7fb70c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c003d80 con 0x7fb714157fc0 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00de50 con 0x7fb714157fc0 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0052b0 con 0x7fb714157fc0 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.598+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb714156740 con 0x7fb714157fc0 2023-10-21T17:38:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.599+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb714158980 con 0x7fb714157fc0 2023-10-21T17:38:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.600+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7141fa5f0 con 0x7fb714157fc0 2023-10-21T17:38:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.601+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb70c005410 con 0x7fb714157fc0 2023-10-21T17:38:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.602+0000 7fb6fbfff640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 0x7fb70006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.602+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fb70c088fb0 con 0x7fb714157fc0 2023-10-21T17:38:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.602+0000 7fb7137fe640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 0x7fb70006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:44.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.604+0000 7fb7137fe640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 0x7fb70006d450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb704034000 tx=0x7fb704009030 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:44.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.606+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7141fa5f0 con 0x7fb714157fc0 2023-10-21T17:38:44.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.801+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb7141fa5b0 con 0x7fb714157fc0 2023-10-21T17:38:44.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.801+0000 7fb6fbfff640 1 -- 172.21.15.53:0/1643798927 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v233) v1 ==== 84+0+209 (secure 0 0 0) 0x7fb70c018090 con 0x7fb714157fc0 2023-10-21T17:38:44.804 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:44.804 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:44.805 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 27, 2023-10-21T17:38:44.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "cephfs_metadata" 2023-10-21T17:38:44.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:44.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:44.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 28, 2023-10-21T17:38:44.806 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "cephfs_data" 2023-10-21T17:38:44.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:44.807 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:38:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 msgr2=0x7fb70006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 0x7fb70006d450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb704034000 tx=0x7fb704009030 comp rx=0 tx=0).stop 2023-10-21T17:38:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 msgr2=0x7fb714158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fb7140cb0e0 tx=0x7fb70c00dc00 comp rx=0 tx=0).stop 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 shutdown_connections 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714157fc0 0x7fb714158440 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb7140b1d30 0x7fb7141fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb70006af90 0x7fb70006d450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 --2- 172.21.15.53:0/1643798927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6fc0039a0 0x7fb714162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:44.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 >> 172.21.15.53:0/1643798927 conn(0x7fb71406a380 msgr2=0x7fb71406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:44.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 shutdown_connections 2023-10-21T17:38:44.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:44.804+0000 7fb719f6c640 1 -- 172.21.15.53:0/1643798927 wait complete. 2023-10-21T17:38:44.825 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-21T17:38:44.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-21T17:38:45.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.058+0000 7f7d6c3b0640 1 Processor -- start 2023-10-21T17:38:45.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.059+0000 7f7d6c3b0640 1 -- start start 2023-10-21T17:38:45.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6c3b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d64070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6c3b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640b1cc0 con 0x7f7d640afbe0 2023-10-21T17:38:45.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6c3b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640b2620 con 0x7f7d64071070 2023-10-21T17:38:45.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6c3b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640b2060 con 0x7f7d6406dfc0 2023-10-21T17:38:45.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6a14c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7d640afbe0 0x7f7d640b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59338/0 (socket says 172.21.15.53:59338) 2023-10-21T17:38:45.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.060+0000 7f7d6a14c640 1 -- 172.21.15.53:0/3050367894 learned_addr learned my addr 172.21.15.53:0/3050367894 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:45.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6994b640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d64070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457133699 0 0) 0x7f7d640b1cc0 con 0x7f7d640afbe0 2023-10-21T17:38:45.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d50003520 con 0x7f7d640afbe0 2023-10-21T17:38:45.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880114517 0 0) 0x7f7d50003520 con 0x7f7d640afbe0 2023-10-21T17:38:45.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 msgr2=0x7f7d64070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:45.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.061+0000 7f7d6914a640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d64070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 >> v1:172.21.15.124:6789/0 conn(0x7f7d64071070 legacy=0x7f7d640cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640b2340 con 0x7f7d640afbe0 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7d60001670 con 0x7f7d640afbe0 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 >> v1:172.21.15.53:6789/0 conn(0x7f7d640afbe0 legacy=0x7f7d640b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d50003f20 0x7f7d500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d50004d00 0x7f7d50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640b2060 con 0x7f7d50004d00 2023-10-21T17:38:45.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5000ba60 con 0x7f7d50003f20 2023-10-21T17:38:45.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5000bc20 con 0x7f7d6406dfc0 2023-10-21T17:38:45.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d50004d00 0x7f7d50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6994b640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d50003f20 0x7f7d500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6a14c640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.062+0000 7f7d6a94d640 1 -- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 msgr2=0x7f7d500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d500039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6a94d640 1 -- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d50003f20 msgr2=0x7f7d500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:45.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d50003f20 0x7f7d500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6a94d640 1 -- 172.21.15.53:0/3050367894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50003520 con 0x7f7d50004d00 2023-10-21T17:38:45.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6a14c640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d6406dfc0 0x7f7d500039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:45.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.063+0000 7f7d6994b640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d50003f20 0x7f7d500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.064+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d50004d00 0x7f7d50005180 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c001e50 tx=0x7f7d500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28d045d261175 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.064+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003b70 con 0x7f7d50004d00 2023-10-21T17:38:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.064+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c003cd0 con 0x7f7d50004d00 2023-10-21T17:38:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.064+0000 7f7d6914a640 1 -- 172.21.15.53:0/3050367894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c005310 con 0x7f7d50004d00 2023-10-21T17:38:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/3050367894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d50004d00 msgr2=0x7f7d50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/3050367894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d50004d00 0x7f7d50005180 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c001e50 tx=0x7f7d500057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6a94d640 1 -- 172.21.15.53:0/3050367894 reap_dead start 2023-10-21T17:38:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/3050367894 shutdown_connections 2023-10-21T17:38:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/3050367894 >> 172.21.15.53:0/3050367894 conn(0x7f7d6406a5d0 msgr2=0x7f7d640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/3050367894 shutdown_connections 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.065+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/3050367894 wait complete. 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.067+0000 7f7d6c3b0640 1 Processor -- start 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.067+0000 7f7d6c3b0640 1 -- start start 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d50004d00 0x7f7d641475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 0x7f7d6415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640ce240 con 0x7f7d641f5790 2023-10-21T17:38:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641f5c10 con 0x7f7d640cddc0 2023-10-21T17:38:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6c3b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6414ab60 con 0x7f7d50004d00 2023-10-21T17:38:45.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6994b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6994b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48896/0 (socket says 172.21.15.53:48896) 2023-10-21T17:38:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.068+0000 7f7d6994b640 1 -- 172.21.15.53:0/2249696180 learned_addr learned my addr 172.21.15.53:0/2249696180 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 0x7f7d6415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 -- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d50004d00 msgr2=0x7f7d641475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d50004d00 0x7f7d641475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 -- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 msgr2=0x7f7d6415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 0x7f7d6415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 -- 172.21.15.53:0/2249696180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6414ae40 con 0x7f7d641f5790 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6a94d640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 0x7f7d6415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6994b640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f7d54011590 tx=0x7f7d54011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54004220 con 0x7f7d641f5790 2023-10-21T17:38:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d54004380 con 0x7f7d641f5790 2023-10-21T17:38:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54015890 con 0x7f7d641f5790 2023-10-21T17:38:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6414b120 con 0x7f7d641f5790 2023-10-21T17:38:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.069+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6414b6e0 con 0x7f7d641f5790 2023-10-21T17:38:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.070+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6404e160 con 0x7f7d641f5790 2023-10-21T17:38:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.072+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7d540159f0 con 0x7f7d641f5790 2023-10-21T17:38:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.072+0000 7f7d5affd640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 0x7f7d4806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.073+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7d54089a50 con 0x7f7d641f5790 2023-10-21T17:38:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.073+0000 7f7d6a14c640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 0x7f7d4806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.074+0000 7f7d6a14c640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 0x7f7d4806d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d60001eb0 tx=0x7f7d60009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.077+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d540548c0 con 0x7f7d641f5790 2023-10-21T17:38:45.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.267+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7d640ce7e0 con 0x7f7d641f5790 2023-10-21T17:38:45.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.504+0000 7f7d5affd640 1 -- 172.21.15.53:0/2249696180 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v234) v1 ==== 144+0+0 (secure 0 0 0) 0x7f7d540220b0 con 0x7f7d641f5790 2023-10-21T17:38:45.507 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_metadata' created 2023-10-21T17:38:45.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 msgr2=0x7f7d4806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 0x7f7d4806d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d60001eb0 tx=0x7f7d60009040 comp rx=0 tx=0).stop 2023-10-21T17:38:45.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 msgr2=0x7f7d640cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f7d54011590 tx=0x7f7d54011a50 comp rx=0 tx=0).stop 2023-10-21T17:38:45.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 shutdown_connections 2023-10-21T17:38:45.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d640cddc0 0x7f7d6415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d641f5790 0x7f7d640cd880 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f7d4806af90 0x7f7d4806d450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 --2- 172.21.15.53:0/2249696180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d50004d00 0x7f7d641475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 >> 172.21.15.53:0/2249696180 conn(0x7f7d6406a5d0 msgr2=0x7f7d640cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:45.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.508+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 shutdown_connections 2023-10-21T17:38:45.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.509+0000 7f7d6c3b0640 1 -- 172.21.15.53:0/2249696180 wait complete. 2023-10-21T17:38:45.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:38:45.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.746+0000 7f937afef640 1 Processor -- start 2023-10-21T17:38:45.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.746+0000 7f937afef640 1 -- start start 2023-10-21T17:38:45.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.748+0000 7f937afef640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93740c2db0 0x7f93740cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.748+0000 7f937afef640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.748+0000 7f937afef640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93740aeb50 con 0x7f9374161030 2023-10-21T17:38:45.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.748+0000 7f937afef640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93740cca90 con 0x7f93740c2db0 2023-10-21T17:38:45.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.748+0000 7f937afef640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93740cf0a0 con 0x7f9374161410 2023-10-21T17:38:45.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f9379fed640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9374161030 0x7f93740c26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59346/0 (socket says 172.21.15.53:59346) 2023-10-21T17:38:45.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f9379fed640 1 -- 172.21.15.53:0/3689945765 learned_addr learned my addr 172.21.15.53:0/3689945765 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:45.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f93797ec640 1 --2- 172.21.15.53:0/3689945765 >> v2:172.21.15.53:3300/0 conn(0x7f93740c2db0 0x7f93740cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f9378feb640 1 -- 172.21.15.53:0/3689945765 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887790764 0 0) 0x7f93740aeb50 con 0x7f9374161030 2023-10-21T17:38:45.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f9378feb640 1 -- 172.21.15.53:0/3689945765 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9360003520 con 0x7f9374161030 2023-10-21T17:38:45.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f937a7ee640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f93797ec640 1 -- 172.21.15.53:0/3689945765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 msgr2=0x7f93740aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:45.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f93797ec640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f93797ec640 1 -- 172.21.15.53:0/3689945765 >> v1:172.21.15.53:6789/0 conn(0x7f9374161030 legacy=0x7f93740c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f93797ec640 1 -- 172.21.15.53:0/3689945765 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93740cc0f0 con 0x7f93740c2db0 2023-10-21T17:38:45.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.749+0000 7f937a7ee640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:45.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f93797ec640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740cfde0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f93640097a0 tx=0x7f936402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a499717c99703984 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f9378feb640 1 -- 172.21.15.53:0/3689945765 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364031780 con 0x7f93740c2db0 2023-10-21T17:38:45.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f9378feb640 1 -- 172.21.15.53:0/3689945765 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93640318e0 con 0x7f93740c2db0 2023-10-21T17:38:45.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f9378feb640 1 -- 172.21.15.53:0/3689945765 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364031c50 con 0x7f93740c2db0 2023-10-21T17:38:45.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 -- 172.21.15.53:0/3689945765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 msgr2=0x7f93740cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740cfde0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f93640097a0 tx=0x7f936402b820 comp rx=0 tx=0).stop 2023-10-21T17:38:45.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 -- 172.21.15.53:0/3689945765 shutdown_connections 2023-10-21T17:38:45.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 --2- 172.21.15.53:0/3689945765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740cfde0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.751+0000 7f937afef640 1 -- 172.21.15.53:0/3689945765 >> 172.21.15.53:0/3689945765 conn(0x7f9374098a50 msgr2=0x7f93740cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:45.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.752+0000 7f937afef640 1 -- 172.21.15.53:0/3689945765 shutdown_connections 2023-10-21T17:38:45.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.752+0000 7f937afef640 1 -- 172.21.15.53:0/3689945765 wait complete. 2023-10-21T17:38:45.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.753+0000 7f937afef640 1 Processor -- start 2023-10-21T17:38:45.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.754+0000 7f937afef640 1 -- start start 2023-10-21T17:38:45.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.754+0000 7f937afef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.754+0000 7f937afef640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9374161030 0x7f9374164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937afef640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937afef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b29a0 con 0x7f93740c2db0 2023-10-21T17:38:45.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937afef640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93740b3370 con 0x7f9374161030 2023-10-21T17:38:45.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937afef640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9374162820 con 0x7f9374161410 2023-10-21T17:38:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48912/0 (socket says 172.21.15.53:48912) 2023-10-21T17:38:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 -- 172.21.15.53:0/715895503 learned_addr learned my addr 172.21.15.53:0/715895503 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937a7ee640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 -- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 msgr2=0x7f93740ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 -- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9374161030 msgr2=0x7f9374164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9374161030 0x7f9374164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f9379fed640 1 -- 172.21.15.53:0/715895503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364009400 con 0x7f93740c2db0 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.755+0000 7f937a7ee640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:45.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f9379fed640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f937000c400 tx=0x7f937000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700074e0 con 0x7f93740c2db0 2023-10-21T17:38:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9370007640 con 0x7f93740c2db0 2023-10-21T17:38:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700059e0 con 0x7f93740c2db0 2023-10-21T17:38:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9374162540 con 0x7f93740c2db0 2023-10-21T17:38:45.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.756+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93740b2630 con 0x7f93740c2db0 2023-10-21T17:38:45.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.757+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937406ef30 con 0x7f93740c2db0 2023-10-21T17:38:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.759+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9370005b40 con 0x7f93740c2db0 2023-10-21T17:38:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.760+0000 7f936a7fc640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 0x7f935806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.760+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f93700894a0 con 0x7f93740c2db0 2023-10-21T17:38:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.760+0000 7f93797ec640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 0x7f935806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:45.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.762+0000 7f93797ec640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 0x7f935806d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9364036000 tx=0x7f9364030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:45.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.764+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93700541f0 con 0x7f93740c2db0 2023-10-21T17:38:45.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:45.960+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f93740b27c0 con 0x7f93740c2db0 2023-10-21T17:38:46.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.523+0000 7f936a7fc640 1 -- 172.21.15.53:0/715895503 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v235) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9370057810 con 0x7f93740c2db0 2023-10-21T17:38:46.525 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_data' created 2023-10-21T17:38:46.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.526+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 msgr2=0x7f935806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.526+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 0x7f935806d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9364036000 tx=0x7f9364030000 comp rx=0 tx=0).stop 2023-10-21T17:38:46.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.526+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 msgr2=0x7f93740b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.526+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f937000c400 tx=0x7f937000c8c0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 shutdown_connections 2023-10-21T17:38:46.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f935806af90 0x7f935806d450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9374161410 0x7f93740ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9374161030 0x7f9374164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 --2- 172.21.15.53:0/715895503 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93740c2db0 0x7f93740b2e30 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.527+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 >> 172.21.15.53:0/715895503 conn(0x7f9374098a50 msgr2=0x7f93740b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:46.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.528+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 shutdown_connections 2023-10-21T17:38:46.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.528+0000 7f937afef640 1 -- 172.21.15.53:0/715895503 wait complete. 2023-10-21T17:38:46.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-21T17:38:46.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.758+0000 7f4c50eff640 1 Processor -- start 2023-10-21T17:38:46.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.758+0000 7f4c50eff640 1 -- start start 2023-10-21T17:38:46.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c50eff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0aff70 0x7f4c4c0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c50eff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c4c0c18d0 0x7f4c4c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c50eff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c161590 con 0x7f4c4c164d10 2023-10-21T17:38:46.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c50eff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c0cc570 con 0x7f4c4c0c18d0 2023-10-21T17:38:46.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c50eff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0c94b0 con 0x7f4c4c0aff70 2023-10-21T17:38:46.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c4bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4c4c164d10 0x7f4c4c0ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59362/0 (socket says 172.21.15.53:59362) 2023-10-21T17:38:46.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c4bfff640 1 -- 172.21.15.53:0/3658727430 learned_addr learned my addr 172.21.15.53:0/3658727430 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:46.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.760+0000 7f4c4affd640 1 --2- 172.21.15.53:0/3658727430 >> v2:172.21.15.53:3300/0 conn(0x7f4c4c0c18d0 0x7f4c4c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.761+0000 7f4c4b7fe640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0aff70 0x7f4c4c0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.761+0000 7f4c4affd640 1 -- 172.21.15.53:0/3658727430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0aff70 msgr2=0x7f4c4c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:46.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.761+0000 7f4c4affd640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0aff70 0x7f4c4c0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.761+0000 7f4c4affd640 1 -- 172.21.15.53:0/3658727430 >> v1:172.21.15.53:6789/0 conn(0x7f4c4c164d10 legacy=0x7f4c4c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.761+0000 7f4c4affd640 1 -- 172.21.15.53:0/3658727430 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c0c9790 con 0x7f4c4c0c18d0 2023-10-21T17:38:46.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.762+0000 7f4c4affd640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c0c18d0 0x7f4c4c0c8b40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0097a0 tx=0x7f4c3c02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be52b6b157856e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:46.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.762+0000 7f4c4a7fc640 1 -- 172.21.15.53:0/3658727430 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c030710 con 0x7f4c4c0c18d0 2023-10-21T17:38:46.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.762+0000 7f4c4a7fc640 1 -- 172.21.15.53:0/3658727430 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c02e050 con 0x7f4c4c0c18d0 2023-10-21T17:38:46.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.762+0000 7f4c4a7fc640 1 -- 172.21.15.53:0/3658727430 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c030a90 con 0x7f4c4c0c18d0 2023-10-21T17:38:46.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 -- 172.21.15.53:0/3658727430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c0c18d0 msgr2=0x7f4c4c0c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c0c18d0 0x7f4c4c0c8b40 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0097a0 tx=0x7f4c3c02b660 comp rx=0 tx=0).stop 2023-10-21T17:38:46.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 -- 172.21.15.53:0/3658727430 shutdown_connections 2023-10-21T17:38:46.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c0c18d0 0x7f4c4c0c8b40 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 --2- 172.21.15.53:0/3658727430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0aff70 0x7f4c4c0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 -- 172.21.15.53:0/3658727430 >> 172.21.15.53:0/3658727430 conn(0x7f4c4c06c050 msgr2=0x7f4c4c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:46.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 -- 172.21.15.53:0/3658727430 shutdown_connections 2023-10-21T17:38:46.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.763+0000 7f4c50eff640 1 -- 172.21.15.53:0/3658727430 wait complete. 2023-10-21T17:38:46.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.765+0000 7f4c50eff640 1 Processor -- start 2023-10-21T17:38:46.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.765+0000 7f4c50eff640 1 -- start start 2023-10-21T17:38:46.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c4c0aff70 0x7f4c4c1f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 0x7f4c4c1f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c1f6850 con 0x7f4c4c164d10 2023-10-21T17:38:46.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c153d20 con 0x7f4c4c0aff70 2023-10-21T17:38:46.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.766+0000 7f4c50eff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0c2ed0 con 0x7f4c4c0c29d0 2023-10-21T17:38:46.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48936/0 (socket says 172.21.15.53:48936) 2023-10-21T17:38:46.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 -- 172.21.15.53:0/552541818 learned_addr learned my addr 172.21.15.53:0/552541818 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:46.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4b7fe640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c4c0aff70 0x7f4c4c1f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4bfff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 0x7f4c4c1f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 -- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 msgr2=0x7f4c4c1f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 0x7f4c4c1f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 -- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c4c0aff70 msgr2=0x7f4c4c1f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:46.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c4c0aff70 0x7f4c4c1f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:46.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4affd640 1 -- 172.21.15.53:0/552541818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009400 con 0x7f4c4c164d10 2023-10-21T17:38:46.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.767+0000 7f4c4bfff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 0x7f4c4c1f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:46.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c4affd640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02bbf0 tx=0x7f4c3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:46.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c032690 con 0x7f4c4c164d10 2023-10-21T17:38:46.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c0327f0 con 0x7f4c4c164d10 2023-10-21T17:38:46.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c032ba0 con 0x7f4c4c164d10 2023-10-21T17:38:46.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c1536d0 con 0x7f4c4c164d10 2023-10-21T17:38:46.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.768+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c0c31b0 con 0x7f4c4c164d10 2023-10-21T17:38:46.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.769+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c074150 con 0x7f4c4c164d10 2023-10-21T17:38:46.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.771+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c3c039070 con 0x7f4c4c164d10 2023-10-21T17:38:46.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.772+0000 7f4c43fff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 0x7f4c2c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:46.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.772+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f4c3c0ab8b0 con 0x7f4c4c164d10 2023-10-21T17:38:46.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.772+0000 7f4c4b7fe640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 0x7f4c2c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:46.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.774+0000 7f4c4b7fe640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 0x7f4c2c06d480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006fd0 tx=0x7f4c34008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:46.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:46.777+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c3c076420 con 0x7f4c4c164d10 2023-10-21T17:38:47.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.010+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f4c4c0b4470 con 0x7f4c4c164d10 2023-10-21T17:38:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.539+0000 7f4c43fff640 1 -- 172.21.15.53:0/552541818 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 29 and data pool 30 v115) v1 ==== 183+0+0 (secure 0 0 0) 0x7f4c3c0400d0 con 0x7f4c4c164d10 2023-10-21T17:38:47.546 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 29 and data pool 30 2023-10-21T17:38:47.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.547+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 msgr2=0x7f4c2c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.547+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 0x7f4c2c06d480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006fd0 tx=0x7f4c34008040 comp rx=0 tx=0).stop 2023-10-21T17:38:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.547+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 msgr2=0x7f4c4c0c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.547+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c02bbf0 tx=0x7f4c3c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 shutdown_connections 2023-10-21T17:38:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f4c2c06afc0 0x7f4c2c06d480 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006fd0 tx=0x7f4c34008040 comp rx=0 tx=0).stop 2023-10-21T17:38:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c4c0c29d0 0x7f4c4c1f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c164d10 0x7f4c4c0c4860 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 --2- 172.21.15.53:0/552541818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c4c0aff70 0x7f4c4c1f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.549+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 >> 172.21.15.53:0/552541818 conn(0x7f4c4c06c050 msgr2=0x7f4c4c0ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:47.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.552+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 shutdown_connections 2023-10-21T17:38:47.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.554+0000 7f4c50eff640 1 -- 172.21.15.53:0/552541818 wait complete. 2023-10-21T17:38:47.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.765+0000 7fbe49059640 1 Processor -- start 2023-10-21T17:38:47.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.765+0000 7fbe49059640 1 -- start start 2023-10-21T17:38:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe49059640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 0x7fbe440b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe49059640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4415f7d0 con 0x7fbe4415ab30 2023-10-21T17:38:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe49059640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4415efb0 con 0x7fbe4415af10 2023-10-21T17:38:47.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe49059640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4415f210 con 0x7fbe440adc70 2023-10-21T17:38:47.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe43fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fbe4415ab30 0x7fbe4415e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59364/0 (socket says 172.21.15.53:59364) 2023-10-21T17:38:47.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe43fff640 1 -- 172.21.15.53:0/29907460 learned_addr learned my addr 172.21.15.53:0/29907460 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:47.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670320350 0 0) 0x7fbe4415f7d0 con 0x7fbe4415ab30 2023-10-21T17:38:47.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe437fe640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 0x7fbe440b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:47.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe30003520 con 0x7fbe4415ab30 2023-10-21T17:38:47.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527469298 0 0) 0x7fbe30003520 con 0x7fbe4415ab30 2023-10-21T17:38:47.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 msgr2=0x7fbe440b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:47.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 0x7fbe440b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 >> v1:172.21.15.124:6789/0 conn(0x7fbe4415af10 legacy=0x7fbe4406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4415f4f0 con 0x7fbe4415ab30 2023-10-21T17:38:47.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fbe34001690 con 0x7fbe4415ab30 2023-10-21T17:38:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 >> v1:172.21.15.53:6789/0 conn(0x7fbe4415ab30 legacy=0x7fbe4415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe300039a0 0x7fbe30003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 0x7fbe300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe30004d00 0x7fbe30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4415f210 con 0x7fbe300039a0 2023-10-21T17:38:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3000bb10 con 0x7fbe30004d00 2023-10-21T17:38:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3000bcd0 con 0x7fbe440adc70 2023-10-21T17:38:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.767+0000 7fbe43fff640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe300039a0 0x7fbe30003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.768+0000 7fbe43fff640 1 -- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 msgr2=0x7fbe300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:47.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.768+0000 7fbe43fff640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe440adc70 0x7fbe300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.768+0000 7fbe43fff640 1 -- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe30004d00 msgr2=0x7fbe30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:47.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.768+0000 7fbe43fff640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe30004d00 0x7fbe30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.768+0000 7fbe43fff640 1 -- 172.21.15.53:0/29907460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30003520 con 0x7fbe300039a0 2023-10-21T17:38:47.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe43fff640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe300039a0 0x7fbe30003dc0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415f960 tx=0x7fbe300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869f7b3b862b6755 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:47.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34005ad0 con 0x7fbe300039a0 2023-10-21T17:38:47.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34005c30 con 0x7fbe300039a0 2023-10-21T17:38:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe42ffd640 1 -- 172.21.15.53:0/29907460 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3402c890 con 0x7fbe300039a0 2023-10-21T17:38:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 -- 172.21.15.53:0/29907460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe300039a0 msgr2=0x7fbe30003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 --2- 172.21.15.53:0/29907460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe300039a0 0x7fbe30003dc0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415f960 tx=0x7fbe300057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe48858640 1 -- 172.21.15.53:0/29907460 reap_dead start 2023-10-21T17:38:47.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 -- 172.21.15.53:0/29907460 shutdown_connections 2023-10-21T17:38:47.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 -- 172.21.15.53:0/29907460 >> 172.21.15.53:0/29907460 conn(0x7fbe440cb0c0 msgr2=0x7fbe440cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:47.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 -- 172.21.15.53:0/29907460 shutdown_connections 2023-10-21T17:38:47.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.770+0000 7fbe49059640 1 -- 172.21.15.53:0/29907460 wait complete. 2023-10-21T17:38:47.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.771+0000 7fbe49059640 1 Processor -- start 2023-10-21T17:38:47.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.771+0000 7fbe49059640 1 -- start start 2023-10-21T17:38:47.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe300039a0 0x7fbe44155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe4415b320 0x7fbe440c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44156ab0 con 0x7fbe440c6070 2023-10-21T17:38:47.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44158490 con 0x7fbe4415b320 2023-10-21T17:38:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.772+0000 7fbe49059640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44158770 con 0x7fbe300039a0 2023-10-21T17:38:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48950/0 (socket says 172.21.15.53:48950) 2023-10-21T17:38:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 -- 172.21.15.53:0/844566337 learned_addr learned my addr 172.21.15.53:0/844566337 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:47.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 -- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe300039a0 msgr2=0x7fbe44155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe300039a0 0x7fbe44155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 -- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe4415b320 msgr2=0x7fbe440c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe4415b320 0x7fbe440c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 -- 172.21.15.53:0/844566337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44156d00 con 0x7fbe440c6070 2023-10-21T17:38:47.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.773+0000 7fbe48858640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415b2f0 tx=0x7fbe3c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.774+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c005d40 con 0x7fbe440c6070 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.774+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c00a610 con 0x7fbe440c6070 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.774+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00a9c0 con 0x7fbe440c6070 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.774+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe44156fe0 con 0x7fbe440c6070 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.774+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe440c64f0 con 0x7fbe440c6070 2023-10-21T17:38:47.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.775+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4404e160 con 0x7fbe440c6070 2023-10-21T17:38:47.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.776+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbe3c00ab20 con 0x7fbe440c6070 2023-10-21T17:38:47.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.777+0000 7fbe40ff9640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 0x7fbe2406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:47.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.777+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fbe3c088d10 con 0x7fbe440c6070 2023-10-21T17:38:47.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.777+0000 7fbe43fff640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 0x7fbe2406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:47.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.778+0000 7fbe43fff640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 0x7fbe2406d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415f960 tx=0x7fbe34007000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:47.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.781+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe3c053790 con 0x7fbe440c6070 2023-10-21T17:38:47.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.974+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe440ada50 con 0x7fbe440c6070 2023-10-21T17:38:47.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.978+0000 7fbe40ff9640 1 -- 172.21.15.53:0/844566337 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20627 (secure 0 0 0) 0x7fbe3c017090 con 0x7fbe440c6070 2023-10-21T17:38:47.980 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:47.980 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":236,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:47.013001+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 msgr2=0x7fbe2406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 0x7fbe2406d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415f960 tx=0x7fbe34007000 comp rx=0 tx=0).stop 2023-10-21T17:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 msgr2=0x7fbe441485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fbe4415b2f0 tx=0x7fbe3c00cb90 comp rx=0 tx=0).stop 2023-10-21T17:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 shutdown_connections 2023-10-21T17:38:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe440c6070 0x7fbe441485e0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe4415b320 0x7fbe440c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fbe2406afc0 0x7fbe2406d480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 --2- 172.21.15.53:0/844566337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe300039a0 0x7fbe44155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 >> 172.21.15.53:0/844566337 conn(0x7fbe440cb0c0 msgr2=0x7fbe440b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 shutdown_connections 2023-10-21T17:38:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:47.981+0000 7fbe49059640 1 -- 172.21.15.53:0/844566337 wait complete. 2023-10-21T17:38:48.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.204+0000 7f2562ba0640 1 Processor -- start 2023-10-21T17:38:48.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.204+0000 7f2562ba0640 1 -- start start 2023-10-21T17:38:48.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.206+0000 7f2562ba0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 0x7f255c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.206+0000 7f2562ba0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f255c0adc70 0x7f255c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.206+0000 7f2562ba0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c15f7d0 con 0x7f255c15af10 2023-10-21T17:38:48.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.206+0000 7f2562ba0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c15f210 con 0x7f255c0adc70 2023-10-21T17:38:48.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.206+0000 7f2562ba0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c15f4f0 con 0x7f255c15ab30 2023-10-21T17:38:48.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.207+0000 7f2561b9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 0x7f255c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.207+0000 7f256139d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f255c0adc70 0x7f255c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.207+0000 7f2561b9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 0x7f255c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:50030/0 (socket says 172.21.15.53:50030) 2023-10-21T17:38:48.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.207+0000 7f2561b9e640 1 -- 172.21.15.53:0/1660980850 learned_addr learned my addr 172.21.15.53:0/1660980850 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:48.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.208+0000 7f256139d640 1 -- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 msgr2=0x7f255c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:48.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.208+0000 7f256139d640 1 --2- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 0x7f255c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.208+0000 7f256139d640 1 -- 172.21.15.53:0/1660980850 >> v1:172.21.15.124:6789/0 conn(0x7f255c15af10 legacy=0x7f255c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.208+0000 7f256139d640 1 -- 172.21.15.53:0/1660980850 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c06c550 con 0x7f255c0adc70 2023-10-21T17:38:48.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.209+0000 7f256139d640 1 --2- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0b00c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f254c0097a0 tx=0x7f254c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71db6b90317984a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.210+0000 7f2560b9c640 1 -- 172.21.15.53:0/1660980850 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c02d040 con 0x7f255c0adc70 2023-10-21T17:38:48.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.210+0000 7f2560b9c640 1 -- 172.21.15.53:0/1660980850 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c031810 con 0x7f255c0adc70 2023-10-21T17:38:48.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.210+0000 7f2560b9c640 1 -- 172.21.15.53:0/1660980850 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c031ba0 con 0x7f255c0adc70 2023-10-21T17:38:48.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 -- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 msgr2=0x7f255c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 --2- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0b00c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f254c0097a0 tx=0x7f254c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 -- 172.21.15.53:0/1660980850 shutdown_connections 2023-10-21T17:38:48.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 --2- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0b00c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 --2- 172.21.15.53:0/1660980850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15ab30 0x7f255c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 -- 172.21.15.53:0/1660980850 >> 172.21.15.53:0/1660980850 conn(0x7f255c0cb0c0 msgr2=0x7f255c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:48.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 -- 172.21.15.53:0/1660980850 shutdown_connections 2023-10-21T17:38:48.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.211+0000 7f2562ba0640 1 -- 172.21.15.53:0/1660980850 wait complete. 2023-10-21T17:38:48.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.213+0000 7f2562ba0640 1 Processor -- start 2023-10-21T17:38:48.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.213+0000 7f2562ba0640 1 -- start start 2023-10-21T17:38:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 0x7f255c1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0acc10 con 0x7f255c15ab30 2023-10-21T17:38:48.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0cc830 con 0x7f255c0adc70 2023-10-21T17:38:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2562ba0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0ccb10 con 0x7f255c15af10 2023-10-21T17:38:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48964/0 (socket says 172.21.15.53:48964) 2023-10-21T17:38:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 -- 172.21.15.53:0/2222200841 learned_addr learned my addr 172.21.15.53:0/2222200841 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:48.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256239f640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 0x7f255c1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f2561b9e640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 -- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 msgr2=0x7f255c1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 0x7f255c1484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.214+0000 7f256139d640 1 -- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 msgr2=0x7f255c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f256139d640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f256139d640 1 -- 172.21.15.53:0/2222200841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c009400 con 0x7f255c15ab30 2023-10-21T17:38:48.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f256239f640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 0x7f255c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:48.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f256139d640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098c0 tx=0x7f254c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c03ace0 con 0x7f255c15ab30 2023-10-21T17:38:48.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c03ae60 con 0x7f255c15ab30 2023-10-21T17:38:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c044640 con 0x7f255c15ab30 2023-10-21T17:38:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c0ccdf0 con 0x7f255c15ab30 2023-10-21T17:38:48.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.215+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0cd3b0 con 0x7f255c15ab30 2023-10-21T17:38:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.216+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c04e160 con 0x7f255c15ab30 2023-10-21T17:38:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.217+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f254c04d440 con 0x7f255c15ab30 2023-10-21T17:38:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.218+0000 7f25527fc640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 0x7f254006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.218+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f254c0ac140 con 0x7f255c15ab30 2023-10-21T17:38:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.219+0000 7f2561b9e640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 0x7f254006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.220+0000 7f2561b9e640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 0x7f254006d480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2558009aa0 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.222+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f254c076a10 con 0x7f255c15ab30 2023-10-21T17:38:48.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.419+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f255c0cf440 con 0x7f255c15ab30 2023-10-21T17:38:48.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.423+0000 7f25527fc640 1 -- 172.21.15.53:0/2222200841 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20627 (secure 0 0 0) 0x7f254c07a030 con 0x7f255c15ab30 2023-10-21T17:38:48.425 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:48.425 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":236,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:47.013001+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:48.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 msgr2=0x7f254006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 0x7f254006d480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2558009aa0 tx=0x7f2558008040 comp rx=0 tx=0).stop 2023-10-21T17:38:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 msgr2=0x7f255c147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f254c0098c0 tx=0x7f254c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 shutdown_connections 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f254006afc0 0x7f254006d480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f255c15af10 0x7f255c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f255c15ab30 0x7f255c147940 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 --2- 172.21.15.53:0/2222200841 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f255c0adc70 0x7f255c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 >> 172.21.15.53:0/2222200841 conn(0x7f255c0cb0c0 msgr2=0x7f255c1597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 shutdown_connections 2023-10-21T17:38:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.426+0000 7f2562ba0640 1 -- 172.21.15.53:0/2222200841 wait complete. 2023-10-21T17:38:48.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-21T17:38:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.666+0000 7fb52871d640 1 Processor -- start 2023-10-21T17:38:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.666+0000 7fb52871d640 1 -- start start 2023-10-21T17:38:48.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.667+0000 7fb52871d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb5200afbe0 0x7fb5200b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.667+0000 7fb52871d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb52006dfc0 0x7fb520070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb52871d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb520071070 0x7fb5200cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb52871d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200b2620 con 0x7fb52006dfc0 2023-10-21T17:38:48.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb52871d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200b2060 con 0x7fb5200afbe0 2023-10-21T17:38:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb52871d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200b2340 con 0x7fb520071070 2023-10-21T17:38:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb52006dfc0 0x7fb520070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb52006dfc0 0x7fb520070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48980/0 (socket says 172.21.15.53:48980) 2023-10-21T17:38:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 -- 172.21.15.53:0/2756211377 learned_addr learned my addr 172.21.15.53:0/2756211377 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 -- 172.21.15.53:0/2756211377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb520071070 msgr2=0x7fb5200cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2756211377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb520071070 0x7fb5200cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 -- 172.21.15.53:0/2756211377 >> v2:172.21.15.124:3300/0 conn(0x7fb5200afbe0 msgr2=0x7fb5200b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2756211377 >> v2:172.21.15.124:3300/0 conn(0x7fb5200afbe0 0x7fb5200b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.668+0000 7fb525cb8640 1 -- 172.21.15.53:0/2756211377 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520070350 con 0x7fb52006dfc0 2023-10-21T17:38:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.670+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2756211377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52006dfc0 0x7fb520070b30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fb52006e640 tx=0x7fb51002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8e98c92d6962df server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.670+0000 7fb5254b7640 1 -- 172.21.15.53:0/2756211377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510031860 con 0x7fb52006dfc0 2023-10-21T17:38:48.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.670+0000 7fb5254b7640 1 -- 172.21.15.53:0/2756211377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5100319c0 con 0x7fb52006dfc0 2023-10-21T17:38:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.670+0000 7fb5254b7640 1 -- 172.21.15.53:0/2756211377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510031d50 con 0x7fb52006dfc0 2023-10-21T17:38:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 -- 172.21.15.53:0/2756211377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52006dfc0 msgr2=0x7fb520070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 --2- 172.21.15.53:0/2756211377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52006dfc0 0x7fb520070b30 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fb52006e640 tx=0x7fb51002b900 comp rx=0 tx=0).stop 2023-10-21T17:38:48.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 -- 172.21.15.53:0/2756211377 shutdown_connections 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 --2- 172.21.15.53:0/2756211377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb520071070 0x7fb5200cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 --2- 172.21.15.53:0/2756211377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb52006dfc0 0x7fb520070b30 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 --2- 172.21.15.53:0/2756211377 >> v2:172.21.15.124:3300/0 conn(0x7fb5200afbe0 0x7fb5200b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 -- 172.21.15.53:0/2756211377 >> 172.21.15.53:0/2756211377 conn(0x7fb52006a5d0 msgr2=0x7fb5200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 -- 172.21.15.53:0/2756211377 shutdown_connections 2023-10-21T17:38:48.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.671+0000 7fb52871d640 1 -- 172.21.15.53:0/2756211377 wait complete. 2023-10-21T17:38:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.673+0000 7fb52871d640 1 Processor -- start 2023-10-21T17:38:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.673+0000 7fb52871d640 1 -- start start 2023-10-21T17:38:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5200afbe0 0x7fb52015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 0x7fb520145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015e4c0 con 0x7fb520071070 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015c0c0 con 0x7fb52015ce40 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb52871d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015e220 con 0x7fb5200afbe0 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb5264b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb5264b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48982/0 (socket says 172.21.15.53:48982) 2023-10-21T17:38:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.674+0000 7fb5264b9640 1 -- 172.21.15.53:0/2719710607 learned_addr learned my addr 172.21.15.53:0/2719710607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 -- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5200afbe0 msgr2=0x7fb52015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5200afbe0 0x7fb52015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5200afbe0 0x7fb52015dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb526cba640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 0x7fb520145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 -- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 msgr2=0x7fb520145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 0x7fb520145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 -- 172.21.15.53:0/2719710607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100094d0 con 0x7fb520071070 2023-10-21T17:38:48.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb526cba640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 0x7fb520145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb5264b9640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00b410 tx=0x7fb51c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011830 con 0x7fb520071070 2023-10-21T17:38:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.675+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c0119b0 con 0x7fb520071070 2023-10-21T17:38:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.676+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011d60 con 0x7fb520071070 2023-10-21T17:38:48.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.676+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52015c360 con 0x7fb520071070 2023-10-21T17:38:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.676+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52015c920 con 0x7fb520071070 2023-10-21T17:38:48.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.677+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52004e160 con 0x7fb520071070 2023-10-21T17:38:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.678+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb51c01a400 con 0x7fb520071070 2023-10-21T17:38:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.679+0000 7fb516ffd640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 0x7fb50406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.679+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fb51c089dc0 con 0x7fb520071070 2023-10-21T17:38:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.680+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 0x7fb50406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:48.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.681+0000 7fb525cb8640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 0x7fb50406d450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb51003c040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:48.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.684+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb51c054830 con 0x7fb520071070 2023-10-21T17:38:48.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:48.917+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5200c5280 con 0x7fb520071070 2023-10-21T17:38:49.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.551+0000 7fb516ffd640 1 -- 172.21.15.53:0/2719710607 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v117) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb51c015020 con 0x7fb520071070 2023-10-21T17:38:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.554+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 msgr2=0x7fb50406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.554+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 0x7fb50406d450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb510036000 tx=0x7fb51003c040 comp rx=0 tx=0).stop 2023-10-21T17:38:49.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.554+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 msgr2=0x7fb5200cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.554+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00b410 tx=0x7fb51c00b8d0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 shutdown_connections 2023-10-21T17:38:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb52015ce40 0x7fb520145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fb50406af90 0x7fb50406d450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5200afbe0 0x7fb52015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 --2- 172.21.15.53:0/2719710607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb520071070 0x7fb5200cd960 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 >> 172.21.15.53:0/2719710607 conn(0x7fb52006a5d0 msgr2=0x7fb5200c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 shutdown_connections 2023-10-21T17:38:49.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.555+0000 7fb52871d640 1 -- 172.21.15.53:0/2719710607 wait complete. 2023-10-21T17:38:49.577 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:38:49.578 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-21T17:38:49.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.808+0000 7f9ac5b1a640 1 Processor -- start 2023-10-21T17:38:49.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.808+0000 7f9ac5b1a640 1 -- start start 2023-10-21T17:38:49.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.809+0000 7f9ac5b1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9ac015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.809+0000 7f9ac5b1a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac00cab60 con 0x7f9ac015a820 2023-10-21T17:38:49.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.809+0000 7f9ac5b1a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac00cb960 con 0x7f9ac00b0680 2023-10-21T17:38:49.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.810+0000 7f9ac5b1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00cbdf0 con 0x7f9ac00ca680 2023-10-21T17:38:49.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.810+0000 7f9ac4b18640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9ac015a820 0x7f9ac00aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59366/0 (socket says 172.21.15.53:59366) 2023-10-21T17:38:49.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.810+0000 7f9ac4b18640 1 -- 172.21.15.53:0/3461768609 learned_addr learned my addr 172.21.15.53:0/3461768609 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:49.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.810+0000 7f9ac5319640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9ac015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.810+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243023975 0 0) 0x7f9ac00cab60 con 0x7f9ac015a820 2023-10-21T17:38:49.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aac003520 con 0x7f9ac015a820 2023-10-21T17:38:49.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1802164603 0 0) 0x7f9aac003520 con 0x7f9ac015a820 2023-10-21T17:38:49.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 msgr2=0x7f9ac015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:49.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9ac015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 >> v1:172.21.15.124:6789/0 conn(0x7f9ac00b0680 legacy=0x7f9ac015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00cb030 con 0x7f9ac015a820 2023-10-21T17:38:49.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9ab0001670 con 0x7f9ac015a820 2023-10-21T17:38:49.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 >> v1:172.21.15.53:6789/0 conn(0x7f9ac015a820 legacy=0x7f9ac00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9aac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aac0047c0 0x7f9aac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9aac005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00cbdf0 con 0x7f9aac0039a0 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac00bb10 con 0x7f9aac0047c0 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac00bcd0 con 0x7f9ac00ca680 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.811+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9aac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9abffff640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aac0047c0 0x7f9aac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac5319640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9aac005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac4b18640 1 -- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 msgr2=0x7f9aac005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:49.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac00ca680 0x7f9aac005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac4b18640 1 -- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aac0047c0 msgr2=0x7f9aac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:49.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9aac0047c0 0x7f9aac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.812+0000 7f9ac4b18640 1 -- 172.21.15.53:0/3461768609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac003520 con 0x7f9aac0039a0 2023-10-21T17:38:49.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.813+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9aac003dc0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c0e0 tx=0x7f9aac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51398cadedcd29c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:49.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.813+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0005ad0 con 0x7f9aac0039a0 2023-10-21T17:38:49.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0005c30 con 0x7f9aac0039a0 2023-10-21T17:38:49.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9abf7fe640 1 -- 172.21.15.53:0/3461768609 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab002c810 con 0x7f9aac0039a0 2023-10-21T17:38:49.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/3461768609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 msgr2=0x7f9aac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/3461768609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9aac003dc0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c0e0 tx=0x7f9aac0057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5319640 1 -- 172.21.15.53:0/3461768609 reap_dead start 2023-10-21T17:38:49.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/3461768609 shutdown_connections 2023-10-21T17:38:49.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/3461768609 >> 172.21.15.53:0/3461768609 conn(0x7f9ac006c050 msgr2=0x7f9ac006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:49.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/3461768609 shutdown_connections 2023-10-21T17:38:49.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.814+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/3461768609 wait complete. 2023-10-21T17:38:49.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.816+0000 7f9ac5b1a640 1 Processor -- start 2023-10-21T17:38:49.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.816+0000 7f9ac5b1a640 1 -- start start 2023-10-21T17:38:49.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.817+0000 7f9ac5b1a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5b1a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 0x7f9ac01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5b1a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac00b2b40 0x7f9ac00b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5b1a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00b3500 con 0x7f9aac0039a0 2023-10-21T17:38:49.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5b1a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0157f90 con 0x7f9ac00b2b40 2023-10-21T17:38:49.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5b1a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac01f1ea0 con 0x7f9ac0156450 2023-10-21T17:38:49.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49000/0 (socket says 172.21.15.53:49000) 2023-10-21T17:38:49.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 -- 172.21.15.53:0/1839187584 learned_addr learned my addr 172.21.15.53:0/1839187584 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:49.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9abffff640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 0x7f9ac01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac5319640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac00b2b40 0x7f9ac00b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 -- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 msgr2=0x7f9ac01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 0x7f9ac01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 -- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac00b2b40 msgr2=0x7f9ac00b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac00b2b40 0x7f9ac00b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9ac4b18640 1 -- 172.21.15.53:0/1839187584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac0156ef0 con 0x7f9aac0039a0 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.818+0000 7f9abffff640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 0x7f9ac01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:49.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9ac4b18640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c200 tx=0x7f9ab0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:49.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab003c040 con 0x7f9aac0039a0 2023-10-21T17:38:49.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab002dcb0 con 0x7f9aac0039a0 2023-10-21T17:38:49.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0007270 con 0x7f9aac0039a0 2023-10-21T17:38:49.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac01571d0 con 0x7f9aac0039a0 2023-10-21T17:38:49.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.819+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac0157790 con 0x7f9aac0039a0 2023-10-21T17:38:49.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.820+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0074150 con 0x7f9aac0039a0 2023-10-21T17:38:49.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.822+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9ab0032070 con 0x7f9aac0039a0 2023-10-21T17:38:49.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.823+0000 7f9abd7fa640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 0x7f9aa006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:49.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.823+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f9ab00ab640 con 0x7f9aac0039a0 2023-10-21T17:38:49.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.824+0000 7f9abffff640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 0x7f9aa006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:49.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.825+0000 7f9abffff640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 0x7f9aa006d480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8007fd0 tx=0x7f9aa8009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:49.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:49.828+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ab0076150 con 0x7f9aac0039a0 2023-10-21T17:38:50.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.068+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9ac0159b90 con 0x7f9aac0039a0 2023-10-21T17:38:50.146 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:38:50.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:38:50.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.562+0000 7f9abd7fa640 1 -- 172.21.15.53:0/1839187584 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v118) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9ab0079770 con 0x7f9aac0039a0 2023-10-21T17:38:50.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 msgr2=0x7f9aa006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 0x7f9aa006d480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8007fd0 tx=0x7f9aa8009040 comp rx=0 tx=0).stop 2023-10-21T17:38:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 msgr2=0x7f9ac01f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c200 tx=0x7f9ab0001930 comp rx=0 tx=0).stop 2023-10-21T17:38:50.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 shutdown_connections 2023-10-21T17:38:50.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac00b2b40 0x7f9ac00b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9aa006afc0 0x7f9aa006d480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac0156450 0x7f9ac01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 --2- 172.21.15.53:0/1839187584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9aac0039a0 0x7f9ac01f1960 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 >> 172.21.15.53:0/1839187584 conn(0x7f9ac006c050 msgr2=0x7f9ac00adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:50.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.567+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 shutdown_connections 2023-10-21T17:38:50.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.568+0000 7f9ac5b1a640 1 -- 172.21.15.53:0/1839187584 wait complete. 2023-10-21T17:38:50.590 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-21T17:38:50.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:50.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.816+0000 7fcfc488f640 1 Processor -- start 2023-10-21T17:38:50.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.816+0000 7fcfc488f640 1 -- start start 2023-10-21T17:38:50.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.818+0000 7fcfc488f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 0x7fcfc00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.818+0000 7fcfc488f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 0x7fcfc00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.818+0000 7fcfc488f640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc015f7d0 con 0x7fcfc015ab30 2023-10-21T17:38:50.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.818+0000 7fcfc488f640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc015efb0 con 0x7fcfc00adc70 2023-10-21T17:38:50.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.818+0000 7fcfc488f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc015f210 con 0x7fcfc015af10 2023-10-21T17:38:50.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.819+0000 7fcfbf59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 0x7fcfc00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.819+0000 7fcfbed9d640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fcfc015ab30 0x7fcfc015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:43838/0 (socket says 172.21.15.53:43838) 2023-10-21T17:38:50.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.819+0000 7fcfbed9d640 1 -- 172.21.15.53:0/159080167 learned_addr learned my addr 172.21.15.53:0/159080167 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:50.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.819+0000 7fcfb7fff640 1 --2- 172.21.15.53:0/159080167 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 0x7fcfc00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfbf59e640 1 -- 172.21.15.53:0/159080167 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 msgr2=0x7fcfc00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:50.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfbf59e640 1 --2- 172.21.15.53:0/159080167 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 0x7fcfc00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfbf59e640 1 -- 172.21.15.53:0/159080167 >> v1:172.21.15.124:6789/0 conn(0x7fcfc015ab30 legacy=0x7fcfc015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfbf59e640 1 -- 172.21.15.53:0/159080167 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc015f4f0 con 0x7fcfc015af10 2023-10-21T17:38:50.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfbe59c640 1 -- 172.21.15.53:0/159080167 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217232826 0 0) 0x7fcfc015f7d0 con 0x7fcfc015ab30 2023-10-21T17:38:50.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.820+0000 7fcfb7fff640 1 --2- 172.21.15.53:0/159080167 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 0x7fcfc00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:38:50.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.821+0000 7fcfbf59e640 1 --2- 172.21.15.53:0/159080167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 0x7fcfc00b0600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009d90 tx=0x7fcfb802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d834178b2c697 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:50.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.822+0000 7fcfbe59c640 1 -- 172.21.15.53:0/159080167 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb802cc60 con 0x7fcfc015af10 2023-10-21T17:38:50.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.822+0000 7fcfbe59c640 1 -- 172.21.15.53:0/159080167 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb802cdc0 con 0x7fcfc015af10 2023-10-21T17:38:50.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.822+0000 7fcfbe59c640 1 -- 172.21.15.53:0/159080167 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8034660 con 0x7fcfc015af10 2023-10-21T17:38:50.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.822+0000 7fcfc488f640 1 -- 172.21.15.53:0/159080167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 msgr2=0x7fcfc00b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.822+0000 7fcfc488f640 1 --2- 172.21.15.53:0/159080167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 0x7fcfc00b0600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009d90 tx=0x7fcfb802beb0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 -- 172.21.15.53:0/159080167 shutdown_connections 2023-10-21T17:38:50.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 --2- 172.21.15.53:0/159080167 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015af10 0x7fcfc00b0600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 --2- 172.21.15.53:0/159080167 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00adc70 0x7fcfc00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 -- 172.21.15.53:0/159080167 >> 172.21.15.53:0/159080167 conn(0x7fcfc00cb0c0 msgr2=0x7fcfc00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:50.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 -- 172.21.15.53:0/159080167 shutdown_connections 2023-10-21T17:38:50.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.823+0000 7fcfc488f640 1 -- 172.21.15.53:0/159080167 wait complete. 2023-10-21T17:38:50.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.825+0000 7fcfc488f640 1 Processor -- start 2023-10-21T17:38:50.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.825+0000 7fcfc488f640 1 -- start start 2023-10-21T17:38:50.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015ab30 0x7fcfc015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 0x7fcfc006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc01f3380 con 0x7fcfc00adc70 2023-10-21T17:38:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc015e140 con 0x7fcfc015af10 2023-10-21T17:38:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfc488f640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc01f2ae0 con 0x7fcfc015ab30 2023-10-21T17:38:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfbed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfbed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49004/0 (socket says 172.21.15.53:49004) 2023-10-21T17:38:50.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfbed9d640 1 -- 172.21.15.53:0/2186150182 learned_addr learned my addr 172.21.15.53:0/2186150182 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:50.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.826+0000 7fcfb7fff640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015ab30 0x7fcfc015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbf59e640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 0x7fcfc006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 -- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015ab30 msgr2=0x7fcfc015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015ab30 0x7fcfc015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 -- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 msgr2=0x7fcfc006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 0x7fcfc006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 -- 172.21.15.53:0/2186150182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009400 con 0x7fcfc00adc70 2023-10-21T17:38:50.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbf59e640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 0x7fcfc006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfbed9d640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c4a0 tx=0x7fcfb000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0005ce0 con 0x7fcfc00adc70 2023-10-21T17:38:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0005e40 con 0x7fcfc00adc70 2023-10-21T17:38:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00072b0 con 0x7fcfc00adc70 2023-10-21T17:38:50.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb8009a70 con 0x7fcfc00adc70 2023-10-21T17:38:50.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.827+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc01f30a0 con 0x7fcfc00adc70 2023-10-21T17:38:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.829+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc004e160 con 0x7fcfc00adc70 2023-10-21T17:38:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.830+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcfb0007410 con 0x7fcfc00adc70 2023-10-21T17:38:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.831+0000 7fcfb77fe640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 0x7fcfa006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.831+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fcfb0089200 con 0x7fcfc00adc70 2023-10-21T17:38:50.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.831+0000 7fcfb7fff640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 0x7fcfa006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.832+0000 7fcfb7fff640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 0x7fcfa006d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfc015e3a0 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:50.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:50.835+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfb0053d10 con 0x7fcfc00adc70 2023-10-21T17:38:51.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.066+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfc00ada50 con 0x7fcfc00adc70 2023-10-21T17:38:51.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.067+0000 7fcfb77fe640 1 -- 172.21.15.53:0/2186150182 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7fcfb0057330 con 0x7fcfc00adc70 2023-10-21T17:38:51.069 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:51.069 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:38:51.069 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:38:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 msgr2=0x7fcfa006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 0x7fcfa006d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfc015e3a0 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2023-10-21T17:38:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 msgr2=0x7fcfc01f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c4a0 tx=0x7fcfb000c960 comp rx=0 tx=0).stop 2023-10-21T17:38:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 shutdown_connections 2023-10-21T17:38:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc015af10 0x7fcfc006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fcfa006af90 0x7fcfa006d450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc015ab30 0x7fcfc015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 --2- 172.21.15.53:0/2186150182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc00adc70 0x7fcfc01f2460 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 >> 172.21.15.53:0/2186150182 conn(0x7fcfc00cb0c0 msgr2=0x7fcfc015e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 shutdown_connections 2023-10-21T17:38:51.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.070+0000 7fcfc488f640 1 -- 172.21.15.53:0/2186150182 wait complete. 2023-10-21T17:38:51.090 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:38:51.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.317+0000 7fde9e2d1640 1 Processor -- start 2023-10-21T17:38:51.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.317+0000 7fde9e2d1640 1 -- start start 2023-10-21T17:38:51.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde981599d0 0x7fde980ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fde98159db0 0x7fde980b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde980b10f0 con 0x7fde98159db0 2023-10-21T17:38:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde980cacf0 con 0x7fde981599d0 2023-10-21T17:38:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9e2d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde980ca3c0 con 0x7fde980cc210 2023-10-21T17:38:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9dad0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fde98159db0 0x7fde980b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9dad0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fde98159db0 0x7fde980b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49018/0 (socket says 172.21.15.53:49018) 2023-10-21T17:38:51.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9dad0640 1 -- 172.21.15.53:0/1369344011 learned_addr learned my addr 172.21.15.53:0/1369344011 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:51.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.319+0000 7fde9cace640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9d2cf640 1 --2- 172.21.15.53:0/1369344011 >> v2:172.21.15.124:3300/0 conn(0x7fde981599d0 0x7fde980ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9dad0640 1 -- 172.21.15.53:0/1369344011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 msgr2=0x7fde980b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:51.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9dad0640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9dad0640 1 -- 172.21.15.53:0/1369344011 >> v2:172.21.15.124:3300/0 conn(0x7fde981599d0 msgr2=0x7fde980ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:51.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9dad0640 1 --2- 172.21.15.53:0/1369344011 >> v2:172.21.15.124:3300/0 conn(0x7fde981599d0 0x7fde980ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.320+0000 7fde9dad0640 1 -- 172.21.15.53:0/1369344011 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980ca6a0 con 0x7fde98159db0 2023-10-21T17:38:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.321+0000 7fde9dad0640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde980b0600 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fde98071350 tx=0x7fde9002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284d339c27ba6bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.321+0000 7fde8ffff640 1 -- 172.21.15.53:0/1369344011 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9002ca70 con 0x7fde98159db0 2023-10-21T17:38:51.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.321+0000 7fde8ffff640 1 -- 172.21.15.53:0/1369344011 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9002cbd0 con 0x7fde98159db0 2023-10-21T17:38:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.321+0000 7fde8ffff640 1 -- 172.21.15.53:0/1369344011 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900343f0 con 0x7fde98159db0 2023-10-21T17:38:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 -- 172.21.15.53:0/1369344011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 msgr2=0x7fde980b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde980b0600 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fde98071350 tx=0x7fde9002bed0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 -- 172.21.15.53:0/1369344011 shutdown_connections 2023-10-21T17:38:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/1369344011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde980b0600 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/1369344011 >> v2:172.21.15.124:3300/0 conn(0x7fde981599d0 0x7fde980ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 -- 172.21.15.53:0/1369344011 >> 172.21.15.53:0/1369344011 conn(0x7fde980cb0c0 msgr2=0x7fde980cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:51.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 -- 172.21.15.53:0/1369344011 shutdown_connections 2023-10-21T17:38:51.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.322+0000 7fde9e2d1640 1 -- 172.21.15.53:0/1369344011 wait complete. 2023-10-21T17:38:51.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.324+0000 7fde9e2d1640 1 Processor -- start 2023-10-21T17:38:51.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.324+0000 7fde9e2d1640 1 -- start start 2023-10-21T17:38:51.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde981599d0 0x7fde98151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980c8360 con 0x7fde98159db0 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9815bbb0 con 0x7fde981599d0 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9e2d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9815be00 con 0x7fde980cc210 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9dad0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9dad0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49034/0 (socket says 172.21.15.53:49034) 2023-10-21T17:38:51.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.325+0000 7fde9dad0640 1 -- 172.21.15.53:0/105268827 learned_addr learned my addr 172.21.15.53:0/105268827 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:51.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9cace640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde981599d0 0x7fde98151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9d2cf640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 -- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 msgr2=0x7fde980af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 -- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde981599d0 msgr2=0x7fde98151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde981599d0 0x7fde98151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 -- 172.21.15.53:0/105268827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90009b30 con 0x7fde98159db0 2023-10-21T17:38:51.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.326+0000 7fde9dad0640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fde90009e50 tx=0x7fde90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.327+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90009510 con 0x7fde98159db0 2023-10-21T17:38:51.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.327+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9003e450 con 0x7fde98159db0 2023-10-21T17:38:51.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.327+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9003e800 con 0x7fde98159db0 2023-10-21T17:38:51.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.327+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9815c0e0 con 0x7fde98159db0 2023-10-21T17:38:51.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.327+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9815c6a0 con 0x7fde98159db0 2023-10-21T17:38:51.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.328+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9804e160 con 0x7fde98159db0 2023-10-21T17:38:51.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.329+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde9003e960 con 0x7fde98159db0 2023-10-21T17:38:51.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.330+0000 7fde8dffb640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 0x7fde7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.330+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fde900aca80 con 0x7fde98159db0 2023-10-21T17:38:51.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.330+0000 7fde9d2cf640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 0x7fde7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.331+0000 7fde9d2cf640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 0x7fde7c06d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde94006fd0 tx=0x7fde94008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.334+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde900774f0 con 0x7fde98159db0 2023-10-21T17:38:51.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.531+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde980c9640 con 0x7fde98159db0 2023-10-21T17:38:51.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.535+0000 7fde8dffb640 1 -- 172.21.15.53:0/105268827 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+20614 (secure 0 0 0) 0x7fde9007ab10 con 0x7fde98159db0 2023-10-21T17:38:51.536 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:51.537 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":237,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:48.537636+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:38:51.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 msgr2=0x7fde7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 0x7fde7c06d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde94006fd0 tx=0x7fde94008040 comp rx=0 tx=0).stop 2023-10-21T17:38:51.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 msgr2=0x7fde9815cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fde90009e50 tx=0x7fde90004a40 comp rx=0 tx=0).stop 2023-10-21T17:38:51.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 shutdown_connections 2023-10-21T17:38:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde98159db0 0x7fde9815cbc0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde981599d0 0x7fde98151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fde7c06af90 0x7fde7c06d450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 --2- 172.21.15.53:0/105268827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde980cc210 0x7fde980af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.537+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 >> 172.21.15.53:0/105268827 conn(0x7fde980cb0c0 msgr2=0x7fde9806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:51.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.538+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 shutdown_connections 2023-10-21T17:38:51.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.538+0000 7fde9e2d1640 1 -- 172.21.15.53:0/105268827 wait complete. 2023-10-21T17:38:51.557 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:38:51.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:38:51.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.774+0000 7fae30eca640 1 Processor -- start 2023-10-21T17:38:51.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.774+0000 7fae30eca640 1 -- start start 2023-10-21T17:38:51.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.775+0000 7fae30eca640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fae2c0caeb0 0x7fae2c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.775+0000 7fae30eca640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c165bc0 con 0x7fae2c0cdd70 2023-10-21T17:38:51.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.775+0000 7fae30eca640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c165730 con 0x7fae2c06a640 2023-10-21T17:38:51.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.775+0000 7fae30eca640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c161570 con 0x7fae2c0caeb0 2023-10-21T17:38:51.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.776+0000 7fae2affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fae2c0cdd70 0x7fae2c0b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59380/0 (socket says 172.21.15.53:59380) 2023-10-21T17:38:51.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.776+0000 7fae2affd640 1 -- 172.21.15.53:0/461989456 learned_addr learned my addr 172.21.15.53:0/461989456 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:51.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.776+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162976286 0 0) 0x7fae2c165bc0 con 0x7fae2c0cdd70 2023-10-21T17:38:51.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.776+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> v2:172.21.15.124:3300/0 conn(0x7fae2c0caeb0 0x7fae2c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.776+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae14003520 con 0x7fae2c0cdd70 2023-10-21T17:38:51.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1747313998 0 0) 0x7fae14003520 con 0x7fae2c0cdd70 2023-10-21T17:38:51.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 >> v2:172.21.15.124:3300/0 conn(0x7fae2c0caeb0 msgr2=0x7fae2c0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:51.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 --2- 172.21.15.53:0/461989456 >> v2:172.21.15.124:3300/0 conn(0x7fae2c0caeb0 0x7fae2c0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 >> v1:172.21.15.124:6789/0 conn(0x7fae2c06a640 legacy=0x7fae2c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c164e00 con 0x7fae2c0cdd70 2023-10-21T17:38:51.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fae20001670 con 0x7fae2c0cdd70 2023-10-21T17:38:51.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 >> v1:172.21.15.53:6789/0 conn(0x7fae2c0cdd70 legacy=0x7fae2c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 0x7fae1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c161570 con 0x7fae14005180 2023-10-21T17:38:51.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1400c0d0 con 0x7fae140047c0 2023-10-21T17:38:51.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1400c290 con 0x7fae14003970 2023-10-21T17:38:51.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> v2:172.21.15.124:3300/0 conn(0x7fae2c0caeb0 0x7fae2c0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:51.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 0x7fae1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2b7fe640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.777+0000 7fae2affd640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.778+0000 7fae2bfff640 1 -- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14003970 msgr2=0x7fae14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:51.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.778+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14003970 0x7fae14003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.778+0000 7fae2bfff640 1 -- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae140047c0 msgr2=0x7fae14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:51.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.778+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae140047c0 0x7fae14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.778+0000 7fae2bfff640 1 -- 172.21.15.53:0/461989456 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14003520 con 0x7fae14005180 2023-10-21T17:38:51.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.779+0000 7fae2bfff640 1 -- 172.21.15.53:0/461989456 reap_dead start 2023-10-21T17:38:51.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.779+0000 7fae2bfff640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 0x7fae1400b980 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0cb2c0 tx=0x7fae140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1045abcfdb0f6c78 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.779+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24030660 con 0x7fae14005180 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.779+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae240307e0 con 0x7fae14005180 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.779+0000 7fae2a7fc640 1 -- 172.21.15.53:0/461989456 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24030b90 con 0x7fae14005180 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 -- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 msgr2=0x7fae1400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 0x7fae1400b980 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0cb2c0 tx=0x7fae140057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 -- 172.21.15.53:0/461989456 shutdown_connections 2023-10-21T17:38:51.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 --2- 172.21.15.53:0/461989456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae14005180 0x7fae1400b980 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 -- 172.21.15.53:0/461989456 >> 172.21.15.53:0/461989456 conn(0x7fae2c06ae60 msgr2=0x7fae2c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:51.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 -- 172.21.15.53:0/461989456 shutdown_connections 2023-10-21T17:38:51.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.780+0000 7fae30eca640 1 -- 172.21.15.53:0/461989456 wait complete. 2023-10-21T17:38:51.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.782+0000 7fae30eca640 1 Processor -- start 2023-10-21T17:38:51.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.782+0000 7fae30eca640 1 -- start start 2023-10-21T17:38:51.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 0x7fae2c0c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae2c0c5a10 0x7fae2c0b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c5310 con 0x7fae2c1537a0 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c61d0 con 0x7fae2c0c5a10 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae30eca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c4e80 con 0x7fae14005180 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae2bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49044/0 (socket says 172.21.15.53:49044) 2023-10-21T17:38:51.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae2bfff640 1 -- 172.21.15.53:0/371758666 learned_addr learned my addr 172.21.15.53:0/371758666 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:51.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae2b7fe640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 0x7fae2c0c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.783+0000 7fae2affd640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae2c0c5a10 0x7fae2c0b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 -- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 msgr2=0x7fae2c0c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 0x7fae2c0c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 -- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae2c0c5a10 msgr2=0x7fae2c0b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:51.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae2c0c5a10 0x7fae2c0b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:51.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 -- 172.21.15.53:0/371758666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0c5ef0 con 0x7fae2c1537a0 2023-10-21T17:38:51.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2b7fe640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 0x7fae2c0c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:38:51.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae2bfff640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fae2402bfc0 tx=0x7fae24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24039860 con 0x7fae2c1537a0 2023-10-21T17:38:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2403f040 con 0x7fae2c1537a0 2023-10-21T17:38:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24039c00 con 0x7fae2c1537a0 2023-10-21T17:38:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.784+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c0c4830 con 0x7fae2c1537a0 2023-10-21T17:38:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.785+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c0c38a0 con 0x7fae2c1537a0 2023-10-21T17:38:51.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.786+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0c5010 con 0x7fae2c1537a0 2023-10-21T17:38:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.786+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae2404d460 con 0x7fae2c1537a0 2023-10-21T17:38:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.788+0000 7fae13fff640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 0x7fae1806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.788+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fae240ac120 con 0x7fae2c1537a0 2023-10-21T17:38:51.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.789+0000 7fae2b7fe640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 0x7fae1806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.790+0000 7fae2b7fe640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 0x7fae1806d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f80 tx=0x7fae1c009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:51.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.792+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae24076a90 con 0x7fae2c1537a0 2023-10-21T17:38:51.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:51.986+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae2c1653c0 con 0x7fae1806afc0 2023-10-21T17:38:52.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.030+0000 7fae13fff640 1 -- 172.21.15.53:0/371758666 <== mgr.37814 v2:172.21.15.124:6832/3930748558 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525568 (secure 0 0 0) 0x7fae1c005860 con 0x7fae1806afc0 2023-10-21T17:38:52.034 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:52.043 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:38:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 msgr2=0x7fae1806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 0x7fae1806d480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f80 tx=0x7fae1c009040 comp rx=0 tx=0).stop 2023-10-21T17:38:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 msgr2=0x7fae2c0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fae2402bfc0 tx=0x7fae24004730 comp rx=0 tx=0).stop 2023-10-21T17:38:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 shutdown_connections 2023-10-21T17:38:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae2c1537a0 0x7fae2c0b4770 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fae2c0c5a10 0x7fae2c0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fae1806afc0 0x7fae1806d480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 --2- 172.21.15.53:0/371758666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fae14005180 0x7fae2c0c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.043+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 >> 172.21.15.53:0/371758666 conn(0x7fae2c06ae60 msgr2=0x7fae2c160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:52.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.044+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 shutdown_connections 2023-10-21T17:38:52.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.044+0000 7fae30eca640 1 -- 172.21.15.53:0/371758666 wait complete. 2023-10-21T17:38:52.072 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-10-21T17:38:51.685452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":678,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4952,"ondisk_log_size":4952,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":286,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439096,"kb_used_data":24436,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754535624,"statfs":{"total":773094113280,"available":772644478976,"internally_reserved":0,"allocated":25022464,"data_stored":13501957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":424463929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002552"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.568738+0000","last_change":"2023-10-21T17:38:47.566020+0000","last_active":"2023-10-21T17:38:47.568738+0000","last_peered":"2023-10-21T17:38:47.568738+0000","last_clean":"2023-10-21T17:38:47.568738+0000","last_became_active":"2023-10-21T17:38:47.565725+0000","last_became_peered":"2023-10-21T17:38:47.565725+0000","last_unstale":"2023-10-21T17:38:47.568738+0000","last_undegraded":"2023-10-21T17:38:47.568738+0000","last_fullsized":"2023-10-21T17:38:47.568738+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:09:02.823367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"221'4873","reported_seq":5806,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.554318+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:38:46.554318+0000","last_peered":"2023-10-21T17:38:46.554318+0000","last_clean":"2023-10-21T17:38:46.554318+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:38:46.554318+0000","last_undegraded":"2023-10-21T17:38:46.554318+0000","last_fullsized":"2023-10-21T17:38:46.554318+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4873,"log_dups_size":0,"ondisk_log_size":4873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"232'1","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538340+0000","last_change":"2023-10-21T17:38:32.397140+0000","last_active":"2023-10-21T17:38:47.538340+0000","last_peered":"2023-10-21T17:38:47.538340+0000","last_clean":"2023-10-21T17:38:47.538340+0000","last_became_active":"2023-10-21T17:38:32.396837+0000","last_became_peered":"2023-10-21T17:38:32.396837+0000","last_unstale":"2023-10-21T17:38:47.538340+0000","last_undegraded":"2023-10-21T17:38:47.538340+0000","last_fullsized":"2023-10-21T17:38:47.538340+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:17.875367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534995+0000","last_change":"2023-10-21T17:38:33.370135+0000","last_active":"2023-10-21T17:38:46.534995+0000","last_peered":"2023-10-21T17:38:46.534995+0000","last_clean":"2023-10-21T17:38:46.534995+0000","last_became_active":"2023-10-21T17:38:33.369798+0000","last_became_peered":"2023-10-21T17:38:33.369798+0000","last_unstale":"2023-10-21T17:38:46.534995+0000","last_undegraded":"2023-10-21T17:38:46.534995+0000","last_fullsized":"2023-10-21T17:38:46.534995+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:25.189245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"236'2","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570893+0000","last_change":"2023-10-21T17:38:46.562481+0000","last_active":"2023-10-21T17:38:47.570893+0000","last_peered":"2023-10-21T17:38:47.570893+0000","last_clean":"2023-10-21T17:38:47.570893+0000","last_became_active":"2023-10-21T17:38:46.562157+0000","last_became_peered":"2023-10-21T17:38:46.562157+0000","last_unstale":"2023-10-21T17:38:47.570893+0000","last_undegraded":"2023-10-21T17:38:47.570893+0000","last_fullsized":"2023-10-21T17:38:47.570893+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:33.520240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565336+0000","last_change":"2023-10-21T17:38:47.558062+0000","last_active":"2023-10-21T17:38:47.565336+0000","last_peered":"2023-10-21T17:38:47.565336+0000","last_clean":"2023-10-21T17:38:47.565336+0000","last_became_active":"2023-10-21T17:38:47.557643+0000","last_became_peered":"2023-10-21T17:38:47.557643+0000","last_unstale":"2023-10-21T17:38:47.565336+0000","last_undegraded":"2023-10-21T17:38:47.565336+0000","last_fullsized":"2023-10-21T17:38:47.565336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:43:53.140469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.526291+0000","last_change":"2023-10-21T17:38:32.411805+0000","last_active":"2023-10-21T17:38:47.526291+0000","last_peered":"2023-10-21T17:38:47.526291+0000","last_clean":"2023-10-21T17:38:47.526291+0000","last_became_active":"2023-10-21T17:38:32.411439+0000","last_became_peered":"2023-10-21T17:38:32.411439+0000","last_unstale":"2023-10-21T17:38:47.526291+0000","last_undegraded":"2023-10-21T17:38:47.526291+0000","last_fullsized":"2023-10-21T17:38:47.526291+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:57.270863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.538119+0000","last_change":"2023-10-21T17:38:33.360420+0000","last_active":"2023-10-21T17:38:46.538119+0000","last_peered":"2023-10-21T17:38:46.538119+0000","last_clean":"2023-10-21T17:38:46.538119+0000","last_became_active":"2023-10-21T17:38:33.360143+0000","last_became_peered":"2023-10-21T17:38:33.360143+0000","last_unstale":"2023-10-21T17:38:46.538119+0000","last_undegraded":"2023-10-21T17:38:46.538119+0000","last_fullsized":"2023-10-21T17:38:46.538119+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:37.181561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.546759+0000","last_change":"2023-10-21T17:38:46.558587+0000","last_active":"2023-10-21T17:38:47.546759+0000","last_peered":"2023-10-21T17:38:47.546759+0000","last_clean":"2023-10-21T17:38:47.546759+0000","last_became_active":"2023-10-21T17:38:46.558305+0000","last_became_peered":"2023-10-21T17:38:46.558305+0000","last_unstale":"2023-10-21T17:38:47.546759+0000","last_undegraded":"2023-10-21T17:38:47.546759+0000","last_fullsized":"2023-10-21T17:38:47.546759+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:23.121781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.551881+0000","last_change":"2023-10-21T17:38:47.565116+0000","last_active":"2023-10-21T17:38:48.551881+0000","last_peered":"2023-10-21T17:38:48.551881+0000","last_clean":"2023-10-21T17:38:48.551881+0000","last_became_active":"2023-10-21T17:38:47.564818+0000","last_became_peered":"2023-10-21T17:38:47.564818+0000","last_unstale":"2023-10-21T17:38:48.551881+0000","last_undegraded":"2023-10-21T17:38:48.551881+0000","last_fullsized":"2023-10-21T17:38:48.551881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:49.072976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.537004+0000","last_change":"2023-10-21T17:38:32.411663+0000","last_active":"2023-10-21T17:38:46.537004+0000","last_peered":"2023-10-21T17:38:46.537004+0000","last_clean":"2023-10-21T17:38:46.537004+0000","last_became_active":"2023-10-21T17:38:32.411409+0000","last_became_peered":"2023-10-21T17:38:32.411409+0000","last_unstale":"2023-10-21T17:38:46.537004+0000","last_undegraded":"2023-10-21T17:38:46.537004+0000","last_fullsized":"2023-10-21T17:38:46.537004+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:00.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559209+0000","last_change":"2023-10-21T17:38:33.354071+0000","last_active":"2023-10-21T17:38:46.559209+0000","last_peered":"2023-10-21T17:38:46.559209+0000","last_clean":"2023-10-21T17:38:46.559209+0000","last_became_active":"2023-10-21T17:38:33.353746+0000","last_became_peered":"2023-10-21T17:38:33.353746+0000","last_unstale":"2023-10-21T17:38:46.559209+0000","last_undegraded":"2023-10-21T17:38:46.559209+0000","last_fullsized":"2023-10-21T17:38:46.559209+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:31.254948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558960+0000","last_change":"2023-10-21T17:38:46.563703+0000","last_active":"2023-10-21T17:38:48.558960+0000","last_peered":"2023-10-21T17:38:48.558960+0000","last_clean":"2023-10-21T17:38:48.558960+0000","last_became_active":"2023-10-21T17:38:46.563077+0000","last_became_peered":"2023-10-21T17:38:46.563077+0000","last_unstale":"2023-10-21T17:38:48.558960+0000","last_undegraded":"2023-10-21T17:38:48.558960+0000","last_fullsized":"2023-10-21T17:38:48.558960+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:49:50.477869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556780+0000","last_change":"2023-10-21T17:38:47.554568+0000","last_active":"2023-10-21T17:38:48.556780+0000","last_peered":"2023-10-21T17:38:48.556780+0000","last_clean":"2023-10-21T17:38:48.556780+0000","last_became_active":"2023-10-21T17:38:47.554270+0000","last_became_peered":"2023-10-21T17:38:47.554270+0000","last_unstale":"2023-10-21T17:38:48.556780+0000","last_undegraded":"2023-10-21T17:38:48.556780+0000","last_fullsized":"2023-10-21T17:38:48.556780+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:48:25.196265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538468+0000","last_change":"2023-10-21T17:38:32.408205+0000","last_active":"2023-10-21T17:38:47.538468+0000","last_peered":"2023-10-21T17:38:47.538468+0000","last_clean":"2023-10-21T17:38:47.538468+0000","last_became_active":"2023-10-21T17:38:32.407638+0000","last_became_peered":"2023-10-21T17:38:32.407638+0000","last_unstale":"2023-10-21T17:38:47.538468+0000","last_undegraded":"2023-10-21T17:38:47.538468+0000","last_fullsized":"2023-10-21T17:38:47.538468+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:20.909728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.561083+0000","last_change":"2023-10-21T17:38:33.362147+0000","last_active":"2023-10-21T17:38:46.561083+0000","last_peered":"2023-10-21T17:38:46.561083+0000","last_clean":"2023-10-21T17:38:46.561083+0000","last_became_active":"2023-10-21T17:38:33.360339+0000","last_became_peered":"2023-10-21T17:38:33.360339+0000","last_unstale":"2023-10-21T17:38:46.561083+0000","last_undegraded":"2023-10-21T17:38:46.561083+0000","last_fullsized":"2023-10-21T17:38:46.561083+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:51:06.624565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558139+0000","last_change":"2023-10-21T17:38:46.568841+0000","last_active":"2023-10-21T17:38:48.558139+0000","last_peered":"2023-10-21T17:38:48.558139+0000","last_clean":"2023-10-21T17:38:48.558139+0000","last_became_active":"2023-10-21T17:38:46.568115+0000","last_became_peered":"2023-10-21T17:38:46.568115+0000","last_unstale":"2023-10-21T17:38:48.558139+0000","last_undegraded":"2023-10-21T17:38:48.558139+0000","last_fullsized":"2023-10-21T17:38:48.558139+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:25.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565189+0000","last_change":"2023-10-21T17:38:47.559677+0000","last_active":"2023-10-21T17:38:47.565189+0000","last_peered":"2023-10-21T17:38:47.565189+0000","last_clean":"2023-10-21T17:38:47.565189+0000","last_became_active":"2023-10-21T17:38:47.559414+0000","last_became_peered":"2023-10-21T17:38:47.559414+0000","last_unstale":"2023-10-21T17:38:47.565189+0000","last_undegraded":"2023-10-21T17:38:47.565189+0000","last_fullsized":"2023-10-21T17:38:47.565189+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:43:53.140469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545658+0000","last_change":"2023-10-21T17:38:32.419011+0000","last_active":"2023-10-21T17:38:47.545658+0000","last_peered":"2023-10-21T17:38:47.545658+0000","last_clean":"2023-10-21T17:38:47.545658+0000","last_became_active":"2023-10-21T17:38:32.418410+0000","last_became_peered":"2023-10-21T17:38:32.418410+0000","last_unstale":"2023-10-21T17:38:47.545658+0000","last_undegraded":"2023-10-21T17:38:47.545658+0000","last_fullsized":"2023-10-21T17:38:47.545658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:22.056635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553378+0000","last_change":"2023-10-21T17:38:33.364849+0000","last_active":"2023-10-21T17:38:47.553378+0000","last_peered":"2023-10-21T17:38:47.553378+0000","last_clean":"2023-10-21T17:38:47.553378+0000","last_became_active":"2023-10-21T17:38:33.363791+0000","last_became_peered":"2023-10-21T17:38:33.363791+0000","last_unstale":"2023-10-21T17:38:47.553378+0000","last_undegraded":"2023-10-21T17:38:47.553378+0000","last_fullsized":"2023-10-21T17:38:47.553378+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:29.919088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.546759+0000","last_change":"2023-10-21T17:38:46.547298+0000","last_active":"2023-10-21T17:38:47.546759+0000","last_peered":"2023-10-21T17:38:47.546759+0000","last_clean":"2023-10-21T17:38:47.546759+0000","last_became_active":"2023-10-21T17:38:46.546520+0000","last_became_peered":"2023-10-21T17:38:46.546520+0000","last_unstale":"2023-10-21T17:38:47.546759+0000","last_undegraded":"2023-10-21T17:38:47.546759+0000","last_fullsized":"2023-10-21T17:38:47.546759+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:37.454918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552423+0000","last_change":"2023-10-21T17:38:47.550517+0000","last_active":"2023-10-21T17:38:47.552423+0000","last_peered":"2023-10-21T17:38:47.552423+0000","last_clean":"2023-10-21T17:38:47.552423+0000","last_became_active":"2023-10-21T17:38:47.550235+0000","last_became_peered":"2023-10-21T17:38:47.550235+0000","last_unstale":"2023-10-21T17:38:47.552423+0000","last_undegraded":"2023-10-21T17:38:47.552423+0000","last_fullsized":"2023-10-21T17:38:47.552423+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:06:46.717693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536611+0000","last_change":"2023-10-21T17:38:32.405103+0000","last_active":"2023-10-21T17:38:46.536611+0000","last_peered":"2023-10-21T17:38:46.536611+0000","last_clean":"2023-10-21T17:38:46.536611+0000","last_became_active":"2023-10-21T17:38:32.404811+0000","last_became_peered":"2023-10-21T17:38:32.404811+0000","last_unstale":"2023-10-21T17:38:46.536611+0000","last_undegraded":"2023-10-21T17:38:46.536611+0000","last_fullsized":"2023-10-21T17:38:46.536611+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:19.532470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532797+0000","last_change":"2023-10-21T17:38:33.351368+0000","last_active":"2023-10-21T17:38:46.532797+0000","last_peered":"2023-10-21T17:38:46.532797+0000","last_clean":"2023-10-21T17:38:46.532797+0000","last_became_active":"2023-10-21T17:38:33.350963+0000","last_became_peered":"2023-10-21T17:38:33.350963+0000","last_unstale":"2023-10-21T17:38:46.532797+0000","last_undegraded":"2023-10-21T17:38:46.532797+0000","last_fullsized":"2023-10-21T17:38:46.532797+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:24.335865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.531083+0000","last_change":"2023-10-21T17:38:46.550779+0000","last_active":"2023-10-21T17:38:47.531083+0000","last_peered":"2023-10-21T17:38:47.531083+0000","last_clean":"2023-10-21T17:38:47.531083+0000","last_became_active":"2023-10-21T17:38:46.550494+0000","last_became_peered":"2023-10-21T17:38:46.550494+0000","last_unstale":"2023-10-21T17:38:47.531083+0000","last_undegraded":"2023-10-21T17:38:47.531083+0000","last_fullsized":"2023-10-21T17:38:47.531083+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:28.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565357+0000","last_change":"2023-10-21T17:38:47.560233+0000","last_active":"2023-10-21T17:38:47.565357+0000","last_peered":"2023-10-21T17:38:47.565357+0000","last_clean":"2023-10-21T17:38:47.565357+0000","last_became_active":"2023-10-21T17:38:47.559324+0000","last_became_peered":"2023-10-21T17:38:47.559324+0000","last_unstale":"2023-10-21T17:38:47.565357+0000","last_undegraded":"2023-10-21T17:38:47.565357+0000","last_fullsized":"2023-10-21T17:38:47.565357+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:56.287349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.551433+0000","last_change":"2023-10-21T17:38:32.405560+0000","last_active":"2023-10-21T17:38:46.551433+0000","last_peered":"2023-10-21T17:38:46.551433+0000","last_clean":"2023-10-21T17:38:46.551433+0000","last_became_active":"2023-10-21T17:38:32.405233+0000","last_became_peered":"2023-10-21T17:38:32.405233+0000","last_unstale":"2023-10-21T17:38:46.551433+0000","last_undegraded":"2023-10-21T17:38:46.551433+0000","last_fullsized":"2023-10-21T17:38:46.551433+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538054+0000","last_change":"2023-10-21T17:38:33.358411+0000","last_active":"2023-10-21T17:38:47.538054+0000","last_peered":"2023-10-21T17:38:47.538054+0000","last_clean":"2023-10-21T17:38:47.538054+0000","last_became_active":"2023-10-21T17:38:33.358101+0000","last_became_peered":"2023-10-21T17:38:33.358101+0000","last_unstale":"2023-10-21T17:38:47.538054+0000","last_undegraded":"2023-10-21T17:38:47.538054+0000","last_fullsized":"2023-10-21T17:38:47.538054+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:31:32.268023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557897+0000","last_change":"2023-10-21T17:38:46.559154+0000","last_active":"2023-10-21T17:38:48.557897+0000","last_peered":"2023-10-21T17:38:48.557897+0000","last_clean":"2023-10-21T17:38:48.557897+0000","last_became_active":"2023-10-21T17:38:46.558296+0000","last_became_peered":"2023-10-21T17:38:46.558296+0000","last_unstale":"2023-10-21T17:38:48.557897+0000","last_undegraded":"2023-10-21T17:38:48.557897+0000","last_fullsized":"2023-10-21T17:38:48.557897+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:41.359330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558279+0000","last_change":"2023-10-21T17:38:47.556014+0000","last_active":"2023-10-21T17:38:48.558279+0000","last_peered":"2023-10-21T17:38:48.558279+0000","last_clean":"2023-10-21T17:38:48.558279+0000","last_became_active":"2023-10-21T17:38:47.555739+0000","last_became_peered":"2023-10-21T17:38:47.555739+0000","last_unstale":"2023-10-21T17:38:48.558279+0000","last_undegraded":"2023-10-21T17:38:48.558279+0000","last_fullsized":"2023-10-21T17:38:48.558279+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:48.820162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"233'1","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552631+0000","last_change":"2023-10-21T17:38:32.416259+0000","last_active":"2023-10-21T17:38:47.552631+0000","last_peered":"2023-10-21T17:38:47.552631+0000","last_clean":"2023-10-21T17:38:47.552631+0000","last_became_active":"2023-10-21T17:38:32.415430+0000","last_became_peered":"2023-10-21T17:38:32.415430+0000","last_unstale":"2023-10-21T17:38:47.552631+0000","last_undegraded":"2023-10-21T17:38:47.552631+0000","last_fullsized":"2023-10-21T17:38:47.552631+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532609+0000","last_change":"2023-10-21T17:38:33.358506+0000","last_active":"2023-10-21T17:38:46.532609+0000","last_peered":"2023-10-21T17:38:46.532609+0000","last_clean":"2023-10-21T17:38:46.532609+0000","last_became_active":"2023-10-21T17:38:33.357627+0000","last_became_peered":"2023-10-21T17:38:33.357627+0000","last_unstale":"2023-10-21T17:38:46.532609+0000","last_undegraded":"2023-10-21T17:38:46.532609+0000","last_fullsized":"2023-10-21T17:38:46.532609+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:10.598342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556150+0000","last_change":"2023-10-21T17:38:46.562461+0000","last_active":"2023-10-21T17:38:48.556150+0000","last_peered":"2023-10-21T17:38:48.556150+0000","last_clean":"2023-10-21T17:38:48.556150+0000","last_became_active":"2023-10-21T17:38:46.562169+0000","last_became_peered":"2023-10-21T17:38:46.562169+0000","last_unstale":"2023-10-21T17:38:48.556150+0000","last_undegraded":"2023-10-21T17:38:48.556150+0000","last_fullsized":"2023-10-21T17:38:48.556150+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:35.077192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555249+0000","last_change":"2023-10-21T17:38:47.539213+0000","last_active":"2023-10-21T17:38:48.555249+0000","last_peered":"2023-10-21T17:38:48.555249+0000","last_clean":"2023-10-21T17:38:48.555249+0000","last_became_active":"2023-10-21T17:38:47.538742+0000","last_became_peered":"2023-10-21T17:38:47.538742+0000","last_unstale":"2023-10-21T17:38:48.555249+0000","last_undegraded":"2023-10-21T17:38:48.555249+0000","last_fullsized":"2023-10-21T17:38:48.555249+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:37.691514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541744+0000","last_change":"2023-10-21T17:38:32.418683+0000","last_active":"2023-10-21T17:38:47.541744+0000","last_peered":"2023-10-21T17:38:47.541744+0000","last_clean":"2023-10-21T17:38:47.541744+0000","last_became_active":"2023-10-21T17:38:32.418325+0000","last_became_peered":"2023-10-21T17:38:32.418325+0000","last_unstale":"2023-10-21T17:38:47.541744+0000","last_undegraded":"2023-10-21T17:38:47.541744+0000","last_fullsized":"2023-10-21T17:38:47.541744+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:36.165616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541407+0000","last_change":"2023-10-21T17:38:33.361024+0000","last_active":"2023-10-21T17:38:47.541407+0000","last_peered":"2023-10-21T17:38:47.541407+0000","last_clean":"2023-10-21T17:38:47.541407+0000","last_became_active":"2023-10-21T17:38:33.360523+0000","last_became_peered":"2023-10-21T17:38:33.360523+0000","last_unstale":"2023-10-21T17:38:47.541407+0000","last_undegraded":"2023-10-21T17:38:47.541407+0000","last_fullsized":"2023-10-21T17:38:47.541407+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:25.912046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.540560+0000","last_change":"2023-10-21T17:38:46.561178+0000","last_active":"2023-10-21T17:38:47.540560+0000","last_peered":"2023-10-21T17:38:47.540560+0000","last_clean":"2023-10-21T17:38:47.540560+0000","last_became_active":"2023-10-21T17:38:46.560869+0000","last_became_peered":"2023-10-21T17:38:46.560869+0000","last_unstale":"2023-10-21T17:38:47.540560+0000","last_undegraded":"2023-10-21T17:38:47.540560+0000","last_fullsized":"2023-10-21T17:38:47.540560+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:03.944786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552897+0000","last_change":"2023-10-21T17:38:47.549334+0000","last_active":"2023-10-21T17:38:47.552897+0000","last_peered":"2023-10-21T17:38:47.552897+0000","last_clean":"2023-10-21T17:38:47.552897+0000","last_became_active":"2023-10-21T17:38:47.548345+0000","last_became_peered":"2023-10-21T17:38:47.548345+0000","last_unstale":"2023-10-21T17:38:47.552897+0000","last_undegraded":"2023-10-21T17:38:47.552897+0000","last_fullsized":"2023-10-21T17:38:47.552897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:00:52.120978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531967+0000","last_change":"2023-10-21T17:38:32.410021+0000","last_active":"2023-10-21T17:38:46.531967+0000","last_peered":"2023-10-21T17:38:46.531967+0000","last_clean":"2023-10-21T17:38:46.531967+0000","last_became_active":"2023-10-21T17:38:32.409751+0000","last_became_peered":"2023-10-21T17:38:32.409751+0000","last_unstale":"2023-10-21T17:38:46.531967+0000","last_undegraded":"2023-10-21T17:38:46.531967+0000","last_fullsized":"2023-10-21T17:38:46.531967+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553995+0000","last_change":"2023-10-21T17:38:33.363577+0000","last_active":"2023-10-21T17:38:47.553995+0000","last_peered":"2023-10-21T17:38:47.553995+0000","last_clean":"2023-10-21T17:38:47.553995+0000","last_became_active":"2023-10-21T17:38:33.360945+0000","last_became_peered":"2023-10-21T17:38:33.360945+0000","last_unstale":"2023-10-21T17:38:47.553995+0000","last_undegraded":"2023-10-21T17:38:47.553995+0000","last_fullsized":"2023-10-21T17:38:47.553995+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:21.404419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555772+0000","last_change":"2023-10-21T17:38:46.550016+0000","last_active":"2023-10-21T17:38:48.555772+0000","last_peered":"2023-10-21T17:38:48.555772+0000","last_clean":"2023-10-21T17:38:48.555772+0000","last_became_active":"2023-10-21T17:38:46.549494+0000","last_became_peered":"2023-10-21T17:38:46.549494+0000","last_unstale":"2023-10-21T17:38:48.555772+0000","last_undegraded":"2023-10-21T17:38:48.555772+0000","last_fullsized":"2023-10-21T17:38:48.555772+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:15:38.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552239+0000","last_change":"2023-10-21T17:38:47.561495+0000","last_active":"2023-10-21T17:38:48.552239+0000","last_peered":"2023-10-21T17:38:48.552239+0000","last_clean":"2023-10-21T17:38:48.552239+0000","last_became_active":"2023-10-21T17:38:47.561226+0000","last_became_peered":"2023-10-21T17:38:47.561226+0000","last_unstale":"2023-10-21T17:38:48.552239+0000","last_undegraded":"2023-10-21T17:38:48.552239+0000","last_fullsized":"2023-10-21T17:38:48.552239+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:31.912580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.547887+0000","last_change":"2023-10-21T17:38:32.416515+0000","last_active":"2023-10-21T17:38:46.547887+0000","last_peered":"2023-10-21T17:38:46.547887+0000","last_clean":"2023-10-21T17:38:46.547887+0000","last_became_active":"2023-10-21T17:38:32.415411+0000","last_became_peered":"2023-10-21T17:38:32.415411+0000","last_unstale":"2023-10-21T17:38:46.547887+0000","last_undegraded":"2023-10-21T17:38:46.547887+0000","last_fullsized":"2023-10-21T17:38:46.547887+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:24.896102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.551013+0000","last_change":"2023-10-21T17:38:33.353848+0000","last_active":"2023-10-21T17:38:47.551013+0000","last_peered":"2023-10-21T17:38:47.551013+0000","last_clean":"2023-10-21T17:38:47.551013+0000","last_became_active":"2023-10-21T17:38:33.351959+0000","last_became_peered":"2023-10-21T17:38:33.351959+0000","last_unstale":"2023-10-21T17:38:47.551013+0000","last_undegraded":"2023-10-21T17:38:47.551013+0000","last_fullsized":"2023-10-21T17:38:47.551013+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.572130+0000","last_change":"2023-10-21T17:38:46.558191+0000","last_active":"2023-10-21T17:38:47.572130+0000","last_peered":"2023-10-21T17:38:47.572130+0000","last_clean":"2023-10-21T17:38:47.572130+0000","last_became_active":"2023-10-21T17:38:46.557875+0000","last_became_peered":"2023-10-21T17:38:46.557875+0000","last_unstale":"2023-10-21T17:38:47.572130+0000","last_undegraded":"2023-10-21T17:38:47.572130+0000","last_fullsized":"2023-10-21T17:38:47.572130+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:04:25.310386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.559400+0000","last_change":"2023-10-21T17:38:47.559460+0000","last_active":"2023-10-21T17:38:48.559400+0000","last_peered":"2023-10-21T17:38:48.559400+0000","last_clean":"2023-10-21T17:38:48.559400+0000","last_became_active":"2023-10-21T17:38:47.559166+0000","last_became_peered":"2023-10-21T17:38:47.559166+0000","last_unstale":"2023-10-21T17:38:48.559400+0000","last_undegraded":"2023-10-21T17:38:48.559400+0000","last_fullsized":"2023-10-21T17:38:48.559400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:57.184166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534658+0000","last_change":"2023-10-21T17:38:32.399236+0000","last_active":"2023-10-21T17:38:46.534658+0000","last_peered":"2023-10-21T17:38:46.534658+0000","last_clean":"2023-10-21T17:38:46.534658+0000","last_became_active":"2023-10-21T17:38:32.398909+0000","last_became_peered":"2023-10-21T17:38:32.398909+0000","last_unstale":"2023-10-21T17:38:46.534658+0000","last_undegraded":"2023-10-21T17:38:46.534658+0000","last_fullsized":"2023-10-21T17:38:46.534658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:04.398168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559637+0000","last_change":"2023-10-21T17:38:33.354105+0000","last_active":"2023-10-21T17:38:46.559637+0000","last_peered":"2023-10-21T17:38:46.559637+0000","last_clean":"2023-10-21T17:38:46.559637+0000","last_became_active":"2023-10-21T17:38:33.353821+0000","last_became_peered":"2023-10-21T17:38:33.353821+0000","last_unstale":"2023-10-21T17:38:46.559637+0000","last_undegraded":"2023-10-21T17:38:46.559637+0000","last_fullsized":"2023-10-21T17:38:46.559637+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:34.936008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557893+0000","last_change":"2023-10-21T17:38:46.566157+0000","last_active":"2023-10-21T17:38:48.557893+0000","last_peered":"2023-10-21T17:38:48.557893+0000","last_clean":"2023-10-21T17:38:48.557893+0000","last_became_active":"2023-10-21T17:38:46.563886+0000","last_became_peered":"2023-10-21T17:38:46.563886+0000","last_unstale":"2023-10-21T17:38:48.557893+0000","last_undegraded":"2023-10-21T17:38:48.557893+0000","last_fullsized":"2023-10-21T17:38:48.557893+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:13.777469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565453+0000","last_change":"2023-10-21T17:38:47.562860+0000","last_active":"2023-10-21T17:38:47.565453+0000","last_peered":"2023-10-21T17:38:47.565453+0000","last_clean":"2023-10-21T17:38:47.565453+0000","last_became_active":"2023-10-21T17:38:47.562527+0000","last_became_peered":"2023-10-21T17:38:47.562527+0000","last_unstale":"2023-10-21T17:38:47.565453+0000","last_undegraded":"2023-10-21T17:38:47.565453+0000","last_fullsized":"2023-10-21T17:38:47.565453+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:44:42.233371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530311+0000","last_change":"2023-10-21T17:38:32.401853+0000","last_active":"2023-10-21T17:38:47.530311+0000","last_peered":"2023-10-21T17:38:47.530311+0000","last_clean":"2023-10-21T17:38:47.530311+0000","last_became_active":"2023-10-21T17:38:32.401567+0000","last_became_peered":"2023-10-21T17:38:32.401567+0000","last_unstale":"2023-10-21T17:38:47.530311+0000","last_undegraded":"2023-10-21T17:38:47.530311+0000","last_fullsized":"2023-10-21T17:38:47.530311+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:24.460794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535354+0000","last_change":"2023-10-21T17:38:33.348917+0000","last_active":"2023-10-21T17:38:46.535354+0000","last_peered":"2023-10-21T17:38:46.535354+0000","last_clean":"2023-10-21T17:38:46.535354+0000","last_became_active":"2023-10-21T17:38:33.348522+0000","last_became_peered":"2023-10-21T17:38:33.348522+0000","last_unstale":"2023-10-21T17:38:46.535354+0000","last_undegraded":"2023-10-21T17:38:46.535354+0000","last_fullsized":"2023-10-21T17:38:46.535354+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:40.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"236'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557489+0000","last_change":"2023-10-21T17:38:46.563174+0000","last_active":"2023-10-21T17:38:48.557489+0000","last_peered":"2023-10-21T17:38:48.557489+0000","last_clean":"2023-10-21T17:38:48.557489+0000","last_became_active":"2023-10-21T17:38:46.562784+0000","last_became_peered":"2023-10-21T17:38:46.562784+0000","last_unstale":"2023-10-21T17:38:48.557489+0000","last_undegraded":"2023-10-21T17:38:48.557489+0000","last_fullsized":"2023-10-21T17:38:48.557489+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:31:10.759058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565041+0000","last_change":"2023-10-21T17:38:47.558254+0000","last_active":"2023-10-21T17:38:47.565041+0000","last_peered":"2023-10-21T17:38:47.565041+0000","last_clean":"2023-10-21T17:38:47.565041+0000","last_became_active":"2023-10-21T17:38:47.557951+0000","last_became_peered":"2023-10-21T17:38:47.557951+0000","last_unstale":"2023-10-21T17:38:47.565041+0000","last_undegraded":"2023-10-21T17:38:47.565041+0000","last_fullsized":"2023-10-21T17:38:47.565041+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:23.084608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529286+0000","last_change":"2023-10-21T17:38:32.403394+0000","last_active":"2023-10-21T17:38:47.529286+0000","last_peered":"2023-10-21T17:38:47.529286+0000","last_clean":"2023-10-21T17:38:47.529286+0000","last_became_active":"2023-10-21T17:38:32.403104+0000","last_became_peered":"2023-10-21T17:38:32.403104+0000","last_unstale":"2023-10-21T17:38:47.529286+0000","last_undegraded":"2023-10-21T17:38:47.529286+0000","last_fullsized":"2023-10-21T17:38:47.529286+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:06:56.765497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545565+0000","last_change":"2023-10-21T17:38:33.366719+0000","last_active":"2023-10-21T17:38:47.545565+0000","last_peered":"2023-10-21T17:38:47.545565+0000","last_clean":"2023-10-21T17:38:47.545565+0000","last_became_active":"2023-10-21T17:38:33.366173+0000","last_became_peered":"2023-10-21T17:38:33.366173+0000","last_unstale":"2023-10-21T17:38:47.545565+0000","last_undegraded":"2023-10-21T17:38:47.545565+0000","last_fullsized":"2023-10-21T17:38:47.545565+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:58.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.543198+0000","last_change":"2023-10-21T17:38:46.554315+0000","last_active":"2023-10-21T17:38:47.543198+0000","last_peered":"2023-10-21T17:38:47.543198+0000","last_clean":"2023-10-21T17:38:47.543198+0000","last_became_active":"2023-10-21T17:38:46.554078+0000","last_became_peered":"2023-10-21T17:38:46.554078+0000","last_unstale":"2023-10-21T17:38:47.543198+0000","last_undegraded":"2023-10-21T17:38:47.543198+0000","last_fullsized":"2023-10-21T17:38:47.543198+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:38.308297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555935+0000","last_change":"2023-10-21T17:38:47.558370+0000","last_active":"2023-10-21T17:38:48.555935+0000","last_peered":"2023-10-21T17:38:48.555935+0000","last_clean":"2023-10-21T17:38:48.555935+0000","last_became_active":"2023-10-21T17:38:47.558070+0000","last_became_peered":"2023-10-21T17:38:47.558070+0000","last_unstale":"2023-10-21T17:38:48.555935+0000","last_undegraded":"2023-10-21T17:38:48.555935+0000","last_fullsized":"2023-10-21T17:38:48.555935+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:35:59.802031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.538996+0000","last_change":"2023-10-21T17:38:32.406815+0000","last_active":"2023-10-21T17:38:46.538996+0000","last_peered":"2023-10-21T17:38:46.538996+0000","last_clean":"2023-10-21T17:38:46.538996+0000","last_became_active":"2023-10-21T17:38:32.406525+0000","last_became_peered":"2023-10-21T17:38:32.406525+0000","last_unstale":"2023-10-21T17:38:46.538996+0000","last_undegraded":"2023-10-21T17:38:46.538996+0000","last_fullsized":"2023-10-21T17:38:46.538996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:01.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532596+0000","last_change":"2023-10-21T17:38:33.343136+0000","last_active":"2023-10-21T17:38:46.532596+0000","last_peered":"2023-10-21T17:38:46.532596+0000","last_clean":"2023-10-21T17:38:46.532596+0000","last_became_active":"2023-10-21T17:38:33.342344+0000","last_became_peered":"2023-10-21T17:38:33.342344+0000","last_unstale":"2023-10-21T17:38:46.532596+0000","last_undegraded":"2023-10-21T17:38:46.532596+0000","last_fullsized":"2023-10-21T17:38:46.532596+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:20:01.763000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557679+0000","last_change":"2023-10-21T17:38:46.567745+0000","last_active":"2023-10-21T17:38:48.557679+0000","last_peered":"2023-10-21T17:38:48.557679+0000","last_clean":"2023-10-21T17:38:48.557679+0000","last_became_active":"2023-10-21T17:38:46.567182+0000","last_became_peered":"2023-10-21T17:38:46.567182+0000","last_unstale":"2023-10-21T17:38:48.557679+0000","last_undegraded":"2023-10-21T17:38:48.557679+0000","last_fullsized":"2023-10-21T17:38:48.557679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:56:15.194684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565379+0000","last_change":"2023-10-21T17:38:47.559506+0000","last_active":"2023-10-21T17:38:47.565379+0000","last_peered":"2023-10-21T17:38:47.565379+0000","last_clean":"2023-10-21T17:38:47.565379+0000","last_became_active":"2023-10-21T17:38:47.559218+0000","last_became_peered":"2023-10-21T17:38:47.559218+0000","last_unstale":"2023-10-21T17:38:47.565379+0000","last_undegraded":"2023-10-21T17:38:47.565379+0000","last_fullsized":"2023-10-21T17:38:47.565379+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:43.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536048+0000","last_change":"2023-10-21T17:38:32.413129+0000","last_active":"2023-10-21T17:38:46.536048+0000","last_peered":"2023-10-21T17:38:46.536048+0000","last_clean":"2023-10-21T17:38:46.536048+0000","last_became_active":"2023-10-21T17:38:32.410406+0000","last_became_peered":"2023-10-21T17:38:32.410406+0000","last_unstale":"2023-10-21T17:38:46.536048+0000","last_undegraded":"2023-10-21T17:38:46.536048+0000","last_fullsized":"2023-10-21T17:38:46.536048+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:30.006550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536664+0000","last_change":"2023-10-21T17:38:33.358480+0000","last_active":"2023-10-21T17:38:46.536664+0000","last_peered":"2023-10-21T17:38:46.536664+0000","last_clean":"2023-10-21T17:38:46.536664+0000","last_became_active":"2023-10-21T17:38:33.358208+0000","last_became_peered":"2023-10-21T17:38:33.358208+0000","last_unstale":"2023-10-21T17:38:46.536664+0000","last_undegraded":"2023-10-21T17:38:46.536664+0000","last_fullsized":"2023-10-21T17:38:46.536664+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:22.754068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555023+0000","last_change":"2023-10-21T17:38:46.548373+0000","last_active":"2023-10-21T17:38:48.555023+0000","last_peered":"2023-10-21T17:38:48.555023+0000","last_clean":"2023-10-21T17:38:48.555023+0000","last_became_active":"2023-10-21T17:38:46.547728+0000","last_became_peered":"2023-10-21T17:38:46.547728+0000","last_unstale":"2023-10-21T17:38:48.555023+0000","last_undegraded":"2023-10-21T17:38:48.555023+0000","last_fullsized":"2023-10-21T17:38:48.555023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:04:11.654326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552067+0000","last_change":"2023-10-21T17:38:47.557513+0000","last_active":"2023-10-21T17:38:48.552067+0000","last_peered":"2023-10-21T17:38:48.552067+0000","last_clean":"2023-10-21T17:38:48.552067+0000","last_became_active":"2023-10-21T17:38:47.556804+0000","last_became_peered":"2023-10-21T17:38:47.556804+0000","last_unstale":"2023-10-21T17:38:48.552067+0000","last_undegraded":"2023-10-21T17:38:48.552067+0000","last_fullsized":"2023-10-21T17:38:48.552067+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:51:20.761534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534290+0000","last_change":"2023-10-21T17:38:32.404935+0000","last_active":"2023-10-21T17:38:46.534290+0000","last_peered":"2023-10-21T17:38:46.534290+0000","last_clean":"2023-10-21T17:38:46.534290+0000","last_became_active":"2023-10-21T17:38:32.404139+0000","last_became_peered":"2023-10-21T17:38:32.404139+0000","last_unstale":"2023-10-21T17:38:46.534290+0000","last_undegraded":"2023-10-21T17:38:46.534290+0000","last_fullsized":"2023-10-21T17:38:46.534290+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:53.425393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541314+0000","last_change":"2023-10-21T17:38:33.357202+0000","last_active":"2023-10-21T17:38:47.541314+0000","last_peered":"2023-10-21T17:38:47.541314+0000","last_clean":"2023-10-21T17:38:47.541314+0000","last_became_active":"2023-10-21T17:38:33.356898+0000","last_became_peered":"2023-10-21T17:38:33.356898+0000","last_unstale":"2023-10-21T17:38:47.541314+0000","last_undegraded":"2023-10-21T17:38:47.541314+0000","last_fullsized":"2023-10-21T17:38:47.541314+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:39.055132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542887+0000","last_change":"2023-10-21T17:38:46.558734+0000","last_active":"2023-10-21T17:38:47.542887+0000","last_peered":"2023-10-21T17:38:47.542887+0000","last_clean":"2023-10-21T17:38:47.542887+0000","last_became_active":"2023-10-21T17:38:46.558210+0000","last_became_peered":"2023-10-21T17:38:46.558210+0000","last_unstale":"2023-10-21T17:38:47.542887+0000","last_undegraded":"2023-10-21T17:38:47.542887+0000","last_fullsized":"2023-10-21T17:38:47.542887+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:35.873121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559702+0000","last_change":"2023-10-21T17:38:47.549325+0000","last_active":"2023-10-21T17:38:47.559702+0000","last_peered":"2023-10-21T17:38:47.559702+0000","last_clean":"2023-10-21T17:38:47.559702+0000","last_became_active":"2023-10-21T17:38:47.549058+0000","last_became_peered":"2023-10-21T17:38:47.549058+0000","last_unstale":"2023-10-21T17:38:47.559702+0000","last_undegraded":"2023-10-21T17:38:47.559702+0000","last_fullsized":"2023-10-21T17:38:47.559702+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:51.612715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"232'8","reported_seq":24,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.526058+0000","last_change":"2023-10-21T17:38:32.411814+0000","last_active":"2023-10-21T17:38:47.526058+0000","last_peered":"2023-10-21T17:38:47.526058+0000","last_clean":"2023-10-21T17:38:47.526058+0000","last_became_active":"2023-10-21T17:38:32.411496+0000","last_became_peered":"2023-10-21T17:38:32.411496+0000","last_unstale":"2023-10-21T17:38:47.526058+0000","last_undegraded":"2023-10-21T17:38:47.526058+0000","last_fullsized":"2023-10-21T17:38:47.526058+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:17.401857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.558839+0000","last_change":"2023-10-21T17:38:33.362626+0000","last_active":"2023-10-21T17:38:46.558839+0000","last_peered":"2023-10-21T17:38:46.558839+0000","last_clean":"2023-10-21T17:38:46.558839+0000","last_became_active":"2023-10-21T17:38:33.361673+0000","last_became_peered":"2023-10-21T17:38:33.361673+0000","last_unstale":"2023-10-21T17:38:46.558839+0000","last_undegraded":"2023-10-21T17:38:46.558839+0000","last_fullsized":"2023-10-21T17:38:46.558839+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:17.775611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530168+0000","last_change":"2023-10-21T17:38:46.560755+0000","last_active":"2023-10-21T17:38:47.530168+0000","last_peered":"2023-10-21T17:38:47.530168+0000","last_clean":"2023-10-21T17:38:47.530168+0000","last_became_active":"2023-10-21T17:38:46.560473+0000","last_became_peered":"2023-10-21T17:38:46.560473+0000","last_unstale":"2023-10-21T17:38:47.530168+0000","last_undegraded":"2023-10-21T17:38:47.530168+0000","last_fullsized":"2023-10-21T17:38:47.530168+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:43.117413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556027+0000","last_change":"2023-10-21T17:38:47.547467+0000","last_active":"2023-10-21T17:38:47.556027+0000","last_peered":"2023-10-21T17:38:47.556027+0000","last_clean":"2023-10-21T17:38:47.556027+0000","last_became_active":"2023-10-21T17:38:47.547084+0000","last_became_peered":"2023-10-21T17:38:47.547084+0000","last_unstale":"2023-10-21T17:38:47.556027+0000","last_undegraded":"2023-10-21T17:38:47.556027+0000","last_fullsized":"2023-10-21T17:38:47.556027+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:45.159463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545340+0000","last_change":"2023-10-21T17:38:32.405540+0000","last_active":"2023-10-21T17:38:47.545340+0000","last_peered":"2023-10-21T17:38:47.545340+0000","last_clean":"2023-10-21T17:38:47.545340+0000","last_became_active":"2023-10-21T17:38:32.405267+0000","last_became_peered":"2023-10-21T17:38:32.405267+0000","last_unstale":"2023-10-21T17:38:47.545340+0000","last_undegraded":"2023-10-21T17:38:47.545340+0000","last_fullsized":"2023-10-21T17:38:47.545340+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:15.824449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550924+0000","last_change":"2023-10-21T17:38:33.358210+0000","last_active":"2023-10-21T17:38:47.550924+0000","last_peered":"2023-10-21T17:38:47.550924+0000","last_clean":"2023-10-21T17:38:47.550924+0000","last_became_active":"2023-10-21T17:38:33.357911+0000","last_became_peered":"2023-10-21T17:38:33.357911+0000","last_unstale":"2023-10-21T17:38:47.550924+0000","last_undegraded":"2023-10-21T17:38:47.550924+0000","last_fullsized":"2023-10-21T17:38:47.550924+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:29.632092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542844+0000","last_change":"2023-10-21T17:38:46.564259+0000","last_active":"2023-10-21T17:38:47.542844+0000","last_peered":"2023-10-21T17:38:47.542844+0000","last_clean":"2023-10-21T17:38:47.542844+0000","last_became_active":"2023-10-21T17:38:46.563937+0000","last_became_peered":"2023-10-21T17:38:46.563937+0000","last_unstale":"2023-10-21T17:38:47.542844+0000","last_undegraded":"2023-10-21T17:38:47.542844+0000","last_fullsized":"2023-10-21T17:38:47.542844+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:54:14.193241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557388+0000","last_change":"2023-10-21T17:38:47.552857+0000","last_active":"2023-10-21T17:38:47.557388+0000","last_peered":"2023-10-21T17:38:47.557388+0000","last_clean":"2023-10-21T17:38:47.557388+0000","last_became_active":"2023-10-21T17:38:47.552609+0000","last_became_peered":"2023-10-21T17:38:47.552609+0000","last_unstale":"2023-10-21T17:38:47.557388+0000","last_undegraded":"2023-10-21T17:38:47.557388+0000","last_fullsized":"2023-10-21T17:38:47.557388+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:58.603914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535330+0000","last_change":"2023-10-21T17:38:32.403441+0000","last_active":"2023-10-21T17:38:46.535330+0000","last_peered":"2023-10-21T17:38:46.535330+0000","last_clean":"2023-10-21T17:38:46.535330+0000","last_became_active":"2023-10-21T17:38:32.403179+0000","last_became_peered":"2023-10-21T17:38:32.403179+0000","last_unstale":"2023-10-21T17:38:46.535330+0000","last_undegraded":"2023-10-21T17:38:46.535330+0000","last_fullsized":"2023-10-21T17:38:46.535330+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:38.273786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535318+0000","last_change":"2023-10-21T17:38:33.351625+0000","last_active":"2023-10-21T17:38:46.535318+0000","last_peered":"2023-10-21T17:38:46.535318+0000","last_clean":"2023-10-21T17:38:46.535318+0000","last_became_active":"2023-10-21T17:38:33.351311+0000","last_became_peered":"2023-10-21T17:38:33.351311+0000","last_unstale":"2023-10-21T17:38:46.535318+0000","last_undegraded":"2023-10-21T17:38:46.535318+0000","last_fullsized":"2023-10-21T17:38:46.535318+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:07.568107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"236'6","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.577695+0000","last_change":"2023-10-21T17:38:46.564895+0000","last_active":"2023-10-21T17:38:47.577695+0000","last_peered":"2023-10-21T17:38:47.577695+0000","last_clean":"2023-10-21T17:38:47.577695+0000","last_became_active":"2023-10-21T17:38:46.564596+0000","last_became_peered":"2023-10-21T17:38:46.564596+0000","last_unstale":"2023-10-21T17:38:47.577695+0000","last_undegraded":"2023-10-21T17:38:47.577695+0000","last_fullsized":"2023-10-21T17:38:47.577695+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:03.944786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.560512+0000","last_change":"2023-10-21T17:38:47.557562+0000","last_active":"2023-10-21T17:38:47.560512+0000","last_peered":"2023-10-21T17:38:47.560512+0000","last_clean":"2023-10-21T17:38:47.560512+0000","last_became_active":"2023-10-21T17:38:47.556767+0000","last_became_peered":"2023-10-21T17:38:47.556767+0000","last_unstale":"2023-10-21T17:38:47.560512+0000","last_undegraded":"2023-10-21T17:38:47.560512+0000","last_fullsized":"2023-10-21T17:38:47.560512+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:32.693500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535404+0000","last_change":"2023-10-21T17:38:32.405640+0000","last_active":"2023-10-21T17:38:46.535404+0000","last_peered":"2023-10-21T17:38:46.535404+0000","last_clean":"2023-10-21T17:38:46.535404+0000","last_became_active":"2023-10-21T17:38:32.404457+0000","last_became_peered":"2023-10-21T17:38:32.404457+0000","last_unstale":"2023-10-21T17:38:46.535404+0000","last_undegraded":"2023-10-21T17:38:46.535404+0000","last_fullsized":"2023-10-21T17:38:46.535404+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:26:53.247310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535265+0000","last_change":"2023-10-21T17:38:33.347261+0000","last_active":"2023-10-21T17:38:46.535265+0000","last_peered":"2023-10-21T17:38:46.535265+0000","last_clean":"2023-10-21T17:38:46.535265+0000","last_became_active":"2023-10-21T17:38:33.346927+0000","last_became_peered":"2023-10-21T17:38:33.346927+0000","last_unstale":"2023-10-21T17:38:46.535265+0000","last_undegraded":"2023-10-21T17:38:46.535265+0000","last_fullsized":"2023-10-21T17:38:46.535265+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:04:01.340329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"236'3","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.569620+0000","last_change":"2023-10-21T17:38:46.564790+0000","last_active":"2023-10-21T17:38:47.569620+0000","last_peered":"2023-10-21T17:38:47.569620+0000","last_clean":"2023-10-21T17:38:47.569620+0000","last_became_active":"2023-10-21T17:38:46.564494+0000","last_became_peered":"2023-10-21T17:38:46.564494+0000","last_unstale":"2023-10-21T17:38:47.569620+0000","last_undegraded":"2023-10-21T17:38:47.569620+0000","last_fullsized":"2023-10-21T17:38:47.569620+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:12.102091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557080+0000","last_change":"2023-10-21T17:38:47.554422+0000","last_active":"2023-10-21T17:38:48.557080+0000","last_peered":"2023-10-21T17:38:48.557080+0000","last_clean":"2023-10-21T17:38:48.557080+0000","last_became_active":"2023-10-21T17:38:47.554111+0000","last_became_peered":"2023-10-21T17:38:47.554111+0000","last_unstale":"2023-10-21T17:38:48.557080+0000","last_undegraded":"2023-10-21T17:38:48.557080+0000","last_fullsized":"2023-10-21T17:38:48.557080+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:04.797866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":23,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559111+0000","last_change":"2023-10-21T17:38:32.416726+0000","last_active":"2023-10-21T17:38:46.559111+0000","last_peered":"2023-10-21T17:38:46.559111+0000","last_clean":"2023-10-21T17:38:46.559111+0000","last_became_active":"2023-10-21T17:38:32.416448+0000","last_became_peered":"2023-10-21T17:38:32.416448+0000","last_unstale":"2023-10-21T17:38:46.559111+0000","last_undegraded":"2023-10-21T17:38:46.559111+0000","last_fullsized":"2023-10-21T17:38:46.559111+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530062+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:47.530062+0000","last_peered":"2023-10-21T17:38:47.530062+0000","last_clean":"2023-10-21T17:38:47.530062+0000","last_became_active":"2023-10-21T17:38:33.361077+0000","last_became_peered":"2023-10-21T17:38:33.361077+0000","last_unstale":"2023-10-21T17:38:47.530062+0000","last_undegraded":"2023-10-21T17:38:47.530062+0000","last_fullsized":"2023-10-21T17:38:47.530062+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:09.263356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552256+0000","last_change":"2023-10-21T17:38:46.545526+0000","last_active":"2023-10-21T17:38:48.552256+0000","last_peered":"2023-10-21T17:38:48.552256+0000","last_clean":"2023-10-21T17:38:48.552256+0000","last_became_active":"2023-10-21T17:38:46.544826+0000","last_became_peered":"2023-10-21T17:38:46.544826+0000","last_unstale":"2023-10-21T17:38:48.552256+0000","last_undegraded":"2023-10-21T17:38:48.552256+0000","last_fullsized":"2023-10-21T17:38:48.552256+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:38.652056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552909+0000","last_change":"2023-10-21T17:38:47.553575+0000","last_active":"2023-10-21T17:38:48.552909+0000","last_peered":"2023-10-21T17:38:48.552909+0000","last_clean":"2023-10-21T17:38:48.552909+0000","last_became_active":"2023-10-21T17:38:47.553314+0000","last_became_peered":"2023-10-21T17:38:47.553314+0000","last_unstale":"2023-10-21T17:38:48.552909+0000","last_undegraded":"2023-10-21T17:38:48.552909+0000","last_fullsized":"2023-10-21T17:38:48.552909+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:16:10.316538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"232'3","reported_seq":19,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.540887+0000","last_change":"2023-10-21T17:38:32.404797+0000","last_active":"2023-10-21T17:38:47.540887+0000","last_peered":"2023-10-21T17:38:47.540887+0000","last_clean":"2023-10-21T17:38:47.540887+0000","last_became_active":"2023-10-21T17:38:32.404524+0000","last_became_peered":"2023-10-21T17:38:32.404524+0000","last_unstale":"2023-10-21T17:38:47.540887+0000","last_undegraded":"2023-10-21T17:38:47.540887+0000","last_fullsized":"2023-10-21T17:38:47.540887+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:07.928480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538195+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:47.538195+0000","last_peered":"2023-10-21T17:38:47.538195+0000","last_clean":"2023-10-21T17:38:47.538195+0000","last_became_active":"2023-10-21T17:38:33.361078+0000","last_became_peered":"2023-10-21T17:38:33.361078+0000","last_unstale":"2023-10-21T17:38:47.538195+0000","last_undegraded":"2023-10-21T17:38:47.538195+0000","last_fullsized":"2023-10-21T17:38:47.538195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:24.254117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557435+0000","last_change":"2023-10-21T17:38:46.560608+0000","last_active":"2023-10-21T17:38:48.557435+0000","last_peered":"2023-10-21T17:38:48.557435+0000","last_clean":"2023-10-21T17:38:48.557435+0000","last_became_active":"2023-10-21T17:38:46.560262+0000","last_became_peered":"2023-10-21T17:38:46.560262+0000","last_unstale":"2023-10-21T17:38:48.557435+0000","last_undegraded":"2023-10-21T17:38:48.557435+0000","last_fullsized":"2023-10-21T17:38:48.557435+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:32.612494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561172+0000","last_change":"2023-10-21T17:38:47.558430+0000","last_active":"2023-10-21T17:38:47.561172+0000","last_peered":"2023-10-21T17:38:47.561172+0000","last_clean":"2023-10-21T17:38:47.561172+0000","last_became_active":"2023-10-21T17:38:47.558111+0000","last_became_peered":"2023-10-21T17:38:47.558111+0000","last_unstale":"2023-10-21T17:38:47.561172+0000","last_undegraded":"2023-10-21T17:38:47.561172+0000","last_fullsized":"2023-10-21T17:38:47.561172+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:08.578306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559258+0000","last_change":"2023-10-21T17:38:32.414100+0000","last_active":"2023-10-21T17:38:46.559258+0000","last_peered":"2023-10-21T17:38:46.559258+0000","last_clean":"2023-10-21T17:38:46.559258+0000","last_became_active":"2023-10-21T17:38:32.413503+0000","last_became_peered":"2023-10-21T17:38:32.413503+0000","last_unstale":"2023-10-21T17:38:46.559258+0000","last_undegraded":"2023-10-21T17:38:46.559258+0000","last_fullsized":"2023-10-21T17:38:46.559258+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:41.928628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532513+0000","last_change":"2023-10-21T17:38:33.359536+0000","last_active":"2023-10-21T17:38:46.532513+0000","last_peered":"2023-10-21T17:38:46.532513+0000","last_clean":"2023-10-21T17:38:46.532513+0000","last_became_active":"2023-10-21T17:38:33.358634+0000","last_became_peered":"2023-10-21T17:38:33.358634+0000","last_unstale":"2023-10-21T17:38:46.532513+0000","last_undegraded":"2023-10-21T17:38:46.532513+0000","last_fullsized":"2023-10-21T17:38:46.532513+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:18.891311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"236'8","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556076+0000","last_change":"2023-10-21T17:38:46.563699+0000","last_active":"2023-10-21T17:38:48.556076+0000","last_peered":"2023-10-21T17:38:48.556076+0000","last_clean":"2023-10-21T17:38:48.556076+0000","last_became_active":"2023-10-21T17:38:46.563400+0000","last_became_peered":"2023-10-21T17:38:46.563400+0000","last_unstale":"2023-10-21T17:38:48.556076+0000","last_undegraded":"2023-10-21T17:38:48.556076+0000","last_fullsized":"2023-10-21T17:38:48.556076+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:36.282720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.555680+0000","last_change":"2023-10-21T17:38:47.552389+0000","last_active":"2023-10-21T17:38:47.555680+0000","last_peered":"2023-10-21T17:38:47.555680+0000","last_clean":"2023-10-21T17:38:47.555680+0000","last_became_active":"2023-10-21T17:38:47.550867+0000","last_became_peered":"2023-10-21T17:38:47.550867+0000","last_unstale":"2023-10-21T17:38:47.555680+0000","last_undegraded":"2023-10-21T17:38:47.555680+0000","last_fullsized":"2023-10-21T17:38:47.555680+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:16:44.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530413+0000","last_change":"2023-10-21T17:38:32.417638+0000","last_active":"2023-10-21T17:38:47.530413+0000","last_peered":"2023-10-21T17:38:47.530413+0000","last_clean":"2023-10-21T17:38:47.530413+0000","last_became_active":"2023-10-21T17:38:32.417302+0000","last_became_peered":"2023-10-21T17:38:32.417302+0000","last_unstale":"2023-10-21T17:38:47.530413+0000","last_undegraded":"2023-10-21T17:38:47.530413+0000","last_fullsized":"2023-10-21T17:38:47.530413+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:22.089541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.545681+0000","last_change":"2023-10-21T17:38:33.349513+0000","last_active":"2023-10-21T17:38:46.545681+0000","last_peered":"2023-10-21T17:38:46.545681+0000","last_clean":"2023-10-21T17:38:46.545681+0000","last_became_active":"2023-10-21T17:38:33.349202+0000","last_became_peered":"2023-10-21T17:38:33.349202+0000","last_unstale":"2023-10-21T17:38:46.545681+0000","last_undegraded":"2023-10-21T17:38:46.545681+0000","last_fullsized":"2023-10-21T17:38:46.545681+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:33:14.663384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.522182+0000","last_change":"2023-10-21T17:38:46.539924+0000","last_active":"2023-10-21T17:38:47.522182+0000","last_peered":"2023-10-21T17:38:47.522182+0000","last_clean":"2023-10-21T17:38:47.522182+0000","last_became_active":"2023-10-21T17:38:46.539284+0000","last_became_peered":"2023-10-21T17:38:46.539284+0000","last_unstale":"2023-10-21T17:38:47.522182+0000","last_undegraded":"2023-10-21T17:38:47.522182+0000","last_fullsized":"2023-10-21T17:38:47.522182+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:41.174817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566536+0000","last_change":"2023-10-21T17:38:47.563567+0000","last_active":"2023-10-21T17:38:47.566536+0000","last_peered":"2023-10-21T17:38:47.566536+0000","last_clean":"2023-10-21T17:38:47.566536+0000","last_became_active":"2023-10-21T17:38:47.563295+0000","last_became_peered":"2023-10-21T17:38:47.563295+0000","last_unstale":"2023-10-21T17:38:47.566536+0000","last_undegraded":"2023-10-21T17:38:47.566536+0000","last_fullsized":"2023-10-21T17:38:47.566536+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:00.316336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"236'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.559065+0000","last_change":"2023-10-21T17:38:46.566907+0000","last_active":"2023-10-21T17:38:48.559065+0000","last_peered":"2023-10-21T17:38:48.559065+0000","last_clean":"2023-10-21T17:38:48.559065+0000","last_became_active":"2023-10-21T17:38:46.566616+0000","last_became_peered":"2023-10-21T17:38:46.566616+0000","last_unstale":"2023-10-21T17:38:48.559065+0000","last_undegraded":"2023-10-21T17:38:48.559065+0000","last_fullsized":"2023-10-21T17:38:48.559065+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:50.826966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531845+0000","last_change":"2023-10-21T17:38:32.410852+0000","last_active":"2023-10-21T17:38:46.531845+0000","last_peered":"2023-10-21T17:38:46.531845+0000","last_clean":"2023-10-21T17:38:46.531845+0000","last_became_active":"2023-10-21T17:38:32.410413+0000","last_became_peered":"2023-10-21T17:38:32.410413+0000","last_unstale":"2023-10-21T17:38:46.531845+0000","last_undegraded":"2023-10-21T17:38:46.531845+0000","last_fullsized":"2023-10-21T17:38:46.531845+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:26.544773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541647+0000","last_change":"2023-10-21T17:38:33.357688+0000","last_active":"2023-10-21T17:38:47.541647+0000","last_peered":"2023-10-21T17:38:47.541647+0000","last_clean":"2023-10-21T17:38:47.541647+0000","last_became_active":"2023-10-21T17:38:33.355135+0000","last_became_peered":"2023-10-21T17:38:33.355135+0000","last_unstale":"2023-10-21T17:38:47.541647+0000","last_undegraded":"2023-10-21T17:38:47.541647+0000","last_fullsized":"2023-10-21T17:38:47.541647+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:10.952499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.554607+0000","last_change":"2023-10-21T17:38:47.552201+0000","last_active":"2023-10-21T17:38:47.554607+0000","last_peered":"2023-10-21T17:38:47.554607+0000","last_clean":"2023-10-21T17:38:47.554607+0000","last_became_active":"2023-10-21T17:38:47.551908+0000","last_became_peered":"2023-10-21T17:38:47.551908+0000","last_unstale":"2023-10-21T17:38:47.554607+0000","last_undegraded":"2023-10-21T17:38:47.554607+0000","last_fullsized":"2023-10-21T17:38:47.554607+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:43:30.804596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"236'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556590+0000","last_change":"2023-10-21T17:38:46.558998+0000","last_active":"2023-10-21T17:38:48.556590+0000","last_peered":"2023-10-21T17:38:48.556590+0000","last_clean":"2023-10-21T17:38:48.556590+0000","last_became_active":"2023-10-21T17:38:46.558681+0000","last_became_peered":"2023-10-21T17:38:46.558681+0000","last_unstale":"2023-10-21T17:38:48.556590+0000","last_undegraded":"2023-10-21T17:38:48.556590+0000","last_fullsized":"2023-10-21T17:38:48.556590+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:18.054503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552420+0000","last_change":"2023-10-21T17:38:32.411637+0000","last_active":"2023-10-21T17:38:46.552420+0000","last_peered":"2023-10-21T17:38:46.552420+0000","last_clean":"2023-10-21T17:38:46.552420+0000","last_became_active":"2023-10-21T17:38:32.411322+0000","last_became_peered":"2023-10-21T17:38:32.411322+0000","last_unstale":"2023-10-21T17:38:46.552420+0000","last_undegraded":"2023-10-21T17:38:46.552420+0000","last_fullsized":"2023-10-21T17:38:46.552420+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.616148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.553870+0000","last_change":"2023-10-21T17:38:33.346021+0000","last_active":"2023-10-21T17:38:46.553870+0000","last_peered":"2023-10-21T17:38:46.553870+0000","last_clean":"2023-10-21T17:38:46.553870+0000","last_became_active":"2023-10-21T17:38:33.345731+0000","last_became_peered":"2023-10-21T17:38:33.345731+0000","last_unstale":"2023-10-21T17:38:46.553870+0000","last_undegraded":"2023-10-21T17:38:46.553870+0000","last_fullsized":"2023-10-21T17:38:46.553870+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:29.919088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558981+0000","last_change":"2023-10-21T17:38:47.558622+0000","last_active":"2023-10-21T17:38:48.558981+0000","last_peered":"2023-10-21T17:38:48.558981+0000","last_clean":"2023-10-21T17:38:48.558981+0000","last_became_active":"2023-10-21T17:38:47.557794+0000","last_became_peered":"2023-10-21T17:38:47.557794+0000","last_unstale":"2023-10-21T17:38:48.558981+0000","last_undegraded":"2023-10-21T17:38:48.558981+0000","last_fullsized":"2023-10-21T17:38:48.558981+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:07.090483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.557679+0000","last_change":"2023-10-21T17:38:32.411640+0000","last_active":"2023-10-21T17:38:46.557679+0000","last_peered":"2023-10-21T17:38:46.557679+0000","last_clean":"2023-10-21T17:38:46.557679+0000","last_became_active":"2023-10-21T17:38:32.411258+0000","last_became_peered":"2023-10-21T17:38:32.411258+0000","last_unstale":"2023-10-21T17:38:46.557679+0000","last_undegraded":"2023-10-21T17:38:46.557679+0000","last_fullsized":"2023-10-21T17:38:46.557679+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552008+0000","last_change":"2023-10-21T17:38:33.341123+0000","last_active":"2023-10-21T17:38:46.552008+0000","last_peered":"2023-10-21T17:38:46.552008+0000","last_clean":"2023-10-21T17:38:46.552008+0000","last_became_active":"2023-10-21T17:38:33.340397+0000","last_became_peered":"2023-10-21T17:38:33.340397+0000","last_unstale":"2023-10-21T17:38:46.552008+0000","last_undegraded":"2023-10-21T17:38:46.552008+0000","last_fullsized":"2023-10-21T17:38:46.552008+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:39.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"236'2","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.572850+0000","last_change":"2023-10-21T17:38:46.566632+0000","last_active":"2023-10-21T17:38:47.572850+0000","last_peered":"2023-10-21T17:38:47.572850+0000","last_clean":"2023-10-21T17:38:47.572850+0000","last_became_active":"2023-10-21T17:38:46.566333+0000","last_became_peered":"2023-10-21T17:38:46.566333+0000","last_unstale":"2023-10-21T17:38:47.572850+0000","last_undegraded":"2023-10-21T17:38:47.572850+0000","last_fullsized":"2023-10-21T17:38:47.572850+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:00:51.103062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556817+0000","last_change":"2023-10-21T17:38:47.561183+0000","last_active":"2023-10-21T17:38:48.556817+0000","last_peered":"2023-10-21T17:38:48.556817+0000","last_clean":"2023-10-21T17:38:48.556817+0000","last_became_active":"2023-10-21T17:38:47.560072+0000","last_became_peered":"2023-10-21T17:38:47.560072+0000","last_unstale":"2023-10-21T17:38:48.556817+0000","last_undegraded":"2023-10-21T17:38:48.556817+0000","last_fullsized":"2023-10-21T17:38:48.556817+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:31.201096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.548545+0000","last_change":"2023-10-21T17:38:32.407056+0000","last_active":"2023-10-21T17:38:47.548545+0000","last_peered":"2023-10-21T17:38:47.548545+0000","last_clean":"2023-10-21T17:38:47.548545+0000","last_became_active":"2023-10-21T17:38:32.405548+0000","last_became_peered":"2023-10-21T17:38:32.405548+0000","last_unstale":"2023-10-21T17:38:47.548545+0000","last_undegraded":"2023-10-21T17:38:47.548545+0000","last_fullsized":"2023-10-21T17:38:47.548545+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:02.940331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529805+0000","last_change":"2023-10-21T17:38:33.350688+0000","last_active":"2023-10-21T17:38:47.529805+0000","last_peered":"2023-10-21T17:38:47.529805+0000","last_clean":"2023-10-21T17:38:47.529805+0000","last_became_active":"2023-10-21T17:38:33.350383+0000","last_became_peered":"2023-10-21T17:38:33.350383+0000","last_unstale":"2023-10-21T17:38:47.529805+0000","last_undegraded":"2023-10-21T17:38:47.529805+0000","last_fullsized":"2023-10-21T17:38:47.529805+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:05.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570003+0000","last_change":"2023-10-21T17:38:46.570202+0000","last_active":"2023-10-21T17:38:47.570003+0000","last_peered":"2023-10-21T17:38:47.570003+0000","last_clean":"2023-10-21T17:38:47.570003+0000","last_became_active":"2023-10-21T17:38:46.569832+0000","last_became_peered":"2023-10-21T17:38:46.569832+0000","last_unstale":"2023-10-21T17:38:47.570003+0000","last_undegraded":"2023-10-21T17:38:47.570003+0000","last_fullsized":"2023-10-21T17:38:47.570003+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:33:06.683362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559579+0000","last_change":"2023-10-21T17:38:47.556629+0000","last_active":"2023-10-21T17:38:47.559579+0000","last_peered":"2023-10-21T17:38:47.559579+0000","last_clean":"2023-10-21T17:38:47.559579+0000","last_became_active":"2023-10-21T17:38:47.555973+0000","last_became_peered":"2023-10-21T17:38:47.555973+0000","last_unstale":"2023-10-21T17:38:47.559579+0000","last_undegraded":"2023-10-21T17:38:47.559579+0000","last_fullsized":"2023-10-21T17:38:47.559579+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:49:13.600687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536443+0000","last_change":"2023-10-21T17:38:32.405671+0000","last_active":"2023-10-21T17:38:46.536443+0000","last_peered":"2023-10-21T17:38:46.536443+0000","last_clean":"2023-10-21T17:38:46.536443+0000","last_became_active":"2023-10-21T17:38:32.404175+0000","last_became_peered":"2023-10-21T17:38:32.404175+0000","last_unstale":"2023-10-21T17:38:46.536443+0000","last_undegraded":"2023-10-21T17:38:46.536443+0000","last_fullsized":"2023-10-21T17:38:46.536443+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:24.818705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542379+0000","last_change":"2023-10-21T17:38:33.360687+0000","last_active":"2023-10-21T17:38:47.542379+0000","last_peered":"2023-10-21T17:38:47.542379+0000","last_clean":"2023-10-21T17:38:47.542379+0000","last_became_active":"2023-10-21T17:38:33.360369+0000","last_became_peered":"2023-10-21T17:38:33.360369+0000","last_unstale":"2023-10-21T17:38:47.542379+0000","last_undegraded":"2023-10-21T17:38:47.542379+0000","last_fullsized":"2023-10-21T17:38:47.542379+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:04.002929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570688+0000","last_change":"2023-10-21T17:38:46.564099+0000","last_active":"2023-10-21T17:38:47.570688+0000","last_peered":"2023-10-21T17:38:47.570688+0000","last_clean":"2023-10-21T17:38:47.570688+0000","last_became_active":"2023-10-21T17:38:46.563814+0000","last_became_peered":"2023-10-21T17:38:46.563814+0000","last_unstale":"2023-10-21T17:38:47.570688+0000","last_undegraded":"2023-10-21T17:38:47.570688+0000","last_fullsized":"2023-10-21T17:38:47.570688+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:38.652056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555998+0000","last_change":"2023-10-21T17:38:47.550728+0000","last_active":"2023-10-21T17:38:48.555998+0000","last_peered":"2023-10-21T17:38:48.555998+0000","last_clean":"2023-10-21T17:38:48.555998+0000","last_became_active":"2023-10-21T17:38:47.549778+0000","last_became_peered":"2023-10-21T17:38:47.549778+0000","last_unstale":"2023-10-21T17:38:48.555998+0000","last_undegraded":"2023-10-21T17:38:48.555998+0000","last_fullsized":"2023-10-21T17:38:48.555998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:02.913327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535472+0000","last_change":"2023-10-21T17:38:32.401420+0000","last_active":"2023-10-21T17:38:46.535472+0000","last_peered":"2023-10-21T17:38:46.535472+0000","last_clean":"2023-10-21T17:38:46.535472+0000","last_became_active":"2023-10-21T17:38:32.394692+0000","last_became_peered":"2023-10-21T17:38:32.394692+0000","last_unstale":"2023-10-21T17:38:46.535472+0000","last_undegraded":"2023-10-21T17:38:46.535472+0000","last_fullsized":"2023-10-21T17:38:46.535472+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536446+0000","last_change":"2023-10-21T17:38:33.352486+0000","last_active":"2023-10-21T17:38:46.536446+0000","last_peered":"2023-10-21T17:38:46.536446+0000","last_clean":"2023-10-21T17:38:46.536446+0000","last_became_active":"2023-10-21T17:38:33.352150+0000","last_became_peered":"2023-10-21T17:38:33.352150+0000","last_unstale":"2023-10-21T17:38:46.536446+0000","last_undegraded":"2023-10-21T17:38:46.536446+0000","last_fullsized":"2023-10-21T17:38:46.536446+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:53.603618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.544594+0000","last_change":"2023-10-21T17:38:46.544612+0000","last_active":"2023-10-21T17:38:47.544594+0000","last_peered":"2023-10-21T17:38:47.544594+0000","last_clean":"2023-10-21T17:38:47.544594+0000","last_became_active":"2023-10-21T17:38:46.544322+0000","last_became_peered":"2023-10-21T17:38:46.544322+0000","last_unstale":"2023-10-21T17:38:47.544594+0000","last_undegraded":"2023-10-21T17:38:47.544594+0000","last_fullsized":"2023-10-21T17:38:47.544594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:47.693310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557984+0000","last_change":"2023-10-21T17:38:47.552093+0000","last_active":"2023-10-21T17:38:47.557984+0000","last_peered":"2023-10-21T17:38:47.557984+0000","last_clean":"2023-10-21T17:38:47.557984+0000","last_became_active":"2023-10-21T17:38:47.551780+0000","last_became_peered":"2023-10-21T17:38:47.551780+0000","last_unstale":"2023-10-21T17:38:47.557984+0000","last_undegraded":"2023-10-21T17:38:47.557984+0000","last_fullsized":"2023-10-21T17:38:47.557984+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:24:07.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"232'4","reported_seq":19,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552776+0000","last_change":"2023-10-21T17:38:32.419554+0000","last_active":"2023-10-21T17:38:46.552776+0000","last_peered":"2023-10-21T17:38:46.552776+0000","last_clean":"2023-10-21T17:38:46.552776+0000","last_became_active":"2023-10-21T17:38:32.419276+0000","last_became_peered":"2023-10-21T17:38:32.419276+0000","last_unstale":"2023-10-21T17:38:46.552776+0000","last_undegraded":"2023-10-21T17:38:46.552776+0000","last_fullsized":"2023-10-21T17:38:46.552776+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:44.216819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534438+0000","last_change":"2023-10-21T17:38:33.350258+0000","last_active":"2023-10-21T17:38:46.534438+0000","last_peered":"2023-10-21T17:38:46.534438+0000","last_clean":"2023-10-21T17:38:46.534438+0000","last_became_active":"2023-10-21T17:38:33.349849+0000","last_became_peered":"2023-10-21T17:38:33.349849+0000","last_unstale":"2023-10-21T17:38:46.534438+0000","last_undegraded":"2023-10-21T17:38:46.534438+0000","last_fullsized":"2023-10-21T17:38:46.534438+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:08.906164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530075+0000","last_change":"2023-10-21T17:38:46.560891+0000","last_active":"2023-10-21T17:38:47.530075+0000","last_peered":"2023-10-21T17:38:47.530075+0000","last_clean":"2023-10-21T17:38:47.530075+0000","last_became_active":"2023-10-21T17:38:46.560549+0000","last_became_peered":"2023-10-21T17:38:46.560549+0000","last_unstale":"2023-10-21T17:38:47.530075+0000","last_undegraded":"2023-10-21T17:38:47.530075+0000","last_fullsized":"2023-10-21T17:38:47.530075+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:51.552421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565097+0000","last_change":"2023-10-21T17:38:47.559132+0000","last_active":"2023-10-21T17:38:47.565097+0000","last_peered":"2023-10-21T17:38:47.565097+0000","last_clean":"2023-10-21T17:38:47.565097+0000","last_became_active":"2023-10-21T17:38:47.558872+0000","last_became_peered":"2023-10-21T17:38:47.558872+0000","last_unstale":"2023-10-21T17:38:47.565097+0000","last_undegraded":"2023-10-21T17:38:47.565097+0000","last_fullsized":"2023-10-21T17:38:47.565097+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:35.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.556204+0000","last_change":"2023-10-21T17:38:33.354963+0000","last_active":"2023-10-21T17:38:46.556204+0000","last_peered":"2023-10-21T17:38:46.556204+0000","last_clean":"2023-10-21T17:38:46.556204+0000","last_became_active":"2023-10-21T17:38:33.354183+0000","last_became_peered":"2023-10-21T17:38:33.354183+0000","last_unstale":"2023-10-21T17:38:46.556204+0000","last_undegraded":"2023-10-21T17:38:46.556204+0000","last_fullsized":"2023-10-21T17:38:46.556204+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:25.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552626+0000","last_change":"2023-10-21T17:38:47.552387+0000","last_active":"2023-10-21T17:38:48.552626+0000","last_peered":"2023-10-21T17:38:48.552626+0000","last_clean":"2023-10-21T17:38:48.552626+0000","last_became_active":"2023-10-21T17:38:47.552122+0000","last_became_peered":"2023-10-21T17:38:47.552122+0000","last_unstale":"2023-10-21T17:38:48.552626+0000","last_undegraded":"2023-10-21T17:38:48.552626+0000","last_fullsized":"2023-10-21T17:38:48.552626+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:27.300690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552309+0000","last_change":"2023-10-21T17:38:33.353725+0000","last_active":"2023-10-21T17:38:46.552309+0000","last_peered":"2023-10-21T17:38:46.552309+0000","last_clean":"2023-10-21T17:38:46.552309+0000","last_became_active":"2023-10-21T17:38:33.353441+0000","last_became_peered":"2023-10-21T17:38:33.353441+0000","last_unstale":"2023-10-21T17:38:46.552309+0000","last_undegraded":"2023-10-21T17:38:46.552309+0000","last_fullsized":"2023-10-21T17:38:46.552309+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:25.189245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566330+0000","last_change":"2023-10-21T17:38:47.561904+0000","last_active":"2023-10-21T17:38:47.566330+0000","last_peered":"2023-10-21T17:38:47.566330+0000","last_clean":"2023-10-21T17:38:47.566330+0000","last_became_active":"2023-10-21T17:38:47.560202+0000","last_became_peered":"2023-10-21T17:38:47.560202+0000","last_unstale":"2023-10-21T17:38:47.566330+0000","last_undegraded":"2023-10-21T17:38:47.566330+0000","last_fullsized":"2023-10-21T17:38:47.566330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:49:34.076182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.551453+0000","last_change":"2023-10-21T17:38:33.340905+0000","last_active":"2023-10-21T17:38:46.551453+0000","last_peered":"2023-10-21T17:38:46.551453+0000","last_clean":"2023-10-21T17:38:46.551453+0000","last_became_active":"2023-10-21T17:38:33.340134+0000","last_became_peered":"2023-10-21T17:38:33.340134+0000","last_unstale":"2023-10-21T17:38:46.551453+0000","last_undegraded":"2023-10-21T17:38:46.551453+0000","last_fullsized":"2023-10-21T17:38:46.551453+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:58.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555361+0000","last_change":"2023-10-21T17:38:47.560945+0000","last_active":"2023-10-21T17:38:48.555361+0000","last_peered":"2023-10-21T17:38:48.555361+0000","last_clean":"2023-10-21T17:38:48.555361+0000","last_became_active":"2023-10-21T17:38:47.560222+0000","last_became_peered":"2023-10-21T17:38:47.560222+0000","last_unstale":"2023-10-21T17:38:48.555361+0000","last_undegraded":"2023-10-21T17:38:48.555361+0000","last_fullsized":"2023-10-21T17:38:48.555361+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:13:20.771952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.548438+0000","last_change":"2023-10-21T17:38:33.354704+0000","last_active":"2023-10-21T17:38:47.548438+0000","last_peered":"2023-10-21T17:38:47.548438+0000","last_clean":"2023-10-21T17:38:47.548438+0000","last_became_active":"2023-10-21T17:38:33.354428+0000","last_became_peered":"2023-10-21T17:38:33.354428+0000","last_unstale":"2023-10-21T17:38:47.548438+0000","last_undegraded":"2023-10-21T17:38:47.548438+0000","last_fullsized":"2023-10-21T17:38:47.548438+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:42.944572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565430+0000","last_change":"2023-10-21T17:38:47.560143+0000","last_active":"2023-10-21T17:38:47.565430+0000","last_peered":"2023-10-21T17:38:47.565430+0000","last_clean":"2023-10-21T17:38:47.565430+0000","last_became_active":"2023-10-21T17:38:47.559565+0000","last_became_peered":"2023-10-21T17:38:47.559565+0000","last_unstale":"2023-10-21T17:38:47.565430+0000","last_undegraded":"2023-10-21T17:38:47.565430+0000","last_fullsized":"2023-10-21T17:38:47.565430+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:46:56.927972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529062+0000","last_change":"2023-10-21T17:38:33.349528+0000","last_active":"2023-10-21T17:38:47.529062+0000","last_peered":"2023-10-21T17:38:47.529062+0000","last_clean":"2023-10-21T17:38:47.529062+0000","last_became_active":"2023-10-21T17:38:33.349070+0000","last_became_peered":"2023-10-21T17:38:33.349070+0000","last_unstale":"2023-10-21T17:38:47.529062+0000","last_undegraded":"2023-10-21T17:38:47.529062+0000","last_fullsized":"2023-10-21T17:38:47.529062+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:10.470691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556409+0000","last_change":"2023-10-21T17:38:47.553702+0000","last_active":"2023-10-21T17:38:47.556409+0000","last_peered":"2023-10-21T17:38:47.556409+0000","last_clean":"2023-10-21T17:38:47.556409+0000","last_became_active":"2023-10-21T17:38:47.553343+0000","last_became_peered":"2023-10-21T17:38:47.553343+0000","last_unstale":"2023-10-21T17:38:47.556409+0000","last_undegraded":"2023-10-21T17:38:47.556409+0000","last_fullsized":"2023-10-21T17:38:47.556409+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:35:50.772804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535231+0000","last_change":"2023-10-21T17:38:33.369586+0000","last_active":"2023-10-21T17:38:46.535231+0000","last_peered":"2023-10-21T17:38:46.535231+0000","last_clean":"2023-10-21T17:38:46.535231+0000","last_became_active":"2023-10-21T17:38:33.369208+0000","last_became_peered":"2023-10-21T17:38:33.369208+0000","last_unstale":"2023-10-21T17:38:46.535231+0000","last_undegraded":"2023-10-21T17:38:46.535231+0000","last_fullsized":"2023-10-21T17:38:46.535231+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:04.246844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557913+0000","last_change":"2023-10-21T17:38:47.563623+0000","last_active":"2023-10-21T17:38:48.557913+0000","last_peered":"2023-10-21T17:38:48.557913+0000","last_clean":"2023-10-21T17:38:48.557913+0000","last_became_active":"2023-10-21T17:38:47.562191+0000","last_became_peered":"2023-10-21T17:38:47.562191+0000","last_unstale":"2023-10-21T17:38:48.557913+0000","last_undegraded":"2023-10-21T17:38:48.557913+0000","last_fullsized":"2023-10-21T17:38:48.557913+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:54:16.894163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534573+0000","last_change":"2023-10-21T17:38:33.362478+0000","last_active":"2023-10-21T17:38:46.534573+0000","last_peered":"2023-10-21T17:38:46.534573+0000","last_clean":"2023-10-21T17:38:46.534573+0000","last_became_active":"2023-10-21T17:38:33.361294+0000","last_became_peered":"2023-10-21T17:38:33.361294+0000","last_unstale":"2023-10-21T17:38:46.534573+0000","last_undegraded":"2023-10-21T17:38:46.534573+0000","last_fullsized":"2023-10-21T17:38:46.534573+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:25.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561640+0000","last_change":"2023-10-21T17:38:47.560964+0000","last_active":"2023-10-21T17:38:47.561640+0000","last_peered":"2023-10-21T17:38:47.561640+0000","last_clean":"2023-10-21T17:38:47.561640+0000","last_became_active":"2023-10-21T17:38:47.560634+0000","last_became_peered":"2023-10-21T17:38:47.560634+0000","last_unstale":"2023-10-21T17:38:47.561640+0000","last_undegraded":"2023-10-21T17:38:47.561640+0000","last_fullsized":"2023-10-21T17:38:47.561640+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:02.913327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541559+0000","last_change":"2023-10-21T17:38:33.352394+0000","last_active":"2023-10-21T17:38:47.541559+0000","last_peered":"2023-10-21T17:38:47.541559+0000","last_clean":"2023-10-21T17:38:47.541559+0000","last_became_active":"2023-10-21T17:38:33.352048+0000","last_became_peered":"2023-10-21T17:38:33.352048+0000","last_unstale":"2023-10-21T17:38:47.541559+0000","last_undegraded":"2023-10-21T17:38:47.541559+0000","last_fullsized":"2023-10-21T17:38:47.541559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:20.548414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557864+0000","last_change":"2023-10-21T17:38:47.548715+0000","last_active":"2023-10-21T17:38:47.557864+0000","last_peered":"2023-10-21T17:38:47.557864+0000","last_clean":"2023-10-21T17:38:47.557864+0000","last_became_active":"2023-10-21T17:38:47.548430+0000","last_became_peered":"2023-10-21T17:38:47.548430+0000","last_unstale":"2023-10-21T17:38:47.557864+0000","last_undegraded":"2023-10-21T17:38:47.557864+0000","last_fullsized":"2023-10-21T17:38:47.557864+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:25.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530506+0000","last_change":"2023-10-21T17:38:33.346563+0000","last_active":"2023-10-21T17:38:47.530506+0000","last_peered":"2023-10-21T17:38:47.530506+0000","last_clean":"2023-10-21T17:38:47.530506+0000","last_became_active":"2023-10-21T17:38:33.343191+0000","last_became_peered":"2023-10-21T17:38:33.343191+0000","last_unstale":"2023-10-21T17:38:47.530506+0000","last_undegraded":"2023-10-21T17:38:47.530506+0000","last_fullsized":"2023-10-21T17:38:47.530506+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:36.794962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570949+0000","last_change":"2023-10-21T17:38:47.569111+0000","last_active":"2023-10-21T17:38:47.570949+0000","last_peered":"2023-10-21T17:38:47.570949+0000","last_clean":"2023-10-21T17:38:47.570949+0000","last_became_active":"2023-10-21T17:38:47.568807+0000","last_became_peered":"2023-10-21T17:38:47.568807+0000","last_unstale":"2023-10-21T17:38:47.570949+0000","last_undegraded":"2023-10-21T17:38:47.570949+0000","last_fullsized":"2023-10-21T17:38:47.570949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:40:16.895474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534426+0000","last_change":"2023-10-21T17:38:33.355761+0000","last_active":"2023-10-21T17:38:46.534426+0000","last_peered":"2023-10-21T17:38:46.534426+0000","last_clean":"2023-10-21T17:38:46.534426+0000","last_became_active":"2023-10-21T17:38:33.355480+0000","last_became_peered":"2023-10-21T17:38:33.355480+0000","last_unstale":"2023-10-21T17:38:46.534426+0000","last_undegraded":"2023-10-21T17:38:46.534426+0000","last_fullsized":"2023-10-21T17:38:46.534426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:12.290129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565242+0000","last_change":"2023-10-21T17:38:47.560623+0000","last_active":"2023-10-21T17:38:47.565242+0000","last_peered":"2023-10-21T17:38:47.565242+0000","last_clean":"2023-10-21T17:38:47.565242+0000","last_became_active":"2023-10-21T17:38:47.558869+0000","last_became_peered":"2023-10-21T17:38:47.558869+0000","last_unstale":"2023-10-21T17:38:47.565242+0000","last_undegraded":"2023-10-21T17:38:47.565242+0000","last_fullsized":"2023-10-21T17:38:47.565242+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:30:48.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542268+0000","last_change":"2023-10-21T17:38:33.366191+0000","last_active":"2023-10-21T17:38:47.542268+0000","last_peered":"2023-10-21T17:38:47.542268+0000","last_clean":"2023-10-21T17:38:47.542268+0000","last_became_active":"2023-10-21T17:38:33.363694+0000","last_became_peered":"2023-10-21T17:38:33.363694+0000","last_unstale":"2023-10-21T17:38:47.542268+0000","last_undegraded":"2023-10-21T17:38:47.542268+0000","last_fullsized":"2023-10-21T17:38:47.542268+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:12.488140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555365+0000","last_change":"2023-10-21T17:38:47.555523+0000","last_active":"2023-10-21T17:38:48.555365+0000","last_peered":"2023-10-21T17:38:48.555365+0000","last_clean":"2023-10-21T17:38:48.555365+0000","last_became_active":"2023-10-21T17:38:47.555224+0000","last_became_peered":"2023-10-21T17:38:47.555224+0000","last_unstale":"2023-10-21T17:38:48.555365+0000","last_undegraded":"2023-10-21T17:38:48.555365+0000","last_fullsized":"2023-10-21T17:38:48.555365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:09.807236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545471+0000","last_change":"2023-10-21T17:38:33.367227+0000","last_active":"2023-10-21T17:38:47.545471+0000","last_peered":"2023-10-21T17:38:47.545471+0000","last_clean":"2023-10-21T17:38:47.545471+0000","last_became_active":"2023-10-21T17:38:33.366922+0000","last_became_peered":"2023-10-21T17:38:33.366922+0000","last_unstale":"2023-10-21T17:38:47.545471+0000","last_undegraded":"2023-10-21T17:38:47.545471+0000","last_fullsized":"2023-10-21T17:38:47.545471+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:33:14.663384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556512+0000","last_change":"2023-10-21T17:38:47.552457+0000","last_active":"2023-10-21T17:38:47.556512+0000","last_peered":"2023-10-21T17:38:47.556512+0000","last_clean":"2023-10-21T17:38:47.556512+0000","last_became_active":"2023-10-21T17:38:47.551709+0000","last_became_peered":"2023-10-21T17:38:47.551709+0000","last_unstale":"2023-10-21T17:38:47.556512+0000","last_undegraded":"2023-10-21T17:38:47.556512+0000","last_fullsized":"2023-10-21T17:38:47.556512+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:28:38.788080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553288+0000","last_change":"2023-10-21T17:38:33.363852+0000","last_active":"2023-10-21T17:38:47.553288+0000","last_peered":"2023-10-21T17:38:47.553288+0000","last_clean":"2023-10-21T17:38:47.553288+0000","last_became_active":"2023-10-21T17:38:33.363427+0000","last_became_peered":"2023-10-21T17:38:33.363427+0000","last_unstale":"2023-10-21T17:38:47.553288+0000","last_undegraded":"2023-10-21T17:38:47.553288+0000","last_fullsized":"2023-10-21T17:38:47.553288+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:39.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558963+0000","last_change":"2023-10-21T17:38:47.558369+0000","last_active":"2023-10-21T17:38:48.558963+0000","last_peered":"2023-10-21T17:38:48.558963+0000","last_clean":"2023-10-21T17:38:48.558963+0000","last_became_active":"2023-10-21T17:38:47.558058+0000","last_became_peered":"2023-10-21T17:38:47.558058+0000","last_unstale":"2023-10-21T17:38:48.558963+0000","last_undegraded":"2023-10-21T17:38:48.558963+0000","last_fullsized":"2023-10-21T17:38:48.558963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:06.514890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.544656+0000","last_change":"2023-10-21T17:38:33.366914+0000","last_active":"2023-10-21T17:38:47.544656+0000","last_peered":"2023-10-21T17:38:47.544656+0000","last_clean":"2023-10-21T17:38:47.544656+0000","last_became_active":"2023-10-21T17:38:33.366337+0000","last_became_peered":"2023-10-21T17:38:33.366337+0000","last_unstale":"2023-10-21T17:38:47.544656+0000","last_undegraded":"2023-10-21T17:38:47.544656+0000","last_fullsized":"2023-10-21T17:38:47.544656+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:15:25.780715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566282+0000","last_change":"2023-10-21T17:38:47.560566+0000","last_active":"2023-10-21T17:38:47.566282+0000","last_peered":"2023-10-21T17:38:47.566282+0000","last_clean":"2023-10-21T17:38:47.566282+0000","last_became_active":"2023-10-21T17:38:47.559800+0000","last_became_peered":"2023-10-21T17:38:47.559800+0000","last_unstale":"2023-10-21T17:38:47.566282+0000","last_undegraded":"2023-10-21T17:38:47.566282+0000","last_fullsized":"2023-10-21T17:38:47.566282+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:34.538156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550826+0000","last_change":"2023-10-21T17:38:33.351005+0000","last_active":"2023-10-21T17:38:47.550826+0000","last_peered":"2023-10-21T17:38:47.550826+0000","last_clean":"2023-10-21T17:38:47.550826+0000","last_became_active":"2023-10-21T17:38:33.350305+0000","last_became_peered":"2023-10-21T17:38:33.350305+0000","last_unstale":"2023-10-21T17:38:47.550826+0000","last_undegraded":"2023-10-21T17:38:47.550826+0000","last_fullsized":"2023-10-21T17:38:47.550826+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:28.150954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565314+0000","last_change":"2023-10-21T17:38:47.556976+0000","last_active":"2023-10-21T17:38:47.565314+0000","last_peered":"2023-10-21T17:38:47.565314+0000","last_clean":"2023-10-21T17:38:47.565314+0000","last_became_active":"2023-10-21T17:38:47.556701+0000","last_became_peered":"2023-10-21T17:38:47.556701+0000","last_unstale":"2023-10-21T17:38:47.565314+0000","last_undegraded":"2023-10-21T17:38:47.565314+0000","last_fullsized":"2023-10-21T17:38:47.565314+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:18.225085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541455+0000","last_change":"2023-10-21T17:38:33.360922+0000","last_active":"2023-10-21T17:38:47.541455+0000","last_peered":"2023-10-21T17:38:47.541455+0000","last_clean":"2023-10-21T17:38:47.541455+0000","last_became_active":"2023-10-21T17:38:33.360628+0000","last_became_peered":"2023-10-21T17:38:33.360628+0000","last_unstale":"2023-10-21T17:38:47.541455+0000","last_undegraded":"2023-10-21T17:38:47.541455+0000","last_fullsized":"2023-10-21T17:38:47.541455+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:01.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557107+0000","last_change":"2023-10-21T17:38:47.554990+0000","last_active":"2023-10-21T17:38:48.557107+0000","last_peered":"2023-10-21T17:38:48.557107+0000","last_clean":"2023-10-21T17:38:48.557107+0000","last_became_active":"2023-10-21T17:38:47.554716+0000","last_became_peered":"2023-10-21T17:38:47.554716+0000","last_unstale":"2023-10-21T17:38:48.557107+0000","last_undegraded":"2023-10-21T17:38:48.557107+0000","last_fullsized":"2023-10-21T17:38:48.557107+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:05.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531752+0000","last_change":"2023-10-21T17:38:33.344462+0000","last_active":"2023-10-21T17:38:46.531752+0000","last_peered":"2023-10-21T17:38:46.531752+0000","last_clean":"2023-10-21T17:38:46.531752+0000","last_became_active":"2023-10-21T17:38:33.344064+0000","last_became_peered":"2023-10-21T17:38:33.344064+0000","last_unstale":"2023-10-21T17:38:46.531752+0000","last_undegraded":"2023-10-21T17:38:46.531752+0000","last_fullsized":"2023-10-21T17:38:46.531752+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:37.232213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550228+0000","last_change":"2023-10-21T17:38:47.544335+0000","last_active":"2023-10-21T17:38:47.550228+0000","last_peered":"2023-10-21T17:38:47.550228+0000","last_clean":"2023-10-21T17:38:47.550228+0000","last_became_active":"2023-10-21T17:38:47.543954+0000","last_became_peered":"2023-10-21T17:38:47.543954+0000","last_unstale":"2023-10-21T17:38:47.550228+0000","last_undegraded":"2023-10-21T17:38:47.550228+0000","last_fullsized":"2023-10-21T17:38:47.550228+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:23.084608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.547688+0000","last_change":"2023-10-21T17:38:33.360718+0000","last_active":"2023-10-21T17:38:46.547688+0000","last_peered":"2023-10-21T17:38:46.547688+0000","last_clean":"2023-10-21T17:38:46.547688+0000","last_became_active":"2023-10-21T17:38:33.359530+0000","last_became_peered":"2023-10-21T17:38:33.359530+0000","last_unstale":"2023-10-21T17:38:46.547688+0000","last_undegraded":"2023-10-21T17:38:46.547688+0000","last_fullsized":"2023-10-21T17:38:46.547688+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:04.002929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.549540+0000","last_change":"2023-10-21T17:38:47.536233+0000","last_active":"2023-10-21T17:38:47.549540+0000","last_peered":"2023-10-21T17:38:47.549540+0000","last_clean":"2023-10-21T17:38:47.549540+0000","last_became_active":"2023-10-21T17:38:47.534706+0000","last_became_peered":"2023-10-21T17:38:47.534706+0000","last_unstale":"2023-10-21T17:38:47.549540+0000","last_undegraded":"2023-10-21T17:38:47.549540+0000","last_fullsized":"2023-10-21T17:38:47.549540+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:24.782875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542162+0000","last_change":"2023-10-21T17:38:33.366456+0000","last_active":"2023-10-21T17:38:47.542162+0000","last_peered":"2023-10-21T17:38:47.542162+0000","last_clean":"2023-10-21T17:38:47.542162+0000","last_became_active":"2023-10-21T17:38:33.366015+0000","last_became_peered":"2023-10-21T17:38:33.366015+0000","last_unstale":"2023-10-21T17:38:47.542162+0000","last_undegraded":"2023-10-21T17:38:47.542162+0000","last_fullsized":"2023-10-21T17:38:47.542162+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:28.700422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559847+0000","last_change":"2023-10-21T17:38:47.555243+0000","last_active":"2023-10-21T17:38:47.559847+0000","last_peered":"2023-10-21T17:38:47.559847+0000","last_clean":"2023-10-21T17:38:47.559847+0000","last_became_active":"2023-10-21T17:38:47.554976+0000","last_became_peered":"2023-10-21T17:38:47.554976+0000","last_unstale":"2023-10-21T17:38:47.559847+0000","last_undegraded":"2023-10-21T17:38:47.559847+0000","last_fullsized":"2023-10-21T17:38:47.559847+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:44.479903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.557628+0000","last_change":"2023-10-21T17:38:33.358688+0000","last_active":"2023-10-21T17:38:46.557628+0000","last_peered":"2023-10-21T17:38:46.557628+0000","last_clean":"2023-10-21T17:38:46.557628+0000","last_became_active":"2023-10-21T17:38:33.354936+0000","last_became_peered":"2023-10-21T17:38:33.354936+0000","last_unstale":"2023-10-21T17:38:46.557628+0000","last_undegraded":"2023-10-21T17:38:46.557628+0000","last_fullsized":"2023-10-21T17:38:46.557628+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:24.335865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556916+0000","last_change":"2023-10-21T17:38:47.552664+0000","last_active":"2023-10-21T17:38:47.556916+0000","last_peered":"2023-10-21T17:38:47.556916+0000","last_clean":"2023-10-21T17:38:47.556916+0000","last_became_active":"2023-10-21T17:38:47.552137+0000","last_became_peered":"2023-10-21T17:38:47.552137+0000","last_unstale":"2023-10-21T17:38:47.556916+0000","last_undegraded":"2023-10-21T17:38:47.556916+0000","last_fullsized":"2023-10-21T17:38:47.556916+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:44:19.967378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.528768+0000","last_change":"2023-10-21T17:38:33.349361+0000","last_active":"2023-10-21T17:38:47.528768+0000","last_peered":"2023-10-21T17:38:47.528768+0000","last_clean":"2023-10-21T17:38:47.528768+0000","last_became_active":"2023-10-21T17:38:33.348672+0000","last_became_peered":"2023-10-21T17:38:33.348672+0000","last_unstale":"2023-10-21T17:38:47.528768+0000","last_undegraded":"2023-10-21T17:38:47.528768+0000","last_fullsized":"2023-10-21T17:38:47.528768+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:37:12.429036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555782+0000","last_change":"2023-10-21T17:38:47.561458+0000","last_active":"2023-10-21T17:38:48.555782+0000","last_peered":"2023-10-21T17:38:48.555782+0000","last_clean":"2023-10-21T17:38:48.555782+0000","last_became_active":"2023-10-21T17:38:47.561144+0000","last_became_peered":"2023-10-21T17:38:47.561144+0000","last_unstale":"2023-10-21T17:38:48.555782+0000","last_undegraded":"2023-10-21T17:38:48.555782+0000","last_fullsized":"2023-10-21T17:38:48.555782+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:25:19.338478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535141+0000","last_change":"2023-10-21T17:38:33.369555+0000","last_active":"2023-10-21T17:38:46.535141+0000","last_peered":"2023-10-21T17:38:46.535141+0000","last_clean":"2023-10-21T17:38:46.535141+0000","last_became_active":"2023-10-21T17:38:33.369212+0000","last_became_peered":"2023-10-21T17:38:33.369212+0000","last_unstale":"2023-10-21T17:38:46.535141+0000","last_undegraded":"2023-10-21T17:38:46.535141+0000","last_fullsized":"2023-10-21T17:38:46.535141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:29.955478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557566+0000","last_change":"2023-10-21T17:38:47.556272+0000","last_active":"2023-10-21T17:38:48.557566+0000","last_peered":"2023-10-21T17:38:48.557566+0000","last_clean":"2023-10-21T17:38:48.557566+0000","last_became_active":"2023-10-21T17:38:47.555903+0000","last_became_peered":"2023-10-21T17:38:47.555903+0000","last_unstale":"2023-10-21T17:38:48.557566+0000","last_undegraded":"2023-10-21T17:38:48.557566+0000","last_fullsized":"2023-10-21T17:38:48.557566+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:28.828809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552542+0000","last_change":"2023-10-21T17:38:33.368484+0000","last_active":"2023-10-21T17:38:47.552542+0000","last_peered":"2023-10-21T17:38:47.552542+0000","last_clean":"2023-10-21T17:38:47.552542+0000","last_became_active":"2023-10-21T17:38:33.366828+0000","last_became_peered":"2023-10-21T17:38:33.366828+0000","last_unstale":"2023-10-21T17:38:47.552542+0000","last_undegraded":"2023-10-21T17:38:47.552542+0000","last_fullsized":"2023-10-21T17:38:47.552542+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:04.935450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556631+0000","last_change":"2023-10-21T17:38:47.552808+0000","last_active":"2023-10-21T17:38:47.556631+0000","last_peered":"2023-10-21T17:38:47.556631+0000","last_clean":"2023-10-21T17:38:47.556631+0000","last_became_active":"2023-10-21T17:38:47.552549+0000","last_became_peered":"2023-10-21T17:38:47.552549+0000","last_unstale":"2023-10-21T17:38:47.556631+0000","last_undegraded":"2023-10-21T17:38:47.556631+0000","last_fullsized":"2023-10-21T17:38:47.556631+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:25:19.338478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541364+0000","last_change":"2023-10-21T17:38:33.363542+0000","last_active":"2023-10-21T17:38:47.541364+0000","last_peered":"2023-10-21T17:38:47.541364+0000","last_clean":"2023-10-21T17:38:47.541364+0000","last_became_active":"2023-10-21T17:38:33.363143+0000","last_became_peered":"2023-10-21T17:38:33.363143+0000","last_unstale":"2023-10-21T17:38:47.541364+0000","last_undegraded":"2023-10-21T17:38:47.541364+0000","last_fullsized":"2023-10-21T17:38:47.541364+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550344+0000","last_change":"2023-10-21T17:38:47.547082+0000","last_active":"2023-10-21T17:38:47.550344+0000","last_peered":"2023-10-21T17:38:47.550344+0000","last_clean":"2023-10-21T17:38:47.550344+0000","last_became_active":"2023-10-21T17:38:47.546777+0000","last_became_peered":"2023-10-21T17:38:47.546777+0000","last_unstale":"2023-10-21T17:38:47.550344+0000","last_undegraded":"2023-10-21T17:38:47.550344+0000","last_fullsized":"2023-10-21T17:38:47.550344+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:33:07.701278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541229+0000","last_change":"2023-10-21T17:38:33.363315+0000","last_active":"2023-10-21T17:38:47.541229+0000","last_peered":"2023-10-21T17:38:47.541229+0000","last_clean":"2023-10-21T17:38:47.541229+0000","last_became_active":"2023-10-21T17:38:33.363007+0000","last_became_peered":"2023-10-21T17:38:33.363007+0000","last_unstale":"2023-10-21T17:38:47.541229+0000","last_undegraded":"2023-10-21T17:38:47.541229+0000","last_fullsized":"2023-10-21T17:38:47.541229+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:45.232763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.564612+0000","last_change":"2023-10-21T17:38:47.557619+0000","last_active":"2023-10-21T17:38:47.564612+0000","last_peered":"2023-10-21T17:38:47.564612+0000","last_clean":"2023-10-21T17:38:47.564612+0000","last_became_active":"2023-10-21T17:38:47.556520+0000","last_became_peered":"2023-10-21T17:38:47.556520+0000","last_unstale":"2023-10-21T17:38:47.564612+0000","last_undegraded":"2023-10-21T17:38:47.564612+0000","last_fullsized":"2023-10-21T17:38:47.564612+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:35.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.528593+0000","last_change":"2023-10-21T17:38:33.352800+0000","last_active":"2023-10-21T17:38:47.528593+0000","last_peered":"2023-10-21T17:38:47.528593+0000","last_clean":"2023-10-21T17:38:47.528593+0000","last_became_active":"2023-10-21T17:38:33.351990+0000","last_became_peered":"2023-10-21T17:38:33.351990+0000","last_unstale":"2023-10-21T17:38:47.528593+0000","last_undegraded":"2023-10-21T17:38:47.528593+0000","last_fullsized":"2023-10-21T17:38:47.528593+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:05.602488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.549558+0000","last_change":"2023-10-21T17:38:47.537064+0000","last_active":"2023-10-21T17:38:47.549558+0000","last_peered":"2023-10-21T17:38:47.549558+0000","last_clean":"2023-10-21T17:38:47.549558+0000","last_became_active":"2023-10-21T17:38:47.536311+0000","last_became_peered":"2023-10-21T17:38:47.536311+0000","last_unstale":"2023-10-21T17:38:47.549558+0000","last_undegraded":"2023-10-21T17:38:47.549558+0000","last_fullsized":"2023-10-21T17:38:47.549558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:11.920952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.554203+0000","last_change":"2023-10-21T17:38:33.351557+0000","last_active":"2023-10-21T17:38:46.554203+0000","last_peered":"2023-10-21T17:38:46.554203+0000","last_clean":"2023-10-21T17:38:46.554203+0000","last_became_active":"2023-10-21T17:38:33.351230+0000","last_became_peered":"2023-10-21T17:38:33.351230+0000","last_unstale":"2023-10-21T17:38:46.554203+0000","last_undegraded":"2023-10-21T17:38:46.554203+0000","last_fullsized":"2023-10-21T17:38:46.554203+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:20:01.763000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557728+0000","last_change":"2023-10-21T17:38:47.551152+0000","last_active":"2023-10-21T17:38:47.557728+0000","last_peered":"2023-10-21T17:38:47.557728+0000","last_clean":"2023-10-21T17:38:47.557728+0000","last_became_active":"2023-10-21T17:38:47.550821+0000","last_became_peered":"2023-10-21T17:38:47.550821+0000","last_unstale":"2023-10-21T17:38:47.557728+0000","last_undegraded":"2023-10-21T17:38:47.557728+0000","last_fullsized":"2023-10-21T17:38:47.557728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:49.083823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529578+0000","last_change":"2023-10-21T17:38:33.354002+0000","last_active":"2023-10-21T17:38:47.529578+0000","last_peered":"2023-10-21T17:38:47.529578+0000","last_clean":"2023-10-21T17:38:47.529578+0000","last_became_active":"2023-10-21T17:38:33.353709+0000","last_became_peered":"2023-10-21T17:38:33.353709+0000","last_unstale":"2023-10-21T17:38:47.529578+0000","last_undegraded":"2023-10-21T17:38:47.529578+0000","last_fullsized":"2023-10-21T17:38:47.529578+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:13.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.553150+0000","last_change":"2023-10-21T17:38:47.568080+0000","last_active":"2023-10-21T17:38:48.553150+0000","last_peered":"2023-10-21T17:38:48.553150+0000","last_clean":"2023-10-21T17:38:48.553150+0000","last_became_active":"2023-10-21T17:38:47.567685+0000","last_became_peered":"2023-10-21T17:38:47.567685+0000","last_unstale":"2023-10-21T17:38:48.553150+0000","last_undegraded":"2023-10-21T17:38:48.553150+0000","last_fullsized":"2023-10-21T17:38:48.553150+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:36.723492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.558999+0000","last_change":"2023-10-21T17:38:33.359316+0000","last_active":"2023-10-21T17:38:46.558999+0000","last_peered":"2023-10-21T17:38:46.558999+0000","last_clean":"2023-10-21T17:38:46.558999+0000","last_became_active":"2023-10-21T17:38:33.356311+0000","last_became_peered":"2023-10-21T17:38:33.356311+0000","last_unstale":"2023-10-21T17:38:46.558999+0000","last_undegraded":"2023-10-21T17:38:46.558999+0000","last_fullsized":"2023-10-21T17:38:46.558999+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:10.002729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556673+0000","last_change":"2023-10-21T17:38:47.554019+0000","last_active":"2023-10-21T17:38:47.556673+0000","last_peered":"2023-10-21T17:38:47.556673+0000","last_clean":"2023-10-21T17:38:47.556673+0000","last_became_active":"2023-10-21T17:38:47.553725+0000","last_became_peered":"2023-10-21T17:38:47.553725+0000","last_unstale":"2023-10-21T17:38:47.556673+0000","last_undegraded":"2023-10-21T17:38:47.556673+0000","last_fullsized":"2023-10-21T17:38:47.556673+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:39:36.444107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535049+0000","last_change":"2023-10-21T17:38:33.354797+0000","last_active":"2023-10-21T17:38:46.535049+0000","last_peered":"2023-10-21T17:38:46.535049+0000","last_clean":"2023-10-21T17:38:46.535049+0000","last_became_active":"2023-10-21T17:38:33.354503+0000","last_became_peered":"2023-10-21T17:38:33.354503+0000","last_unstale":"2023-10-21T17:38:46.535049+0000","last_undegraded":"2023-10-21T17:38:46.535049+0000","last_fullsized":"2023-10-21T17:38:46.535049+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:21.925672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.539797+0000","last_change":"2023-10-21T17:38:47.537503+0000","last_active":"2023-10-21T17:38:47.539797+0000","last_peered":"2023-10-21T17:38:47.539797+0000","last_clean":"2023-10-21T17:38:47.539797+0000","last_became_active":"2023-10-21T17:38:47.536825+0000","last_became_peered":"2023-10-21T17:38:47.536825+0000","last_unstale":"2023-10-21T17:38:47.539797+0000","last_undegraded":"2023-10-21T17:38:47.539797+0000","last_fullsized":"2023-10-21T17:38:47.539797+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:48:25.196265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535171+0000","last_change":"2023-10-21T17:38:33.348939+0000","last_active":"2023-10-21T17:38:46.535171+0000","last_peered":"2023-10-21T17:38:46.535171+0000","last_clean":"2023-10-21T17:38:46.535171+0000","last_became_active":"2023-10-21T17:38:33.348561+0000","last_became_peered":"2023-10-21T17:38:33.348561+0000","last_unstale":"2023-10-21T17:38:46.535171+0000","last_undegraded":"2023-10-21T17:38:46.535171+0000","last_fullsized":"2023-10-21T17:38:46.535171+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:14.527583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561983+0000","last_change":"2023-10-21T17:38:47.558765+0000","last_active":"2023-10-21T17:38:47.561983+0000","last_peered":"2023-10-21T17:38:47.561983+0000","last_clean":"2023-10-21T17:38:47.561983+0000","last_became_active":"2023-10-21T17:38:47.558412+0000","last_became_peered":"2023-10-21T17:38:47.558412+0000","last_unstale":"2023-10-21T17:38:47.561983+0000","last_undegraded":"2023-10-21T17:38:47.561983+0000","last_fullsized":"2023-10-21T17:38:47.561983+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:43.878002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534041+0000","last_change":"2023-10-21T17:38:33.368541+0000","last_active":"2023-10-21T17:38:46.534041+0000","last_peered":"2023-10-21T17:38:46.534041+0000","last_clean":"2023-10-21T17:38:46.534041+0000","last_became_active":"2023-10-21T17:38:33.368186+0000","last_became_peered":"2023-10-21T17:38:33.368186+0000","last_unstale":"2023-10-21T17:38:46.534041+0000","last_undegraded":"2023-10-21T17:38:46.534041+0000","last_fullsized":"2023-10-21T17:38:46.534041+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:10.002729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4873,"ondisk_log_size":4873,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74936,"kb_used_data":3496,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296904,"statfs":{"total":96636764160,"available":96560029696,"internally_reserved":0,"allocated":3579904,"data_stored":2131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.97399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"max":{"1min":1.6419999999999999,"5min":1.841,"15min":2.2549999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.101,"15min":1.1599999999999999},"min":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.724,"15min":1.7669999999999999},"last":2.6160000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1739999999999999,"15min":1.218},"min":{"1min":0.96299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.589,"5min":1.7170000000000001,"15min":1.73},"last":1.615},{"interface":"front","average":{"1min":1.262,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.91200000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.506,"5min":1.792,"15min":1.885},"last":1.411}]},{"osd":2,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":5.3360000000000003},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.86899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":3.556}]},{"osd":3,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2909999999999999,"15min":1.3180000000000001},"min":{"1min":0.998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.472,"5min":2.472,"15min":2.5110000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.486,"15min":1.4670000000000001},"min":{"1min":1.23,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":4.9249999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.383,"15min":1.357},"min":{"1min":0.84599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.536,"5min":1.5129999999999999,"15min":1.496},"min":{"1min":1.1619999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.5649999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.173,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.474,"15min":1.45},"min":{"1min":1.135,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.587,"15min":1.5469999999999999},"min":{"1min":0.85499999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.3100000000000001,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.27},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.488,"15min":1.476},"min":{"1min":0.88800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.919}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42440,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329400,"statfs":{"total":96636764160,"available":96593305600,"internally_reserved":0,"allocated":3002368,"data_stored":1542863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1659999999999999,"15min":1.198},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.728,"5min":1.944,"15min":1.944},"last":1.671},{"interface":"front","average":{"1min":1.256,"5min":1.2050000000000001,"15min":1.222},"min":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.643,"5min":1.7090000000000001,"15min":1.762},"last":1.05}]},{"osd":1,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.171,"15min":1.1919999999999999},"min":{"1min":0.84099999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6379999999999999},"last":0.879},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2989999999999999,"15min":1.319},"min":{"1min":0.98599999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.782,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.1499999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.272,"15min":1.3169999999999999},"min":{"1min":1.006,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5649999999999999,"5min":1.712,"15min":1.8300000000000001},"last":0.999},{"interface":"front","average":{"1min":1.393,"5min":1.3089999999999999,"15min":1.3069999999999999},"min":{"1min":1.1799999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.611,"5min":1.679,"15min":1.929},"last":1.1919999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3919999999999999,"15min":1.417},"min":{"1min":0.90700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.593,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.339},{"interface":"front","average":{"1min":1.468,"5min":1.427,"15min":1.4650000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.90500000000000003},"max":{"1min":1.825,"5min":1.9039999999999999,"15min":2.048},"last":1.2370000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":1.704,"5min":1.77,"15min":1.9950000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.4079999999999999,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.806}]},{"osd":5,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5349999999999999,"15min":1.571},"min":{"1min":1.274,"5min":1.1000000000000001,"15min":1.0029999999999999},"max":{"1min":1.891,"5min":1.956,"15min":2.339},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.325,"15min":1.3759999999999999},"min":{"1min":0.98199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6870000000000001,"5min":1.796,"15min":2.2890000000000001},"last":1.7470000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4970000000000001,"15min":1.536},"min":{"1min":1.1080000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.762,"5min":1.9339999999999999,"15min":2.1200000000000001},"last":1.508},{"interface":"front","average":{"1min":1.641,"5min":1.6259999999999999,"15min":1.6679999999999999},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.0449999999999999},"max":{"1min":1.9570000000000001,"5min":2.1499999999999999,"15min":2.5409999999999999},"last":1.9359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576002,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60468,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311372,"statfs":{"total":96636764160,"available":96574844928,"internally_reserved":0,"allocated":2985984,"data_stored":1539424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.81599999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6319999999999999,"5min":1.6799999999999999,"15min":1.9930000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.64200000000000002,"15min":0.627},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":1.012,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.738,"5min":1.806,"15min":1.8620000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.202,"15min":1.2330000000000001},"min":{"1min":0.88500000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.75,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.2829999999999999,"15min":1.302},"min":{"1min":1.087,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.802},"last":1.238},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.8600000000000001,"15min":1.891},"last":0.95899999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.325,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.732,"5min":1.829,"15min":1.9650000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.4079999999999999,"15min":1.4199999999999999},"min":{"1min":1.038,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.772,"5min":1.881,"15min":1.9139999999999999},"last":1.3380000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2230000000000001,"15min":1.25},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.462,"5min":1.7050000000000001,"15min":1.9910000000000001},"last":1.173},{"interface":"front","average":{"1min":1.46,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":1.1259999999999999,"5min":0.85399999999999998,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":1.877,"15min":1.9399999999999999},"last":1.464}]},{"osd":6,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.482},"min":{"1min":1.081,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8540000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.508,"15min":1.524},"min":{"1min":0.84999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7509999999999999,"5min":1.9730000000000001,"15min":2.52},"last":1.395}]},{"osd":7,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5309999999999999,"15min":1.5329999999999999},"min":{"1min":1.1319999999999999,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":1.96,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.573},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.544,"15min":1.5640000000000001},"min":{"1min":1.018,"5min":0.68400000000000005,"15min":0.56399999999999995},"max":{"1min":1.833,"5min":2.0750000000000002,"15min":2.1349999999999998},"last":1.506}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47972,"kb_used_data":2900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323868,"statfs":{"total":96636764160,"available":96587640832,"internally_reserved":0,"allocated":2969600,"data_stored":1538358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.117,"15min":1.1599999999999999},"min":{"1min":0.72299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.8480000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.659,"5min":1.855,"15min":1.869},"last":1.268}]},{"osd":1,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.58099999999999996},"max":{"1min":1.4890000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.248,"15min":1.268},"min":{"1min":0.93300000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.665,"5min":1.762,"15min":1.762},"last":0.91400000000000003}]},{"osd":2,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2649999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":1.71,"15min":1.7609999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3620000000000001,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5740000000000001,"5min":1.762,"15min":1.762},"last":1.5049999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2729999999999999,"15min":1.3200000000000001},"min":{"1min":0.92100000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4870000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.419,"15min":1.425},"min":{"1min":1.002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7350000000000001,"5min":1.825,"15min":1.9239999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.82699999999999996},"max":{"1min":1.639,"5min":1.8340000000000001,"15min":1.867},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.98799999999999999,"5min":0.879,"15min":0.81000000000000005},"max":{"1min":1.6990000000000001,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4139999999999999,"15min":1.4430000000000001},"min":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.825,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.595,"15min":1.595},"min":{"1min":1.147,"5min":1.147,"15min":1.133},"max":{"1min":1.7210000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.7470000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.534,"15min":1.5189999999999999},"min":{"1min":1.2090000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.833,"5min":2.234,"15min":2.234},"last":1.151},{"interface":"front","average":{"1min":1.55,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.0880000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9279999999999999,"5min":1.984,"15min":2.081},"last":1.6619999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56304,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315536,"statfs":{"total":96636764160,"available":96579108864,"internally_reserved":0,"allocated":2977792,"data_stored":1537834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98399999999999999},"min":{"1min":0.54400000000000004,"5min":0.48899999999999999,"15min":0.432},"max":{"1min":1.4550000000000001,"5min":1.6439999999999999,"15min":1.71},"last":1.107},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.73499999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2010000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0409999999999999,"15min":1.091},"min":{"1min":0.626,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3340000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.875,"5min":1.879,"15min":1.879},"last":1.3140000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1100000000000001,"15min":1.1160000000000001},"min":{"1min":0.59699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.921,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.823,"5min":1.853,"15min":2},"last":1.145}]},{"osd":4,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4350000000000001,"15min":1.4410000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.397},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4710000000000001},"min":{"1min":1.048,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.802,"5min":1.9430000000000001,"15min":2.169},"last":1.712}]},{"osd":5,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.89200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9330000000000001,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.5389999999999999,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.1800000000000002,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.9350000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.546,"15min":1.601},"min":{"1min":1.0189999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.966,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.593,"15min":1.625},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0579999999999998,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.601}]},{"osd":7,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.135,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2789999999999999,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.659,"15min":1.714},"min":{"1min":1.0529999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1760000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":2.073}]}]},{"osd":2,"up_from":13,"seq":55834576001,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52144,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319696,"statfs":{"total":96636764160,"available":96583368704,"internally_reserved":0,"allocated":2985984,"data_stored":1545995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.90400000000000003},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.149,"5min":1.841,"15min":1.841},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.79500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.7529999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.78900000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6539999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.417,"15min":1.395},"min":{"1min":1.151,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.8300000000000001,"5min":1.958,"15min":2.4060000000000001},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.1970000000000001,"15min":1.198},"min":{"1min":0.78400000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.1760000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.92900000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":2.077},"last":1.0209999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":1.1870000000000001,"5min":1.0529999999999999,"15min":1.044},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1680000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.522,"5min":1.468,"15min":1.46},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0329999999999999},"max":{"1min":1.9079999999999999,"5min":1.978,"15min":2.5939999999999999},"last":1.0389999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4279999999999999,"15min":1.4199999999999999},"min":{"1min":1.0800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.169},{"interface":"front","average":{"1min":1.615,"5min":1.6000000000000001,"15min":1.6120000000000001},"min":{"1min":1.286,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":1.9710000000000001,"5min":1.972,"15min":2.3759999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.571,"15min":1.575},"min":{"1min":1.145,"5min":1.1060000000000001,"15min":1.095},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.595,"5min":1.5349999999999999,"15min":1.5289999999999999},"min":{"1min":1.115,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.0880000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.651,"15min":1.647},"min":{"1min":1.337,"5min":1.1080000000000001,"15min":1.0780000000000001},"max":{"1min":2.1240000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5980000000000001,"15min":1.6040000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.2270000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52128,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319712,"statfs":{"total":96636764160,"available":96583385088,"internally_reserved":0,"allocated":2961408,"data_stored":1537686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3120000000000001,"5min":1.367,"15min":1.976},"last":0.629},{"interface":"front","average":{"1min":1.115,"5min":1.147,"15min":1.1619999999999999},"min":{"1min":0.81200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.679,"5min":1.833,"15min":1.8400000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.046,"15min":1.048},"min":{"1min":0.83399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5369999999999999,"5min":1.6990000000000001,"15min":1.79},"last":0.873},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.238,"15min":1.252},"min":{"1min":1.121,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6100000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.397},"min":{"1min":1.0780000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8620000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.498,"5min":1.28,"15min":1.304},"min":{"1min":0.98899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.9339999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3600000000000001,"15min":1.3580000000000001},"min":{"1min":1.0820000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.643,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.575,"5min":1.498,"15min":1.504},"min":{"1min":1.157,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.867,"5min":2.0750000000000002,"15min":2.383},"last":1.151}]},{"osd":5,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.202,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.732,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.524,"15min":1.5429999999999999},"min":{"1min":1.1639999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.8240000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.2769999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.6080000000000001,"15min":1.6060000000000001},"min":{"1min":1.2729999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.085,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.109},{"interface":"front","average":{"1min":1.556,"5min":1.552,"15min":1.5629999999999999},"min":{"1min":1.2390000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.752,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.653,"15min":1.6799999999999999},"min":{"1min":1.298,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.994,"5min":2.1829999999999998,"15min":2.548},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.738,"15min":1.7270000000000001},"min":{"1min":1.401,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1739999999999999,"5min":2.492,"15min":2.548},"last":1.3080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52704,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3559424,"data_stored":2128144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93400000000000005,"15min":0.95199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2989999999999999,"5min":1.514,"15min":1.774},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.036,"15min":1.101},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.506,"5min":1.556,"15min":1.774},"last":0.61699999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.121,"15min":1.171},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1140000000000001,"15min":1.181},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.708,"5min":1.708,"15min":1.851},"last":1.1120000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0960000000000001,"15min":1.133},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6259999999999999,"5min":1.897,"15min":1.9590000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.327,"15min":1.391},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.3149999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.353,"15min":1.4299999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8169999999999999,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":0.877},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5580000000000001,"15min":1.599},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3959999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.401,"15min":1.4059999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.772,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.51,"15min":1.573},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.1150000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.5049999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.847,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.569,"5min":1.611,"15min":1.6759999999999999},"min":{"1min":0.999,"5min":0.996,"15min":0.996},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":1.454}]},{"osd":7,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4890000000000001,"15min":1.5449999999999999},"min":{"1min":0.82799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8380000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.002},{"interface":"front","average":{"1min":1.478,"5min":1.5900000000000001,"15min":1.623},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.952,"5min":2.2519999999999998,"15min":2.262},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:38:52.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:38:52.257 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-10-21T17:38:52.256+0000 7f581ba9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:38:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.350+0000 7f5da5ee5640 1 Processor -- start 2023-10-21T17:38:52.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.350+0000 7f5da5ee5640 1 -- start start 2023-10-21T17:38:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.351+0000 7f5da5ee5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5da015feb0 0x7f5da006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da5ee5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5da006cec0 0x7f5da00cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da5ee5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00ca4b0 con 0x7f5da0160290 2023-10-21T17:38:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da5ee5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da006d3d0 con 0x7f5da015feb0 2023-10-21T17:38:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da5ee5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da006a720 con 0x7f5da006cec0 2023-10-21T17:38:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da4ee3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5da015feb0 0x7f5da006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da56e4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5da0160290 0x7f5da0070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59386/0 (socket says 172.21.15.53:59386) 2023-10-21T17:38:52.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5da56e4640 1 -- 172.21.15.53:0/80846136 learned_addr learned my addr 172.21.15.53:0/80846136 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.352+0000 7f5d97fff640 1 --2- 172.21.15.53:0/80846136 >> v2:172.21.15.124:3300/0 conn(0x7f5da006cec0 0x7f5da00cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.353+0000 7f5da4ee3640 1 -- 172.21.15.53:0/80846136 >> v2:172.21.15.124:3300/0 conn(0x7f5da006cec0 msgr2=0x7f5da00cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.353+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/80846136 >> v2:172.21.15.124:3300/0 conn(0x7f5da006cec0 0x7f5da00cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.353+0000 7f5da4ee3640 1 -- 172.21.15.53:0/80846136 >> v1:172.21.15.53:6789/0 conn(0x7f5da0160290 legacy=0x7f5da0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.353+0000 7f5da4ee3640 1 -- 172.21.15.53:0/80846136 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0069df0 con 0x7f5da015feb0 2023-10-21T17:38:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.353+0000 7f5d977fe640 1 -- 172.21.15.53:0/80846136 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150401332 0 0) 0x7f5d98001110 con 0x7f5da0160290 2023-10-21T17:38:52.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/80846136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da015feb0 0x7f5da006c980 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cfc20 tx=0x7f5d9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb999996d4027edd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5d977fe640 1 -- 172.21.15.53:0/80846136 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c02cc80 con 0x7f5da015feb0 2023-10-21T17:38:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5d977fe640 1 -- 172.21.15.53:0/80846136 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c02cde0 con 0x7f5da015feb0 2023-10-21T17:38:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5d977fe640 1 -- 172.21.15.53:0/80846136 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c034660 con 0x7f5da015feb0 2023-10-21T17:38:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5da5ee5640 1 -- 172.21.15.53:0/80846136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da015feb0 msgr2=0x7f5da006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.354+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/80846136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da015feb0 0x7f5da006c980 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f5da00cfc20 tx=0x7f5d9c02bed0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 -- 172.21.15.53:0/80846136 shutdown_connections 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/80846136 >> v2:172.21.15.124:3300/0 conn(0x7f5da006cec0 0x7f5da00cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/80846136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da015feb0 0x7f5da006c980 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 -- 172.21.15.53:0/80846136 >> 172.21.15.53:0/80846136 conn(0x7f5da00cede0 msgr2=0x7f5da00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 -- 172.21.15.53:0/80846136 shutdown_connections 2023-10-21T17:38:52.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.355+0000 7f5da5ee5640 1 -- 172.21.15.53:0/80846136 wait complete. 2023-10-21T17:38:52.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.356+0000 7f5da5ee5640 1 Processor -- start 2023-10-21T17:38:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.357+0000 7f5da5ee5640 1 -- start start 2023-10-21T17:38:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 0x7f5da0158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5da015feb0 0x7f5da00c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015e010 con 0x7f5da0160290 2023-10-21T17:38:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c6520 con 0x7f5da006cec0 2023-10-21T17:38:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da5ee5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c6800 con 0x7f5da015feb0 2023-10-21T17:38:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49048/0 (socket says 172.21.15.53:49048) 2023-10-21T17:38:52.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 -- 172.21.15.53:0/1900304323 learned_addr learned my addr 172.21.15.53:0/1900304323 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 0x7f5da0158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5d97fff640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5da015feb0 0x7f5da00c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 -- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5da015feb0 msgr2=0x7f5da00c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5da015feb0 0x7f5da00c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 -- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 msgr2=0x7f5da0158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 0x7f5da0158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da56e4640 1 -- 172.21.15.53:0/1900304323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c009a70 con 0x7f5da0160290 2023-10-21T17:38:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.358+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 0x7f5da0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5da56e4640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c780 tx=0x7f5d9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980072d0 con 0x7f5da0160290 2023-10-21T17:38:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98007450 con 0x7f5da0160290 2023-10-21T17:38:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00c6ae0 con 0x7f5da0160290 2023-10-21T17:38:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98001d10 con 0x7f5da0160290 2023-10-21T17:38:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.359+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0156a40 con 0x7f5da0160290 2023-10-21T17:38:52.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.360+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da004e160 con 0x7f5da0160290 2023-10-21T17:38:52.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.361+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d980047e0 con 0x7f5da0160290 2023-10-21T17:38:52.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.362+0000 7f5d957fa640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 0x7f5d8c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.362+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f5d98088f40 con 0x7f5da0160290 2023-10-21T17:38:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.363+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 0x7f5d8c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.364+0000 7f5da4ee3640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 0x7f5d8c06d480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c036000 tx=0x7f5d9c030000 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.367+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d98053a50 con 0x7f5da0160290 2023-10-21T17:38:52.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.560+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5da006a3b0 con 0x7f5d8c06afc0 2023-10-21T17:38:52.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.605+0000 7f5d957fa640 1 -- 172.21.15.53:0/1900304323 <== mgr.37814 v2:172.21.15.124:6832/3930748558 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525568 (secure 0 0 0) 0x7f5d9c02cba0 con 0x7f5d8c06afc0 2023-10-21T17:38:52.614 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:52.618 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:38:52.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.612+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 msgr2=0x7f5d8c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.612+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 0x7f5d8c06d480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c036000 tx=0x7f5d9c030000 comp rx=0 tx=0).stop 2023-10-21T17:38:52.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.612+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 msgr2=0x7f5da00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.612+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c780 tx=0x7f5d9800cc40 comp rx=0 tx=0).stop 2023-10-21T17:38:52.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 shutdown_connections 2023-10-21T17:38:52.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5da0160290 0x7f5da00cc990 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5d8c06afc0 0x7f5d8c06d480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5da015feb0 0x7f5da00c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 --2- 172.21.15.53:0/1900304323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5da006cec0 0x7f5da0158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 >> 172.21.15.53:0/1900304323 conn(0x7f5da00cede0 msgr2=0x7f5da00caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:52.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 shutdown_connections 2023-10-21T17:38:52.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.613+0000 7f5da5ee5640 1 -- 172.21.15.53:0/1900304323 wait complete. 2023-10-21T17:38:52.636 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-10-21T17:38:51.685452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":678,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4952,"ondisk_log_size":4952,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":286,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439096,"kb_used_data":24436,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754535624,"statfs":{"total":773094113280,"available":772644478976,"internally_reserved":0,"allocated":25022464,"data_stored":13501957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":424463929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002552"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.568738+0000","last_change":"2023-10-21T17:38:47.566020+0000","last_active":"2023-10-21T17:38:47.568738+0000","last_peered":"2023-10-21T17:38:47.568738+0000","last_clean":"2023-10-21T17:38:47.568738+0000","last_became_active":"2023-10-21T17:38:47.565725+0000","last_became_peered":"2023-10-21T17:38:47.565725+0000","last_unstale":"2023-10-21T17:38:47.568738+0000","last_undegraded":"2023-10-21T17:38:47.568738+0000","last_fullsized":"2023-10-21T17:38:47.568738+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:09:02.823367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"221'4873","reported_seq":5806,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.554318+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:38:46.554318+0000","last_peered":"2023-10-21T17:38:46.554318+0000","last_clean":"2023-10-21T17:38:46.554318+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:38:46.554318+0000","last_undegraded":"2023-10-21T17:38:46.554318+0000","last_fullsized":"2023-10-21T17:38:46.554318+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4873,"log_dups_size":0,"ondisk_log_size":4873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"232'1","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538340+0000","last_change":"2023-10-21T17:38:32.397140+0000","last_active":"2023-10-21T17:38:47.538340+0000","last_peered":"2023-10-21T17:38:47.538340+0000","last_clean":"2023-10-21T17:38:47.538340+0000","last_became_active":"2023-10-21T17:38:32.396837+0000","last_became_peered":"2023-10-21T17:38:32.396837+0000","last_unstale":"2023-10-21T17:38:47.538340+0000","last_undegraded":"2023-10-21T17:38:47.538340+0000","last_fullsized":"2023-10-21T17:38:47.538340+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:17.875367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534995+0000","last_change":"2023-10-21T17:38:33.370135+0000","last_active":"2023-10-21T17:38:46.534995+0000","last_peered":"2023-10-21T17:38:46.534995+0000","last_clean":"2023-10-21T17:38:46.534995+0000","last_became_active":"2023-10-21T17:38:33.369798+0000","last_became_peered":"2023-10-21T17:38:33.369798+0000","last_unstale":"2023-10-21T17:38:46.534995+0000","last_undegraded":"2023-10-21T17:38:46.534995+0000","last_fullsized":"2023-10-21T17:38:46.534995+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:25.189245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"236'2","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570893+0000","last_change":"2023-10-21T17:38:46.562481+0000","last_active":"2023-10-21T17:38:47.570893+0000","last_peered":"2023-10-21T17:38:47.570893+0000","last_clean":"2023-10-21T17:38:47.570893+0000","last_became_active":"2023-10-21T17:38:46.562157+0000","last_became_peered":"2023-10-21T17:38:46.562157+0000","last_unstale":"2023-10-21T17:38:47.570893+0000","last_undegraded":"2023-10-21T17:38:47.570893+0000","last_fullsized":"2023-10-21T17:38:47.570893+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:33.520240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565336+0000","last_change":"2023-10-21T17:38:47.558062+0000","last_active":"2023-10-21T17:38:47.565336+0000","last_peered":"2023-10-21T17:38:47.565336+0000","last_clean":"2023-10-21T17:38:47.565336+0000","last_became_active":"2023-10-21T17:38:47.557643+0000","last_became_peered":"2023-10-21T17:38:47.557643+0000","last_unstale":"2023-10-21T17:38:47.565336+0000","last_undegraded":"2023-10-21T17:38:47.565336+0000","last_fullsized":"2023-10-21T17:38:47.565336+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:43:53.140469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.526291+0000","last_change":"2023-10-21T17:38:32.411805+0000","last_active":"2023-10-21T17:38:47.526291+0000","last_peered":"2023-10-21T17:38:47.526291+0000","last_clean":"2023-10-21T17:38:47.526291+0000","last_became_active":"2023-10-21T17:38:32.411439+0000","last_became_peered":"2023-10-21T17:38:32.411439+0000","last_unstale":"2023-10-21T17:38:47.526291+0000","last_undegraded":"2023-10-21T17:38:47.526291+0000","last_fullsized":"2023-10-21T17:38:47.526291+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:21:57.270863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.538119+0000","last_change":"2023-10-21T17:38:33.360420+0000","last_active":"2023-10-21T17:38:46.538119+0000","last_peered":"2023-10-21T17:38:46.538119+0000","last_clean":"2023-10-21T17:38:46.538119+0000","last_became_active":"2023-10-21T17:38:33.360143+0000","last_became_peered":"2023-10-21T17:38:33.360143+0000","last_unstale":"2023-10-21T17:38:46.538119+0000","last_undegraded":"2023-10-21T17:38:46.538119+0000","last_fullsized":"2023-10-21T17:38:46.538119+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:37.181561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.546759+0000","last_change":"2023-10-21T17:38:46.558587+0000","last_active":"2023-10-21T17:38:47.546759+0000","last_peered":"2023-10-21T17:38:47.546759+0000","last_clean":"2023-10-21T17:38:47.546759+0000","last_became_active":"2023-10-21T17:38:46.558305+0000","last_became_peered":"2023-10-21T17:38:46.558305+0000","last_unstale":"2023-10-21T17:38:47.546759+0000","last_undegraded":"2023-10-21T17:38:47.546759+0000","last_fullsized":"2023-10-21T17:38:47.546759+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:23.121781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.551881+0000","last_change":"2023-10-21T17:38:47.565116+0000","last_active":"2023-10-21T17:38:48.551881+0000","last_peered":"2023-10-21T17:38:48.551881+0000","last_clean":"2023-10-21T17:38:48.551881+0000","last_became_active":"2023-10-21T17:38:47.564818+0000","last_became_peered":"2023-10-21T17:38:47.564818+0000","last_unstale":"2023-10-21T17:38:48.551881+0000","last_undegraded":"2023-10-21T17:38:48.551881+0000","last_fullsized":"2023-10-21T17:38:48.551881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:49.072976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.537004+0000","last_change":"2023-10-21T17:38:32.411663+0000","last_active":"2023-10-21T17:38:46.537004+0000","last_peered":"2023-10-21T17:38:46.537004+0000","last_clean":"2023-10-21T17:38:46.537004+0000","last_became_active":"2023-10-21T17:38:32.411409+0000","last_became_peered":"2023-10-21T17:38:32.411409+0000","last_unstale":"2023-10-21T17:38:46.537004+0000","last_undegraded":"2023-10-21T17:38:46.537004+0000","last_fullsized":"2023-10-21T17:38:46.537004+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:00.391073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559209+0000","last_change":"2023-10-21T17:38:33.354071+0000","last_active":"2023-10-21T17:38:46.559209+0000","last_peered":"2023-10-21T17:38:46.559209+0000","last_clean":"2023-10-21T17:38:46.559209+0000","last_became_active":"2023-10-21T17:38:33.353746+0000","last_became_peered":"2023-10-21T17:38:33.353746+0000","last_unstale":"2023-10-21T17:38:46.559209+0000","last_undegraded":"2023-10-21T17:38:46.559209+0000","last_fullsized":"2023-10-21T17:38:46.559209+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:31.254948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558960+0000","last_change":"2023-10-21T17:38:46.563703+0000","last_active":"2023-10-21T17:38:48.558960+0000","last_peered":"2023-10-21T17:38:48.558960+0000","last_clean":"2023-10-21T17:38:48.558960+0000","last_became_active":"2023-10-21T17:38:46.563077+0000","last_became_peered":"2023-10-21T17:38:46.563077+0000","last_unstale":"2023-10-21T17:38:48.558960+0000","last_undegraded":"2023-10-21T17:38:48.558960+0000","last_fullsized":"2023-10-21T17:38:48.558960+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:49:50.477869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556780+0000","last_change":"2023-10-21T17:38:47.554568+0000","last_active":"2023-10-21T17:38:48.556780+0000","last_peered":"2023-10-21T17:38:48.556780+0000","last_clean":"2023-10-21T17:38:48.556780+0000","last_became_active":"2023-10-21T17:38:47.554270+0000","last_became_peered":"2023-10-21T17:38:47.554270+0000","last_unstale":"2023-10-21T17:38:48.556780+0000","last_undegraded":"2023-10-21T17:38:48.556780+0000","last_fullsized":"2023-10-21T17:38:48.556780+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:48:25.196265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538468+0000","last_change":"2023-10-21T17:38:32.408205+0000","last_active":"2023-10-21T17:38:47.538468+0000","last_peered":"2023-10-21T17:38:47.538468+0000","last_clean":"2023-10-21T17:38:47.538468+0000","last_became_active":"2023-10-21T17:38:32.407638+0000","last_became_peered":"2023-10-21T17:38:32.407638+0000","last_unstale":"2023-10-21T17:38:47.538468+0000","last_undegraded":"2023-10-21T17:38:47.538468+0000","last_fullsized":"2023-10-21T17:38:47.538468+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:20.909728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.561083+0000","last_change":"2023-10-21T17:38:33.362147+0000","last_active":"2023-10-21T17:38:46.561083+0000","last_peered":"2023-10-21T17:38:46.561083+0000","last_clean":"2023-10-21T17:38:46.561083+0000","last_became_active":"2023-10-21T17:38:33.360339+0000","last_became_peered":"2023-10-21T17:38:33.360339+0000","last_unstale":"2023-10-21T17:38:46.561083+0000","last_undegraded":"2023-10-21T17:38:46.561083+0000","last_fullsized":"2023-10-21T17:38:46.561083+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:51:06.624565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558139+0000","last_change":"2023-10-21T17:38:46.568841+0000","last_active":"2023-10-21T17:38:48.558139+0000","last_peered":"2023-10-21T17:38:48.558139+0000","last_clean":"2023-10-21T17:38:48.558139+0000","last_became_active":"2023-10-21T17:38:46.568115+0000","last_became_peered":"2023-10-21T17:38:46.568115+0000","last_unstale":"2023-10-21T17:38:48.558139+0000","last_undegraded":"2023-10-21T17:38:48.558139+0000","last_fullsized":"2023-10-21T17:38:48.558139+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:48:25.091301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565189+0000","last_change":"2023-10-21T17:38:47.559677+0000","last_active":"2023-10-21T17:38:47.565189+0000","last_peered":"2023-10-21T17:38:47.565189+0000","last_clean":"2023-10-21T17:38:47.565189+0000","last_became_active":"2023-10-21T17:38:47.559414+0000","last_became_peered":"2023-10-21T17:38:47.559414+0000","last_unstale":"2023-10-21T17:38:47.565189+0000","last_undegraded":"2023-10-21T17:38:47.565189+0000","last_fullsized":"2023-10-21T17:38:47.565189+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:43:53.140469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545658+0000","last_change":"2023-10-21T17:38:32.419011+0000","last_active":"2023-10-21T17:38:47.545658+0000","last_peered":"2023-10-21T17:38:47.545658+0000","last_clean":"2023-10-21T17:38:47.545658+0000","last_became_active":"2023-10-21T17:38:32.418410+0000","last_became_peered":"2023-10-21T17:38:32.418410+0000","last_unstale":"2023-10-21T17:38:47.545658+0000","last_undegraded":"2023-10-21T17:38:47.545658+0000","last_fullsized":"2023-10-21T17:38:47.545658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:22.056635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553378+0000","last_change":"2023-10-21T17:38:33.364849+0000","last_active":"2023-10-21T17:38:47.553378+0000","last_peered":"2023-10-21T17:38:47.553378+0000","last_clean":"2023-10-21T17:38:47.553378+0000","last_became_active":"2023-10-21T17:38:33.363791+0000","last_became_peered":"2023-10-21T17:38:33.363791+0000","last_unstale":"2023-10-21T17:38:47.553378+0000","last_undegraded":"2023-10-21T17:38:47.553378+0000","last_fullsized":"2023-10-21T17:38:47.553378+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:29.919088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.546759+0000","last_change":"2023-10-21T17:38:46.547298+0000","last_active":"2023-10-21T17:38:47.546759+0000","last_peered":"2023-10-21T17:38:47.546759+0000","last_clean":"2023-10-21T17:38:47.546759+0000","last_became_active":"2023-10-21T17:38:46.546520+0000","last_became_peered":"2023-10-21T17:38:46.546520+0000","last_unstale":"2023-10-21T17:38:47.546759+0000","last_undegraded":"2023-10-21T17:38:47.546759+0000","last_fullsized":"2023-10-21T17:38:47.546759+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:37.454918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552423+0000","last_change":"2023-10-21T17:38:47.550517+0000","last_active":"2023-10-21T17:38:47.552423+0000","last_peered":"2023-10-21T17:38:47.552423+0000","last_clean":"2023-10-21T17:38:47.552423+0000","last_became_active":"2023-10-21T17:38:47.550235+0000","last_became_peered":"2023-10-21T17:38:47.550235+0000","last_unstale":"2023-10-21T17:38:47.552423+0000","last_undegraded":"2023-10-21T17:38:47.552423+0000","last_fullsized":"2023-10-21T17:38:47.552423+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:06:46.717693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536611+0000","last_change":"2023-10-21T17:38:32.405103+0000","last_active":"2023-10-21T17:38:46.536611+0000","last_peered":"2023-10-21T17:38:46.536611+0000","last_clean":"2023-10-21T17:38:46.536611+0000","last_became_active":"2023-10-21T17:38:32.404811+0000","last_became_peered":"2023-10-21T17:38:32.404811+0000","last_unstale":"2023-10-21T17:38:46.536611+0000","last_undegraded":"2023-10-21T17:38:46.536611+0000","last_fullsized":"2023-10-21T17:38:46.536611+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:19.532470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532797+0000","last_change":"2023-10-21T17:38:33.351368+0000","last_active":"2023-10-21T17:38:46.532797+0000","last_peered":"2023-10-21T17:38:46.532797+0000","last_clean":"2023-10-21T17:38:46.532797+0000","last_became_active":"2023-10-21T17:38:33.350963+0000","last_became_peered":"2023-10-21T17:38:33.350963+0000","last_unstale":"2023-10-21T17:38:46.532797+0000","last_undegraded":"2023-10-21T17:38:46.532797+0000","last_fullsized":"2023-10-21T17:38:46.532797+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:24.335865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.531083+0000","last_change":"2023-10-21T17:38:46.550779+0000","last_active":"2023-10-21T17:38:47.531083+0000","last_peered":"2023-10-21T17:38:47.531083+0000","last_clean":"2023-10-21T17:38:47.531083+0000","last_became_active":"2023-10-21T17:38:46.550494+0000","last_became_peered":"2023-10-21T17:38:46.550494+0000","last_unstale":"2023-10-21T17:38:47.531083+0000","last_undegraded":"2023-10-21T17:38:47.531083+0000","last_fullsized":"2023-10-21T17:38:47.531083+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:28.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565357+0000","last_change":"2023-10-21T17:38:47.560233+0000","last_active":"2023-10-21T17:38:47.565357+0000","last_peered":"2023-10-21T17:38:47.565357+0000","last_clean":"2023-10-21T17:38:47.565357+0000","last_became_active":"2023-10-21T17:38:47.559324+0000","last_became_peered":"2023-10-21T17:38:47.559324+0000","last_unstale":"2023-10-21T17:38:47.565357+0000","last_undegraded":"2023-10-21T17:38:47.565357+0000","last_fullsized":"2023-10-21T17:38:47.565357+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:41:56.287349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.551433+0000","last_change":"2023-10-21T17:38:32.405560+0000","last_active":"2023-10-21T17:38:46.551433+0000","last_peered":"2023-10-21T17:38:46.551433+0000","last_clean":"2023-10-21T17:38:46.551433+0000","last_became_active":"2023-10-21T17:38:32.405233+0000","last_became_peered":"2023-10-21T17:38:32.405233+0000","last_unstale":"2023-10-21T17:38:46.551433+0000","last_undegraded":"2023-10-21T17:38:46.551433+0000","last_fullsized":"2023-10-21T17:38:46.551433+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538054+0000","last_change":"2023-10-21T17:38:33.358411+0000","last_active":"2023-10-21T17:38:47.538054+0000","last_peered":"2023-10-21T17:38:47.538054+0000","last_clean":"2023-10-21T17:38:47.538054+0000","last_became_active":"2023-10-21T17:38:33.358101+0000","last_became_peered":"2023-10-21T17:38:33.358101+0000","last_unstale":"2023-10-21T17:38:47.538054+0000","last_undegraded":"2023-10-21T17:38:47.538054+0000","last_fullsized":"2023-10-21T17:38:47.538054+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:31:32.268023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557897+0000","last_change":"2023-10-21T17:38:46.559154+0000","last_active":"2023-10-21T17:38:48.557897+0000","last_peered":"2023-10-21T17:38:48.557897+0000","last_clean":"2023-10-21T17:38:48.557897+0000","last_became_active":"2023-10-21T17:38:46.558296+0000","last_became_peered":"2023-10-21T17:38:46.558296+0000","last_unstale":"2023-10-21T17:38:48.557897+0000","last_undegraded":"2023-10-21T17:38:48.557897+0000","last_fullsized":"2023-10-21T17:38:48.557897+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:21:41.359330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558279+0000","last_change":"2023-10-21T17:38:47.556014+0000","last_active":"2023-10-21T17:38:48.558279+0000","last_peered":"2023-10-21T17:38:48.558279+0000","last_clean":"2023-10-21T17:38:48.558279+0000","last_became_active":"2023-10-21T17:38:47.555739+0000","last_became_peered":"2023-10-21T17:38:47.555739+0000","last_unstale":"2023-10-21T17:38:48.558279+0000","last_undegraded":"2023-10-21T17:38:48.558279+0000","last_fullsized":"2023-10-21T17:38:48.558279+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:48.820162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"233'1","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552631+0000","last_change":"2023-10-21T17:38:32.416259+0000","last_active":"2023-10-21T17:38:47.552631+0000","last_peered":"2023-10-21T17:38:47.552631+0000","last_clean":"2023-10-21T17:38:47.552631+0000","last_became_active":"2023-10-21T17:38:32.415430+0000","last_became_peered":"2023-10-21T17:38:32.415430+0000","last_unstale":"2023-10-21T17:38:47.552631+0000","last_undegraded":"2023-10-21T17:38:47.552631+0000","last_fullsized":"2023-10-21T17:38:47.552631+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:36:37.140028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532609+0000","last_change":"2023-10-21T17:38:33.358506+0000","last_active":"2023-10-21T17:38:46.532609+0000","last_peered":"2023-10-21T17:38:46.532609+0000","last_clean":"2023-10-21T17:38:46.532609+0000","last_became_active":"2023-10-21T17:38:33.357627+0000","last_became_peered":"2023-10-21T17:38:33.357627+0000","last_unstale":"2023-10-21T17:38:46.532609+0000","last_undegraded":"2023-10-21T17:38:46.532609+0000","last_fullsized":"2023-10-21T17:38:46.532609+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:09:10.598342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556150+0000","last_change":"2023-10-21T17:38:46.562461+0000","last_active":"2023-10-21T17:38:48.556150+0000","last_peered":"2023-10-21T17:38:48.556150+0000","last_clean":"2023-10-21T17:38:48.556150+0000","last_became_active":"2023-10-21T17:38:46.562169+0000","last_became_peered":"2023-10-21T17:38:46.562169+0000","last_unstale":"2023-10-21T17:38:48.556150+0000","last_undegraded":"2023-10-21T17:38:48.556150+0000","last_fullsized":"2023-10-21T17:38:48.556150+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:47:35.077192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555249+0000","last_change":"2023-10-21T17:38:47.539213+0000","last_active":"2023-10-21T17:38:48.555249+0000","last_peered":"2023-10-21T17:38:48.555249+0000","last_clean":"2023-10-21T17:38:48.555249+0000","last_became_active":"2023-10-21T17:38:47.538742+0000","last_became_peered":"2023-10-21T17:38:47.538742+0000","last_unstale":"2023-10-21T17:38:48.555249+0000","last_undegraded":"2023-10-21T17:38:48.555249+0000","last_fullsized":"2023-10-21T17:38:48.555249+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:29:37.691514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541744+0000","last_change":"2023-10-21T17:38:32.418683+0000","last_active":"2023-10-21T17:38:47.541744+0000","last_peered":"2023-10-21T17:38:47.541744+0000","last_clean":"2023-10-21T17:38:47.541744+0000","last_became_active":"2023-10-21T17:38:32.418325+0000","last_became_peered":"2023-10-21T17:38:32.418325+0000","last_unstale":"2023-10-21T17:38:47.541744+0000","last_undegraded":"2023-10-21T17:38:47.541744+0000","last_fullsized":"2023-10-21T17:38:47.541744+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:36.165616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541407+0000","last_change":"2023-10-21T17:38:33.361024+0000","last_active":"2023-10-21T17:38:47.541407+0000","last_peered":"2023-10-21T17:38:47.541407+0000","last_clean":"2023-10-21T17:38:47.541407+0000","last_became_active":"2023-10-21T17:38:33.360523+0000","last_became_peered":"2023-10-21T17:38:33.360523+0000","last_unstale":"2023-10-21T17:38:47.541407+0000","last_undegraded":"2023-10-21T17:38:47.541407+0000","last_fullsized":"2023-10-21T17:38:47.541407+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:25.912046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.540560+0000","last_change":"2023-10-21T17:38:46.561178+0000","last_active":"2023-10-21T17:38:47.540560+0000","last_peered":"2023-10-21T17:38:47.540560+0000","last_clean":"2023-10-21T17:38:47.540560+0000","last_became_active":"2023-10-21T17:38:46.560869+0000","last_became_peered":"2023-10-21T17:38:46.560869+0000","last_unstale":"2023-10-21T17:38:47.540560+0000","last_undegraded":"2023-10-21T17:38:47.540560+0000","last_fullsized":"2023-10-21T17:38:47.540560+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:03.944786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552897+0000","last_change":"2023-10-21T17:38:47.549334+0000","last_active":"2023-10-21T17:38:47.552897+0000","last_peered":"2023-10-21T17:38:47.552897+0000","last_clean":"2023-10-21T17:38:47.552897+0000","last_became_active":"2023-10-21T17:38:47.548345+0000","last_became_peered":"2023-10-21T17:38:47.548345+0000","last_unstale":"2023-10-21T17:38:47.552897+0000","last_undegraded":"2023-10-21T17:38:47.552897+0000","last_fullsized":"2023-10-21T17:38:47.552897+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:00:52.120978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531967+0000","last_change":"2023-10-21T17:38:32.410021+0000","last_active":"2023-10-21T17:38:46.531967+0000","last_peered":"2023-10-21T17:38:46.531967+0000","last_clean":"2023-10-21T17:38:46.531967+0000","last_became_active":"2023-10-21T17:38:32.409751+0000","last_became_peered":"2023-10-21T17:38:32.409751+0000","last_unstale":"2023-10-21T17:38:46.531967+0000","last_undegraded":"2023-10-21T17:38:46.531967+0000","last_fullsized":"2023-10-21T17:38:46.531967+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553995+0000","last_change":"2023-10-21T17:38:33.363577+0000","last_active":"2023-10-21T17:38:47.553995+0000","last_peered":"2023-10-21T17:38:47.553995+0000","last_clean":"2023-10-21T17:38:47.553995+0000","last_became_active":"2023-10-21T17:38:33.360945+0000","last_became_peered":"2023-10-21T17:38:33.360945+0000","last_unstale":"2023-10-21T17:38:47.553995+0000","last_undegraded":"2023-10-21T17:38:47.553995+0000","last_fullsized":"2023-10-21T17:38:47.553995+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:21.404419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555772+0000","last_change":"2023-10-21T17:38:46.550016+0000","last_active":"2023-10-21T17:38:48.555772+0000","last_peered":"2023-10-21T17:38:48.555772+0000","last_clean":"2023-10-21T17:38:48.555772+0000","last_became_active":"2023-10-21T17:38:46.549494+0000","last_became_peered":"2023-10-21T17:38:46.549494+0000","last_unstale":"2023-10-21T17:38:48.555772+0000","last_undegraded":"2023-10-21T17:38:48.555772+0000","last_fullsized":"2023-10-21T17:38:48.555772+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:15:38.899767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552239+0000","last_change":"2023-10-21T17:38:47.561495+0000","last_active":"2023-10-21T17:38:48.552239+0000","last_peered":"2023-10-21T17:38:48.552239+0000","last_clean":"2023-10-21T17:38:48.552239+0000","last_became_active":"2023-10-21T17:38:47.561226+0000","last_became_peered":"2023-10-21T17:38:47.561226+0000","last_unstale":"2023-10-21T17:38:48.552239+0000","last_undegraded":"2023-10-21T17:38:48.552239+0000","last_fullsized":"2023-10-21T17:38:48.552239+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:31.912580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.547887+0000","last_change":"2023-10-21T17:38:32.416515+0000","last_active":"2023-10-21T17:38:46.547887+0000","last_peered":"2023-10-21T17:38:46.547887+0000","last_clean":"2023-10-21T17:38:46.547887+0000","last_became_active":"2023-10-21T17:38:32.415411+0000","last_became_peered":"2023-10-21T17:38:32.415411+0000","last_unstale":"2023-10-21T17:38:46.547887+0000","last_undegraded":"2023-10-21T17:38:46.547887+0000","last_fullsized":"2023-10-21T17:38:46.547887+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:14:24.896102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.551013+0000","last_change":"2023-10-21T17:38:33.353848+0000","last_active":"2023-10-21T17:38:47.551013+0000","last_peered":"2023-10-21T17:38:47.551013+0000","last_clean":"2023-10-21T17:38:47.551013+0000","last_became_active":"2023-10-21T17:38:33.351959+0000","last_became_peered":"2023-10-21T17:38:33.351959+0000","last_unstale":"2023-10-21T17:38:47.551013+0000","last_undegraded":"2023-10-21T17:38:47.551013+0000","last_fullsized":"2023-10-21T17:38:47.551013+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:21.379053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.572130+0000","last_change":"2023-10-21T17:38:46.558191+0000","last_active":"2023-10-21T17:38:47.572130+0000","last_peered":"2023-10-21T17:38:47.572130+0000","last_clean":"2023-10-21T17:38:47.572130+0000","last_became_active":"2023-10-21T17:38:46.557875+0000","last_became_peered":"2023-10-21T17:38:46.557875+0000","last_unstale":"2023-10-21T17:38:47.572130+0000","last_undegraded":"2023-10-21T17:38:47.572130+0000","last_fullsized":"2023-10-21T17:38:47.572130+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:04:25.310386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.559400+0000","last_change":"2023-10-21T17:38:47.559460+0000","last_active":"2023-10-21T17:38:48.559400+0000","last_peered":"2023-10-21T17:38:48.559400+0000","last_clean":"2023-10-21T17:38:48.559400+0000","last_became_active":"2023-10-21T17:38:47.559166+0000","last_became_peered":"2023-10-21T17:38:47.559166+0000","last_unstale":"2023-10-21T17:38:48.559400+0000","last_undegraded":"2023-10-21T17:38:48.559400+0000","last_fullsized":"2023-10-21T17:38:48.559400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:57.184166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534658+0000","last_change":"2023-10-21T17:38:32.399236+0000","last_active":"2023-10-21T17:38:46.534658+0000","last_peered":"2023-10-21T17:38:46.534658+0000","last_clean":"2023-10-21T17:38:46.534658+0000","last_became_active":"2023-10-21T17:38:32.398909+0000","last_became_peered":"2023-10-21T17:38:32.398909+0000","last_unstale":"2023-10-21T17:38:46.534658+0000","last_undegraded":"2023-10-21T17:38:46.534658+0000","last_fullsized":"2023-10-21T17:38:46.534658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:04.398168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559637+0000","last_change":"2023-10-21T17:38:33.354105+0000","last_active":"2023-10-21T17:38:46.559637+0000","last_peered":"2023-10-21T17:38:46.559637+0000","last_clean":"2023-10-21T17:38:46.559637+0000","last_became_active":"2023-10-21T17:38:33.353821+0000","last_became_peered":"2023-10-21T17:38:33.353821+0000","last_unstale":"2023-10-21T17:38:46.559637+0000","last_undegraded":"2023-10-21T17:38:46.559637+0000","last_fullsized":"2023-10-21T17:38:46.559637+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:34.936008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557893+0000","last_change":"2023-10-21T17:38:46.566157+0000","last_active":"2023-10-21T17:38:48.557893+0000","last_peered":"2023-10-21T17:38:48.557893+0000","last_clean":"2023-10-21T17:38:48.557893+0000","last_became_active":"2023-10-21T17:38:46.563886+0000","last_became_peered":"2023-10-21T17:38:46.563886+0000","last_unstale":"2023-10-21T17:38:48.557893+0000","last_undegraded":"2023-10-21T17:38:48.557893+0000","last_fullsized":"2023-10-21T17:38:48.557893+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:13.777469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565453+0000","last_change":"2023-10-21T17:38:47.562860+0000","last_active":"2023-10-21T17:38:47.565453+0000","last_peered":"2023-10-21T17:38:47.565453+0000","last_clean":"2023-10-21T17:38:47.565453+0000","last_became_active":"2023-10-21T17:38:47.562527+0000","last_became_peered":"2023-10-21T17:38:47.562527+0000","last_unstale":"2023-10-21T17:38:47.565453+0000","last_undegraded":"2023-10-21T17:38:47.565453+0000","last_fullsized":"2023-10-21T17:38:47.565453+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:44:42.233371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530311+0000","last_change":"2023-10-21T17:38:32.401853+0000","last_active":"2023-10-21T17:38:47.530311+0000","last_peered":"2023-10-21T17:38:47.530311+0000","last_clean":"2023-10-21T17:38:47.530311+0000","last_became_active":"2023-10-21T17:38:32.401567+0000","last_became_peered":"2023-10-21T17:38:32.401567+0000","last_unstale":"2023-10-21T17:38:47.530311+0000","last_undegraded":"2023-10-21T17:38:47.530311+0000","last_fullsized":"2023-10-21T17:38:47.530311+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:24.460794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535354+0000","last_change":"2023-10-21T17:38:33.348917+0000","last_active":"2023-10-21T17:38:46.535354+0000","last_peered":"2023-10-21T17:38:46.535354+0000","last_clean":"2023-10-21T17:38:46.535354+0000","last_became_active":"2023-10-21T17:38:33.348522+0000","last_became_peered":"2023-10-21T17:38:33.348522+0000","last_unstale":"2023-10-21T17:38:46.535354+0000","last_undegraded":"2023-10-21T17:38:46.535354+0000","last_fullsized":"2023-10-21T17:38:46.535354+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:03:40.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"236'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557489+0000","last_change":"2023-10-21T17:38:46.563174+0000","last_active":"2023-10-21T17:38:48.557489+0000","last_peered":"2023-10-21T17:38:48.557489+0000","last_clean":"2023-10-21T17:38:48.557489+0000","last_became_active":"2023-10-21T17:38:46.562784+0000","last_became_peered":"2023-10-21T17:38:46.562784+0000","last_unstale":"2023-10-21T17:38:48.557489+0000","last_undegraded":"2023-10-21T17:38:48.557489+0000","last_fullsized":"2023-10-21T17:38:48.557489+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:31:10.759058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565041+0000","last_change":"2023-10-21T17:38:47.558254+0000","last_active":"2023-10-21T17:38:47.565041+0000","last_peered":"2023-10-21T17:38:47.565041+0000","last_clean":"2023-10-21T17:38:47.565041+0000","last_became_active":"2023-10-21T17:38:47.557951+0000","last_became_peered":"2023-10-21T17:38:47.557951+0000","last_unstale":"2023-10-21T17:38:47.565041+0000","last_undegraded":"2023-10-21T17:38:47.565041+0000","last_fullsized":"2023-10-21T17:38:47.565041+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:23.084608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529286+0000","last_change":"2023-10-21T17:38:32.403394+0000","last_active":"2023-10-21T17:38:47.529286+0000","last_peered":"2023-10-21T17:38:47.529286+0000","last_clean":"2023-10-21T17:38:47.529286+0000","last_became_active":"2023-10-21T17:38:32.403104+0000","last_became_peered":"2023-10-21T17:38:32.403104+0000","last_unstale":"2023-10-21T17:38:47.529286+0000","last_undegraded":"2023-10-21T17:38:47.529286+0000","last_fullsized":"2023-10-21T17:38:47.529286+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:06:56.765497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545565+0000","last_change":"2023-10-21T17:38:33.366719+0000","last_active":"2023-10-21T17:38:47.545565+0000","last_peered":"2023-10-21T17:38:47.545565+0000","last_clean":"2023-10-21T17:38:47.545565+0000","last_became_active":"2023-10-21T17:38:33.366173+0000","last_became_peered":"2023-10-21T17:38:33.366173+0000","last_unstale":"2023-10-21T17:38:47.545565+0000","last_undegraded":"2023-10-21T17:38:47.545565+0000","last_fullsized":"2023-10-21T17:38:47.545565+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:58.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.543198+0000","last_change":"2023-10-21T17:38:46.554315+0000","last_active":"2023-10-21T17:38:47.543198+0000","last_peered":"2023-10-21T17:38:47.543198+0000","last_clean":"2023-10-21T17:38:47.543198+0000","last_became_active":"2023-10-21T17:38:46.554078+0000","last_became_peered":"2023-10-21T17:38:46.554078+0000","last_unstale":"2023-10-21T17:38:47.543198+0000","last_undegraded":"2023-10-21T17:38:47.543198+0000","last_fullsized":"2023-10-21T17:38:47.543198+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:38.308297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555935+0000","last_change":"2023-10-21T17:38:47.558370+0000","last_active":"2023-10-21T17:38:48.555935+0000","last_peered":"2023-10-21T17:38:48.555935+0000","last_clean":"2023-10-21T17:38:48.555935+0000","last_became_active":"2023-10-21T17:38:47.558070+0000","last_became_peered":"2023-10-21T17:38:47.558070+0000","last_unstale":"2023-10-21T17:38:48.555935+0000","last_undegraded":"2023-10-21T17:38:48.555935+0000","last_fullsized":"2023-10-21T17:38:48.555935+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:35:59.802031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.538996+0000","last_change":"2023-10-21T17:38:32.406815+0000","last_active":"2023-10-21T17:38:46.538996+0000","last_peered":"2023-10-21T17:38:46.538996+0000","last_clean":"2023-10-21T17:38:46.538996+0000","last_became_active":"2023-10-21T17:38:32.406525+0000","last_became_peered":"2023-10-21T17:38:32.406525+0000","last_unstale":"2023-10-21T17:38:46.538996+0000","last_undegraded":"2023-10-21T17:38:46.538996+0000","last_fullsized":"2023-10-21T17:38:46.538996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:01:01.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532596+0000","last_change":"2023-10-21T17:38:33.343136+0000","last_active":"2023-10-21T17:38:46.532596+0000","last_peered":"2023-10-21T17:38:46.532596+0000","last_clean":"2023-10-21T17:38:46.532596+0000","last_became_active":"2023-10-21T17:38:33.342344+0000","last_became_peered":"2023-10-21T17:38:33.342344+0000","last_unstale":"2023-10-21T17:38:46.532596+0000","last_undegraded":"2023-10-21T17:38:46.532596+0000","last_fullsized":"2023-10-21T17:38:46.532596+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:20:01.763000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557679+0000","last_change":"2023-10-21T17:38:46.567745+0000","last_active":"2023-10-21T17:38:48.557679+0000","last_peered":"2023-10-21T17:38:48.557679+0000","last_clean":"2023-10-21T17:38:48.557679+0000","last_became_active":"2023-10-21T17:38:46.567182+0000","last_became_peered":"2023-10-21T17:38:46.567182+0000","last_unstale":"2023-10-21T17:38:48.557679+0000","last_undegraded":"2023-10-21T17:38:48.557679+0000","last_fullsized":"2023-10-21T17:38:48.557679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:56:15.194684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565379+0000","last_change":"2023-10-21T17:38:47.559506+0000","last_active":"2023-10-21T17:38:47.565379+0000","last_peered":"2023-10-21T17:38:47.565379+0000","last_clean":"2023-10-21T17:38:47.565379+0000","last_became_active":"2023-10-21T17:38:47.559218+0000","last_became_peered":"2023-10-21T17:38:47.559218+0000","last_unstale":"2023-10-21T17:38:47.565379+0000","last_undegraded":"2023-10-21T17:38:47.565379+0000","last_fullsized":"2023-10-21T17:38:47.565379+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:16:43.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536048+0000","last_change":"2023-10-21T17:38:32.413129+0000","last_active":"2023-10-21T17:38:46.536048+0000","last_peered":"2023-10-21T17:38:46.536048+0000","last_clean":"2023-10-21T17:38:46.536048+0000","last_became_active":"2023-10-21T17:38:32.410406+0000","last_became_peered":"2023-10-21T17:38:32.410406+0000","last_unstale":"2023-10-21T17:38:46.536048+0000","last_undegraded":"2023-10-21T17:38:46.536048+0000","last_fullsized":"2023-10-21T17:38:46.536048+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:30.006550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536664+0000","last_change":"2023-10-21T17:38:33.358480+0000","last_active":"2023-10-21T17:38:46.536664+0000","last_peered":"2023-10-21T17:38:46.536664+0000","last_clean":"2023-10-21T17:38:46.536664+0000","last_became_active":"2023-10-21T17:38:33.358208+0000","last_became_peered":"2023-10-21T17:38:33.358208+0000","last_unstale":"2023-10-21T17:38:46.536664+0000","last_undegraded":"2023-10-21T17:38:46.536664+0000","last_fullsized":"2023-10-21T17:38:46.536664+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:22.754068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555023+0000","last_change":"2023-10-21T17:38:46.548373+0000","last_active":"2023-10-21T17:38:48.555023+0000","last_peered":"2023-10-21T17:38:48.555023+0000","last_clean":"2023-10-21T17:38:48.555023+0000","last_became_active":"2023-10-21T17:38:46.547728+0000","last_became_peered":"2023-10-21T17:38:46.547728+0000","last_unstale":"2023-10-21T17:38:48.555023+0000","last_undegraded":"2023-10-21T17:38:48.555023+0000","last_fullsized":"2023-10-21T17:38:48.555023+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:04:11.654326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552067+0000","last_change":"2023-10-21T17:38:47.557513+0000","last_active":"2023-10-21T17:38:48.552067+0000","last_peered":"2023-10-21T17:38:48.552067+0000","last_clean":"2023-10-21T17:38:48.552067+0000","last_became_active":"2023-10-21T17:38:47.556804+0000","last_became_peered":"2023-10-21T17:38:47.556804+0000","last_unstale":"2023-10-21T17:38:48.552067+0000","last_undegraded":"2023-10-21T17:38:48.552067+0000","last_fullsized":"2023-10-21T17:38:48.552067+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:51:20.761534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534290+0000","last_change":"2023-10-21T17:38:32.404935+0000","last_active":"2023-10-21T17:38:46.534290+0000","last_peered":"2023-10-21T17:38:46.534290+0000","last_clean":"2023-10-21T17:38:46.534290+0000","last_became_active":"2023-10-21T17:38:32.404139+0000","last_became_peered":"2023-10-21T17:38:32.404139+0000","last_unstale":"2023-10-21T17:38:46.534290+0000","last_undegraded":"2023-10-21T17:38:46.534290+0000","last_fullsized":"2023-10-21T17:38:46.534290+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:53.425393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541314+0000","last_change":"2023-10-21T17:38:33.357202+0000","last_active":"2023-10-21T17:38:47.541314+0000","last_peered":"2023-10-21T17:38:47.541314+0000","last_clean":"2023-10-21T17:38:47.541314+0000","last_became_active":"2023-10-21T17:38:33.356898+0000","last_became_peered":"2023-10-21T17:38:33.356898+0000","last_unstale":"2023-10-21T17:38:47.541314+0000","last_undegraded":"2023-10-21T17:38:47.541314+0000","last_fullsized":"2023-10-21T17:38:47.541314+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:39.055132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542887+0000","last_change":"2023-10-21T17:38:46.558734+0000","last_active":"2023-10-21T17:38:47.542887+0000","last_peered":"2023-10-21T17:38:47.542887+0000","last_clean":"2023-10-21T17:38:47.542887+0000","last_became_active":"2023-10-21T17:38:46.558210+0000","last_became_peered":"2023-10-21T17:38:46.558210+0000","last_unstale":"2023-10-21T17:38:47.542887+0000","last_undegraded":"2023-10-21T17:38:47.542887+0000","last_fullsized":"2023-10-21T17:38:47.542887+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:35.873121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559702+0000","last_change":"2023-10-21T17:38:47.549325+0000","last_active":"2023-10-21T17:38:47.559702+0000","last_peered":"2023-10-21T17:38:47.559702+0000","last_clean":"2023-10-21T17:38:47.559702+0000","last_became_active":"2023-10-21T17:38:47.549058+0000","last_became_peered":"2023-10-21T17:38:47.549058+0000","last_unstale":"2023-10-21T17:38:47.559702+0000","last_undegraded":"2023-10-21T17:38:47.559702+0000","last_fullsized":"2023-10-21T17:38:47.559702+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:51.612715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"232'8","reported_seq":24,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.526058+0000","last_change":"2023-10-21T17:38:32.411814+0000","last_active":"2023-10-21T17:38:47.526058+0000","last_peered":"2023-10-21T17:38:47.526058+0000","last_clean":"2023-10-21T17:38:47.526058+0000","last_became_active":"2023-10-21T17:38:32.411496+0000","last_became_peered":"2023-10-21T17:38:32.411496+0000","last_unstale":"2023-10-21T17:38:47.526058+0000","last_undegraded":"2023-10-21T17:38:47.526058+0000","last_fullsized":"2023-10-21T17:38:47.526058+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:17.401857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.558839+0000","last_change":"2023-10-21T17:38:33.362626+0000","last_active":"2023-10-21T17:38:46.558839+0000","last_peered":"2023-10-21T17:38:46.558839+0000","last_clean":"2023-10-21T17:38:46.558839+0000","last_became_active":"2023-10-21T17:38:33.361673+0000","last_became_peered":"2023-10-21T17:38:33.361673+0000","last_unstale":"2023-10-21T17:38:46.558839+0000","last_undegraded":"2023-10-21T17:38:46.558839+0000","last_fullsized":"2023-10-21T17:38:46.558839+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:17.775611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530168+0000","last_change":"2023-10-21T17:38:46.560755+0000","last_active":"2023-10-21T17:38:47.530168+0000","last_peered":"2023-10-21T17:38:47.530168+0000","last_clean":"2023-10-21T17:38:47.530168+0000","last_became_active":"2023-10-21T17:38:46.560473+0000","last_became_peered":"2023-10-21T17:38:46.560473+0000","last_unstale":"2023-10-21T17:38:47.530168+0000","last_undegraded":"2023-10-21T17:38:47.530168+0000","last_fullsized":"2023-10-21T17:38:47.530168+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:24:43.117413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556027+0000","last_change":"2023-10-21T17:38:47.547467+0000","last_active":"2023-10-21T17:38:47.556027+0000","last_peered":"2023-10-21T17:38:47.556027+0000","last_clean":"2023-10-21T17:38:47.556027+0000","last_became_active":"2023-10-21T17:38:47.547084+0000","last_became_peered":"2023-10-21T17:38:47.547084+0000","last_unstale":"2023-10-21T17:38:47.556027+0000","last_undegraded":"2023-10-21T17:38:47.556027+0000","last_fullsized":"2023-10-21T17:38:47.556027+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:45.159463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545340+0000","last_change":"2023-10-21T17:38:32.405540+0000","last_active":"2023-10-21T17:38:47.545340+0000","last_peered":"2023-10-21T17:38:47.545340+0000","last_clean":"2023-10-21T17:38:47.545340+0000","last_became_active":"2023-10-21T17:38:32.405267+0000","last_became_peered":"2023-10-21T17:38:32.405267+0000","last_unstale":"2023-10-21T17:38:47.545340+0000","last_undegraded":"2023-10-21T17:38:47.545340+0000","last_fullsized":"2023-10-21T17:38:47.545340+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:15.824449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550924+0000","last_change":"2023-10-21T17:38:33.358210+0000","last_active":"2023-10-21T17:38:47.550924+0000","last_peered":"2023-10-21T17:38:47.550924+0000","last_clean":"2023-10-21T17:38:47.550924+0000","last_became_active":"2023-10-21T17:38:33.357911+0000","last_became_peered":"2023-10-21T17:38:33.357911+0000","last_unstale":"2023-10-21T17:38:47.550924+0000","last_undegraded":"2023-10-21T17:38:47.550924+0000","last_fullsized":"2023-10-21T17:38:47.550924+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:29.632092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542844+0000","last_change":"2023-10-21T17:38:46.564259+0000","last_active":"2023-10-21T17:38:47.542844+0000","last_peered":"2023-10-21T17:38:47.542844+0000","last_clean":"2023-10-21T17:38:47.542844+0000","last_became_active":"2023-10-21T17:38:46.563937+0000","last_became_peered":"2023-10-21T17:38:46.563937+0000","last_unstale":"2023-10-21T17:38:47.542844+0000","last_undegraded":"2023-10-21T17:38:47.542844+0000","last_fullsized":"2023-10-21T17:38:47.542844+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:54:14.193241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557388+0000","last_change":"2023-10-21T17:38:47.552857+0000","last_active":"2023-10-21T17:38:47.557388+0000","last_peered":"2023-10-21T17:38:47.557388+0000","last_clean":"2023-10-21T17:38:47.557388+0000","last_became_active":"2023-10-21T17:38:47.552609+0000","last_became_peered":"2023-10-21T17:38:47.552609+0000","last_unstale":"2023-10-21T17:38:47.557388+0000","last_undegraded":"2023-10-21T17:38:47.557388+0000","last_fullsized":"2023-10-21T17:38:47.557388+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:34:58.603914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535330+0000","last_change":"2023-10-21T17:38:32.403441+0000","last_active":"2023-10-21T17:38:46.535330+0000","last_peered":"2023-10-21T17:38:46.535330+0000","last_clean":"2023-10-21T17:38:46.535330+0000","last_became_active":"2023-10-21T17:38:32.403179+0000","last_became_peered":"2023-10-21T17:38:32.403179+0000","last_unstale":"2023-10-21T17:38:46.535330+0000","last_undegraded":"2023-10-21T17:38:46.535330+0000","last_fullsized":"2023-10-21T17:38:46.535330+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:16:38.273786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535318+0000","last_change":"2023-10-21T17:38:33.351625+0000","last_active":"2023-10-21T17:38:46.535318+0000","last_peered":"2023-10-21T17:38:46.535318+0000","last_clean":"2023-10-21T17:38:46.535318+0000","last_became_active":"2023-10-21T17:38:33.351311+0000","last_became_peered":"2023-10-21T17:38:33.351311+0000","last_unstale":"2023-10-21T17:38:46.535318+0000","last_undegraded":"2023-10-21T17:38:46.535318+0000","last_fullsized":"2023-10-21T17:38:46.535318+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:07.568107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"236'6","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.577695+0000","last_change":"2023-10-21T17:38:46.564895+0000","last_active":"2023-10-21T17:38:47.577695+0000","last_peered":"2023-10-21T17:38:47.577695+0000","last_clean":"2023-10-21T17:38:47.577695+0000","last_became_active":"2023-10-21T17:38:46.564596+0000","last_became_peered":"2023-10-21T17:38:46.564596+0000","last_unstale":"2023-10-21T17:38:47.577695+0000","last_undegraded":"2023-10-21T17:38:47.577695+0000","last_fullsized":"2023-10-21T17:38:47.577695+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:16:03.944786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.560512+0000","last_change":"2023-10-21T17:38:47.557562+0000","last_active":"2023-10-21T17:38:47.560512+0000","last_peered":"2023-10-21T17:38:47.560512+0000","last_clean":"2023-10-21T17:38:47.560512+0000","last_became_active":"2023-10-21T17:38:47.556767+0000","last_became_peered":"2023-10-21T17:38:47.556767+0000","last_unstale":"2023-10-21T17:38:47.560512+0000","last_undegraded":"2023-10-21T17:38:47.560512+0000","last_fullsized":"2023-10-21T17:38:47.560512+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:17:32.693500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535404+0000","last_change":"2023-10-21T17:38:32.405640+0000","last_active":"2023-10-21T17:38:46.535404+0000","last_peered":"2023-10-21T17:38:46.535404+0000","last_clean":"2023-10-21T17:38:46.535404+0000","last_became_active":"2023-10-21T17:38:32.404457+0000","last_became_peered":"2023-10-21T17:38:32.404457+0000","last_unstale":"2023-10-21T17:38:46.535404+0000","last_undegraded":"2023-10-21T17:38:46.535404+0000","last_fullsized":"2023-10-21T17:38:46.535404+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:26:53.247310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535265+0000","last_change":"2023-10-21T17:38:33.347261+0000","last_active":"2023-10-21T17:38:46.535265+0000","last_peered":"2023-10-21T17:38:46.535265+0000","last_clean":"2023-10-21T17:38:46.535265+0000","last_became_active":"2023-10-21T17:38:33.346927+0000","last_became_peered":"2023-10-21T17:38:33.346927+0000","last_unstale":"2023-10-21T17:38:46.535265+0000","last_undegraded":"2023-10-21T17:38:46.535265+0000","last_fullsized":"2023-10-21T17:38:46.535265+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:04:01.340329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"236'3","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.569620+0000","last_change":"2023-10-21T17:38:46.564790+0000","last_active":"2023-10-21T17:38:47.569620+0000","last_peered":"2023-10-21T17:38:47.569620+0000","last_clean":"2023-10-21T17:38:47.569620+0000","last_became_active":"2023-10-21T17:38:46.564494+0000","last_became_peered":"2023-10-21T17:38:46.564494+0000","last_unstale":"2023-10-21T17:38:47.569620+0000","last_undegraded":"2023-10-21T17:38:47.569620+0000","last_fullsized":"2023-10-21T17:38:47.569620+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:20:12.102091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557080+0000","last_change":"2023-10-21T17:38:47.554422+0000","last_active":"2023-10-21T17:38:48.557080+0000","last_peered":"2023-10-21T17:38:48.557080+0000","last_clean":"2023-10-21T17:38:48.557080+0000","last_became_active":"2023-10-21T17:38:47.554111+0000","last_became_peered":"2023-10-21T17:38:47.554111+0000","last_unstale":"2023-10-21T17:38:48.557080+0000","last_undegraded":"2023-10-21T17:38:48.557080+0000","last_fullsized":"2023-10-21T17:38:48.557080+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:04.797866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":23,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559111+0000","last_change":"2023-10-21T17:38:32.416726+0000","last_active":"2023-10-21T17:38:46.559111+0000","last_peered":"2023-10-21T17:38:46.559111+0000","last_clean":"2023-10-21T17:38:46.559111+0000","last_became_active":"2023-10-21T17:38:32.416448+0000","last_became_peered":"2023-10-21T17:38:32.416448+0000","last_unstale":"2023-10-21T17:38:46.559111+0000","last_undegraded":"2023-10-21T17:38:46.559111+0000","last_fullsized":"2023-10-21T17:38:46.559111+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:36.459802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530062+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:47.530062+0000","last_peered":"2023-10-21T17:38:47.530062+0000","last_clean":"2023-10-21T17:38:47.530062+0000","last_became_active":"2023-10-21T17:38:33.361077+0000","last_became_peered":"2023-10-21T17:38:33.361077+0000","last_unstale":"2023-10-21T17:38:47.530062+0000","last_undegraded":"2023-10-21T17:38:47.530062+0000","last_fullsized":"2023-10-21T17:38:47.530062+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:42:09.263356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552256+0000","last_change":"2023-10-21T17:38:46.545526+0000","last_active":"2023-10-21T17:38:48.552256+0000","last_peered":"2023-10-21T17:38:48.552256+0000","last_clean":"2023-10-21T17:38:48.552256+0000","last_became_active":"2023-10-21T17:38:46.544826+0000","last_became_peered":"2023-10-21T17:38:46.544826+0000","last_unstale":"2023-10-21T17:38:48.552256+0000","last_undegraded":"2023-10-21T17:38:48.552256+0000","last_fullsized":"2023-10-21T17:38:48.552256+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:38.652056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552909+0000","last_change":"2023-10-21T17:38:47.553575+0000","last_active":"2023-10-21T17:38:48.552909+0000","last_peered":"2023-10-21T17:38:48.552909+0000","last_clean":"2023-10-21T17:38:48.552909+0000","last_became_active":"2023-10-21T17:38:47.553314+0000","last_became_peered":"2023-10-21T17:38:47.553314+0000","last_unstale":"2023-10-21T17:38:48.552909+0000","last_undegraded":"2023-10-21T17:38:48.552909+0000","last_fullsized":"2023-10-21T17:38:48.552909+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:16:10.316538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"232'3","reported_seq":19,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.540887+0000","last_change":"2023-10-21T17:38:32.404797+0000","last_active":"2023-10-21T17:38:47.540887+0000","last_peered":"2023-10-21T17:38:47.540887+0000","last_clean":"2023-10-21T17:38:47.540887+0000","last_became_active":"2023-10-21T17:38:32.404524+0000","last_became_peered":"2023-10-21T17:38:32.404524+0000","last_unstale":"2023-10-21T17:38:47.540887+0000","last_undegraded":"2023-10-21T17:38:47.540887+0000","last_fullsized":"2023-10-21T17:38:47.540887+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:07.928480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.538195+0000","last_change":"2023-10-21T17:38:33.361342+0000","last_active":"2023-10-21T17:38:47.538195+0000","last_peered":"2023-10-21T17:38:47.538195+0000","last_clean":"2023-10-21T17:38:47.538195+0000","last_became_active":"2023-10-21T17:38:33.361078+0000","last_became_peered":"2023-10-21T17:38:33.361078+0000","last_unstale":"2023-10-21T17:38:47.538195+0000","last_undegraded":"2023-10-21T17:38:47.538195+0000","last_fullsized":"2023-10-21T17:38:47.538195+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:07:24.254117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"236'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557435+0000","last_change":"2023-10-21T17:38:46.560608+0000","last_active":"2023-10-21T17:38:48.557435+0000","last_peered":"2023-10-21T17:38:48.557435+0000","last_clean":"2023-10-21T17:38:48.557435+0000","last_became_active":"2023-10-21T17:38:46.560262+0000","last_became_peered":"2023-10-21T17:38:46.560262+0000","last_unstale":"2023-10-21T17:38:48.557435+0000","last_undegraded":"2023-10-21T17:38:48.557435+0000","last_fullsized":"2023-10-21T17:38:48.557435+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:32.612494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561172+0000","last_change":"2023-10-21T17:38:47.558430+0000","last_active":"2023-10-21T17:38:47.561172+0000","last_peered":"2023-10-21T17:38:47.561172+0000","last_clean":"2023-10-21T17:38:47.561172+0000","last_became_active":"2023-10-21T17:38:47.558111+0000","last_became_peered":"2023-10-21T17:38:47.558111+0000","last_unstale":"2023-10-21T17:38:47.561172+0000","last_undegraded":"2023-10-21T17:38:47.561172+0000","last_fullsized":"2023-10-21T17:38:47.561172+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:08.578306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.559258+0000","last_change":"2023-10-21T17:38:32.414100+0000","last_active":"2023-10-21T17:38:46.559258+0000","last_peered":"2023-10-21T17:38:46.559258+0000","last_clean":"2023-10-21T17:38:46.559258+0000","last_became_active":"2023-10-21T17:38:32.413503+0000","last_became_peered":"2023-10-21T17:38:32.413503+0000","last_unstale":"2023-10-21T17:38:46.559258+0000","last_undegraded":"2023-10-21T17:38:46.559258+0000","last_fullsized":"2023-10-21T17:38:46.559258+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:41.928628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.532513+0000","last_change":"2023-10-21T17:38:33.359536+0000","last_active":"2023-10-21T17:38:46.532513+0000","last_peered":"2023-10-21T17:38:46.532513+0000","last_clean":"2023-10-21T17:38:46.532513+0000","last_became_active":"2023-10-21T17:38:33.358634+0000","last_became_peered":"2023-10-21T17:38:33.358634+0000","last_unstale":"2023-10-21T17:38:46.532513+0000","last_undegraded":"2023-10-21T17:38:46.532513+0000","last_fullsized":"2023-10-21T17:38:46.532513+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:18.891311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"236'8","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556076+0000","last_change":"2023-10-21T17:38:46.563699+0000","last_active":"2023-10-21T17:38:48.556076+0000","last_peered":"2023-10-21T17:38:48.556076+0000","last_clean":"2023-10-21T17:38:48.556076+0000","last_became_active":"2023-10-21T17:38:46.563400+0000","last_became_peered":"2023-10-21T17:38:46.563400+0000","last_unstale":"2023-10-21T17:38:48.556076+0000","last_undegraded":"2023-10-21T17:38:48.556076+0000","last_fullsized":"2023-10-21T17:38:48.556076+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:26:36.282720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.555680+0000","last_change":"2023-10-21T17:38:47.552389+0000","last_active":"2023-10-21T17:38:47.555680+0000","last_peered":"2023-10-21T17:38:47.555680+0000","last_clean":"2023-10-21T17:38:47.555680+0000","last_became_active":"2023-10-21T17:38:47.550867+0000","last_became_peered":"2023-10-21T17:38:47.550867+0000","last_unstale":"2023-10-21T17:38:47.555680+0000","last_undegraded":"2023-10-21T17:38:47.555680+0000","last_fullsized":"2023-10-21T17:38:47.555680+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:16:44.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530413+0000","last_change":"2023-10-21T17:38:32.417638+0000","last_active":"2023-10-21T17:38:47.530413+0000","last_peered":"2023-10-21T17:38:47.530413+0000","last_clean":"2023-10-21T17:38:47.530413+0000","last_became_active":"2023-10-21T17:38:32.417302+0000","last_became_peered":"2023-10-21T17:38:32.417302+0000","last_unstale":"2023-10-21T17:38:47.530413+0000","last_undegraded":"2023-10-21T17:38:47.530413+0000","last_fullsized":"2023-10-21T17:38:47.530413+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:22.089541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.545681+0000","last_change":"2023-10-21T17:38:33.349513+0000","last_active":"2023-10-21T17:38:46.545681+0000","last_peered":"2023-10-21T17:38:46.545681+0000","last_clean":"2023-10-21T17:38:46.545681+0000","last_became_active":"2023-10-21T17:38:33.349202+0000","last_became_peered":"2023-10-21T17:38:33.349202+0000","last_unstale":"2023-10-21T17:38:46.545681+0000","last_undegraded":"2023-10-21T17:38:46.545681+0000","last_fullsized":"2023-10-21T17:38:46.545681+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:33:14.663384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.522182+0000","last_change":"2023-10-21T17:38:46.539924+0000","last_active":"2023-10-21T17:38:47.522182+0000","last_peered":"2023-10-21T17:38:47.522182+0000","last_clean":"2023-10-21T17:38:47.522182+0000","last_became_active":"2023-10-21T17:38:46.539284+0000","last_became_peered":"2023-10-21T17:38:46.539284+0000","last_unstale":"2023-10-21T17:38:47.522182+0000","last_undegraded":"2023-10-21T17:38:47.522182+0000","last_fullsized":"2023-10-21T17:38:47.522182+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:41.174817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566536+0000","last_change":"2023-10-21T17:38:47.563567+0000","last_active":"2023-10-21T17:38:47.566536+0000","last_peered":"2023-10-21T17:38:47.566536+0000","last_clean":"2023-10-21T17:38:47.566536+0000","last_became_active":"2023-10-21T17:38:47.563295+0000","last_became_peered":"2023-10-21T17:38:47.563295+0000","last_unstale":"2023-10-21T17:38:47.566536+0000","last_undegraded":"2023-10-21T17:38:47.566536+0000","last_fullsized":"2023-10-21T17:38:47.566536+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:00.316336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"236'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.559065+0000","last_change":"2023-10-21T17:38:46.566907+0000","last_active":"2023-10-21T17:38:48.559065+0000","last_peered":"2023-10-21T17:38:48.559065+0000","last_clean":"2023-10-21T17:38:48.559065+0000","last_became_active":"2023-10-21T17:38:46.566616+0000","last_became_peered":"2023-10-21T17:38:46.566616+0000","last_unstale":"2023-10-21T17:38:48.559065+0000","last_undegraded":"2023-10-21T17:38:48.559065+0000","last_fullsized":"2023-10-21T17:38:48.559065+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:50.826966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531845+0000","last_change":"2023-10-21T17:38:32.410852+0000","last_active":"2023-10-21T17:38:46.531845+0000","last_peered":"2023-10-21T17:38:46.531845+0000","last_clean":"2023-10-21T17:38:46.531845+0000","last_became_active":"2023-10-21T17:38:32.410413+0000","last_became_peered":"2023-10-21T17:38:32.410413+0000","last_unstale":"2023-10-21T17:38:46.531845+0000","last_undegraded":"2023-10-21T17:38:46.531845+0000","last_fullsized":"2023-10-21T17:38:46.531845+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:41:26.544773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541647+0000","last_change":"2023-10-21T17:38:33.357688+0000","last_active":"2023-10-21T17:38:47.541647+0000","last_peered":"2023-10-21T17:38:47.541647+0000","last_clean":"2023-10-21T17:38:47.541647+0000","last_became_active":"2023-10-21T17:38:33.355135+0000","last_became_peered":"2023-10-21T17:38:33.355135+0000","last_unstale":"2023-10-21T17:38:47.541647+0000","last_undegraded":"2023-10-21T17:38:47.541647+0000","last_fullsized":"2023-10-21T17:38:47.541647+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:10.952499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.554607+0000","last_change":"2023-10-21T17:38:47.552201+0000","last_active":"2023-10-21T17:38:47.554607+0000","last_peered":"2023-10-21T17:38:47.554607+0000","last_clean":"2023-10-21T17:38:47.554607+0000","last_became_active":"2023-10-21T17:38:47.551908+0000","last_became_peered":"2023-10-21T17:38:47.551908+0000","last_unstale":"2023-10-21T17:38:47.554607+0000","last_undegraded":"2023-10-21T17:38:47.554607+0000","last_fullsized":"2023-10-21T17:38:47.554607+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:43:30.804596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"236'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556590+0000","last_change":"2023-10-21T17:38:46.558998+0000","last_active":"2023-10-21T17:38:48.556590+0000","last_peered":"2023-10-21T17:38:48.556590+0000","last_clean":"2023-10-21T17:38:48.556590+0000","last_became_active":"2023-10-21T17:38:46.558681+0000","last_became_peered":"2023-10-21T17:38:46.558681+0000","last_unstale":"2023-10-21T17:38:48.556590+0000","last_undegraded":"2023-10-21T17:38:48.556590+0000","last_fullsized":"2023-10-21T17:38:48.556590+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:18.054503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552420+0000","last_change":"2023-10-21T17:38:32.411637+0000","last_active":"2023-10-21T17:38:46.552420+0000","last_peered":"2023-10-21T17:38:46.552420+0000","last_clean":"2023-10-21T17:38:46.552420+0000","last_became_active":"2023-10-21T17:38:32.411322+0000","last_became_peered":"2023-10-21T17:38:32.411322+0000","last_unstale":"2023-10-21T17:38:46.552420+0000","last_undegraded":"2023-10-21T17:38:46.552420+0000","last_fullsized":"2023-10-21T17:38:46.552420+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:21:28.616148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.553870+0000","last_change":"2023-10-21T17:38:33.346021+0000","last_active":"2023-10-21T17:38:46.553870+0000","last_peered":"2023-10-21T17:38:46.553870+0000","last_clean":"2023-10-21T17:38:46.553870+0000","last_became_active":"2023-10-21T17:38:33.345731+0000","last_became_peered":"2023-10-21T17:38:33.345731+0000","last_unstale":"2023-10-21T17:38:46.553870+0000","last_undegraded":"2023-10-21T17:38:46.553870+0000","last_fullsized":"2023-10-21T17:38:46.553870+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:00:29.919088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558981+0000","last_change":"2023-10-21T17:38:47.558622+0000","last_active":"2023-10-21T17:38:48.558981+0000","last_peered":"2023-10-21T17:38:48.558981+0000","last_clean":"2023-10-21T17:38:48.558981+0000","last_became_active":"2023-10-21T17:38:47.557794+0000","last_became_peered":"2023-10-21T17:38:47.557794+0000","last_unstale":"2023-10-21T17:38:48.558981+0000","last_undegraded":"2023-10-21T17:38:48.558981+0000","last_fullsized":"2023-10-21T17:38:48.558981+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:07.090483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.557679+0000","last_change":"2023-10-21T17:38:32.411640+0000","last_active":"2023-10-21T17:38:46.557679+0000","last_peered":"2023-10-21T17:38:46.557679+0000","last_clean":"2023-10-21T17:38:46.557679+0000","last_became_active":"2023-10-21T17:38:32.411258+0000","last_became_peered":"2023-10-21T17:38:32.411258+0000","last_unstale":"2023-10-21T17:38:46.557679+0000","last_undegraded":"2023-10-21T17:38:46.557679+0000","last_fullsized":"2023-10-21T17:38:46.557679+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552008+0000","last_change":"2023-10-21T17:38:33.341123+0000","last_active":"2023-10-21T17:38:46.552008+0000","last_peered":"2023-10-21T17:38:46.552008+0000","last_clean":"2023-10-21T17:38:46.552008+0000","last_became_active":"2023-10-21T17:38:33.340397+0000","last_became_peered":"2023-10-21T17:38:33.340397+0000","last_unstale":"2023-10-21T17:38:46.552008+0000","last_undegraded":"2023-10-21T17:38:46.552008+0000","last_fullsized":"2023-10-21T17:38:46.552008+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:39.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"236'2","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.572850+0000","last_change":"2023-10-21T17:38:46.566632+0000","last_active":"2023-10-21T17:38:47.572850+0000","last_peered":"2023-10-21T17:38:47.572850+0000","last_clean":"2023-10-21T17:38:47.572850+0000","last_became_active":"2023-10-21T17:38:46.566333+0000","last_became_peered":"2023-10-21T17:38:46.566333+0000","last_unstale":"2023-10-21T17:38:47.572850+0000","last_undegraded":"2023-10-21T17:38:47.572850+0000","last_fullsized":"2023-10-21T17:38:47.572850+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:00:51.103062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.556817+0000","last_change":"2023-10-21T17:38:47.561183+0000","last_active":"2023-10-21T17:38:48.556817+0000","last_peered":"2023-10-21T17:38:48.556817+0000","last_clean":"2023-10-21T17:38:48.556817+0000","last_became_active":"2023-10-21T17:38:47.560072+0000","last_became_peered":"2023-10-21T17:38:47.560072+0000","last_unstale":"2023-10-21T17:38:48.556817+0000","last_undegraded":"2023-10-21T17:38:48.556817+0000","last_fullsized":"2023-10-21T17:38:48.556817+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:32:31.201096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"232'2","reported_seq":18,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.548545+0000","last_change":"2023-10-21T17:38:32.407056+0000","last_active":"2023-10-21T17:38:47.548545+0000","last_peered":"2023-10-21T17:38:47.548545+0000","last_clean":"2023-10-21T17:38:47.548545+0000","last_became_active":"2023-10-21T17:38:32.405548+0000","last_became_peered":"2023-10-21T17:38:32.405548+0000","last_unstale":"2023-10-21T17:38:47.548545+0000","last_undegraded":"2023-10-21T17:38:47.548545+0000","last_fullsized":"2023-10-21T17:38:47.548545+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:01:02.940331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529805+0000","last_change":"2023-10-21T17:38:33.350688+0000","last_active":"2023-10-21T17:38:47.529805+0000","last_peered":"2023-10-21T17:38:47.529805+0000","last_clean":"2023-10-21T17:38:47.529805+0000","last_became_active":"2023-10-21T17:38:33.350383+0000","last_became_peered":"2023-10-21T17:38:33.350383+0000","last_unstale":"2023-10-21T17:38:47.529805+0000","last_undegraded":"2023-10-21T17:38:47.529805+0000","last_fullsized":"2023-10-21T17:38:47.529805+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:05.778317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570003+0000","last_change":"2023-10-21T17:38:46.570202+0000","last_active":"2023-10-21T17:38:47.570003+0000","last_peered":"2023-10-21T17:38:47.570003+0000","last_clean":"2023-10-21T17:38:47.570003+0000","last_became_active":"2023-10-21T17:38:46.569832+0000","last_became_peered":"2023-10-21T17:38:46.569832+0000","last_unstale":"2023-10-21T17:38:47.570003+0000","last_undegraded":"2023-10-21T17:38:47.570003+0000","last_fullsized":"2023-10-21T17:38:47.570003+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:33:06.683362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559579+0000","last_change":"2023-10-21T17:38:47.556629+0000","last_active":"2023-10-21T17:38:47.559579+0000","last_peered":"2023-10-21T17:38:47.559579+0000","last_clean":"2023-10-21T17:38:47.559579+0000","last_became_active":"2023-10-21T17:38:47.555973+0000","last_became_peered":"2023-10-21T17:38:47.555973+0000","last_unstale":"2023-10-21T17:38:47.559579+0000","last_undegraded":"2023-10-21T17:38:47.559579+0000","last_fullsized":"2023-10-21T17:38:47.559579+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:49:13.600687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"232'2","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536443+0000","last_change":"2023-10-21T17:38:32.405671+0000","last_active":"2023-10-21T17:38:46.536443+0000","last_peered":"2023-10-21T17:38:46.536443+0000","last_clean":"2023-10-21T17:38:46.536443+0000","last_became_active":"2023-10-21T17:38:32.404175+0000","last_became_peered":"2023-10-21T17:38:32.404175+0000","last_unstale":"2023-10-21T17:38:46.536443+0000","last_undegraded":"2023-10-21T17:38:46.536443+0000","last_fullsized":"2023-10-21T17:38:46.536443+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:11:24.818705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542379+0000","last_change":"2023-10-21T17:38:33.360687+0000","last_active":"2023-10-21T17:38:47.542379+0000","last_peered":"2023-10-21T17:38:47.542379+0000","last_clean":"2023-10-21T17:38:47.542379+0000","last_became_active":"2023-10-21T17:38:33.360369+0000","last_became_peered":"2023-10-21T17:38:33.360369+0000","last_unstale":"2023-10-21T17:38:47.542379+0000","last_undegraded":"2023-10-21T17:38:47.542379+0000","last_fullsized":"2023-10-21T17:38:47.542379+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:04.002929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"236'1","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570688+0000","last_change":"2023-10-21T17:38:46.564099+0000","last_active":"2023-10-21T17:38:47.570688+0000","last_peered":"2023-10-21T17:38:47.570688+0000","last_clean":"2023-10-21T17:38:47.570688+0000","last_became_active":"2023-10-21T17:38:46.563814+0000","last_became_peered":"2023-10-21T17:38:46.563814+0000","last_unstale":"2023-10-21T17:38:47.570688+0000","last_undegraded":"2023-10-21T17:38:47.570688+0000","last_fullsized":"2023-10-21T17:38:47.570688+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:38.652056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555998+0000","last_change":"2023-10-21T17:38:47.550728+0000","last_active":"2023-10-21T17:38:48.555998+0000","last_peered":"2023-10-21T17:38:48.555998+0000","last_clean":"2023-10-21T17:38:48.555998+0000","last_became_active":"2023-10-21T17:38:47.549778+0000","last_became_peered":"2023-10-21T17:38:47.549778+0000","last_unstale":"2023-10-21T17:38:48.555998+0000","last_undegraded":"2023-10-21T17:38:48.555998+0000","last_fullsized":"2023-10-21T17:38:48.555998+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:02.913327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"232'1","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535472+0000","last_change":"2023-10-21T17:38:32.401420+0000","last_active":"2023-10-21T17:38:46.535472+0000","last_peered":"2023-10-21T17:38:46.535472+0000","last_clean":"2023-10-21T17:38:46.535472+0000","last_became_active":"2023-10-21T17:38:32.394692+0000","last_became_peered":"2023-10-21T17:38:32.394692+0000","last_unstale":"2023-10-21T17:38:46.535472+0000","last_undegraded":"2023-10-21T17:38:46.535472+0000","last_fullsized":"2023-10-21T17:38:46.535472+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:09.936554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.536446+0000","last_change":"2023-10-21T17:38:33.352486+0000","last_active":"2023-10-21T17:38:46.536446+0000","last_peered":"2023-10-21T17:38:46.536446+0000","last_clean":"2023-10-21T17:38:46.536446+0000","last_became_active":"2023-10-21T17:38:33.352150+0000","last_became_peered":"2023-10-21T17:38:33.352150+0000","last_unstale":"2023-10-21T17:38:46.536446+0000","last_undegraded":"2023-10-21T17:38:46.536446+0000","last_fullsized":"2023-10-21T17:38:46.536446+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:23:53.603618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.544594+0000","last_change":"2023-10-21T17:38:46.544612+0000","last_active":"2023-10-21T17:38:47.544594+0000","last_peered":"2023-10-21T17:38:47.544594+0000","last_clean":"2023-10-21T17:38:47.544594+0000","last_became_active":"2023-10-21T17:38:46.544322+0000","last_became_peered":"2023-10-21T17:38:46.544322+0000","last_unstale":"2023-10-21T17:38:47.544594+0000","last_undegraded":"2023-10-21T17:38:47.544594+0000","last_fullsized":"2023-10-21T17:38:47.544594+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:47.693310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557984+0000","last_change":"2023-10-21T17:38:47.552093+0000","last_active":"2023-10-21T17:38:47.557984+0000","last_peered":"2023-10-21T17:38:47.557984+0000","last_clean":"2023-10-21T17:38:47.557984+0000","last_became_active":"2023-10-21T17:38:47.551780+0000","last_became_peered":"2023-10-21T17:38:47.551780+0000","last_unstale":"2023-10-21T17:38:47.557984+0000","last_undegraded":"2023-10-21T17:38:47.557984+0000","last_fullsized":"2023-10-21T17:38:47.557984+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:24:07.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"232'4","reported_seq":19,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552776+0000","last_change":"2023-10-21T17:38:32.419554+0000","last_active":"2023-10-21T17:38:46.552776+0000","last_peered":"2023-10-21T17:38:46.552776+0000","last_clean":"2023-10-21T17:38:46.552776+0000","last_became_active":"2023-10-21T17:38:32.419276+0000","last_became_peered":"2023-10-21T17:38:32.419276+0000","last_unstale":"2023-10-21T17:38:46.552776+0000","last_undegraded":"2023-10-21T17:38:46.552776+0000","last_fullsized":"2023-10-21T17:38:46.552776+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:31.354704+0000","last_clean_scrub_stamp":"2023-10-21T17:38:31.354704+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:44.216819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534438+0000","last_change":"2023-10-21T17:38:33.350258+0000","last_active":"2023-10-21T17:38:46.534438+0000","last_peered":"2023-10-21T17:38:46.534438+0000","last_clean":"2023-10-21T17:38:46.534438+0000","last_became_active":"2023-10-21T17:38:33.349849+0000","last_became_peered":"2023-10-21T17:38:33.349849+0000","last_unstale":"2023-10-21T17:38:46.534438+0000","last_undegraded":"2023-10-21T17:38:46.534438+0000","last_fullsized":"2023-10-21T17:38:46.534438+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:08.906164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530075+0000","last_change":"2023-10-21T17:38:46.560891+0000","last_active":"2023-10-21T17:38:47.530075+0000","last_peered":"2023-10-21T17:38:47.530075+0000","last_clean":"2023-10-21T17:38:47.530075+0000","last_became_active":"2023-10-21T17:38:46.560549+0000","last_became_peered":"2023-10-21T17:38:46.560549+0000","last_unstale":"2023-10-21T17:38:47.530075+0000","last_undegraded":"2023-10-21T17:38:47.530075+0000","last_fullsized":"2023-10-21T17:38:47.530075+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:45.489700+0000","last_clean_scrub_stamp":"2023-10-21T17:38:45.489700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:51.552421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565097+0000","last_change":"2023-10-21T17:38:47.559132+0000","last_active":"2023-10-21T17:38:47.565097+0000","last_peered":"2023-10-21T17:38:47.565097+0000","last_clean":"2023-10-21T17:38:47.565097+0000","last_became_active":"2023-10-21T17:38:47.558872+0000","last_became_peered":"2023-10-21T17:38:47.558872+0000","last_unstale":"2023-10-21T17:38:47.565097+0000","last_undegraded":"2023-10-21T17:38:47.565097+0000","last_fullsized":"2023-10-21T17:38:47.565097+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:35.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.556204+0000","last_change":"2023-10-21T17:38:33.354963+0000","last_active":"2023-10-21T17:38:46.556204+0000","last_peered":"2023-10-21T17:38:46.556204+0000","last_clean":"2023-10-21T17:38:46.556204+0000","last_became_active":"2023-10-21T17:38:33.354183+0000","last_became_peered":"2023-10-21T17:38:33.354183+0000","last_unstale":"2023-10-21T17:38:46.556204+0000","last_undegraded":"2023-10-21T17:38:46.556204+0000","last_fullsized":"2023-10-21T17:38:46.556204+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:25.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.552626+0000","last_change":"2023-10-21T17:38:47.552387+0000","last_active":"2023-10-21T17:38:48.552626+0000","last_peered":"2023-10-21T17:38:48.552626+0000","last_clean":"2023-10-21T17:38:48.552626+0000","last_became_active":"2023-10-21T17:38:47.552122+0000","last_became_peered":"2023-10-21T17:38:47.552122+0000","last_unstale":"2023-10-21T17:38:48.552626+0000","last_undegraded":"2023-10-21T17:38:48.552626+0000","last_fullsized":"2023-10-21T17:38:48.552626+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:27.300690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.552309+0000","last_change":"2023-10-21T17:38:33.353725+0000","last_active":"2023-10-21T17:38:46.552309+0000","last_peered":"2023-10-21T17:38:46.552309+0000","last_clean":"2023-10-21T17:38:46.552309+0000","last_became_active":"2023-10-21T17:38:33.353441+0000","last_became_peered":"2023-10-21T17:38:33.353441+0000","last_unstale":"2023-10-21T17:38:46.552309+0000","last_undegraded":"2023-10-21T17:38:46.552309+0000","last_fullsized":"2023-10-21T17:38:46.552309+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:46:25.189245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566330+0000","last_change":"2023-10-21T17:38:47.561904+0000","last_active":"2023-10-21T17:38:47.566330+0000","last_peered":"2023-10-21T17:38:47.566330+0000","last_clean":"2023-10-21T17:38:47.566330+0000","last_became_active":"2023-10-21T17:38:47.560202+0000","last_became_peered":"2023-10-21T17:38:47.560202+0000","last_unstale":"2023-10-21T17:38:47.566330+0000","last_undegraded":"2023-10-21T17:38:47.566330+0000","last_fullsized":"2023-10-21T17:38:47.566330+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:49:34.076182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.551453+0000","last_change":"2023-10-21T17:38:33.340905+0000","last_active":"2023-10-21T17:38:46.551453+0000","last_peered":"2023-10-21T17:38:46.551453+0000","last_clean":"2023-10-21T17:38:46.551453+0000","last_became_active":"2023-10-21T17:38:33.340134+0000","last_became_peered":"2023-10-21T17:38:33.340134+0000","last_unstale":"2023-10-21T17:38:46.551453+0000","last_undegraded":"2023-10-21T17:38:46.551453+0000","last_fullsized":"2023-10-21T17:38:46.551453+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:03:58.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555361+0000","last_change":"2023-10-21T17:38:47.560945+0000","last_active":"2023-10-21T17:38:48.555361+0000","last_peered":"2023-10-21T17:38:48.555361+0000","last_clean":"2023-10-21T17:38:48.555361+0000","last_became_active":"2023-10-21T17:38:47.560222+0000","last_became_peered":"2023-10-21T17:38:47.560222+0000","last_unstale":"2023-10-21T17:38:48.555361+0000","last_undegraded":"2023-10-21T17:38:48.555361+0000","last_fullsized":"2023-10-21T17:38:48.555361+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:13:20.771952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.548438+0000","last_change":"2023-10-21T17:38:33.354704+0000","last_active":"2023-10-21T17:38:47.548438+0000","last_peered":"2023-10-21T17:38:47.548438+0000","last_clean":"2023-10-21T17:38:47.548438+0000","last_became_active":"2023-10-21T17:38:33.354428+0000","last_became_peered":"2023-10-21T17:38:33.354428+0000","last_unstale":"2023-10-21T17:38:47.548438+0000","last_undegraded":"2023-10-21T17:38:47.548438+0000","last_fullsized":"2023-10-21T17:38:47.548438+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:09:42.944572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565430+0000","last_change":"2023-10-21T17:38:47.560143+0000","last_active":"2023-10-21T17:38:47.565430+0000","last_peered":"2023-10-21T17:38:47.565430+0000","last_clean":"2023-10-21T17:38:47.565430+0000","last_became_active":"2023-10-21T17:38:47.559565+0000","last_became_peered":"2023-10-21T17:38:47.559565+0000","last_unstale":"2023-10-21T17:38:47.565430+0000","last_undegraded":"2023-10-21T17:38:47.565430+0000","last_fullsized":"2023-10-21T17:38:47.565430+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:46:56.927972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529062+0000","last_change":"2023-10-21T17:38:33.349528+0000","last_active":"2023-10-21T17:38:47.529062+0000","last_peered":"2023-10-21T17:38:47.529062+0000","last_clean":"2023-10-21T17:38:47.529062+0000","last_became_active":"2023-10-21T17:38:33.349070+0000","last_became_peered":"2023-10-21T17:38:33.349070+0000","last_unstale":"2023-10-21T17:38:47.529062+0000","last_undegraded":"2023-10-21T17:38:47.529062+0000","last_fullsized":"2023-10-21T17:38:47.529062+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:10.470691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556409+0000","last_change":"2023-10-21T17:38:47.553702+0000","last_active":"2023-10-21T17:38:47.556409+0000","last_peered":"2023-10-21T17:38:47.556409+0000","last_clean":"2023-10-21T17:38:47.556409+0000","last_became_active":"2023-10-21T17:38:47.553343+0000","last_became_peered":"2023-10-21T17:38:47.553343+0000","last_unstale":"2023-10-21T17:38:47.556409+0000","last_undegraded":"2023-10-21T17:38:47.556409+0000","last_fullsized":"2023-10-21T17:38:47.556409+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:35:50.772804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535231+0000","last_change":"2023-10-21T17:38:33.369586+0000","last_active":"2023-10-21T17:38:46.535231+0000","last_peered":"2023-10-21T17:38:46.535231+0000","last_clean":"2023-10-21T17:38:46.535231+0000","last_became_active":"2023-10-21T17:38:33.369208+0000","last_became_peered":"2023-10-21T17:38:33.369208+0000","last_unstale":"2023-10-21T17:38:46.535231+0000","last_undegraded":"2023-10-21T17:38:46.535231+0000","last_fullsized":"2023-10-21T17:38:46.535231+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:43:04.246844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557913+0000","last_change":"2023-10-21T17:38:47.563623+0000","last_active":"2023-10-21T17:38:48.557913+0000","last_peered":"2023-10-21T17:38:48.557913+0000","last_clean":"2023-10-21T17:38:48.557913+0000","last_became_active":"2023-10-21T17:38:47.562191+0000","last_became_peered":"2023-10-21T17:38:47.562191+0000","last_unstale":"2023-10-21T17:38:48.557913+0000","last_undegraded":"2023-10-21T17:38:48.557913+0000","last_fullsized":"2023-10-21T17:38:48.557913+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:54:16.894163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534573+0000","last_change":"2023-10-21T17:38:33.362478+0000","last_active":"2023-10-21T17:38:46.534573+0000","last_peered":"2023-10-21T17:38:46.534573+0000","last_clean":"2023-10-21T17:38:46.534573+0000","last_became_active":"2023-10-21T17:38:33.361294+0000","last_became_peered":"2023-10-21T17:38:33.361294+0000","last_unstale":"2023-10-21T17:38:46.534573+0000","last_undegraded":"2023-10-21T17:38:46.534573+0000","last_fullsized":"2023-10-21T17:38:46.534573+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:25.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561640+0000","last_change":"2023-10-21T17:38:47.560964+0000","last_active":"2023-10-21T17:38:47.561640+0000","last_peered":"2023-10-21T17:38:47.561640+0000","last_clean":"2023-10-21T17:38:47.561640+0000","last_became_active":"2023-10-21T17:38:47.560634+0000","last_became_peered":"2023-10-21T17:38:47.560634+0000","last_unstale":"2023-10-21T17:38:47.561640+0000","last_undegraded":"2023-10-21T17:38:47.561640+0000","last_fullsized":"2023-10-21T17:38:47.561640+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:02.913327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541559+0000","last_change":"2023-10-21T17:38:33.352394+0000","last_active":"2023-10-21T17:38:47.541559+0000","last_peered":"2023-10-21T17:38:47.541559+0000","last_clean":"2023-10-21T17:38:47.541559+0000","last_became_active":"2023-10-21T17:38:33.352048+0000","last_became_peered":"2023-10-21T17:38:33.352048+0000","last_unstale":"2023-10-21T17:38:47.541559+0000","last_undegraded":"2023-10-21T17:38:47.541559+0000","last_fullsized":"2023-10-21T17:38:47.541559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:06:20.548414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557864+0000","last_change":"2023-10-21T17:38:47.548715+0000","last_active":"2023-10-21T17:38:47.557864+0000","last_peered":"2023-10-21T17:38:47.557864+0000","last_clean":"2023-10-21T17:38:47.557864+0000","last_became_active":"2023-10-21T17:38:47.548430+0000","last_became_peered":"2023-10-21T17:38:47.548430+0000","last_unstale":"2023-10-21T17:38:47.557864+0000","last_undegraded":"2023-10-21T17:38:47.557864+0000","last_fullsized":"2023-10-21T17:38:47.557864+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:35:25.089467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.530506+0000","last_change":"2023-10-21T17:38:33.346563+0000","last_active":"2023-10-21T17:38:47.530506+0000","last_peered":"2023-10-21T17:38:47.530506+0000","last_clean":"2023-10-21T17:38:47.530506+0000","last_became_active":"2023-10-21T17:38:33.343191+0000","last_became_peered":"2023-10-21T17:38:33.343191+0000","last_unstale":"2023-10-21T17:38:47.530506+0000","last_undegraded":"2023-10-21T17:38:47.530506+0000","last_fullsized":"2023-10-21T17:38:47.530506+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:36.794962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.570949+0000","last_change":"2023-10-21T17:38:47.569111+0000","last_active":"2023-10-21T17:38:47.570949+0000","last_peered":"2023-10-21T17:38:47.570949+0000","last_clean":"2023-10-21T17:38:47.570949+0000","last_became_active":"2023-10-21T17:38:47.568807+0000","last_became_peered":"2023-10-21T17:38:47.568807+0000","last_unstale":"2023-10-21T17:38:47.570949+0000","last_undegraded":"2023-10-21T17:38:47.570949+0000","last_fullsized":"2023-10-21T17:38:47.570949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:40:16.895474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534426+0000","last_change":"2023-10-21T17:38:33.355761+0000","last_active":"2023-10-21T17:38:46.534426+0000","last_peered":"2023-10-21T17:38:46.534426+0000","last_clean":"2023-10-21T17:38:46.534426+0000","last_became_active":"2023-10-21T17:38:33.355480+0000","last_became_peered":"2023-10-21T17:38:33.355480+0000","last_unstale":"2023-10-21T17:38:46.534426+0000","last_undegraded":"2023-10-21T17:38:46.534426+0000","last_fullsized":"2023-10-21T17:38:46.534426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:20:12.290129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565242+0000","last_change":"2023-10-21T17:38:47.560623+0000","last_active":"2023-10-21T17:38:47.565242+0000","last_peered":"2023-10-21T17:38:47.565242+0000","last_clean":"2023-10-21T17:38:47.565242+0000","last_became_active":"2023-10-21T17:38:47.558869+0000","last_became_peered":"2023-10-21T17:38:47.558869+0000","last_unstale":"2023-10-21T17:38:47.565242+0000","last_undegraded":"2023-10-21T17:38:47.565242+0000","last_fullsized":"2023-10-21T17:38:47.565242+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:30:48.745090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542268+0000","last_change":"2023-10-21T17:38:33.366191+0000","last_active":"2023-10-21T17:38:47.542268+0000","last_peered":"2023-10-21T17:38:47.542268+0000","last_clean":"2023-10-21T17:38:47.542268+0000","last_became_active":"2023-10-21T17:38:33.363694+0000","last_became_peered":"2023-10-21T17:38:33.363694+0000","last_unstale":"2023-10-21T17:38:47.542268+0000","last_undegraded":"2023-10-21T17:38:47.542268+0000","last_fullsized":"2023-10-21T17:38:47.542268+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:47:12.488140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555365+0000","last_change":"2023-10-21T17:38:47.555523+0000","last_active":"2023-10-21T17:38:48.555365+0000","last_peered":"2023-10-21T17:38:48.555365+0000","last_clean":"2023-10-21T17:38:48.555365+0000","last_became_active":"2023-10-21T17:38:47.555224+0000","last_became_peered":"2023-10-21T17:38:47.555224+0000","last_unstale":"2023-10-21T17:38:48.555365+0000","last_undegraded":"2023-10-21T17:38:48.555365+0000","last_fullsized":"2023-10-21T17:38:48.555365+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:25:09.807236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.545471+0000","last_change":"2023-10-21T17:38:33.367227+0000","last_active":"2023-10-21T17:38:47.545471+0000","last_peered":"2023-10-21T17:38:47.545471+0000","last_clean":"2023-10-21T17:38:47.545471+0000","last_became_active":"2023-10-21T17:38:33.366922+0000","last_became_peered":"2023-10-21T17:38:33.366922+0000","last_unstale":"2023-10-21T17:38:47.545471+0000","last_undegraded":"2023-10-21T17:38:47.545471+0000","last_fullsized":"2023-10-21T17:38:47.545471+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:33:14.663384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556512+0000","last_change":"2023-10-21T17:38:47.552457+0000","last_active":"2023-10-21T17:38:47.556512+0000","last_peered":"2023-10-21T17:38:47.556512+0000","last_clean":"2023-10-21T17:38:47.556512+0000","last_became_active":"2023-10-21T17:38:47.551709+0000","last_became_peered":"2023-10-21T17:38:47.551709+0000","last_unstale":"2023-10-21T17:38:47.556512+0000","last_undegraded":"2023-10-21T17:38:47.556512+0000","last_fullsized":"2023-10-21T17:38:47.556512+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:28:38.788080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.553288+0000","last_change":"2023-10-21T17:38:33.363852+0000","last_active":"2023-10-21T17:38:47.553288+0000","last_peered":"2023-10-21T17:38:47.553288+0000","last_clean":"2023-10-21T17:38:47.553288+0000","last_became_active":"2023-10-21T17:38:33.363427+0000","last_became_peered":"2023-10-21T17:38:33.363427+0000","last_unstale":"2023-10-21T17:38:47.553288+0000","last_undegraded":"2023-10-21T17:38:47.553288+0000","last_fullsized":"2023-10-21T17:38:47.553288+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:33:39.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.558963+0000","last_change":"2023-10-21T17:38:47.558369+0000","last_active":"2023-10-21T17:38:48.558963+0000","last_peered":"2023-10-21T17:38:48.558963+0000","last_clean":"2023-10-21T17:38:48.558963+0000","last_became_active":"2023-10-21T17:38:47.558058+0000","last_became_peered":"2023-10-21T17:38:47.558058+0000","last_unstale":"2023-10-21T17:38:48.558963+0000","last_undegraded":"2023-10-21T17:38:48.558963+0000","last_fullsized":"2023-10-21T17:38:48.558963+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:06.514890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.544656+0000","last_change":"2023-10-21T17:38:33.366914+0000","last_active":"2023-10-21T17:38:47.544656+0000","last_peered":"2023-10-21T17:38:47.544656+0000","last_clean":"2023-10-21T17:38:47.544656+0000","last_became_active":"2023-10-21T17:38:33.366337+0000","last_became_peered":"2023-10-21T17:38:33.366337+0000","last_unstale":"2023-10-21T17:38:47.544656+0000","last_undegraded":"2023-10-21T17:38:47.544656+0000","last_fullsized":"2023-10-21T17:38:47.544656+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:15:25.780715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.566282+0000","last_change":"2023-10-21T17:38:47.560566+0000","last_active":"2023-10-21T17:38:47.566282+0000","last_peered":"2023-10-21T17:38:47.566282+0000","last_clean":"2023-10-21T17:38:47.566282+0000","last_became_active":"2023-10-21T17:38:47.559800+0000","last_became_peered":"2023-10-21T17:38:47.559800+0000","last_unstale":"2023-10-21T17:38:47.566282+0000","last_undegraded":"2023-10-21T17:38:47.566282+0000","last_fullsized":"2023-10-21T17:38:47.566282+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:23:34.538156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550826+0000","last_change":"2023-10-21T17:38:33.351005+0000","last_active":"2023-10-21T17:38:47.550826+0000","last_peered":"2023-10-21T17:38:47.550826+0000","last_clean":"2023-10-21T17:38:47.550826+0000","last_became_active":"2023-10-21T17:38:33.350305+0000","last_became_peered":"2023-10-21T17:38:33.350305+0000","last_unstale":"2023-10-21T17:38:47.550826+0000","last_undegraded":"2023-10-21T17:38:47.550826+0000","last_fullsized":"2023-10-21T17:38:47.550826+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:58:28.150954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.565314+0000","last_change":"2023-10-21T17:38:47.556976+0000","last_active":"2023-10-21T17:38:47.565314+0000","last_peered":"2023-10-21T17:38:47.565314+0000","last_clean":"2023-10-21T17:38:47.565314+0000","last_became_active":"2023-10-21T17:38:47.556701+0000","last_became_peered":"2023-10-21T17:38:47.556701+0000","last_unstale":"2023-10-21T17:38:47.565314+0000","last_undegraded":"2023-10-21T17:38:47.565314+0000","last_fullsized":"2023-10-21T17:38:47.565314+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:18.225085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541455+0000","last_change":"2023-10-21T17:38:33.360922+0000","last_active":"2023-10-21T17:38:47.541455+0000","last_peered":"2023-10-21T17:38:47.541455+0000","last_clean":"2023-10-21T17:38:47.541455+0000","last_became_active":"2023-10-21T17:38:33.360628+0000","last_became_peered":"2023-10-21T17:38:33.360628+0000","last_unstale":"2023-10-21T17:38:47.541455+0000","last_undegraded":"2023-10-21T17:38:47.541455+0000","last_fullsized":"2023-10-21T17:38:47.541455+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:11:01.407018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557107+0000","last_change":"2023-10-21T17:38:47.554990+0000","last_active":"2023-10-21T17:38:48.557107+0000","last_peered":"2023-10-21T17:38:48.557107+0000","last_clean":"2023-10-21T17:38:48.557107+0000","last_became_active":"2023-10-21T17:38:47.554716+0000","last_became_peered":"2023-10-21T17:38:47.554716+0000","last_unstale":"2023-10-21T17:38:48.557107+0000","last_undegraded":"2023-10-21T17:38:48.557107+0000","last_fullsized":"2023-10-21T17:38:48.557107+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:05.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.531752+0000","last_change":"2023-10-21T17:38:33.344462+0000","last_active":"2023-10-21T17:38:46.531752+0000","last_peered":"2023-10-21T17:38:46.531752+0000","last_clean":"2023-10-21T17:38:46.531752+0000","last_became_active":"2023-10-21T17:38:33.344064+0000","last_became_peered":"2023-10-21T17:38:33.344064+0000","last_unstale":"2023-10-21T17:38:46.531752+0000","last_undegraded":"2023-10-21T17:38:46.531752+0000","last_fullsized":"2023-10-21T17:38:46.531752+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:09:37.232213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550228+0000","last_change":"2023-10-21T17:38:47.544335+0000","last_active":"2023-10-21T17:38:47.550228+0000","last_peered":"2023-10-21T17:38:47.550228+0000","last_clean":"2023-10-21T17:38:47.550228+0000","last_became_active":"2023-10-21T17:38:47.543954+0000","last_became_peered":"2023-10-21T17:38:47.543954+0000","last_unstale":"2023-10-21T17:38:47.550228+0000","last_undegraded":"2023-10-21T17:38:47.550228+0000","last_fullsized":"2023-10-21T17:38:47.550228+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:28:23.084608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.547688+0000","last_change":"2023-10-21T17:38:33.360718+0000","last_active":"2023-10-21T17:38:46.547688+0000","last_peered":"2023-10-21T17:38:46.547688+0000","last_clean":"2023-10-21T17:38:46.547688+0000","last_became_active":"2023-10-21T17:38:33.359530+0000","last_became_peered":"2023-10-21T17:38:33.359530+0000","last_unstale":"2023-10-21T17:38:46.547688+0000","last_undegraded":"2023-10-21T17:38:46.547688+0000","last_fullsized":"2023-10-21T17:38:46.547688+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:04.002929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.549540+0000","last_change":"2023-10-21T17:38:47.536233+0000","last_active":"2023-10-21T17:38:47.549540+0000","last_peered":"2023-10-21T17:38:47.549540+0000","last_clean":"2023-10-21T17:38:47.549540+0000","last_became_active":"2023-10-21T17:38:47.534706+0000","last_became_peered":"2023-10-21T17:38:47.534706+0000","last_unstale":"2023-10-21T17:38:47.549540+0000","last_undegraded":"2023-10-21T17:38:47.549540+0000","last_fullsized":"2023-10-21T17:38:47.549540+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:32:24.782875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.542162+0000","last_change":"2023-10-21T17:38:33.366456+0000","last_active":"2023-10-21T17:38:47.542162+0000","last_peered":"2023-10-21T17:38:47.542162+0000","last_clean":"2023-10-21T17:38:47.542162+0000","last_became_active":"2023-10-21T17:38:33.366015+0000","last_became_peered":"2023-10-21T17:38:33.366015+0000","last_unstale":"2023-10-21T17:38:47.542162+0000","last_undegraded":"2023-10-21T17:38:47.542162+0000","last_fullsized":"2023-10-21T17:38:47.542162+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:46:28.700422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.559847+0000","last_change":"2023-10-21T17:38:47.555243+0000","last_active":"2023-10-21T17:38:47.559847+0000","last_peered":"2023-10-21T17:38:47.559847+0000","last_clean":"2023-10-21T17:38:47.559847+0000","last_became_active":"2023-10-21T17:38:47.554976+0000","last_became_peered":"2023-10-21T17:38:47.554976+0000","last_unstale":"2023-10-21T17:38:47.559847+0000","last_undegraded":"2023-10-21T17:38:47.559847+0000","last_fullsized":"2023-10-21T17:38:47.559847+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:33:44.479903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.557628+0000","last_change":"2023-10-21T17:38:33.358688+0000","last_active":"2023-10-21T17:38:46.557628+0000","last_peered":"2023-10-21T17:38:46.557628+0000","last_clean":"2023-10-21T17:38:46.557628+0000","last_became_active":"2023-10-21T17:38:33.354936+0000","last_became_peered":"2023-10-21T17:38:33.354936+0000","last_unstale":"2023-10-21T17:38:46.557628+0000","last_undegraded":"2023-10-21T17:38:46.557628+0000","last_fullsized":"2023-10-21T17:38:46.557628+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:17:24.335865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556916+0000","last_change":"2023-10-21T17:38:47.552664+0000","last_active":"2023-10-21T17:38:47.556916+0000","last_peered":"2023-10-21T17:38:47.556916+0000","last_clean":"2023-10-21T17:38:47.556916+0000","last_became_active":"2023-10-21T17:38:47.552137+0000","last_became_peered":"2023-10-21T17:38:47.552137+0000","last_unstale":"2023-10-21T17:38:47.556916+0000","last_undegraded":"2023-10-21T17:38:47.556916+0000","last_fullsized":"2023-10-21T17:38:47.556916+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:44:19.967378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.528768+0000","last_change":"2023-10-21T17:38:33.349361+0000","last_active":"2023-10-21T17:38:47.528768+0000","last_peered":"2023-10-21T17:38:47.528768+0000","last_clean":"2023-10-21T17:38:47.528768+0000","last_became_active":"2023-10-21T17:38:33.348672+0000","last_became_peered":"2023-10-21T17:38:33.348672+0000","last_unstale":"2023-10-21T17:38:47.528768+0000","last_undegraded":"2023-10-21T17:38:47.528768+0000","last_fullsized":"2023-10-21T17:38:47.528768+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:37:12.429036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.555782+0000","last_change":"2023-10-21T17:38:47.561458+0000","last_active":"2023-10-21T17:38:48.555782+0000","last_peered":"2023-10-21T17:38:48.555782+0000","last_clean":"2023-10-21T17:38:48.555782+0000","last_became_active":"2023-10-21T17:38:47.561144+0000","last_became_peered":"2023-10-21T17:38:47.561144+0000","last_unstale":"2023-10-21T17:38:48.555782+0000","last_undegraded":"2023-10-21T17:38:48.555782+0000","last_fullsized":"2023-10-21T17:38:48.555782+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:25:19.338478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535141+0000","last_change":"2023-10-21T17:38:33.369555+0000","last_active":"2023-10-21T17:38:46.535141+0000","last_peered":"2023-10-21T17:38:46.535141+0000","last_clean":"2023-10-21T17:38:46.535141+0000","last_became_active":"2023-10-21T17:38:33.369212+0000","last_became_peered":"2023-10-21T17:38:33.369212+0000","last_unstale":"2023-10-21T17:38:46.535141+0000","last_undegraded":"2023-10-21T17:38:46.535141+0000","last_fullsized":"2023-10-21T17:38:46.535141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:14:29.955478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.557566+0000","last_change":"2023-10-21T17:38:47.556272+0000","last_active":"2023-10-21T17:38:48.557566+0000","last_peered":"2023-10-21T17:38:48.557566+0000","last_clean":"2023-10-21T17:38:48.557566+0000","last_became_active":"2023-10-21T17:38:47.555903+0000","last_became_peered":"2023-10-21T17:38:47.555903+0000","last_unstale":"2023-10-21T17:38:48.557566+0000","last_undegraded":"2023-10-21T17:38:48.557566+0000","last_fullsized":"2023-10-21T17:38:48.557566+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:28.828809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.552542+0000","last_change":"2023-10-21T17:38:33.368484+0000","last_active":"2023-10-21T17:38:47.552542+0000","last_peered":"2023-10-21T17:38:47.552542+0000","last_clean":"2023-10-21T17:38:47.552542+0000","last_became_active":"2023-10-21T17:38:33.366828+0000","last_became_peered":"2023-10-21T17:38:33.366828+0000","last_unstale":"2023-10-21T17:38:47.552542+0000","last_undegraded":"2023-10-21T17:38:47.552542+0000","last_fullsized":"2023-10-21T17:38:47.552542+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:04.935450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556631+0000","last_change":"2023-10-21T17:38:47.552808+0000","last_active":"2023-10-21T17:38:47.556631+0000","last_peered":"2023-10-21T17:38:47.556631+0000","last_clean":"2023-10-21T17:38:47.556631+0000","last_became_active":"2023-10-21T17:38:47.552549+0000","last_became_peered":"2023-10-21T17:38:47.552549+0000","last_unstale":"2023-10-21T17:38:47.556631+0000","last_undegraded":"2023-10-21T17:38:47.556631+0000","last_fullsized":"2023-10-21T17:38:47.556631+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:25:19.338478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541364+0000","last_change":"2023-10-21T17:38:33.363542+0000","last_active":"2023-10-21T17:38:47.541364+0000","last_peered":"2023-10-21T17:38:47.541364+0000","last_clean":"2023-10-21T17:38:47.541364+0000","last_became_active":"2023-10-21T17:38:33.363143+0000","last_became_peered":"2023-10-21T17:38:33.363143+0000","last_unstale":"2023-10-21T17:38:47.541364+0000","last_undegraded":"2023-10-21T17:38:47.541364+0000","last_fullsized":"2023-10-21T17:38:47.541364+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:31.552025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.550344+0000","last_change":"2023-10-21T17:38:47.547082+0000","last_active":"2023-10-21T17:38:47.550344+0000","last_peered":"2023-10-21T17:38:47.550344+0000","last_clean":"2023-10-21T17:38:47.550344+0000","last_became_active":"2023-10-21T17:38:47.546777+0000","last_became_peered":"2023-10-21T17:38:47.546777+0000","last_unstale":"2023-10-21T17:38:47.550344+0000","last_undegraded":"2023-10-21T17:38:47.550344+0000","last_fullsized":"2023-10-21T17:38:47.550344+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:33:07.701278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.541229+0000","last_change":"2023-10-21T17:38:33.363315+0000","last_active":"2023-10-21T17:38:47.541229+0000","last_peered":"2023-10-21T17:38:47.541229+0000","last_clean":"2023-10-21T17:38:47.541229+0000","last_became_active":"2023-10-21T17:38:33.363007+0000","last_became_peered":"2023-10-21T17:38:33.363007+0000","last_unstale":"2023-10-21T17:38:47.541229+0000","last_undegraded":"2023-10-21T17:38:47.541229+0000","last_fullsized":"2023-10-21T17:38:47.541229+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:15:45.232763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.564612+0000","last_change":"2023-10-21T17:38:47.557619+0000","last_active":"2023-10-21T17:38:47.564612+0000","last_peered":"2023-10-21T17:38:47.564612+0000","last_clean":"2023-10-21T17:38:47.564612+0000","last_became_active":"2023-10-21T17:38:47.556520+0000","last_became_peered":"2023-10-21T17:38:47.556520+0000","last_unstale":"2023-10-21T17:38:47.564612+0000","last_undegraded":"2023-10-21T17:38:47.564612+0000","last_fullsized":"2023-10-21T17:38:47.564612+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:03:35.356779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.528593+0000","last_change":"2023-10-21T17:38:33.352800+0000","last_active":"2023-10-21T17:38:47.528593+0000","last_peered":"2023-10-21T17:38:47.528593+0000","last_clean":"2023-10-21T17:38:47.528593+0000","last_became_active":"2023-10-21T17:38:33.351990+0000","last_became_peered":"2023-10-21T17:38:33.351990+0000","last_unstale":"2023-10-21T17:38:47.528593+0000","last_undegraded":"2023-10-21T17:38:47.528593+0000","last_fullsized":"2023-10-21T17:38:47.528593+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:00:05.602488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.549558+0000","last_change":"2023-10-21T17:38:47.537064+0000","last_active":"2023-10-21T17:38:47.549558+0000","last_peered":"2023-10-21T17:38:47.549558+0000","last_clean":"2023-10-21T17:38:47.549558+0000","last_became_active":"2023-10-21T17:38:47.536311+0000","last_became_peered":"2023-10-21T17:38:47.536311+0000","last_unstale":"2023-10-21T17:38:47.549558+0000","last_undegraded":"2023-10-21T17:38:47.549558+0000","last_fullsized":"2023-10-21T17:38:47.549558+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:13:11.920952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.554203+0000","last_change":"2023-10-21T17:38:33.351557+0000","last_active":"2023-10-21T17:38:46.554203+0000","last_peered":"2023-10-21T17:38:46.554203+0000","last_clean":"2023-10-21T17:38:46.554203+0000","last_became_active":"2023-10-21T17:38:33.351230+0000","last_became_peered":"2023-10-21T17:38:33.351230+0000","last_unstale":"2023-10-21T17:38:46.554203+0000","last_undegraded":"2023-10-21T17:38:46.554203+0000","last_fullsized":"2023-10-21T17:38:46.554203+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:20:01.763000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.557728+0000","last_change":"2023-10-21T17:38:47.551152+0000","last_active":"2023-10-21T17:38:47.557728+0000","last_peered":"2023-10-21T17:38:47.557728+0000","last_clean":"2023-10-21T17:38:47.557728+0000","last_became_active":"2023-10-21T17:38:47.550821+0000","last_became_peered":"2023-10-21T17:38:47.550821+0000","last_unstale":"2023-10-21T17:38:47.557728+0000","last_undegraded":"2023-10-21T17:38:47.557728+0000","last_fullsized":"2023-10-21T17:38:47.557728+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:46:49.083823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.529578+0000","last_change":"2023-10-21T17:38:33.354002+0000","last_active":"2023-10-21T17:38:47.529578+0000","last_peered":"2023-10-21T17:38:47.529578+0000","last_clean":"2023-10-21T17:38:47.529578+0000","last_became_active":"2023-10-21T17:38:33.353709+0000","last_became_peered":"2023-10-21T17:38:33.353709+0000","last_unstale":"2023-10-21T17:38:47.529578+0000","last_undegraded":"2023-10-21T17:38:47.529578+0000","last_fullsized":"2023-10-21T17:38:47.529578+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:13.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-21T17:38:48.553150+0000","last_change":"2023-10-21T17:38:47.568080+0000","last_active":"2023-10-21T17:38:48.553150+0000","last_peered":"2023-10-21T17:38:48.553150+0000","last_clean":"2023-10-21T17:38:48.553150+0000","last_became_active":"2023-10-21T17:38:47.567685+0000","last_became_peered":"2023-10-21T17:38:47.567685+0000","last_unstale":"2023-10-21T17:38:48.553150+0000","last_undegraded":"2023-10-21T17:38:48.553150+0000","last_fullsized":"2023-10-21T17:38:48.553150+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:36.723492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.558999+0000","last_change":"2023-10-21T17:38:33.359316+0000","last_active":"2023-10-21T17:38:46.558999+0000","last_peered":"2023-10-21T17:38:46.558999+0000","last_clean":"2023-10-21T17:38:46.558999+0000","last_became_active":"2023-10-21T17:38:33.356311+0000","last_became_peered":"2023-10-21T17:38:33.356311+0000","last_unstale":"2023-10-21T17:38:46.558999+0000","last_undegraded":"2023-10-21T17:38:46.558999+0000","last_fullsized":"2023-10-21T17:38:46.558999+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:10.002729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.556673+0000","last_change":"2023-10-21T17:38:47.554019+0000","last_active":"2023-10-21T17:38:47.556673+0000","last_peered":"2023-10-21T17:38:47.556673+0000","last_clean":"2023-10-21T17:38:47.556673+0000","last_became_active":"2023-10-21T17:38:47.553725+0000","last_became_peered":"2023-10-21T17:38:47.553725+0000","last_unstale":"2023-10-21T17:38:47.556673+0000","last_undegraded":"2023-10-21T17:38:47.556673+0000","last_fullsized":"2023-10-21T17:38:47.556673+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:39:36.444107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535049+0000","last_change":"2023-10-21T17:38:33.354797+0000","last_active":"2023-10-21T17:38:46.535049+0000","last_peered":"2023-10-21T17:38:46.535049+0000","last_clean":"2023-10-21T17:38:46.535049+0000","last_became_active":"2023-10-21T17:38:33.354503+0000","last_became_peered":"2023-10-21T17:38:33.354503+0000","last_unstale":"2023-10-21T17:38:46.535049+0000","last_undegraded":"2023-10-21T17:38:46.535049+0000","last_fullsized":"2023-10-21T17:38:46.535049+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:03:21.925672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.539797+0000","last_change":"2023-10-21T17:38:47.537503+0000","last_active":"2023-10-21T17:38:47.539797+0000","last_peered":"2023-10-21T17:38:47.539797+0000","last_clean":"2023-10-21T17:38:47.539797+0000","last_became_active":"2023-10-21T17:38:47.536825+0000","last_became_peered":"2023-10-21T17:38:47.536825+0000","last_unstale":"2023-10-21T17:38:47.539797+0000","last_undegraded":"2023-10-21T17:38:47.539797+0000","last_fullsized":"2023-10-21T17:38:47.539797+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:48:25.196265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.535171+0000","last_change":"2023-10-21T17:38:33.348939+0000","last_active":"2023-10-21T17:38:46.535171+0000","last_peered":"2023-10-21T17:38:46.535171+0000","last_clean":"2023-10-21T17:38:46.535171+0000","last_became_active":"2023-10-21T17:38:33.348561+0000","last_became_peered":"2023-10-21T17:38:33.348561+0000","last_unstale":"2023-10-21T17:38:46.535171+0000","last_undegraded":"2023-10-21T17:38:46.535171+0000","last_fullsized":"2023-10-21T17:38:46.535171+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:14.527583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-21T17:38:47.561983+0000","last_change":"2023-10-21T17:38:47.558765+0000","last_active":"2023-10-21T17:38:47.561983+0000","last_peered":"2023-10-21T17:38:47.561983+0000","last_clean":"2023-10-21T17:38:47.561983+0000","last_became_active":"2023-10-21T17:38:47.558412+0000","last_became_peered":"2023-10-21T17:38:47.558412+0000","last_unstale":"2023-10-21T17:38:47.561983+0000","last_undegraded":"2023-10-21T17:38:47.561983+0000","last_fullsized":"2023-10-21T17:38:47.561983+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:46.507617+0000","last_clean_scrub_stamp":"2023-10-21T17:38:46.507617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:57:43.878002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-21T17:38:46.534041+0000","last_change":"2023-10-21T17:38:33.368541+0000","last_active":"2023-10-21T17:38:46.534041+0000","last_peered":"2023-10-21T17:38:46.534041+0000","last_clean":"2023-10-21T17:38:46.534041+0000","last_became_active":"2023-10-21T17:38:33.368186+0000","last_became_peered":"2023-10-21T17:38:33.368186+0000","last_unstale":"2023-10-21T17:38:46.534041+0000","last_undegraded":"2023-10-21T17:38:46.534041+0000","last_fullsized":"2023-10-21T17:38:46.534041+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:38:32.370648+0000","last_clean_scrub_stamp":"2023-10-21T17:38:32.370648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:10.002729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2224,"num_read_kb":4639,"num_write":587,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4873,"ondisk_log_size":4873,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576000,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74936,"kb_used_data":3496,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296904,"statfs":{"total":96636764160,"available":96560029696,"internally_reserved":0,"allocated":3579904,"data_stored":2131653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.97399999999999998,"5min":0.73399999999999999,"15min":0.72299999999999998},"max":{"1min":1.6419999999999999,"5min":1.841,"15min":2.2549999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.101,"15min":1.1599999999999999},"min":{"1min":0.64300000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.538,"5min":1.724,"15min":1.7669999999999999},"last":2.6160000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1739999999999999,"15min":1.218},"min":{"1min":0.96299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.589,"5min":1.7170000000000001,"15min":1.73},"last":1.615},{"interface":"front","average":{"1min":1.262,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.91200000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.506,"5min":1.792,"15min":1.885},"last":1.411}]},{"osd":2,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.3600000000000001,"15min":1.341},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":5.3360000000000003},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3069999999999999,"15min":1.321},"min":{"1min":0.86899999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":3.556}]},{"osd":3,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2909999999999999,"15min":1.3180000000000001},"min":{"1min":0.998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.472,"5min":2.472,"15min":2.5110000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.486,"15min":1.4670000000000001},"min":{"1min":1.23,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":4.9249999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.383,"15min":1.357},"min":{"1min":0.84599999999999997,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.536,"5min":1.5129999999999999,"15min":1.496},"min":{"1min":1.1619999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.6749999999999998,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.5649999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.173,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.474,"15min":1.45},"min":{"1min":1.135,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Oct 21 17:38:06 2023","interfaces":[{"interface":"back","average":{"1min":1.746,"5min":1.587,"15min":1.5469999999999999},"min":{"1min":0.85499999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":3.3100000000000001,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.27},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.488,"15min":1.476},"min":{"1min":0.88800000000000001,"5min":0.628,"15min":0.628},"max":{"1min":2.879,"5min":2.879,"15min":2.879},"last":1.919}]}]},{"osd":6,"up_from":13,"seq":55834576001,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42440,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329400,"statfs":{"total":96636764160,"available":96593305600,"internally_reserved":0,"allocated":3002368,"data_stored":1542863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1659999999999999,"15min":1.198},"min":{"1min":0.89300000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.728,"5min":1.944,"15min":1.944},"last":1.671},{"interface":"front","average":{"1min":1.256,"5min":1.2050000000000001,"15min":1.222},"min":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.643,"5min":1.7090000000000001,"15min":1.762},"last":1.05}]},{"osd":1,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.171,"15min":1.1919999999999999},"min":{"1min":0.84099999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6379999999999999},"last":0.879},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2989999999999999,"15min":1.319},"min":{"1min":0.98599999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.782,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.1499999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.272,"15min":1.3169999999999999},"min":{"1min":1.006,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5649999999999999,"5min":1.712,"15min":1.8300000000000001},"last":0.999},{"interface":"front","average":{"1min":1.393,"5min":1.3089999999999999,"15min":1.3069999999999999},"min":{"1min":1.1799999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.611,"5min":1.679,"15min":1.929},"last":1.1919999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3919999999999999,"15min":1.417},"min":{"1min":0.90700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.593,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.339},{"interface":"front","average":{"1min":1.468,"5min":1.427,"15min":1.4650000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.90500000000000003},"max":{"1min":1.825,"5min":1.9039999999999999,"15min":2.048},"last":1.2370000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":1.704,"5min":1.77,"15min":1.9950000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.4079999999999999,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6990000000000001,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.806}]},{"osd":5,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.5349999999999999,"15min":1.571},"min":{"1min":1.274,"5min":1.1000000000000001,"15min":1.0029999999999999},"max":{"1min":1.891,"5min":1.956,"15min":2.339},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.325,"15min":1.3759999999999999},"min":{"1min":0.98199999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6870000000000001,"5min":1.796,"15min":2.2890000000000001},"last":1.7470000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:37:57 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4970000000000001,"15min":1.536},"min":{"1min":1.1080000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.762,"5min":1.9339999999999999,"15min":2.1200000000000001},"last":1.508},{"interface":"front","average":{"1min":1.641,"5min":1.6259999999999999,"15min":1.6679999999999999},"min":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.0449999999999999},"max":{"1min":1.9570000000000001,"5min":2.1499999999999999,"15min":2.5409999999999999},"last":1.9359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576002,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60468,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311372,"statfs":{"total":96636764160,"available":96574844928,"internally_reserved":0,"allocated":2985984,"data_stored":1539424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1160000000000001,"15min":1.1539999999999999},"min":{"1min":0.81599999999999995,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6319999999999999,"5min":1.6799999999999999,"15min":1.9930000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.232},"min":{"1min":0.84999999999999998,"5min":0.64200000000000002,"15min":0.627},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2829999999999999,"15min":1.284},"min":{"1min":1.012,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.738,"5min":1.806,"15min":1.8620000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.202,"15min":1.2330000000000001},"min":{"1min":0.88500000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.75,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.81499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.2829999999999999,"15min":1.302},"min":{"1min":1.087,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.802},"last":1.238},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.24,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.5640000000000001,"5min":1.8600000000000001,"15min":1.891},"last":0.95899999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.325,"15min":1.3560000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.732,"5min":1.829,"15min":1.9650000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.4079999999999999,"15min":1.4199999999999999},"min":{"1min":1.038,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.772,"5min":1.881,"15min":1.9139999999999999},"last":1.3380000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2230000000000001,"15min":1.25},"min":{"1min":0.91000000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.462,"5min":1.7050000000000001,"15min":1.9910000000000001},"last":1.173},{"interface":"front","average":{"1min":1.46,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":1.1259999999999999,"5min":0.85399999999999998,"15min":0.80000000000000004},"max":{"1min":1.8400000000000001,"5min":1.877,"15min":1.9399999999999999},"last":1.464}]},{"osd":6,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.482},"min":{"1min":1.081,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8540000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.508,"15min":1.524},"min":{"1min":0.84999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7509999999999999,"5min":1.9730000000000001,"15min":2.52},"last":1.395}]},{"osd":7,"last update":"Sat Oct 21 17:37:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5309999999999999,"15min":1.5329999999999999},"min":{"1min":1.1319999999999999,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":1.96,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.573},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.544,"15min":1.5640000000000001},"min":{"1min":1.018,"5min":0.68400000000000005,"15min":0.56399999999999995},"max":{"1min":1.833,"5min":2.0750000000000002,"15min":2.1349999999999998},"last":1.506}]}]},{"osd":4,"up_from":13,"seq":55834576000,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47972,"kb_used_data":2900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323868,"statfs":{"total":96636764160,"available":96587640832,"internally_reserved":0,"allocated":2969600,"data_stored":1538358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.117,"15min":1.1599999999999999},"min":{"1min":0.72299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.8480000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.218,"15min":1.2170000000000001},"min":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.659,"5min":1.855,"15min":1.869},"last":1.268}]},{"osd":1,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.2350000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.58099999999999996},"max":{"1min":1.4890000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.248,"15min":1.268},"min":{"1min":0.93300000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.665,"5min":1.762,"15min":1.762},"last":0.91400000000000003}]},{"osd":2,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2649999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.6000000000000001,"5min":1.71,"15min":1.7609999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3620000000000001,"15min":1.387},"min":{"1min":1.0900000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.5740000000000001,"5min":1.762,"15min":1.762},"last":1.5049999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2729999999999999,"15min":1.3200000000000001},"min":{"1min":0.92100000000000004,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.4870000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.419,"15min":1.425},"min":{"1min":1.002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7350000000000001,"5min":1.825,"15min":1.9239999999999999},"last":1.1970000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.82699999999999996},"max":{"1min":1.639,"5min":1.8340000000000001,"15min":1.867},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.98799999999999999,"5min":0.879,"15min":0.81000000000000005},"max":{"1min":1.6990000000000001,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4139999999999999,"15min":1.4430000000000001},"min":{"1min":0.88300000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.825,"5min":1.9810000000000001,"15min":2.1600000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.514,"5min":1.595,"15min":1.595},"min":{"1min":1.147,"5min":1.147,"15min":1.133},"max":{"1min":1.7210000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.7470000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.534,"15min":1.5189999999999999},"min":{"1min":1.2090000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.833,"5min":2.234,"15min":2.234},"last":1.151},{"interface":"front","average":{"1min":1.55,"5min":1.5800000000000001,"15min":1.5800000000000001},"min":{"1min":1.0880000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"max":{"1min":1.9279999999999999,"5min":1.984,"15min":2.081},"last":1.6619999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56304,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315536,"statfs":{"total":96636764160,"available":96579108864,"internally_reserved":0,"allocated":2977792,"data_stored":1537834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.97599999999999998,"15min":0.98399999999999999},"min":{"1min":0.54400000000000004,"5min":0.48899999999999999,"15min":0.432},"max":{"1min":1.4550000000000001,"5min":1.6439999999999999,"15min":1.71},"last":1.107},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.258,"15min":1.2629999999999999},"min":{"1min":0.73499999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.637,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2010000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0409999999999999,"15min":1.091},"min":{"1min":0.626,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3340000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.875,"5min":1.879,"15min":1.879},"last":1.3140000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1100000000000001,"15min":1.1160000000000001},"min":{"1min":0.59699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.921,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.23,"5min":1.25,"15min":1.2909999999999999},"min":{"1min":0.877,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.823,"5min":1.853,"15min":2},"last":1.145}]},{"osd":4,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4350000000000001,"15min":1.4410000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.905,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.397},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.444,"15min":1.4710000000000001},"min":{"1min":1.048,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.802,"5min":1.9430000000000001,"15min":2.169},"last":1.712}]},{"osd":5,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.89200000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9330000000000001,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.5389999999999999,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.1800000000000002,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.9350000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.546,"15min":1.601},"min":{"1min":1.0189999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.966,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.593,"15min":1.625},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0579999999999998,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.601}]},{"osd":7,"last update":"Sat Oct 21 17:37:55 2023","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.673,"15min":1.6739999999999999},"min":{"1min":1.135,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2789999999999999,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.659,"15min":1.714},"min":{"1min":1.0529999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1760000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":2.073}]}]},{"osd":2,"up_from":13,"seq":55834576001,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52144,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319696,"statfs":{"total":96636764160,"available":96583368704,"internally_reserved":0,"allocated":2985984,"data_stored":1545995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.90400000000000003},"min":{"1min":0.66100000000000003,"5min":0.63800000000000001,"15min":0.60599999999999998},"max":{"1min":1.149,"5min":1.841,"15min":1.841},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0289999999999999,"15min":1.0509999999999999},"min":{"1min":0.79500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.7529999999999999,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.78900000000000003,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6539999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.417,"15min":1.395},"min":{"1min":1.151,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.8300000000000001,"5min":1.958,"15min":2.4060000000000001},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.1970000000000001,"15min":1.198},"min":{"1min":0.78400000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.1760000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.468,"15min":1.4910000000000001},"min":{"1min":0.92900000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":2.077},"last":1.0209999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":1.1870000000000001,"5min":1.0529999999999999,"15min":1.044},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.1680000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.522,"5min":1.468,"15min":1.46},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0329999999999999},"max":{"1min":1.9079999999999999,"5min":1.978,"15min":2.5939999999999999},"last":1.0389999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4279999999999999,"15min":1.4199999999999999},"min":{"1min":1.0800000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.169},{"interface":"front","average":{"1min":1.615,"5min":1.6000000000000001,"15min":1.6120000000000001},"min":{"1min":1.286,"5min":1.1519999999999999,"15min":1.1519999999999999},"max":{"1min":1.9710000000000001,"5min":1.972,"15min":2.3759999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.571,"15min":1.575},"min":{"1min":1.145,"5min":1.1060000000000001,"15min":1.095},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.595,"5min":1.5349999999999999,"15min":1.5289999999999999},"min":{"1min":1.115,"5min":1.0309999999999999,"15min":1.0309999999999999},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.0880000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:38:02 2023","interfaces":[{"interface":"back","average":{"1min":1.7869999999999999,"5min":1.651,"15min":1.647},"min":{"1min":1.337,"5min":1.1080000000000001,"15min":1.0780000000000001},"max":{"1min":2.1240000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.5980000000000001,"15min":1.6040000000000001},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.2270000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52128,"kb_used_data":2892,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319712,"statfs":{"total":96636764160,"available":96583385088,"internally_reserved":0,"allocated":2961408,"data_stored":1537686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.90500000000000003,"15min":0.94099999999999995},"min":{"1min":0.67400000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3120000000000001,"5min":1.367,"15min":1.976},"last":0.629},{"interface":"front","average":{"1min":1.115,"5min":1.147,"15min":1.1619999999999999},"min":{"1min":0.81200000000000006,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.679,"5min":1.833,"15min":1.8400000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.046,"15min":1.048},"min":{"1min":0.83399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5369999999999999,"5min":1.6990000000000001,"15min":1.79},"last":0.873},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.238,"15min":1.252},"min":{"1min":1.121,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.6100000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.367,"15min":1.397},"min":{"1min":1.0780000000000001,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8620000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.498,"5min":1.28,"15min":1.304},"min":{"1min":0.98899999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.9339999999999999},"last":0.91600000000000004}]},{"osd":4,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3600000000000001,"15min":1.3580000000000001},"min":{"1min":1.0820000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.643,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.575,"5min":1.498,"15min":1.504},"min":{"1min":1.157,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.867,"5min":2.0750000000000002,"15min":2.383},"last":1.151}]},{"osd":5,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.548,"15min":1.5720000000000001},"min":{"1min":1.202,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.732,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.524,"15min":1.5429999999999999},"min":{"1min":1.1639999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.8240000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.2769999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.6080000000000001,"15min":1.6060000000000001},"min":{"1min":1.2729999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.085,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.109},{"interface":"front","average":{"1min":1.556,"5min":1.552,"15min":1.5629999999999999},"min":{"1min":1.2390000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.752,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0589999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:37:59 2023","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.653,"15min":1.6799999999999999},"min":{"1min":1.298,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.994,"5min":2.1829999999999998,"15min":2.548},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.7809999999999999,"5min":1.738,"15min":1.7270000000000001},"min":{"1min":1.401,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.1739999999999999,"5min":2.492,"15min":2.548},"last":1.3080000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52704,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3559424,"data_stored":2128144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.93400000000000005,"15min":0.95199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.2989999999999999,"5min":1.514,"15min":1.774},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.036,"15min":1.101},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.506,"5min":1.556,"15min":1.774},"last":0.61699999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.121,"15min":1.171},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1140000000000001,"15min":1.181},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.708,"5min":1.708,"15min":1.851},"last":1.1120000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0960000000000001,"15min":1.133},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6259999999999999,"5min":1.897,"15min":1.9590000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.327,"15min":1.391},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.3149999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.353,"15min":1.4299999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.8169999999999999,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":0.877},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5580000000000001,"15min":1.599},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3959999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.401,"15min":1.4059999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.772,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.448,"5min":1.51,"15min":1.573},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.1150000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.5049999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4910000000000001,"15min":1.4970000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.847,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.569,"5min":1.611,"15min":1.6759999999999999},"min":{"1min":0.999,"5min":0.996,"15min":0.996},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":1.454}]},{"osd":7,"last update":"Sat Oct 21 17:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4890000000000001,"15min":1.5449999999999999},"min":{"1min":0.82799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8380000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.002},{"interface":"front","average":{"1min":1.478,"5min":1.5900000000000001,"15min":1.623},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.952,"5min":2.2519999999999998,"15min":2.262},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:38:52.646 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:38:52.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:38:52.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.909+0000 7f3f45026640 1 Processor -- start 2023-10-21T17:38:52.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.909+0000 7f3f45026640 1 -- start start 2023-10-21T17:38:52.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.910+0000 7f3f45026640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3f400cd8d0 0x7f3f400cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.910+0000 7f3f45026640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400b8bf0 con 0x7f3f400b1840 2023-10-21T17:38:52.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.910+0000 7f3f45026640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400d39b0 con 0x7f3f400b1d90 2023-10-21T17:38:52.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.910+0000 7f3f45026640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400d2910 con 0x7f3f400cd8d0 2023-10-21T17:38:52.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.911+0000 7f3f3659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3f400b1840 0x7f3f400d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59394/0 (socket says 172.21.15.53:59394) 2023-10-21T17:38:52.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.911+0000 7f3f3659c640 1 -- 172.21.15.53:0/3934527517 learned_addr learned my addr 172.21.15.53:0/3934527517 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:52.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.911+0000 7f3f3ed9d640 1 --2- 172.21.15.53:0/3934527517 >> v2:172.21.15.53:3300/0 conn(0x7f3f400cd8d0 0x7f3f400cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.911+0000 7f3f3e59c640 1 -- 172.21.15.53:0/3934527517 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749053351 0 0) 0x7f3f400b8bf0 con 0x7f3f400b1840 2023-10-21T17:38:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.911+0000 7f3f3e59c640 1 -- 172.21.15.53:0/3934527517 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003520 con 0x7f3f400b1840 2023-10-21T17:38:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.912+0000 7f3f3ed9d640 1 -- 172.21.15.53:0/3934527517 >> v1:172.21.15.124:6789/0 conn(0x7f3f400b1d90 legacy=0x7f3f400d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.912+0000 7f3f3ed9d640 1 -- 172.21.15.53:0/3934527517 >> v1:172.21.15.53:6789/0 conn(0x7f3f400b1840 legacy=0x7f3f400d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.912+0000 7f3f3ed9d640 1 -- 172.21.15.53:0/3934527517 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400ce1f0 con 0x7f3f400cd8d0 2023-10-21T17:38:52.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.913+0000 7f3f3ed9d640 1 --2- 172.21.15.53:0/3934527517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400cd8d0 0x7f3f400cdcb0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097a0 tx=0x7f3f3002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f799ab57353538e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.913+0000 7f3f3e59c640 1 -- 172.21.15.53:0/3934527517 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30030610 con 0x7f3f400cd8d0 2023-10-21T17:38:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.913+0000 7f3f3e59c640 1 -- 172.21.15.53:0/3934527517 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30030770 con 0x7f3f400cd8d0 2023-10-21T17:38:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.913+0000 7f3f3e59c640 1 -- 172.21.15.53:0/3934527517 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30030ae0 con 0x7f3f400cd8d0 2023-10-21T17:38:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 -- 172.21.15.53:0/3934527517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400cd8d0 msgr2=0x7f3f400cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 --2- 172.21.15.53:0/3934527517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400cd8d0 0x7f3f400cdcb0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097a0 tx=0x7f3f3002b960 comp rx=0 tx=0).stop 2023-10-21T17:38:52.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 -- 172.21.15.53:0/3934527517 shutdown_connections 2023-10-21T17:38:52.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 --2- 172.21.15.53:0/3934527517 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400cd8d0 0x7f3f400cdcb0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 -- 172.21.15.53:0/3934527517 >> 172.21.15.53:0/3934527517 conn(0x7f3f4006b6a0 msgr2=0x7f3f4006bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:52.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 -- 172.21.15.53:0/3934527517 shutdown_connections 2023-10-21T17:38:52.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.914+0000 7f3f45026640 1 -- 172.21.15.53:0/3934527517 wait complete. 2023-10-21T17:38:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.917+0000 7f3f45026640 1 Processor -- start 2023-10-21T17:38:52.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.917+0000 7f3f45026640 1 -- start start 2023-10-21T17:38:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f400b1840 0x7f3f400c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f400cd8d0 0x7f3f4015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f401e2d40 con 0x7f3f400b1d90 2023-10-21T17:38:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f401e3020 con 0x7f3f400cd8d0 2023-10-21T17:38:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.918+0000 7f3f45026640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f401e3300 con 0x7f3f400b1840 2023-10-21T17:38:52.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49064/0 (socket says 172.21.15.53:49064) 2023-10-21T17:38:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 -- 172.21.15.53:0/1887148117 learned_addr learned my addr 172.21.15.53:0/1887148117 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3ed9d640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f400b1840 0x7f3f400c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3f59e640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f400cd8d0 0x7f3f4015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 -- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f400b1840 msgr2=0x7f3f400c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f400b1840 0x7f3f400c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 -- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f400cd8d0 msgr2=0x7f3f4015b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f400cd8d0 0x7f3f4015b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 -- 172.21.15.53:0/1887148117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f30009400 con 0x7f3f400b1d90 2023-10-21T17:38:52.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.919+0000 7f3f3659c640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c500 tx=0x7f3f2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.920+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28005ce0 con 0x7f3f400b1d90 2023-10-21T17:38:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.920+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28005e40 con 0x7f3f400b1d90 2023-10-21T17:38:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.920+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28007330 con 0x7f3f400b1d90 2023-10-21T17:38:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.920+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f401e35e0 con 0x7f3f400b1d90 2023-10-21T17:38:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.920+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f400c8c30 con 0x7f3f400b1d90 2023-10-21T17:38:52.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.921+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f401e3b00 con 0x7f3f400b1d90 2023-10-21T17:38:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.923+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f28007550 con 0x7f3f400b1d90 2023-10-21T17:38:52.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.924+0000 7f3f37fff640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 0x7f3f2006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.924+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3f28089360 con 0x7f3f400b1d90 2023-10-21T17:38:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.925+0000 7f3f3ed9d640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 0x7f3f2006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:52.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.926+0000 7f3f3ed9d640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 0x7f3f2006d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f30036000 tx=0x7f3f30008030 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:52.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:52.930+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f401e3b00 con 0x7f3f400b1d90 2023-10-21T17:38:53.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.165+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f401e3b00 con 0x7f3f400b1d90 2023-10-21T17:38:53.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.166+0000 7f3f37fff640 1 -- 172.21.15.53:0/1887148117 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7f3f401e3b00 con 0x7f3f400b1d90 2023-10-21T17:38:53.168 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:38:53.168 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:38:53.168 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:38:53.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 msgr2=0x7f3f2006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 0x7f3f2006d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f30036000 tx=0x7f3f30008030 comp rx=0 tx=0).stop 2023-10-21T17:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 msgr2=0x7f3f4015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c500 tx=0x7f3f2800c9c0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 shutdown_connections 2023-10-21T17:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f400cd8d0 0x7f3f4015b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3f400b1d90 0x7f3f4015a110 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f3f2006af90 0x7f3f2006d450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 --2- 172.21.15.53:0/1887148117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f400b1840 0x7f3f400c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 >> 172.21.15.53:0/1887148117 conn(0x7f3f4006b6a0 msgr2=0x7f3f400b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 shutdown_connections 2023-10-21T17:38:53.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.169+0000 7f3f45026640 1 -- 172.21.15.53:0/1887148117 wait complete. 2023-10-21T17:38:53.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:38:47.013927+0000', 'modified': '2023-10-21T17:38:50.070533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11148': {'gid': 11148, 'name': 'c', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.53:6835/1125921177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 1125921177}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1125921177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:38:53.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:38:53.593 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-21T17:38:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.830+0000 7f5bf32a6640 1 Processor -- start 2023-10-21T17:38:53.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.830+0000 7f5bf32a6640 1 -- start start 2023-10-21T17:38:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.831+0000 7f5bf32a6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 0x7f5bec070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf32a6640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5bec0aff70 0x7f5bec0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf32a6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec071d90 con 0x7f5bec156a10 2023-10-21T17:38:53.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf32a6640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec0b0940 con 0x7f5bec0aff70 2023-10-21T17:38:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf32a6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1598c0 con 0x7f5bec1594e0 2023-10-21T17:38:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf1aa3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5bec156a10 0x7f5bec0caf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59410/0 (socket says 172.21.15.53:59410) 2023-10-21T17:38:53.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/3009185447 learned_addr learned my addr 172.21.15.53:0/3009185447 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 0x7f5bec070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.832+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3009185447 >> v2:172.21.15.124:3300/0 conn(0x7f5bec0aff70 0x7f5bec0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807704884 0 0) 0x7f5bec071d90 con 0x7f5bec156a10 2023-10-21T17:38:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd8003520 con 0x7f5bec156a10 2023-10-21T17:38:53.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100862775 0 0) 0x7f5bd8003520 con 0x7f5bec156a10 2023-10-21T17:38:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 msgr2=0x7f5bec070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 0x7f5bec070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 >> v2:172.21.15.124:3300/0 conn(0x7f5bec0aff70 msgr2=0x7f5bec0b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 --2- 172.21.15.53:0/3009185447 >> v2:172.21.15.124:3300/0 conn(0x7f5bec0aff70 0x7f5bec0b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0cb640 con 0x7f5bec156a10 2023-10-21T17:38:53.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5bdc001670 con 0x7f5bec156a10 2023-10-21T17:38:53.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 >> v1:172.21.15.53:6789/0 conn(0x7f5bec156a10 legacy=0x7f5bec0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bd80047c0 0x7f5bd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 0x7f5bd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0b0940 con 0x7f5bd8003970 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec1598c0 con 0x7f5bd80047c0 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd800db90 con 0x7f5bec1594e0 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.833+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bd80047c0 0x7f5bd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 msgr2=0x7f5bd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec1594e0 0x7f5bd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bd80047c0 msgr2=0x7f5bd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bd80047c0 0x7f5bd8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.834+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3009185447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd8003520 con 0x7f5bd8003970 2023-10-21T17:38:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.835+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bd8003d90 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009ac0 tx=0x7f5bd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3512eede07e1fec server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.835+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be802c6d0 con 0x7f5bd8003970 2023-10-21T17:38:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.835+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be802c830 con 0x7f5bd8003970 2023-10-21T17:38:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.835+0000 7f5bf12a2640 1 -- 172.21.15.53:0/3009185447 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be802cbc0 con 0x7f5bd8003970 2023-10-21T17:38:53.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 -- 172.21.15.53:0/3009185447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 msgr2=0x7f5bd8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/3009185447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bd8003d90 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009ac0 tx=0x7f5bd80057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf2aa5640 1 -- 172.21.15.53:0/3009185447 reap_dead start 2023-10-21T17:38:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 -- 172.21.15.53:0/3009185447 shutdown_connections 2023-10-21T17:38:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 -- 172.21.15.53:0/3009185447 >> 172.21.15.53:0/3009185447 conn(0x7f5bec06c050 msgr2=0x7f5bec06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 -- 172.21.15.53:0/3009185447 shutdown_connections 2023-10-21T17:38:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.836+0000 7f5bf32a6640 1 -- 172.21.15.53:0/3009185447 wait complete. 2023-10-21T17:38:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.838+0000 7f5bf32a6640 1 Processor -- start 2023-10-21T17:38:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.838+0000 7f5bf32a6640 1 -- start start 2023-10-21T17:38:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec0c0fd0 0x7f5bec158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bec154e00 0x7f5bec155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0c7150 con 0x7f5bd8003970 2023-10-21T17:38:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec155780 con 0x7f5bec154e00 2023-10-21T17:38:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf32a6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0c6820 con 0x7f5bec0c0fd0 2023-10-21T17:38:53.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf22a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf22a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49084/0 (socket says 172.21.15.53:49084) 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.839+0000 7f5bf22a4640 1 -- 172.21.15.53:0/4108955569 learned_addr learned my addr 172.21.15.53:0/4108955569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 -- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec0c0fd0 msgr2=0x7f5bec158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec0c0fd0 0x7f5bec158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 -- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bec154e00 msgr2=0x7f5bec155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bec154e00 0x7f5bec155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 -- 172.21.15.53:0/4108955569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0c6b00 con 0x7f5bd8003970 2023-10-21T17:38:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.840+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f5be8037040 tx=0x7f5be8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.841+0000 7f5be2ffd640 1 -- 172.21.15.53:0/4108955569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80044e0 con 0x7f5bd8003970 2023-10-21T17:38:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.841+0000 7f5be2ffd640 1 -- 172.21.15.53:0/4108955569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8003610 con 0x7f5bd8003970 2023-10-21T17:38:53.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.841+0000 7f5be2ffd640 1 -- 172.21.15.53:0/4108955569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be80039c0 con 0x7f5bd8003970 2023-10-21T17:38:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.841+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec0c6de0 con 0x7f5bd8003970 2023-10-21T17:38:53.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.841+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0c7b20 con 0x7f5bd8003970 2023-10-21T17:38:53.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.843+0000 7f5be2ffd640 1 -- 172.21.15.53:0/4108955569 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5be8003b20 con 0x7f5bd8003970 2023-10-21T17:38:53.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.844+0000 7f5be2ffd640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 0x7f5bd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.844+0000 7f5be2ffd640 1 -- 172.21.15.53:0/4108955569 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f5be80abc70 con 0x7f5bd8003970 2023-10-21T17:38:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.845+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 0x7f5bd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.845+0000 7f5bf7170740 1 Processor -- start 2023-10-21T17:38:53.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.845+0000 7f5bf7170740 1 -- start start 2023-10-21T17:38:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 0x7f5bd006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bec154db0 tx=0x7f5bdc009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c895480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c7b17a0 con 0x55a41c8959c0 2023-10-21T17:38:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c803740 con 0x55a41c8c2960 2023-10-21T17:38:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a41c800e30 con 0x55a41c8c2580 2023-10-21T17:38:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.846+0000 7f5bf2aa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49098/0 (socket says 172.21.15.53:49098) 2023-10-21T17:38:53.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 -- 172.21.15.53:0/3591872790 learned_addr learned my addr 172.21.15.53:0/3591872790 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 -- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 msgr2=0x55a41c895480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c895480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 -- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 msgr2=0x55a41c8c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 -- 172.21.15.53:0/3591872790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a41c7c0910 con 0x55a41c8959c0 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0c5550 tx=0x7f5be400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6616d209305c0c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.847+0000 7f5bcffff640 1 -- 172.21.15.53:0/3591872790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4010040 con 0x55a41c8959c0 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bcffff640 1 -- 172.21.15.53:0/3591872790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4003990 con 0x55a41c8959c0 2023-10-21T17:38:53.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bcffff640 1 -- 172.21.15.53:0/3591872790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4003d20 con 0x55a41c8959c0 2023-10-21T17:38:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 -- 172.21.15.53:0/3591872790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 msgr2=0x55a41c8c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0c5550 tx=0x7f5be400b040 comp rx=0 tx=0).stop 2023-10-21T17:38:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 -- 172.21.15.53:0/3591872790 shutdown_connections 2023-10-21T17:38:53.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8959c0 0x55a41c8c5850 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3591872790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c895480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 -- 172.21.15.53:0/3591872790 >> 172.21.15.53:0/3591872790 conn(0x55a41c897790 msgr2=0x55a41c8e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 -- 172.21.15.53:0/3591872790 shutdown_connections 2023-10-21T17:38:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.848+0000 7f5bf7170740 1 -- 172.21.15.53:0/3591872790 wait complete. 2023-10-21T17:38:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.849+0000 7f5bf7170740 1 -- start start 2023-10-21T17:38:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.849+0000 7f5bf7170740 1 Processor -- start 2023-10-21T17:38:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c7fc720 con 0x55a41c8c2580 2023-10-21T17:38:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c79a970 con 0x55a41c8c2960 2023-10-21T17:38:53.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.850+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a41c79a610 con 0x55a41c8c7a60 2023-10-21T17:38:53.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf7170740 10 client.0 _ll_register_callbacks cb 0x55a41c7bc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:38:53.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49112/0 (socket says 172.21.15.53:49112) 2023-10-21T17:38:53.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3725604916 learned_addr learned my addr 172.21.15.53:0/3725604916 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:53.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 msgr2=0x55a41c8b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 msgr2=0x55a41c8d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf22a4640 1 -- 172.21.15.53:0/3725604916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be4009b90 con 0x55a41c8c2580 2023-10-21T17:38:53.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.851+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:53.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f5be802c3a0 tx=0x7f5be807e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9e4e2e47c5ca8b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:38:53.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8078770 con 0x55a41c8c2580 2023-10-21T17:38:53.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f5be8076a20 con 0x55a41c8c2580 2023-10-21T17:38:53.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a41c823160 con 0x55a41c8c2580 2023-10-21T17:38:53.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be8076be0 con 0x55a41c8c2580 2023-10-21T17:38:53.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bf7170740 10 client.11427 fetch_fsmap learned FSMap version 118 2023-10-21T17:38:53.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x55a41c7fada0 con 0x55a41c8c2580 2023-10-21T17:38:53.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a41c823160 con 0x55a41c8c2580 2023-10-21T17:38:53.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.852+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 118) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f5be807a570 con 0x55a41c8c2580 2023-10-21T17:38:53.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 10 client.11427 fetch_fsmap finished waiting for FSMap version 118 2023-10-21T17:38:53.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 10 client.11427 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T17:38:53.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 20 client.11427 populate_metadata read hostname 'smithi053' 2023-10-21T17:38:53.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c8afe90 0x55a41c8b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 4 client.11427 mds_command: new command op to 11148 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:38:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 --> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a41c7cecc0 con 0x55a41c8afe90 2023-10-21T17:38:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c8afe90 0x55a41c8b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c8afe90 0x55a41c8b0270 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5999429453609eee server_cookie=5a9f083f133202c3 in_seq=0 out_seq=0 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.853+0000 7f5bcd7fa640 10 client.11427 ms_handle_connect on v2:172.21.15.53:6834/1125921177 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.854+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/3725604916 <== mds.0 v2:172.21.15.53:6834/1125921177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a41c7cecc0 con 0x55a41c8afe90 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.854+0000 7f5bcd7fa640 10 client.11427 handle_command_reply: tid=0 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.854+0000 7f5bf7170740 1 client.11427 shutdown 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.854+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 msgr2=0x55a41c8f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.854+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f5be802c3a0 tx=0x7f5be807e460 comp rx=0 tx=0).stop 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 shutdown_connections 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c8afe90 0x55a41c8b0270 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2960 0x55a41c8d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3725604916 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2580 0x55a41c8f3950 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bcd7fa640 0 client.11427 ms_handle_reset on v2:172.21.15.53:6834/1125921177 2023-10-21T17:38:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 >> 172.21.15.53:0/3725604916 conn(0x55a41c8959c0 msgr2=0x55a41c899540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 shutdown_connections 2023-10-21T17:38:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 1 -- 172.21.15.53:0/3725604916 wait complete. 2023-10-21T17:38:53.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.855+0000 7f5bf7170740 20 client.11427 trim_cache size 0 max 16384 2023-10-21T17:38:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.872+0000 7f5bf7170740 1 Processor -- start 2023-10-21T17:38:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.872+0000 7f5bf7170740 1 -- start start 2023-10-21T17:38:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c8900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c7a60 0x55a41c8c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c560f20 con 0x55a41c8c2960 2023-10-21T17:38:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c8f3410 con 0x55a41c8c7a60 2023-10-21T17:38:53.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a41c8dc2f0 con 0x55a41c8c2580 2023-10-21T17:38:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf1aa3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf1aa3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49124/0 (socket says 172.21.15.53:49124) 2023-10-21T17:38:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.873+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/3990702872 learned_addr learned my addr 172.21.15.53:0/3990702872 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c8900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c7a60 0x55a41c8c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 msgr2=0x55a41c8900f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c8900f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c7a60 msgr2=0x55a41c8c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c7a60 0x55a41c8c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/3990702872 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a41c799b80 con 0x55a41c8c2960 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d520 tx=0x7f5bdc016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ccac8a917611d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5be0ff9640 1 -- 172.21.15.53:0/3990702872 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc005cf0 con 0x55a41c8c2960 2023-10-21T17:38:53.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.874+0000 7f5be0ff9640 1 -- 172.21.15.53:0/3990702872 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc01a030 con 0x55a41c8c2960 2023-10-21T17:38:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5be0ff9640 1 -- 172.21.15.53:0/3990702872 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0215a0 con 0x55a41c8c2960 2023-10-21T17:38:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 -- 172.21.15.53:0/3990702872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 msgr2=0x55a41c890630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d520 tx=0x7f5bdc016040 comp rx=0 tx=0).stop 2023-10-21T17:38:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 -- 172.21.15.53:0/3990702872 shutdown_connections 2023-10-21T17:38:53.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c7a60 0x55a41c8c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c890630 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 --2- 172.21.15.53:0/3990702872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c2580 0x55a41c8900f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 -- 172.21.15.53:0/3990702872 >> 172.21.15.53:0/3990702872 conn(0x55a41c8959c0 msgr2=0x55a41c8e0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 -- 172.21.15.53:0/3990702872 shutdown_connections 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.875+0000 7f5bf7170740 1 -- 172.21.15.53:0/3990702872 wait complete. 2023-10-21T17:38:53.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.876+0000 7f5bf7170740 1 -- start start 2023-10-21T17:38:53.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.876+0000 7f5bf7170740 1 Processor -- start 2023-10-21T17:38:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 0x55a41c8d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c8da830 con 0x55a41c8c2960 2023-10-21T17:38:53.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a41c8c1a80 con 0x55a41c8c2580 2023-10-21T17:38:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a41c8e18c0 con 0x55a41c8c7a60 2023-10-21T17:38:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.877+0000 7f5bf7170740 10 client.0 _ll_register_callbacks cb 0x7f5be807a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:38:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:49132/0 (socket says 172.21.15.53:49132) 2023-10-21T17:38:53.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/772628444 learned_addr learned my addr 172.21.15.53:0/772628444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 msgr2=0x55a41c8e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf2aa5640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 0x55a41c8d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8e2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 msgr2=0x55a41c8d9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:53.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 0x55a41c8d9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 -- 172.21.15.53:0/772628444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec0cb640 con 0x55a41c8c2960 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 0x55a41c8d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bf1aa3640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007200 tx=0x7f5bdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a2c3dc214f301 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.878+0000 7f5bcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc021da0 con 0x55a41c8c2960 2023-10-21T17:38:53.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f5bdc00d7d0 con 0x55a41c8c2960 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5be4003990 con 0x55a41c8c2960 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bdc00d990 con 0x55a41c8c2960 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 10 client.11433 fetch_fsmap learned FSMap version 118 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x55a41c7bf170 con 0x55a41c8c2960 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4003990 con 0x55a41c8c2960 2023-10-21T17:38:53.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 118) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f5bdc01f050 con 0x55a41c8c2960 2023-10-21T17:38:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 10 client.11433 fetch_fsmap finished waiting for FSMap version 118 2023-10-21T17:38:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 10 client.11433 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T17:38:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 20 client.11433 populate_metadata read hostname 'smithi053' 2023-10-21T17:38:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c89a220 0x55a41c89a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:53.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 4 client.11433 mds_command: new command op to 11148 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:38:53.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.879+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 --> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a41c7f5700 con 0x55a41c89a220 2023-10-21T17:38:53.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c89a220 0x55a41c89a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:53.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bf22a4640 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c89a220 0x55a41c89a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce8f309cfb62743 server_cookie=b549fb01c2d75bba in_seq=0 out_seq=0 2023-10-21T17:38:53.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bcd7fa640 10 client.11433 ms_handle_connect on v2:172.21.15.53:6834/1125921177 2023-10-21T17:38:53.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bcd7fa640 1 -- 172.21.15.53:0/772628444 <== mds.0 v2:172.21.15.53:6834/1125921177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x55a41c7f5700 con 0x55a41c89a220 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bcd7fa640 10 client.11433 handle_command_reply: tid=0 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.880+0000 7f5bf7170740 1 client.11433 shutdown 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 msgr2=0x55a41c948ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc007200 tx=0x7f5bdc000bc0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 shutdown_connections 2023-10-21T17:38:53.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:6834/1125921177,v1:172.21.15.53:6835/1125921177] conn(0x55a41c89a220 0x55a41c89a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55a41c8c7a60 0x55a41c8e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a41c8c2960 0x55a41c948ab0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 --2- 172.21.15.53:0/772628444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55a41c8c2580 0x55a41c8d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 >> 172.21.15.53:0/772628444 conn(0x55a41c8959c0 msgr2=0x55a41c8e2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bcd7fa640 0 client.11433 ms_handle_reset on v2:172.21.15.53:6834/1125921177 2023-10-21T17:38:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 shutdown_connections 2023-10-21T17:38:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 1 -- 172.21.15.53:0/772628444 wait complete. 2023-10-21T17:38:53.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.881+0000 7f5bf7170740 20 client.11433 trim_cache size 0 max 16384 2023-10-21T17:38:53.895 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:38:53.895 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:38:53.895 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:38:53.895 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 11148, 2023-10-21T17:38:53.896 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:38:53.896 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:38:53.896 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "backup_fs", 2023-10-21T17:38:53.896 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.3374723250000002, 2023-10-21T17:38:53.896 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 118, 2023-10-21T17:38:53.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 237, 2023-10-21T17:38:53.897 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 237, 2023-10-21T17:38:53.897 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 33.391515177999999 2023-10-21T17:38:53.897 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:38:53.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 msgr2=0x7f5bd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 0x7f5bd006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bec154db0 tx=0x7f5bdc009040 comp rx=0 tx=0).stop 2023-10-21T17:38:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 msgr2=0x7f5bec0c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f5be8037040 tx=0x7f5be8004730 comp rx=0 tx=0).stop 2023-10-21T17:38:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 shutdown_connections 2023-10-21T17:38:53.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bec154e00 0x7f5bec155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f5bd006af90 0x7f5bd006d450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bec0c0fd0 0x7f5bec158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 --2- 172.21.15.53:0/4108955569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5bd8003970 0x7f5bec0c75e0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 >> 172.21.15.53:0/4108955569 conn(0x7f5bec06c050 msgr2=0x7f5bec0c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:53.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 shutdown_connections 2023-10-21T17:38:53.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:53.882+0000 7f5bf32a6640 1 -- 172.21.15.53:0/4108955569 wait complete. 2023-10-21T17:38:53.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:38:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.117+0000 7f5221221640 1 Processor -- start 2023-10-21T17:38:54.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.117+0000 7f5221221640 1 -- start start 2023-10-21T17:38:54.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f521c15a820 0x7f521c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f521c15ac00 0x7f521c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0cab60 con 0x7f521c15ac00 2023-10-21T17:38:54.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0cb960 con 0x7f521c15a820 2023-10-21T17:38:54.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5221221640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0cbdf0 con 0x7f521c0ca680 2023-10-21T17:38:54.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5220a20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f521c15ac00 0x7f521c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5220a20640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f521c15ac00 0x7f521c0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34048/0 (socket says 172.21.15.53:34048) 2023-10-21T17:38:54.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.118+0000 7f5220a20640 1 -- 172.21.15.53:0/187471941 learned_addr learned my addr 172.21.15.53:0/187471941 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:54.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f521bfff640 1 --2- 172.21.15.53:0/187471941 >> v2:172.21.15.124:3300/0 conn(0x7f521c15a820 0x7f521c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5213fff640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5220a20640 1 -- 172.21.15.53:0/187471941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 msgr2=0x7f521c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5220a20640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5220a20640 1 -- 172.21.15.53:0/187471941 >> v2:172.21.15.124:3300/0 conn(0x7f521c15a820 msgr2=0x7f521c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5220a20640 1 --2- 172.21.15.53:0/187471941 >> v2:172.21.15.124:3300/0 conn(0x7f521c15a820 0x7f521c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.119+0000 7f5220a20640 1 -- 172.21.15.53:0/187471941 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0cb030 con 0x7f521c15ac00 2023-10-21T17:38:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.120+0000 7f5220a20640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15ac00 0x7f521c0c9100 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f521c06da70 tx=0x7f521402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef62d640c638f97a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.120+0000 7f521b7fe640 1 -- 172.21.15.53:0/187471941 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521402c9f0 con 0x7f521c15ac00 2023-10-21T17:38:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.120+0000 7f521b7fe640 1 -- 172.21.15.53:0/187471941 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521402cb50 con 0x7f521c15ac00 2023-10-21T17:38:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.120+0000 7f521b7fe640 1 -- 172.21.15.53:0/187471941 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140343f0 con 0x7f521c15ac00 2023-10-21T17:38:54.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 -- 172.21.15.53:0/187471941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15ac00 msgr2=0x7f521c0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15ac00 0x7f521c0c9100 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f521c06da70 tx=0x7f521402be50 comp rx=0 tx=0).stop 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 -- 172.21.15.53:0/187471941 shutdown_connections 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 --2- 172.21.15.53:0/187471941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15ac00 0x7f521c0c9100 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 --2- 172.21.15.53:0/187471941 >> v2:172.21.15.124:3300/0 conn(0x7f521c15a820 0x7f521c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 -- 172.21.15.53:0/187471941 >> 172.21.15.53:0/187471941 conn(0x7f521c06c050 msgr2=0x7f521c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 -- 172.21.15.53:0/187471941 shutdown_connections 2023-10-21T17:38:54.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.121+0000 7f5221221640 1 -- 172.21.15.53:0/187471941 wait complete. 2023-10-21T17:38:54.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.123+0000 7f5221221640 1 Processor -- start 2023-10-21T17:38:54.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.123+0000 7f5221221640 1 -- start start 2023-10-21T17:38:54.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 0x7f521c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c150270 con 0x7f521c15a820 2023-10-21T17:38:54.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0c8570 con 0x7f521c15ac00 2023-10-21T17:38:54.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5221221640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b1b10 con 0x7f521c0ca680 2023-10-21T17:38:54.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5213fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5213fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34056/0 (socket says 172.21.15.53:34056) 2023-10-21T17:38:54.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.124+0000 7f5213fff640 1 -- 172.21.15.53:0/1869647843 learned_addr learned my addr 172.21.15.53:0/1869647843 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:54.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f521bfff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5220a20640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 0x7f521c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 -- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 msgr2=0x7f521c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 -- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 msgr2=0x7f521c0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 0x7f521c0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 -- 172.21.15.53:0/1869647843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009b30 con 0x7f521c15a820 2023-10-21T17:38:54.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5220a20640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 0x7f521c0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:54.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5213fff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f520400d5f0 tx=0x7f520400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040125f0 con 0x7f521c15a820 2023-10-21T17:38:54.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204012750 con 0x7f521c15a820 2023-10-21T17:38:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204012b00 con 0x7f521c15a820 2023-10-21T17:38:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.125+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c0b1d90 con 0x7f521c15a820 2023-10-21T17:38:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.126+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0b2120 con 0x7f521c15a820 2023-10-21T17:38:54.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.127+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c074150 con 0x7f521c15a820 2023-10-21T17:38:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.128+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5204012c60 con 0x7f521c15a820 2023-10-21T17:38:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.129+0000 7f52197fa640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 0x7f51fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.129+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f520408a1e0 con 0x7f521c15a820 2023-10-21T17:38:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.130+0000 7f521bfff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 0x7f51fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.131+0000 7f521bfff640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 0x7f51fc06d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f520c006fd0 tx=0x7f520c008040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.133+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5204054bc0 con 0x7f521c15a820 2023-10-21T17:38:54.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.355+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f521c0cb5f0 con 0x7f51fc06af90 2023-10-21T17:38:54.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.359+0000 7f52197fa640 1 -- 172.21.15.53:0/1869647843 <== mgr.37814 v2:172.21.15.124:6832/3930748558 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f520c00b180 con 0x7f51fc06af90 2023-10-21T17:38:54.361 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:38:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:38:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:38:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:38:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "backup_fs" 2023-10-21T17:38:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:38:54.363 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:38:54.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 msgr2=0x7f51fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 0x7f51fc06d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f520c006fd0 tx=0x7f520c008040 comp rx=0 tx=0).stop 2023-10-21T17:38:54.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 msgr2=0x7f521c0c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f520400d5f0 tx=0x7f520400dab0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 shutdown_connections 2023-10-21T17:38:54.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f521c15ac00 0x7f521c0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f521c15a820 0x7f521c0c8030 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f51fc06af90 0x7f51fc06d450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 --2- 172.21.15.53:0/1869647843 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f521c0ca680 0x7f521c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.362+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 >> 172.21.15.53:0/1869647843 conn(0x7f521c06c050 msgr2=0x7f521c15b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:54.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.363+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 shutdown_connections 2023-10-21T17:38:54.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.363+0000 7f5221221640 1 -- 172.21.15.53:0/1869647843 wait complete. 2023-10-21T17:38:54.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:38:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.607+0000 7eff9a7a7640 1 Processor -- start 2023-10-21T17:38:54.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.608+0000 7eff9a7a7640 1 -- start start 2023-10-21T17:38:54.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff9a7a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff940cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff9a7a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff94159db0 con 0x7eff940b3c50 2023-10-21T17:38:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff9a7a7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff94071fe0 con 0x7eff940cb5e0 2023-10-21T17:38:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff9a7a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b44c0 con 0x7eff941599d0 2023-10-21T17:38:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff98fa4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7eff940b3c50 0x7eff940caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47012/0 (socket says 172.21.15.53:47012) 2023-10-21T17:38:54.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.609+0000 7eff98fa4640 1 -- 172.21.15.53:0/4259796990 learned_addr learned my addr 172.21.15.53:0/4259796990 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff997a5640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff940cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876795901 0 0) 0x7eff84001110 con 0x7eff940b3c50 2023-10-21T17:38:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff80005730 con 0x7eff940b3c50 2023-10-21T17:38:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12030809 0 0) 0x7eff80005730 con 0x7eff940b3c50 2023-10-21T17:38:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 msgr2=0x7eff940cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff940cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 >> v1:172.21.15.124:6789/0 conn(0x7eff940cb5e0 legacy=0x7eff940cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940713d0 con 0x7eff940b3c50 2023-10-21T17:38:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7eff84001800 con 0x7eff940b3c50 2023-10-21T17:38:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 >> v1:172.21.15.53:6789/0 conn(0x7eff940b3c50 legacy=0x7eff940caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff80005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff800069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff80006f10 0x7eff80007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b44c0 con 0x7eff80005bb0 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8000dd20 con 0x7eff80006f10 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.610+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8000deb0 con 0x7eff941599d0 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff80005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff99fa6640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff80006f10 0x7eff80007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff98fa4640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff800069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 -- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 msgr2=0x7eff800069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff800069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 -- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff80006f10 msgr2=0x7eff80007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:54.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff80006f10 0x7eff80007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff997a5640 1 -- 172.21.15.53:0/4259796990 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80005730 con 0x7eff80005bb0 2023-10-21T17:38:54.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff98fa4640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941599d0 0x7eff800069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:54.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.611+0000 7eff99fa6640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff80006f10 0x7eff80007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:54.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.612+0000 7eff997a5640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff80005fd0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ba60 tx=0x7eff80007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9489aa6362cf4d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90032620 con 0x7eff80005bb0 2023-10-21T17:38:54.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90032780 con 0x7eff80005bb0 2023-10-21T17:38:54.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff8bfff640 1 -- 172.21.15.53:0/4259796990 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90032b10 con 0x7eff80005bb0 2023-10-21T17:38:54.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 -- 172.21.15.53:0/4259796990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 msgr2=0x7eff80005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/4259796990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff80005fd0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ba60 tx=0x7eff80007a00 comp rx=0 tx=0).stop 2023-10-21T17:38:54.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff99fa6640 1 -- 172.21.15.53:0/4259796990 reap_dead start 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 -- 172.21.15.53:0/4259796990 shutdown_connections 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 -- 172.21.15.53:0/4259796990 >> 172.21.15.53:0/4259796990 conn(0x7eff9406c050 msgr2=0x7eff9406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 -- 172.21.15.53:0/4259796990 shutdown_connections 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.613+0000 7eff9a7a7640 1 -- 172.21.15.53:0/4259796990 wait complete. 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.615+0000 7eff9a7a7640 1 Processor -- start 2023-10-21T17:38:54.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.615+0000 7eff9a7a7640 1 -- start start 2023-10-21T17:38:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff941e1f30 0x7eff941e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941556b0 0x7eff940b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941e4dd0 con 0x7eff80005bb0 2023-10-21T17:38:54.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941e5e80 con 0x7eff941e1f30 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.616+0000 7eff9a7a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff941e24a0 con 0x7eff941556b0 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34080/0 (socket says 172.21.15.53:34080) 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 -- 172.21.15.53:0/1675687730 learned_addr learned my addr 172.21.15.53:0/1675687730 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff98fa4640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff941e1f30 0x7eff941e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 -- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941556b0 msgr2=0x7eff940b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941556b0 0x7eff940b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 -- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff941e1f30 msgr2=0x7eff941e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:54.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff941e1f30 0x7eff941e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:54.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 -- 172.21.15.53:0/1675687730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff941e2780 con 0x7eff80005bb0 2023-10-21T17:38:54.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.617+0000 7eff997a5640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7eff90035040 tx=0x7eff90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.618+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90045420 con 0x7eff80005bb0 2023-10-21T17:38:54.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.618+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90045580 con 0x7eff80005bb0 2023-10-21T17:38:54.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.618+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff941e2a60 con 0x7eff80005bb0 2023-10-21T17:38:54.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.618+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90045910 con 0x7eff80005bb0 2023-10-21T17:38:54.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.618+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941e3020 con 0x7eff80005bb0 2023-10-21T17:38:54.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.619+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94074150 con 0x7eff80005bb0 2023-10-21T17:38:54.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.620+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 126) v1 ==== 89331+0+0 (secure 0 0 0) 0x7eff90043070 con 0x7eff80005bb0 2023-10-21T17:38:54.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.621+0000 7eff89ffb640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 0x7eff7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:54.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.621+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7eff900abc30 con 0x7eff80005bb0 2023-10-21T17:38:54.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.622+0000 7eff98fa4640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 0x7eff7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:54.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.623+0000 7eff98fa4640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 0x7eff7806d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff84001eb0 tx=0x7eff84009040 comp rx=0 tx=0).ready entity=mgr.37814 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:54.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.625+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff90076500 con 0x7eff80005bb0 2023-10-21T17:38:54.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:54.869+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7eff940c9c40 con 0x7eff80005bb0 2023-10-21T17:38:55.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.589+0000 7eff89ffb640 1 -- 172.21.15.53:0/1675687730 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v127) v1 ==== 92+0+0 (secure 0 0 0) 0x7eff90006c30 con 0x7eff80005bb0 2023-10-21T17:38:55.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 msgr2=0x7eff7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 0x7eff7806d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff84001eb0 tx=0x7eff84009040 comp rx=0 tx=0).stop 2023-10-21T17:38:55.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 msgr2=0x7eff941e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7eff90035040 tx=0x7eff90004730 comp rx=0 tx=0).stop 2023-10-21T17:38:55.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 shutdown_connections 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7eff7806af90 0x7eff7806d450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff941556b0 0x7eff940b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff941e1f30 0x7eff941e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 --2- 172.21.15.53:0/1675687730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff80005bb0 0x7eff941e3540 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 >> 172.21.15.53:0/1675687730 conn(0x7eff9406c050 msgr2=0x7eff9415b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:55.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 shutdown_connections 2023-10-21T17:38:55.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.592+0000 7eff9a7a7640 1 -- 172.21.15.53:0/1675687730 wait complete. 2023-10-21T17:38:55.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.826+0000 7fc93b7f5640 1 Processor -- start 2023-10-21T17:38:55.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.826+0000 7fc93b7f5640 1 -- start start 2023-10-21T17:38:55.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc93b7f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 0x7fc934070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc93b7f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc934071070 0x7fc9340cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc93b7f5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9340b1cc0 con 0x7fc9340afbe0 2023-10-21T17:38:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc93b7f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9340b2620 con 0x7fc934071070 2023-10-21T17:38:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc93b7f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9340b2060 con 0x7fc93406dfc0 2023-10-21T17:38:55.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc939591640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc9340afbe0 0x7fc9340b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47028/0 (socket says 172.21.15.53:47028) 2023-10-21T17:38:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc939591640 1 -- 172.21.15.53:0/1922156770 learned_addr learned my addr 172.21.15.53:0/1922156770 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:55.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.827+0000 7fc939d92640 1 --2- 172.21.15.53:0/1922156770 >> v2:172.21.15.53:3300/0 conn(0x7fc934071070 0x7fc9340cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc938d90640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 0x7fc934070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc939d92640 1 -- 172.21.15.53:0/1922156770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 msgr2=0x7fc934070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc939d92640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 0x7fc934070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc939d92640 1 -- 172.21.15.53:0/1922156770 >> v1:172.21.15.53:6789/0 conn(0x7fc9340afbe0 legacy=0x7fc9340b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc939d92640 1 -- 172.21.15.53:0/1922156770 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340b2340 con 0x7fc934071070 2023-10-21T17:38:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.828+0000 7fc938d90640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 0x7fc934070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:55.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.830+0000 7fc939d92640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc934071070 0x7fc9340cb130 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0097a0 tx=0x7fc92c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e2e0c3ac99ff3b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:55.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.830+0000 7fc933fff640 1 -- 172.21.15.53:0/1922156770 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c02d040 con 0x7fc934071070 2023-10-21T17:38:55.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.830+0000 7fc933fff640 1 -- 172.21.15.53:0/1922156770 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c031810 con 0x7fc934071070 2023-10-21T17:38:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.830+0000 7fc933fff640 1 -- 172.21.15.53:0/1922156770 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c031ba0 con 0x7fc934071070 2023-10-21T17:38:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 -- 172.21.15.53:0/1922156770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc934071070 msgr2=0x7fc9340cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc934071070 0x7fc9340cb130 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0097a0 tx=0x7fc92c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 -- 172.21.15.53:0/1922156770 shutdown_connections 2023-10-21T17:38:55.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc934071070 0x7fc9340cb130 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/1922156770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc93406dfc0 0x7fc934070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 -- 172.21.15.53:0/1922156770 >> 172.21.15.53:0/1922156770 conn(0x7fc93406a5d0 msgr2=0x7fc9340b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:55.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 -- 172.21.15.53:0/1922156770 shutdown_connections 2023-10-21T17:38:55.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.831+0000 7fc93b7f5640 1 -- 172.21.15.53:0/1922156770 wait complete. 2023-10-21T17:38:55.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.833+0000 7fc93b7f5640 1 Processor -- start 2023-10-21T17:38:55.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.833+0000 7fc93b7f5640 1 -- start start 2023-10-21T17:38:55.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 0x7fc9340c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9340afbe0 0x7fc934155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc934155a70 con 0x7fc93406dfc0 2023-10-21T17:38:55.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93415c8a0 con 0x7fc934071070 2023-10-21T17:38:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc93b7f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93415bc90 con 0x7fc9340afbe0 2023-10-21T17:38:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34116/0 (socket says 172.21.15.53:34116) 2023-10-21T17:38:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 -- 172.21.15.53:0/415658001 learned_addr learned my addr 172.21.15.53:0/415658001 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:55.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc938d90640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 0x7fc9340c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:55.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9340afbe0 msgr2=0x7fc934155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939d92640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9340afbe0 0x7fc934155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:55.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9340afbe0 0x7fc934155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 msgr2=0x7fc9340c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 0x7fc9340c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc939591640 1 -- 172.21.15.53:0/415658001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009400 con 0x7fc93406dfc0 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.834+0000 7fc938d90640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 0x7fc9340c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc939591640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c4a0 tx=0x7fc92800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:55.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928005ce0 con 0x7fc93406dfc0 2023-10-21T17:38:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928005e40 con 0x7fc93406dfc0 2023-10-21T17:38:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928007270 con 0x7fc93406dfc0 2023-10-21T17:38:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93415bf70 con 0x7fc93406dfc0 2023-10-21T17:38:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.835+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93415c530 con 0x7fc93406dfc0 2023-10-21T17:38:55.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.836+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93404e160 con 0x7fc93406dfc0 2023-10-21T17:38:55.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.837+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 127) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc9280073d0 con 0x7fc93406dfc0 2023-10-21T17:38:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.838+0000 7fc931ffb640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 0x7fc91806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.838+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc9280891e0 con 0x7fc93406dfc0 2023-10-21T17:38:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.838+0000 7fc938d90640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 msgr2=0x7fc91806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.838+0000 7fc938d90640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 0x7fc91806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:55.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:55.842+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc928053c70 con 0x7fc93406dfc0 2023-10-21T17:38:56.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.037+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc9340b0ef0 con 0x7fc93406dfc0 2023-10-21T17:38:56.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.039+0000 7fc938d90640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 msgr2=0x7fc91806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:56.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.039+0000 7fc938d90640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 0x7fc91806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:56.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.042+0000 7fc931ffb640 1 -- 172.21.15.53:0/415658001 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc928057290 con 0x7fc93406dfc0 2023-10-21T17:38:56.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 msgr2=0x7fc91806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:56.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 0x7fc91806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 msgr2=0x7fc934152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c4a0 tx=0x7fc92800c960 comp rx=0 tx=0).stop 2023-10-21T17:38:56.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 shutdown_connections 2023-10-21T17:38:56.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fc91806afe0 0x7fc91806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9340afbe0 0x7fc934155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc934071070 0x7fc9340c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 --2- 172.21.15.53:0/415658001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc93406dfc0 0x7fc934152880 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 >> 172.21.15.53:0/415658001 conn(0x7fc93406a5d0 msgr2=0x7fc9340c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:56.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 shutdown_connections 2023-10-21T17:38:56.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.045+0000 7fc93b7f5640 1 -- 172.21.15.53:0/415658001 wait complete. 2023-10-21T17:38:56.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:38:56.137 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.136+0000 7f5474a14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:38:56.139 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.138+0000 7fa2e5c1f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:38:56.191 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.189+0000 7fa2e5c1f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:38:56.193 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.192+0000 7f5474a14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:38:56.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.279+0000 7f9a5db73640 1 Processor -- start 2023-10-21T17:38:56.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.279+0000 7f9a5db73640 1 -- start start 2023-10-21T17:38:56.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.281+0000 7f9a5db73640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a580cd4e0 0x7f9a581612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.281+0000 7f9a5db73640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a580b8c30 con 0x7f9a5815cd70 2023-10-21T17:38:56.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.281+0000 7f9a5db73640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a580cdd80 con 0x7f9a580b4e10 2023-10-21T17:38:56.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.281+0000 7f9a5db73640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a580ce060 con 0x7f9a580cd4e0 2023-10-21T17:38:56.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a577fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9a5815cd70 0x7f9a580b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47036/0 (socket says 172.21.15.53:47036) 2023-10-21T17:38:56.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a577fe640 1 -- 172.21.15.53:0/2582945751 learned_addr learned my addr 172.21.15.53:0/2582945751 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:56.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a57fff640 1 --2- 172.21.15.53:0/2582945751 >> v2:172.21.15.124:3300/0 conn(0x7f9a580cd4e0 0x7f9a581612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291933654 0 0) 0x7f9a580b8c30 con 0x7f9a5815cd70 2023-10-21T17:38:56.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a44003520 con 0x7f9a5815cd70 2023-10-21T17:38:56.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063200658 0 0) 0x7f9a44003520 con 0x7f9a5815cd70 2023-10-21T17:38:56.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.282+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 >> v2:172.21.15.124:3300/0 conn(0x7f9a580cd4e0 msgr2=0x7f9a581612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 --2- 172.21.15.53:0/2582945751 >> v2:172.21.15.124:3300/0 conn(0x7f9a580cd4e0 0x7f9a581612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 >> v1:172.21.15.124:6789/0 conn(0x7f9a580b4e10 legacy=0x7f9a580d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a580ce340 con 0x7f9a5815cd70 2023-10-21T17:38:56.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9a40001670 con 0x7f9a5815cd70 2023-10-21T17:38:56.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 >> v1:172.21.15.53:6789/0 conn(0x7f9a5815cd70 legacy=0x7f9a580b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a440047c0 0x7f9a44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a44005180 0x7f9a4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a580ce060 con 0x7f9a44003970 2023-10-21T17:38:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4400c020 con 0x7f9a44005180 2023-10-21T17:38:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4400c1e0 con 0x7f9a440047c0 2023-10-21T17:38:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a577fe640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a4effd640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a440047c0 0x7f9a44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.283+0000 7f9a57fff640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a44005180 0x7f9a4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a577fe640 1 -- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a440047c0 msgr2=0x7f9a44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a577fe640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a440047c0 0x7f9a44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a577fe640 1 -- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a44005180 msgr2=0x7f9a4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a577fe640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a44005180 0x7f9a4400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a577fe640 1 -- 172.21.15.53:0/2582945751 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a44003520 con 0x7f9a44003970 2023-10-21T17:38:56.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.284+0000 7f9a57fff640 1 -- 172.21.15.53:0/2582945751 reap_dead start 2023-10-21T17:38:56.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.285+0000 7f9a577fe640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a44003d90 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c240 tx=0x7f9a440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73726a4eeb7ae680 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:56.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40005ad0 con 0x7f9a44003970 2023-10-21T17:38:56.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a40005c30 con 0x7f9a44003970 2023-10-21T17:38:56.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a56ffd640 1 -- 172.21.15.53:0/2582945751 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4002cab0 con 0x7f9a44003970 2023-10-21T17:38:56.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 -- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 msgr2=0x7f9a44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a44003d90 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c240 tx=0x7f9a440057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 -- 172.21.15.53:0/2582945751 shutdown_connections 2023-10-21T17:38:56.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 --2- 172.21.15.53:0/2582945751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a44003d90 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 -- 172.21.15.53:0/2582945751 >> 172.21.15.53:0/2582945751 conn(0x7f9a580cf700 msgr2=0x7f9a580cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:56.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 -- 172.21.15.53:0/2582945751 shutdown_connections 2023-10-21T17:38:56.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.286+0000 7f9a5db73640 1 -- 172.21.15.53:0/2582945751 wait complete. 2023-10-21T17:38:56.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.288+0000 7f9a5db73640 1 Processor -- start 2023-10-21T17:38:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.288+0000 7f9a5db73640 1 -- start start 2023-10-21T17:38:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a581584d0 0x7f9a5815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a58157f40 0x7f9a5815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a580ccd60 con 0x7f9a44003970 2023-10-21T17:38:56.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a580cc8d0 con 0x7f9a58157f40 2023-10-21T17:38:56.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a5db73640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5815d900 con 0x7f9a581584d0 2023-10-21T17:38:56.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34138/0 (socket says 172.21.15.53:34138) 2023-10-21T17:38:56.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.289+0000 7f9a577fe640 1 -- 172.21.15.53:0/4048943901 learned_addr learned my addr 172.21.15.53:0/4048943901 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:56.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a57fff640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a58157f40 0x7f9a5815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a581584d0 msgr2=0x7f9a5815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a581584d0 0x7f9a5815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a58157f40 msgr2=0x7f9a5815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a58157f40 0x7f9a5815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 -- 172.21.15.53:0/4048943901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5815ed70 con 0x7f9a44003970 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a577fe640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c240 tx=0x7f9a4002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4003c040 con 0x7f9a44003970 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4002cc80 con 0x7f9a44003970 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.290+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007070 con 0x7f9a44003970 2023-10-21T17:38:56.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.291+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5815ea90 con 0x7f9a44003970 2023-10-21T17:38:56.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.291+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a580cc560 con 0x7f9a44003970 2023-10-21T17:38:56.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.292+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5804e160 con 0x7f9a44003970 2023-10-21T17:38:56.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.293+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 127) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9a40032070 con 0x7f9a44003970 2023-10-21T17:38:56.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.294+0000 7f9a54ff9640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 0x7f9a3806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.294+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f9a400ab2d0 con 0x7f9a44003970 2023-10-21T17:38:56.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.294+0000 7f9a4effd640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 msgr2=0x7f9a3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:56.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.294+0000 7f9a4effd640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 0x7f9a3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:56.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.302+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a40075df0 con 0x7f9a44003970 2023-10-21T17:38:56.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.495+0000 7f9a4effd640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 msgr2=0x7f9a3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:56.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.495+0000 7f9a4effd640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 0x7f9a3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:56.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.498+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9a580cc6f0 con 0x7f9a44003970 2023-10-21T17:38:56.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.503+0000 7f9a54ff9640 1 -- 172.21.15.53:0/4048943901 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9a400460c0 con 0x7f9a44003970 2023-10-21T17:38:56.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 msgr2=0x7f9a3806d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 0x7f9a3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 msgr2=0x7f9a5815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c240 tx=0x7f9a4002e070 comp rx=0 tx=0).stop 2023-10-21T17:38:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 shutdown_connections 2023-10-21T17:38:56.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a58157f40 0x7f9a5815e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7f9a3806b010 0x7f9a3806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a581584d0 0x7f9a5815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 --2- 172.21.15.53:0/4048943901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a44003970 0x7f9a5815d3c0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 >> 172.21.15.53:0/4048943901 conn(0x7f9a580cf700 msgr2=0x7f9a580b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:56.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 shutdown_connections 2023-10-21T17:38:56.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.506+0000 7f9a5db73640 1 -- 172.21.15.53:0/4048943901 wait complete. 2023-10-21T17:38:56.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:38:56.528 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-21T17:38:56.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:38:56.642 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.641+0000 7fa2e5c1f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:38:56.679 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.678+0000 7f5474a14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:38:56.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.691+0000 7fa2e5c1f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:38:56.734 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.733+0000 7f5474a14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:38:56.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.741+0000 7fa2e5c1f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:38:56.787 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.786+0000 7f5474a14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:38:56.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.792+0000 7fce5c261640 1 Processor -- start 2023-10-21T17:38:56.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.793+0000 7fce5c261640 1 -- start start 2023-10-21T17:38:56.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.794+0000 7fce5c261640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fce5415cee0 0x7fce540ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.794+0000 7fce5c261640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540661b0 con 0x7fce5406e5d0 2023-10-21T17:38:56.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.794+0000 7fce5c261640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5415d2c0 con 0x7fce54065d20 2023-10-21T17:38:56.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.794+0000 7fce5c261640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540cd670 con 0x7fce5415cee0 2023-10-21T17:38:56.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.794+0000 7fce5a7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fce5406e5d0 0x7fce5406a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47050/0 (socket says 172.21.15.53:47050) 2023-10-21T17:38:56.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce5a7fe640 1 -- 172.21.15.53:0/617105474 learned_addr learned my addr 172.21.15.53:0/617105474 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:38:56.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce59ffd640 1 --2- 172.21.15.53:0/617105474 >> v2:172.21.15.124:3300/0 conn(0x7fce5415cee0 0x7fce540ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893162430 0 0) 0x7fce540661b0 con 0x7fce5406e5d0 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce40003520 con 0x7fce5406e5d0 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591729369 0 0) 0x7fce40003520 con 0x7fce5406e5d0 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 >> v2:172.21.15.124:3300/0 conn(0x7fce5415cee0 msgr2=0x7fce540ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 --2- 172.21.15.53:0/617105474 >> v2:172.21.15.124:3300/0 conn(0x7fce5415cee0 0x7fce540ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 >> v1:172.21.15.124:6789/0 conn(0x7fce54065d20 legacy=0x7fce5406dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540cd950 con 0x7fce5406e5d0 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fce4c0019f0 con 0x7fce5406e5d0 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 >> v1:172.21.15.53:6789/0 conn(0x7fce5406e5d0 legacy=0x7fce5406a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cd670 con 0x7fce40005180 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.795+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4000c080 con 0x7fce400039c0 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4000c240 con 0x7fce400047c0 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce59ffd640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce597fc640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 -- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce400047c0 msgr2=0x7fce40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce400047c0 0x7fce40004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 -- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce400039c0 msgr2=0x7fce40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:38:56.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce400039c0 0x7fce40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.796+0000 7fce5a7fe640 1 -- 172.21.15.53:0/617105474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40003520 con 0x7fce40005180 2023-10-21T17:38:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.797+0000 7fce5a7fe640 1 -- 172.21.15.53:0/617105474 reap_dead start 2023-10-21T17:38:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.797+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 0x7fce4000b980 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fce540cef50 tx=0x7fce400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ff022a01d3810d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.797+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c034630 con 0x7fce40005180 2023-10-21T17:38:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c02f070 con 0x7fce40005180 2023-10-21T17:38:56.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce58ffb640 1 -- 172.21.15.53:0/617105474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c034a10 con 0x7fce40005180 2023-10-21T17:38:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 -- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 msgr2=0x7fce4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 0x7fce4000b980 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fce540cef50 tx=0x7fce400057f0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 -- 172.21.15.53:0/617105474 shutdown_connections 2023-10-21T17:38:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 --2- 172.21.15.53:0/617105474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce40005180 0x7fce4000b980 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 -- 172.21.15.53:0/617105474 >> 172.21.15.53:0/617105474 conn(0x7fce540cba60 msgr2=0x7fce540cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 -- 172.21.15.53:0/617105474 shutdown_connections 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.798+0000 7fce5c261640 1 -- 172.21.15.53:0/617105474 wait complete. 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.800+0000 7fce5c261640 1 Processor -- start 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.800+0000 7fce5c261640 1 -- start start 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce40005180 0x7fce540b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce5415d470 0x7fce54070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 0x7fce54147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce54071460 con 0x7fce54070fd0 2023-10-21T17:38:56.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5414e710 con 0x7fce5415d470 2023-10-21T17:38:56.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5c261640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b2110 con 0x7fce40005180 2023-10-21T17:38:56.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5a7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 0x7fce54147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5a7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 0x7fce54147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34148/0 (socket says 172.21.15.53:34148) 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.801+0000 7fce5a7fe640 1 -- 172.21.15.53:0/4148109890 learned_addr learned my addr 172.21.15.53:0/4148109890 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce597fc640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce5415d470 0x7fce54070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce40005180 0x7fce540b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce40005180 msgr2=0x7fce540b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce40005180 0x7fce540b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce5415d470 msgr2=0x7fce54070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:38:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce5415d470 0x7fce54070a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540b3830 con 0x7fce54070fd0 2023-10-21T17:38:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce5a7fe640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 0x7fce54147b20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000f80 tx=0x7fce4c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:38:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c001930 con 0x7fce54070fd0 2023-10-21T17:38:56.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c004b60 con 0x7fce54070fd0 2023-10-21T17:38:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.802+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0347b0 con 0x7fce54070fd0 2023-10-21T17:38:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.803+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce540b2370 con 0x7fce54070fd0 2023-10-21T17:38:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.803+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540b2930 con 0x7fce54070fd0 2023-10-21T17:38:56.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.804+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce540b3a50 con 0x7fce54070fd0 2023-10-21T17:38:56.812 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.801+0000 7fa2e5c1f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:38:56.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.805+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 127) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fce4c031030 con 0x7fce54070fd0 2023-10-21T17:38:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.806+0000 7fce4a7fc640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:38:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.806+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fce4c0aaf50 con 0x7fce54070fd0 2023-10-21T17:38:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.806+0000 7fce59ffd640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 msgr2=0x7fce3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.806+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:38:56.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:56.810+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce540b3a50 con 0x7fce54070fd0 2023-10-21T17:38:56.852 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.850+0000 7fa2e5c1f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:38:56.854 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.852+0000 7f5474a14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:38:56.905 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:56.904+0000 7f5474a14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:38:56.961 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:56.959+0000 7fa2e5c1f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:38:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:57.007+0000 7fce59ffd640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 msgr2=0x7fce3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:57.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:57.007+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:38:57.021 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.020+0000 7f5474a14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:38:57.023 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.022+0000 7fa2e5c1f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:57.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:57.024+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fce540b36b0 con 0x7fce3806b010 2023-10-21T17:38:57.072 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.071+0000 7fa2e5c1f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:38:57.087 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.086+0000 7f5474a14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:57.129 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.128+0000 7fa2e5c1f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:38:57.139 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.138+0000 7f5474a14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:38:57.179 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.177+0000 7fa2e5c1f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:38:57.198 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.197+0000 7f5474a14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:38:57.250 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.249+0000 7f5474a14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:38:57.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:57.408+0000 7fce59ffd640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 msgr2=0x7fce3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:57.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:57.408+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:38:57.562 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.560+0000 7fa2e5c1f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:38:57.654 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.653+0000 7f5474a14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:38:57.720 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.718+0000 7fa2e5c1f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:38:57.820 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.819+0000 7f5474a14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:38:57.827 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.826+0000 7fa2e5c1f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:38:57.877 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.876+0000 7fa2e5c1f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:38:57.933 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.932+0000 7f5474a14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:38:57.984 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:57.983+0000 7fa2e5c1f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:57.985 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:57.984+0000 7f5474a14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:38:58.091 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:58.089+0000 7fa2e5c1f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:38:58.098 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:58.097+0000 7f5474a14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:58.209 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:58.208+0000 7f5474a14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:38:58.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:58.209+0000 7fce59ffd640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 msgr2=0x7fce3806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3930748558 2023-10-21T17:38:58.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:58.209+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:38:58.246 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:58.245+0000 7fa2e5c1f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:58.295 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:58.293+0000 7fa2e5c1f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:58.375 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:58.373+0000 7f5474a14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:38:58.415 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:58.414+0000 7fa2e5c1f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:38:58.426 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:58.425+0000 7f5474a14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:38:58.550 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:58.549+0000 7f5474a14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:38:59.246 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:59.244+0000 7fa2e5c1f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:38:59.450 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:59.448+0000 7f5474a14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:38:59.538 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:59.536+0000 7fa2e5c1f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:38:59.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:59.592+0000 7fa2e5c1f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:38:59.705 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:38:59.704+0000 7fa2e5c1f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:38:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:59.728+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 128) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fce4c075bf0 con 0x7fce54070fd0 2023-10-21T17:38:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:59.728+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 msgr2=0x7fce3806d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:38:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:38:59.728+0000 7fce4a7fc640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3930748558,v1:172.21.15.124:6833/3930748558] conn(0x7fce3806b010 0x7fce3806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:38:59.791 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:59.789+0000 7f5474a14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:38:59.862 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:59.861+0000 7f5474a14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:38:59.986 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:38:59.984+0000 7f5474a14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:39:00.017 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.015+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.018 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.015+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.018 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.015+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.018 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.015+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.018 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.015+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.016+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.016+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.016+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.016+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.019 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.016+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.032 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.031+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.033 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.031+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.033 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.031+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.033 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.031+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.033 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.031+0000 7fa2642c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.034 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.032+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.034 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.032+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.034 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.032+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.035 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.032+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.035 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:00.032+0000 7fa2612c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.737+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 129) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fce4c0a8eb0 con 0x7fce54070fd0 2023-10-21T17:39:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.739+0000 7fce4a7fc640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fce380358b0 0x7fce38077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:00.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.739+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fce38035c90 con 0x7fce380358b0 2023-10-21T17:39:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.746+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fce380358b0 0x7fce38077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:00.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.754+0000 7fce59ffd640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fce380358b0 0x7fce38077c10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fce500093d0 tx=0x7fce50008040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:00.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.913+0000 7fce4a7fc640 1 -- 172.21.15.53:0/4148109890 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7fce38035c90 con 0x7fce380358b0 2023-10-21T17:39:00.915 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-10-21T17:39:00.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.916+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fce380358b0 msgr2=0x7fce38077c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:00.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fce380358b0 0x7fce38077c10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fce500093d0 tx=0x7fce50008040 comp rx=0 tx=0).stop 2023-10-21T17:39:00.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 msgr2=0x7fce54147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:00.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 --2- 172.21.15.53:0/4148109890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce54070fd0 0x7fce54147b20 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000f80 tx=0x7fce4c001cb0 comp rx=0 tx=0).stop 2023-10-21T17:39:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5a7fe640 1 -- 172.21.15.53:0/4148109890 reap_dead start 2023-10-21T17:39:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 shutdown_connections 2023-10-21T17:39:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 >> 172.21.15.53:0/4148109890 conn(0x7fce540cba60 msgr2=0x7fce5406dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 shutdown_connections 2023-10-21T17:39:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:00.917+0000 7fce5c261640 1 -- 172.21.15.53:0/4148109890 wait complete. 2023-10-21T17:39:00.940 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-10-21T17:39:00.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:39:01.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.175+0000 7f7d1eab4640 1 Processor -- start 2023-10-21T17:39:01.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.175+0000 7f7d1eab4640 1 -- start start 2023-10-21T17:39:01.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1eab4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d180c18d0 0x7f7d180c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1eab4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7d18164d10 0x7f7d180ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1eab4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d18161590 con 0x7f7d180aff70 2023-10-21T17:39:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1eab4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180cc570 con 0x7f7d180c18d0 2023-10-21T17:39:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1eab4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180c94b0 con 0x7f7d18164d10 2023-10-21T17:39:01.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1d2b1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d180c18d0 0x7f7d180c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1d2b1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7d180c18d0 0x7f7d180c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34158/0 (socket says 172.21.15.53:34158) 2023-10-21T17:39:01.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1d2b1640 1 -- 172.21.15.53:0/1513475824 learned_addr learned my addr 172.21.15.53:0/1513475824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:01.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.177+0000 7f7d1e2b3640 1 --2- 172.21.15.53:0/1513475824 >> v2:172.21.15.124:3300/0 conn(0x7f7d18164d10 0x7f7d180ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.178+0000 7f7d1d2b1640 1 -- 172.21.15.53:0/1513475824 >> v2:172.21.15.124:3300/0 conn(0x7f7d18164d10 msgr2=0x7f7d180ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.178+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/1513475824 >> v2:172.21.15.124:3300/0 conn(0x7f7d18164d10 0x7f7d180ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.178+0000 7f7d1d2b1640 1 -- 172.21.15.53:0/1513475824 >> v1:172.21.15.124:6789/0 conn(0x7f7d180aff70 legacy=0x7f7d180b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.178+0000 7f7d1d2b1640 1 -- 172.21.15.53:0/1513475824 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180c9790 con 0x7f7d180c18d0 2023-10-21T17:39:01.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.179+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/1513475824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180c18d0 0x7f7d180c8b40 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cdf00 tx=0x7f7d0802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39af5d23e95b16e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.179+0000 7f7d1cab0640 1 -- 172.21.15.53:0/1513475824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080317e0 con 0x7f7d180c18d0 2023-10-21T17:39:01.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.179+0000 7f7d1cab0640 1 -- 172.21.15.53:0/1513475824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08031940 con 0x7f7d180c18d0 2023-10-21T17:39:01.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.179+0000 7f7d1cab0640 1 -- 172.21.15.53:0/1513475824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08031cd0 con 0x7f7d180c18d0 2023-10-21T17:39:01.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 -- 172.21.15.53:0/1513475824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180c18d0 msgr2=0x7f7d180c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/1513475824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180c18d0 0x7f7d180c8b40 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cdf00 tx=0x7f7d0802b880 comp rx=0 tx=0).stop 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 -- 172.21.15.53:0/1513475824 shutdown_connections 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/1513475824 >> v2:172.21.15.124:3300/0 conn(0x7f7d18164d10 0x7f7d180ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/1513475824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180c18d0 0x7f7d180c8b40 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 -- 172.21.15.53:0/1513475824 >> 172.21.15.53:0/1513475824 conn(0x7f7d1806c050 msgr2=0x7f7d1806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:01.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 -- 172.21.15.53:0/1513475824 shutdown_connections 2023-10-21T17:39:01.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.180+0000 7f7d1eab4640 1 -- 172.21.15.53:0/1513475824 wait complete. 2023-10-21T17:39:01.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.182+0000 7f7d1eab4640 1 Processor -- start 2023-10-21T17:39:01.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.182+0000 7f7d1eab4640 1 -- start start 2023-10-21T17:39:01.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 0x7f7d180c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 0x7f7d180c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c5a30 con 0x7f7d180aff70 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c5d10 con 0x7f7d18164d10 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1eab4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c5ff0 con 0x7f7d180c18d0 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1dab2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1dab2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34166/0 (socket says 172.21.15.53:34166) 2023-10-21T17:39:01.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.183+0000 7f7d1dab2640 1 -- 172.21.15.53:0/2778486435 learned_addr learned my addr 172.21.15.53:0/2778486435 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:01.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 0x7f7d180c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1e2b3640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 0x7f7d180c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 -- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 msgr2=0x7f7d180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 0x7f7d180c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 -- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 msgr2=0x7f7d180c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 0x7f7d180c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 -- 172.21.15.53:0/2778486435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080094d0 con 0x7f7d180aff70 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 0x7f7d180c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1e2b3640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 0x7f7d180c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1dab2640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c450 tx=0x7f7d1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d140052d0 con 0x7f7d180aff70 2023-10-21T17:39:01.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14005430 con 0x7f7d180aff70 2023-10-21T17:39:01.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14012870 con 0x7f7d180aff70 2023-10-21T17:39:01.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.184+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180c62d0 con 0x7f7d180aff70 2023-10-21T17:39:01.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.185+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18163dd0 con 0x7f7d180aff70 2023-10-21T17:39:01.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.186+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18074150 con 0x7f7d180aff70 2023-10-21T17:39:01.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.187+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 129) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f7d140129d0 con 0x7f7d180aff70 2023-10-21T17:39:01.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.188+0000 7f7d0e7fc640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 0x7f7cfc06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.188+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f7d14089440 con 0x7f7d180aff70 2023-10-21T17:39:01.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.189+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 0x7f7cfc06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.190+0000 7f7d1d2b1640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 0x7f7cfc06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d08036000 tx=0x7f7d0803d040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.193+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d140541c0 con 0x7f7d180aff70 2023-10-21T17:39:01.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.409+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d18163fc0 con 0x7f7cfc06ad10 2023-10-21T17:39:01.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.529+0000 7f7d0e7fc640 1 -- 172.21.15.53:0/2778486435 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f7d08031bb0 con 0x7f7cfc06ad10 2023-10-21T17:39:01.531 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-10-21T17:39:01.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 msgr2=0x7f7cfc06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 0x7f7cfc06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d08036000 tx=0x7f7d0803d040 comp rx=0 tx=0).stop 2023-10-21T17:39:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 msgr2=0x7f7d18163890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c450 tx=0x7f7d1400c910 comp rx=0 tx=0).stop 2023-10-21T17:39:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 shutdown_connections 2023-10-21T17:39:01.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7cfc06ad10 0x7f7cfc06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d18164d10 0x7f7d180c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d180c18d0 0x7f7d180c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 --2- 172.21.15.53:0/2778486435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d180aff70 0x7f7d18163890 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 >> 172.21.15.53:0/2778486435 conn(0x7f7d1806c050 msgr2=0x7f7d180ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 shutdown_connections 2023-10-21T17:39:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.533+0000 7f7d1eab4640 1 -- 172.21.15.53:0/2778486435 wait complete. 2023-10-21T17:39:01.556 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-10-21T17:39:01.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:39:01.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.790+0000 7fcecb59e640 1 Processor -- start 2023-10-21T17:39:01.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.790+0000 7fcecb59e640 1 -- start start 2023-10-21T17:39:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.791+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 0x7fcecc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.791+0000 7fcecb59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcecc15dde0 0x7fcecc06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.791+0000 7fcecb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc069d50 con 0x7fcecc0ce290 2023-10-21T17:39:01.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.791+0000 7fcecb59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc06e6c0 con 0x7fcecc15dde0 2023-10-21T17:39:01.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.791+0000 7fcecb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc06c750 con 0x7fcecc065ab0 2023-10-21T17:39:01.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fceca59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcecc0ce290 0x7fcecc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47054/0 (socket says 172.21.15.53:47054) 2023-10-21T17:39:01.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fceca59c640 1 -- 172.21.15.53:0/1665142661 learned_addr learned my addr 172.21.15.53:0/1665142661 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:01.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359114947 0 0) 0x7fcecc069d50 con 0x7fcecc0ce290 2023-10-21T17:39:01.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb8003520 con 0x7fcecc0ce290 2023-10-21T17:39:01.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1424466827 0 0) 0x7fceb8003520 con 0x7fcecc0ce290 2023-10-21T17:39:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 msgr2=0x7fcecc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.792+0000 7fcec9d9b640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 0x7fcecc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 >> v2:172.21.15.124:3300/0 conn(0x7fcecc15dde0 msgr2=0x7fcecc06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 --2- 172.21.15.53:0/1665142661 >> v2:172.21.15.124:3300/0 conn(0x7fcecc15dde0 0x7fcecc06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc06e120 con 0x7fcecc0ce290 2023-10-21T17:39:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcebc0019f0 con 0x7fcecc0ce290 2023-10-21T17:39:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 >> v1:172.21.15.53:6789/0 conn(0x7fcecc0ce290 legacy=0x7fcecc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fceb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fceb80047c0 0x7fceb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 0x7fceb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc06c750 con 0x7fceb8003970 2023-10-21T17:39:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80054e0 con 0x7fceb80047c0 2023-10-21T17:39:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb800bec0 con 0x7fcecc065ab0 2023-10-21T17:39:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fceb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 -- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 msgr2=0x7fceb800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc065ab0 0x7fceb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 -- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fceb80047c0 msgr2=0x7fceb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fceb80047c0 0x7fceb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.793+0000 7fceca59c640 1 -- 172.21.15.53:0/1665142661 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb8003520 con 0x7fceb8003970 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.795+0000 7fceca59c640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fceb8003d90 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00a860 tx=0x7fceb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78b9b67fee4f41e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.795+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc034630 con 0x7fceb8003970 2023-10-21T17:39:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.795+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc02f070 con 0x7fceb8003970 2023-10-21T17:39:01.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.795+0000 7fcec9d9b640 1 -- 172.21.15.53:0/1665142661 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0349f0 con 0x7fceb8003970 2023-10-21T17:39:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 -- 172.21.15.53:0/1665142661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 msgr2=0x7fceb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1665142661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fceb8003d90 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00a860 tx=0x7fceb80057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecad9d640 1 -- 172.21.15.53:0/1665142661 reap_dead start 2023-10-21T17:39:01.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 -- 172.21.15.53:0/1665142661 shutdown_connections 2023-10-21T17:39:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 -- 172.21.15.53:0/1665142661 >> 172.21.15.53:0/1665142661 conn(0x7fcecc0b08d0 msgr2=0x7fcecc0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 -- 172.21.15.53:0/1665142661 shutdown_connections 2023-10-21T17:39:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.796+0000 7fcecb59e640 1 -- 172.21.15.53:0/1665142661 wait complete. 2023-10-21T17:39:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.798+0000 7fcecb59e640 1 Processor -- start 2023-10-21T17:39:01.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.798+0000 7fcecb59e640 1 -- start start 2023-10-21T17:39:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 0x7fcecc160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 0x7fcecc0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc160cd0 con 0x7fceb8003970 2023-10-21T17:39:01.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0b3c20 con 0x7fcecc160850 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fcecb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0b3050 con 0x7fcecc159e30 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fceca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fceca59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34182/0 (socket says 172.21.15.53:34182) 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.799+0000 7fceca59c640 1 -- 172.21.15.53:0/1237729558 learned_addr learned my addr 172.21.15.53:0/1237729558 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fcecad9d640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 0x7fcecc0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fcec1d9b640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 0x7fcecc160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 -- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 msgr2=0x7fcecc160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 0x7fcecc160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 -- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 msgr2=0x7fcecc0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 0x7fcecc0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:01.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 -- 172.21.15.53:0/1237729558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0b32f0 con 0x7fceb8003970 2023-10-21T17:39:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fcec1d9b640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 0x7fcecc160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fcecad9d640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 0x7fcecc0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fceca59c640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02cba0 tx=0x7fcebc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.800+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc034730 con 0x7fceb8003970 2023-10-21T17:39:01.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.801+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc034890 con 0x7fceb8003970 2023-10-21T17:39:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.801+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc034c00 con 0x7fceb8003970 2023-10-21T17:39:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.801+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc0b35d0 con 0x7fceb8003970 2023-10-21T17:39:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.801+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc156f90 con 0x7fceb8003970 2023-10-21T17:39:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.802+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc074150 con 0x7fceb8003970 2023-10-21T17:39:01.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.803+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 130) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fcebc031030 con 0x7fceb8003970 2023-10-21T17:39:01.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.804+0000 7fcec37fe640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 0x7fceac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:01.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.804+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fcebc0ab990 con 0x7fceb8003970 2023-10-21T17:39:01.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.804+0000 7fcec1d9b640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 0x7fceac06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:01.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.806+0000 7fcec1d9b640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 0x7fceac06d5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006fd0 tx=0x7fceb4008040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:01.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:01.808+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcebc079ab0 con 0x7fceb8003970 2023-10-21T17:39:02.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.035+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcecc15d100 con 0x7fceac06b100 2023-10-21T17:39:02.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.187+0000 7fcec37fe640 1 -- 172.21.15.53:0/1237729558 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fceb400ae20 con 0x7fceac06b100 2023-10-21T17:39:02.189 INFO:teuthology.orchestra.run.smithi053.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-10-21T17:39:02.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 msgr2=0x7fceac06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 0x7fceac06d5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006fd0 tx=0x7fceb4008040 comp rx=0 tx=0).stop 2023-10-21T17:39:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 msgr2=0x7fcecc156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02cba0 tx=0x7fcebc001d80 comp rx=0 tx=0).stop 2023-10-21T17:39:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 shutdown_connections 2023-10-21T17:39:02.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fceac06b100 0x7fceac06d5c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcecc160850 0x7fcecc0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcecc159e30 0x7fcecc160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 --2- 172.21.15.53:0/1237729558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb8003970 0x7fcecc156a50 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 >> 172.21.15.53:0/1237729558 conn(0x7fcecc0b08d0 msgr2=0x7fcecc09e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:02.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 shutdown_connections 2023-10-21T17:39:02.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.191+0000 7fcecb59e640 1 -- 172.21.15.53:0/1237729558 wait complete. 2023-10-21T17:39:02.213 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-10-21T17:39:02.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-21T17:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.451+0000 7f863b59e640 1 Processor -- start 2023-10-21T17:39:02.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.451+0000 7f863b59e640 1 -- start start 2023-10-21T17:39:02.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.452+0000 7f863b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f863c0cf2b0 0x7f863c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 0x7f863c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863b59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c0cd520 con 0x7f863c0d1100 2023-10-21T17:39:02.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863b59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0d1910 con 0x7f863c0cf2b0 2023-10-21T17:39:02.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0ce080 con 0x7f863c0cf690 2023-10-21T17:39:02.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f863c0cf2b0 0x7f863c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f863c0cf2b0 0x7f863c0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34188/0 (socket says 172.21.15.53:34188) 2023-10-21T17:39:02.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 -- 172.21.15.53:0/3929163482 learned_addr learned my addr 172.21.15.53:0/3929163482 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f8639d9b640 1 --2- 172.21.15.53:0/3929163482 >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 0x7f863c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 -- 172.21.15.53:0/3929163482 >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 msgr2=0x7f863c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:02.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 --2- 172.21.15.53:0/3929163482 >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 0x7f863c0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 -- 172.21.15.53:0/3929163482 >> v1:172.21.15.124:6789/0 conn(0x7f863c0d1100 legacy=0x7f863c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f863a59c640 1 -- 172.21.15.53:0/3929163482 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0cd750 con 0x7f863c0cf2b0 2023-10-21T17:39:02.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.453+0000 7f8639d9b640 1 --2- 172.21.15.53:0/3929163482 >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 0x7f863c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:02.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.455+0000 7f863a59c640 1 --2- 172.21.15.53:0/3929163482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c0d34d0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f863c08baa0 tx=0x7f862c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c4004ea4d19ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.455+0000 7f863959a640 1 -- 172.21.15.53:0/3929163482 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c02c9f0 con 0x7f863c0cf2b0 2023-10-21T17:39:02.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.455+0000 7f863959a640 1 -- 172.21.15.53:0/3929163482 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c02cb50 con 0x7f863c0cf2b0 2023-10-21T17:39:02.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.455+0000 7f863959a640 1 -- 172.21.15.53:0/3929163482 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0343f0 con 0x7f863c0cf2b0 2023-10-21T17:39:02.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 -- 172.21.15.53:0/3929163482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 msgr2=0x7f863c0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 --2- 172.21.15.53:0/3929163482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c0d34d0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f863c08baa0 tx=0x7f862c02be50 comp rx=0 tx=0).stop 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 -- 172.21.15.53:0/3929163482 shutdown_connections 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 --2- 172.21.15.53:0/3929163482 >> v2:172.21.15.124:3300/0 conn(0x7f863c0cf690 0x7f863c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 --2- 172.21.15.53:0/3929163482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c0d34d0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 -- 172.21.15.53:0/3929163482 >> 172.21.15.53:0/3929163482 conn(0x7f863c09bc40 msgr2=0x7f863c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 -- 172.21.15.53:0/3929163482 shutdown_connections 2023-10-21T17:39:02.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.456+0000 7f863b59e640 1 -- 172.21.15.53:0/3929163482 wait complete. 2023-10-21T17:39:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.458+0000 7f863b59e640 1 Processor -- start 2023-10-21T17:39:02.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.458+0000 7f863b59e640 1 -- start start 2023-10-21T17:39:02.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f863c0cf690 0x7f863c15ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f863c0d1100 0x7f863c14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1f2860 con 0x7f863c0cf2b0 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c15f4b0 con 0x7f863c0cf690 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0b7180 con 0x7f863c0d1100 2023-10-21T17:39:02.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34200/0 (socket says 172.21.15.53:34200) 2023-10-21T17:39:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863a59c640 1 -- 172.21.15.53:0/3760560299 learned_addr learned my addr 172.21.15.53:0/3760560299 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f863ad9d640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f863c0d1100 0x7f863c14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.460+0000 7f8639d9b640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f863c0cf690 0x7f863c15ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 -- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f863c0d1100 msgr2=0x7f863c14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f863c0d1100 0x7f863c14d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 -- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f863c0cf690 msgr2=0x7f863c15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f863c0cf690 0x7f863c15ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 -- 172.21.15.53:0/3760560299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c009b30 con 0x7f863c0cf2b0 2023-10-21T17:39:02.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863a59c640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f80 tx=0x7f862c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c03e450 con 0x7f863c0cf2b0 2023-10-21T17:39:02.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c03e5f0 con 0x7f863c0cf2b0 2023-10-21T17:39:02.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c03e9a0 con 0x7f863c0cf2b0 2023-10-21T17:39:02.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.461+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c0b7460 con 0x7f863c0cf2b0 2023-10-21T17:39:02.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.462+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c0b7a20 con 0x7f863c0cf2b0 2023-10-21T17:39:02.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.463+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c1f5f30 con 0x7f863c0cf2b0 2023-10-21T17:39:02.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.464+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 130) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f862c03eb00 con 0x7f863c0cf2b0 2023-10-21T17:39:02.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.464+0000 7f8632ffd640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 0x7f861c06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.465+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f862c0acd90 con 0x7f863c0cf2b0 2023-10-21T17:39:02.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.465+0000 7f8639d9b640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 0x7f861c06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.466+0000 7f8639d9b640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 0x7f861c06d5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f863c0b46c0 tx=0x7f8624008040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.469+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f863c1f5f30 con 0x7f863c0cf2b0 2023-10-21T17:39:02.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.689+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f863c0cdd10 con 0x7f861c06b130 2023-10-21T17:39:02.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.706+0000 7f8632ffd640 1 -- 172.21.15.53:0/3760560299 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7f862400c050 con 0x7f861c06b130 2023-10-21T17:39:02.708 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:02.708 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-21T17:39:02.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 msgr2=0x7f861c06d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 0x7f861c06d5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f863c0b46c0 tx=0x7f8624008040 comp rx=0 tx=0).stop 2023-10-21T17:39:02.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 msgr2=0x7f863c1511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f80 tx=0x7f862c004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:02.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 shutdown_connections 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f861c06b130 0x7f861c06d5f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f863c0d1100 0x7f863c14d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f863c0cf690 0x7f863c15ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 --2- 172.21.15.53:0/3760560299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f863c0cf2b0 0x7f863c1511f0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 >> 172.21.15.53:0/3760560299 conn(0x7f863c09bc40 msgr2=0x7f863c0d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:02.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 shutdown_connections 2023-10-21T17:39:02.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.709+0000 7f863b59e640 1 -- 172.21.15.53:0/3760560299 wait complete. 2023-10-21T17:39:02.730 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-21T17:39:02.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-21T17:39:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.955+0000 7f44932a9640 1 Processor -- start 2023-10-21T17:39:02.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.955+0000 7f44932a9640 1 -- start start 2023-10-21T17:39:02.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.956+0000 7f44932a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.956+0000 7f44932a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f448c15aa00 0x7f448c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.956+0000 7f44932a9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c15b310 con 0x7f448c06db60 2023-10-21T17:39:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.956+0000 7f44932a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0ca230 con 0x7f448c15aa00 2023-10-21T17:39:02.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.956+0000 7f44932a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06d290 con 0x7f448c06c3a0 2023-10-21T17:39:02.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.957+0000 7f4491045640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.957+0000 7f4491045640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:35368/0 (socket says 172.21.15.53:35368) 2023-10-21T17:39:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.957+0000 7f4491045640 1 -- 172.21.15.53:0/670211779 learned_addr learned my addr 172.21.15.53:0/670211779 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.957+0000 7f4491846640 1 --2- 172.21.15.53:0/670211779 >> v2:172.21.15.124:3300/0 conn(0x7f448c15aa00 0x7f448c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.958+0000 7f4491045640 1 -- 172.21.15.53:0/670211779 >> v2:172.21.15.124:3300/0 conn(0x7f448c15aa00 msgr2=0x7f448c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:02.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.958+0000 7f4491045640 1 --2- 172.21.15.53:0/670211779 >> v2:172.21.15.124:3300/0 conn(0x7f448c15aa00 0x7f448c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.958+0000 7f4491045640 1 -- 172.21.15.53:0/670211779 >> v1:172.21.15.124:6789/0 conn(0x7f448c06db60 legacy=0x7f448c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:02.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.958+0000 7f4491045640 1 -- 172.21.15.53:0/670211779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0c9860 con 0x7f448c06c3a0 2023-10-21T17:39:02.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.959+0000 7f4491045640 1 --2- 172.21.15.53:0/670211779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e50 tx=0x7f448802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff34a30034a4378f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.960+0000 7f4483fff640 1 -- 172.21.15.53:0/670211779 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448802ca70 con 0x7f448c06c3a0 2023-10-21T17:39:02.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.960+0000 7f4483fff640 1 -- 172.21.15.53:0/670211779 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448802cbd0 con 0x7f448c06c3a0 2023-10-21T17:39:02.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.960+0000 7f4483fff640 1 -- 172.21.15.53:0/670211779 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880343f0 con 0x7f448c06c3a0 2023-10-21T17:39:02.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 -- 172.21.15.53:0/670211779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 msgr2=0x7f448c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 --2- 172.21.15.53:0/670211779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e50 tx=0x7f448802bed0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 -- 172.21.15.53:0/670211779 shutdown_connections 2023-10-21T17:39:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 --2- 172.21.15.53:0/670211779 >> v2:172.21.15.124:3300/0 conn(0x7f448c15aa00 0x7f448c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 --2- 172.21.15.53:0/670211779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06c3a0 0x7f448c0c9cf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 -- 172.21.15.53:0/670211779 >> 172.21.15.53:0/670211779 conn(0x7f448c06a5d0 msgr2=0x7f448c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 -- 172.21.15.53:0/670211779 shutdown_connections 2023-10-21T17:39:02.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.961+0000 7f44932a9640 1 -- 172.21.15.53:0/670211779 wait complete. 2023-10-21T17:39:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.964+0000 7f44932a9640 1 Processor -- start 2023-10-21T17:39:02.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.964+0000 7f44932a9640 1 -- start start 2023-10-21T17:39:02.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06db60 0x7f448c14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 0x7f448c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c165ac0 con 0x7f448c06c3a0 2023-10-21T17:39:02.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c14cf40 con 0x7f448c15aa00 2023-10-21T17:39:02.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f44932a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c7ee0 con 0x7f448c06db60 2023-10-21T17:39:02.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f4491045640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f4491045640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34214/0 (socket says 172.21.15.53:34214) 2023-10-21T17:39:02.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.965+0000 7f4491045640 1 -- 172.21.15.53:0/2025904815 learned_addr learned my addr 172.21.15.53:0/2025904815 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:02.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4490844640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06db60 0x7f448c14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491846640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 0x7f448c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 -- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06db60 msgr2=0x7f448c14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06db60 0x7f448c14ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 -- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 msgr2=0x7f448c0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:02.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 0x7f448c0ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:02.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 -- 172.21.15.53:0/2025904815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009b30 con 0x7f448c06c3a0 2023-10-21T17:39:02.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491846640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 0x7f448c0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:02.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.966+0000 7f4491045640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e50 tx=0x7f4488004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.967+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448803e450 con 0x7f448c06c3a0 2023-10-21T17:39:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.967+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448803e5f0 con 0x7f448c06c3a0 2023-10-21T17:39:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.967+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448803e9a0 con 0x7f448c06c3a0 2023-10-21T17:39:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.967+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c14d150 con 0x7f448c06c3a0 2023-10-21T17:39:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.967+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0c8460 con 0x7f448c06c3a0 2023-10-21T17:39:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.968+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c04e160 con 0x7f448c06c3a0 2023-10-21T17:39:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.969+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 130) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f448803eb00 con 0x7f448c06c3a0 2023-10-21T17:39:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.970+0000 7f4481ffb640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 0x7f447006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.970+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f44880aca80 con 0x7f448c06c3a0 2023-10-21T17:39:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.971+0000 7f4490844640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 0x7f447006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.972+0000 7f4490844640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 0x7f447006d5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f448c159ea0 tx=0x7f447c009040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:02.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:02.974+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44880770c0 con 0x7f448c06c3a0 2023-10-21T17:39:03.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.191+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f448c0c9210 con 0x7f447006b130 2023-10-21T17:39:03.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.208+0000 7f4481ffb640 1 -- 172.21.15.53:0/2025904815 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7f447c007db0 con 0x7f447006b130 2023-10-21T17:39:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:03.210 INFO:teuthology.orchestra.run.smithi053.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:39:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-21T17:39:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 msgr2=0x7f447006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 0x7f447006d5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f448c159ea0 tx=0x7f447c009040 comp rx=0 tx=0).stop 2023-10-21T17:39:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 msgr2=0x7f448c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e50 tx=0x7f4488004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 shutdown_connections 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f447006b130 0x7f447006d5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f448c15aa00 0x7f448c0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f448c06db60 0x7f448c14ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 --2- 172.21.15.53:0/2025904815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f448c06c3a0 0x7f448c0c6430 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.211+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 >> 172.21.15.53:0/2025904815 conn(0x7f448c06a5d0 msgr2=0x7f448c0cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:03.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.212+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 shutdown_connections 2023-10-21T17:39:03.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.212+0000 7f44932a9640 1 -- 172.21.15.53:0/2025904815 wait complete. 2023-10-21T17:39:03.233 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-21T00:00:00", "created": "2023-10-21T17:39:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-21T17:39:03.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:39:03.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.468+0000 7fe89b59e640 1 Processor -- start 2023-10-21T17:39:03.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.468+0000 7fe89b59e640 1 -- start start 2023-10-21T17:39:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.469+0000 7fe89b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe89c1599d0 0x7fe89c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.469+0000 7fe89b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe89c0b3c50 0x7fe89c0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.469+0000 7fe89b59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c159db0 con 0x7fe89c0cb410 2023-10-21T17:39:03.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.469+0000 7fe89b59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c071fe0 con 0x7fe89c1599d0 2023-10-21T17:39:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.469+0000 7fe89b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0b44c0 con 0x7fe89c0b3c50 2023-10-21T17:39:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe89c1599d0 0x7fe89c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe89c1599d0 0x7fe89c0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34220/0 (socket says 172.21.15.53:34220) 2023-10-21T17:39:03.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 -- 172.21.15.53:0/187171467 learned_addr learned my addr 172.21.15.53:0/187171467 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:03.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe899d9b640 1 --2- 172.21.15.53:0/187171467 >> v2:172.21.15.124:3300/0 conn(0x7fe89c0b3c50 0x7fe89c0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 -- 172.21.15.53:0/187171467 >> v2:172.21.15.124:3300/0 conn(0x7fe89c0b3c50 msgr2=0x7fe89c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:03.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 --2- 172.21.15.53:0/187171467 >> v2:172.21.15.124:3300/0 conn(0x7fe89c0b3c50 0x7fe89c0caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 -- 172.21.15.53:0/187171467 >> v1:172.21.15.124:6789/0 conn(0x7fe89c0cb410 legacy=0x7fe89c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:03.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.470+0000 7fe89a59c640 1 -- 172.21.15.53:0/187171467 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0cb860 con 0x7fe89c1599d0 2023-10-21T17:39:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.471+0000 7fe89a59c640 1 --2- 172.21.15.53:0/187171467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0cdcd0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0709e0 tx=0x7fe88c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cc4a5a231d68b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.472+0000 7fe89959a640 1 -- 172.21.15.53:0/187171467 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c02ca80 con 0x7fe89c1599d0 2023-10-21T17:39:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.472+0000 7fe89959a640 1 -- 172.21.15.53:0/187171467 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c02cbe0 con 0x7fe89c1599d0 2023-10-21T17:39:03.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.472+0000 7fe89959a640 1 -- 172.21.15.53:0/187171467 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0343f0 con 0x7fe89c1599d0 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 -- 172.21.15.53:0/187171467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 msgr2=0x7fe89c0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 --2- 172.21.15.53:0/187171467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0cdcd0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0709e0 tx=0x7fe88c02bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 -- 172.21.15.53:0/187171467 shutdown_connections 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 --2- 172.21.15.53:0/187171467 >> v2:172.21.15.124:3300/0 conn(0x7fe89c0b3c50 0x7fe89c0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 --2- 172.21.15.53:0/187171467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0cdcd0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 -- 172.21.15.53:0/187171467 >> 172.21.15.53:0/187171467 conn(0x7fe89c06c050 msgr2=0x7fe89c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 -- 172.21.15.53:0/187171467 shutdown_connections 2023-10-21T17:39:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.473+0000 7fe89b59e640 1 -- 172.21.15.53:0/187171467 wait complete. 2023-10-21T17:39:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.475+0000 7fe89b59e640 1 Processor -- start 2023-10-21T17:39:03.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.475+0000 7fe89b59e640 1 -- start start 2023-10-21T17:39:03.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.476+0000 7fe89b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 0x7fe89c156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 0x7fe89c0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0abe30 con 0x7fe89c1599d0 2023-10-21T17:39:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c1f2380 con 0x7fe89c0cb410 2023-10-21T17:39:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0ab1b0 con 0x7fe89c0b3c50 2023-10-21T17:39:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34236/0 (socket says 172.21.15.53:34236) 2023-10-21T17:39:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 -- 172.21.15.53:0/370854823 learned_addr learned my addr 172.21.15.53:0/370854823 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89a59c640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 0x7fe89c156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe899d9b640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 0x7fe89c0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 -- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 msgr2=0x7fe89c156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 0x7fe89c156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 -- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 msgr2=0x7fe89c0acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 0x7fe89c0acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89ad9d640 1 -- 172.21.15.53:0/370854823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009b30 con 0x7fe89c1599d0 2023-10-21T17:39:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe89a59c640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 0x7fe89c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.477+0000 7fe899d9b640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 0x7fe89c0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe89ad9d640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c390 tx=0x7fe89400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940035f0 con 0x7fe89c1599d0 2023-10-21T17:39:03.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894003750 con 0x7fe89c1599d0 2023-10-21T17:39:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c0ab490 con 0x7fe89c1599d0 2023-10-21T17:39:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894003b00 con 0x7fe89c1599d0 2023-10-21T17:39:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.478+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c0aba50 con 0x7fe89c1599d0 2023-10-21T17:39:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.480+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c074150 con 0x7fe89c1599d0 2023-10-21T17:39:03.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.480+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 130) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe894003c60 con 0x7fe89c1599d0 2023-10-21T17:39:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.481+0000 7fe88affd640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 0x7fe87c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.482+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fe8940897c0 con 0x7fe89c1599d0 2023-10-21T17:39:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.482+0000 7fe89a59c640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 0x7fe87c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.484+0000 7fe89a59c640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 0x7fe87c06d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe88c035000 tx=0x7fe88c030000 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.486+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe894053e00 con 0x7fe89c1599d0 2023-10-21T17:39:03.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.711+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe89c0716b0 con 0x7fe87c06b100 2023-10-21T17:39:03.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.854+0000 7fe88affd640 1 -- 172.21.15.53:0/370854823 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe88c040450 con 0x7fe87c06b100 2023-10-21T17:39:03.856 INFO:teuthology.orchestra.run.smithi053.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-10-21T17:39:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 msgr2=0x7fe87c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 0x7fe87c06d5c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe88c035000 tx=0x7fe88c030000 comp rx=0 tx=0).stop 2023-10-21T17:39:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 msgr2=0x7fe89c0ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c390 tx=0x7fe89400c850 comp rx=0 tx=0).stop 2023-10-21T17:39:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 shutdown_connections 2023-10-21T17:39:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe87c06b100 0x7fe87c06d5c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe89c1599d0 0x7fe89c0ad020 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe89c0cb410 0x7fe89c0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 --2- 172.21.15.53:0/370854823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe89c0b3c50 0x7fe89c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 >> 172.21.15.53:0/370854823 conn(0x7fe89c06c050 msgr2=0x7fe89c0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:03.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 shutdown_connections 2023-10-21T17:39:03.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:03.858+0000 7fe89b59e640 1 -- 172.21.15.53:0/370854823 wait complete. 2023-10-21T17:39:03.881 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-10-21T17:39:03.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:39:04.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.116+0000 7f7ece4d4640 1 Processor -- start 2023-10-21T17:39:04.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.116+0000 7f7ece4d4640 1 -- start start 2023-10-21T17:39:04.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.118+0000 7f7ece4d4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7ec80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.118+0000 7f7ece4d4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec8071140 con 0x7f7ec80cb450 2023-10-21T17:39:04.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.118+0000 7f7ece4d4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec806e150 con 0x7f7ec8160660 2023-10-21T17:39:04.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.118+0000 7f7ece4d4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80cb8c0 con 0x7f7ec8070cc0 2023-10-21T17:39:04.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ecca71640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7ec80cb450 0x7f7ec80b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59382/0 (socket says 172.21.15.53:59382) 2023-10-21T17:39:04.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ecca71640 1 -- 172.21.15.53:0/2030270716 learned_addr learned my addr 172.21.15.53:0/2030270716 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:04.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7ec80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647189852 0 0) 0x7f7ec8071140 con 0x7f7ec80cb450 2023-10-21T17:39:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb0003520 con 0x7f7ec80cb450 2023-10-21T17:39:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.119+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849676270 0 0) 0x7f7eb0003520 con 0x7f7ec80cb450 2023-10-21T17:39:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 msgr2=0x7f7ec80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7ec80caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 >> v1:172.21.15.124:6789/0 conn(0x7f7ec8160660 legacy=0x7f7ec80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec806de70 con 0x7f7ec80cb450 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7ec00019f0 con 0x7f7ec80cb450 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 >> v1:172.21.15.53:6789/0 conn(0x7f7ec80cb450 legacy=0x7f7ec80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7eb00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb0003f70 0x7f7eb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb0004d00 0x7f7eb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec80cb8c0 con 0x7f7eb0003f70 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb000bac0 con 0x7f7eb0004d00 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb000bc80 con 0x7f7ec8070cc0 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb0003f70 0x7f7eb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ecca71640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb0004d00 0x7f7eb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.120+0000 7f7ec7fff640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7eb00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.121+0000 7f7ec77fe640 1 -- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 msgr2=0x7f7eb00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.121+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7ec8070cc0 0x7f7eb00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.121+0000 7f7ec77fe640 1 -- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb0004d00 msgr2=0x7f7eb0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.121+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eb0004d00 0x7f7eb0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.121+0000 7f7ec77fe640 1 -- 172.21.15.53:0/2030270716 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0003520 con 0x7f7eb0003f70 2023-10-21T17:39:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.122+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb0003f70 0x7f7eb00047c0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ac0 tx=0x7f7eb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a83d6ad012486e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.122+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc02c600 con 0x7f7eb0003f70 2023-10-21T17:39:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.122+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc02c760 con 0x7f7eb0003f70 2023-10-21T17:39:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.122+0000 7f7ec6ffd640 1 -- 172.21.15.53:0/2030270716 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc02caf0 con 0x7f7eb0003f70 2023-10-21T17:39:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 -- 172.21.15.53:0/2030270716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb0003f70 msgr2=0x7f7eb00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/2030270716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb0003f70 0x7f7eb00047c0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ac0 tx=0x7f7eb00057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ecca71640 1 -- 172.21.15.53:0/2030270716 reap_dead start 2023-10-21T17:39:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 -- 172.21.15.53:0/2030270716 shutdown_connections 2023-10-21T17:39:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 -- 172.21.15.53:0/2030270716 >> 172.21.15.53:0/2030270716 conn(0x7f7ec806a5d0 msgr2=0x7f7ec80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 -- 172.21.15.53:0/2030270716 shutdown_connections 2023-10-21T17:39:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.123+0000 7f7ece4d4640 1 -- 172.21.15.53:0/2030270716 wait complete. 2023-10-21T17:39:04.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.125+0000 7f7ece4d4640 1 Processor -- start 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.125+0000 7f7ece4d4640 1 -- start start 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 0x7f7ec814b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 0x7f7ec8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec814cac0 con 0x7f7ec8157ae0 2023-10-21T17:39:04.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec814beb0 con 0x7f7ec814cf50 2023-10-21T17:39:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ece4d4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec814c190 con 0x7f7eb0003f70 2023-10-21T17:39:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47052/0 (socket says 172.21.15.53:47052) 2023-10-21T17:39:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.126+0000 7f7ec77fe640 1 -- 172.21.15.53:0/4217325202 learned_addr learned my addr 172.21.15.53:0/4217325202 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:04.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec7fff640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 0x7f7ec814b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ecca71640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 0x7f7ec8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 -- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 msgr2=0x7f7ec814b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 0x7f7ec814b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 -- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 msgr2=0x7f7ec8150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 0x7f7ec8150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 -- 172.21.15.53:0/4217325202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec814c470 con 0x7f7ec8157ae0 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec7fff640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 0x7f7ec814b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ecca71640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 0x7f7ec8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec77fe640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ac0 tx=0x7f7ebc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc003810 con 0x7f7ec8157ae0 2023-10-21T17:39:04.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc003970 con 0x7f7ec8157ae0 2023-10-21T17:39:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc003d00 con 0x7f7ec8157ae0 2023-10-21T17:39:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.127+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec814c750 con 0x7f7ec8157ae0 2023-10-21T17:39:04.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.128+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec814b390 con 0x7f7ec8157ae0 2023-10-21T17:39:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.129+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec814c8e0 con 0x7f7ec8157ae0 2023-10-21T17:39:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.130+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 130) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f7ebc040450 con 0x7f7ec8157ae0 2023-10-21T17:39:04.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.131+0000 7f7ec4ff9640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 0x7f7eb406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.131+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f7ebc0ac090 con 0x7f7ec8157ae0 2023-10-21T17:39:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.131+0000 7f7ec7fff640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 0x7f7eb406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.133+0000 7f7ec7fff640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 0x7f7eb406d5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001630 tx=0x7f7eb8009040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.136+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ebc076760 con 0x7f7ec8157ae0 2023-10-21T17:39:04.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.359+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 131) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f7ebc076b30 con 0x7f7ec8157ae0 2023-10-21T17:39:04.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.360+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec80c9110 con 0x7f7eb406b100 2023-10-21T17:39:04.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.512+0000 7f7ec4ff9640 1 -- 172.21.15.53:0/4217325202 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7f7eb800d050 con 0x7f7eb406b100 2023-10-21T17:39:04.514 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-21T17:39:04.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 msgr2=0x7f7eb406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 0x7f7eb406d5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001630 tx=0x7f7eb8009040 comp rx=0 tx=0).stop 2023-10-21T17:39:04.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 msgr2=0x7f7ec814ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009ac0 tx=0x7f7ebc004730 comp rx=0 tx=0).stop 2023-10-21T17:39:04.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 shutdown_connections 2023-10-21T17:39:04.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f7eb406b100 0x7f7eb406d5c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7ec814cf50 0x7f7ec8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec8157ae0 0x7f7ec814ae50 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 --2- 172.21.15.53:0/4217325202 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eb0003f70 0x7f7ec814b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 >> 172.21.15.53:0/4217325202 conn(0x7f7ec806a5d0 msgr2=0x7f7ec80ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:04.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 shutdown_connections 2023-10-21T17:39:04.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.516+0000 7f7ece4d4640 1 -- 172.21.15.53:0/4217325202 wait complete. 2023-10-21T17:39:04.541 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-21T17:39:04.542 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-21T17:39:04.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.780+0000 7fb6d8000640 1 Processor -- start 2023-10-21T17:39:04.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.780+0000 7fb6d8000640 1 -- start start 2023-10-21T17:39:04.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d8000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb6d0161690 0x7fb6d006dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d8000640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6d00b2f30 0x7fb6d00cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d8000640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d006cc50 con 0x7fb6d00cb150 2023-10-21T17:39:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d8000640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d00cbb40 con 0x7fb6d00b2f30 2023-10-21T17:39:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d8000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d006cf30 con 0x7fb6d0161690 2023-10-21T17:39:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d659d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb6d00b2f30 0x7fb6d00cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d559b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb6d00cb150 0x7fb6d00702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59396/0 (socket says 172.21.15.53:59396) 2023-10-21T17:39:04.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.782+0000 7fb6d559b640 1 -- 172.21.15.53:0/350877502 learned_addr learned my addr 172.21.15.53:0/350877502 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:04.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d5d9c640 1 --2- 172.21.15.53:0/350877502 >> v2:172.21.15.124:3300/0 conn(0x7fb6d0161690 0x7fb6d006dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d659d640 1 -- 172.21.15.53:0/350877502 >> v2:172.21.15.124:3300/0 conn(0x7fb6d0161690 msgr2=0x7fb6d006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d659d640 1 --2- 172.21.15.53:0/350877502 >> v2:172.21.15.124:3300/0 conn(0x7fb6d0161690 0x7fb6d006dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d659d640 1 -- 172.21.15.53:0/350877502 >> v1:172.21.15.53:6789/0 conn(0x7fb6d00cb150 legacy=0x7fb6d00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d659d640 1 -- 172.21.15.53:0/350877502 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d006d210 con 0x7fb6d00b2f30 2023-10-21T17:39:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.783+0000 7fb6d4d9a640 1 -- 172.21.15.53:0/350877502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420418142 0 0) 0x7fb6d006cc50 con 0x7fb6d00cb150 2023-10-21T17:39:04.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.784+0000 7fb6d659d640 1 --2- 172.21.15.53:0/350877502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00cb600 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00b0d50 tx=0x7fb6c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db807022ac85c4c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.784+0000 7fb6d4d9a640 1 -- 172.21.15.53:0/350877502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8031830 con 0x7fb6d00b2f30 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.784+0000 7fb6d4d9a640 1 -- 172.21.15.53:0/350877502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8031990 con 0x7fb6d00b2f30 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.784+0000 7fb6d4d9a640 1 -- 172.21.15.53:0/350877502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8031d20 con 0x7fb6d00b2f30 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 -- 172.21.15.53:0/350877502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 msgr2=0x7fb6d00cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 --2- 172.21.15.53:0/350877502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00cb600 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00b0d50 tx=0x7fb6c802b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 -- 172.21.15.53:0/350877502 shutdown_connections 2023-10-21T17:39:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 --2- 172.21.15.53:0/350877502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00cb600 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 --2- 172.21.15.53:0/350877502 >> v2:172.21.15.124:3300/0 conn(0x7fb6d0161690 0x7fb6d006dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 -- 172.21.15.53:0/350877502 >> 172.21.15.53:0/350877502 conn(0x7fb6d006a5d0 msgr2=0x7fb6d00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 -- 172.21.15.53:0/350877502 shutdown_connections 2023-10-21T17:39:04.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.785+0000 7fb6d8000640 1 -- 172.21.15.53:0/350877502 wait complete. 2023-10-21T17:39:04.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.787+0000 7fb6d8000640 1 Processor -- start 2023-10-21T17:39:04.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.787+0000 7fb6d8000640 1 -- start start 2023-10-21T17:39:04.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6d00cb150 0x7fb6d01f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 0x7fb6d0161f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d01594b0 con 0x7fb6d00b2f30 2023-10-21T17:39:04.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00c8640 con 0x7fb6d0161690 2023-10-21T17:39:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d8000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d01588a0 con 0x7fb6d00cb150 2023-10-21T17:39:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47062/0 (socket says 172.21.15.53:47062) 2023-10-21T17:39:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 -- 172.21.15.53:0/2944680923 learned_addr learned my addr 172.21.15.53:0/2944680923 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:04.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d559b640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6d00cb150 0x7fb6d01f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d659d640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 0x7fb6d0161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 -- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6d00cb150 msgr2=0x7fb6d01f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6d00cb150 0x7fb6d01f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 -- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 msgr2=0x7fb6d0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:04.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 0x7fb6d0161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d5d9c640 1 -- 172.21.15.53:0/2944680923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009400 con 0x7fb6d00b2f30 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.789+0000 7fb6d659d640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 0x7fb6d0161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6d5d9c640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc007e40 tx=0x7fb6cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc014620 con 0x7fb6d00b2f30 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00ee90 con 0x7fb6d00b2f30 2023-10-21T17:39:04.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d0158b80 con 0x7fb6d00b2f30 2023-10-21T17:39:04.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0149c0 con 0x7fb6d00b2f30 2023-10-21T17:39:04.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.790+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d0159140 con 0x7fb6d00b2f30 2023-10-21T17:39:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.791+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d004e160 con 0x7fb6d00b2f30 2023-10-21T17:39:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.792+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 131) v1 ==== 89507+0+0 (secure 0 0 0) 0x7fb6cc014b20 con 0x7fb6d00b2f30 2023-10-21T17:39:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.793+0000 7fb6c67fc640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 0x7fb6b406d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.794+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fb6cc08aa80 con 0x7fb6d00b2f30 2023-10-21T17:39:04.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.794+0000 7fb6d559b640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 0x7fb6b406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:04.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.795+0000 7fb6d559b640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 0x7fb6b406d680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007fd0 tx=0x7fb6c0009040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:04.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:04.798+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6cc055130 con 0x7fb6d00b2f30 2023-10-21T17:39:05.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.014+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 --> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6d00b1990 con 0x7fb6b406b1c0 2023-10-21T17:39:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.182+0000 7fb6c67fc640 1 -- 172.21.15.53:0/2944680923 <== mgr.9277 v2:172.21.15.124:6832/3186075298 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7fb6c0001350 con 0x7fb6b406b1c0 2023-10-21T17:39:05.184 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-10-21T17:39:05.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.185+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 msgr2=0x7fb6b406d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.185+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 0x7fb6b406d680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0007fd0 tx=0x7fb6c0009040 comp rx=0 tx=0).stop 2023-10-21T17:39:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.185+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 msgr2=0x7fb6d00c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.185+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc007e40 tx=0x7fb6cc00e4b0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 shutdown_connections 2023-10-21T17:39:05.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fb6b406b1c0 0x7fb6b406d680 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6d0161690 0x7fb6d0161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6d00cb150 0x7fb6d01f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 --2- 172.21.15.53:0/2944680923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6d00b2f30 0x7fb6d00c8100 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 >> 172.21.15.53:0/2944680923 conn(0x7fb6d006a5d0 msgr2=0x7fb6d00ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:05.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 shutdown_connections 2023-10-21T17:39:05.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.186+0000 7fb6d8000640 1 -- 172.21.15.53:0/2944680923 wait complete. 2023-10-21T17:39:05.209 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-10-21T17:39:05.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:05.210 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-21T17:39:05.270 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:39:05.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.504+0000 7f6b488ce640 1 Processor -- start 2023-10-21T17:39:05.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.505+0000 7f6b488ce640 1 -- start start 2023-10-21T17:39:05.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b488ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b441599d0 0x7f6b440cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b488ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b440b3c50 0x7f6b440caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b488ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b440cb8a0 con 0x7f6b440cb410 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b488ce640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b44159db0 con 0x7f6b441599d0 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b488ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b44071fe0 con 0x7f6b440b3c50 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b42d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b441599d0 0x7f6b440cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:05.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b4359e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6b440cb410 0x7f6b440c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59398/0 (socket says 172.21.15.53:59398) 2023-10-21T17:39:05.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b42d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b441599d0 0x7f6b440cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47076/0 (socket says 172.21.15.53:47076) 2023-10-21T17:39:05.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.506+0000 7f6b4359e640 1 -- 172.21.15.53:0/2555457164 learned_addr learned my addr 172.21.15.53:0/2555457164 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:05.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b3bfff640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b440b3c50 0x7f6b440caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:05.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b42d9d640 1 -- 172.21.15.53:0/2555457164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b440b3c50 msgr2=0x7f6b440caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:05.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b42d9d640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b440b3c50 0x7f6b440caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b42d9d640 1 -- 172.21.15.53:0/2555457164 >> v1:172.21.15.53:6789/0 conn(0x7f6b440cb410 legacy=0x7f6b440c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b4259c640 1 -- 172.21.15.53:0/2555457164 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528026577 0 0) 0x7f6b440cb8a0 con 0x7f6b440cb410 2023-10-21T17:39:05.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.507+0000 7f6b42d9d640 1 -- 172.21.15.53:0/2555457164 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b440b44c0 con 0x7f6b441599d0 2023-10-21T17:39:05.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.508+0000 7f6b42d9d640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b441599d0 0x7f6b440cdcd0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f6b340097a0 tx=0x7f6b3402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbff3a6292c17c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:05.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.508+0000 7f6b4259c640 1 -- 172.21.15.53:0/2555457164 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3402d040 con 0x7f6b441599d0 2023-10-21T17:39:05.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.508+0000 7f6b4259c640 1 -- 172.21.15.53:0/2555457164 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34031810 con 0x7f6b441599d0 2023-10-21T17:39:05.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.508+0000 7f6b4259c640 1 -- 172.21.15.53:0/2555457164 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34031ba0 con 0x7f6b441599d0 2023-10-21T17:39:05.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 -- 172.21.15.53:0/2555457164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b441599d0 msgr2=0x7f6b440cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b441599d0 0x7f6b440cdcd0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f6b340097a0 tx=0x7f6b3402b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 -- 172.21.15.53:0/2555457164 shutdown_connections 2023-10-21T17:39:05.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b440b3c50 0x7f6b440caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 --2- 172.21.15.53:0/2555457164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b441599d0 0x7f6b440cdcd0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 -- 172.21.15.53:0/2555457164 >> 172.21.15.53:0/2555457164 conn(0x7f6b4406c050 msgr2=0x7f6b4406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:05.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 -- 172.21.15.53:0/2555457164 shutdown_connections 2023-10-21T17:39:05.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.509+0000 7f6b488ce640 1 -- 172.21.15.53:0/2555457164 wait complete. 2023-10-21T17:39:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.511+0000 7f6b488ce640 1 Processor -- start 2023-10-21T17:39:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.511+0000 7f6b488ce640 1 -- start start 2023-10-21T17:39:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b440cb410 0x7f6b440ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b441599d0 0x7f6b440aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440a74c0 con 0x7f6b440b3c50 2023-10-21T17:39:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440ad0b0 con 0x7f6b440cb410 2023-10-21T17:39:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b488ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440ad390 con 0x7f6b441599d0 2023-10-21T17:39:05.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b42d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.512+0000 7f6b42d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47078/0 (socket says 172.21.15.53:47078) 2023-10-21T17:39:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 -- 172.21.15.53:0/1010581359 learned_addr learned my addr 172.21.15.53:0/1010581359 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 -- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b441599d0 msgr2=0x7f6b440aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b441599d0 0x7f6b440aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 -- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b440cb410 msgr2=0x7f6b440ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:05.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b440cb410 0x7f6b440ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:05.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 -- 172.21.15.53:0/1010581359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34009400 con 0x7f6b440b3c50 2023-10-21T17:39:05.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b42d9d640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f6b34000c00 tx=0x7f6b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:05.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3403ad60 con 0x7f6b440b3c50 2023-10-21T17:39:05.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.513+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b340443f0 con 0x7f6b440b3c50 2023-10-21T17:39:05.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.514+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b440a76d0 con 0x7f6b440b3c50 2023-10-21T17:39:05.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.514+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340447a0 con 0x7f6b440b3c50 2023-10-21T17:39:05.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.514+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b440acd80 con 0x7f6b440b3c50 2023-10-21T17:39:05.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.515+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b44074150 con 0x7f6b440b3c50 2023-10-21T17:39:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.516+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 131) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f6b3404d450 con 0x7f6b440b3c50 2023-10-21T17:39:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.517+0000 7f6b3b7fe640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 0x7f6b2406d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:05.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.517+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f6b340ac4e0 con 0x7f6b440b3c50 2023-10-21T17:39:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.517+0000 7f6b3bfff640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 0x7f6b2406d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.518+0000 7f6b3bfff640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 0x7f6b2406d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b440ac630 tx=0x7f6b2c008040 comp rx=0 tx=0).ready entity=mgr.9277 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:05.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.521+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b34076a80 con 0x7f6b440b3c50 2023-10-21T17:39:05.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:05.767+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f6b4415b6e0 con 0x7f6b440b3c50 2023-10-21T17:39:06.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.368+0000 7f6b3b7fe640 1 -- 172.21.15.53:0/1010581359 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v132) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6b3407a0a0 con 0x7f6b440b3c50 2023-10-21T17:39:06.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 msgr2=0x7f6b2406d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 0x7f6b2406d650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6b440ac630 tx=0x7f6b2c008040 comp rx=0 tx=0).stop 2023-10-21T17:39:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 msgr2=0x7f6b440ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f6b34000c00 tx=0x7f6b340046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 shutdown_connections 2023-10-21T17:39:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f6b2406b190 0x7f6b2406d650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b441599d0 0x7f6b440aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b440cb410 0x7f6b440ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 --2- 172.21.15.53:0/1010581359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b440b3c50 0x7f6b440ae760 secure :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f6b34000c00 tx=0x7f6b340046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.372+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 >> 172.21.15.53:0/1010581359 conn(0x7f6b4406c050 msgr2=0x7f6b440cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:06.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.373+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 shutdown_connections 2023-10-21T17:39:06.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.373+0000 7f6b488ce640 1 -- 172.21.15.53:0/1010581359 wait complete. 2023-10-21T17:39:06.396 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:39:06.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:39:06.501 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:39:06.588 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:39:06.589 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:06.589 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:06.644 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:06.644 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:06.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:06.901 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:06.900+0000 7f5fce412040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:39:06.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:06.903+0000 7f177b023040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:39:06.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.940+0000 7fe4c7fff640 1 Processor -- start 2023-10-21T17:39:06.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.941+0000 7fe4c7fff640 1 -- start start 2023-10-21T17:39:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.942+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4c80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.942+0000 7fe4c7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4c80d07c0 0x7fe4c80b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.942+0000 7fe4c7fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c80ceb00 con 0x7fe4c815cdf0 2023-10-21T17:39:06.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.942+0000 7fe4c7fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80ce610 con 0x7fe4c80d07c0 2023-10-21T17:39:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.942+0000 7fe4c7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815d1d0 con 0x7fe4c80d1640 2023-10-21T17:39:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe4c815cdf0 0x7fe4c80d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59420/0 (socket says 172.21.15.53:59420) 2023-10-21T17:39:06.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c6ffd640 1 -- 172.21.15.53:0/2133923050 learned_addr learned my addr 172.21.15.53:0/2133923050 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:06.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/2133923050 >> v2:172.21.15.124:3300/0 conn(0x7fe4c80d07c0 0x7fe4c80b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c67fc640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4c80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337889427 0 0) 0x7fe4c80ceb00 con 0x7fe4c815cdf0 2023-10-21T17:39:06.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.943+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b4003520 con 0x7fe4c815cdf0 2023-10-21T17:39:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946744906 0 0) 0x7fe4b4003520 con 0x7fe4c815cdf0 2023-10-21T17:39:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 msgr2=0x7fe4c80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4c80ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 >> v2:172.21.15.124:3300/0 conn(0x7fe4c80d07c0 msgr2=0x7fe4c80b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:06.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 --2- 172.21.15.53:0/2133923050 >> v2:172.21.15.124:3300/0 conn(0x7fe4c80d07c0 0x7fe4c80b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80d0c50 con 0x7fe4c815cdf0 2023-10-21T17:39:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe4b80019f0 con 0x7fe4c815cdf0 2023-10-21T17:39:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 >> v1:172.21.15.53:6789/0 conn(0x7fe4c815cdf0 legacy=0x7fe4c80d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4b4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4b4003f40 0x7fe4b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80ce610 con 0x7fe4b4004d00 2023-10-21T17:39:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c815d1d0 con 0x7fe4b4003f40 2023-10-21T17:39:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b400bab0 con 0x7fe4c80d1640 2023-10-21T17:39:06.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4b4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.944+0000 7fe4c77fe640 1 -- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 msgr2=0x7fe4b4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4b4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c77fe640 1 -- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4b4003f40 msgr2=0x7fe4b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:06.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c67fc640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4b4003f40 0x7fe4b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4b4003f40 0x7fe4b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c77fe640 1 -- 172.21.15.53:0/2133923050 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4003520 con 0x7fe4b4004d00 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c67fc640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4b4003f40 0x7fe4b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.945+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80d1640 0x7fe4b4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a05ac2fccc26d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:06.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002c6f0 con 0x7fe4b4004d00 2023-10-21T17:39:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c002c850 con 0x7fe4b4004d00 2023-10-21T17:39:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c5ffb640 1 -- 172.21.15.53:0/2133923050 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002cbe0 con 0x7fe4b4004d00 2023-10-21T17:39:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c7fff640 1 -- 172.21.15.53:0/2133923050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4b4004d00 msgr2=0x7fe4b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.946+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/2133923050 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4b4004d00 0x7fe4b4005180 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4b40057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.947+0000 7fe4c77fe640 1 -- 172.21.15.53:0/2133923050 reap_dead start 2023-10-21T17:39:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.947+0000 7fe4c7fff640 1 -- 172.21.15.53:0/2133923050 shutdown_connections 2023-10-21T17:39:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.947+0000 7fe4c7fff640 1 -- 172.21.15.53:0/2133923050 >> 172.21.15.53:0/2133923050 conn(0x7fe4c809bcb0 msgr2=0x7fe4c80d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.947+0000 7fe4c7fff640 1 -- 172.21.15.53:0/2133923050 shutdown_connections 2023-10-21T17:39:06.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.947+0000 7fe4c7fff640 1 -- 172.21.15.53:0/2133923050 wait complete. 2023-10-21T17:39:06.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.948+0000 7fe4c7fff640 1 Processor -- start 2023-10-21T17:39:06.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.949+0000 7fe4c7fff640 1 -- start start 2023-10-21T17:39:06.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.949+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 0x7fe4c80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.958 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.949+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80cb750 0x7fe4c80c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.949+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.949+0000 7fe4c7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80ca6d0 con 0x7fe4c80cc190 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80c9ac0 con 0x7fe4c8156040 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80c9da0 con 0x7fe4c80cb750 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47112/0 (socket says 172.21.15.53:47112) 2023-10-21T17:39:06.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 -- 172.21.15.53:0/1004683119 learned_addr learned my addr 172.21.15.53:0/1004683119 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:06.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c67fc640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80cb750 0x7fe4c80c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 0x7fe4c80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:06.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80cb750 msgr2=0x7fe4c80c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80cb750 0x7fe4c80c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 msgr2=0x7fe4c80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:06.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 0x7fe4c80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:06.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c77fe640 1 -- 172.21.15.53:0/1004683119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80ca080 con 0x7fe4c80cc190 2023-10-21T17:39:06.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.950+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 0x7fe4c80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:06.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4c77fe640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:06.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002c6f0 con 0x7fe4c80cc190 2023-10-21T17:39:06.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0004470 con 0x7fe4c80cc190 2023-10-21T17:39:06.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0003850 con 0x7fe4c80cc190 2023-10-21T17:39:06.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c80ca360 con 0x7fe4c80cc190 2023-10-21T17:39:06.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.951+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c81f58c0 con 0x7fe4c80cc190 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.952+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c80ca840 con 0x7fe4c80cc190 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.954+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fe4c003b050 con 0x7fe4c80cc190 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.955+0000 7fe4bf7fe640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 0x7fe4a806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.955+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fe4c00ab810 con 0x7fe4c80cc190 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.955+0000 7fe4c6ffd640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 msgr2=0x7fe4a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:06.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.955+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 0x7fe4a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:06.964 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:06.956+0000 7f177b023040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:39:06.964 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:06.959+0000 7f5fce412040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:39:06.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:06.960+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4c0075ec0 con 0x7fe4c80cc190 2023-10-21T17:39:07.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.152+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4c815c610 con 0x7fe4c80cc190 2023-10-21T17:39:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.156+0000 7fe4c6ffd640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 msgr2=0x7fe4a806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:07.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.156+0000 7fe4c6ffd640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 0x7fe4a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:39:07.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.156+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1004683119 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+21121 (secure 0 0 0) 0x7fe4c00794e0 con 0x7fe4c80cc190 2023-10-21T17:39:07.158 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:07.158 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":238,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:38:59.709943+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.158+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 msgr2=0x7fe4a806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.158+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 0x7fe4a806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.158+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 msgr2=0x7fe4c80c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.158+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ac0 tx=0x7fe4c0004730 comp rx=0 tx=0).stop 2023-10-21T17:39:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 shutdown_connections 2023-10-21T17:39:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe4a806b140 0x7fe4a806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c80cc190 0x7fe4c80c8d20 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c80cb750 0x7fe4c80c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 --2- 172.21.15.53:0/1004683119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c8156040 0x7fe4c80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 >> 172.21.15.53:0/1004683119 conn(0x7fe4c809bcb0 msgr2=0x7fe4c806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 shutdown_connections 2023-10-21T17:39:07.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.159+0000 7fe4c7fff640 1 -- 172.21.15.53:0/1004683119 wait complete. 2023-10-21T17:39:07.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:38:30.810147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:38:31.897170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:38:45.333243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:38:46.027612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:07.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:07.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.395+0000 7fd4d05fc640 1 Processor -- start 2023-10-21T17:39:07.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.396+0000 7fd4d05fc640 1 -- start start 2023-10-21T17:39:07.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.397+0000 7fd4d05fc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4c80cce00 0x7fd4c80ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.397+0000 7fd4d05fc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c8072460 con 0x7fd4c80aeea0 2023-10-21T17:39:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.397+0000 7fd4d05fc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c8071590 con 0x7fd4c815cca0 2023-10-21T17:39:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.397+0000 7fd4d05fc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c80b1900 con 0x7fd4c80cce00 2023-10-21T17:39:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.398+0000 7fd4cdb97640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd4c80aeea0 0x7fd4c80cc6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59426/0 (socket says 172.21.15.53:59426) 2023-10-21T17:39:07.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.398+0000 7fd4cdb97640 1 -- 172.21.15.53:0/526817921 learned_addr learned my addr 172.21.15.53:0/526817921 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:07.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.398+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/526817921 >> v2:172.21.15.124:3300/0 conn(0x7fd4c80cce00 0x7fd4c80ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.398+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155032664 0 0) 0x7fd4c8072460 con 0x7fd4c80aeea0 2023-10-21T17:39:07.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.398+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b4003520 con 0x7fd4c80aeea0 2023-10-21T17:39:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1069873056 0 0) 0x7fd4b4003520 con 0x7fd4c80aeea0 2023-10-21T17:39:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 >> v2:172.21.15.124:3300/0 conn(0x7fd4c80cce00 msgr2=0x7fd4c80ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 --2- 172.21.15.53:0/526817921 >> v2:172.21.15.124:3300/0 conn(0x7fd4c80cce00 0x7fd4c80ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 >> v1:172.21.15.124:6789/0 conn(0x7fd4c815cca0 legacy=0x7fd4c8071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c815d080 con 0x7fd4c80aeea0 2023-10-21T17:39:07.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd4b8001670 con 0x7fd4c80aeea0 2023-10-21T17:39:07.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 >> v1:172.21.15.53:6789/0 conn(0x7fd4c80aeea0 legacy=0x7fd4c80cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 0x7fd4b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40047c0 0x7fd4b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4b40051d0 0x7fd4b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80b1900 con 0x7fd4b40039c0 2023-10-21T17:39:07.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b400c120 con 0x7fd4b40047c0 2023-10-21T17:39:07.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b400c2e0 con 0x7fd4b40051d0 2023-10-21T17:39:07.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4ce398640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 0x7fd4b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4cdb97640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40047c0 0x7fd4b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.399+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4b40051d0 0x7fd4b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ce398640 1 -- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4b40051d0 msgr2=0x7fd4b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:07.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ce398640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4b40051d0 0x7fd4b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ce398640 1 -- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40047c0 msgr2=0x7fd4b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:07.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ce398640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40047c0 0x7fd4b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ce398640 1 -- 172.21.15.53:0/526817921 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b4003520 con 0x7fd4b40039c0 2023-10-21T17:39:07.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4b40051d0 0x7fd4b400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:07.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.400+0000 7fd4ceb99640 1 -- 172.21.15.53:0/526817921 reap_dead start 2023-10-21T17:39:07.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.401+0000 7fd4ce398640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 0x7fd4b4003de0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001720 tx=0x7fd4b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9220a672f355ac server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:07.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.401+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40052d0 con 0x7fd4b40039c0 2023-10-21T17:39:07.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.401+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c402e050 con 0x7fd4b40039c0 2023-10-21T17:39:07.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.401+0000 7fd4cd396640 1 -- 172.21.15.53:0/526817921 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4003830 con 0x7fd4b40039c0 2023-10-21T17:39:07.409 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.399+0000 7f177b023040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:39:07.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 -- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 msgr2=0x7fd4b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 0x7fd4b4003de0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4001720 tx=0x7fd4b40057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 -- 172.21.15.53:0/526817921 shutdown_connections 2023-10-21T17:39:07.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/526817921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b40039c0 0x7fd4b4003de0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 -- 172.21.15.53:0/526817921 >> 172.21.15.53:0/526817921 conn(0x7fd4c806b250 msgr2=0x7fd4c806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:07.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 -- 172.21.15.53:0/526817921 shutdown_connections 2023-10-21T17:39:07.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.402+0000 7fd4d05fc640 1 -- 172.21.15.53:0/526817921 wait complete. 2023-10-21T17:39:07.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.404+0000 7fd4d05fc640 1 Processor -- start 2023-10-21T17:39:07.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.404+0000 7fd4d05fc640 1 -- start start 2023-10-21T17:39:07.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 0x7fd4c80c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4c815a570 0x7fd4c815fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80c59f0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80c5cd0 con 0x7fd4b40039c0 2023-10-21T17:39:07.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4d05fc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80c5fb0 con 0x7fd4c815a570 2023-10-21T17:39:07.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47124/0 (socket says 172.21.15.53:47124) 2023-10-21T17:39:07.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 -- 172.21.15.53:0/1435606986 learned_addr learned my addr 172.21.15.53:0/1435606986 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:07.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4cdb97640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4c815a570 0x7fd4c815fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ce398640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 0x7fd4c80c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4c815a570 msgr2=0x7fd4c815fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4c815a570 0x7fd4c815fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 msgr2=0x7fd4c80c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 0x7fd4c80c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ceb99640 1 -- 172.21.15.53:0/1435606986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c80c6290 con 0x7fd4c8159ed0 2023-10-21T17:39:07.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.405+0000 7fd4ce398640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 0x7fd4c80c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:07.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4ceb99640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80cd3e0 tx=0x7fd4c000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:07.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0016620 con 0x7fd4c8159ed0 2023-10-21T17:39:07.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0016780 con 0x7fd4c8159ed0 2023-10-21T17:39:07.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0016b10 con 0x7fd4c8159ed0 2023-10-21T17:39:07.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c815dbd0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.406+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c80ae1c0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.408+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c815ddc0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.408+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fd4c0016c70 con 0x7fd4c8159ed0 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.409+0000 7fd4beffd640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 0x7fd4ac06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.409+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fd4c008b100 con 0x7fd4c8159ed0 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.409+0000 7fd4ce398640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 msgr2=0x7fd4ac06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:07.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.409+0000 7fd4ce398640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 0x7fd4ac06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:07.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.413+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4c00556a0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.437 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.436+0000 7f5fce412040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:39:07.453 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.451+0000 7f177b023040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:39:07.492 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.490+0000 7f5fce412040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:39:07.502 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.501+0000 7f177b023040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:39:07.544 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.543+0000 7f5fce412040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:39:07.563 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.561+0000 7f177b023040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:39:07.608 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.607+0000 7f5fce412040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:39:07.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.607+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4c804e160 con 0x7fd4c8159ed0 2023-10-21T17:39:07.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.608+0000 7fd4beffd640 1 -- 172.21.15.53:0/1435606986 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v238) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4c0058cc0 con 0x7fd4c8159ed0 2023-10-21T17:39:07.609 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:07.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4ce398640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 msgr2=0x7fd4ac06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:07.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4ce398640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 0x7fd4ac06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:39:07.612 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.610+0000 7f177b023040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:39:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 msgr2=0x7fd4ac06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 0x7fd4ac06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 msgr2=0x7fd4c80adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80cd3e0 tx=0x7fd4c000f520 comp rx=0 tx=0).stop 2023-10-21T17:39:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 shutdown_connections 2023-10-21T17:39:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fd4ac06b140 0x7fd4ac06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c8159ed0 0x7fd4c80adaa0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4c815a570 0x7fd4c815fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.610+0000 7fd4d05fc640 1 --2- 172.21.15.53:0/1435606986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4b40039c0 0x7fd4c80c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.611+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 >> 172.21.15.53:0/1435606986 conn(0x7fd4c806b250 msgr2=0x7fd4c80afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:07.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.611+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 shutdown_connections 2023-10-21T17:39:07.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.611+0000 7fd4d05fc640 1 -- 172.21.15.53:0/1435606986 wait complete. 2023-10-21T17:39:07.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:39:07.667 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.666+0000 7f5fce412040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:39:07.722 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.721+0000 7f177b023040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:39:07.785 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.783+0000 7f177b023040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:39:07.786 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.785+0000 7f5fce412040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:39:07.834 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.832+0000 7f177b023040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:39:07.853 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.852+0000 7f5fce412040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:39:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.861+0000 7f95ef3b0640 1 Processor -- start 2023-10-21T17:39:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.861+0000 7f95ef3b0640 1 -- start start 2023-10-21T17:39:07.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ef3b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f95e80b4e10 0x7f95e81612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ef3b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95e80cd4e0 0x7f95e80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ef3b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e80b8c30 con 0x7f95e815cd70 2023-10-21T17:39:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ef3b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80cdd80 con 0x7f95e80cd4e0 2023-10-21T17:39:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ef3b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80ce060 con 0x7f95e80b4e10 2023-10-21T17:39:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ed94d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95e80cd4e0 0x7f95e80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ed94d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f95e80cd4e0 0x7f95e80d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47134/0 (socket says 172.21.15.53:47134) 2023-10-21T17:39:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ed94d640 1 -- 172.21.15.53:0/2632725242 learned_addr learned my addr 172.21.15.53:0/2632725242 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.863+0000 7f95ec94b640 1 --2- 172.21.15.53:0/2632725242 >> v2:172.21.15.124:3300/0 conn(0x7f95e80b4e10 0x7f95e81612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.864+0000 7f95ed94d640 1 -- 172.21.15.53:0/2632725242 >> v2:172.21.15.124:3300/0 conn(0x7f95e80b4e10 msgr2=0x7f95e81612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.864+0000 7f95ed94d640 1 --2- 172.21.15.53:0/2632725242 >> v2:172.21.15.124:3300/0 conn(0x7f95e80b4e10 0x7f95e81612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.864+0000 7f95ed94d640 1 -- 172.21.15.53:0/2632725242 >> v1:172.21.15.124:6789/0 conn(0x7f95e815cd70 legacy=0x7f95e80b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.864+0000 7f95ed94d640 1 -- 172.21.15.53:0/2632725242 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80ce340 con 0x7f95e80cd4e0 2023-10-21T17:39:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.865+0000 7f95ed94d640 1 --2- 172.21.15.53:0/2632725242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e80cd4e0 0x7f95e80d1650 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f95e80d07d0 tx=0x7f95e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836edb7d385ebb7e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.866+0000 7f95dffff640 1 -- 172.21.15.53:0/2632725242 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e002ca80 con 0x7f95e80cd4e0 2023-10-21T17:39:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.866+0000 7f95dffff640 1 -- 172.21.15.53:0/2632725242 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e002cbe0 con 0x7f95e80cd4e0 2023-10-21T17:39:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.866+0000 7f95dffff640 1 -- 172.21.15.53:0/2632725242 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00343f0 con 0x7f95e80cd4e0 2023-10-21T17:39:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 -- 172.21.15.53:0/2632725242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e80cd4e0 msgr2=0x7f95e80d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/2632725242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e80cd4e0 0x7f95e80d1650 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f95e80d07d0 tx=0x7f95e002bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 -- 172.21.15.53:0/2632725242 shutdown_connections 2023-10-21T17:39:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/2632725242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e80cd4e0 0x7f95e80d1650 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/2632725242 >> v2:172.21.15.124:3300/0 conn(0x7f95e80b4e10 0x7f95e81612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 -- 172.21.15.53:0/2632725242 >> 172.21.15.53:0/2632725242 conn(0x7f95e80cf700 msgr2=0x7f95e80cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:07.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 -- 172.21.15.53:0/2632725242 shutdown_connections 2023-10-21T17:39:07.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.867+0000 7f95ef3b0640 1 -- 172.21.15.53:0/2632725242 wait complete. 2023-10-21T17:39:07.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.868+0000 7f95ef3b0640 1 Processor -- start 2023-10-21T17:39:07.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.869+0000 7f95ef3b0640 1 -- start start 2023-10-21T17:39:07.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.869+0000 7f95ef3b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95e80b4e10 0x7f95e815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.869+0000 7f95ef3b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95e80cd4e0 0x7f95e80c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ef3b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ef3b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80c9d50 con 0x7f95e815cd70 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ef3b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8160390 con 0x7f95e80b4e10 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ef3b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80c9fa0 con 0x7f95e80cd4e0 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47148/0 (socket says 172.21.15.53:47148) 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 -- 172.21.15.53:0/3099183059 learned_addr learned my addr 172.21.15.53:0/3099183059 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:07.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed14c640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95e80b4e10 0x7f95e815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95e80cd4e0 msgr2=0x7f95e80c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ec94b640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95e80cd4e0 0x7f95e80c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:07.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95e80cd4e0 0x7f95e80c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95e80b4e10 msgr2=0x7f95e815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:07.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95e80b4e10 0x7f95e815fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:07.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.870+0000 7f95ed94d640 1 -- 172.21.15.53:0/3099183059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0009b30 con 0x7f95e815cd70 2023-10-21T17:39:07.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ed94d640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f40 tx=0x7f95e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:07.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0009510 con 0x7f95e815cd70 2023-10-21T17:39:07.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e003e450 con 0x7f95e815cd70 2023-10-21T17:39:07.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e003e7e0 con 0x7f95e815cd70 2023-10-21T17:39:07.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e80ca280 con 0x7f95e815cd70 2023-10-21T17:39:07.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.871+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e80ca840 con 0x7f95e815cd70 2023-10-21T17:39:07.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.873+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e804e160 con 0x7f95e815cd70 2023-10-21T17:39:07.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.875+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f95e002cd50 con 0x7f95e815cd70 2023-10-21T17:39:07.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.876+0000 7f95ddffb640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 0x7f95cc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:07.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.876+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f95e00acf70 con 0x7f95e815cd70 2023-10-21T17:39:07.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.876+0000 7f95ed14c640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 msgr2=0x7f95cc06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:07.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.876+0000 7f95ed14c640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 0x7f95cc06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:07.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:07.881+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95e0077780 con 0x7f95e815cd70 2023-10-21T17:39:07.892 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.890+0000 7f177b023040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:39:07.909 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.908+0000 7f5fce412040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:39:07.941 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:07.940+0000 7f177b023040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:39:07.969 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:07.968+0000 7f5fce412040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:39:08.022 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.021+0000 7f5fce412040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:39:08.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.075+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95e80d0a00 con 0x7f95e815cd70 2023-10-21T17:39:08.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.076+0000 7f95ddffb640 1 -- 172.21.15.53:0/3099183059 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v238) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95e007ada0 con 0x7f95e815cd70 2023-10-21T17:39:08.078 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:08.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.077+0000 7f95ed14c640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 msgr2=0x7f95cc06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:08.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.077+0000 7f95ed14c640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 0x7f95cc06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:39:08.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 msgr2=0x7f95cc06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:08.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 0x7f95cc06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 msgr2=0x7f95e815ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009f40 tx=0x7f95e0004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:08.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 shutdown_connections 2023-10-21T17:39:08.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f95cc06b140 0x7f95cc06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95e815cd70 0x7f95e815ae90 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95e80cd4e0 0x7f95e80c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 --2- 172.21.15.53:0/3099183059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95e80b4e10 0x7f95e815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 >> 172.21.15.53:0/3099183059 conn(0x7f95e80cf700 msgr2=0x7f95e80d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:08.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 shutdown_connections 2023-10-21T17:39:08.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.079+0000 7f95ef3b0640 1 -- 172.21.15.53:0/3099183059 wait complete. 2023-10-21T17:39:08.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:39:08.319 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.317+0000 7f177b023040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:39:08.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.358+0000 7ff2164bb640 1 Processor -- start 2023-10-21T17:39:08.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.358+0000 7ff2164bb640 1 -- start start 2023-10-21T17:39:08.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff2164bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff21006dfc0 0x7ff210070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff2164bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2100b1cc0 con 0x7ff2100afbe0 2023-10-21T17:39:08.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff2164bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2100b2620 con 0x7ff210071070 2023-10-21T17:39:08.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff2164bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2100b2060 con 0x7ff21006dfc0 2023-10-21T17:39:08.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff207fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff21006dfc0 0x7ff210070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff20ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff2100afbe0 0x7ff2100b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59428/0 (socket says 172.21.15.53:59428) 2023-10-21T17:39:08.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.360+0000 7ff20ffff640 1 -- 172.21.15.53:0/2279753611 learned_addr learned my addr 172.21.15.53:0/2279753611 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.361+0000 7ff207fff640 1 -- 172.21.15.53:0/2279753611 >> v1:172.21.15.124:6789/0 conn(0x7ff210071070 legacy=0x7ff2100cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:08.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.361+0000 7ff207fff640 1 -- 172.21.15.53:0/2279753611 >> v1:172.21.15.53:6789/0 conn(0x7ff2100afbe0 legacy=0x7ff2100b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.361+0000 7ff207fff640 1 -- 172.21.15.53:0/2279753611 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100b2340 con 0x7ff21006dfc0 2023-10-21T17:39:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.362+0000 7ff207fff640 1 --2- 172.21.15.53:0/2279753611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff21006dfc0 0x7ff210070b30 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff2100b2810 tx=0x7ff1f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1240ba9468e31a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.362+0000 7ff20f7fe640 1 -- 172.21.15.53:0/2279753611 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f802d040 con 0x7ff21006dfc0 2023-10-21T17:39:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.362+0000 7ff20f7fe640 1 -- 172.21.15.53:0/2279753611 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f8031810 con 0x7ff21006dfc0 2023-10-21T17:39:08.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.362+0000 7ff20f7fe640 1 -- 172.21.15.53:0/2279753611 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8031ba0 con 0x7ff21006dfc0 2023-10-21T17:39:08.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 -- 172.21.15.53:0/2279753611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff21006dfc0 msgr2=0x7ff210070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2279753611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff21006dfc0 0x7ff210070b30 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff2100b2810 tx=0x7ff1f802b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 -- 172.21.15.53:0/2279753611 shutdown_connections 2023-10-21T17:39:08.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2279753611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff21006dfc0 0x7ff210070b30 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 -- 172.21.15.53:0/2279753611 >> 172.21.15.53:0/2279753611 conn(0x7ff21006a5d0 msgr2=0x7ff2100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 -- 172.21.15.53:0/2279753611 shutdown_connections 2023-10-21T17:39:08.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.363+0000 7ff2164bb640 1 -- 172.21.15.53:0/2279753611 wait complete. 2023-10-21T17:39:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.365+0000 7ff2164bb640 1 Processor -- start 2023-10-21T17:39:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.365+0000 7ff2164bb640 1 -- start start 2023-10-21T17:39:08.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff21006dfc0 0x7ff210152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2100afbe0 0x7ff2100ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100c23d0 con 0x7ff210071070 2023-10-21T17:39:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100caba0 con 0x7ff21006dfc0 2023-10-21T17:39:08.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff2164bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100c2690 con 0x7ff2100afbe0 2023-10-21T17:39:08.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff207fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff207fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47174/0 (socket says 172.21.15.53:47174) 2023-10-21T17:39:08.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff207fff640 1 -- 172.21.15.53:0/2547926946 learned_addr learned my addr 172.21.15.53:0/2547926946 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:08.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.366+0000 7ff20ffff640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff21006dfc0 0x7ff210152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff214a58640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2100afbe0 0x7ff2100ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 -- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2100afbe0 msgr2=0x7ff2100ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2100afbe0 0x7ff2100ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 -- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff21006dfc0 msgr2=0x7ff210152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff21006dfc0 0x7ff210152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 -- 172.21.15.53:0/2547926946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f8009400 con 0x7ff210071070 2023-10-21T17:39:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff207fff640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8000c00 tx=0x7ff1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f803ad60 con 0x7ff210071070 2023-10-21T17:39:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f80443f0 con 0x7ff210071070 2023-10-21T17:39:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f80447a0 con 0x7ff210071070 2023-10-21T17:39:08.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.367+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2100c2970 con 0x7ff210071070 2023-10-21T17:39:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.368+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2100aeea0 con 0x7ff210071070 2023-10-21T17:39:08.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.369+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2100c2210 con 0x7ff210071070 2023-10-21T17:39:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.370+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7ff1f804d450 con 0x7ff210071070 2023-10-21T17:39:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.371+0000 7ff20d7fa640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 0x7ff1f006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.371+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7ff1f80ac570 con 0x7ff210071070 2023-10-21T17:39:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.371+0000 7ff20ffff640 1 -- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 msgr2=0x7ff1f006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:08.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.371+0000 7ff20ffff640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 0x7ff1f006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:08.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.377+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1f8076c70 con 0x7ff210071070 2023-10-21T17:39:08.431 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.429+0000 7f5fce412040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:39:08.479 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.477+0000 7f177b023040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:39:08.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.567+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2100c2b80 con 0x7ff210071070 2023-10-21T17:39:08.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.568+0000 7ff20d7fa640 1 -- 172.21.15.53:0/2547926946 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v238) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff1f807a290 con 0x7ff210071070 2023-10-21T17:39:08.570 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:08.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 msgr2=0x7ff1f006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:08.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 0x7ff1f006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 msgr2=0x7ff2100c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8000c00 tx=0x7ff1f80046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 shutdown_connections 2023-10-21T17:39:08.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7ff1f006b140 0x7ff1f006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2100afbe0 0x7ff2100ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff210071070 0x7ff2100c2e90 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 --2- 172.21.15.53:0/2547926946 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff21006dfc0 0x7ff210152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 >> 172.21.15.53:0/2547926946 conn(0x7ff21006a5d0 msgr2=0x7ff21006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:08.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 shutdown_connections 2023-10-21T17:39:08.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.571+0000 7ff2164bb640 1 -- 172.21.15.53:0/2547926946 wait complete. 2023-10-21T17:39:08.586 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.585+0000 7f177b023040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:39:08.591 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:39:08.599 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.599+0000 7f5fce412040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:39:08.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.635+0000 7f177b023040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:39:08.715 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.713+0000 7f5fce412040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:39:08.742 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.741+0000 7f177b023040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:39:08.768 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.767+0000 7f5fce412040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:39:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.809+0000 7f6976b16640 1 Processor -- start 2023-10-21T17:39:08.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.809+0000 7f6976b16640 1 -- start start 2023-10-21T17:39:08.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6976b16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015cd70 0x7f69701612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6976b16640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69700b4e10 0x7f69700d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6976b16640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700b8c30 con 0x7f69700cd4e0 2023-10-21T17:39:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6976b16640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700cdd80 con 0x7f69700b4e10 2023-10-21T17:39:08.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6976b16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700ce060 con 0x7f697015cd70 2023-10-21T17:39:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6967fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f69700cd4e0 0x7f69700b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59434/0 (socket says 172.21.15.53:59434) 2023-10-21T17:39:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f69750b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69700b4e10 0x7f69700d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.811+0000 7f6967fff640 1 -- 172.21.15.53:0/4175368794 learned_addr learned my addr 172.21.15.53:0/4175368794 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.812+0000 7f69748b2640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015cd70 0x7f69701612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.812+0000 7f69750b3640 1 -- 172.21.15.53:0/4175368794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015cd70 msgr2=0x7f69701612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.812+0000 7f69750b3640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015cd70 0x7f69701612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.812+0000 7f69750b3640 1 -- 172.21.15.53:0/4175368794 >> v1:172.21.15.53:6789/0 conn(0x7f69700cd4e0 legacy=0x7f69700b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.812+0000 7f69750b3640 1 -- 172.21.15.53:0/4175368794 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700ce340 con 0x7f69700b4e10 2023-10-21T17:39:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.813+0000 7f69750b3640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f69700d1650 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f69680097a0 tx=0x7f696802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6559d826472982 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.813+0000 7f69677fe640 1 -- 172.21.15.53:0/4175368794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696802d040 con 0x7f69700b4e10 2023-10-21T17:39:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.813+0000 7f69677fe640 1 -- 172.21.15.53:0/4175368794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6968031810 con 0x7f69700b4e10 2023-10-21T17:39:08.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.813+0000 7f69677fe640 1 -- 172.21.15.53:0/4175368794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968031ba0 con 0x7f69700b4e10 2023-10-21T17:39:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 -- 172.21.15.53:0/4175368794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 msgr2=0x7f69700d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f69700d1650 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f69680097a0 tx=0x7f696802b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 -- 172.21.15.53:0/4175368794 shutdown_connections 2023-10-21T17:39:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f69700d1650 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 --2- 172.21.15.53:0/4175368794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f697015cd70 0x7f69701612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 -- 172.21.15.53:0/4175368794 >> 172.21.15.53:0/4175368794 conn(0x7f69700cf700 msgr2=0x7f69700cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 -- 172.21.15.53:0/4175368794 shutdown_connections 2023-10-21T17:39:08.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.814+0000 7f6976b16640 1 -- 172.21.15.53:0/4175368794 wait complete. 2023-10-21T17:39:08.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.816+0000 7f6976b16640 1 Processor -- start 2023-10-21T17:39:08.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.816+0000 7f6976b16640 1 -- start start 2023-10-21T17:39:08.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69700cd4e0 0x7f69701e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697015cd70 0x7f697015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c7420 con 0x7f69700b4e10 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c6810 con 0x7f697015cd70 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6976b16640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c6af0 con 0x7f69700cd4e0 2023-10-21T17:39:08.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f69748b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f69748b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47188/0 (socket says 172.21.15.53:47188) 2023-10-21T17:39:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f69748b2640 1 -- 172.21.15.53:0/3975069620 learned_addr learned my addr 172.21.15.53:0/3975069620 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.817+0000 7f6967fff640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69700cd4e0 0x7f69701e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 -- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69700cd4e0 msgr2=0x7f69701e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69700cd4e0 0x7f69701e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 -- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697015cd70 msgr2=0x7f697015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697015cd70 0x7f697015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 -- 172.21.15.53:0/3975069620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968009400 con 0x7f69700b4e10 2023-10-21T17:39:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69748b2640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f696c007ef0 tx=0x7f696c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:08.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c014620 con 0x7f69700b4e10 2023-10-21T17:39:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c014780 con 0x7f69700b4e10 2023-10-21T17:39:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c014b30 con 0x7f69700b4e10 2023-10-21T17:39:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69700c6dd0 con 0x7f69700b4e10 2023-10-21T17:39:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.818+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6970152290 con 0x7f69700b4e10 2023-10-21T17:39:08.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.820+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697004e160 con 0x7f69700b4e10 2023-10-21T17:39:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.820+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f696c014c90 con 0x7f69700b4e10 2023-10-21T17:39:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.821+0000 7f69657fa640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 0x7f695c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.821+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f696c08aad0 con 0x7f69700b4e10 2023-10-21T17:39:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.821+0000 7f6967fff640 1 -- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 msgr2=0x7f695c06d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:08.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.821+0000 7f6967fff640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 0x7f695c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:08.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:08.825+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f696c0551c0 con 0x7f69700b4e10 2023-10-21T17:39:08.849 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:08.847+0000 7f177b023040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:39:08.884 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.883+0000 7f5fce412040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:39:08.998 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:08.996+0000 7f5fce412040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:39:09.005 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:09.005+0000 7f177b023040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:39:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.016+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69700d0a00 con 0x7f69700b4e10 2023-10-21T17:39:09.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.016+0000 7f69657fa640 1 -- 172.21.15.53:0/3975069620 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v238) v1 ==== 107+0+11 (secure 0 0 0) 0x7f696c0587e0 con 0x7f69700b4e10 2023-10-21T17:39:09.018 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 msgr2=0x7f695c06d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:09.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 0x7f695c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 msgr2=0x7f6970151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f696c007ef0 tx=0x7f696c00e4b0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 shutdown_connections 2023-10-21T17:39:09.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7f695c06b140 0x7f695c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f697015cd70 0x7f697015b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69700cd4e0 0x7f69701e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 --2- 172.21.15.53:0/3975069620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69700b4e10 0x7f6970151d50 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.019+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 >> 172.21.15.53:0/3975069620 conn(0x7f69700cf700 msgr2=0x7f69700d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.020+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 shutdown_connections 2023-10-21T17:39:09.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.020+0000 7f6976b16640 1 -- 172.21.15.53:0/3975069620 wait complete. 2023-10-21T17:39:09.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:39:09.054 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:09.053+0000 7f177b023040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:39:09.167 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:09.166+0000 7f5fce412040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:39:09.172 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:09.171+0000 7f177b023040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:39:09.219 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:09.218+0000 7f5fce412040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:39:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.296+0000 7fefcbfff640 1 Processor -- start 2023-10-21T17:39:09.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.296+0000 7fefcbfff640 1 -- start start 2023-10-21T17:39:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcbfff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fefcc0cf690 0x7fefcc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefcc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcbfff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc0cd520 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcbfff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc0d1910 con 0x7fefcc0cf690 2023-10-21T17:39:09.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0ce080 con 0x7fefcc0ce510 2023-10-21T17:39:09.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcaffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fefcc0cf2b0 0x7fefcc0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59438/0 (socket says 172.21.15.53:59438) 2023-10-21T17:39:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcaffd640 1 -- 172.21.15.53:0/1923985308 learned_addr learned my addr 172.21.15.53:0/1923985308 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefcb7fe640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefcc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.298+0000 7fefc27fc640 1 --2- 172.21.15.53:0/1923985308 >> v2:172.21.15.124:3300/0 conn(0x7fefcc0cf690 0x7fefcc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021219532 0 0) 0x7fefcc0cd520 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefac003520 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212853175 0 0) 0x7fefac003520 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 msgr2=0x7fefcc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefcc15c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 >> v2:172.21.15.124:3300/0 conn(0x7fefcc0cf690 msgr2=0x7fefcc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 --2- 172.21.15.53:0/1923985308 >> v2:172.21.15.124:3300/0 conn(0x7fefcc0cf690 0x7fefcc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc0cd750 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fefbc001670 con 0x7fefcc0cf2b0 2023-10-21T17:39:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 >> v1:172.21.15.53:6789/0 conn(0x7fefcc0cf2b0 legacy=0x7fefcc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefac003970 0x7fefac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefac004d00 0x7fefac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0d1910 con 0x7fefac003970 2023-10-21T17:39:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0ce080 con 0x7fefac004d00 2023-10-21T17:39:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.299+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac00ba00 con 0x7fefcc0ce510 2023-10-21T17:39:09.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefac003970 0x7fefac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefc27fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcb7fe640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefac004d00 0x7fefac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 -- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 msgr2=0x7fefac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 -- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefac004d00 msgr2=0x7fefac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefac004d00 0x7fefac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefcaffd640 1 -- 172.21.15.53:0/1923985308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac003520 con 0x7fefac003970 2023-10-21T17:39:09.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.300+0000 7fefc27fc640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefcc0ce510 0x7fefac0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:09.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.301+0000 7fefcaffd640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefac003970 0x7fefac003d90 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c110 tx=0x7fefac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fab229eda66b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:09.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.301+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc005ad0 con 0x7fefac003970 2023-10-21T17:39:09.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.301+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc005c50 con 0x7fefac003970 2023-10-21T17:39:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.301+0000 7fefca7fc640 1 -- 172.21.15.53:0/1923985308 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc02c830 con 0x7fefac003970 2023-10-21T17:39:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 -- 172.21.15.53:0/1923985308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefac003970 msgr2=0x7fefac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 --2- 172.21.15.53:0/1923985308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefac003970 0x7fefac003d90 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c110 tx=0x7fefac0057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcb7fe640 1 -- 172.21.15.53:0/1923985308 reap_dead start 2023-10-21T17:39:09.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 -- 172.21.15.53:0/1923985308 shutdown_connections 2023-10-21T17:39:09.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 -- 172.21.15.53:0/1923985308 >> 172.21.15.53:0/1923985308 conn(0x7fefcc09bc40 msgr2=0x7fefcc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:09.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 -- 172.21.15.53:0/1923985308 shutdown_connections 2023-10-21T17:39:09.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.302+0000 7fefcbfff640 1 -- 172.21.15.53:0/1923985308 wait complete. 2023-10-21T17:39:09.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.304+0000 7fefcbfff640 1 Processor -- start 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.304+0000 7fefcbfff640 1 -- start start 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefac003970 0x7fefcc0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefcc0b6240 0x7fefcc0c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0d2410 con 0x7fefcc0d1a50 2023-10-21T17:39:09.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc1485c0 con 0x7fefcc0b6240 2023-10-21T17:39:09.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.305+0000 7fefcbfff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc1488a0 con 0x7fefac003970 2023-10-21T17:39:09.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47214/0 (socket says 172.21.15.53:47214) 2023-10-21T17:39:09.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 -- 172.21.15.53:0/4281646498 learned_addr learned my addr 172.21.15.53:0/4281646498 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:09.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcaffd640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefac003970 0x7fefcc0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefc27fc640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefcc0b6240 0x7fefcc0c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefac003970 msgr2=0x7fefcc0b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefac003970 0x7fefcc0b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefcc0b6240 msgr2=0x7fefcc0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefcc0b6240 0x7fefcc0c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.306+0000 7fefcb7fe640 1 -- 172.21.15.53:0/4281646498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc148b80 con 0x7fefcc0d1a50 2023-10-21T17:39:09.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefcb7fe640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cfa70 tx=0x7fefc40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:09.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400e070 con 0x7fefcc0d1a50 2023-10-21T17:39:09.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc4011e10 con 0x7fefcc0d1a50 2023-10-21T17:39:09.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4003990 con 0x7fefcc0d1a50 2023-10-21T17:39:09.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc148e60 con 0x7fefcc0d1a50 2023-10-21T17:39:09.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.307+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc0b8000 con 0x7fefcc0d1a50 2023-10-21T17:39:09.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.308+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc148700 con 0x7fefcc0d1a50 2023-10-21T17:39:09.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.309+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fefc4003af0 con 0x7fefcc0d1a50 2023-10-21T17:39:09.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.310+0000 7fefc3fff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 0x7fefb006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.310+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fefc408a820 con 0x7fefcc0d1a50 2023-10-21T17:39:09.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.310+0000 7fefcaffd640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 msgr2=0x7fefb006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:09.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.310+0000 7fefcaffd640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 0x7fefb006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:09.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.315+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefcc148700 con 0x7fefcc0d1a50 2023-10-21T17:39:09.348 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:09.346+0000 7f5fce412040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:39:09.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.511+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fefcc148700 con 0x7fefcc0d1a50 2023-10-21T17:39:09.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.511+0000 7fefcaffd640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 msgr2=0x7fefb006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:09.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.511+0000 7fefc3fff640 1 -- 172.21.15.53:0/4281646498 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v238) v1 ==== 103+0+11 (secure 0 0 0) 0x7fefcc148700 con 0x7fefcc0d1a50 2023-10-21T17:39:09.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.511+0000 7fefcaffd640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 0x7fefb006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:39:09.515 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:09.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.513+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 msgr2=0x7fefb006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:09.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.513+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 0x7fefb006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.513+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 msgr2=0x7fefcc0d1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.513+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0cfa70 tx=0x7fefc40115a0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 shutdown_connections 2023-10-21T17:39:09.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fefb006b140 0x7fefb006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fefcc0d1a50 0x7fefcc0d1ed0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fefcc0b6240 0x7fefcc0c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 --2- 172.21.15.53:0/4281646498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fefac003970 0x7fefcc0b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 >> 172.21.15.53:0/4281646498 conn(0x7fefcc09bc40 msgr2=0x7fefcc0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:09.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 shutdown_connections 2023-10-21T17:39:09.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.514+0000 7fefcbfff640 1 -- 172.21.15.53:0/4281646498 wait complete. 2023-10-21T17:39:09.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:39:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.738+0000 7fe963583640 1 Processor -- start 2023-10-21T17:39:09.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.739+0000 7fe963583640 1 -- start start 2023-10-21T17:39:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe95c06dfc0 0x7fe95c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe95c071070 0x7fe95c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0b2620 con 0x7fe95c06dfc0 2023-10-21T17:39:09.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0b2060 con 0x7fe95c071070 2023-10-21T17:39:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.740+0000 7fe963583640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0b2340 con 0x7fe95c0afbe0 2023-10-21T17:39:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe95c06dfc0 0x7fe95c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe95c06dfc0 0x7fe95c070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47222/0 (socket says 172.21.15.53:47222) 2023-10-21T17:39:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 -- 172.21.15.53:0/1124891838 learned_addr learned my addr 172.21.15.53:0/1124891838 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:09.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe961b20640 1 --2- 172.21.15.53:0/1124891838 >> v2:172.21.15.124:3300/0 conn(0x7fe95c071070 0x7fe95c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe96131f640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 -- 172.21.15.53:0/1124891838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 msgr2=0x7fe95c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 -- 172.21.15.53:0/1124891838 >> v2:172.21.15.124:3300/0 conn(0x7fe95c071070 msgr2=0x7fe95c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:09.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1124891838 >> v2:172.21.15.124:3300/0 conn(0x7fe95c071070 0x7fe95c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.741+0000 7fe960b1e640 1 -- 172.21.15.53:0/1124891838 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c070350 con 0x7fe95c06dfc0 2023-10-21T17:39:09.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.743+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c06dfc0 0x7fe95c070b30 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0b3980 tx=0x7fe94c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cffedd818b2528c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:09.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.743+0000 7fe953fff640 1 -- 172.21.15.53:0/1124891838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c031860 con 0x7fe95c06dfc0 2023-10-21T17:39:09.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.743+0000 7fe953fff640 1 -- 172.21.15.53:0/1124891838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c0319c0 con 0x7fe95c06dfc0 2023-10-21T17:39:09.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.743+0000 7fe953fff640 1 -- 172.21.15.53:0/1124891838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c031d50 con 0x7fe95c06dfc0 2023-10-21T17:39:09.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 -- 172.21.15.53:0/1124891838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c06dfc0 msgr2=0x7fe95c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c06dfc0 0x7fe95c070b30 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0b3980 tx=0x7fe94c02b900 comp rx=0 tx=0).stop 2023-10-21T17:39:09.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 -- 172.21.15.53:0/1124891838 shutdown_connections 2023-10-21T17:39:09.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 --2- 172.21.15.53:0/1124891838 >> v2:172.21.15.124:3300/0 conn(0x7fe95c071070 0x7fe95c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c06dfc0 0x7fe95c070b30 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 --2- 172.21.15.53:0/1124891838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 -- 172.21.15.53:0/1124891838 >> 172.21.15.53:0/1124891838 conn(0x7fe95c06a5d0 msgr2=0x7fe95c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:09.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 -- 172.21.15.53:0/1124891838 shutdown_connections 2023-10-21T17:39:09.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.744+0000 7fe963583640 1 -- 172.21.15.53:0/1124891838 wait complete. 2023-10-21T17:39:09.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.746+0000 7fe963583640 1 Processor -- start 2023-10-21T17:39:09.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.746+0000 7fe963583640 1 -- start start 2023-10-21T17:39:09.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 0x7fe95c14cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c14d4b0 con 0x7fe95c071070 2023-10-21T17:39:09.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c22f0 con 0x7fe95c06dfc0 2023-10-21T17:39:09.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe963583640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c25d0 con 0x7fe95c0afbe0 2023-10-21T17:39:09.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe960b1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe960b1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47238/0 (socket says 172.21.15.53:47238) 2023-10-21T17:39:09.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe960b1e640 1 -- 172.21.15.53:0/1999195182 learned_addr learned my addr 172.21.15.53:0/1999195182 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:09.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe96131f640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 0x7fe95c14cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.747+0000 7fe961b20640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:09.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 msgr2=0x7fe95c157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 msgr2=0x7fe95c14cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 0x7fe95c14cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 -- 172.21.15.53:0/1999195182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c0094d0 con 0x7fe95c071070 2023-10-21T17:39:09.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe96131f640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 0x7fe95c14cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:39:09.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.748+0000 7fe960b1e640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02b940 tx=0x7fe94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:09.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.749+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03ac60 con 0x7fe95c071070 2023-10-21T17:39:09.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.749+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c03ade0 con 0x7fe95c071070 2023-10-21T17:39:09.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.749+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c044630 con 0x7fe95c071070 2023-10-21T17:39:09.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.749+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0c28b0 con 0x7fe95c071070 2023-10-21T17:39:09.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.749+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0c7790 con 0x7fe95c071070 2023-10-21T17:39:09.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.750+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0c2dd0 con 0x7fe95c071070 2023-10-21T17:39:09.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.751+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 132) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fe94c04d450 con 0x7fe95c071070 2023-10-21T17:39:09.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.752+0000 7fe951ffb640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 0x7fe94006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:09.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.752+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fe94c0ac480 con 0x7fe95c071070 2023-10-21T17:39:09.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.752+0000 7fe96131f640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 msgr2=0x7fe94006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:09.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.752+0000 7fe96131f640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 0x7fe94006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:39:09.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.756+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe95c0c2dd0 con 0x7fe95c071070 2023-10-21T17:39:09.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.953+0000 7fe96131f640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 msgr2=0x7fe94006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3186075298 2023-10-21T17:39:09.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.953+0000 7fe96131f640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 0x7fe94006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:39:09.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.960+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe95c0c2710 con 0x7fe95c071070 2023-10-21T17:39:09.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.961+0000 7fe951ffb640 1 -- 172.21.15.53:0/1999195182 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v238) v1 ==== 80+0+476 (secure 0 0 0) 0x7fe95c0c2710 con 0x7fe95c071070 2023-10-21T17:39:09.962 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/583433106 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.963 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3930748558 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.963 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3930748558 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.963 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4019636868 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.964 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2979161442 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.964 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1830851898 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.964 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/4191579832 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.964 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/167361821 2023-10-22T17:38:59.709844+0000 2023-10-21T17:39:09.965 INFO:teuthology.orchestra.run.smithi053.stderr:listed 8 entries 2023-10-21T17:39:09.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 msgr2=0x7fe94006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:39:09.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 0x7fe94006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 msgr2=0x7fe95c1570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:09.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fe94c02b940 tx=0x7fe94c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 shutdown_connections 2023-10-21T17:39:09.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:6832/3186075298,v1:172.21.15.124:6833/3186075298] conn(0x7fe94006b140 0x7fe94006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe95c0afbe0 0x7fe95c157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe95c071070 0x7fe95c1570e0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 --2- 172.21.15.53:0/1999195182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe95c06dfc0 0x7fe95c14cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:09.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 >> 172.21.15.53:0/1999195182 conn(0x7fe95c06a5d0 msgr2=0x7fe95c0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:09.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 shutdown_connections 2023-10-21T17:39:09.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:09.963+0000 7fe963583640 1 -- 172.21.15.53:0/1999195182 wait complete. 2023-10-21T17:39:09.981 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:39:09.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:09.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:39:10.014 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.014+0000 7f177b023040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:39:10.089 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:39:10.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:10.090 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:39:10.142 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:39:10.142 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:10.198 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:39:10.199 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:10.199 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:10.222 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:10.221+0000 7f5fce412040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:39:10.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:39:10.257 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:39:10.257 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:10.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:39:10.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:10.316 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.315+0000 7f177b023040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:39:10.374 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.373+0000 7f177b023040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:39:10.488 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.488+0000 7f177b023040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:39:10.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.532+0000 7f897afdd640 1 Processor -- start 2023-10-21T17:39:10.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.532+0000 7f897afdd640 1 -- start start 2023-10-21T17:39:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897afdd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89741599d0 0x7f89740ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897afdd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 0x7f897406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897afdd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89740cacf0 con 0x7f89740cc210 2023-10-21T17:39:10.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897afdd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89740b10f0 con 0x7f89741599d0 2023-10-21T17:39:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897afdd640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89740ca3c0 con 0x7f89740b07d0 2023-10-21T17:39:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897a7dc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 0x7f897406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f8979fdb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89741599d0 0x7f89740ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:10.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897a7dc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 0x7f897406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:38752/0 (socket says 172.21.15.53:38752) 2023-10-21T17:39:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f8979fdb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f89741599d0 0x7f89740ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47250/0 (socket says 172.21.15.53:47250) 2023-10-21T17:39:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f897a7dc640 1 -- 172.21.15.53:0/1339340460 learned_addr learned my addr 172.21.15.53:0/1339340460 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.534+0000 7f89797da640 1 --1- 172.21.15.53:0/1339340460 >> v1:172.21.15.53:6789/0 conn(0x7f89740cc210 0x7f89740b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59440/0 (socket says 172.21.15.53:59440) 2023-10-21T17:39:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.535+0000 7f8979fdb640 1 -- 172.21.15.53:0/1339340460 >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 msgr2=0x7f897406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:10.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.535+0000 7f8979fdb640 1 --2- 172.21.15.53:0/1339340460 >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 0x7f897406c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.535+0000 7f8979fdb640 1 -- 172.21.15.53:0/1339340460 >> v1:172.21.15.53:6789/0 conn(0x7f89740cc210 legacy=0x7f89740b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:10.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.535+0000 7f8979fdb640 1 -- 172.21.15.53:0/1339340460 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740ca6a0 con 0x7f89741599d0 2023-10-21T17:39:10.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.537+0000 7f8979fdb640 1 --2- 172.21.15.53:0/1339340460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89741599d0 0x7f89740ad730 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f89740c98d0 tx=0x7f897002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3335c95ad3967d22 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:10.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.537+0000 7f8978fd9640 1 -- 172.21.15.53:0/1339340460 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970031830 con 0x7f89741599d0 2023-10-21T17:39:10.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.537+0000 7f8978fd9640 1 -- 172.21.15.53:0/1339340460 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970031990 con 0x7f89741599d0 2023-10-21T17:39:10.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.537+0000 7f8978fd9640 1 -- 172.21.15.53:0/1339340460 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970031d20 con 0x7f89741599d0 2023-10-21T17:39:10.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 -- 172.21.15.53:0/1339340460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89741599d0 msgr2=0x7f89740ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:10.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 --2- 172.21.15.53:0/1339340460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89741599d0 0x7f89740ad730 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f89740c98d0 tx=0x7f897002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 -- 172.21.15.53:0/1339340460 shutdown_connections 2023-10-21T17:39:10.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 --2- 172.21.15.53:0/1339340460 >> v2:172.21.15.124:3300/0 conn(0x7f89740b07d0 0x7f897406c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 --2- 172.21.15.53:0/1339340460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89741599d0 0x7f89740ad730 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 -- 172.21.15.53:0/1339340460 >> 172.21.15.53:0/1339340460 conn(0x7f89740cb0c0 msgr2=0x7f89740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:10.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 -- 172.21.15.53:0/1339340460 shutdown_connections 2023-10-21T17:39:10.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.538+0000 7f897afdd640 1 -- 172.21.15.53:0/1339340460 wait complete. 2023-10-21T17:39:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.540+0000 7f897afdd640 1 Processor -- start 2023-10-21T17:39:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.540+0000 7f897afdd640 1 -- start start 2023-10-21T17:39:10.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740b07d0 0x7f89740afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89741599d0 0x7f89740ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89740ae8f0 con 0x7f89740cc210 2023-10-21T17:39:10.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89740aebd0 con 0x7f89741599d0 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897afdd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89740aeeb0 con 0x7f89740b07d0 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f89797da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f89797da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47266/0 (socket says 172.21.15.53:47266) 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f89797da640 1 -- 172.21.15.53:0/1075613489 learned_addr learned my addr 172.21.15.53:0/1075613489 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f8979fdb640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740b07d0 0x7f89740afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:10.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.541+0000 7f897a7dc640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89741599d0 0x7f89740ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:10.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 -- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740b07d0 msgr2=0x7f89740afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:10.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740b07d0 0x7f89740afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 -- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89741599d0 msgr2=0x7f89740ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:10.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89741599d0 0x7f89740ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 -- 172.21.15.53:0/1075613489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970009400 con 0x7f89740cc210 2023-10-21T17:39:10.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f89797da640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f896400c4d0 tx=0x7f896400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:10.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964007480 con 0x7f89740cc210 2023-10-21T17:39:10.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89640075e0 con 0x7f89740cc210 2023-10-21T17:39:10.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640049e0 con 0x7f89740cc210 2023-10-21T17:39:10.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.542+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89740af190 con 0x7f89740cc210 2023-10-21T17:39:10.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.543+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89741f24d0 con 0x7f89740cc210 2023-10-21T17:39:10.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.544+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897404e160 con 0x7f89740cc210 2023-10-21T17:39:10.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.545+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 133) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8964004b40 con 0x7f89740cc210 2023-10-21T17:39:10.552 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:10.546+0000 7f5fce412040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:39:10.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.546+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f8964088a90 con 0x7f89740cc210 2023-10-21T17:39:10.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.551+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8964053700 con 0x7f89740cc210 2023-10-21T17:39:10.581 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.580+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.581 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.580+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.581 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.580+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.582 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.580+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.582 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.580+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.582 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.581+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.583 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.581+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.583 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.581+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.583 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.581+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.583 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.581+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.593+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.593 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.593+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.593+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.593+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.594 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.593+0000 7f170002f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.595 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.594+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.595 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.594+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.595 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.594+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.595 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.594+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.596 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:39:10.594+0000 7f16fd029640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:39:10.611 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:10.610+0000 7f5fce412040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:39:10.733 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:39:10.732+0000 7f5fce412040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:39:10.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.792+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89740ce370 con 0x7f89740cc210 2023-10-21T17:39:10.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.793+0000 7f896a7fc640 1 -- 172.21.15.53:0/1075613489 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7f8964018070 con 0x7f89740cc210 2023-10-21T17:39:10.795 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:10.795 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:10.795 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:39:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 msgr2=0x7f89741f1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:10.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f896400c4d0 tx=0x7f896400c990 comp rx=0 tx=0).stop 2023-10-21T17:39:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 shutdown_connections 2023-10-21T17:39:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89741599d0 0x7f89740ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f89740cc210 0x7f89741f1f90 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 --2- 172.21.15.53:0/1075613489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740b07d0 0x7f89740afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:10.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 >> 172.21.15.53:0/1075613489 conn(0x7f89740cb0c0 msgr2=0x7f89740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 shutdown_connections 2023-10-21T17:39:10.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:10.796+0000 7f897afdd640 1 -- 172.21.15.53:0/1075613489 wait complete. 2023-10-21T17:39:10.817 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:11.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.037+0000 7f3556b67640 1 Processor -- start 2023-10-21T17:39:11.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.037+0000 7f3556b67640 1 -- start start 2023-10-21T17:39:11.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.038+0000 7f3556b67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f355006b680 0x7f355015d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.038+0000 7f3556b67640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35500ce2c0 0x7f355015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.038+0000 7f3556b67640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35500cde30 con 0x7f3550157480 2023-10-21T17:39:11.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.038+0000 7f3556b67640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3550157860 con 0x7f35500ce2c0 2023-10-21T17:39:11.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.038+0000 7f3556b67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355015e2c0 con 0x7f355006b680 2023-10-21T17:39:11.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3556366640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f35500ce2c0 0x7f355015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3555b65640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3550157480 0x7f355006af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59444/0 (socket says 172.21.15.53:59444) 2023-10-21T17:39:11.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3555b65640 1 -- 172.21.15.53:0/3195370986 learned_addr learned my addr 172.21.15.53:0/3195370986 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:11.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3555364640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f355006b680 0x7f355015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3556366640 1 -- 172.21.15.53:0/3195370986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f355006b680 msgr2=0x7f355015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3556366640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f355006b680 0x7f355015d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3556366640 1 -- 172.21.15.53:0/3195370986 >> v1:172.21.15.53:6789/0 conn(0x7f3550157480 legacy=0x7f355006af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3556366640 1 -- 172.21.15.53:0/3195370986 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500cd500 con 0x7f35500ce2c0 2023-10-21T17:39:11.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.039+0000 7f3554b63640 1 -- 172.21.15.53:0/3195370986 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923998666 0 0) 0x7f35500cde30 con 0x7f3550157480 2023-10-21T17:39:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.041+0000 7f3556366640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35500ce2c0 0x7f355015da90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f354802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6f13786f14a5ed server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.041+0000 7f3554b63640 1 -- 172.21.15.53:0/3195370986 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548031780 con 0x7f35500ce2c0 2023-10-21T17:39:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.041+0000 7f3554b63640 1 -- 172.21.15.53:0/3195370986 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480318e0 con 0x7f35500ce2c0 2023-10-21T17:39:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.041+0000 7f3554b63640 1 -- 172.21.15.53:0/3195370986 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548031c70 con 0x7f35500ce2c0 2023-10-21T17:39:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 -- 172.21.15.53:0/3195370986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35500ce2c0 msgr2=0x7f355015da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35500ce2c0 0x7f355015da90 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f354802b820 comp rx=0 tx=0).stop 2023-10-21T17:39:11.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 -- 172.21.15.53:0/3195370986 shutdown_connections 2023-10-21T17:39:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35500ce2c0 0x7f355015da90 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 --2- 172.21.15.53:0/3195370986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f355006b680 0x7f355015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 -- 172.21.15.53:0/3195370986 >> 172.21.15.53:0/3195370986 conn(0x7f35500d2ae0 msgr2=0x7f35500d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:11.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 -- 172.21.15.53:0/3195370986 shutdown_connections 2023-10-21T17:39:11.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.042+0000 7f3556b67640 1 -- 172.21.15.53:0/3195370986 wait complete. 2023-10-21T17:39:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.044+0000 7f3556b67640 1 Processor -- start 2023-10-21T17:39:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.044+0000 7f3556b67640 1 -- start start 2023-10-21T17:39:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f355006b680 0x7f35501f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35500ce2c0 0x7f3550147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35501f52d0 con 0x7f3550157480 2023-10-21T17:39:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3550146060 con 0x7f355006b680 2023-10-21T17:39:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556b67640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3550146340 con 0x7f35500ce2c0 2023-10-21T17:39:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47288/0 (socket says 172.21.15.53:47288) 2023-10-21T17:39:11.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 -- 172.21.15.53:0/3931433754 learned_addr learned my addr 172.21.15.53:0/3931433754 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:11.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3555364640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35500ce2c0 0x7f3550147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3555b65640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f355006b680 0x7f35501f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 -- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35500ce2c0 msgr2=0x7f3550147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35500ce2c0 0x7f3550147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.045+0000 7f3556366640 1 -- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f355006b680 msgr2=0x7f35501f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f3556366640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f355006b680 0x7f35501f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f3556366640 1 -- 172.21.15.53:0/3931433754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009400 con 0x7f3550157480 2023-10-21T17:39:11.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f3556366640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f35480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548004230 con 0x7f3550157480 2023-10-21T17:39:11.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480043b0 con 0x7f3550157480 2023-10-21T17:39:11.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548004520 con 0x7f3550157480 2023-10-21T17:39:11.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3550146620 con 0x7f3550157480 2023-10-21T17:39:11.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.046+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3550146be0 con 0x7f3550157480 2023-10-21T17:39:11.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.047+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3550146dd0 con 0x7f3550157480 2023-10-21T17:39:11.054 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.048+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 133) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3548032860 con 0x7f3550157480 2023-10-21T17:39:11.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.049+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f35480abaf0 con 0x7f3550157480 2023-10-21T17:39:11.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.053+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35480766b0 con 0x7f3550157480 2023-10-21T17:39:11.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.247+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f355004e160 con 0x7f3550157480 2023-10-21T17:39:11.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.249+0000 7f354e7fc640 1 -- 172.21.15.53:0/3931433754 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21819 (secure 0 0 0) 0x7f3548079cd0 con 0x7f3550157480 2023-10-21T17:39:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:11.251 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":239,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:10.491801+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:11.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.251+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 msgr2=0x7f3550154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.251+0000 7f3556b67640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f35480046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 shutdown_connections 2023-10-21T17:39:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3550157480 0x7f3550154360 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35500ce2c0 0x7f3550147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 --2- 172.21.15.53:0/3931433754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f355006b680 0x7f35501f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 >> 172.21.15.53:0/3931433754 conn(0x7f35500d2ae0 msgr2=0x7f35500b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:11.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 shutdown_connections 2023-10-21T17:39:11.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.252+0000 7f3556b67640 1 -- 172.21.15.53:0/3931433754 wait complete. 2023-10-21T17:39:11.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:38:30.810147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:38:31.897170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:38:45.333243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:38:46.027612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:11.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:11.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.492+0000 7f99f4abb640 1 Processor -- start 2023-10-21T17:39:11.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.493+0000 7f99f4abb640 1 -- start start 2023-10-21T17:39:11.497 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99f4abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99f015a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99f4abb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f015ac90 con 0x7f99f00cdc30 2023-10-21T17:39:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99f4abb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f015a6d0 con 0x7f99f00caef0 2023-10-21T17:39:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99f4abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015a9b0 con 0x7f99f0159bc0 2023-10-21T17:39:11.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99eeffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f99f00cdc30 0x7f99f00cc950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59454/0 (socket says 172.21.15.53:59454) 2023-10-21T17:39:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.494+0000 7f99eeffd640 1 -- 172.21.15.53:0/1822135726 learned_addr learned my addr 172.21.15.53:0/1822135726 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99effff640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99f015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594227329 0 0) 0x7f99f015ac90 con 0x7f99f00cdc30 2023-10-21T17:39:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d8005730 con 0x7f99f00cdc30 2023-10-21T17:39:11.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023522936 0 0) 0x7f99d8005730 con 0x7f99f00cdc30 2023-10-21T17:39:11.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 msgr2=0x7f99f015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99f015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 >> v1:172.21.15.124:6789/0 conn(0x7f99f00caef0 legacy=0x7f99f00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00cc150 con 0x7f99f00cdc30 2023-10-21T17:39:11.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f99e4001670 con 0x7f99f00cdc30 2023-10-21T17:39:11.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.495+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 >> v1:172.21.15.53:6789/0 conn(0x7f99f00cdc30 legacy=0x7f99f00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99d8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99d80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99d8006f10 0x7f99d8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015a9b0 con 0x7f99d8005bb0 2023-10-21T17:39:11.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d800dc70 con 0x7f99d8006f10 2023-10-21T17:39:11.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d800de30 con 0x7f99f0159bc0 2023-10-21T17:39:11.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99d8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99eeffd640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99d80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99effff640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99d8006f10 0x7f99d8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 -- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 msgr2=0x7f99d80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f0159bc0 0x7f99d80069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 -- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99d8006f10 msgr2=0x7f99d8007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99d8006f10 0x7f99d8007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.496+0000 7f99ef7fe640 1 -- 172.21.15.53:0/1822135726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8005730 con 0x7f99d8005bb0 2023-10-21T17:39:11.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.498+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99d8005fd0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f99e0001720 tx=0x7f99d8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1517a7473e721 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.499+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00052d0 con 0x7f99d8005bb0 2023-10-21T17:39:11.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.499+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e002e050 con 0x7f99d8005bb0 2023-10-21T17:39:11.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.499+0000 7f99ee7fc640 1 -- 172.21.15.53:0/1822135726 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00035f0 con 0x7f99d8005bb0 2023-10-21T17:39:11.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 -- 172.21.15.53:0/1822135726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 msgr2=0x7f99d8005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 --2- 172.21.15.53:0/1822135726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99d8005fd0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f99e0001720 tx=0x7f99d8007a00 comp rx=0 tx=0).stop 2023-10-21T17:39:11.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99effff640 1 -- 172.21.15.53:0/1822135726 reap_dead start 2023-10-21T17:39:11.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 -- 172.21.15.53:0/1822135726 shutdown_connections 2023-10-21T17:39:11.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 -- 172.21.15.53:0/1822135726 >> 172.21.15.53:0/1822135726 conn(0x7f99f006c010 msgr2=0x7f99f006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:11.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 -- 172.21.15.53:0/1822135726 shutdown_connections 2023-10-21T17:39:11.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.500+0000 7f99f4abb640 1 -- 172.21.15.53:0/1822135726 wait complete. 2023-10-21T17:39:11.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.502+0000 7f99f4abb640 1 Processor -- start 2023-10-21T17:39:11.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.502+0000 7f99f4abb640 1 -- start start 2023-10-21T17:39:11.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99f00b06d0 0x7f99f00b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f00ae290 0x7f99f00ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00b1090 con 0x7f99d8005bb0 2023-10-21T17:39:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01f23b0 con 0x7f99f00b06d0 2023-10-21T17:39:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99f4abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01561d0 con 0x7f99f00ae290 2023-10-21T17:39:11.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99ef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47306/0 (socket says 172.21.15.53:47306) 2023-10-21T17:39:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.503+0000 7f99ef7fe640 1 -- 172.21.15.53:0/3921916794 learned_addr learned my addr 172.21.15.53:0/3921916794 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99eeffd640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99f00b06d0 0x7f99f00b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 -- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f00ae290 msgr2=0x7f99f00ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f00ae290 0x7f99f00ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 -- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99f00b06d0 msgr2=0x7f99f00b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99f00b06d0 0x7f99f00b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 -- 172.21.15.53:0/3921916794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0153460 con 0x7f99d8005bb0 2023-10-21T17:39:11.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.504+0000 7f99ef7fe640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f99e0008fd0 tx=0x7f99e0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.509+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0000ed0 con 0x7f99d8005bb0 2023-10-21T17:39:11.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.509+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0007640 con 0x7f99d8005bb0 2023-10-21T17:39:11.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.509+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0006c00 con 0x7f99d8005bb0 2023-10-21T17:39:11.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.509+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0155650 con 0x7f99d8005bb0 2023-10-21T17:39:11.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.510+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0155c10 con 0x7f99d8005bb0 2023-10-21T17:39:11.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.511+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0074150 con 0x7f99d8005bb0 2023-10-21T17:39:11.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.512+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 134) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f99e00035f0 con 0x7f99d8005bb0 2023-10-21T17:39:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.513+0000 7f99d7fff640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 0x7f99dc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.513+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f99e00ab7c0 con 0x7f99d8005bb0 2023-10-21T17:39:11.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.515+0000 7f99eeffd640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 0x7f99dc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.518+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99e0076290 con 0x7f99d8005bb0 2023-10-21T17:39:11.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.521+0000 7f99eeffd640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 0x7f99dc06d1b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001eb0 tx=0x7f99e4009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.723+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99f015a410 con 0x7f99d8005bb0 2023-10-21T17:39:11.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.724+0000 7f99d7fff640 1 -- 172.21.15.53:0/3921916794 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v239) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99e004b0a0 con 0x7f99d8005bb0 2023-10-21T17:39:11.726 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:11.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 msgr2=0x7f99dc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 0x7f99dc06d1b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99e4001eb0 tx=0x7f99e4009040 comp rx=0 tx=0).stop 2023-10-21T17:39:11.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 msgr2=0x7f99f01f1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f99e0008fd0 tx=0x7f99e0001120 comp rx=0 tx=0).stop 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 shutdown_connections 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f99dc06acf0 0x7f99dc06d1b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99f00ae290 0x7f99f00ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99f00b06d0 0x7f99f00b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 --2- 172.21.15.53:0/3921916794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99d8005bb0 0x7f99f01f1e70 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 >> 172.21.15.53:0/3921916794 conn(0x7f99f006c010 msgr2=0x7f99f00c2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:11.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 shutdown_connections 2023-10-21T17:39:11.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.727+0000 7f99f4abb640 1 -- 172.21.15.53:0/3921916794 wait complete. 2023-10-21T17:39:11.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:39:11.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.982+0000 7f200c8a1640 1 Processor -- start 2023-10-21T17:39:11.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.982+0000 7f200c8a1640 1 -- start start 2023-10-21T17:39:11.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.983+0000 7f200c8a1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f20080d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.983+0000 7f200c8a1640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f20080ce510 0x7f200815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200c8a1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080cd520 con 0x7f20080cf690 2023-10-21T17:39:11.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200c8a1640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080d1910 con 0x7f20080ce510 2023-10-21T17:39:11.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200c8a1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080ce080 con 0x7f20080cf2b0 2023-10-21T17:39:11.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f1ffffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f20080cf690 0x7f20080d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59468/0 (socket says 172.21.15.53:59468) 2023-10-21T17:39:11.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f1ffffff640 1 -- 172.21.15.53:0/2811164034 learned_addr learned my addr 172.21.15.53:0/2811164034 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:11.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200759e640 1 --2- 172.21.15.53:0/2811164034 >> v2:172.21.15.124:3300/0 conn(0x7f20080ce510 0x7f200815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f2006d9d640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f20080d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670804811 0 0) 0x7f20080cd520 con 0x7f20080cf690 2023-10-21T17:39:11.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.984+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff4003520 con 0x7f20080cf690 2023-10-21T17:39:11.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799842011 0 0) 0x7f1ff4003520 con 0x7f20080cf690 2023-10-21T17:39:11.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 msgr2=0x7f20080d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f20080d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 >> v2:172.21.15.124:3300/0 conn(0x7f20080ce510 msgr2=0x7f200815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 --2- 172.21.15.53:0/2811164034 >> v2:172.21.15.124:3300/0 conn(0x7f20080ce510 0x7f200815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080cd750 con 0x7f20080cf690 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1ff0001670 con 0x7f20080cf690 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 >> v1:172.21.15.53:6789/0 conn(0x7f20080cf690 legacy=0x7f20080d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f1ff4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ff40047c0 0x7f1ff4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f1ff4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080d1910 con 0x7f1ff4003970 2023-10-21T17:39:11.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080ce080 con 0x7f1ff40047c0 2023-10-21T17:39:11.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff400b980 con 0x7f20080cf2b0 2023-10-21T17:39:11.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.985+0000 7f2006d9d640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f1ff4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f200759e640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f1ff4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f2006d9d640 1 -- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 msgr2=0x7f1ff4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f2006d9d640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f1ff4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f1ffffff640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ff40047c0 0x7f1ff4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f2006d9d640 1 -- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ff40047c0 msgr2=0x7f1ff4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f2006d9d640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ff40047c0 0x7f1ff4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f2006d9d640 1 -- 172.21.15.53:0/2811164034 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4003520 con 0x7f1ff4003970 2023-10-21T17:39:11.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.986+0000 7f200759e640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080cf2b0 0x7f1ff4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:11.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.987+0000 7f2006d9d640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f1ff4003d90 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009af0 tx=0x7f1ff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453a20b2c18a18b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.987+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff802c6b0 con 0x7f1ff4003970 2023-10-21T17:39:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.987+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff802c810 con 0x7f1ff4003970 2023-10-21T17:39:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.987+0000 7f200659c640 1 -- 172.21.15.53:0/2811164034 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff802cba0 con 0x7f1ff4003970 2023-10-21T17:39:11.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 -- 172.21.15.53:0/2811164034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 msgr2=0x7f1ff4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 --2- 172.21.15.53:0/2811164034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f1ff4003d90 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009af0 tx=0x7f1ff40057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200759e640 1 -- 172.21.15.53:0/2811164034 reap_dead start 2023-10-21T17:39:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 -- 172.21.15.53:0/2811164034 shutdown_connections 2023-10-21T17:39:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 -- 172.21.15.53:0/2811164034 >> 172.21.15.53:0/2811164034 conn(0x7f200809bc40 msgr2=0x7f20080d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:11.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 -- 172.21.15.53:0/2811164034 shutdown_connections 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.988+0000 7f200c8a1640 1 -- 172.21.15.53:0/2811164034 wait complete. 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.990+0000 7f200c8a1640 1 Processor -- start 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.990+0000 7f200c8a1640 1 -- start start 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 0x7f2008159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 0x7f20080cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080cc060 con 0x7f1ff4003970 2023-10-21T17:39:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20080b6ef0 con 0x7f20080cb6a0 2023-10-21T17:39:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.991+0000 7f200c8a1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200815d1e0 con 0x7f20080b5f40 2023-10-21T17:39:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:11.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47328/0 (socket says 172.21.15.53:47328) 2023-10-21T17:39:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 -- 172.21.15.53:0/686291778 learned_addr learned my addr 172.21.15.53:0/686291778 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f200759e640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 0x7f20080cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 -- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 msgr2=0x7f2008159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f1ffffff640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 0x7f2008159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 0x7f2008159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 -- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 msgr2=0x7f20080cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 0x7f20080cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 -- 172.21.15.53:0/686291778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080b63a0 con 0x7f1ff4003970 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f1ffffff640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 0x7f2008159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f200759e640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 0x7f20080cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:12.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.992+0000 7f2006d9d640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009af0 tx=0x7f1ff8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.993+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8003810 con 0x7f1ff4003970 2023-10-21T17:39:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.993+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff8003970 con 0x7f1ff4003970 2023-10-21T17:39:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.993+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8003ce0 con 0x7f1ff4003970 2023-10-21T17:39:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.993+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20080b6650 con 0x7f1ff4003970 2023-10-21T17:39:12.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.993+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20080b6c10 con 0x7f1ff4003970 2023-10-21T17:39:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.994+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200815d020 con 0x7f1ff4003970 2023-10-21T17:39:12.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.995+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 134) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1ff8034440 con 0x7f1ff4003970 2023-10-21T17:39:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.996+0000 7f1fff7fe640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 0x7f1fe806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.996+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f1ff80abb20 con 0x7f1ff4003970 2023-10-21T17:39:12.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.997+0000 7f1ffffff640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 0x7f1fe806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:11.998+0000 7f1ffffff640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 0x7f1fe806d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001eb0 tx=0x7f1ff0009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.001+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ff80766a0 con 0x7f1ff4003970 2023-10-21T17:39:12.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.196+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f200815d320 con 0x7f1ff4003970 2023-10-21T17:39:12.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.197+0000 7f1fff7fe640 1 -- 172.21.15.53:0/686291778 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ff804c0a0 con 0x7f1ff4003970 2023-10-21T17:39:12.199 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 msgr2=0x7f1fe806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 0x7f1fe806d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001eb0 tx=0x7f1ff0009040 comp rx=0 tx=0).stop 2023-10-21T17:39:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 msgr2=0x7f20080b8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009af0 tx=0x7f1ff8004730 comp rx=0 tx=0).stop 2023-10-21T17:39:12.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 shutdown_connections 2023-10-21T17:39:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1fe806acf0 0x7f1fe806d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20080cb6a0 0x7f20080cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20080b5f40 0x7f2008159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 --2- 172.21.15.53:0/686291778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ff4003970 0x7f20080b8350 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 >> 172.21.15.53:0/686291778 conn(0x7f200809bc40 msgr2=0x7f20080b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 shutdown_connections 2023-10-21T17:39:12.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.200+0000 7f200c8a1640 1 -- 172.21.15.53:0/686291778 wait complete. 2023-10-21T17:39:12.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:39:12.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.455+0000 7fd0b1490640 1 Processor -- start 2023-10-21T17:39:12.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.456+0000 7fd0b1490640 1 -- start start 2023-10-21T17:39:12.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0b1490640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd0ac0afbe0 0x7fd0ac0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0b1490640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0ac06dfc0 0x7fd0ac070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0b1490640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac0b2620 con 0x7fd0ac071070 2023-10-21T17:39:12.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0b1490640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b2060 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0b1490640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0b2340 con 0x7fd0ac0afbe0 2023-10-21T17:39:12.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.457+0000 7fd0a3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0ac06dfc0 0x7fd0ac070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd0ac06dfc0 0x7fd0ac070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47344/0 (socket says 172.21.15.53:47344) 2023-10-21T17:39:12.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 -- 172.21.15.53:0/1240472686 learned_addr learned my addr 172.21.15.53:0/1240472686 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:12.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0aaffd640 1 --2- 172.21.15.53:0/1240472686 >> v2:172.21.15.124:3300/0 conn(0x7fd0ac0afbe0 0x7fd0ac0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 -- 172.21.15.53:0/1240472686 >> v2:172.21.15.124:3300/0 conn(0x7fd0ac0afbe0 msgr2=0x7fd0ac0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:12.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 --2- 172.21.15.53:0/1240472686 >> v2:172.21.15.124:3300/0 conn(0x7fd0ac0afbe0 0x7fd0ac0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 -- 172.21.15.53:0/1240472686 >> v1:172.21.15.124:6789/0 conn(0x7fd0ac071070 legacy=0x7fd0ac0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:12.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.458+0000 7fd0a3fff640 1 -- 172.21.15.53:0/1240472686 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac070350 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.459+0000 7fd0a3fff640 1 --2- 172.21.15.53:0/1240472686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac070b30 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06e640 tx=0x7fd09402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd0713ce860c71 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.460+0000 7fd0aa7fc640 1 -- 172.21.15.53:0/1240472686 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031870 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.460+0000 7fd0aa7fc640 1 -- 172.21.15.53:0/1240472686 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0940319d0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.460+0000 7fd0aa7fc640 1 -- 172.21.15.53:0/1240472686 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031d60 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.460+0000 7fd0b1490640 1 -- 172.21.15.53:0/1240472686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 msgr2=0x7fd0ac070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.460+0000 7fd0b1490640 1 --2- 172.21.15.53:0/1240472686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac070b30 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac06e640 tx=0x7fd09402b910 comp rx=0 tx=0).stop 2023-10-21T17:39:12.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 -- 172.21.15.53:0/1240472686 shutdown_connections 2023-10-21T17:39:12.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 --2- 172.21.15.53:0/1240472686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac070b30 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 --2- 172.21.15.53:0/1240472686 >> v2:172.21.15.124:3300/0 conn(0x7fd0ac0afbe0 0x7fd0ac0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 -- 172.21.15.53:0/1240472686 >> 172.21.15.53:0/1240472686 conn(0x7fd0ac06a5d0 msgr2=0x7fd0ac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:12.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 -- 172.21.15.53:0/1240472686 shutdown_connections 2023-10-21T17:39:12.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.461+0000 7fd0b1490640 1 -- 172.21.15.53:0/1240472686 wait complete. 2023-10-21T17:39:12.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.463+0000 7fd0b1490640 1 Processor -- start 2023-10-21T17:39:12.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.463+0000 7fd0b1490640 1 -- start start 2023-10-21T17:39:12.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.464+0000 7fd0b1490640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.464+0000 7fd0b1490640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0ac071070 0x7fd0ac0cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0b1490640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0ac0afbe0 0x7fd0ac15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0b1490640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0cf630 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0b1490640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac14a460 con 0x7fd0ac071070 2023-10-21T17:39:12.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0b1490640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0cc7c0 con 0x7fd0ac0afbe0 2023-10-21T17:39:12.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47346/0 (socket says 172.21.15.53:47346) 2023-10-21T17:39:12.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 -- 172.21.15.53:0/526323444 learned_addr learned my addr 172.21.15.53:0/526323444 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:12.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0a3fff640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0ac071070 0x7fd0ac0cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0ab7fe640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0ac0afbe0 0x7fd0ac15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 -- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0ac0afbe0 msgr2=0x7fd0ac15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0ac0afbe0 0x7fd0ac15c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 -- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0ac071070 msgr2=0x7fd0ac0cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0ac071070 0x7fd0ac0cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.465+0000 7fd0aaffd640 1 -- 172.21.15.53:0/526323444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940094d0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0aaffd640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00d830 tx=0x7fd09c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00b6c0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00b820 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00bbb0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac0ccaa0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.466+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac0cd060 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.468+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac0cd220 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.468+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 134) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd09c01e460 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.469+0000 7fd0a37fe640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 0x7fd08c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.469+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fd09c08a200 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.470+0000 7fd0a3fff640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 0x7fd08c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.471+0000 7fd0a3fff640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 0x7fd08c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd094036000 tx=0x7fd09403c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.473+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd09c054da0 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.522+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 135) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd09c055170 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.682+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0ac04e160 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.683+0000 7fd0a37fe640 1 -- 172.21.15.53:0/526323444 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v239) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd09c087530 con 0x7fd0ac06dfc0 2023-10-21T17:39:12.684 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:12.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 msgr2=0x7fd08c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 0x7fd08c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd094036000 tx=0x7fd09403c040 comp rx=0 tx=0).stop 2023-10-21T17:39:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 msgr2=0x7fd0ac149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00d830 tx=0x7fd09c00dcf0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 shutdown_connections 2023-10-21T17:39:12.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd08c06acc0 0x7fd08c06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0ac0afbe0 0x7fd0ac15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0ac071070 0x7fd0ac0cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 --2- 172.21.15.53:0/526323444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0ac06dfc0 0x7fd0ac149f20 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 >> 172.21.15.53:0/526323444 conn(0x7fd0ac06a5d0 msgr2=0x7fd0ac15acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 shutdown_connections 2023-10-21T17:39:12.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.686+0000 7fd0b1490640 1 -- 172.21.15.53:0/526323444 wait complete. 2023-10-21T17:39:12.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:39:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.933+0000 7efea6285640 1 Processor -- start 2023-10-21T17:39:12.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.934+0000 7efea6285640 1 -- start start 2023-10-21T17:39:12.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.935+0000 7efea6285640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 0x7efea00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.935+0000 7efea6285640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efea00c8b40 0x7efea00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.935+0000 7efea6285640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00ccbc0 con 0x7efea00aff70 2023-10-21T17:39:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.935+0000 7efea6285640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea00c9510 con 0x7efea00c8b40 2023-10-21T17:39:12.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.935+0000 7efea6285640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea00ce1f0 con 0x7efea015cae0 2023-10-21T17:39:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efea00c8b40 0x7efea00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efea00c8b40 0x7efea00c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47354/0 (socket says 172.21.15.53:47354) 2023-10-21T17:39:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 -- 172.21.15.53:0/732566344 learned_addr learned my addr 172.21.15.53:0/732566344 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea5283640 1 --2- 172.21.15.53:0/732566344 >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 0x7efea00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 -- 172.21.15.53:0/732566344 >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 msgr2=0x7efea00cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 --2- 172.21.15.53:0/732566344 >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 0x7efea00cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 -- 172.21.15.53:0/732566344 >> v1:172.21.15.124:6789/0 conn(0x7efea00aff70 legacy=0x7efea00b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea4a82640 1 -- 172.21.15.53:0/732566344 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00cd590 con 0x7efea00c8b40 2023-10-21T17:39:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.936+0000 7efea5283640 1 --2- 172.21.15.53:0/732566344 >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 0x7efea00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:12.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.938+0000 7efea4a82640 1 --2- 172.21.15.53:0/732566344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c8fd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7efea00caed0 tx=0x7efe9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dbd333c2a0be49 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.938+0000 7efe97fff640 1 -- 172.21.15.53:0/732566344 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90031870 con 0x7efea00c8b40 2023-10-21T17:39:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.938+0000 7efe97fff640 1 -- 172.21.15.53:0/732566344 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe900319d0 con 0x7efea00c8b40 2023-10-21T17:39:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.938+0000 7efe97fff640 1 -- 172.21.15.53:0/732566344 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90031d60 con 0x7efea00c8b40 2023-10-21T17:39:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 -- 172.21.15.53:0/732566344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 msgr2=0x7efea00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 --2- 172.21.15.53:0/732566344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c8fd0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7efea00caed0 tx=0x7efe9002b910 comp rx=0 tx=0).stop 2023-10-21T17:39:12.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 -- 172.21.15.53:0/732566344 shutdown_connections 2023-10-21T17:39:12.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 --2- 172.21.15.53:0/732566344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c8fd0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 --2- 172.21.15.53:0/732566344 >> v2:172.21.15.124:3300/0 conn(0x7efea015cae0 0x7efea00cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 -- 172.21.15.53:0/732566344 >> 172.21.15.53:0/732566344 conn(0x7efea006c050 msgr2=0x7efea006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:12.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 -- 172.21.15.53:0/732566344 shutdown_connections 2023-10-21T17:39:12.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.939+0000 7efea6285640 1 -- 172.21.15.53:0/732566344 wait complete. 2023-10-21T17:39:12.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.941+0000 7efea6285640 1 Processor -- start 2023-10-21T17:39:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.941+0000 7efea6285640 1 -- start start 2023-10-21T17:39:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efea00aff70 0x7efea0157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 0x7efea00c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00b1f90 con 0x7efea00c8b40 2023-10-21T17:39:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c55c0 con 0x7efea015cae0 2023-10-21T17:39:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea6285640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c58a0 con 0x7efea00aff70 2023-10-21T17:39:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea4a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea4a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47358/0 (socket says 172.21.15.53:47358) 2023-10-21T17:39:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.942+0000 7efea4a82640 1 -- 172.21.15.53:0/3980845452 learned_addr learned my addr 172.21.15.53:0/3980845452 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea5283640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efea00aff70 0x7efea0157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea5a84640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 0x7efea00c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 -- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efea00aff70 msgr2=0x7efea0157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efea00aff70 0x7efea0157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 -- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 msgr2=0x7efea00c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:12.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 0x7efea00c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:12.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 -- 172.21.15.53:0/3980845452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900094d0 con 0x7efea00c8b40 2023-10-21T17:39:12.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea5a84640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 0x7efea00c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:12.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efea4a82640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7efe9002b950 tx=0x7efe900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.943+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90031860 con 0x7efea00c8b40 2023-10-21T17:39:12.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.944+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe900443f0 con 0x7efea00c8b40 2023-10-21T17:39:12.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.944+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe900447a0 con 0x7efea00c8b40 2023-10-21T17:39:12.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.944+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea00c5b80 con 0x7efea00c8b40 2023-10-21T17:39:12.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.944+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea00c6140 con 0x7efea00c8b40 2023-10-21T17:39:12.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.945+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea0074150 con 0x7efea00c8b40 2023-10-21T17:39:12.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.946+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 135) v1 ==== 89377+0+0 (secure 0 0 0) 0x7efe90044900 con 0x7efea00c8b40 2023-10-21T17:39:12.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.947+0000 7efe95ffb640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 0x7efe8406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:12.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.947+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7efe9002d040 con 0x7efea00c8b40 2023-10-21T17:39:12.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.948+0000 7efea5283640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 0x7efe8406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:12.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.949+0000 7efea5283640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 0x7efe8406d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00a950 tx=0x7efe9c008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:12.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:12.952+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe90077060 con 0x7efea00c8b40 2023-10-21T17:39:13.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.148+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efea00b4470 con 0x7efea00c8b40 2023-10-21T17:39:13.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.149+0000 7efe95ffb640 1 -- 172.21.15.53:0/3980845452 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 107+0+11 (secure 0 0 0) 0x7efe90035020 con 0x7efea00c8b40 2023-10-21T17:39:13.150 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:13.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.151+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 msgr2=0x7efe8406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.151+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 0x7efe8406d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00a950 tx=0x7efe9c008040 comp rx=0 tx=0).stop 2023-10-21T17:39:13.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.151+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 msgr2=0x7efea00c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.151+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7efe9002b950 tx=0x7efe900046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 shutdown_connections 2023-10-21T17:39:13.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe8406b050 0x7efe8406d510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efea015cae0 0x7efea00c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea00c8b40 0x7efea00c6940 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 --2- 172.21.15.53:0/3980845452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efea00aff70 0x7efea0157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 >> 172.21.15.53:0/3980845452 conn(0x7efea006c050 msgr2=0x7efea0070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 shutdown_connections 2023-10-21T17:39:13.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.152+0000 7efea6285640 1 -- 172.21.15.53:0/3980845452 wait complete. 2023-10-21T17:39:13.172 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:39:13.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.402+0000 7f175e7ce640 1 Processor -- start 2023-10-21T17:39:13.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.403+0000 7f175e7ce640 1 -- start start 2023-10-21T17:39:13.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f175e7ce640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17580c9cf0 0x7f175806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f175e7ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 0x7f17580b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f175e7ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175815b310 con 0x7f175806c3a0 2023-10-21T17:39:13.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f175e7ce640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806e0a0 con 0x7f17580c9cf0 2023-10-21T17:39:13.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f175e7ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175806d290 con 0x7f175815aa00 2023-10-21T17:39:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f1757fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f175806c3a0 0x7f175806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59476/0 (socket says 172.21.15.53:59476) 2023-10-21T17:39:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.404+0000 7f1757fff640 1 -- 172.21.15.53:0/1856039518 learned_addr learned my addr 172.21.15.53:0/1856039518 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:13.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f175cd6b640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 0x7f17580b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005424409 0 0) 0x7f1748001490 con 0x7f175806c3a0 2023-10-21T17:39:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f17577fe640 1 --2- 172.21.15.53:0/1856039518 >> v2:172.21.15.124:3300/0 conn(0x7f17580c9cf0 0x7f175806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1740003520 con 0x7f175806c3a0 2023-10-21T17:39:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807484885 0 0) 0x7f1740003520 con 0x7f175806c3a0 2023-10-21T17:39:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 msgr2=0x7f17580b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 0x7f17580b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 >> v2:172.21.15.124:3300/0 conn(0x7f17580c9cf0 msgr2=0x7f175806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.405+0000 7f1756ffd640 1 --2- 172.21.15.53:0/1856039518 >> v2:172.21.15.124:3300/0 conn(0x7f17580c9cf0 0x7f175806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580c9860 con 0x7f175806c3a0 2023-10-21T17:39:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1748001b80 con 0x7f175806c3a0 2023-10-21T17:39:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 >> v1:172.21.15.53:6789/0 conn(0x7f175806c3a0 legacy=0x7f175806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1740003970 0x7f1740003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 0x7f17400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1740004d00 0x7f1740005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806e0a0 con 0x7f1740003970 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806d290 con 0x7f1740004d00 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174000b980 con 0x7f175815aa00 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1740003970 0x7f1740003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 -- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 msgr2=0x7f17400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f175815aa00 0x7f17400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 -- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1740004d00 msgr2=0x7f1740005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1740004d00 0x7f1740005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.406+0000 7f1757fff640 1 -- 172.21.15.53:0/1856039518 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740003520 con 0x7f1740003970 2023-10-21T17:39:13.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.407+0000 7f1757fff640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1740003970 0x7f1740003d90 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f174800a790 tx=0x7f17400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed43c59243cb6a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748034630 con 0x7f1740003970 2023-10-21T17:39:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174802f070 con 0x7f1740003970 2023-10-21T17:39:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f1756ffd640 1 -- 172.21.15.53:0/1856039518 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480349f0 con 0x7f1740003970 2023-10-21T17:39:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f175e7ce640 1 -- 172.21.15.53:0/1856039518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1740003970 msgr2=0x7f1740003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f175e7ce640 1 --2- 172.21.15.53:0/1856039518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1740003970 0x7f1740003d90 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f174800a790 tx=0x7f17400057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.408+0000 7f175cd6b640 1 -- 172.21.15.53:0/1856039518 reap_dead start 2023-10-21T17:39:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.409+0000 7f175e7ce640 1 -- 172.21.15.53:0/1856039518 shutdown_connections 2023-10-21T17:39:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.409+0000 7f175e7ce640 1 -- 172.21.15.53:0/1856039518 >> 172.21.15.53:0/1856039518 conn(0x7f175806a5d0 msgr2=0x7f17580b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.409+0000 7f175e7ce640 1 -- 172.21.15.53:0/1856039518 shutdown_connections 2023-10-21T17:39:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.409+0000 7f175e7ce640 1 -- 172.21.15.53:0/1856039518 wait complete. 2023-10-21T17:39:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.411+0000 7f175e7ce640 1 Processor -- start 2023-10-21T17:39:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.411+0000 7f175e7ce640 1 -- start start 2023-10-21T17:39:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1740003970 0x7f1758163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 0x7f175815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580c4c40 con 0x7f1758159220 2023-10-21T17:39:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580c2310 con 0x7f17581596a0 2023-10-21T17:39:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175e7ce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580c25f0 con 0x7f1740003970 2023-10-21T17:39:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47388/0 (socket says 172.21.15.53:47388) 2023-10-21T17:39:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 -- 172.21.15.53:0/3063663877 learned_addr learned my addr 172.21.15.53:0/3063663877 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f1757fff640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1740003970 0x7f1758163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175cd6b640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 0x7f175815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 -- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1740003970 msgr2=0x7f1758163dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1740003970 0x7f1758163dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 -- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 msgr2=0x7f175815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 0x7f175815d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f17577fe640 1 -- 172.21.15.53:0/3063663877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580c28d0 con 0x7f1758159220 2023-10-21T17:39:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.412+0000 7f175cd6b640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 0x7f175815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f17577fe640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e860 tx=0x7f174c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c009600 con 0x7f1758159220 2023-10-21T17:39:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c00c940 con 0x7f1758159220 2023-10-21T17:39:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00ccd0 con 0x7f1758159220 2023-10-21T17:39:13.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17580c2bb0 con 0x7f1758159220 2023-10-21T17:39:13.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.413+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580c46c0 con 0x7f1758159220 2023-10-21T17:39:13.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.414+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175804e160 con 0x7f1758159220 2023-10-21T17:39:13.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.416+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 135) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f174c013470 con 0x7f1758159220 2023-10-21T17:39:13.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.416+0000 7f1754ff9640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 0x7f174406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.417+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f174c08a300 con 0x7f1758159220 2023-10-21T17:39:13.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.417+0000 7f1757fff640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 0x7f174406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.418+0000 7f1757fff640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 0x7f174406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1748034000 tx=0x7f1748009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.421+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f174c0548d0 con 0x7f1758159220 2023-10-21T17:39:13.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.615+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f1758073860 con 0x7f1758159220 2023-10-21T17:39:13.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.616+0000 7f1754ff9640 1 -- 172.21.15.53:0/3063663877 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v239) v1 ==== 103+0+11 (secure 0 0 0) 0x7f174c057ef0 con 0x7f1758159220 2023-10-21T17:39:13.617 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:13.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 msgr2=0x7f174406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 0x7f174406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1748034000 tx=0x7f1748009030 comp rx=0 tx=0).stop 2023-10-21T17:39:13.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 msgr2=0x7f17580c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e860 tx=0x7f174c00ed20 comp rx=0 tx=0).stop 2023-10-21T17:39:13.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 shutdown_connections 2023-10-21T17:39:13.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f174406b020 0x7f174406d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17581596a0 0x7f175815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1758159220 0x7f17580c4180 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 --2- 172.21.15.53:0/3063663877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1740003970 0x7f1758163dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 >> 172.21.15.53:0/3063663877 conn(0x7f175806a5d0 msgr2=0x7f17580b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:13.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 shutdown_connections 2023-10-21T17:39:13.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.619+0000 7f175e7ce640 1 -- 172.21.15.53:0/3063663877 wait complete. 2023-10-21T17:39:13.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:13.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.853+0000 7efe3759e640 1 Processor -- start 2023-10-21T17:39:13.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.853+0000 7efe3759e640 1 -- start start 2023-10-21T17:39:13.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.854+0000 7efe3759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe3806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.854+0000 7efe3759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe3806bf60 0x7efe380cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.854+0000 7efe3759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380cb590 con 0x7efe380cf720 2023-10-21T17:39:13.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.854+0000 7efe3759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe380cfc90 con 0x7efe3806bf60 2023-10-21T17:39:13.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.854+0000 7efe3759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380cff70 con 0x7efe380cd8a0 2023-10-21T17:39:13.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe35d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe380cf720 0x7efe3806b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:59490/0 (socket says 172.21.15.53:59490) 2023-10-21T17:39:13.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe35d9b640 1 -- 172.21.15.53:0/3555215275 learned_addr learned my addr 172.21.15.53:0/3555215275 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe36d9d640 1 --2- 172.21.15.53:0/3555215275 >> v2:172.21.15.124:3300/0 conn(0x7efe3806bf60 0x7efe380cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe3659c640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe3806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463828926 0 0) 0x7efe380cb590 con 0x7efe380cf720 2023-10-21T17:39:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.855+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe24003520 con 0x7efe380cf720 2023-10-21T17:39:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881962627 0 0) 0x7efe24003520 con 0x7efe380cf720 2023-10-21T17:39:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 msgr2=0x7efe3806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe3806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 >> v2:172.21.15.124:3300/0 conn(0x7efe3806bf60 msgr2=0x7efe380cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 --2- 172.21.15.53:0/3555215275 >> v2:172.21.15.124:3300/0 conn(0x7efe3806bf60 0x7efe380cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380caf40 con 0x7efe380cf720 2023-10-21T17:39:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7efe20001670 con 0x7efe380cf720 2023-10-21T17:39:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 >> v1:172.21.15.53:6789/0 conn(0x7efe380cf720 legacy=0x7efe3806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe24003970 0x7efe24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24004d00 0x7efe24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380cfc90 con 0x7efe24004d00 2023-10-21T17:39:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380cff70 con 0x7efe24003970 2023-10-21T17:39:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2400b980 con 0x7efe380cd8a0 2023-10-21T17:39:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.856+0000 7efe36d9d640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24004d00 0x7efe24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe35d9b640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe36d9d640 1 -- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 msgr2=0x7efe240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe36d9d640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380cd8a0 0x7efe240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe36d9d640 1 -- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe24003970 msgr2=0x7efe24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe36d9d640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe24003970 0x7efe24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.857+0000 7efe36d9d640 1 -- 172.21.15.53:0/3555215275 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe24003520 con 0x7efe24004d00 2023-10-21T17:39:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.858+0000 7efe36d9d640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24004d00 0x7efe24005180 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a90 tx=0x7efe240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2082a46243c8e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.858+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3002c6b0 con 0x7efe24004d00 2023-10-21T17:39:13.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.858+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3002c810 con 0x7efe24004d00 2023-10-21T17:39:13.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.858+0000 7efe3559a640 1 -- 172.21.15.53:0/3555215275 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3002cba0 con 0x7efe24004d00 2023-10-21T17:39:13.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 -- 172.21.15.53:0/3555215275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24004d00 msgr2=0x7efe24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 --2- 172.21.15.53:0/3555215275 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe24004d00 0x7efe24005180 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a90 tx=0x7efe240057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe36d9d640 1 -- 172.21.15.53:0/3555215275 reap_dead start 2023-10-21T17:39:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 -- 172.21.15.53:0/3555215275 shutdown_connections 2023-10-21T17:39:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 -- 172.21.15.53:0/3555215275 >> 172.21.15.53:0/3555215275 conn(0x7efe38099b60 msgr2=0x7efe380d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 -- 172.21.15.53:0/3555215275 shutdown_connections 2023-10-21T17:39:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.859+0000 7efe3759e640 1 -- 172.21.15.53:0/3555215275 wait complete. 2023-10-21T17:39:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.861+0000 7efe3759e640 1 Processor -- start 2023-10-21T17:39:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.861+0000 7efe3759e640 1 -- start start 2023-10-21T17:39:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe380ca300 0x7efe3809d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380ca6e0 0x7efe3809a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3809cde0 con 0x7efe3809ad20 2023-10-21T17:39:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3809e5e0 con 0x7efe380ca300 2023-10-21T17:39:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe3759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe381f67f0 con 0x7efe380ca6e0 2023-10-21T17:39:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47406/0 (socket says 172.21.15.53:47406) 2023-10-21T17:39:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.862+0000 7efe36d9d640 1 -- 172.21.15.53:0/440572565 learned_addr learned my addr 172.21.15.53:0/440572565 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe3659c640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe380ca300 0x7efe3809d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe35d9b640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380ca6e0 0x7efe3809a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 -- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380ca6e0 msgr2=0x7efe3809a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380ca6e0 0x7efe3809a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 -- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe380ca300 msgr2=0x7efe3809d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe380ca300 0x7efe3809d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 -- 172.21.15.53:0/440572565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe381f5be0 con 0x7efe3809ad20 2023-10-21T17:39:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.863+0000 7efe36d9d640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7efe30009bb0 tx=0x7efe30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.864+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3002c6b0 con 0x7efe3809ad20 2023-10-21T17:39:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.864+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30003830 con 0x7efe3809ad20 2023-10-21T17:39:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.864+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30003be0 con 0x7efe3809ad20 2023-10-21T17:39:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.864+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe381f5ec0 con 0x7efe3809ad20 2023-10-21T17:39:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.864+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe381f6480 con 0x7efe3809ad20 2023-10-21T17:39:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.865+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38151510 con 0x7efe3809ad20 2023-10-21T17:39:13.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.866+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 135) v1 ==== 89377+0+0 (secure 0 0 0) 0x7efe30003d40 con 0x7efe3809ad20 2023-10-21T17:39:13.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.867+0000 7efe2effd640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 0x7efe1806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:13.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.868+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7efe300abb40 con 0x7efe3809ad20 2023-10-21T17:39:13.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.868+0000 7efe3659c640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 0x7efe1806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:13.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.869+0000 7efe3659c640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 0x7efe1806d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe28004b30 tx=0x7efe2800d040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:13.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:13.872+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe38151510 con 0x7efe3809ad20 2023-10-21T17:39:14.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.111+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe38151510 con 0x7efe3809ad20 2023-10-21T17:39:14.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.112+0000 7efe2effd640 1 -- 172.21.15.53:0/440572565 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7efe30042090 con 0x7efe3809ad20 2023-10-21T17:39:14.113 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:14.114 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:14.114 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:39:14.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 msgr2=0x7efe1806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 0x7efe1806d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe28004b30 tx=0x7efe2800d040 comp rx=0 tx=0).stop 2023-10-21T17:39:14.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 msgr2=0x7efe381f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7efe30009bb0 tx=0x7efe30004730 comp rx=0 tx=0).stop 2023-10-21T17:39:14.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 shutdown_connections 2023-10-21T17:39:14.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe1806b050 0x7efe1806d510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3809ad20 0x7efe381f1650 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe380ca6e0 0x7efe3809a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 --2- 172.21.15.53:0/440572565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe380ca300 0x7efe3809d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 >> 172.21.15.53:0/440572565 conn(0x7efe38099b60 msgr2=0x7efe380cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:14.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 shutdown_connections 2023-10-21T17:39:14.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.115+0000 7efe3759e640 1 -- 172.21.15.53:0/440572565 wait complete. 2023-10-21T17:39:14.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:14.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.362+0000 7f9fc2b1b640 1 Processor -- start 2023-10-21T17:39:14.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.362+0000 7f9fc2b1b640 1 -- start start 2023-10-21T17:39:14.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.363+0000 7f9fc2b1b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fbc15caf0 0x7f9fbc0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc2b1b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0b5210 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc2b1b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc15ced0 con 0x7f9fbc0d1eb0 2023-10-21T17:39:14.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc2b1b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0d2470 con 0x7f9fbc15caf0 2023-10-21T17:39:14.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc1318640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9fbc0d0a00 0x7f9fbc0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40870/0 (socket says 172.21.15.53:40870) 2023-10-21T17:39:14.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc1318640 1 -- 172.21.15.53:0/1814884174 learned_addr learned my addr 172.21.15.53:0/1814884174 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183486888 0 0) 0x7f9fbc0b5210 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003520 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.364+0000 7f9fc1b19640 1 --2- 172.21.15.53:0/1814884174 >> v2:172.21.15.124:3300/0 conn(0x7f9fbc15caf0 0x7f9fbc0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311753944 0 0) 0x7f9fa4003520 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 >> v2:172.21.15.124:3300/0 conn(0x7f9fbc15caf0 msgr2=0x7f9fbc0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 --2- 172.21.15.53:0/1814884174 >> v2:172.21.15.124:3300/0 conn(0x7f9fbc15caf0 0x7f9fbc0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 >> v1:172.21.15.124:6789/0 conn(0x7f9fbc0d1eb0 legacy=0x7f9fbc0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc0d2750 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9fb0001670 con 0x7f9fbc0d0a00 2023-10-21T17:39:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 >> v1:172.21.15.53:6789/0 conn(0x7f9fbc0d0a00 legacy=0x7f9fbc0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40039c0 0x7f9fa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa40047c0 0x7f9fa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 0x7f9fa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0d2470 con 0x7f9fa40051d0 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa400c120 con 0x7f9fa40047c0 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa400c2e0 con 0x7f9fa40039c0 2023-10-21T17:39:14.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.365+0000 7f9fc231a640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 0x7f9fa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.366+0000 7f9fc231a640 1 -- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40039c0 msgr2=0x7f9fa4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.366+0000 7f9fc231a640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40039c0 0x7f9fa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.366+0000 7f9fc231a640 1 -- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa40047c0 msgr2=0x7f9fa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.366+0000 7f9fc231a640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa40047c0 0x7f9fa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.366+0000 7f9fc231a640 1 -- 172.21.15.53:0/1814884174 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4003520 con 0x7f9fa40051d0 2023-10-21T17:39:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.367+0000 7f9fc231a640 1 -- 172.21.15.53:0/1814884174 reap_dead start 2023-10-21T17:39:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.367+0000 7f9fc231a640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 0x7f9fa400b980 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001750 tx=0x7f9fa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a02c59eefeedfe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.367+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40052d0 con 0x7f9fa40051d0 2023-10-21T17:39:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.367+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb402e050 con 0x7f9fa40051d0 2023-10-21T17:39:14.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.367+0000 7f9fc0b17640 1 -- 172.21.15.53:0/1814884174 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4003830 con 0x7f9fa40051d0 2023-10-21T17:39:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 msgr2=0x7f9fa400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 0x7f9fa400b980 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001750 tx=0x7f9fa40057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/1814884174 shutdown_connections 2023-10-21T17:39:14.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/1814884174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fa40051d0 0x7f9fa400b980 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/1814884174 >> 172.21.15.53:0/1814884174 conn(0x7f9fbc071870 msgr2=0x7f9fbc071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/1814884174 shutdown_connections 2023-10-21T17:39:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.368+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/1814884174 wait complete. 2023-10-21T17:39:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.370+0000 7f9fc2b1b640 1 Processor -- start 2023-10-21T17:39:14.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.370+0000 7f9fc2b1b640 1 -- start start 2023-10-21T17:39:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40051d0 0x7f9fbc13e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fbc156120 0x7f9fbc0b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc158cb0 con 0x7f9fbc158340 2023-10-21T17:39:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc159250 con 0x7f9fbc156120 2023-10-21T17:39:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc2b1b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc159530 con 0x7f9fa40051d0 2023-10-21T17:39:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38936/0 (socket says 172.21.15.53:38936) 2023-10-21T17:39:14.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 -- 172.21.15.53:0/3763893505 learned_addr learned my addr 172.21.15.53:0/3763893505 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 -- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40051d0 msgr2=0x7f9fbc13e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40051d0 0x7f9fbc13e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 -- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fbc156120 msgr2=0x7f9fbc0b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fbc156120 0x7f9fbc0b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.371+0000 7f9fc231a640 1 -- 172.21.15.53:0/3763893505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc159810 con 0x7f9fbc158340 2023-10-21T17:39:14.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fc231a640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001870 tx=0x7f9fb4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4007030 con 0x7f9fbc158340 2023-10-21T17:39:14.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4007190 con 0x7f9fbc158340 2023-10-21T17:39:14.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4007520 con 0x7f9fbc158340 2023-10-21T17:39:14.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc0b50a0 con 0x7f9fbc158340 2023-10-21T17:39:14.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.372+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc1589a0 con 0x7f9fbc158340 2023-10-21T17:39:14.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.373+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc1590b0 con 0x7f9fbc158340 2023-10-21T17:39:14.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.374+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 135) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f9fb40067d0 con 0x7f9fbc158340 2023-10-21T17:39:14.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.375+0000 7f9fba7fc640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 0x7f9fa806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.376+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f9fb40ab300 con 0x7f9fbc158340 2023-10-21T17:39:14.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.376+0000 7f9fc1b19640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 0x7f9fa806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.377+0000 7f9fc1b19640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 0x7f9fa806d560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009540 tx=0x7f9fac009430 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.380+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fbc1590b0 con 0x7f9fbc158340 2023-10-21T17:39:14.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.540+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9fb4075b80 con 0x7f9fbc158340 2023-10-21T17:39:14.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.577+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fbc1590b0 con 0x7f9fbc158340 2023-10-21T17:39:14.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.579+0000 7f9fba7fc640 1 -- 172.21.15.53:0/3763893505 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21819 (secure 0 0 0) 0x7f9fbc1590b0 con 0x7f9fbc158340 2023-10-21T17:39:14.581 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:14.582 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":239,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:10.491801+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:14.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 msgr2=0x7f9fa806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 0x7f9fa806d560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009540 tx=0x7f9fac009430 comp rx=0 tx=0).stop 2023-10-21T17:39:14.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 msgr2=0x7f9fbc1f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4001870 tx=0x7f9fb4001120 comp rx=0 tx=0).stop 2023-10-21T17:39:14.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 shutdown_connections 2023-10-21T17:39:14.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9fa806b0a0 0x7f9fa806d560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fbc158340 0x7f9fbc1f59d0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fbc156120 0x7f9fbc0b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.582+0000 7f9fc2b1b640 1 --2- 172.21.15.53:0/3763893505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa40051d0 0x7f9fbc13e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.583+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 >> 172.21.15.53:0/3763893505 conn(0x7f9fbc071870 msgr2=0x7f9fbc151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:14.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.583+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 shutdown_connections 2023-10-21T17:39:14.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.583+0000 7f9fc2b1b640 1 -- 172.21.15.53:0/3763893505 wait complete. 2023-10-21T17:39:14.605 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:39:14.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:14.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.821+0000 7fc38d173640 1 Processor -- start 2023-10-21T17:39:14.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.821+0000 7fc38d173640 1 -- start start 2023-10-21T17:39:14.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc38d173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 0x7fc38815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc38d173640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38815f7d0 con 0x7fc3880adc70 2023-10-21T17:39:14.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc38d173640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38815f210 con 0x7fc3880b07d0 2023-10-21T17:39:14.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc38d173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815f4f0 con 0x7fc38815ab30 2023-10-21T17:39:14.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc37ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc3880adc70 0x7fc3880b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40876/0 (socket says 172.21.15.53:40876) 2023-10-21T17:39:14.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc37ffff640 1 -- 172.21.15.53:0/366305058 learned_addr learned my addr 172.21.15.53:0/366305058 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:14.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.823+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610750365 0 0) 0x7fc38815f7d0 con 0x7fc3880adc70 2023-10-21T17:39:14.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc374003520 con 0x7fc3880adc70 2023-10-21T17:39:14.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc387fff640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 0x7fc38815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317976948 0 0) 0x7fc374003520 con 0x7fc3880adc70 2023-10-21T17:39:14.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 msgr2=0x7fc38815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 0x7fc38815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 >> v1:172.21.15.124:6789/0 conn(0x7fc3880b07d0 legacy=0x7fc38806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38806c550 con 0x7fc3880adc70 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc370001670 con 0x7fc3880adc70 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 >> v1:172.21.15.53:6789/0 conn(0x7fc3880adc70 legacy=0x7fc3880b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740039a0 0x7fc374003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3740047c0 0x7fc374004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 0x7fc3740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815f4f0 con 0x7fc3740047c0 2023-10-21T17:39:14.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37400bab0 con 0x7fc3740039a0 2023-10-21T17:39:14.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37400bc70 con 0x7fc38815ab30 2023-10-21T17:39:14.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.824+0000 7fc37ffff640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3740047c0 0x7fc374004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.825+0000 7fc37ffff640 1 -- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 msgr2=0x7fc3740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.825+0000 7fc37ffff640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc38815ab30 0x7fc3740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.825+0000 7fc37ffff640 1 -- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740039a0 msgr2=0x7fc374003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:14.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.825+0000 7fc37ffff640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740039a0 0x7fc374003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.825+0000 7fc37ffff640 1 -- 172.21.15.53:0/366305058 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374003520 con 0x7fc3740047c0 2023-10-21T17:39:14.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.826+0000 7fc37ffff640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3740047c0 0x7fc374004c40 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fc37000c240 tx=0x7fc3740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e3de53a6bca57 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.826+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370005ad0 con 0x7fc3740047c0 2023-10-21T17:39:14.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.826+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc370005c30 con 0x7fc3740047c0 2023-10-21T17:39:14.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.826+0000 7fc3877fe640 1 -- 172.21.15.53:0/366305058 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37002c8b0 con 0x7fc3740047c0 2023-10-21T17:39:14.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 -- 172.21.15.53:0/366305058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3740047c0 msgr2=0x7fc374004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 --2- 172.21.15.53:0/366305058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3740047c0 0x7fc374004c40 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fc37000c240 tx=0x7fc3740057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38c972640 1 -- 172.21.15.53:0/366305058 reap_dead start 2023-10-21T17:39:14.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 -- 172.21.15.53:0/366305058 shutdown_connections 2023-10-21T17:39:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 -- 172.21.15.53:0/366305058 >> 172.21.15.53:0/366305058 conn(0x7fc3880cb0c0 msgr2=0x7fc3880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 -- 172.21.15.53:0/366305058 shutdown_connections 2023-10-21T17:39:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.827+0000 7fc38d173640 1 -- 172.21.15.53:0/366305058 wait complete. 2023-10-21T17:39:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.830+0000 7fc38d173640 1 Processor -- start 2023-10-21T17:39:14.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.830+0000 7fc38d173640 1 -- start start 2023-10-21T17:39:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740047c0 0x7fc3880c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3880cd4d0 0x7fc3881486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc388156440 con 0x7fc3880cd950 2023-10-21T17:39:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880c6970 con 0x7fc3740047c0 2023-10-21T17:39:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38d173640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880c6c50 con 0x7fc3880cd4d0 2023-10-21T17:39:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38c972640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38c972640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38958/0 (socket says 172.21.15.53:38958) 2023-10-21T17:39:14.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.831+0000 7fc38c972640 1 -- 172.21.15.53:0/3947350685 learned_addr learned my addr 172.21.15.53:0/3947350685 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:14.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 -- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3880cd4d0 msgr2=0x7fc3881486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3880cd4d0 0x7fc3881486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc387fff640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740047c0 0x7fc3880c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 -- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740047c0 msgr2=0x7fc3880c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:14.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740047c0 0x7fc3880c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:14.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 -- 172.21.15.53:0/3947350685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3880c6f30 con 0x7fc3880cd950 2023-10-21T17:39:14.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38c972640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fc388072530 tx=0x7fc38000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380005d40 con 0x7fc3880cd950 2023-10-21T17:39:14.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000a610 con 0x7fc3880cd950 2023-10-21T17:39:14.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000a9c0 con 0x7fc3880cd950 2023-10-21T17:39:14.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.832+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880c7210 con 0x7fc3880cd950 2023-10-21T17:39:14.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.833+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880c2b40 con 0x7fc3880cd950 2023-10-21T17:39:14.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.834+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3880c7400 con 0x7fc3880cd950 2023-10-21T17:39:14.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.835+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc38000ab20 con 0x7fc3880cd950 2023-10-21T17:39:14.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.836+0000 7fc3857fa640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 0x7fc36806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:14.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.836+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fc380089280 con 0x7fc3880cd950 2023-10-21T17:39:14.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.836+0000 7fc387fff640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 0x7fc36806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:14.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.837+0000 7fc387fff640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 0x7fc36806d570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc378006a90 tx=0x7fc378008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:14.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:14.840+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3800537b0 con 0x7fc3880cd950 2023-10-21T17:39:15.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.077+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc38804e160 con 0x7fc3880cd950 2023-10-21T17:39:15.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.078+0000 7fc3857fa640 1 -- 172.21.15.53:0/3947350685 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7fc380056dd0 con 0x7fc3880cd950 2023-10-21T17:39:15.080 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:15.080 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:15.081 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:39:15.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.081+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 msgr2=0x7fc36806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.081+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 0x7fc36806d570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc378006a90 tx=0x7fc378008040 comp rx=0 tx=0).stop 2023-10-21T17:39:15.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.081+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 msgr2=0x7fc388148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.081+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fc388072530 tx=0x7fc38000cb90 comp rx=0 tx=0).stop 2023-10-21T17:39:15.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 shutdown_connections 2023-10-21T17:39:15.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc36806b0b0 0x7fc36806d570 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3880cd950 0x7fc388148c20 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3880cd4d0 0x7fc3881486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 --2- 172.21.15.53:0/3947350685 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3740047c0 0x7fc3880c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 >> 172.21.15.53:0/3947350685 conn(0x7fc3880cb0c0 msgr2=0x7fc3880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:15.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 shutdown_connections 2023-10-21T17:39:15.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.082+0000 7fc38d173640 1 -- 172.21.15.53:0/3947350685 wait complete. 2023-10-21T17:39:15.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:15.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.325+0000 7f70229c0640 1 Processor -- start 2023-10-21T17:39:15.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.325+0000 7f70229c0640 1 -- start start 2023-10-21T17:39:15.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.326+0000 7f70229c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 0x7f701c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f70229c0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f701c0cb580 0x7f701c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f70229c0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c15b2b0 con 0x7f701c0c90b0 2023-10-21T17:39:15.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f70229c0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c15bf30 con 0x7f701c0cb580 2023-10-21T17:39:15.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f70229c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15f7a0 con 0x7f701c15ab40 2023-10-21T17:39:15.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f701b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f701c0c90b0 0x7f701c0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40882/0 (socket says 172.21.15.53:40882) 2023-10-21T17:39:15.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f701b7fe640 1 -- 172.21.15.53:0/2647223085 learned_addr learned my addr 172.21.15.53:0/2647223085 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f701bfff640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 0x7f701c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.327+0000 7f7020f5d640 1 --2- 172.21.15.53:0/2647223085 >> v2:172.21.15.124:3300/0 conn(0x7f701c0cb580 0x7f701c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282166292 0 0) 0x7f701c15b2b0 con 0x7f701c0c90b0 2023-10-21T17:39:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7008003520 con 0x7f701c0c90b0 2023-10-21T17:39:15.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480772372 0 0) 0x7f7008003520 con 0x7f701c0c90b0 2023-10-21T17:39:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 msgr2=0x7f701c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 0x7f701c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 >> v2:172.21.15.124:3300/0 conn(0x7f701c0cb580 msgr2=0x7f701c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 --2- 172.21.15.53:0/2647223085 >> v2:172.21.15.124:3300/0 conn(0x7f701c0cb580 0x7f701c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c15b590 con 0x7f701c0c90b0 2023-10-21T17:39:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f7004001670 con 0x7f701c0c90b0 2023-10-21T17:39:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 >> v1:172.21.15.53:6789/0 conn(0x7f701c0c90b0 legacy=0x7f701c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f7008003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 0x7f70080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7008004d00 0x7f7008005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15bf30 con 0x7f7008003970 2023-10-21T17:39:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c15f7a0 con 0x7f7008004d00 2023-10-21T17:39:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.328+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700800b980 con 0x7f701c15ab40 2023-10-21T17:39:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f7008003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 -- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 msgr2=0x7f70080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:15.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c15ab40 0x7f70080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 -- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7008004d00 msgr2=0x7f7008005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:15.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7008004d00 0x7f7008005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.329+0000 7f701bfff640 1 -- 172.21.15.53:0/2647223085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008003520 con 0x7f7008003970 2023-10-21T17:39:15.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.330+0000 7f701bfff640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f7008003d90 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f700c009fc0 tx=0x7f70080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be94f692d13f502c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.330+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c02cc30 con 0x7f7008003970 2023-10-21T17:39:15.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.330+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00f040 con 0x7f7008003970 2023-10-21T17:39:15.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.330+0000 7f701affd640 1 -- 172.21.15.53:0/2647223085 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c034440 con 0x7f7008003970 2023-10-21T17:39:15.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 -- 172.21.15.53:0/2647223085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 msgr2=0x7f7008003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 --2- 172.21.15.53:0/2647223085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f7008003d90 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f700c009fc0 tx=0x7f70080057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f7020f5d640 1 -- 172.21.15.53:0/2647223085 reap_dead start 2023-10-21T17:39:15.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 -- 172.21.15.53:0/2647223085 shutdown_connections 2023-10-21T17:39:15.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 -- 172.21.15.53:0/2647223085 >> 172.21.15.53:0/2647223085 conn(0x7f701c06a380 msgr2=0x7f701c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:15.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 -- 172.21.15.53:0/2647223085 shutdown_connections 2023-10-21T17:39:15.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.331+0000 7f70229c0640 1 -- 172.21.15.53:0/2647223085 wait complete. 2023-10-21T17:39:15.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.333+0000 7f70229c0640 1 Processor -- start 2023-10-21T17:39:15.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.333+0000 7f70229c0640 1 -- start start 2023-10-21T17:39:15.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f701c0c8060 0x7f701c1f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c0c84e0 0x7f701c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c3660 con 0x7f7008003970 2023-10-21T17:39:15.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c3900 con 0x7f701c0c8060 2023-10-21T17:39:15.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f70229c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c3be0 con 0x7f701c0c84e0 2023-10-21T17:39:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f701bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f701bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38982/0 (socket says 172.21.15.53:38982) 2023-10-21T17:39:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.334+0000 7f701bfff640 1 -- 172.21.15.53:0/2318851471 learned_addr learned my addr 172.21.15.53:0/2318851471 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 -- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c0c84e0 msgr2=0x7f701c0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c0c84e0 0x7f701c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 -- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f701c0c8060 msgr2=0x7f701c1f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f701c0c8060 0x7f701c1f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 -- 172.21.15.53:0/2318851471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c0c3ec0 con 0x7f7008003970 2023-10-21T17:39:15.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f701bfff640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f700c037040 tx=0x7f700c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c034b50 con 0x7f7008003970 2023-10-21T17:39:15.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c034cb0 con 0x7f7008003970 2023-10-21T17:39:15.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c0c41a0 con 0x7f7008003970 2023-10-21T17:39:15.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.335+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c003b90 con 0x7f7008003970 2023-10-21T17:39:15.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.336+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c15a040 con 0x7f7008003970 2023-10-21T17:39:15.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.337+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c067490 con 0x7f7008003970 2023-10-21T17:39:15.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.338+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f700c040450 con 0x7f7008003970 2023-10-21T17:39:15.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.339+0000 7f7018ff9640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 0x7f6ffc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.339+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f700c0acbc0 con 0x7f7008003970 2023-10-21T17:39:15.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.339+0000 7f701b7fe640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 0x7f6ffc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.341+0000 7f701b7fe640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 0x7f6ffc06d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f701c06de90 tx=0x7f7004001460 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.343+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f700c0771e0 con 0x7f7008003970 2023-10-21T17:39:15.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.574+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f701c15bb20 con 0x7f7008003970 2023-10-21T17:39:15.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.575+0000 7f7018ff9640 1 -- 172.21.15.53:0/2318851471 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+6153 (secure 0 0 0) 0x7f700c07a800 con 0x7f7008003970 2023-10-21T17:39:15.576 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":118,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:32.860415+0000","modified":"2023-10-21T17:38:35.943962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":37943},"failed":[],"damaged":[],"stopped":[],"info":{"gid_37943":{"gid":37943,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/748765178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":748765178},{"type":"v1","addr":"172.21.15.124:6837","nonce":748765178}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:15.577 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 118 2023-10-21T17:39:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 msgr2=0x7f6ffc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 0x7f6ffc06d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f701c06de90 tx=0x7f7004001460 comp rx=0 tx=0).stop 2023-10-21T17:39:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 msgr2=0x7f701c156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f700c037040 tx=0x7f700c009750 comp rx=0 tx=0).stop 2023-10-21T17:39:15.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 shutdown_connections 2023-10-21T17:39:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6ffc06b0b0 0x7f6ffc06d570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f701c0c84e0 0x7f701c0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f701c0c8060 0x7f701c1f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 --2- 172.21.15.53:0/2318851471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7008003970 0x7f701c156a20 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.577+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 >> 172.21.15.53:0/2318851471 conn(0x7f701c06a380 msgr2=0x7f701c154f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:15.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.578+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 shutdown_connections 2023-10-21T17:39:15.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.578+0000 7f70229c0640 1 -- 172.21.15.53:0/2318851471 wait complete. 2023-10-21T17:39:15.599 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:15.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.808+0000 7f7438c85640 1 Processor -- start 2023-10-21T17:39:15.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.808+0000 7f7438c85640 1 -- start start 2023-10-21T17:39:15.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7438c85640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f74341599d0 0x7f74340ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7438c85640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f74340b07d0 0x7f743406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7438c85640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74340cacf0 con 0x7f74340cc210 2023-10-21T17:39:15.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7438c85640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340b10f0 con 0x7f74340b07d0 2023-10-21T17:39:15.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7438c85640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340ca3c0 con 0x7f74341599d0 2023-10-21T17:39:15.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f742bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f74340cc210 0x7f74340b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40888/0 (socket says 172.21.15.53:40888) 2023-10-21T17:39:15.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f742bfff640 1 -- 172.21.15.53:0/1575588155 learned_addr learned my addr 172.21.15.53:0/1575588155 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:15.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.810+0000 7f7433fff640 1 --2- 172.21.15.53:0/1575588155 >> v2:172.21.15.53:3300/0 conn(0x7f74340b07d0 0x7f743406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f74337fe640 1 --2- 172.21.15.53:0/1575588155 >> v2:172.21.15.124:3300/0 conn(0x7f74341599d0 0x7f74340ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f7433fff640 1 -- 172.21.15.53:0/1575588155 >> v2:172.21.15.124:3300/0 conn(0x7f74341599d0 msgr2=0x7f74340ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f7433fff640 1 --2- 172.21.15.53:0/1575588155 >> v2:172.21.15.124:3300/0 conn(0x7f74341599d0 0x7f74340ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f7433fff640 1 -- 172.21.15.53:0/1575588155 >> v1:172.21.15.53:6789/0 conn(0x7f74340cc210 legacy=0x7f74340b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f7432ffd640 1 -- 172.21.15.53:0/1575588155 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729395607 0 0) 0x7f74340cacf0 con 0x7f74340cc210 2023-10-21T17:39:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.811+0000 7f7433fff640 1 -- 172.21.15.53:0/1575588155 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340ca6a0 con 0x7f74340b07d0 2023-10-21T17:39:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.812+0000 7f7433fff640 1 --2- 172.21.15.53:0/1575588155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74340b07d0 0x7f743406c310 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f7434071350 tx=0x7f742c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b5b302d522af07 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.812+0000 7f7432ffd640 1 -- 172.21.15.53:0/1575588155 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c02cc80 con 0x7f74340b07d0 2023-10-21T17:39:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.812+0000 7f7432ffd640 1 -- 172.21.15.53:0/1575588155 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c02cde0 con 0x7f74340b07d0 2023-10-21T17:39:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.812+0000 7f7432ffd640 1 -- 172.21.15.53:0/1575588155 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c034660 con 0x7f74340b07d0 2023-10-21T17:39:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 -- 172.21.15.53:0/1575588155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74340b07d0 msgr2=0x7f743406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 --2- 172.21.15.53:0/1575588155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74340b07d0 0x7f743406c310 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f7434071350 tx=0x7f742c02bed0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 -- 172.21.15.53:0/1575588155 shutdown_connections 2023-10-21T17:39:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 --2- 172.21.15.53:0/1575588155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74340b07d0 0x7f743406c310 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 --2- 172.21.15.53:0/1575588155 >> v2:172.21.15.124:3300/0 conn(0x7f74341599d0 0x7f74340ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 -- 172.21.15.53:0/1575588155 >> 172.21.15.53:0/1575588155 conn(0x7f74340cb0c0 msgr2=0x7f74340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 -- 172.21.15.53:0/1575588155 shutdown_connections 2023-10-21T17:39:15.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.813+0000 7f7438c85640 1 -- 172.21.15.53:0/1575588155 wait complete. 2023-10-21T17:39:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.815+0000 7f7438c85640 1 Processor -- start 2023-10-21T17:39:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.815+0000 7f7438c85640 1 -- start start 2023-10-21T17:39:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74340b07d0 0x7f7434148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74340cc210 0x7f74340c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7434148f50 con 0x7f74341599d0 2023-10-21T17:39:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c6950 con 0x7f74340cc210 2023-10-21T17:39:15.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7438c85640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340c6c30 con 0x7f74340b07d0 2023-10-21T17:39:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7433fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7433fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:38988/0 (socket says 172.21.15.53:38988) 2023-10-21T17:39:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.816+0000 7f7433fff640 1 -- 172.21.15.53:0/4234453410 learned_addr learned my addr 172.21.15.53:0/4234453410 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f74337fe640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74340b07d0 0x7f7434148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f742bfff640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74340cc210 0x7f74340c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 -- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74340b07d0 msgr2=0x7f7434148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74340b07d0 0x7f7434148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 -- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74340cc210 msgr2=0x7f74340c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74340cc210 0x7f74340c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:15.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 -- 172.21.15.53:0/4234453410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009a70 con 0x7f74341599d0 2023-10-21T17:39:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7433fff640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f742c009e80 tx=0x7f742c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c040650 con 0x7f74341599d0 2023-10-21T17:39:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c0407b0 con 0x7f74341599d0 2023-10-21T17:39:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c040b60 con 0x7f74341599d0 2023-10-21T17:39:15.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.817+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340c6f10 con 0x7f74341599d0 2023-10-21T17:39:15.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.818+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74340c6610 con 0x7f74341599d0 2023-10-21T17:39:15.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.819+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74340c78f0 con 0x7f74341599d0 2023-10-21T17:39:15.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.820+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f742c040cc0 con 0x7f74341599d0 2023-10-21T17:39:15.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.821+0000 7f7430ff9640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 0x7f741406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:15.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.821+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f742c0acfe0 con 0x7f74341599d0 2023-10-21T17:39:15.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.822+0000 7f74337fe640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 0x7f741406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:15.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.823+0000 7f74337fe640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 0x7f741406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7424006fd0 tx=0x7f7424008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:15.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:15.825+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f742c077510 con 0x7f74341599d0 2023-10-21T17:39:16.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.013+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74340c7710 con 0x7f74341599d0 2023-10-21T17:39:16.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.016+0000 7f7430ff9640 1 -- 172.21.15.53:0/4234453410 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21819 (secure 0 0 0) 0x7f74340c7710 con 0x7f74341599d0 2023-10-21T17:39:16.018 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:16.019 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":239,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:10.491801+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:38:30.810147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-21T17:38:31.897170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:16.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.018+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 msgr2=0x7f741406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.018+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 0x7f741406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7424006fd0 tx=0x7f7424008040 comp rx=0 tx=0).stop 2023-10-21T17:39:16.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.018+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 msgr2=0x7f74340ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f742c009e80 tx=0x7f742c004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:16.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 shutdown_connections 2023-10-21T17:39:16.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f741406b0b0 0x7f741406d570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74341599d0 0x7f74340ce850 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f74340cc210 0x7f74340c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 --2- 172.21.15.53:0/4234453410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f74340b07d0 0x7f7434148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 >> 172.21.15.53:0/4234453410 conn(0x7f74340cb0c0 msgr2=0x7f7434158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:16.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 shutdown_connections 2023-10-21T17:39:16.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.019+0000 7f7438c85640 1 -- 172.21.15.53:0/4234453410 wait complete. 2023-10-21T17:39:16.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:39:16.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.256+0000 7f011adce640 1 Processor -- start 2023-10-21T17:39:16.259 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.257+0000 7f011adce640 1 -- start start 2023-10-21T17:39:16.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011adce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f01140c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011adce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f011406db60 0x7f011406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011adce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011415b310 con 0x7f011415aa00 2023-10-21T17:39:16.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011adce640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01140ca230 con 0x7f011406db60 2023-10-21T17:39:16.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011adce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011406d290 con 0x7f011406c3a0 2023-10-21T17:39:16.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f0113fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f011406db60 0x7f011406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011936b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f011415aa00 0x7f01140b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40894/0 (socket says 172.21.15.53:40894) 2023-10-21T17:39:16.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f011936b640 1 -- 172.21.15.53:0/2755253188 learned_addr learned my addr 172.21.15.53:0/2755253188 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:16.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.258+0000 7f0118b6a640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f01140c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f0113fff640 1 -- 172.21.15.53:0/2755253188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 msgr2=0x7f01140c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:16.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f0113fff640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f01140c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f0113fff640 1 -- 172.21.15.53:0/2755253188 >> v1:172.21.15.53:6789/0 conn(0x7f011415aa00 legacy=0x7f01140b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f0113fff640 1 -- 172.21.15.53:0/2755253188 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140c9860 con 0x7f011406db60 2023-10-21T17:39:16.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f01137fe640 1 -- 172.21.15.53:0/2755253188 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999525681 0 0) 0x7f010c001490 con 0x7f011415aa00 2023-10-21T17:39:16.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.259+0000 7f0118b6a640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f01140c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:16.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.260+0000 7f0113fff640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011406db60 0x7f011406c780 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f01000097a0 tx=0x7f010002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e10c961e81ce28 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.260+0000 7f01137fe640 1 -- 172.21.15.53:0/2755253188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010002d040 con 0x7f011406db60 2023-10-21T17:39:16.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.260+0000 7f01137fe640 1 -- 172.21.15.53:0/2755253188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0100031810 con 0x7f011406db60 2023-10-21T17:39:16.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.260+0000 7f01137fe640 1 -- 172.21.15.53:0/2755253188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0100031ba0 con 0x7f011406db60 2023-10-21T17:39:16.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 -- 172.21.15.53:0/2755253188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011406db60 msgr2=0x7f011406c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011406db60 0x7f011406c780 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f01000097a0 tx=0x7f010002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 -- 172.21.15.53:0/2755253188 shutdown_connections 2023-10-21T17:39:16.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011406db60 0x7f011406c780 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 --2- 172.21.15.53:0/2755253188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f01140c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 -- 172.21.15.53:0/2755253188 >> 172.21.15.53:0/2755253188 conn(0x7f011406a5d0 msgr2=0x7f01140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:16.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 -- 172.21.15.53:0/2755253188 shutdown_connections 2023-10-21T17:39:16.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.261+0000 7f011adce640 1 -- 172.21.15.53:0/2755253188 wait complete. 2023-10-21T17:39:16.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.263+0000 7f011adce640 1 Processor -- start 2023-10-21T17:39:16.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.263+0000 7f011adce640 1 -- start start 2023-10-21T17:39:16.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f0114151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f011406db60 0x7f0114153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0114150b20 con 0x7f011415aa00 2023-10-21T17:39:16.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011414ffa0 con 0x7f011406db60 2023-10-21T17:39:16.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011adce640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0114151680 con 0x7f011406c3a0 2023-10-21T17:39:16.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.264+0000 7f011936b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39006/0 (socket says 172.21.15.53:39006) 2023-10-21T17:39:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 -- 172.21.15.53:0/2899797753 learned_addr learned my addr 172.21.15.53:0/2899797753 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f0113fff640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f011406db60 0x7f0114153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f0118b6a640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f0114151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 -- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 msgr2=0x7f0114151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f0114151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 -- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f011406db60 msgr2=0x7f0114153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f011406db60 0x7f0114153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 -- 172.21.15.53:0/2899797753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009400 con 0x7f011415aa00 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f011936b640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c780 tx=0x7f010c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.265+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00ce90 con 0x7f011415aa00 2023-10-21T17:39:16.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.266+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c00a640 con 0x7f011415aa00 2023-10-21T17:39:16.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.266+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00a9d0 con 0x7f011415aa00 2023-10-21T17:39:16.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.266+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01141501f0 con 0x7f011415aa00 2023-10-21T17:39:16.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.266+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01141507b0 con 0x7f011415aa00 2023-10-21T17:39:16.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.267+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011404e160 con 0x7f011415aa00 2023-10-21T17:39:16.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.268+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f010c00ab30 con 0x7f011415aa00 2023-10-21T17:39:16.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.269+0000 7f01117fa640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 0x7f00f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.270+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f010c0894b0 con 0x7f011415aa00 2023-10-21T17:39:16.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.270+0000 7f0118b6a640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 0x7f00f806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.271+0000 7f0118b6a640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 0x7f00f806d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f011415a3f0 tx=0x7f0108008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.274+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f010c053a40 con 0x7f011415aa00 2023-10-21T17:39:16.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.502+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f011415b310 con 0x7f011415aa00 2023-10-21T17:39:16.504 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:39:16.503+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:39:16.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.507+0000 7f01117fa640 1 -- 172.21.15.53:0/2899797753 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f011415b310 con 0x7f011415aa00 2023-10-21T17:39:16.508 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:39:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.509+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 msgr2=0x7f00f806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.509+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 0x7f00f806d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f011415a3f0 tx=0x7f0108008040 comp rx=0 tx=0).stop 2023-10-21T17:39:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.509+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 msgr2=0x7f01141541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.509+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c780 tx=0x7f010c00cc40 comp rx=0 tx=0).stop 2023-10-21T17:39:16.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 shutdown_connections 2023-10-21T17:39:16.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f00f806b0b0 0x7f00f806d570 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f011415aa00 0x7f01141541d0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f011406db60 0x7f0114153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 --2- 172.21.15.53:0/2899797753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f011406c3a0 0x7f0114151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 >> 172.21.15.53:0/2899797753 conn(0x7f011406a5d0 msgr2=0x7f01140b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:16.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 shutdown_connections 2023-10-21T17:39:16.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.510+0000 7f011adce640 1 -- 172.21.15.53:0/2899797753 wait complete. 2023-10-21T17:39:16.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:39:16.545 INFO:tasks.ceph.mds.d.smithi124.stderr: -4658> 2023-10-21T17:38:37.351+0000 7efef8cb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:39:16.626 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:39:16.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.743+0000 7f04ff59e640 1 Processor -- start 2023-10-21T17:39:16.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.743+0000 7f04ff59e640 1 -- start start 2023-10-21T17:39:16.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04ff59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05000cb5e0 0x7f05000cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04ff59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 0x7f05000cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04ff59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0500159db0 con 0x7f05001599d0 2023-10-21T17:39:16.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04ff59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0500071fe0 con 0x7f05000cb5e0 2023-10-21T17:39:16.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04ff59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05000b44c0 con 0x7f05000b3c50 2023-10-21T17:39:16.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04fdd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05000cb5e0 0x7f05000cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04fdd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f05000cb5e0 0x7f05000cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39020/0 (socket says 172.21.15.53:39020) 2023-10-21T17:39:16.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.745+0000 7f04fdd9b640 1 -- 172.21.15.53:0/945336996 learned_addr learned my addr 172.21.15.53:0/945336996 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:16.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fdd9b640 1 -- 172.21.15.53:0/945336996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 msgr2=0x7f05000cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:16.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fed9d640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 0x7f05000cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fdd9b640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 0x7f05000cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fdd9b640 1 -- 172.21.15.53:0/945336996 >> v1:172.21.15.124:6789/0 conn(0x7f05001599d0 legacy=0x7f05000caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:16.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fdd9b640 1 -- 172.21.15.53:0/945336996 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05000713d0 con 0x7f05000cb5e0 2023-10-21T17:39:16.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.746+0000 7f04fed9d640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 0x7f05000cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:16.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.747+0000 7f04fdd9b640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000cb5e0 0x7f05000cdcd0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009870 tx=0x7f04e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e3ff1d4d75932 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.747+0000 7f04fd59a640 1 -- 172.21.15.53:0/945336996 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8031640 con 0x7f05000cb5e0 2023-10-21T17:39:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.747+0000 7f04fd59a640 1 -- 172.21.15.53:0/945336996 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e80317a0 con 0x7f05000cb5e0 2023-10-21T17:39:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.747+0000 7f04fd59a640 1 -- 172.21.15.53:0/945336996 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8031b30 con 0x7f05000cb5e0 2023-10-21T17:39:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 -- 172.21.15.53:0/945336996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000cb5e0 msgr2=0x7f05000cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000cb5e0 0x7f05000cdcd0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009870 tx=0x7f04e802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 -- 172.21.15.53:0/945336996 shutdown_connections 2023-10-21T17:39:16.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05000b3c50 0x7f05000cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 --2- 172.21.15.53:0/945336996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000cb5e0 0x7f05000cdcd0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 -- 172.21.15.53:0/945336996 >> 172.21.15.53:0/945336996 conn(0x7f050006c050 msgr2=0x7f050006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:16.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 -- 172.21.15.53:0/945336996 shutdown_connections 2023-10-21T17:39:16.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.748+0000 7f04ff59e640 1 -- 172.21.15.53:0/945336996 wait complete. 2023-10-21T17:39:16.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.750+0000 7f04ff59e640 1 Processor -- start 2023-10-21T17:39:16.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.750+0000 7f04ff59e640 1 -- start start 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000cb5e0 0x7f0500145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 0x7f0500145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05001efca0 con 0x7f05000b3c50 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500154540 con 0x7f05000cb5e0 2023-10-21T17:39:16.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04ff59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0500154820 con 0x7f05001599d0 2023-10-21T17:39:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04fe59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39030/0 (socket says 172.21.15.53:39030) 2023-10-21T17:39:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04fe59c640 1 -- 172.21.15.53:0/2531129190 learned_addr learned my addr 172.21.15.53:0/2531129190 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04fdd9b640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000cb5e0 0x7f0500145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.751+0000 7f04fed9d640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 0x7f0500145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 -- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 msgr2=0x7f0500145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 0x7f0500145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 -- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000cb5e0 msgr2=0x7f0500145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000cb5e0 0x7f0500145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:16.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 -- 172.21.15.53:0/2531129190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e80094d0 con 0x7f05000b3c50 2023-10-21T17:39:16.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fed9d640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 0x7f0500145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:16.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04fe59c640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b3f0 tx=0x7f04f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0011830 con 0x7f05000b3c50 2023-10-21T17:39:16.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f0011990 con 0x7f05000b3c50 2023-10-21T17:39:16.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f0011d40 con 0x7f05000b3c50 2023-10-21T17:39:16.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0500154b00 con 0x7f05000b3c50 2023-10-21T17:39:16.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.752+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0500143190 con 0x7f05000b3c50 2023-10-21T17:39:16.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.753+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500074150 con 0x7f05000b3c50 2023-10-21T17:39:16.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.754+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f04f001a400 con 0x7f05000b3c50 2023-10-21T17:39:16.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.755+0000 7f04f6ffd640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 0x7f04e006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:16.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.755+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7912+0+0 (secure 0 0 0) 0x7f04f008a200 con 0x7f05000b3c50 2023-10-21T17:39:16.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.756+0000 7f04fdd9b640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 0x7f04e006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.757+0000 7f04fdd9b640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 0x7f04e006d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04e8036000 tx=0x7f04e803c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:16.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.760+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04f0054610 con 0x7f05000b3c50 2023-10-21T17:39:16.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:16.981+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0500158fc0 con 0x7f05000b3c50 2023-10-21T17:39:17.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.563+0000 7f04f6ffd640 1 -- 172.21.15.53:0/2531129190 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 104+0+0 (secure 0 0 0) 0x7f04f0057c30 con 0x7f05000b3c50 2023-10-21T17:39:17.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 msgr2=0x7f04e006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 0x7f04e006d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04e8036000 tx=0x7f04e803c040 comp rx=0 tx=0).stop 2023-10-21T17:39:17.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 msgr2=0x7f0500153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f04f000b3f0 tx=0x7f04f000b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 shutdown_connections 2023-10-21T17:39:17.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f04e006b0b0 0x7f04e006d570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05001599d0 0x7f0500145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05000cb5e0 0x7f0500145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 --2- 172.21.15.53:0/2531129190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05000b3c50 0x7f0500153bd0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 >> 172.21.15.53:0/2531129190 conn(0x7f050006c050 msgr2=0x7f05000affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:17.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 shutdown_connections 2023-10-21T17:39:17.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.567+0000 7f04ff59e640 1 -- 172.21.15.53:0/2531129190 wait complete. 2023-10-21T17:39:17.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:39:17.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.822+0000 7f5641689640 1 Processor -- start 2023-10-21T17:39:17.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.822+0000 7f5641689640 1 -- start start 2023-10-21T17:39:17.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f5641689640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06c3a0 0x7f563c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f5641689640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f563c06db60 0x7f563c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f5641689640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c15b310 con 0x7f563c15aa00 2023-10-21T17:39:17.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f5641689640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0ca230 con 0x7f563c06db60 2023-10-21T17:39:17.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f5641689640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c06d290 con 0x7f563c06c3a0 2023-10-21T17:39:17.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f563b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f563c15aa00 0x7f563c0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40910/0 (socket says 172.21.15.53:40910) 2023-10-21T17:39:17.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f563b7fe640 1 -- 172.21.15.53:0/3058932337 learned_addr learned my addr 172.21.15.53:0/3058932337 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:17.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3058932337 >> v2:172.21.15.53:3300/0 conn(0x7f563c06db60 0x7f563c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.824+0000 7f563affd640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06c3a0 0x7f563c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f5639ffb640 1 -- 172.21.15.53:0/3058932337 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247370280 0 0) 0x7f5634001490 con 0x7f563c15aa00 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f5639ffb640 1 -- 172.21.15.53:0/3058932337 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5624003520 con 0x7f563c15aa00 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f563a7fc640 1 -- 172.21.15.53:0/3058932337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06c3a0 msgr2=0x7f563c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06c3a0 0x7f563c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f563a7fc640 1 -- 172.21.15.53:0/3058932337 >> v1:172.21.15.53:6789/0 conn(0x7f563c15aa00 legacy=0x7f563c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.825+0000 7f563a7fc640 1 -- 172.21.15.53:0/3058932337 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0c9860 con 0x7f563c06db60 2023-10-21T17:39:17.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.826+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06db60 0x7f563c06c780 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dba772322df11ea server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:17.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.826+0000 7f5639ffb640 1 -- 172.21.15.53:0/3058932337 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563002d040 con 0x7f563c06db60 2023-10-21T17:39:17.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.826+0000 7f5639ffb640 1 -- 172.21.15.53:0/3058932337 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630031810 con 0x7f563c06db60 2023-10-21T17:39:17.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.826+0000 7f5639ffb640 1 -- 172.21.15.53:0/3058932337 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630031ba0 con 0x7f563c06db60 2023-10-21T17:39:17.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 -- 172.21.15.53:0/3058932337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06db60 msgr2=0x7f563c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06db60 0x7f563c06c780 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 -- 172.21.15.53:0/3058932337 shutdown_connections 2023-10-21T17:39:17.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06db60 0x7f563c06c780 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 --2- 172.21.15.53:0/3058932337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06c3a0 0x7f563c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 -- 172.21.15.53:0/3058932337 >> 172.21.15.53:0/3058932337 conn(0x7f563c06a5d0 msgr2=0x7f563c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:17.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 -- 172.21.15.53:0/3058932337 shutdown_connections 2023-10-21T17:39:17.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.827+0000 7f5641689640 1 -- 172.21.15.53:0/3058932337 wait complete. 2023-10-21T17:39:17.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.829+0000 7f5641689640 1 Processor -- start 2023-10-21T17:39:17.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.829+0000 7f5641689640 1 -- start start 2023-10-21T17:39:17.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 0x7f563c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f563c15aa00 0x7f563c14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1536e0 con 0x7f563c06c3a0 2023-10-21T17:39:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1585b0 con 0x7f563c15aa00 2023-10-21T17:39:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f5641689640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c158890 con 0x7f563c06db60 2023-10-21T17:39:17.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f563affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.830+0000 7f563affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39050/0 (socket says 172.21.15.53:39050) 2023-10-21T17:39:17.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 -- 172.21.15.53:0/3254807259 learned_addr learned my addr 172.21.15.53:0/3254807259 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:17.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563b7fe640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f563c15aa00 0x7f563c14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 0x7f563c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 -- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 msgr2=0x7f563c155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 0x7f563c155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 -- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f563c15aa00 msgr2=0x7f563c14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f563c15aa00 0x7f563c14e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 -- 172.21.15.53:0/3254807259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5630009400 con 0x7f563c06c3a0 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 0x7f563c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:17.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.831+0000 7f563affd640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f562c00d800 tx=0x7f562c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.832+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00b6c0 con 0x7f563c06c3a0 2023-10-21T17:39:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.832+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c00b820 con 0x7f563c06c3a0 2023-10-21T17:39:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.832+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c00bbb0 con 0x7f563c06c3a0 2023-10-21T17:39:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.832+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c158b70 con 0x7f563c06c3a0 2023-10-21T17:39:17.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.832+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c159130 con 0x7f563c06c3a0 2023-10-21T17:39:17.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.833+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c1592c0 con 0x7f563c06c3a0 2023-10-21T17:39:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.834+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f562c00bd10 con 0x7f563c06c3a0 2023-10-21T17:39:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.835+0000 7f56237fe640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 0x7f562806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.835+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7912+0+0 (secure 0 0 0) 0x7f562c08a7a0 con 0x7f563c06c3a0 2023-10-21T17:39:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.836+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 0x7f562806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:17.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.837+0000 7f563a7fc640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 0x7f562806d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5630036000 tx=0x7f563003c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:17.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:17.840+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f562c054d30 con 0x7f563c06c3a0 2023-10-21T17:39:18.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.031+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f563c15b090 con 0x7f563c06c3a0 2023-10-21T17:39:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.570+0000 7f56237fe640 1 -- 172.21.15.53:0/3254807259 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v241) v1 ==== 181+0+0 (secure 0 0 0) 0x7f562c058350 con 0x7f563c06c3a0 2023-10-21T17:39:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:39:18.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 msgr2=0x7f562806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 0x7f562806d5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5630036000 tx=0x7f563003c040 comp rx=0 tx=0).stop 2023-10-21T17:39:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 msgr2=0x7f563c157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f562c00d800 tx=0x7f562c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 shutdown_connections 2023-10-21T17:39:18.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f562806b100 0x7f562806d5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f563c15aa00 0x7f563c14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f563c06db60 0x7f563c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 --2- 172.21.15.53:0/3254807259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f563c06c3a0 0x7f563c157fd0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 >> 172.21.15.53:0/3254807259 conn(0x7f563c06a5d0 msgr2=0x7f563c0c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:18.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 shutdown_connections 2023-10-21T17:39:18.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.574+0000 7f5641689640 1 -- 172.21.15.53:0/3254807259 wait complete. 2023-10-21T17:39:18.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:39:18.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.827+0000 7fd8de7c9640 1 Processor -- start 2023-10-21T17:39:18.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.827+0000 7fd8de7c9640 1 -- start start 2023-10-21T17:39:18.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8de7c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 0x7fd8d815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8de7c9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d80cab60 con 0x7fd8d815a820 2023-10-21T17:39:18.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8de7c9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d80cb960 con 0x7fd8d80b0680 2023-10-21T17:39:18.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8de7c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80cbdf0 con 0x7fd8d80ca680 2023-10-21T17:39:18.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8dd7c7640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd8d815a820 0x7fd8d80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40922/0 (socket says 172.21.15.53:40922) 2023-10-21T17:39:18.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8dd7c7640 1 -- 172.21.15.53:0/3115353016 learned_addr learned my addr 172.21.15.53:0/3115353016 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:18.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777871945 0 0) 0x7fd8d80cab60 con 0x7fd8d815a820 2023-10-21T17:39:18.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8c0003520 con 0x7fd8d815a820 2023-10-21T17:39:18.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.829+0000 7fd8ddfc8640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 0x7fd8d815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 32458947 0 0) 0x7fd8c0003520 con 0x7fd8d815a820 2023-10-21T17:39:18.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 msgr2=0x7fd8d815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:18.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 0x7fd8d815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 >> v1:172.21.15.124:6789/0 conn(0x7fd8d80b0680 legacy=0x7fd8d815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:18.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80cb030 con 0x7fd8d815a820 2023-10-21T17:39:18.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd8c8001670 con 0x7fd8d815a820 2023-10-21T17:39:18.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 >> v1:172.21.15.53:6789/0 conn(0x7fd8d815a820 legacy=0x7fd8d80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8c00039a0 0x7fd8c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c00047c0 0x7fd8c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 0x7fd8c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80cbdf0 con 0x7fd8c00039a0 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00054e0 con 0x7fd8c00047c0 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c000bec0 con 0x7fd8d80ca680 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.830+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8c00039a0 0x7fd8c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dcfc6640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c00047c0 0x7fd8c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dd7c7640 1 -- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 msgr2=0x7fd8c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d80ca680 0x7fd8c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dd7c7640 1 -- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c00047c0 msgr2=0x7fd8c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c00047c0 0x7fd8c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dd7c7640 1 -- 172.21.15.53:0/3115353016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c0003520 con 0x7fd8c00039a0 2023-10-21T17:39:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.831+0000 7fd8dcfc6640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8c00047c0 0x7fd8c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.832+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8c00039a0 0x7fd8c0003dc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800c240 tx=0x7fd8c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf1c44e6b579a91 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:18.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.832+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8005ad0 con 0x7fd8c00039a0 2023-10-21T17:39:18.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.832+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c8005c30 con 0x7fd8c00039a0 2023-10-21T17:39:18.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.832+0000 7fd8cffff640 1 -- 172.21.15.53:0/3115353016 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c802c8b0 con 0x7fd8c00039a0 2023-10-21T17:39:18.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 -- 172.21.15.53:0/3115353016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8c00039a0 msgr2=0x7fd8c0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/3115353016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8c00039a0 0x7fd8c0003dc0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800c240 tx=0x7fd8c00057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8ddfc8640 1 -- 172.21.15.53:0/3115353016 reap_dead start 2023-10-21T17:39:18.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 -- 172.21.15.53:0/3115353016 shutdown_connections 2023-10-21T17:39:18.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 -- 172.21.15.53:0/3115353016 >> 172.21.15.53:0/3115353016 conn(0x7fd8d806c050 msgr2=0x7fd8d806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:18.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 -- 172.21.15.53:0/3115353016 shutdown_connections 2023-10-21T17:39:18.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.833+0000 7fd8de7c9640 1 -- 172.21.15.53:0/3115353016 wait complete. 2023-10-21T17:39:18.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.835+0000 7fd8de7c9640 1 Processor -- start 2023-10-21T17:39:18.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.835+0000 7fd8de7c9640 1 -- start start 2023-10-21T17:39:18.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 0x7fd8d80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 0x7fd8d815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80c77f0 con 0x7fd8d80c47f0 2023-10-21T17:39:18.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80c6be0 con 0x7fd8d8158b30 2023-10-21T17:39:18.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8de7c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80c6ec0 con 0x7fd8d8158f10 2023-10-21T17:39:18.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39060/0 (socket says 172.21.15.53:39060) 2023-10-21T17:39:18.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 -- 172.21.15.53:0/2398119153 learned_addr learned my addr 172.21.15.53:0/2398119153 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:18.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 0x7fd8d80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8dcfc6640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 0x7fd8d815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 -- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 msgr2=0x7fd8d815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 0x7fd8d815cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 -- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 msgr2=0x7fd8d80c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:18.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 0x7fd8d80c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:18.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.836+0000 7fd8ddfc8640 1 -- 172.21.15.53:0/2398119153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80c71a0 con 0x7fd8d80c47f0 2023-10-21T17:39:18.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8dcfc6640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 0x7fd8d815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:18.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 0x7fd8d80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:18.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8ddfc8640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80ad940 tx=0x7fd8d0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:18.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000ebc0 con 0x7fd8d80c47f0 2023-10-21T17:39:18.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000ed40 con 0x7fd8d80c47f0 2023-10-21T17:39:18.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0014660 con 0x7fd8d80c47f0 2023-10-21T17:39:18.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d80c7480 con 0x7fd8d80c47f0 2023-10-21T17:39:18.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.837+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d815b000 con 0x7fd8d80c47f0 2023-10-21T17:39:18.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.838+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d80c7960 con 0x7fd8d80c47f0 2023-10-21T17:39:18.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.839+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd8d00147c0 con 0x7fd8d80c47f0 2023-10-21T17:39:18.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.840+0000 7fd8cdffb640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 0x7fd8c406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:18.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.840+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7474+0+0 (secure 0 0 0) 0x7fd8d008a3f0 con 0x7fd8d80c47f0 2023-10-21T17:39:18.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.841+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 0x7fd8c406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:18.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.842+0000 7fd8dd7c7640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 0x7fd8c406d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8034000 tx=0x7fd8c8009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:18.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:18.845+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8d0054b10 con 0x7fd8d80c47f0 2023-10-21T17:39:19.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.041+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8d80cb5c0 con 0x7fd8d80c47f0 2023-10-21T17:39:19.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.574+0000 7fd8cdffb640 1 -- 172.21.15.53:0/2398119153 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v242) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd8d0058130 con 0x7fd8d80c47f0 2023-10-21T17:39:19.576 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:39:19.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.580+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 msgr2=0x7fd8c406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.580+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 0x7fd8c406d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8034000 tx=0x7fd8c8009030 comp rx=0 tx=0).stop 2023-10-21T17:39:19.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.580+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 msgr2=0x7fd8d815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.580+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80ad940 tx=0x7fd8d0007e80 comp rx=0 tx=0).stop 2023-10-21T17:39:19.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 shutdown_connections 2023-10-21T17:39:19.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd8c406b0b0 0x7fd8c406d570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8d80c47f0 0x7fd8d815d220 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8d8158f10 0x7fd8d815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 --2- 172.21.15.53:0/2398119153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8d8158b30 0x7fd8d80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.581+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 >> 172.21.15.53:0/2398119153 conn(0x7fd8d806c050 msgr2=0x7fd8d80b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:19.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.583+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 shutdown_connections 2023-10-21T17:39:19.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.583+0000 7fd8de7c9640 1 -- 172.21.15.53:0/2398119153 wait complete. 2023-10-21T17:39:19.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:19.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.811+0000 7fdf03fff640 1 Processor -- start 2023-10-21T17:39:19.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.811+0000 7fdf03fff640 1 -- start start 2023-10-21T17:39:19.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdf04165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf03fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0415f7d0 con 0x7fdf040adc70 2023-10-21T17:39:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf03fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0415efb0 con 0x7fdf0415ab30 2023-10-21T17:39:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf03fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0415f210 con 0x7fdf040b07d0 2023-10-21T17:39:19.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf027fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdf040adc70 0x7fdf040b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40938/0 (socket says 172.21.15.53:40938) 2023-10-21T17:39:19.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.813+0000 7fdf027fc640 1 -- 172.21.15.53:0/3953589578 learned_addr learned my addr 172.21.15.53:0/3953589578 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:19.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf037fe640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdf04165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267561964 0 0) 0x7fdf0415f7d0 con 0x7fdf040adc70 2023-10-21T17:39:19.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef0003520 con 0x7fdf040adc70 2023-10-21T17:39:19.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693467617 0 0) 0x7fdef0003520 con 0x7fdf040adc70 2023-10-21T17:39:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 msgr2=0x7fdf04165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdf04165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 >> v1:172.21.15.124:6789/0 conn(0x7fdf0415ab30 legacy=0x7fdf0415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0415f4f0 con 0x7fdf040adc70 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.814+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fdeec001670 con 0x7fdf040adc70 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 >> v1:172.21.15.53:6789/0 conn(0x7fdf040adc70 legacy=0x7fdf040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00039a0 0x7fdef0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef00047c0 0x7fdef0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdef0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0415f210 con 0x7fdef00039a0 2023-10-21T17:39:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000ba90 con 0x7fdef00047c0 2023-10-21T17:39:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef000bc50 con 0x7fdf040b07d0 2023-10-21T17:39:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00039a0 0x7fdef0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf027fc640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef00047c0 0x7fdef0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.815+0000 7fdf037fe640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdef0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.816+0000 7fdf02ffd640 1 -- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 msgr2=0x7fdef0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.816+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf040b07d0 0x7fdef0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.816+0000 7fdf02ffd640 1 -- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef00047c0 msgr2=0x7fdef0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.816+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdef00047c0 0x7fdef0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.816+0000 7fdf02ffd640 1 -- 172.21.15.53:0/3953589578 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0003520 con 0x7fdef00039a0 2023-10-21T17:39:19.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.817+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00039a0 0x7fdef0003dc0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fdf0415f140 tx=0x7fdef00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fb95a6604e23fe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4003b70 con 0x7fdef00039a0 2023-10-21T17:39:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4003cd0 con 0x7fdef00039a0 2023-10-21T17:39:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf01ffb640 1 -- 172.21.15.53:0/3953589578 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4005310 con 0x7fdef00039a0 2023-10-21T17:39:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf03fff640 1 -- 172.21.15.53:0/3953589578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00039a0 msgr2=0x7fdef0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3953589578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdef00039a0 0x7fdef0003dc0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fdf0415f140 tx=0x7fdef00057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf037fe640 1 -- 172.21.15.53:0/3953589578 reap_dead start 2023-10-21T17:39:19.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf03fff640 1 -- 172.21.15.53:0/3953589578 shutdown_connections 2023-10-21T17:39:19.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.818+0000 7fdf03fff640 1 -- 172.21.15.53:0/3953589578 >> 172.21.15.53:0/3953589578 conn(0x7fdf040cb0c0 msgr2=0x7fdf040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:19.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.819+0000 7fdf03fff640 1 -- 172.21.15.53:0/3953589578 shutdown_connections 2023-10-21T17:39:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.819+0000 7fdf03fff640 1 -- 172.21.15.53:0/3953589578 wait complete. 2023-10-21T17:39:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.820+0000 7fdf03fff640 1 Processor -- start 2023-10-21T17:39:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.821+0000 7fdf03fff640 1 -- start start 2023-10-21T17:39:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.821+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef00039a0 0x7fdf0414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.821+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf041578d0 0x7fdf04157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf03fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040b2640 con 0x7fdf040ccbd0 2023-10-21T17:39:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf03fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c5710 con 0x7fdf041578d0 2023-10-21T17:39:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf03fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04158270 con 0x7fdef00039a0 2023-10-21T17:39:19.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39068/0 (socket says 172.21.15.53:39068) 2023-10-21T17:39:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 -- 172.21.15.53:0/3484030654 learned_addr learned my addr 172.21.15.53:0/3484030654 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 -- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef00039a0 msgr2=0x7fdf0414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef00039a0 0x7fdf0414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 -- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf041578d0 msgr2=0x7fdf04157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf041578d0 0x7fdf04157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.822+0000 7fdf037fe640 1 -- 172.21.15.53:0/3484030654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040ce0d0 con 0x7fdf040ccbd0 2023-10-21T17:39:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdf037fe640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fdf040b1700 tx=0x7fdefc011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc011ee0 con 0x7fdf040ccbd0 2023-10-21T17:39:19.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc004240 con 0x7fdf040ccbd0 2023-10-21T17:39:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040c59b0 con 0x7fdf040ccbd0 2023-10-21T17:39:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00f740 con 0x7fdf040ccbd0 2023-10-21T17:39:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.823+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040cd830 con 0x7fdf040ccbd0 2023-10-21T17:39:19.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.824+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0404e160 con 0x7fdf040ccbd0 2023-10-21T17:39:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.825+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fdefc00f8a0 con 0x7fdf040ccbd0 2023-10-21T17:39:19.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.826+0000 7fdefb7fe640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 0x7fdee406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:19.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.826+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fdefc089e40 con 0x7fdf040ccbd0 2023-10-21T17:39:19.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.827+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 0x7fdee406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:19.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.828+0000 7fdf02ffd640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 0x7fdee406d5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdef4034000 tx=0x7fdef4030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:19.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:19.831+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdefc054640 con 0x7fdf040ccbd0 2023-10-21T17:39:20.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.031+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf040ada50 con 0x7fdf040ccbd0 2023-10-21T17:39:20.036 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.035+0000 7fdefb7fe640 1 -- 172.21.15.53:0/3484030654 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18237 (secure 0 0 0) 0x7fdefc057c60 con 0x7fdf040ccbd0 2023-10-21T17:39:20.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:20.037 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":242,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:19.571410+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6836/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:20.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 msgr2=0x7fdee406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 0x7fdee406d5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdef4034000 tx=0x7fdef4030000 comp rx=0 tx=0).stop 2023-10-21T17:39:20.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 msgr2=0x7fdf040cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fdf040b1700 tx=0x7fdefc011cf0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 shutdown_connections 2023-10-21T17:39:20.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdee406b0e0 0x7fdee406d5a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf040ccbd0 0x7fdf040cd050 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf041578d0 0x7fdf04157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 --2- 172.21.15.53:0/3484030654 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdef00039a0 0x7fdf0414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 >> 172.21.15.53:0/3484030654 conn(0x7fdf040cb0c0 msgr2=0x7fdf0406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:20.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 shutdown_connections 2023-10-21T17:39:20.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.038+0000 7fdf03fff640 1 -- 172.21.15.53:0/3484030654 wait complete. 2023-10-21T17:39:20.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:38:45.333243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:38:46.027612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:20.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:20.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.278+0000 7f4ca7733640 1 Processor -- start 2023-10-21T17:39:20.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.279+0000 7f4ca7733640 1 -- start start 2023-10-21T17:39:20.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.280+0000 7f4ca7733640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca015aad0 0x7f4ca00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.280+0000 7f4ca7733640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4ca006d410 0x7f4ca00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.280+0000 7f4ca7733640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca015bf30 con 0x7f4ca00c82b0 2023-10-21T17:39:20.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.280+0000 7f4ca7733640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca015b320 con 0x7f4ca006d410 2023-10-21T17:39:20.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.280+0000 7f4ca7733640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca015b600 con 0x7f4ca015aad0 2023-10-21T17:39:20.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca4cce640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4ca00c82b0 0x7f4ca006cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40952/0 (socket says 172.21.15.53:40952) 2023-10-21T17:39:20.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca4cce640 1 -- 172.21.15.53:0/2665939502 learned_addr learned my addr 172.21.15.53:0/2665939502 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:20.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/2665939502 >> v2:172.21.15.53:3300/0 conn(0x7f4ca006d410 0x7f4ca00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca54cf640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca015aad0 0x7f4ca00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4c97fff640 1 -- 172.21.15.53:0/2665939502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526583266 0 0) 0x7f4ca015bf30 con 0x7f4ca00c82b0 2023-10-21T17:39:20.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4c97fff640 1 -- 172.21.15.53:0/2665939502 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c8c003520 con 0x7f4ca00c82b0 2023-10-21T17:39:20.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/2665939502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca015aad0 msgr2=0x7f4ca00b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:20.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca015aad0 0x7f4ca00b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/2665939502 >> v1:172.21.15.53:6789/0 conn(0x7f4ca00c82b0 legacy=0x7f4ca006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.281+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/2665939502 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca015b8e0 con 0x7f4ca006d410 2023-10-21T17:39:20.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.283+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca006d410 0x7f4ca00c8710 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009d90 tx=0x7f4c9802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf397467ad49544 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.283+0000 7f4c97fff640 1 -- 172.21.15.53:0/2665939502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9802cc60 con 0x7f4ca006d410 2023-10-21T17:39:20.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.283+0000 7f4c97fff640 1 -- 172.21.15.53:0/2665939502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9802cdc0 con 0x7f4ca006d410 2023-10-21T17:39:20.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.283+0000 7f4c97fff640 1 -- 172.21.15.53:0/2665939502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98034660 con 0x7f4ca006d410 2023-10-21T17:39:20.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 -- 172.21.15.53:0/2665939502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca006d410 msgr2=0x7f4ca00c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca006d410 0x7f4ca00c8710 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009d90 tx=0x7f4c9802beb0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 -- 172.21.15.53:0/2665939502 shutdown_connections 2023-10-21T17:39:20.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca006d410 0x7f4ca00c8710 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 --2- 172.21.15.53:0/2665939502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca015aad0 0x7f4ca00b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 -- 172.21.15.53:0/2665939502 >> 172.21.15.53:0/2665939502 conn(0x7f4ca0069d50 msgr2=0x7f4ca006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:20.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 -- 172.21.15.53:0/2665939502 shutdown_connections 2023-10-21T17:39:20.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.284+0000 7f4ca7733640 1 -- 172.21.15.53:0/2665939502 wait complete. 2023-10-21T17:39:20.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.286+0000 7f4ca7733640 1 Processor -- start 2023-10-21T17:39:20.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.286+0000 7f4ca7733640 1 -- start start 2023-10-21T17:39:20.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca006d410 0x7f4ca014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca00c82b0 0x7f4ca00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca0151600 con 0x7f4ca015aad0 2023-10-21T17:39:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01469b0 con 0x7f4ca006d410 2023-10-21T17:39:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca7733640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca014b450 con 0x7f4ca00c82b0 2023-10-21T17:39:20.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39076/0 (socket says 172.21.15.53:39076) 2023-10-21T17:39:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/3711011894 learned_addr learned my addr 172.21.15.53:0/3711011894 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca4cce640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca00c82b0 0x7f4ca00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca54cf640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca006d410 0x7f4ca014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca00c82b0 msgr2=0x7f4ca00b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca00c82b0 0x7f4ca00b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca006d410 msgr2=0x7f4ca014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca006d410 0x7f4ca014d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.287+0000 7f4ca5cd0640 1 -- 172.21.15.53:0/3711011894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009a70 con 0x7f4ca015aad0 2023-10-21T17:39:20.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4ca5cd0640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009d90 tx=0x7f4c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98040650 con 0x7f4ca015aad0 2023-10-21T17:39:20.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c980407b0 con 0x7f4ca015aad0 2023-10-21T17:39:20.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98040b60 con 0x7f4ca015aad0 2023-10-21T17:39:20.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca014b730 con 0x7f4ca015aad0 2023-10-21T17:39:20.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.288+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca01517e0 con 0x7f4ca015aad0 2023-10-21T17:39:20.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.290+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca014bbf0 con 0x7f4ca015aad0 2023-10-21T17:39:20.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.290+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4c98040cc0 con 0x7f4ca015aad0 2023-10-21T17:39:20.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.292+0000 7f4c95ffb640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 0x7f4c8406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.292+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f4c980acc40 con 0x7f4ca015aad0 2023-10-21T17:39:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.292+0000 7f4ca54cf640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 0x7f4c8406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.294+0000 7f4ca54cf640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 0x7f4c8406d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00cdcb0 tx=0x7f4c9c008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.296+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c98077470 con 0x7f4ca015aad0 2023-10-21T17:39:20.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:39:20.395 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:39:20.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.497+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ca004e160 con 0x7f4ca015aad0 2023-10-21T17:39:20.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.498+0000 7f4c95ffb640 1 -- 172.21.15.53:0/3711011894 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v242) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c98009450 con 0x7f4ca015aad0 2023-10-21T17:39:20.499 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 msgr2=0x7f4c8406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 0x7f4c8406d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00cdcb0 tx=0x7f4c9c008040 comp rx=0 tx=0).stop 2023-10-21T17:39:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 msgr2=0x7f4ca00b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009d90 tx=0x7f4c98004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 shutdown_connections 2023-10-21T17:39:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4c8406b0b0 0x7f4c8406d570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ca015aad0 0x7f4ca00b1890 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca00c82b0 0x7f4ca00b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 --2- 172.21.15.53:0/3711011894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca006d410 0x7f4ca014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 >> 172.21.15.53:0/3711011894 conn(0x7f4ca0069d50 msgr2=0x7f4ca015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 shutdown_connections 2023-10-21T17:39:20.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.501+0000 7f4ca7733640 1 -- 172.21.15.53:0/3711011894 wait complete. 2023-10-21T17:39:20.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:39:20.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.793+0000 7f7252e6d640 1 Processor -- start 2023-10-21T17:39:20.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.794+0000 7f7252e6d640 1 -- start start 2023-10-21T17:39:20.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.795+0000 7f7252e6d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f724c0b0680 0x7f724c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.795+0000 7f7252e6d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f724c0ca680 0x7f724c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.795+0000 7f7252e6d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c0cab60 con 0x7f724c15a820 2023-10-21T17:39:20.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.795+0000 7f7252e6d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c0cb960 con 0x7f724c0ca680 2023-10-21T17:39:20.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.795+0000 7f7252e6d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c0cbdf0 con 0x7f724c0b0680 2023-10-21T17:39:20.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f724c0ca680 0x7f724c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f724c0ca680 0x7f724c15ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39088/0 (socket says 172.21.15.53:39088) 2023-10-21T17:39:20.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 -- 172.21.15.53:0/2061216185 learned_addr learned my addr 172.21.15.53:0/2061216185 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:20.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725166a640 1 --2- 172.21.15.53:0/2061216185 >> v2:172.21.15.124:3300/0 conn(0x7f724c0b0680 0x7f724c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 -- 172.21.15.53:0/2061216185 >> v2:172.21.15.124:3300/0 conn(0x7f724c0b0680 msgr2=0x7f724c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:20.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 --2- 172.21.15.53:0/2061216185 >> v2:172.21.15.124:3300/0 conn(0x7f724c0b0680 0x7f724c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 -- 172.21.15.53:0/2061216185 >> v1:172.21.15.124:6789/0 conn(0x7f724c15a820 legacy=0x7f724c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:20.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.796+0000 7f725266c640 1 -- 172.21.15.53:0/2061216185 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c0cb030 con 0x7f724c0ca680 2023-10-21T17:39:20.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f725266c640 1 --2- 172.21.15.53:0/2061216185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c15ea80 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f724c0718e0 tx=0x7f724402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52821ce7ee686e4b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f7250e69640 1 -- 172.21.15.53:0/2061216185 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724402c9f0 con 0x7f724c0ca680 2023-10-21T17:39:20.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f7250e69640 1 -- 172.21.15.53:0/2061216185 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724402cb50 con 0x7f724c0ca680 2023-10-21T17:39:20.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f7250e69640 1 -- 172.21.15.53:0/2061216185 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440343f0 con 0x7f724c0ca680 2023-10-21T17:39:20.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f7252e6d640 1 -- 172.21.15.53:0/2061216185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 msgr2=0x7f724c15ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.798+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2061216185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c15ea80 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f724c0718e0 tx=0x7f724402be50 comp rx=0 tx=0).stop 2023-10-21T17:39:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 -- 172.21.15.53:0/2061216185 shutdown_connections 2023-10-21T17:39:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2061216185 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c15ea80 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2061216185 >> v2:172.21.15.124:3300/0 conn(0x7f724c0b0680 0x7f724c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 -- 172.21.15.53:0/2061216185 >> 172.21.15.53:0/2061216185 conn(0x7f724c06c050 msgr2=0x7f724c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:20.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 -- 172.21.15.53:0/2061216185 shutdown_connections 2023-10-21T17:39:20.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.799+0000 7f7252e6d640 1 -- 172.21.15.53:0/2061216185 wait complete. 2023-10-21T17:39:20.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.801+0000 7f7252e6d640 1 Processor -- start 2023-10-21T17:39:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.801+0000 7f7252e6d640 1 -- start start 2023-10-21T17:39:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f724c0b0680 0x7f724c1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f724c15a820 0x7f724c0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c14cf10 con 0x7f724c0ca680 2023-10-21T17:39:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c14c290 con 0x7f724c0b0680 2023-10-21T17:39:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f7252e6d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c14c570 con 0x7f724c15a820 2023-10-21T17:39:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39098/0 (socket says 172.21.15.53:39098) 2023-10-21T17:39:20.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 -- 172.21.15.53:0/2296372071 learned_addr learned my addr 172.21.15.53:0/2296372071 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:20.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 -- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f724c15a820 msgr2=0x7f724c0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f724c15a820 0x7f724c0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 -- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f724c0b0680 msgr2=0x7f724c1f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:20.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f724c0b0680 0x7f724c1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:20.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.802+0000 7f725166a640 1 -- 172.21.15.53:0/2296372071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009b30 con 0x7f724c0ca680 2023-10-21T17:39:20.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f725166a640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f723c00d800 tx=0x7f723c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00b6c0 con 0x7f724c0ca680 2023-10-21T17:39:20.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c00b820 con 0x7f724c0ca680 2023-10-21T17:39:20.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c14c850 con 0x7f724c0ca680 2023-10-21T17:39:20.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00bbb0 con 0x7f724c0ca680 2023-10-21T17:39:20.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.803+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c1f1da0 con 0x7f724c0ca680 2023-10-21T17:39:20.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.804+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c074150 con 0x7f724c0ca680 2023-10-21T17:39:20.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.806+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f723c00bd10 con 0x7f724c0ca680 2023-10-21T17:39:20.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.807+0000 7f72427fc640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 0x7f723006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:20.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.807+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f723c08a500 con 0x7f724c0ca680 2023-10-21T17:39:20.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.807+0000 7f7251e6b640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 0x7f723006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:20.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.808+0000 7f7251e6b640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 0x7f723006d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7248006fd0 tx=0x7f7248006bc0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:20.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:20.811+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f723c054cf0 con 0x7f724c0ca680 2023-10-21T17:39:21.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.004+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f724c1f1f60 con 0x7f724c0ca680 2023-10-21T17:39:21.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.005+0000 7f72427fc640 1 -- 172.21.15.53:0/2296372071 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v242) v1 ==== 107+0+11 (secure 0 0 0) 0x7f723c058310 con 0x7f724c0ca680 2023-10-21T17:39:21.006 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 msgr2=0x7f723006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 0x7f723006d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7248006fd0 tx=0x7f7248006bc0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 msgr2=0x7f724c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f723c00d800 tx=0x7f723c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 shutdown_connections 2023-10-21T17:39:21.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f723006b0b0 0x7f723006d570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f724c15a820 0x7f724c0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f724c0ca680 0x7f724c0cd050 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 --2- 172.21.15.53:0/2296372071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f724c0b0680 0x7f724c1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 >> 172.21.15.53:0/2296372071 conn(0x7f724c06c050 msgr2=0x7f724c0abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:21.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 shutdown_connections 2023-10-21T17:39:21.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.008+0000 7f7252e6d640 1 -- 172.21.15.53:0/2296372071 wait complete. 2023-10-21T17:39:21.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:39:21.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.260+0000 7f6cccabb640 1 Processor -- start 2023-10-21T17:39:21.262 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.261+0000 7f6cccabb640 1 -- start start 2023-10-21T17:39:21.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.262+0000 7f6cccabb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6cc80d00e0 0x7f6cc80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.262+0000 7f6cccabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc80d0b90 0x7f6cc80b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.262+0000 7f6cccabb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80d2870 con 0x7f6cc815ccb0 2023-10-21T17:39:21.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.262+0000 7f6cccabb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc80d1150 con 0x7f6cc80d00e0 2023-10-21T17:39:21.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.262+0000 7f6cccabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80d1430 con 0x7f6cc80d0b90 2023-10-21T17:39:21.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc77fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6cc815ccb0 0x7f6cc80cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40962/0 (socket says 172.21.15.53:40962) 2023-10-21T17:39:21.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc77fe640 1 -- 172.21.15.53:0/1228866692 learned_addr learned my addr 172.21.15.53:0/1228866692 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:21.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc6ffd640 1 --2- 172.21.15.53:0/1228866692 >> v2:172.21.15.53:3300/0 conn(0x7f6cc80d00e0 0x7f6cc80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc7fff640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc80d0b90 0x7f6cc80b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc67fc640 1 -- 172.21.15.53:0/1228866692 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015047135 0 0) 0x7f6cc80d2870 con 0x7f6cc815ccb0 2023-10-21T17:39:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc67fc640 1 -- 172.21.15.53:0/1228866692 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb0003520 con 0x7f6cc815ccb0 2023-10-21T17:39:21.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc6ffd640 1 -- 172.21.15.53:0/1228866692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc80d0b90 msgr2=0x7f6cc80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc6ffd640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc80d0b90 0x7f6cc80b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc6ffd640 1 -- 172.21.15.53:0/1228866692 >> v1:172.21.15.53:6789/0 conn(0x7f6cc815ccb0 legacy=0x7f6cc80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.263+0000 7f6cc6ffd640 1 -- 172.21.15.53:0/1228866692 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80d1710 con 0x7f6cc80d00e0 2023-10-21T17:39:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.265+0000 7f6cc6ffd640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc80d1fb0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0097a0 tx=0x7f6cbc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72922a5ce9ef93a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.265+0000 7f6cc67fc640 1 -- 172.21.15.53:0/1228866692 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc02d040 con 0x7f6cc80d00e0 2023-10-21T17:39:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.265+0000 7f6cc67fc640 1 -- 172.21.15.53:0/1228866692 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc031810 con 0x7f6cc80d00e0 2023-10-21T17:39:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.265+0000 7f6cc67fc640 1 -- 172.21.15.53:0/1228866692 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc031ba0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 -- 172.21.15.53:0/1228866692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 msgr2=0x7f6cc80d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc80d1fb0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0097a0 tx=0x7f6cbc02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 -- 172.21.15.53:0/1228866692 shutdown_connections 2023-10-21T17:39:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc80d0b90 0x7f6cc80b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 --2- 172.21.15.53:0/1228866692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc80d1fb0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 -- 172.21.15.53:0/1228866692 >> 172.21.15.53:0/1228866692 conn(0x7f6cc8071660 msgr2=0x7f6cc8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 -- 172.21.15.53:0/1228866692 shutdown_connections 2023-10-21T17:39:21.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.266+0000 7f6cccabb640 1 -- 172.21.15.53:0/1228866692 wait complete. 2023-10-21T17:39:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.268+0000 7f6cccabb640 1 Processor -- start 2023-10-21T17:39:21.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.268+0000 7f6cccabb640 1 -- start start 2023-10-21T17:39:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cc80d0b90 0x7f6cc815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 0x7f6cc815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc815abd0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81f1a10 con 0x7f6cc80d0b90 2023-10-21T17:39:21.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cccabb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80c6820 con 0x7f6cc815ccb0 2023-10-21T17:39:21.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cc77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cc77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39102/0 (socket says 172.21.15.53:39102) 2023-10-21T17:39:21.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cc77fe640 1 -- 172.21.15.53:0/498632862 learned_addr learned my addr 172.21.15.53:0/498632862 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cc7fff640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 0x7f6cc815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.269+0000 7f6cc77fe640 1 -- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 msgr2=0x7f6cc815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc77fe640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 0x7f6cc815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc77fe640 1 -- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cc80d0b90 msgr2=0x7f6cc815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc77fe640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cc80d0b90 0x7f6cc815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc77fe640 1 -- 172.21.15.53:0/498632862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc009400 con 0x7f6cc80d00e0 2023-10-21T17:39:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc7fff640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 0x7f6cc815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cc77fe640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800cbc0 tx=0x7f6cb800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800ad40 con 0x7f6cc80d00e0 2023-10-21T17:39:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8012400 con 0x7f6cc80d00e0 2023-10-21T17:39:21.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800aea0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc815f3a0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.270+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc815f0c0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.272+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8074150 con 0x7f6cc80d00e0 2023-10-21T17:39:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.273+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6cb8007070 con 0x7f6cc80d00e0 2023-10-21T17:39:21.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.274+0000 7f6caffff640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 0x7f6cb406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.274+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f6cb8089f40 con 0x7f6cc80d00e0 2023-10-21T17:39:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.274+0000 7f6cc6ffd640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 0x7f6cb406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.275+0000 7f6cc6ffd640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 0x7f6cb406d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc036000 tx=0x7f6cbc030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.278+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6cb80547d0 con 0x7f6cc80d00e0 2023-10-21T17:39:21.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.472+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cc806a110 con 0x7f6cc80d00e0 2023-10-21T17:39:21.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.472+0000 7f6caffff640 1 -- 172.21.15.53:0/498632862 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v242) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6cb8017020 con 0x7f6cc80d00e0 2023-10-21T17:39:21.474 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 msgr2=0x7f6cb406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 0x7f6cb406d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc036000 tx=0x7f6cbc030000 comp rx=0 tx=0).stop 2023-10-21T17:39:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 msgr2=0x7f6cc815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800cbc0 tx=0x7f6cb800a430 comp rx=0 tx=0).stop 2023-10-21T17:39:21.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 shutdown_connections 2023-10-21T17:39:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f6cb406b0b0 0x7f6cb406d570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cc815ccb0 0x7f6cc815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cc80d0b90 0x7f6cc815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 --2- 172.21.15.53:0/498632862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6cc80d00e0 0x7f6cc815a690 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.475+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 >> 172.21.15.53:0/498632862 conn(0x7f6cc8071660 msgr2=0x7f6cc80cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:21.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.476+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 shutdown_connections 2023-10-21T17:39:21.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.476+0000 7f6cccabb640 1 -- 172.21.15.53:0/498632862 wait complete. 2023-10-21T17:39:21.498 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:21.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.720+0000 7faeca7a9640 1 Processor -- start 2023-10-21T17:39:21.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.721+0000 7faeca7a9640 1 -- start start 2023-10-21T17:39:21.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 0x7faec40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 0x7faec40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faec40cae70 0x7faec40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec415bf30 con 0x7faec40cae70 2023-10-21T17:39:21.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec40cb840 con 0x7faec40c90b0 2023-10-21T17:39:21.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.722+0000 7faeca7a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec415b2b0 con 0x7faec415ab40 2023-10-21T17:39:21.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.723+0000 7faec8d46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faec40cae70 0x7faec40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.723+0000 7faec8d46640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faec40cae70 0x7faec40cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39116/0 (socket says 172.21.15.53:39116) 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.723+0000 7faec8d46640 1 -- 172.21.15.53:0/2688540250 learned_addr learned my addr 172.21.15.53:0/2688540250 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.723+0000 7faec3fff640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 0x7faec40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.723+0000 7faebbfff640 1 --2- 172.21.15.53:0/2688540250 >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 0x7faec40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec8d46640 1 -- 172.21.15.53:0/2688540250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 msgr2=0x7faec40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec8d46640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 0x7faec40ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec8d46640 1 -- 172.21.15.53:0/2688540250 >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 msgr2=0x7faec40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:21.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec8d46640 1 --2- 172.21.15.53:0/2688540250 >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 0x7faec40adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec8d46640 1 -- 172.21.15.53:0/2688540250 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec415b590 con 0x7faec40cae70 2023-10-21T17:39:21.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faec3fff640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 0x7faec40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:21.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.724+0000 7faebbfff640 1 --2- 172.21.15.53:0/2688540250 >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 0x7faec40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:21.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.725+0000 7faec8d46640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec40cae70 0x7faec40cb300 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7faec406e470 tx=0x7faebc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476d8a82338bdf71 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.725+0000 7faec37fe640 1 -- 172.21.15.53:0/2688540250 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc02ca80 con 0x7faec40cae70 2023-10-21T17:39:21.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.725+0000 7faec37fe640 1 -- 172.21.15.53:0/2688540250 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc02cbe0 con 0x7faec40cae70 2023-10-21T17:39:21.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.725+0000 7faec37fe640 1 -- 172.21.15.53:0/2688540250 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0343f0 con 0x7faec40cae70 2023-10-21T17:39:21.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 -- 172.21.15.53:0/2688540250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec40cae70 msgr2=0x7faec40cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec40cae70 0x7faec40cb300 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7faec406e470 tx=0x7faebc02bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 -- 172.21.15.53:0/2688540250 shutdown_connections 2023-10-21T17:39:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec40cae70 0x7faec40cb300 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 --2- 172.21.15.53:0/2688540250 >> v2:172.21.15.124:3300/0 conn(0x7faec40c90b0 0x7faec40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 --2- 172.21.15.53:0/2688540250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec415ab40 0x7faec40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 -- 172.21.15.53:0/2688540250 >> 172.21.15.53:0/2688540250 conn(0x7faec406a380 msgr2=0x7faec40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 -- 172.21.15.53:0/2688540250 shutdown_connections 2023-10-21T17:39:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.726+0000 7faeca7a9640 1 -- 172.21.15.53:0/2688540250 wait complete. 2023-10-21T17:39:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.728+0000 7faeca7a9640 1 Processor -- start 2023-10-21T17:39:21.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.728+0000 7faeca7a9640 1 -- start start 2023-10-21T17:39:21.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec40c90b0 0x7faec40b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 0x7faec414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40b3be0 con 0x7faec415ab40 2023-10-21T17:39:21.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c5620 con 0x7faec40cae70 2023-10-21T17:39:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faeca7a9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec40b1b40 con 0x7faec40c90b0 2023-10-21T17:39:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faec8d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faec8d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39128/0 (socket says 172.21.15.53:39128) 2023-10-21T17:39:21.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.729+0000 7faec8d46640 1 -- 172.21.15.53:0/1003560697 learned_addr learned my addr 172.21.15.53:0/1003560697 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faebbfff640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 0x7faec414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 -- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec40c90b0 msgr2=0x7faec40b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec3fff640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec40c90b0 0x7faec40b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec40c90b0 0x7faec40b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 -- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 msgr2=0x7faec414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 0x7faec414b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 -- 172.21.15.53:0/1003560697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc009b30 con 0x7faec415ab40 2023-10-21T17:39:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faebbfff640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 0x7faec414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec8d46640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7faebc036040 tx=0x7faebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc009510 con 0x7faec415ab40 2023-10-21T17:39:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc03e450 con 0x7faec415ab40 2023-10-21T17:39:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc03e7e0 con 0x7faec415ab40 2023-10-21T17:39:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.730+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec40b1910 con 0x7faec415ab40 2023-10-21T17:39:21.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.731+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec414b8b0 con 0x7faec415ab40 2023-10-21T17:39:21.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.732+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec40b3d20 con 0x7faec415ab40 2023-10-21T17:39:21.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.733+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7faebc02cd50 con 0x7faec415ab40 2023-10-21T17:39:21.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.734+0000 7faec17fa640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 0x7faea406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:21.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.734+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7faebc0acdf0 con 0x7faec415ab40 2023-10-21T17:39:21.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.734+0000 7faec3fff640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 0x7faea406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:21.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.736+0000 7faec3fff640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 0x7faea406d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d8b0 tx=0x7faeb4008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:21.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.739+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faec40b3d20 con 0x7faec415ab40 2023-10-21T17:39:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.972+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec40b3d20 con 0x7faec415ab40 2023-10-21T17:39:21.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.974+0000 7faec17fa640 1 -- 172.21.15.53:0/1003560697 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4933 (secure 0 0 0) 0x7faec40b3d20 con 0x7faec415ab40 2023-10-21T17:39:21.975 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:21.976 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1533795007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":1533795007},{"type":"v1","addr":"172.21.15.124:6837","nonce":1533795007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:21.976 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 120 2023-10-21T17:39:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.976+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 msgr2=0x7faea406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.976+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 0x7faea406d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d8b0 tx=0x7faeb4008040 comp rx=0 tx=0).stop 2023-10-21T17:39:21.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.976+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 msgr2=0x7faec4158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.976+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7faebc036040 tx=0x7faebc004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 shutdown_connections 2023-10-21T17:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faea406b0b0 0x7faea406d570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faec415ab40 0x7faec4158fc0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faec40cae70 0x7faec414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 --2- 172.21.15.53:0/1003560697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec40c90b0 0x7faec40b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 >> 172.21.15.53:0/1003560697 conn(0x7faec406a380 msgr2=0x7faec40c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 shutdown_connections 2023-10-21T17:39:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:21.977+0000 7faeca7a9640 1 -- 172.21.15.53:0/1003560697 wait complete. 2023-10-21T17:39:21.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.225+0000 7efce4cbb640 1 Processor -- start 2023-10-21T17:39:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.226+0000 7efce4cbb640 1 -- start start 2023-10-21T17:39:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efce4cbb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efce015cca0 0x7efce01614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efce4cbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce00cab40 0x7efce00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efce4cbb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00c9020 con 0x7efce015d080 2023-10-21T17:39:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efce4cbb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00ccc20 con 0x7efce015cca0 2023-10-21T17:39:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efce4cbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c92e0 con 0x7efce00cab40 2023-10-21T17:39:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.227+0000 7efcdf7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efce015cca0 0x7efce01614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efce015cca0 0x7efce01614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:36606/0 (socket says 172.21.15.53:36606) 2023-10-21T17:39:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 -- 172.21.15.53:0/1843100392 learned_addr learned my addr 172.21.15.53:0/1843100392 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcd7fff640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce00cab40 0x7efce00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 -- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce00cab40 msgr2=0x7efce00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce00cab40 0x7efce00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 -- 172.21.15.53:0/1843100392 >> v1:172.21.15.124:6789/0 conn(0x7efce015d080 legacy=0x7efce00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.228+0000 7efcdf7fe640 1 -- 172.21.15.53:0/1843100392 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00cc7b0 con 0x7efce015cca0 2023-10-21T17:39:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.230+0000 7efcdf7fe640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015cca0 0x7efce01614b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427a2e793d1aa4c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.230+0000 7efcdeffd640 1 -- 172.21.15.53:0/1843100392 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00317c0 con 0x7efce015cca0 2023-10-21T17:39:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.230+0000 7efcdeffd640 1 -- 172.21.15.53:0/1843100392 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0031920 con 0x7efce015cca0 2023-10-21T17:39:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.230+0000 7efcdeffd640 1 -- 172.21.15.53:0/1843100392 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0031cb0 con 0x7efce015cca0 2023-10-21T17:39:22.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 -- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015cca0 msgr2=0x7efce01614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015cca0 0x7efce01614b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002b860 comp rx=0 tx=0).stop 2023-10-21T17:39:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 -- 172.21.15.53:0/1843100392 shutdown_connections 2023-10-21T17:39:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce00cab40 0x7efce00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 --2- 172.21.15.53:0/1843100392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015cca0 0x7efce01614b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 -- 172.21.15.53:0/1843100392 >> 172.21.15.53:0/1843100392 conn(0x7efce00cb0c0 msgr2=0x7efce00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:22.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 -- 172.21.15.53:0/1843100392 shutdown_connections 2023-10-21T17:39:22.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.231+0000 7efce4cbb640 1 -- 172.21.15.53:0/1843100392 wait complete. 2023-10-21T17:39:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.233+0000 7efce4cbb640 1 Processor -- start 2023-10-21T17:39:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.233+0000 7efce4cbb640 1 -- start start 2023-10-21T17:39:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce015cca0 0x7efce0157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015d080 0x7efce00ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01e7740 con 0x7efce00cab40 2023-10-21T17:39:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01522e0 con 0x7efce015d080 2023-10-21T17:39:22.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efce4cbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce0156fe0 con 0x7efce015cca0 2023-10-21T17:39:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efcdf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39140/0 (socket says 172.21.15.53:39140) 2023-10-21T17:39:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.234+0000 7efcdf7fe640 1 -- 172.21.15.53:0/2627002789 learned_addr learned my addr 172.21.15.53:0/2627002789 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdffff640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015d080 0x7efce00ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcd7fff640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce015cca0 0x7efce0157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 -- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce015cca0 msgr2=0x7efce0157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce015cca0 0x7efce0157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 -- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015d080 msgr2=0x7efce00ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015d080 0x7efce00ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 -- 172.21.15.53:0/2627002789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00094d0 con 0x7efce00cab40 2023-10-21T17:39:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.235+0000 7efcdf7fe640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c00 tx=0x7efcd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.236+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032860 con 0x7efce00cab40 2023-10-21T17:39:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.236+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd00329c0 con 0x7efce00cab40 2023-10-21T17:39:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.236+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce01566b0 con 0x7efce00cab40 2023-10-21T17:39:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.236+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032d70 con 0x7efce00cab40 2023-10-21T17:39:22.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.236+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce0156c70 con 0x7efce00cab40 2023-10-21T17:39:22.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.237+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce004e160 con 0x7efce00cab40 2023-10-21T17:39:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.238+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7efcd004e450 con 0x7efce00cab40 2023-10-21T17:39:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.239+0000 7efcdcff9640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 0x7efcc006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.239+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7efcd00ac0e0 con 0x7efce00cab40 2023-10-21T17:39:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.239+0000 7efcd7fff640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 0x7efcc006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.241+0000 7efcd7fff640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 0x7efcc006d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efce00b4540 tx=0x7efcc8008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.243+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcd0076960 con 0x7efce00cab40 2023-10-21T17:39:22.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.435+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efce00c8ae0 con 0x7efce00cab40 2023-10-21T17:39:22.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.439+0000 7efcdcff9640 1 -- 172.21.15.53:0/2627002789 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18237 (secure 0 0 0) 0x7efcd0079f80 con 0x7efce00cab40 2023-10-21T17:39:22.441 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:22.441 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":242,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:19.571410+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6836/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:22.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.441+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 msgr2=0x7efcc006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.441+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 0x7efcc006d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efce00b4540 tx=0x7efcc8008040 comp rx=0 tx=0).stop 2023-10-21T17:39:22.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.441+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 msgr2=0x7efce0151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.441+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7efcd0000c00 tx=0x7efcd00046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 shutdown_connections 2023-10-21T17:39:22.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efcc006b0b0 0x7efcc006d570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efce015d080 0x7efce00ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efce015cca0 0x7efce0157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 --2- 172.21.15.53:0/2627002789 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efce00cab40 0x7efce0151da0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 >> 172.21.15.53:0/2627002789 conn(0x7efce00cb0c0 msgr2=0x7efce00ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 shutdown_connections 2023-10-21T17:39:22.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.442+0000 7efce4cbb640 1 -- 172.21.15.53:0/2627002789 wait complete. 2023-10-21T17:39:22.463 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-21T17:39:22.464 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:22.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.695+0000 7f9a50d9e640 1 Processor -- start 2023-10-21T17:39:22.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.695+0000 7f9a50d9e640 1 -- start start 2023-10-21T17:39:22.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a50d9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a4c1599d0 0x7f9a4c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a50d9e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a4c0cc210 0x7f9a4c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a50d9e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0cacf0 con 0x7f9a4c159db0 2023-10-21T17:39:22.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a50d9e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b10f0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a50d9e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ca3c0 con 0x7f9a4c0cc210 2023-10-21T17:39:22.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a4b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a4c1599d0 0x7f9a4c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a4b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a4c1599d0 0x7f9a4c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39142/0 (socket says 172.21.15.53:39142) 2023-10-21T17:39:22.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.697+0000 7f9a4b7fe640 1 -- 172.21.15.53:0/2835501399 learned_addr learned my addr 172.21.15.53:0/2835501399 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:22.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.698+0000 7f9a4affd640 1 --2- 172.21.15.53:0/2835501399 >> v2:172.21.15.124:3300/0 conn(0x7f9a4c0cc210 0x7f9a4c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.698+0000 7f9a4b7fe640 1 -- 172.21.15.53:0/2835501399 >> v2:172.21.15.124:3300/0 conn(0x7f9a4c0cc210 msgr2=0x7f9a4c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:22.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.698+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/2835501399 >> v2:172.21.15.124:3300/0 conn(0x7f9a4c0cc210 0x7f9a4c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.698+0000 7f9a4b7fe640 1 -- 172.21.15.53:0/2835501399 >> v1:172.21.15.124:6789/0 conn(0x7f9a4c159db0 legacy=0x7f9a4c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:22.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.698+0000 7f9a4b7fe640 1 -- 172.21.15.53:0/2835501399 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0ca6a0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.699+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/2835501399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ad730 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009870 tx=0x7f9a3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d201e8d2aeb46c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.700+0000 7f9a4a7fc640 1 -- 172.21.15.53:0/2835501399 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c031870 con 0x7f9a4c1599d0 2023-10-21T17:39:22.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.700+0000 7f9a4a7fc640 1 -- 172.21.15.53:0/2835501399 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0319d0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.700+0000 7f9a4a7fc640 1 -- 172.21.15.53:0/2835501399 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c031d60 con 0x7f9a4c1599d0 2023-10-21T17:39:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.700+0000 7f9a50d9e640 1 -- 172.21.15.53:0/2835501399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 msgr2=0x7f9a4c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.700+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/2835501399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ad730 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009870 tx=0x7f9a3c02b910 comp rx=0 tx=0).stop 2023-10-21T17:39:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 -- 172.21.15.53:0/2835501399 shutdown_connections 2023-10-21T17:39:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/2835501399 >> v2:172.21.15.124:3300/0 conn(0x7f9a4c0cc210 0x7f9a4c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/2835501399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ad730 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 -- 172.21.15.53:0/2835501399 >> 172.21.15.53:0/2835501399 conn(0x7f9a4c0cb0c0 msgr2=0x7f9a4c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 -- 172.21.15.53:0/2835501399 shutdown_connections 2023-10-21T17:39:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.701+0000 7f9a50d9e640 1 -- 172.21.15.53:0/2835501399 wait complete. 2023-10-21T17:39:22.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.702+0000 7f9a50d9e640 1 Processor -- start 2023-10-21T17:39:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.703+0000 7f9a50d9e640 1 -- start start 2023-10-21T17:39:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.703+0000 7f9a50d9e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 0x7f9a4c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.703+0000 7f9a50d9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.703+0000 7f9a50d9e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 0x7f9a4c0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a50d9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ae240 con 0x7f9a4c1599d0 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a50d9e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ae520 con 0x7f9a4c0cc210 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a50d9e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ae800 con 0x7f9a4c159db0 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39156/0 (socket says 172.21.15.53:39156) 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 -- 172.21.15.53:0/3574812576 learned_addr learned my addr 172.21.15.53:0/3574812576 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:22.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 0x7f9a4c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 -- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 msgr2=0x7f9a4c0cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4bfff640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 0x7f9a4c0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 0x7f9a4c0cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 -- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 msgr2=0x7f9a4c150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 0x7f9a4c150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4affd640 1 -- 172.21.15.53:0/3574812576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0094d0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 0x7f9a4c150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.704+0000 7f9a4bfff640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 0x7f9a4c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a4affd640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d800 tx=0x7f9a4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000b6c0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000b820 con 0x7f9a4c1599d0 2023-10-21T17:39:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000bbb0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c0aeae0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.705+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c0af000 con 0x7f9a4c1599d0 2023-10-21T17:39:22.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.706+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c04e160 con 0x7f9a4c1599d0 2023-10-21T17:39:22.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.707+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9a4000bd10 con 0x7f9a4c1599d0 2023-10-21T17:39:22.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.708+0000 7f9a33fff640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 0x7f9a3806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:22.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.708+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f9a4008a5d0 con 0x7f9a4c1599d0 2023-10-21T17:39:22.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.709+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 0x7f9a3806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:22.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.710+0000 7f9a4b7fe640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 0x7f9a3806d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c036000 tx=0x7f9a3c03c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:22.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.712+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a40054d30 con 0x7f9a4c1599d0 2023-10-21T17:39:22.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.943+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a4c0ae380 con 0x7f9a4c1599d0 2023-10-21T17:39:22.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.944+0000 7f9a33fff640 1 -- 172.21.15.53:0/3574812576 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f9a4c0ae380 con 0x7f9a4c1599d0 2023-10-21T17:39:22.946 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:22.946 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1533795007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":1533795007},{"type":"v1","addr":"172.21.15.124:6837","nonce":1533795007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:22.947 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 120 2023-10-21T17:39:22.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 msgr2=0x7f9a3806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 0x7f9a3806d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c036000 tx=0x7f9a3c03c040 comp rx=0 tx=0).stop 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 msgr2=0x7f9a4c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d800 tx=0x7f9a4000dcc0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 shutdown_connections 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9a3806b0b0 0x7f9a3806d570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a4c159db0 0x7f9a4c0cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a4c1599d0 0x7f9a4c0ce8d0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 --2- 172.21.15.53:0/3574812576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a4c0cc210 0x7f9a4c150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:22.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 >> 172.21.15.53:0/3574812576 conn(0x7f9a4c0cb0c0 msgr2=0x7f9a4c0ad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:22.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 shutdown_connections 2023-10-21T17:39:22.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:22.947+0000 7f9a50d9e640 1 -- 172.21.15.53:0/3574812576 wait complete. 2023-10-21T17:39:22.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.193+0000 7fec7a072640 1 Processor -- start 2023-10-21T17:39:23.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.193+0000 7fec7a072640 1 -- start start 2023-10-21T17:39:23.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.194+0000 7fec7a072640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec740cc210 0x7fec740b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7a072640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740cacf0 con 0x7fec74159db0 2023-10-21T17:39:23.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7a072640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740ca3c0 con 0x7fec741599d0 2023-10-21T17:39:23.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7a072640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec740b10f0 con 0x7fec740cc210 2023-10-21T17:39:23.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7886f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec740cc210 0x7fec740b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec79871640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fec74159db0 0x7fec7406c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40972/0 (socket says 172.21.15.53:40972) 2023-10-21T17:39:23.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec79871640 1 -- 172.21.15.53:0/3077237950 learned_addr learned my addr 172.21.15.53:0/3077237950 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:23.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7886f640 1 -- 172.21.15.53:0/3077237950 >> v1:172.21.15.124:6789/0 conn(0x7fec741599d0 legacy=0x7fec740ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:23.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.195+0000 7fec7886f640 1 -- 172.21.15.53:0/3077237950 >> v1:172.21.15.53:6789/0 conn(0x7fec74159db0 legacy=0x7fec7406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.196+0000 7fec7886f640 1 -- 172.21.15.53:0/3077237950 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740ca6a0 con 0x7fec740cc210 2023-10-21T17:39:23.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.196+0000 7fec6bfff640 1 -- 172.21.15.53:0/3077237950 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878223321 0 0) 0x7fec740cacf0 con 0x7fec74159db0 2023-10-21T17:39:23.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.197+0000 7fec7886f640 1 --2- 172.21.15.53:0/3077237950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec740cc210 0x7fec740b00c0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fec640097a0 tx=0x7fec6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca0cc34ef285c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.197+0000 7fec6bfff640 1 -- 172.21.15.53:0/3077237950 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64031830 con 0x7fec740cc210 2023-10-21T17:39:23.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.197+0000 7fec6bfff640 1 -- 172.21.15.53:0/3077237950 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64031990 con 0x7fec740cc210 2023-10-21T17:39:23.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.197+0000 7fec6bfff640 1 -- 172.21.15.53:0/3077237950 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64031d20 con 0x7fec740cc210 2023-10-21T17:39:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 -- 172.21.15.53:0/3077237950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec740cc210 msgr2=0x7fec740b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 --2- 172.21.15.53:0/3077237950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec740cc210 0x7fec740b00c0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fec640097a0 tx=0x7fec6402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 -- 172.21.15.53:0/3077237950 shutdown_connections 2023-10-21T17:39:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 --2- 172.21.15.53:0/3077237950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec740cc210 0x7fec740b00c0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 -- 172.21.15.53:0/3077237950 >> 172.21.15.53:0/3077237950 conn(0x7fec740cb0c0 msgr2=0x7fec740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 -- 172.21.15.53:0/3077237950 shutdown_connections 2023-10-21T17:39:23.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.198+0000 7fec7a072640 1 -- 172.21.15.53:0/3077237950 wait complete. 2023-10-21T17:39:23.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.200+0000 7fec7a072640 1 Processor -- start 2023-10-21T17:39:23.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.200+0000 7fec7a072640 1 -- start start 2023-10-21T17:39:23.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec740cc210 0x7fec74150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec741599d0 0x7fec741f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74155010 con 0x7fec74159db0 2023-10-21T17:39:23.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741509c0 con 0x7fec740cc210 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7a072640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec741f62d0 con 0x7fec741599d0 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec79871640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec79871640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39174/0 (socket says 172.21.15.53:39174) 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec79871640 1 -- 172.21.15.53:0/792153053 learned_addr learned my addr 172.21.15.53:0/792153053 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec79070640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec740cc210 0x7fec74150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.201+0000 7fec7886f640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec741599d0 0x7fec741f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 -- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec741599d0 msgr2=0x7fec741f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec741599d0 0x7fec741f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 -- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec740cc210 msgr2=0x7fec74150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec740cc210 0x7fec74150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 -- 172.21.15.53:0/792153053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64009400 con 0x7fec74159db0 2023-10-21T17:39:23.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec79871640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c450 tx=0x7fec6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0074e0 con 0x7fec74159db0 2023-10-21T17:39:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c007660 con 0x7fec74159db0 2023-10-21T17:39:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec741f65b0 con 0x7fec74159db0 2023-10-21T17:39:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c004a20 con 0x7fec74159db0 2023-10-21T17:39:23.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.202+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec74154ca0 con 0x7fec74159db0 2023-10-21T17:39:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.203+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec74155180 con 0x7fec74159db0 2023-10-21T17:39:23.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.204+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fec6c004b80 con 0x7fec74159db0 2023-10-21T17:39:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.205+0000 7fec69ffb640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 0x7fec5806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.205+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fec6c089e70 con 0x7fec74159db0 2023-10-21T17:39:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.206+0000 7fec79070640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 0x7fec5806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.207+0000 7fec79070640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 0x7fec5806d5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec70005520 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.210+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec6c0545d0 con 0x7fec74159db0 2023-10-21T17:39:23.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.442+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7404e160 con 0x7fec74159db0 2023-10-21T17:39:23.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.443+0000 7fec69ffb640 1 -- 172.21.15.53:0/792153053 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fec6c057bf0 con 0x7fec74159db0 2023-10-21T17:39:23.445 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:23.445 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1533795007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":1533795007},{"type":"v1","addr":"172.21.15.124:6837","nonce":1533795007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:38:47.013927+0000","modified":"2023-10-21T17:38:50.070533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11148":{"gid":11148,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":8,"addr":"172.21.15.53:6835/1125921177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1125921177},{"type":"v1","addr":"172.21.15.53:6835","nonce":1125921177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-21T17:39:23.445 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 120 2023-10-21T17:39:23.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 msgr2=0x7fec5806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 0x7fec5806d5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec70005520 tx=0x7fec70009040 comp rx=0 tx=0).stop 2023-10-21T17:39:23.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 msgr2=0x7fec740701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c450 tx=0x7fec6c00c910 comp rx=0 tx=0).stop 2023-10-21T17:39:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 shutdown_connections 2023-10-21T17:39:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fec5806b0e0 0x7fec5806d5a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec74159db0 0x7fec740701d0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec741599d0 0x7fec741f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 --2- 172.21.15.53:0/792153053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec740cc210 0x7fec74150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 >> 172.21.15.53:0/792153053 conn(0x7fec740cb0c0 msgr2=0x7fec740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 shutdown_connections 2023-10-21T17:39:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.446+0000 7fec7a072640 1 -- 172.21.15.53:0/792153053 wait complete. 2023-10-21T17:39:23.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.678+0000 7f9b0f78a640 1 Processor -- start 2023-10-21T17:39:23.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.678+0000 7f9b0f78a640 1 -- start start 2023-10-21T17:39:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0f78a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9b08164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0f78a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0806dee0 con 0x7f9b08159ac0 2023-10-21T17:39:23.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0f78a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0815d850 con 0x7f9b080ad790 2023-10-21T17:39:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0f78a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0806dcf0 con 0x7f9b080709e0 2023-10-21T17:39:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0d526640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9b08159ac0 0x7f9b0806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:40986/0 (socket says 172.21.15.53:40986) 2023-10-21T17:39:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0d526640 1 -- 172.21.15.53:0/2545335823 learned_addr learned my addr 172.21.15.53:0/2545335823 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735700508 0 0) 0x7f9b0806dee0 con 0x7f9b08159ac0 2023-10-21T17:39:23.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af4005730 con 0x7f9b08159ac0 2023-10-21T17:39:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.680+0000 7f9b0dd27640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9b08164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783048647 0 0) 0x7f9af4005730 con 0x7f9b08159ac0 2023-10-21T17:39:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 msgr2=0x7f9b08164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9b08164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 >> v1:172.21.15.124:6789/0 conn(0x7f9b080ad790 legacy=0x7f9b080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0815db30 con 0x7f9b08159ac0 2023-10-21T17:39:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f9b04001670 con 0x7f9b08159ac0 2023-10-21T17:39:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 >> v1:172.21.15.53:6789/0 conn(0x7f9b08159ac0 legacy=0x7f9b0806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9af4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9af40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4006f10 0x7f9af4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400dd70 con 0x7f9af4005bb0 2023-10-21T17:39:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400dee0 con 0x7f9af4006f10 2023-10-21T17:39:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af400e0a0 con 0x7f9b080709e0 2023-10-21T17:39:23.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.681+0000 7f9b0d526640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9af4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0d526640 1 -- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 msgr2=0x7f9af40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0cd25640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9af40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0d526640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b080709e0 0x7f9af40069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0d526640 1 -- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4006f10 msgr2=0x7f9af4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:23.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0d526640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af4006f10 0x7f9af4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.682+0000 7f9b0d526640 1 -- 172.21.15.53:0/2545335823 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4005730 con 0x7f9af4005bb0 2023-10-21T17:39:23.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9b0d526640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9af4005fd0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c0e0 tx=0x7f9af4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83774292878c883e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04005ad0 con 0x7f9af4005bb0 2023-10-21T17:39:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04005c50 con 0x7f9af4005bb0 2023-10-21T17:39:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9afbfff640 1 -- 172.21.15.53:0/2545335823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0402c7f0 con 0x7f9af4005bb0 2023-10-21T17:39:23.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9b0f78a640 1 -- 172.21.15.53:0/2545335823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 msgr2=0x7f9af4005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/2545335823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9af4005fd0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c0e0 tx=0x7f9af4007a00 comp rx=0 tx=0).stop 2023-10-21T17:39:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.683+0000 7f9b0dd27640 1 -- 172.21.15.53:0/2545335823 reap_dead start 2023-10-21T17:39:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.684+0000 7f9b0f78a640 1 -- 172.21.15.53:0/2545335823 shutdown_connections 2023-10-21T17:39:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.684+0000 7f9b0f78a640 1 -- 172.21.15.53:0/2545335823 >> 172.21.15.53:0/2545335823 conn(0x7f9b0806a380 msgr2=0x7f9b080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:23.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.684+0000 7f9b0f78a640 1 -- 172.21.15.53:0/2545335823 shutdown_connections 2023-10-21T17:39:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.684+0000 7f9b0f78a640 1 -- 172.21.15.53:0/2545335823 wait complete. 2023-10-21T17:39:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.685+0000 7f9b0f78a640 1 Processor -- start 2023-10-21T17:39:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.685+0000 7f9b0f78a640 1 -- start start 2023-10-21T17:39:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0815a3f0 0x7f9b081452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b080b2810 0x7f9b080ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b081f28f0 con 0x7f9af4005bb0 2023-10-21T17:39:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b081f2bd0 con 0x7f9b080b2810 2023-10-21T17:39:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0f78a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b081f2eb0 con 0x7f9b0815a3f0 2023-10-21T17:39:23.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.686+0000 7f9b0d526640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:39192/0 (socket says 172.21.15.53:39192) 2023-10-21T17:39:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 -- 172.21.15.53:0/737097599 learned_addr learned my addr 172.21.15.53:0/737097599 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 -- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0815a3f0 msgr2=0x7f9b081452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0815a3f0 0x7f9b081452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 -- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b080b2810 msgr2=0x7f9b080ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b080b2810 0x7f9b080ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 -- 172.21.15.53:0/737097599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b081f3190 con 0x7f9af4005bb0 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9b0d526640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b04001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.687+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0402cc00 con 0x7f9af4005bb0 2023-10-21T17:39:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.688+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0402cd60 con 0x7f9af4005bb0 2023-10-21T17:39:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.688+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04007270 con 0x7f9af4005bb0 2023-10-21T17:39:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.688+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b080b1540 con 0x7f9af4005bb0 2023-10-21T17:39:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.688+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0815a870 con 0x7f9af4005bb0 2023-10-21T17:39:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.689+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b080b16d0 con 0x7f9af4005bb0 2023-10-21T17:39:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.690+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9b04032070 con 0x7f9af4005bb0 2023-10-21T17:39:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.690+0000 7f9af9ffb640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 0x7f9aec06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.691+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f9b0403c040 con 0x7f9af4005bb0 2023-10-21T17:39:23.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.691+0000 7f9b0cd25640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 0x7f9aec06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:23.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.692+0000 7f9b0cd25640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 0x7f9aec06d5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b08156950 tx=0x7f9afc009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:23.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.695+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b04075900 con 0x7f9af4005bb0 2023-10-21T17:39:23.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.888+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b08067490 con 0x7f9af4005bb0 2023-10-21T17:39:23.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.889+0000 7f9af9ffb640 1 -- 172.21.15.53:0/737097599 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18237 (secure 0 0 0) 0x7f9b04078f20 con 0x7f9af4005bb0 2023-10-21T17:39:23.891 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:23.892 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":242,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:19.571410+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:38:45.333243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:38:46.027612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6836/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:23.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 msgr2=0x7f9aec06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 0x7f9aec06d5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b08156950 tx=0x7f9afc009040 comp rx=0 tx=0).stop 2023-10-21T17:39:23.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 msgr2=0x7f9b081f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:23.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b04001930 comp rx=0 tx=0).stop 2023-10-21T17:39:23.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 shutdown_connections 2023-10-21T17:39:23.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f9aec06b0e0 0x7f9aec06d5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b080b2810 0x7f9b080ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.892+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b0815a3f0 0x7f9b081452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.893+0000 7f9b0f78a640 1 --2- 172.21.15.53:0/737097599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9af4005bb0 0x7f9b081f22a0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:23.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.893+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 >> 172.21.15.53:0/737097599 conn(0x7f9b0806a380 msgr2=0x7f9b08156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:23.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.893+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 shutdown_connections 2023-10-21T17:39:23.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:23.893+0000 7f9b0f78a640 1 -- 172.21.15.53:0/737097599 wait complete. 2023-10-21T17:39:23.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-21T17:39:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.144+0000 7fc11a316640 1 Processor -- start 2023-10-21T17:39:24.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.144+0000 7fc11a316640 1 -- start start 2023-10-21T17:39:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.145+0000 7fc11a316640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 0x7fc11415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.145+0000 7fc11a316640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 0x7fc11415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.145+0000 7fc11a316640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1140cab60 con 0x7fc11415a820 2023-10-21T17:39:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.145+0000 7fc11a316640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1140cb960 con 0x7fc1140b0680 2023-10-21T17:39:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.145+0000 7fc11a316640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140cbdf0 con 0x7fc1140ca680 2023-10-21T17:39:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.146+0000 7fc118b13640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 0x7fc11415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.146+0000 7fc118b13640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 0x7fc11415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51368/0 (socket says 172.21.15.53:51368) 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.146+0000 7fc118b13640 1 -- 172.21.15.53:0/4122287959 learned_addr learned my addr 172.21.15.53:0/4122287959 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.146+0000 7fc119b15640 1 --2- 172.21.15.53:0/4122287959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 0x7fc11415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.147+0000 7fc119b15640 1 -- 172.21.15.53:0/4122287959 >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 msgr2=0x7fc11415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.147+0000 7fc119b15640 1 --2- 172.21.15.53:0/4122287959 >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 0x7fc11415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.147+0000 7fc119b15640 1 -- 172.21.15.53:0/4122287959 >> v1:172.21.15.124:6789/0 conn(0x7fc11415a820 legacy=0x7fc1140aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.147+0000 7fc119b15640 1 -- 172.21.15.53:0/4122287959 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140cb030 con 0x7fc1140ca680 2023-10-21T17:39:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.148+0000 7fc119b15640 1 --2- 172.21.15.53:0/4122287959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 0x7fc11415ea80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c02be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b44f5a32ba20963 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.149+0000 7fc107fff640 1 -- 172.21.15.53:0/4122287959 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c02cbd0 con 0x7fc1140ca680 2023-10-21T17:39:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.149+0000 7fc107fff640 1 -- 172.21.15.53:0/4122287959 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c02cd30 con 0x7fc1140ca680 2023-10-21T17:39:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.149+0000 7fc107fff640 1 -- 172.21.15.53:0/4122287959 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c02cea0 con 0x7fc1140ca680 2023-10-21T17:39:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 -- 172.21.15.53:0/4122287959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 msgr2=0x7fc11415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 --2- 172.21.15.53:0/4122287959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 0x7fc11415ea80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009d90 tx=0x7fc10c02be20 comp rx=0 tx=0).stop 2023-10-21T17:39:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 -- 172.21.15.53:0/4122287959 shutdown_connections 2023-10-21T17:39:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 --2- 172.21.15.53:0/4122287959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140ca680 0x7fc11415ea80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 --2- 172.21.15.53:0/4122287959 >> v2:172.21.15.124:3300/0 conn(0x7fc1140b0680 0x7fc11415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 -- 172.21.15.53:0/4122287959 >> 172.21.15.53:0/4122287959 conn(0x7fc11406c050 msgr2=0x7fc11406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 -- 172.21.15.53:0/4122287959 shutdown_connections 2023-10-21T17:39:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.150+0000 7fc11a316640 1 -- 172.21.15.53:0/4122287959 wait complete. 2023-10-21T17:39:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.152+0000 7fc11a316640 1 Processor -- start 2023-10-21T17:39:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.152+0000 7fc11a316640 1 -- start start 2023-10-21T17:39:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140b0680 0x7fc11415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11415a820 0x7fc11415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11415bc10 con 0x7fc1140ca680 2023-10-21T17:39:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140c2130 con 0x7fc11415a820 2023-10-21T17:39:24.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc11a316640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11415b140 con 0x7fc1140b0680 2023-10-21T17:39:24.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc118b13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc118b13640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57578/0 (socket says 172.21.15.53:57578) 2023-10-21T17:39:24.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.153+0000 7fc118b13640 1 -- 172.21.15.53:0/754230123 learned_addr learned my addr 172.21.15.53:0/754230123 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:24.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc119314640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140b0680 0x7fc11415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc119b15640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11415a820 0x7fc11415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 -- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140b0680 msgr2=0x7fc11415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140b0680 0x7fc11415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 -- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11415a820 msgr2=0x7fc11415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11415a820 0x7fc11415c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 -- 172.21.15.53:0/754230123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009a70 con 0x7fc1140ca680 2023-10-21T17:39:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc118b13640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e840 tx=0x7fc10800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.154+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800c8e0 con 0x7fc1140ca680 2023-10-21T17:39:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.155+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800ca40 con 0x7fc1140ca680 2023-10-21T17:39:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.155+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc114158770 con 0x7fc1140ca680 2023-10-21T17:39:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.155+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800cdf0 con 0x7fc1140ca680 2023-10-21T17:39:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.155+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11415b930 con 0x7fc1140ca680 2023-10-21T17:39:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.156+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc114074150 con 0x7fc1140ca680 2023-10-21T17:39:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.157+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc108013470 con 0x7fc1140ca680 2023-10-21T17:39:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.158+0000 7fc105ffb640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 0x7fc0f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.158+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fc10808a040 con 0x7fc1140ca680 2023-10-21T17:39:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.159+0000 7fc119314640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 0x7fc0f806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.161+0000 7fc119314640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 0x7fc0f806d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc110001640 tx=0x7fc110009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.163+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc108054840 con 0x7fc1140ca680 2023-10-21T17:39:24.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.391+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fc1140cc990 con 0x7fc1140ca680 2023-10-21T17:39:24.394 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:39:24.393+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:39:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.396+0000 7fc105ffb640 1 -- 172.21.15.53:0/754230123 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc108057e60 con 0x7fc1140ca680 2023-10-21T17:39:24.398 INFO:teuthology.orchestra.run.smithi053.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:39:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 msgr2=0x7fc0f806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 0x7fc0f806d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc110001640 tx=0x7fc110009040 comp rx=0 tx=0).stop 2023-10-21T17:39:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 msgr2=0x7fc11415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e840 tx=0x7fc10800ed00 comp rx=0 tx=0).stop 2023-10-21T17:39:24.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 shutdown_connections 2023-10-21T17:39:24.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc0f806b0b0 0x7fc0f806d570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11415a820 0x7fc11415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1140ca680 0x7fc11415d4a0 secure :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fc10800e840 tx=0x7fc10800ed00 comp rx=0 tx=0).stop 2023-10-21T17:39:24.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 --2- 172.21.15.53:0/754230123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1140b0680 0x7fc11415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.399+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 >> 172.21.15.53:0/754230123 conn(0x7fc11406c050 msgr2=0x7fc1140ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:24.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.400+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 shutdown_connections 2023-10-21T17:39:24.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.400+0000 7fc11a316640 1 -- 172.21.15.53:0/754230123 wait complete. 2023-10-21T17:39:24.420 INFO:tasks.ceph.mds.c.smithi053.stderr: -1971> 2023-10-21T17:38:52.256+0000 7f581ba9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:39:24.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-21T17:39:24.487 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:39:24.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.607+0000 7fa627fff640 1 Processor -- start 2023-10-21T17:39:24.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.607+0000 7fa627fff640 1 -- start start 2023-10-21T17:39:24.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.609+0000 7fa627fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa628069fc0 0x7fa628071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.609+0000 7fa627fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa62806a3a0 0x7fa628160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.609+0000 7fa627fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa628160d10 con 0x7fa62806ec00 2023-10-21T17:39:24.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.609+0000 7fa627fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa628072070 con 0x7fa62806a3a0 2023-10-21T17:39:24.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.609+0000 7fa627fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa628071580 con 0x7fa628069fc0 2023-10-21T17:39:24.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa61e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa62806ec00 0x7fa6280c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47752/0 (socket says 172.21.15.53:47752) 2023-10-21T17:39:24.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa61e7fc640 1 -- 172.21.15.53:0/4035934288 learned_addr learned my addr 172.21.15.53:0/4035934288 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:24.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6277fe640 1 --2- 172.21.15.53:0/4035934288 >> v2:172.21.15.53:3300/0 conn(0x7fa62806a3a0 0x7fa628160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa626ffd640 1 --2- 172.21.15.53:0/4035934288 >> v2:172.21.15.124:3300/0 conn(0x7fa628069fc0 0x7fa628071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6267fc640 1 -- 172.21.15.53:0/4035934288 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466141807 0 0) 0x7fa628160d10 con 0x7fa62806ec00 2023-10-21T17:39:24.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6267fc640 1 -- 172.21.15.53:0/4035934288 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa614003520 con 0x7fa62806ec00 2023-10-21T17:39:24.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6277fe640 1 -- 172.21.15.53:0/4035934288 >> v2:172.21.15.124:3300/0 conn(0x7fa628069fc0 msgr2=0x7fa628071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6277fe640 1 --2- 172.21.15.53:0/4035934288 >> v2:172.21.15.124:3300/0 conn(0x7fa628069fc0 0x7fa628071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6277fe640 1 -- 172.21.15.53:0/4035934288 >> v1:172.21.15.53:6789/0 conn(0x7fa62806ec00 legacy=0x7fa6280c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.610+0000 7fa6277fe640 1 -- 172.21.15.53:0/4035934288 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628072330 con 0x7fa62806a3a0 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa6277fe640 1 --2- 172.21.15.53:0/4035934288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa62806a3a0 0x7fa628160080 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fa628070de0 tx=0x7fa62002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b13d54f79f9811 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa6267fc640 1 -- 172.21.15.53:0/4035934288 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62002cc80 con 0x7fa62806a3a0 2023-10-21T17:39:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa6267fc640 1 -- 172.21.15.53:0/4035934288 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62002cde0 con 0x7fa62806a3a0 2023-10-21T17:39:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa6267fc640 1 -- 172.21.15.53:0/4035934288 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620034660 con 0x7fa62806a3a0 2023-10-21T17:39:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa627fff640 1 -- 172.21.15.53:0/4035934288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa62806a3a0 msgr2=0x7fa628160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa627fff640 1 --2- 172.21.15.53:0/4035934288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa62806a3a0 0x7fa628160080 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fa628070de0 tx=0x7fa62002bed0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa627fff640 1 -- 172.21.15.53:0/4035934288 shutdown_connections 2023-10-21T17:39:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa627fff640 1 --2- 172.21.15.53:0/4035934288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa62806a3a0 0x7fa628160080 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.612+0000 7fa627fff640 1 --2- 172.21.15.53:0/4035934288 >> v2:172.21.15.124:3300/0 conn(0x7fa628069fc0 0x7fa628071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.613+0000 7fa627fff640 1 -- 172.21.15.53:0/4035934288 >> 172.21.15.53:0/4035934288 conn(0x7fa6280cf5f0 msgr2=0x7fa6280cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:24.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.613+0000 7fa627fff640 1 -- 172.21.15.53:0/4035934288 shutdown_connections 2023-10-21T17:39:24.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.613+0000 7fa627fff640 1 -- 172.21.15.53:0/4035934288 wait complete. 2023-10-21T17:39:24.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.615+0000 7fa627fff640 1 Processor -- start 2023-10-21T17:39:24.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.615+0000 7fa627fff640 1 -- start start 2023-10-21T17:39:24.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa62806a3a0 0x7fa6280b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa62806ec00 0x7fa6280b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280b53d0 con 0x7fa628069fc0 2023-10-21T17:39:24.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6281e7790 con 0x7fa62806a3a0 2023-10-21T17:39:24.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.616+0000 7fa627fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280b4330 con 0x7fa62806ec00 2023-10-21T17:39:24.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57614/0 (socket says 172.21.15.53:57614) 2023-10-21T17:39:24.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 -- 172.21.15.53:0/2141856055 learned_addr learned my addr 172.21.15.53:0/2141856055 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa6277fe640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa62806ec00 0x7fa6280b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 -- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa62806ec00 msgr2=0x7fa6280b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa62806ec00 0x7fa6280b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 -- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa62806a3a0 msgr2=0x7fa6280b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa62806a3a0 0x7fa6280b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:24.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.617+0000 7fa626ffd640 1 -- 172.21.15.53:0/2141856055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620009a70 con 0x7fa628069fc0 2023-10-21T17:39:24.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa626ffd640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e870 tx=0x7fa61800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180094f0 con 0x7fa628069fc0 2023-10-21T17:39:24.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61800c940 con 0x7fa628069fc0 2023-10-21T17:39:24.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6280b4610 con 0x7fa628069fc0 2023-10-21T17:39:24.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800ccf0 con 0x7fa628069fc0 2023-10-21T17:39:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.618+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6280b4bd0 con 0x7fa628069fc0 2023-10-21T17:39:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.619+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa628074150 con 0x7fa628069fc0 2023-10-21T17:39:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.620+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa618013470 con 0x7fa628069fc0 2023-10-21T17:39:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.621+0000 7fa624ff9640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 0x7fa60806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:24.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.621+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7146+0+0 (secure 0 0 0) 0x7fa61808a130 con 0x7fa628069fc0 2023-10-21T17:39:24.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.624+0000 7fa61e7fc640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 0x7fa60806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:24.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.625+0000 7fa61e7fc640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 0x7fa60806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6281f58d0 tx=0x7fa610007930 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:24.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.627+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa618054840 con 0x7fa628069fc0 2023-10-21T17:39:24.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:24.854+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa6280cb560 con 0x7fa628069fc0 2023-10-21T17:39:25.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.400+0000 7fa624ff9640 1 -- 172.21.15.53:0/2141856055 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 107+0+0 (secure 0 0 0) 0x7fa618057e60 con 0x7fa628069fc0 2023-10-21T17:39:25.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 msgr2=0x7fa60806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 0x7fa60806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6281f58d0 tx=0x7fa610007930 comp rx=0 tx=0).stop 2023-10-21T17:39:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 msgr2=0x7fa62815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e870 tx=0x7fa61800ed30 comp rx=0 tx=0).stop 2023-10-21T17:39:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 shutdown_connections 2023-10-21T17:39:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa60806b0b0 0x7fa60806d570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa62806ec00 0x7fa6280b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa62806a3a0 0x7fa6280b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 --2- 172.21.15.53:0/2141856055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa628069fc0 0x7fa62815b6e0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 >> 172.21.15.53:0/2141856055 conn(0x7fa6280cf5f0 msgr2=0x7fa6280c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:25.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 shutdown_connections 2023-10-21T17:39:25.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.404+0000 7fa627fff640 1 -- 172.21.15.53:0/2141856055 wait complete. 2023-10-21T17:39:25.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-21T17:39:25.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.654+0000 7f67a759e640 1 Processor -- start 2023-10-21T17:39:25.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.654+0000 7f67a759e640 1 -- start start 2023-10-21T17:39:25.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 0x7f67a80d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a80d2870 con 0x7f67a815ccb0 2023-10-21T17:39:25.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a80d1150 con 0x7f67a80d0b90 2023-10-21T17:39:25.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80d1430 con 0x7f67a80d00e0 2023-10-21T17:39:25.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a659c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f67a815ccb0 0x7f67a80cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47764/0 (socket says 172.21.15.53:47764) 2023-10-21T17:39:25.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a659c640 1 -- 172.21.15.53:0/2823658526 learned_addr learned my addr 172.21.15.53:0/2823658526 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f679ffff640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 0x7f67a80d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:25.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081701337 0 0) 0x7f67a80d2870 con 0x7f67a815ccb0 2023-10-21T17:39:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.656+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6794003520 con 0x7f67a815ccb0 2023-10-21T17:39:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1967215141 0 0) 0x7f6794003520 con 0x7f67a815ccb0 2023-10-21T17:39:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 msgr2=0x7f67a80d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 0x7f67a80d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 >> v1:172.21.15.124:6789/0 conn(0x7f67a80d0b90 legacy=0x7f67a80b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a80d1710 con 0x7f67a815ccb0 2023-10-21T17:39:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6798001670 con 0x7f67a815ccb0 2023-10-21T17:39:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 >> v1:172.21.15.53:6789/0 conn(0x7f67a815ccb0 legacy=0x7f67a80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 0x7f67940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6794004d00 0x7f6794005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80d1430 con 0x7f6794003f70 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679400bac0 con 0x7f6794004d00 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679400bc80 con 0x7f67a80d00e0 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.657+0000 7f679ffff640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:25.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f679ffff640 1 -- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 msgr2=0x7f67940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f679ffff640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80d00e0 0x7f67940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f679ffff640 1 -- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6794004d00 msgr2=0x7f6794005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f67a6d9d640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6794004d00 0x7f6794005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f679ffff640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6794004d00 0x7f6794005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f679ffff640 1 -- 172.21.15.53:0/2823658526 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794003520 con 0x7f6794003f70 2023-10-21T17:39:25.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.658+0000 7f67a6d9d640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6794004d00 0x7f6794005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.659+0000 7f679ffff640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67940047c0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f6790009ac0 tx=0x7f67940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3f8c9a2353cff8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.659+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679002c630 con 0x7f6794003f70 2023-10-21T17:39:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.659+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679002c790 con 0x7f6794003f70 2023-10-21T17:39:25.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.659+0000 7f67a5d9b640 1 -- 172.21.15.53:0/2823658526 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679002cb20 con 0x7f6794003f70 2023-10-21T17:39:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 -- 172.21.15.53:0/2823658526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 msgr2=0x7f67940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 --2- 172.21.15.53:0/2823658526 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67940047c0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f6790009ac0 tx=0x7f67940057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a6d9d640 1 -- 172.21.15.53:0/2823658526 reap_dead start 2023-10-21T17:39:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 -- 172.21.15.53:0/2823658526 shutdown_connections 2023-10-21T17:39:25.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 -- 172.21.15.53:0/2823658526 >> 172.21.15.53:0/2823658526 conn(0x7f67a8071660 msgr2=0x7f67a8071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 -- 172.21.15.53:0/2823658526 shutdown_connections 2023-10-21T17:39:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.660+0000 7f67a759e640 1 -- 172.21.15.53:0/2823658526 wait complete. 2023-10-21T17:39:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.662+0000 7f67a759e640 1 Processor -- start 2023-10-21T17:39:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.662+0000 7f67a759e640 1 -- start start 2023-10-21T17:39:25.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67a80c8d90 0x7f67a80c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80cbf00 0x7f67a80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80cc8c0 con 0x7f6794003f70 2023-10-21T17:39:25.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80c8780 con 0x7f67a80c8d90 2023-10-21T17:39:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80c72a0 con 0x7f67a80cbf00 2023-10-21T17:39:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57644/0 (socket says 172.21.15.53:57644) 2023-10-21T17:39:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.663+0000 7f67a659c640 1 -- 172.21.15.53:0/1564459230 learned_addr learned my addr 172.21.15.53:0/1564459230 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:25.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 -- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80cbf00 msgr2=0x7f67a80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80cbf00 0x7f67a80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 -- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67a80c8d90 msgr2=0x7f67a80c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67a80c8d90 0x7f67a80c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 -- 172.21.15.53:0/1564459230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a80c7580 con 0x7f6794003f70 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a659c640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f679800c790 tx=0x7f679800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:25.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800cea0 con 0x7f6794003f70 2023-10-21T17:39:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67980074e0 con 0x7f6794003f70 2023-10-21T17:39:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798004a20 con 0x7f6794003f70 2023-10-21T17:39:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.664+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a80a94e0 con 0x7f6794003f70 2023-10-21T17:39:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.665+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a80a9aa0 con 0x7f6794003f70 2023-10-21T17:39:25.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.666+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a80a9c90 con 0x7f6794003f70 2023-10-21T17:39:25.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.667+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6798004b80 con 0x7f6794003f70 2023-10-21T17:39:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.668+0000 7f679effd640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 0x7f678806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.668+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7146+0+0 (secure 0 0 0) 0x7f6798089410 con 0x7f6794003f70 2023-10-21T17:39:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.668+0000 7f679ffff640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 0x7f678806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.670+0000 7f679ffff640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 0x7f678806d5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6790036000 tx=0x7f679002e000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:25.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.672+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6798053bc0 con 0x7f6794003f70 2023-10-21T17:39:25.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:25.866+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67a8074150 con 0x7f6794003f70 2023-10-21T17:39:26.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.637+0000 7f679effd640 1 -- 172.21.15.53:0/1564459230 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v244) v1 ==== 190+0+0 (secure 0 0 0) 0x7f67980571e0 con 0x7f6794003f70 2023-10-21T17:39:26.639 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_metadata' removed 2023-10-21T17:39:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.642+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 msgr2=0x7f678806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.642+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 0x7f678806d5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6790036000 tx=0x7f679002e000 comp rx=0 tx=0).stop 2023-10-21T17:39:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.642+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 msgr2=0x7f67a81f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.642+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f679800c790 tx=0x7f679800cc50 comp rx=0 tx=0).stop 2023-10-21T17:39:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 shutdown_connections 2023-10-21T17:39:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f678806b0e0 0x7f678806d5a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67a80cbf00 0x7f67a80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f67a80c8d90 0x7f67a80c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 --2- 172.21.15.53:0/1564459230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6794003f70 0x7f67a81f1cd0 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 >> 172.21.15.53:0/1564459230 conn(0x7f67a8071660 msgr2=0x7f67a8073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 shutdown_connections 2023-10-21T17:39:26.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.643+0000 7f67a759e640 1 -- 172.21.15.53:0/1564459230 wait complete. 2023-10-21T17:39:26.671 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-21T17:39:26.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.882+0000 7f4d6b59e640 1 Processor -- start 2023-10-21T17:39:26.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.882+0000 7f4d6b59e640 1 -- start start 2023-10-21T17:39:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d6c15a820 0x7f4d6c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4d6c15ac00 0x7f4d6c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cab60 con 0x7f4d6c15a820 2023-10-21T17:39:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cb960 con 0x7f4d6c15ac00 2023-10-21T17:39:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0cbdf0 con 0x7f4d6c0ca680 2023-10-21T17:39:26.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d6c15a820 0x7f4d6c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4d6c15a820 0x7f4d6c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57656/0 (socket says 172.21.15.53:57656) 2023-10-21T17:39:26.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.884+0000 7f4d6a59c640 1 -- 172.21.15.53:0/4211161733 learned_addr learned my addr 172.21.15.53:0/4211161733 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6a59c640 1 -- 172.21.15.53:0/4211161733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 msgr2=0x7f4d6c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:26.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/4211161733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6ad9d640 1 --2- 172.21.15.53:0/4211161733 >> v2:172.21.15.124:3300/0 conn(0x7f4d6c15ac00 0x7f4d6c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6a59c640 1 -- 172.21.15.53:0/4211161733 >> v2:172.21.15.124:3300/0 conn(0x7f4d6c15ac00 msgr2=0x7f4d6c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/4211161733 >> v2:172.21.15.124:3300/0 conn(0x7f4d6c15ac00 0x7f4d6c0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.885+0000 7f4d6a59c640 1 -- 172.21.15.53:0/4211161733 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0cb030 con 0x7f4d6c15a820 2023-10-21T17:39:26.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.886+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/4211161733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c0aff70 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ce340 tx=0x7f4d5c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82192f4a7e7d1c07 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.886+0000 7f4d69d9b640 1 -- 172.21.15.53:0/4211161733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0317e0 con 0x7f4d6c15a820 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.886+0000 7f4d69d9b640 1 -- 172.21.15.53:0/4211161733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c031940 con 0x7f4d6c15a820 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.886+0000 7f4d69d9b640 1 -- 172.21.15.53:0/4211161733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c031cd0 con 0x7f4d6c15a820 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 -- 172.21.15.53:0/4211161733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 msgr2=0x7f4d6c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/4211161733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c0aff70 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0ce340 tx=0x7f4d5c02b880 comp rx=0 tx=0).stop 2023-10-21T17:39:26.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 -- 172.21.15.53:0/4211161733 shutdown_connections 2023-10-21T17:39:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/4211161733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/4211161733 >> v2:172.21.15.124:3300/0 conn(0x7f4d6c15ac00 0x7f4d6c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/4211161733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c0aff70 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 -- 172.21.15.53:0/4211161733 >> 172.21.15.53:0/4211161733 conn(0x7f4d6c06c050 msgr2=0x7f4d6c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:26.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 -- 172.21.15.53:0/4211161733 shutdown_connections 2023-10-21T17:39:26.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.887+0000 7f4d6b59e640 1 -- 172.21.15.53:0/4211161733 wait complete. 2023-10-21T17:39:26.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.889+0000 7f4d6b59e640 1 Processor -- start 2023-10-21T17:39:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.889+0000 7f4d6b59e640 1 -- start start 2023-10-21T17:39:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d6c15ac00 0x7f4d6c15b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c55c0 con 0x7f4d6c15a820 2023-10-21T17:39:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c5810 con 0x7f4d6c15ac00 2023-10-21T17:39:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.890+0000 7f4d6b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0c5af0 con 0x7f4d6c0ca680 2023-10-21T17:39:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57662/0 (socket says 172.21.15.53:57662) 2023-10-21T17:39:26.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 -- 172.21.15.53:0/73050770 learned_addr learned my addr 172.21.15.53:0/73050770 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d6ad9d640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d6c15ac00 0x7f4d6c15b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 -- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 msgr2=0x7f4d6c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 -- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d6c15ac00 msgr2=0x7f4d6c15b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:26.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d6c15ac00 0x7f4d6c15b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 -- 172.21.15.53:0/73050770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0094d0 con 0x7f4d6c15a820 2023-10-21T17:39:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.891+0000 7f4d63fff640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400b4a0 tx=0x7f4d5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.892+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54010610 con 0x7f4d6c15a820 2023-10-21T17:39:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.892+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d54010770 con 0x7f4d6c15a820 2023-10-21T17:39:26.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.892+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54010b20 con 0x7f4d6c15a820 2023-10-21T17:39:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.892+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c0c5dd0 con 0x7f4d6c15a820 2023-10-21T17:39:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.892+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c0c6390 con 0x7f4d6c15a820 2023-10-21T17:39:26.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.893+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c0c65b0 con 0x7f4d6c15a820 2023-10-21T17:39:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.894+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4d54010c80 con 0x7f4d6c15a820 2023-10-21T17:39:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.895+0000 7f4d62ffd640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 0x7f4d4c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.895+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6702+0+0 (secure 0 0 0) 0x7f4d54089a90 con 0x7f4d6c15a820 2023-10-21T17:39:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.896+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 0x7f4d4c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:26.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.897+0000 7f4d6a59c640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 0x7f4d4c06d570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c036000 tx=0x7f4d5c03d040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:26.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:26.900+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d540543f0 con 0x7f4d6c15a820 2023-10-21T17:39:27.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.093+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d6c0b1310 con 0x7f4d6c15a820 2023-10-21T17:39:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.653+0000 7f4d62ffd640 1 -- 172.21.15.53:0/73050770 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v245) v1 ==== 178+0+0 (secure 0 0 0) 0x7f4d54057a10 con 0x7f4d6c15a820 2023-10-21T17:39:27.655 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_data' removed 2023-10-21T17:39:27.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.659+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 msgr2=0x7f4d4c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.659+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 0x7f4d4c06d570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c036000 tx=0x7f4d5c03d040 comp rx=0 tx=0).stop 2023-10-21T17:39:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.659+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 msgr2=0x7f4d6c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.659+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400b4a0 tx=0x7f4d5400b960 comp rx=0 tx=0).stop 2023-10-21T17:39:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 shutdown_connections 2023-10-21T17:39:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4d4c06b0b0 0x7f4d4c06d570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d6c15ac00 0x7f4d6c15b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4d6c15a820 0x7f4d6c15b200 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 --2- 172.21.15.53:0/73050770 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d6c0ca680 0x7f4d6c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.663+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 >> 172.21.15.53:0/73050770 conn(0x7f4d6c06c050 msgr2=0x7f4d6c0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:27.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.664+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 shutdown_connections 2023-10-21T17:39:27.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.664+0000 7f4d6b59e640 1 -- 172.21.15.53:0/73050770 wait complete. 2023-10-21T17:39:27.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:39:27.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.892+0000 7fb2df581640 1 Processor -- start 2023-10-21T17:39:27.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.892+0000 7fb2df581640 1 -- start start 2023-10-21T17:39:27.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.894+0000 7fb2df581640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2d80b0680 0x7fb2d815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.894+0000 7fb2df581640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80ca680 0x7fb2d815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.894+0000 7fb2df581640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80cab60 con 0x7fb2d815a820 2023-10-21T17:39:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.894+0000 7fb2df581640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d80cb960 con 0x7fb2d80b0680 2023-10-21T17:39:27.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.894+0000 7fb2df581640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80cbdf0 con 0x7fb2d80ca680 2023-10-21T17:39:27.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.895+0000 7fb2ddd7e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2d80b0680 0x7fb2d815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.895+0000 7fb2ddd7e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2d80b0680 0x7fb2d815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51462/0 (socket says 172.21.15.53:51462) 2023-10-21T17:39:27.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.895+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1198703081 learned_addr learned my addr 172.21.15.53:0/1198703081 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.895+0000 7fb2de57f640 1 --1- 172.21.15.53:0/1198703081 >> v1:172.21.15.124:6789/0 conn(0x7fb2d815a820 0x7fb2d80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:48690/0 (socket says 172.21.15.53:48690) 2023-10-21T17:39:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.895+0000 7fb2ded80640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80ca680 0x7fb2d815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.896+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80ca680 msgr2=0x7fb2d815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:27.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.896+0000 7fb2ddd7e640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80ca680 0x7fb2d815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.896+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1198703081 >> v1:172.21.15.124:6789/0 conn(0x7fb2d815a820 legacy=0x7fb2d80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.896+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1198703081 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80cb030 con 0x7fb2d80b0680 2023-10-21T17:39:27.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.897+0000 7fb2ddd7e640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d80b0680 0x7fb2d815e540 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009870 tx=0x7fb2c802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b941093d5489ef7e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:27.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.898+0000 7fb2dd57d640 1 -- 172.21.15.53:0/1198703081 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8031640 con 0x7fb2d80b0680 2023-10-21T17:39:27.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.898+0000 7fb2dd57d640 1 -- 172.21.15.53:0/1198703081 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80317a0 con 0x7fb2d80b0680 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.898+0000 7fb2dd57d640 1 -- 172.21.15.53:0/1198703081 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8031b30 con 0x7fb2d80b0680 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 -- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d80b0680 msgr2=0x7fb2d815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d80b0680 0x7fb2d815e540 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009870 tx=0x7fb2c802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 -- 172.21.15.53:0/1198703081 shutdown_connections 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80ca680 0x7fb2d815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 --2- 172.21.15.53:0/1198703081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d80b0680 0x7fb2d815e540 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 -- 172.21.15.53:0/1198703081 >> 172.21.15.53:0/1198703081 conn(0x7fb2d806c050 msgr2=0x7fb2d806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:27.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 -- 172.21.15.53:0/1198703081 shutdown_connections 2023-10-21T17:39:27.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.899+0000 7fb2df581640 1 -- 172.21.15.53:0/1198703081 wait complete. 2023-10-21T17:39:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.901+0000 7fb2df581640 1 Processor -- start 2023-10-21T17:39:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.901+0000 7fb2df581640 1 -- start start 2023-10-21T17:39:27.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80b0680 0x7fb2d814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 0x7fb2d80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c87a0 con 0x7fb2d80ca680 2023-10-21T17:39:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c58b0 con 0x7fb2d815a820 2023-10-21T17:39:27.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2df581640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80c6020 con 0x7fb2d80b0680 2023-10-21T17:39:27.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2ddd7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2ddd7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57668/0 (socket says 172.21.15.53:57668) 2023-10-21T17:39:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.902+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1881821108 learned_addr learned my addr 172.21.15.53:0/1881821108 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2de57f640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80b0680 0x7fb2d814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ded80640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 0x7fb2d80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80b0680 msgr2=0x7fb2d814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80b0680 0x7fb2d814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 msgr2=0x7fb2d80c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 0x7fb2d80c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 -- 172.21.15.53:0/1881821108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80094d0 con 0x7fb2d80ca680 2023-10-21T17:39:27.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ded80640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 0x7fb2d80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.903+0000 7fb2ddd7e640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000c00 tx=0x7fb2c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.904+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803a930 con 0x7fb2d80ca680 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.904+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c803aa90 con 0x7fb2d80ca680 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.904+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803ae40 con 0x7fb2d80ca680 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.904+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d80c7dd0 con 0x7fb2d80ca680 2023-10-21T17:39:27.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.904+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8156630 con 0x7fb2d80ca680 2023-10-21T17:39:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.905+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8074150 con 0x7fb2d80ca680 2023-10-21T17:39:27.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.906+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb2c802f050 con 0x7fb2d80ca680 2023-10-21T17:39:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.907+0000 7fb2ceffd640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 0x7fb2bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.907+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb2c80abb00 con 0x7fb2d80ca680 2023-10-21T17:39:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.907+0000 7fb2de57f640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 0x7fb2bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:27.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.909+0000 7fb2de57f640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 0x7fb2bc06d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000f60 tx=0x7fb2d4000ed0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:27.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:27.911+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2c80ade60 con 0x7fb2d80ca680 2023-10-21T17:39:28.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.113+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb2d80c8040 con 0x7fb2d80ca680 2023-10-21T17:39:28.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.120+0000 7fb2ceffd640 1 -- 172.21.15.53:0/1881821108 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2c8076460 con 0x7fb2d80ca680 2023-10-21T17:39:28.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.122+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 msgr2=0x7fb2bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.122+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 0x7fb2bc06d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000f60 tx=0x7fb2d4000ed0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.122+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 msgr2=0x7fb2d80c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.122+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000c00 tx=0x7fb2c80046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 shutdown_connections 2023-10-21T17:39:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb2bc06b0b0 0x7fb2bc06d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2d815a820 0x7fb2d80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2d80ca680 0x7fb2d80c8260 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 --2- 172.21.15.53:0/1881821108 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2d80b0680 0x7fb2d814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 >> 172.21.15.53:0/1881821108 conn(0x7fb2d806c050 msgr2=0x7fb2d80ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:28.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 shutdown_connections 2023-10-21T17:39:28.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.123+0000 7fb2df581640 1 -- 172.21.15.53:0/1881821108 wait complete. 2023-10-21T17:39:28.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:39:28.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.369+0000 7f49e0d47640 1 Processor -- start 2023-10-21T17:39:28.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.369+0000 7f49e0d47640 1 -- start start 2023-10-21T17:39:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.370+0000 7f49e0d47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc15aab0 0x7f49dc070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.370+0000 7f49e0d47640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 0x7f49dc0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.370+0000 7f49e0d47640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49dc0cb350 0x7f49dc0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.370+0000 7f49e0d47640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc15ae90 con 0x7f49dc0cb350 2023-10-21T17:39:28.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49e0d47640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc0cbdb0 con 0x7f49dc0caef0 2023-10-21T17:39:28.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49e0d47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0cb7f0 con 0x7f49dc15aab0 2023-10-21T17:39:28.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49dbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49dc0cb350 0x7f49dc0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49dbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49dc0cb350 0x7f49dc0b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57680/0 (socket says 172.21.15.53:57680) 2023-10-21T17:39:28.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49dbfff640 1 -- 172.21.15.53:0/2722551385 learned_addr learned my addr 172.21.15.53:0/2722551385 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:28.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49daffd640 1 --2- 172.21.15.53:0/2722551385 >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 0x7f49dc0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.371+0000 7f49db7fe640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc15aab0 0x7f49dc070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49dbfff640 1 -- 172.21.15.53:0/2722551385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc15aab0 msgr2=0x7f49dc070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:28.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49dbfff640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc15aab0 0x7f49dc070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49dbfff640 1 -- 172.21.15.53:0/2722551385 >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 msgr2=0x7f49dc0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:28.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49dbfff640 1 --2- 172.21.15.53:0/2722551385 >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 0x7f49dc0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49dbfff640 1 -- 172.21.15.53:0/2722551385 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0cbad0 con 0x7f49dc0cb350 2023-10-21T17:39:28.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.372+0000 7f49daffd640 1 --2- 172.21.15.53:0/2722551385 >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 0x7f49dc0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.373+0000 7f49dbfff640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0cb350 0x7f49dc0b04a0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f49dc06e530 tx=0x7f49d402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97c328e8bc94a53 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.373+0000 7f49da7fc640 1 -- 172.21.15.53:0/2722551385 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d40317d0 con 0x7f49dc0cb350 2023-10-21T17:39:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.373+0000 7f49da7fc640 1 -- 172.21.15.53:0/2722551385 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4031930 con 0x7f49dc0cb350 2023-10-21T17:39:28.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.373+0000 7f49da7fc640 1 -- 172.21.15.53:0/2722551385 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4031cc0 con 0x7f49dc0cb350 2023-10-21T17:39:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 -- 172.21.15.53:0/2722551385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0cb350 msgr2=0x7f49dc0b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0cb350 0x7f49dc0b04a0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f49dc06e530 tx=0x7f49d402b870 comp rx=0 tx=0).stop 2023-10-21T17:39:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 -- 172.21.15.53:0/2722551385 shutdown_connections 2023-10-21T17:39:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0cb350 0x7f49dc0b04a0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2722551385 >> v2:172.21.15.124:3300/0 conn(0x7f49dc0caef0 0x7f49dc0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2722551385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc15aab0 0x7f49dc070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 -- 172.21.15.53:0/2722551385 >> 172.21.15.53:0/2722551385 conn(0x7f49dc06c010 msgr2=0x7f49dc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:28.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 -- 172.21.15.53:0/2722551385 shutdown_connections 2023-10-21T17:39:28.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.374+0000 7f49e0d47640 1 -- 172.21.15.53:0/2722551385 wait complete. 2023-10-21T17:39:28.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.376+0000 7f49e0d47640 1 Processor -- start 2023-10-21T17:39:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.376+0000 7f49e0d47640 1 -- start start 2023-10-21T17:39:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 0x7f49dc157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49dc15aab0 0x7f49dc158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0ad530 con 0x7f49dc0caef0 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc150060 con 0x7f49dc15aab0 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49e0d47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc0ccf70 con 0x7f49dc0cb350 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49db7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57690/0 (socket says 172.21.15.53:57690) 2023-10-21T17:39:28.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49db7fe640 1 -- 172.21.15.53:0/2408143325 learned_addr learned my addr 172.21.15.53:0/2408143325 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49daffd640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 0x7f49dc157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.377+0000 7f49dbfff640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49dc15aab0 0x7f49dc158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 -- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 msgr2=0x7f49dc157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 0x7f49dc157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 -- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49dc15aab0 msgr2=0x7f49dc158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49dc15aab0 0x7f49dc158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 -- 172.21.15.53:0/2408143325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40094d0 con 0x7f49dc0caef0 2023-10-21T17:39:28.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49daffd640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 0x7f49dc157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:28.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49db7fe640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f49cc007e40 tx=0x7f49cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc014620 con 0x7f49dc0caef0 2023-10-21T17:39:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00ee90 con 0x7f49dc0caef0 2023-10-21T17:39:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0149c0 con 0x7f49dc0caef0 2023-10-21T17:39:28.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc0acb90 con 0x7f49dc0caef0 2023-10-21T17:39:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.378+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc0ad150 con 0x7f49dc0caef0 2023-10-21T17:39:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.380+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc0c8fc0 con 0x7f49dc0caef0 2023-10-21T17:39:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.380+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f49cc014b20 con 0x7f49dc0caef0 2023-10-21T17:39:28.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.381+0000 7f49d3fff640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 0x7f49bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.382+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f49cc08a5d0 con 0x7f49dc0caef0 2023-10-21T17:39:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.382+0000 7f49daffd640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 0x7f49bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.383+0000 7f49daffd640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 0x7f49bc06d570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49dc1f5330 tx=0x7f49c4008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.386+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49cc055110 con 0x7f49dc0caef0 2023-10-21T17:39:28.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.580+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f49dc074150 con 0x7f49dc0caef0 2023-10-21T17:39:28.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.587+0000 7f49d3fff640 1 -- 172.21.15.53:0/2408143325 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v80) v1 ==== 115+0+0 (secure 0 0 0) 0x7f49cc058730 con 0x7f49dc0caef0 2023-10-21T17:39:28.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 msgr2=0x7f49bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 0x7f49bc06d570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49dc1f5330 tx=0x7f49c4008040 comp rx=0 tx=0).stop 2023-10-21T17:39:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 msgr2=0x7f49dc14fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f49cc007e40 tx=0x7f49cc00e4b0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 shutdown_connections 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f49bc06b0b0 0x7f49bc06d570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49dc15aab0 0x7f49dc158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49dc0cb350 0x7f49dc157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 --2- 172.21.15.53:0/2408143325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49dc0caef0 0x7f49dc14fb20 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 >> 172.21.15.53:0/2408143325 conn(0x7f49dc06c010 msgr2=0x7f49dc15bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:28.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 shutdown_connections 2023-10-21T17:39:28.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.590+0000 7f49e0d47640 1 -- 172.21.15.53:0/2408143325 wait complete. 2023-10-21T17:39:28.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:39:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.841+0000 7fcfef59e640 1 Processor -- start 2023-10-21T17:39:28.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.841+0000 7fcfef59e640 1 -- start start 2023-10-21T17:39:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.842+0000 7fcfef59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 0x7fcff00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfef59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcff00b3c50 0x7fcff00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfef59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff0159db0 con 0x7fcff00cb410 2023-10-21T17:39:28.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfef59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff0071fe0 con 0x7fcff00b3c50 2023-10-21T17:39:28.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfef59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff00b44c0 con 0x7fcff01599d0 2023-10-21T17:39:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfe7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcff00b3c50 0x7fcff00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfe7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcff00b3c50 0x7fcff00caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57706/0 (socket says 172.21.15.53:57706) 2023-10-21T17:39:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.843+0000 7fcfe7fff640 1 -- 172.21.15.53:0/1618997614 learned_addr learned my addr 172.21.15.53:0/1618997614 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:28.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfe7fff640 1 -- 172.21.15.53:0/1618997614 >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 msgr2=0x7fcff00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfee59c640 1 --2- 172.21.15.53:0/1618997614 >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 0x7fcff00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfe7fff640 1 --2- 172.21.15.53:0/1618997614 >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 0x7fcff00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfe7fff640 1 -- 172.21.15.53:0/1618997614 >> v1:172.21.15.124:6789/0 conn(0x7fcff00cb410 legacy=0x7fcff00c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfe7fff640 1 -- 172.21.15.53:0/1618997614 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff00cb860 con 0x7fcff00b3c50 2023-10-21T17:39:28.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.844+0000 7fcfee59c640 1 --2- 172.21.15.53:0/1618997614 >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 0x7fcff00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:28.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.845+0000 7fcfe7fff640 1 --2- 172.21.15.53:0/1618997614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00caed0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fcff006e150 tx=0x7fcfd802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60815ebd18e1c4de server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.845+0000 7fcfedd9b640 1 -- 172.21.15.53:0/1618997614 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8031870 con 0x7fcff00b3c50 2023-10-21T17:39:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.845+0000 7fcfedd9b640 1 -- 172.21.15.53:0/1618997614 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80319d0 con 0x7fcff00b3c50 2023-10-21T17:39:28.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.845+0000 7fcfedd9b640 1 -- 172.21.15.53:0/1618997614 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8031d60 con 0x7fcff00b3c50 2023-10-21T17:39:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 -- 172.21.15.53:0/1618997614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 msgr2=0x7fcff00caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 --2- 172.21.15.53:0/1618997614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00caed0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7fcff006e150 tx=0x7fcfd802b910 comp rx=0 tx=0).stop 2023-10-21T17:39:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 -- 172.21.15.53:0/1618997614 shutdown_connections 2023-10-21T17:39:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 --2- 172.21.15.53:0/1618997614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00caed0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 --2- 172.21.15.53:0/1618997614 >> v2:172.21.15.124:3300/0 conn(0x7fcff01599d0 0x7fcff00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 -- 172.21.15.53:0/1618997614 >> 172.21.15.53:0/1618997614 conn(0x7fcff006c050 msgr2=0x7fcff006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 -- 172.21.15.53:0/1618997614 shutdown_connections 2023-10-21T17:39:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.846+0000 7fcfef59e640 1 -- 172.21.15.53:0/1618997614 wait complete. 2023-10-21T17:39:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.848+0000 7fcfef59e640 1 Processor -- start 2023-10-21T17:39:28.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.848+0000 7fcfef59e640 1 -- start start 2023-10-21T17:39:28.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcff00cb410 0x7fcff0157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcff01599d0 0x7fcff0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00ac7c0 con 0x7fcff00b3c50 2023-10-21T17:39:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00ab8d0 con 0x7fcff01599d0 2023-10-21T17:39:28.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfef59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00abbb0 con 0x7fcff00cb410 2023-10-21T17:39:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfee59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfee59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57710/0 (socket says 172.21.15.53:57710) 2023-10-21T17:39:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.849+0000 7fcfee59c640 1 -- 172.21.15.53:0/484962166 learned_addr learned my addr 172.21.15.53:0/484962166 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 -- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcff00cb410 msgr2=0x7fcff0157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcff00cb410 0x7fcff0157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 -- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcff01599d0 msgr2=0x7fcff0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcff01599d0 0x7fcff0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 -- 172.21.15.53:0/484962166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80094d0 con 0x7fcff00b3c50 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfee59c640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c7f0 tx=0x7fcfe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.850+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007820 con 0x7fcff00b3c50 2023-10-21T17:39:28.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.851+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe0007980 con 0x7fcff00b3c50 2023-10-21T17:39:28.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.851+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007cf0 con 0x7fcff00b3c50 2023-10-21T17:39:28.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.851+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff00abe90 con 0x7fcff00b3c50 2023-10-21T17:39:28.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.851+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff00ac450 con 0x7fcff00b3c50 2023-10-21T17:39:28.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.852+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0074150 con 0x7fcff00b3c50 2023-10-21T17:39:28.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.853+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcfe00113f0 con 0x7fcff00b3c50 2023-10-21T17:39:28.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.854+0000 7fcfe6ffd640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 0x7fcfd006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:28.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.854+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fcfe008a0e0 con 0x7fcff00b3c50 2023-10-21T17:39:28.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.854+0000 7fcfe7fff640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 0x7fcfd006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:28.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.856+0000 7fcfe7fff640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 0x7fcfd006d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8036000 tx=0x7fcfd8030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:28.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:28.858+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfe0054ad0 con 0x7fcff00b3c50 2023-10-21T17:39:29.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.051+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fcff00afbb0 con 0x7fcff00b3c50 2023-10-21T17:39:29.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.058+0000 7fcfe6ffd640 1 -- 172.21.15.53:0/484962166 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v81) v1 ==== 120+0+0 (secure 0 0 0) 0x7fcfe00580f0 con 0x7fcff00b3c50 2023-10-21T17:39:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 msgr2=0x7fcfd006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 0x7fcfd006d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8036000 tx=0x7fcfd8030000 comp rx=0 tx=0).stop 2023-10-21T17:39:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 msgr2=0x7fcff00acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c7f0 tx=0x7fcfe000ccb0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 shutdown_connections 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcfd006b0b0 0x7fcfd006d570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcff01599d0 0x7fcff0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcff00cb410 0x7fcff0157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 --2- 172.21.15.53:0/484962166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcff00b3c50 0x7fcff00acc50 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 >> 172.21.15.53:0/484962166 conn(0x7fcff006c050 msgr2=0x7fcff00aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:29.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 shutdown_connections 2023-10-21T17:39:29.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.061+0000 7fcfef59e640 1 -- 172.21.15.53:0/484962166 wait complete. 2023-10-21T17:39:29.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-21T17:39:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.307+0000 7f325759e640 1 Processor -- start 2023-10-21T17:39:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.307+0000 7f325759e640 1 -- start start 2023-10-21T17:39:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f325759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f325815ab30 0x7f325815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f325759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325815f7d0 con 0x7f32580b07d0 2023-10-21T17:39:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f325759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325815f210 con 0x7f32580adc70 2023-10-21T17:39:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f325759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325815f4f0 con 0x7f325815ab30 2023-10-21T17:39:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f3256d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32580b07d0 0x7f325806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47778/0 (socket says 172.21.15.53:47778) 2023-10-21T17:39:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.308+0000 7f3256d9d640 1 -- 172.21.15.53:0/3035835568 learned_addr learned my addr 172.21.15.53:0/3035835568 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f325659c640 1 --2- 172.21.15.53:0/3035835568 >> v2:172.21.15.124:3300/0 conn(0x7f325815ab30 0x7f325815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263586730 0 0) 0x7f325815f7d0 con 0x7f32580b07d0 2023-10-21T17:39:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3244003520 con 0x7f32580b07d0 2023-10-21T17:39:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146196418 0 0) 0x7f3244003520 con 0x7f32580b07d0 2023-10-21T17:39:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 >> v2:172.21.15.124:3300/0 conn(0x7f325815ab30 msgr2=0x7f325815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 --2- 172.21.15.53:0/3035835568 >> v2:172.21.15.124:3300/0 conn(0x7f325815ab30 0x7f325815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 >> v1:172.21.15.124:6789/0 conn(0x7f32580adc70 legacy=0x7f32580b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325806c550 con 0x7f32580b07d0 2023-10-21T17:39:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f32500019f0 con 0x7f32580b07d0 2023-10-21T17:39:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 >> v1:172.21.15.53:6789/0 conn(0x7f32580b07d0 legacy=0x7f325806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3244003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32440047c0 0x7f3244004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32440051d0 0x7f324400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325815f4f0 con 0x7f32440039c0 2023-10-21T17:39:29.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324400c100 con 0x7f32440051d0 2023-10-21T17:39:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.309+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324400c2c0 con 0x7f32440047c0 2023-10-21T17:39:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3244003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 -- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32440047c0 msgr2=0x7f3244004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32440047c0 0x7f3244004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 -- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32440051d0 msgr2=0x7f324400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32440051d0 0x7f324400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f325659c640 1 -- 172.21.15.53:0/3035835568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244003520 con 0x7f32440039c0 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.310+0000 7f3256d9d640 1 -- 172.21.15.53:0/3035835568 reap_dead start 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.311+0000 7f325659c640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3244003dc0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f324800ba00 tx=0x7f32440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241e85280ef69de9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.311+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248032640 con 0x7f32440039c0 2023-10-21T17:39:29.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.311+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32480327a0 con 0x7f32440039c0 2023-10-21T17:39:29.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.311+0000 7f3255d9b640 1 -- 172.21.15.53:0/3035835568 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248032b30 con 0x7f32440039c0 2023-10-21T17:39:29.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 -- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 msgr2=0x7f3244003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3244003dc0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f324800ba00 tx=0x7f32440057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 -- 172.21.15.53:0/3035835568 shutdown_connections 2023-10-21T17:39:29.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 --2- 172.21.15.53:0/3035835568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3244003dc0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 -- 172.21.15.53:0/3035835568 >> 172.21.15.53:0/3035835568 conn(0x7f32580cb0c0 msgr2=0x7f32580cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 -- 172.21.15.53:0/3035835568 shutdown_connections 2023-10-21T17:39:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.312+0000 7f325759e640 1 -- 172.21.15.53:0/3035835568 wait complete. 2023-10-21T17:39:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.314+0000 7f325759e640 1 Processor -- start 2023-10-21T17:39:29.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.314+0000 7f325759e640 1 -- start start 2023-10-21T17:39:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 0x7f32580c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f325815c6e0 0x7f32580cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580cd590 con 0x7f32440039c0 2023-10-21T17:39:29.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580ac130 con 0x7f32580ccbf0 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580cd070 con 0x7f325815c6e0 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57726/0 (socket says 172.21.15.53:57726) 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.315+0000 7f325659c640 1 -- 172.21.15.53:0/334367233 learned_addr learned my addr 172.21.15.53:0/334367233 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f324ffff640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 0x7f32580c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 -- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f325815c6e0 msgr2=0x7f32580cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f325815c6e0 0x7f32580cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 -- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 msgr2=0x7f32580c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 0x7f32580c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 -- 172.21.15.53:0/334367233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580c7ba0 con 0x7f32440039c0 2023-10-21T17:39:29.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f324ffff640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 0x7f32580c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325659c640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f324802bec0 tx=0x7f3248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324803c9c0 con 0x7f32440039c0 2023-10-21T17:39:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324803cb40 con 0x7f32440039c0 2023-10-21T17:39:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248030410 con 0x7f32440039c0 2023-10-21T17:39:29.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.316+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32580cd2b0 con 0x7f32440039c0 2023-10-21T17:39:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.317+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32580abd50 con 0x7f32440039c0 2023-10-21T17:39:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.318+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32580abf10 con 0x7f32440039c0 2023-10-21T17:39:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.318+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3248030570 con 0x7f32440039c0 2023-10-21T17:39:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.319+0000 7f324effd640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 0x7f323806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.319+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f32480ac510 con 0x7f32440039c0 2023-10-21T17:39:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.320+0000 7f324ffff640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 0x7f323806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.321+0000 7f324ffff640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 0x7f323806d570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32580c4ee0 tx=0x7f3240009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.324+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3248076fa0 con 0x7f32440039c0 2023-10-21T17:39:29.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.586+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f32580ab8d0 con 0x7f32440039c0 2023-10-21T17:39:29.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.658+0000 7f324effd640 1 -- 172.21.15.53:0/334367233 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3215) v1 ==== 186+0+0 (secure 0 0 0) 0x7f32580ab8d0 con 0x7f32440039c0 2023-10-21T17:39:29.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.661+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 msgr2=0x7f323806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.661+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 0x7f323806d570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32580c4ee0 tx=0x7f3240009040 comp rx=0 tx=0).stop 2023-10-21T17:39:29.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.661+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 msgr2=0x7f3258157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.661+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f324802bec0 tx=0x7f3248004730 comp rx=0 tx=0).stop 2023-10-21T17:39:29.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 shutdown_connections 2023-10-21T17:39:29.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f323806b0b0 0x7f323806d570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f325815c6e0 0x7f32580cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32580ccbf0 0x7f32580c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 --2- 172.21.15.53:0/334367233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32440039c0 0x7f3258157d90 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 >> 172.21.15.53:0/334367233 conn(0x7f32580cb0c0 msgr2=0x7f32580aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:29.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 shutdown_connections 2023-10-21T17:39:29.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.662+0000 7f325759e640 1 -- 172.21.15.53:0/334367233 wait complete. 2023-10-21T17:39:29.683 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-10-21T17:39:29.683 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-10-21T17:39:29.683 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-21T17:39:29.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:29.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.903+0000 7ff6d3cc9640 1 Processor -- start 2023-10-21T17:39:29.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.904+0000 7ff6d3cc9640 1 -- start start 2023-10-21T17:39:29.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.905+0000 7ff6d3cc9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.905+0000 7ff6d3cc9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 0x7ff6cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.905+0000 7ff6d3cc9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc06a7e0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.905+0000 7ff6d3cc9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc15eb30 con 0x7ff6cc0cc130 2023-10-21T17:39:29.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.905+0000 7ff6d3cc9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cda90 con 0x7ff6cc161260 2023-10-21T17:39:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57728/0 (socket says 172.21.15.53:57728) 2023-10-21T17:39:29.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 -- 172.21.15.53:0/1629812758 learned_addr learned my addr 172.21.15.53:0/1629812758 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:29.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d2266640 1 --2- 172.21.15.53:0/1629812758 >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 0x7ff6cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 -- 172.21.15.53:0/1629812758 >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 msgr2=0x7ff6cc070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 --2- 172.21.15.53:0/1629812758 >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 0x7ff6cc070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 -- 172.21.15.53:0/1629812758 >> v1:172.21.15.124:6789/0 conn(0x7ff6cc0cbd50 legacy=0x7ff6cc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d1264640 1 -- 172.21.15.53:0/1629812758 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc06a190 con 0x7ff6cc0cc130 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.906+0000 7ff6d2266640 1 --2- 172.21.15.53:0/1629812758 >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 0x7ff6cc070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:29.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.907+0000 7ff6d1264640 1 --2- 172.21.15.53:0/1629812758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc1607b0 tx=0x7ff6c002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6c42cd077beebe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d0a63640 1 -- 172.21.15.53:0/1629812758 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00317e0 con 0x7ff6cc0cc130 2023-10-21T17:39:29.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d0a63640 1 -- 172.21.15.53:0/1629812758 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0031940 con 0x7ff6cc0cc130 2023-10-21T17:39:29.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d0a63640 1 -- 172.21.15.53:0/1629812758 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0031cd0 con 0x7ff6cc0cc130 2023-10-21T17:39:29.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1629812758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cc130 msgr2=0x7ff6cc0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1629812758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc1607b0 tx=0x7ff6c002b880 comp rx=0 tx=0).stop 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1629812758 shutdown_connections 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1629812758 >> v2:172.21.15.124:3300/0 conn(0x7ff6cc161260 0x7ff6cc070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1629812758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cc130 0x7ff6cc0cd4b0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1629812758 >> 172.21.15.53:0/1629812758 conn(0x7ff6cc0b1480 msgr2=0x7ff6cc06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1629812758 shutdown_connections 2023-10-21T17:39:29.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.908+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1629812758 wait complete. 2023-10-21T17:39:29.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.910+0000 7ff6d3cc9640 1 Processor -- start 2023-10-21T17:39:29.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.910+0000 7ff6d3cc9640 1 -- start start 2023-10-21T17:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6cc0cc130 0x7ff6cc0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 0x7ff6cc0ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ac5d0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b3350 con 0x7ff6cc161260 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d3cc9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b3630 con 0x7ff6cc0cc130 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.911+0000 7ff6d1a65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57732/0 (socket says 172.21.15.53:57732) 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 -- 172.21.15.53:0/1749140847 learned_addr learned my addr 172.21.15.53:0/1749140847 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1264640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6cc0cc130 0x7ff6cc0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 -- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6cc0cc130 msgr2=0x7ff6cc0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6cc0cc130 0x7ff6cc0b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 -- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 msgr2=0x7ff6cc0ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d2266640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 0x7ff6cc0ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 0x7ff6cc0ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 -- 172.21.15.53:0/1749140847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00094d0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d2266640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 0x7ff6cc0ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.912+0000 7ff6d1a65640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00ba50 tx=0x7ff6bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.913+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00cbe0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.913+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc00cd80 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.913+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc0b3910 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.913+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc014680 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.913+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc0b3ed0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.914+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc067490 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.915+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff6bc0147e0 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.916+0000 7ff6ca7fc640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 0x7ff6b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:29.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.916+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7ff6bc089b50 con 0x7ff6cc0cbd50 2023-10-21T17:39:29.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.916+0000 7ff6d1264640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 0x7ff6b806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:29.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.918+0000 7ff6d1264640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 0x7ff6b806d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0036000 tx=0x7ff6c003d040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:29.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:29.920+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6bc054670 con 0x7ff6cc0cbd50 2023-10-21T17:39:30.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.119+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6cc065c20 con 0x7ff6cc0cbd50 2023-10-21T17:39:30.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.121+0000 7ff6ca7fc640 1 -- 172.21.15.53:0/1749140847 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v245) v1 ==== 74+0+14711 (secure 0 0 0) 0x7ff6bc019020 con 0x7ff6cc0cbd50 2023-10-21T17:39:30.122 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:30.122 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":245,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:27.638987+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1125921177":"2023-10-22T17:39:24.393371+0000","172.21.15.124:6837/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2382130374":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6833/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/3892957852":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1730889116":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2972397739":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2560598913":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2025087542":"2023-10-22T17:39:10.491716+0000","172.21.15.53:6834/1125921177":"2023-10-22T17:39:24.393371+0000","172.21.15.124:0/758149659":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/2786131718":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/583433106":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6833/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6836/748765178":"2023-10-22T17:39:16.504297+0000","172.21.15.124:0/2150821102":"2023-10-22T17:39:10.491716+0000","172.21.15.124:6832/3930748558":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4019636868":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/2979161442":"2023-10-22T17:38:59.709844+0000","172.21.15.124:6832/3186075298":"2023-10-22T17:39:10.491716+0000","172.21.15.124:0/1830851898":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/4191579832":"2023-10-22T17:38:59.709844+0000","172.21.15.124:0/167361821":"2023-10-22T17:38:59.709844+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.123+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 msgr2=0x7ff6b806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.123+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 0x7ff6b806d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0036000 tx=0x7ff6c003d040 comp rx=0 tx=0).stop 2023-10-21T17:39:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.123+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 msgr2=0x7ff6cc0ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.123+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00ba50 tx=0x7ff6bc00bf10 comp rx=0 tx=0).stop 2023-10-21T17:39:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 shutdown_connections 2023-10-21T17:39:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff6b806b0b0 0x7ff6b806d570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6cc161260 0x7ff6cc0ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6cc0cc130 0x7ff6cc0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 --2- 172.21.15.53:0/1749140847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6cc0cbd50 0x7ff6cc0ac090 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 >> 172.21.15.53:0/1749140847 conn(0x7ff6cc0b1480 msgr2=0x7ff6cc15fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 shutdown_connections 2023-10-21T17:39:30.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.124+0000 7ff6d3cc9640 1 -- 172.21.15.53:0/1749140847 wait complete. 2023-10-21T17:39:30.146 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:30.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:30.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.377+0000 7fd3da2b4640 1 Processor -- start 2023-10-21T17:39:30.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.377+0000 7fd3da2b4640 1 -- start start 2023-10-21T17:39:30.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.378+0000 7fd3da2b4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd3d415aab0 0x7fd3d4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.378+0000 7fd3da2b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3d40aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.378+0000 7fd3da2b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d415ae90 con 0x7fd3d40cb350 2023-10-21T17:39:30.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.378+0000 7fd3da2b4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d40cbdb0 con 0x7fd3d415aab0 2023-10-21T17:39:30.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.378+0000 7fd3da2b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40cb7f0 con 0x7fd3d40caef0 2023-10-21T17:39:30.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3d9ab3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd3d40cb350 0x7fd3d40c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47782/0 (socket says 172.21.15.53:47782) 2023-10-21T17:39:30.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/3238559599 learned_addr learned my addr 172.21.15.53:0/3238559599 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:30.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/3238559599 >> v2:172.21.15.124:3300/0 conn(0x7fd3d415aab0 0x7fd3d4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3d40aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110549164 0 0) 0x7fd3cc001490 con 0x7fd3d40cb350 2023-10-21T17:39:30.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.379+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3c0003520 con 0x7fd3d40cb350 2023-10-21T17:39:30.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223470624 0 0) 0x7fd3c0003520 con 0x7fd3d40cb350 2023-10-21T17:39:30.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 msgr2=0x7fd3d40aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:30.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3d40aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 >> v2:172.21.15.124:3300/0 conn(0x7fd3d415aab0 msgr2=0x7fd3d4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 --2- 172.21.15.53:0/3238559599 >> v2:172.21.15.124:3300/0 conn(0x7fd3d415aab0 0x7fd3d4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40cbad0 con 0x7fd3d40cb350 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd3cc001b80 con 0x7fd3d40cb350 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 >> v1:172.21.15.53:6789/0 conn(0x7fd3d40cb350 legacy=0x7fd3d40c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c0003970 0x7fd3c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c00047c0 0x7fd3c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40cbdb0 con 0x7fd3c00047c0 2023-10-21T17:39:30.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40cb7f0 con 0x7fd3c0003970 2023-10-21T17:39:30.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c000b9b0 con 0x7fd3d40caef0 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.380+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c00047c0 0x7fd3c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c0003970 0x7fd3c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d9ab3640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d8ab1640 1 -- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 msgr2=0x7fd3c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d8ab1640 1 -- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c0003970 msgr2=0x7fd3c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:30.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c0003970 0x7fd3c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d8ab1640 1 -- 172.21.15.53:0/3238559599 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0003520 con 0x7fd3c00047c0 2023-10-21T17:39:30.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d9ab3640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3d40caef0 0x7fd3c0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:30.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.381+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c0003970 0x7fd3c0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.382+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c00047c0 0x7fd3c0004c40 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ac0 tx=0x7fd3c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f227b5705178a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c402c6b0 con 0x7fd3c00047c0 2023-10-21T17:39:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c402c810 con 0x7fd3c00047c0 2023-10-21T17:39:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3cbfff640 1 -- 172.21.15.53:0/3238559599 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c402cba0 con 0x7fd3c00047c0 2023-10-21T17:39:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3da2b4640 1 -- 172.21.15.53:0/3238559599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c00047c0 msgr2=0x7fd3c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/3238559599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3c00047c0 0x7fd3c0004c40 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009ac0 tx=0x7fd3c00057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/3238559599 reap_dead start 2023-10-21T17:39:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3da2b4640 1 -- 172.21.15.53:0/3238559599 shutdown_connections 2023-10-21T17:39:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.383+0000 7fd3da2b4640 1 -- 172.21.15.53:0/3238559599 >> 172.21.15.53:0/3238559599 conn(0x7fd3d406c010 msgr2=0x7fd3d406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.384+0000 7fd3da2b4640 1 -- 172.21.15.53:0/3238559599 shutdown_connections 2023-10-21T17:39:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.384+0000 7fd3da2b4640 1 -- 172.21.15.53:0/3238559599 wait complete. 2023-10-21T17:39:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.385+0000 7fd3da2b4640 1 Processor -- start 2023-10-21T17:39:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.386+0000 7fd3da2b4640 1 -- start start 2023-10-21T17:39:30.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c00047c0 0x7fd3d4150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3d40cea40 0x7fd3d40ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41f3440 con 0x7fd3d40c7780 2023-10-21T17:39:30.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40c8140 con 0x7fd3d40cea40 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3da2b4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d41f27c0 con 0x7fd3c00047c0 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3d9ab3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3d9ab3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57754/0 (socket says 172.21.15.53:57754) 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/4259311868 learned_addr learned my addr 172.21.15.53:0/4259311868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.387+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c00047c0 0x7fd3d4150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d8ab1640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3d40cea40 0x7fd3d40ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c00047c0 msgr2=0x7fd3d4150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c00047c0 0x7fd3d4150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3d40cea40 msgr2=0x7fd3d40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3d40cea40 0x7fd3d40ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 -- 172.21.15.53:0/4259311868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d41f2aa0 con 0x7fd3d40c7780 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3d9ab3640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40cf590 tx=0x7fd3cc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004d60 con 0x7fd3d40c7780 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc00bb80 con 0x7fd3d40c7780 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc007070 con 0x7fd3d40c7780 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d41f2d80 con 0x7fd3d40c7780 2023-10-21T17:39:30.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.388+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d4150db0 con 0x7fd3d40c7780 2023-10-21T17:39:30.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.390+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d40aef40 con 0x7fd3d40c7780 2023-10-21T17:39:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.390+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd3cc0071d0 con 0x7fd3d40c7780 2023-10-21T17:39:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.391+0000 7fd3c9ffb640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 0x7fd3b806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.392+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fd3cc089ee0 con 0x7fd3d40c7780 2023-10-21T17:39:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.392+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 0x7fd3b806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.393+0000 7fd3d92b2640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 0x7fd3b806d5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009b10 tx=0x7fd3d0008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.396+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3d40aef40 con 0x7fd3d40c7780 2023-10-21T17:39:30.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.591+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3d415ae90 con 0x7fd3d40c7780 2023-10-21T17:39:30.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.592+0000 7fd3c9ffb640 1 -- 172.21.15.53:0/4259311868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v245) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3d415ae90 con 0x7fd3d40c7780 2023-10-21T17:39:30.593 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:30.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 msgr2=0x7fd3b806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 0x7fd3b806d5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009b10 tx=0x7fd3d0008040 comp rx=0 tx=0).stop 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 msgr2=0x7fd3d40c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40cf590 tx=0x7fd3cc00b4a0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 shutdown_connections 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd3b806b0e0 0x7fd3b806d5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3d40c7780 0x7fd3d40c7c00 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3d40cea40 0x7fd3d40ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 --2- 172.21.15.53:0/4259311868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c00047c0 0x7fd3d4150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 >> 172.21.15.53:0/4259311868 conn(0x7fd3d406c010 msgr2=0x7fd3d415cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 shutdown_connections 2023-10-21T17:39:30.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.595+0000 7fd3da2b4640 1 -- 172.21.15.53:0/4259311868 wait complete. 2023-10-21T17:39:30.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-21T17:39:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.834+0000 7fcc5a62d640 1 Processor -- start 2023-10-21T17:39:30.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.834+0000 7fcc5a62d640 1 -- start start 2023-10-21T17:39:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc5a62d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc540c90b0 0x7fcc540cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc5a62d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5415b2b0 con 0x7fcc540cb3b0 2023-10-21T17:39:30.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc5a62d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc5415b590 con 0x7fcc5415ab40 2023-10-21T17:39:30.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc5a62d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5415bf30 con 0x7fcc540c90b0 2023-10-21T17:39:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc58bca640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcc540cb3b0 0x7fcc5415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47794/0 (socket says 172.21.15.53:47794) 2023-10-21T17:39:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc58bca640 1 -- 172.21.15.53:0/2924122045 learned_addr learned my addr 172.21.15.53:0/2924122045 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.836+0000 7fcc537fe640 1 --2- 172.21.15.53:0/2924122045 >> v2:172.21.15.124:3300/0 conn(0x7fcc540c90b0 0x7fcc540cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181417622 0 0) 0x7fcc5415b2b0 con 0x7fcc540cb3b0 2023-10-21T17:39:30.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc3c003520 con 0x7fcc540cb3b0 2023-10-21T17:39:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc537fe640 1 -- 172.21.15.53:0/2924122045 >> v1:172.21.15.124:6789/0 conn(0x7fcc5415ab40 legacy=0x7fcc540ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1653329968 0 0) 0x7fcc3c003520 con 0x7fcc540cb3b0 2023-10-21T17:39:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc537fe640 1 -- 172.21.15.53:0/2924122045 >> v1:172.21.15.53:6789/0 conn(0x7fcc540cb3b0 legacy=0x7fcc5415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.837+0000 7fcc537fe640 1 -- 172.21.15.53:0/2924122045 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5415b870 con 0x7fcc540c90b0 2023-10-21T17:39:30.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.839+0000 7fcc537fe640 1 --2- 172.21.15.53:0/2924122045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540c90b0 0x7fcc540cae70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097a0 tx=0x7fcc4802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6924aa94a7e50085 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.839+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4802f050 con 0x7fcc540c90b0 2023-10-21T17:39:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.839+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48031500 con 0x7fcc540c90b0 2023-10-21T17:39:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.839+0000 7fcc52ffd640 1 -- 172.21.15.53:0/2924122045 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0019f0 con 0x7fcc540c90b0 2023-10-21T17:39:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 -- 172.21.15.53:0/2924122045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540c90b0 msgr2=0x7fcc540cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/2924122045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540c90b0 0x7fcc540cae70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fcc480097a0 tx=0x7fcc4802b820 comp rx=0 tx=0).stop 2023-10-21T17:39:30.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 -- 172.21.15.53:0/2924122045 shutdown_connections 2023-10-21T17:39:30.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/2924122045 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540c90b0 0x7fcc540cae70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 -- 172.21.15.53:0/2924122045 >> 172.21.15.53:0/2924122045 conn(0x7fcc5406a380 msgr2=0x7fcc540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:30.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 -- 172.21.15.53:0/2924122045 shutdown_connections 2023-10-21T17:39:30.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.840+0000 7fcc5a62d640 1 -- 172.21.15.53:0/2924122045 wait complete. 2023-10-21T17:39:30.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.842+0000 7fcc5a62d640 1 Processor -- start 2023-10-21T17:39:30.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.842+0000 7fcc5a62d640 1 -- start start 2023-10-21T17:39:30.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 0x7fcc541f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540cb3b0 0x7fcc5415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc541f3410 con 0x7fcc5415ab40 2023-10-21T17:39:30.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540c8590 con 0x7fcc540cb3b0 2023-10-21T17:39:30.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc5a62d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540abd10 con 0x7fcc540c90b0 2023-10-21T17:39:30.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc58bca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc58bca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57768/0 (socket says 172.21.15.53:57768) 2023-10-21T17:39:30.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.843+0000 7fcc58bca640 1 -- 172.21.15.53:0/1466749560 learned_addr learned my addr 172.21.15.53:0/1466749560 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:30.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc53fff640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 0x7fcc541f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc537fe640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540cb3b0 0x7fcc5415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 -- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 msgr2=0x7fcc541f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 0x7fcc541f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 -- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540cb3b0 msgr2=0x7fcc5415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:30.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540cb3b0 0x7fcc5415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:30.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 -- 172.21.15.53:0/1466749560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc48009400 con 0x7fcc5415ab40 2023-10-21T17:39:30.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc53fff640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 0x7fcc541f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:30.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc58bca640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c001f70 tx=0x7fcc4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c012400 con 0x7fcc5415ab40 2023-10-21T17:39:30.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c012560 con 0x7fcc5415ab40 2023-10-21T17:39:30.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc541557d0 con 0x7fcc5415ab40 2023-10-21T17:39:30.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.844+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c012910 con 0x7fcc5415ab40 2023-10-21T17:39:30.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.845+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc54155d90 con 0x7fcc5415ab40 2023-10-21T17:39:30.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.846+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc54067490 con 0x7fcc5415ab40 2023-10-21T17:39:30.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.847+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcc4c012a70 con 0x7fcc5415ab40 2023-10-21T17:39:30.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.848+0000 7fcc50ff9640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 0x7fcc4006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:30.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.848+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fcc4c08a470 con 0x7fcc5415ab40 2023-10-21T17:39:30.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.848+0000 7fcc53fff640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 0x7fcc4006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:30.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.850+0000 7fcc53fff640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 0x7fcc4006d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc54154d70 tx=0x7fcc44009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:30.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:30.852+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc4c054f70 con 0x7fcc5415ab40 2023-10-21T17:39:31.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.116+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fcc540b0ef0 con 0x7fcc5415ab40 2023-10-21T17:39:31.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.672+0000 7fcc50ff9640 1 -- 172.21.15.53:0/1466749560 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3217) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcc4c058590 con 0x7fcc5415ab40 2023-10-21T17:39:31.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 msgr2=0x7fcc4006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 0x7fcc4006d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc54154d70 tx=0x7fcc44009040 comp rx=0 tx=0).stop 2023-10-21T17:39:31.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 msgr2=0x7fcc541f1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c001f70 tx=0x7fcc4c00a430 comp rx=0 tx=0).stop 2023-10-21T17:39:31.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 shutdown_connections 2023-10-21T17:39:31.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fcc4006b0b0 0x7fcc4006d570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc5415ab40 0x7fcc541f1cb0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc540cb3b0 0x7fcc5415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 --2- 172.21.15.53:0/1466749560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc540c90b0 0x7fcc541f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 >> 172.21.15.53:0/1466749560 conn(0x7fcc5406a380 msgr2=0x7fcc5406cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:31.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.675+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 shutdown_connections 2023-10-21T17:39:31.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.676+0000 7fcc5a62d640 1 -- 172.21.15.53:0/1466749560 wait complete. 2023-10-21T17:39:31.698 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:39:31.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.921+0000 7f561681a640 1 Processor -- start 2023-10-21T17:39:31.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.922+0000 7f561681a640 1 -- start start 2023-10-21T17:39:31.924 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.923+0000 7f561681a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56100cb5e0 0x7f56100cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.923+0000 7f561681a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56100b3c50 0x7f56100cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.923+0000 7f561681a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5610159db0 con 0x7f56101599d0 2023-10-21T17:39:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.923+0000 7f561681a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5610071fe0 con 0x7f56100cb5e0 2023-10-21T17:39:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.923+0000 7f561681a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56100b44c0 con 0x7f56100b3c50 2023-10-21T17:39:31.925 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615818640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f56101599d0 0x7f56100caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47796/0 (socket says 172.21.15.53:47796) 2023-10-21T17:39:31.926 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615818640 1 -- 172.21.15.53:0/1472082140 learned_addr learned my addr 172.21.15.53:0/1472082140 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615017640 1 --2- 172.21.15.53:0/1472082140 >> v2:172.21.15.53:3300/0 conn(0x7f56100cb5e0 0x7f56100cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5616019640 1 --2- 172.21.15.53:0/1472082140 >> v2:172.21.15.124:3300/0 conn(0x7f56100b3c50 0x7f56100cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5614816640 1 -- 172.21.15.53:0/1472082140 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695253656 0 0) 0x7f560c001110 con 0x7f56101599d0 2023-10-21T17:39:31.927 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5614816640 1 -- 172.21.15.53:0/1472082140 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55fc005730 con 0x7f56101599d0 2023-10-21T17:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615017640 1 -- 172.21.15.53:0/1472082140 >> v2:172.21.15.124:3300/0 conn(0x7f56100b3c50 msgr2=0x7f56100cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615017640 1 --2- 172.21.15.53:0/1472082140 >> v2:172.21.15.124:3300/0 conn(0x7f56100b3c50 0x7f56100cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.924+0000 7f5615017640 1 -- 172.21.15.53:0/1472082140 >> v1:172.21.15.53:6789/0 conn(0x7f56101599d0 legacy=0x7f56100caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.925+0000 7f5615017640 1 -- 172.21.15.53:0/1472082140 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56100713d0 con 0x7f56100cb5e0 2023-10-21T17:39:31.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f5615017640 1 --2- 172.21.15.53:0/1472082140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100cb5e0 0x7f56100cdcd0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f56100cd830 tx=0x7f560002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f3a320ccea8b01 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f5614816640 1 -- 172.21.15.53:0/1472082140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600031830 con 0x7f56100cb5e0 2023-10-21T17:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f5614816640 1 -- 172.21.15.53:0/1472082140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600031990 con 0x7f56100cb5e0 2023-10-21T17:39:31.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f5614816640 1 -- 172.21.15.53:0/1472082140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600031d20 con 0x7f56100cb5e0 2023-10-21T17:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f561681a640 1 -- 172.21.15.53:0/1472082140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100cb5e0 msgr2=0x7f56100cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.926+0000 7f561681a640 1 --2- 172.21.15.53:0/1472082140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100cb5e0 0x7f56100cdcd0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f56100cd830 tx=0x7f560002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 -- 172.21.15.53:0/1472082140 shutdown_connections 2023-10-21T17:39:31.930 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 --2- 172.21.15.53:0/1472082140 >> v2:172.21.15.124:3300/0 conn(0x7f56100b3c50 0x7f56100cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 --2- 172.21.15.53:0/1472082140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100cb5e0 0x7f56100cdcd0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 -- 172.21.15.53:0/1472082140 >> 172.21.15.53:0/1472082140 conn(0x7f561006c050 msgr2=0x7f561006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 -- 172.21.15.53:0/1472082140 shutdown_connections 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.927+0000 7f561681a640 1 -- 172.21.15.53:0/1472082140 wait complete. 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.928+0000 7f561681a640 1 Processor -- start 2023-10-21T17:39:31.931 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.929+0000 7f561681a640 1 -- start start 2023-10-21T17:39:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 0x7f561015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 0x7f561015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56101f1a10 con 0x7f56100b3c50 2023-10-21T17:39:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5610141330 con 0x7f56101599d0 2023-10-21T17:39:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f561681a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5610141050 con 0x7f56100cb5e0 2023-10-21T17:39:31.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57790/0 (socket says 172.21.15.53:57790) 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 -- 172.21.15.53:0/3012955542 learned_addr learned my addr 172.21.15.53:0/3012955542 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5616019640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 0x7f561015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615017640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 0x7f561015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 -- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 msgr2=0x7f561015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 0x7f561015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 -- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 msgr2=0x7f561015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 0x7f561015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615818640 1 -- 172.21.15.53:0/3012955542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009400 con 0x7f56100b3c50 2023-10-21T17:39:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5615017640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 0x7f561015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:31.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.930+0000 7f5616019640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 0x7f561015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f5615818640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c700 tx=0x7f560c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c005ce0 con 0x7f56100b3c50 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c005e40 con 0x7f56100b3c50 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c007290 con 0x7f56100b3c50 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56101f1350 con 0x7f56100b3c50 2023-10-21T17:39:31.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.931+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5610143f50 con 0x7f56100b3c50 2023-10-21T17:39:31.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.932+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56101f11b0 con 0x7f56100b3c50 2023-10-21T17:39:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.933+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f560c0074b0 con 0x7f56100b3c50 2023-10-21T17:39:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.935+0000 7f56067fc640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 0x7f55f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.935+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f560c089120 con 0x7f56100b3c50 2023-10-21T17:39:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.935+0000 7f5615017640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 0x7f55f406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:31.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.936+0000 7f5615017640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 0x7f55f406d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5600036000 tx=0x7f5600008030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:31.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:31.938+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56101f11b0 con 0x7f56100b3c50 2023-10-21T17:39:32.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.133+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5610159db0 con 0x7f56100b3c50 2023-10-21T17:39:32.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.139+0000 7f56067fc640 1 -- 172.21.15.53:0/3012955542 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5610159db0 con 0x7f56100b3c50 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 msgr2=0x7f55f406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 0x7f55f406d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5600036000 tx=0x7f5600008030 comp rx=0 tx=0).stop 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 msgr2=0x7f5610143a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c700 tx=0x7f560c00cb00 comp rx=0 tx=0).stop 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 shutdown_connections 2023-10-21T17:39:32.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f55f406b0b0 0x7f55f406d570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56101599d0 0x7f561015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56100cb5e0 0x7f561015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 --2- 172.21.15.53:0/3012955542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56100b3c50 0x7f5610143a10 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 >> 172.21.15.53:0/3012955542 conn(0x7f561006c050 msgr2=0x7f561015c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:32.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 shutdown_connections 2023-10-21T17:39:32.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.142+0000 7f561681a640 1 -- 172.21.15.53:0/3012955542 wait complete. 2023-10-21T17:39:32.162 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:39:32.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:39:32.284 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:39:32.409 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:39:32.409 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:32.410 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:32.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:39:32.473 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:39:32.474 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:32.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:39:32.535 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:32.536 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:32.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:39:32.600 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:39:32.600 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:32.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:39:32.660 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:39:32.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:32.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.895+0000 7fa5526df640 1 Processor -- start 2023-10-21T17:39:32.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.896+0000 7fa5526df640 1 -- start start 2023-10-21T17:39:32.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.897+0000 7fa5526df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.897+0000 7fa5526df640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa54c0ce6e0 0x7fa54c0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.897+0000 7fa5526df640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c15d040 con 0x7fa54c0d16f0 2023-10-21T17:39:32.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.897+0000 7fa5526df640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c0d0650 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.897+0000 7fa5526df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0d0930 con 0x7fa54c15cc60 2023-10-21T17:39:32.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.898+0000 7fa551ede640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa54c0ce6e0 0x7fa54c0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.898+0000 7fa5516dd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.898+0000 7fa551ede640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa54c0ce6e0 0x7fa54c0cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:51598/0 (socket says 172.21.15.53:51598) 2023-10-21T17:39:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.898+0000 7fa550edc640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fa54c0d16f0 0x7fa54c0cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:48730/0 (socket says 172.21.15.53:48730) 2023-10-21T17:39:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.898+0000 7fa551ede640 1 -- 172.21.15.53:0/2548602704 learned_addr learned my addr 172.21.15.53:0/2548602704 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.899+0000 7fa551ede640 1 -- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 msgr2=0x7fa54c0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:32.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.899+0000 7fa551ede640 1 --2- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.899+0000 7fa551ede640 1 -- 172.21.15.53:0/2548602704 >> v1:172.21.15.124:6789/0 conn(0x7fa54c0d16f0 legacy=0x7fa54c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.899+0000 7fa551ede640 1 -- 172.21.15.53:0/2548602704 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0d0c10 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.900+0000 7fa551ede640 1 --2- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c0cf880 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c4e51a2a2a9a79 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:32.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.901+0000 7fa543fff640 1 -- 172.21.15.53:0/2548602704 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54402cb00 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.901+0000 7fa543fff640 1 -- 172.21.15.53:0/2548602704 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54402cc60 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.901+0000 7fa543fff640 1 -- 172.21.15.53:0/2548602704 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440343f0 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.901+0000 7fa5526df640 1 -- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 msgr2=0x7fa54c0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.901+0000 7fa5526df640 1 --2- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c0cf880 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54402bf60 comp rx=0 tx=0).stop 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 -- 172.21.15.53:0/2548602704 shutdown_connections 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 --2- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c0cf880 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 --2- 172.21.15.53:0/2548602704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c0cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 -- 172.21.15.53:0/2548602704 >> 172.21.15.53:0/2548602704 conn(0x7fa54c09bcf0 msgr2=0x7fa54c0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:32.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 -- 172.21.15.53:0/2548602704 shutdown_connections 2023-10-21T17:39:32.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.902+0000 7fa5526df640 1 -- 172.21.15.53:0/2548602704 wait complete. 2023-10-21T17:39:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.903+0000 7fa5526df640 1 Processor -- start 2023-10-21T17:39:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 -- start start 2023-10-21T17:39:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c14f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c154b00 con 0x7fa54c0d16f0 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1560d0 con 0x7fa54c0ce6e0 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.904+0000 7fa5526df640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c15b330 con 0x7fa54c15cc60 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57792/0 (socket says 172.21.15.53:57792) 2023-10-21T17:39:32.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 -- 172.21.15.53:0/4036745022 learned_addr learned my addr 172.21.15.53:0/4036745022 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa5516dd640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c14f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa551ede640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 -- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 msgr2=0x7fa54c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 -- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 msgr2=0x7fa54c14f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:32.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c14f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:32.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa550edc640 1 -- 172.21.15.53:0/4036745022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544009b30 con 0x7fa54c0d16f0 2023-10-21T17:39:32.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa551ede640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:32.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.905+0000 7fa5516dd640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c14f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:32.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa550edc640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c410 tx=0x7fa53c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:32.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0074e0 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c007640 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c154540 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0049e0 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.906+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c15afc0 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.907+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c15ae20 con 0x7fa54c0d16f0 2023-10-21T17:39:32.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.908+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa53c004b40 con 0x7fa54c0d16f0 2023-10-21T17:39:32.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.909+0000 7fa541ffb640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:32.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.909+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fa53c089700 con 0x7fa54c0d16f0 2023-10-21T17:39:32.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.910+0000 7fa5516dd640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:32.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.911+0000 7fa5516dd640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 0x7fa53006d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa54c160400 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:32.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:32.913+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa54c15ae20 con 0x7fa54c0d16f0 2023-10-21T17:39:33.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.146+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa54c15ae20 con 0x7fa54c0d16f0 2023-10-21T17:39:33.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.147+0000 7fa541ffb640 1 -- 172.21.15.53:0/4036745022 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+3709 (secure 0 0 0) 0x7fa54c15ae20 con 0x7fa54c0d16f0 2023-10-21T17:39:33.150 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:33.150 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9112,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1761722692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1761722692},{"type":"v1","addr":"172.21.15.124:6835","nonce":1761722692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":9490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1533795007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":1533795007},{"type":"v1","addr":"172.21.15.124:6837","nonce":1533795007}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":11145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/430867761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":430867761},{"type":"v1","addr":"172.21.15.53:6833","nonce":430867761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":11847,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2878295332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2878295332},{"type":"v1","addr":"172.21.15.53:6835","nonce":2878295332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[]} 2023-10-21T17:39:33.150 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 122 2023-10-21T17:39:33.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 msgr2=0x7fa53006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 0x7fa53006d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa54c160400 tx=0x7fa548008040 comp rx=0 tx=0).stop 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 msgr2=0x7fa54c14f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00c410 tx=0x7fa53c00c8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 shutdown_connections 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fa53006b0b0 0x7fa53006d570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa54c15cc60 0x7fa54c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa54c0d16f0 0x7fa54c14f9f0 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 --2- 172.21.15.53:0/4036745022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa54c0ce6e0 0x7fa54c14f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 >> 172.21.15.53:0/4036745022 conn(0x7fa54c09bcf0 msgr2=0x7fa54c0b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 shutdown_connections 2023-10-21T17:39:33.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.150+0000 7fa5526df640 1 -- 172.21.15.53:0/4036745022 wait complete. 2023-10-21T17:39:33.171 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:39:33.172 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:39:33.172 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:39:33.172 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:39:33.173 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:39:33.173 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:39:33.173 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:39:33.173 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:39:33.174 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:39:33.174 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:39:33.174 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:39:33.174 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:39:33.175 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:39:33.175 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:39:33.175 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:39:33.175 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:39:33.191 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:39:33.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:39:33.192 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:39:33.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:39:33.199 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:39:33.199 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:39:33.200 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:39:33.200 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:39:33.240 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:39:33.241 DEBUG:teuthology.parallel:result is None 2023-10-21T17:39:33.247 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:39:33.248 DEBUG:teuthology.parallel:result is None 2023-10-21T17:39:33.248 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:39:33.249 DEBUG:teuthology.parallel:result is None 2023-10-21T17:39:33.256 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:39:33.257 DEBUG:teuthology.parallel:result is None 2023-10-21T17:39:33.257 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:39:33.407 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:39:33.410 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:39:33.415 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:39:33.417 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:39:33.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.446+0000 7fc7b8bfc640 1 Processor -- start 2023-10-21T17:39:33.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.446+0000 7fc7b8bfc640 1 -- start start 2023-10-21T17:39:33.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b8bfc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 0x7fc7b40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b8bfc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc7b406db60 0x7fc7b406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b8bfc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b415b310 con 0x7fc7b415aa00 2023-10-21T17:39:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b8bfc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b40ca230 con 0x7fc7b406db60 2023-10-21T17:39:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b8bfc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b406d290 con 0x7fc7b406c3a0 2023-10-21T17:39:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc7b415aa00 0x7fc7b40b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47820/0 (socket says 172.21.15.53:47820) 2023-10-21T17:39:33.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b2d9d640 1 -- 172.21.15.53:0/686831609 learned_addr learned my addr 172.21.15.53:0/686831609 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7abfff640 1 --2- 172.21.15.53:0/686831609 >> v2:172.21.15.124:3300/0 conn(0x7fc7b406db60 0x7fc7b406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.447+0000 7fc7b259c640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 0x7fc7b40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7b1d9b640 1 -- 172.21.15.53:0/686831609 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778637569 0 0) 0x7fc7ac001490 con 0x7fc7b415aa00 2023-10-21T17:39:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7b1d9b640 1 -- 172.21.15.53:0/686831609 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a0003520 con 0x7fc7b415aa00 2023-10-21T17:39:33.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7abfff640 1 -- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 msgr2=0x7fc7b40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7abfff640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 0x7fc7b40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7abfff640 1 -- 172.21.15.53:0/686831609 >> v1:172.21.15.53:6789/0 conn(0x7fc7b415aa00 legacy=0x7fc7b40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7abfff640 1 -- 172.21.15.53:0/686831609 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b40c9860 con 0x7fc7b406db60 2023-10-21T17:39:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.448+0000 7fc7b259c640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 0x7fc7b40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:39:33.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.449+0000 7fc7abfff640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406db60 0x7fc7b406c780 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3aa0aa532a0002 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:33.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b1d9b640 1 -- 172.21.15.53:0/686831609 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c02d040 con 0x7fc7b406db60 2023-10-21T17:39:33.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b1d9b640 1 -- 172.21.15.53:0/686831609 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c031810 con 0x7fc7b406db60 2023-10-21T17:39:33.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b1d9b640 1 -- 172.21.15.53:0/686831609 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c031ba0 con 0x7fc7b406db60 2023-10-21T17:39:33.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406db60 msgr2=0x7fc7b406c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406db60 0x7fc7b406c780 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0097a0 tx=0x7fc79c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/686831609 shutdown_connections 2023-10-21T17:39:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406db60 0x7fc7b406c780 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/686831609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406c3a0 0x7fc7b40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/686831609 >> 172.21.15.53:0/686831609 conn(0x7fc7b406a5d0 msgr2=0x7fc7b40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/686831609 shutdown_connections 2023-10-21T17:39:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.450+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/686831609 wait complete. 2023-10-21T17:39:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.452+0000 7fc7b8bfc640 1 Processor -- start 2023-10-21T17:39:33.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.452+0000 7fc7b8bfc640 1 -- start start 2023-10-21T17:39:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406c3a0 0x7fc7b414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406db60 0x7fc7b414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40c7440 con 0x7fc7b415aa00 2023-10-21T17:39:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b414c480 con 0x7fc7b406c3a0 2023-10-21T17:39:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b8bfc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40c7690 con 0x7fc7b406db60 2023-10-21T17:39:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57824/0 (socket says 172.21.15.53:57824) 2023-10-21T17:39:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.453+0000 7fc7b2d9d640 1 -- 172.21.15.53:0/3127966192 learned_addr learned my addr 172.21.15.53:0/3127966192 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:33.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b259c640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406c3a0 0x7fc7b414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7abfff640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406db60 0x7fc7b414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 -- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406db60 msgr2=0x7fc7b414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406db60 0x7fc7b414bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 -- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406c3a0 msgr2=0x7fc7b414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:33.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406c3a0 0x7fc7b414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:33.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 -- 172.21.15.53:0/3127966192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009400 con 0x7fc7b415aa00 2023-10-21T17:39:33.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b2d9d640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00cae0 tx=0x7fc7ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:33.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00ac40 con 0x7fc7b415aa00 2023-10-21T17:39:33.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00ada0 con 0x7fc7b415aa00 2023-10-21T17:39:33.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac012620 con 0x7fc7b415aa00 2023-10-21T17:39:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c009400 con 0x7fc7b415aa00 2023-10-21T17:39:33.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.454+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b40c7c50 con 0x7fc7b415aa00 2023-10-21T17:39:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.456+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b404e160 con 0x7fc7b415aa00 2023-10-21T17:39:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.457+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc7ac007070 con 0x7fc7b415aa00 2023-10-21T17:39:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.458+0000 7fc7aaffd640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 0x7fc79406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.458+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc7ac089ac0 con 0x7fc7b415aa00 2023-10-21T17:39:33.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.458+0000 7fc7b259c640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 0x7fc79406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.459+0000 7fc7b259c640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 0x7fc79406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41549e0 tx=0x7fc7a4008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:33.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.462+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7ac058060 con 0x7fc7b415aa00 2023-10-21T17:39:33.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:33.668+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc7b415b310 con 0x7fc7b415aa00 2023-10-21T17:39:35.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.180+0000 7fc7aaffd640 1 -- 172.21.15.53:0/3127966192 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v250) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc7b415b310 con 0x7fc7b415aa00 2023-10-21T17:39:35.183 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:39:35.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 msgr2=0x7fc79406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 0x7fc79406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7b41549e0 tx=0x7fc7a4008040 comp rx=0 tx=0).stop 2023-10-21T17:39:35.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 msgr2=0x7fc7b4153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00cae0 tx=0x7fc7ac00a060 comp rx=0 tx=0).stop 2023-10-21T17:39:35.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 shutdown_connections 2023-10-21T17:39:35.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc79406b0b0 0x7fc79406d570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7b415aa00 0x7fc7b4153b90 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7b406db60 0x7fc7b414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 --2- 172.21.15.53:0/3127966192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7b406c3a0 0x7fc7b414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.184+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 >> 172.21.15.53:0/3127966192 conn(0x7fc7b406a5d0 msgr2=0x7fc7b40cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:35.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.185+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 shutdown_connections 2023-10-21T17:39:35.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.185+0000 7fc7b8bfc640 1 -- 172.21.15.53:0/3127966192 wait complete. 2023-10-21T17:39:35.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:39:35.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.420+0000 7f8fad1b0640 1 Processor -- start 2023-10-21T17:39:35.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.420+0000 7f8fad1b0640 1 -- start start 2023-10-21T17:39:35.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fad1b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8fa8164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fad1b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa806dee0 con 0x7f8fa8159ac0 2023-10-21T17:39:35.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fad1b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa815d850 con 0x7f8fa80ad790 2023-10-21T17:39:35.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fad1b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa806dcf0 con 0x7f8fa80709e0 2023-10-21T17:39:35.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fa6d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8fa8159ac0 0x7f8fa806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55778/0 (socket says 172.21.15.53:55778) 2023-10-21T17:39:35.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fa6d9d640 1 -- 172.21.15.53:0/3008459318 learned_addr learned my addr 172.21.15.53:0/3008459318 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:35.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973829536 0 0) 0x7f8fa806dee0 con 0x7f8fa8159ac0 2023-10-21T17:39:35.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f90005730 con 0x7f8fa8159ac0 2023-10-21T17:39:35.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.422+0000 7f8fa759e640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8fa8164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637150218 0 0) 0x7f8f90005730 con 0x7f8fa8159ac0 2023-10-21T17:39:35.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 msgr2=0x7f8fa8164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8fa8164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 >> v1:172.21.15.124:6789/0 conn(0x7f8fa80ad790 legacy=0x7f8fa80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa815db30 con 0x7f8fa8159ac0 2023-10-21T17:39:35.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8f98001670 con 0x7f8fa8159ac0 2023-10-21T17:39:35.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 >> v1:172.21.15.53:6789/0 conn(0x7f8fa8159ac0 legacy=0x7f8fa806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f90005bb0 0x7f8f90005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f900069d0 0x7f8f90006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8f90007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9000dc70 con 0x7f8f90005bb0 2023-10-21T17:39:35.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9000dde0 con 0x7f8f900069d0 2023-10-21T17:39:35.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9000dfa0 con 0x7f8fa80709e0 2023-10-21T17:39:35.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.423+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f90005bb0 0x7f8f90005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa659c640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f900069d0 0x7f8f90006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa6d9d640 1 -- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 msgr2=0x7f8f90007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa759e640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8f90007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8f90007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa6d9d640 1 -- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f900069d0 msgr2=0x7f8f90006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f900069d0 0x7f8f90006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa6d9d640 1 -- 172.21.15.53:0/3008459318 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f90005730 con 0x7f8f90005bb0 2023-10-21T17:39:35.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.424+0000 7f8fa759e640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fa80709e0 0x7f8f90007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:35.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.425+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f90005bb0 0x7f8f90005fd0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f8f98001eb0 tx=0x7f8f90007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccf4b1be75b694c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.425+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98005ad0 con 0x7f8f90005bb0 2023-10-21T17:39:35.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.425+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f98005c30 con 0x7f8f90005bb0 2023-10-21T17:39:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fa5d9b640 1 -- 172.21.15.53:0/3008459318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802c7a0 con 0x7f8f90005bb0 2023-10-21T17:39:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 -- 172.21.15.53:0/3008459318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f90005bb0 msgr2=0x7f8f90005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/3008459318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f90005bb0 0x7f8f90005fd0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f8f98001eb0 tx=0x7f8f90007a00 comp rx=0 tx=0).stop 2023-10-21T17:39:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fa759e640 1 -- 172.21.15.53:0/3008459318 reap_dead start 2023-10-21T17:39:35.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 -- 172.21.15.53:0/3008459318 shutdown_connections 2023-10-21T17:39:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 -- 172.21.15.53:0/3008459318 >> 172.21.15.53:0/3008459318 conn(0x7f8fa806a380 msgr2=0x7f8fa80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 -- 172.21.15.53:0/3008459318 shutdown_connections 2023-10-21T17:39:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.426+0000 7f8fad1b0640 1 -- 172.21.15.53:0/3008459318 wait complete. 2023-10-21T17:39:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.428+0000 7f8fad1b0640 1 Processor -- start 2023-10-21T17:39:35.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.428+0000 7f8fad1b0640 1 -- start start 2023-10-21T17:39:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.429+0000 7f8fad1b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90005bb0 0x7f8fa80ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fad1b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fa8154f10 0x7f8fa815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fad1b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fad1b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80ac110 con 0x7f8fa80b3a40 2023-10-21T17:39:35.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fad1b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80ac3f0 con 0x7f8fa8154f10 2023-10-21T17:39:35.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fad1b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80ac6d0 con 0x7f8f90005bb0 2023-10-21T17:39:35.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fa759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44786/0 (socket says 172.21.15.53:44786) 2023-10-21T17:39:35.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fa759e640 1 -- 172.21.15.53:0/2144912412 learned_addr learned my addr 172.21.15.53:0/2144912412 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:35.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90005bb0 0x7f8fa80ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.430+0000 7f8fa659c640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fa8154f10 0x7f8fa815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 -- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90005bb0 msgr2=0x7f8fa80ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90005bb0 0x7f8fa80ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 -- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fa8154f10 msgr2=0x7f8fa815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fa8154f10 0x7f8fa815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 -- 172.21.15.53:0/2144912412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80b3520 con 0x7f8fa80b3a40 2023-10-21T17:39:35.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fa759e640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80ce750 tx=0x7f8fa000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0017400 con 0x7f8fa80b3a40 2023-10-21T17:39:35.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0011040 con 0x7f8fa80b3a40 2023-10-21T17:39:35.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00177c0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.431+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa80b3210 con 0x7f8fa80b3a40 2023-10-21T17:39:35.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.432+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa815cbd0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.433+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa80ac890 con 0x7f8fa80b3a40 2023-10-21T17:39:35.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.434+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8fa00203b0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.435+0000 7f8f8f7fe640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 0x7f8f9406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.435+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f8fa0089bb0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.435+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 0x7f8f9406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.437+0000 7f8fa6d9d640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 0x7f8f9406d570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f98034000 tx=0x7f8f98009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.439+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fa00549a0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.701+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8fa8067490 con 0x7f8fa80b3a40 2023-10-21T17:39:35.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.702+0000 7f8f8f7fe640 1 -- 172.21.15.53:0/2144912412 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v51) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8fa0057fc0 con 0x7f8fa80b3a40 2023-10-21T17:39:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:35.705 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:39:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:39:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.705 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:39:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:39:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:39:35.706 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:39:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.707 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:39:35.708 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:39:35.709 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:39:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:39:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:39:35.710 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:39:35.711 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:39:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:39:35.713 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.715 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.716 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:39:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:39:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.717 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:39:35.718 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:39:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.720 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:39:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:39:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.722 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.723 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:39:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:39:35.725 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:39:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:39:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:39:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:39:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.728 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:39:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:39:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:39:35.729 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:39:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:39:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.730 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:39:35.731 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:39:35.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.732 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:39:35.732 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:39:35.733 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:39:35.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:39:35.733 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:39:35.733 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:39:35.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.734 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:39:35.734 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.736 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:39:35.736 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:39:35.736 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:35.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:35.738 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:39:35.738 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:39:35.738 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:39:35.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:39:35.738 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:39:35.739 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:39:35.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:35.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:39:35.739 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:39:35.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.704+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 msgr2=0x7f8f9406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.704+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 0x7f8f9406d570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f98034000 tx=0x7f8f98009030 comp rx=0 tx=0).stop 2023-10-21T17:39:35.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 msgr2=0x7f8fa80b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80ce750 tx=0x7f8fa000f4a0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 shutdown_connections 2023-10-21T17:39:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f9406b0b0 0x7f8f9406d570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fa80b3a40 0x7f8fa80b3ec0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fa8154f10 0x7f8fa815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 --2- 172.21.15.53:0/2144912412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f90005bb0 0x7f8fa80ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 >> 172.21.15.53:0/2144912412 conn(0x7f8fa806a380 msgr2=0x7f8fa806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 shutdown_connections 2023-10-21T17:39:35.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.705+0000 7f8fad1b0640 1 -- 172.21.15.53:0/2144912412 wait complete. 2023-10-21T17:39:35.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:35.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.968+0000 7fda8c9ed640 1 Processor -- start 2023-10-21T17:39:35.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.968+0000 7fda8c9ed640 1 -- start start 2023-10-21T17:39:35.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda8c9ed640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda880b07d0 0x7fda88165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda8c9ed640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8815f7d0 con 0x7fda8815ab30 2023-10-21T17:39:35.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda8c9ed640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8815efb0 con 0x7fda880adc70 2023-10-21T17:39:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda8c9ed640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8815f210 con 0x7fda880b07d0 2023-10-21T17:39:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda86d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fda8815ab30 0x7fda8815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55794/0 (socket says 172.21.15.53:55794) 2023-10-21T17:39:35.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda86d9d640 1 -- 172.21.15.53:0/1848610213 learned_addr learned my addr 172.21.15.53:0/1848610213 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.970+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336279806 0 0) 0x7fda8815f7d0 con 0x7fda8815ab30 2023-10-21T17:39:35.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda70003520 con 0x7fda8815ab30 2023-10-21T17:39:35.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda8759e640 1 --2- 172.21.15.53:0/1848610213 >> v2:172.21.15.124:3300/0 conn(0x7fda880b07d0 0x7fda88165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3876421367 0 0) 0x7fda70003520 con 0x7fda8815ab30 2023-10-21T17:39:35.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 >> v2:172.21.15.124:3300/0 conn(0x7fda880b07d0 msgr2=0x7fda88165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 --2- 172.21.15.53:0/1848610213 >> v2:172.21.15.124:3300/0 conn(0x7fda880b07d0 0x7fda88165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 >> v1:172.21.15.124:6789/0 conn(0x7fda880adc70 legacy=0x7fda880b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8815f4f0 con 0x7fda8815ab30 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fda78001670 con 0x7fda8815ab30 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 >> v1:172.21.15.53:6789/0 conn(0x7fda8815ab30 legacy=0x7fda8815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda700039c0 0x7fda70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.977 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda700051d0 0x7fda7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8815f210 con 0x7fda700047c0 2023-10-21T17:39:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7000c120 con 0x7fda700051d0 2023-10-21T17:39:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7000c2e0 con 0x7fda700039c0 2023-10-21T17:39:35.978 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.971+0000 7fda8659c640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8759e640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda700051d0 0x7fda7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda86d9d640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda700039c0 0x7fda70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8659c640 1 -- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda700039c0 msgr2=0x7fda70003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8659c640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda700039c0 0x7fda70003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8659c640 1 -- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda700051d0 msgr2=0x7fda7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8659c640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda700051d0 0x7fda7000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8659c640 1 -- 172.21.15.53:0/1848610213 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70003520 con 0x7fda700047c0 2023-10-21T17:39:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.972+0000 7fda8759e640 1 -- 172.21.15.53:0/1848610213 reap_dead start 2023-10-21T17:39:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.973+0000 7fda8659c640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda70004c40 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fda7c001720 tx=0x7fda700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d2b72822a3c831 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.973+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0052d0 con 0x7fda700047c0 2023-10-21T17:39:35.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.973+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7c02e050 con 0x7fda700047c0 2023-10-21T17:39:35.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.973+0000 7fda85d9b640 1 -- 172.21.15.53:0/1848610213 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c003830 con 0x7fda700047c0 2023-10-21T17:39:35.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 -- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 msgr2=0x7fda70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda70004c40 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fda7c001720 tx=0x7fda700057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 -- 172.21.15.53:0/1848610213 shutdown_connections 2023-10-21T17:39:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/1848610213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda70004c40 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 -- 172.21.15.53:0/1848610213 >> 172.21.15.53:0/1848610213 conn(0x7fda880cb0c0 msgr2=0x7fda880cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 -- 172.21.15.53:0/1848610213 shutdown_connections 2023-10-21T17:39:35.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.974+0000 7fda8c9ed640 1 -- 172.21.15.53:0/1848610213 wait complete. 2023-10-21T17:39:35.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.976+0000 7fda8c9ed640 1 Processor -- start 2023-10-21T17:39:35.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.976+0000 7fda8c9ed640 1 -- start start 2023-10-21T17:39:35.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 0x7fda880c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8806cc50 0x7fda8806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8806d610 con 0x7fda700047c0 2023-10-21T17:39:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c5c50 con 0x7fda88143aa0 2023-10-21T17:39:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda8c9ed640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda88143f20 con 0x7fda8806cc50 2023-10-21T17:39:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda86d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda86d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44804/0 (socket says 172.21.15.53:44804) 2023-10-21T17:39:35.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.977+0000 7fda86d9d640 1 -- 172.21.15.53:0/730312021 learned_addr learned my addr 172.21.15.53:0/730312021 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:35.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda8659c640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 0x7fda880c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda8759e640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8806cc50 0x7fda8806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 -- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8806cc50 msgr2=0x7fda8806d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8806cc50 0x7fda8806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 -- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 msgr2=0x7fda880c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 0x7fda880c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 -- 172.21.15.53:0/730312021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda880c5eb0 con 0x7fda700047c0 2023-10-21T17:39:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda8659c640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 0x7fda880c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:35.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda86d9d640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e730 tx=0x7fda7800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.978+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7800c6c0 con 0x7fda700047c0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.979+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7800c840 con 0x7fda700047c0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.979+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7800cbf0 con 0x7fda700047c0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.979+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda880c6190 con 0x7fda700047c0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.979+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda880c6750 con 0x7fda700047c0 2023-10-21T17:39:35.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.980+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8804e160 con 0x7fda700047c0 2023-10-21T17:39:35.990 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.981+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fda7801d460 con 0x7fda700047c0 2023-10-21T17:39:35.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.982+0000 7fda6f7fe640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 0x7fda7406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:35.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.982+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fda78089910 con 0x7fda700047c0 2023-10-21T17:39:35.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.982+0000 7fda8659c640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 0x7fda7406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:35.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.984+0000 7fda8659c640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 0x7fda7406d5c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda7c033000 tx=0x7fda7c009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:35.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:35.986+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda780548b0 con 0x7fda700047c0 2023-10-21T17:39:36.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.180+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda880ada50 con 0x7fda700047c0 2023-10-21T17:39:36.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.183+0000 7fda6f7fe640 1 -- 172.21.15.53:0/730312021 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v250) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fda78057ed0 con 0x7fda700047c0 2023-10-21T17:39:36.185 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:36.185 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":250,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:35.178880+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 msgr2=0x7fda7406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 0x7fda7406d5c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda7c033000 tx=0x7fda7c009030 comp rx=0 tx=0).stop 2023-10-21T17:39:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 msgr2=0x7fda880b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e730 tx=0x7fda7800ebf0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 shutdown_connections 2023-10-21T17:39:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fda7406b100 0x7fda7406d5c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda8806cc50 0x7fda8806d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda88143aa0 0x7fda880c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 --2- 172.21.15.53:0/730312021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda700047c0 0x7fda880b3360 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.185+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 >> 172.21.15.53:0/730312021 conn(0x7fda880cb0c0 msgr2=0x7fda880ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:36.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.186+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 shutdown_connections 2023-10-21T17:39:36.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.186+0000 7fda8c9ed640 1 -- 172.21.15.53:0/730312021 wait complete. 2023-10-21T17:39:36.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:36.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:36.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.426+0000 7f8f90be2640 1 Processor -- start 2023-10-21T17:39:36.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.426+0000 7f8f90be2640 1 -- start start 2023-10-21T17:39:36.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.427+0000 7f8f90be2640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f8c0d1120 0x7f8f8c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f90be2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 0x7f8f8c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f90be2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c0d0a80 con 0x7f8f8c0d1500 2023-10-21T17:39:36.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f90be2640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8c161620 con 0x7f8f8c0d1120 2023-10-21T17:39:36.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f90be2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b1fe0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 0x7f8f8c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f8affd640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8f8c0d1500 0x7f8f8c0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:36430/0 (socket says 172.21.15.53:36430) 2023-10-21T17:39:36.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f8b7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f8c0d1120 0x7f8f8c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 0x7f8f8c1610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:60356/0 (socket says 172.21.15.53:60356) 2023-10-21T17:39:36.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.428+0000 7f8f8affd640 1 -- 172.21.15.53:0/4116080020 learned_addr learned my addr 172.21.15.53:0/4116080020 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:36.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.429+0000 7f8f8b7fe640 1 -- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 msgr2=0x7f8f8c1610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:36.432 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.429+0000 7f8f8b7fe640 1 --2- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 0x7f8f8c1610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.429+0000 7f8f8b7fe640 1 -- 172.21.15.53:0/4116080020 >> v1:172.21.15.124:6789/0 conn(0x7f8f8c0d1500 legacy=0x7f8f8c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.429+0000 7f8f8b7fe640 1 -- 172.21.15.53:0/4116080020 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c0b4580 con 0x7f8f8c0d1120 2023-10-21T17:39:36.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.430+0000 7f8f8b7fe640 1 --2- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0d04a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009de0 tx=0x7f8f7c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac13f7bdf5b9a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:36.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.431+0000 7f8f8a7fc640 1 -- 172.21.15.53:0/4116080020 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c02cc00 con 0x7f8f8c0d1120 2023-10-21T17:39:36.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.431+0000 7f8f8a7fc640 1 -- 172.21.15.53:0/4116080020 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c02cd60 con 0x7f8f8c0d1120 2023-10-21T17:39:36.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.431+0000 7f8f8a7fc640 1 -- 172.21.15.53:0/4116080020 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c034660 con 0x7f8f8c0d1120 2023-10-21T17:39:36.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 -- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 msgr2=0x7f8f8c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 --2- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0d04a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009de0 tx=0x7f8f7c02be50 comp rx=0 tx=0).stop 2023-10-21T17:39:36.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 -- 172.21.15.53:0/4116080020 shutdown_connections 2023-10-21T17:39:36.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 --2- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c15dcd0 0x7f8f8c1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 --2- 172.21.15.53:0/4116080020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0d04a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 -- 172.21.15.53:0/4116080020 >> 172.21.15.53:0/4116080020 conn(0x7f8f8c0d2960 msgr2=0x7f8f8c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:36.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 -- 172.21.15.53:0/4116080020 shutdown_connections 2023-10-21T17:39:36.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.432+0000 7f8f90be2640 1 -- 172.21.15.53:0/4116080020 wait complete. 2023-10-21T17:39:36.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.434+0000 7f8f90be2640 1 Processor -- start 2023-10-21T17:39:36.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.434+0000 7f8f90be2640 1 -- start start 2023-10-21T17:39:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c0d1500 0x7f8f8c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0c99a0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b7400 con 0x7f8f8c0d1120 2023-10-21T17:39:36.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f90be2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0b76e0 con 0x7f8f8c0d1500 2023-10-21T17:39:36.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44806/0 (socket says 172.21.15.53:44806) 2023-10-21T17:39:36.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 -- 172.21.15.53:0/3490466126 learned_addr learned my addr 172.21.15.53:0/3490466126 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:36.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8b7fe640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8affd640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c0d1500 0x7f8f8c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 -- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c0d1500 msgr2=0x7f8f8c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c0d1500 0x7f8f8c0cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 -- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 msgr2=0x7f8f8c0b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.436+0000 7f8f8bfff640 1 -- 172.21.15.53:0/3490466126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c009a70 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f8bfff640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e7f0 tx=0x7f8f8400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:36.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84009510 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8400ee40 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c0b79c0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400cb80 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.437+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c0b7f80 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.439+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c15c430 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.440+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8f8400cce0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.441+0000 7f8f83fff640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 0x7f8f6c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:36.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.441+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f8f840899f0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.442+0000 7f8f8b7fe640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 0x7f8f6c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:36.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.443+0000 7f8f8b7fe640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 0x7f8f6c06d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c036000 tx=0x7f8f7c030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:36.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.446+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f84054850 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.638+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f8c074150 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.639+0000 7f8f83fff640 1 -- 172.21.15.53:0/3490466126 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v250) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f840210a0 con 0x7f8f8c15dcd0 2023-10-21T17:39:36.847 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 msgr2=0x7f8f6c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 0x7f8f6c06d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c036000 tx=0x7f8f7c030000 comp rx=0 tx=0).stop 2023-10-21T17:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 msgr2=0x7f8f8c1f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e7f0 tx=0x7f8f8400ebf0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 shutdown_connections 2023-10-21T17:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f8f6c06b0b0 0x7f8f6c06d570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f8c15dcd0 0x7f8f8c1f18f0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f8c0d1500 0x7f8f8c0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 --2- 172.21.15.53:0/3490466126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f8c0d1120 0x7f8f8c0b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:36.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 >> 172.21.15.53:0/3490466126 conn(0x7f8f8c0d2960 msgr2=0x7f8f8c0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.642+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 shutdown_connections 2023-10-21T17:39:36.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:36.643+0000 7f8f90be2640 1 -- 172.21.15.53:0/3490466126 wait complete. 2023-10-21T17:39:36.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:39:37.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.085+0000 7f707d378640 1 Processor -- start 2023-10-21T17:39:37.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.085+0000 7f707d378640 1 -- start start 2023-10-21T17:39:37.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f707d378640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f707815cd90 0x7f70780702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f707d378640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078160940 0x7f7078070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f707d378640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780c48f0 con 0x7f7078070d50 2023-10-21T17:39:37.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f707d378640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70780c4d80 con 0x7f707815cd90 2023-10-21T17:39:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f707d378640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c3ce0 con 0x7f7078160940 2023-10-21T17:39:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f7076ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f707815cd90 0x7f70780702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f70777fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f7078070d50 0x7f70780cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55800/0 (socket says 172.21.15.53:55800) 2023-10-21T17:39:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.087+0000 7f70777fe640 1 -- 172.21.15.53:0/1065190318 learned_addr learned my addr 172.21.15.53:0/1065190318 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f70767fc640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078160940 0x7f7078070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7075ffb640 1 -- 172.21.15.53:0/1065190318 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723101484 0 0) 0x7f70780c48f0 con 0x7f7078070d50 2023-10-21T17:39:37.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7075ffb640 1 -- 172.21.15.53:0/1065190318 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7060003520 con 0x7f7078070d50 2023-10-21T17:39:37.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7076ffd640 1 -- 172.21.15.53:0/1065190318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078160940 msgr2=0x7f7078070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:37.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7076ffd640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078160940 0x7f7078070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7076ffd640 1 -- 172.21.15.53:0/1065190318 >> v1:172.21.15.53:6789/0 conn(0x7f7078070d50 legacy=0x7f70780cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.088+0000 7f7076ffd640 1 -- 172.21.15.53:0/1065190318 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780c3fc0 con 0x7f707815cd90 2023-10-21T17:39:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.090+0000 7f7076ffd640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780702d0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f70680097a0 tx=0x7f706802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdce4bbfee81dbe server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.090+0000 7f7075ffb640 1 -- 172.21.15.53:0/1065190318 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068031830 con 0x7f707815cd90 2023-10-21T17:39:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.090+0000 7f7075ffb640 1 -- 172.21.15.53:0/1065190318 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068031990 con 0x7f707815cd90 2023-10-21T17:39:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.090+0000 7f7075ffb640 1 -- 172.21.15.53:0/1065190318 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068031d20 con 0x7f707815cd90 2023-10-21T17:39:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 -- 172.21.15.53:0/1065190318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 msgr2=0x7f70780702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780702d0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f70680097a0 tx=0x7f706802b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 -- 172.21.15.53:0/1065190318 shutdown_connections 2023-10-21T17:39:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078160940 0x7f7078070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 --2- 172.21.15.53:0/1065190318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780702d0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 -- 172.21.15.53:0/1065190318 >> 172.21.15.53:0/1065190318 conn(0x7f707806a380 msgr2=0x7f70780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 -- 172.21.15.53:0/1065190318 shutdown_connections 2023-10-21T17:39:37.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.091+0000 7f707d378640 1 -- 172.21.15.53:0/1065190318 wait complete. 2023-10-21T17:39:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.093+0000 7f707d378640 1 Processor -- start 2023-10-21T17:39:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.093+0000 7f707d378640 1 -- start start 2023-10-21T17:39:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078070d50 0x7f707815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7078160940 0x7f70780c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780b2890 con 0x7f707815cd90 2023-10-21T17:39:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780b2b70 con 0x7f7078160940 2023-10-21T17:39:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f707d378640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780b2e50 con 0x7f7078070d50 2023-10-21T17:39:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f70767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f70767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44826/0 (socket says 172.21.15.53:44826) 2023-10-21T17:39:37.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.094+0000 7f70767fc640 1 -- 172.21.15.53:0/2356519855 learned_addr learned my addr 172.21.15.53:0/2356519855 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f7076ffd640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078070d50 0x7f707815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 -- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078070d50 msgr2=0x7f707815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078070d50 0x7f707815fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 -- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7078160940 msgr2=0x7f70780c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70777fe640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7078160940 0x7f70780c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7078160940 0x7f70780c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 -- 172.21.15.53:0/2356519855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009400 con 0x7f707815cd90 2023-10-21T17:39:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f70767fc640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f706c00d5f0 tx=0x7f706c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0125f0 con 0x7f707815cd90 2023-10-21T17:39:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c012750 con 0x7f707815cd90 2023-10-21T17:39:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c012b00 con 0x7f707815cd90 2023-10-21T17:39:37.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.095+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780b3130 con 0x7f707815cd90 2023-10-21T17:39:37.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.096+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780c94d0 con 0x7f707815cd90 2023-10-21T17:39:37.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.097+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70780c9690 con 0x7f707815cd90 2023-10-21T17:39:37.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.098+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f706c012c60 con 0x7f707815cd90 2023-10-21T17:39:37.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.098+0000 7f705f7fe640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 0x7f706406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.099+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f706c089cf0 con 0x7f707815cd90 2023-10-21T17:39:37.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.099+0000 7f7076ffd640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 0x7f706406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.100+0000 7f7076ffd640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 0x7f706406d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7068036000 tx=0x7f7068008030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.103+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f706c054bc0 con 0x7f707815cd90 2023-10-21T17:39:37.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.302+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7078067490 con 0x7f707815cd90 2023-10-21T17:39:37.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.303+0000 7f705f7fe640 1 -- 172.21.15.53:0/2356519855 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v250) v1 ==== 84+0+65 (secure 0 0 0) 0x7f706c0581e0 con 0x7f707815cd90 2023-10-21T17:39:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:37.306 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:39:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:39:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:39:37.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:37.307 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:39:37.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.305+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 msgr2=0x7f706406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.305+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 0x7f706406d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7068036000 tx=0x7f7068008030 comp rx=0 tx=0).stop 2023-10-21T17:39:37.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.305+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 msgr2=0x7f70780cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.305+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f706c00d5f0 tx=0x7f706c00dab0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 shutdown_connections 2023-10-21T17:39:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f706406b0e0 0x7f706406d5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7078160940 0x7f70780c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707815cd90 0x7f70780cb780 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 --2- 172.21.15.53:0/2356519855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7078070d50 0x7f707815fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 >> 172.21.15.53:0/2356519855 conn(0x7f707806a380 msgr2=0x7f7078158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:37.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 shutdown_connections 2023-10-21T17:39:37.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.306+0000 7f707d378640 1 -- 172.21.15.53:0/2356519855 wait complete. 2023-10-21T17:39:37.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:39:37.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:39:37.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.564+0000 7f4929941640 1 Processor -- start 2023-10-21T17:39:37.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.564+0000 7f4929941640 1 -- start start 2023-10-21T17:39:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f4929941640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49240c9080 0x7f49240c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f4929941640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49240c7ad0 con 0x7f4924070b50 2023-10-21T17:39:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f4929941640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4924160e60 con 0x7f492415a820 2023-10-21T17:39:37.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f4929941640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49240c9a40 con 0x7f49240c9080 2023-10-21T17:39:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f491bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f49240c9080 0x7f49240c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f49237fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4924070b50 0x7f49240c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55810/0 (socket says 172.21.15.53:55810) 2023-10-21T17:39:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.566+0000 7f49237fe640 1 -- 172.21.15.53:0/3779605373 learned_addr learned my addr 172.21.15.53:0/3779605373 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:37.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.567+0000 7f491bfff640 1 -- 172.21.15.53:0/3779605373 >> v1:172.21.15.124:6789/0 conn(0x7f492415a820 legacy=0x7f4924160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.567+0000 7f491bfff640 1 -- 172.21.15.53:0/3779605373 >> v1:172.21.15.53:6789/0 conn(0x7f4924070b50 legacy=0x7f49240c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.567+0000 7f491bfff640 1 -- 172.21.15.53:0/3779605373 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49240cb210 con 0x7f49240c9080 2023-10-21T17:39:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.568+0000 7f491bfff640 1 --2- 172.21.15.53:0/3779605373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49240c9080 0x7f49240c9500 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097a0 tx=0x7f490c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29c54f760ae9e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.568+0000 7f49227fc640 1 -- 172.21.15.53:0/3779605373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c030610 con 0x7f49240c9080 2023-10-21T17:39:37.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f49227fc640 1 -- 172.21.15.53:0/3779605373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c030770 con 0x7f49240c9080 2023-10-21T17:39:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f49227fc640 1 -- 172.21.15.53:0/3779605373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c030b00 con 0x7f49240c9080 2023-10-21T17:39:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 -- 172.21.15.53:0/3779605373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49240c9080 msgr2=0x7f49240c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 --2- 172.21.15.53:0/3779605373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49240c9080 0x7f49240c9500 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097a0 tx=0x7f490c02b960 comp rx=0 tx=0).stop 2023-10-21T17:39:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 -- 172.21.15.53:0/3779605373 shutdown_connections 2023-10-21T17:39:37.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 --2- 172.21.15.53:0/3779605373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49240c9080 0x7f49240c9500 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 -- 172.21.15.53:0/3779605373 >> 172.21.15.53:0/3779605373 conn(0x7f4924069d50 msgr2=0x7f492406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 -- 172.21.15.53:0/3779605373 shutdown_connections 2023-10-21T17:39:37.574 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.569+0000 7f4929941640 1 -- 172.21.15.53:0/3779605373 wait complete. 2023-10-21T17:39:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 Processor -- start 2023-10-21T17:39:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 -- start start 2023-10-21T17:39:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49240c9080 0x7f492415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f492415a820 0x7f4924140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492415de30 con 0x7f4924070b50 2023-10-21T17:39:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492415d1b0 con 0x7f492415a820 2023-10-21T17:39:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4929941640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492415d490 con 0x7f49240c9080 2023-10-21T17:39:37.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4922ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4922ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44830/0 (socket says 172.21.15.53:44830) 2023-10-21T17:39:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.572+0000 7f4922ffd640 1 -- 172.21.15.53:0/2795301960 learned_addr learned my addr 172.21.15.53:0/2795301960 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f49237fe640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f492415a820 0x7f4924140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 -- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49240c9080 msgr2=0x7f492415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49240c9080 0x7f492415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 -- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f492415a820 msgr2=0x7f4924140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f492415a820 0x7f4924140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 -- 172.21.15.53:0/2795301960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c009400 con 0x7f4924070b50 2023-10-21T17:39:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.573+0000 7f4922ffd640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f491400c400 tx=0x7f491400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.574+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140052d0 con 0x7f4924070b50 2023-10-21T17:39:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.574+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4914005430 con 0x7f4924070b50 2023-10-21T17:39:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.574+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914003830 con 0x7f4924070b50 2023-10-21T17:39:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.574+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492415d770 con 0x7f4924070b50 2023-10-21T17:39:37.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.574+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4924153cd0 con 0x7f4924070b50 2023-10-21T17:39:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.575+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4924153e60 con 0x7f4924070b50 2023-10-21T17:39:37.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.576+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4914003990 con 0x7f4924070b50 2023-10-21T17:39:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.578+0000 7f4920ff9640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 0x7f490406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.578+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f4914088e10 con 0x7f4924070b50 2023-10-21T17:39:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.579+0000 7f491bfff640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 0x7f490406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:37.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.579+0000 7f491bfff640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 0x7f490406d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f490c036000 tx=0x7f490c03b040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:37.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.582+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4914053d40 con 0x7f4924070b50 2023-10-21T17:39:37.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:37.780+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f492404e160 con 0x7f4924070b50 2023-10-21T17:39:38.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.200+0000 7f4920ff9640 1 -- 172.21.15.53:0/2795301960 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v251) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4914017020 con 0x7f4924070b50 2023-10-21T17:39:38.202 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:39:38.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 msgr2=0x7f490406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 0x7f490406d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f490c036000 tx=0x7f490c03b040 comp rx=0 tx=0).stop 2023-10-21T17:39:38.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 msgr2=0x7f4924153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f491400c400 tx=0x7f491400c8c0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 shutdown_connections 2023-10-21T17:39:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f490406b0b0 0x7f490406d570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f492415a820 0x7f4924140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49240c9080 0x7f492415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 --2- 172.21.15.53:0/2795301960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4924070b50 0x7f4924153790 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.204+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 >> 172.21.15.53:0/2795301960 conn(0x7f4924069d50 msgr2=0x7f492406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.205+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 shutdown_connections 2023-10-21T17:39:38.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.205+0000 7f4929941640 1 -- 172.21.15.53:0/2795301960 wait complete. 2023-10-21T17:39:38.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:39:38.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.453+0000 7f70d4815640 1 Processor -- start 2023-10-21T17:39:38.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.453+0000 7f70d4815640 1 -- start start 2023-10-21T17:39:38.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.455+0000 7f70d4815640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70d00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.455+0000 7f70d4815640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00c8c60 con 0x7f70d006cfa0 2023-10-21T17:39:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.455+0000 7f70d4815640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00cbe20 con 0x7f70d0070690 2023-10-21T17:39:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.455+0000 7f70d4815640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b1ac0 con 0x7f70d00cb500 2023-10-21T17:39:38.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.456+0000 7f70c7fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f70d006cfa0 0x7f70d015b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55820/0 (socket says 172.21.15.53:55820) 2023-10-21T17:39:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.456+0000 7f70c7fff640 1 -- 172.21.15.53:0/3545861973 learned_addr learned my addr 172.21.15.53:0/3545861973 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.456+0000 7f70ce59c640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70d00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.456+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462469680 0 0) 0x7f70d00c8c60 con 0x7f70d006cfa0 2023-10-21T17:39:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.456+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70bc003520 con 0x7f70d006cfa0 2023-10-21T17:39:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669917088 0 0) 0x7f70bc003520 con 0x7f70d006cfa0 2023-10-21T17:39:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 msgr2=0x7f70d00cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70d00cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 >> v1:172.21.15.124:6789/0 conn(0x7f70d0070690 legacy=0x7f70d0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00b1da0 con 0x7f70d006cfa0 2023-10-21T17:39:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f70b8001670 con 0x7f70d006cfa0 2023-10-21T17:39:38.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 >> v1:172.21.15.53:6789/0 conn(0x7f70d006cfa0 legacy=0x7f70d015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70bc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70bc003f20 0x7f70bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70bc004d00 0x7f70bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b1ac0 con 0x7f70bc003f20 2023-10-21T17:39:38.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00bb10 con 0x7f70bc004d00 2023-10-21T17:39:38.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00bcd0 con 0x7f70d00cb500 2023-10-21T17:39:38.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.457+0000 7f70c7fff640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70bc003f20 0x7f70bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70c7fff640 1 -- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 msgr2=0x7f70bc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:38.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70ce59c640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70bc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70c7fff640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70d00cb500 0x7f70bc0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70ced9d640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70bc004d00 0x7f70bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70c7fff640 1 -- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70bc004d00 msgr2=0x7f70bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70c7fff640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70bc004d00 0x7f70bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70c7fff640 1 -- 172.21.15.53:0/3545861973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc003520 con 0x7f70bc003f20 2023-10-21T17:39:38.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.458+0000 7f70ced9d640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70bc004d00 0x7f70bc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:38.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.459+0000 7f70c7fff640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70bc003f20 0x7f70bc0047c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c210 tx=0x7f70bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250477897c769384 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:38.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.459+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b802cae0 con 0x7f70bc003f20 2023-10-21T17:39:38.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.459+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b802cc40 con 0x7f70bc003f20 2023-10-21T17:39:38.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.459+0000 7f70cdd9b640 1 -- 172.21.15.53:0/3545861973 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80058d0 con 0x7f70bc003f20 2023-10-21T17:39:38.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 -- 172.21.15.53:0/3545861973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70bc003f20 msgr2=0x7f70bc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 --2- 172.21.15.53:0/3545861973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70bc003f20 0x7f70bc0047c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c210 tx=0x7f70bc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70ced9d640 1 -- 172.21.15.53:0/3545861973 reap_dead start 2023-10-21T17:39:38.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 -- 172.21.15.53:0/3545861973 shutdown_connections 2023-10-21T17:39:38.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 -- 172.21.15.53:0/3545861973 >> 172.21.15.53:0/3545861973 conn(0x7f70d006a140 msgr2=0x7f70d006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:38.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 -- 172.21.15.53:0/3545861973 shutdown_connections 2023-10-21T17:39:38.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.460+0000 7f70d4815640 1 -- 172.21.15.53:0/3545861973 wait complete. 2023-10-21T17:39:38.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.462+0000 7f70d4815640 1 Processor -- start 2023-10-21T17:39:38.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.462+0000 7f70d4815640 1 -- start start 2023-10-21T17:39:38.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc003f20 0x7f70d00c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70d0152420 0x7f70d0158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00c36c0 con 0x7f70d0152b30 2023-10-21T17:39:38.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00c2db0 con 0x7f70d0152420 2023-10-21T17:39:38.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.463+0000 7f70d4815640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0153330 con 0x7f70bc003f20 2023-10-21T17:39:38.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44848/0 (socket says 172.21.15.53:44848) 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 -- 172.21.15.53:0/2697502889 learned_addr learned my addr 172.21.15.53:0/2697502889 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70ce59c640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc003f20 0x7f70d00c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70ced9d640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70d0152420 0x7f70d0158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 -- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc003f20 msgr2=0x7f70d00c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc003f20 0x7f70d00c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 -- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70d0152420 msgr2=0x7f70d0158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:38.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70d0152420 0x7f70d0158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:38.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 -- 172.21.15.53:0/2697502889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00c51d0 con 0x7f70d0152b30 2023-10-21T17:39:38.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.464+0000 7f70c7fff640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c210 tx=0x7f70b8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:38.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.465+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8039070 con 0x7f70d0152b30 2023-10-21T17:39:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.465+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b8001430 con 0x7f70d0152b30 2023-10-21T17:39:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.465+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80015a0 con 0x7f70d0152b30 2023-10-21T17:39:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.465+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d00c3070 con 0x7f70d0152b30 2023-10-21T17:39:38.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.465+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d0152880 con 0x7f70d0152b30 2023-10-21T17:39:38.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.466+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0071cb0 con 0x7f70d0152b30 2023-10-21T17:39:38.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.468+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f70b8032070 con 0x7f70d0152b30 2023-10-21T17:39:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.469+0000 7f70c6ffd640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 0x7f70b006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.469+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f70b80ab480 con 0x7f70d0152b30 2023-10-21T17:39:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.469+0000 7f70ce59c640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 0x7f70b006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:38.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.471+0000 7f70ce59c640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 0x7f70b006d5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009a10 tx=0x7f70c0008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:38.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.473+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70b8076570 con 0x7f70d0152b30 2023-10-21T17:39:38.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:38.671+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f70d00c7bd0 con 0x7f70d0152b30 2023-10-21T17:39:39.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.207+0000 7f70c6ffd640 1 -- 172.21.15.53:0/2697502889 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v252) v1 ==== 170+0+0 (secure 0 0 0) 0x7f70b80460d0 con 0x7f70d0152b30 2023-10-21T17:39:39.209 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:39:39.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 msgr2=0x7f70b006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 0x7f70b006d5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009a10 tx=0x7f70c0008040 comp rx=0 tx=0).stop 2023-10-21T17:39:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 msgr2=0x7f70d00c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c210 tx=0x7f70b8001a00 comp rx=0 tx=0).stop 2023-10-21T17:39:39.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 shutdown_connections 2023-10-21T17:39:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f70b006b0e0 0x7f70b006d5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70d0152420 0x7f70d0158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70d0152b30 0x7f70d00c3b50 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 --2- 172.21.15.53:0/2697502889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc003f20 0x7f70d00c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.210+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 >> 172.21.15.53:0/2697502889 conn(0x7f70d006a140 msgr2=0x7f70d00cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.211+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 shutdown_connections 2023-10-21T17:39:39.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.211+0000 7f70d4815640 1 -- 172.21.15.53:0/2697502889 wait complete. 2023-10-21T17:39:39.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:39:39.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.440+0000 7faa61cd0640 1 Processor -- start 2023-10-21T17:39:39.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.441+0000 7faa61cd0640 1 -- start start 2023-10-21T17:39:39.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.442+0000 7faa61cd0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faa5c0b88b0 0x7faa5c0d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:39.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.442+0000 7faa61cd0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c0d15f0 con 0x7faa5c0b4750 2023-10-21T17:39:39.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.442+0000 7faa61cd0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c0d18d0 con 0x7faa5c15cf50 2023-10-21T17:39:39.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.442+0000 7faa61cd0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c0d2270 con 0x7faa5c0b88b0 2023-10-21T17:39:39.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa5bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faa5c0b4750 0x7faa5c0b5e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55826/0 (socket says 172.21.15.53:55826) 2023-10-21T17:39:39.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa52ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7faa5c0b88b0 0x7faa5c0d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:39.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa5bfff640 1 -- 172.21.15.53:0/4068658157 learned_addr learned my addr 172.21.15.53:0/4068658157 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:39.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa52ffd640 1 -- 172.21.15.53:0/4068658157 >> v1:172.21.15.124:6789/0 conn(0x7faa5c15cf50 legacy=0x7faa5c0d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa52ffd640 1 -- 172.21.15.53:0/4068658157 >> v1:172.21.15.53:6789/0 conn(0x7faa5c0b4750 legacy=0x7faa5c0b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa52ffd640 1 -- 172.21.15.53:0/4068658157 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0d1bb0 con 0x7faa5c0b88b0 2023-10-21T17:39:39.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.443+0000 7faa5affd640 1 -- 172.21.15.53:0/4068658157 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443399902 0 0) 0x7faa5c0d15f0 con 0x7faa5c0b4750 2023-10-21T17:39:39.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.444+0000 7faa52ffd640 1 --2- 172.21.15.53:0/4068658157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b88b0 0x7faa5c0d2690 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7faa440097a0 tx=0x7faa4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22758fc99e905364 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:39.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa5affd640 1 -- 172.21.15.53:0/4068658157 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44031830 con 0x7faa5c0b88b0 2023-10-21T17:39:39.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa5affd640 1 -- 172.21.15.53:0/4068658157 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44031990 con 0x7faa5c0b88b0 2023-10-21T17:39:39.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa5affd640 1 -- 172.21.15.53:0/4068658157 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44031d20 con 0x7faa5c0b88b0 2023-10-21T17:39:39.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa61cd0640 1 -- 172.21.15.53:0/4068658157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b88b0 msgr2=0x7faa5c0d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4068658157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b88b0 0x7faa5c0d2690 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7faa440097a0 tx=0x7faa4402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa61cd0640 1 -- 172.21.15.53:0/4068658157 shutdown_connections 2023-10-21T17:39:39.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4068658157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b88b0 0x7faa5c0d2690 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.445+0000 7faa61cd0640 1 -- 172.21.15.53:0/4068658157 >> 172.21.15.53:0/4068658157 conn(0x7faa5c0cf750 msgr2=0x7faa5c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:39.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.446+0000 7faa61cd0640 1 -- 172.21.15.53:0/4068658157 shutdown_connections 2023-10-21T17:39:39.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.446+0000 7faa61cd0640 1 -- 172.21.15.53:0/4068658157 wait complete. 2023-10-21T17:39:39.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.447+0000 7faa61cd0640 1 Processor -- start 2023-10-21T17:39:39.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.448+0000 7faa61cd0640 1 -- start start 2023-10-21T17:39:39.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:39.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 0x7faa5c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 0x7faa5c0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c15f720 con 0x7faa5c0b4750 2023-10-21T17:39:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0c6d40 con 0x7faa5c15cf50 2023-10-21T17:39:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa61cd0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0c7020 con 0x7faa5c1f2dc0 2023-10-21T17:39:39.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44852/0 (socket says 172.21.15.53:44852) 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 -- 172.21.15.53:0/4050521427 learned_addr learned my addr 172.21.15.53:0/4050521427 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa52ffd640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 0x7faa5c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 -- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 msgr2=0x7faa5c0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5bfff640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 0x7faa5c0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:39.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 0x7faa5c0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 -- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 msgr2=0x7faa5c15f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:39.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 0x7faa5c15f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:39.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.449+0000 7faa5b7fe640 1 -- 172.21.15.53:0/4050521427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44009400 con 0x7faa5c0b4750 2023-10-21T17:39:39.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa5bfff640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 0x7faa5c0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:39.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa52ffd640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 0x7faa5c15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:39.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa5b7fe640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c730 tx=0x7faa4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:39.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c003b70 con 0x7faa5c0b4750 2023-10-21T17:39:39.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c003cf0 con 0x7faa5c0b4750 2023-10-21T17:39:39.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c003ea0 con 0x7faa5c0b4750 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c0c7300 con 0x7faa5c0b4750 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.450+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c0c78c0 con 0x7faa5c0b4750 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.452+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c067900 con 0x7faa5c0b4750 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.453+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7faa4c00aa90 con 0x7faa5c0b4750 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.454+0000 7faa58ff9640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 0x7faa3c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:39.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.454+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6045+0+0 (secure 0 0 0) 0x7faa4c089a60 con 0x7faa5c0b4750 2023-10-21T17:39:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.454+0000 7faa52ffd640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 0x7faa3c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:39.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.455+0000 7faa52ffd640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 0x7faa3c06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa44036000 tx=0x7faa44008030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:39.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.459+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa4c0545c0 con 0x7faa5c0b4750 2023-10-21T17:39:39.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:39.691+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faa5c0c7ae0 con 0x7faa5c0b4750 2023-10-21T17:39:40.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.357+0000 7faa58ff9640 1 -- 172.21.15.53:0/4050521427 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v124) v1 ==== 174+0+0 (secure 0 0 0) 0x7faa4c057be0 con 0x7faa5c0b4750 2023-10-21T17:39:40.358 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 31 and data pool 32 2023-10-21T17:39:40.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 msgr2=0x7faa3c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 0x7faa3c06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa44036000 tx=0x7faa44008030 comp rx=0 tx=0).stop 2023-10-21T17:39:40.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 msgr2=0x7faa5c0c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00c730 tx=0x7faa4c00cbf0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 shutdown_connections 2023-10-21T17:39:40.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7faa3c06b0b0 0x7faa3c06d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa5c1f2dc0 0x7faa5c0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa5c15cf50 0x7faa5c15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 --2- 172.21.15.53:0/4050521427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faa5c0b4750 0x7faa5c0c6760 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 >> 172.21.15.53:0/4050521427 conn(0x7faa5c0cf750 msgr2=0x7faa5c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:40.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 shutdown_connections 2023-10-21T17:39:40.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.360+0000 7faa61cd0640 1 -- 172.21.15.53:0/4050521427 wait complete. 2023-10-21T17:39:40.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.575+0000 7f565a5a7640 1 Processor -- start 2023-10-21T17:39:40.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.575+0000 7f565a5a7640 1 -- start start 2023-10-21T17:39:40.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.576+0000 7f565a5a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f565415cdf0 0x7f56540ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:40.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.576+0000 7f565a5a7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540ceb00 con 0x7f56540d07c0 2023-10-21T17:39:40.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.576+0000 7f565a5a7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540ce820 con 0x7f56540d1640 2023-10-21T17:39:40.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.576+0000 7f565a5a7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56540ce610 con 0x7f565415cdf0 2023-10-21T17:39:40.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f5658da4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f56540d07c0 0x7f56540d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:55836/0 (socket says 172.21.15.53:55836) 2023-10-21T17:39:40.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f5658da4640 1 -- 172.21.15.53:0/1038297226 learned_addr learned my addr 172.21.15.53:0/1038297226 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:40.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f56595a5640 1 --2- 172.21.15.53:0/1038297226 >> v2:172.21.15.53:3300/0 conn(0x7f565415cdf0 0x7f56540ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:40.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f5653fff640 1 -- 172.21.15.53:0/1038297226 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646182868 0 0) 0x7f56540ceb00 con 0x7f56540d07c0 2023-10-21T17:39:40.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f5653fff640 1 -- 172.21.15.53:0/1038297226 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c003520 con 0x7f56540d07c0 2023-10-21T17:39:40.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f56595a5640 1 -- 172.21.15.53:0/1038297226 >> v1:172.21.15.124:6789/0 conn(0x7f56540d1640 legacy=0x7f56540b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:40.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f56595a5640 1 -- 172.21.15.53:0/1038297226 >> v1:172.21.15.53:6789/0 conn(0x7f56540d07c0 legacy=0x7f56540d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.577+0000 7f56595a5640 1 -- 172.21.15.53:0/1038297226 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540b5960 con 0x7f565415cdf0 2023-10-21T17:39:40.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.578+0000 7f56595a5640 1 --2- 172.21.15.53:0/1038297226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f565415cdf0 0x7f56540ce0d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f564402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff69d462fc49e5be server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:40.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.579+0000 7f5653fff640 1 -- 172.21.15.53:0/1038297226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564402cb40 con 0x7f565415cdf0 2023-10-21T17:39:40.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.579+0000 7f5653fff640 1 -- 172.21.15.53:0/1038297226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564402cca0 con 0x7f565415cdf0 2023-10-21T17:39:40.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.579+0000 7f5653fff640 1 -- 172.21.15.53:0/1038297226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644034660 con 0x7f565415cdf0 2023-10-21T17:39:40.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.579+0000 7f565a5a7640 1 -- 172.21.15.53:0/1038297226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f565415cdf0 msgr2=0x7f56540ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.579+0000 7f565a5a7640 1 --2- 172.21.15.53:0/1038297226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f565415cdf0 0x7f56540ce0d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f564402bd90 comp rx=0 tx=0).stop 2023-10-21T17:39:40.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.580+0000 7f565a5a7640 1 -- 172.21.15.53:0/1038297226 shutdown_connections 2023-10-21T17:39:40.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.580+0000 7f565a5a7640 1 --2- 172.21.15.53:0/1038297226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f565415cdf0 0x7f56540ce0d0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.580+0000 7f565a5a7640 1 -- 172.21.15.53:0/1038297226 >> 172.21.15.53:0/1038297226 conn(0x7f565409bcb0 msgr2=0x7f56540d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:40.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.580+0000 7f565a5a7640 1 -- 172.21.15.53:0/1038297226 shutdown_connections 2023-10-21T17:39:40.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.580+0000 7f565a5a7640 1 -- 172.21.15.53:0/1038297226 wait complete. 2023-10-21T17:39:40.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.581+0000 7f565a5a7640 1 Processor -- start 2023-10-21T17:39:40.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.582+0000 7f565a5a7640 1 -- start start 2023-10-21T17:39:40.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.582+0000 7f565a5a7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:40.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f565a5a7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 0x7f565409d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:40.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f565a5a7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 0x7f56540cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:40.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f565a5a7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56540c7c10 con 0x7f56540d07c0 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f565a5a7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56540c7300 con 0x7f565415cdf0 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f565a5a7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565415a860 con 0x7f56540d1640 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44872/0 (socket says 172.21.15.53:44872) 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 -- 172.21.15.53:0/3643639760 learned_addr learned my addr 172.21.15.53:0/3643639760 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:40.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f5658da4640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 0x7f565409d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:40.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f5659da6640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 0x7f56540cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:40.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 -- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 msgr2=0x7f565409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 0x7f565409d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 -- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 msgr2=0x7f56540cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 0x7f56540cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f56595a5640 1 -- 172.21.15.53:0/3643639760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009a70 con 0x7f56540d07c0 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f5659da6640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 0x7f56540cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.583+0000 7f5658da4640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 0x7f565409d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f56595a5640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f5644004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644040650 con 0x7f56540d07c0 2023-10-21T17:39:40.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56440407f0 con 0x7f56540d07c0 2023-10-21T17:39:40.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644040ba0 con 0x7f56540d07c0 2023-10-21T17:39:40.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565409de80 con 0x7f56540d07c0 2023-10-21T17:39:40.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.584+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56540c7550 con 0x7f56540d07c0 2023-10-21T17:39:40.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.585+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56540b51b0 con 0x7f56540d07c0 2023-10-21T17:39:40.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.586+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5644040d00 con 0x7f56540d07c0 2023-10-21T17:39:40.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.587+0000 7f5651ffb640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 0x7f564006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:40.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.587+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f56440acaa0 con 0x7f56540d07c0 2023-10-21T17:39:40.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.588+0000 7f5658da4640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 0x7f564006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:40.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.589+0000 7f5658da4640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 0x7f564006d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56540cee90 tx=0x7f5648009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:40.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.591+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5644077620 con 0x7f56540d07c0 2023-10-21T17:39:40.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.785+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f565406cac0 con 0x7f56540d07c0 2023-10-21T17:39:40.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.788+0000 7f5651ffb640 1 -- 172.21.15.53:0/3643639760 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f5644081020 con 0x7f56540d07c0 2023-10-21T17:39:40.790 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:40.790 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":253,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:39.692663+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:40.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 msgr2=0x7f564006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 0x7f564006d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56540cee90 tx=0x7f5648009040 comp rx=0 tx=0).stop 2023-10-21T17:39:40.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 msgr2=0x7f56540cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:40.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f5644009d90 tx=0x7f5644004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:40.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 shutdown_connections 2023-10-21T17:39:40.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f564006b0b0 0x7f564006d570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565415cdf0 0x7f56540cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56540d1640 0x7f565409d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 --2- 172.21.15.53:0/3643639760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56540d07c0 0x7f56540cabc0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:40.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 >> 172.21.15.53:0/3643639760 conn(0x7f565409bcb0 msgr2=0x7f565415f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:40.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 shutdown_connections 2023-10-21T17:39:40.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:40.791+0000 7f565a5a7640 1 -- 172.21.15.53:0/3643639760 wait complete. 2023-10-21T17:39:40.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.036+0000 7ff1e3fff640 1 Processor -- start 2023-10-21T17:39:41.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.036+0000 7ff1e3fff640 1 -- start start 2023-10-21T17:39:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.037+0000 7ff1e3fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 0x7ff1e40cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.037+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.037+0000 7ff1e3fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e415d040 con 0x7ff1e40d16f0 2023-10-21T17:39:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.037+0000 7ff1e3fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e40d0650 con 0x7ff1e415cc60 2023-10-21T17:39:41.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.037+0000 7ff1e3fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40d0930 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.038+0000 7ff1e37fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.040 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.038+0000 7ff1e37fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:60468/0 (socket says 172.21.15.53:60468) 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.038+0000 7ff1e37fe640 1 -- 172.21.15.53:0/526337113 learned_addr learned my addr 172.21.15.53:0/526337113 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.038+0000 7ff1e2ffd640 1 --2- 172.21.15.53:0/526337113 >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 0x7ff1e40cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.039+0000 7ff1e37fe640 1 -- 172.21.15.53:0/526337113 >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 msgr2=0x7ff1e40cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.039+0000 7ff1e37fe640 1 --2- 172.21.15.53:0/526337113 >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 0x7ff1e40cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.039+0000 7ff1e37fe640 1 -- 172.21.15.53:0/526337113 >> v1:172.21.15.124:6789/0 conn(0x7ff1e40d16f0 legacy=0x7ff1e40cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.041 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.039+0000 7ff1e37fe640 1 -- 172.21.15.53:0/526337113 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40d0c10 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.039+0000 7ff1e2ffd640 1 --2- 172.21.15.53:0/526337113 >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 0x7ff1e40cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.040+0000 7ff1e37fe640 1 --2- 172.21.15.53:0/526337113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1190a43819c7a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e27fc640 1 -- 172.21.15.53:0/526337113 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc02cb00 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e27fc640 1 -- 172.21.15.53:0/526337113 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc02cc60 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e27fc640 1 -- 172.21.15.53:0/526337113 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0343f0 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 -- 172.21.15.53:0/526337113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 msgr2=0x7ff1e40cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/526337113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc02bf60 comp rx=0 tx=0).stop 2023-10-21T17:39:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 -- 172.21.15.53:0/526337113 shutdown_connections 2023-10-21T17:39:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/526337113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e40cf880 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/526337113 >> v2:172.21.15.124:3300/0 conn(0x7ff1e415cc60 0x7ff1e40cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 -- 172.21.15.53:0/526337113 >> 172.21.15.53:0/526337113 conn(0x7ff1e409bcf0 msgr2=0x7ff1e40d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 -- 172.21.15.53:0/526337113 shutdown_connections 2023-10-21T17:39:41.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.041+0000 7ff1e3fff640 1 -- 172.21.15.53:0/526337113 wait complete. 2023-10-21T17:39:41.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.043+0000 7ff1e3fff640 1 Processor -- start 2023-10-21T17:39:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.043+0000 7ff1e3fff640 1 -- start start 2023-10-21T17:39:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e41559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40d16f0 0x7ff1e41f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.047 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41f64f0 con 0x7ff1e415cc60 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e41f38e0 con 0x7ff1e40d16f0 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e3fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e4155f30 con 0x7ff1e40ce6e0 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.044+0000 7ff1e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44876/0 (socket says 172.21.15.53:44876) 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 -- 172.21.15.53:0/2353563245 learned_addr learned my addr 172.21.15.53:0/2353563245 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:41.048 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 -- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 msgr2=0x7ff1e41559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e41559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 -- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40d16f0 msgr2=0x7ff1e41f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40d16f0 0x7ff1e41f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 -- 172.21.15.53:0/2353563245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009b30 con 0x7ff1e415cc60 2023-10-21T17:39:41.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.045+0000 7ff1e37fe640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.046+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc009510 con 0x7ff1e415cc60 2023-10-21T17:39:41.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.046+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc03e450 con 0x7ff1e415cc60 2023-10-21T17:39:41.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.046+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc03e800 con 0x7ff1e415cc60 2023-10-21T17:39:41.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.046+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e415ee60 con 0x7ff1e415cc60 2023-10-21T17:39:41.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.046+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e41f3570 con 0x7ff1e415cc60 2023-10-21T17:39:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.047+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e41f30d0 con 0x7ff1e415cc60 2023-10-21T17:39:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.048+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff1dc03e960 con 0x7ff1e415cc60 2023-10-21T17:39:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.050+0000 7ff1dbfff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 0x7ff1c406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.050+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff1dc0ac8a0 con 0x7ff1e415cc60 2023-10-21T17:39:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.050+0000 7ff1e2ffd640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 0x7ff1c406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.053 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.051+0000 7ff1e2ffd640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 0x7ff1c406d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006ea0 tx=0x7ff1d4008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.055 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.054+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1dc0775e0 con 0x7ff1e415cc60 2023-10-21T17:39:41.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.255+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1e41f3a50 con 0x7ff1e415cc60 2023-10-21T17:39:41.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.260+0000 7ff1dbfff640 1 -- 172.21.15.53:0/2353563245 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+16958 (secure 0 0 0) 0x7ff1dc07ac00 con 0x7ff1e415cc60 2023-10-21T17:39:41.261 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:41.262 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":254,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:41.240488+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:41.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 msgr2=0x7ff1c406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 0x7ff1c406d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4006ea0 tx=0x7ff1d4008040 comp rx=0 tx=0).stop 2023-10-21T17:39:41.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 msgr2=0x7ff1e41f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009e50 tx=0x7ff1dc004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 shutdown_connections 2023-10-21T17:39:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff1c406b0e0 0x7ff1c406d5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1e415cc60 0x7ff1e41f5af0 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.262+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1e40d16f0 0x7ff1e41f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.263+0000 7ff1e3fff640 1 --2- 172.21.15.53:0/2353563245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff1e40ce6e0 0x7ff1e41559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.263+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 >> 172.21.15.53:0/2353563245 conn(0x7ff1e409bcf0 msgr2=0x7ff1e40b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:41.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.263+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 shutdown_connections 2023-10-21T17:39:41.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.263+0000 7ff1e3fff640 1 -- 172.21.15.53:0/2353563245 wait complete. 2023-10-21T17:39:41.284 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:39:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.514+0000 7f2b7c5cc640 1 Processor -- start 2023-10-21T17:39:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.514+0000 7f2b7c5cc640 1 -- start start 2023-10-21T17:39:41.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.515+0000 7f2b7c5cc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b7406dfc0 0x7f2b74070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.515+0000 7f2b7c5cc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2b74071070 0x7f2b740cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.515+0000 7f2b7c5cc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b740b1cc0 con 0x7f2b740afbe0 2023-10-21T17:39:41.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b7c5cc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b740b2620 con 0x7f2b7406dfc0 2023-10-21T17:39:41.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b7c5cc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b740b2060 con 0x7f2b74071070 2023-10-21T17:39:41.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b7406dfc0 0x7f2b74070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2b7406dfc0 0x7f2b74070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44886/0 (socket says 172.21.15.53:44886) 2023-10-21T17:39:41.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 -- 172.21.15.53:0/3977134615 learned_addr learned my addr 172.21.15.53:0/3977134615 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:41.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 -- 172.21.15.53:0/3977134615 >> v2:172.21.15.124:3300/0 conn(0x7f2b74071070 msgr2=0x7f2b740cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:41.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 --2- 172.21.15.53:0/3977134615 >> v2:172.21.15.124:3300/0 conn(0x7f2b74071070 0x7f2b740cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.516+0000 7f2b79b67640 1 -- 172.21.15.53:0/3977134615 >> v1:172.21.15.124:6789/0 conn(0x7f2b740afbe0 legacy=0x7f2b740b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:41.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.517+0000 7f2b79b67640 1 -- 172.21.15.53:0/3977134615 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b740b2340 con 0x7f2b7406dfc0 2023-10-21T17:39:41.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.518+0000 7f2b79b67640 1 --2- 172.21.15.53:0/3977134615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b7406dfc0 0x7f2b74070b30 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f2b740c8d10 tx=0x7f2b6402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69b68b9cf863c44 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.518+0000 7f2b79366640 1 -- 172.21.15.53:0/3977134615 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64031890 con 0x7f2b7406dfc0 2023-10-21T17:39:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.518+0000 7f2b79366640 1 -- 172.21.15.53:0/3977134615 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b640319f0 con 0x7f2b7406dfc0 2023-10-21T17:39:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.518+0000 7f2b79366640 1 -- 172.21.15.53:0/3977134615 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64031d80 con 0x7f2b7406dfc0 2023-10-21T17:39:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3977134615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b7406dfc0 msgr2=0x7f2b74070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3977134615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b7406dfc0 0x7f2b74070b30 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f2b740c8d10 tx=0x7f2b6402b930 comp rx=0 tx=0).stop 2023-10-21T17:39:41.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3977134615 shutdown_connections 2023-10-21T17:39:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3977134615 >> v2:172.21.15.124:3300/0 conn(0x7f2b74071070 0x7f2b740cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3977134615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b7406dfc0 0x7f2b74070b30 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3977134615 >> 172.21.15.53:0/3977134615 conn(0x7f2b7406a5d0 msgr2=0x7f2b740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3977134615 shutdown_connections 2023-10-21T17:39:41.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.519+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3977134615 wait complete. 2023-10-21T17:39:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.521+0000 7f2b7c5cc640 1 Processor -- start 2023-10-21T17:39:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.521+0000 7f2b7c5cc640 1 -- start start 2023-10-21T17:39:41.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 0x7f2b74149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 0x7f2b741f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74150fc0 con 0x7f2b74071070 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741f6c00 con 0x7f2b7406dfc0 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7c5cc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b741f6ee0 con 0x7f2b740afbe0 2023-10-21T17:39:41.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b79b67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b79b67640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44894/0 (socket says 172.21.15.53:44894) 2023-10-21T17:39:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b79b67640 1 -- 172.21.15.53:0/3535751296 learned_addr learned my addr 172.21.15.53:0/3535751296 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.522+0000 7f2b7a368640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 0x7f2b74149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 -- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 msgr2=0x7f2b741f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b7ab69640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 0x7f2b741f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 0x7f2b741f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 -- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 msgr2=0x7f2b74149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 0x7f2b74149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 -- 172.21.15.53:0/3535751296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b640094d0 con 0x7f2b74071070 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b7a368640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 0x7f2b74149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:41.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b7ab69640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 0x7f2b741f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:41.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b79b67640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000c00 tx=0x7f2b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6403acb0 con 0x7f2b74071070 2023-10-21T17:39:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.523+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6403ae10 con 0x7f2b74071070 2023-10-21T17:39:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.524+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b74151210 con 0x7f2b74071070 2023-10-21T17:39:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.524+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64044610 con 0x7f2b74071070 2023-10-21T17:39:41.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.524+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b740c5140 con 0x7f2b74071070 2023-10-21T17:39:41.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.525+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b74151730 con 0x7f2b74071070 2023-10-21T17:39:41.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.526+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2b6404d450 con 0x7f2b74071070 2023-10-21T17:39:41.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.527+0000 7f2b6affd640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 0x7f2b5806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:41.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.527+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2b640ac520 con 0x7f2b74071070 2023-10-21T17:39:41.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.528+0000 7f2b7a368640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 0x7f2b5806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:41.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.529+0000 7f2b7a368640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 0x7f2b5806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b70007e90 tx=0x7f2b700073a0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:41.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.532+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b74151730 con 0x7f2b74071070 2023-10-21T17:39:41.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:41.768+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2b74151730 con 0x7f2b74071070 2023-10-21T17:39:42.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.373+0000 7f2b6affd640 1 -- 172.21.15.53:0/3535751296 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v127) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2b74151730 con 0x7f2b74071070 2023-10-21T17:39:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.376+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 msgr2=0x7f2b5806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 0x7f2b5806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b70007e90 tx=0x7f2b700073a0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 msgr2=0x7f2b74151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000c00 tx=0x7f2b640046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 shutdown_connections 2023-10-21T17:39:42.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2b5806b0b0 0x7f2b5806d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b740afbe0 0x7f2b741f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b74071070 0x7f2b74151a10 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 --2- 172.21.15.53:0/3535751296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b7406dfc0 0x7f2b74149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 >> 172.21.15.53:0/3535751296 conn(0x7f2b7406a5d0 msgr2=0x7f2b740c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:42.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 shutdown_connections 2023-10-21T17:39:42.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.377+0000 7f2b7c5cc640 1 -- 172.21.15.53:0/3535751296 wait complete. 2023-10-21T17:39:42.400 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:39:42.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:39:42.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.628+0000 7f5f434ad640 1 Processor -- start 2023-10-21T17:39:42.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.628+0000 7f5f434ad640 1 -- start start 2023-10-21T17:39:42.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.630+0000 7f5f434ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.630+0000 7f5f434ad640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f3c0ca680 0x7f5f3c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.630+0000 7f5f434ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0cab60 con 0x7f5f3c15ac00 2023-10-21T17:39:42.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.630+0000 7f5f434ad640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0cb960 con 0x7f5f3c0ca680 2023-10-21T17:39:42.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.630+0000 7f5f434ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0cbdf0 con 0x7f5f3c15a820 2023-10-21T17:39:42.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.631+0000 7f5f424ab640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.631+0000 7f5f424ab640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:60500/0 (socket says 172.21.15.53:60500) 2023-10-21T17:39:42.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.631+0000 7f5f424ab640 1 -- 172.21.15.53:0/2918336016 learned_addr learned my addr 172.21.15.53:0/2918336016 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:42.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.631+0000 7f5f41caa640 1 --2- 172.21.15.53:0/2918336016 >> v2:172.21.15.124:3300/0 conn(0x7f5f3c0ca680 0x7f5f3c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.632+0000 7f5f424ab640 1 -- 172.21.15.53:0/2918336016 >> v2:172.21.15.124:3300/0 conn(0x7f5f3c0ca680 msgr2=0x7f5f3c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:42.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.632+0000 7f5f424ab640 1 --2- 172.21.15.53:0/2918336016 >> v2:172.21.15.124:3300/0 conn(0x7f5f3c0ca680 0x7f5f3c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.632+0000 7f5f424ab640 1 -- 172.21.15.53:0/2918336016 >> v1:172.21.15.124:6789/0 conn(0x7f5f3c15ac00 legacy=0x7f5f3c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.632+0000 7f5f424ab640 1 -- 172.21.15.53:0/2918336016 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0cb030 con 0x7f5f3c15a820 2023-10-21T17:39:42.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.633+0000 7f5f424ab640 1 --2- 172.21.15.53:0/2918336016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009870 tx=0x7f5f3802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22a6c6e35c36c55 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:42.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.634+0000 7f5f414a9640 1 -- 172.21.15.53:0/2918336016 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38031640 con 0x7f5f3c15a820 2023-10-21T17:39:42.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.634+0000 7f5f414a9640 1 -- 172.21.15.53:0/2918336016 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380317a0 con 0x7f5f3c15a820 2023-10-21T17:39:42.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.634+0000 7f5f414a9640 1 -- 172.21.15.53:0/2918336016 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38031b30 con 0x7f5f3c15a820 2023-10-21T17:39:42.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.634+0000 7f5f434ad640 1 -- 172.21.15.53:0/2918336016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 msgr2=0x7f5f3c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.634+0000 7f5f434ad640 1 --2- 172.21.15.53:0/2918336016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009870 tx=0x7f5f3802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 -- 172.21.15.53:0/2918336016 shutdown_connections 2023-10-21T17:39:42.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 --2- 172.21.15.53:0/2918336016 >> v2:172.21.15.124:3300/0 conn(0x7f5f3c0ca680 0x7f5f3c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 --2- 172.21.15.53:0/2918336016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15a820 0x7f5f3c0aff70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 -- 172.21.15.53:0/2918336016 >> 172.21.15.53:0/2918336016 conn(0x7f5f3c06c050 msgr2=0x7f5f3c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:42.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 -- 172.21.15.53:0/2918336016 shutdown_connections 2023-10-21T17:39:42.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.635+0000 7f5f434ad640 1 -- 172.21.15.53:0/2918336016 wait complete. 2023-10-21T17:39:42.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.637+0000 7f5f434ad640 1 Processor -- start 2023-10-21T17:39:42.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.637+0000 7f5f434ad640 1 -- start start 2023-10-21T17:39:42.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.638+0000 7f5f434ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f3c0ca680 0x7f5f3c14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.638+0000 7f5f434ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f434ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15ac00 0x7f5f3c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f434ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0ae610 con 0x7f5f3c15a820 2023-10-21T17:39:42.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f434ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c15bfe0 con 0x7f5f3c0ca680 2023-10-21T17:39:42.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f434ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0adc70 con 0x7f5f3c15ac00 2023-10-21T17:39:42.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44902/0 (socket says 172.21.15.53:44902) 2023-10-21T17:39:42.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 -- 172.21.15.53:0/807955678 learned_addr learned my addr 172.21.15.53:0/807955678 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:42.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f42cac640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15ac00 0x7f5f3c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 -- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15ac00 msgr2=0x7f5f3c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15ac00 0x7f5f3c0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f424ab640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f3c0ca680 0x7f5f3c14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.639+0000 7f5f41caa640 1 -- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f3c0ca680 msgr2=0x7f5f3c14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:42.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f41caa640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f3c0ca680 0x7f5f3c14d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:42.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f41caa640 1 -- 172.21.15.53:0/807955678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380094d0 con 0x7f5f3c15a820 2023-10-21T17:39:42.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f41caa640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d5f0 tx=0x7f5f2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:42.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0125f0 con 0x7f5f3c15a820 2023-10-21T17:39:42.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c012750 con 0x7f5f3c15a820 2023-10-21T17:39:42.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c012b00 con 0x7f5f3c15a820 2023-10-21T17:39:42.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c0adf50 con 0x7f5f3c15a820 2023-10-21T17:39:42.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.640+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c14dd90 con 0x7f5f3c15a820 2023-10-21T17:39:42.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.642+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c074150 con 0x7f5f3c15a820 2023-10-21T17:39:42.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.642+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5f2c012c60 con 0x7f5f3c15a820 2023-10-21T17:39:42.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.643+0000 7f5f32ffd640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 0x7f5f2006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:42.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.644+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5f2c08a1d0 con 0x7f5f3c15a820 2023-10-21T17:39:42.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.644+0000 7f5f424ab640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 0x7f5f2006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:42.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.646+0000 7f5f424ab640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 0x7f5f2006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f38036000 tx=0x7f5f3803c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:42.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.648+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f2c054cc0 con 0x7f5f3c15a820 2023-10-21T17:39:42.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:42.884+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5f3c0b2230 con 0x7f5f3c15a820 2023-10-21T17:39:43.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.383+0000 7f5f32ffd640 1 -- 172.21.15.53:0/807955678 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v128) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f2c0582e0 con 0x7f5f3c15a820 2023-10-21T17:39:43.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.386+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 msgr2=0x7f5f2006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.386+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 0x7f5f2006d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f38036000 tx=0x7f5f3803c040 comp rx=0 tx=0).stop 2023-10-21T17:39:43.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.386+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 msgr2=0x7f5f3c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.386+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00d5f0 tx=0x7f5f2c00dab0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 shutdown_connections 2023-10-21T17:39:43.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f2006b0b0 0x7f5f2006d570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f3c15ac00 0x7f5f3c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f3c15a820 0x7f5f3c0cc6e0 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 --2- 172.21.15.53:0/807955678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f3c0ca680 0x7f5f3c14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 >> 172.21.15.53:0/807955678 conn(0x7f5f3c06c050 msgr2=0x7f5f3c0c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:43.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 shutdown_connections 2023-10-21T17:39:43.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.387+0000 7f5f434ad640 1 -- 172.21.15.53:0/807955678 wait complete. 2023-10-21T17:39:43.410 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:39:43.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:43.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.635+0000 7f51df51d640 1 Processor -- start 2023-10-21T17:39:43.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.635+0000 7f51df51d640 1 -- start start 2023-10-21T17:39:43.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f51d815cd90 0x7f51d80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51d8160940 0x7f51d8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8070d50 0x7f51d80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80c4d80 con 0x7f51d8160940 2023-10-21T17:39:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80c48f0 con 0x7f51d815cd90 2023-10-21T17:39:43.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.637+0000 7f51df51d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80c3ce0 con 0x7f51d8070d50 2023-10-21T17:39:43.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51d8160940 0x7f51d8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:43.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f51d8160940 0x7f51d8070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44914/0 (socket says 172.21.15.53:44914) 2023-10-21T17:39:43.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 -- 172.21.15.53:0/3441401661 learned_addr learned my addr 172.21.15.53:0/3441401661 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:43.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 -- 172.21.15.53:0/3441401661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8070d50 msgr2=0x7f51d80cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:43.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 --2- 172.21.15.53:0/3441401661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8070d50 0x7f51d80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 -- 172.21.15.53:0/3441401661 >> v2:172.21.15.124:3300/0 conn(0x7f51d815cd90 msgr2=0x7f51d80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:43.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 --2- 172.21.15.53:0/3441401661 >> v2:172.21.15.124:3300/0 conn(0x7f51d815cd90 0x7f51d80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.638+0000 7f51dcab8640 1 -- 172.21.15.53:0/3441401661 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80c3fc0 con 0x7f51d8160940 2023-10-21T17:39:43.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.639+0000 7f51dcab8640 1 --2- 172.21.15.53:0/3441401661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8160940 0x7f51d8070810 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f51d806e440 tx=0x7f51c802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9d3db55a09821c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.639+0000 7f51cffff640 1 -- 172.21.15.53:0/3441401661 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8031880 con 0x7f51d8160940 2023-10-21T17:39:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51cffff640 1 -- 172.21.15.53:0/3441401661 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80319e0 con 0x7f51d8160940 2023-10-21T17:39:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51cffff640 1 -- 172.21.15.53:0/3441401661 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8031d70 con 0x7f51d8160940 2023-10-21T17:39:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 -- 172.21.15.53:0/3441401661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8160940 msgr2=0x7f51d8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 --2- 172.21.15.53:0/3441401661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8160940 0x7f51d8070810 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f51d806e440 tx=0x7f51c802b920 comp rx=0 tx=0).stop 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 -- 172.21.15.53:0/3441401661 shutdown_connections 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 --2- 172.21.15.53:0/3441401661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8070d50 0x7f51d80cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 --2- 172.21.15.53:0/3441401661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8160940 0x7f51d8070810 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 --2- 172.21.15.53:0/3441401661 >> v2:172.21.15.124:3300/0 conn(0x7f51d815cd90 0x7f51d80702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 -- 172.21.15.53:0/3441401661 >> 172.21.15.53:0/3441401661 conn(0x7f51d806a380 msgr2=0x7f51d80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:43.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 -- 172.21.15.53:0/3441401661 shutdown_connections 2023-10-21T17:39:43.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.640+0000 7f51df51d640 1 -- 172.21.15.53:0/3441401661 wait complete. 2023-10-21T17:39:43.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.642+0000 7f51df51d640 1 Processor -- start 2023-10-21T17:39:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.642+0000 7f51df51d640 1 -- start start 2023-10-21T17:39:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51d815cd90 0x7f51d80c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8160940 0x7f51d80c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80c6e00 con 0x7f51d8070d50 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d815e830 con 0x7f51d815cd90 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51df51d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d81468f0 con 0x7f51d8160940 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51dd2b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51dd2b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44916/0 (socket says 172.21.15.53:44916) 2023-10-21T17:39:43.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.643+0000 7f51dd2b9640 1 -- 172.21.15.53:0/1279066064 learned_addr learned my addr 172.21.15.53:0/1279066064 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 -- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8160940 msgr2=0x7f51d80c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8160940 0x7f51d80c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 -- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51d815cd90 msgr2=0x7f51d80c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51d815cd90 0x7f51d80c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 -- 172.21.15.53:0/1279066064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c80094d0 con 0x7f51d8070d50 2023-10-21T17:39:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51dd2b9640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b4a0 tx=0x7f51d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4010610 con 0x7f51d8070d50 2023-10-21T17:39:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4010770 con 0x7f51d8070d50 2023-10-21T17:39:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4010b20 con 0x7f51d8070d50 2023-10-21T17:39:43.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.644+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d80c7430 con 0x7f51d8070d50 2023-10-21T17:39:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.645+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d815de90 con 0x7f51d8070d50 2023-10-21T17:39:43.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.646+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d8067490 con 0x7f51d8070d50 2023-10-21T17:39:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.647+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f51d4010c80 con 0x7f51d8070d50 2023-10-21T17:39:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.648+0000 7f51cdffb640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 0x7f51bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.648+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f51d4089870 con 0x7f51d8070d50 2023-10-21T17:39:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.648+0000 7f51dcab8640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 0x7f51bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:43.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.650+0000 7f51dcab8640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 0x7f51bc06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51c8036000 tx=0x7f51c8030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:43.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.652+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51d40543f0 con 0x7f51d8070d50 2023-10-21T17:39:43.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.884+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d80cbe70 con 0x7f51d8070d50 2023-10-21T17:39:43.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.885+0000 7f51cdffb640 1 -- 172.21.15.53:0/1279066064 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4934 (secure 0 0 0) 0x7f51d4057a10 con 0x7f51d8070d50 2023-10-21T17:39:43.887 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:43.887 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":38345,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-21T17:39:43.888 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 128 2023-10-21T17:39:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 msgr2=0x7f51bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 0x7f51bc06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51c8036000 tx=0x7f51c8030000 comp rx=0 tx=0).stop 2023-10-21T17:39:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 msgr2=0x7f51d81463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f51d400b4a0 tx=0x7f51d400b960 comp rx=0 tx=0).stop 2023-10-21T17:39:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 shutdown_connections 2023-10-21T17:39:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f51bc06b0b0 0x7f51bc06d570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51d8160940 0x7f51d80c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f51d815cd90 0x7f51d80c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 --2- 172.21.15.53:0/1279066064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d8070d50 0x7f51d81463b0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 >> 172.21.15.53:0/1279066064 conn(0x7f51d806a380 msgr2=0x7f51d80cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.888+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 shutdown_connections 2023-10-21T17:39:43.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:43.889+0000 7f51df51d640 1 -- 172.21.15.53:0/1279066064 wait complete. 2023-10-21T17:39:43.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:44.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.133+0000 7ff62f444640 1 Processor -- start 2023-10-21T17:39:44.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.133+0000 7ff62f444640 1 -- start start 2023-10-21T17:39:44.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62f444640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff62815cd90 0x7ff6280702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62f444640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280c48f0 con 0x7ff628160940 2023-10-21T17:39:44.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62f444640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280c3ce0 con 0x7ff62806c890 2023-10-21T17:39:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62f444640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280c4d80 con 0x7ff62815cd90 2023-10-21T17:39:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62c9df640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff628160940 0x7ff628070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36198/0 (socket says 172.21.15.53:36198) 2023-10-21T17:39:44.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.135+0000 7ff62c9df640 1 -- 172.21.15.53:0/2112044663 learned_addr learned my addr 172.21.15.53:0/2112044663 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/2112044663 >> v2:172.21.15.124:3300/0 conn(0x7ff62815cd90 0x7ff6280702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298209745 0 0) 0x7ff6280c48f0 con 0x7ff628160940 2023-10-21T17:39:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003520 con 0x7ff628160940 2023-10-21T17:39:44.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3453911201 0 0) 0x7ff614003520 con 0x7ff628160940 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 >> v2:172.21.15.124:3300/0 conn(0x7ff62815cd90 msgr2=0x7ff6280702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 --2- 172.21.15.53:0/2112044663 >> v2:172.21.15.124:3300/0 conn(0x7ff62815cd90 0x7ff6280702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 >> v1:172.21.15.124:6789/0 conn(0x7ff62806c890 legacy=0x7ff62806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280c3fc0 con 0x7ff628160940 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff618001670 con 0x7ff628160940 2023-10-21T17:39:44.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 >> v1:172.21.15.53:6789/0 conn(0x7ff628160940 legacy=0x7ff628070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 0x7ff614003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6140047c0 0x7ff614004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff614005180 0x7ff61400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c050 con 0x7ff6140039c0 2023-10-21T17:39:44.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c1c0 con 0x7ff6140047c0 2023-10-21T17:39:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.136+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c380 con 0x7ff614005180 2023-10-21T17:39:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 0x7ff614003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62c9df640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6140047c0 0x7ff614004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 -- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff614005180 msgr2=0x7ff61400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:44.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff614005180 0x7ff61400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 -- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6140047c0 msgr2=0x7ff614004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6140047c0 0x7ff614004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d1e0640 1 -- 172.21.15.53:0/2112044663 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614003520 con 0x7ff6140039c0 2023-10-21T17:39:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62d9e1640 1 -- 172.21.15.53:0/2112044663 reap_dead start 2023-10-21T17:39:44.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.137+0000 7ff62c9df640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6140047c0 0x7ff614004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.138+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 0x7ff614003de0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7ff624009ac0 tx=0x7ff6140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df6c9b7928fe7fd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62402c6d0 con 0x7ff6140039c0 2023-10-21T17:39:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62402c830 con 0x7ff6140039c0 2023-10-21T17:39:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff61ffff640 1 -- 172.21.15.53:0/2112044663 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62402cbc0 con 0x7ff6140039c0 2023-10-21T17:39:44.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff62f444640 1 -- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 msgr2=0x7ff614003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff62f444640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 0x7ff614003de0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7ff624009ac0 tx=0x7ff6140057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff62f444640 1 -- 172.21.15.53:0/2112044663 shutdown_connections 2023-10-21T17:39:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.139+0000 7ff62f444640 1 --2- 172.21.15.53:0/2112044663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6140039c0 0x7ff614003de0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.140+0000 7ff62f444640 1 -- 172.21.15.53:0/2112044663 >> 172.21.15.53:0/2112044663 conn(0x7ff62806a380 msgr2=0x7ff6280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:44.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.140+0000 7ff62f444640 1 -- 172.21.15.53:0/2112044663 shutdown_connections 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.140+0000 7ff62f444640 1 -- 172.21.15.53:0/2112044663 wait complete. 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.141+0000 7ff62f444640 1 Processor -- start 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.142+0000 7ff62f444640 1 -- start start 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.142+0000 7ff62f444640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6140039c0 0x7ff6280b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.142+0000 7ff62f444640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6280b1480 0x7ff628154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62f444640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62f444640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b1a40 con 0x7ff6280c74b0 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62f444640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b1d20 con 0x7ff6280b1480 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62f444640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b2000 con 0x7ff6140039c0 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54470/0 (socket says 172.21.15.53:54470) 2023-10-21T17:39:44.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 -- 172.21.15.53:0/3264952866 learned_addr learned my addr 172.21.15.53:0/3264952866 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6140039c0 0x7ff6280b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62c9df640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6280b1480 0x7ff628154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 -- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6140039c0 msgr2=0x7ff6280b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6140039c0 0x7ff6280b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 -- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6280b1480 msgr2=0x7ff628154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6280b1480 0x7ff628154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.143+0000 7ff62d9e1640 1 -- 172.21.15.53:0/3264952866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280b22e0 con 0x7ff6280c74b0 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff62d9e1640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b0c20 tx=0x7ff62000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620005d40 con 0x7ff6280c74b0 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000ced0 con 0x7ff6280c74b0 2023-10-21T17:39:44.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620005ea0 con 0x7ff6280c74b0 2023-10-21T17:39:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6280b25c0 con 0x7ff6280c74b0 2023-10-21T17:39:44.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.144+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6280b3780 con 0x7ff6280c74b0 2023-10-21T17:39:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.145+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280b27b0 con 0x7ff6280c74b0 2023-10-21T17:39:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.146+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff62000a850 con 0x7ff6280c74b0 2023-10-21T17:39:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.147+0000 7ff61dffb640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 0x7ff60c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.147+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff620088ab0 con 0x7ff6280c74b0 2023-10-21T17:39:44.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.148+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 0x7ff60c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.149+0000 7ff62d1e0640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 0x7ff60c06d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff624036000 tx=0x7ff624030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.152+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6200535a0 con 0x7ff6280c74b0 2023-10-21T17:39:44.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.345+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff628067490 con 0x7ff6280c74b0 2023-10-21T17:39:44.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.348+0000 7ff61dffb640 1 -- 172.21.15.53:0/3264952866 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+16958 (secure 0 0 0) 0x7ff620056bc0 con 0x7ff6280c74b0 2023-10-21T17:39:44.350 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:44.350 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":254,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:41.240488+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:44.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 msgr2=0x7ff60c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 0x7ff60c06d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff624036000 tx=0x7ff624030000 comp rx=0 tx=0).stop 2023-10-21T17:39:44.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 msgr2=0x7ff6280b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b0c20 tx=0x7ff62000cb90 comp rx=0 tx=0).stop 2023-10-21T17:39:44.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 shutdown_connections 2023-10-21T17:39:44.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff60c06b0e0 0x7ff60c06d5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6280c74b0 0x7ff6280b3240 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6280b1480 0x7ff628154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 --2- 172.21.15.53:0/3264952866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6140039c0 0x7ff6280b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 >> 172.21.15.53:0/3264952866 conn(0x7ff62806a380 msgr2=0x7ff6280afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 shutdown_connections 2023-10-21T17:39:44.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.351+0000 7ff62f444640 1 -- 172.21.15.53:0/3264952866 wait complete. 2023-10-21T17:39:44.369 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-21T17:39:44.367+0000 7fc88f001640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:39:44.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:39:44.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:39:44.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.593+0000 7f488b59e640 1 Processor -- start 2023-10-21T17:39:44.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.593+0000 7f488b59e640 1 -- start start 2023-10-21T17:39:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f488c15ab30 0x7f488c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f488c0adc70 0x7f488c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c15af10 0x7f488c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c15f7d0 con 0x7f488c15ab30 2023-10-21T17:39:44.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c15f210 con 0x7f488c0adc70 2023-10-21T17:39:44.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15f4f0 con 0x7f488c15af10 2023-10-21T17:39:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f488c15ab30 0x7f488c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488a59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f488c15ab30 0x7f488c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54480/0 (socket says 172.21.15.53:54480) 2023-10-21T17:39:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.595+0000 7f488a59c640 1 -- 172.21.15.53:0/3066574238 learned_addr learned my addr 172.21.15.53:0/3066574238 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f4889d9b640 1 --2- 172.21.15.53:0/3066574238 >> v2:172.21.15.124:3300/0 conn(0x7f488c0adc70 0x7f488c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488ad9d640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c15af10 0x7f488c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488a59c640 1 -- 172.21.15.53:0/3066574238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c15af10 msgr2=0x7f488c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488a59c640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c15af10 0x7f488c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488a59c640 1 -- 172.21.15.53:0/3066574238 >> v2:172.21.15.124:3300/0 conn(0x7f488c0adc70 msgr2=0x7f488c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488a59c640 1 --2- 172.21.15.53:0/3066574238 >> v2:172.21.15.124:3300/0 conn(0x7f488c0adc70 0x7f488c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.596+0000 7f488a59c640 1 -- 172.21.15.53:0/3066574238 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c06c550 con 0x7f488c15ab30 2023-10-21T17:39:44.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.597+0000 7f488a59c640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15ab30 0x7f488c15e8a0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c56c0 tx=0x7f487c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b052ff67748e9c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.597+0000 7f488959a640 1 -- 172.21.15.53:0/3066574238 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c031880 con 0x7f488c15ab30 2023-10-21T17:39:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488959a640 1 -- 172.21.15.53:0/3066574238 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c0319e0 con 0x7f488c15ab30 2023-10-21T17:39:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488959a640 1 -- 172.21.15.53:0/3066574238 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c031d70 con 0x7f488c15ab30 2023-10-21T17:39:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 -- 172.21.15.53:0/3066574238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15ab30 msgr2=0x7f488c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15ab30 0x7f488c15e8a0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c56c0 tx=0x7f487c02b920 comp rx=0 tx=0).stop 2023-10-21T17:39:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 -- 172.21.15.53:0/3066574238 shutdown_connections 2023-10-21T17:39:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c15af10 0x7f488c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 --2- 172.21.15.53:0/3066574238 >> v2:172.21.15.124:3300/0 conn(0x7f488c0adc70 0x7f488c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 --2- 172.21.15.53:0/3066574238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15ab30 0x7f488c15e8a0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 -- 172.21.15.53:0/3066574238 >> 172.21.15.53:0/3066574238 conn(0x7f488c0cb0c0 msgr2=0x7f488c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 -- 172.21.15.53:0/3066574238 shutdown_connections 2023-10-21T17:39:44.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.598+0000 7f488b59e640 1 -- 172.21.15.53:0/3066574238 wait complete. 2023-10-21T17:39:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.600+0000 7f488b59e640 1 Processor -- start 2023-10-21T17:39:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.600+0000 7f488b59e640 1 -- start start 2023-10-21T17:39:44.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 0x7f488c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f488c15ab30 0x7f488c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1f6900 con 0x7f488c15af10 2023-10-21T17:39:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1f6d20 con 0x7f488c15ab30 2023-10-21T17:39:44.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.601+0000 7f488b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0c8500 con 0x7f488c0adc70 2023-10-21T17:39:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54490/0 (socket says 172.21.15.53:54490) 2023-10-21T17:39:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 -- 172.21.15.53:0/4225214816 learned_addr learned my addr 172.21.15.53:0/4225214816 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488a59c640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 0x7f488c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f4889d9b640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f488c15ab30 0x7f488c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 -- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 msgr2=0x7f488c06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 0x7f488c06cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 -- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f488c15ab30 msgr2=0x7f488c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f488c15ab30 0x7f488c0c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488ad9d640 1 -- 172.21.15.53:0/4225214816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0094d0 con 0x7f488c15af10 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.602+0000 7f488a59c640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 0x7f488c06cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f488ad9d640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f4884007ec0 tx=0x7f488400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884014620 con 0x7f488c15af10 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48840147a0 con 0x7f488c15af10 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884014b50 con 0x7f488c15af10 2023-10-21T17:39:44.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c1f5f60 con 0x7f488c15af10 2023-10-21T17:39:44.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.603+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c1f6520 con 0x7f488c15af10 2023-10-21T17:39:44.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.604+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c04e160 con 0x7f488c15af10 2023-10-21T17:39:44.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.606+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4884014cb0 con 0x7f488c15af10 2023-10-21T17:39:44.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.606+0000 7f487affd640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 0x7f486c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:44.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.606+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f488408a420 con 0x7f488c15af10 2023-10-21T17:39:44.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.607+0000 7f488a59c640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 0x7f486c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:44.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.608+0000 7f488a59c640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 0x7f486c06d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f487c036000 tx=0x7f487c03c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:44.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.611+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4884055070 con 0x7f488c15af10 2023-10-21T17:39:44.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.800+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f488c06c830 con 0x7f486c06b0b0 2023-10-21T17:39:44.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.830+0000 7f487affd640 1 -- 172.21.15.53:0/4225214816 <== mgr.38150 v2:172.21.15.124:6832/3816427222 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287297 (secure 0 0 0) 0x7f487c044400 con 0x7f486c06b0b0 2023-10-21T17:39:44.835 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:44.837 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:39:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.835+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 msgr2=0x7f486c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.835+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 0x7f486c06d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f487c036000 tx=0x7f487c03c040 comp rx=0 tx=0).stop 2023-10-21T17:39:44.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 msgr2=0x7f488c0c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f4884007ec0 tx=0x7f488400e4b0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 shutdown_connections 2023-10-21T17:39:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f486c06b0b0 0x7f486c06d570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f488c15af10 0x7f488c0c7ee0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f488c15ab30 0x7f488c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 --2- 172.21.15.53:0/4225214816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f488c0adc70 0x7f488c06cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 >> 172.21.15.53:0/4225214816 conn(0x7f488c0cb0c0 msgr2=0x7f488c1589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 shutdown_connections 2023-10-21T17:39:44.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:44.836+0000 7f488b59e640 1 -- 172.21.15.53:0/4225214816 wait complete. 2023-10-21T17:39:44.856 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-21T17:39:44.538411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":671,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4933,"ondisk_log_size":4933,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453256,"kb_used_data":26132,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754521464,"statfs":{"total":773094113280,"available":772629979136,"internally_reserved":0,"allocated":26759168,"data_stored":14470548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":437243449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002215"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285868+0000","last_change":"2023-10-21T17:39:40.283996+0000","last_active":"2023-10-21T17:39:40.285868+0000","last_peered":"2023-10-21T17:39:40.285868+0000","last_clean":"2023-10-21T17:39:40.285868+0000","last_became_active":"2023-10-21T17:39:40.283690+0000","last_became_peered":"2023-10-21T17:39:40.283690+0000","last_unstale":"2023-10-21T17:39:40.285868+0000","last_undegraded":"2023-10-21T17:39:40.285868+0000","last_fullsized":"2023-10-21T17:39:40.285868+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:00:13.013899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249831+0000","last_change":"2023-10-21T17:39:40.279882+0000","last_active":"2023-10-21T17:39:41.249831+0000","last_peered":"2023-10-21T17:39:41.249831+0000","last_clean":"2023-10-21T17:39:41.249831+0000","last_became_active":"2023-10-21T17:39:40.279568+0000","last_became_peered":"2023-10-21T17:39:40.279568+0000","last_unstale":"2023-10-21T17:39:41.249831+0000","last_undegraded":"2023-10-21T17:39:41.249831+0000","last_fullsized":"2023-10-21T17:39:41.249831+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:07.488724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.282417+0000","last_change":"2023-10-21T17:39:40.277265+0000","last_active":"2023-10-21T17:39:40.282417+0000","last_peered":"2023-10-21T17:39:40.282417+0000","last_clean":"2023-10-21T17:39:40.282417+0000","last_became_active":"2023-10-21T17:39:40.276998+0000","last_became_peered":"2023-10-21T17:39:40.276998+0000","last_unstale":"2023-10-21T17:39:40.282417+0000","last_undegraded":"2023-10-21T17:39:40.282417+0000","last_fullsized":"2023-10-21T17:39:40.282417+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:53.267061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284282+0000","last_change":"2023-10-21T17:39:40.278307+0000","last_active":"2023-10-21T17:39:40.284282+0000","last_peered":"2023-10-21T17:39:40.284282+0000","last_clean":"2023-10-21T17:39:40.284282+0000","last_became_active":"2023-10-21T17:39:40.277780+0000","last_became_peered":"2023-10-21T17:39:40.277780+0000","last_unstale":"2023-10-21T17:39:40.284282+0000","last_undegraded":"2023-10-21T17:39:40.284282+0000","last_fullsized":"2023-10-21T17:39:40.284282+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:01.301844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284173+0000","last_change":"2023-10-21T17:39:40.277999+0000","last_active":"2023-10-21T17:39:40.284173+0000","last_peered":"2023-10-21T17:39:40.284173+0000","last_clean":"2023-10-21T17:39:40.284173+0000","last_became_active":"2023-10-21T17:39:40.277716+0000","last_became_peered":"2023-10-21T17:39:40.277716+0000","last_unstale":"2023-10-21T17:39:40.284173+0000","last_undegraded":"2023-10-21T17:39:40.284173+0000","last_fullsized":"2023-10-21T17:39:40.284173+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:36.785785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.279365+0000","last_change":"2023-10-21T17:39:40.274132+0000","last_active":"2023-10-21T17:39:40.279365+0000","last_peered":"2023-10-21T17:39:40.279365+0000","last_clean":"2023-10-21T17:39:40.279365+0000","last_became_active":"2023-10-21T17:39:40.273837+0000","last_became_peered":"2023-10-21T17:39:40.273837+0000","last_unstale":"2023-10-21T17:39:40.279365+0000","last_undegraded":"2023-10-21T17:39:40.279365+0000","last_fullsized":"2023-10-21T17:39:40.279365+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:36:58.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249224+0000","last_change":"2023-10-21T17:39:40.277476+0000","last_active":"2023-10-21T17:39:41.249224+0000","last_peered":"2023-10-21T17:39:41.249224+0000","last_clean":"2023-10-21T17:39:41.249224+0000","last_became_active":"2023-10-21T17:39:40.277175+0000","last_became_peered":"2023-10-21T17:39:40.277175+0000","last_unstale":"2023-10-21T17:39:41.249224+0000","last_undegraded":"2023-10-21T17:39:41.249224+0000","last_fullsized":"2023-10-21T17:39:41.249224+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:09.554848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280578+0000","last_change":"2023-10-21T17:39:40.274697+0000","last_active":"2023-10-21T17:39:40.280578+0000","last_peered":"2023-10-21T17:39:40.280578+0000","last_clean":"2023-10-21T17:39:40.280578+0000","last_became_active":"2023-10-21T17:39:40.274194+0000","last_became_peered":"2023-10-21T17:39:40.274194+0000","last_unstale":"2023-10-21T17:39:40.280578+0000","last_undegraded":"2023-10-21T17:39:40.280578+0000","last_fullsized":"2023-10-21T17:39:40.280578+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:10:17.428649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280436+0000","last_change":"2023-10-21T17:39:40.278661+0000","last_active":"2023-10-21T17:39:40.280436+0000","last_peered":"2023-10-21T17:39:40.280436+0000","last_clean":"2023-10-21T17:39:40.280436+0000","last_became_active":"2023-10-21T17:39:40.278377+0000","last_became_peered":"2023-10-21T17:39:40.278377+0000","last_unstale":"2023-10-21T17:39:40.280436+0000","last_undegraded":"2023-10-21T17:39:40.280436+0000","last_fullsized":"2023-10-21T17:39:40.280436+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:29.981635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285992+0000","last_change":"2023-10-21T17:39:40.284923+0000","last_active":"2023-10-21T17:39:40.285992+0000","last_peered":"2023-10-21T17:39:40.285992+0000","last_clean":"2023-10-21T17:39:40.285992+0000","last_became_active":"2023-10-21T17:39:40.281089+0000","last_became_peered":"2023-10-21T17:39:40.281089+0000","last_unstale":"2023-10-21T17:39:40.285992+0000","last_undegraded":"2023-10-21T17:39:40.285992+0000","last_fullsized":"2023-10-21T17:39:40.285992+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:51.348415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.282436+0000","last_change":"2023-10-21T17:39:40.280260+0000","last_active":"2023-10-21T17:39:40.282436+0000","last_peered":"2023-10-21T17:39:40.282436+0000","last_clean":"2023-10-21T17:39:40.282436+0000","last_became_active":"2023-10-21T17:39:40.279970+0000","last_became_peered":"2023-10-21T17:39:40.279970+0000","last_unstale":"2023-10-21T17:39:40.282436+0000","last_undegraded":"2023-10-21T17:39:40.282436+0000","last_fullsized":"2023-10-21T17:39:40.282436+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:21.956042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280270+0000","last_change":"2023-10-21T17:39:40.277448+0000","last_active":"2023-10-21T17:39:40.280270+0000","last_peered":"2023-10-21T17:39:40.280270+0000","last_clean":"2023-10-21T17:39:40.280270+0000","last_became_active":"2023-10-21T17:39:40.277121+0000","last_became_peered":"2023-10-21T17:39:40.277121+0000","last_unstale":"2023-10-21T17:39:40.280270+0000","last_undegraded":"2023-10-21T17:39:40.280270+0000","last_fullsized":"2023-10-21T17:39:40.280270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:07:55.678448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253028+0000","last_change":"2023-10-21T17:39:40.279922+0000","last_active":"2023-10-21T17:39:41.253028+0000","last_peered":"2023-10-21T17:39:41.253028+0000","last_clean":"2023-10-21T17:39:41.253028+0000","last_became_active":"2023-10-21T17:39:40.279029+0000","last_became_peered":"2023-10-21T17:39:40.279029+0000","last_unstale":"2023-10-21T17:39:41.253028+0000","last_undegraded":"2023-10-21T17:39:41.253028+0000","last_fullsized":"2023-10-21T17:39:41.253028+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:13.093898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.261700+0000","last_change":"2023-10-21T17:39:40.260107+0000","last_active":"2023-10-21T17:39:40.261700+0000","last_peered":"2023-10-21T17:39:40.261700+0000","last_clean":"2023-10-21T17:39:40.261700+0000","last_became_active":"2023-10-21T17:39:40.259164+0000","last_became_peered":"2023-10-21T17:39:40.259164+0000","last_unstale":"2023-10-21T17:39:40.261700+0000","last_undegraded":"2023-10-21T17:39:40.261700+0000","last_fullsized":"2023-10-21T17:39:40.261700+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:53.267061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280385+0000","last_change":"2023-10-21T17:39:40.278320+0000","last_active":"2023-10-21T17:39:40.280385+0000","last_peered":"2023-10-21T17:39:40.280385+0000","last_clean":"2023-10-21T17:39:40.280385+0000","last_became_active":"2023-10-21T17:39:40.277166+0000","last_became_peered":"2023-10-21T17:39:40.277166+0000","last_unstale":"2023-10-21T17:39:40.280385+0000","last_undegraded":"2023-10-21T17:39:40.280385+0000","last_fullsized":"2023-10-21T17:39:40.280385+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:05:08.170636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286858+0000","last_change":"2023-10-21T17:39:40.285094+0000","last_active":"2023-10-21T17:39:40.286858+0000","last_peered":"2023-10-21T17:39:40.286858+0000","last_clean":"2023-10-21T17:39:40.286858+0000","last_became_active":"2023-10-21T17:39:40.284749+0000","last_became_peered":"2023-10-21T17:39:40.284749+0000","last_unstale":"2023-10-21T17:39:40.286858+0000","last_undegraded":"2023-10-21T17:39:40.286858+0000","last_fullsized":"2023-10-21T17:39:40.286858+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:40:29.137446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.254037+0000","last_change":"2023-10-21T17:39:40.283462+0000","last_active":"2023-10-21T17:39:41.254037+0000","last_peered":"2023-10-21T17:39:41.254037+0000","last_clean":"2023-10-21T17:39:41.254037+0000","last_became_active":"2023-10-21T17:39:40.281926+0000","last_became_peered":"2023-10-21T17:39:40.281926+0000","last_unstale":"2023-10-21T17:39:41.254037+0000","last_undegraded":"2023-10-21T17:39:41.254037+0000","last_fullsized":"2023-10-21T17:39:41.254037+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:42:34.391024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285473+0000","last_change":"2023-10-21T17:39:40.280739+0000","last_active":"2023-10-21T17:39:40.285473+0000","last_peered":"2023-10-21T17:39:40.285473+0000","last_clean":"2023-10-21T17:39:40.285473+0000","last_became_active":"2023-10-21T17:39:40.280069+0000","last_became_peered":"2023-10-21T17:39:40.280069+0000","last_unstale":"2023-10-21T17:39:40.285473+0000","last_undegraded":"2023-10-21T17:39:40.285473+0000","last_fullsized":"2023-10-21T17:39:40.285473+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:12.232845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286444+0000","last_change":"2023-10-21T17:39:40.285848+0000","last_active":"2023-10-21T17:39:40.286444+0000","last_peered":"2023-10-21T17:39:40.286444+0000","last_clean":"2023-10-21T17:39:40.286444+0000","last_became_active":"2023-10-21T17:39:40.285405+0000","last_became_peered":"2023-10-21T17:39:40.285405+0000","last_unstale":"2023-10-21T17:39:40.286444+0000","last_undegraded":"2023-10-21T17:39:40.286444+0000","last_fullsized":"2023-10-21T17:39:40.286444+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250176+0000","last_change":"2023-10-21T17:39:40.284648+0000","last_active":"2023-10-21T17:39:41.250176+0000","last_peered":"2023-10-21T17:39:41.250176+0000","last_clean":"2023-10-21T17:39:41.250176+0000","last_became_active":"2023-10-21T17:39:40.284214+0000","last_became_peered":"2023-10-21T17:39:40.284214+0000","last_unstale":"2023-10-21T17:39:41.250176+0000","last_undegraded":"2023-10-21T17:39:41.250176+0000","last_fullsized":"2023-10-21T17:39:41.250176+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:52.495370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250151+0000","last_change":"2023-10-21T17:39:40.270709+0000","last_active":"2023-10-21T17:39:41.250151+0000","last_peered":"2023-10-21T17:39:41.250151+0000","last_clean":"2023-10-21T17:39:41.250151+0000","last_became_active":"2023-10-21T17:39:40.270369+0000","last_became_peered":"2023-10-21T17:39:40.270369+0000","last_unstale":"2023-10-21T17:39:41.250151+0000","last_undegraded":"2023-10-21T17:39:41.250151+0000","last_fullsized":"2023-10-21T17:39:41.250151+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253548+0000","last_change":"2023-10-21T17:39:40.267480+0000","last_active":"2023-10-21T17:39:41.253548+0000","last_peered":"2023-10-21T17:39:41.253548+0000","last_clean":"2023-10-21T17:39:41.253548+0000","last_became_active":"2023-10-21T17:39:40.267074+0000","last_became_peered":"2023-10-21T17:39:40.267074+0000","last_unstale":"2023-10-21T17:39:41.253548+0000","last_undegraded":"2023-10-21T17:39:41.253548+0000","last_fullsized":"2023-10-21T17:39:41.253548+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:05.724202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.287846+0000","last_change":"2023-10-21T17:39:40.287294+0000","last_active":"2023-10-21T17:39:40.287846+0000","last_peered":"2023-10-21T17:39:40.287846+0000","last_clean":"2023-10-21T17:39:40.287846+0000","last_became_active":"2023-10-21T17:39:40.286990+0000","last_became_peered":"2023-10-21T17:39:40.286990+0000","last_unstale":"2023-10-21T17:39:40.287846+0000","last_undegraded":"2023-10-21T17:39:40.287846+0000","last_fullsized":"2023-10-21T17:39:40.287846+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251199+0000","last_change":"2023-10-21T17:39:40.270092+0000","last_active":"2023-10-21T17:39:41.251199+0000","last_peered":"2023-10-21T17:39:41.251199+0000","last_clean":"2023-10-21T17:39:41.251199+0000","last_became_active":"2023-10-21T17:39:40.269725+0000","last_became_peered":"2023-10-21T17:39:40.269725+0000","last_unstale":"2023-10-21T17:39:41.251199+0000","last_undegraded":"2023-10-21T17:39:41.251199+0000","last_fullsized":"2023-10-21T17:39:41.251199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:18.601428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250830+0000","last_change":"2023-10-21T17:39:40.275179+0000","last_active":"2023-10-21T17:39:41.250830+0000","last_peered":"2023-10-21T17:39:41.250830+0000","last_clean":"2023-10-21T17:39:41.250830+0000","last_became_active":"2023-10-21T17:39:40.274025+0000","last_became_peered":"2023-10-21T17:39:40.274025+0000","last_unstale":"2023-10-21T17:39:41.250830+0000","last_undegraded":"2023-10-21T17:39:41.250830+0000","last_fullsized":"2023-10-21T17:39:41.250830+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:58.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251182+0000","last_change":"2023-10-21T17:39:40.271219+0000","last_active":"2023-10-21T17:39:41.251182+0000","last_peered":"2023-10-21T17:39:41.251182+0000","last_clean":"2023-10-21T17:39:41.251182+0000","last_became_active":"2023-10-21T17:39:40.270928+0000","last_became_peered":"2023-10-21T17:39:40.270928+0000","last_unstale":"2023-10-21T17:39:41.251182+0000","last_undegraded":"2023-10-21T17:39:41.251182+0000","last_fullsized":"2023-10-21T17:39:41.251182+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.281864+0000","last_change":"2023-10-21T17:39:40.276685+0000","last_active":"2023-10-21T17:39:40.281864+0000","last_peered":"2023-10-21T17:39:40.281864+0000","last_clean":"2023-10-21T17:39:40.281864+0000","last_became_active":"2023-10-21T17:39:40.276416+0000","last_became_peered":"2023-10-21T17:39:40.276416+0000","last_unstale":"2023-10-21T17:39:40.281864+0000","last_undegraded":"2023-10-21T17:39:40.281864+0000","last_fullsized":"2023-10-21T17:39:40.281864+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:14.398414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286126+0000","last_change":"2023-10-21T17:39:40.284175+0000","last_active":"2023-10-21T17:39:40.286126+0000","last_peered":"2023-10-21T17:39:40.286126+0000","last_clean":"2023-10-21T17:39:40.286126+0000","last_became_active":"2023-10-21T17:39:40.283773+0000","last_became_peered":"2023-10-21T17:39:40.283773+0000","last_unstale":"2023-10-21T17:39:40.286126+0000","last_undegraded":"2023-10-21T17:39:40.286126+0000","last_fullsized":"2023-10-21T17:39:40.286126+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:00.719344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.261079+0000","last_change":"2023-10-21T17:39:40.258467+0000","last_active":"2023-10-21T17:39:40.261079+0000","last_peered":"2023-10-21T17:39:40.261079+0000","last_clean":"2023-10-21T17:39:40.261079+0000","last_became_active":"2023-10-21T17:39:40.257660+0000","last_became_peered":"2023-10-21T17:39:40.257660+0000","last_unstale":"2023-10-21T17:39:40.261079+0000","last_undegraded":"2023-10-21T17:39:40.261079+0000","last_fullsized":"2023-10-21T17:39:40.261079+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:25.602312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.278613+0000","last_change":"2023-10-21T17:39:40.276636+0000","last_active":"2023-10-21T17:39:40.278613+0000","last_peered":"2023-10-21T17:39:40.278613+0000","last_clean":"2023-10-21T17:39:40.278613+0000","last_became_active":"2023-10-21T17:39:40.274303+0000","last_became_peered":"2023-10-21T17:39:40.274303+0000","last_unstale":"2023-10-21T17:39:40.278613+0000","last_undegraded":"2023-10-21T17:39:40.278613+0000","last_fullsized":"2023-10-21T17:39:40.278613+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:12.232845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250429+0000","last_change":"2023-10-21T17:39:40.270012+0000","last_active":"2023-10-21T17:39:41.250429+0000","last_peered":"2023-10-21T17:39:41.250429+0000","last_clean":"2023-10-21T17:39:41.250429+0000","last_became_active":"2023-10-21T17:39:40.269726+0000","last_became_peered":"2023-10-21T17:39:40.269726+0000","last_unstale":"2023-10-21T17:39:41.250429+0000","last_undegraded":"2023-10-21T17:39:41.250429+0000","last_fullsized":"2023-10-21T17:39:41.250429+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:57.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286209+0000","last_change":"2023-10-21T17:39:40.284346+0000","last_active":"2023-10-21T17:39:40.286209+0000","last_peered":"2023-10-21T17:39:40.286209+0000","last_clean":"2023-10-21T17:39:40.286209+0000","last_became_active":"2023-10-21T17:39:40.282055+0000","last_became_peered":"2023-10-21T17:39:40.282055+0000","last_unstale":"2023-10-21T17:39:40.286209+0000","last_undegraded":"2023-10-21T17:39:40.286209+0000","last_fullsized":"2023-10-21T17:39:40.286209+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:14.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249729+0000","last_change":"2023-10-21T17:39:40.265515+0000","last_active":"2023-10-21T17:39:41.249729+0000","last_peered":"2023-10-21T17:39:41.249729+0000","last_clean":"2023-10-21T17:39:41.249729+0000","last_became_active":"2023-10-21T17:39:40.264706+0000","last_became_peered":"2023-10-21T17:39:40.264706+0000","last_unstale":"2023-10-21T17:39:41.249729+0000","last_undegraded":"2023-10-21T17:39:41.249729+0000","last_fullsized":"2023-10-21T17:39:41.249729+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:19.664508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":15,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.382236+0000","last_change":"2023-10-21T17:39:39.225724+0000","last_active":"2023-10-21T17:39:40.382236+0000","last_peered":"2023-10-21T17:39:40.382236+0000","last_clean":"2023-10-21T17:39:40.382236+0000","last_became_active":"2023-10-21T17:39:39.223965+0000","last_became_peered":"2023-10-21T17:39:39.223965+0000","last_unstale":"2023-10-21T17:39:40.382236+0000","last_undegraded":"2023-10-21T17:39:40.382236+0000","last_fullsized":"2023-10-21T17:39:40.382236+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:53.712155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.283773+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:40.283773+0000","last_peered":"2023-10-21T17:39:40.283773+0000","last_clean":"2023-10-21T17:39:40.283773+0000","last_became_active":"2023-10-21T17:39:40.282270+0000","last_became_peered":"2023-10-21T17:39:40.282270+0000","last_unstale":"2023-10-21T17:39:40.283773+0000","last_undegraded":"2023-10-21T17:39:40.283773+0000","last_fullsized":"2023-10-21T17:39:40.283773+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:47.453960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253031+0000","last_change":"2023-10-21T17:39:39.240537+0000","last_active":"2023-10-21T17:39:41.253031+0000","last_peered":"2023-10-21T17:39:41.253031+0000","last_clean":"2023-10-21T17:39:41.253031+0000","last_became_active":"2023-10-21T17:39:39.240212+0000","last_became_peered":"2023-10-21T17:39:39.240212+0000","last_unstale":"2023-10-21T17:39:41.253031+0000","last_undegraded":"2023-10-21T17:39:41.253031+0000","last_fullsized":"2023-10-21T17:39:41.253031+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:57.552470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280553+0000","last_change":"2023-10-21T17:39:40.278403+0000","last_active":"2023-10-21T17:39:40.280553+0000","last_peered":"2023-10-21T17:39:40.280553+0000","last_clean":"2023-10-21T17:39:40.280553+0000","last_became_active":"2023-10-21T17:39:40.278115+0000","last_became_peered":"2023-10-21T17:39:40.278115+0000","last_unstale":"2023-10-21T17:39:40.280553+0000","last_undegraded":"2023-10-21T17:39:40.280553+0000","last_fullsized":"2023-10-21T17:39:40.280553+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:12:43.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.382188+0000","last_change":"2023-10-21T17:39:39.239260+0000","last_active":"2023-10-21T17:39:40.382188+0000","last_peered":"2023-10-21T17:39:40.382188+0000","last_clean":"2023-10-21T17:39:40.382188+0000","last_became_active":"2023-10-21T17:39:39.235006+0000","last_became_peered":"2023-10-21T17:39:39.235006+0000","last_unstale":"2023-10-21T17:39:40.382188+0000","last_undegraded":"2023-10-21T17:39:40.382188+0000","last_fullsized":"2023-10-21T17:39:40.382188+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271707+0000","last_change":"2023-10-21T17:39:40.268385+0000","last_active":"2023-10-21T17:39:40.271707+0000","last_peered":"2023-10-21T17:39:40.271707+0000","last_clean":"2023-10-21T17:39:40.271707+0000","last_became_active":"2023-10-21T17:39:40.267876+0000","last_became_peered":"2023-10-21T17:39:40.267876+0000","last_unstale":"2023-10-21T17:39:40.271707+0000","last_undegraded":"2023-10-21T17:39:40.271707+0000","last_fullsized":"2023-10-21T17:39:40.271707+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:14.798225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377189+0000","last_change":"2023-10-21T17:39:39.246981+0000","last_active":"2023-10-21T17:39:40.377189+0000","last_peered":"2023-10-21T17:39:40.377189+0000","last_clean":"2023-10-21T17:39:40.377189+0000","last_became_active":"2023-10-21T17:39:39.246676+0000","last_became_peered":"2023-10-21T17:39:39.246676+0000","last_unstale":"2023-10-21T17:39:40.377189+0000","last_undegraded":"2023-10-21T17:39:40.377189+0000","last_fullsized":"2023-10-21T17:39:40.377189+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:08.549227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251107+0000","last_change":"2023-10-21T17:39:40.273974+0000","last_active":"2023-10-21T17:39:41.251107+0000","last_peered":"2023-10-21T17:39:41.251107+0000","last_clean":"2023-10-21T17:39:41.251107+0000","last_became_active":"2023-10-21T17:39:40.273689+0000","last_became_peered":"2023-10-21T17:39:40.273689+0000","last_unstale":"2023-10-21T17:39:41.251107+0000","last_undegraded":"2023-10-21T17:39:41.251107+0000","last_fullsized":"2023-10-21T17:39:41.251107+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.379044+0000","last_change":"2023-10-21T17:39:39.236436+0000","last_active":"2023-10-21T17:39:40.379044+0000","last_peered":"2023-10-21T17:39:40.379044+0000","last_clean":"2023-10-21T17:39:40.379044+0000","last_became_active":"2023-10-21T17:39:39.236185+0000","last_became_peered":"2023-10-21T17:39:39.236185+0000","last_unstale":"2023-10-21T17:39:40.379044+0000","last_undegraded":"2023-10-21T17:39:40.379044+0000","last_fullsized":"2023-10-21T17:39:40.379044+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:51.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285020+0000","last_change":"2023-10-21T17:39:40.275825+0000","last_active":"2023-10-21T17:39:40.285020+0000","last_peered":"2023-10-21T17:39:40.285020+0000","last_clean":"2023-10-21T17:39:40.285020+0000","last_became_active":"2023-10-21T17:39:40.275521+0000","last_became_peered":"2023-10-21T17:39:40.275521+0000","last_unstale":"2023-10-21T17:39:40.285020+0000","last_undegraded":"2023-10-21T17:39:40.285020+0000","last_fullsized":"2023-10-21T17:39:40.285020+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:52.151361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.268669+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:40.268669+0000","last_peered":"2023-10-21T17:39:40.268669+0000","last_clean":"2023-10-21T17:39:40.268669+0000","last_became_active":"2023-10-21T17:39:39.241978+0000","last_became_peered":"2023-10-21T17:39:39.241978+0000","last_unstale":"2023-10-21T17:39:40.268669+0000","last_undegraded":"2023-10-21T17:39:40.268669+0000","last_fullsized":"2023-10-21T17:39:40.268669+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:13.792604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284056+0000","last_change":"2023-10-21T17:39:40.278021+0000","last_active":"2023-10-21T17:39:40.284056+0000","last_peered":"2023-10-21T17:39:40.284056+0000","last_clean":"2023-10-21T17:39:40.284056+0000","last_became_active":"2023-10-21T17:39:40.276836+0000","last_became_peered":"2023-10-21T17:39:40.276836+0000","last_unstale":"2023-10-21T17:39:40.284056+0000","last_undegraded":"2023-10-21T17:39:40.284056+0000","last_fullsized":"2023-10-21T17:39:40.284056+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:44.011868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.270979+0000","last_change":"2023-10-21T17:39:39.241068+0000","last_active":"2023-10-21T17:39:40.270979+0000","last_peered":"2023-10-21T17:39:40.270979+0000","last_clean":"2023-10-21T17:39:40.270979+0000","last_became_active":"2023-10-21T17:39:39.240781+0000","last_became_peered":"2023-10-21T17:39:39.240781+0000","last_unstale":"2023-10-21T17:39:40.270979+0000","last_undegraded":"2023-10-21T17:39:40.270979+0000","last_fullsized":"2023-10-21T17:39:40.270979+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:06.555504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.248936+0000","last_change":"2023-10-21T17:39:40.278404+0000","last_active":"2023-10-21T17:39:41.248936+0000","last_peered":"2023-10-21T17:39:41.248936+0000","last_clean":"2023-10-21T17:39:41.248936+0000","last_became_active":"2023-10-21T17:39:40.276735+0000","last_became_peered":"2023-10-21T17:39:40.276735+0000","last_unstale":"2023-10-21T17:39:41.248936+0000","last_undegraded":"2023-10-21T17:39:41.248936+0000","last_fullsized":"2023-10-21T17:39:41.248936+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.248977+0000","last_change":"2023-10-21T17:39:39.229264+0000","last_active":"2023-10-21T17:39:40.248977+0000","last_peered":"2023-10-21T17:39:40.248977+0000","last_clean":"2023-10-21T17:39:40.248977+0000","last_became_active":"2023-10-21T17:39:39.227950+0000","last_became_peered":"2023-10-21T17:39:39.227950+0000","last_unstale":"2023-10-21T17:39:40.248977+0000","last_undegraded":"2023-10-21T17:39:40.248977+0000","last_fullsized":"2023-10-21T17:39:40.248977+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252603+0000","last_change":"2023-10-21T17:39:40.262960+0000","last_active":"2023-10-21T17:39:41.252603+0000","last_peered":"2023-10-21T17:39:41.252603+0000","last_clean":"2023-10-21T17:39:41.252603+0000","last_became_active":"2023-10-21T17:39:40.262214+0000","last_became_peered":"2023-10-21T17:39:40.262214+0000","last_unstale":"2023-10-21T17:39:41.252603+0000","last_undegraded":"2023-10-21T17:39:41.252603+0000","last_fullsized":"2023-10-21T17:39:41.252603+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:08.816431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.269629+0000","last_change":"2023-10-21T17:39:39.244340+0000","last_active":"2023-10-21T17:39:40.269629+0000","last_peered":"2023-10-21T17:39:40.269629+0000","last_clean":"2023-10-21T17:39:40.269629+0000","last_became_active":"2023-10-21T17:39:39.243245+0000","last_became_peered":"2023-10-21T17:39:39.243245+0000","last_unstale":"2023-10-21T17:39:40.269629+0000","last_undegraded":"2023-10-21T17:39:40.269629+0000","last_fullsized":"2023-10-21T17:39:40.269629+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.283333+0000","last_change":"2023-10-21T17:39:40.279028+0000","last_active":"2023-10-21T17:39:40.283333+0000","last_peered":"2023-10-21T17:39:40.283333+0000","last_clean":"2023-10-21T17:39:40.283333+0000","last_became_active":"2023-10-21T17:39:40.278727+0000","last_became_peered":"2023-10-21T17:39:40.278727+0000","last_unstale":"2023-10-21T17:39:40.283333+0000","last_undegraded":"2023-10-21T17:39:40.283333+0000","last_fullsized":"2023-10-21T17:39:40.283333+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:34.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.374845+0000","last_change":"2023-10-21T17:39:39.238814+0000","last_active":"2023-10-21T17:39:40.374845+0000","last_peered":"2023-10-21T17:39:40.374845+0000","last_clean":"2023-10-21T17:39:40.374845+0000","last_became_active":"2023-10-21T17:39:39.238497+0000","last_became_peered":"2023-10-21T17:39:39.238497+0000","last_unstale":"2023-10-21T17:39:40.374845+0000","last_undegraded":"2023-10-21T17:39:40.374845+0000","last_fullsized":"2023-10-21T17:39:40.374845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:37.376711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250099+0000","last_change":"2023-10-21T17:39:40.280192+0000","last_active":"2023-10-21T17:39:41.250099+0000","last_peered":"2023-10-21T17:39:41.250099+0000","last_clean":"2023-10-21T17:39:41.250099+0000","last_became_active":"2023-10-21T17:39:40.279897+0000","last_became_peered":"2023-10-21T17:39:40.279897+0000","last_unstale":"2023-10-21T17:39:41.250099+0000","last_undegraded":"2023-10-21T17:39:41.250099+0000","last_fullsized":"2023-10-21T17:39:41.250099+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:16.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"253'6","reported_seq":17,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.383878+0000","last_change":"2023-10-21T17:39:39.237379+0000","last_active":"2023-10-21T17:39:40.383878+0000","last_peered":"2023-10-21T17:39:40.383878+0000","last_clean":"2023-10-21T17:39:40.383878+0000","last_became_active":"2023-10-21T17:39:39.236089+0000","last_became_peered":"2023-10-21T17:39:39.236089+0000","last_unstale":"2023-10-21T17:39:40.383878+0000","last_undegraded":"2023-10-21T17:39:40.383878+0000","last_fullsized":"2023-10-21T17:39:40.383878+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:13.628441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250026+0000","last_change":"2023-10-21T17:39:40.283845+0000","last_active":"2023-10-21T17:39:41.250026+0000","last_peered":"2023-10-21T17:39:41.250026+0000","last_clean":"2023-10-21T17:39:41.250026+0000","last_became_active":"2023-10-21T17:39:40.283046+0000","last_became_peered":"2023-10-21T17:39:40.283046+0000","last_unstale":"2023-10-21T17:39:41.250026+0000","last_undegraded":"2023-10-21T17:39:41.250026+0000","last_fullsized":"2023-10-21T17:39:41.250026+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:49:18.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377454+0000","last_change":"2023-10-21T17:39:39.244723+0000","last_active":"2023-10-21T17:39:40.377454+0000","last_peered":"2023-10-21T17:39:40.377454+0000","last_clean":"2023-10-21T17:39:40.377454+0000","last_became_active":"2023-10-21T17:39:39.244145+0000","last_became_peered":"2023-10-21T17:39:39.244145+0000","last_unstale":"2023-10-21T17:39:40.377454+0000","last_undegraded":"2023-10-21T17:39:40.377454+0000","last_fullsized":"2023-10-21T17:39:40.377454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:15.752098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250622+0000","last_change":"2023-10-21T17:39:40.272813+0000","last_active":"2023-10-21T17:39:41.250622+0000","last_peered":"2023-10-21T17:39:41.250622+0000","last_clean":"2023-10-21T17:39:41.250622+0000","last_became_active":"2023-10-21T17:39:40.272544+0000","last_became_peered":"2023-10-21T17:39:40.272544+0000","last_unstale":"2023-10-21T17:39:41.250622+0000","last_undegraded":"2023-10-21T17:39:41.250622+0000","last_fullsized":"2023-10-21T17:39:41.250622+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:23.620934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271218+0000","last_change":"2023-10-21T17:39:39.229845+0000","last_active":"2023-10-21T17:39:40.271218+0000","last_peered":"2023-10-21T17:39:40.271218+0000","last_clean":"2023-10-21T17:39:40.271218+0000","last_became_active":"2023-10-21T17:39:39.229136+0000","last_became_peered":"2023-10-21T17:39:39.229136+0000","last_unstale":"2023-10-21T17:39:40.271218+0000","last_undegraded":"2023-10-21T17:39:40.271218+0000","last_fullsized":"2023-10-21T17:39:40.271218+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:33.975640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280017+0000","last_change":"2023-10-21T17:39:40.276873+0000","last_active":"2023-10-21T17:39:40.280017+0000","last_peered":"2023-10-21T17:39:40.280017+0000","last_clean":"2023-10-21T17:39:40.280017+0000","last_became_active":"2023-10-21T17:39:40.276503+0000","last_became_peered":"2023-10-21T17:39:40.276503+0000","last_unstale":"2023-10-21T17:39:40.280017+0000","last_undegraded":"2023-10-21T17:39:40.280017+0000","last_fullsized":"2023-10-21T17:39:40.280017+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271783+0000","last_change":"2023-10-21T17:39:39.227400+0000","last_active":"2023-10-21T17:39:40.271783+0000","last_peered":"2023-10-21T17:39:40.271783+0000","last_clean":"2023-10-21T17:39:40.271783+0000","last_became_active":"2023-10-21T17:39:39.226822+0000","last_became_peered":"2023-10-21T17:39:39.226822+0000","last_unstale":"2023-10-21T17:39:40.271783+0000","last_undegraded":"2023-10-21T17:39:40.271783+0000","last_fullsized":"2023-10-21T17:39:40.271783+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.203739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284081+0000","last_change":"2023-10-21T17:39:40.281657+0000","last_active":"2023-10-21T17:39:40.284081+0000","last_peered":"2023-10-21T17:39:40.284081+0000","last_clean":"2023-10-21T17:39:40.284081+0000","last_became_active":"2023-10-21T17:39:40.281069+0000","last_became_peered":"2023-10-21T17:39:40.281069+0000","last_unstale":"2023-10-21T17:39:40.284081+0000","last_undegraded":"2023-10-21T17:39:40.284081+0000","last_fullsized":"2023-10-21T17:39:40.284081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:32:39.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":20,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250036+0000","last_change":"2023-10-21T17:39:39.235563+0000","last_active":"2023-10-21T17:39:41.250036+0000","last_peered":"2023-10-21T17:39:41.250036+0000","last_clean":"2023-10-21T17:39:41.250036+0000","last_became_active":"2023-10-21T17:39:39.235286+0000","last_became_peered":"2023-10-21T17:39:39.235286+0000","last_unstale":"2023-10-21T17:39:41.250036+0000","last_undegraded":"2023-10-21T17:39:41.250036+0000","last_fullsized":"2023-10-21T17:39:41.250036+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:56.950389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280559+0000","last_change":"2023-10-21T17:39:40.274495+0000","last_active":"2023-10-21T17:39:40.280559+0000","last_peered":"2023-10-21T17:39:40.280559+0000","last_clean":"2023-10-21T17:39:40.280559+0000","last_became_active":"2023-10-21T17:39:40.273782+0000","last_became_peered":"2023-10-21T17:39:40.273782+0000","last_unstale":"2023-10-21T17:39:40.280559+0000","last_undegraded":"2023-10-21T17:39:40.280559+0000","last_fullsized":"2023-10-21T17:39:40.280559+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:28.755979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377135+0000","last_change":"2023-10-21T17:39:39.236238+0000","last_active":"2023-10-21T17:39:40.377135+0000","last_peered":"2023-10-21T17:39:40.377135+0000","last_clean":"2023-10-21T17:39:40.377135+0000","last_became_active":"2023-10-21T17:39:39.234980+0000","last_became_peered":"2023-10-21T17:39:39.234980+0000","last_unstale":"2023-10-21T17:39:40.377135+0000","last_undegraded":"2023-10-21T17:39:40.377135+0000","last_fullsized":"2023-10-21T17:39:40.377135+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:21.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284566+0000","last_change":"2023-10-21T17:39:40.277381+0000","last_active":"2023-10-21T17:39:40.284566+0000","last_peered":"2023-10-21T17:39:40.284566+0000","last_clean":"2023-10-21T17:39:40.284566+0000","last_became_active":"2023-10-21T17:39:40.277018+0000","last_became_peered":"2023-10-21T17:39:40.277018+0000","last_unstale":"2023-10-21T17:39:40.284566+0000","last_undegraded":"2023-10-21T17:39:40.284566+0000","last_fullsized":"2023-10-21T17:39:40.284566+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:11.077151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251068+0000","last_change":"2023-10-21T17:39:39.248091+0000","last_active":"2023-10-21T17:39:41.251068+0000","last_peered":"2023-10-21T17:39:41.251068+0000","last_clean":"2023-10-21T17:39:41.251068+0000","last_became_active":"2023-10-21T17:39:39.247775+0000","last_became_peered":"2023-10-21T17:39:39.247775+0000","last_unstale":"2023-10-21T17:39:41.251068+0000","last_undegraded":"2023-10-21T17:39:41.251068+0000","last_fullsized":"2023-10-21T17:39:41.251068+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:43.183503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251243+0000","last_change":"2023-10-21T17:39:40.268499+0000","last_active":"2023-10-21T17:39:41.251243+0000","last_peered":"2023-10-21T17:39:41.251243+0000","last_clean":"2023-10-21T17:39:41.251243+0000","last_became_active":"2023-10-21T17:39:40.268106+0000","last_became_peered":"2023-10-21T17:39:40.268106+0000","last_unstale":"2023-10-21T17:39:41.251243+0000","last_undegraded":"2023-10-21T17:39:41.251243+0000","last_fullsized":"2023-10-21T17:39:41.251243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:09.587502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253013+0000","last_change":"2023-10-21T17:39:39.237722+0000","last_active":"2023-10-21T17:39:41.253013+0000","last_peered":"2023-10-21T17:39:41.253013+0000","last_clean":"2023-10-21T17:39:41.253013+0000","last_became_active":"2023-10-21T17:39:39.235932+0000","last_became_peered":"2023-10-21T17:39:39.235932+0000","last_unstale":"2023-10-21T17:39:41.253013+0000","last_undegraded":"2023-10-21T17:39:41.253013+0000","last_fullsized":"2023-10-21T17:39:41.253013+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:08.581881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285958+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:40.285958+0000","last_peered":"2023-10-21T17:39:40.285958+0000","last_clean":"2023-10-21T17:39:40.285958+0000","last_became_active":"2023-10-21T17:39:40.282699+0000","last_became_peered":"2023-10-21T17:39:40.282699+0000","last_unstale":"2023-10-21T17:39:40.285958+0000","last_undegraded":"2023-10-21T17:39:40.285958+0000","last_fullsized":"2023-10-21T17:39:40.285958+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:33.969141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.272521+0000","last_change":"2023-10-21T17:39:39.247859+0000","last_active":"2023-10-21T17:39:40.272521+0000","last_peered":"2023-10-21T17:39:40.272521+0000","last_clean":"2023-10-21T17:39:40.272521+0000","last_became_active":"2023-10-21T17:39:39.245045+0000","last_became_peered":"2023-10-21T17:39:39.245045+0000","last_unstale":"2023-10-21T17:39:40.272521+0000","last_undegraded":"2023-10-21T17:39:40.272521+0000","last_fullsized":"2023-10-21T17:39:40.272521+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:22.615313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286282+0000","last_change":"2023-10-21T17:39:40.283322+0000","last_active":"2023-10-21T17:39:40.286282+0000","last_peered":"2023-10-21T17:39:40.286282+0000","last_clean":"2023-10-21T17:39:40.286282+0000","last_became_active":"2023-10-21T17:39:40.282539+0000","last_became_peered":"2023-10-21T17:39:40.282539+0000","last_unstale":"2023-10-21T17:39:40.286282+0000","last_undegraded":"2023-10-21T17:39:40.286282+0000","last_fullsized":"2023-10-21T17:39:40.286282+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:38:27.772200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.381566+0000","last_change":"2023-10-21T17:39:39.242639+0000","last_active":"2023-10-21T17:39:40.381566+0000","last_peered":"2023-10-21T17:39:40.381566+0000","last_clean":"2023-10-21T17:39:40.381566+0000","last_became_active":"2023-10-21T17:39:39.242334+0000","last_became_peered":"2023-10-21T17:39:39.242334+0000","last_unstale":"2023-10-21T17:39:40.381566+0000","last_undegraded":"2023-10-21T17:39:40.381566+0000","last_fullsized":"2023-10-21T17:39:40.381566+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:44.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250941+0000","last_change":"2023-10-21T17:39:40.270958+0000","last_active":"2023-10-21T17:39:41.250941+0000","last_peered":"2023-10-21T17:39:41.250941+0000","last_clean":"2023-10-21T17:39:41.250941+0000","last_became_active":"2023-10-21T17:39:40.270640+0000","last_became_peered":"2023-10-21T17:39:40.270640+0000","last_unstale":"2023-10-21T17:39:41.250941+0000","last_undegraded":"2023-10-21T17:39:41.250941+0000","last_fullsized":"2023-10-21T17:39:41.250941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:21.522147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252521+0000","last_change":"2023-10-21T17:39:39.246257+0000","last_active":"2023-10-21T17:39:41.252521+0000","last_peered":"2023-10-21T17:39:41.252521+0000","last_clean":"2023-10-21T17:39:41.252521+0000","last_became_active":"2023-10-21T17:39:39.245252+0000","last_became_peered":"2023-10-21T17:39:39.245252+0000","last_unstale":"2023-10-21T17:39:41.252521+0000","last_undegraded":"2023-10-21T17:39:41.252521+0000","last_fullsized":"2023-10-21T17:39:41.252521+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:18.223298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253815+0000","last_change":"2023-10-21T17:39:40.282652+0000","last_active":"2023-10-21T17:39:41.253815+0000","last_peered":"2023-10-21T17:39:41.253815+0000","last_clean":"2023-10-21T17:39:41.253815+0000","last_became_active":"2023-10-21T17:39:40.282108+0000","last_became_peered":"2023-10-21T17:39:40.282108+0000","last_unstale":"2023-10-21T17:39:41.253815+0000","last_undegraded":"2023-10-21T17:39:41.253815+0000","last_fullsized":"2023-10-21T17:39:41.253815+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:38.363235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252540+0000","last_change":"2023-10-21T17:39:39.242559+0000","last_active":"2023-10-21T17:39:41.252540+0000","last_peered":"2023-10-21T17:39:41.252540+0000","last_clean":"2023-10-21T17:39:41.252540+0000","last_became_active":"2023-10-21T17:39:39.236694+0000","last_became_peered":"2023-10-21T17:39:39.236694+0000","last_unstale":"2023-10-21T17:39:41.252540+0000","last_undegraded":"2023-10-21T17:39:41.252540+0000","last_fullsized":"2023-10-21T17:39:41.252540+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:58.202607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253983+0000","last_change":"2023-10-21T17:39:40.268873+0000","last_active":"2023-10-21T17:39:41.253983+0000","last_peered":"2023-10-21T17:39:41.253983+0000","last_clean":"2023-10-21T17:39:41.253983+0000","last_became_active":"2023-10-21T17:39:40.268515+0000","last_became_peered":"2023-10-21T17:39:40.268515+0000","last_unstale":"2023-10-21T17:39:41.253983+0000","last_undegraded":"2023-10-21T17:39:41.253983+0000","last_fullsized":"2023-10-21T17:39:41.253983+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:24.479277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.245568+0000","last_change":"2023-10-21T17:39:39.241987+0000","last_active":"2023-10-21T17:39:40.245568+0000","last_peered":"2023-10-21T17:39:40.245568+0000","last_clean":"2023-10-21T17:39:40.245568+0000","last_became_active":"2023-10-21T17:39:39.241676+0000","last_became_peered":"2023-10-21T17:39:39.241676+0000","last_unstale":"2023-10-21T17:39:40.245568+0000","last_undegraded":"2023-10-21T17:39:40.245568+0000","last_fullsized":"2023-10-21T17:39:40.245568+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:33.595143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.248937+0000","last_change":"2023-10-21T17:39:40.282789+0000","last_active":"2023-10-21T17:39:41.248937+0000","last_peered":"2023-10-21T17:39:41.248937+0000","last_clean":"2023-10-21T17:39:41.248937+0000","last_became_active":"2023-10-21T17:39:40.282503+0000","last_became_peered":"2023-10-21T17:39:40.282503+0000","last_unstale":"2023-10-21T17:39:41.248937+0000","last_undegraded":"2023-10-21T17:39:41.248937+0000","last_fullsized":"2023-10-21T17:39:41.248937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.381961+0000","last_change":"2023-10-21T17:39:39.245748+0000","last_active":"2023-10-21T17:39:40.381961+0000","last_peered":"2023-10-21T17:39:40.381961+0000","last_clean":"2023-10-21T17:39:40.381961+0000","last_became_active":"2023-10-21T17:39:39.245479+0000","last_became_peered":"2023-10-21T17:39:39.245479+0000","last_unstale":"2023-10-21T17:39:40.381961+0000","last_undegraded":"2023-10-21T17:39:40.381961+0000","last_fullsized":"2023-10-21T17:39:40.381961+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280342+0000","last_change":"2023-10-21T17:39:40.277680+0000","last_active":"2023-10-21T17:39:40.280342+0000","last_peered":"2023-10-21T17:39:40.280342+0000","last_clean":"2023-10-21T17:39:40.280342+0000","last_became_active":"2023-10-21T17:39:40.277441+0000","last_became_peered":"2023-10-21T17:39:40.277441+0000","last_unstale":"2023-10-21T17:39:40.280342+0000","last_undegraded":"2023-10-21T17:39:40.280342+0000","last_fullsized":"2023-10-21T17:39:40.280342+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:19.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.275748+0000","last_change":"2023-10-21T17:39:39.246309+0000","last_active":"2023-10-21T17:39:40.275748+0000","last_peered":"2023-10-21T17:39:40.275748+0000","last_clean":"2023-10-21T17:39:40.275748+0000","last_became_active":"2023-10-21T17:39:39.245706+0000","last_became_peered":"2023-10-21T17:39:39.245706+0000","last_unstale":"2023-10-21T17:39:40.275748+0000","last_undegraded":"2023-10-21T17:39:40.275748+0000","last_fullsized":"2023-10-21T17:39:40.275748+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:35.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285188+0000","last_change":"2023-10-21T17:39:40.281463+0000","last_active":"2023-10-21T17:39:40.285188+0000","last_peered":"2023-10-21T17:39:40.285188+0000","last_clean":"2023-10-21T17:39:40.285188+0000","last_became_active":"2023-10-21T17:39:40.281137+0000","last_became_peered":"2023-10-21T17:39:40.281137+0000","last_unstale":"2023-10-21T17:39:40.285188+0000","last_undegraded":"2023-10-21T17:39:40.285188+0000","last_fullsized":"2023-10-21T17:39:40.285188+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:37.556324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.246065+0000","last_change":"2023-10-21T17:39:39.227041+0000","last_active":"2023-10-21T17:39:40.246065+0000","last_peered":"2023-10-21T17:39:40.246065+0000","last_clean":"2023-10-21T17:39:40.246065+0000","last_became_active":"2023-10-21T17:39:39.225535+0000","last_became_peered":"2023-10-21T17:39:39.225535+0000","last_unstale":"2023-10-21T17:39:40.246065+0000","last_undegraded":"2023-10-21T17:39:40.246065+0000","last_fullsized":"2023-10-21T17:39:40.246065+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:20.950421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285434+0000","last_change":"2023-10-21T17:39:40.283081+0000","last_active":"2023-10-21T17:39:40.285434+0000","last_peered":"2023-10-21T17:39:40.285434+0000","last_clean":"2023-10-21T17:39:40.285434+0000","last_became_active":"2023-10-21T17:39:40.282782+0000","last_became_peered":"2023-10-21T17:39:40.282782+0000","last_unstale":"2023-10-21T17:39:40.285434+0000","last_undegraded":"2023-10-21T17:39:40.285434+0000","last_fullsized":"2023-10-21T17:39:40.285434+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:31.084424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.249479+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:40.249479+0000","last_peered":"2023-10-21T17:39:40.249479+0000","last_clean":"2023-10-21T17:39:40.249479+0000","last_became_active":"2023-10-21T17:39:39.241658+0000","last_became_peered":"2023-10-21T17:39:39.241658+0000","last_unstale":"2023-10-21T17:39:40.249479+0000","last_undegraded":"2023-10-21T17:39:40.249479+0000","last_fullsized":"2023-10-21T17:39:40.249479+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271687+0000","last_change":"2023-10-21T17:39:40.267491+0000","last_active":"2023-10-21T17:39:40.271687+0000","last_peered":"2023-10-21T17:39:40.271687+0000","last_clean":"2023-10-21T17:39:40.271687+0000","last_became_active":"2023-10-21T17:39:40.267154+0000","last_became_peered":"2023-10-21T17:39:40.267154+0000","last_unstale":"2023-10-21T17:39:40.271687+0000","last_undegraded":"2023-10-21T17:39:40.271687+0000","last_fullsized":"2023-10-21T17:39:40.271687+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:51.348415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253254+0000","last_change":"2023-10-21T17:39:39.247298+0000","last_active":"2023-10-21T17:39:41.253254+0000","last_peered":"2023-10-21T17:39:41.253254+0000","last_clean":"2023-10-21T17:39:41.253254+0000","last_became_active":"2023-10-21T17:39:39.247016+0000","last_became_peered":"2023-10-21T17:39:39.247016+0000","last_unstale":"2023-10-21T17:39:41.253254+0000","last_undegraded":"2023-10-21T17:39:41.253254+0000","last_fullsized":"2023-10-21T17:39:41.253254+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:58.778201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280334+0000","last_change":"2023-10-21T17:39:40.275609+0000","last_active":"2023-10-21T17:39:40.280334+0000","last_peered":"2023-10-21T17:39:40.280334+0000","last_clean":"2023-10-21T17:39:40.280334+0000","last_became_active":"2023-10-21T17:39:40.275290+0000","last_became_peered":"2023-10-21T17:39:40.275290+0000","last_unstale":"2023-10-21T17:39:40.280334+0000","last_undegraded":"2023-10-21T17:39:40.280334+0000","last_fullsized":"2023-10-21T17:39:40.280334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:59.855693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271370+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:40.271370+0000","last_peered":"2023-10-21T17:39:40.271370+0000","last_clean":"2023-10-21T17:39:40.271370+0000","last_became_active":"2023-10-21T17:39:39.238047+0000","last_became_peered":"2023-10-21T17:39:39.238047+0000","last_unstale":"2023-10-21T17:39:40.271370+0000","last_undegraded":"2023-10-21T17:39:40.271370+0000","last_fullsized":"2023-10-21T17:39:40.271370+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250243+0000","last_change":"2023-10-21T17:39:40.279881+0000","last_active":"2023-10-21T17:39:41.250243+0000","last_peered":"2023-10-21T17:39:41.250243+0000","last_clean":"2023-10-21T17:39:41.250243+0000","last_became_active":"2023-10-21T17:39:40.279567+0000","last_became_peered":"2023-10-21T17:39:40.279567+0000","last_unstale":"2023-10-21T17:39:41.250243+0000","last_undegraded":"2023-10-21T17:39:41.250243+0000","last_fullsized":"2023-10-21T17:39:41.250243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:23.894434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.269288+0000","last_change":"2023-10-21T17:39:39.244421+0000","last_active":"2023-10-21T17:39:40.269288+0000","last_peered":"2023-10-21T17:39:40.269288+0000","last_clean":"2023-10-21T17:39:40.269288+0000","last_became_active":"2023-10-21T17:39:39.244026+0000","last_became_peered":"2023-10-21T17:39:39.244026+0000","last_unstale":"2023-10-21T17:39:40.269288+0000","last_undegraded":"2023-10-21T17:39:40.269288+0000","last_fullsized":"2023-10-21T17:39:40.269288+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:24.596691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253780+0000","last_change":"2023-10-21T17:39:40.275308+0000","last_active":"2023-10-21T17:39:41.253780+0000","last_peered":"2023-10-21T17:39:41.253780+0000","last_clean":"2023-10-21T17:39:41.253780+0000","last_became_active":"2023-10-21T17:39:40.274998+0000","last_became_peered":"2023-10-21T17:39:40.274998+0000","last_unstale":"2023-10-21T17:39:41.253780+0000","last_undegraded":"2023-10-21T17:39:41.253780+0000","last_fullsized":"2023-10-21T17:39:41.253780+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:38.521395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252009+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:41.252009+0000","last_peered":"2023-10-21T17:39:41.252009+0000","last_clean":"2023-10-21T17:39:41.252009+0000","last_became_active":"2023-10-21T17:39:39.236393+0000","last_became_peered":"2023-10-21T17:39:39.236393+0000","last_unstale":"2023-10-21T17:39:41.252009+0000","last_undegraded":"2023-10-21T17:39:41.252009+0000","last_fullsized":"2023-10-21T17:39:41.252009+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:40.507879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.277397+0000","last_change":"2023-10-21T17:39:40.270271+0000","last_active":"2023-10-21T17:39:40.277397+0000","last_peered":"2023-10-21T17:39:40.277397+0000","last_clean":"2023-10-21T17:39:40.277397+0000","last_became_active":"2023-10-21T17:39:40.269983+0000","last_became_peered":"2023-10-21T17:39:40.269983+0000","last_unstale":"2023-10-21T17:39:40.277397+0000","last_undegraded":"2023-10-21T17:39:40.277397+0000","last_fullsized":"2023-10-21T17:39:40.277397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:56.117575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251499+0000","last_change":"2023-10-21T17:39:39.247657+0000","last_active":"2023-10-21T17:39:41.251499+0000","last_peered":"2023-10-21T17:39:41.251499+0000","last_clean":"2023-10-21T17:39:41.251499+0000","last_became_active":"2023-10-21T17:39:39.247152+0000","last_became_peered":"2023-10-21T17:39:39.247152+0000","last_unstale":"2023-10-21T17:39:41.251499+0000","last_undegraded":"2023-10-21T17:39:41.251499+0000","last_fullsized":"2023-10-21T17:39:41.251499+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:46.448339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"239'4895","reported_seq":5898,"reported_epoch":252,"state":"active+clean","last_fresh":"2023-10-21T17:39:39.236490+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:39:39.236490+0000","last_peered":"2023-10-21T17:39:39.236490+0000","last_clean":"2023-10-21T17:39:39.236490+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:39:39.236490+0000","last_undegraded":"2023-10-21T17:39:39.236490+0000","last_fullsized":"2023-10-21T17:39:39.236490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4895,"ondisk_log_size":4895,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576010,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75132,"kb_used_data":3700,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296708,"statfs":{"total":96636764160,"available":96559828992,"internally_reserved":0,"allocated":3788800,"data_stored":2250274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.667},{"interface":"front","average":{"1min":1.286,"5min":1.135,"15min":1.177},"min":{"1min":0.74299999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.81200000000000006}]},{"osd":1,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1619999999999999,"15min":1.212},"min":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.615,"5min":1.7170000000000001,"15min":1.73},"last":1.194},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.26,"15min":1.298},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.611,"5min":1.792,"15min":1.885},"last":1.5600000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.3759999999999999,"15min":1.3480000000000001},"min":{"1min":0.78100000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":1.139}]},{"osd":3,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2849999999999999,"15min":1.3129999999999999},"min":{"1min":0.624,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6739999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.343},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4990000000000001,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.383}]},{"osd":4,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4019999999999999,"15min":1.367},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9039999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5129999999999999,"15min":1.492},"min":{"1min":0.76100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.804,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.4199999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.508,"15min":1.421},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.837,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.246},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.49,"15min":1.456},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.036,"5min":2.613,"15min":2.613},"last":1.5069999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1019999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.4810000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.919,"5min":2.879,"15min":2.879},"last":1.5980000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42628,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329212,"statfs":{"total":96636764160,"available":96593113088,"internally_reserved":0,"allocated":3194880,"data_stored":1659894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.133,"15min":1.1890000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.671,"5min":1.944,"15min":1.944},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.163,"15min":1.212},"min":{"1min":0.71499999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.607,"5min":1.6839999999999999,"15min":1.762},"last":1.1220000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.121,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.474,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.258,"15min":1.302},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5029999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.5049999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.254,"15min":1.3049999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.532,"5min":1.706,"15min":1.8300000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.92200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.516,"5min":1.679,"15min":1.929},"last":0.83199999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.349,"15min":1.403},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.96,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4019999999999999,"15min":1.4510000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.5800000000000001,"5min":1.871,"15min":2.048},"last":1.1970000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.28,"15min":1.341},"min":{"1min":0.97299999999999998,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.409,"15min":1.4330000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.806,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5489999999999999,"15min":1.5669999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.339},"last":1.552},{"interface":"front","average":{"1min":1.369,"5min":1.3260000000000001,"15min":1.369},"min":{"1min":1.0169999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4650000000000001,"15min":1.526},"min":{"1min":1.012,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.627,"15min":1.663},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0449999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.5409999999999999},"last":1.6499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64824,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307016,"statfs":{"total":96636764160,"available":96570384384,"internally_reserved":0,"allocated":3186688,"data_stored":1659782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.1439999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.611,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.161,"15min":1.218},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.343}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2729999999999999,"15min":1.27},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6719999999999999,"5min":1.738,"15min":1.8480000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1659999999999999,"15min":1.2230000000000001},"min":{"1min":0.74399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.532,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.82999999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.802},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.224,"15min":1.244},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.202}]},{"osd":3,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3080000000000001,"15min":1.343},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.639,"5min":1.758,"15min":1.829},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.379,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.897},"last":1.2549999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.234,"15min":1.25},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.9910000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.415,"15min":1.4039999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.4930000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1.0369999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8360000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":1.1399999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.52},"last":1.661}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":0.96199999999999997,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.0249999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.5289999999999999,"15min":1.5649999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.1349999999999998},"last":1.5600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52356,"kb_used_data":3128,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319484,"statfs":{"total":96636764160,"available":96583151616,"internally_reserved":0,"allocated":3203072,"data_stored":1666588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0980000000000001,"15min":1.1559999999999999},"min":{"1min":0.59799999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6850000000000001,"5min":1.746,"15min":1.8480000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.179,"15min":1.2130000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5269999999999999,"5min":1.659,"15min":1.869},"last":1.105}]},{"osd":1,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.234,"15min":1.236},"min":{"1min":0.754,"5min":0.754,"15min":0.58099999999999996},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2270000000000001,"15min":1.2569999999999999},"min":{"1min":0.79000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6870000000000001,"5min":1.762,"15min":1.762},"last":0.73499999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.24,"15min":1.26},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":1.71,"15min":1.7609999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3560000000000001,"15min":1.3859999999999999},"min":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3069999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.2509999999999999,"15min":1.3089999999999999},"min":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5640000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.395,"15min":1.4219999999999999},"min":{"1min":0.98599999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.611,"5min":1.825,"15min":1.9239999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3500000000000001,"15min":1.3420000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.7430000000000001,"5min":1.8340000000000001,"15min":1.867},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.3340000000000001,"15min":1.3580000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.81000000000000005},"max":{"1min":1.776,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":0.85299999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.405,"15min":1.4419999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.732,"5min":1.8979999999999999,"15min":2.1600000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.5860000000000001,"15min":1.5940000000000001},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.9330000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.2250000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.869,"5min":2.0430000000000001,"15min":2.234},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5740000000000001,"15min":1.5820000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.8759999999999999,"5min":1.984,"15min":2.081},"last":1.3480000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56528,"kb_used_data":3136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315312,"statfs":{"total":96636764160,"available":96578879488,"internally_reserved":0,"allocated":3211264,"data_stored":1662032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.462,"5min":1.6439999999999999,"15min":1.71},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.234,"15min":1.258},"min":{"1min":0.68000000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0800000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2350000000000001,"5min":1.835,"15min":1.835},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3240000000000001},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.6870000000000001,"5min":1.879,"15min":1.879},"last":1.036}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0740000000000001,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6539999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.753},{"interface":"front","average":{"1min":1.2,"5min":1.23,"15min":1.286},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2},"last":1.24}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.403,"15min":1.4370000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.143},{"interface":"front","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.4650000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.169},"last":1.5549999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4239999999999999,"15min":1.4910000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9379999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.365},{"interface":"front","average":{"1min":1.528,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.92600000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.484}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.5900000000000001},"min":{"1min":0.92800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1779999999999999,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.849},{"interface":"front","average":{"1min":1.472,"5min":1.552,"15min":1.6160000000000001},"min":{"1min":0.998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0209999999999999,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.633}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.637,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.921,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.038,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1230000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.9199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56512,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315328,"statfs":{"total":96636764160,"available":96578895872,"internally_reserved":0,"allocated":3194880,"data_stored":1660434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93100000000000005,"15min":0.90900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.585,"5min":1.907,"15min":2.0059999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.321,"15min":1.361},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.585,"5min":1.958,"15min":1.958},"last":0.66200000000000003}]},{"osd":3,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1950000000000001,"15min":1.1930000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.466,"15min":1.4830000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.252}]},{"osd":4,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4510000000000001,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4470000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.5289999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.4059999999999999,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.845,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.45},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6240000000000001,"15min":1.613},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.7430000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.5309999999999999,"15min":1.5620000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.855,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.919},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.5169999999999999,"15min":1.52},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.631,"15min":1.6339999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9910000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5780000000000001,"15min":1.593},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0739999999999998,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":3.0230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52340,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319500,"statfs":{"total":96636764160,"available":96583168000,"internally_reserved":0,"allocated":3186688,"data_stored":1659782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.92700000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.411,"5min":1.411,"15min":1.976},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.0980000000000001,"15min":1.147},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.601,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.81100000000000005}]},{"osd":2,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.006,"15min":1.042},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.79},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.179,"15min":1.226},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5960000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.298,"15min":1.371},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9239999999999999,"5min":2.0539999999999998,"15min":2.129},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.2509999999999999,"15min":1.2929999999999999},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.85799999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0139999999999998,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.45,"15min":1.484},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0089999999999999,"5min":2.0750000000000002,"15min":2.383},"last":1.329}]},{"osd":5,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.516,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.98,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.593},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.464,"15min":1.5249999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.1000000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.4139999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.569,"15min":1.595},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.2069999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.4890000000000001,"15min":1.5469999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.927,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0229999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.609,"15min":1.663},"min":{"1min":0.95999999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1349999999999998,"5min":2.1829999999999998,"15min":2.548},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.696,"15min":1.7090000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.282,"5min":2.492,"15min":2.548},"last":1.5369999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576010,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52936,"kb_used_data":3704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318904,"statfs":{"total":96636764160,"available":96582557696,"internally_reserved":0,"allocated":3792896,"data_stored":2251762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":2.9249999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":1.085},"min":{"1min":0.61299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.556,"15min":1.748},"last":1.0209999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0980000000000001,"15min":1.163},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.504,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":2.1840000000000002},{"interface":"front","average":{"1min":1.046,"5min":1.0780000000000001,"15min":1.171},"min":{"1min":0.67500000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4259999999999999,"5min":1.708,"15min":1.851},"last":0.628}]},{"osd":3,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6359999999999999,"5min":1.897,"15min":1.9590000000000001},"last":2.597},{"interface":"front","average":{"1min":1.242,"5min":1.3140000000000001,"15min":1.3819999999999999},"min":{"1min":0.95599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.702,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":2.343}]},{"osd":4,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3340000000000001,"15min":1.421},"min":{"1min":0.877,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9470000000000001,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.764},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5289999999999999,"15min":1.5880000000000001},"min":{"1min":1.0620000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.917,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":2.113}]},{"osd":5,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.389,"15min":1.401},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9239999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.845},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":2.242}]},{"osd":6,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.82899999999999996,"5min":0.75,"15min":0.75},"max":{"1min":1.7310000000000001,"5min":2.0009999999999999,"15min":2.0139999999999998},"last":2.073},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.609,"15min":1.671},"min":{"1min":1.095,"5min":0.996,"15min":0.996},"max":{"1min":2.0920000000000001,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":2.2919999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4570000000000001,"15min":1.5309999999999999},"min":{"1min":1.002,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9910000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.571,"15min":1.6180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1819999999999999,"5min":2.2519999999999998,"15min":2.262},"last":2.3809999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:39:44.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:39:45.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.131+0000 7fce570d1640 1 Processor -- start 2023-10-21T17:39:45.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.131+0000 7fce570d1640 1 -- start start 2023-10-21T17:39:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.132+0000 7fce570d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce570d1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5015f7d0 con 0x7fce5015ab30 2023-10-21T17:39:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce570d1640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5015efb0 con 0x7fce5015af10 2023-10-21T17:39:45.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce570d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5015f210 con 0x7fce500adc70 2023-10-21T17:39:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce560cf640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fce5015ab30 0x7fce5015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36212/0 (socket says 172.21.15.53:36212) 2023-10-21T17:39:45.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce560cf640 1 -- 172.21.15.53:0/2016590398 learned_addr learned my addr 172.21.15.53:0/2016590398 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:45.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce558ce640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.133+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463246228 0 0) 0x7fce5015f7d0 con 0x7fce5015ab30 2023-10-21T17:39:45.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c003520 con 0x7fce5015ab30 2023-10-21T17:39:45.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1846991161 0 0) 0x7fce3c003520 con 0x7fce5015ab30 2023-10-21T17:39:45.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 msgr2=0x7fce500b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce500b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 >> v1:172.21.15.124:6789/0 conn(0x7fce5015af10 legacy=0x7fce5006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5015f4f0 con 0x7fce5015ab30 2023-10-21T17:39:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fce4c001690 con 0x7fce5015ab30 2023-10-21T17:39:45.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 >> v1:172.21.15.53:6789/0 conn(0x7fce5015ab30 legacy=0x7fce5015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce3c0039a0 0x7fce3c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce3c004d00 0x7fce3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5015f210 con 0x7fce3c0039a0 2023-10-21T17:39:45.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c00ba60 con 0x7fce3c004d00 2023-10-21T17:39:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c010030 con 0x7fce500adc70 2023-10-21T17:39:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.134+0000 7fce560cf640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce3c0039a0 0x7fce3c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce558ce640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce568d0640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce3c004d00 0x7fce3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce560cf640 1 -- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 msgr2=0x7fce3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce560cf640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce500adc70 0x7fce3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce560cf640 1 -- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce3c004d00 msgr2=0x7fce3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce560cf640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce3c004d00 0x7fce3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce560cf640 1 -- 172.21.15.53:0/2016590398 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c003520 con 0x7fce3c0039a0 2023-10-21T17:39:45.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.135+0000 7fce568d0640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce3c004d00 0x7fce3c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:45.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.136+0000 7fce560cf640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce3c0039a0 0x7fce3c003dc0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001eb0 tx=0x7fce3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57284bc9506f64bf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.136+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c005ad0 con 0x7fce3c0039a0 2023-10-21T17:39:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.136+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c005c30 con 0x7fce3c0039a0 2023-10-21T17:39:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.136+0000 7fce550cd640 1 -- 172.21.15.53:0/2016590398 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c02c890 con 0x7fce3c0039a0 2023-10-21T17:39:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 -- 172.21.15.53:0/2016590398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce3c0039a0 msgr2=0x7fce3c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 --2- 172.21.15.53:0/2016590398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce3c0039a0 0x7fce3c003dc0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001eb0 tx=0x7fce3c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce568d0640 1 -- 172.21.15.53:0/2016590398 reap_dead start 2023-10-21T17:39:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 -- 172.21.15.53:0/2016590398 shutdown_connections 2023-10-21T17:39:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 -- 172.21.15.53:0/2016590398 >> 172.21.15.53:0/2016590398 conn(0x7fce500cb0c0 msgr2=0x7fce500cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 -- 172.21.15.53:0/2016590398 shutdown_connections 2023-10-21T17:39:45.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.137+0000 7fce570d1640 1 -- 172.21.15.53:0/2016590398 wait complete. 2023-10-21T17:39:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.139+0000 7fce570d1640 1 Processor -- start 2023-10-21T17:39:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.139+0000 7fce570d1640 1 -- start start 2023-10-21T17:39:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce3c0039a0 0x7fce500c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 0x7fce50159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5015d730 con 0x7fce500c6220 2023-10-21T17:39:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce501597d0 con 0x7fce50158e10 2023-10-21T17:39:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce570d1640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500c7e60 con 0x7fce3c0039a0 2023-10-21T17:39:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce558ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce558ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54508/0 (socket says 172.21.15.53:54508) 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.140+0000 7fce558ce640 1 -- 172.21.15.53:0/2468284044 learned_addr learned my addr 172.21.15.53:0/2468284044 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce568d0640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 0x7fce50159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce560cf640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce3c0039a0 0x7fce500c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 -- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce3c0039a0 msgr2=0x7fce500c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce3c0039a0 0x7fce500c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 -- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 msgr2=0x7fce50159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 0x7fce50159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 -- 172.21.15.53:0/2468284044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce50155110 con 0x7fce500c6220 2023-10-21T17:39:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce568d0640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 0x7fce50159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce558ce640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fce40011600 tx=0x7fce40011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.141+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000f920 con 0x7fce500c6220 2023-10-21T17:39:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.142+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4000fa80 con 0x7fce500c6220 2023-10-21T17:39:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.142+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000fdf0 con 0x7fce500c6220 2023-10-21T17:39:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.142+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5015cab0 con 0x7fce500c6220 2023-10-21T17:39:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.142+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5015d070 con 0x7fce500c6220 2023-10-21T17:39:45.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.143+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5004e160 con 0x7fce500c6220 2023-10-21T17:39:45.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.144+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fce40015460 con 0x7fce500c6220 2023-10-21T17:39:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.145+0000 7fce46ffd640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 0x7fce3406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.145+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fce40089ff0 con 0x7fce500c6220 2023-10-21T17:39:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.145+0000 7fce560cf640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 0x7fce3406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.146+0000 7fce560cf640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 0x7fce3406d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034000 tx=0x7fce4c009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.149+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce40054b90 con 0x7fce500c6220 2023-10-21T17:39:45.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.338+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce5006c830 con 0x7fce3406b0b0 2023-10-21T17:39:45.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.360+0000 7fce46ffd640 1 -- 172.21.15.53:0/2468284044 <== mgr.38150 v2:172.21.15.124:6832/3816427222 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287297 (secure 0 0 0) 0x7fce4c001450 con 0x7fce3406b0b0 2023-10-21T17:39:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:45.366 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:39:45.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.366+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 msgr2=0x7fce3406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 0x7fce3406d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fce4c034000 tx=0x7fce4c009030 comp rx=0 tx=0).stop 2023-10-21T17:39:45.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 msgr2=0x7fce50154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fce40011600 tx=0x7fce40011ac0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 shutdown_connections 2023-10-21T17:39:45.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fce3406b0b0 0x7fce3406d570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce50158e10 0x7fce50159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce500c6220 0x7fce50154bd0 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 --2- 172.21.15.53:0/2468284044 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce3c0039a0 0x7fce500c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 >> 172.21.15.53:0/2468284044 conn(0x7fce500cb0c0 msgr2=0x7fce500b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 shutdown_connections 2023-10-21T17:39:45.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.367+0000 7fce570d1640 1 -- 172.21.15.53:0/2468284044 wait complete. 2023-10-21T17:39:45.389 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-21T17:39:44.538411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":671,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4933,"ondisk_log_size":4933,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453256,"kb_used_data":26132,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754521464,"statfs":{"total":773094113280,"available":772629979136,"internally_reserved":0,"allocated":26759168,"data_stored":14470548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":437243449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002215"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285868+0000","last_change":"2023-10-21T17:39:40.283996+0000","last_active":"2023-10-21T17:39:40.285868+0000","last_peered":"2023-10-21T17:39:40.285868+0000","last_clean":"2023-10-21T17:39:40.285868+0000","last_became_active":"2023-10-21T17:39:40.283690+0000","last_became_peered":"2023-10-21T17:39:40.283690+0000","last_unstale":"2023-10-21T17:39:40.285868+0000","last_undegraded":"2023-10-21T17:39:40.285868+0000","last_fullsized":"2023-10-21T17:39:40.285868+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:00:13.013899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249831+0000","last_change":"2023-10-21T17:39:40.279882+0000","last_active":"2023-10-21T17:39:41.249831+0000","last_peered":"2023-10-21T17:39:41.249831+0000","last_clean":"2023-10-21T17:39:41.249831+0000","last_became_active":"2023-10-21T17:39:40.279568+0000","last_became_peered":"2023-10-21T17:39:40.279568+0000","last_unstale":"2023-10-21T17:39:41.249831+0000","last_undegraded":"2023-10-21T17:39:41.249831+0000","last_fullsized":"2023-10-21T17:39:41.249831+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:07.488724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.282417+0000","last_change":"2023-10-21T17:39:40.277265+0000","last_active":"2023-10-21T17:39:40.282417+0000","last_peered":"2023-10-21T17:39:40.282417+0000","last_clean":"2023-10-21T17:39:40.282417+0000","last_became_active":"2023-10-21T17:39:40.276998+0000","last_became_peered":"2023-10-21T17:39:40.276998+0000","last_unstale":"2023-10-21T17:39:40.282417+0000","last_undegraded":"2023-10-21T17:39:40.282417+0000","last_fullsized":"2023-10-21T17:39:40.282417+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:53.267061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284282+0000","last_change":"2023-10-21T17:39:40.278307+0000","last_active":"2023-10-21T17:39:40.284282+0000","last_peered":"2023-10-21T17:39:40.284282+0000","last_clean":"2023-10-21T17:39:40.284282+0000","last_became_active":"2023-10-21T17:39:40.277780+0000","last_became_peered":"2023-10-21T17:39:40.277780+0000","last_unstale":"2023-10-21T17:39:40.284282+0000","last_undegraded":"2023-10-21T17:39:40.284282+0000","last_fullsized":"2023-10-21T17:39:40.284282+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:01.301844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284173+0000","last_change":"2023-10-21T17:39:40.277999+0000","last_active":"2023-10-21T17:39:40.284173+0000","last_peered":"2023-10-21T17:39:40.284173+0000","last_clean":"2023-10-21T17:39:40.284173+0000","last_became_active":"2023-10-21T17:39:40.277716+0000","last_became_peered":"2023-10-21T17:39:40.277716+0000","last_unstale":"2023-10-21T17:39:40.284173+0000","last_undegraded":"2023-10-21T17:39:40.284173+0000","last_fullsized":"2023-10-21T17:39:40.284173+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:36.785785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.279365+0000","last_change":"2023-10-21T17:39:40.274132+0000","last_active":"2023-10-21T17:39:40.279365+0000","last_peered":"2023-10-21T17:39:40.279365+0000","last_clean":"2023-10-21T17:39:40.279365+0000","last_became_active":"2023-10-21T17:39:40.273837+0000","last_became_peered":"2023-10-21T17:39:40.273837+0000","last_unstale":"2023-10-21T17:39:40.279365+0000","last_undegraded":"2023-10-21T17:39:40.279365+0000","last_fullsized":"2023-10-21T17:39:40.279365+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:36:58.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249224+0000","last_change":"2023-10-21T17:39:40.277476+0000","last_active":"2023-10-21T17:39:41.249224+0000","last_peered":"2023-10-21T17:39:41.249224+0000","last_clean":"2023-10-21T17:39:41.249224+0000","last_became_active":"2023-10-21T17:39:40.277175+0000","last_became_peered":"2023-10-21T17:39:40.277175+0000","last_unstale":"2023-10-21T17:39:41.249224+0000","last_undegraded":"2023-10-21T17:39:41.249224+0000","last_fullsized":"2023-10-21T17:39:41.249224+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:09.554848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280578+0000","last_change":"2023-10-21T17:39:40.274697+0000","last_active":"2023-10-21T17:39:40.280578+0000","last_peered":"2023-10-21T17:39:40.280578+0000","last_clean":"2023-10-21T17:39:40.280578+0000","last_became_active":"2023-10-21T17:39:40.274194+0000","last_became_peered":"2023-10-21T17:39:40.274194+0000","last_unstale":"2023-10-21T17:39:40.280578+0000","last_undegraded":"2023-10-21T17:39:40.280578+0000","last_fullsized":"2023-10-21T17:39:40.280578+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:10:17.428649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280436+0000","last_change":"2023-10-21T17:39:40.278661+0000","last_active":"2023-10-21T17:39:40.280436+0000","last_peered":"2023-10-21T17:39:40.280436+0000","last_clean":"2023-10-21T17:39:40.280436+0000","last_became_active":"2023-10-21T17:39:40.278377+0000","last_became_peered":"2023-10-21T17:39:40.278377+0000","last_unstale":"2023-10-21T17:39:40.280436+0000","last_undegraded":"2023-10-21T17:39:40.280436+0000","last_fullsized":"2023-10-21T17:39:40.280436+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:29.981635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285992+0000","last_change":"2023-10-21T17:39:40.284923+0000","last_active":"2023-10-21T17:39:40.285992+0000","last_peered":"2023-10-21T17:39:40.285992+0000","last_clean":"2023-10-21T17:39:40.285992+0000","last_became_active":"2023-10-21T17:39:40.281089+0000","last_became_peered":"2023-10-21T17:39:40.281089+0000","last_unstale":"2023-10-21T17:39:40.285992+0000","last_undegraded":"2023-10-21T17:39:40.285992+0000","last_fullsized":"2023-10-21T17:39:40.285992+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:51.348415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.282436+0000","last_change":"2023-10-21T17:39:40.280260+0000","last_active":"2023-10-21T17:39:40.282436+0000","last_peered":"2023-10-21T17:39:40.282436+0000","last_clean":"2023-10-21T17:39:40.282436+0000","last_became_active":"2023-10-21T17:39:40.279970+0000","last_became_peered":"2023-10-21T17:39:40.279970+0000","last_unstale":"2023-10-21T17:39:40.282436+0000","last_undegraded":"2023-10-21T17:39:40.282436+0000","last_fullsized":"2023-10-21T17:39:40.282436+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:21.956042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280270+0000","last_change":"2023-10-21T17:39:40.277448+0000","last_active":"2023-10-21T17:39:40.280270+0000","last_peered":"2023-10-21T17:39:40.280270+0000","last_clean":"2023-10-21T17:39:40.280270+0000","last_became_active":"2023-10-21T17:39:40.277121+0000","last_became_peered":"2023-10-21T17:39:40.277121+0000","last_unstale":"2023-10-21T17:39:40.280270+0000","last_undegraded":"2023-10-21T17:39:40.280270+0000","last_fullsized":"2023-10-21T17:39:40.280270+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:07:55.678448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253028+0000","last_change":"2023-10-21T17:39:40.279922+0000","last_active":"2023-10-21T17:39:41.253028+0000","last_peered":"2023-10-21T17:39:41.253028+0000","last_clean":"2023-10-21T17:39:41.253028+0000","last_became_active":"2023-10-21T17:39:40.279029+0000","last_became_peered":"2023-10-21T17:39:40.279029+0000","last_unstale":"2023-10-21T17:39:41.253028+0000","last_undegraded":"2023-10-21T17:39:41.253028+0000","last_fullsized":"2023-10-21T17:39:41.253028+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:13.093898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.261700+0000","last_change":"2023-10-21T17:39:40.260107+0000","last_active":"2023-10-21T17:39:40.261700+0000","last_peered":"2023-10-21T17:39:40.261700+0000","last_clean":"2023-10-21T17:39:40.261700+0000","last_became_active":"2023-10-21T17:39:40.259164+0000","last_became_peered":"2023-10-21T17:39:40.259164+0000","last_unstale":"2023-10-21T17:39:40.261700+0000","last_undegraded":"2023-10-21T17:39:40.261700+0000","last_fullsized":"2023-10-21T17:39:40.261700+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:04:53.267061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280385+0000","last_change":"2023-10-21T17:39:40.278320+0000","last_active":"2023-10-21T17:39:40.280385+0000","last_peered":"2023-10-21T17:39:40.280385+0000","last_clean":"2023-10-21T17:39:40.280385+0000","last_became_active":"2023-10-21T17:39:40.277166+0000","last_became_peered":"2023-10-21T17:39:40.277166+0000","last_unstale":"2023-10-21T17:39:40.280385+0000","last_undegraded":"2023-10-21T17:39:40.280385+0000","last_fullsized":"2023-10-21T17:39:40.280385+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:05:08.170636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286858+0000","last_change":"2023-10-21T17:39:40.285094+0000","last_active":"2023-10-21T17:39:40.286858+0000","last_peered":"2023-10-21T17:39:40.286858+0000","last_clean":"2023-10-21T17:39:40.286858+0000","last_became_active":"2023-10-21T17:39:40.284749+0000","last_became_peered":"2023-10-21T17:39:40.284749+0000","last_unstale":"2023-10-21T17:39:40.286858+0000","last_undegraded":"2023-10-21T17:39:40.286858+0000","last_fullsized":"2023-10-21T17:39:40.286858+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:40:29.137446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.254037+0000","last_change":"2023-10-21T17:39:40.283462+0000","last_active":"2023-10-21T17:39:41.254037+0000","last_peered":"2023-10-21T17:39:41.254037+0000","last_clean":"2023-10-21T17:39:41.254037+0000","last_became_active":"2023-10-21T17:39:40.281926+0000","last_became_peered":"2023-10-21T17:39:40.281926+0000","last_unstale":"2023-10-21T17:39:41.254037+0000","last_undegraded":"2023-10-21T17:39:41.254037+0000","last_fullsized":"2023-10-21T17:39:41.254037+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:42:34.391024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285473+0000","last_change":"2023-10-21T17:39:40.280739+0000","last_active":"2023-10-21T17:39:40.285473+0000","last_peered":"2023-10-21T17:39:40.285473+0000","last_clean":"2023-10-21T17:39:40.285473+0000","last_became_active":"2023-10-21T17:39:40.280069+0000","last_became_peered":"2023-10-21T17:39:40.280069+0000","last_unstale":"2023-10-21T17:39:40.285473+0000","last_undegraded":"2023-10-21T17:39:40.285473+0000","last_fullsized":"2023-10-21T17:39:40.285473+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:12.232845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286444+0000","last_change":"2023-10-21T17:39:40.285848+0000","last_active":"2023-10-21T17:39:40.286444+0000","last_peered":"2023-10-21T17:39:40.286444+0000","last_clean":"2023-10-21T17:39:40.286444+0000","last_became_active":"2023-10-21T17:39:40.285405+0000","last_became_peered":"2023-10-21T17:39:40.285405+0000","last_unstale":"2023-10-21T17:39:40.286444+0000","last_undegraded":"2023-10-21T17:39:40.286444+0000","last_fullsized":"2023-10-21T17:39:40.286444+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250176+0000","last_change":"2023-10-21T17:39:40.284648+0000","last_active":"2023-10-21T17:39:41.250176+0000","last_peered":"2023-10-21T17:39:41.250176+0000","last_clean":"2023-10-21T17:39:41.250176+0000","last_became_active":"2023-10-21T17:39:40.284214+0000","last_became_peered":"2023-10-21T17:39:40.284214+0000","last_unstale":"2023-10-21T17:39:41.250176+0000","last_undegraded":"2023-10-21T17:39:41.250176+0000","last_fullsized":"2023-10-21T17:39:41.250176+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:52.495370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250151+0000","last_change":"2023-10-21T17:39:40.270709+0000","last_active":"2023-10-21T17:39:41.250151+0000","last_peered":"2023-10-21T17:39:41.250151+0000","last_clean":"2023-10-21T17:39:41.250151+0000","last_became_active":"2023-10-21T17:39:40.270369+0000","last_became_peered":"2023-10-21T17:39:40.270369+0000","last_unstale":"2023-10-21T17:39:41.250151+0000","last_undegraded":"2023-10-21T17:39:41.250151+0000","last_fullsized":"2023-10-21T17:39:41.250151+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253548+0000","last_change":"2023-10-21T17:39:40.267480+0000","last_active":"2023-10-21T17:39:41.253548+0000","last_peered":"2023-10-21T17:39:41.253548+0000","last_clean":"2023-10-21T17:39:41.253548+0000","last_became_active":"2023-10-21T17:39:40.267074+0000","last_became_peered":"2023-10-21T17:39:40.267074+0000","last_unstale":"2023-10-21T17:39:41.253548+0000","last_undegraded":"2023-10-21T17:39:41.253548+0000","last_fullsized":"2023-10-21T17:39:41.253548+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:05.724202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.287846+0000","last_change":"2023-10-21T17:39:40.287294+0000","last_active":"2023-10-21T17:39:40.287846+0000","last_peered":"2023-10-21T17:39:40.287846+0000","last_clean":"2023-10-21T17:39:40.287846+0000","last_became_active":"2023-10-21T17:39:40.286990+0000","last_became_peered":"2023-10-21T17:39:40.286990+0000","last_unstale":"2023-10-21T17:39:40.287846+0000","last_undegraded":"2023-10-21T17:39:40.287846+0000","last_fullsized":"2023-10-21T17:39:40.287846+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251199+0000","last_change":"2023-10-21T17:39:40.270092+0000","last_active":"2023-10-21T17:39:41.251199+0000","last_peered":"2023-10-21T17:39:41.251199+0000","last_clean":"2023-10-21T17:39:41.251199+0000","last_became_active":"2023-10-21T17:39:40.269725+0000","last_became_peered":"2023-10-21T17:39:40.269725+0000","last_unstale":"2023-10-21T17:39:41.251199+0000","last_undegraded":"2023-10-21T17:39:41.251199+0000","last_fullsized":"2023-10-21T17:39:41.251199+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:18.601428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250830+0000","last_change":"2023-10-21T17:39:40.275179+0000","last_active":"2023-10-21T17:39:41.250830+0000","last_peered":"2023-10-21T17:39:41.250830+0000","last_clean":"2023-10-21T17:39:41.250830+0000","last_became_active":"2023-10-21T17:39:40.274025+0000","last_became_peered":"2023-10-21T17:39:40.274025+0000","last_unstale":"2023-10-21T17:39:41.250830+0000","last_undegraded":"2023-10-21T17:39:41.250830+0000","last_fullsized":"2023-10-21T17:39:41.250830+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:58.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251182+0000","last_change":"2023-10-21T17:39:40.271219+0000","last_active":"2023-10-21T17:39:41.251182+0000","last_peered":"2023-10-21T17:39:41.251182+0000","last_clean":"2023-10-21T17:39:41.251182+0000","last_became_active":"2023-10-21T17:39:40.270928+0000","last_became_peered":"2023-10-21T17:39:40.270928+0000","last_unstale":"2023-10-21T17:39:41.251182+0000","last_undegraded":"2023-10-21T17:39:41.251182+0000","last_fullsized":"2023-10-21T17:39:41.251182+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.281864+0000","last_change":"2023-10-21T17:39:40.276685+0000","last_active":"2023-10-21T17:39:40.281864+0000","last_peered":"2023-10-21T17:39:40.281864+0000","last_clean":"2023-10-21T17:39:40.281864+0000","last_became_active":"2023-10-21T17:39:40.276416+0000","last_became_peered":"2023-10-21T17:39:40.276416+0000","last_unstale":"2023-10-21T17:39:40.281864+0000","last_undegraded":"2023-10-21T17:39:40.281864+0000","last_fullsized":"2023-10-21T17:39:40.281864+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:14.398414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286126+0000","last_change":"2023-10-21T17:39:40.284175+0000","last_active":"2023-10-21T17:39:40.286126+0000","last_peered":"2023-10-21T17:39:40.286126+0000","last_clean":"2023-10-21T17:39:40.286126+0000","last_became_active":"2023-10-21T17:39:40.283773+0000","last_became_peered":"2023-10-21T17:39:40.283773+0000","last_unstale":"2023-10-21T17:39:40.286126+0000","last_undegraded":"2023-10-21T17:39:40.286126+0000","last_fullsized":"2023-10-21T17:39:40.286126+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:28:00.719344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.261079+0000","last_change":"2023-10-21T17:39:40.258467+0000","last_active":"2023-10-21T17:39:40.261079+0000","last_peered":"2023-10-21T17:39:40.261079+0000","last_clean":"2023-10-21T17:39:40.261079+0000","last_became_active":"2023-10-21T17:39:40.257660+0000","last_became_peered":"2023-10-21T17:39:40.257660+0000","last_unstale":"2023-10-21T17:39:40.261079+0000","last_undegraded":"2023-10-21T17:39:40.261079+0000","last_fullsized":"2023-10-21T17:39:40.261079+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:25.602312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.278613+0000","last_change":"2023-10-21T17:39:40.276636+0000","last_active":"2023-10-21T17:39:40.278613+0000","last_peered":"2023-10-21T17:39:40.278613+0000","last_clean":"2023-10-21T17:39:40.278613+0000","last_became_active":"2023-10-21T17:39:40.274303+0000","last_became_peered":"2023-10-21T17:39:40.274303+0000","last_unstale":"2023-10-21T17:39:40.278613+0000","last_undegraded":"2023-10-21T17:39:40.278613+0000","last_fullsized":"2023-10-21T17:39:40.278613+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:12.232845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250429+0000","last_change":"2023-10-21T17:39:40.270012+0000","last_active":"2023-10-21T17:39:41.250429+0000","last_peered":"2023-10-21T17:39:41.250429+0000","last_clean":"2023-10-21T17:39:41.250429+0000","last_became_active":"2023-10-21T17:39:40.269726+0000","last_became_peered":"2023-10-21T17:39:40.269726+0000","last_unstale":"2023-10-21T17:39:41.250429+0000","last_undegraded":"2023-10-21T17:39:41.250429+0000","last_fullsized":"2023-10-21T17:39:41.250429+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:57.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286209+0000","last_change":"2023-10-21T17:39:40.284346+0000","last_active":"2023-10-21T17:39:40.286209+0000","last_peered":"2023-10-21T17:39:40.286209+0000","last_clean":"2023-10-21T17:39:40.286209+0000","last_became_active":"2023-10-21T17:39:40.282055+0000","last_became_peered":"2023-10-21T17:39:40.282055+0000","last_unstale":"2023-10-21T17:39:40.286209+0000","last_undegraded":"2023-10-21T17:39:40.286209+0000","last_fullsized":"2023-10-21T17:39:40.286209+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:55:14.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.249729+0000","last_change":"2023-10-21T17:39:40.265515+0000","last_active":"2023-10-21T17:39:41.249729+0000","last_peered":"2023-10-21T17:39:41.249729+0000","last_clean":"2023-10-21T17:39:41.249729+0000","last_became_active":"2023-10-21T17:39:40.264706+0000","last_became_peered":"2023-10-21T17:39:40.264706+0000","last_unstale":"2023-10-21T17:39:41.249729+0000","last_undegraded":"2023-10-21T17:39:41.249729+0000","last_fullsized":"2023-10-21T17:39:41.249729+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:19.664508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":15,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.382236+0000","last_change":"2023-10-21T17:39:39.225724+0000","last_active":"2023-10-21T17:39:40.382236+0000","last_peered":"2023-10-21T17:39:40.382236+0000","last_clean":"2023-10-21T17:39:40.382236+0000","last_became_active":"2023-10-21T17:39:39.223965+0000","last_became_peered":"2023-10-21T17:39:39.223965+0000","last_unstale":"2023-10-21T17:39:40.382236+0000","last_undegraded":"2023-10-21T17:39:40.382236+0000","last_fullsized":"2023-10-21T17:39:40.382236+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:53.712155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.283773+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:40.283773+0000","last_peered":"2023-10-21T17:39:40.283773+0000","last_clean":"2023-10-21T17:39:40.283773+0000","last_became_active":"2023-10-21T17:39:40.282270+0000","last_became_peered":"2023-10-21T17:39:40.282270+0000","last_unstale":"2023-10-21T17:39:40.283773+0000","last_undegraded":"2023-10-21T17:39:40.283773+0000","last_fullsized":"2023-10-21T17:39:40.283773+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:47.453960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253031+0000","last_change":"2023-10-21T17:39:39.240537+0000","last_active":"2023-10-21T17:39:41.253031+0000","last_peered":"2023-10-21T17:39:41.253031+0000","last_clean":"2023-10-21T17:39:41.253031+0000","last_became_active":"2023-10-21T17:39:39.240212+0000","last_became_peered":"2023-10-21T17:39:39.240212+0000","last_unstale":"2023-10-21T17:39:41.253031+0000","last_undegraded":"2023-10-21T17:39:41.253031+0000","last_fullsized":"2023-10-21T17:39:41.253031+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:57.552470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280553+0000","last_change":"2023-10-21T17:39:40.278403+0000","last_active":"2023-10-21T17:39:40.280553+0000","last_peered":"2023-10-21T17:39:40.280553+0000","last_clean":"2023-10-21T17:39:40.280553+0000","last_became_active":"2023-10-21T17:39:40.278115+0000","last_became_peered":"2023-10-21T17:39:40.278115+0000","last_unstale":"2023-10-21T17:39:40.280553+0000","last_undegraded":"2023-10-21T17:39:40.280553+0000","last_fullsized":"2023-10-21T17:39:40.280553+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:12:43.718942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.382188+0000","last_change":"2023-10-21T17:39:39.239260+0000","last_active":"2023-10-21T17:39:40.382188+0000","last_peered":"2023-10-21T17:39:40.382188+0000","last_clean":"2023-10-21T17:39:40.382188+0000","last_became_active":"2023-10-21T17:39:39.235006+0000","last_became_peered":"2023-10-21T17:39:39.235006+0000","last_unstale":"2023-10-21T17:39:40.382188+0000","last_undegraded":"2023-10-21T17:39:40.382188+0000","last_fullsized":"2023-10-21T17:39:40.382188+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271707+0000","last_change":"2023-10-21T17:39:40.268385+0000","last_active":"2023-10-21T17:39:40.271707+0000","last_peered":"2023-10-21T17:39:40.271707+0000","last_clean":"2023-10-21T17:39:40.271707+0000","last_became_active":"2023-10-21T17:39:40.267876+0000","last_became_peered":"2023-10-21T17:39:40.267876+0000","last_unstale":"2023-10-21T17:39:40.271707+0000","last_undegraded":"2023-10-21T17:39:40.271707+0000","last_fullsized":"2023-10-21T17:39:40.271707+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:14.798225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377189+0000","last_change":"2023-10-21T17:39:39.246981+0000","last_active":"2023-10-21T17:39:40.377189+0000","last_peered":"2023-10-21T17:39:40.377189+0000","last_clean":"2023-10-21T17:39:40.377189+0000","last_became_active":"2023-10-21T17:39:39.246676+0000","last_became_peered":"2023-10-21T17:39:39.246676+0000","last_unstale":"2023-10-21T17:39:40.377189+0000","last_undegraded":"2023-10-21T17:39:40.377189+0000","last_fullsized":"2023-10-21T17:39:40.377189+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:08.549227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251107+0000","last_change":"2023-10-21T17:39:40.273974+0000","last_active":"2023-10-21T17:39:41.251107+0000","last_peered":"2023-10-21T17:39:41.251107+0000","last_clean":"2023-10-21T17:39:41.251107+0000","last_became_active":"2023-10-21T17:39:40.273689+0000","last_became_peered":"2023-10-21T17:39:40.273689+0000","last_unstale":"2023-10-21T17:39:41.251107+0000","last_undegraded":"2023-10-21T17:39:41.251107+0000","last_fullsized":"2023-10-21T17:39:41.251107+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.379044+0000","last_change":"2023-10-21T17:39:39.236436+0000","last_active":"2023-10-21T17:39:40.379044+0000","last_peered":"2023-10-21T17:39:40.379044+0000","last_clean":"2023-10-21T17:39:40.379044+0000","last_became_active":"2023-10-21T17:39:39.236185+0000","last_became_peered":"2023-10-21T17:39:39.236185+0000","last_unstale":"2023-10-21T17:39:40.379044+0000","last_undegraded":"2023-10-21T17:39:40.379044+0000","last_fullsized":"2023-10-21T17:39:40.379044+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:51.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285020+0000","last_change":"2023-10-21T17:39:40.275825+0000","last_active":"2023-10-21T17:39:40.285020+0000","last_peered":"2023-10-21T17:39:40.285020+0000","last_clean":"2023-10-21T17:39:40.285020+0000","last_became_active":"2023-10-21T17:39:40.275521+0000","last_became_peered":"2023-10-21T17:39:40.275521+0000","last_unstale":"2023-10-21T17:39:40.285020+0000","last_undegraded":"2023-10-21T17:39:40.285020+0000","last_fullsized":"2023-10-21T17:39:40.285020+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:52.151361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.268669+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:40.268669+0000","last_peered":"2023-10-21T17:39:40.268669+0000","last_clean":"2023-10-21T17:39:40.268669+0000","last_became_active":"2023-10-21T17:39:39.241978+0000","last_became_peered":"2023-10-21T17:39:39.241978+0000","last_unstale":"2023-10-21T17:39:40.268669+0000","last_undegraded":"2023-10-21T17:39:40.268669+0000","last_fullsized":"2023-10-21T17:39:40.268669+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:13.792604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284056+0000","last_change":"2023-10-21T17:39:40.278021+0000","last_active":"2023-10-21T17:39:40.284056+0000","last_peered":"2023-10-21T17:39:40.284056+0000","last_clean":"2023-10-21T17:39:40.284056+0000","last_became_active":"2023-10-21T17:39:40.276836+0000","last_became_peered":"2023-10-21T17:39:40.276836+0000","last_unstale":"2023-10-21T17:39:40.284056+0000","last_undegraded":"2023-10-21T17:39:40.284056+0000","last_fullsized":"2023-10-21T17:39:40.284056+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:44.011868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.270979+0000","last_change":"2023-10-21T17:39:39.241068+0000","last_active":"2023-10-21T17:39:40.270979+0000","last_peered":"2023-10-21T17:39:40.270979+0000","last_clean":"2023-10-21T17:39:40.270979+0000","last_became_active":"2023-10-21T17:39:39.240781+0000","last_became_peered":"2023-10-21T17:39:39.240781+0000","last_unstale":"2023-10-21T17:39:40.270979+0000","last_undegraded":"2023-10-21T17:39:40.270979+0000","last_fullsized":"2023-10-21T17:39:40.270979+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:06.555504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.248936+0000","last_change":"2023-10-21T17:39:40.278404+0000","last_active":"2023-10-21T17:39:41.248936+0000","last_peered":"2023-10-21T17:39:41.248936+0000","last_clean":"2023-10-21T17:39:41.248936+0000","last_became_active":"2023-10-21T17:39:40.276735+0000","last_became_peered":"2023-10-21T17:39:40.276735+0000","last_unstale":"2023-10-21T17:39:41.248936+0000","last_undegraded":"2023-10-21T17:39:41.248936+0000","last_fullsized":"2023-10-21T17:39:41.248936+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.248977+0000","last_change":"2023-10-21T17:39:39.229264+0000","last_active":"2023-10-21T17:39:40.248977+0000","last_peered":"2023-10-21T17:39:40.248977+0000","last_clean":"2023-10-21T17:39:40.248977+0000","last_became_active":"2023-10-21T17:39:39.227950+0000","last_became_peered":"2023-10-21T17:39:39.227950+0000","last_unstale":"2023-10-21T17:39:40.248977+0000","last_undegraded":"2023-10-21T17:39:40.248977+0000","last_fullsized":"2023-10-21T17:39:40.248977+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252603+0000","last_change":"2023-10-21T17:39:40.262960+0000","last_active":"2023-10-21T17:39:41.252603+0000","last_peered":"2023-10-21T17:39:41.252603+0000","last_clean":"2023-10-21T17:39:41.252603+0000","last_became_active":"2023-10-21T17:39:40.262214+0000","last_became_peered":"2023-10-21T17:39:40.262214+0000","last_unstale":"2023-10-21T17:39:41.252603+0000","last_undegraded":"2023-10-21T17:39:41.252603+0000","last_fullsized":"2023-10-21T17:39:41.252603+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:08.816431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.269629+0000","last_change":"2023-10-21T17:39:39.244340+0000","last_active":"2023-10-21T17:39:40.269629+0000","last_peered":"2023-10-21T17:39:40.269629+0000","last_clean":"2023-10-21T17:39:40.269629+0000","last_became_active":"2023-10-21T17:39:39.243245+0000","last_became_peered":"2023-10-21T17:39:39.243245+0000","last_unstale":"2023-10-21T17:39:40.269629+0000","last_undegraded":"2023-10-21T17:39:40.269629+0000","last_fullsized":"2023-10-21T17:39:40.269629+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.283333+0000","last_change":"2023-10-21T17:39:40.279028+0000","last_active":"2023-10-21T17:39:40.283333+0000","last_peered":"2023-10-21T17:39:40.283333+0000","last_clean":"2023-10-21T17:39:40.283333+0000","last_became_active":"2023-10-21T17:39:40.278727+0000","last_became_peered":"2023-10-21T17:39:40.278727+0000","last_unstale":"2023-10-21T17:39:40.283333+0000","last_undegraded":"2023-10-21T17:39:40.283333+0000","last_fullsized":"2023-10-21T17:39:40.283333+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:34.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.374845+0000","last_change":"2023-10-21T17:39:39.238814+0000","last_active":"2023-10-21T17:39:40.374845+0000","last_peered":"2023-10-21T17:39:40.374845+0000","last_clean":"2023-10-21T17:39:40.374845+0000","last_became_active":"2023-10-21T17:39:39.238497+0000","last_became_peered":"2023-10-21T17:39:39.238497+0000","last_unstale":"2023-10-21T17:39:40.374845+0000","last_undegraded":"2023-10-21T17:39:40.374845+0000","last_fullsized":"2023-10-21T17:39:40.374845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:37.376711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250099+0000","last_change":"2023-10-21T17:39:40.280192+0000","last_active":"2023-10-21T17:39:41.250099+0000","last_peered":"2023-10-21T17:39:41.250099+0000","last_clean":"2023-10-21T17:39:41.250099+0000","last_became_active":"2023-10-21T17:39:40.279897+0000","last_became_peered":"2023-10-21T17:39:40.279897+0000","last_unstale":"2023-10-21T17:39:41.250099+0000","last_undegraded":"2023-10-21T17:39:41.250099+0000","last_fullsized":"2023-10-21T17:39:41.250099+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:16.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"253'6","reported_seq":17,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.383878+0000","last_change":"2023-10-21T17:39:39.237379+0000","last_active":"2023-10-21T17:39:40.383878+0000","last_peered":"2023-10-21T17:39:40.383878+0000","last_clean":"2023-10-21T17:39:40.383878+0000","last_became_active":"2023-10-21T17:39:39.236089+0000","last_became_peered":"2023-10-21T17:39:39.236089+0000","last_unstale":"2023-10-21T17:39:40.383878+0000","last_undegraded":"2023-10-21T17:39:40.383878+0000","last_fullsized":"2023-10-21T17:39:40.383878+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:13.628441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250026+0000","last_change":"2023-10-21T17:39:40.283845+0000","last_active":"2023-10-21T17:39:41.250026+0000","last_peered":"2023-10-21T17:39:41.250026+0000","last_clean":"2023-10-21T17:39:41.250026+0000","last_became_active":"2023-10-21T17:39:40.283046+0000","last_became_peered":"2023-10-21T17:39:40.283046+0000","last_unstale":"2023-10-21T17:39:41.250026+0000","last_undegraded":"2023-10-21T17:39:41.250026+0000","last_fullsized":"2023-10-21T17:39:41.250026+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:49:18.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377454+0000","last_change":"2023-10-21T17:39:39.244723+0000","last_active":"2023-10-21T17:39:40.377454+0000","last_peered":"2023-10-21T17:39:40.377454+0000","last_clean":"2023-10-21T17:39:40.377454+0000","last_became_active":"2023-10-21T17:39:39.244145+0000","last_became_peered":"2023-10-21T17:39:39.244145+0000","last_unstale":"2023-10-21T17:39:40.377454+0000","last_undegraded":"2023-10-21T17:39:40.377454+0000","last_fullsized":"2023-10-21T17:39:40.377454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:15.752098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250622+0000","last_change":"2023-10-21T17:39:40.272813+0000","last_active":"2023-10-21T17:39:41.250622+0000","last_peered":"2023-10-21T17:39:41.250622+0000","last_clean":"2023-10-21T17:39:41.250622+0000","last_became_active":"2023-10-21T17:39:40.272544+0000","last_became_peered":"2023-10-21T17:39:40.272544+0000","last_unstale":"2023-10-21T17:39:41.250622+0000","last_undegraded":"2023-10-21T17:39:41.250622+0000","last_fullsized":"2023-10-21T17:39:41.250622+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:23.620934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271218+0000","last_change":"2023-10-21T17:39:39.229845+0000","last_active":"2023-10-21T17:39:40.271218+0000","last_peered":"2023-10-21T17:39:40.271218+0000","last_clean":"2023-10-21T17:39:40.271218+0000","last_became_active":"2023-10-21T17:39:39.229136+0000","last_became_peered":"2023-10-21T17:39:39.229136+0000","last_unstale":"2023-10-21T17:39:40.271218+0000","last_undegraded":"2023-10-21T17:39:40.271218+0000","last_fullsized":"2023-10-21T17:39:40.271218+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:33.975640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280017+0000","last_change":"2023-10-21T17:39:40.276873+0000","last_active":"2023-10-21T17:39:40.280017+0000","last_peered":"2023-10-21T17:39:40.280017+0000","last_clean":"2023-10-21T17:39:40.280017+0000","last_became_active":"2023-10-21T17:39:40.276503+0000","last_became_peered":"2023-10-21T17:39:40.276503+0000","last_unstale":"2023-10-21T17:39:40.280017+0000","last_undegraded":"2023-10-21T17:39:40.280017+0000","last_fullsized":"2023-10-21T17:39:40.280017+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271783+0000","last_change":"2023-10-21T17:39:39.227400+0000","last_active":"2023-10-21T17:39:40.271783+0000","last_peered":"2023-10-21T17:39:40.271783+0000","last_clean":"2023-10-21T17:39:40.271783+0000","last_became_active":"2023-10-21T17:39:39.226822+0000","last_became_peered":"2023-10-21T17:39:39.226822+0000","last_unstale":"2023-10-21T17:39:40.271783+0000","last_undegraded":"2023-10-21T17:39:40.271783+0000","last_fullsized":"2023-10-21T17:39:40.271783+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.203739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284081+0000","last_change":"2023-10-21T17:39:40.281657+0000","last_active":"2023-10-21T17:39:40.284081+0000","last_peered":"2023-10-21T17:39:40.284081+0000","last_clean":"2023-10-21T17:39:40.284081+0000","last_became_active":"2023-10-21T17:39:40.281069+0000","last_became_peered":"2023-10-21T17:39:40.281069+0000","last_unstale":"2023-10-21T17:39:40.284081+0000","last_undegraded":"2023-10-21T17:39:40.284081+0000","last_fullsized":"2023-10-21T17:39:40.284081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:32:39.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":20,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250036+0000","last_change":"2023-10-21T17:39:39.235563+0000","last_active":"2023-10-21T17:39:41.250036+0000","last_peered":"2023-10-21T17:39:41.250036+0000","last_clean":"2023-10-21T17:39:41.250036+0000","last_became_active":"2023-10-21T17:39:39.235286+0000","last_became_peered":"2023-10-21T17:39:39.235286+0000","last_unstale":"2023-10-21T17:39:41.250036+0000","last_undegraded":"2023-10-21T17:39:41.250036+0000","last_fullsized":"2023-10-21T17:39:41.250036+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:56.950389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280559+0000","last_change":"2023-10-21T17:39:40.274495+0000","last_active":"2023-10-21T17:39:40.280559+0000","last_peered":"2023-10-21T17:39:40.280559+0000","last_clean":"2023-10-21T17:39:40.280559+0000","last_became_active":"2023-10-21T17:39:40.273782+0000","last_became_peered":"2023-10-21T17:39:40.273782+0000","last_unstale":"2023-10-21T17:39:40.280559+0000","last_undegraded":"2023-10-21T17:39:40.280559+0000","last_fullsized":"2023-10-21T17:39:40.280559+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:28.755979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.377135+0000","last_change":"2023-10-21T17:39:39.236238+0000","last_active":"2023-10-21T17:39:40.377135+0000","last_peered":"2023-10-21T17:39:40.377135+0000","last_clean":"2023-10-21T17:39:40.377135+0000","last_became_active":"2023-10-21T17:39:39.234980+0000","last_became_peered":"2023-10-21T17:39:39.234980+0000","last_unstale":"2023-10-21T17:39:40.377135+0000","last_undegraded":"2023-10-21T17:39:40.377135+0000","last_fullsized":"2023-10-21T17:39:40.377135+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:21.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.284566+0000","last_change":"2023-10-21T17:39:40.277381+0000","last_active":"2023-10-21T17:39:40.284566+0000","last_peered":"2023-10-21T17:39:40.284566+0000","last_clean":"2023-10-21T17:39:40.284566+0000","last_became_active":"2023-10-21T17:39:40.277018+0000","last_became_peered":"2023-10-21T17:39:40.277018+0000","last_unstale":"2023-10-21T17:39:40.284566+0000","last_undegraded":"2023-10-21T17:39:40.284566+0000","last_fullsized":"2023-10-21T17:39:40.284566+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:44:11.077151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251068+0000","last_change":"2023-10-21T17:39:39.248091+0000","last_active":"2023-10-21T17:39:41.251068+0000","last_peered":"2023-10-21T17:39:41.251068+0000","last_clean":"2023-10-21T17:39:41.251068+0000","last_became_active":"2023-10-21T17:39:39.247775+0000","last_became_peered":"2023-10-21T17:39:39.247775+0000","last_unstale":"2023-10-21T17:39:41.251068+0000","last_undegraded":"2023-10-21T17:39:41.251068+0000","last_fullsized":"2023-10-21T17:39:41.251068+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:43.183503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251243+0000","last_change":"2023-10-21T17:39:40.268499+0000","last_active":"2023-10-21T17:39:41.251243+0000","last_peered":"2023-10-21T17:39:41.251243+0000","last_clean":"2023-10-21T17:39:41.251243+0000","last_became_active":"2023-10-21T17:39:40.268106+0000","last_became_peered":"2023-10-21T17:39:40.268106+0000","last_unstale":"2023-10-21T17:39:41.251243+0000","last_undegraded":"2023-10-21T17:39:41.251243+0000","last_fullsized":"2023-10-21T17:39:41.251243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:09.587502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253013+0000","last_change":"2023-10-21T17:39:39.237722+0000","last_active":"2023-10-21T17:39:41.253013+0000","last_peered":"2023-10-21T17:39:41.253013+0000","last_clean":"2023-10-21T17:39:41.253013+0000","last_became_active":"2023-10-21T17:39:39.235932+0000","last_became_peered":"2023-10-21T17:39:39.235932+0000","last_unstale":"2023-10-21T17:39:41.253013+0000","last_undegraded":"2023-10-21T17:39:41.253013+0000","last_fullsized":"2023-10-21T17:39:41.253013+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:08.581881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285958+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:40.285958+0000","last_peered":"2023-10-21T17:39:40.285958+0000","last_clean":"2023-10-21T17:39:40.285958+0000","last_became_active":"2023-10-21T17:39:40.282699+0000","last_became_peered":"2023-10-21T17:39:40.282699+0000","last_unstale":"2023-10-21T17:39:40.285958+0000","last_undegraded":"2023-10-21T17:39:40.285958+0000","last_fullsized":"2023-10-21T17:39:40.285958+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:09:33.969141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.272521+0000","last_change":"2023-10-21T17:39:39.247859+0000","last_active":"2023-10-21T17:39:40.272521+0000","last_peered":"2023-10-21T17:39:40.272521+0000","last_clean":"2023-10-21T17:39:40.272521+0000","last_became_active":"2023-10-21T17:39:39.245045+0000","last_became_peered":"2023-10-21T17:39:39.245045+0000","last_unstale":"2023-10-21T17:39:40.272521+0000","last_undegraded":"2023-10-21T17:39:40.272521+0000","last_fullsized":"2023-10-21T17:39:40.272521+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:22.615313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.286282+0000","last_change":"2023-10-21T17:39:40.283322+0000","last_active":"2023-10-21T17:39:40.286282+0000","last_peered":"2023-10-21T17:39:40.286282+0000","last_clean":"2023-10-21T17:39:40.286282+0000","last_became_active":"2023-10-21T17:39:40.282539+0000","last_became_peered":"2023-10-21T17:39:40.282539+0000","last_unstale":"2023-10-21T17:39:40.286282+0000","last_undegraded":"2023-10-21T17:39:40.286282+0000","last_fullsized":"2023-10-21T17:39:40.286282+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:38:27.772200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.381566+0000","last_change":"2023-10-21T17:39:39.242639+0000","last_active":"2023-10-21T17:39:40.381566+0000","last_peered":"2023-10-21T17:39:40.381566+0000","last_clean":"2023-10-21T17:39:40.381566+0000","last_became_active":"2023-10-21T17:39:39.242334+0000","last_became_peered":"2023-10-21T17:39:39.242334+0000","last_unstale":"2023-10-21T17:39:40.381566+0000","last_undegraded":"2023-10-21T17:39:40.381566+0000","last_fullsized":"2023-10-21T17:39:40.381566+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:44.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250941+0000","last_change":"2023-10-21T17:39:40.270958+0000","last_active":"2023-10-21T17:39:41.250941+0000","last_peered":"2023-10-21T17:39:41.250941+0000","last_clean":"2023-10-21T17:39:41.250941+0000","last_became_active":"2023-10-21T17:39:40.270640+0000","last_became_peered":"2023-10-21T17:39:40.270640+0000","last_unstale":"2023-10-21T17:39:41.250941+0000","last_undegraded":"2023-10-21T17:39:41.250941+0000","last_fullsized":"2023-10-21T17:39:41.250941+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:21.522147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252521+0000","last_change":"2023-10-21T17:39:39.246257+0000","last_active":"2023-10-21T17:39:41.252521+0000","last_peered":"2023-10-21T17:39:41.252521+0000","last_clean":"2023-10-21T17:39:41.252521+0000","last_became_active":"2023-10-21T17:39:39.245252+0000","last_became_peered":"2023-10-21T17:39:39.245252+0000","last_unstale":"2023-10-21T17:39:41.252521+0000","last_undegraded":"2023-10-21T17:39:41.252521+0000","last_fullsized":"2023-10-21T17:39:41.252521+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:18.223298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253815+0000","last_change":"2023-10-21T17:39:40.282652+0000","last_active":"2023-10-21T17:39:41.253815+0000","last_peered":"2023-10-21T17:39:41.253815+0000","last_clean":"2023-10-21T17:39:41.253815+0000","last_became_active":"2023-10-21T17:39:40.282108+0000","last_became_peered":"2023-10-21T17:39:40.282108+0000","last_unstale":"2023-10-21T17:39:41.253815+0000","last_undegraded":"2023-10-21T17:39:41.253815+0000","last_fullsized":"2023-10-21T17:39:41.253815+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:38.363235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252540+0000","last_change":"2023-10-21T17:39:39.242559+0000","last_active":"2023-10-21T17:39:41.252540+0000","last_peered":"2023-10-21T17:39:41.252540+0000","last_clean":"2023-10-21T17:39:41.252540+0000","last_became_active":"2023-10-21T17:39:39.236694+0000","last_became_peered":"2023-10-21T17:39:39.236694+0000","last_unstale":"2023-10-21T17:39:41.252540+0000","last_undegraded":"2023-10-21T17:39:41.252540+0000","last_fullsized":"2023-10-21T17:39:41.252540+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:58.202607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253983+0000","last_change":"2023-10-21T17:39:40.268873+0000","last_active":"2023-10-21T17:39:41.253983+0000","last_peered":"2023-10-21T17:39:41.253983+0000","last_clean":"2023-10-21T17:39:41.253983+0000","last_became_active":"2023-10-21T17:39:40.268515+0000","last_became_peered":"2023-10-21T17:39:40.268515+0000","last_unstale":"2023-10-21T17:39:41.253983+0000","last_undegraded":"2023-10-21T17:39:41.253983+0000","last_fullsized":"2023-10-21T17:39:41.253983+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:24.479277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.245568+0000","last_change":"2023-10-21T17:39:39.241987+0000","last_active":"2023-10-21T17:39:40.245568+0000","last_peered":"2023-10-21T17:39:40.245568+0000","last_clean":"2023-10-21T17:39:40.245568+0000","last_became_active":"2023-10-21T17:39:39.241676+0000","last_became_peered":"2023-10-21T17:39:39.241676+0000","last_unstale":"2023-10-21T17:39:40.245568+0000","last_undegraded":"2023-10-21T17:39:40.245568+0000","last_fullsized":"2023-10-21T17:39:40.245568+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:33.595143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.248937+0000","last_change":"2023-10-21T17:39:40.282789+0000","last_active":"2023-10-21T17:39:41.248937+0000","last_peered":"2023-10-21T17:39:41.248937+0000","last_clean":"2023-10-21T17:39:41.248937+0000","last_became_active":"2023-10-21T17:39:40.282503+0000","last_became_peered":"2023-10-21T17:39:40.282503+0000","last_unstale":"2023-10-21T17:39:41.248937+0000","last_undegraded":"2023-10-21T17:39:41.248937+0000","last_fullsized":"2023-10-21T17:39:41.248937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.381961+0000","last_change":"2023-10-21T17:39:39.245748+0000","last_active":"2023-10-21T17:39:40.381961+0000","last_peered":"2023-10-21T17:39:40.381961+0000","last_clean":"2023-10-21T17:39:40.381961+0000","last_became_active":"2023-10-21T17:39:39.245479+0000","last_became_peered":"2023-10-21T17:39:39.245479+0000","last_unstale":"2023-10-21T17:39:40.381961+0000","last_undegraded":"2023-10-21T17:39:40.381961+0000","last_fullsized":"2023-10-21T17:39:40.381961+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280342+0000","last_change":"2023-10-21T17:39:40.277680+0000","last_active":"2023-10-21T17:39:40.280342+0000","last_peered":"2023-10-21T17:39:40.280342+0000","last_clean":"2023-10-21T17:39:40.280342+0000","last_became_active":"2023-10-21T17:39:40.277441+0000","last_became_peered":"2023-10-21T17:39:40.277441+0000","last_unstale":"2023-10-21T17:39:40.280342+0000","last_undegraded":"2023-10-21T17:39:40.280342+0000","last_fullsized":"2023-10-21T17:39:40.280342+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:19.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.275748+0000","last_change":"2023-10-21T17:39:39.246309+0000","last_active":"2023-10-21T17:39:40.275748+0000","last_peered":"2023-10-21T17:39:40.275748+0000","last_clean":"2023-10-21T17:39:40.275748+0000","last_became_active":"2023-10-21T17:39:39.245706+0000","last_became_peered":"2023-10-21T17:39:39.245706+0000","last_unstale":"2023-10-21T17:39:40.275748+0000","last_undegraded":"2023-10-21T17:39:40.275748+0000","last_fullsized":"2023-10-21T17:39:40.275748+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:35.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285188+0000","last_change":"2023-10-21T17:39:40.281463+0000","last_active":"2023-10-21T17:39:40.285188+0000","last_peered":"2023-10-21T17:39:40.285188+0000","last_clean":"2023-10-21T17:39:40.285188+0000","last_became_active":"2023-10-21T17:39:40.281137+0000","last_became_peered":"2023-10-21T17:39:40.281137+0000","last_unstale":"2023-10-21T17:39:40.285188+0000","last_undegraded":"2023-10-21T17:39:40.285188+0000","last_fullsized":"2023-10-21T17:39:40.285188+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:37.556324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.246065+0000","last_change":"2023-10-21T17:39:39.227041+0000","last_active":"2023-10-21T17:39:40.246065+0000","last_peered":"2023-10-21T17:39:40.246065+0000","last_clean":"2023-10-21T17:39:40.246065+0000","last_became_active":"2023-10-21T17:39:39.225535+0000","last_became_peered":"2023-10-21T17:39:39.225535+0000","last_unstale":"2023-10-21T17:39:40.246065+0000","last_undegraded":"2023-10-21T17:39:40.246065+0000","last_fullsized":"2023-10-21T17:39:40.246065+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:20.950421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.285434+0000","last_change":"2023-10-21T17:39:40.283081+0000","last_active":"2023-10-21T17:39:40.285434+0000","last_peered":"2023-10-21T17:39:40.285434+0000","last_clean":"2023-10-21T17:39:40.285434+0000","last_became_active":"2023-10-21T17:39:40.282782+0000","last_became_peered":"2023-10-21T17:39:40.282782+0000","last_unstale":"2023-10-21T17:39:40.285434+0000","last_undegraded":"2023-10-21T17:39:40.285434+0000","last_fullsized":"2023-10-21T17:39:40.285434+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:08:31.084424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.249479+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:40.249479+0000","last_peered":"2023-10-21T17:39:40.249479+0000","last_clean":"2023-10-21T17:39:40.249479+0000","last_became_active":"2023-10-21T17:39:39.241658+0000","last_became_peered":"2023-10-21T17:39:39.241658+0000","last_unstale":"2023-10-21T17:39:40.249479+0000","last_undegraded":"2023-10-21T17:39:40.249479+0000","last_fullsized":"2023-10-21T17:39:40.249479+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271687+0000","last_change":"2023-10-21T17:39:40.267491+0000","last_active":"2023-10-21T17:39:40.271687+0000","last_peered":"2023-10-21T17:39:40.271687+0000","last_clean":"2023-10-21T17:39:40.271687+0000","last_became_active":"2023-10-21T17:39:40.267154+0000","last_became_peered":"2023-10-21T17:39:40.267154+0000","last_unstale":"2023-10-21T17:39:40.271687+0000","last_undegraded":"2023-10-21T17:39:40.271687+0000","last_fullsized":"2023-10-21T17:39:40.271687+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:51.348415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253254+0000","last_change":"2023-10-21T17:39:39.247298+0000","last_active":"2023-10-21T17:39:41.253254+0000","last_peered":"2023-10-21T17:39:41.253254+0000","last_clean":"2023-10-21T17:39:41.253254+0000","last_became_active":"2023-10-21T17:39:39.247016+0000","last_became_peered":"2023-10-21T17:39:39.247016+0000","last_unstale":"2023-10-21T17:39:41.253254+0000","last_undegraded":"2023-10-21T17:39:41.253254+0000","last_fullsized":"2023-10-21T17:39:41.253254+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:58.778201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.280334+0000","last_change":"2023-10-21T17:39:40.275609+0000","last_active":"2023-10-21T17:39:40.280334+0000","last_peered":"2023-10-21T17:39:40.280334+0000","last_clean":"2023-10-21T17:39:40.280334+0000","last_became_active":"2023-10-21T17:39:40.275290+0000","last_became_peered":"2023-10-21T17:39:40.275290+0000","last_unstale":"2023-10-21T17:39:40.280334+0000","last_undegraded":"2023-10-21T17:39:40.280334+0000","last_fullsized":"2023-10-21T17:39:40.280334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:37:59.855693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.271370+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:40.271370+0000","last_peered":"2023-10-21T17:39:40.271370+0000","last_clean":"2023-10-21T17:39:40.271370+0000","last_became_active":"2023-10-21T17:39:39.238047+0000","last_became_peered":"2023-10-21T17:39:39.238047+0000","last_unstale":"2023-10-21T17:39:40.271370+0000","last_undegraded":"2023-10-21T17:39:40.271370+0000","last_fullsized":"2023-10-21T17:39:40.271370+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.250243+0000","last_change":"2023-10-21T17:39:40.279881+0000","last_active":"2023-10-21T17:39:41.250243+0000","last_peered":"2023-10-21T17:39:41.250243+0000","last_clean":"2023-10-21T17:39:41.250243+0000","last_became_active":"2023-10-21T17:39:40.279567+0000","last_became_peered":"2023-10-21T17:39:40.279567+0000","last_unstale":"2023-10-21T17:39:41.250243+0000","last_undegraded":"2023-10-21T17:39:41.250243+0000","last_fullsized":"2023-10-21T17:39:41.250243+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:23.894434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":11,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.269288+0000","last_change":"2023-10-21T17:39:39.244421+0000","last_active":"2023-10-21T17:39:40.269288+0000","last_peered":"2023-10-21T17:39:40.269288+0000","last_clean":"2023-10-21T17:39:40.269288+0000","last_became_active":"2023-10-21T17:39:39.244026+0000","last_became_peered":"2023-10-21T17:39:39.244026+0000","last_unstale":"2023-10-21T17:39:40.269288+0000","last_undegraded":"2023-10-21T17:39:40.269288+0000","last_fullsized":"2023-10-21T17:39:40.269288+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:24.596691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.253780+0000","last_change":"2023-10-21T17:39:40.275308+0000","last_active":"2023-10-21T17:39:41.253780+0000","last_peered":"2023-10-21T17:39:41.253780+0000","last_clean":"2023-10-21T17:39:41.253780+0000","last_became_active":"2023-10-21T17:39:40.274998+0000","last_became_peered":"2023-10-21T17:39:40.274998+0000","last_unstale":"2023-10-21T17:39:41.253780+0000","last_undegraded":"2023-10-21T17:39:41.253780+0000","last_fullsized":"2023-10-21T17:39:41.253780+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:38.521395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.252009+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:41.252009+0000","last_peered":"2023-10-21T17:39:41.252009+0000","last_clean":"2023-10-21T17:39:41.252009+0000","last_became_active":"2023-10-21T17:39:39.236393+0000","last_became_peered":"2023-10-21T17:39:39.236393+0000","last_unstale":"2023-10-21T17:39:41.252009+0000","last_undegraded":"2023-10-21T17:39:41.252009+0000","last_fullsized":"2023-10-21T17:39:41.252009+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:40.507879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-21T17:39:40.277397+0000","last_change":"2023-10-21T17:39:40.270271+0000","last_active":"2023-10-21T17:39:40.277397+0000","last_peered":"2023-10-21T17:39:40.277397+0000","last_clean":"2023-10-21T17:39:40.277397+0000","last_became_active":"2023-10-21T17:39:40.269983+0000","last_became_peered":"2023-10-21T17:39:40.269983+0000","last_unstale":"2023-10-21T17:39:40.277397+0000","last_undegraded":"2023-10-21T17:39:40.277397+0000","last_fullsized":"2023-10-21T17:39:40.277397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:56.117575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-21T17:39:41.251499+0000","last_change":"2023-10-21T17:39:39.247657+0000","last_active":"2023-10-21T17:39:41.251499+0000","last_peered":"2023-10-21T17:39:41.251499+0000","last_clean":"2023-10-21T17:39:41.251499+0000","last_became_active":"2023-10-21T17:39:39.247152+0000","last_became_peered":"2023-10-21T17:39:39.247152+0000","last_unstale":"2023-10-21T17:39:41.251499+0000","last_undegraded":"2023-10-21T17:39:41.251499+0000","last_fullsized":"2023-10-21T17:39:41.251499+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:46.448339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"239'4895","reported_seq":5898,"reported_epoch":252,"state":"active+clean","last_fresh":"2023-10-21T17:39:39.236490+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:39:39.236490+0000","last_peered":"2023-10-21T17:39:39.236490+0000","last_clean":"2023-10-21T17:39:39.236490+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:39:39.236490+0000","last_undegraded":"2023-10-21T17:39:39.236490+0000","last_fullsized":"2023-10-21T17:39:39.236490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4895,"ondisk_log_size":4895,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576010,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75132,"kb_used_data":3700,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296708,"statfs":{"total":96636764160,"available":96559828992,"internally_reserved":0,"allocated":3788800,"data_stored":2250274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.667},{"interface":"front","average":{"1min":1.286,"5min":1.135,"15min":1.177},"min":{"1min":0.74299999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.81200000000000006}]},{"osd":1,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1619999999999999,"15min":1.212},"min":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.615,"5min":1.7170000000000001,"15min":1.73},"last":1.194},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.26,"15min":1.298},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.611,"5min":1.792,"15min":1.885},"last":1.5600000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.3759999999999999,"15min":1.3480000000000001},"min":{"1min":0.78100000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":1.139}]},{"osd":3,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2849999999999999,"15min":1.3129999999999999},"min":{"1min":0.624,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6739999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.343},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4990000000000001,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.383}]},{"osd":4,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4019999999999999,"15min":1.367},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9039999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5129999999999999,"15min":1.492},"min":{"1min":0.76100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.804,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.4199999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.508,"15min":1.421},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.837,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.246},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.49,"15min":1.456},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.036,"5min":2.613,"15min":2.613},"last":1.5069999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1019999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.4810000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.919,"5min":2.879,"15min":2.879},"last":1.5980000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42628,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329212,"statfs":{"total":96636764160,"available":96593113088,"internally_reserved":0,"allocated":3194880,"data_stored":1659894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.133,"15min":1.1890000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.671,"5min":1.944,"15min":1.944},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.163,"15min":1.212},"min":{"1min":0.71499999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.607,"5min":1.6839999999999999,"15min":1.762},"last":1.1220000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.121,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.474,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.258,"15min":1.302},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5029999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.5049999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.254,"15min":1.3049999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.532,"5min":1.706,"15min":1.8300000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.92200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.516,"5min":1.679,"15min":1.929},"last":0.83199999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.349,"15min":1.403},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.96,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4019999999999999,"15min":1.4510000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.5800000000000001,"5min":1.871,"15min":2.048},"last":1.1970000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.28,"15min":1.341},"min":{"1min":0.97299999999999998,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.409,"15min":1.4330000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.806,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5489999999999999,"15min":1.5669999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.339},"last":1.552},{"interface":"front","average":{"1min":1.369,"5min":1.3260000000000001,"15min":1.369},"min":{"1min":1.0169999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.3009999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4650000000000001,"15min":1.526},"min":{"1min":1.012,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.627,"15min":1.663},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0449999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.5409999999999999},"last":1.6499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64824,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307016,"statfs":{"total":96636764160,"available":96570384384,"internally_reserved":0,"allocated":3186688,"data_stored":1659782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.1439999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.611,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.161,"15min":1.218},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.343}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2729999999999999,"15min":1.27},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6719999999999999,"5min":1.738,"15min":1.8480000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1659999999999999,"15min":1.2230000000000001},"min":{"1min":0.74399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.532,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":0.82999999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.802},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.224,"15min":1.244},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.202}]},{"osd":3,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3080000000000001,"15min":1.343},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.639,"5min":1.758,"15min":1.829},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.379,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.897},"last":1.2549999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.234,"15min":1.25},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.9910000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.415,"15min":1.4039999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.4930000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1.0369999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8360000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":1.1399999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.52},"last":1.661}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":0.96199999999999997,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.0249999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.569,"5min":1.5289999999999999,"15min":1.5649999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.1349999999999998},"last":1.5600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52356,"kb_used_data":3128,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319484,"statfs":{"total":96636764160,"available":96583151616,"internally_reserved":0,"allocated":3203072,"data_stored":1666588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.0980000000000001,"15min":1.1559999999999999},"min":{"1min":0.59799999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.6850000000000001,"5min":1.746,"15min":1.8480000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.179,"15min":1.2130000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5269999999999999,"5min":1.659,"15min":1.869},"last":1.105}]},{"osd":1,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.234,"15min":1.236},"min":{"1min":0.754,"5min":0.754,"15min":0.58099999999999996},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.2270000000000001,"15min":1.2569999999999999},"min":{"1min":0.79000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6870000000000001,"5min":1.762,"15min":1.762},"last":0.73499999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.24,"15min":1.26},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.605,"5min":1.71,"15min":1.7609999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3560000000000001,"15min":1.3859999999999999},"min":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3069999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.2509999999999999,"15min":1.3089999999999999},"min":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.5640000000000001,"5min":1.7190000000000001,"15min":1.7949999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.395,"15min":1.4219999999999999},"min":{"1min":0.98599999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.611,"5min":1.825,"15min":1.9239999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3500000000000001,"15min":1.3420000000000001},"min":{"1min":0.872,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.7430000000000001,"5min":1.8340000000000001,"15min":1.867},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.3340000000000001,"15min":1.3580000000000001},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.81000000000000005},"max":{"1min":1.776,"5min":1.8899999999999999,"15min":1.9810000000000001},"last":0.85299999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.405,"15min":1.4419999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.732,"5min":1.8979999999999999,"15min":2.1600000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.5860000000000001,"15min":1.5940000000000001},"min":{"1min":1.056,"5min":1.056,"15min":1.056},"max":{"1min":1.9330000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.2250000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:38:52 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5049999999999999,"15min":1.514},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.869,"5min":2.0430000000000001,"15min":2.234},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5740000000000001,"15min":1.5820000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.8759999999999999,"5min":1.984,"15min":2.081},"last":1.3480000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576010,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56528,"kb_used_data":3136,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315312,"statfs":{"total":96636764160,"available":96578879488,"internally_reserved":0,"allocated":3211264,"data_stored":1662032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.462,"5min":1.6439999999999999,"15min":1.71},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.234,"15min":1.258},"min":{"1min":0.68000000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0800000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2350000000000001,"5min":1.835,"15min":1.835},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3240000000000001},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.6870000000000001,"5min":1.879,"15min":1.879},"last":1.036}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0740000000000001,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6539999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.753},{"interface":"front","average":{"1min":1.2,"5min":1.23,"15min":1.286},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2},"last":1.24}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.403,"15min":1.4370000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.143},{"interface":"front","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.4650000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.169},"last":1.5549999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4239999999999999,"15min":1.4910000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9379999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.365},{"interface":"front","average":{"1min":1.528,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.92600000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.484}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.5900000000000001},"min":{"1min":0.92800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1779999999999999,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.849},{"interface":"front","average":{"1min":1.472,"5min":1.552,"15min":1.6160000000000001},"min":{"1min":0.998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0209999999999999,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.633}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.637,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.921,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.038,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1230000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.9199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56512,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315328,"statfs":{"total":96636764160,"available":96578895872,"internally_reserved":0,"allocated":3194880,"data_stored":1660434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93100000000000005,"15min":0.90900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.585,"5min":1.907,"15min":2.0059999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.321,"15min":1.361},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.585,"5min":1.958,"15min":1.958},"last":0.66200000000000003}]},{"osd":3,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1950000000000001,"15min":1.1930000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.466,"15min":1.4830000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.252}]},{"osd":4,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4510000000000001,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4470000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.5289999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.4059999999999999,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.845,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.45},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6240000000000001,"15min":1.613},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.7430000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.5309999999999999,"15min":1.5620000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.855,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.919},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.5169999999999999,"15min":1.52},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.1160000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.631,"15min":1.6339999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9910000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5780000000000001,"15min":1.593},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0739999999999998,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":3.0230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52340,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319500,"statfs":{"total":96636764160,"available":96583168000,"internally_reserved":0,"allocated":3186688,"data_stored":1659782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.92700000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.411,"5min":1.411,"15min":1.976},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.0980000000000001,"15min":1.147},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.601,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.81100000000000005}]},{"osd":2,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.006,"15min":1.042},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.79},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.179,"15min":1.226},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5960000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.298,"15min":1.371},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9239999999999999,"5min":2.0539999999999998,"15min":2.129},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.2509999999999999,"15min":1.2929999999999999},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.85799999999999998}]},{"osd":4,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0139999999999998,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.45,"15min":1.484},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0089999999999999,"5min":2.0750000000000002,"15min":2.383},"last":1.329}]},{"osd":5,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.516,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.98,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.593},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.464,"15min":1.5249999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.1000000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.4139999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.569,"15min":1.595},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.2069999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.4890000000000001,"15min":1.5469999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.927,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.0229999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.609,"15min":1.663},"min":{"1min":0.95999999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1349999999999998,"5min":2.1829999999999998,"15min":2.548},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.696,"15min":1.7090000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.282,"5min":2.492,"15min":2.548},"last":1.5369999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576010,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52936,"kb_used_data":3704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318904,"statfs":{"total":96636764160,"available":96582557696,"internally_reserved":0,"allocated":3792896,"data_stored":2251762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":2.9249999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":1.085},"min":{"1min":0.61299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.556,"15min":1.748},"last":1.0209999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0980000000000001,"15min":1.163},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.504,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":2.1840000000000002},{"interface":"front","average":{"1min":1.046,"5min":1.0780000000000001,"15min":1.171},"min":{"1min":0.67500000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4259999999999999,"5min":1.708,"15min":1.851},"last":0.628}]},{"osd":3,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6359999999999999,"5min":1.897,"15min":1.9590000000000001},"last":2.597},{"interface":"front","average":{"1min":1.242,"5min":1.3140000000000001,"15min":1.3819999999999999},"min":{"1min":0.95599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.702,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":2.343}]},{"osd":4,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3340000000000001,"15min":1.421},"min":{"1min":0.877,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9470000000000001,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.764},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5289999999999999,"15min":1.5880000000000001},"min":{"1min":1.0620000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.917,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":2.113}]},{"osd":5,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.389,"15min":1.401},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9239999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.845},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":2.242}]},{"osd":6,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.82899999999999996,"5min":0.75,"15min":0.75},"max":{"1min":1.7310000000000001,"5min":2.0009999999999999,"15min":2.0139999999999998},"last":2.073},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.609,"15min":1.671},"min":{"1min":1.095,"5min":0.996,"15min":0.996},"max":{"1min":2.0920000000000001,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":2.2919999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4570000000000001,"15min":1.5309999999999999},"min":{"1min":1.002,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9910000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.571,"15min":1.6180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1819999999999999,"5min":2.2519999999999998,"15min":2.262},"last":2.3809999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:39:45.395 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:39:45.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:39:45.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.664+0000 7f281178a640 1 Processor -- start 2023-10-21T17:39:45.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.664+0000 7f281178a640 1 -- start start 2023-10-21T17:39:45.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f280c15cf90 0x7f280c1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f280c06b1e0 0x7f280c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c0ceca0 0x7f280c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c15d370 con 0x7f280c15cf90 2023-10-21T17:39:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c161910 con 0x7f280c06b1e0 2023-10-21T17:39:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f281178a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0cf670 con 0x7f280c0ceca0 2023-10-21T17:39:45.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f280affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f280c15cf90 0x7f280c1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f280affd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f280c15cf90 0x7f280c1613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54522/0 (socket says 172.21.15.53:54522) 2023-10-21T17:39:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.666+0000 7f280affd640 1 -- 172.21.15.53:0/697035882 learned_addr learned my addr 172.21.15.53:0/697035882 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280a7fc640 1 --2- 172.21.15.53:0/697035882 >> v2:172.21.15.124:3300/0 conn(0x7f280c06b1e0 0x7f280c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280b7fe640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c0ceca0 0x7f280c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280affd640 1 -- 172.21.15.53:0/697035882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c0ceca0 msgr2=0x7f280c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280affd640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c0ceca0 0x7f280c0cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280affd640 1 -- 172.21.15.53:0/697035882 >> v2:172.21.15.124:3300/0 conn(0x7f280c06b1e0 msgr2=0x7f280c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280affd640 1 --2- 172.21.15.53:0/697035882 >> v2:172.21.15.124:3300/0 conn(0x7f280c06b1e0 0x7f280c06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.667+0000 7f280affd640 1 -- 172.21.15.53:0/697035882 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c06b750 con 0x7f280c15cf90 2023-10-21T17:39:45.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.668+0000 7f280affd640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c15cf90 0x7f280c1613d0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f280c06fa50 tx=0x7f27fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34a6969c77629ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.668+0000 7f2809ffb640 1 -- 172.21.15.53:0/697035882 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc031870 con 0x7f280c15cf90 2023-10-21T17:39:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.668+0000 7f2809ffb640 1 -- 172.21.15.53:0/697035882 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc0319d0 con 0x7f280c15cf90 2023-10-21T17:39:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.668+0000 7f2809ffb640 1 -- 172.21.15.53:0/697035882 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc031d60 con 0x7f280c15cf90 2023-10-21T17:39:45.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 -- 172.21.15.53:0/697035882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c15cf90 msgr2=0x7f280c1613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c15cf90 0x7f280c1613d0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f280c06fa50 tx=0x7f27fc02b910 comp rx=0 tx=0).stop 2023-10-21T17:39:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 -- 172.21.15.53:0/697035882 shutdown_connections 2023-10-21T17:39:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c0ceca0 0x7f280c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 --2- 172.21.15.53:0/697035882 >> v2:172.21.15.124:3300/0 conn(0x7f280c06b1e0 0x7f280c06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 --2- 172.21.15.53:0/697035882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c15cf90 0x7f280c1613d0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 -- 172.21.15.53:0/697035882 >> 172.21.15.53:0/697035882 conn(0x7f280c06e630 msgr2=0x7f280c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 -- 172.21.15.53:0/697035882 shutdown_connections 2023-10-21T17:39:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.669+0000 7f281178a640 1 -- 172.21.15.53:0/697035882 wait complete. 2023-10-21T17:39:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.671+0000 7f281178a640 1 Processor -- start 2023-10-21T17:39:45.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.671+0000 7f281178a640 1 -- start start 2023-10-21T17:39:45.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f280c0ceca0 0x7f280c142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c15cf90 0x7f280c0c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0cc820 con 0x7f280c06b1e0 2023-10-21T17:39:45.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c142ad0 con 0x7f280c0ceca0 2023-10-21T17:39:45.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.672+0000 7f281178a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c15f670 con 0x7f280c15cf90 2023-10-21T17:39:45.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54526/0 (socket says 172.21.15.53:54526) 2023-10-21T17:39:45.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 -- 172.21.15.53:0/954564733 learned_addr learned my addr 172.21.15.53:0/954564733 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280b7fe640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c15cf90 0x7f280c0c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280a7fc640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f280c0ceca0 0x7f280c142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 -- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c15cf90 msgr2=0x7f280c0c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c15cf90 0x7f280c0c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 -- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f280c0ceca0 msgr2=0x7f280c142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f280c0ceca0 0x7f280c142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 -- 172.21.15.53:0/954564733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0094d0 con 0x7f280c06b1e0 2023-10-21T17:39:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.673+0000 7f280affd640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.674+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc03abe0 con 0x7f280c06b1e0 2023-10-21T17:39:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.674+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc03ad40 con 0x7f280c06b1e0 2023-10-21T17:39:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.674+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc044610 con 0x7f280c06b1e0 2023-10-21T17:39:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.674+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c15eaf0 con 0x7f280c06b1e0 2023-10-21T17:39:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.674+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c15f0b0 con 0x7f280c06b1e0 2023-10-21T17:39:45.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.675+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c0cc990 con 0x7f280c06b1e0 2023-10-21T17:39:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.676+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f27fc04d450 con 0x7f280c06b1e0 2023-10-21T17:39:45.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.677+0000 7f28037fe640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 0x7f27ec06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.677+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f27fc02d040 con 0x7f280c06b1e0 2023-10-21T17:39:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.678+0000 7f280a7fc640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 0x7f27ec06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.679+0000 7f280a7fc640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 0x7f27ec06d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f280c1605c0 tx=0x7f27f4003a30 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:45.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.681+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27fc076d50 con 0x7f280c06b1e0 2023-10-21T17:39:45.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.946+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f280c06fbc0 con 0x7f280c06b1e0 2023-10-21T17:39:45.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.951+0000 7f28037fe640 1 -- 172.21.15.53:0/954564733 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v52) v1 ==== 190+0+147 (secure 0 0 0) 0x7f27fc07a370 con 0x7f280c06b1e0 2023-10-21T17:39:45.953 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:39:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:39:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:39:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:39:45.953 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:39:45.954 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:39:45.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 msgr2=0x7f27ec06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 0x7f27ec06d5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f280c1605c0 tx=0x7f27f4003a30 comp rx=0 tx=0).stop 2023-10-21T17:39:45.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 msgr2=0x7f280c15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:45.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f27fc000c00 tx=0x7f27fc0046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 shutdown_connections 2023-10-21T17:39:45.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f27ec06b0e0 0x7f27ec06d5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f280c15cf90 0x7f280c0c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f280c0ceca0 0x7f280c142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 --2- 172.21.15.53:0/954564733 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f280c06b1e0 0x7f280c15e510 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 >> 172.21.15.53:0/954564733 conn(0x7f280c06e630 msgr2=0x7f280c0b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 shutdown_connections 2023-10-21T17:39:45.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:45.954+0000 7f281178a640 1 -- 172.21.15.53:0/954564733 wait complete. 2023-10-21T17:39:45.974 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:46.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.208+0000 7fd709dbd640 1 Processor -- start 2023-10-21T17:39:46.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.208+0000 7fd709dbd640 1 -- start start 2023-10-21T17:39:46.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd70415ab30 0x7fd70415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd70415af10 0x7fd7040b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70415f7d0 con 0x7fd70415af10 2023-10-21T17:39:46.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70415f210 con 0x7fd70415ab30 2023-10-21T17:39:46.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.209+0000 7fd709dbd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70415f4f0 con 0x7fd7040adc70 2023-10-21T17:39:46.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd70415af10 0x7fd7040b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:46.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd70415af10 0x7fd7040b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54532/0 (socket says 172.21.15.53:54532) 2023-10-21T17:39:46.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 -- 172.21.15.53:0/4175825038 learned_addr learned my addr 172.21.15.53:0/4175825038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:46.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd703fff640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:46.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd708dbb640 1 --2- 172.21.15.53:0/4175825038 >> v2:172.21.15.124:3300/0 conn(0x7fd70415ab30 0x7fd70415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:46.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 -- 172.21.15.53:0/4175825038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 msgr2=0x7fd7040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:46.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 -- 172.21.15.53:0/4175825038 >> v2:172.21.15.124:3300/0 conn(0x7fd70415ab30 msgr2=0x7fd70415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 --2- 172.21.15.53:0/4175825038 >> v2:172.21.15.124:3300/0 conn(0x7fd70415ab30 0x7fd70415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.210+0000 7fd7095bc640 1 -- 172.21.15.53:0/4175825038 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70406c550 con 0x7fd70415af10 2023-10-21T17:39:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.211+0000 7fd7095bc640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415af10 0x7fd7040b0600 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fd70406ada0 tx=0x7fd6fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2503c783a2326b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:46.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd7037fe640 1 -- 172.21.15.53:0/4175825038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc02ca80 con 0x7fd70415af10 2023-10-21T17:39:46.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd7037fe640 1 -- 172.21.15.53:0/4175825038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc02cbe0 con 0x7fd70415af10 2023-10-21T17:39:46.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd7037fe640 1 -- 172.21.15.53:0/4175825038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0343f0 con 0x7fd70415af10 2023-10-21T17:39:46.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 -- 172.21.15.53:0/4175825038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415af10 msgr2=0x7fd7040b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:46.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415af10 0x7fd7040b0600 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fd70406ada0 tx=0x7fd6fc02bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 -- 172.21.15.53:0/4175825038 shutdown_connections 2023-10-21T17:39:46.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 --2- 172.21.15.53:0/4175825038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415af10 0x7fd7040b0600 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 --2- 172.21.15.53:0/4175825038 >> v2:172.21.15.124:3300/0 conn(0x7fd70415ab30 0x7fd70415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.212+0000 7fd709dbd640 1 -- 172.21.15.53:0/4175825038 >> 172.21.15.53:0/4175825038 conn(0x7fd7040cb0c0 msgr2=0x7fd7040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:46.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.213+0000 7fd709dbd640 1 -- 172.21.15.53:0/4175825038 shutdown_connections 2023-10-21T17:39:46.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.213+0000 7fd709dbd640 1 -- 172.21.15.53:0/4175825038 wait complete. 2023-10-21T17:39:46.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.214+0000 7fd709dbd640 1 Processor -- start 2023-10-21T17:39:46.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.214+0000 7fd709dbd640 1 -- start start 2023-10-21T17:39:46.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd70415af10 0x7fd70415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd704150110 con 0x7fd70415ab30 2023-10-21T17:39:46.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70414fa70 con 0x7fd70415af10 2023-10-21T17:39:46.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.215+0000 7fd709dbd640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040c8160 con 0x7fd7040adc70 2023-10-21T17:39:46.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:46.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54538/0 (socket says 172.21.15.53:54538) 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 -- 172.21.15.53:0/2578726290 learned_addr learned my addr 172.21.15.53:0/2578726290 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 -- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 msgr2=0x7fd7040c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 -- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd70415af10 msgr2=0x7fd70415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd70415af10 0x7fd70415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 -- 172.21.15.53:0/2578726290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc009b30 con 0x7fd70415ab30 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.216+0000 7fd703fff640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000d800 tx=0x7fd6f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.217+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00096a0 con 0x7fd70415ab30 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.217+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f0009800 con 0x7fd70415ab30 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.217+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7040c8440 con 0x7fd70415ab30 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.217+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0009990 con 0x7fd70415ab30 2023-10-21T17:39:46.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.217+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7040c42b0 con 0x7fd70415ab30 2023-10-21T17:39:46.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.218+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd704150380 con 0x7fd70415ab30 2023-10-21T17:39:46.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.219+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd6f0009af0 con 0x7fd70415ab30 2023-10-21T17:39:46.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.220+0000 7fd7017fa640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 0x7fd6e806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:46.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.220+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd6f0089fe0 con 0x7fd70415ab30 2023-10-21T17:39:46.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.221+0000 7fd708dbb640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 0x7fd6e806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:46.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.222+0000 7fd708dbb640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 0x7fd6e806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8006fd0 tx=0x7fd6f8008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:46.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.224+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6f0054c40 con 0x7fd70415ab30 2023-10-21T17:39:46.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.458+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd70404e160 con 0x7fd70415ab30 2023-10-21T17:39:46.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.459+0000 7fd7017fa640 1 -- 172.21.15.53:0/2578726290 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4934 (secure 0 0 0) 0x7fd6f0058260 con 0x7fd70415ab30 2023-10-21T17:39:46.461 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:46.461 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":38345,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-21T17:39:46.461 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 128 2023-10-21T17:39:46.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 msgr2=0x7fd6e806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 0x7fd6e806d570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8006fd0 tx=0x7fd6f8008040 comp rx=0 tx=0).stop 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 msgr2=0x7fd7040b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000d800 tx=0x7fd6f000dcc0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 shutdown_connections 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fd6e806b0b0 0x7fd6e806d570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd70415af10 0x7fd70415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd70415ab30 0x7fd7040b1ef0 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 --2- 172.21.15.53:0/2578726290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7040adc70 0x7fd7040c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 >> 172.21.15.53:0/2578726290 conn(0x7fd7040cb0c0 msgr2=0x7fd7040b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.462+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 shutdown_connections 2023-10-21T17:39:46.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:46.463+0000 7fd709dbd640 1 -- 172.21.15.53:0/2578726290 wait complete. 2023-10-21T17:39:46.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:39:39.693015+0000', 'modified': '2023-10-21T17:39:42.885777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 38348}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_38348': {'gid': 38348, 'name': 'b', 'rank': 0, 'incarnation': 124, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6836/2789835635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6834', 'nonce': 2789835635}, {'type': 'v1', 'addr': '172.21.15.124:6836', 'nonce': 2789835635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:39:46.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:39:46.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-21T17:39:47.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.129+0000 7fdd234ea640 1 Processor -- start 2023-10-21T17:39:47.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.129+0000 7fdd234ea640 1 -- start start 2023-10-21T17:39:47.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd234ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0c8b40 0x7fdd1c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd234ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1c0aff70 0x7fdd1c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd234ea640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1c0ccbc0 con 0x7fdd1c15cae0 2023-10-21T17:39:47.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd234ea640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0b0940 con 0x7fdd1c0aff70 2023-10-21T17:39:47.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd234ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0ce1f0 con 0x7fdd1c0c8b40 2023-10-21T17:39:47.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.130+0000 7fdd22ce9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1c0aff70 0x7fdd1c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdd1c0aff70 0x7fdd1c0b0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54548/0 (socket says 172.21.15.53:54548) 2023-10-21T17:39:47.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 -- 172.21.15.53:0/3757993401 learned_addr learned my addr 172.21.15.53:0/3757993401 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0c8b40 0x7fdd1c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 -- 172.21.15.53:0/3757993401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0c8b40 msgr2=0x7fdd1c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0c8b40 0x7fdd1c0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 -- 172.21.15.53:0/3757993401 >> v1:172.21.15.124:6789/0 conn(0x7fdd1c15cae0 legacy=0x7fdd1c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.131+0000 7fdd22ce9640 1 -- 172.21.15.53:0/3757993401 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0c94c0 con 0x7fdd1c0aff70 2023-10-21T17:39:47.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.133+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c0aff70 0x7fdd1c0b0400 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009e50 tx=0x7fdd1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e0f7bac9ceae3c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.133+0000 7fdd214e6640 1 -- 172.21.15.53:0/3757993401 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1402ca80 con 0x7fdd1c0aff70 2023-10-21T17:39:47.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.133+0000 7fdd214e6640 1 -- 172.21.15.53:0/3757993401 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1402cbe0 con 0x7fdd1c0aff70 2023-10-21T17:39:47.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.133+0000 7fdd214e6640 1 -- 172.21.15.53:0/3757993401 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140343f0 con 0x7fdd1c0aff70 2023-10-21T17:39:47.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 -- 172.21.15.53:0/3757993401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c0aff70 msgr2=0x7fdd1c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c0aff70 0x7fdd1c0b0400 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009e50 tx=0x7fdd1402bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 -- 172.21.15.53:0/3757993401 shutdown_connections 2023-10-21T17:39:47.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0c8b40 0x7fdd1c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 --2- 172.21.15.53:0/3757993401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c0aff70 0x7fdd1c0b0400 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 -- 172.21.15.53:0/3757993401 >> 172.21.15.53:0/3757993401 conn(0x7fdd1c06c050 msgr2=0x7fdd1c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 -- 172.21.15.53:0/3757993401 shutdown_connections 2023-10-21T17:39:47.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.134+0000 7fdd234ea640 1 -- 172.21.15.53:0/3757993401 wait complete. 2023-10-21T17:39:47.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.136+0000 7fdd234ea640 1 Processor -- start 2023-10-21T17:39:47.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.136+0000 7fdd234ea640 1 -- start start 2023-10-21T17:39:47.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0aff70 0x7fdd1c15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1c0c8b40 0x7fdd1c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c15cec0 con 0x7fdd1c15cae0 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c157e30 con 0x7fdd1c0c8b40 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd234ea640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c15d160 con 0x7fdd1c0aff70 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd22ce9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd22ce9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54562/0 (socket says 172.21.15.53:54562) 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.137+0000 7fdd22ce9640 1 -- 172.21.15.53:0/4165380521 learned_addr learned my addr 172.21.15.53:0/4165380521 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1c0c8b40 0x7fdd1c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 -- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0aff70 msgr2=0x7fdd1c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0aff70 0x7fdd1c15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 -- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1c0c8b40 msgr2=0x7fdd1c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1c0c8b40 0x7fdd1c1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 -- 172.21.15.53:0/4165380521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009b30 con 0x7fdd1c15cae0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009e50 tx=0x7fdd14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.138+0000 7fdd12ffd640 1 -- 172.21.15.53:0/4165380521 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14009510 con 0x7fdd1c15cae0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.139+0000 7fdd12ffd640 1 -- 172.21.15.53:0/4165380521 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1403e450 con 0x7fdd1c15cae0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.139+0000 7fdd12ffd640 1 -- 172.21.15.53:0/4165380521 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1403e7e0 con 0x7fdd1c15cae0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.139+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c1f2770 con 0x7fdd1c15cae0 2023-10-21T17:39:47.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.139+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c1f2d30 con 0x7fdd1c15cae0 2023-10-21T17:39:47.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.141+0000 7fdd12ffd640 1 -- 172.21.15.53:0/4165380521 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fdd1402cd50 con 0x7fdd1c15cae0 2023-10-21T17:39:47.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.142+0000 7fdd12ffd640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 0x7fdd0006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.142+0000 7fdd12ffd640 1 -- 172.21.15.53:0/4165380521 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdd140ac800 con 0x7fdd1c15cae0 2023-10-21T17:39:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.143+0000 7fdd273bd740 1 Processor -- start 2023-10-21T17:39:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.143+0000 7fdd224e8640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 0x7fdd0006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.143+0000 7fdd273bd740 1 -- start start 2023-10-21T17:39:47.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd224e8640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 0x7fdd0006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd18006fd0 tx=0x7fdd18006bc0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 0x555a94e97f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 0x555a94e98440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94d887a0 con 0x555a94e95170 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94dda740 con 0x555a94e95550 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.144+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a94dd7e30 con 0x555a94e685b0 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54566/0 (socket says 172.21.15.53:54566) 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 -- 172.21.15.53:0/3663560898 learned_addr learned my addr 172.21.15.53:0/3663560898 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 0x555a94e97f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 -- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 msgr2=0x555a94e98440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 0x555a94e98440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 0x555a94e98440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 -- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 msgr2=0x555a94e97f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 0x555a94e97f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd224e8640 1 -- 172.21.15.53:0/3663560898 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a94d97910 con 0x555a94e95170 2023-10-21T17:39:47.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 0x555a94e97f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:47.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.145+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 0x555a94e98440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:47.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.146+0000 7fdd224e8640 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdd18013860 tx=0x7fdd1800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03b4274c425d6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.146+0000 7fdcfffff640 1 -- 172.21.15.53:0/3663560898 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18012b10 con 0x555a94e95170 2023-10-21T17:39:47.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.146+0000 7fdcfffff640 1 -- 172.21.15.53:0/3663560898 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18019040 con 0x555a94e95170 2023-10-21T17:39:47.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.146+0000 7fdcfffff640 1 -- 172.21.15.53:0/3663560898 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801d3f0 con 0x555a94e95170 2023-10-21T17:39:47.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 -- 172.21.15.53:0/3663560898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 msgr2=0x555a94e68070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdd18013860 tx=0x7fdd1800e000 comp rx=0 tx=0).stop 2023-10-21T17:39:47.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 -- 172.21.15.53:0/3663560898 shutdown_connections 2023-10-21T17:39:47.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e685b0 0x555a94e98440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95550 0x555a94e97f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3663560898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95170 0x555a94e68070 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 -- 172.21.15.53:0/3663560898 >> 172.21.15.53:0/3663560898 conn(0x555a94e6a380 msgr2=0x555a94eb2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 -- 172.21.15.53:0/3663560898 shutdown_connections 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.147+0000 7fdd273bd740 1 -- 172.21.15.53:0/3663560898 wait complete. 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.148+0000 7fdd273bd740 1 -- start start 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.148+0000 7fdd273bd740 1 Processor -- start 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94ec5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94dd3720 con 0x555a94e95550 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94d71970 con 0x555a94e95170 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a94d71610 con 0x555a94e9a700 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd273bd740 10 client.0 _ll_register_callbacks cb 0x555a94d939d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54578/0 (socket says 172.21.15.53:54578) 2023-10-21T17:39:47.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1486045753 learned_addr learned my addr 172.21.15.53:0/1486045753 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.149+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94ec5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 msgr2=0x555a94e9ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e9ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 msgr2=0x555a94ec5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94ec5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1486045753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd18013570 con 0x555a94e95550 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00deb0 tx=0x7fdd0c00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f7bbf1d6ca3292 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdcfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:39:47.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c014680 con 0x555a94e95550 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a94dfa160 con 0x555a94e95550 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdd0c0147e0 con 0x555a94e95550 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.150+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd0c0149a0 con 0x555a94e95550 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 10 client.12063 fetch_fsmap learned FSMap version 128 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x555a94dd1da0 con 0x555a94e95550 2023-10-21T17:39:47.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a94dfa160 con 0x555a94e95550 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd0c014b00 con 0x555a94e95550 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 10 client.12063 fetch_fsmap finished waiting for FSMap version 128 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 10 client.12063 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 20 client.12063 populate_metadata read hostname 'smithi053' 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94eab8d0 0x555a94ec64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 4 client.12063 mds_command: new command op to 38348 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:39:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.151+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 --> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a94da5cc0 con 0x555a94eab8d0 2023-10-21T17:39:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.152+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94eab8d0 0x555a94ec64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.153+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94eab8d0 0x555a94ec64b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8cf91d643b808f server_cookie=adf13c6f39ab04b2 in_seq=0 out_seq=0 2023-10-21T17:39:47.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.153+0000 7fdcfd7fa640 10 client.12063 ms_handle_connect on v2:172.21.15.124:6834/2789835635 2023-10-21T17:39:47.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1486045753 <== mds.0 v2:172.21.15.124:6834/2789835635 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555a94da5cc0 con 0x555a94eab8d0 2023-10-21T17:39:47.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdcfd7fa640 10 client.12063 handle_command_reply: tid=0 2023-10-21T17:39:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 client.12063 shutdown 2023-10-21T17:39:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 msgr2=0x555a94e9a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00deb0 tx=0x7fdd0c00f5b0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 shutdown_connections 2023-10-21T17:39:47.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94eab8d0 0x555a94ec64b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e9ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e9a1c0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1486045753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94ec5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdcfd7fa640 0 client.12063 ms_handle_reset on v2:172.21.15.124:6834/2789835635 2023-10-21T17:39:47.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 >> 172.21.15.53:0/1486045753 conn(0x555a94e685b0 msgr2=0x555a94e6bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 shutdown_connections 2023-10-21T17:39:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.155+0000 7fdd273bd740 1 -- 172.21.15.53:0/1486045753 wait complete. 2023-10-21T17:39:47.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.156+0000 7fdd273bd740 20 client.12063 trim_cache size 0 max 16384 2023-10-21T17:39:47.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.172+0000 7fdd273bd740 1 Processor -- start 2023-10-21T17:39:47.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.172+0000 7fdd273bd740 1 -- start start 2023-10-21T17:39:47.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94eadf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94b37f20 con 0x555a94e95550 2023-10-21T17:39:47.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94eaea00 con 0x555a94e95170 2023-10-21T17:39:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a94eaebe0 con 0x555a94e9a700 2023-10-21T17:39:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.173+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54592/0 (socket says 172.21.15.53:54592) 2023-10-21T17:39:47.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 -- 172.21.15.53:0/3035792958 learned_addr learned my addr 172.21.15.53:0/3035792958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 -- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 msgr2=0x555a94e97940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 -- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 msgr2=0x555a94eadf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94eadf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 -- 172.21.15.53:0/3035792958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a94d70b80 con 0x555a94e95550 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001eb0 tx=0x7fdd0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97beeab28f4b964b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.174+0000 7fdd10ff9640 1 -- 172.21.15.53:0/3035792958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c004220 con 0x555a94e95550 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd10ff9640 1 -- 172.21.15.53:0/3035792958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c00c030 con 0x555a94e95550 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd10ff9640 1 -- 172.21.15.53:0/3035792958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00fab0 con 0x555a94e95550 2023-10-21T17:39:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 -- 172.21.15.53:0/3035792958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 msgr2=0x555a94eae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c001eb0 tx=0x7fdd0c014000 comp rx=0 tx=0).stop 2023-10-21T17:39:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 -- 172.21.15.53:0/3035792958 shutdown_connections 2023-10-21T17:39:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e9a700 0x555a94e97940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94eae4c0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 --2- 172.21.15.53:0/3035792958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e95170 0x555a94eadf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 -- 172.21.15.53:0/3035792958 >> 172.21.15.53:0/3035792958 conn(0x555a94e685b0 msgr2=0x555a94e63810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 -- 172.21.15.53:0/3035792958 shutdown_connections 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.175+0000 7fdd273bd740 1 -- 172.21.15.53:0/3035792958 wait complete. 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.176+0000 7fdd273bd740 1 -- start start 2023-10-21T17:39:47.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.176+0000 7fdd273bd740 1 Processor -- start 2023-10-21T17:39:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e95170 0x555a94f01360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 0x555a94e877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94f018a0 con 0x555a94e95550 2023-10-21T17:39:47.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a94eb4f40 con 0x555a94e9a700 2023-10-21T17:39:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a94e991e0 con 0x555a94e95170 2023-10-21T17:39:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd273bd740 10 client.0 _ll_register_callbacks cb 0x7fdd0c00d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:39:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.177+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54602/0 (socket says 172.21.15.53:54602) 2023-10-21T17:39:47.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1842037864 learned_addr learned my addr 172.21.15.53:0/1842037864 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e95170 0x555a94f01360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 0x555a94e877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e95170 msgr2=0x555a94f01360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e95170 0x555a94f01360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 msgr2=0x555a94e877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 0x555a94e877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 -- 172.21.15.53:0/1842037864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a94dd1da0 con 0x555a94e95550 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd22ce9640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 0x555a94e877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdd21ce7640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00d440 tx=0x7fdd0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa843dd3f5e01761 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.178+0000 7fdcfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c014c40 con 0x555a94e95550 2023-10-21T17:39:47.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdd0c000e40 con 0x555a94e95550 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd18019040 con 0x555a94e95550 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd0c00fab0 con 0x555a94e95550 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 10 client.12069 fetch_fsmap learned FSMap version 128 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x555a94d96170 con 0x555a94e95550 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18019040 con 0x555a94e95550 2023-10-21T17:39:47.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 128) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd0c02e050 con 0x555a94e95550 2023-10-21T17:39:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 10 client.12069 fetch_fsmap finished waiting for FSMap version 128 2023-10-21T17:39:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 10 client.12069 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T17:39:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 20 client.12069 populate_metadata read hostname 'smithi053' 2023-10-21T17:39:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94e87e30 0x555a94eadd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 4 client.12069 mds_command: new command op to 38348 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.179+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 --> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a94dcc700 con 0x555a94e87e30 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.180+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94e87e30 0x555a94eadd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.181+0000 7fdd224e8640 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94e87e30 0x555a94eadd30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56535f0f9dbfc820 server_cookie=fab73a383bc59124 in_seq=0 out_seq=0 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.181+0000 7fdcfd7fa640 10 client.12069 ms_handle_connect on v2:172.21.15.124:6834/2789835635 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdcfd7fa640 1 -- 172.21.15.53:0/1842037864 <== mds.0 v2:172.21.15.124:6834/2789835635 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x555a94dcc700 con 0x555a94e87e30 2023-10-21T17:39:47.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdcfd7fa640 10 client.12069 handle_command_reply: tid=0 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 client.12069 shutdown 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 msgr2=0x555a94e885a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00d440 tx=0x7fdd0c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 shutdown_connections 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:6834/2789835635,v1:172.21.15.124:6836/2789835635] conn(0x555a94e87e30 0x555a94eadd30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x555a94e9a700 0x555a94e877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a94e95550 0x555a94e885a0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 --2- 172.21.15.53:0/1842037864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x555a94e95170 0x555a94f01360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdcfd7fa640 0 client.12069 ms_handle_reset on v2:172.21.15.124:6834/2789835635 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 >> 172.21.15.53:0/1842037864 conn(0x555a94e685b0 msgr2=0x555a94e63810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 shutdown_connections 2023-10-21T17:39:47.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.182+0000 7fdd273bd740 1 -- 172.21.15.53:0/1842037864 wait complete. 2023-10-21T17:39:47.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd273bd740 20 client.12069 trim_cache size 0 max 16384 2023-10-21T17:39:47.194 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:39:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:39:47.194 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 38348, 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8227027429999998, 2023-10-21T17:39:47.195 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 128, 2023-10-21T17:39:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 254, 2023-10-21T17:39:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 254, 2023-10-21T17:39:47.196 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.761837771 2023-10-21T17:39:47.196 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:39:47.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 msgr2=0x7fdd0006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 0x7fdd0006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd18006fd0 tx=0x7fdd18006bc0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 msgr2=0x7fdd1c1f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009e50 tx=0x7fdd14004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:47.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 shutdown_connections 2023-10-21T17:39:47.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdd0006b0b0 0x7fdd0006d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd1c15cae0 0x7fdd1c1f1f20 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd1c0c8b40 0x7fdd1c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 --2- 172.21.15.53:0/4165380521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd1c0aff70 0x7fdd1c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 >> 172.21.15.53:0/4165380521 conn(0x7fdd1c06c050 msgr2=0x7fdd1c071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 shutdown_connections 2023-10-21T17:39:47.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.183+0000 7fdd234ea640 1 -- 172.21.15.53:0/4165380521 wait complete. 2023-10-21T17:39:47.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:39:47.433 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.432+0000 7f1bf1bc0640 1 Processor -- start 2023-10-21T17:39:47.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.432+0000 7f1bf1bc0640 1 -- start start 2023-10-21T17:39:47.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1bf1bc0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1bec0709e0 0x7f1bec164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1bf1bc0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec06dee0 con 0x7f1bec0ad790 2023-10-21T17:39:47.435 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1bf1bc0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec15d850 con 0x7f1bec159ac0 2023-10-21T17:39:47.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1bf1bc0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec06dcf0 con 0x7f1bec0709e0 2023-10-21T17:39:47.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1be2ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1bec0ad790 0x7f1bec0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36222/0 (socket says 172.21.15.53:36222) 2023-10-21T17:39:47.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1be2ffd640 1 -- 172.21.15.53:0/3885187791 learned_addr learned my addr 172.21.15.53:0/3885187791 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:47.436 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.434+0000 7f1bebfff640 1 --2- 172.21.15.53:0/3885187791 >> v2:172.21.15.53:3300/0 conn(0x7f1bec0709e0 0x7f1bec164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.435+0000 7f1beaffd640 1 -- 172.21.15.53:0/3885187791 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652190930 0 0) 0x7f1bec06dee0 con 0x7f1bec0ad790 2023-10-21T17:39:47.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.435+0000 7f1beaffd640 1 -- 172.21.15.53:0/3885187791 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd8005730 con 0x7f1bec0ad790 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.435+0000 7f1bebfff640 1 -- 172.21.15.53:0/3885187791 >> v1:172.21.15.124:6789/0 conn(0x7f1bec159ac0 legacy=0x7f1bec06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.435+0000 7f1bebfff640 1 -- 172.21.15.53:0/3885187791 >> v1:172.21.15.53:6789/0 conn(0x7f1bec0ad790 legacy=0x7f1bec0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.435+0000 7f1bebfff640 1 -- 172.21.15.53:0/3885187791 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec15db30 con 0x7f1bec0709e0 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.436+0000 7f1bebfff640 1 --2- 172.21.15.53:0/3885187791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec164a60 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009d90 tx=0x7f1be402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48560f4ef2e2c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.436+0000 7f1beaffd640 1 -- 172.21.15.53:0/3885187791 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be402cc60 con 0x7f1bec0709e0 2023-10-21T17:39:47.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.436+0000 7f1beaffd640 1 -- 172.21.15.53:0/3885187791 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be402cdc0 con 0x7f1bec0709e0 2023-10-21T17:39:47.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.436+0000 7f1beaffd640 1 -- 172.21.15.53:0/3885187791 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4034660 con 0x7f1bec0709e0 2023-10-21T17:39:47.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/3885187791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 msgr2=0x7f1bec164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/3885187791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec164a60 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009d90 tx=0x7f1be402beb0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/3885187791 shutdown_connections 2023-10-21T17:39:47.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/3885187791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec164a60 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/3885187791 >> 172.21.15.53:0/3885187791 conn(0x7f1bec06a380 msgr2=0x7f1bec0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/3885187791 shutdown_connections 2023-10-21T17:39:47.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.437+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/3885187791 wait complete. 2023-10-21T17:39:47.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.439+0000 7f1bf1bc0640 1 Processor -- start 2023-10-21T17:39:47.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.439+0000 7f1bf1bc0640 1 -- start start 2023-10-21T17:39:47.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bec0ad790 0x7f1bec1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 0x7f1bec157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1f6490 con 0x7f1bec0709e0 2023-10-21T17:39:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1425f0 con 0x7f1bec159ac0 2023-10-21T17:39:47.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bf1bc0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1f6920 con 0x7f1bec0ad790 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1beb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1beb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54632/0 (socket says 172.21.15.53:54632) 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1beb7fe640 1 -- 172.21.15.53:0/2829368036 learned_addr learned my addr 172.21.15.53:0/2829368036 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.440+0000 7f1bebfff640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 0x7f1bec157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 -- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bec0ad790 msgr2=0x7f1bec1573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bec0ad790 0x7f1bec1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 -- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 msgr2=0x7f1bec157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 0x7f1bec157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 -- 172.21.15.53:0/2829368036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009a70 con 0x7f1bec0709e0 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1bebfff640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 0x7f1bec157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1beb7fe640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b3f0 tx=0x7f1bdc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc011830 con 0x7f1bec0709e0 2023-10-21T17:39:47.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc011990 con 0x7f1bec0709e0 2023-10-21T17:39:47.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc011d40 con 0x7f1bec0709e0 2023-10-21T17:39:47.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec15c050 con 0x7f1bec0709e0 2023-10-21T17:39:47.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.441+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1f5e40 con 0x7f1bec0709e0 2023-10-21T17:39:47.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.443+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec067490 con 0x7f1bec0709e0 2023-10-21T17:39:47.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.444+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1bdc01a400 con 0x7f1bec0709e0 2023-10-21T17:39:47.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.445+0000 7f1be8ff9640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 0x7f1bcc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:47.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.445+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1bdc089a00 con 0x7f1bec0709e0 2023-10-21T17:39:47.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.446+0000 7f1be2ffd640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 0x7f1bcc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:47.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.447+0000 7f1be2ffd640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 0x7f1bcc06d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0cf280 tx=0x7f1bd4007930 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:47.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.450+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bdc054590 con 0x7f1bec0709e0 2023-10-21T17:39:47.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.683+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1bec0b0ff0 con 0x7f1bec0709e0 2023-10-21T17:39:47.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.684+0000 7f1be8ff9640 1 -- 172.21.15.53:0/2829368036 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v128) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1bdc015070 con 0x7f1bec0709e0 2023-10-21T17:39:47.685 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:39:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.686+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 msgr2=0x7f1bcc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.686+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 0x7f1bcc06d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0cf280 tx=0x7f1bd4007930 comp rx=0 tx=0).stop 2023-10-21T17:39:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.686+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 msgr2=0x7f1bec1420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:47.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.686+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b3f0 tx=0x7f1bdc00b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 shutdown_connections 2023-10-21T17:39:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1bcc06b0b0 0x7f1bcc06d570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1bec159ac0 0x7f1bec157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1bec0ad790 0x7f1bec1573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 --2- 172.21.15.53:0/2829368036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bec0709e0 0x7f1bec1420b0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:47.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 >> 172.21.15.53:0/2829368036 conn(0x7f1bec06a380 msgr2=0x7f1bec0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:47.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 shutdown_connections 2023-10-21T17:39:47.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:47.687+0000 7f1bf1bc0640 1 -- 172.21.15.53:0/2829368036 wait complete. 2023-10-21T17:39:47.705 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:39:47.706 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:39:47.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:39:47.706 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:39:47.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:39:47.707 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:39:47.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:39:47.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:39:47.708 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:39:47.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:47.709 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:39:47.762 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:39:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:39:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:39:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:39:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.764 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:39:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:39:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:39:47.764 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:39:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:39:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:39:47.765 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:39:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:39:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:39:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:39:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 36505sec preferred_lft 36505sec 2023-10-21T17:39:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:39:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.766 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:39:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:39:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.768 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:39:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:39:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:39:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.769 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:39:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:39:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:39:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:39:47.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:47.771 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:39:47.867 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:39:47.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:47.871 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:39:47.925 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:39:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:39:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:39:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:39:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:39:47.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:47.928 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:39:47.928 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:39:47.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:39:47.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:39:47.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:39:47.929 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:39:48.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:48.129 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:39:48.182 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:39:48.183 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:39:48.184 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:39:48.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:48.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:39:48.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:39:48.301 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:39:48.359 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:39:48.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:39:48.360 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:39:48.414 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:39:48.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:48.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:39:48.571 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:39:48.572 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:39:48.606 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:39:48.606 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:39:48.607 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:39:48.607 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:39:48.607 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:39:48.609 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:39:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:39:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:39:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:39:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:39:48.613 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:39:48.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:48.614 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:39:48.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:39:48.718 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:39:48.801 INFO:teuthology.orchestra.run.smithi053.stdout:client9685 2023-10-21T17:39:48.801 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:39:48.802 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:39:48.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-21T17:39:49.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.106+0000 7fdec660e640 1 Processor -- start 2023-10-21T17:39:49.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.106+0000 7fdec660e640 1 -- start start 2023-10-21T17:39:49.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdec660e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdec00b1750 0x7fdec006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdec660e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdec006d240 0x7fdec0070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdec660e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec006d670 con 0x7fdec015aad0 2023-10-21T17:39:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdec660e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec015bf30 con 0x7fdec006d240 2023-10-21T17:39:49.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdec660e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec00b1bb0 con 0x7fdec00b1750 2023-10-21T17:39:49.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdebffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdec015aad0 0x7fdec00c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36238/0 (socket says 172.21.15.53:36238) 2023-10-21T17:39:49.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.108+0000 7fdebffff640 1 -- 172.21.15.53:0/1120171343 learned_addr learned my addr 172.21.15.53:0/1120171343 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:49.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdec4bab640 1 --2- 172.21.15.53:0/1120171343 >> v2:172.21.15.53:3300/0 conn(0x7fdec006d240 0x7fdec0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdeb7fff640 1 --2- 172.21.15.53:0/1120171343 >> v2:172.21.15.124:3300/0 conn(0x7fdec00b1750 0x7fdec006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdebf7fe640 1 -- 172.21.15.53:0/1120171343 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786699153 0 0) 0x7fdec006d670 con 0x7fdec015aad0 2023-10-21T17:39:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdebf7fe640 1 -- 172.21.15.53:0/1120171343 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeac003520 con 0x7fdec015aad0 2023-10-21T17:39:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdec4bab640 1 -- 172.21.15.53:0/1120171343 >> v2:172.21.15.124:3300/0 conn(0x7fdec00b1750 msgr2=0x7fdec006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdec4bab640 1 --2- 172.21.15.53:0/1120171343 >> v2:172.21.15.124:3300/0 conn(0x7fdec00b1750 0x7fdec006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdec4bab640 1 -- 172.21.15.53:0/1120171343 >> v1:172.21.15.53:6789/0 conn(0x7fdec015aad0 legacy=0x7fdec00c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.109+0000 7fdec4bab640 1 -- 172.21.15.53:0/1120171343 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec015b320 con 0x7fdec006d240 2023-10-21T17:39:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.110+0000 7fdec4bab640 1 --2- 172.21.15.53:0/1120171343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec006d240 0x7fdec0070570 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fdec006e330 tx=0x7fdeb802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211fd034dabdb1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdebf7fe640 1 -- 172.21.15.53:0/1120171343 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb802cc70 con 0x7fdec006d240 2023-10-21T17:39:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdebf7fe640 1 -- 172.21.15.53:0/1120171343 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb802cdd0 con 0x7fdec006d240 2023-10-21T17:39:49.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdebf7fe640 1 -- 172.21.15.53:0/1120171343 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8034660 con 0x7fdec006d240 2023-10-21T17:39:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 -- 172.21.15.53:0/1120171343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec006d240 msgr2=0x7fdec0070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 --2- 172.21.15.53:0/1120171343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec006d240 0x7fdec0070570 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fdec006e330 tx=0x7fdeb802bec0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 -- 172.21.15.53:0/1120171343 shutdown_connections 2023-10-21T17:39:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 --2- 172.21.15.53:0/1120171343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec006d240 0x7fdec0070570 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 --2- 172.21.15.53:0/1120171343 >> v2:172.21.15.124:3300/0 conn(0x7fdec00b1750 0x7fdec006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 -- 172.21.15.53:0/1120171343 >> 172.21.15.53:0/1120171343 conn(0x7fdec0069d50 msgr2=0x7fdec006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:49.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 -- 172.21.15.53:0/1120171343 shutdown_connections 2023-10-21T17:39:49.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.111+0000 7fdec660e640 1 -- 172.21.15.53:0/1120171343 wait complete. 2023-10-21T17:39:49.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.113+0000 7fdec660e640 1 Processor -- start 2023-10-21T17:39:49.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.113+0000 7fdec660e640 1 -- start start 2023-10-21T17:39:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 0x7fdec00c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdec015aad0 0x7fdec01566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c5c80 con 0x7fdec00b1750 2023-10-21T17:39:49.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c5f60 con 0x7fdec006d240 2023-10-21T17:39:49.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.114+0000 7fdec660e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec00c6240 con 0x7fdec015aad0 2023-10-21T17:39:49.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54654/0 (socket says 172.21.15.53:54654) 2023-10-21T17:39:49.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 -- 172.21.15.53:0/2100131279 learned_addr learned my addr 172.21.15.53:0/2100131279 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:49.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdebffff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 0x7fdec00c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 -- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdec015aad0 msgr2=0x7fdec01566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdec015aad0 0x7fdec01566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 -- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 msgr2=0x7fdec00c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 0x7fdec00c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 -- 172.21.15.53:0/2100131279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009a70 con 0x7fdec00b1750 2023-10-21T17:39:49.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdebffff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 0x7fdec00c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:49.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.115+0000 7fdeb7fff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9b0 tx=0x7fdea800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.116+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8009670 con 0x7fdec00b1750 2023-10-21T17:39:49.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.116+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea800c9a0 con 0x7fdec00b1750 2023-10-21T17:39:49.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.116+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800cd50 con 0x7fdec00b1750 2023-10-21T17:39:49.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.116+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec00c6520 con 0x7fdec00b1750 2023-10-21T17:39:49.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.116+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec0148910 con 0x7fdec00b1750 2023-10-21T17:39:49.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.117+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec004e160 con 0x7fdec00b1750 2023-10-21T17:39:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.118+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fdea8013470 con 0x7fdec00b1750 2023-10-21T17:39:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.119+0000 7fdebd7fa640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 0x7fdea006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.119+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdea8089ef0 con 0x7fdec00b1750 2023-10-21T17:39:49.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.120+0000 7fdebffff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 0x7fdea006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.121+0000 7fdebffff640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 0x7fdea006d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005880 tx=0x7fdeb0009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.124+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdea80549e0 con 0x7fdec00b1750 2023-10-21T17:39:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.354+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdec00b4610 con 0x7fdec00b1750 2023-10-21T17:39:49.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.418+0000 7fdebd7fa640 1 -- 172.21.15.53:0/2100131279 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v129) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdea8058000 con 0x7fdec00b1750 2023-10-21T17:39:49.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 msgr2=0x7fdea006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 0x7fdea006d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0005880 tx=0x7fdeb0009040 comp rx=0 tx=0).stop 2023-10-21T17:39:49.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 msgr2=0x7fdec01483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9b0 tx=0x7fdea800ee70 comp rx=0 tx=0).stop 2023-10-21T17:39:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 shutdown_connections 2023-10-21T17:39:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fdea006b0b0 0x7fdea006d570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdec015aad0 0x7fdec01566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdec00b1750 0x7fdec01483d0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 --2- 172.21.15.53:0/2100131279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdec006d240 0x7fdec00c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 >> 172.21.15.53:0/2100131279 conn(0x7fdec0069d50 msgr2=0x7fdec0154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 shutdown_connections 2023-10-21T17:39:49.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.422+0000 7fdec660e640 1 -- 172.21.15.53:0/2100131279 wait complete. 2023-10-21T17:39:49.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:49.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.669+0000 7f5322a4d640 1 Processor -- start 2023-10-21T17:39:49.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.669+0000 7f5322a4d640 1 -- start start 2023-10-21T17:39:49.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5322a4d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f531c1599d0 0x7f531c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5322a4d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 0x7f531c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5322a4d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c0cacf0 con 0x7f531c159db0 2023-10-21T17:39:49.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5322a4d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0b10f0 con 0x7f531c1599d0 2023-10-21T17:39:49.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5322a4d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0ca3c0 con 0x7f531c0cc210 2023-10-21T17:39:49.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f532224c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f531c159db0 0x7f531c15d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36240/0 (socket says 172.21.15.53:36240) 2023-10-21T17:39:49.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f5321a4b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f531c1599d0 0x7f531c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.671+0000 7f532224c640 1 -- 172.21.15.53:0/883940487 learned_addr learned my addr 172.21.15.53:0/883940487 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:49.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f532124a640 1 --2- 172.21.15.53:0/883940487 >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 0x7f531c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5320a49640 1 -- 172.21.15.53:0/883940487 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353655523 0 0) 0x7f531c0cacf0 con 0x7f531c159db0 2023-10-21T17:39:49.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5320a49640 1 -- 172.21.15.53:0/883940487 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5308005730 con 0x7f531c159db0 2023-10-21T17:39:49.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5321a4b640 1 -- 172.21.15.53:0/883940487 >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 msgr2=0x7f531c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:49.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5321a4b640 1 --2- 172.21.15.53:0/883940487 >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 0x7f531c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5321a4b640 1 -- 172.21.15.53:0/883940487 >> v1:172.21.15.53:6789/0 conn(0x7f531c159db0 legacy=0x7f531c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f5321a4b640 1 -- 172.21.15.53:0/883940487 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0ca6a0 con 0x7f531c1599d0 2023-10-21T17:39:49.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.672+0000 7f532124a640 1 --2- 172.21.15.53:0/883940487 >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 0x7f531c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:49.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.674+0000 7f5321a4b640 1 --2- 172.21.15.53:0/883940487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c1599d0 0x7f531c0ad730 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f53180097a0 tx=0x7f531802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f8011c5a45b213 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.674+0000 7f5320a49640 1 -- 172.21.15.53:0/883940487 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318031830 con 0x7f531c1599d0 2023-10-21T17:39:49.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.674+0000 7f5320a49640 1 -- 172.21.15.53:0/883940487 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318031990 con 0x7f531c1599d0 2023-10-21T17:39:49.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.674+0000 7f5320a49640 1 -- 172.21.15.53:0/883940487 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318031d20 con 0x7f531c1599d0 2023-10-21T17:39:49.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 -- 172.21.15.53:0/883940487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c1599d0 msgr2=0x7f531c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 --2- 172.21.15.53:0/883940487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c1599d0 0x7f531c0ad730 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f53180097a0 tx=0x7f531802b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 -- 172.21.15.53:0/883940487 shutdown_connections 2023-10-21T17:39:49.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 --2- 172.21.15.53:0/883940487 >> v2:172.21.15.124:3300/0 conn(0x7f531c0cc210 0x7f531c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 --2- 172.21.15.53:0/883940487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c1599d0 0x7f531c0ad730 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 -- 172.21.15.53:0/883940487 >> 172.21.15.53:0/883940487 conn(0x7f531c0cb0c0 msgr2=0x7f531c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:49.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 -- 172.21.15.53:0/883940487 shutdown_connections 2023-10-21T17:39:49.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.675+0000 7f5322a4d640 1 -- 172.21.15.53:0/883940487 wait complete. 2023-10-21T17:39:49.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.677+0000 7f5322a4d640 1 Processor -- start 2023-10-21T17:39:49.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.677+0000 7f5322a4d640 1 -- start start 2023-10-21T17:39:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f531c0cc210 0x7f531c14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 0x7f531c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c14ffa0 con 0x7f531c159db0 2023-10-21T17:39:49.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0ac2e0 con 0x7f531c1599d0 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f5322a4d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0c61e0 con 0x7f531c0cc210 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.678+0000 7f532224c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54672/0 (socket says 172.21.15.53:54672) 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 -- 172.21.15.53:0/1075572232 learned_addr learned my addr 172.21.15.53:0/1075572232 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f5321a4b640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f531c0cc210 0x7f531c14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532124a640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 0x7f531c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 -- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f531c0cc210 msgr2=0x7f531c14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f531c0cc210 0x7f531c14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 -- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 msgr2=0x7f531c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 0x7f531c0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 -- 172.21.15.53:0/1075572232 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009400 con 0x7f531c159db0 2023-10-21T17:39:49.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532124a640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 0x7f531c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:49.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.679+0000 7f532224c640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f531400caa0 tx=0x7f531400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.680+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400aba0 con 0x7f531c159db0 2023-10-21T17:39:49.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.680+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531400ad00 con 0x7f531c159db0 2023-10-21T17:39:49.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.680+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314012640 con 0x7f531c159db0 2023-10-21T17:39:49.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.680+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c0c64c0 con 0x7f531c159db0 2023-10-21T17:39:49.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.680+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0c5c50 con 0x7f531c159db0 2023-10-21T17:39:49.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.681+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c0ac7e0 con 0x7f531c159db0 2023-10-21T17:39:49.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.682+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5314007070 con 0x7f531c159db0 2023-10-21T17:39:49.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.683+0000 7f53127fc640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 0x7f530006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:49.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.683+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5314089aa0 con 0x7f531c159db0 2023-10-21T17:39:49.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.684+0000 7f5321a4b640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 0x7f530006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:49.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.685+0000 7f5321a4b640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 0x7f530006d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5318036000 tx=0x7f531803c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:49.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.689+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f531c0ac7e0 con 0x7f531c159db0 2023-10-21T17:39:49.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.887+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f531c0ac7e0 con 0x7f531c159db0 2023-10-21T17:39:49.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.890+0000 7f53127fc640 1 -- 172.21.15.53:0/1075572232 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f531c0ac7e0 con 0x7f531c159db0 2023-10-21T17:39:49.891 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:49.891 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":254,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:41.240488+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:49.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.892+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 msgr2=0x7f530006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.892+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 0x7f530006d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5318036000 tx=0x7f531803c040 comp rx=0 tx=0).stop 2023-10-21T17:39:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.892+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 msgr2=0x7f531c0acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.892+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f531400caa0 tx=0x7f531400cf60 comp rx=0 tx=0).stop 2023-10-21T17:39:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 shutdown_connections 2023-10-21T17:39:49.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f530006b0b0 0x7f530006d570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f531c159db0 0x7f531c0acac0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f531c1599d0 0x7f531c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 --2- 172.21.15.53:0/1075572232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f531c0cc210 0x7f531c14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 >> 172.21.15.53:0/1075572232 conn(0x7f531c0cb0c0 msgr2=0x7f531c158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 shutdown_connections 2023-10-21T17:39:49.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:49.893+0000 7f5322a4d640 1 -- 172.21.15.53:0/1075572232 wait complete. 2023-10-21T17:39:49.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:39:37.842433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:39:38.736221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:39:49.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:39:50.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.147+0000 7f11847b0640 1 Processor -- start 2023-10-21T17:39:50.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.147+0000 7f11847b0640 1 -- start start 2023-10-21T17:39:50.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.149+0000 7f11847b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f117c071260 0x7f117c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.149+0000 7f11847b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0c7ad0 con 0x7f117c15a820 2023-10-21T17:39:50.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.149+0000 7f11847b0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c160e60 con 0x7f117c0c9080 2023-10-21T17:39:50.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.149+0000 7f11847b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c0c9a40 con 0x7f117c071260 2023-10-21T17:39:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f1182d4d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f117c071260 0x7f117c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f118254c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f117c15a820 0x7f117c160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36248/0 (socket says 172.21.15.53:36248) 2023-10-21T17:39:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f118254c640 1 -- 172.21.15.53:0/955217876 learned_addr learned my addr 172.21.15.53:0/955217876 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:50.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f1182d4d640 1 -- 172.21.15.53:0/955217876 >> v1:172.21.15.124:6789/0 conn(0x7f117c0c9080 legacy=0x7f117c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f1182d4d640 1 -- 172.21.15.53:0/955217876 >> v1:172.21.15.53:6789/0 conn(0x7f117c15a820 legacy=0x7f117c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.151+0000 7f1182d4d640 1 -- 172.21.15.53:0/955217876 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0cb210 con 0x7f117c071260 2023-10-21T17:39:50.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.150+0000 7f118154a640 1 -- 172.21.15.53:0/955217876 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737673007 0 0) 0x7f117c0c7ad0 con 0x7f117c15a820 2023-10-21T17:39:50.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.152+0000 7f1182d4d640 1 --2- 172.21.15.53:0/955217876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c071260 0x7f117c0c9500 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f1174009d90 tx=0x7f117402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c196b73515546b62 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.152+0000 7f118154a640 1 -- 172.21.15.53:0/955217876 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117402ccf0 con 0x7f117c071260 2023-10-21T17:39:50.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.152+0000 7f118154a640 1 -- 172.21.15.53:0/955217876 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117402ce50 con 0x7f117c071260 2023-10-21T17:39:50.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.152+0000 7f118154a640 1 -- 172.21.15.53:0/955217876 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174034660 con 0x7f117c071260 2023-10-21T17:39:50.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 -- 172.21.15.53:0/955217876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c071260 msgr2=0x7f117c0c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 --2- 172.21.15.53:0/955217876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c071260 0x7f117c0c9500 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f1174009d90 tx=0x7f117402bf40 comp rx=0 tx=0).stop 2023-10-21T17:39:50.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 -- 172.21.15.53:0/955217876 shutdown_connections 2023-10-21T17:39:50.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 --2- 172.21.15.53:0/955217876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c071260 0x7f117c0c9500 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 -- 172.21.15.53:0/955217876 >> 172.21.15.53:0/955217876 conn(0x7f117c069d50 msgr2=0x7f117c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:50.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 -- 172.21.15.53:0/955217876 shutdown_connections 2023-10-21T17:39:50.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.153+0000 7f11847b0640 1 -- 172.21.15.53:0/955217876 wait complete. 2023-10-21T17:39:50.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.155+0000 7f11847b0640 1 Processor -- start 2023-10-21T17:39:50.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.155+0000 7f11847b0640 1 -- start start 2023-10-21T17:39:50.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f117c071260 0x7f117c14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 0x7f117c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c14fd40 con 0x7f117c15a820 2023-10-21T17:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c1550a0 con 0x7f117c071260 2023-10-21T17:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f11847b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c153480 con 0x7f117c0c9080 2023-10-21T17:39:50.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f1182d4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f1182d4d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54696/0 (socket says 172.21.15.53:54696) 2023-10-21T17:39:50.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.156+0000 7f1182d4d640 1 -- 172.21.15.53:0/966850081 learned_addr learned my addr 172.21.15.53:0/966850081 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:50.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f118254c640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f117c071260 0x7f117c14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 -- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 msgr2=0x7f117c15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1181d4b640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 0x7f117c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 0x7f117c15d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 -- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f117c071260 msgr2=0x7f117c14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f117c071260 0x7f117c14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 -- 172.21.15.53:0/966850081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174009a70 con 0x7f117c15a820 2023-10-21T17:39:50.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1181d4b640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 0x7f117c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:50.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.157+0000 7f1182d4d640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f117402dfd0 tx=0x7f1174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.158+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117403d600 con 0x7f117c15a820 2023-10-21T17:39:50.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.158+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117403d760 con 0x7f117c15a820 2023-10-21T17:39:50.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.158+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117403daf0 con 0x7f117c15a820 2023-10-21T17:39:50.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.158+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c153760 con 0x7f117c15a820 2023-10-21T17:39:50.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.158+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c15d710 con 0x7f117c15a820 2023-10-21T17:39:50.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.159+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c158b00 con 0x7f117c15a820 2023-10-21T17:39:50.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.160+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f117403b040 con 0x7f117c15a820 2023-10-21T17:39:50.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.161+0000 7f1172ffd640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 0x7f116006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.161+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f11740ab980 con 0x7f117c15a820 2023-10-21T17:39:50.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.161+0000 7f118254c640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 0x7f116006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.162+0000 7f118254c640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 0x7f116006d5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1178000f60 tx=0x7f1178007bb0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.165+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1174076500 con 0x7f117c15a820 2023-10-21T17:39:50.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.361+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f117c0ce0d0 con 0x7f117c15a820 2023-10-21T17:39:50.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.362+0000 7f1172ffd640 1 -- 172.21.15.53:0/966850081 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v254) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1174079b20 con 0x7f117c15a820 2023-10-21T17:39:50.364 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:39:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.365+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 msgr2=0x7f116006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.365+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 0x7f116006d5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1178000f60 tx=0x7f1178007bb0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.365+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 msgr2=0x7f117c0b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.365+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f117402dfd0 tx=0x7f1174004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.365+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 shutdown_connections 2023-10-21T17:39:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f116006b0e0 0x7f116006d5a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f117c15a820 0x7f117c0b3ed0 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f117c0c9080 0x7f117c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 --2- 172.21.15.53:0/966850081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f117c071260 0x7f117c14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 >> 172.21.15.53:0/966850081 conn(0x7f117c069d50 msgr2=0x7f117c0c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 shutdown_connections 2023-10-21T17:39:50.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.366+0000 7f11847b0640 1 -- 172.21.15.53:0/966850081 wait complete. 2023-10-21T17:39:50.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:39:50.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:39:50.540 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:39:50.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.619+0000 7f327ed1d640 1 Processor -- start 2023-10-21T17:39:50.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.620+0000 7f327ed1d640 1 -- start start 2023-10-21T17:39:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f327ed1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32780cd4e0 0x7f32781612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f327ed1d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32780b8c30 con 0x7f32780b4e10 2023-10-21T17:39:50.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f327ed1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32780cdd80 con 0x7f327815cd70 2023-10-21T17:39:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f327ed1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32780ce060 con 0x7f32780cd4e0 2023-10-21T17:39:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f3277fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f32780b4e10 0x7f32780d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36252/0 (socket says 172.21.15.53:36252) 2023-10-21T17:39:50.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.622+0000 7f3277fff640 1 -- 172.21.15.53:0/2931320418 learned_addr learned my addr 172.21.15.53:0/2931320418 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:50.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95709831 0 0) 0x7f32780b8c30 con 0x7f32780b4e10 2023-10-21T17:39:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3268003520 con 0x7f32780b4e10 2023-10-21T17:39:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f327d2ba640 1 --2- 172.21.15.53:0/2931320418 >> v2:172.21.15.124:3300/0 conn(0x7f32780cd4e0 0x7f32781612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368451506 0 0) 0x7f3268003520 con 0x7f32780b4e10 2023-10-21T17:39:50.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 >> v2:172.21.15.124:3300/0 conn(0x7f32780cd4e0 msgr2=0x7f32781612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 --2- 172.21.15.53:0/2931320418 >> v2:172.21.15.124:3300/0 conn(0x7f32780cd4e0 0x7f32781612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 >> v1:172.21.15.124:6789/0 conn(0x7f327815cd70 legacy=0x7f32780b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780ce340 con 0x7f32780b4e10 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3264001670 con 0x7f32780b4e10 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 >> v1:172.21.15.53:6789/0 conn(0x7f32780b4e10 legacy=0x7f32780d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268003970 0x7f3268003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32680047c0 0x7f3268004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 0x7f326800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32780ce060 con 0x7f3268005180 2023-10-21T17:39:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326800c0d0 con 0x7f3268003970 2023-10-21T17:39:50.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.623+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326800c290 con 0x7f32680047c0 2023-10-21T17:39:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 0x7f326800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327cab9640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268003970 0x7f3268003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f3277fff640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32680047c0 0x7f3268004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 -- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32680047c0 msgr2=0x7f3268004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:50.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32680047c0 0x7f3268004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 -- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268003970 msgr2=0x7f3268003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268003970 0x7f3268003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f327d2ba640 1 -- 172.21.15.53:0/2931320418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268003520 con 0x7f3268005180 2023-10-21T17:39:50.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.624+0000 7f3277fff640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32680047c0 0x7f3268004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.625+0000 7f327d2ba640 1 -- 172.21.15.53:0/2931320418 reap_dead start 2023-10-21T17:39:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.626+0000 7f327d2ba640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 0x7f326800b980 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3270007360 tx=0x7f32680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e18cf4d457369d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.626+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327002ccf0 con 0x7f3268005180 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.626+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327002ce90 con 0x7f3268005180 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.626+0000 7f32777fe640 1 -- 172.21.15.53:0/2931320418 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270003bd0 con 0x7f3268005180 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 -- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 msgr2=0x7f326800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 0x7f326800b980 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3270007360 tx=0x7f32680057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 -- 172.21.15.53:0/2931320418 shutdown_connections 2023-10-21T17:39:50.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 --2- 172.21.15.53:0/2931320418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3268005180 0x7f326800b980 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 -- 172.21.15.53:0/2931320418 >> 172.21.15.53:0/2931320418 conn(0x7f32780cf700 msgr2=0x7f32780cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 -- 172.21.15.53:0/2931320418 shutdown_connections 2023-10-21T17:39:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.627+0000 7f327ed1d640 1 -- 172.21.15.53:0/2931320418 wait complete. 2023-10-21T17:39:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.629+0000 7f327ed1d640 1 Processor -- start 2023-10-21T17:39:50.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.629+0000 7f327ed1d640 1 -- start start 2023-10-21T17:39:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 0x7f32781f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f327815fb90 0x7f327815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32781f3350 con 0x7f3278160010 2023-10-21T17:39:50.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327815f700 con 0x7f3268005180 2023-10-21T17:39:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327ed1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327815df30 con 0x7f327815fb90 2023-10-21T17:39:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327d2ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327d2ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54722/0 (socket says 172.21.15.53:54722) 2023-10-21T17:39:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.630+0000 7f327d2ba640 1 -- 172.21.15.53:0/1835941543 learned_addr learned my addr 172.21.15.53:0/1835941543 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:50.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327cab9640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 0x7f32781f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 -- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f327815fb90 msgr2=0x7f327815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f327815fb90 0x7f327815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 -- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 msgr2=0x7f32781f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 0x7f32781f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 -- 172.21.15.53:0/1835941543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327815e210 con 0x7f3278160010 2023-10-21T17:39:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327cab9640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 0x7f32781f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:39:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.631+0000 7f327d2ba640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f327002c760 tx=0x7f3270004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.632+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270004450 con 0x7f3278160010 2023-10-21T17:39:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.632+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32700045b0 con 0x7f3278160010 2023-10-21T17:39:50.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.632+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327815f0b0 con 0x7f3278160010 2023-10-21T17:39:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.632+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327003f470 con 0x7f3278160010 2023-10-21T17:39:50.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.632+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32781f2620 con 0x7f3278160010 2023-10-21T17:39:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.633+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327804e160 con 0x7f3278160010 2023-10-21T17:39:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.634+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3270034440 con 0x7f3278160010 2023-10-21T17:39:50.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.635+0000 7f32757fa640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 0x7f325c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.635+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f327003b070 con 0x7f3278160010 2023-10-21T17:39:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.636+0000 7f327cab9640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 0x7f325c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.637+0000 7f327cab9640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 0x7f325c06d570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f327815ee30 tx=0x7f326c009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:50.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.639+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3270076070 con 0x7f3278160010 2023-10-21T17:39:50.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.836+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32780d0a00 con 0x7f3278160010 2023-10-21T17:39:50.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.837+0000 7f32757fa640 1 -- 172.21.15.53:0/1835941543 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v254) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3270079690 con 0x7f3278160010 2023-10-21T17:39:50.838 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:39:50.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.839+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 msgr2=0x7f325c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.839+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 0x7f325c06d570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f327815ee30 tx=0x7f326c009040 comp rx=0 tx=0).stop 2023-10-21T17:39:50.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.839+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 msgr2=0x7f327815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:50.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f327002c760 tx=0x7f3270004ab0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 shutdown_connections 2023-10-21T17:39:50.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f325c06b0b0 0x7f325c06d570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3278160010 0x7f327815d900 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f327815fb90 0x7f327815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 --2- 172.21.15.53:0/1835941543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3268005180 0x7f32781f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:50.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 >> 172.21.15.53:0/1835941543 conn(0x7f32780cf700 msgr2=0x7f32780d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:50.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 shutdown_connections 2023-10-21T17:39:50.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:50.840+0000 7f327ed1d640 1 -- 172.21.15.53:0/1835941543 wait complete. 2023-10-21T17:39:50.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:39:51.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.085+0000 7f209b11d640 1 Processor -- start 2023-10-21T17:39:51.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.085+0000 7f209b11d640 1 -- start start 2023-10-21T17:39:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f209b11d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f209406dfc0 0x7f2094070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f209b11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f209b11d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940b1cc0 con 0x7f20940afbe0 2023-10-21T17:39:51.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f209b11d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940b2620 con 0x7f209406dfc0 2023-10-21T17:39:51.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f209b11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940b2060 con 0x7f2094071070 2023-10-21T17:39:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f2098eb9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f20940afbe0 0x7f20940b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36258/0 (socket says 172.21.15.53:36258) 2023-10-21T17:39:51.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f2093fff640 1 --2- 172.21.15.53:0/2970937542 >> v2:172.21.15.53:3300/0 conn(0x7f209406dfc0 0x7f2094070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f2098eb9640 1 -- 172.21.15.53:0/2970937542 learned_addr learned my addr 172.21.15.53:0/2970937542 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.087+0000 7f20996ba640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.088+0000 7f2093fff640 1 -- 172.21.15.53:0/2970937542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 msgr2=0x7f20940cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.088+0000 7f2093fff640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.088+0000 7f2093fff640 1 -- 172.21.15.53:0/2970937542 >> v1:172.21.15.53:6789/0 conn(0x7f20940afbe0 legacy=0x7f20940b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.088+0000 7f2093fff640 1 -- 172.21.15.53:0/2970937542 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940b2340 con 0x7f209406dfc0 2023-10-21T17:39:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.088+0000 7f20937fe640 1 -- 172.21.15.53:0/2970937542 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977863663 0 0) 0x7f20940b1cc0 con 0x7f20940afbe0 2023-10-21T17:39:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.089+0000 7f2093fff640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f2094070b30 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f20800097a0 tx=0x7f208002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccde5b770117b808 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.089+0000 7f20937fe640 1 -- 172.21.15.53:0/2970937542 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208002d040 con 0x7f209406dfc0 2023-10-21T17:39:51.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.089+0000 7f20937fe640 1 -- 172.21.15.53:0/2970937542 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080031810 con 0x7f209406dfc0 2023-10-21T17:39:51.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.089+0000 7f20937fe640 1 -- 172.21.15.53:0/2970937542 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080031ba0 con 0x7f209406dfc0 2023-10-21T17:39:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 -- 172.21.15.53:0/2970937542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 msgr2=0x7f2094070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f2094070b30 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f20800097a0 tx=0x7f208002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 -- 172.21.15.53:0/2970937542 shutdown_connections 2023-10-21T17:39:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 --2- 172.21.15.53:0/2970937542 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f2094070b30 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 -- 172.21.15.53:0/2970937542 >> 172.21.15.53:0/2970937542 conn(0x7f209406a5d0 msgr2=0x7f20940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:51.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 -- 172.21.15.53:0/2970937542 shutdown_connections 2023-10-21T17:39:51.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.090+0000 7f209b11d640 1 -- 172.21.15.53:0/2970937542 wait complete. 2023-10-21T17:39:51.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.092+0000 7f209b11d640 1 Processor -- start 2023-10-21T17:39:51.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.092+0000 7f209b11d640 1 -- start start 2023-10-21T17:39:51.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 0x7f20940c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2094145450 con 0x7f209406dfc0 2023-10-21T17:39:51.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209415c560 con 0x7f20940afbe0 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f209b11d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940cca20 con 0x7f2094071070 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f2098eb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f2098eb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54738/0 (socket says 172.21.15.53:54738) 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.093+0000 7f2098eb9640 1 -- 172.21.15.53:0/1683480958 learned_addr learned my addr 172.21.15.53:0/1683480958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f20996ba640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 0x7f20940c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2093fff640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 -- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 msgr2=0x7f20940c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 -- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 msgr2=0x7f20940c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 0x7f20940c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 -- 172.21.15.53:0/1683480958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2080009400 con 0x7f209406dfc0 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f20996ba640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 0x7f20940c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f2098eb9640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f208800c430 tx=0x7f208800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880074e0 con 0x7f209406dfc0 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2088007660 con 0x7f209406dfc0 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088004a20 con 0x7f209406dfc0 2023-10-21T17:39:51.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.094+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20940ccd00 con 0x7f209406dfc0 2023-10-21T17:39:51.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.095+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209415cae0 con 0x7f209406dfc0 2023-10-21T17:39:51.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.096+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209404e160 con 0x7f209406dfc0 2023-10-21T17:39:51.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.097+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2088004b80 con 0x7f209406dfc0 2023-10-21T17:39:51.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.097+0000 7f20917fa640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 0x7f207806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.098+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2088089a30 con 0x7f209406dfc0 2023-10-21T17:39:51.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.098+0000 7f2093fff640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 0x7f207806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.099+0000 7f2093fff640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 0x7f207806d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2080036000 tx=0x7f2080008030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.102+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20880545c0 con 0x7f209406dfc0 2023-10-21T17:39:51.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.296+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20940b0ef0 con 0x7f209406dfc0 2023-10-21T17:39:51.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.297+0000 7f20917fa640 1 -- 172.21.15.53:0/1683480958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v254) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2088057be0 con 0x7f209406dfc0 2023-10-21T17:39:51.299 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:39:51.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 msgr2=0x7f207806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 0x7f207806d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2080036000 tx=0x7f2080008030 comp rx=0 tx=0).stop 2023-10-21T17:39:51.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 msgr2=0x7f20940cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f208800c430 tx=0x7f208800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 shutdown_connections 2023-10-21T17:39:51.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f207806b0b0 0x7f207806d570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20940afbe0 0x7f20940c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2094071070 0x7f20940c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 --2- 172.21.15.53:0/1683480958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f209406dfc0 0x7f20940cee50 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 >> 172.21.15.53:0/1683480958 conn(0x7f209406a5d0 msgr2=0x7f20940ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:51.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 shutdown_connections 2023-10-21T17:39:51.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.300+0000 7f209b11d640 1 -- 172.21.15.53:0/1683480958 wait complete. 2023-10-21T17:39:51.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:39:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.539+0000 7f3b3359e640 1 Processor -- start 2023-10-21T17:39:51.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.539+0000 7f3b3359e640 1 -- start start 2023-10-21T17:39:51.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b3359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3b340c5da0 0x7f3b34165a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b3359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340c4900 con 0x7f3b340cef50 2023-10-21T17:39:51.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b3359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3415f560 con 0x7f3b3415ab70 2023-10-21T17:39:51.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b3359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b340cf4c0 con 0x7f3b340c5da0 2023-10-21T17:39:51.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b31d9b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f3b340cef50 0x7f3b3415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36260/0 (socket says 172.21.15.53:36260) 2023-10-21T17:39:51.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.540+0000 7f3b31d9b640 1 -- 172.21.15.53:0/908304596 learned_addr learned my addr 172.21.15.53:0/908304596 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:51.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b32d9d640 1 --2- 172.21.15.53:0/908304596 >> v2:172.21.15.124:3300/0 conn(0x7f3b340c5da0 0x7f3b34165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796319282 0 0) 0x7f3b340c4900 con 0x7f3b340cef50 2023-10-21T17:39:51.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b20003520 con 0x7f3b340cef50 2023-10-21T17:39:51.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014594622 0 0) 0x7f3b20003520 con 0x7f3b340cef50 2023-10-21T17:39:51.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 >> v2:172.21.15.124:3300/0 conn(0x7f3b340c5da0 msgr2=0x7f3b34165a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:51.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 --2- 172.21.15.53:0/908304596 >> v2:172.21.15.124:3300/0 conn(0x7f3b340c5da0 0x7f3b34165a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 >> v1:172.21.15.124:6789/0 conn(0x7f3b3415ab70 legacy=0x7f3b3415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:51.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340cf7a0 con 0x7f3b340cef50 2023-10-21T17:39:51.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f3b1c001670 con 0x7f3b340cef50 2023-10-21T17:39:51.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 >> v1:172.21.15.53:6789/0 conn(0x7f3b340cef50 legacy=0x7f3b3415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 0x7f3b20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.541+0000 7f3b3159a640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200047c0 0x7f3b20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3159a640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b20005180 0x7f3b2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340cf4c0 con 0x7f3b200039c0 2023-10-21T17:39:51.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2000c080 con 0x7f3b200047c0 2023-10-21T17:39:51.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2000c240 con 0x7f3b20005180 2023-10-21T17:39:51.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 0x7f3b20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b31d9b640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200047c0 0x7f3b20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 -- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b20005180 msgr2=0x7f3b2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:51.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b20005180 0x7f3b2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 -- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200047c0 msgr2=0x7f3b20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:51.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200047c0 0x7f3b20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b3259c640 1 -- 172.21.15.53:0/908304596 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20003520 con 0x7f3b200039c0 2023-10-21T17:39:51.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.542+0000 7f3b32d9d640 1 -- 172.21.15.53:0/908304596 reap_dead start 2023-10-21T17:39:51.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.543+0000 7f3b3259c640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 0x7f3b20003de0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001e50 tx=0x7f3b200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74329cb57cd3ec66 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24003b70 con 0x7f3b200039c0 2023-10-21T17:39:51.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b24003cd0 con 0x7f3b200039c0 2023-10-21T17:39:51.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3159a640 1 -- 172.21.15.53:0/908304596 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24005310 con 0x7f3b200039c0 2023-10-21T17:39:51.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 -- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 msgr2=0x7f3b20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 0x7f3b20003de0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f3b24001e50 tx=0x7f3b200057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 -- 172.21.15.53:0/908304596 shutdown_connections 2023-10-21T17:39:51.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 --2- 172.21.15.53:0/908304596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b200039c0 0x7f3b20003de0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 -- 172.21.15.53:0/908304596 >> 172.21.15.53:0/908304596 conn(0x7f3b340d07f0 msgr2=0x7f3b340d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:51.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 -- 172.21.15.53:0/908304596 shutdown_connections 2023-10-21T17:39:51.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.544+0000 7f3b3359e640 1 -- 172.21.15.53:0/908304596 wait complete. 2023-10-21T17:39:51.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.547+0000 7f3b3359e640 1 Processor -- start 2023-10-21T17:39:51.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.547+0000 7f3b3359e640 1 -- start start 2023-10-21T17:39:51.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200039c0 0x7f3b34155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 0x7f3b34157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34159b50 con 0x7f3b34158b70 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3406b870 con 0x7f3b200039c0 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b3359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34156c20 con 0x7f3b34156720 2023-10-21T17:39:51.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b32d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b32d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54764/0 (socket says 172.21.15.53:54764) 2023-10-21T17:39:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.548+0000 7f3b32d9d640 1 -- 172.21.15.53:0/3388148639 learned_addr learned my addr 172.21.15.53:0/3388148639 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b3259c640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200039c0 0x7f3b34155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b31d9b640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 0x7f3b34157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 -- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 msgr2=0x7f3b34157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 0x7f3b34157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 -- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200039c0 msgr2=0x7f3b34155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200039c0 0x7f3b34155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 -- 172.21.15.53:0/3388148639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3406bac0 con 0x7f3b34158b70 2023-10-21T17:39:51.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b31d9b640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 0x7f3b34157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:51.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b32d9d640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f3b3409e690 tx=0x7f3b2c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00c620 con 0x7f3b34158b70 2023-10-21T17:39:51.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c00c780 con 0x7f3b34158b70 2023-10-21T17:39:51.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3406bda0 con 0x7f3b34158b70 2023-10-21T17:39:51.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.549+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00cb30 con 0x7f3b34158b70 2023-10-21T17:39:51.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.550+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b34159560 con 0x7f3b34158b70 2023-10-21T17:39:51.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.551+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3406c2c0 con 0x7f3b34158b70 2023-10-21T17:39:51.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.552+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3b2c00cc90 con 0x7f3b34158b70 2023-10-21T17:39:51.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.553+0000 7f3b2affd640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 0x7f3b1406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:51.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.553+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3b2c0898f0 con 0x7f3b34158b70 2023-10-21T17:39:51.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.553+0000 7f3b3259c640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 0x7f3b1406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:51.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.555+0000 7f3b3259c640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 0x7f3b1406d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034000 tx=0x7f3b24009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:51.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.557+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b3406c2c0 con 0x7f3b34158b70 2023-10-21T17:39:51.754 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.753+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3b3406c2c0 con 0x7f3b34158b70 2023-10-21T17:39:51.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.754+0000 7f3b2affd640 1 -- 172.21.15.53:0/3388148639 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v254) v1 ==== 84+0+209 (secure 0 0 0) 0x7f3b3406c2c0 con 0x7f3b34158b70 2023-10-21T17:39:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:51.755 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:39:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:39:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:39:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 31, 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "cephfs_metadata" 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 32, 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": "cephfs_data" 2023-10-21T17:39:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:39:51.758 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:39:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.756+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 msgr2=0x7f3b1406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.756+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 0x7f3b1406d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b24034000 tx=0x7f3b24009030 comp rx=0 tx=0).stop 2023-10-21T17:39:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.756+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 msgr2=0x7f3b34159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:51.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.756+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f3b3409e690 tx=0x7f3b2c00ec00 comp rx=0 tx=0).stop 2023-10-21T17:39:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 shutdown_connections 2023-10-21T17:39:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3b1406b0b0 0x7f3b1406d570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b34158b70 0x7f3b34159020 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b34156720 0x7f3b34157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 --2- 172.21.15.53:0/3388148639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b200039c0 0x7f3b34155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 >> 172.21.15.53:0/3388148639 conn(0x7f3b340d07f0 msgr2=0x7f3b3406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 shutdown_connections 2023-10-21T17:39:51.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:51.757+0000 7f3b3359e640 1 -- 172.21.15.53:0/3388148639 wait complete. 2023-10-21T17:39:51.775 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-21T17:39:51.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-21T17:39:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.005+0000 7f87bca0d640 1 Processor -- start 2023-10-21T17:39:52.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.005+0000 7f87bca0d640 1 -- start start 2023-10-21T17:39:52.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.006+0000 7f87bca0d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 0x7f87b80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.006+0000 7f87bca0d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8159db0 con 0x7f87b80b3c50 2023-10-21T17:39:52.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.006+0000 7f87bca0d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8071fe0 con 0x7f87b80cb5e0 2023-10-21T17:39:52.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.006+0000 7f87bca0d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b44c0 con 0x7f87b81599d0 2023-10-21T17:39:52.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.007+0000 7f87ae59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f87b80b3c50 0x7f87b80caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36266/0 (socket says 172.21.15.53:36266) 2023-10-21T17:39:52.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.007+0000 7f87ae59c640 1 -- 172.21.15.53:0/4086363365 learned_addr learned my addr 172.21.15.53:0/4086363365 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.007+0000 7f87b6d9d640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 0x7f87b80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.007+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711285506 0 0) 0x7f87a0001110 con 0x7f87b80b3c50 2023-10-21T17:39:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.007+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a4005730 con 0x7f87b80b3c50 2023-10-21T17:39:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1372663113 0 0) 0x7f87a4005730 con 0x7f87b80b3c50 2023-10-21T17:39:52.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 msgr2=0x7f87b80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 0x7f87b80cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 >> v1:172.21.15.124:6789/0 conn(0x7f87b80cb5e0 legacy=0x7f87b80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80713d0 con 0x7f87b80b3c50 2023-10-21T17:39:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f87a0001800 con 0x7f87b80b3c50 2023-10-21T17:39:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 >> v1:172.21.15.53:6789/0 conn(0x7f87b80b3c50 legacy=0x7f87b80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a4005bb0 0x7f87a4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a40069d0 0x7f87a4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 0x7f87a4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80b44c0 con 0x7f87a40069d0 2023-10-21T17:39:52.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a400dc70 con 0x7f87a4005bb0 2023-10-21T17:39:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a400de30 con 0x7f87b81599d0 2023-10-21T17:39:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.008+0000 7f87ae59c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a40069d0 0x7f87a4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87ae59c640 1 -- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 msgr2=0x7f87a4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87ae59c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87b81599d0 0x7f87a4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87b6d9d640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a4005bb0 0x7f87a4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87ae59c640 1 -- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a4005bb0 msgr2=0x7f87a4005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87ae59c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87a4005bb0 0x7f87a4005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.009+0000 7f87ae59c640 1 -- 172.21.15.53:0/4086363365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4005730 con 0x7f87a40069d0 2023-10-21T17:39:52.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.010+0000 7f87ae59c640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a40069d0 0x7f87a4006e50 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f87a0001eb0 tx=0x7f87a4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7013647c7a7a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.010+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0005ad0 con 0x7f87a40069d0 2023-10-21T17:39:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.010+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0005c50 con 0x7f87a40069d0 2023-10-21T17:39:52.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.010+0000 7f87b659c640 1 -- 172.21.15.53:0/4086363365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a002c790 con 0x7f87a40069d0 2023-10-21T17:39:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 -- 172.21.15.53:0/4086363365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a40069d0 msgr2=0x7f87a4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 --2- 172.21.15.53:0/4086363365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87a40069d0 0x7f87a4006e50 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f87a0001eb0 tx=0x7f87a4007a00 comp rx=0 tx=0).stop 2023-10-21T17:39:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87b759e640 1 -- 172.21.15.53:0/4086363365 reap_dead start 2023-10-21T17:39:52.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 -- 172.21.15.53:0/4086363365 shutdown_connections 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 -- 172.21.15.53:0/4086363365 >> 172.21.15.53:0/4086363365 conn(0x7f87b806c050 msgr2=0x7f87b806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 -- 172.21.15.53:0/4086363365 shutdown_connections 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.011+0000 7f87bca0d640 1 -- 172.21.15.53:0/4086363365 wait complete. 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.013+0000 7f87bca0d640 1 Processor -- start 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.013+0000 7f87bca0d640 1 -- start start 2023-10-21T17:39:52.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a40069d0 0x7f87b815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87b814e970 0x7f87b8153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8154680 con 0x7f87b8154200 2023-10-21T17:39:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8153690 con 0x7f87b814e970 2023-10-21T17:39:52.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87bca0d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8152b10 con 0x7f87a40069d0 2023-10-21T17:39:52.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87b759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87b759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54784/0 (socket says 172.21.15.53:54784) 2023-10-21T17:39:52.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.014+0000 7f87b759e640 1 -- 172.21.15.53:0/2237328239 learned_addr learned my addr 172.21.15.53:0/2237328239 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:52.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b6d9d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a40069d0 0x7f87b815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87ae59c640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87b814e970 0x7f87b8153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 -- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a40069d0 msgr2=0x7f87b815b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a40069d0 0x7f87b815b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 -- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87b814e970 msgr2=0x7f87b8153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87b814e970 0x7f87b8153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 -- 172.21.15.53:0/2237328239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8152df0 con 0x7f87b8154200 2023-10-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87b759e640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cbf80 tx=0x7f87b000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b00074e0 con 0x7f87b8154200 2023-10-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b0007640 con 0x7f87b8154200 2023-10-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.015+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004260 con 0x7f87b8154200 2023-10-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.016+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b81530d0 con 0x7f87b8154200 2023-10-21T17:39:52.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.016+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b815b1c0 con 0x7f87b8154200 2023-10-21T17:39:52.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.017+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8074150 con 0x7f87b8154200 2023-10-21T17:39:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.018+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f87b00057c0 con 0x7f87b8154200 2023-10-21T17:39:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.019+0000 7f87affff640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 0x7f879806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.019+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f87b0088d70 con 0x7f87b8154200 2023-10-21T17:39:52.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.020+0000 7f87b6d9d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 0x7f879806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.021+0000 7f87b6d9d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 0x7f879806d570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87a8007520 tx=0x7f87a800d040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.024+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87b0053860 con 0x7f87b8154200 2023-10-21T17:39:52.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.216+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87b80cde70 con 0x7f87b8154200 2023-10-21T17:39:52.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.451+0000 7f87affff640 1 -- 172.21.15.53:0/2237328239 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v255) v1 ==== 144+0+0 (secure 0 0 0) 0x7f87b000cea0 con 0x7f87b8154200 2023-10-21T17:39:52.453 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_metadata' created 2023-10-21T17:39:52.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 msgr2=0x7f879806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 0x7f879806d570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87a8007520 tx=0x7f87a800d040 comp rx=0 tx=0).stop 2023-10-21T17:39:52.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 msgr2=0x7f87b814f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f87b80cbf80 tx=0x7f87b000c820 comp rx=0 tx=0).stop 2023-10-21T17:39:52.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 shutdown_connections 2023-10-21T17:39:52.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f879806b0b0 0x7f879806d570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87b8154200 0x7f87b814f1d0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87b814e970 0x7f87b8153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 --2- 172.21.15.53:0/2237328239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87a40069d0 0x7f87b815b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.456+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 >> 172.21.15.53:0/2237328239 conn(0x7f87b806c050 msgr2=0x7f87b80cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:52.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.457+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 shutdown_connections 2023-10-21T17:39:52.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.458+0000 7f87bca0d640 1 -- 172.21.15.53:0/2237328239 wait complete. 2023-10-21T17:39:52.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:39:52.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.696+0000 7f4f5d813640 1 Processor -- start 2023-10-21T17:39:52.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.696+0000 7f4f5d813640 1 -- start start 2023-10-21T17:39:52.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5d813640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f58069d50 0x7f4f5806a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5d813640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580cc580 con 0x7f4f5815d4d0 2023-10-21T17:39:52.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5d813640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580cdcf0 con 0x7f4f58161bf0 2023-10-21T17:39:52.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5d813640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580c8d50 con 0x7f4f58069d50 2023-10-21T17:39:52.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5c811640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4f5815d4d0 0x7f4f580cb260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36274/0 (socket says 172.21.15.53:36274) 2023-10-21T17:39:52.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.698+0000 7f4f5c811640 1 -- 172.21.15.53:0/2427403974 learned_addr learned my addr 172.21.15.53:0/2427403974 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4ffff640 1 --2- 172.21.15.53:0/2427403974 >> v2:172.21.15.53:3300/0 conn(0x7f4f58069d50 0x7f4f5806a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4f7fe640 1 -- 172.21.15.53:0/2427403974 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942982533 0 0) 0x7f4f580cc580 con 0x7f4f5815d4d0 2023-10-21T17:39:52.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4f7fe640 1 -- 172.21.15.53:0/2427403974 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f40003520 con 0x7f4f5815d4d0 2023-10-21T17:39:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2427403974 >> v1:172.21.15.124:6789/0 conn(0x7f4f58161bf0 legacy=0x7f4f58163fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2427403974 >> v1:172.21.15.53:6789/0 conn(0x7f4f5815d4d0 legacy=0x7f4f580cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.699+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2427403974 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580cdfd0 con 0x7f4f58069d50 2023-10-21T17:39:52.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.701+0000 7f4f4ffff640 1 --2- 172.21.15.53:0/2427403974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f58069d50 0x7f4f5806a1c0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097d0 tx=0x7f4f4802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250c5c1388b14170 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.701+0000 7f4f4f7fe640 1 -- 172.21.15.53:0/2427403974 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4802d040 con 0x7f4f58069d50 2023-10-21T17:39:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.701+0000 7f4f4f7fe640 1 -- 172.21.15.53:0/2427403974 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48031810 con 0x7f4f58069d50 2023-10-21T17:39:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.701+0000 7f4f4f7fe640 1 -- 172.21.15.53:0/2427403974 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48031ba0 con 0x7f4f58069d50 2023-10-21T17:39:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 -- 172.21.15.53:0/2427403974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f58069d50 msgr2=0x7f4f5806a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2427403974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f58069d50 0x7f4f5806a1c0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f4f480097d0 tx=0x7f4f4802b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 -- 172.21.15.53:0/2427403974 shutdown_connections 2023-10-21T17:39:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2427403974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f58069d50 0x7f4f5806a1c0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 -- 172.21.15.53:0/2427403974 >> 172.21.15.53:0/2427403974 conn(0x7f4f580ce9f0 msgr2=0x7f4f580cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 -- 172.21.15.53:0/2427403974 shutdown_connections 2023-10-21T17:39:52.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.702+0000 7f4f5d813640 1 -- 172.21.15.53:0/2427403974 wait complete. 2023-10-21T17:39:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.704+0000 7f4f5d813640 1 Processor -- start 2023-10-21T17:39:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.704+0000 7f4f5d813640 1 -- start start 2023-10-21T17:39:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 0x7f4f580c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 0x7f4f581584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5815aeb0 con 0x7f4f5815d4d0 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f58159f90 con 0x7f4f58069d50 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f5d813640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5815a230 con 0x7f4f58161bf0 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f4ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54810/0 (socket says 172.21.15.53:54810) 2023-10-21T17:39:52.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.705+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2825488992 learned_addr learned my addr 172.21.15.53:0/2825488992 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f5d012640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 0x7f4f581584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f5c811640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 0x7f4f580c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 msgr2=0x7f4f581584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 0x7f4f581584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 msgr2=0x7f4f580c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 0x7f4f580c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 -- 172.21.15.53:0/2825488992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48009400 con 0x7f4f5815d4d0 2023-10-21T17:39:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f5d012640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 0x7f4f581584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f5c811640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 0x7f4f580c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:52.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4ffff640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48006e10 con 0x7f4f5815d4d0 2023-10-21T17:39:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4803ad60 con 0x7f4f5815d4d0 2023-10-21T17:39:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48044660 con 0x7f4f5815d4d0 2023-10-21T17:39:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.706+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5815a510 con 0x7f4f5815d4d0 2023-10-21T17:39:52.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.707+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5815aad0 con 0x7f4f5815d4d0 2023-10-21T17:39:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.708+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f580c2100 con 0x7f4f5815d4d0 2023-10-21T17:39:52.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.709+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f4f4804d440 con 0x7f4f5815d4d0 2023-10-21T17:39:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.710+0000 7f4f4d7fa640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 0x7f4f4406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.710+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f4f480ac140 con 0x7f4f5815d4d0 2023-10-21T17:39:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.711+0000 7f4f5c811640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 0x7f4f4406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:52.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.712+0000 7f4f5c811640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 0x7f4f4406d570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f54001560 tx=0x7f4f54009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:52.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.716+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f48076a40 con 0x7f4f5815d4d0 2023-10-21T17:39:52.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:52.909+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4f5804e160 con 0x7f4f5815d4d0 2023-10-21T17:39:53.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.466+0000 7f4f4d7fa640 1 -- 172.21.15.53:0/2825488992 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v256) v1 ==== 176+0+0 (secure 0 0 0) 0x7f4f4807a060 con 0x7f4f5815d4d0 2023-10-21T17:39:53.468 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_data' created 2023-10-21T17:39:53.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 msgr2=0x7f4f4406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 0x7f4f4406d570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4f54001560 tx=0x7f4f54009040 comp rx=0 tx=0).stop 2023-10-21T17:39:53.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 msgr2=0x7f4f58157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f480046c0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 shutdown_connections 2023-10-21T17:39:53.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f4f4406b0b0 0x7f4f4406d570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f58161bf0 0x7f4f581584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f5815d4d0 0x7f4f58157f60 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 --2- 172.21.15.53:0/2825488992 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f58069d50 0x7f4f580c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 >> 172.21.15.53:0/2825488992 conn(0x7f4f580ce9f0 msgr2=0x7f4f58071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:53.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.470+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 shutdown_connections 2023-10-21T17:39:53.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.471+0000 7f4f5d813640 1 -- 172.21.15.53:0/2825488992 wait complete. 2023-10-21T17:39:53.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-21T17:39:53.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.699+0000 7f5fa711d640 1 Processor -- start 2023-10-21T17:39:53.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.699+0000 7f5fa711d640 1 -- start start 2023-10-21T17:39:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.700+0000 7f5fa711d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fa015cd80 0x7f5fa00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa711d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa711d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa015d160 con 0x7f5fa00b6730 2023-10-21T17:39:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa711d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa00b7b90 con 0x7f5fa015cd80 2023-10-21T17:39:53.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa711d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa00b8020 con 0x7f5fa00cdf70 2023-10-21T17:39:53.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa4eb9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fa015cd80 0x7f5fa00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa4eb9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5fa015cd80 0x7f5fa00b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:39906/0 (socket says 172.21.15.53:39906) 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.701+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3489290791 learned_addr learned my addr 172.21.15.53:0/3489290791 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.702+0000 7f5fa56ba640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.702+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 msgr2=0x7f5fa00d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.702+0000 7f5fa4eb9640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.702+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3489290791 >> v1:172.21.15.124:6789/0 conn(0x7f5fa00b6730 legacy=0x7f5fa00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.702+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3489290791 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa00b6f80 con 0x7f5fa015cd80 2023-10-21T17:39:53.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.704+0000 7f5fa4eb9640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00b87d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009e50 tx=0x7f5f9c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd169959d90b6fc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:53.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.704+0000 7f5f977fe640 1 -- 172.21.15.53:0/3489290791 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c02cb00 con 0x7f5fa015cd80 2023-10-21T17:39:53.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.704+0000 7f5f977fe640 1 -- 172.21.15.53:0/3489290791 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c02cc60 con 0x7f5fa015cd80 2023-10-21T17:39:53.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.704+0000 7f5f977fe640 1 -- 172.21.15.53:0/3489290791 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0343f0 con 0x7f5fa015cd80 2023-10-21T17:39:53.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 -- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 msgr2=0x7f5fa00b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00b87d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009e50 tx=0x7f5f9c02bf60 comp rx=0 tx=0).stop 2023-10-21T17:39:53.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 -- 172.21.15.53:0/3489290791 shutdown_connections 2023-10-21T17:39:53.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3489290791 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00b87d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 -- 172.21.15.53:0/3489290791 >> 172.21.15.53:0/3489290791 conn(0x7f5fa006b5d0 msgr2=0x7f5fa006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:53.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 -- 172.21.15.53:0/3489290791 shutdown_connections 2023-10-21T17:39:53.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.705+0000 7f5fa711d640 1 -- 172.21.15.53:0/3489290791 wait complete. 2023-10-21T17:39:53.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.707+0000 7f5fa711d640 1 Processor -- start 2023-10-21T17:39:53.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.707+0000 7f5fa711d640 1 -- start start 2023-10-21T17:39:53.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00d2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa01416e0 con 0x7f5fa00b6730 2023-10-21T17:39:53.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa01f5c20 con 0x7f5fa015cd80 2023-10-21T17:39:53.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa711d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa01f5f00 con 0x7f5fa00cdf70 2023-10-21T17:39:53.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.708+0000 7f5fa4eb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:53.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54816/0 (socket says 172.21.15.53:54816) 2023-10-21T17:39:53.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3982522829 learned_addr learned my addr 172.21.15.53:0/3982522829 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:53.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 msgr2=0x7f5fa00d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 msgr2=0x7f5fa00d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:53.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:53.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 -- 172.21.15.53:0/3982522829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c009b30 con 0x7f5fa00b6730 2023-10-21T17:39:53.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.709+0000 7f5fa4eb9640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000f80 tx=0x7f5f9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:53.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.710+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c03e450 con 0x7f5fa00b6730 2023-10-21T17:39:53.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.710+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c03e5f0 con 0x7f5fa00b6730 2023-10-21T17:39:53.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.710+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa01f61e0 con 0x7f5fa00b6730 2023-10-21T17:39:53.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.710+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c03e9a0 con 0x7f5fa00b6730 2023-10-21T17:39:53.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.710+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa01f67a0 con 0x7f5fa00b6730 2023-10-21T17:39:53.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.711+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa004e160 con 0x7f5fa00b6730 2023-10-21T17:39:53.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.712+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5f9c03eb00 con 0x7f5fa00b6730 2023-10-21T17:39:53.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.713+0000 7f5f957fa640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 0x7f5f8c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:53.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.713+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f5f9c0acbc0 con 0x7f5fa00b6730 2023-10-21T17:39:53.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.714+0000 7f5f97fff640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 0x7f5f8c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:53.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.715+0000 7f5f97fff640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 0x7f5f8c06d570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000f60 tx=0x7f5f900079d0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:53.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.718+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f9c077520 con 0x7f5fa00b6730 2023-10-21T17:39:53.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:53.946+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f5fa00b5f00 con 0x7f5fa00b6730 2023-10-21T17:39:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.490+0000 7f5f957fa640 1 -- 172.21.15.53:0/3982522829 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 33 and data pool 34 v130) v1 ==== 183+0+0 (secure 0 0 0) 0x7f5f9c03edb0 con 0x7f5fa00b6730 2023-10-21T17:39:54.492 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 33 and data pool 34 2023-10-21T17:39:54.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.498+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 msgr2=0x7f5f8c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.498+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 0x7f5f8c06d570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f90000f60 tx=0x7f5f900079d0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.498+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 msgr2=0x7f5fa01411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.498+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000f80 tx=0x7f5f9c004a40 comp rx=0 tx=0).stop 2023-10-21T17:39:54.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.498+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 shutdown_connections 2023-10-21T17:39:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f5f8c06b0b0 0x7f5f8c06d570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fa015cd80 0x7f5fa00d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fa00cdf70 0x7f5fa00d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.502 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 --2- 172.21.15.53:0/3982522829 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fa00b6730 0x7f5fa01411a0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 >> 172.21.15.53:0/3982522829 conn(0x7f5fa006b5d0 msgr2=0x7f5fa0159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 shutdown_connections 2023-10-21T17:39:54.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.499+0000 7f5fa711d640 1 -- 172.21.15.53:0/3982522829 wait complete. 2023-10-21T17:39:54.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:54.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.719+0000 7fc415bf0640 1 Processor -- start 2023-10-21T17:39:54.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.719+0000 7fc415bf0640 1 -- start start 2023-10-21T17:39:54.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc415bf0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4100ad790 0x7fc4100702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc415bf0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc410070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc415bf0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc41006dee0 con 0x7fc410159ac0 2023-10-21T17:39:54.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc415bf0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41006dcf0 con 0x7fc4100ad790 2023-10-21T17:39:54.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc415bf0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41015d850 con 0x7fc41015e330 2023-10-21T17:39:54.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc40effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4100ad790 0x7fc4100702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:54.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc40f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc410159ac0 0x7fc41006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57690/0 (socket says 172.21.15.53:57690) 2023-10-21T17:39:54.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.720+0000 7fc40f7fe640 1 -- 172.21.15.53:0/1556916337 learned_addr learned my addr 172.21.15.53:0/1556916337 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:54.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40ffff640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc410070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:54.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40effd640 1 -- 172.21.15.53:0/1556916337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 msgr2=0x7fc410070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40effd640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc410070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40effd640 1 -- 172.21.15.53:0/1556916337 >> v1:172.21.15.53:6789/0 conn(0x7fc410159ac0 legacy=0x7fc41006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40effd640 1 -- 172.21.15.53:0/1556916337 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41015db30 con 0x7fc4100ad790 2023-10-21T17:39:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.721+0000 7fc40e7fc640 1 -- 172.21.15.53:0/1556916337 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732652844 0 0) 0x7fc41006dee0 con 0x7fc410159ac0 2023-10-21T17:39:54.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.722+0000 7fc40effd640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100702d0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009d90 tx=0x7fc3f802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7655561be2bf10 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.722+0000 7fc40e7fc640 1 -- 172.21.15.53:0/1556916337 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f802cc50 con 0x7fc4100ad790 2023-10-21T17:39:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.722+0000 7fc40e7fc640 1 -- 172.21.15.53:0/1556916337 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f802cdb0 con 0x7fc4100ad790 2023-10-21T17:39:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.722+0000 7fc40e7fc640 1 -- 172.21.15.53:0/1556916337 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8034660 con 0x7fc4100ad790 2023-10-21T17:39:54.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 -- 172.21.15.53:0/1556916337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 msgr2=0x7fc4100702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100702d0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009d90 tx=0x7fc3f802bea0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 -- 172.21.15.53:0/1556916337 shutdown_connections 2023-10-21T17:39:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc410070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 --2- 172.21.15.53:0/1556916337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100702d0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 -- 172.21.15.53:0/1556916337 >> 172.21.15.53:0/1556916337 conn(0x7fc41006a380 msgr2=0x7fc4100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 -- 172.21.15.53:0/1556916337 shutdown_connections 2023-10-21T17:39:54.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.723+0000 7fc415bf0640 1 -- 172.21.15.53:0/1556916337 wait complete. 2023-10-21T17:39:54.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.725+0000 7fc415bf0640 1 Processor -- start 2023-10-21T17:39:54.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.725+0000 7fc415bf0640 1 -- start start 2023-10-21T17:39:54.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc410159ac0 0x7fc4100ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc4100cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100aca40 con 0x7fc4100ad790 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101f5f30 con 0x7fc410159ac0 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc415bf0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101f6210 con 0x7fc41015e330 2023-10-21T17:39:54.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc40f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53164/0 (socket says 172.21.15.53:53164) 2023-10-21T17:39:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.726+0000 7fc40f7fe640 1 -- 172.21.15.53:0/2763000868 learned_addr learned my addr 172.21.15.53:0/2763000868 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40effd640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc410159ac0 0x7fc4100ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 -- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 msgr2=0x7fc4100cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc4100cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 -- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc410159ac0 msgr2=0x7fc4100ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc410159ac0 0x7fc4100ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 -- 172.21.15.53:0/2763000868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009a70 con 0x7fc4100ad790 2023-10-21T17:39:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc40f7fe640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c4d0 tx=0x7fc40000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:54.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000074e0 con 0x7fc4100ad790 2023-10-21T17:39:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400007660 con 0x7fc4100ad790 2023-10-21T17:39:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4101f64f0 con 0x7fc4100ad790 2023-10-21T17:39:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.727+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400004a20 con 0x7fc4100ad790 2023-10-21T17:39:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.728+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4101f6ab0 con 0x7fc4100ad790 2023-10-21T17:39:54.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.729+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410067490 con 0x7fc4100ad790 2023-10-21T17:39:54.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.730+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc400004b80 con 0x7fc4100ad790 2023-10-21T17:39:54.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.730+0000 7fc407fff640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 0x7fc3f006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.731+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc4000894d0 con 0x7fc4100ad790 2023-10-21T17:39:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.731+0000 7fc40effd640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 0x7fc3f006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:54.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.732+0000 7fc40effd640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 0x7fc3f006d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8036000 tx=0x7fc3f8030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:54.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.735+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc400053d00 con 0x7fc4100ad790 2023-10-21T17:39:54.929 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.928+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4100b0ff0 con 0x7fc4100ad790 2023-10-21T17:39:54.933 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.932+0000 7fc407fff640 1 -- 172.21.15.53:0/2763000868 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+20631 (secure 0 0 0) 0x7fc400057320 con 0x7fc4100ad790 2023-10-21T17:39:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:54.934 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":257,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:53.948288+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:54.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.934+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 msgr2=0x7fc3f006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 0x7fc3f006d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8036000 tx=0x7fc3f8030000 comp rx=0 tx=0).stop 2023-10-21T17:39:54.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 msgr2=0x7fc4100ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c4d0 tx=0x7fc40000c990 comp rx=0 tx=0).stop 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 shutdown_connections 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc3f006b0b0 0x7fc3f006d570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc41015e330 0x7fc4100cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc410159ac0 0x7fc4100ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 --2- 172.21.15.53:0/2763000868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4100ad790 0x7fc4100ab8c0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 >> 172.21.15.53:0/2763000868 conn(0x7fc41006a380 msgr2=0x7fc41006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 shutdown_connections 2023-10-21T17:39:54.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:54.935+0000 7fc415bf0640 1 -- 172.21.15.53:0/2763000868 wait complete. 2023-10-21T17:39:54.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:55.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.182+0000 7f2085aca640 1 Processor -- start 2023-10-21T17:39:55.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.183+0000 7f2085aca640 1 -- start start 2023-10-21T17:39:55.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.184+0000 7f2085aca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20800d16f0 0x7f20800cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.184+0000 7f2085aca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.184+0000 7f2085aca640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20800cea50 con 0x7f208015cc60 2023-10-21T17:39:55.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.184+0000 7f2085aca640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208015d040 con 0x7f20800d16f0 2023-10-21T17:39:55.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.184+0000 7f2085aca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20800d0650 con 0x7f20800cfa50 2023-10-21T17:39:55.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20800d16f0 0x7f20800cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20800d16f0 0x7f20800cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53176/0 (socket says 172.21.15.53:53176) 2023-10-21T17:39:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 -- 172.21.15.53:0/2527245679 learned_addr learned my addr 172.21.15.53:0/2527245679 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:55.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f20852c9640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 -- 172.21.15.53:0/2527245679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 msgr2=0x7f20800ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 -- 172.21.15.53:0/2527245679 >> v1:172.21.15.124:6789/0 conn(0x7f208015cc60 legacy=0x7f20800cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.185+0000 7f207ffff640 1 -- 172.21.15.53:0/2527245679 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800d0930 con 0x7f20800d16f0 2023-10-21T17:39:55.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.187+0000 7f207ffff640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20800d16f0 0x7f20800cdfd0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f206c009870 tx=0x7f206c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3bf67b18794079 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.187+0000 7f207f7fe640 1 -- 172.21.15.53:0/2527245679 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c030840 con 0x7f20800d16f0 2023-10-21T17:39:55.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.187+0000 7f207f7fe640 1 -- 172.21.15.53:0/2527245679 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c0309a0 con 0x7f20800d16f0 2023-10-21T17:39:55.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.187+0000 7f207f7fe640 1 -- 172.21.15.53:0/2527245679 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c030d30 con 0x7f20800d16f0 2023-10-21T17:39:55.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 -- 172.21.15.53:0/2527245679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20800d16f0 msgr2=0x7f20800cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20800d16f0 0x7f20800cdfd0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f206c009870 tx=0x7f206c02b980 comp rx=0 tx=0).stop 2023-10-21T17:39:55.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 -- 172.21.15.53:0/2527245679 shutdown_connections 2023-10-21T17:39:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 --2- 172.21.15.53:0/2527245679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20800d16f0 0x7f20800cdfd0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 -- 172.21.15.53:0/2527245679 >> 172.21.15.53:0/2527245679 conn(0x7f208009bcf0 msgr2=0x7f20800d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 -- 172.21.15.53:0/2527245679 shutdown_connections 2023-10-21T17:39:55.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.188+0000 7f2085aca640 1 -- 172.21.15.53:0/2527245679 wait complete. 2023-10-21T17:39:55.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.190+0000 7f2085aca640 1 Processor -- start 2023-10-21T17:39:55.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.190+0000 7f2085aca640 1 -- start start 2023-10-21T17:39:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20800d16f0 0x7f20801557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801f20f0 con 0x7f208015cc60 2023-10-21T17:39:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20800b5a20 con 0x7f20800d16f0 2023-10-21T17:39:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2085aca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20800b5d00 con 0x7f20800cfa50 2023-10-21T17:39:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f20852c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f20852c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53180/0 (socket says 172.21.15.53:53180) 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f20852c9640 1 -- 172.21.15.53:0/4226947922 learned_addr learned my addr 172.21.15.53:0/4226947922 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f2084ac8640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.191+0000 7f207ffff640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20800d16f0 0x7f20801557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 -- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 msgr2=0x7f20800b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 -- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20800d16f0 msgr2=0x7f20801557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20800d16f0 0x7f20801557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 -- 172.21.15.53:0/4226947922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0094d0 con 0x7f208015cc60 2023-10-21T17:39:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f20852c9640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f2078007cf0 tx=0x7f207800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078014420 con 0x7f208015cc60 2023-10-21T17:39:55.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20780145a0 con 0x7f208015cc60 2023-10-21T17:39:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078014950 con 0x7f208015cc60 2023-10-21T17:39:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20800b5fe0 con 0x7f208015cc60 2023-10-21T17:39:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.192+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20800b65a0 con 0x7f208015cc60 2023-10-21T17:39:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.194+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208004e160 con 0x7f208015cc60 2023-10-21T17:39:55.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.194+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2078014ab0 con 0x7f208015cc60 2023-10-21T17:39:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.195+0000 7f207d7fa640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 0x7f206406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.195+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f207808a750 con 0x7f208015cc60 2023-10-21T17:39:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.196+0000 7f2084ac8640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 0x7f206406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.197+0000 7f2084ac8640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 0x7f206406d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2074000fb0 tx=0x7f2074009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.200+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2078055040 con 0x7f208015cc60 2023-10-21T17:39:55.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.402+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20800d3080 con 0x7f208015cc60 2023-10-21T17:39:55.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.403+0000 7f207d7fa640 1 -- 172.21.15.53:0/4226947922 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+20631 (secure 0 0 0) 0x7f2078058660 con 0x7f208015cc60 2023-10-21T17:39:55.405 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:55.406 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":257,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:53.948288+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 msgr2=0x7f206406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 0x7f206406d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2074000fb0 tx=0x7f2074009040 comp rx=0 tx=0).stop 2023-10-21T17:39:55.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 msgr2=0x7f20800b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f2078007cf0 tx=0x7f207800e4b0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 shutdown_connections 2023-10-21T17:39:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f206406b0b0 0x7f206406d570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f208015cc60 0x7f20800b7be0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20800d16f0 0x7f20801557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 --2- 172.21.15.53:0/4226947922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20800cfa50 0x7f20800b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 >> 172.21.15.53:0/4226947922 conn(0x7f208009bcf0 msgr2=0x7f208006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 shutdown_connections 2023-10-21T17:39:55.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.406+0000 7f2085aca640 1 -- 172.21.15.53:0/4226947922 wait complete. 2023-10-21T17:39:55.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-21T17:39:55.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.655+0000 7f103fd33640 1 Processor -- start 2023-10-21T17:39:55.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.655+0000 7f103fd33640 1 -- start start 2023-10-21T17:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103fd33640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1038161690 0x7f103806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103fd33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f10380cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103fd33640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103806cc50 con 0x7f10380cb150 2023-10-21T17:39:55.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103fd33640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10380cbb40 con 0x7f1038161690 2023-10-21T17:39:55.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103fd33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103806cf30 con 0x7f10380b2f30 2023-10-21T17:39:55.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103d2ce640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f10380cb150 0x7f10380702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57706/0 (socket says 172.21.15.53:57706) 2023-10-21T17:39:55.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103d2ce640 1 -- 172.21.15.53:0/3110937060 learned_addr learned my addr 172.21.15.53:0/3110937060 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:55.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103dacf640 1 --2- 172.21.15.53:0/3110937060 >> v2:172.21.15.124:3300/0 conn(0x7f1038161690 0x7f103806dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.657+0000 7f103e2d0640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f10380cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831048222 0 0) 0x7f103806cc50 con 0x7f10380cb150 2023-10-21T17:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1024003520 con 0x7f10380cb150 2023-10-21T17:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333197016 0 0) 0x7f1024003520 con 0x7f10380cb150 2023-10-21T17:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103dacf640 1 -- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 msgr2=0x7f10380cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:55.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103dacf640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f10380cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103dacf640 1 -- 172.21.15.53:0/3110937060 >> v1:172.21.15.53:6789/0 conn(0x7f10380cb150 legacy=0x7f10380702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.658+0000 7f103dacf640 1 -- 172.21.15.53:0/3110937060 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103806d210 con 0x7f1038161690 2023-10-21T17:39:55.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.660+0000 7f103dacf640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f103806dad0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1034009d90 tx=0x7f103402bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d359f51aa070d271 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.660+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028001690 con 0x7f1038161690 2023-10-21T17:39:55.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.660+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103402c8a0 con 0x7f1038161690 2023-10-21T17:39:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.660+0000 7f103cacd640 1 -- 172.21.15.53:0/3110937060 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103402cc30 con 0x7f1038161690 2023-10-21T17:39:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 -- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 msgr2=0x7f103806dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f103806dad0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1034009d90 tx=0x7f103402bd70 comp rx=0 tx=0).stop 2023-10-21T17:39:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 -- 172.21.15.53:0/3110937060 shutdown_connections 2023-10-21T17:39:55.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f10380cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 --2- 172.21.15.53:0/3110937060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f103806dad0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 -- 172.21.15.53:0/3110937060 >> 172.21.15.53:0/3110937060 conn(0x7f103806a5d0 msgr2=0x7f10380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 -- 172.21.15.53:0/3110937060 shutdown_connections 2023-10-21T17:39:55.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.661+0000 7f103fd33640 1 -- 172.21.15.53:0/3110937060 wait complete. 2023-10-21T17:39:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.663+0000 7f103fd33640 1 Processor -- start 2023-10-21T17:39:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.663+0000 7f103fd33640 1 -- start start 2023-10-21T17:39:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f1038161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f10380c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c3660 con 0x7f10380cb150 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c29e0 con 0x7f1038161690 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103fd33640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10380c2cc0 con 0x7f10380b2f30 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103d2ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103d2ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53184/0 (socket says 172.21.15.53:53184) 2023-10-21T17:39:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.664+0000 7f103d2ce640 1 -- 172.21.15.53:0/3279955126 learned_addr learned my addr 172.21.15.53:0/3279955126 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103dacf640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f1038161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103e2d0640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f10380c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 -- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 msgr2=0x7f1038161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f1038161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 -- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 msgr2=0x7f10380c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f10380c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 -- 172.21.15.53:0/3279955126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034009a70 con 0x7f10380cb150 2023-10-21T17:39:55.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103e2d0640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f10380c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103d2ce640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f102800c780 tx=0x7f102800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10280074e0 con 0x7f10380cb150 2023-10-21T17:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1028007660 con 0x7f10380cb150 2023-10-21T17:39:55.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10380c2fa0 con 0x7f10380cb150 2023-10-21T17:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.665+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028004a20 con 0x7f10380cb150 2023-10-21T17:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.666+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380c2670 con 0x7f10380cb150 2023-10-21T17:39:55.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.667+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103804e160 con 0x7f10380cb150 2023-10-21T17:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.668+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1028004b80 con 0x7f10380cb150 2023-10-21T17:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.669+0000 7f102e7fc640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:55.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.669+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f10280896e0 con 0x7f10380cb150 2023-10-21T17:39:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.670+0000 7f103dacf640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:55.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.671+0000 7f103dacf640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 0x7f101c06d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1034036000 tx=0x7f1034030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:55.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.675+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1028053e70 con 0x7f10380cb150 2023-10-21T17:39:55.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:55.921+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f10380b4110 con 0x7f10380cb150 2023-10-21T17:39:56.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.512+0000 7f102e7fc640 1 -- 172.21.15.53:0/3279955126 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v133) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1028057490 con 0x7f10380cb150 2023-10-21T17:39:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 msgr2=0x7f101c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 0x7f101c06d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1034036000 tx=0x7f1034030000 comp rx=0 tx=0).stop 2023-10-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 msgr2=0x7f1038161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f102800c780 tx=0x7f102800cc40 comp rx=0 tx=0).stop 2023-10-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 shutdown_connections 2023-10-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f101c06b0b0 0x7f101c06d570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1038161690 0x7f10380c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f10380cb150 0x7f1038161fb0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 --2- 172.21.15.53:0/3279955126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10380b2f30 0x7f1038161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 >> 172.21.15.53:0/3279955126 conn(0x7f103806a5d0 msgr2=0x7f10380733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 shutdown_connections 2023-10-21T17:39:56.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.516+0000 7f103fd33640 1 -- 172.21.15.53:0/3279955126 wait complete. 2023-10-21T17:39:56.539 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:39:56.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-21T17:39:56.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.771+0000 7f2a871c0640 1 Processor -- start 2023-10-21T17:39:56.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.771+0000 7f2a871c0640 1 -- start start 2023-10-21T17:39:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.772+0000 7f2a871c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.772+0000 7f2a871c0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2a8006a460 0x7f2a800ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.772+0000 7f2a871c0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a800ce210 con 0x7f2a80071c30 2023-10-21T17:39:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.772+0000 7f2a871c0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a800cdee0 con 0x7f2a8006a460 2023-10-21T17:39:56.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.772+0000 7f2a871c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80161530 con 0x7f2a801601b0 2023-10-21T17:39:56.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.773+0000 7f2a84f5c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.773+0000 7f2a84f5c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:51922/0 (socket says 172.21.15.53:51922) 2023-10-21T17:39:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.773+0000 7f2a8575d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2a8006a460 0x7f2a800ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.773+0000 7f2a84f5c640 1 -- 172.21.15.53:0/712557221 learned_addr learned my addr 172.21.15.53:0/712557221 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.774+0000 7f2a84f5c640 1 -- 172.21.15.53:0/712557221 >> v2:172.21.15.124:3300/0 conn(0x7f2a8006a460 msgr2=0x7f2a800ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.774+0000 7f2a84f5c640 1 --2- 172.21.15.53:0/712557221 >> v2:172.21.15.124:3300/0 conn(0x7f2a8006a460 0x7f2a800ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.774+0000 7f2a84f5c640 1 -- 172.21.15.53:0/712557221 >> v1:172.21.15.124:6789/0 conn(0x7f2a80071c30 legacy=0x7f2a80069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.774+0000 7f2a84f5c640 1 -- 172.21.15.53:0/712557221 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80161810 con 0x7f2a801601b0 2023-10-21T17:39:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.775+0000 7f2a84f5c640 1 --2- 172.21.15.53:0/712557221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009870 tx=0x7f2a7402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f73a396a42942d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:56.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.776+0000 7f2a7f7fe640 1 -- 172.21.15.53:0/712557221 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74030840 con 0x7f2a801601b0 2023-10-21T17:39:56.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.776+0000 7f2a7f7fe640 1 -- 172.21.15.53:0/712557221 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a740309a0 con 0x7f2a801601b0 2023-10-21T17:39:56.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.776+0000 7f2a7f7fe640 1 -- 172.21.15.53:0/712557221 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74030d30 con 0x7f2a801601b0 2023-10-21T17:39:56.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 -- 172.21.15.53:0/712557221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 msgr2=0x7f2a800cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 --2- 172.21.15.53:0/712557221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009870 tx=0x7f2a7402b980 comp rx=0 tx=0).stop 2023-10-21T17:39:56.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 -- 172.21.15.53:0/712557221 shutdown_connections 2023-10-21T17:39:56.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 --2- 172.21.15.53:0/712557221 >> v2:172.21.15.124:3300/0 conn(0x7f2a8006a460 0x7f2a800ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 --2- 172.21.15.53:0/712557221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a801601b0 0x7f2a800cd430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 -- 172.21.15.53:0/712557221 >> 172.21.15.53:0/712557221 conn(0x7f2a8006d430 msgr2=0x7f2a8006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 -- 172.21.15.53:0/712557221 shutdown_connections 2023-10-21T17:39:56.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.777+0000 7f2a871c0640 1 -- 172.21.15.53:0/712557221 wait complete. 2023-10-21T17:39:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.779+0000 7f2a871c0640 1 Processor -- start 2023-10-21T17:39:56.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.779+0000 7f2a871c0640 1 -- start start 2023-10-21T17:39:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a8006a460 0x7f2a80149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a80071c30 0x7f2a8015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800ad340 con 0x7f2a801601b0 2023-10-21T17:39:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800c9b90 con 0x7f2a8006a460 2023-10-21T17:39:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a871c0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8014c740 con 0x7f2a80071c30 2023-10-21T17:39:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.780+0000 7f2a8575d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53196/0 (socket says 172.21.15.53:53196) 2023-10-21T17:39:56.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 -- 172.21.15.53:0/1622755899 learned_addr learned my addr 172.21.15.53:0/1622755899 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:56.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a84f5c640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a8006a460 0x7f2a80149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a7ffff640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a80071c30 0x7f2a8015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 -- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a80071c30 msgr2=0x7f2a8015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a80071c30 0x7f2a8015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 -- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a8006a460 msgr2=0x7f2a80149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a8006a460 0x7f2a80149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 -- 172.21.15.53:0/1622755899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a740094d0 con 0x7f2a801601b0 2023-10-21T17:39:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.781+0000 7f2a8575d640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009cb0 tx=0x7f2a7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.782+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78009b20 con 0x7f2a801601b0 2023-10-21T17:39:56.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.782+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7800f040 con 0x7f2a801601b0 2023-10-21T17:39:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.782+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780146c0 con 0x7f2a801601b0 2023-10-21T17:39:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.782+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a800aca50 con 0x7f2a801601b0 2023-10-21T17:39:56.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.782+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a800accf0 con 0x7f2a801601b0 2023-10-21T17:39:56.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.783+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a800ad4f0 con 0x7f2a801601b0 2023-10-21T17:39:56.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.784+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2a78014820 con 0x7f2a801601b0 2023-10-21T17:39:56.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.785+0000 7f2a7d7fa640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 0x7f2a6406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:56.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.785+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f2a7808ae10 con 0x7f2a801601b0 2023-10-21T17:39:56.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.786+0000 7f2a84f5c640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 0x7f2a6406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:56.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.787+0000 7f2a84f5c640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 0x7f2a6406d5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a74036000 tx=0x7f2a7403b040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:56.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:56.789+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a800ad4f0 con 0x7f2a801601b0 2023-10-21T17:39:57.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.020+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a800ad4f0 con 0x7f2a801601b0 2023-10-21T17:39:57.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.523+0000 7f2a7d7fa640 1 -- 172.21.15.53:0/1622755899 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v134) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2a800ad4f0 con 0x7f2a801601b0 2023-10-21T17:39:57.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 msgr2=0x7f2a6406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 0x7f2a6406d5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a74036000 tx=0x7f2a7403b040 comp rx=0 tx=0).stop 2023-10-21T17:39:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 msgr2=0x7f2a80148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009cb0 tx=0x7f2a7800c490 comp rx=0 tx=0).stop 2023-10-21T17:39:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 shutdown_connections 2023-10-21T17:39:57.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a6406b0e0 0x7f2a6406d5a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a801601b0 0x7f2a80148e20 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a80071c30 0x7f2a8015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 --2- 172.21.15.53:0/1622755899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a8006a460 0x7f2a80149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 >> 172.21.15.53:0/1622755899 conn(0x7f2a8006d430 msgr2=0x7f2a800b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:57.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 shutdown_connections 2023-10-21T17:39:57.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.527+0000 7f2a871c0640 1 -- 172.21.15.53:0/1622755899 wait complete. 2023-10-21T17:39:57.549 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-21T17:39:57.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:57.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.771+0000 7fe4c5040640 1 Processor -- start 2023-10-21T17:39:57.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.771+0000 7fe4c5040640 1 -- start start 2023-10-21T17:39:57.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.773+0000 7fe4c5040640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 0x7fe4c00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.773+0000 7fe4c5040640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 0x7fe4c00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.773+0000 7fe4c5040640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00cacf0 con 0x7fe4c0159db0 2023-10-21T17:39:57.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.773+0000 7fe4c5040640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c00b10f0 con 0x7fe4c00cc210 2023-10-21T17:39:57.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.773+0000 7fe4c5040640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00ca3c0 con 0x7fe4c01599d0 2023-10-21T17:39:57.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.774+0000 7fe4bf7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 0x7fe4c00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.774+0000 7fe4bf7fe640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 0x7fe4c00b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:46746/0 (socket says 172.21.15.53:46746) 2023-10-21T17:39:57.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.774+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/2337335070 learned_addr learned my addr 172.21.15.53:0/2337335070 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.774+0000 7fe4bffff640 1 --2- 172.21.15.53:0/2337335070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 0x7fe4c00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.775+0000 7fe4bffff640 1 -- 172.21.15.53:0/2337335070 >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 msgr2=0x7fe4c00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.775+0000 7fe4bffff640 1 --2- 172.21.15.53:0/2337335070 >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 0x7fe4c00b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.775+0000 7fe4bffff640 1 -- 172.21.15.53:0/2337335070 >> v1:172.21.15.124:6789/0 conn(0x7fe4c0159db0 legacy=0x7fe4c015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.775+0000 7fe4bffff640 1 -- 172.21.15.53:0/2337335070 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00ca6a0 con 0x7fe4c01599d0 2023-10-21T17:39:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.776+0000 7fe4bffff640 1 --2- 172.21.15.53:0/2337335070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 0x7fe4c00ad730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00097a0 tx=0x7fe4b002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a10054a3b7bb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.776+0000 7fe4beffd640 1 -- 172.21.15.53:0/2337335070 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b002d040 con 0x7fe4c01599d0 2023-10-21T17:39:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4beffd640 1 -- 172.21.15.53:0/2337335070 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0031810 con 0x7fe4c01599d0 2023-10-21T17:39:57.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4beffd640 1 -- 172.21.15.53:0/2337335070 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0031ba0 con 0x7fe4c01599d0 2023-10-21T17:39:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 -- 172.21.15.53:0/2337335070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 msgr2=0x7fe4c00ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 --2- 172.21.15.53:0/2337335070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 0x7fe4c00ad730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00097a0 tx=0x7fe4b002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 -- 172.21.15.53:0/2337335070 shutdown_connections 2023-10-21T17:39:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 --2- 172.21.15.53:0/2337335070 >> v2:172.21.15.124:3300/0 conn(0x7fe4c00cc210 0x7fe4c00b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 --2- 172.21.15.53:0/2337335070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c01599d0 0x7fe4c00ad730 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.777+0000 7fe4c5040640 1 -- 172.21.15.53:0/2337335070 >> 172.21.15.53:0/2337335070 conn(0x7fe4c00cb0c0 msgr2=0x7fe4c00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:57.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.778+0000 7fe4c5040640 1 -- 172.21.15.53:0/2337335070 shutdown_connections 2023-10-21T17:39:57.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.778+0000 7fe4c5040640 1 -- 172.21.15.53:0/2337335070 wait complete. 2023-10-21T17:39:57.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.780+0000 7fe4c5040640 1 Processor -- start 2023-10-21T17:39:57.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.780+0000 7fe4c5040640 1 -- start start 2023-10-21T17:39:57.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c00cc210 0x7fe4c00ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c0159db0 0x7fe4c0156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c01f7b40 con 0x7fe4c01599d0 2023-10-21T17:39:57.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c01fa020 con 0x7fe4c0159db0 2023-10-21T17:39:57.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c5040640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00ae610 con 0x7fe4c00cc210 2023-10-21T17:39:57.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4bf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53204/0 (socket says 172.21.15.53:53204) 2023-10-21T17:39:57.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1185792049 learned_addr learned my addr 172.21.15.53:0/1185792049 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:57.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4bffff640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c00cc210 0x7fe4c00ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.782+0000 7fe4c483f640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c0159db0 0x7fe4c0156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c00cc210 msgr2=0x7fe4c00ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c00cc210 0x7fe4c00ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c0159db0 msgr2=0x7fe4c0156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:57.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c0159db0 0x7fe4c0156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:57.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 -- 172.21.15.53:0/1185792049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b0009400 con 0x7fe4c01599d0 2023-10-21T17:39:57.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bf7fe640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e900 tx=0x7fe4b400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:57.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400ca00 con 0x7fe4c01599d0 2023-10-21T17:39:57.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400cb60 con 0x7fe4c01599d0 2023-10-21T17:39:57.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.783+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4013470 con 0x7fe4c01599d0 2023-10-21T17:39:57.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.784+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c01fa550 con 0x7fe4c01599d0 2023-10-21T17:39:57.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.784+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c01fab10 con 0x7fe4c01599d0 2023-10-21T17:39:57.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.785+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c004e160 con 0x7fe4c01599d0 2023-10-21T17:39:57.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.786+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe4b40135d0 con 0x7fe4c01599d0 2023-10-21T17:39:57.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.787+0000 7fe4bcff9640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 0x7fe4ac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:57.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.787+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fe4b408a100 con 0x7fe4c01599d0 2023-10-21T17:39:57.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.788+0000 7fe4bffff640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 0x7fe4ac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:57.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.789+0000 7fe4bffff640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 0x7fe4ac06d570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0036000 tx=0x7fe4b003c040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:57.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:57.794+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4b40549c0 con 0x7fe4c01599d0 2023-10-21T17:39:58.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.027+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4c01fa270 con 0x7fe4c01599d0 2023-10-21T17:39:58.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.028+0000 7fe4bcff9640 1 -- 172.21.15.53:0/1185792049 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7fe4b4057fe0 con 0x7fe4c01599d0 2023-10-21T17:39:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:58.030 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:39:58.030 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:39:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 msgr2=0x7fe4ac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 0x7fe4ac06d570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0036000 tx=0x7fe4b003c040 comp rx=0 tx=0).stop 2023-10-21T17:39:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 msgr2=0x7fe4c0156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e900 tx=0x7fe4b400edc0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 shutdown_connections 2023-10-21T17:39:58.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fe4ac06b0b0 0x7fe4ac06d570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe4c0159db0 0x7fe4c0156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4c01599d0 0x7fe4c0156520 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 --2- 172.21.15.53:0/1185792049 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4c00cc210 0x7fe4c00ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.034 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 >> 172.21.15.53:0/1185792049 conn(0x7fe4c00cb0c0 msgr2=0x7fe4c006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:58.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 shutdown_connections 2023-10-21T17:39:58.035 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.031+0000 7fe4c5040640 1 -- 172.21.15.53:0/1185792049 wait complete. 2023-10-21T17:39:58.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:39:58.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.278+0000 7ff4af59e640 1 Processor -- start 2023-10-21T17:39:58.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.278+0000 7ff4af59e640 1 -- start start 2023-10-21T17:39:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4af59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff4b006bf60 0x7ff4b0167c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4af59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00cb590 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4af59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00cfc90 con 0x7ff4b00cf720 2023-10-21T17:39:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4af59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b00cff70 con 0x7ff4b006bf60 2023-10-21T17:39:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4ae59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff4b00cd8a0 0x7ff4b006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57720/0 (socket says 172.21.15.53:57720) 2023-10-21T17:39:58.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4ae59c640 1 -- 172.21.15.53:0/1531465245 learned_addr learned my addr 172.21.15.53:0/1531465245 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4aed9d640 1 --2- 172.21.15.53:0/1531465245 >> v2:172.21.15.124:3300/0 conn(0x7ff4b006bf60 0x7ff4b0167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.280+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467732987 0 0) 0x7ff4b00cb590 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c003520 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3403767951 0 0) 0x7ff49c003520 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 >> v2:172.21.15.124:3300/0 conn(0x7ff4b006bf60 msgr2=0x7ff4b0167c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:58.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 --2- 172.21.15.53:0/1531465245 >> v2:172.21.15.124:3300/0 conn(0x7ff4b006bf60 0x7ff4b0167c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 >> v1:172.21.15.124:6789/0 conn(0x7ff4b00cf720 legacy=0x7ff4b006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00caf40 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7ff4a00019f0 con 0x7ff4b00cd8a0 2023-10-21T17:39:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 >> v1:172.21.15.53:6789/0 conn(0x7ff4b00cd8a0 legacy=0x7ff4b006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 0x7ff49c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff49c0047c0 0x7ff49c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c005180 0x7ff49c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00cff70 con 0x7ff49c0039c0 2023-10-21T17:39:58.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00c080 con 0x7ff49c0047c0 2023-10-21T17:39:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00c240 con 0x7ff49c005180 2023-10-21T17:39:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.281+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 0x7ff49c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4ae59c640 1 -- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c005180 msgr2=0x7ff49c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c005180 0x7ff49c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4ae59c640 1 -- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff49c0047c0 msgr2=0x7ff49c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff49c0047c0 0x7ff49c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4ae59c640 1 -- 172.21.15.53:0/1531465245 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c003520 con 0x7ff49c0039c0 2023-10-21T17:39:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.282+0000 7ff4aed9d640 1 -- 172.21.15.53:0/1531465245 reap_dead start 2023-10-21T17:39:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.283+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 0x7ff49c003de0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000a760 tx=0x7ff49c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359d6cdd0e37492d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.283+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0034630 con 0x7ff49c0039c0 2023-10-21T17:39:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.283+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a002f070 con 0x7ff49c0039c0 2023-10-21T17:39:58.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.283+0000 7ff4ad59a640 1 -- 172.21.15.53:0/1531465245 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0034a50 con 0x7ff49c0039c0 2023-10-21T17:39:58.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 -- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 msgr2=0x7ff49c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 0x7ff49c003de0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000a760 tx=0x7ff49c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 -- 172.21.15.53:0/1531465245 shutdown_connections 2023-10-21T17:39:58.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 --2- 172.21.15.53:0/1531465245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff49c0039c0 0x7ff49c003de0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 -- 172.21.15.53:0/1531465245 >> 172.21.15.53:0/1531465245 conn(0x7ff4b0099b60 msgr2=0x7ff4b00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:58.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 -- 172.21.15.53:0/1531465245 shutdown_connections 2023-10-21T17:39:58.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.284+0000 7ff4af59e640 1 -- 172.21.15.53:0/1531465245 wait complete. 2023-10-21T17:39:58.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.286+0000 7ff4af59e640 1 Processor -- start 2023-10-21T17:39:58.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.286+0000 7ff4af59e640 1 -- start start 2023-10-21T17:39:58.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c0039c0 0x7ff4b0154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 0x7ff4b00ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c5a90 con 0x7ff4b009cd50 2023-10-21T17:39:58.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c6610 con 0x7ff4b009d1d0 2023-10-21T17:39:58.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4af59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c4fb0 con 0x7ff49c0039c0 2023-10-21T17:39:58.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4add9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4add9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53224/0 (socket says 172.21.15.53:53224) 2023-10-21T17:39:58.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.287+0000 7ff4add9b640 1 -- 172.21.15.53:0/2888555706 learned_addr learned my addr 172.21.15.53:0/2888555706 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:58.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c0039c0 0x7ff4b0154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4aed9d640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 0x7ff4b00ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 -- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c0039c0 msgr2=0x7ff4b0154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c0039c0 0x7ff4b0154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 -- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 msgr2=0x7ff4b00ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 0x7ff4b00ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 -- 172.21.15.53:0/2888555706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00c5290 con 0x7ff4b009cd50 2023-10-21T17:39:58.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4aed9d640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 0x7ff4b00ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:58.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4add9b640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7ff49800c500 tx=0x7ff49800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980052d0 con 0x7ff4b009cd50 2023-10-21T17:39:58.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498005430 con 0x7ff4b009cd50 2023-10-21T17:39:58.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498012870 con 0x7ff4b009cd50 2023-10-21T17:39:58.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.288+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b00c5fc0 con 0x7ff4b009cd50 2023-10-21T17:39:58.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.289+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b00c4380 con 0x7ff4b009cd50 2023-10-21T17:39:58.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.290+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b00c6780 con 0x7ff4b009cd50 2023-10-21T17:39:58.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.291+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff4980129d0 con 0x7ff4b009cd50 2023-10-21T17:39:58.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.292+0000 7ff4a6ffd640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 0x7ff49006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.292+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7ff498089bc0 con 0x7ff4b009cd50 2023-10-21T17:39:58.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.292+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 0x7ff49006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.294+0000 7ff4ae59c640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 0x7ff49006d5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0034000 tx=0x7ff4a002f000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.305 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.304+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff498057e70 con 0x7ff4b009cd50 2023-10-21T17:39:58.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.510+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b004e160 con 0x7ff4b009cd50 2023-10-21T17:39:58.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.512+0000 7ff4a6ffd640 1 -- 172.21.15.53:0/2888555706 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+20618 (secure 0 0 0) 0x7ff498057a00 con 0x7ff4b009cd50 2023-10-21T17:39:58.514 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:58.515 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":258,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:39:55.490421+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:39:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.515+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 msgr2=0x7ff49006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.515+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 0x7ff49006d5a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0034000 tx=0x7ff4a002f000 comp rx=0 tx=0).stop 2023-10-21T17:39:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.515+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 msgr2=0x7ff4b00c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.515+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7ff49800c500 tx=0x7ff49800c9c0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 shutdown_connections 2023-10-21T17:39:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ff49006b0e0 0x7ff49006d5a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4b009d1d0 0x7ff4b00ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4b009cd50 0x7ff4b00c9af0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 --2- 172.21.15.53:0/2888555706 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff49c0039c0 0x7ff4b0154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 >> 172.21.15.53:0/2888555706 conn(0x7ff4b0099b60 msgr2=0x7ff4b00b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 shutdown_connections 2023-10-21T17:39:58.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.516+0000 7ff4af59e640 1 -- 172.21.15.53:0/2888555706 wait complete. 2023-10-21T17:39:58.536 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:39:58.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:39:58.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.754+0000 7f3a77fff640 1 Processor -- start 2023-10-21T17:39:58.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.754+0000 7f3a77fff640 1 -- start start 2023-10-21T17:39:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3a780cd8a0 0x7f3a7806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cf720 0x7f3a7806b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a7806bd90 0x7f3a781608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7806c220 con 0x7f3a7806bd90 2023-10-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a780cb590 con 0x7f3a780cd8a0 2023-10-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.756+0000 7f3a77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780cfc90 con 0x7f3a780cf720 2023-10-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a7806bd90 0x7f3a781608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3a7806bd90 0x7f3a781608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53226/0 (socket says 172.21.15.53:53226) 2023-10-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 -- 172.21.15.53:0/4109369483 learned_addr learned my addr 172.21.15.53:0/4109369483 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a76ffd640 1 --2- 172.21.15.53:0/4109369483 >> v2:172.21.15.124:3300/0 conn(0x7f3a780cd8a0 0x7f3a7806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a6ffff640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cf720 0x7f3a7806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 -- 172.21.15.53:0/4109369483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cf720 msgr2=0x7f3a7806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cf720 0x7f3a7806b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 -- 172.21.15.53:0/4109369483 >> v2:172.21.15.124:3300/0 conn(0x7f3a780cd8a0 msgr2=0x7f3a7806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 --2- 172.21.15.53:0/4109369483 >> v2:172.21.15.124:3300/0 conn(0x7f3a780cd8a0 0x7f3a7806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.757+0000 7f3a777fe640 1 -- 172.21.15.53:0/4109369483 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780cff70 con 0x7f3a7806bd90 2023-10-21T17:39:58.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.758+0000 7f3a777fe640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781608f0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f3a7808ae50 tx=0x7f3a7002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbbd65ea2fff928 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.759+0000 7f3a767fc640 1 -- 172.21.15.53:0/4109369483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7002ca80 con 0x7f3a7806bd90 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.759+0000 7f3a767fc640 1 -- 172.21.15.53:0/4109369483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7002cbe0 con 0x7f3a7806bd90 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.759+0000 7f3a767fc640 1 -- 172.21.15.53:0/4109369483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a700343f0 con 0x7f3a7806bd90 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 -- 172.21.15.53:0/4109369483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 msgr2=0x7f3a781608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781608f0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f3a7808ae50 tx=0x7f3a7002bee0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 -- 172.21.15.53:0/4109369483 shutdown_connections 2023-10-21T17:39:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cf720 0x7f3a7806b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 --2- 172.21.15.53:0/4109369483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781608f0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 --2- 172.21.15.53:0/4109369483 >> v2:172.21.15.124:3300/0 conn(0x7f3a780cd8a0 0x7f3a7806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 -- 172.21.15.53:0/4109369483 >> 172.21.15.53:0/4109369483 conn(0x7f3a78099b60 msgr2=0x7f3a780d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:58.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 -- 172.21.15.53:0/4109369483 shutdown_connections 2023-10-21T17:39:58.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.760+0000 7f3a77fff640 1 -- 172.21.15.53:0/4109369483 wait complete. 2023-10-21T17:39:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.762+0000 7f3a77fff640 1 Processor -- start 2023-10-21T17:39:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.762+0000 7f3a77fff640 1 -- start start 2023-10-21T17:39:58.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cd8a0 0x7f3a7815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a780cf720 0x7f3a780cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781f6550 con 0x7f3a7806bd90 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781f5cb0 con 0x7f3a780cf720 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781f5f90 con 0x7f3a780cd8a0 2023-10-21T17:39:58.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53234/0 (socket says 172.21.15.53:53234) 2023-10-21T17:39:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.763+0000 7f3a76ffd640 1 -- 172.21.15.53:0/1903899499 learned_addr learned my addr 172.21.15.53:0/1903899499 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a6ffff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cd8a0 0x7f3a7815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a777fe640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a780cf720 0x7f3a780cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 -- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cd8a0 msgr2=0x7f3a7815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cd8a0 0x7f3a7815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 -- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a780cf720 msgr2=0x7f3a780cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a780cf720 0x7f3a780cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 -- 172.21.15.53:0/1903899499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a70009b30 con 0x7f3a7806bd90 2023-10-21T17:39:58.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a76ffd640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800d800 tx=0x7f3a6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.764+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680096a0 con 0x7f3a7806bd90 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.765+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68009800 con 0x7f3a7806bd90 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.765+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68009990 con 0x7f3a7806bd90 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.765+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a781f6270 con 0x7f3a7806bd90 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.765+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a780ca690 con 0x7f3a7806bd90 2023-10-21T17:39:58.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.766+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a780c4270 con 0x7f3a7806bd90 2023-10-21T17:39:58.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.767+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3a68009af0 con 0x7f3a7806bd90 2023-10-21T17:39:58.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.768+0000 7f3a6f7fe640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 0x7f3a5806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.768+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3a6808a410 con 0x7f3a7806bd90 2023-10-21T17:39:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.768+0000 7f3a6ffff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 0x7f3a5806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:58.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.770+0000 7f3a6ffff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 0x7f3a5806d570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a780c64a0 tx=0x7f3a60008040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:58.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.772+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a68054c70 con 0x7f3a7806bd90 2023-10-21T17:39:58.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:58.962+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a780cb220 con 0x7f3a5806b0b0 2023-10-21T17:39:59.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.006+0000 7f3a6f7fe640 1 -- 172.21.15.53:0/1903899499 <== mgr.38150 v2:172.21.15.124:6832/3816427222 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526182 (secure 0 0 0) 0x7f3a6000b200 con 0x7f3a5806b0b0 2023-10-21T17:39:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:59.018 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:39:59.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 msgr2=0x7f3a5806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 0x7f3a5806d570 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a780c64a0 tx=0x7f3a60008040 comp rx=0 tx=0).stop 2023-10-21T17:39:59.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 msgr2=0x7f3a781f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800d800 tx=0x7f3a6800dcc0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 shutdown_connections 2023-10-21T17:39:59.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f3a5806b0b0 0x7f3a5806d570 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a780cf720 0x7f3a780cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a780cd8a0 0x7f3a7815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 --2- 172.21.15.53:0/1903899499 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a7806bd90 0x7f3a781f56d0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 >> 172.21.15.53:0/1903899499 conn(0x7f3a78099b60 msgr2=0x7f3a780ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:59.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 shutdown_connections 2023-10-21T17:39:59.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.018+0000 7f3a77fff640 1 -- 172.21.15.53:0/1903899499 wait complete. 2023-10-21T17:39:59.045 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2023-10-21T17:39:58.543569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":718,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4974,"ondisk_log_size":4974,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":328,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453780,"kb_used_data":26652,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754520940,"statfs":{"total":773094113280,"available":772629442560,"internally_reserved":0,"allocated":27291648,"data_stored":14760343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":437243449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002131"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504444+0000","last_change":"2023-10-21T17:39:54.490986+0000","last_active":"2023-10-21T17:39:55.504444+0000","last_peered":"2023-10-21T17:39:55.504444+0000","last_clean":"2023-10-21T17:39:55.504444+0000","last_became_active":"2023-10-21T17:39:54.490465+0000","last_became_peered":"2023-10-21T17:39:54.490465+0000","last_unstale":"2023-10-21T17:39:55.504444+0000","last_undegraded":"2023-10-21T17:39:55.504444+0000","last_fullsized":"2023-10-21T17:39:55.504444+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:43.007785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832939+0000","last_change":"2023-10-21T17:39:53.501960+0000","last_active":"2023-10-21T17:39:55.832939+0000","last_peered":"2023-10-21T17:39:55.832939+0000","last_clean":"2023-10-21T17:39:55.832939+0000","last_became_active":"2023-10-21T17:39:53.501683+0000","last_became_peered":"2023-10-21T17:39:53.501683+0000","last_unstale":"2023-10-21T17:39:55.832939+0000","last_undegraded":"2023-10-21T17:39:55.832939+0000","last_fullsized":"2023-10-21T17:39:55.832939+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:34:13.628412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483597+0000","last_change":"2023-10-21T17:39:40.283996+0000","last_active":"2023-10-21T17:39:54.483597+0000","last_peered":"2023-10-21T17:39:54.483597+0000","last_clean":"2023-10-21T17:39:54.483597+0000","last_became_active":"2023-10-21T17:39:40.283690+0000","last_became_peered":"2023-10-21T17:39:40.283690+0000","last_unstale":"2023-10-21T17:39:54.483597+0000","last_undegraded":"2023-10-21T17:39:54.483597+0000","last_fullsized":"2023-10-21T17:39:54.483597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:37:50.486075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832491+0000","last_change":"2023-10-21T17:39:54.513144+0000","last_active":"2023-10-21T17:39:55.832491+0000","last_peered":"2023-10-21T17:39:55.832491+0000","last_clean":"2023-10-21T17:39:55.832491+0000","last_became_active":"2023-10-21T17:39:54.512836+0000","last_became_peered":"2023-10-21T17:39:54.512836+0000","last_unstale":"2023-10-21T17:39:55.832491+0000","last_undegraded":"2023-10-21T17:39:55.832491+0000","last_fullsized":"2023-10-21T17:39:55.832491+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:59.066123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"257'4","reported_seq":16,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504600+0000","last_change":"2023-10-21T17:39:53.504447+0000","last_active":"2023-10-21T17:39:55.504600+0000","last_peered":"2023-10-21T17:39:55.504600+0000","last_clean":"2023-10-21T17:39:55.504600+0000","last_became_active":"2023-10-21T17:39:53.504179+0000","last_became_peered":"2023-10-21T17:39:53.504179+0000","last_unstale":"2023-10-21T17:39:55.504600+0000","last_undegraded":"2023-10-21T17:39:55.504600+0000","last_fullsized":"2023-10-21T17:39:55.504600+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:36.904495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489453+0000","last_change":"2023-10-21T17:39:40.279882+0000","last_active":"2023-10-21T17:39:54.489453+0000","last_peered":"2023-10-21T17:39:54.489453+0000","last_clean":"2023-10-21T17:39:54.489453+0000","last_became_active":"2023-10-21T17:39:40.279568+0000","last_became_peered":"2023-10-21T17:39:40.279568+0000","last_unstale":"2023-10-21T17:39:54.489453+0000","last_undegraded":"2023-10-21T17:39:54.489453+0000","last_fullsized":"2023-10-21T17:39:54.489453+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:07.488724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.871394+0000","last_change":"2023-10-21T17:39:54.521730+0000","last_active":"2023-10-21T17:39:55.871394+0000","last_peered":"2023-10-21T17:39:55.871394+0000","last_clean":"2023-10-21T17:39:55.871394+0000","last_became_active":"2023-10-21T17:39:54.519697+0000","last_became_peered":"2023-10-21T17:39:54.519697+0000","last_unstale":"2023-10-21T17:39:55.871394+0000","last_undegraded":"2023-10-21T17:39:55.871394+0000","last_fullsized":"2023-10-21T17:39:55.871394+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:08.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832220+0000","last_change":"2023-10-21T17:39:53.499872+0000","last_active":"2023-10-21T17:39:55.832220+0000","last_peered":"2023-10-21T17:39:55.832220+0000","last_clean":"2023-10-21T17:39:55.832220+0000","last_became_active":"2023-10-21T17:39:53.499116+0000","last_became_peered":"2023-10-21T17:39:53.499116+0000","last_unstale":"2023-10-21T17:39:55.832220+0000","last_undegraded":"2023-10-21T17:39:55.832220+0000","last_fullsized":"2023-10-21T17:39:55.832220+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:33:30.710008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.495450+0000","last_change":"2023-10-21T17:39:40.277265+0000","last_active":"2023-10-21T17:39:54.495450+0000","last_peered":"2023-10-21T17:39:54.495450+0000","last_clean":"2023-10-21T17:39:54.495450+0000","last_became_active":"2023-10-21T17:39:40.276998+0000","last_became_peered":"2023-10-21T17:39:40.276998+0000","last_unstale":"2023-10-21T17:39:54.495450+0000","last_undegraded":"2023-10-21T17:39:54.495450+0000","last_fullsized":"2023-10-21T17:39:54.495450+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:14.379177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504700+0000","last_change":"2023-10-21T17:39:54.494099+0000","last_active":"2023-10-21T17:39:55.504700+0000","last_peered":"2023-10-21T17:39:55.504700+0000","last_clean":"2023-10-21T17:39:55.504700+0000","last_became_active":"2023-10-21T17:39:54.493629+0000","last_became_peered":"2023-10-21T17:39:54.493629+0000","last_unstale":"2023-10-21T17:39:55.504700+0000","last_undegraded":"2023-10-21T17:39:55.504700+0000","last_fullsized":"2023-10-21T17:39:55.504700+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:58.263674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509742+0000","last_change":"2023-10-21T17:39:53.497524+0000","last_active":"2023-10-21T17:39:55.509742+0000","last_peered":"2023-10-21T17:39:55.509742+0000","last_clean":"2023-10-21T17:39:55.509742+0000","last_became_active":"2023-10-21T17:39:53.496700+0000","last_became_peered":"2023-10-21T17:39:53.496700+0000","last_unstale":"2023-10-21T17:39:55.509742+0000","last_undegraded":"2023-10-21T17:39:55.509742+0000","last_fullsized":"2023-10-21T17:39:55.509742+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:11:03.008674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499044+0000","last_change":"2023-10-21T17:39:40.278307+0000","last_active":"2023-10-21T17:39:54.499044+0000","last_peered":"2023-10-21T17:39:54.499044+0000","last_clean":"2023-10-21T17:39:54.499044+0000","last_became_active":"2023-10-21T17:39:40.277780+0000","last_became_peered":"2023-10-21T17:39:40.277780+0000","last_unstale":"2023-10-21T17:39:54.499044+0000","last_undegraded":"2023-10-21T17:39:54.499044+0000","last_fullsized":"2023-10-21T17:39:54.499044+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:14.855269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869695+0000","last_change":"2023-10-21T17:39:54.516346+0000","last_active":"2023-10-21T17:39:55.869695+0000","last_peered":"2023-10-21T17:39:55.869695+0000","last_clean":"2023-10-21T17:39:55.869695+0000","last_became_active":"2023-10-21T17:39:54.514717+0000","last_became_peered":"2023-10-21T17:39:54.514717+0000","last_unstale":"2023-10-21T17:39:55.869695+0000","last_undegraded":"2023-10-21T17:39:55.869695+0000","last_fullsized":"2023-10-21T17:39:55.869695+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:03.232494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870736+0000","last_change":"2023-10-21T17:39:53.507641+0000","last_active":"2023-10-21T17:39:55.870736+0000","last_peered":"2023-10-21T17:39:55.870736+0000","last_clean":"2023-10-21T17:39:55.870736+0000","last_became_active":"2023-10-21T17:39:53.507321+0000","last_became_peered":"2023-10-21T17:39:53.507321+0000","last_unstale":"2023-10-21T17:39:55.870736+0000","last_undegraded":"2023-10-21T17:39:55.870736+0000","last_fullsized":"2023-10-21T17:39:55.870736+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:52.644827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501560+0000","last_change":"2023-10-21T17:39:40.277999+0000","last_active":"2023-10-21T17:39:54.501560+0000","last_peered":"2023-10-21T17:39:54.501560+0000","last_clean":"2023-10-21T17:39:54.501560+0000","last_became_active":"2023-10-21T17:39:40.277716+0000","last_became_peered":"2023-10-21T17:39:40.277716+0000","last_unstale":"2023-10-21T17:39:54.501560+0000","last_undegraded":"2023-10-21T17:39:54.501560+0000","last_fullsized":"2023-10-21T17:39:54.501560+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:44.306053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869044+0000","last_change":"2023-10-21T17:39:54.515865+0000","last_active":"2023-10-21T17:39:55.869044+0000","last_peered":"2023-10-21T17:39:55.869044+0000","last_clean":"2023-10-21T17:39:55.869044+0000","last_became_active":"2023-10-21T17:39:54.513264+0000","last_became_peered":"2023-10-21T17:39:54.513264+0000","last_unstale":"2023-10-21T17:39:55.869044+0000","last_undegraded":"2023-10-21T17:39:55.869044+0000","last_fullsized":"2023-10-21T17:39:55.869044+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:09.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869713+0000","last_change":"2023-10-21T17:39:53.507072+0000","last_active":"2023-10-21T17:39:55.869713+0000","last_peered":"2023-10-21T17:39:55.869713+0000","last_clean":"2023-10-21T17:39:55.869713+0000","last_became_active":"2023-10-21T17:39:53.506740+0000","last_became_peered":"2023-10-21T17:39:53.506740+0000","last_unstale":"2023-10-21T17:39:55.869713+0000","last_undegraded":"2023-10-21T17:39:55.869713+0000","last_fullsized":"2023-10-21T17:39:55.869713+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:48:02.855105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479821+0000","last_change":"2023-10-21T17:39:40.274132+0000","last_active":"2023-10-21T17:39:54.479821+0000","last_peered":"2023-10-21T17:39:54.479821+0000","last_clean":"2023-10-21T17:39:54.479821+0000","last_became_active":"2023-10-21T17:39:40.273837+0000","last_became_peered":"2023-10-21T17:39:40.273837+0000","last_unstale":"2023-10-21T17:39:54.479821+0000","last_undegraded":"2023-10-21T17:39:54.479821+0000","last_fullsized":"2023-10-21T17:39:54.479821+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:53.620703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867613+0000","last_change":"2023-10-21T17:39:54.510342+0000","last_active":"2023-10-21T17:39:55.867613+0000","last_peered":"2023-10-21T17:39:55.867613+0000","last_clean":"2023-10-21T17:39:55.867613+0000","last_became_active":"2023-10-21T17:39:54.508631+0000","last_became_peered":"2023-10-21T17:39:54.508631+0000","last_unstale":"2023-10-21T17:39:55.867613+0000","last_undegraded":"2023-10-21T17:39:55.867613+0000","last_fullsized":"2023-10-21T17:39:55.867613+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:13.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867247+0000","last_change":"2023-10-21T17:39:53.498530+0000","last_active":"2023-10-21T17:39:55.867247+0000","last_peered":"2023-10-21T17:39:55.867247+0000","last_clean":"2023-10-21T17:39:55.867247+0000","last_became_active":"2023-10-21T17:39:53.498139+0000","last_became_peered":"2023-10-21T17:39:53.498139+0000","last_unstale":"2023-10-21T17:39:55.867247+0000","last_undegraded":"2023-10-21T17:39:55.867247+0000","last_fullsized":"2023-10-21T17:39:55.867247+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:49.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500633+0000","last_change":"2023-10-21T17:39:40.277476+0000","last_active":"2023-10-21T17:39:54.500633+0000","last_peered":"2023-10-21T17:39:54.500633+0000","last_clean":"2023-10-21T17:39:54.500633+0000","last_became_active":"2023-10-21T17:39:40.277175+0000","last_became_peered":"2023-10-21T17:39:40.277175+0000","last_unstale":"2023-10-21T17:39:54.500633+0000","last_undegraded":"2023-10-21T17:39:54.500633+0000","last_fullsized":"2023-10-21T17:39:54.500633+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:09.554848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.505660+0000","last_change":"2023-10-21T17:39:54.520767+0000","last_active":"2023-10-21T17:39:55.505660+0000","last_peered":"2023-10-21T17:39:55.505660+0000","last_clean":"2023-10-21T17:39:55.505660+0000","last_became_active":"2023-10-21T17:39:54.520506+0000","last_became_peered":"2023-10-21T17:39:54.520506+0000","last_unstale":"2023-10-21T17:39:55.505660+0000","last_undegraded":"2023-10-21T17:39:55.505660+0000","last_fullsized":"2023-10-21T17:39:55.505660+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:20.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.831153+0000","last_change":"2023-10-21T17:39:53.506244+0000","last_active":"2023-10-21T17:39:55.831153+0000","last_peered":"2023-10-21T17:39:55.831153+0000","last_clean":"2023-10-21T17:39:55.831153+0000","last_became_active":"2023-10-21T17:39:53.505972+0000","last_became_peered":"2023-10-21T17:39:53.505972+0000","last_unstale":"2023-10-21T17:39:55.831153+0000","last_undegraded":"2023-10-21T17:39:55.831153+0000","last_fullsized":"2023-10-21T17:39:55.831153+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:50:40.003316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501597+0000","last_change":"2023-10-21T17:39:40.274697+0000","last_active":"2023-10-21T17:39:54.501597+0000","last_peered":"2023-10-21T17:39:54.501597+0000","last_clean":"2023-10-21T17:39:54.501597+0000","last_became_active":"2023-10-21T17:39:40.274194+0000","last_became_peered":"2023-10-21T17:39:40.274194+0000","last_unstale":"2023-10-21T17:39:54.501597+0000","last_undegraded":"2023-10-21T17:39:54.501597+0000","last_fullsized":"2023-10-21T17:39:54.501597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:17.782806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504408+0000","last_change":"2023-10-21T17:39:54.509960+0000","last_active":"2023-10-21T17:39:55.504408+0000","last_peered":"2023-10-21T17:39:55.504408+0000","last_clean":"2023-10-21T17:39:55.504408+0000","last_became_active":"2023-10-21T17:39:54.509633+0000","last_became_peered":"2023-10-21T17:39:54.509633+0000","last_unstale":"2023-10-21T17:39:55.504408+0000","last_undegraded":"2023-10-21T17:39:55.504408+0000","last_fullsized":"2023-10-21T17:39:55.504408+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:10:31.680455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.833133+0000","last_change":"2023-10-21T17:39:53.507648+0000","last_active":"2023-10-21T17:39:55.833133+0000","last_peered":"2023-10-21T17:39:55.833133+0000","last_clean":"2023-10-21T17:39:55.833133+0000","last_became_active":"2023-10-21T17:39:53.507371+0000","last_became_peered":"2023-10-21T17:39:53.507371+0000","last_unstale":"2023-10-21T17:39:55.833133+0000","last_undegraded":"2023-10-21T17:39:55.833133+0000","last_fullsized":"2023-10-21T17:39:55.833133+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:04:41.283912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484092+0000","last_change":"2023-10-21T17:39:40.278661+0000","last_active":"2023-10-21T17:39:54.484092+0000","last_peered":"2023-10-21T17:39:54.484092+0000","last_clean":"2023-10-21T17:39:54.484092+0000","last_became_active":"2023-10-21T17:39:40.278377+0000","last_became_peered":"2023-10-21T17:39:40.278377+0000","last_unstale":"2023-10-21T17:39:54.484092+0000","last_undegraded":"2023-10-21T17:39:54.484092+0000","last_fullsized":"2023-10-21T17:39:54.484092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:10:48.303305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867351+0000","last_change":"2023-10-21T17:39:54.508518+0000","last_active":"2023-10-21T17:39:55.867351+0000","last_peered":"2023-10-21T17:39:55.867351+0000","last_clean":"2023-10-21T17:39:55.867351+0000","last_became_active":"2023-10-21T17:39:54.508204+0000","last_became_peered":"2023-10-21T17:39:54.508204+0000","last_unstale":"2023-10-21T17:39:55.867351+0000","last_undegraded":"2023-10-21T17:39:55.867351+0000","last_fullsized":"2023-10-21T17:39:55.867351+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:28.194036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867988+0000","last_change":"2023-10-21T17:39:53.506000+0000","last_active":"2023-10-21T17:39:55.867988+0000","last_peered":"2023-10-21T17:39:55.867988+0000","last_clean":"2023-10-21T17:39:55.867988+0000","last_became_active":"2023-10-21T17:39:53.503681+0000","last_became_peered":"2023-10-21T17:39:53.503681+0000","last_unstale":"2023-10-21T17:39:55.867988+0000","last_undegraded":"2023-10-21T17:39:55.867988+0000","last_fullsized":"2023-10-21T17:39:55.867988+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:27:51.117136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497638+0000","last_change":"2023-10-21T17:39:40.284923+0000","last_active":"2023-10-21T17:39:54.497638+0000","last_peered":"2023-10-21T17:39:54.497638+0000","last_clean":"2023-10-21T17:39:54.497638+0000","last_became_active":"2023-10-21T17:39:40.281089+0000","last_became_peered":"2023-10-21T17:39:40.281089+0000","last_unstale":"2023-10-21T17:39:54.497638+0000","last_undegraded":"2023-10-21T17:39:54.497638+0000","last_fullsized":"2023-10-21T17:39:54.497638+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:15.736471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510430+0000","last_change":"2023-10-21T17:39:54.505970+0000","last_active":"2023-10-21T17:39:55.510430+0000","last_peered":"2023-10-21T17:39:55.510430+0000","last_clean":"2023-10-21T17:39:55.510430+0000","last_became_active":"2023-10-21T17:39:54.503538+0000","last_became_peered":"2023-10-21T17:39:54.503538+0000","last_unstale":"2023-10-21T17:39:55.510430+0000","last_undegraded":"2023-10-21T17:39:55.510430+0000","last_fullsized":"2023-10-21T17:39:55.510430+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:25:14.685731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"257'6","reported_seq":18,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504186+0000","last_change":"2023-10-21T17:39:53.491144+0000","last_active":"2023-10-21T17:39:55.504186+0000","last_peered":"2023-10-21T17:39:55.504186+0000","last_clean":"2023-10-21T17:39:55.504186+0000","last_became_active":"2023-10-21T17:39:53.490763+0000","last_became_peered":"2023-10-21T17:39:53.490763+0000","last_unstale":"2023-10-21T17:39:55.504186+0000","last_undegraded":"2023-10-21T17:39:55.504186+0000","last_fullsized":"2023-10-21T17:39:55.504186+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:42:54.016567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.495574+0000","last_change":"2023-10-21T17:39:40.280260+0000","last_active":"2023-10-21T17:39:54.495574+0000","last_peered":"2023-10-21T17:39:54.495574+0000","last_clean":"2023-10-21T17:39:54.495574+0000","last_became_active":"2023-10-21T17:39:40.279970+0000","last_became_peered":"2023-10-21T17:39:40.279970+0000","last_unstale":"2023-10-21T17:39:54.495574+0000","last_undegraded":"2023-10-21T17:39:54.495574+0000","last_fullsized":"2023-10-21T17:39:54.495574+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:09.241928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867873+0000","last_change":"2023-10-21T17:39:54.513118+0000","last_active":"2023-10-21T17:39:55.867873+0000","last_peered":"2023-10-21T17:39:55.867873+0000","last_clean":"2023-10-21T17:39:55.867873+0000","last_became_active":"2023-10-21T17:39:54.512866+0000","last_became_peered":"2023-10-21T17:39:54.512866+0000","last_unstale":"2023-10-21T17:39:55.867873+0000","last_undegraded":"2023-10-21T17:39:55.867873+0000","last_fullsized":"2023-10-21T17:39:55.867873+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:10.798745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"257'3","reported_seq":15,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866439+0000","last_change":"2023-10-21T17:39:53.506783+0000","last_active":"2023-10-21T17:39:55.866439+0000","last_peered":"2023-10-21T17:39:55.866439+0000","last_clean":"2023-10-21T17:39:55.866439+0000","last_became_active":"2023-10-21T17:39:53.503658+0000","last_became_peered":"2023-10-21T17:39:53.503658+0000","last_unstale":"2023-10-21T17:39:55.866439+0000","last_undegraded":"2023-10-21T17:39:55.866439+0000","last_fullsized":"2023-10-21T17:39:55.866439+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:15.658186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478808+0000","last_change":"2023-10-21T17:39:40.277448+0000","last_active":"2023-10-21T17:39:54.478808+0000","last_peered":"2023-10-21T17:39:54.478808+0000","last_clean":"2023-10-21T17:39:54.478808+0000","last_became_active":"2023-10-21T17:39:40.277121+0000","last_became_peered":"2023-10-21T17:39:40.277121+0000","last_unstale":"2023-10-21T17:39:54.478808+0000","last_undegraded":"2023-10-21T17:39:54.478808+0000","last_fullsized":"2023-10-21T17:39:54.478808+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:11.317461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510152+0000","last_change":"2023-10-21T17:39:54.508131+0000","last_active":"2023-10-21T17:39:55.510152+0000","last_peered":"2023-10-21T17:39:55.510152+0000","last_clean":"2023-10-21T17:39:55.510152+0000","last_became_active":"2023-10-21T17:39:54.507875+0000","last_became_peered":"2023-10-21T17:39:54.507875+0000","last_unstale":"2023-10-21T17:39:55.510152+0000","last_undegraded":"2023-10-21T17:39:55.510152+0000","last_fullsized":"2023-10-21T17:39:55.510152+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:51.425048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866450+0000","last_change":"2023-10-21T17:39:53.503802+0000","last_active":"2023-10-21T17:39:55.866450+0000","last_peered":"2023-10-21T17:39:55.866450+0000","last_clean":"2023-10-21T17:39:55.866450+0000","last_became_active":"2023-10-21T17:39:53.503435+0000","last_became_peered":"2023-10-21T17:39:53.503435+0000","last_unstale":"2023-10-21T17:39:55.866450+0000","last_undegraded":"2023-10-21T17:39:55.866450+0000","last_fullsized":"2023-10-21T17:39:55.866450+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:18:13.675672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483318+0000","last_change":"2023-10-21T17:39:40.279922+0000","last_active":"2023-10-21T17:39:54.483318+0000","last_peered":"2023-10-21T17:39:54.483318+0000","last_clean":"2023-10-21T17:39:54.483318+0000","last_became_active":"2023-10-21T17:39:40.279029+0000","last_became_peered":"2023-10-21T17:39:40.279029+0000","last_unstale":"2023-10-21T17:39:54.483318+0000","last_undegraded":"2023-10-21T17:39:54.483318+0000","last_fullsized":"2023-10-21T17:39:54.483318+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:13.093898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867229+0000","last_change":"2023-10-21T17:39:54.509200+0000","last_active":"2023-10-21T17:39:55.867229+0000","last_peered":"2023-10-21T17:39:55.867229+0000","last_clean":"2023-10-21T17:39:55.867229+0000","last_became_active":"2023-10-21T17:39:54.508841+0000","last_became_peered":"2023-10-21T17:39:54.508841+0000","last_unstale":"2023-10-21T17:39:55.867229+0000","last_undegraded":"2023-10-21T17:39:55.867229+0000","last_fullsized":"2023-10-21T17:39:55.867229+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:23.840619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869197+0000","last_change":"2023-10-21T17:39:53.505087+0000","last_active":"2023-10-21T17:39:55.869197+0000","last_peered":"2023-10-21T17:39:55.869197+0000","last_clean":"2023-10-21T17:39:55.869197+0000","last_became_active":"2023-10-21T17:39:53.504790+0000","last_became_peered":"2023-10-21T17:39:53.504790+0000","last_unstale":"2023-10-21T17:39:55.869197+0000","last_undegraded":"2023-10-21T17:39:55.869197+0000","last_fullsized":"2023-10-21T17:39:55.869197+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:52.644827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.508959+0000","last_change":"2023-10-21T17:39:40.260107+0000","last_active":"2023-10-21T17:39:54.508959+0000","last_peered":"2023-10-21T17:39:54.508959+0000","last_clean":"2023-10-21T17:39:54.508959+0000","last_became_active":"2023-10-21T17:39:40.259164+0000","last_became_peered":"2023-10-21T17:39:40.259164+0000","last_unstale":"2023-10-21T17:39:54.508959+0000","last_undegraded":"2023-10-21T17:39:54.508959+0000","last_fullsized":"2023-10-21T17:39:54.508959+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:53.185535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869024+0000","last_change":"2023-10-21T17:39:54.515983+0000","last_active":"2023-10-21T17:39:55.869024+0000","last_peered":"2023-10-21T17:39:55.869024+0000","last_clean":"2023-10-21T17:39:55.869024+0000","last_became_active":"2023-10-21T17:39:54.515693+0000","last_became_peered":"2023-10-21T17:39:54.515693+0000","last_unstale":"2023-10-21T17:39:55.869024+0000","last_undegraded":"2023-10-21T17:39:55.869024+0000","last_fullsized":"2023-10-21T17:39:55.869024+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:26.912523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"257'8","reported_seq":20,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865395+0000","last_change":"2023-10-21T17:39:53.505646+0000","last_active":"2023-10-21T17:39:55.865395+0000","last_peered":"2023-10-21T17:39:55.865395+0000","last_clean":"2023-10-21T17:39:55.865395+0000","last_became_active":"2023-10-21T17:39:53.500640+0000","last_became_peered":"2023-10-21T17:39:53.500640+0000","last_unstale":"2023-10-21T17:39:55.865395+0000","last_undegraded":"2023-10-21T17:39:55.865395+0000","last_fullsized":"2023-10-21T17:39:55.865395+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:17.968217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497608+0000","last_change":"2023-10-21T17:39:40.278320+0000","last_active":"2023-10-21T17:39:54.497608+0000","last_peered":"2023-10-21T17:39:54.497608+0000","last_clean":"2023-10-21T17:39:54.497608+0000","last_became_active":"2023-10-21T17:39:40.277166+0000","last_became_peered":"2023-10-21T17:39:40.277166+0000","last_unstale":"2023-10-21T17:39:54.497608+0000","last_undegraded":"2023-10-21T17:39:54.497608+0000","last_fullsized":"2023-10-21T17:39:54.497608+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:18:25.386838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867886+0000","last_change":"2023-10-21T17:39:54.514722+0000","last_active":"2023-10-21T17:39:55.867886+0000","last_peered":"2023-10-21T17:39:55.867886+0000","last_clean":"2023-10-21T17:39:55.867886+0000","last_became_active":"2023-10-21T17:39:54.513397+0000","last_became_peered":"2023-10-21T17:39:54.513397+0000","last_unstale":"2023-10-21T17:39:55.867886+0000","last_undegraded":"2023-10-21T17:39:55.867886+0000","last_fullsized":"2023-10-21T17:39:55.867886+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:03.232494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866216+0000","last_change":"2023-10-21T17:39:53.497481+0000","last_active":"2023-10-21T17:39:55.866216+0000","last_peered":"2023-10-21T17:39:55.866216+0000","last_clean":"2023-10-21T17:39:55.866216+0000","last_became_active":"2023-10-21T17:39:53.494746+0000","last_became_peered":"2023-10-21T17:39:53.494746+0000","last_unstale":"2023-10-21T17:39:55.866216+0000","last_undegraded":"2023-10-21T17:39:55.866216+0000","last_fullsized":"2023-10-21T17:39:55.866216+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:47:45.315502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496336+0000","last_change":"2023-10-21T17:39:40.285094+0000","last_active":"2023-10-21T17:39:54.496336+0000","last_peered":"2023-10-21T17:39:54.496336+0000","last_clean":"2023-10-21T17:39:54.496336+0000","last_became_active":"2023-10-21T17:39:40.284749+0000","last_became_peered":"2023-10-21T17:39:40.284749+0000","last_unstale":"2023-10-21T17:39:54.496336+0000","last_undegraded":"2023-10-21T17:39:54.496336+0000","last_fullsized":"2023-10-21T17:39:54.496336+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:32.664957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509769+0000","last_change":"2023-10-21T17:39:54.514819+0000","last_active":"2023-10-21T17:39:55.509769+0000","last_peered":"2023-10-21T17:39:55.509769+0000","last_clean":"2023-10-21T17:39:55.509769+0000","last_became_active":"2023-10-21T17:39:54.514490+0000","last_became_peered":"2023-10-21T17:39:54.514490+0000","last_unstale":"2023-10-21T17:39:55.509769+0000","last_undegraded":"2023-10-21T17:39:55.509769+0000","last_fullsized":"2023-10-21T17:39:55.509769+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:58.557859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504691+0000","last_change":"2023-10-21T17:39:53.507652+0000","last_active":"2023-10-21T17:39:55.504691+0000","last_peered":"2023-10-21T17:39:55.504691+0000","last_clean":"2023-10-21T17:39:55.504691+0000","last_became_active":"2023-10-21T17:39:53.507367+0000","last_became_peered":"2023-10-21T17:39:53.507367+0000","last_unstale":"2023-10-21T17:39:55.504691+0000","last_undegraded":"2023-10-21T17:39:55.504691+0000","last_fullsized":"2023-10-21T17:39:55.504691+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:17:59.353833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485435+0000","last_change":"2023-10-21T17:39:40.283462+0000","last_active":"2023-10-21T17:39:54.485435+0000","last_peered":"2023-10-21T17:39:54.485435+0000","last_clean":"2023-10-21T17:39:54.485435+0000","last_became_active":"2023-10-21T17:39:40.281926+0000","last_became_peered":"2023-10-21T17:39:40.281926+0000","last_unstale":"2023-10-21T17:39:54.485435+0000","last_undegraded":"2023-10-21T17:39:54.485435+0000","last_fullsized":"2023-10-21T17:39:54.485435+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:42:34.391024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508310+0000","last_change":"2023-10-21T17:39:54.510345+0000","last_active":"2023-10-21T17:39:55.508310+0000","last_peered":"2023-10-21T17:39:55.508310+0000","last_clean":"2023-10-21T17:39:55.508310+0000","last_became_active":"2023-10-21T17:39:54.510078+0000","last_became_peered":"2023-10-21T17:39:54.510078+0000","last_unstale":"2023-10-21T17:39:55.508310+0000","last_undegraded":"2023-10-21T17:39:55.508310+0000","last_fullsized":"2023-10-21T17:39:55.508310+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:22.489131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868163+0000","last_change":"2023-10-21T17:39:53.510589+0000","last_active":"2023-10-21T17:39:55.868163+0000","last_peered":"2023-10-21T17:39:55.868163+0000","last_clean":"2023-10-21T17:39:55.868163+0000","last_became_active":"2023-10-21T17:39:53.510298+0000","last_became_peered":"2023-10-21T17:39:53.510298+0000","last_unstale":"2023-10-21T17:39:55.868163+0000","last_undegraded":"2023-10-21T17:39:55.868163+0000","last_fullsized":"2023-10-21T17:39:55.868163+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:19.793960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497679+0000","last_change":"2023-10-21T17:39:40.280739+0000","last_active":"2023-10-21T17:39:54.497679+0000","last_peered":"2023-10-21T17:39:54.497679+0000","last_clean":"2023-10-21T17:39:54.497679+0000","last_became_active":"2023-10-21T17:39:40.280069+0000","last_became_peered":"2023-10-21T17:39:40.280069+0000","last_unstale":"2023-10-21T17:39:54.497679+0000","last_undegraded":"2023-10-21T17:39:54.497679+0000","last_fullsized":"2023-10-21T17:39:54.497679+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:19.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510862+0000","last_change":"2023-10-21T17:39:54.523357+0000","last_active":"2023-10-21T17:39:55.510862+0000","last_peered":"2023-10-21T17:39:55.510862+0000","last_clean":"2023-10-21T17:39:55.510862+0000","last_became_active":"2023-10-21T17:39:54.523016+0000","last_became_peered":"2023-10-21T17:39:54.523016+0000","last_unstale":"2023-10-21T17:39:55.510862+0000","last_undegraded":"2023-10-21T17:39:55.510862+0000","last_fullsized":"2023-10-21T17:39:55.510862+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:46.615117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504189+0000","last_change":"2023-10-21T17:39:53.507060+0000","last_active":"2023-10-21T17:39:55.504189+0000","last_peered":"2023-10-21T17:39:55.504189+0000","last_clean":"2023-10-21T17:39:55.504189+0000","last_became_active":"2023-10-21T17:39:53.506751+0000","last_became_peered":"2023-10-21T17:39:53.506751+0000","last_unstale":"2023-10-21T17:39:55.504189+0000","last_undegraded":"2023-10-21T17:39:55.504189+0000","last_fullsized":"2023-10-21T17:39:55.504189+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:24.020377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.505125+0000","last_change":"2023-10-21T17:39:40.285848+0000","last_active":"2023-10-21T17:39:54.505125+0000","last_peered":"2023-10-21T17:39:54.505125+0000","last_clean":"2023-10-21T17:39:54.505125+0000","last_became_active":"2023-10-21T17:39:40.285405+0000","last_became_peered":"2023-10-21T17:39:40.285405+0000","last_unstale":"2023-10-21T17:39:54.505125+0000","last_undegraded":"2023-10-21T17:39:54.505125+0000","last_fullsized":"2023-10-21T17:39:54.505125+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868777+0000","last_change":"2023-10-21T17:39:54.491851+0000","last_active":"2023-10-21T17:39:55.868777+0000","last_peered":"2023-10-21T17:39:55.868777+0000","last_clean":"2023-10-21T17:39:55.868777+0000","last_became_active":"2023-10-21T17:39:54.491379+0000","last_became_peered":"2023-10-21T17:39:54.491379+0000","last_unstale":"2023-10-21T17:39:55.868777+0000","last_undegraded":"2023-10-21T17:39:55.868777+0000","last_fullsized":"2023-10-21T17:39:55.868777+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:26:31.568634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506954+0000","last_change":"2023-10-21T17:39:53.502790+0000","last_active":"2023-10-21T17:39:55.506954+0000","last_peered":"2023-10-21T17:39:55.506954+0000","last_clean":"2023-10-21T17:39:55.506954+0000","last_became_active":"2023-10-21T17:39:53.500918+0000","last_became_peered":"2023-10-21T17:39:53.500918+0000","last_unstale":"2023-10-21T17:39:55.506954+0000","last_undegraded":"2023-10-21T17:39:55.506954+0000","last_fullsized":"2023-10-21T17:39:55.506954+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:19.527820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484553+0000","last_change":"2023-10-21T17:39:40.284648+0000","last_active":"2023-10-21T17:39:54.484553+0000","last_peered":"2023-10-21T17:39:54.484553+0000","last_clean":"2023-10-21T17:39:54.484553+0000","last_became_active":"2023-10-21T17:39:40.284214+0000","last_became_peered":"2023-10-21T17:39:40.284214+0000","last_unstale":"2023-10-21T17:39:54.484553+0000","last_undegraded":"2023-10-21T17:39:54.484553+0000","last_fullsized":"2023-10-21T17:39:54.484553+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:52.495370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.507313+0000","last_change":"2023-10-21T17:39:54.495077+0000","last_active":"2023-10-21T17:39:55.507313+0000","last_peered":"2023-10-21T17:39:55.507313+0000","last_clean":"2023-10-21T17:39:55.507313+0000","last_became_active":"2023-10-21T17:39:54.494437+0000","last_became_peered":"2023-10-21T17:39:54.494437+0000","last_unstale":"2023-10-21T17:39:55.507313+0000","last_undegraded":"2023-10-21T17:39:55.507313+0000","last_fullsized":"2023-10-21T17:39:55.507313+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:52.634139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868130+0000","last_change":"2023-10-21T17:39:53.503144+0000","last_active":"2023-10-21T17:39:55.868130+0000","last_peered":"2023-10-21T17:39:55.868130+0000","last_clean":"2023-10-21T17:39:55.868130+0000","last_became_active":"2023-10-21T17:39:53.502799+0000","last_became_peered":"2023-10-21T17:39:53.502799+0000","last_unstale":"2023-10-21T17:39:55.868130+0000","last_undegraded":"2023-10-21T17:39:55.868130+0000","last_fullsized":"2023-10-21T17:39:55.868130+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:36.731729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504174+0000","last_change":"2023-10-21T17:39:40.270709+0000","last_active":"2023-10-21T17:39:54.504174+0000","last_peered":"2023-10-21T17:39:54.504174+0000","last_clean":"2023-10-21T17:39:54.504174+0000","last_became_active":"2023-10-21T17:39:40.270369+0000","last_became_peered":"2023-10-21T17:39:40.270369+0000","last_unstale":"2023-10-21T17:39:54.504174+0000","last_undegraded":"2023-10-21T17:39:54.504174+0000","last_fullsized":"2023-10-21T17:39:54.504174+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869379+0000","last_change":"2023-10-21T17:39:54.512950+0000","last_active":"2023-10-21T17:39:55.869379+0000","last_peered":"2023-10-21T17:39:55.869379+0000","last_clean":"2023-10-21T17:39:55.869379+0000","last_became_active":"2023-10-21T17:39:54.511870+0000","last_became_peered":"2023-10-21T17:39:54.511870+0000","last_unstale":"2023-10-21T17:39:55.869379+0000","last_undegraded":"2023-10-21T17:39:55.869379+0000","last_fullsized":"2023-10-21T17:39:55.869379+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:23.840619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869408+0000","last_change":"2023-10-21T17:39:53.510949+0000","last_active":"2023-10-21T17:39:55.869408+0000","last_peered":"2023-10-21T17:39:55.869408+0000","last_clean":"2023-10-21T17:39:55.869408+0000","last_became_active":"2023-10-21T17:39:53.510623+0000","last_became_peered":"2023-10-21T17:39:53.510623+0000","last_unstale":"2023-10-21T17:39:55.869408+0000","last_undegraded":"2023-10-21T17:39:55.869408+0000","last_fullsized":"2023-10-21T17:39:55.869408+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:48.160505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483153+0000","last_change":"2023-10-21T17:39:40.267480+0000","last_active":"2023-10-21T17:39:54.483153+0000","last_peered":"2023-10-21T17:39:54.483153+0000","last_clean":"2023-10-21T17:39:54.483153+0000","last_became_active":"2023-10-21T17:39:40.267074+0000","last_became_peered":"2023-10-21T17:39:40.267074+0000","last_unstale":"2023-10-21T17:39:54.483153+0000","last_undegraded":"2023-10-21T17:39:54.483153+0000","last_fullsized":"2023-10-21T17:39:54.483153+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:05.724202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870560+0000","last_change":"2023-10-21T17:39:54.506708+0000","last_active":"2023-10-21T17:39:55.870560+0000","last_peered":"2023-10-21T17:39:55.870560+0000","last_clean":"2023-10-21T17:39:55.870560+0000","last_became_active":"2023-10-21T17:39:54.506421+0000","last_became_peered":"2023-10-21T17:39:54.506421+0000","last_unstale":"2023-10-21T17:39:55.870560+0000","last_undegraded":"2023-10-21T17:39:55.870560+0000","last_fullsized":"2023-10-21T17:39:55.870560+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:33.273447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865500+0000","last_change":"2023-10-21T17:39:53.493647+0000","last_active":"2023-10-21T17:39:55.865500+0000","last_peered":"2023-10-21T17:39:55.865500+0000","last_clean":"2023-10-21T17:39:55.865500+0000","last_became_active":"2023-10-21T17:39:53.493328+0000","last_became_peered":"2023-10-21T17:39:53.493328+0000","last_unstale":"2023-10-21T17:39:55.865500+0000","last_undegraded":"2023-10-21T17:39:55.865500+0000","last_fullsized":"2023-10-21T17:39:55.865500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:19:28.838008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497725+0000","last_change":"2023-10-21T17:39:40.287294+0000","last_active":"2023-10-21T17:39:54.497725+0000","last_peered":"2023-10-21T17:39:54.497725+0000","last_clean":"2023-10-21T17:39:54.497725+0000","last_became_active":"2023-10-21T17:39:40.286990+0000","last_became_peered":"2023-10-21T17:39:40.286990+0000","last_unstale":"2023-10-21T17:39:54.497725+0000","last_undegraded":"2023-10-21T17:39:54.497725+0000","last_fullsized":"2023-10-21T17:39:54.497725+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:39.411032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866301+0000","last_change":"2023-10-21T17:39:54.508541+0000","last_active":"2023-10-21T17:39:55.866301+0000","last_peered":"2023-10-21T17:39:55.866301+0000","last_clean":"2023-10-21T17:39:55.866301+0000","last_became_active":"2023-10-21T17:39:54.508263+0000","last_became_peered":"2023-10-21T17:39:54.508263+0000","last_unstale":"2023-10-21T17:39:55.866301+0000","last_undegraded":"2023-10-21T17:39:55.866301+0000","last_fullsized":"2023-10-21T17:39:55.866301+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:40.233750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869943+0000","last_change":"2023-10-21T17:39:53.500685+0000","last_active":"2023-10-21T17:39:55.869943+0000","last_peered":"2023-10-21T17:39:55.869943+0000","last_clean":"2023-10-21T17:39:55.869943+0000","last_became_active":"2023-10-21T17:39:53.500261+0000","last_became_peered":"2023-10-21T17:39:53.500261+0000","last_unstale":"2023-10-21T17:39:55.869943+0000","last_undegraded":"2023-10-21T17:39:55.869943+0000","last_fullsized":"2023-10-21T17:39:55.869943+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:48.160505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482374+0000","last_change":"2023-10-21T17:39:40.270092+0000","last_active":"2023-10-21T17:39:54.482374+0000","last_peered":"2023-10-21T17:39:54.482374+0000","last_clean":"2023-10-21T17:39:54.482374+0000","last_became_active":"2023-10-21T17:39:40.269725+0000","last_became_peered":"2023-10-21T17:39:40.269725+0000","last_unstale":"2023-10-21T17:39:54.482374+0000","last_undegraded":"2023-10-21T17:39:54.482374+0000","last_fullsized":"2023-10-21T17:39:54.482374+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:18.601428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867545+0000","last_change":"2023-10-21T17:39:54.515960+0000","last_active":"2023-10-21T17:39:55.867545+0000","last_peered":"2023-10-21T17:39:55.867545+0000","last_clean":"2023-10-21T17:39:55.867545+0000","last_became_active":"2023-10-21T17:39:54.515687+0000","last_became_peered":"2023-10-21T17:39:54.515687+0000","last_unstale":"2023-10-21T17:39:55.867545+0000","last_undegraded":"2023-10-21T17:39:55.867545+0000","last_fullsized":"2023-10-21T17:39:55.867545+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:31.004177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504911+0000","last_change":"2023-10-21T17:39:53.505990+0000","last_active":"2023-10-21T17:39:55.504911+0000","last_peered":"2023-10-21T17:39:55.504911+0000","last_clean":"2023-10-21T17:39:55.504911+0000","last_became_active":"2023-10-21T17:39:53.505715+0000","last_became_peered":"2023-10-21T17:39:53.505715+0000","last_unstale":"2023-10-21T17:39:55.504911+0000","last_undegraded":"2023-10-21T17:39:55.504911+0000","last_fullsized":"2023-10-21T17:39:55.504911+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:27:38.955413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.486010+0000","last_change":"2023-10-21T17:39:40.275179+0000","last_active":"2023-10-21T17:39:54.486010+0000","last_peered":"2023-10-21T17:39:54.486010+0000","last_clean":"2023-10-21T17:39:54.486010+0000","last_became_active":"2023-10-21T17:39:40.274025+0000","last_became_peered":"2023-10-21T17:39:40.274025+0000","last_unstale":"2023-10-21T17:39:54.486010+0000","last_undegraded":"2023-10-21T17:39:54.486010+0000","last_fullsized":"2023-10-21T17:39:54.486010+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:58.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867987+0000","last_change":"2023-10-21T17:39:54.516243+0000","last_active":"2023-10-21T17:39:55.867987+0000","last_peered":"2023-10-21T17:39:55.867987+0000","last_clean":"2023-10-21T17:39:55.867987+0000","last_became_active":"2023-10-21T17:39:54.515934+0000","last_became_peered":"2023-10-21T17:39:54.515934+0000","last_unstale":"2023-10-21T17:39:55.867987+0000","last_undegraded":"2023-10-21T17:39:55.867987+0000","last_fullsized":"2023-10-21T17:39:55.867987+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:08.560941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868091+0000","last_change":"2023-10-21T17:39:53.508492+0000","last_active":"2023-10-21T17:39:55.868091+0000","last_peered":"2023-10-21T17:39:55.868091+0000","last_clean":"2023-10-21T17:39:55.868091+0000","last_became_active":"2023-10-21T17:39:53.508074+0000","last_became_peered":"2023-10-21T17:39:53.508074+0000","last_unstale":"2023-10-21T17:39:55.868091+0000","last_undegraded":"2023-10-21T17:39:55.868091+0000","last_fullsized":"2023-10-21T17:39:55.868091+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:22.348143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503230+0000","last_change":"2023-10-21T17:39:40.271219+0000","last_active":"2023-10-21T17:39:54.503230+0000","last_peered":"2023-10-21T17:39:54.503230+0000","last_clean":"2023-10-21T17:39:54.503230+0000","last_became_active":"2023-10-21T17:39:40.270928+0000","last_became_peered":"2023-10-21T17:39:40.270928+0000","last_unstale":"2023-10-21T17:39:54.503230+0000","last_undegraded":"2023-10-21T17:39:54.503230+0000","last_fullsized":"2023-10-21T17:39:54.503230+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868121+0000","last_change":"2023-10-21T17:39:54.515451+0000","last_active":"2023-10-21T17:39:55.868121+0000","last_peered":"2023-10-21T17:39:55.868121+0000","last_clean":"2023-10-21T17:39:55.868121+0000","last_became_active":"2023-10-21T17:39:54.514828+0000","last_became_peered":"2023-10-21T17:39:54.514828+0000","last_unstale":"2023-10-21T17:39:55.868121+0000","last_undegraded":"2023-10-21T17:39:55.868121+0000","last_fullsized":"2023-10-21T17:39:55.868121+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:55.218745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506188+0000","last_change":"2023-10-21T17:39:53.500862+0000","last_active":"2023-10-21T17:39:55.506188+0000","last_peered":"2023-10-21T17:39:55.506188+0000","last_clean":"2023-10-21T17:39:55.506188+0000","last_became_active":"2023-10-21T17:39:53.500544+0000","last_became_peered":"2023-10-21T17:39:53.500544+0000","last_unstale":"2023-10-21T17:39:55.506188+0000","last_undegraded":"2023-10-21T17:39:55.506188+0000","last_fullsized":"2023-10-21T17:39:55.506188+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:54.638360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.507490+0000","last_change":"2023-10-21T17:39:40.276685+0000","last_active":"2023-10-21T17:39:54.507490+0000","last_peered":"2023-10-21T17:39:54.507490+0000","last_clean":"2023-10-21T17:39:54.507490+0000","last_became_active":"2023-10-21T17:39:40.276416+0000","last_became_peered":"2023-10-21T17:39:40.276416+0000","last_unstale":"2023-10-21T17:39:54.507490+0000","last_undegraded":"2023-10-21T17:39:54.507490+0000","last_fullsized":"2023-10-21T17:39:54.507490+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:29.584375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.507401+0000","last_change":"2023-10-21T17:39:54.512213+0000","last_active":"2023-10-21T17:39:55.507401+0000","last_peered":"2023-10-21T17:39:55.507401+0000","last_clean":"2023-10-21T17:39:55.507401+0000","last_became_active":"2023-10-21T17:39:54.509442+0000","last_became_peered":"2023-10-21T17:39:54.509442+0000","last_unstale":"2023-10-21T17:39:55.507401+0000","last_undegraded":"2023-10-21T17:39:55.507401+0000","last_fullsized":"2023-10-21T17:39:55.507401+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:31:48.847242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506180+0000","last_change":"2023-10-21T17:39:53.489911+0000","last_active":"2023-10-21T17:39:55.506180+0000","last_peered":"2023-10-21T17:39:55.506180+0000","last_clean":"2023-10-21T17:39:55.506180+0000","last_became_active":"2023-10-21T17:39:53.489592+0000","last_became_peered":"2023-10-21T17:39:53.489592+0000","last_unstale":"2023-10-21T17:39:55.506180+0000","last_undegraded":"2023-10-21T17:39:55.506180+0000","last_fullsized":"2023-10-21T17:39:55.506180+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:32.255435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479090+0000","last_change":"2023-10-21T17:39:40.284175+0000","last_active":"2023-10-21T17:39:54.479090+0000","last_peered":"2023-10-21T17:39:54.479090+0000","last_clean":"2023-10-21T17:39:54.479090+0000","last_became_active":"2023-10-21T17:39:40.283773+0000","last_became_peered":"2023-10-21T17:39:40.283773+0000","last_unstale":"2023-10-21T17:39:54.479090+0000","last_undegraded":"2023-10-21T17:39:54.479090+0000","last_fullsized":"2023-10-21T17:39:54.479090+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:23:01.051661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869801+0000","last_change":"2023-10-21T17:39:54.512287+0000","last_active":"2023-10-21T17:39:55.869801+0000","last_peered":"2023-10-21T17:39:55.869801+0000","last_clean":"2023-10-21T17:39:55.869801+0000","last_became_active":"2023-10-21T17:39:54.506913+0000","last_became_peered":"2023-10-21T17:39:54.506913+0000","last_unstale":"2023-10-21T17:39:55.869801+0000","last_undegraded":"2023-10-21T17:39:55.869801+0000","last_fullsized":"2023-10-21T17:39:55.869801+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:26.912523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.503730+0000","last_change":"2023-10-21T17:39:53.497849+0000","last_active":"2023-10-21T17:39:55.503730+0000","last_peered":"2023-10-21T17:39:55.503730+0000","last_clean":"2023-10-21T17:39:55.503730+0000","last_became_active":"2023-10-21T17:39:53.497539+0000","last_became_peered":"2023-10-21T17:39:53.497539+0000","last_unstale":"2023-10-21T17:39:55.503730+0000","last_undegraded":"2023-10-21T17:39:55.503730+0000","last_fullsized":"2023-10-21T17:39:55.503730+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:17.368029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.506981+0000","last_change":"2023-10-21T17:39:40.258467+0000","last_active":"2023-10-21T17:39:54.506981+0000","last_peered":"2023-10-21T17:39:54.506981+0000","last_clean":"2023-10-21T17:39:54.506981+0000","last_became_active":"2023-10-21T17:39:40.257660+0000","last_became_peered":"2023-10-21T17:39:40.257660+0000","last_unstale":"2023-10-21T17:39:54.506981+0000","last_undegraded":"2023-10-21T17:39:54.506981+0000","last_fullsized":"2023-10-21T17:39:54.506981+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:03:12.244419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865600+0000","last_change":"2023-10-21T17:39:54.501955+0000","last_active":"2023-10-21T17:39:55.865600+0000","last_peered":"2023-10-21T17:39:55.865600+0000","last_clean":"2023-10-21T17:39:55.865600+0000","last_became_active":"2023-10-21T17:39:54.501672+0000","last_became_peered":"2023-10-21T17:39:54.501672+0000","last_unstale":"2023-10-21T17:39:55.865600+0000","last_undegraded":"2023-10-21T17:39:55.865600+0000","last_fullsized":"2023-10-21T17:39:55.865600+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:45.216861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504103+0000","last_change":"2023-10-21T17:39:53.504406+0000","last_active":"2023-10-21T17:39:55.504103+0000","last_peered":"2023-10-21T17:39:55.504103+0000","last_clean":"2023-10-21T17:39:55.504103+0000","last_became_active":"2023-10-21T17:39:53.504097+0000","last_became_peered":"2023-10-21T17:39:53.504097+0000","last_unstale":"2023-10-21T17:39:55.504103+0000","last_undegraded":"2023-10-21T17:39:55.504103+0000","last_fullsized":"2023-10-21T17:39:55.504103+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:51.597031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509057+0000","last_change":"2023-10-21T17:39:40.276636+0000","last_active":"2023-10-21T17:39:54.509057+0000","last_peered":"2023-10-21T17:39:54.509057+0000","last_clean":"2023-10-21T17:39:54.509057+0000","last_became_active":"2023-10-21T17:39:40.274303+0000","last_became_peered":"2023-10-21T17:39:40.274303+0000","last_unstale":"2023-10-21T17:39:54.509057+0000","last_undegraded":"2023-10-21T17:39:54.509057+0000","last_fullsized":"2023-10-21T17:39:54.509057+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:45.096371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832643+0000","last_change":"2023-10-21T17:39:54.511696+0000","last_active":"2023-10-21T17:39:55.832643+0000","last_peered":"2023-10-21T17:39:55.832643+0000","last_clean":"2023-10-21T17:39:55.832643+0000","last_became_active":"2023-10-21T17:39:54.511178+0000","last_became_peered":"2023-10-21T17:39:54.511178+0000","last_unstale":"2023-10-21T17:39:55.832643+0000","last_undegraded":"2023-10-21T17:39:55.832643+0000","last_fullsized":"2023-10-21T17:39:55.832643+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:00.085614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.503566+0000","last_change":"2023-10-21T17:39:53.500780+0000","last_active":"2023-10-21T17:39:55.503566+0000","last_peered":"2023-10-21T17:39:55.503566+0000","last_clean":"2023-10-21T17:39:55.503566+0000","last_became_active":"2023-10-21T17:39:53.500473+0000","last_became_peered":"2023-10-21T17:39:53.500473+0000","last_unstale":"2023-10-21T17:39:55.503566+0000","last_undegraded":"2023-10-21T17:39:55.503566+0000","last_fullsized":"2023-10-21T17:39:55.503566+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:37.713072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478509+0000","last_change":"2023-10-21T17:39:40.270012+0000","last_active":"2023-10-21T17:39:54.478509+0000","last_peered":"2023-10-21T17:39:54.478509+0000","last_clean":"2023-10-21T17:39:54.478509+0000","last_became_active":"2023-10-21T17:39:40.269726+0000","last_became_peered":"2023-10-21T17:39:40.269726+0000","last_unstale":"2023-10-21T17:39:54.478509+0000","last_undegraded":"2023-10-21T17:39:54.478509+0000","last_fullsized":"2023-10-21T17:39:54.478509+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:57.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870487+0000","last_change":"2023-10-21T17:39:54.520704+0000","last_active":"2023-10-21T17:39:55.870487+0000","last_peered":"2023-10-21T17:39:55.870487+0000","last_clean":"2023-10-21T17:39:55.870487+0000","last_became_active":"2023-10-21T17:39:54.519741+0000","last_became_peered":"2023-10-21T17:39:54.519741+0000","last_unstale":"2023-10-21T17:39:55.870487+0000","last_undegraded":"2023-10-21T17:39:55.870487+0000","last_fullsized":"2023-10-21T17:39:55.870487+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:45.733225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.505019+0000","last_change":"2023-10-21T17:39:53.493859+0000","last_active":"2023-10-21T17:39:55.505019+0000","last_peered":"2023-10-21T17:39:55.505019+0000","last_clean":"2023-10-21T17:39:55.505019+0000","last_became_active":"2023-10-21T17:39:53.493248+0000","last_became_peered":"2023-10-21T17:39:53.493248+0000","last_unstale":"2023-10-21T17:39:55.505019+0000","last_undegraded":"2023-10-21T17:39:55.505019+0000","last_fullsized":"2023-10-21T17:39:55.505019+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:44.715214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.477800+0000","last_change":"2023-10-21T17:39:40.284346+0000","last_active":"2023-10-21T17:39:54.477800+0000","last_peered":"2023-10-21T17:39:54.477800+0000","last_clean":"2023-10-21T17:39:54.477800+0000","last_became_active":"2023-10-21T17:39:40.282055+0000","last_became_peered":"2023-10-21T17:39:40.282055+0000","last_unstale":"2023-10-21T17:39:54.477800+0000","last_undegraded":"2023-10-21T17:39:54.477800+0000","last_fullsized":"2023-10-21T17:39:54.477800+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:33.876859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866645+0000","last_change":"2023-10-21T17:39:54.513870+0000","last_active":"2023-10-21T17:39:55.866645+0000","last_peered":"2023-10-21T17:39:55.866645+0000","last_clean":"2023-10-21T17:39:55.866645+0000","last_became_active":"2023-10-21T17:39:54.513631+0000","last_became_peered":"2023-10-21T17:39:54.513631+0000","last_unstale":"2023-10-21T17:39:55.866645+0000","last_undegraded":"2023-10-21T17:39:55.866645+0000","last_fullsized":"2023-10-21T17:39:55.866645+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:31:48.847242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.480877+0000","last_change":"2023-10-21T17:39:40.265515+0000","last_active":"2023-10-21T17:39:54.480877+0000","last_peered":"2023-10-21T17:39:54.480877+0000","last_clean":"2023-10-21T17:39:54.480877+0000","last_became_active":"2023-10-21T17:39:40.264706+0000","last_became_peered":"2023-10-21T17:39:40.264706+0000","last_unstale":"2023-10-21T17:39:54.480877+0000","last_undegraded":"2023-10-21T17:39:54.480877+0000","last_fullsized":"2023-10-21T17:39:54.480877+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:19.664508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483840+0000","last_change":"2023-10-21T17:39:39.225724+0000","last_active":"2023-10-21T17:39:54.483840+0000","last_peered":"2023-10-21T17:39:54.483840+0000","last_clean":"2023-10-21T17:39:54.483840+0000","last_became_active":"2023-10-21T17:39:39.223965+0000","last_became_peered":"2023-10-21T17:39:39.223965+0000","last_unstale":"2023-10-21T17:39:54.483840+0000","last_undegraded":"2023-10-21T17:39:54.483840+0000","last_fullsized":"2023-10-21T17:39:54.483840+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:53.712155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868661+0000","last_change":"2023-10-21T17:39:54.502119+0000","last_active":"2023-10-21T17:39:55.868661+0000","last_peered":"2023-10-21T17:39:55.868661+0000","last_clean":"2023-10-21T17:39:55.868661+0000","last_became_active":"2023-10-21T17:39:54.501820+0000","last_became_peered":"2023-10-21T17:39:54.501820+0000","last_unstale":"2023-10-21T17:39:55.868661+0000","last_undegraded":"2023-10-21T17:39:55.868661+0000","last_fullsized":"2023-10-21T17:39:55.868661+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:37.749741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502544+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:54.502544+0000","last_peered":"2023-10-21T17:39:54.502544+0000","last_clean":"2023-10-21T17:39:54.502544+0000","last_became_active":"2023-10-21T17:39:40.282270+0000","last_became_peered":"2023-10-21T17:39:40.282270+0000","last_unstale":"2023-10-21T17:39:54.502544+0000","last_undegraded":"2023-10-21T17:39:54.502544+0000","last_fullsized":"2023-10-21T17:39:54.502544+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:36.571340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484790+0000","last_change":"2023-10-21T17:39:39.240537+0000","last_active":"2023-10-21T17:39:54.484790+0000","last_peered":"2023-10-21T17:39:54.484790+0000","last_clean":"2023-10-21T17:39:54.484790+0000","last_became_active":"2023-10-21T17:39:39.240212+0000","last_became_peered":"2023-10-21T17:39:39.240212+0000","last_unstale":"2023-10-21T17:39:54.484790+0000","last_undegraded":"2023-10-21T17:39:54.484790+0000","last_fullsized":"2023-10-21T17:39:54.484790+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:57.552470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869344+0000","last_change":"2023-10-21T17:39:54.514201+0000","last_active":"2023-10-21T17:39:55.869344+0000","last_peered":"2023-10-21T17:39:55.869344+0000","last_clean":"2023-10-21T17:39:55.869344+0000","last_became_active":"2023-10-21T17:39:54.513317+0000","last_became_peered":"2023-10-21T17:39:54.513317+0000","last_unstale":"2023-10-21T17:39:55.869344+0000","last_undegraded":"2023-10-21T17:39:55.869344+0000","last_fullsized":"2023-10-21T17:39:55.869344+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:17:51.734321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482297+0000","last_change":"2023-10-21T17:39:40.278403+0000","last_active":"2023-10-21T17:39:54.482297+0000","last_peered":"2023-10-21T17:39:54.482297+0000","last_clean":"2023-10-21T17:39:54.482297+0000","last_became_active":"2023-10-21T17:39:40.278115+0000","last_became_peered":"2023-10-21T17:39:40.278115+0000","last_unstale":"2023-10-21T17:39:54.482297+0000","last_undegraded":"2023-10-21T17:39:54.482297+0000","last_fullsized":"2023-10-21T17:39:54.482297+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:16.014478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.511140+0000","last_change":"2023-10-21T17:39:39.239260+0000","last_active":"2023-10-21T17:39:54.511140+0000","last_peered":"2023-10-21T17:39:54.511140+0000","last_clean":"2023-10-21T17:39:54.511140+0000","last_became_active":"2023-10-21T17:39:39.235006+0000","last_became_peered":"2023-10-21T17:39:39.235006+0000","last_unstale":"2023-10-21T17:39:54.511140+0000","last_undegraded":"2023-10-21T17:39:54.511140+0000","last_fullsized":"2023-10-21T17:39:54.511140+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870926+0000","last_change":"2023-10-21T17:39:54.504803+0000","last_active":"2023-10-21T17:39:55.870926+0000","last_peered":"2023-10-21T17:39:55.870926+0000","last_clean":"2023-10-21T17:39:55.870926+0000","last_became_active":"2023-10-21T17:39:54.504547+0000","last_became_peered":"2023-10-21T17:39:54.504547+0000","last_unstale":"2023-10-21T17:39:55.870926+0000","last_undegraded":"2023-10-21T17:39:55.870926+0000","last_fullsized":"2023-10-21T17:39:55.870926+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:55.656371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.510878+0000","last_change":"2023-10-21T17:39:40.268385+0000","last_active":"2023-10-21T17:39:54.510878+0000","last_peered":"2023-10-21T17:39:54.510878+0000","last_clean":"2023-10-21T17:39:54.510878+0000","last_became_active":"2023-10-21T17:39:40.267876+0000","last_became_peered":"2023-10-21T17:39:40.267876+0000","last_unstale":"2023-10-21T17:39:54.510878+0000","last_undegraded":"2023-10-21T17:39:54.510878+0000","last_fullsized":"2023-10-21T17:39:54.510878+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:32.664957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502522+0000","last_change":"2023-10-21T17:39:39.246981+0000","last_active":"2023-10-21T17:39:54.502522+0000","last_peered":"2023-10-21T17:39:54.502522+0000","last_clean":"2023-10-21T17:39:54.502522+0000","last_became_active":"2023-10-21T17:39:39.246676+0000","last_became_peered":"2023-10-21T17:39:39.246676+0000","last_unstale":"2023-10-21T17:39:54.502522+0000","last_undegraded":"2023-10-21T17:39:54.502522+0000","last_fullsized":"2023-10-21T17:39:54.502522+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:08.549227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508776+0000","last_change":"2023-10-21T17:39:54.516931+0000","last_active":"2023-10-21T17:39:55.508776+0000","last_peered":"2023-10-21T17:39:55.508776+0000","last_clean":"2023-10-21T17:39:55.508776+0000","last_became_active":"2023-10-21T17:39:54.516656+0000","last_became_peered":"2023-10-21T17:39:54.516656+0000","last_unstale":"2023-10-21T17:39:55.508776+0000","last_undegraded":"2023-10-21T17:39:55.508776+0000","last_fullsized":"2023-10-21T17:39:55.508776+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:49.781653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504996+0000","last_change":"2023-10-21T17:39:40.273974+0000","last_active":"2023-10-21T17:39:54.504996+0000","last_peered":"2023-10-21T17:39:54.504996+0000","last_clean":"2023-10-21T17:39:54.504996+0000","last_became_active":"2023-10-21T17:39:40.273689+0000","last_became_peered":"2023-10-21T17:39:40.273689+0000","last_unstale":"2023-10-21T17:39:54.504996+0000","last_undegraded":"2023-10-21T17:39:54.504996+0000","last_fullsized":"2023-10-21T17:39:54.504996+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509152+0000","last_change":"2023-10-21T17:39:39.236436+0000","last_active":"2023-10-21T17:39:54.509152+0000","last_peered":"2023-10-21T17:39:54.509152+0000","last_clean":"2023-10-21T17:39:54.509152+0000","last_became_active":"2023-10-21T17:39:39.236185+0000","last_became_peered":"2023-10-21T17:39:39.236185+0000","last_unstale":"2023-10-21T17:39:54.509152+0000","last_undegraded":"2023-10-21T17:39:54.509152+0000","last_fullsized":"2023-10-21T17:39:54.509152+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:51.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508996+0000","last_change":"2023-10-21T17:39:54.518570+0000","last_active":"2023-10-21T17:39:55.508996+0000","last_peered":"2023-10-21T17:39:55.508996+0000","last_clean":"2023-10-21T17:39:55.508996+0000","last_became_active":"2023-10-21T17:39:54.517636+0000","last_became_peered":"2023-10-21T17:39:54.517636+0000","last_unstale":"2023-10-21T17:39:55.508996+0000","last_undegraded":"2023-10-21T17:39:55.508996+0000","last_fullsized":"2023-10-21T17:39:55.508996+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:32.034612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497521+0000","last_change":"2023-10-21T17:39:40.275825+0000","last_active":"2023-10-21T17:39:54.497521+0000","last_peered":"2023-10-21T17:39:54.497521+0000","last_clean":"2023-10-21T17:39:54.497521+0000","last_became_active":"2023-10-21T17:39:40.275521+0000","last_became_peered":"2023-10-21T17:39:40.275521+0000","last_unstale":"2023-10-21T17:39:54.497521+0000","last_undegraded":"2023-10-21T17:39:54.497521+0000","last_fullsized":"2023-10-21T17:39:54.497521+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:44:32.834263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496168+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:54.496168+0000","last_peered":"2023-10-21T17:39:54.496168+0000","last_clean":"2023-10-21T17:39:54.496168+0000","last_became_active":"2023-10-21T17:39:39.241978+0000","last_became_peered":"2023-10-21T17:39:39.241978+0000","last_unstale":"2023-10-21T17:39:54.496168+0000","last_undegraded":"2023-10-21T17:39:54.496168+0000","last_fullsized":"2023-10-21T17:39:54.496168+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:13.792604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509482+0000","last_change":"2023-10-21T17:39:54.518139+0000","last_active":"2023-10-21T17:39:55.509482+0000","last_peered":"2023-10-21T17:39:55.509482+0000","last_clean":"2023-10-21T17:39:55.509482+0000","last_became_active":"2023-10-21T17:39:54.517862+0000","last_became_peered":"2023-10-21T17:39:54.517862+0000","last_unstale":"2023-10-21T17:39:55.509482+0000","last_undegraded":"2023-10-21T17:39:55.509482+0000","last_fullsized":"2023-10-21T17:39:55.509482+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:15.523450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509972+0000","last_change":"2023-10-21T17:39:40.278021+0000","last_active":"2023-10-21T17:39:54.509972+0000","last_peered":"2023-10-21T17:39:54.509972+0000","last_clean":"2023-10-21T17:39:54.509972+0000","last_became_active":"2023-10-21T17:39:40.276836+0000","last_became_peered":"2023-10-21T17:39:40.276836+0000","last_unstale":"2023-10-21T17:39:54.509972+0000","last_undegraded":"2023-10-21T17:39:54.509972+0000","last_fullsized":"2023-10-21T17:39:54.509972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:37.173241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479260+0000","last_change":"2023-10-21T17:39:39.241068+0000","last_active":"2023-10-21T17:39:54.479260+0000","last_peered":"2023-10-21T17:39:54.479260+0000","last_clean":"2023-10-21T17:39:54.479260+0000","last_became_active":"2023-10-21T17:39:39.240781+0000","last_became_peered":"2023-10-21T17:39:39.240781+0000","last_unstale":"2023-10-21T17:39:54.479260+0000","last_undegraded":"2023-10-21T17:39:54.479260+0000","last_fullsized":"2023-10-21T17:39:54.479260+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:06.555504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508529+0000","last_change":"2023-10-21T17:39:54.515178+0000","last_active":"2023-10-21T17:39:55.508529+0000","last_peered":"2023-10-21T17:39:55.508529+0000","last_clean":"2023-10-21T17:39:55.508529+0000","last_became_active":"2023-10-21T17:39:54.514908+0000","last_became_peered":"2023-10-21T17:39:54.514908+0000","last_unstale":"2023-10-21T17:39:55.508529+0000","last_undegraded":"2023-10-21T17:39:55.508529+0000","last_fullsized":"2023-10-21T17:39:55.508529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:41.630527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499702+0000","last_change":"2023-10-21T17:39:40.278404+0000","last_active":"2023-10-21T17:39:54.499702+0000","last_peered":"2023-10-21T17:39:54.499702+0000","last_clean":"2023-10-21T17:39:54.499702+0000","last_became_active":"2023-10-21T17:39:40.276735+0000","last_became_peered":"2023-10-21T17:39:40.276735+0000","last_unstale":"2023-10-21T17:39:54.499702+0000","last_undegraded":"2023-10-21T17:39:54.499702+0000","last_fullsized":"2023-10-21T17:39:54.499702+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.523045+0000","last_change":"2023-10-21T17:39:39.229264+0000","last_active":"2023-10-21T17:39:54.523045+0000","last_peered":"2023-10-21T17:39:54.523045+0000","last_clean":"2023-10-21T17:39:54.523045+0000","last_became_active":"2023-10-21T17:39:39.227950+0000","last_became_peered":"2023-10-21T17:39:39.227950+0000","last_unstale":"2023-10-21T17:39:54.523045+0000","last_undegraded":"2023-10-21T17:39:54.523045+0000","last_fullsized":"2023-10-21T17:39:54.523045+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867036+0000","last_change":"2023-10-21T17:39:54.505800+0000","last_active":"2023-10-21T17:39:55.867036+0000","last_peered":"2023-10-21T17:39:55.867036+0000","last_clean":"2023-10-21T17:39:55.867036+0000","last_became_active":"2023-10-21T17:39:54.505347+0000","last_became_peered":"2023-10-21T17:39:54.505347+0000","last_unstale":"2023-10-21T17:39:55.867036+0000","last_undegraded":"2023-10-21T17:39:55.867036+0000","last_fullsized":"2023-10-21T17:39:55.867036+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:17:51.734321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483055+0000","last_change":"2023-10-21T17:39:40.262960+0000","last_active":"2023-10-21T17:39:54.483055+0000","last_peered":"2023-10-21T17:39:54.483055+0000","last_clean":"2023-10-21T17:39:54.483055+0000","last_became_active":"2023-10-21T17:39:40.262214+0000","last_became_peered":"2023-10-21T17:39:40.262214+0000","last_unstale":"2023-10-21T17:39:54.483055+0000","last_undegraded":"2023-10-21T17:39:54.483055+0000","last_fullsized":"2023-10-21T17:39:54.483055+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:08.816431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497756+0000","last_change":"2023-10-21T17:39:39.244340+0000","last_active":"2023-10-21T17:39:54.497756+0000","last_peered":"2023-10-21T17:39:54.497756+0000","last_clean":"2023-10-21T17:39:54.497756+0000","last_became_active":"2023-10-21T17:39:39.243245+0000","last_became_peered":"2023-10-21T17:39:39.243245+0000","last_unstale":"2023-10-21T17:39:54.497756+0000","last_undegraded":"2023-10-21T17:39:54.497756+0000","last_fullsized":"2023-10-21T17:39:54.497756+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506646+0000","last_change":"2023-10-21T17:39:54.511146+0000","last_active":"2023-10-21T17:39:55.506646+0000","last_peered":"2023-10-21T17:39:55.506646+0000","last_clean":"2023-10-21T17:39:55.506646+0000","last_became_active":"2023-10-21T17:39:54.510047+0000","last_became_peered":"2023-10-21T17:39:54.510047+0000","last_unstale":"2023-10-21T17:39:55.506646+0000","last_undegraded":"2023-10-21T17:39:55.506646+0000","last_fullsized":"2023-10-21T17:39:55.506646+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:50.714206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499628+0000","last_change":"2023-10-21T17:39:40.279028+0000","last_active":"2023-10-21T17:39:54.499628+0000","last_peered":"2023-10-21T17:39:54.499628+0000","last_clean":"2023-10-21T17:39:54.499628+0000","last_became_active":"2023-10-21T17:39:40.278727+0000","last_became_peered":"2023-10-21T17:39:40.278727+0000","last_unstale":"2023-10-21T17:39:54.499628+0000","last_undegraded":"2023-10-21T17:39:54.499628+0000","last_fullsized":"2023-10-21T17:39:54.499628+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:39.467095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":19,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.507640+0000","last_change":"2023-10-21T17:39:39.238814+0000","last_active":"2023-10-21T17:39:54.507640+0000","last_peered":"2023-10-21T17:39:54.507640+0000","last_clean":"2023-10-21T17:39:54.507640+0000","last_became_active":"2023-10-21T17:39:39.238497+0000","last_became_peered":"2023-10-21T17:39:39.238497+0000","last_unstale":"2023-10-21T17:39:54.507640+0000","last_undegraded":"2023-10-21T17:39:54.507640+0000","last_fullsized":"2023-10-21T17:39:54.507640+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:37.376711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869001+0000","last_change":"2023-10-21T17:39:54.514413+0000","last_active":"2023-10-21T17:39:55.869001+0000","last_peered":"2023-10-21T17:39:55.869001+0000","last_clean":"2023-10-21T17:39:55.869001+0000","last_became_active":"2023-10-21T17:39:54.513642+0000","last_became_peered":"2023-10-21T17:39:54.513642+0000","last_unstale":"2023-10-21T17:39:55.869001+0000","last_undegraded":"2023-10-21T17:39:55.869001+0000","last_fullsized":"2023-10-21T17:39:55.869001+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:50.797174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504085+0000","last_change":"2023-10-21T17:39:40.280192+0000","last_active":"2023-10-21T17:39:54.504085+0000","last_peered":"2023-10-21T17:39:54.504085+0000","last_clean":"2023-10-21T17:39:54.504085+0000","last_became_active":"2023-10-21T17:39:40.279897+0000","last_became_peered":"2023-10-21T17:39:40.279897+0000","last_unstale":"2023-10-21T17:39:54.504085+0000","last_undegraded":"2023-10-21T17:39:54.504085+0000","last_fullsized":"2023-10-21T17:39:54.504085+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:16.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"255'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479512+0000","last_change":"2023-10-21T17:39:39.237379+0000","last_active":"2023-10-21T17:39:54.479512+0000","last_peered":"2023-10-21T17:39:54.479512+0000","last_clean":"2023-10-21T17:39:54.479512+0000","last_became_active":"2023-10-21T17:39:39.236089+0000","last_became_peered":"2023-10-21T17:39:39.236089+0000","last_unstale":"2023-10-21T17:39:54.479512+0000","last_undegraded":"2023-10-21T17:39:54.479512+0000","last_fullsized":"2023-10-21T17:39:54.479512+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:13.628441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504970+0000","last_change":"2023-10-21T17:39:54.505105+0000","last_active":"2023-10-21T17:39:55.504970+0000","last_peered":"2023-10-21T17:39:55.504970+0000","last_clean":"2023-10-21T17:39:55.504970+0000","last_became_active":"2023-10-21T17:39:54.502129+0000","last_became_peered":"2023-10-21T17:39:54.502129+0000","last_unstale":"2023-10-21T17:39:55.504970+0000","last_undegraded":"2023-10-21T17:39:55.504970+0000","last_fullsized":"2023-10-21T17:39:55.504970+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:51.037591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484342+0000","last_change":"2023-10-21T17:39:40.283845+0000","last_active":"2023-10-21T17:39:54.484342+0000","last_peered":"2023-10-21T17:39:54.484342+0000","last_clean":"2023-10-21T17:39:54.484342+0000","last_became_active":"2023-10-21T17:39:40.283046+0000","last_became_peered":"2023-10-21T17:39:40.283046+0000","last_unstale":"2023-10-21T17:39:54.484342+0000","last_undegraded":"2023-10-21T17:39:54.484342+0000","last_fullsized":"2023-10-21T17:39:54.484342+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:49:18.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502617+0000","last_change":"2023-10-21T17:39:39.244723+0000","last_active":"2023-10-21T17:39:54.502617+0000","last_peered":"2023-10-21T17:39:54.502617+0000","last_clean":"2023-10-21T17:39:54.502617+0000","last_became_active":"2023-10-21T17:39:39.244145+0000","last_became_peered":"2023-10-21T17:39:39.244145+0000","last_unstale":"2023-10-21T17:39:54.502617+0000","last_undegraded":"2023-10-21T17:39:54.502617+0000","last_fullsized":"2023-10-21T17:39:54.502617+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:15.752098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.833095+0000","last_change":"2023-10-21T17:39:54.511121+0000","last_active":"2023-10-21T17:39:55.833095+0000","last_peered":"2023-10-21T17:39:55.833095+0000","last_clean":"2023-10-21T17:39:55.833095+0000","last_became_active":"2023-10-21T17:39:54.510828+0000","last_became_peered":"2023-10-21T17:39:54.510828+0000","last_unstale":"2023-10-21T17:39:55.833095+0000","last_undegraded":"2023-10-21T17:39:55.833095+0000","last_fullsized":"2023-10-21T17:39:55.833095+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:57.717949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504751+0000","last_change":"2023-10-21T17:39:40.272813+0000","last_active":"2023-10-21T17:39:54.504751+0000","last_peered":"2023-10-21T17:39:54.504751+0000","last_clean":"2023-10-21T17:39:54.504751+0000","last_became_active":"2023-10-21T17:39:40.272544+0000","last_became_peered":"2023-10-21T17:39:40.272544+0000","last_unstale":"2023-10-21T17:39:54.504751+0000","last_undegraded":"2023-10-21T17:39:54.504751+0000","last_fullsized":"2023-10-21T17:39:54.504751+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:23.620934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501962+0000","last_change":"2023-10-21T17:39:39.229845+0000","last_active":"2023-10-21T17:39:54.501962+0000","last_peered":"2023-10-21T17:39:54.501962+0000","last_clean":"2023-10-21T17:39:54.501962+0000","last_became_active":"2023-10-21T17:39:39.229136+0000","last_became_peered":"2023-10-21T17:39:39.229136+0000","last_unstale":"2023-10-21T17:39:54.501962+0000","last_undegraded":"2023-10-21T17:39:54.501962+0000","last_fullsized":"2023-10-21T17:39:54.501962+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:33.975640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867568+0000","last_change":"2023-10-21T17:39:54.511484+0000","last_active":"2023-10-21T17:39:55.867568+0000","last_peered":"2023-10-21T17:39:55.867568+0000","last_clean":"2023-10-21T17:39:55.867568+0000","last_became_active":"2023-10-21T17:39:54.511070+0000","last_became_peered":"2023-10-21T17:39:54.511070+0000","last_unstale":"2023-10-21T17:39:55.867568+0000","last_undegraded":"2023-10-21T17:39:55.867568+0000","last_fullsized":"2023-10-21T17:39:55.867568+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:08.687825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504877+0000","last_change":"2023-10-21T17:39:40.276873+0000","last_active":"2023-10-21T17:39:54.504877+0000","last_peered":"2023-10-21T17:39:54.504877+0000","last_clean":"2023-10-21T17:39:54.504877+0000","last_became_active":"2023-10-21T17:39:40.276503+0000","last_became_peered":"2023-10-21T17:39:40.276503+0000","last_unstale":"2023-10-21T17:39:54.504877+0000","last_undegraded":"2023-10-21T17:39:54.504877+0000","last_fullsized":"2023-10-21T17:39:54.504877+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:34.926709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501910+0000","last_change":"2023-10-21T17:39:39.227400+0000","last_active":"2023-10-21T17:39:54.501910+0000","last_peered":"2023-10-21T17:39:54.501910+0000","last_clean":"2023-10-21T17:39:54.501910+0000","last_became_active":"2023-10-21T17:39:39.226822+0000","last_became_peered":"2023-10-21T17:39:39.226822+0000","last_unstale":"2023-10-21T17:39:54.501910+0000","last_undegraded":"2023-10-21T17:39:54.501910+0000","last_fullsized":"2023-10-21T17:39:54.501910+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.203739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832416+0000","last_change":"2023-10-21T17:39:54.510329+0000","last_active":"2023-10-21T17:39:55.832416+0000","last_peered":"2023-10-21T17:39:55.832416+0000","last_clean":"2023-10-21T17:39:55.832416+0000","last_became_active":"2023-10-21T17:39:54.509095+0000","last_became_peered":"2023-10-21T17:39:54.509095+0000","last_unstale":"2023-10-21T17:39:55.832416+0000","last_undegraded":"2023-10-21T17:39:55.832416+0000","last_fullsized":"2023-10-21T17:39:55.832416+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:41.483301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509687+0000","last_change":"2023-10-21T17:39:40.281657+0000","last_active":"2023-10-21T17:39:54.509687+0000","last_peered":"2023-10-21T17:39:54.509687+0000","last_clean":"2023-10-21T17:39:54.509687+0000","last_became_active":"2023-10-21T17:39:40.281069+0000","last_became_peered":"2023-10-21T17:39:40.281069+0000","last_unstale":"2023-10-21T17:39:54.509687+0000","last_undegraded":"2023-10-21T17:39:54.509687+0000","last_fullsized":"2023-10-21T17:39:54.509687+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:01.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485060+0000","last_change":"2023-10-21T17:39:39.235563+0000","last_active":"2023-10-21T17:39:54.485060+0000","last_peered":"2023-10-21T17:39:54.485060+0000","last_clean":"2023-10-21T17:39:54.485060+0000","last_became_active":"2023-10-21T17:39:39.235286+0000","last_became_peered":"2023-10-21T17:39:39.235286+0000","last_unstale":"2023-10-21T17:39:54.485060+0000","last_undegraded":"2023-10-21T17:39:54.485060+0000","last_fullsized":"2023-10-21T17:39:54.485060+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:56.950389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866603+0000","last_change":"2023-10-21T17:39:54.518702+0000","last_active":"2023-10-21T17:39:55.866603+0000","last_peered":"2023-10-21T17:39:55.866603+0000","last_clean":"2023-10-21T17:39:55.866603+0000","last_became_active":"2023-10-21T17:39:54.518397+0000","last_became_peered":"2023-10-21T17:39:54.518397+0000","last_unstale":"2023-10-21T17:39:55.866603+0000","last_undegraded":"2023-10-21T17:39:55.866603+0000","last_fullsized":"2023-10-21T17:39:55.866603+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:47:56.028968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501495+0000","last_change":"2023-10-21T17:39:40.274495+0000","last_active":"2023-10-21T17:39:54.501495+0000","last_peered":"2023-10-21T17:39:54.501495+0000","last_clean":"2023-10-21T17:39:54.501495+0000","last_became_active":"2023-10-21T17:39:40.273782+0000","last_became_peered":"2023-10-21T17:39:40.273782+0000","last_unstale":"2023-10-21T17:39:54.501495+0000","last_undegraded":"2023-10-21T17:39:54.501495+0000","last_fullsized":"2023-10-21T17:39:54.501495+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:27.378721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484345+0000","last_change":"2023-10-21T17:39:39.236238+0000","last_active":"2023-10-21T17:39:54.484345+0000","last_peered":"2023-10-21T17:39:54.484345+0000","last_clean":"2023-10-21T17:39:54.484345+0000","last_became_active":"2023-10-21T17:39:39.234980+0000","last_became_peered":"2023-10-21T17:39:39.234980+0000","last_unstale":"2023-10-21T17:39:54.484345+0000","last_undegraded":"2023-10-21T17:39:54.484345+0000","last_fullsized":"2023-10-21T17:39:54.484345+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:21.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867250+0000","last_change":"2023-10-21T17:39:54.510233+0000","last_active":"2023-10-21T17:39:55.867250+0000","last_peered":"2023-10-21T17:39:55.867250+0000","last_clean":"2023-10-21T17:39:55.867250+0000","last_became_active":"2023-10-21T17:39:54.508196+0000","last_became_peered":"2023-10-21T17:39:54.508196+0000","last_unstale":"2023-10-21T17:39:55.867250+0000","last_undegraded":"2023-10-21T17:39:55.867250+0000","last_fullsized":"2023-10-21T17:39:55.867250+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:40:47.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501799+0000","last_change":"2023-10-21T17:39:40.277381+0000","last_active":"2023-10-21T17:39:54.501799+0000","last_peered":"2023-10-21T17:39:54.501799+0000","last_clean":"2023-10-21T17:39:54.501799+0000","last_became_active":"2023-10-21T17:39:40.277018+0000","last_became_peered":"2023-10-21T17:39:40.277018+0000","last_unstale":"2023-10-21T17:39:54.501799+0000","last_undegraded":"2023-10-21T17:39:54.501799+0000","last_fullsized":"2023-10-21T17:39:54.501799+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:25:00.433924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489611+0000","last_change":"2023-10-21T17:39:39.248091+0000","last_active":"2023-10-21T17:39:54.489611+0000","last_peered":"2023-10-21T17:39:54.489611+0000","last_clean":"2023-10-21T17:39:54.489611+0000","last_became_active":"2023-10-21T17:39:39.247775+0000","last_became_peered":"2023-10-21T17:39:39.247775+0000","last_unstale":"2023-10-21T17:39:54.489611+0000","last_undegraded":"2023-10-21T17:39:54.489611+0000","last_fullsized":"2023-10-21T17:39:54.489611+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:43.183503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.831529+0000","last_change":"2023-10-21T17:39:54.518461+0000","last_active":"2023-10-21T17:39:55.831529+0000","last_peered":"2023-10-21T17:39:55.831529+0000","last_clean":"2023-10-21T17:39:55.831529+0000","last_became_active":"2023-10-21T17:39:54.517478+0000","last_became_peered":"2023-10-21T17:39:54.517478+0000","last_unstale":"2023-10-21T17:39:55.831529+0000","last_undegraded":"2023-10-21T17:39:55.831529+0000","last_fullsized":"2023-10-21T17:39:55.831529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:17:11.907848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.488175+0000","last_change":"2023-10-21T17:39:40.268499+0000","last_active":"2023-10-21T17:39:54.488175+0000","last_peered":"2023-10-21T17:39:54.488175+0000","last_clean":"2023-10-21T17:39:54.488175+0000","last_became_active":"2023-10-21T17:39:40.268106+0000","last_became_peered":"2023-10-21T17:39:40.268106+0000","last_unstale":"2023-10-21T17:39:54.488175+0000","last_undegraded":"2023-10-21T17:39:54.488175+0000","last_fullsized":"2023-10-21T17:39:54.488175+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:09.587502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485037+0000","last_change":"2023-10-21T17:39:39.237722+0000","last_active":"2023-10-21T17:39:54.485037+0000","last_peered":"2023-10-21T17:39:54.485037+0000","last_clean":"2023-10-21T17:39:54.485037+0000","last_became_active":"2023-10-21T17:39:39.235932+0000","last_became_peered":"2023-10-21T17:39:39.235932+0000","last_unstale":"2023-10-21T17:39:54.485037+0000","last_undegraded":"2023-10-21T17:39:54.485037+0000","last_fullsized":"2023-10-21T17:39:54.485037+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:08.581881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868619+0000","last_change":"2023-10-21T17:39:54.517167+0000","last_active":"2023-10-21T17:39:55.868619+0000","last_peered":"2023-10-21T17:39:55.868619+0000","last_clean":"2023-10-21T17:39:55.868619+0000","last_became_active":"2023-10-21T17:39:54.513938+0000","last_became_peered":"2023-10-21T17:39:54.513938+0000","last_unstale":"2023-10-21T17:39:55.868619+0000","last_undegraded":"2023-10-21T17:39:55.868619+0000","last_fullsized":"2023-10-21T17:39:55.868619+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:48:03.873117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500167+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:54.500167+0000","last_peered":"2023-10-21T17:39:54.500167+0000","last_clean":"2023-10-21T17:39:54.500167+0000","last_became_active":"2023-10-21T17:39:40.282699+0000","last_became_peered":"2023-10-21T17:39:40.282699+0000","last_unstale":"2023-10-21T17:39:54.500167+0000","last_undegraded":"2023-10-21T17:39:54.500167+0000","last_fullsized":"2023-10-21T17:39:54.500167+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:28:01.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503872+0000","last_change":"2023-10-21T17:39:39.247859+0000","last_active":"2023-10-21T17:39:54.503872+0000","last_peered":"2023-10-21T17:39:54.503872+0000","last_clean":"2023-10-21T17:39:54.503872+0000","last_became_active":"2023-10-21T17:39:39.245045+0000","last_became_peered":"2023-10-21T17:39:39.245045+0000","last_unstale":"2023-10-21T17:39:54.503872+0000","last_undegraded":"2023-10-21T17:39:54.503872+0000","last_fullsized":"2023-10-21T17:39:54.503872+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:22.615313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832963+0000","last_change":"2023-10-21T17:39:54.505811+0000","last_active":"2023-10-21T17:39:55.832963+0000","last_peered":"2023-10-21T17:39:55.832963+0000","last_clean":"2023-10-21T17:39:55.832963+0000","last_became_active":"2023-10-21T17:39:54.505555+0000","last_became_peered":"2023-10-21T17:39:54.505555+0000","last_unstale":"2023-10-21T17:39:55.832963+0000","last_undegraded":"2023-10-21T17:39:55.832963+0000","last_fullsized":"2023-10-21T17:39:55.832963+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:20.065152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.498935+0000","last_change":"2023-10-21T17:39:40.283322+0000","last_active":"2023-10-21T17:39:54.498935+0000","last_peered":"2023-10-21T17:39:54.498935+0000","last_clean":"2023-10-21T17:39:54.498935+0000","last_became_active":"2023-10-21T17:39:40.282539+0000","last_became_peered":"2023-10-21T17:39:40.282539+0000","last_unstale":"2023-10-21T17:39:54.498935+0000","last_undegraded":"2023-10-21T17:39:54.498935+0000","last_fullsized":"2023-10-21T17:39:54.498935+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:49.124101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500254+0000","last_change":"2023-10-21T17:39:39.242639+0000","last_active":"2023-10-21T17:39:54.500254+0000","last_peered":"2023-10-21T17:39:54.500254+0000","last_clean":"2023-10-21T17:39:54.500254+0000","last_became_active":"2023-10-21T17:39:39.242334+0000","last_became_peered":"2023-10-21T17:39:39.242334+0000","last_unstale":"2023-10-21T17:39:54.500254+0000","last_undegraded":"2023-10-21T17:39:54.500254+0000","last_fullsized":"2023-10-21T17:39:54.500254+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:44.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868017+0000","last_change":"2023-10-21T17:39:54.512580+0000","last_active":"2023-10-21T17:39:55.868017+0000","last_peered":"2023-10-21T17:39:55.868017+0000","last_clean":"2023-10-21T17:39:55.868017+0000","last_became_active":"2023-10-21T17:39:54.511005+0000","last_became_peered":"2023-10-21T17:39:54.511005+0000","last_unstale":"2023-10-21T17:39:55.868017+0000","last_undegraded":"2023-10-21T17:39:55.868017+0000","last_fullsized":"2023-10-21T17:39:55.868017+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:27:10.349105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.487231+0000","last_change":"2023-10-21T17:39:40.270958+0000","last_active":"2023-10-21T17:39:54.487231+0000","last_peered":"2023-10-21T17:39:54.487231+0000","last_clean":"2023-10-21T17:39:54.487231+0000","last_became_active":"2023-10-21T17:39:40.270640+0000","last_became_peered":"2023-10-21T17:39:40.270640+0000","last_unstale":"2023-10-21T17:39:54.487231+0000","last_undegraded":"2023-10-21T17:39:54.487231+0000","last_fullsized":"2023-10-21T17:39:54.487231+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:21.522147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484668+0000","last_change":"2023-10-21T17:39:39.246257+0000","last_active":"2023-10-21T17:39:54.484668+0000","last_peered":"2023-10-21T17:39:54.484668+0000","last_clean":"2023-10-21T17:39:54.484668+0000","last_became_active":"2023-10-21T17:39:39.245252+0000","last_became_peered":"2023-10-21T17:39:39.245252+0000","last_unstale":"2023-10-21T17:39:54.484668+0000","last_undegraded":"2023-10-21T17:39:54.484668+0000","last_fullsized":"2023-10-21T17:39:54.484668+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:18.223298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867540+0000","last_change":"2023-10-21T17:39:54.515461+0000","last_active":"2023-10-21T17:39:55.867540+0000","last_peered":"2023-10-21T17:39:55.867540+0000","last_clean":"2023-10-21T17:39:55.867540+0000","last_became_active":"2023-10-21T17:39:54.514176+0000","last_became_peered":"2023-10-21T17:39:54.514176+0000","last_unstale":"2023-10-21T17:39:55.867540+0000","last_undegraded":"2023-10-21T17:39:55.867540+0000","last_fullsized":"2023-10-21T17:39:55.867540+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:42:40.444872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484703+0000","last_change":"2023-10-21T17:39:40.282652+0000","last_active":"2023-10-21T17:39:54.484703+0000","last_peered":"2023-10-21T17:39:54.484703+0000","last_clean":"2023-10-21T17:39:54.484703+0000","last_became_active":"2023-10-21T17:39:40.282108+0000","last_became_peered":"2023-10-21T17:39:40.282108+0000","last_unstale":"2023-10-21T17:39:54.484703+0000","last_undegraded":"2023-10-21T17:39:54.484703+0000","last_fullsized":"2023-10-21T17:39:54.484703+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:38.363235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483498+0000","last_change":"2023-10-21T17:39:39.242559+0000","last_active":"2023-10-21T17:39:54.483498+0000","last_peered":"2023-10-21T17:39:54.483498+0000","last_clean":"2023-10-21T17:39:54.483498+0000","last_became_active":"2023-10-21T17:39:39.236694+0000","last_became_peered":"2023-10-21T17:39:39.236694+0000","last_unstale":"2023-10-21T17:39:54.483498+0000","last_undegraded":"2023-10-21T17:39:54.483498+0000","last_fullsized":"2023-10-21T17:39:54.483498+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:58.202607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509429+0000","last_change":"2023-10-21T17:39:54.510437+0000","last_active":"2023-10-21T17:39:55.509429+0000","last_peered":"2023-10-21T17:39:55.509429+0000","last_clean":"2023-10-21T17:39:55.509429+0000","last_became_active":"2023-10-21T17:39:54.510176+0000","last_became_peered":"2023-10-21T17:39:54.510176+0000","last_unstale":"2023-10-21T17:39:55.509429+0000","last_undegraded":"2023-10-21T17:39:55.509429+0000","last_fullsized":"2023-10-21T17:39:55.509429+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:52.104608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485236+0000","last_change":"2023-10-21T17:39:40.268873+0000","last_active":"2023-10-21T17:39:54.485236+0000","last_peered":"2023-10-21T17:39:54.485236+0000","last_clean":"2023-10-21T17:39:54.485236+0000","last_became_active":"2023-10-21T17:39:40.268515+0000","last_became_peered":"2023-10-21T17:39:40.268515+0000","last_unstale":"2023-10-21T17:39:54.485236+0000","last_undegraded":"2023-10-21T17:39:54.485236+0000","last_fullsized":"2023-10-21T17:39:54.485236+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:24.479277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.506703+0000","last_change":"2023-10-21T17:39:39.241987+0000","last_active":"2023-10-21T17:39:54.506703+0000","last_peered":"2023-10-21T17:39:54.506703+0000","last_clean":"2023-10-21T17:39:54.506703+0000","last_became_active":"2023-10-21T17:39:39.241676+0000","last_became_peered":"2023-10-21T17:39:39.241676+0000","last_unstale":"2023-10-21T17:39:54.506703+0000","last_undegraded":"2023-10-21T17:39:54.506703+0000","last_fullsized":"2023-10-21T17:39:54.506703+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:33.595143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866648+0000","last_change":"2023-10-21T17:39:54.513824+0000","last_active":"2023-10-21T17:39:55.866648+0000","last_peered":"2023-10-21T17:39:55.866648+0000","last_clean":"2023-10-21T17:39:55.866648+0000","last_became_active":"2023-10-21T17:39:54.513565+0000","last_became_peered":"2023-10-21T17:39:54.513565+0000","last_unstale":"2023-10-21T17:39:55.866648+0000","last_undegraded":"2023-10-21T17:39:55.866648+0000","last_fullsized":"2023-10-21T17:39:55.866648+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:49.781653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499612+0000","last_change":"2023-10-21T17:39:40.282789+0000","last_active":"2023-10-21T17:39:54.499612+0000","last_peered":"2023-10-21T17:39:54.499612+0000","last_clean":"2023-10-21T17:39:54.499612+0000","last_became_active":"2023-10-21T17:39:40.282503+0000","last_became_peered":"2023-10-21T17:39:40.282503+0000","last_unstale":"2023-10-21T17:39:54.499612+0000","last_undegraded":"2023-10-21T17:39:54.499612+0000","last_fullsized":"2023-10-21T17:39:54.499612+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.498247+0000","last_change":"2023-10-21T17:39:39.245748+0000","last_active":"2023-10-21T17:39:54.498247+0000","last_peered":"2023-10-21T17:39:54.498247+0000","last_clean":"2023-10-21T17:39:54.498247+0000","last_became_active":"2023-10-21T17:39:39.245479+0000","last_became_peered":"2023-10-21T17:39:39.245479+0000","last_unstale":"2023-10-21T17:39:54.498247+0000","last_undegraded":"2023-10-21T17:39:54.498247+0000","last_fullsized":"2023-10-21T17:39:54.498247+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869557+0000","last_change":"2023-10-21T17:39:54.505611+0000","last_active":"2023-10-21T17:39:55.869557+0000","last_peered":"2023-10-21T17:39:55.869557+0000","last_clean":"2023-10-21T17:39:55.869557+0000","last_became_active":"2023-10-21T17:39:54.504970+0000","last_became_peered":"2023-10-21T17:39:54.504970+0000","last_unstale":"2023-10-21T17:39:55.869557+0000","last_undegraded":"2023-10-21T17:39:55.869557+0000","last_fullsized":"2023-10-21T17:39:55.869557+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:45.733225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497501+0000","last_change":"2023-10-21T17:39:40.277680+0000","last_active":"2023-10-21T17:39:54.497501+0000","last_peered":"2023-10-21T17:39:54.497501+0000","last_clean":"2023-10-21T17:39:54.497501+0000","last_became_active":"2023-10-21T17:39:40.277441+0000","last_became_peered":"2023-10-21T17:39:40.277441+0000","last_unstale":"2023-10-21T17:39:54.497501+0000","last_undegraded":"2023-10-21T17:39:54.497501+0000","last_fullsized":"2023-10-21T17:39:54.497501+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"254'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.508198+0000","last_change":"2023-10-21T17:39:39.246309+0000","last_active":"2023-10-21T17:39:54.508198+0000","last_peered":"2023-10-21T17:39:54.508198+0000","last_clean":"2023-10-21T17:39:54.508198+0000","last_became_active":"2023-10-21T17:39:39.245706+0000","last_became_peered":"2023-10-21T17:39:39.245706+0000","last_unstale":"2023-10-21T17:39:54.508198+0000","last_undegraded":"2023-10-21T17:39:54.508198+0000","last_fullsized":"2023-10-21T17:39:54.508198+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:35.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866002+0000","last_change":"2023-10-21T17:39:54.506231+0000","last_active":"2023-10-21T17:39:55.866002+0000","last_peered":"2023-10-21T17:39:55.866002+0000","last_clean":"2023-10-21T17:39:55.866002+0000","last_became_active":"2023-10-21T17:39:54.505929+0000","last_became_peered":"2023-10-21T17:39:54.505929+0000","last_unstale":"2023-10-21T17:39:55.866002+0000","last_undegraded":"2023-10-21T17:39:55.866002+0000","last_fullsized":"2023-10-21T17:39:55.866002+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:26.485478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499532+0000","last_change":"2023-10-21T17:39:40.281463+0000","last_active":"2023-10-21T17:39:54.499532+0000","last_peered":"2023-10-21T17:39:54.499532+0000","last_clean":"2023-10-21T17:39:54.499532+0000","last_became_active":"2023-10-21T17:39:40.281137+0000","last_became_peered":"2023-10-21T17:39:40.281137+0000","last_unstale":"2023-10-21T17:39:54.499532+0000","last_undegraded":"2023-10-21T17:39:54.499532+0000","last_fullsized":"2023-10-21T17:39:54.499532+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:03:12.244419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.513312+0000","last_change":"2023-10-21T17:39:39.227041+0000","last_active":"2023-10-21T17:39:54.513312+0000","last_peered":"2023-10-21T17:39:54.513312+0000","last_clean":"2023-10-21T17:39:54.513312+0000","last_became_active":"2023-10-21T17:39:39.225535+0000","last_became_peered":"2023-10-21T17:39:39.225535+0000","last_unstale":"2023-10-21T17:39:54.513312+0000","last_undegraded":"2023-10-21T17:39:54.513312+0000","last_fullsized":"2023-10-21T17:39:54.513312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:20.950421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504487+0000","last_change":"2023-10-21T17:39:54.509781+0000","last_active":"2023-10-21T17:39:55.504487+0000","last_peered":"2023-10-21T17:39:55.504487+0000","last_clean":"2023-10-21T17:39:55.504487+0000","last_became_active":"2023-10-21T17:39:54.508599+0000","last_became_peered":"2023-10-21T17:39:54.508599+0000","last_unstale":"2023-10-21T17:39:55.504487+0000","last_undegraded":"2023-10-21T17:39:55.504487+0000","last_fullsized":"2023-10-21T17:39:55.504487+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:08.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501366+0000","last_change":"2023-10-21T17:39:40.283081+0000","last_active":"2023-10-21T17:39:54.501366+0000","last_peered":"2023-10-21T17:39:54.501366+0000","last_clean":"2023-10-21T17:39:54.501366+0000","last_became_active":"2023-10-21T17:39:40.282782+0000","last_became_peered":"2023-10-21T17:39:40.282782+0000","last_unstale":"2023-10-21T17:39:54.501366+0000","last_undegraded":"2023-10-21T17:39:54.501366+0000","last_fullsized":"2023-10-21T17:39:54.501366+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:37.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.511228+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:54.511228+0000","last_peered":"2023-10-21T17:39:54.511228+0000","last_clean":"2023-10-21T17:39:54.511228+0000","last_became_active":"2023-10-21T17:39:39.241658+0000","last_became_peered":"2023-10-21T17:39:39.241658+0000","last_unstale":"2023-10-21T17:39:54.511228+0000","last_undegraded":"2023-10-21T17:39:54.511228+0000","last_fullsized":"2023-10-21T17:39:54.511228+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870778+0000","last_change":"2023-10-21T17:39:54.516147+0000","last_active":"2023-10-21T17:39:55.870778+0000","last_peered":"2023-10-21T17:39:55.870778+0000","last_clean":"2023-10-21T17:39:55.870778+0000","last_became_active":"2023-10-21T17:39:54.515856+0000","last_became_peered":"2023-10-21T17:39:54.515856+0000","last_unstale":"2023-10-21T17:39:55.870778+0000","last_undegraded":"2023-10-21T17:39:55.870778+0000","last_fullsized":"2023-10-21T17:39:55.870778+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:20.545832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.510788+0000","last_change":"2023-10-21T17:39:40.267491+0000","last_active":"2023-10-21T17:39:54.510788+0000","last_peered":"2023-10-21T17:39:54.510788+0000","last_clean":"2023-10-21T17:39:54.510788+0000","last_became_active":"2023-10-21T17:39:40.267154+0000","last_became_peered":"2023-10-21T17:39:40.267154+0000","last_unstale":"2023-10-21T17:39:54.510788+0000","last_undegraded":"2023-10-21T17:39:54.510788+0000","last_fullsized":"2023-10-21T17:39:54.510788+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:14.379177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483519+0000","last_change":"2023-10-21T17:39:39.247298+0000","last_active":"2023-10-21T17:39:54.483519+0000","last_peered":"2023-10-21T17:39:54.483519+0000","last_clean":"2023-10-21T17:39:54.483519+0000","last_became_active":"2023-10-21T17:39:39.247016+0000","last_became_peered":"2023-10-21T17:39:39.247016+0000","last_unstale":"2023-10-21T17:39:54.483519+0000","last_undegraded":"2023-10-21T17:39:54.483519+0000","last_fullsized":"2023-10-21T17:39:54.483519+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:58.778201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510462+0000","last_change":"2023-10-21T17:39:54.523451+0000","last_active":"2023-10-21T17:39:55.510462+0000","last_peered":"2023-10-21T17:39:55.510462+0000","last_clean":"2023-10-21T17:39:55.510462+0000","last_became_active":"2023-10-21T17:39:54.523163+0000","last_became_peered":"2023-10-21T17:39:54.523163+0000","last_unstale":"2023-10-21T17:39:55.510462+0000","last_undegraded":"2023-10-21T17:39:55.510462+0000","last_fullsized":"2023-10-21T17:39:55.510462+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:43.836182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.480665+0000","last_change":"2023-10-21T17:39:40.275609+0000","last_active":"2023-10-21T17:39:54.480665+0000","last_peered":"2023-10-21T17:39:54.480665+0000","last_clean":"2023-10-21T17:39:54.480665+0000","last_became_active":"2023-10-21T17:39:40.275290+0000","last_became_peered":"2023-10-21T17:39:40.275290+0000","last_unstale":"2023-10-21T17:39:54.480665+0000","last_undegraded":"2023-10-21T17:39:54.480665+0000","last_fullsized":"2023-10-21T17:39:54.480665+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:22.217845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502055+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:54.502055+0000","last_peered":"2023-10-21T17:39:54.502055+0000","last_clean":"2023-10-21T17:39:54.502055+0000","last_became_active":"2023-10-21T17:39:39.238047+0000","last_became_peered":"2023-10-21T17:39:39.238047+0000","last_unstale":"2023-10-21T17:39:54.502055+0000","last_undegraded":"2023-10-21T17:39:54.502055+0000","last_fullsized":"2023-10-21T17:39:54.502055+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867207+0000","last_change":"2023-10-21T17:39:54.511000+0000","last_active":"2023-10-21T17:39:55.867207+0000","last_peered":"2023-10-21T17:39:55.867207+0000","last_clean":"2023-10-21T17:39:55.867207+0000","last_became_active":"2023-10-21T17:39:54.510718+0000","last_became_peered":"2023-10-21T17:39:54.510718+0000","last_unstale":"2023-10-21T17:39:55.867207+0000","last_undegraded":"2023-10-21T17:39:55.867207+0000","last_fullsized":"2023-10-21T17:39:55.867207+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:54.319191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489520+0000","last_change":"2023-10-21T17:39:40.279881+0000","last_active":"2023-10-21T17:39:54.489520+0000","last_peered":"2023-10-21T17:39:54.489520+0000","last_clean":"2023-10-21T17:39:54.489520+0000","last_became_active":"2023-10-21T17:39:40.279567+0000","last_became_peered":"2023-10-21T17:39:40.279567+0000","last_unstale":"2023-10-21T17:39:54.489520+0000","last_undegraded":"2023-10-21T17:39:54.489520+0000","last_fullsized":"2023-10-21T17:39:54.489520+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:23.894434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496282+0000","last_change":"2023-10-21T17:39:39.244421+0000","last_active":"2023-10-21T17:39:54.496282+0000","last_peered":"2023-10-21T17:39:54.496282+0000","last_clean":"2023-10-21T17:39:54.496282+0000","last_became_active":"2023-10-21T17:39:39.244026+0000","last_became_peered":"2023-10-21T17:39:39.244026+0000","last_unstale":"2023-10-21T17:39:54.496282+0000","last_undegraded":"2023-10-21T17:39:54.496282+0000","last_fullsized":"2023-10-21T17:39:54.496282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:24.596691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506919+0000","last_change":"2023-10-21T17:39:54.520421+0000","last_active":"2023-10-21T17:39:55.506919+0000","last_peered":"2023-10-21T17:39:55.506919+0000","last_clean":"2023-10-21T17:39:55.506919+0000","last_became_active":"2023-10-21T17:39:54.520078+0000","last_became_peered":"2023-10-21T17:39:54.520078+0000","last_unstale":"2023-10-21T17:39:55.506919+0000","last_undegraded":"2023-10-21T17:39:55.506919+0000","last_fullsized":"2023-10-21T17:39:55.506919+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:32.191887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485197+0000","last_change":"2023-10-21T17:39:40.275308+0000","last_active":"2023-10-21T17:39:54.485197+0000","last_peered":"2023-10-21T17:39:54.485197+0000","last_clean":"2023-10-21T17:39:54.485197+0000","last_became_active":"2023-10-21T17:39:40.274998+0000","last_became_peered":"2023-10-21T17:39:40.274998+0000","last_unstale":"2023-10-21T17:39:54.485197+0000","last_undegraded":"2023-10-21T17:39:54.485197+0000","last_fullsized":"2023-10-21T17:39:54.485197+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:38.521395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482593+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:54.482593+0000","last_peered":"2023-10-21T17:39:54.482593+0000","last_clean":"2023-10-21T17:39:54.482593+0000","last_became_active":"2023-10-21T17:39:39.236393+0000","last_became_peered":"2023-10-21T17:39:39.236393+0000","last_unstale":"2023-10-21T17:39:54.482593+0000","last_undegraded":"2023-10-21T17:39:54.482593+0000","last_fullsized":"2023-10-21T17:39:54.482593+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:40.507879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867389+0000","last_change":"2023-10-21T17:39:54.508906+0000","last_active":"2023-10-21T17:39:55.867389+0000","last_peered":"2023-10-21T17:39:55.867389+0000","last_clean":"2023-10-21T17:39:55.867389+0000","last_became_active":"2023-10-21T17:39:54.507753+0000","last_became_peered":"2023-10-21T17:39:54.507753+0000","last_unstale":"2023-10-21T17:39:55.867389+0000","last_undegraded":"2023-10-21T17:39:55.867389+0000","last_fullsized":"2023-10-21T17:39:55.867389+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:16:06.148164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478839+0000","last_change":"2023-10-21T17:39:40.270271+0000","last_active":"2023-10-21T17:39:54.478839+0000","last_peered":"2023-10-21T17:39:54.478839+0000","last_clean":"2023-10-21T17:39:54.478839+0000","last_became_active":"2023-10-21T17:39:40.269983+0000","last_became_peered":"2023-10-21T17:39:40.269983+0000","last_unstale":"2023-10-21T17:39:54.478839+0000","last_undegraded":"2023-10-21T17:39:54.478839+0000","last_fullsized":"2023-10-21T17:39:54.478839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:33.823325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503415+0000","last_change":"2023-10-21T17:39:39.247657+0000","last_active":"2023-10-21T17:39:54.503415+0000","last_peered":"2023-10-21T17:39:54.503415+0000","last_clean":"2023-10-21T17:39:54.503415+0000","last_became_active":"2023-10-21T17:39:39.247152+0000","last_became_peered":"2023-10-21T17:39:39.247152+0000","last_unstale":"2023-10-21T17:39:54.503415+0000","last_undegraded":"2023-10-21T17:39:54.503415+0000","last_fullsized":"2023-10-21T17:39:54.503415+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:46.448339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"239'4895","reported_seq":5903,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497340+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:39:54.497340+0000","last_peered":"2023-10-21T17:39:54.497340+0000","last_clean":"2023-10-21T17:39:54.497340+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:39:54.497340+0000","last_undegraded":"2023-10-21T17:39:54.497340+0000","last_fullsized":"2023-10-21T17:39:54.497340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4895,"ondisk_log_size":4895,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75204,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296636,"statfs":{"total":96636764160,"available":96559755264,"internally_reserved":0,"allocated":3858432,"data_stored":2287981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.135,"15min":1.177},"min":{"1min":0.74299999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1759999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1619999999999999,"15min":1.212},"min":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.615,"5min":1.7170000000000001,"15min":1.73},"last":1.119},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.26,"15min":1.298},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.611,"5min":1.792,"15min":1.885},"last":1.4630000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.3759999999999999,"15min":1.3480000000000001},"min":{"1min":0.78100000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.90700000000000003}]},{"osd":3,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2849999999999999,"15min":1.3129999999999999},"min":{"1min":0.624,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6739999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.294},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4990000000000001,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.361}]},{"osd":4,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4019999999999999,"15min":1.367},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9039999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5129999999999999,"15min":1.492},"min":{"1min":0.76100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.804,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.494}]},{"osd":5,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.508,"15min":1.421},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.837,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.532},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.49,"15min":1.456},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.036,"5min":2.613,"15min":2.613},"last":1.016}]},{"osd":6,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1019999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.4810000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.919,"5min":2.879,"15min":2.879},"last":1.4199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576014,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42692,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329148,"statfs":{"total":96636764160,"available":96593047552,"internally_reserved":0,"allocated":3260416,"data_stored":1697031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.133,"15min":1.1890000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.671,"5min":1.944,"15min":1.944},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.163,"15min":1.212},"min":{"1min":0.71499999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.607,"5min":1.6839999999999999,"15min":1.762},"last":1.3680000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.121,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.474,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.258,"15min":1.302},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5029999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.5289999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.254,"15min":1.3049999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.532,"5min":1.706,"15min":1.8300000000000001},"last":1.244},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.92200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.516,"5min":1.679,"15min":1.929},"last":1.587}]},{"osd":3,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.349,"15min":1.403},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.96,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.766},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4019999999999999,"15min":1.4510000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.5800000000000001,"5min":1.871,"15min":2.048},"last":1.484}]},{"osd":4,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.28,"15min":1.341},"min":{"1min":0.97299999999999998,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.706},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.409,"15min":1.4330000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.806,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.4179999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5489999999999999,"15min":1.5669999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.339},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3260000000000001,"15min":1.369},"min":{"1min":1.0169999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.448}]},{"osd":7,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4650000000000001,"15min":1.526},"min":{"1min":1.012,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.627,"15min":1.663},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0449999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.5409999999999999},"last":1.7450000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576015,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64892,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306948,"statfs":{"total":96636764160,"available":96570314752,"internally_reserved":0,"allocated":3252224,"data_stored":1696955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.1439999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.611,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.161,"15min":1.218},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.2949999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2729999999999999,"15min":1.27},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6719999999999999,"5min":1.738,"15min":1.8480000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1659999999999999,"15min":1.2230000000000001},"min":{"1min":0.74399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.532,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":1.0309999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.802},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.224,"15min":1.244},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.8009999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3080000000000001,"15min":1.343},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.639,"5min":1.758,"15min":1.829},"last":1.101},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.379,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.897},"last":1.5820000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.234,"15min":1.25},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.9910000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.415,"15min":1.4039999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.528}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1.0369999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8360000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.542,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":1.1399999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.52},"last":1.8779999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":0.96199999999999997,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.0249999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":1.639},{"interface":"front","average":{"1min":1.569,"5min":1.5289999999999999,"15min":1.5649999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.1349999999999998},"last":1.772}]}]},{"osd":4,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52416,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319424,"statfs":{"total":96636764160,"available":96583090176,"internally_reserved":0,"allocated":3264512,"data_stored":1698567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.153},"min":{"1min":0.79700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.383,"5min":1.746,"15min":1.8480000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.76500000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5109999999999999,"5min":1.659,"15min":1.869},"last":1.1599999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.58099999999999996},"max":{"1min":1.8380000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.401},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7470000000000001,"5min":1.762,"15min":1.762},"last":1.6000000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.224,"15min":1.254},"min":{"1min":0.77000000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.534},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3420000000000001,"15min":1.3799999999999999},"min":{"1min":0.88700000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.833,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.78}]},{"osd":3,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.236,"15min":1.3049999999999999},"min":{"1min":0.90300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.734,"5min":1.734,"15min":1.7949999999999999},"last":1.246},{"interface":"front","average":{"1min":1.417,"5min":1.389,"15min":1.425},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.728}]},{"osd":5,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.3440000000000001},"min":{"1min":0.93500000000000005,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.911},{"interface":"front","average":{"1min":1.371,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.131,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.1600000000000001},"last":1.96},{"interface":"front","average":{"1min":1.605,"5min":1.5860000000000001,"15min":1.597},"min":{"1min":1.2250000000000001,"5min":1.056,"15min":1.056},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":2.016}]},{"osd":7,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.5249999999999999,"15min":1.5229999999999999},"min":{"1min":0.94699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.587,"15min":1.593},"min":{"1min":1.036,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":2.052}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56592,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315248,"statfs":{"total":96636764160,"available":96578813952,"internally_reserved":0,"allocated":3276800,"data_stored":1700470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.462,"5min":1.6439999999999999,"15min":1.71},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.234,"15min":1.258},"min":{"1min":0.68000000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2350000000000001,"5min":1.835,"15min":1.835},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3240000000000001},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.6870000000000001,"5min":1.879,"15min":1.879},"last":1.091}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0740000000000001,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6539999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2,"5min":1.23,"15min":1.286},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2},"last":1.022}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.403,"15min":1.4370000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.4650000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.169},"last":1.2609999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4239999999999999,"15min":1.4910000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9379999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.92600000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.5900000000000001},"min":{"1min":0.92800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1779999999999999,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.552,"15min":1.6160000000000001},"min":{"1min":0.998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0209999999999999,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.869}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.637,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.921,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.452},{"interface":"front","average":{"1min":1.585,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.038,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1230000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.579}]}]},{"osd":2,"up_from":13,"seq":55834576014,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56592,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315248,"statfs":{"total":96636764160,"available":96578813952,"internally_reserved":0,"allocated":3276800,"data_stored":1699753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93100000000000005,"15min":0.90900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.585,"5min":1.907,"15min":2.0059999999999998},"last":0.96299999999999997}]},{"osd":1,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.321,"15min":1.361},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.585,"5min":1.958,"15min":1.958},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1950000000000001,"15min":1.1930000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.466,"15min":1.4830000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.288}]},{"osd":4,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4510000000000001,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.476},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4470000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.337}]},{"osd":5,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.4059999999999999,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.845,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6240000000000001,"15min":1.613},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.548}]},{"osd":6,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.5309999999999999,"15min":1.5620000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.855,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.5169999999999999,"15min":1.52},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.3799999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.631,"15min":1.6339999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9910000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5780000000000001,"15min":1.593},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0739999999999998,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.52}]}]},{"osd":1,"up_from":13,"seq":55834576013,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52392,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319448,"statfs":{"total":96636764160,"available":96583114752,"internally_reserved":0,"allocated":3244032,"data_stored":1689406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.92700000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.411,"5min":1.411,"15min":1.976},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.878,"5min":1.0980000000000001,"15min":1.147},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.601,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.006,"15min":1.042},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.79},"last":1.363},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.179,"15min":1.226},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5960000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.298,"15min":1.371},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9239999999999999,"5min":2.0539999999999998,"15min":2.129},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.2509999999999999,"15min":1.2929999999999999},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.94199999999999995}]},{"osd":4,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0139999999999998,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.516},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.45,"15min":1.484},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0089999999999999,"5min":2.0750000000000002,"15min":2.383},"last":1.0469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.516,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.98,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.194},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.464,"15min":1.5249999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.1000000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.7869999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.569,"15min":1.595},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.2069999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.4890000000000001,"15min":1.5469999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.927,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.643}]},{"osd":7,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.609,"15min":1.663},"min":{"1min":0.95999999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1349999999999998,"5min":2.1829999999999998,"15min":2.548},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.696,"15min":1.7090000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.282,"5min":2.492,"15min":2.548},"last":1.425}]}]},{"osd":0,"up_from":13,"seq":55834576013,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53000,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318840,"statfs":{"total":96636764160,"available":96582492160,"internally_reserved":0,"allocated":3858432,"data_stored":2290180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.179},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":1.085},"min":{"1min":0.61299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.556,"15min":1.748},"last":0.94499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0980000000000001,"15min":1.163},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.504,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0780000000000001,"15min":1.171},"min":{"1min":0.67500000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4259999999999999,"5min":1.708,"15min":1.851},"last":1.0449999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6359999999999999,"5min":1.897,"15min":1.9590000000000001},"last":1.319},{"interface":"front","average":{"1min":1.242,"5min":1.3140000000000001,"15min":1.3819999999999999},"min":{"1min":0.95599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.702,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.6739999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3340000000000001,"15min":1.421},"min":{"1min":0.877,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9470000000000001,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5289999999999999,"15min":1.5880000000000001},"min":{"1min":1.0620000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.917,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.988}]},{"osd":5,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.389,"15min":1.401},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9239999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.157},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":2.4089999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.82899999999999996,"5min":0.75,"15min":0.75},"max":{"1min":1.7310000000000001,"5min":2.0009999999999999,"15min":2.0139999999999998},"last":1.54},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.609,"15min":1.671},"min":{"1min":1.095,"5min":0.996,"15min":0.996},"max":{"1min":2.0920000000000001,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":2.5259999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4570000000000001,"15min":1.5309999999999999},"min":{"1min":1.002,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9910000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.571,"15min":1.6180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1819999999999999,"5min":2.2519999999999998,"15min":2.262},"last":1.6439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:39:59.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.326+0000 7f19bdba0640 1 Processor -- start 2023-10-21T17:39:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.327+0000 7f19bdba0640 1 -- start start 2023-10-21T17:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bdba0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19b80af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bdba0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b806ba80 con 0x7f19b80cfcc0 2023-10-21T17:39:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bdba0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b806bcd0 con 0x7f19b80ad400 2023-10-21T17:39:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bdba0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80ceba0 con 0x7f19b815ca60 2023-10-21T17:39:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bd39f640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f19b80cfcc0 0x7f19b80ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57734/0 (socket says 172.21.15.53:57734) 2023-10-21T17:39:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.328+0000 7f19bd39f640 1 -- 172.21.15.53:0/4049241502 learned_addr learned my addr 172.21.15.53:0/4049241502 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:59.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19b80af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154752530 0 0) 0x7f19b806ba80 con 0x7f19b80cfcc0 2023-10-21T17:39:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19a4003520 con 0x7f19b80cfcc0 2023-10-21T17:39:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1005847890 0 0) 0x7f19a4003520 con 0x7f19b80cfcc0 2023-10-21T17:39:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 msgr2=0x7f19b80af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19b80af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 >> v1:172.21.15.124:6789/0 conn(0x7f19b80ad400 legacy=0x7f19b80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b806bfb0 con 0x7f19b80cfcc0 2023-10-21T17:39:59.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f19b0001670 con 0x7f19b80cfcc0 2023-10-21T17:39:59.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 >> v1:172.21.15.53:6789/0 conn(0x7f19b80cfcc0 legacy=0x7f19b80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19a40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a4003f20 0x7f19a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80ceba0 con 0x7f19a4003f20 2023-10-21T17:39:59.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a400bb10 con 0x7f19a4004d00 2023-10-21T17:39:59.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.329+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a400bcd0 con 0x7f19b815ca60 2023-10-21T17:39:59.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a4003f20 0x7f19a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 -- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 msgr2=0x7f19a40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19a40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19bd39f640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19a40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 -- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19a4004d00 msgr2=0x7f19a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:59.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19a4004d00 0x7f19a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19b7fff640 1 -- 172.21.15.53:0/4049241502 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4003520 con 0x7f19a4003f20 2023-10-21T17:39:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.330+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19b815ca60 0x7f19a40039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:39:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.331+0000 7f19b7fff640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a4003f20 0x7f19a40047c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f19a0001750 tx=0x7f19a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc47047d43e011c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.331+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a00052d0 con 0x7f19a4003f20 2023-10-21T17:39:59.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.331+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a002e050 con 0x7f19a4003f20 2023-10-21T17:39:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.331+0000 7f19b77fe640 1 -- 172.21.15.53:0/4049241502 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a00035f0 con 0x7f19a4003f20 2023-10-21T17:39:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 -- 172.21.15.53:0/4049241502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a4003f20 msgr2=0x7f19a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 --2- 172.21.15.53:0/4049241502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19a4003f20 0x7f19a40047c0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f19a0001750 tx=0x7f19a40057f0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bd39f640 1 -- 172.21.15.53:0/4049241502 reap_dead start 2023-10-21T17:39:59.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 -- 172.21.15.53:0/4049241502 shutdown_connections 2023-10-21T17:39:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 -- 172.21.15.53:0/4049241502 >> 172.21.15.53:0/4049241502 conn(0x7f19b80d0830 msgr2=0x7f19b80d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 -- 172.21.15.53:0/4049241502 shutdown_connections 2023-10-21T17:39:59.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.332+0000 7f19bdba0640 1 -- 172.21.15.53:0/4049241502 wait complete. 2023-10-21T17:39:59.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.334+0000 7f19bdba0640 1 Processor -- start 2023-10-21T17:39:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.334+0000 7f19bdba0640 1 -- start start 2023-10-21T17:39:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19a4003f20 0x7f19b8142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19b80c70c0 0x7f19b80c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c6c30 con 0x7f19b81f1ae0 2023-10-21T17:39:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c6020 con 0x7f19b80c70c0 2023-10-21T17:39:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bdba0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c6300 con 0x7f19a4003f20 2023-10-21T17:39:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bd39f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bd39f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53256/0 (socket says 172.21.15.53:53256) 2023-10-21T17:39:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.335+0000 7f19bd39f640 1 -- 172.21.15.53:0/1592979734 learned_addr learned my addr 172.21.15.53:0/1592979734 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19a4003f20 0x7f19b8142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 -- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19a4003f20 msgr2=0x7f19b8142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19b7fff640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19b80c70c0 0x7f19b80c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19a4003f20 0x7f19b8142f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 -- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19b80c70c0 msgr2=0x7f19b80c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19b80c70c0 0x7f19b80c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 -- 172.21.15.53:0/1592979734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80c65e0 con 0x7f19b81f1ae0 2023-10-21T17:39:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bd39f640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f19b0000ff0 tx=0x7f19b000d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007440 con 0x7f19b81f1ae0 2023-10-21T17:39:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b00075c0 con 0x7f19b81f1ae0 2023-10-21T17:39:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0004a20 con 0x7f19b81f1ae0 2023-10-21T17:39:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.336+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80c68c0 con 0x7f19b81f1ae0 2023-10-21T17:39:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.337+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b80c5ca0 con 0x7f19b81f1ae0 2023-10-21T17:39:59.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.338+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b804e160 con 0x7f19b81f1ae0 2023-10-21T17:39:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.339+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f19b0004b80 con 0x7f19b81f1ae0 2023-10-21T17:39:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.340+0000 7f19b57fa640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 0x7f199806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.340+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f19b0089b40 con 0x7f19b81f1ae0 2023-10-21T17:39:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.340+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 0x7f199806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.342+0000 7f19bcb9e640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 0x7f199806d570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19a8007520 tx=0x7f19a800d040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.344+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19b0054260 con 0x7f19b81f1ae0 2023-10-21T17:39:59.371 INFO:tasks.ceph.mds.c.smithi053.stderr:2023-10-21T17:39:59.370+0000 7f6fab305640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:39:59.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.535+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19b80cf090 con 0x7f199806b0b0 2023-10-21T17:39:59.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.576+0000 7f19b57fa640 1 -- 172.21.15.53:0/1592979734 <== mgr.38150 v2:172.21.15.124:6832/3816427222 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526182 (secure 0 0 0) 0x7f19a8011040 con 0x7f199806b0b0 2023-10-21T17:39:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:39:59.589 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:39:59.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 msgr2=0x7f199806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 0x7f199806d570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f19a8007520 tx=0x7f19a800d040 comp rx=0 tx=0).stop 2023-10-21T17:39:59.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 msgr2=0x7f19b8142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f19b0000ff0 tx=0x7f19b000d9b0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 shutdown_connections 2023-10-21T17:39:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f199806b0b0 0x7f199806d570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19b81f1ae0 0x7f19b8142370 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19b80c70c0 0x7f19b80c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 --2- 172.21.15.53:0/1592979734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19a4003f20 0x7f19b8142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 >> 172.21.15.53:0/1592979734 conn(0x7f19b80d0830 msgr2=0x7f19b806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:59.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.585+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 shutdown_connections 2023-10-21T17:39:59.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.586+0000 7f19bdba0640 1 -- 172.21.15.53:0/1592979734 wait complete. 2023-10-21T17:39:59.610 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2023-10-21T17:39:58.543569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":718,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4974,"ondisk_log_size":4974,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":328,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453780,"kb_used_data":26652,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754520940,"statfs":{"total":773094113280,"available":772629442560,"internally_reserved":0,"allocated":27291648,"data_stored":14760343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":437243449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002131"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504444+0000","last_change":"2023-10-21T17:39:54.490986+0000","last_active":"2023-10-21T17:39:55.504444+0000","last_peered":"2023-10-21T17:39:55.504444+0000","last_clean":"2023-10-21T17:39:55.504444+0000","last_became_active":"2023-10-21T17:39:54.490465+0000","last_became_peered":"2023-10-21T17:39:54.490465+0000","last_unstale":"2023-10-21T17:39:55.504444+0000","last_undegraded":"2023-10-21T17:39:55.504444+0000","last_fullsized":"2023-10-21T17:39:55.504444+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:04:43.007785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832939+0000","last_change":"2023-10-21T17:39:53.501960+0000","last_active":"2023-10-21T17:39:55.832939+0000","last_peered":"2023-10-21T17:39:55.832939+0000","last_clean":"2023-10-21T17:39:55.832939+0000","last_became_active":"2023-10-21T17:39:53.501683+0000","last_became_peered":"2023-10-21T17:39:53.501683+0000","last_unstale":"2023-10-21T17:39:55.832939+0000","last_undegraded":"2023-10-21T17:39:55.832939+0000","last_fullsized":"2023-10-21T17:39:55.832939+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:34:13.628412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483597+0000","last_change":"2023-10-21T17:39:40.283996+0000","last_active":"2023-10-21T17:39:54.483597+0000","last_peered":"2023-10-21T17:39:54.483597+0000","last_clean":"2023-10-21T17:39:54.483597+0000","last_became_active":"2023-10-21T17:39:40.283690+0000","last_became_peered":"2023-10-21T17:39:40.283690+0000","last_unstale":"2023-10-21T17:39:54.483597+0000","last_undegraded":"2023-10-21T17:39:54.483597+0000","last_fullsized":"2023-10-21T17:39:54.483597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:37:50.486075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832491+0000","last_change":"2023-10-21T17:39:54.513144+0000","last_active":"2023-10-21T17:39:55.832491+0000","last_peered":"2023-10-21T17:39:55.832491+0000","last_clean":"2023-10-21T17:39:55.832491+0000","last_became_active":"2023-10-21T17:39:54.512836+0000","last_became_peered":"2023-10-21T17:39:54.512836+0000","last_unstale":"2023-10-21T17:39:55.832491+0000","last_undegraded":"2023-10-21T17:39:55.832491+0000","last_fullsized":"2023-10-21T17:39:55.832491+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:59.066123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"257'4","reported_seq":16,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504600+0000","last_change":"2023-10-21T17:39:53.504447+0000","last_active":"2023-10-21T17:39:55.504600+0000","last_peered":"2023-10-21T17:39:55.504600+0000","last_clean":"2023-10-21T17:39:55.504600+0000","last_became_active":"2023-10-21T17:39:53.504179+0000","last_became_peered":"2023-10-21T17:39:53.504179+0000","last_unstale":"2023-10-21T17:39:55.504600+0000","last_undegraded":"2023-10-21T17:39:55.504600+0000","last_fullsized":"2023-10-21T17:39:55.504600+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:36.904495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489453+0000","last_change":"2023-10-21T17:39:40.279882+0000","last_active":"2023-10-21T17:39:54.489453+0000","last_peered":"2023-10-21T17:39:54.489453+0000","last_clean":"2023-10-21T17:39:54.489453+0000","last_became_active":"2023-10-21T17:39:40.279568+0000","last_became_peered":"2023-10-21T17:39:40.279568+0000","last_unstale":"2023-10-21T17:39:54.489453+0000","last_undegraded":"2023-10-21T17:39:54.489453+0000","last_fullsized":"2023-10-21T17:39:54.489453+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:07.488724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.871394+0000","last_change":"2023-10-21T17:39:54.521730+0000","last_active":"2023-10-21T17:39:55.871394+0000","last_peered":"2023-10-21T17:39:55.871394+0000","last_clean":"2023-10-21T17:39:55.871394+0000","last_became_active":"2023-10-21T17:39:54.519697+0000","last_became_peered":"2023-10-21T17:39:54.519697+0000","last_unstale":"2023-10-21T17:39:55.871394+0000","last_undegraded":"2023-10-21T17:39:55.871394+0000","last_fullsized":"2023-10-21T17:39:55.871394+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:08.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832220+0000","last_change":"2023-10-21T17:39:53.499872+0000","last_active":"2023-10-21T17:39:55.832220+0000","last_peered":"2023-10-21T17:39:55.832220+0000","last_clean":"2023-10-21T17:39:55.832220+0000","last_became_active":"2023-10-21T17:39:53.499116+0000","last_became_peered":"2023-10-21T17:39:53.499116+0000","last_unstale":"2023-10-21T17:39:55.832220+0000","last_undegraded":"2023-10-21T17:39:55.832220+0000","last_fullsized":"2023-10-21T17:39:55.832220+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:33:30.710008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.495450+0000","last_change":"2023-10-21T17:39:40.277265+0000","last_active":"2023-10-21T17:39:54.495450+0000","last_peered":"2023-10-21T17:39:54.495450+0000","last_clean":"2023-10-21T17:39:54.495450+0000","last_became_active":"2023-10-21T17:39:40.276998+0000","last_became_peered":"2023-10-21T17:39:40.276998+0000","last_unstale":"2023-10-21T17:39:54.495450+0000","last_undegraded":"2023-10-21T17:39:54.495450+0000","last_fullsized":"2023-10-21T17:39:54.495450+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:14.379177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504700+0000","last_change":"2023-10-21T17:39:54.494099+0000","last_active":"2023-10-21T17:39:55.504700+0000","last_peered":"2023-10-21T17:39:55.504700+0000","last_clean":"2023-10-21T17:39:55.504700+0000","last_became_active":"2023-10-21T17:39:54.493629+0000","last_became_peered":"2023-10-21T17:39:54.493629+0000","last_unstale":"2023-10-21T17:39:55.504700+0000","last_undegraded":"2023-10-21T17:39:55.504700+0000","last_fullsized":"2023-10-21T17:39:55.504700+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:46:58.263674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509742+0000","last_change":"2023-10-21T17:39:53.497524+0000","last_active":"2023-10-21T17:39:55.509742+0000","last_peered":"2023-10-21T17:39:55.509742+0000","last_clean":"2023-10-21T17:39:55.509742+0000","last_became_active":"2023-10-21T17:39:53.496700+0000","last_became_peered":"2023-10-21T17:39:53.496700+0000","last_unstale":"2023-10-21T17:39:55.509742+0000","last_undegraded":"2023-10-21T17:39:55.509742+0000","last_fullsized":"2023-10-21T17:39:55.509742+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:11:03.008674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499044+0000","last_change":"2023-10-21T17:39:40.278307+0000","last_active":"2023-10-21T17:39:54.499044+0000","last_peered":"2023-10-21T17:39:54.499044+0000","last_clean":"2023-10-21T17:39:54.499044+0000","last_became_active":"2023-10-21T17:39:40.277780+0000","last_became_peered":"2023-10-21T17:39:40.277780+0000","last_unstale":"2023-10-21T17:39:54.499044+0000","last_undegraded":"2023-10-21T17:39:54.499044+0000","last_fullsized":"2023-10-21T17:39:54.499044+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:24:14.855269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869695+0000","last_change":"2023-10-21T17:39:54.516346+0000","last_active":"2023-10-21T17:39:55.869695+0000","last_peered":"2023-10-21T17:39:55.869695+0000","last_clean":"2023-10-21T17:39:55.869695+0000","last_became_active":"2023-10-21T17:39:54.514717+0000","last_became_peered":"2023-10-21T17:39:54.514717+0000","last_unstale":"2023-10-21T17:39:55.869695+0000","last_undegraded":"2023-10-21T17:39:55.869695+0000","last_fullsized":"2023-10-21T17:39:55.869695+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:03.232494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870736+0000","last_change":"2023-10-21T17:39:53.507641+0000","last_active":"2023-10-21T17:39:55.870736+0000","last_peered":"2023-10-21T17:39:55.870736+0000","last_clean":"2023-10-21T17:39:55.870736+0000","last_became_active":"2023-10-21T17:39:53.507321+0000","last_became_peered":"2023-10-21T17:39:53.507321+0000","last_unstale":"2023-10-21T17:39:55.870736+0000","last_undegraded":"2023-10-21T17:39:55.870736+0000","last_fullsized":"2023-10-21T17:39:55.870736+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:52.644827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501560+0000","last_change":"2023-10-21T17:39:40.277999+0000","last_active":"2023-10-21T17:39:54.501560+0000","last_peered":"2023-10-21T17:39:54.501560+0000","last_clean":"2023-10-21T17:39:54.501560+0000","last_became_active":"2023-10-21T17:39:40.277716+0000","last_became_peered":"2023-10-21T17:39:40.277716+0000","last_unstale":"2023-10-21T17:39:54.501560+0000","last_undegraded":"2023-10-21T17:39:54.501560+0000","last_fullsized":"2023-10-21T17:39:54.501560+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:44.306053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869044+0000","last_change":"2023-10-21T17:39:54.515865+0000","last_active":"2023-10-21T17:39:55.869044+0000","last_peered":"2023-10-21T17:39:55.869044+0000","last_clean":"2023-10-21T17:39:55.869044+0000","last_became_active":"2023-10-21T17:39:54.513264+0000","last_became_peered":"2023-10-21T17:39:54.513264+0000","last_unstale":"2023-10-21T17:39:55.869044+0000","last_undegraded":"2023-10-21T17:39:55.869044+0000","last_fullsized":"2023-10-21T17:39:55.869044+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:10:09.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869713+0000","last_change":"2023-10-21T17:39:53.507072+0000","last_active":"2023-10-21T17:39:55.869713+0000","last_peered":"2023-10-21T17:39:55.869713+0000","last_clean":"2023-10-21T17:39:55.869713+0000","last_became_active":"2023-10-21T17:39:53.506740+0000","last_became_peered":"2023-10-21T17:39:53.506740+0000","last_unstale":"2023-10-21T17:39:55.869713+0000","last_undegraded":"2023-10-21T17:39:55.869713+0000","last_fullsized":"2023-10-21T17:39:55.869713+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:48:02.855105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479821+0000","last_change":"2023-10-21T17:39:40.274132+0000","last_active":"2023-10-21T17:39:54.479821+0000","last_peered":"2023-10-21T17:39:54.479821+0000","last_clean":"2023-10-21T17:39:54.479821+0000","last_became_active":"2023-10-21T17:39:40.273837+0000","last_became_peered":"2023-10-21T17:39:40.273837+0000","last_unstale":"2023-10-21T17:39:54.479821+0000","last_undegraded":"2023-10-21T17:39:54.479821+0000","last_fullsized":"2023-10-21T17:39:54.479821+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:53.620703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867613+0000","last_change":"2023-10-21T17:39:54.510342+0000","last_active":"2023-10-21T17:39:55.867613+0000","last_peered":"2023-10-21T17:39:55.867613+0000","last_clean":"2023-10-21T17:39:55.867613+0000","last_became_active":"2023-10-21T17:39:54.508631+0000","last_became_peered":"2023-10-21T17:39:54.508631+0000","last_unstale":"2023-10-21T17:39:55.867613+0000","last_undegraded":"2023-10-21T17:39:55.867613+0000","last_fullsized":"2023-10-21T17:39:55.867613+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:08:13.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867247+0000","last_change":"2023-10-21T17:39:53.498530+0000","last_active":"2023-10-21T17:39:55.867247+0000","last_peered":"2023-10-21T17:39:55.867247+0000","last_clean":"2023-10-21T17:39:55.867247+0000","last_became_active":"2023-10-21T17:39:53.498139+0000","last_became_peered":"2023-10-21T17:39:53.498139+0000","last_unstale":"2023-10-21T17:39:55.867247+0000","last_undegraded":"2023-10-21T17:39:55.867247+0000","last_fullsized":"2023-10-21T17:39:55.867247+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:49.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500633+0000","last_change":"2023-10-21T17:39:40.277476+0000","last_active":"2023-10-21T17:39:54.500633+0000","last_peered":"2023-10-21T17:39:54.500633+0000","last_clean":"2023-10-21T17:39:54.500633+0000","last_became_active":"2023-10-21T17:39:40.277175+0000","last_became_peered":"2023-10-21T17:39:40.277175+0000","last_unstale":"2023-10-21T17:39:54.500633+0000","last_undegraded":"2023-10-21T17:39:54.500633+0000","last_fullsized":"2023-10-21T17:39:54.500633+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:09.554848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.505660+0000","last_change":"2023-10-21T17:39:54.520767+0000","last_active":"2023-10-21T17:39:55.505660+0000","last_peered":"2023-10-21T17:39:55.505660+0000","last_clean":"2023-10-21T17:39:55.505660+0000","last_became_active":"2023-10-21T17:39:54.520506+0000","last_became_peered":"2023-10-21T17:39:54.520506+0000","last_unstale":"2023-10-21T17:39:55.505660+0000","last_undegraded":"2023-10-21T17:39:55.505660+0000","last_fullsized":"2023-10-21T17:39:55.505660+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:20.811972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.831153+0000","last_change":"2023-10-21T17:39:53.506244+0000","last_active":"2023-10-21T17:39:55.831153+0000","last_peered":"2023-10-21T17:39:55.831153+0000","last_clean":"2023-10-21T17:39:55.831153+0000","last_became_active":"2023-10-21T17:39:53.505972+0000","last_became_peered":"2023-10-21T17:39:53.505972+0000","last_unstale":"2023-10-21T17:39:55.831153+0000","last_undegraded":"2023-10-21T17:39:55.831153+0000","last_fullsized":"2023-10-21T17:39:55.831153+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:50:40.003316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501597+0000","last_change":"2023-10-21T17:39:40.274697+0000","last_active":"2023-10-21T17:39:54.501597+0000","last_peered":"2023-10-21T17:39:54.501597+0000","last_clean":"2023-10-21T17:39:54.501597+0000","last_became_active":"2023-10-21T17:39:40.274194+0000","last_became_peered":"2023-10-21T17:39:40.274194+0000","last_unstale":"2023-10-21T17:39:54.501597+0000","last_undegraded":"2023-10-21T17:39:54.501597+0000","last_fullsized":"2023-10-21T17:39:54.501597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:17.782806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504408+0000","last_change":"2023-10-21T17:39:54.509960+0000","last_active":"2023-10-21T17:39:55.504408+0000","last_peered":"2023-10-21T17:39:55.504408+0000","last_clean":"2023-10-21T17:39:55.504408+0000","last_became_active":"2023-10-21T17:39:54.509633+0000","last_became_peered":"2023-10-21T17:39:54.509633+0000","last_unstale":"2023-10-21T17:39:55.504408+0000","last_undegraded":"2023-10-21T17:39:55.504408+0000","last_fullsized":"2023-10-21T17:39:55.504408+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:10:31.680455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.833133+0000","last_change":"2023-10-21T17:39:53.507648+0000","last_active":"2023-10-21T17:39:55.833133+0000","last_peered":"2023-10-21T17:39:55.833133+0000","last_clean":"2023-10-21T17:39:55.833133+0000","last_became_active":"2023-10-21T17:39:53.507371+0000","last_became_peered":"2023-10-21T17:39:53.507371+0000","last_unstale":"2023-10-21T17:39:55.833133+0000","last_undegraded":"2023-10-21T17:39:55.833133+0000","last_fullsized":"2023-10-21T17:39:55.833133+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:04:41.283912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484092+0000","last_change":"2023-10-21T17:39:40.278661+0000","last_active":"2023-10-21T17:39:54.484092+0000","last_peered":"2023-10-21T17:39:54.484092+0000","last_clean":"2023-10-21T17:39:54.484092+0000","last_became_active":"2023-10-21T17:39:40.278377+0000","last_became_peered":"2023-10-21T17:39:40.278377+0000","last_unstale":"2023-10-21T17:39:54.484092+0000","last_undegraded":"2023-10-21T17:39:54.484092+0000","last_fullsized":"2023-10-21T17:39:54.484092+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:10:48.303305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867351+0000","last_change":"2023-10-21T17:39:54.508518+0000","last_active":"2023-10-21T17:39:55.867351+0000","last_peered":"2023-10-21T17:39:55.867351+0000","last_clean":"2023-10-21T17:39:55.867351+0000","last_became_active":"2023-10-21T17:39:54.508204+0000","last_became_peered":"2023-10-21T17:39:54.508204+0000","last_unstale":"2023-10-21T17:39:55.867351+0000","last_undegraded":"2023-10-21T17:39:55.867351+0000","last_fullsized":"2023-10-21T17:39:55.867351+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:28.194036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867988+0000","last_change":"2023-10-21T17:39:53.506000+0000","last_active":"2023-10-21T17:39:55.867988+0000","last_peered":"2023-10-21T17:39:55.867988+0000","last_clean":"2023-10-21T17:39:55.867988+0000","last_became_active":"2023-10-21T17:39:53.503681+0000","last_became_peered":"2023-10-21T17:39:53.503681+0000","last_unstale":"2023-10-21T17:39:55.867988+0000","last_undegraded":"2023-10-21T17:39:55.867988+0000","last_fullsized":"2023-10-21T17:39:55.867988+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:27:51.117136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497638+0000","last_change":"2023-10-21T17:39:40.284923+0000","last_active":"2023-10-21T17:39:54.497638+0000","last_peered":"2023-10-21T17:39:54.497638+0000","last_clean":"2023-10-21T17:39:54.497638+0000","last_became_active":"2023-10-21T17:39:40.281089+0000","last_became_peered":"2023-10-21T17:39:40.281089+0000","last_unstale":"2023-10-21T17:39:54.497638+0000","last_undegraded":"2023-10-21T17:39:54.497638+0000","last_fullsized":"2023-10-21T17:39:54.497638+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:54:15.736471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510430+0000","last_change":"2023-10-21T17:39:54.505970+0000","last_active":"2023-10-21T17:39:55.510430+0000","last_peered":"2023-10-21T17:39:55.510430+0000","last_clean":"2023-10-21T17:39:55.510430+0000","last_became_active":"2023-10-21T17:39:54.503538+0000","last_became_peered":"2023-10-21T17:39:54.503538+0000","last_unstale":"2023-10-21T17:39:55.510430+0000","last_undegraded":"2023-10-21T17:39:55.510430+0000","last_fullsized":"2023-10-21T17:39:55.510430+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:25:14.685731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"257'6","reported_seq":18,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504186+0000","last_change":"2023-10-21T17:39:53.491144+0000","last_active":"2023-10-21T17:39:55.504186+0000","last_peered":"2023-10-21T17:39:55.504186+0000","last_clean":"2023-10-21T17:39:55.504186+0000","last_became_active":"2023-10-21T17:39:53.490763+0000","last_became_peered":"2023-10-21T17:39:53.490763+0000","last_unstale":"2023-10-21T17:39:55.504186+0000","last_undegraded":"2023-10-21T17:39:55.504186+0000","last_fullsized":"2023-10-21T17:39:55.504186+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:42:54.016567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.495574+0000","last_change":"2023-10-21T17:39:40.280260+0000","last_active":"2023-10-21T17:39:54.495574+0000","last_peered":"2023-10-21T17:39:54.495574+0000","last_clean":"2023-10-21T17:39:54.495574+0000","last_became_active":"2023-10-21T17:39:40.279970+0000","last_became_peered":"2023-10-21T17:39:40.279970+0000","last_unstale":"2023-10-21T17:39:54.495574+0000","last_undegraded":"2023-10-21T17:39:54.495574+0000","last_fullsized":"2023-10-21T17:39:54.495574+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:02:09.241928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867873+0000","last_change":"2023-10-21T17:39:54.513118+0000","last_active":"2023-10-21T17:39:55.867873+0000","last_peered":"2023-10-21T17:39:55.867873+0000","last_clean":"2023-10-21T17:39:55.867873+0000","last_became_active":"2023-10-21T17:39:54.512866+0000","last_became_peered":"2023-10-21T17:39:54.512866+0000","last_unstale":"2023-10-21T17:39:55.867873+0000","last_undegraded":"2023-10-21T17:39:55.867873+0000","last_fullsized":"2023-10-21T17:39:55.867873+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:54:10.798745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"257'3","reported_seq":15,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866439+0000","last_change":"2023-10-21T17:39:53.506783+0000","last_active":"2023-10-21T17:39:55.866439+0000","last_peered":"2023-10-21T17:39:55.866439+0000","last_clean":"2023-10-21T17:39:55.866439+0000","last_became_active":"2023-10-21T17:39:53.503658+0000","last_became_peered":"2023-10-21T17:39:53.503658+0000","last_unstale":"2023-10-21T17:39:55.866439+0000","last_undegraded":"2023-10-21T17:39:55.866439+0000","last_fullsized":"2023-10-21T17:39:55.866439+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:02:15.658186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478808+0000","last_change":"2023-10-21T17:39:40.277448+0000","last_active":"2023-10-21T17:39:54.478808+0000","last_peered":"2023-10-21T17:39:54.478808+0000","last_clean":"2023-10-21T17:39:54.478808+0000","last_became_active":"2023-10-21T17:39:40.277121+0000","last_became_peered":"2023-10-21T17:39:40.277121+0000","last_unstale":"2023-10-21T17:39:54.478808+0000","last_undegraded":"2023-10-21T17:39:54.478808+0000","last_fullsized":"2023-10-21T17:39:54.478808+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:13:11.317461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510152+0000","last_change":"2023-10-21T17:39:54.508131+0000","last_active":"2023-10-21T17:39:55.510152+0000","last_peered":"2023-10-21T17:39:55.510152+0000","last_clean":"2023-10-21T17:39:55.510152+0000","last_became_active":"2023-10-21T17:39:54.507875+0000","last_became_peered":"2023-10-21T17:39:54.507875+0000","last_unstale":"2023-10-21T17:39:55.510152+0000","last_undegraded":"2023-10-21T17:39:55.510152+0000","last_fullsized":"2023-10-21T17:39:55.510152+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:51.425048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866450+0000","last_change":"2023-10-21T17:39:53.503802+0000","last_active":"2023-10-21T17:39:55.866450+0000","last_peered":"2023-10-21T17:39:55.866450+0000","last_clean":"2023-10-21T17:39:55.866450+0000","last_became_active":"2023-10-21T17:39:53.503435+0000","last_became_peered":"2023-10-21T17:39:53.503435+0000","last_unstale":"2023-10-21T17:39:55.866450+0000","last_undegraded":"2023-10-21T17:39:55.866450+0000","last_fullsized":"2023-10-21T17:39:55.866450+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:18:13.675672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483318+0000","last_change":"2023-10-21T17:39:40.279922+0000","last_active":"2023-10-21T17:39:54.483318+0000","last_peered":"2023-10-21T17:39:54.483318+0000","last_clean":"2023-10-21T17:39:54.483318+0000","last_became_active":"2023-10-21T17:39:40.279029+0000","last_became_peered":"2023-10-21T17:39:40.279029+0000","last_unstale":"2023-10-21T17:39:54.483318+0000","last_undegraded":"2023-10-21T17:39:54.483318+0000","last_fullsized":"2023-10-21T17:39:54.483318+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:22:13.093898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867229+0000","last_change":"2023-10-21T17:39:54.509200+0000","last_active":"2023-10-21T17:39:55.867229+0000","last_peered":"2023-10-21T17:39:55.867229+0000","last_clean":"2023-10-21T17:39:55.867229+0000","last_became_active":"2023-10-21T17:39:54.508841+0000","last_became_peered":"2023-10-21T17:39:54.508841+0000","last_unstale":"2023-10-21T17:39:55.867229+0000","last_undegraded":"2023-10-21T17:39:55.867229+0000","last_fullsized":"2023-10-21T17:39:55.867229+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:23.840619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869197+0000","last_change":"2023-10-21T17:39:53.505087+0000","last_active":"2023-10-21T17:39:55.869197+0000","last_peered":"2023-10-21T17:39:55.869197+0000","last_clean":"2023-10-21T17:39:55.869197+0000","last_became_active":"2023-10-21T17:39:53.504790+0000","last_became_peered":"2023-10-21T17:39:53.504790+0000","last_unstale":"2023-10-21T17:39:55.869197+0000","last_undegraded":"2023-10-21T17:39:55.869197+0000","last_fullsized":"2023-10-21T17:39:55.869197+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:52.644827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.508959+0000","last_change":"2023-10-21T17:39:40.260107+0000","last_active":"2023-10-21T17:39:54.508959+0000","last_peered":"2023-10-21T17:39:54.508959+0000","last_clean":"2023-10-21T17:39:54.508959+0000","last_became_active":"2023-10-21T17:39:40.259164+0000","last_became_peered":"2023-10-21T17:39:40.259164+0000","last_unstale":"2023-10-21T17:39:54.508959+0000","last_undegraded":"2023-10-21T17:39:54.508959+0000","last_fullsized":"2023-10-21T17:39:54.508959+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:53.185535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869024+0000","last_change":"2023-10-21T17:39:54.515983+0000","last_active":"2023-10-21T17:39:55.869024+0000","last_peered":"2023-10-21T17:39:55.869024+0000","last_clean":"2023-10-21T17:39:55.869024+0000","last_became_active":"2023-10-21T17:39:54.515693+0000","last_became_peered":"2023-10-21T17:39:54.515693+0000","last_unstale":"2023-10-21T17:39:55.869024+0000","last_undegraded":"2023-10-21T17:39:55.869024+0000","last_fullsized":"2023-10-21T17:39:55.869024+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:26.912523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"257'8","reported_seq":20,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865395+0000","last_change":"2023-10-21T17:39:53.505646+0000","last_active":"2023-10-21T17:39:55.865395+0000","last_peered":"2023-10-21T17:39:55.865395+0000","last_clean":"2023-10-21T17:39:55.865395+0000","last_became_active":"2023-10-21T17:39:53.500640+0000","last_became_peered":"2023-10-21T17:39:53.500640+0000","last_unstale":"2023-10-21T17:39:55.865395+0000","last_undegraded":"2023-10-21T17:39:55.865395+0000","last_fullsized":"2023-10-21T17:39:55.865395+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:01:17.968217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497608+0000","last_change":"2023-10-21T17:39:40.278320+0000","last_active":"2023-10-21T17:39:54.497608+0000","last_peered":"2023-10-21T17:39:54.497608+0000","last_clean":"2023-10-21T17:39:54.497608+0000","last_became_active":"2023-10-21T17:39:40.277166+0000","last_became_peered":"2023-10-21T17:39:40.277166+0000","last_unstale":"2023-10-21T17:39:54.497608+0000","last_undegraded":"2023-10-21T17:39:54.497608+0000","last_fullsized":"2023-10-21T17:39:54.497608+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:18:25.386838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867886+0000","last_change":"2023-10-21T17:39:54.514722+0000","last_active":"2023-10-21T17:39:55.867886+0000","last_peered":"2023-10-21T17:39:55.867886+0000","last_clean":"2023-10-21T17:39:55.867886+0000","last_became_active":"2023-10-21T17:39:54.513397+0000","last_became_peered":"2023-10-21T17:39:54.513397+0000","last_unstale":"2023-10-21T17:39:55.867886+0000","last_undegraded":"2023-10-21T17:39:55.867886+0000","last_fullsized":"2023-10-21T17:39:55.867886+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:43:03.232494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866216+0000","last_change":"2023-10-21T17:39:53.497481+0000","last_active":"2023-10-21T17:39:55.866216+0000","last_peered":"2023-10-21T17:39:55.866216+0000","last_clean":"2023-10-21T17:39:55.866216+0000","last_became_active":"2023-10-21T17:39:53.494746+0000","last_became_peered":"2023-10-21T17:39:53.494746+0000","last_unstale":"2023-10-21T17:39:55.866216+0000","last_undegraded":"2023-10-21T17:39:55.866216+0000","last_fullsized":"2023-10-21T17:39:55.866216+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:47:45.315502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496336+0000","last_change":"2023-10-21T17:39:40.285094+0000","last_active":"2023-10-21T17:39:54.496336+0000","last_peered":"2023-10-21T17:39:54.496336+0000","last_clean":"2023-10-21T17:39:54.496336+0000","last_became_active":"2023-10-21T17:39:40.284749+0000","last_became_peered":"2023-10-21T17:39:40.284749+0000","last_unstale":"2023-10-21T17:39:54.496336+0000","last_undegraded":"2023-10-21T17:39:54.496336+0000","last_fullsized":"2023-10-21T17:39:54.496336+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:32.664957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509769+0000","last_change":"2023-10-21T17:39:54.514819+0000","last_active":"2023-10-21T17:39:55.509769+0000","last_peered":"2023-10-21T17:39:55.509769+0000","last_clean":"2023-10-21T17:39:55.509769+0000","last_became_active":"2023-10-21T17:39:54.514490+0000","last_became_peered":"2023-10-21T17:39:54.514490+0000","last_unstale":"2023-10-21T17:39:55.509769+0000","last_undegraded":"2023-10-21T17:39:55.509769+0000","last_fullsized":"2023-10-21T17:39:55.509769+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:08:58.557859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504691+0000","last_change":"2023-10-21T17:39:53.507652+0000","last_active":"2023-10-21T17:39:55.504691+0000","last_peered":"2023-10-21T17:39:55.504691+0000","last_clean":"2023-10-21T17:39:55.504691+0000","last_became_active":"2023-10-21T17:39:53.507367+0000","last_became_peered":"2023-10-21T17:39:53.507367+0000","last_unstale":"2023-10-21T17:39:55.504691+0000","last_undegraded":"2023-10-21T17:39:55.504691+0000","last_fullsized":"2023-10-21T17:39:55.504691+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:17:59.353833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485435+0000","last_change":"2023-10-21T17:39:40.283462+0000","last_active":"2023-10-21T17:39:54.485435+0000","last_peered":"2023-10-21T17:39:54.485435+0000","last_clean":"2023-10-21T17:39:54.485435+0000","last_became_active":"2023-10-21T17:39:40.281926+0000","last_became_peered":"2023-10-21T17:39:40.281926+0000","last_unstale":"2023-10-21T17:39:54.485435+0000","last_undegraded":"2023-10-21T17:39:54.485435+0000","last_fullsized":"2023-10-21T17:39:54.485435+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:42:34.391024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508310+0000","last_change":"2023-10-21T17:39:54.510345+0000","last_active":"2023-10-21T17:39:55.508310+0000","last_peered":"2023-10-21T17:39:55.508310+0000","last_clean":"2023-10-21T17:39:55.508310+0000","last_became_active":"2023-10-21T17:39:54.510078+0000","last_became_peered":"2023-10-21T17:39:54.510078+0000","last_unstale":"2023-10-21T17:39:55.508310+0000","last_undegraded":"2023-10-21T17:39:55.508310+0000","last_fullsized":"2023-10-21T17:39:55.508310+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:22.489131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868163+0000","last_change":"2023-10-21T17:39:53.510589+0000","last_active":"2023-10-21T17:39:55.868163+0000","last_peered":"2023-10-21T17:39:55.868163+0000","last_clean":"2023-10-21T17:39:55.868163+0000","last_became_active":"2023-10-21T17:39:53.510298+0000","last_became_peered":"2023-10-21T17:39:53.510298+0000","last_unstale":"2023-10-21T17:39:55.868163+0000","last_undegraded":"2023-10-21T17:39:55.868163+0000","last_fullsized":"2023-10-21T17:39:55.868163+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:00:19.793960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497679+0000","last_change":"2023-10-21T17:39:40.280739+0000","last_active":"2023-10-21T17:39:54.497679+0000","last_peered":"2023-10-21T17:39:54.497679+0000","last_clean":"2023-10-21T17:39:54.497679+0000","last_became_active":"2023-10-21T17:39:40.280069+0000","last_became_peered":"2023-10-21T17:39:40.280069+0000","last_unstale":"2023-10-21T17:39:54.497679+0000","last_undegraded":"2023-10-21T17:39:54.497679+0000","last_fullsized":"2023-10-21T17:39:54.497679+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:19.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510862+0000","last_change":"2023-10-21T17:39:54.523357+0000","last_active":"2023-10-21T17:39:55.510862+0000","last_peered":"2023-10-21T17:39:55.510862+0000","last_clean":"2023-10-21T17:39:55.510862+0000","last_became_active":"2023-10-21T17:39:54.523016+0000","last_became_peered":"2023-10-21T17:39:54.523016+0000","last_unstale":"2023-10-21T17:39:55.510862+0000","last_undegraded":"2023-10-21T17:39:55.510862+0000","last_fullsized":"2023-10-21T17:39:55.510862+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:37:46.615117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504189+0000","last_change":"2023-10-21T17:39:53.507060+0000","last_active":"2023-10-21T17:39:55.504189+0000","last_peered":"2023-10-21T17:39:55.504189+0000","last_clean":"2023-10-21T17:39:55.504189+0000","last_became_active":"2023-10-21T17:39:53.506751+0000","last_became_peered":"2023-10-21T17:39:53.506751+0000","last_unstale":"2023-10-21T17:39:55.504189+0000","last_undegraded":"2023-10-21T17:39:55.504189+0000","last_fullsized":"2023-10-21T17:39:55.504189+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:02:24.020377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.505125+0000","last_change":"2023-10-21T17:39:40.285848+0000","last_active":"2023-10-21T17:39:54.505125+0000","last_peered":"2023-10-21T17:39:54.505125+0000","last_clean":"2023-10-21T17:39:54.505125+0000","last_became_active":"2023-10-21T17:39:40.285405+0000","last_became_peered":"2023-10-21T17:39:40.285405+0000","last_unstale":"2023-10-21T17:39:54.505125+0000","last_undegraded":"2023-10-21T17:39:54.505125+0000","last_fullsized":"2023-10-21T17:39:54.505125+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868777+0000","last_change":"2023-10-21T17:39:54.491851+0000","last_active":"2023-10-21T17:39:55.868777+0000","last_peered":"2023-10-21T17:39:55.868777+0000","last_clean":"2023-10-21T17:39:55.868777+0000","last_became_active":"2023-10-21T17:39:54.491379+0000","last_became_peered":"2023-10-21T17:39:54.491379+0000","last_unstale":"2023-10-21T17:39:55.868777+0000","last_undegraded":"2023-10-21T17:39:55.868777+0000","last_fullsized":"2023-10-21T17:39:55.868777+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:26:31.568634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"257'1","reported_seq":13,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506954+0000","last_change":"2023-10-21T17:39:53.502790+0000","last_active":"2023-10-21T17:39:55.506954+0000","last_peered":"2023-10-21T17:39:55.506954+0000","last_clean":"2023-10-21T17:39:55.506954+0000","last_became_active":"2023-10-21T17:39:53.500918+0000","last_became_peered":"2023-10-21T17:39:53.500918+0000","last_unstale":"2023-10-21T17:39:55.506954+0000","last_undegraded":"2023-10-21T17:39:55.506954+0000","last_fullsized":"2023-10-21T17:39:55.506954+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:19.527820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484553+0000","last_change":"2023-10-21T17:39:40.284648+0000","last_active":"2023-10-21T17:39:54.484553+0000","last_peered":"2023-10-21T17:39:54.484553+0000","last_clean":"2023-10-21T17:39:54.484553+0000","last_became_active":"2023-10-21T17:39:40.284214+0000","last_became_peered":"2023-10-21T17:39:40.284214+0000","last_unstale":"2023-10-21T17:39:54.484553+0000","last_undegraded":"2023-10-21T17:39:54.484553+0000","last_fullsized":"2023-10-21T17:39:54.484553+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:36:52.495370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.507313+0000","last_change":"2023-10-21T17:39:54.495077+0000","last_active":"2023-10-21T17:39:55.507313+0000","last_peered":"2023-10-21T17:39:55.507313+0000","last_clean":"2023-10-21T17:39:55.507313+0000","last_became_active":"2023-10-21T17:39:54.494437+0000","last_became_peered":"2023-10-21T17:39:54.494437+0000","last_unstale":"2023-10-21T17:39:55.507313+0000","last_undegraded":"2023-10-21T17:39:55.507313+0000","last_fullsized":"2023-10-21T17:39:55.507313+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:52.634139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868130+0000","last_change":"2023-10-21T17:39:53.503144+0000","last_active":"2023-10-21T17:39:55.868130+0000","last_peered":"2023-10-21T17:39:55.868130+0000","last_clean":"2023-10-21T17:39:55.868130+0000","last_became_active":"2023-10-21T17:39:53.502799+0000","last_became_peered":"2023-10-21T17:39:53.502799+0000","last_unstale":"2023-10-21T17:39:55.868130+0000","last_undegraded":"2023-10-21T17:39:55.868130+0000","last_fullsized":"2023-10-21T17:39:55.868130+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:36.731729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504174+0000","last_change":"2023-10-21T17:39:40.270709+0000","last_active":"2023-10-21T17:39:54.504174+0000","last_peered":"2023-10-21T17:39:54.504174+0000","last_clean":"2023-10-21T17:39:54.504174+0000","last_became_active":"2023-10-21T17:39:40.270369+0000","last_became_peered":"2023-10-21T17:39:40.270369+0000","last_unstale":"2023-10-21T17:39:54.504174+0000","last_undegraded":"2023-10-21T17:39:54.504174+0000","last_fullsized":"2023-10-21T17:39:54.504174+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:14:40.862214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869379+0000","last_change":"2023-10-21T17:39:54.512950+0000","last_active":"2023-10-21T17:39:55.869379+0000","last_peered":"2023-10-21T17:39:55.869379+0000","last_clean":"2023-10-21T17:39:55.869379+0000","last_became_active":"2023-10-21T17:39:54.511870+0000","last_became_peered":"2023-10-21T17:39:54.511870+0000","last_unstale":"2023-10-21T17:39:55.869379+0000","last_undegraded":"2023-10-21T17:39:55.869379+0000","last_fullsized":"2023-10-21T17:39:55.869379+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:41:23.840619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869408+0000","last_change":"2023-10-21T17:39:53.510949+0000","last_active":"2023-10-21T17:39:55.869408+0000","last_peered":"2023-10-21T17:39:55.869408+0000","last_clean":"2023-10-21T17:39:55.869408+0000","last_became_active":"2023-10-21T17:39:53.510623+0000","last_became_peered":"2023-10-21T17:39:53.510623+0000","last_unstale":"2023-10-21T17:39:55.869408+0000","last_undegraded":"2023-10-21T17:39:55.869408+0000","last_fullsized":"2023-10-21T17:39:55.869408+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:48.160505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483153+0000","last_change":"2023-10-21T17:39:40.267480+0000","last_active":"2023-10-21T17:39:54.483153+0000","last_peered":"2023-10-21T17:39:54.483153+0000","last_clean":"2023-10-21T17:39:54.483153+0000","last_became_active":"2023-10-21T17:39:40.267074+0000","last_became_peered":"2023-10-21T17:39:40.267074+0000","last_unstale":"2023-10-21T17:39:54.483153+0000","last_undegraded":"2023-10-21T17:39:54.483153+0000","last_fullsized":"2023-10-21T17:39:54.483153+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:12:05.724202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870560+0000","last_change":"2023-10-21T17:39:54.506708+0000","last_active":"2023-10-21T17:39:55.870560+0000","last_peered":"2023-10-21T17:39:55.870560+0000","last_clean":"2023-10-21T17:39:55.870560+0000","last_became_active":"2023-10-21T17:39:54.506421+0000","last_became_peered":"2023-10-21T17:39:54.506421+0000","last_unstale":"2023-10-21T17:39:55.870560+0000","last_undegraded":"2023-10-21T17:39:55.870560+0000","last_fullsized":"2023-10-21T17:39:55.870560+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:33.273447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865500+0000","last_change":"2023-10-21T17:39:53.493647+0000","last_active":"2023-10-21T17:39:55.865500+0000","last_peered":"2023-10-21T17:39:55.865500+0000","last_clean":"2023-10-21T17:39:55.865500+0000","last_became_active":"2023-10-21T17:39:53.493328+0000","last_became_peered":"2023-10-21T17:39:53.493328+0000","last_unstale":"2023-10-21T17:39:55.865500+0000","last_undegraded":"2023-10-21T17:39:55.865500+0000","last_fullsized":"2023-10-21T17:39:55.865500+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:19:28.838008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497725+0000","last_change":"2023-10-21T17:39:40.287294+0000","last_active":"2023-10-21T17:39:54.497725+0000","last_peered":"2023-10-21T17:39:54.497725+0000","last_clean":"2023-10-21T17:39:54.497725+0000","last_became_active":"2023-10-21T17:39:40.286990+0000","last_became_peered":"2023-10-21T17:39:40.286990+0000","last_unstale":"2023-10-21T17:39:54.497725+0000","last_undegraded":"2023-10-21T17:39:54.497725+0000","last_fullsized":"2023-10-21T17:39:54.497725+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:07:39.411032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866301+0000","last_change":"2023-10-21T17:39:54.508541+0000","last_active":"2023-10-21T17:39:55.866301+0000","last_peered":"2023-10-21T17:39:55.866301+0000","last_clean":"2023-10-21T17:39:55.866301+0000","last_became_active":"2023-10-21T17:39:54.508263+0000","last_became_peered":"2023-10-21T17:39:54.508263+0000","last_unstale":"2023-10-21T17:39:55.866301+0000","last_undegraded":"2023-10-21T17:39:55.866301+0000","last_fullsized":"2023-10-21T17:39:55.866301+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:44:40.233750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869943+0000","last_change":"2023-10-21T17:39:53.500685+0000","last_active":"2023-10-21T17:39:55.869943+0000","last_peered":"2023-10-21T17:39:55.869943+0000","last_clean":"2023-10-21T17:39:55.869943+0000","last_became_active":"2023-10-21T17:39:53.500261+0000","last_became_peered":"2023-10-21T17:39:53.500261+0000","last_unstale":"2023-10-21T17:39:55.869943+0000","last_undegraded":"2023-10-21T17:39:55.869943+0000","last_fullsized":"2023-10-21T17:39:55.869943+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:48.160505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482374+0000","last_change":"2023-10-21T17:39:40.270092+0000","last_active":"2023-10-21T17:39:54.482374+0000","last_peered":"2023-10-21T17:39:54.482374+0000","last_clean":"2023-10-21T17:39:54.482374+0000","last_became_active":"2023-10-21T17:39:40.269725+0000","last_became_peered":"2023-10-21T17:39:40.269725+0000","last_unstale":"2023-10-21T17:39:54.482374+0000","last_undegraded":"2023-10-21T17:39:54.482374+0000","last_fullsized":"2023-10-21T17:39:54.482374+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:18.601428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867545+0000","last_change":"2023-10-21T17:39:54.515960+0000","last_active":"2023-10-21T17:39:55.867545+0000","last_peered":"2023-10-21T17:39:55.867545+0000","last_clean":"2023-10-21T17:39:55.867545+0000","last_became_active":"2023-10-21T17:39:54.515687+0000","last_became_peered":"2023-10-21T17:39:54.515687+0000","last_unstale":"2023-10-21T17:39:55.867545+0000","last_undegraded":"2023-10-21T17:39:55.867545+0000","last_fullsized":"2023-10-21T17:39:55.867545+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:31.004177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504911+0000","last_change":"2023-10-21T17:39:53.505990+0000","last_active":"2023-10-21T17:39:55.504911+0000","last_peered":"2023-10-21T17:39:55.504911+0000","last_clean":"2023-10-21T17:39:55.504911+0000","last_became_active":"2023-10-21T17:39:53.505715+0000","last_became_peered":"2023-10-21T17:39:53.505715+0000","last_unstale":"2023-10-21T17:39:55.504911+0000","last_undegraded":"2023-10-21T17:39:55.504911+0000","last_fullsized":"2023-10-21T17:39:55.504911+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:27:38.955413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.486010+0000","last_change":"2023-10-21T17:39:40.275179+0000","last_active":"2023-10-21T17:39:54.486010+0000","last_peered":"2023-10-21T17:39:54.486010+0000","last_clean":"2023-10-21T17:39:54.486010+0000","last_became_active":"2023-10-21T17:39:40.274025+0000","last_became_peered":"2023-10-21T17:39:40.274025+0000","last_unstale":"2023-10-21T17:39:54.486010+0000","last_undegraded":"2023-10-21T17:39:54.486010+0000","last_fullsized":"2023-10-21T17:39:54.486010+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:58.558091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867987+0000","last_change":"2023-10-21T17:39:54.516243+0000","last_active":"2023-10-21T17:39:55.867987+0000","last_peered":"2023-10-21T17:39:55.867987+0000","last_clean":"2023-10-21T17:39:55.867987+0000","last_became_active":"2023-10-21T17:39:54.515934+0000","last_became_peered":"2023-10-21T17:39:54.515934+0000","last_unstale":"2023-10-21T17:39:55.867987+0000","last_undegraded":"2023-10-21T17:39:55.867987+0000","last_fullsized":"2023-10-21T17:39:55.867987+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:29:08.560941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"257'2","reported_seq":14,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868091+0000","last_change":"2023-10-21T17:39:53.508492+0000","last_active":"2023-10-21T17:39:55.868091+0000","last_peered":"2023-10-21T17:39:55.868091+0000","last_clean":"2023-10-21T17:39:55.868091+0000","last_became_active":"2023-10-21T17:39:53.508074+0000","last_became_peered":"2023-10-21T17:39:53.508074+0000","last_unstale":"2023-10-21T17:39:55.868091+0000","last_undegraded":"2023-10-21T17:39:55.868091+0000","last_fullsized":"2023-10-21T17:39:55.868091+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:32:22.348143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503230+0000","last_change":"2023-10-21T17:39:40.271219+0000","last_active":"2023-10-21T17:39:54.503230+0000","last_peered":"2023-10-21T17:39:54.503230+0000","last_clean":"2023-10-21T17:39:54.503230+0000","last_became_active":"2023-10-21T17:39:40.270928+0000","last_became_peered":"2023-10-21T17:39:40.270928+0000","last_unstale":"2023-10-21T17:39:54.503230+0000","last_undegraded":"2023-10-21T17:39:54.503230+0000","last_fullsized":"2023-10-21T17:39:54.503230+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:53.009674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868121+0000","last_change":"2023-10-21T17:39:54.515451+0000","last_active":"2023-10-21T17:39:55.868121+0000","last_peered":"2023-10-21T17:39:55.868121+0000","last_clean":"2023-10-21T17:39:55.868121+0000","last_became_active":"2023-10-21T17:39:54.514828+0000","last_became_peered":"2023-10-21T17:39:54.514828+0000","last_unstale":"2023-10-21T17:39:55.868121+0000","last_undegraded":"2023-10-21T17:39:55.868121+0000","last_fullsized":"2023-10-21T17:39:55.868121+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:30:55.218745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506188+0000","last_change":"2023-10-21T17:39:53.500862+0000","last_active":"2023-10-21T17:39:55.506188+0000","last_peered":"2023-10-21T17:39:55.506188+0000","last_clean":"2023-10-21T17:39:55.506188+0000","last_became_active":"2023-10-21T17:39:53.500544+0000","last_became_peered":"2023-10-21T17:39:53.500544+0000","last_unstale":"2023-10-21T17:39:55.506188+0000","last_undegraded":"2023-10-21T17:39:55.506188+0000","last_fullsized":"2023-10-21T17:39:55.506188+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:54.638360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.507490+0000","last_change":"2023-10-21T17:39:40.276685+0000","last_active":"2023-10-21T17:39:54.507490+0000","last_peered":"2023-10-21T17:39:54.507490+0000","last_clean":"2023-10-21T17:39:54.507490+0000","last_became_active":"2023-10-21T17:39:40.276416+0000","last_became_peered":"2023-10-21T17:39:40.276416+0000","last_unstale":"2023-10-21T17:39:54.507490+0000","last_undegraded":"2023-10-21T17:39:54.507490+0000","last_fullsized":"2023-10-21T17:39:54.507490+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:29.584375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.507401+0000","last_change":"2023-10-21T17:39:54.512213+0000","last_active":"2023-10-21T17:39:55.507401+0000","last_peered":"2023-10-21T17:39:55.507401+0000","last_clean":"2023-10-21T17:39:55.507401+0000","last_became_active":"2023-10-21T17:39:54.509442+0000","last_became_peered":"2023-10-21T17:39:54.509442+0000","last_unstale":"2023-10-21T17:39:55.507401+0000","last_undegraded":"2023-10-21T17:39:55.507401+0000","last_fullsized":"2023-10-21T17:39:55.507401+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:31:48.847242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506180+0000","last_change":"2023-10-21T17:39:53.489911+0000","last_active":"2023-10-21T17:39:55.506180+0000","last_peered":"2023-10-21T17:39:55.506180+0000","last_clean":"2023-10-21T17:39:55.506180+0000","last_became_active":"2023-10-21T17:39:53.489592+0000","last_became_peered":"2023-10-21T17:39:53.489592+0000","last_unstale":"2023-10-21T17:39:55.506180+0000","last_undegraded":"2023-10-21T17:39:55.506180+0000","last_fullsized":"2023-10-21T17:39:55.506180+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:05:32.255435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479090+0000","last_change":"2023-10-21T17:39:40.284175+0000","last_active":"2023-10-21T17:39:54.479090+0000","last_peered":"2023-10-21T17:39:54.479090+0000","last_clean":"2023-10-21T17:39:54.479090+0000","last_became_active":"2023-10-21T17:39:40.283773+0000","last_became_peered":"2023-10-21T17:39:40.283773+0000","last_unstale":"2023-10-21T17:39:54.479090+0000","last_undegraded":"2023-10-21T17:39:54.479090+0000","last_fullsized":"2023-10-21T17:39:54.479090+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:23:01.051661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869801+0000","last_change":"2023-10-21T17:39:54.512287+0000","last_active":"2023-10-21T17:39:55.869801+0000","last_peered":"2023-10-21T17:39:55.869801+0000","last_clean":"2023-10-21T17:39:55.869801+0000","last_became_active":"2023-10-21T17:39:54.506913+0000","last_became_peered":"2023-10-21T17:39:54.506913+0000","last_unstale":"2023-10-21T17:39:55.869801+0000","last_undegraded":"2023-10-21T17:39:55.869801+0000","last_fullsized":"2023-10-21T17:39:55.869801+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:26.912523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.503730+0000","last_change":"2023-10-21T17:39:53.497849+0000","last_active":"2023-10-21T17:39:55.503730+0000","last_peered":"2023-10-21T17:39:55.503730+0000","last_clean":"2023-10-21T17:39:55.503730+0000","last_became_active":"2023-10-21T17:39:53.497539+0000","last_became_peered":"2023-10-21T17:39:53.497539+0000","last_unstale":"2023-10-21T17:39:55.503730+0000","last_undegraded":"2023-10-21T17:39:55.503730+0000","last_fullsized":"2023-10-21T17:39:55.503730+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:17.368029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.506981+0000","last_change":"2023-10-21T17:39:40.258467+0000","last_active":"2023-10-21T17:39:54.506981+0000","last_peered":"2023-10-21T17:39:54.506981+0000","last_clean":"2023-10-21T17:39:54.506981+0000","last_became_active":"2023-10-21T17:39:40.257660+0000","last_became_peered":"2023-10-21T17:39:40.257660+0000","last_unstale":"2023-10-21T17:39:54.506981+0000","last_undegraded":"2023-10-21T17:39:54.506981+0000","last_fullsized":"2023-10-21T17:39:54.506981+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:03:12.244419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.865600+0000","last_change":"2023-10-21T17:39:54.501955+0000","last_active":"2023-10-21T17:39:55.865600+0000","last_peered":"2023-10-21T17:39:55.865600+0000","last_clean":"2023-10-21T17:39:55.865600+0000","last_became_active":"2023-10-21T17:39:54.501672+0000","last_became_peered":"2023-10-21T17:39:54.501672+0000","last_unstale":"2023-10-21T17:39:55.865600+0000","last_undegraded":"2023-10-21T17:39:55.865600+0000","last_fullsized":"2023-10-21T17:39:55.865600+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:15:45.216861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504103+0000","last_change":"2023-10-21T17:39:53.504406+0000","last_active":"2023-10-21T17:39:55.504103+0000","last_peered":"2023-10-21T17:39:55.504103+0000","last_clean":"2023-10-21T17:39:55.504103+0000","last_became_active":"2023-10-21T17:39:53.504097+0000","last_became_peered":"2023-10-21T17:39:53.504097+0000","last_unstale":"2023-10-21T17:39:55.504103+0000","last_undegraded":"2023-10-21T17:39:55.504103+0000","last_fullsized":"2023-10-21T17:39:55.504103+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:51.597031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509057+0000","last_change":"2023-10-21T17:39:40.276636+0000","last_active":"2023-10-21T17:39:54.509057+0000","last_peered":"2023-10-21T17:39:54.509057+0000","last_clean":"2023-10-21T17:39:54.509057+0000","last_became_active":"2023-10-21T17:39:40.274303+0000","last_became_peered":"2023-10-21T17:39:40.274303+0000","last_unstale":"2023-10-21T17:39:54.509057+0000","last_undegraded":"2023-10-21T17:39:54.509057+0000","last_fullsized":"2023-10-21T17:39:54.509057+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:45.096371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832643+0000","last_change":"2023-10-21T17:39:54.511696+0000","last_active":"2023-10-21T17:39:55.832643+0000","last_peered":"2023-10-21T17:39:55.832643+0000","last_clean":"2023-10-21T17:39:55.832643+0000","last_became_active":"2023-10-21T17:39:54.511178+0000","last_became_peered":"2023-10-21T17:39:54.511178+0000","last_unstale":"2023-10-21T17:39:55.832643+0000","last_undegraded":"2023-10-21T17:39:55.832643+0000","last_fullsized":"2023-10-21T17:39:55.832643+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:00.085614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.503566+0000","last_change":"2023-10-21T17:39:53.500780+0000","last_active":"2023-10-21T17:39:55.503566+0000","last_peered":"2023-10-21T17:39:55.503566+0000","last_clean":"2023-10-21T17:39:55.503566+0000","last_became_active":"2023-10-21T17:39:53.500473+0000","last_became_peered":"2023-10-21T17:39:53.500473+0000","last_unstale":"2023-10-21T17:39:55.503566+0000","last_undegraded":"2023-10-21T17:39:55.503566+0000","last_fullsized":"2023-10-21T17:39:55.503566+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:37.713072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478509+0000","last_change":"2023-10-21T17:39:40.270012+0000","last_active":"2023-10-21T17:39:54.478509+0000","last_peered":"2023-10-21T17:39:54.478509+0000","last_clean":"2023-10-21T17:39:54.478509+0000","last_became_active":"2023-10-21T17:39:40.269726+0000","last_became_peered":"2023-10-21T17:39:40.269726+0000","last_unstale":"2023-10-21T17:39:54.478509+0000","last_undegraded":"2023-10-21T17:39:54.478509+0000","last_fullsized":"2023-10-21T17:39:54.478509+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:58:57.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870487+0000","last_change":"2023-10-21T17:39:54.520704+0000","last_active":"2023-10-21T17:39:55.870487+0000","last_peered":"2023-10-21T17:39:55.870487+0000","last_clean":"2023-10-21T17:39:55.870487+0000","last_became_active":"2023-10-21T17:39:54.519741+0000","last_became_peered":"2023-10-21T17:39:54.519741+0000","last_unstale":"2023-10-21T17:39:55.870487+0000","last_undegraded":"2023-10-21T17:39:55.870487+0000","last_fullsized":"2023-10-21T17:39:55.870487+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:45.733225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.505019+0000","last_change":"2023-10-21T17:39:53.493859+0000","last_active":"2023-10-21T17:39:55.505019+0000","last_peered":"2023-10-21T17:39:55.505019+0000","last_clean":"2023-10-21T17:39:55.505019+0000","last_became_active":"2023-10-21T17:39:53.493248+0000","last_became_peered":"2023-10-21T17:39:53.493248+0000","last_unstale":"2023-10-21T17:39:55.505019+0000","last_undegraded":"2023-10-21T17:39:55.505019+0000","last_fullsized":"2023-10-21T17:39:55.505019+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:52.434750+0000","last_clean_scrub_stamp":"2023-10-21T17:39:52.434750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:44.715214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.477800+0000","last_change":"2023-10-21T17:39:40.284346+0000","last_active":"2023-10-21T17:39:54.477800+0000","last_peered":"2023-10-21T17:39:54.477800+0000","last_clean":"2023-10-21T17:39:54.477800+0000","last_became_active":"2023-10-21T17:39:40.282055+0000","last_became_peered":"2023-10-21T17:39:40.282055+0000","last_unstale":"2023-10-21T17:39:54.477800+0000","last_undegraded":"2023-10-21T17:39:54.477800+0000","last_fullsized":"2023-10-21T17:39:54.477800+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:33.876859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866645+0000","last_change":"2023-10-21T17:39:54.513870+0000","last_active":"2023-10-21T17:39:55.866645+0000","last_peered":"2023-10-21T17:39:55.866645+0000","last_clean":"2023-10-21T17:39:55.866645+0000","last_became_active":"2023-10-21T17:39:54.513631+0000","last_became_peered":"2023-10-21T17:39:54.513631+0000","last_unstale":"2023-10-21T17:39:55.866645+0000","last_undegraded":"2023-10-21T17:39:55.866645+0000","last_fullsized":"2023-10-21T17:39:55.866645+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:31:48.847242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.480877+0000","last_change":"2023-10-21T17:39:40.265515+0000","last_active":"2023-10-21T17:39:54.480877+0000","last_peered":"2023-10-21T17:39:54.480877+0000","last_clean":"2023-10-21T17:39:54.480877+0000","last_became_active":"2023-10-21T17:39:40.264706+0000","last_became_peered":"2023-10-21T17:39:40.264706+0000","last_unstale":"2023-10-21T17:39:54.480877+0000","last_undegraded":"2023-10-21T17:39:54.480877+0000","last_fullsized":"2023-10-21T17:39:54.480877+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:19.664508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483840+0000","last_change":"2023-10-21T17:39:39.225724+0000","last_active":"2023-10-21T17:39:54.483840+0000","last_peered":"2023-10-21T17:39:54.483840+0000","last_clean":"2023-10-21T17:39:54.483840+0000","last_became_active":"2023-10-21T17:39:39.223965+0000","last_became_peered":"2023-10-21T17:39:39.223965+0000","last_unstale":"2023-10-21T17:39:54.483840+0000","last_undegraded":"2023-10-21T17:39:54.483840+0000","last_fullsized":"2023-10-21T17:39:54.483840+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:53:53.712155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868661+0000","last_change":"2023-10-21T17:39:54.502119+0000","last_active":"2023-10-21T17:39:55.868661+0000","last_peered":"2023-10-21T17:39:55.868661+0000","last_clean":"2023-10-21T17:39:55.868661+0000","last_became_active":"2023-10-21T17:39:54.501820+0000","last_became_peered":"2023-10-21T17:39:54.501820+0000","last_unstale":"2023-10-21T17:39:55.868661+0000","last_undegraded":"2023-10-21T17:39:55.868661+0000","last_fullsized":"2023-10-21T17:39:55.868661+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:44:37.749741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502544+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:54.502544+0000","last_peered":"2023-10-21T17:39:54.502544+0000","last_clean":"2023-10-21T17:39:54.502544+0000","last_became_active":"2023-10-21T17:39:40.282270+0000","last_became_peered":"2023-10-21T17:39:40.282270+0000","last_unstale":"2023-10-21T17:39:54.502544+0000","last_undegraded":"2023-10-21T17:39:54.502544+0000","last_fullsized":"2023-10-21T17:39:54.502544+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:58:36.571340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484790+0000","last_change":"2023-10-21T17:39:39.240537+0000","last_active":"2023-10-21T17:39:54.484790+0000","last_peered":"2023-10-21T17:39:54.484790+0000","last_clean":"2023-10-21T17:39:54.484790+0000","last_became_active":"2023-10-21T17:39:39.240212+0000","last_became_peered":"2023-10-21T17:39:39.240212+0000","last_unstale":"2023-10-21T17:39:54.484790+0000","last_undegraded":"2023-10-21T17:39:54.484790+0000","last_fullsized":"2023-10-21T17:39:54.484790+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:44:57.552470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869344+0000","last_change":"2023-10-21T17:39:54.514201+0000","last_active":"2023-10-21T17:39:55.869344+0000","last_peered":"2023-10-21T17:39:55.869344+0000","last_clean":"2023-10-21T17:39:55.869344+0000","last_became_active":"2023-10-21T17:39:54.513317+0000","last_became_peered":"2023-10-21T17:39:54.513317+0000","last_unstale":"2023-10-21T17:39:55.869344+0000","last_undegraded":"2023-10-21T17:39:55.869344+0000","last_fullsized":"2023-10-21T17:39:55.869344+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:17:51.734321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482297+0000","last_change":"2023-10-21T17:39:40.278403+0000","last_active":"2023-10-21T17:39:54.482297+0000","last_peered":"2023-10-21T17:39:54.482297+0000","last_clean":"2023-10-21T17:39:54.482297+0000","last_became_active":"2023-10-21T17:39:40.278115+0000","last_became_peered":"2023-10-21T17:39:40.278115+0000","last_unstale":"2023-10-21T17:39:54.482297+0000","last_undegraded":"2023-10-21T17:39:54.482297+0000","last_fullsized":"2023-10-21T17:39:54.482297+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:58:16.014478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.511140+0000","last_change":"2023-10-21T17:39:39.239260+0000","last_active":"2023-10-21T17:39:54.511140+0000","last_peered":"2023-10-21T17:39:54.511140+0000","last_clean":"2023-10-21T17:39:54.511140+0000","last_became_active":"2023-10-21T17:39:39.235006+0000","last_became_peered":"2023-10-21T17:39:39.235006+0000","last_unstale":"2023-10-21T17:39:54.511140+0000","last_undegraded":"2023-10-21T17:39:54.511140+0000","last_fullsized":"2023-10-21T17:39:54.511140+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870926+0000","last_change":"2023-10-21T17:39:54.504803+0000","last_active":"2023-10-21T17:39:55.870926+0000","last_peered":"2023-10-21T17:39:55.870926+0000","last_clean":"2023-10-21T17:39:55.870926+0000","last_became_active":"2023-10-21T17:39:54.504547+0000","last_became_peered":"2023-10-21T17:39:54.504547+0000","last_unstale":"2023-10-21T17:39:55.870926+0000","last_undegraded":"2023-10-21T17:39:55.870926+0000","last_fullsized":"2023-10-21T17:39:55.870926+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:02:55.656371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.510878+0000","last_change":"2023-10-21T17:39:40.268385+0000","last_active":"2023-10-21T17:39:54.510878+0000","last_peered":"2023-10-21T17:39:54.510878+0000","last_clean":"2023-10-21T17:39:54.510878+0000","last_became_active":"2023-10-21T17:39:40.267876+0000","last_became_peered":"2023-10-21T17:39:40.267876+0000","last_unstale":"2023-10-21T17:39:54.510878+0000","last_undegraded":"2023-10-21T17:39:54.510878+0000","last_fullsized":"2023-10-21T17:39:54.510878+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:12:32.664957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502522+0000","last_change":"2023-10-21T17:39:39.246981+0000","last_active":"2023-10-21T17:39:54.502522+0000","last_peered":"2023-10-21T17:39:54.502522+0000","last_clean":"2023-10-21T17:39:54.502522+0000","last_became_active":"2023-10-21T17:39:39.246676+0000","last_became_peered":"2023-10-21T17:39:39.246676+0000","last_unstale":"2023-10-21T17:39:54.502522+0000","last_undegraded":"2023-10-21T17:39:54.502522+0000","last_fullsized":"2023-10-21T17:39:54.502522+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:51:08.549227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508776+0000","last_change":"2023-10-21T17:39:54.516931+0000","last_active":"2023-10-21T17:39:55.508776+0000","last_peered":"2023-10-21T17:39:55.508776+0000","last_clean":"2023-10-21T17:39:55.508776+0000","last_became_active":"2023-10-21T17:39:54.516656+0000","last_became_peered":"2023-10-21T17:39:54.516656+0000","last_unstale":"2023-10-21T17:39:55.508776+0000","last_undegraded":"2023-10-21T17:39:55.508776+0000","last_fullsized":"2023-10-21T17:39:55.508776+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:49.781653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504996+0000","last_change":"2023-10-21T17:39:40.273974+0000","last_active":"2023-10-21T17:39:54.504996+0000","last_peered":"2023-10-21T17:39:54.504996+0000","last_clean":"2023-10-21T17:39:54.504996+0000","last_became_active":"2023-10-21T17:39:40.273689+0000","last_became_peered":"2023-10-21T17:39:40.273689+0000","last_unstale":"2023-10-21T17:39:54.504996+0000","last_undegraded":"2023-10-21T17:39:54.504996+0000","last_fullsized":"2023-10-21T17:39:54.504996+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509152+0000","last_change":"2023-10-21T17:39:39.236436+0000","last_active":"2023-10-21T17:39:54.509152+0000","last_peered":"2023-10-21T17:39:54.509152+0000","last_clean":"2023-10-21T17:39:54.509152+0000","last_became_active":"2023-10-21T17:39:39.236185+0000","last_became_peered":"2023-10-21T17:39:39.236185+0000","last_unstale":"2023-10-21T17:39:54.509152+0000","last_undegraded":"2023-10-21T17:39:54.509152+0000","last_fullsized":"2023-10-21T17:39:54.509152+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:21:51.145740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508996+0000","last_change":"2023-10-21T17:39:54.518570+0000","last_active":"2023-10-21T17:39:55.508996+0000","last_peered":"2023-10-21T17:39:55.508996+0000","last_clean":"2023-10-21T17:39:55.508996+0000","last_became_active":"2023-10-21T17:39:54.517636+0000","last_became_peered":"2023-10-21T17:39:54.517636+0000","last_unstale":"2023-10-21T17:39:55.508996+0000","last_undegraded":"2023-10-21T17:39:55.508996+0000","last_fullsized":"2023-10-21T17:39:55.508996+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:36:32.034612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497521+0000","last_change":"2023-10-21T17:39:40.275825+0000","last_active":"2023-10-21T17:39:54.497521+0000","last_peered":"2023-10-21T17:39:54.497521+0000","last_clean":"2023-10-21T17:39:54.497521+0000","last_became_active":"2023-10-21T17:39:40.275521+0000","last_became_peered":"2023-10-21T17:39:40.275521+0000","last_unstale":"2023-10-21T17:39:54.497521+0000","last_undegraded":"2023-10-21T17:39:54.497521+0000","last_fullsized":"2023-10-21T17:39:54.497521+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:44:32.834263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496168+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:54.496168+0000","last_peered":"2023-10-21T17:39:54.496168+0000","last_clean":"2023-10-21T17:39:54.496168+0000","last_became_active":"2023-10-21T17:39:39.241978+0000","last_became_peered":"2023-10-21T17:39:39.241978+0000","last_unstale":"2023-10-21T17:39:54.496168+0000","last_undegraded":"2023-10-21T17:39:54.496168+0000","last_fullsized":"2023-10-21T17:39:54.496168+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:44:13.792604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509482+0000","last_change":"2023-10-21T17:39:54.518139+0000","last_active":"2023-10-21T17:39:55.509482+0000","last_peered":"2023-10-21T17:39:55.509482+0000","last_clean":"2023-10-21T17:39:55.509482+0000","last_became_active":"2023-10-21T17:39:54.517862+0000","last_became_peered":"2023-10-21T17:39:54.517862+0000","last_unstale":"2023-10-21T17:39:55.509482+0000","last_undegraded":"2023-10-21T17:39:55.509482+0000","last_fullsized":"2023-10-21T17:39:55.509482+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:15.523450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509972+0000","last_change":"2023-10-21T17:39:40.278021+0000","last_active":"2023-10-21T17:39:54.509972+0000","last_peered":"2023-10-21T17:39:54.509972+0000","last_clean":"2023-10-21T17:39:54.509972+0000","last_became_active":"2023-10-21T17:39:40.276836+0000","last_became_peered":"2023-10-21T17:39:40.276836+0000","last_unstale":"2023-10-21T17:39:54.509972+0000","last_undegraded":"2023-10-21T17:39:54.509972+0000","last_fullsized":"2023-10-21T17:39:54.509972+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:34:37.173241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479260+0000","last_change":"2023-10-21T17:39:39.241068+0000","last_active":"2023-10-21T17:39:54.479260+0000","last_peered":"2023-10-21T17:39:54.479260+0000","last_clean":"2023-10-21T17:39:54.479260+0000","last_became_active":"2023-10-21T17:39:39.240781+0000","last_became_peered":"2023-10-21T17:39:39.240781+0000","last_unstale":"2023-10-21T17:39:54.479260+0000","last_undegraded":"2023-10-21T17:39:54.479260+0000","last_fullsized":"2023-10-21T17:39:54.479260+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:10:06.555504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.508529+0000","last_change":"2023-10-21T17:39:54.515178+0000","last_active":"2023-10-21T17:39:55.508529+0000","last_peered":"2023-10-21T17:39:55.508529+0000","last_clean":"2023-10-21T17:39:55.508529+0000","last_became_active":"2023-10-21T17:39:54.514908+0000","last_became_peered":"2023-10-21T17:39:54.514908+0000","last_unstale":"2023-10-21T17:39:55.508529+0000","last_undegraded":"2023-10-21T17:39:55.508529+0000","last_fullsized":"2023-10-21T17:39:55.508529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:41.630527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499702+0000","last_change":"2023-10-21T17:39:40.278404+0000","last_active":"2023-10-21T17:39:54.499702+0000","last_peered":"2023-10-21T17:39:54.499702+0000","last_clean":"2023-10-21T17:39:54.499702+0000","last_became_active":"2023-10-21T17:39:40.276735+0000","last_became_peered":"2023-10-21T17:39:40.276735+0000","last_unstale":"2023-10-21T17:39:54.499702+0000","last_undegraded":"2023-10-21T17:39:54.499702+0000","last_fullsized":"2023-10-21T17:39:54.499702+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:05:58.107147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.523045+0000","last_change":"2023-10-21T17:39:39.229264+0000","last_active":"2023-10-21T17:39:54.523045+0000","last_peered":"2023-10-21T17:39:54.523045+0000","last_clean":"2023-10-21T17:39:54.523045+0000","last_became_active":"2023-10-21T17:39:39.227950+0000","last_became_peered":"2023-10-21T17:39:39.227950+0000","last_unstale":"2023-10-21T17:39:54.523045+0000","last_undegraded":"2023-10-21T17:39:54.523045+0000","last_fullsized":"2023-10-21T17:39:54.523045+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:26:36.550703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867036+0000","last_change":"2023-10-21T17:39:54.505800+0000","last_active":"2023-10-21T17:39:55.867036+0000","last_peered":"2023-10-21T17:39:55.867036+0000","last_clean":"2023-10-21T17:39:55.867036+0000","last_became_active":"2023-10-21T17:39:54.505347+0000","last_became_peered":"2023-10-21T17:39:54.505347+0000","last_unstale":"2023-10-21T17:39:55.867036+0000","last_undegraded":"2023-10-21T17:39:55.867036+0000","last_fullsized":"2023-10-21T17:39:55.867036+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:17:51.734321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483055+0000","last_change":"2023-10-21T17:39:40.262960+0000","last_active":"2023-10-21T17:39:54.483055+0000","last_peered":"2023-10-21T17:39:54.483055+0000","last_clean":"2023-10-21T17:39:54.483055+0000","last_became_active":"2023-10-21T17:39:40.262214+0000","last_became_peered":"2023-10-21T17:39:40.262214+0000","last_unstale":"2023-10-21T17:39:54.483055+0000","last_undegraded":"2023-10-21T17:39:54.483055+0000","last_fullsized":"2023-10-21T17:39:54.483055+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:42:08.816431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497756+0000","last_change":"2023-10-21T17:39:39.244340+0000","last_active":"2023-10-21T17:39:54.497756+0000","last_peered":"2023-10-21T17:39:54.497756+0000","last_clean":"2023-10-21T17:39:54.497756+0000","last_became_active":"2023-10-21T17:39:39.243245+0000","last_became_peered":"2023-10-21T17:39:39.243245+0000","last_unstale":"2023-10-21T17:39:54.497756+0000","last_undegraded":"2023-10-21T17:39:54.497756+0000","last_fullsized":"2023-10-21T17:39:54.497756+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:08:54.585657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506646+0000","last_change":"2023-10-21T17:39:54.511146+0000","last_active":"2023-10-21T17:39:55.506646+0000","last_peered":"2023-10-21T17:39:55.506646+0000","last_clean":"2023-10-21T17:39:55.506646+0000","last_became_active":"2023-10-21T17:39:54.510047+0000","last_became_peered":"2023-10-21T17:39:54.510047+0000","last_unstale":"2023-10-21T17:39:55.506646+0000","last_undegraded":"2023-10-21T17:39:55.506646+0000","last_fullsized":"2023-10-21T17:39:55.506646+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:50.714206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499628+0000","last_change":"2023-10-21T17:39:40.279028+0000","last_active":"2023-10-21T17:39:54.499628+0000","last_peered":"2023-10-21T17:39:54.499628+0000","last_clean":"2023-10-21T17:39:54.499628+0000","last_became_active":"2023-10-21T17:39:40.278727+0000","last_became_peered":"2023-10-21T17:39:40.278727+0000","last_unstale":"2023-10-21T17:39:54.499628+0000","last_undegraded":"2023-10-21T17:39:54.499628+0000","last_fullsized":"2023-10-21T17:39:54.499628+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:49:39.467095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":19,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.507640+0000","last_change":"2023-10-21T17:39:39.238814+0000","last_active":"2023-10-21T17:39:54.507640+0000","last_peered":"2023-10-21T17:39:54.507640+0000","last_clean":"2023-10-21T17:39:54.507640+0000","last_became_active":"2023-10-21T17:39:39.238497+0000","last_became_peered":"2023-10-21T17:39:39.238497+0000","last_unstale":"2023-10-21T17:39:54.507640+0000","last_undegraded":"2023-10-21T17:39:54.507640+0000","last_fullsized":"2023-10-21T17:39:54.507640+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:59:37.376711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869001+0000","last_change":"2023-10-21T17:39:54.514413+0000","last_active":"2023-10-21T17:39:55.869001+0000","last_peered":"2023-10-21T17:39:55.869001+0000","last_clean":"2023-10-21T17:39:55.869001+0000","last_became_active":"2023-10-21T17:39:54.513642+0000","last_became_peered":"2023-10-21T17:39:54.513642+0000","last_unstale":"2023-10-21T17:39:55.869001+0000","last_undegraded":"2023-10-21T17:39:55.869001+0000","last_fullsized":"2023-10-21T17:39:55.869001+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:50.797174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504085+0000","last_change":"2023-10-21T17:39:40.280192+0000","last_active":"2023-10-21T17:39:54.504085+0000","last_peered":"2023-10-21T17:39:54.504085+0000","last_clean":"2023-10-21T17:39:54.504085+0000","last_became_active":"2023-10-21T17:39:40.279897+0000","last_became_peered":"2023-10-21T17:39:40.279897+0000","last_unstale":"2023-10-21T17:39:54.504085+0000","last_undegraded":"2023-10-21T17:39:54.504085+0000","last_fullsized":"2023-10-21T17:39:54.504085+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:16.757719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"255'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.479512+0000","last_change":"2023-10-21T17:39:39.237379+0000","last_active":"2023-10-21T17:39:54.479512+0000","last_peered":"2023-10-21T17:39:54.479512+0000","last_clean":"2023-10-21T17:39:54.479512+0000","last_became_active":"2023-10-21T17:39:39.236089+0000","last_became_peered":"2023-10-21T17:39:39.236089+0000","last_unstale":"2023-10-21T17:39:54.479512+0000","last_undegraded":"2023-10-21T17:39:54.479512+0000","last_fullsized":"2023-10-21T17:39:54.479512+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:53:13.628441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504970+0000","last_change":"2023-10-21T17:39:54.505105+0000","last_active":"2023-10-21T17:39:55.504970+0000","last_peered":"2023-10-21T17:39:55.504970+0000","last_clean":"2023-10-21T17:39:55.504970+0000","last_became_active":"2023-10-21T17:39:54.502129+0000","last_became_peered":"2023-10-21T17:39:54.502129+0000","last_unstale":"2023-10-21T17:39:55.504970+0000","last_undegraded":"2023-10-21T17:39:55.504970+0000","last_fullsized":"2023-10-21T17:39:55.504970+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:51.037591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484342+0000","last_change":"2023-10-21T17:39:40.283845+0000","last_active":"2023-10-21T17:39:54.484342+0000","last_peered":"2023-10-21T17:39:54.484342+0000","last_clean":"2023-10-21T17:39:54.484342+0000","last_became_active":"2023-10-21T17:39:40.283046+0000","last_became_peered":"2023-10-21T17:39:40.283046+0000","last_unstale":"2023-10-21T17:39:54.484342+0000","last_undegraded":"2023-10-21T17:39:54.484342+0000","last_fullsized":"2023-10-21T17:39:54.484342+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:49:18.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502617+0000","last_change":"2023-10-21T17:39:39.244723+0000","last_active":"2023-10-21T17:39:54.502617+0000","last_peered":"2023-10-21T17:39:54.502617+0000","last_clean":"2023-10-21T17:39:54.502617+0000","last_became_active":"2023-10-21T17:39:39.244145+0000","last_became_peered":"2023-10-21T17:39:39.244145+0000","last_unstale":"2023-10-21T17:39:54.502617+0000","last_undegraded":"2023-10-21T17:39:54.502617+0000","last_fullsized":"2023-10-21T17:39:54.502617+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:03:15.752098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.833095+0000","last_change":"2023-10-21T17:39:54.511121+0000","last_active":"2023-10-21T17:39:55.833095+0000","last_peered":"2023-10-21T17:39:55.833095+0000","last_clean":"2023-10-21T17:39:55.833095+0000","last_became_active":"2023-10-21T17:39:54.510828+0000","last_became_peered":"2023-10-21T17:39:54.510828+0000","last_unstale":"2023-10-21T17:39:55.833095+0000","last_undegraded":"2023-10-21T17:39:55.833095+0000","last_fullsized":"2023-10-21T17:39:55.833095+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:36:57.717949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504751+0000","last_change":"2023-10-21T17:39:40.272813+0000","last_active":"2023-10-21T17:39:54.504751+0000","last_peered":"2023-10-21T17:39:54.504751+0000","last_clean":"2023-10-21T17:39:54.504751+0000","last_became_active":"2023-10-21T17:39:40.272544+0000","last_became_peered":"2023-10-21T17:39:40.272544+0000","last_unstale":"2023-10-21T17:39:54.504751+0000","last_undegraded":"2023-10-21T17:39:54.504751+0000","last_fullsized":"2023-10-21T17:39:54.504751+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:23.620934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501962+0000","last_change":"2023-10-21T17:39:39.229845+0000","last_active":"2023-10-21T17:39:54.501962+0000","last_peered":"2023-10-21T17:39:54.501962+0000","last_clean":"2023-10-21T17:39:54.501962+0000","last_became_active":"2023-10-21T17:39:39.229136+0000","last_became_peered":"2023-10-21T17:39:39.229136+0000","last_unstale":"2023-10-21T17:39:54.501962+0000","last_undegraded":"2023-10-21T17:39:54.501962+0000","last_fullsized":"2023-10-21T17:39:54.501962+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:33.975640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867568+0000","last_change":"2023-10-21T17:39:54.511484+0000","last_active":"2023-10-21T17:39:55.867568+0000","last_peered":"2023-10-21T17:39:55.867568+0000","last_clean":"2023-10-21T17:39:55.867568+0000","last_became_active":"2023-10-21T17:39:54.511070+0000","last_became_peered":"2023-10-21T17:39:54.511070+0000","last_unstale":"2023-10-21T17:39:55.867568+0000","last_undegraded":"2023-10-21T17:39:55.867568+0000","last_fullsized":"2023-10-21T17:39:55.867568+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:52:08.687825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.504877+0000","last_change":"2023-10-21T17:39:40.276873+0000","last_active":"2023-10-21T17:39:54.504877+0000","last_peered":"2023-10-21T17:39:54.504877+0000","last_clean":"2023-10-21T17:39:54.504877+0000","last_became_active":"2023-10-21T17:39:40.276503+0000","last_became_peered":"2023-10-21T17:39:40.276503+0000","last_unstale":"2023-10-21T17:39:54.504877+0000","last_undegraded":"2023-10-21T17:39:54.504877+0000","last_fullsized":"2023-10-21T17:39:54.504877+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:45:34.926709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501910+0000","last_change":"2023-10-21T17:39:39.227400+0000","last_active":"2023-10-21T17:39:54.501910+0000","last_peered":"2023-10-21T17:39:54.501910+0000","last_clean":"2023-10-21T17:39:54.501910+0000","last_became_active":"2023-10-21T17:39:39.226822+0000","last_became_peered":"2023-10-21T17:39:39.226822+0000","last_unstale":"2023-10-21T17:39:54.501910+0000","last_undegraded":"2023-10-21T17:39:54.501910+0000","last_fullsized":"2023-10-21T17:39:54.501910+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:18:27.203739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832416+0000","last_change":"2023-10-21T17:39:54.510329+0000","last_active":"2023-10-21T17:39:55.832416+0000","last_peered":"2023-10-21T17:39:55.832416+0000","last_clean":"2023-10-21T17:39:55.832416+0000","last_became_active":"2023-10-21T17:39:54.509095+0000","last_became_peered":"2023-10-21T17:39:54.509095+0000","last_unstale":"2023-10-21T17:39:55.832416+0000","last_undegraded":"2023-10-21T17:39:55.832416+0000","last_fullsized":"2023-10-21T17:39:55.832416+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:24:41.483301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.509687+0000","last_change":"2023-10-21T17:39:40.281657+0000","last_active":"2023-10-21T17:39:54.509687+0000","last_peered":"2023-10-21T17:39:54.509687+0000","last_clean":"2023-10-21T17:39:54.509687+0000","last_became_active":"2023-10-21T17:39:40.281069+0000","last_became_peered":"2023-10-21T17:39:40.281069+0000","last_unstale":"2023-10-21T17:39:54.509687+0000","last_undegraded":"2023-10-21T17:39:54.509687+0000","last_fullsized":"2023-10-21T17:39:54.509687+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:01.271644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485060+0000","last_change":"2023-10-21T17:39:39.235563+0000","last_active":"2023-10-21T17:39:54.485060+0000","last_peered":"2023-10-21T17:39:54.485060+0000","last_clean":"2023-10-21T17:39:54.485060+0000","last_became_active":"2023-10-21T17:39:39.235286+0000","last_became_peered":"2023-10-21T17:39:39.235286+0000","last_unstale":"2023-10-21T17:39:54.485060+0000","last_undegraded":"2023-10-21T17:39:54.485060+0000","last_fullsized":"2023-10-21T17:39:54.485060+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:56.950389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866603+0000","last_change":"2023-10-21T17:39:54.518702+0000","last_active":"2023-10-21T17:39:55.866603+0000","last_peered":"2023-10-21T17:39:55.866603+0000","last_clean":"2023-10-21T17:39:55.866603+0000","last_became_active":"2023-10-21T17:39:54.518397+0000","last_became_peered":"2023-10-21T17:39:54.518397+0000","last_unstale":"2023-10-21T17:39:55.866603+0000","last_undegraded":"2023-10-21T17:39:55.866603+0000","last_fullsized":"2023-10-21T17:39:55.866603+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:47:56.028968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501495+0000","last_change":"2023-10-21T17:39:40.274495+0000","last_active":"2023-10-21T17:39:54.501495+0000","last_peered":"2023-10-21T17:39:54.501495+0000","last_clean":"2023-10-21T17:39:54.501495+0000","last_became_active":"2023-10-21T17:39:40.273782+0000","last_became_peered":"2023-10-21T17:39:40.273782+0000","last_unstale":"2023-10-21T17:39:54.501495+0000","last_undegraded":"2023-10-21T17:39:54.501495+0000","last_fullsized":"2023-10-21T17:39:54.501495+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:07:27.378721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484345+0000","last_change":"2023-10-21T17:39:39.236238+0000","last_active":"2023-10-21T17:39:54.484345+0000","last_peered":"2023-10-21T17:39:54.484345+0000","last_clean":"2023-10-21T17:39:54.484345+0000","last_became_active":"2023-10-21T17:39:39.234980+0000","last_became_peered":"2023-10-21T17:39:39.234980+0000","last_unstale":"2023-10-21T17:39:54.484345+0000","last_undegraded":"2023-10-21T17:39:54.484345+0000","last_fullsized":"2023-10-21T17:39:54.484345+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:20:21.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867250+0000","last_change":"2023-10-21T17:39:54.510233+0000","last_active":"2023-10-21T17:39:55.867250+0000","last_peered":"2023-10-21T17:39:55.867250+0000","last_clean":"2023-10-21T17:39:55.867250+0000","last_became_active":"2023-10-21T17:39:54.508196+0000","last_became_peered":"2023-10-21T17:39:54.508196+0000","last_unstale":"2023-10-21T17:39:55.867250+0000","last_undegraded":"2023-10-21T17:39:55.867250+0000","last_fullsized":"2023-10-21T17:39:55.867250+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:40:47.495803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501799+0000","last_change":"2023-10-21T17:39:40.277381+0000","last_active":"2023-10-21T17:39:54.501799+0000","last_peered":"2023-10-21T17:39:54.501799+0000","last_clean":"2023-10-21T17:39:54.501799+0000","last_became_active":"2023-10-21T17:39:40.277018+0000","last_became_peered":"2023-10-21T17:39:40.277018+0000","last_unstale":"2023-10-21T17:39:54.501799+0000","last_undegraded":"2023-10-21T17:39:54.501799+0000","last_fullsized":"2023-10-21T17:39:54.501799+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:25:00.433924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489611+0000","last_change":"2023-10-21T17:39:39.248091+0000","last_active":"2023-10-21T17:39:54.489611+0000","last_peered":"2023-10-21T17:39:54.489611+0000","last_clean":"2023-10-21T17:39:54.489611+0000","last_became_active":"2023-10-21T17:39:39.247775+0000","last_became_peered":"2023-10-21T17:39:39.247775+0000","last_unstale":"2023-10-21T17:39:54.489611+0000","last_undegraded":"2023-10-21T17:39:54.489611+0000","last_fullsized":"2023-10-21T17:39:54.489611+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:43.183503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.831529+0000","last_change":"2023-10-21T17:39:54.518461+0000","last_active":"2023-10-21T17:39:55.831529+0000","last_peered":"2023-10-21T17:39:55.831529+0000","last_clean":"2023-10-21T17:39:55.831529+0000","last_became_active":"2023-10-21T17:39:54.517478+0000","last_became_peered":"2023-10-21T17:39:54.517478+0000","last_unstale":"2023-10-21T17:39:55.831529+0000","last_undegraded":"2023-10-21T17:39:55.831529+0000","last_fullsized":"2023-10-21T17:39:55.831529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:17:11.907848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.488175+0000","last_change":"2023-10-21T17:39:40.268499+0000","last_active":"2023-10-21T17:39:54.488175+0000","last_peered":"2023-10-21T17:39:54.488175+0000","last_clean":"2023-10-21T17:39:54.488175+0000","last_became_active":"2023-10-21T17:39:40.268106+0000","last_became_peered":"2023-10-21T17:39:40.268106+0000","last_unstale":"2023-10-21T17:39:54.488175+0000","last_undegraded":"2023-10-21T17:39:54.488175+0000","last_fullsized":"2023-10-21T17:39:54.488175+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:09.587502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485037+0000","last_change":"2023-10-21T17:39:39.237722+0000","last_active":"2023-10-21T17:39:54.485037+0000","last_peered":"2023-10-21T17:39:54.485037+0000","last_clean":"2023-10-21T17:39:54.485037+0000","last_became_active":"2023-10-21T17:39:39.235932+0000","last_became_peered":"2023-10-21T17:39:39.235932+0000","last_unstale":"2023-10-21T17:39:54.485037+0000","last_undegraded":"2023-10-21T17:39:54.485037+0000","last_fullsized":"2023-10-21T17:39:54.485037+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:55:08.581881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868619+0000","last_change":"2023-10-21T17:39:54.517167+0000","last_active":"2023-10-21T17:39:55.868619+0000","last_peered":"2023-10-21T17:39:55.868619+0000","last_clean":"2023-10-21T17:39:55.868619+0000","last_became_active":"2023-10-21T17:39:54.513938+0000","last_became_peered":"2023-10-21T17:39:54.513938+0000","last_unstale":"2023-10-21T17:39:55.868619+0000","last_undegraded":"2023-10-21T17:39:55.868619+0000","last_fullsized":"2023-10-21T17:39:55.868619+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:48:03.873117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500167+0000","last_change":"2023-10-21T17:39:40.282997+0000","last_active":"2023-10-21T17:39:54.500167+0000","last_peered":"2023-10-21T17:39:54.500167+0000","last_clean":"2023-10-21T17:39:54.500167+0000","last_became_active":"2023-10-21T17:39:40.282699+0000","last_became_peered":"2023-10-21T17:39:40.282699+0000","last_unstale":"2023-10-21T17:39:54.500167+0000","last_undegraded":"2023-10-21T17:39:54.500167+0000","last_fullsized":"2023-10-21T17:39:54.500167+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:28:01.093561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503872+0000","last_change":"2023-10-21T17:39:39.247859+0000","last_active":"2023-10-21T17:39:54.503872+0000","last_peered":"2023-10-21T17:39:54.503872+0000","last_clean":"2023-10-21T17:39:54.503872+0000","last_became_active":"2023-10-21T17:39:39.245045+0000","last_became_peered":"2023-10-21T17:39:39.245045+0000","last_unstale":"2023-10-21T17:39:54.503872+0000","last_undegraded":"2023-10-21T17:39:54.503872+0000","last_fullsized":"2023-10-21T17:39:54.503872+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:16:22.615313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.832963+0000","last_change":"2023-10-21T17:39:54.505811+0000","last_active":"2023-10-21T17:39:55.832963+0000","last_peered":"2023-10-21T17:39:55.832963+0000","last_clean":"2023-10-21T17:39:55.832963+0000","last_became_active":"2023-10-21T17:39:54.505555+0000","last_became_peered":"2023-10-21T17:39:54.505555+0000","last_unstale":"2023-10-21T17:39:55.832963+0000","last_undegraded":"2023-10-21T17:39:55.832963+0000","last_fullsized":"2023-10-21T17:39:55.832963+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:21:20.065152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.498935+0000","last_change":"2023-10-21T17:39:40.283322+0000","last_active":"2023-10-21T17:39:54.498935+0000","last_peered":"2023-10-21T17:39:54.498935+0000","last_clean":"2023-10-21T17:39:54.498935+0000","last_became_active":"2023-10-21T17:39:40.282539+0000","last_became_peered":"2023-10-21T17:39:40.282539+0000","last_unstale":"2023-10-21T17:39:54.498935+0000","last_undegraded":"2023-10-21T17:39:54.498935+0000","last_fullsized":"2023-10-21T17:39:54.498935+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:12:49.124101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.500254+0000","last_change":"2023-10-21T17:39:39.242639+0000","last_active":"2023-10-21T17:39:54.500254+0000","last_peered":"2023-10-21T17:39:54.500254+0000","last_clean":"2023-10-21T17:39:54.500254+0000","last_became_active":"2023-10-21T17:39:39.242334+0000","last_became_peered":"2023-10-21T17:39:39.242334+0000","last_unstale":"2023-10-21T17:39:54.500254+0000","last_undegraded":"2023-10-21T17:39:54.500254+0000","last_fullsized":"2023-10-21T17:39:54.500254+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:01:44.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.868017+0000","last_change":"2023-10-21T17:39:54.512580+0000","last_active":"2023-10-21T17:39:55.868017+0000","last_peered":"2023-10-21T17:39:55.868017+0000","last_clean":"2023-10-21T17:39:55.868017+0000","last_became_active":"2023-10-21T17:39:54.511005+0000","last_became_peered":"2023-10-21T17:39:54.511005+0000","last_unstale":"2023-10-21T17:39:55.868017+0000","last_undegraded":"2023-10-21T17:39:55.868017+0000","last_fullsized":"2023-10-21T17:39:55.868017+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:27:10.349105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.487231+0000","last_change":"2023-10-21T17:39:40.270958+0000","last_active":"2023-10-21T17:39:54.487231+0000","last_peered":"2023-10-21T17:39:54.487231+0000","last_clean":"2023-10-21T17:39:54.487231+0000","last_became_active":"2023-10-21T17:39:40.270640+0000","last_became_peered":"2023-10-21T17:39:40.270640+0000","last_unstale":"2023-10-21T17:39:54.487231+0000","last_undegraded":"2023-10-21T17:39:54.487231+0000","last_fullsized":"2023-10-21T17:39:54.487231+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:21.522147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484668+0000","last_change":"2023-10-21T17:39:39.246257+0000","last_active":"2023-10-21T17:39:54.484668+0000","last_peered":"2023-10-21T17:39:54.484668+0000","last_clean":"2023-10-21T17:39:54.484668+0000","last_became_active":"2023-10-21T17:39:39.245252+0000","last_became_peered":"2023-10-21T17:39:39.245252+0000","last_unstale":"2023-10-21T17:39:54.484668+0000","last_undegraded":"2023-10-21T17:39:54.484668+0000","last_fullsized":"2023-10-21T17:39:54.484668+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:18.223298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867540+0000","last_change":"2023-10-21T17:39:54.515461+0000","last_active":"2023-10-21T17:39:55.867540+0000","last_peered":"2023-10-21T17:39:55.867540+0000","last_clean":"2023-10-21T17:39:55.867540+0000","last_became_active":"2023-10-21T17:39:54.514176+0000","last_became_peered":"2023-10-21T17:39:54.514176+0000","last_unstale":"2023-10-21T17:39:55.867540+0000","last_undegraded":"2023-10-21T17:39:55.867540+0000","last_fullsized":"2023-10-21T17:39:55.867540+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:42:40.444872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.484703+0000","last_change":"2023-10-21T17:39:40.282652+0000","last_active":"2023-10-21T17:39:54.484703+0000","last_peered":"2023-10-21T17:39:54.484703+0000","last_clean":"2023-10-21T17:39:54.484703+0000","last_became_active":"2023-10-21T17:39:40.282108+0000","last_became_peered":"2023-10-21T17:39:40.282108+0000","last_unstale":"2023-10-21T17:39:54.484703+0000","last_undegraded":"2023-10-21T17:39:54.484703+0000","last_fullsized":"2023-10-21T17:39:54.484703+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:14:38.363235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483498+0000","last_change":"2023-10-21T17:39:39.242559+0000","last_active":"2023-10-21T17:39:54.483498+0000","last_peered":"2023-10-21T17:39:54.483498+0000","last_clean":"2023-10-21T17:39:54.483498+0000","last_became_active":"2023-10-21T17:39:39.236694+0000","last_became_peered":"2023-10-21T17:39:39.236694+0000","last_unstale":"2023-10-21T17:39:54.483498+0000","last_undegraded":"2023-10-21T17:39:54.483498+0000","last_fullsized":"2023-10-21T17:39:54.483498+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:03:58.202607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.509429+0000","last_change":"2023-10-21T17:39:54.510437+0000","last_active":"2023-10-21T17:39:55.509429+0000","last_peered":"2023-10-21T17:39:55.509429+0000","last_clean":"2023-10-21T17:39:55.509429+0000","last_became_active":"2023-10-21T17:39:54.510176+0000","last_became_peered":"2023-10-21T17:39:54.510176+0000","last_unstale":"2023-10-21T17:39:55.509429+0000","last_undegraded":"2023-10-21T17:39:55.509429+0000","last_fullsized":"2023-10-21T17:39:55.509429+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:52.104608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485236+0000","last_change":"2023-10-21T17:39:40.268873+0000","last_active":"2023-10-21T17:39:54.485236+0000","last_peered":"2023-10-21T17:39:54.485236+0000","last_clean":"2023-10-21T17:39:54.485236+0000","last_became_active":"2023-10-21T17:39:40.268515+0000","last_became_peered":"2023-10-21T17:39:40.268515+0000","last_unstale":"2023-10-21T17:39:54.485236+0000","last_undegraded":"2023-10-21T17:39:54.485236+0000","last_fullsized":"2023-10-21T17:39:54.485236+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:24.479277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.506703+0000","last_change":"2023-10-21T17:39:39.241987+0000","last_active":"2023-10-21T17:39:54.506703+0000","last_peered":"2023-10-21T17:39:54.506703+0000","last_clean":"2023-10-21T17:39:54.506703+0000","last_became_active":"2023-10-21T17:39:39.241676+0000","last_became_peered":"2023-10-21T17:39:39.241676+0000","last_unstale":"2023-10-21T17:39:54.506703+0000","last_undegraded":"2023-10-21T17:39:54.506703+0000","last_fullsized":"2023-10-21T17:39:54.506703+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:33.595143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866648+0000","last_change":"2023-10-21T17:39:54.513824+0000","last_active":"2023-10-21T17:39:55.866648+0000","last_peered":"2023-10-21T17:39:55.866648+0000","last_clean":"2023-10-21T17:39:55.866648+0000","last_became_active":"2023-10-21T17:39:54.513565+0000","last_became_peered":"2023-10-21T17:39:54.513565+0000","last_unstale":"2023-10-21T17:39:55.866648+0000","last_undegraded":"2023-10-21T17:39:55.866648+0000","last_fullsized":"2023-10-21T17:39:55.866648+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:49.781653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499612+0000","last_change":"2023-10-21T17:39:40.282789+0000","last_active":"2023-10-21T17:39:54.499612+0000","last_peered":"2023-10-21T17:39:54.499612+0000","last_clean":"2023-10-21T17:39:54.499612+0000","last_became_active":"2023-10-21T17:39:40.282503+0000","last_became_peered":"2023-10-21T17:39:40.282503+0000","last_unstale":"2023-10-21T17:39:54.499612+0000","last_undegraded":"2023-10-21T17:39:54.499612+0000","last_fullsized":"2023-10-21T17:39:54.499612+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:13:21.357890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.498247+0000","last_change":"2023-10-21T17:39:39.245748+0000","last_active":"2023-10-21T17:39:54.498247+0000","last_peered":"2023-10-21T17:39:54.498247+0000","last_clean":"2023-10-21T17:39:54.498247+0000","last_became_active":"2023-10-21T17:39:39.245479+0000","last_became_peered":"2023-10-21T17:39:39.245479+0000","last_unstale":"2023-10-21T17:39:54.498247+0000","last_undegraded":"2023-10-21T17:39:54.498247+0000","last_fullsized":"2023-10-21T17:39:54.498247+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.869557+0000","last_change":"2023-10-21T17:39:54.505611+0000","last_active":"2023-10-21T17:39:55.869557+0000","last_peered":"2023-10-21T17:39:55.869557+0000","last_clean":"2023-10-21T17:39:55.869557+0000","last_became_active":"2023-10-21T17:39:54.504970+0000","last_became_peered":"2023-10-21T17:39:54.504970+0000","last_unstale":"2023-10-21T17:39:55.869557+0000","last_undegraded":"2023-10-21T17:39:55.869557+0000","last_fullsized":"2023-10-21T17:39:55.869557+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:29:45.733225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497501+0000","last_change":"2023-10-21T17:39:40.277680+0000","last_active":"2023-10-21T17:39:54.497501+0000","last_peered":"2023-10-21T17:39:54.497501+0000","last_clean":"2023-10-21T17:39:54.497501+0000","last_became_active":"2023-10-21T17:39:40.277441+0000","last_became_peered":"2023-10-21T17:39:40.277441+0000","last_unstale":"2023-10-21T17:39:54.497501+0000","last_undegraded":"2023-10-21T17:39:54.497501+0000","last_fullsized":"2023-10-21T17:39:54.497501+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:13:49.789589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"254'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.508198+0000","last_change":"2023-10-21T17:39:39.246309+0000","last_active":"2023-10-21T17:39:54.508198+0000","last_peered":"2023-10-21T17:39:54.508198+0000","last_clean":"2023-10-21T17:39:54.508198+0000","last_became_active":"2023-10-21T17:39:39.245706+0000","last_became_peered":"2023-10-21T17:39:39.245706+0000","last_unstale":"2023-10-21T17:39:54.508198+0000","last_undegraded":"2023-10-21T17:39:54.508198+0000","last_fullsized":"2023-10-21T17:39:54.508198+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:22:35.456778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.866002+0000","last_change":"2023-10-21T17:39:54.506231+0000","last_active":"2023-10-21T17:39:55.866002+0000","last_peered":"2023-10-21T17:39:55.866002+0000","last_clean":"2023-10-21T17:39:55.866002+0000","last_became_active":"2023-10-21T17:39:54.505929+0000","last_became_peered":"2023-10-21T17:39:54.505929+0000","last_unstale":"2023-10-21T17:39:55.866002+0000","last_undegraded":"2023-10-21T17:39:55.866002+0000","last_fullsized":"2023-10-21T17:39:55.866002+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:48:26.485478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.499532+0000","last_change":"2023-10-21T17:39:40.281463+0000","last_active":"2023-10-21T17:39:54.499532+0000","last_peered":"2023-10-21T17:39:54.499532+0000","last_clean":"2023-10-21T17:39:54.499532+0000","last_became_active":"2023-10-21T17:39:40.281137+0000","last_became_peered":"2023-10-21T17:39:40.281137+0000","last_unstale":"2023-10-21T17:39:54.499532+0000","last_undegraded":"2023-10-21T17:39:54.499532+0000","last_fullsized":"2023-10-21T17:39:54.499532+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:03:12.244419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.513312+0000","last_change":"2023-10-21T17:39:39.227041+0000","last_active":"2023-10-21T17:39:54.513312+0000","last_peered":"2023-10-21T17:39:54.513312+0000","last_clean":"2023-10-21T17:39:54.513312+0000","last_became_active":"2023-10-21T17:39:39.225535+0000","last_became_peered":"2023-10-21T17:39:39.225535+0000","last_unstale":"2023-10-21T17:39:54.513312+0000","last_undegraded":"2023-10-21T17:39:54.513312+0000","last_fullsized":"2023-10-21T17:39:54.513312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:48:20.950421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.504487+0000","last_change":"2023-10-21T17:39:54.509781+0000","last_active":"2023-10-21T17:39:55.504487+0000","last_peered":"2023-10-21T17:39:55.504487+0000","last_clean":"2023-10-21T17:39:55.504487+0000","last_became_active":"2023-10-21T17:39:54.508599+0000","last_became_peered":"2023-10-21T17:39:54.508599+0000","last_unstale":"2023-10-21T17:39:55.504487+0000","last_undegraded":"2023-10-21T17:39:55.504487+0000","last_fullsized":"2023-10-21T17:39:55.504487+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:50:08.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.501366+0000","last_change":"2023-10-21T17:39:40.283081+0000","last_active":"2023-10-21T17:39:54.501366+0000","last_peered":"2023-10-21T17:39:54.501366+0000","last_clean":"2023-10-21T17:39:54.501366+0000","last_became_active":"2023-10-21T17:39:40.282782+0000","last_became_peered":"2023-10-21T17:39:40.282782+0000","last_unstale":"2023-10-21T17:39:54.501366+0000","last_undegraded":"2023-10-21T17:39:54.501366+0000","last_fullsized":"2023-10-21T17:39:54.501366+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:14:37.852801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.511228+0000","last_change":"2023-10-21T17:39:39.242272+0000","last_active":"2023-10-21T17:39:54.511228+0000","last_peered":"2023-10-21T17:39:54.511228+0000","last_clean":"2023-10-21T17:39:54.511228+0000","last_became_active":"2023-10-21T17:39:39.241658+0000","last_became_peered":"2023-10-21T17:39:39.241658+0000","last_unstale":"2023-10-21T17:39:54.511228+0000","last_undegraded":"2023-10-21T17:39:54.511228+0000","last_fullsized":"2023-10-21T17:39:54.511228+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:19:48.405632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.870778+0000","last_change":"2023-10-21T17:39:54.516147+0000","last_active":"2023-10-21T17:39:55.870778+0000","last_peered":"2023-10-21T17:39:55.870778+0000","last_clean":"2023-10-21T17:39:55.870778+0000","last_became_active":"2023-10-21T17:39:54.515856+0000","last_became_peered":"2023-10-21T17:39:54.515856+0000","last_unstale":"2023-10-21T17:39:55.870778+0000","last_undegraded":"2023-10-21T17:39:55.870778+0000","last_fullsized":"2023-10-21T17:39:55.870778+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:20.545832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.510788+0000","last_change":"2023-10-21T17:39:40.267491+0000","last_active":"2023-10-21T17:39:54.510788+0000","last_peered":"2023-10-21T17:39:54.510788+0000","last_clean":"2023-10-21T17:39:54.510788+0000","last_became_active":"2023-10-21T17:39:40.267154+0000","last_became_peered":"2023-10-21T17:39:40.267154+0000","last_unstale":"2023-10-21T17:39:54.510788+0000","last_undegraded":"2023-10-21T17:39:54.510788+0000","last_fullsized":"2023-10-21T17:39:54.510788+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:07:14.379177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.483519+0000","last_change":"2023-10-21T17:39:39.247298+0000","last_active":"2023-10-21T17:39:54.483519+0000","last_peered":"2023-10-21T17:39:54.483519+0000","last_clean":"2023-10-21T17:39:54.483519+0000","last_became_active":"2023-10-21T17:39:39.247016+0000","last_became_peered":"2023-10-21T17:39:39.247016+0000","last_unstale":"2023-10-21T17:39:54.483519+0000","last_undegraded":"2023-10-21T17:39:54.483519+0000","last_fullsized":"2023-10-21T17:39:54.483519+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:36:58.778201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.510462+0000","last_change":"2023-10-21T17:39:54.523451+0000","last_active":"2023-10-21T17:39:55.510462+0000","last_peered":"2023-10-21T17:39:55.510462+0000","last_clean":"2023-10-21T17:39:55.510462+0000","last_became_active":"2023-10-21T17:39:54.523163+0000","last_became_peered":"2023-10-21T17:39:54.523163+0000","last_unstale":"2023-10-21T17:39:55.510462+0000","last_undegraded":"2023-10-21T17:39:55.510462+0000","last_fullsized":"2023-10-21T17:39:55.510462+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:43.836182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.480665+0000","last_change":"2023-10-21T17:39:40.275609+0000","last_active":"2023-10-21T17:39:54.480665+0000","last_peered":"2023-10-21T17:39:54.480665+0000","last_clean":"2023-10-21T17:39:54.480665+0000","last_became_active":"2023-10-21T17:39:40.275290+0000","last_became_peered":"2023-10-21T17:39:40.275290+0000","last_unstale":"2023-10-21T17:39:54.480665+0000","last_undegraded":"2023-10-21T17:39:54.480665+0000","last_fullsized":"2023-10-21T17:39:54.480665+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:17:22.217845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.502055+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:54.502055+0000","last_peered":"2023-10-21T17:39:54.502055+0000","last_clean":"2023-10-21T17:39:54.502055+0000","last_became_active":"2023-10-21T17:39:39.238047+0000","last_became_peered":"2023-10-21T17:39:39.238047+0000","last_unstale":"2023-10-21T17:39:54.502055+0000","last_undegraded":"2023-10-21T17:39:54.502055+0000","last_fullsized":"2023-10-21T17:39:54.502055+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:10:48.769258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867207+0000","last_change":"2023-10-21T17:39:54.511000+0000","last_active":"2023-10-21T17:39:55.867207+0000","last_peered":"2023-10-21T17:39:55.867207+0000","last_clean":"2023-10-21T17:39:55.867207+0000","last_became_active":"2023-10-21T17:39:54.510718+0000","last_became_peered":"2023-10-21T17:39:54.510718+0000","last_unstale":"2023-10-21T17:39:55.867207+0000","last_undegraded":"2023-10-21T17:39:55.867207+0000","last_fullsized":"2023-10-21T17:39:55.867207+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:46:54.319191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.489520+0000","last_change":"2023-10-21T17:39:40.279881+0000","last_active":"2023-10-21T17:39:54.489520+0000","last_peered":"2023-10-21T17:39:54.489520+0000","last_clean":"2023-10-21T17:39:54.489520+0000","last_became_active":"2023-10-21T17:39:40.279567+0000","last_became_peered":"2023-10-21T17:39:40.279567+0000","last_unstale":"2023-10-21T17:39:54.489520+0000","last_undegraded":"2023-10-21T17:39:54.489520+0000","last_fullsized":"2023-10-21T17:39:54.489520+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:33:23.894434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.496282+0000","last_change":"2023-10-21T17:39:39.244421+0000","last_active":"2023-10-21T17:39:54.496282+0000","last_peered":"2023-10-21T17:39:54.496282+0000","last_clean":"2023-10-21T17:39:54.496282+0000","last_became_active":"2023-10-21T17:39:39.244026+0000","last_became_peered":"2023-10-21T17:39:39.244026+0000","last_unstale":"2023-10-21T17:39:54.496282+0000","last_undegraded":"2023-10-21T17:39:54.496282+0000","last_fullsized":"2023-10-21T17:39:54.496282+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:42:24.596691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.506919+0000","last_change":"2023-10-21T17:39:54.520421+0000","last_active":"2023-10-21T17:39:55.506919+0000","last_peered":"2023-10-21T17:39:55.506919+0000","last_clean":"2023-10-21T17:39:55.506919+0000","last_became_active":"2023-10-21T17:39:54.520078+0000","last_became_peered":"2023-10-21T17:39:54.520078+0000","last_unstale":"2023-10-21T17:39:55.506919+0000","last_undegraded":"2023-10-21T17:39:55.506919+0000","last_fullsized":"2023-10-21T17:39:55.506919+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:17:32.191887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.485197+0000","last_change":"2023-10-21T17:39:40.275308+0000","last_active":"2023-10-21T17:39:54.485197+0000","last_peered":"2023-10-21T17:39:54.485197+0000","last_clean":"2023-10-21T17:39:54.485197+0000","last_became_active":"2023-10-21T17:39:40.274998+0000","last_became_peered":"2023-10-21T17:39:40.274998+0000","last_unstale":"2023-10-21T17:39:54.485197+0000","last_undegraded":"2023-10-21T17:39:54.485197+0000","last_fullsized":"2023-10-21T17:39:54.485197+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:59:38.521395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.482593+0000","last_change":"2023-10-21T17:39:39.238364+0000","last_active":"2023-10-21T17:39:54.482593+0000","last_peered":"2023-10-21T17:39:54.482593+0000","last_clean":"2023-10-21T17:39:54.482593+0000","last_became_active":"2023-10-21T17:39:39.236393+0000","last_became_peered":"2023-10-21T17:39:39.236393+0000","last_unstale":"2023-10-21T17:39:54.482593+0000","last_undegraded":"2023-10-21T17:39:54.482593+0000","last_fullsized":"2023-10-21T17:39:54.482593+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:40.507879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-21T17:39:55.867389+0000","last_change":"2023-10-21T17:39:54.508906+0000","last_active":"2023-10-21T17:39:55.867389+0000","last_peered":"2023-10-21T17:39:55.867389+0000","last_clean":"2023-10-21T17:39:55.867389+0000","last_became_active":"2023-10-21T17:39:54.507753+0000","last_became_peered":"2023-10-21T17:39:54.507753+0000","last_unstale":"2023-10-21T17:39:55.867389+0000","last_undegraded":"2023-10-21T17:39:55.867389+0000","last_fullsized":"2023-10-21T17:39:55.867389+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:53.452761+0000","last_clean_scrub_stamp":"2023-10-21T17:39:53.452761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:16:06.148164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.478839+0000","last_change":"2023-10-21T17:39:40.270271+0000","last_active":"2023-10-21T17:39:54.478839+0000","last_peered":"2023-10-21T17:39:54.478839+0000","last_clean":"2023-10-21T17:39:54.478839+0000","last_became_active":"2023-10-21T17:39:40.269983+0000","last_became_peered":"2023-10-21T17:39:40.269983+0000","last_unstale":"2023-10-21T17:39:54.478839+0000","last_undegraded":"2023-10-21T17:39:54.478839+0000","last_fullsized":"2023-10-21T17:39:54.478839+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:39.200955+0000","last_clean_scrub_stamp":"2023-10-21T17:39:39.200955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:33.823325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.503415+0000","last_change":"2023-10-21T17:39:39.247657+0000","last_active":"2023-10-21T17:39:54.503415+0000","last_peered":"2023-10-21T17:39:54.503415+0000","last_clean":"2023-10-21T17:39:54.503415+0000","last_became_active":"2023-10-21T17:39:39.247152+0000","last_became_peered":"2023-10-21T17:39:39.247152+0000","last_unstale":"2023-10-21T17:39:54.503415+0000","last_undegraded":"2023-10-21T17:39:54.503415+0000","last_fullsized":"2023-10-21T17:39:54.503415+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:39:38.195334+0000","last_clean_scrub_stamp":"2023-10-21T17:39:38.195334+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:04:46.448339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"239'4895","reported_seq":5903,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-21T17:39:54.497340+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:39:54.497340+0000","last_peered":"2023-10-21T17:39:54.497340+0000","last_clean":"2023-10-21T17:39:54.497340+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:39:54.497340+0000","last_undegraded":"2023-10-21T17:39:54.497340+0000","last_fullsized":"2023-10-21T17:39:54.497340+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2416,"num_read_kb":5051,"num_write":627,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4895,"ondisk_log_size":4895,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576013,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75204,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296636,"statfs":{"total":96636764160,"available":96559755264,"internally_reserved":0,"allocated":3858432,"data_stored":2287981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1950000000000001,"15min":1.206},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.135,"15min":1.177},"min":{"1min":0.74299999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.6160000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1759999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.1619999999999999,"15min":1.212},"min":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.615,"5min":1.7170000000000001,"15min":1.73},"last":1.119},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.26,"15min":1.298},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.611,"5min":1.792,"15min":1.885},"last":1.4630000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.3759999999999999,"15min":1.3480000000000001},"min":{"1min":0.78100000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3260000000000001,"15min":1.325},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.90700000000000003}]},{"osd":3,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2849999999999999,"15min":1.3129999999999999},"min":{"1min":0.624,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.6739999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.294},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.4990000000000001,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.361}]},{"osd":4,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.4019999999999999,"15min":1.367},"min":{"1min":0.79900000000000004,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9039999999999999,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5129999999999999,"15min":1.492},"min":{"1min":0.76100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.804,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.494}]},{"osd":5,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.508,"15min":1.421},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.837,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.532},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.49,"15min":1.456},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.036,"5min":2.613,"15min":2.613},"last":1.016}]},{"osd":6,"last update":"Sat Oct 21 17:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1019999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5209999999999999,"15min":1.4810000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.919,"5min":2.879,"15min":2.879},"last":1.4199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576014,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42692,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329148,"statfs":{"total":96636764160,"available":96593047552,"internally_reserved":0,"allocated":3260416,"data_stored":1697031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.133,"15min":1.1890000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.671,"5min":1.944,"15min":1.944},"last":1.8069999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.163,"15min":1.212},"min":{"1min":0.71499999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.607,"5min":1.6839999999999999,"15min":1.762},"last":1.3680000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.121,"15min":1.175},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.474,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.258,"15min":1.302},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5029999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.5289999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.254,"15min":1.3049999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.532,"5min":1.706,"15min":1.8300000000000001},"last":1.244},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2809999999999999,"15min":1.296},"min":{"1min":0.92200000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.516,"5min":1.679,"15min":1.929},"last":1.587}]},{"osd":3,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.349,"15min":1.403},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.96,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.766},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.4019999999999999,"15min":1.4510000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.5800000000000001,"5min":1.871,"15min":2.048},"last":1.484}]},{"osd":4,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.28,"15min":1.341},"min":{"1min":0.97299999999999998,"5min":0.85999999999999999,"15min":0.79000000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.706},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.409,"15min":1.4330000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.806,"5min":1.8759999999999999,"15min":2.2149999999999999},"last":1.4179999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5489999999999999,"15min":1.5669999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.339},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3260000000000001,"15min":1.369},"min":{"1min":1.0169999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.3250000000000002,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.448}]},{"osd":7,"last update":"Sat Oct 21 17:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4650000000000001,"15min":1.526},"min":{"1min":1.012,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.627,"15min":1.663},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0449999999999999},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.5409999999999999},"last":1.7450000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576015,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64892,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306948,"statfs":{"total":96636764160,"available":96570314752,"internally_reserved":0,"allocated":3252224,"data_stored":1696955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.0940000000000001,"15min":1.1439999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.611,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.161,"15min":1.218},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6839999999999999,"5min":1.8009999999999999,"15min":2.2909999999999999},"last":1.2949999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2729999999999999,"15min":1.27},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6719999999999999,"5min":1.738,"15min":1.8480000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1659999999999999,"15min":1.2230000000000001},"min":{"1min":0.74399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.532,"5min":1.7829999999999999,"15min":2.4529999999999998},"last":1.0309999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.284,"15min":1.2969999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.802},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.224,"15min":1.244},"min":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.8009999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3080000000000001,"15min":1.343},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.639,"5min":1.758,"15min":1.829},"last":1.101},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.379,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.843,"5min":1.843,"15min":1.897},"last":1.5820000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.234,"15min":1.25},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.9910000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.415,"15min":1.4039999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.528}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.464,"15min":1.4830000000000001},"min":{"1min":1.0369999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8360000000000001,"5min":1.8939999999999999,"15min":2.3929999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.542,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":1.1399999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.52},"last":1.8779999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":0.96199999999999997,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.0249999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":1.639},{"interface":"front","average":{"1min":1.569,"5min":1.5289999999999999,"15min":1.5649999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.1349999999999998},"last":1.772}]}]},{"osd":4,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52416,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319424,"statfs":{"total":96636764160,"available":96583090176,"internally_reserved":0,"allocated":3264512,"data_stored":1698567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.153},"min":{"1min":0.79700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.383,"5min":1.746,"15min":1.8480000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.76500000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5109999999999999,"5min":1.659,"15min":1.869},"last":1.1599999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.58099999999999996},"max":{"1min":1.8380000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.401},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7470000000000001,"5min":1.762,"15min":1.762},"last":1.6000000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.224,"15min":1.254},"min":{"1min":0.77000000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.534},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3420000000000001,"15min":1.3799999999999999},"min":{"1min":0.88700000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.833,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.78}]},{"osd":3,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.236,"15min":1.3049999999999999},"min":{"1min":0.90300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.734,"5min":1.734,"15min":1.7949999999999999},"last":1.246},{"interface":"front","average":{"1min":1.417,"5min":1.389,"15min":1.425},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.728}]},{"osd":5,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.3440000000000001},"min":{"1min":0.93500000000000005,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.911},{"interface":"front","average":{"1min":1.371,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.131,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.1600000000000001},"last":1.96},{"interface":"front","average":{"1min":1.605,"5min":1.5860000000000001,"15min":1.597},"min":{"1min":1.2250000000000001,"5min":1.056,"15min":1.056},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":2.016}]},{"osd":7,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.5249999999999999,"15min":1.5229999999999999},"min":{"1min":0.94699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.587,"15min":1.593},"min":{"1min":1.036,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":2.052}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56592,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315248,"statfs":{"total":96636764160,"available":96578813952,"internally_reserved":0,"allocated":3276800,"data_stored":1700470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.462,"5min":1.6439999999999999,"15min":1.71},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.234,"15min":1.258},"min":{"1min":0.68000000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.53800000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2350000000000001,"5min":1.835,"15min":1.835},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2849999999999999,"15min":1.3240000000000001},"min":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.57299999999999995},"max":{"1min":1.6870000000000001,"5min":1.879,"15min":1.879},"last":1.091}]},{"osd":2,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0740000000000001,"15min":1.109},"min":{"1min":0.57199999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6539999999999999,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2,"5min":1.23,"15min":1.286},"min":{"1min":0.57499999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":2},"last":1.022}]},{"osd":4,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.403,"15min":1.4370000000000001},"min":{"1min":0.873,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.4650000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.169},"last":1.2609999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4239999999999999,"15min":1.4910000000000001},"min":{"1min":0.89900000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.9379999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.526,"15min":1.5629999999999999},"min":{"1min":0.92600000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.3100000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.538,"15min":1.5900000000000001},"min":{"1min":0.92800000000000005,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1779999999999999,"5min":2.4990000000000001,"15min":2.7410000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.552,"15min":1.6160000000000001},"min":{"1min":0.998,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0209999999999999,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.869}]},{"osd":7,"last update":"Sat Oct 21 17:38:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.637,"15min":1.665},"min":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":1.921,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":1.452},{"interface":"front","average":{"1min":1.585,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.038,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":2.1230000000000002,"5min":2.5880000000000001,"15min":2.8940000000000001},"last":1.579}]}]},{"osd":2,"up_from":13,"seq":55834576014,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56592,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315248,"statfs":{"total":96636764160,"available":96578813952,"internally_reserved":0,"allocated":3276800,"data_stored":1699753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":0.93100000000000005,"15min":0.90900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":1.0389999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.585,"5min":1.907,"15min":2.0059999999999998},"last":0.96299999999999997}]},{"osd":1,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1819999999999999,"15min":1.1919999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.321,"15min":1.361},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.585,"5min":1.958,"15min":1.958},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1950000000000001,"15min":1.1930000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.466,"15min":1.4830000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.288}]},{"osd":4,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4510000000000001,"15min":1.425},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.476},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.4399999999999999,"15min":1.4470000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.337}]},{"osd":5,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.4059999999999999,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.845,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.6240000000000001,"15min":1.613},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.548}]},{"osd":6,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.5309999999999999,"15min":1.5620000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.855,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.5169999999999999,"15min":1.52},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.3799999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:39:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.631,"15min":1.6339999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.9910000000000001,"5min":2.1520000000000001,"15min":2.4630000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.5780000000000001,"15min":1.593},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0739999999999998,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.52}]}]},{"osd":1,"up_from":13,"seq":55834576013,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52392,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319448,"statfs":{"total":96636764160,"available":96583114752,"internally_reserved":0,"allocated":3244032,"data_stored":1689406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.872,"15min":0.92700000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.411,"5min":1.411,"15min":1.976},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.878,"5min":1.0980000000000001,"15min":1.147},"min":{"1min":0.437,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.601,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.006,"15min":1.042},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.79},"last":1.363},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.179,"15min":1.226},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5960000000000001,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":1.1100000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.298,"15min":1.371},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.9239999999999999,"5min":2.0539999999999998,"15min":2.129},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.2509999999999999,"15min":1.2929999999999999},"min":{"1min":0.57599999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.94199999999999995}]},{"osd":4,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2989999999999999,"15min":1.3480000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0139999999999998,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":1.516},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.45,"15min":1.484},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0089999999999999,"5min":2.0750000000000002,"15min":2.383},"last":1.0469999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.516,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.98,"5min":2.3969999999999998,"15min":2.4750000000000001},"last":1.194},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.464,"15min":1.5249999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.1000000000000001,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.7869999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.569,"15min":1.595},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.2069999999999999,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.4890000000000001,"15min":1.5469999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.927,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.643}]},{"osd":7,"last update":"Sat Oct 21 17:39:03 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.609,"15min":1.663},"min":{"1min":0.95999999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1349999999999998,"5min":2.1829999999999998,"15min":2.548},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.696,"15min":1.7090000000000001},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.282,"5min":2.492,"15min":2.548},"last":1.425}]}]},{"osd":0,"up_from":13,"seq":55834576013,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53000,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318840,"statfs":{"total":96636764160,"available":96582492160,"internally_reserved":0,"allocated":3858432,"data_stored":2290180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.94899999999999995},"min":{"1min":0.64500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.179},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.98399999999999999,"15min":1.085},"min":{"1min":0.61299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.556,"15min":1.748},"last":0.94499999999999995}]},{"osd":2,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0980000000000001,"15min":1.163},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.504,"5min":1.8049999999999999,"15min":2.0590000000000002},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.0780000000000001,"15min":1.171},"min":{"1min":0.67500000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4259999999999999,"5min":1.708,"15min":1.851},"last":1.0449999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.079,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6359999999999999,"5min":1.897,"15min":1.9590000000000001},"last":1.319},{"interface":"front","average":{"1min":1.242,"5min":1.3140000000000001,"15min":1.3819999999999999},"min":{"1min":0.95599999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.702,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.6739999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.3340000000000001,"15min":1.421},"min":{"1min":0.877,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9470000000000001,"5min":1.9570000000000001,"15min":2.0910000000000002},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5289999999999999,"15min":1.5880000000000001},"min":{"1min":1.0620000000000001,"5min":1.018,"15min":1.018},"max":{"1min":1.917,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.988}]},{"osd":5,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.389,"15min":1.401},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.9239999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.157},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5,"15min":1.5669999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":2.4089999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.452,"15min":1.4850000000000001},"min":{"1min":0.82899999999999996,"5min":0.75,"15min":0.75},"max":{"1min":1.7310000000000001,"5min":2.0009999999999999,"15min":2.0139999999999998},"last":1.54},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.609,"15min":1.671},"min":{"1min":1.095,"5min":0.996,"15min":0.996},"max":{"1min":2.0920000000000001,"5min":2.2309999999999999,"15min":2.4940000000000002},"last":2.5259999999999998}]},{"osd":7,"last update":"Sat Oct 21 17:39:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4570000000000001,"15min":1.5309999999999999},"min":{"1min":1.002,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9910000000000001,"5min":2.1619999999999999,"15min":2.3100000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.571,"15min":1.6180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1819999999999999,"5min":2.2519999999999998,"15min":2.262},"last":1.6439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:39:59.621 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:39:59.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:39:59.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.880+0000 7efe439f5640 1 Processor -- start 2023-10-21T17:39:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.880+0000 7efe439f5640 1 -- start start 2023-10-21T17:39:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.881+0000 7efe439f5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efe3c159ac0 0x7efe3c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.881+0000 7efe439f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efe3c0ad790 0x7efe3c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.881+0000 7efe439f5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c06dee0 con 0x7efe3c15e330 2023-10-21T17:39:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe439f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c15d850 con 0x7efe3c0ad790 2023-10-21T17:39:59.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe439f5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c15db30 con 0x7efe3c159ac0 2023-10-21T17:39:59.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe40f90640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efe3c0ad790 0x7efe3c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe41f92640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efe3c15e330 0x7efe3c06c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57742/0 (socket says 172.21.15.53:57742) 2023-10-21T17:39:59.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe41f92640 1 -- 172.21.15.53:0/2148293407 learned_addr learned my addr 172.21.15.53:0/2148293407 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:39:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe41791640 1 --2- 172.21.15.53:0/2148293407 >> v2:172.21.15.124:3300/0 conn(0x7efe3c159ac0 0x7efe3c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe40f90640 1 -- 172.21.15.53:0/2148293407 >> v2:172.21.15.124:3300/0 conn(0x7efe3c159ac0 msgr2=0x7efe3c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:39:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe40f90640 1 --2- 172.21.15.53:0/2148293407 >> v2:172.21.15.124:3300/0 conn(0x7efe3c159ac0 0x7efe3c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe40f90640 1 -- 172.21.15.53:0/2148293407 >> v1:172.21.15.53:6789/0 conn(0x7efe3c15e330 legacy=0x7efe3c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe33fff640 1 -- 172.21.15.53:0/2148293407 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338311278 0 0) 0x7efe3c06dee0 con 0x7efe3c15e330 2023-10-21T17:39:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.882+0000 7efe40f90640 1 -- 172.21.15.53:0/2148293407 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c15de10 con 0x7efe3c0ad790 2023-10-21T17:39:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe40f90640 1 --2- 172.21.15.53:0/2148293407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c0702d0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06e790 tx=0x7efe2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f1cb1032cf9370 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe33fff640 1 -- 172.21.15.53:0/2148293407 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031830 con 0x7efe3c0ad790 2023-10-21T17:39:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe33fff640 1 -- 172.21.15.53:0/2148293407 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c031990 con 0x7efe3c0ad790 2023-10-21T17:39:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe33fff640 1 -- 172.21.15.53:0/2148293407 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031d20 con 0x7efe3c0ad790 2023-10-21T17:39:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe439f5640 1 -- 172.21.15.53:0/2148293407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 msgr2=0x7efe3c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.884+0000 7efe439f5640 1 --2- 172.21.15.53:0/2148293407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c0702d0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7efe3c06e790 tx=0x7efe2c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 -- 172.21.15.53:0/2148293407 shutdown_connections 2023-10-21T17:39:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 --2- 172.21.15.53:0/2148293407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c0702d0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 --2- 172.21.15.53:0/2148293407 >> v2:172.21.15.124:3300/0 conn(0x7efe3c159ac0 0x7efe3c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 -- 172.21.15.53:0/2148293407 >> 172.21.15.53:0/2148293407 conn(0x7efe3c06a380 msgr2=0x7efe3c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:39:59.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 -- 172.21.15.53:0/2148293407 shutdown_connections 2023-10-21T17:39:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.885+0000 7efe439f5640 1 -- 172.21.15.53:0/2148293407 wait complete. 2023-10-21T17:39:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.886+0000 7efe439f5640 1 Processor -- start 2023-10-21T17:39:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 -- start start 2023-10-21T17:39:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe3c159ac0 0x7efe3c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe3c15e330 0x7efe3c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c158b10 con 0x7efe3c0ad790 2023-10-21T17:39:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c157e90 con 0x7efe3c159ac0 2023-10-21T17:39:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.887+0000 7efe439f5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c158170 con 0x7efe3c15e330 2023-10-21T17:39:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53270/0 (socket says 172.21.15.53:53270) 2023-10-21T17:39:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 -- 172.21.15.53:0/3503879033 learned_addr learned my addr 172.21.15.53:0/3503879033 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:39:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 -- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe3c15e330 msgr2=0x7efe3c15b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe3c15e330 0x7efe3c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 -- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe3c159ac0 msgr2=0x7efe3c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:39:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe3c159ac0 0x7efe3c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:39:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.888+0000 7efe41791640 1 -- 172.21.15.53:0/3503879033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c009400 con 0x7efe3c0ad790 2023-10-21T17:39:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe41791640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d800 tx=0x7efe3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800b6c0 con 0x7efe3c0ad790 2023-10-21T17:39:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3800b820 con 0x7efe3c0ad790 2023-10-21T17:39:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800bbb0 con 0x7efe3c0ad790 2023-10-21T17:39:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c158450 con 0x7efe3c0ad790 2023-10-21T17:39:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.889+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c14da60 con 0x7efe3c0ad790 2023-10-21T17:39:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.890+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c067490 con 0x7efe3c0ad790 2023-10-21T17:39:59.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.891+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7efe3800bd10 con 0x7efe3c0ad790 2023-10-21T17:39:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.892+0000 7efe31ffb640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 0x7efe2006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:39:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.892+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7efe3808a510 con 0x7efe3c0ad790 2023-10-21T17:39:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.893+0000 7efe40f90640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 0x7efe2006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:39:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.894+0000 7efe40f90640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 0x7efe2006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c030000 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:39:59.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:39:59.896+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe38054d30 con 0x7efe3c0ad790 2023-10-21T17:40:00.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.130+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe3c0b0ff0 con 0x7efe3c0ad790 2023-10-21T17:40:00.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.131+0000 7efe31ffb640 1 -- 172.21.15.53:0/3503879033 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7efe38058350 con 0x7efe3c0ad790 2023-10-21T17:40:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:00.134 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:00.134 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:40:00.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 msgr2=0x7efe2006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 0x7efe2006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe2c036000 tx=0x7efe2c030000 comp rx=0 tx=0).stop 2023-10-21T17:40:00.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 msgr2=0x7efe3c14d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d800 tx=0x7efe3800dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 shutdown_connections 2023-10-21T17:40:00.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7efe2006b0b0 0x7efe2006d570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efe3c15e330 0x7efe3c15b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efe3c159ac0 0x7efe3c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 --2- 172.21.15.53:0/3503879033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe3c0ad790 0x7efe3c14d520 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.134+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 >> 172.21.15.53:0/3503879033 conn(0x7efe3c06a380 msgr2=0x7efe3c1567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.135+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 shutdown_connections 2023-10-21T17:40:00.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.135+0000 7efe439f5640 1 -- 172.21.15.53:0/3503879033 wait complete. 2023-10-21T17:40:00.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 134, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:39:53.949575+0000', 'modified': '2023-10-21T17:39:57.022551+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 38345}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_38345': {'gid': 38345, 'name': 'c', 'rank': 0, 'incarnation': 130, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.53:6835/1218730184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6833', 'nonce': 1218730184}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 1218730184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:40:00.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:40:00.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-21T17:40:00.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.798+0000 7fc22dce7640 1 Processor -- start 2023-10-21T17:40:00.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.798+0000 7fc22dce7640 1 -- start start 2023-10-21T17:40:00.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc22dce7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 0x7fc2281612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc22dce7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2280b8c30 con 0x7fc2280b4e10 2023-10-21T17:40:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc22dce7640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2280cdd80 con 0x7fc22815cd70 2023-10-21T17:40:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc22dce7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280ce060 con 0x7fc2280cd4e0 2023-10-21T17:40:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc226ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc2280b4e10 0x7fc2280d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57744/0 (socket says 172.21.15.53:57744) 2023-10-21T17:40:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.800+0000 7fc226ffd640 1 -- 172.21.15.53:0/3330323678 learned_addr learned my addr 172.21.15.53:0/3330323678 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:00.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc227fff640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 0x7fc2281612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759265060 0 0) 0x7fc2280b8c30 con 0x7fc2280b4e10 2023-10-21T17:40:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc208003520 con 0x7fc2280b4e10 2023-10-21T17:40:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2009799841 0 0) 0x7fc208003520 con 0x7fc2280b4e10 2023-10-21T17:40:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 msgr2=0x7fc2281612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 0x7fc2281612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 >> v1:172.21.15.124:6789/0 conn(0x7fc22815cd70 legacy=0x7fc2280b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280ce340 con 0x7fc2280b4e10 2023-10-21T17:40:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc210001690 con 0x7fc2280b4e10 2023-10-21T17:40:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 >> v1:172.21.15.53:6789/0 conn(0x7fc2280b4e10 legacy=0x7fc2280d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2080039a0 0x7fc208003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080047c0 0x7fc208004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 0x7fc208005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280ce060 con 0x7fc2080039a0 2023-10-21T17:40:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20800ba30 con 0x7fc2080047c0 2023-10-21T17:40:00.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.801+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20800bbf0 con 0x7fc2280cd4e0 2023-10-21T17:40:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2080039a0 0x7fc208003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 -- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 msgr2=0x7fc208005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280cd4e0 0x7fc208005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 -- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080047c0 msgr2=0x7fc208004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080047c0 0x7fc208004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080047c0 0x7fc208004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc2277fe640 1 -- 172.21.15.53:0/3330323678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208003520 con 0x7fc2080039a0 2023-10-21T17:40:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.802+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080047c0 0x7fc208004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.803+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2080039a0 0x7fc208003dc0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fc218001720 tx=0x7fc2080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6305d6f6a01643 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.803+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180052d0 con 0x7fc2080039a0 2023-10-21T17:40:00.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.803+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21802e050 con 0x7fc2080039a0 2023-10-21T17:40:00.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.803+0000 7fc2267fc640 1 -- 172.21.15.53:0/3330323678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180035f0 con 0x7fc2080039a0 2023-10-21T17:40:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 -- 172.21.15.53:0/3330323678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2080039a0 msgr2=0x7fc208003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3330323678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2080039a0 0x7fc208003dc0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fc218001720 tx=0x7fc2080057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc227fff640 1 -- 172.21.15.53:0/3330323678 reap_dead start 2023-10-21T17:40:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 -- 172.21.15.53:0/3330323678 shutdown_connections 2023-10-21T17:40:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 -- 172.21.15.53:0/3330323678 >> 172.21.15.53:0/3330323678 conn(0x7fc2280cf700 msgr2=0x7fc2280cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 -- 172.21.15.53:0/3330323678 shutdown_connections 2023-10-21T17:40:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.804+0000 7fc22dce7640 1 -- 172.21.15.53:0/3330323678 wait complete. 2023-10-21T17:40:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.806+0000 7fc22dce7640 1 Processor -- start 2023-10-21T17:40:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.806+0000 7fc22dce7640 1 -- start start 2023-10-21T17:40:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080039a0 0x7fc2281515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 0x7fc2280b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c71a0 con 0x7fc2280c8430 2023-10-21T17:40:00.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22815f030 con 0x7fc2080039a0 2023-10-21T17:40:00.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc22dce7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c7390 con 0x7fc2280c6cf0 2023-10-21T17:40:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53280/0 (socket says 172.21.15.53:53280) 2023-10-21T17:40:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.807+0000 7fc226ffd640 1 -- 172.21.15.53:0/3043584664 learned_addr learned my addr 172.21.15.53:0/3043584664 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080039a0 0x7fc2281515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc227fff640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 0x7fc2280b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 -- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 msgr2=0x7fc2280b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 0x7fc2280b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 -- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080039a0 msgr2=0x7fc2281515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080039a0 0x7fc2281515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 -- 172.21.15.53:0/3043584664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280c7670 con 0x7fc2280c8430 2023-10-21T17:40:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc227fff640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 0x7fc2280b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.808+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d810 tx=0x7fc21000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.809+0000 7fc21ffff640 1 -- 172.21.15.53:0/3043584664 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100074e0 con 0x7fc2280c8430 2023-10-21T17:40:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.809+0000 7fc21ffff640 1 -- 172.21.15.53:0/3043584664 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210007640 con 0x7fc2280c8430 2023-10-21T17:40:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.809+0000 7fc21ffff640 1 -- 172.21.15.53:0/3043584664 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210004a20 con 0x7fc2280c8430 2023-10-21T17:40:00.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.809+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2280c7950 con 0x7fc2280c8430 2023-10-21T17:40:00.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.809+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2280c7f10 con 0x7fc2280c8430 2023-10-21T17:40:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.812+0000 7fc21ffff640 1 -- 172.21.15.53:0/3043584664 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc210004b80 con 0x7fc2280c8430 2023-10-21T17:40:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.813+0000 7fc21ffff640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 0x7fc20c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.813+0000 7fc21ffff640 1 -- 172.21.15.53:0/3043584664 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc210089ce0 con 0x7fc2280c8430 2023-10-21T17:40:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.813+0000 7fc23087e740 1 Processor -- start 2023-10-21T17:40:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.813+0000 7fc23087e740 1 -- start start 2023-10-21T17:40:00.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 0x7fc20c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0df80 0x55c749de0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749de13c0 0x55c749e11250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749d017a0 con 0x55c749e0e360 2023-10-21T17:40:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749d53740 con 0x55c749e0df80 2023-10-21T17:40:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.814+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c749d50e30 con 0x55c749de13c0 2023-10-21T17:40:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53286/0 (socket says 172.21.15.53:53286) 2023-10-21T17:40:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 -- 172.21.15.53:0/415868609 learned_addr learned my addr 172.21.15.53:0/415868609 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 0x7fc20c06d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc218033000 tx=0x7fc218009030 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 -- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749de13c0 msgr2=0x55c749e11250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749de13c0 0x55c749e11250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 -- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0df80 msgr2=0x55c749de0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0df80 0x55c749de0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 -- 172.21.15.53:0/415868609 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749d10910 con 0x55c749e0e360 2023-10-21T17:40:00.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.815+0000 7fc226ffd640 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fc2100877e0 tx=0x7fc210004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb2e50e6b0e593 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc21d7fa640 1 -- 172.21.15.53:0/415868609 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210060320 con 0x55c749e0e360 2023-10-21T17:40:00.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc21d7fa640 1 -- 172.21.15.53:0/415868609 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2100547e0 con 0x55c749e0e360 2023-10-21T17:40:00.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc21d7fa640 1 -- 172.21.15.53:0/415868609 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21005e290 con 0x55c749e0e360 2023-10-21T17:40:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 -- 172.21.15.53:0/415868609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 msgr2=0x55c749e10d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fc2100877e0 tx=0x7fc210004e30 comp rx=0 tx=0).stop 2023-10-21T17:40:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 -- 172.21.15.53:0/415868609 shutdown_connections 2023-10-21T17:40:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749de13c0 0x55c749e11250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0e360 0x55c749e10d10 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 --2- 172.21.15.53:0/415868609 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0df80 0x55c749de0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 -- 172.21.15.53:0/415868609 >> 172.21.15.53:0/415868609 conn(0x55c749de3190 msgr2=0x55c749e2bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.816+0000 7fc23087e740 1 -- 172.21.15.53:0/415868609 shutdown_connections 2023-10-21T17:40:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.817+0000 7fc23087e740 1 -- 172.21.15.53:0/415868609 wait complete. 2023-10-21T17:40:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.817+0000 7fc23087e740 1 -- start start 2023-10-21T17:40:00.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.817+0000 7fc23087e740 1 Processor -- start 2023-10-21T17:40:00.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749dfe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749dfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749d4c720 con 0x55c749dfba80 2023-10-21T17:40:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749cea970 con 0x55c749e0e360 2023-10-21T17:40:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.818+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c749cea610 con 0x55c749e0df80 2023-10-21T17:40:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc23087e740 10 client.0 _ll_register_callbacks cb 0x55c749d0c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:40:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53294/0 (socket says 172.21.15.53:53294) 2023-10-21T17:40:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 -- 172.21.15.53:0/3829037756 learned_addr learned my addr 172.21.15.53:0/3829037756 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749dfe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749dfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 -- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 msgr2=0x55c749dfe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749dfe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 -- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 msgr2=0x55c749dfe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749dfe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc227fff640 1 -- 172.21.15.53:0/3829037756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2280c7f10 con 0x55c749dfba80 2023-10-21T17:40:00.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.819+0000 7fc226ffd640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749dfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc227fff640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fc2280d07a0 tx=0x7fc220007c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430ca2c558f32751 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc2167fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:40:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000f930 con 0x55c749dfba80 2023-10-21T17:40:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c749d73160 con 0x55c749dfba80 2023-10-21T17:40:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc22000fa90 con 0x55c749dfba80 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc22000fc50 con 0x55c749dfba80 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc23087e740 10 client.12210 fetch_fsmap learned FSMap version 134 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) v3 -- 0x55c749d4ada0 con 0x55c749dfba80 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.820+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c749d73160 con 0x55c749dfba80 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 134) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fc220014070 con 0x55c749dfba80 2023-10-21T17:40:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 10 client.12210 fetch_fsmap finished waiting for FSMap version 134 2023-10-21T17:40:00.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 10 client.12210 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T17:40:00.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 20 client.12210 populate_metadata read hostname 'smithi053' 2023-10-21T17:40:00.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749e46e40 0x55c749e49230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 4 client.12210 mds_command: new command op to 38345 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:40:00.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 --> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c749d1ecc0 con 0x55c749e46e40 2023-10-21T17:40:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749e46e40 0x55c749e49230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749e46e40 0x55c749e49230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d1e612d56c0bdb server_cookie=5f8654b15a2cb864 in_seq=0 out_seq=0 2023-10-21T17:40:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.821+0000 7fc2167fc640 10 client.12210 ms_handle_connect on v2:172.21.15.53:6833/1218730184 2023-10-21T17:40:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.822+0000 7fc2167fc640 1 -- 172.21.15.53:0/3829037756 <== mds.0 v2:172.21.15.53:6833/1218730184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c749d1ecc0 con 0x55c749e46e40 2023-10-21T17:40:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.822+0000 7fc2167fc640 10 client.12210 handle_command_reply: tid=0 2023-10-21T17:40:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.822+0000 7fc23087e740 1 client.12210 shutdown 2023-10-21T17:40:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 msgr2=0x55c749dfd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fc2280d07a0 tx=0x7fc220007c40 comp rx=0 tx=0).stop 2023-10-21T17:40:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 shutdown_connections 2023-10-21T17:40:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749e46e40 0x55c749e49230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1e0 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749dfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 --2- 172.21.15.53:0/3829037756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749dfe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 >> 172.21.15.53:0/3829037756 conn(0x55c749de13c0 msgr2=0x55c749e107b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc2167fc640 0 client.12210 ms_handle_reset on v2:172.21.15.53:6833/1218730184 2023-10-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 shutdown_connections 2023-10-21T17:40:00.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 1 -- 172.21.15.53:0/3829037756 wait complete. 2023-10-21T17:40:00.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.823+0000 7fc23087e740 20 client.12210 trim_cache size 0 max 16384 2023-10-21T17:40:00.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.839+0000 7fc23087e740 1 Processor -- start 2023-10-21T17:40:00.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.839+0000 7fc23087e740 1 -- start start 2023-10-21T17:40:00.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749e22af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749de0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749ab0f20 con 0x55c749dfba80 2023-10-21T17:40:00.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749e3e570 con 0x55c749e0e360 2023-10-21T17:40:00.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c749e0cd20 con 0x55c749e0df80 2023-10-21T17:40:00.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc2277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc2277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53306/0 (socket says 172.21.15.53:53306) 2023-10-21T17:40:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.840+0000 7fc2277fe640 1 -- 172.21.15.53:0/3882900361 learned_addr learned my addr 172.21.15.53:0/3882900361 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 -- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 msgr2=0x55c749e22af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749e22af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 -- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 msgr2=0x55c749de0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749de0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 -- 172.21.15.53:0/3882900361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749ce9b80 con 0x55c749dfba80 2023-10-21T17:40:00.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc2277fe640 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc218001720 tx=0x7fc21802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61442d500111887 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc21dffb640 1 -- 172.21.15.53:0/3882900361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218007640 con 0x55c749dfba80 2023-10-21T17:40:00.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc21dffb640 1 -- 172.21.15.53:0/3882900361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180067d0 con 0x55c749dfba80 2023-10-21T17:40:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.841+0000 7fc21dffb640 1 -- 172.21.15.53:0/3882900361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218006b40 con 0x55c749dfba80 2023-10-21T17:40:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 -- 172.21.15.53:0/3882900361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 msgr2=0x55c749dfd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fc218001720 tx=0x7fc21802d000 comp rx=0 tx=0).stop 2023-10-21T17:40:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 -- 172.21.15.53:0/3882900361 shutdown_connections 2023-10-21T17:40:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749de0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749e0df80 0x55c749e22af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 --2- 172.21.15.53:0/3882900361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749dfba80 0x55c749dfd1c0 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 -- 172.21.15.53:0/3882900361 >> 172.21.15.53:0/3882900361 conn(0x55c749de13c0 msgr2=0x55c749e2c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 -- 172.21.15.53:0/3882900361 shutdown_connections 2023-10-21T17:40:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.842+0000 7fc23087e740 1 -- 172.21.15.53:0/3882900361 wait complete. 2023-10-21T17:40:00.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.843+0000 7fc23087e740 1 -- start start 2023-10-21T17:40:00.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.843+0000 7fc23087e740 1 Processor -- start 2023-10-21T17:40:00.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749dfba80 0x55c749e93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749e10750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749ddff60 con 0x55c749e0df80 2023-10-21T17:40:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749de0c50 con 0x55c749e0e360 2023-10-21T17:40:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c749e0b860 con 0x55c749dfba80 2023-10-21T17:40:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc23087e740 10 client.0 _ll_register_callbacks cb 0x7fc220007440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:40:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc226ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53312/0 (socket says 172.21.15.53:53312) 2023-10-21T17:40:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.844+0000 7fc226ffd640 1 -- 172.21.15.53:0/2046599846 learned_addr learned my addr 172.21.15.53:0/2046599846 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2277fe640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749dfba80 0x55c749e93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 -- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749dfba80 msgr2=0x55c749e93660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749dfba80 0x55c749e93660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 -- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 msgr2=0x55c749e10750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749e10750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 -- 172.21.15.53:0/2046599846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc208003520 con 0x55c749e0df80 2023-10-21T17:40:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc226ffd640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc21005fdf0 tx=0x7fc210001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279a20d7515c2f11 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2167fc640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:40:00.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210073070 con 0x55c749e0df80 2023-10-21T17:40:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc210054950 con 0x55c749e0df80 2023-10-21T17:40:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2100547e0 con 0x55c749e0df80 2023-10-21T17:40:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc210060320 con 0x55c749e0df80 2023-10-21T17:40:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc23087e740 10 client.12216 fetch_fsmap learned FSMap version 134 2023-10-21T17:40:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) v3 -- 0x55c749d0f170 con 0x55c749e0df80 2023-10-21T17:40:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.845+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2100547e0 con 0x55c749e0df80 2023-10-21T17:40:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 134) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fc210076020 con 0x55c749e0df80 2023-10-21T17:40:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 10 client.12216 fetch_fsmap finished waiting for FSMap version 134 2023-10-21T17:40:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 10 client.12216 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-21T17:40:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 20 client.12216 populate_metadata read hostname 'smithi053' 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749dfab40 0x55c749dfaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 4 client.12216 mds_command: new command op to 38345 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 --> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c749d45700 con 0x55c749dfab40 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc2277fe640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749dfab40 0x55c749dfaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc2277fe640 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749dfab40 0x55c749dfaf20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c9eeeb0ba1fc04 server_cookie=de44b8b36a64b2de in_seq=0 out_seq=0 2023-10-21T17:40:00.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.846+0000 7fc2167fc640 10 client.12216 ms_handle_connect on v2:172.21.15.53:6833/1218730184 2023-10-21T17:40:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc2167fc640 1 -- 172.21.15.53:0/2046599846 <== mds.0 v2:172.21.15.53:6833/1218730184 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x55c749d45700 con 0x55c749dfab40 2023-10-21T17:40:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc2167fc640 10 client.12216 handle_command_reply: tid=0 2023-10-21T17:40:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 client.12216 shutdown 2023-10-21T17:40:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 msgr2=0x55c749e0bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fc21005fdf0 tx=0x7fc210001a20 comp rx=0 tx=0).stop 2023-10-21T17:40:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 shutdown_connections 2023-10-21T17:40:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:6833/1218730184,v1:172.21.15.53:6835/1218730184] conn(0x55c749dfab40 0x55c749dfaf20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55c749e0e360 0x55c749e10750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c749e0df80 0x55c749e0bd30 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc2167fc640 0 client.12216 ms_handle_reset on v2:172.21.15.53:6833/1218730184 2023-10-21T17:40:00.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 --2- 172.21.15.53:0/2046599846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55c749dfba80 0x55c749e93660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 >> 172.21.15.53:0/2046599846 conn(0x55c749de13c0 msgr2=0x55c749de7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 shutdown_connections 2023-10-21T17:40:00.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 1 -- 172.21.15.53:0/2046599846 wait complete. 2023-10-21T17:40:00.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.847+0000 7fc23087e740 20 client.12216 trim_cache size 0 max 16384 2023-10-21T17:40:00.864 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:40:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:40:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:40:00.864 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 38345, 2023-10-21T17:40:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:40:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:40:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "backup_fs", 2023-10-21T17:40:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.3447942849999999, 2023-10-21T17:40:00.865 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 134, 2023-10-21T17:40:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 258, 2023-10-21T17:40:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 258, 2023-10-21T17:40:00.866 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 27.435387502000001 2023-10-21T17:40:00.866 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:40:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 msgr2=0x7fc20c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 0x7fc20c06d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc218033000 tx=0x7fc218009030 comp rx=0 tx=0).stop 2023-10-21T17:40:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 msgr2=0x7fc2280c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d810 tx=0x7fc21000dcd0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 shutdown_connections 2023-10-21T17:40:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fc20c06b0b0 0x7fc20c06d570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2280c6cf0 0x7fc2280b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2280c8430 0x7fc2280c67b0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 --2- 172.21.15.53:0/3043584664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2080039a0 0x7fc2281515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:00.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 >> 172.21.15.53:0/3043584664 conn(0x7fc2280cf700 msgr2=0x7fc2280d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 shutdown_connections 2023-10-21T17:40:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:00.848+0000 7fc22dce7640 1 -- 172.21.15.53:0/3043584664 wait complete. 2023-10-21T17:40:00.870 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:40:01.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.087+0000 7f78780da640 1 Processor -- start 2023-10-21T17:40:01.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.087+0000 7f78780da640 1 -- start start 2023-10-21T17:40:01.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.088+0000 7f78780da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f787006dfc0 0x7f7870070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.088+0000 7f78780da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.088+0000 7f78780da640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78700b1cc0 con 0x7f78700afbe0 2023-10-21T17:40:01.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.088+0000 7f78780da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700b2620 con 0x7f787006dfc0 2023-10-21T17:40:01.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.088+0000 7f78780da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78700b2060 con 0x7f7870071070 2023-10-21T17:40:01.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875e76640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f78700afbe0 0x7f78700b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57750/0 (socket says 172.21.15.53:57750) 2023-10-21T17:40:01.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875e76640 1 -- 172.21.15.53:0/295734651 learned_addr learned my addr 172.21.15.53:0/295734651 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:01.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875675640 1 --2- 172.21.15.53:0/295734651 >> v2:172.21.15.53:3300/0 conn(0x7f787006dfc0 0x7f7870070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7876677640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7874e74640 1 -- 172.21.15.53:0/295734651 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820733513 0 0) 0x7f78700b1cc0 con 0x7f78700afbe0 2023-10-21T17:40:01.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7874e74640 1 -- 172.21.15.53:0/295734651 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c003520 con 0x7f78700afbe0 2023-10-21T17:40:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875675640 1 -- 172.21.15.53:0/295734651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 msgr2=0x7f78700cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875675640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875675640 1 -- 172.21.15.53:0/295734651 >> v1:172.21.15.53:6789/0 conn(0x7f78700afbe0 legacy=0x7f78700b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.089+0000 7f7875675640 1 -- 172.21.15.53:0/295734651 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700b2340 con 0x7f787006dfc0 2023-10-21T17:40:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.090+0000 7f7875675640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787006dfc0 0x7f7870070b30 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f78600097a0 tx=0x7f786002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9861d2bc1b5c4ce server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f7874e74640 1 -- 172.21.15.53:0/295734651 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786002d040 con 0x7f787006dfc0 2023-10-21T17:40:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f7874e74640 1 -- 172.21.15.53:0/295734651 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860031810 con 0x7f787006dfc0 2023-10-21T17:40:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f7874e74640 1 -- 172.21.15.53:0/295734651 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860031ba0 con 0x7f787006dfc0 2023-10-21T17:40:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 -- 172.21.15.53:0/295734651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787006dfc0 msgr2=0x7f7870070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787006dfc0 0x7f7870070b30 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f78600097a0 tx=0x7f786002b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 -- 172.21.15.53:0/295734651 shutdown_connections 2023-10-21T17:40:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 --2- 172.21.15.53:0/295734651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f787006dfc0 0x7f7870070b30 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.091+0000 7f78780da640 1 -- 172.21.15.53:0/295734651 >> 172.21.15.53:0/295734651 conn(0x7f787006a5d0 msgr2=0x7f78700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.092+0000 7f78780da640 1 -- 172.21.15.53:0/295734651 shutdown_connections 2023-10-21T17:40:01.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.092+0000 7f78780da640 1 -- 172.21.15.53:0/295734651 wait complete. 2023-10-21T17:40:01.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.093+0000 7f78780da640 1 Processor -- start 2023-10-21T17:40:01.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.093+0000 7f78780da640 1 -- start start 2023-10-21T17:40:01.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787006dfc0 0x7f7870151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78700cc6f0 con 0x7f78700afbe0 2023-10-21T17:40:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78700cf5e0 con 0x7f787006dfc0 2023-10-21T17:40:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.094+0000 7f78780da640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78700c4a60 con 0x7f7870071070 2023-10-21T17:40:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53328/0 (socket says 172.21.15.53:53328) 2023-10-21T17:40:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 -- 172.21.15.53:0/923893678 learned_addr learned my addr 172.21.15.53:0/923893678 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7875e76640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787006dfc0 0x7f7870151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 -- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 msgr2=0x7f78700cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 -- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787006dfc0 msgr2=0x7f7870151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787006dfc0 0x7f7870151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 -- 172.21.15.53:0/923893678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860009400 con 0x7f78700afbe0 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.095+0000 7f7876677640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f7868007e30 tx=0x7f786800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.096+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868014620 con 0x7f78700afbe0 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.096+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786800ee90 con 0x7f78700afbe0 2023-10-21T17:40:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.096+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78700c4d40 con 0x7f78700afbe0 2023-10-21T17:40:01.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.096+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680149c0 con 0x7f78700afbe0 2023-10-21T17:40:01.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.096+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78700cc940 con 0x7f78700afbe0 2023-10-21T17:40:01.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.097+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787004e160 con 0x7f78700afbe0 2023-10-21T17:40:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.098+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7868014b20 con 0x7f78700afbe0 2023-10-21T17:40:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.099+0000 7f78667fc640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 0x7f785406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.099+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f786808a920 con 0x7f78700afbe0 2023-10-21T17:40:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.100+0000 7f7875e76640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 0x7f785406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.101+0000 7f7875e76640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 0x7f785406d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f786c0011e0 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.103+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7868055180 con 0x7f78700afbe0 2023-10-21T17:40:01.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.322+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7870070630 con 0x7f785406b0b0 2023-10-21T17:40:01.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.326+0000 7f78667fc640 1 -- 172.21.15.53:0/923893678 <== mgr.38150 v2:172.21.15.124:6832/3816427222 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f786c005890 con 0x7f785406b0b0 2023-10-21T17:40:01.327 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:40:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:40:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "backup_fs" 2023-10-21T17:40:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:01.328 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:40:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 msgr2=0x7f785406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 0x7f785406d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f786c0011e0 tx=0x7f786c009040 comp rx=0 tx=0).stop 2023-10-21T17:40:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 msgr2=0x7f78700c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f7868007e30 tx=0x7f786800e4b0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 shutdown_connections 2023-10-21T17:40:01.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f785406b0b0 0x7f785406d570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78700afbe0 0x7f78700c3bb0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7870071070 0x7f78700cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 --2- 172.21.15.53:0/923893678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f787006dfc0 0x7f7870151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 >> 172.21.15.53:0/923893678 conn(0x7f787006a5d0 msgr2=0x7f78701591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:01.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 shutdown_connections 2023-10-21T17:40:01.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.328+0000 7f78780da640 1 -- 172.21.15.53:0/923893678 wait complete. 2023-10-21T17:40:01.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:40:01.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.569+0000 7f2a1ea26640 1 Processor -- start 2023-10-21T17:40:01.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.569+0000 7f2a1ea26640 1 -- start start 2023-10-21T17:40:01.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a180afbe0 0x7f2a180b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a1806dfc0 0x7f2a18070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2a18071070 0x7f2a180cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180b2620 con 0x7f2a180afbe0 2023-10-21T17:40:01.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180b2060 con 0x7f2a18071070 2023-10-21T17:40:01.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.570+0000 7f2a1ea26640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180b2340 con 0x7f2a1806dfc0 2023-10-21T17:40:01.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a180afbe0 0x7f2a180b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2a180afbe0 0x7f2a180b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53334/0 (socket says 172.21.15.53:53334) 2023-10-21T17:40:01.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 -- 172.21.15.53:0/1020564363 learned_addr learned my addr 172.21.15.53:0/1020564363 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:01.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 -- 172.21.15.53:0/1020564363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a1806dfc0 msgr2=0x7f2a18070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:01.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1020564363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a1806dfc0 0x7f2a18070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 -- 172.21.15.53:0/1020564363 >> v2:172.21.15.124:3300/0 conn(0x7f2a18071070 msgr2=0x7f2a180cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:01.575 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1020564363 >> v2:172.21.15.124:3300/0 conn(0x7f2a18071070 0x7f2a180cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.571+0000 7f2a17fff640 1 -- 172.21.15.53:0/1020564363 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18070350 con 0x7f2a180afbe0 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.572+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1020564363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a180afbe0 0x7f2a180b15b0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f2a1815e970 tx=0x7f2a0802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6401dec1dfc15d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a16ffd640 1 -- 172.21.15.53:0/1020564363 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08031880 con 0x7f2a180afbe0 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a16ffd640 1 -- 172.21.15.53:0/1020564363 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a080319e0 con 0x7f2a180afbe0 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a16ffd640 1 -- 172.21.15.53:0/1020564363 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08031d70 con 0x7f2a180afbe0 2023-10-21T17:40:01.576 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1020564363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a180afbe0 msgr2=0x7f2a180b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1020564363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a180afbe0 0x7f2a180b15b0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f2a1815e970 tx=0x7f2a0802b920 comp rx=0 tx=0).stop 2023-10-21T17:40:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1020564363 shutdown_connections 2023-10-21T17:40:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1020564363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a1806dfc0 0x7f2a18070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1020564363 >> v2:172.21.15.124:3300/0 conn(0x7f2a18071070 0x7f2a180cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1020564363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a180afbe0 0x7f2a180b15b0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.573+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1020564363 >> 172.21.15.53:0/1020564363 conn(0x7f2a1806a5d0 msgr2=0x7f2a180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:01.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.574+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1020564363 shutdown_connections 2023-10-21T17:40:01.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.574+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1020564363 wait complete. 2023-10-21T17:40:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.575+0000 7f2a1ea26640 1 Processor -- start 2023-10-21T17:40:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.576+0000 7f2a1ea26640 1 -- start start 2023-10-21T17:40:01.580 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.576+0000 7f2a1ea26640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a1ea26640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a18071070 0x7f2a18151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a1ea26640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a180afbe0 0x7f2a1815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a1ea26640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c3420 con 0x7f2a1806dfc0 2023-10-21T17:40:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a1ea26640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c3140 con 0x7f2a18071070 2023-10-21T17:40:01.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a1ea26640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1815d090 con 0x7f2a180afbe0 2023-10-21T17:40:01.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53336/0 (socket says 172.21.15.53:53336) 2023-10-21T17:40:01.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 -- 172.21.15.53:0/1700274295 learned_addr learned my addr 172.21.15.53:0/1700274295 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:01.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 -- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a180afbe0 msgr2=0x7f2a1815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a180afbe0 0x7f2a1815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 -- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a18071070 msgr2=0x7f2a18151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:01.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a18071070 0x7f2a18151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:01.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.577+0000 7f2a17fff640 1 -- 172.21.15.53:0/1700274295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080094d0 con 0x7f2a1806dfc0 2023-10-21T17:40:01.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a17fff640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f2a08000c00 tx=0x7f2a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0803ad30 con 0x7f2a1806dfc0 2023-10-21T17:40:01.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0803ae90 con 0x7f2a1806dfc0 2023-10-21T17:40:01.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08044630 con 0x7f2a1806dfc0 2023-10-21T17:40:01.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1815d370 con 0x7f2a1806dfc0 2023-10-21T17:40:01.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.578+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a180c50d0 con 0x7f2a1806dfc0 2023-10-21T17:40:01.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.579+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1815d890 con 0x7f2a1806dfc0 2023-10-21T17:40:01.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.580+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 136) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2a0804d450 con 0x7f2a1806dfc0 2023-10-21T17:40:01.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.581+0000 7f2a14ff9640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 0x7f2a0406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:01.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.581+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f2a080ac950 con 0x7f2a1806dfc0 2023-10-21T17:40:01.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.582+0000 7f2a177fe640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 0x7f2a0406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:01.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.583+0000 7f2a177fe640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 0x7f2a0406d570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a181f6180 tx=0x7f2a0c006bf0 comp rx=0 tx=0).ready entity=mgr.38150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:01.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.585+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a1815d890 con 0x7f2a1806dfc0 2023-10-21T17:40:01.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:01.830+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2a1815d890 con 0x7f2a1806dfc0 2023-10-21T17:40:02.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.565+0000 7f2a14ff9640 1 -- 172.21.15.53:0/1700274295 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v137) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2a1815d890 con 0x7f2a1806dfc0 2023-10-21T17:40:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.568+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 msgr2=0x7f2a0406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.568+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 0x7f2a0406d570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a181f6180 tx=0x7f2a0c006bf0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 msgr2=0x7f2a180c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f2a08000c00 tx=0x7f2a080046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 shutdown_connections 2023-10-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f2a0406b0b0 0x7f2a0406d570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a180afbe0 0x7f2a1815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a18071070 0x7f2a18151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 --2- 172.21.15.53:0/1700274295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a1806dfc0 0x7f2a180c4b90 secure :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f2a08000c00 tx=0x7f2a080046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.569+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 >> 172.21.15.53:0/1700274295 conn(0x7f2a1806a5d0 msgr2=0x7f2a1815e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.570+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 shutdown_connections 2023-10-21T17:40:02.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.570+0000 7f2a1ea26640 1 -- 172.21.15.53:0/1700274295 wait complete. 2023-10-21T17:40:02.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:40:02.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.798+0000 7ffb533f0640 1 Processor -- start 2023-10-21T17:40:02.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.798+0000 7ffb533f0640 1 -- start start 2023-10-21T17:40:02.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffb4c0cb0f0 0x7ffb4c06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cd750 0x7ffb4c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c0ca8f0 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c069d50 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.799+0000 7ffb533f0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0cdc00 con 0x7ffb4c0cd750 2023-10-21T17:40:02.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53358/0 (socket says 172.21.15.53:53358) 2023-10-21T17:40:02.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 -- 172.21.15.53:0/2158258638 learned_addr learned my addr 172.21.15.53:0/2158258638 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:02.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5118c640 1 --2- 172.21.15.53:0/2158258638 >> v2:172.21.15.124:3300/0 conn(0x7ffb4c0cb0f0 0x7ffb4c06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5198d640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cd750 0x7ffb4c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 -- 172.21.15.53:0/2158258638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cd750 msgr2=0x7ffb4c0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:02.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cd750 0x7ffb4c0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 -- 172.21.15.53:0/2158258638 >> v2:172.21.15.124:3300/0 conn(0x7ffb4c0cb0f0 msgr2=0x7ffb4c06f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:02.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 --2- 172.21.15.53:0/2158258638 >> v2:172.21.15.124:3300/0 conn(0x7ffb4c0cb0f0 0x7ffb4c06f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.800+0000 7ffb5098b640 1 -- 172.21.15.53:0/2158258638 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c06fc00 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.801+0000 7ffb5098b640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0cd1d0 tx=0x7ffb3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fa843ecc1a45a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:02.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.802+0000 7ffb43fff640 1 -- 172.21.15.53:0/2158258638 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c031870 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.802+0000 7ffb43fff640 1 -- 172.21.15.53:0/2158258638 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c0319d0 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.802+0000 7ffb43fff640 1 -- 172.21.15.53:0/2158258638 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c031d60 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 -- 172.21.15.53:0/2158258638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb4d0 msgr2=0x7ffb4c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:02.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0cd1d0 tx=0x7ffb3c02b910 comp rx=0 tx=0).stop 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 -- 172.21.15.53:0/2158258638 shutdown_connections 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cd750 0x7ffb4c0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 --2- 172.21.15.53:0/2158258638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb4d0 0x7ffb4c0ca3b0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 --2- 172.21.15.53:0/2158258638 >> v2:172.21.15.124:3300/0 conn(0x7ffb4c0cb0f0 0x7ffb4c06f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 -- 172.21.15.53:0/2158258638 >> 172.21.15.53:0/2158258638 conn(0x7ffb4c0cc2d0 msgr2=0x7ffb4c0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:02.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 -- 172.21.15.53:0/2158258638 shutdown_connections 2023-10-21T17:40:02.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.803+0000 7ffb533f0640 1 -- 172.21.15.53:0/2158258638 wait complete. 2023-10-21T17:40:02.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.805+0000 7ffb533f0640 1 Processor -- start 2023-10-21T17:40:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.805+0000 7ffb533f0640 1 -- start start 2023-10-21T17:40:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cb4d0 0x7ffb4c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 0x7ffb4c0adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c15bd60 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0ace10 con 0x7ffb4c0cd750 2023-10-21T17:40:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb533f0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0ad0f0 con 0x7ffb4c0cb4d0 2023-10-21T17:40:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53366/0 (socket says 172.21.15.53:53366) 2023-10-21T17:40:02.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 -- 172.21.15.53:0/703883677 learned_addr learned my addr 172.21.15.53:0/703883677 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5198d640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 0x7ffb4c0adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cb4d0 msgr2=0x7ffb4c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5098b640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cb4d0 0x7ffb4c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cb4d0 0x7ffb4c15d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 msgr2=0x7ffb4c0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 0x7ffb4c0adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5118c640 1 -- 172.21.15.53:0/703883677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0094d0 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.806+0000 7ffb5198d640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 0x7ffb4c0adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb5118c640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800d800 tx=0x7ffb4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:02.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb480096a0 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48009800 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48009990 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c0ad3d0 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.807+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c0ad990 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.808+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c04e160 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.810+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 137) v1 ==== 89450+0+0 (secure 0 0 0) 0x7ffb48009af0 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.811+0000 7ffb41ffb640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 0x7ffb3006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:02.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.811+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7ffb4808a400 con 0x7ffb4c0cb0f0 2023-10-21T17:40:02.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.811+0000 7ffb5098b640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 msgr2=0x7ffb3006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:02.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.811+0000 7ffb5098b640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 0x7ffb3006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:02.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:02.816+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb48054c60 con 0x7ffb4c0cb0f0 2023-10-21T17:40:03.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.012+0000 7ffb5098b640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 msgr2=0x7ffb3006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:03.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.012+0000 7ffb5098b640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 0x7ffb3006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:03.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.014+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ffb4c0b57d0 con 0x7ffb4c0cb0f0 2023-10-21T17:40:03.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.021+0000 7ffb41ffb640 1 -- 172.21.15.53:0/703883677 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffb48058280 con 0x7ffb4c0cb0f0 2023-10-21T17:40:03.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 msgr2=0x7ffb3006d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 0x7ffb3006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 msgr2=0x7ffb4c15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800d800 tx=0x7ffb4800dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 shutdown_connections 2023-10-21T17:40:03.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7ffb3006b100 0x7ffb3006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb4c0cd750 0x7ffb4c0adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb4c0cb4d0 0x7ffb4c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 --2- 172.21.15.53:0/703883677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb4c0cb0f0 0x7ffb4c15b820 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 >> 172.21.15.53:0/703883677 conn(0x7ffb4c0cc2d0 msgr2=0x7ffb4c0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:03.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 shutdown_connections 2023-10-21T17:40:03.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.024+0000 7ffb533f0640 1 -- 172.21.15.53:0/703883677 wait complete. 2023-10-21T17:40:03.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:40:03.089 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.087+0000 7efd93cb6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:03.141 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.140+0000 7efd93cb6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:03.142 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.141+0000 7fe46342d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:03.203 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.202+0000 7fe46342d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:03.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.264+0000 7f1f7ab01640 1 Processor -- start 2023-10-21T17:40:03.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.264+0000 7f1f7ab01640 1 -- start start 2023-10-21T17:40:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.265+0000 7f1f7ab01640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f7406dfc0 0x7f1f74070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.265+0000 7f1f7ab01640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 0x7f1f740cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.265+0000 7f1f7ab01640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740b1cc0 con 0x7f1f740afbe0 2023-10-21T17:40:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.265+0000 7f1f7ab01640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740b2620 con 0x7f1f7406dfc0 2023-10-21T17:40:03.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.265+0000 7f1f7ab01640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b2060 con 0x7f1f74071070 2023-10-21T17:40:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f7889d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1f740afbe0 0x7f1f740b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:57766/0 (socket says 172.21.15.53:57766) 2023-10-21T17:40:03.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f7889d640 1 -- 172.21.15.53:0/3016137562 learned_addr learned my addr 172.21.15.53:0/3016137562 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f73fff640 1 --2- 172.21.15.53:0/3016137562 >> v2:172.21.15.124:3300/0 conn(0x7f1f7406dfc0 0x7f1f74070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f7909e640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 0x7f1f740cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419512538 0 0) 0x7f1f740b1cc0 con 0x7f1f740afbe0 2023-10-21T17:40:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f5c003520 con 0x7f1f740afbe0 2023-10-21T17:40:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.266+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3278208496 0 0) 0x7f1f5c003520 con 0x7f1f740afbe0 2023-10-21T17:40:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 msgr2=0x7f1f740cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 0x7f1f740cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 >> v2:172.21.15.124:3300/0 conn(0x7f1f7406dfc0 msgr2=0x7f1f74070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 --2- 172.21.15.53:0/3016137562 >> v2:172.21.15.124:3300/0 conn(0x7f1f7406dfc0 0x7f1f74070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740b2340 con 0x7f1f740afbe0 2023-10-21T17:40:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1f64001670 con 0x7f1f740afbe0 2023-10-21T17:40:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 >> v1:172.21.15.53:6789/0 conn(0x7f1f740afbe0 legacy=0x7f1f740b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f5c003970 0x7f1f5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 0x7f1f5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f5c004d00 0x7f1f5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b2620 con 0x7f1f5c004d00 2023-10-21T17:40:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740b2060 con 0x7f1f5c003970 2023-10-21T17:40:03.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c00b980 con 0x7f1f74071070 2023-10-21T17:40:03.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f5c004d00 0x7f1f5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 -- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 msgr2=0x7f1f5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:03.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7889d640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f5c003970 0x7f1f5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f74071070 0x7f1f5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 -- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f5c003970 msgr2=0x7f1f5c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f5c003970 0x7f1f5c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7909e640 1 -- 172.21.15.53:0/3016137562 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c003520 con 0x7f1f5c004d00 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.267+0000 7f1f7889d640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f5c003970 0x7f1f5c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.269+0000 7f1f7909e640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f5c004d00 0x7f1f5c005180 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009f90 tx=0x7f1f5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a09dfcd348e68d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.269+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c02cc20 con 0x7f1f5c004d00 2023-10-21T17:40:03.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.269+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00f040 con 0x7f1f5c004d00 2023-10-21T17:40:03.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.269+0000 7f1f737fe640 1 -- 172.21.15.53:0/3016137562 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c034440 con 0x7f1f5c004d00 2023-10-21T17:40:03.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 -- 172.21.15.53:0/3016137562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f5c004d00 msgr2=0x7f1f5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/3016137562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f5c004d00 0x7f1f5c005180 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009f90 tx=0x7f1f5c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7909e640 1 -- 172.21.15.53:0/3016137562 reap_dead start 2023-10-21T17:40:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 -- 172.21.15.53:0/3016137562 shutdown_connections 2023-10-21T17:40:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 -- 172.21.15.53:0/3016137562 >> 172.21.15.53:0/3016137562 conn(0x7f1f7406a5d0 msgr2=0x7f1f740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 -- 172.21.15.53:0/3016137562 shutdown_connections 2023-10-21T17:40:03.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.270+0000 7f1f7ab01640 1 -- 172.21.15.53:0/3016137562 wait complete. 2023-10-21T17:40:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.273+0000 7f1f7ab01640 1 Processor -- start 2023-10-21T17:40:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.273+0000 7f1f7ab01640 1 -- start start 2023-10-21T17:40:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.274+0000 7f1f7ab01640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740aeda0 0x7f1f74159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7ab01640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 0x7f1f74159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7ab01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7ab01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7415e500 con 0x7f1f740af180 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7ab01640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c9c60 con 0x7f1f740aeda0 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7ab01640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c90e0 con 0x7f1f740c4770 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53384/0 (socket says 172.21.15.53:53384) 2023-10-21T17:40:03.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 -- 172.21.15.53:0/2164434738 learned_addr learned my addr 172.21.15.53:0/2164434738 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7889d640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740aeda0 0x7f1f74159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f73fff640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 0x7f1f74159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 msgr2=0x7f1f74159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 0x7f1f74159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740aeda0 msgr2=0x7f1f74159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740aeda0 0x7f1f74159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f7909e640 1 -- 172.21.15.53:0/2164434738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740c93c0 con 0x7f1f740af180 2023-10-21T17:40:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.275+0000 7f1f73fff640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 0x7f1f74159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f7909e640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c037040 tx=0x7f1f6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:03.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c034d50 con 0x7f1f740af180 2023-10-21T17:40:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c003990 con 0x7f1f740af180 2023-10-21T17:40:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c003d20 con 0x7f1f740af180 2023-10-21T17:40:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740c96a0 con 0x7f1f740af180 2023-10-21T17:40:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.276+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740c8d90 con 0x7f1f740af180 2023-10-21T17:40:03.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.277+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7404e160 con 0x7f1f740af180 2023-10-21T17:40:03.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.278+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 137) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f1f6c004030 con 0x7f1f740af180 2023-10-21T17:40:03.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.280+0000 7f1f717fa640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 0x7f1f6006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.280+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f1f6c0ab570 con 0x7f1f740af180 2023-10-21T17:40:03.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.280+0000 7f1f7889d640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 msgr2=0x7f1f6006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:03.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.280+0000 7f1f7889d640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 0x7f1f6006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:03.286 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.284+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f6c075dc0 con 0x7f1f740af180 2023-10-21T17:40:03.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.477+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f1f740b0ef0 con 0x7f1f740af180 2023-10-21T17:40:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.481+0000 7f1f7889d640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 msgr2=0x7f1f6006d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:03.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.481+0000 7f1f7889d640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 0x7f1f6006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:03.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.483+0000 7f1f717fa640 1 -- 172.21.15.53:0/2164434738 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1f6c0793e0 con 0x7f1f740af180 2023-10-21T17:40:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 msgr2=0x7f1f6006d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 0x7f1f6006d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 msgr2=0x7f1f740c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.487 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c037040 tx=0x7f1f6c009750 comp rx=0 tx=0).stop 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 shutdown_connections 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7f1f6006b100 0x7f1f6006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f740af180 0x7f1f740c2230 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740c4770 0x7f1f74159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 --2- 172.21.15.53:0/2164434738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740aeda0 0x7f1f74159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.485+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 >> 172.21.15.53:0/2164434738 conn(0x7f1f7406a5d0 msgr2=0x7f1f7415f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.486+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 shutdown_connections 2023-10-21T17:40:03.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.486+0000 7f1f7ab01640 1 -- 172.21.15.53:0/2164434738 wait complete. 2023-10-21T17:40:03.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:03.504 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-21T17:40:03.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-10-21T17:40:03.596 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.594+0000 7efd93cb6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:03.647 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.645+0000 7efd93cb6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:03.690 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.689+0000 7fe46342d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:03.696 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.695+0000 7efd93cb6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:03.746 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.745+0000 7fe46342d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:03.756 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.755+0000 7efd93cb6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:03.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.757+0000 7fb4bdf63640 1 Processor -- start 2023-10-21T17:40:03.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.757+0000 7fb4bdf63640 1 -- start start 2023-10-21T17:40:03.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bdf63640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4b815e220 0x7fb4b80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bdf63640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb4b80c6170 0x7fb4b8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bdf63640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b806a6e0 con 0x7fb4b806a290 2023-10-21T17:40:03.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bdf63640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b815f090 con 0x7fb4b815e220 2023-10-21T17:40:03.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bdf63640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80aed20 con 0x7fb4b80c6170 2023-10-21T17:40:03.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bcf61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4b815e220 0x7fb4b80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bcf61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb4b815e220 0x7fb4b80ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53396/0 (socket says 172.21.15.53:53396) 2023-10-21T17:40:03.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.759+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1091212743 learned_addr learned my addr 172.21.15.53:0/1091212743 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:03.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.760+0000 7fb4b7fff640 1 --2- 172.21.15.53:0/1091212743 >> v2:172.21.15.124:3300/0 conn(0x7fb4b80c6170 0x7fb4b8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.760+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1091212743 >> v2:172.21.15.124:3300/0 conn(0x7fb4b80c6170 msgr2=0x7fb4b8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:03.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.760+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1091212743 >> v2:172.21.15.124:3300/0 conn(0x7fb4b80c6170 0x7fb4b8069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.760+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1091212743 >> v1:172.21.15.53:6789/0 conn(0x7fb4b806a290 legacy=0x7fb4b80b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.760+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1091212743 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b815e760 con 0x7fb4b815e220 2023-10-21T17:40:03.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.761+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1091212743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b815e220 0x7fb4b80ae740 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fb4b806cb10 tx=0x7fb4ac02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b492d94f591ace server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:03.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.761+0000 7fb4b77fe640 1 -- 172.21.15.53:0/1091212743 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac030870 con 0x7fb4b815e220 2023-10-21T17:40:03.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.761+0000 7fb4b77fe640 1 -- 172.21.15.53:0/1091212743 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac0309d0 con 0x7fb4b815e220 2023-10-21T17:40:03.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.761+0000 7fb4b77fe640 1 -- 172.21.15.53:0/1091212743 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac030d60 con 0x7fb4b815e220 2023-10-21T17:40:03.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1091212743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b815e220 msgr2=0x7fb4b80ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 --2- 172.21.15.53:0/1091212743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b815e220 0x7fb4b80ae740 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fb4b806cb10 tx=0x7fb4ac02b9b0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1091212743 shutdown_connections 2023-10-21T17:40:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 --2- 172.21.15.53:0/1091212743 >> v2:172.21.15.124:3300/0 conn(0x7fb4b80c6170 0x7fb4b8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 --2- 172.21.15.53:0/1091212743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b815e220 0x7fb4b80ae740 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1091212743 >> 172.21.15.53:0/1091212743 conn(0x7fb4b80cf5f0 msgr2=0x7fb4b80cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:03.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1091212743 shutdown_connections 2023-10-21T17:40:03.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.762+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1091212743 wait complete. 2023-10-21T17:40:03.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.764+0000 7fb4bdf63640 1 Processor -- start 2023-10-21T17:40:03.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.764+0000 7fb4bdf63640 1 -- start start 2023-10-21T17:40:03.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4b806a290 0x7fb4b81526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b80c6170 0x7fb4b81f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4b815e220 0x7fb4b81efca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b814b730 con 0x7fb4b80c6170 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81f01e0 con 0x7fb4b806a290 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bdf63640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80c89c0 con 0x7fb4b815e220 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b80c6170 0x7fb4b81f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b80c6170 0x7fb4b81f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:53404/0 (socket says 172.21.15.53:53404) 2023-10-21T17:40:03.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4b7fff640 1 -- 172.21.15.53:0/1603376462 learned_addr learned my addr 172.21.15.53:0/1603376462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:03.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4b806a290 0x7fb4b81526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.765+0000 7fb4bd762640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4b815e220 0x7fb4b81efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:03.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4b815e220 msgr2=0x7fb4b81efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4b815e220 0x7fb4b81efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4b806a290 msgr2=0x7fb4b81526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4b806a290 0x7fb4b81526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0094d0 con 0x7fb4b80c6170 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4bd762640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4b815e220 0x7fb4b81efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b7fff640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4b80c6170 0x7fb4b81f4370 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400d5f0 tx=0x7fb4a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:03.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40125f0 con 0x7fb4b80c6170 2023-10-21T17:40:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a4012750 con 0x7fb4b80c6170 2023-10-21T17:40:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4012b00 con 0x7fb4b80c6170 2023-10-21T17:40:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b80c8ca0 con 0x7fb4b80c6170 2023-10-21T17:40:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.766+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b81f3b70 con 0x7fb4b80c6170 2023-10-21T17:40:03.773 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.767+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b815f500 con 0x7fb4b80c6170 2023-10-21T17:40:03.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.769+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 137) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fb4a4012c60 con 0x7fb4b80c6170 2023-10-21T17:40:03.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.769+0000 7fb4b57fa640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:03.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.770+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fb4a408a390 con 0x7fb4b80c6170 2023-10-21T17:40:03.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.770+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 msgr2=0x7fb49c06d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:03.775 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.770+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:03.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.774+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4b815f500 con 0x7fb4b80c6170 2023-10-21T17:40:03.801 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.800+0000 7fe46342d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:03.806 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.804+0000 7efd93cb6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:03.865 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.864+0000 7fe46342d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:03.916 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.914+0000 7efd93cb6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:03.916 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:03.915+0000 7fe46342d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:03.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.971+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 msgr2=0x7fb49c06d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:03.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.971+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:03.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:03.977+0000 7efd93cb6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:03.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:03.987+0000 7fb4bdf63640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4b815ed20 con 0x7fb49c06b130 2023-10-21T17:40:04.028 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.026+0000 7efd93cb6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:04.032 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.031+0000 7fe46342d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:04.086 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.084+0000 7efd93cb6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:04.098 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.097+0000 7fe46342d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:04.136 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.135+0000 7efd93cb6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:04.150 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.149+0000 7fe46342d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:04.209 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.207+0000 7fe46342d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:04.262 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.260+0000 7fe46342d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:04.372+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 msgr2=0x7fb49c06d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:04.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:04.372+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:40:04.521 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.520+0000 7efd93cb6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:04.664 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.663+0000 7fe46342d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:04.679 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.678+0000 7efd93cb6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:04.786 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.785+0000 7efd93cb6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:04.831 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.830+0000 7fe46342d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:04.836 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.834+0000 7efd93cb6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:04.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:04.941+0000 7efd93cb6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:04.944 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.943+0000 7fe46342d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:04.996 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:04.995+0000 7fe46342d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:05.050 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:05.049+0000 7efd93cb6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:05.108 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:05.107+0000 7fe46342d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:05.173+0000 7fb4bcf61640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 msgr2=0x7fb49c06d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/3816427222 2023-10-21T17:40:05.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:05.173+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:40:05.206 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:05.205+0000 7efd93cb6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:05.219 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:05.218+0000 7fe46342d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:05.255 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:05.254+0000 7efd93cb6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:05.374 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:05.373+0000 7efd93cb6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:05.382 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:05.381+0000 7fe46342d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:05.433 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:05.432+0000 7fe46342d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:05.558 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:05.556+0000 7fe46342d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:06.203 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.202+0000 7efd93cb6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:06.431 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:06.430+0000 7fe46342d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:06.496 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.495+0000 7efd93cb6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:06.552 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.552+0000 7efd93cb6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:06.666 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.666+0000 7efd93cb6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:06.686+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 138) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fb4a4054f80 con 0x7fb4b80c6170 2023-10-21T17:40:06.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:06.687+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 msgr2=0x7fb49c06d5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:06.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:06.687+0000 7fb4b57fa640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/3816427222,v1:172.21.15.124:6833/3816427222] conn(0x7fb49c06b130 0x7fb49c06d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:06.748 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:06.747+0000 7fe46342d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:06.811 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:06.810+0000 7fe46342d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:06.942 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.942 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.942 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.942 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.942 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.941+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.946 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:06.945+0000 7fe46342d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:06.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.957+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.957+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.957+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.957+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.959 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.958+0000 7efd106f7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.959 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.959+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.959 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.959+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.959+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.959+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:06.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:06.959+0000 7efd136fd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:07.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.754+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 139) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fb4a403f260 con 0x7fb4b80c6170 2023-10-21T17:40:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.755+0000 7fb4b57fa640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fb49c035690 0x7fb49c035a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:07.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.755+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 --> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb49c01d740 con 0x7fb49c035690 2023-10-21T17:40:07.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.759+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fb49c035690 0x7fb49c035a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:07.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.766+0000 7fb4bcf61640 1 --2- 172.21.15.53:0/1603376462 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fb49c035690 0x7fb49c035a70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac036000 tx=0x7fb4ac008030 comp rx=0 tx=0).ready entity=mgr.9769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:07.773 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:07.773+0000 7efd1a74b640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-10-21T17:40:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:07.772+0000 7fb4b57fa640 1 -- 172.21.15.53:0/1603376462 <== mgr.9769 v2:172.21.15.124:6832/2228122799 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7fb49c01d740 con 0x7fb49c035690 2023-10-21T17:40:07.774 INFO:teuthology.orchestra.run.smithi053.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-10-21T17:40:07.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-21T17:40:07.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:07.790 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-21T17:40:07.853 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:40:08.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.084+0000 7f8e90c61640 1 Processor -- start 2023-10-21T17:40:08.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.084+0000 7f8e90c61640 1 -- start start 2023-10-21T17:40:08.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.085+0000 7f8e90c61640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0cd8b0 0x7f8e8c0cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.085+0000 7f8e90c61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.085+0000 7f8e90c61640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c0cb390 con 0x7f8e8c0ca4d0 2023-10-21T17:40:08.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.085+0000 7f8e90c61640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0ce1d0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.085+0000 7f8e90c61640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0cadd0 con 0x7f8e8c0cd8b0 2023-10-21T17:40:08.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.086+0000 7f8e81d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:08.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.086+0000 7f8e81d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47168/0 (socket says 172.21.15.53:47168) 2023-10-21T17:40:08.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.086+0000 7f8e81d9b640 1 -- 172.21.15.53:0/1006479841 learned_addr learned my addr 172.21.15.53:0/1006479841 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:08.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.086+0000 7f8e8a59c640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0cd8b0 0x7f8e8c0cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:08.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.087+0000 7f8e81d9b640 1 -- 172.21.15.53:0/1006479841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0cd8b0 msgr2=0x7f8e8c0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:08.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.087+0000 7f8e81d9b640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0cd8b0 0x7f8e8c0cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.087+0000 7f8e81d9b640 1 -- 172.21.15.53:0/1006479841 >> v1:172.21.15.53:6789/0 conn(0x7f8e8c0ca4d0 legacy=0x7f8e8c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.087+0000 7f8e81d9b640 1 -- 172.21.15.53:0/1006479841 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0cb0b0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.088+0000 7f8e81d9b640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400bb10 tx=0x7f8e7402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70833bbbad82f90 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:08.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.089+0000 7f8e89d9b640 1 -- 172.21.15.53:0/1006479841 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74036440 con 0x7f8e8c06eea0 2023-10-21T17:40:08.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.089+0000 7f8e89d9b640 1 -- 172.21.15.53:0/1006479841 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e740365a0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.089+0000 7f8e89d9b640 1 -- 172.21.15.53:0/1006479841 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74036930 con 0x7f8e8c06eea0 2023-10-21T17:40:08.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 -- 172.21.15.53:0/1006479841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 msgr2=0x7f8e8c15a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400bb10 tx=0x7f8e7402e150 comp rx=0 tx=0).stop 2023-10-21T17:40:08.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 -- 172.21.15.53:0/1006479841 shutdown_connections 2023-10-21T17:40:08.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c15a8f0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 --2- 172.21.15.53:0/1006479841 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0cd8b0 0x7f8e8c0cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 -- 172.21.15.53:0/1006479841 >> 172.21.15.53:0/1006479841 conn(0x7f8e8c0cebd0 msgr2=0x7f8e8c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:08.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 -- 172.21.15.53:0/1006479841 shutdown_connections 2023-10-21T17:40:08.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.090+0000 7f8e90c61640 1 -- 172.21.15.53:0/1006479841 wait complete. 2023-10-21T17:40:08.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.092+0000 7f8e90c61640 1 Processor -- start 2023-10-21T17:40:08.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.092+0000 7f8e90c61640 1 -- start start 2023-10-21T17:40:08.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0ca4d0 0x7f8e8c1e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e8c0cd8b0 0x7f8e8c1e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c1e6fd0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0c7440 con 0x7f8e8c0cd8b0 2023-10-21T17:40:08.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e90c61640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0c6830 con 0x7f8e8c0ca4d0 2023-10-21T17:40:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47186/0 (socket says 172.21.15.53:47186) 2023-10-21T17:40:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 -- 172.21.15.53:0/2022646141 learned_addr learned my addr 172.21.15.53:0/2022646141 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8ad9d640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e8c0cd8b0 0x7f8e8c1e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:08.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 -- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0ca4d0 msgr2=0x7f8e8c1e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0ca4d0 0x7f8e8c1e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 -- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e8c0cd8b0 msgr2=0x7f8e8c1e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e8c0cd8b0 0x7f8e8c1e5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.093+0000 7f8e8a59c640 1 -- 172.21.15.53:0/2022646141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7400b770 con 0x7f8e8c06eea0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e8a59c640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e8a0 tx=0x7f8e7c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0094f0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c00c940 con 0x7f8e8c06eea0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c0c6b10 con 0x7f8e8c06eea0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00ccf0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.094+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c0c70d0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.095+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c04e160 con 0x7f8e8c06eea0 2023-10-21T17:40:08.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.097+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 139) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8e7c013470 con 0x7f8e8c06eea0 2023-10-21T17:40:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.097+0000 7f8e837fe640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 0x7f8e6c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:08.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.098+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f8e7c0899f0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.098+0000 7f8e81d9b640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 0x7f8e6c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:08.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.099+0000 7f8e81d9b640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 0x7f8e6c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e74038000 tx=0x7f8e74030000 comp rx=0 tx=0).ready entity=mgr.9769 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:08.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.103+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e7c054720 con 0x7f8e8c06eea0 2023-10-21T17:40:08.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.359+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8e8c0cfae0 con 0x7f8e8c06eea0 2023-10-21T17:40:08.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.759+0000 7f8e837fe640 1 -- 172.21.15.53:0/2022646141 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v140) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8e7c057d40 con 0x7f8e8c06eea0 2023-10-21T17:40:08.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 msgr2=0x7f8e6c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 0x7f8e6c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e74038000 tx=0x7f8e74030000 comp rx=0 tx=0).stop 2023-10-21T17:40:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 msgr2=0x7f8e8c14e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00e8a0 tx=0x7f8e7c00ed60 comp rx=0 tx=0).stop 2023-10-21T17:40:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 shutdown_connections 2023-10-21T17:40:08.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f8e6c06ad10 0x7f8e6c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e8c0cd8b0 0x7f8e8c1e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e8c0ca4d0 0x7f8e8c1e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 --2- 172.21.15.53:0/2022646141 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e8c06eea0 0x7f8e8c14e380 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:08.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.766+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 >> 172.21.15.53:0/2022646141 conn(0x7f8e8c0cebd0 msgr2=0x7f8e8c15c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:08.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.767+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 shutdown_connections 2023-10-21T17:40:08.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:08.767+0000 7f8e90c61640 1 -- 172.21.15.53:0/2022646141 wait complete. 2023-10-21T17:40:08.789 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:40:08.790 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:40:08.890 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:40:08.980 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:40:08.980 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:08.981 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:09.034 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:09.034 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:09.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:09.277 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.276+0000 7f8c1304c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:09.294 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.294+0000 7f909c60f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:09.333 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.332+0000 7f8c1304c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:09.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.335+0000 7f37eb646640 1 Processor -- start 2023-10-21T17:40:09.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.335+0000 7f37eb646640 1 -- start start 2023-10-21T17:40:09.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37eb646640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37e40c90b0 0x7f37e40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37eb646640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40cb3b0 0x7f37e415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37eb646640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e415b2b0 con 0x7f37e415ab40 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37eb646640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e415bf30 con 0x7f37e40c90b0 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37eb646640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cb820 con 0x7f37e40cb3b0 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37e8be1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37e40c90b0 0x7f37e40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37e8be1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f37e40c90b0 0x7f37e40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47220/0 (socket says 172.21.15.53:47220) 2023-10-21T17:40:09.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37e8be1640 1 -- 172.21.15.53:0/3924125142 learned_addr learned my addr 172.21.15.53:0/3924125142 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:09.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.337+0000 7f37e9be3640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40cb3b0 0x7f37e415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.338+0000 7f37e8be1640 1 -- 172.21.15.53:0/3924125142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40cb3b0 msgr2=0x7f37e415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:09.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.338+0000 7f37e8be1640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40cb3b0 0x7f37e415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.338+0000 7f37e8be1640 1 -- 172.21.15.53:0/3924125142 >> v1:172.21.15.124:6789/0 conn(0x7f37e415ab40 legacy=0x7f37e40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.338+0000 7f37e8be1640 1 -- 172.21.15.53:0/3924125142 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e415b590 con 0x7f37e40c90b0 2023-10-21T17:40:09.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.339+0000 7f37e8be1640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40c90b0 0x7f37e40cae70 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885db853e06c9bb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:09.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.339+0000 7f37dbfff640 1 -- 172.21.15.53:0/3924125142 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4030840 con 0x7f37e40c90b0 2023-10-21T17:40:09.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.339+0000 7f37dbfff640 1 -- 172.21.15.53:0/3924125142 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40309a0 con 0x7f37e40c90b0 2023-10-21T17:40:09.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.339+0000 7f37dbfff640 1 -- 172.21.15.53:0/3924125142 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4030d30 con 0x7f37e40c90b0 2023-10-21T17:40:09.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 -- 172.21.15.53:0/3924125142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40c90b0 msgr2=0x7f37e40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40c90b0 0x7f37e40cae70 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d402b980 comp rx=0 tx=0).stop 2023-10-21T17:40:09.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 -- 172.21.15.53:0/3924125142 shutdown_connections 2023-10-21T17:40:09.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40cb3b0 0x7f37e415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 --2- 172.21.15.53:0/3924125142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40c90b0 0x7f37e40cae70 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 -- 172.21.15.53:0/3924125142 >> 172.21.15.53:0/3924125142 conn(0x7f37e406a380 msgr2=0x7f37e40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:09.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 -- 172.21.15.53:0/3924125142 shutdown_connections 2023-10-21T17:40:09.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.340+0000 7f37eb646640 1 -- 172.21.15.53:0/3924125142 wait complete. 2023-10-21T17:40:09.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.342+0000 7f37eb646640 1 Processor -- start 2023-10-21T17:40:09.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.342+0000 7f37eb646640 1 -- start start 2023-10-21T17:40:09.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 0x7f37e4151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 0x7f37e40abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4150c20 con 0x7f37e40cb3b0 2023-10-21T17:40:09.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e415e0e0 con 0x7f37e415ab40 2023-10-21T17:40:09.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37eb646640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e415e3c0 con 0x7f37e40c90b0 2023-10-21T17:40:09.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37e8be1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37e8be1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47224/0 (socket says 172.21.15.53:47224) 2023-10-21T17:40:09.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37e8be1640 1 -- 172.21.15.53:0/450322895 learned_addr learned my addr 172.21.15.53:0/450322895 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:09.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37e93e2640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 0x7f37e4151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.343+0000 7f37e9be3640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 0x7f37e40abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.347+0000 7f909c60f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:09.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 msgr2=0x7f37e4151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 0x7f37e4151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 msgr2=0x7f37e40abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 0x7f37e40abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 -- 172.21.15.53:0/450322895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40094d0 con 0x7f37e40cb3b0 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e9be3640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 0x7f37e40abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:09.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e93e2640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 0x7f37e4151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:40:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37e8be1640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4039a90 con 0x7f37e40cb3b0 2023-10-21T17:40:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d4039bf0 con 0x7f37e40cb3b0 2023-10-21T17:40:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4043410 con 0x7f37e40cb3b0 2023-10-21T17:40:09.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e4153a80 con 0x7f37e40cb3b0 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.344+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e4153730 con 0x7f37e40cb3b0 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.345+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e415e580 con 0x7f37e40cb3b0 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.346+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f37d402f390 con 0x7f37e40cb3b0 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.347+0000 7f37d9ffb640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 0x7f37c806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.347+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f37d40ac4b0 con 0x7f37e40cb3b0 2023-10-21T17:40:09.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.348+0000 7f37e93e2640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 msgr2=0x7f37c806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:09.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.348+0000 7f37e93e2640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 0x7f37c806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:09.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.356+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37e415e580 con 0x7f37e40cb3b0 2023-10-21T17:40:09.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.549+0000 7f37e93e2640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 msgr2=0x7f37c806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:09.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.549+0000 7f37e93e2640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 0x7f37c806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:09.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.552+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37e415e580 con 0x7f37e40cb3b0 2023-10-21T17:40:09.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.555+0000 7f37d9ffb640 1 -- 172.21.15.53:0/450322895 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v259) v1 ==== 74+0+21250 (secure 0 0 0) 0x7f37d4098320 con 0x7f37e40cb3b0 2023-10-21T17:40:09.557 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:09.557 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":259,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:06.671369+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:09.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 msgr2=0x7f37c806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 0x7f37c806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 msgr2=0x7f37e41506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d4004730 comp rx=0 tx=0).stop 2023-10-21T17:40:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 shutdown_connections 2023-10-21T17:40:09.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f37c806b0b0 0x7f37c806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37e415ab40 0x7f37e40abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37e40cb3b0 0x7f37e41506e0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 --2- 172.21.15.53:0/450322895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37e40c90b0 0x7f37e4151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 >> 172.21.15.53:0/450322895 conn(0x7f37e406a380 msgr2=0x7f37e4156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:09.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 shutdown_connections 2023-10-21T17:40:09.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.558+0000 7f37eb646640 1 -- 172.21.15.53:0/450322895 wait complete. 2023-10-21T17:40:09.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:39:37.842433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:39:38.736221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:39:52.278617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:39:52.975528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:40:09.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:40:09.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.793+0000 7ff7856a2640 1 Processor -- start 2023-10-21T17:40:09.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.793+0000 7ff7856a2640 1 -- start start 2023-10-21T17:40:09.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7856a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff7800b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7856a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7800b6cd0 0x7ff7800cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7856a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800b7e70 con 0x7ff7800cde70 2023-10-21T17:40:09.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7856a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7800ce300 con 0x7ff7800b6cd0 2023-10-21T17:40:09.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7856a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78015d080 con 0x7ff78015cca0 2023-10-21T17:40:09.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff7767fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7800b6cd0 0x7ff7800cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff77f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7ff7800cde70 0x7ff7800b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45358/0 (socket says 172.21.15.53:45358) 2023-10-21T17:40:09.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.795+0000 7ff77f7fe640 1 -- 172.21.15.53:0/1238814871 learned_addr learned my addr 172.21.15.53:0/1238814871 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:09.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.796+0000 7ff77effd640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff7800b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.799 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.797+0000 7f909c60f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:09.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.796+0000 7ff7767fc640 1 -- 172.21.15.53:0/1238814871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 msgr2=0x7ff7800b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:09.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.796+0000 7ff7767fc640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff7800b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.796+0000 7ff7767fc640 1 -- 172.21.15.53:0/1238814871 >> v1:172.21.15.53:6789/0 conn(0x7ff7800cde70 legacy=0x7ff7800b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.796+0000 7ff7767fc640 1 -- 172.21.15.53:0/1238814871 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800b7260 con 0x7ff7800b6cd0 2023-10-21T17:40:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.797+0000 7ff7767fc640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cd930 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7ff7680097a0 tx=0x7ff76802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f8be6d8f5c7e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.797+0000 7ff77e7fc640 1 -- 172.21.15.53:0/1238814871 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768030800 con 0x7ff7800b6cd0 2023-10-21T17:40:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.797+0000 7ff77e7fc640 1 -- 172.21.15.53:0/1238814871 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768030960 con 0x7ff7800b6cd0 2023-10-21T17:40:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.797+0000 7ff77e7fc640 1 -- 172.21.15.53:0/1238814871 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768030cf0 con 0x7ff7800b6cd0 2023-10-21T17:40:09.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 -- 172.21.15.53:0/1238814871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800b6cd0 msgr2=0x7ff7800cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cd930 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7ff7680097a0 tx=0x7ff76802b940 comp rx=0 tx=0).stop 2023-10-21T17:40:09.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 -- 172.21.15.53:0/1238814871 shutdown_connections 2023-10-21T17:40:09.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cd930 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 --2- 172.21.15.53:0/1238814871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff7800b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 -- 172.21.15.53:0/1238814871 >> 172.21.15.53:0/1238814871 conn(0x7ff7800718a0 msgr2=0x7ff780071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:09.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 -- 172.21.15.53:0/1238814871 shutdown_connections 2023-10-21T17:40:09.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.798+0000 7ff7856a2640 1 -- 172.21.15.53:0/1238814871 wait complete. 2023-10-21T17:40:09.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.800+0000 7ff7856a2640 1 Processor -- start 2023-10-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.800+0000 7ff7856a2640 1 -- start start 2023-10-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff78015f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff780147480 con 0x7ff7800cde70 2023-10-21T17:40:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c9d30 con 0x7ff7800b6cd0 2023-10-21T17:40:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7856a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800ca2f0 con 0x7ff78015cca0 2023-10-21T17:40:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47244/0 (socket says 172.21.15.53:47244) 2023-10-21T17:40:09.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 -- 172.21.15.53:0/399217869 learned_addr learned my addr 172.21.15.53:0/399217869 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff77f7fe640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff78015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff77effd640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 -- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 msgr2=0x7ff78015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff78015f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 -- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7800b6cd0 msgr2=0x7ff7800cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.801+0000 7ff7767fc640 1 -- 172.21.15.53:0/399217869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768009400 con 0x7ff7800cde70 2023-10-21T17:40:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff7767fc640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7ff768009890 tx=0x7ff7680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76803e070 con 0x7ff7800cde70 2023-10-21T17:40:09.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff768039b00 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768039e90 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78015ded0 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.802+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7800c9f80 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.803+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7800c6570 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.804+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7ff76804b400 con 0x7ff7800cde70 2023-10-21T17:40:09.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.806+0000 7ff777fff640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 0x7ff76006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.806+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7ff7680ac760 con 0x7ff7800cde70 2023-10-21T17:40:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.806+0000 7ff77effd640 1 -- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 msgr2=0x7ff76006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.806+0000 7ff77effd640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 0x7ff76006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:09.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.810+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff768076cd0 con 0x7ff7800cde70 2023-10-21T17:40:09.820 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.819+0000 7f8c1304c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:09.849 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.849+0000 7f909c60f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:09.875 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.874+0000 7f8c1304c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:09.898 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.898+0000 7f909c60f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:09.928 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.927+0000 7f8c1304c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:09.958 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:09.958+0000 7f909c60f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:09.992 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:09.991+0000 7f8c1304c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.999+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff78004e160 con 0x7ff7800cde70 2023-10-21T17:40:10.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:09.999+0000 7ff777fff640 1 -- 172.21.15.53:0/399217869 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v259) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff76807a2f0 con 0x7ff7800cde70 2023-10-21T17:40:10.001 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:40:10.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 msgr2=0x7ff76006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:10.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 0x7ff76006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 msgr2=0x7ff78015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7ff768009890 tx=0x7ff7680046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 shutdown_connections 2023-10-21T17:40:10.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7ff76006b0b0 0x7ff76006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff78015cca0 0x7ff78015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7800cde70 0x7ff78015ef40 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 --2- 172.21.15.53:0/399217869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7800b6cd0 0x7ff7800cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 >> 172.21.15.53:0/399217869 conn(0x7ff7800718a0 msgr2=0x7ff78015ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:10.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 shutdown_connections 2023-10-21T17:40:10.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.002+0000 7ff7856a2640 1 -- 172.21.15.53:0/399217869 wait complete. 2023-10-21T17:40:10.009 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.006+0000 7f909c60f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:10.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:40:10.047 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.046+0000 7f8c1304c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:10.117 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.116+0000 7f909c60f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:10.165 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.164+0000 7f8c1304c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:10.178 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.178+0000 7f909c60f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:10.228 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.227+0000 7f909c60f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:10.231 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.230+0000 7f8c1304c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:10.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.232+0000 7fc1213a5640 1 Processor -- start 2023-10-21T17:40:10.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.233+0000 7fc1213a5640 1 -- start start 2023-10-21T17:40:10.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.234+0000 7fc1213a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15d0c0 0x7fc11c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.234+0000 7fc1213a5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc11c0b30d0 0x7fc11c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.234+0000 7fc1213a5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c06ce20 con 0x7fc11c15cce0 2023-10-21T17:40:10.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.234+0000 7fc1213a5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0cf850 con 0x7fc11c0b30d0 2023-10-21T17:40:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.234+0000 7fc1213a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c06c060 con 0x7fc11c15d0c0 2023-10-21T17:40:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc11c15cce0 0x7fc11c0cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45372/0 (socket says 172.21.15.53:45372) 2023-10-21T17:40:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11affd640 1 -- 172.21.15.53:0/3816628080 learned_addr learned my addr 172.21.15.53:0/3816628080 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/3816628080 >> v2:172.21.15.53:3300/0 conn(0x7fc11c0b30d0 0x7fc11c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11a7fc640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15d0c0 0x7fc11c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc119ffb640 1 -- 172.21.15.53:0/3816628080 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436270477 0 0) 0x7fc11c06ce20 con 0x7fc11c15cce0 2023-10-21T17:40:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc119ffb640 1 -- 172.21.15.53:0/3816628080 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc104003520 con 0x7fc11c15cce0 2023-10-21T17:40:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11b7fe640 1 -- 172.21.15.53:0/3816628080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15d0c0 msgr2=0x7fc11c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15d0c0 0x7fc11c0b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11b7fe640 1 -- 172.21.15.53:0/3816628080 >> v1:172.21.15.53:6789/0 conn(0x7fc11c15cce0 legacy=0x7fc11c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.235+0000 7fc11b7fe640 1 -- 172.21.15.53:0/3816628080 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c06c340 con 0x7fc11c0b30d0 2023-10-21T17:40:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.237+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0b30d0 0x7fc11c0b2a70 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fc114009d90 tx=0x7fc11402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9973add80d683162 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.237+0000 7fc119ffb640 1 -- 172.21.15.53:0/3816628080 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11402cc60 con 0x7fc11c0b30d0 2023-10-21T17:40:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.237+0000 7fc119ffb640 1 -- 172.21.15.53:0/3816628080 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11402cdc0 con 0x7fc11c0b30d0 2023-10-21T17:40:10.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.237+0000 7fc119ffb640 1 -- 172.21.15.53:0/3816628080 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114034640 con 0x7fc11c0b30d0 2023-10-21T17:40:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 -- 172.21.15.53:0/3816628080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0b30d0 msgr2=0x7fc11c0b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0b30d0 0x7fc11c0b2a70 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fc114009d90 tx=0x7fc11402beb0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 -- 172.21.15.53:0/3816628080 shutdown_connections 2023-10-21T17:40:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0b30d0 0x7fc11c0b2a70 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 --2- 172.21.15.53:0/3816628080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15d0c0 0x7fc11c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 -- 172.21.15.53:0/3816628080 >> 172.21.15.53:0/3816628080 conn(0x7fc11c06e650 msgr2=0x7fc11c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 -- 172.21.15.53:0/3816628080 shutdown_connections 2023-10-21T17:40:10.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.238+0000 7fc1213a5640 1 -- 172.21.15.53:0/3816628080 wait complete. 2023-10-21T17:40:10.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.240+0000 7fc1213a5640 1 Processor -- start 2023-10-21T17:40:10.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.240+0000 7fc1213a5640 1 -- start start 2023-10-21T17:40:10.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 0x7fc11c1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15cce0 0x7fc11c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c1f25f0 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15eb00 con 0x7fc11c0b30d0 2023-10-21T17:40:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc1213a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15df60 con 0x7fc11c15cce0 2023-10-21T17:40:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47274/0 (socket says 172.21.15.53:47274) 2023-10-21T17:40:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc11b7fe640 1 -- 172.21.15.53:0/1487123113 learned_addr learned my addr 172.21.15.53:0/1487123113 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:10.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc11a7fc640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15cce0 0x7fc11c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.241+0000 7fc11affd640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 0x7fc11c1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 -- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15cce0 msgr2=0x7fc11c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15cce0 0x7fc11c0ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 -- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 msgr2=0x7fc11c1f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 0x7fc11c1f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 -- 172.21.15.53:0/1487123113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009a70 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11affd640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 0x7fc11c1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc11b7fe640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fc11402c4c0 tx=0x7fc114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11403e610 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11403e770 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11403eb20 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c15e1d0 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.242+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c15e790 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.244+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c160710 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.245+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fc11403b050 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.245+0000 7fc1037fe640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 0x7fc10806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.246+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fc1140ac100 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.246+0000 7fc11affd640 1 -- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 msgr2=0x7fc10806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:10.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.246+0000 7fc11affd640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 0x7fc10806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:10.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.250+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1140768f0 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.285 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.285+0000 7f909c60f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:10.286 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.285+0000 7f8c1304c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:10.335 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.335+0000 7f909c60f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:10.346 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.345+0000 7f8c1304c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:10.401 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.400+0000 7f8c1304c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:10.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.441+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc11c04e160 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.443 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.442+0000 7fc1037fe640 1 -- 172.21.15.53:0/1487123113 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v259) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc114009450 con 0x7fc11c0ca8d0 2023-10-21T17:40:10.443 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:40:10.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 msgr2=0x7fc10806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 0x7fc10806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 msgr2=0x7fc11c15f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fc11402c4c0 tx=0x7fc114004a40 comp rx=0 tx=0).stop 2023-10-21T17:40:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 shutdown_connections 2023-10-21T17:40:10.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc10806b0b0 0x7fc10806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc11c0ca8d0 0x7fc11c15f2e0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc11c15cce0 0x7fc11c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 --2- 172.21.15.53:0/1487123113 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc11c0b30d0 0x7fc11c1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 >> 172.21.15.53:0/1487123113 conn(0x7fc11c06e650 msgr2=0x7fc11c0b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:10.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 shutdown_connections 2023-10-21T17:40:10.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.444+0000 7fc1213a5640 1 -- 172.21.15.53:0/1487123113 wait complete. 2023-10-21T17:40:10.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:40:10.719 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.718+0000 7f909c60f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:10.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.725+0000 7fc033a5d640 1 Processor -- start 2023-10-21T17:40:10.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.725+0000 7fc033a5d640 1 -- start start 2023-10-21T17:40:10.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.726+0000 7fc033a5d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 0x7fc02c0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.726+0000 7fc033a5d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c0cf840 con 0x7fc02c06bc80 2023-10-21T17:40:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.726+0000 7fc033a5d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c06cd90 con 0x7fc02c0cf3e0 2023-10-21T17:40:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.726+0000 7fc033a5d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c160ca0 con 0x7fc02c15ce10 2023-10-21T17:40:10.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.727+0000 7fc031ffa640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc02c06bc80 0x7fc02c0cb770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45388/0 (socket says 172.21.15.53:45388) 2023-10-21T17:40:10.729 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.727+0000 7fc031ffa640 1 -- 172.21.15.53:0/3338316637 learned_addr learned my addr 172.21.15.53:0/3338316637 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.727+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 0x7fc02c0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.727+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619480433 0 0) 0x7fc02c0cf840 con 0x7fc02c06bc80 2023-10-21T17:40:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.727+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc018003520 con 0x7fc02c06bc80 2023-10-21T17:40:10.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665568726 0 0) 0x7fc018003520 con 0x7fc02c06bc80 2023-10-21T17:40:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 msgr2=0x7fc02c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 0x7fc02c0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 >> v1:172.21.15.124:6789/0 conn(0x7fc02c0cf3e0 legacy=0x7fc02c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c06c180 con 0x7fc02c06bc80 2023-10-21T17:40:10.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc024001670 con 0x7fc02c06bc80 2023-10-21T17:40:10.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 >> v1:172.21.15.53:6789/0 conn(0x7fc02c06bc80 legacy=0x7fc02c0cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0180039a0 0x7fc018003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180047c0 0x7fc018004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 0x7fc018005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c160ca0 con 0x7fc0180039a0 2023-10-21T17:40:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01800bb10 con 0x7fc0180047c0 2023-10-21T17:40:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01800bcd0 con 0x7fc02c15ce10 2023-10-21T17:40:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0180039a0 0x7fc018003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.728+0000 7fc030ff8640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180047c0 0x7fc018004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc0317f9640 1 -- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 msgr2=0x7fc018005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c15ce10 0x7fc018005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc0317f9640 1 -- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180047c0 msgr2=0x7fc018004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180047c0 0x7fc018004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc0317f9640 1 -- 172.21.15.53:0/3338316637 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc018003520 con 0x7fc0180039a0 2023-10-21T17:40:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.729+0000 7fc030ff8640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180047c0 0x7fc018004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.730+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0180039a0 0x7fc018003dc0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc028009fc0 tx=0x7fc0180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92920a5b1985f131 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.730+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02802cc60 con 0x7fc0180039a0 2023-10-21T17:40:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.730+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800f040 con 0x7fc0180039a0 2023-10-21T17:40:10.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.730+0000 7fc023fff640 1 -- 172.21.15.53:0/3338316637 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028034440 con 0x7fc0180039a0 2023-10-21T17:40:10.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 -- 172.21.15.53:0/3338316637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0180039a0 msgr2=0x7fc018003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3338316637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0180039a0 0x7fc018003dc0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7fc028009fc0 tx=0x7fc0180057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc031ffa640 1 -- 172.21.15.53:0/3338316637 reap_dead start 2023-10-21T17:40:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 -- 172.21.15.53:0/3338316637 shutdown_connections 2023-10-21T17:40:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 -- 172.21.15.53:0/3338316637 >> 172.21.15.53:0/3338316637 conn(0x7fc02c06e630 msgr2=0x7fc02c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 -- 172.21.15.53:0/3338316637 shutdown_connections 2023-10-21T17:40:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.731+0000 7fc033a5d640 1 -- 172.21.15.53:0/3338316637 wait complete. 2023-10-21T17:40:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.733+0000 7fc033a5d640 1 Processor -- start 2023-10-21T17:40:10.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.733+0000 7fc033a5d640 1 -- start start 2023-10-21T17:40:10.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180039a0 0x7fc02c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c4640 0x7fc02c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c7b20 con 0x7fc02c14eb10 2023-10-21T17:40:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c53b0 con 0x7fc0180039a0 2023-10-21T17:40:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc033a5d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0c7d00 con 0x7fc02c0c4640 2023-10-21T17:40:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47302/0 (socket says 172.21.15.53:47302) 2023-10-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 -- 172.21.15.53:0/3244741009 learned_addr learned my addr 172.21.15.53:0/3244741009 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180039a0 0x7fc02c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc031ffa640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c4640 0x7fc02c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 -- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c4640 msgr2=0x7fc02c0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c4640 0x7fc02c0c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 -- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180039a0 msgr2=0x7fc02c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.734+0000 7fc030ff8640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180039a0 0x7fc02c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc030ff8640 1 -- 172.21.15.53:0/3244741009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0c7fe0 con 0x7fc02c14eb10 2023-10-21T17:40:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc030ff8640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:10.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c005480 con 0x7fc02c14eb10 2023-10-21T17:40:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c003610 con 0x7fc02c14eb10 2023-10-21T17:40:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003980 con 0x7fc02c14eb10 2023-10-21T17:40:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c0c82c0 con 0x7fc02c14eb10 2023-10-21T17:40:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.735+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c0ab8d0 con 0x7fc02c14eb10 2023-10-21T17:40:10.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.737+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c04e160 con 0x7fc02c14eb10 2023-10-21T17:40:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.738+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fc01c003ae0 con 0x7fc02c14eb10 2023-10-21T17:40:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.738+0000 7fc021ffb640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 0x7fc01006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.739+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fc01c08a0d0 con 0x7fc02c14eb10 2023-10-21T17:40:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.739+0000 7fc0317f9640 1 -- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 msgr2=0x7fc01006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:10.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.739+0000 7fc0317f9640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 0x7fc01006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:10.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.742+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc01c0546f0 con 0x7fc02c14eb10 2023-10-21T17:40:10.816 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.815+0000 7f8c1304c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:10.879 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.878+0000 7f909c60f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:10.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.934+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc02c073780 con 0x7fc02c14eb10 2023-10-21T17:40:10.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.935+0000 7fc021ffb640 1 -- 172.21.15.53:0/3244741009 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v259) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc01c001230 con 0x7fc02c14eb10 2023-10-21T17:40:10.937 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:40:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.937+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 msgr2=0x7fc01006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.937+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 0x7fc01006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.937+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 msgr2=0x7fc02c14ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.937+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000f60 tx=0x7fc01c00d9f0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 shutdown_connections 2023-10-21T17:40:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7fc01006b0b0 0x7fc01006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc02c0c4640 0x7fc02c0c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc02c14eb10 0x7fc02c14ef90 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 --2- 172.21.15.53:0/3244741009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0180039a0 0x7fc02c15f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 >> 172.21.15.53:0/3244741009 conn(0x7fc02c06e630 msgr2=0x7fc02c0cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 shutdown_connections 2023-10-21T17:40:10.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:10.938+0000 7fc033a5d640 1 -- 172.21.15.53:0/3244741009 wait complete. 2023-10-21T17:40:10.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:40:10.987 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:10.987+0000 7f909c60f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:10.989 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:10.988+0000 7f8c1304c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:11.037 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.037+0000 7f909c60f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:11.107 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.105+0000 7f8c1304c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:11.146 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.146+0000 7f909c60f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:11.160 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.159+0000 7f8c1304c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:11.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.168+0000 7f539c1c5640 1 Processor -- start 2023-10-21T17:40:11.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.169+0000 7f539c1c5640 1 -- start start 2023-10-21T17:40:11.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.170+0000 7f539c1c5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f539415cd70 0x7f53941612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.170+0000 7f539c1c5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940b4e10 0x7f53940d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.170+0000 7f539c1c5640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940b8c30 con 0x7f53940cd4e0 2023-10-21T17:40:11.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.170+0000 7f539c1c5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940cdd80 con 0x7f539415cd70 2023-10-21T17:40:11.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.170+0000 7f539c1c5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53940ce060 con 0x7f53940b4e10 2023-10-21T17:40:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f539415cd70 0x7f53941612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f539415cd70 0x7f53941612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47304/0 (socket says 172.21.15.53:47304) 2023-10-21T17:40:11.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 -- 172.21.15.53:0/2776713007 learned_addr learned my addr 172.21.15.53:0/2776713007 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:11.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 -- 172.21.15.53:0/2776713007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940b4e10 msgr2=0x7f53940d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:11.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f539a762640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940b4e10 0x7f53940d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940b4e10 0x7f53940d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 -- 172.21.15.53:0/2776713007 >> v1:172.21.15.124:6789/0 conn(0x7f53940cd4e0 legacy=0x7f53940b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.171+0000 7f5399f61640 1 -- 172.21.15.53:0/2776713007 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940ce340 con 0x7f539415cd70 2023-10-21T17:40:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.172+0000 7f5399f61640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53941612f0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f5390009870 tx=0x7f539002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774d70771cf47f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f5398f5f640 1 -- 172.21.15.53:0/2776713007 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390031640 con 0x7f539415cd70 2023-10-21T17:40:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f5398f5f640 1 -- 172.21.15.53:0/2776713007 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53900317a0 con 0x7f539415cd70 2023-10-21T17:40:11.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f5398f5f640 1 -- 172.21.15.53:0/2776713007 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390031b30 con 0x7f539415cd70 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 -- 172.21.15.53:0/2776713007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 msgr2=0x7f53941612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53941612f0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f5390009870 tx=0x7f539002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 -- 172.21.15.53:0/2776713007 shutdown_connections 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940b4e10 0x7f53940d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2776713007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53941612f0 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 -- 172.21.15.53:0/2776713007 >> 172.21.15.53:0/2776713007 conn(0x7f53940cf700 msgr2=0x7f53940cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 -- 172.21.15.53:0/2776713007 shutdown_connections 2023-10-21T17:40:11.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.173+0000 7f539c1c5640 1 -- 172.21.15.53:0/2776713007 wait complete. 2023-10-21T17:40:11.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.175+0000 7f539c1c5640 1 Processor -- start 2023-10-21T17:40:11.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.175+0000 7f539c1c5640 1 -- start start 2023-10-21T17:40:11.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53940b4e10 0x7f53941e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940cd4e0 0x7f53940cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53941e6750 con 0x7f539415cd70 2023-10-21T17:40:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53941e79c0 con 0x7f53940b4e10 2023-10-21T17:40:11.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539c1c5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53941e6940 con 0x7f53940cd4e0 2023-10-21T17:40:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.176+0000 7f539a762640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47318/0 (socket says 172.21.15.53:47318) 2023-10-21T17:40:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 -- 172.21.15.53:0/2593493234 learned_addr learned my addr 172.21.15.53:0/2593493234 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f5399f61640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53940b4e10 0x7f53941e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f5399760640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940cd4e0 0x7f53940cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 -- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940cd4e0 msgr2=0x7f53940cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940cd4e0 0x7f53940cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 -- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53940b4e10 msgr2=0x7f53941e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53940b4e10 0x7f53941e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 -- 172.21.15.53:0/2593493234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900094d0 con 0x7f539415cd70 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f539a762640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ccb0 tx=0x7f538c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.177+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00ce60 con 0x7f539415cd70 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.178+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c003990 con 0x7f539415cd70 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.178+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c003d20 con 0x7f539415cd70 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.178+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53941e6c20 con 0x7f539415cd70 2023-10-21T17:40:11.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.178+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53941e71e0 con 0x7f539415cd70 2023-10-21T17:40:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.179+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539404e160 con 0x7f539415cd70 2023-10-21T17:40:11.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.181+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f538c003e80 con 0x7f539415cd70 2023-10-21T17:40:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.182+0000 7f538a7fc640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 0x7f537806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.182+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f538c08a5d0 con 0x7f539415cd70 2023-10-21T17:40:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.182+0000 7f5399f61640 1 -- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 msgr2=0x7f537806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:11.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.182+0000 7f5399f61640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 0x7f537806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:11.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.186+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f538c054ab0 con 0x7f539415cd70 2023-10-21T17:40:11.256 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.255+0000 7f909c60f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:11.280 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.279+0000 7f8c1304c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:11.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.379+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53940c6220 con 0x7f539415cd70 2023-10-21T17:40:11.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.380+0000 7f538a7fc640 1 -- 172.21.15.53:0/2593493234 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v259) v1 ==== 107+0+11 (secure 0 0 0) 0x7f538c01d0b0 con 0x7f539415cd70 2023-10-21T17:40:11.381 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:40:11.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.382+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 msgr2=0x7f537806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:11.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.382+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 0x7f537806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.382+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 msgr2=0x7f53940cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.382+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ccb0 tx=0x7f538c00a5c0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 shutdown_connections 2023-10-21T17:40:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f537806b0b0 0x7f537806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f539415cd70 0x7f53940cc2a0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f53940cd4e0 0x7f53940cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 --2- 172.21.15.53:0/2593493234 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f53940b4e10 0x7f53941e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 >> 172.21.15.53:0/2593493234 conn(0x7f53940cf700 msgr2=0x7f53940d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 shutdown_connections 2023-10-21T17:40:11.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.383+0000 7f539c1c5640 1 -- 172.21.15.53:0/2593493234 wait complete. 2023-10-21T17:40:11.394 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.393+0000 7f8c1304c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:11.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:40:11.413 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.412+0000 7f909c60f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:11.466 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.466+0000 7f909c60f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:11.564 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.562+0000 7f8c1304c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:11.587 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:11.586+0000 7f909c60f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:11.616 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.615+0000 7f8c1304c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:11.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.654+0000 7f6dbcd68640 1 Processor -- start 2023-10-21T17:40:11.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.654+0000 7f6dbcd68640 1 -- start start 2023-10-21T17:40:11.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6dbcd68640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6db8071f10 0x7f6db815d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6dbcd68640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80cd860 con 0x7f6db80cd400 2023-10-21T17:40:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6dbcd68640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db806fc60 con 0x7f6db8159a40 2023-10-21T17:40:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6dbcd68640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db806ee00 con 0x7f6db8071f10 2023-10-21T17:40:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6daffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6db80cd400 0x7f6db8071800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45396/0 (socket says 172.21.15.53:45396) 2023-10-21T17:40:11.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.656+0000 7f6daffff640 1 -- 172.21.15.53:0/3114199303 learned_addr learned my addr 172.21.15.53:0/3114199303 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:11.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db6d9d640 1 --2- 172.21.15.53:0/3114199303 >> v2:172.21.15.124:3300/0 conn(0x7f6db8071f10 0x7f6db815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db5d9b640 1 -- 172.21.15.53:0/3114199303 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190086224 0 0) 0x7f6db80cd860 con 0x7f6db80cd400 2023-10-21T17:40:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db5d9b640 1 -- 172.21.15.53:0/3114199303 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da4003520 con 0x7f6db80cd400 2023-10-21T17:40:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db6d9d640 1 -- 172.21.15.53:0/3114199303 >> v1:172.21.15.124:6789/0 conn(0x7f6db8159a40 legacy=0x7f6db80b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db6d9d640 1 -- 172.21.15.53:0/3114199303 >> v1:172.21.15.53:6789/0 conn(0x7f6db80cd400 legacy=0x7f6db8071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.657+0000 7f6db6d9d640 1 -- 172.21.15.53:0/3114199303 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db806f0e0 con 0x7f6db8071f10 2023-10-21T17:40:11.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.658+0000 7f6db6d9d640 1 --2- 172.21.15.53:0/3114199303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db815d870 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009d90 tx=0x7f6db002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4939f829d4f9fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.659+0000 7f6db5d9b640 1 -- 172.21.15.53:0/3114199303 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db002cd00 con 0x7f6db8071f10 2023-10-21T17:40:11.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.659+0000 7f6db5d9b640 1 -- 172.21.15.53:0/3114199303 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db002ce60 con 0x7f6db8071f10 2023-10-21T17:40:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.659+0000 7f6db5d9b640 1 -- 172.21.15.53:0/3114199303 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0034640 con 0x7f6db8071f10 2023-10-21T17:40:11.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 -- 172.21.15.53:0/3114199303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 msgr2=0x7f6db815d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/3114199303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db815d870 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009d90 tx=0x7f6db002bf50 comp rx=0 tx=0).stop 2023-10-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 -- 172.21.15.53:0/3114199303 shutdown_connections 2023-10-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/3114199303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db815d870 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 -- 172.21.15.53:0/3114199303 >> 172.21.15.53:0/3114199303 conn(0x7f6db80cf8c0 msgr2=0x7f6db80cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 -- 172.21.15.53:0/3114199303 shutdown_connections 2023-10-21T17:40:11.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.660+0000 7f6dbcd68640 1 -- 172.21.15.53:0/3114199303 wait complete. 2023-10-21T17:40:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.661+0000 7f6dbcd68640 1 Processor -- start 2023-10-21T17:40:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.662+0000 7f6dbcd68640 1 -- start start 2023-10-21T17:40:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db80c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6db80cd400 0x7f6db80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c7980 con 0x7f6db8159a40 2023-10-21T17:40:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c6e40 con 0x7f6db8071f10 2023-10-21T17:40:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6dbcd68640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c70e0 con 0x7f6db80cd400 2023-10-21T17:40:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47330/0 (socket says 172.21.15.53:47330) 2023-10-21T17:40:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 -- 172.21.15.53:0/834905301 learned_addr learned my addr 172.21.15.53:0/834905301 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db659c640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db80c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6daffff640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6db80cd400 0x7f6db80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 -- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6db80cd400 msgr2=0x7f6db80c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.669 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6db80cd400 0x7f6db80c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 -- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 msgr2=0x7f6db80c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db80c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.663+0000 7f6db6d9d640 1 -- 172.21.15.53:0/834905301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009a70 con 0x7f6db8159a40 2023-10-21T17:40:11.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6db6d9d640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f6db002c560 tx=0x7f6db0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:11.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004310 con 0x7f6db8159a40 2023-10-21T17:40:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db0004470 con 0x7f6db8159a40 2023-10-21T17:40:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004800 con 0x7f6db8159a40 2023-10-21T17:40:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db80c73c0 con 0x7f6db8159a40 2023-10-21T17:40:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.664+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db8149b40 con 0x7f6db8159a40 2023-10-21T17:40:11.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.665+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db804e160 con 0x7f6db8159a40 2023-10-21T17:40:11.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.666+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f6db003b070 con 0x7f6db8159a40 2023-10-21T17:40:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.667+0000 7f6daeffd640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 0x7f6d9806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.667+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f6db00acc00 con 0x7f6db8159a40 2023-10-21T17:40:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.667+0000 7f6db659c640 1 -- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 msgr2=0x7f6d9806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.667+0000 7f6db659c640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 0x7f6d9806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:11.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.671+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6db00773a0 con 0x7f6db8159a40 2023-10-21T17:40:11.744 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:11.743+0000 7f8c1304c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:11.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.864+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f6db80715e0 con 0x7f6db8159a40 2023-10-21T17:40:11.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.865+0000 7f6daeffd640 1 -- 172.21.15.53:0/834905301 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v259) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6db0042020 con 0x7f6db8159a40 2023-10-21T17:40:11.866 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:40:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 msgr2=0x7f6d9806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 0x7f6d9806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 msgr2=0x7f6db80b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f6db002c560 tx=0x7f6db0004a40 comp rx=0 tx=0).stop 2023-10-21T17:40:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 shutdown_connections 2023-10-21T17:40:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f6d9806b0b0 0x7f6d9806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8159a40 0x7f6db80b09c0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6db80cd400 0x7f6db80c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 --2- 172.21.15.53:0/834905301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6db8071f10 0x7f6db80c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 >> 172.21.15.53:0/834905301 conn(0x7f6db80cf8c0 msgr2=0x7f6db80ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.867+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 shutdown_connections 2023-10-21T17:40:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:11.868+0000 7f6dbcd68640 1 -- 172.21.15.53:0/834905301 wait complete. 2023-10-21T17:40:11.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:40:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.100+0000 7f9723494640 1 Processor -- start 2023-10-21T17:40:12.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.100+0000 7f9723494640 1 -- start start 2023-10-21T17:40:12.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.101+0000 7f9723494640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 0x7f971c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.101+0000 7f9723494640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f971c0cd4e0 0x7f971c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.101+0000 7f9723494640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c0b8c30 con 0x7f971c15cd70 2023-10-21T17:40:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.101+0000 7f9723494640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c0cdd80 con 0x7f971c0cd4e0 2023-10-21T17:40:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.101+0000 7f9723494640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0ce060 con 0x7f971c0b4e10 2023-10-21T17:40:12.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9721230640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f971c15cd70 0x7f971c0b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45398/0 (socket says 172.21.15.53:45398) 2023-10-21T17:40:12.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9721230640 1 -- 172.21.15.53:0/2517860417 learned_addr learned my addr 172.21.15.53:0/2517860417 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:12.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9720a2f640 1 --2- 172.21.15.53:0/2517860417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 0x7f971c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9721a31640 1 --2- 172.21.15.53:0/2517860417 >> v2:172.21.15.124:3300/0 conn(0x7f971c0cd4e0 0x7f971c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9713fff640 1 -- 172.21.15.53:0/2517860417 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525795042 0 0) 0x7f971c0b8c30 con 0x7f971c15cd70 2023-10-21T17:40:12.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9713fff640 1 -- 172.21.15.53:0/2517860417 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9708003520 con 0x7f971c15cd70 2023-10-21T17:40:12.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9720a2f640 1 -- 172.21.15.53:0/2517860417 >> v2:172.21.15.124:3300/0 conn(0x7f971c0cd4e0 msgr2=0x7f971c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:12.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9720a2f640 1 --2- 172.21.15.53:0/2517860417 >> v2:172.21.15.124:3300/0 conn(0x7f971c0cd4e0 0x7f971c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9720a2f640 1 -- 172.21.15.53:0/2517860417 >> v1:172.21.15.53:6789/0 conn(0x7f971c15cd70 legacy=0x7f971c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.102+0000 7f9720a2f640 1 -- 172.21.15.53:0/2517860417 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0ce340 con 0x7f971c0b4e10 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.103+0000 7f9720a2f640 1 --2- 172.21.15.53:0/2517860417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 0x7f971c1612f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f970c0097a0 tx=0x7f970c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c8785b598c0e71 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9713fff640 1 -- 172.21.15.53:0/2517860417 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c02d040 con 0x7f971c0b4e10 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9713fff640 1 -- 172.21.15.53:0/2517860417 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c031810 con 0x7f971c0b4e10 2023-10-21T17:40:12.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9713fff640 1 -- 172.21.15.53:0/2517860417 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c031b80 con 0x7f971c0b4e10 2023-10-21T17:40:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 -- 172.21.15.53:0/2517860417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 msgr2=0x7f971c1612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 --2- 172.21.15.53:0/2517860417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 0x7f971c1612f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f970c0097a0 tx=0x7f970c02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 -- 172.21.15.53:0/2517860417 shutdown_connections 2023-10-21T17:40:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 --2- 172.21.15.53:0/2517860417 >> v2:172.21.15.124:3300/0 conn(0x7f971c0cd4e0 0x7f971c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 --2- 172.21.15.53:0/2517860417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0b4e10 0x7f971c1612f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.104+0000 7f9723494640 1 -- 172.21.15.53:0/2517860417 >> 172.21.15.53:0/2517860417 conn(0x7f971c0cf700 msgr2=0x7f971c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.105+0000 7f9723494640 1 -- 172.21.15.53:0/2517860417 shutdown_connections 2023-10-21T17:40:12.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.105+0000 7f9723494640 1 -- 172.21.15.53:0/2517860417 wait complete. 2023-10-21T17:40:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.106+0000 7f9723494640 1 Processor -- start 2023-10-21T17:40:12.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.106+0000 7f9723494640 1 -- start start 2023-10-21T17:40:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f971c0b4e10 0x7f971c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 0x7f971c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0cc490 con 0x7f971c15cd70 2023-10-21T17:40:12.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0cc770 con 0x7f971c0b4e10 2023-10-21T17:40:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.107+0000 7f9723494640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0cca50 con 0x7f971c0cd4e0 2023-10-21T17:40:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47342/0 (socket says 172.21.15.53:47342) 2023-10-21T17:40:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 -- 172.21.15.53:0/1159041081 learned_addr learned my addr 172.21.15.53:0/1159041081 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:12.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9720a2f640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 0x7f971c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721230640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f971c0b4e10 0x7f971c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 msgr2=0x7f971c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 0x7f971c0cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f971c0b4e10 msgr2=0x7f971c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f971c0b4e10 0x7f971c157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 -- 172.21.15.53:0/1159041081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009400 con 0x7f971c15cd70 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9720a2f640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 0x7f971c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.108+0000 7f9721a31640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f971400e860 tx=0x7f971400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.109+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400c6d0 con 0x7f971c15cd70 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.109+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971400c830 con 0x7f971c15cd70 2023-10-21T17:40:12.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.109+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c0cada0 con 0x7f971c15cd70 2023-10-21T17:40:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.109+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400cbe0 con 0x7f971c15cd70 2023-10-21T17:40:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.109+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c158570 con 0x7f971c15cd70 2023-10-21T17:40:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.110+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c0cb2c0 con 0x7f971c15cd70 2023-10-21T17:40:12.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.111+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 140) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f9714013470 con 0x7f971c15cd70 2023-10-21T17:40:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.112+0000 7f9711ffb640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 0x7f970006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.113+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f971408a470 con 0x7f971c15cd70 2023-10-21T17:40:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.113+0000 7f9721230640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 msgr2=0x7f970006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.113+0000 7f9721230640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 0x7f970006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:12.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.117+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f971c0cb2c0 con 0x7f971c15cd70 2023-10-21T17:40:12.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.314+0000 7f9721230640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 msgr2=0x7f970006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/2228122799 2023-10-21T17:40:12.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.314+0000 7f9721230640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 0x7f970006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.348+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f971c0cb2c0 con 0x7f971c15cd70 2023-10-21T17:40:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.349+0000 7f9711ffb640 1 -- 172.21.15.53:0/1159041081 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v259) v1 ==== 81+0+593 (secure 0 0 0) 0x7f971c0cb2c0 con 0x7f971c15cd70 2023-10-21T17:40:12.350 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3074714843 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.350 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/3816427222 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.351 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/920825767 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.351 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3189508227 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.351 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/959805027 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.351 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1957744482 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.351 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/844552797 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.352 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2190223666 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.352 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/3816427222 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.352 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2179186961 2023-10-22T17:40:06.671249+0000 2023-10-21T17:40:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:listed 10 entries 2023-10-21T17:40:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 msgr2=0x7f970006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 0x7f970006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 msgr2=0x7f971c0b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f971400e860 tx=0x7f971400ed20 comp rx=0 tx=0).stop 2023-10-21T17:40:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 shutdown_connections 2023-10-21T17:40:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:6832/2228122799,v1:172.21.15.124:6833/2228122799] conn(0x7f970006b0b0 0x7f970006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f971c15cd70 0x7f971c0b6b50 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f971c0cd4e0 0x7f971c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 --2- 172.21.15.53:0/1159041081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f971c0b4e10 0x7f971c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 >> 172.21.15.53:0/1159041081 conn(0x7f971c0cf700 msgr2=0x7f971c0d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 shutdown_connections 2023-10-21T17:40:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.351+0000 7f9723494640 1 -- 172.21.15.53:0/1159041081 wait complete. 2023-10-21T17:40:12.369 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:40:12.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:12.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:40:12.420 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.419+0000 7f909c60f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:12.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:40:12.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:12.464 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:40:12.514 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:40:12.514 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:12.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:40:12.570 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:12.570 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:12.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:40:12.626 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:40:12.626 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:12.630 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:12.629+0000 7f8c1304c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:12.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:40:12.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:12.725 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.724+0000 7f909c60f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:12.784 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.783+0000 7f909c60f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:12.896 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.895+0000 7f909c60f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:12.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.899+0000 7f8426690640 1 Processor -- start 2023-10-21T17:40:12.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.900+0000 7f8426690640 1 -- start start 2023-10-21T17:40:12.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.901+0000 7f8426690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f84200cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.901+0000 7f8426690640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200b8bf0 con 0x7f84200b1840 2023-10-21T17:40:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.901+0000 7f8426690640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200d39b0 con 0x7f84200b1d90 2023-10-21T17:40:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.901+0000 7f8426690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200d2910 con 0x7f84200cd8d0 2023-10-21T17:40:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f84177fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f84200b1840 0x7f84200d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:45410/0 (socket says 172.21.15.53:45410) 2023-10-21T17:40:12.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f84177fe640 1 -- 172.21.15.53:0/146737509 learned_addr learned my addr 172.21.15.53:0/146737509 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:12.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841ffff640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f84200cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349818325 0 0) 0x7f84200b8bf0 con 0x7f84200b1840 2023-10-21T17:40:12.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840c003520 con 0x7f84200b1840 2023-10-21T17:40:12.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1994146597 0 0) 0x7f840c003520 con 0x7f84200b1840 2023-10-21T17:40:12.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 msgr2=0x7f84200cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:12.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f84200cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 >> v1:172.21.15.124:6789/0 conn(0x7f84200b1d90 legacy=0x7f84200d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200ce1f0 con 0x7f84200b1840 2023-10-21T17:40:12.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f8408001670 con 0x7f84200b1840 2023-10-21T17:40:12.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.902+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 >> v1:172.21.15.53:6789/0 conn(0x7f84200b1840 legacy=0x7f84200d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c0039a0 0x7f840c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f840c0047c0 0x7f840c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f840c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200d2910 con 0x7f840c0039a0 2023-10-21T17:40:12.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00ba30 con 0x7f840c0047c0 2023-10-21T17:40:12.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c00bbf0 con 0x7f84200cd8d0 2023-10-21T17:40:12.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c0039a0 0x7f840c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f8424c2d640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f840c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f84177fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f840c0047c0 0x7f840c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 -- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 msgr2=0x7f840c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f84200cd8d0 0x7f840c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 -- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f840c0047c0 msgr2=0x7f840c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f840c0047c0 0x7f840c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f841ffff640 1 -- 172.21.15.53:0/146737509 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c003520 con 0x7f840c0039a0 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.903+0000 7f84177fe640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f840c0047c0 0x7f840c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:12.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.905+0000 7f841ffff640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c0039a0 0x7f840c003dc0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f8410009f90 tx=0x7f840c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1deb8b9a31dbb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.909+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841002cbd0 con 0x7f840c0039a0 2023-10-21T17:40:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.909+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841000f040 con 0x7f840c0039a0 2023-10-21T17:40:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.909+0000 7f841f7fe640 1 -- 172.21.15.53:0/146737509 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410034440 con 0x7f840c0039a0 2023-10-21T17:40:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 -- 172.21.15.53:0/146737509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c0039a0 msgr2=0x7f840c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 --2- 172.21.15.53:0/146737509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f840c0039a0 0x7f840c003dc0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f8410009f90 tx=0x7f840c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8424c2d640 1 -- 172.21.15.53:0/146737509 reap_dead start 2023-10-21T17:40:12.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 -- 172.21.15.53:0/146737509 shutdown_connections 2023-10-21T17:40:12.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 -- 172.21.15.53:0/146737509 >> 172.21.15.53:0/146737509 conn(0x7f842006b6a0 msgr2=0x7f842006bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:12.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 -- 172.21.15.53:0/146737509 shutdown_connections 2023-10-21T17:40:12.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.910+0000 7f8426690640 1 -- 172.21.15.53:0/146737509 wait complete. 2023-10-21T17:40:12.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.913+0000 7f8426690640 1 Processor -- start 2023-10-21T17:40:12.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.914+0000 7f8426690640 1 -- start start 2023-10-21T17:40:12.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f840c0039a0 0x7f84201f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84200cb800 0x7f8420156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:12.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200cbc80 con 0x7f84201588a0 2023-10-21T17:40:12.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200c6890 con 0x7f84200cb800 2023-10-21T17:40:12.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.915+0000 7f8426690640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200c9580 con 0x7f840c0039a0 2023-10-21T17:40:12.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.916+0000 7f84177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.916+0000 7f84177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47374/0 (socket says 172.21.15.53:47374) 2023-10-21T17:40:12.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.916+0000 7f84177fe640 1 -- 172.21.15.53:0/2169895371 learned_addr learned my addr 172.21.15.53:0/2169895371 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:12.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f841ffff640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f840c0039a0 0x7f84201f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:12.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 -- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f840c0039a0 msgr2=0x7f84201f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f840c0039a0 0x7f84201f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 -- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84200cb800 msgr2=0x7f8420156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:12.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84200cb800 0x7f8420156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:12.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 -- 172.21.15.53:0/2169895371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420156180 con 0x7f84201588a0 2023-10-21T17:40:12.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.917+0000 7f84177fe640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f840800db40 tx=0x7f8408007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:12.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.918+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080075a0 con 0x7f84201588a0 2023-10-21T17:40:12.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.918+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408007700 con 0x7f84201588a0 2023-10-21T17:40:12.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.918+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080049e0 con 0x7f84201588a0 2023-10-21T17:40:12.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.918+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8420156460 con 0x7f84201588a0 2023-10-21T17:40:12.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.918+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8420156a20 con 0x7f84201588a0 2023-10-21T17:40:12.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.920+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8420066d80 con 0x7f84201588a0 2023-10-21T17:40:12.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.921+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 141) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8408004b40 con 0x7f84201588a0 2023-10-21T17:40:12.923 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.922+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f8408089360 con 0x7f84201588a0 2023-10-21T17:40:12.928 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:12.927+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8408053f00 con 0x7f84201588a0 2023-10-21T17:40:12.958 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:12.954+0000 7f8c1304c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:12.989 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.989 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.989 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.989 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.990 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:12.991 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:12.988+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.002 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.002+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.002 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.002+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.003 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.002+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.003 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.002+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.003 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.002+0000 7f901e4c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.004 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.003+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.004 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.003+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.004 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.003+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.005 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.003+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.005 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:13.003+0000 7f901fcc6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:13.021 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:13.020+0000 7f8c1304c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:13.142 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:13.141+0000 7f8c1304c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:13.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.162+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84200b7c30 con 0x7f84201588a0 2023-10-21T17:40:13.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.163+0000 7f841d7fa640 1 -- 172.21.15.53:0/2169895371 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7f8408019070 con 0x7f84201588a0 2023-10-21T17:40:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:13.165 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:13.166 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:40:13.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 msgr2=0x7f84200cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f840800db40 tx=0x7f8408007e10 comp rx=0 tx=0).stop 2023-10-21T17:40:13.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 shutdown_connections 2023-10-21T17:40:13.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f84200cb800 0x7f8420156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84201588a0 0x7f84200cb2c0 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 --2- 172.21.15.53:0/2169895371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f840c0039a0 0x7f84201f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 >> 172.21.15.53:0/2169895371 conn(0x7f842006b6a0 msgr2=0x7f84200b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:13.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 shutdown_connections 2023-10-21T17:40:13.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.166+0000 7f8426690640 1 -- 172.21.15.53:0/2169895371 wait complete. 2023-10-21T17:40:13.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:13.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.403+0000 7f7ef096b640 1 Processor -- start 2023-10-21T17:40:13.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.403+0000 7f7ef096b640 1 -- start start 2023-10-21T17:40:13.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.405+0000 7f7ef096b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eec0d1640 0x7f7eec0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.405+0000 7f7ef096b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7eec0d07c0 0x7f7eec0b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.405+0000 7f7ef096b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec0ceb00 con 0x7f7eec15cdf0 2023-10-21T17:40:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.405+0000 7f7ef096b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec0ce610 con 0x7f7eec0d1640 2023-10-21T17:40:13.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.405+0000 7f7ef096b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec15d1d0 con 0x7f7eec0d07c0 2023-10-21T17:40:13.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eec0d1640 0x7f7eec0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7eec0d1640 0x7f7eec0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47386/0 (socket says 172.21.15.53:47386) 2023-10-21T17:40:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 -- 172.21.15.53:0/2595000587 learned_addr learned my addr 172.21.15.53:0/2595000587 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eeb59e640 1 --2- 172.21.15.53:0/2595000587 >> v2:172.21.15.124:3300/0 conn(0x7f7eec0d07c0 0x7f7eec0b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 -- 172.21.15.53:0/2595000587 >> v2:172.21.15.124:3300/0 conn(0x7f7eec0d07c0 msgr2=0x7f7eec0b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 --2- 172.21.15.53:0/2595000587 >> v2:172.21.15.124:3300/0 conn(0x7f7eec0d07c0 0x7f7eec0b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 -- 172.21.15.53:0/2595000587 >> v1:172.21.15.124:6789/0 conn(0x7f7eec15cdf0 legacy=0x7f7eec0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.406+0000 7f7eea59c640 1 -- 172.21.15.53:0/2595000587 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0ce820 con 0x7f7eec0d1640 2023-10-21T17:40:13.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.407+0000 7f7eea59c640 1 --2- 172.21.15.53:0/2595000587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec0d1640 0x7f7eec0ce0d0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f7eec08c080 tx=0x7f7ee002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8cec09c986bc5f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ee9d9b640 1 -- 172.21.15.53:0/2595000587 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0031750 con 0x7f7eec0d1640 2023-10-21T17:40:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ee9d9b640 1 -- 172.21.15.53:0/2595000587 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee00318b0 con 0x7f7eec0d1640 2023-10-21T17:40:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ee9d9b640 1 -- 172.21.15.53:0/2595000587 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0031c40 con 0x7f7eec0d1640 2023-10-21T17:40:13.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 -- 172.21.15.53:0/2595000587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec0d1640 msgr2=0x7f7eec0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 --2- 172.21.15.53:0/2595000587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec0d1640 0x7f7eec0ce0d0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f7eec08c080 tx=0x7f7ee002b7f0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 -- 172.21.15.53:0/2595000587 shutdown_connections 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 --2- 172.21.15.53:0/2595000587 >> v2:172.21.15.124:3300/0 conn(0x7f7eec0d07c0 0x7f7eec0b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 --2- 172.21.15.53:0/2595000587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec0d1640 0x7f7eec0ce0d0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.408+0000 7f7ef096b640 1 -- 172.21.15.53:0/2595000587 >> 172.21.15.53:0/2595000587 conn(0x7f7eec09bcb0 msgr2=0x7f7eec0d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:13.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.409+0000 7f7ef096b640 1 -- 172.21.15.53:0/2595000587 shutdown_connections 2023-10-21T17:40:13.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.409+0000 7f7ef096b640 1 -- 172.21.15.53:0/2595000587 wait complete. 2023-10-21T17:40:13.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.410+0000 7f7ef096b640 1 Processor -- start 2023-10-21T17:40:13.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.410+0000 7f7ef096b640 1 -- start start 2023-10-21T17:40:13.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 0x7f7eec1e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 0x7f7eec1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec1e5ad0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec1584b0 con 0x7f7eec0d1640 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.411+0000 7f7ef096b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec158790 con 0x7f7eec0d07c0 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47390/0 (socket says 172.21.15.53:47390) 2023-10-21T17:40:13.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 -- 172.21.15.53:0/1776911566 learned_addr learned my addr 172.21.15.53:0/1776911566 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eea59c640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 0x7f7eec1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eead9d640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 0x7f7eec1e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 -- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 msgr2=0x7f7eec1e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 0x7f7eec1e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 -- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 msgr2=0x7f7eec1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 0x7f7eec1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eeb59e640 1 -- 172.21.15.53:0/1776911566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00094d0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eea59c640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 0x7f7eec1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.412+0000 7f7eead9d640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 0x7f7eec1e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:13.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7eeb59e640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4007e60 tx=0x7f7ee400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4014420 con 0x7f7eec15cdf0 2023-10-21T17:40:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee40145a0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4014950 con 0x7f7eec15cdf0 2023-10-21T17:40:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec158a70 con 0x7f7eec15cdf0 2023-10-21T17:40:13.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.413+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec1e57e0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.414+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec158f90 con 0x7f7eec15cdf0 2023-10-21T17:40:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.415+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 141) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f7ee4014ab0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.416+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f7ee408a270 con 0x7f7eec15cdf0 2023-10-21T17:40:13.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.421+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ee4054f00 con 0x7f7eec15cdf0 2023-10-21T17:40:13.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.610+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eec0d35c0 con 0x7f7eec15cdf0 2023-10-21T17:40:13.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.614+0000 7f7ed37fe640 1 -- 172.21.15.53:0/1776911566 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21885 (secure 0 0 0) 0x7f7ee4058520 con 0x7f7eec15cdf0 2023-10-21T17:40:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:13.616 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":260,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:12.899743+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:13.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.616+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 msgr2=0x7f7eec0cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.616+0000 7f7ef096b640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4007e60 tx=0x7f7ee400e4b0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 shutdown_connections 2023-10-21T17:40:13.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eec15cdf0 0x7f7eec0cb010 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7eec0d1640 0x7f7eec1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 --2- 172.21.15.53:0/1776911566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7eec0d07c0 0x7f7eec1e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 >> 172.21.15.53:0/1776911566 conn(0x7f7eec09bcb0 msgr2=0x7f7eec15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:13.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 shutdown_connections 2023-10-21T17:40:13.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.617+0000 7f7ef096b640 1 -- 172.21.15.53:0/1776911566 wait complete. 2023-10-21T17:40:13.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:39:37.842433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:39:38.736221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:39:52.278617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:39:52.975528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:40:13.638 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:40:13.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.857+0000 7fbf94aca640 1 Processor -- start 2023-10-21T17:40:13.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.857+0000 7fbf94aca640 1 -- start start 2023-10-21T17:40:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf94aca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf90070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf94aca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf900cb350 0x7fbf900b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf94aca640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9015ae90 con 0x7fbf900caef0 2023-10-21T17:40:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf94aca640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf900cbdb0 con 0x7fbf900cb350 2023-10-21T17:40:13.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf94aca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900cb7f0 con 0x7fbf9015aab0 2023-10-21T17:40:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf8ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf900cb350 0x7fbf900b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf8ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf900cb350 0x7fbf900b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47394/0 (socket says 172.21.15.53:47394) 2023-10-21T17:40:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.859+0000 7fbf8ffff640 1 -- 172.21.15.53:0/77167586 learned_addr learned my addr 172.21.15.53:0/77167586 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:13.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.860+0000 7fbf8f7fe640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf90070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.860+0000 7fbf8ffff640 1 -- 172.21.15.53:0/77167586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 msgr2=0x7fbf90070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.860+0000 7fbf8ffff640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf90070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.860+0000 7fbf8ffff640 1 -- 172.21.15.53:0/77167586 >> v1:172.21.15.124:6789/0 conn(0x7fbf900caef0 legacy=0x7fbf900aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.860+0000 7fbf8ffff640 1 -- 172.21.15.53:0/77167586 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900cbad0 con 0x7fbf900cb350 2023-10-21T17:40:13.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.861+0000 7fbf8ffff640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900b1c60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009e50 tx=0x7fbf8802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eec88535a4b7b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:13.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.861+0000 7fbf8e7fc640 1 -- 172.21.15.53:0/77167586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8802c9f0 con 0x7fbf900cb350 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.861+0000 7fbf8e7fc640 1 -- 172.21.15.53:0/77167586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8802cb50 con 0x7fbf900cb350 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.861+0000 7fbf8e7fc640 1 -- 172.21.15.53:0/77167586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880343f0 con 0x7fbf900cb350 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 -- 172.21.15.53:0/77167586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 msgr2=0x7fbf900b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900b1c60 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009e50 tx=0x7fbf8802be50 comp rx=0 tx=0).stop 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 -- 172.21.15.53:0/77167586 shutdown_connections 2023-10-21T17:40:13.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900b1c60 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 --2- 172.21.15.53:0/77167586 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf90070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 -- 172.21.15.53:0/77167586 >> 172.21.15.53:0/77167586 conn(0x7fbf9006c010 msgr2=0x7fbf9006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 -- 172.21.15.53:0/77167586 shutdown_connections 2023-10-21T17:40:13.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.862+0000 7fbf94aca640 1 -- 172.21.15.53:0/77167586 wait complete. 2023-10-21T17:40:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.864+0000 7fbf94aca640 1 Processor -- start 2023-10-21T17:40:13.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.864+0000 7fbf94aca640 1 -- start start 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf900caef0 0x7fbf9014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf900cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900c21d0 con 0x7fbf900cb350 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90159160 con 0x7fbf900caef0 2023-10-21T17:40:13.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf94aca640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901e51c0 con 0x7fbf9015aab0 2023-10-21T17:40:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf8effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf8effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47398/0 (socket says 172.21.15.53:47398) 2023-10-21T17:40:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.865+0000 7fbf8effd640 1 -- 172.21.15.53:0/367933294 learned_addr learned my addr 172.21.15.53:0/367933294 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8f7fe640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf900caef0 0x7fbf9014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8ffff640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf900cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 -- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 msgr2=0x7fbf900cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf900cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 -- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf900caef0 msgr2=0x7fbf9014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf900caef0 0x7fbf9014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 -- 172.21.15.53:0/367933294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88009b30 con 0x7fbf900cb350 2023-10-21T17:40:13.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf8effd640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400b3f0 tx=0x7fbf8400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84011830 con 0x7fbf900cb350 2023-10-21T17:40:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf84011990 con 0x7fbf900cb350 2023-10-21T17:40:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf90158ba0 con 0x7fbf900cb350 2023-10-21T17:40:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.866+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84011d40 con 0x7fbf900cb350 2023-10-21T17:40:13.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.867+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900c2660 con 0x7fbf900cb350 2023-10-21T17:40:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.868+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf900c28e0 con 0x7fbf900cb350 2023-10-21T17:40:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.869+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 141) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbf8401a400 con 0x7fbf900cb350 2023-10-21T17:40:13.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.870+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fbf840898a0 con 0x7fbf900cb350 2023-10-21T17:40:13.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.874+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf84054440 con 0x7fbf900cb350 2023-10-21T17:40:13.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.919+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 142) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbf84053ef0 con 0x7fbf900cb350 2023-10-21T17:40:13.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.920+0000 7fbf77fff640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 0x7fbf7c0758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:13.922 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.921+0000 7fbf8f7fe640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 0x7fbf7c0758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:13.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:13.934+0000 7fbf8f7fe640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 0x7fbf7c0758a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf80006fd0 tx=0x7fbf80008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.079+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf90074150 con 0x7fbf900cb350 2023-10-21T17:40:14.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.079+0000 7fbf77fff640 1 -- 172.21.15.53:0/367933294 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v260) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf84071fb0 con 0x7fbf900cb350 2023-10-21T17:40:14.081 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:40:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 msgr2=0x7fbf7c0758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 0x7fbf7c0758a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf80006fd0 tx=0x7fbf80008040 comp rx=0 tx=0).stop 2023-10-21T17:40:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 msgr2=0x7fbf900c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400b3f0 tx=0x7fbf8400b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 shutdown_connections 2023-10-21T17:40:14.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf7c0734b0 0x7fbf7c0758a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf9015aab0 0x7fbf900cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf900cb350 0x7fbf900c4800 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 --2- 172.21.15.53:0/367933294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf900caef0 0x7fbf9014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.082+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 >> 172.21.15.53:0/367933294 conn(0x7fbf9006c010 msgr2=0x7fbf901593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:14.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.083+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 shutdown_connections 2023-10-21T17:40:14.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.083+0000 7fbf94aca640 1 -- 172.21.15.53:0/367933294 wait complete. 2023-10-21T17:40:14.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:40:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.320+0000 7f756c9a8640 1 Processor -- start 2023-10-21T17:40:14.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.320+0000 7f756c9a8640 1 -- start start 2023-10-21T17:40:14.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.321+0000 7f756c9a8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f75680cf690 0x7f75680d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.321+0000 7f756c9a8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75680cd520 con 0x7f75680cf2b0 2023-10-21T17:40:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.321+0000 7f756c9a8640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75680d1910 con 0x7f75680ce510 2023-10-21T17:40:14.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.321+0000 7f756c9a8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75680ce080 con 0x7f75680cf690 2023-10-21T17:40:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.321+0000 7f756659c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f75680cf690 0x7f75680d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f7566d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f75680cf2b0 0x7f75680d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36584/0 (socket says 172.21.15.53:36584) 2023-10-21T17:40:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f7566d9d640 1 -- 172.21.15.53:0/3971150801 learned_addr learned my addr 172.21.15.53:0/3971150801 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:14.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f756659c640 1 -- 172.21.15.53:0/3971150801 >> v1:172.21.15.124:6789/0 conn(0x7f75680ce510 legacy=0x7f756815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:14.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f756659c640 1 -- 172.21.15.53:0/3971150801 >> v1:172.21.15.53:6789/0 conn(0x7f75680cf2b0 legacy=0x7f75680d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f7565d9b640 1 -- 172.21.15.53:0/3971150801 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812120289 0 0) 0x7f75680cd520 con 0x7f75680cf2b0 2023-10-21T17:40:14.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.322+0000 7f756659c640 1 -- 172.21.15.53:0/3971150801 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680cd750 con 0x7f75680cf690 2023-10-21T17:40:14.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.324+0000 7f756659c640 1 --2- 172.21.15.53:0/3971150801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f75680d34d0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad483d790e596ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.324+0000 7f7565d9b640 1 -- 172.21.15.53:0/3971150801 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755002cbd0 con 0x7f75680cf690 2023-10-21T17:40:14.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.324+0000 7f7565d9b640 1 -- 172.21.15.53:0/3971150801 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755002cd30 con 0x7f75680cf690 2023-10-21T17:40:14.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.324+0000 7f7565d9b640 1 -- 172.21.15.53:0/3971150801 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755002cea0 con 0x7f75680cf690 2023-10-21T17:40:14.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 -- 172.21.15.53:0/3971150801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 msgr2=0x7f75680d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 --2- 172.21.15.53:0/3971150801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f75680d34d0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f7550009d90 tx=0x7f755002be20 comp rx=0 tx=0).stop 2023-10-21T17:40:14.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 -- 172.21.15.53:0/3971150801 shutdown_connections 2023-10-21T17:40:14.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 --2- 172.21.15.53:0/3971150801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f75680d34d0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 -- 172.21.15.53:0/3971150801 >> 172.21.15.53:0/3971150801 conn(0x7f756809bc40 msgr2=0x7f75680d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:14.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 -- 172.21.15.53:0/3971150801 shutdown_connections 2023-10-21T17:40:14.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.325+0000 7f756c9a8640 1 -- 172.21.15.53:0/3971150801 wait complete. 2023-10-21T17:40:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.327+0000 7f756c9a8640 1 Processor -- start 2023-10-21T17:40:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.327+0000 7f756c9a8640 1 -- start start 2023-10-21T17:40:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75680ce510 0x7f756809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 0x7f756809da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680cbcf0 con 0x7f75680cf690 2023-10-21T17:40:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756809e0e0 con 0x7f75680ce510 2023-10-21T17:40:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756c9a8640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756809e3c0 con 0x7f75680cf2b0 2023-10-21T17:40:14.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45432/0 (socket says 172.21.15.53:45432) 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f756759e640 1 -- 172.21.15.53:0/923045816 learned_addr learned my addr 172.21.15.53:0/923045816 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.328+0000 7f7566d9d640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75680ce510 0x7f756809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756659c640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 0x7f756809da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 -- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 msgr2=0x7f756809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 0x7f756809da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 -- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75680ce510 msgr2=0x7f756809d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75680ce510 0x7f756809d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 -- 172.21.15.53:0/923045816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550009a70 con 0x7f75680cf690 2023-10-21T17:40:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756659c640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 0x7f756809da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:14.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756759e640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f756000c410 tx=0x7f756000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560011790 con 0x7f75680cf690 2023-10-21T17:40:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75600118f0 con 0x7f75680cf690 2023-10-21T17:40:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560011ce0 con 0x7f75680cf690 2023-10-21T17:40:14.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.329+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756809e6a0 con 0x7f75680cf690 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.330+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756809ec60 con 0x7f75680cf690 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.331+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756809ee50 con 0x7f75680cf690 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.332+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 142) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f756001c460 con 0x7f75680cf690 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.332+0000 7f755f7fe640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 0x7f754806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.333+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f75600898d0 con 0x7f75680cf690 2023-10-21T17:40:14.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.333+0000 7f7566d9d640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 0x7f754806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.334+0000 7f7566d9d640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 0x7f754806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7558000f60 tx=0x7f7558007b50 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.337+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75600542f0 con 0x7f75680cf690 2023-10-21T17:40:14.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.529+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7568074150 con 0x7f75680cf690 2023-10-21T17:40:14.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.530+0000 7f755f7fe640 1 -- 172.21.15.53:0/923045816 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v260) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7560057910 con 0x7f75680cf690 2023-10-21T17:40:14.531 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:40:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 msgr2=0x7f754806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 0x7f754806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7558000f60 tx=0x7f7558007b50 comp rx=0 tx=0).stop 2023-10-21T17:40:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 msgr2=0x7f756809f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f756000c410 tx=0x7f756000c8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 shutdown_connections 2023-10-21T17:40:14.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f754806acc0 0x7f754806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75680cf690 0x7f756809f180 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75680cf2b0 0x7f756809da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 --2- 172.21.15.53:0/923045816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75680ce510 0x7f756809d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.532+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 >> 172.21.15.53:0/923045816 conn(0x7f756809bc40 msgr2=0x7f75680b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:14.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.533+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 shutdown_connections 2023-10-21T17:40:14.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.533+0000 7f756c9a8640 1 -- 172.21.15.53:0/923045816 wait complete. 2023-10-21T17:40:14.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:40:14.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.775+0000 7fbc3699e640 1 Processor -- start 2023-10-21T17:40:14.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.775+0000 7fbc3699e640 1 -- start start 2023-10-21T17:40:14.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3699e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc30069fc0 0x7fbc30071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3699e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 0x7fbc300c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3699e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc30160d10 con 0x7fbc300ca210 2023-10-21T17:40:14.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3699e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc30072070 con 0x7fbc30069fc0 2023-10-21T17:40:14.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3699e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30071580 con 0x7fbc3006ec00 2023-10-21T17:40:14.778 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.776+0000 7fbc3599c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc30069fc0 0x7fbc30071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbc30069fc0 0x7fbc30071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45448/0 (socket says 172.21.15.53:45448) 2023-10-21T17:40:14.779 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 -- 172.21.15.53:0/2856270250 learned_addr learned my addr 172.21.15.53:0/2856270250 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:14.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3519b640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 0x7fbc300c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 -- 172.21.15.53:0/2856270250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 msgr2=0x7fbc300c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:14.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 0x7fbc300c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 -- 172.21.15.53:0/2856270250 >> v1:172.21.15.124:6789/0 conn(0x7fbc300ca210 legacy=0x7fbc30160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:14.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3599c640 1 -- 172.21.15.53:0/2856270250 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30072330 con 0x7fbc30069fc0 2023-10-21T17:40:14.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.777+0000 7fbc3519b640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 0x7fbc300c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:14.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.779+0000 7fbc3599c640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc30069fc0 0x7fbc30071860 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009870 tx=0x7fbc2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11fe7cac6d948ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.779+0000 7fbc3499a640 1 -- 172.21.15.53:0/2856270250 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20031640 con 0x7fbc30069fc0 2023-10-21T17:40:14.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.779+0000 7fbc3499a640 1 -- 172.21.15.53:0/2856270250 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200317a0 con 0x7fbc30069fc0 2023-10-21T17:40:14.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.779+0000 7fbc3499a640 1 -- 172.21.15.53:0/2856270250 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20031b30 con 0x7fbc30069fc0 2023-10-21T17:40:14.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 -- 172.21.15.53:0/2856270250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc30069fc0 msgr2=0x7fbc30071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc30069fc0 0x7fbc30071860 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009870 tx=0x7fbc2002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 -- 172.21.15.53:0/2856270250 shutdown_connections 2023-10-21T17:40:14.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc3006ec00 0x7fbc300c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2856270250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc30069fc0 0x7fbc30071860 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 -- 172.21.15.53:0/2856270250 >> 172.21.15.53:0/2856270250 conn(0x7fbc300cf5f0 msgr2=0x7fbc300cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:14.784 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 -- 172.21.15.53:0/2856270250 shutdown_connections 2023-10-21T17:40:14.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.780+0000 7fbc3699e640 1 -- 172.21.15.53:0/2856270250 wait complete. 2023-10-21T17:40:14.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.782+0000 7fbc3699e640 1 Processor -- start 2023-10-21T17:40:14.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.782+0000 7fbc3699e640 1 -- start start 2023-10-21T17:40:14.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc30069fc0 0x7fbc301525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 0x7fbc301f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300b2790 con 0x7fbc3006ec00 2023-10-21T17:40:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300b2440 con 0x7fbc30069fc0 2023-10-21T17:40:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3699e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc301e80c0 con 0x7fbc300ca210 2023-10-21T17:40:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3519b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3519b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45452/0 (socket says 172.21.15.53:45452) 2023-10-21T17:40:14.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3519b640 1 -- 172.21.15.53:0/2706950694 learned_addr learned my addr 172.21.15.53:0/2706950694 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:14.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3599c640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc30069fc0 0x7fbc301525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.783+0000 7fbc3619d640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 0x7fbc301f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 -- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 msgr2=0x7fbc301f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 0x7fbc301f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 -- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc30069fc0 msgr2=0x7fbc301525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc30069fc0 0x7fbc301525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 -- 172.21.15.53:0/2706950694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200094d0 con 0x7fbc3006ec00 2023-10-21T17:40:14.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3619d640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 0x7fbc301f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:14.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3519b640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c7f0 tx=0x7fbc2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00cea0 con 0x7fbc3006ec00 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c007a30 con 0x7fbc3006ec00 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c007dc0 con 0x7fbc3006ec00 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc301e83a0 con 0x7fbc3006ec00 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.784+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc301530a0 con 0x7fbc3006ec00 2023-10-21T17:40:14.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.785+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc300b28d0 con 0x7fbc3006ec00 2023-10-21T17:40:14.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.786+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 142) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbc2c0113f0 con 0x7fbc3006ec00 2023-10-21T17:40:14.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.787+0000 7fbc267fc640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 0x7fbc1406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:14.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.787+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fbc2c089c50 con 0x7fbc3006ec00 2023-10-21T17:40:14.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.788+0000 7fbc3599c640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 0x7fbc1406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:14.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.789+0000 7fbc3599c640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 0x7fbc1406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc20036000 tx=0x7fbc2003c040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:14.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.792+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc300b28d0 con 0x7fbc3006ec00 2023-10-21T17:40:14.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.939+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 143) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fbc2c054ad0 con 0x7fbc3006ec00 2023-10-21T17:40:14.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.991+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc300b28d0 con 0x7fbc3006ec00 2023-10-21T17:40:14.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.991+0000 7fbc267fc640 1 -- 172.21.15.53:0/2706950694 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v260) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc300b28d0 con 0x7fbc3006ec00 2023-10-21T17:40:14.993 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:40:14.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 msgr2=0x7fbc1406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 0x7fbc1406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc20036000 tx=0x7fbc2003c040 comp rx=0 tx=0).stop 2023-10-21T17:40:14.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 msgr2=0x7fbc301f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:14.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c7f0 tx=0x7fbc2c00ccb0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 shutdown_connections 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbc1406acf0 0x7fbc1406d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbc300ca210 0x7fbc301f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc3006ec00 0x7fbc301f5ef0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 --2- 172.21.15.53:0/2706950694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbc30069fc0 0x7fbc301525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 >> 172.21.15.53:0/2706950694 conn(0x7fbc300cf5f0 msgr2=0x7fbc300cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:14.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.994+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 shutdown_connections 2023-10-21T17:40:14.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:14.995+0000 7fbc3699e640 1 -- 172.21.15.53:0/2706950694 wait complete. 2023-10-21T17:40:15.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:40:15.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.237+0000 7fdef3124640 1 Processor -- start 2023-10-21T17:40:15.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.238+0000 7fdef3124640 1 -- start start 2023-10-21T17:40:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.239+0000 7fdef3124640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 0x7fdeec06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.239+0000 7fdef3124640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdeec0cb150 0x7fdeec0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.239+0000 7fdef3124640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeec06cc50 con 0x7fdeec0702b0 2023-10-21T17:40:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.239+0000 7fdef3124640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeec0cbb10 con 0x7fdeec0cb150 2023-10-21T17:40:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.239+0000 7fdef3124640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec06cf30 con 0x7fdeec161690 2023-10-21T17:40:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdeec0cb150 0x7fdeec0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdeec0cb150 0x7fdeec0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45462/0 (socket says 172.21.15.53:45462) 2023-10-21T17:40:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 -- 172.21.15.53:0/1756766965 learned_addr learned my addr 172.21.15.53:0/1756766965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdef0ec0640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 0x7fdeec06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 -- 172.21.15.53:0/1756766965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 msgr2=0x7fdeec06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 0x7fdeec06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 -- 172.21.15.53:0/1756766965 >> v1:172.21.15.124:6789/0 conn(0x7fdeec0702b0 legacy=0x7fdeec070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:15.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdeebfff640 1 -- 172.21.15.53:0/1756766965 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec06d210 con 0x7fdeec0cb150 2023-10-21T17:40:15.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.240+0000 7fdef0ec0640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 0x7fdeec06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:15.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.241+0000 7fdeebfff640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec0cb5d0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fded8009870 tx=0x7fded802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db007b1c0d265b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdeeb7fe640 1 -- 172.21.15.53:0/1756766965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8031640 con 0x7fdeec0cb150 2023-10-21T17:40:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdeeb7fe640 1 -- 172.21.15.53:0/1756766965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded80317a0 con 0x7fdeec0cb150 2023-10-21T17:40:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdeeb7fe640 1 -- 172.21.15.53:0/1756766965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8031b30 con 0x7fdeec0cb150 2023-10-21T17:40:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 -- 172.21.15.53:0/1756766965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 msgr2=0x7fdeec0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec0cb5d0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fded8009870 tx=0x7fded802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 -- 172.21.15.53:0/1756766965 shutdown_connections 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec0cb5d0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 --2- 172.21.15.53:0/1756766965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec161690 0x7fdeec06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 -- 172.21.15.53:0/1756766965 >> 172.21.15.53:0/1756766965 conn(0x7fdeec06a5d0 msgr2=0x7fdeec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 -- 172.21.15.53:0/1756766965 shutdown_connections 2023-10-21T17:40:15.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.242+0000 7fdef3124640 1 -- 172.21.15.53:0/1756766965 wait complete. 2023-10-21T17:40:15.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.244+0000 7fdef3124640 1 Processor -- start 2023-10-21T17:40:15.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.244+0000 7fdef3124640 1 -- start start 2023-10-21T17:40:15.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec0702b0 0x7fdeec15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeec161690 0x7fdeec14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.250 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec159250 con 0x7fdeec0cb150 2023-10-21T17:40:15.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec159530 con 0x7fdeec161690 2023-10-21T17:40:15.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.245+0000 7fdef3124640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec159810 con 0x7fdeec0702b0 2023-10-21T17:40:15.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45478/0 (socket says 172.21.15.53:45478) 2023-10-21T17:40:15.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 -- 172.21.15.53:0/689725386 learned_addr learned my addr 172.21.15.53:0/689725386 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:15.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdef0ec0640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec0702b0 0x7fdeec15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 -- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec0702b0 msgr2=0x7fdeec15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec0702b0 0x7fdeec15a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 -- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeec161690 msgr2=0x7fdeec14d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeec161690 0x7fdeec14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 -- 172.21.15.53:0/689725386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80094d0 con 0x7fdeec0cb150 2023-10-21T17:40:15.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.246+0000 7fdeebfff640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fded8009870 tx=0x7fded80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.247+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded803ac60 con 0x7fdeec0cb150 2023-10-21T17:40:15.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.247+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded803ade0 con 0x7fdeec0cb150 2023-10-21T17:40:15.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.247+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8044630 con 0x7fdeec0cb150 2023-10-21T17:40:15.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.247+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec159af0 con 0x7fdeec0cb150 2023-10-21T17:40:15.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.247+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec14d590 con 0x7fdeec0cb150 2023-10-21T17:40:15.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.248+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec15a010 con 0x7fdeec0cb150 2023-10-21T17:40:15.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.249+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 143) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fded802f050 con 0x7fdeec0cb150 2023-10-21T17:40:15.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.250+0000 7fdee97fa640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 0x7fded006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.250+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fded80ac440 con 0x7fdeec0cb150 2023-10-21T17:40:15.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.251+0000 7fdef0ec0640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 0x7fded006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.252+0000 7fdef0ec0640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 0x7fded006d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdee000b840 tx=0x7fdee0008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.254+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdeec15a010 con 0x7fdeec0cb150 2023-10-21T17:40:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.448+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdeec15a010 con 0x7fdeec0cb150 2023-10-21T17:40:15.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.449+0000 7fdee97fa640 1 -- 172.21.15.53:0/689725386 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v260) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdeec15a010 con 0x7fdeec0cb150 2023-10-21T17:40:15.451 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:40:15.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 msgr2=0x7fded006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 0x7fded006d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdee000b840 tx=0x7fdee0008040 comp rx=0 tx=0).stop 2023-10-21T17:40:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 msgr2=0x7fdeec1f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fded8009870 tx=0x7fded80046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 shutdown_connections 2023-10-21T17:40:15.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fded006b020 0x7fded006d4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdeec161690 0x7fdeec14d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdeec0cb150 0x7fdeec1f64e0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 --2- 172.21.15.53:0/689725386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdeec0702b0 0x7fdeec15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 >> 172.21.15.53:0/689725386 conn(0x7fdeec06a5d0 msgr2=0x7fdeec0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:15.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 shutdown_connections 2023-10-21T17:40:15.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.452+0000 7fdef3124640 1 -- 172.21.15.53:0/689725386 wait complete. 2023-10-21T17:40:15.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:40:15.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.681+0000 7fb69729b640 1 Processor -- start 2023-10-21T17:40:15.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.681+0000 7fb69729b640 1 -- start start 2023-10-21T17:40:15.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6900b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69006dfc0 0x7fb690070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb690071070 0x7fb6900cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900b2620 con 0x7fb69006dfc0 2023-10-21T17:40:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900b2060 con 0x7fb690071070 2023-10-21T17:40:15.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb69729b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b2340 con 0x7fb6900afbe0 2023-10-21T17:40:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.683+0000 7fb694836640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69006dfc0 0x7fb690070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb69006dfc0 0x7fb690070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45494/0 (socket says 172.21.15.53:45494) 2023-10-21T17:40:15.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 -- 172.21.15.53:0/4183604791 learned_addr learned my addr 172.21.15.53:0/4183604791 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:15.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb695037640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6900b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb695838640 1 --2- 172.21.15.53:0/4183604791 >> v2:172.21.15.124:3300/0 conn(0x7fb690071070 0x7fb6900cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 -- 172.21.15.53:0/4183604791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 msgr2=0x7fb6900b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6900b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 -- 172.21.15.53:0/4183604791 >> v2:172.21.15.124:3300/0 conn(0x7fb690071070 msgr2=0x7fb6900cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 --2- 172.21.15.53:0/4183604791 >> v2:172.21.15.124:3300/0 conn(0x7fb690071070 0x7fb6900cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.684+0000 7fb694836640 1 -- 172.21.15.53:0/4183604791 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690070350 con 0x7fb69006dfc0 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.685+0000 7fb694836640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb690070b30 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b3980 tx=0x7fb68002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac944175c076b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.685+0000 7fb687fff640 1 -- 172.21.15.53:0/4183604791 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680031880 con 0x7fb69006dfc0 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb687fff640 1 -- 172.21.15.53:0/4183604791 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800319e0 con 0x7fb69006dfc0 2023-10-21T17:40:15.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb687fff640 1 -- 172.21.15.53:0/4183604791 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680031d70 con 0x7fb69006dfc0 2023-10-21T17:40:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 -- 172.21.15.53:0/4183604791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 msgr2=0x7fb690070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb690070b30 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fb6900b3980 tx=0x7fb68002b920 comp rx=0 tx=0).stop 2023-10-21T17:40:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 -- 172.21.15.53:0/4183604791 shutdown_connections 2023-10-21T17:40:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 --2- 172.21.15.53:0/4183604791 >> v2:172.21.15.124:3300/0 conn(0x7fb690071070 0x7fb6900cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb690070b30 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 --2- 172.21.15.53:0/4183604791 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6900b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 -- 172.21.15.53:0/4183604791 >> 172.21.15.53:0/4183604791 conn(0x7fb69006a5d0 msgr2=0x7fb6900b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 -- 172.21.15.53:0/4183604791 shutdown_connections 2023-10-21T17:40:15.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.686+0000 7fb69729b640 1 -- 172.21.15.53:0/4183604791 wait complete. 2023-10-21T17:40:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.688+0000 7fb69729b640 1 Processor -- start 2023-10-21T17:40:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.688+0000 7fb69729b640 1 -- start start 2023-10-21T17:40:15.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb690071070 0x7fb6900c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6901f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c9d10 con 0x7fb69006dfc0 2023-10-21T17:40:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c9ff0 con 0x7fb690071070 2023-10-21T17:40:15.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.689+0000 7fb69729b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900ca2d0 con 0x7fb6900afbe0 2023-10-21T17:40:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45496/0 (socket says 172.21.15.53:45496) 2023-10-21T17:40:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 -- 172.21.15.53:0/3808398361 learned_addr learned my addr 172.21.15.53:0/3808398361 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695838640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6901f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 -- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 msgr2=0x7fb6901f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb694836640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb690071070 0x7fb6900c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6901f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 -- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb690071070 msgr2=0x7fb6900c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb690071070 0x7fb6900c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 -- 172.21.15.53:0/3808398361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800094d0 con 0x7fb69006dfc0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.690+0000 7fb695037640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00d800 tx=0x7fb68c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.691+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00b6c0 con 0x7fb69006dfc0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.691+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c00b820 con 0x7fb69006dfc0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.691+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00bbb0 con 0x7fb69006dfc0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.691+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900ca5b0 con 0x7fb69006dfc0 2023-10-21T17:40:15.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.691+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900c94c0 con 0x7fb69006dfc0 2023-10-21T17:40:15.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.692+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69004e160 con 0x7fb69006dfc0 2023-10-21T17:40:15.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.693+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 143) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb68c00bd10 con 0x7fb69006dfc0 2023-10-21T17:40:15.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.694+0000 7fb685ffb640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 0x7fb67406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:15.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.694+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fb68c08a7e0 con 0x7fb69006dfc0 2023-10-21T17:40:15.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.695+0000 7fb694836640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 0x7fb67406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:15.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.696+0000 7fb694836640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 0x7fb67406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb680036000 tx=0x7fb68003c040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:15.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.698+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb68c054d00 con 0x7fb69006dfc0 2023-10-21T17:40:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.894+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6900b0ef0 con 0x7fb69006dfc0 2023-10-21T17:40:15.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.894+0000 7fb685ffb640 1 -- 172.21.15.53:0/3808398361 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v260) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb68c058320 con 0x7fb69006dfc0 2023-10-21T17:40:15.896 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:40:15.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 msgr2=0x7fb67406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 0x7fb67406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb680036000 tx=0x7fb68003c040 comp rx=0 tx=0).stop 2023-10-21T17:40:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 msgr2=0x7fb6900c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00d800 tx=0x7fb68c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 shutdown_connections 2023-10-21T17:40:15.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb67406b020 0x7fb67406d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6900afbe0 0x7fb6901f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb690071070 0x7fb6900c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 --2- 172.21.15.53:0/3808398361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb69006dfc0 0x7fb6900c9730 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 >> 172.21.15.53:0/3808398361 conn(0x7fb69006a5d0 msgr2=0x7fb6900c6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:15.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 shutdown_connections 2023-10-21T17:40:15.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:15.897+0000 7fb69729b640 1 -- 172.21.15.53:0/3808398361 wait complete. 2023-10-21T17:40:15.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:16.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.139+0000 7fc0da685640 1 Processor -- start 2023-10-21T17:40:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.139+0000 7fc0da685640 1 -- start start 2023-10-21T17:40:16.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.140+0000 7fc0da685640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0d415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.140+0000 7fc0da685640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d415f7d0 con 0x7fc0d40b07d0 2023-10-21T17:40:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.140+0000 7fc0da685640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d415f210 con 0x7fc0d40adc70 2023-10-21T17:40:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.140+0000 7fc0da685640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415f4f0 con 0x7fc0d415ab30 2023-10-21T17:40:16.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.141+0000 7fc0d9e84640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc0d40b07d0 0x7fc0d406cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36586/0 (socket says 172.21.15.53:36586) 2023-10-21T17:40:16.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.141+0000 7fc0d9e84640 1 -- 172.21.15.53:0/3620727515 learned_addr learned my addr 172.21.15.53:0/3620727515 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.141+0000 7fc0d9683640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0d415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.141+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057112790 0 0) 0x7fc0d415f7d0 con 0x7fc0d40b07d0 2023-10-21T17:40:16.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.141+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c0003520 con 0x7fc0d40b07d0 2023-10-21T17:40:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458176992 0 0) 0x7fc0c0003520 con 0x7fc0d40b07d0 2023-10-21T17:40:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 msgr2=0x7fc0d415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0d415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 >> v1:172.21.15.124:6789/0 conn(0x7fc0d40adc70 legacy=0x7fc0d40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d406c550 con 0x7fc0d40b07d0 2023-10-21T17:40:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fc0cc001670 con 0x7fc0d40b07d0 2023-10-21T17:40:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 >> v1:172.21.15.53:6789/0 conn(0x7fc0d40b07d0 legacy=0x7fc0d406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00039a0 0x7fc0c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415f4f0 con 0x7fc0c00047c0 2023-10-21T17:40:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c000ba60 con 0x7fc0c00039a0 2023-10-21T17:40:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c000bc20 con 0x7fc0d415ab30 2023-10-21T17:40:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0d9683640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00039a0 0x7fc0c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.142+0000 7fc0d9e84640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d8e82640 1 -- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 msgr2=0x7fc0c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d8e82640 1 -- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00039a0 msgr2=0x7fc0c0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:16.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00039a0 0x7fc0c0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d8e82640 1 -- 172.21.15.53:0/3620727515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0003520 con 0x7fc0c00047c0 2023-10-21T17:40:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.143+0000 7fc0d9e84640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d415ab30 0x7fc0c0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.144+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4001720 tx=0x7fc0c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754ced0c458babae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.144+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40052d0 con 0x7fc0c00047c0 2023-10-21T17:40:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.144+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c402e050 con 0x7fc0c00047c0 2023-10-21T17:40:16.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.144+0000 7fc0cbfff640 1 -- 172.21.15.53:0/3620727515 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40035f0 con 0x7fc0c00047c0 2023-10-21T17:40:16.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 -- 172.21.15.53:0/3620727515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0c00047c0 msgr2=0x7fc0c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 --2- 172.21.15.53:0/3620727515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0c00047c0 0x7fc0c0004c40 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4001720 tx=0x7fc0c00057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0d9e84640 1 -- 172.21.15.53:0/3620727515 reap_dead start 2023-10-21T17:40:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 -- 172.21.15.53:0/3620727515 shutdown_connections 2023-10-21T17:40:16.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 -- 172.21.15.53:0/3620727515 >> 172.21.15.53:0/3620727515 conn(0x7fc0d40cb0c0 msgr2=0x7fc0d40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 -- 172.21.15.53:0/3620727515 shutdown_connections 2023-10-21T17:40:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.145+0000 7fc0da685640 1 -- 172.21.15.53:0/3620727515 wait complete. 2023-10-21T17:40:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.147+0000 7fc0da685640 1 Processor -- start 2023-10-21T17:40:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.147+0000 7fc0da685640 1 -- start start 2023-10-21T17:40:16.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00047c0 0x7fc0d40c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 0x7fc0d40b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40b3420 con 0x7fc0d41e44e0 2023-10-21T17:40:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40c21e0 con 0x7fc0c00047c0 2023-10-21T17:40:16.156 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0da685640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d41e4e60 con 0x7fc0d40b2a60 2023-10-21T17:40:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d8e82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d8e82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45512/0 (socket says 172.21.15.53:45512) 2023-10-21T17:40:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d8e82640 1 -- 172.21.15.53:0/1089105838 learned_addr learned my addr 172.21.15.53:0/1089105838 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d9683640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00047c0 0x7fc0d40c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d9e84640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 0x7fc0d40b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.148+0000 7fc0d8e82640 1 -- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 msgr2=0x7fc0d40b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 0x7fc0d40b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d8e82640 1 -- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00047c0 msgr2=0x7fc0d40c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00047c0 0x7fc0d40c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d8e82640 1 -- 172.21.15.53:0/1089105838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d41e5380 con 0x7fc0d41e44e0 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d9e84640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 0x7fc0d40b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0d8e82640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005040 tx=0x7fc0c4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40076c0 con 0x7fc0d41e44e0 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c40069e0 con 0x7fc0d41e44e0 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4006d50 con 0x7fc0d41e44e0 2023-10-21T17:40:16.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d4152710 con 0x7fc0d41e44e0 2023-10-21T17:40:16.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.149+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d40c2410 con 0x7fc0d41e44e0 2023-10-21T17:40:16.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.151+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d404e160 con 0x7fc0d41e44e0 2023-10-21T17:40:16.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.151+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 143) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc0c40035f0 con 0x7fc0d41e44e0 2023-10-21T17:40:16.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.152+0000 7fc0c9ffb640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 0x7fc0b806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.152+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fc0c40ac020 con 0x7fc0d41e44e0 2023-10-21T17:40:16.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.153+0000 7fc0d9683640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 0x7fc0b806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.154+0000 7fc0d9683640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 0x7fc0b806d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0007380 tx=0x7fc0d000b040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.157+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0c40764d0 con 0x7fc0d41e44e0 2023-10-21T17:40:16.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.395+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0d40ada50 con 0x7fc0d41e44e0 2023-10-21T17:40:16.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.396+0000 7fc0c9ffb640 1 -- 172.21.15.53:0/1089105838 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7fc0c404b090 con 0x7fc0d41e44e0 2023-10-21T17:40:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:16.398 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:16.398 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:40:16.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 msgr2=0x7fc0b806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 0x7fc0b806d510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0007380 tx=0x7fc0d000b040 comp rx=0 tx=0).stop 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 msgr2=0x7fc0d41e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005040 tx=0x7fc0c4001120 comp rx=0 tx=0).stop 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 shutdown_connections 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc0b806b050 0x7fc0b806d510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d40b2a60 0x7fc0d40b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0d41e44e0 0x7fc0d41e4920 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 --2- 172.21.15.53:0/1089105838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0c00047c0 0x7fc0d40c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.399+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 >> 172.21.15.53:0/1089105838 conn(0x7fc0d40cb0c0 msgr2=0x7fc0d406d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:16.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.400+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 shutdown_connections 2023-10-21T17:40:16.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.400+0000 7fc0da685640 1 -- 172.21.15.53:0/1089105838 wait complete. 2023-10-21T17:40:16.421 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:16.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.633+0000 7f7f2a2a2640 1 Processor -- start 2023-10-21T17:40:16.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.634+0000 7f7f2a2a2640 1 -- start start 2023-10-21T17:40:16.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2a2a2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f240709e0 0x7f7f24164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2a2a2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2406dee0 con 0x7f7f240ad790 2023-10-21T17:40:16.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2a2a2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2415d850 con 0x7f7f24159ac0 2023-10-21T17:40:16.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2a2a2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2406dcf0 con 0x7f7f240709e0 2023-10-21T17:40:16.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2883f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f240709e0 0x7f7f24164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2883f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f240709e0 0x7f7f24164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45524/0 (socket says 172.21.15.53:45524) 2023-10-21T17:40:16.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f2883f640 1 -- 172.21.15.53:0/3121598569 learned_addr learned my addr 172.21.15.53:0/3121598569 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:16.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.635+0000 7f7f1bfff640 1 --1- 172.21.15.53:0/3121598569 >> v1:172.21.15.53:6789/0 conn(0x7f7f240ad790 0x7f7f240702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36592/0 (socket says 172.21.15.53:36592) 2023-10-21T17:40:16.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.636+0000 7f7f2883f640 1 -- 172.21.15.53:0/3121598569 >> v1:172.21.15.124:6789/0 conn(0x7f7f24159ac0 legacy=0x7f7f2406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:16.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.636+0000 7f7f2883f640 1 -- 172.21.15.53:0/3121598569 >> v1:172.21.15.53:6789/0 conn(0x7f7f240ad790 legacy=0x7f7f240702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.636+0000 7f7f2883f640 1 -- 172.21.15.53:0/3121598569 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2415db30 con 0x7f7f240709e0 2023-10-21T17:40:16.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.637+0000 7f7f2883f640 1 --2- 172.21.15.53:0/3121598569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f24164a60 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba59505311ee7d54 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.637+0000 7f7f237fe640 1 -- 172.21.15.53:0/3121598569 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c02cb10 con 0x7f7f240709e0 2023-10-21T17:40:16.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.637+0000 7f7f237fe640 1 -- 172.21.15.53:0/3121598569 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c02cc70 con 0x7f7f240709e0 2023-10-21T17:40:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.637+0000 7f7f237fe640 1 -- 172.21.15.53:0/3121598569 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0343f0 con 0x7f7f240709e0 2023-10-21T17:40:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/3121598569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 msgr2=0x7f7f24164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/3121598569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f24164a60 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c02bf70 comp rx=0 tx=0).stop 2023-10-21T17:40:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/3121598569 shutdown_connections 2023-10-21T17:40:16.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/3121598569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f24164a60 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/3121598569 >> 172.21.15.53:0/3121598569 conn(0x7f7f2406a380 msgr2=0x7f7f240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/3121598569 shutdown_connections 2023-10-21T17:40:16.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.638+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/3121598569 wait complete. 2023-10-21T17:40:16.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.640+0000 7f7f2a2a2640 1 Processor -- start 2023-10-21T17:40:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.640+0000 7f7f2a2a2640 1 -- start start 2023-10-21T17:40:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f240ad790 0x7f7f2415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 0x7f7f240af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240b3860 con 0x7f7f240709e0 2023-10-21T17:40:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240b1b60 con 0x7f7f240ad790 2023-10-21T17:40:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f2a2a2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240b2f90 con 0x7f7f24159ac0 2023-10-21T17:40:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45526/0 (socket says 172.21.15.53:45526) 2023-10-21T17:40:16.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f23fff640 1 -- 172.21.15.53:0/865681818 learned_addr learned my addr 172.21.15.53:0/865681818 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.641+0000 7f7f1bfff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f240ad790 0x7f7f2415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f2883f640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 0x7f7f240af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 -- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 msgr2=0x7f7f240af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 0x7f7f240af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 -- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f240ad790 msgr2=0x7f7f2415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f240ad790 0x7f7f2415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 -- 172.21.15.53:0/865681818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009b30 con 0x7f7f240709e0 2023-10-21T17:40:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f2883f640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 0x7f7f240af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f23fff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400b420 tx=0x7f7f1400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14011830 con 0x7f7f240709e0 2023-10-21T17:40:16.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f14011990 con 0x7f7f240709e0 2023-10-21T17:40:16.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14011d40 con 0x7f7f240709e0 2023-10-21T17:40:16.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.642+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240b1880 con 0x7f7f240709e0 2023-10-21T17:40:16.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.643+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f240b34f0 con 0x7f7f240709e0 2023-10-21T17:40:16.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.644+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f24067490 con 0x7f7f240709e0 2023-10-21T17:40:16.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.645+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 143) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7f1401b460 con 0x7f7f240709e0 2023-10-21T17:40:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.646+0000 7f7f217fa640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 0x7f7f0406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.646+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f7f14089c40 con 0x7f7f240709e0 2023-10-21T17:40:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.646+0000 7f7f1bfff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 0x7f7f0406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:16.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.648+0000 7f7f1bfff640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 0x7f7f0406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f240c5090 tx=0x7f7f0c007c80 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:16.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.652+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f140542e0 con 0x7f7f240709e0 2023-10-21T17:40:16.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.849+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f240b0ff0 con 0x7f7f240709e0 2023-10-21T17:40:16.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.851+0000 7f7f217fa640 1 -- 172.21.15.53:0/865681818 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21885 (secure 0 0 0) 0x7f7f14057900 con 0x7f7f240709e0 2023-10-21T17:40:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:16.853 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":260,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:12.899743+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 msgr2=0x7f7f0406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 0x7f7f0406d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f240c5090 tx=0x7f7f0c007c80 comp rx=0 tx=0).stop 2023-10-21T17:40:16.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 msgr2=0x7f7f2415ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400b420 tx=0x7f7f1400b8e0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 shutdown_connections 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f7f0406b070 0x7f7f0406d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f24159ac0 0x7f7f240af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f240ad790 0x7f7f2415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 --2- 172.21.15.53:0/865681818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f240709e0 0x7f7f2415ccc0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.854+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 >> 172.21.15.53:0/865681818 conn(0x7f7f2406a380 msgr2=0x7f7f241579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.855+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 shutdown_connections 2023-10-21T17:40:16.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:16.855+0000 7f7f2a2a2640 1 -- 172.21.15.53:0/865681818 wait complete. 2023-10-21T17:40:16.875 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:40:16.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:17.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.102+0000 7f6b21a83640 1 Processor -- start 2023-10-21T17:40:17.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.102+0000 7f6b21a83640 1 -- start start 2023-10-21T17:40:17.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c15cd90 0x7f6b1c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.105 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b1c160940 0x7f6b1c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6b1c070d50 0x7f6b1c0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c4d80 con 0x7f6b1c160940 2023-10-21T17:40:17.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c48f0 con 0x7f6b1c070d50 2023-10-21T17:40:17.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b21a83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0c3ce0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b12ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b1c160940 0x7f6b1c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b12ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6b1c160940 0x7f6b1c070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45540/0 (socket says 172.21.15.53:45540) 2023-10-21T17:40:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.104+0000 7f6b12ffd640 1 -- 172.21.15.53:0/2383284911 learned_addr learned my addr 172.21.15.53:0/2383284911 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b1bfff640 1 --2- 172.21.15.53:0/2383284911 >> v2:172.21.15.124:3300/0 conn(0x7f6b1c070d50 0x7f6b1c0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b1b7fe640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c15cd90 0x7f6b1c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b12ffd640 1 -- 172.21.15.53:0/2383284911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c15cd90 msgr2=0x7f6b1c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c15cd90 0x7f6b1c0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b12ffd640 1 -- 172.21.15.53:0/2383284911 >> v2:172.21.15.124:3300/0 conn(0x7f6b1c070d50 msgr2=0x7f6b1c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/2383284911 >> v2:172.21.15.124:3300/0 conn(0x7f6b1c070d50 0x7f6b1c0cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.105+0000 7f6b12ffd640 1 -- 172.21.15.53:0/2383284911 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0c3fc0 con 0x7f6b1c160940 2023-10-21T17:40:17.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.106+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c160940 0x7f6b1c070810 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0ce470 tx=0x7f6b0402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc19e619a07603c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.106+0000 7f6b1affd640 1 -- 172.21.15.53:0/2383284911 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04031880 con 0x7f6b1c160940 2023-10-21T17:40:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.106+0000 7f6b1affd640 1 -- 172.21.15.53:0/2383284911 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b040319e0 con 0x7f6b1c160940 2023-10-21T17:40:17.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.106+0000 7f6b1affd640 1 -- 172.21.15.53:0/2383284911 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04031d70 con 0x7f6b1c160940 2023-10-21T17:40:17.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 -- 172.21.15.53:0/2383284911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c160940 msgr2=0x7f6b1c070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c160940 0x7f6b1c070810 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0ce470 tx=0x7f6b0402b920 comp rx=0 tx=0).stop 2023-10-21T17:40:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 -- 172.21.15.53:0/2383284911 shutdown_connections 2023-10-21T17:40:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 --2- 172.21.15.53:0/2383284911 >> v2:172.21.15.124:3300/0 conn(0x7f6b1c070d50 0x7f6b1c0cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c160940 0x7f6b1c070810 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 --2- 172.21.15.53:0/2383284911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c15cd90 0x7f6b1c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 -- 172.21.15.53:0/2383284911 >> 172.21.15.53:0/2383284911 conn(0x7f6b1c06a380 msgr2=0x7f6b1c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 -- 172.21.15.53:0/2383284911 shutdown_connections 2023-10-21T17:40:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.107+0000 7f6b21a83640 1 -- 172.21.15.53:0/2383284911 wait complete. 2023-10-21T17:40:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.109+0000 7f6b21a83640 1 Processor -- start 2023-10-21T17:40:17.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.109+0000 7f6b21a83640 1 -- start start 2023-10-21T17:40:17.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 0x7f6b1c14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b1c160940 0x7f6b1c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1f3570 con 0x7f6b1c15cd90 2023-10-21T17:40:17.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1f2c40 con 0x7f6b1c160940 2023-10-21T17:40:17.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.110+0000 7f6b21a83640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1f2f20 con 0x7f6b1c070d50 2023-10-21T17:40:17.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b12ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b12ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45550/0 (socket says 172.21.15.53:45550) 2023-10-21T17:40:17.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b12ffd640 1 -- 172.21.15.53:0/1031310938 learned_addr learned my addr 172.21.15.53:0/1031310938 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:17.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b1b7fe640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 0x7f6b1c14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b1bfff640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b1c160940 0x7f6b1c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b12ffd640 1 -- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 msgr2=0x7f6b1c14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.111+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 0x7f6b1c14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b12ffd640 1 -- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b1c160940 msgr2=0x7f6b1c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b1c160940 0x7f6b1c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b12ffd640 1 -- 172.21.15.53:0/1031310938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040094d0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b1b7fe640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 0x7f6b1c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:40:17.118 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b12ffd640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f6b040090b0 tx=0x7f6b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0403ad20 con 0x7f6b1c15cd90 2023-10-21T17:40:17.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0403ae80 con 0x7f6b1c15cd90 2023-10-21T17:40:17.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04044630 con 0x7f6b1c15cd90 2023-10-21T17:40:17.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.112+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c1f3200 con 0x7f6b1c15cd90 2023-10-21T17:40:17.119 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.113+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c1f28c0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.114+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c067490 con 0x7f6b1c15cd90 2023-10-21T17:40:17.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.114+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6b0404d450 con 0x7f6b1c15cd90 2023-10-21T17:40:17.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.115+0000 7f6b18ff9640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 0x7f6afc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.116+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f6b040ac9e0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.116+0000 7f6b1b7fe640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 0x7f6afc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.117+0000 7f6b1b7fe640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 0x7f6afc06d570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e4a0 tx=0x7f6b0c007650 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.120+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b04076ed0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.351+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b1c0b0ef0 con 0x7f6b1c15cd90 2023-10-21T17:40:17.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.352+0000 7f6b18ff9640 1 -- 172.21.15.53:0/1031310938 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7f6b04035070 con 0x7f6b1c15cd90 2023-10-21T17:40:17.354 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:17.354 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:17.354 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:40:17.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 msgr2=0x7f6afc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 0x7f6afc06d570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e4a0 tx=0x7f6b0c007650 comp rx=0 tx=0).stop 2023-10-21T17:40:17.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 msgr2=0x7f6b1c15d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f6b040090b0 tx=0x7f6b040046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 shutdown_connections 2023-10-21T17:40:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6afc06b0b0 0x7f6afc06d570 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b1c160940 0x7f6b1c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b1c15cd90 0x7f6b1c15d350 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 --2- 172.21.15.53:0/1031310938 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b1c070d50 0x7f6b1c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.355+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 >> 172.21.15.53:0/1031310938 conn(0x7f6b1c06a380 msgr2=0x7f6b1c158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.356+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 shutdown_connections 2023-10-21T17:40:17.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.356+0000 7f6b21a83640 1 -- 172.21.15.53:0/1031310938 wait complete. 2023-10-21T17:40:17.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:17.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.606+0000 7fb7557e9640 1 Processor -- start 2023-10-21T17:40:17.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.607+0000 7fb7557e9640 1 -- start start 2023-10-21T17:40:17.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7557e9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7500adc70 0x7fb7500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7557e9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb75015af10 0x7fb7500b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7557e9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75015f7d0 con 0x7fb75015ab30 2023-10-21T17:40:17.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7557e9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75015efb0 con 0x7fb7500adc70 2023-10-21T17:40:17.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7557e9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75015f210 con 0x7fb75015af10 2023-10-21T17:40:17.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7477fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7500adc70 0x7fb7500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7477fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb7500adc70 0x7fb7500b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45566/0 (socket says 172.21.15.53:45566) 2023-10-21T17:40:17.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb7477fe640 1 -- 172.21.15.53:0/1435731404 learned_addr learned my addr 172.21.15.53:0/1435731404 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:17.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.608+0000 7fb74ffff640 1 --1- 172.21.15.53:0/1435731404 >> v1:172.21.15.53:6789/0 conn(0x7fb75015ab30 0x7fb75015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36600/0 (socket says 172.21.15.53:36600) 2023-10-21T17:40:17.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb754fe8640 1 --2- 172.21.15.53:0/1435731404 >> v2:172.21.15.124:3300/0 conn(0x7fb75015af10 0x7fb7500b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb7477fe640 1 -- 172.21.15.53:0/1435731404 >> v2:172.21.15.124:3300/0 conn(0x7fb75015af10 msgr2=0x7fb7500b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb7477fe640 1 --2- 172.21.15.53:0/1435731404 >> v2:172.21.15.124:3300/0 conn(0x7fb75015af10 0x7fb7500b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb7477fe640 1 -- 172.21.15.53:0/1435731404 >> v1:172.21.15.53:6789/0 conn(0x7fb75015ab30 legacy=0x7fb75015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb7477fe640 1 -- 172.21.15.53:0/1435731404 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75015f4f0 con 0x7fb7500adc70 2023-10-21T17:40:17.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.609+0000 7fb74f7fe640 1 -- 172.21.15.53:0/1435731404 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099025379 0 0) 0x7fb75015f7d0 con 0x7fb75015ab30 2023-10-21T17:40:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.610+0000 7fb7477fe640 1 --2- 172.21.15.53:0/1435731404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7500b00c0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb7500ca780 tx=0x7fb73802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4365c1328e3093 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.610+0000 7fb74f7fe640 1 -- 172.21.15.53:0/1435731404 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738030860 con 0x7fb7500adc70 2023-10-21T17:40:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.610+0000 7fb74f7fe640 1 -- 172.21.15.53:0/1435731404 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7380309c0 con 0x7fb7500adc70 2023-10-21T17:40:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.610+0000 7fb74f7fe640 1 -- 172.21.15.53:0/1435731404 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738030d50 con 0x7fb7500adc70 2023-10-21T17:40:17.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 -- 172.21.15.53:0/1435731404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 msgr2=0x7fb7500b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 --2- 172.21.15.53:0/1435731404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7500b00c0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb7500ca780 tx=0x7fb73802b9a0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 -- 172.21.15.53:0/1435731404 shutdown_connections 2023-10-21T17:40:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 --2- 172.21.15.53:0/1435731404 >> v2:172.21.15.124:3300/0 conn(0x7fb75015af10 0x7fb7500b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 --2- 172.21.15.53:0/1435731404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7500b00c0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 -- 172.21.15.53:0/1435731404 >> 172.21.15.53:0/1435731404 conn(0x7fb7500cb0c0 msgr2=0x7fb7500cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 -- 172.21.15.53:0/1435731404 shutdown_connections 2023-10-21T17:40:17.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.611+0000 7fb7557e9640 1 -- 172.21.15.53:0/1435731404 wait complete. 2023-10-21T17:40:17.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.613+0000 7fb7557e9640 1 Processor -- start 2023-10-21T17:40:17.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.613+0000 7fb7557e9640 1 -- start start 2023-10-21T17:40:17.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb75015ab30 0x7fb7500c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 0x7fb7501ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501f1440 con 0x7fb7500adc70 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500c6170 con 0x7fb75015ab30 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7557e9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500c5840 con 0x7fb75015af10 2023-10-21T17:40:17.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb74ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45570/0 (socket says 172.21.15.53:45570) 2023-10-21T17:40:17.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb74ffff640 1 -- 172.21.15.53:0/3448853666 learned_addr learned my addr 172.21.15.53:0/3448853666 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:17.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.614+0000 7fb7477fe640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb75015ab30 0x7fb7500c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb754fe8640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 0x7fb7501ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 -- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 msgr2=0x7fb7501ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 0x7fb7501ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 -- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb75015ab30 msgr2=0x7fb7500c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb75015ab30 0x7fb7500c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 -- 172.21.15.53:0/3448853666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380094d0 con 0x7fb7500adc70 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb754fe8640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 0x7fb7501ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:17.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74ffff640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c4b0 tx=0x7fb74000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740007480 con 0x7fb7500adc70 2023-10-21T17:40:17.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400075e0 con 0x7fb7500adc70 2023-10-21T17:40:17.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740004a20 con 0x7fb7500adc70 2023-10-21T17:40:17.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7500c5b20 con 0x7fb7500adc70 2023-10-21T17:40:17.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.615+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7501f1ba0 con 0x7fb7500adc70 2023-10-21T17:40:17.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.616+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75004e160 con 0x7fb7500adc70 2023-10-21T17:40:17.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.618+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb740004b80 con 0x7fb7500adc70 2023-10-21T17:40:17.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.618+0000 7fb74d7fa640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 0x7fb73006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:17.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.619+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fb740089540 con 0x7fb7500adc70 2023-10-21T17:40:17.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.619+0000 7fb7477fe640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 0x7fb73006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:17.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.620+0000 7fb7477fe640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 0x7fb73006d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb738036000 tx=0x7fb738008030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:17.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.623+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb740053ad0 con 0x7fb7500adc70 2023-10-21T17:40:17.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.858+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7500ada50 con 0x7fb7500adc70 2023-10-21T17:40:17.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.859+0000 7fb74d7fa640 1 -- 172.21.15.53:0/3448853666 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+6159 (secure 0 0 0) 0x7fb740018020 con 0x7fb7500adc70 2023-10-21T17:40:17.861 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:17.861 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":134,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:39.693015+0000","modified":"2023-10-21T17:39:42.885777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38348},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38348":{"gid":38348,"name":"b","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/2789835635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2789835635},{"type":"v1","addr":"172.21.15.124:6836","nonce":2789835635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:17.861 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 134 2023-10-21T17:40:17.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 msgr2=0x7fb73006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 0x7fb73006d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb738036000 tx=0x7fb738008030 comp rx=0 tx=0).stop 2023-10-21T17:40:17.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 msgr2=0x7fb7501f1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:17.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c4b0 tx=0x7fb74000c970 comp rx=0 tx=0).stop 2023-10-21T17:40:17.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 shutdown_connections 2023-10-21T17:40:17.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fb73006b0b0 0x7fb73006d570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb75015af10 0x7fb7501ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb75015ab30 0x7fb7500c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 --2- 172.21.15.53:0/3448853666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7500adc70 0x7fb7501f1660 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:17.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 >> 172.21.15.53:0/3448853666 conn(0x7fb7500cb0c0 msgr2=0x7fb75006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:17.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 shutdown_connections 2023-10-21T17:40:17.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:17.862+0000 7fb7557e9640 1 -- 172.21.15.53:0/3448853666 wait complete. 2023-10-21T17:40:17.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.105+0000 7fee611dc640 1 Processor -- start 2023-10-21T17:40:18.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.106+0000 7fee611dc640 1 -- start start 2023-10-21T17:40:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.107+0000 7fee611dc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c0cbfe0 0x7fee5c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.107+0000 7fee611dc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c06a730 con 0x7fee5c070b40 2023-10-21T17:40:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.108+0000 7fee611dc640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c15e2b0 con 0x7fee5c065eb0 2023-10-21T17:40:18.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.108+0000 7fee611dc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0cd7c0 con 0x7fee5c0cbfe0 2023-10-21T17:40:18.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.108+0000 7fee5b59e640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fee5c070b40 0x7fee5c156c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36608/0 (socket says 172.21.15.53:36608) 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.108+0000 7fee5b59e640 1 -- 172.21.15.53:0/1985167377 learned_addr learned my addr 172.21.15.53:0/1985167377 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.108+0000 7fee5ad9d640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c0cbfe0 0x7fee5c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443176814 0 0) 0x7fee5c06a730 con 0x7fee5c070b40 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee48003520 con 0x7fee5c070b40 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729973698 0 0) 0x7fee48003520 con 0x7fee5c070b40 2023-10-21T17:40:18.112 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c0cbfe0 msgr2=0x7fee5c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c0cbfe0 0x7fee5c15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 >> v1:172.21.15.124:6789/0 conn(0x7fee5c065eb0 legacy=0x7fee5c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0cdaa0 con 0x7fee5c070b40 2023-10-21T17:40:18.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fee54001670 con 0x7fee5c070b40 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 >> v1:172.21.15.53:6789/0 conn(0x7fee5c070b40 legacy=0x7fee5c156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee480039a0 0x7fee48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee48005180 0x7fee4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0cd7c0 con 0x7fee480047c0 2023-10-21T17:40:18.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4800bfd0 con 0x7fee48005180 2023-10-21T17:40:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4800c190 con 0x7fee480039a0 2023-10-21T17:40:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5ad9d640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c0cbfe0 0x7fee5c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:40:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.109+0000 7fee5259c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5b59e640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee48005180 0x7fee4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5259c640 1 -- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee480039a0 msgr2=0x7fee48003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:18.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5259c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee480039a0 0x7fee48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5259c640 1 -- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee48005180 msgr2=0x7fee4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:18.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5259c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee48005180 0x7fee4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.116 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5259c640 1 -- 172.21.15.53:0/1985167377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48003520 con 0x7fee480047c0 2023-10-21T17:40:18.117 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.110+0000 7fee5b59e640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee48005180 0x7fee4800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.111+0000 7fee5259c640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee48004c40 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fee440016a0 tx=0x7fee480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fbaec29b31d926 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.111+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee440052d0 con 0x7fee480047c0 2023-10-21T17:40:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.111+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4402e050 con 0x7fee480047c0 2023-10-21T17:40:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee5a59c640 1 -- 172.21.15.53:0/1985167377 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44003830 con 0x7fee480047c0 2023-10-21T17:40:18.120 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 -- 172.21.15.53:0/1985167377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 msgr2=0x7fee48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 --2- 172.21.15.53:0/1985167377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee48004c40 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fee440016a0 tx=0x7fee480057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee5b59e640 1 -- 172.21.15.53:0/1985167377 reap_dead start 2023-10-21T17:40:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 -- 172.21.15.53:0/1985167377 shutdown_connections 2023-10-21T17:40:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 -- 172.21.15.53:0/1985167377 >> 172.21.15.53:0/1985167377 conn(0x7fee5c0b1480 msgr2=0x7fee5c06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:18.121 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 -- 172.21.15.53:0/1985167377 shutdown_connections 2023-10-21T17:40:18.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.112+0000 7fee611dc640 1 -- 172.21.15.53:0/1985167377 wait complete. 2023-10-21T17:40:18.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.114+0000 7fee611dc640 1 Processor -- start 2023-10-21T17:40:18.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.114+0000 7fee611dc640 1 -- start start 2023-10-21T17:40:18.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.122 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee5c161060 0x7fee5c0b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 0x7fee5c0b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0ac4b0 con 0x7fee480047c0 2023-10-21T17:40:18.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c157da0 con 0x7fee5c161060 2023-10-21T17:40:18.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.115+0000 7fee611dc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0b42c0 con 0x7fee5c1614e0 2023-10-21T17:40:18.123 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45588/0 (socket says 172.21.15.53:45588) 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 -- 172.21.15.53:0/2571601904 learned_addr learned my addr 172.21.15.53:0/2571601904 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5259c640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee5c161060 0x7fee5c0b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5b59e640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 0x7fee5c0b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 -- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 msgr2=0x7fee5c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 0x7fee5c0b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 -- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee5c161060 msgr2=0x7fee5c0b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee5c161060 0x7fee5c0b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5ad9d640 1 -- 172.21.15.53:0/2571601904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0b45a0 con 0x7fee480047c0 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.116+0000 7fee5b59e640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 0x7fee5c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee5ad9d640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009e50 tx=0x7fee4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.125 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c003990 con 0x7fee480047c0 2023-10-21T17:40:18.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c00f040 con 0x7fee480047c0 2023-10-21T17:40:18.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c003d50 con 0x7fee480047c0 2023-10-21T17:40:18.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c0b4880 con 0x7fee480047c0 2023-10-21T17:40:18.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.117+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c0b3ee0 con 0x7fee480047c0 2023-10-21T17:40:18.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.118+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c067490 con 0x7fee480047c0 2023-10-21T17:40:18.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.119+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fee4c004030 con 0x7fee480047c0 2023-10-21T17:40:18.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.120+0000 7fee53fff640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 0x7fee3c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.120+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fee4c08b3e0 con 0x7fee480047c0 2023-10-21T17:40:18.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.121+0000 7fee5259c640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 0x7fee3c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.122+0000 7fee5259c640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 0x7fee3c06d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee44033000 tx=0x7fee44009030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.125+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee4c0557b0 con 0x7fee480047c0 2023-10-21T17:40:18.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.317+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee5c0c9500 con 0x7fee480047c0 2023-10-21T17:40:18.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.322+0000 7fee53fff640 1 -- 172.21.15.53:0/2571601904 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21885 (secure 0 0 0) 0x7fee4c058dd0 con 0x7fee480047c0 2023-10-21T17:40:18.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:18.324 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":260,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:12.899743+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:39:37.842433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-21T17:39:38.736221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:18.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.324+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 msgr2=0x7fee3c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.324+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 0x7fee3c06d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fee44033000 tx=0x7fee44009030 comp rx=0 tx=0).stop 2023-10-21T17:40:18.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.324+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 msgr2=0x7fee5c1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.324+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009e50 tx=0x7fee4c00c490 comp rx=0 tx=0).stop 2023-10-21T17:40:18.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 shutdown_connections 2023-10-21T17:40:18.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fee3c06b0b0 0x7fee3c06d570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee5c1614e0 0x7fee5c0b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee5c161060 0x7fee5c0b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 --2- 172.21.15.53:0/2571601904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee480047c0 0x7fee5c1573c0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 >> 172.21.15.53:0/2571601904 conn(0x7fee5c0b1480 msgr2=0x7fee5c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:18.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 shutdown_connections 2023-10-21T17:40:18.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.325+0000 7fee611dc640 1 -- 172.21.15.53:0/2571601904 wait complete. 2023-10-21T17:40:18.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:40:18.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.571+0000 7f1344c72640 1 Processor -- start 2023-10-21T17:40:18.573 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.571+0000 7f1344c72640 1 -- start start 2023-10-21T17:40:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1340159db0 0x7f13400cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1340071dc0 0x7f13400cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 0x7f13400cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1340072250 con 0x7f1340159db0 2023-10-21T17:40:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134015e200 con 0x7f13400cbbe0 2023-10-21T17:40:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f1344c72640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134015e4e0 con 0x7f1340071dc0 2023-10-21T17:40:18.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f133f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1340159db0 0x7f13400cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f133f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1340159db0 0x7f13400cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45592/0 (socket says 172.21.15.53:45592) 2023-10-21T17:40:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.573+0000 7f133f7fe640 1 -- 172.21.15.53:0/4179495761 learned_addr learned my addr 172.21.15.53:0/4179495761 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133ffff640 1 --2- 172.21.15.53:0/4179495761 >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 0x7f13400cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133f7fe640 1 -- 172.21.15.53:0/4179495761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1340071dc0 msgr2=0x7f13400cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:18.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133effd640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1340071dc0 0x7f13400cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133f7fe640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1340071dc0 0x7f13400cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133f7fe640 1 -- 172.21.15.53:0/4179495761 >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 msgr2=0x7f13400cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133f7fe640 1 --2- 172.21.15.53:0/4179495761 >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 0x7f13400cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133f7fe640 1 -- 172.21.15.53:0/4179495761 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400cb180 con 0x7f1340159db0 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.574+0000 7f133ffff640 1 --2- 172.21.15.53:0/4179495761 >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 0x7f13400cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:18.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.575+0000 7f133f7fe640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f13400cb6a0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f13400ca300 tx=0x7f133002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8133607823399ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.575+0000 7f133e7fc640 1 -- 172.21.15.53:0/4179495761 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330031870 con 0x7f1340159db0 2023-10-21T17:40:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.575+0000 7f133e7fc640 1 -- 172.21.15.53:0/4179495761 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13300319d0 con 0x7f1340159db0 2023-10-21T17:40:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.575+0000 7f133e7fc640 1 -- 172.21.15.53:0/4179495761 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330031d60 con 0x7f1340159db0 2023-10-21T17:40:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 -- 172.21.15.53:0/4179495761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 msgr2=0x7f13400cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f13400cb6a0 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f13400ca300 tx=0x7f133002b910 comp rx=0 tx=0).stop 2023-10-21T17:40:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 -- 172.21.15.53:0/4179495761 shutdown_connections 2023-10-21T17:40:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1340071dc0 0x7f13400cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 --2- 172.21.15.53:0/4179495761 >> v2:172.21.15.124:3300/0 conn(0x7f13400cbbe0 0x7f13400cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 --2- 172.21.15.53:0/4179495761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f13400cb6a0 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 -- 172.21.15.53:0/4179495761 >> 172.21.15.53:0/4179495761 conn(0x7f13400cf5f0 msgr2=0x7f13400cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 -- 172.21.15.53:0/4179495761 shutdown_connections 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.576+0000 7f1344c72640 1 -- 172.21.15.53:0/4179495761 wait complete. 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.578+0000 7f1344c72640 1 Processor -- start 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.578+0000 7f1344c72640 1 -- start start 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13400cbbe0 0x7f13400b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1340156d20 0x7f13400afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13400c3530 con 0x7f1340159db0 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340156b40 con 0x7f1340156d20 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f1344c72640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13401f2400 con 0x7f13400cbbe0 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f133effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f133effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45600/0 (socket says 172.21.15.53:45600) 2023-10-21T17:40:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.579+0000 7f133effd640 1 -- 172.21.15.53:0/3791468618 learned_addr learned my addr 172.21.15.53:0/3791468618 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133f7fe640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13400cbbe0 0x7f13400b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133ffff640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1340156d20 0x7f13400afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 -- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13400cbbe0 msgr2=0x7f13400b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13400cbbe0 0x7f13400b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 -- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1340156d20 msgr2=0x7f13400afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1340156d20 0x7f13400afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 -- 172.21.15.53:0/3791468618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300094d0 con 0x7f1340159db0 2023-10-21T17:40:18.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.580+0000 7f133effd640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f132800c570 tx=0x7f132800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.591 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.581+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328011800 con 0x7f1340159db0 2023-10-21T17:40:18.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.581+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1328011960 con 0x7f1340159db0 2023-10-21T17:40:18.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.581+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328011d10 con 0x7f1340159db0 2023-10-21T17:40:18.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.581+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13400aba50 con 0x7f1340159db0 2023-10-21T17:40:18.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.581+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13400ac010 con 0x7f1340159db0 2023-10-21T17:40:18.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.582+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1340074150 con 0x7f1340159db0 2023-10-21T17:40:18.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.583+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1328018360 con 0x7f1340159db0 2023-10-21T17:40:18.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.584+0000 7f1337fff640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 0x7f132006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:18.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.584+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f132808a390 con 0x7f1340159db0 2023-10-21T17:40:18.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.585+0000 7f133f7fe640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 0x7f132006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:18.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.586+0000 7f133f7fe640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 0x7f132006d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1330036000 tx=0x7f133003c040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:18.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.588+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1328054750 con 0x7f1340159db0 2023-10-21T17:40:18.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.826+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f134015dd20 con 0x7f1340159db0 2023-10-21T17:40:18.828 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:40:18.827+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:40:18.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.830+0000 7f1337fff640 1 -- 172.21.15.53:0/3791468618 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v135) v1 ==== 161+0+0 (secure 0 0 0) 0x7f132801f090 con 0x7f1340159db0 2023-10-21T17:40:18.832 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:40:18.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 msgr2=0x7f132006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 0x7f132006d570 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1330036000 tx=0x7f133003c040 comp rx=0 tx=0).stop 2023-10-21T17:40:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 msgr2=0x7f1340156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f132800c570 tx=0x7f132800ca30 comp rx=0 tx=0).stop 2023-10-21T17:40:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 shutdown_connections 2023-10-21T17:40:18.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f132006b0b0 0x7f132006d570 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1340156d20 0x7f13400afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1340159db0 0x7f1340156600 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 --2- 172.21.15.53:0/3791468618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13400cbbe0 0x7f13400b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 >> 172.21.15.53:0/3791468618 conn(0x7f13400cf5f0 msgr2=0x7f13400af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 shutdown_connections 2023-10-21T17:40:18.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:18.835+0000 7f1344c72640 1 -- 172.21.15.53:0/3791468618 wait complete. 2023-10-21T17:40:18.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:40:18.865 INFO:tasks.ceph.mds.b.smithi124.stderr: -4320> 2023-10-21T17:39:44.367+0000 7fc88f001640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:40:18.962 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:40:19.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.065+0000 7feabbb73640 1 Processor -- start 2023-10-21T17:40:19.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.065+0000 7feabbb73640 1 -- start start 2023-10-21T17:40:19.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.066+0000 7feabbb73640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feab40afbe0 0x7feab40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.066+0000 7feabbb73640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feab406dfc0 0x7feab4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feabbb73640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40b2620 con 0x7feab4071070 2023-10-21T17:40:19.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feabbb73640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40b2060 con 0x7feab406dfc0 2023-10-21T17:40:19.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feabbb73640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40b2340 con 0x7feab40afbe0 2023-10-21T17:40:19.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab910e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7feab406dfc0 0x7feab4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feaba110640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feab4071070 0x7feab40cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36610/0 (socket says 172.21.15.53:36610) 2023-10-21T17:40:19.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feaba110640 1 -- 172.21.15.53:0/2264843423 learned_addr learned my addr 172.21.15.53:0/2264843423 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:19.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab990f640 1 --2- 172.21.15.53:0/2264843423 >> v2:172.21.15.124:3300/0 conn(0x7feab40afbe0 0x7feab40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab890d640 1 -- 172.21.15.53:0/2264843423 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630994340 0 0) 0x7feab40b2620 con 0x7feab4071070 2023-10-21T17:40:19.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab890d640 1 -- 172.21.15.53:0/2264843423 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa0003520 con 0x7feab4071070 2023-10-21T17:40:19.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab910e640 1 -- 172.21.15.53:0/2264843423 >> v2:172.21.15.124:3300/0 conn(0x7feab40afbe0 msgr2=0x7feab40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:19.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab910e640 1 --2- 172.21.15.53:0/2264843423 >> v2:172.21.15.124:3300/0 conn(0x7feab40afbe0 0x7feab40b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab910e640 1 -- 172.21.15.53:0/2264843423 >> v1:172.21.15.53:6789/0 conn(0x7feab4071070 legacy=0x7feab40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.067+0000 7feab910e640 1 -- 172.21.15.53:0/2264843423 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4070350 con 0x7feab406dfc0 2023-10-21T17:40:19.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.069+0000 7feab910e640 1 --2- 172.21.15.53:0/2264843423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab406dfc0 0x7feab4070b30 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7feab406e640 tx=0x7feaa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c70a75bbe4739c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:19.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.069+0000 7feab890d640 1 -- 172.21.15.53:0/2264843423 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031830 con 0x7feab406dfc0 2023-10-21T17:40:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.069+0000 7feab890d640 1 -- 172.21.15.53:0/2264843423 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4031990 con 0x7feab406dfc0 2023-10-21T17:40:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.069+0000 7feab890d640 1 -- 172.21.15.53:0/2264843423 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4031d20 con 0x7feab406dfc0 2023-10-21T17:40:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 -- 172.21.15.53:0/2264843423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab406dfc0 msgr2=0x7feab4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 --2- 172.21.15.53:0/2264843423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab406dfc0 0x7feab4070b30 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7feab406e640 tx=0x7feaa402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 -- 172.21.15.53:0/2264843423 shutdown_connections 2023-10-21T17:40:19.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 --2- 172.21.15.53:0/2264843423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab406dfc0 0x7feab4070b30 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 --2- 172.21.15.53:0/2264843423 >> v2:172.21.15.124:3300/0 conn(0x7feab40afbe0 0x7feab40b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 -- 172.21.15.53:0/2264843423 >> 172.21.15.53:0/2264843423 conn(0x7feab406a5d0 msgr2=0x7feab40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:19.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 -- 172.21.15.53:0/2264843423 shutdown_connections 2023-10-21T17:40:19.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.070+0000 7feabbb73640 1 -- 172.21.15.53:0/2264843423 wait complete. 2023-10-21T17:40:19.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.071+0000 7feabbb73640 1 Processor -- start 2023-10-21T17:40:19.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.072+0000 7feabbb73640 1 -- start start 2023-10-21T17:40:19.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.072+0000 7feabbb73640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feab406dfc0 0x7feab40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feabbb73640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 0x7feab414db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feabbb73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feabbb73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40c3780 con 0x7feab40afbe0 2023-10-21T17:40:19.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feabbb73640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab40ceda0 con 0x7feab4071070 2023-10-21T17:40:19.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feabbb73640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab40c2de0 con 0x7feab406dfc0 2023-10-21T17:40:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45620/0 (socket says 172.21.15.53:45620) 2023-10-21T17:40:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 -- 172.21.15.53:0/2243109573 learned_addr learned my addr 172.21.15.53:0/2243109573 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feab990f640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feab406dfc0 0x7feab40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feab910e640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 0x7feab414db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 -- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feab406dfc0 msgr2=0x7feab40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feab406dfc0 0x7feab40ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 -- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 msgr2=0x7feab414db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 0x7feab414db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feaba110640 1 -- 172.21.15.53:0/2243109573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009400 con 0x7feab40afbe0 2023-10-21T17:40:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.073+0000 7feab910e640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 0x7feab414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feaba110640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7feaac00cae0 tx=0x7feaac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00ac40 con 0x7feab40afbe0 2023-10-21T17:40:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac00ada0 con 0x7feab40afbe0 2023-10-21T17:40:19.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac012640 con 0x7feab40afbe0 2023-10-21T17:40:19.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab40c30c0 con 0x7feab40afbe0 2023-10-21T17:40:19.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.074+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab40ceab0 con 0x7feab40afbe0 2023-10-21T17:40:19.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.075+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab40c38c0 con 0x7feab40afbe0 2023-10-21T17:40:19.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.076+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7feaac007070 con 0x7feab40afbe0 2023-10-21T17:40:19.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.077+0000 7feaaa7fc640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 0x7fea9806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:19.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.077+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 7955+0+0 (secure 0 0 0) 0x7feaac08a2f0 con 0x7feab40afbe0 2023-10-21T17:40:19.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.078+0000 7feab990f640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 0x7fea9806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:19.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.079+0000 7feab990f640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 0x7fea9806d540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feab0006fd0 tx=0x7feab0008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:19.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.081+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feab40c38c0 con 0x7feab40afbe0 2023-10-21T17:40:19.320 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.319+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feab40c38c0 con 0x7feab40afbe0 2023-10-21T17:40:19.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.955+0000 7feaaa7fc640 1 -- 172.21.15.53:0/2243109573 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v136) v1 ==== 104+0+0 (secure 0 0 0) 0x7feab40c38c0 con 0x7feab40afbe0 2023-10-21T17:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 msgr2=0x7fea9806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 0x7fea9806d540 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feab0006fd0 tx=0x7feab0008040 comp rx=0 tx=0).stop 2023-10-21T17:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 msgr2=0x7feab40c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:19.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7feaac00cae0 tx=0x7feaac00a060 comp rx=0 tx=0).stop 2023-10-21T17:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 shutdown_connections 2023-10-21T17:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fea9806b080 0x7fea9806d540 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40afbe0 0x7feab40c3ba0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feab4071070 0x7feab414db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 --2- 172.21.15.53:0/2243109573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feab406dfc0 0x7feab40ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 >> 172.21.15.53:0/2243109573 conn(0x7feab406a5d0 msgr2=0x7feab406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.958+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 shutdown_connections 2023-10-21T17:40:19.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:19.959+0000 7feabbb73640 1 -- 172.21.15.53:0/2243109573 wait complete. 2023-10-21T17:40:19.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:40:20.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.220+0000 7f61f359e640 1 Processor -- start 2023-10-21T17:40:20.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.221+0000 7f61f359e640 1 -- start start 2023-10-21T17:40:20.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f61f40cf690 0x7f61f40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61f415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40cd520 con 0x7f61f40cf2b0 2023-10-21T17:40:20.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f40d1910 con 0x7f61f40cf690 2023-10-21T17:40:20.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40ce080 con 0x7f61f40ce510 2023-10-21T17:40:20.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f61f40cf2b0 0x7f61f40d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36624/0 (socket says 172.21.15.53:36624) 2023-10-21T17:40:20.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.222+0000 7f61f259c640 1 -- 172.21.15.53:0/416620359 learned_addr learned my addr 172.21.15.53:0/416620359 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.223+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384315694 0 0) 0x7f61f40cd520 con 0x7f61f40cf2b0 2023-10-21T17:40:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.223+0000 7f61f1d9b640 1 --2- 172.21.15.53:0/416620359 >> v2:172.21.15.124:3300/0 conn(0x7f61f40cf690 0x7f61f40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.223+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61f415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.223+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61dc003520 con 0x7f61f40cf2b0 2023-10-21T17:40:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.223+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313800894 0 0) 0x7f61dc003520 con 0x7f61f40cf2b0 2023-10-21T17:40:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 msgr2=0x7f61f415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61f415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 >> v2:172.21.15.124:3300/0 conn(0x7f61f40cf690 msgr2=0x7f61f40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:20.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 --2- 172.21.15.53:0/416620359 >> v2:172.21.15.124:3300/0 conn(0x7f61f40cf690 0x7f61f40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40cd750 con 0x7f61f40cf2b0 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f61e40019f0 con 0x7f61f40cf2b0 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 >> v1:172.21.15.53:6789/0 conn(0x7f61f40cf2b0 legacy=0x7f61f40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61dc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc003f40 0x7f61dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40d1910 con 0x7f61dc004d00 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40ce080 con 0x7f61dc003f40 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc00ba30 con 0x7f61f40ce510 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.224+0000 7f61f259c640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61dc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f1d9b640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc003f40 0x7f61dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f2d9d640 1 -- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 msgr2=0x7f61dc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f40ce510 0x7f61dc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f2d9d640 1 -- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc003f40 msgr2=0x7f61dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc003f40 0x7f61dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.225+0000 7f61f2d9d640 1 -- 172.21.15.53:0/416620359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc003520 con 0x7f61dc004d00 2023-10-21T17:40:20.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.226+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ac0 tx=0x7f61dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5638db6404b4f5f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:20.234 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.226+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec02c6b0 con 0x7f61dc004d00 2023-10-21T17:40:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.226+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec02c810 con 0x7f61dc004d00 2023-10-21T17:40:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.226+0000 7f61f159a640 1 -- 172.21.15.53:0/416620359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec02cba0 con 0x7f61dc004d00 2023-10-21T17:40:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 -- 172.21.15.53:0/416620359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61dc004d00 msgr2=0x7f61dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 --2- 172.21.15.53:0/416620359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ac0 tx=0x7f61dc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.235 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f2d9d640 1 -- 172.21.15.53:0/416620359 reap_dead start 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 -- 172.21.15.53:0/416620359 shutdown_connections 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 -- 172.21.15.53:0/416620359 >> 172.21.15.53:0/416620359 conn(0x7f61f409bc40 msgr2=0x7f61f40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 -- 172.21.15.53:0/416620359 shutdown_connections 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.227+0000 7f61f359e640 1 -- 172.21.15.53:0/416620359 wait complete. 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.229+0000 7f61f359e640 1 Processor -- start 2023-10-21T17:40:20.236 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.229+0000 7f61f359e640 1 -- start start 2023-10-21T17:40:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc004d00 0x7f61f414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f409dfd0 0x7f61f415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415dfa0 con 0x7f61f415f580 2023-10-21T17:40:20.237 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415e280 con 0x7f61dc004d00 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415e560 con 0x7f61f409dfd0 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45626/0 (socket says 172.21.15.53:45626) 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.230+0000 7f61f2d9d640 1 -- 172.21.15.53:0/2657879965 learned_addr learned my addr 172.21.15.53:0/2657879965 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f259c640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc004d00 0x7f61f414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.238 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f1d9b640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f409dfd0 0x7f61f415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 -- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f409dfd0 msgr2=0x7f61f415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f409dfd0 0x7f61f415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 -- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc004d00 msgr2=0x7f61f414fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc004d00 0x7f61f414fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.239 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 -- 172.21.15.53:0/2657879965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f415e840 con 0x7f61f415f580 2023-10-21T17:40:20.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.231+0000 7f61f2d9d640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f61ec037040 tx=0x7f61ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:20.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.232+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec02c6b0 con 0x7f61f415f580 2023-10-21T17:40:20.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.232+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec003810 con 0x7f61f415f580 2023-10-21T17:40:20.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.232+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec003bc0 con 0x7f61f415f580 2023-10-21T17:40:20.240 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.232+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f415eb20 con 0x7f61f415f580 2023-10-21T17:40:20.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.232+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f409fdb0 con 0x7f61f415f580 2023-10-21T17:40:20.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.233+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f415ed10 con 0x7f61f415f580 2023-10-21T17:40:20.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.234+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f61ec003d20 con 0x7f61f415f580 2023-10-21T17:40:20.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.235+0000 7f61e2ffd640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 0x7f61d406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:20.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.235+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 7955+0+0 (secure 0 0 0) 0x7f61ec0aba70 con 0x7f61f415f580 2023-10-21T17:40:20.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.236+0000 7f61f259c640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 0x7f61d406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:20.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.237+0000 7f61f259c640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 0x7f61d406d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001330 tx=0x7f61e4006040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:20.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.239+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61ec075f80 con 0x7f61f415f580 2023-10-21T17:40:20.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.438+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61f40cda00 con 0x7f61f415f580 2023-10-21T17:40:20.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:40:20.775 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:40:20.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.974+0000 7f61e2ffd640 1 -- 172.21.15.53:0/2657879965 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v262) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61ec0390d0 con 0x7f61f415f580 2023-10-21T17:40:20.976 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:40:20.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.978+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 msgr2=0x7f61d406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.978+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 0x7f61d406d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61e4001330 tx=0x7f61e4006040 comp rx=0 tx=0).stop 2023-10-21T17:40:20.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.978+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 msgr2=0x7f61f406afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:20.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.978+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f61ec037040 tx=0x7f61ec004730 comp rx=0 tx=0).stop 2023-10-21T17:40:20.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.978+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 shutdown_connections 2023-10-21T17:40:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f61d406b0b0 0x7f61d406d570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61f415f580 0x7f61f406afb0 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f61f409dfd0 0x7f61f415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 --2- 172.21.15.53:0/2657879965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f61dc004d00 0x7f61f414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:20.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 >> 172.21.15.53:0/2657879965 conn(0x7f61f409bc40 msgr2=0x7f61f41e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.979+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 shutdown_connections 2023-10-21T17:40:20.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:20.980+0000 7f61f359e640 1 -- 172.21.15.53:0/2657879965 wait complete. 2023-10-21T17:40:21.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:40:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.212+0000 7f5db3fff640 1 Processor -- start 2023-10-21T17:40:21.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.212+0000 7f5db3fff640 1 -- start start 2023-10-21T17:40:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.213+0000 7f5db3fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5db40cf690 0x7f5db40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5db40ce510 0x7f5db415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db3fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cd520 con 0x7f5db40cf2b0 2023-10-21T17:40:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db3fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db40d1910 con 0x7f5db40ce510 2023-10-21T17:40:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db3fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db40ce080 con 0x7f5db40cf690 2023-10-21T17:40:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5db40ce510 0x7f5db415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5db40ce510 0x7f5db415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45642/0 (socket says 172.21.15.53:45642) 2023-10-21T17:40:21.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db37fe640 1 -- 172.21.15.53:0/470794823 learned_addr learned my addr 172.21.15.53:0/470794823 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:21.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db2ffd640 1 --1- 172.21.15.53:0/470794823 >> v1:172.21.15.53:6789/0 conn(0x7f5db40cf2b0 0x7f5db40d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36640/0 (socket says 172.21.15.53:36640) 2023-10-21T17:40:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db27fc640 1 --2- 172.21.15.53:0/470794823 >> v2:172.21.15.124:3300/0 conn(0x7f5db40cf690 0x7f5db40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db37fe640 1 -- 172.21.15.53:0/470794823 >> v2:172.21.15.124:3300/0 conn(0x7f5db40cf690 msgr2=0x7f5db40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:21.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.214+0000 7f5db37fe640 1 --2- 172.21.15.53:0/470794823 >> v2:172.21.15.124:3300/0 conn(0x7f5db40cf690 0x7f5db40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.215+0000 7f5db37fe640 1 -- 172.21.15.53:0/470794823 >> v1:172.21.15.53:6789/0 conn(0x7f5db40cf2b0 legacy=0x7f5db40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.215+0000 7f5db37fe640 1 -- 172.21.15.53:0/470794823 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40cd750 con 0x7f5db40ce510 2023-10-21T17:40:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db37fe640 1 --2- 172.21.15.53:0/470794823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db415c820 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f5db408baa0 tx=0x7f5dac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69dd929cfc4b0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db1ffb640 1 -- 172.21.15.53:0/470794823 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac02ca80 con 0x7f5db40ce510 2023-10-21T17:40:21.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db1ffb640 1 -- 172.21.15.53:0/470794823 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac02cbe0 con 0x7f5db40ce510 2023-10-21T17:40:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db1ffb640 1 -- 172.21.15.53:0/470794823 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0343f0 con 0x7f5db40ce510 2023-10-21T17:40:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db3fff640 1 -- 172.21.15.53:0/470794823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 msgr2=0x7f5db415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.216+0000 7f5db3fff640 1 --2- 172.21.15.53:0/470794823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db415c820 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f5db408baa0 tx=0x7f5dac02bee0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 -- 172.21.15.53:0/470794823 shutdown_connections 2023-10-21T17:40:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 --2- 172.21.15.53:0/470794823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db415c820 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 --2- 172.21.15.53:0/470794823 >> v2:172.21.15.124:3300/0 conn(0x7f5db40cf690 0x7f5db40d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 -- 172.21.15.53:0/470794823 >> 172.21.15.53:0/470794823 conn(0x7f5db409bc40 msgr2=0x7f5db40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 -- 172.21.15.53:0/470794823 shutdown_connections 2023-10-21T17:40:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.217+0000 7f5db3fff640 1 -- 172.21.15.53:0/470794823 wait complete. 2023-10-21T17:40:21.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.218+0000 7f5db3fff640 1 Processor -- start 2023-10-21T17:40:21.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 -- start start 2023-10-21T17:40:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 0x7f5db415b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 0x7f5db415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db409d510 con 0x7f5db40ce510 2023-10-21T17:40:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db409d9a0 con 0x7f5db40cf690 2023-10-21T17:40:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.219+0000 7f5db3fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db409c900 con 0x7f5db40cf2b0 2023-10-21T17:40:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45658/0 (socket says 172.21.15.53:45658) 2023-10-21T17:40:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 -- 172.21.15.53:0/734856310 learned_addr learned my addr 172.21.15.53:0/734856310 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db37fe640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 0x7f5db415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db27fc640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 0x7f5db415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 -- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 msgr2=0x7f5db415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 0x7f5db415b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 -- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 msgr2=0x7f5db415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 0x7f5db415ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db2ffd640 1 -- 172.21.15.53:0/734856310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac009b30 con 0x7f5db40ce510 2023-10-21T17:40:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db27fc640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 0x7f5db415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.220+0000 7f5db37fe640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 0x7f5db415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5db2ffd640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c4d0 tx=0x7f5da400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4007480 con 0x7f5db40ce510 2023-10-21T17:40:21.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40075e0 con 0x7f5db40ce510 2023-10-21T17:40:21.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db409cbe0 con 0x7f5db40ce510 2023-10-21T17:40:21.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40049e0 con 0x7f5db40ce510 2023-10-21T17:40:21.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.221+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db409d1a0 con 0x7f5db40ce510 2023-10-21T17:40:21.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.222+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4074150 con 0x7f5db40ce510 2023-10-21T17:40:21.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.223+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5da4004b40 con 0x7f5db40ce510 2023-10-21T17:40:21.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.224+0000 7f5d9b7fe640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 0x7f5da006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:21.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.224+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 7517+0+0 (secure 0 0 0) 0x7f5da4089190 con 0x7f5db40ce510 2023-10-21T17:40:21.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.225+0000 7f5db27fc640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 0x7f5da006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:21.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.226+0000 7f5db27fc640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 0x7f5da006d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:21.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.229+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5da4053870 con 0x7f5db40ce510 2023-10-21T17:40:21.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.429+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5db40d0070 con 0x7f5db40ce510 2023-10-21T17:40:21.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.978+0000 7f5d9b7fe640 1 -- 172.21.15.53:0/734856310 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v263) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5da40180d0 con 0x7f5db40ce510 2023-10-21T17:40:21.980 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:40:21.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.984+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 msgr2=0x7f5da006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.984+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 0x7f5da006d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5da8006fd0 tx=0x7f5da8008040 comp rx=0 tx=0).stop 2023-10-21T17:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 msgr2=0x7f5db409ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f5da400c4d0 tx=0x7f5da400c990 comp rx=0 tx=0).stop 2023-10-21T17:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 shutdown_connections 2023-10-21T17:40:21.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f5da006b0b0 0x7f5da006d570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5db40cf690 0x7f5db415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5db40cf2b0 0x7f5db415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 --2- 172.21.15.53:0/734856310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5db40ce510 0x7f5db409ef30 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:21.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.985+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 >> 172.21.15.53:0/734856310 conn(0x7f5db409bc40 msgr2=0x7f5db415fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.986+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 shutdown_connections 2023-10-21T17:40:21.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:21.987+0000 7f5db3fff640 1 -- 172.21.15.53:0/734856310 wait complete. 2023-10-21T17:40:22.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:22.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.209+0000 7f140ed6c640 1 Processor -- start 2023-10-21T17:40:22.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.209+0000 7f140ed6c640 1 -- start start 2023-10-21T17:40:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140ed6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f14080ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140ed6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14080c90b0 0x7f14080adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140ed6c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140815b2b0 con 0x7f14080cae70 2023-10-21T17:40:22.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140ed6c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140815bf30 con 0x7f14080c90b0 2023-10-21T17:40:22.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140ed6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140815f7a0 con 0x7f140815ab40 2023-10-21T17:40:22.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f13fffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14080c90b0 0x7f14080adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f13fffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f14080c90b0 0x7f14080adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45662/0 (socket says 172.21.15.53:45662) 2023-10-21T17:40:22.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f13fffff640 1 -- 172.21.15.53:0/3556965790 learned_addr learned my addr 172.21.15.53:0/3556965790 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.211+0000 7f140d309640 1 --1- 172.21.15.53:0/3556965790 >> v1:172.21.15.53:6789/0 conn(0x7f14080cae70 0x7f14080702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36648/0 (socket says 172.21.15.53:36648) 2023-10-21T17:40:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f140cb08640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f14080ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f13fffff640 1 -- 172.21.15.53:0/3556965790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 msgr2=0x7f14080ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f13fffff640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f14080ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f13fffff640 1 -- 172.21.15.53:0/3556965790 >> v1:172.21.15.53:6789/0 conn(0x7f14080cae70 legacy=0x7f14080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f13fffff640 1 -- 172.21.15.53:0/3556965790 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140815b590 con 0x7f14080c90b0 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.212+0000 7f140cb08640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f14080ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.213+0000 7f13fffff640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f14080adc60 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba60e21573b87c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.213+0000 7f13ff7fe640 1 -- 172.21.15.53:0/3556965790 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8030840 con 0x7f14080c90b0 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.213+0000 7f13ff7fe640 1 -- 172.21.15.53:0/3556965790 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80309a0 con 0x7f14080c90b0 2023-10-21T17:40:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.213+0000 7f13ff7fe640 1 -- 172.21.15.53:0/3556965790 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8030d30 con 0x7f14080c90b0 2023-10-21T17:40:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 -- 172.21.15.53:0/3556965790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 msgr2=0x7f14080adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f14080adc60 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802b980 comp rx=0 tx=0).stop 2023-10-21T17:40:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 -- 172.21.15.53:0/3556965790 shutdown_connections 2023-10-21T17:40:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f14080adc60 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 --2- 172.21.15.53:0/3556965790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f14080ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 -- 172.21.15.53:0/3556965790 >> 172.21.15.53:0/3556965790 conn(0x7f140806a380 msgr2=0x7f14080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 -- 172.21.15.53:0/3556965790 shutdown_connections 2023-10-21T17:40:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.214+0000 7f140ed6c640 1 -- 172.21.15.53:0/3556965790 wait complete. 2023-10-21T17:40:22.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.216+0000 7f140ed6c640 1 Processor -- start 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.216+0000 7f140ed6c640 1 -- start start 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 0x7f1408153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f1408155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c56a0 con 0x7f14080c90b0 2023-10-21T17:40:22.222 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c3b30 con 0x7f14080cae70 2023-10-21T17:40:22.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140ed6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c50e0 con 0x7f140815ab40 2023-10-21T17:40:22.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140cb08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140cb08640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45668/0 (socket says 172.21.15.53:45668) 2023-10-21T17:40:22.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.217+0000 7f140cb08640 1 -- 172.21.15.53:0/1599485168 learned_addr learned my addr 172.21.15.53:0/1599485168 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:22.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 -- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 msgr2=0x7f1408155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f13fffff640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 0x7f1408153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140d309640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f1408155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f1408155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 -- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 msgr2=0x7f1408153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 0x7f1408153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 -- 172.21.15.53:0/1599485168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80094d0 con 0x7f14080c90b0 2023-10-21T17:40:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f13fffff640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 0x7f1408153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140d309640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f1408155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140cb08640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f140400c7f0 tx=0x7f140400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404007a30 con 0x7f14080c90b0 2023-10-21T17:40:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404007b90 con 0x7f14080c90b0 2023-10-21T17:40:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400ce40 con 0x7f14080c90b0 2023-10-21T17:40:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14080c53c0 con 0x7f14080c90b0 2023-10-21T17:40:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.218+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14081539a0 con 0x7f14080c90b0 2023-10-21T17:40:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.220+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14080c3620 con 0x7f14080c90b0 2023-10-21T17:40:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.221+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f14040113f0 con 0x7f14080c90b0 2023-10-21T17:40:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.222+0000 7f13fd7fa640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 0x7f13f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.222+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f140408a2b0 con 0x7f14080c90b0 2023-10-21T17:40:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.223+0000 7f13fffff640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 0x7f13f406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.224+0000 7f13fffff640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 0x7f13f406d570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036000 tx=0x7f13f803b040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.226+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14080c3620 con 0x7f14080c90b0 2023-10-21T17:40:22.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.419+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14080c3620 con 0x7f14080c90b0 2023-10-21T17:40:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.422+0000 7f13fd7fa640 1 -- 172.21.15.53:0/1599485168 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f14080c3620 con 0x7f14080c90b0 2023-10-21T17:40:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:22.424 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":263,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:21.975193+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:6834/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 msgr2=0x7f13f406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 0x7f13f406d570 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13f8036000 tx=0x7f13f803b040 comp rx=0 tx=0).stop 2023-10-21T17:40:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 msgr2=0x7f1408158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f140400c7f0 tx=0x7f140400cbf0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 shutdown_connections 2023-10-21T17:40:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f13f406b0b0 0x7f13f406d570 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f140815ab40 0x7f1408155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14080cae70 0x7f1408153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 --2- 172.21.15.53:0/1599485168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14080c90b0 0x7f1408158130 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 >> 172.21.15.53:0/1599485168 conn(0x7f140806a380 msgr2=0x7f1408156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 shutdown_connections 2023-10-21T17:40:22.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.425+0000 7f140ed6c640 1 -- 172.21.15.53:0/1599485168 wait complete. 2023-10-21T17:40:22.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-21T17:39:52.278617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-21T17:39:52.975528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:40:22.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:40:22.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.667+0000 7f6f9759e640 1 Processor -- start 2023-10-21T17:40:22.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.667+0000 7f6f9759e640 1 -- start start 2023-10-21T17:40:22.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.668+0000 7f6f9759e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f980c1710 0x7f6f980c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.668+0000 7f6f9759e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f981622f0 con 0x7f6f981657e0 2023-10-21T17:40:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.669+0000 7f6f9759e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f98164d00 con 0x7f6f980c1af0 2023-10-21T17:40:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.669+0000 7f6f9759e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f981625d0 con 0x7f6f980c1710 2023-10-21T17:40:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.669+0000 7f6f96d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6f981657e0 0x7f6f9815e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36650/0 (socket says 172.21.15.53:36650) 2023-10-21T17:40:22.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.669+0000 7f6f96d9d640 1 -- 172.21.15.53:0/2114585329 learned_addr learned my addr 172.21.15.53:0/2114585329 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.669+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> v2:172.21.15.124:3300/0 conn(0x7f6f980c1710 0x7f6f980c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656155042 0 0) 0x7f6f981622f0 con 0x7f6f981657e0 2023-10-21T17:40:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f84003520 con 0x7f6f981657e0 2023-10-21T17:40:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3031249431 0 0) 0x7f6f84003520 con 0x7f6f981657e0 2023-10-21T17:40:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 >> v2:172.21.15.124:3300/0 conn(0x7f6f980c1710 msgr2=0x7f6f980c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 --2- 172.21.15.53:0/2114585329 >> v2:172.21.15.124:3300/0 conn(0x7f6f980c1710 0x7f6f980c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 >> v1:172.21.15.124:6789/0 conn(0x7f6f980c1af0 legacy=0x7f6f98161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98164fe0 con 0x7f6f981657e0 2023-10-21T17:40:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f6f900019f0 con 0x7f6f981657e0 2023-10-21T17:40:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> v2:172.21.15.124:3300/0 conn(0x7f6f980c1710 0x7f6f980c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:40:22.678 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 >> v1:172.21.15.53:6789/0 conn(0x7f6f981657e0 legacy=0x7f6f9815e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 0x7f6f84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f840047c0 0x7f6f84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f84005180 0x7f6f8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981625d0 con 0x7f6f84003970 2023-10-21T17:40:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400c030 con 0x7f6f840047c0 2023-10-21T17:40:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8400c1f0 con 0x7f6f84005180 2023-10-21T17:40:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.670+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 0x7f6f84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f95d9b640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f840047c0 0x7f6f84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f9659c640 1 -- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f84005180 msgr2=0x7f6f8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f96d9d640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f84005180 0x7f6f8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f84005180 0x7f6f8400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f9659c640 1 -- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f840047c0 msgr2=0x7f6f84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f840047c0 0x7f6f84004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f9659c640 1 -- 172.21.15.53:0/2114585329 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f84003520 con 0x7f6f84003970 2023-10-21T17:40:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.671+0000 7f6f96d9d640 1 -- 172.21.15.53:0/2114585329 reap_dead start 2023-10-21T17:40:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.672+0000 7f6f9659c640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 0x7f6f84003d90 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009c90 tx=0x7f6f840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b2d78181fce71c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.672+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8802c8f0 con 0x7f6f84003970 2023-10-21T17:40:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.672+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8802ca50 con 0x7f6f84003970 2023-10-21T17:40:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.672+0000 7f6f9559a640 1 -- 172.21.15.53:0/2114585329 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8802ce40 con 0x7f6f84003970 2023-10-21T17:40:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 -- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 msgr2=0x7f6f84003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 0x7f6f84003d90 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009c90 tx=0x7f6f840057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 -- 172.21.15.53:0/2114585329 shutdown_connections 2023-10-21T17:40:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 --2- 172.21.15.53:0/2114585329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f84003970 0x7f6f84003d90 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 -- 172.21.15.53:0/2114585329 >> 172.21.15.53:0/2114585329 conn(0x7f6f9806c010 msgr2=0x7f6f9806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 -- 172.21.15.53:0/2114585329 shutdown_connections 2023-10-21T17:40:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.673+0000 7f6f9759e640 1 -- 172.21.15.53:0/2114585329 wait complete. 2023-10-21T17:40:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.675+0000 7f6f9759e640 1 Processor -- start 2023-10-21T17:40:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.675+0000 7f6f9759e640 1 -- start start 2023-10-21T17:40:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f84003970 0x7f6f98155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f980b1100 0x7f6f9815df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f98156030 con 0x7f6f9815e450 2023-10-21T17:40:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9815e8d0 con 0x7f6f84003970 2023-10-21T17:40:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.676+0000 7f6f9759e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980c3600 con 0x7f6f980b1100 2023-10-21T17:40:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.688 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45676/0 (socket says 172.21.15.53:45676) 2023-10-21T17:40:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 -- 172.21.15.53:0/1108695987 learned_addr learned my addr 172.21.15.53:0/1108695987 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f95d9b640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f980b1100 0x7f6f9815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f9659c640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f84003970 0x7f6f98155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 -- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f980b1100 msgr2=0x7f6f9815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.689 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f980b1100 0x7f6f9815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 -- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f84003970 msgr2=0x7f6f98155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f84003970 0x7f6f98155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 -- 172.21.15.53:0/1108695987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f98070d60 con 0x7f6f9815e450 2023-10-21T17:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.677+0000 7f6f96d9d640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f6f900012e0 tx=0x7f6f9000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.690 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.678+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000de90 con 0x7f6f9815e450 2023-10-21T17:40:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.678+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000bb20 con 0x7f6f9815e450 2023-10-21T17:40:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.678+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000bed0 con 0x7f6f9815e450 2023-10-21T17:40:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.678+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f98071040 con 0x7f6f9815e450 2023-10-21T17:40:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.678+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f980c2f90 con 0x7f6f9815e450 2023-10-21T17:40:22.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.679+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f98074150 con 0x7f6f9815e450 2023-10-21T17:40:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.680+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6f90007030 con 0x7f6f9815e450 2023-10-21T17:40:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.681+0000 7f6f8effd640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 0x7f6f7806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.681+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f6f90089b50 con 0x7f6f9815e450 2023-10-21T17:40:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.682+0000 7f6f9659c640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 0x7f6f7806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:22.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.683+0000 7f6f9659c640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 0x7f6f7806d5a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6f88036000 tx=0x7f6f88030000 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:22.693 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.685+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f90054180 con 0x7f6f9815e450 2023-10-21T17:40:22.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.877+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f980c3740 con 0x7f6f9815e450 2023-10-21T17:40:22.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.878+0000 7f6f8effd640 1 -- 172.21.15.53:0/1108695987 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v263) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f900577a0 con 0x7f6f9815e450 2023-10-21T17:40:22.881 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:40:22.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 msgr2=0x7f6f7806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 0x7f6f7806d5a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6f88036000 tx=0x7f6f88030000 comp rx=0 tx=0).stop 2023-10-21T17:40:22.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 msgr2=0x7f6f98070070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:22.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f6f900012e0 tx=0x7f6f9000b4a0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 shutdown_connections 2023-10-21T17:40:22.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f7806b0e0 0x7f6f7806d5a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f9815e450 0x7f6f98070070 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f980b1100 0x7f6f9815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 --2- 172.21.15.53:0/1108695987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f84003970 0x7f6f98155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:22.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.880+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 >> 172.21.15.53:0/1108695987 conn(0x7f6f9806c010 msgr2=0x7f6f981613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:22.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.881+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 shutdown_connections 2023-10-21T17:40:22.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:22.881+0000 7f6f9759e640 1 -- 172.21.15.53:0/1108695987 wait complete. 2023-10-21T17:40:22.902 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-21T17:40:23.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.136+0000 7f6cd125d640 1 Processor -- start 2023-10-21T17:40:23.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.136+0000 7f6cd125d640 1 -- start start 2023-10-21T17:40:23.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cd125d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ccc06c3a0 0x7f6ccc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cd125d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6ccc15aa00 0x7f6ccc0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cd125d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc15b310 con 0x7f6ccc06db60 2023-10-21T17:40:23.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cd125d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc0ca230 con 0x7f6ccc15aa00 2023-10-21T17:40:23.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cd125d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc06d290 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6ccb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6ccc15aa00 0x7f6ccc0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cca59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6ccc06db60 0x7f6ccc06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36652/0 (socket says 172.21.15.53:36652) 2023-10-21T17:40:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6cca59c640 1 -- 172.21.15.53:0/29928634 learned_addr learned my addr 172.21.15.53:0/29928634 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.138+0000 7f6ccad9d640 1 --2- 172.21.15.53:0/29928634 >> v2:172.21.15.124:3300/0 conn(0x7f6ccc06c3a0 0x7f6ccc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.142 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.139+0000 7f6ccb59e640 1 -- 172.21.15.53:0/29928634 >> v2:172.21.15.124:3300/0 conn(0x7f6ccc06c3a0 msgr2=0x7f6ccc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:23.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.139+0000 7f6ccb59e640 1 --2- 172.21.15.53:0/29928634 >> v2:172.21.15.124:3300/0 conn(0x7f6ccc06c3a0 0x7f6ccc0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.139+0000 7f6ccb59e640 1 -- 172.21.15.53:0/29928634 >> v1:172.21.15.53:6789/0 conn(0x7f6ccc06db60 legacy=0x7f6ccc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.143 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.139+0000 7f6ccb59e640 1 -- 172.21.15.53:0/29928634 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0c9860 con 0x7f6ccc15aa00 2023-10-21T17:40:23.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.140+0000 7f6ccb59e640 1 --2- 172.21.15.53:0/29928634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc15aa00 0x7f6ccc0b2a90 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0716b0 tx=0x7f6cc402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b2a4b54df04e34 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.144 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.140+0000 7f6cc9d9b640 1 -- 172.21.15.53:0/29928634 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc402ee80 con 0x7f6ccc15aa00 2023-10-21T17:40:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.140+0000 7f6cc9d9b640 1 -- 172.21.15.53:0/29928634 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4036440 con 0x7f6ccc15aa00 2023-10-21T17:40:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.140+0000 7f6cc9d9b640 1 -- 172.21.15.53:0/29928634 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40367b0 con 0x7f6ccc15aa00 2023-10-21T17:40:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 -- 172.21.15.53:0/29928634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc15aa00 msgr2=0x7f6ccc0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 --2- 172.21.15.53:0/29928634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc15aa00 0x7f6ccc0b2a90 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0716b0 tx=0x7f6cc402e0d0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.145 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 -- 172.21.15.53:0/29928634 shutdown_connections 2023-10-21T17:40:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 --2- 172.21.15.53:0/29928634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc15aa00 0x7f6ccc0b2a90 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 --2- 172.21.15.53:0/29928634 >> v2:172.21.15.124:3300/0 conn(0x7f6ccc06c3a0 0x7f6ccc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 -- 172.21.15.53:0/29928634 >> 172.21.15.53:0/29928634 conn(0x7f6ccc06a5d0 msgr2=0x7f6ccc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 -- 172.21.15.53:0/29928634 shutdown_connections 2023-10-21T17:40:23.146 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.141+0000 7f6cd125d640 1 -- 172.21.15.53:0/29928634 wait complete. 2023-10-21T17:40:23.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.143+0000 7f6cd125d640 1 Processor -- start 2023-10-21T17:40:23.148 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.143+0000 7f6cd125d640 1 -- start start 2023-10-21T17:40:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ccc06db60 0x7f6ccc0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ccc15aa00 0x7f6ccc0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc1e4830 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c8150 con 0x7f6ccc06db60 2023-10-21T17:40:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6cd125d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c8430 con 0x7f6ccc15aa00 2023-10-21T17:40:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6ccad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45686/0 (socket says 172.21.15.53:45686) 2023-10-21T17:40:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.144+0000 7f6ccad9d640 1 -- 172.21.15.53:0/2130011149 learned_addr learned my addr 172.21.15.53:0/2130011149 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:23.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 -- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ccc15aa00 msgr2=0x7f6ccc0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ccc15aa00 0x7f6ccc0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 -- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ccc06db60 msgr2=0x7f6ccc0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ccc06db60 0x7f6ccc0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 -- 172.21.15.53:0/2130011149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009a70 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6ccad9d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00d800 tx=0x7f6cbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00b6c0 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00b820 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00bbb0 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.145+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc1e4a10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.146+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc0c7d80 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.147+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc0c7f10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.148+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6cbc00bd10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.149+0000 7f6cb37fe640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 0x7f6cb806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.149+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f6cbc08a550 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.150+0000 7f6cca59c640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 0x7f6cb806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.151+0000 7f6cca59c640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 0x7f6cb806d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0cb050 tx=0x7f6cc0007a40 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.153+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ccc0c7f10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.344+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ccc0c7f10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.345+0000 7f6cb37fe640 1 -- 172.21.15.53:0/2130011149 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v263) v1 ==== 107+0+11 (secure 0 0 0) 0x7f6ccc0c7f10 con 0x7f6ccc06c3a0 2023-10-21T17:40:23.348 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:40:23.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 msgr2=0x7f6cb806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 0x7f6cb806d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0cb050 tx=0x7f6cc0007a40 comp rx=0 tx=0).stop 2023-10-21T17:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 msgr2=0x7f6ccc0c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00d800 tx=0x7f6cbc00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 shutdown_connections 2023-10-21T17:40:23.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6cb806b0b0 0x7f6cb806d570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ccc15aa00 0x7f6ccc0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ccc06db60 0x7f6ccc0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 --2- 172.21.15.53:0/2130011149 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ccc06c3a0 0x7f6ccc0c7840 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 >> 172.21.15.53:0/2130011149 conn(0x7f6ccc06a5d0 msgr2=0x7f6ccc0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:23.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 shutdown_connections 2023-10-21T17:40:23.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.348+0000 7f6cd125d640 1 -- 172.21.15.53:0/2130011149 wait complete. 2023-10-21T17:40:23.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-21T17:40:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.597+0000 7f9d7e24d640 1 Processor -- start 2023-10-21T17:40:23.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.598+0000 7f9d7e24d640 1 -- start start 2023-10-21T17:40:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7e24d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d780d00e0 0x7f9d780d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7e24d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 0x7f9d780b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7e24d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780d2870 con 0x7f9d7815ccb0 2023-10-21T17:40:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7e24d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d780d1150 con 0x7f9d780d00e0 2023-10-21T17:40:23.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7e24d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780d1430 con 0x7f9d780d0b90 2023-10-21T17:40:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.599+0000 7f9d7ca4a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d780d00e0 0x7f9d780d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9d780d00e0 0x7f9d780d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45696/0 (socket says 172.21.15.53:45696) 2023-10-21T17:40:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 -- 172.21.15.53:0/1722413359 learned_addr learned my addr 172.21.15.53:0/1722413359 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:23.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7d24b640 1 --1- 172.21.15.53:0/1722413359 >> v1:172.21.15.53:6789/0 conn(0x7f9d7815ccb0 0x7f9d780cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36668/0 (socket says 172.21.15.53:36668) 2023-10-21T17:40:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7da4c640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 0x7f9d780b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 -- 172.21.15.53:0/1722413359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 msgr2=0x7f9d780b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 0x7f9d780b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 -- 172.21.15.53:0/1722413359 >> v1:172.21.15.53:6789/0 conn(0x7f9d7815ccb0 legacy=0x7f9d780cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d77fff640 1 -- 172.21.15.53:0/1722413359 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093726542 0 0) 0x7f9d780d2870 con 0x7f9d7815ccb0 2023-10-21T17:40:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7ca4a640 1 -- 172.21.15.53:0/1722413359 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780d1710 con 0x7f9d780d00e0 2023-10-21T17:40:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.600+0000 7f9d7da4c640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 0x7f9d780b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.601+0000 7f9d7ca4a640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d780d00e0 0x7f9d780d1fb0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009870 tx=0x7f9d6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6546f48715a57427 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.602+0000 7f9d77fff640 1 -- 172.21.15.53:0/1722413359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64030840 con 0x7f9d780d00e0 2023-10-21T17:40:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.602+0000 7f9d77fff640 1 -- 172.21.15.53:0/1722413359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d640309a0 con 0x7f9d780d00e0 2023-10-21T17:40:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.602+0000 7f9d77fff640 1 -- 172.21.15.53:0/1722413359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64030d30 con 0x7f9d780d00e0 2023-10-21T17:40:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.602+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1722413359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d780d00e0 msgr2=0x7f9d780d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.602+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d780d00e0 0x7f9d780d1fb0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009870 tx=0x7f9d6402b980 comp rx=0 tx=0).stop 2023-10-21T17:40:23.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1722413359 shutdown_connections 2023-10-21T17:40:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d0b90 0x7f9d780b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1722413359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d780d00e0 0x7f9d780d1fb0 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1722413359 >> 172.21.15.53:0/1722413359 conn(0x7f9d78071660 msgr2=0x7f9d78071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1722413359 shutdown_connections 2023-10-21T17:40:23.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.603+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1722413359 wait complete. 2023-10-21T17:40:23.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.604+0000 7f9d7e24d640 1 Processor -- start 2023-10-21T17:40:23.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.605+0000 7f9d7e24d640 1 -- start start 2023-10-21T17:40:23.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d00e0 0x7f9d78158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d780d0b90 0x7f9d7815f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780c7cd0 con 0x7f9d7815ccb0 2023-10-21T17:40:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780c7fb0 con 0x7f9d780d0b90 2023-10-21T17:40:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7e24d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7815eb50 con 0x7f9d780d00e0 2023-10-21T17:40:23.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:45700/0 (socket says 172.21.15.53:45700) 2023-10-21T17:40:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 -- 172.21.15.53:0/1221947814 learned_addr learned my addr 172.21.15.53:0/1221947814 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7d24b640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d00e0 0x7f9d78158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7ca4a640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d780d0b90 0x7f9d7815f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 -- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d00e0 msgr2=0x7f9d78158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d00e0 0x7f9d78158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 -- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d780d0b90 msgr2=0x7f9d7815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d780d0b90 0x7f9d7815f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.606+0000 7f9d7da4c640 1 -- 172.21.15.53:0/1221947814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640094d0 con 0x7f9d7815ccb0 2023-10-21T17:40:23.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d7da4c640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c5e0 tx=0x7f9d7000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70011800 con 0x7f9d7815ccb0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d700119a0 con 0x7f9d7815ccb0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70011d50 con 0x7f9d7815ccb0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7815ee30 con 0x7f9d7815ccb0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.607+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d78159490 con 0x7f9d7815ccb0 2023-10-21T17:40:23.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.609+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7815f350 con 0x7f9d7815ccb0 2023-10-21T17:40:23.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.610+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9d70022350 con 0x7f9d7815ccb0 2023-10-21T17:40:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.611+0000 7f9d75ffb640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 0x7f9d5c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.611+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f9d7008a180 con 0x7f9d7815ccb0 2023-10-21T17:40:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.612+0000 7f9d7d24b640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 0x7f9d5c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:23.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.613+0000 7f9d7d24b640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 0x7f9d5c06d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0012e0 tx=0x7f9d6c005eb0 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:23.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.616+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d7815f350 con 0x7f9d7815ccb0 2023-10-21T17:40:23.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.808+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d7815f350 con 0x7f9d7815ccb0 2023-10-21T17:40:23.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.809+0000 7f9d75ffb640 1 -- 172.21.15.53:0/1221947814 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v263) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d7815f350 con 0x7f9d7815ccb0 2023-10-21T17:40:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:40:23.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 msgr2=0x7f9d5c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 0x7f9d5c06d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0012e0 tx=0x7f9d6c005eb0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 msgr2=0x7f9d7806b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c5e0 tx=0x7f9d7000caa0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 shutdown_connections 2023-10-21T17:40:23.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f9d5c06b0b0 0x7f9d5c06d570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d7815ccb0 0x7f9d7806b560 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d780d0b90 0x7f9d7815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.812+0000 7f9d7e24d640 1 --2- 172.21.15.53:0/1221947814 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d780d00e0 0x7f9d78158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:23.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.813+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 >> 172.21.15.53:0/1221947814 conn(0x7f9d78071660 msgr2=0x7f9d78071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:23.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.813+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 shutdown_connections 2023-10-21T17:40:23.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:23.813+0000 7f9d7e24d640 1 -- 172.21.15.53:0/1221947814 wait complete. 2023-10-21T17:40:23.834 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:24.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.056+0000 7fd87499e640 1 Processor -- start 2023-10-21T17:40:24.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.056+0000 7fd87499e640 1 -- start start 2023-10-21T17:40:24.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.058+0000 7fd87499e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 0x7fd87006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.058+0000 7fd87499e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd8700ad790 0x7fd8700702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.058+0000 7fd87499e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87006dee0 con 0x7fd87015e330 2023-10-21T17:40:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.058+0000 7fd87499e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87015d850 con 0x7fd8700ad790 2023-10-21T17:40:24.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.058+0000 7fd87499e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87015db30 con 0x7fd870159ac0 2023-10-21T17:40:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.059+0000 7fd86e59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 0x7fd87006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.059+0000 7fd865d9b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd8700ad790 0x7fd8700702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.059+0000 7fd86e59c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 0x7fd87006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:54714/0 (socket says 172.21.15.53:54714) 2023-10-21T17:40:24.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.059+0000 7fd86e59c640 1 -- 172.21.15.53:0/3124258111 learned_addr learned my addr 172.21.15.53:0/3124258111 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.060+0000 7fd865d9b640 1 -- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 msgr2=0x7fd87006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.060+0000 7fd865d9b640 1 --2- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 0x7fd87006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.060+0000 7fd865d9b640 1 -- 172.21.15.53:0/3124258111 >> v1:172.21.15.124:6789/0 conn(0x7fd87015e330 legacy=0x7fd87006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.060+0000 7fd865d9b640 1 -- 172.21.15.53:0/3124258111 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87015de10 con 0x7fd8700ad790 2023-10-21T17:40:24.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.061+0000 7fd865d9b640 1 --2- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8700ad790 0x7fd8700702d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd8580097a0 tx=0x7fd85802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4dddfefb1bd0008 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.061+0000 7fd86dd9b640 1 -- 172.21.15.53:0/3124258111 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858031830 con 0x7fd8700ad790 2023-10-21T17:40:24.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd86dd9b640 1 -- 172.21.15.53:0/3124258111 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858031990 con 0x7fd8700ad790 2023-10-21T17:40:24.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd86dd9b640 1 -- 172.21.15.53:0/3124258111 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858031d20 con 0x7fd8700ad790 2023-10-21T17:40:24.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 -- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8700ad790 msgr2=0x7fd8700702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.065 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 --2- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8700ad790 0x7fd8700702d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd8580097a0 tx=0x7fd85802b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 -- 172.21.15.53:0/3124258111 shutdown_connections 2023-10-21T17:40:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 --2- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8700ad790 0x7fd8700702d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 --2- 172.21.15.53:0/3124258111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd870159ac0 0x7fd87006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 -- 172.21.15.53:0/3124258111 >> 172.21.15.53:0/3124258111 conn(0x7fd87006a380 msgr2=0x7fd8700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:24.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 -- 172.21.15.53:0/3124258111 shutdown_connections 2023-10-21T17:40:24.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.062+0000 7fd87499e640 1 -- 172.21.15.53:0/3124258111 wait complete. 2023-10-21T17:40:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.064+0000 7fd87499e640 1 Processor -- start 2023-10-21T17:40:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.064+0000 7fd87499e640 1 -- start start 2023-10-21T17:40:24.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8700ad790 0x7fd87014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd870159ac0 0x7fd870154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87014c120 con 0x7fd87015e330 2023-10-21T17:40:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd870155a20 con 0x7fd870159ac0 2023-10-21T17:40:24.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd87499e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd870155740 con 0x7fd8700ad790 2023-10-21T17:40:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd86ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd86ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46210/0 (socket says 172.21.15.53:46210) 2023-10-21T17:40:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.065+0000 7fd86ed9d640 1 -- 172.21.15.53:0/1386966378 learned_addr learned my addr 172.21.15.53:0/1386966378 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd865d9b640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd870159ac0 0x7fd870154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 -- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8700ad790 msgr2=0x7fd87014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86e59c640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8700ad790 0x7fd87014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8700ad790 0x7fd87014bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 -- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd870159ac0 msgr2=0x7fd870154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd870159ac0 0x7fd870154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 -- 172.21.15.53:0/1386966378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd858009400 con 0x7fd87015e330 2023-10-21T17:40:24.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd86ed9d640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c6b0 tx=0x7fd86800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.066+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868003b70 con 0x7fd87015e330 2023-10-21T17:40:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.067+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868003cf0 con 0x7fd87015e330 2023-10-21T17:40:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.067+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8701500c0 con 0x7fd87015e330 2023-10-21T17:40:24.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.067+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868003ea0 con 0x7fd87015e330 2023-10-21T17:40:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.067+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870150680 con 0x7fd87015e330 2023-10-21T17:40:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.068+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870067490 con 0x7fd87015e330 2023-10-21T17:40:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.069+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd868005310 con 0x7fd87015e330 2023-10-21T17:40:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.070+0000 7fd8677fe640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 0x7fd85006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.070+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fd868089d60 con 0x7fd87015e330 2023-10-21T17:40:24.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.071+0000 7fd86e59c640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 0x7fd85006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.072+0000 7fd86e59c640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 0x7fd85006d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bc20 tx=0x7fd860008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.075+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd868054540 con 0x7fd87015e330 2023-10-21T17:40:24.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.305+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8701538f0 con 0x7fd87015e330 2023-10-21T17:40:24.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.307+0000 7fd8677fe640 1 -- 172.21.15.53:0/1386966378 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4936 (secure 0 0 0) 0x7fd868057b60 con 0x7fd87015e330 2023-10-21T17:40:24.309 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:24.309 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":136,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2097440682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2097440682},{"type":"v1","addr":"172.21.15.124:6836","nonce":2097440682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":136},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:24.310 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 136 2023-10-21T17:40:24.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 msgr2=0x7fd85006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 0x7fd85006d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd86000bc20 tx=0x7fd860008040 comp rx=0 tx=0).stop 2023-10-21T17:40:24.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 msgr2=0x7fd870158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c6b0 tx=0x7fd86800cb70 comp rx=0 tx=0).stop 2023-10-21T17:40:24.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 shutdown_connections 2023-10-21T17:40:24.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd85006b0b0 0x7fd85006d570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd87015e330 0x7fd870158430 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd870159ac0 0x7fd870154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 --2- 172.21.15.53:0/1386966378 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8700ad790 0x7fd87014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.309+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 >> 172.21.15.53:0/1386966378 conn(0x7fd87006a380 msgr2=0x7fd8700ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:24.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.310+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 shutdown_connections 2023-10-21T17:40:24.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.310+0000 7fd87499e640 1 -- 172.21.15.53:0/1386966378 wait complete. 2023-10-21T17:40:24.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:24.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.548+0000 7f222407c640 1 Processor -- start 2023-10-21T17:40:24.551 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.548+0000 7f222407c640 1 -- start start 2023-10-21T17:40:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f222407c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f221c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f222407c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221c0c48f0 con 0x7f221c06c890 2023-10-21T17:40:24.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f222407c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221c0c3ce0 con 0x7f221c160940 2023-10-21T17:40:24.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f222407c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0c4d80 con 0x7f221c15cd90 2023-10-21T17:40:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f2222619640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f221c06c890 0x7f221c06cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37098/0 (socket says 172.21.15.53:37098) 2023-10-21T17:40:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.550+0000 7f2222619640 1 -- 172.21.15.53:0/2708136559 learned_addr learned my addr 172.21.15.53:0/2708136559 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2221e18640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f221c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800916374 0 0) 0x7f221c0c48f0 con 0x7f221c06c890 2023-10-21T17:40:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2208003520 con 0x7f221c06c890 2023-10-21T17:40:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3870269737 0 0) 0x7f2208003520 con 0x7f221c06c890 2023-10-21T17:40:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 msgr2=0x7f221c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.551+0000 7f2220e16640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f221c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 >> v1:172.21.15.124:6789/0 conn(0x7f221c160940 legacy=0x7f221c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c0c3fc0 con 0x7f221c06c890 2023-10-21T17:40:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2214001670 con 0x7f221c06c890 2023-10-21T17:40:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 >> v1:172.21.15.53:6789/0 conn(0x7f221c06c890 legacy=0x7f221c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22080039a0 0x7f2208003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f22080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f2208005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220800b980 con 0x7f2208004d00 2023-10-21T17:40:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220800bac0 con 0x7f22080039a0 2023-10-21T17:40:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220800bc80 con 0x7f221c15cd90 2023-10-21T17:40:24.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2222619640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f2208005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2221e18640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22080039a0 0x7f2208003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2221617640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f22080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.552+0000 7f2222619640 1 -- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 msgr2=0x7f22080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2222619640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f22080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2222619640 1 -- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22080039a0 msgr2=0x7f2208003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:24.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2222619640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22080039a0 0x7f2208003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2222619640 1 -- 172.21.15.53:0/2708136559 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208003520 con 0x7f2208004d00 2023-10-21T17:40:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2221e18640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22080039a0 0x7f2208003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:24.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.553+0000 7f2221617640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c15cd90 0x7f22080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f2222619640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f2208005180 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f221400c240 tx=0x7f22080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518d218a4af6ebc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214005ad0 con 0x7f2208004d00 2023-10-21T17:40:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2214005c30 con 0x7f2208004d00 2023-10-21T17:40:24.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f2220e16640 1 -- 172.21.15.53:0/2708136559 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221402c8a0 con 0x7f2208004d00 2023-10-21T17:40:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f222407c640 1 -- 172.21.15.53:0/2708136559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 msgr2=0x7f2208005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f222407c640 1 --2- 172.21.15.53:0/2708136559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f2208005180 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f221400c240 tx=0x7f22080057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.554+0000 7f2222619640 1 -- 172.21.15.53:0/2708136559 reap_dead start 2023-10-21T17:40:24.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.555+0000 7f222407c640 1 -- 172.21.15.53:0/2708136559 shutdown_connections 2023-10-21T17:40:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.555+0000 7f222407c640 1 -- 172.21.15.53:0/2708136559 >> 172.21.15.53:0/2708136559 conn(0x7f221c06a380 msgr2=0x7f221c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.555+0000 7f222407c640 1 -- 172.21.15.53:0/2708136559 shutdown_connections 2023-10-21T17:40:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.555+0000 7f222407c640 1 -- 172.21.15.53:0/2708136559 wait complete. 2023-10-21T17:40:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.556+0000 7f222407c640 1 Processor -- start 2023-10-21T17:40:24.565 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.557+0000 7f222407c640 1 -- start start 2023-10-21T17:40:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c0c9f30 0x7f221c15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f221c0ca3b0 0x7f221c15fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c15e660 con 0x7f2208004d00 2023-10-21T17:40:24.566 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c15dcc0 con 0x7f221c0ca3b0 2023-10-21T17:40:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f222407c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c15dfa0 con 0x7f221c0c9f30 2023-10-21T17:40:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46236/0 (socket says 172.21.15.53:46236) 2023-10-21T17:40:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 -- 172.21.15.53:0/3602858408 learned_addr learned my addr 172.21.15.53:0/3602858408 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:24.567 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2222619640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f221c0ca3b0 0x7f221c15fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 -- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c0c9f30 msgr2=0x7f221c15f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c0c9f30 0x7f221c15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 -- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f221c0ca3b0 msgr2=0x7f221c15fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f221c0ca3b0 0x7f221c15fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.568 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.558+0000 7f2221e18640 1 -- 172.21.15.53:0/3602858408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c15e280 con 0x7f2208004d00 2023-10-21T17:40:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f2221e18640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f221800f350 tx=0x7f221800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218003b90 con 0x7f2208004d00 2023-10-21T17:40:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2218011040 con 0x7f2208004d00 2023-10-21T17:40:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218004020 con 0x7f2208004d00 2023-10-21T17:40:24.569 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221c152820 con 0x7f2208004d00 2023-10-21T17:40:24.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.559+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c15ea80 con 0x7f2208004d00 2023-10-21T17:40:24.570 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.560+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221c15e7a0 con 0x7f2208004d00 2023-10-21T17:40:24.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.561+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2218004180 con 0x7f2208004d00 2023-10-21T17:40:24.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.563+0000 7f22127fc640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 0x7f220006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:24.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.563+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f221808a7f0 con 0x7f2208004d00 2023-10-21T17:40:24.571 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.563+0000 7f2221617640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 0x7f220006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.565+0000 7f2221617640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 0x7f220006d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f221c0cba40 tx=0x7f220c009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:24.572 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.566+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f221c15e7a0 con 0x7f2208004d00 2023-10-21T17:40:24.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.759+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f221c15e7a0 con 0x7f2208004d00 2023-10-21T17:40:24.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.762+0000 7f22127fc640 1 -- 172.21.15.53:0/3602858408 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18305 (secure 0 0 0) 0x7f221c15e7a0 con 0x7f2208004d00 2023-10-21T17:40:24.765 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:24.765 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":263,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:21.975193+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:6834/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 msgr2=0x7f220006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 0x7f220006d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f221c0cba40 tx=0x7f220c009040 comp rx=0 tx=0).stop 2023-10-21T17:40:24.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 msgr2=0x7f221c1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f221800f350 tx=0x7f221800f810 comp rx=0 tx=0).stop 2023-10-21T17:40:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 shutdown_connections 2023-10-21T17:40:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f220006b0e0 0x7f220006d5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f221c0ca3b0 0x7f221c15fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f221c0c9f30 0x7f221c15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 --2- 172.21.15.53:0/3602858408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2208004d00 0x7f221c1522e0 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 >> 172.21.15.53:0/3602858408 conn(0x7f221c06a380 msgr2=0x7f221c0af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 shutdown_connections 2023-10-21T17:40:24.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:24.765+0000 7f222407c640 1 -- 172.21.15.53:0/3602858408 wait complete. 2023-10-21T17:40:24.788 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-21T17:40:24.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.011+0000 7fbf77fff640 1 Processor -- start 2023-10-21T17:40:25.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.012+0000 7fbf77fff640 1 -- start start 2023-10-21T17:40:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.013+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.013+0000 7fbf77fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf780b07d0 0x7fbf7806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.013+0000 7fbf77fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7815f7d0 con 0x7fbf780adc70 2023-10-21T17:40:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.013+0000 7fbf77fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7815f210 con 0x7fbf780b07d0 2023-10-21T17:40:25.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.014+0000 7fbf77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7815f4f0 con 0x7fbf7815ab30 2023-10-21T17:40:25.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.014+0000 7fbf777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf780b07d0 0x7fbf7806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.014+0000 7fbf777fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbf780b07d0 0x7fbf7806cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46248/0 (socket says 172.21.15.53:46248) 2023-10-21T17:40:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.014+0000 7fbf777fe640 1 -- 172.21.15.53:0/934836349 learned_addr learned my addr 172.21.15.53:0/934836349 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.014+0000 7fbf76ffd640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.015+0000 7fbf777fe640 1 -- 172.21.15.53:0/934836349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 msgr2=0x7fbf7815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:25.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.015+0000 7fbf777fe640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.015+0000 7fbf777fe640 1 -- 172.21.15.53:0/934836349 >> v1:172.21.15.53:6789/0 conn(0x7fbf780adc70 legacy=0x7fbf780b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.015+0000 7fbf777fe640 1 -- 172.21.15.53:0/934836349 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7806c550 con 0x7fbf780b07d0 2023-10-21T17:40:25.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.017+0000 7fbf777fe640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780b07d0 0x7fbf7806cd50 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009e50 tx=0x7fbf7002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c6fcd3cb13ef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.017+0000 7fbf75ffb640 1 -- 172.21.15.53:0/934836349 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7002cb00 con 0x7fbf780b07d0 2023-10-21T17:40:25.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.017+0000 7fbf75ffb640 1 -- 172.21.15.53:0/934836349 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7002cc60 con 0x7fbf780b07d0 2023-10-21T17:40:25.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.017+0000 7fbf75ffb640 1 -- 172.21.15.53:0/934836349 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700343f0 con 0x7fbf780b07d0 2023-10-21T17:40:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 -- 172.21.15.53:0/934836349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780b07d0 msgr2=0x7fbf7806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780b07d0 0x7fbf7806cd50 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009e50 tx=0x7fbf7002bf60 comp rx=0 tx=0).stop 2023-10-21T17:40:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 -- 172.21.15.53:0/934836349 shutdown_connections 2023-10-21T17:40:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780b07d0 0x7fbf7806cd50 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 --2- 172.21.15.53:0/934836349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 -- 172.21.15.53:0/934836349 >> 172.21.15.53:0/934836349 conn(0x7fbf780cb0c0 msgr2=0x7fbf780cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:25.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 -- 172.21.15.53:0/934836349 shutdown_connections 2023-10-21T17:40:25.022 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.018+0000 7fbf77fff640 1 -- 172.21.15.53:0/934836349 wait complete. 2023-10-21T17:40:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.020+0000 7fbf77fff640 1 Processor -- start 2023-10-21T17:40:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.020+0000 7fbf77fff640 1 -- start start 2023-10-21T17:40:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.023 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 0x7fbf7815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780c6f90 con 0x7fbf780adc70 2023-10-21T17:40:25.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7815c360 con 0x7fbf780b07d0 2023-10-21T17:40:25.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf77fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7815ddb0 con 0x7fbf7815ab30 2023-10-21T17:40:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf76ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46252/0 (socket says 172.21.15.53:46252) 2023-10-21T17:40:25.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.021+0000 7fbf76ffd640 1 -- 172.21.15.53:0/4000556613 learned_addr learned my addr 172.21.15.53:0/4000556613 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf767fc640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 0x7fbf7815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf777fe640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 -- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 msgr2=0x7fbf7815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 -- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 msgr2=0x7fbf7815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 0x7fbf7815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 -- 172.21.15.53:0/4000556613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70009b30 con 0x7fbf780adc70 2023-10-21T17:40:25.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf767fc640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 0x7fbf7815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:25.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf76ffd640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e870 tx=0x7fbf6800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.022+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68009510 con 0x7fbf780adc70 2023-10-21T17:40:25.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.023+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6800eec0 con 0x7fbf780adc70 2023-10-21T17:40:25.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.023+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800cb80 con 0x7fbf780adc70 2023-10-21T17:40:25.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.023+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf780c5df0 con 0x7fbf780adc70 2023-10-21T17:40:25.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.023+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf780c63b0 con 0x7fbf780adc70 2023-10-21T17:40:25.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.024+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf780c7100 con 0x7fbf780adc70 2023-10-21T17:40:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.025+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbf6800cd20 con 0x7fbf780adc70 2023-10-21T17:40:25.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.026+0000 7fbf5f7fe640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 0x7fbf6406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.026+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fbf6808a220 con 0x7fbf780adc70 2023-10-21T17:40:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.027+0000 7fbf767fc640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 0x7fbf6406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.028+0000 7fbf767fc640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 0x7fbf6406d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf7806e230 tx=0x7fbf6c009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.030+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf780c7100 con 0x7fbf780adc70 2023-10-21T17:40:25.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.263+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf780c70d0 con 0x7fbf780adc70 2023-10-21T17:40:25.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.265+0000 7fbf5f7fe640 1 -- 172.21.15.53:0/4000556613 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4936 (secure 0 0 0) 0x7fbf680548d0 con 0x7fbf780adc70 2023-10-21T17:40:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:25.267 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":136,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2097440682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2097440682},{"type":"v1","addr":"172.21.15.124:6836","nonce":2097440682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":136},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:25.268 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 136 2023-10-21T17:40:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.267+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 msgr2=0x7fbf6406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.267+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 0x7fbf6406d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf7806e230 tx=0x7fbf6c009040 comp rx=0 tx=0).stop 2023-10-21T17:40:25.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 msgr2=0x7fbf780c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e870 tx=0x7fbf6800ec70 comp rx=0 tx=0).stop 2023-10-21T17:40:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 shutdown_connections 2023-10-21T17:40:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fbf6406b0b0 0x7fbf6406d570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf7815ab30 0x7fbf7815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf780b07d0 0x7fbf7815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 --2- 172.21.15.53:0/4000556613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf780adc70 0x7fbf780c5710 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 >> 172.21.15.53:0/4000556613 conn(0x7fbf780cb0c0 msgr2=0x7fbf7806d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 shutdown_connections 2023-10-21T17:40:25.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.268+0000 7fbf77fff640 1 -- 172.21.15.53:0/4000556613 wait complete. 2023-10-21T17:40:25.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:25.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.512+0000 7fce80f7c640 1 Processor -- start 2023-10-21T17:40:25.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.512+0000 7fce80f7c640 1 -- start start 2023-10-21T17:40:25.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.513+0000 7fce80f7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cac00 0x7fce7c1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.513+0000 7fce80f7c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fce7c0cb450 0x7fce7c0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.513+0000 7fce80f7c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce7c0c9d50 0x7fce7c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.513+0000 7fce80f7c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c161850 con 0x7fce7c0c9d50 2023-10-21T17:40:25.516 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce80f7c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c160fb0 con 0x7fce7c0cb450 2023-10-21T17:40:25.517 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce80f7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c161290 con 0x7fce7c0cac00 2023-10-21T17:40:25.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce7c0c9d50 0x7fce7c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fce7c0c9d50 0x7fce7c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46256/0 (socket says 172.21.15.53:46256) 2023-10-21T17:40:25.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 -- 172.21.15.53:0/1955833116 learned_addr learned my addr 172.21.15.53:0/1955833116 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7a59c640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cac00 0x7fce7c1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce79d9b640 1 --2- 172.21.15.53:0/1955833116 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0cb450 0x7fce7c0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 -- 172.21.15.53:0/1955833116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cac00 msgr2=0x7fce7c1608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cac00 0x7fce7c1608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 -- 172.21.15.53:0/1955833116 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0cb450 msgr2=0x7fce7c0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:25.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 --2- 172.21.15.53:0/1955833116 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0cb450 0x7fce7c0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.514+0000 7fce7ad9d640 1 -- 172.21.15.53:0/1955833116 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c161570 con 0x7fce7c0c9d50 2023-10-21T17:40:25.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce7ad9d640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0c9d50 0x7fce7c06a410 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fce7c071cf0 tx=0x7fce7402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e89f37a6b79787b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce7959a640 1 -- 172.21.15.53:0/1955833116 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740317d0 con 0x7fce7c0c9d50 2023-10-21T17:40:25.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce7959a640 1 -- 172.21.15.53:0/1955833116 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74031930 con 0x7fce7c0c9d50 2023-10-21T17:40:25.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce7959a640 1 -- 172.21.15.53:0/1955833116 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74031cc0 con 0x7fce7c0c9d50 2023-10-21T17:40:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce80f7c640 1 -- 172.21.15.53:0/1955833116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0c9d50 msgr2=0x7fce7c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.516+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0c9d50 0x7fce7c06a410 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fce7c071cf0 tx=0x7fce7402b870 comp rx=0 tx=0).stop 2023-10-21T17:40:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 -- 172.21.15.53:0/1955833116 shutdown_connections 2023-10-21T17:40:25.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0c9d50 0x7fce7c06a410 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1955833116 >> v2:172.21.15.124:3300/0 conn(0x7fce7c0cb450 0x7fce7c0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1955833116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cac00 0x7fce7c1608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 -- 172.21.15.53:0/1955833116 >> 172.21.15.53:0/1955833116 conn(0x7fce7c0cf890 msgr2=0x7fce7c0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 -- 172.21.15.53:0/1955833116 shutdown_connections 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.517+0000 7fce80f7c640 1 -- 172.21.15.53:0/1955833116 wait complete. 2023-10-21T17:40:25.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.518+0000 7fce80f7c640 1 Processor -- start 2023-10-21T17:40:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.519+0000 7fce80f7c640 1 -- start start 2023-10-21T17:40:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 0x7fce7c1f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 0x7fce7c0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1f2e80 con 0x7fce7c0cac00 2023-10-21T17:40:25.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0b21e0 con 0x7fce7c0c9d50 2023-10-21T17:40:25.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce80f7c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1f28c0 con 0x7fce7c0cb450 2023-10-21T17:40:25.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46272/0 (socket says 172.21.15.53:46272) 2023-10-21T17:40:25.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 -- 172.21.15.53:0/1042943781 learned_addr learned my addr 172.21.15.53:0/1042943781 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:25.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce7a59c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 0x7fce7c1f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce7ad9d640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 0x7fce7c0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 -- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 msgr2=0x7fce7c0ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 0x7fce7c0ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 -- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 msgr2=0x7fce7c1f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 0x7fce7c1f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce79d9b640 1 -- 172.21.15.53:0/1042943781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740094d0 con 0x7fce7c0cac00 2023-10-21T17:40:25.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce7ad9d640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 0x7fce7c0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:25.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.520+0000 7fce7a59c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 0x7fce7c1f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:25.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce79d9b640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d800 tx=0x7fce6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400b6c0 con 0x7fce7c0cac00 2023-10-21T17:40:25.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6400b820 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6400bbb0 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1f2ba0 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.521+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c0c43f0 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.522+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c066d80 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.523+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fce6400bd10 con 0x7fce7c0cac00 2023-10-21T17:40:25.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.524+0000 7fce72ffd640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 0x7fce5c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:25.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.524+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fce6408a480 con 0x7fce7c0cac00 2023-10-21T17:40:25.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.525+0000 7fce7a59c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 0x7fce5c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:25.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.526+0000 7fce7a59c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 0x7fce5c06d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce7c1efa60 tx=0x7fce6c00b040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:25.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.528+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce64054cf0 con 0x7fce7c0cac00 2023-10-21T17:40:25.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.756+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce7c0b1c50 con 0x7fce7c0cac00 2023-10-21T17:40:25.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.757+0000 7fce72ffd640 1 -- 172.21.15.53:0/1042943781 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4936 (secure 0 0 0) 0x7fce64058310 con 0x7fce7c0cac00 2023-10-21T17:40:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:25.760 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":136,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2097440682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2097440682},{"type":"v1","addr":"172.21.15.124:6836","nonce":2097440682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":136},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:39:53.949575+0000","modified":"2023-10-21T17:39:57.022551+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38345},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38345":{"gid":38345,"name":"c","rank":0,"incarnation":130,"state":"up:active","state_seq":7,"addr":"172.21.15.53:6835/1218730184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":1218730184},{"type":"v1","addr":"172.21.15.53:6835","nonce":1218730184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-21T17:40:25.760 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 136 2023-10-21T17:40:25.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.759+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 msgr2=0x7fce5c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.759+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 0x7fce5c06d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce7c1efa60 tx=0x7fce6c00b040 comp rx=0 tx=0).stop 2023-10-21T17:40:25.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.759+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 msgr2=0x7fce7c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:25.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.759+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d800 tx=0x7fce6400dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 shutdown_connections 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fce5c06b0b0 0x7fce5c06d570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce7c0cb450 0x7fce7c0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce7c0cac00 0x7fce7c0ad8e0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 --2- 172.21.15.53:0/1042943781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce7c0c9d50 0x7fce7c1f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:25.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 >> 172.21.15.53:0/1042943781 conn(0x7fce7c0cf890 msgr2=0x7fce7c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 shutdown_connections 2023-10-21T17:40:25.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.760+0000 7fce80f7c640 1 -- 172.21.15.53:0/1042943781 wait complete. 2023-10-21T17:40:25.779 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:25.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.995+0000 7fd9f1773640 1 Processor -- start 2023-10-21T17:40:25.998 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.996+0000 7fd9f1773640 1 -- start start 2023-10-21T17:40:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9f1773640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9f1773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 0x7fd9ec0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9f1773640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec0cacf0 con 0x7fd9ec1599d0 2023-10-21T17:40:26.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9f1773640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b10f0 con 0x7fd9ec0cc210 2023-10-21T17:40:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9f1773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0ade40 con 0x7fd9ec159db0 2023-10-21T17:40:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9e3fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46274/0 (socket says 172.21.15.53:46274) 2023-10-21T17:40:26.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.997+0000 7fd9e3fff640 1 -- 172.21.15.53:0/3097249080 learned_addr learned my addr 172.21.15.53:0/3097249080 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9f0f72640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 0x7fd9ec0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9e3fff640 1 -- 172.21.15.53:0/3097249080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 msgr2=0x7fd9ec0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9e3fff640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 0x7fd9ec0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9e3fff640 1 -- 172.21.15.53:0/3097249080 >> v1:172.21.15.124:6789/0 conn(0x7fd9ec1599d0 legacy=0x7fd9ec0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:26.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9e3fff640 1 -- 172.21.15.53:0/3097249080 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0ca3c0 con 0x7fd9ec0cc210 2023-10-21T17:40:26.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.998+0000 7fd9f0f72640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 0x7fd9ec0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.999+0000 7fd9e3fff640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009870 tx=0x7fd9d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248bd3209d7551ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.999+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/3097249080 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4031640 con 0x7fd9ec0cc210 2023-10-21T17:40:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.999+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/3097249080 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d40317a0 con 0x7fd9ec0cc210 2023-10-21T17:40:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:25.999+0000 7fd9eb7fe640 1 -- 172.21.15.53:0/3097249080 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4031b30 con 0x7fd9ec0cc210 2023-10-21T17:40:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 -- 172.21.15.53:0/3097249080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec0cc210 msgr2=0x7fd9ec0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009870 tx=0x7fd9d402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 -- 172.21.15.53:0/3097249080 shutdown_connections 2023-10-21T17:40:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec159db0 0x7fd9ec0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3097249080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec0cc210 0x7fd9ec0b00c0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 -- 172.21.15.53:0/3097249080 >> 172.21.15.53:0/3097249080 conn(0x7fd9ec0cb0c0 msgr2=0x7fd9ec0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:26.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 -- 172.21.15.53:0/3097249080 shutdown_connections 2023-10-21T17:40:26.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.000+0000 7fd9f1773640 1 -- 172.21.15.53:0/3097249080 wait complete. 2023-10-21T17:40:26.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.002+0000 7fd9f1773640 1 Processor -- start 2023-10-21T17:40:26.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.002+0000 7fd9f1773640 1 -- start start 2023-10-21T17:40:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec0cc210 0x7fd9ec1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1599d0 0x7fd9ec15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0aed80 con 0x7fd9ec159db0 2023-10-21T17:40:26.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec156e00 con 0x7fd9ec1599d0 2023-10-21T17:40:26.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f1773640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0ae100 con 0x7fd9ec0cc210 2023-10-21T17:40:26.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f0f72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f0f72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46280/0 (socket says 172.21.15.53:46280) 2023-10-21T17:40:26.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.003+0000 7fd9f0f72640 1 -- 172.21.15.53:0/3222212462 learned_addr learned my addr 172.21.15.53:0/3222212462 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:26.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9ebfff640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec0cc210 0x7fd9ec1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9e3fff640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1599d0 0x7fd9ec15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 -- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec0cc210 msgr2=0x7fd9ec1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec0cc210 0x7fd9ec1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 -- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1599d0 msgr2=0x7fd9ec15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1599d0 0x7fd9ec15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 -- 172.21.15.53:0/3222212462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40094d0 con 0x7fd9ec159db0 2023-10-21T17:40:26.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.004+0000 7fd9f0f72640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400cd10 tx=0x7fd9e400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.005+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4003b90 con 0x7fd9ec159db0 2023-10-21T17:40:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.005+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e4003d10 con 0x7fd9ec159db0 2023-10-21T17:40:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.005+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4004260 con 0x7fd9ec159db0 2023-10-21T17:40:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.005+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec0ae3e0 con 0x7fd9ec159db0 2023-10-21T17:40:26.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.005+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec0ae9a0 con 0x7fd9ec159db0 2023-10-21T17:40:26.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.006+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0aeec0 con 0x7fd9ec159db0 2023-10-21T17:40:26.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.007+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd9e400b040 con 0x7fd9ec159db0 2023-10-21T17:40:26.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.008+0000 7fd9e97fa640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 0x7fd9cc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.008+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fd9e408a720 con 0x7fd9ec159db0 2023-10-21T17:40:26.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.009+0000 7fd9ebfff640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 0x7fd9cc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.010+0000 7fd9ebfff640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 0x7fd9cc06d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc004f60 tx=0x7fd9dc007e30 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.012+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9e4054e90 con 0x7fd9ec159db0 2023-10-21T17:40:26.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.210+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9ec04e160 con 0x7fd9ec159db0 2023-10-21T17:40:26.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.212+0000 7fd9e97fa640 1 -- 172.21.15.53:0/3222212462 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18305 (secure 0 0 0) 0x7fd9e4003e80 con 0x7fd9ec159db0 2023-10-21T17:40:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:26.215 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":263,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:21.975193+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-21T17:39:52.278617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-21T17:39:52.975528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:6834/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:26.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 msgr2=0x7fd9cc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 0x7fd9cc06d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc004f60 tx=0x7fd9dc007e30 comp rx=0 tx=0).stop 2023-10-21T17:40:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 msgr2=0x7fd9ec1f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400cd10 tx=0x7fd9e400a650 comp rx=0 tx=0).stop 2023-10-21T17:40:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 shutdown_connections 2023-10-21T17:40:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd9cc06b0b0 0x7fd9cc06d570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9ec159db0 0x7fd9ec1f2460 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1599d0 0x7fd9ec15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 --2- 172.21.15.53:0/3222212462 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec0cc210 0x7fd9ec1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.215+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 >> 172.21.15.53:0/3222212462 conn(0x7fd9ec0cb0c0 msgr2=0x7fd9ec158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:26.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.216+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 shutdown_connections 2023-10-21T17:40:26.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.216+0000 7fd9f1773640 1 -- 172.21.15.53:0/3222212462 wait complete. 2023-10-21T17:40:26.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-21T17:40:26.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.458+0000 7f07a2fef640 1 Processor -- start 2023-10-21T17:40:26.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.458+0000 7f07a2fef640 1 -- start start 2023-10-21T17:40:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a2fef640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f079c1599d0 0x7f079c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a2fef640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a2fef640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0cb8a0 con 0x7f079c0cb410 2023-10-21T17:40:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a2fef640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c159db0 con 0x7f079c1599d0 2023-10-21T17:40:26.463 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a2fef640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c071fe0 con 0x7f079c0b3c50 2023-10-21T17:40:26.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a1fed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f079c1599d0 0x7f079c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a1fed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f079c1599d0 0x7f079c0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46288/0 (socket says 172.21.15.53:46288) 2023-10-21T17:40:26.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a27ee640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f079c0cb410 0x7f079c0c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37116/0 (socket says 172.21.15.53:37116) 2023-10-21T17:40:26.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.460+0000 7f07a1fed640 1 -- 172.21.15.53:0/549145971 learned_addr learned my addr 172.21.15.53:0/549145971 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:26.464 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a1fed640 1 -- 172.21.15.53:0/549145971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 msgr2=0x7f079c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a17ec640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a1fed640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a1fed640 1 -- 172.21.15.53:0/549145971 >> v1:172.21.15.53:6789/0 conn(0x7f079c0cb410 legacy=0x7f079c0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a1fed640 1 -- 172.21.15.53:0/549145971 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0b44c0 con 0x7f079c1599d0 2023-10-21T17:40:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.461+0000 7f07a0feb640 1 -- 172.21.15.53:0/549145971 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507022800 0 0) 0x7f079c0cb8a0 con 0x7f079c0cb410 2023-10-21T17:40:26.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.462+0000 7f07a1fed640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c1599d0 0x7f079c0cdcd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f0798009870 tx=0x7f079802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93d0153235295c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.462+0000 7f07a0feb640 1 -- 172.21.15.53:0/549145971 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798030840 con 0x7f079c1599d0 2023-10-21T17:40:26.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.462+0000 7f07a0feb640 1 -- 172.21.15.53:0/549145971 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980309a0 con 0x7f079c1599d0 2023-10-21T17:40:26.467 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.462+0000 7f07a0feb640 1 -- 172.21.15.53:0/549145971 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798030d30 con 0x7f079c1599d0 2023-10-21T17:40:26.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 -- 172.21.15.53:0/549145971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c1599d0 msgr2=0x7f079c0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c1599d0 0x7f079c0cdcd0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f0798009870 tx=0x7f079802b980 comp rx=0 tx=0).stop 2023-10-21T17:40:26.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 -- 172.21.15.53:0/549145971 shutdown_connections 2023-10-21T17:40:26.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.468 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 --2- 172.21.15.53:0/549145971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c1599d0 0x7f079c0cdcd0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 -- 172.21.15.53:0/549145971 >> 172.21.15.53:0/549145971 conn(0x7f079c06c050 msgr2=0x7f079c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:26.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 -- 172.21.15.53:0/549145971 shutdown_connections 2023-10-21T17:40:26.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.463+0000 7f07a2fef640 1 -- 172.21.15.53:0/549145971 wait complete. 2023-10-21T17:40:26.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.465+0000 7f07a2fef640 1 Processor -- start 2023-10-21T17:40:26.469 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.465+0000 7f07a2fef640 1 -- start start 2023-10-21T17:40:26.471 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 0x7f079c0aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0a74c0 con 0x7f079c0cb410 2023-10-21T17:40:26.472 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0ad0b0 con 0x7f079c1599d0 2023-10-21T17:40:26.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a2fef640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0ad390 con 0x7f079c0b3c50 2023-10-21T17:40:26.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a17ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.466+0000 7f07a1fed640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46294/0 (socket says 172.21.15.53:46294) 2023-10-21T17:40:26.473 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a1fed640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:54842/0 (socket says 172.21.15.53:54842) 2023-10-21T17:40:26.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 -- 172.21.15.53:0/3947823835 learned_addr learned my addr 172.21.15.53:0/3947823835 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:26.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a27ee640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 0x7f079c0aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 -- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 msgr2=0x7f079c0ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 -- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 msgr2=0x7f079c0aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 0x7f079c0aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 -- 172.21.15.53:0/3947823835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980094d0 con 0x7f079c0cb410 2023-10-21T17:40:26.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a27ee640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 0x7f079c0aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:40:26.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.467+0000 7f07a17ec640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c7f0 tx=0x7f078c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.468+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c004220 con 0x7f079c0cb410 2023-10-21T17:40:26.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.468+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0043a0 con 0x7f079c0cb410 2023-10-21T17:40:26.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.468+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c007860 con 0x7f079c0cb410 2023-10-21T17:40:26.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.468+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c0a76d0 con 0x7f079c0cb410 2023-10-21T17:40:26.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.468+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c0acd80 con 0x7f079c0cb410 2023-10-21T17:40:26.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.469+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c074150 con 0x7f079c0cb410 2023-10-21T17:40:26.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.470+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f078c007bd0 con 0x7f079c0cb410 2023-10-21T17:40:26.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.472+0000 7f07927fc640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 0x7f078006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.472+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f078c08a010 con 0x7f079c0cb410 2023-10-21T17:40:26.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.472+0000 7f07a1fed640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 0x7f078006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.478 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.474+0000 7f07a1fed640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 0x7f078006d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0798036000 tx=0x7f0798008030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.477+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f078c054710 con 0x7f079c0cb410 2023-10-21T17:40:26.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.735+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f079c0ad550 con 0x7f079c0cb410 2023-10-21T17:40:26.738 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:40:26.736+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:40:26.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.739+0000 7f07927fc640 1 -- 172.21.15.53:0/3947823835 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) v1 ==== 167+0+0 (secure 0 0 0) 0x7f078c01e020 con 0x7f079c0cb410 2023-10-21T17:40:26.742 INFO:teuthology.orchestra.run.smithi053.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:40:26.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.742+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 msgr2=0x7f078006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.742+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 0x7f078006d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0798036000 tx=0x7f0798008030 comp rx=0 tx=0).stop 2023-10-21T17:40:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.742+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 msgr2=0x7f079c0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.742+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c7f0 tx=0x7f078c00ccb0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 shutdown_connections 2023-10-21T17:40:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f078006b0b0 0x7f078006d570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f079c1599d0 0x7f079c0aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f079c0cb410 0x7f079c0ac840 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 --2- 172.21.15.53:0/3947823835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f079c0b3c50 0x7f079c0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 >> 172.21.15.53:0/3947823835 conn(0x7f079c06c050 msgr2=0x7f079c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 shutdown_connections 2023-10-21T17:40:26.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.743+0000 7f07a2fef640 1 -- 172.21.15.53:0/3947823835 wait complete. 2023-10-21T17:40:26.757 INFO:tasks.ceph.mds.c.smithi053.stderr: -1725> 2023-10-21T17:39:59.370+0000 7f6fab305640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:40:26.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-21T17:40:26.826 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:40:26.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.966+0000 7f22e359e640 1 Processor -- start 2023-10-21T17:40:26.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.967+0000 7f22e359e640 1 -- start start 2023-10-21T17:40:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.968+0000 7f22e359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22e41599d0 0x7f22e40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.968+0000 7f22e359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4159db0 con 0x7f22e40cb5e0 2023-10-21T17:40:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.968+0000 7f22e359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4071fe0 con 0x7f22e40b3c50 2023-10-21T17:40:26.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.968+0000 7f22e359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40b44c0 con 0x7f22e41599d0 2023-10-21T17:40:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22e41599d0 0x7f22e40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e2d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f22e40cb5e0 0x7f22e40cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37142/0 (socket says 172.21.15.53:37142) 2023-10-21T17:40:26.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e2d9d640 1 -- 172.21.15.53:0/2660994586 learned_addr learned my addr 172.21.15.53:0/2660994586 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:26.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e259c640 1 -- 172.21.15.53:0/2660994586 >> v1:172.21.15.124:6789/0 conn(0x7f22e40b3c50 legacy=0x7f22e40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e259c640 1 -- 172.21.15.53:0/2660994586 >> v1:172.21.15.53:6789/0 conn(0x7f22e40cb5e0 legacy=0x7f22e40cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e259c640 1 -- 172.21.15.53:0/2660994586 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40713d0 con 0x7f22e41599d0 2023-10-21T17:40:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.969+0000 7f22e159a640 1 -- 172.21.15.53:0/2660994586 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131215519 0 0) 0x7f22dc001490 con 0x7f22e40cb5e0 2023-10-21T17:40:26.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.970+0000 7f22e259c640 1 --2- 172.21.15.53:0/2660994586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e41599d0 0x7f22e40cdcd0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc087d81b2b09ba5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.970+0000 7f22e159a640 1 -- 172.21.15.53:0/2660994586 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4031830 con 0x7f22e41599d0 2023-10-21T17:40:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e159a640 1 -- 172.21.15.53:0/2660994586 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4031990 con 0x7f22e41599d0 2023-10-21T17:40:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e159a640 1 -- 172.21.15.53:0/2660994586 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4031d20 con 0x7f22e41599d0 2023-10-21T17:40:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 -- 172.21.15.53:0/2660994586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e41599d0 msgr2=0x7f22e40cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.975 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 --2- 172.21.15.53:0/2660994586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e41599d0 0x7f22e40cdcd0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097a0 tx=0x7f22d402b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 -- 172.21.15.53:0/2660994586 shutdown_connections 2023-10-21T17:40:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 --2- 172.21.15.53:0/2660994586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e41599d0 0x7f22e40cdcd0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 -- 172.21.15.53:0/2660994586 >> 172.21.15.53:0/2660994586 conn(0x7f22e406c050 msgr2=0x7f22e406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 -- 172.21.15.53:0/2660994586 shutdown_connections 2023-10-21T17:40:26.976 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.971+0000 7f22e359e640 1 -- 172.21.15.53:0/2660994586 wait complete. 2023-10-21T17:40:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.973+0000 7f22e359e640 1 Processor -- start 2023-10-21T17:40:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.973+0000 7f22e359e640 1 -- start start 2023-10-21T17:40:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 0x7f22e40a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e41599d0 0x7f22e40a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b29e0 con 0x7f22e40b3c50 2023-10-21T17:40:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4147100 con 0x7f22e40cb5e0 2023-10-21T17:40:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4147e30 con 0x7f22e41599d0 2023-10-21T17:40:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46318/0 (socket says 172.21.15.53:46318) 2023-10-21T17:40:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.974+0000 7f22e259c640 1 -- 172.21.15.53:0/1055529748 learned_addr learned my addr 172.21.15.53:0/1055529748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e1d9b640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 0x7f22e40a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 -- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e41599d0 msgr2=0x7f22e40a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e41599d0 0x7f22e40a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 -- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 msgr2=0x7f22e40a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 0x7f22e40a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 -- 172.21.15.53:0/1055529748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009400 con 0x7f22e40b3c50 2023-10-21T17:40:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e1d9b640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 0x7f22e40a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22e259c640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f22d40098c0 tx=0x7f22d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.975+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d403aa60 con 0x7f22e40b3c50 2023-10-21T17:40:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.976+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d403abc0 con 0x7f22e40b3c50 2023-10-21T17:40:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.976+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40423f0 con 0x7f22e40b3c50 2023-10-21T17:40:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.976+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e4147500 con 0x7f22e40b3c50 2023-10-21T17:40:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.976+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e4147ac0 con 0x7f22e40b3c50 2023-10-21T17:40:26.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.977+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e4074150 con 0x7f22e40b3c50 2023-10-21T17:40:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.978+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f22d4030380 con 0x7f22e40b3c50 2023-10-21T17:40:26.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.979+0000 7f22daffd640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 0x7f22c406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.979+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f22d40abd90 con 0x7f22e40b3c50 2023-10-21T17:40:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.979+0000 7f22e1d9b640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 0x7f22c406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:26.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.980+0000 7f22e1d9b640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 0x7f22c406d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f22e40b30e0 tx=0x7f22cc009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:26.986 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:26.983+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22d4076360 con 0x7f22e40b3c50 2023-10-21T17:40:27.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.217+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f22e4070b30 con 0x7f22e40b3c50 2023-10-21T17:40:27.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.743+0000 7f22daffd640 1 -- 172.21.15.53:0/1055529748 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v138) v1 ==== 107+0+0 (secure 0 0 0) 0x7f22d40470b0 con 0x7f22e40b3c50 2023-10-21T17:40:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 msgr2=0x7f22c406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 0x7f22c406d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f22e40b30e0 tx=0x7f22cc009040 comp rx=0 tx=0).stop 2023-10-21T17:40:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 msgr2=0x7f22e4146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f22d40098c0 tx=0x7f22d40046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:27.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 shutdown_connections 2023-10-21T17:40:27.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c406b0b0 0x7f22c406d570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:27.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e41599d0 0x7f22e40a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:27.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e40cb5e0 0x7f22e40a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:27.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 --2- 172.21.15.53:0/1055529748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e40b3c50 0x7f22e4146bc0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:27.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 >> 172.21.15.53:0/1055529748 conn(0x7f22e406c050 msgr2=0x7f22e4159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:27.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 shutdown_connections 2023-10-21T17:40:27.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:27.747+0000 7f22e359e640 1 -- 172.21.15.53:0/1055529748 wait complete. 2023-10-21T17:40:27.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-21T17:40:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.002+0000 7fa095933640 1 Processor -- start 2023-10-21T17:40:28.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.002+0000 7fa095933640 1 -- start start 2023-10-21T17:40:28.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa095933640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09006c3a0 0x7fa0900c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa095933640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa09015aa00 0x7fa0900b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa095933640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09015b310 con 0x7fa09006db60 2023-10-21T17:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa095933640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0900ca230 con 0x7fa09015aa00 2023-10-21T17:40:28.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa095933640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09006d290 con 0x7fa09006c3a0 2023-10-21T17:40:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa09015aa00 0x7fa0900b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa09015aa00 0x7fa0900b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46328/0 (socket says 172.21.15.53:46328) 2023-10-21T17:40:28.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 -- 172.21.15.53:0/2279824335 learned_addr learned my addr 172.21.15.53:0/2279824335 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 -- 172.21.15.53:0/2279824335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09006c3a0 msgr2=0x7fa0900c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 --2- 172.21.15.53:0/2279824335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09006c3a0 0x7fa0900c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 -- 172.21.15.53:0/2279824335 >> v1:172.21.15.124:6789/0 conn(0x7fa09006db60 legacy=0x7fa09006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:28.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.003+0000 7fa08f7fe640 1 -- 172.21.15.53:0/2279824335 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900c9860 con 0x7fa09015aa00 2023-10-21T17:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.004+0000 7fa08f7fe640 1 --2- 172.21.15.53:0/2279824335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09015aa00 0x7fa0900b2a90 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa08802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549838d1d9154013 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.005+0000 7fa08dffb640 1 -- 172.21.15.53:0/2279824335 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088031640 con 0x7fa09015aa00 2023-10-21T17:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.005+0000 7fa08dffb640 1 -- 172.21.15.53:0/2279824335 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0880317a0 con 0x7fa09015aa00 2023-10-21T17:40:28.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.005+0000 7fa08dffb640 1 -- 172.21.15.53:0/2279824335 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088031b30 con 0x7fa09015aa00 2023-10-21T17:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.005+0000 7fa095933640 1 -- 172.21.15.53:0/2279824335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09015aa00 msgr2=0x7fa0900b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:28.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.005+0000 7fa095933640 1 --2- 172.21.15.53:0/2279824335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09015aa00 0x7fa0900b2a90 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa08802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 -- 172.21.15.53:0/2279824335 shutdown_connections 2023-10-21T17:40:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 --2- 172.21.15.53:0/2279824335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09015aa00 0x7fa0900b2a90 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 --2- 172.21.15.53:0/2279824335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09006c3a0 0x7fa0900c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 -- 172.21.15.53:0/2279824335 >> 172.21.15.53:0/2279824335 conn(0x7fa09006a5d0 msgr2=0x7fa0900b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:28.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 -- 172.21.15.53:0/2279824335 shutdown_connections 2023-10-21T17:40:28.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.006+0000 7fa095933640 1 -- 172.21.15.53:0/2279824335 wait complete. 2023-10-21T17:40:28.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.007+0000 7fa095933640 1 Processor -- start 2023-10-21T17:40:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.009+0000 7fa095933640 1 -- start start 2023-10-21T17:40:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 0x7fa0900c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09015aa00 0x7fa09014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa090155900 con 0x7fa09006c3a0 2023-10-21T17:40:28.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09014ed80 con 0x7fa09006db60 2023-10-21T17:40:28.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa090155620 con 0x7fa09015aa00 2023-10-21T17:40:28.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:28.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46344/0 (socket says 172.21.15.53:46344) 2023-10-21T17:40:28.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 -- 172.21.15.53:0/1421892816 learned_addr learned my addr 172.21.15.53:0/1421892816 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:28.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 -- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09015aa00 msgr2=0x7fa09014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:28.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09015aa00 0x7fa09014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 -- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 msgr2=0x7fa0900c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:28.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08e7fc640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 0x7fa0900c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:28.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 0x7fa0900c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:28.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 -- 172.21.15.53:0/1421892816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880094d0 con 0x7fa09006c3a0 2023-10-21T17:40:28.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08e7fc640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 0x7fa0900c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa08effd640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fa08000b4a0 tx=0x7fa08000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080010610 con 0x7fa09006c3a0 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080010770 con 0x7fa09006c3a0 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080010b20 con 0x7fa09006c3a0 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa090154620 con 0x7fa09006c3a0 2023-10-21T17:40:28.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.010+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa090154be0 con 0x7fa09006c3a0 2023-10-21T17:40:28.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.012+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa090154480 con 0x7fa09006c3a0 2023-10-21T17:40:28.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.013+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa080010c80 con 0x7fa09006c3a0 2023-10-21T17:40:28.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.013+0000 7fa087fff640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 0x7fa07006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:28.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.014+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fa080089bb0 con 0x7fa09006c3a0 2023-10-21T17:40:28.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.014+0000 7fa08e7fc640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 0x7fa07006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:28.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.017+0000 7fa08e7fc640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 0x7fa07006d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa0900ce300 tx=0x7fa078009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:28.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.018+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa090154480 con 0x7fa09006c3a0 2023-10-21T17:40:28.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:28.216+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa090154480 con 0x7fa09006c3a0 2023-10-21T17:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.035+0000 7fa087fff640 1 -- 172.21.15.53:0/1421892816 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v265) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa090154480 con 0x7fa09006c3a0 2023-10-21T17:40:29.037 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_metadata' removed 2023-10-21T17:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.040+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 msgr2=0x7fa07006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.040+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 0x7fa07006d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa0900ce300 tx=0x7fa078009040 comp rx=0 tx=0).stop 2023-10-21T17:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.040+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 msgr2=0x7fa0900c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.040+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fa08000b4a0 tx=0x7fa08000b960 comp rx=0 tx=0).stop 2023-10-21T17:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 shutdown_connections 2023-10-21T17:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa07006b100 0x7fa07006d5c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa09015aa00 0x7fa09014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa09006db60 0x7fa0900c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 --2- 172.21.15.53:0/1421892816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa09006c3a0 0x7fa0900c5810 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.041+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 >> 172.21.15.53:0/1421892816 conn(0x7fa09006a5d0 msgr2=0x7fa090155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:29.044 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.043+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 shutdown_connections 2023-10-21T17:40:29.045 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.044+0000 7fa095933640 1 -- 172.21.15.53:0/1421892816 wait complete. 2023-10-21T17:40:29.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-21T17:40:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.293+0000 7f851d2e6640 1 Processor -- start 2023-10-21T17:40:29.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.294+0000 7f851d2e6640 1 -- start start 2023-10-21T17:40:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.295+0000 7f851d2e6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85180aff70 0x7f85180b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.295+0000 7f851d2e6640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 0x7f85180ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.295+0000 7f851d2e6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85180c9250 con 0x7f85180c18d0 2023-10-21T17:40:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.295+0000 7f851d2e6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518161590 con 0x7f85180aff70 2023-10-21T17:40:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.295+0000 7f851d2e6640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85180cc570 con 0x7f8518164d10 2023-10-21T17:40:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85180aff70 0x7f85180b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f85180aff70 0x7f85180b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46348/0 (socket says 172.21.15.53:46348) 2023-10-21T17:40:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 -- 172.21.15.53:0/962149387 learned_addr learned my addr 172.21.15.53:0/962149387 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f85177fe640 1 --1- 172.21.15.53:0/962149387 >> v1:172.21.15.53:6789/0 conn(0x7f85180c18d0 0x7f85180c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37152/0 (socket says 172.21.15.53:37152) 2023-10-21T17:40:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 -- 172.21.15.53:0/962149387 >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 msgr2=0x7f85180ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f851cae5640 1 --2- 172.21.15.53:0/962149387 >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 0x7f85180ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 --2- 172.21.15.53:0/962149387 >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 0x7f85180ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 -- 172.21.15.53:0/962149387 >> v1:172.21.15.53:6789/0 conn(0x7f85180c18d0 legacy=0x7f85180c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8517fff640 1 -- 172.21.15.53:0/962149387 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180c94b0 con 0x7f85180aff70 2023-10-21T17:40:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f8516ffd640 1 -- 172.21.15.53:0/962149387 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767240922 0 0) 0x7f85180c9250 con 0x7f85180c18d0 2023-10-21T17:40:29.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.296+0000 7f851cae5640 1 --2- 172.21.15.53:0/962149387 >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 0x7f85180ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:29.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.297+0000 7f8517fff640 1 --2- 172.21.15.53:0/962149387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f85180b0350 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f85180cdf00 tx=0x7f850802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67997b057c7a0d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.298+0000 7f8516ffd640 1 -- 172.21.15.53:0/962149387 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031880 con 0x7f85180aff70 2023-10-21T17:40:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.298+0000 7f8516ffd640 1 -- 172.21.15.53:0/962149387 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080319e0 con 0x7f85180aff70 2023-10-21T17:40:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.298+0000 7f8516ffd640 1 -- 172.21.15.53:0/962149387 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031d70 con 0x7f85180aff70 2023-10-21T17:40:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.298+0000 7f851d2e6640 1 -- 172.21.15.53:0/962149387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 msgr2=0x7f85180b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.302 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.298+0000 7f851d2e6640 1 --2- 172.21.15.53:0/962149387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f85180b0350 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f85180cdf00 tx=0x7f850802b920 comp rx=0 tx=0).stop 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 -- 172.21.15.53:0/962149387 shutdown_connections 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 --2- 172.21.15.53:0/962149387 >> v2:172.21.15.124:3300/0 conn(0x7f8518164d10 0x7f85180ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 --2- 172.21.15.53:0/962149387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f85180b0350 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 -- 172.21.15.53:0/962149387 >> 172.21.15.53:0/962149387 conn(0x7f851806c050 msgr2=0x7f851806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 -- 172.21.15.53:0/962149387 shutdown_connections 2023-10-21T17:40:29.303 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.299+0000 7f851d2e6640 1 -- 172.21.15.53:0/962149387 wait complete. 2023-10-21T17:40:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.300+0000 7f851d2e6640 1 Processor -- start 2023-10-21T17:40:29.304 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.301+0000 7f851d2e6640 1 -- start start 2023-10-21T17:40:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.301+0000 7f851d2e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.301+0000 7f851d2e6640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85180c18d0 0x7f85180b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f851d2e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8518164d10 0x7f8518161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f851d2e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851814c3a0 con 0x7f85180aff70 2023-10-21T17:40:29.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f851d2e6640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851814f3b0 con 0x7f85180c18d0 2023-10-21T17:40:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f851d2e6640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c67d0 con 0x7f8518164d10 2023-10-21T17:40:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46354/0 (socket says 172.21.15.53:46354) 2023-10-21T17:40:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 -- 172.21.15.53:0/1735775368 learned_addr learned my addr 172.21.15.53:0/1735775368 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:29.307 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f851cae5640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8518164d10 0x7f8518161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 -- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8518164d10 msgr2=0x7f8518161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f85177fe640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85180c18d0 0x7f85180b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8518164d10 0x7f8518161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 -- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85180c18d0 msgr2=0x7f85180b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:29.308 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85180c18d0 0x7f85180b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.302+0000 7f8517fff640 1 -- 172.21.15.53:0/1735775368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080094d0 con 0x7f85180aff70 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f8517fff640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f850802b960 tx=0x7f85080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031870 con 0x7f85180aff70 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080443f0 con 0x7f85180aff70 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180c6ab0 con 0x7f85180aff70 2023-10-21T17:40:29.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85080447a0 con 0x7f85180aff70 2023-10-21T17:40:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.303+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85180b3f30 con 0x7f85180aff70 2023-10-21T17:40:29.310 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.304+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85180c6610 con 0x7f85180aff70 2023-10-21T17:40:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.305+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f850804d450 con 0x7f85180aff70 2023-10-21T17:40:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.306+0000 7f8514ff9640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.306+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6745+0+0 (secure 0 0 0) 0x7f85080ac870 con 0x7f85180aff70 2023-10-21T17:40:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.307+0000 7f85177fe640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:29.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.308+0000 7f85177fe640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 0x7f850406d570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f850c007fd0 tx=0x7f850c009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:29.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.311+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8508077000 con 0x7f85180aff70 2023-10-21T17:40:29.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:29.506+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85180c6fd0 con 0x7f85180aff70 2023-10-21T17:40:30.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.385+0000 7f8514ff9640 1 -- 172.21.15.53:0/1735775368 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v266) v1 ==== 178+0+0 (secure 0 0 0) 0x7f850807a620 con 0x7f85180aff70 2023-10-21T17:40:30.387 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'backup_fs_data' removed 2023-10-21T17:40:30.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 msgr2=0x7f850406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 0x7f850406d570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f850c007fd0 tx=0x7f850c009040 comp rx=0 tx=0).stop 2023-10-21T17:40:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 msgr2=0x7f851814c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f850802b960 tx=0x7f85080046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 shutdown_connections 2023-10-21T17:40:30.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8518164d10 0x7f8518161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85180c18d0 0x7f85180b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 --2- 172.21.15.53:0/1735775368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85180aff70 0x7f851814c5f0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 >> 172.21.15.53:0/1735775368 conn(0x7f851806c050 msgr2=0x7f85180ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:30.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.389+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 shutdown_connections 2023-10-21T17:40:30.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.390+0000 7f851d2e6640 1 -- 172.21.15.53:0/1735775368 wait complete. 2023-10-21T17:40:30.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:40:30.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.636+0000 7fd08947c640 1 Processor -- start 2023-10-21T17:40:30.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.637+0000 7fd08947c640 1 -- start start 2023-10-21T17:40:30.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.638+0000 7fd08947c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 0x7fd0840b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.638+0000 7fd08947c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08406e1e0 con 0x7fd0840cb150 2023-10-21T17:40:30.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.639+0000 7fd08947c640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0840cba50 con 0x7fd084161690 2023-10-21T17:40:30.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.639+0000 7fd08947c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08406cc50 con 0x7fd0840cb600 2023-10-21T17:40:30.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.639+0000 7fd07bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd0840cb150 0x7fd0840b1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37162/0 (socket says 172.21.15.53:37162) 2023-10-21T17:40:30.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.639+0000 7fd07bfff640 1 -- 172.21.15.53:0/3067923366 learned_addr learned my addr 172.21.15.53:0/3067923366 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.639+0000 7fd0837fe640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 0x7fd0840b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944656516 0 0) 0x7fd08406e1e0 con 0x7fd0840cb150 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd070003520 con 0x7fd0840cb150 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789199122 0 0) 0x7fd070003520 con 0x7fd0840cb150 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 msgr2=0x7fd0840b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:30.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 0x7fd0840b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 >> v1:172.21.15.124:6789/0 conn(0x7fd084161690 legacy=0x7fd08406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08406cf30 con 0x7fd0840cb150 2023-10-21T17:40:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd06c001690 con 0x7fd0840cb150 2023-10-21T17:40:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 >> v1:172.21.15.53:6789/0 conn(0x7fd0840cb150 legacy=0x7fd0840b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd070003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 0x7fd0700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd070004d00 0x7fd070005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08406cc50 con 0x7fd0700039a0 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000ba30 con 0x7fd070004d00 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.640+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07000bbc0 con 0x7fd0840cb600 2023-10-21T17:40:30.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd070003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 -- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 msgr2=0x7fd0700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0840cb600 0x7fd0700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 -- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd070004d00 msgr2=0x7fd070005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd070004d00 0x7fd070005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.641+0000 7fd082ffd640 1 -- 172.21.15.53:0/3067923366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070003520 con 0x7fd0700039a0 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.642+0000 7fd082ffd640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd070003d80 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd074001890 tx=0x7fd0700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e4c152af0f6c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.642+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074033620 con 0x7fd0700039a0 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.642+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740337a0 con 0x7fd0700039a0 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.642+0000 7fd0827fc640 1 -- 172.21.15.53:0/3067923366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074033b50 con 0x7fd0700039a0 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 -- 172.21.15.53:0/3067923366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 msgr2=0x7fd070003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 --2- 172.21.15.53:0/3067923366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd070003d80 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fd074001890 tx=0x7fd0700057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd0837fe640 1 -- 172.21.15.53:0/3067923366 reap_dead start 2023-10-21T17:40:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 -- 172.21.15.53:0/3067923366 shutdown_connections 2023-10-21T17:40:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 -- 172.21.15.53:0/3067923366 >> 172.21.15.53:0/3067923366 conn(0x7fd08406a5d0 msgr2=0x7fd0840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 -- 172.21.15.53:0/3067923366 shutdown_connections 2023-10-21T17:40:30.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.643+0000 7fd08947c640 1 -- 172.21.15.53:0/3067923366 wait complete. 2023-10-21T17:40:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.645+0000 7fd08947c640 1 Processor -- start 2023-10-21T17:40:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.645+0000 7fd08947c640 1 -- start start 2023-10-21T17:40:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd084161a70 0x7fd08415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0840b3030 0x7fd0840b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084161ef0 con 0x7fd0700039a0 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840b39f0 con 0x7fd0840b3030 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.646+0000 7fd08947c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084162a40 con 0x7fd084161a70 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46376/0 (socket says 172.21.15.53:46376) 2023-10-21T17:40:30.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 -- 172.21.15.53:0/2511020170 learned_addr learned my addr 172.21.15.53:0/2511020170 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd0837fe640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0840b3030 0x7fd0840b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd07bfff640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd084161a70 0x7fd08415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 -- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd084161a70 msgr2=0x7fd08415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd084161a70 0x7fd08415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 -- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0840b3030 msgr2=0x7fd0840b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0840b3030 0x7fd0840b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.647+0000 7fd082ffd640 1 -- 172.21.15.53:0/2511020170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840c2100 con 0x7fd0700039a0 2023-10-21T17:40:30.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd082ffd640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fd074000c00 tx=0x7fd074001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:30.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074007040 con 0x7fd0700039a0 2023-10-21T17:40:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740071a0 con 0x7fd0700039a0 2023-10-21T17:40:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0840c15c0 con 0x7fd0700039a0 2023-10-21T17:40:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074007510 con 0x7fd0700039a0 2023-10-21T17:40:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.648+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd084162480 con 0x7fd0700039a0 2023-10-21T17:40:30.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.650+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08404e160 con 0x7fd0700039a0 2023-10-21T17:40:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.650+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd074031070 con 0x7fd0700039a0 2023-10-21T17:40:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.651+0000 7fd07b7fe640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.651+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7fd074040070 con 0x7fd0700039a0 2023-10-21T17:40:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.652+0000 7fd07bfff640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:30.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.653+0000 7fd07bfff640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 0x7fd06406d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd06c001880 tx=0x7fd06c001670 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:30.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.656+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0740764a0 con 0x7fd0700039a0 2023-10-21T17:40:30.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.850+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd084073900 con 0x7fd0700039a0 2023-10-21T17:40:30.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.857+0000 7fd07b7fe640 1 -- 172.21.15.53:0/2511020170 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd074079ac0 con 0x7fd0700039a0 2023-10-21T17:40:30.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 msgr2=0x7fd06406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 0x7fd06406d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd06c001880 tx=0x7fd06c001670 comp rx=0 tx=0).stop 2023-10-21T17:40:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 msgr2=0x7fd0840b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:30.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fd074000c00 tx=0x7fd074001230 comp rx=0 tx=0).stop 2023-10-21T17:40:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 shutdown_connections 2023-10-21T17:40:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fd06406b0b0 0x7fd06406d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0840b3030 0x7fd0840b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd084161a70 0x7fd08415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 --2- 172.21.15.53:0/2511020170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0700039a0 0x7fd0840b4150 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.860+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 >> 172.21.15.53:0/2511020170 conn(0x7fd08406a5d0 msgr2=0x7fd0840701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.861+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 shutdown_connections 2023-10-21T17:40:30.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:30.861+0000 7fd08947c640 1 -- 172.21.15.53:0/2511020170 wait complete. 2023-10-21T17:40:30.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:40:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.123+0000 7f0098ee7640 1 Processor -- start 2023-10-21T17:40:31.124 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.123+0000 7f0098ee7640 1 -- start start 2023-10-21T17:40:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b1750 0x7f009415d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.126 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 0x7f00940cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f00940702d0 0x7f0094070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009415dcf0 con 0x7f00940702d0 2023-10-21T17:40:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0094070ca0 con 0x7f00940b40e0 2023-10-21T17:40:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.124+0000 7f0098ee7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00940b30a0 con 0x7f00940b1750 2023-10-21T17:40:31.127 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f00940702d0 0x7f0094070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f00940702d0 0x7f0094070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46382/0 (socket says 172.21.15.53:46382) 2023-10-21T17:40:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 -- 172.21.15.53:0/112449741 learned_addr learned my addr 172.21.15.53:0/112449741 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f009259c640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b1750 0x7f009415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0091d9b640 1 --2- 172.21.15.53:0/112449741 >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 0x7f00940cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.128 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 -- 172.21.15.53:0/112449741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b1750 msgr2=0x7f009415d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b1750 0x7f009415d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 -- 172.21.15.53:0/112449741 >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 msgr2=0x7f00940cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 --2- 172.21.15.53:0/112449741 >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 0x7f00940cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0092d9d640 1 -- 172.21.15.53:0/112449741 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940b3320 con 0x7f00940702d0 2023-10-21T17:40:31.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.125+0000 7f0091d9b640 1 --2- 172.21.15.53:0/112449741 >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 0x7f00940cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.126+0000 7f0092d9d640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094070760 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f009406abc0 tx=0x7f008c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3ca1597a19dd33 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f009159a640 1 -- 172.21.15.53:0/112449741 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c02c9e0 con 0x7f00940702d0 2023-10-21T17:40:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f009159a640 1 -- 172.21.15.53:0/112449741 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c02cb40 con 0x7f00940702d0 2023-10-21T17:40:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f009159a640 1 -- 172.21.15.53:0/112449741 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c0343f0 con 0x7f00940702d0 2023-10-21T17:40:31.131 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 -- 172.21.15.53:0/112449741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 msgr2=0x7f0094070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094070760 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f009406abc0 tx=0x7f008c02be40 comp rx=0 tx=0).stop 2023-10-21T17:40:31.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 -- 172.21.15.53:0/112449741 shutdown_connections 2023-10-21T17:40:31.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094070760 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 --2- 172.21.15.53:0/112449741 >> v2:172.21.15.124:3300/0 conn(0x7f00940b40e0 0x7f00940cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.132 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 --2- 172.21.15.53:0/112449741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b1750 0x7f009415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 -- 172.21.15.53:0/112449741 >> 172.21.15.53:0/112449741 conn(0x7f0094069d50 msgr2=0x7f009406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:31.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.127+0000 7f0098ee7640 1 -- 172.21.15.53:0/112449741 shutdown_connections 2023-10-21T17:40:31.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.128+0000 7f0098ee7640 1 -- 172.21.15.53:0/112449741 wait complete. 2023-10-21T17:40:31.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.129+0000 7f0098ee7640 1 Processor -- start 2023-10-21T17:40:31.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.129+0000 7f0098ee7640 1 -- start start 2023-10-21T17:40:31.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.130+0000 7f0098ee7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.130+0000 7f0098ee7640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00940b1750 0x7f00940cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.130+0000 7f0098ee7640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b40e0 0x7f00940c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.130+0000 7f0098ee7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00941568a0 con 0x7f00940702d0 2023-10-21T17:40:31.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.130+0000 7f0098ee7640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094145970 con 0x7f00940b1750 2023-10-21T17:40:31.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f0098ee7640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0094155c20 con 0x7f00940b40e0 2023-10-21T17:40:31.136 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46384/0 (socket says 172.21.15.53:46384) 2023-10-21T17:40:31.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 -- 172.21.15.53:0/430739114 learned_addr learned my addr 172.21.15.53:0/430739114 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:31.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f0092d9d640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b40e0 0x7f00940c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f0091d9b640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00940b1750 0x7f00940cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.137 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 -- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b40e0 msgr2=0x7f00940c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b40e0 0x7f00940c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 -- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00940b1750 msgr2=0x7f00940cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00940b1750 0x7f00940cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.131+0000 7f009259c640 1 -- 172.21.15.53:0/430739114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c009b30 con 0x7f00940702d0 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f009259c640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f008400d800 tx=0x7f008400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840096a0 con 0x7f00940702d0 2023-10-21T17:40:31.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084009800 con 0x7f00940702d0 2023-10-21T17:40:31.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084009b90 con 0x7f00940702d0 2023-10-21T17:40:31.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0094155f00 con 0x7f00940702d0 2023-10-21T17:40:31.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.132+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00941564c0 con 0x7f00940702d0 2023-10-21T17:40:31.139 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.133+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0094156b10 con 0x7f00940702d0 2023-10-21T17:40:31.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.134+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0084009cf0 con 0x7f00940702d0 2023-10-21T17:40:31.140 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.135+0000 7f008affd640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 0x7f007406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.135+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f008408a140 con 0x7f00940702d0 2023-10-21T17:40:31.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.136+0000 7f0091d9b640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 0x7f007406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.137+0000 7f0091d9b640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 0x7f007406d570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00940b4800 tx=0x7f007c008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.141 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.140+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0084054c10 con 0x7f00940702d0 2023-10-21T17:40:31.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.331+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f009404e160 con 0x7f00940702d0 2023-10-21T17:40:31.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.338+0000 7f008affd640 1 -- 172.21.15.53:0/430739114 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v86) v1 ==== 115+0+0 (secure 0 0 0) 0x7f008400b440 con 0x7f00940702d0 2023-10-21T17:40:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.341+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 msgr2=0x7f007406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.341+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 0x7f007406d570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00940b4800 tx=0x7f007c008040 comp rx=0 tx=0).stop 2023-10-21T17:40:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.341+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 msgr2=0x7f0094145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.341+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f008400d800 tx=0x7f008400dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 shutdown_connections 2023-10-21T17:40:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f007406b0b0 0x7f007406d570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00940b40e0 0x7f00940c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00940b1750 0x7f00940cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 --2- 172.21.15.53:0/430739114 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f00940702d0 0x7f0094145430 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 >> 172.21.15.53:0/430739114 conn(0x7f0094069d50 msgr2=0x7f0094151510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 shutdown_connections 2023-10-21T17:40:31.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.342+0000 7f0098ee7640 1 -- 172.21.15.53:0/430739114 wait complete. 2023-10-21T17:40:31.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:40:31.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.586+0000 7f03b4ca9640 1 Processor -- start 2023-10-21T17:40:31.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.586+0000 7f03b4ca9640 1 -- start start 2023-10-21T17:40:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03b4ca9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f03b00cb600 0x7f03b00b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03b4ca9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b006e1e0 con 0x7f03b0161690 2023-10-21T17:40:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03b4ca9640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b00cba50 con 0x7f03b00cb150 2023-10-21T17:40:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03b4ca9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b006cc50 con 0x7f03b00cb600 2023-10-21T17:40:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03ae59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f03b0161690 0x7f03b006dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37176/0 (socket says 172.21.15.53:37176) 2023-10-21T17:40:31.590 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.588+0000 7f03ae59c640 1 -- 172.21.15.53:0/2463532869 learned_addr learned my addr 172.21.15.53:0/2463532869 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03aed9d640 1 --2- 172.21.15.53:0/2463532869 >> v2:172.21.15.124:3300/0 conn(0x7f03b00cb600 0x7f03b00b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988924623 0 0) 0x7f03b006e1e0 con 0x7f03b0161690 2023-10-21T17:40:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f039c003520 con 0x7f03b0161690 2023-10-21T17:40:31.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2284030840 0 0) 0x7f039c003520 con 0x7f03b0161690 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 >> v2:172.21.15.124:3300/0 conn(0x7f03b00cb600 msgr2=0x7f03b00b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 --2- 172.21.15.53:0/2463532869 >> v2:172.21.15.124:3300/0 conn(0x7f03b00cb600 0x7f03b00b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 >> v1:172.21.15.124:6789/0 conn(0x7f03b00cb150 legacy=0x7f03b00b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b006cf30 con 0x7f03b0161690 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f03a0001a10 con 0x7f03b0161690 2023-10-21T17:40:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 >> v1:172.21.15.53:6789/0 conn(0x7f03b0161690 legacy=0x7f03b006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f039c0039c0 0x7f039c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.589+0000 7f03ad59a640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f039c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b006cc50 con 0x7f039c005180 2023-10-21T17:40:31.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c00c080 con 0x7f039c0047c0 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c00c240 con 0x7f039c0039c0 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f039c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03add9b640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03ae59c640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f039c0039c0 0x7f039c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 -- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f039c0039c0 msgr2=0x7f039c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:31.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f039c0039c0 0x7f039c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 -- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f039c0047c0 msgr2=0x7f039c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f039c0047c0 0x7f039c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.590+0000 7f03aed9d640 1 -- 172.21.15.53:0/2463532869 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c003520 con 0x7f039c005180 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.591+0000 7f03aed9d640 1 -- 172.21.15.53:0/2463532869 reap_dead start 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.591+0000 7f03aed9d640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f039c00b980 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f039c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffab46700c91c669 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8004020 con 0x7f039c005180 2023-10-21T17:40:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8004180 con 0x7f039c005180 2023-10-21T17:40:31.598 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03ad59a640 1 -- 172.21.15.53:0/2463532869 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a802c710 con 0x7f039c005180 2023-10-21T17:40:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03b4ca9640 1 -- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 msgr2=0x7f039c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f039c00b980 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f039c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.600 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03b4ca9640 1 -- 172.21.15.53:0/2463532869 shutdown_connections 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/2463532869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f039c00b980 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.592+0000 7f03b4ca9640 1 -- 172.21.15.53:0/2463532869 >> 172.21.15.53:0/2463532869 conn(0x7f03b006a5d0 msgr2=0x7f03b00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.593+0000 7f03b4ca9640 1 -- 172.21.15.53:0/2463532869 shutdown_connections 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.593+0000 7f03b4ca9640 1 -- 172.21.15.53:0/2463532869 wait complete. 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.595+0000 7f03b4ca9640 1 Processor -- start 2023-10-21T17:40:31.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.595+0000 7f03b4ca9640 1 -- start start 2023-10-21T17:40:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b01575a0 0x7f03b0157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b0163850 0x7f03b0163cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0157f60 con 0x7f039c005180 2023-10-21T17:40:31.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b014a5e0 con 0x7f03b01575a0 2023-10-21T17:40:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03b4ca9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00cd3d0 con 0x7f03b0163850 2023-10-21T17:40:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46396/0 (socket says 172.21.15.53:46396) 2023-10-21T17:40:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 -- 172.21.15.53:0/1491646333 learned_addr learned my addr 172.21.15.53:0/1491646333 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:31.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03add9b640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b01575a0 0x7f03b0157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 -- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b0163850 msgr2=0x7f03b0163cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b0163850 0x7f03b0163cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 -- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b01575a0 msgr2=0x7f03b0157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b01575a0 0x7f03b0157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.596+0000 7f03ae59c640 1 -- 172.21.15.53:0/1491646333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00cd6b0 con 0x7f039c005180 2023-10-21T17:40:31.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03ae59c640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f03a00020c0 tx=0x7f03a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0004d60 con 0x7f039c005180 2023-10-21T17:40:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a000be80 con 0x7f039c005180 2023-10-21T17:40:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a00072f0 con 0x7f039c005180 2023-10-21T17:40:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b00cd990 con 0x7f039c005180 2023-10-21T17:40:31.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.597+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b00cdf50 con 0x7f039c005180 2023-10-21T17:40:31.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.599+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b004e160 con 0x7f039c005180 2023-10-21T17:40:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.599+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f03a0007450 con 0x7f039c005180 2023-10-21T17:40:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.600+0000 7f03a6ffd640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 0x7f039006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.600+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f03a0089ea0 con 0x7f039c005180 2023-10-21T17:40:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.601+0000 7f03add9b640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 0x7f039006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:31.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.602+0000 7f03add9b640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 0x7f039006d5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0398007fd0 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:31.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.605+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03a00547d0 con 0x7f039c005180 2023-10-21T17:40:31.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.801+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f03b00afc70 con 0x7f039c005180 2023-10-21T17:40:31.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.808+0000 7f03a6ffd640 1 -- 172.21.15.53:0/1491646333 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v87) v1 ==== 120+0+0 (secure 0 0 0) 0x7f03a00190b0 con 0x7f039c005180 2023-10-21T17:40:31.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.810+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 msgr2=0x7f039006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.810+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 0x7f039006d5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0398007fd0 tx=0x7f0398009040 comp rx=0 tx=0).stop 2023-10-21T17:40:31.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 msgr2=0x7f03b00ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:31.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f03a00020c0 tx=0x7f03a000b4a0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 shutdown_connections 2023-10-21T17:40:31.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f039006b0e0 0x7f039006d5a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b0163850 0x7f03b0163cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b01575a0 0x7f03b0157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 --2- 172.21.15.53:0/1491646333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f039c005180 0x7f03b00ccdf0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:31.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 >> 172.21.15.53:0/1491646333 conn(0x7f03b006a5d0 msgr2=0x7f03b00b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:31.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 shutdown_connections 2023-10-21T17:40:31.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:31.811+0000 7f03b4ca9640 1 -- 172.21.15.53:0/1491646333 wait complete. 2023-10-21T17:40:31.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-21T17:40:32.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.054+0000 7f15a3b3a640 1 Processor -- start 2023-10-21T17:40:32.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.055+0000 7f15a3b3a640 1 -- start start 2023-10-21T17:40:32.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.056+0000 7f15a3b3a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f159c159ac0 0x7f159c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.056+0000 7f15a3b3a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.056+0000 7f15a3b3a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f159c06dee0 con 0x7f159c0ad790 2023-10-21T17:40:32.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.056+0000 7f15a3b3a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159c15d850 con 0x7f159c159ac0 2023-10-21T17:40:32.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.056+0000 7f15a3b3a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c15db30 con 0x7f159c0709e0 2023-10-21T17:40:32.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.057+0000 7f15a18d6640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f159c159ac0 0x7f159c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.057+0000 7f15a10d5640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f159c0ad790 0x7f159c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:52588/0 (socket says 172.21.15.53:52588) 2023-10-21T17:40:32.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.057+0000 7f15a10d5640 1 -- 172.21.15.53:0/3216516453 learned_addr learned my addr 172.21.15.53:0/3216516453 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.057+0000 7f15a20d7640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.058+0000 7f15a18d6640 1 -- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 msgr2=0x7f159c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.058+0000 7f15a18d6640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.058+0000 7f15a18d6640 1 -- 172.21.15.53:0/3216516453 >> v1:172.21.15.124:6789/0 conn(0x7f159c0ad790 legacy=0x7f159c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.058+0000 7f15a18d6640 1 -- 172.21.15.53:0/3216516453 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c15de10 con 0x7f159c159ac0 2023-10-21T17:40:32.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.058+0000 7f15a08d4640 1 -- 172.21.15.53:0/3216516453 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868112333 0 0) 0x7f159c06dee0 con 0x7f159c0ad790 2023-10-21T17:40:32.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.059+0000 7f15a18d6640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c06d5e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f15980097a0 tx=0x7f159802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637f65c5f3e96b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.062 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a08d4640 1 -- 172.21.15.53:0/3216516453 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598031830 con 0x7f159c159ac0 2023-10-21T17:40:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a08d4640 1 -- 172.21.15.53:0/3216516453 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598031990 con 0x7f159c159ac0 2023-10-21T17:40:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a08d4640 1 -- 172.21.15.53:0/3216516453 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598031d20 con 0x7f159c159ac0 2023-10-21T17:40:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 msgr2=0x7f159c06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c06d5e0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f15980097a0 tx=0x7f159802b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.063 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3216516453 shutdown_connections 2023-10-21T17:40:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3216516453 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c06d5e0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.060+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3216516453 >> 172.21.15.53:0/3216516453 conn(0x7f159c06a380 msgr2=0x7f159c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.061+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3216516453 shutdown_connections 2023-10-21T17:40:32.064 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.061+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3216516453 wait complete. 2023-10-21T17:40:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.062+0000 7f15a3b3a640 1 Processor -- start 2023-10-21T17:40:32.066 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.062+0000 7f15a3b3a640 1 -- start start 2023-10-21T17:40:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c157860 con 0x7f159c0ad790 2023-10-21T17:40:32.067 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c158370 con 0x7f159c159ac0 2023-10-21T17:40:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.063+0000 7f15a3b3a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14db30 con 0x7f159c0709e0 2023-10-21T17:40:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46410/0 (socket says 172.21.15.53:46410) 2023-10-21T17:40:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 -- 172.21.15.53:0/3788356322 learned_addr learned my addr 172.21.15.53:0/3788356322 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:32.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a20d7640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a18d6640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 -- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 msgr2=0x7f159c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 -- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 msgr2=0x7f159c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.064+0000 7f15a10d5640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15a10d5640 1 -- 172.21.15.53:0/3788356322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009400 con 0x7f159c0ad790 2023-10-21T17:40:32.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15a20d7640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:40:32.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15a10d5640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c4a0 tx=0x7f158c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c005ce0 con 0x7f159c0ad790 2023-10-21T17:40:32.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c005e40 con 0x7f159c0ad790 2023-10-21T17:40:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0072b0 con 0x7f159c0ad790 2023-10-21T17:40:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c157a40 con 0x7f159c0ad790 2023-10-21T17:40:32.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.065+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c158000 con 0x7f159c0ad790 2023-10-21T17:40:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.067+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c158520 con 0x7f159c0ad790 2023-10-21T17:40:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.068+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f158c007410 con 0x7f159c0ad790 2023-10-21T17:40:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.068+0000 7f15927fc640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 0x7f158006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.069+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f158c089170 con 0x7f159c0ad790 2023-10-21T17:40:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.069+0000 7f15a18d6640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 0x7f158006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.070+0000 7f15a18d6640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 0x7f158006d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1598036000 tx=0x7f1598008030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.077+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f159c158520 con 0x7f159c0ad790 2023-10-21T17:40:32.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.338+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f159c1584e0 con 0x7f159c0ad790 2023-10-21T17:40:32.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.390+0000 7f15927fc640 1 -- 172.21.15.53:0/3788356322 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3273) v1 ==== 187+0+0 (secure 0 0 0) 0x7f158c057770 con 0x7f159c0ad790 2023-10-21T17:40:32.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 msgr2=0x7f158006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 0x7f158006d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1598036000 tx=0x7f1598008030 comp rx=0 tx=0).stop 2023-10-21T17:40:32.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 msgr2=0x7f159c15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c4a0 tx=0x7f158c00c960 comp rx=0 tx=0).stop 2023-10-21T17:40:32.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 shutdown_connections 2023-10-21T17:40:32.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f158006b0e0 0x7f158006d5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f159c159ac0 0x7f159c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f159c0ad790 0x7f159c15bcb0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 --2- 172.21.15.53:0/3788356322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f159c0709e0 0x7f159c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.393+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 >> 172.21.15.53:0/3788356322 conn(0x7f159c06a380 msgr2=0x7f159c0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:32.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.394+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 shutdown_connections 2023-10-21T17:40:32.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.394+0000 7f15a3b3a640 1 -- 172.21.15.53:0/3788356322 wait complete. 2023-10-21T17:40:32.412 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-10-21T17:40:32.412 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-21T17:40:32.413 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-21T17:40:32.413 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.622+0000 7f22e6694640 1 Processor -- start 2023-10-21T17:40:32.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.623+0000 7f22e6694640 1 -- start start 2023-10-21T17:40:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.624+0000 7f22e6694640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e00ad790 0x7f22e00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.624+0000 7f22e6694640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22e015e330 0x7f22e0070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.624+0000 7f22e6694640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e006dee0 con 0x7f22e0159ac0 2023-10-21T17:40:32.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.624+0000 7f22e6694640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e006dcf0 con 0x7f22e015e330 2023-10-21T17:40:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.624+0000 7f22e6694640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e015d850 con 0x7f22e00ad790 2023-10-21T17:40:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22e015e330 0x7f22e0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f22e015e330 0x7f22e0070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46422/0 (socket says 172.21.15.53:46422) 2023-10-21T17:40:32.627 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 -- 172.21.15.53:0/1802898411 learned_addr learned my addr 172.21.15.53:0/1802898411 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 -- 172.21.15.53:0/1802898411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e00ad790 msgr2=0x7f22e00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 --2- 172.21.15.53:0/1802898411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e00ad790 0x7f22e00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 -- 172.21.15.53:0/1802898411 >> v1:172.21.15.124:6789/0 conn(0x7f22e0159ac0 legacy=0x7f22e006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.625+0000 7f22e4c31640 1 -- 172.21.15.53:0/1802898411 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e015db30 con 0x7f22e015e330 2023-10-21T17:40:32.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.626+0000 7f22e4c31640 1 --2- 172.21.15.53:0/1802898411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e0070810 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d802db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ab7b97edd440af server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22deffd640 1 -- 172.21.15.53:0/1802898411 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d802f8c0 con 0x7f22e015e330 2023-10-21T17:40:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22deffd640 1 -- 172.21.15.53:0/1802898411 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d802fa20 con 0x7f22e015e330 2023-10-21T17:40:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22deffd640 1 -- 172.21.15.53:0/1802898411 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d802fdb0 con 0x7f22e015e330 2023-10-21T17:40:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 -- 172.21.15.53:0/1802898411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 msgr2=0x7f22e0070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 --2- 172.21.15.53:0/1802898411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e0070810 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d802db00 comp rx=0 tx=0).stop 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 -- 172.21.15.53:0/1802898411 shutdown_connections 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 --2- 172.21.15.53:0/1802898411 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e00ad790 0x7f22e00702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 --2- 172.21.15.53:0/1802898411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e0070810 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 -- 172.21.15.53:0/1802898411 >> 172.21.15.53:0/1802898411 conn(0x7f22e006a380 msgr2=0x7f22e00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.627+0000 7f22e6694640 1 -- 172.21.15.53:0/1802898411 shutdown_connections 2023-10-21T17:40:32.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.628+0000 7f22e6694640 1 -- 172.21.15.53:0/1802898411 wait complete. 2023-10-21T17:40:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.629+0000 7f22e6694640 1 Processor -- start 2023-10-21T17:40:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.629+0000 7f22e6694640 1 -- start start 2023-10-21T17:40:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 0x7f22e01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e0159ac0 0x7f22e014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e014f310 con 0x7f22e015e330 2023-10-21T17:40:32.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e014f5f0 con 0x7f22e00ad790 2023-10-21T17:40:32.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.630+0000 7f22e6694640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e014f8d0 con 0x7f22e0159ac0 2023-10-21T17:40:32.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46430/0 (socket says 172.21.15.53:46430) 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 -- 172.21.15.53:0/2517957265 learned_addr learned my addr 172.21.15.53:0/2517957265 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22dffff640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 0x7f22e01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22df7fe640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e0159ac0 0x7f22e014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 -- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e0159ac0 msgr2=0x7f22e014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e0159ac0 0x7f22e014eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 -- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 msgr2=0x7f22e01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 0x7f22e01503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22e4c31640 1 -- 172.21.15.53:0/2517957265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80094d0 con 0x7f22e015e330 2023-10-21T17:40:32.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.631+0000 7f22dffff640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 0x7f22e01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:32.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22e4c31640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8037be0 con 0x7f22e015e330 2023-10-21T17:40:32.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8037d40 con 0x7f22e015e330 2023-10-21T17:40:32.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e014fbb0 con 0x7f22e015e330 2023-10-21T17:40:32.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d8040660 con 0x7f22e015e330 2023-10-21T17:40:32.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.632+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e014f020 con 0x7f22e015e330 2023-10-21T17:40:32.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.633+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e01500d0 con 0x7f22e015e330 2023-10-21T17:40:32.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.634+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f22d80407c0 con 0x7f22e015e330 2023-10-21T17:40:32.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.635+0000 7f22dcff9640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 0x7f22c006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:32.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.635+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f22d8033070 con 0x7f22e015e330 2023-10-21T17:40:32.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.636+0000 7f22dffff640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 0x7f22c006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:32.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.637+0000 7f22dffff640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 0x7f22c006d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22d0001dc0 tx=0x7f22d0009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:32.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.639+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22e01500d0 con 0x7f22e015e330 2023-10-21T17:40:32.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.832+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e01500d0 con 0x7f22e015e330 2023-10-21T17:40:32.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.833+0000 7f22dcff9640 1 -- 172.21.15.53:0/2517957265 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v266) v1 ==== 74+0+14775 (secure 0 0 0) 0x7f22e01500d0 con 0x7f22e015e330 2023-10-21T17:40:32.835 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:32.835 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":266,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:30.382190+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.53:6835/1218730184":"2023-10-22T17:40:26.738076+0000","172.21.15.53:6833/1218730184":"2023-10-22T17:40:26.738076+0000","172.21.15.124:6836/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:6834/2789835635":"2023-10-22T17:40:18.828174+0000","172.21.15.124:0/3510583289":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6833/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/2228122799":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/320015324":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/2321799593":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1514005818":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3074714843":"2023-10-22T17:40:06.671249+0000","172.21.15.124:6833/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1386309274":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/920825767":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/1751665876":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/3189508227":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/959805027":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2467387267":"2023-10-22T17:40:12.899681+0000","172.21.15.124:0/1957744482":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/844552797":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2190223666":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/4116456697":"2023-10-22T17:40:12.899681+0000","172.21.15.124:6832/3816427222":"2023-10-22T17:40:06.671249+0000","172.21.15.124:0/2179186961":"2023-10-22T17:40:06.671249+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 msgr2=0x7f22c006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 0x7f22c006d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22d0001dc0 tx=0x7f22d0009040 comp rx=0 tx=0).stop 2023-10-21T17:40:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 msgr2=0x7f22e00aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d80046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 shutdown_connections 2023-10-21T17:40:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f22c006b0b0 0x7f22c006d570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22e015e330 0x7f22e00aee30 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22e0159ac0 0x7f22e014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 --2- 172.21.15.53:0/2517957265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22e00ad790 0x7f22e01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 >> 172.21.15.53:0/2517957265 conn(0x7f22e006a380 msgr2=0x7f22e006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 shutdown_connections 2023-10-21T17:40:32.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:32.836+0000 7f22e6694640 1 -- 172.21.15.53:0/2517957265 wait complete. 2023-10-21T17:40:32.854 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:40:32.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:40:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.075+0000 7f97fee02640 1 Processor -- start 2023-10-21T17:40:33.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.076+0000 7f97fee02640 1 -- start start 2023-10-21T17:40:33.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fee02640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97f81432f0 0x7f97f80cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fee02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 0x7f97f8159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fee02640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80cbb30 con 0x7f97f80cb6b0 2023-10-21T17:40:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fee02640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f81436d0 con 0x7f97f81432f0 2023-10-21T17:40:33.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fee02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815d440 con 0x7f97f80b4240 2023-10-21T17:40:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fcb9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97f81432f0 0x7f97f80cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fcb9e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f97f81432f0 0x7f97f80cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46446/0 (socket says 172.21.15.53:46446) 2023-10-21T17:40:33.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.077+0000 7f97fcb9e640 1 -- 172.21.15.53:0/3122742317 learned_addr learned my addr 172.21.15.53:0/3122742317 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fd39f640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 0x7f97f8159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fcb9e640 1 -- 172.21.15.53:0/3122742317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 msgr2=0x7f97f8159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:33.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fcb9e640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 0x7f97f8159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fcb9e640 1 -- 172.21.15.53:0/3122742317 >> v1:172.21.15.124:6789/0 conn(0x7f97f80cb6b0 legacy=0x7f97f80b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fcb9e640 1 -- 172.21.15.53:0/3122742317 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f815d690 con 0x7f97f81432f0 2023-10-21T17:40:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.078+0000 7f97fd39f640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 0x7f97f8159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:33.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.079+0000 7f97fcb9e640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80cb170 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e80 tx=0x7f97e802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd3ea17d00cb5a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.079+0000 7f97f77fe640 1 -- 172.21.15.53:0/3122742317 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e802c9c0 con 0x7f97f81432f0 2023-10-21T17:40:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.079+0000 7f97f77fe640 1 -- 172.21.15.53:0/3122742317 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e802cb20 con 0x7f97f81432f0 2023-10-21T17:40:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.079+0000 7f97f77fe640 1 -- 172.21.15.53:0/3122742317 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80343f0 con 0x7f97f81432f0 2023-10-21T17:40:33.084 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 -- 172.21.15.53:0/3122742317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 msgr2=0x7f97f80cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80cb170 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009e80 tx=0x7f97e802be20 comp rx=0 tx=0).stop 2023-10-21T17:40:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 -- 172.21.15.53:0/3122742317 shutdown_connections 2023-10-21T17:40:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80b4240 0x7f97f8159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 --2- 172.21.15.53:0/3122742317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80cb170 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 -- 172.21.15.53:0/3122742317 >> 172.21.15.53:0/3122742317 conn(0x7f97f8069d50 msgr2=0x7f97f806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 -- 172.21.15.53:0/3122742317 shutdown_connections 2023-10-21T17:40:33.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.080+0000 7f97fee02640 1 -- 172.21.15.53:0/3122742317 wait complete. 2023-10-21T17:40:33.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.081+0000 7f97fee02640 1 Processor -- start 2023-10-21T17:40:33.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.082+0000 7f97fee02640 1 -- start start 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.082+0000 7f97fee02640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97f80b4240 0x7f97f8133330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fee02640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80cb6b0 0x7f97f8157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fee02640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fee02640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80ca640 con 0x7f97f81432f0 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fee02640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8139ff0 con 0x7f97f80b4240 2023-10-21T17:40:33.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fee02640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c9da0 con 0x7f97f80cb6b0 2023-10-21T17:40:33.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46450/0 (socket says 172.21.15.53:46450) 2023-10-21T17:40:33.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 -- 172.21.15.53:0/1125296488 learned_addr learned my addr 172.21.15.53:0/1125296488 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:33.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 -- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80cb6b0 msgr2=0x7f97f8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80cb6b0 0x7f97f8157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 -- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97f80b4240 msgr2=0x7f97f8133330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97f80b4240 0x7f97f8133330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.083+0000 7f97fd39f640 1 -- 172.21.15.53:0/1125296488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009b30 con 0x7f97f81432f0 2023-10-21T17:40:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97fd39f640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c7f0 tx=0x7f97f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007820 con 0x7f97f81432f0 2023-10-21T17:40:33.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0007980 con 0x7f97f81432f0 2023-10-21T17:40:33.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007d70 con 0x7f97f81432f0 2023-10-21T17:40:33.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80ca080 con 0x7f97f81432f0 2023-10-21T17:40:33.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.084+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f80cb0b0 con 0x7f97f81432f0 2023-10-21T17:40:33.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.085+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f804e160 con 0x7f97f81432f0 2023-10-21T17:40:33.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.086+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f97f00113f0 con 0x7f97f81432f0 2023-10-21T17:40:33.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.087+0000 7f97f57fa640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 0x7f97d806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.087+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f97f008a100 con 0x7f97f81432f0 2023-10-21T17:40:33.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.088+0000 7f97fcb9e640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 0x7f97d806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.089+0000 7f97fcb9e640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 0x7f97d806d5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97e8035000 tx=0x7f97e802e000 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.091+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97f0054ac0 con 0x7f97f81432f0 2023-10-21T17:40:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.276+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97f813a160 con 0x7f97f81432f0 2023-10-21T17:40:33.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.276+0000 7f97f57fa640 1 -- 172.21.15.53:0/1125296488 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v266) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97f00580e0 con 0x7f97f81432f0 2023-10-21T17:40:33.278 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:40:33.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 msgr2=0x7f97d806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 0x7f97d806d5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97e8035000 tx=0x7f97e802e000 comp rx=0 tx=0).stop 2023-10-21T17:40:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 msgr2=0x7f97f80c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f97f000c7f0 tx=0x7f97f000ccb0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 shutdown_connections 2023-10-21T17:40:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f97d806b0e0 0x7f97d806d5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f97f81432f0 0x7f97f80c92e0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97f80cb6b0 0x7f97f8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 --2- 172.21.15.53:0/1125296488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97f80b4240 0x7f97f8133330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 >> 172.21.15.53:0/1125296488 conn(0x7f97f8069d50 msgr2=0x7f97f8070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 shutdown_connections 2023-10-21T17:40:33.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.279+0000 7f97fee02640 1 -- 172.21.15.53:0/1125296488 wait complete. 2023-10-21T17:40:33.297 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-21T17:40:33.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.504+0000 7ffacffff640 1 Processor -- start 2023-10-21T17:40:33.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.504+0000 7ffacffff640 1 -- start start 2023-10-21T17:40:33.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.505+0000 7ffacffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 0x7ffad00cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.507 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.505+0000 7ffacffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffad00cb990 0x7ffad00ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.505+0000 7ffacffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad006cfd0 con 0x7ffad006dd00 2023-10-21T17:40:33.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.505+0000 7ffacffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad00cbe20 con 0x7ffad00cb990 2023-10-21T17:40:33.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.505+0000 7ffacffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad006d870 con 0x7ffad015a970 2023-10-21T17:40:33.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffad00cb990 0x7ffad00ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffad00cb990 0x7ffad00ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46452/0 (socket says 172.21.15.53:46452) 2023-10-21T17:40:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 -- 172.21.15.53:0/1378960974 learned_addr learned my addr 172.21.15.53:0/1378960974 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffaceffd640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 0x7ffad00cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 -- 172.21.15.53:0/1378960974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 msgr2=0x7ffad00cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 0x7ffad00cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 -- 172.21.15.53:0/1378960974 >> v1:172.21.15.124:6789/0 conn(0x7ffad006dd00 legacy=0x7ffad00c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:33.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffacf7fe640 1 -- 172.21.15.53:0/1378960974 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad015ad50 con 0x7ffad00cb990 2023-10-21T17:40:33.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.506+0000 7ffaceffd640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 0x7ffad00cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:33.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.507+0000 7ffacf7fe640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad00cb990 0x7ffad00ca570 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c22f56925daabf server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacdffb640 1 -- 172.21.15.53:0/1378960974 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac802c960 con 0x7ffad00cb990 2023-10-21T17:40:33.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacdffb640 1 -- 172.21.15.53:0/1378960974 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac802cac0 con 0x7ffad00cb990 2023-10-21T17:40:33.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacdffb640 1 -- 172.21.15.53:0/1378960974 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac802ce50 con 0x7ffad00cb990 2023-10-21T17:40:33.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 -- 172.21.15.53:0/1378960974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad00cb990 msgr2=0x7ffad00ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad00cb990 0x7ffad00ca570 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802bdc0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 -- 172.21.15.53:0/1378960974 shutdown_connections 2023-10-21T17:40:33.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad00cb990 0x7ffad00ca570 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 --2- 172.21.15.53:0/1378960974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad015a970 0x7ffad00cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 -- 172.21.15.53:0/1378960974 >> 172.21.15.53:0/1378960974 conn(0x7ffad00ce9d0 msgr2=0x7ffad00cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:33.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 -- 172.21.15.53:0/1378960974 shutdown_connections 2023-10-21T17:40:33.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.508+0000 7ffacffff640 1 -- 172.21.15.53:0/1378960974 wait complete. 2023-10-21T17:40:33.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.510+0000 7ffacffff640 1 Processor -- start 2023-10-21T17:40:33.515 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.510+0000 7ffacffff640 1 -- start start 2023-10-21T17:40:33.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 0x7ffad01f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 0x7ffad00c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.518 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00c54b0 con 0x7ffad006dd00 2023-10-21T17:40:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01f64c0 con 0x7ffad015a970 2023-10-21T17:40:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.511+0000 7ffacffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01f1ed0 con 0x7ffad00cb990 2023-10-21T17:40:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:46454/0 (socket says 172.21.15.53:46454) 2023-10-21T17:40:33.519 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 -- 172.21.15.53:0/2769183888 learned_addr learned my addr 172.21.15.53:0/2769183888 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:33.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffacf7fe640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 0x7ffad00c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 -- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 msgr2=0x7ffad01f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7fface7fc640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 0x7ffad01f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 0x7ffad01f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 -- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 msgr2=0x7ffad00c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:33.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 0x7ffad00c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:33.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 -- 172.21.15.53:0/2769183888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8009b30 con 0x7ffad006dd00 2023-10-21T17:40:33.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffacf7fe640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 0x7ffad00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:33.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7fface7fc640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 0x7ffad01f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:33.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.512+0000 7ffaceffd640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c7f0 tx=0x7ffac000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.513+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00077c0 con 0x7ffad006dd00 2023-10-21T17:40:33.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.513+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0007920 con 0x7ffad006dd00 2023-10-21T17:40:33.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.513+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0007cb0 con 0x7ffad006dd00 2023-10-21T17:40:33.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.513+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00cc970 con 0x7ffad006dd00 2023-10-21T17:40:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.513+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00cc320 con 0x7ffad006dd00 2023-10-21T17:40:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.514+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad00cc180 con 0x7ffad006dd00 2023-10-21T17:40:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.515+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffac00113f0 con 0x7ffad006dd00 2023-10-21T17:40:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.516+0000 7ffab7fff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 0x7ffabc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:33.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.516+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7ffac008a120 con 0x7ffad006dd00 2023-10-21T17:40:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.517+0000 7fface7fc640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 0x7ffabc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:33.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.518+0000 7fface7fc640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 0x7ffabc06d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b10f0 tx=0x7ffac4008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:33.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.520+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffad00cc180 con 0x7ffad006dd00 2023-10-21T17:40:33.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:33.783+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7ffad00cc180 con 0x7ffad006dd00 2023-10-21T17:40:34.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.404+0000 7ffab7fff640 1 -- 172.21.15.53:0/2769183888 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3275) v1 ==== 158+0+0 (secure 0 0 0) 0x7ffad00cc180 con 0x7ffad006dd00 2023-10-21T17:40:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 msgr2=0x7ffabc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 0x7ffabc06d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffad00b10f0 tx=0x7ffac4008040 comp rx=0 tx=0).stop 2023-10-21T17:40:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 msgr2=0x7ffad01f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c7f0 tx=0x7ffac000cbf0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 shutdown_connections 2023-10-21T17:40:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ffabc06b0b0 0x7ffabc06d570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.407+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffad015a970 0x7ffad00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.408+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffad00cb990 0x7ffad01f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.408+0000 7ffacffff640 1 --2- 172.21.15.53:0/2769183888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffad006dd00 0x7ffad01f5f80 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.408+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 >> 172.21.15.53:0/2769183888 conn(0x7ffad00ce9d0 msgr2=0x7ffad00b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.408+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 shutdown_connections 2023-10-21T17:40:34.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.408+0000 7ffacffff640 1 -- 172.21.15.53:0/2769183888 wait complete. 2023-10-21T17:40:34.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-21T17:40:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.642+0000 7f915f20e640 1 Processor -- start 2023-10-21T17:40:34.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.643+0000 7f915f20e640 1 -- start start 2023-10-21T17:40:34.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915f20e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91580afbe0 0x7f91580b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:34.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915f20e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580b2620 con 0x7f91580cb130 2023-10-21T17:40:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915f20e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580b2060 con 0x7f915806dfc0 2023-10-21T17:40:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915f20e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91580b2340 con 0x7f91580afbe0 2023-10-21T17:40:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915cfaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91580afbe0 0x7f91580b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915cfaa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f91580afbe0 0x7f91580b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44236/0 (socket says 172.21.15.53:44236) 2023-10-21T17:40:34.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915cfaa640 1 -- 172.21.15.53:0/3428980448 learned_addr learned my addr 172.21.15.53:0/3428980448 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:34.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.644+0000 7f915d7ab640 1 --1- 172.21.15.53:0/3428980448 >> v1:172.21.15.53:6789/0 conn(0x7f91580cb130 0x7f915806c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37256/0 (socket says 172.21.15.53:37256) 2023-10-21T17:40:34.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.645+0000 7f915cfaa640 1 -- 172.21.15.53:0/3428980448 >> v1:172.21.15.124:6789/0 conn(0x7f915806dfc0 legacy=0x7f9158070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:34.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.645+0000 7f915cfaa640 1 -- 172.21.15.53:0/3428980448 >> v1:172.21.15.53:6789/0 conn(0x7f91580cb130 legacy=0x7f915806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.645+0000 7f915cfaa640 1 -- 172.21.15.53:0/3428980448 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158070350 con 0x7f91580afbe0 2023-10-21T17:40:34.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f915cfaa640 1 --2- 172.21.15.53:0/3428980448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91580afbe0 0x7f91580b15b0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec66909ad1b3e95 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:34.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f91577fe640 1 -- 172.21.15.53:0/3428980448 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c030860 con 0x7f91580afbe0 2023-10-21T17:40:34.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f91577fe640 1 -- 172.21.15.53:0/3428980448 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c0309c0 con 0x7f91580afbe0 2023-10-21T17:40:34.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f91577fe640 1 -- 172.21.15.53:0/3428980448 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c030d50 con 0x7f91580afbe0 2023-10-21T17:40:34.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f915f20e640 1 -- 172.21.15.53:0/3428980448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91580afbe0 msgr2=0x7f91580b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.646+0000 7f915f20e640 1 --2- 172.21.15.53:0/3428980448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91580afbe0 0x7f91580b15b0 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c02b9a0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.647+0000 7f915f20e640 1 -- 172.21.15.53:0/3428980448 shutdown_connections 2023-10-21T17:40:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.647+0000 7f915f20e640 1 --2- 172.21.15.53:0/3428980448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91580afbe0 0x7f91580b15b0 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.647+0000 7f915f20e640 1 -- 172.21.15.53:0/3428980448 >> 172.21.15.53:0/3428980448 conn(0x7f915806a5d0 msgr2=0x7f91580b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:34.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.647+0000 7f915f20e640 1 -- 172.21.15.53:0/3428980448 shutdown_connections 2023-10-21T17:40:34.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.647+0000 7f915f20e640 1 -- 172.21.15.53:0/3428980448 wait complete. 2023-10-21T17:40:34.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.648+0000 7f915f20e640 1 Processor -- start 2023-10-21T17:40:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.649+0000 7f915f20e640 1 -- start start 2023-10-21T17:40:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.649+0000 7f915f20e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.649+0000 7f915f20e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91580afbe0 0x7f91580c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915f20e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91580cb130 0x7f91580ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:34.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915f20e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915815e000 con 0x7f915806dfc0 2023-10-21T17:40:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915f20e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915815d3c0 con 0x7f91580cb130 2023-10-21T17:40:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915f20e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915815d660 con 0x7f91580afbe0 2023-10-21T17:40:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44244/0 (socket says 172.21.15.53:44244) 2023-10-21T17:40:34.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 -- 172.21.15.53:0/261615538 learned_addr learned my addr 172.21.15.53:0/261615538 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:34.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 -- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91580afbe0 msgr2=0x7f91580c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91580afbe0 0x7f91580c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 -- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91580cb130 msgr2=0x7f91580ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91580cb130 0x7f91580ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 -- 172.21.15.53:0/261615538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c0094d0 con 0x7f915806dfc0 2023-10-21T17:40:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.650+0000 7f915cfaa640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.651+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c030810 con 0x7f915806dfc0 2023-10-21T17:40:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.651+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c043400 con 0x7f915806dfc0 2023-10-21T17:40:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.651+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c043790 con 0x7f915806dfc0 2023-10-21T17:40:34.656 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.651+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915815d940 con 0x7f915806dfc0 2023-10-21T17:40:34.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.651+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580cd2f0 con 0x7f915806dfc0 2023-10-21T17:40:34.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.652+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915804e160 con 0x7f915806dfc0 2023-10-21T17:40:34.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.653+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f914c0438f0 con 0x7f915806dfc0 2023-10-21T17:40:34.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.654+0000 7f91557fa640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 0x7f913c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:34.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.655+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f914c03e070 con 0x7f915806dfc0 2023-10-21T17:40:34.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.655+0000 7f9157fff640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 0x7f913c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:34.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.656+0000 7f9157fff640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 0x7f913c06d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f91580cc840 tx=0x7f9144006ba0 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:34.660 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.659+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f914c077d80 con 0x7f915806dfc0 2023-10-21T17:40:34.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.848+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f91580c5280 con 0x7f915806dfc0 2023-10-21T17:40:34.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.854+0000 7f91557fa640 1 -- 172.21.15.53:0/261615538 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v88)=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f914c07b3a0 con 0x7f915806dfc0 2023-10-21T17:40:34.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 msgr2=0x7f913c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 0x7f913c06d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f91580cc840 tx=0x7f9144006ba0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 msgr2=0x7f9158154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:34.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f914c009870 tx=0x7f914c004730 comp rx=0 tx=0).stop 2023-10-21T17:40:34.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 shutdown_connections 2023-10-21T17:40:34.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f913c06b0b0 0x7f913c06d570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91580cb130 0x7f91580ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91580afbe0 0x7f91580c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 --2- 172.21.15.53:0/261615538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f915806dfc0 0x7f9158154f60 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:34.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 >> 172.21.15.53:0/261615538 conn(0x7f915806a5d0 msgr2=0x7f91580c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:34.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 shutdown_connections 2023-10-21T17:40:34.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:34.857+0000 7f915f20e640 1 -- 172.21.15.53:0/261615538 wait complete. 2023-10-21T17:40:34.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:40:34.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:40:34.994 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:40:35.115 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:40:35.115 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:35.116 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:35.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:40:35.179 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:40:35.180 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:35.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:40:35.240 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:35.240 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:35.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:40:35.303 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:40:35.303 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:35.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:40:35.361 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-21T17:40:35.362 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:35.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.597+0000 7fe9088ad640 1 Processor -- start 2023-10-21T17:40:35.599 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.598+0000 7fe9088ad640 1 -- start start 2023-10-21T17:40:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.599+0000 7fe9088ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe9041648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.599+0000 7fe9088ad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040cacf0 con 0x7fe9041599d0 2023-10-21T17:40:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.599+0000 7fe9088ad640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040ca3c0 con 0x7fe9040cc210 2023-10-21T17:40:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.599+0000 7fe9088ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040b10f0 con 0x7fe9040b07d0 2023-10-21T17:40:35.601 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9037fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe9041599d0 0x7fe9040ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37268/0 (socket says 172.21.15.53:37268) 2023-10-21T17:40:35.602 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9037fe640 1 -- 172.21.15.53:0/3331459366 learned_addr learned my addr 172.21.15.53:0/3331459366 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:35.603 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844119329 0 0) 0x7fe9040cacf0 con 0x7fe9041599d0 2023-10-21T17:40:35.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f0003520 con 0x7fe9041599d0 2023-10-21T17:40:35.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe903fff640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe9041648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479262036 0 0) 0x7fe8f0003520 con 0x7fe9041599d0 2023-10-21T17:40:35.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 msgr2=0x7fe9041648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:35.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe9041648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.600+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 >> v1:172.21.15.124:6789/0 conn(0x7fe9040cc210 legacy=0x7fe9040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:35.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040ca6a0 con 0x7fe9041599d0 2023-10-21T17:40:35.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe8f40019f0 con 0x7fe9041599d0 2023-10-21T17:40:35.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 >> v1:172.21.15.53:6789/0 conn(0x7fe9041599d0 legacy=0x7fe9040ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.605 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f00039a0 0x7fe8f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe8f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f0004d00 0x7fe8f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040b10f0 con 0x7fe8f00039a0 2023-10-21T17:40:35.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000bb90 con 0x7fe8f0004d00 2023-10-21T17:40:35.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000bd50 con 0x7fe9040b07d0 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9037fe640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f00039a0 0x7fe8f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe902ffd640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe8f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9037fe640 1 -- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 msgr2=0x7fe8f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9037fe640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe8f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9037fe640 1 -- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f0004d00 msgr2=0x7fe8f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:35.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.601+0000 7fe9037fe640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f0004d00 0x7fe8f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.602+0000 7fe9037fe640 1 -- 172.21.15.53:0/3331459366 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0003520 con 0x7fe8f00039a0 2023-10-21T17:40:35.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.602+0000 7fe902ffd640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040b07d0 0x7fe8f00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:35.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9037fe640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f00039a0 0x7fe8f0003dc0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a750 tx=0x7fe8f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b3bea508874c8d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:35.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4034630 con 0x7fe8f00039a0 2023-10-21T17:40:35.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f402f070 con 0x7fe8f00039a0 2023-10-21T17:40:35.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9027fc640 1 -- 172.21.15.53:0/3331459366 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40349f0 con 0x7fe8f00039a0 2023-10-21T17:40:35.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9088ad640 1 -- 172.21.15.53:0/3331459366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f00039a0 msgr2=0x7fe8f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.611 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.603+0000 7fe9088ad640 1 --2- 172.21.15.53:0/3331459366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8f00039a0 0x7fe8f0003dc0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a750 tx=0x7fe8f00057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.604+0000 7fe903fff640 1 -- 172.21.15.53:0/3331459366 reap_dead start 2023-10-21T17:40:35.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.604+0000 7fe9088ad640 1 -- 172.21.15.53:0/3331459366 shutdown_connections 2023-10-21T17:40:35.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.604+0000 7fe9088ad640 1 -- 172.21.15.53:0/3331459366 >> 172.21.15.53:0/3331459366 conn(0x7fe9040cb0c0 msgr2=0x7fe9040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:35.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.604+0000 7fe9088ad640 1 -- 172.21.15.53:0/3331459366 shutdown_connections 2023-10-21T17:40:35.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.604+0000 7fe9088ad640 1 -- 172.21.15.53:0/3331459366 wait complete. 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.606+0000 7fe9088ad640 1 Processor -- start 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.606+0000 7fe9088ad640 1 -- start start 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.607+0000 7fe9088ad640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 0x7fe9040c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.607+0000 7fe9088ad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.607+0000 7fe9088ad640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 0x7fe904070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe9088ad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041407d0 con 0x7fe9040c5710 2023-10-21T17:40:35.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe9088ad640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904070a90 con 0x7fe8f00039a0 2023-10-21T17:40:35.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe9088ad640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c7480 con 0x7fe9040700d0 2023-10-21T17:40:35.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44260/0 (socket says 172.21.15.53:44260) 2023-10-21T17:40:35.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 -- 172.21.15.53:0/1423763709 learned_addr learned my addr 172.21.15.53:0/1423763709 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe9037fe640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 0x7fe9040c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe903fff640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 0x7fe904070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 -- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 msgr2=0x7fe904070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 0x7fe904070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 -- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 msgr2=0x7fe9040c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 0x7fe9040c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe902ffd640 1 -- 172.21.15.53:0/1423763709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9041404f0 con 0x7fe9040c5710 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe9037fe640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 0x7fe9040c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.608+0000 7fe903fff640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 0x7fe904070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe902ffd640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d7a0 tx=0x7fe8ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec005270 con 0x7fe9040c5710 2023-10-21T17:40:35.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec0053f0 con 0x7fe9040c5710 2023-10-21T17:40:35.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9040c7720 con 0x7fe9040c5710 2023-10-21T17:40:35.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00b8f0 con 0x7fe9040c5710 2023-10-21T17:40:35.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.609+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9040c7ce0 con 0x7fe9040c5710 2023-10-21T17:40:35.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.610+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9040c7ea0 con 0x7fe9040c5710 2023-10-21T17:40:35.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.611+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe8ec00ba50 con 0x7fe9040c5710 2023-10-21T17:40:35.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.612+0000 7fe8fbfff640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 0x7fe8e406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:35.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.613+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7fe8ec089d90 con 0x7fe9040c5710 2023-10-21T17:40:35.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.613+0000 7fe9037fe640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 0x7fe8e406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:35.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.614+0000 7fe9037fe640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 0x7fe8e406d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4034000 tx=0x7fe8f402f000 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:35.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.617+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8ec054760 con 0x7fe9040c5710 2023-10-21T17:40:35.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.850+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9040ce210 con 0x7fe9040c5710 2023-10-21T17:40:35.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.851+0000 7fe8fbfff640 1 -- 172.21.15.53:0/1423763709 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+3712 (secure 0 0 0) 0x7fe8ec057d80 con 0x7fe9040c5710 2023-10-21T17:40:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:35.852 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":138,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/4141679395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":4141679395},{"type":"v1","addr":"172.21.15.124:6837","nonce":4141679395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":9961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2097440682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2097440682},{"type":"v1","addr":"172.21.15.124:6836","nonce":2097440682}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":136},{"gid":11931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6834/1598788675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1598788675},{"type":"v1","addr":"172.21.15.53:6834","nonce":1598788675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":12606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3091584361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6833","nonce":3091584361},{"type":"v1","addr":"172.21.15.53:6835","nonce":3091584361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[]} 2023-10-21T17:40:35.853 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 138 2023-10-21T17:40:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 msgr2=0x7fe8e406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 0x7fe8e406d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4034000 tx=0x7fe8f402f000 comp rx=0 tx=0).stop 2023-10-21T17:40:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 msgr2=0x7fe9040c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:35.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d7a0 tx=0x7fe8ec00dc60 comp rx=0 tx=0).stop 2023-10-21T17:40:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 shutdown_connections 2023-10-21T17:40:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe8e406b0b0 0x7fe8e406d570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9040700d0 0x7fe904070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9040c5710 0x7fe9040c5b90 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.853+0000 7fe9088ad640 1 --2- 172.21.15.53:0/1423763709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8f00039a0 0x7fe9040c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.854+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 >> 172.21.15.53:0/1423763709 conn(0x7fe9040cb0c0 msgr2=0x7fe9040ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.854+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 shutdown_connections 2023-10-21T17:40:35.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:35.854+0000 7fe9088ad640 1 -- 172.21.15.53:0/1423763709 wait complete. 2023-10-21T17:40:35.875 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-21T17:40:35.875 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-21T17:40:35.875 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:40:35.875 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:40:35.876 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-21T17:40:35.876 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-21T17:40:35.876 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:40:35.877 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:40:35.877 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-21T17:40:35.877 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-21T17:40:35.877 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:40:35.878 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:40:35.878 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-21T17:40:35.878 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-21T17:40:35.878 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:40:35.879 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:40:35.899 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:40:35.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-21T17:40:35.901 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:40:35.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-21T17:40:35.909 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:40:35.910 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-21T17:40:35.911 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:40:35.911 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-21T17:40:35.950 INFO:tasks.ceph.mds.c:Started 2023-10-21T17:40:35.950 DEBUG:teuthology.parallel:result is None 2023-10-21T17:40:35.958 INFO:tasks.ceph.mds.a:Started 2023-10-21T17:40:35.958 DEBUG:teuthology.parallel:result is None 2023-10-21T17:40:35.959 INFO:tasks.ceph.mds.b:Started 2023-10-21T17:40:35.959 DEBUG:teuthology.parallel:result is None 2023-10-21T17:40:35.965 INFO:tasks.ceph.mds.d:Started 2023-10-21T17:40:35.965 DEBUG:teuthology.parallel:result is None 2023-10-21T17:40:35.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-21T17:40:36.118 INFO:tasks.ceph.mds.a.smithi053.stdout:starting mds.a at 2023-10-21T17:40:36.120 INFO:tasks.ceph.mds.c.smithi053.stdout:starting mds.c at 2023-10-21T17:40:36.127 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2023-10-21T17:40:36.127 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:40:36.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.161+0000 7f0c73fff640 1 Processor -- start 2023-10-21T17:40:36.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.161+0000 7f0c73fff640 1 -- start start 2023-10-21T17:40:36.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.162+0000 7f0c73fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c740b3c50 0x7f0c740cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:36.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.162+0000 7f0c73fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c74159db0 con 0x7f0c741599d0 2023-10-21T17:40:36.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.162+0000 7f0c73fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c74071fe0 con 0x7f0c740cb5e0 2023-10-21T17:40:36.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.162+0000 7f0c73fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c740b44c0 con 0x7f0c740b3c50 2023-10-21T17:40:36.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c72ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0c741599d0 0x7f0c740caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37282/0 (socket says 172.21.15.53:37282) 2023-10-21T17:40:36.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c72ffd640 1 -- 172.21.15.53:0/2603782847 learned_addr learned my addr 172.21.15.53:0/2603782847 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:36.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c737fe640 1 --2- 172.21.15.53:0/2603782847 >> v2:172.21.15.124:3300/0 conn(0x7f0c740b3c50 0x7f0c740cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:36.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331945076 0 0) 0x7f0c64001110 con 0x7f0c741599d0 2023-10-21T17:40:36.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c60003520 con 0x7f0c741599d0 2023-10-21T17:40:36.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443055115 0 0) 0x7f0c74071fe0 con 0x7f0c740cb5e0 2023-10-21T17:40:36.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c64001110 con 0x7f0c740cb5e0 2023-10-21T17:40:36.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116617402 0 0) 0x7f0c60003520 con 0x7f0c741599d0 2023-10-21T17:40:36.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c737fe640 1 -- 172.21.15.53:0/2603782847 >> v1:172.21.15.124:6789/0 conn(0x7f0c740cb5e0 legacy=0x7f0c740cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:36.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c737fe640 1 -- 172.21.15.53:0/2603782847 >> v1:172.21.15.53:6789/0 conn(0x7f0c741599d0 legacy=0x7f0c740caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:36.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.163+0000 7f0c737fe640 1 -- 172.21.15.53:0/2603782847 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740713d0 con 0x7f0c740b3c50 2023-10-21T17:40:36.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c737fe640 1 --2- 172.21.15.53:0/2603782847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c740cdcd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009d90 tx=0x7f0c6c02be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc011f136b58f98 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:36.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64001800 con 0x7f0c740b3c50 2023-10-21T17:40:36.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c02c950 con 0x7f0c740b3c50 2023-10-21T17:40:36.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c727fc640 1 -- 172.21.15.53:0/2603782847 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c02cce0 con 0x7f0c740b3c50 2023-10-21T17:40:36.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c73fff640 1 -- 172.21.15.53:0/2603782847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 msgr2=0x7f0c740cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:36.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.165+0000 7f0c73fff640 1 --2- 172.21.15.53:0/2603782847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c740cdcd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009d90 tx=0x7f0c6c02be20 comp rx=0 tx=0).stop 2023-10-21T17:40:36.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.166+0000 7f0c73fff640 1 -- 172.21.15.53:0/2603782847 shutdown_connections 2023-10-21T17:40:36.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.166+0000 7f0c73fff640 1 --2- 172.21.15.53:0/2603782847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c740cdcd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:36.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.166+0000 7f0c73fff640 1 -- 172.21.15.53:0/2603782847 >> 172.21.15.53:0/2603782847 conn(0x7f0c7406c050 msgr2=0x7f0c7406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:36.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.166+0000 7f0c73fff640 1 -- 172.21.15.53:0/2603782847 shutdown_connections 2023-10-21T17:40:36.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.166+0000 7f0c73fff640 1 -- 172.21.15.53:0/2603782847 wait complete. 2023-10-21T17:40:36.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.167+0000 7f0c73fff640 1 Processor -- start 2023-10-21T17:40:36.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.168+0000 7f0c73fff640 1 -- start start 2023-10-21T17:40:36.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.168+0000 7f0c73fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c74156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:36.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.168+0000 7f0c73fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740cb5e0 0x7f0c74159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:36.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c73fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:36.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c73fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7415b540 con 0x7f0c741599d0 2023-10-21T17:40:36.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c73fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74156fe0 con 0x7f0c740b3c50 2023-10-21T17:40:36.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c73fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7415a930 con 0x7f0c740cb5e0 2023-10-21T17:40:36.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:36.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44306/0 (socket says 172.21.15.53:44306) 2023-10-21T17:40:36.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 -- 172.21.15.53:0/976807420 learned_addr learned my addr 172.21.15.53:0/976807420 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:36.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c72ffd640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c74156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:36.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c6bfff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740cb5e0 0x7f0c74159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:36.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 -- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740cb5e0 msgr2=0x7f0c74159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:36.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740cb5e0 0x7f0c74159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:36.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 -- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 msgr2=0x7f0c74156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:36.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c74156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.169+0000 7f0c737fe640 1 -- 172.21.15.53:0/976807420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c009a70 con 0x7f0c741599d0 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c737fe640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009d90 tx=0x7f0c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c009450 con 0x7f0c741599d0 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c034c50 con 0x7f0c741599d0 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c003990 con 0x7f0c741599d0 2023-10-21T17:40:36.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7415ac10 con 0x7f0c741599d0 2023-10-21T17:40:36.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.170+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7415b1d0 con 0x7f0c741599d0 2023-10-21T17:40:36.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.171+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7415b6b0 con 0x7f0c741599d0 2023-10-21T17:40:36.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.172+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0c6c040450 con 0x7f0c741599d0 2023-10-21T17:40:36.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.173+0000 7f0c6b7fe640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 0x7f0c5406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:36.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.173+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f0c6c0ac680 con 0x7f0c741599d0 2023-10-21T17:40:36.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.174+0000 7f0c72ffd640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 0x7f0c5406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:36.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.174+0000 7f0c72ffd640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 0x7f0c5406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c64001390 tx=0x7f0c640012c0 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:36.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.177+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c6c077200 con 0x7f0c741599d0 2023-10-21T17:40:36.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:36.383+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0c74074150 con 0x7f0c741599d0 2023-10-21T17:40:37.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.439+0000 7f0c6b7fe640 1 -- 172.21.15.53:0/976807420 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v271) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0c6c07a820 con 0x7f0c741599d0 2023-10-21T17:40:37.441 INFO:teuthology.orchestra.run.smithi053.stderr: removed all blocklist entries 2023-10-21T17:40:37.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.442+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 msgr2=0x7f0c5406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.442+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 0x7f0c5406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c64001390 tx=0x7f0c640012c0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.444 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 msgr2=0x7f0c740ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009d90 tx=0x7f0c6c004a40 comp rx=0 tx=0).stop 2023-10-21T17:40:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 shutdown_connections 2023-10-21T17:40:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f0c5406b0b0 0x7f0c5406d570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c741599d0 0x7f0c740ae900 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c740cb5e0 0x7f0c74159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 --2- 172.21.15.53:0/976807420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c740b3c50 0x7f0c74156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 >> 172.21.15.53:0/976807420 conn(0x7f0c7406c050 msgr2=0x7f0c740b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:37.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 shutdown_connections 2023-10-21T17:40:37.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.443+0000 7f0c73fff640 1 -- 172.21.15.53:0/976807420 wait complete. 2023-10-21T17:40:37.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-21T17:40:37.691 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.690+0000 7f462aa21640 1 Processor -- start 2023-10-21T17:40:37.692 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.690+0000 7f462aa21640 1 -- start start 2023-10-21T17:40:37.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462aa21640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f46240cc210 0x7f46240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462aa21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4624159db0 0x7f46240b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462aa21640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240cacf0 con 0x7f46241599d0 2023-10-21T17:40:37.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462aa21640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240b10f0 con 0x7f46240cc210 2023-10-21T17:40:37.694 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462aa21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240ade40 con 0x7f4624159db0 2023-10-21T17:40:37.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.692+0000 7f462921e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f46240cc210 0x7f46240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f46240cc210 0x7f46240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44310/0 (socket says 172.21.15.53:44310) 2023-10-21T17:40:37.695 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 -- 172.21.15.53:0/1335007266 learned_addr learned my addr 172.21.15.53:0/1335007266 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462a220640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4624159db0 0x7f46240b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 -- 172.21.15.53:0/1335007266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4624159db0 msgr2=0x7f46240b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4624159db0 0x7f46240b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 -- 172.21.15.53:0/1335007266 >> v1:172.21.15.124:6789/0 conn(0x7f46241599d0 legacy=0x7f46240ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.693+0000 7f462921e640 1 -- 172.21.15.53:0/1335007266 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240ca3c0 con 0x7f46240cc210 2023-10-21T17:40:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.694+0000 7f462921e640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f46240b00c0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f4614009870 tx=0x7f461402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e5d52ec88a1461 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f4628a1d640 1 -- 172.21.15.53:0/1335007266 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614030840 con 0x7f46240cc210 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f4628a1d640 1 -- 172.21.15.53:0/1335007266 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46140309a0 con 0x7f46240cc210 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f4628a1d640 1 -- 172.21.15.53:0/1335007266 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614030d30 con 0x7f46240cc210 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 -- 172.21.15.53:0/1335007266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 msgr2=0x7f46240b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f46240b00c0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f4614009870 tx=0x7f461402b980 comp rx=0 tx=0).stop 2023-10-21T17:40:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 -- 172.21.15.53:0/1335007266 shutdown_connections 2023-10-21T17:40:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4624159db0 0x7f46240b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 --2- 172.21.15.53:0/1335007266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f46240b00c0 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 -- 172.21.15.53:0/1335007266 >> 172.21.15.53:0/1335007266 conn(0x7f46240cb0c0 msgr2=0x7f46240cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.695+0000 7f462aa21640 1 -- 172.21.15.53:0/1335007266 shutdown_connections 2023-10-21T17:40:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.696+0000 7f462aa21640 1 -- 172.21.15.53:0/1335007266 wait complete. 2023-10-21T17:40:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.697+0000 7f462aa21640 1 Processor -- start 2023-10-21T17:40:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 -- start start 2023-10-21T17:40:37.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46241599d0 0x7f4624153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4624159db0 0x7f46240aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624150310 con 0x7f46240cc210 2023-10-21T17:40:37.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624151650 con 0x7f4624159db0 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.698+0000 7f462aa21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46241500c0 con 0x7f46241599d0 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44326/0 (socket says 172.21.15.53:44326) 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 -- 172.21.15.53:0/227673773 learned_addr learned my addr 172.21.15.53:0/227673773 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f462921e640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46241599d0 0x7f4624153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f462a220640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4624159db0 0x7f46240aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 -- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46241599d0 msgr2=0x7f4624153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46241599d0 0x7f4624153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 -- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4624159db0 msgr2=0x7f46240aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4624159db0 0x7f46240aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.699+0000 7f4629a1f640 1 -- 172.21.15.53:0/227673773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46140094d0 con 0x7f46240cc210 2023-10-21T17:40:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f4629a1f640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f462000c450 tx=0x7f462000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:37.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200052d0 con 0x7f46240cc210 2023-10-21T17:40:37.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4620005430 con 0x7f46240cc210 2023-10-21T17:40:37.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620012870 con 0x7f46240cc210 2023-10-21T17:40:37.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46241505b0 con 0x7f46240cc210 2023-10-21T17:40:37.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.700+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4624150b70 con 0x7f46240cc210 2023-10-21T17:40:37.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.701+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462404e160 con 0x7f46240cc210 2023-10-21T17:40:37.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.702+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f46200129d0 con 0x7f46240cc210 2023-10-21T17:40:37.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.703+0000 7f461a7fc640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 0x7f460806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:37.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.703+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f46200893b0 con 0x7f46240cc210 2023-10-21T17:40:37.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.704+0000 7f462921e640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 0x7f460806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:37.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.705+0000 7f462921e640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 0x7f460806d5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4614036000 tx=0x7f4614008030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:37.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.707+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4620054290 con 0x7f46240cc210 2023-10-21T17:40:37.970 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.969+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f46240cb4a0 con 0x7f46240cc210 2023-10-21T17:40:37.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.970+0000 7f461a7fc640 1 -- 172.21.15.53:0/227673773 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f46200578b0 con 0x7f46240cc210 2023-10-21T17:40:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:37.972 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:40:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_dump": [ 2023-10-21T17:40:37.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.a", 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlaBQjDhAACnRToyyymM6xShdqKpjj8Q==", 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:40:37.973 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:40:37.974 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.974 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.b", 2023-10-21T17:40:37.975 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlcJbCMhAAzP6vJPe1wVo48zLeKGZarQ==", 2023-10-21T17:40:37.975 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:40:37.975 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:40:37.976 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.976 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.c", 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNlhdjGIBAAuHdFEjhhUJp+h4LHsE5gJA==", 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:40:37.977 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mds.d", 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlnAbiBxAA2Vu8mwKu+kBDu3fdb4lwHw==", 2023-10-21T17:40:37.978 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:40:37.979 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:40:37.979 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.979 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.0", 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC29TNl2KiKARAAldJDYEEr26dyLlEiomY/Kw==", 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.980 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.981 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.981 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.1", 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC59TNlz67BGRAASEJUNZRRTaZtIcwvIMl8YQ==", 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.982 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.2", 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQC89TNlKxBtNRAAsx09il/PV7X3+UCvm51/wA==", 2023-10-21T17:40:37.983 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.3", 2023-10-21T17:40:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDA9TNl1RGrExAAY4lkS+lUligoWCQmdsLA0Q==", 2023-10-21T17:40:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.985 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.986 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.4", 2023-10-21T17:40:37.987 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDH9TNljyoXHxAA57Kmc4PUgJ7HWyQ+fxQtiQ==", 2023-10-21T17:40:37.987 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.987 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.987 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.987 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.5", 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDK9TNlPVsgNxAA5vsy9/tVRwcGKlWXKa0ibQ==", 2023-10-21T17:40:37.988 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.989 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.989 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.989 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.989 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.6", 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDO9TNl4Fq5ExAAoudoOHVjUdG7iJTx+OoqnQ==", 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.990 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.991 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "osd.7", 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQDR9TNlEi84LBAApBQEk1ZpegQyQUpKTij3pA==", 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow profile osd", 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile osd", 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:37.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.993 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.0", 2023-10-21T17:40:37.993 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA==", 2023-10-21T17:40:37.993 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow r", 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.995 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.1", 2023-10-21T17:40:37.995 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlbgozLxAAPa4VWYFYPSa48uPvies+mg==", 2023-10-21T17:40:37.995 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.995 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.995 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.996 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.2", 2023-10-21T17:40:37.997 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlk0mVIhAAI7Mk8ruaHaT/scG3GBYdZg==", 2023-10-21T17:40:37.997 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.997 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.997 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:40:37.997 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.3", 2023-10-21T17:40:37.998 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCz9TNlkT4wOhAAmxQ5SWTbzhObaxfCfosvPA==", 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow", 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow r", 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow rw", 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow rwx" 2023-10-21T17:40:37.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "client.admin", 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCw9TNlIDdEORAATI0H/3SBOro5xcUjtPLcdw==", 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:38.000 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:40:38.001 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": "allow *", 2023-10-21T17:40:38.001 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow *", 2023-10-21T17:40:38.001 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:38.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:38.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.x", 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCy9TNl2XXwABAAho1RYs3VNqeXJe9C8xu0Ew==", 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:40:38.002 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:40:38.003 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:38.003 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:38.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-10-21T17:40:38.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:38.003 INFO:teuthology.orchestra.run.smithi053.stdout: "entity": "mgr.y", 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "AQCx9TNl89WaLxAADfG48tXhxsx5wrFSfJwSCQ==", 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": { 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": "allow *", 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: "mon": "allow profile mgr", 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": "allow *" 2023-10-21T17:40:38.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:38.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:38.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-10-21T17:40:38.005 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:40:38.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 msgr2=0x7f460806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 0x7f460806d5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4614036000 tx=0x7f4614008030 comp rx=0 tx=0).stop 2023-10-21T17:40:38.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 msgr2=0x7f462414fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f462000c450 tx=0x7f462000c910 comp rx=0 tx=0).stop 2023-10-21T17:40:38.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 shutdown_connections 2023-10-21T17:40:38.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f460806b0e0 0x7f460806d5a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4624159db0 0x7f46240aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46241599d0 0x7f4624153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 --2- 172.21.15.53:0/227673773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46240cc210 0x7f462414fb80 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.973+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 >> 172.21.15.53:0/227673773 conn(0x7f46240cb0c0 msgr2=0x7f4624155890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.974+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 shutdown_connections 2023-10-21T17:40:38.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:37.974+0000 7f462aa21640 1 -- 172.21.15.53:0/227673773 wait complete. 2023-10-21T17:40:38.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:38.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.240+0000 7fc844b47640 1 Processor -- start 2023-10-21T17:40:38.241 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.240+0000 7fc844b47640 1 -- start start 2023-10-21T17:40:38.242 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc844b47640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc840070db0 0x7fc84015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc844b47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 0x7fc84006b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc844b47640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400c9850 con 0x7fc840071190 2023-10-21T17:40:38.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc844b47640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84006a870 con 0x7fc840070db0 2023-10-21T17:40:38.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc844b47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84015e280 con 0x7fc84006ad00 2023-10-21T17:40:38.243 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc83e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc840070db0 0x7fc84015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc83e59c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc840070db0 0x7fc84015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44342/0 (socket says 172.21.15.53:44342) 2023-10-21T17:40:38.244 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.241+0000 7fc83e59c640 1 -- 172.21.15.53:0/1765301324 learned_addr learned my addr 172.21.15.53:0/1765301324 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:38.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83ed9d640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 0x7fc84006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83e59c640 1 -- 172.21.15.53:0/1765301324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 msgr2=0x7fc84006b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:38.245 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83e59c640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 0x7fc84006b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83e59c640 1 -- 172.21.15.53:0/1765301324 >> v1:172.21.15.124:6789/0 conn(0x7fc840071190 legacy=0x7fc84015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83e59c640 1 -- 172.21.15.53:0/1765301324 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84015eb90 con 0x7fc840070db0 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.242+0000 7fc83ed9d640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 0x7fc84006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.243+0000 7fc83e59c640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc840070db0 0x7fc84015dd40 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42de98c1bc84779f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.244+0000 7fc83d59a640 1 -- 172.21.15.53:0/1765301324 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830030840 con 0x7fc840070db0 2023-10-21T17:40:38.246 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.244+0000 7fc83d59a640 1 -- 172.21.15.53:0/1765301324 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8300309a0 con 0x7fc840070db0 2023-10-21T17:40:38.247 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.244+0000 7fc83d59a640 1 -- 172.21.15.53:0/1765301324 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830030d30 con 0x7fc840070db0 2023-10-21T17:40:38.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.244+0000 7fc844b47640 1 -- 172.21.15.53:0/1765301324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc840070db0 msgr2=0x7fc84015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.244+0000 7fc844b47640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc840070db0 0x7fc84015dd40 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83002b980 comp rx=0 tx=0).stop 2023-10-21T17:40:38.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 -- 172.21.15.53:0/1765301324 shutdown_connections 2023-10-21T17:40:38.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc84006ad00 0x7fc84006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.248 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 --2- 172.21.15.53:0/1765301324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc840070db0 0x7fc84015dd40 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 -- 172.21.15.53:0/1765301324 >> 172.21.15.53:0/1765301324 conn(0x7fc8400b1480 msgr2=0x7fc84006cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:38.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 -- 172.21.15.53:0/1765301324 shutdown_connections 2023-10-21T17:40:38.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.245+0000 7fc844b47640 1 -- 172.21.15.53:0/1765301324 wait complete. 2023-10-21T17:40:38.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.246+0000 7fc844b47640 1 Processor -- start 2023-10-21T17:40:38.249 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.247+0000 7fc844b47640 1 -- start start 2023-10-21T17:40:38.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 0x7fc84014c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.251 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 0x7fc8401e5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84014b6c0 con 0x7fc84006ad00 2023-10-21T17:40:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84014b9a0 con 0x7fc840071190 2023-10-21T17:40:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc844b47640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84014bc80 con 0x7fc840070db0 2023-10-21T17:40:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.252 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44358/0 (socket says 172.21.15.53:44358) 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 -- 172.21.15.53:0/2440154581 learned_addr learned my addr 172.21.15.53:0/2440154581 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83dd9b640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 0x7fc84014c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83ed9d640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 0x7fc8401e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 -- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 msgr2=0x7fc84014c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 0x7fc84014c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.253 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 -- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 msgr2=0x7fc8401e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 0x7fc8401e5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83e59c640 1 -- 172.21.15.53:0/2440154581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300094d0 con 0x7fc84006ad00 2023-10-21T17:40:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.248+0000 7fc83ed9d640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 0x7fc8401e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc83dd9b640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 0x7fc84014c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:38.254 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc83e59c640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc830004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300307f0 con 0x7fc84006ad00 2023-10-21T17:40:38.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830043400 con 0x7fc84006ad00 2023-10-21T17:40:38.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300437b0 con 0x7fc84006ad00 2023-10-21T17:40:38.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84014bf60 con 0x7fc84006ad00 2023-10-21T17:40:38.255 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.249+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84014cca0 con 0x7fc84006ad00 2023-10-21T17:40:38.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.251+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8400cd5c0 con 0x7fc84006ad00 2023-10-21T17:40:38.256 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.252+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc830043910 con 0x7fc84006ad00 2023-10-21T17:40:38.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.253+0000 7fc82effd640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 0x7fc82006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.253+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fc83003e070 con 0x7fc84006ad00 2023-10-21T17:40:38.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.253+0000 7fc83dd9b640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 0x7fc82006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.257 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.255+0000 7fc83dd9b640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 0x7fc82006d570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8400c5bd0 tx=0x7fc834009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.258 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.257+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8300aee60 con 0x7fc84006ad00 2023-10-21T17:40:38.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.452+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc840067490 con 0x7fc84006ad00 2023-10-21T17:40:38.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.454+0000 7fc82effd640 1 -- 172.21.15.53:0/2440154581 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fc830034090 con 0x7fc84006ad00 2023-10-21T17:40:38.455 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:38.455 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":271,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:37.436273+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 msgr2=0x7fc82006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 0x7fc82006d570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8400c5bd0 tx=0x7fc834009040 comp rx=0 tx=0).stop 2023-10-21T17:40:38.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 msgr2=0x7fc8400c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc830004730 comp rx=0 tx=0).stop 2023-10-21T17:40:38.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 shutdown_connections 2023-10-21T17:40:38.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fc82006b0b0 0x7fc82006d570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc840071190 0x7fc8401e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc840070db0 0x7fc84014c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 --2- 172.21.15.53:0/2440154581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc84006ad00 0x7fc8400c2290 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 >> 172.21.15.53:0/2440154581 conn(0x7fc8400b1480 msgr2=0x7fc84015d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:38.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 shutdown_connections 2023-10-21T17:40:38.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.456+0000 7fc844b47640 1 -- 172.21.15.53:0/2440154581 wait complete. 2023-10-21T17:40:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:40:38.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:40:38.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.702+0000 7f4253fff640 1 Processor -- start 2023-10-21T17:40:38.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.702+0000 7f4253fff640 1 -- start start 2023-10-21T17:40:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.704+0000 7f4253fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42540cdc30 0x7f42540cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.704+0000 7f4253fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f4254159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.704+0000 7f4253fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425415ac90 con 0x7f42540caef0 2023-10-21T17:40:38.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.704+0000 7f4253fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425415a6d0 con 0x7f42540cdc30 2023-10-21T17:40:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.704+0000 7f4253fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425415a9b0 con 0x7f42540cce90 2023-10-21T17:40:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42540cdc30 0x7f42540cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f42540cdc30 0x7f42540cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44374/0 (socket says 172.21.15.53:44374) 2023-10-21T17:40:38.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 -- 172.21.15.53:0/2593476966 learned_addr learned my addr 172.21.15.53:0/2593476966 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f42537fe640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f4254159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 -- 172.21.15.53:0/2593476966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 msgr2=0x7f4254159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:38.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f4254159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 -- 172.21.15.53:0/2593476966 >> v1:172.21.15.124:6789/0 conn(0x7f42540caef0 legacy=0x7f42540cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:38.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.705+0000 7f424bfff640 1 -- 172.21.15.53:0/2593476966 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42540cc150 con 0x7f42540cdc30 2023-10-21T17:40:38.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f424bfff640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540cdc30 0x7f42540cc950 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f425406e510 tx=0x7f423c02e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca950c8854829799 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f42527fc640 1 -- 172.21.15.53:0/2593476966 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c02ee50 con 0x7f42540cdc30 2023-10-21T17:40:38.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f42527fc640 1 -- 172.21.15.53:0/2593476966 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c036440 con 0x7f42540cdc30 2023-10-21T17:40:38.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f42527fc640 1 -- 172.21.15.53:0/2593476966 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0367d0 con 0x7f42540cdc30 2023-10-21T17:40:38.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f4253fff640 1 -- 172.21.15.53:0/2593476966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540cdc30 msgr2=0x7f42540cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.707+0000 7f4253fff640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540cdc30 0x7f42540cc950 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f425406e510 tx=0x7f423c02e0a0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 -- 172.21.15.53:0/2593476966 shutdown_connections 2023-10-21T17:40:38.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f4254159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 --2- 172.21.15.53:0/2593476966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540cdc30 0x7f42540cc950 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 -- 172.21.15.53:0/2593476966 >> 172.21.15.53:0/2593476966 conn(0x7f425406c010 msgr2=0x7f425406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:38.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 -- 172.21.15.53:0/2593476966 shutdown_connections 2023-10-21T17:40:38.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.708+0000 7f4253fff640 1 -- 172.21.15.53:0/2593476966 wait complete. 2023-10-21T17:40:38.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.709+0000 7f4253fff640 1 Processor -- start 2023-10-21T17:40:38.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.710+0000 7f4253fff640 1 -- start start 2023-10-21T17:40:38.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f42540ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 0x7f42540ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42540af850 con 0x7f42540caef0 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425414b130 con 0x7f42540cdc30 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4253fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42540aed00 con 0x7f42540cce90 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44382/0 (socket says 172.21.15.53:44382) 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 -- 172.21.15.53:0/3733494085 learned_addr learned my addr 172.21.15.53:0/3733494085 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:38.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f424bfff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f42540ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 -- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 msgr2=0x7f42540ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f42540ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 -- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 msgr2=0x7f42540ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f42537fe640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 0x7f42540ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 0x7f42540ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f4252ffd640 1 -- 172.21.15.53:0/3733494085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c00b770 con 0x7f42540caef0 2023-10-21T17:40:38.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.711+0000 7f42537fe640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 0x7f42540ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:38.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f4252ffd640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f424400b3f0 tx=0x7f424400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244011830 con 0x7f42540caef0 2023-10-21T17:40:38.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244011990 con 0x7f42540caef0 2023-10-21T17:40:38.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244011d40 con 0x7f42540caef0 2023-10-21T17:40:38.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425414aae0 con 0x7f42540caef0 2023-10-21T17:40:38.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.712+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425414a590 con 0x7f42540caef0 2023-10-21T17:40:38.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.713+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425414a940 con 0x7f42540caef0 2023-10-21T17:40:38.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.714+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f424401a400 con 0x7f42540caef0 2023-10-21T17:40:38.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.715+0000 7f424b7fe640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 0x7f423406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:38.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.715+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f42440897f0 con 0x7f42540caef0 2023-10-21T17:40:38.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.716+0000 7f424bfff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 0x7f423406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:38.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.717+0000 7f424bfff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 0x7f423406d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f423c038000 tx=0x7f423c008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:38.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.720+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f425414a940 con 0x7f42540caef0 2023-10-21T17:40:38.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.911+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f425414a940 con 0x7f42540caef0 2023-10-21T17:40:38.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.912+0000 7f424b7fe640 1 -- 172.21.15.53:0/3733494085 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v271) v1 ==== 93+0+10 (secure 0 0 0) 0x7f425414a940 con 0x7f42540caef0 2023-10-21T17:40:38.913 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:40:38.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 msgr2=0x7f423406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 0x7f423406d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f423c038000 tx=0x7f423c008040 comp rx=0 tx=0).stop 2023-10-21T17:40:38.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 msgr2=0x7f425414a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:38.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f424400b3f0 tx=0x7f424400b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 shutdown_connections 2023-10-21T17:40:38.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f423406b0b0 0x7f423406d570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42540cdc30 0x7f42540ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42540cce90 0x7f42540ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 --2- 172.21.15.53:0/3733494085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f42540caef0 0x7f425414a050 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:38.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 >> 172.21.15.53:0/3733494085 conn(0x7f425406c010 msgr2=0x7f42541581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:38.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 shutdown_connections 2023-10-21T17:40:38.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:38.914+0000 7f4253fff640 1 -- 172.21.15.53:0/3733494085 wait complete. 2023-10-21T17:40:38.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-21T17:40:39.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.156+0000 7fe4168e0640 1 Processor -- start 2023-10-21T17:40:39.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.156+0000 7fe4168e0640 1 -- start start 2023-10-21T17:40:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.157+0000 7fe4168e0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe4100cd760 0x7fe4100cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe4168e0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41015af50 con 0x7fe41015ab70 2023-10-21T17:40:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe4168e0640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100ced20 con 0x7fe4100b3c30 2023-10-21T17:40:39.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe4168e0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ce110 con 0x7fe4100cd760 2023-10-21T17:40:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe40ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe41015ab70 0x7fe4100cffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37288/0 (socket says 172.21.15.53:37288) 2023-10-21T17:40:39.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe40ffff640 1 -- 172.21.15.53:0/3770702183 learned_addr learned my addr 172.21.15.53:0/3770702183 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:39.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe40f7fe640 1 --2- 172.21.15.53:0/3770702183 >> v2:172.21.15.124:3300/0 conn(0x7fe4100cd760 0x7fe4100cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866331821 0 0) 0x7fe41015af50 con 0x7fe41015ab70 2023-10-21T17:40:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.158+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8003520 con 0x7fe41015ab70 2023-10-21T17:40:39.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146951239 0 0) 0x7fe3f8003520 con 0x7fe41015ab70 2023-10-21T17:40:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 >> v2:172.21.15.124:3300/0 conn(0x7fe4100cd760 msgr2=0x7fe4100cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 --2- 172.21.15.53:0/3770702183 >> v2:172.21.15.124:3300/0 conn(0x7fe4100cd760 0x7fe4100cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 >> v1:172.21.15.124:6789/0 conn(0x7fe4100b3c30 legacy=0x7fe4100b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100cf000 con 0x7fe41015ab70 2023-10-21T17:40:39.166 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe400001690 con 0x7fe41015ab70 2023-10-21T17:40:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 >> v1:172.21.15.53:6789/0 conn(0x7fe41015ab70 legacy=0x7fe4100cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f80039c0 0x7fe3f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f80047c0 0x7fe3f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.167 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 0x7fe3f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100ce110 con 0x7fe3f8005180 2023-10-21T17:40:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800c080 con 0x7fe3f80047c0 2023-10-21T17:40:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f800c240 con 0x7fe3f80039c0 2023-10-21T17:40:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.159+0000 7fe414e7d640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 0x7fe3f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe40f7fe640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f80047c0 0x7fe3f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe40ffff640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f80039c0 0x7fe3f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe414e7d640 1 -- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f80039c0 msgr2=0x7fe3f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe414e7d640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3f80039c0 0x7fe3f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe414e7d640 1 -- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f80047c0 msgr2=0x7fe3f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe414e7d640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f80047c0 0x7fe3f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.160+0000 7fe414e7d640 1 -- 172.21.15.53:0/3770702183 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8003520 con 0x7fe3f8005180 2023-10-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.161+0000 7fe414e7d640 1 -- 172.21.15.53:0/3770702183 reap_dead start 2023-10-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.161+0000 7fe414e7d640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 0x7fe3f800b980 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fe408001e50 tx=0x7fe3f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b253781142946dab server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.161+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408003b70 con 0x7fe3f8005180 2023-10-21T17:40:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.161+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408003d10 con 0x7fe3f8005180 2023-10-21T17:40:39.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.161+0000 7fe40effd640 1 -- 172.21.15.53:0/3770702183 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408005310 con 0x7fe3f8005180 2023-10-21T17:40:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 -- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 msgr2=0x7fe3f800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 0x7fe3f800b980 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fe408001e50 tx=0x7fe3f80057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 -- 172.21.15.53:0/3770702183 shutdown_connections 2023-10-21T17:40:39.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 --2- 172.21.15.53:0/3770702183 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe3f8005180 0x7fe3f800b980 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 -- 172.21.15.53:0/3770702183 >> 172.21.15.53:0/3770702183 conn(0x7fe410069f30 msgr2=0x7fe41006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 -- 172.21.15.53:0/3770702183 shutdown_connections 2023-10-21T17:40:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.162+0000 7fe4168e0640 1 -- 172.21.15.53:0/3770702183 wait complete. 2023-10-21T17:40:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.164+0000 7fe4168e0640 1 Processor -- start 2023-10-21T17:40:39.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.164+0000 7fe4168e0640 1 -- start start 2023-10-21T17:40:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f8005180 0x7fe410156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe410157ee0 0x7fe410158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101588a0 con 0x7fe4100c5fb0 2023-10-21T17:40:39.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410157070 con 0x7fe3f8005180 2023-10-21T17:40:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.165+0000 7fe4168e0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100c48e0 con 0x7fe410157ee0 2023-10-21T17:40:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44404/0 (socket says 172.21.15.53:44404) 2023-10-21T17:40:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 -- 172.21.15.53:0/1332329300 learned_addr learned my addr 172.21.15.53:0/1332329300 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:39.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 -- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe410157ee0 msgr2=0x7fe410158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe410157ee0 0x7fe410158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 -- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f8005180 msgr2=0x7fe410156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f8005180 0x7fe410156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.166+0000 7fe40f7fe640 1 -- 172.21.15.53:0/1332329300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100c4bc0 con 0x7fe4100c5fb0 2023-10-21T17:40:39.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe40f7fe640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe404011570 tx=0x7fe404011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404015930 con 0x7fe4100c5fb0 2023-10-21T17:40:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe404015a90 con 0x7fe4100c5fb0 2023-10-21T17:40:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100c4ea0 con 0x7fe4100c5fb0 2023-10-21T17:40:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404015e60 con 0x7fe4100c5fb0 2023-10-21T17:40:39.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.167+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100c5460 con 0x7fe4100c5fb0 2023-10-21T17:40:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.168+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410066d80 con 0x7fe4100c5fb0 2023-10-21T17:40:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.169+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe40401d400 con 0x7fe4100c5fb0 2023-10-21T17:40:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.170+0000 7fe40cff9640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 0x7fe3fc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.170+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fe40408a080 con 0x7fe4100c5fb0 2023-10-21T17:40:39.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.171+0000 7fe40ffff640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 0x7fe3fc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.172+0000 7fe40ffff640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 0x7fe3fc06d570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4100b6b60 tx=0x7fe400000f50 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.175+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4040bcca0 con 0x7fe4100c5fb0 2023-10-21T17:40:39.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.373+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe4100cc820 con 0x7fe4100c5fb0 2023-10-21T17:40:39.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.374+0000 7fe40cff9640 1 -- 172.21.15.53:0/1332329300 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v271) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe404054ed0 con 0x7fe4100c5fb0 2023-10-21T17:40:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:39.375 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:40:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "poolnum": 1, 2023-10-21T17:40:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "poolname": ".mgr" 2023-10-21T17:40:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:39.376 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:40:39.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.376+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 msgr2=0x7fe3fc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.376+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 0x7fe3fc06d570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe4100b6b60 tx=0x7fe400000f50 comp rx=0 tx=0).stop 2023-10-21T17:40:39.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 msgr2=0x7fe4100c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe404011570 tx=0x7fe404011a30 comp rx=0 tx=0).stop 2023-10-21T17:40:39.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 shutdown_connections 2023-10-21T17:40:39.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fe3fc06b0b0 0x7fe3fc06d570 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe410157ee0 0x7fe410158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe4100c5fb0 0x7fe4100c6430 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 --2- 172.21.15.53:0/1332329300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe3f8005180 0x7fe410156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 >> 172.21.15.53:0/1332329300 conn(0x7fe410069f30 msgr2=0x7fe4100c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:39.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 shutdown_connections 2023-10-21T17:40:39.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.377+0000 7fe4168e0640 1 -- 172.21.15.53:0/1332329300 wait complete. 2023-10-21T17:40:39.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-21T17:40:39.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-21T17:40:39.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.633+0000 7f096b59e640 1 Processor -- start 2023-10-21T17:40:39.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.633+0000 7f096b59e640 1 -- start start 2023-10-21T17:40:39.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f096c165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c15ae90 con 0x7f096c15aab0 2023-10-21T17:40:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096b59e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c0cbdb0 con 0x7f096c070b50 2023-10-21T17:40:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0cb600 con 0x7f096c0b0670 2023-10-21T17:40:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f096c15aab0 0x7f096c0caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37298/0 (socket says 172.21.15.53:37298) 2023-10-21T17:40:39.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.635+0000 7f096a59c640 1 -- 172.21.15.53:0/991887474 learned_addr learned my addr 172.21.15.53:0/991887474 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:39.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096ad9d640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f096c165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514028409 0 0) 0x7f095c001110 con 0x7f096c15aab0 2023-10-21T17:40:39.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0958003520 con 0x7f096c15aab0 2023-10-21T17:40:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900309679 0 0) 0x7f0958003520 con 0x7f096c15aab0 2023-10-21T17:40:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 msgr2=0x7f096c165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f096c165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 >> v1:172.21.15.124:6789/0 conn(0x7f096c070b50 legacy=0x7f096c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0cb7f0 con 0x7f096c15aab0 2023-10-21T17:40:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f095c001800 con 0x7f096c15aab0 2023-10-21T17:40:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 >> v1:172.21.15.53:6789/0 conn(0x7f096c15aab0 legacy=0x7f096c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f09580039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0958003f40 0x7f09580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0958004d00 0x7f0958005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095800bb10 con 0x7f0958004d00 2023-10-21T17:40:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095800bc80 con 0x7f0958003f40 2023-10-21T17:40:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.636+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095800be40 con 0x7f096c0b0670 2023-10-21T17:40:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0958004d00 0x7f0958005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.642 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f0969d9b640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0958003f40 0x7f09580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096a59c640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f09580039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 -- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 msgr2=0x7f09580039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0b0670 0x7f09580039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 -- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0958003f40 msgr2=0x7f09580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:39.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0958003f40 0x7f09580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.637+0000 7f096ad9d640 1 -- 172.21.15.53:0/991887474 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958003520 con 0x7f0958004d00 2023-10-21T17:40:39.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.638+0000 7f096ad9d640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0958004d00 0x7f0958005180 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f0964009fc0 tx=0x7f09580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feabbd3f7f800a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096402cc20 con 0x7f0958004d00 2023-10-21T17:40:39.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096400f040 con 0x7f0958004d00 2023-10-21T17:40:39.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096959a640 1 -- 172.21.15.53:0/991887474 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0964034440 con 0x7f0958004d00 2023-10-21T17:40:39.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096b59e640 1 -- 172.21.15.53:0/991887474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0958004d00 msgr2=0x7f0958005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096b59e640 1 --2- 172.21.15.53:0/991887474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0958004d00 0x7f0958005180 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f0964009fc0 tx=0x7f09580057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096ad9d640 1 -- 172.21.15.53:0/991887474 reap_dead start 2023-10-21T17:40:39.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096b59e640 1 -- 172.21.15.53:0/991887474 shutdown_connections 2023-10-21T17:40:39.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096b59e640 1 -- 172.21.15.53:0/991887474 >> 172.21.15.53:0/991887474 conn(0x7f096c06c010 msgr2=0x7f096c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:39.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.639+0000 7f096b59e640 1 -- 172.21.15.53:0/991887474 shutdown_connections 2023-10-21T17:40:39.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.640+0000 7f096b59e640 1 -- 172.21.15.53:0/991887474 wait complete. 2023-10-21T17:40:39.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.641+0000 7f096b59e640 1 Processor -- start 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.641+0000 7f096b59e640 1 -- start start 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f096c1f3110 0x7f096c0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c6710 0x7f096c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15c630 con 0x7f096c1f2d30 2023-10-21T17:40:39.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c0c6d40 con 0x7f096c1f3110 2023-10-21T17:40:39.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.642+0000 7f096b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15c880 con 0x7f096c0c6710 2023-10-21T17:40:39.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44418/0 (socket says 172.21.15.53:44418) 2023-10-21T17:40:39.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 -- 172.21.15.53:0/1802502671 learned_addr learned my addr 172.21.15.53:0/1802502671 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:39.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 -- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c6710 msgr2=0x7f096c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c6710 0x7f096c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 -- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f096c1f3110 msgr2=0x7f096c0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f0969d9b640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f096c1f3110 0x7f096c0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f096c1f3110 0x7f096c0c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:39.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.643+0000 7f096a59c640 1 -- 172.21.15.53:0/1802502671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c15cb60 con 0x7f096c1f2d30 2023-10-21T17:40:39.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f096a59c640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e840 tx=0x7f095c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00c8d0 con 0x7f096c1f2d30 2023-10-21T17:40:39.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c00ca30 con 0x7f096c1f2d30 2023-10-21T17:40:39.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00cdc0 con 0x7f096c1f2d30 2023-10-21T17:40:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c15ce40 con 0x7f096c1f2d30 2023-10-21T17:40:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.644+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c0af550 con 0x7f096c1f2d30 2023-10-21T17:40:39.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.645+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c15d080 con 0x7f096c1f2d30 2023-10-21T17:40:39.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.646+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f095c01d460 con 0x7f096c1f2d30 2023-10-21T17:40:39.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.647+0000 7f0962ffd640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 0x7f094c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:39.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.647+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f095c0897e0 con 0x7f096c1f2d30 2023-10-21T17:40:39.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.648+0000 7f0969d9b640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 0x7f094c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:39.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.649+0000 7f0969d9b640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 0x7f094c06d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0954007fd0 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:39.655 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.652+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f096c15d080 con 0x7f096c1f2d30 2023-10-21T17:40:39.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:39.845+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f096c15d080 con 0x7f096c1f2d30 2023-10-21T17:40:40.470 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.468+0000 7f0962ffd640 1 -- 172.21.15.53:0/1802502671 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v272) v1 ==== 138+0+0 (secure 0 0 0) 0x7f096c15d080 con 0x7f096c1f2d30 2023-10-21T17:40:40.470 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' created 2023-10-21T17:40:40.474 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.472+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 msgr2=0x7f094c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.472+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 0x7f094c06d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0954007fd0 tx=0x7f0954009040 comp rx=0 tx=0).stop 2023-10-21T17:40:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.472+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 msgr2=0x7f096c1f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.472+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e840 tx=0x7f095c00ed00 comp rx=0 tx=0).stop 2023-10-21T17:40:40.475 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 shutdown_connections 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f094c06b0b0 0x7f094c06d570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f096c0c6710 0x7f096c0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f096c1f3110 0x7f096c0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 --2- 172.21.15.53:0/1802502671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f096c1f2d30 0x7f096c1f3590 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 >> 172.21.15.53:0/1802502671 conn(0x7f096c06c010 msgr2=0x7f096c0adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:40.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 shutdown_connections 2023-10-21T17:40:40.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.473+0000 7f096b59e640 1 -- 172.21.15.53:0/1802502671 wait complete. 2023-10-21T17:40:40.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-21T17:40:40.724 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.723+0000 7f41e1883640 1 Processor -- start 2023-10-21T17:40:40.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.723+0000 7f41e1883640 1 -- start start 2023-10-21T17:40:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.725+0000 7f41e1883640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41dc0b3a00 0x7f41dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.725+0000 7f41e1883640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc0cb500 0x7f41dc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.725+0000 7f41e1883640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0cb0d0 con 0x7f41dc15d580 2023-10-21T17:40:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.725+0000 7f41e1883640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc0cbef0 con 0x7f41dc0b3a00 2023-10-21T17:40:40.727 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.725+0000 7f41e1883640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0b32d0 con 0x7f41dc0cb500 2023-10-21T17:40:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41daffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f41dc0b3a00 0x7f41dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41d3fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41dc15d580 0x7f41dc0b2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37308/0 (socket says 172.21.15.53:37308) 2023-10-21T17:40:40.728 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41d3fff640 1 -- 172.21.15.53:0/3408520500 learned_addr learned my addr 172.21.15.53:0/3408520500 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:40.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41db7fe640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc0cb500 0x7f41dc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41daffd640 1 -- 172.21.15.53:0/3408520500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc0cb500 msgr2=0x7f41dc0cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:40.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41daffd640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc0cb500 0x7f41dc0cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41daffd640 1 -- 172.21.15.53:0/3408520500 >> v1:172.21.15.53:6789/0 conn(0x7f41dc15d580 legacy=0x7f41dc0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.730 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41daffd640 1 -- 172.21.15.53:0/3408520500 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc06d8a0 con 0x7f41dc0b3a00 2023-10-21T17:40:40.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.726+0000 7f41da7fc640 1 -- 172.21.15.53:0/3408520500 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924810186 0 0) 0x7f41dc0cb0d0 con 0x7f41dc15d580 2023-10-21T17:40:40.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.727+0000 7f41daffd640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0b3a00 0x7f41dc06d360 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2584956e30cb2594 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:40.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41da7fc640 1 -- 172.21.15.53:0/3408520500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc02d040 con 0x7f41dc0b3a00 2023-10-21T17:40:40.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41da7fc640 1 -- 172.21.15.53:0/3408520500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc031810 con 0x7f41dc0b3a00 2023-10-21T17:40:40.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41da7fc640 1 -- 172.21.15.53:0/3408520500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc031ba0 con 0x7f41dc0b3a00 2023-10-21T17:40:40.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 -- 172.21.15.53:0/3408520500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0b3a00 msgr2=0x7f41dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0b3a00 0x7f41dc06d360 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0097a0 tx=0x7f41cc02b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 -- 172.21.15.53:0/3408520500 shutdown_connections 2023-10-21T17:40:40.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc0cb500 0x7f41dc0cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.732 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 --2- 172.21.15.53:0/3408520500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0b3a00 0x7f41dc06d360 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.728+0000 7f41e1883640 1 -- 172.21.15.53:0/3408520500 >> 172.21.15.53:0/3408520500 conn(0x7f41dc06a140 msgr2=0x7f41dc06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:40.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.729+0000 7f41e1883640 1 -- 172.21.15.53:0/3408520500 shutdown_connections 2023-10-21T17:40:40.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.729+0000 7f41e1883640 1 -- 172.21.15.53:0/3408520500 wait complete. 2023-10-21T17:40:40.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.730+0000 7f41e1883640 1 Processor -- start 2023-10-21T17:40:40.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 -- start start 2023-10-21T17:40:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 0x7f41dc1f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc15d580 0x7f41dc0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c7370 con 0x7f41dc0cb500 2023-10-21T17:40:40.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c7650 con 0x7f41dc0b3a00 2023-10-21T17:40:40.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.731+0000 7f41e1883640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c7930 con 0x7f41dc15d580 2023-10-21T17:40:40.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44444/0 (socket says 172.21.15.53:44444) 2023-10-21T17:40:40.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 -- 172.21.15.53:0/1167397647 learned_addr learned my addr 172.21.15.53:0/1167397647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:40.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41db7fe640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc15d580 0x7f41dc0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41daffd640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 0x7f41dc1f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 -- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc15d580 msgr2=0x7f41dc0c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc15d580 0x7f41dc0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 -- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 msgr2=0x7f41dc1f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:40.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 0x7f41dc1f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:40.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41d3fff640 1 -- 172.21.15.53:0/1167397647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009400 con 0x7f41dc0cb500 2023-10-21T17:40:40.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.732+0000 7f41daffd640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 0x7f41dc1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:40.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41d3fff640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c410 tx=0x7f41c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:40.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c40074e0 con 0x7f41dc0cb500 2023-10-21T17:40:40.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c4007660 con 0x7f41dc0cb500 2023-10-21T17:40:40.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004a20 con 0x7f41dc0cb500 2023-10-21T17:40:40.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0c7c10 con 0x7f41dc0cb500 2023-10-21T17:40:40.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.733+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc1f2800 con 0x7f41dc0cb500 2023-10-21T17:40:40.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.734+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc0c8130 con 0x7f41dc0cb500 2023-10-21T17:40:40.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.737+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f41c4004b80 con 0x7f41dc0cb500 2023-10-21T17:40:40.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.738+0000 7f41d8ff9640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 0x7f41bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:40.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.738+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f41c4089e50 con 0x7f41dc0cb500 2023-10-21T17:40:40.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.739+0000 7f41daffd640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 0x7f41bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:40.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.740+0000 7f41daffd640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 0x7f41bc06d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc03c040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:40.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.742+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41c40586d0 con 0x7f41dc0cb500 2023-10-21T17:40:40.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:40.936+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f41dc071cb0 con 0x7f41dc0cb500 2023-10-21T17:40:41.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.475+0000 7f41d8ff9640 1 -- 172.21.15.53:0/1167397647 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v273) v1 ==== 170+0+0 (secure 0 0 0) 0x7f41c4058260 con 0x7f41dc0cb500 2023-10-21T17:40:41.477 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' created 2023-10-21T17:40:41.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 msgr2=0x7f41bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 0x7f41bc06d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41cc036000 tx=0x7f41cc03c040 comp rx=0 tx=0).stop 2023-10-21T17:40:41.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 msgr2=0x7f41dc0c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f41c400c410 tx=0x7f41c400c8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 shutdown_connections 2023-10-21T17:40:41.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f41bc06b0b0 0x7f41bc06d570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f41dc15d580 0x7f41dc0c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41dc0cb500 0x7f41dc0c6990 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 --2- 172.21.15.53:0/1167397647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41dc0b3a00 0x7f41dc1f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 >> 172.21.15.53:0/1167397647 conn(0x7f41dc06a140 msgr2=0x7f41dc160d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:41.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 shutdown_connections 2023-10-21T17:40:41.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.479+0000 7f41e1883640 1 -- 172.21.15.53:0/1167397647 wait complete. 2023-10-21T17:40:41.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-21T17:40:41.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.706+0000 7f135434e640 1 Processor -- start 2023-10-21T17:40:41.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.706+0000 7f135434e640 1 -- start start 2023-10-21T17:40:41.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.707+0000 7f135434e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f134c160660 0x7f134c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.707+0000 7f135434e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f134c0c9910 0x7f134c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.707+0000 7f135434e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134c06e150 con 0x7f134c0cb450 2023-10-21T17:40:41.709 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.707+0000 7f135434e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134c06de70 con 0x7f134c160660 2023-10-21T17:40:41.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.707+0000 7f135434e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134c0c8e30 con 0x7f134c0c9910 2023-10-21T17:40:41.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f134c160660 0x7f134c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:41.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f134c160660 0x7f134c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44454/0 (socket says 172.21.15.53:44454) 2023-10-21T17:40:41.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 -- 172.21.15.53:0/702781228 learned_addr learned my addr 172.21.15.53:0/702781228 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:41.710 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13528eb640 1 --1- 172.21.15.53:0/702781228 >> v1:172.21.15.53:6789/0 conn(0x7f134c0cb450 0x7f134c0b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:37316/0 (socket says 172.21.15.53:37316) 2023-10-21T17:40:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 -- 172.21.15.53:0/702781228 >> v2:172.21.15.124:3300/0 conn(0x7f134c0c9910 msgr2=0x7f134c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 --2- 172.21.15.53:0/702781228 >> v2:172.21.15.124:3300/0 conn(0x7f134c0c9910 0x7f134c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 -- 172.21.15.53:0/702781228 >> v1:172.21.15.53:6789/0 conn(0x7f134c0cb450 legacy=0x7f134c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.712 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.708+0000 7f13520ea640 1 -- 172.21.15.53:0/702781228 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0c9110 con 0x7f134c160660 2023-10-21T17:40:41.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.709+0000 7f13520ea640 1 --2- 172.21.15.53:0/702781228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c160660 0x7f134c070cc0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f134c0b0d50 tx=0x7f133c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65f46b2d8bb43f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:41.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f13510e8640 1 -- 172.21.15.53:0/702781228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c02cb10 con 0x7f134c160660 2023-10-21T17:40:41.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f13510e8640 1 -- 172.21.15.53:0/702781228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c02cc70 con 0x7f134c160660 2023-10-21T17:40:41.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f13510e8640 1 -- 172.21.15.53:0/702781228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c0343f0 con 0x7f134c160660 2023-10-21T17:40:41.713 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 -- 172.21.15.53:0/702781228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c160660 msgr2=0x7f134c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 --2- 172.21.15.53:0/702781228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c160660 0x7f134c070cc0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f134c0b0d50 tx=0x7f133c02bf70 comp rx=0 tx=0).stop 2023-10-21T17:40:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 -- 172.21.15.53:0/702781228 shutdown_connections 2023-10-21T17:40:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 --2- 172.21.15.53:0/702781228 >> v2:172.21.15.124:3300/0 conn(0x7f134c0c9910 0x7f134c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 --2- 172.21.15.53:0/702781228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c160660 0x7f134c070cc0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 -- 172.21.15.53:0/702781228 >> 172.21.15.53:0/702781228 conn(0x7f134c06a5d0 msgr2=0x7f134c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:41.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.710+0000 7f135434e640 1 -- 172.21.15.53:0/702781228 shutdown_connections 2023-10-21T17:40:41.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.711+0000 7f135434e640 1 -- 172.21.15.53:0/702781228 wait complete. 2023-10-21T17:40:41.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.712+0000 7f135434e640 1 Processor -- start 2023-10-21T17:40:41.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.713+0000 7f135434e640 1 -- start start 2023-10-21T17:40:41.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.713+0000 7f135434e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f135434e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134c0cb450 0x7f134c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f135434e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f134c160660 0x7f134c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f135434e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c1f2e20 con 0x7f134c0c9910 2023-10-21T17:40:41.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f135434e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134c154920 con 0x7f134c160660 2023-10-21T17:40:41.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f135434e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134c154bc0 con 0x7f134c0cb450 2023-10-21T17:40:41.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:41.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44460/0 (socket says 172.21.15.53:44460) 2023-10-21T17:40:41.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 -- 172.21.15.53:0/1512581102 learned_addr learned my addr 172.21.15.53:0/1512581102 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:41.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13518e9640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134c0cb450 0x7f134c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:41.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13528eb640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f134c160660 0x7f134c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:41.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 -- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134c0cb450 msgr2=0x7f134c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134c0cb450 0x7f134c153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 -- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f134c160660 msgr2=0x7f134c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f134c160660 0x7f134c0c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.714+0000 7f13520ea640 1 -- 172.21.15.53:0/1512581102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c009b30 con 0x7f134c0c9910 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f13520ea640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f133c000f80 tx=0x7f133c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:41.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c03e450 con 0x7f134c0c9910 2023-10-21T17:40:41.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c03e5f0 con 0x7f134c0c9910 2023-10-21T17:40:41.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c03e9a0 con 0x7f134c0c9910 2023-10-21T17:40:41.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134c154ea0 con 0x7f134c0c9910 2023-10-21T17:40:41.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.715+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134c1f27a0 con 0x7f134c0c9910 2023-10-21T17:40:41.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.716+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134c04e160 con 0x7f134c0c9910 2023-10-21T17:40:41.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.717+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f133c03eb00 con 0x7f134c0c9910 2023-10-21T17:40:41.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.719+0000 7f134affd640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 0x7f133806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:41.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.719+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f133c0ac850 con 0x7f134c0c9910 2023-10-21T17:40:41.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.720+0000 7f13518e9640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 0x7f133806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:41.723 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.721+0000 7f13518e9640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 0x7f133806d5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f134c15f020 tx=0x7f1340008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:41.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.724+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f133c0774d0 con 0x7f134c0c9910 2023-10-21T17:40:41.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:41.956+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f134c0c3510 con 0x7f134c0c9910 2023-10-21T17:40:42.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.506+0000 7f134affd640 1 -- 172.21.15.53:0/1512581102 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 35 and data pool 36 v140) v1 ==== 174+0+0 (secure 0 0 0) 0x7f133c03edb0 con 0x7f134c0c9910 2023-10-21T17:40:42.508 INFO:teuthology.orchestra.run.smithi053.stderr:new fs with metadata pool 35 and data pool 36 2023-10-21T17:40:42.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 msgr2=0x7f133806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 0x7f133806d5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f134c15f020 tx=0x7f1340008040 comp rx=0 tx=0).stop 2023-10-21T17:40:42.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 msgr2=0x7f134c1f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f133c000f80 tx=0x7f133c004a40 comp rx=0 tx=0).stop 2023-10-21T17:40:42.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 shutdown_connections 2023-10-21T17:40:42.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f133806b0e0 0x7f133806d5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f134c160660 0x7f134c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134c0cb450 0x7f134c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.513 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 --2- 172.21.15.53:0/1512581102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f134c0c9910 0x7f134c1f32b0 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.510+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 >> 172.21.15.53:0/1512581102 conn(0x7f134c06a5d0 msgr2=0x7f134c0cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:42.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.511+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 shutdown_connections 2023-10-21T17:40:42.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.511+0000 7f135434e640 1 -- 172.21.15.53:0/1512581102 wait complete. 2023-10-21T17:40:42.551 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:42.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.730+0000 7f72e1abb640 1 Processor -- start 2023-10-21T17:40:42.731 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.730+0000 7f72e1abb640 1 -- start start 2023-10-21T17:40:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.731+0000 7f72e1abb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f72dc159ac0 0x7f72dc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.731+0000 7f72e1abb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc06dee0 con 0x7f72dc0ad790 2023-10-21T17:40:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.731+0000 7f72e1abb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc15d850 con 0x7f72dc0709e0 2023-10-21T17:40:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.731+0000 7f72e1abb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc15db30 con 0x7f72dc159ac0 2023-10-21T17:40:42.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f72dc159ac0 0x7f72dc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f72dc159ac0 0x7f72dc06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44468/0 (socket says 172.21.15.53:44468) 2023-10-21T17:40:42.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 -- 172.21.15.53:0/3175624010 learned_addr learned my addr 172.21.15.53:0/3175624010 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:42.735 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 -- 172.21.15.53:0/3175624010 >> v1:172.21.15.124:6789/0 conn(0x7f72dc0709e0 legacy=0x7f72dc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 -- 172.21.15.53:0/3175624010 >> v1:172.21.15.53:6789/0 conn(0x7f72dc0ad790 legacy=0x7f72dc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.732+0000 7f72db7fe640 1 -- 172.21.15.53:0/3175624010 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc15de10 con 0x7f72dc159ac0 2023-10-21T17:40:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.733+0000 7f72db7fe640 1 --2- 172.21.15.53:0/3175624010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc06d5e0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7879c9292559701 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.733+0000 7f72da7fc640 1 -- 172.21.15.53:0/3175624010 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc030860 con 0x7f72dc159ac0 2023-10-21T17:40:42.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.733+0000 7f72da7fc640 1 -- 172.21.15.53:0/3175624010 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc0309c0 con 0x7f72dc159ac0 2023-10-21T17:40:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72da7fc640 1 -- 172.21.15.53:0/3175624010 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc030d50 con 0x7f72dc159ac0 2023-10-21T17:40:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 -- 172.21.15.53:0/3175624010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 msgr2=0x7f72dc06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 --2- 172.21.15.53:0/3175624010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc06d5e0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc02b9a0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 -- 172.21.15.53:0/3175624010 shutdown_connections 2023-10-21T17:40:42.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 --2- 172.21.15.53:0/3175624010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc06d5e0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 -- 172.21.15.53:0/3175624010 >> 172.21.15.53:0/3175624010 conn(0x7f72dc06a380 msgr2=0x7f72dc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 -- 172.21.15.53:0/3175624010 shutdown_connections 2023-10-21T17:40:42.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.734+0000 7f72e1abb640 1 -- 172.21.15.53:0/3175624010 wait complete. 2023-10-21T17:40:42.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.736+0000 7f72e1abb640 1 Processor -- start 2023-10-21T17:40:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.736+0000 7f72e1abb640 1 -- start start 2023-10-21T17:40:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72dc0709e0 0x7f72dc1e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 0x7f72dc0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:42.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc158130 con 0x7f72dc159ac0 2023-10-21T17:40:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc158410 con 0x7f72dc0ad790 2023-10-21T17:40:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72e1abb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1586f0 con 0x7f72dc0709e0 2023-10-21T17:40:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44478/0 (socket says 172.21.15.53:44478) 2023-10-21T17:40:42.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.737+0000 7f72dbfff640 1 -- 172.21.15.53:0/4250981365 learned_addr learned my addr 172.21.15.53:0/4250981365 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72db7fe640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72dc0709e0 0x7f72dc1e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72daffd640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 0x7f72dc0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 -- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72dc0709e0 msgr2=0x7f72dc1e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72dc0709e0 0x7f72dc1e39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.743 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 -- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 msgr2=0x7f72dc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 0x7f72dc0af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 -- 172.21.15.53:0/4250981365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0094d0 con 0x7f72dc159ac0 2023-10-21T17:40:42.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72daffd640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 0x7f72dc0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:42.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72dbfff640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b9e0 tx=0x7f72d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:42.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400cb20 con 0x7f72dc159ac0 2023-10-21T17:40:42.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400cc80 con 0x7f72dc159ac0 2023-10-21T17:40:42.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc1589d0 con 0x7f72dc159ac0 2023-10-21T17:40:42.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4014680 con 0x7f72dc159ac0 2023-10-21T17:40:42.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.738+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc1591d0 con 0x7f72dc159ac0 2023-10-21T17:40:42.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.739+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc158ef0 con 0x7f72dc159ac0 2023-10-21T17:40:42.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.741+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f72d401d3b0 con 0x7f72dc159ac0 2023-10-21T17:40:42.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.741+0000 7f72c3fff640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 0x7f72c806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:42.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.742+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f72d4089d70 con 0x7f72dc159ac0 2023-10-21T17:40:42.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.742+0000 7f72db7fe640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 0x7f72c806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:42.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.744+0000 7f72db7fe640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 0x7f72c806d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72cc036000 tx=0x7f72cc03b040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:42.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.744+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72d4054970 con 0x7f72dc159ac0 2023-10-21T17:40:42.936 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.934+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72dc158ef0 con 0x7f72dc159ac0 2023-10-21T17:40:42.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.937+0000 7f72c3fff640 1 -- 172.21.15.53:0/4250981365 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v274) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f72dc158ef0 con 0x7f72dc159ac0 2023-10-21T17:40:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:42.940 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":274,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:41.957577+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:42.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 msgr2=0x7f72c806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 0x7f72c806d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72cc036000 tx=0x7f72cc03b040 comp rx=0 tx=0).stop 2023-10-21T17:40:42.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 msgr2=0x7f72dc13fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:42.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b9e0 tx=0x7f72d400bea0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 shutdown_connections 2023-10-21T17:40:42.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f72c806b0b0 0x7f72c806d570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72dc159ac0 0x7f72dc13fe50 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f72dc0ad790 0x7f72dc0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 --2- 172.21.15.53:0/4250981365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72dc0709e0 0x7f72dc1e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:42.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.940+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 >> 172.21.15.53:0/4250981365 conn(0x7f72dc06a380 msgr2=0x7f72dc156970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:42.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.941+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 shutdown_connections 2023-10-21T17:40:42.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:42.941+0000 7f72e1abb640 1 -- 172.21.15.53:0/4250981365 wait complete. 2023-10-21T17:40:42.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:43.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.186+0000 7ff747fff640 1 Processor -- start 2023-10-21T17:40:43.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.186+0000 7ff747fff640 1 -- start start 2023-10-21T17:40:43.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff747fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff748161320 0x7ff7480cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff747fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 0x7ff7480b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff747fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff748165470 con 0x7ff74806c710 2023-10-21T17:40:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff747fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff748165750 con 0x7ff748161320 2023-10-21T17:40:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff747fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff748165a30 con 0x7ff748164f70 2023-10-21T17:40:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff746ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff748161320 0x7ff7480cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff746ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff748161320 0x7ff7480cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44488/0 (socket says 172.21.15.53:44488) 2023-10-21T17:40:43.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff746ffd640 1 -- 172.21.15.53:0/3158849252 learned_addr learned my addr 172.21.15.53:0/3158849252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:43.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.188+0000 7ff73ffff640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 0x7ff7480b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.189+0000 7ff746ffd640 1 -- 172.21.15.53:0/3158849252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 msgr2=0x7ff7480b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:43.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.189+0000 7ff746ffd640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 0x7ff7480b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.189+0000 7ff746ffd640 1 -- 172.21.15.53:0/3158849252 >> v1:172.21.15.53:6789/0 conn(0x7ff74806c710 legacy=0x7ff7480c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.189+0000 7ff746ffd640 1 -- 172.21.15.53:0/3158849252 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748165d10 con 0x7ff748161320 2023-10-21T17:40:43.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.189+0000 7ff73ffff640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 0x7ff7480b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:43.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.190+0000 7ff746ffd640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748161320 0x7ff7480cc210 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c18ff21c9ca3f6e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.190+0000 7ff7467fc640 1 -- 172.21.15.53:0/3158849252 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73802f830 con 0x7ff748161320 2023-10-21T17:40:43.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.190+0000 7ff7467fc640 1 -- 172.21.15.53:0/3158849252 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73802f990 con 0x7ff748161320 2023-10-21T17:40:43.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.190+0000 7ff7467fc640 1 -- 172.21.15.53:0/3158849252 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73802fd20 con 0x7ff748161320 2023-10-21T17:40:43.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 -- 172.21.15.53:0/3158849252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748161320 msgr2=0x7ff7480cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748161320 0x7ff7480cc210 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7ff738009870 tx=0x7ff73802ba10 comp rx=0 tx=0).stop 2023-10-21T17:40:43.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 -- 172.21.15.53:0/3158849252 shutdown_connections 2023-10-21T17:40:43.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff748164f70 0x7ff7480b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 --2- 172.21.15.53:0/3158849252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748161320 0x7ff7480cc210 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 -- 172.21.15.53:0/3158849252 >> 172.21.15.53:0/3158849252 conn(0x7ff7480cb0c0 msgr2=0x7ff7480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:43.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 -- 172.21.15.53:0/3158849252 shutdown_connections 2023-10-21T17:40:43.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.191+0000 7ff747fff640 1 -- 172.21.15.53:0/3158849252 wait complete. 2023-10-21T17:40:43.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.193+0000 7ff747fff640 1 Processor -- start 2023-10-21T17:40:43.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.193+0000 7ff747fff640 1 -- start start 2023-10-21T17:40:43.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff74806c710 0x7ff7480b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 0x7ff7480ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481fa080 con 0x7ff748164f70 2023-10-21T17:40:43.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481f7090 con 0x7ff748161320 2023-10-21T17:40:43.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff747fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481f7370 con 0x7ff74806c710 2023-10-21T17:40:43.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.199 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44490/0 (socket says 172.21.15.53:44490) 2023-10-21T17:40:43.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff7477fe640 1 -- 172.21.15.53:0/2672597512 learned_addr learned my addr 172.21.15.53:0/2672597512 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:43.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff73ffff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 0x7ff7480ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff7477fe640 1 -- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff74806c710 msgr2=0x7ff7480b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.194+0000 7ff746ffd640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff74806c710 0x7ff7480b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff7477fe640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff74806c710 0x7ff7480b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff7477fe640 1 -- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 msgr2=0x7ff7480ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff7477fe640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 0x7ff7480ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff7477fe640 1 -- 172.21.15.53:0/2672597512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380094d0 con 0x7ff748164f70 2023-10-21T17:40:43.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff73ffff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 0x7ff7480ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:43.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff7477fe640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c730 tx=0x7ff74000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740003b70 con 0x7ff748164f70 2023-10-21T17:40:43.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740003cd0 con 0x7ff748164f70 2023-10-21T17:40:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74000aa90 con 0x7ff748164f70 2023-10-21T17:40:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7481f7650 con 0x7ff748164f70 2023-10-21T17:40:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.195+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7480b17a0 con 0x7ff748164f70 2023-10-21T17:40:43.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.196+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7480b1960 con 0x7ff748164f70 2023-10-21T17:40:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.197+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff74000abf0 con 0x7ff748164f70 2023-10-21T17:40:43.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.199+0000 7ff73f7fe640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 0x7ff72806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.199+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff740089ab0 con 0x7ff748164f70 2023-10-21T17:40:43.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.200+0000 7ff746ffd640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 0x7ff72806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.201+0000 7ff746ffd640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 0x7ff72806d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff738036000 tx=0x7ff738039040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.206 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.203+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7400545a0 con 0x7ff748164f70 2023-10-21T17:40:43.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.402+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff74804e160 con 0x7ff748164f70 2023-10-21T17:40:43.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.405+0000 7ff73f7fe640 1 -- 172.21.15.53:0/2672597512 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v274) v1 ==== 74+0+16907 (secure 0 0 0) 0x7ff740057bc0 con 0x7ff748164f70 2023-10-21T17:40:43.407 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:43.407 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":274,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:41.957577+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:43.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 msgr2=0x7ff72806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 0x7ff72806d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff738036000 tx=0x7ff738039040 comp rx=0 tx=0).stop 2023-10-21T17:40:43.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 msgr2=0x7ff7480ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c730 tx=0x7ff74000cbf0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 shutdown_connections 2023-10-21T17:40:43.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7ff72806b0e0 0x7ff72806d5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff748164f70 0x7ff7480ae5e0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff748161320 0x7ff7480ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 --2- 172.21.15.53:0/2672597512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff74806c710 0x7ff7480b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 >> 172.21.15.53:0/2672597512 conn(0x7ff7480cb0c0 msgr2=0x7ff74815f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:43.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 shutdown_connections 2023-10-21T17:40:43.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.408+0000 7ff747fff640 1 -- 172.21.15.53:0/2672597512 wait complete. 2023-10-21T17:40:43.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-21T17:40:43.658 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.657+0000 7f2e72a89640 1 Processor -- start 2023-10-21T17:40:43.659 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.657+0000 7f2e72a89640 1 -- start start 2023-10-21T17:40:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e72a89640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e6c06dfc0 0x7f2e6c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e72a89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e6c071070 0x7f2e6c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e72a89640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0b1cc0 con 0x7f2e6c0afbe0 2023-10-21T17:40:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e72a89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0b2620 con 0x7f2e6c071070 2023-10-21T17:40:43.661 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e72a89640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0b2060 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e71026640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e6c071070 0x7f2e6c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e71026640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2e6c071070 0x7f2e6c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44498/0 (socket says 172.21.15.53:44498) 2023-10-21T17:40:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e71026640 1 -- 172.21.15.53:0/1356610605 learned_addr learned my addr 172.21.15.53:0/1356610605 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e6bfff640 1 --2- 172.21.15.53:0/1356610605 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c06dfc0 0x7f2e6c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e71026640 1 -- 172.21.15.53:0/1356610605 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c06dfc0 msgr2=0x7f2e6c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.659+0000 7f2e71026640 1 --2- 172.21.15.53:0/1356610605 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c06dfc0 0x7f2e6c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.660+0000 7f2e71026640 1 -- 172.21.15.53:0/1356610605 >> v1:172.21.15.124:6789/0 conn(0x7f2e6c0afbe0 legacy=0x7f2e6c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:43.663 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.660+0000 7f2e71026640 1 -- 172.21.15.53:0/1356610605 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0b2340 con 0x7f2e6c071070 2023-10-21T17:40:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.661+0000 7f2e71026640 1 --2- 172.21.15.53:0/1356610605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cb130 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b3980 tx=0x7f2e6402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68086384dd5b44c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.661+0000 7f2e6b7fe640 1 -- 172.21.15.53:0/1356610605 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6402ca80 con 0x7f2e6c071070 2023-10-21T17:40:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.661+0000 7f2e6b7fe640 1 -- 172.21.15.53:0/1356610605 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6402cbe0 con 0x7f2e6c071070 2023-10-21T17:40:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.661+0000 7f2e6b7fe640 1 -- 172.21.15.53:0/1356610605 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640343f0 con 0x7f2e6c071070 2023-10-21T17:40:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 -- 172.21.15.53:0/1356610605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c071070 msgr2=0x7f2e6c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 --2- 172.21.15.53:0/1356610605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cb130 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0b3980 tx=0x7f2e6402bee0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 -- 172.21.15.53:0/1356610605 shutdown_connections 2023-10-21T17:40:43.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 --2- 172.21.15.53:0/1356610605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cb130 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 --2- 172.21.15.53:0/1356610605 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c06dfc0 0x7f2e6c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 -- 172.21.15.53:0/1356610605 >> 172.21.15.53:0/1356610605 conn(0x7f2e6c06a5d0 msgr2=0x7f2e6c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 -- 172.21.15.53:0/1356610605 shutdown_connections 2023-10-21T17:40:43.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.662+0000 7f2e72a89640 1 -- 172.21.15.53:0/1356610605 wait complete. 2023-10-21T17:40:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.664+0000 7f2e72a89640 1 Processor -- start 2023-10-21T17:40:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.664+0000 7f2e72a89640 1 -- start start 2023-10-21T17:40:43.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c0afbe0 0x7f2e6c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0c40b0 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c1f2710 con 0x7f2e6c071070 2023-10-21T17:40:43.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e72a89640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0c3470 con 0x7f2e6c0afbe0 2023-10-21T17:40:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:44512/0 (socket says 172.21.15.53:44512) 2023-10-21T17:40:43.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 -- 172.21.15.53:0/3234534180 learned_addr learned my addr 172.21.15.53:0/3234534180 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e6bfff640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 -- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c0afbe0 msgr2=0x7f2e6c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c0afbe0 0x7f2e6c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 -- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 msgr2=0x7f2e6c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:43.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e70825640 1 -- 172.21.15.53:0/3234534180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64009b30 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.666+0000 7f2e6bfff640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e70825640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b3f0 tx=0x7f2e6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011830 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e60011990 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011d40 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c0c50a0 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.667+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c0c39f0 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.668+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c0c41f0 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.669+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2e6001a400 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.670+0000 7f2e697fa640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 0x7f2e5006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.670+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2e60089990 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.671+0000 7f2e6bfff640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 0x7f2e5006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:43.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.672+0000 7f2e6bfff640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 0x7f2e5006d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c150ac0 tx=0x7f2e58008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:43.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.675+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e6c0c41f0 con 0x7f2e6c06dfc0 2023-10-21T17:40:43.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:43.911+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2e6c0c41f0 con 0x7f2e6c06dfc0 2023-10-21T17:40:44.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.541+0000 7f2e697fa640 1 -- 172.21.15.53:0/3234534180 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v142) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2e6c0c41f0 con 0x7f2e6c06dfc0 2023-10-21T17:40:44.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 msgr2=0x7f2e5006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 0x7f2e5006d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c150ac0 tx=0x7f2e58008040 comp rx=0 tx=0).stop 2023-10-21T17:40:44.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 msgr2=0x7f2e6c150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b3f0 tx=0x7f2e6000b8b0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 shutdown_connections 2023-10-21T17:40:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2e5006b0b0 0x7f2e5006d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c0afbe0 0x7f2e6c0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c071070 0x7f2e6c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 --2- 172.21.15.53:0/3234534180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e6c06dfc0 0x7f2e6c150580 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 >> 172.21.15.53:0/3234534180 conn(0x7f2e6c06a5d0 msgr2=0x7f2e6c0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 shutdown_connections 2023-10-21T17:40:44.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.545+0000 7f2e72a89640 1 -- 172.21.15.53:0/3234534180 wait complete. 2023-10-21T17:40:44.569 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-21T17:40:44.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-21T17:40:44.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.804+0000 7fa07359e640 1 Processor -- start 2023-10-21T17:40:44.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.805+0000 7fa07359e640 1 -- start start 2023-10-21T17:40:44.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.806+0000 7fa07359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa0740ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.806+0000 7fa07359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740ce9b0 con 0x7fa0740b2180 2023-10-21T17:40:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.806+0000 7fa07359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740cec90 con 0x7fa07406bcb0 2023-10-21T17:40:44.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.806+0000 7fa07359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740cef70 con 0x7fa07415ccd0 2023-10-21T17:40:44.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.807+0000 7fa072d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa0740b2180 0x7fa0740cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46826/0 (socket says 172.21.15.53:46826) 2023-10-21T17:40:44.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.807+0000 7fa072d9d640 1 -- 172.21.15.53:0/295286678 learned_addr learned my addr 172.21.15.53:0/295286678 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:44.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.807+0000 7fa07259c640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa0740ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.807+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743557854 0 0) 0x7fa0740ce9b0 con 0x7fa0740b2180 2023-10-21T17:40:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.807+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003520 con 0x7fa0740b2180 2023-10-21T17:40:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193501082 0 0) 0x7fa05c003520 con 0x7fa0740b2180 2023-10-21T17:40:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 msgr2=0x7fa0740ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:44.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa0740ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 >> v1:172.21.15.124:6789/0 conn(0x7fa07406bcb0 legacy=0x7fa074160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074160880 con 0x7fa0740b2180 2023-10-21T17:40:44.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fa06c0019f0 con 0x7fa0740b2180 2023-10-21T17:40:44.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07259c640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa0740ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:40:44.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 >> v1:172.21.15.53:6789/0 conn(0x7fa0740b2180 legacy=0x7fa0740cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa05c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa05c003f30 0x7fa05c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa05c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740cef70 con 0x7fa05c004d00 2023-10-21T17:40:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c00ba90 con 0x7fa05c003f30 2023-10-21T17:40:44.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c00bc50 con 0x7fa07415ccd0 2023-10-21T17:40:44.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.808+0000 7fa072d9d640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa05c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa07259c640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa05c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa071d9b640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa05c003f30 0x7fa05c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa072d9d640 1 -- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 msgr2=0x7fa05c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa072d9d640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415ccd0 0x7fa05c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa072d9d640 1 -- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa05c003f30 msgr2=0x7fa05c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa072d9d640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa05c003f30 0x7fa05c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.809+0000 7fa072d9d640 1 -- 172.21.15.53:0/295286678 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009400 con 0x7fa05c004d00 2023-10-21T17:40:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.810+0000 7fa072d9d640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa05c005180 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fa06c002230 tx=0x7fa05c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8092a54e3528589 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.810+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c034630 con 0x7fa05c004d00 2023-10-21T17:40:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.810+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c02f070 con 0x7fa05c004d00 2023-10-21T17:40:44.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.810+0000 7fa07159a640 1 -- 172.21.15.53:0/295286678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c034a50 con 0x7fa05c004d00 2023-10-21T17:40:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 -- 172.21.15.53:0/295286678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 msgr2=0x7fa05c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 --2- 172.21.15.53:0/295286678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa05c005180 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fa06c002230 tx=0x7fa05c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa072d9d640 1 -- 172.21.15.53:0/295286678 reap_dead start 2023-10-21T17:40:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 -- 172.21.15.53:0/295286678 shutdown_connections 2023-10-21T17:40:44.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 -- 172.21.15.53:0/295286678 >> 172.21.15.53:0/295286678 conn(0x7fa0740cd180 msgr2=0x7fa0740cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 -- 172.21.15.53:0/295286678 shutdown_connections 2023-10-21T17:40:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.811+0000 7fa07359e640 1 -- 172.21.15.53:0/295286678 wait complete. 2023-10-21T17:40:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.813+0000 7fa07359e640 1 Processor -- start 2023-10-21T17:40:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.813+0000 7fa07359e640 1 -- start start 2023-10-21T17:40:44.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415dac0 0x7fa07415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0741f2180 0x7fa0741f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741f3450 con 0x7fa05c004d00 2023-10-21T17:40:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415e630 con 0x7fa0741f2180 2023-10-21T17:40:44.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07415e910 con 0x7fa07415dac0 2023-10-21T17:40:44.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34912/0 (socket says 172.21.15.53:34912) 2023-10-21T17:40:44.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.814+0000 7fa07259c640 1 -- 172.21.15.53:0/3681830036 learned_addr learned my addr 172.21.15.53:0/3681830036 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:44.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa071d9b640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415dac0 0x7fa07415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 -- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415dac0 msgr2=0x7fa07415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415dac0 0x7fa07415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 -- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0741f2180 msgr2=0x7fa0741f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0741f2180 0x7fa0741f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 -- 172.21.15.53:0/3681830036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07415ebf0 con 0x7fa05c004d00 2023-10-21T17:40:44.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa07259c640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fa064011cd0 tx=0x7fa06400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.815+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400fd20 con 0x7fa05c004d00 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.816+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400fe80 con 0x7fa05c004d00 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.816+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064004240 con 0x7fa05c004d00 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.816+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07415eed0 con 0x7fa05c004d00 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.816+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0741f2b40 con 0x7fa05c004d00 2023-10-21T17:40:44.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.817+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07415f0c0 con 0x7fa05c004d00 2023-10-21T17:40:44.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.818+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa0640043a0 con 0x7fa05c004d00 2023-10-21T17:40:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.819+0000 7fa062ffd640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 0x7fa05406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.819+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa06408a5f0 con 0x7fa05c004d00 2023-10-21T17:40:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.820+0000 7fa071d9b640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 0x7fa05406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:44.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.821+0000 7fa071d9b640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 0x7fa05406d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa068007fd0 tx=0x7fa068009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:44.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:44.823+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa064055210 con 0x7fa05c004d00 2023-10-21T17:40:45.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.060+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa074074150 con 0x7fa05c004d00 2023-10-21T17:40:45.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.556+0000 7fa062ffd640 1 -- 172.21.15.53:0/3681830036 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v143) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa064058830 con 0x7fa05c004d00 2023-10-21T17:40:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 msgr2=0x7fa05406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 0x7fa05406d570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa068007fd0 tx=0x7fa068009040 comp rx=0 tx=0).stop 2023-10-21T17:40:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 msgr2=0x7fa0741532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7fa064011cd0 tx=0x7fa06400f610 comp rx=0 tx=0).stop 2023-10-21T17:40:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 shutdown_connections 2023-10-21T17:40:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7fa05406b0b0 0x7fa05406d570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0741f2180 0x7fa0741f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa07415dac0 0x7fa07415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 --2- 172.21.15.53:0/3681830036 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa05c004d00 0x7fa0741532e0 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 >> 172.21.15.53:0/3681830036 conn(0x7fa0740cd180 msgr2=0x7fa0740b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 shutdown_connections 2023-10-21T17:40:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.560+0000 7fa07359e640 1 -- 172.21.15.53:0/3681830036 wait complete. 2023-10-21T17:40:45.584 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-21T17:40:45.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:45.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.814+0000 7febaddf4640 1 Processor -- start 2023-10-21T17:40:45.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.814+0000 7febaddf4640 1 -- start start 2023-10-21T17:40:45.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febaddf4640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feba8071010 0x7feba815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febaddf4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba815d7d0 con 0x7feba8150440 2023-10-21T17:40:45.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febaddf4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba80714a0 con 0x7feba815d380 2023-10-21T17:40:45.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febaddf4640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba80c9f90 con 0x7feba8071010 2023-10-21T17:40:45.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febacdf2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feba8150440 0x7feba80c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46834/0 (socket says 172.21.15.53:46834) 2023-10-21T17:40:45.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.816+0000 7febacdf2640 1 -- 172.21.15.53:0/3118832002 learned_addr learned my addr 172.21.15.53:0/3118832002 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:45.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368186315 0 0) 0x7feba815d7d0 con 0x7feba8150440 2023-10-21T17:40:45.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba7fff640 1 --2- 172.21.15.53:0/3118832002 >> v2:172.21.15.124:3300/0 conn(0x7feba8071010 0x7feba815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb98003520 con 0x7feba8150440 2023-10-21T17:40:45.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373295715 0 0) 0x7feb98003520 con 0x7feba8150440 2023-10-21T17:40:45.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 >> v2:172.21.15.124:3300/0 conn(0x7feba8071010 msgr2=0x7feba815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:45.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 --2- 172.21.15.53:0/3118832002 >> v2:172.21.15.124:3300/0 conn(0x7feba8071010 0x7feba815ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 >> v1:172.21.15.124:6789/0 conn(0x7feba815d380 legacy=0x7feba80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80ca270 con 0x7feba8150440 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7feb9c001670 con 0x7feba8150440 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.817+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 >> v1:172.21.15.53:6789/0 conn(0x7feba8150440 legacy=0x7feba80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 0x7feb98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980047c0 0x7feb98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb98005180 0x7feb9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80c9f90 con 0x7feb980039c0 2023-10-21T17:40:45.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9800c080 con 0x7feb980047c0 2023-10-21T17:40:45.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9800c240 con 0x7feb98005180 2023-10-21T17:40:45.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 0x7feb98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba7fff640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980047c0 0x7feb98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febad5f3640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb98005180 0x7feb9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 -- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb98005180 msgr2=0x7feb9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:45.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb98005180 0x7feb9800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 -- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980047c0 msgr2=0x7feb98004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:45.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980047c0 0x7feb98004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febacdf2640 1 -- 172.21.15.53:0/3118832002 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98003520 con 0x7feb980039c0 2023-10-21T17:40:45.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7febad5f3640 1 -- 172.21.15.53:0/3118832002 reap_dead start 2023-10-21T17:40:45.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.818+0000 7feba7fff640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980047c0 0x7feb98004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:45.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.819+0000 7febacdf2640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 0x7feb98003de0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7feb9c007bc0 tx=0x7feb980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902cb21a8c38653 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:45.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02ebf0 con 0x7feb980039c0 2023-10-21T17:40:45.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c02ed50 con 0x7feb980039c0 2023-10-21T17:40:45.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7feba77fe640 1 -- 172.21.15.53:0/3118832002 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c005b10 con 0x7feb980039c0 2023-10-21T17:40:45.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 -- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 msgr2=0x7feb98003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 0x7feb98003de0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7feb9c007bc0 tx=0x7feb980057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 -- 172.21.15.53:0/3118832002 shutdown_connections 2023-10-21T17:40:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 --2- 172.21.15.53:0/3118832002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb980039c0 0x7feb98003de0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 -- 172.21.15.53:0/3118832002 >> 172.21.15.53:0/3118832002 conn(0x7feba80cb0c0 msgr2=0x7feba80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:45.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.820+0000 7febaddf4640 1 -- 172.21.15.53:0/3118832002 shutdown_connections 2023-10-21T17:40:45.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.821+0000 7febaddf4640 1 -- 172.21.15.53:0/3118832002 wait complete. 2023-10-21T17:40:45.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.822+0000 7febaddf4640 1 Processor -- start 2023-10-21T17:40:45.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.823+0000 7febaddf4640 1 -- start start 2023-10-21T17:40:45.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.823+0000 7febaddf4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 0x7feba80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.823+0000 7febaddf4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febaddf4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 0x7feba8158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febaddf4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8158ca0 con 0x7feba8147a00 2023-10-21T17:40:45.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febaddf4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80af7c0 con 0x7feb980039c0 2023-10-21T17:40:45.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febaddf4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba80afaa0 con 0x7feba81582e0 2023-10-21T17:40:45.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34926/0 (socket says 172.21.15.53:34926) 2023-10-21T17:40:45.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 -- 172.21.15.53:0/450453132 learned_addr learned my addr 172.21.15.53:0/450453132 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:45.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febacdf2640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 0x7feba80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febad5f3640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 0x7feba8158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 -- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 msgr2=0x7feba8158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 0x7feba8158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 -- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 msgr2=0x7feba80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:45.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 0x7feba80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:45.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7feba7fff640 1 -- 172.21.15.53:0/450453132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80afd80 con 0x7feba8147a00 2023-10-21T17:40:45.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febacdf2640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 0x7feba80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.824+0000 7febad5f3640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 0x7feba8158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7feba7fff640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7feb9400f6a0 tx=0x7feb9400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9400dc50 con 0x7feba8147a00 2023-10-21T17:40:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9400ddb0 con 0x7feba8147a00 2023-10-21T17:40:45.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94012680 con 0x7feba8147a00 2023-10-21T17:40:45.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba80b0060 con 0x7feba8147a00 2023-10-21T17:40:45.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.825+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba8147e80 con 0x7feba8147a00 2023-10-21T17:40:45.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.826+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba804e160 con 0x7feba8147a00 2023-10-21T17:40:45.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.827+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7feb940127e0 con 0x7feba8147a00 2023-10-21T17:40:45.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.828+0000 7feba57fa640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 0x7feb8c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:45.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.828+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feb94089fe0 con 0x7feba8147a00 2023-10-21T17:40:45.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.829+0000 7febacdf2640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 0x7feb8c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:45.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.830+0000 7febacdf2640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 0x7feb8c06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb9c036000 tx=0x7feb9c009030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:45.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:45.833+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb94054a20 con 0x7feba8147a00 2023-10-21T17:40:46.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.072+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feba80b45c0 con 0x7feba8147a00 2023-10-21T17:40:46.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.073+0000 7feba57fa640 1 -- 172.21.15.53:0/450453132 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7feb94020070 con 0x7feba8147a00 2023-10-21T17:40:46.074 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:46.075 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:40:46.075 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:40:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 msgr2=0x7feb8c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 0x7feb8c06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb9c036000 tx=0x7feb9c009030 comp rx=0 tx=0).stop 2023-10-21T17:40:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 msgr2=0x7feba80aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7feb9400f6a0 tx=0x7feb9400fb60 comp rx=0 tx=0).stop 2023-10-21T17:40:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 shutdown_connections 2023-10-21T17:40:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7feb8c06b0b0 0x7feb8c06d570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feba81582e0 0x7feba8158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feba8147a00 0x7feba80aee90 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 --2- 172.21.15.53:0/450453132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb980039c0 0x7feba80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 >> 172.21.15.53:0/450453132 conn(0x7feba80cb0c0 msgr2=0x7feba80cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 shutdown_connections 2023-10-21T17:40:46.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.076+0000 7febaddf4640 1 -- 172.21.15.53:0/450453132 wait complete. 2023-10-21T17:40:46.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:40:46.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.323+0000 7f8c96e68640 1 Processor -- start 2023-10-21T17:40:46.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.324+0000 7f8c96e68640 1 -- start start 2023-10-21T17:40:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.325+0000 7f8c96e68640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c90159ac0 0x7f8c9006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.325+0000 7f8c96e68640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8c900709e0 0x7f8c900c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.325+0000 7f8c96e68640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9006dee0 con 0x7f8c900ad790 2023-10-21T17:40:46.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.325+0000 7f8c96e68640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9015d850 con 0x7f8c90159ac0 2023-10-21T17:40:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.325+0000 7f8c96e68640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9015db30 con 0x7f8c900709e0 2023-10-21T17:40:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c94c04640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c90159ac0 0x7f8c9006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c8ffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8c900ad790 0x7f8c900702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46844/0 (socket says 172.21.15.53:46844) 2023-10-21T17:40:46.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c8ffff640 1 -- 172.21.15.53:0/803178380 learned_addr learned my addr 172.21.15.53:0/803178380 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:46.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c95405640 1 --2- 172.21.15.53:0/803178380 >> v2:172.21.15.124:3300/0 conn(0x7f8c900709e0 0x7f8c900c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c94c04640 1 -- 172.21.15.53:0/803178380 >> v2:172.21.15.124:3300/0 conn(0x7f8c900709e0 msgr2=0x7f8c900c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:46.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c94c04640 1 --2- 172.21.15.53:0/803178380 >> v2:172.21.15.124:3300/0 conn(0x7f8c900709e0 0x7f8c900c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c94c04640 1 -- 172.21.15.53:0/803178380 >> v1:172.21.15.53:6789/0 conn(0x7f8c900ad790 legacy=0x7f8c900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.326+0000 7f8c94c04640 1 -- 172.21.15.53:0/803178380 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9015de10 con 0x7f8c90159ac0 2023-10-21T17:40:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.328+0000 7f8c94c04640 1 --2- 172.21.15.53:0/803178380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90159ac0 0x7f8c9006d5e0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f8c900caab0 tx=0x7f8c8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60cee7f7dcb9b68 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.328+0000 7f8c8f7fe640 1 -- 172.21.15.53:0/803178380 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031830 con 0x7f8c90159ac0 2023-10-21T17:40:46.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.328+0000 7f8c8f7fe640 1 -- 172.21.15.53:0/803178380 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80031990 con 0x7f8c90159ac0 2023-10-21T17:40:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.328+0000 7f8c8f7fe640 1 -- 172.21.15.53:0/803178380 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031d20 con 0x7f8c90159ac0 2023-10-21T17:40:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 -- 172.21.15.53:0/803178380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90159ac0 msgr2=0x7f8c9006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 --2- 172.21.15.53:0/803178380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90159ac0 0x7f8c9006d5e0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f8c900caab0 tx=0x7f8c8002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 -- 172.21.15.53:0/803178380 shutdown_connections 2023-10-21T17:40:46.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 --2- 172.21.15.53:0/803178380 >> v2:172.21.15.124:3300/0 conn(0x7f8c900709e0 0x7f8c900c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 --2- 172.21.15.53:0/803178380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c90159ac0 0x7f8c9006d5e0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 -- 172.21.15.53:0/803178380 >> 172.21.15.53:0/803178380 conn(0x7f8c9006a380 msgr2=0x7f8c900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:46.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 -- 172.21.15.53:0/803178380 shutdown_connections 2023-10-21T17:40:46.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.329+0000 7f8c96e68640 1 -- 172.21.15.53:0/803178380 wait complete. 2023-10-21T17:40:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.331+0000 7f8c96e68640 1 Processor -- start 2023-10-21T17:40:46.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.331+0000 7f8c96e68640 1 -- start start 2023-10-21T17:40:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c900ad790 0x7f8c9015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c90159ac0 0x7f8c9015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9015a250 con 0x7f8c900709e0 2023-10-21T17:40:46.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c901477e0 con 0x7f8c900ad790 2023-10-21T17:40:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c96e68640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b2be0 con 0x7f8c90159ac0 2023-10-21T17:40:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34956/0 (socket says 172.21.15.53:34956) 2023-10-21T17:40:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 -- 172.21.15.53:0/2289975422 learned_addr learned my addr 172.21.15.53:0/2289975422 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:46.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 -- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c90159ac0 msgr2=0x7f8c9015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c90159ac0 0x7f8c9015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 -- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c900ad790 msgr2=0x7f8c9015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c900ad790 0x7f8c9015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.332+0000 7f8c94c04640 1 -- 172.21.15.53:0/2289975422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009400 con 0x7f8c900709e0 2023-10-21T17:40:46.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c94c04640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8003aa60 con 0x7f8c900709e0 2023-10-21T17:40:46.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8003abe0 con 0x7f8c900709e0 2023-10-21T17:40:46.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800443f0 con 0x7f8c900709e0 2023-10-21T17:40:46.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c900b2340 con 0x7f8c900709e0 2023-10-21T17:40:46.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.333+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900b2900 con 0x7f8c900709e0 2023-10-21T17:40:46.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.335+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90067490 con 0x7f8c900709e0 2023-10-21T17:40:46.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.335+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8c8004d450 con 0x7f8c900709e0 2023-10-21T17:40:46.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.337+0000 7f8c8d7fa640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 0x7f8c7c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.337+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8c800abd90 con 0x7f8c900709e0 2023-10-21T17:40:46.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.337+0000 7f8c8ffff640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 0x7f8c7c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.339+0000 7f8c8ffff640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 0x7f8c7c06d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8c900cdcd0 tx=0x7f8c84007a40 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.341+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c800769c0 con 0x7f8c900709e0 2023-10-21T17:40:46.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.540+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c900b0ff0 con 0x7f8c900709e0 2023-10-21T17:40:46.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.542+0000 7f8c8d7fa640 1 -- 172.21.15.53:0/2289975422 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v275) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f8c80035070 con 0x7f8c900709e0 2023-10-21T17:40:46.543 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:46.544 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":275,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:43.507189+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:40:46.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 msgr2=0x7f8c7c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 0x7f8c7c06d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8c900cdcd0 tx=0x7f8c84007a40 comp rx=0 tx=0).stop 2023-10-21T17:40:46.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 msgr2=0x7f8c901472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c800046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 shutdown_connections 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f8c7c06b0e0 0x7f8c7c06d5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8c90159ac0 0x7f8c9015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8c900ad790 0x7f8c9015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 --2- 172.21.15.53:0/2289975422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c900709e0 0x7f8c901472a0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.544+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 >> 172.21.15.53:0/2289975422 conn(0x7f8c9006a380 msgr2=0x7f8c900cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:46.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.545+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 shutdown_connections 2023-10-21T17:40:46.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.545+0000 7f8c96e68640 1 -- 172.21.15.53:0/2289975422 wait complete. 2023-10-21T17:40:46.566 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-21T17:40:46.567 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:40:46.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.798+0000 7f6f69f21640 1 Processor -- start 2023-10-21T17:40:46.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.798+0000 7f6f69f21640 1 -- start start 2023-10-21T17:40:46.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f6415ab30 0x7f6f6415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f640adc70 0x7f6f640b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f6415af10 0x7f6f640b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6415f7d0 con 0x7f6f6415ab30 2023-10-21T17:40:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6415f210 con 0x7f6f6415af10 2023-10-21T17:40:46.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f69f21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6415f4f0 con 0x7f6f640adc70 2023-10-21T17:40:46.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.800+0000 7f6f68f1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f6415ab30 0x7f6f6415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6f6415ab30 0x7f6f6415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34966/0 (socket says 172.21.15.53:34966) 2023-10-21T17:40:46.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 -- 172.21.15.53:0/1553559483 learned_addr learned my addr 172.21.15.53:0/1553559483 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:46.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f69720640 1 --2- 172.21.15.53:0/1553559483 >> v2:172.21.15.124:3300/0 conn(0x7f6f6415af10 0x7f6f640b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f640adc70 0x7f6f640b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 -- 172.21.15.53:0/1553559483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f640adc70 msgr2=0x7f6f640b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:46.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f640adc70 0x7f6f640b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.805 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 -- 172.21.15.53:0/1553559483 >> v2:172.21.15.124:3300/0 conn(0x7f6f6415af10 msgr2=0x7f6f640b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:46.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/1553559483 >> v2:172.21.15.124:3300/0 conn(0x7f6f6415af10 0x7f6f640b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.801+0000 7f6f68f1f640 1 -- 172.21.15.53:0/1553559483 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6406c550 con 0x7f6f6415ab30 2023-10-21T17:40:46.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.802+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f6415ab30 0x7f6f6415e8a0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6f640cb000 tx=0x7f6f6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593538e48580c1a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.803+0000 7f6f5b7fe640 1 -- 172.21.15.53:0/1553559483 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60031870 con 0x7f6f6415ab30 2023-10-21T17:40:46.806 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.803+0000 7f6f5b7fe640 1 -- 172.21.15.53:0/1553559483 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f600319d0 con 0x7f6f6415ab30 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.803+0000 7f6f5b7fe640 1 -- 172.21.15.53:0/1553559483 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60031d60 con 0x7f6f6415ab30 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.803+0000 7f6f69f21640 1 -- 172.21.15.53:0/1553559483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f6415ab30 msgr2=0x7f6f6415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.803+0000 7f6f69f21640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f6415ab30 0x7f6f6415e8a0 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6f640cb000 tx=0x7f6f6002b910 comp rx=0 tx=0).stop 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 -- 172.21.15.53:0/1553559483 shutdown_connections 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f640adc70 0x7f6f640b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 --2- 172.21.15.53:0/1553559483 >> v2:172.21.15.124:3300/0 conn(0x7f6f6415af10 0x7f6f640b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 --2- 172.21.15.53:0/1553559483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f6415ab30 0x7f6f6415e8a0 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 -- 172.21.15.53:0/1553559483 >> 172.21.15.53:0/1553559483 conn(0x7f6f640cb0c0 msgr2=0x7f6f640cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:46.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 -- 172.21.15.53:0/1553559483 shutdown_connections 2023-10-21T17:40:46.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.804+0000 7f6f69f21640 1 -- 172.21.15.53:0/1553559483 wait complete. 2023-10-21T17:40:46.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.805+0000 7f6f69f21640 1 Processor -- start 2023-10-21T17:40:46.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.806+0000 7f6f69f21640 1 -- start start 2023-10-21T17:40:46.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 0x7f6f6415ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f6415af10 0x7f6f640c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6415d040 con 0x7f6f640adc70 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640c7d40 con 0x7f6f6415ab30 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f69f21640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640c8020 con 0x7f6f6415af10 2023-10-21T17:40:46.811 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34976/0 (socket says 172.21.15.53:34976) 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 -- 172.21.15.53:0/2507686060 learned_addr learned my addr 172.21.15.53:0/2507686060 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 -- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f6415af10 msgr2=0x7f6f640c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f6415af10 0x7f6f640c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 -- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 msgr2=0x7f6f6415ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:46.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 0x7f6f6415ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 0x7f6f6415ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:46.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f68f1f640 1 -- 172.21.15.53:0/2507686060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600094d0 con 0x7f6f640adc70 2023-10-21T17:40:46.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.807+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 0x7f6f6415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:46.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f68f1f640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002b950 tx=0x7f6f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6003ad30 con 0x7f6f640adc70 2023-10-21T17:40:46.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f600443f0 con 0x7f6f640adc70 2023-10-21T17:40:46.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f640c8300 con 0x7f6f640adc70 2023-10-21T17:40:46.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600447a0 con 0x7f6f640adc70 2023-10-21T17:40:46.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.808+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f640c79d0 con 0x7f6f640adc70 2023-10-21T17:40:46.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.810+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6415b720 con 0x7f6f640adc70 2023-10-21T17:40:46.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.810+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6f6004d450 con 0x7f6f640adc70 2023-10-21T17:40:46.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.812+0000 7f6f597fa640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 0x7f6f5006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:46.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.812+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6f6002d040 con 0x7f6f640adc70 2023-10-21T17:40:46.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.813+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 0x7f6f5006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:46.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.814+0000 7f6f5bfff640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 0x7f6f5006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:46.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:46.816+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f6415b720 con 0x7f6f640adc70 2023-10-21T17:40:46.874 INFO:tasks.ceph.mds.b.smithi124.stderr:2023-10-21T17:40:46.872+0000 7f59a2008640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:40:47.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.013+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 --> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f6406c830 con 0x7f6f5006b0b0 2023-10-21T17:40:47.042 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.041+0000 7f6f597fa640 1 -- 172.21.15.53:0/2507686060 <== mgr.9841 v2:172.21.15.124:6832/596989303 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287276 (secure 0 0 0) 0x7f6f54004030 con 0x7f6f5006b0b0 2023-10-21T17:40:47.046 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:47.048 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:40:47.049 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.046+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 msgr2=0x7f6f5006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.046+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 0x7f6f5006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f54006fd0 tx=0x7f6f54008040 comp rx=0 tx=0).stop 2023-10-21T17:40:47.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.046+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 msgr2=0x7f6f641576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.046+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f6f6002b950 tx=0x7f6f600046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 shutdown_connections 2023-10-21T17:40:47.050 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6f5006b0b0 0x7f6f5006d570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f6415af10 0x7f6f640c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f6415ab30 0x7f6f6415ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 --2- 172.21.15.53:0/2507686060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f640adc70 0x7f6f641576a0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 >> 172.21.15.53:0/2507686060 conn(0x7f6f640cb0c0 msgr2=0x7f6f640b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:47.051 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 shutdown_connections 2023-10-21T17:40:47.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.047+0000 7f6f69f21640 1 -- 172.21.15.53:0/2507686060 wait complete. 2023-10-21T17:40:47.069 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-21T17:40:46.950298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":695,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4947,"ondisk_log_size":4947,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460992,"kb_used_data":28288,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754513728,"statfs":{"total":773094113280,"available":772622057472,"internally_reserved":0,"allocated":28966912,"data_stored":15695148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002112"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545746+0000","last_change":"2023-10-21T17:40:42.541474+0000","last_active":"2023-10-21T17:40:42.545746+0000","last_peered":"2023-10-21T17:40:42.545746+0000","last_clean":"2023-10-21T17:40:42.545746+0000","last_became_active":"2023-10-21T17:40:42.540917+0000","last_became_peered":"2023-10-21T17:40:42.540917+0000","last_unstale":"2023-10-21T17:40:42.545746+0000","last_undegraded":"2023-10-21T17:40:42.545746+0000","last_fullsized":"2023-10-21T17:40:42.545746+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:35.102557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.552547+0000","last_change":"2023-10-21T17:40:42.551089+0000","last_active":"2023-10-21T17:40:42.552547+0000","last_peered":"2023-10-21T17:40:42.552547+0000","last_clean":"2023-10-21T17:40:42.552547+0000","last_became_active":"2023-10-21T17:40:42.550812+0000","last_became_peered":"2023-10-21T17:40:42.550812+0000","last_unstale":"2023-10-21T17:40:42.552547+0000","last_undegraded":"2023-10-21T17:40:42.552547+0000","last_fullsized":"2023-10-21T17:40:42.552547+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:51.392394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515857+0000","last_change":"2023-10-21T17:40:42.549313+0000","last_active":"2023-10-21T17:40:43.515857+0000","last_peered":"2023-10-21T17:40:43.515857+0000","last_clean":"2023-10-21T17:40:43.515857+0000","last_became_active":"2023-10-21T17:40:42.549026+0000","last_became_peered":"2023-10-21T17:40:42.549026+0000","last_unstale":"2023-10-21T17:40:43.515857+0000","last_undegraded":"2023-10-21T17:40:43.515857+0000","last_fullsized":"2023-10-21T17:40:43.515857+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:03.570137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515724+0000","last_change":"2023-10-21T17:40:42.549746+0000","last_active":"2023-10-21T17:40:43.515724+0000","last_peered":"2023-10-21T17:40:43.515724+0000","last_clean":"2023-10-21T17:40:43.515724+0000","last_became_active":"2023-10-21T17:40:42.532435+0000","last_became_peered":"2023-10-21T17:40:42.532435+0000","last_unstale":"2023-10-21T17:40:43.515724+0000","last_undegraded":"2023-10-21T17:40:43.515724+0000","last_fullsized":"2023-10-21T17:40:43.515724+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:41:31.405739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.529173+0000","last_change":"2023-10-21T17:40:42.526485+0000","last_active":"2023-10-21T17:40:42.529173+0000","last_peered":"2023-10-21T17:40:42.529173+0000","last_clean":"2023-10-21T17:40:42.529173+0000","last_became_active":"2023-10-21T17:40:42.526222+0000","last_became_peered":"2023-10-21T17:40:42.526222+0000","last_unstale":"2023-10-21T17:40:42.529173+0000","last_undegraded":"2023-10-21T17:40:42.529173+0000","last_fullsized":"2023-10-21T17:40:42.529173+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:55.928152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541090+0000","last_change":"2023-10-21T17:40:42.536351+0000","last_active":"2023-10-21T17:40:42.541090+0000","last_peered":"2023-10-21T17:40:42.541090+0000","last_clean":"2023-10-21T17:40:42.541090+0000","last_became_active":"2023-10-21T17:40:42.536084+0000","last_became_peered":"2023-10-21T17:40:42.536084+0000","last_unstale":"2023-10-21T17:40:42.541090+0000","last_undegraded":"2023-10-21T17:40:42.541090+0000","last_fullsized":"2023-10-21T17:40:42.541090+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:40.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.537030+0000","last_change":"2023-10-21T17:40:42.532275+0000","last_active":"2023-10-21T17:40:42.537030+0000","last_peered":"2023-10-21T17:40:42.537030+0000","last_clean":"2023-10-21T17:40:42.537030+0000","last_became_active":"2023-10-21T17:40:42.531933+0000","last_became_peered":"2023-10-21T17:40:42.531933+0000","last_unstale":"2023-10-21T17:40:42.537030+0000","last_undegraded":"2023-10-21T17:40:42.537030+0000","last_fullsized":"2023-10-21T17:40:42.537030+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:39.590353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515091+0000","last_change":"2023-10-21T17:40:42.525429+0000","last_active":"2023-10-21T17:40:43.515091+0000","last_peered":"2023-10-21T17:40:43.515091+0000","last_clean":"2023-10-21T17:40:43.515091+0000","last_became_active":"2023-10-21T17:40:42.524591+0000","last_became_peered":"2023-10-21T17:40:42.524591+0000","last_unstale":"2023-10-21T17:40:43.515091+0000","last_undegraded":"2023-10-21T17:40:43.515091+0000","last_fullsized":"2023-10-21T17:40:43.515091+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:47.082610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545436+0000","last_change":"2023-10-21T17:40:42.537534+0000","last_active":"2023-10-21T17:40:42.545436+0000","last_peered":"2023-10-21T17:40:42.545436+0000","last_clean":"2023-10-21T17:40:42.545436+0000","last_became_active":"2023-10-21T17:40:42.537236+0000","last_became_peered":"2023-10-21T17:40:42.537236+0000","last_unstale":"2023-10-21T17:40:42.545436+0000","last_undegraded":"2023-10-21T17:40:42.545436+0000","last_fullsized":"2023-10-21T17:40:42.545436+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:32.249928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553130+0000","last_change":"2023-10-21T17:40:42.549430+0000","last_active":"2023-10-21T17:40:42.553130+0000","last_peered":"2023-10-21T17:40:42.553130+0000","last_clean":"2023-10-21T17:40:42.553130+0000","last_became_active":"2023-10-21T17:40:42.549059+0000","last_became_peered":"2023-10-21T17:40:42.549059+0000","last_unstale":"2023-10-21T17:40:42.553130+0000","last_undegraded":"2023-10-21T17:40:42.553130+0000","last_fullsized":"2023-10-21T17:40:42.553130+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:01.221675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547464+0000","last_change":"2023-10-21T17:40:42.546712+0000","last_active":"2023-10-21T17:40:42.547464+0000","last_peered":"2023-10-21T17:40:42.547464+0000","last_clean":"2023-10-21T17:40:42.547464+0000","last_became_active":"2023-10-21T17:40:42.546181+0000","last_became_peered":"2023-10-21T17:40:42.546181+0000","last_unstale":"2023-10-21T17:40:42.547464+0000","last_undegraded":"2023-10-21T17:40:42.547464+0000","last_fullsized":"2023-10-21T17:40:42.547464+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:09:41.079080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546774+0000","last_change":"2023-10-21T17:40:42.542201+0000","last_active":"2023-10-21T17:40:42.546774+0000","last_peered":"2023-10-21T17:40:42.546774+0000","last_clean":"2023-10-21T17:40:42.546774+0000","last_became_active":"2023-10-21T17:40:42.541692+0000","last_became_peered":"2023-10-21T17:40:42.541692+0000","last_unstale":"2023-10-21T17:40:42.546774+0000","last_undegraded":"2023-10-21T17:40:42.546774+0000","last_fullsized":"2023-10-21T17:40:42.546774+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:29:03.361855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515707+0000","last_change":"2023-10-21T17:40:42.548391+0000","last_active":"2023-10-21T17:40:43.515707+0000","last_peered":"2023-10-21T17:40:43.515707+0000","last_clean":"2023-10-21T17:40:43.515707+0000","last_became_active":"2023-10-21T17:40:42.548104+0000","last_became_peered":"2023-10-21T17:40:42.548104+0000","last_unstale":"2023-10-21T17:40:43.515707+0000","last_undegraded":"2023-10-21T17:40:43.515707+0000","last_fullsized":"2023-10-21T17:40:43.515707+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:32:43.706722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.554931+0000","last_change":"2023-10-21T17:40:42.554249+0000","last_active":"2023-10-21T17:40:42.554931+0000","last_peered":"2023-10-21T17:40:42.554931+0000","last_clean":"2023-10-21T17:40:42.554931+0000","last_became_active":"2023-10-21T17:40:42.553948+0000","last_became_peered":"2023-10-21T17:40:42.553948+0000","last_unstale":"2023-10-21T17:40:42.554931+0000","last_undegraded":"2023-10-21T17:40:42.554931+0000","last_fullsized":"2023-10-21T17:40:42.554931+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:46.821913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515113+0000","last_change":"2023-10-21T17:40:42.549092+0000","last_active":"2023-10-21T17:40:43.515113+0000","last_peered":"2023-10-21T17:40:43.515113+0000","last_clean":"2023-10-21T17:40:43.515113+0000","last_became_active":"2023-10-21T17:40:42.543791+0000","last_became_peered":"2023-10-21T17:40:42.543791+0000","last_unstale":"2023-10-21T17:40:43.515113+0000","last_undegraded":"2023-10-21T17:40:43.515113+0000","last_fullsized":"2023-10-21T17:40:43.515113+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:48.102101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.518625+0000","last_change":"2023-10-21T17:40:42.535868+0000","last_active":"2023-10-21T17:40:43.518625+0000","last_peered":"2023-10-21T17:40:43.518625+0000","last_clean":"2023-10-21T17:40:43.518625+0000","last_became_active":"2023-10-21T17:40:42.535588+0000","last_became_peered":"2023-10-21T17:40:42.535588+0000","last_unstale":"2023-10-21T17:40:43.518625+0000","last_undegraded":"2023-10-21T17:40:43.518625+0000","last_fullsized":"2023-10-21T17:40:43.518625+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:30:36.869057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519696+0000","last_change":"2023-10-21T17:40:42.530418+0000","last_active":"2023-10-21T17:40:43.519696+0000","last_peered":"2023-10-21T17:40:43.519696+0000","last_clean":"2023-10-21T17:40:43.519696+0000","last_became_active":"2023-10-21T17:40:42.530061+0000","last_became_peered":"2023-10-21T17:40:42.530061+0000","last_unstale":"2023-10-21T17:40:43.519696+0000","last_undegraded":"2023-10-21T17:40:43.519696+0000","last_fullsized":"2023-10-21T17:40:43.519696+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:30:53.616708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516329+0000","last_change":"2023-10-21T17:40:42.553732+0000","last_active":"2023-10-21T17:40:43.516329+0000","last_peered":"2023-10-21T17:40:43.516329+0000","last_clean":"2023-10-21T17:40:43.516329+0000","last_became_active":"2023-10-21T17:40:42.552636+0000","last_became_peered":"2023-10-21T17:40:42.552636+0000","last_unstale":"2023-10-21T17:40:43.516329+0000","last_undegraded":"2023-10-21T17:40:43.516329+0000","last_fullsized":"2023-10-21T17:40:43.516329+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:30:53.616708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519640+0000","last_change":"2023-10-21T17:40:42.548597+0000","last_active":"2023-10-21T17:40:43.519640+0000","last_peered":"2023-10-21T17:40:43.519640+0000","last_clean":"2023-10-21T17:40:43.519640+0000","last_became_active":"2023-10-21T17:40:42.547150+0000","last_became_peered":"2023-10-21T17:40:42.547150+0000","last_unstale":"2023-10-21T17:40:43.519640+0000","last_undegraded":"2023-10-21T17:40:43.519640+0000","last_fullsized":"2023-10-21T17:40:43.519640+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:51:14.501138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519941+0000","last_change":"2023-10-21T17:40:42.532634+0000","last_active":"2023-10-21T17:40:43.519941+0000","last_peered":"2023-10-21T17:40:43.519941+0000","last_clean":"2023-10-21T17:40:43.519941+0000","last_became_active":"2023-10-21T17:40:42.532366+0000","last_became_peered":"2023-10-21T17:40:42.532366+0000","last_unstale":"2023-10-21T17:40:43.519941+0000","last_undegraded":"2023-10-21T17:40:43.519941+0000","last_fullsized":"2023-10-21T17:40:43.519941+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:27.870606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.527116+0000","last_change":"2023-10-21T17:40:42.526094+0000","last_active":"2023-10-21T17:40:42.527116+0000","last_peered":"2023-10-21T17:40:42.527116+0000","last_clean":"2023-10-21T17:40:42.527116+0000","last_became_active":"2023-10-21T17:40:42.525767+0000","last_became_peered":"2023-10-21T17:40:42.525767+0000","last_unstale":"2023-10-21T17:40:42.527116+0000","last_undegraded":"2023-10-21T17:40:42.527116+0000","last_fullsized":"2023-10-21T17:40:42.527116+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:16.796664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545787+0000","last_change":"2023-10-21T17:40:42.540489+0000","last_active":"2023-10-21T17:40:42.545787+0000","last_peered":"2023-10-21T17:40:42.545787+0000","last_clean":"2023-10-21T17:40:42.545787+0000","last_became_active":"2023-10-21T17:40:42.540224+0000","last_became_peered":"2023-10-21T17:40:42.540224+0000","last_unstale":"2023-10-21T17:40:42.545787+0000","last_undegraded":"2023-10-21T17:40:42.545787+0000","last_fullsized":"2023-10-21T17:40:42.545787+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:20.628311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.537009+0000","last_change":"2023-10-21T17:40:42.531940+0000","last_active":"2023-10-21T17:40:42.537009+0000","last_peered":"2023-10-21T17:40:42.537009+0000","last_clean":"2023-10-21T17:40:42.537009+0000","last_became_active":"2023-10-21T17:40:42.531537+0000","last_became_peered":"2023-10-21T17:40:42.531537+0000","last_unstale":"2023-10-21T17:40:42.537009+0000","last_undegraded":"2023-10-21T17:40:42.537009+0000","last_fullsized":"2023-10-21T17:40:42.537009+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:23:43.875691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550816+0000","last_change":"2023-10-21T17:40:42.548450+0000","last_active":"2023-10-21T17:40:42.550816+0000","last_peered":"2023-10-21T17:40:42.550816+0000","last_clean":"2023-10-21T17:40:42.550816+0000","last_became_active":"2023-10-21T17:40:42.548187+0000","last_became_peered":"2023-10-21T17:40:42.548187+0000","last_unstale":"2023-10-21T17:40:42.550816+0000","last_undegraded":"2023-10-21T17:40:42.550816+0000","last_fullsized":"2023-10-21T17:40:42.550816+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:56:36.331352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545727+0000","last_change":"2023-10-21T17:40:42.540179+0000","last_active":"2023-10-21T17:40:42.545727+0000","last_peered":"2023-10-21T17:40:42.545727+0000","last_clean":"2023-10-21T17:40:42.545727+0000","last_became_active":"2023-10-21T17:40:42.539531+0000","last_became_peered":"2023-10-21T17:40:42.539531+0000","last_unstale":"2023-10-21T17:40:42.545727+0000","last_undegraded":"2023-10-21T17:40:42.545727+0000","last_fullsized":"2023-10-21T17:40:42.545727+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:03:11.510221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545210+0000","last_change":"2023-10-21T17:40:42.539742+0000","last_active":"2023-10-21T17:40:42.545210+0000","last_peered":"2023-10-21T17:40:42.545210+0000","last_clean":"2023-10-21T17:40:42.545210+0000","last_became_active":"2023-10-21T17:40:42.539428+0000","last_became_peered":"2023-10-21T17:40:42.539428+0000","last_unstale":"2023-10-21T17:40:42.545210+0000","last_undegraded":"2023-10-21T17:40:42.545210+0000","last_fullsized":"2023-10-21T17:40:42.545210+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:12.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.542415+0000","last_change":"2023-10-21T17:40:42.539420+0000","last_active":"2023-10-21T17:40:42.542415+0000","last_peered":"2023-10-21T17:40:42.542415+0000","last_clean":"2023-10-21T17:40:42.542415+0000","last_became_active":"2023-10-21T17:40:42.539124+0000","last_became_peered":"2023-10-21T17:40:42.539124+0000","last_unstale":"2023-10-21T17:40:42.542415+0000","last_undegraded":"2023-10-21T17:40:42.542415+0000","last_fullsized":"2023-10-21T17:40:42.542415+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:22.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.534661+0000","last_change":"2023-10-21T17:40:42.531050+0000","last_active":"2023-10-21T17:40:42.534661+0000","last_peered":"2023-10-21T17:40:42.534661+0000","last_clean":"2023-10-21T17:40:42.534661+0000","last_became_active":"2023-10-21T17:40:42.530744+0000","last_became_peered":"2023-10-21T17:40:42.530744+0000","last_unstale":"2023-10-21T17:40:42.534661+0000","last_undegraded":"2023-10-21T17:40:42.534661+0000","last_fullsized":"2023-10-21T17:40:42.534661+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:55.928152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517155+0000","last_change":"2023-10-21T17:40:42.527652+0000","last_active":"2023-10-21T17:40:43.517155+0000","last_peered":"2023-10-21T17:40:43.517155+0000","last_clean":"2023-10-21T17:40:43.517155+0000","last_became_active":"2023-10-21T17:40:42.521681+0000","last_became_peered":"2023-10-21T17:40:42.521681+0000","last_unstale":"2023-10-21T17:40:43.517155+0000","last_undegraded":"2023-10-21T17:40:43.517155+0000","last_fullsized":"2023-10-21T17:40:43.517155+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:15.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.533924+0000","last_change":"2023-10-21T17:40:42.531040+0000","last_active":"2023-10-21T17:40:42.533924+0000","last_peered":"2023-10-21T17:40:42.533924+0000","last_clean":"2023-10-21T17:40:42.533924+0000","last_became_active":"2023-10-21T17:40:42.530736+0000","last_became_peered":"2023-10-21T17:40:42.530736+0000","last_unstale":"2023-10-21T17:40:42.533924+0000","last_undegraded":"2023-10-21T17:40:42.533924+0000","last_fullsized":"2023-10-21T17:40:42.533924+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:12.384348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515088+0000","last_change":"2023-10-21T17:40:42.544786+0000","last_active":"2023-10-21T17:40:43.515088+0000","last_peered":"2023-10-21T17:40:43.515088+0000","last_clean":"2023-10-21T17:40:43.515088+0000","last_became_active":"2023-10-21T17:40:42.544510+0000","last_became_peered":"2023-10-21T17:40:42.544510+0000","last_unstale":"2023-10-21T17:40:43.515088+0000","last_undegraded":"2023-10-21T17:40:43.515088+0000","last_fullsized":"2023-10-21T17:40:43.515088+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:29.499788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550834+0000","last_change":"2023-10-21T17:40:42.548700+0000","last_active":"2023-10-21T17:40:42.550834+0000","last_peered":"2023-10-21T17:40:42.550834+0000","last_clean":"2023-10-21T17:40:42.550834+0000","last_became_active":"2023-10-21T17:40:42.547072+0000","last_became_peered":"2023-10-21T17:40:42.547072+0000","last_unstale":"2023-10-21T17:40:42.550834+0000","last_undegraded":"2023-10-21T17:40:42.550834+0000","last_fullsized":"2023-10-21T17:40:42.550834+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:22.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"274'4","reported_seq":15,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553920+0000","last_change":"2023-10-21T17:40:41.515775+0000","last_active":"2023-10-21T17:40:42.553920+0000","last_peered":"2023-10-21T17:40:42.553920+0000","last_clean":"2023-10-21T17:40:42.553920+0000","last_became_active":"2023-10-21T17:40:41.515484+0000","last_became_peered":"2023-10-21T17:40:41.515484+0000","last_unstale":"2023-10-21T17:40:42.553920+0000","last_undegraded":"2023-10-21T17:40:42.553920+0000","last_fullsized":"2023-10-21T17:40:42.553920+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:14.265654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.552877+0000","last_change":"2023-10-21T17:40:42.549625+0000","last_active":"2023-10-21T17:40:42.552877+0000","last_peered":"2023-10-21T17:40:42.552877+0000","last_clean":"2023-10-21T17:40:42.552877+0000","last_became_active":"2023-10-21T17:40:42.548133+0000","last_became_peered":"2023-10-21T17:40:42.548133+0000","last_unstale":"2023-10-21T17:40:42.552877+0000","last_undegraded":"2023-10-21T17:40:42.552877+0000","last_fullsized":"2023-10-21T17:40:42.552877+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:02.987638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549385+0000","last_change":"2023-10-21T17:40:41.512308+0000","last_active":"2023-10-21T17:40:42.549385+0000","last_peered":"2023-10-21T17:40:42.549385+0000","last_clean":"2023-10-21T17:40:42.549385+0000","last_became_active":"2023-10-21T17:40:41.511439+0000","last_became_peered":"2023-10-21T17:40:41.511439+0000","last_unstale":"2023-10-21T17:40:42.549385+0000","last_undegraded":"2023-10-21T17:40:42.549385+0000","last_fullsized":"2023-10-21T17:40:42.549385+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.353094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546780+0000","last_change":"2023-10-21T17:40:42.542988+0000","last_active":"2023-10-21T17:40:42.546780+0000","last_peered":"2023-10-21T17:40:42.546780+0000","last_clean":"2023-10-21T17:40:42.546780+0000","last_became_active":"2023-10-21T17:40:42.542719+0000","last_became_peered":"2023-10-21T17:40:42.542719+0000","last_unstale":"2023-10-21T17:40:42.546780+0000","last_undegraded":"2023-10-21T17:40:42.546780+0000","last_fullsized":"2023-10-21T17:40:42.546780+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:14.512713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550502+0000","last_change":"2023-10-21T17:40:41.511235+0000","last_active":"2023-10-21T17:40:42.550502+0000","last_peered":"2023-10-21T17:40:42.550502+0000","last_clean":"2023-10-21T17:40:42.550502+0000","last_became_active":"2023-10-21T17:40:41.505475+0000","last_became_peered":"2023-10-21T17:40:41.505475+0000","last_unstale":"2023-10-21T17:40:42.550502+0000","last_undegraded":"2023-10-21T17:40:42.550502+0000","last_fullsized":"2023-10-21T17:40:42.550502+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:48.812842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540322+0000","last_change":"2023-10-21T17:40:42.536242+0000","last_active":"2023-10-21T17:40:42.540322+0000","last_peered":"2023-10-21T17:40:42.540322+0000","last_clean":"2023-10-21T17:40:42.540322+0000","last_became_active":"2023-10-21T17:40:42.535463+0000","last_became_peered":"2023-10-21T17:40:42.535463+0000","last_unstale":"2023-10-21T17:40:42.540322+0000","last_undegraded":"2023-10-21T17:40:42.540322+0000","last_fullsized":"2023-10-21T17:40:42.540322+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:17.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550609+0000","last_change":"2023-10-21T17:40:41.507550+0000","last_active":"2023-10-21T17:40:42.550609+0000","last_peered":"2023-10-21T17:40:42.550609+0000","last_clean":"2023-10-21T17:40:42.550609+0000","last_became_active":"2023-10-21T17:40:41.505155+0000","last_became_peered":"2023-10-21T17:40:41.505155+0000","last_unstale":"2023-10-21T17:40:42.550609+0000","last_undegraded":"2023-10-21T17:40:42.550609+0000","last_fullsized":"2023-10-21T17:40:42.550609+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:25.764299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516025+0000","last_change":"2023-10-21T17:40:42.551931+0000","last_active":"2023-10-21T17:40:43.516025+0000","last_peered":"2023-10-21T17:40:43.516025+0000","last_clean":"2023-10-21T17:40:43.516025+0000","last_became_active":"2023-10-21T17:40:42.550131+0000","last_became_peered":"2023-10-21T17:40:42.550131+0000","last_unstale":"2023-10-21T17:40:43.516025+0000","last_undegraded":"2023-10-21T17:40:43.516025+0000","last_fullsized":"2023-10-21T17:40:43.516025+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.224336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553070+0000","last_change":"2023-10-21T17:40:41.506400+0000","last_active":"2023-10-21T17:40:42.553070+0000","last_peered":"2023-10-21T17:40:42.553070+0000","last_clean":"2023-10-21T17:40:42.553070+0000","last_became_active":"2023-10-21T17:40:41.506020+0000","last_became_peered":"2023-10-21T17:40:41.506020+0000","last_unstale":"2023-10-21T17:40:42.553070+0000","last_undegraded":"2023-10-21T17:40:42.553070+0000","last_fullsized":"2023-10-21T17:40:42.553070+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:55:16.988226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547548+0000","last_change":"2023-10-21T17:40:42.546477+0000","last_active":"2023-10-21T17:40:42.547548+0000","last_peered":"2023-10-21T17:40:42.547548+0000","last_clean":"2023-10-21T17:40:42.547548+0000","last_became_active":"2023-10-21T17:40:42.546203+0000","last_became_peered":"2023-10-21T17:40:42.546203+0000","last_unstale":"2023-10-21T17:40:42.547548+0000","last_undegraded":"2023-10-21T17:40:42.547548+0000","last_fullsized":"2023-10-21T17:40:42.547548+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:30:23.660382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547737+0000","last_change":"2023-10-21T17:40:41.512089+0000","last_active":"2023-10-21T17:40:42.547737+0000","last_peered":"2023-10-21T17:40:42.547737+0000","last_clean":"2023-10-21T17:40:42.547737+0000","last_became_active":"2023-10-21T17:40:41.511790+0000","last_became_peered":"2023-10-21T17:40:41.511790+0000","last_unstale":"2023-10-21T17:40:42.547737+0000","last_undegraded":"2023-10-21T17:40:42.547737+0000","last_fullsized":"2023-10-21T17:40:42.547737+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:38.103936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.556771+0000","last_change":"2023-10-21T17:40:42.556268+0000","last_active":"2023-10-21T17:40:42.556771+0000","last_peered":"2023-10-21T17:40:42.556771+0000","last_clean":"2023-10-21T17:40:42.556771+0000","last_became_active":"2023-10-21T17:40:42.555873+0000","last_became_peered":"2023-10-21T17:40:42.555873+0000","last_unstale":"2023-10-21T17:40:42.556771+0000","last_undegraded":"2023-10-21T17:40:42.556771+0000","last_fullsized":"2023-10-21T17:40:42.556771+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:53.272545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515723+0000","last_change":"2023-10-21T17:40:41.499396+0000","last_active":"2023-10-21T17:40:43.515723+0000","last_peered":"2023-10-21T17:40:43.515723+0000","last_clean":"2023-10-21T17:40:43.515723+0000","last_became_active":"2023-10-21T17:40:41.498697+0000","last_became_peered":"2023-10-21T17:40:41.498697+0000","last_unstale":"2023-10-21T17:40:43.515723+0000","last_undegraded":"2023-10-21T17:40:43.515723+0000","last_fullsized":"2023-10-21T17:40:43.515723+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:18.446164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519711+0000","last_change":"2023-10-21T17:40:42.530842+0000","last_active":"2023-10-21T17:40:43.519711+0000","last_peered":"2023-10-21T17:40:43.519711+0000","last_clean":"2023-10-21T17:40:43.519711+0000","last_became_active":"2023-10-21T17:40:42.530436+0000","last_became_peered":"2023-10-21T17:40:42.530436+0000","last_unstale":"2023-10-21T17:40:43.519711+0000","last_undegraded":"2023-10-21T17:40:43.519711+0000","last_fullsized":"2023-10-21T17:40:43.519711+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:22:54.419654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515623+0000","last_change":"2023-10-21T17:40:41.501427+0000","last_active":"2023-10-21T17:40:43.515623+0000","last_peered":"2023-10-21T17:40:43.515623+0000","last_clean":"2023-10-21T17:40:43.515623+0000","last_became_active":"2023-10-21T17:40:41.498937+0000","last_became_peered":"2023-10-21T17:40:41.498937+0000","last_unstale":"2023-10-21T17:40:43.515623+0000","last_undegraded":"2023-10-21T17:40:43.515623+0000","last_fullsized":"2023-10-21T17:40:43.515623+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:01.646372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515282+0000","last_change":"2023-10-21T17:40:42.552347+0000","last_active":"2023-10-21T17:40:43.515282+0000","last_peered":"2023-10-21T17:40:43.515282+0000","last_clean":"2023-10-21T17:40:43.515282+0000","last_became_active":"2023-10-21T17:40:42.551606+0000","last_became_peered":"2023-10-21T17:40:42.551606+0000","last_unstale":"2023-10-21T17:40:43.515282+0000","last_undegraded":"2023-10-21T17:40:43.515282+0000","last_fullsized":"2023-10-21T17:40:43.515282+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:08.081639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519969+0000","last_change":"2023-10-21T17:40:41.514484+0000","last_active":"2023-10-21T17:40:43.519969+0000","last_peered":"2023-10-21T17:40:43.519969+0000","last_clean":"2023-10-21T17:40:43.519969+0000","last_became_active":"2023-10-21T17:40:41.513572+0000","last_became_peered":"2023-10-21T17:40:41.513572+0000","last_unstale":"2023-10-21T17:40:43.519969+0000","last_undegraded":"2023-10-21T17:40:43.519969+0000","last_fullsized":"2023-10-21T17:40:43.519969+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:02.553599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545905+0000","last_change":"2023-10-21T17:40:42.541292+0000","last_active":"2023-10-21T17:40:42.545905+0000","last_peered":"2023-10-21T17:40:42.545905+0000","last_clean":"2023-10-21T17:40:42.545905+0000","last_became_active":"2023-10-21T17:40:42.540984+0000","last_became_peered":"2023-10-21T17:40:42.540984+0000","last_unstale":"2023-10-21T17:40:42.545905+0000","last_undegraded":"2023-10-21T17:40:42.545905+0000","last_fullsized":"2023-10-21T17:40:42.545905+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:16.647471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517382+0000","last_change":"2023-10-21T17:40:41.502046+0000","last_active":"2023-10-21T17:40:43.517382+0000","last_peered":"2023-10-21T17:40:43.517382+0000","last_clean":"2023-10-21T17:40:43.517382+0000","last_became_active":"2023-10-21T17:40:41.498316+0000","last_became_peered":"2023-10-21T17:40:41.498316+0000","last_unstale":"2023-10-21T17:40:43.517382+0000","last_undegraded":"2023-10-21T17:40:43.517382+0000","last_fullsized":"2023-10-21T17:40:43.517382+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:23:42.859153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515614+0000","last_change":"2023-10-21T17:40:42.523579+0000","last_active":"2023-10-21T17:40:43.515614+0000","last_peered":"2023-10-21T17:40:43.515614+0000","last_clean":"2023-10-21T17:40:43.515614+0000","last_became_active":"2023-10-21T17:40:42.523014+0000","last_became_peered":"2023-10-21T17:40:42.523014+0000","last_unstale":"2023-10-21T17:40:43.515614+0000","last_undegraded":"2023-10-21T17:40:43.515614+0000","last_fullsized":"2023-10-21T17:40:43.515614+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:13.186717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"274'3","reported_seq":15,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519904+0000","last_change":"2023-10-21T17:40:41.501154+0000","last_active":"2023-10-21T17:40:43.519904+0000","last_peered":"2023-10-21T17:40:43.519904+0000","last_clean":"2023-10-21T17:40:43.519904+0000","last_became_active":"2023-10-21T17:40:41.500856+0000","last_became_peered":"2023-10-21T17:40:41.500856+0000","last_unstale":"2023-10-21T17:40:43.519904+0000","last_undegraded":"2023-10-21T17:40:43.519904+0000","last_fullsized":"2023-10-21T17:40:43.519904+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:28.483250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.554636+0000","last_change":"2023-10-21T17:40:42.554029+0000","last_active":"2023-10-21T17:40:42.554636+0000","last_peered":"2023-10-21T17:40:42.554636+0000","last_clean":"2023-10-21T17:40:42.554636+0000","last_became_active":"2023-10-21T17:40:42.552973+0000","last_became_peered":"2023-10-21T17:40:42.552973+0000","last_unstale":"2023-10-21T17:40:42.554636+0000","last_undegraded":"2023-10-21T17:40:42.554636+0000","last_fullsized":"2023-10-21T17:40:42.554636+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:42:53.351945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"274'6","reported_seq":17,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.557216+0000","last_change":"2023-10-21T17:40:41.515864+0000","last_active":"2023-10-21T17:40:42.557216+0000","last_peered":"2023-10-21T17:40:42.557216+0000","last_clean":"2023-10-21T17:40:42.557216+0000","last_became_active":"2023-10-21T17:40:41.515566+0000","last_became_peered":"2023-10-21T17:40:41.515566+0000","last_unstale":"2023-10-21T17:40:42.557216+0000","last_undegraded":"2023-10-21T17:40:42.557216+0000","last_fullsized":"2023-10-21T17:40:42.557216+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:42.134246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546756+0000","last_change":"2023-10-21T17:40:42.542021+0000","last_active":"2023-10-21T17:40:42.546756+0000","last_peered":"2023-10-21T17:40:42.546756+0000","last_clean":"2023-10-21T17:40:42.546756+0000","last_became_active":"2023-10-21T17:40:42.538806+0000","last_became_peered":"2023-10-21T17:40:42.538806+0000","last_unstale":"2023-10-21T17:40:42.546756+0000","last_undegraded":"2023-10-21T17:40:42.546756+0000","last_fullsized":"2023-10-21T17:40:42.546756+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:34.933250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516773+0000","last_change":"2023-10-21T17:40:41.497629+0000","last_active":"2023-10-21T17:40:43.516773+0000","last_peered":"2023-10-21T17:40:43.516773+0000","last_clean":"2023-10-21T17:40:43.516773+0000","last_became_active":"2023-10-21T17:40:41.497228+0000","last_became_peered":"2023-10-21T17:40:41.497228+0000","last_unstale":"2023-10-21T17:40:43.516773+0000","last_undegraded":"2023-10-21T17:40:43.516773+0000","last_fullsized":"2023-10-21T17:40:43.516773+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:43.179333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516284+0000","last_change":"2023-10-21T17:40:42.550129+0000","last_active":"2023-10-21T17:40:43.516284+0000","last_peered":"2023-10-21T17:40:43.516284+0000","last_clean":"2023-10-21T17:40:43.516284+0000","last_became_active":"2023-10-21T17:40:42.544475+0000","last_became_peered":"2023-10-21T17:40:42.544475+0000","last_unstale":"2023-10-21T17:40:43.516284+0000","last_undegraded":"2023-10-21T17:40:43.516284+0000","last_fullsized":"2023-10-21T17:40:43.516284+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:51:14.501138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.539765+0000","last_change":"2023-10-21T17:40:41.512572+0000","last_active":"2023-10-21T17:40:42.539765+0000","last_peered":"2023-10-21T17:40:42.539765+0000","last_clean":"2023-10-21T17:40:42.539765+0000","last_became_active":"2023-10-21T17:40:41.512047+0000","last_became_peered":"2023-10-21T17:40:41.512047+0000","last_unstale":"2023-10-21T17:40:42.539765+0000","last_undegraded":"2023-10-21T17:40:42.539765+0000","last_fullsized":"2023-10-21T17:40:42.539765+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:40.718850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519503+0000","last_change":"2023-10-21T17:40:42.530817+0000","last_active":"2023-10-21T17:40:43.519503+0000","last_peered":"2023-10-21T17:40:43.519503+0000","last_clean":"2023-10-21T17:40:43.519503+0000","last_became_active":"2023-10-21T17:40:42.530517+0000","last_became_peered":"2023-10-21T17:40:42.530517+0000","last_unstale":"2023-10-21T17:40:43.519503+0000","last_undegraded":"2023-10-21T17:40:43.519503+0000","last_fullsized":"2023-10-21T17:40:43.519503+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:05:55.535354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.539062+0000","last_change":"2023-10-21T17:40:41.516565+0000","last_active":"2023-10-21T17:40:42.539062+0000","last_peered":"2023-10-21T17:40:42.539062+0000","last_clean":"2023-10-21T17:40:42.539062+0000","last_became_active":"2023-10-21T17:40:41.515183+0000","last_became_peered":"2023-10-21T17:40:41.515183+0000","last_unstale":"2023-10-21T17:40:42.539062+0000","last_undegraded":"2023-10-21T17:40:42.539062+0000","last_fullsized":"2023-10-21T17:40:42.539062+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:16.107024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.520055+0000","last_change":"2023-10-21T17:40:42.546988+0000","last_active":"2023-10-21T17:40:43.520055+0000","last_peered":"2023-10-21T17:40:43.520055+0000","last_clean":"2023-10-21T17:40:43.520055+0000","last_became_active":"2023-10-21T17:40:42.546706+0000","last_became_peered":"2023-10-21T17:40:42.546706+0000","last_unstale":"2023-10-21T17:40:43.520055+0000","last_undegraded":"2023-10-21T17:40:43.520055+0000","last_fullsized":"2023-10-21T17:40:43.520055+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:39.824618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"274'8","reported_seq":19,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.555839+0000","last_change":"2023-10-21T17:40:41.512974+0000","last_active":"2023-10-21T17:40:42.555839+0000","last_peered":"2023-10-21T17:40:42.555839+0000","last_clean":"2023-10-21T17:40:42.555839+0000","last_became_active":"2023-10-21T17:40:41.512707+0000","last_became_peered":"2023-10-21T17:40:41.512707+0000","last_unstale":"2023-10-21T17:40:42.555839+0000","last_undegraded":"2023-10-21T17:40:42.555839+0000","last_fullsized":"2023-10-21T17:40:42.555839+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:43.179333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541251+0000","last_change":"2023-10-21T17:40:42.537465+0000","last_active":"2023-10-21T17:40:42.541251+0000","last_peered":"2023-10-21T17:40:42.541251+0000","last_clean":"2023-10-21T17:40:42.541251+0000","last_became_active":"2023-10-21T17:40:42.537127+0000","last_became_peered":"2023-10-21T17:40:42.537127+0000","last_unstale":"2023-10-21T17:40:42.541251+0000","last_undegraded":"2023-10-21T17:40:42.541251+0000","last_fullsized":"2023-10-21T17:40:42.541251+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:20.628311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547861+0000","last_change":"2023-10-21T17:40:41.503085+0000","last_active":"2023-10-21T17:40:42.547861+0000","last_peered":"2023-10-21T17:40:42.547861+0000","last_clean":"2023-10-21T17:40:42.547861+0000","last_became_active":"2023-10-21T17:40:41.502733+0000","last_became_peered":"2023-10-21T17:40:41.502733+0000","last_unstale":"2023-10-21T17:40:42.547861+0000","last_undegraded":"2023-10-21T17:40:42.547861+0000","last_fullsized":"2023-10-21T17:40:42.547861+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:41.988063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545333+0000","last_change":"2023-10-21T17:40:42.542268+0000","last_active":"2023-10-21T17:40:42.545333+0000","last_peered":"2023-10-21T17:40:42.545333+0000","last_clean":"2023-10-21T17:40:42.545333+0000","last_became_active":"2023-10-21T17:40:42.541999+0000","last_became_peered":"2023-10-21T17:40:42.541999+0000","last_unstale":"2023-10-21T17:40:42.545333+0000","last_undegraded":"2023-10-21T17:40:42.545333+0000","last_fullsized":"2023-10-21T17:40:42.545333+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:42:53.351945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.555476+0000","last_change":"2023-10-21T17:40:41.498860+0000","last_active":"2023-10-21T17:40:42.555476+0000","last_peered":"2023-10-21T17:40:42.555476+0000","last_clean":"2023-10-21T17:40:42.555476+0000","last_became_active":"2023-10-21T17:40:41.497936+0000","last_became_peered":"2023-10-21T17:40:41.497936+0000","last_unstale":"2023-10-21T17:40:42.555476+0000","last_undegraded":"2023-10-21T17:40:42.555476+0000","last_fullsized":"2023-10-21T17:40:42.555476+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:52.256006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541065+0000","last_change":"2023-10-21T17:40:42.536241+0000","last_active":"2023-10-21T17:40:42.541065+0000","last_peered":"2023-10-21T17:40:42.541065+0000","last_clean":"2023-10-21T17:40:42.541065+0000","last_became_active":"2023-10-21T17:40:42.535733+0000","last_became_peered":"2023-10-21T17:40:42.535733+0000","last_unstale":"2023-10-21T17:40:42.541065+0000","last_undegraded":"2023-10-21T17:40:42.541065+0000","last_fullsized":"2023-10-21T17:40:42.541065+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:12.384348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549255+0000","last_change":"2023-10-21T17:40:41.512355+0000","last_active":"2023-10-21T17:40:42.549255+0000","last_peered":"2023-10-21T17:40:42.549255+0000","last_clean":"2023-10-21T17:40:42.549255+0000","last_became_active":"2023-10-21T17:40:41.512016+0000","last_became_peered":"2023-10-21T17:40:41.512016+0000","last_unstale":"2023-10-21T17:40:42.549255+0000","last_undegraded":"2023-10-21T17:40:42.549255+0000","last_fullsized":"2023-10-21T17:40:42.549255+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:17.483951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515910+0000","last_change":"2023-10-21T17:40:42.548197+0000","last_active":"2023-10-21T17:40:43.515910+0000","last_peered":"2023-10-21T17:40:43.515910+0000","last_clean":"2023-10-21T17:40:43.515910+0000","last_became_active":"2023-10-21T17:40:42.545636+0000","last_became_peered":"2023-10-21T17:40:42.545636+0000","last_unstale":"2023-10-21T17:40:43.515910+0000","last_undegraded":"2023-10-21T17:40:43.515910+0000","last_fullsized":"2023-10-21T17:40:43.515910+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:30:36.869057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516803+0000","last_change":"2023-10-21T17:40:41.501403+0000","last_active":"2023-10-21T17:40:43.516803+0000","last_peered":"2023-10-21T17:40:43.516803+0000","last_clean":"2023-10-21T17:40:43.516803+0000","last_became_active":"2023-10-21T17:40:41.501026+0000","last_became_peered":"2023-10-21T17:40:41.501026+0000","last_unstale":"2023-10-21T17:40:43.516803+0000","last_undegraded":"2023-10-21T17:40:43.516803+0000","last_fullsized":"2023-10-21T17:40:43.516803+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:42.134246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549970+0000","last_change":"2023-10-21T17:40:42.547341+0000","last_active":"2023-10-21T17:40:42.549970+0000","last_peered":"2023-10-21T17:40:42.549970+0000","last_clean":"2023-10-21T17:40:42.549970+0000","last_became_active":"2023-10-21T17:40:42.547021+0000","last_became_peered":"2023-10-21T17:40:42.547021+0000","last_unstale":"2023-10-21T17:40:42.549970+0000","last_undegraded":"2023-10-21T17:40:42.549970+0000","last_fullsized":"2023-10-21T17:40:42.549970+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:39:18.909871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516402+0000","last_change":"2023-10-21T17:40:41.503144+0000","last_active":"2023-10-21T17:40:43.516402+0000","last_peered":"2023-10-21T17:40:43.516402+0000","last_clean":"2023-10-21T17:40:43.516402+0000","last_became_active":"2023-10-21T17:40:41.502566+0000","last_became_peered":"2023-10-21T17:40:41.502566+0000","last_unstale":"2023-10-21T17:40:43.516402+0000","last_undegraded":"2023-10-21T17:40:43.516402+0000","last_fullsized":"2023-10-21T17:40:43.516402+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:57.369330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545119+0000","last_change":"2023-10-21T17:40:42.541871+0000","last_active":"2023-10-21T17:40:42.545119+0000","last_peered":"2023-10-21T17:40:42.545119+0000","last_clean":"2023-10-21T17:40:42.545119+0000","last_became_active":"2023-10-21T17:40:42.541593+0000","last_became_peered":"2023-10-21T17:40:42.541593+0000","last_unstale":"2023-10-21T17:40:42.545119+0000","last_undegraded":"2023-10-21T17:40:42.545119+0000","last_fullsized":"2023-10-21T17:40:42.545119+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:39.120475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515739+0000","last_change":"2023-10-21T17:40:41.500455+0000","last_active":"2023-10-21T17:40:43.515739+0000","last_peered":"2023-10-21T17:40:43.515739+0000","last_clean":"2023-10-21T17:40:43.515739+0000","last_became_active":"2023-10-21T17:40:41.498478+0000","last_became_peered":"2023-10-21T17:40:41.498478+0000","last_unstale":"2023-10-21T17:40:43.515739+0000","last_undegraded":"2023-10-21T17:40:43.515739+0000","last_fullsized":"2023-10-21T17:40:43.515739+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:17.029701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519796+0000","last_change":"2023-10-21T17:40:42.521477+0000","last_active":"2023-10-21T17:40:43.519796+0000","last_peered":"2023-10-21T17:40:43.519796+0000","last_clean":"2023-10-21T17:40:43.519796+0000","last_became_active":"2023-10-21T17:40:42.521145+0000","last_became_peered":"2023-10-21T17:40:42.521145+0000","last_unstale":"2023-10-21T17:40:43.519796+0000","last_undegraded":"2023-10-21T17:40:43.519796+0000","last_fullsized":"2023-10-21T17:40:43.519796+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:45:17.066518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.536075+0000","last_change":"2023-10-21T17:40:41.505227+0000","last_active":"2023-10-21T17:40:42.536075+0000","last_peered":"2023-10-21T17:40:42.536075+0000","last_clean":"2023-10-21T17:40:42.536075+0000","last_became_active":"2023-10-21T17:40:41.504712+0000","last_became_peered":"2023-10-21T17:40:41.504712+0000","last_unstale":"2023-10-21T17:40:42.536075+0000","last_undegraded":"2023-10-21T17:40:42.536075+0000","last_fullsized":"2023-10-21T17:40:42.536075+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:47.883648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541304+0000","last_change":"2023-10-21T17:40:42.537178+0000","last_active":"2023-10-21T17:40:42.541304+0000","last_peered":"2023-10-21T17:40:42.541304+0000","last_clean":"2023-10-21T17:40:42.541304+0000","last_became_active":"2023-10-21T17:40:42.536879+0000","last_became_peered":"2023-10-21T17:40:42.536879+0000","last_unstale":"2023-10-21T17:40:42.541304+0000","last_undegraded":"2023-10-21T17:40:42.541304+0000","last_fullsized":"2023-10-21T17:40:42.541304+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:56:36.331352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516528+0000","last_change":"2023-10-21T17:40:41.496704+0000","last_active":"2023-10-21T17:40:43.516528+0000","last_peered":"2023-10-21T17:40:43.516528+0000","last_clean":"2023-10-21T17:40:43.516528+0000","last_became_active":"2023-10-21T17:40:41.496063+0000","last_became_peered":"2023-10-21T17:40:41.496063+0000","last_unstale":"2023-10-21T17:40:43.516528+0000","last_undegraded":"2023-10-21T17:40:43.516528+0000","last_fullsized":"2023-10-21T17:40:43.516528+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:45.805375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515499+0000","last_change":"2023-10-21T17:40:42.545415+0000","last_active":"2023-10-21T17:40:43.515499+0000","last_peered":"2023-10-21T17:40:43.515499+0000","last_clean":"2023-10-21T17:40:43.515499+0000","last_became_active":"2023-10-21T17:40:42.545122+0000","last_became_peered":"2023-10-21T17:40:42.545122+0000","last_unstale":"2023-10-21T17:40:43.515499+0000","last_undegraded":"2023-10-21T17:40:43.515499+0000","last_fullsized":"2023-10-21T17:40:43.515499+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:56:10.172790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517744+0000","last_change":"2023-10-21T17:40:41.513429+0000","last_active":"2023-10-21T17:40:43.517744+0000","last_peered":"2023-10-21T17:40:43.517744+0000","last_clean":"2023-10-21T17:40:43.517744+0000","last_became_active":"2023-10-21T17:40:41.513158+0000","last_became_peered":"2023-10-21T17:40:41.513158+0000","last_unstale":"2023-10-21T17:40:43.517744+0000","last_undegraded":"2023-10-21T17:40:43.517744+0000","last_fullsized":"2023-10-21T17:40:43.517744+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:16.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519007+0000","last_change":"2023-10-21T17:40:42.523684+0000","last_active":"2023-10-21T17:40:43.519007+0000","last_peered":"2023-10-21T17:40:43.519007+0000","last_clean":"2023-10-21T17:40:43.519007+0000","last_became_active":"2023-10-21T17:40:42.523337+0000","last_became_peered":"2023-10-21T17:40:42.523337+0000","last_unstale":"2023-10-21T17:40:43.519007+0000","last_undegraded":"2023-10-21T17:40:43.519007+0000","last_fullsized":"2023-10-21T17:40:43.519007+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.224336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540788+0000","last_change":"2023-10-21T17:40:41.514879+0000","last_active":"2023-10-21T17:40:42.540788+0000","last_peered":"2023-10-21T17:40:42.540788+0000","last_clean":"2023-10-21T17:40:42.540788+0000","last_became_active":"2023-10-21T17:40:41.512216+0000","last_became_peered":"2023-10-21T17:40:41.512216+0000","last_unstale":"2023-10-21T17:40:42.540788+0000","last_undegraded":"2023-10-21T17:40:42.540788+0000","last_fullsized":"2023-10-21T17:40:42.540788+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:20.372191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515431+0000","last_change":"2023-10-21T17:40:42.544408+0000","last_active":"2023-10-21T17:40:43.515431+0000","last_peered":"2023-10-21T17:40:43.515431+0000","last_clean":"2023-10-21T17:40:43.515431+0000","last_became_active":"2023-10-21T17:40:42.544127+0000","last_became_peered":"2023-10-21T17:40:42.544127+0000","last_unstale":"2023-10-21T17:40:43.515431+0000","last_undegraded":"2023-10-21T17:40:43.515431+0000","last_fullsized":"2023-10-21T17:40:43.515431+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:45.734436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.514154+0000","last_change":"2023-10-21T17:40:41.511513+0000","last_active":"2023-10-21T17:40:42.514154+0000","last_peered":"2023-10-21T17:40:42.514154+0000","last_clean":"2023-10-21T17:40:42.514154+0000","last_became_active":"2023-10-21T17:40:41.510807+0000","last_became_peered":"2023-10-21T17:40:41.510807+0000","last_unstale":"2023-10-21T17:40:42.514154+0000","last_undegraded":"2023-10-21T17:40:42.514154+0000","last_fullsized":"2023-10-21T17:40:42.514154+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:31.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540803+0000","last_change":"2023-10-21T17:40:42.539021+0000","last_active":"2023-10-21T17:40:42.540803+0000","last_peered":"2023-10-21T17:40:42.540803+0000","last_clean":"2023-10-21T17:40:42.540803+0000","last_became_active":"2023-10-21T17:40:42.535926+0000","last_became_peered":"2023-10-21T17:40:42.535926+0000","last_unstale":"2023-10-21T17:40:42.540803+0000","last_undegraded":"2023-10-21T17:40:42.540803+0000","last_fullsized":"2023-10-21T17:40:42.540803+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:58.385869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.542011+0000","last_change":"2023-10-21T17:40:41.516234+0000","last_active":"2023-10-21T17:40:42.542011+0000","last_peered":"2023-10-21T17:40:42.542011+0000","last_clean":"2023-10-21T17:40:42.542011+0000","last_became_active":"2023-10-21T17:40:41.514086+0000","last_became_peered":"2023-10-21T17:40:41.514086+0000","last_unstale":"2023-10-21T17:40:42.542011+0000","last_undegraded":"2023-10-21T17:40:42.542011+0000","last_fullsized":"2023-10-21T17:40:42.542011+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:06:09.422391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.536966+0000","last_change":"2023-10-21T17:40:42.531150+0000","last_active":"2023-10-21T17:40:42.536966+0000","last_peered":"2023-10-21T17:40:42.536966+0000","last_clean":"2023-10-21T17:40:42.536966+0000","last_became_active":"2023-10-21T17:40:42.530845+0000","last_became_peered":"2023-10-21T17:40:42.530845+0000","last_unstale":"2023-10-21T17:40:42.536966+0000","last_undegraded":"2023-10-21T17:40:42.536966+0000","last_fullsized":"2023-10-21T17:40:42.536966+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:56.883561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516645+0000","last_change":"2023-10-21T17:40:41.502880+0000","last_active":"2023-10-21T17:40:43.516645+0000","last_peered":"2023-10-21T17:40:43.516645+0000","last_clean":"2023-10-21T17:40:43.516645+0000","last_became_active":"2023-10-21T17:40:41.500906+0000","last_became_peered":"2023-10-21T17:40:41.500906+0000","last_unstale":"2023-10-21T17:40:43.516645+0000","last_undegraded":"2023-10-21T17:40:43.516645+0000","last_fullsized":"2023-10-21T17:40:43.516645+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:00.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516213+0000","last_change":"2023-10-21T17:40:42.548094+0000","last_active":"2023-10-21T17:40:43.516213+0000","last_peered":"2023-10-21T17:40:43.516213+0000","last_clean":"2023-10-21T17:40:43.516213+0000","last_became_active":"2023-10-21T17:40:42.534066+0000","last_became_peered":"2023-10-21T17:40:42.534066+0000","last_unstale":"2023-10-21T17:40:43.516213+0000","last_undegraded":"2023-10-21T17:40:43.516213+0000","last_fullsized":"2023-10-21T17:40:43.516213+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:05:55.535354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515856+0000","last_change":"2023-10-21T17:40:41.501333+0000","last_active":"2023-10-21T17:40:43.515856+0000","last_peered":"2023-10-21T17:40:43.515856+0000","last_clean":"2023-10-21T17:40:43.515856+0000","last_became_active":"2023-10-21T17:40:41.498706+0000","last_became_peered":"2023-10-21T17:40:41.498706+0000","last_unstale":"2023-10-21T17:40:43.515856+0000","last_undegraded":"2023-10-21T17:40:43.515856+0000","last_fullsized":"2023-10-21T17:40:43.515856+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:17:58.907796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.534916+0000","last_change":"2023-10-21T17:40:42.531782+0000","last_active":"2023-10-21T17:40:42.534916+0000","last_peered":"2023-10-21T17:40:42.534916+0000","last_clean":"2023-10-21T17:40:42.534916+0000","last_became_active":"2023-10-21T17:40:42.531516+0000","last_became_peered":"2023-10-21T17:40:42.531516+0000","last_unstale":"2023-10-21T17:40:42.534916+0000","last_undegraded":"2023-10-21T17:40:42.534916+0000","last_fullsized":"2023-10-21T17:40:42.534916+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:07.166062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519237+0000","last_change":"2023-10-21T17:40:41.508269+0000","last_active":"2023-10-21T17:40:43.519237+0000","last_peered":"2023-10-21T17:40:43.519237+0000","last_clean":"2023-10-21T17:40:43.519237+0000","last_became_active":"2023-10-21T17:40:41.505327+0000","last_became_peered":"2023-10-21T17:40:41.505327+0000","last_unstale":"2023-10-21T17:40:43.519237+0000","last_undegraded":"2023-10-21T17:40:43.519237+0000","last_fullsized":"2023-10-21T17:40:43.519237+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:41:30.389201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516451+0000","last_change":"2023-10-21T17:40:42.552662+0000","last_active":"2023-10-21T17:40:43.516451+0000","last_peered":"2023-10-21T17:40:43.516451+0000","last_clean":"2023-10-21T17:40:43.516451+0000","last_became_active":"2023-10-21T17:40:42.552112+0000","last_became_peered":"2023-10-21T17:40:42.552112+0000","last_unstale":"2023-10-21T17:40:43.516451+0000","last_undegraded":"2023-10-21T17:40:43.516451+0000","last_fullsized":"2023-10-21T17:40:43.516451+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:22:54.419654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516084+0000","last_change":"2023-10-21T17:40:41.496477+0000","last_active":"2023-10-21T17:40:43.516084+0000","last_peered":"2023-10-21T17:40:43.516084+0000","last_clean":"2023-10-21T17:40:43.516084+0000","last_became_active":"2023-10-21T17:40:41.495477+0000","last_became_peered":"2023-10-21T17:40:41.495477+0000","last_unstale":"2023-10-21T17:40:43.516084+0000","last_undegraded":"2023-10-21T17:40:43.516084+0000","last_fullsized":"2023-10-21T17:40:43.516084+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:38.573814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516677+0000","last_change":"2023-10-21T17:40:42.538268+0000","last_active":"2023-10-21T17:40:43.516677+0000","last_peered":"2023-10-21T17:40:43.516677+0000","last_clean":"2023-10-21T17:40:43.516677+0000","last_became_active":"2023-10-21T17:40:42.537965+0000","last_became_peered":"2023-10-21T17:40:42.537965+0000","last_unstale":"2023-10-21T17:40:43.516677+0000","last_undegraded":"2023-10-21T17:40:43.516677+0000","last_fullsized":"2023-10-21T17:40:43.516677+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:32.249928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"260'4909","reported_seq":5965,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-21T17:40:41.506872+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:40:41.506872+0000","last_peered":"2023-10-21T17:40:41.506872+0000","last_clean":"2023-10-21T17:40:41.506872+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:40:41.506872+0000","last_undegraded":"2023-10-21T17:40:41.506872+0000","last_fullsized":"2023-10-21T17:40:41.506872+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4909,"ondisk_log_size":4909,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75420,"kb_used_data":3980,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296420,"statfs":{"total":96636764160,"available":96559534080,"internally_reserved":0,"allocated":4075520,"data_stored":2404647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7190000000000001,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.014},{"interface":"front","average":{"1min":1.044,"5min":1.119,"15min":1.161},"min":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1930000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1619999999999999,"15min":1.204},"min":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.056},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.232,"15min":1.2909999999999999},"min":{"1min":0.65000000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.351}]},{"osd":2,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.361,"15min":1.3400000000000001},"min":{"1min":0.72599999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0430000000000001,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.306,"15min":1.3149999999999999},"min":{"1min":0.76600000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.736,"5min":3.556,"15min":3.556},"last":1.6240000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.274,"15min":1.3089999999999999},"min":{"1min":0.78600000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.782,"5min":2.472,"15min":2.5110000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.476,"15min":1.464},"min":{"1min":0.69999999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2280000000000002,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.248}]},{"osd":4,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.391,"15min":1.363},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.669,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.569},{"interface":"front","average":{"1min":1.472,"5min":1.4950000000000001,"15min":1.4930000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3420000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.2809999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.486,"15min":1.4199999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.052,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.486},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0819999999999999,"5min":2.613,"15min":2.613},"last":1.403}]},{"osd":6,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.599,"15min":1.5549999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.2309999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5309999999999999,"15min":1.492},"min":{"1min":0.71999999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2850000000000001,"5min":2.879,"15min":2.879},"last":1.6619999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576023,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65092,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306748,"statfs":{"total":96636764160,"available":96570109952,"internally_reserved":0,"allocated":3461120,"data_stored":1812015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.181},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.8069999999999999,"5min":1.944,"15min":1.944},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.147,"15min":1.2},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.605,"5min":1.6619999999999999,"15min":1.762},"last":1.4490000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1080000000000001,"15min":1.1699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.595,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":1.234},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.226,"15min":1.292},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6779999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.25,"15min":1.3009999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8300000000000001},"last":1.585},{"interface":"front","average":{"1min":1.256,"5min":1.26,"15min":1.294},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.929},"last":0.98599999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3220000000000001,"15min":1.3959999999999999},"min":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.9770000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.391,"15min":1.4390000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.754}]},{"osd":4,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3029999999999999,"15min":1.3380000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.9910000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.423,"15min":1.4330000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.911,"5min":1.911,"15min":2.2149999999999999},"last":1.8460000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.5640000000000001,"15min":1.5669999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.339},"last":1.9450000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3420000000000001,"15min":1.363},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.863,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.8029999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.472,"15min":1.524},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":2.0550000000000002},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.659},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.21,"5min":2.21,"15min":2.5409999999999999},"last":2.0219999999999998}]}]},{"osd":5,"up_from":13,"seq":55834576024,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48472,"kb_used_data":3400,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323368,"statfs":{"total":96636764160,"available":96587128832,"internally_reserved":0,"allocated":3481600,"data_stored":1818807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.089,"15min":1.133},"min":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.361,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.206},"min":{"1min":0.72999999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.994,"5min":1.994,"15min":2.2909999999999999},"last":0.752}]},{"osd":1,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2789999999999999,"15min":1.274},"min":{"1min":0.93300000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.754,"5min":1.754,"15min":1.8480000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.127,"15min":1.2130000000000001},"min":{"1min":0.82999999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5860000000000001,"5min":1.75,"15min":2.4529999999999998},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.288,"15min":1.2929999999999999},"min":{"1min":0.85099999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2070000000000001,"15min":1.238},"min":{"1min":0.75,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.8009999999999999,"5min":2.266,"15min":2.266},"last":0.872}]},{"osd":3,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3,"15min":1.341},"min":{"1min":0.84399999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.383,"5min":1.361,"15min":1.4019999999999999},"min":{"1min":0.82499999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8300000000000001,"5min":1.843,"15min":1.897},"last":0.83199999999999996}]},{"osd":4,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.2769999999999999,"15min":1.2589999999999999},"min":{"1min":0.90800000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8360000000000001,"5min":1.976,"15min":1.976},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.548,"5min":1.4399999999999999,"15min":1.413},"min":{"1min":0.92900000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.0249999999999999,"5min":2.093,"15min":2.093},"last":0.85399999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4710000000000001,"15min":1.4810000000000001},"min":{"1min":1.1819999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":2.3929999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.625,"5min":1.5109999999999999,"15min":1.528},"min":{"1min":1.147,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.9710000000000001,"5min":2.1579999999999999,"15min":2.52},"last":0.92700000000000005}]},{"osd":7,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.526,"15min":1.5389999999999999},"min":{"1min":1.2350000000000001,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.1019999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.5389999999999999,"15min":1.5680000000000001},"min":{"1min":1.2150000000000001,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.133},"last":0.94199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834576022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52628,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319212,"statfs":{"total":96636764160,"available":96582873088,"internally_reserved":0,"allocated":3477504,"data_stored":1813717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.153},"min":{"1min":0.79700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.383,"5min":1.746,"15min":1.8480000000000001},"last":0.872},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.76500000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5109999999999999,"5min":1.659,"15min":1.869},"last":1.3799999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.58099999999999996},"max":{"1min":1.8380000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7470000000000001,"5min":1.762,"15min":1.762},"last":1.337}]},{"osd":2,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.224,"15min":1.254},"min":{"1min":0.77000000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3420000000000001,"15min":1.3799999999999999},"min":{"1min":0.88700000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.833,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1279999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.236,"15min":1.3049999999999999},"min":{"1min":0.90300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.734,"5min":1.734,"15min":1.7949999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.389,"15min":1.425},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.085}]},{"osd":5,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.3440000000000001},"min":{"1min":0.93500000000000005,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.492},{"interface":"front","average":{"1min":1.371,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.4239999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.131,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.1600000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.605,"5min":1.5860000000000001,"15min":1.597},"min":{"1min":1.2250000000000001,"5min":1.056,"15min":1.056},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.611}]},{"osd":7,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.5249999999999999,"15min":1.5229999999999999},"min":{"1min":0.94699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.587,"15min":1.593},"min":{"1min":1.036,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.1899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56776,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315064,"statfs":{"total":96636764160,"available":96578625536,"internally_reserved":0,"allocated":3469312,"data_stored":1812107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97599999999999998},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5029999999999999,"5min":1.6439999999999999,"15min":1.71},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.165,"15min":1.234},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3640000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.7090000000000001,"5min":1.835,"15min":1.835},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.234,"15min":1.3},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.024,"15min":1.1000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4990000000000001,"5min":1.921,"15min":2.1850000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1759999999999999,"15min":1.2669999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.679,"5min":1.9870000000000001,"15min":2},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.751,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0579999999999998,"5min":2.262,"15min":2.262},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4059999999999999,"15min":1.462},"min":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.401,"15min":1.4850000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0089999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.522,"15min":1.5600000000000001},"min":{"1min":1.02,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0259999999999998,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.526,"15min":1.589},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.5349999999999999,"15min":1.6100000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1110000000000002,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":0.81999999999999995}]},{"osd":7,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.6060000000000001,"15min":1.659},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.121,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.605,"5min":1.609,"15min":1.698},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.2890000000000001,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834576023,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56784,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315056,"statfs":{"total":96636764160,"available":96578617344,"internally_reserved":0,"allocated":3473408,"data_stored":1818751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.90800000000000003},"min":{"1min":0.61899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4470000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.081},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.95299999999999996,"15min":1.022},"min":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.353,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.64900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.79,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7010000000000001,"5min":1.881,"15min":1.958},"last":1.1279999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.155,"15min":1.1739999999999999},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.4319999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.298}]},{"osd":4,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.45,"15min":1.425},"min":{"1min":0.97399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.867,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.0780000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8620000000000001,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.395}]},{"osd":5,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3520000000000001,"15min":1.401},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5920000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.607,"15min":1.6080000000000001},"min":{"1min":1.1319999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9470000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.369}]},{"osd":6,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.512,"15min":1.5509999999999999},"min":{"1min":1.171,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.974,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.333}]},{"osd":7,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.617,"15min":1.6279999999999999},"min":{"1min":1.224,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0750000000000002,"5min":2.1240000000000001,"15min":2.4630000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":1.2190000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.4550000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52624,"kb_used_data":3384,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319216,"statfs":{"total":96636764160,"available":96582877184,"internally_reserved":0,"allocated":3465216,"data_stored":1812105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86799999999999999,"15min":0.92100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.976},"last":0.442},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.024,"15min":1.125},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6870000000000001,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.95299999999999996,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.363,"5min":1.746,"15min":1.79},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.1339999999999999,"15min":1.2110000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.677,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.248,"15min":1.3460000000000001},"min":{"1min":0.48499999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6160000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.208,"15min":1.268},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.8080000000000001,"5min":1.948,"15min":1.948},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.228,"15min":1.325},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.819,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.347,"5min":1.425,"15min":1.476},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.383},"last":1.1639999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.45,"15min":1.54},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.98,"5min":2.2749999999999999,"15min":2.4750000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.4239999999999999,"15min":1.5049999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0539999999999998,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.97199999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4970000000000001,"15min":1.573},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.895,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.4159999999999999,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.931,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.042}]},{"osd":7,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.5740000000000001,"15min":1.641},"min":{"1min":0.75,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":2.1829999999999998,"15min":2.548},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.351,"5min":1.6120000000000001,"15min":1.6870000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.9319999999999999,"5min":2.3730000000000002,"15min":2.548},"last":1.129}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53196,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318644,"statfs":{"total":96636764160,"available":96582291456,"internally_reserved":0,"allocated":4063232,"data_stored":2402999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":0.95999999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.075},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3080000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5309999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.3740000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.421}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.804}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:40:47.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:40:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.345+0000 7f772359e640 1 Processor -- start 2023-10-21T17:40:47.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.345+0000 7f772359e640 1 -- start start 2023-10-21T17:40:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f772359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772415cd00 0x7f77240b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f772359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7724072140 0x7f77240cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f772359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77240cd060 con 0x7f77240ccc00 2023-10-21T17:40:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f772359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77240b17e0 con 0x7f7724072140 2023-10-21T17:40:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f772359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7724071380 con 0x7f772415cd00 2023-10-21T17:40:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f771bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7724072140 0x7f77240cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.347+0000 7f7722d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f77240ccc00 0x7f7724069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46852/0 (socket says 172.21.15.53:46852) 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f771bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7724072140 0x7f77240cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34982/0 (socket says 172.21.15.53:34982) 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f7722d9d640 1 -- 172.21.15.53:0/2050055693 learned_addr learned my addr 172.21.15.53:0/2050055693 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f772259c640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772415cd00 0x7f77240b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f771bfff640 1 -- 172.21.15.53:0/2050055693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772415cd00 msgr2=0x7f77240b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f771bfff640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772415cd00 0x7f77240b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f771bfff640 1 -- 172.21.15.53:0/2050055693 >> v1:172.21.15.53:6789/0 conn(0x7f77240ccc00 legacy=0x7f7724069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f771bfff640 1 -- 172.21.15.53:0/2050055693 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724071660 con 0x7f7724072140 2023-10-21T17:40:47.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.348+0000 7f7721d9b640 1 -- 172.21.15.53:0/2050055693 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054287858 0 0) 0x7f77240cd060 con 0x7f77240ccc00 2023-10-21T17:40:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.349+0000 7f771bfff640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724072140 0x7f77240cc6c0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f770c0097a0 tx=0x7f770c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af4103c0eddbf6b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f7721d9b640 1 -- 172.21.15.53:0/2050055693 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c031830 con 0x7f7724072140 2023-10-21T17:40:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f7721d9b640 1 -- 172.21.15.53:0/2050055693 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c031990 con 0x7f7724072140 2023-10-21T17:40:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f7721d9b640 1 -- 172.21.15.53:0/2050055693 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c031d20 con 0x7f7724072140 2023-10-21T17:40:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 -- 172.21.15.53:0/2050055693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724072140 msgr2=0x7f77240cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724072140 0x7f77240cc6c0 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f770c0097a0 tx=0x7f770c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 -- 172.21.15.53:0/2050055693 shutdown_connections 2023-10-21T17:40:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7724072140 0x7f77240cc6c0 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 --2- 172.21.15.53:0/2050055693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f772415cd00 0x7f77240b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 -- 172.21.15.53:0/2050055693 >> 172.21.15.53:0/2050055693 conn(0x7f772406b280 msgr2=0x7f772406b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.350+0000 7f772359e640 1 -- 172.21.15.53:0/2050055693 shutdown_connections 2023-10-21T17:40:47.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.351+0000 7f772359e640 1 -- 172.21.15.53:0/2050055693 wait complete. 2023-10-21T17:40:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.352+0000 7f772359e640 1 Processor -- start 2023-10-21T17:40:47.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.352+0000 7f772359e640 1 -- start start 2023-10-21T17:40:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7724072140 0x7f77241f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77240ccc00 0x7f7724152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240c6da0 con 0x7f772415cd00 2023-10-21T17:40:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240c6fd0 con 0x7f7724072140 2023-10-21T17:40:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.353+0000 7f772359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77240c72b0 con 0x7f77240ccc00 2023-10-21T17:40:47.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:34992/0 (socket says 172.21.15.53:34992) 2023-10-21T17:40:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 -- 172.21.15.53:0/4114184568 learned_addr learned my addr 172.21.15.53:0/4114184568 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f771bfff640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77240ccc00 0x7f7724152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 -- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77240ccc00 msgr2=0x7f7724152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77240ccc00 0x7f7724152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 -- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7724072140 msgr2=0x7f77241f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7724072140 0x7f77241f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 -- 172.21.15.53:0/4114184568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c009400 con 0x7f772415cd00 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.354+0000 7f7722d9d640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c7a0 tx=0x7f771c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.355+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00a640 con 0x7f772415cd00 2023-10-21T17:40:47.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.355+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c00a7a0 con 0x7f772415cd00 2023-10-21T17:40:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.355+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c00abd0 con 0x7f772415cd00 2023-10-21T17:40:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.355+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77240c7590 con 0x7f772415cd00 2023-10-21T17:40:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.355+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772415f330 con 0x7f772415cd00 2023-10-21T17:40:47.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.356+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77240c7ab0 con 0x7f772415cd00 2023-10-21T17:40:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.357+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f771c00ad30 con 0x7f772415cd00 2023-10-21T17:40:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.358+0000 7f771affd640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 0x7f770406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.358+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f771c089020 con 0x7f772415cd00 2023-10-21T17:40:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.359+0000 7f772259c640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 0x7f770406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.360+0000 7f772259c640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 0x7f770406d5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7714006fd0 tx=0x7f7714008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.362+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f771c053bc0 con 0x7f772415cd00 2023-10-21T17:40:47.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.551+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 --> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7724071c20 con 0x7f770406b0e0 2023-10-21T17:40:47.579 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.577+0000 7f771affd640 1 -- 172.21.15.53:0/4114184568 <== mgr.9841 v2:172.21.15.124:6832/596989303 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287276 (secure 0 0 0) 0x7f771400be20 con 0x7f770406b0e0 2023-10-21T17:40:47.583 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:47.584 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:40:47.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 msgr2=0x7f770406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 0x7f770406d5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7714006fd0 tx=0x7f7714008040 comp rx=0 tx=0).stop 2023-10-21T17:40:47.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 msgr2=0x7f77240c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c7a0 tx=0x7f771c00cc60 comp rx=0 tx=0).stop 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 shutdown_connections 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f770406b0e0 0x7f770406d5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f772415cd00 0x7f77240c6860 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77240ccc00 0x7f7724152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.583+0000 7f772359e640 1 --2- 172.21.15.53:0/4114184568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7724072140 0x7f77241f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.584+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 >> 172.21.15.53:0/4114184568 conn(0x7f772406b280 msgr2=0x7f772415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:47.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.584+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 shutdown_connections 2023-10-21T17:40:47.588 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.584+0000 7f772359e640 1 -- 172.21.15.53:0/4114184568 wait complete. 2023-10-21T17:40:47.605 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-21T17:40:46.950298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":695,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4947,"ondisk_log_size":4947,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":460992,"kb_used_data":28288,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754513728,"statfs":{"total":773094113280,"available":772622057472,"internally_reserved":0,"allocated":28966912,"data_stored":15695148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002112"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545746+0000","last_change":"2023-10-21T17:40:42.541474+0000","last_active":"2023-10-21T17:40:42.545746+0000","last_peered":"2023-10-21T17:40:42.545746+0000","last_clean":"2023-10-21T17:40:42.545746+0000","last_became_active":"2023-10-21T17:40:42.540917+0000","last_became_peered":"2023-10-21T17:40:42.540917+0000","last_unstale":"2023-10-21T17:40:42.545746+0000","last_undegraded":"2023-10-21T17:40:42.545746+0000","last_fullsized":"2023-10-21T17:40:42.545746+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:45:35.102557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.552547+0000","last_change":"2023-10-21T17:40:42.551089+0000","last_active":"2023-10-21T17:40:42.552547+0000","last_peered":"2023-10-21T17:40:42.552547+0000","last_clean":"2023-10-21T17:40:42.552547+0000","last_became_active":"2023-10-21T17:40:42.550812+0000","last_became_peered":"2023-10-21T17:40:42.550812+0000","last_unstale":"2023-10-21T17:40:42.552547+0000","last_undegraded":"2023-10-21T17:40:42.552547+0000","last_fullsized":"2023-10-21T17:40:42.552547+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:13:51.392394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515857+0000","last_change":"2023-10-21T17:40:42.549313+0000","last_active":"2023-10-21T17:40:43.515857+0000","last_peered":"2023-10-21T17:40:43.515857+0000","last_clean":"2023-10-21T17:40:43.515857+0000","last_became_active":"2023-10-21T17:40:42.549026+0000","last_became_peered":"2023-10-21T17:40:42.549026+0000","last_unstale":"2023-10-21T17:40:43.515857+0000","last_undegraded":"2023-10-21T17:40:43.515857+0000","last_fullsized":"2023-10-21T17:40:43.515857+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:03.570137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515724+0000","last_change":"2023-10-21T17:40:42.549746+0000","last_active":"2023-10-21T17:40:43.515724+0000","last_peered":"2023-10-21T17:40:43.515724+0000","last_clean":"2023-10-21T17:40:43.515724+0000","last_became_active":"2023-10-21T17:40:42.532435+0000","last_became_peered":"2023-10-21T17:40:42.532435+0000","last_unstale":"2023-10-21T17:40:43.515724+0000","last_undegraded":"2023-10-21T17:40:43.515724+0000","last_fullsized":"2023-10-21T17:40:43.515724+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:41:31.405739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.529173+0000","last_change":"2023-10-21T17:40:42.526485+0000","last_active":"2023-10-21T17:40:42.529173+0000","last_peered":"2023-10-21T17:40:42.529173+0000","last_clean":"2023-10-21T17:40:42.529173+0000","last_became_active":"2023-10-21T17:40:42.526222+0000","last_became_peered":"2023-10-21T17:40:42.526222+0000","last_unstale":"2023-10-21T17:40:42.529173+0000","last_undegraded":"2023-10-21T17:40:42.529173+0000","last_fullsized":"2023-10-21T17:40:42.529173+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:55.928152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541090+0000","last_change":"2023-10-21T17:40:42.536351+0000","last_active":"2023-10-21T17:40:42.541090+0000","last_peered":"2023-10-21T17:40:42.541090+0000","last_clean":"2023-10-21T17:40:42.541090+0000","last_became_active":"2023-10-21T17:40:42.536084+0000","last_became_peered":"2023-10-21T17:40:42.536084+0000","last_unstale":"2023-10-21T17:40:42.541090+0000","last_undegraded":"2023-10-21T17:40:42.541090+0000","last_fullsized":"2023-10-21T17:40:42.541090+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:50:40.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.537030+0000","last_change":"2023-10-21T17:40:42.532275+0000","last_active":"2023-10-21T17:40:42.537030+0000","last_peered":"2023-10-21T17:40:42.537030+0000","last_clean":"2023-10-21T17:40:42.537030+0000","last_became_active":"2023-10-21T17:40:42.531933+0000","last_became_peered":"2023-10-21T17:40:42.531933+0000","last_unstale":"2023-10-21T17:40:42.537030+0000","last_undegraded":"2023-10-21T17:40:42.537030+0000","last_fullsized":"2023-10-21T17:40:42.537030+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:39.590353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515091+0000","last_change":"2023-10-21T17:40:42.525429+0000","last_active":"2023-10-21T17:40:43.515091+0000","last_peered":"2023-10-21T17:40:43.515091+0000","last_clean":"2023-10-21T17:40:43.515091+0000","last_became_active":"2023-10-21T17:40:42.524591+0000","last_became_peered":"2023-10-21T17:40:42.524591+0000","last_unstale":"2023-10-21T17:40:43.515091+0000","last_undegraded":"2023-10-21T17:40:43.515091+0000","last_fullsized":"2023-10-21T17:40:43.515091+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:02:47.082610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545436+0000","last_change":"2023-10-21T17:40:42.537534+0000","last_active":"2023-10-21T17:40:42.545436+0000","last_peered":"2023-10-21T17:40:42.545436+0000","last_clean":"2023-10-21T17:40:42.545436+0000","last_became_active":"2023-10-21T17:40:42.537236+0000","last_became_peered":"2023-10-21T17:40:42.537236+0000","last_unstale":"2023-10-21T17:40:42.545436+0000","last_undegraded":"2023-10-21T17:40:42.545436+0000","last_fullsized":"2023-10-21T17:40:42.545436+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:32.249928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553130+0000","last_change":"2023-10-21T17:40:42.549430+0000","last_active":"2023-10-21T17:40:42.553130+0000","last_peered":"2023-10-21T17:40:42.553130+0000","last_clean":"2023-10-21T17:40:42.553130+0000","last_became_active":"2023-10-21T17:40:42.549059+0000","last_became_peered":"2023-10-21T17:40:42.549059+0000","last_unstale":"2023-10-21T17:40:42.553130+0000","last_undegraded":"2023-10-21T17:40:42.553130+0000","last_fullsized":"2023-10-21T17:40:42.553130+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:01.221675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547464+0000","last_change":"2023-10-21T17:40:42.546712+0000","last_active":"2023-10-21T17:40:42.547464+0000","last_peered":"2023-10-21T17:40:42.547464+0000","last_clean":"2023-10-21T17:40:42.547464+0000","last_became_active":"2023-10-21T17:40:42.546181+0000","last_became_peered":"2023-10-21T17:40:42.546181+0000","last_unstale":"2023-10-21T17:40:42.547464+0000","last_undegraded":"2023-10-21T17:40:42.547464+0000","last_fullsized":"2023-10-21T17:40:42.547464+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:09:41.079080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546774+0000","last_change":"2023-10-21T17:40:42.542201+0000","last_active":"2023-10-21T17:40:42.546774+0000","last_peered":"2023-10-21T17:40:42.546774+0000","last_clean":"2023-10-21T17:40:42.546774+0000","last_became_active":"2023-10-21T17:40:42.541692+0000","last_became_peered":"2023-10-21T17:40:42.541692+0000","last_unstale":"2023-10-21T17:40:42.546774+0000","last_undegraded":"2023-10-21T17:40:42.546774+0000","last_fullsized":"2023-10-21T17:40:42.546774+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:29:03.361855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515707+0000","last_change":"2023-10-21T17:40:42.548391+0000","last_active":"2023-10-21T17:40:43.515707+0000","last_peered":"2023-10-21T17:40:43.515707+0000","last_clean":"2023-10-21T17:40:43.515707+0000","last_became_active":"2023-10-21T17:40:42.548104+0000","last_became_peered":"2023-10-21T17:40:42.548104+0000","last_unstale":"2023-10-21T17:40:43.515707+0000","last_undegraded":"2023-10-21T17:40:43.515707+0000","last_fullsized":"2023-10-21T17:40:43.515707+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:32:43.706722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.554931+0000","last_change":"2023-10-21T17:40:42.554249+0000","last_active":"2023-10-21T17:40:42.554931+0000","last_peered":"2023-10-21T17:40:42.554931+0000","last_clean":"2023-10-21T17:40:42.554931+0000","last_became_active":"2023-10-21T17:40:42.553948+0000","last_became_peered":"2023-10-21T17:40:42.553948+0000","last_unstale":"2023-10-21T17:40:42.554931+0000","last_undegraded":"2023-10-21T17:40:42.554931+0000","last_fullsized":"2023-10-21T17:40:42.554931+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:46.821913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515113+0000","last_change":"2023-10-21T17:40:42.549092+0000","last_active":"2023-10-21T17:40:43.515113+0000","last_peered":"2023-10-21T17:40:43.515113+0000","last_clean":"2023-10-21T17:40:43.515113+0000","last_became_active":"2023-10-21T17:40:42.543791+0000","last_became_peered":"2023-10-21T17:40:42.543791+0000","last_unstale":"2023-10-21T17:40:43.515113+0000","last_undegraded":"2023-10-21T17:40:43.515113+0000","last_fullsized":"2023-10-21T17:40:43.515113+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:45:48.102101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.518625+0000","last_change":"2023-10-21T17:40:42.535868+0000","last_active":"2023-10-21T17:40:43.518625+0000","last_peered":"2023-10-21T17:40:43.518625+0000","last_clean":"2023-10-21T17:40:43.518625+0000","last_became_active":"2023-10-21T17:40:42.535588+0000","last_became_peered":"2023-10-21T17:40:42.535588+0000","last_unstale":"2023-10-21T17:40:43.518625+0000","last_undegraded":"2023-10-21T17:40:43.518625+0000","last_fullsized":"2023-10-21T17:40:43.518625+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:30:36.869057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519696+0000","last_change":"2023-10-21T17:40:42.530418+0000","last_active":"2023-10-21T17:40:43.519696+0000","last_peered":"2023-10-21T17:40:43.519696+0000","last_clean":"2023-10-21T17:40:43.519696+0000","last_became_active":"2023-10-21T17:40:42.530061+0000","last_became_peered":"2023-10-21T17:40:42.530061+0000","last_unstale":"2023-10-21T17:40:43.519696+0000","last_undegraded":"2023-10-21T17:40:43.519696+0000","last_fullsized":"2023-10-21T17:40:43.519696+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:30:53.616708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516329+0000","last_change":"2023-10-21T17:40:42.553732+0000","last_active":"2023-10-21T17:40:43.516329+0000","last_peered":"2023-10-21T17:40:43.516329+0000","last_clean":"2023-10-21T17:40:43.516329+0000","last_became_active":"2023-10-21T17:40:42.552636+0000","last_became_peered":"2023-10-21T17:40:42.552636+0000","last_unstale":"2023-10-21T17:40:43.516329+0000","last_undegraded":"2023-10-21T17:40:43.516329+0000","last_fullsized":"2023-10-21T17:40:43.516329+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:30:53.616708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519640+0000","last_change":"2023-10-21T17:40:42.548597+0000","last_active":"2023-10-21T17:40:43.519640+0000","last_peered":"2023-10-21T17:40:43.519640+0000","last_clean":"2023-10-21T17:40:43.519640+0000","last_became_active":"2023-10-21T17:40:42.547150+0000","last_became_peered":"2023-10-21T17:40:42.547150+0000","last_unstale":"2023-10-21T17:40:43.519640+0000","last_undegraded":"2023-10-21T17:40:43.519640+0000","last_fullsized":"2023-10-21T17:40:43.519640+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:51:14.501138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519941+0000","last_change":"2023-10-21T17:40:42.532634+0000","last_active":"2023-10-21T17:40:43.519941+0000","last_peered":"2023-10-21T17:40:43.519941+0000","last_clean":"2023-10-21T17:40:43.519941+0000","last_became_active":"2023-10-21T17:40:42.532366+0000","last_became_peered":"2023-10-21T17:40:42.532366+0000","last_unstale":"2023-10-21T17:40:43.519941+0000","last_undegraded":"2023-10-21T17:40:43.519941+0000","last_fullsized":"2023-10-21T17:40:43.519941+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:27.870606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.527116+0000","last_change":"2023-10-21T17:40:42.526094+0000","last_active":"2023-10-21T17:40:42.527116+0000","last_peered":"2023-10-21T17:40:42.527116+0000","last_clean":"2023-10-21T17:40:42.527116+0000","last_became_active":"2023-10-21T17:40:42.525767+0000","last_became_peered":"2023-10-21T17:40:42.525767+0000","last_unstale":"2023-10-21T17:40:42.527116+0000","last_undegraded":"2023-10-21T17:40:42.527116+0000","last_fullsized":"2023-10-21T17:40:42.527116+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:21:16.796664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545787+0000","last_change":"2023-10-21T17:40:42.540489+0000","last_active":"2023-10-21T17:40:42.545787+0000","last_peered":"2023-10-21T17:40:42.545787+0000","last_clean":"2023-10-21T17:40:42.545787+0000","last_became_active":"2023-10-21T17:40:42.540224+0000","last_became_peered":"2023-10-21T17:40:42.540224+0000","last_unstale":"2023-10-21T17:40:42.545787+0000","last_undegraded":"2023-10-21T17:40:42.545787+0000","last_fullsized":"2023-10-21T17:40:42.545787+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:20.628311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.537009+0000","last_change":"2023-10-21T17:40:42.531940+0000","last_active":"2023-10-21T17:40:42.537009+0000","last_peered":"2023-10-21T17:40:42.537009+0000","last_clean":"2023-10-21T17:40:42.537009+0000","last_became_active":"2023-10-21T17:40:42.531537+0000","last_became_peered":"2023-10-21T17:40:42.531537+0000","last_unstale":"2023-10-21T17:40:42.537009+0000","last_undegraded":"2023-10-21T17:40:42.537009+0000","last_fullsized":"2023-10-21T17:40:42.537009+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:23:43.875691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550816+0000","last_change":"2023-10-21T17:40:42.548450+0000","last_active":"2023-10-21T17:40:42.550816+0000","last_peered":"2023-10-21T17:40:42.550816+0000","last_clean":"2023-10-21T17:40:42.550816+0000","last_became_active":"2023-10-21T17:40:42.548187+0000","last_became_peered":"2023-10-21T17:40:42.548187+0000","last_unstale":"2023-10-21T17:40:42.550816+0000","last_undegraded":"2023-10-21T17:40:42.550816+0000","last_fullsized":"2023-10-21T17:40:42.550816+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:56:36.331352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545727+0000","last_change":"2023-10-21T17:40:42.540179+0000","last_active":"2023-10-21T17:40:42.545727+0000","last_peered":"2023-10-21T17:40:42.545727+0000","last_clean":"2023-10-21T17:40:42.545727+0000","last_became_active":"2023-10-21T17:40:42.539531+0000","last_became_peered":"2023-10-21T17:40:42.539531+0000","last_unstale":"2023-10-21T17:40:42.545727+0000","last_undegraded":"2023-10-21T17:40:42.545727+0000","last_fullsized":"2023-10-21T17:40:42.545727+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:03:11.510221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545210+0000","last_change":"2023-10-21T17:40:42.539742+0000","last_active":"2023-10-21T17:40:42.545210+0000","last_peered":"2023-10-21T17:40:42.545210+0000","last_clean":"2023-10-21T17:40:42.545210+0000","last_became_active":"2023-10-21T17:40:42.539428+0000","last_became_peered":"2023-10-21T17:40:42.539428+0000","last_unstale":"2023-10-21T17:40:42.545210+0000","last_undegraded":"2023-10-21T17:40:42.545210+0000","last_fullsized":"2023-10-21T17:40:42.545210+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:53:12.174291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.542415+0000","last_change":"2023-10-21T17:40:42.539420+0000","last_active":"2023-10-21T17:40:42.542415+0000","last_peered":"2023-10-21T17:40:42.542415+0000","last_clean":"2023-10-21T17:40:42.542415+0000","last_became_active":"2023-10-21T17:40:42.539124+0000","last_became_peered":"2023-10-21T17:40:42.539124+0000","last_unstale":"2023-10-21T17:40:42.542415+0000","last_undegraded":"2023-10-21T17:40:42.542415+0000","last_fullsized":"2023-10-21T17:40:42.542415+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:22.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.534661+0000","last_change":"2023-10-21T17:40:42.531050+0000","last_active":"2023-10-21T17:40:42.534661+0000","last_peered":"2023-10-21T17:40:42.534661+0000","last_clean":"2023-10-21T17:40:42.534661+0000","last_became_active":"2023-10-21T17:40:42.530744+0000","last_became_peered":"2023-10-21T17:40:42.530744+0000","last_unstale":"2023-10-21T17:40:42.534661+0000","last_undegraded":"2023-10-21T17:40:42.534661+0000","last_fullsized":"2023-10-21T17:40:42.534661+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:50:55.928152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517155+0000","last_change":"2023-10-21T17:40:42.527652+0000","last_active":"2023-10-21T17:40:43.517155+0000","last_peered":"2023-10-21T17:40:43.517155+0000","last_clean":"2023-10-21T17:40:43.517155+0000","last_became_active":"2023-10-21T17:40:42.521681+0000","last_became_peered":"2023-10-21T17:40:42.521681+0000","last_unstale":"2023-10-21T17:40:43.517155+0000","last_undegraded":"2023-10-21T17:40:43.517155+0000","last_fullsized":"2023-10-21T17:40:43.517155+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:15.282192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.533924+0000","last_change":"2023-10-21T17:40:42.531040+0000","last_active":"2023-10-21T17:40:42.533924+0000","last_peered":"2023-10-21T17:40:42.533924+0000","last_clean":"2023-10-21T17:40:42.533924+0000","last_became_active":"2023-10-21T17:40:42.530736+0000","last_became_peered":"2023-10-21T17:40:42.530736+0000","last_unstale":"2023-10-21T17:40:42.533924+0000","last_undegraded":"2023-10-21T17:40:42.533924+0000","last_fullsized":"2023-10-21T17:40:42.533924+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:12.384348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515088+0000","last_change":"2023-10-21T17:40:42.544786+0000","last_active":"2023-10-21T17:40:43.515088+0000","last_peered":"2023-10-21T17:40:43.515088+0000","last_clean":"2023-10-21T17:40:43.515088+0000","last_became_active":"2023-10-21T17:40:42.544510+0000","last_became_peered":"2023-10-21T17:40:42.544510+0000","last_unstale":"2023-10-21T17:40:43.515088+0000","last_undegraded":"2023-10-21T17:40:43.515088+0000","last_fullsized":"2023-10-21T17:40:43.515088+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:29.499788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550834+0000","last_change":"2023-10-21T17:40:42.548700+0000","last_active":"2023-10-21T17:40:42.550834+0000","last_peered":"2023-10-21T17:40:42.550834+0000","last_clean":"2023-10-21T17:40:42.550834+0000","last_became_active":"2023-10-21T17:40:42.547072+0000","last_became_peered":"2023-10-21T17:40:42.547072+0000","last_unstale":"2023-10-21T17:40:42.550834+0000","last_undegraded":"2023-10-21T17:40:42.550834+0000","last_fullsized":"2023-10-21T17:40:42.550834+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:22.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"274'4","reported_seq":15,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553920+0000","last_change":"2023-10-21T17:40:41.515775+0000","last_active":"2023-10-21T17:40:42.553920+0000","last_peered":"2023-10-21T17:40:42.553920+0000","last_clean":"2023-10-21T17:40:42.553920+0000","last_became_active":"2023-10-21T17:40:41.515484+0000","last_became_peered":"2023-10-21T17:40:41.515484+0000","last_unstale":"2023-10-21T17:40:42.553920+0000","last_undegraded":"2023-10-21T17:40:42.553920+0000","last_fullsized":"2023-10-21T17:40:42.553920+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:14.265654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.552877+0000","last_change":"2023-10-21T17:40:42.549625+0000","last_active":"2023-10-21T17:40:42.552877+0000","last_peered":"2023-10-21T17:40:42.552877+0000","last_clean":"2023-10-21T17:40:42.552877+0000","last_became_active":"2023-10-21T17:40:42.548133+0000","last_became_peered":"2023-10-21T17:40:42.548133+0000","last_unstale":"2023-10-21T17:40:42.552877+0000","last_undegraded":"2023-10-21T17:40:42.552877+0000","last_fullsized":"2023-10-21T17:40:42.552877+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:29:02.987638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549385+0000","last_change":"2023-10-21T17:40:41.512308+0000","last_active":"2023-10-21T17:40:42.549385+0000","last_peered":"2023-10-21T17:40:42.549385+0000","last_clean":"2023-10-21T17:40:42.549385+0000","last_became_active":"2023-10-21T17:40:41.511439+0000","last_became_peered":"2023-10-21T17:40:41.511439+0000","last_unstale":"2023-10-21T17:40:42.549385+0000","last_undegraded":"2023-10-21T17:40:42.549385+0000","last_fullsized":"2023-10-21T17:40:42.549385+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:44:51.353094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546780+0000","last_change":"2023-10-21T17:40:42.542988+0000","last_active":"2023-10-21T17:40:42.546780+0000","last_peered":"2023-10-21T17:40:42.546780+0000","last_clean":"2023-10-21T17:40:42.546780+0000","last_became_active":"2023-10-21T17:40:42.542719+0000","last_became_peered":"2023-10-21T17:40:42.542719+0000","last_unstale":"2023-10-21T17:40:42.546780+0000","last_undegraded":"2023-10-21T17:40:42.546780+0000","last_fullsized":"2023-10-21T17:40:42.546780+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:14.512713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550502+0000","last_change":"2023-10-21T17:40:41.511235+0000","last_active":"2023-10-21T17:40:42.550502+0000","last_peered":"2023-10-21T17:40:42.550502+0000","last_clean":"2023-10-21T17:40:42.550502+0000","last_became_active":"2023-10-21T17:40:41.505475+0000","last_became_peered":"2023-10-21T17:40:41.505475+0000","last_unstale":"2023-10-21T17:40:42.550502+0000","last_undegraded":"2023-10-21T17:40:42.550502+0000","last_fullsized":"2023-10-21T17:40:42.550502+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:29:48.812842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540322+0000","last_change":"2023-10-21T17:40:42.536242+0000","last_active":"2023-10-21T17:40:42.540322+0000","last_peered":"2023-10-21T17:40:42.540322+0000","last_clean":"2023-10-21T17:40:42.540322+0000","last_became_active":"2023-10-21T17:40:42.535463+0000","last_became_peered":"2023-10-21T17:40:42.535463+0000","last_unstale":"2023-10-21T17:40:42.540322+0000","last_undegraded":"2023-10-21T17:40:42.540322+0000","last_fullsized":"2023-10-21T17:40:42.540322+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:17.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.550609+0000","last_change":"2023-10-21T17:40:41.507550+0000","last_active":"2023-10-21T17:40:42.550609+0000","last_peered":"2023-10-21T17:40:42.550609+0000","last_clean":"2023-10-21T17:40:42.550609+0000","last_became_active":"2023-10-21T17:40:41.505155+0000","last_became_peered":"2023-10-21T17:40:41.505155+0000","last_unstale":"2023-10-21T17:40:42.550609+0000","last_undegraded":"2023-10-21T17:40:42.550609+0000","last_fullsized":"2023-10-21T17:40:42.550609+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:50:25.764299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516025+0000","last_change":"2023-10-21T17:40:42.551931+0000","last_active":"2023-10-21T17:40:43.516025+0000","last_peered":"2023-10-21T17:40:43.516025+0000","last_clean":"2023-10-21T17:40:43.516025+0000","last_became_active":"2023-10-21T17:40:42.550131+0000","last_became_peered":"2023-10-21T17:40:42.550131+0000","last_unstale":"2023-10-21T17:40:43.516025+0000","last_undegraded":"2023-10-21T17:40:43.516025+0000","last_fullsized":"2023-10-21T17:40:43.516025+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.224336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.553070+0000","last_change":"2023-10-21T17:40:41.506400+0000","last_active":"2023-10-21T17:40:42.553070+0000","last_peered":"2023-10-21T17:40:42.553070+0000","last_clean":"2023-10-21T17:40:42.553070+0000","last_became_active":"2023-10-21T17:40:41.506020+0000","last_became_peered":"2023-10-21T17:40:41.506020+0000","last_unstale":"2023-10-21T17:40:42.553070+0000","last_undegraded":"2023-10-21T17:40:42.553070+0000","last_fullsized":"2023-10-21T17:40:42.553070+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:55:16.988226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547548+0000","last_change":"2023-10-21T17:40:42.546477+0000","last_active":"2023-10-21T17:40:42.547548+0000","last_peered":"2023-10-21T17:40:42.547548+0000","last_clean":"2023-10-21T17:40:42.547548+0000","last_became_active":"2023-10-21T17:40:42.546203+0000","last_became_peered":"2023-10-21T17:40:42.546203+0000","last_unstale":"2023-10-21T17:40:42.547548+0000","last_undegraded":"2023-10-21T17:40:42.547548+0000","last_fullsized":"2023-10-21T17:40:42.547548+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:30:23.660382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547737+0000","last_change":"2023-10-21T17:40:41.512089+0000","last_active":"2023-10-21T17:40:42.547737+0000","last_peered":"2023-10-21T17:40:42.547737+0000","last_clean":"2023-10-21T17:40:42.547737+0000","last_became_active":"2023-10-21T17:40:41.511790+0000","last_became_peered":"2023-10-21T17:40:41.511790+0000","last_unstale":"2023-10-21T17:40:42.547737+0000","last_undegraded":"2023-10-21T17:40:42.547737+0000","last_fullsized":"2023-10-21T17:40:42.547737+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:38.103936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.556771+0000","last_change":"2023-10-21T17:40:42.556268+0000","last_active":"2023-10-21T17:40:42.556771+0000","last_peered":"2023-10-21T17:40:42.556771+0000","last_clean":"2023-10-21T17:40:42.556771+0000","last_became_active":"2023-10-21T17:40:42.555873+0000","last_became_peered":"2023-10-21T17:40:42.555873+0000","last_unstale":"2023-10-21T17:40:42.556771+0000","last_undegraded":"2023-10-21T17:40:42.556771+0000","last_fullsized":"2023-10-21T17:40:42.556771+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:53.272545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515723+0000","last_change":"2023-10-21T17:40:41.499396+0000","last_active":"2023-10-21T17:40:43.515723+0000","last_peered":"2023-10-21T17:40:43.515723+0000","last_clean":"2023-10-21T17:40:43.515723+0000","last_became_active":"2023-10-21T17:40:41.498697+0000","last_became_peered":"2023-10-21T17:40:41.498697+0000","last_unstale":"2023-10-21T17:40:43.515723+0000","last_undegraded":"2023-10-21T17:40:43.515723+0000","last_fullsized":"2023-10-21T17:40:43.515723+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:55:18.446164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519711+0000","last_change":"2023-10-21T17:40:42.530842+0000","last_active":"2023-10-21T17:40:43.519711+0000","last_peered":"2023-10-21T17:40:43.519711+0000","last_clean":"2023-10-21T17:40:43.519711+0000","last_became_active":"2023-10-21T17:40:42.530436+0000","last_became_peered":"2023-10-21T17:40:42.530436+0000","last_unstale":"2023-10-21T17:40:43.519711+0000","last_undegraded":"2023-10-21T17:40:43.519711+0000","last_fullsized":"2023-10-21T17:40:43.519711+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:22:54.419654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515623+0000","last_change":"2023-10-21T17:40:41.501427+0000","last_active":"2023-10-21T17:40:43.515623+0000","last_peered":"2023-10-21T17:40:43.515623+0000","last_clean":"2023-10-21T17:40:43.515623+0000","last_became_active":"2023-10-21T17:40:41.498937+0000","last_became_peered":"2023-10-21T17:40:41.498937+0000","last_unstale":"2023-10-21T17:40:43.515623+0000","last_undegraded":"2023-10-21T17:40:43.515623+0000","last_fullsized":"2023-10-21T17:40:43.515623+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:35:01.646372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515282+0000","last_change":"2023-10-21T17:40:42.552347+0000","last_active":"2023-10-21T17:40:43.515282+0000","last_peered":"2023-10-21T17:40:43.515282+0000","last_clean":"2023-10-21T17:40:43.515282+0000","last_became_active":"2023-10-21T17:40:42.551606+0000","last_became_peered":"2023-10-21T17:40:42.551606+0000","last_unstale":"2023-10-21T17:40:43.515282+0000","last_undegraded":"2023-10-21T17:40:43.515282+0000","last_fullsized":"2023-10-21T17:40:43.515282+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:22:08.081639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519969+0000","last_change":"2023-10-21T17:40:41.514484+0000","last_active":"2023-10-21T17:40:43.519969+0000","last_peered":"2023-10-21T17:40:43.519969+0000","last_clean":"2023-10-21T17:40:43.519969+0000","last_became_active":"2023-10-21T17:40:41.513572+0000","last_became_peered":"2023-10-21T17:40:41.513572+0000","last_unstale":"2023-10-21T17:40:43.519969+0000","last_undegraded":"2023-10-21T17:40:43.519969+0000","last_fullsized":"2023-10-21T17:40:43.519969+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:02.553599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545905+0000","last_change":"2023-10-21T17:40:42.541292+0000","last_active":"2023-10-21T17:40:42.545905+0000","last_peered":"2023-10-21T17:40:42.545905+0000","last_clean":"2023-10-21T17:40:42.545905+0000","last_became_active":"2023-10-21T17:40:42.540984+0000","last_became_peered":"2023-10-21T17:40:42.540984+0000","last_unstale":"2023-10-21T17:40:42.545905+0000","last_undegraded":"2023-10-21T17:40:42.545905+0000","last_fullsized":"2023-10-21T17:40:42.545905+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:08:16.647471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517382+0000","last_change":"2023-10-21T17:40:41.502046+0000","last_active":"2023-10-21T17:40:43.517382+0000","last_peered":"2023-10-21T17:40:43.517382+0000","last_clean":"2023-10-21T17:40:43.517382+0000","last_became_active":"2023-10-21T17:40:41.498316+0000","last_became_peered":"2023-10-21T17:40:41.498316+0000","last_unstale":"2023-10-21T17:40:43.517382+0000","last_undegraded":"2023-10-21T17:40:43.517382+0000","last_fullsized":"2023-10-21T17:40:43.517382+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:23:42.859153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515614+0000","last_change":"2023-10-21T17:40:42.523579+0000","last_active":"2023-10-21T17:40:43.515614+0000","last_peered":"2023-10-21T17:40:43.515614+0000","last_clean":"2023-10-21T17:40:43.515614+0000","last_became_active":"2023-10-21T17:40:42.523014+0000","last_became_peered":"2023-10-21T17:40:42.523014+0000","last_unstale":"2023-10-21T17:40:43.515614+0000","last_undegraded":"2023-10-21T17:40:43.515614+0000","last_fullsized":"2023-10-21T17:40:43.515614+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:51:13.186717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"274'3","reported_seq":15,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519904+0000","last_change":"2023-10-21T17:40:41.501154+0000","last_active":"2023-10-21T17:40:43.519904+0000","last_peered":"2023-10-21T17:40:43.519904+0000","last_clean":"2023-10-21T17:40:43.519904+0000","last_became_active":"2023-10-21T17:40:41.500856+0000","last_became_peered":"2023-10-21T17:40:41.500856+0000","last_unstale":"2023-10-21T17:40:43.519904+0000","last_undegraded":"2023-10-21T17:40:43.519904+0000","last_fullsized":"2023-10-21T17:40:43.519904+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:25:28.483250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.554636+0000","last_change":"2023-10-21T17:40:42.554029+0000","last_active":"2023-10-21T17:40:42.554636+0000","last_peered":"2023-10-21T17:40:42.554636+0000","last_clean":"2023-10-21T17:40:42.554636+0000","last_became_active":"2023-10-21T17:40:42.552973+0000","last_became_peered":"2023-10-21T17:40:42.552973+0000","last_unstale":"2023-10-21T17:40:42.554636+0000","last_undegraded":"2023-10-21T17:40:42.554636+0000","last_fullsized":"2023-10-21T17:40:42.554636+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:42:53.351945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"274'6","reported_seq":17,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.557216+0000","last_change":"2023-10-21T17:40:41.515864+0000","last_active":"2023-10-21T17:40:42.557216+0000","last_peered":"2023-10-21T17:40:42.557216+0000","last_clean":"2023-10-21T17:40:42.557216+0000","last_became_active":"2023-10-21T17:40:41.515566+0000","last_became_peered":"2023-10-21T17:40:41.515566+0000","last_unstale":"2023-10-21T17:40:42.557216+0000","last_undegraded":"2023-10-21T17:40:42.557216+0000","last_fullsized":"2023-10-21T17:40:42.557216+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:42.134246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.546756+0000","last_change":"2023-10-21T17:40:42.542021+0000","last_active":"2023-10-21T17:40:42.546756+0000","last_peered":"2023-10-21T17:40:42.546756+0000","last_clean":"2023-10-21T17:40:42.546756+0000","last_became_active":"2023-10-21T17:40:42.538806+0000","last_became_peered":"2023-10-21T17:40:42.538806+0000","last_unstale":"2023-10-21T17:40:42.546756+0000","last_undegraded":"2023-10-21T17:40:42.546756+0000","last_fullsized":"2023-10-21T17:40:42.546756+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:34.933250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516773+0000","last_change":"2023-10-21T17:40:41.497629+0000","last_active":"2023-10-21T17:40:43.516773+0000","last_peered":"2023-10-21T17:40:43.516773+0000","last_clean":"2023-10-21T17:40:43.516773+0000","last_became_active":"2023-10-21T17:40:41.497228+0000","last_became_peered":"2023-10-21T17:40:41.497228+0000","last_unstale":"2023-10-21T17:40:43.516773+0000","last_undegraded":"2023-10-21T17:40:43.516773+0000","last_fullsized":"2023-10-21T17:40:43.516773+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:43.179333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516284+0000","last_change":"2023-10-21T17:40:42.550129+0000","last_active":"2023-10-21T17:40:43.516284+0000","last_peered":"2023-10-21T17:40:43.516284+0000","last_clean":"2023-10-21T17:40:43.516284+0000","last_became_active":"2023-10-21T17:40:42.544475+0000","last_became_peered":"2023-10-21T17:40:42.544475+0000","last_unstale":"2023-10-21T17:40:43.516284+0000","last_undegraded":"2023-10-21T17:40:43.516284+0000","last_fullsized":"2023-10-21T17:40:43.516284+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:51:14.501138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.539765+0000","last_change":"2023-10-21T17:40:41.512572+0000","last_active":"2023-10-21T17:40:42.539765+0000","last_peered":"2023-10-21T17:40:42.539765+0000","last_clean":"2023-10-21T17:40:42.539765+0000","last_became_active":"2023-10-21T17:40:41.512047+0000","last_became_peered":"2023-10-21T17:40:41.512047+0000","last_unstale":"2023-10-21T17:40:42.539765+0000","last_undegraded":"2023-10-21T17:40:42.539765+0000","last_fullsized":"2023-10-21T17:40:42.539765+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:50:40.718850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519503+0000","last_change":"2023-10-21T17:40:42.530817+0000","last_active":"2023-10-21T17:40:43.519503+0000","last_peered":"2023-10-21T17:40:43.519503+0000","last_clean":"2023-10-21T17:40:43.519503+0000","last_became_active":"2023-10-21T17:40:42.530517+0000","last_became_peered":"2023-10-21T17:40:42.530517+0000","last_unstale":"2023-10-21T17:40:43.519503+0000","last_undegraded":"2023-10-21T17:40:43.519503+0000","last_fullsized":"2023-10-21T17:40:43.519503+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:05:55.535354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.539062+0000","last_change":"2023-10-21T17:40:41.516565+0000","last_active":"2023-10-21T17:40:42.539062+0000","last_peered":"2023-10-21T17:40:42.539062+0000","last_clean":"2023-10-21T17:40:42.539062+0000","last_became_active":"2023-10-21T17:40:41.515183+0000","last_became_peered":"2023-10-21T17:40:41.515183+0000","last_unstale":"2023-10-21T17:40:42.539062+0000","last_undegraded":"2023-10-21T17:40:42.539062+0000","last_fullsized":"2023-10-21T17:40:42.539062+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:25:16.107024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.520055+0000","last_change":"2023-10-21T17:40:42.546988+0000","last_active":"2023-10-21T17:40:43.520055+0000","last_peered":"2023-10-21T17:40:43.520055+0000","last_clean":"2023-10-21T17:40:43.520055+0000","last_became_active":"2023-10-21T17:40:42.546706+0000","last_became_peered":"2023-10-21T17:40:42.546706+0000","last_unstale":"2023-10-21T17:40:43.520055+0000","last_undegraded":"2023-10-21T17:40:43.520055+0000","last_fullsized":"2023-10-21T17:40:43.520055+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:27:39.824618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"274'8","reported_seq":19,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.555839+0000","last_change":"2023-10-21T17:40:41.512974+0000","last_active":"2023-10-21T17:40:42.555839+0000","last_peered":"2023-10-21T17:40:42.555839+0000","last_clean":"2023-10-21T17:40:42.555839+0000","last_became_active":"2023-10-21T17:40:41.512707+0000","last_became_peered":"2023-10-21T17:40:41.512707+0000","last_unstale":"2023-10-21T17:40:42.555839+0000","last_undegraded":"2023-10-21T17:40:42.555839+0000","last_fullsized":"2023-10-21T17:40:42.555839+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:38:43.179333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541251+0000","last_change":"2023-10-21T17:40:42.537465+0000","last_active":"2023-10-21T17:40:42.541251+0000","last_peered":"2023-10-21T17:40:42.541251+0000","last_clean":"2023-10-21T17:40:42.541251+0000","last_became_active":"2023-10-21T17:40:42.537127+0000","last_became_peered":"2023-10-21T17:40:42.537127+0000","last_unstale":"2023-10-21T17:40:42.541251+0000","last_undegraded":"2023-10-21T17:40:42.541251+0000","last_fullsized":"2023-10-21T17:40:42.541251+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:01:20.628311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.547861+0000","last_change":"2023-10-21T17:40:41.503085+0000","last_active":"2023-10-21T17:40:42.547861+0000","last_peered":"2023-10-21T17:40:42.547861+0000","last_clean":"2023-10-21T17:40:42.547861+0000","last_became_active":"2023-10-21T17:40:41.502733+0000","last_became_peered":"2023-10-21T17:40:41.502733+0000","last_unstale":"2023-10-21T17:40:42.547861+0000","last_undegraded":"2023-10-21T17:40:42.547861+0000","last_fullsized":"2023-10-21T17:40:42.547861+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:43:41.988063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545333+0000","last_change":"2023-10-21T17:40:42.542268+0000","last_active":"2023-10-21T17:40:42.545333+0000","last_peered":"2023-10-21T17:40:42.545333+0000","last_clean":"2023-10-21T17:40:42.545333+0000","last_became_active":"2023-10-21T17:40:42.541999+0000","last_became_peered":"2023-10-21T17:40:42.541999+0000","last_unstale":"2023-10-21T17:40:42.545333+0000","last_undegraded":"2023-10-21T17:40:42.545333+0000","last_fullsized":"2023-10-21T17:40:42.545333+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:42:53.351945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"274'2","reported_seq":13,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.555476+0000","last_change":"2023-10-21T17:40:41.498860+0000","last_active":"2023-10-21T17:40:42.555476+0000","last_peered":"2023-10-21T17:40:42.555476+0000","last_clean":"2023-10-21T17:40:42.555476+0000","last_became_active":"2023-10-21T17:40:41.497936+0000","last_became_peered":"2023-10-21T17:40:41.497936+0000","last_unstale":"2023-10-21T17:40:42.555476+0000","last_undegraded":"2023-10-21T17:40:42.555476+0000","last_fullsized":"2023-10-21T17:40:42.555476+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:28:52.256006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541065+0000","last_change":"2023-10-21T17:40:42.536241+0000","last_active":"2023-10-21T17:40:42.541065+0000","last_peered":"2023-10-21T17:40:42.541065+0000","last_clean":"2023-10-21T17:40:42.541065+0000","last_became_active":"2023-10-21T17:40:42.535733+0000","last_became_peered":"2023-10-21T17:40:42.535733+0000","last_unstale":"2023-10-21T17:40:42.541065+0000","last_undegraded":"2023-10-21T17:40:42.541065+0000","last_fullsized":"2023-10-21T17:40:42.541065+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:21:12.384348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"274'1","reported_seq":12,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549255+0000","last_change":"2023-10-21T17:40:41.512355+0000","last_active":"2023-10-21T17:40:42.549255+0000","last_peered":"2023-10-21T17:40:42.549255+0000","last_clean":"2023-10-21T17:40:42.549255+0000","last_became_active":"2023-10-21T17:40:41.512016+0000","last_became_peered":"2023-10-21T17:40:41.512016+0000","last_unstale":"2023-10-21T17:40:42.549255+0000","last_undegraded":"2023-10-21T17:40:42.549255+0000","last_fullsized":"2023-10-21T17:40:42.549255+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:50:17.483951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515910+0000","last_change":"2023-10-21T17:40:42.548197+0000","last_active":"2023-10-21T17:40:43.515910+0000","last_peered":"2023-10-21T17:40:43.515910+0000","last_clean":"2023-10-21T17:40:43.515910+0000","last_became_active":"2023-10-21T17:40:42.545636+0000","last_became_peered":"2023-10-21T17:40:42.545636+0000","last_unstale":"2023-10-21T17:40:43.515910+0000","last_undegraded":"2023-10-21T17:40:43.515910+0000","last_fullsized":"2023-10-21T17:40:43.515910+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:30:36.869057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516803+0000","last_change":"2023-10-21T17:40:41.501403+0000","last_active":"2023-10-21T17:40:43.516803+0000","last_peered":"2023-10-21T17:40:43.516803+0000","last_clean":"2023-10-21T17:40:43.516803+0000","last_became_active":"2023-10-21T17:40:41.501026+0000","last_became_peered":"2023-10-21T17:40:41.501026+0000","last_unstale":"2023-10-21T17:40:43.516803+0000","last_undegraded":"2023-10-21T17:40:43.516803+0000","last_fullsized":"2023-10-21T17:40:43.516803+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:01:42.134246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.549970+0000","last_change":"2023-10-21T17:40:42.547341+0000","last_active":"2023-10-21T17:40:42.549970+0000","last_peered":"2023-10-21T17:40:42.549970+0000","last_clean":"2023-10-21T17:40:42.549970+0000","last_became_active":"2023-10-21T17:40:42.547021+0000","last_became_peered":"2023-10-21T17:40:42.547021+0000","last_unstale":"2023-10-21T17:40:42.549970+0000","last_undegraded":"2023-10-21T17:40:42.549970+0000","last_fullsized":"2023-10-21T17:40:42.549970+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:39:18.909871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516402+0000","last_change":"2023-10-21T17:40:41.503144+0000","last_active":"2023-10-21T17:40:43.516402+0000","last_peered":"2023-10-21T17:40:43.516402+0000","last_clean":"2023-10-21T17:40:43.516402+0000","last_became_active":"2023-10-21T17:40:41.502566+0000","last_became_peered":"2023-10-21T17:40:41.502566+0000","last_unstale":"2023-10-21T17:40:43.516402+0000","last_undegraded":"2023-10-21T17:40:43.516402+0000","last_fullsized":"2023-10-21T17:40:43.516402+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:57.369330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.545119+0000","last_change":"2023-10-21T17:40:42.541871+0000","last_active":"2023-10-21T17:40:42.545119+0000","last_peered":"2023-10-21T17:40:42.545119+0000","last_clean":"2023-10-21T17:40:42.545119+0000","last_became_active":"2023-10-21T17:40:42.541593+0000","last_became_peered":"2023-10-21T17:40:42.541593+0000","last_unstale":"2023-10-21T17:40:42.545119+0000","last_undegraded":"2023-10-21T17:40:42.545119+0000","last_fullsized":"2023-10-21T17:40:42.545119+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:59:39.120475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515739+0000","last_change":"2023-10-21T17:40:41.500455+0000","last_active":"2023-10-21T17:40:43.515739+0000","last_peered":"2023-10-21T17:40:43.515739+0000","last_clean":"2023-10-21T17:40:43.515739+0000","last_became_active":"2023-10-21T17:40:41.498478+0000","last_became_peered":"2023-10-21T17:40:41.498478+0000","last_unstale":"2023-10-21T17:40:43.515739+0000","last_undegraded":"2023-10-21T17:40:43.515739+0000","last_fullsized":"2023-10-21T17:40:43.515739+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T03:30:17.029701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519796+0000","last_change":"2023-10-21T17:40:42.521477+0000","last_active":"2023-10-21T17:40:43.519796+0000","last_peered":"2023-10-21T17:40:43.519796+0000","last_clean":"2023-10-21T17:40:43.519796+0000","last_became_active":"2023-10-21T17:40:42.521145+0000","last_became_peered":"2023-10-21T17:40:42.521145+0000","last_unstale":"2023-10-21T17:40:43.519796+0000","last_undegraded":"2023-10-21T17:40:43.519796+0000","last_fullsized":"2023-10-21T17:40:43.519796+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:45:17.066518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.536075+0000","last_change":"2023-10-21T17:40:41.505227+0000","last_active":"2023-10-21T17:40:42.536075+0000","last_peered":"2023-10-21T17:40:42.536075+0000","last_clean":"2023-10-21T17:40:42.536075+0000","last_became_active":"2023-10-21T17:40:41.504712+0000","last_became_peered":"2023-10-21T17:40:41.504712+0000","last_unstale":"2023-10-21T17:40:42.536075+0000","last_undegraded":"2023-10-21T17:40:42.536075+0000","last_fullsized":"2023-10-21T17:40:42.536075+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:02:47.883648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.541304+0000","last_change":"2023-10-21T17:40:42.537178+0000","last_active":"2023-10-21T17:40:42.541304+0000","last_peered":"2023-10-21T17:40:42.541304+0000","last_clean":"2023-10-21T17:40:42.541304+0000","last_became_active":"2023-10-21T17:40:42.536879+0000","last_became_peered":"2023-10-21T17:40:42.536879+0000","last_unstale":"2023-10-21T17:40:42.541304+0000","last_undegraded":"2023-10-21T17:40:42.541304+0000","last_fullsized":"2023-10-21T17:40:42.541304+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T04:56:36.331352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516528+0000","last_change":"2023-10-21T17:40:41.496704+0000","last_active":"2023-10-21T17:40:43.516528+0000","last_peered":"2023-10-21T17:40:43.516528+0000","last_clean":"2023-10-21T17:40:43.516528+0000","last_became_active":"2023-10-21T17:40:41.496063+0000","last_became_peered":"2023-10-21T17:40:41.496063+0000","last_unstale":"2023-10-21T17:40:43.516528+0000","last_undegraded":"2023-10-21T17:40:43.516528+0000","last_fullsized":"2023-10-21T17:40:43.516528+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:15:45.805375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515499+0000","last_change":"2023-10-21T17:40:42.545415+0000","last_active":"2023-10-21T17:40:43.515499+0000","last_peered":"2023-10-21T17:40:43.515499+0000","last_clean":"2023-10-21T17:40:43.515499+0000","last_became_active":"2023-10-21T17:40:42.545122+0000","last_became_peered":"2023-10-21T17:40:42.545122+0000","last_unstale":"2023-10-21T17:40:43.515499+0000","last_undegraded":"2023-10-21T17:40:43.515499+0000","last_fullsized":"2023-10-21T17:40:43.515499+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:56:10.172790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"274'2","reported_seq":14,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.517744+0000","last_change":"2023-10-21T17:40:41.513429+0000","last_active":"2023-10-21T17:40:43.517744+0000","last_peered":"2023-10-21T17:40:43.517744+0000","last_clean":"2023-10-21T17:40:43.517744+0000","last_became_active":"2023-10-21T17:40:41.513158+0000","last_became_peered":"2023-10-21T17:40:41.513158+0000","last_unstale":"2023-10-21T17:40:43.517744+0000","last_undegraded":"2023-10-21T17:40:43.517744+0000","last_fullsized":"2023-10-21T17:40:43.517744+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:56:16.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519007+0000","last_change":"2023-10-21T17:40:42.523684+0000","last_active":"2023-10-21T17:40:43.519007+0000","last_peered":"2023-10-21T17:40:43.519007+0000","last_clean":"2023-10-21T17:40:43.519007+0000","last_became_active":"2023-10-21T17:40:42.523337+0000","last_became_peered":"2023-10-21T17:40:42.523337+0000","last_unstale":"2023-10-21T17:40:43.519007+0000","last_undegraded":"2023-10-21T17:40:43.519007+0000","last_fullsized":"2023-10-21T17:40:43.519007+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T01:49:24.224336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540788+0000","last_change":"2023-10-21T17:40:41.514879+0000","last_active":"2023-10-21T17:40:42.540788+0000","last_peered":"2023-10-21T17:40:42.540788+0000","last_clean":"2023-10-21T17:40:42.540788+0000","last_became_active":"2023-10-21T17:40:41.512216+0000","last_became_peered":"2023-10-21T17:40:41.512216+0000","last_unstale":"2023-10-21T17:40:42.540788+0000","last_undegraded":"2023-10-21T17:40:42.540788+0000","last_fullsized":"2023-10-21T17:40:42.540788+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:22:20.372191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515431+0000","last_change":"2023-10-21T17:40:42.544408+0000","last_active":"2023-10-21T17:40:43.515431+0000","last_peered":"2023-10-21T17:40:43.515431+0000","last_clean":"2023-10-21T17:40:43.515431+0000","last_became_active":"2023-10-21T17:40:42.544127+0000","last_became_peered":"2023-10-21T17:40:42.544127+0000","last_unstale":"2023-10-21T17:40:43.515431+0000","last_undegraded":"2023-10-21T17:40:43.515431+0000","last_fullsized":"2023-10-21T17:40:43.515431+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:37:45.734436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.514154+0000","last_change":"2023-10-21T17:40:41.511513+0000","last_active":"2023-10-21T17:40:42.514154+0000","last_peered":"2023-10-21T17:40:42.514154+0000","last_clean":"2023-10-21T17:40:42.514154+0000","last_became_active":"2023-10-21T17:40:41.510807+0000","last_became_peered":"2023-10-21T17:40:41.510807+0000","last_unstale":"2023-10-21T17:40:42.514154+0000","last_undegraded":"2023-10-21T17:40:42.514154+0000","last_fullsized":"2023-10-21T17:40:42.514154+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:45:31.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.540803+0000","last_change":"2023-10-21T17:40:42.539021+0000","last_active":"2023-10-21T17:40:42.540803+0000","last_peered":"2023-10-21T17:40:42.540803+0000","last_clean":"2023-10-21T17:40:42.540803+0000","last_became_active":"2023-10-21T17:40:42.535926+0000","last_became_peered":"2023-10-21T17:40:42.535926+0000","last_unstale":"2023-10-21T17:40:42.540803+0000","last_undegraded":"2023-10-21T17:40:42.540803+0000","last_fullsized":"2023-10-21T17:40:42.540803+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:13:58.385869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":11,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.542011+0000","last_change":"2023-10-21T17:40:41.516234+0000","last_active":"2023-10-21T17:40:42.542011+0000","last_peered":"2023-10-21T17:40:42.542011+0000","last_clean":"2023-10-21T17:40:42.542011+0000","last_became_active":"2023-10-21T17:40:41.514086+0000","last_became_peered":"2023-10-21T17:40:41.514086+0000","last_unstale":"2023-10-21T17:40:42.542011+0000","last_undegraded":"2023-10-21T17:40:42.542011+0000","last_fullsized":"2023-10-21T17:40:42.542011+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T23:06:09.422391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.536966+0000","last_change":"2023-10-21T17:40:42.531150+0000","last_active":"2023-10-21T17:40:42.536966+0000","last_peered":"2023-10-21T17:40:42.536966+0000","last_clean":"2023-10-21T17:40:42.536966+0000","last_became_active":"2023-10-21T17:40:42.530845+0000","last_became_peered":"2023-10-21T17:40:42.530845+0000","last_unstale":"2023-10-21T17:40:42.536966+0000","last_undegraded":"2023-10-21T17:40:42.536966+0000","last_fullsized":"2023-10-21T17:40:42.536966+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:20:56.883561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516645+0000","last_change":"2023-10-21T17:40:41.502880+0000","last_active":"2023-10-21T17:40:43.516645+0000","last_peered":"2023-10-21T17:40:43.516645+0000","last_clean":"2023-10-21T17:40:43.516645+0000","last_became_active":"2023-10-21T17:40:41.500906+0000","last_became_peered":"2023-10-21T17:40:41.500906+0000","last_unstale":"2023-10-21T17:40:43.516645+0000","last_undegraded":"2023-10-21T17:40:43.516645+0000","last_fullsized":"2023-10-21T17:40:43.516645+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T00:38:00.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516213+0000","last_change":"2023-10-21T17:40:42.548094+0000","last_active":"2023-10-21T17:40:43.516213+0000","last_peered":"2023-10-21T17:40:43.516213+0000","last_clean":"2023-10-21T17:40:43.516213+0000","last_became_active":"2023-10-21T17:40:42.534066+0000","last_became_peered":"2023-10-21T17:40:42.534066+0000","last_unstale":"2023-10-21T17:40:43.516213+0000","last_undegraded":"2023-10-21T17:40:43.516213+0000","last_fullsized":"2023-10-21T17:40:43.516213+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T02:05:55.535354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.515856+0000","last_change":"2023-10-21T17:40:41.501333+0000","last_active":"2023-10-21T17:40:43.515856+0000","last_peered":"2023-10-21T17:40:43.515856+0000","last_clean":"2023-10-21T17:40:43.515856+0000","last_became_active":"2023-10-21T17:40:41.498706+0000","last_became_peered":"2023-10-21T17:40:41.498706+0000","last_unstale":"2023-10-21T17:40:43.515856+0000","last_undegraded":"2023-10-21T17:40:43.515856+0000","last_fullsized":"2023-10-21T17:40:43.515856+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T20:17:58.907796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":9,"reported_epoch":274,"state":"active+clean","last_fresh":"2023-10-21T17:40:42.534916+0000","last_change":"2023-10-21T17:40:42.531782+0000","last_active":"2023-10-21T17:40:42.534916+0000","last_peered":"2023-10-21T17:40:42.534916+0000","last_clean":"2023-10-21T17:40:42.534916+0000","last_became_active":"2023-10-21T17:40:42.531516+0000","last_became_peered":"2023-10-21T17:40:42.531516+0000","last_unstale":"2023-10-21T17:40:42.534916+0000","last_undegraded":"2023-10-21T17:40:42.534916+0000","last_fullsized":"2023-10-21T17:40:42.534916+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:04:07.166062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.519237+0000","last_change":"2023-10-21T17:40:41.508269+0000","last_active":"2023-10-21T17:40:43.519237+0000","last_peered":"2023-10-21T17:40:43.519237+0000","last_clean":"2023-10-21T17:40:43.519237+0000","last_became_active":"2023-10-21T17:40:41.505327+0000","last_became_peered":"2023-10-21T17:40:41.505327+0000","last_unstale":"2023-10-21T17:40:43.519237+0000","last_undegraded":"2023-10-21T17:40:43.519237+0000","last_fullsized":"2023-10-21T17:40:43.519237+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T18:41:30.389201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516451+0000","last_change":"2023-10-21T17:40:42.552662+0000","last_active":"2023-10-21T17:40:43.516451+0000","last_peered":"2023-10-21T17:40:43.516451+0000","last_clean":"2023-10-21T17:40:43.516451+0000","last_became_active":"2023-10-21T17:40:42.552112+0000","last_became_peered":"2023-10-21T17:40:42.552112+0000","last_unstale":"2023-10-21T17:40:43.516451+0000","last_undegraded":"2023-10-21T17:40:43.516451+0000","last_fullsized":"2023-10-21T17:40:43.516451+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-23T05:22:54.419654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516084+0000","last_change":"2023-10-21T17:40:41.496477+0000","last_active":"2023-10-21T17:40:43.516084+0000","last_peered":"2023-10-21T17:40:43.516084+0000","last_clean":"2023-10-21T17:40:43.516084+0000","last_became_active":"2023-10-21T17:40:41.495477+0000","last_became_peered":"2023-10-21T17:40:41.495477+0000","last_unstale":"2023-10-21T17:40:43.516084+0000","last_undegraded":"2023-10-21T17:40:43.516084+0000","last_fullsized":"2023-10-21T17:40:43.516084+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":272,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:40.452710+0000","last_clean_scrub_stamp":"2023-10-21T17:40:40.452710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T19:30:38.573814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":11,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-21T17:40:43.516677+0000","last_change":"2023-10-21T17:40:42.538268+0000","last_active":"2023-10-21T17:40:43.516677+0000","last_peered":"2023-10-21T17:40:43.516677+0000","last_clean":"2023-10-21T17:40:43.516677+0000","last_became_active":"2023-10-21T17:40:42.537965+0000","last_became_peered":"2023-10-21T17:40:42.537965+0000","last_unstale":"2023-10-21T17:40:43.516677+0000","last_undegraded":"2023-10-21T17:40:43.516677+0000","last_fullsized":"2023-10-21T17:40:43.516677+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T17:40:41.469248+0000","last_clean_scrub_stamp":"2023-10-21T17:40:41.469248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T17:45:32.249928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"260'4909","reported_seq":5965,"reported_epoch":273,"state":"active+clean","last_fresh":"2023-10-21T17:40:41.506872+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:40:41.506872+0000","last_peered":"2023-10-21T17:40:41.506872+0000","last_clean":"2023-10-21T17:40:41.506872+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:40:41.506872+0000","last_undegraded":"2023-10-21T17:40:41.506872+0000","last_fullsized":"2023-10-21T17:40:41.506872+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4909,"ondisk_log_size":4909,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75420,"kb_used_data":3980,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296420,"statfs":{"total":96636764160,"available":96559534080,"internally_reserved":0,"allocated":4075520,"data_stored":2404647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1779999999999999,"15min":1.198},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.7190000000000001,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.014},{"interface":"front","average":{"1min":1.044,"5min":1.119,"15min":1.161},"min":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1930000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1619999999999999,"15min":1.204},"min":{"1min":0.60599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.056},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.232,"15min":1.2909999999999999},"min":{"1min":0.65000000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.351}]},{"osd":2,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.361,"15min":1.3400000000000001},"min":{"1min":0.72599999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0430000000000001,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.306,"15min":1.3149999999999999},"min":{"1min":0.76600000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.736,"5min":3.556,"15min":3.556},"last":1.6240000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.274,"15min":1.3089999999999999},"min":{"1min":0.78600000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.782,"5min":2.472,"15min":2.5110000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.476,"15min":1.464},"min":{"1min":0.69999999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2280000000000002,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.248}]},{"osd":4,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.391,"15min":1.363},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.669,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.569},{"interface":"front","average":{"1min":1.472,"5min":1.4950000000000001,"15min":1.4930000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3420000000000001,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.2809999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.486,"15min":1.4199999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.052,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.486},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0819999999999999,"5min":2.613,"15min":2.613},"last":1.403}]},{"osd":6,"last update":"Sat Oct 21 17:40:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.599,"15min":1.5549999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.2309999999999999,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5309999999999999,"15min":1.492},"min":{"1min":0.71999999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2850000000000001,"5min":2.879,"15min":2.879},"last":1.6619999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576023,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65092,"kb_used_data":3380,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306748,"statfs":{"total":96636764160,"available":96570109952,"internally_reserved":0,"allocated":3461120,"data_stored":1812015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.113,"15min":1.181},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.8069999999999999,"5min":1.944,"15min":1.944},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.147,"15min":1.2},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.605,"5min":1.6619999999999999,"15min":1.762},"last":1.4490000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1080000000000001,"15min":1.1699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.595,"5min":1.6100000000000001,"15min":1.6299999999999999},"last":1.234},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.226,"15min":1.292},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6779999999999999,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.25,"15min":1.3009999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8300000000000001},"last":1.585},{"interface":"front","average":{"1min":1.256,"5min":1.26,"15min":1.294},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.929},"last":0.98599999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.3220000000000001,"15min":1.3959999999999999},"min":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.9770000000000001,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.391,"15min":1.4390000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.754}]},{"osd":4,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3029999999999999,"15min":1.3380000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.9910000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.423,"15min":1.4330000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.911,"5min":1.911,"15min":2.2149999999999999},"last":1.8460000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.5640000000000001,"15min":1.5669999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.339},"last":1.9450000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3420000000000001,"15min":1.363},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.863,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.8029999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.472,"15min":1.524},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":2.0550000000000002},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.659},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.21,"5min":2.21,"15min":2.5409999999999999},"last":2.0219999999999998}]}]},{"osd":5,"up_from":13,"seq":55834576024,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48472,"kb_used_data":3400,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323368,"statfs":{"total":96636764160,"available":96587128832,"internally_reserved":0,"allocated":3481600,"data_stored":1818807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.089,"15min":1.133},"min":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.361,"5min":1.6319999999999999,"15min":1.9930000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.206},"min":{"1min":0.72999999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.994,"5min":1.994,"15min":2.2909999999999999},"last":0.752}]},{"osd":1,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2789999999999999,"15min":1.274},"min":{"1min":0.93300000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.754,"5min":1.754,"15min":1.8480000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.127,"15min":1.2130000000000001},"min":{"1min":0.82999999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5860000000000001,"5min":1.75,"15min":2.4529999999999998},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.288,"15min":1.2929999999999999},"min":{"1min":0.85099999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2070000000000001,"15min":1.238},"min":{"1min":0.75,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.8009999999999999,"5min":2.266,"15min":2.266},"last":0.872}]},{"osd":3,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3,"15min":1.341},"min":{"1min":0.84399999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.383,"5min":1.361,"15min":1.4019999999999999},"min":{"1min":0.82499999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8300000000000001,"5min":1.843,"15min":1.897},"last":0.83199999999999996}]},{"osd":4,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.2769999999999999,"15min":1.2589999999999999},"min":{"1min":0.90800000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8360000000000001,"5min":1.976,"15min":1.976},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.548,"5min":1.4399999999999999,"15min":1.413},"min":{"1min":0.92900000000000005,"5min":0.80100000000000005,"15min":0.80000000000000004},"max":{"1min":2.0249999999999999,"5min":2.093,"15min":2.093},"last":0.85399999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4710000000000001,"15min":1.4810000000000001},"min":{"1min":1.1819999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":2.3929999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.625,"5min":1.5109999999999999,"15min":1.528},"min":{"1min":1.147,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.9710000000000001,"5min":2.1579999999999999,"15min":2.52},"last":0.92700000000000005}]},{"osd":7,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6899999999999999,"5min":1.526,"15min":1.5389999999999999},"min":{"1min":1.2350000000000001,"5min":0.90300000000000002,"15min":0.58499999999999996},"max":{"1min":2.1019999999999999,"5min":2.1059999999999999,"15min":2.2530000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.6839999999999999,"5min":1.5389999999999999,"15min":1.5680000000000001},"min":{"1min":1.2150000000000001,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.133},"last":0.94199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834576022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52628,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319212,"statfs":{"total":96636764160,"available":96582873088,"internally_reserved":0,"allocated":3477504,"data_stored":1813717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.089,"15min":1.153},"min":{"1min":0.79700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.383,"5min":1.746,"15min":1.8480000000000001},"last":0.872},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.161,"15min":1.2090000000000001},"min":{"1min":0.76500000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5109999999999999,"5min":1.659,"15min":1.869},"last":1.3799999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.58099999999999996},"max":{"1min":1.8380000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.73499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7470000000000001,"5min":1.762,"15min":1.762},"last":1.337}]},{"osd":2,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.224,"15min":1.254},"min":{"1min":0.77000000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3420000000000001,"15min":1.3799999999999999},"min":{"1min":0.88700000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.833,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1279999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.236,"15min":1.3049999999999999},"min":{"1min":0.90300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.734,"5min":1.734,"15min":1.7949999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.389,"15min":1.425},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.085}]},{"osd":5,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.363,"15min":1.3440000000000001},"min":{"1min":0.93500000000000005,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.492},{"interface":"front","average":{"1min":1.371,"5min":1.3480000000000001,"15min":1.3640000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.4239999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.131,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.1600000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.605,"5min":1.5860000000000001,"15min":1.597},"min":{"1min":1.2250000000000001,"5min":1.056,"15min":1.056},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.611}]},{"osd":7,"last update":"Sat Oct 21 17:39:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.5249999999999999,"15min":1.5229999999999999},"min":{"1min":0.94699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.641,"5min":1.587,"15min":1.593},"min":{"1min":1.036,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.1899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56776,"kb_used_data":3388,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315064,"statfs":{"total":96636764160,"available":96578625536,"internally_reserved":0,"allocated":3469312,"data_stored":1812107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95799999999999996,"15min":0.97599999999999998},"min":{"1min":0.48999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5029999999999999,"5min":1.6439999999999999,"15min":1.71},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.165,"15min":1.234},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3640000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.97499999999999998,"15min":1.0600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.7090000000000001,"5min":1.835,"15min":1.835},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.234,"15min":1.3},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.024,"15min":1.1000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.4990000000000001,"5min":1.921,"15min":2.1850000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1759999999999999,"15min":1.2669999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.679,"5min":1.9870000000000001,"15min":2},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.751,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0579999999999998,"5min":2.262,"15min":2.262},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4059999999999999,"15min":1.462},"min":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.401,"15min":1.4850000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.0089999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.546,"5min":1.522,"15min":1.5600000000000001},"min":{"1min":1.02,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0259999999999998,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.871}]},{"osd":6,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5980000000000001,"5min":1.526,"15min":1.589},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.5349999999999999,"15min":1.6100000000000001},"min":{"1min":0.90900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.1110000000000002,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":0.81999999999999995}]},{"osd":7,"last update":"Sat Oct 21 17:40:01 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.6060000000000001,"15min":1.659},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.121,"5min":2.3420000000000001,"15min":2.3460000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.605,"5min":1.609,"15min":1.698},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.2890000000000001,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834576023,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56784,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315056,"statfs":{"total":96636764160,"available":96578617344,"internally_reserved":0,"allocated":3473408,"data_stored":1818751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.90800000000000003},"min":{"1min":0.61899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4470000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.081},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.95299999999999996,"15min":1.022},"min":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.353,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.64900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.79,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.222},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7010000000000001,"5min":1.881,"15min":1.958},"last":1.1279999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.155,"15min":1.1739999999999999},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.321,"5min":1.4319999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.298}]},{"osd":4,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.45,"15min":1.425},"min":{"1min":0.97399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.867,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.0780000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8620000000000001,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.395}]},{"osd":5,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3520000000000001,"15min":1.401},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5920000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.607,"15min":1.6080000000000001},"min":{"1min":1.1319999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9470000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.369}]},{"osd":6,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.512,"15min":1.5509999999999999},"min":{"1min":1.171,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.974,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.333}]},{"osd":7,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.617,"15min":1.6279999999999999},"min":{"1min":1.224,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0750000000000002,"5min":2.1240000000000001,"15min":2.4630000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":1.2190000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.4550000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576022,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52624,"kb_used_data":3384,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319216,"statfs":{"total":96636764160,"available":96582877184,"internally_reserved":0,"allocated":3465216,"data_stored":1812105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.86799999999999999,"15min":0.92100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.976},"last":0.442},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.024,"15min":1.125},"min":{"1min":0.41299999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.6870000000000001,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.70199999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.95299999999999996,"15min":1.0209999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.363,"5min":1.746,"15min":1.79},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.1339999999999999,"15min":1.2110000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.677,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.248,"15min":1.3460000000000001},"min":{"1min":0.48499999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6160000000000001,"5min":2.0539999999999998,"15min":2.129},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.208,"15min":1.268},"min":{"1min":0.501,"5min":0.437,"15min":0.437},"max":{"1min":1.8080000000000001,"5min":1.948,"15min":1.948},"last":0.55200000000000005}]},{"osd":4,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.228,"15min":1.325},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.819,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.347,"5min":1.425,"15min":1.476},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.383},"last":1.1639999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.45,"15min":1.54},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.98,"5min":2.2749999999999999,"15min":2.4750000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.4239999999999999,"15min":1.5049999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0539999999999998,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.97199999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4970000000000001,"15min":1.573},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.895,"5min":2.4319999999999999,"15min":2.4319999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.4159999999999999,"15min":1.5209999999999999},"min":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.931,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.042}]},{"osd":7,"last update":"Sat Oct 21 17:40:03 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.5740000000000001,"15min":1.641},"min":{"1min":0.75,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":2.1829999999999998,"15min":2.548},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.351,"5min":1.6120000000000001,"15min":1.6870000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.9319999999999999,"5min":2.3730000000000002,"15min":2.548},"last":1.129}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53196,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318644,"statfs":{"total":96636764160,"available":96582291456,"internally_reserved":0,"allocated":4063232,"data_stored":2402999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":0.95999999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.075},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.3080000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5309999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.3740000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.421}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.804}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:40:47.612 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-21T17:40:47.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-21T17:40:47.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.889+0000 7f6270890640 1 Processor -- start 2023-10-21T17:40:47.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.890+0000 7f6270890640 1 -- start start 2023-10-21T17:40:47.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.891+0000 7f6270890640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f626c06cb20 0x7f626c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.891+0000 7f6270890640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f626c06a290 0x7f626c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.891+0000 7f6270890640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626c06aa40 con 0x7f626c161630 2023-10-21T17:40:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.891+0000 7f6270890640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c06ad20 con 0x7f626c06a290 2023-10-21T17:40:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.891+0000 7f6270890640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626c06b000 con 0x7f626c06cb20 2023-10-21T17:40:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f626c06a290 0x7f626c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f626c06a290 0x7f626c0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35002/0 (socket says 172.21.15.53:35002) 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 -- 172.21.15.53:0/3352870418 learned_addr learned my addr 172.21.15.53:0/3352870418 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f6269d9b640 1 --2- 172.21.15.53:0/3352870418 >> v2:172.21.15.124:3300/0 conn(0x7f626c06cb20 0x7f626c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 -- 172.21.15.53:0/3352870418 >> v2:172.21.15.124:3300/0 conn(0x7f626c06cb20 msgr2=0x7f626c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 --2- 172.21.15.53:0/3352870418 >> v2:172.21.15.124:3300/0 conn(0x7f626c06cb20 0x7f626c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 -- 172.21.15.53:0/3352870418 >> v1:172.21.15.124:6789/0 conn(0x7f626c161630 legacy=0x7f626c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:47.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.892+0000 7f626ad9d640 1 -- 172.21.15.53:0/3352870418 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c06b2e0 con 0x7f626c06a290 2023-10-21T17:40:47.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.893+0000 7f626ad9d640 1 --2- 172.21.15.53:0/3352870418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06a290 0x7f626c0cd270 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f626c06eb20 tx=0x7f626402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d623230be12fb8d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.894+0000 7f626959a640 1 -- 172.21.15.53:0/3352870418 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626402c9f0 con 0x7f626c06a290 2023-10-21T17:40:47.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.894+0000 7f626959a640 1 -- 172.21.15.53:0/3352870418 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626402cb50 con 0x7f626c06a290 2023-10-21T17:40:47.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.894+0000 7f626959a640 1 -- 172.21.15.53:0/3352870418 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640343f0 con 0x7f626c06a290 2023-10-21T17:40:47.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.894+0000 7f6270890640 1 -- 172.21.15.53:0/3352870418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06a290 msgr2=0x7f626c0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.894+0000 7f6270890640 1 --2- 172.21.15.53:0/3352870418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06a290 0x7f626c0cd270 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f626c06eb20 tx=0x7f626402be50 comp rx=0 tx=0).stop 2023-10-21T17:40:47.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 -- 172.21.15.53:0/3352870418 shutdown_connections 2023-10-21T17:40:47.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 --2- 172.21.15.53:0/3352870418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06a290 0x7f626c0cd270 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 --2- 172.21.15.53:0/3352870418 >> v2:172.21.15.124:3300/0 conn(0x7f626c06cb20 0x7f626c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.898 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 -- 172.21.15.53:0/3352870418 >> 172.21.15.53:0/3352870418 conn(0x7f626c0b0b80 msgr2=0x7f626c0b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:47.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 -- 172.21.15.53:0/3352870418 shutdown_connections 2023-10-21T17:40:47.899 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.895+0000 7f6270890640 1 -- 172.21.15.53:0/3352870418 wait complete. 2023-10-21T17:40:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.897+0000 7f6270890640 1 Processor -- start 2023-10-21T17:40:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.897+0000 7f6270890640 1 -- start start 2023-10-21T17:40:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 0x7f626c0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f626c161630 0x7f626c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1f3260 con 0x7f626c06cb20 2023-10-21T17:40:47.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1f3540 con 0x7f626c161630 2023-10-21T17:40:47.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6270890640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1f3820 con 0x7f626c06a290 2023-10-21T17:40:47.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6269d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6269d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35018/0 (socket says 172.21.15.53:35018) 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f6269d9b640 1 -- 172.21.15.53:0/4208799709 learned_addr learned my addr 172.21.15.53:0/4208799709 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f626ad9d640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f626c161630 0x7f626c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.898+0000 7f626a59c640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 0x7f626c0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 -- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 msgr2=0x7f626c0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 0x7f626c0ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 -- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f626c161630 msgr2=0x7f626c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f626c161630 0x7f626c0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 -- 172.21.15.53:0/4208799709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264009b30 con 0x7f626c06cb20 2023-10-21T17:40:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f626a59c640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 0x7f626c0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6269d9b640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f626000d5d0 tx=0x7f626000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600125f0 con 0x7f626c06cb20 2023-10-21T17:40:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6260012750 con 0x7f626c06cb20 2023-10-21T17:40:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260012ae0 con 0x7f626c06cb20 2023-10-21T17:40:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c1f3b00 con 0x7f626c06cb20 2023-10-21T17:40:47.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.899+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c0c6720 con 0x7f626c06cb20 2023-10-21T17:40:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.901+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c06d420 con 0x7f626c06cb20 2023-10-21T17:40:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.902+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6260012c40 con 0x7f626c06cb20 2023-10-21T17:40:47.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.902+0000 7f625affd640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 0x7f624c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.903+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f626008a070 con 0x7f626c06cb20 2023-10-21T17:40:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.903+0000 7f626a59c640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 0x7f624c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:47.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.904+0000 7f626a59c640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 0x7f624c06d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f625c00a810 tx=0x7f625c008040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:47.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:47.907+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6260054c10 con 0x7f626c06cb20 2023-10-21T17:40:48.172 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.171+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f626c0c68e0 con 0x7f626c06cb20 2023-10-21T17:40:48.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.176+0000 7f625affd640 1 -- 172.21.15.53:0/4208799709 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v54) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6260058230 con 0x7f626c06cb20 2023-10-21T17:40:48.178 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2023-10-21T17:40:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQCz9TNlViucFhAAH8uurOUsX09lJZtUHRzzYA== 2023-10-21T17:40:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: caps mds = "allow" 2023-10-21T17:40:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: caps mon = "allow r" 2023-10-21T17:40:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-21T17:40:48.179 INFO:teuthology.orchestra.run.smithi053.stderr:updated caps for client.0 2023-10-21T17:40:48.180 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 msgr2=0x7f624c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 0x7f624c06d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f625c00a810 tx=0x7f625c008040 comp rx=0 tx=0).stop 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 msgr2=0x7f626c1f27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f626000d5d0 tx=0x7f626000da90 comp rx=0 tx=0).stop 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 shutdown_connections 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f624c06b0b0 0x7f624c06d570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f626c161630 0x7f626c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f626c06cb20 0x7f626c1f27a0 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 --2- 172.21.15.53:0/4208799709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f626c06a290 0x7f626c0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 >> 172.21.15.53:0/4208799709 conn(0x7f626c0b0b80 msgr2=0x7f626c06c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:48.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 shutdown_connections 2023-10-21T17:40:48.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.179+0000 7f6270890640 1 -- 172.21.15.53:0/4208799709 wait complete. 2023-10-21T17:40:48.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:40:48.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.413+0000 7f87fe002640 1 Processor -- start 2023-10-21T17:40:48.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.414+0000 7f87fe002640 1 -- start start 2023-10-21T17:40:48.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.415+0000 7f87fe002640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87f80add60 0x7f87f8162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:48.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87fe002640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f80ae140 con 0x7f87f806d4c0 2023-10-21T17:40:48.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87fe002640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f80ccf20 con 0x7f87f80cb150 2023-10-21T17:40:48.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87fe002640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f8161f10 con 0x7f87f80add60 2023-10-21T17:40:48.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87f6ffd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f87f806d4c0 0x7f87f80cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46856/0 (socket says 172.21.15.53:46856) 2023-10-21T17:40:48.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87f6ffd640 1 -- 172.21.15.53:0/4256073188 learned_addr learned my addr 172.21.15.53:0/4256073188 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:48.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.416+0000 7f87f77fe640 1 --2- 172.21.15.53:0/4256073188 >> v2:172.21.15.53:3300/0 conn(0x7f87f80add60 0x7f87f8162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:48.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.417+0000 7f87f77fe640 1 -- 172.21.15.53:0/4256073188 >> v1:172.21.15.124:6789/0 conn(0x7f87f80cb150 legacy=0x7f87f8164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.417+0000 7f87f67fc640 1 -- 172.21.15.53:0/4256073188 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774886967 0 0) 0x7f87f80ae140 con 0x7f87f806d4c0 2023-10-21T17:40:48.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.417+0000 7f87f77fe640 1 -- 172.21.15.53:0/4256073188 >> v1:172.21.15.53:6789/0 conn(0x7f87f806d4c0 legacy=0x7f87f80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.417+0000 7f87f77fe640 1 -- 172.21.15.53:0/4256073188 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80cd1c0 con 0x7f87f80add60 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.418+0000 7f87f77fe640 1 --2- 172.21.15.53:0/4256073188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80add60 0x7f87f8162330 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa3e06b01380f39 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87f67fc640 1 -- 172.21.15.53:0/4256073188 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e802cc60 con 0x7f87f80add60 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87f67fc640 1 -- 172.21.15.53:0/4256073188 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e802cdc0 con 0x7f87f80add60 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87f67fc640 1 -- 172.21.15.53:0/4256073188 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8034660 con 0x7f87f80add60 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87fe002640 1 -- 172.21.15.53:0/4256073188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80add60 msgr2=0x7f87f8162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87fe002640 1 --2- 172.21.15.53:0/4256073188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80add60 0x7f87f8162330 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009d90 tx=0x7f87e802beb0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87fe002640 1 -- 172.21.15.53:0/4256073188 shutdown_connections 2023-10-21T17:40:48.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87fe002640 1 --2- 172.21.15.53:0/4256073188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80add60 0x7f87f8162330 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.419+0000 7f87fe002640 1 -- 172.21.15.53:0/4256073188 >> 172.21.15.53:0/4256073188 conn(0x7f87f806a380 msgr2=0x7f87f80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:48.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.420+0000 7f87fe002640 1 -- 172.21.15.53:0/4256073188 shutdown_connections 2023-10-21T17:40:48.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.420+0000 7f87fe002640 1 -- 172.21.15.53:0/4256073188 wait complete. 2023-10-21T17:40:48.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.421+0000 7f87fe002640 1 Processor -- start 2023-10-21T17:40:48.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.422+0000 7f87fe002640 1 -- start start 2023-10-21T17:40:48.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87f806d4c0 0x7f87f81576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:48.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87f80add60 0x7f87f815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:48.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:48.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f815d520 con 0x7f87f80cb150 2023-10-21T17:40:48.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f815d780 con 0x7f87f806d4c0 2023-10-21T17:40:48.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87fe002640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f815da60 con 0x7f87f80add60 2023-10-21T17:40:48.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:48.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35038/0 (socket says 172.21.15.53:35038) 2023-10-21T17:40:48.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87f7fff640 1 -- 172.21.15.53:0/3182486890 learned_addr learned my addr 172.21.15.53:0/3182486890 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:48.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.423+0000 7f87f77fe640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87f806d4c0 0x7f87f81576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:48.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 -- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87f80add60 msgr2=0x7f87f815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f6ffd640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87f80add60 0x7f87f815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87f80add60 0x7f87f815eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 -- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87f806d4c0 msgr2=0x7f87f81576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87f806d4c0 0x7f87f81576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 -- 172.21.15.53:0/3182486890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8009a70 con 0x7f87f80cb150 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87f7fff640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f87f0001eb0 tx=0x7f87f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00052d0 con 0x7f87f80cb150 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0005430 con 0x7f87f80cb150 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0003810 con 0x7f87f80cb150 2023-10-21T17:40:48.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.424+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f815dd40 con 0x7f87f80cb150 2023-10-21T17:40:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.425+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f80b1480 con 0x7f87f80cb150 2023-10-21T17:40:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.426+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f8067490 con 0x7f87f80cb150 2023-10-21T17:40:48.430 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.427+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f87f0003970 con 0x7f87f80cb150 2023-10-21T17:40:48.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.428+0000 7f87effff640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 0x7f87d806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:48.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.428+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f87f0089080 con 0x7f87f80cb150 2023-10-21T17:40:48.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.428+0000 7f87f77fe640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 0x7f87d806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:48.431 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.430+0000 7f87f77fe640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 0x7f87d806d5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87e8036000 tx=0x7f87e8030000 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:48.434 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.433+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87f0053dc0 con 0x7f87f80cb150 2023-10-21T17:40:48.666 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.665+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87f80ad740 con 0x7f87f80cb150 2023-10-21T17:40:48.667 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.666+0000 7f87effff640 1 -- 172.21.15.53:0/3182486890 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f87f00573e0 con 0x7f87f80cb150 2023-10-21T17:40:48.667 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:48.667 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:40:48.668 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:40:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 msgr2=0x7f87d806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 0x7f87d806d5c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f87e8036000 tx=0x7f87e8030000 comp rx=0 tx=0).stop 2023-10-21T17:40:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 msgr2=0x7f87f815cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f87f0001eb0 tx=0x7f87f000c890 comp rx=0 tx=0).stop 2023-10-21T17:40:48.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 shutdown_connections 2023-10-21T17:40:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f87d806b100 0x7f87d806d5c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87f80cb150 0x7f87f815cfe0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.668+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87f80add60 0x7f87f815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.669+0000 7f87fe002640 1 --2- 172.21.15.53:0/3182486890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87f806d4c0 0x7f87f81576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:48.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.669+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 >> 172.21.15.53:0/3182486890 conn(0x7f87f806a380 msgr2=0x7f87f8163740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.669+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 shutdown_connections 2023-10-21T17:40:48.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:48.669+0000 7f87fe002640 1 -- 172.21.15.53:0/3182486890 wait complete. 2023-10-21T17:40:49.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-21T17:40:41.958072+0000', 'modified': '2023-10-21T17:40:45.062063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 38723}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_38723': {'gid': 38723, 'name': 'b', 'rank': 0, 'incarnation': 140, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/2127148028', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6835', 'nonce': 2127148028}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 2127148028}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [36], 'metadata_pool': 35, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-21T17:40:49.090 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-21T17:40:49.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-21T17:40:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.322+0000 7f281f056640 1 Processor -- start 2023-10-21T17:40:49.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.322+0000 7f281f056640 1 -- start start 2023-10-21T17:40:49.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.323+0000 7f281f056640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f28180cb600 0x7f28180b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.323+0000 7f281f056640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281806e1e0 con 0x7f2818161690 2023-10-21T17:40:49.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.323+0000 7f281f056640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28180cba50 con 0x7f28180cb150 2023-10-21T17:40:49.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.323+0000 7f281f056640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281806cc50 con 0x7f28180cb600 2023-10-21T17:40:49.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.324+0000 7f281cdf2640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2818161690 0x7f281806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46870/0 (socket says 172.21.15.53:46870) 2023-10-21T17:40:49.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.324+0000 7f281cdf2640 1 -- 172.21.15.53:0/134421657 learned_addr learned my addr 172.21.15.53:0/134421657 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.324+0000 7f281d5f3640 1 --2- 172.21.15.53:0/134421657 >> v2:172.21.15.124:3300/0 conn(0x7f28180cb600 0x7f28180b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.324+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022931533 0 0) 0x7f281806e1e0 con 0x7f2818161690 2023-10-21T17:40:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.324+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003520 con 0x7f2818161690 2023-10-21T17:40:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096990244 0 0) 0x7f2800003520 con 0x7f2818161690 2023-10-21T17:40:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 >> v2:172.21.15.124:3300/0 conn(0x7f28180cb600 msgr2=0x7f28180b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 --2- 172.21.15.53:0/134421657 >> v2:172.21.15.124:3300/0 conn(0x7f28180cb600 0x7f28180b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 >> v1:172.21.15.124:6789/0 conn(0x7f28180cb150 legacy=0x7f28180b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281806cf30 con 0x7f2818161690 2023-10-21T17:40:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f2808001a10 con 0x7f2818161690 2023-10-21T17:40:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 >> v1:172.21.15.53:6789/0 conn(0x7f2818161690 legacy=0x7f281806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28000039c0 0x7f2800003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28000047c0 0x7f2800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 0x7f280000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281806cc50 con 0x7f2800005180 2023-10-21T17:40:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280000c080 con 0x7f28000039c0 2023-10-21T17:40:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280000c240 con 0x7f28000047c0 2023-10-21T17:40:49.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f281d5f3640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 0x7f280000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f281d5f3640 1 -- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28000047c0 msgr2=0x7f2800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.325+0000 7f2817fff640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28000047c0 0x7f2800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281cdf2640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28000039c0 0x7f2800003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281d5f3640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28000047c0 0x7f2800004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281d5f3640 1 -- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28000039c0 msgr2=0x7f2800003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281d5f3640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28000039c0 0x7f2800003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281d5f3640 1 -- 172.21.15.53:0/134421657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800003520 con 0x7f2800005180 2023-10-21T17:40:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f2817fff640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28000047c0 0x7f2800004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.326+0000 7f281cdf2640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28000039c0 0x7f2800003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:49.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.327+0000 7f281d5f3640 1 -- 172.21.15.53:0/134421657 reap_dead start 2023-10-21T17:40:49.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.327+0000 7f281d5f3640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 0x7f280000b980 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ac0 tx=0x7f28000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a42cbeb85cc13e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.327+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810004020 con 0x7f2800005180 2023-10-21T17:40:49.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.327+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810004180 con 0x7f2800005180 2023-10-21T17:40:49.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.327+0000 7f28177fe640 1 -- 172.21.15.53:0/134421657 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281002c710 con 0x7f2800005180 2023-10-21T17:40:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 -- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 msgr2=0x7f280000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 0x7f280000b980 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f2810009ac0 tx=0x7f28000057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 -- 172.21.15.53:0/134421657 shutdown_connections 2023-10-21T17:40:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 --2- 172.21.15.53:0/134421657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2800005180 0x7f280000b980 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 -- 172.21.15.53:0/134421657 >> 172.21.15.53:0/134421657 conn(0x7f281806a5d0 msgr2=0x7f28180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 -- 172.21.15.53:0/134421657 shutdown_connections 2023-10-21T17:40:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.328+0000 7f281f056640 1 -- 172.21.15.53:0/134421657 wait complete. 2023-10-21T17:40:49.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.330+0000 7f281f056640 1 Processor -- start 2023-10-21T17:40:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.330+0000 7f281f056640 1 -- start start 2023-10-21T17:40:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2800005180 0x7f2818162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28180c4af0 0x7f28180c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28181638e0 con 0x7f28180c20e0 2023-10-21T17:40:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28180c2aa0 con 0x7f28180c4af0 2023-10-21T17:40:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f281f056640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28181f6b90 con 0x7f2800005180 2023-10-21T17:40:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35056/0 (socket says 172.21.15.53:35056) 2023-10-21T17:40:49.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 -- 172.21.15.53:0/1914314228 learned_addr learned my addr 172.21.15.53:0/1914314228 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 -- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2800005180 msgr2=0x7f2818162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2800005180 0x7f2818162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 -- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28180c4af0 msgr2=0x7f28180c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28180c4af0 0x7f28180c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.331+0000 7f2817fff640 1 -- 172.21.15.53:0/1914314228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2818162c60 con 0x7f28180c20e0 2023-10-21T17:40:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f2817fff640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f280c00d7d0 tx=0x7f280c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f28157fa640 1 -- 172.21.15.53:0/1914314228 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c017070 con 0x7f28180c20e0 2023-10-21T17:40:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f28157fa640 1 -- 172.21.15.53:0/1914314228 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c005070 con 0x7f28180c20e0 2023-10-21T17:40:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f28157fa640 1 -- 172.21.15.53:0/1914314228 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c005440 con 0x7f28180c20e0 2023-10-21T17:40:49.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2818162f40 con 0x7f28180c20e0 2023-10-21T17:40:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.332+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2818163500 con 0x7f28180c20e0 2023-10-21T17:40:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.335+0000 7f28157fa640 1 -- 172.21.15.53:0/1914314228 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f280c007030 con 0x7f28180c20e0 2023-10-21T17:40:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.336+0000 7f28157fa640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 0x7f280406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.336+0000 7f28157fa640 1 -- 172.21.15.53:0/1914314228 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f280c0895c0 con 0x7f28180c20e0 2023-10-21T17:40:49.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.336+0000 7f2821beb740 1 Processor -- start 2023-10-21T17:40:49.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.336+0000 7f2821beb740 1 -- start start 2023-10-21T17:40:49.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.336+0000 7f281cdf2640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 0x7f280406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b725f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b724e27a0 con 0x564b725c23c0 2023-10-21T17:40:49.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b72534740 con 0x564b725ef360 2023-10-21T17:40:49.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b72531e30 con 0x564b725eef80 2023-10-21T17:40:49.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.337+0000 7f281d5f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35072/0 (socket says 172.21.15.53:35072) 2023-10-21T17:40:49.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 -- 172.21.15.53:0/2756342419 learned_addr learned my addr 172.21.15.53:0/2756342419 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f2817fff640 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b725f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281cdf2640 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 -- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 msgr2=0x564b725c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 -- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 msgr2=0x564b725f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b725f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 -- 172.21.15.53:0/2756342419 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b724f1910 con 0x564b725c23c0 2023-10-21T17:40:49.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281cdf2640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 0x7f280406d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2808000f80 tx=0x7f2808009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.338+0000 7f281d5f3640 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f2810009bb0 tx=0x7f281002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f624eed80ce9610e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f27fe7fc640 1 -- 172.21.15.53:0/2756342419 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281003a070 con 0x564b725c23c0 2023-10-21T17:40:49.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f27fe7fc640 1 -- 172.21.15.53:0/2756342419 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281000f000 con 0x564b725c23c0 2023-10-21T17:40:49.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f27fe7fc640 1 -- 172.21.15.53:0/2756342419 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281002c710 con 0x564b725c23c0 2023-10-21T17:40:49.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 -- 172.21.15.53:0/2756342419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 msgr2=0x564b725f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f2810009bb0 tx=0x7f281002e070 comp rx=0 tx=0).stop 2023-10-21T17:40:49.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 -- 172.21.15.53:0/2756342419 shutdown_connections 2023-10-21T17:40:49.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725c23c0 0x564b725f2250 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b725f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 --2- 172.21.15.53:0/2756342419 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 -- 172.21.15.53:0/2756342419 >> 172.21.15.53:0/2756342419 conn(0x564b725c4190 msgr2=0x564b7260cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.339+0000 7f2821beb740 1 -- 172.21.15.53:0/2756342419 shutdown_connections 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.340+0000 7f2821beb740 1 -- 172.21.15.53:0/2756342419 wait complete. 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.340+0000 7f2821beb740 1 -- start start 2023-10-21T17:40:49.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.340+0000 7f2821beb740 1 Processor -- start 2023-10-21T17:40:49.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b725dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7252d720 con 0x564b725dd0a0 2023-10-21T17:40:49.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b724cb970 con 0x564b725eef80 2023-10-21T17:40:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.341+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b724cb610 con 0x564b725ef360 2023-10-21T17:40:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f2821beb740 10 client.0 _ll_register_callbacks cb 0x564b724ed9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:40:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35078/0 (socket says 172.21.15.53:35078) 2023-10-21T17:40:49.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 -- 172.21.15.53:0/1167135248 learned_addr learned my addr 172.21.15.53:0/1167135248 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f2817fff640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281cdf2640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b725dfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 -- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 msgr2=0x564b725e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725e0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 -- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 msgr2=0x564b725dfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b725dfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f281d5f3640 1 -- 172.21.15.53:0/1167135248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2800003520 con 0x564b725dd0a0 2023-10-21T17:40:49.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.342+0000 7f2817fff640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:49.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f281d5f3640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f28100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41478b0b7ad61bbc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f27f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:40:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281002ce80 con 0x564b725dd0a0 2023-10-21T17:40:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b72554160 con 0x564b725dd0a0 2023-10-21T17:40:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2810003610 con 0x564b725dd0a0 2023-10-21T17:40:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28100037d0 con 0x564b725dd0a0 2023-10-21T17:40:49.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f2821beb740 10 client.12849 fetch_fsmap learned FSMap version 143 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x564b7252bda0 con 0x564b725dd0a0 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.343+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b72554160 con 0x564b725dd0a0 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2810003930 con 0x564b725dd0a0 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 10 client.12849 fetch_fsmap finished waiting for FSMap version 143 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 10 client.12849 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T17:40:49.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 20 client.12849 populate_metadata read hostname 'smithi053' 2023-10-21T17:40:49.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b726104f0 0x564b725f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 4 client.12849 mds_command: new command op to 38723 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-21T17:40:49.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 --> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b724ffcc0 con 0x564b726104f0 2023-10-21T17:40:49.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.344+0000 7f281cdf2640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b726104f0 0x564b725f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.345+0000 7f281cdf2640 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b726104f0 0x564b725f3d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0d853c4518d745 server_cookie=bd209403d72ba458 in_seq=0 out_seq=0 2023-10-21T17:40:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.345+0000 7f27f77fe640 10 client.12849 ms_handle_connect on v2:172.21.15.124:6835/2127148028 2023-10-21T17:40:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f27f77fe640 1 -- 172.21.15.53:0/1167135248 <== mds.0 v2:172.21.15.124:6835/2127148028 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b724ffcc0 con 0x564b726104f0 2023-10-21T17:40:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f27f77fe640 10 client.12849 handle_command_reply: tid=0 2023-10-21T17:40:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 client.12849 shutdown 2023-10-21T17:40:49.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 msgr2=0x564b725de870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f28100046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 shutdown_connections 2023-10-21T17:40:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725de870 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b726104f0 0x564b725f3d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 --2- 172.21.15.53:0/1167135248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b725dfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 >> 172.21.15.53:0/1167135248 conn(0x564b725c23c0 msgr2=0x564b725f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f27f77fe640 0 client.12849 ms_handle_reset on v2:172.21.15.124:6835/2127148028 2023-10-21T17:40:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 shutdown_connections 2023-10-21T17:40:49.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.347+0000 7f2821beb740 1 -- 172.21.15.53:0/1167135248 wait complete. 2023-10-21T17:40:49.366 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.348+0000 7f2821beb740 20 client.12849 trim_cache size 0 max 16384 2023-10-21T17:40:49.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.364+0000 7f2821beb740 1 Processor -- start 2023-10-21T17:40:49.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.364+0000 7f2821beb740 1 -- start start 2023-10-21T17:40:49.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b7260d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b72291f20 con 0x564b725dd0a0 2023-10-21T17:40:49.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b725c3850 con 0x564b725ef360 2023-10-21T17:40:49.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b725df600 con 0x564b725eef80 2023-10-21T17:40:49.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f281cdf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f281cdf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35080/0 (socket says 172.21.15.53:35080) 2023-10-21T17:40:49.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.365+0000 7f281cdf2640 1 -- 172.21.15.53:0/3337547774 learned_addr learned my addr 172.21.15.53:0/3337547774 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 -- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 msgr2=0x564b725c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 -- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 msgr2=0x564b7260d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b7260d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 -- 172.21.15.53:0/3337547774 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b724cab80 con 0x564b725dd0a0 2023-10-21T17:40:49.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f281cdf2640 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f280800dfe0 tx=0x7f2808013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84db3846952cabb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f27feffd640 1 -- 172.21.15.53:0/3337547774 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808023dd0 con 0x564b725dd0a0 2023-10-21T17:40:49.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f27feffd640 1 -- 172.21.15.53:0/3337547774 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808023040 con 0x564b725dd0a0 2023-10-21T17:40:49.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.366+0000 7f27feffd640 1 -- 172.21.15.53:0/3337547774 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28080233d0 con 0x564b725dd0a0 2023-10-21T17:40:49.375 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 -- 172.21.15.53:0/3337547774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 msgr2=0x564b725df0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f280800dfe0 tx=0x7f2808013000 comp rx=0 tx=0).stop 2023-10-21T17:40:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 -- 172.21.15.53:0/3337547774 shutdown_connections 2023-10-21T17:40:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725ef360 0x564b7260d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725eef80 0x564b725c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.376 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 --2- 172.21.15.53:0/3337547774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b725df0c0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 -- 172.21.15.53:0/3337547774 >> 172.21.15.53:0/3337547774 conn(0x564b725c23c0 msgr2=0x564b725c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 -- 172.21.15.53:0/3337547774 shutdown_connections 2023-10-21T17:40:49.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.367+0000 7f2821beb740 1 -- 172.21.15.53:0/3337547774 wait complete. 2023-10-21T17:40:49.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.368+0000 7f2821beb740 1 -- start start 2023-10-21T17:40:49.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.368+0000 7f2821beb740 1 Processor -- start 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b7260d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7265b980 con 0x564b725dd0a0 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b725e07c0 con 0x564b725eef80 2023-10-21T17:40:49.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b7260dc60 con 0x564b725ef360 2023-10-21T17:40:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f2821beb740 10 client.0 _ll_register_callbacks cb 0x7f2810009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-21T17:40:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f281cdf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f281cdf2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35084/0 (socket says 172.21.15.53:35084) 2023-10-21T17:40:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.369+0000 7f281cdf2640 1 -- 172.21.15.53:0/101623384 learned_addr learned my addr 172.21.15.53:0/101623384 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281d5f3640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f2817fff640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b7260d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 -- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 msgr2=0x564b725dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 -- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 msgr2=0x564b7260d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.380 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b7260d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 -- 172.21.15.53:0/101623384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281806cf30 con 0x564b725dd0a0 2023-10-21T17:40:49.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f2817fff640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b7260d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:49.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f281cdf2640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f2808001b40 tx=0x7f2808010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56df77cd76e8299c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f27f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-10-21T17:40:49.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808033070 con 0x564b725dd0a0 2023-10-21T17:40:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mon.0 v2:172.21.15.53:3300/0 2 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f280801b040 con 0x564b725dd0a0 2023-10-21T17:40:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.370+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f281000f000 con 0x564b725dd0a0 2023-10-21T17:40:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=143) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2808023580 con 0x564b725dd0a0 2023-10-21T17:40:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 10 client.12855 fetch_fsmap learned FSMap version 143 2023-10-21T17:40:49.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=143,monmap=2+}) v3 -- 0x564b724f0170 con 0x564b725dd0a0 2023-10-21T17:40:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mon.0 v2:172.21.15.53:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281000f000 con 0x564b725dd0a0 2023-10-21T17:40:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mon.0 v2:172.21.15.53:3300/0 5 ==== fsmap(e 143) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2808013470 con 0x564b725dd0a0 2023-10-21T17:40:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 10 client.12855 fetch_fsmap finished waiting for FSMap version 143 2023-10-21T17:40:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 10 client.12855 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-21T17:40:49.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 20 client.12855 populate_metadata read hostname 'smithi053' 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b725c6d20 0x564b725c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 4 client.12855 mds_command: new command op to 38723 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.371+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 --> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b72526700 con 0x564b725c6d20 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.372+0000 7f2817fff640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b725c6d20 0x564b725c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.372+0000 7f2817fff640 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b725c6d20 0x564b725c7100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92def2768ab12a5 server_cookie=14b2adc66a9325c3 in_seq=0 out_seq=0 2023-10-21T17:40:49.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.372+0000 7f27f77fe640 10 client.12855 ms_handle_connect on v2:172.21.15.124:6835/2127148028 2023-10-21T17:40:49.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.373+0000 7f27f77fe640 1 -- 172.21.15.53:0/101623384 <== mds.0 v2:172.21.15.124:6835/2127148028 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x564b72526700 con 0x564b725c6d20 2023-10-21T17:40:49.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.373+0000 7f27f77fe640 10 client.12855 handle_command_reply: tid=0 2023-10-21T17:40:49.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.373+0000 7f2821beb740 1 client.12855 shutdown 2023-10-21T17:40:49.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.373+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 msgr2=0x564b72657d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.373+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f2808001b40 tx=0x7f2808010000 comp rx=0 tx=0).stop 2023-10-21T17:40:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 shutdown_connections 2023-10-21T17:40:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564b725ef360 0x564b725dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:6835/2127148028,v1:172.21.15.124:6837/2127148028] conn(0x564b725c6d20 0x564b725c7100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564b725eef80 0x564b7260d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 --2- 172.21.15.53:0/101623384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b725dd0a0 0x564b72657d80 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f27f77fe640 0 client.12855 ms_handle_reset on v2:172.21.15.124:6835/2127148028 2023-10-21T17:40:49.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 >> 172.21.15.53:0/101623384 conn(0x564b725c23c0 msgr2=0x564b725c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 shutdown_connections 2023-10-21T17:40:49.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 1 -- 172.21.15.53:0/101623384 wait complete. 2023-10-21T17:40:49.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f2821beb740 20 client.12855 trim_cache size 0 max 16384 2023-10-21T17:40:49.388 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:40:49.388 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_fsid": "d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56", 2023-10-21T17:40:49.388 INFO:teuthology.orchestra.run.smithi053.stdout: "whoami": 0, 2023-10-21T17:40:49.388 INFO:teuthology.orchestra.run.smithi053.stdout: "id": 38723, 2023-10-21T17:40:49.388 INFO:teuthology.orchestra.run.smithi053.stdout: "want_state": "up:active", 2023-10-21T17:40:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: "state": "up:active", 2023-10-21T17:40:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: "fs_name": "cephfs", 2023-10-21T17:40:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: "rank_uptime": 6.8630567080000002, 2023-10-21T17:40:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: "mdsmap_epoch": 143, 2023-10-21T17:40:49.389 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch": 275, 2023-10-21T17:40:49.390 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_epoch_barrier": 275, 2023-10-21T17:40:49.390 INFO:teuthology.orchestra.run.smithi053.stdout: "uptime": 13.242387730000001 2023-10-21T17:40:49.390 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:40:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 msgr2=0x7f280406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 0x7f280406d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2808000f80 tx=0x7f2808009040 comp rx=0 tx=0).stop 2023-10-21T17:40:49.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 msgr2=0x7f28180c2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.374+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f280c00d7d0 tx=0x7f280c00dc90 comp rx=0 tx=0).stop 2023-10-21T17:40:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 shutdown_connections 2023-10-21T17:40:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f280406b0b0 0x7f280406d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28180c4af0 0x7f28180c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28180c20e0 0x7f28180c2560 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 --2- 172.21.15.53:0/1914314228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2800005180 0x7f2818162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 >> 172.21.15.53:0/1914314228 conn(0x7f281806a5d0 msgr2=0x7f28180b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 shutdown_connections 2023-10-21T17:40:49.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.375+0000 7f281f056640 1 -- 172.21.15.53:0/1914314228 wait complete. 2023-10-21T17:40:49.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-21T17:40:49.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.615+0000 7f62e359e640 1 Processor -- start 2023-10-21T17:40:49.616 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.615+0000 7f62e359e640 1 -- start start 2023-10-21T17:40:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62e359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f62e40ca620 0x7f62e40b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62e359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e40cade0 con 0x7f62e40cbb60 2023-10-21T17:40:49.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62e359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e40addf0 con 0x7f62e40ada10 2023-10-21T17:40:49.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62e359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e406c270 con 0x7f62e40ca620 2023-10-21T17:40:49.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62dbfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f62e40cbb60 0x7f62e415d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46876/0 (socket says 172.21.15.53:46876) 2023-10-21T17:40:49.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62dbfff640 1 -- 172.21.15.53:0/889163075 learned_addr learned my addr 172.21.15.53:0/889163075 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.617+0000 7f62e2d9d640 1 --2- 172.21.15.53:0/889163075 >> v2:172.21.15.124:3300/0 conn(0x7f62e40ca620 0x7f62e40b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391288142 0 0) 0x7f62e40cade0 con 0x7f62e40cbb60 2023-10-21T17:40:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d0003520 con 0x7f62e40cbb60 2023-10-21T17:40:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4279676481 0 0) 0x7f62d0003520 con 0x7f62e40cbb60 2023-10-21T17:40:49.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 >> v2:172.21.15.124:3300/0 conn(0x7f62e40ca620 msgr2=0x7f62e40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 --2- 172.21.15.53:0/889163075 >> v2:172.21.15.124:3300/0 conn(0x7f62e40ca620 0x7f62e40b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 >> v1:172.21.15.124:6789/0 conn(0x7f62e40ada10 legacy=0x7f62e40cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e406c550 con 0x7f62e40cbb60 2023-10-21T17:40:49.622 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f62cc001670 con 0x7f62e40cbb60 2023-10-21T17:40:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 >> v1:172.21.15.53:6789/0 conn(0x7f62e40cbb60 legacy=0x7f62e415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d00039c0 0x7f62d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 0x7f62d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00051d0 0x7f62d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.623 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e406c270 con 0x7f62d00047c0 2023-10-21T17:40:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d000c120 con 0x7f62d00051d0 2023-10-21T17:40:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d000c2e0 con 0x7f62d00039c0 2023-10-21T17:40:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.618+0000 7f62dbfff640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 0x7f62d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62e259c640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d00039c0 0x7f62d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.624 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62e2d9d640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00051d0 0x7f62d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62dbfff640 1 -- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d00039c0 msgr2=0x7f62d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62dbfff640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62d00039c0 0x7f62d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62dbfff640 1 -- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00051d0 msgr2=0x7f62d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62dbfff640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00051d0 0x7f62d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62dbfff640 1 -- 172.21.15.53:0/889163075 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0003520 con 0x7f62d00047c0 2023-10-21T17:40:49.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.619+0000 7f62e2d9d640 1 -- 172.21.15.53:0/889163075 reap_dead start 2023-10-21T17:40:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.620+0000 7f62dbfff640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 0x7f62d0004c40 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001eb0 tx=0x7f62d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bfcfdafcd2e20e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc005ad0 con 0x7f62d00047c0 2023-10-21T17:40:49.629 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc005c30 con 0x7f62d00047c0 2023-10-21T17:40:49.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e1d9b640 1 -- 172.21.15.53:0/889163075 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0072b0 con 0x7f62d00047c0 2023-10-21T17:40:49.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 -- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 msgr2=0x7f62d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 0x7f62d0004c40 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f62cc001eb0 tx=0x7f62d00057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 -- 172.21.15.53:0/889163075 shutdown_connections 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 --2- 172.21.15.53:0/889163075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62d00047c0 0x7f62d0004c40 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 -- 172.21.15.53:0/889163075 >> 172.21.15.53:0/889163075 conn(0x7f62e40ce9d0 msgr2=0x7f62e40cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 -- 172.21.15.53:0/889163075 shutdown_connections 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.621+0000 7f62e359e640 1 -- 172.21.15.53:0/889163075 wait complete. 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.623+0000 7f62e359e640 1 Processor -- start 2023-10-21T17:40:49.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.623+0000 7f62e359e640 1 -- start start 2023-10-21T17:40:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00047c0 0x7f62e4097aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62e41669a0 0x7f62e40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e40c4e40 con 0x7f62e409a020 2023-10-21T17:40:49.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e41673e0 con 0x7f62d00047c0 2023-10-21T17:40:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.624+0000 7f62e359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e409a9e0 con 0x7f62e41669a0 2023-10-21T17:40:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35102/0 (socket says 172.21.15.53:35102) 2023-10-21T17:40:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 -- 172.21.15.53:0/3130264412 learned_addr learned my addr 172.21.15.53:0/3130264412 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:49.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 -- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62e41669a0 msgr2=0x7f62e40c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62e41669a0 0x7f62e40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 -- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00047c0 msgr2=0x7f62e4097aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e259c640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00047c0 0x7f62e4097aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00047c0 0x7f62e4097aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 -- 172.21.15.53:0/3130264412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40c3f20 con 0x7f62e409a020 2023-10-21T17:40:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.625+0000 7f62e2d9d640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f62e40ce2d0 tx=0x7f62dc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.626+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc016420 con 0x7f62e409a020 2023-10-21T17:40:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.626+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc0165a0 con 0x7f62e409a020 2023-10-21T17:40:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.626+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc016950 con 0x7f62e409a020 2023-10-21T17:40:49.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.626+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40c41c0 con 0x7f62e409a020 2023-10-21T17:40:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.626+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e40c4780 con 0x7f62e409a020 2023-10-21T17:40:49.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.627+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e4067280 con 0x7f62e409a020 2023-10-21T17:40:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.628+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f62dc016ab0 con 0x7f62e409a020 2023-10-21T17:40:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.629+0000 7f62daffd640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 0x7f62c406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.629+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f62dc08acb0 con 0x7f62e409a020 2023-10-21T17:40:49.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.630+0000 7f62e259c640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 0x7f62c406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:49.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.631+0000 7f62e259c640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 0x7f62c406d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62d4007fd0 tx=0x7f62d4009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:49.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.634+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62dc055890 con 0x7f62e409a020 2023-10-21T17:40:49.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.866+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62e40c4f80 con 0x7f62e409a020 2023-10-21T17:40:49.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.867+0000 7f62daffd640 1 -- 172.21.15.53:0/3130264412 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v143) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62e40c4f80 con 0x7f62e409a020 2023-10-21T17:40:49.868 INFO:teuthology.orchestra.run.smithi053.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-21T17:40:49.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.869+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 msgr2=0x7f62c406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.869+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 0x7f62c406d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f62d4007fd0 tx=0x7f62d4009040 comp rx=0 tx=0).stop 2023-10-21T17:40:49.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.869+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 msgr2=0x7f62e409a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:49.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.869+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f62e40ce2d0 tx=0x7f62dc00f520 comp rx=0 tx=0).stop 2023-10-21T17:40:49.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 shutdown_connections 2023-10-21T17:40:49.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f62c406b0e0 0x7f62c406d5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62e409a020 0x7f62e409a4a0 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f62e41669a0 0x7f62e40c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 --2- 172.21.15.53:0/3130264412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f62d00047c0 0x7f62e4097aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:49.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 >> 172.21.15.53:0/3130264412 conn(0x7f62e40ce9d0 msgr2=0x7f62e40701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:49.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 shutdown_connections 2023-10-21T17:40:49.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:49.870+0000 7f62e359e640 1 -- 172.21.15.53:0/3130264412 wait complete. 2023-10-21T17:40:49.891 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-21T17:40:49.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-21T17:40:49.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi053.front.sepia.ceph.com 2023-10-21T17:40:49.892 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-21T17:40:49.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-21T17:40:49.893 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-21T17:40:49.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-21T17:40:49.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-21T17:40:49.894 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:40:49.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:49.895 DEBUG:teuthology.orchestra.run.smithi053:> ip addr 2023-10-21T17:40:49.950 INFO:teuthology.orchestra.run.smithi053.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-21T17:40:49.950 INFO:teuthology.orchestra.run.smithi053.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-21T17:40:49.951 INFO:teuthology.orchestra.run.smithi053.stdout: inet 127.0.0.1/8 scope host lo 2023-10-21T17:40:49.951 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.951 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 ::1/128 scope host 2023-10-21T17:40:49.951 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.951 INFO:teuthology.orchestra.run.smithi053.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:40:49.952 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:da brd ff:ff:ff:ff:ff:ff 2023-10-21T17:40:49.952 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f0 2023-10-21T17:40:49.952 INFO:teuthology.orchestra.run.smithi053.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:40:49.952 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0c:c4:7a:6b:fd:db brd ff:ff:ff:ff:ff:ff 2023-10-21T17:40:49.952 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp7s0f1 2023-10-21T17:40:49.953 INFO:teuthology.orchestra.run.smithi053.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-21T17:40:49.953 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:18 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:40:49.953 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f0 2023-10-21T17:40:49.953 INFO:teuthology.orchestra.run.smithi053.stdout: inet 172.21.15.53/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-21T17:40:49.953 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft 36443sec preferred_lft 36443sec 2023-10-21T17:40:49.954 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::ae1f:6bff:fef8:3718/64 scope link noprefixroute 2023-10-21T17:40:49.954 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.954 INFO:teuthology.orchestra.run.smithi053.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-21T17:40:49.954 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether ac:1f:6b:f8:37:19 brd ff:ff:ff:ff:ff:ff 2023-10-21T17:40:49.954 INFO:teuthology.orchestra.run.smithi053.stdout: altname enp1s0f1 2023-10-21T17:40:49.955 INFO:teuthology.orchestra.run.smithi053.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-21T17:40:49.955 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff 2023-10-21T17:40:49.955 INFO:teuthology.orchestra.run.smithi053.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-21T17:40:49.955 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.955 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::e4ea:56ff:fef6:4799/64 scope link 2023-10-21T17:40:49.956 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.956 INFO:teuthology.orchestra.run.smithi053.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:40:49.956 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 0a:d3:88:31:05:8b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:40:49.956 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::8d3:88ff:fe31:58b/64 scope link 2023-10-21T17:40:49.956 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.957 INFO:teuthology.orchestra.run.smithi053.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-21T17:40:49.957 INFO:teuthology.orchestra.run.smithi053.stdout: link/ether 4e:1f:73:e1:a6:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:40:49.957 INFO:teuthology.orchestra.run.smithi053.stdout: inet6 fe80::4c1f:73ff:fee1:a65c/64 scope link 2023-10-21T17:40:49.957 INFO:teuthology.orchestra.run.smithi053.stdout: valid_lft forever preferred_lft forever 2023-10-21T17:40:49.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:49.959 DEBUG:teuthology.orchestra.run.smithi053:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-21T17:40:50.054 INFO:teuthology.orchestra.run.smithi053.stdout:1 2023-10-21T17:40:50.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.058 DEBUG:teuthology.orchestra.run.smithi053:> route 2023-10-21T17:40:50.119 INFO:teuthology.orchestra.run.smithi053.stdout:Kernel IP routing table 2023-10-21T17:40:50.120 INFO:teuthology.orchestra.run.smithi053.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-21T17:40:50.120 INFO:teuthology.orchestra.run.smithi053.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-21T17:40:50.120 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-21T17:40:50.120 INFO:teuthology.orchestra.run.smithi053.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-21T17:40:50.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.121 DEBUG:teuthology.orchestra.run.smithi053:> (cd / && exec bash -c ' 2023-10-21T17:40:50.121 DEBUG:teuthology.orchestra.run.smithi053:> set -e 2023-10-21T17:40:50.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-21T17:40:50.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-21T17:40:50.123 DEBUG:teuthology.orchestra.run.smithi053:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-21T17:40:50.123 DEBUG:teuthology.orchestra.run.smithi053:> ') 2023-10-21T17:40:50.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.323 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:40:50.379 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:40:50.379 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:40:50.380 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:40:50.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link set brx.0 up 2023-10-21T17:40:50.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:40:50.485 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-21T17:40:50.549 INFO:teuthology.orchestra.run.smithi053.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-21T17:40:50.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-21T17:40:50.550 DEBUG:teuthology.orchestra.run.smithi053:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:40:50.604 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-21T17:40:50.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:40:50.753 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:40:50.754 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:40:50.790 INFO:teuthology.orchestra.run.smithi053.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-21T17:40:50.790 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-21T17:40:50.790 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: resolved to: "172.21.15.53:3300,172.21.15.124:3300,172.21.15.124:3301" 2023-10-21T17:40:50.790 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: trying mount with new device syntax: 0@d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56.cephfs=/ 2023-10-21T17:40:50.791 INFO:teuthology.orchestra.run.smithi053.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.53:3300/172.21.15.124:3300/172.21.15.124:3301" will pass to kernel 2023-10-21T17:40:50.794 INFO:teuthology.orchestra.run.smithi053.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-21T17:40:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: You just mounted a file system that supports labels which does not 2023-10-21T17:40:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-21T17:40:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: applications will generate AVC messages and not be allowed access to 2023-10-21T17:40:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-21T17:40:50.799 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-21T17:40:50.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-21T17:40:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:50.899 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-21T17:40:50.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:40:50.954 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:40:50.982 INFO:teuthology.orchestra.run.smithi053.stdout:client10126 2023-10-21T17:40:50.982 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:40:50.983 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:40:51.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-21T17:40:51.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.345+0000 7f69b18a3640 1 Processor -- start 2023-10-21T17:40:51.347 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.346+0000 7f69b18a3640 1 -- start start 2023-10-21T17:40:51.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b18a3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69ac0c8b40 0x7f69ac0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b18a3640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 0x7f69ac0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b18a3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ac0c94f0 con 0x7f69ac15cae0 2023-10-21T17:40:51.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b18a3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac0ccbc0 con 0x7f69ac0c8b40 2023-10-21T17:40:51.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b18a3640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac0ce1f0 con 0x7f69ac0aff70 2023-10-21T17:40:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b08a1640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f69ac15cae0 0x7f69ac0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46892/0 (socket says 172.21.15.53:46892) 2023-10-21T17:40:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69b08a1640 1 -- 172.21.15.53:0/4178679000 learned_addr learned my addr 172.21.15.53:0/4178679000 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.347+0000 7f69a3fff640 1 --2- 172.21.15.53:0/4178679000 >> v2:172.21.15.53:3300/0 conn(0x7f69ac0c8b40 0x7f69ac0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a37fe640 1 -- 172.21.15.53:0/4178679000 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860854271 0 0) 0x7f69ac0c94f0 con 0x7f69ac15cae0 2023-10-21T17:40:51.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a37fe640 1 -- 172.21.15.53:0/4178679000 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6994003520 con 0x7f69ac15cae0 2023-10-21T17:40:51.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69b10a2640 1 --2- 172.21.15.53:0/4178679000 >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 0x7f69ac0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a3fff640 1 -- 172.21.15.53:0/4178679000 >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 msgr2=0x7f69ac0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:51.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a3fff640 1 --2- 172.21.15.53:0/4178679000 >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 0x7f69ac0b03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a3fff640 1 -- 172.21.15.53:0/4178679000 >> v1:172.21.15.53:6789/0 conn(0x7f69ac15cae0 legacy=0x7f69ac0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69a3fff640 1 -- 172.21.15.53:0/4178679000 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0b0900 con 0x7f69ac0c8b40 2023-10-21T17:40:51.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.348+0000 7f69b10a2640 1 --2- 172.21.15.53:0/4178679000 >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 0x7f69ac0b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:51.352 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.349+0000 7f69a3fff640 1 --2- 172.21.15.53:0/4178679000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c8fb0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0b4420 tx=0x7f699c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5f70810a2af43b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.349+0000 7f69a37fe640 1 -- 172.21.15.53:0/4178679000 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c031830 con 0x7f69ac0c8b40 2023-10-21T17:40:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.349+0000 7f69a37fe640 1 -- 172.21.15.53:0/4178679000 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c031990 con 0x7f69ac0c8b40 2023-10-21T17:40:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.349+0000 7f69a37fe640 1 -- 172.21.15.53:0/4178679000 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c031d20 con 0x7f69ac0c8b40 2023-10-21T17:40:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 -- 172.21.15.53:0/4178679000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 msgr2=0x7f69ac0c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 --2- 172.21.15.53:0/4178679000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c8fb0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0b4420 tx=0x7f699c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 -- 172.21.15.53:0/4178679000 shutdown_connections 2023-10-21T17:40:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 --2- 172.21.15.53:0/4178679000 >> v2:172.21.15.124:3300/0 conn(0x7f69ac0aff70 0x7f69ac0b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 --2- 172.21.15.53:0/4178679000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c8fb0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 -- 172.21.15.53:0/4178679000 >> 172.21.15.53:0/4178679000 conn(0x7f69ac06c050 msgr2=0x7f69ac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:51.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 -- 172.21.15.53:0/4178679000 shutdown_connections 2023-10-21T17:40:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.350+0000 7f69b18a3640 1 -- 172.21.15.53:0/4178679000 wait complete. 2023-10-21T17:40:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.352+0000 7f69b18a3640 1 Processor -- start 2023-10-21T17:40:51.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.352+0000 7f69b18a3640 1 -- start start 2023-10-21T17:40:51.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69ac0aff70 0x7f69ac14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.358 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69ac15cae0 0x7f69ac0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0c4df0 con 0x7f69ac0c8b40 2023-10-21T17:40:51.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0c73f0 con 0x7f69ac0aff70 2023-10-21T17:40:51.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69b18a3640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0c76d0 con 0x7f69ac15cae0 2023-10-21T17:40:51.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35122/0 (socket says 172.21.15.53:35122) 2023-10-21T17:40:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.353+0000 7f69a3fff640 1 -- 172.21.15.53:0/2313603073 learned_addr learned my addr 172.21.15.53:0/2313603073 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69b08a1640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69ac0aff70 0x7f69ac14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69b10a2640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69ac15cae0 0x7f69ac0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 -- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69ac15cae0 msgr2=0x7f69ac0c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69ac15cae0 0x7f69ac0c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 -- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69ac0aff70 msgr2=0x7f69ac14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69ac0aff70 0x7f69ac14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 -- 172.21.15.53:0/2313603073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c009400 con 0x7f69ac0c8b40 2023-10-21T17:40:51.361 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.354+0000 7f69a3fff640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f699c009890 tx=0x7f699c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.355+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c03a6f0 con 0x7f69ac0c8b40 2023-10-21T17:40:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.355+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c03a870 con 0x7f69ac0c8b40 2023-10-21T17:40:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.355+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c03ac20 con 0x7f69ac0c8b40 2023-10-21T17:40:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.355+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac0c79b0 con 0x7f69ac0c8b40 2023-10-21T17:40:51.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.355+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac14cd80 con 0x7f69ac0c8b40 2023-10-21T17:40:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.356+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ac074150 con 0x7f69ac0c8b40 2023-10-21T17:40:51.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.357+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f699c043410 con 0x7f69ac0c8b40 2023-10-21T17:40:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.358+0000 7f69a17fa640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 0x7f699806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.358+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f699c0ab9b0 con 0x7f69ac0c8b40 2023-10-21T17:40:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.359+0000 7f69b08a1640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 0x7f699806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.360+0000 7f69b08a1640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 0x7f699806d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69a8001560 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.362+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f699c076540 con 0x7f69ac0c8b40 2023-10-21T17:40:51.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.577+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 --> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ac0cc570 con 0x7f699806b0b0 2023-10-21T17:40:51.582 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.581+0000 7f69a17fa640 1 -- 172.21.15.53:0/2313603073 <== mgr.9841 v2:172.21.15.124:6832/596989303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f69a8001eb0 con 0x7f699806b0b0 2023-10-21T17:40:51.582 INFO:teuthology.orchestra.run.smithi053.stdout:[ 2023-10-21T17:40:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2023-10-21T17:40:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "cephfs" 2023-10-21T17:40:51.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-10-21T17:40:51.583 INFO:teuthology.orchestra.run.smithi053.stdout:] 2023-10-21T17:40:51.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.583+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 msgr2=0x7f699806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.583+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 0x7f699806d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69a8001560 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2023-10-21T17:40:51.585 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.583+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 msgr2=0x7f69ac0c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.583+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f699c009890 tx=0x7f699c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 shutdown_connections 2023-10-21T17:40:51.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f699806b0b0 0x7f699806d570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69ac15cae0 0x7f69ac0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.586 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69ac0c8b40 0x7f69ac0c4370 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 --2- 172.21.15.53:0/2313603073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69ac0aff70 0x7f69ac14c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 >> 172.21.15.53:0/2313603073 conn(0x7f69ac06c050 msgr2=0x7f69ac0caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:51.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 shutdown_connections 2023-10-21T17:40:51.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.584+0000 7f69b18a3640 1 -- 172.21.15.53:0/2313603073 wait complete. 2023-10-21T17:40:51.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-21T17:40:51.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.835+0000 7f6c320aa640 1 Processor -- start 2023-10-21T17:40:51.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.836+0000 7f6c320aa640 1 -- start start 2023-10-21T17:40:51.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.837+0000 7f6c320aa640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c2c0d00e0 0x7f6c2c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.837+0000 7f6c320aa640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6c2c0d0b90 0x7f6c2c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.837+0000 7f6c320aa640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0d2870 con 0x7f6c2c15ccb0 2023-10-21T17:40:51.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.837+0000 7f6c320aa640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d1150 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.837+0000 7f6c320aa640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0d1430 con 0x7f6c2c0d0b90 2023-10-21T17:40:51.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c310a8640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6c2c15ccb0 0x7f6c2c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46898/0 (socket says 172.21.15.53:46898) 2023-10-21T17:40:51.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c308a7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c2c0d00e0 0x7f6c2c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c310a8640 1 -- 172.21.15.53:0/3077658697 learned_addr learned my addr 172.21.15.53:0/3077658697 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c318a9640 1 --2- 172.21.15.53:0/3077658697 >> v2:172.21.15.124:3300/0 conn(0x7f6c2c0d0b90 0x7f6c2c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c308a7640 1 -- 172.21.15.53:0/3077658697 >> v2:172.21.15.124:3300/0 conn(0x7f6c2c0d0b90 msgr2=0x7f6c2c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:51.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c308a7640 1 --2- 172.21.15.53:0/3077658697 >> v2:172.21.15.124:3300/0 conn(0x7f6c2c0d0b90 0x7f6c2c0b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c308a7640 1 -- 172.21.15.53:0/3077658697 >> v1:172.21.15.53:6789/0 conn(0x7f6c2c15ccb0 legacy=0x7f6c2c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c308a7640 1 -- 172.21.15.53:0/3077658697 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0d1710 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.838+0000 7f6c23fff640 1 -- 172.21.15.53:0/3077658697 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018055768 0 0) 0x7f6c2c0d2870 con 0x7f6c2c15ccb0 2023-10-21T17:40:51.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.840+0000 7f6c308a7640 1 --2- 172.21.15.53:0/3077658697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0d1fb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0d2520 tx=0x7f6c1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c554b8b9e507ad05 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.840+0000 7f6c23fff640 1 -- 172.21.15.53:0/3077658697 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c031830 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.840+0000 7f6c23fff640 1 -- 172.21.15.53:0/3077658697 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c031990 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.840+0000 7f6c23fff640 1 -- 172.21.15.53:0/3077658697 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c031d20 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 -- 172.21.15.53:0/3077658697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 msgr2=0x7f6c2c0d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 --2- 172.21.15.53:0/3077658697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0d1fb0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0d2520 tx=0x7f6c1c02b8d0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 -- 172.21.15.53:0/3077658697 shutdown_connections 2023-10-21T17:40:51.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 --2- 172.21.15.53:0/3077658697 >> v2:172.21.15.124:3300/0 conn(0x7f6c2c0d0b90 0x7f6c2c0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 --2- 172.21.15.53:0/3077658697 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0d1fb0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 -- 172.21.15.53:0/3077658697 >> 172.21.15.53:0/3077658697 conn(0x7f6c2c071660 msgr2=0x7f6c2c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:51.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 -- 172.21.15.53:0/3077658697 shutdown_connections 2023-10-21T17:40:51.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.841+0000 7f6c320aa640 1 -- 172.21.15.53:0/3077658697 wait complete. 2023-10-21T17:40:51.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.843+0000 7f6c320aa640 1 Processor -- start 2023-10-21T17:40:51.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.843+0000 7f6c320aa640 1 -- start start 2023-10-21T17:40:51.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c2c0d0b90 0x7f6c2c0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 0x7f6c2c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c76c0 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c6ab0 con 0x7f6c2c0d0b90 2023-10-21T17:40:51.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c320aa640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0c6d90 con 0x7f6c2c15ccb0 2023-10-21T17:40:51.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c310a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.844+0000 7f6c310a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35138/0 (socket says 172.21.15.53:35138) 2023-10-21T17:40:51.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 -- 172.21.15.53:0/2943242309 learned_addr learned my addr 172.21.15.53:0/2943242309 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c308a7640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c2c0d0b90 0x7f6c2c0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c318a9640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 0x7f6c2c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 -- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 msgr2=0x7f6c2c14c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 0x7f6c2c14c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 -- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c2c0d0b90 msgr2=0x7f6c2c0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c2c0d0b90 0x7f6c2c0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 -- 172.21.15.53:0/2943242309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009400 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c318a9640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 0x7f6c2c14c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.845+0000 7f6c310a8640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c7c0 tx=0x7f6c2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.846+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800a640 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.846+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2800a7c0 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.846+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800ab70 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.846+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c0c7070 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.846+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c0c66f0 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.847+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c074150 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.848+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 144) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6c2800acd0 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.849+0000 7f6c21ffb640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 0x7f6c1006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.849+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6c28089280 con 0x7f6c2c0d00e0 2023-10-21T17:40:51.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.849+0000 7f6c308a7640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 0x7f6c1006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:51.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.851+0000 7f6c308a7640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 0x7f6c1006d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c036000 tx=0x7f6c1c008030 comp rx=0 tx=0).ready entity=mgr.9841 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:51.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:51.853+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c28053d70 con 0x7f6c2c0d00e0 2023-10-21T17:40:52.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.094+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6c2c0b4bf0 con 0x7f6c2c0d00e0 2023-10-21T17:40:52.589 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.588+0000 7f6c21ffb640 1 -- 172.21.15.53:0/2943242309 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v145) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6c280180a0 con 0x7f6c2c0d00e0 2023-10-21T17:40:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 msgr2=0x7f6c1006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 0x7f6c1006d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c036000 tx=0x7f6c1c008030 comp rx=0 tx=0).stop 2023-10-21T17:40:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 msgr2=0x7f6c2c0cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c7c0 tx=0x7f6c2800cc80 comp rx=0 tx=0).stop 2023-10-21T17:40:52.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 shutdown_connections 2023-10-21T17:40:52.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f6c1006b0b0 0x7f6c1006d570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c2c15ccb0 0x7f6c2c14c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c2c0d0b90 0x7f6c2c0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 --2- 172.21.15.53:0/2943242309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c2c0d00e0 0x7f6c2c0cbf60 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 >> 172.21.15.53:0/2943242309 conn(0x7f6c2c071660 msgr2=0x7f6c2c0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:52.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 shutdown_connections 2023-10-21T17:40:52.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.592+0000 7f6c320aa640 1 -- 172.21.15.53:0/2943242309 wait complete. 2023-10-21T17:40:52.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-21T17:40:52.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.820+0000 7f66ce31d640 1 Processor -- start 2023-10-21T17:40:52.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.820+0000 7f66ce31d640 1 -- start start 2023-10-21T17:40:52.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66ce31d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f66c80cd250 0x7f66c815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66ce31d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c806b6a0 con 0x7f66c80cea30 2023-10-21T17:40:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66ce31d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c80b19e0 con 0x7f66c815cdb0 2023-10-21T17:40:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66ce31d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c8069ef0 con 0x7f66c80cd250 2023-10-21T17:40:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66c77fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f66c80cea30 0x7f66c80ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46912/0 (socket says 172.21.15.53:46912) 2023-10-21T17:40:52.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.822+0000 7f66c77fe640 1 -- 172.21.15.53:0/192908773 learned_addr learned my addr 172.21.15.53:0/192908773 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:52.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66cc8ba640 1 --2- 172.21.15.53:0/192908773 >> v2:172.21.15.124:3300/0 conn(0x7f66c80cd250 0x7f66c815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546376090 0 0) 0x7f66c806b6a0 con 0x7f66c80cea30 2023-10-21T17:40:52.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b4003520 con 0x7f66c80cea30 2023-10-21T17:40:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 376977904 0 0) 0x7f66b4003520 con 0x7f66c80cea30 2023-10-21T17:40:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 >> v2:172.21.15.124:3300/0 conn(0x7f66c80cd250 msgr2=0x7f66c815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 --2- 172.21.15.53:0/192908773 >> v2:172.21.15.124:3300/0 conn(0x7f66c80cd250 0x7f66c815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 >> v1:172.21.15.124:6789/0 conn(0x7f66c815cdb0 legacy=0x7f66c806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c806a1d0 con 0x7f66c80cea30 2023-10-21T17:40:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f66b0001670 con 0x7f66c80cea30 2023-10-21T17:40:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 >> v1:172.21.15.53:6789/0 conn(0x7f66c80cea30 legacy=0x7f66c80ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.823+0000 7f66c6ffd640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b4003970 0x7f66b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c6ffd640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 0x7f66b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c6ffd640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66b4005180 0x7f66b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8069ef0 con 0x7f66b40047c0 2023-10-21T17:40:52.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b400c020 con 0x7f66b4003970 2023-10-21T17:40:52.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b400c1e0 con 0x7f66b4005180 2023-10-21T17:40:52.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c77fe640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 0x7f66b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66cc8ba640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66b4005180 0x7f66b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c7fff640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b4003970 0x7f66b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c77fe640 1 -- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66b4005180 msgr2=0x7f66b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.824+0000 7f66c77fe640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66b4005180 0x7f66b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.825+0000 7f66c77fe640 1 -- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b4003970 msgr2=0x7f66b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:52.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.825+0000 7f66c77fe640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b4003970 0x7f66b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.825+0000 7f66c77fe640 1 -- 172.21.15.53:0/192908773 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b4003520 con 0x7f66b40047c0 2023-10-21T17:40:52.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.825+0000 7f66cc8ba640 1 -- 172.21.15.53:0/192908773 reap_dead start 2023-10-21T17:40:52.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.826+0000 7f66c77fe640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 0x7f66b4004c40 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c240 tx=0x7f66b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da17fcf351632916 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.827+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0005ad0 con 0x7f66b40047c0 2023-10-21T17:40:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.827+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0005c30 con 0x7f66b40047c0 2023-10-21T17:40:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.827+0000 7f66c6ffd640 1 -- 172.21.15.53:0/192908773 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b002ca90 con 0x7f66b40047c0 2023-10-21T17:40:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.827+0000 7f66ce31d640 1 -- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 msgr2=0x7f66b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.827+0000 7f66ce31d640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 0x7f66b4004c40 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c240 tx=0x7f66b40057f0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.828+0000 7f66ce31d640 1 -- 172.21.15.53:0/192908773 shutdown_connections 2023-10-21T17:40:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.828+0000 7f66ce31d640 1 --2- 172.21.15.53:0/192908773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66b40047c0 0x7f66b4004c40 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.828+0000 7f66ce31d640 1 -- 172.21.15.53:0/192908773 >> 172.21.15.53:0/192908773 conn(0x7f66c80b0af0 msgr2=0x7f66c80b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.828+0000 7f66ce31d640 1 -- 172.21.15.53:0/192908773 shutdown_connections 2023-10-21T17:40:52.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.828+0000 7f66ce31d640 1 -- 172.21.15.53:0/192908773 wait complete. 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.830+0000 7f66ce31d640 1 Processor -- start 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.830+0000 7f66ce31d640 1 -- start start 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 0x7f66c81454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66c81e8730 0x7f66c81e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.836 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81e7760 con 0x7f66c8158ef0 2023-10-21T17:40:52.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8159890 con 0x7f66b40047c0 2023-10-21T17:40:52.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66ce31d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81e82a0 con 0x7f66c81e8730 2023-10-21T17:40:52.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66c77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35184/0 (socket says 172.21.15.53:35184) 2023-10-21T17:40:52.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66c77fe640 1 -- 172.21.15.53:0/3766781548 learned_addr learned my addr 172.21.15.53:0/3766781548 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66cc8ba640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66c81e8730 0x7f66c81e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.831+0000 7f66c7fff640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 0x7f66c81454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66c81e8730 msgr2=0x7f66c81e8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66c81e8730 0x7f66c81e8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 msgr2=0x7f66c81454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 0x7f66c81454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 -- 172.21.15.53:0/3766781548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c81e7970 con 0x7f66c8158ef0 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c7fff640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 0x7f66c81454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c77fe640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f66b0001930 tx=0x7f66b002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b003c040 con 0x7f66c8158ef0 2023-10-21T17:40:52.839 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b002cc60 con 0x7f66c8158ef0 2023-10-21T17:40:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c81e7c50 con 0x7f66c8158ef0 2023-10-21T17:40:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.832+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0007070 con 0x7f66c8158ef0 2023-10-21T17:40:52.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.833+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c81e90f0 con 0x7f66c8158ef0 2023-10-21T17:40:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.834+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c804e160 con 0x7f66c8158ef0 2023-10-21T17:40:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.835+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 145) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f66b0032070 con 0x7f66c8158ef0 2023-10-21T17:40:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.836+0000 7f66c4ff9640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 0x7f66a806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:52.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.836+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f66b00ab170 con 0x7f66c8158ef0 2023-10-21T17:40:52.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.836+0000 7f66c7fff640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 msgr2=0x7f66a806d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:52.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.836+0000 7f66c7fff640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 0x7f66a806d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:52.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:52.844+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66b0075c60 con 0x7f66c8158ef0 2023-10-21T17:40:53.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.037+0000 7f66c7fff640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 msgr2=0x7f66a806d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:53.038 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.037+0000 7f66c7fff640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 0x7f66a806d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:53.046 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.045+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f66c81e83e0 con 0x7f66c8158ef0 2023-10-21T17:40:53.052 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.051+0000 7f66c4ff9640 1 -- 172.21.15.53:0/3766781548 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v89)=0 v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7f66b0079280 con 0x7f66c8158ef0 2023-10-21T17:40:53.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 msgr2=0x7f66a806d5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:53.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 0x7f66a806d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 msgr2=0x7f66c8159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f66b0001930 tx=0x7f66b002e070 comp rx=0 tx=0).stop 2023-10-21T17:40:53.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 shutdown_connections 2023-10-21T17:40:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f66a806b130 0x7f66a806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f66c81e8730 0x7f66c81e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66c8158ef0 0x7f66c8159350 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 --2- 172.21.15.53:0/3766781548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f66b40047c0 0x7f66c81454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.057 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 >> 172.21.15.53:0/3766781548 conn(0x7f66c80b0af0 msgr2=0x7f66c815d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:53.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 shutdown_connections 2023-10-21T17:40:53.058 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.054+0000 7f66ce31d640 1 -- 172.21.15.53:0/3766781548 wait complete. 2023-10-21T17:40:53.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-21T17:40:53.134 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.133+0000 7f28dc0c9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:53.139 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.138+0000 7f3172e12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:40:53.187 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.185+0000 7f28dc0c9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:53.195 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.194+0000 7f3172e12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:40:53.284 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.283+0000 7f2d50c7e640 1 Processor -- start 2023-10-21T17:40:53.285 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.284+0000 7f2d50c7e640 1 -- start start 2023-10-21T17:40:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d50c7e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d50c7e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2d4c0ce510 0x7f2d4c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d50c7e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4c0cd520 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d50c7e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4c0d1910 con 0x7f2d4c0ce510 2023-10-21T17:40:53.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d50c7e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0ce080 con 0x7f2d4c0cf690 2023-10-21T17:40:53.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d4b7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f2d4c0cf2b0 0x7f2d4c0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46920/0 (socket says 172.21.15.53:46920) 2023-10-21T17:40:53.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.285+0000 7f2d4b7fe640 1 -- 172.21.15.53:0/2409070120 learned_addr learned my addr 172.21.15.53:0/2409070120 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:53.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4bfff640 1 --2- 172.21.15.53:0/2409070120 >> v2:172.21.15.53:3300/0 conn(0x7f2d4c0ce510 0x7f2d4c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.288 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4affd640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4a7fc640 1 -- 172.21.15.53:0/2409070120 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54791623 0 0) 0x7f2d4c0cd520 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4a7fc640 1 -- 172.21.15.53:0/2409070120 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d34003520 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4bfff640 1 -- 172.21.15.53:0/2409070120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 msgr2=0x7f2d4c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:53.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4bfff640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4bfff640 1 -- 172.21.15.53:0/2409070120 >> v1:172.21.15.53:6789/0 conn(0x7f2d4c0cf2b0 legacy=0x7f2d4c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.286+0000 7f2d4bfff640 1 -- 172.21.15.53:0/2409070120 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0cd750 con 0x7f2d4c0ce510 2023-10-21T17:40:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.287+0000 7f2d4bfff640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c15c820 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf78d965706652d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:53.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d4a7fc640 1 -- 172.21.15.53:0/2409070120 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4402cbc0 con 0x7f2d4c0ce510 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d4a7fc640 1 -- 172.21.15.53:0/2409070120 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4402cd20 con 0x7f2d4c0ce510 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d4a7fc640 1 -- 172.21.15.53:0/2409070120 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d44034660 con 0x7f2d4c0ce510 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2409070120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0ce510 msgr2=0x7f2d4c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c15c820 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009d90 tx=0x7f2d4402be10 comp rx=0 tx=0).stop 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2409070120 shutdown_connections 2023-10-21T17:40:53.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c15c820 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2409070120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2409070120 >> 172.21.15.53:0/2409070120 conn(0x7f2d4c09bc40 msgr2=0x7f2d4c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2409070120 shutdown_connections 2023-10-21T17:40:53.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.288+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2409070120 wait complete. 2023-10-21T17:40:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.291+0000 7f2d50c7e640 1 Processor -- start 2023-10-21T17:40:53.295 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.291+0000 7f2d50c7e640 1 -- start start 2023-10-21T17:40:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0caa00 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.296 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c15fdc0 con 0x7f2d4c0ce510 2023-10-21T17:40:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d50c7e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1600a0 con 0x7f2d4c0cf690 2023-10-21T17:40:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d4affd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35204/0 (socket says 172.21.15.53:35204) 2023-10-21T17:40:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d4affd640 1 -- 172.21.15.53:0/2412359022 learned_addr learned my addr 172.21.15.53:0/2412359022 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:53.297 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d4bfff640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.293+0000 7f2d4b7fe640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 msgr2=0x7f2d4c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d4c0ce510 msgr2=0x7f2d4c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.298 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 -- 172.21.15.53:0/2412359022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009a70 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d4affd640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e870 tx=0x7f2d4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400094f0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000c940 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000ccf0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.299 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c0c99e0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.294+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c0c9fa0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.300 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.296+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c074150 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.297+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 145) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f2d40013470 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.297+0000 7f2d33fff640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 0x7f2d3806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.297+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2d40089cd0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.298+0000 7f2d4b7fe640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 msgr2=0x7f2d3806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:53.301 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.298+0000 7f2d4b7fe640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 0x7f2d3806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:53.306 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.306+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d40054910 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.497+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f2d4c0d0070 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.499+0000 7f2d4b7fe640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 msgr2=0x7f2d3806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:53.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.499+0000 7f2d4b7fe640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 0x7f2d3806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:53.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.504+0000 7f2d33fff640 1 -- 172.21.15.53:0/2412359022 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v90)=0 v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2d400170c0 con 0x7f2d4c0cf2b0 2023-10-21T17:40:53.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.506+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 msgr2=0x7f2d3806d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:53.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.506+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 0x7f2d3806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.506+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 msgr2=0x7f2d4c15b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.506+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000e870 tx=0x7f2d4000ed30 comp rx=0 tx=0).stop 2023-10-21T17:40:53.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 shutdown_connections 2023-10-21T17:40:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7f2d3806b100 0x7f2d3806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d4c0cf690 0x7f2d4c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2d4c0cf2b0 0x7f2d4c15b660 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 --2- 172.21.15.53:0/2412359022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d4c0ce510 0x7f2d4c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.509 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 >> 172.21.15.53:0/2412359022 conn(0x7f2d4c09bc40 msgr2=0x7f2d4c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:53.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 shutdown_connections 2023-10-21T17:40:53.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.507+0000 7f2d50c7e640 1 -- 172.21.15.53:0/2412359022 wait complete. 2023-10-21T17:40:53.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:53.527 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-21T17:40:53.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-21T17:40:53.639 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.638+0000 7f28dc0c9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:53.679 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.678+0000 7f3172e12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:40:53.691 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.690+0000 7f28dc0c9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:53.734 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.733+0000 7f3172e12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:40:53.741 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.739+0000 7f28dc0c9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.785+0000 7efed73b0640 1 Processor -- start 2023-10-21T17:40:53.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.785+0000 7efed73b0640 1 -- start start 2023-10-21T17:40:53.787 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.786+0000 7f3172e12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.786+0000 7efed73b0640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efed0069d50 0x7efed006a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.786+0000 7efed73b0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efed015b0a0 0x7efed015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed73b0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed0071c40 con 0x7efed00ca090 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed73b0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed0071380 con 0x7efed015b0a0 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed73b0640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed006a670 con 0x7efed0069d50 2023-10-21T17:40:53.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed494b640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7efed00ca090 0x7efed00ca520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:46932/0 (socket says 172.21.15.53:46932) 2023-10-21T17:40:53.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed494b640 1 -- 172.21.15.53:0/2789849767 learned_addr learned my addr 172.21.15.53:0/2789849767 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:53.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed594d640 1 --2- 172.21.15.53:0/2789849767 >> v2:172.21.15.53:3300/0 conn(0x7efed015b0a0 0x7efed015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efed514c640 1 --2- 172.21.15.53:0/2789849767 >> v2:172.21.15.124:3300/0 conn(0x7efed0069d50 0x7efed006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efec7fff640 1 -- 172.21.15.53:0/2789849767 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085722859 0 0) 0x7efed0071c40 con 0x7efed00ca090 2023-10-21T17:40:53.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.787+0000 7efec7fff640 1 -- 172.21.15.53:0/2789849767 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efebc003520 con 0x7efed00ca090 2023-10-21T17:40:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.788+0000 7efed594d640 1 -- 172.21.15.53:0/2789849767 >> v2:172.21.15.124:3300/0 conn(0x7efed0069d50 msgr2=0x7efed006a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.788+0000 7efed594d640 1 --2- 172.21.15.53:0/2789849767 >> v2:172.21.15.124:3300/0 conn(0x7efed0069d50 0x7efed006a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.788+0000 7efed594d640 1 -- 172.21.15.53:0/2789849767 >> v1:172.21.15.53:6789/0 conn(0x7efed00ca090 legacy=0x7efed00ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.791 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.788+0000 7efed594d640 1 -- 172.21.15.53:0/2789849767 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00715f0 con 0x7efed015b0a0 2023-10-21T17:40:53.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.789+0000 7efed594d640 1 --2- 172.21.15.53:0/2789849767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed015b0a0 0x7efed015b4c0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7efed006ab20 tx=0x7efec802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61951056fc74ea7d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.789+0000 7efec7fff640 1 -- 172.21.15.53:0/2789849767 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec802cc70 con 0x7efed015b0a0 2023-10-21T17:40:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.789+0000 7efec7fff640 1 -- 172.21.15.53:0/2789849767 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec802cdd0 con 0x7efed015b0a0 2023-10-21T17:40:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.789+0000 7efec7fff640 1 -- 172.21.15.53:0/2789849767 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8034660 con 0x7efed015b0a0 2023-10-21T17:40:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 -- 172.21.15.53:0/2789849767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed015b0a0 msgr2=0x7efed015b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 --2- 172.21.15.53:0/2789849767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed015b0a0 0x7efed015b4c0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7efed006ab20 tx=0x7efec802bec0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 -- 172.21.15.53:0/2789849767 shutdown_connections 2023-10-21T17:40:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 --2- 172.21.15.53:0/2789849767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed015b0a0 0x7efed015b4c0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 --2- 172.21.15.53:0/2789849767 >> v2:172.21.15.124:3300/0 conn(0x7efed0069d50 0x7efed006a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 -- 172.21.15.53:0/2789849767 >> 172.21.15.53:0/2789849767 conn(0x7efed00cf8c0 msgr2=0x7efed00cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 -- 172.21.15.53:0/2789849767 shutdown_connections 2023-10-21T17:40:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.790+0000 7efed73b0640 1 -- 172.21.15.53:0/2789849767 wait complete. 2023-10-21T17:40:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.792+0000 7efed73b0640 1 Processor -- start 2023-10-21T17:40:53.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.792+0000 7efed73b0640 1 -- start start 2023-10-21T17:40:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 0x7efed00cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efed00ca090 0x7efed0159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efed015b0a0 0x7efed00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed00a7810 con 0x7efed0069d50 2023-10-21T17:40:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed00cdc70 con 0x7efed015b0a0 2023-10-21T17:40:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed73b0640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed00cdf50 con 0x7efed00ca090 2023-10-21T17:40:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 0x7efed00cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 0x7efed00cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:35224/0 (socket says 172.21.15.53:35224) 2023-10-21T17:40:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 -- 172.21.15.53:0/1151354987 learned_addr learned my addr 172.21.15.53:0/1151354987 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efed00ca090 0x7efed0159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed594d640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efed015b0a0 0x7efed00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efed00ca090 msgr2=0x7efed0159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efed00ca090 0x7efed0159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efed015b0a0 msgr2=0x7efed00cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efed015b0a0 0x7efed00cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed514c640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec8009a70 con 0x7efed0069d50 2023-10-21T17:40:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.793+0000 7efed594d640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efed015b0a0 0x7efed00cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:53.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efed514c640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 0x7efed00cd590 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e870 tx=0x7efecc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:53.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0094f0 con 0x7efed0069d50 2023-10-21T17:40:53.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc00c940 con 0x7efed0069d50 2023-10-21T17:40:53.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00ccf0 con 0x7efed0069d50 2023-10-21T17:40:53.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed00ce230 con 0x7efed0069d50 2023-10-21T17:40:53.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.794+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed00ce7f0 con 0x7efed0069d50 2023-10-21T17:40:53.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.796+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed004e160 con 0x7efed0069d50 2023-10-21T17:40:53.802 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.797+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 145) v1 ==== 89450+0+0 (secure 0 0 0) 0x7efecc013470 con 0x7efed0069d50 2023-10-21T17:40:53.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.798+0000 7efec5ffb640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:53.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.798+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6145+0+0 (secure 0 0 0) 0x7efecc089e40 con 0x7efed0069d50 2023-10-21T17:40:53.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.798+0000 7efed494b640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 msgr2=0x7efeb406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:53.803 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.798+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:40:53.804 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.801+0000 7f28dc0c9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:53.804 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.802+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efecc0548a0 con 0x7efed0069d50 2023-10-21T17:40:53.851 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.850+0000 7f28dc0c9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:53.854 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.852+0000 7f3172e12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:40:53.905 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:53.904+0000 7f3172e12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:40:53.962 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:53.960+0000 7f28dc0c9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.999+0000 7efed494b640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 msgr2=0x7efeb406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:54.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:53.999+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:40:54.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:54.017+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efed00aed00 con 0x7efeb406b100 2023-10-21T17:40:54.021 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.020+0000 7f3172e12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:40:54.024 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.023+0000 7f28dc0c9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:54.073 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.072+0000 7f28dc0c9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:54.088 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.086+0000 7f3172e12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:54.131 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.129+0000 7f28dc0c9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:54.139 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.138+0000 7f3172e12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:40:54.180 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.179+0000 7f28dc0c9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:54.198 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.196+0000 7f3172e12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:40:54.250 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.248+0000 7f3172e12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:40:54.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:54.400+0000 7efed494b640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 msgr2=0x7efeb406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:54.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:54.400+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-21T17:40:54.556 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.554+0000 7f28dc0c9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:54.644 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.643+0000 7f3172e12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:40:54.713 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.711+0000 7f28dc0c9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:54.810 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.809+0000 7f3172e12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:40:54.819 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.818+0000 7f28dc0c9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:54.869 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.867+0000 7f28dc0c9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:54.922 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.921+0000 7f3172e12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:40:54.974 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:54.973+0000 7f3172e12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:40:54.975 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:54.974+0000 7f28dc0c9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:55.080 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:55.079+0000 7f28dc0c9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:55.087 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:55.086+0000 7f3172e12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:55.199 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:55.198+0000 7f3172e12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:40:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:55.201+0000 7efed494b640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 msgr2=0x7efeb406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/596989303 2023-10-21T17:40:55.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:55.201+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-21T17:40:55.236 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:55.234+0000 7f28dc0c9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:55.285 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:55.283+0000 7f28dc0c9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:55.362 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:55.361+0000 7f3172e12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:40:55.403 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:55.402+0000 7f28dc0c9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:55.413 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:55.412+0000 7f3172e12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:40:55.537 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:55.536+0000 7f3172e12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:40:56.240 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.239+0000 7f28dc0c9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:56.418 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:56.416+0000 7f3172e12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:40:56.532 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.531+0000 7f28dc0c9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:56.589 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.588+0000 7f28dc0c9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:56.702 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.701+0000 7f28dc0c9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:56.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:56.723+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 146) v1 ==== 88957+0+0 (secure 0 0 0) 0x7efecc054c70 con 0x7efed0069d50 2023-10-21T17:40:56.725 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:56.724+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 msgr2=0x7efeb406d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:40:56.726 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:56.724+0000 7efec5ffb640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/596989303,v1:172.21.15.124:6833/596989303] conn(0x7efeb406b100 0x7efeb406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:56.732 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:56.731+0000 7f3172e12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:40:56.796 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:56.795+0000 7f3172e12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:40:56.888 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.886+0000 7f285bb53640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.888 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.886+0000 7f285bb53640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.888 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.886+0000 7f285bb53640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.889 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.886+0000 7f285bb53640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.889 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.886+0000 7f285bb53640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.889 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.887+0000 7f285930e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.890 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.887+0000 7f285930e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.890 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.887+0000 7f285930e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.890 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.887+0000 7f285930e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.890 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:40:56.887+0000 7f285930e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:40:56.922 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:40:56.921+0000 7f3172e12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:40:57.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.735+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mgrmap(e 147) v1 ==== 89084+0+0 (secure 0 0 0) 0x7efecc056380 con 0x7efed0069d50 2023-10-21T17:40:57.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.736+0000 7efec5ffb640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7efeb4035800 0x7efeb4075650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:57.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.736+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 --> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efeb40283b0 con 0x7efeb4035800 2023-10-21T17:40:57.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.740+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7efeb4035800 0x7efeb4075650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:57.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.744+0000 7efed494b640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7efeb4035800 0x7efeb4075650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efed015a940 tx=0x7efec0009040 comp rx=0 tx=0).ready entity=mgr.10150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:57.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.907+0000 7efec5ffb640 1 -- 172.21.15.53:0/1151354987 <== mgr.10150 v2:172.21.15.124:6832/1070399472 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efeb40283b0 con 0x7efeb4035800 2023-10-21T17:40:57.909 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule set for path snap_test_dir1 2023-10-21T17:40:57.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7efeb4035800 msgr2=0x7efeb4075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:57.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7efeb4035800 0x7efeb4075650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efed015a940 tx=0x7efec0009040 comp rx=0 tx=0).stop 2023-10-21T17:40:57.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 msgr2=0x7efed00cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:57.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 --2- 172.21.15.53:0/1151354987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efed0069d50 0x7efed00cd590 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7efecc00e870 tx=0x7efecc00ed30 comp rx=0 tx=0).stop 2023-10-21T17:40:57.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed594d640 1 -- 172.21.15.53:0/1151354987 reap_dead start 2023-10-21T17:40:57.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 shutdown_connections 2023-10-21T17:40:57.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 >> 172.21.15.53:0/1151354987 conn(0x7efed00cf8c0 msgr2=0x7efed015c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:57.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 shutdown_connections 2023-10-21T17:40:57.914 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:57.911+0000 7efed73b0640 1 -- 172.21.15.53:0/1151354987 wait complete. 2023-10-21T17:40:57.934 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-21T17:40:57.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-21T17:40:58.168 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.167+0000 7f5742c25640 1 Processor -- start 2023-10-21T17:40:58.169 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.167+0000 7f5742c25640 1 -- start start 2023-10-21T17:40:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5742c25640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f573c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.170 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5742c25640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c159db0 con 0x7f573c1599d0 2023-10-21T17:40:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5742c25640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c071fe0 con 0x7f573c0b3c50 2023-10-21T17:40:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5742c25640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0b44c0 con 0x7f573c0cb5e0 2023-10-21T17:40:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5741c23640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f573c1599d0 0x7f573c0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47366/0 (socket says 172.21.15.53:47366) 2023-10-21T17:40:58.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.169+0000 7f5741c23640 1 -- 172.21.15.53:0/2467709409 learned_addr learned my addr 172.21.15.53:0/2467709409 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:40:58.173 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5741422640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f573c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217008169 0 0) 0x7f5738001110 con 0x7f573c1599d0 2023-10-21T17:40:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728005730 con 0x7f573c1599d0 2023-10-21T17:40:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680073495 0 0) 0x7f5728005730 con 0x7f573c1599d0 2023-10-21T17:40:58.174 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 msgr2=0x7f573c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f573c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 >> v1:172.21.15.124:6789/0 conn(0x7f573c0b3c50 legacy=0x7f573c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0713d0 con 0x7f573c1599d0 2023-10-21T17:40:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5738001800 con 0x7f573c1599d0 2023-10-21T17:40:58.175 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 >> v1:172.21.15.53:6789/0 conn(0x7f573c1599d0 legacy=0x7f573c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f5728005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5728006180 0x7f57280069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f5728007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0b44c0 con 0x7f5728006f10 2023-10-21T17:40:58.176 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572800dcd0 con 0x7f5728006180 2023-10-21T17:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.170+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572800de90 con 0x7f573c0cb5e0 2023-10-21T17:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f5728007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5741422640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5728006180 0x7f57280069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5741c23640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f5728005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.177 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 -- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 msgr2=0x7f5728005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f5728005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 -- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5728006180 msgr2=0x7f57280069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5728006180 0x7f57280069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5742424640 1 -- 172.21.15.53:0/2467709409 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728005730 con 0x7f5728006f10 2023-10-21T17:40:58.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.171+0000 7f5741c23640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0cb5e0 0x7f5728005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.172+0000 7f5742424640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f5728007390 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f5734001e50 tx=0x7f5728007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e238aa8463c69e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.172+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734003b70 con 0x7f5728006f10 2023-10-21T17:40:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.172+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734003cd0 con 0x7f5728006f10 2023-10-21T17:40:58.179 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.172+0000 7f5740c21640 1 -- 172.21.15.53:0/2467709409 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734005310 con 0x7f5728006f10 2023-10-21T17:40:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 -- 172.21.15.53:0/2467709409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 msgr2=0x7f5728007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 --2- 172.21.15.53:0/2467709409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f5728007390 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f5734001e50 tx=0x7f5728007a00 comp rx=0 tx=0).stop 2023-10-21T17:40:58.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742424640 1 -- 172.21.15.53:0/2467709409 reap_dead start 2023-10-21T17:40:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 -- 172.21.15.53:0/2467709409 shutdown_connections 2023-10-21T17:40:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 -- 172.21.15.53:0/2467709409 >> 172.21.15.53:0/2467709409 conn(0x7f573c06c050 msgr2=0x7f573c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 -- 172.21.15.53:0/2467709409 shutdown_connections 2023-10-21T17:40:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.173+0000 7f5742c25640 1 -- 172.21.15.53:0/2467709409 wait complete. 2023-10-21T17:40:58.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.175+0000 7f5742c25640 1 Processor -- start 2023-10-21T17:40:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.175+0000 7f5742c25640 1 -- start start 2023-10-21T17:40:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 0x7f573c1e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b3090 0x7f573c1e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1e5440 con 0x7f5728006f10 2023-10-21T17:40:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1f38e0 con 0x7f573c0b3090 2023-10-21T17:40:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.176+0000 7f5742c25640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1e4830 con 0x7f573c0b2c10 2023-10-21T17:40:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40956/0 (socket says 172.21.15.53:40956) 2023-10-21T17:40:58.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 -- 172.21.15.53:0/518132887 learned_addr learned my addr 172.21.15.53:0/518132887 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741422640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 0x7f573c1e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 -- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 msgr2=0x7f573c1e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 0x7f573c1e5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 -- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b3090 msgr2=0x7f573c1e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b3090 0x7f573c1e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 -- 172.21.15.53:0/518132887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c1e4b10 con 0x7f5728006f10 2023-10-21T17:40:58.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741422640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 0x7f573c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:58.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.177+0000 7f5741c23640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f573800d820 tx=0x7f573800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.178+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380074e0 con 0x7f5728006f10 2023-10-21T17:40:58.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.178+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738007640 con 0x7f5728006f10 2023-10-21T17:40:58.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.178+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380049e0 con 0x7f5728006f10 2023-10-21T17:40:58.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.178+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c1e4df0 con 0x7f5728006f10 2023-10-21T17:40:58.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.178+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c1e4500 con 0x7f5728006f10 2023-10-21T17:40:58.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.179+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c074150 con 0x7f5728006f10 2023-10-21T17:40:58.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.180+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 147) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f5738004b40 con 0x7f5728006f10 2023-10-21T17:40:58.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.181+0000 7f57327fc640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 0x7f572006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.181+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f5738088e10 con 0x7f5728006f10 2023-10-21T17:40:58.189 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.181+0000 7f5741422640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 0x7f572006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.182+0000 7f5741422640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 0x7f572006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f573c1e7600 tx=0x7f572c008040 comp rx=0 tx=0).ready entity=mgr.10150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.185+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f573808c030 con 0x7f5728006f10 2023-10-21T17:40:58.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.404+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 --> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f573c071990 con 0x7f572006ad40 2023-10-21T17:40:58.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.546+0000 7f57327fc640 1 -- 172.21.15.53:0/518132887 <== mgr.10150 v2:172.21.15.124:6832/1070399472 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f572c012920 con 0x7f572006ad40 2023-10-21T17:40:58.548 INFO:teuthology.orchestra.run.smithi053.stdout:Retention added to path snap_test_dir1 2023-10-21T17:40:58.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.550+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 msgr2=0x7f572006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.550+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 0x7f572006d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f573c1e7600 tx=0x7f572c008040 comp rx=0 tx=0).stop 2023-10-21T17:40:58.552 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.550+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 msgr2=0x7f573c1f33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.550+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f573800d820 tx=0x7f573800dce0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 shutdown_connections 2023-10-21T17:40:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f573c0b3090 0x7f573c1e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f573c0b2c10 0x7f573c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f572006ad40 0x7f572006d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 --2- 172.21.15.53:0/518132887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5728006f10 0x7f573c1f33a0 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 >> 172.21.15.53:0/518132887 conn(0x7f573c06c050 msgr2=0x7f573c158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:58.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 shutdown_connections 2023-10-21T17:40:58.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.551+0000 7f5742c25640 1 -- 172.21.15.53:0/518132887 wait complete. 2023-10-21T17:40:58.575 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-21T17:40:58.575 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-21-17_41_00 in ~2s... 2023-10-21T17:40:58.576 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-21T17:40:58.576 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-21T17:40:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.805+0000 7effe71c9640 1 Processor -- start 2023-10-21T17:40:58.807 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.806+0000 7effe71c9640 1 -- start start 2023-10-21T17:40:58.808 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effe015ae30 0x7effe00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7effe00cd470 0x7effe00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00cde60 con 0x7effe015ae30 2023-10-21T17:40:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00cebf0 con 0x7effe00cd470 2023-10-21T17:40:58.809 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.807+0000 7effe71c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cbc60 con 0x7effe00af920 2023-10-21T17:40:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effe015ae30 0x7effe00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7effe015ae30 0x7effe00cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40964/0 (socket says 172.21.15.53:40964) 2023-10-21T17:40:58.810 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 -- 172.21.15.53:0/1329093477 learned_addr learned my addr 172.21.15.53:0/1329093477 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:58.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effdffff640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe5766640 1 --2- 172.21.15.53:0/1329093477 >> v2:172.21.15.124:3300/0 conn(0x7effe00cd470 0x7effe00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 -- 172.21.15.53:0/1329093477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 msgr2=0x7effe00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:58.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 -- 172.21.15.53:0/1329093477 >> v2:172.21.15.124:3300/0 conn(0x7effe00cd470 msgr2=0x7effe00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 --2- 172.21.15.53:0/1329093477 >> v2:172.21.15.124:3300/0 conn(0x7effe00cd470 0x7effe00cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.808+0000 7effe4f65640 1 -- 172.21.15.53:0/1329093477 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00cbf40 con 0x7effe015ae30 2023-10-21T17:40:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.809+0000 7effe4f65640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe015ae30 0x7effe00cf640 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7effe0072500 tx=0x7effd402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51007e96ab3067 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.810+0000 7effdf7fe640 1 -- 172.21.15.53:0/1329093477 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4031860 con 0x7effe015ae30 2023-10-21T17:40:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.810+0000 7effdf7fe640 1 -- 172.21.15.53:0/1329093477 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd40319c0 con 0x7effe015ae30 2023-10-21T17:40:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.810+0000 7effdf7fe640 1 -- 172.21.15.53:0/1329093477 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4031d50 con 0x7effe015ae30 2023-10-21T17:40:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.810+0000 7effe71c9640 1 -- 172.21.15.53:0/1329093477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe015ae30 msgr2=0x7effe00cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.810+0000 7effe71c9640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe015ae30 0x7effe00cf640 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7effe0072500 tx=0x7effd402b900 comp rx=0 tx=0).stop 2023-10-21T17:40:58.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 -- 172.21.15.53:0/1329093477 shutdown_connections 2023-10-21T17:40:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 --2- 172.21.15.53:0/1329093477 >> v2:172.21.15.124:3300/0 conn(0x7effe00cd470 0x7effe00cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 --2- 172.21.15.53:0/1329093477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe015ae30 0x7effe00cf640 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 -- 172.21.15.53:0/1329093477 >> 172.21.15.53:0/1329093477 conn(0x7effe006e650 msgr2=0x7effe006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:58.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 -- 172.21.15.53:0/1329093477 shutdown_connections 2023-10-21T17:40:58.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.811+0000 7effe71c9640 1 -- 172.21.15.53:0/1329093477 wait complete. 2023-10-21T17:40:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.813+0000 7effe71c9640 1 Processor -- start 2023-10-21T17:40:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.813+0000 7effe71c9640 1 -- start start 2023-10-21T17:40:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effe015ae30 0x7effe014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00c7a00 con 0x7effe00cd470 2023-10-21T17:40:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0151b00 con 0x7effe015ae30 2023-10-21T17:40:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effe71c9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe015d430 con 0x7effe00af920 2023-10-21T17:40:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effdffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effdffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40968/0 (socket says 172.21.15.53:40968) 2023-10-21T17:40:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.814+0000 7effdffff640 1 -- 172.21.15.53:0/344234469 learned_addr learned my addr 172.21.15.53:0/344234469 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effe5766640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effe015ae30 0x7effe014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effe4f65640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 -- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 msgr2=0x7effe015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 -- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effe015ae30 msgr2=0x7effe014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effe015ae30 0x7effe014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 -- 172.21.15.53:0/344234469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40094d0 con 0x7effe00cd470 2023-10-21T17:40:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effe4f65640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:40:58.821 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdffff640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d800 tx=0x7effcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc0096a0 con 0x7effe00cd470 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc009800 con 0x7effe00cd470 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc009990 con 0x7effe00cd470 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.815+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe015c880 con 0x7effe00cd470 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.816+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe015d0c0 con 0x7effe00cd470 2023-10-21T17:40:58.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.817+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe004e160 con 0x7effe00cd470 2023-10-21T17:40:58.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.818+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 147) v1 ==== 89084+0+0 (secure 0 0 0) 0x7effcc009af0 con 0x7effe00cd470 2023-10-21T17:40:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.819+0000 7effdd7fa640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 0x7effc406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.819+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7effcc089bc0 con 0x7effe00cd470 2023-10-21T17:40:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.819+0000 7effe4f65640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 0x7effc406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:58.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.821+0000 7effe4f65640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 0x7effc406d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7effd4036000 tx=0x7effd403c040 comp rx=0 tx=0).ready entity=mgr.10150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:58.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:58.824+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effcc054af0 con 0x7effe00cd470 2023-10-21T17:40:59.039 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.038+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 --> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effe00cee40 con 0x7effc406ad10 2023-10-21T17:40:59.056 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.055+0000 7effdd7fa640 1 -- 172.21.15.53:0/344234469 <== mgr.10150 v2:172.21.15.124:6832/1070399472 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7effd403a420 con 0x7effc406ad10 2023-10-21T17:40:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:40:59.057 INFO:teuthology.orchestra.run.smithi053.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-21T17:40:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.057+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 msgr2=0x7effc406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 0x7effc406d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7effd4036000 tx=0x7effd403c040 comp rx=0 tx=0).stop 2023-10-21T17:40:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 msgr2=0x7effe01f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d800 tx=0x7effcc00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 shutdown_connections 2023-10-21T17:40:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7effe015ae30 0x7effe014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7effc406ad10 0x7effc406d1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effe00cd470 0x7effe01f17f0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 --2- 172.21.15.53:0/344234469 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7effe00af920 0x7effe015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 >> 172.21.15.53:0/344234469 conn(0x7effe006e650 msgr2=0x7effe006c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 shutdown_connections 2023-10-21T17:40:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.058+0000 7effe71c9640 1 -- 172.21.15.53:0/344234469 wait complete. 2023-10-21T17:40:59.080 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-21T17:40:59.080 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-21T17:40:59.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-21T17:40:59.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.310+0000 7fa1061f4640 1 Processor -- start 2023-10-21T17:40:59.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.310+0000 7fa1061f4640 1 -- start start 2023-10-21T17:40:59.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1061f4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa10015cae0 0x7fa1000cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1061f4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa1000c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1061f4640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1000ccbc0 con 0x7fa1000c8b40 2023-10-21T17:40:59.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1061f4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1000c9510 con 0x7fa10015cae0 2023-10-21T17:40:59.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1061f4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000ce1f0 con 0x7fa1000b0680 2023-10-21T17:40:59.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1051f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa10015cae0 0x7fa1000cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:59.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1051f2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa10015cae0 0x7fa1000cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40976/0 (socket says 172.21.15.53:40976) 2023-10-21T17:40:59.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.312+0000 7fa1051f2640 1 -- 172.21.15.53:0/2914114650 learned_addr learned my addr 172.21.15.53:0/2914114650 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:59.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.313+0000 7fa1051f2640 1 -- 172.21.15.53:0/2914114650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 msgr2=0x7fa1000c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:59.316 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.313+0000 7fa1051f2640 1 --2- 172.21.15.53:0/2914114650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa1000c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.313+0000 7fa1051f2640 1 -- 172.21.15.53:0/2914114650 >> v1:172.21.15.124:6789/0 conn(0x7fa1000c8b40 legacy=0x7fa1000aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:40:59.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.313+0000 7fa1051f2640 1 -- 172.21.15.53:0/2914114650 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1000cd590 con 0x7fa10015cae0 2023-10-21T17:40:59.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.314+0000 7fa1051f2640 1 --2- 172.21.15.53:0/2914114650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10015cae0 0x7fa1000cd050 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009870 tx=0x7fa0fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604c8fe683c63727 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:59.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.314+0000 7fa0f7fff640 1 -- 172.21.15.53:0/2914114650 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc031640 con 0x7fa10015cae0 2023-10-21T17:40:59.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.314+0000 7fa0f7fff640 1 -- 172.21.15.53:0/2914114650 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc0317a0 con 0x7fa10015cae0 2023-10-21T17:40:59.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.314+0000 7fa0f7fff640 1 -- 172.21.15.53:0/2914114650 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc031b30 con 0x7fa10015cae0 2023-10-21T17:40:59.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 -- 172.21.15.53:0/2914114650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10015cae0 msgr2=0x7fa1000cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 --2- 172.21.15.53:0/2914114650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10015cae0 0x7fa1000cd050 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009870 tx=0x7fa0fc02b8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 -- 172.21.15.53:0/2914114650 shutdown_connections 2023-10-21T17:40:59.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 --2- 172.21.15.53:0/2914114650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa1000c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 --2- 172.21.15.53:0/2914114650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa10015cae0 0x7fa1000cd050 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 -- 172.21.15.53:0/2914114650 >> 172.21.15.53:0/2914114650 conn(0x7fa10006c050 msgr2=0x7fa10006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:59.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 -- 172.21.15.53:0/2914114650 shutdown_connections 2023-10-21T17:40:59.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.315+0000 7fa1061f4640 1 -- 172.21.15.53:0/2914114650 wait complete. 2023-10-21T17:40:59.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.317+0000 7fa1061f4640 1 Processor -- start 2023-10-21T17:40:59.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.317+0000 7fa1061f4640 1 -- start start 2023-10-21T17:40:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa10015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa10015cae0 0x7fa1000b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000c64a0 con 0x7fa1000c8b40 2023-10-21T17:40:59.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000c4920 con 0x7fa10015cae0 2023-10-21T17:40:59.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1061f4640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000c5b70 con 0x7fa1000b0680 2023-10-21T17:40:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.318+0000 7fa1049f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:59.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:40992/0 (socket says 172.21.15.53:40992) 2023-10-21T17:40:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 -- 172.21.15.53:0/1631444441 learned_addr learned my addr 172.21.15.53:0/1631444441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:40:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 -- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 msgr2=0x7fa10015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1051f2640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa10015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1059f3640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa10015cae0 0x7fa1000b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:59.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa10015f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 -- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa10015cae0 msgr2=0x7fa1000b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa10015cae0 0x7fa1000b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 -- 172.21.15.53:0/1631444441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0094d0 con 0x7fa1000c8b40 2023-10-21T17:40:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.319+0000 7fa1049f1640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c430 tx=0x7fa0f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:59.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.320+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011790 con 0x7fa1000c8b40 2023-10-21T17:40:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.320+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f00118f0 con 0x7fa1000c8b40 2023-10-21T17:40:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.320+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011ce0 con 0x7fa1000c8b40 2023-10-21T17:40:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.320+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1000c5e50 con 0x7fa1000c8b40 2023-10-21T17:40:59.328 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.320+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10015f830 con 0x7fa1000c8b40 2023-10-21T17:40:59.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.321+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1000c6610 con 0x7fa1000c8b40 2023-10-21T17:40:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.322+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 147) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa0f001c460 con 0x7fa1000c8b40 2023-10-21T17:40:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.323+0000 7fa0f5ffb640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 0x7fa0e406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:40:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.323+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7fa0f0089610 con 0x7fa1000c8b40 2023-10-21T17:40:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.324+0000 7fa1051f2640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 0x7fa0e406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:40:59.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.325+0000 7fa1051f2640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 0x7fa0e406d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc036000 tx=0x7fa0fc03c040 comp rx=0 tx=0).ready entity=mgr.10150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:40:59.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.327+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0f00545d0 con 0x7fa1000c8b40 2023-10-21T17:40:59.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.407+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 148) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fa0f00549a0 con 0x7fa1000c8b40 2023-10-21T17:40:59.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.558+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 --> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1000cc570 con 0x7fa0e406ad10 2023-10-21T17:40:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.709+0000 7fa0f5ffb640 1 -- 172.21.15.53:0/1631444441 <== mgr.10150 v2:172.21.15.124:6832/1070399472 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa0fc031a90 con 0x7fa0e406ad10 2023-10-21T17:40:59.711 INFO:teuthology.orchestra.run.smithi053.stdout:Schedule removed for path snap_test_dir1 2023-10-21T17:40:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 msgr2=0x7fa0e406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 0x7fa0e406d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc036000 tx=0x7fa0fc03c040 comp rx=0 tx=0).stop 2023-10-21T17:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 msgr2=0x7fa1000b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c430 tx=0x7fa0f000c8f0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 shutdown_connections 2023-10-21T17:40:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa10015cae0 0x7fa1000b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa0e406ad10 0x7fa0e406d1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1000c8b40 0x7fa1000b0d50 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 --2- 172.21.15.53:0/1631444441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1000b0680 0x7fa10015f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:40:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 >> 172.21.15.53:0/1631444441 conn(0x7fa10006c050 msgr2=0x7fa1000ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:40:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 shutdown_connections 2023-10-21T17:40:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:40:59.713+0000 7fa1061f4640 1 -- 172.21.15.53:0/1631444441 wait complete. 2023-10-21T17:40:59.735 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-21T17:40:59.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:59.736 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-21T17:40:59.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:40:59.799 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-21T17:40:59.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-21T17:41:00.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.094+0000 7f7c4ffff640 1 Processor -- start 2023-10-21T17:41:00.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.094+0000 7f7c4ffff640 1 -- start start 2023-10-21T17:41:00.096 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.095+0000 7f7c4ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.095+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.095+0000 7f7c4ffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c500d2870 con 0x7f7c500d0b90 2023-10-21T17:41:00.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.095+0000 7f7c4ffff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c500d1150 con 0x7f7c5015ccb0 2023-10-21T17:41:00.097 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.095+0000 7f7c4ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c500d1430 con 0x7f7c500d00e0 2023-10-21T17:41:00.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.096+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.096+0000 7f7c4effd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.096+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:51092/0 (socket says 172.21.15.53:51092) 2023-10-21T17:41:00.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.096+0000 7f7c4effd640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:35300/0 (socket says 172.21.15.53:35300) 2023-10-21T17:41:00.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.096+0000 7f7c4f7fe640 1 -- 172.21.15.53:0/207829391 learned_addr learned my addr 172.21.15.53:0/207829391 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:00.099 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.097+0000 7f7c4f7fe640 1 -- 172.21.15.53:0/207829391 >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 msgr2=0x7f7c500d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:00.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.097+0000 7f7c4f7fe640 1 --2- 172.21.15.53:0/207829391 >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.097+0000 7f7c4f7fe640 1 -- 172.21.15.53:0/207829391 >> v1:172.21.15.124:6789/0 conn(0x7f7c500d0b90 legacy=0x7f7c500cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.097+0000 7f7c4f7fe640 1 -- 172.21.15.53:0/207829391 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500d1710 con 0x7f7c500d00e0 2023-10-21T17:41:00.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.097+0000 7f7c4effd640 1 --2- 172.21.15.53:0/207829391 >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:41:00.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.098+0000 7f7c4f7fe640 1 --2- 172.21.15.53:0/207829391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1ad5f9b9433703 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:00.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.098+0000 7f7c4dffb640 1 -- 172.21.15.53:0/207829391 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48031640 con 0x7f7c500d00e0 2023-10-21T17:41:00.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4dffb640 1 -- 172.21.15.53:0/207829391 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c480317a0 con 0x7f7c500d00e0 2023-10-21T17:41:00.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4dffb640 1 -- 172.21.15.53:0/207829391 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48031b30 con 0x7f7c500d00e0 2023-10-21T17:41:00.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 -- 172.21.15.53:0/207829391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 msgr2=0x7f7c500b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/207829391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4802b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 -- 172.21.15.53:0/207829391 shutdown_connections 2023-10-21T17:41:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/207829391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c500d00e0 0x7f7c500b6060 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/207829391 >> v2:172.21.15.124:3300/0 conn(0x7f7c5015ccb0 0x7f7c500d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.099+0000 7f7c4ffff640 1 -- 172.21.15.53:0/207829391 >> 172.21.15.53:0/207829391 conn(0x7f7c50071660 msgr2=0x7f7c50071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.100+0000 7f7c4ffff640 1 -- 172.21.15.53:0/207829391 shutdown_connections 2023-10-21T17:41:00.104 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.100+0000 7f7c4ffff640 1 -- 172.21.15.53:0/207829391 wait complete. 2023-10-21T17:41:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.101+0000 7f7c4ffff640 1 Processor -- start 2023-10-21T17:41:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.102+0000 7f7c4ffff640 1 -- start start 2023-10-21T17:41:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c500d00e0 0x7f7c501f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c5015ccb0 0x7f7c500b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501efdd0 con 0x7f7c500d0b90 2023-10-21T17:41:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501f3310 con 0x7f7c500d00e0 2023-10-21T17:41:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5015edd0 con 0x7f7c5015ccb0 2023-10-21T17:41:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41006/0 (socket says 172.21.15.53:41006) 2023-10-21T17:41:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 -- 172.21.15.53:0/365342658 learned_addr learned my addr 172.21.15.53:0/365342658 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 -- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c5015ccb0 msgr2=0x7f7c500b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c5015ccb0 0x7f7c500b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4e7fc640 1 -- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c500d00e0 msgr2=0x7f7c501f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.103+0000 7f7c4effd640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c500d00e0 0x7f7c501f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c4e7fc640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c500d00e0 0x7f7c501f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c4e7fc640 1 -- 172.21.15.53:0/365342658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40009a70 con 0x7f7c500d0b90 2023-10-21T17:41:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c4e7fc640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800c430 tx=0x7f7c3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380052d0 con 0x7f7c500d0b90 2023-10-21T17:41:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c38005450 con 0x7f7c500d0b90 2023-10-21T17:41:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38007260 con 0x7f7c500d0b90 2023-10-21T17:41:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c480094d0 con 0x7f7c500d0b90 2023-10-21T17:41:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.104+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c501f3580 con 0x7f7c500d0b90 2023-10-21T17:41:00.111 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.105+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50074150 con 0x7f7c500d0b90 2023-10-21T17:41:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.106+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 148) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f7c380073c0 con 0x7f7c500d0b90 2023-10-21T17:41:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.107+0000 7f7c47fff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 0x7f7c3006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.108+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f7c38088fb0 con 0x7f7c500d0b90 2023-10-21T17:41:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.108+0000 7f7c4effd640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 0x7f7c3006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.109+0000 7f7c4effd640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 0x7f7c3006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c40007520 tx=0x7f7c4000b040 comp rx=0 tx=0).ready entity=mgr.10150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.112+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c38053be0 con 0x7f7c500d0b90 2023-10-21T17:41:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.357+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f7c500c9950 con 0x7f7c500d0b90 2023-10-21T17:41:00.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.421+0000 7f7c47fff640 1 -- 172.21.15.53:0/365342658 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v149) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7c38057200 con 0x7f7c500d0b90 2023-10-21T17:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 msgr2=0x7f7c3006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 0x7f7c3006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c40007520 tx=0x7f7c4000b040 comp rx=0 tx=0).stop 2023-10-21T17:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 msgr2=0x7f7c500b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800c430 tx=0x7f7c3800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 shutdown_connections 2023-10-21T17:41:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c5015ccb0 0x7f7c500b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7c3006afe0 0x7f7c3006d4a0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c40007520 tx=0x7f7c4000b040 comp rx=0 tx=0).stop 2023-10-21T17:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c500d0b90 0x7f7c500b37f0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 --2- 172.21.15.53:0/365342658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c500d00e0 0x7f7c501f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 >> 172.21.15.53:0/365342658 conn(0x7f7c50071660 msgr2=0x7f7c5006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.425+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 shutdown_connections 2023-10-21T17:41:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.426+0000 7f7c4ffff640 1 -- 172.21.15.53:0/365342658 wait complete. 2023-10-21T17:41:00.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-21T17:41:00.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:41:00.574 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-21T17:41:00.667 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-21T17:41:00.668 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:41:00.668 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:00.722 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:41:00.722 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:00.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:41:00.952 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:00.950+0000 7f274f83c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:41:00.989 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:00.987+0000 7f21b7e32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-21T17:41:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.992+0000 7fd18e997640 1 Processor -- start 2023-10-21T17:41:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.992+0000 7fd18e997640 1 -- start start 2023-10-21T17:41:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd18e997640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd18815aeb0 0x7fd1880cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd18e997640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cb580 con 0x7fd18815aad0 2023-10-21T17:41:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd18e997640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cc0e0 con 0x7fd1880cc570 2023-10-21T17:41:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd18e997640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880b2390 con 0x7fd18815aeb0 2023-10-21T17:41:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd187fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd18815aad0 0x7fd1880ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47388/0 (socket says 172.21.15.53:47388) 2023-10-21T17:41:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd187fff640 1 -- 172.21.15.53:0/2004437590 learned_addr learned my addr 172.21.15.53:0/2004437590 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd17ffff640 1 --2- 172.21.15.53:0/2004437590 >> v2:172.21.15.124:3300/0 conn(0x7fd18815aeb0 0x7fd1880cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.994+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362463063 0 0) 0x7fd1880cb580 con 0x7fd18815aad0 2023-10-21T17:41:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd174003520 con 0x7fd18815aad0 2023-10-21T17:41:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480133702 0 0) 0x7fd174003520 con 0x7fd18815aad0 2023-10-21T17:41:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 >> v2:172.21.15.124:3300/0 conn(0x7fd18815aeb0 msgr2=0x7fd1880cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 --2- 172.21.15.53:0/2004437590 >> v2:172.21.15.124:3300/0 conn(0x7fd18815aeb0 0x7fd1880cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 >> v1:172.21.15.124:6789/0 conn(0x7fd1880cc570 legacy=0x7fd1880cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880cb7b0 con 0x7fd18815aad0 2023-10-21T17:41:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fd1780017f0 con 0x7fd18815aad0 2023-10-21T17:41:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 >> v1:172.21.15.53:6789/0 conn(0x7fd18815aad0 legacy=0x7fd1880ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1740039c0 0x7fd174003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1740047c0 0x7fd174004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 0x7fd17400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b2390 con 0x7fd174005180 2023-10-21T17:41:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17400c060 con 0x7fd1740039c0 2023-10-21T17:41:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.995+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17400c220 con 0x7fd1740047c0 2023-10-21T17:41:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 0x7fd17400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd187fff640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1740039c0 0x7fd174003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd17ffff640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1740047c0 0x7fd174004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 -- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1740047c0 msgr2=0x7fd174004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1740047c0 0x7fd174004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 -- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1740039c0 msgr2=0x7fd174003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1740039c0 0x7fd174003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd18cf34640 1 -- 172.21.15.53:0/2004437590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174003520 con 0x7fd174005180 2023-10-21T17:41:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.996+0000 7fd187fff640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1740039c0 0x7fd174003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.997+0000 7fd18cf34640 1 -- 172.21.15.53:0/2004437590 reap_dead start 2023-10-21T17:41:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.997+0000 7fd18cf34640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 0x7fd17400b980 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fd180001e50 tx=0x7fd1740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683edeadce20d9d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.997+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180003b70 con 0x7fd174005180 2023-10-21T17:41:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.997+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180003d10 con 0x7fd174005180 2023-10-21T17:41:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.997+0000 7fd1877fe640 1 -- 172.21.15.53:0/2004437590 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180005310 con 0x7fd174005180 2023-10-21T17:41:01.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 -- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 msgr2=0x7fd17400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 0x7fd17400b980 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fd180001e50 tx=0x7fd1740057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 -- 172.21.15.53:0/2004437590 shutdown_connections 2023-10-21T17:41:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 --2- 172.21.15.53:0/2004437590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd174005180 0x7fd17400b980 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 -- 172.21.15.53:0/2004437590 >> 172.21.15.53:0/2004437590 conn(0x7fd1880d09e0 msgr2=0x7fd1880d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 -- 172.21.15.53:0/2004437590 shutdown_connections 2023-10-21T17:41:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:00.998+0000 7fd18e997640 1 -- 172.21.15.53:0/2004437590 wait complete. 2023-10-21T17:41:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.000+0000 7fd18e997640 1 Processor -- start 2023-10-21T17:41:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.000+0000 7fd18e997640 1 -- start start 2023-10-21T17:41:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd174005180 0x7fd1880c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880c4990 0x7fd188158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880cedc0 con 0x7fd1880c4510 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b4860 con 0x7fd174005180 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd18e997640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b4b40 con 0x7fd1880c4990 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.001+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41030/0 (socket says 172.21.15.53:41030) 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 -- 172.21.15.53:0/3650995724 learned_addr learned my addr 172.21.15.53:0/3650995724 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd187fff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd174005180 0x7fd1880c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880c4990 msgr2=0x7fd188158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880c4990 0x7fd188158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd174005180 msgr2=0x7fd1880c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd174005180 0x7fd1880c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 -- 172.21.15.53:0/3650995724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880c4f60 con 0x7fd1880c4510 2023-10-21T17:41:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd17ffff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fd170011570 tx=0x7fd170011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.002+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170015930 con 0x7fd1880c4510 2023-10-21T17:41:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.003+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170015a90 con 0x7fd1880c4510 2023-10-21T17:41:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.003+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170015e20 con 0x7fd1880c4510 2023-10-21T17:41:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.003+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1880c5240 con 0x7fd1880c4510 2023-10-21T17:41:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.003+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18806c070 con 0x7fd1880c4510 2023-10-21T17:41:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.004+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd188067490 con 0x7fd1880c4510 2023-10-21T17:41:01.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.006+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd17000f460 con 0x7fd1880c4510 2023-10-21T17:41:01.014 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.004+0000 7f274f83c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:41:01.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.007+0000 7fd1857fa640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 0x7fd16806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.007+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7fd17008a360 con 0x7fd1880c4510 2023-10-21T17:41:01.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.007+0000 7fd187fff640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 msgr2=0x7fd16806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:01.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.007+0000 7fd187fff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 0x7fd16806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:01.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.012+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd170054fa0 con 0x7fd1880c4510 2023-10-21T17:41:01.052 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.050+0000 7f21b7e32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-21T17:41:01.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.208+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1880b4c80 con 0x7fd1880c4510 2023-10-21T17:41:01.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.208+0000 7fd187fff640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 msgr2=0x7fd16806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:01.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.208+0000 7fd187fff640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 0x7fd16806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:41:01.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.209+0000 7fd1857fa640 1 -- 172.21.15.53:0/3650995724 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v276) v1 ==== 74+0+17525 (secure 0 0 0) 0x7fd1700585c0 con 0x7fd1880c4510 2023-10-21T17:41:01.211 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:01.211 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":276,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:56.706711+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/612474845":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3975797466":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2074523274":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2461568637":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6832/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2747533012":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/410485344":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3555819866":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/1112463946":"2023-10-22T17:40:56.706577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:41:01.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 msgr2=0x7fd16806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:01.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 0x7fd16806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 msgr2=0x7fd188157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fd170011570 tx=0x7fd170011a30 comp rx=0 tx=0).stop 2023-10-21T17:41:01.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 shutdown_connections 2023-10-21T17:41:01.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1880c4990 0x7fd188158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd16806af90 0x7fd16806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.214 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1880c4510 0x7fd188157ae0 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 --2- 172.21.15.53:0/3650995724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd174005180 0x7fd1880c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.211+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 >> 172.21.15.53:0/3650995724 conn(0x7fd1880d09e0 msgr2=0x7fd18815f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:01.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.212+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 shutdown_connections 2023-10-21T17:41:01.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.212+0000 7fd18e997640 1 -- 172.21.15.53:0/3650995724 wait complete. 2023-10-21T17:41:01.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:40:39.912054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '274', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:40:40.993087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '275', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:41:01.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:41:01.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.435+0000 7f8af4875640 1 Processor -- start 2023-10-21T17:41:01.437 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.436+0000 7f8af4875640 1 -- start start 2023-10-21T17:41:01.438 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.437+0000 7f8af4875640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015a9f0 0x7f8af00ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.437+0000 7f8af4875640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8af00ae390 0x7f8af00cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.437+0000 7f8af4875640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00cb330 con 0x7f8af015e630 2023-10-21T17:41:01.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.437+0000 7f8af4875640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00661f0 con 0x7f8af00ae390 2023-10-21T17:41:01.439 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.437+0000 7f8af4875640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00cda00 con 0x7f8af015a9f0 2023-10-21T17:41:01.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.438+0000 7f8aeffff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f8af015e630 0x7f8af00cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47392/0 (socket says 172.21.15.53:47392) 2023-10-21T17:41:01.440 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.438+0000 7f8aeffff640 1 -- 172.21.15.53:0/3951858967 learned_addr learned my addr 172.21.15.53:0/3951858967 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:01.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.438+0000 7f8aeeffd640 1 --2- 172.21.15.53:0/3951858967 >> v2:172.21.15.124:3300/0 conn(0x7f8af00ae390 0x7f8af00cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.438+0000 7f8aef7fe640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015a9f0 0x7f8af00ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.439+0000 7f8aeeffd640 1 -- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015a9f0 msgr2=0x7f8af00ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.441 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.439+0000 7f8aeeffd640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015a9f0 0x7f8af00ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.439+0000 7f8aeeffd640 1 -- 172.21.15.53:0/3951858967 >> v1:172.21.15.53:6789/0 conn(0x7f8af015e630 legacy=0x7f8af00cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.439+0000 7f8aeeffd640 1 -- 172.21.15.53:0/3951858967 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00cdce0 con 0x7f8af00ae390 2023-10-21T17:41:01.442 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.440+0000 7f8aeeffd640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af00ae390 0x7f8af00cadf0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40097a0 tx=0x7f8ae402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e62b28e69fd496 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.443+0000 7f8aee7fc640 1 -- 172.21.15.53:0/3951858967 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae402d040 con 0x7f8af00ae390 2023-10-21T17:41:01.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.443+0000 7f8aee7fc640 1 -- 172.21.15.53:0/3951858967 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4031810 con 0x7f8af00ae390 2023-10-21T17:41:01.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8aee7fc640 1 -- 172.21.15.53:0/3951858967 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4031ba0 con 0x7f8af00ae390 2023-10-21T17:41:01.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 -- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af00ae390 msgr2=0x7f8af00cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af00ae390 0x7f8af00cadf0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40097a0 tx=0x7f8ae402b8b0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 -- 172.21.15.53:0/3951858967 shutdown_connections 2023-10-21T17:41:01.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af00ae390 0x7f8af00cadf0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 --2- 172.21.15.53:0/3951858967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015a9f0 0x7f8af00ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 -- 172.21.15.53:0/3951858967 >> 172.21.15.53:0/3951858967 conn(0x7f8af00b08a0 msgr2=0x7f8af00b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:01.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 -- 172.21.15.53:0/3951858967 shutdown_connections 2023-10-21T17:41:01.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.444+0000 7f8af4875640 1 -- 172.21.15.53:0/3951858967 wait complete. 2023-10-21T17:41:01.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.446+0000 7f8af4875640 1 Processor -- start 2023-10-21T17:41:01.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.446+0000 7f8af4875640 1 -- start start 2023-10-21T17:41:01.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af015a9f0 0x7f8af01598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015e630 0x7f8af0159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af006e240 con 0x7f8af00ae390 2023-10-21T17:41:01.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b16b0 con 0x7f8af015a9f0 2023-10-21T17:41:01.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8af4875640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00c80e0 con 0x7f8af015e630 2023-10-21T17:41:01.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.447+0000 7f8aef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41032/0 (socket says 172.21.15.53:41032) 2023-10-21T17:41:01.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 -- 172.21.15.53:0/2920641894 learned_addr learned my addr 172.21.15.53:0/2920641894 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:01.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 -- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015e630 msgr2=0x7f8af0159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015e630 0x7f8af0159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 -- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af015a9f0 msgr2=0x7f8af01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af015a9f0 0x7f8af01598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 -- 172.21.15.53:0/2920641894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4009400 con 0x7f8af00ae390 2023-10-21T17:41:01.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8aef7fe640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000e870 tx=0x7f8ae000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00094f0 con 0x7f8af00ae390 2023-10-21T17:41:01.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000c940 con 0x7f8af00ae390 2023-10-21T17:41:01.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.448+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af00c83c0 con 0x7f8af00ae390 2023-10-21T17:41:01.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.449+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000ccf0 con 0x7f8af00ae390 2023-10-21T17:41:01.454 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.449+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af00c8980 con 0x7f8af00ae390 2023-10-21T17:41:01.455 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.450+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af0074150 con 0x7f8af00ae390 2023-10-21T17:41:01.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.451+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f8ae0013470 con 0x7f8af00ae390 2023-10-21T17:41:01.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.451+0000 7f8ad7fff640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 0x7f8adc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.451+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f8ae0089c20 con 0x7f8af00ae390 2023-10-21T17:41:01.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.452+0000 7f8aeeffd640 1 -- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 msgr2=0x7f8adc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:01.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.452+0000 7f8aeeffd640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 0x7f8adc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:01.457 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.455+0000 7f274f83c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:41:01.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.455+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ae0054850 con 0x7f8af00ae390 2023-10-21T17:41:01.510 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.508+0000 7f274f83c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:41:01.527 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.526+0000 7f21b7e32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-21T17:41:01.559 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.558+0000 7f274f83c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:41:01.582 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.581+0000 7f21b7e32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-21T17:41:01.621 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.620+0000 7f274f83c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:41:01.634 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.633+0000 7f21b7e32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-21T17:41:01.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.647+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af006b670 con 0x7f8af00ae390 2023-10-21T17:41:01.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.648+0000 7f8ad7fff640 1 -- 172.21.15.53:0/2920641894 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v276) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ae00170a0 con 0x7f8af00ae390 2023-10-21T17:41:01.649 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 msgr2=0x7f8adc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 0x7f8adc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 msgr2=0x7f8af00c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000e870 tx=0x7f8ae000ed30 comp rx=0 tx=0).stop 2023-10-21T17:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 shutdown_connections 2023-10-21T17:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8af015e630 0x7f8af0159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8af015a9f0 0x7f8af01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f8adc06af90 0x7f8adc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 --2- 172.21.15.53:0/2920641894 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af00ae390 0x7f8af00c4880 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 >> 172.21.15.53:0/2920641894 conn(0x7f8af00b08a0 msgr2=0x7f8af00cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:01.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 shutdown_connections 2023-10-21T17:41:01.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.650+0000 7f8af4875640 1 -- 172.21.15.53:0/2920641894 wait complete. 2023-10-21T17:41:01.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:41:01.671 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.668+0000 7f274f83c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:41:01.700 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.698+0000 7f21b7e32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-21T17:41:01.752 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.751+0000 7f21b7e32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-21T17:41:01.780 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.778+0000 7f274f83c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:41:01.842 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.840+0000 7f274f83c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:41:01.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.867+0000 7f334c589640 1 Processor -- start 2023-10-21T17:41:01.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.868+0000 7f334c589640 1 -- start start 2023-10-21T17:41:01.870 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.869+0000 7f21b7e32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-21T17:41:01.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.869+0000 7f334c589640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33440cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.869+0000 7f334c589640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3344072480 con 0x7f334406a460 2023-10-21T17:41:01.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.869+0000 7f334c589640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334406f730 con 0x7f3344160280 2023-10-21T17:41:01.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.869+0000 7f334c589640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3344071ef0 con 0x7f33440712e0 2023-10-21T17:41:01.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f334ab26640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f334406a460 0x7f33440cdaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47400/0 (socket says 172.21.15.53:47400) 2023-10-21T17:41:01.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f334ab26640 1 -- 172.21.15.53:0/1073591374 learned_addr learned my addr 172.21.15.53:0/1073591374 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:01.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f3349b24640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33440cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123099225 0 0) 0x7f3344072480 con 0x7f334406a460 2023-10-21T17:41:01.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3330003520 con 0x7f334406a460 2023-10-21T17:41:01.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.870+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103219184 0 0) 0x7f3330003520 con 0x7f334406a460 2023-10-21T17:41:01.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 msgr2=0x7f33440cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33440cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 >> v1:172.21.15.124:6789/0 conn(0x7f3344160280 legacy=0x7f3344069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334406fa10 con 0x7f334406a460 2023-10-21T17:41:01.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f333c001a10 con 0x7f334406a460 2023-10-21T17:41:01.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 >> v1:172.21.15.53:6789/0 conn(0x7f334406a460 legacy=0x7f33440cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3330003ee0 0x7f33300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330004d00 0x7f3330005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344071ef0 con 0x7f3330003ee0 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333000b9e0 con 0x7f3330004d00 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333000bba0 con 0x7f33440712e0 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f3349b24640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3330003ee0 0x7f33300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f334ab26640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330004d00 0x7f3330005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.871+0000 7f334a325640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.872+0000 7f3349b24640 1 -- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 msgr2=0x7f33300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.872+0000 7f3349b24640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33440712e0 0x7f33300039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.872+0000 7f3349b24640 1 -- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330004d00 msgr2=0x7f3330005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.872+0000 7f3349b24640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330004d00 0x7f3330005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.872+0000 7f3349b24640 1 -- 172.21.15.53:0/1073591374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330003520 con 0x7f3330003ee0 2023-10-21T17:41:01.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.873+0000 7f3349b24640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3330003ee0 0x7f33300047c0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ac0 tx=0x7f33300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9a97143400f389 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333402c600 con 0x7f3330003ee0 2023-10-21T17:41:01.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333402c760 con 0x7f3330003ee0 2023-10-21T17:41:01.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f3349323640 1 -- 172.21.15.53:0/1073591374 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333402caf0 con 0x7f3330003ee0 2023-10-21T17:41:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 -- 172.21.15.53:0/1073591374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3330003ee0 msgr2=0x7f33300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 --2- 172.21.15.53:0/1073591374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3330003ee0 0x7f33300047c0 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ac0 tx=0x7f33300057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334ab26640 1 -- 172.21.15.53:0/1073591374 reap_dead start 2023-10-21T17:41:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 -- 172.21.15.53:0/1073591374 shutdown_connections 2023-10-21T17:41:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 -- 172.21.15.53:0/1073591374 >> 172.21.15.53:0/1073591374 conn(0x7f33440cf8c0 msgr2=0x7f33440cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 -- 172.21.15.53:0/1073591374 shutdown_connections 2023-10-21T17:41:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.874+0000 7f334c589640 1 -- 172.21.15.53:0/1073591374 wait complete. 2023-10-21T17:41:01.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.876+0000 7f334c589640 1 Processor -- start 2023-10-21T17:41:01.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.876+0000 7f334c589640 1 -- start start 2023-10-21T17:41:01.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 0x7f334414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f334415c090 0x7f33440c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334415f230 con 0x7f334415bc30 2023-10-21T17:41:01.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33441570e0 con 0x7f3330003ee0 2023-10-21T17:41:01.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f334c589640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334415e620 con 0x7f334415c090 2023-10-21T17:41:01.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f3349b24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f3349b24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41042/0 (socket says 172.21.15.53:41042) 2023-10-21T17:41:01.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.877+0000 7f3349b24640 1 -- 172.21.15.53:0/2097605420 learned_addr learned my addr 172.21.15.53:0/2097605420 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:01.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 -- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f334415c090 msgr2=0x7f33440c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f334415c090 0x7f33440c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 -- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 msgr2=0x7f334414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:01.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f334a325640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 0x7f334414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:01.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 0x7f334414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:01.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 -- 172.21.15.53:0/2097605420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334415e900 con 0x7f334415bc30 2023-10-21T17:41:01.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f334a325640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 0x7f334414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:01.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f3349b24640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ac0 tx=0x7f3334004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:01.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334003810 con 0x7f334415bc30 2023-10-21T17:41:01.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334003970 con 0x7f334415bc30 2023-10-21T17:41:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334415ebe0 con 0x7f334415bc30 2023-10-21T17:41:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.878+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334003d00 con 0x7f334415bc30 2023-10-21T17:41:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.879+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334415e310 con 0x7f334415bc30 2023-10-21T17:41:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.880+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334404e160 con 0x7f334415bc30 2023-10-21T17:41:01.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.881+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3334040450 con 0x7f334415bc30 2023-10-21T17:41:01.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.882+0000 7f333affd640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 0x7f332806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:01.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.882+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f33340abb90 con 0x7f334415bc30 2023-10-21T17:41:01.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.882+0000 7f334a325640 1 -- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 msgr2=0x7f332806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:01.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.882+0000 7f334a325640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 0x7f332806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:01.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:01.886+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33340766c0 con 0x7f334415bc30 2023-10-21T17:41:01.892 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.891+0000 7f274f83c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:41:01.940 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.939+0000 7f21b7e32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-21T17:41:01.949 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.948+0000 7f274f83c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:41:01.992 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:01.991+0000 7f21b7e32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-21T17:41:01.999 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:01.998+0000 7f274f83c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:41:02.052 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.050+0000 7f21b7e32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-21T17:41:02.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.077+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33440c2e60 con 0x7f334415bc30 2023-10-21T17:41:02.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.078+0000 7f333affd640 1 -- 172.21.15.53:0/2097605420 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v276) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3334003e60 con 0x7f334415bc30 2023-10-21T17:41:02.079 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:41:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.080+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 msgr2=0x7f332806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:02.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.080+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 0x7f332806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 msgr2=0x7f33440c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ac0 tx=0x7f3334004730 comp rx=0 tx=0).stop 2023-10-21T17:41:02.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 shutdown_connections 2023-10-21T17:41:02.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f334415c090 0x7f33440c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.085 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f332806af90 0x7f332806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f334415bc30 0x7f33440c7010 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 --2- 172.21.15.53:0/2097605420 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3330003ee0 0x7f334414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 >> 172.21.15.53:0/2097605420 conn(0x7f33440cf8c0 msgr2=0x7f3344160660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:02.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 shutdown_connections 2023-10-21T17:41:02.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.081+0000 7f334c589640 1 -- 172.21.15.53:0/2097605420 wait complete. 2023-10-21T17:41:02.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:41:02.106 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.104+0000 7f21b7e32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-21T17:41:02.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.308+0000 7f7f08f41640 1 Processor -- start 2023-10-21T17:41:02.309 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.308+0000 7f7f08f41640 1 -- start start 2023-10-21T17:41:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.309+0000 7f7f08f41640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f041601b0 0x7f7f040cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.309+0000 7f7f08f41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.311 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f08f41640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f040ce210 con 0x7f7f04072090 2023-10-21T17:41:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f08f41640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040cdee0 con 0x7f7f041601b0 2023-10-21T17:41:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f08f41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04161530 con 0x7f7f04071c30 2023-10-21T17:41:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f041601b0 0x7f7f040cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7f041601b0 0x7f7f040cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41054/0 (socket says 172.21.15.53:41054) 2023-10-21T17:41:02.312 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 -- 172.21.15.53:0/2940143972 learned_addr learned my addr 172.21.15.53:0/2940143972 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:02.313 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f01d9b640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 -- 172.21.15.53:0/2940143972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 msgr2=0x7f7f04069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:02.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 -- 172.21.15.53:0/2940143972 >> v1:172.21.15.124:6789/0 conn(0x7f7f04072090 legacy=0x7f7f040ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:02.314 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f0259c640 1 -- 172.21.15.53:0/2940143972 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04161810 con 0x7f7f041601b0 2023-10-21T17:41:02.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.310+0000 7f7f01d9b640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:02.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.312+0000 7f7f0259c640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f041601b0 0x7f7f040cd430 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009870 tx=0x7f7ef402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d39db4b6df362bb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:02.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.312+0000 7f7f0159a640 1 -- 172.21.15.53:0/2940143972 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031640 con 0x7f7f041601b0 2023-10-21T17:41:02.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.312+0000 7f7f0159a640 1 -- 172.21.15.53:0/2940143972 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef40317a0 con 0x7f7f041601b0 2023-10-21T17:41:02.315 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.312+0000 7f7f0159a640 1 -- 172.21.15.53:0/2940143972 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4031b30 con 0x7f7f041601b0 2023-10-21T17:41:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 -- 172.21.15.53:0/2940143972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f041601b0 msgr2=0x7f7f040cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f041601b0 0x7f7f040cd430 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009870 tx=0x7f7ef402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 -- 172.21.15.53:0/2940143972 shutdown_connections 2023-10-21T17:41:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.317 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 --2- 172.21.15.53:0/2940143972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f041601b0 0x7f7f040cd430 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 -- 172.21.15.53:0/2940143972 >> 172.21.15.53:0/2940143972 conn(0x7f7f0406d430 msgr2=0x7f7f0406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 -- 172.21.15.53:0/2940143972 shutdown_connections 2023-10-21T17:41:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.313+0000 7f7f08f41640 1 -- 172.21.15.53:0/2940143972 wait complete. 2023-10-21T17:41:02.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.315+0000 7f7f08f41640 1 Processor -- start 2023-10-21T17:41:02.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.315+0000 7f7f08f41640 1 -- start start 2023-10-21T17:41:02.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f041601b0 0x7f7f041f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040ac6e0 con 0x7f7f04072090 2023-10-21T17:41:02.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040abe40 con 0x7f7f041601b0 2023-10-21T17:41:02.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f08f41640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040ac120 con 0x7f7f04071c30 2023-10-21T17:41:02.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f01d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41058/0 (socket says 172.21.15.53:41058) 2023-10-21T17:41:02.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.316+0000 7f7f01d9b640 1 -- 172.21.15.53:0/1517823257 learned_addr learned my addr 172.21.15.53:0/1517823257 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 -- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 msgr2=0x7f7f04159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 -- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f041601b0 msgr2=0x7f7f041f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f041601b0 0x7f7f041f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 -- 172.21.15.53:0/1517823257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40094d0 con 0x7f7f04072090 2023-10-21T17:41:02.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f01d9b640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800ce00 tx=0x7f7ef800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:02.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800f890 con 0x7f7f04072090 2023-10-21T17:41:02.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef800f9f0 con 0x7f7f04072090 2023-10-21T17:41:02.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800fdc0 con 0x7f7f04072090 2023-10-21T17:41:02.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.317+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040ac400 con 0x7f7f04072090 2023-10-21T17:41:02.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.318+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f040b4d30 con 0x7f7f04072090 2023-10-21T17:41:02.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.319+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f041f2940 con 0x7f7f04072090 2023-10-21T17:41:02.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.320+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f7ef800b040 con 0x7f7f04072090 2023-10-21T17:41:02.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.321+0000 7f7ef2ffd640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 0x7f7ee406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.321+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f7ef808a640 con 0x7f7f04072090 2023-10-21T17:41:02.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.321+0000 7f7f0259c640 1 -- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 msgr2=0x7f7ee406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:02.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.321+0000 7f7f0259c640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 0x7f7ee406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:02.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.325+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f041f2940 con 0x7f7f04072090 2023-10-21T17:41:02.382 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.380+0000 7f274f83c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:41:02.510 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.509+0000 7f21b7e32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-21T17:41:02.520 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.519+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f041f2940 con 0x7f7f04072090 2023-10-21T17:41:02.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.520+0000 7f7ef2ffd640 1 -- 172.21.15.53:0/1517823257 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v276) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ef801d020 con 0x7f7f04072090 2023-10-21T17:41:02.521 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:41:02.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 msgr2=0x7f7ee406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:02.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 0x7f7ee406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 msgr2=0x7f7f040b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800ce00 tx=0x7f7ef800a650 comp rx=0 tx=0).stop 2023-10-21T17:41:02.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 shutdown_connections 2023-10-21T17:41:02.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f041601b0 0x7f7f041f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f7ee406af90 0x7f7ee406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f04072090 0x7f7f040b47f0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 --2- 172.21.15.53:0/1517823257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f04071c30 0x7f7f04159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 >> 172.21.15.53:0/1517823257 conn(0x7f7f0406d430 msgr2=0x7f7f040b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:02.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 shutdown_connections 2023-10-21T17:41:02.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.522+0000 7f7f08f41640 1 -- 172.21.15.53:0/1517823257 wait complete. 2023-10-21T17:41:02.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-21T17:41:02.543 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.541+0000 7f274f83c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:41:02.656 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.654+0000 7f274f83c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:41:02.682 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.680+0000 7f21b7e32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-21T17:41:02.705 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.704+0000 7f274f83c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:41:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.744+0000 7fa914483640 1 Processor -- start 2023-10-21T17:41:02.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.744+0000 7fa914483640 1 -- start start 2023-10-21T17:41:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.745+0000 7fa914483640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa90c06dfc0 0x7fa90c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.745+0000 7fa914483640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c0b1cc0 con 0x7fa90c071070 2023-10-21T17:41:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa914483640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c0b2620 con 0x7fa90c0afbe0 2023-10-21T17:41:02.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa914483640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c0b2060 con 0x7fa90c06dfc0 2023-10-21T17:41:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa911a1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa90c06dfc0 0x7fa90c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa912a20640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa90c071070 0x7fa90c0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:47414/0 (socket says 172.21.15.53:47414) 2023-10-21T17:41:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa911a1e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa90c06dfc0 0x7fa90c070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41064/0 (socket says 172.21.15.53:41064) 2023-10-21T17:41:02.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa912a20640 1 -- 172.21.15.53:0/4218748800 learned_addr learned my addr 172.21.15.53:0/4218748800 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa911a1e640 1 -- 172.21.15.53:0/4218748800 >> v1:172.21.15.124:6789/0 conn(0x7fa90c0afbe0 legacy=0x7fa90c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa911a1e640 1 -- 172.21.15.53:0/4218748800 >> v1:172.21.15.53:6789/0 conn(0x7fa90c071070 legacy=0x7fa90c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa911a1e640 1 -- 172.21.15.53:0/4218748800 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0b2340 con 0x7fa90c06dfc0 2023-10-21T17:41:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.746+0000 7fa91121d640 1 -- 172.21.15.53:0/4218748800 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761802166 0 0) 0x7fa90c0b1cc0 con 0x7fa90c071070 2023-10-21T17:41:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.748+0000 7fa911a1e640 1 --2- 172.21.15.53:0/4218748800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c06dfc0 0x7fa90c070b30 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b2810 tx=0x7fa90002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aacbd2e1abede8d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.748+0000 7fa91121d640 1 -- 172.21.15.53:0/4218748800 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900031830 con 0x7fa90c06dfc0 2023-10-21T17:41:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.748+0000 7fa91121d640 1 -- 172.21.15.53:0/4218748800 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900031990 con 0x7fa90c06dfc0 2023-10-21T17:41:02.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.748+0000 7fa91121d640 1 -- 172.21.15.53:0/4218748800 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900031d20 con 0x7fa90c06dfc0 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 -- 172.21.15.53:0/4218748800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c06dfc0 msgr2=0x7fa90c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 --2- 172.21.15.53:0/4218748800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c06dfc0 0x7fa90c070b30 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b2810 tx=0x7fa90002b8d0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 -- 172.21.15.53:0/4218748800 shutdown_connections 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 --2- 172.21.15.53:0/4218748800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c06dfc0 0x7fa90c070b30 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 -- 172.21.15.53:0/4218748800 >> 172.21.15.53:0/4218748800 conn(0x7fa90c06a5d0 msgr2=0x7fa90c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:02.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 -- 172.21.15.53:0/4218748800 shutdown_connections 2023-10-21T17:41:02.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.749+0000 7fa914483640 1 -- 172.21.15.53:0/4218748800 wait complete. 2023-10-21T17:41:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.751+0000 7fa914483640 1 Processor -- start 2023-10-21T17:41:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.751+0000 7fa914483640 1 -- start start 2023-10-21T17:41:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90c06dfc0 0x7fa90c14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90c0afbe0 0x7fa90c1efb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0a7ad0 con 0x7fa90c071070 2023-10-21T17:41:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c14b190 con 0x7fa90c0afbe0 2023-10-21T17:41:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa914483640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0a77f0 con 0x7fa90c06dfc0 2023-10-21T17:41:02.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa911a1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa911a1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41074/0 (socket says 172.21.15.53:41074) 2023-10-21T17:41:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa911a1e640 1 -- 172.21.15.53:0/2276740771 learned_addr learned my addr 172.21.15.53:0/2276740771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa912a20640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90c0afbe0 0x7fa90c1efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.752+0000 7fa91221f640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90c06dfc0 0x7fa90c14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:02.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90c06dfc0 msgr2=0x7fa90c14ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90c06dfc0 0x7fa90c14ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90c0afbe0 msgr2=0x7fa90c1efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90c0afbe0 0x7fa90c1efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 -- 172.21.15.53:0/2276740771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900009400 con 0x7fa90c071070 2023-10-21T17:41:02.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa911a1e640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90003aa60 con 0x7fa90c071070 2023-10-21T17:41:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90003abe0 con 0x7fa90c071070 2023-10-21T17:41:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000443f0 con 0x7fa90c071070 2023-10-21T17:41:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c1f3e10 con 0x7fa90c071070 2023-10-21T17:41:02.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.753+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c1f43d0 con 0x7fa90c071070 2023-10-21T17:41:02.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.755+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c1f3c70 con 0x7fa90c071070 2023-10-21T17:41:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.756+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa90004d450 con 0x7fa90c071070 2023-10-21T17:41:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.757+0000 7fa8feffd640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 0x7fa8f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.757+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7fa9000abf40 con 0x7fa90c071070 2023-10-21T17:41:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.757+0000 7fa91221f640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 msgr2=0x7fa8f006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:02.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.757+0000 7fa91221f640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 0x7fa8f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:02.762 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.761+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa90c1f3c70 con 0x7fa90c071070 2023-10-21T17:41:02.799 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.798+0000 7f21b7e32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-21T17:41:02.813 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.812+0000 7f274f83c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:41:02.852 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.851+0000 7f21b7e32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-21T17:41:02.920 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:02.918+0000 7f274f83c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:41:02.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.958+0000 7fa91221f640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 msgr2=0x7fa8f006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:02.959 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.958+0000 7fa91221f640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 0x7fa8f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:41:02.966 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:02.965+0000 7f21b7e32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:41:02.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.967+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa90c1f3c70 con 0x7fa90c071070 2023-10-21T17:41:02.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.968+0000 7fa8feffd640 1 -- 172.21.15.53:0/2276740771 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v276) v1 ==== 81+0+592 (secure 0 0 0) 0x7fa90c1f3c70 con 0x7fa90c071070 2023-10-21T17:41:02.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/612474845 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.969 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3975797466 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2074523274 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2461568637 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6832/596989303 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/2747533012 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.970 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/410485344 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.971 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/3555819866 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.971 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:6833/596989303 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.971 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.124:0/1112463946 2023-10-22T17:40:56.706577+0000 2023-10-21T17:41:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:listed 10 entries 2023-10-21T17:41:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 msgr2=0x7fa8f006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:02.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 0x7fa8f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 msgr2=0x7fa90c159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa9000046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 shutdown_connections 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa90c0afbe0 0x7fa90c1efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa8f006af90 0x7fa8f006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa90c071070 0x7fa90c159d30 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 --2- 172.21.15.53:0/2276740771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa90c06dfc0 0x7fa90c14ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.970+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 >> 172.21.15.53:0/2276740771 conn(0x7fa90c06a5d0 msgr2=0x7fa90c15ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.971+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 shutdown_connections 2023-10-21T17:41:02.974 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:02.971+0000 7fa914483640 1 -- 172.21.15.53:0/2276740771 wait complete. 2023-10-21T17:41:02.990 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-21T17:41:02.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:02.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-21T17:41:03.078 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:03.076+0000 7f274f83c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:41:03.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:41:03.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:03.084 DEBUG:teuthology.orchestra.run.smithi053:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-21T17:41:03.084 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:03.082+0000 7f21b7e32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-21T17:41:03.128 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:03.126+0000 7f274f83c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:41:03.133 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:41:03.133 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:03.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:41:03.190 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:41:03.190 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:03.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:41:03.247 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:41:03.247 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:03.256 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:03.255+0000 7f21b7e32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-21T17:41:03.257 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:03.251+0000 7f274f83c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:41:03.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:41:03.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:41:03.314 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:03.307+0000 7f21b7e32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-21T17:41:03.438 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:03.437+0000 7f21b7e32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-21T17:41:03.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.527+0000 7f9e7c8bb640 1 Processor -- start 2023-10-21T17:41:03.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.527+0000 7f9e7c8bb640 1 -- start start 2023-10-21T17:41:03.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e7c8bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e780c90b0 0x7f9e780cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:03.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e7c8bb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7815b2b0 con 0x7f9e7815ab40 2023-10-21T17:41:03.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e7c8bb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7815b590 con 0x7f9e780cb3b0 2023-10-21T17:41:03.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e7c8bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7815bf30 con 0x7f9e780c90b0 2023-10-21T17:41:03.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e6dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e780c90b0 0x7f9e780cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:03.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e6dd9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9e780c90b0 0x7f9e780cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41090/0 (socket says 172.21.15.53:41090) 2023-10-21T17:41:03.531 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.529+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/2209094716 learned_addr learned my addr 172.21.15.53:0/2209094716 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:03.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.530+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/2209094716 >> v1:172.21.15.124:6789/0 conn(0x7f9e780cb3b0 legacy=0x7f9e7815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.530+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/2209094716 >> v1:172.21.15.53:6789/0 conn(0x7f9e7815ab40 legacy=0x7f9e780ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.530+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/2209094716 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7815b870 con 0x7f9e780c90b0 2023-10-21T17:41:03.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.530+0000 7f9e75d9b640 1 -- 172.21.15.53:0/2209094716 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073959504 0 0) 0x7f9e7815b2b0 con 0x7f9e7815ab40 2023-10-21T17:41:03.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.531+0000 7f9e6dd9b640 1 --2- 172.21.15.53:0/2209094716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780cae70 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009870 tx=0x7f9e6002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cd3a79104129d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:03.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.531+0000 7f9e75d9b640 1 -- 172.21.15.53:0/2209094716 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002f840 con 0x7f9e780c90b0 2023-10-21T17:41:03.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.531+0000 7f9e75d9b640 1 -- 172.21.15.53:0/2209094716 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6002f9a0 con 0x7f9e780c90b0 2023-10-21T17:41:03.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.531+0000 7f9e75d9b640 1 -- 172.21.15.53:0/2209094716 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002fd30 con 0x7f9e780c90b0 2023-10-21T17:41:03.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/2209094716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 msgr2=0x7f9e780cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/2209094716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780cae70 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009870 tx=0x7f9e6002ba20 comp rx=0 tx=0).stop 2023-10-21T17:41:03.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/2209094716 shutdown_connections 2023-10-21T17:41:03.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/2209094716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780cae70 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/2209094716 >> 172.21.15.53:0/2209094716 conn(0x7f9e7806a380 msgr2=0x7f9e780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:03.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/2209094716 shutdown_connections 2023-10-21T17:41:03.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.532+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/2209094716 wait complete. 2023-10-21T17:41:03.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.533+0000 7f9e7c8bb640 1 Processor -- start 2023-10-21T17:41:03.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.534+0000 7f9e7c8bb640 1 -- start start 2023-10-21T17:41:03.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:03.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e780cb3b0 0x7f9e780ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:03.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e7815ab40 0x7f9e781530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:03.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78158ed0 con 0x7f9e780c90b0 2023-10-21T17:41:03.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780abea0 con 0x7f9e780cb3b0 2023-10-21T17:41:03.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7c8bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78158530 con 0x7f9e7815ab40 2023-10-21T17:41:03.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:03.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41094/0 (socket says 172.21.15.53:41094) 2023-10-21T17:41:03.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 -- 172.21.15.53:0/758680565 learned_addr learned my addr 172.21.15.53:0/758680565 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:03.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e6dd9b640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e780cb3b0 0x7f9e780ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:03.540 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e7815ab40 msgr2=0x7f9e781530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e7815ab40 0x7f9e781530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e780cb3b0 msgr2=0x7f9e780ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e780cb3b0 0x7f9e780ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.535+0000 7f9e7659c640 1 -- 172.21.15.53:0/758680565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600094d0 con 0x7f9e780c90b0 2023-10-21T17:41:03.541 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e7659c640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c780 tx=0x7f9e6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:03.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68003da0 con 0x7f9e780c90b0 2023-10-21T17:41:03.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6800ce90 con 0x7f9e780c90b0 2023-10-21T17:41:03.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68005270 con 0x7f9e780c90b0 2023-10-21T17:41:03.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e78158810 con 0x7f9e780c90b0 2023-10-21T17:41:03.542 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.536+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e78154480 con 0x7f9e780c90b0 2023-10-21T17:41:03.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.537+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e78071450 con 0x7f9e780c90b0 2023-10-21T17:41:03.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.538+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9e680053d0 con 0x7f9e780c90b0 2023-10-21T17:41:03.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.539+0000 7f9e6f7fe640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 0x7f9e5806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:03.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.539+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7f9e68089880 con 0x7f9e780c90b0 2023-10-21T17:41:03.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.539+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 msgr2=0x7f9e5806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:03.544 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.539+0000 7f9e6dd9b640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 0x7f9e5806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:03.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.544+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e68054550 con 0x7f9e780c90b0 2023-10-21T17:41:03.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.740+0000 7f9e6dd9b640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 msgr2=0x7f9e5806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:03.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.740+0000 7f9e6dd9b640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 0x7f9e5806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:41:03.776 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.775+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e78067490 con 0x7f9e780c90b0 2023-10-21T17:41:03.777 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.775+0000 7f9e6f7fe640 1 -- 172.21.15.53:0/758680565 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f9e68057b70 con 0x7f9e780c90b0 2023-10-21T17:41:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:03.777 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:41:03.778 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:41:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 msgr2=0x7f9e5806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 0x7f9e5806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 msgr2=0x7f9e780ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c780 tx=0x7f9e6800cc40 comp rx=0 tx=0).stop 2023-10-21T17:41:03.780 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 shutdown_connections 2023-10-21T17:41:03.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e7815ab40 0x7f9e781530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e780cb3b0 0x7f9e780ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7f9e5806af90 0x7f9e5806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.781 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 --2- 172.21.15.53:0/758680565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e780c90b0 0x7f9e780ab8c0 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:03.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 >> 172.21.15.53:0/758680565 conn(0x7f9e7806a380 msgr2=0x7f9e780aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:03.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 shutdown_connections 2023-10-21T17:41:03.782 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.778+0000 7f9e7c8bb640 1 -- 172.21.15.53:0/758680565 wait complete. 2023-10-21T17:41:03.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:41:04.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.999+0000 7fa650c07640 1 Processor -- start 2023-10-21T17:41:04.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:03.999+0000 7fa650c07640 1 -- start start 2023-10-21T17:41:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa650c07640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c069d50 0x7fa64c06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa650c07640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa64c0cb880 0x7fa64c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa650c07640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0ce550 con 0x7fa64c0cbce0 2023-10-21T17:41:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa650c07640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c1609d0 con 0x7fa64c0cb880 2023-10-21T17:41:04.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa650c07640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cdcf0 con 0x7fa64c069d50 2023-10-21T17:41:04.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.001+0000 7fa649d9b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa64c0cb880 0x7fa64c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa64c0cb880 0x7fa64c0b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:35414/0 (socket says 172.21.15.53:35414) 2023-10-21T17:41:04.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 -- 172.21.15.53:0/4135253532 learned_addr learned my addr 172.21.15.53:0/4135253532 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa64a59c640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c069d50 0x7fa64c06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 -- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c069d50 msgr2=0x7fa64c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c069d50 0x7fa64c06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 -- 172.21.15.53:0/4135253532 >> v1:172.21.15.124:6789/0 conn(0x7fa64c0cbce0 legacy=0x7fa64c160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.002+0000 7fa649d9b640 1 -- 172.21.15.53:0/4135253532 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c06a670 con 0x7fa64c0cb880 2023-10-21T17:41:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.003+0000 7fa649d9b640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c0cb880 0x7fa64c0b1310 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa640009870 tx=0x7fa64002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f3f47cd0a3056a server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.004+0000 7fa64959a640 1 -- 172.21.15.53:0/4135253532 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640030840 con 0x7fa64c0cb880 2023-10-21T17:41:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.004+0000 7fa64959a640 1 -- 172.21.15.53:0/4135253532 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6400309a0 con 0x7fa64c0cb880 2023-10-21T17:41:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.004+0000 7fa64959a640 1 -- 172.21.15.53:0/4135253532 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640030d30 con 0x7fa64c0cb880 2023-10-21T17:41:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.004+0000 7fa650c07640 1 -- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c0cb880 msgr2=0x7fa64c0b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.004+0000 7fa650c07640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c0cb880 0x7fa64c0b1310 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa640009870 tx=0x7fa64002b980 comp rx=0 tx=0).stop 2023-10-21T17:41:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 -- 172.21.15.53:0/4135253532 shutdown_connections 2023-10-21T17:41:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c0cb880 0x7fa64c0b1310 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 --2- 172.21.15.53:0/4135253532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c069d50 0x7fa64c06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 -- 172.21.15.53:0/4135253532 >> 172.21.15.53:0/4135253532 conn(0x7fa64c06d450 msgr2=0x7fa64c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:04.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 -- 172.21.15.53:0/4135253532 shutdown_connections 2023-10-21T17:41:04.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.005+0000 7fa650c07640 1 -- 172.21.15.53:0/4135253532 wait complete. 2023-10-21T17:41:04.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.007+0000 7fa650c07640 1 Processor -- start 2023-10-21T17:41:04.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.007+0000 7fa650c07640 1 -- start start 2023-10-21T17:41:04.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c069d50 0x7fa64c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c0cbce0 0x7fa64c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0ac8a0 con 0x7fa64c0cb880 2023-10-21T17:41:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0acb80 con 0x7fa64c069d50 2023-10-21T17:41:04.014 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa650c07640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0ace60 con 0x7fa64c0cbce0 2023-10-21T17:41:04.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa649d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa649d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:41098/0 (socket says 172.21.15.53:41098) 2023-10-21T17:41:04.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa649d9b640 1 -- 172.21.15.53:0/773451824 learned_addr learned my addr 172.21.15.53:0/773451824 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:04.015 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa64a59c640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c069d50 0x7fa64c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.008+0000 7fa64ad9d640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c0cbce0 0x7fa64c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c0cbce0 msgr2=0x7fa64c0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c0cbce0 0x7fa64c0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c069d50 msgr2=0x7fa64c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c069d50 0x7fa64c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 -- 172.21.15.53:0/773451824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400094d0 con 0x7fa64c0cb880 2023-10-21T17:41:04.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa649d9b640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640039a90 con 0x7fa64c0cb880 2023-10-21T17:41:04.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640039bf0 con 0x7fa64c0cb880 2023-10-21T17:41:04.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640043410 con 0x7fa64c0cb880 2023-10-21T17:41:04.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.009+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0ad140 con 0x7fa64c0cb880 2023-10-21T17:41:04.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.010+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c15a8f0 con 0x7fa64c0cb880 2023-10-21T17:41:04.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.011+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c0ad660 con 0x7fa64c0cb880 2023-10-21T17:41:04.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.012+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa64002f390 con 0x7fa64c0cb880 2023-10-21T17:41:04.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.013+0000 7fa63affd640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 0x7fa62c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.013+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7fa6400abd40 con 0x7fa64c0cb880 2023-10-21T17:41:04.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.013+0000 7fa64a59c640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 msgr2=0x7fa62c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:04.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.013+0000 7fa64a59c640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 0x7fa62c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:04.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.018+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa64c0ad660 con 0x7fa64c0cb880 2023-10-21T17:41:04.088 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.086+0000 7f274f83c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:41:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.214+0000 7fa64a59c640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 msgr2=0x7fa62c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:04.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.214+0000 7fa64a59c640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 0x7fa62c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-21T17:41:04.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.216+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa64c0ad660 con 0x7fa64c0cb880 2023-10-21T17:41:04.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.219+0000 7fa63affd640 1 -- 172.21.15.53:0/773451824 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v276) v1 ==== 74+0+17525 (secure 0 0 0) 0x7fa64c0ad660 con 0x7fa64c0cb880 2023-10-21T17:41:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:04.221 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":276,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:40:56.706711+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:0/612474845":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3975797466":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2074523274":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2461568637":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6832/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2747533012":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/410485344":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3555819866":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/1112463946":"2023-10-22T17:40:56.706577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:41:04.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 msgr2=0x7fa62c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:04.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 0x7fa62c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 msgr2=0x7fa64c0c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640004730 comp rx=0 tx=0).stop 2023-10-21T17:41:04.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 shutdown_connections 2023-10-21T17:41:04.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.221+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa64c0cbce0 0x7fa64c0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fa62c06af90 0x7fa62c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa64c0cb880 0x7fa64c0c5ad0 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 --2- 172.21.15.53:0/773451824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa64c069d50 0x7fa64c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 >> 172.21.15.53:0/773451824 conn(0x7fa64c06d450 msgr2=0x7fa64c0c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:04.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 shutdown_connections 2023-10-21T17:41:04.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.222+0000 7fa650c07640 1 -- 172.21.15.53:0/773451824 wait complete. 2023-10-21T17:41:04.240 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-21T16:01:49.093973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-21T17:40:39.912054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '274', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-21T17:40:40.993087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '275', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-21T17:41:04.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-21T17:41:04.296 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:04.295+0000 7f21b7e32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-21T17:41:04.382 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.381+0000 7f274f83c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:41:04.439 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.438+0000 7f274f83c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:41:04.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.444+0000 7fd9c0ec2640 1 Processor -- start 2023-10-21T17:41:04.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.444+0000 7fd9c0ec2640 1 -- start start 2023-10-21T17:41:04.447 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9c0ec2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9bc0b1750 0x7fd9bc06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9c0ec2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9c0ec2640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc15bf30 con 0x7fd9bc15aad0 2023-10-21T17:41:04.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9c0ec2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc06d6b0 con 0x7fd9bc0b1750 2023-10-21T17:41:04.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9c0ec2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0b1bb0 con 0x7fd9bc06d240 2023-10-21T17:41:04.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9bc0b1750 0x7fd9bc06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd9bc0b1750 0x7fd9bc06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48344/0 (socket says 172.21.15.53:48344) 2023-10-21T17:41:04.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 -- 172.21.15.53:0/2438922678 learned_addr learned my addr 172.21.15.53:0/2438922678 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 -- 172.21.15.53:0/2438922678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 msgr2=0x7fd9bc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 --2- 172.21.15.53:0/2438922678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.446+0000 7fd9b9d9b640 1 -- 172.21.15.53:0/2438922678 >> v1:172.21.15.124:6789/0 conn(0x7fd9bc15aad0 legacy=0x7fd9bc0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.447+0000 7fd9b9d9b640 1 -- 172.21.15.53:0/2438922678 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc15b320 con 0x7fd9bc0b1750 2023-10-21T17:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.448+0000 7fd9b9d9b640 1 --2- 172.21.15.53:0/2438922678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc06cd00 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceb831ca02f92f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.448+0000 7fd9b959a640 1 -- 172.21.15.53:0/2438922678 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0031640 con 0x7fd9bc0b1750 2023-10-21T17:41:04.451 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.448+0000 7fd9b959a640 1 -- 172.21.15.53:0/2438922678 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00317a0 con 0x7fd9bc0b1750 2023-10-21T17:41:04.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.448+0000 7fd9b959a640 1 -- 172.21.15.53:0/2438922678 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0031b30 con 0x7fd9bc0b1750 2023-10-21T17:41:04.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/2438922678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc0b1750 msgr2=0x7fd9bc06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/2438922678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc06cd00 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/2438922678 shutdown_connections 2023-10-21T17:41:04.452 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/2438922678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/2438922678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc06cd00 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/2438922678 >> 172.21.15.53:0/2438922678 conn(0x7fd9bc069d50 msgr2=0x7fd9bc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:04.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/2438922678 shutdown_connections 2023-10-21T17:41:04.453 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.449+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/2438922678 wait complete. 2023-10-21T17:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.451+0000 7fd9c0ec2640 1 Processor -- start 2023-10-21T17:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.451+0000 7fd9c0ec2640 1 -- start start 2023-10-21T17:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.456 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f2ee0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f22d0 con 0x7fd9bc0b1750 2023-10-21T17:41:04.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9c0ec2640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f25b0 con 0x7fd9bc06d240 2023-10-21T17:41:04.457 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9bad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9bad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48360/0 (socket says 172.21.15.53:48360) 2023-10-21T17:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.452+0000 7fd9bad9d640 1 -- 172.21.15.53:0/1256027641 learned_addr learned my addr 172.21.15.53:0/1256027641 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9b9d9b640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9ba59c640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.458 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 -- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 msgr2=0x7fd9bc154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 -- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9bc0b1750 msgr2=0x7fd9bc159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 -- 172.21.15.53:0/1256027641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00094d0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.459 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9bad9d640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400b970 tx=0x7fd9b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.453+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cb40 con 0x7fd9bc15aad0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.454+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400ccc0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.454+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc1f2890 con 0x7fd9bc15aad0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.454+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4014680 con 0x7fd9bc15aad0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.454+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc1597c0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.460 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.455+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc1f3050 con 0x7fd9bc15aad0 2023-10-21T17:41:04.461 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.456+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 149) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd9b401d3b0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.457+0000 7fd9aaffd640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 0x7fd99c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.457+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6575+0+0 (secure 0 0 0) 0x7fd9b4089b40 con 0x7fd9bc15aad0 2023-10-21T17:41:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.457+0000 7fd9ba59c640 1 -- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 msgr2=0x7fd99c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.124:6832/1070399472 2023-10-21T17:41:04.462 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.457+0000 7fd9ba59c640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 0x7fd99c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-21T17:41:04.466 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.465+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9b40546e0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.554 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.554+0000 7f274f83c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:41:04.577 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.576+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 150) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd9b4054ab0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.577+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 msgr2=0x7fd99c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-21T17:41:04.578 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.577+0000 7fd9aaffd640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 0x7fd99c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.646 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:04.644+0000 7f21b7e32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-21T17:41:04.651 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d1b8c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.651 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d1b8c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.652 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d1b8c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.652 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d1b8c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.652 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d1b8c640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.652 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d338f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.652 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d338f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.653 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d338f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.653 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d338f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.653 INFO:tasks.ceph.mgr.x.smithi124.stderr:2023-10-21T17:41:04.650+0000 7f26d338f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-21T17:41:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.667+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9bc04e160 con 0x7fd9bc15aad0 2023-10-21T17:41:04.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.667+0000 7fd9aaffd640 1 -- 172.21.15.53:0/1256027641 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v277) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b40721e0 con 0x7fd9bc15aad0 2023-10-21T17:41:04.669 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 1 2023-10-21T17:41:04.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 msgr2=0x7fd9bc0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400b970 tx=0x7fd9b400be30 comp rx=0 tx=0).stop 2023-10-21T17:41:04.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 shutdown_connections 2023-10-21T17:41:04.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:6832/1070399472,v1:172.21.15.124:6833/1070399472] conn(0x7fd99c06af90 0x7fd99c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9bc15aad0 0x7fd9bc0b2a60 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.672 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9bc0b1750 0x7fd9bc159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 --2- 172.21.15.53:0/1256027641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9bc06d240 0x7fd9bc154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 >> 172.21.15.53:0/1256027641 conn(0x7fd9bc069d50 msgr2=0x7fd9bc155150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.670+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 shutdown_connections 2023-10-21T17:41:04.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.671+0000 7fd9c0ec2640 1 -- 172.21.15.53:0/1256027641 wait complete. 2023-10-21T17:41:04.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-21T17:41:04.709 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:04.708+0000 7f21b7e32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-21T17:41:04.829 INFO:tasks.ceph.mgr.y.smithi053.stderr:2023-10-21T17:41:04.828+0000 7f21b7e32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-21T17:41:04.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.899+0000 7f229677a640 1 Processor -- start 2023-10-21T17:41:04.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.899+0000 7f229677a640 1 -- start start 2023-10-21T17:41:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.900+0000 7f229677a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22901599d0 0x7f22900cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.900+0000 7f229677a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22900b3c50 0x7f22900caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.900+0000 7f229677a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900cb8a0 con 0x7f22900cb410 2023-10-21T17:41:04.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.900+0000 7f229677a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2290159db0 con 0x7f22900b3c50 2023-10-21T17:41:04.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.900+0000 7f229677a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290071fe0 con 0x7f22901599d0 2023-10-21T17:41:04.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2295f79640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f22900cb410 0x7f22900c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58734/0 (socket says 172.21.15.53:58734) 2023-10-21T17:41:04.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2295f79640 1 -- 172.21.15.53:0/3080322343 learned_addr learned my addr 172.21.15.53:0/3080322343 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:04.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2294f77640 1 --2- 172.21.15.53:0/3080322343 >> v2:172.21.15.124:3300/0 conn(0x7f22900b3c50 0x7f22900caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914429808 0 0) 0x7f22900cb8a0 con 0x7f22900cb410 2023-10-21T17:41:04.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22901599d0 0x7f22900cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f227c003520 con 0x7f22900cb410 2023-10-21T17:41:04.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.901+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766530016 0 0) 0x7f227c003520 con 0x7f22900cb410 2023-10-21T17:41:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22901599d0 msgr2=0x7f22900cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22901599d0 0x7f22900cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 >> v2:172.21.15.124:3300/0 conn(0x7f22900b3c50 msgr2=0x7f22900caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 --2- 172.21.15.53:0/3080322343 >> v2:172.21.15.124:3300/0 conn(0x7f22900b3c50 0x7f22900caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900b44c0 con 0x7f22900cb410 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f22880019f0 con 0x7f22900cb410 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 >> v1:172.21.15.53:6789/0 conn(0x7f22900cb410 legacy=0x7f22900c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f227c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f227c0047c0 0x7f227c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f227c005180 0x7f227c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290159db0 con 0x7f227c003970 2023-10-21T17:41:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290071fe0 con 0x7f227c0047c0 2023-10-21T17:41:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c00bfc0 con 0x7f227c005180 2023-10-21T17:41:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2294f77640 1 --2- 172.21.15.53:0/3080322343 >> v2:172.21.15.124:3300/0 conn(0x7f22900b3c50 0x7f22900caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22901599d0 0x7f22900cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:04.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f227c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295f79640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f227c005180 0x7f227c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2294f77640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f227c0047c0 0x7f227c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295778640 1 -- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f227c005180 msgr2=0x7f227c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f227c005180 0x7f227c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.902+0000 7f2295778640 1 -- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f227c0047c0 msgr2=0x7f227c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.903+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f227c0047c0 0x7f227c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.903+0000 7f2295778640 1 -- 172.21.15.53:0/3080322343 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c003520 con 0x7f227c003970 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.903+0000 7f2295f79640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f227c005180 0x7f227c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.903+0000 7f2294f77640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f227c0047c0 0x7f227c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f2295778640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f227c003d90 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f229006e150 tx=0x7f227c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc17158b2eb56af3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c033600 con 0x7f227c003970 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c02f050 con 0x7f227c003970 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f2287fff640 1 -- 172.21.15.53:0/3080322343 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c033980 con 0x7f227c003970 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f229677a640 1 -- 172.21.15.53:0/3080322343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 msgr2=0x7f227c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f229677a640 1 --2- 172.21.15.53:0/3080322343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f227c003d90 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7f229006e150 tx=0x7f227c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f2295f79640 1 -- 172.21.15.53:0/3080322343 reap_dead start 2023-10-21T17:41:04.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f229677a640 1 -- 172.21.15.53:0/3080322343 shutdown_connections 2023-10-21T17:41:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.904+0000 7f229677a640 1 -- 172.21.15.53:0/3080322343 >> 172.21.15.53:0/3080322343 conn(0x7f229006c050 msgr2=0x7f229006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.905+0000 7f229677a640 1 -- 172.21.15.53:0/3080322343 shutdown_connections 2023-10-21T17:41:04.913 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.905+0000 7f229677a640 1 -- 172.21.15.53:0/3080322343 wait complete. 2023-10-21T17:41:04.915 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.906+0000 7f229677a640 1 Processor -- start 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.907+0000 7f229677a640 1 -- start start 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 0x7f22900b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22900b0640 0x7f22900b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901571a0 con 0x7f227c003970 2023-10-21T17:41:04.916 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900b1000 con 0x7f22901544a0 2023-10-21T17:41:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f229677a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c4740 con 0x7f22900b0640 2023-10-21T17:41:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48386/0 (socket says 172.21.15.53:48386) 2023-10-21T17:41:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 -- 172.21.15.53:0/911936162 learned_addr learned my addr 172.21.15.53:0/911936162 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:04.917 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2294f77640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 0x7f22900b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295f79640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22900b0640 0x7f22900b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 -- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22900b0640 msgr2=0x7f22900b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22900b0640 0x7f22900b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 -- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 msgr2=0x7f22900b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 0x7f22900b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:04.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2295778640 1 -- 172.21.15.53:0/911936162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290154a60 con 0x7f227c003970 2023-10-21T17:41:04.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.908+0000 7f2294f77640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 0x7f22900b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:04.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f2295778640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f228c02bdc0 tx=0x7f228c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:04.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c03d850 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c03d9b0 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290154d40 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c03dd40 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.909+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290155300 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.910+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290155520 con 0x7f227c003970 2023-10-21T17:41:04.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.911+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 150) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f228c03c3a0 con 0x7f227c003970 2023-10-21T17:41:04.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.912+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f228c0ab480 con 0x7f227c003970 2023-10-21T17:41:04.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:04.917+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f228c0763d0 con 0x7f227c003970 2023-10-21T17:41:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.109+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2290074150 con 0x7f227c003970 2023-10-21T17:41:05.110 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.109+0000 7f2285ffb640 1 -- 172.21.15.53:0/911936162 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v277) v1 ==== 104+0+11 (secure 0 0 0) 0x7f228c0799f0 con 0x7f227c003970 2023-10-21T17:41:05.111 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 32 2023-10-21T17:41:05.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 msgr2=0x7f22900c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.113 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f228c02bdc0 tx=0x7f228c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 shutdown_connections 2023-10-21T17:41:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22900b0640 0x7f22900b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22901544a0 0x7f22900b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 --2- 172.21.15.53:0/911936162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f227c003970 0x7f22900c4200 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.114 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 >> 172.21.15.53:0/911936162 conn(0x7f229006c050 msgr2=0x7f229015a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 shutdown_connections 2023-10-21T17:41:05.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.112+0000 7f229677a640 1 -- 172.21.15.53:0/911936162 wait complete. 2023-10-21T17:41:05.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-21T17:41:05.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.406+0000 7f2345d0a640 1 Processor -- start 2023-10-21T17:41:05.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.406+0000 7f2345d0a640 1 -- start start 2023-10-21T17:41:05.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.407+0000 7f2345d0a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f23400cf2b0 0x7f23400d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345d0a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f23400ce510 0x7f234015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.409 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345d0a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23400cd520 con 0x7f23400cf690 2023-10-21T17:41:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345d0a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400d1910 con 0x7f23400ce510 2023-10-21T17:41:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345d0a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23400ce080 con 0x7f23400cf2b0 2023-10-21T17:41:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f23400ce510 0x7f234015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f23400ce510 0x7f234015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48402/0 (socket says 172.21.15.53:48402) 2023-10-21T17:41:05.410 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 -- 172.21.15.53:0/227979179 learned_addr learned my addr 172.21.15.53:0/227979179 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:05.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2344d08640 1 --2- 172.21.15.53:0/227979179 >> v2:172.21.15.124:3300/0 conn(0x7f23400cf2b0 0x7f23400d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 -- 172.21.15.53:0/227979179 >> v2:172.21.15.124:3300/0 conn(0x7f23400cf2b0 msgr2=0x7f23400d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 --2- 172.21.15.53:0/227979179 >> v2:172.21.15.124:3300/0 conn(0x7f23400cf2b0 0x7f23400d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.408+0000 7f2345509640 1 -- 172.21.15.53:0/227979179 >> v1:172.21.15.124:6789/0 conn(0x7f23400cf690 legacy=0x7f23400d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.409+0000 7f2345509640 1 -- 172.21.15.53:0/227979179 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400cd750 con 0x7f23400ce510 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.410+0000 7f2345509640 1 --2- 172.21.15.53:0/227979179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400ce510 0x7f234015c820 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f23400b54b0 tx=0x7f233802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d0078d240113c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.410+0000 7f233f7fe640 1 -- 172.21.15.53:0/227979179 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23380317e0 con 0x7f23400ce510 2023-10-21T17:41:05.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.410+0000 7f233f7fe640 1 -- 172.21.15.53:0/227979179 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338031940 con 0x7f23400ce510 2023-10-21T17:41:05.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.410+0000 7f233f7fe640 1 -- 172.21.15.53:0/227979179 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338031cd0 con 0x7f23400ce510 2023-10-21T17:41:05.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 -- 172.21.15.53:0/227979179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400ce510 msgr2=0x7f234015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 --2- 172.21.15.53:0/227979179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400ce510 0x7f234015c820 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f23400b54b0 tx=0x7f233802b880 comp rx=0 tx=0).stop 2023-10-21T17:41:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 -- 172.21.15.53:0/227979179 shutdown_connections 2023-10-21T17:41:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 --2- 172.21.15.53:0/227979179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400ce510 0x7f234015c820 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 --2- 172.21.15.53:0/227979179 >> v2:172.21.15.124:3300/0 conn(0x7f23400cf2b0 0x7f23400d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 -- 172.21.15.53:0/227979179 >> 172.21.15.53:0/227979179 conn(0x7f234009bc40 msgr2=0x7f23400d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 -- 172.21.15.53:0/227979179 shutdown_connections 2023-10-21T17:41:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.411+0000 7f2345d0a640 1 -- 172.21.15.53:0/227979179 wait complete. 2023-10-21T17:41:05.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.414+0000 7f2345d0a640 1 Processor -- start 2023-10-21T17:41:05.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.414+0000 7f2345d0a640 1 -- start start 2023-10-21T17:41:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23400ce510 0x7f23401f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23400cf2b0 0x7f234009ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234009df90 con 0x7f23400cf690 2023-10-21T17:41:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234009d380 con 0x7f23400cf2b0 2023-10-21T17:41:05.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.415+0000 7f2345d0a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234009d660 con 0x7f23400ce510 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48412/0 (socket says 172.21.15.53:48412) 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 -- 172.21.15.53:0/1826557650 learned_addr learned my addr 172.21.15.53:0/1826557650 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f233ffff640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23400cf2b0 0x7f234009ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2344d08640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23400ce510 0x7f23401f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 -- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23400ce510 msgr2=0x7f23401f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23400ce510 0x7f23401f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 -- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23400cf2b0 msgr2=0x7f234009ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23400cf2b0 0x7f234009ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 -- 172.21.15.53:0/1826557650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380094d0 con 0x7f23400cf690 2023-10-21T17:41:05.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.416+0000 7f2345509640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f23380090b0 tx=0x7f23380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.417+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338032880 con 0x7f23400cf690 2023-10-21T17:41:05.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.417+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338032a20 con 0x7f23400cf690 2023-10-21T17:41:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.417+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338032dd0 con 0x7f23400cf690 2023-10-21T17:41:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.417+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234009d940 con 0x7f23400cf690 2023-10-21T17:41:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.417+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234009cfc0 con 0x7f23400cf690 2023-10-21T17:41:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.418+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234009e100 con 0x7f23400cf690 2023-10-21T17:41:05.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.419+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 150) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f233804e450 con 0x7f23400cf690 2023-10-21T17:41:05.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.421+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f23380ab5e0 con 0x7f23400cf690 2023-10-21T17:41:05.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.425+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2338076620 con 0x7f23400cf690 2023-10-21T17:41:05.583 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.582+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 151) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f23380760d0 con 0x7f23400cf690 2023-10-21T17:41:05.584 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.583+0000 7f233d7fa640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 0x7f2320073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.595 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.594+0000 7f2344d08640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 0x7f2320073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.606+0000 7f2344d08640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 0x7f2320073160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2330009f30 tx=0x7f233000b040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.627+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2340074150 con 0x7f23400cf690 2023-10-21T17:41:05.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.628+0000 7f233d7fa640 1 -- 172.21.15.53:0/1826557650 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v277) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2338080030 con 0x7f23400cf690 2023-10-21T17:41:05.629 INFO:teuthology.orchestra.run.smithi053.stdout:pg_num: 64 2023-10-21T17:41:05.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 msgr2=0x7f2320073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 0x7f2320073160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2330009f30 tx=0x7f233000b040 comp rx=0 tx=0).stop 2023-10-21T17:41:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 msgr2=0x7f23400b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f23380090b0 tx=0x7f23380046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 shutdown_connections 2023-10-21T17:41:05.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f2320070d70 0x7f2320073160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23400cf690 0x7f23400b6cf0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23400cf2b0 0x7f234009ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 --2- 172.21.15.53:0/1826557650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23400ce510 0x7f23401f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 >> 172.21.15.53:0/1826557650 conn(0x7f234009bc40 msgr2=0x7f23400b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 shutdown_connections 2023-10-21T17:41:05.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.630+0000 7f2345d0a640 1 -- 172.21.15.53:0/1826557650 wait complete. 2023-10-21T17:41:05.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:41:05.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.875+0000 7f1a91b1d640 1 Processor -- start 2023-10-21T17:41:05.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.875+0000 7f1a91b1d640 1 -- start start 2023-10-21T17:41:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a91b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a91b1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a91b1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8c15b2b0 con 0x7f1a8c15ab40 2023-10-21T17:41:05.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a91b1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8c15bf30 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a91b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0cb840 con 0x7f1a8c0c90b0 2023-10-21T17:41:05.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a8bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a8bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48428/0 (socket says 172.21.15.53:48428) 2023-10-21T17:41:05.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.877+0000 7f1a8bfff640 1 -- 172.21.15.53:0/3453661764 learned_addr learned my addr 172.21.15.53:0/3453661764 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:05.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.878+0000 7f1a82ffd640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.878+0000 7f1a8bfff640 1 -- 172.21.15.53:0/3453661764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 msgr2=0x7f1a8c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:05.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.878+0000 7f1a8bfff640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.878+0000 7f1a8bfff640 1 -- 172.21.15.53:0/3453661764 >> v1:172.21.15.124:6789/0 conn(0x7f1a8c15ab40 legacy=0x7f1a8c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:05.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.878+0000 7f1a8bfff640 1 -- 172.21.15.53:0/3453661764 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c15b590 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.879+0000 7f1a8bfff640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e50 tx=0x7f1a8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3fd073321e6f0d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.879+0000 7f1a8affd640 1 -- 172.21.15.53:0/3453661764 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402ca70 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.879+0000 7f1a8affd640 1 -- 172.21.15.53:0/3453661764 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8402cbd0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.879+0000 7f1a8affd640 1 -- 172.21.15.53:0/3453661764 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840343f0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 -- 172.21.15.53:0/3453661764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 msgr2=0x7f1a8c15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009e50 tx=0x7f1a8402bed0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 -- 172.21.15.53:0/3453661764 shutdown_connections 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/3453661764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15e790 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 -- 172.21.15.53:0/3453661764 >> 172.21.15.53:0/3453661764 conn(0x7f1a8c06a380 msgr2=0x7f1a8c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 -- 172.21.15.53:0/3453661764 shutdown_connections 2023-10-21T17:41:05.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.880+0000 7f1a91b1d640 1 -- 172.21.15.53:0/3453661764 wait complete. 2023-10-21T17:41:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.883+0000 7f1a91b1d640 1 Processor -- start 2023-10-21T17:41:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.883+0000 7f1a91b1d640 1 -- start start 2023-10-21T17:41:05.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a8c15ab40 0x7f1a8c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c1f65b0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c1f3420 con 0x7f1a8c15ab40 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a91b1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c1f69d0 con 0x7f1a8c0c90b0 2023-10-21T17:41:05.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48438/0 (socket says 172.21.15.53:48438) 2023-10-21T17:41:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.884+0000 7f1a82ffd640 1 -- 172.21.15.53:0/2971759997 learned_addr learned my addr 172.21.15.53:0/2971759997 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a8b7fe640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a8bfff640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a8c15ab40 0x7f1a8c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 -- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 msgr2=0x7f1a8c1f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c1f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 -- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a8c15ab40 msgr2=0x7f1a8c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a8c15ab40 0x7f1a8c15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 -- 172.21.15.53:0/2971759997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009b30 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.890 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.885+0000 7f1a82ffd640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400d620 tx=0x7f1a7400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.886+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400b620 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.886+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7400b780 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.886+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400bb30 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.886+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c1f2340 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.886+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c1f5ef0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.887+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c1f66f0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.888+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 151) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1a7400bc90 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.889+0000 7f1a88ff9640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 0x7f1a7006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.889+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f1a74089b90 con 0x7f1a8c0cb3b0 2023-10-21T17:41:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.890+0000 7f1a8b7fe640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 0x7f1a7006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:05.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.891+0000 7f1a8b7fe640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 0x7f1a7006d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c004ea0 tx=0x7f1a7c009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:05.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:05.895+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a8c1f66f0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:06.129 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.128+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a8c1f66f0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:06.130 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.129+0000 7f1a88ff9640 1 -- 172.21.15.53:0/2971759997 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f1a8c1f66f0 con 0x7f1a8c0cb3b0 2023-10-21T17:41:06.131 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:06.131 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:41:06.131 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:41:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 msgr2=0x7f1a7006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.133 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 0x7f1a7006d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c004ea0 tx=0x7f1a7c009040 comp rx=0 tx=0).stop 2023-10-21T17:41:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 msgr2=0x7f1a8c15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400d620 tx=0x7f1a7400dae0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 shutdown_connections 2023-10-21T17:41:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1a7006acf0 0x7f1a7006d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.134 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a8c15ab40 0x7f1a8c15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a8c0cb3b0 0x7f1a8c15d930 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 --2- 172.21.15.53:0/2971759997 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8c0c90b0 0x7f1a8c1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 >> 172.21.15.53:0/2971759997 conn(0x7f1a8c06a380 msgr2=0x7f1a8c06cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 shutdown_connections 2023-10-21T17:41:06.135 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.132+0000 7f1a91b1d640 1 -- 172.21.15.53:0/2971759997 wait complete. 2023-10-21T17:41:06.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:41:06.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.375+0000 7fe041924640 1 Processor -- start 2023-10-21T17:41:06.377 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.376+0000 7fe041924640 1 -- start start 2023-10-21T17:41:06.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe041924640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe03c0cb580 0x7fe03c165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.378 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe041924640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c15b2b0 con 0x7fe03c15ab40 2023-10-21T17:41:06.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe041924640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c15b590 con 0x7fe03c0c90b0 2023-10-21T17:41:06.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe041924640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c15bf30 con 0x7fe03c0cb580 2023-10-21T17:41:06.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe03affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe03c15ab40 0x7fe03c0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58744/0 (socket says 172.21.15.53:58744) 2023-10-21T17:41:06.379 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.377+0000 7fe03affd640 1 -- 172.21.15.53:0/3347268743 learned_addr learned my addr 172.21.15.53:0/3347268743 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:06.381 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe03b7fe640 1 --2- 172.21.15.53:0/3347268743 >> v2:172.21.15.124:3300/0 conn(0x7fe03c0cb580 0x7fe03c165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208360247 0 0) 0x7fe03c15b2b0 con 0x7fe03c15ab40 2023-10-21T17:41:06.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe024003520 con 0x7fe03c15ab40 2023-10-21T17:41:06.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713785713 0 0) 0x7fe024003520 con 0x7fe03c15ab40 2023-10-21T17:41:06.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 >> v2:172.21.15.124:3300/0 conn(0x7fe03c0cb580 msgr2=0x7fe03c165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:06.382 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 --2- 172.21.15.53:0/3347268743 >> v2:172.21.15.124:3300/0 conn(0x7fe03c0cb580 0x7fe03c165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 >> v1:172.21.15.124:6789/0 conn(0x7fe03c0c90b0 legacy=0x7fe03c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c15b870 con 0x7fe03c15ab40 2023-10-21T17:41:06.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fe02c0019f0 con 0x7fe03c15ab40 2023-10-21T17:41:06.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 >> v1:172.21.15.53:6789/0 conn(0x7fe03c15ab40 legacy=0x7fe03c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.383 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240039c0 0x7fe024003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.378+0000 7fe039ffb640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 0x7fe024004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe039ffb640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe024005180 0x7fe02400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c15bf30 con 0x7fe0240047c0 2023-10-21T17:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02400c080 con 0x7fe0240039c0 2023-10-21T17:41:06.384 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02400c240 con 0x7fe024005180 2023-10-21T17:41:06.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 0x7fe024004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03affd640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240039c0 0x7fe024003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 -- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe024005180 msgr2=0x7fe02400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:06.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03b7fe640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe024005180 0x7fe02400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.385 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe024005180 0x7fe02400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 -- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240039c0 msgr2=0x7fe024003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:06.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240039c0 0x7fe024003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03a7fc640 1 -- 172.21.15.53:0/3347268743 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024003520 con 0x7fe0240047c0 2023-10-21T17:41:06.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03b7fe640 1 -- 172.21.15.53:0/3347268743 reap_dead start 2023-10-21T17:41:06.386 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.379+0000 7fe03affd640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240039c0 0x7fe024003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:06.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 0x7fe024004c40 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fe030001720 tx=0x7fe0240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d326ceffe530f72 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300052d0 con 0x7fe0240047c0 2023-10-21T17:41:06.390 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03002e050 con 0x7fe0240047c0 2023-10-21T17:41:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe039ffb640 1 -- 172.21.15.53:0/3347268743 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030003830 con 0x7fe0240047c0 2023-10-21T17:41:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe041924640 1 -- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 msgr2=0x7fe024004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe041924640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 0x7fe024004c40 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fe030001720 tx=0x7fe0240057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe041924640 1 -- 172.21.15.53:0/3347268743 shutdown_connections 2023-10-21T17:41:06.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.381+0000 7fe041924640 1 --2- 172.21.15.53:0/3347268743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0240047c0 0x7fe024004c40 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.382+0000 7fe041924640 1 -- 172.21.15.53:0/3347268743 >> 172.21.15.53:0/3347268743 conn(0x7fe03c06a380 msgr2=0x7fe03c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.382+0000 7fe041924640 1 -- 172.21.15.53:0/3347268743 shutdown_connections 2023-10-21T17:41:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.382+0000 7fe041924640 1 -- 172.21.15.53:0/3347268743 wait complete. 2023-10-21T17:41:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.383+0000 7fe041924640 1 Processor -- start 2023-10-21T17:41:06.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.383+0000 7fe041924640 1 -- start start 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240047c0 0x7fe03c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe03c1556f0 0x7fe03c155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c15f7a0 con 0x7fe03c151d80 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1f22e0 con 0x7fe0240047c0 2023-10-21T17:41:06.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.384+0000 7fe041924640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c152740 con 0x7fe03c1556f0 2023-10-21T17:41:06.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48450/0 (socket says 172.21.15.53:48450) 2023-10-21T17:41:06.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 -- 172.21.15.53:0/1586220794 learned_addr learned my addr 172.21.15.53:0/1586220794 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:06.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03affd640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240047c0 0x7fe03c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03b7fe640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe03c1556f0 0x7fe03c155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 -- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe03c1556f0 msgr2=0x7fe03c155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe03c1556f0 0x7fe03c155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 -- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240047c0 msgr2=0x7fe03c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240047c0 0x7fe03c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.385+0000 7fe03a7fc640 1 -- 172.21.15.53:0/1586220794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c1ede20 con 0x7fe03c151d80 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe03a7fc640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fe030034040 tx=0x7fe030001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300076c0 con 0x7fe03c151d80 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300069e0 con 0x7fe03c151d80 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030006d50 con 0x7fe03c151d80 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c1f1660 con 0x7fe03c151d80 2023-10-21T17:41:06.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.386+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c1f1c20 con 0x7fe03c151d80 2023-10-21T17:41:06.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.387+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c067490 con 0x7fe03c151d80 2023-10-21T17:41:06.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.388+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 151) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe0300035f0 con 0x7fe03c151d80 2023-10-21T17:41:06.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.389+0000 7fe023fff640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 0x7fe02806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.389+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7fe0300ab530 con 0x7fe03c151d80 2023-10-21T17:41:06.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.390+0000 7fe03affd640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 0x7fe02806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.392+0000 7fe03affd640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 0x7fe02806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0013b0 tx=0x7fe02c009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.398 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.393+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0300764b0 con 0x7fe03c151d80 2023-10-21T17:41:06.597 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.596+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fe030076880 con 0x7fe03c151d80 2023-10-21T17:41:06.606 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.605+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe03c0b0ef0 con 0x7fe03c151d80 2023-10-21T17:41:06.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.608+0000 7fe023fff640 1 -- 172.21.15.53:0/1586220794 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v277) v1 ==== 74+0+18033 (secure 0 0 0) 0x7fe03007de50 con 0x7fe03c151d80 2023-10-21T17:41:06.610 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:06.610 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":277,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:41:04.558424+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6832/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3025084903":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2615301846":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/1018395068":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2173042720":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3250485253":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/59627499":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/612474845":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3975797466":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2074523274":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2461568637":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6832/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2747533012":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/410485344":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3555819866":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/1112463946":"2023-10-22T17:40:56.706577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:41:06.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 msgr2=0x7fe02806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 0x7fe02806d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0013b0 tx=0x7fe02c009040 comp rx=0 tx=0).stop 2023-10-21T17:41:06.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 msgr2=0x7fe03c152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fe030034040 tx=0x7fe030001120 comp rx=0 tx=0).stop 2023-10-21T17:41:06.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 shutdown_connections 2023-10-21T17:41:06.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe02806acc0 0x7fe02806d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe03c1556f0 0x7fe03c155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe03c151d80 0x7fe03c152200 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 --2- 172.21.15.53:0/1586220794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0240047c0 0x7fe03c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 >> 172.21.15.53:0/1586220794 conn(0x7fe03c06a380 msgr2=0x7fe03c0c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:06.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.611+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 shutdown_connections 2023-10-21T17:41:06.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.612+0000 7fe041924640 1 -- 172.21.15.53:0/1586220794 wait complete. 2023-10-21T17:41:06.633 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:41:06.634 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:41:06.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.851+0000 7f26cc44b640 1 Processor -- start 2023-10-21T17:41:06.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.851+0000 7f26cc44b640 1 -- start start 2023-10-21T17:41:06.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.853+0000 7f26cc44b640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 0x7f26c40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.853+0000 7f26cc44b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.853+0000 7f26cc44b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c415b2b0 con 0x7f26c40cae70 2023-10-21T17:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.853+0000 7f26cc44b640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c415bf30 con 0x7f26c415ab40 2023-10-21T17:41:06.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.853+0000 7f26cc44b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c415f7a0 con 0x7f26c40c90b0 2023-10-21T17:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.854+0000 7f26ca1e7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 0x7f26c40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.854+0000 7f26c99e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.854+0000 7f26ca1e7640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 0x7f26c40ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.53:47160/0 (socket says 172.21.15.53:47160) 2023-10-21T17:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.854+0000 7f26c99e6640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.53:54866/0 (socket says 172.21.15.53:54866) 2023-10-21T17:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.854+0000 7f26ca1e7640 1 -- 172.21.15.53:0/3121932304 learned_addr learned my addr 172.21.15.53:0/3121932304 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.855+0000 7f26c99e6640 1 -- 172.21.15.53:0/3121932304 >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 msgr2=0x7f26c40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.855+0000 7f26c99e6640 1 --2- 172.21.15.53:0/3121932304 >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 0x7f26c40ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.855+0000 7f26c99e6640 1 -- 172.21.15.53:0/3121932304 >> v1:172.21.15.124:6789/0 conn(0x7f26c40cae70 legacy=0x7f26c40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.855+0000 7f26c99e6640 1 -- 172.21.15.53:0/3121932304 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c415b590 con 0x7f26c40c90b0 2023-10-21T17:41:06.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.856+0000 7f26c99e6640 1 --2- 172.21.15.53:0/3121932304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f26b40097a0 tx=0x7f26b402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189504a15c72560d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26c91e5640 1 -- 172.21.15.53:0/3121932304 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b402d040 con 0x7f26c40c90b0 2023-10-21T17:41:06.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26c91e5640 1 -- 172.21.15.53:0/3121932304 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b4031810 con 0x7f26c40c90b0 2023-10-21T17:41:06.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26c91e5640 1 -- 172.21.15.53:0/3121932304 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4031ba0 con 0x7f26c40c90b0 2023-10-21T17:41:06.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26cc44b640 1 -- 172.21.15.53:0/3121932304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 msgr2=0x7f26c40adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26cc44b640 1 --2- 172.21.15.53:0/3121932304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f26b40097a0 tx=0x7f26b402b8b0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26cc44b640 1 -- 172.21.15.53:0/3121932304 shutdown_connections 2023-10-21T17:41:06.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26cc44b640 1 --2- 172.21.15.53:0/3121932304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40adc60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.857+0000 7f26cc44b640 1 --2- 172.21.15.53:0/3121932304 >> v2:172.21.15.124:3300/0 conn(0x7f26c415ab40 0x7f26c40ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.858+0000 7f26cc44b640 1 -- 172.21.15.53:0/3121932304 >> 172.21.15.53:0/3121932304 conn(0x7f26c406a380 msgr2=0x7f26c40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:06.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.858+0000 7f26cc44b640 1 -- 172.21.15.53:0/3121932304 shutdown_connections 2023-10-21T17:41:06.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.858+0000 7f26cc44b640 1 -- 172.21.15.53:0/3121932304 wait complete. 2023-10-21T17:41:06.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.859+0000 7f26cc44b640 1 Processor -- start 2023-10-21T17:41:06.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.860+0000 7f26cc44b640 1 -- start start 2023-10-21T17:41:06.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c40cae70 0x7f26c41440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40ac910 con 0x7f26c415ab40 2023-10-21T17:41:06.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c41508b0 con 0x7f26c40cae70 2023-10-21T17:41:06.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26cc44b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4144610 con 0x7f26c40c90b0 2023-10-21T17:41:06.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26ca9e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26ca9e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48452/0 (socket says 172.21.15.53:48452) 2023-10-21T17:41:06.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26ca9e8640 1 -- 172.21.15.53:0/2985345540 learned_addr learned my addr 172.21.15.53:0/2985345540 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:06.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26ca1e7640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.861+0000 7f26c99e6640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c40cae70 0x7f26c41440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 -- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 msgr2=0x7f26c40ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 -- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c40cae70 msgr2=0x7f26c41440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:06.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c40cae70 0x7f26c41440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:06.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 -- 172.21.15.53:0/2985345540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4009400 con 0x7f26c415ab40 2023-10-21T17:41:06.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26ca9e8640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c7a0 tx=0x7f26bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc003b70 con 0x7f26c415ab40 2023-10-21T17:41:06.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc003cf0 con 0x7f26c415ab40 2023-10-21T17:41:06.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c40acb60 con 0x7f26c415ab40 2023-10-21T17:41:06.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.862+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc003ea0 con 0x7f26c415ab40 2023-10-21T17:41:06.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.863+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c40ad120 con 0x7f26c415ab40 2023-10-21T17:41:06.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.864+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c4067490 con 0x7f26c415ab40 2023-10-21T17:41:06.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.865+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f26bc00aa90 con 0x7f26c415ab40 2023-10-21T17:41:06.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.866+0000 7f26baffd640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 0x7f26a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:06.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.866+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f26bc089aa0 con 0x7f26c415ab40 2023-10-21T17:41:06.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.866+0000 7f26ca1e7640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 0x7f26a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:06.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.867+0000 7f26ca1e7640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 0x7f26a806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009b60 tx=0x7f26c0008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:06.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:06.870+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26bc054510 con 0x7f26c415ab40 2023-10-21T17:41:07.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.101+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26c40cbc20 con 0x7f26c415ab40 2023-10-21T17:41:07.103 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.102+0000 7f26baffd640 1 -- 172.21.15.53:0/2985345540 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f26bc057b30 con 0x7f26c415ab40 2023-10-21T17:41:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:07.104 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:41:07.104 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:41:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 msgr2=0x7f26a806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 0x7f26a806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009b60 tx=0x7f26c0008040 comp rx=0 tx=0).stop 2023-10-21T17:41:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 msgr2=0x7f26c4159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00c7a0 tx=0x7f26bc00cc60 comp rx=0 tx=0).stop 2023-10-21T17:41:07.107 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 shutdown_connections 2023-10-21T17:41:07.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f26a806af30 0x7f26a806d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26c415ab40 0x7f26c4159000 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c40cae70 0x7f26c41440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 --2- 172.21.15.53:0/2985345540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c40c90b0 0x7f26c40ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.108 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.105+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 >> 172.21.15.53:0/2985345540 conn(0x7f26c406a380 msgr2=0x7f26c40cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.106+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 shutdown_connections 2023-10-21T17:41:07.109 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.106+0000 7f26cc44b640 1 -- 172.21.15.53:0/2985345540 wait complete. 2023-10-21T17:41:07.128 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:41:07.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.355+0000 7fdfe7fff640 1 Processor -- start 2023-10-21T17:41:07.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.356+0000 7fdfe7fff640 1 -- start start 2023-10-21T17:41:07.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.357+0000 7fdfe7fff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdfe815ab30 0x7fdfe815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.357+0000 7fdfe7fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfe80b07d0 0x7fdfe806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.357+0000 7fdfe7fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe815f7d0 con 0x7fdfe80adc70 2023-10-21T17:41:07.359 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.357+0000 7fdfe7fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815f210 con 0x7fdfe80b07d0 2023-10-21T17:41:07.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.357+0000 7fdfe7fff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815f4f0 con 0x7fdfe815ab30 2023-10-21T17:41:07.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfe80b07d0 0x7fdfe806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdfe80b07d0 0x7fdfe806cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48464/0 (socket says 172.21.15.53:48464) 2023-10-21T17:41:07.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 -- 172.21.15.53:0/2417755041 learned_addr learned my addr 172.21.15.53:0/2417755041 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:07.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe6ffd640 1 --2- 172.21.15.53:0/2417755041 >> v2:172.21.15.124:3300/0 conn(0x7fdfe815ab30 0x7fdfe815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 -- 172.21.15.53:0/2417755041 >> v2:172.21.15.124:3300/0 conn(0x7fdfe815ab30 msgr2=0x7fdfe815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:07.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 --2- 172.21.15.53:0/2417755041 >> v2:172.21.15.124:3300/0 conn(0x7fdfe815ab30 0x7fdfe815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 -- 172.21.15.53:0/2417755041 >> v1:172.21.15.124:6789/0 conn(0x7fdfe80adc70 legacy=0x7fdfe80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:07.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.358+0000 7fdfe77fe640 1 -- 172.21.15.53:0/2417755041 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe806c550 con 0x7fdfe80b07d0 2023-10-21T17:41:07.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.359+0000 7fdfe77fe640 1 --2- 172.21.15.53:0/2417755041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80b07d0 0x7fdfe806cd50 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fdfe806ada0 tx=0x7fdfe002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4cce494fc714bb server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe67fc640 1 -- 172.21.15.53:0/2417755041 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe002ca80 con 0x7fdfe80b07d0 2023-10-21T17:41:07.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe67fc640 1 -- 172.21.15.53:0/2417755041 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe002cbe0 con 0x7fdfe80b07d0 2023-10-21T17:41:07.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe67fc640 1 -- 172.21.15.53:0/2417755041 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00343f0 con 0x7fdfe80b07d0 2023-10-21T17:41:07.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2417755041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80b07d0 msgr2=0x7fdfe806cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2417755041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80b07d0 0x7fdfe806cd50 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fdfe806ada0 tx=0x7fdfe002bee0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2417755041 shutdown_connections 2023-10-21T17:41:07.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2417755041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80b07d0 0x7fdfe806cd50 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/2417755041 >> v2:172.21.15.124:3300/0 conn(0x7fdfe815ab30 0x7fdfe815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.360+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2417755041 >> 172.21.15.53:0/2417755041 conn(0x7fdfe80cb0c0 msgr2=0x7fdfe80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:07.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.361+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2417755041 shutdown_connections 2023-10-21T17:41:07.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.361+0000 7fdfe7fff640 1 -- 172.21.15.53:0/2417755041 wait complete. 2023-10-21T17:41:07.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.362+0000 7fdfe7fff640 1 Processor -- start 2023-10-21T17:41:07.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 -- start start 2023-10-21T17:41:07.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 0x7fdfe80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.367 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfe815ab30 0x7fdfe80af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c79e0 con 0x7fdfe80adc70 2023-10-21T17:41:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c6600 con 0x7fdfe80b07d0 2023-10-21T17:41:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.363+0000 7fdfe7fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80c7040 con 0x7fdfe815ab30 2023-10-21T17:41:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.368 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48466/0 (socket says 172.21.15.53:48466) 2023-10-21T17:41:07.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/1299750643 learned_addr learned my addr 172.21.15.53:0/1299750643 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:07.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfdffff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 0x7fdfe80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfe815ab30 msgr2=0x7fdfe80af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.369 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfe815ab30 0x7fdfe80af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 msgr2=0x7fdfe80c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 0x7fdfe80c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfe6ffd640 1 -- 172.21.15.53:0/1299750643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009b30 con 0x7fdfe80adc70 2023-10-21T17:41:07.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.364+0000 7fdfdffff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 0x7fdfe80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:07.370 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe6ffd640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800d5f0 tx=0x7fdfd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80125f0 con 0x7fdfe80adc70 2023-10-21T17:41:07.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8012750 con 0x7fdfe80adc70 2023-10-21T17:41:07.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8012b00 con 0x7fdfe80adc70 2023-10-21T17:41:07.371 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe80c7320 con 0x7fdfe80adc70 2023-10-21T17:41:07.372 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.365+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe80c8340 con 0x7fdfe80adc70 2023-10-21T17:41:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.366+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe804e160 con 0x7fdfe80adc70 2023-10-21T17:41:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.367+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdfd8012c60 con 0x7fdfe80adc70 2023-10-21T17:41:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.368+0000 7fdfe4ff9640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 0x7fdfc806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.373 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.368+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7fdfd808a090 con 0x7fdfe80adc70 2023-10-21T17:41:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.369+0000 7fdfdffff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 0x7fdfc806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.370+0000 7fdfdffff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 0x7fdfc806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0003eb0 tx=0x7fdfd0009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.374 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.372+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfd80bcca0 con 0x7fdfe80adc70 2023-10-21T17:41:07.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.607+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe80ada50 con 0x7fdfe80adc70 2023-10-21T17:41:07.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.608+0000 7fdfe4ff9640 1 -- 172.21.15.53:0/1299750643 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fdfd8054b90 con 0x7fdfe80adc70 2023-10-21T17:41:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:07.610 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":143,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-21T17:40:41.958072+0000","modified":"2023-10-21T17:40:45.062063+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":38723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_38723":{"gid":38723,"name":"b","rank":0,"incarnation":140,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2127148028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2127148028},{"type":"v1","addr":"172.21.15.124:6837","nonce":2127148028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-21T17:41:07.610 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 143 2023-10-21T17:41:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 msgr2=0x7fdfc806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 0x7fdfc806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0003eb0 tx=0x7fdfd0009040 comp rx=0 tx=0).stop 2023-10-21T17:41:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 msgr2=0x7fdfe81f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800d5f0 tx=0x7fdfd800dab0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.613 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 shutdown_connections 2023-10-21T17:41:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdfc806af00 0x7fdfc806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfe815ab30 0x7fdfe80af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfe80b07d0 0x7fdfe80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 --2- 172.21.15.53:0/1299750643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdfe80adc70 0x7fdfe81f0eb0 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.614 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.611+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 >> 172.21.15.53:0/1299750643 conn(0x7fdfe80cb0c0 msgr2=0x7fdfe80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:07.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.612+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 shutdown_connections 2023-10-21T17:41:07.615 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.612+0000 7fdfe7fff640 1 -- 172.21.15.53:0/1299750643 wait complete. 2023-10-21T17:41:07.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:41:07.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.854+0000 7f575dcbb640 1 Processor -- start 2023-10-21T17:41:07.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.854+0000 7f575dcbb640 1 -- start start 2023-10-21T17:41:07.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f575dcbb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 0x7f575815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f575dcbb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57580adc70 0x7f57580b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f575dcbb640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575815f7d0 con 0x7f575815af10 2023-10-21T17:41:07.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f575dcbb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575815f210 con 0x7f57580adc70 2023-10-21T17:41:07.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f575dcbb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575815f4f0 con 0x7f575815ab30 2023-10-21T17:41:07.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f5757fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57580adc70 0x7f57580b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f5757fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f57580adc70 0x7f57580b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48474/0 (socket says 172.21.15.53:48474) 2023-10-21T17:41:07.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.856+0000 7f5757fff640 1 -- 172.21.15.53:0/3279505949 learned_addr learned my addr 172.21.15.53:0/3279505949 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:07.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f5757fff640 1 -- 172.21.15.53:0/3279505949 >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 msgr2=0x7f575815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:07.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f575ccb9640 1 --2- 172.21.15.53:0/3279505949 >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 0x7f575815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f5757fff640 1 --2- 172.21.15.53:0/3279505949 >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 0x7f575815e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f5757fff640 1 -- 172.21.15.53:0/3279505949 >> v1:172.21.15.124:6789/0 conn(0x7f575815af10 legacy=0x7f57580701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:07.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f5757fff640 1 -- 172.21.15.53:0/3279505949 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575806c550 con 0x7f57580adc70 2023-10-21T17:41:07.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.857+0000 7f575ccb9640 1 --2- 172.21.15.53:0/3279505949 >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 0x7f575815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.858+0000 7f5757fff640 1 --2- 172.21.15.53:0/3279505949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580b00c0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f57580c56c0 tx=0x7f574402b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a581496d7ff4f91c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f57577fe640 1 -- 172.21.15.53:0/3279505949 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744031640 con 0x7f57580adc70 2023-10-21T17:41:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f57577fe640 1 -- 172.21.15.53:0/3279505949 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57440317a0 con 0x7f57580adc70 2023-10-21T17:41:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f57577fe640 1 -- 172.21.15.53:0/3279505949 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744031b30 con 0x7f57580adc70 2023-10-21T17:41:07.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 -- 172.21.15.53:0/3279505949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 msgr2=0x7f57580b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 --2- 172.21.15.53:0/3279505949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580b00c0 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f57580c56c0 tx=0x7f574402b8e0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 -- 172.21.15.53:0/3279505949 shutdown_connections 2023-10-21T17:41:07.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 --2- 172.21.15.53:0/3279505949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580b00c0 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 --2- 172.21.15.53:0/3279505949 >> v2:172.21.15.124:3300/0 conn(0x7f575815ab30 0x7f575815e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 -- 172.21.15.53:0/3279505949 >> 172.21.15.53:0/3279505949 conn(0x7f57580cb0c0 msgr2=0x7f57580cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:07.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 -- 172.21.15.53:0/3279505949 shutdown_connections 2023-10-21T17:41:07.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.859+0000 7f575dcbb640 1 -- 172.21.15.53:0/3279505949 wait complete. 2023-10-21T17:41:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.861+0000 7f575dcbb640 1 Processor -- start 2023-10-21T17:41:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.861+0000 7f575dcbb640 1 -- start start 2023-10-21T17:41:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f575815ab30 0x7f575815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 0x7f575815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57580c6e20 con 0x7f57580adc70 2023-10-21T17:41:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57580c7100 con 0x7f575815af10 2023-10-21T17:41:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575dcbb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57580c73e0 con 0x7f575815ab30 2023-10-21T17:41:07.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.862+0000 7f575ccb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48486/0 (socket says 172.21.15.53:48486) 2023-10-21T17:41:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 -- 172.21.15.53:0/2936624156 learned_addr learned my addr 172.21.15.53:0/2936624156 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 -- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f575815ab30 msgr2=0x7f575815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f575815ab30 0x7f575815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 -- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 msgr2=0x7f575815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575d4ba640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 0x7f575815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 0x7f575815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 -- 172.21.15.53:0/2936624156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440094d0 con 0x7f57580adc70 2023-10-21T17:41:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575d4ba640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 0x7f575815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:07.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.863+0000 7f575ccb9640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c7f0 tx=0x7f574c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.864+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c007a30 con 0x7f57580adc70 2023-10-21T17:41:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.864+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c007b90 con 0x7f57580adc70 2023-10-21T17:41:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.864+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c00ce40 con 0x7f57580adc70 2023-10-21T17:41:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.864+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57580c76c0 con 0x7f57580adc70 2023-10-21T17:41:07.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.864+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575815e110 con 0x7f57580adc70 2023-10-21T17:41:07.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.865+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57580c7be0 con 0x7f57580adc70 2023-10-21T17:41:07.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.866+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f574c0113f0 con 0x7f57580adc70 2023-10-21T17:41:07.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.867+0000 7f57557fa640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 0x7f573c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:07.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.867+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f574c089ef0 con 0x7f57580adc70 2023-10-21T17:41:07.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.868+0000 7f5757fff640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 0x7f573c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:07.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.869+0000 7f5757fff640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 0x7f573c06d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5744036000 tx=0x7f5744030000 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:07.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:07.871+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57580c7be0 con 0x7f57580adc70 2023-10-21T17:41:08.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.067+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57580c54b0 con 0x7f57580adc70 2023-10-21T17:41:08.070 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.069+0000 7f57557fa640 1 -- 172.21.15.53:0/2936624156 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v277) v1 ==== 74+0+18033 (secure 0 0 0) 0x7f574c01f020 con 0x7f57580adc70 2023-10-21T17:41:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:08.071 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":277,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:41:04.558424+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-21T17:40:39.912054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"274","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-21T17:40:40.993087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6832/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3025084903":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2615301846":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/1018395068":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2173042720":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3250485253":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/59627499":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/612474845":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3975797466":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2074523274":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2461568637":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6832/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2747533012":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/410485344":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3555819866":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/1112463946":"2023-10-22T17:40:56.706577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:41:08.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 msgr2=0x7f573c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 0x7f573c06d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5744036000 tx=0x7f5744030000 comp rx=0 tx=0).stop 2023-10-21T17:41:08.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 msgr2=0x7f57580c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f574c00c7f0 tx=0x7f574c00cbf0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 shutdown_connections 2023-10-21T17:41:08.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f573c06af00 0x7f573c06d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f575815af10 0x7f575815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f575815ab30 0x7f575815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 --2- 172.21.15.53:0/2936624156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57580adc70 0x7f57580c6840 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 >> 172.21.15.53:0/2936624156 conn(0x7f57580cb0c0 msgr2=0x7f57580acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:08.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 shutdown_connections 2023-10-21T17:41:08.076 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.072+0000 7f575dcbb640 1 -- 172.21.15.53:0/2936624156 wait complete. 2023-10-21T17:41:08.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-21T17:41:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.328+0000 7f4dbb59e640 1 Processor -- start 2023-10-21T17:41:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.328+0000 7f4dbb59e640 1 -- start start 2023-10-21T17:41:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.329+0000 7f4dbb59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4dbc0c18d0 0x7f4dbc0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.329+0000 7f4dbb59e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dbc164d10 0x7f4dbc0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.329+0000 7f4dbb59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dbc161590 con 0x7f4dbc0aff70 2023-10-21T17:41:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.329+0000 7f4dbb59e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dbc0cc570 con 0x7f4dbc164d10 2023-10-21T17:41:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.329+0000 7f4dbb59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dbc0c94b0 con 0x7f4dbc0c18d0 2023-10-21T17:41:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dbc164d10 0x7f4dbc0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4dbc164d10 0x7f4dbc0ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48494/0 (socket says 172.21.15.53:48494) 2023-10-21T17:41:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 -- 172.21.15.53:0/4154413859 learned_addr learned my addr 172.21.15.53:0/4154413859 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dba59c640 1 --1- 172.21.15.53:0/4154413859 >> v1:172.21.15.53:6789/0 conn(0x7f4dbc0aff70 0x7f4dbc0b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58746/0 (socket says 172.21.15.53:58746) 2023-10-21T17:41:08.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4db3fff640 1 --2- 172.21.15.53:0/4154413859 >> v2:172.21.15.124:3300/0 conn(0x7f4dbc0c18d0 0x7f4dbc0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 -- 172.21.15.53:0/4154413859 >> v2:172.21.15.124:3300/0 conn(0x7f4dbc0c18d0 msgr2=0x7f4dbc0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 --2- 172.21.15.53:0/4154413859 >> v2:172.21.15.124:3300/0 conn(0x7f4dbc0c18d0 0x7f4dbc0c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 -- 172.21.15.53:0/4154413859 >> v1:172.21.15.53:6789/0 conn(0x7f4dbc0aff70 legacy=0x7f4dbc0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.330+0000 7f4dbad9d640 1 -- 172.21.15.53:0/4154413859 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc0c9790 con 0x7f4dbc164d10 2023-10-21T17:41:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4dbad9d640 1 --2- 172.21.15.53:0/4154413859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc0ccd30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0cc010 tx=0x7f4db402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875ee27aaf547d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4db9d9b640 1 -- 172.21.15.53:0/4154413859 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db402c9f0 con 0x7f4dbc164d10 2023-10-21T17:41:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4db9d9b640 1 -- 172.21.15.53:0/4154413859 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db402cb50 con 0x7f4dbc164d10 2023-10-21T17:41:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4db9d9b640 1 -- 172.21.15.53:0/4154413859 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db40343f0 con 0x7f4dbc164d10 2023-10-21T17:41:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4dbb59e640 1 -- 172.21.15.53:0/4154413859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 msgr2=0x7f4dbc0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.332+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/4154413859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc0ccd30 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0cc010 tx=0x7f4db402be50 comp rx=0 tx=0).stop 2023-10-21T17:41:08.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 -- 172.21.15.53:0/4154413859 shutdown_connections 2023-10-21T17:41:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/4154413859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc0ccd30 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/4154413859 >> v2:172.21.15.124:3300/0 conn(0x7f4dbc0c18d0 0x7f4dbc0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 -- 172.21.15.53:0/4154413859 >> 172.21.15.53:0/4154413859 conn(0x7f4dbc06c050 msgr2=0x7f4dbc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 -- 172.21.15.53:0/4154413859 shutdown_connections 2023-10-21T17:41:08.337 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.333+0000 7f4dbb59e640 1 -- 172.21.15.53:0/4154413859 wait complete. 2023-10-21T17:41:08.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.334+0000 7f4dbb59e640 1 Processor -- start 2023-10-21T17:41:08.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.335+0000 7f4dbb59e640 1 -- start start 2023-10-21T17:41:08.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.335+0000 7f4dbb59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4dbc0aff70 0x7f4dbc0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.335+0000 7f4dbb59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4dbc0c18d0 0x7f4dbc1447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbb59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbb59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc0b3670 con 0x7f4dbc164d10 2023-10-21T17:41:08.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbb59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc0b32d0 con 0x7f4dbc0c18d0 2023-10-21T17:41:08.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbb59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dbc0b3fd0 con 0x7f4dbc0aff70 2023-10-21T17:41:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48498/0 (socket says 172.21.15.53:48498) 2023-10-21T17:41:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 -- 172.21.15.53:0/2564933801 learned_addr learned my addr 172.21.15.53:0/2564933801 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4db3fff640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4dbc0c18d0 0x7f4dbc1447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 -- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4dbc0aff70 msgr2=0x7f4dbc0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4dbc0aff70 0x7f4dbc0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 -- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4dbc0c18d0 msgr2=0x7f4dbc1447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4dbc0c18d0 0x7f4dbc1447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.336+0000 7f4dbad9d640 1 -- 172.21.15.53:0/2564933801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4009b30 con 0x7f4dbc164d10 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4dbad9d640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f4db4036040 tx=0x7f4db4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db403e450 con 0x7f4dbc164d10 2023-10-21T17:41:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db403e5b0 con 0x7f4dbc164d10 2023-10-21T17:41:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db403e960 con 0x7f4dbc164d10 2023-10-21T17:41:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dbc0c7db0 con 0x7f4dbc164d10 2023-10-21T17:41:08.343 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.337+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc164540 con 0x7f4dbc164d10 2023-10-21T17:41:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.338+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dbc0b37b0 con 0x7f4dbc164d10 2023-10-21T17:41:08.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.339+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f4db403eac0 con 0x7f4dbc164d10 2023-10-21T17:41:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.340+0000 7f4db2ffd640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.340+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f4db40aca00 con 0x7f4dbc164d10 2023-10-21T17:41:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.341+0000 7f4dba59c640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.342+0000 7f4dba59c640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc160b40 tx=0x7f4dac0079c0 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.344+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4db4077590 con 0x7f4dbc164d10 2023-10-21T17:41:08.581 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.580+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4dbc074150 con 0x7f4dbc164d10 2023-10-21T17:41:08.583 INFO:tasks.ceph.mon.a.smithi053.stderr:2023-10-21T17:41:08.582+0000 7f4050d3e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-21T17:41:08.587 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.585+0000 7f4db2ffd640 1 -- 172.21.15.53:0/2564933801 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v144) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4db4043090 con 0x7f4dbc164d10 2023-10-21T17:41:08.587 INFO:teuthology.orchestra.run.smithi053.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-21T17:41:08.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 msgr2=0x7f4d9c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc160b40 tx=0x7f4dac0079c0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 msgr2=0x7f4dbc155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f4db4036040 tx=0x7f4db4004a40 comp rx=0 tx=0).stop 2023-10-21T17:41:08.592 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 shutdown_connections 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4d9c06af00 0x7f4d9c06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4dbc164d10 0x7f4dbc155f40 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4dbc0c18d0 0x7f4dbc1447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 --2- 172.21.15.53:0/2564933801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4dbc0aff70 0x7f4dbc0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 >> 172.21.15.53:0/2564933801 conn(0x7f4dbc06c050 msgr2=0x7f4dbc0ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:08.593 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 shutdown_connections 2023-10-21T17:41:08.594 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.590+0000 7f4dbb59e640 1 -- 172.21.15.53:0/2564933801 wait complete. 2023-10-21T17:41:08.612 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-21T17:41:08.621 INFO:tasks.ceph.mds.b.smithi124.stderr: -3921> 2023-10-21T17:40:46.872+0000 7f59a2008640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-21T17:41:08.696 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2023-10-21T17:41:08.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.837+0000 7fc9fef0c640 1 Processor -- start 2023-10-21T17:41:08.838 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.837+0000 7fc9fef0c640 1 -- start start 2023-10-21T17:41:08.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.838+0000 7fc9fef0c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.838+0000 7fc9fef0c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.838+0000 7fc9fef0c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f815b310 con 0x7fc9f815aa00 2023-10-21T17:41:08.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.838+0000 7fc9fef0c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80ca230 con 0x7fc9f806c3a0 2023-10-21T17:41:08.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.838+0000 7fc9fef0c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f806d290 con 0x7fc9f806db60 2023-10-21T17:41:08.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.841 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48510/0 (socket says 172.21.15.53:48510) 2023-10-21T17:41:08.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 -- 172.21.15.53:0/82237598 learned_addr learned my addr 172.21.15.53:0/82237598 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:08.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9f7fff640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 -- 172.21.15.53:0/82237598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 msgr2=0x7fc9f806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:08.842 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.843 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 -- 172.21.15.53:0/82237598 >> v1:172.21.15.53:6789/0 conn(0x7fc9f815aa00 legacy=0x7fc9f80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.839+0000 7fc9fcca8640 1 -- 172.21.15.53:0/82237598 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80c9860 con 0x7fc9f806c3a0 2023-10-21T17:41:08.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.841+0000 7fc9fcca8640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009870 tx=0x7fc9ec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3460a1489677717b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.841+0000 7fc9f77fe640 1 -- 172.21.15.53:0/82237598 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec030840 con 0x7fc9f806c3a0 2023-10-21T17:41:08.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.841+0000 7fc9f77fe640 1 -- 172.21.15.53:0/82237598 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0309a0 con 0x7fc9f806c3a0 2023-10-21T17:41:08.844 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.841+0000 7fc9f77fe640 1 -- 172.21.15.53:0/82237598 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec030d30 con 0x7fc9f806c3a0 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 -- 172.21.15.53:0/82237598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 msgr2=0x7fc9f80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009870 tx=0x7fc9ec02b980 comp rx=0 tx=0).stop 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 -- 172.21.15.53:0/82237598 shutdown_connections 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/82237598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f80c9cf0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.845 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 -- 172.21.15.53:0/82237598 >> 172.21.15.53:0/82237598 conn(0x7fc9f806a5d0 msgr2=0x7fc9f80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:08.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 -- 172.21.15.53:0/82237598 shutdown_connections 2023-10-21T17:41:08.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.842+0000 7fc9fef0c640 1 -- 172.21.15.53:0/82237598 wait complete. 2023-10-21T17:41:08.847 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.844+0000 7fc9fef0c640 1 Processor -- start 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.844+0000 7fc9fef0c640 1 -- start start 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f8154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 0x7fc9f80b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81573a0 con 0x7fc9f806c3a0 2023-10-21T17:41:08.848 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81567d0 con 0x7fc9f815aa00 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fef0c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f815d3d0 con 0x7fc9f806db60 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fcca8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fcca8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48514/0 (socket says 172.21.15.53:48514) 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.845+0000 7fc9fcca8640 1 -- 172.21.15.53:0/2051510490 learned_addr learned my addr 172.21.15.53:0/2051510490 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fd4a9640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 0x7fc9f80b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9f7fff640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f8154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 -- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 msgr2=0x7fc9f8154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f8154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 -- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 msgr2=0x7fc9f80b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 0x7fc9f80b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:08.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 -- 172.21.15.53:0/2051510490 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec0094d0 con 0x7fc9f806c3a0 2023-10-21T17:41:08.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fd4a9640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 0x7fc9f80b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:08.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.846+0000 7fc9fcca8640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec000c00 tx=0x7fc9ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.847+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec039c90 con 0x7fc9f806c3a0 2023-10-21T17:41:08.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.847+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec039df0 con 0x7fc9f806c3a0 2023-10-21T17:41:08.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.847+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec043640 con 0x7fc9f806c3a0 2023-10-21T17:41:08.852 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.847+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8156a70 con 0x7fc9f806c3a0 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.847+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8157030 con 0x7fc9f806c3a0 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.848+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f804e160 con 0x7fc9f806c3a0 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.849+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 152) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc9ec0437a0 con 0x7fc9f806c3a0 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.850+0000 7fc9f57fa640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.850+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7fc9ec03e070 con 0x7fc9f806c3a0 2023-10-21T17:41:08.853 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.850+0000 7fc9f7fff640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:08.854 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.852+0000 7fc9f7fff640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80aeb90 tx=0x7fc9e4008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:08.856 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:08.855+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9ec0777e0 con 0x7fc9f806c3a0 2023-10-21T17:41:09.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.085+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9f80c94e0 con 0x7fc9f806c3a0 2023-10-21T17:41:09.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.411+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc9ec077bb0 con 0x7fc9f806c3a0 2023-10-21T17:41:09.612 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.611+0000 7fc9f57fa640 1 -- 172.21.15.53:0/2051510490 <== mon.0 v2:172.21.15.53:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v145) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9ec07d920 con 0x7fc9f806c3a0 2023-10-21T17:41:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.615+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 msgr2=0x7fc9dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.615+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80aeb90 tx=0x7fc9e4008040 comp rx=0 tx=0).stop 2023-10-21T17:41:09.617 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 msgr2=0x7fc9f815ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec000c00 tx=0x7fc9ec004730 comp rx=0 tx=0).stop 2023-10-21T17:41:09.618 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 shutdown_connections 2023-10-21T17:41:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc9dc06af00 0x7fc9dc06d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9f815aa00 0x7fc9f80b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9f806db60 0x7fc9f8154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 --2- 172.21.15.53:0/2051510490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9f806c3a0 0x7fc9f815ce90 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.619 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 >> 172.21.15.53:0/2051510490 conn(0x7fc9f806a5d0 msgr2=0x7fc9f80cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:09.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 shutdown_connections 2023-10-21T17:41:09.620 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.616+0000 7fc9fef0c640 1 -- 172.21.15.53:0/2051510490 wait complete. 2023-10-21T17:41:09.640 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-21T17:41:09.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.871+0000 7fdb47fff640 1 Processor -- start 2023-10-21T17:41:09.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.871+0000 7fdb47fff640 1 -- start start 2023-10-21T17:41:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.872+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 0x7fdb480d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.872+0000 7fdb47fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb480d00e0 0x7fdb480b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.872+0000 7fdb47fff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb480d2870 con 0x7fdb480d0b90 2023-10-21T17:41:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.872+0000 7fdb47fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb480d1150 con 0x7fdb480d00e0 2023-10-21T17:41:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.872+0000 7fdb47fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480d1430 con 0x7fdb4815ccb0 2023-10-21T17:41:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb480d00e0 0x7fdb480b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fdb480d00e0 0x7fdb480b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48524/0 (socket says 172.21.15.53:48524) 2023-10-21T17:41:09.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 -- 172.21.15.53:0/2503215369 learned_addr learned my addr 172.21.15.53:0/2503215369 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 0x7fdb480d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 -- 172.21.15.53:0/2503215369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 msgr2=0x7fdb480d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 0x7fdb480d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 -- 172.21.15.53:0/2503215369 >> v1:172.21.15.124:6789/0 conn(0x7fdb480d0b90 legacy=0x7fdb480cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb477fe640 1 -- 172.21.15.53:0/2503215369 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480d1710 con 0x7fdb480d00e0 2023-10-21T17:41:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.873+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 0x7fdb480d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb477fe640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb480d00e0 0x7fdb480b6060 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009870 tx=0x7fdb4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1ef835f7fb2906 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb467fc640 1 -- 172.21.15.53:0/2503215369 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40031640 con 0x7fdb480d00e0 2023-10-21T17:41:09.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb467fc640 1 -- 172.21.15.53:0/2503215369 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb400317a0 con 0x7fdb480d00e0 2023-10-21T17:41:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb467fc640 1 -- 172.21.15.53:0/2503215369 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb40031b30 con 0x7fdb480d00e0 2023-10-21T17:41:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb47fff640 1 -- 172.21.15.53:0/2503215369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb480d00e0 msgr2=0x7fdb480b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.875+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb480d00e0 0x7fdb480b6060 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009870 tx=0x7fdb4002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 -- 172.21.15.53:0/2503215369 shutdown_connections 2023-10-21T17:41:09.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb480d00e0 0x7fdb480b6060 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2503215369 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb4815ccb0 0x7fdb480d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 -- 172.21.15.53:0/2503215369 >> 172.21.15.53:0/2503215369 conn(0x7fdb48071660 msgr2=0x7fdb48071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 -- 172.21.15.53:0/2503215369 shutdown_connections 2023-10-21T17:41:09.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.876+0000 7fdb47fff640 1 -- 172.21.15.53:0/2503215369 wait complete. 2023-10-21T17:41:09.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.877+0000 7fdb47fff640 1 Processor -- start 2023-10-21T17:41:09.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.878+0000 7fdb47fff640 1 -- start start 2023-10-21T17:41:09.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.878+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 0x7fdb481f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 0x7fdb480b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb47fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb47fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4815ec30 con 0x7fdb4815ccb0 2023-10-21T17:41:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb47fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4815dfb0 con 0x7fdb480d0b90 2023-10-21T17:41:09.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb47fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4815e290 con 0x7fdb480d00e0 2023-10-21T17:41:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48528/0 (socket says 172.21.15.53:48528) 2023-10-21T17:41:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 -- 172.21.15.53:0/2469580488 learned_addr learned my addr 172.21.15.53:0/2469580488 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 0x7fdb481f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 -- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 msgr2=0x7fdb481f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 0x7fdb481f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb3e7fc640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 0x7fdb480b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 -- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 msgr2=0x7fdb480b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 0x7fdb480b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:09.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb477fe640 1 -- 172.21.15.53:0/2469580488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400094d0 con 0x7fdb4815ccb0 2023-10-21T17:41:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 0x7fdb481f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.879+0000 7fdb3e7fc640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 0x7fdb480b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb477fe640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fdb4002b930 tx=0x7fdb400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4003ab10 con 0x7fdb4815ccb0 2023-10-21T17:41:09.886 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4003ac90 con 0x7fdb4815ccb0 2023-10-21T17:41:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb400443f0 con 0x7fdb4815ccb0 2023-10-21T17:41:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4815e570 con 0x7fdb4815ccb0 2023-10-21T17:41:09.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.880+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb481f5300 con 0x7fdb4815ccb0 2023-10-21T17:41:09.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.881+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb481f54f0 con 0x7fdb4815ccb0 2023-10-21T17:41:09.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.882+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb4002f050 con 0x7fdb4815ccb0 2023-10-21T17:41:09.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.884+0000 7fdb44ff9640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 0x7fdb2806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:09.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.884+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7fdb400abd70 con 0x7fdb4815ccb0 2023-10-21T17:41:09.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.884+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 0x7fdb2806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:09.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.886+0000 7fdb46ffd640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 0x7fdb2806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb38007f40 tx=0x7fdb38007490 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:09.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:09.888+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb400766e0 con 0x7fdb4815ccb0 2023-10-21T17:41:10.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.084+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdb480b5ae0 con 0x7fdb4815ccb0 2023-10-21T17:41:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.623+0000 7fdb44ff9640 1 -- 172.21.15.53:0/2469580488 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v279) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdb400350d0 con 0x7fdb4815ccb0 2023-10-21T17:41:10.625 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_metadata' removed 2023-10-21T17:41:10.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 msgr2=0x7fdb2806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 0x7fdb2806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb38007f40 tx=0x7fdb38007490 comp rx=0 tx=0).stop 2023-10-21T17:41:10.630 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 msgr2=0x7fdb480b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7fdb4002b930 tx=0x7fdb400046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 shutdown_connections 2023-10-21T17:41:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdb2806af90 0x7fdb2806d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.631 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb4815ccb0 0x7fdb480b2b40 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb480d0b90 0x7fdb480b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 --2- 172.21.15.53:0/2469580488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb480d00e0 0x7fdb481f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.629+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 >> 172.21.15.53:0/2469580488 conn(0x7fdb48071660 msgr2=0x7fdb480cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.630+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 shutdown_connections 2023-10-21T17:41:10.632 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.630+0000 7fdb47fff640 1 -- 172.21.15.53:0/2469580488 wait complete. 2023-10-21T17:41:10.658 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-21T17:41:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.885+0000 7feb31ebb640 1 Processor -- start 2023-10-21T17:41:10.887 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.886+0000 7feb31ebb640 1 -- start start 2023-10-21T17:41:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb31ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb2c069fc0 0x7feb2c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb31ebb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feb2c06ec00 0x7feb2c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.888 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb31ebb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2c160d10 con 0x7feb2c0ca210 2023-10-21T17:41:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb31ebb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2c072070 con 0x7feb2c06ec00 2023-10-21T17:41:10.889 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb31ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c071580 con 0x7feb2c069fc0 2023-10-21T17:41:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb316ba640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7feb2c0ca210 0x7feb2c160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:58760/0 (socket says 172.21.15.53:58760) 2023-10-21T17:41:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.887+0000 7feb316ba640 1 -- 172.21.15.53:0/2637142787 learned_addr learned my addr 172.21.15.53:0/2637142787 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:10.891 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2bfff640 1 --2- 172.21.15.53:0/2637142787 >> v2:172.21.15.124:3300/0 conn(0x7feb2c06ec00 0x7feb2c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb30eb9640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb2c069fc0 0x7feb2c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548982590 0 0) 0x7feb2c160d10 con 0x7feb2c0ca210 2023-10-21T17:41:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb1c003520 con 0x7feb2c0ca210 2023-10-21T17:41:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1777098568 0 0) 0x7feb1c003520 con 0x7feb2c0ca210 2023-10-21T17:41:10.892 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb2c069fc0 msgr2=0x7feb2c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:10.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb2c069fc0 0x7feb2c071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 >> v2:172.21.15.124:3300/0 conn(0x7feb2c06ec00 msgr2=0x7feb2c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:10.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 --2- 172.21.15.53:0/2637142787 >> v2:172.21.15.124:3300/0 conn(0x7feb2c06ec00 0x7feb2c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c072330 con 0x7feb2c0ca210 2023-10-21T17:41:10.893 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7feb240019f0 con 0x7feb2c0ca210 2023-10-21T17:41:10.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 >> v1:172.21.15.53:6789/0 conn(0x7feb2c0ca210 legacy=0x7feb2c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1c003970 0x7feb1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1c0047c0 0x7feb1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.894 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c005180 0x7feb1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c072070 con 0x7feb1c0047c0 2023-10-21T17:41:10.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c071580 con 0x7feb1c003970 2023-10-21T17:41:10.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c00bf70 con 0x7feb1c005180 2023-10-21T17:41:10.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.888+0000 7feb30eb9640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb2c069fc0 0x7feb2c071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:10.895 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1c0047c0 0x7feb1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb30eb9640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1c003970 0x7feb1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 -- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c005180 msgr2=0x7feb1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:10.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c005180 0x7feb1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 -- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1c003970 msgr2=0x7feb1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:10.896 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1c003970 0x7feb1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb2bfff640 1 -- 172.21.15.53:0/2637142787 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c003520 con 0x7feb1c0047c0 2023-10-21T17:41:10.897 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.889+0000 7feb30eb9640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb1c003970 0x7feb1c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:10.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb2bfff640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1c0047c0 0x7feb1c004c40 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7feb18009a90 tx=0x7feb1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a78dcd71ff3909b server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:10.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1802c6b0 con 0x7feb1c0047c0 2023-10-21T17:41:10.900 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1802c810 con 0x7feb1c0047c0 2023-10-21T17:41:10.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb2b7fe640 1 -- 172.21.15.53:0/2637142787 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1802cba0 con 0x7feb1c0047c0 2023-10-21T17:41:10.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb31ebb640 1 -- 172.21.15.53:0/2637142787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1c0047c0 msgr2=0x7feb1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb31ebb640 1 --2- 172.21.15.53:0/2637142787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb1c0047c0 0x7feb1c004c40 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7feb18009a90 tx=0x7feb1c0057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.891+0000 7feb316ba640 1 -- 172.21.15.53:0/2637142787 reap_dead start 2023-10-21T17:41:10.901 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.892+0000 7feb31ebb640 1 -- 172.21.15.53:0/2637142787 shutdown_connections 2023-10-21T17:41:10.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.892+0000 7feb31ebb640 1 -- 172.21.15.53:0/2637142787 >> 172.21.15.53:0/2637142787 conn(0x7feb2c0cf5f0 msgr2=0x7feb2c0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:10.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.892+0000 7feb31ebb640 1 -- 172.21.15.53:0/2637142787 shutdown_connections 2023-10-21T17:41:10.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.892+0000 7feb31ebb640 1 -- 172.21.15.53:0/2637142787 wait complete. 2023-10-21T17:41:10.902 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.893+0000 7feb31ebb640 1 Processor -- start 2023-10-21T17:41:10.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.894+0000 7feb31ebb640 1 -- start start 2023-10-21T17:41:10.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 0x7feb2c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 0x7feb2c0b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.903 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0b1af0 con 0x7feb2c0b1130 2023-10-21T17:41:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0af030 con 0x7feb2c0b0320 2023-10-21T17:41:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb31ebb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c0afe90 con 0x7feb1c0047c0 2023-10-21T17:41:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48550/0 (socket says 172.21.15.53:48550) 2023-10-21T17:41:10.904 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 -- 172.21.15.53:0/1576751726 learned_addr learned my addr 172.21.15.53:0/1576751726 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:10.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb2bfff640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 0x7feb2c0b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb30eb9640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 0x7feb2c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 -- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 msgr2=0x7feb2c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 0x7feb2c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.905 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 -- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 msgr2=0x7feb2c0b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:10.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 0x7feb2c0b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:10.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb316ba640 1 -- 172.21.15.53:0/1576751726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c156e00 con 0x7feb2c0b1130 2023-10-21T17:41:10.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb30eb9640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 0x7feb2c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:10.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.895+0000 7feb2bfff640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 0x7feb2c0b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:10.906 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb316ba640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c700 tx=0x7feb2400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24004d60 con 0x7feb2c0b1130 2023-10-21T17:41:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2400a640 con 0x7feb2c0b1130 2023-10-21T17:41:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2400a9b0 con 0x7feb2c0b1130 2023-10-21T17:41:10.907 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c0af280 con 0x7feb2c0b1130 2023-10-21T17:41:10.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.896+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c0af840 con 0x7feb2c0b1130 2023-10-21T17:41:10.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.897+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c074150 con 0x7feb2c0b1130 2023-10-21T17:41:10.908 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.898+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb2400ab10 con 0x7feb2c0b1130 2023-10-21T17:41:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.899+0000 7feb297fa640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 0x7feb1006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.899+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(279..279 src has 1..279) v4 ==== 6567+0+0 (secure 0 0 0) 0x7feb24088c10 con 0x7feb2c0b1130 2023-10-21T17:41:10.909 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.900+0000 7feb30eb9640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 0x7feb1006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.901+0000 7feb30eb9640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 0x7feb1006d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb20006e80 tx=0x7feb20008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:10.910 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:10.904+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb240537b0 con 0x7feb2c0b1130 2023-10-21T17:41:11.115 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.113+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb2c0affd0 con 0x7feb2c0b1130 2023-10-21T17:41:11.628 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.627+0000 7feb297fa640 1 -- 172.21.15.53:0/1576751726 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v280) v1 ==== 169+0+0 (secure 0 0 0) 0x7feb24056dd0 con 0x7feb2c0b1130 2023-10-21T17:41:11.629 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'cephfs_data' removed 2023-10-21T17:41:11.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.634+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 msgr2=0x7feb1006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.634+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 0x7feb1006d480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb20006e80 tx=0x7feb20008040 comp rx=0 tx=0).stop 2023-10-21T17:41:11.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.634+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 msgr2=0x7feb2c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.634+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c700 tx=0x7feb2400cbc0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 shutdown_connections 2023-10-21T17:41:11.637 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7feb1006afc0 0x7feb1006d480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb2c0b1130 0x7feb2c0b15b0 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb2c0b0320 0x7feb2c0b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 --2- 172.21.15.53:0/1576751726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb1c0047c0 0x7feb2c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 >> 172.21.15.53:0/1576751726 conn(0x7feb2c0cf5f0 msgr2=0x7feb2c15fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:11.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 shutdown_connections 2023-10-21T17:41:11.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.636+0000 7feb31ebb640 1 -- 172.21.15.53:0/1576751726 wait complete. 2023-10-21T17:41:11.664 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-21T17:41:11.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.865+0000 7f18b0202640 1 Processor -- start 2023-10-21T17:41:11.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.865+0000 7f18b0202640 1 -- start start 2023-10-21T17:41:11.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.866+0000 7f18b0202640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 0x7f18a8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.866+0000 7f18b0202640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f18a8071070 0x7f18a80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.866+0000 7f18b0202640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80b1cc0 con 0x7f18a80afbe0 2023-10-21T17:41:11.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.866+0000 7f18b0202640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a80b2620 con 0x7f18a8071070 2023-10-21T17:41:11.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.866+0000 7f18b0202640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80b2060 con 0x7f18a806dfc0 2023-10-21T17:41:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.867+0000 7f18ad79d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 0x7f18a8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.867+0000 7f18ae79f640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f18a8071070 0x7f18a80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.867+0000 7f18adf9e640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f18a80afbe0 0x7f18a80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.53:58092/0 (socket says 172.21.15.53:58092) 2023-10-21T17:41:11.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.867+0000 7f18adf9e640 1 -- 172.21.15.53:0/3449294617 learned_addr learned my addr 172.21.15.53:0/3449294617 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:11.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18acf9c640 1 -- 172.21.15.53:0/3449294617 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261874100 0 0) 0x7f18a80b1cc0 con 0x7f18a80afbe0 2023-10-21T17:41:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18acf9c640 1 -- 172.21.15.53:0/3449294617 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1894003520 con 0x7f18a80afbe0 2023-10-21T17:41:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18ae79f640 1 -- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 msgr2=0x7f18a8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18ae79f640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 0x7f18a8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18ae79f640 1 -- 172.21.15.53:0/3449294617 >> v1:172.21.15.124:6789/0 conn(0x7f18a80afbe0 legacy=0x7f18a80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18ae79f640 1 -- 172.21.15.53:0/3449294617 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80b2340 con 0x7f18a8071070 2023-10-21T17:41:11.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.868+0000 7f18ad79d640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 0x7f18a8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-21T17:41:11.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.870+0000 7f18ae79f640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a8071070 0x7f18a80cb130 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009d90 tx=0x7f18a002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7379c725b21dc1ae server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:11.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.871+0000 7f18acf9c640 1 -- 172.21.15.53:0/3449294617 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a002cc60 con 0x7f18a8071070 2023-10-21T17:41:11.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.871+0000 7f18acf9c640 1 -- 172.21.15.53:0/3449294617 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a002cdc0 con 0x7f18a8071070 2023-10-21T17:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.871+0000 7f18acf9c640 1 -- 172.21.15.53:0/3449294617 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0034640 con 0x7f18a8071070 2023-10-21T17:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.871+0000 7f18b0202640 1 -- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a8071070 msgr2=0x7f18a80cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.871+0000 7f18b0202640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a8071070 0x7f18a80cb130 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009d90 tx=0x7f18a002beb0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 -- 172.21.15.53:0/3449294617 shutdown_connections 2023-10-21T17:41:11.874 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a806dfc0 0x7f18a8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 --2- 172.21.15.53:0/3449294617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a8071070 0x7f18a80cb130 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 -- 172.21.15.53:0/3449294617 >> 172.21.15.53:0/3449294617 conn(0x7f18a806a5d0 msgr2=0x7f18a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 -- 172.21.15.53:0/3449294617 shutdown_connections 2023-10-21T17:41:11.875 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.872+0000 7f18b0202640 1 -- 172.21.15.53:0/3449294617 wait complete. 2023-10-21T17:41:11.877 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.874+0000 7f18b0202640 1 Processor -- start 2023-10-21T17:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.874+0000 7f18b0202640 1 -- start start 2023-10-21T17:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a806dfc0 0x7f18a80c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a8071070 0x7f18a814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.878 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a814e3a0 con 0x7f18a80afbe0 2023-10-21T17:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cf520 con 0x7f18a806dfc0 2023-10-21T17:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18b0202640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80c8d40 con 0x7f18a8071070 2023-10-21T17:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18ae79f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.879 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18ae79f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48556/0 (socket says 172.21.15.53:48556) 2023-10-21T17:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18ae79f640 1 -- 172.21.15.53:0/240991038 learned_addr learned my addr 172.21.15.53:0/240991038 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18ad79d640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a8071070 0x7f18a814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.875+0000 7f18adf9e640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a806dfc0 0x7f18a80c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 -- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a8071070 msgr2=0x7f18a814de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.880 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a8071070 0x7f18a814de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 -- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a806dfc0 msgr2=0x7f18a80c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a806dfc0 0x7f18a80c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 -- 172.21.15.53:0/240991038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c009400 con 0x7f18a80afbe0 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18ae79f640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009d90 tx=0x7f18a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0004880 con 0x7f18a80afbe0 2023-10-21T17:41:11.881 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0009450 con 0x7f18a80afbe0 2023-10-21T17:41:11.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0034640 con 0x7f18a80afbe0 2023-10-21T17:41:11.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a0009a70 con 0x7f18a80afbe0 2023-10-21T17:41:11.882 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.876+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80c9300 con 0x7f18a80afbe0 2023-10-21T17:41:11.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.877+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a804e160 con 0x7f18a80afbe0 2023-10-21T17:41:11.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.878+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f18a003b030 con 0x7f18a80afbe0 2023-10-21T17:41:11.883 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.880+0000 7f189a7fc640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 0x7f188c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.880+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f18a00abae0 con 0x7f18a80afbe0 2023-10-21T17:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.881+0000 7f18adf9e640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 0x7f188c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:11.884 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.882+0000 7f18adf9e640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 0x7f188c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f18a4007fd0 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:11.885 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:11.884+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18a0076970 con 0x7f18a80afbe0 2023-10-21T17:41:12.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.080+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f18a80c5280 con 0x7f18a80afbe0 2023-10-21T17:41:12.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.087+0000 7f189a7fc640 1 -- 172.21.15.53:0/240991038 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18a0079f90 con 0x7f18a80afbe0 2023-10-21T17:41:12.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 msgr2=0x7f188c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 0x7f188c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f18a4007fd0 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2023-10-21T17:41:12.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 msgr2=0x7f18a80ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009d90 tx=0x7f18a0004a40 comp rx=0 tx=0).stop 2023-10-21T17:41:12.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 shutdown_connections 2023-10-21T17:41:12.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f188c06af90 0x7f188c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f18a80afbe0 0x7f18a80ceb20 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18a8071070 0x7f18a814de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 --2- 172.21.15.53:0/240991038 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18a806dfc0 0x7f18a80c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 >> 172.21.15.53:0/240991038 conn(0x7f18a806a5d0 msgr2=0x7f18a815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:12.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 shutdown_connections 2023-10-21T17:41:12.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.091+0000 7f18b0202640 1 -- 172.21.15.53:0/240991038 wait complete. 2023-10-21T17:41:12.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-21T17:41:12.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.338+0000 7fe1d6135640 1 Processor -- start 2023-10-21T17:41:12.340 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.338+0000 7fe1d6135640 1 -- start start 2023-10-21T17:41:12.341 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.340+0000 7fe1d6135640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c8fb0 0x7fe1d00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.340+0000 7fe1d6135640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015c630 0x7fe1d015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.340+0000 7fe1d6135640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00b0260 con 0x7fe1d00c9390 2023-10-21T17:41:12.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.340+0000 7fe1d6135640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d01612d0 con 0x7fe1d015c630 2023-10-21T17:41:12.342 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.340+0000 7fe1d6135640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0161590 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015c630 0x7fe1d015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe1d015c630 0x7fe1d015ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48572/0 (socket says 172.21.15.53:48572) 2023-10-21T17:41:12.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 -- 172.21.15.53:0/942734641 learned_addr learned my addr 172.21.15.53:0/942734641 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:12.344 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5133640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c8fb0 0x7fe1d00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 -- 172.21.15.53:0/942734641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c8fb0 msgr2=0x7fe1d00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:12.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c8fb0 0x7fe1d00cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 -- 172.21.15.53:0/942734641 >> v1:172.21.15.124:6789/0 conn(0x7fe1d00c9390 legacy=0x7fe1d0160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:12.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.341+0000 7fe1d5934640 1 -- 172.21.15.53:0/942734641 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00704b0 con 0x7fe1d015c630 2023-10-21T17:41:12.345 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.342+0000 7fe1d5934640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015c630 0x7fe1d015ca50 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009e50 tx=0x7fe1c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ec5a01c86162b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1c7fff640 1 -- 172.21.15.53:0/942734641 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c802cb00 con 0x7fe1d015c630 2023-10-21T17:41:12.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1c7fff640 1 -- 172.21.15.53:0/942734641 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c802cc60 con 0x7fe1d015c630 2023-10-21T17:41:12.346 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1c7fff640 1 -- 172.21.15.53:0/942734641 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80343f0 con 0x7fe1d015c630 2023-10-21T17:41:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 -- 172.21.15.53:0/942734641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015c630 msgr2=0x7fe1d015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015c630 0x7fe1d015ca50 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009e50 tx=0x7fe1c802bf60 comp rx=0 tx=0).stop 2023-10-21T17:41:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 -- 172.21.15.53:0/942734641 shutdown_connections 2023-10-21T17:41:12.348 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d015c630 0x7fe1d015ca50 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 --2- 172.21.15.53:0/942734641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c8fb0 0x7fe1d00cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 -- 172.21.15.53:0/942734641 >> 172.21.15.53:0/942734641 conn(0x7fe1d006c010 msgr2=0x7fe1d006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.343+0000 7fe1d6135640 1 -- 172.21.15.53:0/942734641 shutdown_connections 2023-10-21T17:41:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.344+0000 7fe1d6135640 1 -- 172.21.15.53:0/942734641 wait complete. 2023-10-21T17:41:12.349 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.345+0000 7fe1d6135640 1 Processor -- start 2023-10-21T17:41:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.345+0000 7fe1d6135640 1 -- start start 2023-10-21T17:41:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.346+0000 7fe1d6135640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.346+0000 7fe1d6135640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c9390 0x7fe1d014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.346+0000 7fe1d6135640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1d015c630 0x7fe1d015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.350 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.346+0000 7fe1d6135640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d01574a0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.346+0000 7fe1d6135640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d015d6e0 con 0x7fe1d015c630 2023-10-21T17:41:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d6135640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d015d910 con 0x7fe1d00c9390 2023-10-21T17:41:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48582/0 (socket says 172.21.15.53:48582) 2023-10-21T17:41:12.351 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 -- 172.21.15.53:0/1815837958 learned_addr learned my addr 172.21.15.53:0/1815837958 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 -- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c9390 msgr2=0x7fe1d014ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c9390 0x7fe1d014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5934640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1d015c630 0x7fe1d015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 -- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1d015c630 msgr2=0x7fe1d015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1d015c630 0x7fe1d015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.347+0000 7fe1d5133640 1 -- 172.21.15.53:0/1815837958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8009b30 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1d5133640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d800 tx=0x7fe1cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.354 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00b6c0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc00b820 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00bbb0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d015dbf0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.348+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d015e1b0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.355 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.349+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d0074150 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.350+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe1cc00bd10 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.351+0000 7fe1c5ffb640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 0x7fe1b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.351+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fe1cc089e20 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.356 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.352+0000 7fe1d4932640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 0x7fe1b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.353+0000 7fe1d4932640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 0x7fe1b406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00c8970 tx=0x7fe1c00079b0 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.357 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.356+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1cc054c90 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.550 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.549+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe1d00c9a00 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.556+0000 7fe1c5ffb640 1 -- 172.21.15.53:0/1815837958 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v92) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe1cc0582b0 con 0x7fe1d00c8fb0 2023-10-21T17:41:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 msgr2=0x7fe1b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 0x7fe1b406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00c8970 tx=0x7fe1c00079b0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 msgr2=0x7fe1d00b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d800 tx=0x7fe1cc00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 shutdown_connections 2023-10-21T17:41:12.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fe1b406af90 0x7fe1b406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1d015c630 0x7fe1d015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1d00c9390 0x7fe1d014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 --2- 172.21.15.53:0/1815837958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1d00c8fb0 0x7fe1d00b3980 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.558+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 >> 172.21.15.53:0/1815837958 conn(0x7fe1d006c010 msgr2=0x7fe1d01606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.559+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 shutdown_connections 2023-10-21T17:41:12.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.559+0000 7fe1d6135640 1 -- 172.21.15.53:0/1815837958 wait complete. 2023-10-21T17:41:12.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-21T17:41:12.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.810+0000 7fed8584b640 1 Processor -- start 2023-10-21T17:41:12.812 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.810+0000 7fed8584b640 1 -- start start 2023-10-21T17:41:12.813 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed8584b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed800c8e70 0x7fed800ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed8584b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 0x7fed800b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed8584b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8006d490 con 0x7fed800cae90 2023-10-21T17:41:12.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed8584b640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8006c880 con 0x7fed800c8e70 2023-10-21T17:41:12.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed8584b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8006cb60 con 0x7fed8006e3a0 2023-10-21T17:41:12.814 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed7effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed800c8e70 0x7fed800ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed7effd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed800c8e70 0x7fed800ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48596/0 (socket says 172.21.15.53:48596) 2023-10-21T17:41:12.815 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.812+0000 7fed7effd640 1 -- 172.21.15.53:0/3147172748 learned_addr learned my addr 172.21.15.53:0/3147172748 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:12.816 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed7effd640 1 -- 172.21.15.53:0/3147172748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 msgr2=0x7fed800b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:12.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed77fff640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 0x7fed800b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed7effd640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 0x7fed800b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed7effd640 1 -- 172.21.15.53:0/3147172748 >> v1:172.21.15.124:6789/0 conn(0x7fed800cae90 legacy=0x7fed800cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:12.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed7effd640 1 -- 172.21.15.53:0/3147172748 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8006ce40 con 0x7fed800c8e70 2023-10-21T17:41:12.817 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.813+0000 7fed77fff640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 0x7fed800b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:12.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.814+0000 7fed7effd640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed800c8e70 0x7fed800ca0b0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fed70009870 tx=0x7fed7002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb80dee125509f35 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.814+0000 7fed7e7fc640 1 -- 172.21.15.53:0/3147172748 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70031640 con 0x7fed800c8e70 2023-10-21T17:41:12.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed7e7fc640 1 -- 172.21.15.53:0/3147172748 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700317a0 con 0x7fed800c8e70 2023-10-21T17:41:12.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed7e7fc640 1 -- 172.21.15.53:0/3147172748 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70031b30 con 0x7fed800c8e70 2023-10-21T17:41:12.818 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 -- 172.21.15.53:0/3147172748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed800c8e70 msgr2=0x7fed800ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed800c8e70 0x7fed800ca0b0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7fed70009870 tx=0x7fed7002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 -- 172.21.15.53:0/3147172748 shutdown_connections 2023-10-21T17:41:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed8006e3a0 0x7fed800b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 --2- 172.21.15.53:0/3147172748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed800c8e70 0x7fed800ca0b0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.819 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 -- 172.21.15.53:0/3147172748 >> 172.21.15.53:0/3147172748 conn(0x7fed8006a5d0 msgr2=0x7fed800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 -- 172.21.15.53:0/3147172748 shutdown_connections 2023-10-21T17:41:12.820 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.815+0000 7fed8584b640 1 -- 172.21.15.53:0/3147172748 wait complete. 2023-10-21T17:41:12.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.817+0000 7fed8584b640 1 Processor -- start 2023-10-21T17:41:12.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.817+0000 7fed8584b640 1 -- start start 2023-10-21T17:41:12.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 0x7fed80152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.822 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed800cae90 0x7fed800c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed800c7810 con 0x7fed8006e3a0 2023-10-21T17:41:12.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801532f0 con 0x7fed800c8e70 2023-10-21T17:41:12.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.818+0000 7fed8584b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed80152010 con 0x7fed800cae90 2023-10-21T17:41:12.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.823 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48612/0 (socket says 172.21.15.53:48612) 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 -- 172.21.15.53:0/1503055785 learned_addr learned my addr 172.21.15.53:0/1503055785 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed77fff640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 0x7fed80152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 -- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed800cae90 msgr2=0x7fed800c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed800cae90 0x7fed800c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 -- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 msgr2=0x7fed80152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:12.824 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 0x7fed80152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:12.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 -- 172.21.15.53:0/1503055785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed700094d0 con 0x7fed8006e3a0 2023-10-21T17:41:12.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed77fff640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 0x7fed80152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:12.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.819+0000 7fed7effd640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fed70009870 tx=0x7fed700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.820+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7003ac60 con 0x7fed8006e3a0 2023-10-21T17:41:12.825 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.820+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7003ade0 con 0x7fed8006e3a0 2023-10-21T17:41:12.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.820+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70044630 con 0x7fed8006e3a0 2023-10-21T17:41:12.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.820+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed801522d0 con 0x7fed8006e3a0 2023-10-21T17:41:12.826 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.820+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed80152890 con 0x7fed8006e3a0 2023-10-21T17:41:12.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.821+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8004e160 con 0x7fed8006e3a0 2023-10-21T17:41:12.827 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.822+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fed7002f050 con 0x7fed8006e3a0 2023-10-21T17:41:12.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.823+0000 7fed7cff9640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 0x7fed6006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:12.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.823+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fed700abc20 con 0x7fed8006e3a0 2023-10-21T17:41:12.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.824+0000 7fed77fff640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 0x7fed6006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:12.828 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.825+0000 7fed77fff640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 0x7fed6006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed68006fd0 tx=0x7fed68008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:12.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:12.827+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed700768e0 con 0x7fed8006e3a0 2023-10-21T17:41:13.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.018+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fed800c93f0 con 0x7fed8006e3a0 2023-10-21T17:41:13.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.026+0000 7fed7cff9640 1 -- 172.21.15.53:0/1503055785 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v93) v1 ==== 120+0+0 (secure 0 0 0) 0x7fed7004e020 con 0x7fed8006e3a0 2023-10-21T17:41:13.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.028+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 msgr2=0x7fed6006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.028+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 0x7fed6006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fed68006fd0 tx=0x7fed68008040 comp rx=0 tx=0).stop 2023-10-21T17:41:13.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.028+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 msgr2=0x7fed800c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.028+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fed70009870 tx=0x7fed700046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 shutdown_connections 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fed6006af90 0x7fed6006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed800cae90 0x7fed800c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed800c8e70 0x7fed80152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 --2- 172.21.15.53:0/1503055785 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed8006e3a0 0x7fed800c72d0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 >> 172.21.15.53:0/1503055785 conn(0x7fed8006a5d0 msgr2=0x7fed800cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:13.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 shutdown_connections 2023-10-21T17:41:13.033 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.029+0000 7fed8584b640 1 -- 172.21.15.53:0/1503055785 wait complete. 2023-10-21T17:41:13.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-21T17:41:13.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.263+0000 7fcc8359e640 1 Processor -- start 2023-10-21T17:41:13.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.263+0000 7fcc8359e640 1 -- start start 2023-10-21T17:41:13.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8359e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcc8415a970 0x7fcc840cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8359e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc8406dd00 0x7fcc840cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8359e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc840ca9e0 con 0x7fcc840c9e60 2023-10-21T17:41:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8359e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8406cfd0 con 0x7fcc8415a970 2023-10-21T17:41:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8359e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8406d870 con 0x7fcc8406dd00 2023-10-21T17:41:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcc8415a970 0x7fcc840cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:13.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcc8415a970 0x7fcc840cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48626/0 (socket says 172.21.15.53:48626) 2023-10-21T17:41:13.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 -- 172.21.15.53:0/4203196698 learned_addr learned my addr 172.21.15.53:0/4203196698 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:13.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 -- 172.21.15.53:0/4203196698 >> v2:172.21.15.124:3300/0 conn(0x7fcc8406dd00 msgr2=0x7fcc840cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:13.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 --2- 172.21.15.53:0/4203196698 >> v2:172.21.15.124:3300/0 conn(0x7fcc8406dd00 0x7fcc840cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.265+0000 7fcc8259c640 1 -- 172.21.15.53:0/4203196698 >> v1:172.21.15.124:6789/0 conn(0x7fcc840c9e60 legacy=0x7fcc840ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.266+0000 7fcc8259c640 1 -- 172.21.15.53:0/4203196698 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8415ad50 con 0x7fcc8415a970 2023-10-21T17:41:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.267+0000 7fcc8259c640 1 --2- 172.21.15.53:0/4203196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc840cb450 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fcc840caf90 tx=0x7fcc7402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb30f208c81b8a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.267+0000 7fcc81d9b640 1 -- 172.21.15.53:0/4203196698 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7402c950 con 0x7fcc8415a970 2023-10-21T17:41:13.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.267+0000 7fcc81d9b640 1 -- 172.21.15.53:0/4203196698 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7402cab0 con 0x7fcc8415a970 2023-10-21T17:41:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.267+0000 7fcc81d9b640 1 -- 172.21.15.53:0/4203196698 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7402ce40 con 0x7fcc8415a970 2023-10-21T17:41:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 -- 172.21.15.53:0/4203196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 msgr2=0x7fcc840cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 --2- 172.21.15.53:0/4203196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc840cb450 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fcc840caf90 tx=0x7fcc7402bdb0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 -- 172.21.15.53:0/4203196698 shutdown_connections 2023-10-21T17:41:13.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 --2- 172.21.15.53:0/4203196698 >> v2:172.21.15.124:3300/0 conn(0x7fcc8406dd00 0x7fcc840cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 --2- 172.21.15.53:0/4203196698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc840cb450 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 -- 172.21.15.53:0/4203196698 >> 172.21.15.53:0/4203196698 conn(0x7fcc840ce9d0 msgr2=0x7fcc840cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:13.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 -- 172.21.15.53:0/4203196698 shutdown_connections 2023-10-21T17:41:13.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.268+0000 7fcc8359e640 1 -- 172.21.15.53:0/4203196698 wait complete. 2023-10-21T17:41:13.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.270+0000 7fcc8359e640 1 Processor -- start 2023-10-21T17:41:13.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.270+0000 7fcc8359e640 1 -- start start 2023-10-21T17:41:13.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc8406dd00 0x7fcc841f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc840c9e60 0x7fcc841595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84159b30 con 0x7fcc8415a970 2023-10-21T17:41:13.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc841f3650 con 0x7fcc840c9e60 2023-10-21T17:41:13.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc8359e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc841f38f0 con 0x7fcc8406dd00 2023-10-21T17:41:13.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:13.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc82d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:48634/0 (socket says 172.21.15.53:48634) 2023-10-21T17:41:13.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.271+0000 7fcc82d9d640 1 -- 172.21.15.53:0/2647787478 learned_addr learned my addr 172.21.15.53:0/2647787478 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:13.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc8259c640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc8406dd00 0x7fcc841f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:13.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc79d9b640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc840c9e60 0x7fcc841595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:13.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 -- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc8406dd00 msgr2=0x7fcc841f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc8406dd00 0x7fcc841f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 -- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc840c9e60 msgr2=0x7fcc841595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc840c9e60 0x7fcc841595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 -- 172.21.15.53:0/2647787478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74009b30 con 0x7fcc8415a970 2023-10-21T17:41:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc82d9d640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c390 tx=0x7fcc7c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c0035f0 con 0x7fcc8415a970 2023-10-21T17:41:13.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c003750 con 0x7fcc8415a970 2023-10-21T17:41:13.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c003b00 con 0x7fcc8415a970 2023-10-21T17:41:13.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.272+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc841f3bd0 con 0x7fcc8415a970 2023-10-21T17:41:13.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.273+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc841f4190 con 0x7fcc8415a970 2023-10-21T17:41:13.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.274+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc840c8a00 con 0x7fcc8415a970 2023-10-21T17:41:13.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.275+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc7c003c60 con 0x7fcc8415a970 2023-10-21T17:41:13.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.276+0000 7fcc7b7fe640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 0x7fcc6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:13.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.276+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fcc7c088b40 con 0x7fcc8415a970 2023-10-21T17:41:13.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.277+0000 7fcc8259c640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 0x7fcc6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:13.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.278+0000 7fcc8259c640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 0x7fcc6406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc74035000 tx=0x7fcc74030000 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:13.283 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.281+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc7c0539b0 con 0x7fcc8415a970 2023-10-21T17:41:13.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.546+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fcc8404e160 con 0x7fcc8415a970 2023-10-21T17:41:13.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.643+0000 7fcc7b7fe640 1 -- 172.21.15.53:0/2647787478 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3309) v1 ==== 155+0+0 (secure 0 0 0) 0x7fcc7c0170e0 con 0x7fcc8415a970 2023-10-21T17:41:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 msgr2=0x7fcc6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 0x7fcc6406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcc74035000 tx=0x7fcc74030000 comp rx=0 tx=0).stop 2023-10-21T17:41:13.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 msgr2=0x7fcc841f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c390 tx=0x7fcc7c00c850 comp rx=0 tx=0).stop 2023-10-21T17:41:13.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 shutdown_connections 2023-10-21T17:41:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc6406af90 0x7fcc6406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc8415a970 0x7fcc841f3110 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc840c9e60 0x7fcc841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 --2- 172.21.15.53:0/2647787478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc8406dd00 0x7fcc841f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:13.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 >> 172.21.15.53:0/2647787478 conn(0x7fcc840ce9d0 msgr2=0x7fcc84069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:13.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 shutdown_connections 2023-10-21T17:41:13.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:13.647+0000 7fcc8359e640 1 -- 172.21.15.53:0/2647787478 wait complete. 2023-10-21T17:41:13.667 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-21T17:41:13.668 INFO:tasks.cephfs_test_runner: 2023-10-21T17:41:13.668 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-21T17:41:13.669 INFO:tasks.cephfs_test_runner:Ran 14 tests in 5892.578s 2023-10-21T17:41:13.669 INFO:tasks.cephfs_test_runner: 2023-10-21T17:41:13.669 INFO:tasks.cephfs_test_runner:OK 2023-10-21T17:41:13.670 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-21T17:41:13.679 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-21T17:41:13.730 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-21T17:41:13.730 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:41:13.731 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:13.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:41:13.794 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-10-21T17:41:13.794 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:13.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-21T17:41:13.853 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:41:13.853 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:13.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:41:13.915 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2023-10-21T17:41:13.915 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-21T17:41:13.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054284,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-21T17:41:13.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:13.970 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:41:14.034 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-21T17:41:14.034 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-21T17:41:14.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.036 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-21T17:41:14.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-21T17:41:14.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-10-21T17:41:14.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.377 DEBUG:teuthology.orchestra.run.smithi053:> ip netns list 2023-10-21T17:41:14.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo ip link delete ceph-brx 2023-10-21T17:41:14.574 INFO:teuthology.orchestra.run.smithi053.stderr:Cannot find device "ceph-brx" 2023-10-21T17:41:14.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:41:14.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.578 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T17:41:14.639 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-21T17:41:14.639 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-21T17:41:14.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.641 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-21T17:41:14.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.742 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-21T17:41:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.833 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-21T17:41:14.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.942 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2023-10-21T17:41:14.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-21T17:41:14.992 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2023-10-21T17:41:15.132 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2023-10-21T17:41:15.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:41:15.136 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-21T17:41:15.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-21T17:41:15.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:41:15.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.384+0000 7fdf2b59e640 1 Processor -- start 2023-10-21T17:41:15.387 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.384+0000 7fdf2b59e640 1 -- start start 2023-10-21T17:41:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2b59e640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf2c0adc70 0x7fdf2c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf2c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2b59e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c15f7d0 con 0x7fdf2c15ab30 2023-10-21T17:41:15.388 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2b59e640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c15efb0 con 0x7fdf2c0adc70 2023-10-21T17:41:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15f210 con 0x7fdf2c15af10 2023-10-21T17:41:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2a59c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fdf2c15ab30 0x7fdf2c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53108/0 (socket says 172.21.15.53:53108) 2023-10-21T17:41:15.389 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.386+0000 7fdf2a59c640 1 -- 172.21.15.53:0/2302411650 learned_addr learned my addr 172.21.15.53:0/2302411650 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310767846 0 0) 0x7fdf2c15f7d0 con 0x7fdf2c15ab30 2023-10-21T17:41:15.391 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf18003520 con 0x7fdf2c15ab30 2023-10-21T17:41:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf23fff640 1 --2- 172.21.15.53:0/2302411650 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c0adc70 0x7fdf2c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf2ad9d640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf2c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247521557 0 0) 0x7fdf18003520 con 0x7fdf2c15ab30 2023-10-21T17:41:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 msgr2=0x7fdf2c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.392 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf2c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c0adc70 msgr2=0x7fdf2c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 --2- 172.21.15.53:0/2302411650 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c0adc70 0x7fdf2c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c15f4f0 con 0x7fdf2c15ab30 2023-10-21T17:41:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fdf1c001690 con 0x7fdf2c15ab30 2023-10-21T17:41:15.393 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 >> v1:172.21.15.53:6789/0 conn(0x7fdf2c15ab30 legacy=0x7fdf2c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf18003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18003ff0 0x7fdf180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf18004d00 0x7fdf18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15efb0 con 0x7fdf18004d00 2023-10-21T17:41:15.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15f210 con 0x7fdf18003ff0 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1800bae0 con 0x7fdf2c15af10 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf23fff640 1 --2- 172.21.15.53:0/2302411650 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c0adc70 0x7fdf2c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.387+0000 7fdf2ad9d640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf18004d00 0x7fdf18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf18003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2ad9d640 1 -- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 msgr2=0x7fdf18003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.395 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2ad9d640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf18003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2ad9d640 1 -- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18003ff0 msgr2=0x7fdf180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf23fff640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18003ff0 0x7fdf180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2ad9d640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18003ff0 0x7fdf180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2ad9d640 1 -- 172.21.15.53:0/2302411650 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf18003520 con 0x7fdf18004d00 2023-10-21T17:41:15.396 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf23fff640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18003ff0 0x7fdf180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.388+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c15af10 0x7fdf18003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:15.397 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.389+0000 7fdf2ad9d640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf18004d00 0x7fdf18005180 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007360 tx=0x7fdf180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c897443963bcd2d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2402ccd0 con 0x7fdf18004d00 2023-10-21T17:41:15.399 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2402ce30 con 0x7fdf18004d00 2023-10-21T17:41:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf29d9b640 1 -- 172.21.15.53:0/2302411650 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24034660 con 0x7fdf18004d00 2023-10-21T17:41:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2b59e640 1 -- 172.21.15.53:0/2302411650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf18004d00 msgr2=0x7fdf18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/2302411650 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf18004d00 0x7fdf18005180 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007360 tx=0x7fdf180057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2ad9d640 1 -- 172.21.15.53:0/2302411650 reap_dead start 2023-10-21T17:41:15.400 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2b59e640 1 -- 172.21.15.53:0/2302411650 shutdown_connections 2023-10-21T17:41:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2b59e640 1 -- 172.21.15.53:0/2302411650 >> 172.21.15.53:0/2302411650 conn(0x7fdf2c0cb0c0 msgr2=0x7fdf2c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.390+0000 7fdf2b59e640 1 -- 172.21.15.53:0/2302411650 shutdown_connections 2023-10-21T17:41:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.391+0000 7fdf2b59e640 1 -- 172.21.15.53:0/2302411650 wait complete. 2023-10-21T17:41:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.392+0000 7fdf2b59e640 1 Processor -- start 2023-10-21T17:41:15.401 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.392+0000 7fdf2b59e640 1 -- start start 2023-10-21T17:41:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 0x7fdf2c14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c0c7330 0x7fdf2c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b1eb0 con 0x7fdf2c0b2340 2023-10-21T17:41:15.402 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0c7c20 con 0x7fdf18004d00 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.393+0000 7fdf2b59e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c151eb0 con 0x7fdf2c0c7330 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54356/0 (socket says 172.21.15.53:54356) 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 -- 172.21.15.53:0/97438999 learned_addr learned my addr 172.21.15.53:0/97438999 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 0x7fdf2c14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.403 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 -- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c0c7330 msgr2=0x7fdf2c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c0c7330 0x7fdf2c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 -- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 msgr2=0x7fdf2c14ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 0x7fdf2c14ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 -- 172.21.15.53:0/97438999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c14f440 con 0x7fdf2c0b2340 2023-10-21T17:41:15.404 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 0x7fdf2c14ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.394+0000 7fdf23fff640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400f5a0 tx=0x7fdf1400fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.395+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14014890 con 0x7fdf2c0b2340 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.395+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1400c030 con 0x7fdf2c0b2340 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.395+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14014bf0 con 0x7fdf2c0b2340 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.395+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0c7940 con 0x7fdf2c0b2340 2023-10-21T17:41:15.405 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.395+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0b1b40 con 0x7fdf2c0b2340 2023-10-21T17:41:15.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.396+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c0ca850 con 0x7fdf2c0b2340 2023-10-21T17:41:15.406 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.397+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf14014d50 con 0x7fdf2c0b2340 2023-10-21T17:41:15.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.398+0000 7fdf22ffd640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 0x7fdf0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.398+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fdf14089dd0 con 0x7fdf2c0b2340 2023-10-21T17:41:15.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.399+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 0x7fdf0c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.407 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.400+0000 7fdf2a59c640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 0x7fdf0c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c001eb0 tx=0x7fdf1c009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.408 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.403+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf14054aa0 con 0x7fdf2c0b2340 2023-10-21T17:41:15.596 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.594+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf2c06c830 con 0x7fdf0c06af90 2023-10-21T17:41:15.604 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.603+0000 7fdf22ffd640 1 -- 172.21.15.53:0/97438999 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48418 (secure 0 0 0) 0x7fdf1c00d040 con 0x7fdf0c06af90 2023-10-21T17:41:15.605 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:15.606 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:41:15.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 msgr2=0x7fdf0c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.607 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 0x7fdf0c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c001eb0 tx=0x7fdf1c009040 comp rx=0 tx=0).stop 2023-10-21T17:41:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 msgr2=0x7fdf2c0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400f5a0 tx=0x7fdf1400fa60 comp rx=0 tx=0).stop 2023-10-21T17:41:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 shutdown_connections 2023-10-21T17:41:15.608 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fdf0c06af90 0x7fdf0c06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c0c7330 0x7fdf2c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf2c0b2340 0x7fdf2c0c6df0 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 --2- 172.21.15.53:0/97438999 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf18004d00 0x7fdf2c14ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 >> 172.21.15.53:0/97438999 conn(0x7fdf2c0cb0c0 msgr2=0x7fdf2c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:15.609 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 shutdown_connections 2023-10-21T17:41:15.610 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.605+0000 7fdf2b59e640 1 -- 172.21.15.53:0/97438999 wait complete. 2023-10-21T17:41:15.627 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-21T17:41:14.602217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462640,"kb_used_data":29968,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754512080,"statfs":{"total":773094113280,"available":772620369920,"internally_reserved":0,"allocated":30687232,"data_stored":17246620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668451,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.009917"},"pg_stats":[{"pgid":"1.0","version":"277'4931","reported_seq":6047,"reported_epoch":277,"state":"active+clean","last_fresh":"2023-10-21T17:41:06.721440+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:41:06.721440+0000","last_peered":"2023-10-21T17:41:06.721440+0000","last_clean":"2023-10-21T17:41:06.721440+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:41:06.721440+0000","last_undegraded":"2023-10-21T17:41:06.721440+0000","last_fullsized":"2023-10-21T17:41:06.721440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576026,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75800,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296040,"statfs":{"total":96636764160,"available":96559144960,"internally_reserved":0,"allocated":4468736,"data_stored":2773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.76700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.724,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.282},{"interface":"front","average":{"1min":1.165,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.77100000000000002,"5min":0.64300000000000002,"15min":0.63800000000000001},"max":{"1min":1.5069999999999999,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0329999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.175,"15min":1.1910000000000001},"min":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.57099999999999995},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.78600000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.909,"15min":1.909},"last":1.4970000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3540000000000001,"15min":1.3320000000000001},"min":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.65700000000000003},"max":{"1min":1.849,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.577},{"interface":"front","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.667,"5min":3.556,"15min":3.556},"last":1.2050000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.306},"min":{"1min":0.623,"5min":0.623,"15min":0.50700000000000001},"max":{"1min":2.3439999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.387},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.486,"15min":1.4610000000000001},"min":{"1min":1.087,"5min":0.69999999999999996,"15min":0.58899999999999997},"max":{"1min":1.859,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.643}]},{"osd":4,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3859999999999999,"15min":1.361},"min":{"1min":0.64100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7350000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.90000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3799999999999999,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5,"15min":1.431},"min":{"1min":0.74399999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9770000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.514,"15min":1.464},"min":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0379999999999998,"5min":2.613,"15min":2.613},"last":1.4430000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.613,"15min":1.5640000000000001},"min":{"1min":1.181,"5min":0.625,"15min":0.625},"max":{"1min":1.964,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.696},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5600000000000001,"15min":1.4990000000000001},"min":{"1min":0.99299999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2410000000000001,"5min":2.879,"15min":2.879},"last":1.54}]}]},{"osd":6,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65460,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306380,"statfs":{"total":96636764160,"available":96569733120,"internally_reserved":0,"allocated":3842048,"data_stored":2168491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.119,"15min":1.1679999999999999},"min":{"1min":0.70499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6319999999999999,"5min":1.944,"15min":1.944},"last":1.173},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.147,"15min":1.1890000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5249999999999999,"5min":1.6619999999999999,"15min":1.762},"last":1.028}]},{"osd":1,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.121,"15min":1.1579999999999999},"min":{"1min":0.73199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4990000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.198,"15min":1.278},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.5620000000000001,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2450000000000001,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.266,"15min":1.288},"min":{"1min":0.80700000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6519999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.242}]},{"osd":3,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3009999999999999,"15min":1.3839999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.4790000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3009999999999999,"15min":1.3360000000000001},"min":{"1min":0.47699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.9910000000000001,"5min":2.419,"15min":2.419},"last":1.288},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4299999999999999},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0299999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.552,"15min":1.556},"min":{"1min":0.77400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9450000000000001,"5min":2.1299999999999999,"15min":2.1739999999999999},"last":1.419},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3560000000000001,"15min":1.361},"min":{"1min":0.56100000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.9219999999999999,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.0740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4690000000000001,"15min":1.5169999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0550000000000002,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.119},{"interface":"front","average":{"1min":1.663,"5min":1.639,"15min":1.651},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0219999999999998,"5min":2.21,"15min":2.2610000000000001},"last":1.6359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576028,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48504,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323336,"statfs":{"total":96636764160,"available":96587096064,"internally_reserved":0,"allocated":3518464,"data_stored":1840766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.103,"15min":1.125},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.671,"5min":1.671,"15min":1.9930000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1559999999999999,"15min":1.2},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9550000000000001,"5min":1.994,"15min":2.2909999999999999},"last":1.2070000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.482},{"interface":"front","average":{"1min":1.137,"5min":1.129,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.738,"5min":1.75,"15min":2.4529999999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.282,"15min":1.2789999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1990000000000001,"15min":1.2290000000000001},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.595,"5min":2.266,"15min":2.266},"last":1.518}]},{"osd":3,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3120000000000001,"15min":1.333},"min":{"1min":0.79000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9079999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.881},"last":1.573}]},{"osd":4,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2909999999999999,"15min":1.2609999999999999},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9450000000000001,"5min":1.976,"15min":1.976},"last":1.46},{"interface":"front","average":{"1min":1.361,"5min":1.448,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0489999999999999,"5min":2.093,"15min":2.093},"last":1.923}]},{"osd":6,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4670000000000001,"15min":1.474},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7909999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5249999999999999,"15min":1.5209999999999999},"min":{"1min":0.91900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.875,"5min":2.1579999999999999,"15min":2.2269999999999999},"last":1.3740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.55,"15min":1.538},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.58499999999999996},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.331},{"interface":"front","average":{"1min":1.488,"5min":1.552,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0070000000000001,"5min":2.0569999999999999,"15min":2.1299999999999999},"last":1.413}]}]},{"osd":4,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52688,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319152,"statfs":{"total":96636764160,"available":96582811648,"internally_reserved":0,"allocated":3543040,"data_stored":1854192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.1379999999999999},"min":{"1min":0.68500000000000005,"5min":0.59799999999999998,"15min":0.48999999999999999},"max":{"1min":1.5149999999999999,"5min":1.746,"15min":1.8480000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.147,"15min":1.202},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.869},"last":1.264}]},{"osd":1,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7250000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.179,"15min":1.2390000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6000000000000001,"5min":1.762,"15min":1.762},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2010000000000001,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.534,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.349,"15min":1.3759999999999999},"min":{"1min":0.90200000000000002,"5min":0.88700000000000001,"15min":0.82199999999999995},"max":{"1min":1.78,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.4430000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2350000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3700000000000001,"15min":1.419},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.60999999999999999},"max":{"1min":1.728,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.194}]},{"osd":5,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.359,"15min":1.3500000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.982,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":0.85799999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.97699999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.96,"5min":2.145,"15min":2.1600000000000001},"last":1.413},{"interface":"front","average":{"1min":1.528,"5min":1.5640000000000001,"15min":1.5960000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.016,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.5549999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4970000000000001,"15min":1.5189999999999999},"min":{"1min":0.998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.137},{"interface":"front","average":{"1min":1.554,"5min":1.5669999999999999,"15min":1.5900000000000001},"min":{"1min":0.95199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.052,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.534}]}]},{"osd":3,"up_from":13,"seq":55834576026,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57148,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314692,"statfs":{"total":96636764160,"available":96578244608,"internally_reserved":0,"allocated":3850240,"data_stored":2169034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1919999999999999,"5min":1.6439999999999999,"15min":1.71},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.1559999999999999,"15min":1.2250000000000001},"min":{"1min":0.54300000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.7709999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.498}]},{"osd":1,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":1.038},"min":{"1min":0.51400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4239999999999999,"5min":1.835,"15min":1.835},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.181,"15min":1.2789999999999999},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.456,"5min":1.911,"15min":1.911},"last":1.6659999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.0880000000000001},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.921,"15min":2.1850000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1439999999999999,"15min":1.2509999999999999},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.605,"5min":1.9870000000000001,"15min":2},"last":1.7749999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3180000000000001,"15min":1.4099999999999999},"min":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.502,"5min":2.262,"15min":2.262},"last":1.867},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3740000000000001,"15min":1.4379999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.9430000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.927}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.369,"15min":1.464},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6579999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":2.089},{"interface":"front","average":{"1min":1.421,"5min":1.502,"15min":1.546},"min":{"1min":0.871,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8069999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":2.1509999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4830000000000001,"15min":1.5629999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7789999999999999,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":2.488},{"interface":"front","average":{"1min":1.304,"5min":1.4970000000000001,"15min":1.585},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":2.2280000000000002}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5669999999999999,"15min":1.6419999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.8520000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":2.4100000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.5649999999999999,"15min":1.6679999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.766,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":2.3010000000000002}]}]},{"osd":2,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3510272,"data_stored":1840710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.90800000000000003},"min":{"1min":0.61899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4470000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.95299999999999996,"15min":1.022},"min":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.353,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":0.78100000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.64900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.79,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7010000000000001,"5min":1.881,"15min":1.958},"last":1.0900000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.155,"15min":1.1739999999999999},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.423},{"interface":"front","average":{"1min":1.321,"5min":1.4319999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.357}]},{"osd":4,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.45,"15min":1.425},"min":{"1min":0.97399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.867,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.139},{"interface":"front","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.0780000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8620000000000001,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.4470000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3520000000000001,"15min":1.401},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5920000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.22},{"interface":"front","average":{"1min":1.544,"5min":1.607,"15min":1.6080000000000001},"min":{"1min":1.1319999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9470000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.472}]},{"osd":6,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.512,"15min":1.5509999999999999},"min":{"1min":1.171,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.292},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.974,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.617,"15min":1.6279999999999999},"min":{"1min":1.224,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0750000000000002,"5min":2.1240000000000001,"15min":2.4630000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":1.2190000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576026,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52660,"kb_used_data":3432,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319180,"statfs":{"total":96636764160,"available":96582840320,"internally_reserved":0,"allocated":3514368,"data_stored":1840800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.976},"last":1.105},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.115},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":1.756,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":1.056}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":1.0169999999999999},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2370000000000001,"5min":1.746,"15min":1.79},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.135,"15min":1.2010000000000001},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.26,"15min":1.335},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8240000000000001,"5min":1.9239999999999999,"15min":2.129},"last":1.345},{"interface":"front","average":{"1min":1.119,"5min":1.208,"15min":1.252},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.437},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.206,"15min":1.3140000000000001},"min":{"1min":0.61899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.423,"15min":1.468},"min":{"1min":0.90900000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0539999999999998,"5min":2.1070000000000002,"15min":2.383},"last":1.788}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.522},"min":{"1min":0.92000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.3969999999999998},"last":1.542},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4079999999999999,"15min":1.492},"min":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8660000000000001,"5min":2.1000000000000001,"15min":2.3359999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4650000000000001,"15min":1.5509999999999999},"min":{"1min":0.876,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":2.2069999999999999,"15min":2.4319999999999999},"last":1.391},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.385,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.794,"5min":2.1360000000000001,"15min":2.5310000000000001},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.5549999999999999,"15min":1.621},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1259999999999999,"5min":2.1739999999999999,"15min":2.548},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.583,"15min":1.663},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9550000000000001,"5min":2.3730000000000002,"15min":2.492},"last":1.2909999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53564,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318276,"statfs":{"total":96636764160,"available":96581914624,"internally_reserved":0,"allocated":4440064,"data_stored":2759374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.323},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":1.2190000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6259999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":1.274}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.1579999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.806}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.522}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.113},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.9430000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.681},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4730000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:41:15.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:41:15.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.853+0000 7fd66cd8a640 1 Processor -- start 2023-10-21T17:41:15.855 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.853+0000 7fd66cd8a640 1 -- start start 2023-10-21T17:41:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd66cd8a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6680c90b0 0x7fd6680cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd66cd8a640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd6680cb3b0 0x7fd66815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.857 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd66cd8a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66815b2b0 con 0x7fd66815ab40 2023-10-21T17:41:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd66cd8a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66815bf30 con 0x7fd6680c90b0 2023-10-21T17:41:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd66cd8a640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680cb820 con 0x7fd6680cb3b0 2023-10-21T17:41:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6680c90b0 0x7fd6680cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.858 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6680c90b0 0x7fd6680cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54366/0 (socket says 172.21.15.53:54366) 2023-10-21T17:41:15.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 -- 172.21.15.53:0/714538884 learned_addr learned my addr 172.21.15.53:0/714538884 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 -- 172.21.15.53:0/714538884 >> v2:172.21.15.124:3300/0 conn(0x7fd6680cb3b0 msgr2=0x7fd66815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 --2- 172.21.15.53:0/714538884 >> v2:172.21.15.124:3300/0 conn(0x7fd6680cb3b0 0x7fd66815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.860 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 -- 172.21.15.53:0/714538884 >> v1:172.21.15.124:6789/0 conn(0x7fd66815ab40 legacy=0x7fd6680ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.855+0000 7fd665d9b640 1 -- 172.21.15.53:0/714538884 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66815b590 con 0x7fd6680c90b0 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.856+0000 7fd665d9b640 1 --2- 172.21.15.53:0/714538884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680c90b0 0x7fd6680cae70 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b0ec0 tx=0x7fd65c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14d73dd02b0eba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.857+0000 7fd66559a640 1 -- 172.21.15.53:0/714538884 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c031870 con 0x7fd6680c90b0 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.857+0000 7fd66559a640 1 -- 172.21.15.53:0/714538884 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c0319d0 con 0x7fd6680c90b0 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.857+0000 7fd66559a640 1 -- 172.21.15.53:0/714538884 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c031d60 con 0x7fd6680c90b0 2023-10-21T17:41:15.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 -- 172.21.15.53:0/714538884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680c90b0 msgr2=0x7fd6680cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/714538884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680c90b0 0x7fd6680cae70 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7fd6680b0ec0 tx=0x7fd65c02b910 comp rx=0 tx=0).stop 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 -- 172.21.15.53:0/714538884 shutdown_connections 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/714538884 >> v2:172.21.15.124:3300/0 conn(0x7fd6680cb3b0 0x7fd66815e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/714538884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680c90b0 0x7fd6680cae70 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 -- 172.21.15.53:0/714538884 >> 172.21.15.53:0/714538884 conn(0x7fd66806a380 msgr2=0x7fd6680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:15.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 -- 172.21.15.53:0/714538884 shutdown_connections 2023-10-21T17:41:15.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.858+0000 7fd66cd8a640 1 -- 172.21.15.53:0/714538884 wait complete. 2023-10-21T17:41:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.860+0000 7fd66cd8a640 1 Processor -- start 2023-10-21T17:41:15.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.860+0000 7fd66cd8a640 1 -- start start 2023-10-21T17:41:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6680c90b0 0x7fd6680c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd66815ab40 0x7fd668159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680c3ef0 con 0x7fd6680cb3b0 2023-10-21T17:41:15.865 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668153b10 con 0x7fd66815ab40 2023-10-21T17:41:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd66cd8a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668158940 con 0x7fd6680c90b0 2023-10-21T17:41:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54370/0 (socket says 172.21.15.53:54370) 2023-10-21T17:41:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 -- 172.21.15.53:0/1954412312 learned_addr learned my addr 172.21.15.53:0/1954412312 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:15.866 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd666d9d640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd66815ab40 0x7fd668159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 -- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6680c90b0 msgr2=0x7fd6680c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6680c90b0 0x7fd6680c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 -- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd66815ab40 msgr2=0x7fd668159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd66815ab40 0x7fd668159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:15.867 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.861+0000 7fd665d9b640 1 -- 172.21.15.53:0/1954412312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0094d0 con 0x7fd6680cb3b0 2023-10-21T17:41:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd665d9b640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c03ad10 con 0x7fd6680cb3b0 2023-10-21T17:41:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c03ae70 con 0x7fd6680cb3b0 2023-10-21T17:41:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c044610 con 0x7fd6680cb3b0 2023-10-21T17:41:15.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680c5d90 con 0x7fd6680cb3b0 2023-10-21T17:41:15.869 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.862+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680c3830 con 0x7fd6680cb3b0 2023-10-21T17:41:15.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.863+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6680cd9d0 con 0x7fd6680cb3b0 2023-10-21T17:41:15.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.864+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd65c044770 con 0x7fd6680cb3b0 2023-10-21T17:41:15.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.865+0000 7fd656ffd640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 0x7fd64806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:15.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.865+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fd65c02d040 con 0x7fd6680cb3b0 2023-10-21T17:41:15.870 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.866+0000 7fd66659c640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 0x7fd64806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:15.871 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.867+0000 7fd66659c640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 0x7fd64806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd658009650 tx=0x7fd658006ba0 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:15.872 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:15.870+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd65c077110 con 0x7fd6680cb3b0 2023-10-21T17:41:16.061 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.059+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd66815bb50 con 0x7fd64806af90 2023-10-21T17:41:16.068 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.066+0000 7fd656ffd640 1 -- 172.21.15.53:0/1954412312 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48418 (secure 0 0 0) 0x7fd65800b3e0 con 0x7fd64806af90 2023-10-21T17:41:16.069 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:16.069 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:41:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.068+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 msgr2=0x7fd64806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.068+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 0x7fd64806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd658009650 tx=0x7fd658006ba0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 msgr2=0x7fd6681491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.071 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c0046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 shutdown_connections 2023-10-21T17:41:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd64806af90 0x7fd64806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd66815ab40 0x7fd668159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6680cb3b0 0x7fd6681491a0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.072 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 --2- 172.21.15.53:0/1954412312 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd6680c90b0 0x7fd6680c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 >> 172.21.15.53:0/1954412312 conn(0x7fd66806a380 msgr2=0x7fd66806d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 shutdown_connections 2023-10-21T17:41:16.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.069+0000 7fd66cd8a640 1 -- 172.21.15.53:0/1954412312 wait complete. 2023-10-21T17:41:16.090 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-21T17:41:14.602217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462640,"kb_used_data":29968,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754512080,"statfs":{"total":773094113280,"available":772620369920,"internally_reserved":0,"allocated":30687232,"data_stored":17246620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668451,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.009917"},"pg_stats":[{"pgid":"1.0","version":"277'4931","reported_seq":6047,"reported_epoch":277,"state":"active+clean","last_fresh":"2023-10-21T17:41:06.721440+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:41:06.721440+0000","last_peered":"2023-10-21T17:41:06.721440+0000","last_clean":"2023-10-21T17:41:06.721440+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:41:06.721440+0000","last_undegraded":"2023-10-21T17:41:06.721440+0000","last_fullsized":"2023-10-21T17:41:06.721440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576026,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75800,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296040,"statfs":{"total":96636764160,"available":96559144960,"internally_reserved":0,"allocated":4468736,"data_stored":2773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.76700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.724,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.282},{"interface":"front","average":{"1min":1.165,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.77100000000000002,"5min":0.64300000000000002,"15min":0.63800000000000001},"max":{"1min":1.5069999999999999,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0329999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.175,"15min":1.1910000000000001},"min":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.57099999999999995},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.78600000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.909,"15min":1.909},"last":1.4970000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3540000000000001,"15min":1.3320000000000001},"min":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.65700000000000003},"max":{"1min":1.849,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.577},{"interface":"front","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.667,"5min":3.556,"15min":3.556},"last":1.2050000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.306},"min":{"1min":0.623,"5min":0.623,"15min":0.50700000000000001},"max":{"1min":2.3439999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.387},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.486,"15min":1.4610000000000001},"min":{"1min":1.087,"5min":0.69999999999999996,"15min":0.58899999999999997},"max":{"1min":1.859,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.643}]},{"osd":4,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3859999999999999,"15min":1.361},"min":{"1min":0.64100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7350000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.90000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3799999999999999,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5,"15min":1.431},"min":{"1min":0.74399999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9770000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.514,"15min":1.464},"min":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0379999999999998,"5min":2.613,"15min":2.613},"last":1.4430000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.613,"15min":1.5640000000000001},"min":{"1min":1.181,"5min":0.625,"15min":0.625},"max":{"1min":1.964,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.696},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5600000000000001,"15min":1.4990000000000001},"min":{"1min":0.99299999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2410000000000001,"5min":2.879,"15min":2.879},"last":1.54}]}]},{"osd":6,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65460,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306380,"statfs":{"total":96636764160,"available":96569733120,"internally_reserved":0,"allocated":3842048,"data_stored":2168491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.119,"15min":1.1679999999999999},"min":{"1min":0.70499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6319999999999999,"5min":1.944,"15min":1.944},"last":1.173},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.147,"15min":1.1890000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5249999999999999,"5min":1.6619999999999999,"15min":1.762},"last":1.028}]},{"osd":1,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.121,"15min":1.1579999999999999},"min":{"1min":0.73199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4990000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.198,"15min":1.278},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.5620000000000001,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2450000000000001,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.266,"15min":1.288},"min":{"1min":0.80700000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6519999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.242}]},{"osd":3,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3009999999999999,"15min":1.3839999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.4790000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3009999999999999,"15min":1.3360000000000001},"min":{"1min":0.47699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.9910000000000001,"5min":2.419,"15min":2.419},"last":1.288},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4299999999999999},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0299999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.552,"15min":1.556},"min":{"1min":0.77400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9450000000000001,"5min":2.1299999999999999,"15min":2.1739999999999999},"last":1.419},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3560000000000001,"15min":1.361},"min":{"1min":0.56100000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.9219999999999999,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.0740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4690000000000001,"15min":1.5169999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0550000000000002,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.119},{"interface":"front","average":{"1min":1.663,"5min":1.639,"15min":1.651},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0219999999999998,"5min":2.21,"15min":2.2610000000000001},"last":1.6359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576028,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48504,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323336,"statfs":{"total":96636764160,"available":96587096064,"internally_reserved":0,"allocated":3518464,"data_stored":1840766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.103,"15min":1.125},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.671,"5min":1.671,"15min":1.9930000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1559999999999999,"15min":1.2},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9550000000000001,"5min":1.994,"15min":2.2909999999999999},"last":1.2070000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.482},{"interface":"front","average":{"1min":1.137,"5min":1.129,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.738,"5min":1.75,"15min":2.4529999999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.282,"15min":1.2789999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1990000000000001,"15min":1.2290000000000001},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.595,"5min":2.266,"15min":2.266},"last":1.518}]},{"osd":3,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3120000000000001,"15min":1.333},"min":{"1min":0.79000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9079999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.881},"last":1.573}]},{"osd":4,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2909999999999999,"15min":1.2609999999999999},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9450000000000001,"5min":1.976,"15min":1.976},"last":1.46},{"interface":"front","average":{"1min":1.361,"5min":1.448,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0489999999999999,"5min":2.093,"15min":2.093},"last":1.923}]},{"osd":6,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4670000000000001,"15min":1.474},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7909999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5249999999999999,"15min":1.5209999999999999},"min":{"1min":0.91900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.875,"5min":2.1579999999999999,"15min":2.2269999999999999},"last":1.3740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.55,"15min":1.538},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.58499999999999996},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.331},{"interface":"front","average":{"1min":1.488,"5min":1.552,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0070000000000001,"5min":2.0569999999999999,"15min":2.1299999999999999},"last":1.413}]}]},{"osd":4,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52688,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319152,"statfs":{"total":96636764160,"available":96582811648,"internally_reserved":0,"allocated":3543040,"data_stored":1854192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.1379999999999999},"min":{"1min":0.68500000000000005,"5min":0.59799999999999998,"15min":0.48999999999999999},"max":{"1min":1.5149999999999999,"5min":1.746,"15min":1.8480000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.147,"15min":1.202},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.869},"last":1.264}]},{"osd":1,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7250000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.179,"15min":1.2390000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6000000000000001,"5min":1.762,"15min":1.762},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2010000000000001,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.534,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.349,"15min":1.3759999999999999},"min":{"1min":0.90200000000000002,"5min":0.88700000000000001,"15min":0.82199999999999995},"max":{"1min":1.78,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.4430000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2350000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3700000000000001,"15min":1.419},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.60999999999999999},"max":{"1min":1.728,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.194}]},{"osd":5,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.359,"15min":1.3500000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.982,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":0.85799999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.97699999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.96,"5min":2.145,"15min":2.1600000000000001},"last":1.413},{"interface":"front","average":{"1min":1.528,"5min":1.5640000000000001,"15min":1.5960000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.016,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.5549999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4970000000000001,"15min":1.5189999999999999},"min":{"1min":0.998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.137},{"interface":"front","average":{"1min":1.554,"5min":1.5669999999999999,"15min":1.5900000000000001},"min":{"1min":0.95199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.052,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.534}]}]},{"osd":3,"up_from":13,"seq":55834576026,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57148,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314692,"statfs":{"total":96636764160,"available":96578244608,"internally_reserved":0,"allocated":3850240,"data_stored":2169034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1919999999999999,"5min":1.6439999999999999,"15min":1.71},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.1559999999999999,"15min":1.2250000000000001},"min":{"1min":0.54300000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.7709999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.498}]},{"osd":1,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":1.038},"min":{"1min":0.51400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4239999999999999,"5min":1.835,"15min":1.835},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.181,"15min":1.2789999999999999},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.456,"5min":1.911,"15min":1.911},"last":1.6659999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.0880000000000001},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.921,"15min":2.1850000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1439999999999999,"15min":1.2509999999999999},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.605,"5min":1.9870000000000001,"15min":2},"last":1.7749999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3180000000000001,"15min":1.4099999999999999},"min":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.502,"5min":2.262,"15min":2.262},"last":1.867},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3740000000000001,"15min":1.4379999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.9430000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.927}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.369,"15min":1.464},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6579999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":2.089},{"interface":"front","average":{"1min":1.421,"5min":1.502,"15min":1.546},"min":{"1min":0.871,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8069999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":2.1509999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4830000000000001,"15min":1.5629999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7789999999999999,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":2.488},{"interface":"front","average":{"1min":1.304,"5min":1.4970000000000001,"15min":1.585},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":2.2280000000000002}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5669999999999999,"15min":1.6419999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.8520000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":2.4100000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.5649999999999999,"15min":1.6679999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.766,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":2.3010000000000002}]}]},{"osd":2,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3510272,"data_stored":1840710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.90800000000000003},"min":{"1min":0.61899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4470000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.95299999999999996,"15min":1.022},"min":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.353,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":0.78100000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.64900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.79,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7010000000000001,"5min":1.881,"15min":1.958},"last":1.0900000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.155,"15min":1.1739999999999999},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.423},{"interface":"front","average":{"1min":1.321,"5min":1.4319999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.357}]},{"osd":4,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.45,"15min":1.425},"min":{"1min":0.97399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.867,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.139},{"interface":"front","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.0780000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8620000000000001,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.4470000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3520000000000001,"15min":1.401},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5920000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.22},{"interface":"front","average":{"1min":1.544,"5min":1.607,"15min":1.6080000000000001},"min":{"1min":1.1319999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9470000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.472}]},{"osd":6,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.512,"15min":1.5509999999999999},"min":{"1min":1.171,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.292},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.974,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.617,"15min":1.6279999999999999},"min":{"1min":1.224,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0750000000000002,"5min":2.1240000000000001,"15min":2.4630000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":1.2190000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576026,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52660,"kb_used_data":3432,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319180,"statfs":{"total":96636764160,"available":96582840320,"internally_reserved":0,"allocated":3514368,"data_stored":1840800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.976},"last":1.105},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.115},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":1.756,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":1.056}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":1.0169999999999999},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2370000000000001,"5min":1.746,"15min":1.79},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.135,"15min":1.2010000000000001},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.26,"15min":1.335},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8240000000000001,"5min":1.9239999999999999,"15min":2.129},"last":1.345},{"interface":"front","average":{"1min":1.119,"5min":1.208,"15min":1.252},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.437},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.206,"15min":1.3140000000000001},"min":{"1min":0.61899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.423,"15min":1.468},"min":{"1min":0.90900000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0539999999999998,"5min":2.1070000000000002,"15min":2.383},"last":1.788}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.522},"min":{"1min":0.92000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.3969999999999998},"last":1.542},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4079999999999999,"15min":1.492},"min":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8660000000000001,"5min":2.1000000000000001,"15min":2.3359999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4650000000000001,"15min":1.5509999999999999},"min":{"1min":0.876,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":2.2069999999999999,"15min":2.4319999999999999},"last":1.391},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.385,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.794,"5min":2.1360000000000001,"15min":2.5310000000000001},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.5549999999999999,"15min":1.621},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1259999999999999,"5min":2.1739999999999999,"15min":2.548},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.583,"15min":1.663},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9550000000000001,"5min":2.3730000000000002,"15min":2.492},"last":1.2909999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53564,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318276,"statfs":{"total":96636764160,"available":96581914624,"internally_reserved":0,"allocated":4440064,"data_stored":2759374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.323},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":1.2190000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6259999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":1.274}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.1579999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.806}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.522}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.113},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.9430000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.681},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4730000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:41:16.092 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-21T17:41:16.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:41:16.318 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.316+0000 7f5afd1cc640 1 Processor -- start 2023-10-21T17:41:16.319 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.316+0000 7f5afd1cc640 1 -- start start 2023-10-21T17:41:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af815ab40 0x7f5af80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5af80c90b0 0x7f5af80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5af80cae70 0x7f5af80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af815bf30 con 0x7f5af80c90b0 2023-10-21T17:41:16.321 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80cb840 con 0x7f5af80cae70 2023-10-21T17:41:16.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5afd1cc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815b2b0 con 0x7f5af815ab40 2023-10-21T17:41:16.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5aeffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5af80c90b0 0x7f5af80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5aeffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5af80c90b0 0x7f5af80adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54380/0 (socket says 172.21.15.53:54380) 2023-10-21T17:41:16.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.318+0000 7f5aeffff640 1 -- 172.21.15.53:0/1954544495 learned_addr learned my addr 172.21.15.53:0/1954544495 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:16.322 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5af6d9d640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af815ab40 0x7f5af80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.323 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5af759e640 1 --2- 172.21.15.53:0/1954544495 >> v2:172.21.15.124:3300/0 conn(0x7f5af80cae70 0x7f5af80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5aeffff640 1 -- 172.21.15.53:0/1954544495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af815ab40 msgr2=0x7f5af80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:16.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5aeffff640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af815ab40 0x7f5af80ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.324 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5aeffff640 1 -- 172.21.15.53:0/1954544495 >> v2:172.21.15.124:3300/0 conn(0x7f5af80cae70 msgr2=0x7f5af80cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:16.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5aeffff640 1 --2- 172.21.15.53:0/1954544495 >> v2:172.21.15.124:3300/0 conn(0x7f5af80cae70 0x7f5af80cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.319+0000 7f5aeffff640 1 -- 172.21.15.53:0/1954544495 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af815b590 con 0x7f5af80c90b0 2023-10-21T17:41:16.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.320+0000 7f5aeffff640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af80c90b0 0x7f5af80adc60 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f5af80c8d50 tx=0x7f5ae002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad244e7fa0ff00 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.321+0000 7f5af659c640 1 -- 172.21.15.53:0/1954544495 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0031870 con 0x7f5af80c90b0 2023-10-21T17:41:16.325 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.321+0000 7f5af659c640 1 -- 172.21.15.53:0/1954544495 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae00319d0 con 0x7f5af80c90b0 2023-10-21T17:41:16.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.321+0000 7f5af659c640 1 -- 172.21.15.53:0/1954544495 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0031d60 con 0x7f5af80c90b0 2023-10-21T17:41:16.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1954544495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af80c90b0 msgr2=0x7f5af80adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af80c90b0 0x7f5af80adc60 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7f5af80c8d50 tx=0x7f5ae002b910 comp rx=0 tx=0).stop 2023-10-21T17:41:16.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1954544495 shutdown_connections 2023-10-21T17:41:16.326 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1954544495 >> v2:172.21.15.124:3300/0 conn(0x7f5af80cae70 0x7f5af80cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af80c90b0 0x7f5af80adc60 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1954544495 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af815ab40 0x7f5af80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1954544495 >> 172.21.15.53:0/1954544495 conn(0x7f5af806a380 msgr2=0x7f5af80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:16.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1954544495 shutdown_connections 2023-10-21T17:41:16.327 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.322+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1954544495 wait complete. 2023-10-21T17:41:16.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.324+0000 7f5afd1cc640 1 Processor -- start 2023-10-21T17:41:16.329 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.324+0000 7f5afd1cc640 1 -- start start 2023-10-21T17:41:16.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af80c90b0 0x7f5af80ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af80cae70 0x7f5af8153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80ad620 con 0x7f5af815ab40 2023-10-21T17:41:16.330 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80ada40 con 0x7f5af80cae70 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5afd1cc640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80c84d0 con 0x7f5af80c90b0 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54386/0 (socket says 172.21.15.53:54386) 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 -- 172.21.15.53:0/1287988226 learned_addr learned my addr 172.21.15.53:0/1287988226 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af6d9d640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af80c90b0 0x7f5af80ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.331 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 -- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af80c90b0 msgr2=0x7f5af80ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af80c90b0 0x7f5af80ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5aeffff640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af80cae70 0x7f5af8153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.325+0000 7f5af759e640 1 -- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af80cae70 msgr2=0x7f5af8153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5af759e640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af80cae70 0x7f5af8153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.332 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5af759e640 1 -- 172.21.15.53:0/1287988226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00094d0 con 0x7f5af815ab40 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5af759e640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007e90 tx=0x7f5af000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0014420 con 0x7f5af815ab40 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af00145a0 con 0x7f5af815ab40 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0014950 con 0x7f5af815ab40 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af80ac9a0 con 0x7f5af815ab40 2023-10-21T17:41:16.333 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.326+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af80acf60 con 0x7f5af815ab40 2023-10-21T17:41:16.334 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.328+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af80ad760 con 0x7f5af815ab40 2023-10-21T17:41:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.329+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5af0014ab0 con 0x7f5af815ab40 2023-10-21T17:41:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.329+0000 7f5aef7fe640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 0x7f5ad806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.330+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f5af008a170 con 0x7f5af815ab40 2023-10-21T17:41:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.330+0000 7f5af6d9d640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 0x7f5ad806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.335 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.331+0000 7f5af6d9d640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 0x7f5ad806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009b40 tx=0x7f5ae8009190 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.334+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5af80ad760 con 0x7f5af815ab40 2023-10-21T17:41:16.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.523+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5af815bb50 con 0x7f5ad806af90 2023-10-21T17:41:16.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.530+0000 7f5aef7fe640 1 -- 172.21.15.53:0/1287988226 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48418 (secure 0 0 0) 0x7f5ae8011a60 con 0x7f5ad806af90 2023-10-21T17:41:16.532 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:16.533 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:41:16.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 msgr2=0x7f5ad806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 0x7f5ad806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009b40 tx=0x7f5ae8009190 comp rx=0 tx=0).stop 2023-10-21T17:41:16.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 msgr2=0x7f5af81f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007e90 tx=0x7f5af000e4b0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 shutdown_connections 2023-10-21T17:41:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5ad806af90 0x7f5ad806d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af815ab40 0x7f5af81f5d00 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5af80cae70 0x7f5af8153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 --2- 172.21.15.53:0/1287988226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5af80c90b0 0x7f5af80ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.532+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 >> 172.21.15.53:0/1287988226 conn(0x7f5af806a380 msgr2=0x7f5af80ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:16.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.533+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 shutdown_connections 2023-10-21T17:41:16.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.533+0000 7f5afd1cc640 1 -- 172.21.15.53:0/1287988226 wait complete. 2023-10-21T17:41:16.554 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-21T17:41:14.602217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462640,"kb_used_data":29968,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754512080,"statfs":{"total":773094113280,"available":772620369920,"internally_reserved":0,"allocated":30687232,"data_stored":17246620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668451,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.009917"},"pg_stats":[{"pgid":"1.0","version":"277'4931","reported_seq":6047,"reported_epoch":277,"state":"active+clean","last_fresh":"2023-10-21T17:41:06.721440+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:41:06.721440+0000","last_peered":"2023-10-21T17:41:06.721440+0000","last_clean":"2023-10-21T17:41:06.721440+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:41:06.721440+0000","last_undegraded":"2023-10-21T17:41:06.721440+0000","last_fullsized":"2023-10-21T17:41:06.721440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576026,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75800,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296040,"statfs":{"total":96636764160,"available":96559144960,"internally_reserved":0,"allocated":4468736,"data_stored":2773253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.76700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.724,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":1.282},{"interface":"front","average":{"1min":1.165,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.77100000000000002,"5min":0.64300000000000002,"15min":0.63800000000000001},"max":{"1min":1.5069999999999999,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0329999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.175,"15min":1.1910000000000001},"min":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.57099999999999995},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.78600000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.909,"15min":1.909},"last":1.4970000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3540000000000001,"15min":1.3320000000000001},"min":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.65700000000000003},"max":{"1min":1.849,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.577},{"interface":"front","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.667,"5min":3.556,"15min":3.556},"last":1.2050000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.306},"min":{"1min":0.623,"5min":0.623,"15min":0.50700000000000001},"max":{"1min":2.3439999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.387},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.486,"15min":1.4610000000000001},"min":{"1min":1.087,"5min":0.69999999999999996,"15min":0.58899999999999997},"max":{"1min":1.859,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.643}]},{"osd":4,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3859999999999999,"15min":1.361},"min":{"1min":0.64100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7350000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.90000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3799999999999999,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5,"15min":1.431},"min":{"1min":0.74399999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9770000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.514,"15min":1.464},"min":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0379999999999998,"5min":2.613,"15min":2.613},"last":1.4430000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.613,"15min":1.5640000000000001},"min":{"1min":1.181,"5min":0.625,"15min":0.625},"max":{"1min":1.964,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.696},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5600000000000001,"15min":1.4990000000000001},"min":{"1min":0.99299999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2410000000000001,"5min":2.879,"15min":2.879},"last":1.54}]}]},{"osd":6,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65460,"kb_used_data":3752,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306380,"statfs":{"total":96636764160,"available":96569733120,"internally_reserved":0,"allocated":3842048,"data_stored":2168491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.119,"15min":1.1679999999999999},"min":{"1min":0.70499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6319999999999999,"5min":1.944,"15min":1.944},"last":1.173},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.147,"15min":1.1890000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5249999999999999,"5min":1.6619999999999999,"15min":1.762},"last":1.028}]},{"osd":1,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.121,"15min":1.1579999999999999},"min":{"1min":0.73199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4990000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.198,"15min":1.278},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.5620000000000001,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2450000000000001,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.266,"15min":1.288},"min":{"1min":0.80700000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6519999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.242}]},{"osd":3,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3009999999999999,"15min":1.3839999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.4790000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3009999999999999,"15min":1.3360000000000001},"min":{"1min":0.47699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.9910000000000001,"5min":2.419,"15min":2.419},"last":1.288},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4299999999999999},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0299999999999998},"last":1.5800000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.552,"15min":1.556},"min":{"1min":0.77400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9450000000000001,"5min":2.1299999999999999,"15min":2.1739999999999999},"last":1.419},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3560000000000001,"15min":1.361},"min":{"1min":0.56100000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.9219999999999999,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.0740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4690000000000001,"15min":1.5169999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0550000000000002,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.119},{"interface":"front","average":{"1min":1.663,"5min":1.639,"15min":1.651},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0219999999999998,"5min":2.21,"15min":2.2610000000000001},"last":1.6359999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576028,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48504,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323336,"statfs":{"total":96636764160,"available":96587096064,"internally_reserved":0,"allocated":3518464,"data_stored":1840766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.103,"15min":1.125},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.671,"5min":1.671,"15min":1.9930000000000001},"last":1.153},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1559999999999999,"15min":1.2},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9550000000000001,"5min":1.994,"15min":2.2909999999999999},"last":1.2070000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.482},{"interface":"front","average":{"1min":1.137,"5min":1.129,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.738,"5min":1.75,"15min":2.4529999999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.282,"15min":1.2789999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1990000000000001,"15min":1.2290000000000001},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.595,"5min":2.266,"15min":2.266},"last":1.518}]},{"osd":3,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3120000000000001,"15min":1.333},"min":{"1min":0.79000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9079999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.881},"last":1.573}]},{"osd":4,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2909999999999999,"15min":1.2609999999999999},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9450000000000001,"5min":1.976,"15min":1.976},"last":1.46},{"interface":"front","average":{"1min":1.361,"5min":1.448,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0489999999999999,"5min":2.093,"15min":2.093},"last":1.923}]},{"osd":6,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4670000000000001,"15min":1.474},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7909999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5249999999999999,"15min":1.5209999999999999},"min":{"1min":0.91900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.875,"5min":2.1579999999999999,"15min":2.2269999999999999},"last":1.3740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.55,"15min":1.538},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.58499999999999996},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.331},{"interface":"front","average":{"1min":1.488,"5min":1.552,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0070000000000001,"5min":2.0569999999999999,"15min":2.1299999999999999},"last":1.413}]}]},{"osd":4,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52688,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319152,"statfs":{"total":96636764160,"available":96582811648,"internally_reserved":0,"allocated":3543040,"data_stored":1854192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.1379999999999999},"min":{"1min":0.68500000000000005,"5min":0.59799999999999998,"15min":0.48999999999999999},"max":{"1min":1.5149999999999999,"5min":1.746,"15min":1.8480000000000001},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.087,"5min":1.147,"15min":1.202},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.869},"last":1.264}]},{"osd":1,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7250000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.179,"15min":1.2390000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6000000000000001,"5min":1.762,"15min":1.762},"last":0.79800000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2010000000000001,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.534,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.349,"15min":1.3759999999999999},"min":{"1min":0.90200000000000002,"5min":0.88700000000000001,"15min":0.82199999999999995},"max":{"1min":1.78,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.4430000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2350000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3700000000000001,"15min":1.419},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.60999999999999999},"max":{"1min":1.728,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.194}]},{"osd":5,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.359,"15min":1.3500000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.982,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":0.85799999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.97699999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.96,"5min":2.145,"15min":2.1600000000000001},"last":1.413},{"interface":"front","average":{"1min":1.528,"5min":1.5640000000000001,"15min":1.5960000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.016,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.5549999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4970000000000001,"15min":1.5189999999999999},"min":{"1min":0.998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.137},{"interface":"front","average":{"1min":1.554,"5min":1.5669999999999999,"15min":1.5900000000000001},"min":{"1min":0.95199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.052,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.534}]}]},{"osd":3,"up_from":13,"seq":55834576026,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57148,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314692,"statfs":{"total":96636764160,"available":96578244608,"internally_reserved":0,"allocated":3850240,"data_stored":2169034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1919999999999999,"5min":1.6439999999999999,"15min":1.71},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.1559999999999999,"15min":1.2250000000000001},"min":{"1min":0.54300000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.7709999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.498}]},{"osd":1,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":1.038},"min":{"1min":0.51400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4239999999999999,"5min":1.835,"15min":1.835},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.181,"15min":1.2789999999999999},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.456,"5min":1.911,"15min":1.911},"last":1.6659999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.0880000000000001},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.921,"15min":2.1850000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1439999999999999,"15min":1.2509999999999999},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.605,"5min":1.9870000000000001,"15min":2},"last":1.7749999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3180000000000001,"15min":1.4099999999999999},"min":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.502,"5min":2.262,"15min":2.262},"last":1.867},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3740000000000001,"15min":1.4379999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.9430000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.927}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.369,"15min":1.464},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6579999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":2.089},{"interface":"front","average":{"1min":1.421,"5min":1.502,"15min":1.546},"min":{"1min":0.871,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8069999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":2.1509999999999998}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4830000000000001,"15min":1.5629999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7789999999999999,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":2.488},{"interface":"front","average":{"1min":1.304,"5min":1.4970000000000001,"15min":1.585},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":2.2280000000000002}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5669999999999999,"15min":1.6419999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.8520000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":2.4100000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.5649999999999999,"15min":1.6679999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.766,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":2.3010000000000002}]}]},{"osd":2,"up_from":13,"seq":55834576027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3510272,"data_stored":1840710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.92400000000000004,"15min":0.90800000000000003},"min":{"1min":0.61899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.4470000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.95299999999999996,"15min":1.022},"min":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.353,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":0.78100000000000003}]},{"osd":1,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.64900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.79,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2629999999999999,"15min":1.3460000000000001},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7010000000000001,"5min":1.881,"15min":1.958},"last":1.0900000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.155,"15min":1.1739999999999999},"min":{"1min":0.56299999999999994,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.5720000000000001,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.423},{"interface":"front","average":{"1min":1.321,"5min":1.4319999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.661,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.357}]},{"osd":4,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.45,"15min":1.425},"min":{"1min":0.97399999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.867,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.139},{"interface":"front","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.4450000000000001},"min":{"1min":1.0780000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.8620000000000001,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.4470000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3520000000000001,"15min":1.401},"min":{"1min":1.022,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5920000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.22},{"interface":"front","average":{"1min":1.544,"5min":1.607,"15min":1.6080000000000001},"min":{"1min":1.1319999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.9470000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.472}]},{"osd":6,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.512,"15min":1.5509999999999999},"min":{"1min":1.171,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.919,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.292},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.5149999999999999},"min":{"1min":1.028,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.974,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.0269999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.617,"15min":1.6279999999999999},"min":{"1min":1.224,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0750000000000002,"5min":2.1240000000000001,"15min":2.4630000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.609,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":1.2190000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0230000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.3300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576026,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52660,"kb_used_data":3432,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319180,"statfs":{"total":96636764160,"available":96582840320,"internally_reserved":0,"allocated":3514368,"data_stored":1840800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.976},"last":1.105},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.115},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":1.756,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":1.056}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":1.0169999999999999},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2370000000000001,"5min":1.746,"15min":1.79},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.135,"15min":1.2010000000000001},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.26,"15min":1.335},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8240000000000001,"5min":1.9239999999999999,"15min":2.129},"last":1.345},{"interface":"front","average":{"1min":1.119,"5min":1.208,"15min":1.252},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.437},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.206,"15min":1.3140000000000001},"min":{"1min":0.61899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.423,"15min":1.468},"min":{"1min":0.90900000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0539999999999998,"5min":2.1070000000000002,"15min":2.383},"last":1.788}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.522},"min":{"1min":0.92000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.3969999999999998},"last":1.542},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4079999999999999,"15min":1.492},"min":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8660000000000001,"5min":2.1000000000000001,"15min":2.3359999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4650000000000001,"15min":1.5509999999999999},"min":{"1min":0.876,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":2.2069999999999999,"15min":2.4319999999999999},"last":1.391},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.385,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.794,"5min":2.1360000000000001,"15min":2.5310000000000001},"last":1.4850000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.5549999999999999,"15min":1.621},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1259999999999999,"5min":2.1739999999999999,"15min":2.548},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.583,"15min":1.663},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9550000000000001,"5min":2.3730000000000002,"15min":2.492},"last":1.2909999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53564,"kb_used_data":4336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318276,"statfs":{"total":96636764160,"available":96581914624,"internally_reserved":0,"allocated":4440064,"data_stored":2759374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.323},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":1.2190000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.6259999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":1.274}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.1579999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.806}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.522}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.113},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.9430000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.681},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.4730000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:41:16.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-21T17:41:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.781+0000 7fb3aa9a5640 1 Processor -- start 2023-10-21T17:41:16.783 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.781+0000 7fb3aa9a5640 1 -- start start 2023-10-21T17:41:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3a415ab30 0x7fb3a415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a40adc70 0x7fb3a40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 0x7fb3a40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a415f7d0 con 0x7fb3a415ab30 2023-10-21T17:41:16.785 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a415f210 con 0x7fb3a415af10 2023-10-21T17:41:16.786 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.782+0000 7fb3aa9a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a415f4f0 con 0x7fb3a40adc70 2023-10-21T17:41:16.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3a415ab30 0x7fb3a415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.787 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3a415ab30 0x7fb3a415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54390/0 (socket says 172.21.15.53:54390) 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 -- 172.21.15.53:0/3298291671 learned_addr learned my addr 172.21.15.53:0/3298291671 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3aa1a4640 1 --2- 172.21.15.53:0/3298291671 >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 0x7fb3a40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a91a2640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a40adc70 0x7fb3a40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 -- 172.21.15.53:0/3298291671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a40adc70 msgr2=0x7fb3a40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a40adc70 0x7fb3a40b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.788 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 -- 172.21.15.53:0/3298291671 >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 msgr2=0x7fb3a40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:16.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/3298291671 >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 0x7fb3a40b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3a99a3640 1 -- 172.21.15.53:0/3298291671 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a406c550 con 0x7fb3a415ab30 2023-10-21T17:41:16.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.783+0000 7fb3aa1a4640 1 --2- 172.21.15.53:0/3298291671 >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 0x7fb3a40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:16.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.785+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415ab30 0x7fb3a415e8a0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40cb000 tx=0x7fb3a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439fcb0651160dc server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.789 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.785+0000 7fb3a89a1640 1 -- 172.21.15.53:0/3298291671 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0031860 con 0x7fb3a415ab30 2023-10-21T17:41:16.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.785+0000 7fb3a89a1640 1 -- 172.21.15.53:0/3298291671 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a00319c0 con 0x7fb3a415ab30 2023-10-21T17:41:16.790 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.785+0000 7fb3a89a1640 1 -- 172.21.15.53:0/3298291671 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0031d50 con 0x7fb3a415ab30 2023-10-21T17:41:16.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/3298291671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415ab30 msgr2=0x7fb3a415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415ab30 0x7fb3a415e8a0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40cb000 tx=0x7fb3a002b900 comp rx=0 tx=0).stop 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/3298291671 shutdown_connections 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a40adc70 0x7fb3a40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/3298291671 >> v2:172.21.15.124:3300/0 conn(0x7fb3a415af10 0x7fb3a40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/3298291671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415ab30 0x7fb3a415e8a0 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/3298291671 >> 172.21.15.53:0/3298291671 conn(0x7fb3a40cb0c0 msgr2=0x7fb3a40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:16.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/3298291671 shutdown_connections 2023-10-21T17:41:16.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.786+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/3298291671 wait complete. 2023-10-21T17:41:16.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.788+0000 7fb3aa9a5640 1 Processor -- start 2023-10-21T17:41:16.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.788+0000 7fb3aa9a5640 1 -- start start 2023-10-21T17:41:16.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 0x7fb3a414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.794 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 0x7fb3a40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40c84d0 con 0x7fb3a415af10 2023-10-21T17:41:16.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40cc880 con 0x7fb3a40adc70 2023-10-21T17:41:16.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa9a5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40ccb60 con 0x7fb3a415ab30 2023-10-21T17:41:16.795 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54400/0 (socket says 172.21.15.53:54400) 2023-10-21T17:41:16.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 -- 172.21.15.53:0/1582111865 learned_addr learned my addr 172.21.15.53:0/1582111865 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:16.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 0x7fb3a414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3a91a2640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 0x7fb3a40c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.796 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 -- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 msgr2=0x7fb3a40c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 0x7fb3a40c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 -- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 msgr2=0x7fb3a414deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.789+0000 7fb3aa1a4640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 0x7fb3a414deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3aa1a4640 1 -- 172.21.15.53:0/1582111865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00094d0 con 0x7fb3a415af10 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 0x7fb3a414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:16.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3a91a2640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 0x7fb3a40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3aa1a4640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007df0 tx=0x7fb39c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003990 con 0x7fb3a415af10 2023-10-21T17:41:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c00f040 con 0x7fb3a415af10 2023-10-21T17:41:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003d30 con 0x7fb3a415af10 2023-10-21T17:41:16.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a40cce40 con 0x7fb3a415af10 2023-10-21T17:41:16.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.790+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a40cd400 con 0x7fb3a415af10 2023-10-21T17:41:16.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.791+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a40cd620 con 0x7fb3a415af10 2023-10-21T17:41:16.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.792+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb39c004030 con 0x7fb3a415af10 2023-10-21T17:41:16.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.794+0000 7fb39a7fc640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 0x7fb38806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:16.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.794+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fb39c089d70 con 0x7fb3a415af10 2023-10-21T17:41:16.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.794+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 0x7fb38806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:16.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.795+0000 7fb3a99a3640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 0x7fb38806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0036000 tx=0x7fb3a003c040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:16.801 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.798+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb39c054a40 con 0x7fb3a415af10 2023-10-21T17:41:16.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.992+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3a404e160 con 0x7fb3a415af10 2023-10-21T17:41:16.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.995+0000 7fb39a7fc640 1 -- 172.21.15.53:0/1582111865 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v280) v1 ==== 74+0+14517 (secure 0 0 0) 0x7fb39c01a090 con 0x7fb3a415af10 2023-10-21T17:41:16.997 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:16.998 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":280,"fsid":"d7047b51-7d5c-4d1a-a8c1-6ce2e690fd56","created":"2023-10-21T16:01:35.921120+0000","modified":"2023-10-21T17:41:11.624033+0000","last_up_change":"2023-10-21T16:01:47.773219+0000","last_in_change":"2023-10-21T16:01:39.439175+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-21T16:01:49.093973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a16f7f4-3a30-462a-8bf0-41e8182c027c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6809","nonce":716916716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6811","nonce":716916716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6815","nonce":716916716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":716916716},{"type":"v1","addr":"172.21.15.53:6813","nonce":716916716}]},"public_addr":"172.21.15.53:6809/716916716","cluster_addr":"172.21.15.53:6811/716916716","heartbeat_back_addr":"172.21.15.53:6815/716916716","heartbeat_front_addr":"172.21.15.53:6813/716916716","state":["exists","up"]},{"osd":1,"uuid":"c97255a9-80ac-47bc-8525-410c20b486ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6801","nonce":752205661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6803","nonce":752205661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6807","nonce":752205661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":752205661},{"type":"v1","addr":"172.21.15.53:6805","nonce":752205661}]},"public_addr":"172.21.15.53:6801/752205661","cluster_addr":"172.21.15.53:6803/752205661","heartbeat_back_addr":"172.21.15.53:6807/752205661","heartbeat_front_addr":"172.21.15.53:6805/752205661","state":["exists","up"]},{"osd":2,"uuid":"f35949b0-800b-4e69-a1e7-aeeb07bc546a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6817","nonce":3435187098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6819","nonce":3435187098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6823","nonce":3435187098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3435187098},{"type":"v1","addr":"172.21.15.53:6821","nonce":3435187098}]},"public_addr":"172.21.15.53:6817/3435187098","cluster_addr":"172.21.15.53:6819/3435187098","heartbeat_back_addr":"172.21.15.53:6823/3435187098","heartbeat_front_addr":"172.21.15.53:6821/3435187098","state":["exists","up"]},{"osd":3,"uuid":"2a33ca07-a7e3-4e34-9c42-35cafc6374f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6825","nonce":1838632921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6827","nonce":1838632921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6831","nonce":1838632921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":1838632921},{"type":"v1","addr":"172.21.15.53:6829","nonce":1838632921}]},"public_addr":"172.21.15.53:6825/1838632921","cluster_addr":"172.21.15.53:6827/1838632921","heartbeat_back_addr":"172.21.15.53:6831/1838632921","heartbeat_front_addr":"172.21.15.53:6829/1838632921","state":["exists","up"]},{"osd":4,"uuid":"5c45510d-a729-470e-bca4-4a10297aeb06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6801","nonce":1000673860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6803","nonce":1000673860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6807","nonce":1000673860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":1000673860},{"type":"v1","addr":"172.21.15.124:6805","nonce":1000673860}]},"public_addr":"172.21.15.124:6801/1000673860","cluster_addr":"172.21.15.124:6803/1000673860","heartbeat_back_addr":"172.21.15.124:6807/1000673860","heartbeat_front_addr":"172.21.15.124:6805/1000673860","state":["exists","up"]},{"osd":5,"uuid":"80f7d069-f0f5-4f93-b281-46d047166231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6809","nonce":3990975910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6811","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6813","nonce":3990975910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6822","nonce":3990975910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6815","nonce":3990975910},{"type":"v1","addr":"172.21.15.124:6818","nonce":3990975910}]},"public_addr":"172.21.15.124:6809/3990975910","cluster_addr":"172.21.15.124:6813/3990975910","heartbeat_back_addr":"172.21.15.124:6822/3990975910","heartbeat_front_addr":"172.21.15.124:6818/3990975910","state":["exists","up"]},{"osd":6,"uuid":"ed6646ad-cdd5-41d8-b1da-6fccc6570435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6812","nonce":3032138525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6816","nonce":3032138525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6821","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6823","nonce":3032138525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6817","nonce":3032138525},{"type":"v1","addr":"172.21.15.124:6819","nonce":3032138525}]},"public_addr":"172.21.15.124:6812/3032138525","cluster_addr":"172.21.15.124:6816/3032138525","heartbeat_back_addr":"172.21.15.124:6823/3032138525","heartbeat_front_addr":"172.21.15.124:6819/3032138525","state":["exists","up"]},{"osd":7,"uuid":"ffe40f79-ab92-472f-a4b2-fbdc385ebc6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6825","nonce":1993556734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6827","nonce":1993556734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6831","nonce":1993556734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":1993556734},{"type":"v1","addr":"172.21.15.124:6829","nonce":1993556734}]},"public_addr":"172.21.15.124:6825/1993556734","cluster_addr":"172.21.15.124:6827/1993556734","heartbeat_back_addr":"172.21.15.124:6831/1993556734","heartbeat_front_addr":"172.21.15.124:6829/1993556734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.949225+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.990383+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.744093+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.819879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:45.015830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.735316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.780019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-21T16:01:44.801167+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6835/2127148028":"2023-10-22T17:41:08.582493+0000","172.21.15.124:6832/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3025084903":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2615301846":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/1018395068":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/2173042720":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3250485253":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/59627499":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/612474845":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/3975797466":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2074523274":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/2461568637":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6832/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6837/2127148028":"2023-10-22T17:41:08.582493+0000","172.21.15.124:0/2747533012":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/410485344":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/1070399472":"2023-10-22T17:41:04.558304+0000","172.21.15.124:0/3555819866":"2023-10-22T17:40:56.706577+0000","172.21.15.124:6833/596989303":"2023-10-22T17:40:56.706577+0000","172.21.15.124:0/1112463946":"2023-10-22T17:40:56.706577+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":108,"length":1},{"begin":219,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-21T17:41:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.997+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 msgr2=0x7fb38806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:17.000 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.997+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 0x7fb38806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0036000 tx=0x7fb3a003c040 comp rx=0 tx=0).stop 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.997+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 msgr2=0x7fb3a40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.997+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007df0 tx=0x7fb39c00d4a0 comp rx=0 tx=0).stop 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 shutdown_connections 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fb38806af90 0x7fb38806d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a415af10 0x7fb3a40cd920 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:17.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3a415ab30 0x7fb3a40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 --2- 172.21.15.53:0/1582111865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3a40adc70 0x7fb3a414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 >> 172.21.15.53:0/1582111865 conn(0x7fb3a40cb0c0 msgr2=0x7fb3a406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 shutdown_connections 2023-10-21T17:41:17.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:16.998+0000 7fb3aa9a5640 1 -- 172.21.15.53:0/1582111865 wait complete. 2023-10-21T17:41:18.020 INFO:tasks.ceph:Scrubbing osd.0 2023-10-21T17:41:18.021 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.260+0000 7ff587fff640 1 Processor -- start 2023-10-21T17:41:18.261 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.260+0000 7ff587fff640 1 -- start start 2023-10-21T17:41:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.261+0000 7ff587fff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff5880cf2b0 0x7ff5880d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.261+0000 7ff587fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 0x7ff58815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.261+0000 7ff587fff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5880cd520 con 0x7ff5880cf690 2023-10-21T17:41:18.263 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.261+0000 7ff587fff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5880d1910 con 0x7ff5880cf2b0 2023-10-21T17:41:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.261+0000 7ff587fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880ce080 con 0x7ff5880ce510 2023-10-21T17:41:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff5880cf2b0 0x7ff5880d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff5880cf2b0 0x7ff5880d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54416/0 (socket says 172.21.15.53:54416) 2023-10-21T17:41:18.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 -- 172.21.15.53:0/3209340592 learned_addr learned my addr 172.21.15.53:0/3209340592 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:18.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff5877fe640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 0x7ff58815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 -- 172.21.15.53:0/3209340592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 msgr2=0x7ff58815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 0x7ff58815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 -- 172.21.15.53:0/3209340592 >> v1:172.21.15.53:6789/0 conn(0x7ff5880cf690 legacy=0x7ff5880d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff586ffd640 1 -- 172.21.15.53:0/3209340592 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880cd750 con 0x7ff5880cf2b0 2023-10-21T17:41:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.262+0000 7ff5877fe640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 0x7ff58815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff586ffd640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5880d34d0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7ff578009e50 tx=0x7ff57802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b8bbb34f747aea server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff585ffb640 1 -- 172.21.15.53:0/3209340592 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57802ca80 con 0x7ff5880cf2b0 2023-10-21T17:41:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff585ffb640 1 -- 172.21.15.53:0/3209340592 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57802cbe0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff585ffb640 1 -- 172.21.15.53:0/3209340592 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5780343f0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 -- 172.21.15.53:0/3209340592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 msgr2=0x7ff5880d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5880d34d0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7ff578009e50 tx=0x7ff57802bee0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 -- 172.21.15.53:0/3209340592 shutdown_connections 2023-10-21T17:41:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880ce510 0x7ff58815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 --2- 172.21.15.53:0/3209340592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5880d34d0 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.263+0000 7ff587fff640 1 -- 172.21.15.53:0/3209340592 >> 172.21.15.53:0/3209340592 conn(0x7ff58809bc40 msgr2=0x7ff5880d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:18.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.264+0000 7ff587fff640 1 -- 172.21.15.53:0/3209340592 shutdown_connections 2023-10-21T17:41:18.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.264+0000 7ff587fff640 1 -- 172.21.15.53:0/3209340592 wait complete. 2023-10-21T17:41:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 Processor -- start 2023-10-21T17:41:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 -- start start 2023-10-21T17:41:18.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5880ce510 0x7ff5880a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5881f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880cf690 0x7ff58809c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881f5e90 con 0x7ff5880cf2b0 2023-10-21T17:41:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881f60c0 con 0x7ff5880ce510 2023-10-21T17:41:18.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.266+0000 7ff587fff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881f63a0 con 0x7ff5880cf690 2023-10-21T17:41:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5881f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5881f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54424/0 (socket says 172.21.15.53:54424) 2023-10-21T17:41:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 -- 172.21.15.53:0/2411768353 learned_addr learned my addr 172.21.15.53:0/2411768353 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff586ffd640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5880ce510 0x7ff5880a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5877fe640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880cf690 0x7ff58809c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 -- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880cf690 msgr2=0x7ff58809c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5880cf690 0x7ff58809c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 -- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5880ce510 msgr2=0x7ff5880a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5880ce510 0x7ff5880a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff5867fc640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009b30 con 0x7ff5880cf2b0 2023-10-21T17:41:18.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.268+0000 7ff586ffd640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5880ce510 0x7ff5880a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff5867fc640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5881f5950 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff57000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700052d0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570005430 con 0x7ff5880cf2b0 2023-10-21T17:41:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570003830 con 0x7ff5880cf2b0 2023-10-21T17:41:18.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5881f6680 con 0x7ff5880cf2b0 2023-10-21T17:41:18.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.269+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58809caf0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.270+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5880caa30 con 0x7ff5880cf2b0 2023-10-21T17:41:18.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.271+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff570003990 con 0x7ff5880cf2b0 2023-10-21T17:41:18.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.272+0000 7ff57f7fe640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7ff56806af90 0x7ff56806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.272+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7ff570088ee0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.272+0000 7ff57f7fe640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7ff5680719e0 0x7ff568073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.272+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff568074510 con 0x7ff5680719e0 2023-10-21T17:41:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.272+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff57008b4c0 con 0x7ff5880cf2b0 2023-10-21T17:41:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.273+0000 7ff586ffd640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7ff56806af90 0x7ff56806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.273+0000 7ff5877fe640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7ff5680719e0 0x7ff568073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.273+0000 7ff5877fe640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7ff5680719e0 0x7ff568073e40 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.274+0000 7ff586ffd640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7ff56806af90 0x7ff56806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff578035000 tx=0x7ff578030000 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.274+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== osd.0 v2:172.21.15.53:6808/716916716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7ff568074510 con 0x7ff5680719e0 2023-10-21T17:41:18.289 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.288+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 --> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff5880cdd10 con 0x7ff5680719e0 2023-10-21T17:41:18.290 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.288+0000 7ff57f7fe640 1 -- 172.21.15.53:0/2411768353 <== osd.0 v2:172.21.15.53:6808/716916716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff58000c040 con 0x7ff5680719e0 2023-10-21T17:41:18.290 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:18.291 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:18.291 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:18.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7ff5680719e0 msgr2=0x7ff568073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:6808/716916716,v1:172.21.15.53:6809/716916716] conn(0x7ff5680719e0 0x7ff568073e40 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7ff56806af90 msgr2=0x7ff56806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7ff56806af90 0x7ff56806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff578035000 tx=0x7ff578030000 comp rx=0 tx=0).stop 2023-10-21T17:41:18.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 msgr2=0x7ff5881f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 --2- 172.21.15.53:0/2411768353 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5880cf2b0 0x7ff5881f5950 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff57000c8c0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff5877fe640 1 -- 172.21.15.53:0/2411768353 reap_dead start 2023-10-21T17:41:18.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 shutdown_connections 2023-10-21T17:41:18.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 >> 172.21.15.53:0/2411768353 conn(0x7ff58809bc40 msgr2=0x7ff588073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:18.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 shutdown_connections 2023-10-21T17:41:18.294 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.289+0000 7ff587fff640 1 -- 172.21.15.53:0/2411768353 wait complete. 2023-10-21T17:41:18.307 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-21T17:41:18.521 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.520+0000 7f12a92bb640 1 Processor -- start 2023-10-21T17:41:18.522 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.520+0000 7f12a92bb640 1 -- start start 2023-10-21T17:41:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.521+0000 7f12a92bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a415ccb0 0x7f12a40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.521+0000 7f12a92bb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a40d0b90 0x7f12a40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.523 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.521+0000 7f12a92bb640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 0x7f12a40d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f12a92bb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40d0380 con 0x7f12a40d0b90 2023-10-21T17:41:18.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f12a92bb640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a40d2870 con 0x7f12a40cff10 2023-10-21T17:41:18.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f12a92bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40d1150 con 0x7f12a415ccb0 2023-10-21T17:41:18.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a40d0b90 0x7f12a40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.524 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a40d0b90 0x7f12a40cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54426/0 (socket says 172.21.15.53:54426) 2023-10-21T17:41:18.525 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 -- 172.21.15.53:0/1794553376 learned_addr learned my addr 172.21.15.53:0/1794553376 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:18.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 -- 172.21.15.53:0/1794553376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a415ccb0 msgr2=0x7f12a40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.526 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1794553376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a415ccb0 0x7f12a40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f12a8aba640 1 --2- 172.21.15.53:0/1794553376 >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 0x7f12a40d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 -- 172.21.15.53:0/1794553376 >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 msgr2=0x7f12a40d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1794553376 >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 0x7f12a40d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f129b7fe640 1 -- 172.21.15.53:0/1794553376 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40d1430 con 0x7f12a40d0b90 2023-10-21T17:41:18.527 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.522+0000 7f12a8aba640 1 --2- 172.21.15.53:0/1794553376 >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 0x7f12a40d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.523+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1794553376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40d0b90 0x7f12a40cf9d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b5a80 tx=0x7f128c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0f0db0bc5be48c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.524+0000 7f12a37fe640 1 -- 172.21.15.53:0/1794553376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c031870 con 0x7f12a40d0b90 2023-10-21T17:41:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.524+0000 7f12a37fe640 1 -- 172.21.15.53:0/1794553376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c0319d0 con 0x7f12a40d0b90 2023-10-21T17:41:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.524+0000 7f12a37fe640 1 -- 172.21.15.53:0/1794553376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c031d60 con 0x7f12a40d0b90 2023-10-21T17:41:18.528 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.524+0000 7f12a92bb640 1 -- 172.21.15.53:0/1794553376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40d0b90 msgr2=0x7f12a40cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1794553376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40d0b90 0x7f12a40cf9d0 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7f12a40b5a80 tx=0x7f128c02b910 comp rx=0 tx=0).stop 2023-10-21T17:41:18.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 -- 172.21.15.53:0/1794553376 shutdown_connections 2023-10-21T17:41:18.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1794553376 >> v2:172.21.15.124:3300/0 conn(0x7f12a40cff10 0x7f12a40d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1794553376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40d0b90 0x7f12a40cf9d0 unknown :-1 s=CLOSED pgs=2943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.529 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1794553376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a415ccb0 0x7f12a40d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 -- 172.21.15.53:0/1794553376 >> 172.21.15.53:0/1794553376 conn(0x7f12a4071660 msgr2=0x7f12a4071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:18.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 -- 172.21.15.53:0/1794553376 shutdown_connections 2023-10-21T17:41:18.530 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.525+0000 7f12a92bb640 1 -- 172.21.15.53:0/1794553376 wait complete. 2023-10-21T17:41:18.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.526+0000 7f12a92bb640 1 Processor -- start 2023-10-21T17:41:18.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.527+0000 7f12a92bb640 1 -- start start 2023-10-21T17:41:18.532 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.527+0000 7f12a92bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a92bb640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a40d0b90 0x7f12a415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a92bb640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 0x7f12a41f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a92bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4160090 con 0x7f12a40cff10 2023-10-21T17:41:18.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a92bb640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c7a50 con 0x7f12a415ccb0 2023-10-21T17:41:18.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a92bb640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40c7d30 con 0x7f12a40d0b90 2023-10-21T17:41:18.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54440/0 (socket says 172.21.15.53:54440) 2023-10-21T17:41:18.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 -- 172.21.15.53:0/1244310771 learned_addr learned my addr 172.21.15.53:0/1244310771 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:18.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a8aba640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 0x7f12a41f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.534 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 -- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a40d0b90 msgr2=0x7f12a415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a40d0b90 0x7f12a415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a40d0b90 0x7f12a415fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 -- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 msgr2=0x7f12a41f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 0x7f12a41f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.535 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a3fff640 1 -- 172.21.15.53:0/1244310771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0094d0 con 0x7f12a40cff10 2023-10-21T17:41:18.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.528+0000 7f12a8aba640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 0x7f12a41f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:18.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a3fff640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f129400ba50 tx=0x7f129400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400cbe0 con 0x7f12a40cff10 2023-10-21T17:41:18.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129400cd60 con 0x7f12a40cff10 2023-10-21T17:41:18.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a40c8010 con 0x7f12a40cff10 2023-10-21T17:41:18.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294014680 con 0x7f12a40cff10 2023-10-21T17:41:18.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.529+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a415d280 con 0x7f12a40cff10 2023-10-21T17:41:18.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.531+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a40b8bb0 con 0x7f12a40cff10 2023-10-21T17:41:18.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.531+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f129401d3b0 con 0x7f12a40cff10 2023-10-21T17:41:18.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.532+0000 7f12a17fa640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 0x7f128406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.538 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.533+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f12940898b0 con 0x7f12a40cff10 2023-10-21T17:41:18.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.533+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 0x7f128406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.534+0000 7f129b7fe640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 0x7f128406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f128c036000 tx=0x7f128c03c040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:18.539 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.537+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1294054720 con 0x7f12a40cff10 2023-10-21T17:41:18.733 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.731+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a40d19f0 con 0x7f128406af90 2023-10-21T17:41:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.732+0000 7f12a17fa640 1 -- 172.21.15.53:0/1244310771 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f128c031cc0 con 0x7f128406af90 2023-10-21T17:41:18.734 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 0 to deep-scrub 2023-10-21T17:41:18.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.734+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 msgr2=0x7f128406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.734+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 0x7f128406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f128c036000 tx=0x7f128c03c040 comp rx=0 tx=0).stop 2023-10-21T17:41:18.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.734+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 msgr2=0x7f12a40c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.734+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f129400ba50 tx=0x7f129400bf10 comp rx=0 tx=0).stop 2023-10-21T17:41:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 shutdown_connections 2023-10-21T17:41:18.738 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128406af90 0x7f128406d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a415ccb0 0x7f12a41f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a40d0b90 0x7f12a415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 --2- 172.21.15.53:0/1244310771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a40cff10 0x7f12a40c73e0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 >> 172.21.15.53:0/1244310771 conn(0x7f12a4071660 msgr2=0x7f12a40b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:18.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 shutdown_connections 2023-10-21T17:41:18.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.735+0000 7f12a92bb640 1 -- 172.21.15.53:0/1244310771 wait complete. 2023-10-21T17:41:18.756 INFO:tasks.ceph:Scrubbing osd.1 2023-10-21T17:41:18.756 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.985+0000 7fcc801c5640 1 Processor -- start 2023-10-21T17:41:18.987 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.986+0000 7fcc801c5640 1 -- start start 2023-10-21T17:41:18.988 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.987+0000 7fcc801c5640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc78161690 0x7fcc7806dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.987+0000 7fcc801c5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc780cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.987+0000 7fcc801c5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7806cc50 con 0x7fcc780cb150 2023-10-21T17:41:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.987+0000 7fcc801c5640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc780cbb40 con 0x7fcc78161690 2023-10-21T17:41:18.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.987+0000 7fcc801c5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7806cf30 con 0x7fcc780b2f30 2023-10-21T17:41:18.991 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7d760640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fcc780cb150 0x7fcc780702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53126/0 (socket says 172.21.15.53:53126) 2023-10-21T17:41:18.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7d760640 1 -- 172.21.15.53:0/414610051 learned_addr learned my addr 172.21.15.53:0/414610051 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:18.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7df61640 1 --2- 172.21.15.53:0/414610051 >> v2:172.21.15.124:3300/0 conn(0x7fcc78161690 0x7fcc7806dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7e762640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc780cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.992 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341540189 0 0) 0x7fcc7806cc50 con 0x7fcc780cb150 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.988+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc64003520 con 0x7fcc780cb150 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133926749 0 0) 0x7fcc64003520 con 0x7fcc780cb150 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 msgr2=0x7fcc780cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc780cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 >> v2:172.21.15.124:3300/0 conn(0x7fcc78161690 msgr2=0x7fcc7806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.993 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 --2- 172.21.15.53:0/414610051 >> v2:172.21.15.124:3300/0 conn(0x7fcc78161690 0x7fcc7806dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7806d210 con 0x7fcc780cb150 2023-10-21T17:41:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7fcc68001690 con 0x7fcc780cb150 2023-10-21T17:41:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 >> v1:172.21.15.53:6789/0 conn(0x7fcc780cb150 legacy=0x7fcc780702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc64003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.994 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc64003ef0 0x7fcc640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc64004d00 0x7fcc64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc780cbb40 con 0x7fcc64004d00 2023-10-21T17:41:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7806cf30 con 0x7fcc64003ef0 2023-10-21T17:41:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6400b980 con 0x7fcc780b2f30 2023-10-21T17:41:18.995 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7e762640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc64004d00 0x7fcc64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.989+0000 7fcc7d760640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc64003ef0 0x7fcc640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7df61640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc64003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7e762640 1 -- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 msgr2=0x7fcc64003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7e762640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780b2f30 0x7fcc64003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.996 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7e762640 1 -- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc64003ef0 msgr2=0x7fcc640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7e762640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc64003ef0 0x7fcc640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:18.997 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.990+0000 7fcc7e762640 1 -- 172.21.15.53:0/414610051 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64003520 con 0x7fcc64004d00 2023-10-21T17:41:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.991+0000 7fcc7e762640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc64004d00 0x7fcc64005180 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009fc0 tx=0x7fcc640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17036a95e687ea9e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.991+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7002cc00 con 0x7fcc64004d00 2023-10-21T17:41:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.991+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7000f040 con 0x7fcc64004d00 2023-10-21T17:41:19.001 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.991+0000 7fcc7cf5f640 1 -- 172.21.15.53:0/414610051 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70034440 con 0x7fcc64004d00 2023-10-21T17:41:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 -- 172.21.15.53:0/414610051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc64004d00 msgr2=0x7fcc64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 --2- 172.21.15.53:0/414610051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc64004d00 0x7fcc64005180 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009fc0 tx=0x7fcc640057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc7e762640 1 -- 172.21.15.53:0/414610051 reap_dead start 2023-10-21T17:41:19.002 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 -- 172.21.15.53:0/414610051 shutdown_connections 2023-10-21T17:41:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 -- 172.21.15.53:0/414610051 >> 172.21.15.53:0/414610051 conn(0x7fcc7806a5d0 msgr2=0x7fcc780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 -- 172.21.15.53:0/414610051 shutdown_connections 2023-10-21T17:41:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.992+0000 7fcc801c5640 1 -- 172.21.15.53:0/414610051 wait complete. 2023-10-21T17:41:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.994+0000 7fcc801c5640 1 Processor -- start 2023-10-21T17:41:19.003 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.994+0000 7fcc801c5640 1 -- start start 2023-10-21T17:41:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 0x7fcc7814db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc780c20e0 0x7fcc7814e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780c7e70 0x7fcc78150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7814e7f0 con 0x7fcc780c7a90 2023-10-21T17:41:19.004 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7814ea40 con 0x7fcc780c20e0 2023-10-21T17:41:19.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc801c5640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7814ed20 con 0x7fcc780c7e70 2023-10-21T17:41:19.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc7df61640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 0x7fcc7814db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc7df61640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 0x7fcc7814db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54462/0 (socket says 172.21.15.53:54462) 2023-10-21T17:41:19.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc7df61640 1 -- 172.21.15.53:0/3993236429 learned_addr learned my addr 172.21.15.53:0/3993236429 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:19.005 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.995+0000 7fcc7d760640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc780c20e0 0x7fcc7814e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 -- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780c7e70 msgr2=0x7fcc78150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc780c7e70 0x7fcc78150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 -- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc780c20e0 msgr2=0x7fcc7814e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc780c20e0 0x7fcc7814e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.006 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7814f000 con 0x7fcc780c7a90 2023-10-21T17:41:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7d760640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc780c20e0 0x7fcc7814e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc7df61640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 0x7fcc7814db60 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7fcc74010860 tx=0x7fcc74010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74009690 con 0x7fcc780c7a90 2023-10-21T17:41:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7400eb60 con 0x7fcc780c7a90 2023-10-21T17:41:19.007 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7401c400 con 0x7fcc780c7a90 2023-10-21T17:41:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7814f2e0 con 0x7fcc780c7a90 2023-10-21T17:41:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.996+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc781512f0 con 0x7fcc780c7a90 2023-10-21T17:41:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.997+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc7814ebb0 con 0x7fcc780c7a90 2023-10-21T17:41:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.999+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc7401c560 con 0x7fcc780c7a90 2023-10-21T17:41:19.008 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:18.999+0000 7fcc6e7fc640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc5c06af90 0x7fcc5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fcc74089f90 con 0x7fcc780c7a90 2023-10-21T17:41:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc6e7fc640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7fcc5c0719e0 0x7fcc5c073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.009 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc5c074510 con 0x7fcc5c0719e0 2023-10-21T17:41:19.010 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc7408c570 con 0x7fcc780c7a90 2023-10-21T17:41:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc7e762640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7fcc5c0719e0 0x7fcc5c073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.000+0000 7fcc7d760640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc5c06af90 0x7fcc5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.001+0000 7fcc7e762640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7fcc5c0719e0 0x7fcc5c073e40 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.001+0000 7fcc7d760640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc5c06af90 0x7fcc5c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001eb0 tx=0x7fcc68000ed0 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.002+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== osd.1 v2:172.21.15.53:6800/752205661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fcc5c074510 con 0x7fcc5c0719e0 2023-10-21T17:41:19.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.016+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 --> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcc780b1990 con 0x7fcc5c0719e0 2023-10-21T17:41:19.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.016+0000 7fcc6e7fc640 1 -- 172.21.15.53:0/3993236429 <== osd.1 v2:172.21.15.53:6800/752205661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcc7002db10 con 0x7fcc5c0719e0 2023-10-21T17:41:19.018 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:19.018 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:19.018 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:19.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7fcc5c0719e0 msgr2=0x7fcc5c073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:6800/752205661,v1:172.21.15.53:6801/752205661] conn(0x7fcc5c0719e0 0x7fcc5c073e40 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc5c06af90 msgr2=0x7fcc5c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcc5c06af90 0x7fcc5c06d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001eb0 tx=0x7fcc68000ed0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 msgr2=0x7fcc7814db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 --2- 172.21.15.53:0/3993236429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc780c7a90 0x7fcc7814db60 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7fcc74010860 tx=0x7fcc74010d20 comp rx=0 tx=0).stop 2023-10-21T17:41:19.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc7e762640 1 -- 172.21.15.53:0/3993236429 reap_dead start 2023-10-21T17:41:19.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 shutdown_connections 2023-10-21T17:41:19.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 >> 172.21.15.53:0/3993236429 conn(0x7fcc7806a5d0 msgr2=0x7fcc780b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 shutdown_connections 2023-10-21T17:41:19.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.017+0000 7fcc801c5640 1 -- 172.21.15.53:0/3993236429 wait complete. 2023-10-21T17:41:19.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-21T17:41:19.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.263+0000 7f03d1852640 1 Processor -- start 2023-10-21T17:41:19.264 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.263+0000 7f03d1852640 1 -- start start 2023-10-21T17:41:19.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.264+0000 7f03d1852640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc1599d0 0x7f03cc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.265 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.264+0000 7f03d1852640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.264+0000 7f03d1852640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc159db0 con 0x7f03cc0b3c50 2023-10-21T17:41:19.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.264+0000 7f03d1852640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03cc071fe0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.266 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.264+0000 7f03d1852640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0b44c0 con 0x7f03cc1599d0 2023-10-21T17:41:19.267 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54474/0 (socket says 172.21.15.53:54474) 2023-10-21T17:41:19.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 -- 172.21.15.53:0/1319546940 learned_addr learned my addr 172.21.15.53:0/1319546940 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:19.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d0850640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc1599d0 0x7f03cc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 -- 172.21.15.53:0/1319546940 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc1599d0 msgr2=0x7f03cc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:19.268 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc1599d0 0x7f03cc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 -- 172.21.15.53:0/1319546940 >> v1:172.21.15.124:6789/0 conn(0x7f03cc0b3c50 legacy=0x7f03cc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:19.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.265+0000 7f03d1051640 1 -- 172.21.15.53:0/1319546940 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc0713d0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.269 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.266+0000 7f03d1051640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009e50 tx=0x7f03c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517e8aa14f861160 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03cb7fe640 1 -- 172.21.15.53:0/1319546940 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c402ca80 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.270 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03cb7fe640 1 -- 172.21.15.53:0/1319546940 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c402cbe0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03cb7fe640 1 -- 172.21.15.53:0/1319546940 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40343f0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 -- 172.21.15.53:0/1319546940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 msgr2=0x7f03cc0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009e50 tx=0x7f03c402bee0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 -- 172.21.15.53:0/1319546940 shutdown_connections 2023-10-21T17:41:19.271 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc0cc0c0 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 --2- 172.21.15.53:0/1319546940 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc1599d0 0x7f03cc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 -- 172.21.15.53:0/1319546940 >> 172.21.15.53:0/1319546940 conn(0x7f03cc06c050 msgr2=0x7f03cc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.267+0000 7f03d1852640 1 -- 172.21.15.53:0/1319546940 shutdown_connections 2023-10-21T17:41:19.272 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.268+0000 7f03d1852640 1 -- 172.21.15.53:0/1319546940 wait complete. 2023-10-21T17:41:19.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.269+0000 7f03d1852640 1 Processor -- start 2023-10-21T17:41:19.273 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.269+0000 7f03d1852640 1 -- start start 2023-10-21T17:41:19.274 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc0b3c50 0x7f03cc153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 0x7f03cc14a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0b0e70 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.275 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0affc0 con 0x7f03cc1599d0 2023-10-21T17:41:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.270+0000 7f03d1852640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0b0260 con 0x7f03cc0b3c50 2023-10-21T17:41:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54484/0 (socket says 172.21.15.53:54484) 2023-10-21T17:41:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 -- 172.21.15.53:0/368713644 learned_addr learned my addr 172.21.15.53:0/368713644 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:19.276 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03d1051640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 0x7f03cc14a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03d0850640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc0b3c50 0x7f03cc153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 -- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc0b3c50 msgr2=0x7f03cc153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc0b3c50 0x7f03cc153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 -- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 msgr2=0x7f03cc14a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 0x7f03cc14a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 -- 172.21.15.53:0/368713644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4009b30 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03d1051640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 0x7f03cc14a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.271+0000 7f03cbfff640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c430 tx=0x7f03b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.272+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011790 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.272+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b80118f0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.272+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011ce0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.272+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc0b0540 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.272+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc0b0b00 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.279 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.273+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc074150 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.274+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03b801c460 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.275+0000 7f03c97fa640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 0x7f03b006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.275+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f03b80898c0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.276+0000 7f03d0850640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 0x7f03b006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.277+0000 7f03d0850640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 0x7f03b006d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03c0006fd0 tx=0x7f03c0008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.280+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03b80546a0 con 0x7f03cc0cb5e0 2023-10-21T17:41:19.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.475+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f03cc071990 con 0x7f03b006afc0 2023-10-21T17:41:19.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.476+0000 7f03c97fa640 1 -- 172.21.15.53:0/368713644 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03c000e4c0 con 0x7f03b006afc0 2023-10-21T17:41:19.477 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 1 to deep-scrub 2023-10-21T17:41:19.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 msgr2=0x7f03b006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 0x7f03b006d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03c0006fd0 tx=0x7f03c0008040 comp rx=0 tx=0).stop 2023-10-21T17:41:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 msgr2=0x7f03cc149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c430 tx=0x7f03b800c8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 shutdown_connections 2023-10-21T17:41:19.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f03b006afc0 0x7f03b006d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03cc1599d0 0x7f03cc14a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03cc0cb5e0 0x7f03cc149ad0 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 --2- 172.21.15.53:0/368713644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03cc0b3c50 0x7f03cc153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 >> 172.21.15.53:0/368713644 conn(0x7f03cc06c050 msgr2=0x7f03cc0ca420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 shutdown_connections 2023-10-21T17:41:19.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.478+0000 7f03d1852640 1 -- 172.21.15.53:0/368713644 wait complete. 2023-10-21T17:41:19.497 INFO:tasks.ceph:Scrubbing osd.2 2023-10-21T17:41:19.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:19.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.732+0000 7fd869367640 1 Processor -- start 2023-10-21T17:41:19.734 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.733+0000 7fd869367640 1 -- start start 2023-10-21T17:41:19.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.734+0000 7fd869367640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.734+0000 7fd869367640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd8640d0b90 0x7fd8640b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.734+0000 7fd869367640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640d2870 con 0x7fd86415ccb0 2023-10-21T17:41:19.736 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.734+0000 7fd869367640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640d1150 con 0x7fd8640d0b90 2023-10-21T17:41:19.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.734+0000 7fd869367640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640d1430 con 0x7fd8640d00e0 2023-10-21T17:41:19.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd863fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fd86415ccb0 0x7fd8640cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53130/0 (socket says 172.21.15.53:53130) 2023-10-21T17:41:19.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd863fff640 1 -- 172.21.15.53:0/3877513373 learned_addr learned my addr 172.21.15.53:0/3877513373 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:19.737 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd868b66640 1 --2- 172.21.15.53:0/3877513373 >> v2:172.21.15.53:3300/0 conn(0x7fd8640d0b90 0x7fd8640b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd85bfff640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd868b66640 1 -- 172.21.15.53:0/3877513373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 msgr2=0x7fd8640d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:19.739 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd868b66640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd868b66640 1 -- 172.21.15.53:0/3877513373 >> v1:172.21.15.53:6789/0 conn(0x7fd86415ccb0 legacy=0x7fd8640cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.735+0000 7fd868b66640 1 -- 172.21.15.53:0/3877513373 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640d1710 con 0x7fd8640d0b90 2023-10-21T17:41:19.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.736+0000 7fd868b66640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640b6060 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009d90 tx=0x7fd85c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beae2fe840e0a37d server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.737+0000 7fd8637fe640 1 -- 172.21.15.53:0/3877513373 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c02cc50 con 0x7fd8640d0b90 2023-10-21T17:41:19.740 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.737+0000 7fd8637fe640 1 -- 172.21.15.53:0/3877513373 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c02cdb0 con 0x7fd8640d0b90 2023-10-21T17:41:19.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.737+0000 7fd8637fe640 1 -- 172.21.15.53:0/3877513373 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c034660 con 0x7fd8640d0b90 2023-10-21T17:41:19.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.737+0000 7fd869367640 1 -- 172.21.15.53:0/3877513373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 msgr2=0x7fd8640b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.737+0000 7fd869367640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640b6060 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009d90 tx=0x7fd85c02bea0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 -- 172.21.15.53:0/3877513373 shutdown_connections 2023-10-21T17:41:19.741 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640b6060 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 --2- 172.21.15.53:0/3877513373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 -- 172.21.15.53:0/3877513373 >> 172.21.15.53:0/3877513373 conn(0x7fd864071660 msgr2=0x7fd864071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 -- 172.21.15.53:0/3877513373 shutdown_connections 2023-10-21T17:41:19.742 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.738+0000 7fd869367640 1 -- 172.21.15.53:0/3877513373 wait complete. 2023-10-21T17:41:19.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.739+0000 7fd869367640 1 Processor -- start 2023-10-21T17:41:19.744 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.740+0000 7fd869367640 1 -- start start 2023-10-21T17:41:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd86415ccb0 0x7fd8641f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c9780 con 0x7fd8640d0b90 2023-10-21T17:41:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c7430 con 0x7fd86415ccb0 2023-10-21T17:41:19.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd869367640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c76d0 con 0x7fd8640d00e0 2023-10-21T17:41:19.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd85bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd85bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54502/0 (socket says 172.21.15.53:54502) 2023-10-21T17:41:19.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd85bfff640 1 -- 172.21.15.53:0/498168532 learned_addr learned my addr 172.21.15.53:0/498168532 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:19.746 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.741+0000 7fd868b66640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd86415ccb0 0x7fd8641f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 -- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 msgr2=0x7fd8640c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd863fff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8640d00e0 0x7fd8640c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 -- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd86415ccb0 msgr2=0x7fd8641f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.747 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd86415ccb0 0x7fd8641f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c009a70 con 0x7fd8640d0b90 2023-10-21T17:41:19.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd868b66640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd86415ccb0 0x7fd8641f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:19.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd85bfff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640c81b0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00d800 tx=0x7fd84c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.748 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00b6c0 con 0x7fd8640d0b90 2023-10-21T17:41:19.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00b820 con 0x7fd8640d0b90 2023-10-21T17:41:19.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640c79b0 con 0x7fd8640d0b90 2023-10-21T17:41:19.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00bbb0 con 0x7fd8640d0b90 2023-10-21T17:41:19.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.742+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8640c9470 con 0x7fd8640d0b90 2023-10-21T17:41:19.749 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.743+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8640b5b10 con 0x7fd8640d0b90 2023-10-21T17:41:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.745+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd84c00bd10 con 0x7fd8640d0b90 2023-10-21T17:41:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd8617fa640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd84406af90 0x7fd84406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fd84c089e90 con 0x7fd8640d0b90 2023-10-21T17:41:19.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd8617fa640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7fd8440719e0 0x7fd844073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd844074510 con 0x7fd8440719e0 2023-10-21T17:41:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84c08c550 con 0x7fd8640d0b90 2023-10-21T17:41:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd863fff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd84406af90 0x7fd84406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.746+0000 7fd868b66640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7fd8440719e0 0x7fd844073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:19.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.747+0000 7fd868b66640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7fd8440719e0 0x7fd844073e40 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.747+0000 7fd863fff640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd84406af90 0x7fd84406d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd854005880 tx=0x7fd854009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:19.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.748+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== osd.2 v2:172.21.15.53:6816/3435187098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd844074510 con 0x7fd8440719e0 2023-10-21T17:41:19.763 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.762+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 --> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8640d19f0 con 0x7fd8440719e0 2023-10-21T17:41:19.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd8617fa640 1 -- 172.21.15.53:0/498168532 <== osd.2 v2:172.21.15.53:6816/3435187098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd85c009450 con 0x7fd8440719e0 2023-10-21T17:41:19.764 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:19.764 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:19.765 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:19.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7fd8440719e0 msgr2=0x7fd844073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:6816/3435187098,v1:172.21.15.53:6817/3435187098] conn(0x7fd8440719e0 0x7fd844073e40 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd84406af90 msgr2=0x7fd84406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fd84406af90 0x7fd84406d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd854005880 tx=0x7fd854009040 comp rx=0 tx=0).stop 2023-10-21T17:41:19.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 msgr2=0x7fd8640c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:19.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd869367640 1 --2- 172.21.15.53:0/498168532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8640d0b90 0x7fd8640c81b0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00d800 tx=0x7fd84c00dcc0 comp rx=0 tx=0).stop 2023-10-21T17:41:19.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.763+0000 7fd868b66640 1 -- 172.21.15.53:0/498168532 reap_dead start 2023-10-21T17:41:19.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.764+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 shutdown_connections 2023-10-21T17:41:19.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.764+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 >> 172.21.15.53:0/498168532 conn(0x7fd864071660 msgr2=0x7fd864073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:19.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.764+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 shutdown_connections 2023-10-21T17:41:19.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:19.764+0000 7fd869367640 1 -- 172.21.15.53:0/498168532 wait complete. 2023-10-21T17:41:19.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-21T17:41:20.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.009+0000 7f6862d7a640 1 Processor -- start 2023-10-21T17:41:20.011 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.009+0000 7f6862d7a640 1 -- start start 2023-10-21T17:41:20.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6862d7a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f685c15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.012 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6862d7a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0cab60 con 0x7f685c0b0680 2023-10-21T17:41:20.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6862d7a640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c0cb960 con 0x7f685c15a820 2023-10-21T17:41:20.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6862d7a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cbdf0 con 0x7f685c0ca680 2023-10-21T17:41:20.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6861577640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f685c0b0680 0x7f685c15e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53144/0 (socket says 172.21.15.53:53144) 2023-10-21T17:41:20.013 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.011+0000 7f6861577640 1 -- 172.21.15.53:0/4287770004 learned_addr learned my addr 172.21.15.53:0/4287770004 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:20.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6862579640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f685c15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392873376 0 0) 0x7f685c0cab60 con 0x7f685c0b0680 2023-10-21T17:41:20.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848003520 con 0x7f685c0b0680 2023-10-21T17:41:20.016 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336344139 0 0) 0x7f6848003520 con 0x7f685c0b0680 2023-10-21T17:41:20.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 msgr2=0x7f685c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f685c15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 >> v1:172.21.15.124:6789/0 conn(0x7f685c15a820 legacy=0x7f685c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0cb030 con 0x7f685c0b0680 2023-10-21T17:41:20.017 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f684c001670 con 0x7f685c0b0680 2023-10-21T17:41:20.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 >> v1:172.21.15.53:6789/0 conn(0x7f685c0b0680 legacy=0x7f685c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68480039a0 0x7f6848003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f68480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6848004d00 0x7f6848005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.018 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cbdf0 con 0x7f68480039a0 2023-10-21T17:41:20.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684800bab0 con 0x7f6848004d00 2023-10-21T17:41:20.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.012+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684800bc70 con 0x7f685c0ca680 2023-10-21T17:41:20.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68480039a0 0x7f6848003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6862579640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6848004d00 0x7f6848005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.019 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861577640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f68480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 -- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 msgr2=0x7f68480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f68480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 -- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6848004d00 msgr2=0x7f6848005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6848004d00 0x7f6848005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861d78640 1 -- 172.21.15.53:0/4287770004 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848003520 con 0x7f68480039a0 2023-10-21T17:41:20.020 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6862579640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6848004d00 0x7f6848005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:20.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.013+0000 7f6861577640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0ca680 0x7f68480047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:20.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.014+0000 7f6861d78640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68480039a0 0x7f6848003dc0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f6858001750 tx=0x7f68480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdded860eb26fefd server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68580052d0 con 0x7f68480039a0 2023-10-21T17:41:20.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685802e050 con 0x7f68480039a0 2023-10-21T17:41:20.021 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6860d76640 1 -- 172.21.15.53:0/4287770004 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68580035f0 con 0x7f68480039a0 2023-10-21T17:41:20.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 -- 172.21.15.53:0/4287770004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68480039a0 msgr2=0x7f6848003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 --2- 172.21.15.53:0/4287770004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68480039a0 0x7f6848003dc0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7f6858001750 tx=0x7f68480057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.024 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862579640 1 -- 172.21.15.53:0/4287770004 reap_dead start 2023-10-21T17:41:20.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 -- 172.21.15.53:0/4287770004 shutdown_connections 2023-10-21T17:41:20.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 -- 172.21.15.53:0/4287770004 >> 172.21.15.53:0/4287770004 conn(0x7f685c06c050 msgr2=0x7f685c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 -- 172.21.15.53:0/4287770004 shutdown_connections 2023-10-21T17:41:20.025 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.015+0000 7f6862d7a640 1 -- 172.21.15.53:0/4287770004 wait complete. 2023-10-21T17:41:20.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.017+0000 7f6862d7a640 1 Processor -- start 2023-10-21T17:41:20.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.017+0000 7f6862d7a640 1 -- start start 2023-10-21T17:41:20.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.018+0000 7f6862d7a640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 0x7f685c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.018+0000 7f6862d7a640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0c6f30 0x7f685c150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.026 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.018+0000 7f6862d7a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.018+0000 7f6862d7a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c150850 con 0x7f685c0c7d20 2023-10-21T17:41:20.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.018+0000 7f6862d7a640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c6170 con 0x7f68480039a0 2023-10-21T17:41:20.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862d7a640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0c6450 con 0x7f685c0c6f30 2023-10-21T17:41:20.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.027 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54532/0 (socket says 172.21.15.53:54532) 2023-10-21T17:41:20.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 -- 172.21.15.53:0/2798933652 learned_addr learned my addr 172.21.15.53:0/2798933652 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:20.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6861d78640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 0x7f685c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6861577640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0c6f30 0x7f685c150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 -- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0c6f30 msgr2=0x7f685c150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.028 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0c6f30 0x7f685c150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 -- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 msgr2=0x7f685c0c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 0x7f685c0c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6862579640 1 -- 172.21.15.53:0/2798933652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0c6730 con 0x7f685c0c7d20 2023-10-21T17:41:20.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.019+0000 7f6861d78640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 0x7f685c0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:20.029 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f6862579640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f685c0ad940 tx=0x7f685400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854017400 con 0x7f685c0c7d20 2023-10-21T17:41:20.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854011040 con 0x7f685c0c7d20 2023-10-21T17:41:20.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0c6a10 con 0x7f685c0c7d20 2023-10-21T17:41:20.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854017780 con 0x7f685c0c7d20 2023-10-21T17:41:20.030 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.020+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0c41e0 con 0x7f685c0c7d20 2023-10-21T17:41:20.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.021+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c0c6c00 con 0x7f685c0c7d20 2023-10-21T17:41:20.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.022+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68540203b0 con 0x7f685c0c7d20 2023-10-21T17:41:20.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.023+0000 7f68527fc640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 0x7f684006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.031 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.023+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f6854089c00 con 0x7f685c0c7d20 2023-10-21T17:41:20.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.024+0000 7f6861d78640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 0x7f684006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.025+0000 7f6861d78640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 0x7f684006d480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6858033000 tx=0x7f6858009030 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.032 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.027+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6854054830 con 0x7f685c0c7d20 2023-10-21T17:41:20.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.224+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f685c0c62e0 con 0x7f684006afc0 2023-10-21T17:41:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.225+0000 7f68527fc640 1 -- 172.21.15.53:0/2798933652 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f685c0c62e0 con 0x7f684006afc0 2023-10-21T17:41:20.227 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 2 to deep-scrub 2023-10-21T17:41:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 msgr2=0x7f684006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 0x7f684006d480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6858033000 tx=0x7f6858009030 comp rx=0 tx=0).stop 2023-10-21T17:41:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 msgr2=0x7f685c0c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 secure :-1 s=READY pgs=2955 cs=0 l=1 rev1=1 crypto rx=0x7f685c0ad940 tx=0x7f685400f4a0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 shutdown_connections 2023-10-21T17:41:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f684006afc0 0x7f684006d480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f685c0c7d20 0x7f685c0c3ca0 unknown :-1 s=CLOSED pgs=2955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f685c0c6f30 0x7f685c150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.228+0000 7f6862d7a640 1 --2- 172.21.15.53:0/2798933652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68480039a0 0x7f685c0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.229+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 >> 172.21.15.53:0/2798933652 conn(0x7f685c06c050 msgr2=0x7f685c159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.229+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 shutdown_connections 2023-10-21T17:41:20.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.229+0000 7f6862d7a640 1 -- 172.21.15.53:0/2798933652 wait complete. 2023-10-21T17:41:20.248 INFO:tasks.ceph:Scrubbing osd.3 2023-10-21T17:41:20.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:20.476 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.475+0000 7f779e305640 1 Processor -- start 2023-10-21T17:41:20.477 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.475+0000 7f779e305640 1 -- start start 2023-10-21T17:41:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.477+0000 7f779e305640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779806c3a0 0x7f77980c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.477+0000 7f779e305640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f779806db60 0x7f779806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.477+0000 7f779e305640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779815b310 con 0x7f779815aa00 2023-10-21T17:41:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.477+0000 7f779e305640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980ca230 con 0x7f779806db60 2023-10-21T17:41:20.479 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.477+0000 7f779e305640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779806d290 con 0x7f779806c3a0 2023-10-21T17:41:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f779806db60 0x7f779806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f779806db60 0x7f779806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54538/0 (socket says 172.21.15.53:54538) 2023-10-21T17:41:20.480 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 -- 172.21.15.53:0/4218156710 learned_addr learned my addr 172.21.15.53:0/4218156710 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f7797fff640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779806c3a0 0x7f77980c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 -- 172.21.15.53:0/4218156710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779806c3a0 msgr2=0x7f77980c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.481 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779806c3a0 0x7f77980c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 -- 172.21.15.53:0/4218156710 >> v1:172.21.15.124:6789/0 conn(0x7f779815aa00 legacy=0x7f77980b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.478+0000 7f77977fe640 1 -- 172.21.15.53:0/4218156710 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980c9860 con 0x7f779806db60 2023-10-21T17:41:20.482 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.480+0000 7f77977fe640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806db60 0x7f779806c780 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f778c009870 tx=0x7f778c02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7ebe1e8bede76e server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.480+0000 7f7796ffd640 1 -- 172.21.15.53:0/4218156710 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c02f8c0 con 0x7f779806db60 2023-10-21T17:41:20.483 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f7796ffd640 1 -- 172.21.15.53:0/4218156710 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c02fa20 con 0x7f779806db60 2023-10-21T17:41:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f7796ffd640 1 -- 172.21.15.53:0/4218156710 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c02fdb0 con 0x7f779806db60 2023-10-21T17:41:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 -- 172.21.15.53:0/4218156710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806db60 msgr2=0x7f779806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806db60 0x7f779806c780 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7f778c009870 tx=0x7f778c02db00 comp rx=0 tx=0).stop 2023-10-21T17:41:20.484 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 -- 172.21.15.53:0/4218156710 shutdown_connections 2023-10-21T17:41:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806db60 0x7f779806c780 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 --2- 172.21.15.53:0/4218156710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779806c3a0 0x7f77980c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.485 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 -- 172.21.15.53:0/4218156710 >> 172.21.15.53:0/4218156710 conn(0x7f779806a5d0 msgr2=0x7f77980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 -- 172.21.15.53:0/4218156710 shutdown_connections 2023-10-21T17:41:20.486 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.481+0000 7f779e305640 1 -- 172.21.15.53:0/4218156710 wait complete. 2023-10-21T17:41:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.483+0000 7f779e305640 1 Processor -- start 2023-10-21T17:41:20.488 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.483+0000 7f779e305640 1 -- start start 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 0x7f7798152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f779806db60 0x7f779815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779815aa00 0x7f779815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980c66b0 con 0x7f779806c3a0 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980c6b40 con 0x7f779806db60 2023-10-21T17:41:20.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f779e305640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77980c79e0 con 0x7f779815aa00 2023-10-21T17:41:20.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f7797fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 0x7f7798152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f7797fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 0x7f7798152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54546/0 (socket says 172.21.15.53:54546) 2023-10-21T17:41:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.484+0000 7f7797fff640 1 -- 172.21.15.53:0/3997385063 learned_addr learned my addr 172.21.15.53:0/3997385063 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f77977fe640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f779806db60 0x7f779815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 -- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779815aa00 msgr2=0x7f779815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f779815aa00 0x7f779815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 -- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f779806db60 msgr2=0x7f779815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f779806db60 0x7f779815d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0094d0 con 0x7f779806c3a0 2023-10-21T17:41:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7797fff640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 0x7f7798152360 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f778800c7f0 tx=0x7f778800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788004220 con 0x7f779806c3a0 2023-10-21T17:41:20.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.485+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77880043a0 con 0x7f779806c3a0 2023-10-21T17:41:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.486+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77980c5d80 con 0x7f779806c3a0 2023-10-21T17:41:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.486+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788007860 con 0x7f779806c3a0 2023-10-21T17:41:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.486+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77980c6340 con 0x7f779806c3a0 2023-10-21T17:41:20.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.487+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77980c8f30 con 0x7f779806c3a0 2023-10-21T17:41:20.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.489+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7788007bd0 con 0x7f779806c3a0 2023-10-21T17:41:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.490+0000 7f7794ff9640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f778406af90 0x7f778406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.490+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f77880899e0 con 0x7f779806c3a0 2023-10-21T17:41:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.490+0000 7f7794ff9640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f77840719e0 0x7f7784073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.490+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7784074510 con 0x7f77840719e0 2023-10-21T17:41:20.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.490+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778808e2c0 con 0x7f779806c3a0 2023-10-21T17:41:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.491+0000 7f779c8a2640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f77840719e0 0x7f7784073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.491+0000 7f77977fe640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f778406af90 0x7f778406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.491+0000 7f779c8a2640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f77840719e0 0x7f7784073e40 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.492+0000 7f77977fe640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f778406af90 0x7f778406d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f778c038000 tx=0x7f778c008030 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.492+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== osd.3 v2:172.21.15.53:6824/1838632921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f7784074510 con 0x7f77840719e0 2023-10-21T17:41:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.507+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 --> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f779806ccc0 con 0x7f77840719e0 2023-10-21T17:41:20.508 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.507+0000 7f7794ff9640 1 -- 172.21.15.53:0/3997385063 <== osd.3 v2:172.21.15.53:6824/1838632921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f779000ac40 con 0x7f77840719e0 2023-10-21T17:41:20.509 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:20.509 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:20.509 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:20.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f77840719e0 msgr2=0x7f7784073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.510 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:6824/1838632921,v1:172.21.15.53:6825/1838632921] conn(0x7f77840719e0 0x7f7784073e40 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f778406af90 msgr2=0x7f778406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f778406af90 0x7f778406d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f778c038000 tx=0x7f778c008030 comp rx=0 tx=0).stop 2023-10-21T17:41:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 msgr2=0x7f7798152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 --2- 172.21.15.53:0/3997385063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f779806c3a0 0x7f7798152360 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7f778800c7f0 tx=0x7f778800ccb0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779c8a2640 1 -- 172.21.15.53:0/3997385063 reap_dead start 2023-10-21T17:41:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 shutdown_connections 2023-10-21T17:41:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 >> 172.21.15.53:0/3997385063 conn(0x7f779806a5d0 msgr2=0x7f779806d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.508+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 shutdown_connections 2023-10-21T17:41:20.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.509+0000 7f779e305640 1 -- 172.21.15.53:0/3997385063 wait complete. 2023-10-21T17:41:20.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-21T17:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.748+0000 7f5d28ba9640 1 Processor -- start 2023-10-21T17:41:20.750 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.749+0000 7f5d28ba9640 1 -- start start 2023-10-21T17:41:20.751 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d28ba9640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d2415ae30 0x7f5d240cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d28ba9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d240cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d28ba9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d240cba20 con 0x7f5d240af920 2023-10-21T17:41:20.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d28ba9640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d240cde60 con 0x7f5d2415ae30 2023-10-21T17:41:20.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d28ba9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240cebf0 con 0x7f5d240cd470 2023-10-21T17:41:20.752 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.750+0000 7f5d1bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f5d240af920 0x7f5d240cb310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53160/0 (socket says 172.21.15.53:53160) 2023-10-21T17:41:20.753 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d1bfff640 1 -- 172.21.15.53:0/3886050481 learned_addr learned my addr 172.21.15.53:0/3886050481 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:20.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d2259c640 1 --2- 172.21.15.53:0/3886050481 >> v2:172.21.15.124:3300/0 conn(0x7f5d2415ae30 0x7f5d240cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d240cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.755 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599076159 0 0) 0x7f5d0c001110 con 0x7f5d240af920 2023-10-21T17:41:20.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d10003520 con 0x7f5d240af920 2023-10-21T17:41:20.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974293252 0 0) 0x7f5d10003520 con 0x7f5d240af920 2023-10-21T17:41:20.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 msgr2=0x7f5d240cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d240cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.756 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 >> v2:172.21.15.124:3300/0 conn(0x7f5d2415ae30 msgr2=0x7f5d240cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 --2- 172.21.15.53:0/3886050481 >> v2:172.21.15.124:3300/0 conn(0x7f5d2415ae30 0x7f5d240cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240cbc60 con 0x7f5d240af920 2023-10-21T17:41:20.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f5d0c001800 con 0x7f5d240af920 2023-10-21T17:41:20.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.751+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 >> v1:172.21.15.53:6789/0 conn(0x7f5d240af920 legacy=0x7f5d240cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.757 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d10003970 0x7f5d10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d10004d00 0x7f5d10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240cde60 con 0x7f5d10004d00 2023-10-21T17:41:20.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240cebf0 con 0x7f5d10003970 2023-10-21T17:41:20.758 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1000ba30 con 0x7f5d240cd470 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d2259c640 1 --2- 172.21.15.53:0/3886050481 >> v2:172.21.15.124:3300/0 conn(0x7f5d2415ae30 0x7f5d240cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d10004d00 0x7f5d10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d2259c640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d10003970 0x7f5d10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d1bfff640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 -- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 msgr2=0x7f5d100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.759 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 -- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d10003970 msgr2=0x7f5d10003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:20.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d10003970 0x7f5d10003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.752+0000 7f5d22d9d640 1 -- 172.21.15.53:0/3886050481 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d10003520 con 0x7f5d10004d00 2023-10-21T17:41:20.760 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.753+0000 7f5d2259c640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d10003970 0x7f5d10003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:20.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.753+0000 7f5d1bfff640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d240cd470 0x7f5d100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:20.761 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.754+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d10004d00 0x7f5d10005180 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009fc0 tx=0x7f5d100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd36d7d26254e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.754+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02cbf0 con 0x7f5d10004d00 2023-10-21T17:41:20.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00f040 con 0x7f5d10004d00 2023-10-21T17:41:20.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d21d9b640 1 -- 172.21.15.53:0/3886050481 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c034440 con 0x7f5d10004d00 2023-10-21T17:41:20.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 -- 172.21.15.53:0/3886050481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d10004d00 msgr2=0x7f5d10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.764 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/3886050481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d10004d00 0x7f5d10005180 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009fc0 tx=0x7f5d100057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d22d9d640 1 -- 172.21.15.53:0/3886050481 reap_dead start 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 -- 172.21.15.53:0/3886050481 shutdown_connections 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 -- 172.21.15.53:0/3886050481 >> 172.21.15.53:0/3886050481 conn(0x7f5d2406e650 msgr2=0x7f5d2406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 -- 172.21.15.53:0/3886050481 shutdown_connections 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.755+0000 7f5d28ba9640 1 -- 172.21.15.53:0/3886050481 wait complete. 2023-10-21T17:41:20.765 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.757+0000 7f5d28ba9640 1 Processor -- start 2023-10-21T17:41:20.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.757+0000 7f5d28ba9640 1 -- start start 2023-10-21T17:41:20.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.758+0000 7f5d28ba9640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d10004d00 0x7f5d240c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.758+0000 7f5d28ba9640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d240c9290 0x7f5d2415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.766 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d28ba9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d28ba9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2415bbe0 con 0x7f5d2415b750 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d28ba9640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240c7cb0 con 0x7f5d240c9290 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d28ba9640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240c7f90 con 0x7f5d10004d00 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54568/0 (socket says 172.21.15.53:54568) 2023-10-21T17:41:20.767 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 -- 172.21.15.53:0/967463514 learned_addr learned my addr 172.21.15.53:0/967463514 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:20.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d2259c640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d10004d00 0x7f5d240c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d1bfff640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d240c9290 0x7f5d2415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 -- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d10004d00 msgr2=0x7f5d240c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d10004d00 0x7f5d240c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.768 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 -- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d240c9290 msgr2=0x7f5d2415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d240c9290 0x7f5d2415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.759+0000 7f5d22d9d640 1 -- 172.21.15.53:0/967463514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240c8270 con 0x7f5d2415b750 2023-10-21T17:41:20.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d22d9d640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c002230 tx=0x7f5d1c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003990 con 0x7f5d2415b750 2023-10-21T17:41:20.769 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c003af0 con 0x7f5d2415b750 2023-10-21T17:41:20.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003ea0 con 0x7f5d2415b750 2023-10-21T17:41:20.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d240c8550 con 0x7f5d2415b750 2023-10-21T17:41:20.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.760+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d241591b0 con 0x7f5d2415b750 2023-10-21T17:41:20.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.762+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d240c8a70 con 0x7f5d2415b750 2023-10-21T17:41:20.770 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.763+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5d1c034900 con 0x7f5d2415b750 2023-10-21T17:41:20.771 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.764+0000 7f5d1affd640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 0x7f5d0406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:20.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.764+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f5d1c0ab560 con 0x7f5d2415b750 2023-10-21T17:41:20.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.764+0000 7f5d2259c640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 0x7f5d0406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:20.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.766+0000 7f5d2259c640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 0x7f5d0406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d14006fd0 tx=0x7f5d14008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:20.772 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.768+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d240c8a70 con 0x7f5d2415b750 2023-10-21T17:41:20.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.959+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d240cee40 con 0x7f5d0406afc0 2023-10-21T17:41:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.960+0000 7f5d1affd640 1 -- 172.21.15.53:0/967463514 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5d1400c040 con 0x7f5d0406afc0 2023-10-21T17:41:20.961 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 3 to deep-scrub 2023-10-21T17:41:20.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 msgr2=0x7f5d0406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 0x7f5d0406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d14006fd0 tx=0x7f5d14008040 comp rx=0 tx=0).stop 2023-10-21T17:41:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 msgr2=0x7f5d240ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c002230 tx=0x7f5d1c009750 comp rx=0 tx=0).stop 2023-10-21T17:41:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 shutdown_connections 2023-10-21T17:41:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f5d0406afc0 0x7f5d0406d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2415b750 0x7f5d240ad040 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d240c9290 0x7f5d2415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 --2- 172.21.15.53:0/967463514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d10004d00 0x7f5d240c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 >> 172.21.15.53:0/967463514 conn(0x7f5d2406e650 msgr2=0x7f5d240b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 shutdown_connections 2023-10-21T17:41:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:20.962+0000 7f5d28ba9640 1 -- 172.21.15.53:0/967463514 wait complete. 2023-10-21T17:41:20.982 INFO:tasks.ceph:Scrubbing osd.4 2023-10-21T17:41:20.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:21.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:21.095 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:21.190 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.189+0000 7f32ea705640 1 Processor -- start 2023-10-21T17:41:21.191 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.189+0000 7f32ea705640 1 -- start start 2023-10-21T17:41:21.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.190+0000 7f32ea705640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32e4070cc0 0x7f32e40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.192 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.190+0000 7f32ea705640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32e40cb450 0x7f32e40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.190+0000 7f32ea705640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e4071170 con 0x7f32e4160660 2023-10-21T17:41:21.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.190+0000 7f32ea705640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e406e150 con 0x7f32e4070cc0 2023-10-21T17:41:21.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.190+0000 7f32ea705640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e406de70 con 0x7f32e40cb450 2023-10-21T17:41:21.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32e4070cc0 0x7f32e40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.193 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f32e4070cc0 0x7f32e40caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54578/0 (socket says 172.21.15.53:54578) 2023-10-21T17:41:21.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 -- 172.21.15.53:0/1271362322 learned_addr learned my addr 172.21.15.53:0/1271362322 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e8ca2640 1 --2- 172.21.15.53:0/1271362322 >> v2:172.21.15.124:3300/0 conn(0x7f32e40cb450 0x7f32e40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 -- 172.21.15.53:0/1271362322 >> v2:172.21.15.124:3300/0 conn(0x7f32e40cb450 msgr2=0x7f32e40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:21.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 --2- 172.21.15.53:0/1271362322 >> v2:172.21.15.124:3300/0 conn(0x7f32e40cb450 0x7f32e40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.194 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 -- 172.21.15.53:0/1271362322 >> v1:172.21.15.53:6789/0 conn(0x7f32e4160660 legacy=0x7f32e40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.195 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.191+0000 7f32e37fe640 1 -- 172.21.15.53:0/1271362322 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40c8e30 con 0x7f32e4070cc0 2023-10-21T17:41:21.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.192+0000 7f32e37fe640 1 --2- 172.21.15.53:0/1271362322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e40caf10 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f32e40b1760 tx=0x7f32d802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfbee3881bc3160 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32e2ffd640 1 -- 172.21.15.53:0/1271362322 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8030870 con 0x7f32e4070cc0 2023-10-21T17:41:21.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32e2ffd640 1 -- 172.21.15.53:0/1271362322 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d80309d0 con 0x7f32e4070cc0 2023-10-21T17:41:21.196 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32e2ffd640 1 -- 172.21.15.53:0/1271362322 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8030d60 con 0x7f32e4070cc0 2023-10-21T17:41:21.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32ea705640 1 -- 172.21.15.53:0/1271362322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 msgr2=0x7f32e40caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32ea705640 1 --2- 172.21.15.53:0/1271362322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e40caf10 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f32e40b1760 tx=0x7f32d802b9b0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.193+0000 7f32ea705640 1 -- 172.21.15.53:0/1271362322 shutdown_connections 2023-10-21T17:41:21.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.194+0000 7f32ea705640 1 --2- 172.21.15.53:0/1271362322 >> v2:172.21.15.124:3300/0 conn(0x7f32e40cb450 0x7f32e40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.197 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.194+0000 7f32ea705640 1 --2- 172.21.15.53:0/1271362322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e40caf10 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.194+0000 7f32ea705640 1 -- 172.21.15.53:0/1271362322 >> 172.21.15.53:0/1271362322 conn(0x7f32e406a5d0 msgr2=0x7f32e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.194+0000 7f32ea705640 1 -- 172.21.15.53:0/1271362322 shutdown_connections 2023-10-21T17:41:21.198 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.194+0000 7f32ea705640 1 -- 172.21.15.53:0/1271362322 wait complete. 2023-10-21T17:41:21.200 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.196+0000 7f32ea705640 1 Processor -- start 2023-10-21T17:41:21.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.196+0000 7f32ea705640 1 -- start start 2023-10-21T17:41:21.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e4152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e40cb450 0x7f32e40c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.201 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32e4160660 0x7f32e415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415e830 con 0x7f32e4070cc0 2023-10-21T17:41:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e41521e0 con 0x7f32e4160660 2023-10-21T17:41:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.197+0000 7f32ea705640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e41524c0 con 0x7f32e40cb450 2023-10-21T17:41:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e4152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.202 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e4152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54594/0 (socket says 172.21.15.53:54594) 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 -- 172.21.15.53:0/2975085968 learned_addr learned my addr 172.21.15.53:0/2975085968 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e8ca2640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32e4160660 0x7f32e415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e37fe640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e40cb450 0x7f32e40c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 -- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e40cb450 msgr2=0x7f32e40c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e40cb450 0x7f32e40c12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.203 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 -- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32e4160660 msgr2=0x7f32e415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32e4160660 0x7f32e415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80094d0 con 0x7f32e4070cc0 2023-10-21T17:41:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e37fe640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e40cb450 0x7f32e40c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e8ca2640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32e4160660 0x7f32e415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:21.204 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.198+0000 7f32e3fff640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e4152fa0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c400 tx=0x7f32d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.199+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d40052d0 con 0x7f32e4070cc0 2023-10-21T17:41:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.199+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4005430 con 0x7f32e4070cc0 2023-10-21T17:41:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.199+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4003830 con 0x7f32e4070cc0 2023-10-21T17:41:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.199+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e41527a0 con 0x7f32e4070cc0 2023-10-21T17:41:21.205 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.199+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e40c1820 con 0x7f32e4070cc0 2023-10-21T17:41:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.200+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32e40c93f0 con 0x7f32e4070cc0 2023-10-21T17:41:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.201+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f32d4003990 con 0x7f32e4070cc0 2023-10-21T17:41:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e0ff9640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f32d006af90 0x7f32d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f32d4088dd0 con 0x7f32e4070cc0 2023-10-21T17:41:21.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e0ff9640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f32d00719e0 0x7f32d0073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32d0074510 con 0x7f32d00719e0 2023-10-21T17:41:21.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32d408c020 con 0x7f32e4070cc0 2023-10-21T17:41:21.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.202+0000 7f32e37fe640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f32d006af90 0x7f32d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.203+0000 7f32e8ca2640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f32d00719e0 0x7f32d0073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.203+0000 7f32e8ca2640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f32d00719e0 0x7f32d0073e40 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.204+0000 7f32e37fe640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f32d006af90 0x7f32d006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036000 tx=0x7f32d8008030 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.205+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== osd.4 v2:172.21.15.124:6800/1000673860 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f32d0074510 con 0x7f32d00719e0 2023-10-21T17:41:21.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.219+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 --> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f32e40c9110 con 0x7f32d00719e0 2023-10-21T17:41:21.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.220+0000 7f32e0ff9640 1 -- 172.21.15.53:0/2975085968 <== osd.4 v2:172.21.15.124:6800/1000673860 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f32dc007480 con 0x7f32d00719e0 2023-10-21T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:21.222 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:21.223 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f32d00719e0 msgr2=0x7f32d0073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6800/1000673860,v1:172.21.15.124:6801/1000673860] conn(0x7f32d00719e0 0x7f32d0073e40 crc :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f32d006af90 msgr2=0x7f32d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f32d006af90 0x7f32d006d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036000 tx=0x7f32d8008030 comp rx=0 tx=0).stop 2023-10-21T17:41:21.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 msgr2=0x7f32e4152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 --2- 172.21.15.53:0/2975085968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32e4070cc0 0x7f32e4152fa0 secure :-1 s=READY pgs=2963 cs=0 l=1 rev1=1 crypto rx=0x7f32d400c400 tx=0x7f32d400c8c0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32e8ca2640 1 -- 172.21.15.53:0/2975085968 reap_dead start 2023-10-21T17:41:21.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 shutdown_connections 2023-10-21T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 >> 172.21.15.53:0/2975085968 conn(0x7f32e406a5d0 msgr2=0x7f32e40ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 shutdown_connections 2023-10-21T17:41:21.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.221+0000 7f32ea705640 1 -- 172.21.15.53:0/2975085968 wait complete. 2023-10-21T17:41:21.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-21T17:41:21.489 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.488+0000 7f93e962e640 1 Processor -- start 2023-10-21T17:41:21.490 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.488+0000 7f93e962e640 1 -- start start 2023-10-21T17:41:21.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e962e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93e415cf30 0x7f93e40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e962e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40cd520 con 0x7f93e40cf2b0 2023-10-21T17:41:21.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e962e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40d1910 con 0x7f93e40cf690 2023-10-21T17:41:21.491 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e962e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40ce080 con 0x7f93e415cf30 2023-10-21T17:41:21.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e8e2d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93e415cf30 0x7f93e40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.492 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e8e2d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f93e415cf30 0x7f93e40d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54610/0 (socket says 172.21.15.53:54610) 2023-10-21T17:41:21.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.490+0000 7f93e8e2d640 1 -- 172.21.15.53:0/3223181359 learned_addr learned my addr 172.21.15.53:0/3223181359 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.491+0000 7f93e8e2d640 1 -- 172.21.15.53:0/3223181359 >> v1:172.21.15.124:6789/0 conn(0x7f93e40cf690 legacy=0x7f93e415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:21.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.491+0000 7f93e8e2d640 1 -- 172.21.15.53:0/3223181359 >> v1:172.21.15.53:6789/0 conn(0x7f93e40cf2b0 legacy=0x7f93e40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.493 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.491+0000 7f93e8e2d640 1 -- 172.21.15.53:0/3223181359 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40cd750 con 0x7f93e415cf30 2023-10-21T17:41:21.494 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.492+0000 7f93e8e2d640 1 --2- 172.21.15.53:0/3223181359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e415cf30 0x7f93e40d34d0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009e50 tx=0x7f93dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cc9306b4612410 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.492+0000 7f93e37fe640 1 -- 172.21.15.53:0/3223181359 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc02ca80 con 0x7f93e415cf30 2023-10-21T17:41:21.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.492+0000 7f93e37fe640 1 -- 172.21.15.53:0/3223181359 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc02cbe0 con 0x7f93e415cf30 2023-10-21T17:41:21.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.492+0000 7f93e37fe640 1 -- 172.21.15.53:0/3223181359 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0343f0 con 0x7f93e415cf30 2023-10-21T17:41:21.495 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 -- 172.21.15.53:0/3223181359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e415cf30 msgr2=0x7f93e40d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 --2- 172.21.15.53:0/3223181359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e415cf30 0x7f93e40d34d0 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009e50 tx=0x7f93dc02bee0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 -- 172.21.15.53:0/3223181359 shutdown_connections 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 --2- 172.21.15.53:0/3223181359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e415cf30 0x7f93e40d34d0 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 -- 172.21.15.53:0/3223181359 >> 172.21.15.53:0/3223181359 conn(0x7f93e409bc40 msgr2=0x7f93e40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 -- 172.21.15.53:0/3223181359 shutdown_connections 2023-10-21T17:41:21.496 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.493+0000 7f93e962e640 1 -- 172.21.15.53:0/3223181359 wait complete. 2023-10-21T17:41:21.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.495+0000 7f93e962e640 1 Processor -- start 2023-10-21T17:41:21.498 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.495+0000 7f93e962e640 1 -- start start 2023-10-21T17:41:21.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93e40cf2b0 0x7f93e40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93e415cf30 0x7f93e40b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b74c0 con 0x7f93e40cf690 2023-10-21T17:41:21.499 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b71e0 con 0x7f93e40cf2b0 2023-10-21T17:41:21.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93e962e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b56d0 con 0x7f93e415cf30 2023-10-21T17:41:21.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54624/0 (socket says 172.21.15.53:54624) 2023-10-21T17:41:21.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.496+0000 7f93dbfff640 1 -- 172.21.15.53:0/4136354647 learned_addr learned my addr 172.21.15.53:0/4136354647 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.500 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93e3fff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93e40cf2b0 0x7f93e40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 -- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93e415cf30 msgr2=0x7f93e40b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93e415cf30 0x7f93e40b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 -- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93e40cf2b0 msgr2=0x7f93e40b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93e40cf2b0 0x7f93e40b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 -- 172.21.15.53:0/4136354647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc009b30 con 0x7f93e40cf690 2023-10-21T17:41:21.501 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93dbfff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c430 tx=0x7f93cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc011790 con 0x7f93e40cf690 2023-10-21T17:41:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc0118f0 con 0x7f93e40cf690 2023-10-21T17:41:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc011ce0 con 0x7f93e40cf690 2023-10-21T17:41:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.497+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e40b59b0 con 0x7f93e40cf690 2023-10-21T17:41:21.503 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.498+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e415a710 con 0x7f93e40cf690 2023-10-21T17:41:21.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.499+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e40b5530 con 0x7f93e40cf690 2023-10-21T17:41:21.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.500+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f93cc01c460 con 0x7f93e40cf690 2023-10-21T17:41:21.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.501+0000 7f93e17fa640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 0x7f93c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.504 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.501+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f93cc089980 con 0x7f93e40cf690 2023-10-21T17:41:21.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.502+0000 7f93e3fff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 0x7f93c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.505 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.503+0000 7f93e3fff640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 0x7f93c406d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000f60 tx=0x7f93d4009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.506 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.505+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93e40b5530 con 0x7f93e40cf690 2023-10-21T17:41:21.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.696+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f93e40b5840 con 0x7f93c406afc0 2023-10-21T17:41:21.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.697+0000 7f93e17fa640 1 -- 172.21.15.53:0/4136354647 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f93e40b5840 con 0x7f93c406afc0 2023-10-21T17:41:21.699 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 4 to deep-scrub 2023-10-21T17:41:21.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 msgr2=0x7f93c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 0x7f93c406d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000f60 tx=0x7f93d4009040 comp rx=0 tx=0).stop 2023-10-21T17:41:21.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 msgr2=0x7f93e415a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00c430 tx=0x7f93cc00c8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 shutdown_connections 2023-10-21T17:41:21.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f93c406afc0 0x7f93c406d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93e415cf30 0x7f93e40b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e40cf690 0x7f93e415a1d0 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 --2- 172.21.15.53:0/4136354647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93e40cf2b0 0x7f93e40b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 >> 172.21.15.53:0/4136354647 conn(0x7f93e409bc40 msgr2=0x7f93e415fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 shutdown_connections 2023-10-21T17:41:21.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.700+0000 7f93e962e640 1 -- 172.21.15.53:0/4136354647 wait complete. 2023-10-21T17:41:21.720 INFO:tasks.ceph:Scrubbing osd.5 2023-10-21T17:41:21.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:21.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.937+0000 7fcbd8d6c640 1 Processor -- start 2023-10-21T17:41:21.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.937+0000 7fcbd8d6c640 1 -- start start 2023-10-21T17:41:21.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcbd41599d0 0x7fcbd40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40b3c50 0x7fcbd40caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcbd40cb410 0x7fcbd40c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40cb8a0 con 0x7fcbd41599d0 2023-10-21T17:41:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd4159db0 con 0x7fcbd40cb410 2023-10-21T17:41:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.938+0000 7fcbd8d6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4071fe0 con 0x7fcbd40b3c50 2023-10-21T17:41:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcbd41599d0 0x7fcbd40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcbd41599d0 0x7fcbd40cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54630/0 (socket says 172.21.15.53:54630) 2023-10-21T17:41:21.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 -- 172.21.15.53:0/3477054441 learned_addr learned my addr 172.21.15.53:0/3477054441 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 -- 172.21.15.53:0/3477054441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40b3c50 msgr2=0x7fcbd40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:21.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/3477054441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40b3c50 0x7fcbd40caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 -- 172.21.15.53:0/3477054441 >> v2:172.21.15.124:3300/0 conn(0x7fcbd40cb410 msgr2=0x7fcbd40c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:21.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/3477054441 >> v2:172.21.15.124:3300/0 conn(0x7fcbd40cb410 0x7fcbd40c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.943 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.939+0000 7fcbd37fe640 1 -- 172.21.15.53:0/3477054441 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40b44c0 con 0x7fcbd41599d0 2023-10-21T17:41:21.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.940+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/3477054441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd40cdcd0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40cbf80 tx=0x7fcbc402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bd77af07069258 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd2ffd640 1 -- 172.21.15.53:0/3477054441 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4031880 con 0x7fcbd41599d0 2023-10-21T17:41:21.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd2ffd640 1 -- 172.21.15.53:0/3477054441 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc40319e0 con 0x7fcbd41599d0 2023-10-21T17:41:21.944 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd2ffd640 1 -- 172.21.15.53:0/3477054441 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4031d70 con 0x7fcbd41599d0 2023-10-21T17:41:21.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/3477054441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 msgr2=0x7fcbd40cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/3477054441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd40cdcd0 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40cbf80 tx=0x7fcbc402b920 comp rx=0 tx=0).stop 2023-10-21T17:41:21.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/3477054441 shutdown_connections 2023-10-21T17:41:21.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/3477054441 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40b3c50 0x7fcbd40caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.945 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/3477054441 >> v2:172.21.15.124:3300/0 conn(0x7fcbd40cb410 0x7fcbd40c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/3477054441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd40cdcd0 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.941+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/3477054441 >> 172.21.15.53:0/3477054441 conn(0x7fcbd406c050 msgr2=0x7fcbd406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.942+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/3477054441 shutdown_connections 2023-10-21T17:41:21.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.942+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/3477054441 wait complete. 2023-10-21T17:41:21.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.943+0000 7fcbd8d6c640 1 Processor -- start 2023-10-21T17:41:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.944+0000 7fcbd8d6c640 1 -- start start 2023-10-21T17:41:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.944+0000 7fcbd8d6c640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbd40b3c50 0x7fcbd41479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd8d6c640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40cb410 0x7fcbd40c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd8d6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd8d6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40c9d00 con 0x7fcbd41599d0 2023-10-21T17:41:21.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd8d6c640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd414def0 con 0x7fcbd40b3c50 2023-10-21T17:41:21.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd8d6c640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4147ef0 con 0x7fcbd40cb410 2023-10-21T17:41:21.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54632/0 (socket says 172.21.15.53:54632) 2023-10-21T17:41:21.950 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 -- 172.21.15.53:0/1916068447 learned_addr learned my addr 172.21.15.53:0/1916068447 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbd40b3c50 0x7fcbd41479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 -- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40cb410 msgr2=0x7fcbd40c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbd40cb410 0x7fcbd40c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 -- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbd40b3c50 msgr2=0x7fcbd41479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.951 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbd40b3c50 0x7fcbd41479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.945+0000 7fcbd3fff640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc40094d0 con 0x7fcbd41599d0 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd3fff640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd415c280 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b970 tx=0x7fcbcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00cb40 con 0x7fcbd41599d0 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00ccc0 con 0x7fcbd41599d0 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc014680 con 0x7fcbd41599d0 2023-10-21T17:41:21.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40c8c60 con 0x7fcbd41599d0 2023-10-21T17:41:21.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.946+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40c9220 con 0x7fcbd41599d0 2023-10-21T17:41:21.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.947+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcbd40716b0 con 0x7fcbd41599d0 2023-10-21T17:41:21.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.948+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcbcc01d3b0 con 0x7fcbd41599d0 2023-10-21T17:41:21.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.949+0000 7fcbd0ff9640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcbb406af90 0x7fcbb406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.949+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fcbcc089900 con 0x7fcbd41599d0 2023-10-21T17:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.949+0000 7fcbd0ff9640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7fcbb40719e0 0x7fcbb4073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.949+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbb4074510 con 0x7fcbb40719e0 2023-10-21T17:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.949+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbcc08e2a0 con 0x7fcbd41599d0 2023-10-21T17:41:21.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.950+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcbb406af90 0x7fcbb406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.950+0000 7fcbcbfff640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7fcbb40719e0 0x7fcbb4073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.951+0000 7fcbd37fe640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcbb406af90 0x7fcbb406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4036000 tx=0x7fcbc403c040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.951+0000 7fcbcbfff640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7fcbb40719e0 0x7fcbb4073e40 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:21.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.953+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== osd.5 v2:172.21.15.124:6808/3990975910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fcbb4074510 con 0x7fcbb40719e0 2023-10-21T17:41:21.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.967+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 --> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcbd40713d0 con 0x7fcbb40719e0 2023-10-21T17:41:21.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.968+0000 7fcbd0ff9640 1 -- 172.21.15.53:0/1916068447 <== osd.5 v2:172.21.15.124:6808/3990975910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcbbc009840 con 0x7fcbb40719e0 2023-10-21T17:41:21.970 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:21.970 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:21.970 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7fcbb40719e0 msgr2=0x7fcbb4073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6808/3990975910,v1:172.21.15.124:6809/3990975910] conn(0x7fcbb40719e0 0x7fcbb4073e40 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:21.971 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcbb406af90 msgr2=0x7fcbb406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fcbb406af90 0x7fcbb406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4036000 tx=0x7fcbc403c040 comp rx=0 tx=0).stop 2023-10-21T17:41:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 msgr2=0x7fcbd415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 --2- 172.21.15.53:0/1916068447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcbd41599d0 0x7fcbd415c280 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b970 tx=0x7fcbcc00be30 comp rx=0 tx=0).stop 2023-10-21T17:41:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd3fff640 1 -- 172.21.15.53:0/1916068447 reap_dead start 2023-10-21T17:41:21.972 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 shutdown_connections 2023-10-21T17:41:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 >> 172.21.15.53:0/1916068447 conn(0x7fcbd406c050 msgr2=0x7fcbd40b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 shutdown_connections 2023-10-21T17:41:21.973 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:21.969+0000 7fcbd8d6c640 1 -- 172.21.15.53:0/1916068447 wait complete. 2023-10-21T17:41:21.987 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-21T17:41:22.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.209+0000 7f825896b640 1 Processor -- start 2023-10-21T17:41:22.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.209+0000 7f825896b640 1 -- start start 2023-10-21T17:41:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f825896b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82540c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f825896b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8254161590 con 0x7f82540aff70 2023-10-21T17:41:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f825896b640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82540cc570 con 0x7f8254164d10 2023-10-21T17:41:22.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f825896b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c94b0 con 0x7f82540c18d0 2023-10-21T17:41:22.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f8252d9d640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f82540aff70 0x7f82540b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53178/0 (socket says 172.21.15.53:53178) 2023-10-21T17:41:22.213 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.211+0000 7f8252d9d640 1 -- 172.21.15.53:0/236336376 learned_addr learned my addr 172.21.15.53:0/236336376 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:22.215 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365907424 0 0) 0x7f8254161590 con 0x7f82540aff70 2023-10-21T17:41:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f824a59c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82540c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8240003520 con 0x7f82540aff70 2023-10-21T17:41:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212809629 0 0) 0x7f8240003520 con 0x7f82540aff70 2023-10-21T17:41:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 msgr2=0x7f82540c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82540c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 >> v1:172.21.15.124:6789/0 conn(0x7f8254164d10 legacy=0x7f82540ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82540c9790 con 0x7f82540aff70 2023-10-21T17:41:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f82440019f0 con 0x7f82540aff70 2023-10-21T17:41:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 >> v1:172.21.15.53:6789/0 conn(0x7f82540aff70 legacy=0x7f82540b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.217 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8240004000 0x7f82400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f82400051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c94b0 con 0x7f8240004d70 2023-10-21T17:41:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824000bc80 con 0x7f8240004000 2023-10-21T17:41:22.218 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824000be40 con 0x7f82540c18d0 2023-10-21T17:41:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.212+0000 7f825359e640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f82400051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f8252d9d640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f824a59c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8240004000 0x7f82400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f825359e640 1 -- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 msgr2=0x7f82400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.219 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f825359e640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c18d0 0x7f82400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f825359e640 1 -- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8240004000 msgr2=0x7f82400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f825359e640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8240004000 0x7f82400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f825359e640 1 -- 172.21.15.53:0/236336376 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240003520 con 0x7f8240004d70 2023-10-21T17:41:22.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.213+0000 7f824a59c640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8240004000 0x7f82400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:22.220 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.214+0000 7f825359e640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f82400051f0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f824c003d80 tx=0x7f82400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c959be1020367fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.214+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c032600 con 0x7f8240004d70 2023-10-21T17:41:22.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.214+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c02e050 con 0x7f8240004d70 2023-10-21T17:41:22.221 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.214+0000 7f825259c640 1 -- 172.21.15.53:0/236336376 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0329e0 con 0x7f8240004d70 2023-10-21T17:41:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 -- 172.21.15.53:0/236336376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 msgr2=0x7f82400051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 --2- 172.21.15.53:0/236336376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f82400051f0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f824c003d80 tx=0x7f82400057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825359e640 1 -- 172.21.15.53:0/236336376 reap_dead start 2023-10-21T17:41:22.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 -- 172.21.15.53:0/236336376 shutdown_connections 2023-10-21T17:41:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 -- 172.21.15.53:0/236336376 >> 172.21.15.53:0/236336376 conn(0x7f825406c050 msgr2=0x7f825406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 -- 172.21.15.53:0/236336376 shutdown_connections 2023-10-21T17:41:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.215+0000 7f825896b640 1 -- 172.21.15.53:0/236336376 wait complete. 2023-10-21T17:41:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.217+0000 7f825896b640 1 Processor -- start 2023-10-21T17:41:22.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.217+0000 7f825896b640 1 -- start start 2023-10-21T17:41:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825415f7c0 0x7f825415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c6b80 0x7f82540c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c8410 con 0x7f8240004d70 2023-10-21T17:41:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8254160180 con 0x7f825415f7c0 2023-10-21T17:41:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f825896b640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82540c7550 con 0x7f82540c6b80 2023-10-21T17:41:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f8252d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f8252d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54640/0 (socket says 172.21.15.53:54640) 2023-10-21T17:41:22.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.218+0000 7f8252d9d640 1 -- 172.21.15.53:0/2610372902 learned_addr learned my addr 172.21.15.53:0/2610372902 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 -- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c6b80 msgr2=0x7f82540c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c6b80 0x7f82540c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 -- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825415f7c0 msgr2=0x7f825415fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825415f7c0 0x7f825415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 -- 172.21.15.53:0/2610372902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82541622a0 con 0x7f8240004d70 2023-10-21T17:41:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f8252d9d640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f8244007a10 tx=0x7f824400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824400edd0 con 0x7f8240004d70 2023-10-21T17:41:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244013420 con 0x7f8240004d70 2023-10-21T17:41:22.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82440137b0 con 0x7f8240004d70 2023-10-21T17:41:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825415be70 con 0x7f8240004d70 2023-10-21T17:41:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.219+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82540c7a70 con 0x7f8240004d70 2023-10-21T17:41:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.221+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82540c1fe0 con 0x7f8240004d70 2023-10-21T17:41:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.222+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8244013910 con 0x7f8240004d70 2023-10-21T17:41:22.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.222+0000 7f824bfff640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 0x7f823406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.223+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f8244089fa0 con 0x7f8240004d70 2023-10-21T17:41:22.231 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.223+0000 7f824a59c640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 0x7f823406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.224+0000 7f824a59c640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 0x7f823406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f823c006fd0 tx=0x7f823c009350 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.232 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.227+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82440bcca0 con 0x7f8240004d70 2023-10-21T17:41:22.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.416+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f82540cc810 con 0x7f823406af90 2023-10-21T17:41:22.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.417+0000 7f824bfff640 1 -- 172.21.15.53:0/2610372902 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f823c00e540 con 0x7f823406af90 2023-10-21T17:41:22.419 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 5 to deep-scrub 2023-10-21T17:41:22.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 msgr2=0x7f823406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 0x7f823406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f823c006fd0 tx=0x7f823c009350 comp rx=0 tx=0).stop 2023-10-21T17:41:22.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 msgr2=0x7f825415b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f8244007a10 tx=0x7f824400e530 comp rx=0 tx=0).stop 2023-10-21T17:41:22.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 shutdown_connections 2023-10-21T17:41:22.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f823406af90 0x7f823406d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82540c6b80 0x7f82540c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f825415f7c0 0x7f825415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 --2- 172.21.15.53:0/2610372902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8240004d70 0x7f825415b930 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 >> 172.21.15.53:0/2610372902 conn(0x7f825406c050 msgr2=0x7f8254070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 shutdown_connections 2023-10-21T17:41:22.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.420+0000 7f825896b640 1 -- 172.21.15.53:0/2610372902 wait complete. 2023-10-21T17:41:22.440 INFO:tasks.ceph:Scrubbing osd.6 2023-10-21T17:41:22.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:22.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.667+0000 7fc251259640 1 Processor -- start 2023-10-21T17:41:22.668 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.667+0000 7fc251259640 1 -- start start 2023-10-21T17:41:22.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.668+0000 7fc251259640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.668+0000 7fc251259640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0ce510 0x7fc24c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.670 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.668+0000 7fc251259640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c0cd520 con 0x7fc24c0cf690 2023-10-21T17:41:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.668+0000 7fc251259640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0d1910 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.668+0000 7fc251259640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0ce080 con 0x7fc24c0ce510 2023-10-21T17:41:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54656/0 (socket says 172.21.15.53:54656) 2023-10-21T17:41:22.671 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 -- 172.21.15.53:0/2954503500 learned_addr learned my addr 172.21.15.53:0/2954503500 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:22.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 -- 172.21.15.53:0/2954503500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0ce510 msgr2=0x7fc24c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 --2- 172.21.15.53:0/2954503500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0ce510 0x7fc24c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 -- 172.21.15.53:0/2954503500 >> v1:172.21.15.124:6789/0 conn(0x7fc24c0cf690 legacy=0x7fc24c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.673 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.669+0000 7fc24bfff640 1 -- 172.21.15.53:0/2954503500 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0cd750 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.670+0000 7fc24bfff640 1 --2- 172.21.15.53:0/2954503500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd82f6956c30d03 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.674 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.671+0000 7fc24b7fe640 1 -- 172.21.15.53:0/2954503500 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c02c9e0 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.671+0000 7fc24b7fe640 1 -- 172.21.15.53:0/2954503500 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c02cb40 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.671+0000 7fc24b7fe640 1 -- 172.21.15.53:0/2954503500 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0343f0 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.675 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.671+0000 7fc251259640 1 -- 172.21.15.53:0/2954503500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0cf2b0 msgr2=0x7fc24c0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.671+0000 7fc251259640 1 --2- 172.21.15.53:0/2954503500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c02be40 comp rx=0 tx=0).stop 2023-10-21T17:41:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 -- 172.21.15.53:0/2954503500 shutdown_connections 2023-10-21T17:41:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 --2- 172.21.15.53:0/2954503500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0ce510 0x7fc24c15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 --2- 172.21.15.53:0/2954503500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0cf2b0 0x7fc24c0d34d0 unknown :-1 s=CLOSED pgs=2972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.676 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 -- 172.21.15.53:0/2954503500 >> 172.21.15.53:0/2954503500 conn(0x7fc24c09bc40 msgr2=0x7fc24c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 -- 172.21.15.53:0/2954503500 shutdown_connections 2023-10-21T17:41:22.677 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.672+0000 7fc251259640 1 -- 172.21.15.53:0/2954503500 wait complete. 2023-10-21T17:41:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.673+0000 7fc251259640 1 Processor -- start 2023-10-21T17:41:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.674+0000 7fc251259640 1 -- start start 2023-10-21T17:41:22.679 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 0x7fc24c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc24c0cf2b0 0x7fc24c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0cf690 0x7fc24c09f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0cc520 con 0x7fc24c0ce510 2023-10-21T17:41:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0cb480 con 0x7fc24c0cf2b0 2023-10-21T17:41:22.680 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc251259640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0cb760 con 0x7fc24c0cf690 2023-10-21T17:41:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 0x7fc24c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 0x7fc24c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54666/0 (socket says 172.21.15.53:54666) 2023-10-21T17:41:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 -- 172.21.15.53:0/2049060607 learned_addr learned my addr 172.21.15.53:0/2049060607 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 -- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0cf690 msgr2=0x7fc24c09f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.681 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc24c0cf690 0x7fc24c09f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 -- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc24c0cf2b0 msgr2=0x7fc24c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc24c0cf2b0 0x7fc24c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.675+0000 7fc24bfff640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009b30 con 0x7fc24c0ce510 2023-10-21T17:41:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc24bfff640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 0x7fc24c0b44d0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.682 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c03e450 con 0x7fc24c0ce510 2023-10-21T17:41:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c03e5b0 con 0x7fc24c0ce510 2023-10-21T17:41:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c0cba40 con 0x7fc24c0ce510 2023-10-21T17:41:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c03e960 con 0x7fc24c0ce510 2023-10-21T17:41:22.683 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.676+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c0cc000 con 0x7fc24c0ce510 2023-10-21T17:41:22.684 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.677+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc24c0cdd10 con 0x7fc24c0ce510 2023-10-21T17:41:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.678+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc23c03eac0 con 0x7fc24c0ce510 2023-10-21T17:41:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.679+0000 7fc2497fa640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc22c06af90 0x7fc22c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.679+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fc23c0ac920 con 0x7fc24c0ce510 2023-10-21T17:41:22.685 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.680+0000 7fc2497fa640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7fc22c0719e0 0x7fc22c073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.680+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc22c074510 con 0x7fc22c0719e0 2023-10-21T17:41:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.680+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc23c0b12a0 con 0x7fc24c0ce510 2023-10-21T17:41:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.680+0000 7fc2437fe640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc22c06af90 0x7fc22c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.680+0000 7fc250a58640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7fc22c0719e0 0x7fc22c073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.686 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.681+0000 7fc2437fe640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc22c06af90 0x7fc22c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0b5230 tx=0x7fc234006ba0 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.682+0000 7fc250a58640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7fc22c0719e0 0x7fc22c073e40 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.687 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.685+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== osd.6 v2:172.21.15.124:6810/3032138525 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc22c074510 con 0x7fc22c0719e0 2023-10-21T17:41:22.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.699+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 --> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc24c0cda30 con 0x7fc22c0719e0 2023-10-21T17:41:22.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.700+0000 7fc2497fa640 1 -- 172.21.15.53:0/2049060607 <== osd.6 v2:172.21.15.124:6810/3032138525 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc2440097f0 con 0x7fc22c0719e0 2023-10-21T17:41:22.701 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:22.702 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:22.702 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.700+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7fc22c0719e0 msgr2=0x7fc22c073e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.700+0000 7fc251259640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6810/3032138525,v1:172.21.15.124:6812/3032138525] conn(0x7fc22c0719e0 0x7fc22c073e40 crc :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc22c06af90 msgr2=0x7fc22c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fc22c06af90 0x7fc22c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0b5230 tx=0x7fc234006ba0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 msgr2=0x7fc24c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 --2- 172.21.15.53:0/2049060607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc24c0ce510 0x7fc24c0b44d0 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c004a40 comp rx=0 tx=0).stop 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc250a58640 1 -- 172.21.15.53:0/2049060607 reap_dead start 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 shutdown_connections 2023-10-21T17:41:22.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 >> 172.21.15.53:0/2049060607 conn(0x7fc24c09bc40 msgr2=0x7fc24c15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:22.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 shutdown_connections 2023-10-21T17:41:22.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.701+0000 7fc251259640 1 -- 172.21.15.53:0/2049060607 wait complete. 2023-10-21T17:41:22.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-21T17:41:22.946 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.945+0000 7f1defd1d640 1 Processor -- start 2023-10-21T17:41:22.947 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.945+0000 7f1defd1d640 1 -- start start 2023-10-21T17:41:22.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1defd1d640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1de806dfc0 0x7f1de8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.948 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1defd1d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de80b1cc0 con 0x7f1de80afbe0 2023-10-21T17:41:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1defd1d640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de80b2620 con 0x7f1de8071070 2023-10-21T17:41:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1defd1d640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de80b2060 con 0x7f1de806dfc0 2023-10-21T17:41:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1dedab9640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1de80afbe0 0x7f1de80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53180/0 (socket says 172.21.15.53:53180) 2023-10-21T17:41:22.949 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.947+0000 7f1dedab9640 1 -- 172.21.15.53:0/246508365 learned_addr learned my addr 172.21.15.53:0/246508365 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:22.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/246508365 >> v2:172.21.15.124:3300/0 conn(0x7f1de806dfc0 0x7f1de8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272389898 0 0) 0x7f1de80b1cc0 con 0x7f1de80afbe0 2023-10-21T17:41:22.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd4003520 con 0x7f1de80afbe0 2023-10-21T17:41:22.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3481670858 0 0) 0x7f1dd4003520 con 0x7f1de80afbe0 2023-10-21T17:41:22.952 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 >> v2:172.21.15.124:3300/0 conn(0x7f1de806dfc0 msgr2=0x7f1de8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 --2- 172.21.15.53:0/246508365 >> v2:172.21.15.124:3300/0 conn(0x7f1de806dfc0 0x7f1de8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 >> v1:172.21.15.124:6789/0 conn(0x7f1de8071070 legacy=0x7f1de80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80b2340 con 0x7f1de80afbe0 2023-10-21T17:41:22.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.948+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f1de4001670 con 0x7f1de80afbe0 2023-10-21T17:41:22.953 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 >> v1:172.21.15.53:6789/0 conn(0x7f1de80afbe0 legacy=0x7f1de80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd4003970 0x7f1dd4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd40047c0 0x7f1dd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1dd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80b2060 con 0x7f1dd4005180 2023-10-21T17:41:22.954 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd400c050 con 0x7f1dd40047c0 2023-10-21T17:41:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd400c210 con 0x7f1dd4003970 2023-10-21T17:41:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1dee2ba640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1dd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd40047c0 0x7f1dd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.949+0000 7f1dedab9640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd4003970 0x7f1dd4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.955 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dee2ba640 1 -- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd4003970 msgr2=0x7f1dd4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dee2ba640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd4003970 0x7f1dd4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dee2ba640 1 -- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd40047c0 msgr2=0x7f1dd4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dee2ba640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd40047c0 0x7f1dd4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dee2ba640 1 -- 172.21.15.53:0/246508365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4003520 con 0x7f1dd4005180 2023-10-21T17:41:22.956 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.950+0000 7f1dedab9640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd4003970 0x7f1dd4003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:22.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.951+0000 7f1dee2ba640 1 -- 172.21.15.53:0/246508365 reap_dead start 2023-10-21T17:41:22.957 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.951+0000 7f1dee2ba640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1dd400b980 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f1de8071450 tx=0x7f1dd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4253f166234529 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.951+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0003b70 con 0x7f1dd4005180 2023-10-21T17:41:22.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.951+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0003cd0 con 0x7f1dd4005180 2023-10-21T17:41:22.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.951+0000 7f1decab7640 1 -- 172.21.15.53:0/246508365 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0005310 con 0x7f1dd4005180 2023-10-21T17:41:22.960 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 -- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 msgr2=0x7f1dd400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1dd400b980 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 crypto rx=0x7f1de8071450 tx=0x7f1dd40057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 -- 172.21.15.53:0/246508365 shutdown_connections 2023-10-21T17:41:22.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 --2- 172.21.15.53:0/246508365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1dd400b980 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 -- 172.21.15.53:0/246508365 >> 172.21.15.53:0/246508365 conn(0x7f1de806a5d0 msgr2=0x7f1de80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:22.961 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 -- 172.21.15.53:0/246508365 shutdown_connections 2023-10-21T17:41:22.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.952+0000 7f1defd1d640 1 -- 172.21.15.53:0/246508365 wait complete. 2023-10-21T17:41:22.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.954+0000 7f1defd1d640 1 Processor -- start 2023-10-21T17:41:22.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.954+0000 7f1defd1d640 1 -- start start 2023-10-21T17:41:22.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.962 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 0x7f1de81525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1de8152b30 0x7f1de81f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de8152fb0 con 0x7f1dd4005180 2023-10-21T17:41:22.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de81501a0 con 0x7f1de815d8b0 2023-10-21T17:41:22.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1defd1d640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de81f3600 con 0x7f1de8152b30 2023-10-21T17:41:22.963 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1dedab9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1dedab9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54678/0 (socket says 172.21.15.53:54678) 2023-10-21T17:41:22.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.955+0000 7f1dedab9640 1 -- 172.21.15.53:0/4200917140 learned_addr learned my addr 172.21.15.53:0/4200917140 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:22.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 -- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1de8152b30 msgr2=0x7f1de81f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.964 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1de8152b30 0x7f1de81f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 -- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 msgr2=0x7f1de81525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:22.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 0x7f1de81525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 0x7f1de81525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:22.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 -- 172.21.15.53:0/4200917140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de815caf0 con 0x7f1dd4005180 2023-10-21T17:41:22.965 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 0x7f1de81525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:22.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dedab9640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f1de400e730 tx=0x7f1de400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400ee40 con 0x7f1dd4005180 2023-10-21T17:41:22.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de400c8d0 con 0x7f1dd4005180 2023-10-21T17:41:22.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de400cc40 con 0x7f1dd4005180 2023-10-21T17:41:22.966 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de815cdd0 con 0x7f1dd4005180 2023-10-21T17:41:22.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.956+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de815d390 con 0x7f1dd4005180 2023-10-21T17:41:22.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.957+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de804e160 con 0x7f1dd4005180 2023-10-21T17:41:22.967 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.959+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1de401d460 con 0x7f1dd4005180 2023-10-21T17:41:22.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.960+0000 7f1dde7fc640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 0x7f1dcc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:22.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.960+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f1de4089850 con 0x7f1dd4005180 2023-10-21T17:41:22.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.960+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 0x7f1dcc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:22.968 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.961+0000 7f1ded2b8640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 0x7f1dcc06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8006b10 tx=0x7f1dd8008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:22.969 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:22.964+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1de4058150 con 0x7f1dd4005180 2023-10-21T17:41:23.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.156+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de8070630 con 0x7f1dcc06af90 2023-10-21T17:41:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.157+0000 7f1dde7fc640 1 -- 172.21.15.53:0/4200917140 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1dd80096f0 con 0x7f1dcc06af90 2023-10-21T17:41:23.158 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 6 to deep-scrub 2023-10-21T17:41:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.159+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 msgr2=0x7f1dcc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.159+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 0x7f1dcc06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8006b10 tx=0x7f1dd8008040 comp rx=0 tx=0).stop 2023-10-21T17:41:23.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.159+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 msgr2=0x7f1de814fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.159+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7f1de400e730 tx=0x7f1de400ebf0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 shutdown_connections 2023-10-21T17:41:23.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f1dcc06af90 0x7f1dcc06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1de8152b30 0x7f1de81f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1de815d8b0 0x7f1de81525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 --2- 172.21.15.53:0/4200917140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1dd4005180 0x7f1de814fc60 unknown :-1 s=CLOSED pgs=2976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 >> 172.21.15.53:0/4200917140 conn(0x7f1de806a5d0 msgr2=0x7f1de80c5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:23.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 shutdown_connections 2023-10-21T17:41:23.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.160+0000 7f1defd1d640 1 -- 172.21.15.53:0/4200917140 wait complete. 2023-10-21T17:41:23.179 INFO:tasks.ceph:Scrubbing osd.7 2023-10-21T17:41:23.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-21T17:41:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.409+0000 7fec8f890640 1 Processor -- start 2023-10-21T17:41:23.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.409+0000 7fec8f890640 1 -- start start 2023-10-21T17:41:23.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8f890640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec8806c3a0 0x7fec880c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8f890640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec8815aa00 0x7fec880b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8f890640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8815b310 con 0x7fec8806db60 2023-10-21T17:41:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8f890640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880ca230 con 0x7fec8806c3a0 2023-10-21T17:41:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8f890640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8806d290 con 0x7fec8815aa00 2023-10-21T17:41:23.413 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8d62c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec8806c3a0 0x7fec880c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8d62c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fec8806c3a0 0x7fec880c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54690/0 (socket says 172.21.15.53:54690) 2023-10-21T17:41:23.414 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.411+0000 7fec8d62c640 1 -- 172.21.15.53:0/1041823388 learned_addr learned my addr 172.21.15.53:0/1041823388 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:23.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.412+0000 7fec8de2d640 1 --2- 172.21.15.53:0/1041823388 >> v2:172.21.15.124:3300/0 conn(0x7fec8815aa00 0x7fec880b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.412+0000 7fec8d62c640 1 -- 172.21.15.53:0/1041823388 >> v2:172.21.15.124:3300/0 conn(0x7fec8815aa00 msgr2=0x7fec880b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.412+0000 7fec8d62c640 1 --2- 172.21.15.53:0/1041823388 >> v2:172.21.15.124:3300/0 conn(0x7fec8815aa00 0x7fec880b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.412+0000 7fec8d62c640 1 -- 172.21.15.53:0/1041823388 >> v1:172.21.15.53:6789/0 conn(0x7fec8806db60 legacy=0x7fec8806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.412+0000 7fec8d62c640 1 -- 172.21.15.53:0/1041823388 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880c9860 con 0x7fec8806c3a0 2023-10-21T17:41:23.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.413+0000 7fec8d62c640 1 --2- 172.21.15.53:0/1041823388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c9cf0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fec880c57a0 tx=0x7fec8402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc1d149882ebe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec7bfff640 1 -- 172.21.15.53:0/1041823388 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84030860 con 0x7fec8806c3a0 2023-10-21T17:41:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec7bfff640 1 -- 172.21.15.53:0/1041823388 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec840309c0 con 0x7fec8806c3a0 2023-10-21T17:41:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec7bfff640 1 -- 172.21.15.53:0/1041823388 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84030d50 con 0x7fec8806c3a0 2023-10-21T17:41:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 -- 172.21.15.53:0/1041823388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 msgr2=0x7fec880c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 --2- 172.21.15.53:0/1041823388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c9cf0 secure :-1 s=READY pgs=2978 cs=0 l=1 rev1=1 crypto rx=0x7fec880c57a0 tx=0x7fec8402b9a0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 -- 172.21.15.53:0/1041823388 shutdown_connections 2023-10-21T17:41:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 --2- 172.21.15.53:0/1041823388 >> v2:172.21.15.124:3300/0 conn(0x7fec8815aa00 0x7fec880b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 --2- 172.21.15.53:0/1041823388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c9cf0 unknown :-1 s=CLOSED pgs=2978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 -- 172.21.15.53:0/1041823388 >> 172.21.15.53:0/1041823388 conn(0x7fec8806a5d0 msgr2=0x7fec880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:23.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 -- 172.21.15.53:0/1041823388 shutdown_connections 2023-10-21T17:41:23.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.414+0000 7fec8f890640 1 -- 172.21.15.53:0/1041823388 wait complete. 2023-10-21T17:41:23.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.416+0000 7fec8f890640 1 Processor -- start 2023-10-21T17:41:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.416+0000 7fec8f890640 1 -- start start 2023-10-21T17:41:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8806db60 0x7fec881500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8815aa00 0x7fec88154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.421 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec880c36a0 con 0x7fec8806c3a0 2023-10-21T17:41:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec880c7b60 con 0x7fec8806db60 2023-10-21T17:41:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.417+0000 7fec8f890640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec88150610 con 0x7fec8815aa00 2023-10-21T17:41:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54704/0 (socket says 172.21.15.53:54704) 2023-10-21T17:41:23.422 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 -- 172.21.15.53:0/486252969 learned_addr learned my addr 172.21.15.53:0/486252969 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8ce2b640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8806db60 0x7fec881500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 -- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8815aa00 msgr2=0x7fec88154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8de2d640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8815aa00 0x7fec88154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8815aa00 0x7fec88154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.423 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 -- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8806db60 msgr2=0x7fec881500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8806db60 0x7fec881500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.418+0000 7fec8d62c640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840094d0 con 0x7fec8806c3a0 2023-10-21T17:41:23.424 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.419+0000 7fec8d62c640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c3160 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.425 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.420+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84039c90 con 0x7fec8806c3a0 2023-10-21T17:41:23.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.420+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880c58b0 con 0x7fec8806c3a0 2023-10-21T17:41:23.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.420+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84039df0 con 0x7fec8806c3a0 2023-10-21T17:41:23.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.420+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84043620 con 0x7fec8806c3a0 2023-10-21T17:41:23.426 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.420+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880c5e70 con 0x7fec8806c3a0 2023-10-21T17:41:23.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.421+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec880c9210 con 0x7fec8806c3a0 2023-10-21T17:41:23.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.423+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fec84043780 con 0x7fec8806c3a0 2023-10-21T17:41:23.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.423+0000 7fec79ffb640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fec6c06af90 0x7fec6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.424+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7fec8403e070 con 0x7fec8806c3a0 2023-10-21T17:41:23.427 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.424+0000 7fec79ffb640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fec6c0719d0 0x7fec6c073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.424+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec6c074500 con 0x7fec6c0719d0 2023-10-21T17:41:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.424+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=280) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec840b12c0 con 0x7fec8806c3a0 2023-10-21T17:41:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.424+0000 7fec8ce2b640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fec6c06af90 0x7fec6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.428 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.425+0000 7fec8de2d640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fec6c0719d0 0x7fec6c073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.426+0000 7fec8ce2b640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fec6c06af90 0x7fec6c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec880b4210 tx=0x7fec7c009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.426+0000 7fec8de2d640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fec6c0719d0 0x7fec6c073e30 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.429 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.428+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== osd.7 v2:172.21.15.124:6824/1993556734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fec6c074500 con 0x7fec6c0719d0 2023-10-21T17:41:23.445 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.444+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 --> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec880c8f30 con 0x7fec6c0719d0 2023-10-21T17:41:23.446 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.445+0000 7fec79ffb640 1 -- 172.21.15.53:0/486252969 <== osd.7 v2:172.21.15.124:6824/1993556734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fec80014380 con 0x7fec6c0719d0 2023-10-21T17:41:23.446 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-10-21T17:41:23.447 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-21T17:41:23.447 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-10-21T17:41:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.445+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fec6c0719d0 msgr2=0x7fec6c073e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.445+0000 7fec8f890640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6824/1993556734,v1:172.21.15.124:6825/1993556734] conn(0x7fec6c0719d0 0x7fec6c073e30 crc :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.445+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fec6c06af90 msgr2=0x7fec6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.448 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.445+0000 7fec8f890640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7fec6c06af90 0x7fec6c06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec880b4210 tx=0x7fec7c009040 comp rx=0 tx=0).stop 2023-10-21T17:41:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 msgr2=0x7fec880c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 --2- 172.21.15.53:0/486252969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec8806c3a0 0x7fec880c3160 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7fec84000c00 tx=0x7fec84004730 comp rx=0 tx=0).stop 2023-10-21T17:41:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8de2d640 1 -- 172.21.15.53:0/486252969 reap_dead start 2023-10-21T17:41:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 shutdown_connections 2023-10-21T17:41:23.449 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 >> 172.21.15.53:0/486252969 conn(0x7fec8806a5d0 msgr2=0x7fec88159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:23.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 shutdown_connections 2023-10-21T17:41:23.450 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.446+0000 7fec8f890640 1 -- 172.21.15.53:0/486252969 wait complete. 2023-10-21T17:41:23.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-21T17:41:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.695+0000 7f4e3da59640 1 Processor -- start 2023-10-21T17:41:23.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.695+0000 7f4e3da59640 1 -- start start 2023-10-21T17:41:23.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3da59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e38070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3da59640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e38071d90 con 0x7f4e38156a10 2023-10-21T17:41:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3da59640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e38071180 con 0x7f4e380b0680 2023-10-21T17:41:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3da59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38071460 con 0x7f4e381594e0 2023-10-21T17:41:23.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3ca57640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4e38156a10 0x7f4e380aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:53188/0 (socket says 172.21.15.53:53188) 2023-10-21T17:41:23.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.697+0000 7f4e3ca57640 1 -- 172.21.15.53:0/1202567357 learned_addr learned my addr 172.21.15.53:0/1202567357 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:23.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e38070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.702 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126746678 0 0) 0x7f4e38071d90 con 0x7f4e38156a10 2023-10-21T17:41:23.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e20004630 con 0x7f4e38156a10 2023-10-21T17:41:23.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3137390454 0 0) 0x7f4e20004630 con 0x7f4e38156a10 2023-10-21T17:41:23.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 msgr2=0x7f4e38070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:23.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e38070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.703 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.698+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 >> v1:172.21.15.124:6789/0 conn(0x7f4e380b0680 legacy=0x7f4e380c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38071740 con 0x7f4e38156a10 2023-10-21T17:41:23.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f4e340019f0 con 0x7f4e38156a10 2023-10-21T17:41:23.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 >> v1:172.21.15.53:6789/0 conn(0x7f4e38156a10 legacy=0x7f4e380aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e20004ab0 0x7f4e20004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e200058d0 0x7f4e20005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e20006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38071460 con 0x7f4e200058d0 2023-10-21T17:41:23.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2000cb40 con 0x7f4e20004ab0 2023-10-21T17:41:23.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200035a0 con 0x7f4e381594e0 2023-10-21T17:41:23.705 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e2ffff640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e200058d0 0x7f4e20005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e3ca57640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e20004ab0 0x7f4e20004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e2ffff640 1 -- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 msgr2=0x7f4e20006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:23.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.699+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e20006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.706 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e2ffff640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e20006290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e2ffff640 1 -- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e20004ab0 msgr2=0x7f4e20004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:23.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e2ffff640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e20004ab0 0x7f4e20004ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e2ffff640 1 -- 172.21.15.53:0/1202567357 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20004630 con 0x7f4e200058d0 2023-10-21T17:41:23.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.700+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e381594e0 0x7f4e20006290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:23.707 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.701+0000 7f4e2ffff640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e200058d0 0x7f4e20005d50 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f4e28001720 tx=0x7f4e20006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894f0baaa3d3f72c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.701+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280052d0 con 0x7f4e200058d0 2023-10-21T17:41:23.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.701+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2802e050 con 0x7f4e200058d0 2023-10-21T17:41:23.708 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.701+0000 7f4e2f7fe640 1 -- 172.21.15.53:0/1202567357 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280035f0 con 0x7f4e200058d0 2023-10-21T17:41:23.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 -- 172.21.15.53:0/1202567357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e200058d0 msgr2=0x7f4e20005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.714 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1202567357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e200058d0 0x7f4e20005d50 secure :-1 s=READY pgs=2981 cs=0 l=1 rev1=1 crypto rx=0x7f4e28001720 tx=0x7f4e20006900 comp rx=0 tx=0).stop 2023-10-21T17:41:23.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3d258640 1 -- 172.21.15.53:0/1202567357 reap_dead start 2023-10-21T17:41:23.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 -- 172.21.15.53:0/1202567357 shutdown_connections 2023-10-21T17:41:23.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 -- 172.21.15.53:0/1202567357 >> 172.21.15.53:0/1202567357 conn(0x7f4e3806c050 msgr2=0x7f4e3806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:23.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 -- 172.21.15.53:0/1202567357 shutdown_connections 2023-10-21T17:41:23.715 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.702+0000 7f4e3da59640 1 -- 172.21.15.53:0/1202567357 wait complete. 2023-10-21T17:41:23.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.704+0000 7f4e3da59640 1 Processor -- start 2023-10-21T17:41:23.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.704+0000 7f4e3da59640 1 -- start start 2023-10-21T17:41:23.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200058d0 0x7f4e3814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e380af230 0x7f4e380af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.716 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380afbd0 con 0x7f4e38158210 2023-10-21T17:41:23.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c2dd0 con 0x7f4e200058d0 2023-10-21T17:41:23.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3da59640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e38158bd0 con 0x7f4e380af230 2023-10-21T17:41:23.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3d258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.717 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3d258640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:54720/0 (socket says 172.21.15.53:54720) 2023-10-21T17:41:23.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3d258640 1 -- 172.21.15.53:0/1580556345 learned_addr learned my addr 172.21.15.53:0/1580556345 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:23.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.705+0000 7f4e3ca57640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200058d0 0x7f4e3814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e2ffff640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e380af230 0x7f4e380af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 -- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e380af230 msgr2=0x7f4e380af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.718 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e380af230 0x7f4e380af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 -- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200058d0 msgr2=0x7f4e3814eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200058d0 0x7f4e3814eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 -- 172.21.15.53:0/1580556345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380c21c0 con 0x7f4e38158210 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3d258640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f4e3806d9d0 tx=0x7f4e30010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000e870 con 0x7f4e38158210 2023-10-21T17:41:23.719 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3000e9f0 con 0x7f4e38158210 2023-10-21T17:41:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000eda0 con 0x7f4e38158210 2023-10-21T17:41:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e380c24a0 con 0x7f4e38158210 2023-10-21T17:41:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.706+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e380c2a60 con 0x7f4e38158210 2023-10-21T17:41:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.708+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e380c2c20 con 0x7f4e38158210 2023-10-21T17:41:23.720 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.709+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4e3001c400 con 0x7f4e38158210 2023-10-21T17:41:23.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.709+0000 7f4e2d7fa640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 0x7f4e2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:23.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.710+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f4e30089830 con 0x7f4e38158210 2023-10-21T17:41:23.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.710+0000 7f4e3ca57640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 0x7f4e2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:23.721 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.711+0000 7f4e3ca57640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 0x7f4e2406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001330 tx=0x7f4e34009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:23.722 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.714+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e30054600 con 0x7f4e38158210 2023-10-21T17:41:23.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.910+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e380cbd80 con 0x7f4e2406af90 2023-10-21T17:41:23.912 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.911+0000 7f4e2d7fa640 1 -- 172.21.15.53:0/1580556345 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4e34005e80 con 0x7f4e2406af90 2023-10-21T17:41:23.913 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 7 to deep-scrub 2023-10-21T17:41:23.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 msgr2=0x7f4e2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 0x7f4e2406d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e34001330 tx=0x7f4e34009040 comp rx=0 tx=0).stop 2023-10-21T17:41:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 msgr2=0x7f4e38158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7f4e3806d9d0 tx=0x7f4e30010c20 comp rx=0 tx=0).stop 2023-10-21T17:41:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 shutdown_connections 2023-10-21T17:41:23.919 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f4e2406af90 0x7f4e2406d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4e38158210 0x7f4e38158690 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e380af230 0x7f4e380af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 --2- 172.21.15.53:0/1580556345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200058d0 0x7f4e3814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 >> 172.21.15.53:0/1580556345 conn(0x7f4e3806c050 msgr2=0x7f4e38154b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:23.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.914+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 shutdown_connections 2023-10-21T17:41:23.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:23.915+0000 7f4e3da59640 1 -- 172.21.15.53:0/1580556345 wait complete. 2023-10-21T17:41:23.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:41:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.145+0000 7f0b22c75640 1 Processor -- start 2023-10-21T17:41:24.147 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.146+0000 7f0b22c75640 1 -- start start 2023-10-21T17:41:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.147+0000 7f0b22c75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c15cd90 0x7f0b1c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b22c75640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b1c06c890 0x7f0b1c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b22c75640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1c0c48f0 con 0x7f0b1c160940 2023-10-21T17:41:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b22c75640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1c0c4d80 con 0x7f0b1c06c890 2023-10-21T17:41:24.149 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b22c75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c0c3ce0 con 0x7f0b1c15cd90 2023-10-21T17:41:24.150 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b21212640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b1c06c890 0x7f0b1c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b21212640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b1c06c890 0x7f0b1c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57660/0 (socket says 172.21.15.53:57660) 2023-10-21T17:41:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b21212640 1 -- 172.21.15.53:0/3491176917 learned_addr learned my addr 172.21.15.53:0/3491176917 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.148+0000 7f0b20a11640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c15cd90 0x7f0b1c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.151 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.149+0000 7f0b21212640 1 -- 172.21.15.53:0/3491176917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c15cd90 msgr2=0x7f0b1c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.149+0000 7f0b21212640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c15cd90 0x7f0b1c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.149+0000 7f0b21212640 1 -- 172.21.15.53:0/3491176917 >> v1:172.21.15.124:6789/0 conn(0x7f0b1c160940 legacy=0x7f0b1c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:24.152 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.149+0000 7f0b21212640 1 -- 172.21.15.53:0/3491176917 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c0c3fc0 con 0x7f0b1c06c890 2023-10-21T17:41:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.150+0000 7f0b21212640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c06c890 0x7f0b1c06cd20 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009870 tx=0x7f0b1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494965513e110e2f server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:24.153 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.150+0000 7f0b1b7fe640 1 -- 172.21.15.53:0/3491176917 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031640 con 0x7f0b1c06c890 2023-10-21T17:41:24.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.150+0000 7f0b1b7fe640 1 -- 172.21.15.53:0/3491176917 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b140317a0 con 0x7f0b1c06c890 2023-10-21T17:41:24.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.150+0000 7f0b1b7fe640 1 -- 172.21.15.53:0/3491176917 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031b30 con 0x7f0b1c06c890 2023-10-21T17:41:24.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 -- 172.21.15.53:0/3491176917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c06c890 msgr2=0x7f0b1c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:24.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c06c890 0x7f0b1c06cd20 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009870 tx=0x7f0b1402b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.154 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 -- 172.21.15.53:0/3491176917 shutdown_connections 2023-10-21T17:41:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c06c890 0x7f0b1c06cd20 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 --2- 172.21.15.53:0/3491176917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c15cd90 0x7f0b1c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 -- 172.21.15.53:0/3491176917 >> 172.21.15.53:0/3491176917 conn(0x7f0b1c06a380 msgr2=0x7f0b1c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 -- 172.21.15.53:0/3491176917 shutdown_connections 2023-10-21T17:41:24.155 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.151+0000 7f0b22c75640 1 -- 172.21.15.53:0/3491176917 wait complete. 2023-10-21T17:41:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.153+0000 7f0b22c75640 1 Processor -- start 2023-10-21T17:41:24.157 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.153+0000 7f0b22c75640 1 -- start start 2023-10-21T17:41:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c06c890 0x7f0b1c14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b1c160940 0x7f0b1c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c160230 con 0x7f0b1c15cd90 2023-10-21T17:41:24.158 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c1578c0 con 0x7f0b1c160940 2023-10-21T17:41:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b22c75640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c156cb0 con 0x7f0b1c06c890 2023-10-21T17:41:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.154+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57664/0 (socket says 172.21.15.53:57664) 2023-10-21T17:41:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 -- 172.21.15.53:0/1355434277 learned_addr learned my addr 172.21.15.53:0/1355434277 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:24.159 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b20a11640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c06c890 0x7f0b1c14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b21212640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b1c160940 0x7f0b1c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 -- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c06c890 msgr2=0x7f0b1c14c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c06c890 0x7f0b1c14c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 -- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b1c160940 msgr2=0x7f0b1c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b1c160940 0x7f0b1c15db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 -- 172.21.15.53:0/1355434277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140094d0 con 0x7f0b1c15cd90 2023-10-21T17:41:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.155+0000 7f0b1bfff640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b3f0 tx=0x7f0b0400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.156+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04011830 con 0x7f0b1c15cd90 2023-10-21T17:41:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.156+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b04011990 con 0x7f0b1c15cd90 2023-10-21T17:41:24.161 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.156+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04011d40 con 0x7f0b1c15cd90 2023-10-21T17:41:24.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.156+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c156f90 con 0x7f0b1c15cd90 2023-10-21T17:41:24.162 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.156+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c157550 con 0x7f0b1c15cd90 2023-10-21T17:41:24.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.157+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c067490 con 0x7f0b1c15cd90 2023-10-21T17:41:24.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.158+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b0401a400 con 0x7f0b1c15cd90 2023-10-21T17:41:24.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.159+0000 7f0b197fa640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 0x7f0afc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:24.163 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.159+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f0b040898a0 con 0x7f0b1c15cd90 2023-10-21T17:41:24.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.160+0000 7f0b20a11640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 0x7f0afc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:24.164 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.161+0000 7f0b20a11640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 0x7f0afc06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f30 tx=0x7f0b0c00b040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:24.165 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.164+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b040545e0 con 0x7f0b1c15cd90 2023-10-21T17:41:24.353 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.352+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b1c0c4580 con 0x7f0afc06af90 2023-10-21T17:41:24.360 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.359+0000 7f0b197fa640 1 -- 172.21.15.53:0/1355434277 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48473 (secure 0 0 0) 0x7f0b0c009b20 con 0x7f0afc06af90 2023-10-21T17:41:24.361 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:24.361 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:41:24.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 msgr2=0x7f0afc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:24.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 0x7f0afc06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f30 tx=0x7f0b0c00b040 comp rx=0 tx=0).stop 2023-10-21T17:41:24.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 msgr2=0x7f0b1c156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:24.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b3f0 tx=0x7f0b0400b8b0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 shutdown_connections 2023-10-21T17:41:24.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f0afc06af90 0x7f0afc06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b1c160940 0x7f0b1c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b1c15cd90 0x7f0b1c156260 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 --2- 172.21.15.53:0/1355434277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b1c06c890 0x7f0b1c14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:24.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.361+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 >> 172.21.15.53:0/1355434277 conn(0x7f0b1c06a380 msgr2=0x7f0b1c0ad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:24.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.362+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 shutdown_connections 2023-10-21T17:41:24.365 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:24.362+0000 7f0b22c75640 1 -- 172.21.15.53:0/1355434277 wait complete. 2023-10-21T17:41:24.381 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-10-21T17:41:22.604511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":461536,"kb_used_data":28840,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754513184,"statfs":{"total":773094113280,"available":772621500416,"internally_reserved":0,"allocated":29532160,"data_stored":16018320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.974376"},"pg_stats":[{"pgid":"1.0","version":"277'4931","reported_seq":6049,"reported_epoch":279,"state":"active+clean","last_fresh":"2023-10-21T17:41:10.956386+0000","last_change":"2023-10-21T16:01:51.047593+0000","last_active":"2023-10-21T17:41:10.956386+0000","last_peered":"2023-10-21T17:41:10.956386+0000","last_clean":"2023-10-21T17:41:10.956386+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:41:10.956386+0000","last_undegraded":"2023-10-21T17:41:10.956386+0000","last_fullsized":"2023-10-21T17:41:10.956386+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-21T16:01:49.781498+0000","last_clean_scrub_stamp":"2023-10-21T16:01:49.781498+0000","objects_scrubbed":0,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T21:57:29.577106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4931,"ondisk_log_size":4931,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75476,"kb_used_data":4040,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296364,"statfs":{"total":96636764160,"available":96559476736,"internally_reserved":0,"allocated":4136960,"data_stored":2445066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.76700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.724,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.77100000000000002,"5min":0.64300000000000002,"15min":0.63800000000000001},"max":{"1min":1.5069999999999999,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.1339999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.175,"15min":1.1910000000000001},"min":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.57099999999999995},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.78600000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.909,"15min":1.909},"last":1.2929999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3540000000000001,"15min":1.3320000000000001},"min":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.65700000000000003},"max":{"1min":1.849,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.667,"5min":3.556,"15min":3.556},"last":1.8919999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.306},"min":{"1min":0.623,"5min":0.623,"15min":0.50700000000000001},"max":{"1min":2.3439999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.486,"15min":1.4610000000000001},"min":{"1min":1.087,"5min":0.69999999999999996,"15min":0.58899999999999997},"max":{"1min":1.859,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.538}]},{"osd":4,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3859999999999999,"15min":1.361},"min":{"1min":0.64100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7350000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":2.1080000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.90000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3799999999999999,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":2.0430000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5,"15min":1.431},"min":{"1min":0.74399999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9770000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.748},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.514,"15min":1.464},"min":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0379999999999998,"5min":2.613,"15min":2.613},"last":2.012}]},{"osd":6,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.613,"15min":1.5640000000000001},"min":{"1min":1.181,"5min":0.625,"15min":0.625},"max":{"1min":1.964,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":2.169},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5600000000000001,"15min":1.4990000000000001},"min":{"1min":0.99299999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2410000000000001,"5min":2.879,"15min":2.879},"last":1.952}]}]},{"osd":6,"up_from":13,"seq":55834576029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65168,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306672,"statfs":{"total":96636764160,"available":96570032128,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.119,"15min":1.1679999999999999},"min":{"1min":0.70499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6319999999999999,"5min":1.944,"15min":1.944},"last":1.704},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.147,"15min":1.1890000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5249999999999999,"5min":1.6619999999999999,"15min":1.762},"last":0.81100000000000005}]},{"osd":1,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.121,"15min":1.1579999999999999},"min":{"1min":0.73199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4990000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.198,"15min":1.278},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.5620000000000001,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.2170000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2450000000000001,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.266,"15min":1.288},"min":{"1min":0.80700000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6519999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0740000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3009999999999999,"15min":1.3839999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.629}]},{"osd":4,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3009999999999999,"15min":1.3360000000000001},"min":{"1min":0.47699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.9910000000000001,"5min":2.419,"15min":2.419},"last":1.452},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4299999999999999},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0299999999999998},"last":1.7549999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.552,"15min":1.556},"min":{"1min":0.77400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9450000000000001,"5min":2.1299999999999999,"15min":2.1739999999999999},"last":1.804},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3560000000000001,"15min":1.361},"min":{"1min":0.56100000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.9219999999999999,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.544}]},{"osd":7,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4690000000000001,"15min":1.5169999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0550000000000002,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.663,"5min":1.639,"15min":1.651},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0219999999999998,"5min":2.21,"15min":2.2610000000000001},"last":1.8819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48528,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323312,"statfs":{"total":96636764160,"available":96587071488,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.103,"15min":1.125},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.671,"5min":1.671,"15min":1.9930000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1559999999999999,"15min":1.2},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9550000000000001,"5min":1.994,"15min":2.2909999999999999},"last":1.325}]},{"osd":1,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.137,"5min":1.129,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.738,"5min":1.75,"15min":2.4529999999999998},"last":1.0309999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.282,"15min":1.2789999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1990000000000001,"15min":1.2290000000000001},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.595,"5min":2.266,"15min":2.266},"last":1.4219999999999999}]},{"osd":3,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3120000000000001,"15min":1.333},"min":{"1min":0.79000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9079999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.488},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.881},"last":1.5529999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2909999999999999,"15min":1.2609999999999999},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9450000000000001,"5min":1.976,"15min":1.976},"last":1.9059999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.448,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0489999999999999,"5min":2.093,"15min":2.093},"last":1.6279999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4670000000000001,"15min":1.474},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7909999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5249999999999999,"15min":1.5209999999999999},"min":{"1min":0.91900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.875,"5min":2.1579999999999999,"15min":2.2269999999999999},"last":1.8460000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.55,"15min":1.538},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.58499999999999996},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.552,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0070000000000001,"5min":2.0569999999999999,"15min":2.1299999999999999},"last":1.8080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52692,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319148,"statfs":{"total":96636764160,"available":96582807552,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.1379999999999999},"min":{"1min":0.68500000000000005,"5min":0.59799999999999998,"15min":0.48999999999999999},"max":{"1min":1.5149999999999999,"5min":1.746,"15min":1.8480000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.087,"5min":1.147,"15min":1.202},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.869},"last":0.86799999999999999}]},{"osd":1,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7250000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.179,"15min":1.2390000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6000000000000001,"5min":1.762,"15min":1.762},"last":1.3600000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2010000000000001,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.534,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.343,"5min":1.349,"15min":1.3759999999999999},"min":{"1min":0.90200000000000002,"5min":0.88700000000000001,"15min":0.82199999999999995},"max":{"1min":1.78,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.302}]},{"osd":3,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2350000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.091},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3700000000000001,"15min":1.419},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.60999999999999999},"max":{"1min":1.728,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.359,"15min":1.3500000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.982,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":0.85799999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.2170000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.97699999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.96,"5min":2.145,"15min":2.1600000000000001},"last":1.143},{"interface":"front","average":{"1min":1.528,"5min":1.5640000000000001,"15min":1.5960000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.016,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.4219999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4970000000000001,"15min":1.5189999999999999},"min":{"1min":0.998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.464},{"interface":"front","average":{"1min":1.554,"5min":1.5669999999999999,"15min":1.5900000000000001},"min":{"1min":0.95199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.052,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.1990000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56852,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314988,"statfs":{"total":96636764160,"available":96578547712,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1919999999999999,"5min":1.6439999999999999,"15min":1.71},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.167,"5min":1.1559999999999999,"15min":1.2250000000000001},"min":{"1min":0.54300000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.7709999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.92500000000000004}]},{"osd":1,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":1.038},"min":{"1min":0.51400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4239999999999999,"5min":1.835,"15min":1.835},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.181,"15min":1.2789999999999999},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.456,"5min":1.911,"15min":1.911},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.0880000000000001},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.921,"15min":2.1850000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1439999999999999,"15min":1.2509999999999999},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.605,"5min":1.9870000000000001,"15min":2},"last":0.71399999999999997}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3180000000000001,"15min":1.4099999999999999},"min":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.502,"5min":2.262,"15min":2.262},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3740000000000001,"15min":1.4379999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.9430000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.79700000000000004}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.369,"15min":1.464},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6579999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.421,"5min":1.502,"15min":1.546},"min":{"1min":0.871,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8069999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.093}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4830000000000001,"15min":1.5629999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7789999999999999,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.304,"5min":1.4970000000000001,"15min":1.585},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.002}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5669999999999999,"15min":1.6419999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.8520000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.298},{"interface":"front","average":{"1min":1.377,"5min":1.5649999999999999,"15min":1.6679999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.766,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":1.0640000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56852,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314988,"statfs":{"total":96636764160,"available":96578547712,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.448,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.93200000000000005,"15min":1.0049999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1779999999999999,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":0.80500000000000005}]},{"osd":1,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.1359999999999999,"15min":1.175},"min":{"1min":0.55900000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.494,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.085,"5min":1.222,"15min":1.327},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.958},"last":1.0860000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.141,"15min":1.1659999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.911,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.421,"15min":1.456},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9770000000000001,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.2410000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.429,"15min":1.423},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.875,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.696},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.3680000000000001,"15min":1.4299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.0750000000000002,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.536}]},{"osd":5,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.3939999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8879999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":0.755,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2490000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.4750000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4550000000000001,"15min":1.538},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1549999999999998,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.45,"15min":1.504},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7390000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.4970000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.581,"15min":1.619},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.149,"5min":2.149,"15min":2.4630000000000001},"last":1.821},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.5329999999999999,"15min":1.577},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.036,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.5740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52692,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319148,"statfs":{"total":96636764160,"available":96582807552,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.976},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.115},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":1.756,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.97499999999999998}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":1.0169999999999999},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2370000000000001,"5min":1.746,"15min":1.79},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.135,"15min":1.2010000000000001},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.26,"15min":1.335},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8240000000000001,"5min":1.9239999999999999,"15min":2.129},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.119,"5min":1.208,"15min":1.252},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.437},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.91500000000000004}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.206,"15min":1.3140000000000001},"min":{"1min":0.61899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":1.022},{"interface":"front","average":{"1min":1.375,"5min":1.423,"15min":1.468},"min":{"1min":0.90900000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0539999999999998,"5min":2.1070000000000002,"15min":2.383},"last":0.94699999999999995}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.522},"min":{"1min":0.92000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.3969999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4079999999999999,"15min":1.492},"min":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8660000000000001,"5min":2.1000000000000001,"15min":2.3359999999999999},"last":1.246}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4650000000000001,"15min":1.5509999999999999},"min":{"1min":0.876,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":2.2069999999999999,"15min":2.4319999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.385,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.794,"5min":2.1360000000000001,"15min":2.5310000000000001},"last":1.0740000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.5549999999999999,"15min":1.621},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1259999999999999,"5min":2.1739999999999999,"15min":2.548},"last":1.196},{"interface":"front","average":{"1min":1.502,"5min":1.583,"15min":1.663},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9550000000000001,"5min":2.3730000000000002,"15min":2.492},"last":1.2709999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53276,"kb_used_data":4040,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318564,"statfs":{"total":96636764160,"available":96582209536,"internally_reserved":0,"allocated":4136960,"data_stored":2445066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.48099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9249999999999998,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":1.081},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.748},"last":1.264}]},{"osd":2,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.121,"15min":1.1659999999999999},"min":{"1min":0.60599999999999998,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.21},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.1539999999999999},"min":{"1min":0.51200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.851},"last":1.4730000000000001}]},{"osd":3,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.109},"min":{"1min":0.57999999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.516},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.375},"min":{"1min":0.67800000000000005,"5min":0.496,"15min":0.496},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.446}]},{"osd":4,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.878,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.853,"5min":1.9570000000000001,"15min":2.032},"last":1.889},{"interface":"front","average":{"1min":1.607,"5min":1.53,"15min":1.5860000000000001},"min":{"1min":1.109,"5min":1.018,"15min":1.018},"max":{"1min":2.113,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.752}]},{"osd":5,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":0.81899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.925,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.639},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5029999999999999,"15min":1.5609999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.6819999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.4179999999999999,"15min":1.4770000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.617,"15min":1.6659999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5259999999999998,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.5720000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:40:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4319999999999999,"15min":1.5189999999999999},"min":{"1min":0.93700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.528,"15min":1.6020000000000001},"min":{"1min":0.85799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.7669999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:41:24.388 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-21T16:01:49.781498+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=21, tm_hour=16, tm_min=1, tm_sec=49, tm_wday=5, tm_yday=294, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=21, tm_hour=17, tm_min=41, tm_sec=17, tm_wday=5, tm_yday=294, tm_isdst=0) 2023-10-21T17:41:24.389 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-21T17:41:44.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-21T17:41:44.633 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.632+0000 7f129ffff640 1 Processor -- start 2023-10-21T17:41:44.634 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.632+0000 7f129ffff640 1 -- start start 2023-10-21T17:41:44.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.633+0000 7f129ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a0160290 0x7f12a00cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.633+0000 7f129ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f12a00cb150 0x7f12a00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.633+0000 7f129ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a00cb5a0 con 0x7f12a015feb0 2023-10-21T17:41:44.635 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.633+0000 7f129ffff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a006d3d0 con 0x7f12a0160290 2023-10-21T17:41:44.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.633+0000 7f129ffff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a006a720 con 0x7f12a00cb150 2023-10-21T17:41:44.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a0160290 0x7f12a00cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:44.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f12a0160290 0x7f12a00cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47142/0 (socket says 172.21.15.53:47142) 2023-10-21T17:41:44.636 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 -- 172.21.15.53:0/4178582148 learned_addr learned my addr 172.21.15.53:0/4178582148 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:44.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 -- 172.21.15.53:0/4178582148 >> v2:172.21.15.124:3300/0 conn(0x7f12a00cb150 msgr2=0x7f12a00c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:44.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 --2- 172.21.15.53:0/4178582148 >> v2:172.21.15.124:3300/0 conn(0x7f12a00cb150 0x7f12a00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 -- 172.21.15.53:0/4178582148 >> v1:172.21.15.53:6789/0 conn(0x7f12a015feb0 legacy=0x7f12a006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.638 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.634+0000 7f129e7fc640 1 -- 172.21.15.53:0/4178582148 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0069df0 con 0x7f12a0160290 2023-10-21T17:41:44.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.635+0000 7f129e7fc640 1 --2- 172.21.15.53:0/4178582148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a0160290 0x7f12a00cac10 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f12a0065bb0 tx=0x7f128802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b33cd965f6b5a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:44.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129dffb640 1 -- 172.21.15.53:0/4178582148 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288030870 con 0x7f12a0160290 2023-10-21T17:41:44.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129dffb640 1 -- 172.21.15.53:0/4178582148 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880309d0 con 0x7f12a0160290 2023-10-21T17:41:44.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129dffb640 1 -- 172.21.15.53:0/4178582148 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288030d60 con 0x7f12a0160290 2023-10-21T17:41:44.639 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 -- 172.21.15.53:0/4178582148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a0160290 msgr2=0x7f12a00cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 --2- 172.21.15.53:0/4178582148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a0160290 0x7f12a00cac10 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7f12a0065bb0 tx=0x7f128802b9b0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 -- 172.21.15.53:0/4178582148 shutdown_connections 2023-10-21T17:41:44.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 --2- 172.21.15.53:0/4178582148 >> v2:172.21.15.124:3300/0 conn(0x7f12a00cb150 0x7f12a00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 --2- 172.21.15.53:0/4178582148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a0160290 0x7f12a00cac10 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.640 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 -- 172.21.15.53:0/4178582148 >> 172.21.15.53:0/4178582148 conn(0x7f12a00cede0 msgr2=0x7f12a00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:44.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 -- 172.21.15.53:0/4178582148 shutdown_connections 2023-10-21T17:41:44.641 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.636+0000 7f129ffff640 1 -- 172.21.15.53:0/4178582148 wait complete. 2023-10-21T17:41:44.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.638+0000 7f129ffff640 1 Processor -- start 2023-10-21T17:41:44.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.638+0000 7f129ffff640 1 -- start start 2023-10-21T17:41:44.643 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 0x7f12a01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 0x7f12a015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a01f56d0 con 0x7f12a015feb0 2023-10-21T17:41:44.644 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a01f5990 con 0x7f12a00cb150 2023-10-21T17:41:44.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a01f5c70 con 0x7f12a0160290 2023-10-21T17:41:44.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.639+0000 7f129e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:44.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47150/0 (socket says 172.21.15.53:47150) 2023-10-21T17:41:44.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 -- 172.21.15.53:0/4227787068 learned_addr learned my addr 172.21.15.53:0/4227787068 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:44.645 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 -- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 msgr2=0x7f12a015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129f7fe640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 0x7f12a015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:44.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129effd640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 0x7f12a01550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:44.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 0x7f12a015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 -- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 msgr2=0x7f12a01550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.646 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 0x7f12a01550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 -- 172.21.15.53:0/4227787068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880094d0 con 0x7f12a015feb0 2023-10-21T17:41:44.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129f7fe640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 0x7f12a015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:44.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129effd640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 0x7f12a01550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:44.647 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.640+0000 7f129e7fc640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f1288004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.641+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288030840 con 0x7f12a015feb0 2023-10-21T17:41:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.641+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288043400 con 0x7f12a015feb0 2023-10-21T17:41:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.641+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a01f5f50 con 0x7f12a015feb0 2023-10-21T17:41:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.641+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880437b0 con 0x7f12a015feb0 2023-10-21T17:41:44.648 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.641+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a01555f0 con 0x7f12a015feb0 2023-10-21T17:41:44.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.642+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a004e160 con 0x7f12a015feb0 2023-10-21T17:41:44.649 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.643+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1288043910 con 0x7f12a015feb0 2023-10-21T17:41:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.644+0000 7f12977fe640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 0x7f128006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.645+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f128803e070 con 0x7f12a015feb0 2023-10-21T17:41:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.645+0000 7f129effd640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 0x7f128006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:44.650 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.646+0000 7f129effd640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 0x7f128006d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f12900012e0 tx=0x7f1290009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:44.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.649+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1288077db0 con 0x7f12a015feb0 2023-10-21T17:41:44.840 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.838+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 --> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f12a01f5b00 con 0x7f128006afc0 2023-10-21T17:41:44.846 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.845+0000 7f12977fe640 1 -- 172.21.15.53:0/4227787068 <== mgr.38828 v2:172.21.15.124:6832/3322763271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48173 (secure 0 0 0) 0x7f12a01f5b00 con 0x7f128006afc0 2023-10-21T17:41:44.847 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:44.847 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2023-10-21T17:41:44.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.847+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 msgr2=0x7f128006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.849 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.847+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 0x7f128006d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f12900012e0 tx=0x7f1290009040 comp rx=0 tx=0).stop 2023-10-21T17:41:44.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.847+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 msgr2=0x7f12a015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:44.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.847+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f1288004730 comp rx=0 tx=0).stop 2023-10-21T17:41:44.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 shutdown_connections 2023-10-21T17:41:44.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f128006afc0 0x7f128006d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.850 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12a0160290 0x7f12a015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a015feb0 0x7f12a015dd50 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 --2- 172.21.15.53:0/4227787068 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12a00cb150 0x7f12a01550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:44.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 >> 172.21.15.53:0/4227787068 conn(0x7f12a00cede0 msgr2=0x7f12a00af820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:44.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 shutdown_connections 2023-10-21T17:41:44.851 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:44.848+0000 7f129ffff640 1 -- 172.21.15.53:0/4227787068 wait complete. 2023-10-21T17:41:44.869 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-21T17:41:44.611554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4932,"ondisk_log_size":4932,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":461520,"kb_used_data":28840,"kb_used_omap":12,"kb_used_meta":432563,"kb_avail":754513200,"statfs":{"total":773094113280,"available":772621516800,"internally_reserved":0,"allocated":29532160,"data_stored":16018320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":442945081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003574"},"pg_stats":[{"pgid":"1.0","version":"280'4932","reported_seq":6059,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-10-21T17:41:25.629735+0000","last_change":"2023-10-21T17:41:25.629853+0000","last_active":"2023-10-21T17:41:25.629735+0000","last_peered":"2023-10-21T17:41:25.629735+0000","last_clean":"2023-10-21T17:41:25.629735+0000","last_became_active":"2023-10-21T16:01:51.044586+0000","last_became_peered":"2023-10-21T16:01:51.044586+0000","last_unstale":"2023-10-21T17:41:25.629735+0000","last_undegraded":"2023-10-21T17:41:25.629735+0000","last_fullsized":"2023-10-21T17:41:25.629735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"280'4932","last_scrub_stamp":"2023-10-21T17:41:25.629581+0000","last_deep_scrub":"280'4932","last_deep_scrub_stamp":"2023-10-21T17:41:25.629581+0000","last_clean_scrub_stamp":"2023-10-21T17:41:25.629581+0000","objects_scrubbed":2,"log_size":4932,"log_dups_size":0,"ondisk_log_size":4932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-22T22:16:11.252159+0000","scrub_duration":0.013827951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4932,"ondisk_log_size":4932,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75472,"kb_used_data":4040,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296368,"statfs":{"total":96636764160,"available":96559480832,"internally_reserved":0,"allocated":4136960,"data_stored":2445066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.171,"15min":1.1930000000000001},"min":{"1min":0.76700000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.724,"5min":1.9199999999999999,"15min":2.2549999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1459999999999999,"15min":1.1579999999999999},"min":{"1min":0.77100000000000002,"5min":0.64300000000000002,"15min":0.63800000000000001},"max":{"1min":1.5069999999999999,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.92600000000000005}]},{"osd":1,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.175,"15min":1.1910000000000001},"min":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.57099999999999995},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.258,"15min":1.2909999999999999},"min":{"1min":0.78600000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.909,"15min":1.909},"last":1.532}]},{"osd":2,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3540000000000001,"15min":1.3320000000000001},"min":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.65700000000000003},"max":{"1min":1.849,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.667,"5min":3.556,"15min":3.556},"last":1.411}]},{"osd":3,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.306},"min":{"1min":0.623,"5min":0.623,"15min":0.50700000000000001},"max":{"1min":2.3439999999999999,"5min":2.472,"15min":2.5110000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.486,"15min":1.4610000000000001},"min":{"1min":1.087,"5min":0.69999999999999996,"15min":0.58899999999999997},"max":{"1min":1.859,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":1.3759999999999999}]},{"osd":4,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3859999999999999,"15min":1.361},"min":{"1min":0.64100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7350000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.27},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.90000000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3799999999999999,"5min":2.6749999999999998,"15min":2.6749999999999998},"last":1.452}]},{"osd":5,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5,"15min":1.431},"min":{"1min":0.74399999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.9770000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.514,"15min":1.464},"min":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.0379999999999998,"5min":2.613,"15min":2.613},"last":1.091}]},{"osd":6,"last update":"Sat Oct 21 17:41:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.613,"15min":1.5640000000000001},"min":{"1min":1.181,"5min":0.625,"15min":0.625},"max":{"1min":1.964,"5min":3.3100000000000001,"15min":3.3100000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5600000000000001,"15min":1.4990000000000001},"min":{"1min":0.99299999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.2410000000000001,"5min":2.879,"15min":2.879},"last":1.5629999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":65168,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306672,"statfs":{"total":96636764160,"available":96570032128,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.119,"15min":1.1679999999999999},"min":{"1min":0.70499999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.6319999999999999,"5min":1.944,"15min":1.944},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.147,"15min":1.1890000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5249999999999999,"5min":1.6619999999999999,"15min":1.762},"last":1.5620000000000001}]},{"osd":1,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.121,"15min":1.1579999999999999},"min":{"1min":0.73199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4990000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.198,"15min":1.278},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.5620000000000001,"5min":1.8120000000000001,"15min":1.9359999999999999},"last":1.4430000000000001}]},{"osd":2,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2450000000000001,"15min":1.294},"min":{"1min":0.77100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.236},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.266,"15min":1.288},"min":{"1min":0.80700000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6519999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.597}]},{"osd":3,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3009999999999999,"15min":1.3839999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8839999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.861},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.429},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.637}]},{"osd":4,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3009999999999999,"15min":1.3360000000000001},"min":{"1min":0.47699999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.9910000000000001,"5min":2.419,"15min":2.419},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4319999999999999,"15min":1.4299999999999999},"min":{"1min":0.61099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0299999999999998},"last":1.4990000000000001}]},{"osd":5,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.552,"15min":1.556},"min":{"1min":0.77400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9450000000000001,"5min":2.1299999999999999,"15min":2.1739999999999999},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.3560000000000001,"15min":1.361},"min":{"1min":0.56100000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.9219999999999999,"5min":2.3250000000000002,"15min":2.3250000000000002},"last":1.6839999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:41:05 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4690000000000001,"15min":1.5169999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.0550000000000002,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.788},{"interface":"front","average":{"1min":1.663,"5min":1.639,"15min":1.651},"min":{"1min":0.71499999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.0219999999999998,"5min":2.21,"15min":2.2610000000000001},"last":1.7450000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48528,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323312,"statfs":{"total":96636764160,"available":96587071488,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.103,"15min":1.125},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.671,"5min":1.671,"15min":1.9930000000000001},"last":1.347},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1559999999999999,"15min":1.2},"min":{"1min":0.70199999999999996,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9550000000000001,"5min":1.994,"15min":2.2909999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.26,"15min":1.2649999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.129,"15min":1.2030000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.738,"5min":1.75,"15min":2.4529999999999998},"last":0.91500000000000004}]},{"osd":2,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.282,"15min":1.2789999999999999},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1990000000000001,"15min":1.2290000000000001},"min":{"1min":0.83599999999999997,"5min":0.73299999999999998,"15min":0.67500000000000004},"max":{"1min":1.595,"5min":2.266,"15min":2.266},"last":1.131}]},{"osd":3,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3120000000000001,"15min":1.333},"min":{"1min":0.79000000000000004,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.9079999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.363,"15min":1.3939999999999999},"min":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.6910000000000001,"5min":1.843,"15min":1.881},"last":1.524}]},{"osd":4,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2909999999999999,"15min":1.2609999999999999},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.9450000000000001,"5min":1.976,"15min":1.976},"last":1.492},{"interface":"front","average":{"1min":1.361,"5min":1.448,"15min":1.417},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.0489999999999999,"5min":2.093,"15min":2.093},"last":1.413}]},{"osd":6,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4670000000000001,"15min":1.474},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7909999999999999,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5249999999999999,"15min":1.5209999999999999},"min":{"1min":0.91900000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.875,"5min":2.1579999999999999,"15min":2.2269999999999999},"last":1.609}]},{"osd":7,"last update":"Sat Oct 21 17:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.55,"15min":1.538},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.58499999999999996},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.552,"15min":1.5609999999999999},"min":{"1min":0.94199999999999995,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":2.0070000000000001,"5min":2.0569999999999999,"15min":2.1299999999999999},"last":1.5920000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576032,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52692,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319148,"statfs":{"total":96636764160,"available":96582807552,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.1379999999999999},"min":{"1min":0.68500000000000005,"5min":0.59799999999999998,"15min":0.48999999999999999},"max":{"1min":1.5149999999999999,"5min":1.746,"15min":1.8480000000000001},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.147,"15min":1.202},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.869},"last":1.357}]},{"osd":1,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2030000000000001,"15min":1.2250000000000001},"min":{"1min":0.69399999999999995,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7250000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.179,"15min":1.2390000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6000000000000001,"5min":1.762,"15min":1.762},"last":1.552}]},{"osd":2,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2010000000000001,"15min":1.2450000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.534,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.349,"15min":1.3759999999999999},"min":{"1min":0.90200000000000002,"5min":0.88700000000000001,"15min":0.82199999999999995},"max":{"1min":1.78,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.51}]},{"osd":3,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.2350000000000001,"15min":1.2909999999999999},"min":{"1min":0.95099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3700000000000001,"15min":1.419},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.60999999999999999},"max":{"1min":1.728,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.75}]},{"osd":5,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.359,"15min":1.3500000000000001},"min":{"1min":0.93700000000000006,"5min":0.872,"15min":0.82699999999999996},"max":{"1min":1.982,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.597},{"interface":"front","average":{"1min":1.401,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":0.85799999999999998,"5min":0.85299999999999998,"15min":0.81000000000000005},"max":{"1min":1.9159999999999999,"5min":1.9390000000000001,"15min":1.9810000000000001},"last":1.292}]},{"osd":6,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4099999999999999,"15min":1.4490000000000001},"min":{"1min":0.97699999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.96,"5min":2.145,"15min":2.1600000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.5640000000000001,"15min":1.5960000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":2.016,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.6859999999999999}]},{"osd":7,"last update":"Sat Oct 21 17:40:55 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4970000000000001,"15min":1.5189999999999999},"min":{"1min":0.998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8460000000000001,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.474},{"interface":"front","average":{"1min":1.554,"5min":1.5669999999999999,"15min":1.5900000000000001},"min":{"1min":0.95199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.052,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.8600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576032,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56852,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314988,"statfs":{"total":96636764160,"available":96578547712,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.1919999999999999,"5min":1.6439999999999999,"15min":1.71},"last":1.226},{"interface":"front","average":{"1min":1.167,"5min":1.1559999999999999,"15min":1.2250000000000001},"min":{"1min":0.54300000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.7709999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.627}]},{"osd":1,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.94299999999999995,"15min":1.038},"min":{"1min":0.51400000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.4239999999999999,"5min":1.835,"15min":1.835},"last":1.123},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.181,"15min":1.2789999999999999},"min":{"1min":0.57699999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.456,"5min":1.911,"15min":1.911},"last":1.585}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.002,"15min":1.0880000000000001},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.611,"5min":1.921,"15min":2.1850000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1439999999999999,"15min":1.2509999999999999},"min":{"1min":0.57699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.605,"5min":1.9870000000000001,"15min":2},"last":1.3140000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3180000000000001,"15min":1.4099999999999999},"min":{"1min":0.71399999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.502,"5min":2.262,"15min":2.262},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.3740000000000001,"15min":1.4379999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.9430000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.446}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.369,"15min":1.464},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6579999999999999,"5min":2.0310000000000001,"15min":2.5329999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.502,"15min":1.546},"min":{"1min":0.871,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.8069999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.8109999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4830000000000001,"15min":1.5629999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7789999999999999,"5min":2.4049999999999998,"15min":2.7410000000000001},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.304,"5min":1.4970000000000001,"15min":1.585},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.3380000000000001,"15min":2.6840000000000002},"last":1.9810000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5669999999999999,"15min":1.6419999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.8520000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.9330000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.5649999999999999,"15min":1.6679999999999999},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.766,"5min":2.4580000000000002,"15min":2.8940000000000001},"last":2.1629999999999998}]}]},{"osd":2,"up_from":13,"seq":55834576033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56852,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314988,"statfs":{"total":96636764160,"available":96578547712,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.90400000000000003,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.448,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.93200000000000005,"15min":1.0049999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1779999999999999,"5min":1.7529999999999999,"15min":2.0059999999999998},"last":1.103}]},{"osd":1,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.1359999999999999,"15min":1.175},"min":{"1min":0.55900000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.494,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.879},{"interface":"front","average":{"1min":1.085,"5min":1.222,"15min":1.327},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.958},"last":1.405}]},{"osd":3,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.141,"15min":1.1659999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.911,"5min":4.8609999999999998,"15min":4.8609999999999998},"last":1.264},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.421,"15min":1.456},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9770000000000001,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.3200000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.429,"15min":1.423},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.875,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.472},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.3680000000000001,"15min":1.4299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.0750000000000002,"5min":2.1699999999999999,"15min":2.5939999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3280000000000001,"15min":1.3939999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8879999999999999,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.585,"15min":1.5960000000000001},"min":{"1min":0.755,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2490000000000001,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4550000000000001,"15min":1.538},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1549999999999998,"5min":2.3450000000000002,"15min":2.3980000000000001},"last":1.714},{"interface":"front","average":{"1min":1.272,"5min":1.45,"15min":1.504},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.7390000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.371}]},{"osd":7,"last update":"Sat Oct 21 17:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.581,"15min":1.619},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.149,"5min":2.149,"15min":2.4630000000000001},"last":1.643},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.5329999999999999,"15min":1.577},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.036,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.607}]}]},{"osd":1,"up_from":13,"seq":55834576032,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52692,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319148,"statfs":{"total":96636764160,"available":96582807552,"internally_reserved":0,"allocated":3543040,"data_stored":1854698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.42399999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.976},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.115},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":1.756,"5min":1.8260000000000001,"15min":1.8400000000000001},"last":0.86499999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94399999999999995,"15min":1.0169999999999999},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2370000000000001,"5min":1.746,"15min":1.79},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.135,"15min":1.2010000000000001},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":1.8839999999999999},"last":1.345}]},{"osd":3,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.26,"15min":1.335},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8240000000000001,"5min":1.9239999999999999,"15min":2.129},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.208,"15min":1.252},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.437},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.3080000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.206,"15min":1.3140000000000001},"min":{"1min":0.61899999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7230000000000001,"5min":2.0139999999999998,"15min":2.0819999999999999},"last":1.143},{"interface":"front","average":{"1min":1.375,"5min":1.423,"15min":1.468},"min":{"1min":0.90900000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":2.0539999999999998,"5min":2.1070000000000002,"15min":2.383},"last":1.25}]},{"osd":5,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.425,"15min":1.522},"min":{"1min":0.92000000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.3969999999999998},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4079999999999999,"15min":1.492},"min":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8660000000000001,"5min":2.1000000000000001,"15min":2.3359999999999999},"last":1.3859999999999999}]},{"osd":6,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4650000000000001,"15min":1.5509999999999999},"min":{"1min":0.876,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0179999999999998,"5min":2.2069999999999999,"15min":2.4319999999999999},"last":1.546},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.385,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.794,"5min":2.1360000000000001,"15min":2.5310000000000001},"last":1.4730000000000001}]},{"osd":7,"last update":"Sat Oct 21 17:41:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.5549999999999999,"15min":1.621},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1259999999999999,"5min":2.1739999999999999,"15min":2.548},"last":1.581},{"interface":"front","average":{"1min":1.502,"5min":1.583,"15min":1.663},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9550000000000001,"5min":2.3730000000000002,"15min":2.492},"last":1.643}]}]},{"osd":0,"up_from":13,"seq":55834576032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53264,"kb_used_data":4040,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318576,"statfs":{"total":96636764160,"available":96582221824,"internally_reserved":0,"allocated":4136960,"data_stored":2445066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.93799999999999994,"15min":0.95399999999999996},"min":{"1min":0.36899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.323,"5min":2.9249999999999998,"15min":2.9249999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97799999999999998,"15min":1.0669999999999999},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.42199999999999999},"max":{"1min":1.357,"5min":1.718,"15min":1.748},"last":1.1719999999999999}]},{"osd":2,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.107,"15min":1.149},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.6259999999999999,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.2889999999999999},{"interface":"front","average":{"1min":1,"5min":1.02,"15min":1.145},"min":{"1min":0.45700000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.4730000000000001,"5min":1.708,"15min":1.851},"last":0.93500000000000005}]},{"osd":3,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.0589999999999999,"15min":1.0960000000000001},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5680000000000001,"5min":2.597,"15min":2.597},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.272,"15min":1.355},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4850000000000001,"5min":2.343,"15min":2.343},"last":1.5880000000000001}]},{"osd":4,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3169999999999999,"15min":1.3879999999999999},"min":{"1min":0.75,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.889,"5min":1.9570000000000001,"15min":2.032},"last":1.504},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5149999999999999,"15min":1.5700000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.806,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.796}]},{"osd":5,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3400000000000001,"15min":1.383},"min":{"1min":0.81499999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.639,"5min":1.925,"15min":2.1309999999999998},"last":1.869},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4830000000000001,"15min":1.54},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.851,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.4470000000000001}]},{"osd":6,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.389,"15min":1.4610000000000001},"min":{"1min":0.85699999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.722,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.629},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.607,"15min":1.6439999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.984,"5min":2.5259999999999998,"15min":2.5259999999999998},"last":1.714}]},{"osd":7,"last update":"Sat Oct 21 17:41:23 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4279999999999999,"15min":1.5089999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.016,"5min":2.1760000000000002,"15min":2.3100000000000001},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.514,"15min":1.5840000000000001},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.921,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.758}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-21T17:41:44.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-21T17:41:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.071+0000 7f410ffff640 1 Processor -- start 2023-10-21T17:41:45.073 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.071+0000 7f410ffff640 1 -- start start 2023-10-21T17:41:45.074 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410ffff640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f41100c8b40 0x7f41100c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100c94f0 con 0x7f41100aff70 2023-10-21T17:41:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410ffff640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41100ccbc0 con 0x7f411015cae0 2023-10-21T17:41:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410ffff640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41100ce1f0 con 0x7f41100c8b40 2023-10-21T17:41:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f41100aff70 0x7f41100b03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:39346/0 (socket says 172.21.15.53:39346) 2023-10-21T17:41:45.075 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.073+0000 7f410f7fe640 1 -- 172.21.15.53:0/2413218265 learned_addr learned my addr 172.21.15.53:0/2413218265 (peer_addr_for_me v1:172.21.15.53:0/0) 2023-10-21T17:41:45.077 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f4107fff640 1 --2- 172.21.15.53:0/2413218265 >> v2:172.21.15.124:3300/0 conn(0x7f41100c8b40 0x7f41100c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989903238 0 0) 0x7f41100c94f0 con 0x7f41100aff70 2023-10-21T17:41:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40fc003520 con 0x7f41100aff70 2023-10-21T17:41:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881567234 0 0) 0x7f40fc003520 con 0x7f41100aff70 2023-10-21T17:41:45.078 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 >> v2:172.21.15.124:3300/0 conn(0x7f41100c8b40 msgr2=0x7f41100c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 --2- 172.21.15.53:0/2413218265 >> v2:172.21.15.124:3300/0 conn(0x7f41100c8b40 0x7f41100c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 >> v1:172.21.15.124:6789/0 conn(0x7f411015cae0 legacy=0x7f41100cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100c9720 con 0x7f41100aff70 2023-10-21T17:41:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3860881559 0 0) 0x7f41080019f0 con 0x7f41100aff70 2023-10-21T17:41:45.079 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.074+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 >> v1:172.21.15.53:6789/0 conn(0x7f41100aff70 legacy=0x7f41100b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40fc0039c0 0x7f40fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc0047c0 0x7f40fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 0x7f40fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41100ce1f0 con 0x7f40fc005180 2023-10-21T17:41:45.080 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc00c080 con 0x7f40fc0039c0 2023-10-21T17:41:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc00c240 con 0x7f40fc0047c0 2023-10-21T17:41:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410f7fe640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 0x7f40fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f410effd640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40fc0039c0 0x7f40fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.075+0000 7f4107fff640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc0047c0 0x7f40fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.081 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 -- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc0047c0 msgr2=0x7f40fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:45.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc0047c0 0x7f40fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 -- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40fc0039c0 msgr2=0x7f40fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:45.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40fc0039c0 0x7f40fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.082 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 -- 172.21.15.53:0/2413218265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc003520 con 0x7f40fc005180 2023-10-21T17:41:45.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410effd640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40fc0039c0 0x7f40fc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-21T17:41:45.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.076+0000 7f410f7fe640 1 -- 172.21.15.53:0/2413218265 reap_dead start 2023-10-21T17:41:45.083 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410f7fe640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 0x7f40fc00b980 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f410800a760 tx=0x7f40fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809a18e5874af24c server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108034630 con 0x7f40fc005180 2023-10-21T17:41:45.086 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410802f070 con 0x7f40fc005180 2023-10-21T17:41:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410e7fc640 1 -- 172.21.15.53:0/2413218265 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108034a10 con 0x7f40fc005180 2023-10-21T17:41:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410ffff640 1 -- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 msgr2=0x7f40fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.077+0000 7f410ffff640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 0x7f40fc00b980 secure :-1 s=READY pgs=2989 cs=0 l=1 rev1=1 crypto rx=0x7f410800a760 tx=0x7f40fc0057f0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.078+0000 7f410ffff640 1 -- 172.21.15.53:0/2413218265 shutdown_connections 2023-10-21T17:41:45.087 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.078+0000 7f410ffff640 1 --2- 172.21.15.53:0/2413218265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f40fc005180 0x7f40fc00b980 unknown :-1 s=CLOSED pgs=2989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.078+0000 7f410ffff640 1 -- 172.21.15.53:0/2413218265 >> 172.21.15.53:0/2413218265 conn(0x7f411006c050 msgr2=0x7f411006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.078+0000 7f410ffff640 1 -- 172.21.15.53:0/2413218265 shutdown_connections 2023-10-21T17:41:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.078+0000 7f410ffff640 1 -- 172.21.15.53:0/2413218265 wait complete. 2023-10-21T17:41:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.079+0000 7f410ffff640 1 Processor -- start 2023-10-21T17:41:45.088 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.079+0000 7f410ffff640 1 -- start start 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc005180 0x7f4110159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41100b38e0 0x7f41100c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41100c4b30 con 0x7f41100c46a0 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4110159f70 con 0x7f41100b38e0 2023-10-21T17:41:45.089 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.080+0000 7f410ffff640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41100b1110 con 0x7f40fc005180 2023-10-21T17:41:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47164/0 (socket says 172.21.15.53:47164) 2023-10-21T17:41:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 -- 172.21.15.53:0/683295460 learned_addr learned my addr 172.21.15.53:0/683295460 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:45.090 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 -- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc005180 msgr2=0x7f4110159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410effd640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc005180 0x7f4110159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f4107fff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41100b38e0 0x7f41100c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc005180 0x7f4110159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 -- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41100b38e0 msgr2=0x7f41100c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41100b38e0 0x7f41100c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.091 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 -- 172.21.15.53:0/683295460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100b13b0 con 0x7f41100c46a0 2023-10-21T17:41:45.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.081+0000 7f410f7fe640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f4108002230 tx=0x7f4108001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.082+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108001930 con 0x7f41100c46a0 2023-10-21T17:41:45.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.082+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4108004b60 con 0x7f41100c46a0 2023-10-21T17:41:45.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.082+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41080347b0 con 0x7f41100c46a0 2023-10-21T17:41:45.092 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.082+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41100b1690 con 0x7f41100c46a0 2023-10-21T17:41:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.082+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41100b1c50 con 0x7f41100c46a0 2023-10-21T17:41:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.083+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4110074150 con 0x7f41100c46a0 2023-10-21T17:41:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.084+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4108031030 con 0x7f41100c46a0 2023-10-21T17:41:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.085+0000 7f410cff9640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 0x7f40f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.093 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.085+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f41080aaf50 con 0x7f41100c46a0 2023-10-21T17:41:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.085+0000 7f410effd640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 0x7f40f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.094 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.086+0000 7f410effd640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 0x7f40f006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4100007fd0 tx=0x7f4100009040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.095 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.089+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4108075b80 con 0x7f41100c46a0 2023-10-21T17:41:45.282 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.281+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f411015ba50 con 0x7f41100c46a0 2023-10-21T17:41:45.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.286+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v94)=0 v94) v1 ==== 123+0+0 (secure 0 0 0) 0x7f41080791a0 con 0x7f41100c46a0 2023-10-21T17:41:45.287 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.286+0000 7f410cff9640 1 -- 172.21.15.53:0/683295460 <== mon.0 v2:172.21.15.53:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4108075b80 con 0x7f41100c46a0 2023-10-21T17:41:45.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 msgr2=0x7f40f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 0x7f40f006d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4100007fd0 tx=0x7f4100009040 comp rx=0 tx=0).stop 2023-10-21T17:41:45.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 msgr2=0x7f411014dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7f4108002230 tx=0x7f4108001cb0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.291 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 shutdown_connections 2023-10-21T17:41:45.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f40f006af90 0x7f40f006d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41100c46a0 0x7f411014dc80 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41100b38e0 0x7f41100c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.292 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 --2- 172.21.15.53:0/683295460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40fc005180 0x7f4110159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 >> 172.21.15.53:0/683295460 conn(0x7f411006c050 msgr2=0x7f411015bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:45.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 shutdown_connections 2023-10-21T17:41:45.293 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.289+0000 7f410ffff640 1 -- 172.21.15.53:0/683295460 wait complete. 2023-10-21T17:41:45.310 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-21T17:41:45.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-21T17:41:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.541+0000 7f315514e640 1 Processor -- start 2023-10-21T17:41:45.543 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.541+0000 7f315514e640 1 -- start start 2023-10-21T17:41:45.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f315514e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3150159ac0 0x7f315006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.545 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f315514e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31500ad790 0x7f31500702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f315514e640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315006dee0 con 0x7f315015e330 2023-10-21T17:41:45.546 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f315514e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315015d850 con 0x7f3150159ac0 2023-10-21T17:41:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f315514e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315015db30 con 0x7f31500ad790 2023-10-21T17:41:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f314ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3150159ac0 0x7f315006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f314ed9d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3150159ac0 0x7f315006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47180/0 (socket says 172.21.15.53:47180) 2023-10-21T17:41:45.547 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.543+0000 7f314ed9d640 1 -- 172.21.15.53:0/220632548 learned_addr learned my addr 172.21.15.53:0/220632548 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.544+0000 7f314ed9d640 1 -- 172.21.15.53:0/220632548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31500ad790 msgr2=0x7f31500702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:45.548 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.544+0000 7f314ed9d640 1 --2- 172.21.15.53:0/220632548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31500ad790 0x7f31500702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.544+0000 7f314ed9d640 1 -- 172.21.15.53:0/220632548 >> v1:172.21.15.124:6789/0 conn(0x7f315015e330 legacy=0x7f315006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-21T17:41:45.549 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.544+0000 7f314ed9d640 1 -- 172.21.15.53:0/220632548 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315015de10 con 0x7f3150159ac0 2023-10-21T17:41:45.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.545+0000 7f314ed9d640 1 --2- 172.21.15.53:0/220632548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3150159ac0 0x7f315006d5e0 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f3140009870 tx=0x7f314002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b70da3a0e85466 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.545+0000 7f314dd9b640 1 -- 172.21.15.53:0/220632548 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140031640 con 0x7f3150159ac0 2023-10-21T17:41:45.553 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.545+0000 7f314dd9b640 1 -- 172.21.15.53:0/220632548 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f31400317a0 con 0x7f3150159ac0 2023-10-21T17:41:45.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.545+0000 7f314dd9b640 1 -- 172.21.15.53:0/220632548 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140031b50 con 0x7f3150159ac0 2023-10-21T17:41:45.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 -- 172.21.15.53:0/220632548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3150159ac0 msgr2=0x7f315006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.554 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 --2- 172.21.15.53:0/220632548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3150159ac0 0x7f315006d5e0 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7f3140009870 tx=0x7f314002b8f0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 -- 172.21.15.53:0/220632548 shutdown_connections 2023-10-21T17:41:45.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 --2- 172.21.15.53:0/220632548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f31500ad790 0x7f31500702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.555 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 --2- 172.21.15.53:0/220632548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3150159ac0 0x7f315006d5e0 unknown :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 -- 172.21.15.53:0/220632548 >> 172.21.15.53:0/220632548 conn(0x7f315006a380 msgr2=0x7f31500b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:45.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 -- 172.21.15.53:0/220632548 shutdown_connections 2023-10-21T17:41:45.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.546+0000 7f315514e640 1 -- 172.21.15.53:0/220632548 wait complete. 2023-10-21T17:41:45.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.548+0000 7f315514e640 1 Processor -- start 2023-10-21T17:41:45.556 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.548+0000 7f315514e640 1 -- start start 2023-10-21T17:41:45.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 0x7f315014c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f315015e330 0x7f315014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.557 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315014e1f0 con 0x7f31500ad790 2023-10-21T17:41:45.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315014a4e0 con 0x7f315015e330 2023-10-21T17:41:45.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f315514e640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3150158c20 con 0x7f3150159ac0 2023-10-21T17:41:45.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f314ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f314ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:47184/0 (socket says 172.21.15.53:47184) 2023-10-21T17:41:45.558 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.549+0000 7f314ed9d640 1 -- 172.21.15.53:0/2066304283 learned_addr learned my addr 172.21.15.53:0/2066304283 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-10-21T17:41:45.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314e59c640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 0x7f315014c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 -- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 msgr2=0x7f315014c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314f59e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f315015e330 0x7f315014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.559 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 0x7f315014c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 -- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f315015e330 msgr2=0x7f315014c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f315015e330 0x7f315014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 -- 172.21.15.53:0/2066304283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31400094d0 con 0x7f31500ad790 2023-10-21T17:41:45.560 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314e59c640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 0x7f315014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-21T17:41:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f314ed9d640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7f3140000c00 tx=0x7f31400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.550+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314003ac60 con 0x7f31500ad790 2023-10-21T17:41:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.551+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f314003ade0 con 0x7f31500ad790 2023-10-21T17:41:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.551+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140044650 con 0x7f31500ad790 2023-10-21T17:41:45.561 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.551+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315014dc30 con 0x7f31500ad790 2023-10-21T17:41:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.551+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315014a780 con 0x7f31500ad790 2023-10-21T17:41:45.562 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.552+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3150067490 con 0x7f31500ad790 2023-10-21T17:41:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.553+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 153) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f314002f050 con 0x7f31500ad790 2023-10-21T17:41:45.563 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.554+0000 7f31377fe640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 0x7f313c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-21T17:41:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.554+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f31400abc70 con 0x7f31500ad790 2023-10-21T17:41:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.554+0000 7f314e59c640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 0x7f313c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-21T17:41:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.555+0000 7f314e59c640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 0x7f313c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3144006fd0 tx=0x7f3144008040 comp rx=0 tx=0).ready entity=mgr.38828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-21T17:41:45.564 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.558+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31400768a0 con 0x7f31500ad790 2023-10-21T17:41:45.792 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.791+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31500b0ff0 con 0x7f31500ad790 2023-10-21T17:41:45.793 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.792+0000 7f31377fe640 1 -- 172.21.15.53:0/2066304283 <== mon.0 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+3711 (secure 0 0 0) 0x7f3140035090 con 0x7f31500ad790 2023-10-21T17:41:45.794 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-10-21T17:41:45.794 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":145,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10057,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3306109649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3306109649},{"type":"v1","addr":"172.21.15.124:6836","nonce":3306109649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":12699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/2601438765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":2601438765},{"type":"v1","addr":"172.21.15.53:6833","nonce":2601438765}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3218875748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3218875748},{"type":"v1","addr":"172.21.15.53:6835","nonce":3218875748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":139},{"gid":38909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/587217669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":587217669},{"type":"v1","addr":"172.21.15.124:6837","nonce":587217669}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":145}],"filesystems":[]} 2023-10-21T17:41:45.795 INFO:teuthology.orchestra.run.smithi053.stderr:dumped fsmap epoch 145 2023-10-21T17:41:45.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 msgr2=0x7f313c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 0x7f313c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3144006fd0 tx=0x7f3144008040 comp rx=0 tx=0).stop 2023-10-21T17:41:45.797 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 msgr2=0x7f31501586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-21T17:41:45.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 secure :-1 s=READY pgs=2992 cs=0 l=1 rev1=1 crypto rx=0x7f3140000c00 tx=0x7f31400046c0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 shutdown_connections 2023-10-21T17:41:45.798 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:6832/3322763271,v1:172.21.15.124:6833/3322763271] conn(0x7f313c06af90 0x7f313c06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f315015e330 0x7f315014c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3150159ac0 0x7f315014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.799 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 --2- 172.21.15.53:0/2066304283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f31500ad790 0x7f31501586e0 unknown :-1 s=CLOSED pgs=2992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-21T17:41:45.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 >> 172.21.15.53:0/2066304283 conn(0x7f315006a380 msgr2=0x7f3150156210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-21T17:41:45.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 shutdown_connections 2023-10-21T17:41:45.800 INFO:teuthology.orchestra.run.smithi053.stderr:2023-10-21T17:41:45.795+0000 7f315514e640 1 -- 172.21.15.53:0/2066304283 wait complete. 2023-10-21T17:41:45.817 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-21T17:41:45.817 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-21T17:41:45.817 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-21T17:41:45.818 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:45.839 INFO:tasks.ceph.mds.a:Stopped 2023-10-21T17:41:45.839 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-21T17:41:45.839 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:45.905 INFO:tasks.ceph.mds.c:Stopped 2023-10-21T17:41:45.906 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-21T17:41:45.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:45.930 INFO:tasks.ceph.mds.b:Stopped 2023-10-21T17:41:45.931 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-21T17:41:45.931 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:45.997 INFO:tasks.ceph.mds.d:Stopped 2023-10-21T17:41:45.998 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-21T17:41:45.999 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-21T17:41:45.999 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.069 INFO:tasks.ceph.osd.0:Stopped 2023-10-21T17:41:46.069 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-21T17:41:46.070 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.130 INFO:tasks.ceph.osd.1:Stopped 2023-10-21T17:41:46.131 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-21T17:41:46.132 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.197 INFO:tasks.ceph.osd.2:Stopped 2023-10-21T17:41:46.197 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-21T17:41:46.198 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.277 INFO:tasks.ceph.osd.3:Stopped 2023-10-21T17:41:46.278 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-21T17:41:46.278 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.339 INFO:tasks.ceph.osd.4:Stopped 2023-10-21T17:41:46.340 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-21T17:41:46.340 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.402 INFO:tasks.ceph.osd.5:Stopped 2023-10-21T17:41:46.402 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-21T17:41:46.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.473 INFO:tasks.ceph.osd.6:Stopped 2023-10-21T17:41:46.474 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-21T17:41:46.474 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:46.571 INFO:tasks.ceph.osd.7:Stopped 2023-10-21T17:41:46.572 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-21T17:41:46.572 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-21T17:41:46.572 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:51.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:51.448 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:52.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-10-21T17:41:52.674 INFO:tasks.ceph.mgr.y:Stopped 2023-10-21T17:41:52.675 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-21T17:41:52.675 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:52.759 INFO:tasks.ceph.mgr.x:Stopped 2023-10-21T17:41:52.760 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-21T17:41:52.760 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-21T17:41:52.760 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:52.834 INFO:tasks.ceph.mon.a:Stopped 2023-10-21T17:41:52.835 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-21T17:41:52.835 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:52.895 INFO:tasks.ceph.mon.b:Stopped 2023-10-21T17:41:52.896 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-21T17:41:52.896 INFO:teuthology.orchestra.run:waiting for 300 2023-10-21T17:41:52.931 INFO:tasks.ceph.mon.c:Stopped 2023-10-21T17:41:52.931 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-21T17:41:52.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-21T17:41:53.046 INFO:teuthology.orchestra.run.smithi053.stdout:2023-10-21T16:16:32.316263+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-10-21T17:41:53.047 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-21T17:41:53.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-21T17:41:53.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-21T17:41:53.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-21T17:41:53.310 INFO:teuthology.orchestra.run.smithi053.stdout:2023-10-21T16:16:32.316263+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-10-21T17:41:53.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:41:53.312 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-21T17:41:53.579 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:41:53.580 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-21T17:41:53.755 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:41:53.755 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-21T17:41:53.929 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:41:53.929 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-21T17:41:54.113 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T17:41:54.114 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-21T17:41:54.624 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T17:41:54.624 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-21T17:41:54.783 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T17:41:54.784 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-21T17:41:54.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T17:41:54.954 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-21T17:41:55.137 INFO:tasks.ceph:Archiving mon data... 2023-10-21T17:41:55.139 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/data/mon.a.tgz 2023-10-21T17:41:55.139 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2023-10-21T17:41:55.199 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.2BnOJLMnbj 2023-10-21T17:41:55.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2BnOJLMnbj 2023-10-21T17:41:55.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.2BnOJLMnbj 2023-10-21T17:41:56.084 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.2BnOJLMnbj is 4MB 2023-10-21T17:41:56.253 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.2BnOJLMnbj 2023-10-21T17:41:56.316 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/data/mon.b.tgz 2023-10-21T17:41:56.316 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-21T17:41:56.374 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.YKmV877oMG 2023-10-21T17:41:56.374 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YKmV877oMG 2023-10-21T17:41:57.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-10-21T17:41:57.184 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.YKmV877oMG 2023-10-21T17:41:57.334 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.YKmV877oMG is 4MB 2023-10-21T17:41:57.490 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.YKmV877oMG 2023-10-21T17:41:57.551 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/data/mon.c.tgz 2023-10-21T17:41:57.551 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2023-10-21T17:41:57.605 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.5Cg0yUya3N 2023-10-21T17:41:57.605 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5Cg0yUya3N 2023-10-21T17:41:58.289 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.5Cg0yUya3N 2023-10-21T17:41:58.437 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.5Cg0yUya3N is 4MB 2023-10-21T17:41:58.599 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.5Cg0yUya3N 2023-10-21T17:41:58.659 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-21T17:41:58.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-21T17:41:58.669 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-21T17:41:58.775 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-21T17:41:58.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-21T17:41:58.785 DEBUG:teuthology.orchestra.run.smithi124:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-21T17:41:58.890 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-21T17:41:58.895 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-21T17:41:58.908 INFO:tasks.ceph:Archiving crash dumps... 2023-10-21T17:41:58.910 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi053/crash 2023-10-21T17:41:58.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-21T17:41:58.990 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi124/crash 2023-10-21T17:41:58.991 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-21T17:41:59.087 INFO:tasks.ceph:Shutting down logrotate 2023-10-21T17:41:59.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:59.097 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:59.228 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:59.338 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-21T17:41:59.441 INFO:tasks.ceph:Compressing logs... 2023-10-21T17:41:59.442 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-21T17:41:59.451 DEBUG:teuthology.orchestra.run.smithi124:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-21T17:41:59.542 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87797.log 2023-10-21T17:41:59.543 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-21T17:41:59.544 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87797.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.87797.log.gz 2023-10-21T17:41:59.544 INFO:teuthology.orchestra.run.smithi124.stderr: /var/log/ceph/ceph-client.admin.87755.log 2023-10-21T17:41:59.545 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87626.log 2023-10-21T17:41:59.546 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87755.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87755.log.gz 2023-10-21T17:41:59.547 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93313.log 2023-10-21T17:41:59.547 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99867.log 2023-10-21T17:41:59.548 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87626.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87626.log.gz 2023-10-21T17:41:59.548 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87837.log 2023-10-21T17:41:59.548 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.93313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.93313.log.gz 2023-10-21T17:41:59.550 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108924.log 2023-10-21T17:41:59.550 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99867.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99867.log.gz 2023-10-21T17:41:59.550 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156405.log 2023-10-21T17:41:59.551 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87878.log 2023-10-21T17:41:59.552 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87837.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-21T17:41:59.552 INFO:teuthology.orchestra.run.smithi124.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87837.log.gz 2023-10-21T17:41:59.552 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87878.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-21T17:41:59.552 INFO:teuthology.orchestra.run.smithi124.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87878.log.gz 2023-10-21T17:41:59.553 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-21T17:41:59.554 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87667.log 2023-10-21T17:41:59.555 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93009.log 2023-10-21T17:41:59.555 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156405.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.108924.log.gz 2023-10-21T17:41:59.556 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156405.log.gz 2023-10-21T17:41:59.556 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119713.log 2023-10-21T17:41:59.556 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112445.log 2023-10-21T17:41:59.556 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93009.log.gz 2023-10-21T17:41:59.556 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119713.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119713.log.gz 2023-10-21T17:41:59.557 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156964.log 2023-10-21T17:41:59.559 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-21T17:41:59.559 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87667.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87667.log.gz 2023-10-21T17:41:59.559 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-21T17:41:59.560 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87707.log 2023-10-21T17:41:59.560 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112445.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112445.log.gz 2023-10-21T17:41:59.560 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112361.log 2023-10-21T17:41:59.561 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152606.log 2023-10-21T17:41:59.561 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156964.log.gz 2023-10-21T17:41:59.561 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112361.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112361.log.gz 2023-10-21T17:41:59.561 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92045.log 2023-10-21T17:41:59.561 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152606.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152606.log.gz 2023-10-21T17:41:59.562 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137312.log 2023-10-21T17:41:59.563 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-21T17:41:59.563 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87707.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87707.log.gz 2023-10-21T17:41:59.564 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92045.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92045.log.gz 2023-10-21T17:41:59.564 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102067.log 2023-10-21T17:41:59.564 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137312.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137312.log.gz 2023-10-21T17:41:59.565 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120741.log 2023-10-21T17:41:59.565 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102067.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102067.log.gz 2023-10-21T17:41:59.565 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124418.log 2023-10-21T17:41:59.567 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93706.log 2023-10-21T17:41:59.567 INFO:teuthology.orchestra.run.smithi124.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-21T17:41:59.568 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-21T17:41:59.568 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.3.93706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.93706.log.gz 2023-10-21T17:41:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157072.log 2023-10-21T17:41:59.569 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120741.log.gz 2023-10-21T17:41:59.569 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124418.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124418.log.gz 2023-10-21T17:41:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128738.log 2023-10-21T17:41:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157072.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157072.log.gz 2023-10-21T17:41:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146059.log 2023-10-21T17:41:59.570 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135808.log 2023-10-21T17:41:59.570 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128738.log.gz 2023-10-21T17:41:59.572 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-21T17:41:59.572 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-21T17:41:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146059.log.gz 2023-10-21T17:41:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136181.log 2023-10-21T17:41:59.573 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135808.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135808.log.gz 2023-10-21T17:41:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136020.log 2023-10-21T17:41:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103706.log 2023-10-21T17:41:59.574 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136181.log.gz 2023-10-21T17:41:59.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136020.log.gz 2023-10-21T17:41:59.575 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87918.log 2023-10-21T17:41:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117716.log 2023-10-21T17:41:59.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103706.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-21T17:41:59.577 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103706.log.gz 2023-10-21T17:41:59.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117716.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117716.log.gz 2023-10-21T17:41:59.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91737.log 2023-10-21T17:41:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149518.log 2023-10-21T17:41:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133726.log 2023-10-21T17:41:59.579 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91737.log.gz 2023-10-21T17:41:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149518.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149518.log.gz 2023-10-21T17:41:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142355.log 2023-10-21T17:41:59.581 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-21T17:41:59.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158030.log 2023-10-21T17:41:59.581 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.133726.log.gz 2023-10-21T17:41:59.582 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.87918.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87918.log.gz 2023-10-21T17:41:59.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108696.log 2023-10-21T17:41:59.583 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.142355.log.gz 2023-10-21T17:41:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158030.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158030.log.gz 2023-10-21T17:41:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147971.log 2023-10-21T17:41:59.584 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108696.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108696.log.gz 2023-10-21T17:41:59.585 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152930.log 2023-10-21T17:41:59.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148920.log 2023-10-21T17:41:59.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147971.log.gz 2023-10-21T17:41:59.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152930.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.152930.log.gz 2023-10-21T17:41:59.587 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mon.c.log: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-21T17:41:59.588 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95099.log 2023-10-21T17:41:59.589 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116395.log 2023-10-21T17:41:59.589 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148920.log.gz 2023-10-21T17:41:59.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95099.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95099.log.gz 2023-10-21T17:41:59.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97674.log 2023-10-21T17:41:59.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125285.log 2023-10-21T17:41:59.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97674.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.116395.log.gz 2023-10-21T17:41:59.592 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97674.log.gz 2023-10-21T17:41:59.593 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115751.log 2023-10-21T17:41:59.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126773.log 2023-10-21T17:41:59.595 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125285.log.gz 2023-10-21T17:41:59.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115751.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115751.log.gz 2023-10-21T17:41:59.596 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125994.log 2023-10-21T17:41:59.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126773.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.126773.log.gz 2023-10-21T17:41:59.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116918.log 2023-10-21T17:41:59.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92129.log 2023-10-21T17:41:59.598 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125994.log.gz 2023-10-21T17:41:59.599 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116918.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116918.log.gz 2023-10-21T17:41:59.599 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125113.log 2023-10-21T17:41:59.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101262.log 2023-10-21T17:41:59.600 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92129.log.gz 2023-10-21T17:41:59.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125113.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125113.log.gz 2023-10-21T17:41:59.602 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150795.log 2023-10-21T17:41:59.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.101262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101262.log.gz 2023-10-21T17:41:59.603 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109363.log 2023-10-21T17:41:59.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150795.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150795.log.gz 2023-10-21T17:41:59.604 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108177.log 2023-10-21T17:41:59.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98577.log 2023-10-21T17:41:59.605 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109363.log.gz 2023-10-21T17:41:59.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108177.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108177.log.gz 2023-10-21T17:41:59.606 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107229.log 2023-10-21T17:41:59.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138898.log 2023-10-21T17:41:59.608 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98577.log.gz 2023-10-21T17:41:59.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107229.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107229.log.gz 2023-10-21T17:41:59.609 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152480.log 2023-10-21T17:41:59.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138898.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138898.log.gz 2023-10-21T17:41:59.610 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109464.log 2023-10-21T17:41:59.611 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152480.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.152480.log.gz /var/log/ceph/ceph-client.admin.141821.log 2023-10-21T17:41:59.611 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109464.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109464.log.gz 2023-10-21T17:41:59.613 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134039.log 2023-10-21T17:41:59.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119755.log 2023-10-21T17:41:59.614 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141821.log.gz 2023-10-21T17:41:59.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134039.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134039.log.gz 2023-10-21T17:41:59.615 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112846.log 2023-10-21T17:41:59.616 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121119.log 2023-10-21T17:41:59.616 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119755.log.gz 2023-10-21T17:41:59.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112846.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112846.log.gz 2023-10-21T17:41:59.617 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.153860.log 2023-10-21T17:41:59.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142135.log 2023-10-21T17:41:59.618 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121119.log.gz 2023-10-21T17:41:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153860.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153860.log.gz 2023-10-21T17:41:59.619 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142836.log 2023-10-21T17:41:59.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111796.log 2023-10-21T17:41:59.621 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142135.log.gz 2023-10-21T17:41:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142836.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142836.log.gz 2023-10-21T17:41:59.622 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158366.log 2023-10-21T17:41:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111796.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111796.log.gz 2023-10-21T17:41:59.623 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103588.log 2023-10-21T17:41:59.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122164.log 2023-10-21T17:41:59.624 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158366.log.gz 2023-10-21T17:41:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103588.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103588.log.gz 2023-10-21T17:41:59.625 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94144.log 2023-10-21T17:41:59.626 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136265.log 2023-10-21T17:41:59.627 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122164.log.gz 2023-10-21T17:41:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94144.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94144.log.gz 2023-10-21T17:41:59.628 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114701.log 2023-10-21T17:41:59.629 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116059.log 2023-10-21T17:41:59.629 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136265.log.gz 2023-10-21T17:41:59.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114701.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114701.log.gz 2023-10-21T17:41:59.630 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124334.log 2023-10-21T17:41:59.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100671.log 2023-10-21T17:41:59.632 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116059.log.gz 2023-10-21T17:41:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124334.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124334.log.gz 2023-10-21T17:41:59.633 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114302.log 2023-10-21T17:41:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100671.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100671.log.gz 2023-10-21T17:41:59.634 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129463.log 2023-10-21T17:41:59.635 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114302.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81424.log 2023-10-21T17:41:59.635 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114302.log.gz 2023-10-21T17:41:59.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.129463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129463.log.gz 2023-10-21T17:41:59.636 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103765.log 2023-10-21T17:41:59.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.tmp-client.admin.81424.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81424.log.gz 2023-10-21T17:41:59.637 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97800.log 2023-10-21T17:41:59.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103765.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103765.log.gz 2023-10-21T17:41:59.639 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157946.log 2023-10-21T17:41:59.639 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123979.log 2023-10-21T17:41:59.640 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97800.log.gz 2023-10-21T17:41:59.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157946.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157946.log.gz 2023-10-21T17:41:59.641 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112930.log 2023-10-21T17:41:59.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136580.log 2023-10-21T17:41:59.642 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123979.log.gz 2023-10-21T17:41:59.643 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112930.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112930.log.gz 2023-10-21T17:41:59.644 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91687.log 2023-10-21T17:41:59.645 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136580.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136580.log.gz 2023-10-21T17:41:59.645 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158156.log 2023-10-21T17:41:59.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140459.log 2023-10-21T17:41:59.646 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91687.log.gz 2023-10-21T17:41:59.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158156.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158156.log.gz 2023-10-21T17:41:59.647 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100587.log 2023-10-21T17:41:59.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140459.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140459.log.gz 2023-10-21T17:41:59.649 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129000.log 2023-10-21T17:41:59.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98709.log 2023-10-21T17:41:59.650 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100587.log.gz 2023-10-21T17:41:59.651 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129000.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129000.log.gz 2023-10-21T17:41:59.651 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112582.log 2023-10-21T17:41:59.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155815.log 2023-10-21T17:41:59.653 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98709.log.gz 2023-10-21T17:41:59.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112582.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112582.log.gz 2023-10-21T17:41:59.654 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151557.log 2023-10-21T17:41:59.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123853.log 2023-10-21T17:41:59.655 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155815.log.gz 2023-10-21T17:41:59.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151557.log.gz 2023-10-21T17:41:59.656 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156023.log 2023-10-21T17:41:59.657 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155117.log 2023-10-21T17:41:59.657 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123853.log.gz 2023-10-21T17:41:59.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156023.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156023.log.gz 2023-10-21T17:41:59.658 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150337.log 2023-10-21T17:41:59.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108840.log 2023-10-21T17:41:59.659 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155117.log.gz 2023-10-21T17:41:59.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150337.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150337.log.gz 2023-10-21T17:41:59.662 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108526.log 2023-10-21T17:41:59.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94801.log 2023-10-21T17:41:59.663 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108840.log.gz 2023-10-21T17:41:59.663 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108526.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.108526.log.gz 2023-10-21T17:41:59.664 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155897.log 2023-10-21T17:41:59.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94801.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94801.log.gz 2023-10-21T17:41:59.665 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96832.log 2023-10-21T17:41:59.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97758.log 2023-10-21T17:41:59.666 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155897.log.gz 2023-10-21T17:41:59.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96832.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96832.log.gz 2023-10-21T17:41:59.667 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136139.log 2023-10-21T17:41:59.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150607.log 2023-10-21T17:41:59.668 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97758.log.gz 2023-10-21T17:41:59.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145526.log 2023-10-21T17:41:59.669 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136139.log.gz 2023-10-21T17:41:59.670 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129652.log 2023-10-21T17:41:59.670 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.150607.log.gz 2023-10-21T17:41:59.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145526.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145526.log.gz 2023-10-21T17:41:59.671 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141268.log 2023-10-21T17:41:59.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147304.log 2023-10-21T17:41:59.673 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129652.log.gz 2023-10-21T17:41:59.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138288.log 2023-10-21T17:41:59.674 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141268.log.gz 2023-10-21T17:41:59.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147304.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147304.log.gz 2023-10-21T17:41:59.675 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127847.log 2023-10-21T17:41:59.676 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117590.log 2023-10-21T17:41:59.677 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.138288.log.gz 2023-10-21T17:41:59.678 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127847.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127847.log.gz -5 2023-10-21T17:41:59.678 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94130.log 2023-10-21T17:41:59.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117590.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117590.log.gz 2023-10-21T17:41:59.680 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151897.log 2023-10-21T17:41:59.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141693.log 2023-10-21T17:41:59.681 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94130.log.gz 2023-10-21T17:41:59.682 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117758.log 2023-10-21T17:41:59.682 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151897.log.gz 2023-10-21T17:41:59.684 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138120.log 2023-10-21T17:41:59.684 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141693.log.gz 2023-10-21T17:41:59.685 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117758.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117758.log.gz 2023-10-21T17:41:59.685 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105282.log 2023-10-21T17:41:59.686 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153337.log 2023-10-21T17:41:59.686 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138120.log.gz 2023-10-21T17:41:59.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.105282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124249.log 2023-10-21T17:41:59.688 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105282.log.gz 2023-10-21T17:41:59.689 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129210.log 2023-10-21T17:41:59.689 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153337.log.gz 2023-10-21T17:41:59.690 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124249.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124249.log.gz -5 2023-10-21T17:41:59.690 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.137610.log 2023-10-21T17:41:59.691 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107271.log 2023-10-21T17:41:59.692 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.129210.log.gz 2023-10-21T17:41:59.693 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126609.log 2023-10-21T17:41:59.693 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137610.log.gz 2023-10-21T17:41:59.694 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157386.log 2023-10-21T17:41:59.695 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107271.log.gz 2023-10-21T17:41:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126609.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.126609.log.gz 2023-10-21T17:41:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152107.log 2023-10-21T17:41:59.697 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155262.log 2023-10-21T17:41:59.697 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157386.log.gz 2023-10-21T17:41:59.698 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146227.log 2023-10-21T17:41:59.699 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152107.log.gz 2023-10-21T17:41:59.700 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124105.log 2023-10-21T17:41:59.700 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155262.log.gz 2023-10-21T17:41:59.701 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158809.log 2023-10-21T17:41:59.701 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146227.log.gz 2023-10-21T17:41:59.702 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150481.log 2023-10-21T17:41:59.702 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124105.log.gz 2023-10-21T17:41:59.703 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158809.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.158809.log.gz /var/log/ceph/ceph-client.admin.139386.log 2023-10-21T17:41:59.703 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.705 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98205.log 2023-10-21T17:41:59.705 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.150481.log.gz 2023-10-21T17:41:59.706 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.139386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154679.log 2023-10-21T17:41:59.706 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139386.log.gz 2023-10-21T17:41:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134949.log 2023-10-21T17:41:59.707 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98205.log.gz 2023-10-21T17:41:59.708 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93587.log 2023-10-21T17:41:59.708 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154679.log.gz 2023-10-21T17:41:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104773.log 2023-10-21T17:41:59.710 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134949.log.gz 2023-10-21T17:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93587.log.gz 2023-10-21T17:41:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151131.log 2023-10-21T17:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.104773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115242.log 2023-10-21T17:41:59.712 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104773.log.gz 2023-10-21T17:41:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151131.log: gzip -5 87.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.151131.log.gz -- 2023-10-21T17:41:59.713 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.140848.log 2023-10-21T17:41:59.714 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118920.log 2023-10-21T17:41:59.715 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115242.log.gz 2023-10-21T17:41:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140848.log: gzip -5 87.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.140848.log.gz -- 2023-10-21T17:41:59.716 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.135867.log 2023-10-21T17:41:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100537.log 2023-10-21T17:41:59.718 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118920.log.gz 2023-10-21T17:41:59.719 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135867.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.135867.log.gz /var/log/ceph/ceph-client.admin.146395.log 2023-10-21T17:41:59.719 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100537.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100537.log.gz 2023-10-21T17:41:59.720 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123758.log 2023-10-21T17:41:59.721 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.152402.log 2023-10-21T17:41:59.722 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146395.log.gz 2023-10-21T17:41:59.723 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123758.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123758.log.gz 2023-10-21T17:41:59.723 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97968.log 2023-10-21T17:41:59.724 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.152402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.152402.log.gz 2023-10-21T17:41:59.724 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98451.log 2023-10-21T17:41:59.725 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97968.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97968.log.gz 2023-10-21T17:41:59.725 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92171.log 2023-10-21T17:41:59.727 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154781.log 2023-10-21T17:41:59.727 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98451.log.gz 2023-10-21T17:41:59.728 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140078.log 2023-10-21T17:41:59.728 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92171.log.gz 2023-10-21T17:41:59.729 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109559.log 2023-10-21T17:41:59.729 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154781.log.gz 2023-10-21T17:41:59.730 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140078.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140078.log.gz 2023-10-21T17:41:59.731 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124021.log 2023-10-21T17:41:59.732 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145009.log 2023-10-21T17:41:59.732 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109559.log.gz 2023-10-21T17:41:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124021.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.124021.log.gz 2023-10-21T17:41:59.733 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154739.log 2023-10-21T17:41:59.735 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156279.log 2023-10-21T17:41:59.735 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145009.log.gz 2023-10-21T17:41:59.736 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134207.log 2023-10-21T17:41:59.736 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154739.log.gz 2023-10-21T17:41:59.737 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91831.log 2023-10-21T17:41:59.737 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156279.log.gz 2023-10-21T17:41:59.738 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134207.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134207.log.gz 2023-10-21T17:41:59.739 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153142.log 2023-10-21T17:41:59.740 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113490.log 2023-10-21T17:41:59.740 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91831.log.gz 2023-10-21T17:41:59.741 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153142.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153142.log.gz 2023-10-21T17:41:59.742 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138354.log 2023-10-21T17:41:59.743 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119046.log 2023-10-21T17:41:59.744 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113490.log.gz 2023-10-21T17:41:59.744 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138354.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.115879.log -- replaced with /var/log/ceph/ceph-client.admin.138354.log.gz 2023-10-21T17:41:59.745 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.746 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103226.log 2023-10-21T17:41:59.746 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119046.log.gz 2023-10-21T17:41:59.747 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115879.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115879.log.gz 2023-10-21T17:41:59.748 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103352.log 2023-10-21T17:41:59.749 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124063.log 2023-10-21T17:41:59.749 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103226.log.gz 2023-10-21T17:41:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103352.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103352.log.gz 2023-10-21T17:41:59.750 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133159.log 2023-10-21T17:41:59.752 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117860.log 2023-10-21T17:41:59.752 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124063.log.gz 2023-10-21T17:41:59.753 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133159.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133159.log.gz 2023-10-21T17:41:59.754 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108346.log 2023-10-21T17:41:59.755 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107060.log 2023-10-21T17:41:59.755 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117860.log.gz 2023-10-21T17:41:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108346.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108346.log.gz 2023-10-21T17:41:59.756 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94818.log 2023-10-21T17:41:59.758 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118878.log 2023-10-21T17:41:59.758 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107060.log.gz 2023-10-21T17:41:59.759 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156153.log 2023-10-21T17:41:59.759 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94818.log.gz 2023-10-21T17:41:59.760 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154553.log 2023-10-21T17:41:59.761 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118878.log.gz 2023-10-21T17:41:59.762 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156153.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156153.log.gz 2023-10-21T17:41:59.762 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128141.log 2023-10-21T17:41:59.763 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137060.log 2023-10-21T17:41:59.763 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154553.log.gz 2023-10-21T17:41:59.764 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121954.log 2023-10-21T17:41:59.764 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128141.log.gz 2023-10-21T17:41:59.766 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145263.log 2023-10-21T17:41:59.766 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137060.log.gz 2023-10-21T17:41:59.767 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102462.log 2023-10-21T17:41:59.767 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121954.log.gz 2023-10-21T17:41:59.769 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145263.log.gz 2023-10-21T17:41:59.769 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140974.log 2023-10-21T17:41:59.770 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119850.log 2023-10-21T17:41:59.770 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102462.log.gz 2023-10-21T17:41:59.771 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149959.log 2023-10-21T17:41:59.771 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140974.log.gz 2023-10-21T17:41:59.773 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119850.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.119850.log.gz 2023-10-21T17:41:59.773 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.154091.log 2023-10-21T17:41:59.774 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149959.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149959.log.gz 2023-10-21T17:41:59.774 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137737.log 2023-10-21T17:41:59.776 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98409.log 2023-10-21T17:41:59.776 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154091.log.gz 2023-10-21T17:41:59.777 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120174.log 2023-10-21T17:41:59.777 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137737.log.gz 2023-10-21T17:41:59.778 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146628.log 2023-10-21T17:41:59.778 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98409.log.gz 2023-10-21T17:41:59.779 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107669.log 2023-10-21T17:41:59.780 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120174.log.gz 2023-10-21T17:41:59.781 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99573.log 2023-10-21T17:41:59.781 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146628.log.gz 2023-10-21T17:41:59.782 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156065.log 2023-10-21T17:41:59.783 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107669.log.gz 2023-10-21T17:41:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99573.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99573.log.gz 2023-10-21T17:41:59.784 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149665.log 2023-10-21T17:41:59.785 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117987.log 2023-10-21T17:41:59.785 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156065.log.gz 2023-10-21T17:41:59.786 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116876.log 2023-10-21T17:41:59.786 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149665.log.gz 2023-10-21T17:41:59.788 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135962.log 2023-10-21T17:41:59.788 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117987.log.gz 2023-10-21T17:41:59.789 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116876.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116876.log.gz 2023-10-21T17:41:59.789 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118750.log 2023-10-21T17:41:59.791 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133904.log 2023-10-21T17:41:59.791 INFO:teuthology.orchestra.run.smithi053.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.135962.log.gz 2023-10-21T17:41:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115665.log 2023-10-21T17:41:59.792 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118750.log.gz 2023-10-21T17:41:59.793 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102974.log 2023-10-21T17:41:59.793 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133904.log.gz 2023-10-21T17:41:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103529.log 2023-10-21T17:41:59.795 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115665.log.gz 2023-10-21T17:41:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102974.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102974.log.gz 2023-10-21T17:41:59.796 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150169.log 2023-10-21T17:41:59.797 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133516.log 2023-10-21T17:41:59.798 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103529.log.gz 2023-10-21T17:41:59.799 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140638.log 2023-10-21T17:41:59.799 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150169.log.gz 2023-10-21T17:41:59.800 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114827.log 2023-10-21T17:41:59.800 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133516.log.gz 2023-10-21T17:41:59.801 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153615.log 2023-10-21T17:41:59.801 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.140638.log.gz 2023-10-21T17:41:59.803 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114827.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114827.log.gz 2023-10-21T17:41:59.805 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124147.log 2023-10-21T17:41:59.806 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153615.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153615.log.gz 2023-10-21T17:41:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148465.log 2023-10-21T17:41:59.807 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133810.log 2023-10-21T17:41:59.807 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124147.log.gz 2023-10-21T17:41:59.808 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148465.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148465.log.gz 2023-10-21T17:41:59.808 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158851.log 2023-10-21T17:41:59.809 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156573.log 2023-10-21T17:41:59.809 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133810.log.gz 2023-10-21T17:41:59.810 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97212.log 2023-10-21T17:41:59.810 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158851.log.gz 2023-10-21T17:41:59.811 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135159.log 2023-10-21T17:41:59.811 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156573.log.gz 2023-10-21T17:41:59.813 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139670.log 2023-10-21T17:41:59.813 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97212.log.gz 2023-10-21T17:41:59.814 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102546.log 2023-10-21T17:41:59.814 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135159.log.gz 2023-10-21T17:41:59.815 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.139670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100757.log 2023-10-21T17:41:59.815 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139670.log.gz 2023-10-21T17:41:59.816 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128566.log 2023-10-21T17:41:59.817 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102546.log.gz 2023-10-21T17:41:59.818 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137695.log 2023-10-21T17:41:59.818 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100757.log.gz 2023-10-21T17:41:59.819 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128566.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.128566.log.gz -5 2023-10-21T17:41:59.819 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.154949.log 2023-10-21T17:41:59.820 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137695.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137695.log.gz -5 2023-10-21T17:41:59.821 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99783.log 2023-10-21T17:41:59.822 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123377.log 2023-10-21T17:41:59.822 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154949.log.gz 2023-10-21T17:41:59.823 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111439.log 2023-10-21T17:41:59.823 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99783.log.gz 2023-10-21T17:41:59.824 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123377.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.123377.log.gz 2023-10-21T17:41:59.825 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102504.log 2023-10-21T17:41:59.826 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136453.log 2023-10-21T17:41:59.826 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111439.log.gz 2023-10-21T17:41:59.827 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108484.log 2023-10-21T17:41:59.827 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102504.log.gz 2023-10-21T17:41:59.828 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121543.log 2023-10-21T17:41:59.828 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136453.log.gz 2023-10-21T17:41:59.830 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111922.log 2023-10-21T17:41:59.830 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108484.log.gz 2023-10-21T17:41:59.831 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119340.log 2023-10-21T17:41:59.831 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121543.log.gz 2023-10-21T17:41:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153986.log 2023-10-21T17:41:59.832 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111922.log.gz 2023-10-21T17:41:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150085.log 2023-10-21T17:41:59.834 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119340.log.gz 2023-10-21T17:41:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153986.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.153986.log.gz -5 2023-10-21T17:41:59.835 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.149833.log 2023-10-21T17:41:59.836 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100365.log 2023-10-21T17:41:59.836 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150085.log.gz 2023-10-21T17:41:59.837 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115158.log 2023-10-21T17:41:59.838 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149833.log.gz 2023-10-21T17:41:59.839 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154301.log 2023-10-21T17:41:59.839 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100365.log.gz 2023-10-21T17:41:59.840 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115158.log.gz 2023-10-21T17:41:59.841 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137144.log 2023-10-21T17:41:59.842 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154301.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154301.log.gz 2023-10-21T17:41:59.842 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153487.log 2023-10-21T17:41:59.844 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113110.log 2023-10-21T17:41:59.844 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137144.log.gz 2023-10-21T17:41:59.844 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97128.log 2023-10-21T17:41:59.845 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153487.log.gz 2023-10-21T17:41:59.846 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146269.log 2023-10-21T17:41:59.846 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113110.log.gz 2023-10-21T17:41:59.848 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113196.log 2023-10-21T17:41:59.848 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97128.log.gz 2023-10-21T17:41:59.849 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146269.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146269.log.gz 2023-10-21T17:41:59.849 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146143.log 2023-10-21T17:41:59.850 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145347.log 2023-10-21T17:41:59.850 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113196.log.gz 2023-10-21T17:41:59.852 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127094.log 2023-10-21T17:41:59.852 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146143.log.gz 2023-10-21T17:41:59.854 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101584.log 2023-10-21T17:41:59.854 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145347.log.gz 2023-10-21T17:41:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127094.log.gz 2023-10-21T17:41:59.855 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120132.log 2023-10-21T17:41:59.856 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149088.log 2023-10-21T17:41:59.856 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101584.log.gz 2023-10-21T17:41:59.857 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93545.log 2023-10-21T17:41:59.857 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120132.log.gz 2023-10-21T17:41:59.858 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97464.log 2023-10-21T17:41:59.859 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.149088.log.gz 2023-10-21T17:41:59.860 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112319.log 2023-10-21T17:41:59.860 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93545.log.gz 2023-10-21T17:41:59.861 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150963.log 2023-10-21T17:41:59.861 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97464.log.gz 2023-10-21T17:41:59.862 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129126.log 2023-10-21T17:41:59.862 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112319.log.gz 2023-10-21T17:41:59.864 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101949.log 2023-10-21T17:41:59.864 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150963.log.gz 2023-10-21T17:41:59.865 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127595.log 2023-10-21T17:41:59.865 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129126.log.gz 2023-10-21T17:41:59.866 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154259.log 2023-10-21T17:41:59.866 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101949.log.gz 2023-10-21T17:41:59.867 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136748.log 2023-10-21T17:41:59.868 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127595.log.gz 2023-10-21T17:41:59.869 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154259.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154259.log.gz 2023-10-21T17:41:59.869 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128480.log 2023-10-21T17:41:59.870 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99279.log 2023-10-21T17:41:59.870 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136748.log.gz 2023-10-21T17:41:59.871 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106872.log 2023-10-21T17:41:59.872 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128480.log.gz 2023-10-21T17:41:59.873 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150523.log 2023-10-21T17:41:59.873 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99279.log.gz 2023-10-21T17:41:59.874 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106071.log 2023-10-21T17:41:59.874 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106872.log.gz 2023-10-21T17:41:59.875 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158977.log 2023-10-21T17:41:59.876 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150523.log.gz 2023-10-21T17:41:59.876 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106071.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106071.log.gz 2023-10-21T17:41:59.877 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157547.log 2023-10-21T17:41:59.878 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120531.log 2023-10-21T17:41:59.879 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158977.log.gz 2023-10-21T17:41:59.879 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157547.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157547.log.gz 2023-10-21T17:41:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115585.log 2023-10-21T17:41:59.880 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151939.log 2023-10-21T17:41:59.881 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120531.log.gz 2023-10-21T17:41:59.882 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151348.log 2023-10-21T17:41:59.882 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115585.log.gz 2023-10-21T17:41:59.883 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117674.log 2023-10-21T17:41:59.884 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151939.log.gz 2023-10-21T17:41:59.885 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109297.log 2023-10-21T17:41:59.885 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.151348.log.gz 2023-10-21T17:41:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94809.log 2023-10-21T17:41:59.886 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117674.log.gz 2023-10-21T17:41:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134123.log 2023-10-21T17:41:59.887 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109297.log.gz 2023-10-21T17:41:59.888 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95141.log 2023-10-21T17:41:59.889 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94809.log.gz 2023-10-21T17:41:59.890 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138078.log 2023-10-21T17:41:59.890 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134123.log.gz 2023-10-21T17:41:59.891 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95141.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95141.log.gz -5 2023-10-21T17:41:59.891 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111880.log 2023-10-21T17:41:59.892 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138078.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.138078.log.gz 2023-10-21T17:41:59.893 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114785.log 2023-10-21T17:41:59.894 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100629.log 2023-10-21T17:41:59.894 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111880.log.gz 2023-10-21T17:41:59.895 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100069.log 2023-10-21T17:41:59.895 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114785.log.gz 2023-10-21T17:41:59.896 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131987.log 2023-10-21T17:41:59.897 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100629.log.gz 2023-10-21T17:41:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100069.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100069.log.gz 2023-10-21T17:41:59.898 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158324.log 2023-10-21T17:41:59.899 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.131987.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131987.log.gz 2023-10-21T17:41:59.900 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100407.log 2023-10-21T17:41:59.901 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158324.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.158324.log.gz 2023-10-21T17:41:59.901 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91453.log 2023-10-21T17:41:59.902 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141226.log 2023-10-21T17:41:59.902 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100407.log.gz 2023-10-21T17:41:59.903 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91453.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91453.log.gz 2023-10-21T17:41:59.904 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153902.log 2023-10-21T17:41:59.905 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.135580.log 2023-10-21T17:41:59.905 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141226.log.gz 2023-10-21T17:41:59.906 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153902.log.gz 2023-10-21T17:41:59.906 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152065.log 2023-10-21T17:41:59.907 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.135580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.135580.log.gz 2023-10-21T17:41:59.907 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97716.log 2023-10-21T17:41:59.909 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152065.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152065.log.gz 2023-10-21T17:41:59.909 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121585.log 2023-10-21T17:41:59.910 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119797.log 2023-10-21T17:41:59.911 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97716.log.gz 2023-10-21T17:41:59.911 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121585.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121585.log.gz 2023-10-21T17:41:59.911 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107879.log 2023-10-21T17:41:59.913 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127805.log 2023-10-21T17:41:59.914 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119797.log.gz 2023-10-21T17:41:59.914 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107879.log: gzip -5 --verbose 88.4% -- /var/log/ceph/ceph-client.admin.128057.log -- replaced with /var/log/ceph/ceph-client.admin.107879.log.gz 2023-10-21T17:41:59.915 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.916 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127805.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.127805.log.gz --verbose 2023-10-21T17:41:59.916 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/ceph-client.admin.149266.log 2023-10-21T17:41:59.917 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133600.log 2023-10-21T17:41:59.917 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128057.log.gz 2023-10-21T17:41:59.918 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142229.log 2023-10-21T17:41:59.918 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149266.log.gz 2023-10-21T17:41:59.920 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97926.log 2023-10-21T17:41:59.920 INFO:teuthology.orchestra.run.smithi053.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.133600.log.gz 2023-10-21T17:41:59.920 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142229.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142229.log.gz 2023-10-21T17:41:59.921 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117800.log 2023-10-21T17:41:59.922 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148682.log 2023-10-21T17:41:59.923 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97926.log.gz 2023-10-21T17:41:59.923 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117800.log: gzip -5 --verbose 87.1% -- -- replaced with /var/log/ceph/ceph-client.admin.117800.log.gz 2023-10-21T17:41:59.923 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.136976.log 2023-10-21T17:41:59.925 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148682.log.gz 2023-10-21T17:41:59.925 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117945.log 2023-10-21T17:41:59.926 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136976.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136976.log.gz -5 2023-10-21T17:41:59.926 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101831.log 2023-10-21T17:41:59.927 INFO:teuthology.orchestra.run.smithi124.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-21T17:41:59.928 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134429.log 2023-10-21T17:41:59.928 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117945.log.gz 2023-10-21T17:41:59.929 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101831.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101831.log.gz 2023-10-21T17:41:59.929 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.133201.log 2023-10-21T17:41:59.930 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109177.log 2023-10-21T17:41:59.930 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134429.log.gz 2023-10-21T17:41:59.931 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94793.log 2023-10-21T17:41:59.932 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133201.log.gz 2023-10-21T17:41:59.932 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.109177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109177.log.gz 2023-10-21T17:41:59.933 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149749.log 2023-10-21T17:41:59.934 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157673.log 2023-10-21T17:41:59.934 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94793.log.gz 2023-10-21T17:41:59.935 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115284.log 2023-10-21T17:41:59.935 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149749.log.gz 2023-10-21T17:41:59.937 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113448.log 2023-10-21T17:41:59.937 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157673.log.gz 2023-10-21T17:41:59.938 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115284.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115284.log.gz 2023-10-21T17:41:59.938 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149623.log 2023-10-21T17:41:59.939 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128400.log 2023-10-21T17:41:59.939 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113448.log.gz 2023-10-21T17:41:59.941 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103100.log 2023-10-21T17:41:59.941 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149623.log.gz 2023-10-21T17:41:59.942 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147803.log 2023-10-21T17:41:59.942 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128400.log.gz 2023-10-21T17:41:59.943 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100967.log 2023-10-21T17:41:59.943 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103100.log.gz 2023-10-21T17:41:59.944 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129276.log 2023-10-21T17:41:59.945 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147803.log.gz 2023-10-21T17:41:59.946 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126650.log 2023-10-21T17:41:59.946 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100967.log.gz 2023-10-21T17:41:59.947 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97590.log 2023-10-21T17:41:59.947 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129276.log.gz 2023-10-21T17:41:59.948 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158492.log 2023-10-21T17:41:59.948 INFO:teuthology.orchestra.run.smithi053.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.126650.log.gz 2023-10-21T17:41:59.949 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113556.log 2023-10-21T17:41:59.949 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97590.log.gz 2023-10-21T17:41:59.950 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93361.log 2023-10-21T17:41:59.951 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158492.log.gz 2023-10-21T17:41:59.952 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116834.log 2023-10-21T17:41:59.952 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113556.log.gz 2023-10-21T17:41:59.953 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111670.log 2023-10-21T17:41:59.953 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93361.log.gz 2023-10-21T17:41:59.955 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116834.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.119256.log 2023-10-21T17:41:59.955 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116834.log.gz 2023-10-21T17:41:59.956 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155075.log 2023-10-21T17:41:59.956 INFO:teuthology.orchestra.run.smithi053.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.111670.log.gz 2023-10-21T17:41:59.957 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156447.log 2023-10-21T17:41:59.958 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155075.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.119256.log.gz 2023-10-21T17:41:59.958 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.155075.log.gz 2023-10-21T17:41:59.958 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.138541.log 2023-10-21T17:41:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128830.log 2023-10-21T17:41:59.960 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156447.log.gz 2023-10-21T17:41:59.960 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.138541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.138541.log.gz 2023-10-21T17:41:59.961 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125201.log 2023-10-21T17:41:59.962 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99489.log 2023-10-21T17:41:59.963 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128830.log.gz 2023-10-21T17:41:59.964 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121203.log 2023-10-21T17:41:59.964 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125201.log.gz 2023-10-21T17:41:59.965 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112151.log 2023-10-21T17:41:59.965 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99489.log.gz 2023-10-21T17:41:59.966 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135327.log 2023-10-21T17:41:59.967 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121203.log.gz 2023-10-21T17:41:59.968 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100027.log 2023-10-21T17:41:59.968 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112151.log.gz 2023-10-21T17:41:59.969 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93403.log 2023-10-21T17:41:59.969 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100027.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.135327.log.gz 2023-10-21T17:41:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119647.log 2023-10-21T17:41:59.970 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100027.log.gz 2023-10-21T17:41:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148339.log 2023-10-21T17:41:59.972 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93403.log.gz 2023-10-21T17:41:59.973 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119647.log.gz 2023-10-21T17:41:59.973 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116227.log 2023-10-21T17:41:59.974 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142085.log 2023-10-21T17:41:59.974 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.148339.log.gz 2023-10-21T17:41:59.975 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142043.log 2023-10-21T17:41:59.975 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116227.log.gz 2023-10-21T17:41:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142085.log: gzip -5 --verbose 87.7% -- /var/log/ceph/ceph-client.admin.124544.log -- replaced with /var/log/ceph/ceph-client.admin.142085.log.gz 2023-10-21T17:41:59.977 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:41:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142043.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142043.log.gz 2023-10-21T17:41:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118239.log 2023-10-21T17:41:59.979 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115326.log 2023-10-21T17:41:59.980 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124544.log.gz 2023-10-21T17:41:59.981 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147929.log 2023-10-21T17:41:59.981 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118239.log.gz 2023-10-21T17:41:59.982 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147052.log 2023-10-21T17:41:59.982 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115326.log.gz 2023-10-21T17:41:59.983 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152888.log 2023-10-21T17:41:59.983 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147929.log.gz 2023-10-21T17:41:59.984 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124376.log 2023-10-21T17:41:59.985 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147052.log.gz 2023-10-21T17:41:59.985 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152888.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152888.log.gz 2023-10-21T17:41:59.986 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133327.log 2023-10-21T17:41:59.987 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103647.log 2023-10-21T17:41:59.987 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124376.log.gz 2023-10-21T17:41:59.988 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133327.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133327.log.gz 2023-10-21T17:41:59.989 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120573.log 2023-10-21T17:41:59.990 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159196.log 2023-10-21T17:41:59.990 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103647.log.gz 2023-10-21T17:41:59.991 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133243.log 2023-10-21T17:41:59.991 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120573.log.gz 2023-10-21T17:41:59.992 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141142.log 2023-10-21T17:41:59.993 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159196.log.gz 2023-10-21T17:41:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133243.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133243.log.gz 2023-10-21T17:41:59.994 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147262.log 2023-10-21T17:41:59.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159019.log 2023-10-21T17:41:59.995 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141142.log.gz 2023-10-21T17:41:59.996 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115835.log 2023-10-21T17:41:59.997 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147262.log.gz 2023-10-21T17:41:59.997 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159019.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159019.log.gz 2023-10-21T17:41:59.998 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137825.log 2023-10-21T17:41:59.999 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120699.log 2023-10-21T17:41:59.999 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115835.log.gz 2023-10-21T17:42:00.000 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149046.log 2023-10-21T17:42:00.000 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137825.log.gz 2023-10-21T17:42:00.001 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120699.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.120699.log.gz -- 2023-10-21T17:42:00.002 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.150879.log 2023-10-21T17:42:00.003 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116269.log 2023-10-21T17:42:00.003 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149046.log.gz 2023-10-21T17:42:00.004 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102378.log 2023-10-21T17:42:00.004 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150879.log.gz 2023-10-21T17:42:00.005 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145578.log 2023-10-21T17:42:00.005 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116269.log.gz 2023-10-21T17:42:00.006 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136307.log 2023-10-21T17:42:00.006 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102378.log.gz 2023-10-21T17:42:00.007 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145578.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145578.log.gz 2023-10-21T17:42:00.008 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159553.log 2023-10-21T17:42:00.009 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136307.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136307.log.gz 2023-10-21T17:42:00.009 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-21T17:42:00.010 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108390.log 2023-10-21T17:42:00.010 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159553.log.gz 2023-10-21T17:42:00.011 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116185.log 2023-10-21T17:42:00.013 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139611.log 2023-10-21T17:42:00.013 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108390.log.gz 2023-10-21T17:42:00.014 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146353.log 2023-10-21T17:42:00.014 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116185.log.gz 2023-10-21T17:42:00.015 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.139611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101340.log 2023-10-21T17:42:00.016 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139611.log.gz 2023-10-21T17:42:00.017 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146353.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146353.log.gz 2023-10-21T17:42:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105019.log 2023-10-21T17:42:00.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101340.log: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-21T17:42:00.019 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126036.log 2023-10-21T17:42:00.019 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101340.log.gz 2023-10-21T17:42:00.020 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.105019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127469.log 2023-10-21T17:42:00.020 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105019.log.gz 2023-10-21T17:42:00.021 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126036.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126036.log.gz 2023-10-21T17:42:00.021 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135243.log 2023-10-21T17:42:00.022 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123580.log 2023-10-21T17:42:00.023 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127469.log.gz 2023-10-21T17:42:00.024 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134165.log 2023-10-21T17:42:00.024 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135243.log.gz 2023-10-21T17:42:00.025 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151173.log 2023-10-21T17:42:00.026 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123580.log.gz 2023-10-21T17:42:00.027 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127889.log 88.3% 2023-10-21T17:42:00.027 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.134165.log.gz 2023-10-21T17:42:00.028 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126450.log 2023-10-21T17:42:00.028 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151173.log.gz 2023-10-21T17:42:00.029 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99363.log 2023-10-21T17:42:00.029 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127889.log.gz 2023-10-21T17:42:00.031 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156615.log 2023-10-21T17:42:00.031 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126450.log.gz 2023-10-21T17:42:00.032 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108654.log 2023-10-21T17:42:00.032 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99363.log.gz 2023-10-21T17:42:00.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140932.log 2023-10-21T17:42:00.033 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.156615.log.gz 2023-10-21T17:42:00.035 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108654.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.108654.log.gz --verbose 2023-10-21T17:42:00.035 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/ceph-client.admin.140764.log 2023-10-21T17:42:00.036 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113364.log 2023-10-21T17:42:00.036 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140932.log.gz 2023-10-21T17:42:00.037 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136790.log 2023-10-21T17:42:00.037 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140764.log.gz 2023-10-21T17:42:00.039 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113364.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113364.log.gz 2023-10-21T17:42:00.039 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119527.log 2023-10-21T17:42:00.040 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136790.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136790.log.gz 2023-10-21T17:42:00.040 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113322.log 2023-10-21T17:42:00.041 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.119527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.119527.log.gz 2023-10-21T17:42:00.041 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108096.log 2023-10-21T17:42:00.043 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122038.log 2023-10-21T17:42:00.043 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113322.log.gz 2023-10-21T17:42:00.044 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108096.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108096.log.gz 2023-10-21T17:42:00.044 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133852.log 2023-10-21T17:42:00.045 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123664.log 2023-10-21T17:42:00.046 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122038.log.gz 2023-10-21T17:42:00.046 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133852.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133852.log.gzgzip 2023-10-21T17:42:00.047 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.127332.log 2023-10-21T17:42:00.048 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107103.log 2023-10-21T17:42:00.049 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123664.log.gz 2023-10-21T17:42:00.049 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127332.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127332.log.gz 2023-10-21T17:42:00.050 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118071.log 2023-10-21T17:42:00.051 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107103.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107103.log.gz 2023-10-21T17:42:00.051 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102008.log 2023-10-21T17:42:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157862.log 2023-10-21T17:42:00.052 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118071.log.gz 2023-10-21T17:42:00.053 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102008.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102008.log.gz 2023-10-21T17:42:00.053 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129084.log 2023-10-21T17:42:00.055 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99124.log 2023-10-21T17:42:00.055 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157862.log.gz 2023-10-21T17:42:00.056 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135075.log 2023-10-21T17:42:00.056 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129084.log.gz 2023-10-21T17:42:00.057 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99124.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99124.log.gz 2023-10-21T17:42:00.057 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128099.log 2023-10-21T17:42:00.058 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108780.log 2023-10-21T17:42:00.058 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135075.log.gz 2023-10-21T17:42:00.059 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128099.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128099.log.gz 2023-10-21T17:42:00.060 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108990.log 2023-10-21T17:42:00.061 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156681.log 2023-10-21T17:42:00.061 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108780.log.gz 2023-10-21T17:42:00.062 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108990.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.158198.log -- replaced with /var/log/ceph/ceph-client.admin.108990.log.gz 2023-10-21T17:42:00.062 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.063 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98877.log 2023-10-21T17:42:00.064 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156681.log.gz 2023-10-21T17:42:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158198.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158198.log.gz 2023-10-21T17:42:00.065 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116970.log 2023-10-21T17:42:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98877.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98877.log.gz 2023-10-21T17:42:00.066 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133955.log 2023-10-21T17:42:00.067 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127637.log 2023-10-21T17:42:00.067 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116970.log.gz 2023-10-21T17:42:00.068 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133955.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133955.log.gz 2023-10-21T17:42:00.069 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127154.log 2023-10-21T17:42:00.069 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153100.log 2023-10-21T17:42:00.070 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127637.log.gz 2023-10-21T17:42:00.071 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127154.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.148508.log -- replaced with /var/log/ceph/ceph-client.admin.127154.log.gz 2023-10-21T17:42:00.071 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.072 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97044.log 2023-10-21T17:42:00.073 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153100.log.gz 2023-10-21T17:42:00.074 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98289.log 2023-10-21T17:42:00.074 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.148508.log.gz 2023-10-21T17:42:00.075 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159300.log 2023-10-21T17:42:00.075 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97044.log.gz 2023-10-21T17:42:00.077 INFO:teuthology.orchestra.run.smithi124.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-21T17:42:00.077 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137781.log 2023-10-21T17:42:00.077 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98289.log.gz 2023-10-21T17:42:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159300.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159300.log.gz 2023-10-21T17:42:00.078 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92967.log 2023-10-21T17:42:00.079 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142728.log 2023-10-21T17:42:00.079 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137781.log.gz 2023-10-21T17:42:00.080 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98163.log 2023-10-21T17:42:00.081 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92967.log.gz 2023-10-21T17:42:00.081 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94482.log 2023-10-21T17:42:00.082 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142728.log.gz 2023-10-21T17:42:00.083 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158893.log 2023-10-21T17:42:00.084 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98163.log.gz 2023-10-21T17:42:00.084 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94482.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94482.log.gz 2023-10-21T17:42:00.084 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128225.log 2023-10-21T17:42:00.086 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113238.log 2023-10-21T17:42:00.086 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158893.log.gz 2023-10-21T17:42:00.087 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128225.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128225.log.gz 2023-10-21T17:42:00.088 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107355.log 2023-10-21T17:42:00.089 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132029.log 2023-10-21T17:42:00.089 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113238.log.gz 2023-10-21T17:42:00.090 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94479.log 2023-10-21T17:42:00.090 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107355.log.gz 2023-10-21T17:42:00.091 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.132029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138246.log 2023-10-21T17:42:00.092 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.132029.log.gz 2023-10-21T17:42:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94479.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94479.log.gz 2023-10-21T17:42:00.093 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134865.log 2023-10-21T17:42:00.094 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155556.log 2023-10-21T17:42:00.094 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.138246.log.gz 2023-10-21T17:42:00.095 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122772.log 2023-10-21T17:42:00.096 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134865.log.gz 2023-10-21T17:42:00.097 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113068.log 2023-10-21T17:42:00.097 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155556.log.gz 2023-10-21T17:42:00.098 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99236.log 2023-10-21T17:42:00.098 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122772.log.gz 2023-10-21T17:42:00.099 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97842.log 2023-10-21T17:42:00.099 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113068.log.gz 2023-10-21T17:42:00.100 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99236.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99236.log.gz 2023-10-21T17:42:00.101 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127376.log 2023-10-21T17:42:00.102 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118029.log 2023-10-21T17:42:00.103 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97842.log.gz 2023-10-21T17:42:00.103 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111712.log 2023-10-21T17:42:00.104 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127376.log.gz 2023-10-21T17:42:00.105 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146670.log 2023-10-21T17:42:00.105 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118029.log.gz 2023-10-21T17:42:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111712.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111712.log.gz 2023-10-21T17:42:00.106 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118962.log 2023-10-21T17:42:00.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148423.log 2023-10-21T17:42:00.108 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146670.log.gz 2023-10-21T17:42:00.109 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145704.log 2023-10-21T17:42:00.109 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118962.log.gz 2023-10-21T17:42:00.110 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151855.log 2023-10-21T17:42:00.110 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148423.log.gz 2023-10-21T17:42:00.112 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113406.log 2023-10-21T17:42:00.112 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145704.log.gz 2023-10-21T17:42:00.113 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151855.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151855.log.gz 2023-10-21T17:42:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122080.log 2023-10-21T17:42:00.114 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157226.log 2023-10-21T17:42:00.115 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113406.log.gz 2023-10-21T17:42:00.116 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112277.log 2023-10-21T17:42:00.116 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122080.log.gz 2023-10-21T17:42:00.117 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157268.log 2023-10-21T17:42:00.118 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157226.log.gz 2023-10-21T17:42:00.119 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112277.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112277.log.gz 2023-10-21T17:42:00.119 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112760.log 2023-10-21T17:42:00.120 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117464.log 2023-10-21T17:42:00.120 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157268.log.gz 2023-10-21T17:42:00.121 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156237.log 2023-10-21T17:42:00.121 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112760.log.gz 2023-10-21T17:42:00.122 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140680.log 2023-10-21T17:42:00.123 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117464.log.gz 2023-10-21T17:42:00.124 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112006.log 2023-10-21T17:42:00.124 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156237.log.gz 2023-10-21T17:42:00.125 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118664.log 2023-10-21T17:42:00.125 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140680.log.gz 2023-10-21T17:42:00.126 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147220.log 2023-10-21T17:42:00.127 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112006.log.gz 2023-10-21T17:42:00.128 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118664.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118664.log.gz 2023-10-21T17:42:00.128 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95309.log 2023-10-21T17:42:00.129 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117422.log 2023-10-21T17:42:00.129 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147220.log.gz 2023-10-21T17:42:00.130 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146882.log 2023-10-21T17:42:00.131 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95309.log.gz 2023-10-21T17:42:00.132 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118579.log 2023-10-21T17:42:00.132 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117422.log.gz 2023-10-21T17:42:00.133 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146882.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146882.log.gz 2023-10-21T17:42:00.134 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108570.log 2023-10-21T17:42:00.135 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140503.log 2023-10-21T17:42:00.135 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118579.log.gz 2023-10-21T17:42:00.136 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100449.log 2023-10-21T17:42:00.136 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108570.log.gz 2023-10-21T17:42:00.137 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145830.log 2023-10-21T17:42:00.137 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140503.log.gz 2023-10-21T17:42:00.138 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135393.log 2023-10-21T17:42:00.139 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100449.log.gz 2023-10-21T17:42:00.140 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155033.log 2023-10-21T17:42:00.140 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145830.log.gz 2023-10-21T17:42:00.141 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128183.log 2023-10-21T17:42:00.141 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135393.log.gz 2023-10-21T17:42:00.142 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87824.log 2023-10-21T17:42:00.142 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155033.log.gz 2023-10-21T17:42:00.144 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94136.log 2023-10-21T17:42:00.144 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128183.log.gz 2023-10-21T17:42:00.145 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.87824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108016.log 2023-10-21T17:42:00.145 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87824.log.gz 2023-10-21T17:42:00.147 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94136.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.94136.log.gz -- 2023-10-21T17:42:00.147 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.155981.log 2023-10-21T17:42:00.148 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108016.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.108016.log.gz -5 2023-10-21T17:42:00.148 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.116461.log 2023-10-21T17:42:00.149 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105545.log 2023-10-21T17:42:00.149 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155981.log.gz 2023-10-21T17:42:00.150 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118622.log 2023-10-21T17:42:00.151 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116461.log.gz 2023-10-21T17:42:00.152 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.105545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159061.log 2023-10-21T17:42:00.152 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105545.log.gz 2023-10-21T17:42:00.153 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136874.log 2023-10-21T17:42:00.153 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118622.log.gz 2023-10-21T17:42:00.154 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142421.log 2023-10-21T17:42:00.154 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159061.log.gz 2023-10-21T17:42:00.155 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151473.log 2023-10-21T17:42:00.156 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136874.log.gz 2023-10-21T17:42:00.157 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159104.log 2023-10-21T17:42:00.157 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142421.log.gz 2023-10-21T17:42:00.158 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151473.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.151473.log.gz -- 2023-10-21T17:42:00.158 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.155219.log 2023-10-21T17:42:00.160 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151771.log 2023-10-21T17:42:00.160 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159104.log.gz 2023-10-21T17:42:00.161 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150649.log 2023-10-21T17:42:00.161 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155219.log.gz 2023-10-21T17:42:00.162 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139344.log 2023-10-21T17:42:00.163 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151771.log.gz 2023-10-21T17:42:00.164 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116726.log 2023-10-21T17:42:00.164 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150649.log.gz 2023-10-21T17:42:00.165 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.139344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114218.log 2023-10-21T17:42:00.165 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139344.log.gz 2023-10-21T17:42:00.166 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122831.log 2023-10-21T17:42:00.167 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116726.log.gz 2023-10-21T17:42:00.167 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117191.log 2023-10-21T17:42:00.168 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114218.log.gz 2023-10-21T17:42:00.169 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123496.log 2023-10-21T17:42:00.169 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122831.log.gz 2023-10-21T17:42:00.170 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117191.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117191.log.gz 2023-10-21T17:42:00.171 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119214.log 2023-10-21T17:42:00.172 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128874.log 2023-10-21T17:42:00.172 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123496.log.gz 2023-10-21T17:42:00.173 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107753.log 2023-10-21T17:42:00.173 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119214.log.gz 2023-10-21T17:42:00.174 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122544.log 2023-10-21T17:42:00.175 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128874.log.gz 2023-10-21T17:42:00.176 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107753.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107753.log.gz 2023-10-21T17:42:00.176 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157988.log 2023-10-21T17:42:00.177 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.122544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122544.log.gz 2023-10-21T17:42:00.177 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125411.log 2023-10-21T17:42:00.179 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152974.log 2023-10-21T17:42:00.179 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157988.log.gz 2023-10-21T17:42:00.180 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95057.log 2023-10-21T17:42:00.180 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125411.log.gz 2023-10-21T17:42:00.181 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150127.log 2023-10-21T17:42:00.181 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152974.log.gz 2023-10-21T17:42:00.182 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147761.log 2023-10-21T17:42:00.183 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95057.log.gz 2023-10-21T17:42:00.184 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150127.log: gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.150127.log.gz -- 2023-10-21T17:42:00.184 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.134733.log 2023-10-21T17:42:00.185 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155346.log 2023-10-21T17:42:00.185 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147761.log.gz 2023-10-21T17:42:00.186 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140197.log 2023-10-21T17:42:00.187 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134733.log.gz 2023-10-21T17:42:00.188 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155346.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155346.log.gz 2023-10-21T17:42:00.188 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102126.log 2023-10-21T17:42:00.190 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145305.log 2023-10-21T17:42:00.190 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140197.log.gz 2023-10-21T17:42:00.191 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102126.log.gz 2023-10-21T17:42:00.191 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149392.log 2023-10-21T17:42:00.192 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97338.log 2023-10-21T17:42:00.192 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145305.log.gz 2023-10-21T17:42:00.193 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99081.log 2023-10-21T17:42:00.194 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149392.log.gz 2023-10-21T17:42:00.195 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118706.log 2023-10-21T17:42:00.195 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97338.log.gz 2023-10-21T17:42:00.196 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99081.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.99081.log.gz 2023-10-21T17:42:00.196 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91875.log 2023-10-21T17:42:00.198 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95645.log 2023-10-21T17:42:00.198 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.118706.log.gz 2023-10-21T17:42:00.199 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96207.log 2023-10-21T17:42:00.199 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91875.log.gz 2023-10-21T17:42:00.200 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151089.log 2023-10-21T17:42:00.200 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95645.log.gz 2023-10-21T17:42:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.96207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96207.log.gz 2023-10-21T17:42:00.201 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157631.log 2023-10-21T17:42:00.202 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151643.log 2023-10-21T17:42:00.203 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151089.log.gz 2023-10-21T17:42:00.203 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157631.log.gz 2023-10-21T17:42:00.204 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103310.log 2023-10-21T17:42:00.205 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158282.log 2023-10-21T17:42:00.206 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151643.log.gz 2023-10-21T17:42:00.206 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158450.log 2023-10-21T17:42:00.207 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103310.log.gz 2023-10-21T17:42:00.208 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136706.log 2023-10-21T17:42:00.209 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158282.log.gz 2023-10-21T17:42:00.209 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158450.log: gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.93713.log -- replaced with /var/log/ceph/ceph-client.admin.158450.log.gz 2023-10-21T17:42:00.209 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.211 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153776.log 2023-10-21T17:42:00.211 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136706.log.gz 2023-10-21T17:42:00.212 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152023.log 2023-10-21T17:42:00.212 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93713.log.gz 2023-10-21T17:42:00.213 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141609.log 2023-10-21T17:42:00.213 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153776.log.gz 2023-10-21T17:42:00.214 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102168.log 2023-10-21T17:42:00.215 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152023.log.gz 2023-10-21T17:42:00.216 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141609.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141609.log.gz -5 2023-10-21T17:42:00.216 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107439.log 2023-10-21T17:42:00.217 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111084.log 2023-10-21T17:42:00.217 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102168.log.gz 2023-10-21T17:42:00.219 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101713.log 2023-10-21T17:42:00.219 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107439.log.gz 2023-10-21T17:42:00.220 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118376.log 2023-10-21T17:42:00.220 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111084.log.gz 2023-10-21T17:42:00.221 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101713.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101713.log.gzgzip 2023-10-21T17:42:00.221 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94787.log 2023-10-21T17:42:00.223 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96918.log 2023-10-21T17:42:00.223 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118376.log.gz 2023-10-21T17:42:00.224 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141184.log 2023-10-21T17:42:00.224 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94787.log.gz 2023-10-21T17:42:00.225 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96960.log 2023-10-21T17:42:00.226 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96918.log.gz 2023-10-21T17:42:00.226 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141184.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.134991.log -- replaced with /var/log/ceph/ceph-client.admin.141184.log.gz 2023-10-21T17:42:00.226 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.228 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126691.log 2023-10-21T17:42:00.228 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96960.log.gz 2023-10-21T17:42:00.229 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98997.log 2023-10-21T17:42:00.230 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134991.log.gz 2023-10-21T17:42:00.230 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145620.log 2023-10-21T17:42:00.231 INFO:teuthology.orchestra.run.smithi053.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.126691.log.gz 2023-10-21T17:42:00.232 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98997.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-21T17:42:00.232 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98997.log.gz 2023-10-21T17:42:00.233 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114260.log 2023-10-21T17:42:00.233 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145620.log.gz 2023-10-21T17:42:00.234 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145389.log 2023-10-21T17:42:00.236 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114260.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.114260.log.gz /var/log/ceph/ceph-client.admin.109517.log 2023-10-21T17:42:00.236 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.237 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145389.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145389.log.gz 2023-10-21T17:42:00.237 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145872.log 2023-10-21T17:42:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109517.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109517.log.gz 2023-10-21T17:42:00.239 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94481.log 2023-10-21T17:42:00.240 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145872.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145872.log.gz 2023-10-21T17:42:00.241 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103268.log 2023-10-21T17:42:00.242 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94481.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94481.log.gz 2023-10-21T17:42:00.242 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134907.log 2023-10-21T17:42:00.243 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103268.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103268.log.gz 2023-10-21T17:42:00.244 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114617.log 2023-10-21T17:42:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134907.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134907.log.gz 2023-10-21T17:42:00.245 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115973.log 2023-10-21T17:42:00.247 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114617.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114617.log.gz 2023-10-21T17:42:00.247 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92087.log 2023-10-21T17:42:00.248 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124845.log 2023-10-21T17:42:00.248 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115973.log.gz 2023-10-21T17:42:00.249 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154175.log 2023-10-21T17:42:00.249 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92087.log.gz 2023-10-21T17:42:00.251 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95561.log 2023-10-21T17:42:00.251 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124845.log.gz 2023-10-21T17:42:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154175.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154175.log.gz 2023-10-21T17:42:00.252 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153421.log 2023-10-21T17:42:00.253 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152838.log 2023-10-21T17:42:00.253 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95561.log.gz 2023-10-21T17:42:00.255 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153421.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153421.log.gz 2023-10-21T17:42:00.255 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121462.log 2023-10-21T17:42:00.256 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120657.log 2023-10-21T17:42:00.256 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152838.log.gz 2023-10-21T17:42:00.258 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121462.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.129602.log -- replaced with /var/log/ceph/ceph-client.admin.121462.log.gz 2023-10-21T17:42:00.258 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.259 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120657.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.120657.log.gz --verbose 2023-10-21T17:42:00.259 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/ceph-client.admin.147136.log 2023-10-21T17:42:00.260 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129602.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129602.log.gz 2023-10-21T17:42:00.260 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152149.log 2023-10-21T17:42:00.261 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97002.log 2023-10-21T17:42:00.262 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147136.log.gz 2023-10-21T17:42:00.263 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118836.log 2023-10-21T17:42:00.263 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.152149.log.gz 2023-10-21T17:42:00.264 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121818.log 2023-10-21T17:42:00.265 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97002.log.gz 2023-10-21T17:42:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118836.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118836.log.gz 2023-10-21T17:42:00.266 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151515.log 2023-10-21T17:42:00.267 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153269.log 2023-10-21T17:42:00.267 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121818.log.gz 2023-10-21T17:42:00.268 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151515.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.151515.log.gz 2023-10-21T17:42:00.268 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.128788.log 2023-10-21T17:42:00.270 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122357.log 2023-10-21T17:42:00.270 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.153269.log.gz 2023-10-21T17:42:00.271 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128788.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128788.log.gz 2023-10-21T17:42:00.271 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100799.log 2023-10-21T17:42:00.272 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141957.log 2023-10-21T17:42:00.273 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122357.log.gz 2023-10-21T17:42:00.274 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100281.log 2023-10-21T17:42:00.274 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100799.log.gz 2023-10-21T17:42:00.278 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95519.log 2023-10-21T17:42:00.278 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141957.log.gz 2023-10-21T17:42:00.278 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100281.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100281.log.gz 2023-10-21T17:42:00.279 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112403.log 2023-10-21T17:42:00.279 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127679.log 2023-10-21T17:42:00.279 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95519.log.gz 2023-10-21T17:42:00.280 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126568.log 2023-10-21T17:42:00.280 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112403.log.gz 2023-10-21T17:42:00.281 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126203.log 2023-10-21T17:42:00.281 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127679.log.gz 2023-10-21T17:42:00.281 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126568.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.126568.log.gz 2023-10-21T17:42:00.282 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152794.log 2023-10-21T17:42:00.283 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159636.log 2023-10-21T17:42:00.283 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126203.log.gz 2023-10-21T17:42:00.284 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152794.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.152794.log.gz 2023-10-21T17:42:00.284 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136496.log 2023-10-21T17:42:00.286 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101506.log 2023-10-21T17:42:00.286 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159636.log.gz 2023-10-21T17:42:00.287 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136496.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136496.log.gz 2023-10-21T17:42:00.287 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127721.log 2023-10-21T17:42:00.289 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101506.log: gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.101506.log.gz 2023-10-21T17:42:00.289 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.138162.log 2023-10-21T17:42:00.290 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127721.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127721.log.gz 2023-10-21T17:42:00.290 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111575.log 2023-10-21T17:42:00.291 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150691.log 2023-10-21T17:42:00.291 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138162.log.gz 2023-10-21T17:42:00.293 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111575.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111575.log.gz 2023-10-21T17:42:00.293 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141100.log 2023-10-21T17:42:00.293 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96342.log 2023-10-21T17:42:00.294 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150691.log.gz 2023-10-21T17:42:00.295 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141100.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141100.log.gz 2023-10-21T17:42:00.295 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137994.log 2023-10-21T17:42:00.296 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134675.log 2023-10-21T17:42:00.296 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96342.log.gz 2023-10-21T17:42:00.297 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137994.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137994.log.gz 2023-10-21T17:42:00.298 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101654.log 2023-10-21T17:42:00.299 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137487.log 2023-10-21T17:42:00.299 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134675.log.gz 2023-10-21T17:42:00.300 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101654.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101654.log.gz 2023-10-21T17:42:00.300 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101382.log 2023-10-21T17:42:00.301 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112487.log 2023-10-21T17:42:00.302 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137487.log.gz 2023-10-21T17:42:00.302 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101382.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101382.log.gz -5 2023-10-21T17:42:00.303 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.121868.log 2023-10-21T17:42:00.304 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125243.log 2023-10-21T17:42:00.304 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112487.log.gz 2023-10-21T17:42:00.305 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121868.log.gz 2023-10-21T17:42:00.305 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128015.log 2023-10-21T17:42:00.306 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124292.log 2023-10-21T17:42:00.307 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125243.log.gz 2023-10-21T17:42:00.308 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128015.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.128015.log.gz 2023-10-21T17:42:00.308 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115793.log 2023-10-21T17:42:00.309 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113018.log 2023-10-21T17:42:00.309 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124292.log.gz 2023-10-21T17:42:00.310 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150837.log 2023-10-21T17:42:00.310 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115793.log.gz 2023-10-21T17:42:00.311 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87456.log 2023-10-21T17:42:00.312 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113018.log.gz 2023-10-21T17:42:00.313 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150837.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150837.log.gz 2023-10-21T17:42:00.313 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112974.log 2023-10-21T17:42:00.314 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.87456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137102.log 2023-10-21T17:42:00.315 INFO:teuthology.orchestra.run.smithi053.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.87456.log.gz 2023-10-21T17:42:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112974.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112974.log.gz 2023-10-21T17:42:00.316 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148962.log 2023-10-21T17:42:00.317 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107008.log 2023-10-21T17:42:00.317 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137102.log.gz 2023-10-21T17:42:00.318 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147346.log 2023-10-21T17:42:00.318 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148962.log.gz 2023-10-21T17:42:00.319 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91579.log 2023-10-21T17:42:00.320 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107008.log.gz 2023-10-21T17:42:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147346.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147346.log.gz 2023-10-21T17:42:00.321 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119088.log 2023-10-21T17:42:00.322 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115032.log 2023-10-21T17:42:00.322 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91579.log.gz 2023-10-21T17:42:00.323 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137909.log 88.4% 2023-10-21T17:42:00.323 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119088.log.gz 2023-10-21T17:42:00.325 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151729.log 2023-10-21T17:42:00.325 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115032.log.gz 2023-10-21T17:42:00.326 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137909.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137909.log.gz 2023-10-21T17:42:00.326 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140722.log 2023-10-21T17:42:00.327 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102294.log 2023-10-21T17:42:00.327 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151729.log.gz 2023-10-21T17:42:00.329 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140722.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.140722.log.gz --verbose 2023-10-21T17:42:00.329 INFO:teuthology.orchestra.run.smithi053.stderr: -- /var/log/ceph/ceph-client.admin.125495.log 2023-10-21T17:42:00.330 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116353.log 2023-10-21T17:42:00.331 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102294.log.gz 2023-10-21T17:42:00.331 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125495.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.125495.log.gz 2023-10-21T17:42:00.331 INFO:teuthology.orchestra.run.smithi053.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.108262.log 2023-10-21T17:42:00.333 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134591.log 2023-10-21T17:42:00.333 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116353.log.gz 2023-10-21T17:42:00.334 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121161.log 88.9% 2023-10-21T17:42:00.334 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108262.log.gz 2023-10-21T17:42:00.335 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103984.log 2023-10-21T17:42:00.336 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134591.log.gz 2023-10-21T17:42:00.337 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108738.log 2023-10-21T17:42:00.337 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121161.log.gz 2023-10-21T17:42:00.339 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120489.log 2023-10-21T17:42:00.339 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103984.log.gz 2023-10-21T17:42:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108738.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108738.log.gz 2023-10-21T17:42:00.340 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98535.log 2023-10-21T17:42:00.341 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154469.log 2023-10-21T17:42:00.341 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120489.log.gz 2023-10-21T17:42:00.342 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98535.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98535.log.gz 2023-10-21T17:42:00.342 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111397.log 2023-10-21T17:42:00.344 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114869.log 2023-10-21T17:42:00.344 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154469.log.gz 2023-10-21T17:42:00.345 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111397.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111397.log.gz 2023-10-21T17:42:00.345 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108434.log 2023-10-21T17:42:00.346 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152708.log 2023-10-21T17:42:00.346 INFO:teuthology.orchestra.run.smithi053.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114869.log.gz 2023-10-21T17:42:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108434.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108434.log.gz 2023-10-21T17:42:00.347 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94810.log 2023-10-21T17:42:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148878.log 2023-10-21T17:42:00.349 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152708.log.gz 2023-10-21T17:42:00.349 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94810.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94810.log.gz 2023-10-21T17:42:00.350 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95687.log 2023-10-21T17:42:00.351 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157173.log 88.4% 2023-10-21T17:42:00.352 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.148878.log.gz 2023-10-21T17:42:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95687.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95687.log.gz 2023-10-21T17:42:00.352 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91357.log 2023-10-21T17:42:00.354 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120615.log 2023-10-21T17:42:00.354 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157173.log.gz 2023-10-21T17:42:00.355 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91357.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91357.log.gz 2023-10-21T17:42:00.356 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155939.log 2023-10-21T17:42:00.357 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120615.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.120615.log.gz 2023-10-21T17:42:00.357 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.150043.log 2023-10-21T17:42:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155939.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155939.log.gz 2023-10-21T17:42:00.358 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159258.log 2023-10-21T17:42:00.359 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87713.log 2023-10-21T17:42:00.360 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150043.log.gz 2023-10-21T17:42:00.360 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159258.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159258.log.gz 2023-10-21T17:42:00.361 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153184.log 2023-10-21T17:42:00.362 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.87713.log: gzip -5 --verbose 88.2% -- /var/log/ceph/ceph-client.admin.92363.log -- replaced with /var/log/ceph/ceph-client.admin.87713.log.gz 2023-10-21T17:42:00.362 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.363 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153184.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153184.log.gz 2023-10-21T17:42:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98835.log 2023-10-21T17:42:00.364 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146490.log 2023-10-21T17:42:00.365 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92363.log.gz 2023-10-21T17:42:00.365 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98835.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98835.log.gz 2023-10-21T17:42:00.366 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127238.log 2023-10-21T17:42:00.367 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146490.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146490.log.gz 2023-10-21T17:42:00.367 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104510.log 2023-10-21T17:42:00.368 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127238.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127238.log.gz 2023-10-21T17:42:00.368 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124628.log 2023-10-21T17:42:00.369 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.104510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157767.log 2023-10-21T17:42:00.369 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104510.log.gz 2023-10-21T17:42:00.370 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124628.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124628.log.gz 2023-10-21T17:42:00.371 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142608.log 2023-10-21T17:42:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142794.log 2023-10-21T17:42:00.372 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157767.log.gz 2023-10-21T17:42:00.372 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.142608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.142608.log.gz 2023-10-21T17:42:00.373 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153226.log 2023-10-21T17:42:00.374 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93755.log 2023-10-21T17:42:00.374 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142794.log.gz 2023-10-21T17:42:00.375 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153226.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153226.log.gz 2023-10-21T17:42:00.376 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148836.log 2023-10-21T17:42:00.377 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111194.log 2023-10-21T17:42:00.377 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93755.log.gz 2023-10-21T17:42:00.378 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148836.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148836.log.gz 2023-10-21T17:42:00.379 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107187.log 2023-10-21T17:42:00.380 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111194.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111194.log.gz 2023-10-21T17:42:00.380 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141999.log 2023-10-21T17:42:00.381 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97548.log 2023-10-21T17:42:00.381 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107187.log.gz 2023-10-21T17:42:00.382 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135742.log 2023-10-21T17:42:00.382 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141999.log.gz 2023-10-21T17:42:00.383 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.147641.log 2023-10-21T17:42:00.384 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97548.log.gz 2023-10-21T17:42:00.385 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123538.log 2023-10-21T17:42:00.385 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135742.log.gz 2023-10-21T17:42:00.386 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.147641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.147641.log.gz 2023-10-21T17:42:00.386 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94478.log 2023-10-21T17:42:00.387 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136223.log 2023-10-21T17:42:00.388 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123538.log.gz 2023-10-21T17:42:00.388 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102336.log 2023-10-21T17:42:00.388 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94478.log.gz 2023-10-21T17:42:00.389 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154217.log 2023-10-21T17:42:00.389 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136223.log.gz 2023-10-21T17:42:00.390 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102336.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.102336.log.gz 2023-10-21T17:42:00.391 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.107837.log 2023-10-21T17:42:00.392 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154217.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.154217.log.gz /var/log/ceph/ceph-client.admin.118456.log 2023-10-21T17:42:00.392 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.393 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117548.log 2023-10-21T17:42:00.393 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107837.log.gz 2023-10-21T17:42:00.394 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154865.log 2023-10-21T17:42:00.394 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118456.log.gz 2023-10-21T17:42:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117548.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117548.log.gz 2023-10-21T17:42:00.395 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156195.log 2023-10-21T17:42:00.396 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159427.log 2023-10-21T17:42:00.396 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154865.log.gz 2023-10-21T17:42:00.397 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156195.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156195.log.gz 2023-10-21T17:42:00.398 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129710.log 2023-10-21T17:42:00.399 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159427.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159427.log.gz 2023-10-21T17:42:00.399 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155472.log 2023-10-21T17:42:00.400 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124670.log 2023-10-21T17:42:00.400 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129710.log.gz 2023-10-21T17:42:00.401 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120216.log 2023-10-21T17:42:00.401 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155472.log.gz 2023-10-21T17:42:00.402 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124670.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124670.log.gz 2023-10-21T17:42:00.402 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94147.log 2023-10-21T17:42:00.404 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142686.log 2023-10-21T17:42:00.404 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120216.log.gz 2023-10-21T17:42:00.405 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137867.log 2023-10-21T17:42:00.405 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94147.log.gz 2023-10-21T17:42:00.406 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154637.log 2023-10-21T17:42:00.406 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142686.log.gz 2023-10-21T17:42:00.407 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123937.log 2023-10-21T17:42:00.407 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.137867.log.gz 2023-10-21T17:42:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154637.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154637.log.gz 2023-10-21T17:42:00.408 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154595.log 2023-10-21T17:42:00.409 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149308.log 2023-10-21T17:42:00.409 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123937.log.gz 2023-10-21T17:42:00.411 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135920.log 2023-10-21T17:42:00.411 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154595.log.gz 2023-10-21T17:42:00.412 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116101.log 2023-10-21T17:42:00.412 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.149308.log.gz 2023-10-21T17:42:00.413 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152750.log 2023-10-21T17:42:00.413 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135920.log.gz 2023-10-21T17:42:00.414 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116101.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.154991.log -- replaced with /var/log/ceph/ceph-client.admin.116101.log.gz 2023-10-21T17:42:00.414 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:00.415 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158935.log 2023-10-21T17:42:00.415 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152750.log.gz 2023-10-21T17:42:00.416 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133464.log 2023-10-21T17:42:00.417 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154991.log.gz 2023-10-21T17:42:00.418 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103058.log 2023-10-21T17:42:00.418 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158935.log.gz 2023-10-21T17:42:00.419 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133464.log.gz 2023-10-21T17:42:00.419 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150295.log 2023-10-21T17:42:00.420 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97506.log 2023-10-21T17:42:00.420 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103058.log.gz 2023-10-21T17:42:00.421 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98793.log 2023-10-21T17:42:00.422 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150295.log.gz 2023-10-21T17:42:00.423 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117632.log 2023-10-21T17:42:00.423 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97506.log.gz 2023-10-21T17:42:00.424 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149707.log 2023-10-21T17:42:00.424 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98793.log.gz 2023-10-21T17:42:00.425 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117632.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117632.log.gz 2023-10-21T17:42:00.425 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114099.log 2023-10-21T17:42:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149707.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149707.log.gz 2023-10-21T17:42:00.427 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96642.log 2023-10-21T17:42:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114099.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114099.log.gz 2023-10-21T17:42:00.428 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-21T17:42:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.2.96642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.96642.log.gz 2023-10-21T17:42:00.429 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93839.log 2023-10-21T17:42:00.430 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155304.log 2023-10-21T17:42:00.432 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93839.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93839.log.gz 2023-10-21T17:42:00.432 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99825.log 2023-10-21T17:42:00.433 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155304.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155304.log.gz 2023-10-21T17:42:00.434 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136664.log 2023-10-21T17:42:00.434 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133558.log 2023-10-21T17:42:00.435 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99825.log.gz 2023-10-21T17:42:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136664.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136664.log.gz 2023-10-21T17:42:00.436 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125916.log 2023-10-21T17:42:00.437 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157589.log 2023-10-21T17:42:00.438 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133558.log.gz 2023-10-21T17:42:00.438 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.125916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125916.log.gz 2023-10-21T17:42:00.439 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116311.log 2023-10-21T17:42:00.440 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116648.log 2023-10-21T17:42:00.440 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157589.log.gz 2023-10-21T17:42:00.441 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116311.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116311.log.gz 2023-10-21T17:42:00.442 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148015.log 2023-10-21T17:42:00.442 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.116648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145974.log 2023-10-21T17:42:00.443 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116648.log.gz 2023-10-21T17:42:00.444 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148015.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.148015.log.gz -5 2023-10-21T17:42:00.444 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.146838.log 2023-10-21T17:42:00.445 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95435.log 2023-10-21T17:42:00.446 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145974.log.gz 2023-10-21T17:42:00.447 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146838.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146838.log.gz 2023-10-21T17:42:00.447 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140890.log 2023-10-21T17:42:00.448 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150565.log 2023-10-21T17:42:00.448 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95435.log.gz 2023-10-21T17:42:00.449 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140890.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140890.log.gz 2023-10-21T17:42:00.450 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155598.log 2023-10-21T17:42:00.450 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119971.log 2023-10-21T17:42:00.451 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.150565.log.gz 2023-10-21T17:42:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155598.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155598.log.gz 2023-10-21T17:42:00.452 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126732.log 2023-10-21T17:42:00.453 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95893.log 2023-10-21T17:42:00.454 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119971.log.gz 2023-10-21T17:42:00.455 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126732.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126732.log.gz 2023-10-21T17:42:00.455 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140281.log 2023-10-21T17:42:00.456 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92795.log 2023-10-21T17:42:00.456 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95893.log.gz 2023-10-21T17:42:00.457 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140281.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140281.log.gz 2023-10-21T17:42:00.458 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148109.log 2023-10-21T17:42:00.459 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106334.log 2023-10-21T17:42:00.460 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148109.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148109.log.gz 2023-10-21T17:42:00.460 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121287.log 2023-10-21T17:42:00.460 INFO:teuthology.orchestra.run.smithi053.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.92795.log.gz 2023-10-21T17:42:00.461 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100493.log 2023-10-21T17:42:00.462 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106334.log.gz 2023-10-21T17:42:00.462 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121287.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121287.log.gz 2023-10-21T17:42:00.463 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129042.log 2023-10-21T17:42:00.463 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92297.log 2023-10-21T17:42:00.464 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100493.log.gz 2023-10-21T17:42:00.465 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129042.log.gz 2023-10-21T17:42:00.465 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141352.log 2023-10-21T17:42:00.466 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120909.log 2023-10-21T17:42:00.467 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141352.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141352.log.gz 2023-10-21T17:42:00.467 INFO:teuthology.orchestra.run.smithi053.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.92297.log.gz 2023-10-21T17:42:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137228.log 2023-10-21T17:42:00.468 INFO:teuthology.orchestra.run.smithi053.stderr:gzip/var/log/ceph/ceph-client.admin.120909.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94467.log 2023-10-21T17:42:00.469 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120909.log.gz 2023-10-21T17:42:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137228.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137228.log.gz 2023-10-21T17:42:00.470 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142187.log 2023-10-21T17:42:00.471 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100197.log 2023-10-21T17:42:00.471 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94467.log.gz 2023-10-21T17:42:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142187.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142187.log.gz 2023-10-21T17:42:00.472 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106619.log 2023-10-21T17:42:00.473 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124945.log 2023-10-21T17:42:00.474 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100197.log.gz 2023-10-21T17:42:00.474 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106619.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106619.log.gz 2023-10-21T17:42:00.475 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122291.log 2023-10-21T17:42:00.476 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151268.log 2023-10-21T17:42:00.476 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124945.log.gz 2023-10-21T17:42:00.477 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136934.log 2023-10-21T17:42:00.477 INFO:teuthology.orchestra.run.smithi053.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.122291.log.gz 2023-10-21T17:42:00.478 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134821.log 2023-10-21T17:42:00.479 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151268.log.gz 2023-10-21T17:42:00.479 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136934.log.gz 2023-10-21T17:42:00.480 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154343.log 2023-10-21T17:42:00.481 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100925.log 2023-10-21T17:42:00.481 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134821.log.gz 2023-10-21T17:42:00.482 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154343.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.154343.log.gz 2023-10-21T17:42:00.482 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.156868.log 2023-10-21T17:42:00.483 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159146.log 2023-10-21T17:42:00.484 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100925.log.gz 2023-10-21T17:42:00.484 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.156868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.156868.log.gz 2023-10-21T17:42:00.485 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106914.log 2023-10-21T17:42:00.486 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5/var/log/ceph/ceph-client.admin.159146.log: --verbose -- /var/log/ceph/ceph-client.admin.133642.log 2023-10-21T17:42:00.486 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159146.log.gz 2023-10-21T17:42:00.487 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106914.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106914.log.gz 2023-10-21T17:42:00.488 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111627.log 2023-10-21T17:42:00.489 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5/var/log/ceph/ceph-client.admin.133642.log: --verbose -- /var/log/ceph/ceph-client.admin.119130.log 2023-10-21T17:42:00.490 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133642.log.gz 2023-10-21T17:42:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111627.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111627.log.gz 2023-10-21T17:42:00.491 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129560.log 2023-10-21T17:42:00.492 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117903.log 2023-10-21T17:42:00.492 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119130.log.gz 2023-10-21T17:42:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129560.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129560.log.gz 2023-10-21T17:42:00.493 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91632.log 2023-10-21T17:42:00.494 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126144.log 2023-10-21T17:42:00.494 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117903.log.gz 2023-10-21T17:42:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91632.log.gz 2023-10-21T17:42:00.495 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93319.log 2023-10-21T17:42:00.497 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115074.log 2023-10-21T17:42:00.497 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126144.log.gz 2023-10-21T17:42:00.498 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93319.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93319.log.gz 2023-10-21T17:42:00.498 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114743.log 2023-10-21T17:42:00.499 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149917.log 2023-10-21T17:42:00.499 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115074.log.gz 2023-10-21T17:42:00.500 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114743.log.gz 2023-10-21T17:42:00.500 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101075.log 2023-10-21T17:42:00.501 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91534.log 2023-10-21T17:42:00.502 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149917.log.gz 2023-10-21T17:42:00.502 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101075.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101075.log.gz 2023-10-21T17:42:00.503 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111355.log 2023-10-21T17:42:00.504 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95393.log 2023-10-21T17:42:00.504 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91534.log.gz 2023-10-21T17:42:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111355.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111355.log.gz 2023-10-21T17:42:00.505 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152522.log 2023-10-21T17:42:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95393.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95393.log.gz 2023-10-21T17:42:00.507 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117369.log 2023-10-21T17:42:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152522.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152522.log.gz 2023-10-21T17:42:00.508 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111838.log 2023-10-21T17:42:00.509 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119172.log 2023-10-21T17:42:00.510 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117369.log.gz 2023-10-21T17:42:00.510 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111838.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111838.log.gz 2023-10-21T17:42:00.511 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106746.log 2023-10-21T17:42:00.512 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119172.log.gz 2023-10-21T17:42:00.512 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121774.log 2023-10-21T17:42:00.513 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91917.log 2023-10-21T17:42:00.514 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106746.log.gz 2023-10-21T17:42:00.515 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121774.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121774.log.gz 2023-10-21T17:42:00.515 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127553.log 2023-10-21T17:42:00.516 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91917.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91917.log.gz 2023-10-21T17:42:00.516 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93093.log 2023-10-21T17:42:00.517 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99657.log 2023-10-21T17:42:00.518 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127553.log.gz 2023-10-21T17:42:00.519 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93093.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93093.log.gz 2023-10-21T17:42:00.519 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155388.log 2023-10-21T17:42:00.520 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99657.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.99657.log.gz 2023-10-21T17:42:00.520 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121628.log 2023-10-21T17:42:00.521 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101772.log 2023-10-21T17:42:00.522 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.155388.log.gz 2023-10-21T17:42:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121628.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121628.log.gz 2023-10-21T17:42:00.523 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145788.log 2023-10-21T17:42:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101772.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101772.log.gz 2023-10-21T17:42:00.524 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113821.log 2023-10-21T17:42:00.526 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146101.log 2023-10-21T17:42:00.526 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145788.log.gz 2023-10-21T17:42:00.527 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113821.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113821.log.gz 2023-10-21T17:42:00.527 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141735.log 2023-10-21T17:42:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146101.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146101.log.gz 2023-10-21T17:42:00.529 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120783.log 2023-10-21T17:42:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141735.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141735.log.gz 2023-10-21T17:42:00.530 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93671.log 2023-10-21T17:42:00.531 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152564.log 2023-10-21T17:42:00.532 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120783.log.gz 2023-10-21T17:42:00.533 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93671.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93671.log.gz 2023-10-21T17:42:00.533 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129168.log 2023-10-21T17:42:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152564.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152564.log.gz 2023-10-21T17:42:00.534 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92213.log 2023-10-21T17:42:00.535 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120825.log 2023-10-21T17:42:00.536 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129168.log.gz 2023-10-21T17:42:00.537 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92213.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92213.log.gz 2023-10-21T17:42:00.537 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135285.log 2023-10-21T17:42:00.538 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120825.log.gz 2023-10-21T17:42:00.538 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99741.log 2023-10-21T17:42:00.539 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114386.log 2023-10-21T17:42:00.540 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135285.log.gz 2023-10-21T17:42:00.541 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99741.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99741.log.gz 2023-10-21T17:42:00.541 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103142.log 2023-10-21T17:42:00.542 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157006.log 2023-10-21T17:42:00.542 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114386.log.gz 2023-10-21T17:42:00.543 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103142.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103142.log.gz 2023-10-21T17:42:00.543 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145102.log 2023-10-21T17:42:00.544 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94470.log 2023-10-21T17:42:00.545 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157006.log.gz 2023-10-21T17:42:00.545 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145102.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145102.log.gz 2023-10-21T17:42:00.546 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125327.log 2023-10-21T17:42:00.547 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97422.log 2023-10-21T17:42:00.547 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94470.log.gz 2023-10-21T17:42:00.548 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125327.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.125327.log.gz 2023-10-21T17:42:00.548 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99447.log 2023-10-21T17:42:00.549 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115923.log 2023-10-21T17:42:00.550 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97422.log.gz 2023-10-21T17:42:00.550 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99447.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99447.log.gz 2023-10-21T17:42:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138769.log 2023-10-21T17:42:00.551 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115505.log 2023-10-21T17:42:00.552 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115923.log.gz 2023-10-21T17:42:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138769.log.gz 2023-10-21T17:42:00.553 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114532.log 2023-10-21T17:42:00.554 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155735.log 2023-10-21T17:42:00.555 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115505.log.gz 2023-10-21T17:42:00.555 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114532.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114532.log.gz 2023-10-21T17:42:00.556 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133768.log 2023-10-21T17:42:00.556 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98079.log 2023-10-21T17:42:00.557 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155735.log.gz 2023-10-21T17:42:00.558 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133768.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133768.log.gz 2023-10-21T17:42:00.558 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2023-10-21T17:42:00.559 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146926.log 2023-10-21T17:42:00.559 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98079.log.gz 2023-10-21T17:42:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2023-10-21T17:42:00.560 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128650.log 2023-10-21T17:42:00.561 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91779.log 2023-10-21T17:42:00.562 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146926.log.gz 2023-10-21T17:42:00.562 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128650.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128650.log.gz 2023-10-21T17:42:00.563 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154823.log 2023-10-21T17:42:00.564 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100841.log 2023-10-21T17:42:00.564 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91779.log.gz 2023-10-21T17:42:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154823.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154823.log.gz 2023-10-21T17:42:00.565 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142948.log 2023-10-21T17:42:00.566 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117506.log 2023-10-21T17:42:00.567 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100841.log.gz 2023-10-21T17:42:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142948.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142948.log.gz 2023-10-21T17:42:00.568 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118537.log 2023-10-21T17:42:00.569 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121077.log 2023-10-21T17:42:00.569 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117506.log.gz 2023-10-21T17:42:00.570 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118537.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118537.log.gz 2023-10-21T17:42:00.570 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114575.log 2023-10-21T17:42:00.571 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102210.log 2023-10-21T17:42:00.572 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121077.log.gz 2023-10-21T17:42:00.572 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114575.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114575.log.gz 2023-10-21T17:42:00.573 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93629.log 2023-10-21T17:42:00.574 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154427.log 2023-10-21T17:42:00.574 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102210.log.gz 2023-10-21T17:42:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93629.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93629.log.gz 2023-10-21T17:42:00.575 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124765.log 2023-10-21T17:42:00.576 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142895.log 2023-10-21T17:42:00.577 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154427.log.gz 2023-10-21T17:42:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124765.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.124765.log.gz 2023-10-21T17:42:00.577 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101424.log 2023-10-21T17:42:00.578 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147010.log 2023-10-21T17:42:00.579 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142895.log.gz 2023-10-21T17:42:00.580 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101424.log.gz 2023-10-21T17:42:00.580 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103184.log 2023-10-21T17:42:00.581 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155159.log 2023-10-21T17:42:00.582 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147010.log.gz 2023-10-21T17:42:00.582 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103184.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103184.log.gz 2023-10-21T17:42:00.582 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97170.log 2023-10-21T17:42:00.583 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155514.log 2023-10-21T17:42:00.584 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155159.log.gz 2023-10-21T17:42:00.585 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97170.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97170.log.gz 2023-10-21T17:42:00.585 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123622.log 2023-10-21T17:42:00.586 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149476.log 2023-10-21T17:42:00.586 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155514.log.gz 2023-10-21T17:42:00.587 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123622.log.gz 2023-10-21T17:42:00.587 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103942.log 2023-10-21T17:42:00.588 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119004.log 2023-10-21T17:42:00.589 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149476.log.gz 2023-10-21T17:42:00.590 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103942.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103942.log.gz 2023-10-21T17:42:00.590 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159343.log 2023-10-21T17:42:00.591 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93051.log 2023-10-21T17:42:00.591 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119004.log.gz 2023-10-21T17:42:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159343.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159343.log.gz 2023-10-21T17:42:00.592 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137186.log 2023-10-21T17:42:00.593 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150253.log 2023-10-21T17:42:00.594 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93051.log.gz 2023-10-21T17:42:00.594 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137186.log.gz 2023-10-21T17:42:00.595 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107397.log 2023-10-21T17:42:00.596 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133997.log 2023-10-21T17:42:00.596 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150253.log.gz 2023-10-21T17:42:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107397.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107397.log.gz 2023-10-21T17:42:00.597 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124189.log 2023-10-21T17:42:00.598 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99405.log 2023-10-21T17:42:00.599 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133997.log.gz 2023-10-21T17:42:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124189.log.gz 2023-10-21T17:42:00.600 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97380.log 2023-10-21T17:42:00.601 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99405.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99405.log.gz 2023-10-21T17:42:00.601 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98367.log 2023-10-21T17:42:00.602 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148783.log 2023-10-21T17:42:00.603 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97380.log.gz 2023-10-21T17:42:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98367.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98367.log.gz 2023-10-21T17:42:00.604 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151981.log 2023-10-21T17:42:00.605 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148783.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148783.log.gz 2023-10-21T17:42:00.605 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93277.log 2023-10-21T17:42:00.606 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99699.log 2023-10-21T17:42:00.607 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151981.log.gz 2023-10-21T17:42:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93277.log.gz 2023-10-21T17:42:00.608 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120394.log 2023-10-21T17:42:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99699.log.gz 2023-10-21T17:42:00.609 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-21T17:42:00.610 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137653.log 2023-10-21T17:42:00.611 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120394.log.gz 2023-10-21T17:42:00.612 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158408.log 2023-10-21T17:42:00.613 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125157.log 2023-10-21T17:42:00.614 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137653.log.gz 2023-10-21T17:42:00.615 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148237.log 2023-10-21T17:42:00.615 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158408.log.gz 2023-10-21T17:42:00.617 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126102.log 2023-10-21T17:42:00.617 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125157.log.gz 2023-10-21T17:42:00.618 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105808.log 2023-10-21T17:42:00.618 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148237.log.gz 2023-10-21T17:42:00.619 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114030.log 2023-10-21T17:42:00.619 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126102.log.gz 2023-10-21T17:42:00.621 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.105808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126273.log 2023-10-21T17:42:00.621 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105808.log.gz 2023-10-21T17:42:00.622 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159511.log 2023-10-21T17:42:00.622 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114030.log.gz 2023-10-21T17:42:00.624 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126273.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126273.log.gz 2023-10-21T17:42:00.624 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117107.log 2023-10-21T17:42:00.626 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120300.log 2023-10-21T17:42:00.626 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159511.log.gz 2023-10-21T17:42:00.627 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125071.log 2023-10-21T17:42:00.627 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117107.log.gz 2023-10-21T17:42:00.628 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128958.log 2023-10-21T17:42:00.629 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120300.log.gz 2023-10-21T17:42:00.630 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128608.log 2023-10-21T17:42:00.630 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125071.log.gz 2023-10-21T17:42:00.631 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158725.log 2023-10-21T17:42:00.631 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128958.log.gz 2023-10-21T17:42:00.633 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118113.log 2023-10-21T17:42:00.633 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128608.log.gz 2023-10-21T17:42:00.634 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140554.log 2023-10-21T17:42:00.635 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158725.log.gz 2023-10-21T17:42:00.636 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137018.log 2023-10-21T17:42:00.636 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118113.log.gz 2023-10-21T17:42:00.637 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154133.log 2023-10-21T17:42:00.637 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140554.log.gz 2023-10-21T17:42:00.638 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125663.log 2023-10-21T17:42:00.639 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137018.log.gz 2023-10-21T17:42:00.640 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107711.log 2023-10-21T17:42:00.641 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154133.log.gz 2023-10-21T17:42:00.641 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149791.log 2023-10-21T17:42:00.642 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107711.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.125663.log.gz 2023-10-21T17:42:00.643 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122122.log 2023-10-21T17:42:00.643 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107711.log.gz 2023-10-21T17:42:00.644 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104247.log 2023-10-21T17:42:00.645 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.149791.log.gz 2023-10-21T17:42:00.646 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153379.log 2023-10-21T17:42:00.646 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122122.log.gz 2023-10-21T17:42:00.647 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.104247.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104247.log.gz -5 2023-10-21T17:42:00.647 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.137568.log 2023-10-21T17:42:00.649 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153016.log 2023-10-21T17:42:00.649 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153379.log.gz 2023-10-21T17:42:00.650 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126933.log 2023-10-21T17:42:00.650 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137568.log.gz 2023-10-21T17:42:00.652 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120993.log 2023-10-21T17:42:00.652 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153016.log.gz 2023-10-21T17:42:00.653 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140365.log 2023-10-21T17:42:00.653 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126933.log.gz 2023-10-21T17:42:00.655 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141527.log 2023-10-21T17:42:00.655 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120993.log.gz 2023-10-21T17:42:00.656 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97632.log 2023-10-21T17:42:00.657 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140365.log.gz 2023-10-21T17:42:00.658 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141527.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.141527.log.gz -5 2023-10-21T17:42:00.658 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.150751.log 2023-10-21T17:42:00.659 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103411.log 2023-10-21T17:42:00.660 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97632.log.gz 2023-10-21T17:42:00.661 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122206.log 2023-10-21T17:42:00.661 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150751.log.gz 2023-10-21T17:42:00.662 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153818.log 2023-10-21T17:42:00.662 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103411.log.gz 2023-10-21T17:42:00.664 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141310.log 2023-10-21T17:42:00.664 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122206.log.gz 2023-10-21T17:42:00.665 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138845.log 2023-10-21T17:42:00.665 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153818.log.gz 2023-10-21T17:42:00.666 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115116.log 2023-10-21T17:42:00.667 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141310.log.gz 2023-10-21T17:42:00.668 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125621.log 2023-10-21T17:42:00.668 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138845.log.gz 2023-10-21T17:42:00.669 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123810.log 2023-10-21T17:42:00.670 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115116.log.gz 2023-10-21T17:42:00.671 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156321.log 2023-10-21T17:42:00.671 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125621.log.gz 2023-10-21T17:42:00.673 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94145.log 2023-10-21T17:42:00.673 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123810.log.gz 2023-10-21T17:42:00.674 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112193.log 2023-10-21T17:42:00.674 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156321.log.gz 2023-10-21T17:42:00.675 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127427.log 2023-10-21T17:42:00.675 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94145.log.gz 2023-10-21T17:42:00.677 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157505.log 2023-10-21T17:42:00.677 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112193.log.gz 2023-10-21T17:42:00.679 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142313.log 2023-10-21T17:42:00.679 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127427.log.gz 2023-10-21T17:42:00.680 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150211.log 2023-10-21T17:42:00.680 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157505.log.gz 2023-10-21T17:42:00.681 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99187.log 2023-10-21T17:42:00.681 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142313.log.gz 2023-10-21T17:42:00.683 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149434.log 2023-10-21T17:42:00.683 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150211.log.gz 2023-10-21T17:42:00.684 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157114.log 2023-10-21T17:42:00.684 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99187.log.gz 2023-10-21T17:42:00.685 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121382.log 2023-10-21T17:42:00.686 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149434.log.gz 2023-10-21T17:42:00.687 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128916.log 2023-10-21T17:42:00.687 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157114.log.gz 2023-10-21T17:42:00.688 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151047.log 2023-10-21T17:42:00.689 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121382.log.gz 2023-10-21T17:42:00.690 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121730.log 2023-10-21T17:42:00.690 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128916.log.gz 2023-10-21T17:42:00.692 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120258.log 2023-10-21T17:42:00.692 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151047.log.gz 2023-10-21T17:42:00.693 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98121.log 2023-10-21T17:42:00.693 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121730.log.gz 2023-10-21T17:42:00.694 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99039.log 2023-10-21T17:42:00.694 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120258.log.gz 2023-10-21T17:42:00.696 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150921.log 2023-10-21T17:42:00.696 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98121.log.gz 2023-10-21T17:42:00.697 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100113.log 2023-10-21T17:42:00.698 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99039.log.gz 2023-10-21T17:42:00.699 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148381.log 2023-10-21T17:42:00.699 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150921.log.gz 2023-10-21T17:42:00.700 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95477.log 2023-10-21T17:42:00.700 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100113.log.gz 2023-10-21T17:42:00.702 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133421.log 2023-10-21T17:42:00.702 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148381.log.gz 2023-10-21T17:42:00.703 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114971.log 2023-10-21T17:42:00.704 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95477.log.gz 2023-10-21T17:42:00.704 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159469.log 2023-10-21T17:42:00.705 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133421.log.gz 2023-10-21T17:42:00.706 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146968.log 2023-10-21T17:42:00.706 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114971.log.gz 2023-10-21T17:42:00.708 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115410.log 2023-10-21T17:42:00.708 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159469.log.gz 2023-10-21T17:42:00.709 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134509.log 2023-10-21T17:42:00.709 INFO:teuthology.orchestra.run.smithi053.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146968.log.gz 2023-10-21T17:42:00.710 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150397.log 2023-10-21T17:42:00.711 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115410.log.gz 2023-10-21T17:42:00.712 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153944.log 2023-10-21T17:42:00.712 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134509.log.gz 2023-10-21T17:42:00.713 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117233.log 2023-10-21T17:42:00.713 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150397.log.gz 2023-10-21T17:42:00.715 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145483.log 2023-10-21T17:42:00.715 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.153944.log.gz 2023-10-21T17:42:00.716 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158114.log 2023-10-21T17:42:00.717 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117233.log.gz 2023-10-21T17:42:00.718 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112888.log 2023-10-21T17:42:00.718 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145483.log.gz 2023-10-21T17:42:00.719 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158114.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158114.log.gz 2023-10-21T17:42:00.719 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138036.log 2023-10-21T17:42:00.721 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146311.log 2023-10-21T17:42:00.721 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112888.log.gz 2023-10-21T17:42:00.722 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134081.log 2023-10-21T17:42:00.722 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138036.log.gz 2023-10-21T17:42:00.724 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135033.log 2023-10-21T17:42:00.724 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146311.log.gz 2023-10-21T17:42:00.725 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127973.log 2023-10-21T17:42:00.725 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134081.log.gz 2023-10-21T17:42:00.727 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107585.log 2023-10-21T17:42:00.727 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135033.log.gz 2023-10-21T17:42:00.728 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107543.log 2023-10-21T17:42:00.728 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127973.log.gz 2023-10-21T17:42:00.730 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106830.log 2023-10-21T17:42:00.730 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107585.log.gz 2023-10-21T17:42:00.731 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102420.log 2023-10-21T17:42:00.731 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107543.log.gz 2023-10-21T17:42:00.732 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149004.log 2023-10-21T17:42:00.732 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106830.log.gz 2023-10-21T17:42:00.734 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113280.log 2023-10-21T17:42:00.734 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102420.log.gz 2023-10-21T17:42:00.736 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117275.log 2023-10-21T17:42:00.736 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149004.log.gz 2023-10-21T17:42:00.737 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113743.log 2023-10-21T17:42:00.737 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113280.log.gz 2023-10-21T17:42:00.738 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148195.log 2023-10-21T17:42:00.738 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117275.log.gz 2023-10-21T17:42:00.739 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.0.113743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113743.log.gz 2023-10-21T17:42:00.740 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122664.log 2023-10-21T17:42:00.740 INFO:teuthology.orchestra.run.smithi053.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-21T17:42:00.741 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94483.log 2023-10-21T17:42:00.741 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.148195.log.gz 2023-10-21T17:42:00.742 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122664.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122664.log.gz 2023-10-21T17:42:00.743 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133040.log 2023-10-21T17:42:00.744 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107921.log 2023-10-21T17:42:00.744 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94483.log.gz 2023-10-21T17:42:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133040.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133040.log.gz 2023-10-21T17:42:00.745 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115707.log 2023-10-21T17:42:00.746 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123284.log 2023-10-21T17:42:00.747 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107921.log.gz 2023-10-21T17:42:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115707.log.gz 2023-10-21T17:42:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135700.log 2023-10-21T17:42:00.748 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123284.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123284.log.gz 2023-10-21T17:42:00.749 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-21T17:42:00.750 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142990.log 2023-10-21T17:42:00.751 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135700.log.gz 2023-10-21T17:42:00.752 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95183.log 2023-10-21T17:42:00.753 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151431.log 2023-10-21T17:42:00.753 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142990.log.gz 2023-10-21T17:42:00.755 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119929.log 2023-10-21T17:42:00.755 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95183.log.gz 2023-10-21T17:42:00.756 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93235.log 2023-10-21T17:42:00.757 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151431.log.gz 2023-10-21T17:42:00.758 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145221.log 2023-10-21T17:42:00.758 INFO:teuthology.orchestra.run.smithi053.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119929.log.gz 2023-10-21T17:42:00.759 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136538.log 2023-10-21T17:42:00.760 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93235.log.gz 2023-10-21T17:42:00.760 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99531.log 2023-10-21T17:42:00.761 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145221.log.gz 2023-10-21T17:42:00.762 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113929.log 2023-10-21T17:42:00.763 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136538.log.gz 2023-10-21T17:42:00.764 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151599.log 2023-10-21T17:42:00.764 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99531.log.gz 2023-10-21T17:42:00.765 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134777.log 2023-10-21T17:42:00.765 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113929.log.gz 2023-10-21T17:42:00.766 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118197.log 2023-10-21T17:42:00.767 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151599.log.gz 2023-10-21T17:42:00.768 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126509.log 2023-10-21T17:42:00.768 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134777.log.gz 2023-10-21T17:42:00.770 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98955.log 2023-10-21T17:42:00.770 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118197.log.gz 2023-10-21T17:42:00.771 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133684.log 2023-10-21T17:42:00.771 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126509.log.gz 2023-10-21T17:42:00.772 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125369.log 2023-10-21T17:42:00.772 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98955.log.gz 2023-10-21T17:42:00.774 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133684.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.95729.log 2023-10-21T17:42:00.775 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.133684.log.gz 2023-10-21T17:42:00.775 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113863.log 2023-10-21T17:42:00.776 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125369.log.gz 2023-10-21T17:42:00.777 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100155.log 2023-10-21T17:42:00.777 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.95729.log.gz 2023-10-21T17:42:00.778 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152666.log 2023-10-21T17:42:00.779 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113863.log.gz 2023-10-21T17:42:00.780 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124502.log 2023-10-21T17:42:00.780 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100155.log.gz 2023-10-21T17:42:00.781 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115368.log 2023-10-21T17:42:00.781 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152666.log.gz 2023-10-21T17:42:00.783 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156489.log 2023-10-21T17:42:00.783 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124502.log.gz 2023-10-21T17:42:00.785 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137951.log 2023-10-21T17:42:00.785 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115368.log.gz 2023-10-21T17:42:00.786 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93177.log 2023-10-21T17:42:00.786 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156489.log.gz 2023-10-21T17:42:00.787 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149350.log 2023-10-21T17:42:00.787 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137951.log.gz 2023-10-21T17:42:00.789 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93135.log 2023-10-21T17:42:00.789 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93177.log.gz 2023-10-21T17:42:00.790 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147719.log 2023-10-21T17:42:00.790 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149350.log.gz 2023-10-21T17:42:00.792 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93135.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.93135.log.gz 2023-10-21T17:42:00.792 INFO:teuthology.orchestra.run.smithi053.stderr: /var/log/ceph/ceph-client.admin.112802.log 2023-10-21T17:42:00.793 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91959.log 2023-10-21T17:42:00.794 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147719.log.gz 2023-10-21T17:42:00.795 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98247.log 2023-10-21T17:42:00.795 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112802.log.gz 2023-10-21T17:42:00.796 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.91959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135201.log 2023-10-21T17:42:00.796 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91959.log.gz 2023-10-21T17:42:00.798 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154511.log 2023-10-21T17:42:00.798 INFO:teuthology.orchestra.run.smithi053.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98247.log.gz 2023-10-21T17:42:00.799 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95225.log 2023-10-21T17:42:00.800 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135201.log.gz 2023-10-21T17:42:00.801 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96876.log 2023-10-21T17:42:00.801 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154511.log.gz 2023-10-21T17:42:00.802 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93797.log 2023-10-21T17:42:00.802 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95225.log.gz 2023-10-21T17:42:00.803 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96876.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96876.log.gz 2023-10-21T17:42:00.803 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156109.log 2023-10-21T17:42:00.805 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129864.log 2023-10-21T17:42:00.805 INFO:teuthology.orchestra.run.smithi053.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93797.log.gz 2023-10-21T17:42:00.806 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94811.log 2023-10-21T17:42:00.807 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156109.log.gz 2023-10-21T17:42:00.808 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158767.log 2023-10-21T17:42:00.808 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129864.log.gz 2023-10-21T17:42:00.809 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94819.log 2023-10-21T17:42:00.810 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94811.log.gz 2023-10-21T17:42:00.811 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114344.log 2023-10-21T17:42:00.811 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158767.log.gz 2023-10-21T17:42:00.812 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145662.log 2023-10-21T17:42:00.812 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94819.log.gz 2023-10-21T17:42:00.813 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111481.log 2023-10-21T17:42:00.814 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114344.log.gz 2023-10-21T17:42:00.815 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146017.log 2023-10-21T17:42:00.815 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145662.log.gz 2023-10-21T17:42:00.816 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96772.log 2023-10-21T17:42:00.817 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111481.log.gz 2023-10-21T17:42:00.818 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136622.log 2023-10-21T17:42:00.818 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146017.log.gz 2023-10-21T17:42:00.820 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.96772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142271.log 2023-10-21T17:42:00.820 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96772.log.gz 2023-10-21T17:42:00.821 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97254.log 2023-10-21T17:42:00.821 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136622.log.gz 2023-10-21T17:42:00.822 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.142271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121245.log 2023-10-21T17:42:00.822 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142271.log.gz 2023-10-21T17:42:00.823 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158240.log 2023-10-21T17:42:00.824 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97254.log.gz 2023-10-21T17:42:00.825 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121245.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-21T17:42:00.826 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121245.log.gz 2023-10-21T17:42:00.827 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125453.log 2023-10-21T17:42:00.827 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158240.log.gz 2023-10-21T17:42:00.828 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126332.log 2023-10-21T17:42:00.830 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138619.log 2023-10-21T17:42:00.830 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125453.log.gz 2023-10-21T17:42:00.831 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107795.log 2023-10-21T17:42:00.831 INFO:teuthology.orchestra.run.smithi053.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126332.log.gz 2023-10-21T17:42:00.832 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138619.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.138619.log.gz 2023-10-21T17:42:00.833 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154385.log 2023-10-21T17:42:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107795.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107795.log.gz 2023-10-21T17:42:00.834 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127196.log 2023-10-21T17:42:00.836 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134291.log 2023-10-21T17:42:00.836 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154385.log.gz 2023-10-21T17:42:00.837 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116143.log 2023-10-21T17:42:00.837 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127196.log.gz 2023-10-21T17:42:00.839 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116768.log 2023-10-21T17:42:00.839 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134291.log.gz 2023-10-21T17:42:00.839 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-21T17:42:00.840 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98493.log 2023-10-21T17:42:00.840 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116143.log.gz 2023-10-21T17:42:00.841 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92003.log 2023-10-21T17:42:00.841 INFO:teuthology.orchestra.run.smithi053.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116768.log.gz 2023-10-21T17:42:00.843 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121035.log 2023-10-21T17:42:00.843 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98493.log.gz 2023-10-21T17:42:00.844 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128694.log 2023-10-21T17:42:00.845 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92003.log.gz 2023-10-21T17:42:00.846 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145914.log 2023-10-21T17:42:00.846 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121035.log.gz 2023-10-21T17:42:00.847 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126391.log 2023-10-21T17:42:00.847 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128694.log.gz 2023-10-21T17:42:00.849 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145914.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-21T17:42:00.849 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145914.log.gz 2023-10-21T17:42:00.850 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.126391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116015.log 2023-10-21T17:42:00.850 INFO:teuthology.orchestra.run.smithi053.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126391.log.gz 2023-10-21T17:42:00.851 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158072.log 2023-10-21T17:42:00.853 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.116015.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116015.log.gz -5 2023-10-21T17:42:00.853 INFO:teuthology.orchestra.run.smithi053.stderr: --verbose -- /var/log/ceph/ceph-client.admin.120951.log 2023-10-21T17:42:00.854 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158072.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158072.log.gz 2023-10-21T17:42:00.855 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147454.log 2023-10-21T17:42:00.856 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101890.log 2023-10-21T17:42:00.857 INFO:teuthology.orchestra.run.smithi053.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120951.log.gz 2023-10-21T17:42:00.858 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107627.log 2023-10-21T17:42:00.858 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147454.log.gz 2023-10-21T17:42:00.859 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.101890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137407.log 2023-10-21T17:42:00.859 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101890.log.gz 2023-10-21T17:42:00.861 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138661.log 2023-10-21T17:42:00.861 INFO:teuthology.orchestra.run.smithi053.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107627.log.gz 2023-10-21T17:42:00.862 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157310.log 2023-10-21T17:42:00.863 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137407.log.gz 2023-10-21T17:42:00.864 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121670.log 2023-10-21T17:42:00.864 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138661.log.gz 2023-10-21T17:42:00.865 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157310.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-mds.c.log 2023-10-21T17:42:00.866 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.157310.log.gz 2023-10-21T17:42:00.867 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141058.log 2023-10-21T17:42:00.867 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121670.log.gz 2023-10-21T17:42:00.868 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112066.log 2023-10-21T17:42:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141058.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141058.log.gz 2023-10-21T17:42:00.870 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95603.log 2023-10-21T17:42:00.871 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112066.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112066.log.gz 2023-10-21T17:42:00.872 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134333.log 2023-10-21T17:42:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95603.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95603.log.gz 2023-10-21T17:42:00.874 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127511.log 2023-10-21T17:42:00.875 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134333.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134333.log.gz 2023-10-21T17:42:00.876 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94458.log 2023-10-21T17:42:00.877 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127511.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127511.log.gz 2023-10-21T17:42:00.878 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141777.log 2023-10-21T17:42:00.879 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94458.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94458.log.gz 2023-10-21T17:42:00.880 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108219.log 2023-10-21T17:42:00.881 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141777.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141777.log.gz 2023-10-21T17:42:00.882 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141915.log 2023-10-21T17:42:00.883 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108219.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108219.log.gz 2023-10-21T17:42:00.884 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146796.log 2023-10-21T17:42:00.886 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141915.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141915.log.gz 2023-10-21T17:42:00.887 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97296.log 2023-10-21T17:42:00.887 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146796.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146796.log.gz 2023-10-21T17:42:00.888 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118794.log 2023-10-21T17:42:00.891 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97296.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97296.log.gz 2023-10-21T17:42:00.891 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111754.log 2023-10-21T17:42:00.891 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118794.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118794.log.gz 2023-10-21T17:42:00.892 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156531.log 2023-10-21T17:42:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111754.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111754.log.gz 2023-10-21T17:42:00.895 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129822.log 2023-10-21T17:42:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156531.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156531.log.gz 2023-10-21T17:42:00.896 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147094.log 2023-10-21T17:42:00.898 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129822.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129822.log.gz 2023-10-21T17:42:00.898 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148151.log 2023-10-21T17:42:00.900 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147094.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147094.log.gz 2023-10-21T17:42:00.901 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147887.log 2023-10-21T17:42:00.902 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148151.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148151.log.gz 2023-10-21T17:42:00.903 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109255.log 2023-10-21T17:42:00.904 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147887.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147887.log.gz 2023-10-21T17:42:00.905 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157810.log 2023-10-21T17:42:00.906 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109255.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109255.log.gz 2023-10-21T17:42:00.907 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124987.log 2023-10-21T17:42:00.907 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157810.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157810.log.gz 2023-10-21T17:42:00.908 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99947.log 2023-10-21T17:42:00.910 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124987.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124987.log.gz 2023-10-21T17:42:00.911 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122622.log 2023-10-21T17:42:00.912 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99947.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99947.log.gz 2023-10-21T17:42:00.913 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140596.log 2023-10-21T17:42:00.914 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122622.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122622.log.gz 2023-10-21T17:42:00.915 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129769.log 2023-10-21T17:42:00.916 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140596.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140596.log.gz 2023-10-21T17:42:00.917 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134633.log 2023-10-21T17:42:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.129769.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129769.log.gz 2023-10-21T17:42:00.919 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92255.log 2023-10-21T17:42:00.920 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134633.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134633.log.gz 2023-10-21T17:42:00.921 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153058.log 2023-10-21T17:42:00.922 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.92255.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92255.log.gz 2023-10-21T17:42:00.923 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120446.log 2023-10-21T17:42:00.924 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153058.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153058.log.gz 2023-10-21T17:42:00.925 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149875.log 2023-10-21T17:42:00.926 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120446.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120446.log.gz 2023-10-21T17:42:00.927 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106788.log 2023-10-21T17:42:00.928 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149875.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.149875.log.gz 2023-10-21T17:42:00.929 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112109.log 2023-10-21T17:42:00.930 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.106788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106788.log.gz 2023-10-21T17:42:00.931 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119605.log/var/log/ceph/ceph-client.admin.112109.log: 2023-10-21T17:42:00.932 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112109.log.gz 2023-10-21T17:42:00.933 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145746.log 2023-10-21T17:42:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.119605.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119605.log.gz 2023-10-21T17:42:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118155.log 2023-10-21T17:42:00.936 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.145746.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145746.log.gz 2023-10-21T17:42:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103470.log 2023-10-21T17:42:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118155.log.gz 2023-10-21T17:42:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97884.log 2023-10-21T17:42:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103470.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103470.log.gz 2023-10-21T17:42:00.941 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124586.log 2023-10-21T17:42:00.943 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97884.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97884.log.gz 2023-10-21T17:42:00.944 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147178.log 2023-10-21T17:42:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124586.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124586.log.gz 2023-10-21T17:42:00.945 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94137.log 2023-10-21T17:42:00.947 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147178.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147178.log.gz 2023-10-21T17:42:00.948 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121910.log 2023-10-21T17:42:00.948 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94137.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94137.log.gz 2023-10-21T17:42:00.949 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95351.log 2023-10-21T17:42:00.951 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121910.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121910.log.gz 2023-10-21T17:42:00.952 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159385.log 2023-10-21T17:42:00.952 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95351.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95351.log.gz 2023-10-21T17:42:00.953 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148616.log 2023-10-21T17:42:00.956 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159385.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159385.log.gz 2023-10-21T17:42:00.956 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155430.log 2023-10-21T17:42:00.957 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148616.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148616.log.gz 2023-10-21T17:42:00.957 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154907.log 2023-10-21T17:42:00.959 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155430.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155430.log.gz 2023-10-21T17:42:00.960 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111313.log 2023-10-21T17:42:00.961 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.154907.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154907.log.gz 2023-10-21T17:42:00.962 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108304.log 2023-10-21T17:42:00.962 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111313.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111313.log.gz 2023-10-21T17:42:00.963 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100883.log 2023-10-21T17:42:00.965 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108304.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108304.log.gz 2023-10-21T17:42:00.965 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150001.log 2023-10-21T17:42:00.967 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100883.log.gz 2023-10-21T17:42:00.968 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155640.log 2023-10-21T17:42:00.969 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150001.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150001.log.gz 2023-10-21T17:42:00.970 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125579.log 2023-10-21T17:42:00.971 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.155640.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155640.log.gz 2023-10-21T17:42:00.971 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147845.log 2023-10-21T17:42:00.973 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125579.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125579.log.gz 2023-10-21T17:42:00.974 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98667.log 2023-10-21T17:42:00.975 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147845.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147845.log.gz 2023-10-21T17:42:00.976 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114911.log 2023-10-21T17:42:00.976 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.98667.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98667.log.gz 2023-10-21T17:42:00.977 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157904.log 2023-10-21T17:42:00.979 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114911.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114911.log.gz 2023-10-21T17:42:00.980 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118281.log 2023-10-21T17:42:00.981 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.157904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.157904.log.gz 2023-10-21T17:42:00.982 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133285.log 2023-10-21T17:42:00.983 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.118281.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118281.log.gz 2023-10-21T17:42:00.984 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120013.log 2023-10-21T17:42:00.984 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.133285.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.133285.log.gz 2023-10-21T17:42:00.985 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101009.log 2023-10-21T17:42:00.987 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120013.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120013.log.gz 2023-10-21T17:42:00.988 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108882.log/var/log/ceph/ceph-client.admin.101009.log: 2023-10-21T17:42:00.989 INFO:teuthology.orchestra.run.smithi053.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101009.log.gz 2023-10-21T17:42:00.990 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103016.log 2023-10-21T17:42:00.991 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108882.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108882.log.gz 2023-10-21T17:42:00.992 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120867.log 2023-10-21T17:42:00.993 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103016.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103016.log.gz 2023-10-21T17:42:00.994 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150439.log 2023-10-21T17:42:00.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.120867.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120867.log.gz 2023-10-21T17:42:00.996 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103883.log 2023-10-21T17:42:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.150439.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150439.log.gz 2023-10-21T17:42:00.997 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151005.log 2023-10-21T17:42:00.999 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.103883.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103883.log.gz 2023-10-21T17:42:01.000 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127763.log 2023-10-21T17:42:01.001 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151005.log.gz 2023-10-21T17:42:01.002 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137270.log 2023-10-21T17:42:01.003 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127763.log.gz 2023-10-21T17:42:01.004 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93923.log 2023-10-21T17:42:01.005 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.137270.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137270.log.gz 2023-10-21T17:42:01.006 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146754.log 2023-10-21T17:42:01.007 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93923.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93923.log.gz 2023-10-21T17:42:01.008 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125537.log 2023-10-21T17:42:01.009 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146754.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146754.log.gz 2023-10-21T17:42:01.010 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95015.log 2023-10-21T17:42:01.011 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125537.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125537.log.gz 2023-10-21T17:42:01.012 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146185.log 2023-10-21T17:42:01.014 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95015.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95015.log.gz 2023-10-21T17:42:01.014 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122884.log 2023-10-21T17:42:01.015 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.146185.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146185.log.gz 2023-10-21T17:42:01.016 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100323.log 2023-10-21T17:42:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122884.log.gz 2023-10-21T17:42:01.018 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128522.log 2023-10-21T17:42:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100323.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100323.log.gz 2023-10-21T17:42:01.020 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151813.log 2023-10-21T17:42:01.021 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128522.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128522.log.gz 2023-10-21T17:42:01.022 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148297.log 2023-10-21T17:42:01.024 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.151813.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151813.log.gz 2023-10-21T17:42:01.024 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102252.log 2023-10-21T17:42:01.025 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148297.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148297.log.gz 2023-10-21T17:42:01.026 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124460.log 2023-10-21T17:42:01.027 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.102252.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102252.log.gz 2023-10-21T17:42:01.028 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107313.log 2023-10-21T17:42:01.029 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.124460.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124460.log.gz 2023-10-21T17:42:01.030 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113971.log 2023-10-21T17:42:01.031 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107313.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107313.log.gz 2023-10-21T17:42:01.032 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112235.log 2023-10-21T17:42:01.033 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113971.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113971.log.gz 2023-10-21T17:42:01.034 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146712.log 2023-10-21T17:42:01.035 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112235.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112235.log.gz 2023-10-21T17:42:01.037 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.146712.log: /var/log/ceph/ceph-client.admin.152215.log 2023-10-21T17:42:01.037 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146712.log.gz 2023-10-21T17:42:01.038 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122248.log 2023-10-21T17:42:01.039 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.152215.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152215.log.gz 2023-10-21T17:42:01.040 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148574.log 2023-10-21T17:42:01.041 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122248.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122248.log.gz 2023-10-21T17:42:01.042 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141447.log 2023-10-21T17:42:01.043 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148574.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148574.log.gz 2023-10-21T17:42:01.044 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140323.log 2023-10-21T17:42:01.045 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141447.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141447.log.gz 2023-10-21T17:42:01.046 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151687.log 2023-10-21T17:42:01.048 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140323.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140323.log.gz 2023-10-21T17:42:01.049 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109405.log/var/log/ceph/ceph-client.admin.151687.log: 2023-10-21T17:42:01.049 INFO:teuthology.orchestra.run.smithi053.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.151687.log.gz 2023-10-21T17:42:01.050 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99321.log 2023-10-21T17:42:01.052 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.109405.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109405.log.gz 2023-10-21T17:42:01.053 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100715.log 2023-10-21T17:42:01.054 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99321.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99321.log.gz 2023-10-21T17:42:01.055 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135117.log 2023-10-21T17:42:01.056 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100715.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100715.log.gz 2023-10-21T17:42:01.057 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93445.log 2023-10-21T17:42:01.058 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135117.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135117.log.gz 2023-10-21T17:42:01.059 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97086.log 2023-10-21T17:42:01.060 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93445.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93445.log.gz 2023-10-21T17:42:01.061 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136401.log 2023-10-21T17:42:01.062 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.97086.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97086.log.gz 2023-10-21T17:42:01.063 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148059.log 2023-10-21T17:42:01.064 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136401.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136401.log.gz 2023-10-21T17:42:01.065 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122730.log 2023-10-21T17:42:01.066 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148059.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148059.log.gz 2023-10-21T17:42:01.067 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127052.log 2023-10-21T17:42:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.122730.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122730.log.gz 2023-10-21T17:42:01.069 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93965.log 2023-10-21T17:42:01.070 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127052.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127052.log.gz 2023-10-21T17:42:01.071 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141865.log 2023-10-21T17:42:01.072 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93965.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93965.log.gz 2023-10-21T17:42:01.073 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138727.log 2023-10-21T17:42:01.075 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141865.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141865.log.gz 2023-10-21T17:42:01.075 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95267.log 2023-10-21T17:42:01.076 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138727.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138727.log.gz 2023-10-21T17:42:01.077 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99615.log 2023-10-21T17:42:01.078 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.95267.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95267.log.gz 2023-10-21T17:42:01.079 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115200.log 2023-10-21T17:42:01.081 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.99615.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99615.log.gz 2023-10-21T17:42:01.081 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107145.log 2023-10-21T17:42:01.082 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.115200.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115200.log.gz 2023-10-21T17:42:01.083 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148724.log 2023-10-21T17:42:01.084 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107145.log.gz 2023-10-21T17:42:01.085 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94134.log 2023-10-21T17:42:01.086 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.148724.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148724.log.gz 2023-10-21T17:42:01.087 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141016.log 2023-10-21T17:42:01.088 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.94134.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94134.log.gz 2023-10-21T17:42:01.089 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140239.log 2023-10-21T17:42:01.090 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141016.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141016.log.gz 2023-10-21T17:42:01.091 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140806.log 2023-10-21T17:42:01.092 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140239.log.gz 2023-10-21T17:42:01.093 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138204.log 2023-10-21T17:42:01.094 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.140806.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140806.log.gz 2023-10-21T17:42:01.095 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112662.log 2023-10-21T17:42:01.096 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.138204.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138204.log.gz 2023-10-21T17:42:01.097 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141651.log 2023-10-21T17:42:01.098 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.112662.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112662.log.gz 2023-10-21T17:42:01.100 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.141651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134249.log 2023-10-21T17:42:01.100 INFO:teuthology.orchestra.run.smithi053.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141651.log.gz 2023-10-21T17:42:01.102 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135658.log 2023-10-21T17:42:01.103 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.134249.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134249.log.gz 2023-10-21T17:42:01.104 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121996.log 2023-10-21T17:42:01.105 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.135658.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135658.log.gz 2023-10-21T17:42:01.106 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149147.log 2023-10-21T17:42:01.108 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.121996.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121996.log.gz 2023-10-21T17:42:01.108 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159596.log 2023-10-21T17:42:01.109 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.149147.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149147.log.gz 2023-10-21T17:42:01.110 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153546.log 2023-10-21T17:42:01.111 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159596.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159596.log.gz 2023-10-21T17:42:01.112 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114659.log 2023-10-21T17:42:01.113 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153546.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153546.log.gz 2023-10-21T17:42:01.114 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-21T17:42:01.115 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114659.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114659.log.gz 2023-10-21T17:42:01.116 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103824.log 2023-10-21T17:42:01.119 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.103824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103824.log.gz 2023-10-21T17:42:01.120 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108612.log 2023-10-21T17:42:01.122 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.108612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100239.log 2023-10-21T17:42:01.124 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108612.log.gz 2023-10-21T17:42:01.125 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147388.log 2023-10-21T17:42:01.128 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.100239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100239.log.gz 2023-10-21T17:42:01.131 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.147388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117149.log 2023-10-21T17:42:01.131 INFO:teuthology.orchestra.run.smithi053.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.147388.log.gz 2023-10-21T17:42:01.133 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153734.log 2023-10-21T17:42:01.135 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.117149.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117149.log.gz 2023-10-21T17:42:01.137 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111964.log 2023-10-21T17:42:01.139 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.153734.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153734.log.gz 2023-10-21T17:42:01.140 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127931.log 2023-10-21T17:42:01.141 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.111964.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111964.log.gz 2023-10-21T17:42:01.143 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.127931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136832.log 2023-10-21T17:42:01.145 INFO:teuthology.orchestra.run.smithi053.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127931.log.gz 2023-10-21T17:42:01.145 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125729.log 2023-10-21T17:42:01.146 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.136832.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136832.log.gz 2023-10-21T17:42:01.149 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.125729.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125729.log.gz 2023-10-21T17:42:01.150 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159678.log 2023-10-21T17:42:01.151 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156363.log 2023-10-21T17:42:01.153 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.159678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125029.log 2023-10-21T17:42:01.154 INFO:teuthology.orchestra.run.smithi053.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93881.log 2023-10-21T17:42:01.155 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.159678.log.gz 2023-10-21T17:42:01.156 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.156363.log: /var/log/ceph/ceph-client.admin.125029.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.156363.log.gz 2023-10-21T17:42:01.158 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.93881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123895.log 2023-10-21T17:42:01.160 INFO:teuthology.orchestra.run.smithi053.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158683.log 2023-10-21T17:42:01.160 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125029.log.gz 2023-10-21T17:42:01.161 INFO:teuthology.orchestra.run.smithi053.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93881.log.gz 2023-10-21T17:42:01.162 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.123895.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123895.log.gz 2023-10-21T17:42:01.165 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.158683.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158683.log.gz 2023-10-21T17:42:01.166 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139712.log 2023-10-21T17:42:01.169 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.139712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128320.log 2023-10-21T17:42:01.170 INFO:teuthology.orchestra.run.smithi053.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139712.log.gz 2023-10-21T17:42:01.172 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94146.log 2023-10-21T17:42:01.175 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.128320.log: /var/log/ceph/ceph-client.admin.94146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107499.log 2023-10-21T17:42:01.176 INFO:teuthology.orchestra.run.smithi053.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128320.log.gz 2023-10-21T17:42:01.177 INFO:teuthology.orchestra.run.smithi053.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94146.log.gz 2023-10-21T17:42:01.178 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.107499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107499.log.gz 2023-10-21T17:42:01.179 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113152.log 2023-10-21T17:42:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.113152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114480.log 2023-10-21T17:42:01.183 INFO:teuthology.orchestra.run.smithi053.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113152.log.gz 2023-10-21T17:42:01.185 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.114480.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114480.log.gz 2023-10-21T17:42:01.331 INFO:teuthology.orchestra.run.smithi124.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-21T17:42:01.603 INFO:teuthology.orchestra.run.smithi124.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-21T17:42:01.635 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-21T17:42:01.772 INFO:teuthology.orchestra.run.smithi124.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-21T17:42:02.152 INFO:teuthology.orchestra.run.smithi053.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-21T17:42:03.441 INFO:teuthology.orchestra.run.smithi053.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-21T17:42:04.540 INFO:teuthology.orchestra.run.smithi124.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-21T17:42:04.780 INFO:teuthology.orchestra.run.smithi124.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-21T17:42:04.958 INFO:teuthology.orchestra.run.smithi124.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-21T17:42:05.878 INFO:teuthology.orchestra.run.smithi053.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-21T17:42:06.121 INFO:teuthology.orchestra.run.smithi053.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-21T17:42:06.405 INFO:teuthology.orchestra.run.smithi053.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-21T17:42:06.611 INFO:teuthology.orchestra.run.smithi124.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-21T17:42:06.614 INFO:teuthology.orchestra.run.smithi124.stderr: 2023-10-21T17:42:06.615 INFO:teuthology.orchestra.run.smithi124.stderr:real 0m7.111s 2023-10-21T17:42:06.615 INFO:teuthology.orchestra.run.smithi124.stderr:user 0m27.095s 2023-10-21T17:42:06.615 INFO:teuthology.orchestra.run.smithi124.stderr:sys 0m2.224s 2023-10-21T17:42:07.477 INFO:teuthology.orchestra.run.smithi053.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-21T17:42:07.482 INFO:teuthology.orchestra.run.smithi053.stderr: 2023-10-21T17:42:07.482 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m7.980s 2023-10-21T17:42:07.483 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m27.824s 2023-10-21T17:42:07.483 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m5.196s 2023-10-21T17:42:07.484 INFO:tasks.ceph:Archiving logs... 2023-10-21T17:42:07.484 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi053/log 2023-10-21T17:42:07.485 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-21T17:42:12.105 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi124/log 2023-10-21T17:42:12.106 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-21T17:42:15.726 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-21T17:42:15.736 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-21T17:42:15.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-21T17:42:15.746 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-21T17:42:15.907 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-21T17:42:15.920 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-21T17:42:15.921 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-21T17:42:15.931 DEBUG:teuthology.orchestra.run.smithi124:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-21T17:42:15.984 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2023-10-21T17:42:15.990 INFO:teuthology.orchestra.run.smithi124.stderr:bash: line 1: ntpq: command not found 2023-10-21T17:42:15.995 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-21T17:42:15.995 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-10-21T17:42:15.995 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 0 +1637us[+1637us] +/- 73ms 2023-10-21T17:42:15.996 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 8 377 191 -1159us[-1191us] +/- 61ms 2023-10-21T17:42:15.996 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +172us[ +172us] +/- 67ms 2023-10-21T17:42:15.996 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T17:42:15.999 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-21T17:42:16.000 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2023-10-21T17:42:16.000 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 65 +1633us[+1633us] +/- 72ms 2023-10-21T17:42:16.000 INFO:teuthology.orchestra.run.smithi124.stdout:^* hv02.front.sepia.ceph.com 3 8 377 257 -1208us[-1229us] +/- 60ms 2023-10-21T17:42:16.000 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 253 +229us[ +229us] +/- 63ms 2023-10-21T17:42:16.001 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-21T17:42:16.001 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-21T17:42:16.011 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-21T17:42:16.011 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-21T17:42:16.037 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-21T17:42:16.248 DEBUG:teuthology.orchestra.run.smithi124:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-21T17:42:16.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-21T17:42:16.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:42:16.626 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-21T17:42:16.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:42:16.739 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-21T17:42:16.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-21T17:42:16.784 INFO:teuthology.task.internal:Duration was 6831.080997 seconds 2023-10-21T17:42:16.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-21T17:42:16.828 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-21T17:42:16.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-21T17:42:16.837 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-21T17:42:17.008 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-21T17:42:17.019 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-21T17:42:17.404 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-21T17:42:17.405 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2023-10-21T17:42:17.405 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-21T17:42:17.487 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi124.front.sepia.ceph.com 2023-10-21T17:42:17.487 DEBUG:teuthology.orchestra.run.smithi124:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-21T17:42:17.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-21T17:42:17.567 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-21T17:42:17.576 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-21T17:42:17.677 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-21T17:42:17.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-21T17:42:17.684 DEBUG:teuthology.orchestra.run.smithi124:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-21T17:42:17.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-21T17:42:17.984 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-21T17:42:17.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-21T17:42:18.093 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-21T17:42:18.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-21T17:42:18.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-21T17:42:18.223 DEBUG:teuthology.orchestra.run.smithi124:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-21T17:42:18.336 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2023-10-21T17:42:18.338 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = core 2023-10-21T17:42:18.394 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-21T17:42:18.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:42:18.449 DEBUG:teuthology.orchestra.run.smithi124:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-21T17:42:18.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-21T17:42:18.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-21T17:42:18.518 INFO:teuthology.task.internal:Transferring archived files... 2023-10-21T17:42:18.518 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi053 2023-10-21T17:42:18.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-21T17:42:18.636 DEBUG:teuthology.misc:Transferring archived files from smithi124:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368/remote/smithi124 2023-10-21T17:42:18.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-21T17:42:18.745 INFO:teuthology.task.internal:Removing archive directory... 2023-10-21T17:42:18.745 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-21T17:42:18.755 DEBUG:teuthology.orchestra.run.smithi124:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-21T17:42:18.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-21T17:42:18.828 INFO:teuthology.task.internal:Not uploading archives. 2023-10-21T17:42:18.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-21T17:42:18.887 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-21T17:42:18.888 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-21T17:42:18.897 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-21T17:42:18.950 INFO:teuthology.orchestra.run.smithi053.stdout: 393622 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 21 17:42 /home/ubuntu/cephtest 2023-10-21T17:42:18.959 INFO:teuthology.orchestra.run.smithi124.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 21 17:42 /home/ubuntu/cephtest 2023-10-21T17:42:18.962 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-21T17:42:18.973 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-21T17:42:19.008 INFO:teuthology.nuke:Checking targets against current locks 2023-10-21T17:42:19.034 DEBUG:teuthology.nuke:shortname: smithi053 2023-10-21T17:42:19.035 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T17:42:19.056 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 15:39:31.284207', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6fbGUyaNZpvY8oPhmMhokG1MwWoELqZEkefGzsAeko4jFlkENtZa/IW7SEG+imR/eOC9to3WOcrryOomavqFs='} 2023-10-21T17:42:19.060 DEBUG:teuthology.nuke:shortname: smithi124 2023-10-21T17:42:19.061 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T17:42:19.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:46:15-fs-main-distro-default-smithi/7434368', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 15:39:31.282771', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuo3XS6+QNnM2uGCiSRrpQv3oO0av8aS5sVXQ0s6Hpb+uYzKM7UdSZpafuhOAmzXO0hreAgrGR6pCLCsexlrdo='} 2023-10-21T17:42:19.111 INFO:teuthology.orchestra.console.smithi053:Power off 2023-10-21T17:42:19.112 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T17:42:19.133 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-10-21T17:42:19.134 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:19.149 INFO:teuthology.orchestra.console.smithi124:Power off 2023-10-21T17:42:19.149 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T17:42:19.159 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-10-21T17:42:19.172 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-10-21T17:42:19.172 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:19.195 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-10-21T17:42:23.161 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:23.196 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:23.271 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-10-21T17:42:23.305 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-10-21T17:42:27.273 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:27.306 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T17:42:27.383 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-10-21T17:42:27.415 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-10-21T17:42:27.484 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-10-21T17:42:27.516 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-10-21T17:42:27.621 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 client.1-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6831.080996751785 failure_reason: '"2023-10-21T16:16:32.316263+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-21T17:42:27.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T17:42:27.703 INFO:teuthology.run:FAIL